2024-04-07T15:49:23.933 INFO:root:teuthology version: 0.0.1.dev267+g6c63784 2024-04-07T15:49:23.933 DEBUG:teuthology.run:Teuthology command: teuthology --name yuriw-2024-04-07_14:37:12-rados-wip-yuri3-testing-2024-04-05-0825-distro-default-smithi --description rados/upgrade/parallel/{0-random-distro$/{ubuntu_22.04} 0-start 1-tasks mon_election/classic upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api test_rbd_python}} --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2024-04-07_14:37:12-rados-wip-yuri3-testing-2024-04-05-0825-distro-default-smithi/7643840 --verbose -- /home/teuthworker/archive/yuriw-2024-04-07_14:37:12-rados-wip-yuri3-testing-2024-04-05-0825-distro-default-smithi/7643840/orig.config.yaml 2024-04-07T15:49:23.977 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-07T15:49:24.061 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/yuriw-2024-04-07_14:37:12-rados-wip-yuri3-testing-2024-04-05-0825-distro-default-smithi/7643840 branch: wip-yuri3-testing-2024-04-05-0825 description: rados/upgrade/parallel/{0-random-distro$/{ubuntu_22.04} 0-start 1-tasks mon_election/classic upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api test_rbd_python}} email: yweinste@redhat.com first_in_suite: false job_id: '7643840' kernel: kdb: 1 sha1: distro ktype: distro last_in_suite: false machine_type: smithi meta: - desc: 'run run randomized correctness test for rados operations on an erasure-coded pool ' - desc: 'object class functional tests ' - desc: 'generate read/write load with rados objects ranging from 1MB to 25MB ' - desc: 'run basic import/export cli tests for rbd ' - desc: 'librbd C and C++ api tests ' - desc: 'librbd python api tests ' name: yuriw-2024-04-07_14:37:12-rados-wip-yuri3-testing-2024-04-05-0825-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 10 os_type: ubuntu os_version: '22.04' overrides: admin_socket: branch: wip-yuri3-testing-2024-04-05-0825 ceph: conf: global: mon election default strategy: 1 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: debug ms: 1 debug osd: 20 osd shutdown pgref assert: true create_rbd_pool: true flavor: default log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) sha1: adce92d056533f22dc1ee37e3141d098b046ec1b ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: extra_system_packages: - python3-pytest flavor: default sha1: adce92d056533f22dc1ee37e3141d098b046ec1b workunit: branch: wip-yuri3-testing-2024-04-05-0825 sha1: adce92d056533f22dc1ee37e3141d098b046ec1b owner: scheduled_yuriw@teuthology priority: 99 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mon.c - mgr.y - mds.a - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - node-exporter.a - alertmanager.a - - mon.b - mds.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - prometheus.a - grafana.a - node-exporter.b seed: 5988 sha1: adce92d056533f22dc1ee37e3141d098b046ec1b sleep_before_teardown: 0 subset: 111/120000 suite: rados suite_branch: wip-yuri3-testing-2024-04-05-0825 suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_adce92d056533f22dc1ee37e3141d098b046ec1b/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: adce92d056533f22dc1ee37e3141d098b046ec1b targets: smithi100.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMvCZrXBXfjZD3eHBRmkPE6vfHXACCa9JHtdsxjkOCe1gSulfAtRRHBJXmkZyuhdWpF+PnJbuepLZr/Ept4h9yA= smithi144.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMeJBXxAI0FxiausJKk6WgkEXzjIjP/zQyK16+mbO2O273fwK1Lu67a5kMJb58Wd1VkWo/ZQpRXn+gclCx3dev0= tasks: - install: branch: reef exclude_packages: - ceph-volume - print: '**** done install task...' - print: '**** done start installing reef cephadm ...' - cephadm: compiled_cephadm_branch: reef conf: osd: osd_class_default_list: '*' osd_class_load_list: '*' image: quay.ceph.io/ceph-ci/ceph:reef - print: '**** done end installing reef cephadm ...' - print: '**** done start cephadm.shell ceph config set mgr...' - cephadm.shell: mon.a: - ceph config set mgr mgr/cephadm/use_repo_digest true --force - print: '**** done cephadm.shell ceph config set mgr...' - print: '**** done start telemetry reef...' - workunit: clients: client.0: - test_telemetry_reef.sh - print: '**** done end telemetry reef...' - print: '**** done start parallel' - parallel: - workload - upgrade-sequence - print: '**** done end parallel' - print: '**** done start telemetry x...' - workunit: clients: client.0: - test_telemetry_reef_x.sh - print: '**** done end telemetry x...' teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 6c637841c215537a4502385240412f1966e0faab timestamp: 2024-04-07_14:37:12 tube: smithi upgrade-sequence: sequential: - print: '**** done start upgrade, wait' - cephadm.shell: env: - sha1 mon.a: - ceph config set global log_to_journald false --force - ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 - while ceph orch upgrade status | jq '.in_progress' | grep true ; do ceph orch ps ; ceph versions ; sleep 30 ; done - ceph orch ps - ceph versions - ceph versions | jq -e '.overall | length == 1' - ceph versions | jq -e '.overall | keys' | grep $sha1 - print: '**** done end upgrade, wait...' user: yuriw verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 workload: full_sequential: - print: '**** done start ec-rados-default.yaml' - rados: clients: - client.0 ec_pool: true objects: 50 op_weights: append: 100 copy_from: 50 delete: 50 read: 100 rmattr: 25 rollback: 50 setattr: 25 snap_create: 50 snap_remove: 50 write: 0 ops: 4000 write_append_excl: false - print: '**** done end ec-rados-default.yaml' - print: '**** done start rados_api.yaml' - workunit: branch: reef clients: client.0: - cls - print: '**** done end rados_api.yaml' - print: '**** done start rados_loadgenbig.yaml' - workunit: branch: reef clients: client.0: - rados/load-gen-big.sh - print: '**** done end rados_loadgenbig.yaml' - print: '**** done start rbd_import_export.yaml' - workunit: branch: reef clients: client.1: - rbd/import_export.sh env: RBD_CREATE_ARGS: --new-format - print: '**** done end rbd_import_export.yaml' - print: '**** done start test_rbd_api.yaml' - workunit: branch: reef clients: client.0: - rbd/test_librbd.sh env: RBD_FEATURES: '61' - print: '**** done end test_rbd_api.yaml' - print: '**** done start test_rbd_python.yaml' - workunit: branch: reef clients: client.0: - rbd/test_librbd_python.sh env: RBD_FEATURES: '61' - print: '**** done end test_rbd_python.yaml' 2024-04-07T15:49:24.061 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_adce92d056533f22dc1ee37e3141d098b046ec1b/qa; will attempt to use it 2024-04-07T15:49:24.062 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_adce92d056533f22dc1ee37e3141d098b046ec1b/qa/tasks 2024-04-07T15:49:24.062 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-07T15:49:24.063 INFO:teuthology.task.internal:Checking packages... 2024-04-07T15:49:24.085 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash 'adce92d056533f22dc1ee37e3141d098b046ec1b' 2024-04-07T15:49:24.085 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-07T15:49:24.086 INFO:teuthology.packaging:ref: None 2024-04-07T15:49:24.086 INFO:teuthology.packaging:tag: None 2024-04-07T15:49:24.086 INFO:teuthology.packaging:branch: wip-yuri3-testing-2024-04-05-0825 2024-04-07T15:49:24.086 INFO:teuthology.packaging:sha1: adce92d056533f22dc1ee37e3141d098b046ec1b 2024-04-07T15:49:24.086 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=wip-yuri3-testing-2024-04-05-0825 2024-04-07T15:49:24.223 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2785-gadce92d0-1jammy 2024-04-07T15:49:24.224 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-07T15:49:24.230 INFO:teuthology.task.internal:no buildpackages task found 2024-04-07T15:49:24.230 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-07T15:49:24.240 INFO:teuthology.task.internal:Saving configuration 2024-04-07T15:49:24.256 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-07T15:49:24.262 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-07T15:49:24.287 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi100.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-04-07_14:37:12-rados-wip-yuri3-testing-2024-04-05-0825-distro-default-smithi/7643840', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-07 15:43:44.207867', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIMbgAcaq5r+SpYxnc+xsPGa8ZQ7yWgSGpccv/21xcnkV'} 2024-04-07T15:49:24.307 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi144.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-04-07_14:37:12-rados-wip-yuri3-testing-2024-04-05-0825-distro-default-smithi/7643840', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-07 15:43:44.206274', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIFl6AtacunckDc+LieAD+i7yTepRBczn/9FS6i1EK4jG'} 2024-04-07T15:49:24.307 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-07T15:49:24.313 INFO:teuthology.task.internal:roles: ubuntu@smithi100.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'mds.a', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a', 'alertmanager.a'] 2024-04-07T15:49:24.313 INFO:teuthology.task.internal:roles: ubuntu@smithi144.front.sepia.ceph.com - ['mon.b', 'mds.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'grafana.a', 'node-exporter.b'] 2024-04-07T15:49:24.313 INFO:teuthology.run_tasks:Running task console_log... 2024-04-07T15:49:24.387 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f9dfe51b550>, signals=[15]) 2024-04-07T15:49:24.387 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-07T15:49:24.394 INFO:teuthology.task.internal:Opening connections... 2024-04-07T15:49:24.394 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi100.front.sepia.ceph.com 2024-04-07T15:49:24.396 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi100.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-07T15:49:24.481 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi144.front.sepia.ceph.com 2024-04-07T15:49:24.481 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi144.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-07T15:49:24.558 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-07T15:49:24.565 DEBUG:teuthology.orchestra.run.smithi100:> uname -m 2024-04-07T15:49:24.570 INFO:teuthology.orchestra.run.smithi100.stdout:x86_64 2024-04-07T15:49:24.571 DEBUG:teuthology.orchestra.run.smithi100:> cat /etc/os-release 2024-04-07T15:49:24.622 INFO:teuthology.orchestra.run.smithi100.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-04-07T15:49:24.622 INFO:teuthology.orchestra.run.smithi100.stdout:NAME="Ubuntu" 2024-04-07T15:49:24.622 INFO:teuthology.orchestra.run.smithi100.stdout:VERSION_ID="22.04" 2024-04-07T15:49:24.622 INFO:teuthology.orchestra.run.smithi100.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-04-07T15:49:24.622 INFO:teuthology.orchestra.run.smithi100.stdout:VERSION_CODENAME=jammy 2024-04-07T15:49:24.622 INFO:teuthology.orchestra.run.smithi100.stdout:ID=ubuntu 2024-04-07T15:49:24.622 INFO:teuthology.orchestra.run.smithi100.stdout:ID_LIKE=debian 2024-04-07T15:49:24.622 INFO:teuthology.orchestra.run.smithi100.stdout:HOME_URL="https://www.ubuntu.com/" 2024-04-07T15:49:24.623 INFO:teuthology.orchestra.run.smithi100.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-04-07T15:49:24.623 INFO:teuthology.orchestra.run.smithi100.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-04-07T15:49:24.623 INFO:teuthology.orchestra.run.smithi100.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-04-07T15:49:24.623 INFO:teuthology.orchestra.run.smithi100.stdout:UBUNTU_CODENAME=jammy 2024-04-07T15:49:24.623 INFO:teuthology.lock.ops:Updating smithi100.front.sepia.ceph.com on lock server 2024-04-07T15:49:24.659 DEBUG:teuthology.orchestra.run.smithi144:> uname -m 2024-04-07T15:49:24.664 INFO:teuthology.orchestra.run.smithi144.stdout:x86_64 2024-04-07T15:49:24.665 DEBUG:teuthology.orchestra.run.smithi144:> cat /etc/os-release 2024-04-07T15:49:24.712 INFO:teuthology.orchestra.run.smithi144.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-04-07T15:49:24.712 INFO:teuthology.orchestra.run.smithi144.stdout:NAME="Ubuntu" 2024-04-07T15:49:24.712 INFO:teuthology.orchestra.run.smithi144.stdout:VERSION_ID="22.04" 2024-04-07T15:49:24.712 INFO:teuthology.orchestra.run.smithi144.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-04-07T15:49:24.712 INFO:teuthology.orchestra.run.smithi144.stdout:VERSION_CODENAME=jammy 2024-04-07T15:49:24.712 INFO:teuthology.orchestra.run.smithi144.stdout:ID=ubuntu 2024-04-07T15:49:24.712 INFO:teuthology.orchestra.run.smithi144.stdout:ID_LIKE=debian 2024-04-07T15:49:24.712 INFO:teuthology.orchestra.run.smithi144.stdout:HOME_URL="https://www.ubuntu.com/" 2024-04-07T15:49:24.712 INFO:teuthology.orchestra.run.smithi144.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-04-07T15:49:24.712 INFO:teuthology.orchestra.run.smithi144.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-04-07T15:49:24.713 INFO:teuthology.orchestra.run.smithi144.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-04-07T15:49:24.713 INFO:teuthology.orchestra.run.smithi144.stdout:UBUNTU_CODENAME=jammy 2024-04-07T15:49:24.713 INFO:teuthology.lock.ops:Updating smithi144.front.sepia.ceph.com on lock server 2024-04-07T15:49:24.741 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-07T15:49:24.751 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-07T15:49:24.762 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-07T15:49:24.762 DEBUG:teuthology.orchestra.run.smithi100:> test '!' -e /home/ubuntu/cephtest 2024-04-07T15:49:24.765 DEBUG:teuthology.orchestra.run.smithi144:> test '!' -e /home/ubuntu/cephtest 2024-04-07T15:49:24.769 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-07T15:49:24.775 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-07T15:49:24.775 DEBUG:teuthology.orchestra.run.smithi100:> test -z $(ls -A /var/lib/ceph) 2024-04-07T15:49:24.812 DEBUG:teuthology.orchestra.run.smithi144:> test -z $(ls -A /var/lib/ceph) 2024-04-07T15:49:24.817 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-07T15:49:24.869 INFO:teuthology.run_tasks:Running task kernel... 2024-04-07T15:49:24.884 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-04-07T15:49:24.884 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-04-07T15:49:24.884 DEBUG:teuthology.orchestra.run.smithi100:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-07T15:49:24.884 DEBUG:teuthology.orchestra.run.smithi144:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-07T15:49:24.889 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-07T15:49:24.889 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-07T15:49:24.889 DEBUG:teuthology.orchestra.run.smithi100:> uname -r 2024-04-07T15:49:24.890 DEBUG:teuthology.orchestra.run.smithi144:> uname -r 2024-04-07T15:49:24.936 INFO:teuthology.orchestra.run.smithi144.stdout:5.15.0-56-generic 2024-04-07T15:49:24.937 INFO:teuthology.task.kernel:Running kernel on smithi144: 5.15.0-56-generic 2024-04-07T15:49:24.937 DEBUG:teuthology.orchestra.run.smithi144:> sudo apt-get clean 2024-04-07T15:49:24.942 INFO:teuthology.orchestra.run.smithi100.stdout:5.15.0-56-generic 2024-04-07T15:49:24.942 INFO:teuthology.task.kernel:Running kernel on smithi100: 5.15.0-56-generic 2024-04-07T15:49:24.942 DEBUG:teuthology.orchestra.run.smithi100:> sudo apt-get clean 2024-04-07T15:49:25.063 DEBUG:teuthology.orchestra.run.smithi144:> sudo apt-get update 2024-04-07T15:49:25.083 DEBUG:teuthology.orchestra.run.smithi100:> sudo apt-get update 2024-04-07T15:49:25.215 INFO:teuthology.orchestra.run.smithi100.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-07T15:49:25.216 INFO:teuthology.orchestra.run.smithi100.stdout:Get:2 http://security.ubuntu.com/ubuntu jammy-security InRelease [110 kB] 2024-04-07T15:49:25.224 INFO:teuthology.orchestra.run.smithi100.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [119 kB] 2024-04-07T15:49:25.309 INFO:teuthology.orchestra.run.smithi100.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [109 kB] 2024-04-07T15:49:25.346 INFO:teuthology.orchestra.run.smithi144.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-07T15:49:25.346 INFO:teuthology.orchestra.run.smithi144.stdout:Get:2 http://security.ubuntu.com/ubuntu jammy-security InRelease [110 kB] 2024-04-07T15:49:25.392 INFO:teuthology.orchestra.run.smithi144.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [119 kB] 2024-04-07T15:49:25.449 INFO:teuthology.orchestra.run.smithi100.stdout:Get:5 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [436 kB] 2024-04-07T15:49:25.518 INFO:teuthology.orchestra.run.smithi100.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [1,519 kB] 2024-04-07T15:49:25.527 INFO:teuthology.orchestra.run.smithi100.stdout:Get:7 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,303 kB] 2024-04-07T15:49:25.569 INFO:teuthology.orchestra.run.smithi100.stdout:Get:8 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [233 kB] 2024-04-07T15:49:25.572 INFO:teuthology.orchestra.run.smithi100.stdout:Get:9 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [11.4 kB] 2024-04-07T15:49:25.573 INFO:teuthology.orchestra.run.smithi100.stdout:Get:10 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [1,616 kB] 2024-04-07T15:49:25.595 INFO:teuthology.orchestra.run.smithi100.stdout:Get:11 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [33.7 kB] 2024-04-07T15:49:25.595 INFO:teuthology.orchestra.run.smithi100.stdout:Get:12 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [271 kB] 2024-04-07T15:49:25.597 INFO:teuthology.orchestra.run.smithi100.stdout:Get:13 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [520 B] 2024-04-07T15:49:25.598 INFO:teuthology.orchestra.run.smithi100.stdout:Get:14 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [852 kB] 2024-04-07T15:49:25.605 INFO:teuthology.orchestra.run.smithi100.stdout:Get:15 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [599 kB] 2024-04-07T15:49:25.611 INFO:teuthology.orchestra.run.smithi100.stdout:Get:16 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [163 kB] 2024-04-07T15:49:25.612 INFO:teuthology.orchestra.run.smithi100.stdout:Get:17 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [16.8 kB] 2024-04-07T15:49:25.631 INFO:teuthology.orchestra.run.smithi100.stdout:Get:18 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.1 kB] 2024-04-07T15:49:25.631 INFO:teuthology.orchestra.run.smithi100.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [602 kB] 2024-04-07T15:49:25.631 INFO:teuthology.orchestra.run.smithi100.stdout:Get:20 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-04-07T15:49:25.635 INFO:teuthology.orchestra.run.smithi100.stdout:Get:21 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,476 B] 2024-04-07T15:49:25.635 INFO:teuthology.orchestra.run.smithi100.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [293 kB] 2024-04-07T15:49:25.635 INFO:teuthology.orchestra.run.smithi100.stdout:Get:23 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [260 B] 2024-04-07T15:49:25.638 INFO:teuthology.orchestra.run.smithi100.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [16.1 kB] 2024-04-07T15:49:25.638 INFO:teuthology.orchestra.run.smithi100.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [35.2 kB] 2024-04-07T15:49:25.638 INFO:teuthology.orchestra.run.smithi100.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [1,648 kB] 2024-04-07T15:49:25.654 INFO:teuthology.orchestra.run.smithi100.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [275 kB] 2024-04-07T15:49:25.656 INFO:teuthology.orchestra.run.smithi100.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [520 B] 2024-04-07T15:49:25.656 INFO:teuthology.orchestra.run.smithi100.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,060 kB] 2024-04-07T15:49:25.665 INFO:teuthology.orchestra.run.smithi100.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [698 kB] 2024-04-07T15:49:25.671 INFO:teuthology.orchestra.run.smithi100.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [241 kB] 2024-04-07T15:49:25.673 INFO:teuthology.orchestra.run.smithi100.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [22.1 kB] 2024-04-07T15:49:25.677 INFO:teuthology.orchestra.run.smithi100.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [6,028 B] 2024-04-07T15:49:25.678 INFO:teuthology.orchestra.run.smithi100.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [49.6 kB] 2024-04-07T15:49:25.678 INFO:teuthology.orchestra.run.smithi100.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [12.0 kB] 2024-04-07T15:49:25.679 INFO:teuthology.orchestra.run.smithi100.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [472 B] 2024-04-07T15:49:25.679 INFO:teuthology.orchestra.run.smithi100.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [67.1 kB] 2024-04-07T15:49:25.681 INFO:teuthology.orchestra.run.smithi100.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [59.2 kB] 2024-04-07T15:49:25.699 INFO:teuthology.orchestra.run.smithi100.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.0 kB] 2024-04-07T15:49:25.699 INFO:teuthology.orchestra.run.smithi100.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-04-07T15:49:25.699 INFO:teuthology.orchestra.run.smithi100.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [28.4 kB] 2024-04-07T15:49:25.700 INFO:teuthology.orchestra.run.smithi100.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [17.2 kB] 2024-04-07T15:49:25.700 INFO:teuthology.orchestra.run.smithi100.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.2 kB] 2024-04-07T15:49:25.701 INFO:teuthology.orchestra.run.smithi100.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [644 B] 2024-04-07T15:49:25.746 INFO:teuthology.orchestra.run.smithi144.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [109 kB] 2024-04-07T15:49:25.749 INFO:teuthology.orchestra.run.smithi144.stdout:Get:5 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,303 kB] 2024-04-07T15:49:25.885 INFO:teuthology.orchestra.run.smithi144.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [1,519 kB] 2024-04-07T15:49:26.163 INFO:teuthology.orchestra.run.smithi144.stdout:Get:7 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [436 kB] 2024-04-07T15:49:26.181 INFO:teuthology.orchestra.run.smithi144.stdout:Get:8 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [233 kB] 2024-04-07T15:49:26.190 INFO:teuthology.orchestra.run.smithi144.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [602 kB] 2024-04-07T15:49:26.191 INFO:teuthology.orchestra.run.smithi144.stdout:Get:10 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [11.4 kB] 2024-04-07T15:49:26.191 INFO:teuthology.orchestra.run.smithi144.stdout:Get:11 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [33.7 kB] 2024-04-07T15:49:26.193 INFO:teuthology.orchestra.run.smithi144.stdout:Get:12 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [1,616 kB] 2024-04-07T15:49:26.229 INFO:teuthology.orchestra.run.smithi144.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [293 kB] 2024-04-07T15:49:26.244 INFO:teuthology.orchestra.run.smithi144.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [16.1 kB] 2024-04-07T15:49:26.245 INFO:teuthology.orchestra.run.smithi144.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [1,648 kB] 2024-04-07T15:49:26.284 INFO:teuthology.orchestra.run.smithi144.stdout:Get:16 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [271 kB] 2024-04-07T15:49:26.295 INFO:teuthology.orchestra.run.smithi144.stdout:Get:17 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [520 B] 2024-04-07T15:49:26.316 INFO:teuthology.orchestra.run.smithi144.stdout:Get:18 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [599 kB] 2024-04-07T15:49:26.320 INFO:teuthology.orchestra.run.smithi144.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [35.2 kB] 2024-04-07T15:49:26.321 INFO:teuthology.orchestra.run.smithi144.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [275 kB] 2024-04-07T15:49:26.327 INFO:teuthology.orchestra.run.smithi144.stdout:Get:21 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [852 kB] 2024-04-07T15:49:26.335 INFO:teuthology.orchestra.run.smithi144.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [520 B] 2024-04-07T15:49:26.500 INFO:teuthology.orchestra.run.smithi144.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,060 kB] 2024-04-07T15:49:26.505 INFO:teuthology.orchestra.run.smithi144.stdout:Get:24 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [163 kB] 2024-04-07T15:49:26.911 INFO:teuthology.orchestra.run.smithi144.stdout:Get:25 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [16.8 kB] 2024-04-07T15:49:26.917 INFO:teuthology.orchestra.run.smithi144.stdout:Get:26 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-04-07T15:49:26.918 INFO:teuthology.orchestra.run.smithi144.stdout:Get:27 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.1 kB] 2024-04-07T15:49:26.931 INFO:teuthology.orchestra.run.smithi144.stdout:Get:28 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,476 B] 2024-04-07T15:49:26.933 INFO:teuthology.orchestra.run.smithi144.stdout:Get:29 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [260 B] 2024-04-07T15:49:27.133 INFO:teuthology.orchestra.run.smithi144.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [698 kB] 2024-04-07T15:49:27.170 INFO:teuthology.orchestra.run.smithi144.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [241 kB] 2024-04-07T15:49:27.183 INFO:teuthology.orchestra.run.smithi144.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [22.1 kB] 2024-04-07T15:49:27.184 INFO:teuthology.orchestra.run.smithi144.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [49.6 kB] 2024-04-07T15:49:27.187 INFO:teuthology.orchestra.run.smithi144.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [6,028 B] 2024-04-07T15:49:27.188 INFO:teuthology.orchestra.run.smithi144.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [12.0 kB] 2024-04-07T15:49:27.188 INFO:teuthology.orchestra.run.smithi144.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [472 B] 2024-04-07T15:49:27.188 INFO:teuthology.orchestra.run.smithi144.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [67.1 kB] 2024-04-07T15:49:27.215 INFO:teuthology.orchestra.run.smithi144.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [59.2 kB] 2024-04-07T15:49:27.218 INFO:teuthology.orchestra.run.smithi144.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.0 kB] 2024-04-07T15:49:27.382 INFO:teuthology.orchestra.run.smithi144.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-04-07T15:49:27.464 INFO:teuthology.orchestra.run.smithi144.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [17.2 kB] 2024-04-07T15:49:27.548 INFO:teuthology.orchestra.run.smithi144.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [28.4 kB] 2024-04-07T15:49:27.629 INFO:teuthology.orchestra.run.smithi144.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.2 kB] 2024-04-07T15:49:27.657 INFO:teuthology.orchestra.run.smithi144.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [644 B] 2024-04-07T15:49:31.357 INFO:teuthology.orchestra.run.smithi100.stdout:Fetched 12.6 MB in 2s (7,325 kB/s) 2024-04-07T15:49:31.972 INFO:teuthology.orchestra.run.smithi144.stdout:Fetched 12.6 MB in 3s (4,975 kB/s) 2024-04-07T15:49:32.679 INFO:teuthology.orchestra.run.smithi100.stdout:Reading package lists... 2024-04-07T15:49:32.696 DEBUG:teuthology.orchestra.run.smithi100:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-04-07T15:49:32.755 INFO:teuthology.orchestra.run.smithi100.stdout:Reading package lists... 2024-04-07T15:49:32.952 INFO:teuthology.orchestra.run.smithi100.stdout:Building dependency tree... 2024-04-07T15:49:32.952 INFO:teuthology.orchestra.run.smithi100.stdout:Reading state information... 2024-04-07T15:49:33.113 INFO:teuthology.orchestra.run.smithi100.stdout:The following packages were automatically installed and are no longer required: 2024-04-07T15:49:33.114 INFO:teuthology.orchestra.run.smithi100.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-07T15:49:33.114 INFO:teuthology.orchestra.run.smithi100.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-07T15:49:33.114 INFO:teuthology.orchestra.run.smithi100.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-07T15:49:33.114 INFO:teuthology.orchestra.run.smithi100.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-04-07T15:49:33.114 INFO:teuthology.orchestra.run.smithi100.stdout: python2.7-minimal 2024-04-07T15:49:33.114 INFO:teuthology.orchestra.run.smithi100.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-07T15:49:33.115 INFO:teuthology.orchestra.run.smithi100.stdout:The following additional packages will be installed: 2024-04-07T15:49:33.115 INFO:teuthology.orchestra.run.smithi100.stdout: linux-generic linux-headers-5.15.0-101 linux-headers-5.15.0-101-generic 2024-04-07T15:49:33.115 INFO:teuthology.orchestra.run.smithi100.stdout: linux-headers-generic linux-image-5.15.0-101-generic 2024-04-07T15:49:33.116 INFO:teuthology.orchestra.run.smithi100.stdout: linux-modules-5.15.0-101-generic linux-modules-extra-5.15.0-101-generic 2024-04-07T15:49:33.117 INFO:teuthology.orchestra.run.smithi100.stdout:Suggested packages: 2024-04-07T15:49:33.117 INFO:teuthology.orchestra.run.smithi100.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-04-07T15:49:33.117 INFO:teuthology.orchestra.run.smithi100.stdout:Recommended packages: 2024-04-07T15:49:33.117 INFO:teuthology.orchestra.run.smithi100.stdout: thermald 2024-04-07T15:49:33.163 INFO:teuthology.orchestra.run.smithi100.stdout:The following NEW packages will be installed: 2024-04-07T15:49:33.163 INFO:teuthology.orchestra.run.smithi100.stdout: linux-headers-5.15.0-101 linux-headers-5.15.0-101-generic 2024-04-07T15:49:33.163 INFO:teuthology.orchestra.run.smithi100.stdout: linux-image-5.15.0-101-generic linux-modules-5.15.0-101-generic 2024-04-07T15:49:33.164 INFO:teuthology.orchestra.run.smithi100.stdout: linux-modules-extra-5.15.0-101-generic 2024-04-07T15:49:33.165 INFO:teuthology.orchestra.run.smithi100.stdout:The following packages will be upgraded: 2024-04-07T15:49:33.165 INFO:teuthology.orchestra.run.smithi100.stdout: linux-generic linux-headers-generic linux-image-generic 2024-04-07T15:49:33.221 INFO:teuthology.orchestra.run.smithi100.stdout:3 upgraded, 5 newly installed, 0 to remove and 312 not upgraded. 2024-04-07T15:49:33.221 INFO:teuthology.orchestra.run.smithi100.stdout:Need to get 113 MB of archives. 2024-04-07T15:49:33.221 INFO:teuthology.orchestra.run.smithi100.stdout:After this operation, 582 MB of additional disk space will be used. 2024-04-07T15:49:33.221 INFO:teuthology.orchestra.run.smithi100.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-101-generic amd64 5.15.0-101.111 [22.6 MB] 2024-04-07T15:49:33.315 INFO:teuthology.orchestra.run.smithi144.stdout:Reading package lists... 2024-04-07T15:49:33.333 DEBUG:teuthology.orchestra.run.smithi144:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-04-07T15:49:33.394 INFO:teuthology.orchestra.run.smithi144.stdout:Reading package lists... 2024-04-07T15:49:33.589 INFO:teuthology.orchestra.run.smithi144.stdout:Building dependency tree... 2024-04-07T15:49:33.589 INFO:teuthology.orchestra.run.smithi144.stdout:Reading state information... 2024-04-07T15:49:33.761 INFO:teuthology.orchestra.run.smithi144.stdout:The following packages were automatically installed and are no longer required: 2024-04-07T15:49:33.761 INFO:teuthology.orchestra.run.smithi144.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-07T15:49:33.761 INFO:teuthology.orchestra.run.smithi144.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-07T15:49:33.762 INFO:teuthology.orchestra.run.smithi144.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-07T15:49:33.762 INFO:teuthology.orchestra.run.smithi144.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-04-07T15:49:33.762 INFO:teuthology.orchestra.run.smithi144.stdout: python2.7-minimal 2024-04-07T15:49:33.762 INFO:teuthology.orchestra.run.smithi144.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-07T15:49:33.763 INFO:teuthology.orchestra.run.smithi144.stdout:The following additional packages will be installed: 2024-04-07T15:49:33.763 INFO:teuthology.orchestra.run.smithi144.stdout: linux-generic linux-headers-5.15.0-101 linux-headers-5.15.0-101-generic 2024-04-07T15:49:33.763 INFO:teuthology.orchestra.run.smithi144.stdout: linux-headers-generic linux-image-5.15.0-101-generic 2024-04-07T15:49:33.763 INFO:teuthology.orchestra.run.smithi144.stdout: linux-modules-5.15.0-101-generic linux-modules-extra-5.15.0-101-generic 2024-04-07T15:49:33.764 INFO:teuthology.orchestra.run.smithi144.stdout:Suggested packages: 2024-04-07T15:49:33.765 INFO:teuthology.orchestra.run.smithi144.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-04-07T15:49:33.765 INFO:teuthology.orchestra.run.smithi144.stdout:Recommended packages: 2024-04-07T15:49:33.765 INFO:teuthology.orchestra.run.smithi144.stdout: thermald 2024-04-07T15:49:33.799 INFO:teuthology.orchestra.run.smithi100.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-101-generic amd64 5.15.0-101.111 [11.5 MB] 2024-04-07T15:49:33.820 INFO:teuthology.orchestra.run.smithi144.stdout:The following NEW packages will be installed: 2024-04-07T15:49:33.820 INFO:teuthology.orchestra.run.smithi144.stdout: linux-headers-5.15.0-101 linux-headers-5.15.0-101-generic 2024-04-07T15:49:33.820 INFO:teuthology.orchestra.run.smithi144.stdout: linux-image-5.15.0-101-generic linux-modules-5.15.0-101-generic 2024-04-07T15:49:33.820 INFO:teuthology.orchestra.run.smithi144.stdout: linux-modules-extra-5.15.0-101-generic 2024-04-07T15:49:33.821 INFO:teuthology.orchestra.run.smithi144.stdout:The following packages will be upgraded: 2024-04-07T15:49:33.822 INFO:teuthology.orchestra.run.smithi144.stdout: linux-generic linux-headers-generic linux-image-generic 2024-04-07T15:49:33.935 INFO:teuthology.orchestra.run.smithi144.stdout:3 upgraded, 5 newly installed, 0 to remove and 312 not upgraded. 2024-04-07T15:49:33.935 INFO:teuthology.orchestra.run.smithi144.stdout:Need to get 113 MB of archives. 2024-04-07T15:49:33.935 INFO:teuthology.orchestra.run.smithi144.stdout:After this operation, 582 MB of additional disk space will be used. 2024-04-07T15:49:33.935 INFO:teuthology.orchestra.run.smithi144.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-101-generic amd64 5.15.0-101.111 [22.6 MB] 2024-04-07T15:49:33.977 INFO:teuthology.orchestra.run.smithi100.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-101-generic amd64 5.15.0-101.111 [63.9 MB] 2024-04-07T15:49:35.167 INFO:teuthology.orchestra.run.smithi144.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-101-generic amd64 5.15.0-101.111 [11.5 MB] 2024-04-07T15:49:35.458 INFO:teuthology.orchestra.run.smithi144.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-101-generic amd64 5.15.0-101.111 [63.9 MB] 2024-04-07T15:49:35.736 INFO:teuthology.orchestra.run.smithi100.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.101.98 [1,698 B] 2024-04-07T15:49:35.736 INFO:teuthology.orchestra.run.smithi100.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.101.98 [2,486 B] 2024-04-07T15:49:35.736 INFO:teuthology.orchestra.run.smithi100.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-101 all 5.15.0-101.111 [12.3 MB] 2024-04-07T15:49:36.030 INFO:teuthology.orchestra.run.smithi100.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-101-generic amd64 5.15.0-101.111 [2,876 kB] 2024-04-07T15:49:36.086 INFO:teuthology.orchestra.run.smithi100.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.101.98 [2,332 B] 2024-04-07T15:49:36.587 INFO:teuthology.orchestra.run.smithi100.stdout:Fetched 113 MB in 3s (38.8 MB/s) 2024-04-07T15:49:36.708 INFO:teuthology.orchestra.run.smithi144.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.101.98 [1,698 B] 2024-04-07T15:49:36.708 INFO:teuthology.orchestra.run.smithi144.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.101.98 [2,486 B] 2024-04-07T15:49:36.708 INFO:teuthology.orchestra.run.smithi144.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-101 all 5.15.0-101.111 [12.3 MB] 2024-04-07T15:49:36.746 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package linux-modules-5.15.0-101-generic. 2024-04-07T15:49:37.182 INFO:teuthology.orchestra.run.smithi144.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-101-generic amd64 5.15.0-101.111 [2,876 kB] 2024-04-07T15:49:37.441 INFO:teuthology.orchestra.run.smithi144.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.101.98 [2,332 B] 2024-04-07T15:49:37.835 INFO:teuthology.orchestra.run.smithi144.stdout:Fetched 113 MB in 4s (31.3 MB/s) 2024-04-07T15:49:38.009 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package linux-modules-5.15.0-101-generic. 2024-04-07T15:49:38.778 INFO:teuthology.orchestra.run.smithi100.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-04-07T15:49:38.782 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../0-linux-modules-5.15.0-101-generic_5.15.0-101.111_amd64.deb ... 2024-04-07T15:49:38.852 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking linux-modules-5.15.0-101-generic (5.15.0-101.111) ... 2024-04-07T15:49:40.018 INFO:teuthology.orchestra.run.smithi144.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-04-07T15:49:40.022 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../0-linux-modules-5.15.0-101-generic_5.15.0-101.111_amd64.deb ... 2024-04-07T15:49:40.106 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking linux-modules-5.15.0-101-generic (5.15.0-101.111) ... 2024-04-07T15:49:40.919 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package linux-image-5.15.0-101-generic. 2024-04-07T15:49:40.941 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../1-linux-image-5.15.0-101-generic_5.15.0-101.111_amd64.deb ... 2024-04-07T15:49:41.027 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking linux-image-5.15.0-101-generic (5.15.0-101.111) ... 2024-04-07T15:49:41.472 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-101-generic. 2024-04-07T15:49:41.493 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-101-generic_5.15.0-101.111_amd64.deb ... 2024-04-07T15:49:41.530 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking linux-modules-extra-5.15.0-101-generic (5.15.0-101.111) ... 2024-04-07T15:49:42.241 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package linux-image-5.15.0-101-generic. 2024-04-07T15:49:42.262 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../1-linux-image-5.15.0-101-generic_5.15.0-101.111_amd64.deb ... 2024-04-07T15:49:42.351 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking linux-image-5.15.0-101-generic (5.15.0-101.111) ... 2024-04-07T15:49:43.144 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-101-generic. 2024-04-07T15:49:43.165 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-101-generic_5.15.0-101.111_amd64.deb ... 2024-04-07T15:49:43.202 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking linux-modules-extra-5.15.0-101-generic (5.15.0-101.111) ... 2024-04-07T15:49:47.715 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../3-linux-generic_5.15.0.101.98_amd64.deb ... 2024-04-07T15:49:47.840 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking linux-generic (5.15.0.101.98) over (5.15.0.56.54) ... 2024-04-07T15:49:48.139 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.101.98_amd64.deb ... 2024-04-07T15:49:48.277 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking linux-image-generic (5.15.0.101.98) over (5.15.0.56.54) ... 2024-04-07T15:49:48.579 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package linux-headers-5.15.0-101. 2024-04-07T15:49:48.590 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../5-linux-headers-5.15.0-101_5.15.0-101.111_all.deb ... 2024-04-07T15:49:48.629 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking linux-headers-5.15.0-101 (5.15.0-101.111) ... 2024-04-07T15:49:49.396 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../3-linux-generic_5.15.0.101.98_amd64.deb ... 2024-04-07T15:49:49.523 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking linux-generic (5.15.0.101.98) over (5.15.0.56.54) ... 2024-04-07T15:49:49.890 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.101.98_amd64.deb ... 2024-04-07T15:49:50.018 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking linux-image-generic (5.15.0.101.98) over (5.15.0.56.54) ... 2024-04-07T15:49:50.279 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package linux-headers-5.15.0-101. 2024-04-07T15:49:50.301 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../5-linux-headers-5.15.0-101_5.15.0-101.111_all.deb ... 2024-04-07T15:49:50.337 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking linux-headers-5.15.0-101 (5.15.0-101.111) ... 2024-04-07T15:49:54.455 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package linux-headers-5.15.0-101-generic. 2024-04-07T15:49:54.492 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../6-linux-headers-5.15.0-101-generic_5.15.0-101.111_amd64.deb ... 2024-04-07T15:49:54.530 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking linux-headers-5.15.0-101-generic (5.15.0-101.111) ... 2024-04-07T15:49:56.288 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package linux-headers-5.15.0-101-generic. 2024-04-07T15:49:56.325 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../6-linux-headers-5.15.0-101-generic_5.15.0-101.111_amd64.deb ... 2024-04-07T15:49:56.363 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking linux-headers-5.15.0-101-generic (5.15.0-101.111) ... 2024-04-07T15:49:57.140 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.101.98_amd64.deb ... 2024-04-07T15:49:57.266 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking linux-headers-generic (5.15.0.101.98) over (5.15.0.56.54) ... 2024-04-07T15:49:57.591 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up linux-headers-5.15.0-101 (5.15.0-101.111) ... 2024-04-07T15:49:57.704 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up linux-headers-5.15.0-101-generic (5.15.0-101.111) ... 2024-04-07T15:49:57.830 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up linux-headers-generic (5.15.0.101.98) ... 2024-04-07T15:49:57.959 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up linux-image-5.15.0-101-generic (5.15.0-101.111) ... 2024-04-07T15:49:58.952 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.101.98_amd64.deb ... 2024-04-07T15:49:59.076 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking linux-headers-generic (5.15.0.101.98) over (5.15.0.56.54) ... 2024-04-07T15:49:59.300 INFO:teuthology.orchestra.run.smithi100.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-04-07T15:49:59.300 INFO:teuthology.orchestra.run.smithi100.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-04-07T15:49:59.301 INFO:teuthology.orchestra.run.smithi100.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-101-generic 2024-04-07T15:49:59.301 INFO:teuthology.orchestra.run.smithi100.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-101-generic 2024-04-07T15:49:59.418 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up linux-headers-5.15.0-101 (5.15.0-101.111) ... 2024-04-07T15:49:59.470 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up linux-modules-extra-5.15.0-101-generic (5.15.0-101.111) ... 2024-04-07T15:49:59.540 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up linux-headers-5.15.0-101-generic (5.15.0-101.111) ... 2024-04-07T15:49:59.666 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up linux-headers-generic (5.15.0.101.98) ... 2024-04-07T15:49:59.820 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up linux-image-5.15.0-101-generic (5.15.0-101.111) ... 2024-04-07T15:50:01.259 INFO:teuthology.orchestra.run.smithi144.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-04-07T15:50:01.259 INFO:teuthology.orchestra.run.smithi144.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-04-07T15:50:01.259 INFO:teuthology.orchestra.run.smithi144.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-101-generic 2024-04-07T15:50:01.259 INFO:teuthology.orchestra.run.smithi144.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-101-generic 2024-04-07T15:50:01.397 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up linux-modules-5.15.0-101-generic (5.15.0-101.111) ... 2024-04-07T15:50:01.589 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up linux-modules-extra-5.15.0-101-generic (5.15.0-101.111) ... 2024-04-07T15:50:02.764 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up linux-image-generic (5.15.0.101.98) ... 2024-04-07T15:50:02.890 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up linux-generic (5.15.0.101.98) ... 2024-04-07T15:50:03.006 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up linux-modules-5.15.0-101-generic (5.15.0-101.111) ... 2024-04-07T15:50:03.016 INFO:teuthology.orchestra.run.smithi100.stdout:Processing triggers for linux-image-5.15.0-101-generic (5.15.0-101.111) ... 2024-04-07T15:50:03.075 INFO:teuthology.orchestra.run.smithi100.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-04-07T15:50:03.075 INFO:teuthology.orchestra.run.smithi100.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-101-generic 2024-04-07T15:50:04.374 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up linux-image-generic (5.15.0.101.98) ... 2024-04-07T15:50:04.475 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up linux-generic (5.15.0.101.98) ... 2024-04-07T15:50:04.576 INFO:teuthology.orchestra.run.smithi144.stdout:Processing triggers for linux-image-5.15.0-101-generic (5.15.0-101.111) ... 2024-04-07T15:50:04.642 INFO:teuthology.orchestra.run.smithi144.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-04-07T15:50:04.642 INFO:teuthology.orchestra.run.smithi144.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-101-generic 2024-04-07T15:50:17.429 INFO:teuthology.orchestra.run.smithi100.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-04-07T15:50:17.429 INFO:teuthology.orchestra.run.smithi100.stdout:Sourcing file `/etc/default/grub' 2024-04-07T15:50:17.440 INFO:teuthology.orchestra.run.smithi100.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-04-07T15:50:17.464 INFO:teuthology.orchestra.run.smithi100.stdout:Generating grub configuration file ... 2024-04-07T15:50:17.738 INFO:teuthology.orchestra.run.smithi100.stdout:Found linux image: /boot/vmlinuz-5.15.0-101-generic 2024-04-07T15:50:17.759 INFO:teuthology.orchestra.run.smithi100.stdout:Found initrd image: /boot/initrd.img-5.15.0-101-generic 2024-04-07T15:50:18.035 INFO:teuthology.orchestra.run.smithi100.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-04-07T15:50:18.040 INFO:teuthology.orchestra.run.smithi100.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-04-07T15:50:18.108 INFO:teuthology.orchestra.run.smithi100.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-04-07T15:50:18.113 INFO:teuthology.orchestra.run.smithi100.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-04-07T15:50:18.286 INFO:teuthology.orchestra.run.smithi100.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-04-07T15:50:18.286 INFO:teuthology.orchestra.run.smithi100.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-04-07T15:50:18.286 INFO:teuthology.orchestra.run.smithi100.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-04-07T15:50:18.333 INFO:teuthology.orchestra.run.smithi100.stdout:done 2024-04-07T15:50:18.580 INFO:teuthology.orchestra.run.smithi144.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-04-07T15:50:18.581 INFO:teuthology.orchestra.run.smithi144.stdout:Sourcing file `/etc/default/grub' 2024-04-07T15:50:18.598 INFO:teuthology.orchestra.run.smithi144.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-04-07T15:50:18.615 INFO:teuthology.orchestra.run.smithi144.stdout:Generating grub configuration file ... 2024-04-07T15:50:18.819 DEBUG:teuthology.orchestra.run.smithi100:> dpkg -s linux-image-generic 2024-04-07T15:50:18.839 INFO:teuthology.orchestra.run.smithi100.stdout:Package: linux-image-generic 2024-04-07T15:50:18.839 INFO:teuthology.orchestra.run.smithi100.stdout:Status: install ok installed 2024-04-07T15:50:18.839 INFO:teuthology.orchestra.run.smithi100.stdout:Priority: optional 2024-04-07T15:50:18.839 INFO:teuthology.orchestra.run.smithi100.stdout:Section: kernel 2024-04-07T15:50:18.839 INFO:teuthology.orchestra.run.smithi100.stdout:Installed-Size: 21 2024-04-07T15:50:18.839 INFO:teuthology.orchestra.run.smithi100.stdout:Maintainer: Ubuntu Kernel Team 2024-04-07T15:50:18.839 INFO:teuthology.orchestra.run.smithi100.stdout:Architecture: amd64 2024-04-07T15:50:18.839 INFO:teuthology.orchestra.run.smithi100.stdout:Source: linux-meta 2024-04-07T15:50:18.840 INFO:teuthology.orchestra.run.smithi100.stdout:Version: 5.15.0.101.98 2024-04-07T15:50:18.840 INFO:teuthology.orchestra.run.smithi100.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.3), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-101), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.3) 2024-04-07T15:50:18.840 INFO:teuthology.orchestra.run.smithi100.stdout:Depends: linux-image-5.15.0-101-generic, linux-modules-extra-5.15.0-101-generic, linux-firmware, intel-microcode, amd64-microcode 2024-04-07T15:50:18.840 INFO:teuthology.orchestra.run.smithi100.stdout:Recommends: thermald 2024-04-07T15:50:18.840 INFO:teuthology.orchestra.run.smithi100.stdout:Description: Generic Linux kernel image 2024-04-07T15:50:18.840 INFO:teuthology.orchestra.run.smithi100.stdout: This package will always depend on the latest generic kernel image 2024-04-07T15:50:18.840 INFO:teuthology.orchestra.run.smithi100.stdout: available. 2024-04-07T15:50:18.840 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-101-generic 2024-04-07T15:50:18.840 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-04-07T15:50:18.840 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-04-07T15:50:18.841 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-04-07T15:50:18.841 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi100.front.sepia.ceph.com, path=None, version=distro) 2024-04-07T15:50:18.841 DEBUG:teuthology.orchestra.run.smithi100:> sudo apt-get clean 2024-04-07T15:50:18.865 INFO:teuthology.orchestra.run.smithi144.stdout:Found linux image: /boot/vmlinuz-5.15.0-101-generic 2024-04-07T15:50:18.880 INFO:teuthology.orchestra.run.smithi144.stdout:Found initrd image: /boot/initrd.img-5.15.0-101-generic 2024-04-07T15:50:19.020 DEBUG:teuthology.orchestra.run.smithi100:> sudo apt-get update 2024-04-07T15:50:19.137 INFO:teuthology.orchestra.run.smithi100.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-07T15:50:19.137 INFO:teuthology.orchestra.run.smithi100.stdout:Hit:2 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-04-07T15:50:19.149 INFO:teuthology.orchestra.run.smithi100.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-04-07T15:50:19.170 INFO:teuthology.orchestra.run.smithi144.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-04-07T15:50:19.170 INFO:teuthology.orchestra.run.smithi100.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-04-07T15:50:19.175 INFO:teuthology.orchestra.run.smithi144.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-04-07T15:50:19.244 INFO:teuthology.orchestra.run.smithi144.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-04-07T15:50:19.250 INFO:teuthology.orchestra.run.smithi144.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-04-07T15:50:19.418 INFO:teuthology.orchestra.run.smithi144.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-04-07T15:50:19.419 INFO:teuthology.orchestra.run.smithi144.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-04-07T15:50:19.419 INFO:teuthology.orchestra.run.smithi144.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-04-07T15:50:19.463 INFO:teuthology.orchestra.run.smithi144.stdout:done 2024-04-07T15:50:19.964 DEBUG:teuthology.orchestra.run.smithi144:> dpkg -s linux-image-generic 2024-04-07T15:50:19.985 INFO:teuthology.orchestra.run.smithi144.stdout:Package: linux-image-generic 2024-04-07T15:50:19.985 INFO:teuthology.orchestra.run.smithi144.stdout:Status: install ok installed 2024-04-07T15:50:19.985 INFO:teuthology.orchestra.run.smithi144.stdout:Priority: optional 2024-04-07T15:50:19.985 INFO:teuthology.orchestra.run.smithi144.stdout:Section: kernel 2024-04-07T15:50:19.986 INFO:teuthology.orchestra.run.smithi144.stdout:Installed-Size: 21 2024-04-07T15:50:19.986 INFO:teuthology.orchestra.run.smithi144.stdout:Maintainer: Ubuntu Kernel Team 2024-04-07T15:50:19.986 INFO:teuthology.orchestra.run.smithi144.stdout:Architecture: amd64 2024-04-07T15:50:19.986 INFO:teuthology.orchestra.run.smithi144.stdout:Source: linux-meta 2024-04-07T15:50:19.986 INFO:teuthology.orchestra.run.smithi144.stdout:Version: 5.15.0.101.98 2024-04-07T15:50:19.986 INFO:teuthology.orchestra.run.smithi144.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.3), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-101), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.3) 2024-04-07T15:50:19.986 INFO:teuthology.orchestra.run.smithi144.stdout:Depends: linux-image-5.15.0-101-generic, linux-modules-extra-5.15.0-101-generic, linux-firmware, intel-microcode, amd64-microcode 2024-04-07T15:50:19.986 INFO:teuthology.orchestra.run.smithi144.stdout:Recommends: thermald 2024-04-07T15:50:19.986 INFO:teuthology.orchestra.run.smithi144.stdout:Description: Generic Linux kernel image 2024-04-07T15:50:19.986 INFO:teuthology.orchestra.run.smithi144.stdout: This package will always depend on the latest generic kernel image 2024-04-07T15:50:19.986 INFO:teuthology.orchestra.run.smithi144.stdout: available. 2024-04-07T15:50:19.986 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-101-generic 2024-04-07T15:50:19.987 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-04-07T15:50:19.987 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-04-07T15:50:19.987 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-04-07T15:50:19.987 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi144.front.sepia.ceph.com, path=None, version=distro) 2024-04-07T15:50:19.987 DEBUG:teuthology.orchestra.run.smithi144:> sudo apt-get clean 2024-04-07T15:50:20.182 DEBUG:teuthology.orchestra.run.smithi144:> sudo apt-get update 2024-04-07T15:50:20.299 INFO:teuthology.orchestra.run.smithi144.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-07T15:50:20.309 INFO:teuthology.orchestra.run.smithi144.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-04-07T15:50:20.329 INFO:teuthology.orchestra.run.smithi144.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-04-07T15:50:20.416 INFO:teuthology.orchestra.run.smithi144.stdout:Hit:4 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-04-07T15:50:20.496 INFO:teuthology.orchestra.run.smithi100.stdout:Reading package lists... 2024-04-07T15:50:20.512 DEBUG:teuthology.orchestra.run.smithi100:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-04-07T15:50:20.573 INFO:teuthology.orchestra.run.smithi100.stdout:Reading package lists... 2024-04-07T15:50:20.771 INFO:teuthology.orchestra.run.smithi100.stdout:Building dependency tree... 2024-04-07T15:50:20.772 INFO:teuthology.orchestra.run.smithi100.stdout:Reading state information... 2024-04-07T15:50:20.948 INFO:teuthology.orchestra.run.smithi100.stdout:linux-image-generic is already the newest version (5.15.0.101.98). 2024-04-07T15:50:20.948 INFO:teuthology.orchestra.run.smithi100.stdout:The following packages were automatically installed and are no longer required: 2024-04-07T15:50:20.948 INFO:teuthology.orchestra.run.smithi100.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-07T15:50:20.948 INFO:teuthology.orchestra.run.smithi100.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-07T15:50:20.948 INFO:teuthology.orchestra.run.smithi100.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-07T15:50:20.949 INFO:teuthology.orchestra.run.smithi100.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-07T15:50:20.949 INFO:teuthology.orchestra.run.smithi100.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-07T15:50:20.949 INFO:teuthology.orchestra.run.smithi100.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-07T15:50:20.991 INFO:teuthology.orchestra.run.smithi100.stdout:0 upgraded, 0 newly installed, 0 to remove and 312 not upgraded. 2024-04-07T15:50:20.993 DEBUG:teuthology.orchestra.run.smithi100:> dpkg -s linux-image-generic 2024-04-07T15:50:21.009 INFO:teuthology.orchestra.run.smithi100.stdout:Package: linux-image-generic 2024-04-07T15:50:21.009 INFO:teuthology.orchestra.run.smithi100.stdout:Status: install ok installed 2024-04-07T15:50:21.009 INFO:teuthology.orchestra.run.smithi100.stdout:Priority: optional 2024-04-07T15:50:21.009 INFO:teuthology.orchestra.run.smithi100.stdout:Section: kernel 2024-04-07T15:50:21.009 INFO:teuthology.orchestra.run.smithi100.stdout:Installed-Size: 21 2024-04-07T15:50:21.009 INFO:teuthology.orchestra.run.smithi100.stdout:Maintainer: Ubuntu Kernel Team 2024-04-07T15:50:21.009 INFO:teuthology.orchestra.run.smithi100.stdout:Architecture: amd64 2024-04-07T15:50:21.009 INFO:teuthology.orchestra.run.smithi100.stdout:Source: linux-meta 2024-04-07T15:50:21.009 INFO:teuthology.orchestra.run.smithi100.stdout:Version: 5.15.0.101.98 2024-04-07T15:50:21.010 INFO:teuthology.orchestra.run.smithi100.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.3), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-101), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.3) 2024-04-07T15:50:21.010 INFO:teuthology.orchestra.run.smithi100.stdout:Depends: linux-image-5.15.0-101-generic, linux-modules-extra-5.15.0-101-generic, linux-firmware, intel-microcode, amd64-microcode 2024-04-07T15:50:21.010 INFO:teuthology.orchestra.run.smithi100.stdout:Recommends: thermald 2024-04-07T15:50:21.010 INFO:teuthology.orchestra.run.smithi100.stdout:Description: Generic Linux kernel image 2024-04-07T15:50:21.010 INFO:teuthology.orchestra.run.smithi100.stdout: This package will always depend on the latest generic kernel image 2024-04-07T15:50:21.010 INFO:teuthology.orchestra.run.smithi100.stdout: available. 2024-04-07T15:50:21.010 DEBUG:teuthology.orchestra.run.smithi100:> mktemp 2024-04-07T15:50:21.057 INFO:teuthology.orchestra.run.smithi100.stdout:/tmp/tmp.O94aQtXwQL 2024-04-07T15:50:21.057 DEBUG:teuthology.orchestra.run.smithi100:> sudo cp /boot/grub/grub.cfg /tmp/tmp.O94aQtXwQL 2024-04-07T15:50:21.114 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod 0666 /tmp/tmp.O94aQtXwQL 2024-04-07T15:50:21.240 DEBUG:teuthology.orchestra.remote:smithi100:/tmp/tmp.O94aQtXwQL is 10KB 2024-04-07T15:50:21.255 DEBUG:teuthology.orchestra.run.smithi100:> rm -fr /tmp/tmp.O94aQtXwQL 2024-04-07T15:50:21.260 DEBUG:teuthology.orchestra.run.smithi100:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-04-07T15:50:21.317 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-07T15:50:21.317 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-04-07T15:50:21.317 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-04-07T15:50:21.384 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-101-generic 2024-04-07T15:50:21.384 DEBUG:teuthology.orchestra.run.smithi100:> sudo update-grub 2024-04-07T15:50:21.716 INFO:teuthology.orchestra.run.smithi144.stdout:Reading package lists... 2024-04-07T15:50:21.734 DEBUG:teuthology.orchestra.run.smithi144:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-04-07T15:50:21.785 INFO:teuthology.orchestra.run.smithi144.stdout:Reading package lists... 2024-04-07T15:50:21.960 INFO:teuthology.orchestra.run.smithi144.stdout:Building dependency tree... 2024-04-07T15:50:21.960 INFO:teuthology.orchestra.run.smithi144.stdout:Reading state information... 2024-04-07T15:50:22.111 INFO:teuthology.orchestra.run.smithi144.stdout:linux-image-generic is already the newest version (5.15.0.101.98). 2024-04-07T15:50:22.111 INFO:teuthology.orchestra.run.smithi144.stdout:The following packages were automatically installed and are no longer required: 2024-04-07T15:50:22.111 INFO:teuthology.orchestra.run.smithi144.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-07T15:50:22.111 INFO:teuthology.orchestra.run.smithi144.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-07T15:50:22.111 INFO:teuthology.orchestra.run.smithi144.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-07T15:50:22.112 INFO:teuthology.orchestra.run.smithi144.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-07T15:50:22.112 INFO:teuthology.orchestra.run.smithi144.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-07T15:50:22.112 INFO:teuthology.orchestra.run.smithi144.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-07T15:50:22.149 INFO:teuthology.orchestra.run.smithi144.stdout:0 upgraded, 0 newly installed, 0 to remove and 312 not upgraded. 2024-04-07T15:50:22.151 DEBUG:teuthology.orchestra.run.smithi144:> dpkg -s linux-image-generic 2024-04-07T15:50:22.206 INFO:teuthology.orchestra.run.smithi144.stdout:Package: linux-image-generic 2024-04-07T15:50:22.206 INFO:teuthology.orchestra.run.smithi144.stdout:Status: install ok installed 2024-04-07T15:50:22.206 INFO:teuthology.orchestra.run.smithi144.stdout:Priority: optional 2024-04-07T15:50:22.206 INFO:teuthology.orchestra.run.smithi144.stdout:Section: kernel 2024-04-07T15:50:22.206 INFO:teuthology.orchestra.run.smithi144.stdout:Installed-Size: 21 2024-04-07T15:50:22.206 INFO:teuthology.orchestra.run.smithi144.stdout:Maintainer: Ubuntu Kernel Team 2024-04-07T15:50:22.206 INFO:teuthology.orchestra.run.smithi144.stdout:Architecture: amd64 2024-04-07T15:50:22.207 INFO:teuthology.orchestra.run.smithi144.stdout:Source: linux-meta 2024-04-07T15:50:22.207 INFO:teuthology.orchestra.run.smithi144.stdout:Version: 5.15.0.101.98 2024-04-07T15:50:22.207 INFO:teuthology.orchestra.run.smithi144.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.3), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-101), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.3) 2024-04-07T15:50:22.207 INFO:teuthology.orchestra.run.smithi144.stdout:Depends: linux-image-5.15.0-101-generic, linux-modules-extra-5.15.0-101-generic, linux-firmware, intel-microcode, amd64-microcode 2024-04-07T15:50:22.207 INFO:teuthology.orchestra.run.smithi144.stdout:Recommends: thermald 2024-04-07T15:50:22.207 INFO:teuthology.orchestra.run.smithi144.stdout:Description: Generic Linux kernel image 2024-04-07T15:50:22.207 INFO:teuthology.orchestra.run.smithi144.stdout: This package will always depend on the latest generic kernel image 2024-04-07T15:50:22.207 INFO:teuthology.orchestra.run.smithi144.stdout: available. 2024-04-07T15:50:22.207 DEBUG:teuthology.orchestra.run.smithi144:> mktemp 2024-04-07T15:50:22.251 INFO:teuthology.orchestra.run.smithi144.stdout:/tmp/tmp.qeYvGqaQK9 2024-04-07T15:50:22.252 DEBUG:teuthology.orchestra.run.smithi144:> sudo cp /boot/grub/grub.cfg /tmp/tmp.qeYvGqaQK9 2024-04-07T15:50:22.308 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod 0666 /tmp/tmp.qeYvGqaQK9 2024-04-07T15:50:22.429 DEBUG:teuthology.orchestra.remote:smithi144:/tmp/tmp.qeYvGqaQK9 is 10KB 2024-04-07T15:50:22.441 DEBUG:teuthology.orchestra.run.smithi144:> rm -fr /tmp/tmp.qeYvGqaQK9 2024-04-07T15:50:22.446 DEBUG:teuthology.orchestra.run.smithi144:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-04-07T15:50:22.499 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-04-07T15:50:22.499 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-04-07T15:50:22.499 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-04-07T15:50:22.517 INFO:teuthology.orchestra.run.smithi100.stderr:Sourcing file `/etc/default/grub' 2024-04-07T15:50:22.519 INFO:teuthology.orchestra.run.smithi100.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-04-07T15:50:22.521 INFO:teuthology.orchestra.run.smithi100.stderr:Generating grub configuration file ... 2024-04-07T15:50:22.566 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-101-generic 2024-04-07T15:50:22.566 DEBUG:teuthology.orchestra.run.smithi144:> sudo update-grub 2024-04-07T15:50:22.705 INFO:teuthology.orchestra.run.smithi100.stderr:Found linux image: /boot/vmlinuz-5.15.0-101-generic 2024-04-07T15:50:22.710 INFO:teuthology.orchestra.run.smithi100.stderr:Found initrd image: /boot/initrd.img-5.15.0-101-generic 2024-04-07T15:50:22.951 INFO:teuthology.orchestra.run.smithi100.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-04-07T15:50:22.956 INFO:teuthology.orchestra.run.smithi100.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-04-07T15:50:22.999 INFO:teuthology.orchestra.run.smithi100.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-04-07T15:50:23.004 INFO:teuthology.orchestra.run.smithi100.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-04-07T15:50:23.131 INFO:teuthology.orchestra.run.smithi100.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-04-07T15:50:23.131 INFO:teuthology.orchestra.run.smithi100.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-04-07T15:50:23.131 INFO:teuthology.orchestra.run.smithi100.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-04-07T15:50:23.150 INFO:teuthology.orchestra.run.smithi100.stderr:done 2024-04-07T15:50:23.169 DEBUG:teuthology.orchestra.run.smithi100:> sudo shutdown -r now 2024-04-07T15:50:23.719 INFO:teuthology.orchestra.run.smithi144.stderr:Sourcing file `/etc/default/grub' 2024-04-07T15:50:23.720 INFO:teuthology.orchestra.run.smithi144.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-04-07T15:50:23.723 INFO:teuthology.orchestra.run.smithi144.stderr:Generating grub configuration file ... 2024-04-07T15:50:23.923 INFO:teuthology.orchestra.run.smithi144.stderr:Found linux image: /boot/vmlinuz-5.15.0-101-generic 2024-04-07T15:50:23.930 INFO:teuthology.orchestra.run.smithi144.stderr:Found initrd image: /boot/initrd.img-5.15.0-101-generic 2024-04-07T15:50:24.161 INFO:teuthology.orchestra.run.smithi144.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-04-07T15:50:24.166 INFO:teuthology.orchestra.run.smithi144.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-04-07T15:50:24.208 INFO:teuthology.orchestra.run.smithi144.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-04-07T15:50:24.213 INFO:teuthology.orchestra.run.smithi144.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-04-07T15:50:24.337 INFO:teuthology.orchestra.run.smithi144.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-04-07T15:50:24.338 INFO:teuthology.orchestra.run.smithi144.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-04-07T15:50:24.338 INFO:teuthology.orchestra.run.smithi144.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-04-07T15:50:24.357 INFO:teuthology.orchestra.run.smithi144.stderr:done 2024-04-07T15:50:24.371 DEBUG:teuthology.orchestra.run.smithi144:> sudo shutdown -r now 2024-04-07T15:50:53.198 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-04-07T15:50:53.199 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi100.front.sepia.ceph.com' 2024-04-07T15:50:53.199 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi100.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-07T15:50:54.374 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-04-07T15:50:54.374 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi144.front.sepia.ceph.com' 2024-04-07T15:50:54.375 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi144.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-07T15:51:27.774 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.100 2024-04-07T15:51:29.054 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.144 2024-04-07T15:51:36.781 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi100.front.sepia.ceph.com' 2024-04-07T15:51:36.782 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi100.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-07T15:51:38.056 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi144.front.sepia.ceph.com' 2024-04-07T15:51:38.057 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi144.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-07T15:51:55.166 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.100 2024-04-07T15:51:56.446 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.144 2024-04-07T15:52:07.178 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi100.front.sepia.ceph.com' 2024-04-07T15:52:07.179 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi100.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-07T15:52:07.428 DEBUG:teuthology.orchestra.run.smithi100:> true 2024-04-07T15:52:07.944 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi100.front.sepia.ceph.com' 2024-04-07T15:52:07.944 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.15.0-101-generic"... 2024-04-07T15:52:07.944 DEBUG:teuthology.orchestra.run.smithi100:> uname -r 2024-04-07T15:52:07.989 INFO:teuthology.orchestra.run.smithi100.stdout:5.15.0-101-generic 2024-04-07T15:52:07.989 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-101-generic vs 5.15.0-101-generic 2024-04-07T15:52:07.989 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-07T15:52:07.990 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-04-07T15:52:08.447 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi144.front.sepia.ceph.com' 2024-04-07T15:52:08.448 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi144.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-07T15:52:08.741 DEBUG:teuthology.orchestra.run.smithi144:> true 2024-04-07T15:52:08.991 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-04-07T15:52:08.991 DEBUG:teuthology.orchestra.run.smithi100:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-07T15:52:09.063 INFO:teuthology.orchestra.run.smithi100.stdout:ttyS1 2024-04-07T15:52:09.092 DEBUG:teuthology.parallel:result is None 2024-04-07T15:52:09.415 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi144.front.sepia.ceph.com' 2024-04-07T15:52:09.415 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.15.0-101-generic"... 2024-04-07T15:52:09.415 DEBUG:teuthology.orchestra.run.smithi144:> uname -r 2024-04-07T15:52:09.460 INFO:teuthology.orchestra.run.smithi144.stdout:5.15.0-101-generic 2024-04-07T15:52:09.461 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-101-generic vs 5.15.0-101-generic 2024-04-07T15:52:09.461 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-07T15:52:09.461 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-04-07T15:52:10.461 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-04-07T15:52:10.461 DEBUG:teuthology.orchestra.run.smithi144:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-07T15:52:10.546 INFO:teuthology.orchestra.run.smithi144.stdout:ttyS1 2024-04-07T15:52:10.562 DEBUG:teuthology.parallel:result is None 2024-04-07T15:52:10.562 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-07T15:52:10.569 INFO:teuthology.task.internal:Creating test directory... 2024-04-07T15:52:10.569 DEBUG:teuthology.orchestra.run.smithi100:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-07T15:52:10.571 DEBUG:teuthology.orchestra.run.smithi144:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-07T15:52:10.576 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-07T15:52:10.582 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-07T15:52:10.593 INFO:teuthology.task.internal:Creating archive directory... 2024-04-07T15:52:10.593 DEBUG:teuthology.orchestra.run.smithi100:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-07T15:52:10.618 DEBUG:teuthology.orchestra.run.smithi144:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-07T15:52:10.722 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-07T15:52:10.730 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-07T15:52:10.730 DEBUG:teuthology.orchestra.run.smithi100:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-07T15:52:10.733 DEBUG:teuthology.orchestra.run.smithi144:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-07T15:52:10.750 INFO:teuthology.orchestra.run.smithi100.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-07T15:52:10.758 INFO:teuthology.orchestra.run.smithi100.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-07T15:52:10.760 INFO:teuthology.orchestra.run.smithi144.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-07T15:52:10.768 INFO:teuthology.orchestra.run.smithi144.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-07T15:52:10.769 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-07T15:52:10.776 INFO:teuthology.task.internal:Configuring sudo... 2024-04-07T15:52:10.776 DEBUG:teuthology.orchestra.run.smithi100:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-07T15:52:10.802 DEBUG:teuthology.orchestra.run.smithi144:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-07T15:52:10.826 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-07T15:52:10.835 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-07T15:52:10.835 DEBUG:teuthology.orchestra.run.smithi100:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-07T15:52:10.870 DEBUG:teuthology.orchestra.run.smithi144:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-07T15:52:10.876 DEBUG:teuthology.orchestra.run.smithi100:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-07T15:52:10.922 DEBUG:teuthology.orchestra.run.smithi100:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-07T15:52:10.969 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-07T15:52:10.970 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-07T15:52:11.031 DEBUG:teuthology.orchestra.run.smithi144:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-07T15:52:11.036 DEBUG:teuthology.orchestra.run.smithi144:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-07T15:52:11.085 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-04-07T15:52:11.085 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-07T15:52:11.150 DEBUG:teuthology.orchestra.run.smithi100:> sudo service rsyslog restart 2024-04-07T15:52:11.152 DEBUG:teuthology.orchestra.run.smithi144:> sudo service rsyslog restart 2024-04-07T15:52:11.230 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-07T15:52:11.237 INFO:teuthology.task.internal:Starting timer... 2024-04-07T15:52:11.238 INFO:teuthology.run_tasks:Running task pcp... 2024-04-07T15:52:11.247 INFO:teuthology.run_tasks:Running task selinux... 2024-04-07T15:52:11.256 DEBUG:teuthology.task.selinux:Excluding smithi100: OS 'ubuntu' does not support SELinux 2024-04-07T15:52:11.256 DEBUG:teuthology.task.selinux:Excluding smithi144: OS 'ubuntu' does not support SELinux 2024-04-07T15:52:11.256 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-07T15:52:11.256 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2024-04-07T15:52:11.256 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-07T15:52:11.256 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-07T15:52:11.265 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-07T15:52:11.285 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-07T15:52:11.286 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi100.front.sepia.ceph.com,smithi144.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-07T15:59:07.550 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi100.front.sepia.ceph.com'), Remote(name='ubuntu@smithi144.front.sepia.ceph.com')] 2024-04-07T15:59:07.551 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi100.front.sepia.ceph.com' 2024-04-07T15:59:07.552 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi100.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-07T15:59:07.630 DEBUG:teuthology.orchestra.run.smithi100:> true 2024-04-07T15:59:07.706 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi100.front.sepia.ceph.com' 2024-04-07T15:59:07.706 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi144.front.sepia.ceph.com' 2024-04-07T15:59:07.707 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi144.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-07T15:59:07.789 DEBUG:teuthology.orchestra.run.smithi144:> true 2024-04-07T15:59:07.866 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi144.front.sepia.ceph.com' 2024-04-07T15:59:07.866 INFO:teuthology.run_tasks:Running task clock... 2024-04-07T15:59:07.876 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-07T15:59:07.876 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-07T15:59:07.877 DEBUG:teuthology.orchestra.run.smithi100:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-07T15:59:07.879 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-07T15:59:07.879 DEBUG:teuthology.orchestra.run.smithi144:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-07T15:59:07.904 INFO:teuthology.orchestra.run.smithi100.stdout: 7 Apr 15:59:07 ntpd[17192]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-04-07T15:59:07.904 INFO:teuthology.orchestra.run.smithi100.stdout: 7 Apr 15:59:07 ntpd[17192]: Command line: ntpd -gq 2024-04-07T15:59:07.904 INFO:teuthology.orchestra.run.smithi100.stdout: 7 Apr 15:59:07 ntpd[17192]: ---------------------------------------------------- 2024-04-07T15:59:07.904 INFO:teuthology.orchestra.run.smithi100.stdout: 7 Apr 15:59:07 ntpd[17192]: ntp-4 is maintained by Network Time Foundation, 2024-04-07T15:59:07.904 INFO:teuthology.orchestra.run.smithi100.stdout: 7 Apr 15:59:07 ntpd[17192]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-04-07T15:59:07.904 INFO:teuthology.orchestra.run.smithi100.stdout: 7 Apr 15:59:07 ntpd[17192]: corporation. Support and training for ntp-4 are 2024-04-07T15:59:07.904 INFO:teuthology.orchestra.run.smithi100.stdout: 7 Apr 15:59:07 ntpd[17192]: available at https://www.nwtime.org/support 2024-04-07T15:59:07.904 INFO:teuthology.orchestra.run.smithi100.stdout: 7 Apr 15:59:07 ntpd[17192]: ---------------------------------------------------- 2024-04-07T15:59:07.905 INFO:teuthology.orchestra.run.smithi100.stdout: 7 Apr 15:59:07 ntpd[17192]: proto: precision = 0.046 usec (-24) 2024-04-07T15:59:07.905 INFO:teuthology.orchestra.run.smithi100.stdout: 7 Apr 15:59:07 ntpd[17192]: basedate set to 2022-02-04 2024-04-07T15:59:07.905 INFO:teuthology.orchestra.run.smithi100.stdout: 7 Apr 15:59:07 ntpd[17192]: gps base set to 2022-02-06 (week 2196) 2024-04-07T15:59:07.906 INFO:teuthology.orchestra.run.smithi100.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-04-07T15:59:07.906 INFO:teuthology.orchestra.run.smithi100.stdout: 7 Apr 15:59:07 ntpd[17192]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-04-07T15:59:07.906 INFO:teuthology.orchestra.run.smithi100.stdout: 7 Apr 15:59:07 ntpd[17192]: restrict ::: KOD does nothing without LIMITED. 2024-04-07T15:59:07.906 INFO:teuthology.orchestra.run.smithi100.stderr:restrict ::: KOD does nothing without LIMITED. 2024-04-07T15:59:07.907 INFO:teuthology.orchestra.run.smithi100.stdout: 7 Apr 15:59:07 ntpd[17192]: Listen and drop on 0 v6wildcard [::]:123 2024-04-07T15:59:07.907 INFO:teuthology.orchestra.run.smithi100.stdout: 7 Apr 15:59:07 ntpd[17192]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-04-07T15:59:07.907 INFO:teuthology.orchestra.run.smithi100.stdout: 7 Apr 15:59:07 ntpd[17192]: Listen normally on 2 lo 127.0.0.1:123 2024-04-07T15:59:07.907 INFO:teuthology.orchestra.run.smithi100.stdout: 7 Apr 15:59:07 ntpd[17192]: Listen normally on 3 enp3s0f1 172.21.15.100:123 2024-04-07T15:59:07.907 INFO:teuthology.orchestra.run.smithi100.stdout: 7 Apr 15:59:07 ntpd[17192]: Listen normally on 4 lo [::1]:123 2024-04-07T15:59:07.907 INFO:teuthology.orchestra.run.smithi100.stdout: 7 Apr 15:59:07 ntpd[17192]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe8f:d733%5]:123 2024-04-07T15:59:07.908 INFO:teuthology.orchestra.run.smithi100.stdout: 7 Apr 15:59:07 ntpd[17192]: Listening on routing socket on fd #22 for interface updates 2024-04-07T15:59:07.932 INFO:teuthology.orchestra.run.smithi144.stdout: 7 Apr 15:59:07 ntpd[17205]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-04-07T15:59:07.932 INFO:teuthology.orchestra.run.smithi144.stdout: 7 Apr 15:59:07 ntpd[17205]: Command line: ntpd -gq 2024-04-07T15:59:07.932 INFO:teuthology.orchestra.run.smithi144.stdout: 7 Apr 15:59:07 ntpd[17205]: ---------------------------------------------------- 2024-04-07T15:59:07.933 INFO:teuthology.orchestra.run.smithi144.stdout: 7 Apr 15:59:07 ntpd[17205]: ntp-4 is maintained by Network Time Foundation, 2024-04-07T15:59:07.933 INFO:teuthology.orchestra.run.smithi144.stdout: 7 Apr 15:59:07 ntpd[17205]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-04-07T15:59:07.933 INFO:teuthology.orchestra.run.smithi144.stdout: 7 Apr 15:59:07 ntpd[17205]: corporation. Support and training for ntp-4 are 2024-04-07T15:59:07.933 INFO:teuthology.orchestra.run.smithi144.stdout: 7 Apr 15:59:07 ntpd[17205]: available at https://www.nwtime.org/support 2024-04-07T15:59:07.933 INFO:teuthology.orchestra.run.smithi144.stdout: 7 Apr 15:59:07 ntpd[17205]: ---------------------------------------------------- 2024-04-07T15:59:07.934 INFO:teuthology.orchestra.run.smithi144.stdout: 7 Apr 15:59:07 ntpd[17205]: proto: precision = 0.056 usec (-24) 2024-04-07T15:59:07.934 INFO:teuthology.orchestra.run.smithi144.stdout: 7 Apr 15:59:07 ntpd[17205]: basedate set to 2022-02-04 2024-04-07T15:59:07.934 INFO:teuthology.orchestra.run.smithi144.stdout: 7 Apr 15:59:07 ntpd[17205]: gps base set to 2022-02-06 (week 2196) 2024-04-07T15:59:07.934 INFO:teuthology.orchestra.run.smithi144.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-04-07T15:59:07.934 INFO:teuthology.orchestra.run.smithi144.stdout: 7 Apr 15:59:07 ntpd[17205]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-04-07T15:59:07.935 INFO:teuthology.orchestra.run.smithi144.stdout: 7 Apr 15:59:07 ntpd[17205]: restrict ::: KOD does nothing without LIMITED. 2024-04-07T15:59:07.935 INFO:teuthology.orchestra.run.smithi144.stderr:restrict ::: KOD does nothing without LIMITED. 2024-04-07T15:59:07.936 INFO:teuthology.orchestra.run.smithi144.stdout: 7 Apr 15:59:07 ntpd[17205]: Listen and drop on 0 v6wildcard [::]:123 2024-04-07T15:59:07.936 INFO:teuthology.orchestra.run.smithi144.stdout: 7 Apr 15:59:07 ntpd[17205]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-04-07T15:59:07.936 INFO:teuthology.orchestra.run.smithi144.stdout: 7 Apr 15:59:07 ntpd[17205]: Listen normally on 2 lo 127.0.0.1:123 2024-04-07T15:59:07.936 INFO:teuthology.orchestra.run.smithi144.stdout: 7 Apr 15:59:07 ntpd[17205]: Listen normally on 3 enp3s0f1 172.21.15.144:123 2024-04-07T15:59:07.936 INFO:teuthology.orchestra.run.smithi144.stdout: 7 Apr 15:59:07 ntpd[17205]: Listen normally on 4 lo [::1]:123 2024-04-07T15:59:07.936 INFO:teuthology.orchestra.run.smithi144.stdout: 7 Apr 15:59:07 ntpd[17205]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:7f5f%5]:123 2024-04-07T15:59:07.936 INFO:teuthology.orchestra.run.smithi144.stdout: 7 Apr 15:59:07 ntpd[17205]: Listening on routing socket on fd #22 for interface updates 2024-04-07T15:59:08.905 INFO:teuthology.orchestra.run.smithi100.stderr: 7 Apr 15:59:08 ntpd[17192]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-07T15:59:08.906 INFO:teuthology.orchestra.run.smithi100.stderr: 7 Apr 15:59:08 ntpd[17192]: can't open /var/log/ntpstats/rawstats.20240407: Permission denied 2024-04-07T15:59:08.906 INFO:teuthology.orchestra.run.smithi100.stderr: 7 Apr 15:59:08 ntpd[17192]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-07T15:59:08.906 INFO:teuthology.orchestra.run.smithi100.stderr: 7 Apr 15:59:08 ntpd[17192]: can't open /var/log/ntpstats/rawstats.20240407: Permission denied 2024-04-07T15:59:08.906 INFO:teuthology.orchestra.run.smithi100.stderr: 7 Apr 15:59:08 ntpd[17192]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-07T15:59:08.906 INFO:teuthology.orchestra.run.smithi100.stderr: 7 Apr 15:59:08 ntpd[17192]: can't open /var/log/ntpstats/peerstats.20240407: Permission denied 2024-04-07T15:59:08.933 INFO:teuthology.orchestra.run.smithi144.stderr: 7 Apr 15:59:08 ntpd[17205]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-07T15:59:08.933 INFO:teuthology.orchestra.run.smithi144.stderr: 7 Apr 15:59:08 ntpd[17205]: can't open /var/log/ntpstats/rawstats.20240407: Permission denied 2024-04-07T15:59:08.933 INFO:teuthology.orchestra.run.smithi144.stderr: 7 Apr 15:59:08 ntpd[17205]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-07T15:59:08.933 INFO:teuthology.orchestra.run.smithi144.stderr: 7 Apr 15:59:08 ntpd[17205]: can't open /var/log/ntpstats/rawstats.20240407: Permission denied 2024-04-07T15:59:08.933 INFO:teuthology.orchestra.run.smithi144.stderr: 7 Apr 15:59:08 ntpd[17205]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-07T15:59:08.933 INFO:teuthology.orchestra.run.smithi144.stderr: 7 Apr 15:59:08 ntpd[17205]: can't open /var/log/ntpstats/rawstats.20240407: Permission denied 2024-04-07T15:59:08.933 INFO:teuthology.orchestra.run.smithi144.stderr: 7 Apr 15:59:08 ntpd[17205]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-07T15:59:08.933 INFO:teuthology.orchestra.run.smithi144.stderr: 7 Apr 15:59:08 ntpd[17205]: can't open /var/log/ntpstats/peerstats.20240407: Permission denied 2024-04-07T15:59:09.905 INFO:teuthology.orchestra.run.smithi100.stderr: 7 Apr 15:59:09 ntpd[17192]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-07T15:59:09.905 INFO:teuthology.orchestra.run.smithi100.stderr: 7 Apr 15:59:09 ntpd[17192]: can't open /var/log/ntpstats/rawstats.20240407: Permission denied 2024-04-07T15:59:09.905 INFO:teuthology.orchestra.run.smithi100.stderr: 7 Apr 15:59:09 ntpd[17192]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-07T15:59:09.905 INFO:teuthology.orchestra.run.smithi100.stderr: 7 Apr 15:59:09 ntpd[17192]: can't open /var/log/ntpstats/rawstats.20240407: Permission denied 2024-04-07T15:59:09.905 INFO:teuthology.orchestra.run.smithi100.stderr: 7 Apr 15:59:09 ntpd[17192]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-07T15:59:09.905 INFO:teuthology.orchestra.run.smithi100.stderr: 7 Apr 15:59:09 ntpd[17192]: can't open /var/log/ntpstats/peerstats.20240407: Permission denied 2024-04-07T15:59:10.905 INFO:teuthology.orchestra.run.smithi100.stderr: 7 Apr 15:59:10 ntpd[17192]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-07T15:59:10.905 INFO:teuthology.orchestra.run.smithi100.stderr: 7 Apr 15:59:10 ntpd[17192]: can't open /var/log/ntpstats/rawstats.20240407: Permission denied 2024-04-07T15:59:10.905 INFO:teuthology.orchestra.run.smithi100.stderr: 7 Apr 15:59:10 ntpd[17192]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-07T15:59:10.905 INFO:teuthology.orchestra.run.smithi100.stderr: 7 Apr 15:59:10 ntpd[17192]: can't open /var/log/ntpstats/rawstats.20240407: Permission denied 2024-04-07T15:59:10.905 INFO:teuthology.orchestra.run.smithi100.stderr: 7 Apr 15:59:10 ntpd[17192]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-07T15:59:10.905 INFO:teuthology.orchestra.run.smithi100.stderr: 7 Apr 15:59:10 ntpd[17192]: can't open /var/log/ntpstats/rawstats.20240407: Permission denied 2024-04-07T15:59:10.905 INFO:teuthology.orchestra.run.smithi100.stderr: 7 Apr 15:59:10 ntpd[17192]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-07T15:59:10.905 INFO:teuthology.orchestra.run.smithi100.stderr: 7 Apr 15:59:10 ntpd[17192]: can't open /var/log/ntpstats/peerstats.20240407: Permission denied 2024-04-07T15:59:10.933 INFO:teuthology.orchestra.run.smithi144.stderr: 7 Apr 15:59:10 ntpd[17205]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-07T15:59:10.933 INFO:teuthology.orchestra.run.smithi144.stderr: 7 Apr 15:59:10 ntpd[17205]: can't open /var/log/ntpstats/rawstats.20240407: Permission denied 2024-04-07T15:59:10.933 INFO:teuthology.orchestra.run.smithi144.stderr: 7 Apr 15:59:10 ntpd[17205]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-07T15:59:10.933 INFO:teuthology.orchestra.run.smithi144.stderr: 7 Apr 15:59:10 ntpd[17205]: can't open /var/log/ntpstats/rawstats.20240407: Permission denied 2024-04-07T15:59:10.933 INFO:teuthology.orchestra.run.smithi144.stderr: 7 Apr 15:59:10 ntpd[17205]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-07T15:59:10.934 INFO:teuthology.orchestra.run.smithi144.stderr: 7 Apr 15:59:10 ntpd[17205]: can't open /var/log/ntpstats/peerstats.20240407: Permission denied 2024-04-07T15:59:11.905 INFO:teuthology.orchestra.run.smithi100.stderr: 7 Apr 15:59:11 ntpd[17192]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-07T15:59:11.905 INFO:teuthology.orchestra.run.smithi100.stderr: 7 Apr 15:59:11 ntpd[17192]: can't open /var/log/ntpstats/rawstats.20240407: Permission denied 2024-04-07T15:59:11.905 INFO:teuthology.orchestra.run.smithi100.stderr: 7 Apr 15:59:11 ntpd[17192]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-07T15:59:11.905 INFO:teuthology.orchestra.run.smithi100.stderr: 7 Apr 15:59:11 ntpd[17192]: can't open /var/log/ntpstats/rawstats.20240407: Permission denied 2024-04-07T15:59:11.905 INFO:teuthology.orchestra.run.smithi100.stderr: 7 Apr 15:59:11 ntpd[17192]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-07T15:59:11.905 INFO:teuthology.orchestra.run.smithi100.stderr: 7 Apr 15:59:11 ntpd[17192]: can't open /var/log/ntpstats/peerstats.20240407: Permission denied 2024-04-07T15:59:12.905 INFO:teuthology.orchestra.run.smithi100.stderr: 7 Apr 15:59:12 ntpd[17192]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-07T15:59:12.905 INFO:teuthology.orchestra.run.smithi100.stderr: 7 Apr 15:59:12 ntpd[17192]: can't open /var/log/ntpstats/rawstats.20240407: Permission denied 2024-04-07T15:59:12.905 INFO:teuthology.orchestra.run.smithi100.stderr: 7 Apr 15:59:12 ntpd[17192]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-07T15:59:12.905 INFO:teuthology.orchestra.run.smithi100.stderr: 7 Apr 15:59:12 ntpd[17192]: can't open /var/log/ntpstats/rawstats.20240407: Permission denied 2024-04-07T15:59:12.905 INFO:teuthology.orchestra.run.smithi100.stderr: 7 Apr 15:59:12 ntpd[17192]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-07T15:59:12.905 INFO:teuthology.orchestra.run.smithi100.stderr: 7 Apr 15:59:12 ntpd[17192]: can't open /var/log/ntpstats/rawstats.20240407: Permission denied 2024-04-07T15:59:12.906 INFO:teuthology.orchestra.run.smithi100.stderr: 7 Apr 15:59:12 ntpd[17192]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-07T15:59:12.906 INFO:teuthology.orchestra.run.smithi100.stderr: 7 Apr 15:59:12 ntpd[17192]: can't open /var/log/ntpstats/peerstats.20240407: Permission denied 2024-04-07T15:59:12.906 INFO:teuthology.orchestra.run.smithi100.stderr: 7 Apr 15:59:12 ntpd[17192]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-07T15:59:12.906 INFO:teuthology.orchestra.run.smithi100.stderr: 7 Apr 15:59:12 ntpd[17192]: can't open /var/log/ntpstats/rawstats.20240407: Permission denied 2024-04-07T15:59:12.906 INFO:teuthology.orchestra.run.smithi100.stderr: 7 Apr 15:59:12 ntpd[17192]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-07T15:59:12.906 INFO:teuthology.orchestra.run.smithi100.stderr: 7 Apr 15:59:12 ntpd[17192]: can't open /var/log/ntpstats/peerstats.20240407: Permission denied 2024-04-07T15:59:12.934 INFO:teuthology.orchestra.run.smithi144.stderr: 7 Apr 15:59:12 ntpd[17205]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-07T15:59:12.934 INFO:teuthology.orchestra.run.smithi144.stderr: 7 Apr 15:59:12 ntpd[17205]: can't open /var/log/ntpstats/rawstats.20240407: Permission denied 2024-04-07T15:59:12.934 INFO:teuthology.orchestra.run.smithi144.stderr: 7 Apr 15:59:12 ntpd[17205]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-07T15:59:12.934 INFO:teuthology.orchestra.run.smithi144.stderr: 7 Apr 15:59:12 ntpd[17205]: can't open /var/log/ntpstats/rawstats.20240407: Permission denied 2024-04-07T15:59:12.934 INFO:teuthology.orchestra.run.smithi144.stderr: 7 Apr 15:59:12 ntpd[17205]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-07T15:59:12.934 INFO:teuthology.orchestra.run.smithi144.stderr: 7 Apr 15:59:12 ntpd[17205]: can't open /var/log/ntpstats/rawstats.20240407: Permission denied 2024-04-07T15:59:12.935 INFO:teuthology.orchestra.run.smithi144.stderr: 7 Apr 15:59:12 ntpd[17205]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-07T15:59:12.935 INFO:teuthology.orchestra.run.smithi144.stderr: 7 Apr 15:59:12 ntpd[17205]: can't open /var/log/ntpstats/rawstats.20240407: Permission denied 2024-04-07T15:59:12.935 INFO:teuthology.orchestra.run.smithi144.stderr: 7 Apr 15:59:12 ntpd[17205]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-07T15:59:12.935 INFO:teuthology.orchestra.run.smithi144.stderr: 7 Apr 15:59:12 ntpd[17205]: can't open /var/log/ntpstats/peerstats.20240407: Permission denied 2024-04-07T15:59:12.935 INFO:teuthology.orchestra.run.smithi144.stderr: 7 Apr 15:59:12 ntpd[17205]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-07T15:59:12.935 INFO:teuthology.orchestra.run.smithi144.stderr: 7 Apr 15:59:12 ntpd[17205]: can't open /var/log/ntpstats/rawstats.20240407: Permission denied 2024-04-07T15:59:12.935 INFO:teuthology.orchestra.run.smithi144.stderr: 7 Apr 15:59:12 ntpd[17205]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-07T15:59:12.935 INFO:teuthology.orchestra.run.smithi144.stderr: 7 Apr 15:59:12 ntpd[17205]: can't open /var/log/ntpstats/peerstats.20240407: Permission denied 2024-04-07T15:59:12.935 INFO:teuthology.orchestra.run.smithi144.stderr: 7 Apr 15:59:12 ntpd[17205]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-07T15:59:12.935 INFO:teuthology.orchestra.run.smithi144.stderr: 7 Apr 15:59:12 ntpd[17205]: can't open /var/log/ntpstats/rawstats.20240407: Permission denied 2024-04-07T15:59:12.935 INFO:teuthology.orchestra.run.smithi144.stderr: 7 Apr 15:59:12 ntpd[17205]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-07T15:59:12.935 INFO:teuthology.orchestra.run.smithi144.stderr: 7 Apr 15:59:12 ntpd[17205]: can't open /var/log/ntpstats/peerstats.20240407: Permission denied 2024-04-07T15:59:13.905 INFO:teuthology.orchestra.run.smithi100.stderr: 7 Apr 15:59:13 ntpd[17192]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-07T15:59:13.905 INFO:teuthology.orchestra.run.smithi100.stderr: 7 Apr 15:59:13 ntpd[17192]: can't open /var/log/ntpstats/rawstats.20240407: Permission denied 2024-04-07T15:59:13.905 INFO:teuthology.orchestra.run.smithi100.stderr: 7 Apr 15:59:13 ntpd[17192]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-07T15:59:13.905 INFO:teuthology.orchestra.run.smithi100.stderr: 7 Apr 15:59:13 ntpd[17192]: can't open /var/log/ntpstats/rawstats.20240407: Permission denied 2024-04-07T15:59:13.905 INFO:teuthology.orchestra.run.smithi100.stderr: 7 Apr 15:59:13 ntpd[17192]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-07T15:59:13.905 INFO:teuthology.orchestra.run.smithi100.stderr: 7 Apr 15:59:13 ntpd[17192]: can't open /var/log/ntpstats/peerstats.20240407: Permission denied 2024-04-07T15:59:14.905 INFO:teuthology.orchestra.run.smithi100.stderr: 7 Apr 15:59:14 ntpd[17192]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-07T15:59:14.905 INFO:teuthology.orchestra.run.smithi100.stderr: 7 Apr 15:59:14 ntpd[17192]: can't open /var/log/ntpstats/rawstats.20240407: Permission denied 2024-04-07T15:59:14.905 INFO:teuthology.orchestra.run.smithi100.stderr: 7 Apr 15:59:14 ntpd[17192]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-07T15:59:14.905 INFO:teuthology.orchestra.run.smithi100.stderr: 7 Apr 15:59:14 ntpd[17192]: can't open /var/log/ntpstats/rawstats.20240407: Permission denied 2024-04-07T15:59:14.905 INFO:teuthology.orchestra.run.smithi100.stderr: 7 Apr 15:59:14 ntpd[17192]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-07T15:59:14.905 INFO:teuthology.orchestra.run.smithi100.stderr: 7 Apr 15:59:14 ntpd[17192]: can't open /var/log/ntpstats/rawstats.20240407: Permission denied 2024-04-07T15:59:14.905 INFO:teuthology.orchestra.run.smithi100.stderr: 7 Apr 15:59:14 ntpd[17192]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-07T15:59:14.905 INFO:teuthology.orchestra.run.smithi100.stderr: 7 Apr 15:59:14 ntpd[17192]: can't open /var/log/ntpstats/peerstats.20240407: Permission denied 2024-04-07T15:59:14.906 INFO:teuthology.orchestra.run.smithi100.stdout: 7 Apr 15:59:14 ntpd[17192]: ntpd: time slew -0.002137 s 2024-04-07T15:59:14.906 INFO:teuthology.orchestra.run.smithi100.stdout:ntpd: time slew -0.002137s 2024-04-07T15:59:14.906 INFO:teuthology.orchestra.run.smithi100.stderr: 7 Apr 15:59:14 ntpd[17192]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-04-07T15:59:14.906 INFO:teuthology.orchestra.run.smithi100.stderr: 7 Apr 15:59:14 ntpd[17192]: can't open /var/log/ntpstats/loopstats.20240407: Permission denied 2024-04-07T15:59:14.934 INFO:teuthology.orchestra.run.smithi144.stderr: 7 Apr 15:59:14 ntpd[17205]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-07T15:59:14.934 INFO:teuthology.orchestra.run.smithi144.stderr: 7 Apr 15:59:14 ntpd[17205]: can't open /var/log/ntpstats/rawstats.20240407: Permission denied 2024-04-07T15:59:14.935 INFO:teuthology.orchestra.run.smithi144.stderr: 7 Apr 15:59:14 ntpd[17205]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-07T15:59:14.935 INFO:teuthology.orchestra.run.smithi144.stderr: 7 Apr 15:59:14 ntpd[17205]: can't open /var/log/ntpstats/rawstats.20240407: Permission denied 2024-04-07T15:59:14.935 INFO:teuthology.orchestra.run.smithi144.stderr: 7 Apr 15:59:14 ntpd[17205]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-07T15:59:14.935 INFO:teuthology.orchestra.run.smithi144.stderr: 7 Apr 15:59:14 ntpd[17205]: can't open /var/log/ntpstats/rawstats.20240407: Permission denied 2024-04-07T15:59:14.935 INFO:teuthology.orchestra.run.smithi144.stderr: 7 Apr 15:59:14 ntpd[17205]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-07T15:59:14.935 INFO:teuthology.orchestra.run.smithi144.stderr: 7 Apr 15:59:14 ntpd[17205]: can't open /var/log/ntpstats/rawstats.20240407: Permission denied 2024-04-07T15:59:14.935 INFO:teuthology.orchestra.run.smithi144.stderr: 7 Apr 15:59:14 ntpd[17205]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-07T15:59:14.935 INFO:teuthology.orchestra.run.smithi144.stderr: 7 Apr 15:59:14 ntpd[17205]: can't open /var/log/ntpstats/peerstats.20240407: Permission denied 2024-04-07T15:59:14.935 INFO:teuthology.orchestra.run.smithi144.stderr: 7 Apr 15:59:14 ntpd[17205]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-07T15:59:14.935 INFO:teuthology.orchestra.run.smithi144.stderr: 7 Apr 15:59:14 ntpd[17205]: can't open /var/log/ntpstats/rawstats.20240407: Permission denied 2024-04-07T15:59:14.935 INFO:teuthology.orchestra.run.smithi144.stderr: 7 Apr 15:59:14 ntpd[17205]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-07T15:59:14.935 INFO:teuthology.orchestra.run.smithi144.stderr: 7 Apr 15:59:14 ntpd[17205]: can't open /var/log/ntpstats/peerstats.20240407: Permission denied 2024-04-07T15:59:14.935 INFO:teuthology.orchestra.run.smithi144.stderr: 7 Apr 15:59:14 ntpd[17205]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-07T15:59:14.935 INFO:teuthology.orchestra.run.smithi144.stderr: 7 Apr 15:59:14 ntpd[17205]: can't open /var/log/ntpstats/rawstats.20240407: Permission denied 2024-04-07T15:59:14.935 INFO:teuthology.orchestra.run.smithi144.stderr: 7 Apr 15:59:14 ntpd[17205]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-07T15:59:14.936 INFO:teuthology.orchestra.run.smithi144.stderr: 7 Apr 15:59:14 ntpd[17205]: can't open /var/log/ntpstats/peerstats.20240407: Permission denied 2024-04-07T15:59:14.936 INFO:teuthology.orchestra.run.smithi144.stdout: 7 Apr 15:59:14 ntpd[17205]: ntpd: time slew -0.000515 s 2024-04-07T15:59:14.936 INFO:teuthology.orchestra.run.smithi144.stdout:ntpd: time slew -0.000515s 2024-04-07T15:59:14.936 INFO:teuthology.orchestra.run.smithi144.stderr: 7 Apr 15:59:14 ntpd[17205]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-04-07T15:59:14.936 INFO:teuthology.orchestra.run.smithi144.stderr: 7 Apr 15:59:14 ntpd[17205]: can't open /var/log/ntpstats/loopstats.20240407: Permission denied 2024-04-07T15:59:14.968 INFO:teuthology.orchestra.run.smithi100.stdout: remote refid st t when poll reach delay offset jitter 2024-04-07T15:59:14.968 INFO:teuthology.orchestra.run.smithi100.stdout:============================================================================== 2024-04-07T15:59:14.968 INFO:teuthology.orchestra.run.smithi100.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-07T15:59:14.968 INFO:teuthology.orchestra.run.smithi100.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-07T15:59:14.968 INFO:teuthology.orchestra.run.smithi100.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-07T15:59:14.968 INFO:teuthology.orchestra.run.smithi100.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-07T15:59:15.040 INFO:teuthology.orchestra.run.smithi144.stdout: remote refid st t when poll reach delay offset jitter 2024-04-07T15:59:15.040 INFO:teuthology.orchestra.run.smithi144.stdout:============================================================================== 2024-04-07T15:59:15.040 INFO:teuthology.orchestra.run.smithi144.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-07T15:59:15.041 INFO:teuthology.orchestra.run.smithi144.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-07T15:59:15.041 INFO:teuthology.orchestra.run.smithi144.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-07T15:59:15.041 INFO:teuthology.orchestra.run.smithi144.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-07T15:59:15.042 INFO:teuthology.run_tasks:Running task install... 2024-04-07T15:59:15.052 DEBUG:teuthology.task.install:project ceph 2024-04-07T15:59:15.052 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'extra_system_packages': ['python3-pytest'], 'flavor': 'default', 'sha1': 'adce92d056533f22dc1ee37e3141d098b046ec1b'}} 2024-04-07T15:59:15.052 DEBUG:teuthology.task.install:config {'branch': 'reef', 'exclude_packages': ['ceph-volume'], 'extra_system_packages': ['python3-pytest'], 'flavor': 'default', 'sha1': 'adce92d056533f22dc1ee37e3141d098b046ec1b'} 2024-04-07T15:59:15.052 INFO:teuthology.task.install:Using flavor: default 2024-04-07T15:59:15.058 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-07T15:59:15.058 INFO:teuthology.task.install:extra packages: [] 2024-04-07T15:59:15.058 DEBUG:teuthology.orchestra.run.smithi100:> sudo apt-key list | grep Ceph 2024-04-07T15:59:15.059 DEBUG:teuthology.orchestra.run.smithi144:> sudo apt-key list | grep Ceph 2024-04-07T15:59:15.116 INFO:teuthology.orchestra.run.smithi100.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-04-07T15:59:15.140 INFO:teuthology.orchestra.run.smithi144.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-04-07T15:59:15.145 INFO:teuthology.orchestra.run.smithi100.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-04-07T15:59:15.145 INFO:teuthology.orchestra.run.smithi100.stdout:uid [ unknown] Ceph.com (release key) 2024-04-07T15:59:15.146 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2024-04-07T15:59:15.146 INFO:teuthology.task.install.deb:Installing system (non-project) packages: python3-pytest on remote deb x86_64 2024-04-07T15:59:15.146 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-07T15:59:15.146 INFO:teuthology.packaging:ref: None 2024-04-07T15:59:15.146 INFO:teuthology.packaging:tag: None 2024-04-07T15:59:15.146 INFO:teuthology.packaging:branch: reef 2024-04-07T15:59:15.146 INFO:teuthology.packaging:sha1: adce92d056533f22dc1ee37e3141d098b046ec1b 2024-04-07T15:59:15.147 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=reef 2024-04-07T15:59:15.165 INFO:teuthology.orchestra.run.smithi144.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-04-07T15:59:15.165 INFO:teuthology.orchestra.run.smithi144.stdout:uid [ unknown] Ceph.com (release key) 2024-04-07T15:59:15.166 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2024-04-07T15:59:15.166 INFO:teuthology.task.install.deb:Installing system (non-project) packages: python3-pytest on remote deb x86_64 2024-04-07T15:59:15.166 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-07T15:59:15.166 INFO:teuthology.packaging:ref: None 2024-04-07T15:59:15.167 INFO:teuthology.packaging:tag: None 2024-04-07T15:59:15.167 INFO:teuthology.packaging:branch: reef 2024-04-07T15:59:15.167 INFO:teuthology.packaging:sha1: adce92d056533f22dc1ee37e3141d098b046ec1b 2024-04-07T15:59:15.167 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=reef 2024-04-07T15:59:15.430 INFO:teuthology.task.install.deb:Pulling from https://2.chacra.ceph.com/r/ceph/reef/267c8ff5b1ff5815581b3b2381d19360e081b37b/ubuntu/jammy/flavors/default/ 2024-04-07T15:59:15.431 INFO:teuthology.task.install.deb:Package version is 18.2.2-1171-g267c8ff5-1jammy 2024-04-07T15:59:15.541 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-04-07T15:59:15.541 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-04-07T15:59:15.554 DEBUG:teuthology.orchestra.run.smithi144:> sudo apt-get update 2024-04-07T15:59:15.588 INFO:teuthology.task.install.deb:Pulling from https://2.chacra.ceph.com/r/ceph/reef/267c8ff5b1ff5815581b3b2381d19360e081b37b/ubuntu/jammy/flavors/default/ 2024-04-07T15:59:15.589 INFO:teuthology.task.install.deb:Package version is 18.2.2-1171-g267c8ff5-1jammy 2024-04-07T15:59:15.759 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-07T15:59:15.759 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-04-07T15:59:15.773 DEBUG:teuthology.orchestra.run.smithi100:> sudo apt-get update 2024-04-07T15:59:15.792 INFO:teuthology.orchestra.run.smithi144.stdout:Ign:1 https://2.chacra.ceph.com/r/ceph/reef/267c8ff5b1ff5815581b3b2381d19360e081b37b/ubuntu/jammy/flavors/default jammy InRelease 2024-04-07T15:59:15.831 INFO:teuthology.orchestra.run.smithi144.stdout:Get:2 https://2.chacra.ceph.com/r/ceph/reef/267c8ff5b1ff5815581b3b2381d19360e081b37b/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2024-04-07T15:59:15.856 INFO:teuthology.orchestra.run.smithi144.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-07T15:59:15.857 INFO:teuthology.orchestra.run.smithi144.stdout:Ign:4 https://2.chacra.ceph.com/r/ceph/reef/267c8ff5b1ff5815581b3b2381d19360e081b37b/ubuntu/jammy/flavors/default jammy Release.gpg 2024-04-07T15:59:15.868 INFO:teuthology.orchestra.run.smithi144.stdout:Hit:5 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-04-07T15:59:15.880 INFO:teuthology.orchestra.run.smithi144.stdout:Get:6 https://2.chacra.ceph.com/r/ceph/reef/267c8ff5b1ff5815581b3b2381d19360e081b37b/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.4 kB] 2024-04-07T15:59:15.928 INFO:teuthology.orchestra.run.smithi144.stdout:Hit:7 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-04-07T15:59:15.993 INFO:teuthology.orchestra.run.smithi144.stdout:Get:8 https://2.chacra.ceph.com/r/ceph/reef/267c8ff5b1ff5815581b3b2381d19360e081b37b/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,409 B] 2024-04-07T15:59:16.012 INFO:teuthology.orchestra.run.smithi144.stdout:Hit:9 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-04-07T15:59:16.055 INFO:teuthology.orchestra.run.smithi100.stdout:Ign:1 https://2.chacra.ceph.com/r/ceph/reef/267c8ff5b1ff5815581b3b2381d19360e081b37b/ubuntu/jammy/flavors/default jammy InRelease 2024-04-07T15:59:16.071 INFO:teuthology.orchestra.run.smithi100.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-07T15:59:16.071 INFO:teuthology.orchestra.run.smithi100.stdout:Hit:3 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-04-07T15:59:16.092 INFO:teuthology.orchestra.run.smithi100.stdout:Get:4 https://2.chacra.ceph.com/r/ceph/reef/267c8ff5b1ff5815581b3b2381d19360e081b37b/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2024-04-07T15:59:16.130 INFO:teuthology.orchestra.run.smithi100.stdout:Ign:5 https://2.chacra.ceph.com/r/ceph/reef/267c8ff5b1ff5815581b3b2381d19360e081b37b/ubuntu/jammy/flavors/default jammy Release.gpg 2024-04-07T15:59:16.144 INFO:teuthology.orchestra.run.smithi100.stdout:Hit:6 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-04-07T15:59:16.167 INFO:teuthology.orchestra.run.smithi100.stdout:Get:7 https://2.chacra.ceph.com/r/ceph/reef/267c8ff5b1ff5815581b3b2381d19360e081b37b/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.4 kB] 2024-04-07T15:59:16.209 INFO:teuthology.orchestra.run.smithi100.stdout:Get:8 https://2.chacra.ceph.com/r/ceph/reef/267c8ff5b1ff5815581b3b2381d19360e081b37b/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,409 B] 2024-04-07T15:59:16.217 INFO:teuthology.orchestra.run.smithi144.stdout:Fetched 28.5 kB in 0s (59.5 kB/s) 2024-04-07T15:59:16.226 INFO:teuthology.orchestra.run.smithi100.stdout:Hit:9 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-04-07T15:59:16.429 INFO:teuthology.orchestra.run.smithi100.stdout:Fetched 28.5 kB in 0s (60.1 kB/s) 2024-04-07T15:59:17.140 INFO:teuthology.orchestra.run.smithi144.stdout:Reading package lists... 2024-04-07T15:59:17.157 DEBUG:teuthology.orchestra.run.smithi144:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=18.2.2-1171-g267c8ff5-1jammy cephadm=18.2.2-1171-g267c8ff5-1jammy ceph-mds=18.2.2-1171-g267c8ff5-1jammy ceph-mgr=18.2.2-1171-g267c8ff5-1jammy ceph-common=18.2.2-1171-g267c8ff5-1jammy ceph-fuse=18.2.2-1171-g267c8ff5-1jammy ceph-test=18.2.2-1171-g267c8ff5-1jammy radosgw=18.2.2-1171-g267c8ff5-1jammy python3-rados=18.2.2-1171-g267c8ff5-1jammy python3-rgw=18.2.2-1171-g267c8ff5-1jammy python3-cephfs=18.2.2-1171-g267c8ff5-1jammy python3-rbd=18.2.2-1171-g267c8ff5-1jammy libcephfs2=18.2.2-1171-g267c8ff5-1jammy libcephfs-dev=18.2.2-1171-g267c8ff5-1jammy librados2=18.2.2-1171-g267c8ff5-1jammy librbd1=18.2.2-1171-g267c8ff5-1jammy rbd-fuse=18.2.2-1171-g267c8ff5-1jammy 2024-04-07T15:59:17.209 INFO:teuthology.orchestra.run.smithi144.stdout:Reading package lists... 2024-04-07T15:59:17.384 INFO:teuthology.orchestra.run.smithi144.stdout:Building dependency tree... 2024-04-07T15:59:17.384 INFO:teuthology.orchestra.run.smithi144.stdout:Reading state information... 2024-04-07T15:59:17.398 INFO:teuthology.orchestra.run.smithi100.stdout:Reading package lists... 2024-04-07T15:59:17.415 DEBUG:teuthology.orchestra.run.smithi100:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=18.2.2-1171-g267c8ff5-1jammy cephadm=18.2.2-1171-g267c8ff5-1jammy ceph-mds=18.2.2-1171-g267c8ff5-1jammy ceph-mgr=18.2.2-1171-g267c8ff5-1jammy ceph-common=18.2.2-1171-g267c8ff5-1jammy ceph-fuse=18.2.2-1171-g267c8ff5-1jammy ceph-test=18.2.2-1171-g267c8ff5-1jammy radosgw=18.2.2-1171-g267c8ff5-1jammy python3-rados=18.2.2-1171-g267c8ff5-1jammy python3-rgw=18.2.2-1171-g267c8ff5-1jammy python3-cephfs=18.2.2-1171-g267c8ff5-1jammy python3-rbd=18.2.2-1171-g267c8ff5-1jammy libcephfs2=18.2.2-1171-g267c8ff5-1jammy libcephfs-dev=18.2.2-1171-g267c8ff5-1jammy librados2=18.2.2-1171-g267c8ff5-1jammy librbd1=18.2.2-1171-g267c8ff5-1jammy rbd-fuse=18.2.2-1171-g267c8ff5-1jammy 2024-04-07T15:59:17.473 INFO:teuthology.orchestra.run.smithi100.stdout:Reading package lists... 2024-04-07T15:59:17.552 INFO:teuthology.orchestra.run.smithi144.stdout:The following packages were automatically installed and are no longer required: 2024-04-07T15:59:17.552 INFO:teuthology.orchestra.run.smithi144.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-07T15:59:17.552 INFO:teuthology.orchestra.run.smithi144.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-07T15:59:17.552 INFO:teuthology.orchestra.run.smithi144.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-07T15:59:17.552 INFO:teuthology.orchestra.run.smithi144.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-07T15:59:17.552 INFO:teuthology.orchestra.run.smithi144.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-07T15:59:17.552 INFO:teuthology.orchestra.run.smithi144.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-07T15:59:17.553 INFO:teuthology.orchestra.run.smithi144.stdout:The following additional packages will be installed: 2024-04-07T15:59:17.553 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-04-07T15:59:17.553 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd ceph-volume jq 2024-04-07T15:59:17.553 INFO:teuthology.orchestra.run.smithi144.stdout: libdouble-conversion3 libfmt8 libjq1 liblttng-ust1 liboath0 libonig5 2024-04-07T15:59:17.553 INFO:teuthology.orchestra.run.smithi144.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-04-07T15:59:17.554 INFO:teuthology.orchestra.run.smithi144.stdout: librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.16.0 nvme-cli 2024-04-07T15:59:17.554 INFO:teuthology.orchestra.run.smithi144.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-04-07T15:59:17.554 INFO:teuthology.orchestra.run.smithi144.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-04-07T15:59:17.554 INFO:teuthology.orchestra.run.smithi144.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-04-07T15:59:17.554 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-04-07T15:59:17.555 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-04-07T15:59:17.555 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-04-07T15:59:17.555 INFO:teuthology.orchestra.run.smithi144.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-04-07T15:59:17.555 INFO:teuthology.orchestra.run.smithi144.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-04-07T15:59:17.555 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-04-07T15:59:17.555 INFO:teuthology.orchestra.run.smithi144.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-04-07T15:59:17.555 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pygments python3-pyinotify python3-pytest python3-repoze.lru 2024-04-07T15:59:17.555 INFO:teuthology.orchestra.run.smithi144.stdout: python3-requests python3-requests-oauthlib python3-routes python3-rsa 2024-04-07T15:59:17.555 INFO:teuthology.orchestra.run.smithi144.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-04-07T15:59:17.555 INFO:teuthology.orchestra.run.smithi144.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-04-07T15:59:17.555 INFO:teuthology.orchestra.run.smithi144.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-04-07T15:59:17.555 INFO:teuthology.orchestra.run.smithi144.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-04-07T15:59:17.555 INFO:teuthology.orchestra.run.smithi144.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n socat xmlstarlet 2024-04-07T15:59:17.556 INFO:teuthology.orchestra.run.smithi144.stdout:Suggested packages: 2024-04-07T15:59:17.556 INFO:teuthology.orchestra.run.smithi144.stdout: python3-influxdb python-jinja2-doc python3-beaker python-mako-doc 2024-04-07T15:59:17.556 INFO:teuthology.orchestra.run.smithi144.stdout: python-natsort-doc python-openssl-doc python3-openssl-dbg httpd-wsgi 2024-04-07T15:59:17.556 INFO:teuthology.orchestra.run.smithi144.stdout: libapache2-mod-python libapache2-mod-scgi libjs-mochikit python-pecan-doc 2024-04-07T15:59:17.556 INFO:teuthology.orchestra.run.smithi144.stdout: python-psutil-doc subversion python-pygments-doc python-pyinotify-doc 2024-04-07T15:59:17.556 INFO:teuthology.orchestra.run.smithi144.stdout: python3-socks python-requests-doc python3-dap python-sklearn-doc ipython3 2024-04-07T15:59:17.556 INFO:teuthology.orchestra.run.smithi144.stdout: python-waitress-doc python-webob-doc python-webtest-doc python-werkzeug-doc 2024-04-07T15:59:17.556 INFO:teuthology.orchestra.run.smithi144.stdout: python3-watchdog 2024-04-07T15:59:17.556 INFO:teuthology.orchestra.run.smithi144.stdout:Recommended packages: 2024-04-07T15:59:17.556 INFO:teuthology.orchestra.run.smithi144.stdout: btrfs-tools 2024-04-07T15:59:17.655 INFO:teuthology.orchestra.run.smithi144.stdout:The following NEW packages will be installed: 2024-04-07T15:59:17.655 INFO:teuthology.orchestra.run.smithi144.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-04-07T15:59:17.655 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-04-07T15:59:17.655 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm jq 2024-04-07T15:59:17.655 INFO:teuthology.orchestra.run.smithi144.stdout: libcephfs-dev libcephfs2 libdouble-conversion3 libfmt8 libjq1 liblttng-ust1 2024-04-07T15:59:17.655 INFO:teuthology.orchestra.run.smithi144.stdout: liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 2024-04-07T15:59:17.655 INFO:teuthology.orchestra.run.smithi144.stdout: librados2 libradosstriper1 librbd1 librdkafka1 librgw2 libsqlite3-mod-ceph 2024-04-07T15:59:17.656 INFO:teuthology.orchestra.run.smithi144.stdout: libthrift-0.16.0 nvme-cli python-asyncssh-doc python-babel-localedata 2024-04-07T15:59:17.656 INFO:teuthology.orchestra.run.smithi144.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-04-07T15:59:17.656 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2024-04-07T15:59:17.656 INFO:teuthology.orchestra.run.smithi144.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-google-auth 2024-04-07T15:59:17.656 INFO:teuthology.orchestra.run.smithi144.stdout: python3-idna python3-iniconfig python3-jaraco.classes 2024-04-07T15:59:17.657 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2024-04-07T15:59:17.657 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2024-04-07T15:59:17.657 INFO:teuthology.orchestra.run.smithi144.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2024-04-07T15:59:17.657 INFO:teuthology.orchestra.run.smithi144.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-04-07T15:59:17.657 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2024-04-07T15:59:17.657 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2024-04-07T15:59:17.657 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pytest python3-rados python3-rbd python3-repoze.lru python3-requests 2024-04-07T15:59:17.657 INFO:teuthology.orchestra.run.smithi144.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-04-07T15:59:17.657 INFO:teuthology.orchestra.run.smithi144.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-04-07T15:59:17.657 INFO:teuthology.orchestra.run.smithi144.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-04-07T15:59:17.657 INFO:teuthology.orchestra.run.smithi144.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-04-07T15:59:17.658 INFO:teuthology.orchestra.run.smithi144.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-04-07T15:59:17.658 INFO:teuthology.orchestra.run.smithi144.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2024-04-07T15:59:17.658 INFO:teuthology.orchestra.run.smithi144.stdout: socat xmlstarlet 2024-04-07T15:59:17.671 INFO:teuthology.orchestra.run.smithi100.stdout:Building dependency tree... 2024-04-07T15:59:17.672 INFO:teuthology.orchestra.run.smithi100.stdout:Reading state information... 2024-04-07T15:59:17.844 INFO:teuthology.orchestra.run.smithi144.stdout:0 upgraded, 109 newly installed, 0 to remove and 299 not upgraded. 2024-04-07T15:59:17.844 INFO:teuthology.orchestra.run.smithi144.stdout:Need to get 162 MB of archives. 2024-04-07T15:59:17.844 INFO:teuthology.orchestra.run.smithi144.stdout:After this operation, 743 MB of additional disk space will be used. 2024-04-07T15:59:17.844 INFO:teuthology.orchestra.run.smithi144.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libfmt8 amd64 8.1.1+ds1-2 [60.2 kB] 2024-04-07T15:59:17.864 INFO:teuthology.orchestra.run.smithi100.stdout:The following packages were automatically installed and are no longer required: 2024-04-07T15:59:17.864 INFO:teuthology.orchestra.run.smithi100.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-07T15:59:17.864 INFO:teuthology.orchestra.run.smithi100.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-07T15:59:17.864 INFO:teuthology.orchestra.run.smithi100.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-07T15:59:17.865 INFO:teuthology.orchestra.run.smithi100.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-07T15:59:17.865 INFO:teuthology.orchestra.run.smithi100.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-07T15:59:17.865 INFO:teuthology.orchestra.run.smithi100.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-07T15:59:17.866 INFO:teuthology.orchestra.run.smithi100.stdout:The following additional packages will be installed: 2024-04-07T15:59:17.866 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-04-07T15:59:17.866 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd ceph-volume jq 2024-04-07T15:59:17.867 INFO:teuthology.orchestra.run.smithi100.stdout: libdouble-conversion3 libfmt8 libjq1 liblttng-ust1 liboath0 libonig5 2024-04-07T15:59:17.867 INFO:teuthology.orchestra.run.smithi100.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-04-07T15:59:17.867 INFO:teuthology.orchestra.run.smithi100.stdout: librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.16.0 nvme-cli 2024-04-07T15:59:17.867 INFO:teuthology.orchestra.run.smithi100.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-04-07T15:59:17.868 INFO:teuthology.orchestra.run.smithi100.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-04-07T15:59:17.868 INFO:teuthology.orchestra.run.smithi100.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-04-07T15:59:17.868 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-04-07T15:59:17.868 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-04-07T15:59:17.868 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-04-07T15:59:17.868 INFO:teuthology.orchestra.run.smithi100.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-04-07T15:59:17.868 INFO:teuthology.orchestra.run.smithi100.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-04-07T15:59:17.868 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-04-07T15:59:17.868 INFO:teuthology.orchestra.run.smithi100.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-04-07T15:59:17.868 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pygments python3-pyinotify python3-pytest python3-repoze.lru 2024-04-07T15:59:17.868 INFO:teuthology.orchestra.run.smithi100.stdout: python3-requests python3-requests-oauthlib python3-routes python3-rsa 2024-04-07T15:59:17.868 INFO:teuthology.orchestra.run.smithi100.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-04-07T15:59:17.868 INFO:teuthology.orchestra.run.smithi100.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-04-07T15:59:17.868 INFO:teuthology.orchestra.run.smithi100.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-04-07T15:59:17.869 INFO:teuthology.orchestra.run.smithi100.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-04-07T15:59:17.869 INFO:teuthology.orchestra.run.smithi100.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n socat xmlstarlet 2024-04-07T15:59:17.869 INFO:teuthology.orchestra.run.smithi100.stdout:Suggested packages: 2024-04-07T15:59:17.869 INFO:teuthology.orchestra.run.smithi100.stdout: python3-influxdb python-jinja2-doc python3-beaker python-mako-doc 2024-04-07T15:59:17.869 INFO:teuthology.orchestra.run.smithi100.stdout: python-natsort-doc python-openssl-doc python3-openssl-dbg httpd-wsgi 2024-04-07T15:59:17.869 INFO:teuthology.orchestra.run.smithi100.stdout: libapache2-mod-python libapache2-mod-scgi libjs-mochikit python-pecan-doc 2024-04-07T15:59:17.869 INFO:teuthology.orchestra.run.smithi100.stdout: python-psutil-doc subversion python-pygments-doc python-pyinotify-doc 2024-04-07T15:59:17.869 INFO:teuthology.orchestra.run.smithi100.stdout: python3-socks python-requests-doc python3-dap python-sklearn-doc ipython3 2024-04-07T15:59:17.869 INFO:teuthology.orchestra.run.smithi100.stdout: python-waitress-doc python-webob-doc python-webtest-doc python-werkzeug-doc 2024-04-07T15:59:17.870 INFO:teuthology.orchestra.run.smithi100.stdout: python3-watchdog 2024-04-07T15:59:17.870 INFO:teuthology.orchestra.run.smithi100.stdout:Recommended packages: 2024-04-07T15:59:17.870 INFO:teuthology.orchestra.run.smithi100.stdout: btrfs-tools 2024-04-07T15:59:17.891 INFO:teuthology.orchestra.run.smithi144.stdout:Get:2 https://2.chacra.ceph.com/r/ceph/reef/267c8ff5b1ff5815581b3b2381d19360e081b37b/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 18.2.2-1171-g267c8ff5-1jammy [3,439 kB] 2024-04-07T15:59:17.967 INFO:teuthology.orchestra.run.smithi100.stdout:The following NEW packages will be installed: 2024-04-07T15:59:17.967 INFO:teuthology.orchestra.run.smithi100.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-04-07T15:59:17.967 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-04-07T15:59:17.967 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm jq 2024-04-07T15:59:17.967 INFO:teuthology.orchestra.run.smithi100.stdout: libcephfs-dev libcephfs2 libdouble-conversion3 libfmt8 libjq1 liblttng-ust1 2024-04-07T15:59:17.967 INFO:teuthology.orchestra.run.smithi100.stdout: liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 2024-04-07T15:59:17.968 INFO:teuthology.orchestra.run.smithi100.stdout: librados2 libradosstriper1 librbd1 librdkafka1 librgw2 libsqlite3-mod-ceph 2024-04-07T15:59:17.968 INFO:teuthology.orchestra.run.smithi100.stdout: libthrift-0.16.0 nvme-cli python-asyncssh-doc python-babel-localedata 2024-04-07T15:59:17.968 INFO:teuthology.orchestra.run.smithi100.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-04-07T15:59:17.969 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2024-04-07T15:59:17.969 INFO:teuthology.orchestra.run.smithi100.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-google-auth 2024-04-07T15:59:17.969 INFO:teuthology.orchestra.run.smithi100.stdout: python3-idna python3-iniconfig python3-jaraco.classes 2024-04-07T15:59:17.969 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2024-04-07T15:59:17.969 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2024-04-07T15:59:17.969 INFO:teuthology.orchestra.run.smithi100.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2024-04-07T15:59:17.969 INFO:teuthology.orchestra.run.smithi100.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-04-07T15:59:17.969 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2024-04-07T15:59:17.969 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2024-04-07T15:59:17.969 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pytest python3-rados python3-rbd python3-repoze.lru python3-requests 2024-04-07T15:59:17.969 INFO:teuthology.orchestra.run.smithi100.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-04-07T15:59:17.969 INFO:teuthology.orchestra.run.smithi100.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-04-07T15:59:17.969 INFO:teuthology.orchestra.run.smithi100.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-04-07T15:59:17.969 INFO:teuthology.orchestra.run.smithi100.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-04-07T15:59:17.969 INFO:teuthology.orchestra.run.smithi100.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-04-07T15:59:17.970 INFO:teuthology.orchestra.run.smithi100.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2024-04-07T15:59:17.970 INFO:teuthology.orchestra.run.smithi100.stdout: socat xmlstarlet 2024-04-07T15:59:18.037 INFO:teuthology.orchestra.run.smithi100.stdout:0 upgraded, 109 newly installed, 0 to remove and 299 not upgraded. 2024-04-07T15:59:18.037 INFO:teuthology.orchestra.run.smithi100.stdout:Need to get 162 MB of archives. 2024-04-07T15:59:18.037 INFO:teuthology.orchestra.run.smithi100.stdout:After this operation, 743 MB of additional disk space will be used. 2024-04-07T15:59:18.037 INFO:teuthology.orchestra.run.smithi100.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libfmt8 amd64 8.1.1+ds1-2 [60.2 kB] 2024-04-07T15:59:18.112 INFO:teuthology.orchestra.run.smithi100.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-04-07T15:59:18.120 INFO:teuthology.orchestra.run.smithi144.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-04-07T15:59:18.156 INFO:teuthology.orchestra.run.smithi100.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-04-07T15:59:18.160 INFO:teuthology.orchestra.run.smithi100.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-04-07T15:59:18.180 INFO:teuthology.orchestra.run.smithi100.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-04-07T15:59:18.196 INFO:teuthology.orchestra.run.smithi100.stdout:Get:6 https://2.chacra.ceph.com/r/ceph/reef/267c8ff5b1ff5815581b3b2381d19360e081b37b/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 18.2.2-1171-g267c8ff5-1jammy [3,439 kB] 2024-04-07T15:59:18.240 INFO:teuthology.orchestra.run.smithi100.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-04-07T15:59:18.245 INFO:teuthology.orchestra.run.smithi100.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-04-07T15:59:18.257 INFO:teuthology.orchestra.run.smithi100.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-04-07T15:59:18.262 INFO:teuthology.orchestra.run.smithi100.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-04-07T15:59:18.262 INFO:teuthology.orchestra.run.smithi100.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-04-07T15:59:18.263 INFO:teuthology.orchestra.run.smithi100.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-04-07T15:59:18.272 INFO:teuthology.orchestra.run.smithi100.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-04-07T15:59:18.272 INFO:teuthology.orchestra.run.smithi100.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-04-07T15:59:18.273 INFO:teuthology.orchestra.run.smithi100.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-04-07T15:59:18.274 INFO:teuthology.orchestra.run.smithi100.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-04-07T15:59:18.291 INFO:teuthology.orchestra.run.smithi100.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-04-07T15:59:18.292 INFO:teuthology.orchestra.run.smithi100.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-04-07T15:59:18.292 INFO:teuthology.orchestra.run.smithi100.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-04-07T15:59:18.292 INFO:teuthology.orchestra.run.smithi100.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-04-07T15:59:18.292 INFO:teuthology.orchestra.run.smithi100.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-04-07T15:59:18.294 INFO:teuthology.orchestra.run.smithi144.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-04-07T15:59:18.309 INFO:teuthology.orchestra.run.smithi144.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-04-07T15:59:18.311 INFO:teuthology.orchestra.run.smithi100.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-04-07T15:59:18.314 INFO:teuthology.orchestra.run.smithi100.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-04-07T15:59:18.315 INFO:teuthology.orchestra.run.smithi100.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-04-07T15:59:18.315 INFO:teuthology.orchestra.run.smithi100.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-04-07T15:59:18.315 INFO:teuthology.orchestra.run.smithi100.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-04-07T15:59:18.331 INFO:teuthology.orchestra.run.smithi100.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-04-07T15:59:18.331 INFO:teuthology.orchestra.run.smithi100.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-04-07T15:59:18.331 INFO:teuthology.orchestra.run.smithi100.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2024-04-07T15:59:18.333 INFO:teuthology.orchestra.run.smithi100.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-04-07T15:59:18.334 INFO:teuthology.orchestra.run.smithi100.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-04-07T15:59:18.338 INFO:teuthology.orchestra.run.smithi144.stdout:Get:6 https://2.chacra.ceph.com/r/ceph/reef/267c8ff5b1ff5815581b3b2381d19360e081b37b/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 18.2.2-1171-g267c8ff5-1jammy [3,105 kB] 2024-04-07T15:59:18.353 INFO:teuthology.orchestra.run.smithi100.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-04-07T15:59:18.359 INFO:teuthology.orchestra.run.smithi100.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-04-07T15:59:18.359 INFO:teuthology.orchestra.run.smithi100.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-04-07T15:59:18.360 INFO:teuthology.orchestra.run.smithi100.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-04-07T15:59:18.361 INFO:teuthology.orchestra.run.smithi100.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-04-07T15:59:18.374 INFO:teuthology.orchestra.run.smithi100.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-04-07T15:59:18.376 INFO:teuthology.orchestra.run.smithi100.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-idna all 3.3-1 [49.3 kB] 2024-04-07T15:59:18.377 INFO:teuthology.orchestra.run.smithi100.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-04-07T15:59:18.378 INFO:teuthology.orchestra.run.smithi100.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-04-07T15:59:18.379 INFO:teuthology.orchestra.run.smithi100.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.1 [181 kB] 2024-04-07T15:59:18.380 INFO:teuthology.orchestra.run.smithi144.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-04-07T15:59:18.395 INFO:teuthology.orchestra.run.smithi100.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-04-07T15:59:18.396 INFO:teuthology.orchestra.run.smithi100.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-04-07T15:59:18.398 INFO:teuthology.orchestra.run.smithi100.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-04-07T15:59:18.474 INFO:teuthology.orchestra.run.smithi100.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-04-07T15:59:18.476 INFO:teuthology.orchestra.run.smithi100.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.1 [108 kB] 2024-04-07T15:59:18.477 INFO:teuthology.orchestra.run.smithi100.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-04-07T15:59:18.478 INFO:teuthology.orchestra.run.smithi100.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-04-07T15:59:18.478 INFO:teuthology.orchestra.run.smithi100.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-04-07T15:59:18.479 INFO:teuthology.orchestra.run.smithi100.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-04-07T15:59:18.480 INFO:teuthology.orchestra.run.smithi100.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-04-07T15:59:18.483 INFO:teuthology.orchestra.run.smithi100.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-04-07T15:59:18.486 INFO:teuthology.orchestra.run.smithi100.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-04-07T15:59:18.510 INFO:teuthology.orchestra.run.smithi100.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-04-07T15:59:18.512 INFO:teuthology.orchestra.run.smithi100.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-04-07T15:59:18.540 INFO:teuthology.orchestra.run.smithi100.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-04-07T15:59:18.543 INFO:teuthology.orchestra.run.smithi100.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-04-07T15:59:18.544 INFO:teuthology.orchestra.run.smithi100.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-04-07T15:59:18.576 INFO:teuthology.orchestra.run.smithi144.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-04-07T15:59:18.578 INFO:teuthology.orchestra.run.smithi100.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-04-07T15:59:18.578 INFO:teuthology.orchestra.run.smithi100.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-04-07T15:59:18.578 INFO:teuthology.orchestra.run.smithi100.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-04-07T15:59:18.579 INFO:teuthology.orchestra.run.smithi100.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-04-07T15:59:18.580 INFO:teuthology.orchestra.run.smithi100.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-04-07T15:59:18.581 INFO:teuthology.orchestra.run.smithi100.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-04-07T15:59:18.586 INFO:teuthology.orchestra.run.smithi144.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-04-07T15:59:18.602 INFO:teuthology.orchestra.run.smithi100.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-04-07T15:59:18.603 INFO:teuthology.orchestra.run.smithi100.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-04-07T15:59:18.608 INFO:teuthology.orchestra.run.smithi100.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-04-07T15:59:18.614 INFO:teuthology.orchestra.run.smithi144.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-04-07T15:59:18.614 INFO:teuthology.orchestra.run.smithi144.stdout:Get:11 https://2.chacra.ceph.com/r/ceph/reef/267c8ff5b1ff5815581b3b2381d19360e081b37b/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 18.2.2-1171-g267c8ff5-1jammy [679 kB] 2024-04-07T15:59:18.626 INFO:teuthology.orchestra.run.smithi144.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-04-07T15:59:18.628 INFO:teuthology.orchestra.run.smithi144.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-04-07T15:59:18.629 INFO:teuthology.orchestra.run.smithi144.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-04-07T15:59:18.631 INFO:teuthology.orchestra.run.smithi100.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-04-07T15:59:18.633 INFO:teuthology.orchestra.run.smithi100.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-04-07T15:59:18.634 INFO:teuthology.orchestra.run.smithi100.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-04-07T15:59:18.639 INFO:teuthology.orchestra.run.smithi100.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-04-07T15:59:18.642 INFO:teuthology.orchestra.run.smithi100.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2024-04-07T15:59:18.648 INFO:teuthology.orchestra.run.smithi100.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-04-07T15:59:18.651 INFO:teuthology.orchestra.run.smithi100.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-04-07T15:59:18.665 INFO:teuthology.orchestra.run.smithi144.stdout:Get:15 https://2.chacra.ceph.com/r/ceph/reef/267c8ff5b1ff5815581b3b2381d19360e081b37b/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 18.2.2-1171-g267c8ff5-1jammy [356 kB] 2024-04-07T15:59:18.666 INFO:teuthology.orchestra.run.smithi100.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-04-07T15:59:18.669 INFO:teuthology.orchestra.run.smithi100.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-04-07T15:59:18.679 INFO:teuthology.orchestra.run.smithi100.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-04-07T15:59:18.687 INFO:teuthology.orchestra.run.smithi144.stdout:Get:16 https://2.chacra.ceph.com/r/ceph/reef/267c8ff5b1ff5815581b3b2381d19360e081b37b/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 18.2.2-1171-g267c8ff5-1jammy [32.8 kB] 2024-04-07T15:59:18.689 INFO:teuthology.orchestra.run.smithi144.stdout:Get:17 https://2.chacra.ceph.com/r/ceph/reef/267c8ff5b1ff5815581b3b2381d19360e081b37b/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 18.2.2-1171-g267c8ff5-1jammy [180 kB] 2024-04-07T15:59:18.699 INFO:teuthology.orchestra.run.smithi100.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-04-07T15:59:18.700 INFO:teuthology.orchestra.run.smithi100.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-04-07T15:59:18.705 INFO:teuthology.orchestra.run.smithi144.stdout:Get:18 https://2.chacra.ceph.com/r/ceph/reef/267c8ff5b1ff5815581b3b2381d19360e081b37b/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 18.2.2-1171-g267c8ff5-1jammy [63.7 kB] 2024-04-07T15:59:18.716 INFO:teuthology.orchestra.run.smithi144.stdout:Get:19 https://2.chacra.ceph.com/r/ceph/reef/267c8ff5b1ff5815581b3b2381d19360e081b37b/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 18.2.2-1171-g267c8ff5-1jammy [330 kB] 2024-04-07T15:59:18.736 INFO:teuthology.orchestra.run.smithi144.stdout:Get:20 https://2.chacra.ceph.com/r/ceph/reef/267c8ff5b1ff5815581b3b2381d19360e081b37b/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 18.2.2-1171-g267c8ff5-1jammy [5,891 kB] 2024-04-07T15:59:18.741 INFO:teuthology.orchestra.run.smithi100.stdout:Get:80 https://2.chacra.ceph.com/r/ceph/reef/267c8ff5b1ff5815581b3b2381d19360e081b37b/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 18.2.2-1171-g267c8ff5-1jammy [3,105 kB] 2024-04-07T15:59:18.810 INFO:teuthology.orchestra.run.smithi144.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-04-07T15:59:18.976 INFO:teuthology.orchestra.run.smithi100.stdout:Get:81 https://2.chacra.ceph.com/r/ceph/reef/267c8ff5b1ff5815581b3b2381d19360e081b37b/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 18.2.2-1171-g267c8ff5-1jammy [679 kB] 2024-04-07T15:59:19.031 INFO:teuthology.orchestra.run.smithi100.stdout:Get:82 https://2.chacra.ceph.com/r/ceph/reef/267c8ff5b1ff5815581b3b2381d19360e081b37b/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 18.2.2-1171-g267c8ff5-1jammy [356 kB] 2024-04-07T15:59:19.043 INFO:teuthology.orchestra.run.smithi100.stdout:Get:83 https://2.chacra.ceph.com/r/ceph/reef/267c8ff5b1ff5815581b3b2381d19360e081b37b/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 18.2.2-1171-g267c8ff5-1jammy [32.8 kB] 2024-04-07T15:59:19.044 INFO:teuthology.orchestra.run.smithi100.stdout:Get:84 https://2.chacra.ceph.com/r/ceph/reef/267c8ff5b1ff5815581b3b2381d19360e081b37b/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 18.2.2-1171-g267c8ff5-1jammy [180 kB] 2024-04-07T15:59:19.051 INFO:teuthology.orchestra.run.smithi100.stdout:Get:85 https://2.chacra.ceph.com/r/ceph/reef/267c8ff5b1ff5815581b3b2381d19360e081b37b/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 18.2.2-1171-g267c8ff5-1jammy [63.7 kB] 2024-04-07T15:59:19.058 INFO:teuthology.orchestra.run.smithi144.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-04-07T15:59:19.060 INFO:teuthology.orchestra.run.smithi100.stdout:Get:86 https://2.chacra.ceph.com/r/ceph/reef/267c8ff5b1ff5815581b3b2381d19360e081b37b/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 18.2.2-1171-g267c8ff5-1jammy [330 kB] 2024-04-07T15:59:19.067 INFO:teuthology.orchestra.run.smithi144.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-04-07T15:59:19.167 INFO:teuthology.orchestra.run.smithi144.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-04-07T15:59:19.175 INFO:teuthology.orchestra.run.smithi144.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-04-07T15:59:19.179 INFO:teuthology.orchestra.run.smithi144.stdout:Get:26 https://2.chacra.ceph.com/r/ceph/reef/267c8ff5b1ff5815581b3b2381d19360e081b37b/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 18.2.2-1171-g267c8ff5-1jammy [112 kB] 2024-04-07T15:59:19.184 INFO:teuthology.orchestra.run.smithi144.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-04-07T15:59:19.197 INFO:teuthology.orchestra.run.smithi144.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-04-07T15:59:19.209 INFO:teuthology.orchestra.run.smithi144.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-04-07T15:59:19.216 INFO:teuthology.orchestra.run.smithi144.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-04-07T15:59:19.220 INFO:teuthology.orchestra.run.smithi144.stdout:Get:31 https://2.chacra.ceph.com/r/ceph/reef/267c8ff5b1ff5815581b3b2381d19360e081b37b/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 18.2.2-1171-g267c8ff5-1jammy [441 kB] 2024-04-07T15:59:19.225 INFO:teuthology.orchestra.run.smithi144.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-04-07T15:59:19.266 INFO:teuthology.orchestra.run.smithi144.stdout:Get:33 https://2.chacra.ceph.com/r/ceph/reef/267c8ff5b1ff5815581b3b2381d19360e081b37b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 18.2.2-1171-g267c8ff5-1jammy [21.9 MB] 2024-04-07T15:59:19.311 INFO:teuthology.orchestra.run.smithi100.stdout:Get:87 https://2.chacra.ceph.com/r/ceph/reef/267c8ff5b1ff5815581b3b2381d19360e081b37b/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 18.2.2-1171-g267c8ff5-1jammy [5,891 kB] 2024-04-07T15:59:19.323 INFO:teuthology.orchestra.run.smithi144.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-04-07T15:59:19.497 INFO:teuthology.orchestra.run.smithi144.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-04-07T15:59:19.664 INFO:teuthology.orchestra.run.smithi144.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-04-07T15:59:19.685 INFO:teuthology.orchestra.run.smithi144.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-04-07T15:59:19.820 INFO:teuthology.orchestra.run.smithi144.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-04-07T15:59:19.837 INFO:teuthology.orchestra.run.smithi144.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-04-07T15:59:19.844 INFO:teuthology.orchestra.run.smithi144.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2024-04-07T15:59:19.918 INFO:teuthology.orchestra.run.smithi144.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-04-07T15:59:19.946 INFO:teuthology.orchestra.run.smithi144.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-04-07T15:59:19.953 INFO:teuthology.orchestra.run.smithi144.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-04-07T15:59:20.075 INFO:teuthology.orchestra.run.smithi144.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-04-07T15:59:20.076 INFO:teuthology.orchestra.run.smithi144.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-04-07T15:59:20.244 INFO:teuthology.orchestra.run.smithi144.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-04-07T15:59:20.406 INFO:teuthology.orchestra.run.smithi144.stdout:Get:47 https://2.chacra.ceph.com/r/ceph/reef/267c8ff5b1ff5815581b3b2381d19360e081b37b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 18.2.2-1171-g267c8ff5-1jammy [5,002 kB] 2024-04-07T15:59:20.407 INFO:teuthology.orchestra.run.smithi100.stdout:Get:88 https://2.chacra.ceph.com/r/ceph/reef/267c8ff5b1ff5815581b3b2381d19360e081b37b/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 18.2.2-1171-g267c8ff5-1jammy [112 kB] 2024-04-07T15:59:20.409 INFO:teuthology.orchestra.run.smithi100.stdout:Get:89 https://2.chacra.ceph.com/r/ceph/reef/267c8ff5b1ff5815581b3b2381d19360e081b37b/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 18.2.2-1171-g267c8ff5-1jammy [441 kB] 2024-04-07T15:59:20.462 INFO:teuthology.orchestra.run.smithi100.stdout:Get:90 https://2.chacra.ceph.com/r/ceph/reef/267c8ff5b1ff5815581b3b2381d19360e081b37b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 18.2.2-1171-g267c8ff5-1jammy [21.9 MB] 2024-04-07T15:59:20.487 INFO:teuthology.orchestra.run.smithi144.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-04-07T15:59:20.553 INFO:teuthology.orchestra.run.smithi144.stdout:Get:49 https://2.chacra.ceph.com/r/ceph/reef/267c8ff5b1ff5815581b3b2381d19360e081b37b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 18.2.2-1171-g267c8ff5-1jammy [241 kB] 2024-04-07T15:59:20.571 INFO:teuthology.orchestra.run.smithi144.stdout:Get:50 https://2.chacra.ceph.com/r/ceph/reef/267c8ff5b1ff5815581b3b2381d19360e081b37b/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 18.2.2-1171-g267c8ff5-1jammy [130 kB] 2024-04-07T15:59:20.578 INFO:teuthology.orchestra.run.smithi144.stdout:Get:51 https://2.chacra.ceph.com/r/ceph/reef/267c8ff5b1ff5815581b3b2381d19360e081b37b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 18.2.2-1171-g267c8ff5-1jammy [1,490 kB] 2024-04-07T15:59:20.627 INFO:teuthology.orchestra.run.smithi144.stdout:Get:52 https://2.chacra.ceph.com/r/ceph/reef/267c8ff5b1ff5815581b3b2381d19360e081b37b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 18.2.2-1171-g267c8ff5-1jammy [6,079 kB] 2024-04-07T15:59:20.641 INFO:teuthology.orchestra.run.smithi144.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-04-07T15:59:20.746 INFO:teuthology.orchestra.run.smithi144.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-idna all 3.3-1 [49.3 kB] 2024-04-07T15:59:20.768 INFO:teuthology.orchestra.run.smithi144.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-04-07T15:59:20.800 INFO:teuthology.orchestra.run.smithi144.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-04-07T15:59:20.812 INFO:teuthology.orchestra.run.smithi144.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.1 [181 kB] 2024-04-07T15:59:20.853 INFO:teuthology.orchestra.run.smithi144.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-04-07T15:59:20.861 INFO:teuthology.orchestra.run.smithi144.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-04-07T15:59:20.891 INFO:teuthology.orchestra.run.smithi144.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-04-07T15:59:20.921 INFO:teuthology.orchestra.run.smithi144.stdout:Get:61 https://2.chacra.ceph.com/r/ceph/reef/267c8ff5b1ff5815581b3b2381d19360e081b37b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 18.2.2-1171-g267c8ff5-1jammy [22.3 MB] 2024-04-07T15:59:21.116 INFO:teuthology.orchestra.run.smithi144.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-04-07T15:59:21.281 INFO:teuthology.orchestra.run.smithi144.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.1 [108 kB] 2024-04-07T15:59:21.639 INFO:teuthology.orchestra.run.smithi144.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-04-07T15:59:21.656 INFO:teuthology.orchestra.run.smithi144.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-04-07T15:59:21.668 INFO:teuthology.orchestra.run.smithi144.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-04-07T15:59:21.721 INFO:teuthology.orchestra.run.smithi144.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-04-07T15:59:21.725 INFO:teuthology.orchestra.run.smithi144.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-04-07T15:59:21.734 INFO:teuthology.orchestra.run.smithi144.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-04-07T15:59:21.821 INFO:teuthology.orchestra.run.smithi144.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-04-07T15:59:21.823 INFO:teuthology.orchestra.run.smithi144.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-04-07T15:59:21.844 INFO:teuthology.orchestra.run.smithi144.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-04-07T15:59:22.055 INFO:teuthology.orchestra.run.smithi144.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-04-07T15:59:22.055 INFO:teuthology.orchestra.run.smithi144.stdout:Get:74 https://2.chacra.ceph.com/r/ceph/reef/267c8ff5b1ff5815581b3b2381d19360e081b37b/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 18.2.2-1171-g267c8ff5-1jammy [14.1 kB] 2024-04-07T15:59:22.056 INFO:teuthology.orchestra.run.smithi144.stdout:Get:75 https://2.chacra.ceph.com/r/ceph/reef/267c8ff5b1ff5815581b3b2381d19360e081b37b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 18.2.2-1171-g267c8ff5-1jammy [831 kB] 2024-04-07T15:59:22.072 INFO:teuthology.orchestra.run.smithi144.stdout:Get:76 https://2.chacra.ceph.com/r/ceph/reef/267c8ff5b1ff5815581b3b2381d19360e081b37b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 18.2.2-1171-g267c8ff5-1jammy [2,153 kB] 2024-04-07T15:59:22.110 INFO:teuthology.orchestra.run.smithi144.stdout:Get:77 https://2.chacra.ceph.com/r/ceph/reef/267c8ff5b1ff5815581b3b2381d19360e081b37b/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 18.2.2-1171-g267c8ff5-1jammy [238 kB] 2024-04-07T15:59:22.114 INFO:teuthology.orchestra.run.smithi144.stdout:Get:78 https://2.chacra.ceph.com/r/ceph/reef/267c8ff5b1ff5815581b3b2381d19360e081b37b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 18.2.2-1171-g267c8ff5-1jammy [144 kB] 2024-04-07T15:59:22.116 INFO:teuthology.orchestra.run.smithi144.stdout:Get:79 https://2.chacra.ceph.com/r/ceph/reef/267c8ff5b1ff5815581b3b2381d19360e081b37b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 18.2.2-1171-g267c8ff5-1jammy [2,239 kB] 2024-04-07T15:59:22.203 INFO:teuthology.orchestra.run.smithi144.stdout:Get:80 https://2.chacra.ceph.com/r/ceph/reef/267c8ff5b1ff5815581b3b2381d19360e081b37b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 18.2.2-1171-g267c8ff5-1jammy [8,625 kB] 2024-04-07T15:59:22.224 INFO:teuthology.orchestra.run.smithi144.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-04-07T15:59:22.397 INFO:teuthology.orchestra.run.smithi144.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-04-07T15:59:22.600 INFO:teuthology.orchestra.run.smithi144.stdout:Get:83 https://2.chacra.ceph.com/r/ceph/reef/267c8ff5b1ff5815581b3b2381d19360e081b37b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 18.2.2-1171-g267c8ff5-1jammy [14.3 kB] 2024-04-07T15:59:22.601 INFO:teuthology.orchestra.run.smithi144.stdout:Get:84 https://2.chacra.ceph.com/r/ceph/reef/267c8ff5b1ff5815581b3b2381d19360e081b37b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 18.2.2-1171-g267c8ff5-1jammy [42.6 MB] 2024-04-07T15:59:22.902 INFO:teuthology.orchestra.run.smithi144.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-04-07T15:59:22.903 INFO:teuthology.orchestra.run.smithi144.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-04-07T15:59:22.906 INFO:teuthology.orchestra.run.smithi144.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-04-07T15:59:22.907 INFO:teuthology.orchestra.run.smithi144.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-04-07T15:59:22.909 INFO:teuthology.orchestra.run.smithi144.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-04-07T15:59:22.911 INFO:teuthology.orchestra.run.smithi144.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-04-07T15:59:22.912 INFO:teuthology.orchestra.run.smithi144.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-04-07T15:59:22.914 INFO:teuthology.orchestra.run.smithi144.stdout:Get:92 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-04-07T15:59:22.933 INFO:teuthology.orchestra.run.smithi144.stdout:Get:93 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-04-07T15:59:23.067 INFO:teuthology.orchestra.run.smithi100.stdout:Get:91 https://2.chacra.ceph.com/r/ceph/reef/267c8ff5b1ff5815581b3b2381d19360e081b37b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 18.2.2-1171-g267c8ff5-1jammy [5,002 kB] 2024-04-07T15:59:23.104 INFO:teuthology.orchestra.run.smithi144.stdout:Get:94 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-04-07T15:59:23.484 INFO:teuthology.orchestra.run.smithi144.stdout:Get:95 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-04-07T15:59:23.507 INFO:teuthology.orchestra.run.smithi144.stdout:Get:96 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-04-07T15:59:23.562 INFO:teuthology.orchestra.run.smithi100.stdout:Get:92 https://2.chacra.ceph.com/r/ceph/reef/267c8ff5b1ff5815581b3b2381d19360e081b37b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 18.2.2-1171-g267c8ff5-1jammy [241 kB] 2024-04-07T15:59:23.588 INFO:teuthology.orchestra.run.smithi100.stdout:Get:93 https://2.chacra.ceph.com/r/ceph/reef/267c8ff5b1ff5815581b3b2381d19360e081b37b/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 18.2.2-1171-g267c8ff5-1jammy [130 kB] 2024-04-07T15:59:23.612 INFO:teuthology.orchestra.run.smithi100.stdout:Get:94 https://2.chacra.ceph.com/r/ceph/reef/267c8ff5b1ff5815581b3b2381d19360e081b37b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 18.2.2-1171-g267c8ff5-1jammy [1,490 kB] 2024-04-07T15:59:23.659 INFO:teuthology.orchestra.run.smithi144.stdout:Get:97 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-04-07T15:59:23.709 INFO:teuthology.orchestra.run.smithi144.stdout:Get:98 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2024-04-07T15:59:23.770 INFO:teuthology.orchestra.run.smithi144.stdout:Get:99 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-04-07T15:59:23.782 INFO:teuthology.orchestra.run.smithi100.stdout:Get:95 https://2.chacra.ceph.com/r/ceph/reef/267c8ff5b1ff5815581b3b2381d19360e081b37b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 18.2.2-1171-g267c8ff5-1jammy [6,079 kB] 2024-04-07T15:59:23.797 INFO:teuthology.orchestra.run.smithi144.stdout:Get:100 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-04-07T15:59:23.800 INFO:teuthology.orchestra.run.smithi144.stdout:Get:101 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-04-07T15:59:23.813 INFO:teuthology.orchestra.run.smithi144.stdout:Get:102 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-04-07T15:59:23.856 INFO:teuthology.orchestra.run.smithi144.stdout:Get:103 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-04-07T15:59:23.857 INFO:teuthology.orchestra.run.smithi144.stdout:Get:104 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-04-07T15:59:24.023 INFO:teuthology.orchestra.run.smithi144.stdout:Get:105 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-04-07T15:59:24.464 INFO:teuthology.orchestra.run.smithi100.stdout:Get:96 https://2.chacra.ceph.com/r/ceph/reef/267c8ff5b1ff5815581b3b2381d19360e081b37b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 18.2.2-1171-g267c8ff5-1jammy [22.3 MB] 2024-04-07T15:59:24.760 INFO:teuthology.orchestra.run.smithi144.stdout:Get:106 https://2.chacra.ceph.com/r/ceph/reef/267c8ff5b1ff5815581b3b2381d19360e081b37b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 18.2.2-1171-g267c8ff5-1jammy [121 kB] 2024-04-07T15:59:24.762 INFO:teuthology.orchestra.run.smithi144.stdout:Get:107 https://2.chacra.ceph.com/r/ceph/reef/267c8ff5b1ff5815581b3b2381d19360e081b37b/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 18.2.2-1171-g267c8ff5-1jammy [38.3 kB] 2024-04-07T15:59:24.829 INFO:teuthology.orchestra.run.smithi144.stdout:Get:108 https://2.chacra.ceph.com/r/ceph/reef/267c8ff5b1ff5815581b3b2381d19360e081b37b/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 18.2.2-1171-g267c8ff5-1jammy [11.6 MB] 2024-04-07T15:59:25.211 INFO:teuthology.orchestra.run.smithi144.stdout:Get:109 https://2.chacra.ceph.com/r/ceph/reef/267c8ff5b1ff5815581b3b2381d19360e081b37b/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 18.2.2-1171-g267c8ff5-1jammy [93.1 kB] 2024-04-07T15:59:25.766 INFO:teuthology.orchestra.run.smithi144.stdout:Fetched 162 MB in 8s (21.5 MB/s) 2024-04-07T15:59:25.869 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package libfmt8:amd64. 2024-04-07T15:59:25.939 INFO:teuthology.orchestra.run.smithi100.stdout:Get:97 https://2.chacra.ceph.com/r/ceph/reef/267c8ff5b1ff5815581b3b2381d19360e081b37b/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 18.2.2-1171-g267c8ff5-1jammy [14.1 kB] 2024-04-07T15:59:25.939 INFO:teuthology.orchestra.run.smithi100.stdout:Get:98 https://2.chacra.ceph.com/r/ceph/reef/267c8ff5b1ff5815581b3b2381d19360e081b37b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 18.2.2-1171-g267c8ff5-1jammy [831 kB] 2024-04-07T15:59:25.947 INFO:teuthology.orchestra.run.smithi144.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165997 files and directories currently installed.) 2024-04-07T15:59:25.952 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../000-libfmt8_8.1.1+ds1-2_amd64.deb ... 2024-04-07T15:59:25.955 INFO:teuthology.orchestra.run.smithi100.stdout:Get:99 https://2.chacra.ceph.com/r/ceph/reef/267c8ff5b1ff5815581b3b2381d19360e081b37b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 18.2.2-1171-g267c8ff5-1jammy [2,153 kB] 2024-04-07T15:59:26.013 INFO:teuthology.orchestra.run.smithi100.stdout:Get:100 https://2.chacra.ceph.com/r/ceph/reef/267c8ff5b1ff5815581b3b2381d19360e081b37b/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 18.2.2-1171-g267c8ff5-1jammy [238 kB] 2024-04-07T15:59:26.018 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking libfmt8:amd64 (8.1.1+ds1-2) ... 2024-04-07T15:59:26.018 INFO:teuthology.orchestra.run.smithi100.stdout:Get:101 https://2.chacra.ceph.com/r/ceph/reef/267c8ff5b1ff5815581b3b2381d19360e081b37b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 18.2.2-1171-g267c8ff5-1jammy [144 kB] 2024-04-07T15:59:26.028 INFO:teuthology.orchestra.run.smithi100.stdout:Get:102 https://2.chacra.ceph.com/r/ceph/reef/267c8ff5b1ff5815581b3b2381d19360e081b37b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 18.2.2-1171-g267c8ff5-1jammy [2,239 kB] 2024-04-07T15:59:26.126 INFO:teuthology.orchestra.run.smithi100.stdout:Get:103 https://2.chacra.ceph.com/r/ceph/reef/267c8ff5b1ff5815581b3b2381d19360e081b37b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 18.2.2-1171-g267c8ff5-1jammy [8,625 kB] 2024-04-07T15:59:26.355 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-04-07T15:59:26.381 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../001-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-04-07T15:59:26.421 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-04-07T15:59:26.508 INFO:teuthology.orchestra.run.smithi100.stdout:Get:104 https://2.chacra.ceph.com/r/ceph/reef/267c8ff5b1ff5815581b3b2381d19360e081b37b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 18.2.2-1171-g267c8ff5-1jammy [14.3 kB] 2024-04-07T15:59:26.508 INFO:teuthology.orchestra.run.smithi100.stdout:Get:105 https://2.chacra.ceph.com/r/ceph/reef/267c8ff5b1ff5815581b3b2381d19360e081b37b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 18.2.2-1171-g267c8ff5-1jammy [42.6 MB] 2024-04-07T15:59:26.775 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-04-07T15:59:26.801 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../002-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-04-07T15:59:26.841 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-04-07T15:59:27.169 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-04-07T15:59:27.195 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../003-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-04-07T15:59:27.236 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-04-07T15:59:27.656 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-04-07T15:59:27.682 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../004-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-07T15:59:27.730 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-07T15:59:28.100 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-04-07T15:59:28.127 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../005-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-07T15:59:28.167 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-07T15:59:28.478 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-04-07T15:59:28.505 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../006-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-07T15:59:28.545 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-07T15:59:28.555 INFO:teuthology.orchestra.run.smithi100.stdout:Get:106 https://2.chacra.ceph.com/r/ceph/reef/267c8ff5b1ff5815581b3b2381d19360e081b37b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 18.2.2-1171-g267c8ff5-1jammy [121 kB] 2024-04-07T15:59:28.558 INFO:teuthology.orchestra.run.smithi100.stdout:Get:107 https://2.chacra.ceph.com/r/ceph/reef/267c8ff5b1ff5815581b3b2381d19360e081b37b/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 18.2.2-1171-g267c8ff5-1jammy [38.3 kB] 2024-04-07T15:59:28.559 INFO:teuthology.orchestra.run.smithi100.stdout:Get:108 https://2.chacra.ceph.com/r/ceph/reef/267c8ff5b1ff5815581b3b2381d19360e081b37b/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 18.2.2-1171-g267c8ff5-1jammy [11.6 MB] 2024-04-07T15:59:28.856 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-04-07T15:59:28.883 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../007-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-04-07T15:59:28.923 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-04-07T15:59:28.973 INFO:teuthology.orchestra.run.smithi100.stdout:Get:109 https://2.chacra.ceph.com/r/ceph/reef/267c8ff5b1ff5815581b3b2381d19360e081b37b/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 18.2.2-1171-g267c8ff5-1jammy [93.1 kB] 2024-04-07T15:59:29.242 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package librados2. 2024-04-07T15:59:29.269 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../008-librados2_18.2.2-1171-g267c8ff5-1jammy_amd64.deb ... 2024-04-07T15:59:29.309 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking librados2 (18.2.2-1171-g267c8ff5-1jammy) ... 2024-04-07T15:59:29.527 INFO:teuthology.orchestra.run.smithi100.stdout:Fetched 162 MB in 11s (14.7 MB/s) 2024-04-07T15:59:29.668 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package libfmt8:amd64. 2024-04-07T15:59:29.754 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package librbd1. 2024-04-07T15:59:29.764 INFO:teuthology.orchestra.run.smithi100.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165997 files and directories currently installed.) 2024-04-07T15:59:29.769 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../000-libfmt8_8.1.1+ds1-2_amd64.deb ... 2024-04-07T15:59:29.780 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../009-librbd1_18.2.2-1171-g267c8ff5-1jammy_amd64.deb ... 2024-04-07T15:59:29.820 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking librbd1 (18.2.2-1171-g267c8ff5-1jammy) ... 2024-04-07T15:59:29.858 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking libfmt8:amd64 (8.1.1+ds1-2) ... 2024-04-07T15:59:30.169 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-04-07T15:59:30.195 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../001-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-04-07T15:59:30.227 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-04-07T15:59:30.248 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package libcephfs2. 2024-04-07T15:59:30.274 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../010-libcephfs2_18.2.2-1171-g267c8ff5-1jammy_amd64.deb ... 2024-04-07T15:59:30.306 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking libcephfs2 (18.2.2-1171-g267c8ff5-1jammy) ... 2024-04-07T15:59:30.547 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-04-07T15:59:30.574 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../002-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-04-07T15:59:30.593 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-rados. 2024-04-07T15:59:30.614 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-04-07T15:59:30.618 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../011-python3-rados_18.2.2-1171-g267c8ff5-1jammy_amd64.deb ... 2024-04-07T15:59:30.650 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-rados (18.2.2-1171-g267c8ff5-1jammy) ... 2024-04-07T15:59:30.959 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-04-07T15:59:30.970 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-04-07T15:59:30.985 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../003-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-04-07T15:59:30.996 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../012-python3-ceph-argparse_18.2.2-1171-g267c8ff5-1jammy_all.deb ... 2024-04-07T15:59:31.025 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-04-07T15:59:31.036 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-ceph-argparse (18.2.2-1171-g267c8ff5-1jammy) ... 2024-04-07T15:59:31.289 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-cephfs. 2024-04-07T15:59:31.304 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../013-python3-cephfs_18.2.2-1171-g267c8ff5-1jammy_amd64.deb ... 2024-04-07T15:59:31.347 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-cephfs (18.2.2-1171-g267c8ff5-1jammy) ... 2024-04-07T15:59:31.645 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-04-07T15:59:31.658 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-ceph-common. 2024-04-07T15:59:31.671 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../004-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-07T15:59:31.684 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../014-python3-ceph-common_18.2.2-1171-g267c8ff5-1jammy_all.deb ... 2024-04-07T15:59:31.720 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-07T15:59:31.724 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-ceph-common (18.2.2-1171-g267c8ff5-1jammy) ... 2024-04-07T15:59:32.002 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-wcwidth. 2024-04-07T15:59:32.028 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../015-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-04-07T15:59:32.071 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-04-07T15:59:32.115 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-04-07T15:59:32.141 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../005-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-07T15:59:32.181 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-07T15:59:32.374 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-prettytable. 2024-04-07T15:59:32.400 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../016-python3-prettytable_2.5.0-2_all.deb ... 2024-04-07T15:59:32.441 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-04-07T15:59:32.526 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-04-07T15:59:32.552 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../006-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-07T15:59:32.593 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-07T15:59:32.693 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-rbd. 2024-04-07T15:59:32.719 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../017-python3-rbd_18.2.2-1171-g267c8ff5-1jammy_amd64.deb ... 2024-04-07T15:59:32.760 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-rbd (18.2.2-1171-g267c8ff5-1jammy) ... 2024-04-07T15:59:32.954 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-04-07T15:59:32.980 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../007-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-04-07T15:59:33.021 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-04-07T15:59:33.104 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-04-07T15:59:33.130 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../018-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-04-07T15:59:33.171 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-04-07T15:59:33.315 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package librados2. 2024-04-07T15:59:33.341 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../008-librados2_18.2.2-1171-g267c8ff5-1jammy_amd64.deb ... 2024-04-07T15:59:33.382 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking librados2 (18.2.2-1171-g267c8ff5-1jammy) ... 2024-04-07T15:59:33.474 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package librgw2. 2024-04-07T15:59:33.500 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../019-librgw2_18.2.2-1171-g267c8ff5-1jammy_amd64.deb ... 2024-04-07T15:59:33.532 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking librgw2 (18.2.2-1171-g267c8ff5-1jammy) ... 2024-04-07T15:59:33.843 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package librbd1. 2024-04-07T15:59:33.869 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../009-librbd1_18.2.2-1171-g267c8ff5-1jammy_amd64.deb ... 2024-04-07T15:59:33.909 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking librbd1 (18.2.2-1171-g267c8ff5-1jammy) ... 2024-04-07T15:59:34.035 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-rgw. 2024-04-07T15:59:34.061 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../020-python3-rgw_18.2.2-1171-g267c8ff5-1jammy_amd64.deb ... 2024-04-07T15:59:34.101 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-rgw (18.2.2-1171-g267c8ff5-1jammy) ... 2024-04-07T15:59:34.345 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package libcephfs2. 2024-04-07T15:59:34.371 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../010-libcephfs2_18.2.2-1171-g267c8ff5-1jammy_amd64.deb ... 2024-04-07T15:59:34.437 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package liboath0:amd64. 2024-04-07T15:59:34.454 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking libcephfs2 (18.2.2-1171-g267c8ff5-1jammy) ... 2024-04-07T15:59:34.463 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../021-liboath0_2.6.7-3build1_amd64.deb ... 2024-04-07T15:59:34.504 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-04-07T15:59:34.731 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-rados. 2024-04-07T15:59:34.747 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../011-python3-rados_18.2.2-1171-g267c8ff5-1jammy_amd64.deb ... 2024-04-07T15:59:34.782 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package libradosstriper1. 2024-04-07T15:59:34.798 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-rados (18.2.2-1171-g267c8ff5-1jammy) ... 2024-04-07T15:59:34.808 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../022-libradosstriper1_18.2.2-1171-g267c8ff5-1jammy_amd64.deb ... 2024-04-07T15:59:34.840 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking libradosstriper1 (18.2.2-1171-g267c8ff5-1jammy) ... 2024-04-07T15:59:35.100 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-04-07T15:59:35.127 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../012-python3-ceph-argparse_18.2.2-1171-g267c8ff5-1jammy_all.deb ... 2024-04-07T15:59:35.167 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-ceph-argparse (18.2.2-1171-g267c8ff5-1jammy) ... 2024-04-07T15:59:35.394 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-cephfs. 2024-04-07T15:59:35.410 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../013-python3-cephfs_18.2.2-1171-g267c8ff5-1jammy_amd64.deb ... 2024-04-07T15:59:35.444 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-cephfs (18.2.2-1171-g267c8ff5-1jammy) ... 2024-04-07T15:59:35.627 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package ceph-common. 2024-04-07T15:59:35.653 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../023-ceph-common_18.2.2-1171-g267c8ff5-1jammy_amd64.deb ... 2024-04-07T15:59:35.693 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking ceph-common (18.2.2-1171-g267c8ff5-1jammy) ... 2024-04-07T15:59:35.747 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-ceph-common. 2024-04-07T15:59:35.774 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../014-python3-ceph-common_18.2.2-1171-g267c8ff5-1jammy_all.deb ... 2024-04-07T15:59:35.813 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-ceph-common (18.2.2-1171-g267c8ff5-1jammy) ... 2024-04-07T15:59:36.125 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-wcwidth. 2024-04-07T15:59:36.152 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../015-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-04-07T15:59:36.262 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-04-07T15:59:36.636 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-prettytable. 2024-04-07T15:59:36.663 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../016-python3-prettytable_2.5.0-2_all.deb ... 2024-04-07T15:59:36.703 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-04-07T15:59:36.930 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-rbd. 2024-04-07T15:59:36.945 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../017-python3-rbd_18.2.2-1171-g267c8ff5-1jammy_amd64.deb ... 2024-04-07T15:59:36.974 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package ceph-base. 2024-04-07T15:59:36.997 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-rbd (18.2.2-1171-g267c8ff5-1jammy) ... 2024-04-07T15:59:37.000 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../024-ceph-base_18.2.2-1171-g267c8ff5-1jammy_amd64.deb ... 2024-04-07T15:59:37.050 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking ceph-base (18.2.2-1171-g267c8ff5-1jammy) ... 2024-04-07T15:59:37.358 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-04-07T15:59:37.385 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../018-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-04-07T15:59:37.425 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-04-07T15:59:37.637 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-04-07T15:59:37.652 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../025-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-04-07T15:59:37.695 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-04-07T15:59:37.753 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package librgw2. 2024-04-07T15:59:37.779 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../019-librgw2_18.2.2-1171-g267c8ff5-1jammy_amd64.deb ... 2024-04-07T15:59:37.819 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking librgw2 (18.2.2-1171-g267c8ff5-1jammy) ... 2024-04-07T15:59:37.998 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-cheroot. 2024-04-07T15:59:38.024 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../026-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-04-07T15:59:38.065 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-04-07T15:59:38.338 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-rgw. 2024-04-07T15:59:38.354 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../020-python3-rgw_18.2.2-1171-g267c8ff5-1jammy_amd64.deb ... 2024-04-07T15:59:38.359 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-04-07T15:59:38.385 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../027-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-04-07T15:59:38.396 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-rgw (18.2.2-1171-g267c8ff5-1jammy) ... 2024-04-07T15:59:38.417 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-04-07T15:59:38.720 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-jaraco.text. 2024-04-07T15:59:38.724 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package liboath0:amd64. 2024-04-07T15:59:38.746 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../028-python3-jaraco.text_3.6.0-2_all.deb ... 2024-04-07T15:59:38.751 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../021-liboath0_2.6.7-3build1_amd64.deb ... 2024-04-07T15:59:38.787 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-04-07T15:59:38.791 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-04-07T15:59:39.102 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package libradosstriper1. 2024-04-07T15:59:39.115 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-04-07T15:59:39.129 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../022-libradosstriper1_18.2.2-1171-g267c8ff5-1jammy_amd64.deb ... 2024-04-07T15:59:39.141 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../029-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-04-07T15:59:39.169 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking libradosstriper1 (18.2.2-1171-g267c8ff5-1jammy) ... 2024-04-07T15:59:39.181 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-04-07T15:59:39.493 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-tempora. 2024-04-07T15:59:39.519 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../030-python3-tempora_4.1.2-1_all.deb ... 2024-04-07T15:59:39.559 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-04-07T15:59:39.605 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package ceph-common. 2024-04-07T15:59:39.632 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../023-ceph-common_18.2.2-1171-g267c8ff5-1jammy_amd64.deb ... 2024-04-07T15:59:39.671 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking ceph-common (18.2.2-1171-g267c8ff5-1jammy) ... 2024-04-07T15:59:39.812 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-portend. 2024-04-07T15:59:39.827 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../031-python3-portend_3.0.0-1_all.deb ... 2024-04-07T15:59:39.870 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-04-07T15:59:40.174 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-04-07T15:59:40.200 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../032-python3-zc.lockfile_2.0-1_all.deb ... 2024-04-07T15:59:40.240 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-04-07T15:59:40.543 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-cherrypy3. 2024-04-07T15:59:40.570 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../033-python3-cherrypy3_18.6.1-4_all.deb ... 2024-04-07T15:59:40.610 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-04-07T15:59:40.944 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package ceph-base. 2024-04-07T15:59:40.971 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../024-ceph-base_18.2.2-1171-g267c8ff5-1jammy_amd64.deb ... 2024-04-07T15:59:41.020 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking ceph-base (18.2.2-1171-g267c8ff5-1jammy) ... 2024-04-07T15:59:41.164 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-natsort. 2024-04-07T15:59:41.191 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../034-python3-natsort_8.0.2-1_all.deb ... 2024-04-07T15:59:41.230 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-04-07T15:59:41.541 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-logutils. 2024-04-07T15:59:41.565 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-04-07T15:59:41.568 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../035-python3-logutils_0.3.3-8_all.deb ... 2024-04-07T15:59:41.592 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../025-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-04-07T15:59:41.608 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-04-07T15:59:41.631 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-04-07T15:59:41.894 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-markupsafe. 2024-04-07T15:59:41.918 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../036-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-04-07T15:59:41.934 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-cheroot. 2024-04-07T15:59:41.952 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-04-07T15:59:41.961 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../026-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-04-07T15:59:42.001 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-04-07T15:59:42.231 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-mako. 2024-04-07T15:59:42.257 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../037-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-04-07T15:59:42.297 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-04-07T15:59:42.337 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-04-07T15:59:42.363 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../027-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-04-07T15:59:42.404 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-04-07T15:59:42.634 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-simplegeneric. 2024-04-07T15:59:42.660 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../038-python3-simplegeneric_0.8.1-3_all.deb ... 2024-04-07T15:59:42.681 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-jaraco.text. 2024-04-07T15:59:42.700 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-04-07T15:59:42.708 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../028-python3-jaraco.text_3.6.0-2_all.deb ... 2024-04-07T15:59:42.740 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-04-07T15:59:43.003 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-singledispatch. 2024-04-07T15:59:43.030 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../039-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-04-07T15:59:43.059 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-04-07T15:59:43.069 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-04-07T15:59:43.086 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../029-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-04-07T15:59:43.126 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-04-07T15:59:43.356 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-webob. 2024-04-07T15:59:43.371 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../040-python3-webob_1%3a1.8.6-1.1_all.deb ... 2024-04-07T15:59:43.406 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2024-04-07T15:59:43.429 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-tempora. 2024-04-07T15:59:43.456 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../030-python3-tempora_4.1.2-1_all.deb ... 2024-04-07T15:59:43.495 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-04-07T15:59:43.717 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-waitress. 2024-04-07T15:59:43.744 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../041-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-04-07T15:59:43.773 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-portend. 2024-04-07T15:59:43.786 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-04-07T15:59:43.799 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../031-python3-portend_3.0.0-1_all.deb ... 2024-04-07T15:59:43.840 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-04-07T15:59:44.095 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-tempita. 2024-04-07T15:59:44.101 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-04-07T15:59:44.122 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../042-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-04-07T15:59:44.128 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../032-python3-zc.lockfile_2.0-1_all.deb ... 2024-04-07T15:59:44.162 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-04-07T15:59:44.167 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-04-07T15:59:44.470 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-cherrypy3. 2024-04-07T15:59:44.473 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-paste. 2024-04-07T15:59:44.497 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../033-python3-cherrypy3_18.6.1-4_all.deb ... 2024-04-07T15:59:44.500 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../043-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-04-07T15:59:44.537 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-04-07T15:59:44.556 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-04-07T15:59:44.865 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-natsort. 2024-04-07T15:59:44.884 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-04-07T15:59:44.892 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../034-python3-natsort_8.0.2-1_all.deb ... 2024-04-07T15:59:44.911 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../044-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-04-07T15:59:44.932 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-04-07T15:59:44.951 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-04-07T15:59:45.243 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-logutils. 2024-04-07T15:59:45.262 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-pastedeploy. 2024-04-07T15:59:45.270 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../035-python3-logutils_0.3.3-8_all.deb ... 2024-04-07T15:59:45.289 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../045-python3-pastedeploy_2.1.1-1_all.deb ... 2024-04-07T15:59:45.310 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-04-07T15:59:45.329 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-04-07T15:59:45.613 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-markupsafe. 2024-04-07T15:59:45.639 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../036-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-04-07T15:59:45.640 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-webtest. 2024-04-07T15:59:45.666 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../046-python3-webtest_2.0.35-1_all.deb ... 2024-04-07T15:59:45.679 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-04-07T15:59:45.707 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-04-07T15:59:45.974 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-mako. 2024-04-07T15:59:45.984 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-pecan. 2024-04-07T15:59:46.000 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../047-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-04-07T15:59:46.001 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../037-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-04-07T15:59:46.035 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-04-07T15:59:46.040 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-04-07T15:59:46.360 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-simplegeneric. 2024-04-07T15:59:46.387 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../038-python3-simplegeneric_0.8.1-3_all.deb ... 2024-04-07T15:59:46.415 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-certifi. 2024-04-07T15:59:46.418 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-04-07T15:59:46.442 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../048-python3-certifi_2020.6.20-1_all.deb ... 2024-04-07T15:59:46.481 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-04-07T15:59:46.809 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-idna. 2024-04-07T15:59:46.836 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../049-python3-idna_3.3-1_all.deb ... 2024-04-07T15:59:46.876 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-idna (3.3-1) ... 2024-04-07T15:59:46.896 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-singledispatch. 2024-04-07T15:59:46.925 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../039-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-04-07T15:59:46.963 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-04-07T15:59:47.162 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-urllib3. 2024-04-07T15:59:47.185 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../050-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-04-07T15:59:47.220 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-04-07T15:59:47.282 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-webob. 2024-04-07T15:59:47.309 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../040-python3-webob_1%3a1.8.6-1.1_all.deb ... 2024-04-07T15:59:47.349 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2024-04-07T15:59:47.532 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-requests. 2024-04-07T15:59:47.559 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../051-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-04-07T15:59:47.590 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-04-07T15:59:47.710 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-waitress. 2024-04-07T15:59:47.737 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../041-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-04-07T15:59:47.780 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-04-07T15:59:47.893 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-werkzeug. 2024-04-07T15:59:47.919 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../052-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.1_all.deb ... 2024-04-07T15:59:47.959 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-04-07T15:59:48.088 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-tempita. 2024-04-07T15:59:48.115 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../042-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-04-07T15:59:48.154 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-04-07T15:59:48.263 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-04-07T15:59:48.289 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../053-ceph-mgr-modules-core_18.2.2-1171-g267c8ff5-1jammy_all.deb ... 2024-04-07T15:59:48.329 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking ceph-mgr-modules-core (18.2.2-1171-g267c8ff5-1jammy) ... 2024-04-07T15:59:48.416 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-paste. 2024-04-07T15:59:48.442 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../043-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-04-07T15:59:48.482 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-04-07T15:59:48.701 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-04-07T15:59:48.716 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../054-libsqlite3-mod-ceph_18.2.2-1171-g267c8ff5-1jammy_amd64.deb ... 2024-04-07T15:59:48.751 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking libsqlite3-mod-ceph (18.2.2-1171-g267c8ff5-1jammy) ... 2024-04-07T15:59:48.802 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-04-07T15:59:48.817 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../044-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-04-07T15:59:48.860 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-04-07T15:59:49.062 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-openssl. 2024-04-07T15:59:49.089 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../055-python3-openssl_21.0.0-1_all.deb ... 2024-04-07T15:59:49.121 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-pastedeploy. 2024-04-07T15:59:49.128 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-04-07T15:59:49.148 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../045-python3-pastedeploy_2.1.1-1_all.deb ... 2024-04-07T15:59:49.179 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-04-07T15:59:49.473 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package ceph-mgr. 2024-04-07T15:59:49.482 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-webtest. 2024-04-07T15:59:49.500 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../056-ceph-mgr_18.2.2-1171-g267c8ff5-1jammy_amd64.deb ... 2024-04-07T15:59:49.509 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../046-python3-webtest_2.0.35-1_all.deb ... 2024-04-07T15:59:49.540 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking ceph-mgr (18.2.2-1171-g267c8ff5-1jammy) ... 2024-04-07T15:59:49.549 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-04-07T15:59:49.885 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-pecan. 2024-04-07T15:59:49.912 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../047-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-04-07T15:59:49.926 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package ceph-mon. 2024-04-07T15:59:49.952 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-04-07T15:59:49.953 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../057-ceph-mon_18.2.2-1171-g267c8ff5-1jammy_amd64.deb ... 2024-04-07T15:59:49.993 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking ceph-mon (18.2.2-1171-g267c8ff5-1jammy) ... 2024-04-07T15:59:50.298 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-certifi. 2024-04-07T15:59:50.325 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../048-python3-certifi_2020.6.20-1_all.deb ... 2024-04-07T15:59:50.364 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-04-07T15:59:50.563 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package ceph-osd. 2024-04-07T15:59:50.590 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../058-ceph-osd_18.2.2-1171-g267c8ff5-1jammy_amd64.deb ... 2024-04-07T15:59:50.621 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking ceph-osd (18.2.2-1171-g267c8ff5-1jammy) ... 2024-04-07T15:59:50.701 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-idna. 2024-04-07T15:59:50.728 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../049-python3-idna_3.3-1_all.deb ... 2024-04-07T15:59:50.767 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-idna (3.3-1) ... 2024-04-07T15:59:51.973 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-urllib3. 2024-04-07T15:59:51.999 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../050-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-04-07T15:59:52.037 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-04-07T15:59:52.217 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package ceph. 2024-04-07T15:59:52.232 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../059-ceph_18.2.2-1171-g267c8ff5-1jammy_amd64.deb ... 2024-04-07T15:59:52.275 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking ceph (18.2.2-1171-g267c8ff5-1jammy) ... 2024-04-07T15:59:52.357 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-requests. 2024-04-07T15:59:52.384 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../051-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-04-07T15:59:52.423 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-04-07T15:59:52.594 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package ceph-fuse. 2024-04-07T15:59:52.621 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../060-ceph-fuse_18.2.2-1171-g267c8ff5-1jammy_amd64.deb ... 2024-04-07T15:59:52.661 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking ceph-fuse (18.2.2-1171-g267c8ff5-1jammy) ... 2024-04-07T15:59:52.735 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-werkzeug. 2024-04-07T15:59:52.762 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../052-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.1_all.deb ... 2024-04-07T15:59:52.801 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-04-07T15:59:53.014 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package ceph-mds. 2024-04-07T15:59:53.029 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../061-ceph-mds_18.2.2-1171-g267c8ff5-1jammy_amd64.deb ... 2024-04-07T15:59:53.064 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking ceph-mds (18.2.2-1171-g267c8ff5-1jammy) ... 2024-04-07T15:59:53.096 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-04-07T15:59:53.111 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../053-ceph-mgr-modules-core_18.2.2-1171-g267c8ff5-1jammy_all.deb ... 2024-04-07T15:59:53.145 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking ceph-mgr-modules-core (18.2.2-1171-g267c8ff5-1jammy) ... 2024-04-07T15:59:53.409 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package cephadm. 2024-04-07T15:59:53.436 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../062-cephadm_18.2.2-1171-g267c8ff5-1jammy_amd64.deb ... 2024-04-07T15:59:53.475 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking cephadm (18.2.2-1171-g267c8ff5-1jammy) ... 2024-04-07T15:59:53.509 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-04-07T15:59:53.536 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../054-libsqlite3-mod-ceph_18.2.2-1171-g267c8ff5-1jammy_amd64.deb ... 2024-04-07T15:59:53.575 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking libsqlite3-mod-ceph (18.2.2-1171-g267c8ff5-1jammy) ... 2024-04-07T15:59:53.787 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-asyncssh. 2024-04-07T15:59:53.813 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../063-python3-asyncssh_2.5.0-1_all.deb ... 2024-04-07T15:59:53.853 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-openssl. 2024-04-07T15:59:53.853 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-04-07T15:59:53.868 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../055-python3-openssl_21.0.0-1_all.deb ... 2024-04-07T15:59:53.903 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-04-07T15:59:54.156 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python-babel-localedata. 2024-04-07T15:59:54.172 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../064-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-04-07T15:59:54.206 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package ceph-mgr. 2024-04-07T15:59:54.223 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-04-07T15:59:54.232 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../056-ceph-mgr_18.2.2-1171-g267c8ff5-1jammy_amd64.deb ... 2024-04-07T15:59:54.272 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking ceph-mgr (18.2.2-1171-g267c8ff5-1jammy) ... 2024-04-07T15:59:54.659 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package ceph-mon. 2024-04-07T15:59:54.686 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../057-ceph-mon_18.2.2-1171-g267c8ff5-1jammy_amd64.deb ... 2024-04-07T15:59:54.725 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking ceph-mon (18.2.2-1171-g267c8ff5-1jammy) ... 2024-04-07T15:59:55.203 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-babel. 2024-04-07T15:59:55.230 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../065-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-04-07T15:59:55.261 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-04-07T15:59:55.354 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package ceph-osd. 2024-04-07T15:59:55.381 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../058-ceph-osd_18.2.2-1171-g267c8ff5-1jammy_amd64.deb ... 2024-04-07T15:59:55.420 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking ceph-osd (18.2.2-1171-g267c8ff5-1jammy) ... 2024-04-07T15:59:55.573 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-jinja2. 2024-04-07T15:59:55.599 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../066-python3-jinja2_3.0.3-1ubuntu0.1_all.deb ... 2024-04-07T15:59:55.631 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-04-07T15:59:55.925 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-04-07T15:59:55.941 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../067-ceph-mgr-cephadm_18.2.2-1171-g267c8ff5-1jammy_all.deb ... 2024-04-07T15:59:55.984 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking ceph-mgr-cephadm (18.2.2-1171-g267c8ff5-1jammy) ... 2024-04-07T15:59:56.440 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package ceph. 2024-04-07T15:59:56.467 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../059-ceph_18.2.2-1171-g267c8ff5-1jammy_amd64.deb ... 2024-04-07T15:59:56.507 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking ceph (18.2.2-1171-g267c8ff5-1jammy) ... 2024-04-07T15:59:56.526 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-bcrypt. 2024-04-07T15:59:56.553 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../068-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-04-07T15:59:56.595 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-04-07T15:59:56.826 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package ceph-fuse. 2024-04-07T15:59:56.854 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../060-ceph-fuse_18.2.2-1171-g267c8ff5-1jammy_amd64.deb ... 2024-04-07T15:59:56.857 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-pluggy. 2024-04-07T15:59:56.883 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../069-python3-pluggy_0.13.0-7.1_all.deb ... 2024-04-07T15:59:56.893 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking ceph-fuse (18.2.2-1171-g267c8ff5-1jammy) ... 2024-04-07T15:59:56.923 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-04-07T15:59:57.235 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-py. 2024-04-07T15:59:57.246 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package ceph-mds. 2024-04-07T15:59:57.262 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../070-python3-py_1.10.0-1_all.deb ... 2024-04-07T15:59:57.273 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../061-ceph-mds_18.2.2-1171-g267c8ff5-1jammy_amd64.deb ... 2024-04-07T15:59:57.301 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-py (1.10.0-1) ... 2024-04-07T15:59:57.312 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking ceph-mds (18.2.2-1171-g267c8ff5-1jammy) ... 2024-04-07T15:59:57.630 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-iniconfig. 2024-04-07T15:59:57.656 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../071-python3-iniconfig_1.1.1-2_all.deb ... 2024-04-07T15:59:57.696 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-04-07T15:59:57.699 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package cephadm. 2024-04-07T15:59:57.726 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../062-cephadm_18.2.2-1171-g267c8ff5-1jammy_amd64.deb ... 2024-04-07T15:59:57.765 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking cephadm (18.2.2-1171-g267c8ff5-1jammy) ... 2024-04-07T15:59:57.958 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-toml. 2024-04-07T15:59:57.984 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../072-python3-toml_0.10.2-1_all.deb ... 2024-04-07T15:59:58.016 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-04-07T15:59:58.077 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-asyncssh. 2024-04-07T15:59:58.104 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../063-python3-asyncssh_2.5.0-1_all.deb ... 2024-04-07T15:59:58.143 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-04-07T15:59:58.294 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-pytest. 2024-04-07T15:59:58.321 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../073-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-04-07T15:59:58.360 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-04-07T15:59:58.429 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python-babel-localedata. 2024-04-07T15:59:58.445 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../064-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-04-07T15:59:58.488 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-04-07T15:59:58.698 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-repoze.lru. 2024-04-07T15:59:58.725 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../074-python3-repoze.lru_0.7-2_all.deb ... 2024-04-07T15:59:58.756 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-04-07T15:59:59.059 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-routes. 2024-04-07T15:59:59.085 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../075-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-04-07T15:59:59.125 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-04-07T15:59:59.451 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-babel. 2024-04-07T15:59:59.454 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-04-07T15:59:59.466 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../065-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-04-07T15:59:59.481 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../076-ceph-mgr-dashboard_18.2.2-1171-g267c8ff5-1jammy_all.deb ... 2024-04-07T15:59:59.509 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-04-07T15:59:59.520 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking ceph-mgr-dashboard (18.2.2-1171-g267c8ff5-1jammy) ... 2024-04-07T15:59:59.821 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-jinja2. 2024-04-07T15:59:59.848 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../066-python3-jinja2_3.0.3-1ubuntu0.1_all.deb ... 2024-04-07T15:59:59.887 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-04-07T16:00:00.265 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-04-07T16:00:00.293 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../067-ceph-mgr-cephadm_18.2.2-1171-g267c8ff5-1jammy_all.deb ... 2024-04-07T16:00:00.341 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking ceph-mgr-cephadm (18.2.2-1171-g267c8ff5-1jammy) ... 2024-04-07T16:00:00.685 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-bcrypt. 2024-04-07T16:00:00.712 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../068-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-04-07T16:00:00.734 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-04-07T16:00:00.750 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../077-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-04-07T16:00:00.752 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-04-07T16:00:00.792 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-04-07T16:00:01.013 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-pluggy. 2024-04-07T16:00:01.028 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../069-python3-pluggy_0.13.0-7.1_all.deb ... 2024-04-07T16:00:01.063 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-04-07T16:00:01.271 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-joblib. 2024-04-07T16:00:01.298 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../078-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-04-07T16:00:01.337 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-04-07T16:00:01.349 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-py. 2024-04-07T16:00:01.376 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../070-python3-py_1.10.0-1_all.deb ... 2024-04-07T16:00:01.416 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-py (1.10.0-1) ... 2024-04-07T16:00:01.691 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-04-07T16:00:01.719 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../079-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-04-07T16:00:01.744 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-iniconfig. 2024-04-07T16:00:01.757 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-04-07T16:00:01.771 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../071-python3-iniconfig_1.1.1-2_all.deb ... 2024-04-07T16:00:01.836 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-04-07T16:00:02.061 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-sklearn. 2024-04-07T16:00:02.088 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../080-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-04-07T16:00:02.127 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-04-07T16:00:02.139 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-toml. 2024-04-07T16:00:02.165 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../072-python3-toml_0.10.2-1_all.deb ... 2024-04-07T16:00:02.205 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-04-07T16:00:02.684 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-pytest. 2024-04-07T16:00:02.710 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../073-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-04-07T16:00:02.750 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-04-07T16:00:02.853 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-04-07T16:00:02.881 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../081-ceph-mgr-diskprediction-local_18.2.2-1171-g267c8ff5-1jammy_all.deb ... 2024-04-07T16:00:02.919 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking ceph-mgr-diskprediction-local (18.2.2-1171-g267c8ff5-1jammy) ... 2024-04-07T16:00:03.095 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-repoze.lru. 2024-04-07T16:00:03.122 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../074-python3-repoze.lru_0.7-2_all.deb ... 2024-04-07T16:00:03.161 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-04-07T16:00:03.439 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-routes. 2024-04-07T16:00:03.455 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../075-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-04-07T16:00:03.489 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-04-07T16:00:03.814 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-cachetools. 2024-04-07T16:00:03.826 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-04-07T16:00:03.841 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../082-python3-cachetools_5.0.0-1_all.deb ... 2024-04-07T16:00:03.853 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../076-ceph-mgr-dashboard_18.2.2-1171-g267c8ff5-1jammy_all.deb ... 2024-04-07T16:00:03.880 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-04-07T16:00:03.892 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking ceph-mgr-dashboard (18.2.2-1171-g267c8ff5-1jammy) ... 2024-04-07T16:00:04.277 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-pyasn1. 2024-04-07T16:00:04.304 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../083-python3-pyasn1_0.4.8-1_all.deb ... 2024-04-07T16:00:04.343 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-04-07T16:00:04.672 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-04-07T16:00:04.699 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../084-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-04-07T16:00:04.738 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-04-07T16:00:05.050 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-rsa. 2024-04-07T16:00:05.072 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-04-07T16:00:05.077 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../085-python3-rsa_4.8-1_all.deb ... 2024-04-07T16:00:05.088 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../077-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-04-07T16:00:05.116 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-rsa (4.8-1) ... 2024-04-07T16:00:05.155 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-04-07T16:00:05.420 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-google-auth. 2024-04-07T16:00:05.447 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../086-python3-google-auth_1.5.1-3_all.deb ... 2024-04-07T16:00:05.478 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-04-07T16:00:05.634 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-joblib. 2024-04-07T16:00:05.661 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../078-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-04-07T16:00:05.701 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-04-07T16:00:05.806 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-04-07T16:00:05.834 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../087-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-04-07T16:00:05.873 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-04-07T16:00:06.046 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-04-07T16:00:06.074 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../079-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-04-07T16:00:06.112 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-04-07T16:00:06.134 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-websocket. 2024-04-07T16:00:06.161 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../088-python3-websocket_1.2.3-1_all.deb ... 2024-04-07T16:00:06.193 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-04-07T16:00:06.415 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-sklearn. 2024-04-07T16:00:06.443 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../080-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-04-07T16:00:06.482 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-04-07T16:00:06.487 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-kubernetes. 2024-04-07T16:00:06.515 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../089-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-04-07T16:00:06.554 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-04-07T16:00:07.184 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-04-07T16:00:07.212 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../081-ceph-mgr-diskprediction-local_18.2.2-1171-g267c8ff5-1jammy_all.deb ... 2024-04-07T16:00:07.243 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-04-07T16:00:07.250 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking ceph-mgr-diskprediction-local (18.2.2-1171-g267c8ff5-1jammy) ... 2024-04-07T16:00:07.260 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../090-ceph-mgr-k8sevents_18.2.2-1171-g267c8ff5-1jammy_all.deb ... 2024-04-07T16:00:07.301 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking ceph-mgr-k8sevents (18.2.2-1171-g267c8ff5-1jammy) ... 2024-04-07T16:00:07.554 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package libonig5:amd64. 2024-04-07T16:00:07.582 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../091-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-04-07T16:00:07.612 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-04-07T16:00:07.941 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package libjq1:amd64. 2024-04-07T16:00:07.968 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../092-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-04-07T16:00:08.007 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-04-07T16:00:08.262 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-cachetools. 2024-04-07T16:00:08.268 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package jq. 2024-04-07T16:00:08.284 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../093-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-04-07T16:00:08.290 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../082-python3-cachetools_5.0.0-1_all.deb ... 2024-04-07T16:00:08.326 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-04-07T16:00:08.328 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-04-07T16:00:08.571 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package socat. 2024-04-07T16:00:08.587 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../094-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-04-07T16:00:08.620 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-04-07T16:00:08.716 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-pyasn1. 2024-04-07T16:00:08.744 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../083-python3-pyasn1_0.4.8-1_all.deb ... 2024-04-07T16:00:08.783 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-04-07T16:00:08.935 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package xmlstarlet. 2024-04-07T16:00:08.962 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../095-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-04-07T16:00:09.001 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-04-07T16:00:09.095 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-04-07T16:00:09.122 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../084-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-04-07T16:00:09.161 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-04-07T16:00:09.338 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package ceph-test. 2024-04-07T16:00:09.354 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../096-ceph-test_18.2.2-1171-g267c8ff5-1jammy_amd64.deb ... 2024-04-07T16:00:09.396 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking ceph-test (18.2.2-1171-g267c8ff5-1jammy) ... 2024-04-07T16:00:09.489 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-rsa. 2024-04-07T16:00:09.517 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../085-python3-rsa_4.8-1_all.deb ... 2024-04-07T16:00:09.556 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-rsa (4.8-1) ... 2024-04-07T16:00:09.868 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-google-auth. 2024-04-07T16:00:09.895 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../086-python3-google-auth_1.5.1-3_all.deb ... 2024-04-07T16:00:09.934 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-04-07T16:00:10.288 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-04-07T16:00:10.316 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../087-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-04-07T16:00:10.354 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-04-07T16:00:10.666 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-websocket. 2024-04-07T16:00:10.693 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../088-python3-websocket_1.2.3-1_all.deb ... 2024-04-07T16:00:10.732 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-04-07T16:00:11.044 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-kubernetes. 2024-04-07T16:00:11.072 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../089-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-04-07T16:00:11.110 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-04-07T16:00:11.276 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package ceph-volume. 2024-04-07T16:00:11.304 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../097-ceph-volume_18.2.2-1171-g267c8ff5-1jammy_all.deb ... 2024-04-07T16:00:11.342 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking ceph-volume (18.2.2-1171-g267c8ff5-1jammy) ... 2024-04-07T16:00:11.588 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package libcephfs-dev. 2024-04-07T16:00:11.604 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../098-libcephfs-dev_18.2.2-1171-g267c8ff5-1jammy_amd64.deb ... 2024-04-07T16:00:11.638 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking libcephfs-dev (18.2.2-1171-g267c8ff5-1jammy) ... 2024-04-07T16:00:11.824 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-04-07T16:00:11.841 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../090-ceph-mgr-k8sevents_18.2.2-1171-g267c8ff5-1jammy_all.deb ... 2024-04-07T16:00:11.874 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking ceph-mgr-k8sevents (18.2.2-1171-g267c8ff5-1jammy) ... 2024-04-07T16:00:12.169 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package libonig5:amd64. 2024-04-07T16:00:12.196 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../091-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-04-07T16:00:12.216 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package nvme-cli. 2024-04-07T16:00:12.235 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-04-07T16:00:12.244 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../099-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2024-04-07T16:00:12.274 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2024-04-07T16:00:12.672 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package libjq1:amd64. 2024-04-07T16:00:12.700 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../092-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-04-07T16:00:12.712 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-04-07T16:00:12.738 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-04-07T16:00:12.740 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../100-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-04-07T16:00:12.770 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-04-07T16:00:13.182 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-pastescript. 2024-04-07T16:00:13.210 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../101-python3-pastescript_2.0.2-4_all.deb ... 2024-04-07T16:00:13.248 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-04-07T16:00:13.274 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package jq. 2024-04-07T16:00:13.290 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../093-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-04-07T16:00:13.332 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-04-07T16:00:13.552 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-psutil. 2024-04-07T16:00:13.580 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../102-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-04-07T16:00:13.593 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package socat. 2024-04-07T16:00:13.618 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-04-07T16:00:13.621 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../094-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-04-07T16:00:13.660 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-04-07T16:00:13.963 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-pygments. 2024-04-07T16:00:13.988 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package xmlstarlet. 2024-04-07T16:00:13.991 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../103-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-04-07T16:00:14.016 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../095-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-04-07T16:00:14.038 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-04-07T16:00:14.055 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-04-07T16:00:14.409 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package ceph-test. 2024-04-07T16:00:14.437 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../096-ceph-test_18.2.2-1171-g267c8ff5-1jammy_amd64.deb ... 2024-04-07T16:00:14.484 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking ceph-test (18.2.2-1171-g267c8ff5-1jammy) ... 2024-04-07T16:00:14.484 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-pyinotify. 2024-04-07T16:00:14.512 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../104-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-04-07T16:00:14.542 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-04-07T16:00:14.836 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-simplejson. 2024-04-07T16:00:14.865 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../105-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-04-07T16:00:14.903 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-04-07T16:00:15.172 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-04-07T16:00:15.200 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../106-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-04-07T16:00:15.247 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-04-07T16:00:16.043 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package radosgw. 2024-04-07T16:00:16.071 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../107-radosgw_18.2.2-1171-g267c8ff5-1jammy_amd64.deb ... 2024-04-07T16:00:16.109 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking radosgw (18.2.2-1171-g267c8ff5-1jammy) ... 2024-04-07T16:00:16.280 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package ceph-volume. 2024-04-07T16:00:16.293 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../097-ceph-volume_18.2.2-1171-g267c8ff5-1jammy_all.deb ... 2024-04-07T16:00:16.321 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking ceph-volume (18.2.2-1171-g267c8ff5-1jammy) ... 2024-04-07T16:00:16.617 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package libcephfs-dev. 2024-04-07T16:00:16.633 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../098-libcephfs-dev_18.2.2-1171-g267c8ff5-1jammy_amd64.deb ... 2024-04-07T16:00:16.675 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking libcephfs-dev (18.2.2-1171-g267c8ff5-1jammy) ... 2024-04-07T16:00:16.829 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package rbd-fuse. 2024-04-07T16:00:16.845 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../108-rbd-fuse_18.2.2-1171-g267c8ff5-1jammy_amd64.deb ... 2024-04-07T16:00:16.895 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking rbd-fuse (18.2.2-1171-g267c8ff5-1jammy) ... 2024-04-07T16:00:17.020 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package nvme-cli. 2024-04-07T16:00:17.048 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../099-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2024-04-07T16:00:17.087 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2024-04-07T16:00:17.158 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-04-07T16:00:17.408 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-04-07T16:00:17.417 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-04-07T16:00:17.424 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../100-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-04-07T16:00:17.466 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-04-07T16:00:17.543 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2024-04-07T16:00:17.806 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-04-07T16:00:17.878 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-pastescript. 2024-04-07T16:00:17.905 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../101-python3-pastescript_2.0.2-4_all.deb ... 2024-04-07T16:00:17.978 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-04-07T16:00:18.138 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-04-07T16:00:18.498 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-psutil. 2024-04-07T16:00:18.525 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../102-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-04-07T16:00:18.564 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-04-07T16:00:18.591 INFO:teuthology.orchestra.run.smithi144.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-04-07T16:00:18.653 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up cephadm (18.2.2-1171-g267c8ff5-1jammy) ... 2024-04-07T16:00:18.892 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-pygments. 2024-04-07T16:00:18.920 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../103-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-04-07T16:00:18.958 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-04-07T16:00:19.144 INFO:teuthology.orchestra.run.smithi144.stdout:Adding system user cephadm....done 2024-04-07T16:00:19.309 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-04-07T16:00:19.405 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-pyinotify. 2024-04-07T16:00:19.432 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../104-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-04-07T16:00:19.471 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-04-07T16:00:19.586 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-04-07T16:00:19.782 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-simplejson. 2024-04-07T16:00:19.809 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../105-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-04-07T16:00:19.820 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-04-07T16:00:19.848 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-04-07T16:00:19.929 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-04-07T16:00:20.126 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-04-07T16:00:20.142 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../106-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-04-07T16:00:20.164 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-04-07T16:00:20.184 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-04-07T16:00:20.432 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-04-07T16:00:20.566 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-py (1.10.0-1) ... 2024-04-07T16:00:20.868 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-04-07T16:00:20.913 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package radosgw. 2024-04-07T16:00:20.941 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../107-radosgw_18.2.2-1171-g267c8ff5-1jammy_amd64.deb ... 2024-04-07T16:00:20.980 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking radosgw (18.2.2-1171-g267c8ff5-1jammy) ... 2024-04-07T16:00:21.228 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-04-07T16:00:21.463 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-04-07T16:00:21.683 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package rbd-fuse. 2024-04-07T16:00:21.710 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../108-rbd-fuse_18.2.2-1171-g267c8ff5-1jammy_amd64.deb ... 2024-04-07T16:00:21.739 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-04-07T16:00:21.749 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking rbd-fuse (18.2.2-1171-g267c8ff5-1jammy) ... 2024-04-07T16:00:21.990 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-04-07T16:00:22.045 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-04-07T16:00:22.250 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-04-07T16:00:22.295 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-04-07T16:00:22.422 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2024-04-07T16:00:22.517 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-ceph-argparse (18.2.2-1171-g267c8ff5-1jammy) ... 2024-04-07T16:00:22.686 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-04-07T16:00:22.785 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-04-07T16:00:22.957 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-04-07T16:00:23.015 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-04-07T16:00:23.220 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-04-07T16:00:23.321 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-04-07T16:00:23.450 INFO:teuthology.orchestra.run.smithi100.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-04-07T16:00:23.523 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up cephadm (18.2.2-1171-g267c8ff5-1jammy) ... 2024-04-07T16:00:23.673 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-04-07T16:00:23.798 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-04-07T16:00:24.039 INFO:teuthology.orchestra.run.smithi100.stdout:Adding system user cephadm....done 2024-04-07T16:00:24.067 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-04-07T16:00:24.188 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-04-07T16:00:24.351 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-04-07T16:00:24.464 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-04-07T16:00:24.686 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-04-07T16:00:24.715 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-04-07T16:00:24.816 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-04-07T16:00:25.093 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-04-07T16:00:25.329 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-04-07T16:00:25.378 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-04-07T16:00:25.487 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-py (1.10.0-1) ... 2024-04-07T16:00:25.555 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-04-07T16:00:25.681 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-04-07T16:00:25.805 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-04-07T16:00:25.807 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-04-07T16:00:26.117 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-04-07T16:00:26.190 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-04-07T16:00:26.441 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-04-07T16:00:26.501 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-04-07T16:00:26.734 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-04-07T16:00:26.752 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-04-07T16:00:26.986 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-04-07T16:00:27.112 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-04-07T16:00:27.236 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-04-07T16:00:27.380 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-04-07T16:00:27.488 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-ceph-argparse (18.2.2-1171-g267c8ff5-1jammy) ... 2024-04-07T16:00:27.723 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-04-07T16:00:27.755 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-04-07T16:00:27.881 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-04-07T16:00:27.983 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-04-07T16:00:28.109 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-ceph-common (18.2.2-1171-g267c8ff5-1jammy) ... 2024-04-07T16:00:28.241 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-04-07T16:00:28.367 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-04-07T16:00:28.402 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-idna (3.3-1) ... 2024-04-07T16:00:28.736 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-04-07T16:00:29.001 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-04-07T16:00:29.102 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-04-07T16:00:29.345 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-04-07T16:00:29.604 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-04-07T16:00:29.875 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-07T16:00:29.914 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-04-07T16:00:30.010 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-toml (0.10.2-1) ... 2024-04-07T16:00:30.269 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-04-07T16:00:30.387 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-04-07T16:00:30.488 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-04-07T16:00:30.582 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-04-07T16:00:30.772 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-04-07T16:00:30.842 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-04-07T16:00:30.976 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-04-07T16:00:31.024 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-04-07T16:00:31.103 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-04-07T16:00:31.275 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-07T16:00:31.401 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up libfmt8:amd64 (8.1.1+ds1-2) ... 2024-04-07T16:00:31.403 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-04-07T16:00:31.535 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-04-07T16:00:31.788 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-04-07T16:00:31.829 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-04-07T16:00:32.039 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-04-07T16:00:32.063 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-04-07T16:00:32.314 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-04-07T16:00:32.399 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-04-07T16:00:32.581 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-04-07T16:00:32.692 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-04-07T16:00:32.875 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-04-07T16:00:33.018 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-04-07T16:00:33.134 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-04-07T16:00:33.260 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-04-07T16:00:33.261 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-04-07T16:00:33.412 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-ceph-common (18.2.2-1171-g267c8ff5-1jammy) ... 2024-04-07T16:00:33.544 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-04-07T16:00:33.654 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-04-07T16:00:33.705 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-idna (3.3-1) ... 2024-04-07T16:00:33.938 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2024-04-07T16:00:34.039 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-04-07T16:00:34.239 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-04-07T16:00:34.499 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-04-07T16:00:34.866 INFO:teuthology.orchestra.run.smithi144.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-04-07T16:00:35.094 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-04-07T16:00:35.220 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-04-07T16:00:35.245 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-07T16:00:35.380 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-toml (0.10.2-1) ... 2024-04-07T16:00:35.588 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-04-07T16:00:35.639 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-04-07T16:00:35.765 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-04-07T16:00:35.839 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-portend (3.0.0-1) ... 2024-04-07T16:00:35.916 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-04-07T16:00:36.090 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-07T16:00:36.216 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-04-07T16:00:36.218 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-04-07T16:00:36.452 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-04-07T16:00:36.534 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-04-07T16:00:36.695 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-07T16:00:36.821 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up libfmt8:amd64 (8.1.1+ds1-2) ... 2024-04-07T16:00:36.835 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-04-07T16:00:36.947 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-04-07T16:00:37.111 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-04-07T16:00:37.220 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-rsa (4.8-1) ... 2024-04-07T16:00:37.240 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-04-07T16:00:37.463 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-04-07T16:00:37.499 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-04-07T16:00:37.723 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-04-07T16:00:37.767 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-04-07T16:00:37.990 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-04-07T16:00:38.001 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-04-07T16:00:38.244 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-04-07T16:00:38.417 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-04-07T16:00:38.504 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-04-07T16:00:38.630 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-04-07T16:00:38.702 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-04-07T16:00:38.914 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-04-07T16:00:39.037 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-04-07T16:00:39.040 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-04-07T16:00:39.155 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up librados2 (18.2.2-1171-g267c8ff5-1jammy) ... 2024-04-07T16:00:39.306 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up librgw2 (18.2.2-1171-g267c8ff5-1jammy) ... 2024-04-07T16:00:39.341 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2024-04-07T16:00:39.432 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up libsqlite3-mod-ceph (18.2.2-1171-g267c8ff5-1jammy) ... 2024-04-07T16:00:39.558 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up libcephfs2 (18.2.2-1171-g267c8ff5-1jammy) ... 2024-04-07T16:00:39.617 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-04-07T16:00:39.684 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up libradosstriper1 (18.2.2-1171-g267c8ff5-1jammy) ... 2024-04-07T16:00:39.860 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-04-07T16:00:39.870 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-04-07T16:00:40.122 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up librbd1 (18.2.2-1171-g267c8ff5-1jammy) ... 2024-04-07T16:00:40.248 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up ceph-mgr-modules-core (18.2.2-1171-g267c8ff5-1jammy) ... 2024-04-07T16:00:40.255 INFO:teuthology.orchestra.run.smithi100.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-04-07T16:00:40.383 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up ceph-fuse (18.2.2-1171-g267c8ff5-1jammy) ... 2024-04-07T16:00:40.505 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-04-07T16:00:40.597 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-04-07T16:00:40.598 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-04-07T16:00:40.631 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-04-07T16:00:41.016 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-04-07T16:00:41.109 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up libcephfs-dev (18.2.2-1171-g267c8ff5-1jammy) ... 2024-04-07T16:00:41.235 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-rados (18.2.2-1171-g267c8ff5-1jammy) ... 2024-04-07T16:00:41.267 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-portend (3.0.0-1) ... 2024-04-07T16:00:41.353 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-04-07T16:00:41.501 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-07T16:00:41.619 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-04-07T16:00:41.928 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-04-07T16:00:42.221 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-04-07T16:00:42.489 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-04-07T16:00:42.566 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-rbd (18.2.2-1171-g267c8ff5-1jammy) ... 2024-04-07T16:00:42.615 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-rsa (4.8-1) ... 2024-04-07T16:00:42.692 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up rbd-fuse (18.2.2-1171-g267c8ff5-1jammy) ... 2024-04-07T16:00:42.818 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-rgw (18.2.2-1171-g267c8ff5-1jammy) ... 2024-04-07T16:00:42.883 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-04-07T16:00:43.134 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-04-07T16:00:43.277 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-cephfs (18.2.2-1171-g267c8ff5-1jammy) ... 2024-04-07T16:00:43.385 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-04-07T16:00:43.403 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up ceph-common (18.2.2-1171-g267c8ff5-1jammy) ... 2024-04-07T16:00:43.745 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-04-07T16:00:43.947 INFO:teuthology.orchestra.run.smithi144.stdout:Adding group ceph....done 2024-04-07T16:00:44.154 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-04-07T16:00:44.289 INFO:teuthology.orchestra.run.smithi144.stdout:Adding system user ceph....done 2024-04-07T16:00:44.416 INFO:teuthology.orchestra.run.smithi144.stdout:Setting system user ceph properties....done 2024-04-07T16:00:44.435 INFO:teuthology.orchestra.run.smithi144.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-04-07T16:00:44.578 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-04-07T16:00:44.590 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-04-07T16:00:44.716 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up librados2 (18.2.2-1171-g267c8ff5-1jammy) ... 2024-04-07T16:00:44.842 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up librgw2 (18.2.2-1171-g267c8ff5-1jammy) ... 2024-04-07T16:00:44.916 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-04-07T16:00:44.943 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up libsqlite3-mod-ceph (18.2.2-1171-g267c8ff5-1jammy) ... 2024-04-07T16:00:45.061 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up libcephfs2 (18.2.2-1171-g267c8ff5-1jammy) ... 2024-04-07T16:00:45.187 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up libradosstriper1 (18.2.2-1171-g267c8ff5-1jammy) ... 2024-04-07T16:00:45.390 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-04-07T16:00:45.421 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up ceph-test (18.2.2-1171-g267c8ff5-1jammy) ... 2024-04-07T16:00:45.547 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up radosgw (18.2.2-1171-g267c8ff5-1jammy) ... 2024-04-07T16:00:45.666 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up librbd1 (18.2.2-1171-g267c8ff5-1jammy) ... 2024-04-07T16:00:45.767 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up ceph-mgr-modules-core (18.2.2-1171-g267c8ff5-1jammy) ... 2024-04-07T16:00:45.877 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up ceph-fuse (18.2.2-1171-g267c8ff5-1jammy) ... 2024-04-07T16:00:46.019 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-04-07T16:00:46.019 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-04-07T16:00:46.087 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-04-07T16:00:46.088 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-04-07T16:00:46.467 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up ceph-base (18.2.2-1171-g267c8ff5-1jammy) ... 2024-04-07T16:00:46.562 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up libcephfs-dev (18.2.2-1171-g267c8ff5-1jammy) ... 2024-04-07T16:00:46.688 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-rados (18.2.2-1171-g267c8ff5-1jammy) ... 2024-04-07T16:00:46.809 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-04-07T16:00:46.814 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-04-07T16:00:47.371 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up ceph-mds (18.2.2-1171-g267c8ff5-1jammy) ... 2024-04-07T16:00:47.711 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-04-07T16:00:47.711 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-04-07T16:00:48.068 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-rbd (18.2.2-1171-g267c8ff5-1jammy) ... 2024-04-07T16:00:48.194 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up rbd-fuse (18.2.2-1171-g267c8ff5-1jammy) ... 2024-04-07T16:00:48.205 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up ceph-mgr (18.2.2-1171-g267c8ff5-1jammy) ... 2024-04-07T16:00:48.320 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-rgw (18.2.2-1171-g267c8ff5-1jammy) ... 2024-04-07T16:00:48.427 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-04-07T16:00:48.428 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-04-07T16:00:48.446 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-cephfs (18.2.2-1171-g267c8ff5-1jammy) ... 2024-04-07T16:00:48.572 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up ceph-common (18.2.2-1171-g267c8ff5-1jammy) ... 2024-04-07T16:00:48.915 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up ceph-osd (18.2.2-1171-g267c8ff5-1jammy) ... 2024-04-07T16:00:49.140 INFO:teuthology.orchestra.run.smithi100.stdout:Adding group ceph....done 2024-04-07T16:00:49.194 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-04-07T16:00:49.194 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-04-07T16:00:49.675 INFO:teuthology.orchestra.run.smithi100.stdout:Adding system user ceph....done 2024-04-07T16:00:49.700 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up ceph-mgr-k8sevents (18.2.2-1171-g267c8ff5-1jammy) ... 2024-04-07T16:00:49.826 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up ceph-mgr-diskprediction-local (18.2.2-1171-g267c8ff5-1jammy) ... 2024-04-07T16:00:49.827 INFO:teuthology.orchestra.run.smithi100.stdout:Setting system user ceph properties....done 2024-04-07T16:00:49.846 INFO:teuthology.orchestra.run.smithi100.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-04-07T16:00:49.977 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up ceph-mon (18.2.2-1171-g267c8ff5-1jammy) ... 2024-04-07T16:00:49.981 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-04-07T16:00:50.171 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-04-07T16:00:50.171 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-04-07T16:00:50.299 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-04-07T16:00:50.662 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up ceph-mgr-cephadm (18.2.2-1171-g267c8ff5-1jammy) ... 2024-04-07T16:00:50.788 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up ceph (18.2.2-1171-g267c8ff5-1jammy) ... 2024-04-07T16:00:50.790 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up ceph-test (18.2.2-1171-g267c8ff5-1jammy) ... 2024-04-07T16:00:50.891 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up radosgw (18.2.2-1171-g267c8ff5-1jammy) ... 2024-04-07T16:00:50.922 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up ceph-mgr-dashboard (18.2.2-1171-g267c8ff5-1jammy) ... 2024-04-07T16:00:51.081 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up ceph-volume (18.2.2-1171-g267c8ff5-1jammy) ... 2024-04-07T16:00:51.342 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-04-07T16:00:51.342 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-04-07T16:00:51.435 INFO:teuthology.orchestra.run.smithi144.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-04-07T16:00:51.810 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up ceph-base (18.2.2-1171-g267c8ff5-1jammy) ... 2024-04-07T16:00:52.127 INFO:teuthology.orchestra.run.smithi144.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-04-07T16:00:52.162 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-04-07T16:00:52.597 INFO:teuthology.orchestra.run.smithi144.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-04-07T16:00:52.598 DEBUG:teuthology.orchestra.run.smithi144:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install python3-pytest 2024-04-07T16:00:52.637 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up ceph-mds (18.2.2-1171-g267c8ff5-1jammy) ... 2024-04-07T16:00:52.693 INFO:teuthology.orchestra.run.smithi144.stdout:Reading package lists... 2024-04-07T16:00:52.863 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-04-07T16:00:52.864 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-04-07T16:00:52.892 INFO:teuthology.orchestra.run.smithi144.stdout:Building dependency tree... 2024-04-07T16:00:52.893 INFO:teuthology.orchestra.run.smithi144.stdout:Reading state information... 2024-04-07T16:00:53.069 INFO:teuthology.orchestra.run.smithi144.stdout:python3-pytest is already the newest version (6.2.5-1ubuntu2). 2024-04-07T16:00:53.069 INFO:teuthology.orchestra.run.smithi144.stdout:python3-pytest set to manually installed. 2024-04-07T16:00:53.069 INFO:teuthology.orchestra.run.smithi144.stdout:The following packages were automatically installed and are no longer required: 2024-04-07T16:00:53.069 INFO:teuthology.orchestra.run.smithi144.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-07T16:00:53.069 INFO:teuthology.orchestra.run.smithi144.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-07T16:00:53.069 INFO:teuthology.orchestra.run.smithi144.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-07T16:00:53.070 INFO:teuthology.orchestra.run.smithi144.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-07T16:00:53.070 INFO:teuthology.orchestra.run.smithi144.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-07T16:00:53.070 INFO:teuthology.orchestra.run.smithi144.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-07T16:00:53.117 INFO:teuthology.orchestra.run.smithi144.stdout:0 upgraded, 0 newly installed, 0 to remove and 299 not upgraded. 2024-04-07T16:00:53.117 INFO:teuthology.orchestra.run.smithi144.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-04-07T16:00:53.121 DEBUG:teuthology.parallel:result is None 2024-04-07T16:00:53.305 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up ceph-mgr (18.2.2-1171-g267c8ff5-1jammy) ... 2024-04-07T16:00:53.519 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-04-07T16:00:53.519 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-04-07T16:00:54.015 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up ceph-osd (18.2.2-1171-g267c8ff5-1jammy) ... 2024-04-07T16:00:54.291 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-04-07T16:00:54.291 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-04-07T16:00:54.741 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up ceph-mgr-k8sevents (18.2.2-1171-g267c8ff5-1jammy) ... 2024-04-07T16:00:54.860 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up ceph-mgr-diskprediction-local (18.2.2-1171-g267c8ff5-1jammy) ... 2024-04-07T16:00:55.002 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up ceph-mon (18.2.2-1171-g267c8ff5-1jammy) ... 2024-04-07T16:00:55.209 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-04-07T16:00:55.209 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-04-07T16:00:55.695 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up ceph-mgr-cephadm (18.2.2-1171-g267c8ff5-1jammy) ... 2024-04-07T16:00:55.838 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up ceph (18.2.2-1171-g267c8ff5-1jammy) ... 2024-04-07T16:00:55.964 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up ceph-mgr-dashboard (18.2.2-1171-g267c8ff5-1jammy) ... 2024-04-07T16:00:56.115 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up ceph-volume (18.2.2-1171-g267c8ff5-1jammy) ... 2024-04-07T16:00:56.460 INFO:teuthology.orchestra.run.smithi100.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-04-07T16:00:57.588 INFO:teuthology.orchestra.run.smithi100.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-04-07T16:00:58.058 INFO:teuthology.orchestra.run.smithi100.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-04-07T16:00:58.061 DEBUG:teuthology.orchestra.run.smithi100:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install python3-pytest 2024-04-07T16:00:58.116 INFO:teuthology.orchestra.run.smithi100.stdout:Reading package lists... 2024-04-07T16:00:58.314 INFO:teuthology.orchestra.run.smithi100.stdout:Building dependency tree... 2024-04-07T16:00:58.315 INFO:teuthology.orchestra.run.smithi100.stdout:Reading state information... 2024-04-07T16:00:58.488 INFO:teuthology.orchestra.run.smithi100.stdout:python3-pytest is already the newest version (6.2.5-1ubuntu2). 2024-04-07T16:00:58.488 INFO:teuthology.orchestra.run.smithi100.stdout:python3-pytest set to manually installed. 2024-04-07T16:00:58.488 INFO:teuthology.orchestra.run.smithi100.stdout:The following packages were automatically installed and are no longer required: 2024-04-07T16:00:58.488 INFO:teuthology.orchestra.run.smithi100.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-07T16:00:58.488 INFO:teuthology.orchestra.run.smithi100.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-07T16:00:58.488 INFO:teuthology.orchestra.run.smithi100.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-07T16:00:58.489 INFO:teuthology.orchestra.run.smithi100.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-07T16:00:58.489 INFO:teuthology.orchestra.run.smithi100.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-07T16:00:58.489 INFO:teuthology.orchestra.run.smithi100.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-07T16:00:58.536 INFO:teuthology.orchestra.run.smithi100.stdout:0 upgraded, 0 newly installed, 0 to remove and 299 not upgraded. 2024-04-07T16:00:58.536 INFO:teuthology.orchestra.run.smithi100.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-04-07T16:00:58.540 DEBUG:teuthology.parallel:result is None 2024-04-07T16:00:58.540 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-07T16:00:58.540 INFO:teuthology.packaging:ref: None 2024-04-07T16:00:58.540 INFO:teuthology.packaging:tag: None 2024-04-07T16:00:58.540 INFO:teuthology.packaging:branch: reef 2024-04-07T16:00:58.540 INFO:teuthology.packaging:sha1: adce92d056533f22dc1ee37e3141d098b046ec1b 2024-04-07T16:00:58.541 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=reef 2024-04-07T16:00:59.051 DEBUG:teuthology.orchestra.run.smithi100:> dpkg-query -W -f '${Version}' ceph 2024-04-07T16:00:59.069 INFO:teuthology.orchestra.run.smithi100.stdout:18.2.2-1171-g267c8ff5-1jammy 2024-04-07T16:00:59.070 INFO:teuthology.packaging:The installed version of ceph is 18.2.2-1171-g267c8ff5-1jammy 2024-04-07T16:00:59.070 INFO:teuthology.task.install:The correct ceph version 18.2.2-1171-g267c8ff5-1jammy is installed. 2024-04-07T16:00:59.072 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-07T16:00:59.072 INFO:teuthology.packaging:ref: None 2024-04-07T16:00:59.072 INFO:teuthology.packaging:tag: None 2024-04-07T16:00:59.072 INFO:teuthology.packaging:branch: reef 2024-04-07T16:00:59.072 INFO:teuthology.packaging:sha1: adce92d056533f22dc1ee37e3141d098b046ec1b 2024-04-07T16:00:59.072 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=reef 2024-04-07T16:00:59.562 DEBUG:teuthology.orchestra.run.smithi144:> dpkg-query -W -f '${Version}' ceph 2024-04-07T16:00:59.581 INFO:teuthology.orchestra.run.smithi144.stdout:18.2.2-1171-g267c8ff5-1jammy 2024-04-07T16:00:59.582 INFO:teuthology.packaging:The installed version of ceph is 18.2.2-1171-g267c8ff5-1jammy 2024-04-07T16:00:59.582 INFO:teuthology.task.install:The correct ceph version 18.2.2-1171-g267c8ff5-1jammy is installed. 2024-04-07T16:00:59.583 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-07T16:00:59.584 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-07T16:00:59.584 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-07T16:00:59.597 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-04-07T16:00:59.597 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-07T16:00:59.637 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-07T16:00:59.638 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-07T16:00:59.638 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/usr/bin/daemon-helper 2024-04-07T16:00:59.651 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-07T16:00:59.707 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-04-07T16:00:59.707 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/usr/bin/daemon-helper 2024-04-07T16:00:59.719 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-07T16:00:59.772 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-07T16:00:59.772 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-07T16:00:59.772 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-07T16:00:59.785 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-07T16:00:59.840 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-04-07T16:00:59.840 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-07T16:00:59.853 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-07T16:00:59.908 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-07T16:00:59.908 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-07T16:00:59.908 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/usr/bin/stdin-killer 2024-04-07T16:00:59.922 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-07T16:00:59.976 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-04-07T16:00:59.976 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/usr/bin/stdin-killer 2024-04-07T16:00:59.988 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-07T16:01:00.043 INFO:teuthology.run_tasks:Running task print... 2024-04-07T16:01:00.052 INFO:teuthology.task.print:**** done install task... 2024-04-07T16:01:00.052 INFO:teuthology.run_tasks:Running task print... 2024-04-07T16:01:00.059 INFO:teuthology.task.print:**** done start installing reef cephadm ... 2024-04-07T16:01:00.059 INFO:teuthology.run_tasks:Running task cephadm... 2024-04-07T16:01:00.175 INFO:tasks.cephadm:Config: {'compiled_cephadm_branch': 'reef', 'conf': {'osd': {'osd_class_default_list': '*', 'osd_class_load_list': '*', 'debug ms': 1, 'debug osd': 20, 'osd shutdown pgref assert': True}, 'global': {'mon election default strategy': 1}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}}, 'image': 'quay.ceph.io/ceph-ci/ceph:reef', 'create_rbd_pool': True, 'flavor': 'default', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'sha1': 'adce92d056533f22dc1ee37e3141d098b046ec1b'} 2024-04-07T16:01:00.175 INFO:tasks.cephadm:Cluster image is quay.ceph.io/ceph-ci/ceph:reef 2024-04-07T16:01:00.176 INFO:tasks.cephadm:Cluster fsid is 0702e3a8-f4f8-11ee-b649-cb9ed24678a4 2024-04-07T16:01:00.176 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-04-07T16:01:00.176 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.100', 'mon.c': '[v2:172.21.15.100:3301,v1:172.21.15.100:6790]', 'mon.b': '172.21.15.144'} 2024-04-07T16:01:00.176 INFO:tasks.cephadm:First mon is mon.a on smithi100 2024-04-07T16:01:00.176 INFO:tasks.cephadm:First mgr is y 2024-04-07T16:01:00.176 INFO:tasks.cephadm:Normalizing hostnames... 2024-04-07T16:01:00.176 DEBUG:teuthology.orchestra.run.smithi100:> sudo hostname $(hostname -s) 2024-04-07T16:01:00.192 DEBUG:teuthology.orchestra.run.smithi144:> sudo hostname $(hostname -s) 2024-04-07T16:01:00.206 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra for reef 2024-04-07T16:01:00.206 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=adce92d056533f22dc1ee37e3141d098b046ec1b 2024-04-07T16:01:00.350 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'adce92d056533f22dc1ee37e3141d098b046ec1b', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=jammy,DIST=jammy,MACHINE_SIZE=gigantic/78309/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-2785-gadce92d0', 'node_name': '172.21.2.12+braggi12', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=jammy,DIST=jammy,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-2785-gadce92d0-1jammy'}, 'url': 'https://3.chacra.ceph.com/r/ceph/wip-yuri3-testing-2024-04-05-0825/adce92d056533f22dc1ee37e3141d098b046ec1b/ubuntu/jammy/flavors/default/', 'distro_codename': 'jammy', 'modified': '2024-04-06 02:32:48.617427', 'distro_version': '22.04', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-yuri3-testing-2024-04-05-0825', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/wip-yuri3-testing-2024-04-05-0825/adce92d056533f22dc1ee37e3141d098b046ec1b/ubuntu/jammy/flavors/default/', 'archs': ['x86_64'], 'distro': 'ubuntu'}] 2024-04-07T16:01:00.644 INFO:tasks.util.chacra:got chacra host 2.chacra.ceph.com, ref reef, sha1 267c8ff5b1ff5815581b3b2381d19360e081b37b from https://shaman.ceph.com/api/search/?project=ceph&distros=ubuntu%2F22.04%2Fx86_64&flavor=default&ref=reef 2024-04-07T16:01:00.646 INFO:tasks.cephadm:Discovered cachra url: https://2.chacra.ceph.com/binaries/ceph/reef/267c8ff5b1ff5815581b3b2381d19360e081b37b/ubuntu/jammy/x86_64/flavors/default/cephadm 2024-04-07T16:01:00.646 DEBUG:teuthology.orchestra.run.smithi100:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/reef/267c8ff5b1ff5815581b3b2381d19360e081b37b/ubuntu/jammy/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-07T16:01:01.163 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-rw-r-- 1 ubuntu ubuntu 216836 Apr 7 16:01 /home/ubuntu/cephtest/cephadm 2024-04-07T16:01:01.163 DEBUG:teuthology.orchestra.run.smithi144:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/reef/267c8ff5b1ff5815581b3b2381d19360e081b37b/ubuntu/jammy/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-07T16:01:01.627 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-rw-r-- 1 ubuntu ubuntu 216836 Apr 7 16:01 /home/ubuntu/cephtest/cephadm 2024-04-07T16:01:01.627 DEBUG:teuthology.orchestra.run.smithi100:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-07T16:01:01.635 DEBUG:teuthology.orchestra.run.smithi144:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-07T16:01:01.661 INFO:tasks.cephadm:Pulling image quay.ceph.io/ceph-ci/ceph:reef on all hosts... 2024-04-07T16:01:01.661 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef pull 2024-04-07T16:01:01.681 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef pull 2024-04-07T16:01:01.796 INFO:teuthology.orchestra.run.smithi100.stderr:Pulling container image quay.ceph.io/ceph-ci/ceph:reef... 2024-04-07T16:01:01.799 INFO:teuthology.orchestra.run.smithi144.stderr:Pulling container image quay.ceph.io/ceph-ci/ceph:reef... 2024-04-07T16:02:08.188 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-04-07T16:02:08.188 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph_version": "ceph version 18.2.2-1171-g267c8ff5 (267c8ff5b1ff5815581b3b2381d19360e081b37b) reef (stable)", 2024-04-07T16:02:08.188 INFO:teuthology.orchestra.run.smithi100.stdout: "image_id": "8a4a01e7186cf27511b5350c5e0ffe2ea18c6322f190fc5b1ff7614b21bec164", 2024-04-07T16:02:08.188 INFO:teuthology.orchestra.run.smithi100.stdout: "repo_digests": [ 2024-04-07T16:02:08.188 INFO:teuthology.orchestra.run.smithi100.stdout: "quay.ceph.io/ceph-ci/ceph@sha256:454b23ea7ae31a2ccc633075d7dca7a328e0d13fe9231772faa01c3d29ed7fc7" 2024-04-07T16:02:08.189 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-04-07T16:02:08.189 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-04-07T16:02:09.564 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-04-07T16:02:09.564 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph_version": "ceph version 18.2.2-1171-g267c8ff5 (267c8ff5b1ff5815581b3b2381d19360e081b37b) reef (stable)", 2024-04-07T16:02:09.564 INFO:teuthology.orchestra.run.smithi144.stdout: "image_id": "8a4a01e7186cf27511b5350c5e0ffe2ea18c6322f190fc5b1ff7614b21bec164", 2024-04-07T16:02:09.564 INFO:teuthology.orchestra.run.smithi144.stdout: "repo_digests": [ 2024-04-07T16:02:09.564 INFO:teuthology.orchestra.run.smithi144.stdout: "quay.ceph.io/ceph-ci/ceph@sha256:454b23ea7ae31a2ccc633075d7dca7a328e0d13fe9231772faa01c3d29ed7fc7" 2024-04-07T16:02:09.564 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-04-07T16:02:09.564 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-04-07T16:02:09.593 DEBUG:teuthology.orchestra.run.smithi100:> sudo mkdir -p /etc/ceph 2024-04-07T16:02:09.607 DEBUG:teuthology.orchestra.run.smithi144:> sudo mkdir -p /etc/ceph 2024-04-07T16:02:09.621 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod 777 /etc/ceph 2024-04-07T16:02:09.661 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod 777 /etc/ceph 2024-04-07T16:02:09.675 INFO:tasks.cephadm:Writing seed config... 2024-04-07T16:02:09.676 INFO:tasks.cephadm: override: [osd] osd_class_default_list = * 2024-04-07T16:02:09.676 INFO:tasks.cephadm: override: [osd] osd_class_load_list = * 2024-04-07T16:02:09.677 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-04-07T16:02:09.677 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-04-07T16:02:09.677 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-04-07T16:02:09.677 INFO:tasks.cephadm: override: [global] mon election default strategy = 1 2024-04-07T16:02:09.677 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-04-07T16:02:09.677 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-04-07T16:02:09.677 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-04-07T16:02:09.677 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-04-07T16:02:09.677 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-04-07T16:02:09.678 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-07T16:02:09.678 DEBUG:teuthology.orchestra.run.smithi100:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-04-07T16:02:09.707 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 0702e3a8-f4f8-11ee-b649-cb9ed24678a4 mon election default strategy = 1 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true osd_class_default_list = * osd_class_load_list = * debug ms = 1 debug osd = 20 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true 2024-04-07T16:02:09.707 DEBUG:teuthology.orchestra.run.smithi100:mon.a> sudo journalctl -f -n 0 -u ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@mon.a.service 2024-04-07T16:02:09.752 DEBUG:teuthology.orchestra.run.smithi100:mgr.y> sudo journalctl -f -n 0 -u ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@mgr.y.service 2024-04-07T16:02:09.795 INFO:tasks.cephadm:Bootstrapping... 2024-04-07T16:02:09.796 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef -v bootstrap --fsid 0702e3a8-f4f8-11ee-b649-cb9ed24678a4 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id y --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.100 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-04-07T16:02:09.957 INFO:teuthology.orchestra.run.smithi100.stdout:-------------------------------------------------------------------------------- 2024-04-07T16:02:09.957 INFO:teuthology.orchestra.run.smithi100.stdout:cephadm ['--image', 'quay.ceph.io/ceph-ci/ceph:reef', '-v', 'bootstrap', '--fsid', '0702e3a8-f4f8-11ee-b649-cb9ed24678a4', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'y', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.100', '--skip-admin-label'] 2024-04-07T16:02:09.957 INFO:teuthology.orchestra.run.smithi100.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-04-07T16:02:09.958 INFO:teuthology.orchestra.run.smithi100.stdout:Verifying podman|docker is present... 2024-04-07T16:02:09.958 INFO:teuthology.orchestra.run.smithi100.stdout:Verifying lvm2 is present... 2024-04-07T16:02:09.958 INFO:teuthology.orchestra.run.smithi100.stdout:Verifying time synchronization is in place... 2024-04-07T16:02:09.963 INFO:teuthology.orchestra.run.smithi100.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-07T16:02:09.963 INFO:teuthology.orchestra.run.smithi100.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-07T16:02:09.968 INFO:teuthology.orchestra.run.smithi100.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-07T16:02:09.968 INFO:teuthology.orchestra.run.smithi100.stdout:systemctl: stdout inactive 2024-04-07T16:02:09.971 INFO:teuthology.orchestra.run.smithi100.stdout:Non-zero exit code 1 from systemctl is-enabled chronyd.service 2024-04-07T16:02:09.971 INFO:teuthology.orchestra.run.smithi100.stdout:systemctl: stderr Failed to get unit file state for chronyd.service: No such file or directory 2024-04-07T16:02:09.975 INFO:teuthology.orchestra.run.smithi100.stdout:Non-zero exit code 3 from systemctl is-active chronyd.service 2024-04-07T16:02:09.975 INFO:teuthology.orchestra.run.smithi100.stdout:systemctl: stdout inactive 2024-04-07T16:02:09.978 INFO:teuthology.orchestra.run.smithi100.stdout:Non-zero exit code 1 from systemctl is-enabled systemd-timesyncd.service 2024-04-07T16:02:09.978 INFO:teuthology.orchestra.run.smithi100.stdout:systemctl: stdout masked 2024-04-07T16:02:09.982 INFO:teuthology.orchestra.run.smithi100.stdout:Non-zero exit code 3 from systemctl is-active systemd-timesyncd.service 2024-04-07T16:02:09.982 INFO:teuthology.orchestra.run.smithi100.stdout:systemctl: stdout inactive 2024-04-07T16:02:09.986 INFO:teuthology.orchestra.run.smithi100.stdout:Non-zero exit code 1 from systemctl is-enabled ntpd.service 2024-04-07T16:02:09.987 INFO:teuthology.orchestra.run.smithi100.stdout:systemctl: stderr Failed to get unit file state for ntpd.service: No such file or directory 2024-04-07T16:02:09.990 INFO:teuthology.orchestra.run.smithi100.stdout:Non-zero exit code 3 from systemctl is-active ntpd.service 2024-04-07T16:02:09.990 INFO:teuthology.orchestra.run.smithi100.stdout:systemctl: stdout inactive 2024-04-07T16:02:09.994 INFO:teuthology.orchestra.run.smithi100.stdout:systemctl: stdout enabled 2024-04-07T16:02:09.998 INFO:teuthology.orchestra.run.smithi100.stdout:systemctl: stdout active 2024-04-07T16:02:09.998 INFO:teuthology.orchestra.run.smithi100.stdout:Unit ntp.service is enabled and running 2024-04-07T16:02:09.998 INFO:teuthology.orchestra.run.smithi100.stdout:Repeating the final host check... 2024-04-07T16:02:09.998 INFO:teuthology.orchestra.run.smithi100.stdout:docker (/usr/bin/docker) is present 2024-04-07T16:02:09.998 INFO:teuthology.orchestra.run.smithi100.stdout:systemctl is present 2024-04-07T16:02:09.998 INFO:teuthology.orchestra.run.smithi100.stdout:lvcreate is present 2024-04-07T16:02:10.001 INFO:teuthology.orchestra.run.smithi100.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-07T16:02:10.001 INFO:teuthology.orchestra.run.smithi100.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-07T16:02:10.006 INFO:teuthology.orchestra.run.smithi100.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-07T16:02:10.006 INFO:teuthology.orchestra.run.smithi100.stdout:systemctl: stdout inactive 2024-04-07T16:02:10.009 INFO:teuthology.orchestra.run.smithi100.stdout:Non-zero exit code 1 from systemctl is-enabled chronyd.service 2024-04-07T16:02:10.009 INFO:teuthology.orchestra.run.smithi100.stdout:systemctl: stderr Failed to get unit file state for chronyd.service: No such file or directory 2024-04-07T16:02:10.013 INFO:teuthology.orchestra.run.smithi100.stdout:Non-zero exit code 3 from systemctl is-active chronyd.service 2024-04-07T16:02:10.013 INFO:teuthology.orchestra.run.smithi100.stdout:systemctl: stdout inactive 2024-04-07T16:02:10.017 INFO:teuthology.orchestra.run.smithi100.stdout:Non-zero exit code 1 from systemctl is-enabled systemd-timesyncd.service 2024-04-07T16:02:10.017 INFO:teuthology.orchestra.run.smithi100.stdout:systemctl: stdout masked 2024-04-07T16:02:10.021 INFO:teuthology.orchestra.run.smithi100.stdout:Non-zero exit code 3 from systemctl is-active systemd-timesyncd.service 2024-04-07T16:02:10.021 INFO:teuthology.orchestra.run.smithi100.stdout:systemctl: stdout inactive 2024-04-07T16:02:10.025 INFO:teuthology.orchestra.run.smithi100.stdout:Non-zero exit code 1 from systemctl is-enabled ntpd.service 2024-04-07T16:02:10.026 INFO:teuthology.orchestra.run.smithi100.stdout:systemctl: stderr Failed to get unit file state for ntpd.service: No such file or directory 2024-04-07T16:02:10.029 INFO:teuthology.orchestra.run.smithi100.stdout:Non-zero exit code 3 from systemctl is-active ntpd.service 2024-04-07T16:02:10.029 INFO:teuthology.orchestra.run.smithi100.stdout:systemctl: stdout inactive 2024-04-07T16:02:10.034 INFO:teuthology.orchestra.run.smithi100.stdout:systemctl: stdout enabled 2024-04-07T16:02:10.039 INFO:teuthology.orchestra.run.smithi100.stdout:systemctl: stdout active 2024-04-07T16:02:10.040 INFO:teuthology.orchestra.run.smithi100.stdout:Unit ntp.service is enabled and running 2024-04-07T16:02:10.040 INFO:teuthology.orchestra.run.smithi100.stdout:Host looks OK 2024-04-07T16:02:10.040 INFO:teuthology.orchestra.run.smithi100.stdout:Cluster fsid: 0702e3a8-f4f8-11ee-b649-cb9ed24678a4 2024-04-07T16:02:10.040 INFO:teuthology.orchestra.run.smithi100.stdout:Acquiring lock 140692689470944 on /run/cephadm/0702e3a8-f4f8-11ee-b649-cb9ed24678a4.lock 2024-04-07T16:02:10.040 INFO:teuthology.orchestra.run.smithi100.stdout:Lock 140692689470944 acquired on /run/cephadm/0702e3a8-f4f8-11ee-b649-cb9ed24678a4.lock 2024-04-07T16:02:10.040 INFO:teuthology.orchestra.run.smithi100.stdout:Verifying IP 172.21.15.100 port 3300 ... 2024-04-07T16:02:10.040 INFO:teuthology.orchestra.run.smithi100.stdout:Verifying IP 172.21.15.100 port 6789 ... 2024-04-07T16:02:10.040 INFO:teuthology.orchestra.run.smithi100.stdout:Base mon IP(s) is [172.21.15.100:3300, 172.21.15.100:6789], mon addrv is [v2:172.21.15.100:3300,v1:172.21.15.100:6789] 2024-04-07T16:02:10.042 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 2024-04-07T16:02:10.043 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.100 metric 100 2024-04-07T16:02:10.043 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/sbin/ip: stdout 158.69.67.47 via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.100 metric 100 2024-04-07T16:02:10.043 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/sbin/ip: stdout 172.17.0.0/16 dev docker0 proto kernel scope link src 172.17.0.1 linkdown 2024-04-07T16:02:10.045 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.100 metric 100 2024-04-07T16:02:10.045 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/sbin/ip: stdout 172.21.0.1 dev enp3s0f1 proto dhcp scope link src 172.21.15.100 metric 100 2024-04-07T16:02:10.045 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/sbin/ip: stdout 172.21.0.2 dev enp3s0f1 proto dhcp scope link src 172.21.15.100 metric 100 2024-04-07T16:02:10.045 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/sbin/ip: stdout 172.21.15.254 dev enp3s0f1 proto dhcp scope link src 172.21.15.100 metric 100 2024-04-07T16:02:10.045 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-04-07T16:02:10.045 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 256 pref medium 2024-04-07T16:02:10.046 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 100 expires 1753sec pref medium 2024-04-07T16:02:10.046 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-04-07T16:02:10.047 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/sbin/ip: stdout inet6 ::1/128 scope host 2024-04-07T16:02:10.047 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-07T16:02:10.047 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-04-07T16:02:10.047 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/sbin/ip: stdout inet6 fe80::ec4:7aff:fe8f:d733/64 scope link 2024-04-07T16:02:10.047 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-07T16:02:10.048 INFO:teuthology.orchestra.run.smithi100.stdout:Mon IP `172.21.15.100` is in CIDR network `172.21.0.0/20` 2024-04-07T16:02:10.048 INFO:teuthology.orchestra.run.smithi100.stdout:Mon IP `172.21.15.100` is in CIDR network `172.21.0.0/20` 2024-04-07T16:02:10.048 INFO:teuthology.orchestra.run.smithi100.stdout:Mon IP `172.21.15.100` is in CIDR network `172.21.0.1/32` 2024-04-07T16:02:10.048 INFO:teuthology.orchestra.run.smithi100.stdout:Mon IP `172.21.15.100` is in CIDR network `172.21.0.1/32` 2024-04-07T16:02:10.048 INFO:teuthology.orchestra.run.smithi100.stdout:Mon IP `172.21.15.100` is in CIDR network `172.21.0.2/32` 2024-04-07T16:02:10.048 INFO:teuthology.orchestra.run.smithi100.stdout:Mon IP `172.21.15.100` is in CIDR network `172.21.0.2/32` 2024-04-07T16:02:10.048 INFO:teuthology.orchestra.run.smithi100.stdout:Mon IP `172.21.15.100` is in CIDR network `172.21.15.254/32` 2024-04-07T16:02:10.049 INFO:teuthology.orchestra.run.smithi100.stdout:Mon IP `172.21.15.100` is in CIDR network `172.21.15.254/32` 2024-04-07T16:02:10.049 INFO:teuthology.orchestra.run.smithi100.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20', '172.21.0.1/32', '172.21.0.1/32', '172.21.0.2/32', '172.21.0.2/32', '172.21.15.254/32', '172.21.15.254/32'] 2024-04-07T16:02:10.049 INFO:teuthology.orchestra.run.smithi100.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-04-07T16:02:10.049 INFO:teuthology.orchestra.run.smithi100.stdout:Pulling container image quay.ceph.io/ceph-ci/ceph:reef... 2024-04-07T16:02:10.273 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/docker: stdout reef: Pulling from ceph-ci/ceph 2024-04-07T16:02:10.273 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/docker: stdout Digest: sha256:454b23ea7ae31a2ccc633075d7dca7a328e0d13fe9231772faa01c3d29ed7fc7 2024-04-07T16:02:10.273 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/docker: stdout Status: Image is up to date for quay.ceph.io/ceph-ci/ceph:reef 2024-04-07T16:02:10.273 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/docker: stdout quay.ceph.io/ceph-ci/ceph:reef 2024-04-07T16:02:11.675 INFO:teuthology.orchestra.run.smithi100.stdout:ceph: stdout ceph version 18.2.2-1171-g267c8ff5 (267c8ff5b1ff5815581b3b2381d19360e081b37b) reef (stable) 2024-04-07T16:02:11.675 INFO:teuthology.orchestra.run.smithi100.stdout:Ceph version: ceph version 18.2.2-1171-g267c8ff5 (267c8ff5b1ff5815581b3b2381d19360e081b37b) reef (stable) 2024-04-07T16:02:11.676 INFO:teuthology.orchestra.run.smithi100.stdout:Extracting ceph user uid/gid from container image... 2024-04-07T16:02:12.863 INFO:teuthology.orchestra.run.smithi100.stdout:stat: stdout 167 167 2024-04-07T16:02:12.863 INFO:teuthology.orchestra.run.smithi100.stdout:Creating initial keys... 2024-04-07T16:02:14.101 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-authtool: stdout AQCFwxJmQFSSIBAA4MIlkrnXZFx2DYz+8egbCw== 2024-04-07T16:02:15.323 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-authtool: stdout AQCGwxJmE1k4MhAAl7I3jjihQCF+VZiwioosmw== 2024-04-07T16:02:16.552 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-authtool: stdout AQCIwxJmiRhhARAAfxNVgYhXBg2ehUR6azfhJQ== 2024-04-07T16:02:16.553 INFO:teuthology.orchestra.run.smithi100.stdout:Creating initial monmap... 2024-04-07T16:02:17.833 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-07T16:02:17.833 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = pacific 2024-04-07T16:02:17.833 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 0702e3a8-f4f8-11ee-b649-cb9ed24678a4 2024-04-07T16:02:17.833 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-07T16:02:17.833 INFO:teuthology.orchestra.run.smithi100.stdout:monmaptool for a [v2:172.21.15.100:3300,v1:172.21.15.100:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-07T16:02:17.833 INFO:teuthology.orchestra.run.smithi100.stdout:setting min_mon_release = pacific 2024-04-07T16:02:17.833 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/monmaptool: set fsid to 0702e3a8-f4f8-11ee-b649-cb9ed24678a4 2024-04-07T16:02:17.833 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-07T16:02:17.833 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-07T16:02:17.833 INFO:teuthology.orchestra.run.smithi100.stdout:Creating mon... 2024-04-07T16:02:19.136 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.575+0000 7f53bca04c80 0 set uid:gid to 167:167 (ceph:ceph) 2024-04-07T16:02:19.136 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.575+0000 7f53bca04c80 1 imported monmap: 2024-04-07T16:02:19.136 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr epoch 0 2024-04-07T16:02:19.137 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr fsid 0702e3a8-f4f8-11ee-b649-cb9ed24678a4 2024-04-07T16:02:19.137 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr last_changed 2024-04-07T16:02:17.280630+0000 2024-04-07T16:02:19.137 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr created 2024-04-07T16:02:17.280630+0000 2024-04-07T16:02:19.137 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr min_mon_release 16 (pacific) 2024-04-07T16:02:19.137 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr election_strategy: 1 2024-04-07T16:02:19.137 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr 0: [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] mon.a 2024-04-07T16:02:19.137 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr 2024-04-07T16:02:19.137 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.575+0000 7f53bca04c80 0 /usr/bin/ceph-mon: set fsid to 0702e3a8-f4f8-11ee-b649-cb9ed24678a4 2024-04-07T16:02:19.137 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.575+0000 7f53bca04c80 4 rocksdb: RocksDB version: 7.9.2 2024-04-07T16:02:19.137 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr 2024-04-07T16:02:19.137 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.575+0000 7f53bca04c80 4 rocksdb: Git sha 0 2024-04-07T16:02:19.137 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.575+0000 7f53bca04c80 4 rocksdb: Compile date 2024-04-06 15:05:06 2024-04-07T16:02:19.137 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.575+0000 7f53bca04c80 4 rocksdb: DB SUMMARY 2024-04-07T16:02:19.137 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr 2024-04-07T16:02:19.138 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.575+0000 7f53bca04c80 4 rocksdb: DB Session ID: ZCSBHR41Q0NOIP01DQSN 2024-04-07T16:02:19.138 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr 2024-04-07T16:02:19.138 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.575+0000 7f53bca04c80 4 rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 0, files: 2024-04-07T16:02:19.138 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr 2024-04-07T16:02:19.138 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.575+0000 7f53bca04c80 4 rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 2024-04-07T16:02:19.138 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr 2024-04-07T16:02:19.138 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.575+0000 7f53bca04c80 4 rocksdb: Options.error_if_exists: 0 2024-04-07T16:02:19.138 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.575+0000 7f53bca04c80 4 rocksdb: Options.create_if_missing: 1 2024-04-07T16:02:19.138 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.575+0000 7f53bca04c80 4 rocksdb: Options.paranoid_checks: 1 2024-04-07T16:02:19.138 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.575+0000 7f53bca04c80 4 rocksdb: Options.flush_verify_memtable_count: 1 2024-04-07T16:02:19.138 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.575+0000 7f53bca04c80 4 rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-07T16:02:19.138 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.575+0000 7f53bca04c80 4 rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-07T16:02:19.138 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.575+0000 7f53bca04c80 4 rocksdb: Options.env: 0x564d702897a0 2024-04-07T16:02:19.138 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.575+0000 7f53bca04c80 4 rocksdb: Options.fs: PosixFileSystem 2024-04-07T16:02:19.139 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.575+0000 7f53bca04c80 4 rocksdb: Options.info_log: 0x564d72052740 2024-04-07T16:02:19.139 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.575+0000 7f53bca04c80 4 rocksdb: Options.max_file_opening_threads: 16 2024-04-07T16:02:19.139 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.575+0000 7f53bca04c80 4 rocksdb: Options.statistics: (nil) 2024-04-07T16:02:19.139 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.575+0000 7f53bca04c80 4 rocksdb: Options.use_fsync: 0 2024-04-07T16:02:19.139 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.575+0000 7f53bca04c80 4 rocksdb: Options.max_log_file_size: 0 2024-04-07T16:02:19.139 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.575+0000 7f53bca04c80 4 rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-07T16:02:19.139 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.575+0000 7f53bca04c80 4 rocksdb: Options.log_file_time_to_roll: 0 2024-04-07T16:02:19.139 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.575+0000 7f53bca04c80 4 rocksdb: Options.keep_log_file_num: 1000 2024-04-07T16:02:19.139 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.575+0000 7f53bca04c80 4 rocksdb: Options.recycle_log_file_num: 0 2024-04-07T16:02:19.139 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.575+0000 7f53bca04c80 4 rocksdb: Options.allow_fallocate: 1 2024-04-07T16:02:19.139 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.575+0000 7f53bca04c80 4 rocksdb: Options.allow_mmap_reads: 0 2024-04-07T16:02:19.139 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.575+0000 7f53bca04c80 4 rocksdb: Options.allow_mmap_writes: 0 2024-04-07T16:02:19.139 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.575+0000 7f53bca04c80 4 rocksdb: Options.use_direct_reads: 0 2024-04-07T16:02:19.139 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.575+0000 7f53bca04c80 4 rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-07T16:02:19.139 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.575+0000 7f53bca04c80 4 rocksdb: Options.create_missing_column_families: 0 2024-04-07T16:02:19.140 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.575+0000 7f53bca04c80 4 rocksdb: Options.db_log_dir: 2024-04-07T16:02:19.140 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.575+0000 7f53bca04c80 4 rocksdb: Options.wal_dir: 2024-04-07T16:02:19.140 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.575+0000 7f53bca04c80 4 rocksdb: Options.table_cache_numshardbits: 6 2024-04-07T16:02:19.140 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.575+0000 7f53bca04c80 4 rocksdb: Options.WAL_ttl_seconds: 0 2024-04-07T16:02:19.140 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.575+0000 7f53bca04c80 4 rocksdb: Options.WAL_size_limit_MB: 0 2024-04-07T16:02:19.140 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.575+0000 7f53bca04c80 4 rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-07T16:02:19.140 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.575+0000 7f53bca04c80 4 rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-07T16:02:19.140 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.575+0000 7f53bca04c80 4 rocksdb: Options.is_fd_close_on_exec: 1 2024-04-07T16:02:19.140 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.575+0000 7f53bca04c80 4 rocksdb: Options.advise_random_on_open: 1 2024-04-07T16:02:19.140 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.575+0000 7f53bca04c80 4 rocksdb: Options.db_write_buffer_size: 0 2024-04-07T16:02:19.140 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.575+0000 7f53bca04c80 4 rocksdb: Options.write_buffer_manager: 0x564d712e9180 2024-04-07T16:02:19.140 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.575+0000 7f53bca04c80 4 rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-07T16:02:19.140 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.575+0000 7f53bca04c80 4 rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-07T16:02:19.140 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.575+0000 7f53bca04c80 4 rocksdb: Options.use_adaptive_mutex: 0 2024-04-07T16:02:19.140 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.575+0000 7f53bca04c80 4 rocksdb: Options.rate_limiter: (nil) 2024-04-07T16:02:19.141 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.575+0000 7f53bca04c80 4 rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-07T16:02:19.141 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.575+0000 7f53bca04c80 4 rocksdb: Options.wal_recovery_mode: 2 2024-04-07T16:02:19.141 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.575+0000 7f53bca04c80 4 rocksdb: Options.enable_thread_tracking: 0 2024-04-07T16:02:19.141 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.575+0000 7f53bca04c80 4 rocksdb: Options.enable_pipelined_write: 0 2024-04-07T16:02:19.141 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.575+0000 7f53bca04c80 4 rocksdb: Options.unordered_write: 0 2024-04-07T16:02:19.141 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.575+0000 7f53bca04c80 4 rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-07T16:02:19.141 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.575+0000 7f53bca04c80 4 rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-07T16:02:19.141 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.575+0000 7f53bca04c80 4 rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-07T16:02:19.141 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.575+0000 7f53bca04c80 4 rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-07T16:02:19.141 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.575+0000 7f53bca04c80 4 rocksdb: Options.row_cache: None 2024-04-07T16:02:19.141 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.575+0000 7f53bca04c80 4 rocksdb: Options.wal_filter: None 2024-04-07T16:02:19.141 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.575+0000 7f53bca04c80 4 rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-07T16:02:19.141 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.575+0000 7f53bca04c80 4 rocksdb: Options.allow_ingest_behind: 0 2024-04-07T16:02:19.141 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.575+0000 7f53bca04c80 4 rocksdb: Options.two_write_queues: 0 2024-04-07T16:02:19.141 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.575+0000 7f53bca04c80 4 rocksdb: Options.manual_wal_flush: 0 2024-04-07T16:02:19.142 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.575+0000 7f53bca04c80 4 rocksdb: Options.wal_compression: 0 2024-04-07T16:02:19.142 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.575+0000 7f53bca04c80 4 rocksdb: Options.atomic_flush: 0 2024-04-07T16:02:19.142 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.575+0000 7f53bca04c80 4 rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-07T16:02:19.142 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.575+0000 7f53bca04c80 4 rocksdb: Options.persist_stats_to_disk: 0 2024-04-07T16:02:19.142 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.575+0000 7f53bca04c80 4 rocksdb: Options.write_dbid_to_manifest: 0 2024-04-07T16:02:19.142 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.575+0000 7f53bca04c80 4 rocksdb: Options.log_readahead_size: 0 2024-04-07T16:02:19.142 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.575+0000 7f53bca04c80 4 rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-07T16:02:19.142 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.575+0000 7f53bca04c80 4 rocksdb: Options.best_efforts_recovery: 0 2024-04-07T16:02:19.142 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.575+0000 7f53bca04c80 4 rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-07T16:02:19.142 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.575+0000 7f53bca04c80 4 rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-07T16:02:19.142 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.575+0000 7f53bca04c80 4 rocksdb: Options.allow_data_in_errors: 0 2024-04-07T16:02:19.142 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.575+0000 7f53bca04c80 4 rocksdb: Options.db_host_id: __hostname__ 2024-04-07T16:02:19.142 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.575+0000 7f53bca04c80 4 rocksdb: Options.enforce_single_del_contracts: true 2024-04-07T16:02:19.142 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.575+0000 7f53bca04c80 4 rocksdb: Options.max_background_jobs: 2 2024-04-07T16:02:19.142 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.575+0000 7f53bca04c80 4 rocksdb: Options.max_background_compactions: -1 2024-04-07T16:02:19.143 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.575+0000 7f53bca04c80 4 rocksdb: Options.max_subcompactions: 1 2024-04-07T16:02:19.143 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.575+0000 7f53bca04c80 4 rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-07T16:02:19.143 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.575+0000 7f53bca04c80 4 rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-07T16:02:19.143 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.575+0000 7f53bca04c80 4 rocksdb: Options.delayed_write_rate : 16777216 2024-04-07T16:02:19.143 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.575+0000 7f53bca04c80 4 rocksdb: Options.max_total_wal_size: 0 2024-04-07T16:02:19.143 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.575+0000 7f53bca04c80 4 rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-07T16:02:19.143 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.575+0000 7f53bca04c80 4 rocksdb: Options.stats_dump_period_sec: 600 2024-04-07T16:02:19.143 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.575+0000 7f53bca04c80 4 rocksdb: Options.stats_persist_period_sec: 600 2024-04-07T16:02:19.143 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.575+0000 7f53bca04c80 4 rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-07T16:02:19.143 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.575+0000 7f53bca04c80 4 rocksdb: Options.max_open_files: -1 2024-04-07T16:02:19.143 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.575+0000 7f53bca04c80 4 rocksdb: Options.bytes_per_sync: 0 2024-04-07T16:02:19.143 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.575+0000 7f53bca04c80 4 rocksdb: Options.wal_bytes_per_sync: 0 2024-04-07T16:02:19.143 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.575+0000 7f53bca04c80 4 rocksdb: Options.strict_bytes_per_sync: 0 2024-04-07T16:02:19.143 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.575+0000 7f53bca04c80 4 rocksdb: Options.compaction_readahead_size: 0 2024-04-07T16:02:19.143 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.575+0000 7f53bca04c80 4 rocksdb: Options.max_background_flushes: -1 2024-04-07T16:02:19.143 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.575+0000 7f53bca04c80 4 rocksdb: Compression algorithms supported: 2024-04-07T16:02:19.144 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.575+0000 7f53bca04c80 4 rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-07T16:02:19.144 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.575+0000 7f53bca04c80 4 rocksdb: kZSTD supported: 0 2024-04-07T16:02:19.144 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.575+0000 7f53bca04c80 4 rocksdb: kXpressCompression supported: 0 2024-04-07T16:02:19.144 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.575+0000 7f53bca04c80 4 rocksdb: kLZ4HCCompression supported: 1 2024-04-07T16:02:19.144 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.575+0000 7f53bca04c80 4 rocksdb: kZlibCompression supported: 1 2024-04-07T16:02:19.144 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.575+0000 7f53bca04c80 4 rocksdb: kSnappyCompression supported: 1 2024-04-07T16:02:19.144 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.575+0000 7f53bca04c80 4 rocksdb: kLZ4Compression supported: 1 2024-04-07T16:02:19.144 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.575+0000 7f53bca04c80 4 rocksdb: kBZip2Compression supported: 0 2024-04-07T16:02:19.144 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.575+0000 7f53bca04c80 4 rocksdb: Fast CRC32 supported: Supported on x86 2024-04-07T16:02:19.144 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.575+0000 7f53bca04c80 4 rocksdb: DMutex implementation: pthread_mutex_t 2024-04-07T16:02:19.144 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.579+0000 7f53bca04c80 4 rocksdb: [db/db_impl/db_impl_open.cc:317] Creating manifest 1 2024-04-07T16:02:19.144 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr 2024-04-07T16:02:19.144 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.579+0000 7f53bca04c80 4 rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000001 2024-04-07T16:02:19.144 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr 2024-04-07T16:02:19.144 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.579+0000 7f53bca04c80 4 rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-07T16:02:19.145 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr 2024-04-07T16:02:19.145 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.579+0000 7f53bca04c80 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-07T16:02:19.145 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.579+0000 7f53bca04c80 4 rocksdb: Options.merge_operator: 2024-04-07T16:02:19.145 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.579+0000 7f53bca04c80 4 rocksdb: Options.compaction_filter: None 2024-04-07T16:02:19.145 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.579+0000 7f53bca04c80 4 rocksdb: Options.compaction_filter_factory: None 2024-04-07T16:02:19.145 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.579+0000 7f53bca04c80 4 rocksdb: Options.sst_partitioner_factory: None 2024-04-07T16:02:19.145 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.579+0000 7f53bca04c80 4 rocksdb: Options.memtable_factory: SkipListFactory 2024-04-07T16:02:19.145 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.579+0000 7f53bca04c80 4 rocksdb: Options.table_factory: BlockBasedTable 2024-04-07T16:02:19.145 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.579+0000 7f53bca04c80 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x564d72052860) 2024-04-07T16:02:19.145 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr cache_index_and_filter_blocks: 1 2024-04-07T16:02:19.145 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr cache_index_and_filter_blocks_with_high_priority: 0 2024-04-07T16:02:19.145 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-07T16:02:19.145 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr pin_top_level_index_and_filter: 1 2024-04-07T16:02:19.145 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr index_type: 0 2024-04-07T16:02:19.145 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr data_block_index_type: 0 2024-04-07T16:02:19.146 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr index_shortening: 1 2024-04-07T16:02:19.146 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr data_block_hash_table_util_ratio: 0.750000 2024-04-07T16:02:19.146 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr checksum: 4 2024-04-07T16:02:19.146 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr no_block_cache: 0 2024-04-07T16:02:19.146 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr block_cache: 0x564d713691f0 2024-04-07T16:02:19.146 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr block_cache_name: BinnedLRUCache 2024-04-07T16:02:19.146 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr block_cache_options: 2024-04-07T16:02:19.146 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr capacity : 536870912 2024-04-07T16:02:19.146 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr num_shard_bits : 4 2024-04-07T16:02:19.146 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr strict_capacity_limit : 0 2024-04-07T16:02:19.146 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr high_pri_pool_ratio: 0.000 2024-04-07T16:02:19.146 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr block_cache_compressed: (nil) 2024-04-07T16:02:19.146 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr persistent_cache: (nil) 2024-04-07T16:02:19.146 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr block_size: 4096 2024-04-07T16:02:19.147 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr block_size_deviation: 10 2024-04-07T16:02:19.147 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr block_restart_interval: 16 2024-04-07T16:02:19.147 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr index_block_restart_interval: 1 2024-04-07T16:02:19.147 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr metadata_block_size: 4096 2024-04-07T16:02:19.147 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr partition_filters: 0 2024-04-07T16:02:19.147 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr use_delta_encoding: 1 2024-04-07T16:02:19.147 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr filter_policy: bloomfilter 2024-04-07T16:02:19.147 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr whole_key_filtering: 1 2024-04-07T16:02:19.147 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr verify_compression: 0 2024-04-07T16:02:19.147 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr read_amp_bytes_per_bit: 0 2024-04-07T16:02:19.147 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr format_version: 5 2024-04-07T16:02:19.147 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr enable_index_compression: 1 2024-04-07T16:02:19.147 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr block_align: 0 2024-04-07T16:02:19.147 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr max_auto_readahead_size: 262144 2024-04-07T16:02:19.147 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr prepopulate_block_cache: 0 2024-04-07T16:02:19.148 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr initial_auto_readahead_size: 8192 2024-04-07T16:02:19.148 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr num_file_reads_for_auto_readahead: 2 2024-04-07T16:02:19.148 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr 2024-04-07T16:02:19.148 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.579+0000 7f53bca04c80 4 rocksdb: Options.write_buffer_size: 33554432 2024-04-07T16:02:19.148 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.579+0000 7f53bca04c80 4 rocksdb: Options.max_write_buffer_number: 2 2024-04-07T16:02:19.148 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.579+0000 7f53bca04c80 4 rocksdb: Options.compression: NoCompression 2024-04-07T16:02:19.148 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.579+0000 7f53bca04c80 4 rocksdb: Options.bottommost_compression: Disabled 2024-04-07T16:02:19.148 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.579+0000 7f53bca04c80 4 rocksdb: Options.prefix_extractor: nullptr 2024-04-07T16:02:19.148 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.579+0000 7f53bca04c80 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-07T16:02:19.148 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.579+0000 7f53bca04c80 4 rocksdb: Options.num_levels: 7 2024-04-07T16:02:19.148 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.579+0000 7f53bca04c80 4 rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-07T16:02:19.148 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.579+0000 7f53bca04c80 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-07T16:02:19.148 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.579+0000 7f53bca04c80 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-07T16:02:19.148 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.579+0000 7f53bca04c80 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-07T16:02:19.148 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.579+0000 7f53bca04c80 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-07T16:02:19.148 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.579+0000 7f53bca04c80 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-07T16:02:19.149 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.579+0000 7f53bca04c80 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-07T16:02:19.149 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.579+0000 7f53bca04c80 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-07T16:02:19.149 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.579+0000 7f53bca04c80 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-07T16:02:19.149 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.579+0000 7f53bca04c80 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-07T16:02:19.149 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.579+0000 7f53bca04c80 4 rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-07T16:02:19.149 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.579+0000 7f53bca04c80 4 rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-07T16:02:19.149 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.579+0000 7f53bca04c80 4 rocksdb: Options.compression_opts.window_bits: -14 2024-04-07T16:02:19.149 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.579+0000 7f53bca04c80 4 rocksdb: Options.compression_opts.level: 32767 2024-04-07T16:02:19.149 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.579+0000 7f53bca04c80 4 rocksdb: Options.compression_opts.strategy: 0 2024-04-07T16:02:19.149 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.579+0000 7f53bca04c80 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-07T16:02:19.149 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.579+0000 7f53bca04c80 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-07T16:02:19.149 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.579+0000 7f53bca04c80 4 rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-07T16:02:19.149 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.579+0000 7f53bca04c80 4 rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-07T16:02:19.149 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.579+0000 7f53bca04c80 4 rocksdb: Options.compression_opts.enabled: false 2024-04-07T16:02:19.149 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.579+0000 7f53bca04c80 4 rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-07T16:02:19.150 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.579+0000 7f53bca04c80 4 rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-07T16:02:19.150 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.579+0000 7f53bca04c80 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-07T16:02:19.150 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.579+0000 7f53bca04c80 4 rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-07T16:02:19.150 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.579+0000 7f53bca04c80 4 rocksdb: Options.target_file_size_base: 67108864 2024-04-07T16:02:19.150 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.579+0000 7f53bca04c80 4 rocksdb: Options.target_file_size_multiplier: 1 2024-04-07T16:02:19.150 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.579+0000 7f53bca04c80 4 rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-07T16:02:19.150 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.579+0000 7f53bca04c80 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-07T16:02:19.150 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.579+0000 7f53bca04c80 4 rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-07T16:02:19.150 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.579+0000 7f53bca04c80 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-07T16:02:19.150 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.579+0000 7f53bca04c80 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-07T16:02:19.150 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.579+0000 7f53bca04c80 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-07T16:02:19.150 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.579+0000 7f53bca04c80 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-07T16:02:19.150 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.579+0000 7f53bca04c80 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-07T16:02:19.150 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.579+0000 7f53bca04c80 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-07T16:02:19.151 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.579+0000 7f53bca04c80 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-07T16:02:19.151 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.579+0000 7f53bca04c80 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-07T16:02:19.151 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.579+0000 7f53bca04c80 4 rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-07T16:02:19.151 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.579+0000 7f53bca04c80 4 rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-07T16:02:19.151 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.579+0000 7f53bca04c80 4 rocksdb: Options.arena_block_size: 1048576 2024-04-07T16:02:19.151 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.579+0000 7f53bca04c80 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-07T16:02:19.151 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.579+0000 7f53bca04c80 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-07T16:02:19.151 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.579+0000 7f53bca04c80 4 rocksdb: Options.disable_auto_compactions: 0 2024-04-07T16:02:19.151 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.579+0000 7f53bca04c80 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-07T16:02:19.151 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.579+0000 7f53bca04c80 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-07T16:02:19.151 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.579+0000 7f53bca04c80 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-07T16:02:19.151 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.579+0000 7f53bca04c80 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-07T16:02:19.151 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.579+0000 7f53bca04c80 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-07T16:02:19.151 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.579+0000 7f53bca04c80 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-07T16:02:19.151 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.579+0000 7f53bca04c80 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-07T16:02:19.151 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.579+0000 7f53bca04c80 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-07T16:02:19.152 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.579+0000 7f53bca04c80 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-07T16:02:19.152 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.579+0000 7f53bca04c80 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-07T16:02:19.152 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.579+0000 7f53bca04c80 4 rocksdb: Options.table_properties_collectors: 2024-04-07T16:02:19.152 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.579+0000 7f53bca04c80 4 rocksdb: Options.inplace_update_support: 0 2024-04-07T16:02:19.152 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.579+0000 7f53bca04c80 4 rocksdb: Options.inplace_update_num_locks: 10000 2024-04-07T16:02:19.152 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.579+0000 7f53bca04c80 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-07T16:02:19.152 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.579+0000 7f53bca04c80 4 rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-07T16:02:19.152 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.579+0000 7f53bca04c80 4 rocksdb: Options.memtable_huge_page_size: 0 2024-04-07T16:02:19.152 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.579+0000 7f53bca04c80 4 rocksdb: Options.bloom_locality: 0 2024-04-07T16:02:19.152 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.579+0000 7f53bca04c80 4 rocksdb: Options.max_successive_merges: 0 2024-04-07T16:02:19.152 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.579+0000 7f53bca04c80 4 rocksdb: Options.optimize_filters_for_hits: 0 2024-04-07T16:02:19.152 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.579+0000 7f53bca04c80 4 rocksdb: Options.paranoid_file_checks: 0 2024-04-07T16:02:19.152 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.579+0000 7f53bca04c80 4 rocksdb: Options.force_consistency_checks: 1 2024-04-07T16:02:19.152 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.579+0000 7f53bca04c80 4 rocksdb: Options.report_bg_io_stats: 0 2024-04-07T16:02:19.152 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.579+0000 7f53bca04c80 4 rocksdb: Options.ttl: 2592000 2024-04-07T16:02:19.153 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.579+0000 7f53bca04c80 4 rocksdb: Options.periodic_compaction_seconds: 0 2024-04-07T16:02:19.153 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.579+0000 7f53bca04c80 4 rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-07T16:02:19.153 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.579+0000 7f53bca04c80 4 rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-07T16:02:19.153 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.579+0000 7f53bca04c80 4 rocksdb: Options.enable_blob_files: false 2024-04-07T16:02:19.153 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.579+0000 7f53bca04c80 4 rocksdb: Options.min_blob_size: 0 2024-04-07T16:02:19.153 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.579+0000 7f53bca04c80 4 rocksdb: Options.blob_file_size: 268435456 2024-04-07T16:02:19.153 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.579+0000 7f53bca04c80 4 rocksdb: Options.blob_compression_type: NoCompression 2024-04-07T16:02:19.153 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.579+0000 7f53bca04c80 4 rocksdb: Options.enable_blob_garbage_collection: false 2024-04-07T16:02:19.153 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.579+0000 7f53bca04c80 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-07T16:02:19.153 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.579+0000 7f53bca04c80 4 rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-07T16:02:19.153 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.579+0000 7f53bca04c80 4 rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-07T16:02:19.153 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.579+0000 7f53bca04c80 4 rocksdb: Options.blob_file_starting_level: 0 2024-04-07T16:02:19.154 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.579+0000 7f53bca04c80 4 rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-07T16:02:19.154 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.579+0000 7f53bca04c80 4 rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000001 succeeded,manifest_file_number is 1, next_file_number is 3, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-04-07T16:02:19.154 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr 2024-04-07T16:02:19.154 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.579+0000 7f53bca04c80 4 rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-04-07T16:02:19.154 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr 2024-04-07T16:02:19.154 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.579+0000 7f53bca04c80 4 rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 8e6083b8-6eb2-43a6-936e-e3f8952735eb 2024-04-07T16:02:19.154 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr 2024-04-07T16:02:19.154 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.579+0000 7f53bca04c80 4 rocksdb: [db/version_set.cc:5047] Creating manifest 5 2024-04-07T16:02:19.154 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr 2024-04-07T16:02:19.154 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.579+0000 7f53bca04c80 4 rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x564d713e0000 2024-04-07T16:02:19.154 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.579+0000 7f53bca04c80 4 rocksdb: DB pointer 0x564d713d2000 2024-04-07T16:02:19.154 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.579+0000 7f53acbe2700 4 rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-07T16:02:19.154 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.579+0000 7f53acbe2700 4 rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-07T16:02:19.154 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr ** DB Stats ** 2024-04-07T16:02:19.154 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr Uptime(secs): 0.0 total, 0.0 interval 2024-04-07T16:02:19.155 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-07T16:02:19.155 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-07T16:02:19.155 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-07T16:02:19.155 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-07T16:02:19.155 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-07T16:02:19.155 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-07T16:02:19.155 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr 2024-04-07T16:02:19.155 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr ** Compaction Stats [default] ** 2024-04-07T16:02:19.155 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-07T16:02:19.155 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-07T16:02:19.155 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr Sum 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 2024-04-07T16:02:19.155 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 2024-04-07T16:02:19.155 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr 2024-04-07T16:02:19.155 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr ** Compaction Stats [default] ** 2024-04-07T16:02:19.155 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-07T16:02:19.156 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-07T16:02:19.156 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr 2024-04-07T16:02:19.156 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-07T16:02:19.156 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr 2024-04-07T16:02:19.156 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr Uptime(secs): 0.0 total, 0.0 interval 2024-04-07T16:02:19.156 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr Flush(GB): cumulative 0.000, interval 0.000 2024-04-07T16:02:19.156 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr AddFile(GB): cumulative 0.000, interval 0.000 2024-04-07T16:02:19.156 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr AddFile(Total Files): cumulative 0, interval 0 2024-04-07T16:02:19.156 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr AddFile(L0 Files): cumulative 0, interval 0 2024-04-07T16:02:19.156 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr AddFile(Keys): cumulative 0, interval 0 2024-04-07T16:02:19.156 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr Cumulative compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-07T16:02:19.156 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-07T16:02:19.156 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-07T16:02:19.156 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr Block cache BinnedLRUCache@0x564d713691f0#7 capacity: 512.00 MB usage: 0.00 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.2e-05 secs_since: 0 2024-04-07T16:02:19.156 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr Block cache entry stats(count,size,portion): Misc(1,0.00 KB,0%) 2024-04-07T16:02:19.157 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr 2024-04-07T16:02:19.157 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr ** File Read Latency Histogram By Level [default] ** 2024-04-07T16:02:19.157 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr 2024-04-07T16:02:19.157 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.583+0000 7f53bca04c80 4 rocksdb: [db/db_impl/db_impl.cc:496] Shutdown: canceling all background work 2024-04-07T16:02:19.157 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.583+0000 7f53bca04c80 4 rocksdb: [db/db_impl/db_impl.cc:704] Shutdown complete 2024-04-07T16:02:19.157 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-04-07T16:02:18.583+0000 7f53bca04c80 0 /usr/bin/ceph-mon: created monfs at /var/lib/ceph/mon/ceph-a for mon.a 2024-04-07T16:02:19.157 INFO:teuthology.orchestra.run.smithi100.stdout:create mon.a on 2024-04-07T16:02:19.381 INFO:teuthology.orchestra.run.smithi100.stdout:systemctl: stderr Removed /etc/systemd/system/multi-user.target.wants/ceph.target. 2024-04-07T16:02:19.589 INFO:teuthology.orchestra.run.smithi100.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /etc/systemd/system/ceph.target. 2024-04-07T16:02:19.838 INFO:teuthology.orchestra.run.smithi100.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4.target -> /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4.target. 2024-04-07T16:02:19.838 INFO:teuthology.orchestra.run.smithi100.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4.target -> /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4.target. 2024-04-07T16:02:20.104 INFO:teuthology.orchestra.run.smithi100.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@mon.a 2024-04-07T16:02:20.104 INFO:teuthology.orchestra.run.smithi100.stdout:systemctl: stderr Failed to reset failed state of unit ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@mon.a.service: Unit ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@mon.a.service not loaded. 2024-04-07T16:02:20.305 INFO:teuthology.orchestra.run.smithi100.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4.target.wants/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@mon.a.service -> /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service. 2024-04-07T16:02:20.315 INFO:teuthology.orchestra.run.smithi100.stdout:firewalld does not appear to be present 2024-04-07T16:02:20.315 INFO:teuthology.orchestra.run.smithi100.stdout:Not possible to enable service . firewalld.service is not available 2024-04-07T16:02:20.315 INFO:teuthology.orchestra.run.smithi100.stdout:Waiting for mon to start... 2024-04-07T16:02:20.315 INFO:teuthology.orchestra.run.smithi100.stdout:Waiting for mon... 2024-04-07T16:02:21.896 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:21 smithi100 bash[20859]: cluster 2024-04-07T16:02:21.624183+0000 mon.a (mon.0) 1 : cluster 1 mon.a is new leader, mons a in quorum (ranks 0) 2024-04-07T16:02:22.440 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout cluster: 2024-04-07T16:02:22.440 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout id: 0702e3a8-f4f8-11ee-b649-cb9ed24678a4 2024-04-07T16:02:22.441 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-04-07T16:02:22.441 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout 2024-04-07T16:02:22.441 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout services: 2024-04-07T16:02:22.441 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.263128s) 2024-04-07T16:02:22.441 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-04-07T16:02:22.441 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-04-07T16:02:22.441 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout 2024-04-07T16:02:22.441 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout data: 2024-04-07T16:02:22.441 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-04-07T16:02:22.441 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-04-07T16:02:22.441 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-04-07T16:02:22.441 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout pgs: 2024-04-07T16:02:22.441 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout 2024-04-07T16:02:22.441 INFO:teuthology.orchestra.run.smithi100.stdout:mon is available 2024-04-07T16:02:22.442 INFO:teuthology.orchestra.run.smithi100.stdout:Assimilating anything we can from ceph.conf... 2024-04-07T16:02:22.944 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:22 smithi100 bash[20859]: cluster 2024-04-07T16:02:21.628906+0000 mon.a (mon.0) 2 : cluster 1 mon.a is new leader, mons a in quorum (ranks 0) 2024-04-07T16:02:22.944 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:22 smithi100 bash[20859]: cluster 2024-04-07T16:02:21.629092+0000 mon.a (mon.0) 3 : cluster 0 monmap e1: 1 mons at {a=[v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-04-07T16:02:22.944 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:22 smithi100 bash[20859]: cluster 2024-04-07T16:02:21.631085+0000 mon.a (mon.0) 4 : cluster 0 fsmap 2024-04-07T16:02:22.944 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:22 smithi100 bash[20859]: cluster 2024-04-07T16:02:21.633279+0000 mon.a (mon.0) 5 : cluster 0 osdmap e1: 0 total, 0 up, 0 in 2024-04-07T16:02:22.944 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:22 smithi100 bash[20859]: cluster 2024-04-07T16:02:21.633699+0000 mon.a (mon.0) 6 : cluster 0 mgrmap e1: no daemons active 2024-04-07T16:02:22.944 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:22 smithi100 bash[20859]: audit 2024-04-07T16:02:21.891976+0000 mon.a (mon.0) 7 : audit 0 from='client.? 172.21.15.100:0/3939926840' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-04-07T16:02:23.944 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:23 smithi100 bash[20859]: audit 2024-04-07T16:02:23.354828+0000 mon.a (mon.0) 8 : audit 1 from='client.? 172.21.15.100:0/141223399' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-07T16:02:23.944 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:23 smithi100 bash[20859]: audit 2024-04-07T16:02:23.361455+0000 mon.a (mon.0) 9 : audit 1 from='client.? 172.21.15.100:0/141223399' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-04-07T16:02:24.005 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout 2024-04-07T16:02:24.005 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout [global] 2024-04-07T16:02:24.005 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout fsid = 0702e3a8-f4f8-11ee-b649-cb9ed24678a4 2024-04-07T16:02:24.005 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.100:3300,v1:172.21.15.100:6789] 2024-04-07T16:02:24.005 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-07T16:02:24.005 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-07T16:02:24.005 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-07T16:02:24.005 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-07T16:02:24.005 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout 2024-04-07T16:02:24.005 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-07T16:02:24.005 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-07T16:02:24.005 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout 2024-04-07T16:02:24.006 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout [osd] 2024-04-07T16:02:24.006 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-07T16:02:24.006 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-07T16:02:24.006 INFO:teuthology.orchestra.run.smithi100.stdout:Generating new minimal ceph.conf... 2024-04-07T16:02:25.444 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:24 smithi100 bash[20859]: audit 2024-04-07T16:02:24.940964+0000 mon.a (mon.0) 10 : audit 0 from='client.? 172.21.15.100:0/1515241174' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:02:25.517 INFO:teuthology.orchestra.run.smithi100.stdout:Restarting the monitor... 2024-04-07T16:02:25.782 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:25 smithi100 systemd[1]: Stopping Ceph mon.a for 0702e3a8-f4f8-11ee-b649-cb9ed24678a4... 2024-04-07T16:02:25.783 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:25 smithi100 bash[20859]: debug 2024-04-07T16:02:25.671+0000 7f593563e700 -1 received signal: Terminated from /sbin/docker-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-stderr=true (PID: 1) UID: 0 2024-04-07T16:02:25.783 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:25 smithi100 bash[20859]: debug 2024-04-07T16:02:25.671+0000 7f593563e700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-04-07T16:02:26.359 INFO:teuthology.orchestra.run.smithi100.stdout:Setting public_network to 172.21.0.0/20,172.21.0.1/32,172.21.15.254/32,172.21.0.2/32 in mon config section 2024-04-07T16:02:26.694 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:26 smithi100 bash[21243]: ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4-mon-a 2024-04-07T16:02:26.694 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:26 smithi100 bash[21306]: Error response from daemon: No such container: ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4-mon-a 2024-04-07T16:02:26.694 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:26 smithi100 systemd[1]: ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@mon.a.service: Deactivated successfully. 2024-04-07T16:02:26.694 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:26 smithi100 systemd[1]: Stopped Ceph mon.a for 0702e3a8-f4f8-11ee-b649-cb9ed24678a4. 2024-04-07T16:02:26.694 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:26 smithi100 systemd[1]: Started Ceph mon.a for 0702e3a8-f4f8-11ee-b649-cb9ed24678a4. 2024-04-07T16:02:27.667 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.387+0000 7fea12530c80 0 set uid:gid to 167:167 (ceph:ceph) 2024-04-07T16:02:27.667 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.387+0000 7fea12530c80 0 ceph version 18.2.2-1171-g267c8ff5 (267c8ff5b1ff5815581b3b2381d19360e081b37b) reef (stable), process ceph-mon, pid 7 2024-04-07T16:02:27.667 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.387+0000 7fea12530c80 0 pidfile_write: ignore empty --pid-file 2024-04-07T16:02:27.667 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.391+0000 7fea12530c80 0 load: jerasure load: lrc 2024-04-07T16:02:27.667 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: RocksDB version: 7.9.2 2024-04-07T16:02:27.667 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Git sha 0 2024-04-07T16:02:27.667 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Compile date 2024-04-06 15:05:06 2024-04-07T16:02:27.667 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: DB SUMMARY 2024-04-07T16:02:27.667 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: DB Session ID: U7U0CSXSWVY9LINC75ZD 2024-04-07T16:02:27.667 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: CURRENT file: CURRENT 2024-04-07T16:02:27.667 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: IDENTITY file: IDENTITY 2024-04-07T16:02:27.667 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-04-07T16:02:27.667 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-04-07T16:02:27.668 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 84562 ; 2024-04-07T16:02:27.668 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.error_if_exists: 0 2024-04-07T16:02:27.668 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.create_if_missing: 0 2024-04-07T16:02:27.668 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.paranoid_checks: 1 2024-04-07T16:02:27.668 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.flush_verify_memtable_count: 1 2024-04-07T16:02:27.668 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-07T16:02:27.668 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-07T16:02:27.668 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.env: 0x55ca397967a0 2024-04-07T16:02:27.668 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.fs: PosixFileSystem 2024-04-07T16:02:27.668 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.info_log: 0x55ca3b3cd520 2024-04-07T16:02:27.668 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.max_file_opening_threads: 16 2024-04-07T16:02:27.668 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.statistics: (nil) 2024-04-07T16:02:27.668 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.use_fsync: 0 2024-04-07T16:02:27.668 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.max_log_file_size: 0 2024-04-07T16:02:27.669 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-07T16:02:27.669 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.log_file_time_to_roll: 0 2024-04-07T16:02:27.669 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.keep_log_file_num: 1000 2024-04-07T16:02:27.669 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.recycle_log_file_num: 0 2024-04-07T16:02:27.669 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.allow_fallocate: 1 2024-04-07T16:02:27.669 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.allow_mmap_reads: 0 2024-04-07T16:02:27.669 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.allow_mmap_writes: 0 2024-04-07T16:02:27.669 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.use_direct_reads: 0 2024-04-07T16:02:27.669 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-07T16:02:27.669 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.create_missing_column_families: 0 2024-04-07T16:02:27.669 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.db_log_dir: 2024-04-07T16:02:27.669 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.wal_dir: 2024-04-07T16:02:27.669 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.table_cache_numshardbits: 6 2024-04-07T16:02:27.669 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.WAL_ttl_seconds: 0 2024-04-07T16:02:27.669 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.WAL_size_limit_MB: 0 2024-04-07T16:02:27.670 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-07T16:02:27.670 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-07T16:02:27.670 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.is_fd_close_on_exec: 1 2024-04-07T16:02:27.670 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.advise_random_on_open: 1 2024-04-07T16:02:27.670 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.db_write_buffer_size: 0 2024-04-07T16:02:27.670 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.write_buffer_manager: 0x55ca3a660320 2024-04-07T16:02:27.670 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-07T16:02:27.670 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-07T16:02:27.670 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.use_adaptive_mutex: 0 2024-04-07T16:02:27.670 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.rate_limiter: (nil) 2024-04-07T16:02:27.670 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-07T16:02:27.670 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.wal_recovery_mode: 2 2024-04-07T16:02:27.672 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.enable_thread_tracking: 0 2024-04-07T16:02:27.672 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.enable_pipelined_write: 0 2024-04-07T16:02:27.672 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.unordered_write: 0 2024-04-07T16:02:27.672 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-07T16:02:27.672 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-07T16:02:27.672 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-07T16:02:27.672 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-07T16:02:27.672 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.row_cache: None 2024-04-07T16:02:27.672 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.wal_filter: None 2024-04-07T16:02:27.672 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-07T16:02:27.672 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.allow_ingest_behind: 0 2024-04-07T16:02:27.672 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.two_write_queues: 0 2024-04-07T16:02:27.672 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.manual_wal_flush: 0 2024-04-07T16:02:27.672 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.wal_compression: 0 2024-04-07T16:02:27.672 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.atomic_flush: 0 2024-04-07T16:02:27.673 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-07T16:02:27.673 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.persist_stats_to_disk: 0 2024-04-07T16:02:27.673 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.write_dbid_to_manifest: 0 2024-04-07T16:02:27.673 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.log_readahead_size: 0 2024-04-07T16:02:27.673 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-07T16:02:27.673 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.best_efforts_recovery: 0 2024-04-07T16:02:27.673 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-07T16:02:27.673 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-07T16:02:27.673 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.allow_data_in_errors: 0 2024-04-07T16:02:27.673 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.db_host_id: __hostname__ 2024-04-07T16:02:27.673 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.enforce_single_del_contracts: true 2024-04-07T16:02:27.673 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.max_background_jobs: 2 2024-04-07T16:02:27.673 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.max_background_compactions: -1 2024-04-07T16:02:27.673 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.max_subcompactions: 1 2024-04-07T16:02:27.673 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-07T16:02:27.673 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-07T16:02:27.674 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.delayed_write_rate : 16777216 2024-04-07T16:02:27.674 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.max_total_wal_size: 0 2024-04-07T16:02:27.674 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-07T16:02:27.674 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.stats_dump_period_sec: 600 2024-04-07T16:02:27.674 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.stats_persist_period_sec: 600 2024-04-07T16:02:27.674 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-07T16:02:27.674 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.max_open_files: -1 2024-04-07T16:02:27.674 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.bytes_per_sync: 0 2024-04-07T16:02:27.674 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.wal_bytes_per_sync: 0 2024-04-07T16:02:27.674 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.strict_bytes_per_sync: 0 2024-04-07T16:02:27.674 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.compaction_readahead_size: 0 2024-04-07T16:02:27.674 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.max_background_flushes: -1 2024-04-07T16:02:27.674 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Compression algorithms supported: 2024-04-07T16:02:27.674 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-07T16:02:27.675 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: kZSTD supported: 0 2024-04-07T16:02:27.675 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: kXpressCompression supported: 0 2024-04-07T16:02:27.675 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: kLZ4HCCompression supported: 1 2024-04-07T16:02:27.675 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: kZlibCompression supported: 1 2024-04-07T16:02:27.675 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: kSnappyCompression supported: 1 2024-04-07T16:02:27.675 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: kLZ4Compression supported: 1 2024-04-07T16:02:27.675 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: kBZip2Compression supported: 0 2024-04-07T16:02:27.675 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Fast CRC32 supported: Supported on x86 2024-04-07T16:02:27.675 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: DMutex implementation: pthread_mutex_t 2024-04-07T16:02:27.675 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-04-07T16:02:27.675 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-07T16:02:27.675 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-07T16:02:27.675 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.merge_operator: 2024-04-07T16:02:27.675 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.compaction_filter: None 2024-04-07T16:02:27.676 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.compaction_filter_factory: None 2024-04-07T16:02:27.676 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.sst_partitioner_factory: None 2024-04-07T16:02:27.676 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.memtable_factory: SkipListFactory 2024-04-07T16:02:27.676 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.table_factory: BlockBasedTable 2024-04-07T16:02:27.676 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55ca3b3cd620) 2024-04-07T16:02:27.676 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: cache_index_and_filter_blocks: 1 2024-04-07T16:02:27.676 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-07T16:02:27.676 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-07T16:02:27.676 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: pin_top_level_index_and_filter: 1 2024-04-07T16:02:27.676 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: index_type: 0 2024-04-07T16:02:27.676 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: data_block_index_type: 0 2024-04-07T16:02:27.676 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: index_shortening: 1 2024-04-07T16:02:27.676 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: data_block_hash_table_util_ratio: 0.750000 2024-04-07T16:02:27.676 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: checksum: 4 2024-04-07T16:02:27.676 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: no_block_cache: 0 2024-04-07T16:02:27.676 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: block_cache: 0x55ca3a6e31f0 2024-04-07T16:02:27.677 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: block_cache_name: BinnedLRUCache 2024-04-07T16:02:27.677 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: block_cache_options: 2024-04-07T16:02:27.677 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: capacity : 536870912 2024-04-07T16:02:27.677 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: num_shard_bits : 4 2024-04-07T16:02:27.677 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: strict_capacity_limit : 0 2024-04-07T16:02:27.677 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: high_pri_pool_ratio: 0.000 2024-04-07T16:02:27.677 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: block_cache_compressed: (nil) 2024-04-07T16:02:27.677 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: persistent_cache: (nil) 2024-04-07T16:02:27.677 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: block_size: 4096 2024-04-07T16:02:27.677 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: block_size_deviation: 10 2024-04-07T16:02:27.677 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: block_restart_interval: 16 2024-04-07T16:02:27.677 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: index_block_restart_interval: 1 2024-04-07T16:02:27.677 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: metadata_block_size: 4096 2024-04-07T16:02:27.677 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: partition_filters: 0 2024-04-07T16:02:27.677 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: use_delta_encoding: 1 2024-04-07T16:02:27.677 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: filter_policy: bloomfilter 2024-04-07T16:02:27.678 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: whole_key_filtering: 1 2024-04-07T16:02:27.678 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: verify_compression: 0 2024-04-07T16:02:27.678 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: read_amp_bytes_per_bit: 0 2024-04-07T16:02:27.678 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: format_version: 5 2024-04-07T16:02:27.678 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: enable_index_compression: 1 2024-04-07T16:02:27.678 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: block_align: 0 2024-04-07T16:02:27.678 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: max_auto_readahead_size: 262144 2024-04-07T16:02:27.678 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: prepopulate_block_cache: 0 2024-04-07T16:02:27.678 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: initial_auto_readahead_size: 8192 2024-04-07T16:02:27.678 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: num_file_reads_for_auto_readahead: 2 2024-04-07T16:02:27.678 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.write_buffer_size: 33554432 2024-04-07T16:02:27.678 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.max_write_buffer_number: 2 2024-04-07T16:02:27.678 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.compression: NoCompression 2024-04-07T16:02:27.678 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.bottommost_compression: Disabled 2024-04-07T16:02:27.679 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.prefix_extractor: nullptr 2024-04-07T16:02:27.679 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-07T16:02:27.679 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.num_levels: 7 2024-04-07T16:02:27.679 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-07T16:02:27.679 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-07T16:02:27.679 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-07T16:02:27.679 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-07T16:02:27.679 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-07T16:02:27.679 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-07T16:02:27.679 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-07T16:02:27.679 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-07T16:02:27.679 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-07T16:02:27.679 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-07T16:02:27.679 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-07T16:02:27.679 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-07T16:02:27.680 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.compression_opts.window_bits: -14 2024-04-07T16:02:27.680 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.compression_opts.level: 32767 2024-04-07T16:02:27.680 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.compression_opts.strategy: 0 2024-04-07T16:02:27.680 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-07T16:02:27.680 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-07T16:02:27.680 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-07T16:02:27.680 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-07T16:02:27.680 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.compression_opts.enabled: false 2024-04-07T16:02:27.680 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-07T16:02:27.680 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-07T16:02:27.680 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-07T16:02:27.680 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-07T16:02:27.680 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.target_file_size_base: 67108864 2024-04-07T16:02:27.680 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.target_file_size_multiplier: 1 2024-04-07T16:02:27.680 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-07T16:02:27.681 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-07T16:02:27.681 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-07T16:02:27.681 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-07T16:02:27.681 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-07T16:02:27.681 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-07T16:02:27.681 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-07T16:02:27.681 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-07T16:02:27.681 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-07T16:02:27.681 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-07T16:02:27.681 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-07T16:02:27.681 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-07T16:02:27.681 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-07T16:02:27.681 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.arena_block_size: 1048576 2024-04-07T16:02:27.681 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-07T16:02:27.681 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-07T16:02:27.682 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.disable_auto_compactions: 0 2024-04-07T16:02:27.682 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-07T16:02:27.682 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-07T16:02:27.682 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-07T16:02:27.682 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-07T16:02:27.682 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-07T16:02:27.682 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-07T16:02:27.682 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-07T16:02:27.682 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-07T16:02:27.682 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-07T16:02:27.682 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-07T16:02:27.682 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.table_properties_collectors: 2024-04-07T16:02:27.682 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.inplace_update_support: 0 2024-04-07T16:02:27.682 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.inplace_update_num_locks: 10000 2024-04-07T16:02:27.682 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-07T16:02:27.683 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-07T16:02:27.683 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.memtable_huge_page_size: 0 2024-04-07T16:02:27.683 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.bloom_locality: 0 2024-04-07T16:02:27.683 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.max_successive_merges: 0 2024-04-07T16:02:27.683 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.optimize_filters_for_hits: 0 2024-04-07T16:02:27.683 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.paranoid_file_checks: 0 2024-04-07T16:02:27.683 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.force_consistency_checks: 1 2024-04-07T16:02:27.683 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.report_bg_io_stats: 0 2024-04-07T16:02:27.683 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.ttl: 2592000 2024-04-07T16:02:27.683 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.periodic_compaction_seconds: 0 2024-04-07T16:02:27.683 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-07T16:02:27.683 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-07T16:02:27.683 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.enable_blob_files: false 2024-04-07T16:02:27.683 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.min_blob_size: 0 2024-04-07T16:02:27.683 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.blob_file_size: 268435456 2024-04-07T16:02:27.683 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.blob_compression_type: NoCompression 2024-04-07T16:02:27.684 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.enable_blob_garbage_collection: false 2024-04-07T16:02:27.684 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-07T16:02:27.684 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-07T16:02:27.684 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-07T16:02:27.684 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.blob_file_starting_level: 0 2024-04-07T16:02:27.684 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-07T16:02:27.684 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-04-07T16:02:27.684 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-04-07T16:02:27.684 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 8e6083b8-6eb2-43a6-936e-e3f8952735eb 2024-04-07T16:02:27.684 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1712505747397191, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-04-07T16:02:27.684 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-04-07T16:02:27.684 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1712505747398293, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 81086, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 256, "table_properties": {"data_size": 79262, "index_size": 234, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 581, "raw_key_size": 10267, "raw_average_key_size": 47, "raw_value_size": 73463, "raw_average_value_size": 341, "num_data_blocks": 10, "num_entries": 215, "num_filter_entries": 215, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1712505747, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "8e6083b8-6eb2-43a6-936e-e3f8952735eb", "db_session_id": "U7U0CSXSWVY9LINC75ZD", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-04-07T16:02:27.684 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1712505747398393, "job": 1, "event": "recovery_finished"} 2024-04-07T16:02:27.684 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.395+0000 7fea12530c80 4 rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-04-07T16:02:27.684 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.399+0000 7fea12530c80 4 rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-07T16:02:27.684 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.399+0000 7fea12530c80 4 rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55ca3a784000 2024-04-07T16:02:27.685 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.399+0000 7fea12530c80 4 rocksdb: DB pointer 0x55ca3a772000 2024-04-07T16:02:27.685 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.399+0000 7fea00d08700 4 rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-07T16:02:27.685 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.399+0000 7fea00d08700 4 rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-07T16:02:27.685 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: ** DB Stats ** 2024-04-07T16:02:27.685 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: Uptime(secs): 0.0 total, 0.0 interval 2024-04-07T16:02:27.685 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-07T16:02:27.685 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-07T16:02:27.685 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-07T16:02:27.685 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-07T16:02:27.685 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-07T16:02:27.685 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-07T16:02:27.685 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: ** Compaction Stats [default] ** 2024-04-07T16:02:27.685 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-07T16:02:27.685 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-07T16:02:27.685 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: L0 2/0 81.02 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 121.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-07T16:02:27.685 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: Sum 2/0 81.02 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 121.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-07T16:02:27.686 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 121.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-07T16:02:27.686 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: ** Compaction Stats [default] ** 2024-04-07T16:02:27.686 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-07T16:02:27.686 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-07T16:02:27.686 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 121.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-07T16:02:27.686 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-07T16:02:27.686 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: Uptime(secs): 0.0 total, 0.0 interval 2024-04-07T16:02:27.686 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: Flush(GB): cumulative 0.000, interval 0.000 2024-04-07T16:02:27.686 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-07T16:02:27.686 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: AddFile(Total Files): cumulative 0, interval 0 2024-04-07T16:02:27.686 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: AddFile(L0 Files): cumulative 0, interval 0 2024-04-07T16:02:27.686 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: AddFile(Keys): cumulative 0, interval 0 2024-04-07T16:02:27.686 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: Cumulative compaction: 0.00 GB write, 18.95 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-07T16:02:27.686 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: Interval compaction: 0.00 GB write, 18.95 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-07T16:02:27.686 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-07T16:02:27.687 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: Block cache BinnedLRUCache@0x55ca3a6e31f0#7 capacity: 512.00 MB usage: 1.19 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.3e-05 secs_since: 0 2024-04-07T16:02:27.687 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: Block cache entry stats(count,size,portion): FilterBlock(2,0.77 KB,0.000146031%) IndexBlock(2,0.42 KB,8.04663e-05%) Misc(1,0.00 KB,0%) 2024-04-07T16:02:27.687 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: ** File Read Latency Histogram By Level [default] ** 2024-04-07T16:02:27.687 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.399+0000 7fea12530c80 0 starting mon.a rank 0 at public addrs [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] at bind addrs [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 0702e3a8-f4f8-11ee-b649-cb9ed24678a4 2024-04-07T16:02:27.687 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.399+0000 7fea12530c80 1 mon.a@-1(???) e1 preinit fsid 0702e3a8-f4f8-11ee-b649-cb9ed24678a4 2024-04-07T16:02:27.687 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.399+0000 7fea12530c80 0 mon.a@-1(???).mds e1 new map 2024-04-07T16:02:27.687 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.399+0000 7fea12530c80 0 mon.a@-1(???).mds e1 print_map 2024-04-07T16:02:27.687 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: e1 2024-04-07T16:02:27.687 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: enable_multiple, ever_enabled_multiple: 1,1 2024-04-07T16:02:27.687 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-07T16:02:27.687 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: legacy client fscid: -1 2024-04-07T16:02:27.687 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: 2024-04-07T16:02:27.687 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: No filesystems configured 2024-04-07T16:02:27.687 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.399+0000 7fea12530c80 0 mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-07T16:02:27.688 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.399+0000 7fea12530c80 0 mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-07T16:02:27.688 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.399+0000 7fea12530c80 0 mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-07T16:02:27.688 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.399+0000 7fea12530c80 0 mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-07T16:02:27.688 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: debug 2024-04-07T16:02:27.399+0000 7fea12530c80 1 mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-04-07T16:02:27.688 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: cluster 2024-04-07T16:02:27.423873+0000 mon.a (mon.0) 1 : cluster 1 mon.a is new leader, mons a in quorum (ranks 0) 2024-04-07T16:02:27.688 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: cluster 2024-04-07T16:02:27.423953+0000 mon.a (mon.0) 2 : cluster 0 monmap e1: 1 mons at {a=[v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-04-07T16:02:27.688 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: cluster 2024-04-07T16:02:27.443950+0000 mon.a (mon.0) 3 : cluster 0 fsmap 2024-04-07T16:02:27.688 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: cluster 2024-04-07T16:02:27.443998+0000 mon.a (mon.0) 4 : cluster 0 osdmap e1: 0 total, 0 up, 0 in 2024-04-07T16:02:27.688 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:27 smithi100 bash[21346]: cluster 2024-04-07T16:02:27.444396+0000 mon.a (mon.0) 5 : cluster 0 mgrmap e1: no daemons active 2024-04-07T16:02:28.271 INFO:teuthology.orchestra.run.smithi100.stdout:Wrote config to /etc/ceph/ceph.conf 2024-04-07T16:02:28.297 INFO:teuthology.orchestra.run.smithi100.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-04-07T16:02:28.297 INFO:teuthology.orchestra.run.smithi100.stdout:Creating mgr... 2024-04-07T16:02:28.297 INFO:teuthology.orchestra.run.smithi100.stdout:Verifying port 0.0.0.0:9283 ... 2024-04-07T16:02:28.298 INFO:teuthology.orchestra.run.smithi100.stdout:Verifying port 0.0.0.0:8765 ... 2024-04-07T16:02:28.623 INFO:teuthology.orchestra.run.smithi100.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@mgr.y 2024-04-07T16:02:28.623 INFO:teuthology.orchestra.run.smithi100.stdout:systemctl: stderr Failed to reset failed state of unit ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@mgr.y.service: Unit ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@mgr.y.service not loaded. 2024-04-07T16:02:28.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:28 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:02:28.840 INFO:teuthology.orchestra.run.smithi100.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4.target.wants/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@mgr.y.service -> /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service. 2024-04-07T16:02:28.848 INFO:teuthology.orchestra.run.smithi100.stdout:firewalld does not appear to be present 2024-04-07T16:02:28.849 INFO:teuthology.orchestra.run.smithi100.stdout:Not possible to enable service . firewalld.service is not available 2024-04-07T16:02:28.849 INFO:teuthology.orchestra.run.smithi100.stdout:firewalld does not appear to be present 2024-04-07T16:02:28.849 INFO:teuthology.orchestra.run.smithi100.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-04-07T16:02:28.849 INFO:teuthology.orchestra.run.smithi100.stdout:Waiting for mgr to start... 2024-04-07T16:02:28.849 INFO:teuthology.orchestra.run.smithi100.stdout:Waiting for mgr... 2024-04-07T16:02:28.943 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:28 smithi100 bash[21346]: audit 2024-04-07T16:02:27.662904+0000 mon.a (mon.0) 6 : audit 1 from='client.? 172.21.15.100:0/4197534852' entity='client.admin' 2024-04-07T16:02:28.944 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:28 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:02:30.693 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:30 smithi100 bash[21346]: audit 2024-04-07T16:02:30.357860+0000 mon.a (mon.0) 7 : audit 0 from='client.? 172.21.15.100:0/558914710' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-07T16:02:30.694 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:02:30 smithi100 bash[21601]: debug 2024-04-07T16:02:30.335+0000 7f57c98d6200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-07T16:02:30.925 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout 2024-04-07T16:02:30.925 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout { 2024-04-07T16:02:30.925 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "fsid": "0702e3a8-f4f8-11ee-b649-cb9ed24678a4", 2024-04-07T16:02:30.926 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "health": { 2024-04-07T16:02:30.926 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-07T16:02:30.926 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-07T16:02:30.926 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-07T16:02:30.926 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout }, 2024-04-07T16:02:30.926 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-07T16:02:30.926 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-07T16:02:30.926 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout 0 2024-04-07T16:02:30.926 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout ], 2024-04-07T16:02:30.926 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-07T16:02:30.926 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "a" 2024-04-07T16:02:30.926 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout ], 2024-04-07T16:02:30.926 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-04-07T16:02:30.927 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-07T16:02:30.927 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-07T16:02:30.927 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-04-07T16:02:30.927 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-07T16:02:30.927 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout }, 2024-04-07T16:02:30.927 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-07T16:02:30.927 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-07T16:02:30.927 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-07T16:02:30.927 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-07T16:02:30.927 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-07T16:02:30.927 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-07T16:02:30.931 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-07T16:02:30.931 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-07T16:02:30.931 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout }, 2024-04-07T16:02:30.931 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-07T16:02:30.931 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-07T16:02:30.932 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-07T16:02:30.932 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-07T16:02:30.932 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-07T16:02:30.932 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-07T16:02:30.932 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-07T16:02:30.932 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-07T16:02:30.932 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-07T16:02:30.932 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout }, 2024-04-07T16:02:30.932 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-07T16:02:30.933 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-07T16:02:30.933 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-07T16:02:30.933 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-07T16:02:30.933 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout }, 2024-04-07T16:02:30.933 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-07T16:02:30.933 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-07T16:02:30.933 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-07T16:02:30.933 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-07T16:02:30.933 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-07T16:02:30.933 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-07T16:02:30.933 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "restful" 2024-04-07T16:02:30.933 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout ], 2024-04-07T16:02:30.933 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-07T16:02:30.933 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout }, 2024-04-07T16:02:30.933 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-07T16:02:30.934 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-07T16:02:30.934 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "modified": "2024-04-07T16:02:21.630195+0000", 2024-04-07T16:02:30.934 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-07T16:02:30.934 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout }, 2024-04-07T16:02:30.934 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-07T16:02:30.934 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout } 2024-04-07T16:02:30.934 INFO:teuthology.orchestra.run.smithi100.stdout:mgr not available, waiting (1/15)... 2024-04-07T16:02:32.194 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:02:31 smithi100 bash[21601]: debug 2024-04-07T16:02:31.891+0000 7f57c98d6200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-07T16:02:32.194 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:02:31 smithi100 bash[21601]: debug 2024-04-07T16:02:31.983+0000 7f57c98d6200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-07T16:02:32.693 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:02:32 smithi100 bash[21601]: debug 2024-04-07T16:02:32.227+0000 7f57c98d6200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-07T16:02:33.213 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:02:33 smithi100 bash[21601]: debug 2024-04-07T16:02:33.011+0000 7f57c98d6200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-07T16:02:33.213 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:02:33 smithi100 bash[21601]: debug 2024-04-07T16:02:33.119+0000 7f57c98d6200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-07T16:02:33.563 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:02:33 smithi100 bash[21601]: debug 2024-04-07T16:02:33.219+0000 7f57c98d6200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-07T16:02:33.943 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:02:33 smithi100 bash[21601]: debug 2024-04-07T16:02:33.571+0000 7f57c98d6200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-07T16:02:33.944 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:02:33 smithi100 bash[21601]: debug 2024-04-07T16:02:33.683+0000 7f57c98d6200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-07T16:02:33.944 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:02:33 smithi100 bash[21601]: debug 2024-04-07T16:02:33.787+0000 7f57c98d6200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-07T16:02:34.369 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:02:34 smithi100 bash[21601]: debug 2024-04-07T16:02:34.139+0000 7f57c98d6200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-07T16:02:34.369 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:34 smithi100 bash[21346]: audit 2024-04-07T16:02:33.993914+0000 mon.a (mon.0) 8 : audit 0 from='client.? 172.21.15.100:0/844078271' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-07T16:02:34.563 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout 2024-04-07T16:02:34.563 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout { 2024-04-07T16:02:34.563 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "fsid": "0702e3a8-f4f8-11ee-b649-cb9ed24678a4", 2024-04-07T16:02:34.563 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "health": { 2024-04-07T16:02:34.563 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-07T16:02:34.563 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-07T16:02:34.563 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-07T16:02:34.563 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout }, 2024-04-07T16:02:34.563 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-07T16:02:34.564 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-07T16:02:34.564 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout 0 2024-04-07T16:02:34.564 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout ], 2024-04-07T16:02:34.564 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-07T16:02:34.564 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "a" 2024-04-07T16:02:34.564 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout ], 2024-04-07T16:02:34.564 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "quorum_age": 6, 2024-04-07T16:02:34.564 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-07T16:02:34.564 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-07T16:02:34.564 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-04-07T16:02:34.564 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-07T16:02:34.564 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout }, 2024-04-07T16:02:34.564 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-07T16:02:34.564 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-07T16:02:34.564 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-07T16:02:34.564 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-07T16:02:34.564 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-07T16:02:34.565 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-07T16:02:34.568 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-07T16:02:34.568 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-07T16:02:34.568 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout }, 2024-04-07T16:02:34.568 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-07T16:02:34.568 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-07T16:02:34.568 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-07T16:02:34.568 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-07T16:02:34.568 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-07T16:02:34.569 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-07T16:02:34.569 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-07T16:02:34.569 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-07T16:02:34.569 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-07T16:02:34.569 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout }, 2024-04-07T16:02:34.569 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-07T16:02:34.569 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-07T16:02:34.569 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-07T16:02:34.569 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-07T16:02:34.569 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout }, 2024-04-07T16:02:34.569 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-07T16:02:34.569 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-07T16:02:34.569 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-07T16:02:34.569 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-07T16:02:34.569 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-07T16:02:34.570 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-07T16:02:34.570 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "restful" 2024-04-07T16:02:34.570 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout ], 2024-04-07T16:02:34.570 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-07T16:02:34.570 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout }, 2024-04-07T16:02:34.570 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-07T16:02:34.570 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-07T16:02:34.570 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "modified": "2024-04-07T16:02:21.630195+0000", 2024-04-07T16:02:34.570 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-07T16:02:34.570 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout }, 2024-04-07T16:02:34.570 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-07T16:02:34.570 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout } 2024-04-07T16:02:34.570 INFO:teuthology.orchestra.run.smithi100.stdout:mgr not available, waiting (2/15)... 2024-04-07T16:02:34.627 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:02:34 smithi100 bash[21601]: debug 2024-04-07T16:02:34.375+0000 7f57c98d6200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-07T16:02:34.627 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:02:34 smithi100 bash[21601]: debug 2024-04-07T16:02:34.635+0000 7f57c98d6200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-07T16:02:34.943 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:02:34 smithi100 bash[21601]: debug 2024-04-07T16:02:34.743+0000 7f57c98d6200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-07T16:02:35.444 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:02:35 smithi100 bash[21601]: debug 2024-04-07T16:02:35.007+0000 7f57c98d6200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-07T16:02:35.444 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:02:35 smithi100 bash[21601]: debug 2024-04-07T16:02:35.243+0000 7f57c98d6200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-07T16:02:35.944 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:02:35 smithi100 bash[21601]: debug 2024-04-07T16:02:35.643+0000 7f57c98d6200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-07T16:02:35.944 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:02:35 smithi100 bash[21601]: debug 2024-04-07T16:02:35.759+0000 7f57c98d6200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-07T16:02:36.444 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:02:36 smithi100 bash[21601]: debug 2024-04-07T16:02:35.999+0000 7f57c98d6200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-07T16:02:37.240 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:02:36 smithi100 bash[21601]: debug 2024-04-07T16:02:36.987+0000 7f57c98d6200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-07T16:02:37.240 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:02:37 smithi100 bash[21601]: debug 2024-04-07T16:02:37.102+0000 7f57c98d6200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-07T16:02:37.507 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:02:37 smithi100 bash[21601]: debug 2024-04-07T16:02:37.402+0000 7f57c98d6200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-07T16:02:37.944 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:37 smithi100 bash[21346]: audit 2024-04-07T16:02:37.567977+0000 mon.a (mon.0) 9 : audit 0 from='client.? 172.21.15.100:0/1225392316' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-07T16:02:37.944 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:02:37 smithi100 bash[21601]: debug 2024-04-07T16:02:37.514+0000 7f57c98d6200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-07T16:02:38.087 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout 2024-04-07T16:02:38.088 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout { 2024-04-07T16:02:38.088 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "fsid": "0702e3a8-f4f8-11ee-b649-cb9ed24678a4", 2024-04-07T16:02:38.088 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "health": { 2024-04-07T16:02:38.088 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-07T16:02:38.088 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-07T16:02:38.088 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-07T16:02:38.088 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout }, 2024-04-07T16:02:38.088 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-07T16:02:38.088 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-07T16:02:38.088 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout 0 2024-04-07T16:02:38.088 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout ], 2024-04-07T16:02:38.088 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-07T16:02:38.088 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "a" 2024-04-07T16:02:38.088 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout ], 2024-04-07T16:02:38.088 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "quorum_age": 10, 2024-04-07T16:02:38.089 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-07T16:02:38.089 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-07T16:02:38.089 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-04-07T16:02:38.089 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-07T16:02:38.089 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout }, 2024-04-07T16:02:38.089 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-07T16:02:38.089 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-07T16:02:38.089 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-07T16:02:38.089 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-07T16:02:38.089 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-07T16:02:38.089 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-07T16:02:38.089 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-07T16:02:38.089 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-07T16:02:38.089 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout }, 2024-04-07T16:02:38.089 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-07T16:02:38.089 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-07T16:02:38.090 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-07T16:02:38.090 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-07T16:02:38.090 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-07T16:02:38.090 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-07T16:02:38.090 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-07T16:02:38.090 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-07T16:02:38.090 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-07T16:02:38.090 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout }, 2024-04-07T16:02:38.090 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-07T16:02:38.090 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-07T16:02:38.090 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-07T16:02:38.090 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-07T16:02:38.090 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout }, 2024-04-07T16:02:38.090 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-07T16:02:38.090 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-07T16:02:38.091 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-07T16:02:38.091 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-07T16:02:38.091 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-07T16:02:38.091 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-07T16:02:38.091 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "restful" 2024-04-07T16:02:38.091 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout ], 2024-04-07T16:02:38.091 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-07T16:02:38.091 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout }, 2024-04-07T16:02:38.091 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-07T16:02:38.091 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-07T16:02:38.091 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "modified": "2024-04-07T16:02:21.630195+0000", 2024-04-07T16:02:38.091 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-07T16:02:38.091 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout }, 2024-04-07T16:02:38.091 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-07T16:02:38.091 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout } 2024-04-07T16:02:38.092 INFO:teuthology.orchestra.run.smithi100.stdout:mgr not available, waiting (3/15)... 2024-04-07T16:02:38.444 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:02:37 smithi100 bash[21601]: debug 2024-04-07T16:02:37.962+0000 7f57c98d6200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-07T16:02:38.444 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:02:38 smithi100 bash[21601]: debug 2024-04-07T16:02:38.082+0000 7f57c98d6200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-07T16:02:38.944 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:38 smithi100 bash[21346]: cluster 2024-04-07T16:02:38.105142+0000 mon.a (mon.0) 10 : cluster 1 Activating manager daemon y 2024-04-07T16:02:38.944 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:38 smithi100 bash[21346]: cluster 2024-04-07T16:02:38.109133+0000 mon.a (mon.0) 11 : cluster 0 mgrmap e2: y(active, starting, since 0.00427875s) 2024-04-07T16:02:38.945 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:38 smithi100 bash[21346]: audit 2024-04-07T16:02:38.110311+0000 mon.a (mon.0) 12 : audit 0 from='mgr.14100 172.21.15.100:0/3097710657' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-07T16:02:38.945 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:38 smithi100 bash[21346]: audit 2024-04-07T16:02:38.110461+0000 mon.a (mon.0) 13 : audit 0 from='mgr.14100 172.21.15.100:0/3097710657' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-07T16:02:38.945 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:38 smithi100 bash[21346]: audit 2024-04-07T16:02:38.110624+0000 mon.a (mon.0) 14 : audit 0 from='mgr.14100 172.21.15.100:0/3097710657' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-07T16:02:38.945 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:38 smithi100 bash[21346]: audit 2024-04-07T16:02:38.110806+0000 mon.a (mon.0) 15 : audit 0 from='mgr.14100 172.21.15.100:0/3097710657' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:02:38.945 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:38 smithi100 bash[21346]: audit 2024-04-07T16:02:38.111385+0000 mon.a (mon.0) 16 : audit 0 from='mgr.14100 172.21.15.100:0/3097710657' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-07T16:02:38.945 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:38 smithi100 bash[21346]: cluster 2024-04-07T16:02:38.117662+0000 mon.a (mon.0) 17 : cluster 1 Manager daemon y is now available 2024-04-07T16:02:38.945 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:38 smithi100 bash[21346]: audit 2024-04-07T16:02:38.129921+0000 mon.a (mon.0) 18 : audit 1 from='mgr.14100 172.21.15.100:0/3097710657' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-04-07T16:02:38.945 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:38 smithi100 bash[21346]: audit 2024-04-07T16:02:38.132412+0000 mon.a (mon.0) 19 : audit 1 from='mgr.14100 172.21.15.100:0/3097710657' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-04-07T16:02:38.945 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:38 smithi100 bash[21346]: audit 2024-04-07T16:02:38.135628+0000 mon.a (mon.0) 20 : audit 1 from='mgr.14100 172.21.15.100:0/3097710657' entity='mgr.y' 2024-04-07T16:02:38.945 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:38 smithi100 bash[21346]: audit 2024-04-07T16:02:38.137456+0000 mon.a (mon.0) 21 : audit 1 from='mgr.14100 172.21.15.100:0/3097710657' entity='mgr.y' 2024-04-07T16:02:38.945 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:38 smithi100 bash[21346]: audit 2024-04-07T16:02:38.139200+0000 mon.a (mon.0) 22 : audit 1 from='mgr.14100 172.21.15.100:0/3097710657' entity='mgr.y' 2024-04-07T16:02:40.444 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:40 smithi100 bash[21346]: cluster 2024-04-07T16:02:39.119929+0000 mon.a (mon.0) 23 : cluster 0 mgrmap e3: y(active, since 1.01506s) 2024-04-07T16:02:42.113 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout 2024-04-07T16:02:42.113 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout { 2024-04-07T16:02:42.113 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "fsid": "0702e3a8-f4f8-11ee-b649-cb9ed24678a4", 2024-04-07T16:02:42.113 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "health": { 2024-04-07T16:02:42.113 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-07T16:02:42.113 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-07T16:02:42.113 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-07T16:02:42.113 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout }, 2024-04-07T16:02:42.113 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-07T16:02:42.114 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-07T16:02:42.114 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout 0 2024-04-07T16:02:42.114 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout ], 2024-04-07T16:02:42.114 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-07T16:02:42.114 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "a" 2024-04-07T16:02:42.114 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout ], 2024-04-07T16:02:42.114 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "quorum_age": 14, 2024-04-07T16:02:42.114 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-07T16:02:42.114 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-07T16:02:42.114 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-04-07T16:02:42.114 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-07T16:02:42.114 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout }, 2024-04-07T16:02:42.114 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-07T16:02:42.114 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-07T16:02:42.114 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-07T16:02:42.115 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-07T16:02:42.115 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-07T16:02:42.115 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-07T16:02:42.115 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-07T16:02:42.115 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-07T16:02:42.115 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout }, 2024-04-07T16:02:42.115 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-07T16:02:42.115 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-07T16:02:42.115 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-07T16:02:42.115 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-07T16:02:42.115 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-07T16:02:42.115 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-07T16:02:42.115 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-07T16:02:42.115 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-07T16:02:42.115 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-07T16:02:42.115 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout }, 2024-04-07T16:02:42.116 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-07T16:02:42.116 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-07T16:02:42.116 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-07T16:02:42.116 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-07T16:02:42.116 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout }, 2024-04-07T16:02:42.116 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-07T16:02:42.116 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-07T16:02:42.116 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-07T16:02:42.116 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-07T16:02:42.116 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-07T16:02:42.116 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-07T16:02:42.116 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "restful" 2024-04-07T16:02:42.116 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout ], 2024-04-07T16:02:42.116 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-07T16:02:42.116 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout }, 2024-04-07T16:02:42.116 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-07T16:02:42.117 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-07T16:02:42.117 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "modified": "2024-04-07T16:02:21.630195+0000", 2024-04-07T16:02:42.117 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-07T16:02:42.117 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout }, 2024-04-07T16:02:42.117 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-07T16:02:42.117 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout } 2024-04-07T16:02:42.117 INFO:teuthology.orchestra.run.smithi100.stdout:mgr is available 2024-04-07T16:02:42.444 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:42 smithi100 bash[21346]: cluster 2024-04-07T16:02:41.116070+0000 mon.a (mon.0) 24 : cluster 0 mgrmap e4: y(active, since 3s) 2024-04-07T16:02:42.444 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:42 smithi100 bash[21346]: audit 2024-04-07T16:02:41.487520+0000 mon.a (mon.0) 25 : audit 0 from='client.? 172.21.15.100:0/1510804586' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-07T16:02:43.694 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:43 smithi100 bash[21346]: audit 2024-04-07T16:02:43.325120+0000 mon.a (mon.0) 26 : audit 1 from='client.? 172.21.15.100:0/3188107396' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-07T16:02:43.878 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout 2024-04-07T16:02:43.878 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout [global] 2024-04-07T16:02:43.878 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout fsid = 0702e3a8-f4f8-11ee-b649-cb9ed24678a4 2024-04-07T16:02:43.878 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-07T16:02:43.878 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-07T16:02:43.878 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-07T16:02:43.878 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-07T16:02:43.878 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout 2024-04-07T16:02:43.879 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-07T16:02:43.879 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-07T16:02:43.879 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout 2024-04-07T16:02:43.880 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout [osd] 2024-04-07T16:02:43.880 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-07T16:02:43.880 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-07T16:02:43.880 INFO:teuthology.orchestra.run.smithi100.stdout:Enabling cephadm module... 2024-04-07T16:02:45.444 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:45 smithi100 bash[21346]: audit 2024-04-07T16:02:45.141819+0000 mon.a (mon.0) 27 : audit 1 from='client.? 172.21.15.100:0/2098070404' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-04-07T16:02:45.444 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:02:45 smithi100 bash[21601]: ignoring --setuser ceph since I am not root 2024-04-07T16:02:45.444 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:02:45 smithi100 bash[21601]: ignoring --setgroup ceph since I am not root 2024-04-07T16:02:45.444 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:02:45 smithi100 bash[21601]: debug 2024-04-07T16:02:45.386+0000 7f7516199200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-07T16:02:46.444 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:46 smithi100 bash[21346]: audit 2024-04-07T16:02:45.199147+0000 mon.a (mon.0) 28 : audit 1 from='client.? 172.21.15.100:0/2098070404' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-04-07T16:02:46.444 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:46 smithi100 bash[21346]: cluster 2024-04-07T16:02:45.199249+0000 mon.a (mon.0) 29 : cluster 0 mgrmap e5: y(active, since 7s) 2024-04-07T16:02:47.303 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:47 smithi100 bash[21346]: audit 2024-04-07T16:02:46.986653+0000 mon.a (mon.0) 30 : audit 0 from='client.? 172.21.15.100:0/4149905068' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-07T16:02:47.303 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:02:46 smithi100 bash[21601]: debug 2024-04-07T16:02:46.958+0000 7f7516199200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-07T16:02:47.303 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:02:47 smithi100 bash[21601]: debug 2024-04-07T16:02:47.058+0000 7f7516199200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-07T16:02:47.572 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout { 2024-04-07T16:02:47.572 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-04-07T16:02:47.572 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-07T16:02:47.572 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "active_name": "y", 2024-04-07T16:02:47.572 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-07T16:02:47.572 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout } 2024-04-07T16:02:47.572 INFO:teuthology.orchestra.run.smithi100.stdout:Waiting for the mgr to restart... 2024-04-07T16:02:47.572 INFO:teuthology.orchestra.run.smithi100.stdout:Waiting for mgr epoch 5... 2024-04-07T16:02:47.588 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:02:47 smithi100 bash[21601]: debug 2024-04-07T16:02:47.306+0000 7f7516199200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-07T16:02:48.444 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:02:48 smithi100 bash[21601]: debug 2024-04-07T16:02:48.102+0000 7f7516199200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-07T16:02:48.444 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:02:48 smithi100 bash[21601]: debug 2024-04-07T16:02:48.214+0000 7f7516199200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-07T16:02:48.444 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:02:48 smithi100 bash[21601]: debug 2024-04-07T16:02:48.322+0000 7f7516199200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-07T16:02:48.944 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:02:48 smithi100 bash[21601]: debug 2024-04-07T16:02:48.674+0000 7f7516199200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-07T16:02:48.944 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:02:48 smithi100 bash[21601]: debug 2024-04-07T16:02:48.790+0000 7f7516199200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-07T16:02:48.945 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:02:48 smithi100 bash[21601]: debug 2024-04-07T16:02:48.894+0000 7f7516199200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-07T16:02:49.694 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:02:49 smithi100 bash[21601]: debug 2024-04-07T16:02:49.246+0000 7f7516199200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-07T16:02:49.694 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:02:49 smithi100 bash[21601]: debug 2024-04-07T16:02:49.482+0000 7f7516199200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-07T16:02:50.124 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:02:49 smithi100 bash[21601]: debug 2024-04-07T16:02:49.742+0000 7f7516199200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-07T16:02:50.124 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:02:49 smithi100 bash[21601]: debug 2024-04-07T16:02:49.854+0000 7f7516199200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-07T16:02:50.444 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:02:50 smithi100 bash[21601]: debug 2024-04-07T16:02:50.126+0000 7f7516199200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-07T16:02:50.444 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:02:50 smithi100 bash[21601]: debug 2024-04-07T16:02:50.366+0000 7f7516199200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-07T16:02:51.125 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:02:50 smithi100 bash[21601]: debug 2024-04-07T16:02:50.770+0000 7f7516199200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-07T16:02:51.125 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:02:50 smithi100 bash[21601]: debug 2024-04-07T16:02:50.886+0000 7f7516199200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-07T16:02:52.444 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:02:51 smithi100 bash[21601]: debug 2024-04-07T16:02:51.130+0000 7f7516199200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-07T16:02:52.444 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:02:52 smithi100 bash[21601]: debug 2024-04-07T16:02:52.162+0000 7f7516199200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-07T16:02:52.444 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:02:52 smithi100 bash[21601]: debug 2024-04-07T16:02:52.278+0000 7f7516199200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-07T16:02:52.944 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:02:52 smithi100 bash[21601]: debug 2024-04-07T16:02:52.566+0000 7f7516199200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-07T16:02:52.944 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:02:52 smithi100 bash[21601]: debug 2024-04-07T16:02:52.678+0000 7f7516199200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-07T16:02:53.444 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:02:53 smithi100 bash[21601]: debug 2024-04-07T16:02:53.126+0000 7f7516199200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-07T16:02:53.444 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:02:53 smithi100 bash[21601]: debug 2024-04-07T16:02:53.246+0000 7f7516199200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-07T16:02:53.445 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:53 smithi100 bash[21346]: cluster 2024-04-07T16:02:53.269457+0000 mon.a (mon.0) 31 : cluster 1 Active manager daemon y restarted 2024-04-07T16:02:53.445 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:53 smithi100 bash[21346]: cluster 2024-04-07T16:02:53.270266+0000 mon.a (mon.0) 32 : cluster 1 Activating manager daemon y 2024-04-07T16:02:53.445 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:53 smithi100 bash[21346]: cluster 2024-04-07T16:02:53.276902+0000 mon.a (mon.0) 33 : cluster 0 osdmap e2: 0 total, 0 up, 0 in 2024-04-07T16:02:53.445 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:53 smithi100 bash[21346]: cluster 2024-04-07T16:02:53.277017+0000 mon.a (mon.0) 34 : cluster 0 mgrmap e6: y(active, starting, since 0.00710217s) 2024-04-07T16:02:53.445 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:53 smithi100 bash[21346]: audit 2024-04-07T16:02:53.279986+0000 mon.a (mon.0) 35 : audit 0 from='mgr.14120 172.21.15.100:0/202144723' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:02:53.445 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:53 smithi100 bash[21346]: audit 2024-04-07T16:02:53.281646+0000 mon.a (mon.0) 36 : audit 0 from='mgr.14120 172.21.15.100:0/202144723' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-07T16:02:53.445 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:53 smithi100 bash[21346]: audit 2024-04-07T16:02:53.282822+0000 mon.a (mon.0) 37 : audit 0 from='mgr.14120 172.21.15.100:0/202144723' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-07T16:02:53.445 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:53 smithi100 bash[21346]: audit 2024-04-07T16:02:53.283233+0000 mon.a (mon.0) 38 : audit 0 from='mgr.14120 172.21.15.100:0/202144723' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-07T16:02:53.445 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:53 smithi100 bash[21346]: audit 2024-04-07T16:02:53.283619+0000 mon.a (mon.0) 39 : audit 0 from='mgr.14120 172.21.15.100:0/202144723' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-07T16:02:53.445 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:53 smithi100 bash[21346]: cluster 2024-04-07T16:02:53.309681+0000 mon.a (mon.0) 40 : cluster 1 Manager daemon y is now available 2024-04-07T16:02:54.694 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:54 smithi100 bash[21346]: cephadm 2024-04-07T16:02:53.328105+0000 mgr.y (mgr.14120) 1 : cephadm 1 Found migration_current of "None". Setting to last migration. 2024-04-07T16:02:54.695 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:54 smithi100 bash[21346]: audit 2024-04-07T16:02:53.335102+0000 mon.a (mon.0) 41 : audit 1 from='mgr.14120 172.21.15.100:0/202144723' entity='mgr.y' 2024-04-07T16:02:54.695 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:54 smithi100 bash[21346]: audit 2024-04-07T16:02:53.340365+0000 mon.a (mon.0) 42 : audit 1 from='mgr.14120 172.21.15.100:0/202144723' entity='mgr.y' 2024-04-07T16:02:54.695 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:54 smithi100 bash[21346]: audit 2024-04-07T16:02:53.345293+0000 mon.a (mon.0) 43 : audit 0 from='mgr.14120 172.21.15.100:0/202144723' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:02:54.695 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:54 smithi100 bash[21346]: audit 2024-04-07T16:02:53.349445+0000 mon.a (mon.0) 44 : audit 0 from='mgr.14120 172.21.15.100:0/202144723' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:02:54.695 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:54 smithi100 bash[21346]: audit 2024-04-07T16:02:53.358336+0000 mon.a (mon.0) 45 : audit 1 from='mgr.14120 172.21.15.100:0/202144723' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-04-07T16:02:54.695 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:54 smithi100 bash[21346]: audit 2024-04-07T16:02:53.361599+0000 mon.a (mon.0) 46 : audit 1 from='mgr.14120 172.21.15.100:0/202144723' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-04-07T16:02:54.695 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:54 smithi100 bash[21346]: cluster 2024-04-07T16:02:54.283377+0000 mon.a (mon.0) 47 : cluster 0 mgrmap e7: y(active, since 1.01344s) 2024-04-07T16:02:54.921 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout { 2024-04-07T16:02:54.921 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-04-07T16:02:54.922 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-07T16:02:54.922 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout } 2024-04-07T16:02:54.922 INFO:teuthology.orchestra.run.smithi100.stdout:mgr epoch 5 is available 2024-04-07T16:02:54.922 INFO:teuthology.orchestra.run.smithi100.stdout:Setting orchestrator backend to cephadm... 2024-04-07T16:02:56.337 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:56 smithi100 bash[21346]: audit 2024-04-07T16:02:54.281266+0000 mgr.y (mgr.14120) 2 : audit 0 from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-07T16:02:56.337 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:56 smithi100 bash[21346]: audit 2024-04-07T16:02:54.293034+0000 mgr.y (mgr.14120) 3 : audit 0 from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-07T16:02:56.337 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:56 smithi100 bash[21346]: audit 2024-04-07T16:02:55.008505+0000 mon.a (mon.0) 48 : audit 1 from='mgr.14120 172.21.15.100:0/202144723' entity='mgr.y' 2024-04-07T16:02:56.337 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:56 smithi100 bash[21346]: audit 2024-04-07T16:02:55.012263+0000 mon.a (mon.0) 49 : audit 1 from='mgr.14120 172.21.15.100:0/202144723' entity='mgr.y' 2024-04-07T16:02:56.338 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:56 smithi100 bash[21346]: audit 2024-04-07T16:02:55.526590+0000 mon.a (mon.0) 50 : audit 0 from='mgr.14120 172.21.15.100:0/202144723' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:02:57.304 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:57 smithi100 bash[21346]: cephadm 2024-04-07T16:02:55.301278+0000 mgr.y (mgr.14120) 4 : cephadm 1 [07/Apr/2024:16:02:55] ENGINE Bus STARTING 2024-04-07T16:02:57.304 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:57 smithi100 bash[21346]: cephadm 2024-04-07T16:02:55.418614+0000 mgr.y (mgr.14120) 5 : cephadm 1 [07/Apr/2024:16:02:55] ENGINE Serving on https://172.21.15.100:7150 2024-04-07T16:02:57.304 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:57 smithi100 bash[21346]: cephadm 2024-04-07T16:02:55.520186+0000 mgr.y (mgr.14120) 6 : cephadm 1 [07/Apr/2024:16:02:55] ENGINE Serving on http://172.21.15.100:8765 2024-04-07T16:02:57.304 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:57 smithi100 bash[21346]: cephadm 2024-04-07T16:02:55.520383+0000 mgr.y (mgr.14120) 7 : cephadm 1 [07/Apr/2024:16:02:55] ENGINE Bus STARTED 2024-04-07T16:02:57.305 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:57 smithi100 bash[21346]: cluster 2024-04-07T16:02:56.018612+0000 mon.a (mon.0) 51 : cluster 0 mgrmap e8: y(active, since 2s) 2024-04-07T16:02:57.305 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:57 smithi100 bash[21346]: audit 2024-04-07T16:02:56.110279+0000 mgr.y (mgr.14120) 8 : audit 0 from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:02:57.305 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:57 smithi100 bash[21346]: audit 2024-04-07T16:02:56.118763+0000 mon.a (mon.0) 52 : audit 1 from='mgr.14120 172.21.15.100:0/202144723' entity='mgr.y' 2024-04-07T16:02:57.305 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:57 smithi100 bash[21346]: audit 2024-04-07T16:02:56.145348+0000 mon.a (mon.0) 53 : audit 0 from='mgr.14120 172.21.15.100:0/202144723' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:02:58.398 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout value unchanged 2024-04-07T16:02:58.398 INFO:teuthology.orchestra.run.smithi100.stdout:Generating ssh key... 2024-04-07T16:02:58.694 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:02:58 smithi100 bash[21346]: audit 2024-04-07T16:02:57.868879+0000 mgr.y (mgr.14120) 9 : audit 0 from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:02:59.863 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:02:59 smithi100 bash[21601]: Generating public/private rsa key pair. 2024-04-07T16:02:59.863 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:02:59 smithi100 bash[21601]: Your identification has been saved in /tmp/tmpgu4v7lph/key. 2024-04-07T16:02:59.863 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:02:59 smithi100 bash[21601]: Your public key has been saved in /tmp/tmpgu4v7lph/key.pub. 2024-04-07T16:02:59.863 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:02:59 smithi100 bash[21601]: The key fingerprint is: 2024-04-07T16:02:59.863 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:02:59 smithi100 bash[21601]: SHA256:0E1taKsMsRna+WehYdDxU9UBehnoAk3/++E+7OBZVV0 ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4 2024-04-07T16:02:59.863 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:02:59 smithi100 bash[21601]: The key's randomart image is: 2024-04-07T16:02:59.863 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:02:59 smithi100 bash[21601]: +---[RSA 3072]----+ 2024-04-07T16:02:59.863 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:02:59 smithi100 bash[21601]: | .+o.oo+ooE| 2024-04-07T16:02:59.864 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:02:59 smithi100 bash[21601]: | +o.=+o+ o +| 2024-04-07T16:02:59.864 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:02:59 smithi100 bash[21601]: | o.Bo.== o o| 2024-04-07T16:02:59.864 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:02:59 smithi100 bash[21601]: | . *.o.ooo .| 2024-04-07T16:02:59.864 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:02:59 smithi100 bash[21601]: | =S+.. . .| 2024-04-07T16:02:59.864 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:02:59 smithi100 bash[21601]: | = o . .| 2024-04-07T16:02:59.864 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:02:59 smithi100 bash[21601]: | o o.o | 2024-04-07T16:02:59.864 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:02:59 smithi100 bash[21601]: | . *o.| 2024-04-07T16:02:59.864 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:02:59 smithi100 bash[21601]: | oo=.| 2024-04-07T16:02:59.864 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:02:59 smithi100 bash[21601]: +----[SHA256]-----+ 2024-04-07T16:03:01.194 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:03:00 smithi100 bash[21346]: audit 2024-04-07T16:02:59.640582+0000 mgr.y (mgr.14120) 10 : audit 0 from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:03:01.194 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:03:00 smithi100 bash[21346]: cephadm 2024-04-07T16:02:59.640815+0000 mgr.y (mgr.14120) 11 : cephadm 1 Generating ssh key... 2024-04-07T16:03:01.194 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:03:00 smithi100 bash[21346]: audit 2024-04-07T16:02:59.818114+0000 mon.a (mon.0) 54 : audit 1 from='mgr.14120 172.21.15.100:0/202144723' entity='mgr.y' 2024-04-07T16:03:01.195 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:03:00 smithi100 bash[21346]: audit 2024-04-07T16:02:59.823284+0000 mon.a (mon.0) 55 : audit 1 from='mgr.14120 172.21.15.100:0/202144723' entity='mgr.y' 2024-04-07T16:03:02.191 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDLTZnpNjBQEdfmMPh6dpTaJvlFhFcpTeRueleNdDsyquChaFisbigQnPSkrOOYH6DriwTA4fhsRmKQEdG0yuilFSg2jjw2eSCvtWkl2OFzMwDG/onqH5d8BvJSFGxxu/ovvIPD7SFLuFH31Wxg1/xTv44Du9ub+WUQNqyctjDtbE4Dq2LidY5PWxYQ293vN00JgpXmqoFHUsEVGHK0gv5ZryDqKx2PjY2ZvRAjlZAx5XsID4rHcKgPYiRd4/uyYUZIl8SVSeDnL7+mv1KeaczFEe26XQQt1yUhwXwMX1FmlV1vE/9WcnRwBqU0NWd/dQADaZrXDllGxF8UlvL6Gq8V6adA2s45H3fqlDcOFxZqeFOHk8t6QDi8dtletOfb1Ou95Fop7wOFNr9lJPKKuaLscitv2MQEXpuORDi68t6z6UxzMGimh/kDc+W06MDQexwKQwZSBgcNckGK1/e17gt/SShwCtYTGdMpK1p18sw1ni4lC+SWFMOGHq7lAl/VnzU= ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4 2024-04-07T16:03:02.191 INFO:teuthology.orchestra.run.smithi100.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-04-07T16:03:02.191 INFO:teuthology.orchestra.run.smithi100.stdout:Adding key to root@localhost authorized_keys... 2024-04-07T16:03:02.191 INFO:teuthology.orchestra.run.smithi100.stdout:Adding host smithi100... 2024-04-07T16:03:02.694 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:03:02 smithi100 bash[21346]: audit 2024-04-07T16:03:01.602837+0000 mgr.y (mgr.14120) 12 : audit 0 from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:03:04.694 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:03:04 smithi100 bash[21346]: audit 2024-04-07T16:03:03.414144+0000 mgr.y (mgr.14120) 13 : audit 0 from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi100", "addr": "172.21.15.100", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:03:04.695 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:03:04 smithi100 bash[21346]: cephadm 2024-04-07T16:03:03.916075+0000 mgr.y (mgr.14120) 14 : cephadm 1 Deploying cephadm binary to smithi100 2024-04-07T16:03:06.879 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout Added host 'smithi100' with addr '172.21.15.100' 2024-04-07T16:03:06.879 INFO:teuthology.orchestra.run.smithi100.stdout:Deploying unmanaged mon service... 2024-04-07T16:03:07.444 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:03:07 smithi100 bash[21346]: audit 2024-04-07T16:03:06.099921+0000 mon.a (mon.0) 56 : audit 1 from='mgr.14120 172.21.15.100:0/202144723' entity='mgr.y' 2024-04-07T16:03:07.445 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:03:07 smithi100 bash[21346]: cephadm 2024-04-07T16:03:06.100249+0000 mgr.y (mgr.14120) 15 : cephadm 1 Added host smithi100 2024-04-07T16:03:07.445 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:03:07 smithi100 bash[21346]: audit 2024-04-07T16:03:06.140550+0000 mon.a (mon.0) 57 : audit 0 from='mgr.14120 172.21.15.100:0/202144723' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:03:09.014 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-04-07T16:03:09.014 INFO:teuthology.orchestra.run.smithi100.stdout:Deploying unmanaged mgr service... 2024-04-07T16:03:09.655 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:03:09 smithi100 bash[21346]: audit 2024-04-07T16:03:08.416424+0000 mon.a (mon.0) 58 : audit 1 from='mgr.14120 172.21.15.100:0/202144723' entity='mgr.y' 2024-04-07T16:03:09.656 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:03:09 smithi100 bash[21346]: audit 2024-04-07T16:03:08.475977+0000 mgr.y (mgr.14120) 16 : audit 0 from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:03:09.656 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:03:09 smithi100 bash[21346]: cephadm 2024-04-07T16:03:08.477336+0000 mgr.y (mgr.14120) 17 : cephadm 1 Saving service mon spec with placement count:5 2024-04-07T16:03:09.656 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:03:09 smithi100 bash[21346]: audit 2024-04-07T16:03:08.482827+0000 mon.a (mon.0) 59 : audit 1 from='mgr.14120 172.21.15.100:0/202144723' entity='mgr.y' 2024-04-07T16:03:09.656 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:03:09 smithi100 bash[21346]: audit 2024-04-07T16:03:08.801048+0000 mon.a (mon.0) 60 : audit 1 from='mgr.14120 172.21.15.100:0/202144723' entity='mgr.y' 2024-04-07T16:03:10.770 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-04-07T16:03:11.694 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:03:11 smithi100 bash[21346]: audit 2024-04-07T16:03:10.236942+0000 mgr.y (mgr.14120) 18 : audit 0 from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:03:11.694 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:03:11 smithi100 bash[21346]: cephadm 2024-04-07T16:03:10.239518+0000 mgr.y (mgr.14120) 19 : cephadm 1 Saving service mgr spec with placement count:2 2024-04-07T16:03:11.695 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:03:11 smithi100 bash[21346]: audit 2024-04-07T16:03:10.242298+0000 mon.a (mon.0) 61 : audit 1 from='mgr.14120 172.21.15.100:0/202144723' entity='mgr.y' 2024-04-07T16:03:13.188 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:03:12 smithi100 bash[21346]: audit 2024-04-07T16:03:11.940591+0000 mon.a (mon.0) 62 : audit 1 from='client.? 172.21.15.100:0/3471951600' entity='client.admin' 2024-04-07T16:03:14.314 INFO:teuthology.orchestra.run.smithi100.stdout:Enabling the dashboard module... 2024-04-07T16:03:14.973 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:03:14 smithi100 bash[21346]: cluster 2024-04-07T16:03:13.284057+0000 mgr.y (mgr.14120) 20 : cluster 0 pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-07T16:03:14.974 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:03:14 smithi100 bash[21346]: audit 2024-04-07T16:03:13.731697+0000 mon.a (mon.0) 63 : audit 1 from='client.? 172.21.15.100:0/1882244162' entity='client.admin' 2024-04-07T16:03:16.444 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:03:16 smithi100 bash[21346]: audit 2024-04-07T16:03:15.032398+0000 mon.a (mon.0) 64 : audit 1 from='mgr.14120 172.21.15.100:0/202144723' entity='mgr.y' 2024-04-07T16:03:16.445 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:03:16 smithi100 bash[21346]: audit 2024-04-07T16:03:15.664152+0000 mon.a (mon.0) 65 : audit 1 from='client.? 172.21.15.100:0/3270310450' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-04-07T16:03:16.445 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:03:16 smithi100 bash[21346]: audit 2024-04-07T16:03:15.718245+0000 mon.a (mon.0) 66 : audit 1 from='mgr.14120 172.21.15.100:0/202144723' entity='mgr.y' 2024-04-07T16:03:16.445 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:03:16 smithi100 bash[21601]: ignoring --setuser ceph since I am not root 2024-04-07T16:03:16.445 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:03:16 smithi100 bash[21601]: ignoring --setgroup ceph since I am not root 2024-04-07T16:03:16.445 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:03:16 smithi100 bash[21601]: debug 2024-04-07T16:03:16.221+0000 7f21d4871200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-07T16:03:17.341 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:03:17 smithi100 bash[21346]: audit 2024-04-07T16:03:16.042477+0000 mon.a (mon.0) 67 : audit 1 from='client.? 172.21.15.100:0/3270310450' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-04-07T16:03:17.341 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:03:17 smithi100 bash[21346]: cluster 2024-04-07T16:03:16.042527+0000 mon.a (mon.0) 68 : cluster 0 mgrmap e9: y(active, since 22s) 2024-04-07T16:03:18.068 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:03:18 smithi100 bash[21346]: audit 2024-04-07T16:03:18.010070+0000 mon.a (mon.0) 69 : audit 0 from='client.? 172.21.15.100:0/1814911948' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-07T16:03:18.068 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:03:17 smithi100 bash[21601]: debug 2024-04-07T16:03:17.801+0000 7f21d4871200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-07T16:03:18.068 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:03:17 smithi100 bash[21601]: debug 2024-04-07T16:03:17.901+0000 7f21d4871200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-07T16:03:18.445 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:03:18 smithi100 bash[21601]: debug 2024-04-07T16:03:18.161+0000 7f21d4871200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-07T16:03:18.574 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout { 2024-04-07T16:03:18.575 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-04-07T16:03:18.575 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-07T16:03:18.575 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "active_name": "y", 2024-04-07T16:03:18.575 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-07T16:03:18.575 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout } 2024-04-07T16:03:18.575 INFO:teuthology.orchestra.run.smithi100.stdout:Waiting for the mgr to restart... 2024-04-07T16:03:18.575 INFO:teuthology.orchestra.run.smithi100.stdout:Waiting for mgr epoch 9... 2024-04-07T16:03:19.206 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:03:18 smithi100 bash[21601]: debug 2024-04-07T16:03:18.957+0000 7f21d4871200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-07T16:03:19.206 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:03:19 smithi100 bash[21601]: debug 2024-04-07T16:03:19.073+0000 7f21d4871200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-07T16:03:19.206 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:03:19 smithi100 bash[21601]: debug 2024-04-07T16:03:19.177+0000 7f21d4871200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-07T16:03:19.944 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:03:19 smithi100 bash[21601]: debug 2024-04-07T16:03:19.537+0000 7f21d4871200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-07T16:03:19.944 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:03:19 smithi100 bash[21601]: debug 2024-04-07T16:03:19.653+0000 7f21d4871200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-07T16:03:19.945 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:03:19 smithi100 bash[21601]: debug 2024-04-07T16:03:19.757+0000 7f21d4871200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-07T16:03:20.444 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:03:20 smithi100 bash[21601]: debug 2024-04-07T16:03:20.109+0000 7f21d4871200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-07T16:03:20.445 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:03:20 smithi100 bash[21601]: debug 2024-04-07T16:03:20.349+0000 7f21d4871200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-07T16:03:20.944 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:03:20 smithi100 bash[21601]: debug 2024-04-07T16:03:20.609+0000 7f21d4871200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-07T16:03:20.944 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:03:20 smithi100 bash[21601]: debug 2024-04-07T16:03:20.717+0000 7f21d4871200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-07T16:03:21.444 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:03:20 smithi100 bash[21601]: debug 2024-04-07T16:03:20.993+0000 7f21d4871200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-07T16:03:21.444 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:03:21 smithi100 bash[21601]: debug 2024-04-07T16:03:21.237+0000 7f21d4871200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-07T16:03:21.944 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:03:21 smithi100 bash[21601]: debug 2024-04-07T16:03:21.641+0000 7f21d4871200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-07T16:03:21.945 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:03:21 smithi100 bash[21601]: debug 2024-04-07T16:03:21.757+0000 7f21d4871200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-07T16:03:22.407 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:03:22 smithi100 bash[21601]: debug 2024-04-07T16:03:22.005+0000 7f21d4871200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-07T16:03:23.429 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:03:23 smithi100 bash[21601]: debug 2024-04-07T16:03:23.021+0000 7f21d4871200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-07T16:03:23.429 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:03:23 smithi100 bash[21601]: debug 2024-04-07T16:03:23.141+0000 7f21d4871200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-07T16:03:23.429 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:03:23 smithi100 bash[21601]: debug 2024-04-07T16:03:23.433+0000 7f21d4871200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-07T16:03:23.694 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:03:23 smithi100 bash[21601]: debug 2024-04-07T16:03:23.549+0000 7f21d4871200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-07T16:03:24.252 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:03:24 smithi100 bash[21601]: debug 2024-04-07T16:03:23.997+0000 7f21d4871200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-07T16:03:24.252 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:03:24 smithi100 bash[21601]: debug 2024-04-07T16:03:24.121+0000 7f21d4871200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-07T16:03:24.252 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:03:24 smithi100 bash[21346]: cluster 2024-04-07T16:03:24.146042+0000 mon.a (mon.0) 70 : cluster 1 Active manager daemon y restarted 2024-04-07T16:03:24.252 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:03:24 smithi100 bash[21346]: cluster 2024-04-07T16:03:24.146795+0000 mon.a (mon.0) 71 : cluster 1 Activating manager daemon y 2024-04-07T16:03:24.252 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:03:24 smithi100 bash[21346]: cluster 2024-04-07T16:03:24.153657+0000 mon.a (mon.0) 72 : cluster 0 osdmap e3: 0 total, 0 up, 0 in 2024-04-07T16:03:24.253 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:03:24 smithi100 bash[21346]: cluster 2024-04-07T16:03:24.153902+0000 mon.a (mon.0) 73 : cluster 0 mgrmap e10: y(active, starting, since 0.00745461s) 2024-04-07T16:03:24.253 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:03:24 smithi100 bash[21346]: audit 2024-04-07T16:03:24.154737+0000 mon.a (mon.0) 74 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:03:24.253 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:03:24 smithi100 bash[21346]: audit 2024-04-07T16:03:24.157028+0000 mon.a (mon.0) 75 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-07T16:03:24.253 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:03:24 smithi100 bash[21346]: audit 2024-04-07T16:03:24.157802+0000 mon.a (mon.0) 76 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-07T16:03:24.253 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:03:24 smithi100 bash[21346]: audit 2024-04-07T16:03:24.157946+0000 mon.a (mon.0) 77 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-07T16:03:24.253 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:03:24 smithi100 bash[21346]: audit 2024-04-07T16:03:24.158115+0000 mon.a (mon.0) 78 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-07T16:03:24.253 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:03:24 smithi100 bash[21346]: cluster 2024-04-07T16:03:24.190016+0000 mon.a (mon.0) 79 : cluster 1 Manager daemon y is now available 2024-04-07T16:03:25.444 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:03:25 smithi100 bash[21346]: audit 2024-04-07T16:03:24.213311+0000 mon.a (mon.0) 80 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:03:25.445 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:03:25 smithi100 bash[21346]: audit 2024-04-07T16:03:24.234298+0000 mon.a (mon.0) 81 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-04-07T16:03:25.445 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:03:25 smithi100 bash[21346]: audit 2024-04-07T16:03:24.237787+0000 mon.a (mon.0) 82 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-04-07T16:03:25.445 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:03:25 smithi100 bash[21346]: cluster 2024-04-07T16:03:25.155973+0000 mon.a (mon.0) 83 : cluster 0 mgrmap e11: y(active, since 1.00953s) 2024-04-07T16:03:25.745 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout { 2024-04-07T16:03:25.745 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-04-07T16:03:25.745 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-07T16:03:25.745 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout } 2024-04-07T16:03:25.746 INFO:teuthology.orchestra.run.smithi100.stdout:mgr epoch 9 is available 2024-04-07T16:03:25.746 INFO:teuthology.orchestra.run.smithi100.stdout:Generating a dashboard self-signed certificate... 2024-04-07T16:03:26.694 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:03:26 smithi100 bash[21346]: cephadm 2024-04-07T16:03:25.229354+0000 mgr.y (mgr.14152) 3 : cephadm 1 [07/Apr/2024:16:03:25] ENGINE Bus STARTING 2024-04-07T16:03:26.694 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:03:26 smithi100 bash[21346]: cephadm 2024-04-07T16:03:25.343976+0000 mgr.y (mgr.14152) 4 : cephadm 1 [07/Apr/2024:16:03:25] ENGINE Serving on https://172.21.15.100:7150 2024-04-07T16:03:26.695 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:03:26 smithi100 bash[21346]: cephadm 2024-04-07T16:03:25.445235+0000 mgr.y (mgr.14152) 5 : cephadm 1 [07/Apr/2024:16:03:25] ENGINE Serving on http://172.21.15.100:8765 2024-04-07T16:03:26.695 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:03:26 smithi100 bash[21346]: cephadm 2024-04-07T16:03:25.445368+0000 mgr.y (mgr.14152) 6 : cephadm 1 [07/Apr/2024:16:03:25] ENGINE Bus STARTED 2024-04-07T16:03:27.445 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:03:27 smithi100 bash[21346]: cluster 2024-04-07T16:03:26.207827+0000 mon.a (mon.0) 84 : cluster 0 mgrmap e12: y(active, since 2s) 2024-04-07T16:03:27.445 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:03:27 smithi100 bash[21346]: audit 2024-04-07T16:03:26.944673+0000 mgr.y (mgr.14152) 7 : audit 0 from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:03:27.445 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:03:27 smithi100 bash[21346]: audit 2024-04-07T16:03:27.078605+0000 mon.a (mon.0) 85 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:03:27.445 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:03:27 smithi100 bash[21346]: audit 2024-04-07T16:03:27.085679+0000 mon.a (mon.0) 86 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:03:27.683 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-04-07T16:03:27.684 INFO:teuthology.orchestra.run.smithi100.stdout:Creating initial admin user... 2024-04-07T16:03:29.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:03:29 smithi100 bash[21346]: audit 2024-04-07T16:03:28.938339+0000 mgr.y (mgr.14152) 8 : audit 0 from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:03:29.763 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$uLgELAW69nOaSoN1FVg.gOPZqB7LjnJBVmxoePknONtT/2Wj5eoiK", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1712505809, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-04-07T16:03:29.764 INFO:teuthology.orchestra.run.smithi100.stdout:Fetching dashboard port number... 2024-04-07T16:03:30.444 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:03:30 smithi100 bash[21346]: audit 2024-04-07T16:03:29.190820+0000 mon.a (mon.0) 87 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:03:31.577 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout 8443 2024-04-07T16:03:31.577 INFO:teuthology.orchestra.run.smithi100.stdout:firewalld does not appear to be present 2024-04-07T16:03:31.577 INFO:teuthology.orchestra.run.smithi100.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-04-07T16:03:31.581 INFO:teuthology.orchestra.run.smithi100.stdout:Ceph Dashboard is now available at: 2024-04-07T16:03:31.581 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-07T16:03:31.581 INFO:teuthology.orchestra.run.smithi100.stdout: URL: https://smithi100.front.sepia.ceph.com:8443/ 2024-04-07T16:03:31.581 INFO:teuthology.orchestra.run.smithi100.stdout: User: admin 2024-04-07T16:03:31.581 INFO:teuthology.orchestra.run.smithi100.stdout: Password: tqv5zn48a9 2024-04-07T16:03:31.581 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-07T16:03:31.581 INFO:teuthology.orchestra.run.smithi100.stdout:Saving cluster configuration to /var/lib/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/config directory 2024-04-07T16:03:31.582 INFO:teuthology.orchestra.run.smithi100.stdout:Enabling autotune for osd_memory_target 2024-04-07T16:03:32.194 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:03:31 smithi100 bash[21346]: audit 2024-04-07T16:03:30.929835+0000 mon.a (mon.0) 88 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:03:32.194 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:03:31 smithi100 bash[21346]: audit 2024-04-07T16:03:31.019487+0000 mon.a (mon.0) 89 : audit 0 from='client.? 172.21.15.100:0/3653820594' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-04-07T16:03:32.195 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:03:31 smithi100 bash[21346]: cluster 2024-04-07T16:03:31.194353+0000 mon.a (mon.0) 90 : cluster 0 mgrmap e13: y(active, since 7s) 2024-04-07T16:03:32.195 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:03:31 smithi100 bash[21346]: audit 2024-04-07T16:03:31.666335+0000 mon.a (mon.0) 91 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:03:35.439 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-04-07T16:03:35.439 INFO:teuthology.orchestra.run.smithi100.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-04-07T16:03:35.439 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-07T16:03:35.439 INFO:teuthology.orchestra.run.smithi100.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 0702e3a8-f4f8-11ee-b649-cb9ed24678a4 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-04-07T16:03:35.439 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-07T16:03:35.440 INFO:teuthology.orchestra.run.smithi100.stdout:Or, if you are only running a single cluster on this host: 2024-04-07T16:03:35.440 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-07T16:03:35.440 INFO:teuthology.orchestra.run.smithi100.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-04-07T16:03:35.440 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-07T16:03:35.440 INFO:teuthology.orchestra.run.smithi100.stdout:Please consider enabling telemetry to help improve Ceph: 2024-04-07T16:03:35.440 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-07T16:03:35.440 INFO:teuthology.orchestra.run.smithi100.stdout: ceph telemetry on 2024-04-07T16:03:35.440 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-07T16:03:35.440 INFO:teuthology.orchestra.run.smithi100.stdout:For more information see: 2024-04-07T16:03:35.440 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-07T16:03:35.440 INFO:teuthology.orchestra.run.smithi100.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-04-07T16:03:35.440 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-07T16:03:35.440 INFO:teuthology.orchestra.run.smithi100.stdout:Bootstrap complete. 2024-04-07T16:03:35.471 INFO:tasks.cephadm:Fetching config... 2024-04-07T16:03:35.471 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-07T16:03:35.472 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-04-07T16:03:35.477 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-04-07T16:03:35.477 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-07T16:03:35.477 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-04-07T16:03:35.526 INFO:tasks.cephadm:Fetching mon keyring... 2024-04-07T16:03:35.527 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-07T16:03:35.527 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/var/lib/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/mon.a/keyring of=/dev/stdout 2024-04-07T16:03:35.581 INFO:tasks.cephadm:Fetching pub ssh key... 2024-04-07T16:03:35.581 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-07T16:03:35.581 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-04-07T16:03:35.627 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-04-07T16:03:35.627 DEBUG:teuthology.orchestra.run.smithi100:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDLTZnpNjBQEdfmMPh6dpTaJvlFhFcpTeRueleNdDsyquChaFisbigQnPSkrOOYH6DriwTA4fhsRmKQEdG0yuilFSg2jjw2eSCvtWkl2OFzMwDG/onqH5d8BvJSFGxxu/ovvIPD7SFLuFH31Wxg1/xTv44Du9ub+WUQNqyctjDtbE4Dq2LidY5PWxYQ293vN00JgpXmqoFHUsEVGHK0gv5ZryDqKx2PjY2ZvRAjlZAx5XsID4rHcKgPYiRd4/uyYUZIl8SVSeDnL7+mv1KeaczFEe26XQQt1yUhwXwMX1FmlV1vE/9WcnRwBqU0NWd/dQADaZrXDllGxF8UlvL6Gq8V6adA2s45H3fqlDcOFxZqeFOHk8t6QDi8dtletOfb1Ou95Fop7wOFNr9lJPKKuaLscitv2MQEXpuORDi68t6z6UxzMGimh/kDc+W06MDQexwKQwZSBgcNckGK1/e17gt/SShwCtYTGdMpK1p18sw1ni4lC+SWFMOGHq7lAl/VnzU= ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-07T16:03:35.691 INFO:teuthology.orchestra.run.smithi100.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDLTZnpNjBQEdfmMPh6dpTaJvlFhFcpTeRueleNdDsyquChaFisbigQnPSkrOOYH6DriwTA4fhsRmKQEdG0yuilFSg2jjw2eSCvtWkl2OFzMwDG/onqH5d8BvJSFGxxu/ovvIPD7SFLuFH31Wxg1/xTv44Du9ub+WUQNqyctjDtbE4Dq2LidY5PWxYQ293vN00JgpXmqoFHUsEVGHK0gv5ZryDqKx2PjY2ZvRAjlZAx5XsID4rHcKgPYiRd4/uyYUZIl8SVSeDnL7+mv1KeaczFEe26XQQt1yUhwXwMX1FmlV1vE/9WcnRwBqU0NWd/dQADaZrXDllGxF8UlvL6Gq8V6adA2s45H3fqlDcOFxZqeFOHk8t6QDi8dtletOfb1Ou95Fop7wOFNr9lJPKKuaLscitv2MQEXpuORDi68t6z6UxzMGimh/kDc+W06MDQexwKQwZSBgcNckGK1/e17gt/SShwCtYTGdMpK1p18sw1ni4lC+SWFMOGHq7lAl/VnzU= ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4 2024-04-07T16:03:35.699 DEBUG:teuthology.orchestra.run.smithi144:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDLTZnpNjBQEdfmMPh6dpTaJvlFhFcpTeRueleNdDsyquChaFisbigQnPSkrOOYH6DriwTA4fhsRmKQEdG0yuilFSg2jjw2eSCvtWkl2OFzMwDG/onqH5d8BvJSFGxxu/ovvIPD7SFLuFH31Wxg1/xTv44Du9ub+WUQNqyctjDtbE4Dq2LidY5PWxYQ293vN00JgpXmqoFHUsEVGHK0gv5ZryDqKx2PjY2ZvRAjlZAx5XsID4rHcKgPYiRd4/uyYUZIl8SVSeDnL7+mv1KeaczFEe26XQQt1yUhwXwMX1FmlV1vE/9WcnRwBqU0NWd/dQADaZrXDllGxF8UlvL6Gq8V6adA2s45H3fqlDcOFxZqeFOHk8t6QDi8dtletOfb1Ou95Fop7wOFNr9lJPKKuaLscitv2MQEXpuORDi68t6z6UxzMGimh/kDc+W06MDQexwKQwZSBgcNckGK1/e17gt/SShwCtYTGdMpK1p18sw1ni4lC+SWFMOGHq7lAl/VnzU= ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-07T16:03:35.723 INFO:teuthology.orchestra.run.smithi144.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDLTZnpNjBQEdfmMPh6dpTaJvlFhFcpTeRueleNdDsyquChaFisbigQnPSkrOOYH6DriwTA4fhsRmKQEdG0yuilFSg2jjw2eSCvtWkl2OFzMwDG/onqH5d8BvJSFGxxu/ovvIPD7SFLuFH31Wxg1/xTv44Du9ub+WUQNqyctjDtbE4Dq2LidY5PWxYQ293vN00JgpXmqoFHUsEVGHK0gv5ZryDqKx2PjY2ZvRAjlZAx5XsID4rHcKgPYiRd4/uyYUZIl8SVSeDnL7+mv1KeaczFEe26XQQt1yUhwXwMX1FmlV1vE/9WcnRwBqU0NWd/dQADaZrXDllGxF8UlvL6Gq8V6adA2s45H3fqlDcOFxZqeFOHk8t6QDi8dtletOfb1Ou95Fop7wOFNr9lJPKKuaLscitv2MQEXpuORDi68t6z6UxzMGimh/kDc+W06MDQexwKQwZSBgcNckGK1/e17gt/SShwCtYTGdMpK1p18sw1ni4lC+SWFMOGHq7lAl/VnzU= ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4 2024-04-07T16:03:35.732 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0702e3a8-f4f8-11ee-b649-cb9ed24678a4 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-04-07T16:03:36.194 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:03:35 smithi100 bash[21346]: audit 2024-04-07T16:03:34.829062+0000 mon.a (mon.0) 92 : audit 1 from='client.? 172.21.15.100:0/4188958735' entity='client.admin' 2024-04-07T16:03:40.680 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/mon.a/config 2024-04-07T16:03:44.013 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-04-07T16:03:44.013 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0702e3a8-f4f8-11ee-b649-cb9ed24678a4 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-04-07T16:03:44.014 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:03:43 smithi100 bash[21346]: audit 2024-04-07T16:03:42.710354+0000 mon.a (mon.0) 93 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:03:44.014 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:03:43 smithi100 bash[21346]: audit 2024-04-07T16:03:42.715279+0000 mon.a (mon.0) 94 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:03:44.014 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:03:43 smithi100 bash[21346]: audit 2024-04-07T16:03:42.716635+0000 mon.a (mon.0) 95 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi100", "name": "osd_memory_target"}]: dispatch 2024-04-07T16:03:44.014 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:03:43 smithi100 bash[21346]: audit 2024-04-07T16:03:42.721818+0000 mon.a (mon.0) 96 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:03:44.014 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:03:43 smithi100 bash[21346]: audit 2024-04-07T16:03:42.755444+0000 mon.a (mon.0) 97 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:03:44.014 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:03:43 smithi100 bash[21346]: audit 2024-04-07T16:03:42.762401+0000 mon.a (mon.0) 98 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:03:44.014 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:03:43 smithi100 bash[21346]: audit 2024-04-07T16:03:43.356145+0000 mon.a (mon.0) 99 : audit 1 from='client.? 172.21.15.100:0/2460384037' entity='client.admin' 2024-04-07T16:03:44.014 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:03:43 smithi100 bash[21346]: audit 2024-04-07T16:03:43.411773+0000 mon.a (mon.0) 100 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:03:44.015 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:03:43 smithi100 bash[21346]: audit 2024-04-07T16:03:43.416353+0000 mon.a (mon.0) 101 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:03:46.195 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:03:45 smithi100 bash[21346]: cluster 2024-04-07T16:03:44.159181+0000 mgr.y (mgr.14152) 9 : cluster 0 pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-07T16:03:48.194 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:03:47 smithi100 bash[21346]: cluster 2024-04-07T16:03:46.159603+0000 mgr.y (mgr.14152) 10 : cluster 0 pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-07T16:03:48.680 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/mon.a/config 2024-04-07T16:03:50.181 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:03:49 smithi100 bash[21346]: cluster 2024-04-07T16:03:48.160070+0000 mgr.y (mgr.14152) 11 : cluster 0 pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-07T16:03:51.842 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi144 2024-04-07T16:03:51.843 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-04-07T16:03:51.843 DEBUG:teuthology.orchestra.run.smithi144:> dd of=/etc/ceph/ceph.conf 2024-04-07T16:03:51.849 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-04-07T16:03:51.849 DEBUG:teuthology.orchestra.run.smithi144:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-07T16:03:51.897 INFO:tasks.cephadm:Adding host smithi144 to orchestrator... 2024-04-07T16:03:51.898 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0702e3a8-f4f8-11ee-b649-cb9ed24678a4 -- ceph orch host add smithi144 2024-04-07T16:03:52.195 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:03:51 smithi100 bash[21346]: cluster 2024-04-07T16:03:50.160449+0000 mgr.y (mgr.14152) 12 : cluster 0 pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-07T16:03:52.195 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:03:51 smithi100 bash[21346]: audit 2024-04-07T16:03:51.231238+0000 mon.a (mon.0) 102 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:03:52.195 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:03:51 smithi100 bash[21346]: audit 2024-04-07T16:03:51.265672+0000 mon.a (mon.0) 103 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:03:52.195 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:03:51 smithi100 bash[21346]: audit 2024-04-07T16:03:51.267243+0000 mon.a (mon.0) 104 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:03:52.195 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:03:51 smithi100 bash[21346]: audit 2024-04-07T16:03:51.268033+0000 mon.a (mon.0) 105 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:03:53.195 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:03:52 smithi100 bash[21346]: audit 2024-04-07T16:03:51.225492+0000 mgr.y (mgr.14152) 13 : audit 0 from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:03:53.195 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:03:52 smithi100 bash[21346]: cephadm 2024-04-07T16:03:51.269266+0000 mgr.y (mgr.14152) 14 : cephadm 1 Updating smithi100:/etc/ceph/ceph.conf 2024-04-07T16:03:53.195 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:03:52 smithi100 bash[21346]: cephadm 2024-04-07T16:03:51.383235+0000 mgr.y (mgr.14152) 15 : cephadm 1 Updating smithi100:/var/lib/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/config/ceph.conf 2024-04-07T16:03:53.195 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:03:52 smithi100 bash[21346]: cephadm 2024-04-07T16:03:51.497881+0000 mgr.y (mgr.14152) 16 : cephadm 1 Updating smithi100:/etc/ceph/ceph.client.admin.keyring 2024-04-07T16:03:53.196 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:03:52 smithi100 bash[21346]: cephadm 2024-04-07T16:03:51.612244+0000 mgr.y (mgr.14152) 17 : cephadm 1 Updating smithi100:/var/lib/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/config/ceph.client.admin.keyring 2024-04-07T16:03:53.196 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:03:52 smithi100 bash[21346]: audit 2024-04-07T16:03:51.728958+0000 mon.a (mon.0) 106 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:03:53.196 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:03:52 smithi100 bash[21346]: audit 2024-04-07T16:03:51.731227+0000 mon.a (mon.0) 107 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:03:53.196 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:03:52 smithi100 bash[21346]: audit 2024-04-07T16:03:51.733551+0000 mon.a (mon.0) 108 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:03:54.195 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:03:53 smithi100 bash[21346]: cluster 2024-04-07T16:03:52.160861+0000 mgr.y (mgr.14152) 18 : cluster 0 pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-07T16:03:56.195 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:03:55 smithi100 bash[21346]: cluster 2024-04-07T16:03:54.161367+0000 mgr.y (mgr.14152) 19 : cluster 0 pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-07T16:03:56.559 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/mon.a/config 2024-04-07T16:03:57.945 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:03:57 smithi100 bash[21346]: cluster 2024-04-07T16:03:56.161894+0000 mgr.y (mgr.14152) 20 : cluster 0 pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-07T16:04:00.195 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:03:59 smithi100 bash[21346]: cluster 2024-04-07T16:03:58.162435+0000 mgr.y (mgr.14152) 21 : cluster 0 pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-07T16:04:01.195 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:00 smithi100 bash[21346]: audit 2024-04-07T16:03:59.180236+0000 mgr.y (mgr.14152) 22 : audit 0 from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi144", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:04:01.195 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:00 smithi100 bash[21346]: cephadm 2024-04-07T16:03:59.825244+0000 mgr.y (mgr.14152) 23 : cephadm 1 Deploying cephadm binary to smithi144 2024-04-07T16:04:02.195 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:01 smithi100 bash[21346]: cluster 2024-04-07T16:04:00.162863+0000 mgr.y (mgr.14152) 24 : cluster 0 pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-07T16:04:02.405 INFO:teuthology.orchestra.run.smithi100.stdout:Added host 'smithi144' with addr '172.21.15.144' 2024-04-07T16:04:03.042 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0702e3a8-f4f8-11ee-b649-cb9ed24678a4 -- ceph orch host ls --format=json 2024-04-07T16:04:03.695 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:03 smithi100 bash[21346]: cluster 2024-04-07T16:04:02.163278+0000 mgr.y (mgr.14152) 25 : cluster 0 pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-07T16:04:03.695 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:03 smithi100 bash[21346]: audit 2024-04-07T16:04:02.411906+0000 mon.a (mon.0) 109 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:04:03.695 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:03 smithi100 bash[21346]: cephadm 2024-04-07T16:04:02.412681+0000 mgr.y (mgr.14152) 26 : cephadm 1 Added host smithi144 2024-04-07T16:04:03.696 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:03 smithi100 bash[21346]: audit 2024-04-07T16:04:02.439622+0000 mon.a (mon.0) 110 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:04:03.696 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:03 smithi100 bash[21346]: audit 2024-04-07T16:04:02.763112+0000 mon.a (mon.0) 111 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:04:05.445 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:05 smithi100 bash[21346]: audit 2024-04-07T16:04:04.137525+0000 mon.a (mon.0) 112 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:04:05.445 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:05 smithi100 bash[21346]: cluster 2024-04-07T16:04:04.163715+0000 mgr.y (mgr.14152) 27 : cluster 0 pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-07T16:04:05.445 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:05 smithi100 bash[21346]: audit 2024-04-07T16:04:05.038812+0000 mon.a (mon.0) 113 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:04:07.695 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:07 smithi100 bash[21346]: cluster 2024-04-07T16:04:06.164160+0000 mgr.y (mgr.14152) 28 : cluster 0 pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-07T16:04:07.711 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/mon.a/config 2024-04-07T16:04:09.695 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:09 smithi100 bash[21346]: cluster 2024-04-07T16:04:08.164579+0000 mgr.y (mgr.14152) 29 : cluster 0 pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-07T16:04:10.315 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-07T16:04:10.315 INFO:teuthology.orchestra.run.smithi100.stdout:[{"addr": "172.21.15.100", "hostname": "smithi100", "labels": [], "status": ""}, {"addr": "172.21.15.144", "hostname": "smithi144", "labels": [], "status": ""}] 2024-04-07T16:04:10.969 INFO:tasks.cephadm:Setting crush tunables to default 2024-04-07T16:04:10.970 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0702e3a8-f4f8-11ee-b649-cb9ed24678a4 -- ceph osd crush tunables default 2024-04-07T16:04:11.446 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:11 smithi100 bash[21346]: audit 2024-04-07T16:04:10.073228+0000 mon.a (mon.0) 114 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:04:11.446 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:11 smithi100 bash[21346]: audit 2024-04-07T16:04:10.079392+0000 mon.a (mon.0) 115 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:04:11.446 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:11 smithi100 bash[21346]: audit 2024-04-07T16:04:10.086422+0000 mon.a (mon.0) 116 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:04:11.446 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:11 smithi100 bash[21346]: audit 2024-04-07T16:04:10.092286+0000 mon.a (mon.0) 117 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:04:11.446 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:11 smithi100 bash[21346]: audit 2024-04-07T16:04:10.093949+0000 mon.a (mon.0) 118 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi144", "name": "osd_memory_target"}]: dispatch 2024-04-07T16:04:11.446 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:11 smithi100 bash[21346]: audit 2024-04-07T16:04:10.095333+0000 mon.a (mon.0) 119 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:04:11.446 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:11 smithi100 bash[21346]: audit 2024-04-07T16:04:10.096457+0000 mon.a (mon.0) 120 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:04:11.446 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:11 smithi100 bash[21346]: cephadm 2024-04-07T16:04:10.098337+0000 mgr.y (mgr.14152) 30 : cephadm 1 Updating smithi144:/etc/ceph/ceph.conf 2024-04-07T16:04:11.446 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:11 smithi100 bash[21346]: audit 2024-04-07T16:04:10.660456+0000 mon.a (mon.0) 121 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:04:11.446 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:11 smithi100 bash[21346]: audit 2024-04-07T16:04:10.664591+0000 mon.a (mon.0) 122 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:04:11.446 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:11 smithi100 bash[21346]: audit 2024-04-07T16:04:10.668809+0000 mon.a (mon.0) 123 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:04:12.445 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:12 smithi100 bash[21346]: cluster 2024-04-07T16:04:10.164861+0000 mgr.y (mgr.14152) 31 : cluster 0 pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-07T16:04:12.445 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:12 smithi100 bash[21346]: cephadm 2024-04-07T16:04:10.212773+0000 mgr.y (mgr.14152) 32 : cephadm 1 Updating smithi144:/var/lib/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/config/ceph.conf 2024-04-07T16:04:12.446 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:12 smithi100 bash[21346]: audit 2024-04-07T16:04:10.321884+0000 mgr.y (mgr.14152) 33 : audit 0 from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-07T16:04:12.446 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:12 smithi100 bash[21346]: cephadm 2024-04-07T16:04:10.327034+0000 mgr.y (mgr.14152) 34 : cephadm 1 Updating smithi144:/etc/ceph/ceph.client.admin.keyring 2024-04-07T16:04:12.446 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:12 smithi100 bash[21346]: cephadm 2024-04-07T16:04:10.491351+0000 mgr.y (mgr.14152) 35 : cephadm 1 Updating smithi144:/var/lib/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/config/ceph.client.admin.keyring 2024-04-07T16:04:14.445 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:14 smithi100 bash[21346]: cluster 2024-04-07T16:04:12.165232+0000 mgr.y (mgr.14152) 36 : cluster 0 pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-07T16:04:15.646 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/mon.a/config 2024-04-07T16:04:16.366 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:16 smithi100 bash[21346]: cluster 2024-04-07T16:04:14.165631+0000 mgr.y (mgr.14152) 37 : cluster 0 pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-07T16:04:18.445 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:18 smithi100 bash[21346]: cluster 2024-04-07T16:04:16.166058+0000 mgr.y (mgr.14152) 38 : cluster 0 pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-07T16:04:19.103 INFO:teuthology.orchestra.run.smithi100.stderr:adjusted tunables profile to default 2024-04-07T16:04:19.445 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:19 smithi100 bash[21346]: audit 2024-04-07T16:04:18.152414+0000 mon.a (mon.0) 124 : audit 1 from='client.? 172.21.15.100:0/860722094' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-07T16:04:19.753 INFO:tasks.cephadm:Adding mon.a on smithi100 2024-04-07T16:04:19.753 INFO:tasks.cephadm:Adding mon.c on smithi100 2024-04-07T16:04:19.754 INFO:tasks.cephadm:Adding mon.b on smithi144 2024-04-07T16:04:19.754 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0702e3a8-f4f8-11ee-b649-cb9ed24678a4 -- ceph orch apply mon '3;smithi100:172.21.15.100=a;smithi100:[v2:172.21.15.100:3301,v1:172.21.15.100:6790]=c;smithi144:172.21.15.144=b' 2024-04-07T16:04:20.445 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:20 smithi100 bash[21346]: cluster 2024-04-07T16:04:18.166463+0000 mgr.y (mgr.14152) 39 : cluster 0 pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-07T16:04:20.445 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:20 smithi100 bash[21346]: audit 2024-04-07T16:04:19.106485+0000 mon.a (mon.0) 125 : audit 1 from='client.? 172.21.15.100:0/860722094' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-07T16:04:20.445 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:20 smithi100 bash[21346]: cluster 2024-04-07T16:04:19.106643+0000 mon.a (mon.0) 126 : cluster 0 osdmap e4: 0 total, 0 up, 0 in 2024-04-07T16:04:20.920 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/config/ceph.conf 2024-04-07T16:04:22.445 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:22 smithi100 bash[21346]: cluster 2024-04-07T16:04:20.166864+0000 mgr.y (mgr.14152) 40 : cluster 0 pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-07T16:04:23.292 INFO:teuthology.orchestra.run.smithi144.stdout:Scheduled mon update... 2024-04-07T16:04:23.978 DEBUG:teuthology.orchestra.run.smithi100:mon.c> sudo journalctl -f -n 0 -u ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@mon.c.service 2024-04-07T16:04:23.981 DEBUG:teuthology.orchestra.run.smithi144:mon.b> sudo journalctl -f -n 0 -u ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@mon.b.service 2024-04-07T16:04:23.983 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-07T16:04:23.983 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0702e3a8-f4f8-11ee-b649-cb9ed24678a4 -- ceph mon dump -f json 2024-04-07T16:04:24.445 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:24 smithi100 bash[21346]: cluster 2024-04-07T16:04:22.167236+0000 mgr.y (mgr.14152) 41 : cluster 0 pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-07T16:04:24.446 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:24 smithi100 bash[21346]: audit 2024-04-07T16:04:23.298679+0000 mon.a (mon.0) 127 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:04:24.446 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:24 smithi100 bash[21346]: audit 2024-04-07T16:04:23.331312+0000 mon.a (mon.0) 128 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:04:24.446 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:24 smithi100 bash[21346]: audit 2024-04-07T16:04:23.334984+0000 mon.a (mon.0) 129 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:04:24.446 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:24 smithi100 bash[21346]: audit 2024-04-07T16:04:23.336706+0000 mon.a (mon.0) 130 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:04:24.446 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:24 smithi100 bash[21346]: audit 2024-04-07T16:04:23.345012+0000 mon.a (mon.0) 131 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:04:24.446 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:24 smithi100 bash[21346]: audit 2024-04-07T16:04:23.349012+0000 mon.a (mon.0) 132 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-07T16:04:24.446 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:24 smithi100 bash[21346]: audit 2024-04-07T16:04:23.350659+0000 mon.a (mon.0) 133 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:04:25.445 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:25 smithi100 bash[21346]: audit 2024-04-07T16:04:23.287209+0000 mgr.y (mgr.14152) 42 : audit 0 from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi100:172.21.15.100=a;smithi100:[v2:172.21.15.100:3301,v1:172.21.15.100:6790]=c;smithi144:172.21.15.144=b", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:04:25.445 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:25 smithi100 bash[21346]: cephadm 2024-04-07T16:04:23.293171+0000 mgr.y (mgr.14152) 43 : cephadm 1 Saving service mon spec with placement smithi100:172.21.15.100=a;smithi100:[v2:172.21.15.100:3301,v1:172.21.15.100:6790]=c;smithi144:172.21.15.144=b;count:3 2024-04-07T16:04:25.445 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:25 smithi100 bash[21346]: cephadm 2024-04-07T16:04:23.352585+0000 mgr.y (mgr.14152) 44 : cephadm 1 Deploying daemon mon.b on smithi144 2024-04-07T16:04:25.567 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/config/ceph.conf 2024-04-07T16:04:26.445 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:26 smithi100 bash[21346]: cluster 2024-04-07T16:04:24.167630+0000 mgr.y (mgr.14152) 45 : cluster 0 pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-07T16:04:27.445 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:27 smithi100 bash[21346]: cluster 2024-04-07T16:04:26.168087+0000 mgr.y (mgr.14152) 46 : cluster 0 pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-07T16:04:28.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:28 smithi144 systemd[1]: Started Ceph mon.b for 0702e3a8-f4f8-11ee-b649-cb9ed24678a4. 2024-04-07T16:04:28.508 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-07T16:04:28.508 INFO:teuthology.orchestra.run.smithi144.stdout:{"epoch":1,"fsid":"0702e3a8-f4f8-11ee-b649-cb9ed24678a4","modified":"2024-04-07T16:02:17.280630Z","created":"2024-04-07T16:02:17.280630Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:3300","nonce":0},{"type":"v1","addr":"172.21.15.100:6789","nonce":0}]},"addr":"172.21.15.100:6789/0","public_addr":"172.21.15.100:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-07T16:04:28.511 INFO:teuthology.orchestra.run.smithi144.stderr:dumped monmap epoch 1 2024-04-07T16:04:29.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:29 smithi100 bash[21346]: audit 2024-04-07T16:04:28.104941+0000 mon.a (mon.0) 134 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:04:29.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:29 smithi100 bash[21346]: audit 2024-04-07T16:04:28.111559+0000 mon.a (mon.0) 135 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:04:29.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:29 smithi100 bash[21346]: audit 2024-04-07T16:04:28.134187+0000 mon.a (mon.0) 136 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:04:29.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:29 smithi100 bash[21346]: audit 2024-04-07T16:04:28.135508+0000 mon.a (mon.0) 137 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-07T16:04:29.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:29 smithi100 bash[21346]: audit 2024-04-07T16:04:28.137241+0000 mon.a (mon.0) 138 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:04:29.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:29 smithi100 bash[21346]: audit 2024-04-07T16:04:28.514999+0000 mon.a (mon.0) 139 : audit 0 from='client.? 172.21.15.144:0/4129686569' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-07T16:04:29.611 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.296+0000 7f27b88a2c80 0 set uid:gid to 167:167 (ceph:ceph) 2024-04-07T16:04:29.611 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.296+0000 7f27b88a2c80 0 ceph version 18.2.2-1171-g267c8ff5 (267c8ff5b1ff5815581b3b2381d19360e081b37b) reef (stable), process ceph-mon, pid 7 2024-04-07T16:04:29.611 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.296+0000 7f27b88a2c80 0 pidfile_write: ignore empty --pid-file 2024-04-07T16:04:29.611 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 0 load: jerasure load: lrc 2024-04-07T16:04:29.611 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: RocksDB version: 7.9.2 2024-04-07T16:04:29.611 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Git sha 0 2024-04-07T16:04:29.611 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Compile date 2024-04-06 15:05:06 2024-04-07T16:04:29.611 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: DB SUMMARY 2024-04-07T16:04:29.611 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: DB Session ID: Z7QIPSO085COQQ7SQO4S 2024-04-07T16:04:29.612 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: CURRENT file: CURRENT 2024-04-07T16:04:29.612 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: IDENTITY file: IDENTITY 2024-04-07T16:04:29.612 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-04-07T16:04:29.612 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-04-07T16:04:29.612 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 511 ; 2024-04-07T16:04:29.612 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.error_if_exists: 0 2024-04-07T16:04:29.612 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.create_if_missing: 0 2024-04-07T16:04:29.612 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.paranoid_checks: 1 2024-04-07T16:04:29.612 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.flush_verify_memtable_count: 1 2024-04-07T16:04:29.612 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-07T16:04:29.612 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-07T16:04:29.612 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.env: 0x55d40613f7a0 2024-04-07T16:04:29.613 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.fs: PosixFileSystem 2024-04-07T16:04:29.613 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.info_log: 0x55d407f55520 2024-04-07T16:04:29.613 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.max_file_opening_threads: 16 2024-04-07T16:04:29.613 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.statistics: (nil) 2024-04-07T16:04:29.613 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.use_fsync: 0 2024-04-07T16:04:29.613 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.max_log_file_size: 0 2024-04-07T16:04:29.614 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-07T16:04:29.614 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.log_file_time_to_roll: 0 2024-04-07T16:04:29.614 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.keep_log_file_num: 1000 2024-04-07T16:04:29.614 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.recycle_log_file_num: 0 2024-04-07T16:04:29.614 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.allow_fallocate: 1 2024-04-07T16:04:29.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.allow_mmap_reads: 0 2024-04-07T16:04:29.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.allow_mmap_writes: 0 2024-04-07T16:04:29.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.use_direct_reads: 0 2024-04-07T16:04:29.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-07T16:04:29.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.create_missing_column_families: 0 2024-04-07T16:04:29.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.db_log_dir: 2024-04-07T16:04:29.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.wal_dir: 2024-04-07T16:04:29.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.table_cache_numshardbits: 6 2024-04-07T16:04:29.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.WAL_ttl_seconds: 0 2024-04-07T16:04:29.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.WAL_size_limit_MB: 0 2024-04-07T16:04:29.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-07T16:04:29.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-07T16:04:29.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.is_fd_close_on_exec: 1 2024-04-07T16:04:29.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.advise_random_on_open: 1 2024-04-07T16:04:29.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.db_write_buffer_size: 0 2024-04-07T16:04:29.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.write_buffer_manager: 0x55d4071e8320 2024-04-07T16:04:29.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-07T16:04:29.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-07T16:04:29.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.use_adaptive_mutex: 0 2024-04-07T16:04:29.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.rate_limiter: (nil) 2024-04-07T16:04:29.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-07T16:04:29.617 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.wal_recovery_mode: 2 2024-04-07T16:04:29.617 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.enable_thread_tracking: 0 2024-04-07T16:04:29.617 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.enable_pipelined_write: 0 2024-04-07T16:04:29.617 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.unordered_write: 0 2024-04-07T16:04:29.617 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-07T16:04:29.617 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-07T16:04:29.617 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-07T16:04:29.617 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-07T16:04:29.617 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.row_cache: None 2024-04-07T16:04:29.617 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.wal_filter: None 2024-04-07T16:04:29.617 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-07T16:04:29.617 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.allow_ingest_behind: 0 2024-04-07T16:04:29.617 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.two_write_queues: 0 2024-04-07T16:04:29.617 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.manual_wal_flush: 0 2024-04-07T16:04:29.618 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.wal_compression: 0 2024-04-07T16:04:29.618 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.atomic_flush: 0 2024-04-07T16:04:29.618 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-07T16:04:29.618 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.persist_stats_to_disk: 0 2024-04-07T16:04:29.618 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.write_dbid_to_manifest: 0 2024-04-07T16:04:29.618 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.log_readahead_size: 0 2024-04-07T16:04:29.618 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-07T16:04:29.618 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.best_efforts_recovery: 0 2024-04-07T16:04:29.618 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-07T16:04:29.618 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-07T16:04:29.618 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.allow_data_in_errors: 0 2024-04-07T16:04:29.618 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.db_host_id: __hostname__ 2024-04-07T16:04:29.619 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.enforce_single_del_contracts: true 2024-04-07T16:04:29.619 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.max_background_jobs: 2 2024-04-07T16:04:29.619 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.max_background_compactions: -1 2024-04-07T16:04:29.619 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.max_subcompactions: 1 2024-04-07T16:04:29.619 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-07T16:04:29.619 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-07T16:04:29.619 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.delayed_write_rate : 16777216 2024-04-07T16:04:29.619 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.max_total_wal_size: 0 2024-04-07T16:04:29.619 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-07T16:04:29.619 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.stats_dump_period_sec: 600 2024-04-07T16:04:29.619 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.stats_persist_period_sec: 600 2024-04-07T16:04:29.619 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-07T16:04:29.620 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.max_open_files: -1 2024-04-07T16:04:29.620 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.bytes_per_sync: 0 2024-04-07T16:04:29.620 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.wal_bytes_per_sync: 0 2024-04-07T16:04:29.620 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.strict_bytes_per_sync: 0 2024-04-07T16:04:29.620 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.compaction_readahead_size: 0 2024-04-07T16:04:29.620 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.max_background_flushes: -1 2024-04-07T16:04:29.620 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Compression algorithms supported: 2024-04-07T16:04:29.620 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-07T16:04:29.620 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: kZSTD supported: 0 2024-04-07T16:04:29.620 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: kXpressCompression supported: 0 2024-04-07T16:04:29.620 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: kLZ4HCCompression supported: 1 2024-04-07T16:04:29.620 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: kZlibCompression supported: 1 2024-04-07T16:04:29.620 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: kSnappyCompression supported: 1 2024-04-07T16:04:29.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: kLZ4Compression supported: 1 2024-04-07T16:04:29.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: kBZip2Compression supported: 0 2024-04-07T16:04:29.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Fast CRC32 supported: Supported on x86 2024-04-07T16:04:29.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: DMutex implementation: pthread_mutex_t 2024-04-07T16:04:29.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-04-07T16:04:29.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-07T16:04:29.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-07T16:04:29.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.merge_operator: 2024-04-07T16:04:29.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.compaction_filter: None 2024-04-07T16:04:29.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.compaction_filter_factory: None 2024-04-07T16:04:29.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.sst_partitioner_factory: None 2024-04-07T16:04:29.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.memtable_factory: SkipListFactory 2024-04-07T16:04:29.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.table_factory: BlockBasedTable 2024-04-07T16:04:29.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55d407f55620) 2024-04-07T16:04:29.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: cache_index_and_filter_blocks: 1 2024-04-07T16:04:29.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-07T16:04:29.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-07T16:04:29.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: pin_top_level_index_and_filter: 1 2024-04-07T16:04:29.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: index_type: 0 2024-04-07T16:04:29.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: data_block_index_type: 0 2024-04-07T16:04:29.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: index_shortening: 1 2024-04-07T16:04:29.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: data_block_hash_table_util_ratio: 0.750000 2024-04-07T16:04:29.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: checksum: 4 2024-04-07T16:04:29.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: no_block_cache: 0 2024-04-07T16:04:29.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: block_cache: 0x55d40726b1f0 2024-04-07T16:04:29.623 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: block_cache_name: BinnedLRUCache 2024-04-07T16:04:29.623 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: block_cache_options: 2024-04-07T16:04:29.623 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: capacity : 536870912 2024-04-07T16:04:29.623 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: num_shard_bits : 4 2024-04-07T16:04:29.623 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: strict_capacity_limit : 0 2024-04-07T16:04:29.623 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: high_pri_pool_ratio: 0.000 2024-04-07T16:04:29.623 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: block_cache_compressed: (nil) 2024-04-07T16:04:29.623 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: persistent_cache: (nil) 2024-04-07T16:04:29.623 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: block_size: 4096 2024-04-07T16:04:29.623 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: block_size_deviation: 10 2024-04-07T16:04:29.623 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: block_restart_interval: 16 2024-04-07T16:04:29.623 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: index_block_restart_interval: 1 2024-04-07T16:04:29.623 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: metadata_block_size: 4096 2024-04-07T16:04:29.623 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: partition_filters: 0 2024-04-07T16:04:29.623 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: use_delta_encoding: 1 2024-04-07T16:04:29.624 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: filter_policy: bloomfilter 2024-04-07T16:04:29.624 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: whole_key_filtering: 1 2024-04-07T16:04:29.624 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: verify_compression: 0 2024-04-07T16:04:29.624 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: read_amp_bytes_per_bit: 0 2024-04-07T16:04:29.624 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: format_version: 5 2024-04-07T16:04:29.624 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: enable_index_compression: 1 2024-04-07T16:04:29.624 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: block_align: 0 2024-04-07T16:04:29.624 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: max_auto_readahead_size: 262144 2024-04-07T16:04:29.624 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: prepopulate_block_cache: 0 2024-04-07T16:04:29.624 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: initial_auto_readahead_size: 8192 2024-04-07T16:04:29.624 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: num_file_reads_for_auto_readahead: 2 2024-04-07T16:04:29.624 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.write_buffer_size: 33554432 2024-04-07T16:04:29.624 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.max_write_buffer_number: 2 2024-04-07T16:04:29.626 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.compression: NoCompression 2024-04-07T16:04:29.626 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.bottommost_compression: Disabled 2024-04-07T16:04:29.626 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.prefix_extractor: nullptr 2024-04-07T16:04:29.626 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-07T16:04:29.626 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.num_levels: 7 2024-04-07T16:04:29.626 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-07T16:04:29.626 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-07T16:04:29.627 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-07T16:04:29.627 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-07T16:04:29.627 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-07T16:04:29.627 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-07T16:04:29.627 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-07T16:04:29.627 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-07T16:04:29.627 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-07T16:04:29.627 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-07T16:04:29.627 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-07T16:04:29.627 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-07T16:04:29.627 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.compression_opts.window_bits: -14 2024-04-07T16:04:29.627 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.compression_opts.level: 32767 2024-04-07T16:04:29.627 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.compression_opts.strategy: 0 2024-04-07T16:04:29.627 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-07T16:04:29.628 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-07T16:04:29.628 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-07T16:04:29.628 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-07T16:04:29.628 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.compression_opts.enabled: false 2024-04-07T16:04:29.628 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-07T16:04:29.628 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-07T16:04:29.628 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-07T16:04:29.628 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-07T16:04:29.628 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.target_file_size_base: 67108864 2024-04-07T16:04:29.628 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.target_file_size_multiplier: 1 2024-04-07T16:04:29.628 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-07T16:04:29.628 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-07T16:04:29.628 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-07T16:04:29.628 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-07T16:04:29.629 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-07T16:04:29.629 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-07T16:04:29.629 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-07T16:04:29.629 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-07T16:04:29.629 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-07T16:04:29.629 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-07T16:04:29.629 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-07T16:04:29.629 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-07T16:04:29.629 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-07T16:04:29.629 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.arena_block_size: 1048576 2024-04-07T16:04:29.629 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-07T16:04:29.629 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-07T16:04:29.629 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.disable_auto_compactions: 0 2024-04-07T16:04:29.629 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-07T16:04:29.630 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-07T16:04:29.630 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-07T16:04:29.630 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-07T16:04:29.630 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-07T16:04:29.630 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-07T16:04:29.630 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-07T16:04:29.630 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-07T16:04:29.630 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-07T16:04:29.630 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-07T16:04:29.630 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.table_properties_collectors: 2024-04-07T16:04:29.630 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.inplace_update_support: 0 2024-04-07T16:04:29.631 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.inplace_update_num_locks: 10000 2024-04-07T16:04:29.631 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-07T16:04:29.631 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-07T16:04:29.631 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.memtable_huge_page_size: 0 2024-04-07T16:04:29.631 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.bloom_locality: 0 2024-04-07T16:04:29.631 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.max_successive_merges: 0 2024-04-07T16:04:29.631 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.optimize_filters_for_hits: 0 2024-04-07T16:04:29.631 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.paranoid_file_checks: 0 2024-04-07T16:04:29.631 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.force_consistency_checks: 1 2024-04-07T16:04:29.631 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.report_bg_io_stats: 0 2024-04-07T16:04:29.631 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.ttl: 2592000 2024-04-07T16:04:29.631 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.periodic_compaction_seconds: 0 2024-04-07T16:04:29.631 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-07T16:04:29.631 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-07T16:04:29.632 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.enable_blob_files: false 2024-04-07T16:04:29.632 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.min_blob_size: 0 2024-04-07T16:04:29.632 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.blob_file_size: 268435456 2024-04-07T16:04:29.632 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.blob_compression_type: NoCompression 2024-04-07T16:04:29.632 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.enable_blob_garbage_collection: false 2024-04-07T16:04:29.632 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-07T16:04:29.632 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-07T16:04:29.632 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-07T16:04:29.632 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.blob_file_starting_level: 0 2024-04-07T16:04:29.632 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.300+0000 7f27b88a2c80 4 rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-07T16:04:29.632 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.304+0000 7f27b88a2c80 4 rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-04-07T16:04:29.632 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.304+0000 7f27b88a2c80 4 rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-04-07T16:04:29.632 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.304+0000 7f27b88a2c80 4 rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: aaaa2253-d796-4d76-9a21-61c166965630 2024-04-07T16:04:29.633 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.304+0000 7f27b88a2c80 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1712505869309096, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-04-07T16:04:29.633 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.304+0000 7f27b88a2c80 4 rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-04-07T16:04:29.633 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.304+0000 7f27b88a2c80 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1712505869309614, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1622, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1712505869, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "aaaa2253-d796-4d76-9a21-61c166965630", "db_session_id": "Z7QIPSO085COQQ7SQO4S", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-04-07T16:04:29.633 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.304+0000 7f27b88a2c80 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1712505869309719, "job": 1, "event": "recovery_finished"} 2024-04-07T16:04:29.633 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.304+0000 7f27b88a2c80 4 rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-04-07T16:04:29.633 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.304+0000 7f27b88a2c80 4 rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-07T16:04:29.633 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.304+0000 7f27b88a2c80 4 rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55d40730c000 2024-04-07T16:04:29.633 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.304+0000 7f27b88a2c80 4 rocksdb: DB pointer 0x55d4072fa000 2024-04-07T16:04:29.633 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.304+0000 7f27a707a700 4 rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-07T16:04:29.633 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.304+0000 7f27a707a700 4 rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-07T16:04:29.634 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: ** DB Stats ** 2024-04-07T16:04:29.634 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: Uptime(secs): 0.0 total, 0.0 interval 2024-04-07T16:04:29.634 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-07T16:04:29.634 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-07T16:04:29.634 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-07T16:04:29.634 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-07T16:04:29.634 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-07T16:04:29.634 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-07T16:04:29.634 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: ** Compaction Stats [default] ** 2024-04-07T16:04:29.635 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-07T16:04:29.635 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-07T16:04:29.635 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: L0 1/0 1.58 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-07T16:04:29.635 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: Sum 1/0 1.58 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-07T16:04:29.635 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-07T16:04:29.635 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: ** Compaction Stats [default] ** 2024-04-07T16:04:29.635 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-07T16:04:29.635 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-07T16:04:29.635 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-07T16:04:29.635 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-07T16:04:29.635 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: Uptime(secs): 0.0 total, 0.0 interval 2024-04-07T16:04:29.635 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: Flush(GB): cumulative 0.000, interval 0.000 2024-04-07T16:04:29.636 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-07T16:04:29.636 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: AddFile(Total Files): cumulative 0, interval 0 2024-04-07T16:04:29.636 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: AddFile(L0 Files): cumulative 0, interval 0 2024-04-07T16:04:29.636 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: AddFile(Keys): cumulative 0, interval 0 2024-04-07T16:04:29.636 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: Cumulative compaction: 0.00 GB write, 0.49 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-07T16:04:29.636 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: Interval compaction: 0.00 GB write, 0.49 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-07T16:04:29.636 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-07T16:04:29.636 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: Block cache BinnedLRUCache@0x55d40726b1f0#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.1e-05 secs_since: 0 2024-04-07T16:04:29.636 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-04-07T16:04:29.636 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: ** File Read Latency Histogram By Level [default] ** 2024-04-07T16:04:29.636 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.304+0000 7f27b88a2c80 0 mon.b does not exist in monmap, will attempt to join an existing cluster 2024-04-07T16:04:29.636 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.304+0000 7f27b88a2c80 0 using public_addr v2:172.21.15.144:0/0 -> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] 2024-04-07T16:04:29.636 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.304+0000 7f27b88a2c80 0 starting mon.b rank -1 at public addrs [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] at bind addrs [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid 0702e3a8-f4f8-11ee-b649-cb9ed24678a4 2024-04-07T16:04:29.637 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.308+0000 7f27b88a2c80 1 mon.b@-1(???) e0 preinit fsid 0702e3a8-f4f8-11ee-b649-cb9ed24678a4 2024-04-07T16:04:29.637 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.320+0000 7f27aa080700 0 mon.b@-1(synchronizing).mds e1 new map 2024-04-07T16:04:29.637 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.320+0000 7f27aa080700 0 mon.b@-1(synchronizing).mds e1 print_map 2024-04-07T16:04:29.637 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: e1 2024-04-07T16:04:29.637 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: enable_multiple, ever_enabled_multiple: 1,1 2024-04-07T16:04:29.637 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-07T16:04:29.637 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: legacy client fscid: -1 2024-04-07T16:04:29.637 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: 2024-04-07T16:04:29.637 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: No filesystems configured 2024-04-07T16:04:29.637 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.320+0000 7f27aa080700 1 mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-04-07T16:04:29.637 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.324+0000 7f27aa080700 1 mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-04-07T16:04:29.638 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.324+0000 7f27aa080700 1 mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-04-07T16:04:29.638 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.324+0000 7f27aa080700 1 mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-04-07T16:04:29.638 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.324+0000 7f27aa080700 1 mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-04-07T16:04:29.638 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.324+0000 7f27aa080700 1 mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-04-07T16:04:29.638 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.324+0000 7f27aa080700 0 mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-07T16:04:29.638 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.324+0000 7f27aa080700 0 mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-07T16:04:29.638 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.324+0000 7f27aa080700 0 mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-07T16:04:29.638 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.324+0000 7f27aa080700 0 mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-07T16:04:29.638 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: audit 2024-04-07T16:03:59.180236+0000 mgr.y (mgr.14152) 22 : audit 0 from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi144", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:04:29.638 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: cephadm 2024-04-07T16:03:59.825244+0000 mgr.y (mgr.14152) 23 : cephadm 1 Deploying cephadm binary to smithi144 2024-04-07T16:04:29.638 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: cluster 2024-04-07T16:04:00.162863+0000 mgr.y (mgr.14152) 24 : cluster 0 pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-07T16:04:29.639 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: cluster 2024-04-07T16:04:02.163278+0000 mgr.y (mgr.14152) 25 : cluster 0 pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-07T16:04:29.639 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: audit 2024-04-07T16:04:02.411906+0000 mon.a (mon.0) 109 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:04:29.639 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: cephadm 2024-04-07T16:04:02.412681+0000 mgr.y (mgr.14152) 26 : cephadm 1 Added host smithi144 2024-04-07T16:04:29.639 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: audit 2024-04-07T16:04:02.439622+0000 mon.a (mon.0) 110 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:04:29.639 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: audit 2024-04-07T16:04:02.763112+0000 mon.a (mon.0) 111 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:04:29.639 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: audit 2024-04-07T16:04:04.137525+0000 mon.a (mon.0) 112 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:04:29.639 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: cluster 2024-04-07T16:04:04.163715+0000 mgr.y (mgr.14152) 27 : cluster 0 pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-07T16:04:29.639 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: audit 2024-04-07T16:04:05.038812+0000 mon.a (mon.0) 113 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:04:29.639 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: cluster 2024-04-07T16:04:06.164160+0000 mgr.y (mgr.14152) 28 : cluster 0 pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-07T16:04:29.639 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: cluster 2024-04-07T16:04:08.164579+0000 mgr.y (mgr.14152) 29 : cluster 0 pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-07T16:04:29.639 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: audit 2024-04-07T16:04:10.073228+0000 mon.a (mon.0) 114 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:04:29.639 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: audit 2024-04-07T16:04:10.079392+0000 mon.a (mon.0) 115 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:04:29.640 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: audit 2024-04-07T16:04:10.086422+0000 mon.a (mon.0) 116 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:04:29.640 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: audit 2024-04-07T16:04:10.092286+0000 mon.a (mon.0) 117 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:04:29.640 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: audit 2024-04-07T16:04:10.093949+0000 mon.a (mon.0) 118 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi144", "name": "osd_memory_target"}]: dispatch 2024-04-07T16:04:29.640 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: audit 2024-04-07T16:04:10.095333+0000 mon.a (mon.0) 119 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:04:29.640 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: audit 2024-04-07T16:04:10.096457+0000 mon.a (mon.0) 120 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:04:29.640 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: cephadm 2024-04-07T16:04:10.098337+0000 mgr.y (mgr.14152) 30 : cephadm 1 Updating smithi144:/etc/ceph/ceph.conf 2024-04-07T16:04:29.640 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: audit 2024-04-07T16:04:10.660456+0000 mon.a (mon.0) 121 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:04:29.640 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: audit 2024-04-07T16:04:10.664591+0000 mon.a (mon.0) 122 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:04:29.640 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: audit 2024-04-07T16:04:10.668809+0000 mon.a (mon.0) 123 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:04:29.640 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: cluster 2024-04-07T16:04:10.164861+0000 mgr.y (mgr.14152) 31 : cluster 0 pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-07T16:04:29.640 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: cephadm 2024-04-07T16:04:10.212773+0000 mgr.y (mgr.14152) 32 : cephadm 1 Updating smithi144:/var/lib/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/config/ceph.conf 2024-04-07T16:04:29.640 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: audit 2024-04-07T16:04:10.321884+0000 mgr.y (mgr.14152) 33 : audit 0 from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-07T16:04:29.641 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: cephadm 2024-04-07T16:04:10.327034+0000 mgr.y (mgr.14152) 34 : cephadm 1 Updating smithi144:/etc/ceph/ceph.client.admin.keyring 2024-04-07T16:04:29.641 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: cephadm 2024-04-07T16:04:10.491351+0000 mgr.y (mgr.14152) 35 : cephadm 1 Updating smithi144:/var/lib/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/config/ceph.client.admin.keyring 2024-04-07T16:04:29.641 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: cluster 2024-04-07T16:04:12.165232+0000 mgr.y (mgr.14152) 36 : cluster 0 pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-07T16:04:29.641 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: cluster 2024-04-07T16:04:14.165631+0000 mgr.y (mgr.14152) 37 : cluster 0 pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-07T16:04:29.641 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: cluster 2024-04-07T16:04:16.166058+0000 mgr.y (mgr.14152) 38 : cluster 0 pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-07T16:04:29.641 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: audit 2024-04-07T16:04:18.152414+0000 mon.a (mon.0) 124 : audit 1 from='client.? 172.21.15.100:0/860722094' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-07T16:04:29.641 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: cluster 2024-04-07T16:04:18.166463+0000 mgr.y (mgr.14152) 39 : cluster 0 pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-07T16:04:29.641 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: audit 2024-04-07T16:04:19.106485+0000 mon.a (mon.0) 125 : audit 1 from='client.? 172.21.15.100:0/860722094' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-07T16:04:29.641 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: cluster 2024-04-07T16:04:19.106643+0000 mon.a (mon.0) 126 : cluster 0 osdmap e4: 0 total, 0 up, 0 in 2024-04-07T16:04:29.641 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: cluster 2024-04-07T16:04:20.166864+0000 mgr.y (mgr.14152) 40 : cluster 0 pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-07T16:04:29.641 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: cluster 2024-04-07T16:04:22.167236+0000 mgr.y (mgr.14152) 41 : cluster 0 pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-07T16:04:29.641 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: audit 2024-04-07T16:04:23.298679+0000 mon.a (mon.0) 127 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:04:29.641 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: audit 2024-04-07T16:04:23.331312+0000 mon.a (mon.0) 128 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:04:29.642 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: audit 2024-04-07T16:04:23.334984+0000 mon.a (mon.0) 129 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:04:29.642 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: audit 2024-04-07T16:04:23.336706+0000 mon.a (mon.0) 130 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:04:29.642 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: audit 2024-04-07T16:04:23.345012+0000 mon.a (mon.0) 131 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:04:29.642 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: audit 2024-04-07T16:04:23.349012+0000 mon.a (mon.0) 132 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-07T16:04:29.642 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: audit 2024-04-07T16:04:23.350659+0000 mon.a (mon.0) 133 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:04:29.642 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: audit 2024-04-07T16:04:23.287209+0000 mgr.y (mgr.14152) 42 : audit 0 from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi100:172.21.15.100=a;smithi100:[v2:172.21.15.100:3301,v1:172.21.15.100:6790]=c;smithi144:172.21.15.144=b", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:04:29.642 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: cephadm 2024-04-07T16:04:23.293171+0000 mgr.y (mgr.14152) 43 : cephadm 1 Saving service mon spec with placement smithi100:172.21.15.100=a;smithi100:[v2:172.21.15.100:3301,v1:172.21.15.100:6790]=c;smithi144:172.21.15.144=b;count:3 2024-04-07T16:04:29.642 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: cephadm 2024-04-07T16:04:23.352585+0000 mgr.y (mgr.14152) 44 : cephadm 1 Deploying daemon mon.b on smithi144 2024-04-07T16:04:29.642 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: cluster 2024-04-07T16:04:24.167630+0000 mgr.y (mgr.14152) 45 : cluster 0 pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-07T16:04:29.642 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: cluster 2024-04-07T16:04:26.168087+0000 mgr.y (mgr.14152) 46 : cluster 0 pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-07T16:04:29.642 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: audit 2024-04-07T16:04:28.104941+0000 mon.a (mon.0) 134 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:04:29.642 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: audit 2024-04-07T16:04:28.111559+0000 mon.a (mon.0) 135 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:04:29.643 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: audit 2024-04-07T16:04:28.134187+0000 mon.a (mon.0) 136 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:04:29.643 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: audit 2024-04-07T16:04:28.135508+0000 mon.a (mon.0) 137 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-07T16:04:29.643 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: audit 2024-04-07T16:04:28.137241+0000 mon.a (mon.0) 138 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:04:29.643 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: audit 2024-04-07T16:04:28.514999+0000 mon.a (mon.0) 139 : audit 0 from='client.? 172.21.15.144:0/4129686569' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-07T16:04:29.643 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:29 smithi144 bash[23522]: debug 2024-04-07T16:04:29.328+0000 7f27aa080700 1 mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-04-07T16:04:30.234 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-07T16:04:30.235 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0702e3a8-f4f8-11ee-b649-cb9ed24678a4 -- ceph mon dump -f json 2024-04-07T16:04:31.695 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:31 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:04:31.695 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:04:31 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:04:32.195 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:04:31 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:04:32.195 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:31 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:04:32.945 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:04:32 smithi100 bash[31660]: debug 2024-04-07T16:04:32.794+0000 7f8ed9a6b700 1 mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-04-07T16:04:33.889 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/mon.b/config 2024-04-07T16:04:34.696 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:34 smithi100 bash[21346]: audit 2024-04-07T16:04:29.341850+0000 mon.a (mon.0) 141 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:04:34.696 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:34 smithi100 bash[21346]: audit 2024-04-07T16:04:29.342028+0000 mon.a (mon.0) 142 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:04:34.696 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:34 smithi100 bash[21346]: cluster 2024-04-07T16:04:29.342282+0000 mon.a (mon.0) 143 : cluster 1 mon.a calling monitor election 2024-04-07T16:04:34.696 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:34 smithi100 bash[21346]: cluster 2024-04-07T16:04:30.168982+0000 mgr.y (mgr.14152) 49 : cluster 0 pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-07T16:04:34.696 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:34 smithi100 bash[21346]: audit 2024-04-07T16:04:30.339288+0000 mon.a (mon.0) 144 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:04:34.696 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:34 smithi100 bash[21346]: audit 2024-04-07T16:04:31.339287+0000 mon.a (mon.0) 145 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:04:34.696 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:34 smithi100 bash[21346]: cluster 2024-04-07T16:04:31.339858+0000 mon.b (mon.1) 1 : cluster 1 mon.b calling monitor election 2024-04-07T16:04:34.696 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:34 smithi100 bash[21346]: cluster 2024-04-07T16:04:32.169512+0000 mgr.y (mgr.14152) 50 : cluster 0 pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-07T16:04:34.696 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:34 smithi100 bash[21346]: audit 2024-04-07T16:04:32.339523+0000 mon.a (mon.0) 146 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:04:34.697 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:34 smithi100 bash[21346]: audit 2024-04-07T16:04:32.800445+0000 mon.a (mon.0) 147 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:04:34.697 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:34 smithi100 bash[21346]: audit 2024-04-07T16:04:33.339687+0000 mon.a (mon.0) 148 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:04:34.697 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:34 smithi100 bash[21346]: audit 2024-04-07T16:04:33.800937+0000 mon.a (mon.0) 149 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:04:34.697 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:34 smithi100 bash[21346]: audit 2024-04-07T16:04:34.339635+0000 mon.a (mon.0) 150 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:04:34.697 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:34 smithi100 bash[21346]: cluster 2024-04-07T16:04:34.366882+0000 mon.a (mon.0) 151 : cluster 1 mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-04-07T16:04:34.697 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:34 smithi100 bash[21346]: cluster 2024-04-07T16:04:34.377293+0000 mon.a (mon.0) 152 : cluster 0 monmap e2: 2 mons at {a=[v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0],b=[v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-04-07T16:04:34.697 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:34 smithi100 bash[21346]: cluster 2024-04-07T16:04:34.396366+0000 mon.a (mon.0) 153 : cluster 0 fsmap 2024-04-07T16:04:34.697 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:34 smithi100 bash[21346]: cluster 2024-04-07T16:04:34.396421+0000 mon.a (mon.0) 154 : cluster 0 osdmap e4: 0 total, 0 up, 0 in 2024-04-07T16:04:34.697 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:34 smithi100 bash[21346]: cluster 2024-04-07T16:04:34.396701+0000 mon.a (mon.0) 155 : cluster 0 mgrmap e13: y(active, since 70s) 2024-04-07T16:04:34.697 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:34 smithi100 bash[21346]: cluster 2024-04-07T16:04:34.396950+0000 mon.a (mon.0) 156 : cluster 1 overall HEALTH_OK 2024-04-07T16:04:34.697 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:34 smithi100 bash[21346]: audit 2024-04-07T16:04:34.403882+0000 mon.a (mon.0) 157 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:04:34.697 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:34 smithi100 bash[21346]: audit 2024-04-07T16:04:34.411402+0000 mon.a (mon.0) 158 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:04:34.697 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:34 smithi100 bash[21346]: audit 2024-04-07T16:04:34.426053+0000 mon.a (mon.0) 159 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:04:34.697 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:34 smithi100 bash[21346]: audit 2024-04-07T16:04:34.433173+0000 mon.a (mon.0) 160 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:04:34.746 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:34 smithi144 bash[23522]: audit 2024-04-07T16:04:29.341850+0000 mon.a (mon.0) 141 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:04:34.746 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:34 smithi144 bash[23522]: audit 2024-04-07T16:04:29.342028+0000 mon.a (mon.0) 142 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:04:34.746 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:34 smithi144 bash[23522]: cluster 2024-04-07T16:04:29.342282+0000 mon.a (mon.0) 143 : cluster 1 mon.a calling monitor election 2024-04-07T16:04:34.746 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:34 smithi144 bash[23522]: cluster 2024-04-07T16:04:30.168982+0000 mgr.y (mgr.14152) 49 : cluster 0 pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-07T16:04:34.746 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:34 smithi144 bash[23522]: audit 2024-04-07T16:04:30.339288+0000 mon.a (mon.0) 144 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:04:34.746 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:34 smithi144 bash[23522]: audit 2024-04-07T16:04:31.339287+0000 mon.a (mon.0) 145 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:04:34.746 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:34 smithi144 bash[23522]: cluster 2024-04-07T16:04:31.339858+0000 mon.b (mon.1) 1 : cluster 1 mon.b calling monitor election 2024-04-07T16:04:34.746 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:34 smithi144 bash[23522]: cluster 2024-04-07T16:04:32.169512+0000 mgr.y (mgr.14152) 50 : cluster 0 pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-07T16:04:34.746 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:34 smithi144 bash[23522]: audit 2024-04-07T16:04:32.339523+0000 mon.a (mon.0) 146 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:04:34.746 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:34 smithi144 bash[23522]: audit 2024-04-07T16:04:32.800445+0000 mon.a (mon.0) 147 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:04:34.746 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:34 smithi144 bash[23522]: audit 2024-04-07T16:04:33.339687+0000 mon.a (mon.0) 148 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:04:34.746 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:34 smithi144 bash[23522]: audit 2024-04-07T16:04:33.800937+0000 mon.a (mon.0) 149 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:04:34.747 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:34 smithi144 bash[23522]: audit 2024-04-07T16:04:34.339635+0000 mon.a (mon.0) 150 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:04:34.747 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:34 smithi144 bash[23522]: cluster 2024-04-07T16:04:34.366882+0000 mon.a (mon.0) 151 : cluster 1 mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-04-07T16:04:34.747 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:34 smithi144 bash[23522]: cluster 2024-04-07T16:04:34.377293+0000 mon.a (mon.0) 152 : cluster 0 monmap e2: 2 mons at {a=[v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0],b=[v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-04-07T16:04:34.747 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:34 smithi144 bash[23522]: cluster 2024-04-07T16:04:34.396366+0000 mon.a (mon.0) 153 : cluster 0 fsmap 2024-04-07T16:04:34.747 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:34 smithi144 bash[23522]: cluster 2024-04-07T16:04:34.396421+0000 mon.a (mon.0) 154 : cluster 0 osdmap e4: 0 total, 0 up, 0 in 2024-04-07T16:04:34.747 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:34 smithi144 bash[23522]: cluster 2024-04-07T16:04:34.396701+0000 mon.a (mon.0) 155 : cluster 0 mgrmap e13: y(active, since 70s) 2024-04-07T16:04:34.747 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:34 smithi144 bash[23522]: cluster 2024-04-07T16:04:34.396950+0000 mon.a (mon.0) 156 : cluster 1 overall HEALTH_OK 2024-04-07T16:04:34.747 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:34 smithi144 bash[23522]: audit 2024-04-07T16:04:34.403882+0000 mon.a (mon.0) 157 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:04:34.747 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:34 smithi144 bash[23522]: audit 2024-04-07T16:04:34.411402+0000 mon.a (mon.0) 158 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:04:34.747 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:34 smithi144 bash[23522]: audit 2024-04-07T16:04:34.426053+0000 mon.a (mon.0) 159 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:04:34.747 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:34 smithi144 bash[23522]: audit 2024-04-07T16:04:34.433173+0000 mon.a (mon.0) 160 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:04:35.695 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:04:35 smithi100 bash[21601]: debug 2024-04-07T16:04:35.338+0000 7f2193091700 -1 mgr.server handle_report got status from non-daemon mon.b 2024-04-07T16:04:40.195 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:04:39 smithi100 bash[31660]: audit 2024-04-07T16:04:29.341850+0000 mon.a (mon.0) 141 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:04:40.196 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:04:39 smithi100 bash[31660]: audit 2024-04-07T16:04:29.342028+0000 mon.a (mon.0) 142 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:04:40.196 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:04:39 smithi100 bash[31660]: cluster 2024-04-07T16:04:29.342282+0000 mon.a (mon.0) 143 : cluster 1 mon.a calling monitor election 2024-04-07T16:04:40.196 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:04:39 smithi100 bash[31660]: cluster 2024-04-07T16:04:30.168982+0000 mgr.y (mgr.14152) 49 : cluster 0 pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-07T16:04:40.196 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:04:39 smithi100 bash[31660]: audit 2024-04-07T16:04:30.339288+0000 mon.a (mon.0) 144 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:04:40.196 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:04:39 smithi100 bash[31660]: audit 2024-04-07T16:04:31.339287+0000 mon.a (mon.0) 145 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:04:40.196 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:04:39 smithi100 bash[31660]: cluster 2024-04-07T16:04:31.339858+0000 mon.b (mon.1) 1 : cluster 1 mon.b calling monitor election 2024-04-07T16:04:40.196 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:04:39 smithi100 bash[31660]: cluster 2024-04-07T16:04:32.169512+0000 mgr.y (mgr.14152) 50 : cluster 0 pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-07T16:04:40.196 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:04:39 smithi100 bash[31660]: audit 2024-04-07T16:04:32.339523+0000 mon.a (mon.0) 146 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:04:40.196 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:04:39 smithi100 bash[31660]: audit 2024-04-07T16:04:32.800445+0000 mon.a (mon.0) 147 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:04:40.196 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:04:39 smithi100 bash[31660]: audit 2024-04-07T16:04:33.339687+0000 mon.a (mon.0) 148 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:04:40.196 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:04:39 smithi100 bash[31660]: audit 2024-04-07T16:04:33.800937+0000 mon.a (mon.0) 149 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:04:40.196 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:04:39 smithi100 bash[31660]: audit 2024-04-07T16:04:34.339635+0000 mon.a (mon.0) 150 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:04:40.197 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:04:39 smithi100 bash[31660]: cluster 2024-04-07T16:04:34.366882+0000 mon.a (mon.0) 151 : cluster 1 mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-04-07T16:04:40.197 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:04:39 smithi100 bash[31660]: cluster 2024-04-07T16:04:34.377293+0000 mon.a (mon.0) 152 : cluster 0 monmap e2: 2 mons at {a=[v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0],b=[v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-04-07T16:04:40.197 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:04:39 smithi100 bash[31660]: cluster 2024-04-07T16:04:34.396366+0000 mon.a (mon.0) 153 : cluster 0 fsmap 2024-04-07T16:04:40.197 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:04:39 smithi100 bash[31660]: cluster 2024-04-07T16:04:34.396421+0000 mon.a (mon.0) 154 : cluster 0 osdmap e4: 0 total, 0 up, 0 in 2024-04-07T16:04:40.197 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:04:39 smithi100 bash[31660]: cluster 2024-04-07T16:04:34.396701+0000 mon.a (mon.0) 155 : cluster 0 mgrmap e13: y(active, since 70s) 2024-04-07T16:04:40.197 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:04:39 smithi100 bash[31660]: cluster 2024-04-07T16:04:34.396950+0000 mon.a (mon.0) 156 : cluster 1 overall HEALTH_OK 2024-04-07T16:04:40.197 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:04:39 smithi100 bash[31660]: audit 2024-04-07T16:04:34.403882+0000 mon.a (mon.0) 157 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:04:40.197 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:04:39 smithi100 bash[31660]: audit 2024-04-07T16:04:34.411402+0000 mon.a (mon.0) 158 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:04:40.197 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:04:39 smithi100 bash[31660]: audit 2024-04-07T16:04:34.426053+0000 mon.a (mon.0) 159 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:04:40.197 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:04:39 smithi100 bash[31660]: audit 2024-04-07T16:04:34.433173+0000 mon.a (mon.0) 160 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:04:40.197 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:04:39 smithi100 bash[31660]: cluster 2024-04-07T16:04:34.170018+0000 mgr.y (mgr.14152) 51 : cluster 0 pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-07T16:04:40.197 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:04:39 smithi100 bash[31660]: audit 2024-04-07T16:04:34.804526+0000 mon.a (mon.0) 163 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:04:40.197 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:04:39 smithi100 bash[31660]: cluster 2024-04-07T16:04:34.804754+0000 mon.a (mon.0) 164 : cluster 1 mon.a calling monitor election 2024-04-07T16:04:40.197 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:04:39 smithi100 bash[31660]: cluster 2024-04-07T16:04:34.805375+0000 mon.b (mon.1) 2 : cluster 1 mon.b calling monitor election 2024-04-07T16:04:40.197 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:04:39 smithi100 bash[31660]: audit 2024-04-07T16:04:34.824248+0000 mon.a (mon.0) 165 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:04:40.198 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:04:39 smithi100 bash[31660]: audit 2024-04-07T16:04:34.824419+0000 mon.a (mon.0) 166 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:04:40.198 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:04:39 smithi100 bash[31660]: audit 2024-04-07T16:04:35.801490+0000 mon.a (mon.0) 167 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:04:40.198 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:04:39 smithi100 bash[31660]: cluster 2024-04-07T16:04:36.170467+0000 mgr.y (mgr.14152) 52 : cluster 0 pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-07T16:04:40.198 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:04:39 smithi100 bash[31660]: audit 2024-04-07T16:04:36.801525+0000 mon.a (mon.0) 168 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:04:40.198 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:04:39 smithi100 bash[31660]: cluster 2024-04-07T16:04:36.803806+0000 mon.c (mon.2) 1 : cluster 1 mon.c calling monitor election 2024-04-07T16:04:40.198 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:04:39 smithi100 bash[31660]: audit 2024-04-07T16:04:37.801986+0000 mon.a (mon.0) 169 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:04:40.198 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:04:39 smithi100 bash[31660]: cluster 2024-04-07T16:04:38.170848+0000 mgr.y (mgr.14152) 53 : cluster 0 pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-07T16:04:40.198 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:04:39 smithi100 bash[31660]: audit 2024-04-07T16:04:38.801941+0000 mon.a (mon.0) 170 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:04:40.198 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:04:39 smithi100 bash[31660]: audit 2024-04-07T16:04:39.802007+0000 mon.a (mon.0) 171 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:04:40.198 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:04:39 smithi100 bash[31660]: cluster 2024-04-07T16:04:39.849186+0000 mon.a (mon.0) 172 : cluster 1 mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-04-07T16:04:40.198 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:04:39 smithi100 bash[31660]: cluster 2024-04-07T16:04:39.856269+0000 mon.a (mon.0) 173 : cluster 0 monmap e3: 3 mons at {a=[v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0],b=[v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0],c=[v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0]} removed_ranks: {} disallowed_leaders: {} 2024-04-07T16:04:40.198 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:04:39 smithi100 bash[31660]: cluster 2024-04-07T16:04:39.876284+0000 mon.a (mon.0) 174 : cluster 0 fsmap 2024-04-07T16:04:40.198 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:04:39 smithi100 bash[31660]: cluster 2024-04-07T16:04:39.876343+0000 mon.a (mon.0) 175 : cluster 0 osdmap e4: 0 total, 0 up, 0 in 2024-04-07T16:04:40.198 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:04:39 smithi100 bash[31660]: cluster 2024-04-07T16:04:39.876641+0000 mon.a (mon.0) 176 : cluster 0 mgrmap e13: y(active, since 75s) 2024-04-07T16:04:40.199 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:04:39 smithi100 bash[31660]: cluster 2024-04-07T16:04:39.876888+0000 mon.a (mon.0) 177 : cluster 1 overall HEALTH_OK 2024-04-07T16:04:40.199 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:04:39 smithi100 bash[31660]: audit 2024-04-07T16:04:39.884795+0000 mon.a (mon.0) 178 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:04:40.199 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:04:39 smithi100 bash[31660]: audit 2024-04-07T16:04:39.890710+0000 mon.a (mon.0) 179 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:04:40.199 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:04:39 smithi100 bash[31660]: audit 2024-04-07T16:04:39.896180+0000 mon.a (mon.0) 180 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:04:40.199 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:04:39 smithi100 bash[31660]: audit 2024-04-07T16:04:39.902674+0000 mon.a (mon.0) 181 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:04:40.199 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:04:39 smithi100 bash[31660]: audit 2024-04-07T16:04:39.909009+0000 mon.a (mon.0) 182 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:04:40.200 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:39 smithi100 bash[21346]: cluster 2024-04-07T16:04:34.170018+0000 mgr.y (mgr.14152) 51 : cluster 0 pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-07T16:04:40.200 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:39 smithi100 bash[21346]: audit 2024-04-07T16:04:34.804526+0000 mon.a (mon.0) 163 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:04:40.200 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:39 smithi100 bash[21346]: cluster 2024-04-07T16:04:34.804754+0000 mon.a (mon.0) 164 : cluster 1 mon.a calling monitor election 2024-04-07T16:04:40.200 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:39 smithi100 bash[21346]: cluster 2024-04-07T16:04:34.805375+0000 mon.b (mon.1) 2 : cluster 1 mon.b calling monitor election 2024-04-07T16:04:40.200 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:39 smithi100 bash[21346]: audit 2024-04-07T16:04:34.824248+0000 mon.a (mon.0) 165 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:04:40.200 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:39 smithi100 bash[21346]: audit 2024-04-07T16:04:34.824419+0000 mon.a (mon.0) 166 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:04:40.200 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:39 smithi100 bash[21346]: audit 2024-04-07T16:04:35.801490+0000 mon.a (mon.0) 167 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:04:40.200 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:39 smithi100 bash[21346]: cluster 2024-04-07T16:04:36.170467+0000 mgr.y (mgr.14152) 52 : cluster 0 pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-07T16:04:40.200 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:39 smithi100 bash[21346]: audit 2024-04-07T16:04:36.801525+0000 mon.a (mon.0) 168 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:04:40.200 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:39 smithi100 bash[21346]: cluster 2024-04-07T16:04:36.803806+0000 mon.c (mon.2) 1 : cluster 1 mon.c calling monitor election 2024-04-07T16:04:40.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:39 smithi100 bash[21346]: audit 2024-04-07T16:04:37.801986+0000 mon.a (mon.0) 169 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:04:40.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:39 smithi100 bash[21346]: cluster 2024-04-07T16:04:38.170848+0000 mgr.y (mgr.14152) 53 : cluster 0 pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-07T16:04:40.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:39 smithi100 bash[21346]: audit 2024-04-07T16:04:38.801941+0000 mon.a (mon.0) 170 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:04:40.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:39 smithi100 bash[21346]: audit 2024-04-07T16:04:39.802007+0000 mon.a (mon.0) 171 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:04:40.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:39 smithi100 bash[21346]: cluster 2024-04-07T16:04:39.849186+0000 mon.a (mon.0) 172 : cluster 1 mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-04-07T16:04:40.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:39 smithi100 bash[21346]: cluster 2024-04-07T16:04:39.856269+0000 mon.a (mon.0) 173 : cluster 0 monmap e3: 3 mons at {a=[v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0],b=[v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0],c=[v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0]} removed_ranks: {} disallowed_leaders: {} 2024-04-07T16:04:40.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:39 smithi100 bash[21346]: cluster 2024-04-07T16:04:39.876284+0000 mon.a (mon.0) 174 : cluster 0 fsmap 2024-04-07T16:04:40.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:39 smithi100 bash[21346]: cluster 2024-04-07T16:04:39.876343+0000 mon.a (mon.0) 175 : cluster 0 osdmap e4: 0 total, 0 up, 0 in 2024-04-07T16:04:40.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:39 smithi100 bash[21346]: cluster 2024-04-07T16:04:39.876641+0000 mon.a (mon.0) 176 : cluster 0 mgrmap e13: y(active, since 75s) 2024-04-07T16:04:40.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:39 smithi100 bash[21346]: cluster 2024-04-07T16:04:39.876888+0000 mon.a (mon.0) 177 : cluster 1 overall HEALTH_OK 2024-04-07T16:04:40.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:39 smithi100 bash[21346]: audit 2024-04-07T16:04:39.884795+0000 mon.a (mon.0) 178 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:04:40.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:39 smithi100 bash[21346]: audit 2024-04-07T16:04:39.890710+0000 mon.a (mon.0) 179 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:04:40.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:39 smithi100 bash[21346]: audit 2024-04-07T16:04:39.896180+0000 mon.a (mon.0) 180 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:04:40.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:39 smithi100 bash[21346]: audit 2024-04-07T16:04:39.902674+0000 mon.a (mon.0) 181 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:04:40.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:39 smithi100 bash[21346]: audit 2024-04-07T16:04:39.909009+0000 mon.a (mon.0) 182 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:04:40.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:39 smithi144 bash[23522]: cluster 2024-04-07T16:04:34.170018+0000 mgr.y (mgr.14152) 51 : cluster 0 pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-07T16:04:40.361 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:39 smithi144 bash[23522]: audit 2024-04-07T16:04:34.804526+0000 mon.a (mon.0) 163 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:04:40.361 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:39 smithi144 bash[23522]: cluster 2024-04-07T16:04:34.804754+0000 mon.a (mon.0) 164 : cluster 1 mon.a calling monitor election 2024-04-07T16:04:40.361 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:39 smithi144 bash[23522]: cluster 2024-04-07T16:04:34.805375+0000 mon.b (mon.1) 2 : cluster 1 mon.b calling monitor election 2024-04-07T16:04:40.361 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:39 smithi144 bash[23522]: audit 2024-04-07T16:04:34.824248+0000 mon.a (mon.0) 165 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:04:40.361 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:39 smithi144 bash[23522]: audit 2024-04-07T16:04:34.824419+0000 mon.a (mon.0) 166 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:04:40.361 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:39 smithi144 bash[23522]: audit 2024-04-07T16:04:35.801490+0000 mon.a (mon.0) 167 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:04:40.361 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:39 smithi144 bash[23522]: cluster 2024-04-07T16:04:36.170467+0000 mgr.y (mgr.14152) 52 : cluster 0 pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-07T16:04:40.361 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:39 smithi144 bash[23522]: audit 2024-04-07T16:04:36.801525+0000 mon.a (mon.0) 168 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:04:40.361 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:39 smithi144 bash[23522]: cluster 2024-04-07T16:04:36.803806+0000 mon.c (mon.2) 1 : cluster 1 mon.c calling monitor election 2024-04-07T16:04:40.361 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:39 smithi144 bash[23522]: audit 2024-04-07T16:04:37.801986+0000 mon.a (mon.0) 169 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:04:40.361 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:39 smithi144 bash[23522]: cluster 2024-04-07T16:04:38.170848+0000 mgr.y (mgr.14152) 53 : cluster 0 pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-07T16:04:40.361 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:39 smithi144 bash[23522]: audit 2024-04-07T16:04:38.801941+0000 mon.a (mon.0) 170 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:04:40.361 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:39 smithi144 bash[23522]: audit 2024-04-07T16:04:39.802007+0000 mon.a (mon.0) 171 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:04:40.361 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:39 smithi144 bash[23522]: cluster 2024-04-07T16:04:39.849186+0000 mon.a (mon.0) 172 : cluster 1 mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-04-07T16:04:40.362 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:39 smithi144 bash[23522]: cluster 2024-04-07T16:04:39.856269+0000 mon.a (mon.0) 173 : cluster 0 monmap e3: 3 mons at {a=[v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0],b=[v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0],c=[v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0]} removed_ranks: {} disallowed_leaders: {} 2024-04-07T16:04:40.362 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:39 smithi144 bash[23522]: cluster 2024-04-07T16:04:39.876284+0000 mon.a (mon.0) 174 : cluster 0 fsmap 2024-04-07T16:04:40.362 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:39 smithi144 bash[23522]: cluster 2024-04-07T16:04:39.876343+0000 mon.a (mon.0) 175 : cluster 0 osdmap e4: 0 total, 0 up, 0 in 2024-04-07T16:04:40.362 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:39 smithi144 bash[23522]: cluster 2024-04-07T16:04:39.876641+0000 mon.a (mon.0) 176 : cluster 0 mgrmap e13: y(active, since 75s) 2024-04-07T16:04:40.362 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:39 smithi144 bash[23522]: cluster 2024-04-07T16:04:39.876888+0000 mon.a (mon.0) 177 : cluster 1 overall HEALTH_OK 2024-04-07T16:04:40.362 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:39 smithi144 bash[23522]: audit 2024-04-07T16:04:39.884795+0000 mon.a (mon.0) 178 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:04:40.362 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:39 smithi144 bash[23522]: audit 2024-04-07T16:04:39.890710+0000 mon.a (mon.0) 179 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:04:40.362 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:39 smithi144 bash[23522]: audit 2024-04-07T16:04:39.896180+0000 mon.a (mon.0) 180 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:04:40.362 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:39 smithi144 bash[23522]: audit 2024-04-07T16:04:39.902674+0000 mon.a (mon.0) 181 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:04:40.362 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:39 smithi144 bash[23522]: audit 2024-04-07T16:04:39.909009+0000 mon.a (mon.0) 182 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:04:40.877 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-07T16:04:40.877 INFO:teuthology.orchestra.run.smithi144.stdout:{"epoch":3,"fsid":"0702e3a8-f4f8-11ee-b649-cb9ed24678a4","modified":"2024-04-07T16:04:34.801459Z","created":"2024-04-07T16:02:17.280630Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:3300","nonce":0},{"type":"v1","addr":"172.21.15.100:6789","nonce":0}]},"addr":"172.21.15.100:6789/0","public_addr":"172.21.15.100:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:3300","nonce":0},{"type":"v1","addr":"172.21.15.144:6789","nonce":0}]},"addr":"172.21.15.144:6789/0","public_addr":"172.21.15.144:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:3301","nonce":0},{"type":"v1","addr":"172.21.15.100:6790","nonce":0}]},"addr":"172.21.15.100:6790/0","public_addr":"172.21.15.100:6790/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-04-07T16:04:40.880 INFO:teuthology.orchestra.run.smithi144.stderr:dumped monmap epoch 3 2024-04-07T16:04:41.195 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:40 smithi100 bash[21346]: audit 2024-04-07T16:04:40.378039+0000 mon.a (mon.0) 183 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:04:41.195 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:40 smithi100 bash[21346]: audit 2024-04-07T16:04:40.379558+0000 mon.a (mon.0) 184 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:04:41.195 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:40 smithi100 bash[21346]: audit 2024-04-07T16:04:40.717680+0000 mon.a (mon.0) 185 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:04:41.195 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:40 smithi100 bash[21346]: audit 2024-04-07T16:04:40.721756+0000 mon.a (mon.0) 186 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:04:41.195 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:40 smithi100 bash[21346]: audit 2024-04-07T16:04:40.726250+0000 mon.a (mon.0) 187 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:04:41.196 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:40 smithi100 bash[21346]: audit 2024-04-07T16:04:40.730999+0000 mon.a (mon.0) 188 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:04:41.196 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:40 smithi100 bash[21346]: audit 2024-04-07T16:04:40.735554+0000 mon.a (mon.0) 189 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:04:41.196 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:40 smithi100 bash[21346]: audit 2024-04-07T16:04:40.773817+0000 mon.a (mon.0) 190 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:04:41.196 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:40 smithi100 bash[21346]: audit 2024-04-07T16:04:40.778298+0000 mon.a (mon.0) 191 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:04:41.196 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:40 smithi100 bash[21346]: audit 2024-04-07T16:04:40.782942+0000 mon.a (mon.0) 192 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:04:41.196 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:40 smithi100 bash[21346]: audit 2024-04-07T16:04:40.788037+0000 mon.a (mon.0) 193 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:04:41.196 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:40 smithi100 bash[21346]: audit 2024-04-07T16:04:40.788940+0000 mon.a (mon.0) 194 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-07T16:04:41.196 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:40 smithi100 bash[21346]: audit 2024-04-07T16:04:40.790348+0000 mon.a (mon.0) 195 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-07T16:04:41.196 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:40 smithi100 bash[21346]: audit 2024-04-07T16:04:40.791643+0000 mon.a (mon.0) 196 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:04:41.196 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:40 smithi100 bash[21346]: audit 2024-04-07T16:04:40.801445+0000 mon.a (mon.0) 197 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:04:41.196 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:40 smithi100 bash[21346]: audit 2024-04-07T16:04:40.884320+0000 mon.a (mon.0) 198 : audit 0 from='client.? 172.21.15.144:0/1587811410' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-07T16:04:41.196 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:04:40 smithi100 bash[31660]: audit 2024-04-07T16:04:40.378039+0000 mon.a (mon.0) 183 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:04:41.197 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:04:40 smithi100 bash[31660]: audit 2024-04-07T16:04:40.379558+0000 mon.a (mon.0) 184 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:04:41.197 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:04:40 smithi100 bash[31660]: audit 2024-04-07T16:04:40.717680+0000 mon.a (mon.0) 185 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:04:41.197 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:04:40 smithi100 bash[31660]: audit 2024-04-07T16:04:40.721756+0000 mon.a (mon.0) 186 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:04:41.197 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:04:40 smithi100 bash[31660]: audit 2024-04-07T16:04:40.726250+0000 mon.a (mon.0) 187 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:04:41.197 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:04:40 smithi100 bash[31660]: audit 2024-04-07T16:04:40.730999+0000 mon.a (mon.0) 188 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:04:41.197 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:04:40 smithi100 bash[31660]: audit 2024-04-07T16:04:40.735554+0000 mon.a (mon.0) 189 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:04:41.197 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:04:40 smithi100 bash[31660]: audit 2024-04-07T16:04:40.773817+0000 mon.a (mon.0) 190 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:04:41.197 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:04:40 smithi100 bash[31660]: audit 2024-04-07T16:04:40.778298+0000 mon.a (mon.0) 191 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:04:41.197 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:04:40 smithi100 bash[31660]: audit 2024-04-07T16:04:40.782942+0000 mon.a (mon.0) 192 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:04:41.197 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:04:40 smithi100 bash[31660]: audit 2024-04-07T16:04:40.788037+0000 mon.a (mon.0) 193 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:04:41.197 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:04:40 smithi100 bash[31660]: audit 2024-04-07T16:04:40.788940+0000 mon.a (mon.0) 194 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-07T16:04:41.197 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:04:40 smithi100 bash[31660]: audit 2024-04-07T16:04:40.790348+0000 mon.a (mon.0) 195 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-07T16:04:41.197 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:04:40 smithi100 bash[31660]: audit 2024-04-07T16:04:40.791643+0000 mon.a (mon.0) 196 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:04:41.197 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:04:40 smithi100 bash[31660]: audit 2024-04-07T16:04:40.801445+0000 mon.a (mon.0) 197 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:04:41.197 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:04:40 smithi100 bash[31660]: audit 2024-04-07T16:04:40.884320+0000 mon.a (mon.0) 198 : audit 0 from='client.? 172.21.15.144:0/1587811410' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-07T16:04:41.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:40 smithi144 bash[23522]: audit 2024-04-07T16:04:40.378039+0000 mon.a (mon.0) 183 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:04:41.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:40 smithi144 bash[23522]: audit 2024-04-07T16:04:40.379558+0000 mon.a (mon.0) 184 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:04:41.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:40 smithi144 bash[23522]: audit 2024-04-07T16:04:40.717680+0000 mon.a (mon.0) 185 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:04:41.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:40 smithi144 bash[23522]: audit 2024-04-07T16:04:40.721756+0000 mon.a (mon.0) 186 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:04:41.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:40 smithi144 bash[23522]: audit 2024-04-07T16:04:40.726250+0000 mon.a (mon.0) 187 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:04:41.361 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:40 smithi144 bash[23522]: audit 2024-04-07T16:04:40.730999+0000 mon.a (mon.0) 188 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:04:41.361 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:40 smithi144 bash[23522]: audit 2024-04-07T16:04:40.735554+0000 mon.a (mon.0) 189 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:04:41.361 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:40 smithi144 bash[23522]: audit 2024-04-07T16:04:40.773817+0000 mon.a (mon.0) 190 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:04:41.361 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:40 smithi144 bash[23522]: audit 2024-04-07T16:04:40.778298+0000 mon.a (mon.0) 191 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:04:41.361 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:40 smithi144 bash[23522]: audit 2024-04-07T16:04:40.782942+0000 mon.a (mon.0) 192 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:04:41.361 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:40 smithi144 bash[23522]: audit 2024-04-07T16:04:40.788037+0000 mon.a (mon.0) 193 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:04:41.361 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:40 smithi144 bash[23522]: audit 2024-04-07T16:04:40.788940+0000 mon.a (mon.0) 194 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-07T16:04:41.361 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:40 smithi144 bash[23522]: audit 2024-04-07T16:04:40.790348+0000 mon.a (mon.0) 195 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-07T16:04:41.361 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:40 smithi144 bash[23522]: audit 2024-04-07T16:04:40.791643+0000 mon.a (mon.0) 196 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:04:41.361 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:40 smithi144 bash[23522]: audit 2024-04-07T16:04:40.801445+0000 mon.a (mon.0) 197 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:04:41.361 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:40 smithi144 bash[23522]: audit 2024-04-07T16:04:40.884320+0000 mon.a (mon.0) 198 : audit 0 from='client.? 172.21.15.144:0/1587811410' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-07T16:04:41.561 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-04-07T16:04:41.561 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0702e3a8-f4f8-11ee-b649-cb9ed24678a4 -- ceph config generate-minimal-conf 2024-04-07T16:04:42.137 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:41 smithi100 bash[21346]: cluster 2024-04-07T16:04:40.171277+0000 mgr.y (mgr.14152) 54 : cluster 0 pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-07T16:04:42.137 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:41 smithi100 bash[21346]: cephadm 2024-04-07T16:04:40.381546+0000 mgr.y (mgr.14152) 55 : cephadm 1 Updating smithi100:/etc/ceph/ceph.conf 2024-04-07T16:04:42.137 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:41 smithi100 bash[21346]: cephadm 2024-04-07T16:04:40.381971+0000 mgr.y (mgr.14152) 56 : cephadm 1 Updating smithi144:/etc/ceph/ceph.conf 2024-04-07T16:04:42.137 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:41 smithi100 bash[21346]: cephadm 2024-04-07T16:04:40.547020+0000 mgr.y (mgr.14152) 57 : cephadm 1 Updating smithi100:/var/lib/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/config/ceph.conf 2024-04-07T16:04:42.137 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:41 smithi100 bash[21346]: cephadm 2024-04-07T16:04:40.547377+0000 mgr.y (mgr.14152) 58 : cephadm 1 Updating smithi144:/var/lib/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/config/ceph.conf 2024-04-07T16:04:42.137 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:41 smithi100 bash[21346]: cephadm 2024-04-07T16:04:40.788461+0000 mgr.y (mgr.14152) 59 : cephadm 1 Reconfiguring mon.a (unknown last config time)... 2024-04-07T16:04:42.137 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:41 smithi100 bash[21346]: cephadm 2024-04-07T16:04:40.793063+0000 mgr.y (mgr.14152) 60 : cephadm 1 Reconfiguring daemon mon.a on smithi100 2024-04-07T16:04:42.137 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:04:41 smithi100 bash[31660]: cluster 2024-04-07T16:04:40.171277+0000 mgr.y (mgr.14152) 54 : cluster 0 pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-07T16:04:42.137 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:04:41 smithi100 bash[31660]: cephadm 2024-04-07T16:04:40.381546+0000 mgr.y (mgr.14152) 55 : cephadm 1 Updating smithi100:/etc/ceph/ceph.conf 2024-04-07T16:04:42.137 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:04:41 smithi100 bash[31660]: cephadm 2024-04-07T16:04:40.381971+0000 mgr.y (mgr.14152) 56 : cephadm 1 Updating smithi144:/etc/ceph/ceph.conf 2024-04-07T16:04:42.138 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:04:41 smithi100 bash[31660]: cephadm 2024-04-07T16:04:40.547020+0000 mgr.y (mgr.14152) 57 : cephadm 1 Updating smithi100:/var/lib/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/config/ceph.conf 2024-04-07T16:04:42.138 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:04:41 smithi100 bash[31660]: cephadm 2024-04-07T16:04:40.547377+0000 mgr.y (mgr.14152) 58 : cephadm 1 Updating smithi144:/var/lib/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/config/ceph.conf 2024-04-07T16:04:42.138 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:04:41 smithi100 bash[31660]: cephadm 2024-04-07T16:04:40.788461+0000 mgr.y (mgr.14152) 59 : cephadm 1 Reconfiguring mon.a (unknown last config time)... 2024-04-07T16:04:42.138 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:04:41 smithi100 bash[31660]: cephadm 2024-04-07T16:04:40.793063+0000 mgr.y (mgr.14152) 60 : cephadm 1 Reconfiguring daemon mon.a on smithi100 2024-04-07T16:04:42.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:41 smithi144 bash[23522]: cluster 2024-04-07T16:04:40.171277+0000 mgr.y (mgr.14152) 54 : cluster 0 pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-07T16:04:42.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:41 smithi144 bash[23522]: cephadm 2024-04-07T16:04:40.381546+0000 mgr.y (mgr.14152) 55 : cephadm 1 Updating smithi100:/etc/ceph/ceph.conf 2024-04-07T16:04:42.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:41 smithi144 bash[23522]: cephadm 2024-04-07T16:04:40.381971+0000 mgr.y (mgr.14152) 56 : cephadm 1 Updating smithi144:/etc/ceph/ceph.conf 2024-04-07T16:04:42.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:41 smithi144 bash[23522]: cephadm 2024-04-07T16:04:40.547020+0000 mgr.y (mgr.14152) 57 : cephadm 1 Updating smithi100:/var/lib/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/config/ceph.conf 2024-04-07T16:04:42.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:41 smithi144 bash[23522]: cephadm 2024-04-07T16:04:40.547377+0000 mgr.y (mgr.14152) 58 : cephadm 1 Updating smithi144:/var/lib/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/config/ceph.conf 2024-04-07T16:04:42.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:41 smithi144 bash[23522]: cephadm 2024-04-07T16:04:40.788461+0000 mgr.y (mgr.14152) 59 : cephadm 1 Reconfiguring mon.a (unknown last config time)... 2024-04-07T16:04:42.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:41 smithi144 bash[23522]: cephadm 2024-04-07T16:04:40.793063+0000 mgr.y (mgr.14152) 60 : cephadm 1 Reconfiguring daemon mon.a on smithi100 2024-04-07T16:04:43.860 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:43 smithi144 bash[23522]: cluster 2024-04-07T16:04:42.171757+0000 mgr.y (mgr.14152) 61 : cluster 0 pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-07T16:04:43.860 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:43 smithi144 bash[23522]: audit 2024-04-07T16:04:42.609222+0000 mon.a (mon.0) 199 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:04:43.860 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:43 smithi144 bash[23522]: audit 2024-04-07T16:04:42.615914+0000 mon.a (mon.0) 200 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:04:43.860 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:43 smithi144 bash[23522]: cephadm 2024-04-07T16:04:42.661314+0000 mgr.y (mgr.14152) 62 : cephadm 1 Reconfiguring mon.c (monmap changed)... 2024-04-07T16:04:43.860 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:43 smithi144 bash[23522]: audit 2024-04-07T16:04:42.661929+0000 mon.a (mon.0) 201 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-07T16:04:43.861 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:43 smithi144 bash[23522]: audit 2024-04-07T16:04:42.663358+0000 mon.a (mon.0) 202 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-07T16:04:43.861 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:43 smithi144 bash[23522]: audit 2024-04-07T16:04:42.664849+0000 mon.a (mon.0) 203 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:04:43.861 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:43 smithi144 bash[23522]: cephadm 2024-04-07T16:04:42.666388+0000 mgr.y (mgr.14152) 63 : cephadm 1 Reconfiguring daemon mon.c on smithi100 2024-04-07T16:04:43.946 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:43 smithi100 bash[21346]: cluster 2024-04-07T16:04:42.171757+0000 mgr.y (mgr.14152) 61 : cluster 0 pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-07T16:04:43.946 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:43 smithi100 bash[21346]: audit 2024-04-07T16:04:42.609222+0000 mon.a (mon.0) 199 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:04:43.946 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:43 smithi100 bash[21346]: audit 2024-04-07T16:04:42.615914+0000 mon.a (mon.0) 200 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:04:43.946 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:43 smithi100 bash[21346]: cephadm 2024-04-07T16:04:42.661314+0000 mgr.y (mgr.14152) 62 : cephadm 1 Reconfiguring mon.c (monmap changed)... 2024-04-07T16:04:43.947 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:43 smithi100 bash[21346]: audit 2024-04-07T16:04:42.661929+0000 mon.a (mon.0) 201 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-07T16:04:43.947 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:43 smithi100 bash[21346]: audit 2024-04-07T16:04:42.663358+0000 mon.a (mon.0) 202 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-07T16:04:43.947 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:43 smithi100 bash[21346]: audit 2024-04-07T16:04:42.664849+0000 mon.a (mon.0) 203 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:04:43.947 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:43 smithi100 bash[21346]: cephadm 2024-04-07T16:04:42.666388+0000 mgr.y (mgr.14152) 63 : cephadm 1 Reconfiguring daemon mon.c on smithi100 2024-04-07T16:04:43.947 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:04:43 smithi100 bash[31660]: cluster 2024-04-07T16:04:42.171757+0000 mgr.y (mgr.14152) 61 : cluster 0 pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-07T16:04:43.948 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:04:43 smithi100 bash[31660]: audit 2024-04-07T16:04:42.609222+0000 mon.a (mon.0) 199 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:04:43.948 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:04:43 smithi100 bash[31660]: audit 2024-04-07T16:04:42.615914+0000 mon.a (mon.0) 200 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:04:43.948 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:04:43 smithi100 bash[31660]: cephadm 2024-04-07T16:04:42.661314+0000 mgr.y (mgr.14152) 62 : cephadm 1 Reconfiguring mon.c (monmap changed)... 2024-04-07T16:04:43.948 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:04:43 smithi100 bash[31660]: audit 2024-04-07T16:04:42.661929+0000 mon.a (mon.0) 201 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-07T16:04:43.948 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:04:43 smithi100 bash[31660]: audit 2024-04-07T16:04:42.663358+0000 mon.a (mon.0) 202 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-07T16:04:43.948 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:04:43 smithi100 bash[31660]: audit 2024-04-07T16:04:42.664849+0000 mon.a (mon.0) 203 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:04:43.948 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:04:43 smithi100 bash[31660]: cephadm 2024-04-07T16:04:42.666388+0000 mgr.y (mgr.14152) 63 : cephadm 1 Reconfiguring daemon mon.c on smithi100 2024-04-07T16:04:45.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:45 smithi144 bash[23522]: cluster 2024-04-07T16:04:44.172232+0000 mgr.y (mgr.14152) 64 : cluster 0 pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-07T16:04:45.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:45 smithi144 bash[23522]: audit 2024-04-07T16:04:44.328261+0000 mon.a (mon.0) 204 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:04:45.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:45 smithi144 bash[23522]: audit 2024-04-07T16:04:44.334505+0000 mon.a (mon.0) 205 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:04:45.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:45 smithi144 bash[23522]: cephadm 2024-04-07T16:04:44.384345+0000 mgr.y (mgr.14152) 65 : cephadm 1 Reconfiguring mon.b (monmap changed)... 2024-04-07T16:04:45.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:45 smithi144 bash[23522]: audit 2024-04-07T16:04:44.384833+0000 mon.a (mon.0) 206 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-07T16:04:45.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:45 smithi144 bash[23522]: audit 2024-04-07T16:04:44.385921+0000 mon.a (mon.0) 207 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-07T16:04:45.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:45 smithi144 bash[23522]: audit 2024-04-07T16:04:44.386904+0000 mon.a (mon.0) 208 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:04:45.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:45 smithi144 bash[23522]: cephadm 2024-04-07T16:04:44.388142+0000 mgr.y (mgr.14152) 66 : cephadm 1 Reconfiguring daemon mon.b on smithi144 2024-04-07T16:04:45.695 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:45 smithi100 bash[21346]: cluster 2024-04-07T16:04:44.172232+0000 mgr.y (mgr.14152) 64 : cluster 0 pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-07T16:04:45.696 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:45 smithi100 bash[21346]: audit 2024-04-07T16:04:44.328261+0000 mon.a (mon.0) 204 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:04:45.696 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:45 smithi100 bash[21346]: audit 2024-04-07T16:04:44.334505+0000 mon.a (mon.0) 205 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:04:45.696 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:45 smithi100 bash[21346]: cephadm 2024-04-07T16:04:44.384345+0000 mgr.y (mgr.14152) 65 : cephadm 1 Reconfiguring mon.b (monmap changed)... 2024-04-07T16:04:45.696 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:45 smithi100 bash[21346]: audit 2024-04-07T16:04:44.384833+0000 mon.a (mon.0) 206 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-07T16:04:45.696 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:45 smithi100 bash[21346]: audit 2024-04-07T16:04:44.385921+0000 mon.a (mon.0) 207 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-07T16:04:45.696 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:45 smithi100 bash[21346]: audit 2024-04-07T16:04:44.386904+0000 mon.a (mon.0) 208 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:04:45.696 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:45 smithi100 bash[21346]: cephadm 2024-04-07T16:04:44.388142+0000 mgr.y (mgr.14152) 66 : cephadm 1 Reconfiguring daemon mon.b on smithi144 2024-04-07T16:04:45.696 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:04:45 smithi100 bash[31660]: cluster 2024-04-07T16:04:44.172232+0000 mgr.y (mgr.14152) 64 : cluster 0 pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-07T16:04:45.696 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:04:45 smithi100 bash[31660]: audit 2024-04-07T16:04:44.328261+0000 mon.a (mon.0) 204 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:04:45.696 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:04:45 smithi100 bash[31660]: audit 2024-04-07T16:04:44.334505+0000 mon.a (mon.0) 205 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:04:45.696 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:04:45 smithi100 bash[31660]: cephadm 2024-04-07T16:04:44.384345+0000 mgr.y (mgr.14152) 65 : cephadm 1 Reconfiguring mon.b (monmap changed)... 2024-04-07T16:04:45.696 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:04:45 smithi100 bash[31660]: audit 2024-04-07T16:04:44.384833+0000 mon.a (mon.0) 206 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-07T16:04:45.696 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:04:45 smithi100 bash[31660]: audit 2024-04-07T16:04:44.385921+0000 mon.a (mon.0) 207 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-07T16:04:45.697 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:04:45 smithi100 bash[31660]: audit 2024-04-07T16:04:44.386904+0000 mon.a (mon.0) 208 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:04:45.697 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:04:45 smithi100 bash[31660]: cephadm 2024-04-07T16:04:44.388142+0000 mgr.y (mgr.14152) 66 : cephadm 1 Reconfiguring daemon mon.b on smithi144 2024-04-07T16:04:46.240 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/mon.a/config 2024-04-07T16:04:47.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:47 smithi100 bash[21346]: audit 2024-04-07T16:04:46.020771+0000 mon.a (mon.0) 209 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:04:47.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:47 smithi100 bash[21346]: audit 2024-04-07T16:04:46.030131+0000 mon.a (mon.0) 210 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:04:47.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:47 smithi100 bash[21346]: audit 2024-04-07T16:04:46.058225+0000 mon.a (mon.0) 211 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:04:47.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:47 smithi100 bash[21346]: audit 2024-04-07T16:04:46.061818+0000 mon.a (mon.0) 212 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:04:47.295 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:47 smithi100 bash[21346]: audit 2024-04-07T16:04:46.063401+0000 mon.a (mon.0) 213 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:04:47.295 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:47 smithi100 bash[21346]: audit 2024-04-07T16:04:46.074570+0000 mon.a (mon.0) 214 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:04:47.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:04:47 smithi100 bash[31660]: audit 2024-04-07T16:04:46.020771+0000 mon.a (mon.0) 209 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:04:47.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:04:47 smithi100 bash[31660]: audit 2024-04-07T16:04:46.030131+0000 mon.a (mon.0) 210 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:04:47.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:04:47 smithi100 bash[31660]: audit 2024-04-07T16:04:46.058225+0000 mon.a (mon.0) 211 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:04:47.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:04:47 smithi100 bash[31660]: audit 2024-04-07T16:04:46.061818+0000 mon.a (mon.0) 212 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:04:47.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:04:47 smithi100 bash[31660]: audit 2024-04-07T16:04:46.063401+0000 mon.a (mon.0) 213 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:04:47.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:04:47 smithi100 bash[31660]: audit 2024-04-07T16:04:46.074570+0000 mon.a (mon.0) 214 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:04:47.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:47 smithi144 bash[23522]: audit 2024-04-07T16:04:46.020771+0000 mon.a (mon.0) 209 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:04:47.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:47 smithi144 bash[23522]: audit 2024-04-07T16:04:46.030131+0000 mon.a (mon.0) 210 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:04:47.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:47 smithi144 bash[23522]: audit 2024-04-07T16:04:46.058225+0000 mon.a (mon.0) 211 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:04:47.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:47 smithi144 bash[23522]: audit 2024-04-07T16:04:46.061818+0000 mon.a (mon.0) 212 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:04:47.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:47 smithi144 bash[23522]: audit 2024-04-07T16:04:46.063401+0000 mon.a (mon.0) 213 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:04:47.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:47 smithi144 bash[23522]: audit 2024-04-07T16:04:46.074570+0000 mon.a (mon.0) 214 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:04:48.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:48 smithi144 bash[23522]: cluster 2024-04-07T16:04:46.172694+0000 mgr.y (mgr.14152) 67 : cluster 0 pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-07T16:04:48.429 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:48 smithi100 bash[21346]: cluster 2024-04-07T16:04:46.172694+0000 mgr.y (mgr.14152) 67 : cluster 0 pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-07T16:04:48.429 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:04:48 smithi100 bash[31660]: cluster 2024-04-07T16:04:46.172694+0000 mgr.y (mgr.14152) 67 : cluster 0 pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-07T16:04:48.815 INFO:teuthology.orchestra.run.smithi100.stdout:# minimal ceph.conf for 0702e3a8-f4f8-11ee-b649-cb9ed24678a4 2024-04-07T16:04:48.815 INFO:teuthology.orchestra.run.smithi100.stdout:[global] 2024-04-07T16:04:48.815 INFO:teuthology.orchestra.run.smithi100.stdout: fsid = 0702e3a8-f4f8-11ee-b649-cb9ed24678a4 2024-04-07T16:04:48.815 INFO:teuthology.orchestra.run.smithi100.stdout: mon_host = [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] 2024-04-07T16:04:49.195 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:49 smithi100 bash[21346]: audit 2024-04-07T16:04:48.822613+0000 mon.c (mon.2) 2 : audit 0 from='client.? 172.21.15.100:0/2387839377' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:04:49.196 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:04:49 smithi100 bash[31660]: audit 2024-04-07T16:04:48.822613+0000 mon.c (mon.2) 2 : audit 0 from='client.? 172.21.15.100:0/2387839377' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:04:49.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:49 smithi144 bash[23522]: audit 2024-04-07T16:04:48.822613+0000 mon.c (mon.2) 2 : audit 0 from='client.? 172.21.15.100:0/2387839377' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:04:49.471 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-04-07T16:04:49.471 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-07T16:04:49.471 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/etc/ceph/ceph.conf 2024-04-07T16:04:49.485 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-07T16:04:49.485 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-07T16:04:49.539 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-04-07T16:04:49.540 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/etc/ceph/ceph.conf 2024-04-07T16:04:49.554 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-04-07T16:04:49.554 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-07T16:04:49.609 INFO:tasks.cephadm:Adding mgr.y on smithi100 2024-04-07T16:04:49.609 INFO:tasks.cephadm:Adding mgr.x on smithi144 2024-04-07T16:04:49.609 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0702e3a8-f4f8-11ee-b649-cb9ed24678a4 -- ceph orch apply mgr '2;smithi100=y;smithi144=x' 2024-04-07T16:04:50.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:50 smithi144 bash[23522]: cluster 2024-04-07T16:04:48.173182+0000 mgr.y (mgr.14152) 68 : cluster 0 pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-07T16:04:50.445 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:50 smithi100 bash[21346]: cluster 2024-04-07T16:04:48.173182+0000 mgr.y (mgr.14152) 68 : cluster 0 pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-07T16:04:50.445 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:04:50 smithi100 bash[31660]: cluster 2024-04-07T16:04:48.173182+0000 mgr.y (mgr.14152) 68 : cluster 0 pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-07T16:04:52.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:52 smithi144 bash[23522]: cluster 2024-04-07T16:04:50.173623+0000 mgr.y (mgr.14152) 69 : cluster 0 pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-07T16:04:52.446 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:52 smithi100 bash[21346]: cluster 2024-04-07T16:04:50.173623+0000 mgr.y (mgr.14152) 69 : cluster 0 pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-07T16:04:52.446 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:04:52 smithi100 bash[31660]: cluster 2024-04-07T16:04:50.173623+0000 mgr.y (mgr.14152) 69 : cluster 0 pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-07T16:04:53.308 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/mon.b/config 2024-04-07T16:04:54.329 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:54 smithi144 bash[23522]: cluster 2024-04-07T16:04:52.174028+0000 mgr.y (mgr.14152) 70 : cluster 0 pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-07T16:04:54.445 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:54 smithi100 bash[21346]: cluster 2024-04-07T16:04:52.174028+0000 mgr.y (mgr.14152) 70 : cluster 0 pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-07T16:04:54.446 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:04:54 smithi100 bash[31660]: cluster 2024-04-07T16:04:52.174028+0000 mgr.y (mgr.14152) 70 : cluster 0 pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-07T16:04:55.729 INFO:teuthology.orchestra.run.smithi144.stdout:Scheduled mgr update... 2024-04-07T16:04:56.330 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:56 smithi144 bash[23522]: cluster 2024-04-07T16:04:54.174376+0000 mgr.y (mgr.14152) 71 : cluster 0 pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-07T16:04:56.330 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:56 smithi144 bash[23522]: audit 2024-04-07T16:04:55.735125+0000 mon.a (mon.0) 215 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:04:56.331 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:56 smithi144 bash[23522]: audit 2024-04-07T16:04:55.781309+0000 mon.a (mon.0) 216 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:04:56.331 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:56 smithi144 bash[23522]: audit 2024-04-07T16:04:55.785064+0000 mon.a (mon.0) 217 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:04:56.331 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:56 smithi144 bash[23522]: audit 2024-04-07T16:04:55.786845+0000 mon.a (mon.0) 218 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:04:56.331 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:56 smithi144 bash[23522]: audit 2024-04-07T16:04:55.796195+0000 mon.a (mon.0) 219 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:04:56.331 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:56 smithi144 bash[23522]: audit 2024-04-07T16:04:55.799293+0000 mon.a (mon.0) 220 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-07T16:04:56.331 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:56 smithi144 bash[23522]: audit 2024-04-07T16:04:55.805414+0000 mon.a (mon.0) 221 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-07T16:04:56.331 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:56 smithi144 bash[23522]: audit 2024-04-07T16:04:55.807040+0000 mon.a (mon.0) 222 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-07T16:04:56.331 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:56 smithi144 bash[23522]: audit 2024-04-07T16:04:55.808847+0000 mon.a (mon.0) 223 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:04:56.446 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:56 smithi100 bash[21346]: cluster 2024-04-07T16:04:54.174376+0000 mgr.y (mgr.14152) 71 : cluster 0 pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-07T16:04:56.446 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:56 smithi100 bash[21346]: audit 2024-04-07T16:04:55.735125+0000 mon.a (mon.0) 215 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:04:56.446 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:56 smithi100 bash[21346]: audit 2024-04-07T16:04:55.781309+0000 mon.a (mon.0) 216 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:04:56.446 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:56 smithi100 bash[21346]: audit 2024-04-07T16:04:55.785064+0000 mon.a (mon.0) 217 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:04:56.446 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:56 smithi100 bash[21346]: audit 2024-04-07T16:04:55.786845+0000 mon.a (mon.0) 218 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:04:56.446 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:56 smithi100 bash[21346]: audit 2024-04-07T16:04:55.796195+0000 mon.a (mon.0) 219 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:04:56.446 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:56 smithi100 bash[21346]: audit 2024-04-07T16:04:55.799293+0000 mon.a (mon.0) 220 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-07T16:04:56.446 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:56 smithi100 bash[21346]: audit 2024-04-07T16:04:55.805414+0000 mon.a (mon.0) 221 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-07T16:04:56.446 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:56 smithi100 bash[21346]: audit 2024-04-07T16:04:55.807040+0000 mon.a (mon.0) 222 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-07T16:04:56.446 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:56 smithi100 bash[21346]: audit 2024-04-07T16:04:55.808847+0000 mon.a (mon.0) 223 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:04:56.447 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:04:56 smithi100 bash[31660]: cluster 2024-04-07T16:04:54.174376+0000 mgr.y (mgr.14152) 71 : cluster 0 pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-07T16:04:56.447 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:04:56 smithi100 bash[31660]: audit 2024-04-07T16:04:55.735125+0000 mon.a (mon.0) 215 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:04:56.447 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:04:56 smithi100 bash[31660]: audit 2024-04-07T16:04:55.781309+0000 mon.a (mon.0) 216 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:04:56.447 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:04:56 smithi100 bash[31660]: audit 2024-04-07T16:04:55.785064+0000 mon.a (mon.0) 217 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:04:56.447 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:04:56 smithi100 bash[31660]: audit 2024-04-07T16:04:55.786845+0000 mon.a (mon.0) 218 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:04:56.447 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:04:56 smithi100 bash[31660]: audit 2024-04-07T16:04:55.796195+0000 mon.a (mon.0) 219 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:04:56.447 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:04:56 smithi100 bash[31660]: audit 2024-04-07T16:04:55.799293+0000 mon.a (mon.0) 220 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-07T16:04:56.447 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:04:56 smithi100 bash[31660]: audit 2024-04-07T16:04:55.805414+0000 mon.a (mon.0) 221 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-07T16:04:56.447 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:04:56 smithi100 bash[31660]: audit 2024-04-07T16:04:55.807040+0000 mon.a (mon.0) 222 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-07T16:04:56.447 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:04:56 smithi100 bash[31660]: audit 2024-04-07T16:04:55.808847+0000 mon.a (mon.0) 223 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:04:56.533 DEBUG:teuthology.orchestra.run.smithi144:mgr.x> sudo journalctl -f -n 0 -u ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@mgr.x.service 2024-04-07T16:04:56.536 INFO:tasks.cephadm:Deploying OSDs... 2024-04-07T16:04:56.536 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-07T16:04:56.536 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/scratch_devs of=/dev/stdout 2024-04-07T16:04:56.542 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-07T16:04:56.543 DEBUG:teuthology.orchestra.run.smithi100:> stat /dev/vg_nvme/lv_1 2024-04-07T16:04:56.589 INFO:teuthology.orchestra.run.smithi100.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-07T16:04:56.589 INFO:teuthology.orchestra.run.smithi100.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-07T16:04:56.589 INFO:teuthology.orchestra.run.smithi100.stdout:Device: 5h/5d Inode: 889 Links: 1 2024-04-07T16:04:56.589 INFO:teuthology.orchestra.run.smithi100.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-07T16:04:56.589 INFO:teuthology.orchestra.run.smithi100.stdout:Access: 2024-04-07 15:55:51.931197181 +0000 2024-04-07T16:04:56.589 INFO:teuthology.orchestra.run.smithi100.stdout:Modify: 2024-04-07 15:55:51.659199610 +0000 2024-04-07T16:04:56.589 INFO:teuthology.orchestra.run.smithi100.stdout:Change: 2024-04-07 15:55:51.659199610 +0000 2024-04-07T16:04:56.590 INFO:teuthology.orchestra.run.smithi100.stdout: Birth: - 2024-04-07T16:04:56.590 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-07T16:04:56.648 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records in 2024-04-07T16:04:56.648 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records out 2024-04-07T16:04:56.648 INFO:teuthology.orchestra.run.smithi100.stderr:512 bytes copied, 0.000463578 s, 1.1 MB/s 2024-04-07T16:04:56.649 DEBUG:teuthology.orchestra.run.smithi100:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-07T16:04:56.698 DEBUG:teuthology.orchestra.run.smithi100:> stat /dev/vg_nvme/lv_2 2024-04-07T16:04:56.746 INFO:teuthology.orchestra.run.smithi100.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-07T16:04:56.746 INFO:teuthology.orchestra.run.smithi100.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-07T16:04:56.746 INFO:teuthology.orchestra.run.smithi100.stdout:Device: 5h/5d Inode: 899 Links: 1 2024-04-07T16:04:56.746 INFO:teuthology.orchestra.run.smithi100.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-07T16:04:56.746 INFO:teuthology.orchestra.run.smithi100.stdout:Access: 2024-04-07 15:55:52.431192714 +0000 2024-04-07T16:04:56.747 INFO:teuthology.orchestra.run.smithi100.stdout:Modify: 2024-04-07 15:55:52.159195143 +0000 2024-04-07T16:04:56.747 INFO:teuthology.orchestra.run.smithi100.stdout:Change: 2024-04-07 15:55:52.159195143 +0000 2024-04-07T16:04:56.747 INFO:teuthology.orchestra.run.smithi100.stdout: Birth: - 2024-04-07T16:04:56.747 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-07T16:04:56.799 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records in 2024-04-07T16:04:56.799 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records out 2024-04-07T16:04:56.799 INFO:teuthology.orchestra.run.smithi100.stderr:512 bytes copied, 0.000451665 s, 1.1 MB/s 2024-04-07T16:04:56.800 DEBUG:teuthology.orchestra.run.smithi100:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-07T16:04:56.851 DEBUG:teuthology.orchestra.run.smithi100:> stat /dev/vg_nvme/lv_3 2024-04-07T16:04:56.896 INFO:teuthology.orchestra.run.smithi100.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-07T16:04:56.897 INFO:teuthology.orchestra.run.smithi100.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-07T16:04:56.897 INFO:teuthology.orchestra.run.smithi100.stdout:Device: 5h/5d Inode: 916 Links: 1 2024-04-07T16:04:56.897 INFO:teuthology.orchestra.run.smithi100.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-07T16:04:56.897 INFO:teuthology.orchestra.run.smithi100.stdout:Access: 2024-04-07 15:55:52.955188034 +0000 2024-04-07T16:04:56.897 INFO:teuthology.orchestra.run.smithi100.stdout:Modify: 2024-04-07 15:55:52.667190606 +0000 2024-04-07T16:04:56.897 INFO:teuthology.orchestra.run.smithi100.stdout:Change: 2024-04-07 15:55:52.667190606 +0000 2024-04-07T16:04:56.897 INFO:teuthology.orchestra.run.smithi100.stdout: Birth: - 2024-04-07T16:04:56.897 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-07T16:04:56.951 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records in 2024-04-07T16:04:56.951 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records out 2024-04-07T16:04:56.951 INFO:teuthology.orchestra.run.smithi100.stderr:512 bytes copied, 0.000335039 s, 1.5 MB/s 2024-04-07T16:04:56.952 DEBUG:teuthology.orchestra.run.smithi100:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-07T16:04:57.003 DEBUG:teuthology.orchestra.run.smithi100:> stat /dev/vg_nvme/lv_4 2024-04-07T16:04:57.048 INFO:teuthology.orchestra.run.smithi100.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-07T16:04:57.048 INFO:teuthology.orchestra.run.smithi100.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-07T16:04:57.048 INFO:teuthology.orchestra.run.smithi100.stdout:Device: 5h/5d Inode: 937 Links: 1 2024-04-07T16:04:57.049 INFO:teuthology.orchestra.run.smithi100.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-07T16:04:57.049 INFO:teuthology.orchestra.run.smithi100.stdout:Access: 2024-04-07 15:55:53.463183496 +0000 2024-04-07T16:04:57.049 INFO:teuthology.orchestra.run.smithi100.stdout:Modify: 2024-04-07 15:55:53.219185675 +0000 2024-04-07T16:04:57.049 INFO:teuthology.orchestra.run.smithi100.stdout:Change: 2024-04-07 15:55:53.219185675 +0000 2024-04-07T16:04:57.049 INFO:teuthology.orchestra.run.smithi100.stdout: Birth: - 2024-04-07T16:04:57.049 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-07T16:04:57.104 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records in 2024-04-07T16:04:57.104 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records out 2024-04-07T16:04:57.104 INFO:teuthology.orchestra.run.smithi100.stderr:512 bytes copied, 0.000288753 s, 1.8 MB/s 2024-04-07T16:04:57.106 DEBUG:teuthology.orchestra.run.smithi100:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-07T16:04:57.155 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-04-07T16:04:57.155 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/scratch_devs of=/dev/stdout 2024-04-07T16:04:57.159 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-07T16:04:57.160 DEBUG:teuthology.orchestra.run.smithi144:> stat /dev/vg_nvme/lv_1 2024-04-07T16:04:57.205 INFO:teuthology.orchestra.run.smithi144.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-07T16:04:57.205 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-07T16:04:57.205 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 5h/5d Inode: 881 Links: 1 2024-04-07T16:04:57.205 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-07T16:04:57.205 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-04-07 15:55:54.384745352 +0000 2024-04-07T16:04:57.205 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-04-07 15:55:54.124747674 +0000 2024-04-07T16:04:57.205 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-04-07 15:55:54.124747674 +0000 2024-04-07T16:04:57.206 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: - 2024-04-07T16:04:57.206 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-07T16:04:57.262 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records in 2024-04-07T16:04:57.262 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records out 2024-04-07T16:04:57.262 INFO:teuthology.orchestra.run.smithi144.stderr:512 bytes copied, 0.000290818 s, 1.8 MB/s 2024-04-07T16:04:57.263 DEBUG:teuthology.orchestra.run.smithi144:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-07T16:04:57.310 DEBUG:teuthology.orchestra.run.smithi144:> stat /dev/vg_nvme/lv_2 2024-04-07T16:04:57.346 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:57 smithi144 bash[23522]: audit 2024-04-07T16:04:55.725124+0000 mgr.y (mgr.14152) 72 : audit 0 from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi100=y;smithi144=x", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:04:57.346 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:57 smithi144 bash[23522]: cephadm 2024-04-07T16:04:55.728616+0000 mgr.y (mgr.14152) 73 : cephadm 1 Saving service mgr spec with placement smithi100=y;smithi144=x;count:2 2024-04-07T16:04:57.346 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:57 smithi144 bash[23522]: cephadm 2024-04-07T16:04:55.810858+0000 mgr.y (mgr.14152) 74 : cephadm 1 Deploying daemon mgr.x on smithi144 2024-04-07T16:04:57.351 INFO:teuthology.orchestra.run.smithi144.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-07T16:04:57.351 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-07T16:04:57.351 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 5h/5d Inode: 901 Links: 1 2024-04-07T16:04:57.351 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-07T16:04:57.352 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-04-07 15:55:54.860741101 +0000 2024-04-07T16:04:57.352 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-04-07 15:55:54.624743208 +0000 2024-04-07T16:04:57.352 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-04-07 15:55:54.624743208 +0000 2024-04-07T16:04:57.352 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: - 2024-04-07T16:04:57.352 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-07T16:04:57.404 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records in 2024-04-07T16:04:57.404 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records out 2024-04-07T16:04:57.405 INFO:teuthology.orchestra.run.smithi144.stderr:512 bytes copied, 0.000401982 s, 1.3 MB/s 2024-04-07T16:04:57.406 DEBUG:teuthology.orchestra.run.smithi144:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-07T16:04:57.445 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:57 smithi100 bash[21346]: audit 2024-04-07T16:04:55.725124+0000 mgr.y (mgr.14152) 72 : audit 0 from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi100=y;smithi144=x", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:04:57.445 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:57 smithi100 bash[21346]: cephadm 2024-04-07T16:04:55.728616+0000 mgr.y (mgr.14152) 73 : cephadm 1 Saving service mgr spec with placement smithi100=y;smithi144=x;count:2 2024-04-07T16:04:57.446 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:57 smithi100 bash[21346]: cephadm 2024-04-07T16:04:55.810858+0000 mgr.y (mgr.14152) 74 : cephadm 1 Deploying daemon mgr.x on smithi144 2024-04-07T16:04:57.446 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:04:57 smithi100 bash[31660]: audit 2024-04-07T16:04:55.725124+0000 mgr.y (mgr.14152) 72 : audit 0 from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi100=y;smithi144=x", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:04:57.446 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:04:57 smithi100 bash[31660]: cephadm 2024-04-07T16:04:55.728616+0000 mgr.y (mgr.14152) 73 : cephadm 1 Saving service mgr spec with placement smithi100=y;smithi144=x;count:2 2024-04-07T16:04:57.446 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:04:57 smithi100 bash[31660]: cephadm 2024-04-07T16:04:55.810858+0000 mgr.y (mgr.14152) 74 : cephadm 1 Deploying daemon mgr.x on smithi144 2024-04-07T16:04:57.455 DEBUG:teuthology.orchestra.run.smithi144:> stat /dev/vg_nvme/lv_3 2024-04-07T16:04:57.502 INFO:teuthology.orchestra.run.smithi144.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-07T16:04:57.502 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-07T16:04:57.502 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 5h/5d Inode: 916 Links: 1 2024-04-07T16:04:57.502 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-07T16:04:57.502 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-04-07 15:55:55.356736671 +0000 2024-04-07T16:04:57.503 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-04-07 15:55:55.100738957 +0000 2024-04-07T16:04:57.503 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-04-07 15:55:55.100738957 +0000 2024-04-07T16:04:57.503 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: - 2024-04-07T16:04:57.503 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-07T16:04:57.562 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records in 2024-04-07T16:04:57.562 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records out 2024-04-07T16:04:57.562 INFO:teuthology.orchestra.run.smithi144.stderr:512 bytes copied, 0.000389459 s, 1.3 MB/s 2024-04-07T16:04:57.562 DEBUG:teuthology.orchestra.run.smithi144:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-07T16:04:57.611 DEBUG:teuthology.orchestra.run.smithi144:> stat /dev/vg_nvme/lv_4 2024-04-07T16:04:57.658 INFO:teuthology.orchestra.run.smithi144.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-07T16:04:57.658 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-07T16:04:57.658 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 5h/5d Inode: 937 Links: 1 2024-04-07T16:04:57.658 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-07T16:04:57.658 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-04-07 15:55:55.828732454 +0000 2024-04-07T16:04:57.658 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-04-07 15:55:55.588734597 +0000 2024-04-07T16:04:57.658 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-04-07 15:55:55.588734597 +0000 2024-04-07T16:04:57.658 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: - 2024-04-07T16:04:57.659 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-07T16:04:57.712 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records in 2024-04-07T16:04:57.713 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records out 2024-04-07T16:04:57.713 INFO:teuthology.orchestra.run.smithi144.stderr:512 bytes copied, 0.000328492 s, 1.6 MB/s 2024-04-07T16:04:57.714 DEBUG:teuthology.orchestra.run.smithi144:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-07T16:04:57.763 INFO:tasks.cephadm:Deploying osd.0 on smithi100 with /dev/vg_nvme/lv_4... 2024-04-07T16:04:57.763 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0702e3a8-f4f8-11ee-b649-cb9ed24678a4 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-07T16:04:58.098 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:57 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:04:58.099 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:04:57 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:04:58.099 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:04:57 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:04:58.099 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:04:57 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:04:58.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:58 smithi144 bash[23522]: cluster 2024-04-07T16:04:56.174767+0000 mgr.y (mgr.14152) 75 : cluster 0 pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-07T16:04:58.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:58 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:04:58.360 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:04:58 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:04:58.360 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:04:58 smithi144 systemd[1]: Started Ceph mgr.x for 0702e3a8-f4f8-11ee-b649-cb9ed24678a4. 2024-04-07T16:04:58.445 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:58 smithi100 bash[21346]: cluster 2024-04-07T16:04:56.174767+0000 mgr.y (mgr.14152) 75 : cluster 0 pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-07T16:04:58.446 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:04:58 smithi100 bash[31660]: cluster 2024-04-07T16:04:56.174767+0000 mgr.y (mgr.14152) 75 : cluster 0 pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-07T16:04:59.205 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:04:59 smithi144 bash[24472]: debug 2024-04-07T16:04:59.059+0000 7f87bd953700 1 -- 172.21.15.144:0/1953147240 <== mon.1 v2:172.21.15.144:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5579bf10c1a0 con 0x5579bfe8e400 2024-04-07T16:04:59.205 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:04:59 smithi144 bash[24472]: debug 2024-04-07T16:04:59.207+0000 7f87c7ad7200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-07T16:04:59.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:59 smithi144 bash[23522]: cluster 2024-04-07T16:04:58.175196+0000 mgr.y (mgr.14152) 76 : cluster 0 pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-07T16:04:59.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:59 smithi144 bash[23522]: audit 2024-04-07T16:04:58.248642+0000 mon.a (mon.0) 224 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:04:59.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:59 smithi144 bash[23522]: audit 2024-04-07T16:04:58.256536+0000 mon.a (mon.0) 225 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:04:59.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:59 smithi144 bash[23522]: audit 2024-04-07T16:04:58.287100+0000 mon.a (mon.0) 226 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:04:59.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:59 smithi144 bash[23522]: audit 2024-04-07T16:04:58.295964+0000 mon.a (mon.0) 227 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:04:59.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:04:59 smithi144 bash[23522]: audit 2024-04-07T16:04:58.364130+0000 mon.a (mon.0) 228 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:04:59.696 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:59 smithi100 bash[21346]: cluster 2024-04-07T16:04:58.175196+0000 mgr.y (mgr.14152) 76 : cluster 0 pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-07T16:04:59.696 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:59 smithi100 bash[21346]: audit 2024-04-07T16:04:58.248642+0000 mon.a (mon.0) 224 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:04:59.696 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:59 smithi100 bash[21346]: audit 2024-04-07T16:04:58.256536+0000 mon.a (mon.0) 225 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:04:59.696 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:59 smithi100 bash[21346]: audit 2024-04-07T16:04:58.287100+0000 mon.a (mon.0) 226 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:04:59.696 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:59 smithi100 bash[21346]: audit 2024-04-07T16:04:58.295964+0000 mon.a (mon.0) 227 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:04:59.696 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:04:59 smithi100 bash[21346]: audit 2024-04-07T16:04:58.364130+0000 mon.a (mon.0) 228 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:04:59.696 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:04:59 smithi100 bash[31660]: cluster 2024-04-07T16:04:58.175196+0000 mgr.y (mgr.14152) 76 : cluster 0 pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-07T16:04:59.696 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:04:59 smithi100 bash[31660]: audit 2024-04-07T16:04:58.248642+0000 mon.a (mon.0) 224 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:04:59.696 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:04:59 smithi100 bash[31660]: audit 2024-04-07T16:04:58.256536+0000 mon.a (mon.0) 225 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:04:59.696 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:04:59 smithi100 bash[31660]: audit 2024-04-07T16:04:58.287100+0000 mon.a (mon.0) 226 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:04:59.696 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:04:59 smithi100 bash[31660]: audit 2024-04-07T16:04:58.295964+0000 mon.a (mon.0) 227 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:04:59.697 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:04:59 smithi100 bash[31660]: audit 2024-04-07T16:04:58.364130+0000 mon.a (mon.0) 228 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:05:01.098 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:05:00 smithi144 bash[23522]: audit 2024-04-07T16:04:59.899057+0000 mon.a (mon.0) 229 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:05:01.098 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:05:00 smithi144 bash[24472]: debug 2024-04-07T16:05:00.759+0000 7f87c7ad7200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-07T16:05:01.099 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:05:00 smithi144 bash[24472]: debug 2024-04-07T16:05:00.855+0000 7f87c7ad7200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-07T16:05:01.099 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:05:01 smithi144 bash[24472]: debug 2024-04-07T16:05:01.103+0000 7f87c7ad7200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-07T16:05:01.195 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:05:00 smithi100 bash[21346]: audit 2024-04-07T16:04:59.899057+0000 mon.a (mon.0) 229 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:05:01.196 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:05:00 smithi100 bash[31660]: audit 2024-04-07T16:04:59.899057+0000 mon.a (mon.0) 229 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:05:02.195 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:05:01 smithi100 bash[21346]: cluster 2024-04-07T16:05:00.175626+0000 mgr.y (mgr.14152) 77 : cluster 0 pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-07T16:05:02.196 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:05:01 smithi100 bash[31660]: cluster 2024-04-07T16:05:00.175626+0000 mgr.y (mgr.14152) 77 : cluster 0 pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-07T16:05:02.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:05:01 smithi144 bash[23522]: cluster 2024-04-07T16:05:00.175626+0000 mgr.y (mgr.14152) 77 : cluster 0 pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-07T16:05:02.360 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:05:01 smithi144 bash[24472]: debug 2024-04-07T16:05:01.883+0000 7f87c7ad7200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-07T16:05:02.360 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:05:01 smithi144 bash[24472]: debug 2024-04-07T16:05:01.991+0000 7f87c7ad7200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-07T16:05:02.360 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:05:02 smithi144 bash[24472]: debug 2024-04-07T16:05:02.091+0000 7f87c7ad7200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-07T16:05:02.443 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/mon.a/config 2024-04-07T16:05:02.859 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:05:02 smithi144 bash[24472]: debug 2024-04-07T16:05:02.439+0000 7f87c7ad7200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-07T16:05:02.859 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:05:02 smithi144 bash[24472]: debug 2024-04-07T16:05:02.551+0000 7f87c7ad7200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-07T16:05:02.860 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:05:02 smithi144 bash[24472]: debug 2024-04-07T16:05:02.651+0000 7f87c7ad7200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-07T16:05:03.359 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:05:03 smithi144 bash[24472]: debug 2024-04-07T16:05:02.999+0000 7f87c7ad7200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-07T16:05:03.359 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:05:03 smithi144 bash[24472]: debug 2024-04-07T16:05:03.243+0000 7f87c7ad7200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-07T16:05:03.860 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:05:03 smithi144 bash[24472]: debug 2024-04-07T16:05:03.503+0000 7f87c7ad7200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-07T16:05:03.860 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:05:03 smithi144 bash[24472]: debug 2024-04-07T16:05:03.611+0000 7f87c7ad7200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-07T16:05:04.196 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:05:04 smithi100 bash[31660]: cluster 2024-04-07T16:05:02.175987+0000 mgr.y (mgr.14152) 78 : cluster 0 pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-07T16:05:04.196 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:05:04 smithi100 bash[31660]: audit 2024-04-07T16:05:03.449273+0000 mon.a (mon.0) 230 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:05:04.196 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:05:04 smithi100 bash[31660]: audit 2024-04-07T16:05:03.455197+0000 mon.a (mon.0) 231 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:05:04.196 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:05:04 smithi100 bash[31660]: audit 2024-04-07T16:05:03.456013+0000 mon.a (mon.0) 232 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:05:04.196 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:05:04 smithi100 bash[31660]: audit 2024-04-07T16:05:03.457036+0000 mon.a (mon.0) 233 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:05:04.196 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:05:04 smithi100 bash[31660]: audit 2024-04-07T16:05:03.463837+0000 mon.a (mon.0) 234 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:05:04.196 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:05:04 smithi100 bash[31660]: audit 2024-04-07T16:05:03.530132+0000 mon.a (mon.0) 235 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-07T16:05:04.196 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:05:04 smithi100 bash[31660]: audit 2024-04-07T16:05:03.531237+0000 mon.a (mon.0) 236 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-07T16:05:04.196 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:05:04 smithi100 bash[31660]: audit 2024-04-07T16:05:03.532039+0000 mon.a (mon.0) 237 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:05:04.196 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:05:03 smithi100 bash[21346]: cluster 2024-04-07T16:05:02.175987+0000 mgr.y (mgr.14152) 78 : cluster 0 pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-07T16:05:04.196 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:05:03 smithi100 bash[21346]: audit 2024-04-07T16:05:03.449273+0000 mon.a (mon.0) 230 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:05:04.196 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:05:04 smithi100 bash[21346]: audit 2024-04-07T16:05:03.455197+0000 mon.a (mon.0) 231 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:05:04.197 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:05:04 smithi100 bash[21346]: audit 2024-04-07T16:05:03.456013+0000 mon.a (mon.0) 232 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:05:04.197 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:05:04 smithi100 bash[21346]: audit 2024-04-07T16:05:03.457036+0000 mon.a (mon.0) 233 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:05:04.197 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:05:04 smithi100 bash[21346]: audit 2024-04-07T16:05:03.463837+0000 mon.a (mon.0) 234 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:05:04.197 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:05:04 smithi100 bash[21346]: audit 2024-04-07T16:05:03.530132+0000 mon.a (mon.0) 235 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-07T16:05:04.197 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:05:04 smithi100 bash[21346]: audit 2024-04-07T16:05:03.531237+0000 mon.a (mon.0) 236 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-07T16:05:04.197 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:05:04 smithi100 bash[21346]: audit 2024-04-07T16:05:03.532039+0000 mon.a (mon.0) 237 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:05:04.360 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:05:03 smithi144 bash[24472]: debug 2024-04-07T16:05:03.879+0000 7f87c7ad7200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-07T16:05:04.360 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:05:04 smithi144 bash[24472]: debug 2024-04-07T16:05:04.119+0000 7f87c7ad7200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-07T16:05:04.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:05:03 smithi144 bash[23522]: cluster 2024-04-07T16:05:02.175987+0000 mgr.y (mgr.14152) 78 : cluster 0 pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-07T16:05:04.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:05:03 smithi144 bash[23522]: audit 2024-04-07T16:05:03.449273+0000 mon.a (mon.0) 230 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:05:04.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:05:03 smithi144 bash[23522]: audit 2024-04-07T16:05:03.455197+0000 mon.a (mon.0) 231 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:05:04.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:05:03 smithi144 bash[23522]: audit 2024-04-07T16:05:03.456013+0000 mon.a (mon.0) 232 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:05:04.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:05:03 smithi144 bash[23522]: audit 2024-04-07T16:05:03.457036+0000 mon.a (mon.0) 233 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:05:04.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:05:03 smithi144 bash[23522]: audit 2024-04-07T16:05:03.463837+0000 mon.a (mon.0) 234 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:05:04.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:05:03 smithi144 bash[23522]: audit 2024-04-07T16:05:03.530132+0000 mon.a (mon.0) 235 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-07T16:05:04.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:05:03 smithi144 bash[23522]: audit 2024-04-07T16:05:03.531237+0000 mon.a (mon.0) 236 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-07T16:05:04.361 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:05:03 smithi144 bash[23522]: audit 2024-04-07T16:05:03.532039+0000 mon.a (mon.0) 237 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:05:04.860 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:05:04 smithi144 bash[24472]: debug 2024-04-07T16:05:04.523+0000 7f87c7ad7200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-07T16:05:04.860 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:05:04 smithi144 bash[24472]: debug 2024-04-07T16:05:04.639+0000 7f87c7ad7200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-07T16:05:05.176 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:05:04 smithi100 bash[21346]: cephadm 2024-04-07T16:05:03.529609+0000 mgr.y (mgr.14152) 79 : cephadm 1 Reconfiguring mgr.y (unknown last config time)... 2024-04-07T16:05:05.176 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:05:04 smithi100 bash[21346]: cephadm 2024-04-07T16:05:03.533272+0000 mgr.y (mgr.14152) 80 : cephadm 1 Reconfiguring daemon mgr.y on smithi100 2024-04-07T16:05:05.176 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:05:04 smithi100 bash[31660]: cephadm 2024-04-07T16:05:03.529609+0000 mgr.y (mgr.14152) 79 : cephadm 1 Reconfiguring mgr.y (unknown last config time)... 2024-04-07T16:05:05.176 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:05:04 smithi100 bash[31660]: cephadm 2024-04-07T16:05:03.533272+0000 mgr.y (mgr.14152) 80 : cephadm 1 Reconfiguring daemon mgr.y on smithi100 2024-04-07T16:05:05.360 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:05:04 smithi144 bash[24472]: debug 2024-04-07T16:05:04.883+0000 7f87c7ad7200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-07T16:05:05.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:05:04 smithi144 bash[23522]: cephadm 2024-04-07T16:05:03.529609+0000 mgr.y (mgr.14152) 79 : cephadm 1 Reconfiguring mgr.y (unknown last config time)... 2024-04-07T16:05:05.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:05:04 smithi144 bash[23522]: cephadm 2024-04-07T16:05:03.533272+0000 mgr.y (mgr.14152) 80 : cephadm 1 Reconfiguring daemon mgr.y on smithi100 2024-04-07T16:05:05.693 INFO:teuthology.orchestra.run.smithi100.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-07T16:05:05.693 INFO:teuthology.orchestra.run.smithi100.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-07T16:05:05.693 INFO:teuthology.orchestra.run.smithi100.stderr: stderr: 10+0 records in 2024-04-07T16:05:05.693 INFO:teuthology.orchestra.run.smithi100.stderr:10+0 records out 2024-04-07T16:05:05.693 INFO:teuthology.orchestra.run.smithi100.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.015491 s, 677 MB/s 2024-04-07T16:05:05.693 INFO:teuthology.orchestra.run.smithi100.stderr:--> Zapping successful for: 2024-04-07T16:05:06.195 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:05:05 smithi100 bash[21346]: cluster 2024-04-07T16:05:04.176426+0000 mgr.y (mgr.14152) 81 : cluster 0 pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-07T16:05:06.196 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:05:05 smithi100 bash[21346]: audit 2024-04-07T16:05:05.746475+0000 mon.a (mon.0) 238 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:05:06.196 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:05:05 smithi100 bash[21346]: audit 2024-04-07T16:05:05.751245+0000 mon.a (mon.0) 239 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:05:06.196 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:05:05 smithi100 bash[21346]: audit 2024-04-07T16:05:05.802447+0000 mon.a (mon.0) 240 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:05:06.196 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:05:05 smithi100 bash[31660]: cluster 2024-04-07T16:05:04.176426+0000 mgr.y (mgr.14152) 81 : cluster 0 pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-07T16:05:06.196 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:05:05 smithi100 bash[31660]: audit 2024-04-07T16:05:05.746475+0000 mon.a (mon.0) 238 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:05:06.196 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:05:05 smithi100 bash[31660]: audit 2024-04-07T16:05:05.751245+0000 mon.a (mon.0) 239 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:05:06.196 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:05:05 smithi100 bash[31660]: audit 2024-04-07T16:05:05.802447+0000 mon.a (mon.0) 240 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:05:06.282 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:05:05 smithi144 bash[24472]: debug 2024-04-07T16:05:05.879+0000 7f87c7ad7200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-07T16:05:06.282 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:05:05 smithi144 bash[24472]: debug 2024-04-07T16:05:05.995+0000 7f87c7ad7200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-07T16:05:06.282 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:05:06 smithi144 bash[24472]: debug 2024-04-07T16:05:06.287+0000 7f87c7ad7200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-07T16:05:06.283 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:05:05 smithi144 bash[23522]: cluster 2024-04-07T16:05:04.176426+0000 mgr.y (mgr.14152) 81 : cluster 0 pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-07T16:05:06.283 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:05:05 smithi144 bash[23522]: audit 2024-04-07T16:05:05.746475+0000 mon.a (mon.0) 238 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:05:06.283 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:05:05 smithi144 bash[23522]: audit 2024-04-07T16:05:05.751245+0000 mon.a (mon.0) 239 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:05:06.283 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:05:05 smithi144 bash[23522]: audit 2024-04-07T16:05:05.802447+0000 mon.a (mon.0) 240 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:05:06.336 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0702e3a8-f4f8-11ee-b649-cb9ed24678a4 -- ceph orch daemon add osd smithi100:vg_nvme/lv_4 2024-04-07T16:05:06.610 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:05:06 smithi144 bash[24472]: debug 2024-04-07T16:05:06.399+0000 7f87c7ad7200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-07T16:05:07.110 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:05:06 smithi144 bash[24472]: debug 2024-04-07T16:05:06.839+0000 7f87c7ad7200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-07T16:05:07.110 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:05:06 smithi144 bash[24472]: debug 2024-04-07T16:05:06.959+0000 7f87c7ad7200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-07T16:05:07.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:05:06 smithi144 bash[23522]: audit 2024-04-07T16:05:06.329970+0000 mon.a (mon.0) 241 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:05:07.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:05:06 smithi144 bash[23522]: audit 2024-04-07T16:05:06.331628+0000 mon.a (mon.0) 242 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:05:07.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:05:06 smithi144 bash[23522]: audit 2024-04-07T16:05:06.339775+0000 mon.a (mon.0) 243 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:05:07.195 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:05:06 smithi100 bash[21346]: audit 2024-04-07T16:05:06.329970+0000 mon.a (mon.0) 241 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:05:07.196 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:05:06 smithi100 bash[21346]: audit 2024-04-07T16:05:06.331628+0000 mon.a (mon.0) 242 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:05:07.196 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:05:06 smithi100 bash[21346]: audit 2024-04-07T16:05:06.339775+0000 mon.a (mon.0) 243 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:05:07.196 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:05:06 smithi100 bash[31660]: audit 2024-04-07T16:05:06.329970+0000 mon.a (mon.0) 241 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:05:07.196 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:05:06 smithi100 bash[31660]: audit 2024-04-07T16:05:06.331628+0000 mon.a (mon.0) 242 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:05:07.196 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:05:06 smithi100 bash[31660]: audit 2024-04-07T16:05:06.339775+0000 mon.a (mon.0) 243 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:05:08.196 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:05:07 smithi100 bash[21346]: cluster 2024-04-07T16:05:06.176759+0000 mgr.y (mgr.14152) 82 : cluster 0 pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-07T16:05:08.196 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:05:07 smithi100 bash[21346]: cluster 2024-04-07T16:05:06.984958+0000 mon.a (mon.0) 244 : cluster 0 Standby manager daemon x started 2024-04-07T16:05:08.196 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:05:07 smithi100 bash[21346]: audit 2024-04-07T16:05:06.986842+0000 mon.b (mon.1) 3 : audit 0 from='mgr.? 172.21.15.144:0/564533397' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-04-07T16:05:08.196 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:05:07 smithi100 bash[21346]: audit 2024-04-07T16:05:06.987628+0000 mon.b (mon.1) 4 : audit 0 from='mgr.? 172.21.15.144:0/564533397' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-07T16:05:08.196 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:05:07 smithi100 bash[21346]: audit 2024-04-07T16:05:06.989031+0000 mon.b (mon.1) 5 : audit 0 from='mgr.? 172.21.15.144:0/564533397' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-04-07T16:05:08.196 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:05:07 smithi100 bash[21346]: audit 2024-04-07T16:05:06.989713+0000 mon.b (mon.1) 6 : audit 0 from='mgr.? 172.21.15.144:0/564533397' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-07T16:05:08.196 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:05:07 smithi100 bash[31660]: cluster 2024-04-07T16:05:06.176759+0000 mgr.y (mgr.14152) 82 : cluster 0 pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-07T16:05:08.196 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:05:07 smithi100 bash[31660]: cluster 2024-04-07T16:05:06.984958+0000 mon.a (mon.0) 244 : cluster 0 Standby manager daemon x started 2024-04-07T16:05:08.197 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:05:07 smithi100 bash[31660]: audit 2024-04-07T16:05:06.986842+0000 mon.b (mon.1) 3 : audit 0 from='mgr.? 172.21.15.144:0/564533397' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-04-07T16:05:08.197 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:05:07 smithi100 bash[31660]: audit 2024-04-07T16:05:06.987628+0000 mon.b (mon.1) 4 : audit 0 from='mgr.? 172.21.15.144:0/564533397' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-07T16:05:08.197 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:05:07 smithi100 bash[31660]: audit 2024-04-07T16:05:06.989031+0000 mon.b (mon.1) 5 : audit 0 from='mgr.? 172.21.15.144:0/564533397' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-04-07T16:05:08.197 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:05:07 smithi100 bash[31660]: audit 2024-04-07T16:05:06.989713+0000 mon.b (mon.1) 6 : audit 0 from='mgr.? 172.21.15.144:0/564533397' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-07T16:05:08.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:05:07 smithi144 bash[23522]: cluster 2024-04-07T16:05:06.176759+0000 mgr.y (mgr.14152) 82 : cluster 0 pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-07T16:05:08.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:05:07 smithi144 bash[23522]: cluster 2024-04-07T16:05:06.984958+0000 mon.a (mon.0) 244 : cluster 0 Standby manager daemon x started 2024-04-07T16:05:08.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:05:07 smithi144 bash[23522]: audit 2024-04-07T16:05:06.986842+0000 mon.b (mon.1) 3 : audit 0 from='mgr.? 172.21.15.144:0/564533397' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-04-07T16:05:08.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:05:07 smithi144 bash[23522]: audit 2024-04-07T16:05:06.987628+0000 mon.b (mon.1) 4 : audit 0 from='mgr.? 172.21.15.144:0/564533397' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-07T16:05:08.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:05:07 smithi144 bash[23522]: audit 2024-04-07T16:05:06.989031+0000 mon.b (mon.1) 5 : audit 0 from='mgr.? 172.21.15.144:0/564533397' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-04-07T16:05:08.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:05:07 smithi144 bash[23522]: audit 2024-04-07T16:05:06.989713+0000 mon.b (mon.1) 6 : audit 0 from='mgr.? 172.21.15.144:0/564533397' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-07T16:05:09.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:05:08 smithi144 bash[23522]: cluster 2024-04-07T16:05:07.957053+0000 mon.a (mon.0) 245 : cluster 0 mgrmap e14: y(active, since 103s), standbys: x 2024-04-07T16:05:09.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:05:08 smithi144 bash[23522]: audit 2024-04-07T16:05:07.957307+0000 mon.a (mon.0) 246 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-07T16:05:09.445 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:05:08 smithi100 bash[21346]: cluster 2024-04-07T16:05:07.957053+0000 mon.a (mon.0) 245 : cluster 0 mgrmap e14: y(active, since 103s), standbys: x 2024-04-07T16:05:09.446 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:05:08 smithi100 bash[21346]: audit 2024-04-07T16:05:07.957307+0000 mon.a (mon.0) 246 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-07T16:05:09.446 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:05:08 smithi100 bash[31660]: cluster 2024-04-07T16:05:07.957053+0000 mon.a (mon.0) 245 : cluster 0 mgrmap e14: y(active, since 103s), standbys: x 2024-04-07T16:05:09.446 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:05:08 smithi100 bash[31660]: audit 2024-04-07T16:05:07.957307+0000 mon.a (mon.0) 246 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-07T16:05:10.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:05:09 smithi144 bash[23522]: cluster 2024-04-07T16:05:08.177249+0000 mgr.y (mgr.14152) 83 : cluster 0 pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-07T16:05:10.446 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:05:09 smithi100 bash[21346]: cluster 2024-04-07T16:05:08.177249+0000 mgr.y (mgr.14152) 83 : cluster 0 pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-07T16:05:10.446 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:05:09 smithi100 bash[31660]: cluster 2024-04-07T16:05:08.177249+0000 mgr.y (mgr.14152) 83 : cluster 0 pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-07T16:05:11.008 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/mon.a/config 2024-04-07T16:05:12.065 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:05:11 smithi100 bash[21346]: cluster 2024-04-07T16:05:10.177757+0000 mgr.y (mgr.14152) 84 : cluster 0 pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-07T16:05:12.065 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:05:11 smithi100 bash[31660]: cluster 2024-04-07T16:05:10.177757+0000 mgr.y (mgr.14152) 84 : cluster 0 pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-07T16:05:12.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:05:11 smithi144 bash[23522]: cluster 2024-04-07T16:05:10.177757+0000 mgr.y (mgr.14152) 84 : cluster 0 pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-07T16:05:14.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:05:13 smithi144 bash[23522]: cluster 2024-04-07T16:05:12.178241+0000 mgr.y (mgr.14152) 85 : cluster 0 pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-07T16:05:14.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:05:13 smithi144 bash[23522]: audit 2024-04-07T16:05:13.595220+0000 mon.a (mon.0) 247 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-07T16:05:14.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:05:13 smithi144 bash[23522]: audit 2024-04-07T16:05:13.601193+0000 mon.a (mon.0) 248 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-07T16:05:14.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:05:13 smithi144 bash[23522]: audit 2024-04-07T16:05:13.602262+0000 mon.a (mon.0) 249 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:05:14.446 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:05:13 smithi100 bash[31660]: cluster 2024-04-07T16:05:12.178241+0000 mgr.y (mgr.14152) 85 : cluster 0 pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-07T16:05:14.446 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:05:13 smithi100 bash[31660]: audit 2024-04-07T16:05:13.595220+0000 mon.a (mon.0) 247 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-07T16:05:14.446 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:05:13 smithi100 bash[31660]: audit 2024-04-07T16:05:13.601193+0000 mon.a (mon.0) 248 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-07T16:05:14.446 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:05:13 smithi100 bash[31660]: audit 2024-04-07T16:05:13.602262+0000 mon.a (mon.0) 249 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:05:14.446 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:05:13 smithi100 bash[21346]: cluster 2024-04-07T16:05:12.178241+0000 mgr.y (mgr.14152) 85 : cluster 0 pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-07T16:05:14.446 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:05:13 smithi100 bash[21346]: audit 2024-04-07T16:05:13.595220+0000 mon.a (mon.0) 247 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-07T16:05:14.446 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:05:13 smithi100 bash[21346]: audit 2024-04-07T16:05:13.601193+0000 mon.a (mon.0) 248 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-07T16:05:14.446 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:05:13 smithi100 bash[21346]: audit 2024-04-07T16:05:13.602262+0000 mon.a (mon.0) 249 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:05:15.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:05:14 smithi144 bash[23522]: audit 2024-04-07T16:05:13.588344+0000 mgr.y (mgr.14152) 86 : audit 0 from='client.14208 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:05:15.445 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:05:14 smithi100 bash[31660]: audit 2024-04-07T16:05:13.588344+0000 mgr.y (mgr.14152) 86 : audit 0 from='client.14208 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:05:15.446 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:05:14 smithi100 bash[21346]: audit 2024-04-07T16:05:13.588344+0000 mgr.y (mgr.14152) 86 : audit 0 from='client.14208 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:05:16.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:05:15 smithi144 bash[23522]: cluster 2024-04-07T16:05:14.178659+0000 mgr.y (mgr.14152) 87 : cluster 0 pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-07T16:05:16.445 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:05:15 smithi100 bash[21346]: cluster 2024-04-07T16:05:14.178659+0000 mgr.y (mgr.14152) 87 : cluster 0 pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-07T16:05:16.445 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:05:15 smithi100 bash[31660]: cluster 2024-04-07T16:05:14.178659+0000 mgr.y (mgr.14152) 87 : cluster 0 pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-07T16:05:18.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:05:17 smithi144 bash[23522]: cluster 2024-04-07T16:05:16.179073+0000 mgr.y (mgr.14152) 88 : cluster 0 pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-07T16:05:18.429 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:05:17 smithi100 bash[21346]: cluster 2024-04-07T16:05:16.179073+0000 mgr.y (mgr.14152) 88 : cluster 0 pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-07T16:05:18.430 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:05:17 smithi100 bash[31660]: cluster 2024-04-07T16:05:16.179073+0000 mgr.y (mgr.14152) 88 : cluster 0 pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-07T16:05:20.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:05:19 smithi144 bash[23522]: cluster 2024-04-07T16:05:18.179501+0000 mgr.y (mgr.14152) 89 : cluster 0 pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-07T16:05:20.445 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:05:19 smithi100 bash[21346]: cluster 2024-04-07T16:05:18.179501+0000 mgr.y (mgr.14152) 89 : cluster 0 pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-07T16:05:20.445 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:05:19 smithi100 bash[31660]: cluster 2024-04-07T16:05:18.179501+0000 mgr.y (mgr.14152) 89 : cluster 0 pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-07T16:05:22.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:05:21 smithi144 bash[23522]: cluster 2024-04-07T16:05:20.179861+0000 mgr.y (mgr.14152) 90 : cluster 0 pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-07T16:05:22.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:05:21 smithi144 bash[23522]: audit 2024-04-07T16:05:21.560039+0000 mon.c (mon.2) 3 : audit 1 from='client.? 172.21.15.100:0/1410850036' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8facd48e-0a0c-4fda-aee2-f001b60a3811"}]: dispatch 2024-04-07T16:05:22.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:05:21 smithi144 bash[23522]: audit 2024-04-07T16:05:21.560637+0000 mon.a (mon.0) 250 : audit 1 from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8facd48e-0a0c-4fda-aee2-f001b60a3811"}]: dispatch 2024-04-07T16:05:22.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:05:21 smithi144 bash[23522]: audit 2024-04-07T16:05:21.568343+0000 mon.a (mon.0) 251 : audit 1 from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8facd48e-0a0c-4fda-aee2-f001b60a3811"}]': finished 2024-04-07T16:05:22.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:05:21 smithi144 bash[23522]: cluster 2024-04-07T16:05:21.568394+0000 mon.a (mon.0) 252 : cluster 0 osdmap e5: 1 total, 0 up, 1 in 2024-04-07T16:05:22.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:05:21 smithi144 bash[23522]: audit 2024-04-07T16:05:21.568519+0000 mon.a (mon.0) 253 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-07T16:05:22.445 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:05:21 smithi100 bash[21346]: cluster 2024-04-07T16:05:20.179861+0000 mgr.y (mgr.14152) 90 : cluster 0 pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-07T16:05:22.446 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:05:21 smithi100 bash[21346]: audit 2024-04-07T16:05:21.560039+0000 mon.c (mon.2) 3 : audit 1 from='client.? 172.21.15.100:0/1410850036' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8facd48e-0a0c-4fda-aee2-f001b60a3811"}]: dispatch 2024-04-07T16:05:22.446 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:05:21 smithi100 bash[21346]: audit 2024-04-07T16:05:21.560637+0000 mon.a (mon.0) 250 : audit 1 from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8facd48e-0a0c-4fda-aee2-f001b60a3811"}]: dispatch 2024-04-07T16:05:22.446 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:05:21 smithi100 bash[21346]: audit 2024-04-07T16:05:21.568343+0000 mon.a (mon.0) 251 : audit 1 from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8facd48e-0a0c-4fda-aee2-f001b60a3811"}]': finished 2024-04-07T16:05:22.446 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:05:21 smithi100 bash[21346]: cluster 2024-04-07T16:05:21.568394+0000 mon.a (mon.0) 252 : cluster 0 osdmap e5: 1 total, 0 up, 1 in 2024-04-07T16:05:22.446 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:05:21 smithi100 bash[21346]: audit 2024-04-07T16:05:21.568519+0000 mon.a (mon.0) 253 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-07T16:05:22.446 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:05:21 smithi100 bash[31660]: cluster 2024-04-07T16:05:20.179861+0000 mgr.y (mgr.14152) 90 : cluster 0 pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-07T16:05:22.446 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:05:21 smithi100 bash[31660]: audit 2024-04-07T16:05:21.560039+0000 mon.c (mon.2) 3 : audit 1 from='client.? 172.21.15.100:0/1410850036' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8facd48e-0a0c-4fda-aee2-f001b60a3811"}]: dispatch 2024-04-07T16:05:22.446 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:05:21 smithi100 bash[31660]: audit 2024-04-07T16:05:21.560637+0000 mon.a (mon.0) 250 : audit 1 from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8facd48e-0a0c-4fda-aee2-f001b60a3811"}]: dispatch 2024-04-07T16:05:22.446 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:05:21 smithi100 bash[31660]: audit 2024-04-07T16:05:21.568343+0000 mon.a (mon.0) 251 : audit 1 from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8facd48e-0a0c-4fda-aee2-f001b60a3811"}]': finished 2024-04-07T16:05:22.446 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:05:21 smithi100 bash[31660]: cluster 2024-04-07T16:05:21.568394+0000 mon.a (mon.0) 252 : cluster 0 osdmap e5: 1 total, 0 up, 1 in 2024-04-07T16:05:22.446 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:05:21 smithi100 bash[31660]: audit 2024-04-07T16:05:21.568519+0000 mon.a (mon.0) 253 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-07T16:05:23.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:05:22 smithi144 bash[23522]: audit 2024-04-07T16:05:22.241879+0000 mon.c (mon.2) 4 : audit 0 from='client.? 172.21.15.100:0/285516' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-07T16:05:23.445 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:05:22 smithi100 bash[21346]: audit 2024-04-07T16:05:22.241879+0000 mon.c (mon.2) 4 : audit 0 from='client.? 172.21.15.100:0/285516' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-07T16:05:23.446 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:05:22 smithi100 bash[31660]: audit 2024-04-07T16:05:22.241879+0000 mon.c (mon.2) 4 : audit 0 from='client.? 172.21.15.100:0/285516' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-07T16:05:24.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:05:23 smithi144 bash[23522]: cluster 2024-04-07T16:05:22.180226+0000 mgr.y (mgr.14152) 91 : cluster 0 pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-07T16:05:24.445 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:05:23 smithi100 bash[21346]: cluster 2024-04-07T16:05:22.180226+0000 mgr.y (mgr.14152) 91 : cluster 0 pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-07T16:05:24.446 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:05:23 smithi100 bash[31660]: cluster 2024-04-07T16:05:22.180226+0000 mgr.y (mgr.14152) 91 : cluster 0 pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-07T16:05:26.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:05:26 smithi144 bash[23522]: cluster 2024-04-07T16:05:24.180577+0000 mgr.y (mgr.14152) 92 : cluster 0 pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-07T16:05:26.445 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:05:25 smithi100 bash[21346]: cluster 2024-04-07T16:05:24.180577+0000 mgr.y (mgr.14152) 92 : cluster 0 pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-07T16:05:26.446 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:05:26 smithi100 bash[31660]: cluster 2024-04-07T16:05:24.180577+0000 mgr.y (mgr.14152) 92 : cluster 0 pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-07T16:05:28.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:05:28 smithi144 bash[23522]: cluster 2024-04-07T16:05:26.180982+0000 mgr.y (mgr.14152) 93 : cluster 0 pgmap v56: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-07T16:05:28.445 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:05:28 smithi100 bash[21346]: cluster 2024-04-07T16:05:26.180982+0000 mgr.y (mgr.14152) 93 : cluster 0 pgmap v56: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-07T16:05:28.446 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:05:28 smithi100 bash[31660]: cluster 2024-04-07T16:05:26.180982+0000 mgr.y (mgr.14152) 93 : cluster 0 pgmap v56: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-07T16:05:30.288 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:05:30 smithi100 bash[21346]: cluster 2024-04-07T16:05:28.181390+0000 mgr.y (mgr.14152) 94 : cluster 0 pgmap v57: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-07T16:05:30.288 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:05:30 smithi100 bash[31660]: cluster 2024-04-07T16:05:28.181390+0000 mgr.y (mgr.14152) 94 : cluster 0 pgmap v57: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-07T16:05:30.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:05:30 smithi144 bash[23522]: cluster 2024-04-07T16:05:28.181390+0000 mgr.y (mgr.14152) 94 : cluster 0 pgmap v57: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-07T16:05:32.195 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:05:32 smithi100 bash[21346]: cluster 2024-04-07T16:05:30.181793+0000 mgr.y (mgr.14152) 95 : cluster 0 pgmap v58: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-07T16:05:32.196 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:05:32 smithi100 bash[31660]: cluster 2024-04-07T16:05:30.181793+0000 mgr.y (mgr.14152) 95 : cluster 0 pgmap v58: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-07T16:05:32.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:05:32 smithi144 bash[23522]: cluster 2024-04-07T16:05:30.181793+0000 mgr.y (mgr.14152) 95 : cluster 0 pgmap v58: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-07T16:05:34.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:05:34 smithi144 bash[23522]: cluster 2024-04-07T16:05:32.182246+0000 mgr.y (mgr.14152) 96 : cluster 0 pgmap v59: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-07T16:05:34.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:05:34 smithi144 bash[23522]: audit 2024-04-07T16:05:33.445497+0000 mon.a (mon.0) 254 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-07T16:05:34.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:05:34 smithi144 bash[23522]: audit 2024-04-07T16:05:33.447351+0000 mon.a (mon.0) 255 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:05:34.420 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:05:34 smithi100 bash[21346]: cluster 2024-04-07T16:05:32.182246+0000 mgr.y (mgr.14152) 96 : cluster 0 pgmap v59: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-07T16:05:34.420 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:05:34 smithi100 bash[21346]: audit 2024-04-07T16:05:33.445497+0000 mon.a (mon.0) 254 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-07T16:05:34.420 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:05:34 smithi100 bash[21346]: audit 2024-04-07T16:05:33.447351+0000 mon.a (mon.0) 255 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:05:34.420 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:05:34 smithi100 bash[31660]: cluster 2024-04-07T16:05:32.182246+0000 mgr.y (mgr.14152) 96 : cluster 0 pgmap v59: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-07T16:05:34.420 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:05:34 smithi100 bash[31660]: audit 2024-04-07T16:05:33.445497+0000 mon.a (mon.0) 254 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-07T16:05:34.420 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:05:34 smithi100 bash[31660]: audit 2024-04-07T16:05:33.447351+0000 mon.a (mon.0) 255 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:05:35.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:05:35 smithi144 bash[23522]: cephadm 2024-04-07T16:05:33.448991+0000 mgr.y (mgr.14152) 97 : cephadm 1 Deploying daemon osd.0 on smithi100 2024-04-07T16:05:35.445 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:05:35 smithi100 bash[21346]: cephadm 2024-04-07T16:05:33.448991+0000 mgr.y (mgr.14152) 97 : cephadm 1 Deploying daemon osd.0 on smithi100 2024-04-07T16:05:35.445 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:05:35 smithi100 bash[31660]: cephadm 2024-04-07T16:05:33.448991+0000 mgr.y (mgr.14152) 97 : cephadm 1 Deploying daemon osd.0 on smithi100 2024-04-07T16:05:36.273 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:05:36 smithi100 bash[31660]: cluster 2024-04-07T16:05:34.182679+0000 mgr.y (mgr.14152) 98 : cluster 0 pgmap v60: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-07T16:05:36.273 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:05:36 smithi100 bash[21346]: cluster 2024-04-07T16:05:34.182679+0000 mgr.y (mgr.14152) 98 : cluster 0 pgmap v60: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-07T16:05:36.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:05:36 smithi144 bash[23522]: cluster 2024-04-07T16:05:34.182679+0000 mgr.y (mgr.14152) 98 : cluster 0 pgmap v60: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-07T16:05:37.246 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:05:37 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:05:37.247 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:05:37 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:05:37.247 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:05:37 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:05:37.514 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:05:37 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:05:37.515 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:05:37 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:05:37.515 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:05:37 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:05:38.313 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:05:38 smithi100 bash[31660]: cluster 2024-04-07T16:05:36.182967+0000 mgr.y (mgr.14152) 99 : cluster 0 pgmap v61: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-07T16:05:38.313 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:05:38 smithi100 bash[31660]: audit 2024-04-07T16:05:37.505178+0000 mon.a (mon.0) 256 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:05:38.313 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:05:38 smithi100 bash[31660]: audit 2024-04-07T16:05:37.511953+0000 mon.a (mon.0) 257 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:05:38.313 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:05:38 smithi100 bash[31660]: audit 2024-04-07T16:05:37.518299+0000 mon.a (mon.0) 258 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:05:38.313 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:05:38 smithi100 bash[21346]: cluster 2024-04-07T16:05:36.182967+0000 mgr.y (mgr.14152) 99 : cluster 0 pgmap v61: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-07T16:05:38.313 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:05:38 smithi100 bash[21346]: audit 2024-04-07T16:05:37.505178+0000 mon.a (mon.0) 256 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:05:38.313 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:05:38 smithi100 bash[21346]: audit 2024-04-07T16:05:37.511953+0000 mon.a (mon.0) 257 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:05:38.313 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:05:38 smithi100 bash[21346]: audit 2024-04-07T16:05:37.518299+0000 mon.a (mon.0) 258 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:05:38.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:05:38 smithi144 bash[23522]: cluster 2024-04-07T16:05:36.182967+0000 mgr.y (mgr.14152) 99 : cluster 0 pgmap v61: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-07T16:05:38.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:05:38 smithi144 bash[23522]: audit 2024-04-07T16:05:37.505178+0000 mon.a (mon.0) 256 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:05:38.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:05:38 smithi144 bash[23522]: audit 2024-04-07T16:05:37.511953+0000 mon.a (mon.0) 257 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:05:38.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:05:38 smithi144 bash[23522]: audit 2024-04-07T16:05:37.518299+0000 mon.a (mon.0) 258 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:05:40.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:05:40 smithi144 bash[23522]: cluster 2024-04-07T16:05:38.183367+0000 mgr.y (mgr.14152) 100 : cluster 0 pgmap v62: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-07T16:05:40.445 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:05:40 smithi100 bash[21346]: cluster 2024-04-07T16:05:38.183367+0000 mgr.y (mgr.14152) 100 : cluster 0 pgmap v62: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-07T16:05:40.445 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:05:40 smithi100 bash[31660]: cluster 2024-04-07T16:05:38.183367+0000 mgr.y (mgr.14152) 100 : cluster 0 pgmap v62: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-07T16:05:42.330 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:05:42 smithi100 bash[21346]: cluster 2024-04-07T16:05:40.183833+0000 mgr.y (mgr.14152) 101 : cluster 0 pgmap v63: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-07T16:05:42.330 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:05:42 smithi100 bash[31660]: cluster 2024-04-07T16:05:40.183833+0000 mgr.y (mgr.14152) 101 : cluster 0 pgmap v63: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-07T16:05:42.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:05:42 smithi144 bash[23522]: cluster 2024-04-07T16:05:40.183833+0000 mgr.y (mgr.14152) 101 : cluster 0 pgmap v63: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-07T16:05:43.945 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:05:43 smithi100 bash[21346]: cluster 2024-04-07T16:05:42.184197+0000 mgr.y (mgr.14152) 102 : cluster 0 pgmap v64: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-07T16:05:43.946 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:05:43 smithi100 bash[21346]: audit 2024-04-07T16:05:42.640987+0000 mon.a (mon.0) 259 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:05:43.946 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:05:43 smithi100 bash[21346]: audit 2024-04-07T16:05:42.646540+0000 mon.a (mon.0) 260 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:05:43.946 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:05:43 smithi100 bash[21346]: audit 2024-04-07T16:05:43.115322+0000 mon.a (mon.0) 261 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:05:43.946 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:05:43 smithi100 bash[21346]: audit 2024-04-07T16:05:43.116366+0000 mon.a (mon.0) 262 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:05:43.946 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:05:43 smithi100 bash[21346]: audit 2024-04-07T16:05:43.122393+0000 mon.a (mon.0) 263 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:05:43.946 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:05:43 smithi100 bash[31660]: cluster 2024-04-07T16:05:42.184197+0000 mgr.y (mgr.14152) 102 : cluster 0 pgmap v64: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-07T16:05:43.946 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:05:43 smithi100 bash[31660]: audit 2024-04-07T16:05:42.640987+0000 mon.a (mon.0) 259 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:05:43.946 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:05:43 smithi100 bash[31660]: audit 2024-04-07T16:05:42.646540+0000 mon.a (mon.0) 260 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:05:43.946 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:05:43 smithi100 bash[31660]: audit 2024-04-07T16:05:43.115322+0000 mon.a (mon.0) 261 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:05:43.946 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:05:43 smithi100 bash[31660]: audit 2024-04-07T16:05:43.116366+0000 mon.a (mon.0) 262 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:05:43.946 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:05:43 smithi100 bash[31660]: audit 2024-04-07T16:05:43.122393+0000 mon.a (mon.0) 263 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:05:44.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:05:43 smithi144 bash[23522]: cluster 2024-04-07T16:05:42.184197+0000 mgr.y (mgr.14152) 102 : cluster 0 pgmap v64: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-07T16:05:44.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:05:43 smithi144 bash[23522]: audit 2024-04-07T16:05:42.640987+0000 mon.a (mon.0) 259 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:05:44.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:05:43 smithi144 bash[23522]: audit 2024-04-07T16:05:42.646540+0000 mon.a (mon.0) 260 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:05:44.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:05:43 smithi144 bash[23522]: audit 2024-04-07T16:05:43.115322+0000 mon.a (mon.0) 261 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:05:44.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:05:43 smithi144 bash[23522]: audit 2024-04-07T16:05:43.116366+0000 mon.a (mon.0) 262 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:05:44.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:05:43 smithi144 bash[23522]: audit 2024-04-07T16:05:43.122393+0000 mon.a (mon.0) 263 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:05:44.945 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:05:44 smithi100 bash[21346]: audit 2024-04-07T16:05:43.703018+0000 mon.c (mon.2) 5 : audit 1 from='osd.0 [v2:172.21.15.100:6802/1636950303,v1:172.21.15.100:6803/1636950303]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-07T16:05:44.945 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:05:44 smithi100 bash[21346]: audit 2024-04-07T16:05:43.703356+0000 mon.a (mon.0) 264 : audit 1 from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-07T16:05:44.946 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:05:44 smithi100 bash[31660]: audit 2024-04-07T16:05:43.703018+0000 mon.c (mon.2) 5 : audit 1 from='osd.0 [v2:172.21.15.100:6802/1636950303,v1:172.21.15.100:6803/1636950303]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-07T16:05:44.946 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:05:44 smithi100 bash[31660]: audit 2024-04-07T16:05:43.703356+0000 mon.a (mon.0) 264 : audit 1 from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-07T16:05:45.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:05:44 smithi144 bash[23522]: audit 2024-04-07T16:05:43.703018+0000 mon.c (mon.2) 5 : audit 1 from='osd.0 [v2:172.21.15.100:6802/1636950303,v1:172.21.15.100:6803/1636950303]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-07T16:05:45.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:05:44 smithi144 bash[23522]: audit 2024-04-07T16:05:43.703356+0000 mon.a (mon.0) 264 : audit 1 from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-07T16:05:45.896 INFO:teuthology.orchestra.run.smithi100.stdout:Created osd(s) 0 on host 'smithi100' 2024-04-07T16:05:45.914 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:05:45 smithi100 bash[21346]: cluster 2024-04-07T16:05:44.184586+0000 mgr.y (mgr.14152) 103 : cluster 0 pgmap v65: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-07T16:05:45.914 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:05:45 smithi100 bash[21346]: audit 2024-04-07T16:05:44.657136+0000 mon.a (mon.0) 265 : audit 1 from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-07T16:05:45.915 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:05:45 smithi100 bash[21346]: cluster 2024-04-07T16:05:44.657250+0000 mon.a (mon.0) 266 : cluster 0 osdmap e6: 1 total, 0 up, 1 in 2024-04-07T16:05:45.915 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:05:45 smithi100 bash[21346]: audit 2024-04-07T16:05:44.657431+0000 mon.a (mon.0) 267 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-07T16:05:45.915 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:05:45 smithi100 bash[21346]: audit 2024-04-07T16:05:44.658219+0000 mon.c (mon.2) 6 : audit 1 from='osd.0 [v2:172.21.15.100:6802/1636950303,v1:172.21.15.100:6803/1636950303]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-04-07T16:05:45.915 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:05:45 smithi100 bash[21346]: audit 2024-04-07T16:05:44.658694+0000 mon.a (mon.0) 268 : audit 1 from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-04-07T16:05:45.915 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:05:45 smithi100 bash[31660]: cluster 2024-04-07T16:05:44.184586+0000 mgr.y (mgr.14152) 103 : cluster 0 pgmap v65: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-07T16:05:45.915 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:05:45 smithi100 bash[31660]: audit 2024-04-07T16:05:44.657136+0000 mon.a (mon.0) 265 : audit 1 from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-07T16:05:45.915 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:05:45 smithi100 bash[31660]: cluster 2024-04-07T16:05:44.657250+0000 mon.a (mon.0) 266 : cluster 0 osdmap e6: 1 total, 0 up, 1 in 2024-04-07T16:05:45.915 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:05:45 smithi100 bash[31660]: audit 2024-04-07T16:05:44.657431+0000 mon.a (mon.0) 267 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-07T16:05:45.915 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:05:45 smithi100 bash[31660]: audit 2024-04-07T16:05:44.658219+0000 mon.c (mon.2) 6 : audit 1 from='osd.0 [v2:172.21.15.100:6802/1636950303,v1:172.21.15.100:6803/1636950303]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-04-07T16:05:45.915 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:05:45 smithi100 bash[31660]: audit 2024-04-07T16:05:44.658694+0000 mon.a (mon.0) 268 : audit 1 from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-04-07T16:05:46.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:05:45 smithi144 bash[23522]: cluster 2024-04-07T16:05:44.184586+0000 mgr.y (mgr.14152) 103 : cluster 0 pgmap v65: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-07T16:05:46.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:05:45 smithi144 bash[23522]: audit 2024-04-07T16:05:44.657136+0000 mon.a (mon.0) 265 : audit 1 from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-07T16:05:46.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:05:45 smithi144 bash[23522]: cluster 2024-04-07T16:05:44.657250+0000 mon.a (mon.0) 266 : cluster 0 osdmap e6: 1 total, 0 up, 1 in 2024-04-07T16:05:46.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:05:45 smithi144 bash[23522]: audit 2024-04-07T16:05:44.657431+0000 mon.a (mon.0) 267 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-07T16:05:46.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:05:45 smithi144 bash[23522]: audit 2024-04-07T16:05:44.658219+0000 mon.c (mon.2) 6 : audit 1 from='osd.0 [v2:172.21.15.100:6802/1636950303,v1:172.21.15.100:6803/1636950303]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-04-07T16:05:46.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:05:45 smithi144 bash[23522]: audit 2024-04-07T16:05:44.658694+0000 mon.a (mon.0) 268 : audit 1 from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-04-07T16:05:46.525 DEBUG:teuthology.orchestra.run.smithi100:osd.0> sudo journalctl -f -n 0 -u ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@osd.0.service 2024-04-07T16:05:46.528 INFO:tasks.cephadm:Deploying osd.1 on smithi100 with /dev/vg_nvme/lv_3... 2024-04-07T16:05:46.528 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0702e3a8-f4f8-11ee-b649-cb9ed24678a4 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-07T16:05:46.946 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:05:46 smithi100 bash[21346]: audit 2024-04-07T16:05:45.658096+0000 mon.a (mon.0) 269 : audit 1 from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]': finished 2024-04-07T16:05:46.946 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:05:46 smithi100 bash[21346]: cluster 2024-04-07T16:05:45.658319+0000 mon.a (mon.0) 270 : cluster 0 osdmap e7: 1 total, 0 up, 1 in 2024-04-07T16:05:46.946 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:05:46 smithi100 bash[21346]: audit 2024-04-07T16:05:45.659050+0000 mon.a (mon.0) 271 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-07T16:05:46.946 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:05:46 smithi100 bash[21346]: audit 2024-04-07T16:05:45.672033+0000 mon.a (mon.0) 272 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-07T16:05:46.946 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:05:46 smithi100 bash[21346]: audit 2024-04-07T16:05:45.847100+0000 mon.a (mon.0) 273 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:05:46.946 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:05:46 smithi100 bash[21346]: audit 2024-04-07T16:05:45.847539+0000 mon.a (mon.0) 274 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:05:46.946 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:05:46 smithi100 bash[21346]: audit 2024-04-07T16:05:45.856718+0000 mon.a (mon.0) 275 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:05:46.946 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:05:46 smithi100 bash[31660]: audit 2024-04-07T16:05:45.658096+0000 mon.a (mon.0) 269 : audit 1 from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]': finished 2024-04-07T16:05:46.946 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:05:46 smithi100 bash[31660]: cluster 2024-04-07T16:05:45.658319+0000 mon.a (mon.0) 270 : cluster 0 osdmap e7: 1 total, 0 up, 1 in 2024-04-07T16:05:46.947 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:05:46 smithi100 bash[31660]: audit 2024-04-07T16:05:45.659050+0000 mon.a (mon.0) 271 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-07T16:05:46.947 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:05:46 smithi100 bash[31660]: audit 2024-04-07T16:05:45.672033+0000 mon.a (mon.0) 272 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-07T16:05:46.947 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:05:46 smithi100 bash[31660]: audit 2024-04-07T16:05:45.847100+0000 mon.a (mon.0) 273 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:05:46.947 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:05:46 smithi100 bash[31660]: audit 2024-04-07T16:05:45.847539+0000 mon.a (mon.0) 274 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:05:46.947 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:05:46 smithi100 bash[31660]: audit 2024-04-07T16:05:45.856718+0000 mon.a (mon.0) 275 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:05:47.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:05:46 smithi144 bash[23522]: audit 2024-04-07T16:05:45.658096+0000 mon.a (mon.0) 269 : audit 1 from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]': finished 2024-04-07T16:05:47.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:05:46 smithi144 bash[23522]: cluster 2024-04-07T16:05:45.658319+0000 mon.a (mon.0) 270 : cluster 0 osdmap e7: 1 total, 0 up, 1 in 2024-04-07T16:05:47.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:05:46 smithi144 bash[23522]: audit 2024-04-07T16:05:45.659050+0000 mon.a (mon.0) 271 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-07T16:05:47.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:05:46 smithi144 bash[23522]: audit 2024-04-07T16:05:45.672033+0000 mon.a (mon.0) 272 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-07T16:05:47.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:05:46 smithi144 bash[23522]: audit 2024-04-07T16:05:45.847100+0000 mon.a (mon.0) 273 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:05:47.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:05:46 smithi144 bash[23522]: audit 2024-04-07T16:05:45.847539+0000 mon.a (mon.0) 274 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:05:47.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:05:46 smithi144 bash[23522]: audit 2024-04-07T16:05:45.856718+0000 mon.a (mon.0) 275 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:05:47.945 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:05:47 smithi100 bash[21346]: cluster 2024-04-07T16:05:44.696598+0000 osd.0 (osd.0) 1 : cluster 0 purged_snaps scrub starts 2024-04-07T16:05:47.946 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:05:47 smithi100 bash[21346]: cluster 2024-04-07T16:05:44.696665+0000 osd.0 (osd.0) 2 : cluster 0 purged_snaps scrub ok 2024-04-07T16:05:47.946 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:05:47 smithi100 bash[21346]: cluster 2024-04-07T16:05:46.185012+0000 mgr.y (mgr.14152) 104 : cluster 0 pgmap v68: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-07T16:05:47.946 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:05:47 smithi100 bash[21346]: cluster 2024-04-07T16:05:46.672096+0000 mon.a (mon.0) 276 : cluster 1 osd.0 [v2:172.21.15.100:6802/1636950303,v1:172.21.15.100:6803/1636950303] boot 2024-04-07T16:05:47.946 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:05:47 smithi100 bash[21346]: cluster 2024-04-07T16:05:46.672185+0000 mon.a (mon.0) 277 : cluster 0 osdmap e8: 1 total, 1 up, 1 in 2024-04-07T16:05:47.946 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:05:47 smithi100 bash[21346]: audit 2024-04-07T16:05:46.672791+0000 mon.a (mon.0) 278 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-07T16:05:47.946 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:05:47 smithi100 bash[31660]: cluster 2024-04-07T16:05:44.696598+0000 osd.0 (osd.0) 1 : cluster 0 purged_snaps scrub starts 2024-04-07T16:05:47.946 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:05:47 smithi100 bash[31660]: cluster 2024-04-07T16:05:44.696665+0000 osd.0 (osd.0) 2 : cluster 0 purged_snaps scrub ok 2024-04-07T16:05:47.946 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:05:47 smithi100 bash[31660]: cluster 2024-04-07T16:05:46.185012+0000 mgr.y (mgr.14152) 104 : cluster 0 pgmap v68: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-07T16:05:47.946 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:05:47 smithi100 bash[31660]: cluster 2024-04-07T16:05:46.672096+0000 mon.a (mon.0) 276 : cluster 1 osd.0 [v2:172.21.15.100:6802/1636950303,v1:172.21.15.100:6803/1636950303] boot 2024-04-07T16:05:47.947 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:05:47 smithi100 bash[31660]: cluster 2024-04-07T16:05:46.672185+0000 mon.a (mon.0) 277 : cluster 0 osdmap e8: 1 total, 1 up, 1 in 2024-04-07T16:05:47.947 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:05:47 smithi100 bash[31660]: audit 2024-04-07T16:05:46.672791+0000 mon.a (mon.0) 278 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-07T16:05:48.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:05:47 smithi144 bash[23522]: cluster 2024-04-07T16:05:44.696598+0000 osd.0 (osd.0) 1 : cluster 0 purged_snaps scrub starts 2024-04-07T16:05:48.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:05:47 smithi144 bash[23522]: cluster 2024-04-07T16:05:44.696665+0000 osd.0 (osd.0) 2 : cluster 0 purged_snaps scrub ok 2024-04-07T16:05:48.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:05:47 smithi144 bash[23522]: cluster 2024-04-07T16:05:46.185012+0000 mgr.y (mgr.14152) 104 : cluster 0 pgmap v68: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-07T16:05:48.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:05:47 smithi144 bash[23522]: cluster 2024-04-07T16:05:46.672096+0000 mon.a (mon.0) 276 : cluster 1 osd.0 [v2:172.21.15.100:6802/1636950303,v1:172.21.15.100:6803/1636950303] boot 2024-04-07T16:05:48.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:05:47 smithi144 bash[23522]: cluster 2024-04-07T16:05:46.672185+0000 mon.a (mon.0) 277 : cluster 0 osdmap e8: 1 total, 1 up, 1 in 2024-04-07T16:05:48.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:05:47 smithi144 bash[23522]: audit 2024-04-07T16:05:46.672791+0000 mon.a (mon.0) 278 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-07T16:05:48.945 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:05:48 smithi100 bash[21346]: cluster 2024-04-07T16:05:47.691833+0000 mon.a (mon.0) 279 : cluster 0 osdmap e9: 1 total, 1 up, 1 in 2024-04-07T16:05:48.945 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:05:48 smithi100 bash[31660]: cluster 2024-04-07T16:05:47.691833+0000 mon.a (mon.0) 279 : cluster 0 osdmap e9: 1 total, 1 up, 1 in 2024-04-07T16:05:49.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:05:48 smithi144 bash[23522]: cluster 2024-04-07T16:05:47.691833+0000 mon.a (mon.0) 279 : cluster 0 osdmap e9: 1 total, 1 up, 1 in 2024-04-07T16:05:49.945 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:05:49 smithi100 bash[21346]: cluster 2024-04-07T16:05:48.185515+0000 mgr.y (mgr.14152) 105 : cluster 0 pgmap v71: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-07T16:05:49.946 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:05:49 smithi100 bash[31660]: cluster 2024-04-07T16:05:48.185515+0000 mgr.y (mgr.14152) 105 : cluster 0 pgmap v71: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-07T16:05:50.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:05:49 smithi144 bash[23522]: cluster 2024-04-07T16:05:48.185515+0000 mgr.y (mgr.14152) 105 : cluster 0 pgmap v71: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-07T16:05:50.692 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/mon.a/config 2024-04-07T16:05:51.945 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:05:51 smithi100 bash[21346]: cluster 2024-04-07T16:05:50.185959+0000 mgr.y (mgr.14152) 106 : cluster 0 pgmap v72: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-07T16:05:51.945 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:05:51 smithi100 bash[31660]: cluster 2024-04-07T16:05:50.185959+0000 mgr.y (mgr.14152) 106 : cluster 0 pgmap v72: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-07T16:05:52.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:05:51 smithi144 bash[23522]: cluster 2024-04-07T16:05:50.185959+0000 mgr.y (mgr.14152) 106 : cluster 0 pgmap v72: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-07T16:05:53.945 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:05:53 smithi100 bash[21346]: cluster 2024-04-07T16:05:52.186456+0000 mgr.y (mgr.14152) 107 : cluster 0 pgmap v73: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-07T16:05:53.945 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:05:53 smithi100 bash[31660]: cluster 2024-04-07T16:05:52.186456+0000 mgr.y (mgr.14152) 107 : cluster 0 pgmap v73: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-07T16:05:54.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:05:53 smithi144 bash[23522]: cluster 2024-04-07T16:05:52.186456+0000 mgr.y (mgr.14152) 107 : cluster 0 pgmap v73: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-07T16:05:54.429 INFO:teuthology.orchestra.run.smithi100.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-07T16:05:54.429 INFO:teuthology.orchestra.run.smithi100.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-07T16:05:54.429 INFO:teuthology.orchestra.run.smithi100.stderr: stderr: 10+0 records in 2024-04-07T16:05:54.430 INFO:teuthology.orchestra.run.smithi100.stderr:10+0 records out 2024-04-07T16:05:54.430 INFO:teuthology.orchestra.run.smithi100.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.014157 s, 741 MB/s 2024-04-07T16:05:54.430 INFO:teuthology.orchestra.run.smithi100.stderr:--> Zapping successful for: 2024-04-07T16:05:55.157 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0702e3a8-f4f8-11ee-b649-cb9ed24678a4 -- ceph orch daemon add osd smithi100:vg_nvme/lv_3 2024-04-07T16:05:56.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:05:55 smithi144 bash[23522]: cluster 2024-04-07T16:05:54.186760+0000 mgr.y (mgr.14152) 108 : cluster 0 pgmap v74: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-07T16:05:56.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:05:55 smithi144 bash[23522]: audit 2024-04-07T16:05:55.599052+0000 mon.a (mon.0) 280 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:05:56.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:05:55 smithi144 bash[23522]: audit 2024-04-07T16:05:55.610764+0000 mon.a (mon.0) 281 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:05:56.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:05:55 smithi144 bash[23522]: audit 2024-04-07T16:05:55.613009+0000 mon.a (mon.0) 282 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-07T16:05:56.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:05:55 smithi144 bash[23522]: audit 2024-04-07T16:05:55.623998+0000 mon.a (mon.0) 283 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:05:56.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:05:55 smithi144 bash[23522]: audit 2024-04-07T16:05:55.626038+0000 mon.a (mon.0) 284 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:05:56.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:05:55 smithi144 bash[23522]: audit 2024-04-07T16:05:55.627831+0000 mon.a (mon.0) 285 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:05:56.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:05:55 smithi144 bash[23522]: audit 2024-04-07T16:05:55.639657+0000 mon.a (mon.0) 286 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:05:56.196 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:05:55 smithi100 bash[21346]: cluster 2024-04-07T16:05:54.186760+0000 mgr.y (mgr.14152) 108 : cluster 0 pgmap v74: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-07T16:05:56.196 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:05:55 smithi100 bash[21346]: audit 2024-04-07T16:05:55.599052+0000 mon.a (mon.0) 280 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:05:56.196 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:05:55 smithi100 bash[21346]: audit 2024-04-07T16:05:55.610764+0000 mon.a (mon.0) 281 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:05:56.196 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:05:55 smithi100 bash[21346]: audit 2024-04-07T16:05:55.613009+0000 mon.a (mon.0) 282 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-07T16:05:56.196 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:05:55 smithi100 bash[21346]: audit 2024-04-07T16:05:55.623998+0000 mon.a (mon.0) 283 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:05:56.196 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:05:55 smithi100 bash[21346]: audit 2024-04-07T16:05:55.626038+0000 mon.a (mon.0) 284 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:05:56.196 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:05:55 smithi100 bash[21346]: audit 2024-04-07T16:05:55.627831+0000 mon.a (mon.0) 285 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:05:56.196 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:05:55 smithi100 bash[21346]: audit 2024-04-07T16:05:55.639657+0000 mon.a (mon.0) 286 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:05:56.196 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:05:55 smithi100 bash[31660]: cluster 2024-04-07T16:05:54.186760+0000 mgr.y (mgr.14152) 108 : cluster 0 pgmap v74: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-07T16:05:56.196 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:05:55 smithi100 bash[31660]: audit 2024-04-07T16:05:55.599052+0000 mon.a (mon.0) 280 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:05:56.196 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:05:55 smithi100 bash[31660]: audit 2024-04-07T16:05:55.610764+0000 mon.a (mon.0) 281 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:05:56.197 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:05:55 smithi100 bash[31660]: audit 2024-04-07T16:05:55.613009+0000 mon.a (mon.0) 282 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-07T16:05:56.197 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:05:55 smithi100 bash[31660]: audit 2024-04-07T16:05:55.623998+0000 mon.a (mon.0) 283 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:05:56.197 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:05:55 smithi100 bash[31660]: audit 2024-04-07T16:05:55.626038+0000 mon.a (mon.0) 284 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:05:56.197 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:05:55 smithi100 bash[31660]: audit 2024-04-07T16:05:55.627831+0000 mon.a (mon.0) 285 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:05:56.197 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:05:55 smithi100 bash[31660]: audit 2024-04-07T16:05:55.639657+0000 mon.a (mon.0) 286 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:05:57.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:05:56 smithi144 bash[23522]: cephadm 2024-04-07T16:05:55.589965+0000 mgr.y (mgr.14152) 109 : cephadm 1 Detected new or changed devices on smithi100 2024-04-07T16:05:57.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:05:56 smithi144 bash[23522]: cephadm 2024-04-07T16:05:55.614393+0000 mgr.y (mgr.14152) 110 : cephadm 1 Adjusting osd_memory_target on smithi100 to 16241M 2024-04-07T16:05:57.195 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:05:56 smithi100 bash[21346]: cephadm 2024-04-07T16:05:55.589965+0000 mgr.y (mgr.14152) 109 : cephadm 1 Detected new or changed devices on smithi100 2024-04-07T16:05:57.196 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:05:56 smithi100 bash[21346]: cephadm 2024-04-07T16:05:55.614393+0000 mgr.y (mgr.14152) 110 : cephadm 1 Adjusting osd_memory_target on smithi100 to 16241M 2024-04-07T16:05:57.196 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:05:56 smithi100 bash[31660]: cephadm 2024-04-07T16:05:55.589965+0000 mgr.y (mgr.14152) 109 : cephadm 1 Detected new or changed devices on smithi100 2024-04-07T16:05:57.196 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:05:56 smithi100 bash[31660]: cephadm 2024-04-07T16:05:55.614393+0000 mgr.y (mgr.14152) 110 : cephadm 1 Adjusting osd_memory_target on smithi100 to 16241M 2024-04-07T16:05:58.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:05:57 smithi144 bash[23522]: cluster 2024-04-07T16:05:56.187174+0000 mgr.y (mgr.14152) 111 : cluster 0 pgmap v75: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-07T16:05:58.195 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:05:57 smithi100 bash[21346]: cluster 2024-04-07T16:05:56.187174+0000 mgr.y (mgr.14152) 111 : cluster 0 pgmap v75: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-07T16:05:58.196 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:05:57 smithi100 bash[31660]: cluster 2024-04-07T16:05:56.187174+0000 mgr.y (mgr.14152) 111 : cluster 0 pgmap v75: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-07T16:05:59.837 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/mon.a/config 2024-04-07T16:06:00.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:05:59 smithi144 bash[23522]: cluster 2024-04-07T16:05:58.187673+0000 mgr.y (mgr.14152) 112 : cluster 0 pgmap v76: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-07T16:06:00.195 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:05:59 smithi100 bash[21346]: cluster 2024-04-07T16:05:58.187673+0000 mgr.y (mgr.14152) 112 : cluster 0 pgmap v76: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-07T16:06:00.196 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:05:59 smithi100 bash[31660]: cluster 2024-04-07T16:05:58.187673+0000 mgr.y (mgr.14152) 112 : cluster 0 pgmap v76: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-07T16:06:02.001 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:06:01 smithi100 bash[21346]: cluster 2024-04-07T16:06:00.188107+0000 mgr.y (mgr.14152) 113 : cluster 0 pgmap v77: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-07T16:06:02.001 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:06:01 smithi100 bash[31660]: cluster 2024-04-07T16:06:00.188107+0000 mgr.y (mgr.14152) 113 : cluster 0 pgmap v77: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-07T16:06:02.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:06:01 smithi144 bash[23522]: cluster 2024-04-07T16:06:00.188107+0000 mgr.y (mgr.14152) 113 : cluster 0 pgmap v77: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-07T16:06:03.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:06:02 smithi144 bash[23522]: audit 2024-04-07T16:06:02.390286+0000 mon.a (mon.0) 287 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-07T16:06:03.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:06:02 smithi144 bash[23522]: audit 2024-04-07T16:06:02.394409+0000 mon.a (mon.0) 288 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-07T16:06:03.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:06:02 smithi144 bash[23522]: audit 2024-04-07T16:06:02.395839+0000 mon.a (mon.0) 289 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:06:03.196 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:06:02 smithi100 bash[21346]: audit 2024-04-07T16:06:02.390286+0000 mon.a (mon.0) 287 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-07T16:06:03.196 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:06:02 smithi100 bash[21346]: audit 2024-04-07T16:06:02.394409+0000 mon.a (mon.0) 288 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-07T16:06:03.196 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:06:02 smithi100 bash[21346]: audit 2024-04-07T16:06:02.395839+0000 mon.a (mon.0) 289 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:06:03.196 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:06:02 smithi100 bash[31660]: audit 2024-04-07T16:06:02.390286+0000 mon.a (mon.0) 287 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-07T16:06:03.196 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:06:02 smithi100 bash[31660]: audit 2024-04-07T16:06:02.394409+0000 mon.a (mon.0) 288 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-07T16:06:03.196 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:06:02 smithi100 bash[31660]: audit 2024-04-07T16:06:02.395839+0000 mon.a (mon.0) 289 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:06:04.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:06:03 smithi144 bash[23522]: cluster 2024-04-07T16:06:02.188393+0000 mgr.y (mgr.14152) 114 : cluster 0 pgmap v78: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-07T16:06:04.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:06:03 smithi144 bash[23522]: audit 2024-04-07T16:06:02.383503+0000 mgr.y (mgr.14152) 115 : audit 0 from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:06:04.196 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:06:03 smithi100 bash[21346]: cluster 2024-04-07T16:06:02.188393+0000 mgr.y (mgr.14152) 114 : cluster 0 pgmap v78: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-07T16:06:04.196 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:06:03 smithi100 bash[21346]: audit 2024-04-07T16:06:02.383503+0000 mgr.y (mgr.14152) 115 : audit 0 from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:06:04.196 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:06:03 smithi100 bash[31660]: cluster 2024-04-07T16:06:02.188393+0000 mgr.y (mgr.14152) 114 : cluster 0 pgmap v78: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-07T16:06:04.196 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:06:03 smithi100 bash[31660]: audit 2024-04-07T16:06:02.383503+0000 mgr.y (mgr.14152) 115 : audit 0 from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:06:06.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:06:05 smithi144 bash[23522]: cluster 2024-04-07T16:06:04.188818+0000 mgr.y (mgr.14152) 116 : cluster 0 pgmap v79: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-07T16:06:06.195 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:06:05 smithi100 bash[21346]: cluster 2024-04-07T16:06:04.188818+0000 mgr.y (mgr.14152) 116 : cluster 0 pgmap v79: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-07T16:06:06.195 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:06:05 smithi100 bash[31660]: cluster 2024-04-07T16:06:04.188818+0000 mgr.y (mgr.14152) 116 : cluster 0 pgmap v79: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-07T16:06:07.992 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:06:07 smithi100 bash[21346]: cluster 2024-04-07T16:06:06.189279+0000 mgr.y (mgr.14152) 117 : cluster 0 pgmap v80: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-07T16:06:07.993 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:06:07 smithi100 bash[31660]: cluster 2024-04-07T16:06:06.189279+0000 mgr.y (mgr.14152) 117 : cluster 0 pgmap v80: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-07T16:06:08.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:06:07 smithi144 bash[23522]: cluster 2024-04-07T16:06:06.189279+0000 mgr.y (mgr.14152) 117 : cluster 0 pgmap v80: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-07T16:06:10.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:06:09 smithi144 bash[23522]: cluster 2024-04-07T16:06:08.189771+0000 mgr.y (mgr.14152) 118 : cluster 0 pgmap v81: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-07T16:06:10.195 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:06:09 smithi100 bash[21346]: cluster 2024-04-07T16:06:08.189771+0000 mgr.y (mgr.14152) 118 : cluster 0 pgmap v81: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-07T16:06:10.196 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:06:09 smithi100 bash[31660]: cluster 2024-04-07T16:06:08.189771+0000 mgr.y (mgr.14152) 118 : cluster 0 pgmap v81: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-07T16:06:11.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:06:10 smithi144 bash[23522]: audit 2024-04-07T16:06:10.257443+0000 mon.a (mon.0) 290 : audit 1 from='client.? 172.21.15.100:0/2918280651' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cb5f6c23-6764-48b6-8b21-97a36b004786"}]: dispatch 2024-04-07T16:06:11.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:06:10 smithi144 bash[23522]: audit 2024-04-07T16:06:10.262390+0000 mon.a (mon.0) 291 : audit 1 from='client.? 172.21.15.100:0/2918280651' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cb5f6c23-6764-48b6-8b21-97a36b004786"}]': finished 2024-04-07T16:06:11.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:06:10 smithi144 bash[23522]: cluster 2024-04-07T16:06:10.262454+0000 mon.a (mon.0) 292 : cluster 0 osdmap e10: 2 total, 1 up, 2 in 2024-04-07T16:06:11.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:06:10 smithi144 bash[23522]: audit 2024-04-07T16:06:10.262635+0000 mon.a (mon.0) 293 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-07T16:06:11.196 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:06:10 smithi100 bash[21346]: audit 2024-04-07T16:06:10.257443+0000 mon.a (mon.0) 290 : audit 1 from='client.? 172.21.15.100:0/2918280651' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cb5f6c23-6764-48b6-8b21-97a36b004786"}]: dispatch 2024-04-07T16:06:11.196 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:06:10 smithi100 bash[21346]: audit 2024-04-07T16:06:10.262390+0000 mon.a (mon.0) 291 : audit 1 from='client.? 172.21.15.100:0/2918280651' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cb5f6c23-6764-48b6-8b21-97a36b004786"}]': finished 2024-04-07T16:06:11.196 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:06:10 smithi100 bash[21346]: cluster 2024-04-07T16:06:10.262454+0000 mon.a (mon.0) 292 : cluster 0 osdmap e10: 2 total, 1 up, 2 in 2024-04-07T16:06:11.196 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:06:10 smithi100 bash[21346]: audit 2024-04-07T16:06:10.262635+0000 mon.a (mon.0) 293 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-07T16:06:11.196 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:06:10 smithi100 bash[31660]: audit 2024-04-07T16:06:10.257443+0000 mon.a (mon.0) 290 : audit 1 from='client.? 172.21.15.100:0/2918280651' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cb5f6c23-6764-48b6-8b21-97a36b004786"}]: dispatch 2024-04-07T16:06:11.196 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:06:10 smithi100 bash[31660]: audit 2024-04-07T16:06:10.262390+0000 mon.a (mon.0) 291 : audit 1 from='client.? 172.21.15.100:0/2918280651' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cb5f6c23-6764-48b6-8b21-97a36b004786"}]': finished 2024-04-07T16:06:11.196 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:06:10 smithi100 bash[31660]: cluster 2024-04-07T16:06:10.262454+0000 mon.a (mon.0) 292 : cluster 0 osdmap e10: 2 total, 1 up, 2 in 2024-04-07T16:06:11.196 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:06:10 smithi100 bash[31660]: audit 2024-04-07T16:06:10.262635+0000 mon.a (mon.0) 293 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-07T16:06:12.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:06:11 smithi144 bash[23522]: cluster 2024-04-07T16:06:10.190183+0000 mgr.y (mgr.14152) 119 : cluster 0 pgmap v82: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-07T16:06:12.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:06:11 smithi144 bash[23522]: audit 2024-04-07T16:06:11.003132+0000 mon.a (mon.0) 294 : audit 0 from='client.? 172.21.15.100:0/2910487424' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-07T16:06:12.196 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:06:11 smithi100 bash[21346]: cluster 2024-04-07T16:06:10.190183+0000 mgr.y (mgr.14152) 119 : cluster 0 pgmap v82: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-07T16:06:12.196 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:06:11 smithi100 bash[21346]: audit 2024-04-07T16:06:11.003132+0000 mon.a (mon.0) 294 : audit 0 from='client.? 172.21.15.100:0/2910487424' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-07T16:06:12.196 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:06:11 smithi100 bash[31660]: cluster 2024-04-07T16:06:10.190183+0000 mgr.y (mgr.14152) 119 : cluster 0 pgmap v82: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-07T16:06:12.196 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:06:11 smithi100 bash[31660]: audit 2024-04-07T16:06:11.003132+0000 mon.a (mon.0) 294 : audit 0 from='client.? 172.21.15.100:0/2910487424' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-07T16:06:13.945 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:06:13 smithi100 bash[21346]: cluster 2024-04-07T16:06:12.190598+0000 mgr.y (mgr.14152) 120 : cluster 0 pgmap v84: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-07T16:06:13.946 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:06:13 smithi100 bash[31660]: cluster 2024-04-07T16:06:12.190598+0000 mgr.y (mgr.14152) 120 : cluster 0 pgmap v84: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-07T16:06:14.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:06:13 smithi144 bash[23522]: cluster 2024-04-07T16:06:12.190598+0000 mgr.y (mgr.14152) 120 : cluster 0 pgmap v84: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-07T16:06:16.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:06:15 smithi144 bash[23522]: cluster 2024-04-07T16:06:14.191073+0000 mgr.y (mgr.14152) 121 : cluster 0 pgmap v85: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-07T16:06:16.195 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:06:15 smithi100 bash[21346]: cluster 2024-04-07T16:06:14.191073+0000 mgr.y (mgr.14152) 121 : cluster 0 pgmap v85: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-07T16:06:16.195 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:06:15 smithi100 bash[31660]: cluster 2024-04-07T16:06:14.191073+0000 mgr.y (mgr.14152) 121 : cluster 0 pgmap v85: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-07T16:06:18.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:06:17 smithi144 bash[23522]: cluster 2024-04-07T16:06:16.191593+0000 mgr.y (mgr.14152) 122 : cluster 0 pgmap v86: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-07T16:06:18.195 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:06:17 smithi100 bash[21346]: cluster 2024-04-07T16:06:16.191593+0000 mgr.y (mgr.14152) 122 : cluster 0 pgmap v86: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-07T16:06:18.196 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:06:17 smithi100 bash[31660]: cluster 2024-04-07T16:06:16.191593+0000 mgr.y (mgr.14152) 122 : cluster 0 pgmap v86: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-07T16:06:20.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:06:19 smithi144 bash[23522]: cluster 2024-04-07T16:06:18.192018+0000 mgr.y (mgr.14152) 123 : cluster 0 pgmap v87: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-07T16:06:20.136 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:06:19 smithi100 bash[21346]: cluster 2024-04-07T16:06:18.192018+0000 mgr.y (mgr.14152) 123 : cluster 0 pgmap v87: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-07T16:06:20.136 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:06:19 smithi100 bash[31660]: cluster 2024-04-07T16:06:18.192018+0000 mgr.y (mgr.14152) 123 : cluster 0 pgmap v87: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-07T16:06:22.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:06:21 smithi144 bash[23522]: cluster 2024-04-07T16:06:20.192514+0000 mgr.y (mgr.14152) 124 : cluster 0 pgmap v88: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-07T16:06:22.195 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:06:21 smithi100 bash[21346]: cluster 2024-04-07T16:06:20.192514+0000 mgr.y (mgr.14152) 124 : cluster 0 pgmap v88: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-07T16:06:22.195 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:06:21 smithi100 bash[31660]: cluster 2024-04-07T16:06:20.192514+0000 mgr.y (mgr.14152) 124 : cluster 0 pgmap v88: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-07T16:06:22.945 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:06:22 smithi100 bash[21346]: audit 2024-04-07T16:06:22.355240+0000 mon.a (mon.0) 295 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-07T16:06:22.946 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:06:22 smithi100 bash[21346]: audit 2024-04-07T16:06:22.356832+0000 mon.a (mon.0) 296 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:06:22.946 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:06:22 smithi100 bash[31660]: audit 2024-04-07T16:06:22.355240+0000 mon.a (mon.0) 295 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-07T16:06:22.946 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:06:22 smithi100 bash[31660]: audit 2024-04-07T16:06:22.356832+0000 mon.a (mon.0) 296 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:06:23.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:06:22 smithi144 bash[23522]: audit 2024-04-07T16:06:22.355240+0000 mon.a (mon.0) 295 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-07T16:06:23.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:06:22 smithi144 bash[23522]: audit 2024-04-07T16:06:22.356832+0000 mon.a (mon.0) 296 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:06:23.945 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:06:23 smithi100 bash[31660]: cluster 2024-04-07T16:06:22.192969+0000 mgr.y (mgr.14152) 125 : cluster 0 pgmap v89: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-07T16:06:23.946 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:06:23 smithi100 bash[31660]: cephadm 2024-04-07T16:06:22.358012+0000 mgr.y (mgr.14152) 126 : cephadm 1 Deploying daemon osd.1 on smithi100 2024-04-07T16:06:23.946 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:06:23 smithi100 bash[21346]: cluster 2024-04-07T16:06:22.192969+0000 mgr.y (mgr.14152) 125 : cluster 0 pgmap v89: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-07T16:06:23.946 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:06:23 smithi100 bash[21346]: cephadm 2024-04-07T16:06:22.358012+0000 mgr.y (mgr.14152) 126 : cephadm 1 Deploying daemon osd.1 on smithi100 2024-04-07T16:06:24.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:06:23 smithi144 bash[23522]: cluster 2024-04-07T16:06:22.192969+0000 mgr.y (mgr.14152) 125 : cluster 0 pgmap v89: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-07T16:06:24.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:06:23 smithi144 bash[23522]: cephadm 2024-04-07T16:06:22.358012+0000 mgr.y (mgr.14152) 126 : cephadm 1 Deploying daemon osd.1 on smithi100 2024-04-07T16:06:26.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:06:25 smithi144 bash[23522]: cluster 2024-04-07T16:06:24.193292+0000 mgr.y (mgr.14152) 127 : cluster 0 pgmap v90: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-07T16:06:26.154 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:06:25 smithi100 bash[21346]: cluster 2024-04-07T16:06:24.193292+0000 mgr.y (mgr.14152) 127 : cluster 0 pgmap v90: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-07T16:06:26.154 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:06:25 smithi100 bash[31660]: cluster 2024-04-07T16:06:24.193292+0000 mgr.y (mgr.14152) 127 : cluster 0 pgmap v90: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-07T16:06:26.445 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:06:26 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:06:26.446 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:06:26 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:06:26.446 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:06:26 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:06:26.446 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:06:26 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:06:26.446 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:06:26 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:06:26.446 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:06:26 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:06:26.447 INFO:journalctl@ceph.osd.0.smithi100.stdout:Apr 07 16:06:26 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:06:26.447 INFO:journalctl@ceph.osd.0.smithi100.stdout:Apr 07 16:06:26 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:06:27.695 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:06:27 smithi100 bash[21346]: cluster 2024-04-07T16:06:26.193654+0000 mgr.y (mgr.14152) 128 : cluster 0 pgmap v91: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-07T16:06:27.696 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:06:27 smithi100 bash[21346]: audit 2024-04-07T16:06:26.531313+0000 mon.a (mon.0) 297 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:06:27.696 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:06:27 smithi100 bash[21346]: audit 2024-04-07T16:06:26.539396+0000 mon.a (mon.0) 298 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:06:27.696 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:06:27 smithi100 bash[21346]: audit 2024-04-07T16:06:26.552673+0000 mon.a (mon.0) 299 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:06:27.696 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:06:27 smithi100 bash[31660]: cluster 2024-04-07T16:06:26.193654+0000 mgr.y (mgr.14152) 128 : cluster 0 pgmap v91: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-07T16:06:27.696 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:06:27 smithi100 bash[31660]: audit 2024-04-07T16:06:26.531313+0000 mon.a (mon.0) 297 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:06:27.696 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:06:27 smithi100 bash[31660]: audit 2024-04-07T16:06:26.539396+0000 mon.a (mon.0) 298 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:06:27.696 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:06:27 smithi100 bash[31660]: audit 2024-04-07T16:06:26.552673+0000 mon.a (mon.0) 299 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:06:27.860 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:06:27 smithi144 bash[23522]: cluster 2024-04-07T16:06:26.193654+0000 mgr.y (mgr.14152) 128 : cluster 0 pgmap v91: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-07T16:06:27.860 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:06:27 smithi144 bash[23522]: audit 2024-04-07T16:06:26.531313+0000 mon.a (mon.0) 297 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:06:27.860 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:06:27 smithi144 bash[23522]: audit 2024-04-07T16:06:26.539396+0000 mon.a (mon.0) 298 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:06:27.860 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:06:27 smithi144 bash[23522]: audit 2024-04-07T16:06:26.552673+0000 mon.a (mon.0) 299 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:06:29.806 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:06:29 smithi100 bash[31660]: cluster 2024-04-07T16:06:28.194108+0000 mgr.y (mgr.14152) 129 : cluster 0 pgmap v92: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-07T16:06:29.807 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:06:29 smithi100 bash[21346]: cluster 2024-04-07T16:06:28.194108+0000 mgr.y (mgr.14152) 129 : cluster 0 pgmap v92: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-07T16:06:29.860 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:06:29 smithi144 bash[23522]: cluster 2024-04-07T16:06:28.194108+0000 mgr.y (mgr.14152) 129 : cluster 0 pgmap v92: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-07T16:06:31.695 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:06:31 smithi100 bash[21346]: cluster 2024-04-07T16:06:30.194514+0000 mgr.y (mgr.14152) 130 : cluster 0 pgmap v93: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-07T16:06:31.695 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:06:31 smithi100 bash[31660]: cluster 2024-04-07T16:06:30.194514+0000 mgr.y (mgr.14152) 130 : cluster 0 pgmap v93: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-07T16:06:31.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:06:31 smithi144 bash[23522]: cluster 2024-04-07T16:06:30.194514+0000 mgr.y (mgr.14152) 130 : cluster 0 pgmap v93: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-07T16:06:33.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:06:32 smithi144 bash[23522]: audit 2024-04-07T16:06:31.704333+0000 mon.a (mon.0) 300 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:06:33.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:06:32 smithi144 bash[23522]: audit 2024-04-07T16:06:31.725863+0000 mon.a (mon.0) 301 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:06:33.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:06:32 smithi144 bash[23522]: audit 2024-04-07T16:06:31.726974+0000 mon.a (mon.0) 302 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:06:33.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:06:32 smithi144 bash[23522]: audit 2024-04-07T16:06:31.728166+0000 mon.a (mon.0) 303 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:06:33.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:06:32 smithi144 bash[23522]: audit 2024-04-07T16:06:31.734341+0000 mon.a (mon.0) 304 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:06:33.195 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:06:32 smithi100 bash[21346]: audit 2024-04-07T16:06:31.704333+0000 mon.a (mon.0) 300 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:06:33.195 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:06:32 smithi100 bash[21346]: audit 2024-04-07T16:06:31.725863+0000 mon.a (mon.0) 301 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:06:33.195 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:06:32 smithi100 bash[21346]: audit 2024-04-07T16:06:31.726974+0000 mon.a (mon.0) 302 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:06:33.196 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:06:32 smithi100 bash[21346]: audit 2024-04-07T16:06:31.728166+0000 mon.a (mon.0) 303 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:06:33.196 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:06:32 smithi100 bash[21346]: audit 2024-04-07T16:06:31.734341+0000 mon.a (mon.0) 304 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:06:33.196 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:06:32 smithi100 bash[31660]: audit 2024-04-07T16:06:31.704333+0000 mon.a (mon.0) 300 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:06:33.196 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:06:32 smithi100 bash[31660]: audit 2024-04-07T16:06:31.725863+0000 mon.a (mon.0) 301 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:06:33.196 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:06:32 smithi100 bash[31660]: audit 2024-04-07T16:06:31.726974+0000 mon.a (mon.0) 302 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:06:33.196 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:06:32 smithi100 bash[31660]: audit 2024-04-07T16:06:31.728166+0000 mon.a (mon.0) 303 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:06:33.196 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:06:32 smithi100 bash[31660]: audit 2024-04-07T16:06:31.734341+0000 mon.a (mon.0) 304 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:06:33.945 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:06:33 smithi100 bash[31660]: cluster 2024-04-07T16:06:32.194839+0000 mgr.y (mgr.14152) 131 : cluster 0 pgmap v94: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-07T16:06:33.945 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:06:33 smithi100 bash[31660]: audit 2024-04-07T16:06:32.763192+0000 mon.c (mon.2) 7 : audit 1 from='osd.1 [v2:172.21.15.100:6810/470179506,v1:172.21.15.100:6811/470179506]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-07T16:06:33.945 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:06:33 smithi100 bash[31660]: audit 2024-04-07T16:06:32.763559+0000 mon.a (mon.0) 305 : audit 1 from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-07T16:06:33.946 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:06:33 smithi100 bash[21346]: cluster 2024-04-07T16:06:32.194839+0000 mgr.y (mgr.14152) 131 : cluster 0 pgmap v94: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-07T16:06:33.946 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:06:33 smithi100 bash[21346]: audit 2024-04-07T16:06:32.763192+0000 mon.c (mon.2) 7 : audit 1 from='osd.1 [v2:172.21.15.100:6810/470179506,v1:172.21.15.100:6811/470179506]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-07T16:06:33.946 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:06:33 smithi100 bash[21346]: audit 2024-04-07T16:06:32.763559+0000 mon.a (mon.0) 305 : audit 1 from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-07T16:06:34.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:06:33 smithi144 bash[23522]: cluster 2024-04-07T16:06:32.194839+0000 mgr.y (mgr.14152) 131 : cluster 0 pgmap v94: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-07T16:06:34.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:06:33 smithi144 bash[23522]: audit 2024-04-07T16:06:32.763192+0000 mon.c (mon.2) 7 : audit 1 from='osd.1 [v2:172.21.15.100:6810/470179506,v1:172.21.15.100:6811/470179506]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-07T16:06:34.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:06:33 smithi144 bash[23522]: audit 2024-04-07T16:06:32.763559+0000 mon.a (mon.0) 305 : audit 1 from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-07T16:06:34.796 INFO:teuthology.orchestra.run.smithi100.stdout:Created osd(s) 1 on host 'smithi100' 2024-04-07T16:06:35.002 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:06:34 smithi100 bash[21346]: audit 2024-04-07T16:06:33.751606+0000 mon.a (mon.0) 306 : audit 1 from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-07T16:06:35.002 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:06:34 smithi100 bash[21346]: cluster 2024-04-07T16:06:33.751801+0000 mon.a (mon.0) 307 : cluster 0 osdmap e11: 2 total, 1 up, 2 in 2024-04-07T16:06:35.003 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:06:34 smithi100 bash[21346]: audit 2024-04-07T16:06:33.751972+0000 mon.a (mon.0) 308 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-07T16:06:35.003 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:06:34 smithi100 bash[21346]: audit 2024-04-07T16:06:33.752584+0000 mon.c (mon.2) 8 : audit 1 from='osd.1 [v2:172.21.15.100:6810/470179506,v1:172.21.15.100:6811/470179506]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-04-07T16:06:35.003 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:06:34 smithi100 bash[21346]: audit 2024-04-07T16:06:33.753136+0000 mon.a (mon.0) 309 : audit 1 from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-04-07T16:06:35.003 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:06:34 smithi100 bash[31660]: audit 2024-04-07T16:06:33.751606+0000 mon.a (mon.0) 306 : audit 1 from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-07T16:06:35.003 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:06:34 smithi100 bash[31660]: cluster 2024-04-07T16:06:33.751801+0000 mon.a (mon.0) 307 : cluster 0 osdmap e11: 2 total, 1 up, 2 in 2024-04-07T16:06:35.003 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:06:34 smithi100 bash[31660]: audit 2024-04-07T16:06:33.751972+0000 mon.a (mon.0) 308 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-07T16:06:35.003 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:06:34 smithi100 bash[31660]: audit 2024-04-07T16:06:33.752584+0000 mon.c (mon.2) 8 : audit 1 from='osd.1 [v2:172.21.15.100:6810/470179506,v1:172.21.15.100:6811/470179506]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-04-07T16:06:35.003 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:06:34 smithi100 bash[31660]: audit 2024-04-07T16:06:33.753136+0000 mon.a (mon.0) 309 : audit 1 from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-04-07T16:06:35.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:06:34 smithi144 bash[23522]: audit 2024-04-07T16:06:33.751606+0000 mon.a (mon.0) 306 : audit 1 from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-07T16:06:35.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:06:34 smithi144 bash[23522]: cluster 2024-04-07T16:06:33.751801+0000 mon.a (mon.0) 307 : cluster 0 osdmap e11: 2 total, 1 up, 2 in 2024-04-07T16:06:35.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:06:34 smithi144 bash[23522]: audit 2024-04-07T16:06:33.751972+0000 mon.a (mon.0) 308 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-07T16:06:35.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:06:34 smithi144 bash[23522]: audit 2024-04-07T16:06:33.752584+0000 mon.c (mon.2) 8 : audit 1 from='osd.1 [v2:172.21.15.100:6810/470179506,v1:172.21.15.100:6811/470179506]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-04-07T16:06:35.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:06:34 smithi144 bash[23522]: audit 2024-04-07T16:06:33.753136+0000 mon.a (mon.0) 309 : audit 1 from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-04-07T16:06:35.428 DEBUG:teuthology.orchestra.run.smithi100:osd.1> sudo journalctl -f -n 0 -u ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@osd.1.service 2024-04-07T16:06:35.433 INFO:tasks.cephadm:Deploying osd.2 on smithi100 with /dev/vg_nvme/lv_2... 2024-04-07T16:06:35.433 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0702e3a8-f4f8-11ee-b649-cb9ed24678a4 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-07T16:06:36.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:06:35 smithi144 bash[23522]: cluster 2024-04-07T16:06:34.195262+0000 mgr.y (mgr.14152) 132 : cluster 0 pgmap v96: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-07T16:06:36.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:06:35 smithi144 bash[23522]: audit 2024-04-07T16:06:34.750869+0000 mon.a (mon.0) 310 : audit 1 from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]': finished 2024-04-07T16:06:36.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:06:35 smithi144 bash[23522]: cluster 2024-04-07T16:06:34.750977+0000 mon.a (mon.0) 311 : cluster 0 osdmap e12: 2 total, 1 up, 2 in 2024-04-07T16:06:36.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:06:35 smithi144 bash[23522]: audit 2024-04-07T16:06:34.751481+0000 mon.a (mon.0) 312 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-07T16:06:36.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:06:35 smithi144 bash[23522]: audit 2024-04-07T16:06:34.761369+0000 mon.a (mon.0) 313 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-07T16:06:36.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:06:35 smithi144 bash[23522]: audit 2024-04-07T16:06:34.771292+0000 mon.a (mon.0) 314 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:06:36.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:06:35 smithi144 bash[23522]: audit 2024-04-07T16:06:34.778069+0000 mon.a (mon.0) 315 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:06:36.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:06:35 smithi144 bash[23522]: audit 2024-04-07T16:06:34.805272+0000 mon.a (mon.0) 316 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:06:36.196 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:06:35 smithi100 bash[31660]: cluster 2024-04-07T16:06:34.195262+0000 mgr.y (mgr.14152) 132 : cluster 0 pgmap v96: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-07T16:06:36.196 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:06:35 smithi100 bash[31660]: audit 2024-04-07T16:06:34.750869+0000 mon.a (mon.0) 310 : audit 1 from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]': finished 2024-04-07T16:06:36.196 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:06:35 smithi100 bash[31660]: cluster 2024-04-07T16:06:34.750977+0000 mon.a (mon.0) 311 : cluster 0 osdmap e12: 2 total, 1 up, 2 in 2024-04-07T16:06:36.196 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:06:35 smithi100 bash[31660]: audit 2024-04-07T16:06:34.751481+0000 mon.a (mon.0) 312 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-07T16:06:36.196 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:06:35 smithi100 bash[31660]: audit 2024-04-07T16:06:34.761369+0000 mon.a (mon.0) 313 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-07T16:06:36.196 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:06:35 smithi100 bash[31660]: audit 2024-04-07T16:06:34.771292+0000 mon.a (mon.0) 314 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:06:36.196 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:06:35 smithi100 bash[31660]: audit 2024-04-07T16:06:34.778069+0000 mon.a (mon.0) 315 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:06:36.196 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:06:35 smithi100 bash[31660]: audit 2024-04-07T16:06:34.805272+0000 mon.a (mon.0) 316 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:06:36.196 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:06:35 smithi100 bash[21346]: cluster 2024-04-07T16:06:34.195262+0000 mgr.y (mgr.14152) 132 : cluster 0 pgmap v96: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-07T16:06:36.196 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:06:35 smithi100 bash[21346]: audit 2024-04-07T16:06:34.750869+0000 mon.a (mon.0) 310 : audit 1 from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]': finished 2024-04-07T16:06:36.197 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:06:35 smithi100 bash[21346]: cluster 2024-04-07T16:06:34.750977+0000 mon.a (mon.0) 311 : cluster 0 osdmap e12: 2 total, 1 up, 2 in 2024-04-07T16:06:36.197 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:06:35 smithi100 bash[21346]: audit 2024-04-07T16:06:34.751481+0000 mon.a (mon.0) 312 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-07T16:06:36.197 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:06:35 smithi100 bash[21346]: audit 2024-04-07T16:06:34.761369+0000 mon.a (mon.0) 313 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-07T16:06:36.197 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:06:35 smithi100 bash[21346]: audit 2024-04-07T16:06:34.771292+0000 mon.a (mon.0) 314 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:06:36.197 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:06:35 smithi100 bash[21346]: audit 2024-04-07T16:06:34.778069+0000 mon.a (mon.0) 315 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:06:36.197 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:06:35 smithi100 bash[21346]: audit 2024-04-07T16:06:34.805272+0000 mon.a (mon.0) 316 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:06:37.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:06:36 smithi144 bash[23522]: cluster 2024-04-07T16:06:33.787203+0000 osd.1 (osd.1) 1 : cluster 0 purged_snaps scrub starts 2024-04-07T16:06:37.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:06:36 smithi144 bash[23522]: cluster 2024-04-07T16:06:33.787312+0000 osd.1 (osd.1) 2 : cluster 0 purged_snaps scrub ok 2024-04-07T16:06:37.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:06:36 smithi144 bash[23522]: audit 2024-04-07T16:06:35.763731+0000 mon.a (mon.0) 317 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-07T16:06:37.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:06:36 smithi144 bash[23522]: cluster 2024-04-07T16:06:35.783617+0000 mon.a (mon.0) 318 : cluster 1 osd.1 [v2:172.21.15.100:6810/470179506,v1:172.21.15.100:6811/470179506] boot 2024-04-07T16:06:37.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:06:36 smithi144 bash[23522]: cluster 2024-04-07T16:06:35.783723+0000 mon.a (mon.0) 319 : cluster 0 osdmap e13: 2 total, 2 up, 2 in 2024-04-07T16:06:37.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:06:36 smithi144 bash[23522]: audit 2024-04-07T16:06:35.783999+0000 mon.a (mon.0) 320 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-07T16:06:37.195 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:06:36 smithi100 bash[21346]: cluster 2024-04-07T16:06:33.787203+0000 osd.1 (osd.1) 1 : cluster 0 purged_snaps scrub starts 2024-04-07T16:06:37.195 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:06:36 smithi100 bash[21346]: cluster 2024-04-07T16:06:33.787312+0000 osd.1 (osd.1) 2 : cluster 0 purged_snaps scrub ok 2024-04-07T16:06:37.196 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:06:36 smithi100 bash[21346]: audit 2024-04-07T16:06:35.763731+0000 mon.a (mon.0) 317 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-07T16:06:37.196 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:06:36 smithi100 bash[21346]: cluster 2024-04-07T16:06:35.783617+0000 mon.a (mon.0) 318 : cluster 1 osd.1 [v2:172.21.15.100:6810/470179506,v1:172.21.15.100:6811/470179506] boot 2024-04-07T16:06:37.196 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:06:36 smithi100 bash[21346]: cluster 2024-04-07T16:06:35.783723+0000 mon.a (mon.0) 319 : cluster 0 osdmap e13: 2 total, 2 up, 2 in 2024-04-07T16:06:37.196 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:06:36 smithi100 bash[21346]: audit 2024-04-07T16:06:35.783999+0000 mon.a (mon.0) 320 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-07T16:06:37.196 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:06:36 smithi100 bash[31660]: cluster 2024-04-07T16:06:33.787203+0000 osd.1 (osd.1) 1 : cluster 0 purged_snaps scrub starts 2024-04-07T16:06:37.196 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:06:36 smithi100 bash[31660]: cluster 2024-04-07T16:06:33.787312+0000 osd.1 (osd.1) 2 : cluster 0 purged_snaps scrub ok 2024-04-07T16:06:37.196 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:06:36 smithi100 bash[31660]: audit 2024-04-07T16:06:35.763731+0000 mon.a (mon.0) 317 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-07T16:06:37.196 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:06:36 smithi100 bash[31660]: cluster 2024-04-07T16:06:35.783617+0000 mon.a (mon.0) 318 : cluster 1 osd.1 [v2:172.21.15.100:6810/470179506,v1:172.21.15.100:6811/470179506] boot 2024-04-07T16:06:37.196 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:06:36 smithi100 bash[31660]: cluster 2024-04-07T16:06:35.783723+0000 mon.a (mon.0) 319 : cluster 0 osdmap e13: 2 total, 2 up, 2 in 2024-04-07T16:06:37.196 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:06:36 smithi100 bash[31660]: audit 2024-04-07T16:06:35.783999+0000 mon.a (mon.0) 320 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-07T16:06:38.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:06:37 smithi144 bash[23522]: cluster 2024-04-07T16:06:36.195722+0000 mgr.y (mgr.14152) 133 : cluster 0 pgmap v99: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-07T16:06:38.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:06:37 smithi144 bash[23522]: cluster 2024-04-07T16:06:36.788923+0000 mon.a (mon.0) 321 : cluster 0 osdmap e14: 2 total, 2 up, 2 in 2024-04-07T16:06:38.195 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:06:37 smithi100 bash[21346]: cluster 2024-04-07T16:06:36.195722+0000 mgr.y (mgr.14152) 133 : cluster 0 pgmap v99: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-07T16:06:38.195 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:06:37 smithi100 bash[21346]: cluster 2024-04-07T16:06:36.788923+0000 mon.a (mon.0) 321 : cluster 0 osdmap e14: 2 total, 2 up, 2 in 2024-04-07T16:06:38.196 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:06:37 smithi100 bash[31660]: cluster 2024-04-07T16:06:36.195722+0000 mgr.y (mgr.14152) 133 : cluster 0 pgmap v99: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-07T16:06:38.196 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:06:37 smithi100 bash[31660]: cluster 2024-04-07T16:06:36.788923+0000 mon.a (mon.0) 321 : cluster 0 osdmap e14: 2 total, 2 up, 2 in 2024-04-07T16:06:39.625 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/mon.a/config 2024-04-07T16:06:39.945 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:06:39 smithi100 bash[21346]: cluster 2024-04-07T16:06:38.196137+0000 mgr.y (mgr.14152) 134 : cluster 0 pgmap v101: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-07T16:06:39.945 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:06:39 smithi100 bash[31660]: cluster 2024-04-07T16:06:38.196137+0000 mgr.y (mgr.14152) 134 : cluster 0 pgmap v101: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-07T16:06:40.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:06:39 smithi144 bash[23522]: cluster 2024-04-07T16:06:38.196137+0000 mgr.y (mgr.14152) 134 : cluster 0 pgmap v101: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-07T16:06:41.945 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:06:41 smithi100 bash[21346]: cluster 2024-04-07T16:06:40.196615+0000 mgr.y (mgr.14152) 135 : cluster 0 pgmap v102: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-07T16:06:41.945 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:06:41 smithi100 bash[31660]: cluster 2024-04-07T16:06:40.196615+0000 mgr.y (mgr.14152) 135 : cluster 0 pgmap v102: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-07T16:06:42.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:06:41 smithi144 bash[23522]: cluster 2024-04-07T16:06:40.196615+0000 mgr.y (mgr.14152) 135 : cluster 0 pgmap v102: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-07T16:06:43.502 INFO:teuthology.orchestra.run.smithi100.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-07T16:06:43.503 INFO:teuthology.orchestra.run.smithi100.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-07T16:06:43.503 INFO:teuthology.orchestra.run.smithi100.stderr: stderr: 10+0 records in 2024-04-07T16:06:43.503 INFO:teuthology.orchestra.run.smithi100.stderr:10+0 records out 2024-04-07T16:06:43.503 INFO:teuthology.orchestra.run.smithi100.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0174499 s, 601 MB/s 2024-04-07T16:06:43.503 INFO:teuthology.orchestra.run.smithi100.stderr:--> Zapping successful for: 2024-04-07T16:06:44.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:06:43 smithi144 bash[23522]: cluster 2024-04-07T16:06:42.197058+0000 mgr.y (mgr.14152) 136 : cluster 0 pgmap v103: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-07T16:06:44.191 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:06:43 smithi100 bash[21346]: cluster 2024-04-07T16:06:42.197058+0000 mgr.y (mgr.14152) 136 : cluster 0 pgmap v103: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-07T16:06:44.191 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:06:43 smithi100 bash[31660]: cluster 2024-04-07T16:06:42.197058+0000 mgr.y (mgr.14152) 136 : cluster 0 pgmap v103: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-07T16:06:44.197 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0702e3a8-f4f8-11ee-b649-cb9ed24678a4 -- ceph orch daemon add osd smithi100:vg_nvme/lv_2 2024-04-07T16:06:46.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:06:45 smithi144 bash[23522]: cluster 2024-04-07T16:06:44.197612+0000 mgr.y (mgr.14152) 137 : cluster 0 pgmap v104: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-07T16:06:46.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:06:45 smithi144 bash[23522]: cephadm 2024-04-07T16:06:44.695888+0000 mgr.y (mgr.14152) 138 : cephadm 1 Detected new or changed devices on smithi100 2024-04-07T16:06:46.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:06:45 smithi144 bash[23522]: audit 2024-04-07T16:06:44.705220+0000 mon.a (mon.0) 322 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:06:46.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:06:45 smithi144 bash[23522]: audit 2024-04-07T16:06:44.715300+0000 mon.a (mon.0) 323 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:06:46.111 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:06:45 smithi144 bash[23522]: audit 2024-04-07T16:06:44.717740+0000 mon.a (mon.0) 324 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-07T16:06:46.111 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:06:45 smithi144 bash[23522]: audit 2024-04-07T16:06:44.719563+0000 mon.a (mon.0) 325 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-07T16:06:46.111 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:06:45 smithi144 bash[23522]: cephadm 2024-04-07T16:06:44.720872+0000 mgr.y (mgr.14152) 139 : cephadm 1 Adjusting osd_memory_target on smithi100 to 8120M 2024-04-07T16:06:46.111 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:06:45 smithi144 bash[23522]: audit 2024-04-07T16:06:44.729242+0000 mon.a (mon.0) 326 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:06:46.111 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:06:45 smithi144 bash[23522]: audit 2024-04-07T16:06:44.731598+0000 mon.a (mon.0) 327 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:06:46.111 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:06:45 smithi144 bash[23522]: audit 2024-04-07T16:06:44.733316+0000 mon.a (mon.0) 328 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:06:46.111 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:06:45 smithi144 bash[23522]: audit 2024-04-07T16:06:44.742807+0000 mon.a (mon.0) 329 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:06:46.195 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:06:45 smithi100 bash[21346]: cluster 2024-04-07T16:06:44.197612+0000 mgr.y (mgr.14152) 137 : cluster 0 pgmap v104: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-07T16:06:46.196 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:06:45 smithi100 bash[21346]: cephadm 2024-04-07T16:06:44.695888+0000 mgr.y (mgr.14152) 138 : cephadm 1 Detected new or changed devices on smithi100 2024-04-07T16:06:46.196 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:06:45 smithi100 bash[21346]: audit 2024-04-07T16:06:44.705220+0000 mon.a (mon.0) 322 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:06:46.196 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:06:45 smithi100 bash[21346]: audit 2024-04-07T16:06:44.715300+0000 mon.a (mon.0) 323 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:06:46.196 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:06:45 smithi100 bash[21346]: audit 2024-04-07T16:06:44.717740+0000 mon.a (mon.0) 324 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-07T16:06:46.196 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:06:45 smithi100 bash[21346]: audit 2024-04-07T16:06:44.719563+0000 mon.a (mon.0) 325 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-07T16:06:46.196 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:06:45 smithi100 bash[21346]: cephadm 2024-04-07T16:06:44.720872+0000 mgr.y (mgr.14152) 139 : cephadm 1 Adjusting osd_memory_target on smithi100 to 8120M 2024-04-07T16:06:46.196 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:06:45 smithi100 bash[21346]: audit 2024-04-07T16:06:44.729242+0000 mon.a (mon.0) 326 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:06:46.196 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:06:45 smithi100 bash[21346]: audit 2024-04-07T16:06:44.731598+0000 mon.a (mon.0) 327 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:06:46.196 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:06:45 smithi100 bash[21346]: audit 2024-04-07T16:06:44.733316+0000 mon.a (mon.0) 328 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:06:46.196 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:06:45 smithi100 bash[21346]: audit 2024-04-07T16:06:44.742807+0000 mon.a (mon.0) 329 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:06:46.196 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:06:45 smithi100 bash[31660]: cluster 2024-04-07T16:06:44.197612+0000 mgr.y (mgr.14152) 137 : cluster 0 pgmap v104: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-07T16:06:46.196 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:06:45 smithi100 bash[31660]: cephadm 2024-04-07T16:06:44.695888+0000 mgr.y (mgr.14152) 138 : cephadm 1 Detected new or changed devices on smithi100 2024-04-07T16:06:46.197 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:06:45 smithi100 bash[31660]: audit 2024-04-07T16:06:44.705220+0000 mon.a (mon.0) 322 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:06:46.197 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:06:45 smithi100 bash[31660]: audit 2024-04-07T16:06:44.715300+0000 mon.a (mon.0) 323 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:06:46.197 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:06:45 smithi100 bash[31660]: audit 2024-04-07T16:06:44.717740+0000 mon.a (mon.0) 324 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-07T16:06:46.197 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:06:45 smithi100 bash[31660]: audit 2024-04-07T16:06:44.719563+0000 mon.a (mon.0) 325 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-07T16:06:46.197 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:06:45 smithi100 bash[31660]: cephadm 2024-04-07T16:06:44.720872+0000 mgr.y (mgr.14152) 139 : cephadm 1 Adjusting osd_memory_target on smithi100 to 8120M 2024-04-07T16:06:46.197 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:06:45 smithi100 bash[31660]: audit 2024-04-07T16:06:44.729242+0000 mon.a (mon.0) 326 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:06:46.197 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:06:45 smithi100 bash[31660]: audit 2024-04-07T16:06:44.731598+0000 mon.a (mon.0) 327 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:06:46.197 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:06:45 smithi100 bash[31660]: audit 2024-04-07T16:06:44.733316+0000 mon.a (mon.0) 328 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:06:46.197 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:06:45 smithi100 bash[31660]: audit 2024-04-07T16:06:44.742807+0000 mon.a (mon.0) 329 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:06:48.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:06:47 smithi144 bash[23522]: cluster 2024-04-07T16:06:46.198137+0000 mgr.y (mgr.14152) 140 : cluster 0 pgmap v105: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-07T16:06:48.195 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:06:47 smithi100 bash[21346]: cluster 2024-04-07T16:06:46.198137+0000 mgr.y (mgr.14152) 140 : cluster 0 pgmap v105: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-07T16:06:48.195 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:06:47 smithi100 bash[31660]: cluster 2024-04-07T16:06:46.198137+0000 mgr.y (mgr.14152) 140 : cluster 0 pgmap v105: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-07T16:06:48.893 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/mon.a/config 2024-04-07T16:06:49.945 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:06:49 smithi100 bash[21346]: cluster 2024-04-07T16:06:48.198642+0000 mgr.y (mgr.14152) 141 : cluster 0 pgmap v106: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-07T16:06:49.945 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:06:49 smithi100 bash[31660]: cluster 2024-04-07T16:06:48.198642+0000 mgr.y (mgr.14152) 141 : cluster 0 pgmap v106: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-07T16:06:50.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:06:49 smithi144 bash[23522]: cluster 2024-04-07T16:06:48.198642+0000 mgr.y (mgr.14152) 141 : cluster 0 pgmap v106: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-07T16:06:52.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:06:51 smithi144 bash[23522]: cluster 2024-04-07T16:06:50.199135+0000 mgr.y (mgr.14152) 142 : cluster 0 pgmap v107: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-07T16:06:52.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:06:51 smithi144 bash[23522]: audit 2024-04-07T16:06:51.508052+0000 mon.a (mon.0) 330 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-07T16:06:52.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:06:51 smithi144 bash[23522]: audit 2024-04-07T16:06:51.513679+0000 mon.a (mon.0) 331 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-07T16:06:52.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:06:51 smithi144 bash[23522]: audit 2024-04-07T16:06:51.514842+0000 mon.a (mon.0) 332 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:06:52.195 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:06:51 smithi100 bash[21346]: cluster 2024-04-07T16:06:50.199135+0000 mgr.y (mgr.14152) 142 : cluster 0 pgmap v107: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-07T16:06:52.195 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:06:51 smithi100 bash[21346]: audit 2024-04-07T16:06:51.508052+0000 mon.a (mon.0) 330 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-07T16:06:52.196 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:06:51 smithi100 bash[21346]: audit 2024-04-07T16:06:51.513679+0000 mon.a (mon.0) 331 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-07T16:06:52.196 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:06:51 smithi100 bash[21346]: audit 2024-04-07T16:06:51.514842+0000 mon.a (mon.0) 332 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:06:52.196 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:06:51 smithi100 bash[31660]: cluster 2024-04-07T16:06:50.199135+0000 mgr.y (mgr.14152) 142 : cluster 0 pgmap v107: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-07T16:06:52.196 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:06:51 smithi100 bash[31660]: audit 2024-04-07T16:06:51.508052+0000 mon.a (mon.0) 330 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-07T16:06:52.196 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:06:51 smithi100 bash[31660]: audit 2024-04-07T16:06:51.513679+0000 mon.a (mon.0) 331 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-07T16:06:52.196 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:06:51 smithi100 bash[31660]: audit 2024-04-07T16:06:51.514842+0000 mon.a (mon.0) 332 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:06:53.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:06:52 smithi144 bash[23522]: audit 2024-04-07T16:06:51.501397+0000 mgr.y (mgr.14152) 143 : audit 0 from='client.24170 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:06:53.195 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:06:52 smithi100 bash[21346]: audit 2024-04-07T16:06:51.501397+0000 mgr.y (mgr.14152) 143 : audit 0 from='client.24170 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:06:53.195 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:06:52 smithi100 bash[31660]: audit 2024-04-07T16:06:51.501397+0000 mgr.y (mgr.14152) 143 : audit 0 from='client.24170 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:06:54.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:06:53 smithi144 bash[23522]: cluster 2024-04-07T16:06:52.199614+0000 mgr.y (mgr.14152) 144 : cluster 0 pgmap v108: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-07T16:06:54.195 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:06:53 smithi100 bash[21346]: cluster 2024-04-07T16:06:52.199614+0000 mgr.y (mgr.14152) 144 : cluster 0 pgmap v108: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-07T16:06:54.195 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:06:53 smithi100 bash[31660]: cluster 2024-04-07T16:06:52.199614+0000 mgr.y (mgr.14152) 144 : cluster 0 pgmap v108: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-07T16:06:56.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:06:55 smithi144 bash[23522]: cluster 2024-04-07T16:06:54.200037+0000 mgr.y (mgr.14152) 145 : cluster 0 pgmap v109: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-07T16:06:56.195 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:06:55 smithi100 bash[21346]: cluster 2024-04-07T16:06:54.200037+0000 mgr.y (mgr.14152) 145 : cluster 0 pgmap v109: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-07T16:06:56.195 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:06:55 smithi100 bash[31660]: cluster 2024-04-07T16:06:54.200037+0000 mgr.y (mgr.14152) 145 : cluster 0 pgmap v109: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-07T16:06:58.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:06:57 smithi144 bash[23522]: cluster 2024-04-07T16:06:56.200524+0000 mgr.y (mgr.14152) 146 : cluster 0 pgmap v110: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-07T16:06:58.195 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:06:57 smithi100 bash[21346]: cluster 2024-04-07T16:06:56.200524+0000 mgr.y (mgr.14152) 146 : cluster 0 pgmap v110: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-07T16:06:58.195 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:06:57 smithi100 bash[31660]: cluster 2024-04-07T16:06:56.200524+0000 mgr.y (mgr.14152) 146 : cluster 0 pgmap v110: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-07T16:07:00.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:06:59 smithi144 bash[23522]: cluster 2024-04-07T16:06:58.201026+0000 mgr.y (mgr.14152) 147 : cluster 0 pgmap v111: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-07T16:07:00.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:06:59 smithi144 bash[23522]: audit 2024-04-07T16:06:59.415890+0000 mon.a (mon.0) 333 : audit 1 from='client.? 172.21.15.100:0/2455690181' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1650aedb-9ce9-4c14-a059-e1f089e5e63f"}]: dispatch 2024-04-07T16:07:00.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:06:59 smithi144 bash[23522]: audit 2024-04-07T16:06:59.421917+0000 mon.a (mon.0) 334 : audit 1 from='client.? 172.21.15.100:0/2455690181' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1650aedb-9ce9-4c14-a059-e1f089e5e63f"}]': finished 2024-04-07T16:07:00.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:06:59 smithi144 bash[23522]: cluster 2024-04-07T16:06:59.421994+0000 mon.a (mon.0) 335 : cluster 0 osdmap e15: 3 total, 2 up, 3 in 2024-04-07T16:07:00.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:06:59 smithi144 bash[23522]: audit 2024-04-07T16:06:59.422262+0000 mon.a (mon.0) 336 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-07T16:07:00.195 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:06:59 smithi100 bash[31660]: cluster 2024-04-07T16:06:58.201026+0000 mgr.y (mgr.14152) 147 : cluster 0 pgmap v111: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-07T16:07:00.195 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:06:59 smithi100 bash[31660]: audit 2024-04-07T16:06:59.415890+0000 mon.a (mon.0) 333 : audit 1 from='client.? 172.21.15.100:0/2455690181' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1650aedb-9ce9-4c14-a059-e1f089e5e63f"}]: dispatch 2024-04-07T16:07:00.195 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:06:59 smithi100 bash[31660]: audit 2024-04-07T16:06:59.421917+0000 mon.a (mon.0) 334 : audit 1 from='client.? 172.21.15.100:0/2455690181' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1650aedb-9ce9-4c14-a059-e1f089e5e63f"}]': finished 2024-04-07T16:07:00.195 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:06:59 smithi100 bash[31660]: cluster 2024-04-07T16:06:59.421994+0000 mon.a (mon.0) 335 : cluster 0 osdmap e15: 3 total, 2 up, 3 in 2024-04-07T16:07:00.195 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:06:59 smithi100 bash[31660]: audit 2024-04-07T16:06:59.422262+0000 mon.a (mon.0) 336 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-07T16:07:00.195 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:06:59 smithi100 bash[21346]: cluster 2024-04-07T16:06:58.201026+0000 mgr.y (mgr.14152) 147 : cluster 0 pgmap v111: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-07T16:07:00.195 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:06:59 smithi100 bash[21346]: audit 2024-04-07T16:06:59.415890+0000 mon.a (mon.0) 333 : audit 1 from='client.? 172.21.15.100:0/2455690181' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1650aedb-9ce9-4c14-a059-e1f089e5e63f"}]: dispatch 2024-04-07T16:07:00.196 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:06:59 smithi100 bash[21346]: audit 2024-04-07T16:06:59.421917+0000 mon.a (mon.0) 334 : audit 1 from='client.? 172.21.15.100:0/2455690181' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1650aedb-9ce9-4c14-a059-e1f089e5e63f"}]': finished 2024-04-07T16:07:00.196 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:06:59 smithi100 bash[21346]: cluster 2024-04-07T16:06:59.421994+0000 mon.a (mon.0) 335 : cluster 0 osdmap e15: 3 total, 2 up, 3 in 2024-04-07T16:07:00.196 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:06:59 smithi100 bash[21346]: audit 2024-04-07T16:06:59.422262+0000 mon.a (mon.0) 336 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-07T16:07:01.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:07:00 smithi144 bash[23522]: audit 2024-04-07T16:07:00.129703+0000 mon.c (mon.2) 9 : audit 0 from='client.? 172.21.15.100:0/2492536129' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-07T16:07:01.195 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:07:00 smithi100 bash[31660]: audit 2024-04-07T16:07:00.129703+0000 mon.c (mon.2) 9 : audit 0 from='client.? 172.21.15.100:0/2492536129' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-07T16:07:01.195 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:07:00 smithi100 bash[21346]: audit 2024-04-07T16:07:00.129703+0000 mon.c (mon.2) 9 : audit 0 from='client.? 172.21.15.100:0/2492536129' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-07T16:07:02.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:07:01 smithi144 bash[23522]: cluster 2024-04-07T16:07:00.201375+0000 mgr.y (mgr.14152) 148 : cluster 0 pgmap v113: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-07T16:07:02.195 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:07:01 smithi100 bash[21346]: cluster 2024-04-07T16:07:00.201375+0000 mgr.y (mgr.14152) 148 : cluster 0 pgmap v113: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-07T16:07:02.195 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:07:01 smithi100 bash[31660]: cluster 2024-04-07T16:07:00.201375+0000 mgr.y (mgr.14152) 148 : cluster 0 pgmap v113: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-07T16:07:04.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:07:03 smithi144 bash[23522]: cluster 2024-04-07T16:07:02.201759+0000 mgr.y (mgr.14152) 149 : cluster 0 pgmap v114: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-07T16:07:04.195 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:07:03 smithi100 bash[21346]: cluster 2024-04-07T16:07:02.201759+0000 mgr.y (mgr.14152) 149 : cluster 0 pgmap v114: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-07T16:07:04.195 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:07:03 smithi100 bash[31660]: cluster 2024-04-07T16:07:02.201759+0000 mgr.y (mgr.14152) 149 : cluster 0 pgmap v114: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-07T16:07:06.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:07:05 smithi144 bash[23522]: cluster 2024-04-07T16:07:04.202230+0000 mgr.y (mgr.14152) 150 : cluster 0 pgmap v115: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-07T16:07:06.195 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:07:05 smithi100 bash[21346]: cluster 2024-04-07T16:07:04.202230+0000 mgr.y (mgr.14152) 150 : cluster 0 pgmap v115: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-07T16:07:06.195 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:07:05 smithi100 bash[31660]: cluster 2024-04-07T16:07:04.202230+0000 mgr.y (mgr.14152) 150 : cluster 0 pgmap v115: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-07T16:07:08.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:07:07 smithi144 bash[23522]: cluster 2024-04-07T16:07:06.202744+0000 mgr.y (mgr.14152) 151 : cluster 0 pgmap v116: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-07T16:07:08.195 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:07:07 smithi100 bash[21346]: cluster 2024-04-07T16:07:06.202744+0000 mgr.y (mgr.14152) 151 : cluster 0 pgmap v116: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-07T16:07:08.195 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:07:07 smithi100 bash[31660]: cluster 2024-04-07T16:07:06.202744+0000 mgr.y (mgr.14152) 151 : cluster 0 pgmap v116: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-07T16:07:09.945 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:07:09 smithi100 bash[31660]: cluster 2024-04-07T16:07:08.203146+0000 mgr.y (mgr.14152) 152 : cluster 0 pgmap v117: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-07T16:07:09.945 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:07:09 smithi100 bash[21346]: cluster 2024-04-07T16:07:08.203146+0000 mgr.y (mgr.14152) 152 : cluster 0 pgmap v117: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-07T16:07:10.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:07:09 smithi144 bash[23522]: cluster 2024-04-07T16:07:08.203146+0000 mgr.y (mgr.14152) 152 : cluster 0 pgmap v117: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-07T16:07:11.945 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:07:11 smithi100 bash[21346]: cluster 2024-04-07T16:07:10.203586+0000 mgr.y (mgr.14152) 153 : cluster 0 pgmap v118: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-07T16:07:11.945 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:07:11 smithi100 bash[21346]: audit 2024-04-07T16:07:11.293763+0000 mon.a (mon.0) 337 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-07T16:07:11.945 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:07:11 smithi100 bash[21346]: audit 2024-04-07T16:07:11.295659+0000 mon.a (mon.0) 338 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:07:11.946 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:07:11 smithi100 bash[31660]: cluster 2024-04-07T16:07:10.203586+0000 mgr.y (mgr.14152) 153 : cluster 0 pgmap v118: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-07T16:07:11.946 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:07:11 smithi100 bash[31660]: audit 2024-04-07T16:07:11.293763+0000 mon.a (mon.0) 337 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-07T16:07:11.946 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:07:11 smithi100 bash[31660]: audit 2024-04-07T16:07:11.295659+0000 mon.a (mon.0) 338 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:07:12.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:07:11 smithi144 bash[23522]: cluster 2024-04-07T16:07:10.203586+0000 mgr.y (mgr.14152) 153 : cluster 0 pgmap v118: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-07T16:07:12.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:07:11 smithi144 bash[23522]: audit 2024-04-07T16:07:11.293763+0000 mon.a (mon.0) 337 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-07T16:07:12.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:07:11 smithi144 bash[23522]: audit 2024-04-07T16:07:11.295659+0000 mon.a (mon.0) 338 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:07:13.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:07:12 smithi144 bash[23522]: cephadm 2024-04-07T16:07:11.297313+0000 mgr.y (mgr.14152) 154 : cephadm 1 Deploying daemon osd.2 on smithi100 2024-04-07T16:07:13.195 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:07:12 smithi100 bash[31660]: cephadm 2024-04-07T16:07:11.297313+0000 mgr.y (mgr.14152) 154 : cephadm 1 Deploying daemon osd.2 on smithi100 2024-04-07T16:07:13.195 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:07:12 smithi100 bash[21346]: cephadm 2024-04-07T16:07:11.297313+0000 mgr.y (mgr.14152) 154 : cephadm 1 Deploying daemon osd.2 on smithi100 2024-04-07T16:07:14.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:07:13 smithi144 bash[23522]: cluster 2024-04-07T16:07:12.204010+0000 mgr.y (mgr.14152) 155 : cluster 0 pgmap v119: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-07T16:07:14.195 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:07:13 smithi100 bash[21346]: cluster 2024-04-07T16:07:12.204010+0000 mgr.y (mgr.14152) 155 : cluster 0 pgmap v119: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-07T16:07:14.195 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:07:13 smithi100 bash[31660]: cluster 2024-04-07T16:07:12.204010+0000 mgr.y (mgr.14152) 155 : cluster 0 pgmap v119: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-07T16:07:15.361 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:07:15 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:07:15.361 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:07:15 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:07:15.361 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:07:15 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:07:15.362 INFO:journalctl@ceph.osd.0.smithi100.stdout:Apr 07 16:07:15 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:07:15.362 INFO:journalctl@ceph.osd.1.smithi100.stdout:Apr 07 16:07:15 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:07:15.619 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:07:15 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:07:15.620 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:07:15 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:07:15.620 INFO:journalctl@ceph.osd.0.smithi100.stdout:Apr 07 16:07:15 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:07:15.620 INFO:journalctl@ceph.osd.1.smithi100.stdout:Apr 07 16:07:15 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:07:15.621 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:07:15 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:07:15.945 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:07:15 smithi100 bash[21346]: cluster 2024-04-07T16:07:14.204454+0000 mgr.y (mgr.14152) 156 : cluster 0 pgmap v120: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-07T16:07:15.945 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:07:15 smithi100 bash[21346]: audit 2024-04-07T16:07:15.660772+0000 mon.a (mon.0) 339 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:07:15.945 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:07:15 smithi100 bash[21346]: audit 2024-04-07T16:07:15.667999+0000 mon.a (mon.0) 340 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:07:15.945 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:07:15 smithi100 bash[21346]: audit 2024-04-07T16:07:15.705239+0000 mon.a (mon.0) 341 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:07:15.945 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:07:15 smithi100 bash[31660]: cluster 2024-04-07T16:07:14.204454+0000 mgr.y (mgr.14152) 156 : cluster 0 pgmap v120: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-07T16:07:15.946 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:07:15 smithi100 bash[31660]: audit 2024-04-07T16:07:15.660772+0000 mon.a (mon.0) 339 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:07:15.946 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:07:15 smithi100 bash[31660]: audit 2024-04-07T16:07:15.667999+0000 mon.a (mon.0) 340 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:07:15.946 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:07:15 smithi100 bash[31660]: audit 2024-04-07T16:07:15.705239+0000 mon.a (mon.0) 341 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:07:16.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:07:15 smithi144 bash[23522]: cluster 2024-04-07T16:07:14.204454+0000 mgr.y (mgr.14152) 156 : cluster 0 pgmap v120: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-07T16:07:16.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:07:15 smithi144 bash[23522]: audit 2024-04-07T16:07:15.660772+0000 mon.a (mon.0) 339 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:07:16.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:07:15 smithi144 bash[23522]: audit 2024-04-07T16:07:15.667999+0000 mon.a (mon.0) 340 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:07:16.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:07:15 smithi144 bash[23522]: audit 2024-04-07T16:07:15.705239+0000 mon.a (mon.0) 341 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:07:17.945 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:07:17 smithi100 bash[21346]: cluster 2024-04-07T16:07:16.204894+0000 mgr.y (mgr.14152) 157 : cluster 0 pgmap v121: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-07T16:07:17.945 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:07:17 smithi100 bash[31660]: cluster 2024-04-07T16:07:16.204894+0000 mgr.y (mgr.14152) 157 : cluster 0 pgmap v121: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-07T16:07:18.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:07:17 smithi144 bash[23522]: cluster 2024-04-07T16:07:16.204894+0000 mgr.y (mgr.14152) 157 : cluster 0 pgmap v121: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-07T16:07:20.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:07:19 smithi144 bash[23522]: cluster 2024-04-07T16:07:18.205448+0000 mgr.y (mgr.14152) 158 : cluster 0 pgmap v122: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-07T16:07:20.195 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:07:19 smithi100 bash[21346]: cluster 2024-04-07T16:07:18.205448+0000 mgr.y (mgr.14152) 158 : cluster 0 pgmap v122: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-07T16:07:20.195 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:07:19 smithi100 bash[31660]: cluster 2024-04-07T16:07:18.205448+0000 mgr.y (mgr.14152) 158 : cluster 0 pgmap v122: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-07T16:07:22.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:07:21 smithi144 bash[23522]: cluster 2024-04-07T16:07:20.205877+0000 mgr.y (mgr.14152) 159 : cluster 0 pgmap v123: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-07T16:07:22.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:07:21 smithi144 bash[23522]: audit 2024-04-07T16:07:20.843575+0000 mon.a (mon.0) 342 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:07:22.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:07:21 smithi144 bash[23522]: audit 2024-04-07T16:07:20.851077+0000 mon.a (mon.0) 343 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:07:22.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:07:21 smithi144 bash[23522]: audit 2024-04-07T16:07:21.327772+0000 mon.a (mon.0) 344 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:07:22.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:07:21 smithi144 bash[23522]: audit 2024-04-07T16:07:21.328661+0000 mon.a (mon.0) 345 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:07:22.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:07:21 smithi144 bash[23522]: audit 2024-04-07T16:07:21.334678+0000 mon.a (mon.0) 346 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:07:22.195 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:07:21 smithi100 bash[21346]: cluster 2024-04-07T16:07:20.205877+0000 mgr.y (mgr.14152) 159 : cluster 0 pgmap v123: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-07T16:07:22.195 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:07:21 smithi100 bash[21346]: audit 2024-04-07T16:07:20.843575+0000 mon.a (mon.0) 342 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:07:22.195 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:07:21 smithi100 bash[21346]: audit 2024-04-07T16:07:20.851077+0000 mon.a (mon.0) 343 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:07:22.195 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:07:21 smithi100 bash[21346]: audit 2024-04-07T16:07:21.327772+0000 mon.a (mon.0) 344 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:07:22.195 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:07:21 smithi100 bash[21346]: audit 2024-04-07T16:07:21.328661+0000 mon.a (mon.0) 345 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:07:22.195 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:07:21 smithi100 bash[21346]: audit 2024-04-07T16:07:21.334678+0000 mon.a (mon.0) 346 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:07:22.196 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:07:21 smithi100 bash[31660]: cluster 2024-04-07T16:07:20.205877+0000 mgr.y (mgr.14152) 159 : cluster 0 pgmap v123: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-07T16:07:22.196 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:07:21 smithi100 bash[31660]: audit 2024-04-07T16:07:20.843575+0000 mon.a (mon.0) 342 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:07:22.196 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:07:21 smithi100 bash[31660]: audit 2024-04-07T16:07:20.851077+0000 mon.a (mon.0) 343 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:07:22.196 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:07:21 smithi100 bash[31660]: audit 2024-04-07T16:07:21.327772+0000 mon.a (mon.0) 344 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:07:22.196 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:07:21 smithi100 bash[31660]: audit 2024-04-07T16:07:21.328661+0000 mon.a (mon.0) 345 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:07:22.196 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:07:21 smithi100 bash[31660]: audit 2024-04-07T16:07:21.334678+0000 mon.a (mon.0) 346 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:07:23.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:07:22 smithi144 bash[23522]: audit 2024-04-07T16:07:21.955705+0000 mon.c (mon.2) 10 : audit 1 from='osd.2 [v2:172.21.15.100:6818/2235490011,v1:172.21.15.100:6819/2235490011]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-07T16:07:23.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:07:22 smithi144 bash[23522]: audit 2024-04-07T16:07:21.956084+0000 mon.a (mon.0) 347 : audit 1 from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-07T16:07:23.195 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:07:22 smithi100 bash[21346]: audit 2024-04-07T16:07:21.955705+0000 mon.c (mon.2) 10 : audit 1 from='osd.2 [v2:172.21.15.100:6818/2235490011,v1:172.21.15.100:6819/2235490011]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-07T16:07:23.195 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:07:22 smithi100 bash[21346]: audit 2024-04-07T16:07:21.956084+0000 mon.a (mon.0) 347 : audit 1 from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-07T16:07:23.195 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:07:22 smithi100 bash[31660]: audit 2024-04-07T16:07:21.955705+0000 mon.c (mon.2) 10 : audit 1 from='osd.2 [v2:172.21.15.100:6818/2235490011,v1:172.21.15.100:6819/2235490011]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-07T16:07:23.195 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:07:22 smithi100 bash[31660]: audit 2024-04-07T16:07:21.956084+0000 mon.a (mon.0) 347 : audit 1 from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-07T16:07:24.067 INFO:teuthology.orchestra.run.smithi100.stdout:Created osd(s) 2 on host 'smithi100' 2024-04-07T16:07:24.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:07:23 smithi144 bash[23522]: cluster 2024-04-07T16:07:22.206345+0000 mgr.y (mgr.14152) 160 : cluster 0 pgmap v124: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-07T16:07:24.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:07:23 smithi144 bash[23522]: audit 2024-04-07T16:07:22.852475+0000 mon.a (mon.0) 348 : audit 1 from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-07T16:07:24.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:07:23 smithi144 bash[23522]: cluster 2024-04-07T16:07:22.852573+0000 mon.a (mon.0) 349 : cluster 0 osdmap e16: 3 total, 2 up, 3 in 2024-04-07T16:07:24.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:07:23 smithi144 bash[23522]: audit 2024-04-07T16:07:22.852839+0000 mon.a (mon.0) 350 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-07T16:07:24.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:07:23 smithi144 bash[23522]: audit 2024-04-07T16:07:22.857391+0000 mon.c (mon.2) 11 : audit 1 from='osd.2 [v2:172.21.15.100:6818/2235490011,v1:172.21.15.100:6819/2235490011]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-04-07T16:07:24.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:07:23 smithi144 bash[23522]: audit 2024-04-07T16:07:22.858123+0000 mon.a (mon.0) 351 : audit 1 from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-04-07T16:07:24.116 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:07:23 smithi100 bash[21346]: cluster 2024-04-07T16:07:22.206345+0000 mgr.y (mgr.14152) 160 : cluster 0 pgmap v124: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-07T16:07:24.116 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:07:23 smithi100 bash[21346]: audit 2024-04-07T16:07:22.852475+0000 mon.a (mon.0) 348 : audit 1 from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-07T16:07:24.116 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:07:23 smithi100 bash[21346]: cluster 2024-04-07T16:07:22.852573+0000 mon.a (mon.0) 349 : cluster 0 osdmap e16: 3 total, 2 up, 3 in 2024-04-07T16:07:24.116 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:07:23 smithi100 bash[21346]: audit 2024-04-07T16:07:22.852839+0000 mon.a (mon.0) 350 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-07T16:07:24.116 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:07:23 smithi100 bash[21346]: audit 2024-04-07T16:07:22.857391+0000 mon.c (mon.2) 11 : audit 1 from='osd.2 [v2:172.21.15.100:6818/2235490011,v1:172.21.15.100:6819/2235490011]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-04-07T16:07:24.117 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:07:23 smithi100 bash[21346]: audit 2024-04-07T16:07:22.858123+0000 mon.a (mon.0) 351 : audit 1 from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-04-07T16:07:24.117 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:07:23 smithi100 bash[31660]: cluster 2024-04-07T16:07:22.206345+0000 mgr.y (mgr.14152) 160 : cluster 0 pgmap v124: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-07T16:07:24.117 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:07:23 smithi100 bash[31660]: audit 2024-04-07T16:07:22.852475+0000 mon.a (mon.0) 348 : audit 1 from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-07T16:07:24.117 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:07:23 smithi100 bash[31660]: cluster 2024-04-07T16:07:22.852573+0000 mon.a (mon.0) 349 : cluster 0 osdmap e16: 3 total, 2 up, 3 in 2024-04-07T16:07:24.117 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:07:23 smithi100 bash[31660]: audit 2024-04-07T16:07:22.852839+0000 mon.a (mon.0) 350 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-07T16:07:24.117 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:07:23 smithi100 bash[31660]: audit 2024-04-07T16:07:22.857391+0000 mon.c (mon.2) 11 : audit 1 from='osd.2 [v2:172.21.15.100:6818/2235490011,v1:172.21.15.100:6819/2235490011]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-04-07T16:07:24.117 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:07:23 smithi100 bash[31660]: audit 2024-04-07T16:07:22.858123+0000 mon.a (mon.0) 351 : audit 1 from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-04-07T16:07:24.691 DEBUG:teuthology.orchestra.run.smithi100:osd.2> sudo journalctl -f -n 0 -u ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@osd.2.service 2024-04-07T16:07:24.694 INFO:tasks.cephadm:Deploying osd.3 on smithi100 with /dev/vg_nvme/lv_1... 2024-04-07T16:07:24.694 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0702e3a8-f4f8-11ee-b649-cb9ed24678a4 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-07T16:07:24.946 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:07:24 smithi100 bash[21346]: audit 2024-04-07T16:07:23.857521+0000 mon.a (mon.0) 352 : audit 1 from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]': finished 2024-04-07T16:07:24.946 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:07:24 smithi100 bash[21346]: cluster 2024-04-07T16:07:23.857785+0000 mon.a (mon.0) 353 : cluster 0 osdmap e17: 3 total, 2 up, 3 in 2024-04-07T16:07:24.946 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:07:24 smithi100 bash[21346]: audit 2024-04-07T16:07:23.858639+0000 mon.a (mon.0) 354 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-07T16:07:24.946 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:07:24 smithi100 bash[21346]: audit 2024-04-07T16:07:23.869873+0000 mon.a (mon.0) 355 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-07T16:07:24.946 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:07:24 smithi100 bash[21346]: audit 2024-04-07T16:07:23.906257+0000 mon.a (mon.0) 356 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:07:24.946 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:07:24 smithi100 bash[21346]: audit 2024-04-07T16:07:23.978025+0000 mon.a (mon.0) 357 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:07:24.946 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:07:24 smithi100 bash[21346]: audit 2024-04-07T16:07:24.059596+0000 mon.a (mon.0) 358 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:07:24.946 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:07:24 smithi100 bash[21346]: audit 2024-04-07T16:07:24.325725+0000 mon.a (mon.0) 359 : audit 1 from='osd.2 ' entity='osd.2' 2024-04-07T16:07:24.946 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:07:24 smithi100 bash[31660]: audit 2024-04-07T16:07:23.857521+0000 mon.a (mon.0) 352 : audit 1 from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]': finished 2024-04-07T16:07:24.946 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:07:24 smithi100 bash[31660]: cluster 2024-04-07T16:07:23.857785+0000 mon.a (mon.0) 353 : cluster 0 osdmap e17: 3 total, 2 up, 3 in 2024-04-07T16:07:24.947 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:07:24 smithi100 bash[31660]: audit 2024-04-07T16:07:23.858639+0000 mon.a (mon.0) 354 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-07T16:07:24.947 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:07:24 smithi100 bash[31660]: audit 2024-04-07T16:07:23.869873+0000 mon.a (mon.0) 355 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-07T16:07:24.947 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:07:24 smithi100 bash[31660]: audit 2024-04-07T16:07:23.906257+0000 mon.a (mon.0) 356 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:07:24.947 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:07:24 smithi100 bash[31660]: audit 2024-04-07T16:07:23.978025+0000 mon.a (mon.0) 357 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:07:24.947 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:07:24 smithi100 bash[31660]: audit 2024-04-07T16:07:24.059596+0000 mon.a (mon.0) 358 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:07:24.947 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:07:24 smithi100 bash[31660]: audit 2024-04-07T16:07:24.325725+0000 mon.a (mon.0) 359 : audit 1 from='osd.2 ' entity='osd.2' 2024-04-07T16:07:25.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:07:24 smithi144 bash[23522]: audit 2024-04-07T16:07:23.857521+0000 mon.a (mon.0) 352 : audit 1 from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]': finished 2024-04-07T16:07:25.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:07:24 smithi144 bash[23522]: cluster 2024-04-07T16:07:23.857785+0000 mon.a (mon.0) 353 : cluster 0 osdmap e17: 3 total, 2 up, 3 in 2024-04-07T16:07:25.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:07:24 smithi144 bash[23522]: audit 2024-04-07T16:07:23.858639+0000 mon.a (mon.0) 354 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-07T16:07:25.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:07:24 smithi144 bash[23522]: audit 2024-04-07T16:07:23.869873+0000 mon.a (mon.0) 355 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-07T16:07:25.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:07:24 smithi144 bash[23522]: audit 2024-04-07T16:07:23.906257+0000 mon.a (mon.0) 356 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:07:25.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:07:24 smithi144 bash[23522]: audit 2024-04-07T16:07:23.978025+0000 mon.a (mon.0) 357 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:07:25.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:07:24 smithi144 bash[23522]: audit 2024-04-07T16:07:24.059596+0000 mon.a (mon.0) 358 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:07:25.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:07:24 smithi144 bash[23522]: audit 2024-04-07T16:07:24.325725+0000 mon.a (mon.0) 359 : audit 1 from='osd.2 ' entity='osd.2' 2024-04-07T16:07:26.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:07:25 smithi144 bash[23522]: cluster 2024-04-07T16:07:22.983005+0000 osd.2 (osd.2) 1 : cluster 0 purged_snaps scrub starts 2024-04-07T16:07:26.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:07:25 smithi144 bash[23522]: cluster 2024-04-07T16:07:22.983150+0000 osd.2 (osd.2) 2 : cluster 0 purged_snaps scrub ok 2024-04-07T16:07:26.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:07:25 smithi144 bash[23522]: cluster 2024-04-07T16:07:24.206626+0000 mgr.y (mgr.14152) 161 : cluster 0 pgmap v127: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-07T16:07:26.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:07:25 smithi144 bash[23522]: audit 2024-04-07T16:07:24.870318+0000 mon.a (mon.0) 360 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-07T16:07:26.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:07:25 smithi144 bash[23522]: cluster 2024-04-07T16:07:25.330974+0000 mon.a (mon.0) 361 : cluster 1 osd.2 [v2:172.21.15.100:6818/2235490011,v1:172.21.15.100:6819/2235490011] boot 2024-04-07T16:07:26.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:07:25 smithi144 bash[23522]: cluster 2024-04-07T16:07:25.331054+0000 mon.a (mon.0) 362 : cluster 0 osdmap e18: 3 total, 3 up, 3 in 2024-04-07T16:07:26.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:07:25 smithi144 bash[23522]: audit 2024-04-07T16:07:25.331268+0000 mon.a (mon.0) 363 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-07T16:07:26.195 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:07:25 smithi100 bash[31660]: cluster 2024-04-07T16:07:22.983005+0000 osd.2 (osd.2) 1 : cluster 0 purged_snaps scrub starts 2024-04-07T16:07:26.195 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:07:25 smithi100 bash[31660]: cluster 2024-04-07T16:07:22.983150+0000 osd.2 (osd.2) 2 : cluster 0 purged_snaps scrub ok 2024-04-07T16:07:26.195 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:07:25 smithi100 bash[31660]: cluster 2024-04-07T16:07:24.206626+0000 mgr.y (mgr.14152) 161 : cluster 0 pgmap v127: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-07T16:07:26.195 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:07:25 smithi100 bash[31660]: audit 2024-04-07T16:07:24.870318+0000 mon.a (mon.0) 360 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-07T16:07:26.195 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:07:25 smithi100 bash[31660]: cluster 2024-04-07T16:07:25.330974+0000 mon.a (mon.0) 361 : cluster 1 osd.2 [v2:172.21.15.100:6818/2235490011,v1:172.21.15.100:6819/2235490011] boot 2024-04-07T16:07:26.196 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:07:25 smithi100 bash[31660]: cluster 2024-04-07T16:07:25.331054+0000 mon.a (mon.0) 362 : cluster 0 osdmap e18: 3 total, 3 up, 3 in 2024-04-07T16:07:26.196 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:07:25 smithi100 bash[31660]: audit 2024-04-07T16:07:25.331268+0000 mon.a (mon.0) 363 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-07T16:07:26.196 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:07:25 smithi100 bash[21346]: cluster 2024-04-07T16:07:22.983005+0000 osd.2 (osd.2) 1 : cluster 0 purged_snaps scrub starts 2024-04-07T16:07:26.196 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:07:25 smithi100 bash[21346]: cluster 2024-04-07T16:07:22.983150+0000 osd.2 (osd.2) 2 : cluster 0 purged_snaps scrub ok 2024-04-07T16:07:26.196 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:07:25 smithi100 bash[21346]: cluster 2024-04-07T16:07:24.206626+0000 mgr.y (mgr.14152) 161 : cluster 0 pgmap v127: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-07T16:07:26.196 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:07:25 smithi100 bash[21346]: audit 2024-04-07T16:07:24.870318+0000 mon.a (mon.0) 360 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-07T16:07:26.196 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:07:25 smithi100 bash[21346]: cluster 2024-04-07T16:07:25.330974+0000 mon.a (mon.0) 361 : cluster 1 osd.2 [v2:172.21.15.100:6818/2235490011,v1:172.21.15.100:6819/2235490011] boot 2024-04-07T16:07:26.196 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:07:25 smithi100 bash[21346]: cluster 2024-04-07T16:07:25.331054+0000 mon.a (mon.0) 362 : cluster 0 osdmap e18: 3 total, 3 up, 3 in 2024-04-07T16:07:26.196 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:07:25 smithi100 bash[21346]: audit 2024-04-07T16:07:25.331268+0000 mon.a (mon.0) 363 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-07T16:07:27.195 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:07:26 smithi100 bash[21346]: audit 2024-04-07T16:07:26.348718+0000 mon.a (mon.0) 364 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:07:27.195 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:07:26 smithi100 bash[31660]: audit 2024-04-07T16:07:26.348718+0000 mon.a (mon.0) 364 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:07:27.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:07:26 smithi144 bash[23522]: audit 2024-04-07T16:07:26.348718+0000 mon.a (mon.0) 364 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:07:28.195 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:07:27 smithi100 bash[21346]: cluster 2024-04-07T16:07:26.207070+0000 mgr.y (mgr.14152) 162 : cluster 0 pgmap v129: 0 pgs: ; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-07T16:07:28.195 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:07:27 smithi100 bash[21346]: audit 2024-04-07T16:07:26.882194+0000 mon.a (mon.0) 365 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-07T16:07:28.195 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:07:27 smithi100 bash[21346]: cluster 2024-04-07T16:07:26.882366+0000 mon.a (mon.0) 366 : cluster 0 osdmap e19: 3 total, 3 up, 3 in 2024-04-07T16:07:28.196 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:07:27 smithi100 bash[21346]: audit 2024-04-07T16:07:26.884328+0000 mon.a (mon.0) 367 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:07:28.196 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:07:27 smithi100 bash[31660]: cluster 2024-04-07T16:07:26.207070+0000 mgr.y (mgr.14152) 162 : cluster 0 pgmap v129: 0 pgs: ; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-07T16:07:28.196 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:07:27 smithi100 bash[31660]: audit 2024-04-07T16:07:26.882194+0000 mon.a (mon.0) 365 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-07T16:07:28.196 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:07:27 smithi100 bash[31660]: cluster 2024-04-07T16:07:26.882366+0000 mon.a (mon.0) 366 : cluster 0 osdmap e19: 3 total, 3 up, 3 in 2024-04-07T16:07:28.196 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:07:27 smithi100 bash[31660]: audit 2024-04-07T16:07:26.884328+0000 mon.a (mon.0) 367 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:07:28.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:07:27 smithi144 bash[23522]: cluster 2024-04-07T16:07:26.207070+0000 mgr.y (mgr.14152) 162 : cluster 0 pgmap v129: 0 pgs: ; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-07T16:07:28.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:07:27 smithi144 bash[23522]: audit 2024-04-07T16:07:26.882194+0000 mon.a (mon.0) 365 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-07T16:07:28.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:07:27 smithi144 bash[23522]: cluster 2024-04-07T16:07:26.882366+0000 mon.a (mon.0) 366 : cluster 0 osdmap e19: 3 total, 3 up, 3 in 2024-04-07T16:07:28.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:07:27 smithi144 bash[23522]: audit 2024-04-07T16:07:26.884328+0000 mon.a (mon.0) 367 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:07:28.904 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/mon.a/config 2024-04-07T16:07:29.195 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:07:28 smithi100 bash[21346]: audit 2024-04-07T16:07:27.882106+0000 mon.a (mon.0) 368 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-07T16:07:29.195 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:07:28 smithi100 bash[21346]: cluster 2024-04-07T16:07:27.882382+0000 mon.a (mon.0) 369 : cluster 0 osdmap e20: 3 total, 3 up, 3 in 2024-04-07T16:07:29.195 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:07:28 smithi100 bash[31660]: audit 2024-04-07T16:07:27.882106+0000 mon.a (mon.0) 368 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-07T16:07:29.195 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:07:28 smithi100 bash[31660]: cluster 2024-04-07T16:07:27.882382+0000 mon.a (mon.0) 369 : cluster 0 osdmap e20: 3 total, 3 up, 3 in 2024-04-07T16:07:29.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:07:28 smithi144 bash[23522]: audit 2024-04-07T16:07:27.882106+0000 mon.a (mon.0) 368 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-07T16:07:29.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:07:28 smithi144 bash[23522]: cluster 2024-04-07T16:07:27.882382+0000 mon.a (mon.0) 369 : cluster 0 osdmap e20: 3 total, 3 up, 3 in 2024-04-07T16:07:30.195 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:07:29 smithi100 bash[21346]: cluster 2024-04-07T16:07:28.207630+0000 mgr.y (mgr.14152) 163 : cluster 0 pgmap v132: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-07T16:07:30.195 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:07:29 smithi100 bash[21346]: cluster 2024-04-07T16:07:28.897407+0000 mon.a (mon.0) 370 : cluster 0 osdmap e21: 3 total, 3 up, 3 in 2024-04-07T16:07:30.195 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:07:29 smithi100 bash[21346]: audit 2024-04-07T16:07:29.274634+0000 mon.a (mon.0) 371 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:07:30.195 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:07:29 smithi100 bash[31660]: cluster 2024-04-07T16:07:28.207630+0000 mgr.y (mgr.14152) 163 : cluster 0 pgmap v132: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-07T16:07:30.195 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:07:29 smithi100 bash[31660]: cluster 2024-04-07T16:07:28.897407+0000 mon.a (mon.0) 370 : cluster 0 osdmap e21: 3 total, 3 up, 3 in 2024-04-07T16:07:30.196 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:07:29 smithi100 bash[31660]: audit 2024-04-07T16:07:29.274634+0000 mon.a (mon.0) 371 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:07:30.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:07:29 smithi144 bash[23522]: cluster 2024-04-07T16:07:28.207630+0000 mgr.y (mgr.14152) 163 : cluster 0 pgmap v132: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-07T16:07:30.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:07:29 smithi144 bash[23522]: cluster 2024-04-07T16:07:28.897407+0000 mon.a (mon.0) 370 : cluster 0 osdmap e21: 3 total, 3 up, 3 in 2024-04-07T16:07:30.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:07:29 smithi144 bash[23522]: audit 2024-04-07T16:07:29.274634+0000 mon.a (mon.0) 371 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:07:32.176 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:07:31 smithi100 bash[31660]: cluster 2024-04-07T16:07:30.913895+0000 mon.a (mon.0) 372 : cluster 0 mgrmap e15: y(active, since 4m), standbys: x 2024-04-07T16:07:32.176 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:07:31 smithi100 bash[31660]: audit 2024-04-07T16:07:31.049846+0000 mon.a (mon.0) 373 : audit 1 from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-07T16:07:32.177 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:07:31 smithi100 bash[21346]: cluster 2024-04-07T16:07:30.913895+0000 mon.a (mon.0) 372 : cluster 0 mgrmap e15: y(active, since 4m), standbys: x 2024-04-07T16:07:32.177 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:07:31 smithi100 bash[21346]: audit 2024-04-07T16:07:31.049846+0000 mon.a (mon.0) 373 : audit 1 from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-07T16:07:32.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:07:31 smithi144 bash[23522]: cluster 2024-04-07T16:07:30.913895+0000 mon.a (mon.0) 372 : cluster 0 mgrmap e15: y(active, since 4m), standbys: x 2024-04-07T16:07:32.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:07:31 smithi144 bash[23522]: audit 2024-04-07T16:07:31.049846+0000 mon.a (mon.0) 373 : audit 1 from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-07T16:07:32.893 INFO:teuthology.orchestra.run.smithi100.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-07T16:07:32.893 INFO:teuthology.orchestra.run.smithi100.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-07T16:07:32.893 INFO:teuthology.orchestra.run.smithi100.stderr: stderr: 10+0 records in 2024-04-07T16:07:32.893 INFO:teuthology.orchestra.run.smithi100.stderr:10+0 records out 2024-04-07T16:07:32.893 INFO:teuthology.orchestra.run.smithi100.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0148623 s, 706 MB/s 2024-04-07T16:07:32.894 INFO:teuthology.orchestra.run.smithi100.stderr:--> Zapping successful for: 2024-04-07T16:07:33.725 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0702e3a8-f4f8-11ee-b649-cb9ed24678a4 -- ceph orch daemon add osd smithi100:vg_nvme/lv_1 2024-04-07T16:07:33.945 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:07:33 smithi100 bash[21346]: cluster 2024-04-07T16:07:30.207962+0000 mgr.y (mgr.14152) 164 : cluster 0 pgmap v134: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-07T16:07:33.946 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:07:33 smithi100 bash[21346]: audit 2024-04-07T16:07:31.917400+0000 mon.a (mon.0) 374 : audit 1 from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-07T16:07:33.946 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:07:33 smithi100 bash[21346]: audit 2024-04-07T16:07:31.918341+0000 mon.a (mon.0) 375 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:07:33.946 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:07:33 smithi100 bash[21346]: audit 2024-04-07T16:07:31.918707+0000 mon.a (mon.0) 376 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:07:33.946 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:07:33 smithi100 bash[21346]: audit 2024-04-07T16:07:31.918987+0000 mon.a (mon.0) 377 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:07:33.946 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:07:33 smithi100 bash[21346]: audit 2024-04-07T16:07:31.924507+0000 mon.b (mon.1) 7 : audit 1 from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-07T16:07:33.946 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:07:33 smithi100 bash[21346]: audit 2024-04-07T16:07:31.925328+0000 mon.a (mon.0) 378 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:07:33.946 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:07:33 smithi100 bash[21346]: audit 2024-04-07T16:07:31.925517+0000 mon.a (mon.0) 379 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:07:33.946 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:07:33 smithi100 bash[21346]: audit 2024-04-07T16:07:31.925736+0000 mon.a (mon.0) 380 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:07:33.946 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:07:33 smithi100 bash[21346]: audit 2024-04-07T16:07:32.837192+0000 mon.b (mon.1) 8 : audit 1 from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-07T16:07:33.947 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:07:33 smithi100 bash[21346]: audit 2024-04-07T16:07:32.862704+0000 mon.a (mon.0) 381 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:07:33.947 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:07:33 smithi100 bash[21346]: audit 2024-04-07T16:07:32.864372+0000 mon.a (mon.0) 382 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:07:33.947 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:07:33 smithi100 bash[21346]: audit 2024-04-07T16:07:32.864576+0000 mon.a (mon.0) 383 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:07:33.947 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:07:33 smithi100 bash[21346]: audit 2024-04-07T16:07:32.864750+0000 mon.a (mon.0) 384 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:07:33.947 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:07:33 smithi100 bash[31660]: cluster 2024-04-07T16:07:30.207962+0000 mgr.y (mgr.14152) 164 : cluster 0 pgmap v134: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-07T16:07:33.947 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:07:33 smithi100 bash[31660]: audit 2024-04-07T16:07:31.917400+0000 mon.a (mon.0) 374 : audit 1 from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-07T16:07:33.947 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:07:33 smithi100 bash[31660]: audit 2024-04-07T16:07:31.918341+0000 mon.a (mon.0) 375 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:07:33.947 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:07:33 smithi100 bash[31660]: audit 2024-04-07T16:07:31.918707+0000 mon.a (mon.0) 376 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:07:33.947 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:07:33 smithi100 bash[31660]: audit 2024-04-07T16:07:31.918987+0000 mon.a (mon.0) 377 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:07:33.947 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:07:33 smithi100 bash[31660]: audit 2024-04-07T16:07:31.924507+0000 mon.b (mon.1) 7 : audit 1 from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-07T16:07:33.947 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:07:33 smithi100 bash[31660]: audit 2024-04-07T16:07:31.925328+0000 mon.a (mon.0) 378 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:07:33.947 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:07:33 smithi100 bash[31660]: audit 2024-04-07T16:07:31.925517+0000 mon.a (mon.0) 379 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:07:33.947 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:07:33 smithi100 bash[31660]: audit 2024-04-07T16:07:31.925736+0000 mon.a (mon.0) 380 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:07:33.948 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:07:33 smithi100 bash[31660]: audit 2024-04-07T16:07:32.837192+0000 mon.b (mon.1) 8 : audit 1 from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-07T16:07:33.948 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:07:33 smithi100 bash[31660]: audit 2024-04-07T16:07:32.862704+0000 mon.a (mon.0) 381 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:07:33.948 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:07:33 smithi100 bash[31660]: audit 2024-04-07T16:07:32.864372+0000 mon.a (mon.0) 382 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:07:33.948 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:07:33 smithi100 bash[31660]: audit 2024-04-07T16:07:32.864576+0000 mon.a (mon.0) 383 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:07:33.948 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:07:33 smithi100 bash[31660]: audit 2024-04-07T16:07:32.864750+0000 mon.a (mon.0) 384 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:07:34.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:07:33 smithi144 bash[23522]: cluster 2024-04-07T16:07:30.207962+0000 mgr.y (mgr.14152) 164 : cluster 0 pgmap v134: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-07T16:07:34.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:07:33 smithi144 bash[23522]: audit 2024-04-07T16:07:31.917400+0000 mon.a (mon.0) 374 : audit 1 from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-07T16:07:34.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:07:33 smithi144 bash[23522]: audit 2024-04-07T16:07:31.918341+0000 mon.a (mon.0) 375 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:07:34.111 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:07:33 smithi144 bash[23522]: audit 2024-04-07T16:07:31.918707+0000 mon.a (mon.0) 376 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:07:34.111 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:07:33 smithi144 bash[23522]: audit 2024-04-07T16:07:31.918987+0000 mon.a (mon.0) 377 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:07:34.111 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:07:33 smithi144 bash[23522]: audit 2024-04-07T16:07:31.924507+0000 mon.b (mon.1) 7 : audit 1 from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-07T16:07:34.111 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:07:33 smithi144 bash[23522]: audit 2024-04-07T16:07:31.925328+0000 mon.a (mon.0) 378 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:07:34.111 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:07:33 smithi144 bash[23522]: audit 2024-04-07T16:07:31.925517+0000 mon.a (mon.0) 379 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:07:34.111 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:07:33 smithi144 bash[23522]: audit 2024-04-07T16:07:31.925736+0000 mon.a (mon.0) 380 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:07:34.111 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:07:33 smithi144 bash[23522]: audit 2024-04-07T16:07:32.837192+0000 mon.b (mon.1) 8 : audit 1 from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-07T16:07:34.111 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:07:33 smithi144 bash[23522]: audit 2024-04-07T16:07:32.862704+0000 mon.a (mon.0) 381 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:07:34.111 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:07:33 smithi144 bash[23522]: audit 2024-04-07T16:07:32.864372+0000 mon.a (mon.0) 382 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:07:34.111 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:07:33 smithi144 bash[23522]: audit 2024-04-07T16:07:32.864576+0000 mon.a (mon.0) 383 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:07:34.111 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:07:33 smithi144 bash[23522]: audit 2024-04-07T16:07:32.864750+0000 mon.a (mon.0) 384 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:07:35.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:07:34 smithi144 bash[23522]: cluster 2024-04-07T16:07:32.208318+0000 mgr.y (mgr.14152) 165 : cluster 0 pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-07T16:07:35.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:07:34 smithi144 bash[23522]: audit 2024-04-07T16:07:32.864097+0000 mon.c (mon.2) 12 : audit 1 from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-07T16:07:35.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:07:34 smithi144 bash[23522]: audit 2024-04-07T16:07:33.759521+0000 mon.c (mon.2) 13 : audit 1 from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-07T16:07:35.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:07:34 smithi144 bash[23522]: cephadm 2024-04-07T16:07:34.067340+0000 mgr.y (mgr.14152) 166 : cephadm 1 Detected new or changed devices on smithi100 2024-04-07T16:07:35.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:07:34 smithi144 bash[23522]: audit 2024-04-07T16:07:34.076393+0000 mon.a (mon.0) 385 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:07:35.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:07:34 smithi144 bash[23522]: audit 2024-04-07T16:07:34.087697+0000 mon.a (mon.0) 386 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:07:35.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:07:34 smithi144 bash[23522]: audit 2024-04-07T16:07:34.090139+0000 mon.a (mon.0) 387 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-07T16:07:35.111 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:07:34 smithi144 bash[23522]: audit 2024-04-07T16:07:34.091842+0000 mon.a (mon.0) 388 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-07T16:07:35.111 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:07:34 smithi144 bash[23522]: audit 2024-04-07T16:07:34.093440+0000 mon.a (mon.0) 389 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-07T16:07:35.111 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:07:34 smithi144 bash[23522]: cephadm 2024-04-07T16:07:34.094718+0000 mgr.y (mgr.14152) 167 : cephadm 1 Adjusting osd_memory_target on smithi100 to 5413M 2024-04-07T16:07:35.111 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:07:34 smithi144 bash[23522]: audit 2024-04-07T16:07:34.104287+0000 mon.a (mon.0) 390 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:07:35.111 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:07:34 smithi144 bash[23522]: audit 2024-04-07T16:07:34.106213+0000 mon.a (mon.0) 391 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:07:35.111 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:07:34 smithi144 bash[23522]: audit 2024-04-07T16:07:34.108293+0000 mon.a (mon.0) 392 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:07:35.111 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:07:34 smithi144 bash[23522]: audit 2024-04-07T16:07:34.117701+0000 mon.a (mon.0) 393 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:07:35.195 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:07:34 smithi100 bash[21346]: cluster 2024-04-07T16:07:32.208318+0000 mgr.y (mgr.14152) 165 : cluster 0 pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-07T16:07:35.196 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:07:34 smithi100 bash[21346]: audit 2024-04-07T16:07:32.864097+0000 mon.c (mon.2) 12 : audit 1 from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-07T16:07:35.196 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:07:34 smithi100 bash[21346]: audit 2024-04-07T16:07:33.759521+0000 mon.c (mon.2) 13 : audit 1 from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-07T16:07:35.196 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:07:34 smithi100 bash[21346]: cephadm 2024-04-07T16:07:34.067340+0000 mgr.y (mgr.14152) 166 : cephadm 1 Detected new or changed devices on smithi100 2024-04-07T16:07:35.196 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:07:34 smithi100 bash[21346]: audit 2024-04-07T16:07:34.076393+0000 mon.a (mon.0) 385 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:07:35.196 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:07:34 smithi100 bash[21346]: audit 2024-04-07T16:07:34.087697+0000 mon.a (mon.0) 386 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:07:35.196 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:07:34 smithi100 bash[21346]: audit 2024-04-07T16:07:34.090139+0000 mon.a (mon.0) 387 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-07T16:07:35.196 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:07:34 smithi100 bash[21346]: audit 2024-04-07T16:07:34.091842+0000 mon.a (mon.0) 388 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-07T16:07:35.196 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:07:34 smithi100 bash[21346]: audit 2024-04-07T16:07:34.093440+0000 mon.a (mon.0) 389 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-07T16:07:35.196 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:07:34 smithi100 bash[21346]: cephadm 2024-04-07T16:07:34.094718+0000 mgr.y (mgr.14152) 167 : cephadm 1 Adjusting osd_memory_target on smithi100 to 5413M 2024-04-07T16:07:35.196 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:07:34 smithi100 bash[21346]: audit 2024-04-07T16:07:34.104287+0000 mon.a (mon.0) 390 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:07:35.196 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:07:34 smithi100 bash[21346]: audit 2024-04-07T16:07:34.106213+0000 mon.a (mon.0) 391 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:07:35.196 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:07:34 smithi100 bash[21346]: audit 2024-04-07T16:07:34.108293+0000 mon.a (mon.0) 392 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:07:35.196 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:07:34 smithi100 bash[21346]: audit 2024-04-07T16:07:34.117701+0000 mon.a (mon.0) 393 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:07:35.197 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:07:34 smithi100 bash[31660]: cluster 2024-04-07T16:07:32.208318+0000 mgr.y (mgr.14152) 165 : cluster 0 pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-07T16:07:35.197 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:07:34 smithi100 bash[31660]: audit 2024-04-07T16:07:32.864097+0000 mon.c (mon.2) 12 : audit 1 from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-07T16:07:35.197 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:07:34 smithi100 bash[31660]: audit 2024-04-07T16:07:33.759521+0000 mon.c (mon.2) 13 : audit 1 from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-07T16:07:35.197 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:07:34 smithi100 bash[31660]: cephadm 2024-04-07T16:07:34.067340+0000 mgr.y (mgr.14152) 166 : cephadm 1 Detected new or changed devices on smithi100 2024-04-07T16:07:35.197 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:07:34 smithi100 bash[31660]: audit 2024-04-07T16:07:34.076393+0000 mon.a (mon.0) 385 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:07:35.197 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:07:34 smithi100 bash[31660]: audit 2024-04-07T16:07:34.087697+0000 mon.a (mon.0) 386 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:07:35.197 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:07:34 smithi100 bash[31660]: audit 2024-04-07T16:07:34.090139+0000 mon.a (mon.0) 387 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-07T16:07:35.197 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:07:34 smithi100 bash[31660]: audit 2024-04-07T16:07:34.091842+0000 mon.a (mon.0) 388 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-07T16:07:35.197 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:07:34 smithi100 bash[31660]: audit 2024-04-07T16:07:34.093440+0000 mon.a (mon.0) 389 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-07T16:07:35.197 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:07:34 smithi100 bash[31660]: cephadm 2024-04-07T16:07:34.094718+0000 mgr.y (mgr.14152) 167 : cephadm 1 Adjusting osd_memory_target on smithi100 to 5413M 2024-04-07T16:07:35.197 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:07:34 smithi100 bash[31660]: audit 2024-04-07T16:07:34.104287+0000 mon.a (mon.0) 390 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:07:35.197 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:07:34 smithi100 bash[31660]: audit 2024-04-07T16:07:34.106213+0000 mon.a (mon.0) 391 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:07:35.198 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:07:34 smithi100 bash[31660]: audit 2024-04-07T16:07:34.108293+0000 mon.a (mon.0) 392 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:07:35.198 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:07:34 smithi100 bash[31660]: audit 2024-04-07T16:07:34.117701+0000 mon.a (mon.0) 393 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:07:36.445 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:07:36 smithi100 bash[21346]: cluster 2024-04-07T16:07:34.208755+0000 mgr.y (mgr.14152) 168 : cluster 0 pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-04-07T16:07:36.445 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:07:36 smithi100 bash[31660]: cluster 2024-04-07T16:07:34.208755+0000 mgr.y (mgr.14152) 168 : cluster 0 pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-04-07T16:07:36.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:07:36 smithi144 bash[23522]: cluster 2024-04-07T16:07:34.208755+0000 mgr.y (mgr.14152) 168 : cluster 0 pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-04-07T16:07:38.420 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/mon.a/config 2024-04-07T16:07:38.442 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:07:38 smithi100 bash[21346]: cluster 2024-04-07T16:07:36.209282+0000 mgr.y (mgr.14152) 169 : cluster 0 pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2024-04-07T16:07:38.442 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:07:38 smithi100 bash[31660]: cluster 2024-04-07T16:07:36.209282+0000 mgr.y (mgr.14152) 169 : cluster 0 pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2024-04-07T16:07:38.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:07:38 smithi144 bash[23522]: cluster 2024-04-07T16:07:36.209282+0000 mgr.y (mgr.14152) 169 : cluster 0 pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2024-04-07T16:07:40.445 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:07:40 smithi100 bash[21346]: cluster 2024-04-07T16:07:38.209923+0000 mgr.y (mgr.14152) 170 : cluster 0 pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 131 KiB/s wr, 9 op/s 2024-04-07T16:07:40.445 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:07:40 smithi100 bash[31660]: cluster 2024-04-07T16:07:38.209923+0000 mgr.y (mgr.14152) 170 : cluster 0 pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 131 KiB/s wr, 9 op/s 2024-04-07T16:07:40.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:07:40 smithi144 bash[23522]: cluster 2024-04-07T16:07:38.209923+0000 mgr.y (mgr.14152) 170 : cluster 0 pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 131 KiB/s wr, 9 op/s 2024-04-07T16:07:41.445 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:07:41 smithi100 bash[21346]: audit 2024-04-07T16:07:41.043207+0000 mon.a (mon.0) 394 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-07T16:07:41.445 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:07:41 smithi100 bash[21346]: audit 2024-04-07T16:07:41.046060+0000 mon.a (mon.0) 395 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-07T16:07:41.445 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:07:41 smithi100 bash[21346]: audit 2024-04-07T16:07:41.046916+0000 mon.a (mon.0) 396 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:07:41.445 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:07:41 smithi100 bash[31660]: audit 2024-04-07T16:07:41.043207+0000 mon.a (mon.0) 394 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-07T16:07:41.446 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:07:41 smithi100 bash[31660]: audit 2024-04-07T16:07:41.046060+0000 mon.a (mon.0) 395 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-07T16:07:41.446 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:07:41 smithi100 bash[31660]: audit 2024-04-07T16:07:41.046916+0000 mon.a (mon.0) 396 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:07:41.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:07:41 smithi144 bash[23522]: audit 2024-04-07T16:07:41.043207+0000 mon.a (mon.0) 394 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-07T16:07:41.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:07:41 smithi144 bash[23522]: audit 2024-04-07T16:07:41.046060+0000 mon.a (mon.0) 395 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-07T16:07:41.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:07:41 smithi144 bash[23522]: audit 2024-04-07T16:07:41.046916+0000 mon.a (mon.0) 396 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:07:42.445 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:07:42 smithi100 bash[21346]: cluster 2024-04-07T16:07:40.210469+0000 mgr.y (mgr.14152) 171 : cluster 0 pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 98 KiB/s wr, 6 op/s 2024-04-07T16:07:42.445 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:07:42 smithi100 bash[21346]: audit 2024-04-07T16:07:41.039564+0000 mgr.y (mgr.14152) 172 : audit 0 from='client.24206 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:07:42.445 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:07:42 smithi100 bash[31660]: cluster 2024-04-07T16:07:40.210469+0000 mgr.y (mgr.14152) 171 : cluster 0 pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 98 KiB/s wr, 6 op/s 2024-04-07T16:07:42.445 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:07:42 smithi100 bash[31660]: audit 2024-04-07T16:07:41.039564+0000 mgr.y (mgr.14152) 172 : audit 0 from='client.24206 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:07:42.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:07:42 smithi144 bash[23522]: cluster 2024-04-07T16:07:40.210469+0000 mgr.y (mgr.14152) 171 : cluster 0 pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 98 KiB/s wr, 6 op/s 2024-04-07T16:07:42.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:07:42 smithi144 bash[23522]: audit 2024-04-07T16:07:41.039564+0000 mgr.y (mgr.14152) 172 : audit 0 from='client.24206 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:07:44.445 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:07:44 smithi100 bash[21346]: cluster 2024-04-07T16:07:42.211071+0000 mgr.y (mgr.14152) 173 : cluster 0 pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-04-07T16:07:44.445 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:07:44 smithi100 bash[31660]: cluster 2024-04-07T16:07:42.211071+0000 mgr.y (mgr.14152) 173 : cluster 0 pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-04-07T16:07:44.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:07:44 smithi144 bash[23522]: cluster 2024-04-07T16:07:42.211071+0000 mgr.y (mgr.14152) 173 : cluster 0 pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-04-07T16:07:46.195 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:07:46 smithi100 bash[21346]: cluster 2024-04-07T16:07:44.211656+0000 mgr.y (mgr.14152) 174 : cluster 0 pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-04-07T16:07:46.195 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:07:46 smithi100 bash[31660]: cluster 2024-04-07T16:07:44.211656+0000 mgr.y (mgr.14152) 174 : cluster 0 pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-04-07T16:07:46.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:07:46 smithi144 bash[23522]: cluster 2024-04-07T16:07:44.211656+0000 mgr.y (mgr.14152) 174 : cluster 0 pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-04-07T16:07:48.445 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:07:48 smithi100 bash[21346]: cluster 2024-04-07T16:07:46.211999+0000 mgr.y (mgr.14152) 175 : cluster 0 pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-04-07T16:07:48.445 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:07:48 smithi100 bash[31660]: cluster 2024-04-07T16:07:46.211999+0000 mgr.y (mgr.14152) 175 : cluster 0 pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-04-07T16:07:48.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:07:48 smithi144 bash[23522]: cluster 2024-04-07T16:07:46.211999+0000 mgr.y (mgr.14152) 175 : cluster 0 pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-04-07T16:07:49.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:07:49 smithi144 bash[23522]: cluster 2024-04-07T16:07:48.212556+0000 mgr.y (mgr.14152) 176 : cluster 0 pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-04-07T16:07:49.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:07:49 smithi144 bash[23522]: audit 2024-04-07T16:07:49.185286+0000 mon.a (mon.0) 397 : audit 1 from='client.? 172.21.15.100:0/2754842238' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "17591c6b-862b-43b2-858b-02f4c3199247"}]: dispatch 2024-04-07T16:07:49.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:07:49 smithi144 bash[23522]: audit 2024-04-07T16:07:49.196517+0000 mon.a (mon.0) 398 : audit 1 from='client.? 172.21.15.100:0/2754842238' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "17591c6b-862b-43b2-858b-02f4c3199247"}]': finished 2024-04-07T16:07:49.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:07:49 smithi144 bash[23522]: cluster 2024-04-07T16:07:49.196658+0000 mon.a (mon.0) 399 : cluster 0 osdmap e22: 4 total, 3 up, 4 in 2024-04-07T16:07:49.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:07:49 smithi144 bash[23522]: audit 2024-04-07T16:07:49.196853+0000 mon.a (mon.0) 400 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-07T16:07:49.695 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:07:49 smithi100 bash[31660]: cluster 2024-04-07T16:07:48.212556+0000 mgr.y (mgr.14152) 176 : cluster 0 pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-04-07T16:07:49.695 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:07:49 smithi100 bash[31660]: audit 2024-04-07T16:07:49.185286+0000 mon.a (mon.0) 397 : audit 1 from='client.? 172.21.15.100:0/2754842238' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "17591c6b-862b-43b2-858b-02f4c3199247"}]: dispatch 2024-04-07T16:07:49.695 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:07:49 smithi100 bash[31660]: audit 2024-04-07T16:07:49.196517+0000 mon.a (mon.0) 398 : audit 1 from='client.? 172.21.15.100:0/2754842238' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "17591c6b-862b-43b2-858b-02f4c3199247"}]': finished 2024-04-07T16:07:49.695 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:07:49 smithi100 bash[31660]: cluster 2024-04-07T16:07:49.196658+0000 mon.a (mon.0) 399 : cluster 0 osdmap e22: 4 total, 3 up, 4 in 2024-04-07T16:07:49.695 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:07:49 smithi100 bash[31660]: audit 2024-04-07T16:07:49.196853+0000 mon.a (mon.0) 400 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-07T16:07:49.695 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:07:49 smithi100 bash[21346]: cluster 2024-04-07T16:07:48.212556+0000 mgr.y (mgr.14152) 176 : cluster 0 pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-04-07T16:07:49.695 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:07:49 smithi100 bash[21346]: audit 2024-04-07T16:07:49.185286+0000 mon.a (mon.0) 397 : audit 1 from='client.? 172.21.15.100:0/2754842238' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "17591c6b-862b-43b2-858b-02f4c3199247"}]: dispatch 2024-04-07T16:07:49.696 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:07:49 smithi100 bash[21346]: audit 2024-04-07T16:07:49.196517+0000 mon.a (mon.0) 398 : audit 1 from='client.? 172.21.15.100:0/2754842238' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "17591c6b-862b-43b2-858b-02f4c3199247"}]': finished 2024-04-07T16:07:49.696 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:07:49 smithi100 bash[21346]: cluster 2024-04-07T16:07:49.196658+0000 mon.a (mon.0) 399 : cluster 0 osdmap e22: 4 total, 3 up, 4 in 2024-04-07T16:07:49.696 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:07:49 smithi100 bash[21346]: audit 2024-04-07T16:07:49.196853+0000 mon.a (mon.0) 400 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-07T16:07:50.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:07:50 smithi144 bash[23522]: audit 2024-04-07T16:07:49.906370+0000 mon.a (mon.0) 401 : audit 0 from='client.? 172.21.15.100:0/501282878' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-07T16:07:50.695 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:07:50 smithi100 bash[21346]: audit 2024-04-07T16:07:49.906370+0000 mon.a (mon.0) 401 : audit 0 from='client.? 172.21.15.100:0/501282878' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-07T16:07:50.695 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:07:50 smithi100 bash[31660]: audit 2024-04-07T16:07:49.906370+0000 mon.a (mon.0) 401 : audit 0 from='client.? 172.21.15.100:0/501282878' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-07T16:07:51.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:07:51 smithi144 bash[23522]: cluster 2024-04-07T16:07:50.213041+0000 mgr.y (mgr.14152) 177 : cluster 0 pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-04-07T16:07:51.695 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:07:51 smithi100 bash[21346]: cluster 2024-04-07T16:07:50.213041+0000 mgr.y (mgr.14152) 177 : cluster 0 pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-04-07T16:07:51.695 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:07:51 smithi100 bash[31660]: cluster 2024-04-07T16:07:50.213041+0000 mgr.y (mgr.14152) 177 : cluster 0 pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-04-07T16:07:53.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:07:53 smithi144 bash[23522]: cluster 2024-04-07T16:07:52.213567+0000 mgr.y (mgr.14152) 178 : cluster 0 pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-04-07T16:07:53.695 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:07:53 smithi100 bash[21346]: cluster 2024-04-07T16:07:52.213567+0000 mgr.y (mgr.14152) 178 : cluster 0 pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-04-07T16:07:53.695 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:07:53 smithi100 bash[31660]: cluster 2024-04-07T16:07:52.213567+0000 mgr.y (mgr.14152) 178 : cluster 0 pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-04-07T16:07:55.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:07:55 smithi144 bash[23522]: cluster 2024-04-07T16:07:54.214072+0000 mgr.y (mgr.14152) 179 : cluster 0 pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-04-07T16:07:55.695 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:07:55 smithi100 bash[21346]: cluster 2024-04-07T16:07:54.214072+0000 mgr.y (mgr.14152) 179 : cluster 0 pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-04-07T16:07:55.695 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:07:55 smithi100 bash[31660]: cluster 2024-04-07T16:07:54.214072+0000 mgr.y (mgr.14152) 179 : cluster 0 pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-04-07T16:07:57.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:07:57 smithi144 bash[23522]: cluster 2024-04-07T16:07:56.214556+0000 mgr.y (mgr.14152) 180 : cluster 0 pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-07T16:07:57.695 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:07:57 smithi100 bash[21346]: cluster 2024-04-07T16:07:56.214556+0000 mgr.y (mgr.14152) 180 : cluster 0 pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-07T16:07:57.695 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:07:57 smithi100 bash[31660]: cluster 2024-04-07T16:07:56.214556+0000 mgr.y (mgr.14152) 180 : cluster 0 pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-07T16:07:59.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:07:59 smithi144 bash[23522]: cluster 2024-04-07T16:07:58.215105+0000 mgr.y (mgr.14152) 181 : cluster 0 pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-07T16:07:59.695 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:07:59 smithi100 bash[21346]: cluster 2024-04-07T16:07:58.215105+0000 mgr.y (mgr.14152) 181 : cluster 0 pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-07T16:07:59.695 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:07:59 smithi100 bash[31660]: cluster 2024-04-07T16:07:58.215105+0000 mgr.y (mgr.14152) 181 : cluster 0 pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-07T16:08:01.569 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:08:01 smithi100 bash[31660]: cluster 2024-04-07T16:08:00.215491+0000 mgr.y (mgr.14152) 182 : cluster 0 pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-07T16:08:01.569 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:08:01 smithi100 bash[31660]: audit 2024-04-07T16:08:01.239549+0000 mon.a (mon.0) 402 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-07T16:08:01.569 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:08:01 smithi100 bash[31660]: audit 2024-04-07T16:08:01.241109+0000 mon.a (mon.0) 403 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:08:01.569 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:08:01 smithi100 bash[21346]: cluster 2024-04-07T16:08:00.215491+0000 mgr.y (mgr.14152) 182 : cluster 0 pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-07T16:08:01.569 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:08:01 smithi100 bash[21346]: audit 2024-04-07T16:08:01.239549+0000 mon.a (mon.0) 402 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-07T16:08:01.569 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:08:01 smithi100 bash[21346]: audit 2024-04-07T16:08:01.241109+0000 mon.a (mon.0) 403 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:08:01.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:08:01 smithi144 bash[23522]: cluster 2024-04-07T16:08:00.215491+0000 mgr.y (mgr.14152) 182 : cluster 0 pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-07T16:08:01.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:08:01 smithi144 bash[23522]: audit 2024-04-07T16:08:01.239549+0000 mon.a (mon.0) 402 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-07T16:08:01.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:08:01 smithi144 bash[23522]: audit 2024-04-07T16:08:01.241109+0000 mon.a (mon.0) 403 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:08:02.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:08:02 smithi144 bash[23522]: cephadm 2024-04-07T16:08:01.242611+0000 mgr.y (mgr.14152) 183 : cephadm 1 Deploying daemon osd.3 on smithi100 2024-04-07T16:08:02.628 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:08:02 smithi100 bash[21346]: cephadm 2024-04-07T16:08:01.242611+0000 mgr.y (mgr.14152) 183 : cephadm 1 Deploying daemon osd.3 on smithi100 2024-04-07T16:08:02.628 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:08:02 smithi100 bash[31660]: cephadm 2024-04-07T16:08:01.242611+0000 mgr.y (mgr.14152) 183 : cephadm 1 Deploying daemon osd.3 on smithi100 2024-04-07T16:08:03.444 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:08:03 smithi100 bash[31660]: cluster 2024-04-07T16:08:02.215967+0000 mgr.y (mgr.14152) 184 : cluster 0 pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-07T16:08:03.445 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:08:03 smithi100 bash[21346]: cluster 2024-04-07T16:08:02.215967+0000 mgr.y (mgr.14152) 184 : cluster 0 pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-07T16:08:03.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:08:03 smithi144 bash[23522]: cluster 2024-04-07T16:08:02.215967+0000 mgr.y (mgr.14152) 184 : cluster 0 pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-07T16:08:05.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:08:05 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:08:05.185 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:08:05 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:08:05.185 INFO:journalctl@ceph.osd.0.smithi100.stdout:Apr 07 16:08:05 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:08:05.186 INFO:journalctl@ceph.osd.1.smithi100.stdout:Apr 07 16:08:05 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:08:05.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:08:05 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:08:05.186 INFO:journalctl@ceph.osd.2.smithi100.stdout:Apr 07 16:08:05 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:08:05.444 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:08:05 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:08:05.445 INFO:journalctl@ceph.osd.0.smithi100.stdout:Apr 07 16:08:05 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:08:05.445 INFO:journalctl@ceph.osd.1.smithi100.stdout:Apr 07 16:08:05 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:08:05.445 INFO:journalctl@ceph.osd.2.smithi100.stdout:Apr 07 16:08:05 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:08:05.445 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:08:05 smithi100 bash[21346]: cluster 2024-04-07T16:08:04.216466+0000 mgr.y (mgr.14152) 185 : cluster 0 pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-07T16:08:05.445 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:08:05 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:08:05.446 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:08:05 smithi100 bash[31660]: cluster 2024-04-07T16:08:04.216466+0000 mgr.y (mgr.14152) 185 : cluster 0 pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-07T16:08:05.446 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:08:05 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:08:05.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:08:05 smithi144 bash[23522]: cluster 2024-04-07T16:08:04.216466+0000 mgr.y (mgr.14152) 185 : cluster 0 pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-07T16:08:06.694 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:08:06 smithi100 bash[21346]: audit 2024-04-07T16:08:05.472181+0000 mon.a (mon.0) 404 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:08:06.695 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:08:06 smithi100 bash[21346]: audit 2024-04-07T16:08:05.477563+0000 mon.a (mon.0) 405 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:08:06.695 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:08:06 smithi100 bash[21346]: audit 2024-04-07T16:08:05.477958+0000 mon.a (mon.0) 406 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:08:06.695 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:08:06 smithi100 bash[31660]: audit 2024-04-07T16:08:05.472181+0000 mon.a (mon.0) 404 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:08:06.695 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:08:06 smithi100 bash[31660]: audit 2024-04-07T16:08:05.477563+0000 mon.a (mon.0) 405 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:08:06.695 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:08:06 smithi100 bash[31660]: audit 2024-04-07T16:08:05.477958+0000 mon.a (mon.0) 406 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:08:06.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:08:06 smithi144 bash[23522]: audit 2024-04-07T16:08:05.472181+0000 mon.a (mon.0) 404 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:08:06.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:08:06 smithi144 bash[23522]: audit 2024-04-07T16:08:05.477563+0000 mon.a (mon.0) 405 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:08:06.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:08:06 smithi144 bash[23522]: audit 2024-04-07T16:08:05.477958+0000 mon.a (mon.0) 406 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:08:07.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:08:07 smithi144 bash[23522]: cluster 2024-04-07T16:08:06.216942+0000 mgr.y (mgr.14152) 186 : cluster 0 pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-07T16:08:07.944 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:08:07 smithi100 bash[21346]: cluster 2024-04-07T16:08:06.216942+0000 mgr.y (mgr.14152) 186 : cluster 0 pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-07T16:08:07.945 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:08:07 smithi100 bash[31660]: cluster 2024-04-07T16:08:06.216942+0000 mgr.y (mgr.14152) 186 : cluster 0 pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-07T16:08:09.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:08:09 smithi144 bash[23522]: cluster 2024-04-07T16:08:08.217496+0000 mgr.y (mgr.14152) 187 : cluster 0 pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-07T16:08:09.944 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:08:09 smithi100 bash[21346]: cluster 2024-04-07T16:08:08.217496+0000 mgr.y (mgr.14152) 187 : cluster 0 pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-07T16:08:09.945 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:08:09 smithi100 bash[31660]: cluster 2024-04-07T16:08:08.217496+0000 mgr.y (mgr.14152) 187 : cluster 0 pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-07T16:08:12.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:08:11 smithi144 bash[23522]: cluster 2024-04-07T16:08:10.218009+0000 mgr.y (mgr.14152) 188 : cluster 0 pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-07T16:08:12.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:08:11 smithi144 bash[23522]: audit 2024-04-07T16:08:10.718203+0000 mon.a (mon.0) 407 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:08:12.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:08:11 smithi144 bash[23522]: audit 2024-04-07T16:08:10.724821+0000 mon.a (mon.0) 408 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:08:12.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:08:11 smithi144 bash[23522]: audit 2024-04-07T16:08:10.725968+0000 mon.a (mon.0) 409 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:08:12.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:08:11 smithi144 bash[23522]: audit 2024-04-07T16:08:10.727293+0000 mon.a (mon.0) 410 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:08:12.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:08:11 smithi144 bash[23522]: audit 2024-04-07T16:08:10.733000+0000 mon.a (mon.0) 411 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:08:12.195 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:08:11 smithi100 bash[21346]: cluster 2024-04-07T16:08:10.218009+0000 mgr.y (mgr.14152) 188 : cluster 0 pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-07T16:08:12.195 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:08:11 smithi100 bash[21346]: audit 2024-04-07T16:08:10.718203+0000 mon.a (mon.0) 407 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:08:12.195 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:08:11 smithi100 bash[21346]: audit 2024-04-07T16:08:10.724821+0000 mon.a (mon.0) 408 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:08:12.195 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:08:11 smithi100 bash[21346]: audit 2024-04-07T16:08:10.725968+0000 mon.a (mon.0) 409 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:08:12.195 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:08:11 smithi100 bash[21346]: audit 2024-04-07T16:08:10.727293+0000 mon.a (mon.0) 410 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:08:12.195 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:08:11 smithi100 bash[21346]: audit 2024-04-07T16:08:10.733000+0000 mon.a (mon.0) 411 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:08:12.195 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:08:11 smithi100 bash[31660]: cluster 2024-04-07T16:08:10.218009+0000 mgr.y (mgr.14152) 188 : cluster 0 pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-07T16:08:12.195 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:08:11 smithi100 bash[31660]: audit 2024-04-07T16:08:10.718203+0000 mon.a (mon.0) 407 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:08:12.195 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:08:11 smithi100 bash[31660]: audit 2024-04-07T16:08:10.724821+0000 mon.a (mon.0) 408 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:08:12.195 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:08:11 smithi100 bash[31660]: audit 2024-04-07T16:08:10.725968+0000 mon.a (mon.0) 409 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:08:12.196 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:08:11 smithi100 bash[31660]: audit 2024-04-07T16:08:10.727293+0000 mon.a (mon.0) 410 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:08:12.196 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:08:11 smithi100 bash[31660]: audit 2024-04-07T16:08:10.733000+0000 mon.a (mon.0) 411 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:08:12.944 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:08:12 smithi100 bash[31660]: audit 2024-04-07T16:08:11.919188+0000 mon.c (mon.2) 14 : audit 1 from='osd.3 [v2:172.21.15.100:6826/3019212393,v1:172.21.15.100:6827/3019212393]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-07T16:08:12.944 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:08:12 smithi100 bash[31660]: audit 2024-04-07T16:08:11.919576+0000 mon.a (mon.0) 412 : audit 1 from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-07T16:08:12.945 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:08:12 smithi100 bash[21346]: audit 2024-04-07T16:08:11.919188+0000 mon.c (mon.2) 14 : audit 1 from='osd.3 [v2:172.21.15.100:6826/3019212393,v1:172.21.15.100:6827/3019212393]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-07T16:08:12.945 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:08:12 smithi100 bash[21346]: audit 2024-04-07T16:08:11.919576+0000 mon.a (mon.0) 412 : audit 1 from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-07T16:08:13.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:08:12 smithi144 bash[23522]: audit 2024-04-07T16:08:11.919188+0000 mon.c (mon.2) 14 : audit 1 from='osd.3 [v2:172.21.15.100:6826/3019212393,v1:172.21.15.100:6827/3019212393]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-07T16:08:13.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:08:12 smithi144 bash[23522]: audit 2024-04-07T16:08:11.919576+0000 mon.a (mon.0) 412 : audit 1 from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-07T16:08:13.839 INFO:teuthology.orchestra.run.smithi100.stdout:Created osd(s) 3 on host 'smithi100' 2024-04-07T16:08:14.060 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:08:13 smithi100 bash[31660]: cluster 2024-04-07T16:08:12.218559+0000 mgr.y (mgr.14152) 189 : cluster 0 pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-07T16:08:14.060 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:08:13 smithi100 bash[31660]: audit 2024-04-07T16:08:12.730795+0000 mon.a (mon.0) 413 : audit 1 from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-07T16:08:14.060 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:08:13 smithi100 bash[31660]: cluster 2024-04-07T16:08:12.730901+0000 mon.a (mon.0) 414 : cluster 0 osdmap e23: 4 total, 3 up, 4 in 2024-04-07T16:08:14.060 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:08:13 smithi100 bash[31660]: audit 2024-04-07T16:08:12.731109+0000 mon.a (mon.0) 415 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-07T16:08:14.060 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:08:13 smithi100 bash[31660]: audit 2024-04-07T16:08:12.734287+0000 mon.c (mon.2) 15 : audit 1 from='osd.3 [v2:172.21.15.100:6826/3019212393,v1:172.21.15.100:6827/3019212393]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-04-07T16:08:14.060 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:08:13 smithi100 bash[31660]: audit 2024-04-07T16:08:12.734699+0000 mon.a (mon.0) 416 : audit 1 from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-04-07T16:08:14.060 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:08:13 smithi100 bash[21346]: cluster 2024-04-07T16:08:12.218559+0000 mgr.y (mgr.14152) 189 : cluster 0 pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-07T16:08:14.060 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:08:13 smithi100 bash[21346]: audit 2024-04-07T16:08:12.730795+0000 mon.a (mon.0) 413 : audit 1 from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-07T16:08:14.061 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:08:13 smithi100 bash[21346]: cluster 2024-04-07T16:08:12.730901+0000 mon.a (mon.0) 414 : cluster 0 osdmap e23: 4 total, 3 up, 4 in 2024-04-07T16:08:14.061 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:08:13 smithi100 bash[21346]: audit 2024-04-07T16:08:12.731109+0000 mon.a (mon.0) 415 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-07T16:08:14.061 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:08:13 smithi100 bash[21346]: audit 2024-04-07T16:08:12.734287+0000 mon.c (mon.2) 15 : audit 1 from='osd.3 [v2:172.21.15.100:6826/3019212393,v1:172.21.15.100:6827/3019212393]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-04-07T16:08:14.061 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:08:13 smithi100 bash[21346]: audit 2024-04-07T16:08:12.734699+0000 mon.a (mon.0) 416 : audit 1 from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-04-07T16:08:14.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:08:13 smithi144 bash[23522]: cluster 2024-04-07T16:08:12.218559+0000 mgr.y (mgr.14152) 189 : cluster 0 pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-07T16:08:14.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:08:13 smithi144 bash[23522]: audit 2024-04-07T16:08:12.730795+0000 mon.a (mon.0) 413 : audit 1 from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-07T16:08:14.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:08:13 smithi144 bash[23522]: cluster 2024-04-07T16:08:12.730901+0000 mon.a (mon.0) 414 : cluster 0 osdmap e23: 4 total, 3 up, 4 in 2024-04-07T16:08:14.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:08:13 smithi144 bash[23522]: audit 2024-04-07T16:08:12.731109+0000 mon.a (mon.0) 415 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-07T16:08:14.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:08:13 smithi144 bash[23522]: audit 2024-04-07T16:08:12.734287+0000 mon.c (mon.2) 15 : audit 1 from='osd.3 [v2:172.21.15.100:6826/3019212393,v1:172.21.15.100:6827/3019212393]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-04-07T16:08:14.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:08:13 smithi144 bash[23522]: audit 2024-04-07T16:08:12.734699+0000 mon.a (mon.0) 416 : audit 1 from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-04-07T16:08:14.521 DEBUG:teuthology.orchestra.run.smithi100:osd.3> sudo journalctl -f -n 0 -u ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@osd.3.service 2024-04-07T16:08:14.524 INFO:tasks.cephadm:Deploying osd.4 on smithi144 with /dev/vg_nvme/lv_4... 2024-04-07T16:08:14.524 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0702e3a8-f4f8-11ee-b649-cb9ed24678a4 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-07T16:08:14.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:08:14 smithi144 bash[23522]: audit 2024-04-07T16:08:13.734770+0000 mon.a (mon.0) 417 : audit 1 from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]': finished 2024-04-07T16:08:14.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:08:14 smithi144 bash[23522]: cluster 2024-04-07T16:08:13.734913+0000 mon.a (mon.0) 418 : cluster 0 osdmap e24: 4 total, 3 up, 4 in 2024-04-07T16:08:14.860 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:08:14 smithi144 bash[23522]: audit 2024-04-07T16:08:13.735628+0000 mon.a (mon.0) 419 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-07T16:08:14.860 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:08:14 smithi144 bash[23522]: audit 2024-04-07T16:08:13.743798+0000 mon.a (mon.0) 420 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-07T16:08:14.860 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:08:14 smithi144 bash[23522]: audit 2024-04-07T16:08:13.817842+0000 mon.a (mon.0) 421 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:08:14.860 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:08:14 smithi144 bash[23522]: audit 2024-04-07T16:08:13.822014+0000 mon.a (mon.0) 422 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:08:14.860 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:08:14 smithi144 bash[23522]: audit 2024-04-07T16:08:13.826829+0000 mon.a (mon.0) 423 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:08:14.945 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:08:14 smithi100 bash[21346]: audit 2024-04-07T16:08:13.734770+0000 mon.a (mon.0) 417 : audit 1 from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]': finished 2024-04-07T16:08:14.945 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:08:14 smithi100 bash[21346]: cluster 2024-04-07T16:08:13.734913+0000 mon.a (mon.0) 418 : cluster 0 osdmap e24: 4 total, 3 up, 4 in 2024-04-07T16:08:14.945 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:08:14 smithi100 bash[21346]: audit 2024-04-07T16:08:13.735628+0000 mon.a (mon.0) 419 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-07T16:08:14.945 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:08:14 smithi100 bash[21346]: audit 2024-04-07T16:08:13.743798+0000 mon.a (mon.0) 420 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-07T16:08:14.945 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:08:14 smithi100 bash[21346]: audit 2024-04-07T16:08:13.817842+0000 mon.a (mon.0) 421 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:08:14.945 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:08:14 smithi100 bash[21346]: audit 2024-04-07T16:08:13.822014+0000 mon.a (mon.0) 422 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:08:14.945 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:08:14 smithi100 bash[21346]: audit 2024-04-07T16:08:13.826829+0000 mon.a (mon.0) 423 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:08:14.945 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:08:14 smithi100 bash[31660]: audit 2024-04-07T16:08:13.734770+0000 mon.a (mon.0) 417 : audit 1 from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]': finished 2024-04-07T16:08:14.945 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:08:14 smithi100 bash[31660]: cluster 2024-04-07T16:08:13.734913+0000 mon.a (mon.0) 418 : cluster 0 osdmap e24: 4 total, 3 up, 4 in 2024-04-07T16:08:14.945 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:08:14 smithi100 bash[31660]: audit 2024-04-07T16:08:13.735628+0000 mon.a (mon.0) 419 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-07T16:08:14.945 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:08:14 smithi100 bash[31660]: audit 2024-04-07T16:08:13.743798+0000 mon.a (mon.0) 420 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-07T16:08:14.946 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:08:14 smithi100 bash[31660]: audit 2024-04-07T16:08:13.817842+0000 mon.a (mon.0) 421 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:08:14.946 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:08:14 smithi100 bash[31660]: audit 2024-04-07T16:08:13.822014+0000 mon.a (mon.0) 422 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:08:14.946 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:08:14 smithi100 bash[31660]: audit 2024-04-07T16:08:13.826829+0000 mon.a (mon.0) 423 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:08:16.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:08:15 smithi144 bash[23522]: cluster 2024-04-07T16:08:12.935537+0000 osd.3 (osd.3) 1 : cluster 0 purged_snaps scrub starts 2024-04-07T16:08:16.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:08:15 smithi144 bash[23522]: cluster 2024-04-07T16:08:12.935603+0000 osd.3 (osd.3) 2 : cluster 0 purged_snaps scrub ok 2024-04-07T16:08:16.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:08:15 smithi144 bash[23522]: cluster 2024-04-07T16:08:14.219084+0000 mgr.y (mgr.14152) 190 : cluster 0 pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-07T16:08:16.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:08:15 smithi144 bash[23522]: cluster 2024-04-07T16:08:14.747471+0000 mon.a (mon.0) 424 : cluster 1 osd.3 [v2:172.21.15.100:6826/3019212393,v1:172.21.15.100:6827/3019212393] boot 2024-04-07T16:08:16.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:08:15 smithi144 bash[23522]: cluster 2024-04-07T16:08:14.747587+0000 mon.a (mon.0) 425 : cluster 0 osdmap e25: 4 total, 4 up, 4 in 2024-04-07T16:08:16.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:08:15 smithi144 bash[23522]: audit 2024-04-07T16:08:14.748924+0000 mon.a (mon.0) 426 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-07T16:08:16.194 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:08:15 smithi100 bash[21346]: cluster 2024-04-07T16:08:12.935537+0000 osd.3 (osd.3) 1 : cluster 0 purged_snaps scrub starts 2024-04-07T16:08:16.194 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:08:15 smithi100 bash[21346]: cluster 2024-04-07T16:08:12.935603+0000 osd.3 (osd.3) 2 : cluster 0 purged_snaps scrub ok 2024-04-07T16:08:16.195 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:08:15 smithi100 bash[21346]: cluster 2024-04-07T16:08:14.219084+0000 mgr.y (mgr.14152) 190 : cluster 0 pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-07T16:08:16.195 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:08:15 smithi100 bash[21346]: cluster 2024-04-07T16:08:14.747471+0000 mon.a (mon.0) 424 : cluster 1 osd.3 [v2:172.21.15.100:6826/3019212393,v1:172.21.15.100:6827/3019212393] boot 2024-04-07T16:08:16.195 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:08:15 smithi100 bash[21346]: cluster 2024-04-07T16:08:14.747587+0000 mon.a (mon.0) 425 : cluster 0 osdmap e25: 4 total, 4 up, 4 in 2024-04-07T16:08:16.195 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:08:15 smithi100 bash[21346]: audit 2024-04-07T16:08:14.748924+0000 mon.a (mon.0) 426 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-07T16:08:16.195 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:08:15 smithi100 bash[31660]: cluster 2024-04-07T16:08:12.935537+0000 osd.3 (osd.3) 1 : cluster 0 purged_snaps scrub starts 2024-04-07T16:08:16.195 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:08:15 smithi100 bash[31660]: cluster 2024-04-07T16:08:12.935603+0000 osd.3 (osd.3) 2 : cluster 0 purged_snaps scrub ok 2024-04-07T16:08:16.195 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:08:15 smithi100 bash[31660]: cluster 2024-04-07T16:08:14.219084+0000 mgr.y (mgr.14152) 190 : cluster 0 pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-07T16:08:16.195 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:08:15 smithi100 bash[31660]: cluster 2024-04-07T16:08:14.747471+0000 mon.a (mon.0) 424 : cluster 1 osd.3 [v2:172.21.15.100:6826/3019212393,v1:172.21.15.100:6827/3019212393] boot 2024-04-07T16:08:16.195 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:08:15 smithi100 bash[31660]: cluster 2024-04-07T16:08:14.747587+0000 mon.a (mon.0) 425 : cluster 0 osdmap e25: 4 total, 4 up, 4 in 2024-04-07T16:08:16.195 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:08:15 smithi100 bash[31660]: audit 2024-04-07T16:08:14.748924+0000 mon.a (mon.0) 426 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-07T16:08:17.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:08:16 smithi144 bash[23522]: cluster 2024-04-07T16:08:15.758110+0000 mon.a (mon.0) 427 : cluster 0 osdmap e26: 4 total, 4 up, 4 in 2024-04-07T16:08:17.194 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:08:16 smithi100 bash[21346]: cluster 2024-04-07T16:08:15.758110+0000 mon.a (mon.0) 427 : cluster 0 osdmap e26: 4 total, 4 up, 4 in 2024-04-07T16:08:17.195 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:08:16 smithi100 bash[31660]: cluster 2024-04-07T16:08:15.758110+0000 mon.a (mon.0) 427 : cluster 0 osdmap e26: 4 total, 4 up, 4 in 2024-04-07T16:08:18.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:08:17 smithi144 bash[23522]: cluster 2024-04-07T16:08:16.219662+0000 mgr.y (mgr.14152) 191 : cluster 0 pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-07T16:08:18.194 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:08:17 smithi100 bash[21346]: cluster 2024-04-07T16:08:16.219662+0000 mgr.y (mgr.14152) 191 : cluster 0 pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-07T16:08:18.195 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:08:17 smithi100 bash[31660]: cluster 2024-04-07T16:08:16.219662+0000 mgr.y (mgr.14152) 191 : cluster 0 pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-07T16:08:19.188 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/mon.b/config 2024-04-07T16:08:20.071 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:08:19 smithi144 bash[23522]: cluster 2024-04-07T16:08:18.220180+0000 mgr.y (mgr.14152) 192 : cluster 0 pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-07T16:08:20.194 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:08:19 smithi100 bash[21346]: cluster 2024-04-07T16:08:18.220180+0000 mgr.y (mgr.14152) 192 : cluster 0 pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-07T16:08:20.195 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:08:19 smithi100 bash[31660]: cluster 2024-04-07T16:08:18.220180+0000 mgr.y (mgr.14152) 192 : cluster 0 pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-07T16:08:21.980 INFO:teuthology.orchestra.run.smithi144.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-07T16:08:21.980 INFO:teuthology.orchestra.run.smithi144.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-07T16:08:21.980 INFO:teuthology.orchestra.run.smithi144.stderr: stderr: 10+0 records in 2024-04-07T16:08:21.980 INFO:teuthology.orchestra.run.smithi144.stderr:10+0 records out 2024-04-07T16:08:21.980 INFO:teuthology.orchestra.run.smithi144.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0145899 s, 719 MB/s 2024-04-07T16:08:21.980 INFO:teuthology.orchestra.run.smithi144.stderr:--> Zapping successful for: 2024-04-07T16:08:22.027 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:08:21 smithi144 bash[23522]: cluster 2024-04-07T16:08:20.220717+0000 mgr.y (mgr.14152) 193 : cluster 0 pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-07T16:08:22.181 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:08:21 smithi100 bash[21346]: cluster 2024-04-07T16:08:20.220717+0000 mgr.y (mgr.14152) 193 : cluster 0 pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-07T16:08:22.181 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:08:21 smithi100 bash[31660]: cluster 2024-04-07T16:08:20.220717+0000 mgr.y (mgr.14152) 193 : cluster 0 pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-07T16:08:22.610 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0702e3a8-f4f8-11ee-b649-cb9ed24678a4 -- ceph orch daemon add osd smithi144:vg_nvme/lv_4 2024-04-07T16:08:24.194 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:08:23 smithi100 bash[21346]: cluster 2024-04-07T16:08:22.221099+0000 mgr.y (mgr.14152) 194 : cluster 0 pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-07T16:08:24.195 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:08:23 smithi100 bash[21346]: cephadm 2024-04-07T16:08:22.861520+0000 mgr.y (mgr.14152) 195 : cephadm 1 Detected new or changed devices on smithi100 2024-04-07T16:08:24.195 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:08:23 smithi100 bash[21346]: audit 2024-04-07T16:08:22.875076+0000 mon.a (mon.0) 428 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:08:24.195 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:08:23 smithi100 bash[21346]: audit 2024-04-07T16:08:22.889423+0000 mon.a (mon.0) 429 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:08:24.195 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:08:23 smithi100 bash[21346]: audit 2024-04-07T16:08:22.892902+0000 mon.a (mon.0) 430 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-07T16:08:24.195 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:08:23 smithi100 bash[21346]: audit 2024-04-07T16:08:22.894896+0000 mon.a (mon.0) 431 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-07T16:08:24.195 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:08:23 smithi100 bash[21346]: audit 2024-04-07T16:08:22.896840+0000 mon.a (mon.0) 432 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-07T16:08:24.195 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:08:23 smithi100 bash[21346]: audit 2024-04-07T16:08:22.898529+0000 mon.a (mon.0) 433 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-07T16:08:24.195 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:08:23 smithi100 bash[21346]: cephadm 2024-04-07T16:08:22.899710+0000 mgr.y (mgr.14152) 196 : cephadm 1 Adjusting osd_memory_target on smithi100 to 4060M 2024-04-07T16:08:24.195 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:08:23 smithi100 bash[21346]: audit 2024-04-07T16:08:22.913259+0000 mon.a (mon.0) 434 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:08:24.195 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:08:23 smithi100 bash[21346]: audit 2024-04-07T16:08:22.915896+0000 mon.a (mon.0) 435 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:08:24.195 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:08:23 smithi100 bash[21346]: audit 2024-04-07T16:08:22.917333+0000 mon.a (mon.0) 436 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:08:24.195 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:08:23 smithi100 bash[21346]: audit 2024-04-07T16:08:22.930464+0000 mon.a (mon.0) 437 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:08:24.196 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:08:23 smithi100 bash[31660]: cluster 2024-04-07T16:08:22.221099+0000 mgr.y (mgr.14152) 194 : cluster 0 pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-07T16:08:24.196 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:08:23 smithi100 bash[31660]: cephadm 2024-04-07T16:08:22.861520+0000 mgr.y (mgr.14152) 195 : cephadm 1 Detected new or changed devices on smithi100 2024-04-07T16:08:24.196 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:08:23 smithi100 bash[31660]: audit 2024-04-07T16:08:22.875076+0000 mon.a (mon.0) 428 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:08:24.196 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:08:23 smithi100 bash[31660]: audit 2024-04-07T16:08:22.889423+0000 mon.a (mon.0) 429 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:08:24.196 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:08:23 smithi100 bash[31660]: audit 2024-04-07T16:08:22.892902+0000 mon.a (mon.0) 430 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-07T16:08:24.196 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:08:23 smithi100 bash[31660]: audit 2024-04-07T16:08:22.894896+0000 mon.a (mon.0) 431 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-07T16:08:24.196 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:08:23 smithi100 bash[31660]: audit 2024-04-07T16:08:22.896840+0000 mon.a (mon.0) 432 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-07T16:08:24.196 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:08:23 smithi100 bash[31660]: audit 2024-04-07T16:08:22.898529+0000 mon.a (mon.0) 433 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-07T16:08:24.196 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:08:23 smithi100 bash[31660]: cephadm 2024-04-07T16:08:22.899710+0000 mgr.y (mgr.14152) 196 : cephadm 1 Adjusting osd_memory_target on smithi100 to 4060M 2024-04-07T16:08:24.196 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:08:23 smithi100 bash[31660]: audit 2024-04-07T16:08:22.913259+0000 mon.a (mon.0) 434 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:08:24.196 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:08:23 smithi100 bash[31660]: audit 2024-04-07T16:08:22.915896+0000 mon.a (mon.0) 435 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:08:24.196 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:08:23 smithi100 bash[31660]: audit 2024-04-07T16:08:22.917333+0000 mon.a (mon.0) 436 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:08:24.197 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:08:23 smithi100 bash[31660]: audit 2024-04-07T16:08:22.930464+0000 mon.a (mon.0) 437 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:08:24.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:08:23 smithi144 bash[23522]: cluster 2024-04-07T16:08:22.221099+0000 mgr.y (mgr.14152) 194 : cluster 0 pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-07T16:08:24.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:08:23 smithi144 bash[23522]: cephadm 2024-04-07T16:08:22.861520+0000 mgr.y (mgr.14152) 195 : cephadm 1 Detected new or changed devices on smithi100 2024-04-07T16:08:24.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:08:23 smithi144 bash[23522]: audit 2024-04-07T16:08:22.875076+0000 mon.a (mon.0) 428 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:08:24.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:08:23 smithi144 bash[23522]: audit 2024-04-07T16:08:22.889423+0000 mon.a (mon.0) 429 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:08:24.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:08:23 smithi144 bash[23522]: audit 2024-04-07T16:08:22.892902+0000 mon.a (mon.0) 430 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-07T16:08:24.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:08:23 smithi144 bash[23522]: audit 2024-04-07T16:08:22.894896+0000 mon.a (mon.0) 431 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-07T16:08:24.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:08:23 smithi144 bash[23522]: audit 2024-04-07T16:08:22.896840+0000 mon.a (mon.0) 432 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-07T16:08:24.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:08:23 smithi144 bash[23522]: audit 2024-04-07T16:08:22.898529+0000 mon.a (mon.0) 433 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-07T16:08:24.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:08:23 smithi144 bash[23522]: cephadm 2024-04-07T16:08:22.899710+0000 mgr.y (mgr.14152) 196 : cephadm 1 Adjusting osd_memory_target on smithi100 to 4060M 2024-04-07T16:08:24.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:08:23 smithi144 bash[23522]: audit 2024-04-07T16:08:22.913259+0000 mon.a (mon.0) 434 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:08:24.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:08:23 smithi144 bash[23522]: audit 2024-04-07T16:08:22.915896+0000 mon.a (mon.0) 435 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:08:24.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:08:23 smithi144 bash[23522]: audit 2024-04-07T16:08:22.917333+0000 mon.a (mon.0) 436 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:08:24.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:08:23 smithi144 bash[23522]: audit 2024-04-07T16:08:22.930464+0000 mon.a (mon.0) 437 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:08:26.194 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:08:25 smithi100 bash[21346]: cluster 2024-04-07T16:08:24.221666+0000 mgr.y (mgr.14152) 197 : cluster 0 pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-07T16:08:26.194 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:08:25 smithi100 bash[31660]: cluster 2024-04-07T16:08:24.221666+0000 mgr.y (mgr.14152) 197 : cluster 0 pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-07T16:08:26.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:08:25 smithi144 bash[23522]: cluster 2024-04-07T16:08:24.221666+0000 mgr.y (mgr.14152) 197 : cluster 0 pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-07T16:08:27.279 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/mon.b/config 2024-04-07T16:08:28.138 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:08:27 smithi144 bash[23522]: cluster 2024-04-07T16:08:26.222221+0000 mgr.y (mgr.14152) 198 : cluster 0 pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-07T16:08:28.194 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:08:27 smithi100 bash[21346]: cluster 2024-04-07T16:08:26.222221+0000 mgr.y (mgr.14152) 198 : cluster 0 pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-07T16:08:28.194 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:08:27 smithi100 bash[31660]: cluster 2024-04-07T16:08:26.222221+0000 mgr.y (mgr.14152) 198 : cluster 0 pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-07T16:08:30.194 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:08:29 smithi100 bash[31660]: cluster 2024-04-07T16:08:28.222818+0000 mgr.y (mgr.14152) 199 : cluster 0 pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-07T16:08:30.194 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:08:29 smithi100 bash[31660]: audit 2024-04-07T16:08:29.702634+0000 mon.a (mon.0) 438 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-07T16:08:30.194 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:08:29 smithi100 bash[31660]: audit 2024-04-07T16:08:29.710089+0000 mon.a (mon.0) 439 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-07T16:08:30.195 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:08:29 smithi100 bash[31660]: audit 2024-04-07T16:08:29.711990+0000 mon.a (mon.0) 440 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:08:30.195 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:08:29 smithi100 bash[21346]: cluster 2024-04-07T16:08:28.222818+0000 mgr.y (mgr.14152) 199 : cluster 0 pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-07T16:08:30.195 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:08:29 smithi100 bash[21346]: audit 2024-04-07T16:08:29.702634+0000 mon.a (mon.0) 438 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-07T16:08:30.195 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:08:29 smithi100 bash[21346]: audit 2024-04-07T16:08:29.710089+0000 mon.a (mon.0) 439 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-07T16:08:30.195 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:08:29 smithi100 bash[21346]: audit 2024-04-07T16:08:29.711990+0000 mon.a (mon.0) 440 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:08:30.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:08:29 smithi144 bash[23522]: cluster 2024-04-07T16:08:28.222818+0000 mgr.y (mgr.14152) 199 : cluster 0 pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-07T16:08:30.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:08:29 smithi144 bash[23522]: audit 2024-04-07T16:08:29.702634+0000 mon.a (mon.0) 438 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-07T16:08:30.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:08:29 smithi144 bash[23522]: audit 2024-04-07T16:08:29.710089+0000 mon.a (mon.0) 439 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-07T16:08:30.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:08:29 smithi144 bash[23522]: audit 2024-04-07T16:08:29.711990+0000 mon.a (mon.0) 440 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:08:31.194 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:08:30 smithi100 bash[21346]: audit 2024-04-07T16:08:29.695114+0000 mgr.y (mgr.14152) 200 : audit 0 from='client.24178 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:08:31.195 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:08:30 smithi100 bash[31660]: audit 2024-04-07T16:08:29.695114+0000 mgr.y (mgr.14152) 200 : audit 0 from='client.24178 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:08:31.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:08:30 smithi144 bash[23522]: audit 2024-04-07T16:08:29.695114+0000 mgr.y (mgr.14152) 200 : audit 0 from='client.24178 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:08:32.194 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:08:31 smithi100 bash[21346]: cluster 2024-04-07T16:08:30.223394+0000 mgr.y (mgr.14152) 201 : cluster 0 pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-07T16:08:32.194 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:08:31 smithi100 bash[31660]: cluster 2024-04-07T16:08:30.223394+0000 mgr.y (mgr.14152) 201 : cluster 0 pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-07T16:08:32.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:08:31 smithi144 bash[23522]: cluster 2024-04-07T16:08:30.223394+0000 mgr.y (mgr.14152) 201 : cluster 0 pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-07T16:08:34.194 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:08:33 smithi100 bash[21346]: cluster 2024-04-07T16:08:32.223950+0000 mgr.y (mgr.14152) 202 : cluster 0 pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-07T16:08:34.194 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:08:33 smithi100 bash[31660]: cluster 2024-04-07T16:08:32.223950+0000 mgr.y (mgr.14152) 202 : cluster 0 pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-07T16:08:34.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:08:33 smithi144 bash[23522]: cluster 2024-04-07T16:08:32.223950+0000 mgr.y (mgr.14152) 202 : cluster 0 pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-07T16:08:36.194 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:08:35 smithi100 bash[21346]: cluster 2024-04-07T16:08:34.224486+0000 mgr.y (mgr.14152) 203 : cluster 0 pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-07T16:08:36.194 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:08:35 smithi100 bash[31660]: cluster 2024-04-07T16:08:34.224486+0000 mgr.y (mgr.14152) 203 : cluster 0 pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-07T16:08:36.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:08:35 smithi144 bash[23522]: cluster 2024-04-07T16:08:34.224486+0000 mgr.y (mgr.14152) 203 : cluster 0 pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-07T16:08:38.194 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:08:37 smithi100 bash[21346]: cluster 2024-04-07T16:08:36.225037+0000 mgr.y (mgr.14152) 204 : cluster 0 pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-07T16:08:38.194 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:08:37 smithi100 bash[21346]: audit 2024-04-07T16:08:37.563782+0000 mon.b (mon.1) 9 : audit 1 from='client.? 172.21.15.144:0/4009428049' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6ecc1205-7c78-461e-8037-640136cb3658"}]: dispatch 2024-04-07T16:08:38.194 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:08:37 smithi100 bash[21346]: audit 2024-04-07T16:08:37.566181+0000 mon.a (mon.0) 441 : audit 1 from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6ecc1205-7c78-461e-8037-640136cb3658"}]: dispatch 2024-04-07T16:08:38.195 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:08:37 smithi100 bash[21346]: audit 2024-04-07T16:08:37.574411+0000 mon.a (mon.0) 442 : audit 1 from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6ecc1205-7c78-461e-8037-640136cb3658"}]': finished 2024-04-07T16:08:38.195 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:08:37 smithi100 bash[21346]: cluster 2024-04-07T16:08:37.574525+0000 mon.a (mon.0) 443 : cluster 0 osdmap e27: 5 total, 4 up, 5 in 2024-04-07T16:08:38.195 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:08:37 smithi100 bash[21346]: audit 2024-04-07T16:08:37.574824+0000 mon.a (mon.0) 444 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-07T16:08:38.195 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:08:37 smithi100 bash[31660]: cluster 2024-04-07T16:08:36.225037+0000 mgr.y (mgr.14152) 204 : cluster 0 pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-07T16:08:38.195 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:08:37 smithi100 bash[31660]: audit 2024-04-07T16:08:37.563782+0000 mon.b (mon.1) 9 : audit 1 from='client.? 172.21.15.144:0/4009428049' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6ecc1205-7c78-461e-8037-640136cb3658"}]: dispatch 2024-04-07T16:08:38.195 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:08:37 smithi100 bash[31660]: audit 2024-04-07T16:08:37.566181+0000 mon.a (mon.0) 441 : audit 1 from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6ecc1205-7c78-461e-8037-640136cb3658"}]: dispatch 2024-04-07T16:08:38.195 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:08:37 smithi100 bash[31660]: audit 2024-04-07T16:08:37.574411+0000 mon.a (mon.0) 442 : audit 1 from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6ecc1205-7c78-461e-8037-640136cb3658"}]': finished 2024-04-07T16:08:38.195 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:08:37 smithi100 bash[31660]: cluster 2024-04-07T16:08:37.574525+0000 mon.a (mon.0) 443 : cluster 0 osdmap e27: 5 total, 4 up, 5 in 2024-04-07T16:08:38.195 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:08:37 smithi100 bash[31660]: audit 2024-04-07T16:08:37.574824+0000 mon.a (mon.0) 444 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-07T16:08:38.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:08:37 smithi144 bash[23522]: cluster 2024-04-07T16:08:36.225037+0000 mgr.y (mgr.14152) 204 : cluster 0 pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-07T16:08:38.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:08:37 smithi144 bash[23522]: audit 2024-04-07T16:08:37.563782+0000 mon.b (mon.1) 9 : audit 1 from='client.? 172.21.15.144:0/4009428049' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6ecc1205-7c78-461e-8037-640136cb3658"}]: dispatch 2024-04-07T16:08:38.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:08:37 smithi144 bash[23522]: audit 2024-04-07T16:08:37.566181+0000 mon.a (mon.0) 441 : audit 1 from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6ecc1205-7c78-461e-8037-640136cb3658"}]: dispatch 2024-04-07T16:08:38.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:08:37 smithi144 bash[23522]: audit 2024-04-07T16:08:37.574411+0000 mon.a (mon.0) 442 : audit 1 from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6ecc1205-7c78-461e-8037-640136cb3658"}]': finished 2024-04-07T16:08:38.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:08:37 smithi144 bash[23522]: cluster 2024-04-07T16:08:37.574525+0000 mon.a (mon.0) 443 : cluster 0 osdmap e27: 5 total, 4 up, 5 in 2024-04-07T16:08:38.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:08:37 smithi144 bash[23522]: audit 2024-04-07T16:08:37.574824+0000 mon.a (mon.0) 444 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-07T16:08:39.194 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:08:38 smithi100 bash[21346]: audit 2024-04-07T16:08:38.258682+0000 mon.b (mon.1) 10 : audit 0 from='client.? 172.21.15.144:0/1910999957' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-07T16:08:39.194 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:08:38 smithi100 bash[31660]: audit 2024-04-07T16:08:38.258682+0000 mon.b (mon.1) 10 : audit 0 from='client.? 172.21.15.144:0/1910999957' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-07T16:08:39.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:08:38 smithi144 bash[23522]: audit 2024-04-07T16:08:38.258682+0000 mon.b (mon.1) 10 : audit 0 from='client.? 172.21.15.144:0/1910999957' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-07T16:08:40.194 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:08:39 smithi100 bash[31660]: cluster 2024-04-07T16:08:38.225556+0000 mgr.y (mgr.14152) 205 : cluster 0 pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-07T16:08:40.194 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:08:39 smithi100 bash[21346]: cluster 2024-04-07T16:08:38.225556+0000 mgr.y (mgr.14152) 205 : cluster 0 pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-07T16:08:40.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:08:39 smithi144 bash[23522]: cluster 2024-04-07T16:08:38.225556+0000 mgr.y (mgr.14152) 205 : cluster 0 pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-07T16:08:42.194 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:08:41 smithi100 bash[31660]: cluster 2024-04-07T16:08:40.226055+0000 mgr.y (mgr.14152) 206 : cluster 0 pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-07T16:08:42.194 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:08:41 smithi100 bash[21346]: cluster 2024-04-07T16:08:40.226055+0000 mgr.y (mgr.14152) 206 : cluster 0 pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-07T16:08:42.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:08:41 smithi144 bash[23522]: cluster 2024-04-07T16:08:40.226055+0000 mgr.y (mgr.14152) 206 : cluster 0 pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-07T16:08:44.194 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:08:43 smithi100 bash[21346]: cluster 2024-04-07T16:08:42.226607+0000 mgr.y (mgr.14152) 207 : cluster 0 pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-07T16:08:44.194 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:08:43 smithi100 bash[31660]: cluster 2024-04-07T16:08:42.226607+0000 mgr.y (mgr.14152) 207 : cluster 0 pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-07T16:08:44.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:08:43 smithi144 bash[23522]: cluster 2024-04-07T16:08:42.226607+0000 mgr.y (mgr.14152) 207 : cluster 0 pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-07T16:08:46.194 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:08:45 smithi100 bash[21346]: cluster 2024-04-07T16:08:44.227142+0000 mgr.y (mgr.14152) 208 : cluster 0 pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-07T16:08:46.194 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:08:45 smithi100 bash[31660]: cluster 2024-04-07T16:08:44.227142+0000 mgr.y (mgr.14152) 208 : cluster 0 pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-07T16:08:46.264 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:08:45 smithi144 bash[23522]: cluster 2024-04-07T16:08:44.227142+0000 mgr.y (mgr.14152) 208 : cluster 0 pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-07T16:08:48.194 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:08:47 smithi100 bash[21346]: cluster 2024-04-07T16:08:46.227698+0000 mgr.y (mgr.14152) 209 : cluster 0 pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-07T16:08:48.194 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:08:47 smithi100 bash[31660]: cluster 2024-04-07T16:08:46.227698+0000 mgr.y (mgr.14152) 209 : cluster 0 pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-07T16:08:48.209 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:08:47 smithi144 bash[23522]: cluster 2024-04-07T16:08:46.227698+0000 mgr.y (mgr.14152) 209 : cluster 0 pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-07T16:08:50.194 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:08:49 smithi100 bash[21346]: cluster 2024-04-07T16:08:48.228234+0000 mgr.y (mgr.14152) 210 : cluster 0 pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-07T16:08:50.194 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:08:49 smithi100 bash[21346]: audit 2024-04-07T16:08:49.291483+0000 mon.a (mon.0) 445 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-07T16:08:50.194 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:08:49 smithi100 bash[21346]: audit 2024-04-07T16:08:49.293263+0000 mon.a (mon.0) 446 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:08:50.194 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:08:49 smithi100 bash[31660]: cluster 2024-04-07T16:08:48.228234+0000 mgr.y (mgr.14152) 210 : cluster 0 pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-07T16:08:50.194 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:08:49 smithi100 bash[31660]: audit 2024-04-07T16:08:49.291483+0000 mon.a (mon.0) 445 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-07T16:08:50.195 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:08:49 smithi100 bash[31660]: audit 2024-04-07T16:08:49.293263+0000 mon.a (mon.0) 446 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:08:50.291 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:08:49 smithi144 bash[23522]: cluster 2024-04-07T16:08:48.228234+0000 mgr.y (mgr.14152) 210 : cluster 0 pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-07T16:08:50.291 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:08:49 smithi144 bash[23522]: audit 2024-04-07T16:08:49.291483+0000 mon.a (mon.0) 445 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-07T16:08:50.291 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:08:49 smithi144 bash[23522]: audit 2024-04-07T16:08:49.293263+0000 mon.a (mon.0) 446 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:08:51.194 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:08:50 smithi100 bash[21346]: cephadm 2024-04-07T16:08:49.294750+0000 mgr.y (mgr.14152) 211 : cephadm 1 Deploying daemon osd.4 on smithi144 2024-04-07T16:08:51.194 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:08:50 smithi100 bash[31660]: cephadm 2024-04-07T16:08:49.294750+0000 mgr.y (mgr.14152) 211 : cephadm 1 Deploying daemon osd.4 on smithi144 2024-04-07T16:08:51.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:08:50 smithi144 bash[23522]: cephadm 2024-04-07T16:08:49.294750+0000 mgr.y (mgr.14152) 211 : cephadm 1 Deploying daemon osd.4 on smithi144 2024-04-07T16:08:52.029 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:08:51 smithi144 bash[23522]: cluster 2024-04-07T16:08:50.228839+0000 mgr.y (mgr.14152) 212 : cluster 0 pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-07T16:08:52.194 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:08:51 smithi100 bash[21346]: cluster 2024-04-07T16:08:50.228839+0000 mgr.y (mgr.14152) 212 : cluster 0 pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-07T16:08:52.194 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:08:51 smithi100 bash[31660]: cluster 2024-04-07T16:08:50.228839+0000 mgr.y (mgr.14152) 212 : cluster 0 pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-07T16:08:53.108 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:08:52 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:08:53.109 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:08:52 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:08:53.608 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:08:53 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:08:53.609 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:08:53 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:08:54.194 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:08:53 smithi100 bash[21346]: cluster 2024-04-07T16:08:52.229379+0000 mgr.y (mgr.14152) 213 : cluster 0 pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-07T16:08:54.194 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:08:53 smithi100 bash[21346]: audit 2024-04-07T16:08:53.273812+0000 mon.a (mon.0) 447 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:08:54.194 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:08:53 smithi100 bash[21346]: audit 2024-04-07T16:08:53.282262+0000 mon.a (mon.0) 448 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:08:54.194 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:08:53 smithi100 bash[21346]: audit 2024-04-07T16:08:53.295283+0000 mon.a (mon.0) 449 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:08:54.194 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:08:53 smithi100 bash[31660]: cluster 2024-04-07T16:08:52.229379+0000 mgr.y (mgr.14152) 213 : cluster 0 pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-07T16:08:54.194 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:08:53 smithi100 bash[31660]: audit 2024-04-07T16:08:53.273812+0000 mon.a (mon.0) 447 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:08:54.194 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:08:53 smithi100 bash[31660]: audit 2024-04-07T16:08:53.282262+0000 mon.a (mon.0) 448 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:08:54.194 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:08:53 smithi100 bash[31660]: audit 2024-04-07T16:08:53.295283+0000 mon.a (mon.0) 449 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:08:54.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:08:53 smithi144 bash[23522]: cluster 2024-04-07T16:08:52.229379+0000 mgr.y (mgr.14152) 213 : cluster 0 pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-07T16:08:54.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:08:53 smithi144 bash[23522]: audit 2024-04-07T16:08:53.273812+0000 mon.a (mon.0) 447 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:08:54.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:08:53 smithi144 bash[23522]: audit 2024-04-07T16:08:53.282262+0000 mon.a (mon.0) 448 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:08:54.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:08:53 smithi144 bash[23522]: audit 2024-04-07T16:08:53.295283+0000 mon.a (mon.0) 449 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:08:56.255 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:08:55 smithi144 bash[23522]: cluster 2024-04-07T16:08:54.229924+0000 mgr.y (mgr.14152) 214 : cluster 0 pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-07T16:08:56.444 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:08:55 smithi100 bash[21346]: cluster 2024-04-07T16:08:54.229924+0000 mgr.y (mgr.14152) 214 : cluster 0 pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-07T16:08:56.444 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:08:55 smithi100 bash[31660]: cluster 2024-04-07T16:08:54.229924+0000 mgr.y (mgr.14152) 214 : cluster 0 pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-07T16:08:58.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:08:57 smithi144 bash[23522]: cluster 2024-04-07T16:08:56.230493+0000 mgr.y (mgr.14152) 215 : cluster 0 pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-07T16:08:58.443 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:08:57 smithi100 bash[21346]: cluster 2024-04-07T16:08:56.230493+0000 mgr.y (mgr.14152) 215 : cluster 0 pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-07T16:08:58.444 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:08:57 smithi100 bash[31660]: cluster 2024-04-07T16:08:56.230493+0000 mgr.y (mgr.14152) 215 : cluster 0 pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-07T16:09:00.264 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:08:59 smithi144 bash[23522]: cluster 2024-04-07T16:08:58.231112+0000 mgr.y (mgr.14152) 216 : cluster 0 pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-07T16:09:00.264 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:08:59 smithi144 bash[23522]: audit 2024-04-07T16:08:59.176968+0000 mon.b (mon.1) 11 : audit 1 from='osd.4 [v2:172.21.15.144:6800/974906744,v1:172.21.15.144:6801/974906744]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-07T16:09:00.264 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:08:59 smithi144 bash[23522]: audit 2024-04-07T16:08:59.179149+0000 mon.a (mon.0) 450 : audit 1 from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-07T16:09:00.265 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:08:59 smithi144 bash[23522]: audit 2024-04-07T16:08:59.848326+0000 mon.a (mon.0) 451 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:09:00.265 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:08:59 smithi144 bash[23522]: audit 2024-04-07T16:08:59.859129+0000 mon.a (mon.0) 452 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:09:00.265 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:08:59 smithi144 bash[23522]: audit 2024-04-07T16:08:59.861132+0000 mon.a (mon.0) 453 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:09:00.265 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:08:59 smithi144 bash[23522]: audit 2024-04-07T16:08:59.863013+0000 mon.a (mon.0) 454 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:09:00.265 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:08:59 smithi144 bash[23522]: audit 2024-04-07T16:08:59.875016+0000 mon.a (mon.0) 455 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:09:00.444 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:08:59 smithi100 bash[21346]: cluster 2024-04-07T16:08:58.231112+0000 mgr.y (mgr.14152) 216 : cluster 0 pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-07T16:09:00.444 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:08:59 smithi100 bash[21346]: audit 2024-04-07T16:08:59.176968+0000 mon.b (mon.1) 11 : audit 1 from='osd.4 [v2:172.21.15.144:6800/974906744,v1:172.21.15.144:6801/974906744]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-07T16:09:00.444 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:08:59 smithi100 bash[21346]: audit 2024-04-07T16:08:59.179149+0000 mon.a (mon.0) 450 : audit 1 from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-07T16:09:00.444 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:08:59 smithi100 bash[21346]: audit 2024-04-07T16:08:59.848326+0000 mon.a (mon.0) 451 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:09:00.444 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:08:59 smithi100 bash[21346]: audit 2024-04-07T16:08:59.859129+0000 mon.a (mon.0) 452 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:09:00.444 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:08:59 smithi100 bash[21346]: audit 2024-04-07T16:08:59.861132+0000 mon.a (mon.0) 453 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:09:00.444 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:08:59 smithi100 bash[21346]: audit 2024-04-07T16:08:59.863013+0000 mon.a (mon.0) 454 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:09:00.444 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:08:59 smithi100 bash[21346]: audit 2024-04-07T16:08:59.875016+0000 mon.a (mon.0) 455 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:09:00.444 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:08:59 smithi100 bash[31660]: cluster 2024-04-07T16:08:58.231112+0000 mgr.y (mgr.14152) 216 : cluster 0 pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-07T16:09:00.445 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:08:59 smithi100 bash[31660]: audit 2024-04-07T16:08:59.176968+0000 mon.b (mon.1) 11 : audit 1 from='osd.4 [v2:172.21.15.144:6800/974906744,v1:172.21.15.144:6801/974906744]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-07T16:09:00.445 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:08:59 smithi100 bash[31660]: audit 2024-04-07T16:08:59.179149+0000 mon.a (mon.0) 450 : audit 1 from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-07T16:09:00.445 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:08:59 smithi100 bash[31660]: audit 2024-04-07T16:08:59.848326+0000 mon.a (mon.0) 451 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:09:00.445 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:08:59 smithi100 bash[31660]: audit 2024-04-07T16:08:59.859129+0000 mon.a (mon.0) 452 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:09:00.445 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:08:59 smithi100 bash[31660]: audit 2024-04-07T16:08:59.861132+0000 mon.a (mon.0) 453 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:09:00.445 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:08:59 smithi100 bash[31660]: audit 2024-04-07T16:08:59.863013+0000 mon.a (mon.0) 454 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:09:00.445 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:08:59 smithi100 bash[31660]: audit 2024-04-07T16:08:59.875016+0000 mon.a (mon.0) 455 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:09:01.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:09:00 smithi144 bash[23522]: audit 2024-04-07T16:08:59.970049+0000 mon.b (mon.1) 12 : audit 1 from='osd.4 [v2:172.21.15.144:6800/974906744,v1:172.21.15.144:6801/974906744]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-04-07T16:09:01.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:09:00 smithi144 bash[23522]: audit 2024-04-07T16:08:59.970239+0000 mon.a (mon.0) 456 : audit 1 from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-07T16:09:01.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:09:00 smithi144 bash[23522]: cluster 2024-04-07T16:08:59.970406+0000 mon.a (mon.0) 457 : cluster 0 osdmap e28: 5 total, 4 up, 5 in 2024-04-07T16:09:01.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:09:00 smithi144 bash[23522]: audit 2024-04-07T16:08:59.971244+0000 mon.a (mon.0) 458 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-07T16:09:01.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:09:00 smithi144 bash[23522]: audit 2024-04-07T16:08:59.983206+0000 mon.a (mon.0) 459 : audit 1 from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-04-07T16:09:01.444 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:09:00 smithi100 bash[21346]: audit 2024-04-07T16:08:59.970049+0000 mon.b (mon.1) 12 : audit 1 from='osd.4 [v2:172.21.15.144:6800/974906744,v1:172.21.15.144:6801/974906744]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-04-07T16:09:01.444 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:09:00 smithi100 bash[21346]: audit 2024-04-07T16:08:59.970239+0000 mon.a (mon.0) 456 : audit 1 from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-07T16:09:01.444 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:09:00 smithi100 bash[21346]: cluster 2024-04-07T16:08:59.970406+0000 mon.a (mon.0) 457 : cluster 0 osdmap e28: 5 total, 4 up, 5 in 2024-04-07T16:09:01.444 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:09:00 smithi100 bash[21346]: audit 2024-04-07T16:08:59.971244+0000 mon.a (mon.0) 458 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-07T16:09:01.444 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:09:00 smithi100 bash[21346]: audit 2024-04-07T16:08:59.983206+0000 mon.a (mon.0) 459 : audit 1 from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-04-07T16:09:01.444 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:09:00 smithi100 bash[31660]: audit 2024-04-07T16:08:59.970049+0000 mon.b (mon.1) 12 : audit 1 from='osd.4 [v2:172.21.15.144:6800/974906744,v1:172.21.15.144:6801/974906744]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-04-07T16:09:01.444 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:09:00 smithi100 bash[31660]: audit 2024-04-07T16:08:59.970239+0000 mon.a (mon.0) 456 : audit 1 from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-07T16:09:01.444 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:09:00 smithi100 bash[31660]: cluster 2024-04-07T16:08:59.970406+0000 mon.a (mon.0) 457 : cluster 0 osdmap e28: 5 total, 4 up, 5 in 2024-04-07T16:09:01.444 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:09:00 smithi100 bash[31660]: audit 2024-04-07T16:08:59.971244+0000 mon.a (mon.0) 458 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-07T16:09:01.444 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:09:00 smithi100 bash[31660]: audit 2024-04-07T16:08:59.983206+0000 mon.a (mon.0) 459 : audit 1 from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-04-07T16:09:02.281 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:09:01 smithi144 bash[23522]: cluster 2024-04-07T16:09:00.231622+0000 mgr.y (mgr.14152) 217 : cluster 0 pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-07T16:09:02.281 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:09:01 smithi144 bash[23522]: audit 2024-04-07T16:09:00.997770+0000 mon.a (mon.0) 460 : audit 1 from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]': finished 2024-04-07T16:09:02.282 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:09:01 smithi144 bash[23522]: cluster 2024-04-07T16:09:00.997974+0000 mon.a (mon.0) 461 : cluster 0 osdmap e29: 5 total, 4 up, 5 in 2024-04-07T16:09:02.282 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:09:01 smithi144 bash[23522]: audit 2024-04-07T16:09:00.998430+0000 mon.a (mon.0) 462 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-07T16:09:02.282 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:09:01 smithi144 bash[23522]: audit 2024-04-07T16:09:01.007753+0000 mon.a (mon.0) 463 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-07T16:09:02.282 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:09:01 smithi144 bash[23522]: audit 2024-04-07T16:09:01.533750+0000 mon.a (mon.0) 464 : audit 1 from='osd.4 ' entity='osd.4' 2024-04-07T16:09:02.444 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:09:01 smithi100 bash[21346]: cluster 2024-04-07T16:09:00.231622+0000 mgr.y (mgr.14152) 217 : cluster 0 pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-07T16:09:02.444 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:09:01 smithi100 bash[21346]: audit 2024-04-07T16:09:00.997770+0000 mon.a (mon.0) 460 : audit 1 from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]': finished 2024-04-07T16:09:02.444 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:09:01 smithi100 bash[21346]: cluster 2024-04-07T16:09:00.997974+0000 mon.a (mon.0) 461 : cluster 0 osdmap e29: 5 total, 4 up, 5 in 2024-04-07T16:09:02.444 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:09:01 smithi100 bash[21346]: audit 2024-04-07T16:09:00.998430+0000 mon.a (mon.0) 462 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-07T16:09:02.444 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:09:01 smithi100 bash[21346]: audit 2024-04-07T16:09:01.007753+0000 mon.a (mon.0) 463 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-07T16:09:02.444 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:09:01 smithi100 bash[21346]: audit 2024-04-07T16:09:01.533750+0000 mon.a (mon.0) 464 : audit 1 from='osd.4 ' entity='osd.4' 2024-04-07T16:09:02.444 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:09:01 smithi100 bash[31660]: cluster 2024-04-07T16:09:00.231622+0000 mgr.y (mgr.14152) 217 : cluster 0 pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-07T16:09:02.444 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:09:01 smithi100 bash[31660]: audit 2024-04-07T16:09:00.997770+0000 mon.a (mon.0) 460 : audit 1 from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]': finished 2024-04-07T16:09:02.444 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:09:01 smithi100 bash[31660]: cluster 2024-04-07T16:09:00.997974+0000 mon.a (mon.0) 461 : cluster 0 osdmap e29: 5 total, 4 up, 5 in 2024-04-07T16:09:02.444 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:09:01 smithi100 bash[31660]: audit 2024-04-07T16:09:00.998430+0000 mon.a (mon.0) 462 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-07T16:09:02.445 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:09:01 smithi100 bash[31660]: audit 2024-04-07T16:09:01.007753+0000 mon.a (mon.0) 463 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-07T16:09:02.445 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:09:01 smithi100 bash[31660]: audit 2024-04-07T16:09:01.533750+0000 mon.a (mon.0) 464 : audit 1 from='osd.4 ' entity='osd.4' 2024-04-07T16:09:02.903 INFO:teuthology.orchestra.run.smithi144.stdout:Created osd(s) 4 on host 'smithi144' 2024-04-07T16:09:03.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:09:02 smithi144 bash[23522]: cluster 2024-04-07T16:09:00.218521+0000 osd.4 (osd.4) 1 : cluster 0 purged_snaps scrub starts 2024-04-07T16:09:03.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:09:02 smithi144 bash[23522]: cluster 2024-04-07T16:09:00.218613+0000 osd.4 (osd.4) 2 : cluster 0 purged_snaps scrub ok 2024-04-07T16:09:03.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:09:02 smithi144 bash[23522]: audit 2024-04-07T16:09:02.006735+0000 mon.a (mon.0) 465 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-07T16:09:03.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:09:02 smithi144 bash[23522]: cluster 2024-04-07T16:09:02.479414+0000 mon.a (mon.0) 466 : cluster 1 osd.4 [v2:172.21.15.144:6800/974906744,v1:172.21.15.144:6801/974906744] boot 2024-04-07T16:09:03.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:09:02 smithi144 bash[23522]: cluster 2024-04-07T16:09:02.479619+0000 mon.a (mon.0) 467 : cluster 0 osdmap e30: 5 total, 5 up, 5 in 2024-04-07T16:09:03.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:09:02 smithi144 bash[23522]: audit 2024-04-07T16:09:02.481212+0000 mon.a (mon.0) 468 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-07T16:09:03.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:09:02 smithi144 bash[23522]: audit 2024-04-07T16:09:02.886484+0000 mon.a (mon.0) 469 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:09:03.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:09:02 smithi144 bash[23522]: audit 2024-04-07T16:09:02.895445+0000 mon.a (mon.0) 470 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:09:03.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:09:02 smithi144 bash[23522]: audit 2024-04-07T16:09:02.918811+0000 mon.a (mon.0) 471 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:09:03.444 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:09:02 smithi100 bash[21346]: cluster 2024-04-07T16:09:00.218521+0000 osd.4 (osd.4) 1 : cluster 0 purged_snaps scrub starts 2024-04-07T16:09:03.444 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:09:02 smithi100 bash[21346]: cluster 2024-04-07T16:09:00.218613+0000 osd.4 (osd.4) 2 : cluster 0 purged_snaps scrub ok 2024-04-07T16:09:03.444 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:09:02 smithi100 bash[21346]: audit 2024-04-07T16:09:02.006735+0000 mon.a (mon.0) 465 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-07T16:09:03.444 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:09:02 smithi100 bash[21346]: cluster 2024-04-07T16:09:02.479414+0000 mon.a (mon.0) 466 : cluster 1 osd.4 [v2:172.21.15.144:6800/974906744,v1:172.21.15.144:6801/974906744] boot 2024-04-07T16:09:03.444 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:09:03 smithi100 bash[21346]: cluster 2024-04-07T16:09:02.479619+0000 mon.a (mon.0) 467 : cluster 0 osdmap e30: 5 total, 5 up, 5 in 2024-04-07T16:09:03.444 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:09:03 smithi100 bash[21346]: audit 2024-04-07T16:09:02.481212+0000 mon.a (mon.0) 468 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-07T16:09:03.445 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:09:03 smithi100 bash[21346]: audit 2024-04-07T16:09:02.886484+0000 mon.a (mon.0) 469 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:09:03.445 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:09:03 smithi100 bash[21346]: audit 2024-04-07T16:09:02.895445+0000 mon.a (mon.0) 470 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:09:03.445 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:09:03 smithi100 bash[21346]: audit 2024-04-07T16:09:02.918811+0000 mon.a (mon.0) 471 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:09:03.445 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:09:03 smithi100 bash[31660]: cluster 2024-04-07T16:09:00.218521+0000 osd.4 (osd.4) 1 : cluster 0 purged_snaps scrub starts 2024-04-07T16:09:03.445 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:09:03 smithi100 bash[31660]: cluster 2024-04-07T16:09:00.218613+0000 osd.4 (osd.4) 2 : cluster 0 purged_snaps scrub ok 2024-04-07T16:09:03.445 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:09:03 smithi100 bash[31660]: audit 2024-04-07T16:09:02.006735+0000 mon.a (mon.0) 465 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-07T16:09:03.445 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:09:03 smithi100 bash[31660]: cluster 2024-04-07T16:09:02.479414+0000 mon.a (mon.0) 466 : cluster 1 osd.4 [v2:172.21.15.144:6800/974906744,v1:172.21.15.144:6801/974906744] boot 2024-04-07T16:09:03.445 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:09:03 smithi100 bash[31660]: cluster 2024-04-07T16:09:02.479619+0000 mon.a (mon.0) 467 : cluster 0 osdmap e30: 5 total, 5 up, 5 in 2024-04-07T16:09:03.445 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:09:03 smithi100 bash[31660]: audit 2024-04-07T16:09:02.481212+0000 mon.a (mon.0) 468 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-07T16:09:03.445 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:09:03 smithi100 bash[31660]: audit 2024-04-07T16:09:02.886484+0000 mon.a (mon.0) 469 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:09:03.445 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:09:03 smithi100 bash[31660]: audit 2024-04-07T16:09:02.895445+0000 mon.a (mon.0) 470 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:09:03.445 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:09:03 smithi100 bash[31660]: audit 2024-04-07T16:09:02.918811+0000 mon.a (mon.0) 471 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:09:03.463 DEBUG:teuthology.orchestra.run.smithi144:osd.4> sudo journalctl -f -n 0 -u ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@osd.4.service 2024-04-07T16:09:03.466 INFO:tasks.cephadm:Deploying osd.5 on smithi144 with /dev/vg_nvme/lv_3... 2024-04-07T16:09:03.466 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0702e3a8-f4f8-11ee-b649-cb9ed24678a4 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-07T16:09:04.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:09:04 smithi144 bash[23522]: cluster 2024-04-07T16:09:02.232172+0000 mgr.y (mgr.14152) 218 : cluster 0 pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-07T16:09:04.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:09:04 smithi144 bash[23522]: cluster 2024-04-07T16:09:03.487180+0000 mon.a (mon.0) 472 : cluster 0 osdmap e31: 5 total, 5 up, 5 in 2024-04-07T16:09:04.443 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:09:04 smithi100 bash[21346]: cluster 2024-04-07T16:09:02.232172+0000 mgr.y (mgr.14152) 218 : cluster 0 pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-07T16:09:04.444 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:09:04 smithi100 bash[21346]: cluster 2024-04-07T16:09:03.487180+0000 mon.a (mon.0) 472 : cluster 0 osdmap e31: 5 total, 5 up, 5 in 2024-04-07T16:09:04.444 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:09:04 smithi100 bash[31660]: cluster 2024-04-07T16:09:02.232172+0000 mgr.y (mgr.14152) 218 : cluster 0 pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-07T16:09:04.444 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:09:04 smithi100 bash[31660]: cluster 2024-04-07T16:09:03.487180+0000 mon.a (mon.0) 472 : cluster 0 osdmap e31: 5 total, 5 up, 5 in 2024-04-07T16:09:05.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:09:05 smithi144 bash[23522]: cluster 2024-04-07T16:09:04.232737+0000 mgr.y (mgr.14152) 219 : cluster 0 pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-07T16:09:05.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:09:05 smithi144 bash[23522]: cluster 2024-04-07T16:09:04.491552+0000 mon.a (mon.0) 473 : cluster 0 osdmap e32: 5 total, 5 up, 5 in 2024-04-07T16:09:05.943 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:09:05 smithi100 bash[21346]: cluster 2024-04-07T16:09:04.232737+0000 mgr.y (mgr.14152) 219 : cluster 0 pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-07T16:09:05.943 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:09:05 smithi100 bash[21346]: cluster 2024-04-07T16:09:04.491552+0000 mon.a (mon.0) 473 : cluster 0 osdmap e32: 5 total, 5 up, 5 in 2024-04-07T16:09:05.944 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:09:05 smithi100 bash[31660]: cluster 2024-04-07T16:09:04.232737+0000 mgr.y (mgr.14152) 219 : cluster 0 pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-07T16:09:05.944 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:09:05 smithi100 bash[31660]: cluster 2024-04-07T16:09:04.491552+0000 mon.a (mon.0) 473 : cluster 0 osdmap e32: 5 total, 5 up, 5 in 2024-04-07T16:09:06.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:09:06 smithi144 bash[23522]: cluster 2024-04-07T16:09:05.494273+0000 mon.a (mon.0) 474 : cluster 0 osdmap e33: 5 total, 5 up, 5 in 2024-04-07T16:09:06.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:09:06 smithi144 bash[23522]: cluster 2024-04-07T16:09:06.492268+0000 mon.a (mon.0) 475 : cluster 0 osdmap e34: 5 total, 5 up, 5 in 2024-04-07T16:09:06.943 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:09:06 smithi100 bash[31660]: cluster 2024-04-07T16:09:05.494273+0000 mon.a (mon.0) 474 : cluster 0 osdmap e33: 5 total, 5 up, 5 in 2024-04-07T16:09:06.944 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:09:06 smithi100 bash[31660]: cluster 2024-04-07T16:09:06.492268+0000 mon.a (mon.0) 475 : cluster 0 osdmap e34: 5 total, 5 up, 5 in 2024-04-07T16:09:06.944 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:09:06 smithi100 bash[21346]: cluster 2024-04-07T16:09:05.494273+0000 mon.a (mon.0) 474 : cluster 0 osdmap e33: 5 total, 5 up, 5 in 2024-04-07T16:09:06.944 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:09:06 smithi100 bash[21346]: cluster 2024-04-07T16:09:06.492268+0000 mon.a (mon.0) 475 : cluster 0 osdmap e34: 5 total, 5 up, 5 in 2024-04-07T16:09:07.743 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/mon.b/config 2024-04-07T16:09:07.764 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:09:07 smithi144 bash[23522]: cluster 2024-04-07T16:09:06.233288+0000 mgr.y (mgr.14152) 220 : cluster 0 pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-07T16:09:07.943 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:09:07 smithi100 bash[21346]: cluster 2024-04-07T16:09:06.233288+0000 mgr.y (mgr.14152) 220 : cluster 0 pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-07T16:09:07.943 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:09:07 smithi100 bash[31660]: cluster 2024-04-07T16:09:06.233288+0000 mgr.y (mgr.14152) 220 : cluster 0 pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-07T16:09:09.762 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:09:09 smithi144 bash[23522]: cluster 2024-04-07T16:09:08.233852+0000 mgr.y (mgr.14152) 221 : cluster 0 pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 140 MiB used, 447 GiB / 447 GiB avail; 121 KiB/s, 0 objects/s recovering 2024-04-07T16:09:09.943 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:09:09 smithi100 bash[21346]: cluster 2024-04-07T16:09:08.233852+0000 mgr.y (mgr.14152) 221 : cluster 0 pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 140 MiB used, 447 GiB / 447 GiB avail; 121 KiB/s, 0 objects/s recovering 2024-04-07T16:09:09.943 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:09:09 smithi100 bash[31660]: cluster 2024-04-07T16:09:08.233852+0000 mgr.y (mgr.14152) 221 : cluster 0 pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 140 MiB used, 447 GiB / 447 GiB avail; 121 KiB/s, 0 objects/s recovering 2024-04-07T16:09:11.394 INFO:teuthology.orchestra.run.smithi144.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-07T16:09:11.394 INFO:teuthology.orchestra.run.smithi144.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-07T16:09:11.395 INFO:teuthology.orchestra.run.smithi144.stderr: stderr: 10+0 records in 2024-04-07T16:09:11.395 INFO:teuthology.orchestra.run.smithi144.stderr:10+0 records out 2024-04-07T16:09:11.395 INFO:teuthology.orchestra.run.smithi144.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0187938 s, 558 MB/s 2024-04-07T16:09:11.395 INFO:teuthology.orchestra.run.smithi144.stderr:--> Zapping successful for: 2024-04-07T16:09:11.726 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:09:11 smithi144 bash[23522]: cluster 2024-04-07T16:09:10.234383+0000 mgr.y (mgr.14152) 222 : cluster 0 pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 140 MiB used, 447 GiB / 447 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-07T16:09:11.943 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:09:11 smithi100 bash[21346]: cluster 2024-04-07T16:09:10.234383+0000 mgr.y (mgr.14152) 222 : cluster 0 pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 140 MiB used, 447 GiB / 447 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-07T16:09:11.944 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:09:11 smithi100 bash[31660]: cluster 2024-04-07T16:09:10.234383+0000 mgr.y (mgr.14152) 222 : cluster 0 pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 140 MiB used, 447 GiB / 447 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-07T16:09:12.061 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0702e3a8-f4f8-11ee-b649-cb9ed24678a4 -- ceph orch daemon add osd smithi144:vg_nvme/lv_3 2024-04-07T16:09:13.694 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:09:13 smithi100 bash[21346]: cluster 2024-04-07T16:09:12.234956+0000 mgr.y (mgr.14152) 223 : cluster 0 pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-04-07T16:09:13.694 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:09:13 smithi100 bash[21346]: cephadm 2024-04-07T16:09:12.411402+0000 mgr.y (mgr.14152) 224 : cephadm 1 Detected new or changed devices on smithi144 2024-04-07T16:09:13.694 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:09:13 smithi100 bash[21346]: audit 2024-04-07T16:09:12.420329+0000 mon.a (mon.0) 476 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:09:13.694 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:09:13 smithi100 bash[21346]: audit 2024-04-07T16:09:12.429821+0000 mon.a (mon.0) 477 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:09:13.694 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:09:13 smithi100 bash[21346]: audit 2024-04-07T16:09:12.432079+0000 mon.a (mon.0) 478 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-07T16:09:13.694 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:09:13 smithi100 bash[21346]: cephadm 2024-04-07T16:09:12.433422+0000 mgr.y (mgr.14152) 225 : cephadm 1 Adjusting osd_memory_target on smithi144 to 17265M 2024-04-07T16:09:13.694 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:09:13 smithi100 bash[21346]: audit 2024-04-07T16:09:12.442226+0000 mon.a (mon.0) 479 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:09:13.694 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:09:13 smithi100 bash[21346]: audit 2024-04-07T16:09:12.444479+0000 mon.a (mon.0) 480 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:09:13.694 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:09:13 smithi100 bash[21346]: audit 2024-04-07T16:09:12.446323+0000 mon.a (mon.0) 481 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:09:13.694 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:09:13 smithi100 bash[21346]: audit 2024-04-07T16:09:12.456555+0000 mon.a (mon.0) 482 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:09:13.695 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:09:13 smithi100 bash[31660]: cluster 2024-04-07T16:09:12.234956+0000 mgr.y (mgr.14152) 223 : cluster 0 pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-04-07T16:09:13.695 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:09:13 smithi100 bash[31660]: cephadm 2024-04-07T16:09:12.411402+0000 mgr.y (mgr.14152) 224 : cephadm 1 Detected new or changed devices on smithi144 2024-04-07T16:09:13.695 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:09:13 smithi100 bash[31660]: audit 2024-04-07T16:09:12.420329+0000 mon.a (mon.0) 476 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:09:13.695 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:09:13 smithi100 bash[31660]: audit 2024-04-07T16:09:12.429821+0000 mon.a (mon.0) 477 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:09:13.695 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:09:13 smithi100 bash[31660]: audit 2024-04-07T16:09:12.432079+0000 mon.a (mon.0) 478 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-07T16:09:13.695 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:09:13 smithi100 bash[31660]: cephadm 2024-04-07T16:09:12.433422+0000 mgr.y (mgr.14152) 225 : cephadm 1 Adjusting osd_memory_target on smithi144 to 17265M 2024-04-07T16:09:13.695 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:09:13 smithi100 bash[31660]: audit 2024-04-07T16:09:12.442226+0000 mon.a (mon.0) 479 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:09:13.695 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:09:13 smithi100 bash[31660]: audit 2024-04-07T16:09:12.444479+0000 mon.a (mon.0) 480 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:09:13.695 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:09:13 smithi100 bash[31660]: audit 2024-04-07T16:09:12.446323+0000 mon.a (mon.0) 481 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:09:13.695 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:09:13 smithi100 bash[31660]: audit 2024-04-07T16:09:12.456555+0000 mon.a (mon.0) 482 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:09:13.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:09:13 smithi144 bash[23522]: cluster 2024-04-07T16:09:12.234956+0000 mgr.y (mgr.14152) 223 : cluster 0 pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-04-07T16:09:13.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:09:13 smithi144 bash[23522]: cephadm 2024-04-07T16:09:12.411402+0000 mgr.y (mgr.14152) 224 : cephadm 1 Detected new or changed devices on smithi144 2024-04-07T16:09:13.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:09:13 smithi144 bash[23522]: audit 2024-04-07T16:09:12.420329+0000 mon.a (mon.0) 476 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:09:13.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:09:13 smithi144 bash[23522]: audit 2024-04-07T16:09:12.429821+0000 mon.a (mon.0) 477 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:09:13.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:09:13 smithi144 bash[23522]: audit 2024-04-07T16:09:12.432079+0000 mon.a (mon.0) 478 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-07T16:09:13.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:09:13 smithi144 bash[23522]: cephadm 2024-04-07T16:09:12.433422+0000 mgr.y (mgr.14152) 225 : cephadm 1 Adjusting osd_memory_target on smithi144 to 17265M 2024-04-07T16:09:13.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:09:13 smithi144 bash[23522]: audit 2024-04-07T16:09:12.442226+0000 mon.a (mon.0) 479 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:09:13.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:09:13 smithi144 bash[23522]: audit 2024-04-07T16:09:12.444479+0000 mon.a (mon.0) 480 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:09:13.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:09:13 smithi144 bash[23522]: audit 2024-04-07T16:09:12.446323+0000 mon.a (mon.0) 481 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:09:13.860 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:09:13 smithi144 bash[23522]: audit 2024-04-07T16:09:12.456555+0000 mon.a (mon.0) 482 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:09:15.693 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:09:15 smithi100 bash[21346]: cluster 2024-04-07T16:09:14.235620+0000 mgr.y (mgr.14152) 226 : cluster 0 pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-04-07T16:09:15.693 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:09:15 smithi100 bash[31660]: cluster 2024-04-07T16:09:14.235620+0000 mgr.y (mgr.14152) 226 : cluster 0 pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-04-07T16:09:15.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:09:15 smithi144 bash[23522]: cluster 2024-04-07T16:09:14.235620+0000 mgr.y (mgr.14152) 226 : cluster 0 pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-04-07T16:09:16.734 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/mon.b/config 2024-04-07T16:09:17.693 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:09:17 smithi100 bash[21346]: cluster 2024-04-07T16:09:16.236180+0000 mgr.y (mgr.14152) 227 : cluster 0 pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-07T16:09:17.693 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:09:17 smithi100 bash[31660]: cluster 2024-04-07T16:09:16.236180+0000 mgr.y (mgr.14152) 227 : cluster 0 pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-07T16:09:17.709 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:09:17 smithi144 bash[23522]: cluster 2024-04-07T16:09:16.236180+0000 mgr.y (mgr.14152) 227 : cluster 0 pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-07T16:09:19.693 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:09:19 smithi100 bash[21346]: cluster 2024-04-07T16:09:18.236730+0000 mgr.y (mgr.14152) 228 : cluster 0 pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-04-07T16:09:19.722 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:09:19 smithi100 bash[21346]: audit 2024-04-07T16:09:19.390343+0000 mon.a (mon.0) 483 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-07T16:09:19.723 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:09:19 smithi100 bash[21346]: audit 2024-04-07T16:09:19.397519+0000 mon.a (mon.0) 484 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-07T16:09:19.723 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:09:19 smithi100 bash[21346]: audit 2024-04-07T16:09:19.399352+0000 mon.a (mon.0) 485 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:09:19.723 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:09:19 smithi100 bash[31660]: cluster 2024-04-07T16:09:18.236730+0000 mgr.y (mgr.14152) 228 : cluster 0 pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-04-07T16:09:19.723 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:09:19 smithi100 bash[31660]: audit 2024-04-07T16:09:19.390343+0000 mon.a (mon.0) 483 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-07T16:09:19.723 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:09:19 smithi100 bash[31660]: audit 2024-04-07T16:09:19.397519+0000 mon.a (mon.0) 484 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-07T16:09:19.723 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:09:19 smithi100 bash[31660]: audit 2024-04-07T16:09:19.399352+0000 mon.a (mon.0) 485 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:09:19.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:09:19 smithi144 bash[23522]: cluster 2024-04-07T16:09:18.236730+0000 mgr.y (mgr.14152) 228 : cluster 0 pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-04-07T16:09:19.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:09:19 smithi144 bash[23522]: audit 2024-04-07T16:09:19.390343+0000 mon.a (mon.0) 483 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-07T16:09:19.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:09:19 smithi144 bash[23522]: audit 2024-04-07T16:09:19.397519+0000 mon.a (mon.0) 484 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-07T16:09:19.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:09:19 smithi144 bash[23522]: audit 2024-04-07T16:09:19.399352+0000 mon.a (mon.0) 485 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:09:20.693 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:09:20 smithi100 bash[21346]: audit 2024-04-07T16:09:19.383492+0000 mgr.y (mgr.14152) 229 : audit 0 from='client.24205 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:09:20.694 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:09:20 smithi100 bash[31660]: audit 2024-04-07T16:09:19.383492+0000 mgr.y (mgr.14152) 229 : audit 0 from='client.24205 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:09:20.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:09:20 smithi144 bash[23522]: audit 2024-04-07T16:09:19.383492+0000 mgr.y (mgr.14152) 229 : audit 0 from='client.24205 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:09:21.693 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:09:21 smithi100 bash[21346]: cluster 2024-04-07T16:09:20.237281+0000 mgr.y (mgr.14152) 230 : cluster 0 pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-07T16:09:21.693 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:09:21 smithi100 bash[31660]: cluster 2024-04-07T16:09:20.237281+0000 mgr.y (mgr.14152) 230 : cluster 0 pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-07T16:09:21.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:09:21 smithi144 bash[23522]: cluster 2024-04-07T16:09:20.237281+0000 mgr.y (mgr.14152) 230 : cluster 0 pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-07T16:09:23.693 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:09:23 smithi100 bash[31660]: cluster 2024-04-07T16:09:22.237829+0000 mgr.y (mgr.14152) 231 : cluster 0 pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-07T16:09:23.693 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:09:23 smithi100 bash[21346]: cluster 2024-04-07T16:09:22.237829+0000 mgr.y (mgr.14152) 231 : cluster 0 pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-07T16:09:23.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:09:23 smithi144 bash[23522]: cluster 2024-04-07T16:09:22.237829+0000 mgr.y (mgr.14152) 231 : cluster 0 pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-07T16:09:25.548 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:09:25 smithi144 bash[23522]: cluster 2024-04-07T16:09:24.238345+0000 mgr.y (mgr.14152) 232 : cluster 0 pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-07T16:09:25.693 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:09:25 smithi100 bash[31660]: cluster 2024-04-07T16:09:24.238345+0000 mgr.y (mgr.14152) 232 : cluster 0 pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-07T16:09:25.693 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:09:25 smithi100 bash[21346]: cluster 2024-04-07T16:09:24.238345+0000 mgr.y (mgr.14152) 232 : cluster 0 pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-07T16:09:27.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:09:27 smithi144 bash[23522]: cluster 2024-04-07T16:09:26.238866+0000 mgr.y (mgr.14152) 233 : cluster 0 pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-07T16:09:27.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:09:27 smithi144 bash[23522]: audit 2024-04-07T16:09:27.194212+0000 mon.b (mon.1) 13 : audit 1 from='client.? 172.21.15.144:0/2373279442' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "54789cc8-1c3c-46a4-a92e-8881564338db"}]: dispatch 2024-04-07T16:09:27.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:09:27 smithi144 bash[23522]: audit 2024-04-07T16:09:27.196883+0000 mon.a (mon.0) 486 : audit 1 from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "54789cc8-1c3c-46a4-a92e-8881564338db"}]: dispatch 2024-04-07T16:09:27.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:09:27 smithi144 bash[23522]: audit 2024-04-07T16:09:27.205587+0000 mon.a (mon.0) 487 : audit 1 from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "54789cc8-1c3c-46a4-a92e-8881564338db"}]': finished 2024-04-07T16:09:27.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:09:27 smithi144 bash[23522]: cluster 2024-04-07T16:09:27.205711+0000 mon.a (mon.0) 488 : cluster 0 osdmap e35: 6 total, 5 up, 6 in 2024-04-07T16:09:27.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:09:27 smithi144 bash[23522]: audit 2024-04-07T16:09:27.206029+0000 mon.a (mon.0) 489 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-07T16:09:27.943 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:09:27 smithi100 bash[21346]: cluster 2024-04-07T16:09:26.238866+0000 mgr.y (mgr.14152) 233 : cluster 0 pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-07T16:09:27.943 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:09:27 smithi100 bash[21346]: audit 2024-04-07T16:09:27.194212+0000 mon.b (mon.1) 13 : audit 1 from='client.? 172.21.15.144:0/2373279442' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "54789cc8-1c3c-46a4-a92e-8881564338db"}]: dispatch 2024-04-07T16:09:27.943 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:09:27 smithi100 bash[21346]: audit 2024-04-07T16:09:27.196883+0000 mon.a (mon.0) 486 : audit 1 from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "54789cc8-1c3c-46a4-a92e-8881564338db"}]: dispatch 2024-04-07T16:09:27.943 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:09:27 smithi100 bash[21346]: audit 2024-04-07T16:09:27.205587+0000 mon.a (mon.0) 487 : audit 1 from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "54789cc8-1c3c-46a4-a92e-8881564338db"}]': finished 2024-04-07T16:09:27.944 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:09:27 smithi100 bash[21346]: cluster 2024-04-07T16:09:27.205711+0000 mon.a (mon.0) 488 : cluster 0 osdmap e35: 6 total, 5 up, 6 in 2024-04-07T16:09:27.944 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:09:27 smithi100 bash[21346]: audit 2024-04-07T16:09:27.206029+0000 mon.a (mon.0) 489 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-07T16:09:27.944 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:09:27 smithi100 bash[31660]: cluster 2024-04-07T16:09:26.238866+0000 mgr.y (mgr.14152) 233 : cluster 0 pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-07T16:09:27.944 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:09:27 smithi100 bash[31660]: audit 2024-04-07T16:09:27.194212+0000 mon.b (mon.1) 13 : audit 1 from='client.? 172.21.15.144:0/2373279442' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "54789cc8-1c3c-46a4-a92e-8881564338db"}]: dispatch 2024-04-07T16:09:27.944 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:09:27 smithi100 bash[31660]: audit 2024-04-07T16:09:27.196883+0000 mon.a (mon.0) 486 : audit 1 from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "54789cc8-1c3c-46a4-a92e-8881564338db"}]: dispatch 2024-04-07T16:09:27.944 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:09:27 smithi100 bash[31660]: audit 2024-04-07T16:09:27.205587+0000 mon.a (mon.0) 487 : audit 1 from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "54789cc8-1c3c-46a4-a92e-8881564338db"}]': finished 2024-04-07T16:09:27.944 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:09:27 smithi100 bash[31660]: cluster 2024-04-07T16:09:27.205711+0000 mon.a (mon.0) 488 : cluster 0 osdmap e35: 6 total, 5 up, 6 in 2024-04-07T16:09:27.944 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:09:27 smithi100 bash[31660]: audit 2024-04-07T16:09:27.206029+0000 mon.a (mon.0) 489 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-07T16:09:28.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:09:28 smithi144 bash[23522]: audit 2024-04-07T16:09:27.904018+0000 mon.b (mon.1) 14 : audit 0 from='client.? 172.21.15.144:0/4205291335' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-07T16:09:28.943 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:09:28 smithi100 bash[21346]: audit 2024-04-07T16:09:27.904018+0000 mon.b (mon.1) 14 : audit 0 from='client.? 172.21.15.144:0/4205291335' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-07T16:09:28.943 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:09:28 smithi100 bash[31660]: audit 2024-04-07T16:09:27.904018+0000 mon.b (mon.1) 14 : audit 0 from='client.? 172.21.15.144:0/4205291335' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-07T16:09:29.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:09:29 smithi144 bash[23522]: cluster 2024-04-07T16:09:28.239393+0000 mgr.y (mgr.14152) 234 : cluster 0 pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-07T16:09:29.943 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:09:29 smithi100 bash[21346]: cluster 2024-04-07T16:09:28.239393+0000 mgr.y (mgr.14152) 234 : cluster 0 pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-07T16:09:29.943 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:09:29 smithi100 bash[31660]: cluster 2024-04-07T16:09:28.239393+0000 mgr.y (mgr.14152) 234 : cluster 0 pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-07T16:09:31.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:09:31 smithi144 bash[23522]: cluster 2024-04-07T16:09:30.239887+0000 mgr.y (mgr.14152) 235 : cluster 0 pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-07T16:09:31.943 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:09:31 smithi100 bash[21346]: cluster 2024-04-07T16:09:30.239887+0000 mgr.y (mgr.14152) 235 : cluster 0 pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-07T16:09:31.943 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:09:31 smithi100 bash[31660]: cluster 2024-04-07T16:09:30.239887+0000 mgr.y (mgr.14152) 235 : cluster 0 pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-07T16:09:33.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:09:33 smithi144 bash[23522]: cluster 2024-04-07T16:09:32.240422+0000 mgr.y (mgr.14152) 236 : cluster 0 pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-07T16:09:33.943 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:09:33 smithi100 bash[21346]: cluster 2024-04-07T16:09:32.240422+0000 mgr.y (mgr.14152) 236 : cluster 0 pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-07T16:09:33.943 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:09:33 smithi100 bash[31660]: cluster 2024-04-07T16:09:32.240422+0000 mgr.y (mgr.14152) 236 : cluster 0 pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-07T16:09:35.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:09:35 smithi144 bash[23522]: cluster 2024-04-07T16:09:34.240943+0000 mgr.y (mgr.14152) 237 : cluster 0 pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-07T16:09:35.943 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:09:35 smithi100 bash[21346]: cluster 2024-04-07T16:09:34.240943+0000 mgr.y (mgr.14152) 237 : cluster 0 pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-07T16:09:35.943 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:09:35 smithi100 bash[31660]: cluster 2024-04-07T16:09:34.240943+0000 mgr.y (mgr.14152) 237 : cluster 0 pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-07T16:09:37.608 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:09:37 smithi144 bash[23522]: cluster 2024-04-07T16:09:36.241477+0000 mgr.y (mgr.14152) 238 : cluster 0 pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-07T16:09:37.943 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:09:37 smithi100 bash[21346]: cluster 2024-04-07T16:09:36.241477+0000 mgr.y (mgr.14152) 238 : cluster 0 pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-07T16:09:37.943 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:09:37 smithi100 bash[31660]: cluster 2024-04-07T16:09:36.241477+0000 mgr.y (mgr.14152) 238 : cluster 0 pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-07T16:09:39.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:09:39 smithi144 bash[23522]: cluster 2024-04-07T16:09:38.241997+0000 mgr.y (mgr.14152) 239 : cluster 0 pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-07T16:09:39.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:09:39 smithi144 bash[23522]: audit 2024-04-07T16:09:38.936302+0000 mon.a (mon.0) 490 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-07T16:09:39.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:09:39 smithi144 bash[23522]: audit 2024-04-07T16:09:38.938105+0000 mon.a (mon.0) 491 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:09:39.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:09:39 smithi144 bash[23522]: cephadm 2024-04-07T16:09:38.939710+0000 mgr.y (mgr.14152) 240 : cephadm 1 Deploying daemon osd.5 on smithi144 2024-04-07T16:09:39.943 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:09:39 smithi100 bash[21346]: cluster 2024-04-07T16:09:38.241997+0000 mgr.y (mgr.14152) 239 : cluster 0 pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-07T16:09:39.943 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:09:39 smithi100 bash[21346]: audit 2024-04-07T16:09:38.936302+0000 mon.a (mon.0) 490 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-07T16:09:39.943 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:09:39 smithi100 bash[21346]: audit 2024-04-07T16:09:38.938105+0000 mon.a (mon.0) 491 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:09:39.943 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:09:39 smithi100 bash[21346]: cephadm 2024-04-07T16:09:38.939710+0000 mgr.y (mgr.14152) 240 : cephadm 1 Deploying daemon osd.5 on smithi144 2024-04-07T16:09:39.943 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:09:39 smithi100 bash[31660]: cluster 2024-04-07T16:09:38.241997+0000 mgr.y (mgr.14152) 239 : cluster 0 pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-07T16:09:39.943 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:09:39 smithi100 bash[31660]: audit 2024-04-07T16:09:38.936302+0000 mon.a (mon.0) 490 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-07T16:09:39.944 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:09:39 smithi100 bash[31660]: audit 2024-04-07T16:09:38.938105+0000 mon.a (mon.0) 491 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:09:39.944 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:09:39 smithi100 bash[31660]: cephadm 2024-04-07T16:09:38.939710+0000 mgr.y (mgr.14152) 240 : cephadm 1 Deploying daemon osd.5 on smithi144 2024-04-07T16:09:41.663 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:09:41 smithi144 bash[23522]: cluster 2024-04-07T16:09:40.242536+0000 mgr.y (mgr.14152) 241 : cluster 0 pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-07T16:09:41.943 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:09:41 smithi100 bash[21346]: cluster 2024-04-07T16:09:40.242536+0000 mgr.y (mgr.14152) 241 : cluster 0 pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-07T16:09:41.943 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:09:41 smithi100 bash[31660]: cluster 2024-04-07T16:09:40.242536+0000 mgr.y (mgr.14152) 241 : cluster 0 pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-07T16:09:42.648 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:09:42 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:09:42.649 INFO:journalctl@ceph.osd.4.smithi144.stdout:Apr 07 16:09:42 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:09:42.649 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:09:42 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:09:42.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:09:42 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:09:42.937 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:09:42 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:09:42.937 INFO:journalctl@ceph.osd.4.smithi144.stdout:Apr 07 16:09:42 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:09:44.193 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:09:43 smithi100 bash[21346]: cluster 2024-04-07T16:09:42.243103+0000 mgr.y (mgr.14152) 242 : cluster 0 pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-07T16:09:44.193 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:09:43 smithi100 bash[21346]: audit 2024-04-07T16:09:42.943917+0000 mon.a (mon.0) 492 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:09:44.193 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:09:43 smithi100 bash[21346]: audit 2024-04-07T16:09:42.954524+0000 mon.a (mon.0) 493 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:09:44.193 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:09:43 smithi100 bash[21346]: audit 2024-04-07T16:09:42.975832+0000 mon.a (mon.0) 494 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:09:44.193 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:09:43 smithi100 bash[31660]: cluster 2024-04-07T16:09:42.243103+0000 mgr.y (mgr.14152) 242 : cluster 0 pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-07T16:09:44.194 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:09:43 smithi100 bash[31660]: audit 2024-04-07T16:09:42.943917+0000 mon.a (mon.0) 492 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:09:44.194 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:09:43 smithi100 bash[31660]: audit 2024-04-07T16:09:42.954524+0000 mon.a (mon.0) 493 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:09:44.194 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:09:43 smithi100 bash[31660]: audit 2024-04-07T16:09:42.975832+0000 mon.a (mon.0) 494 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:09:44.260 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:09:43 smithi144 bash[23522]: cluster 2024-04-07T16:09:42.243103+0000 mgr.y (mgr.14152) 242 : cluster 0 pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-07T16:09:44.260 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:09:43 smithi144 bash[23522]: audit 2024-04-07T16:09:42.943917+0000 mon.a (mon.0) 492 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:09:44.260 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:09:43 smithi144 bash[23522]: audit 2024-04-07T16:09:42.954524+0000 mon.a (mon.0) 493 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:09:44.260 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:09:43 smithi144 bash[23522]: audit 2024-04-07T16:09:42.975832+0000 mon.a (mon.0) 494 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:09:46.192 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:09:45 smithi100 bash[21346]: cluster 2024-04-07T16:09:44.243670+0000 mgr.y (mgr.14152) 243 : cluster 0 pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-07T16:09:46.193 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:09:45 smithi100 bash[31660]: cluster 2024-04-07T16:09:44.243670+0000 mgr.y (mgr.14152) 243 : cluster 0 pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-07T16:09:46.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:09:45 smithi144 bash[23522]: cluster 2024-04-07T16:09:44.243670+0000 mgr.y (mgr.14152) 243 : cluster 0 pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-07T16:09:48.108 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:09:47 smithi144 bash[23522]: cluster 2024-04-07T16:09:46.244200+0000 mgr.y (mgr.14152) 244 : cluster 0 pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-07T16:09:48.193 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:09:47 smithi100 bash[21346]: cluster 2024-04-07T16:09:46.244200+0000 mgr.y (mgr.14152) 244 : cluster 0 pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-07T16:09:48.193 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:09:47 smithi100 bash[31660]: cluster 2024-04-07T16:09:46.244200+0000 mgr.y (mgr.14152) 244 : cluster 0 pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-07T16:09:49.443 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:09:49 smithi100 bash[21346]: audit 2024-04-07T16:09:48.120275+0000 mon.a (mon.0) 495 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:09:49.443 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:09:49 smithi100 bash[21346]: audit 2024-04-07T16:09:48.129900+0000 mon.a (mon.0) 496 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:09:49.443 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:09:49 smithi100 bash[21346]: audit 2024-04-07T16:09:48.656621+0000 mon.a (mon.0) 497 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:09:49.443 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:09:49 smithi100 bash[21346]: audit 2024-04-07T16:09:48.658460+0000 mon.a (mon.0) 498 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:09:49.443 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:09:49 smithi100 bash[21346]: audit 2024-04-07T16:09:48.668782+0000 mon.a (mon.0) 499 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:09:49.443 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:09:49 smithi100 bash[31660]: audit 2024-04-07T16:09:48.120275+0000 mon.a (mon.0) 495 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:09:49.444 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:09:49 smithi100 bash[31660]: audit 2024-04-07T16:09:48.129900+0000 mon.a (mon.0) 496 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:09:49.444 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:09:49 smithi100 bash[31660]: audit 2024-04-07T16:09:48.656621+0000 mon.a (mon.0) 497 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:09:49.444 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:09:49 smithi100 bash[31660]: audit 2024-04-07T16:09:48.658460+0000 mon.a (mon.0) 498 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:09:49.444 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:09:49 smithi100 bash[31660]: audit 2024-04-07T16:09:48.668782+0000 mon.a (mon.0) 499 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:09:49.608 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:09:49 smithi144 bash[23522]: audit 2024-04-07T16:09:48.120275+0000 mon.a (mon.0) 495 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:09:49.608 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:09:49 smithi144 bash[23522]: audit 2024-04-07T16:09:48.129900+0000 mon.a (mon.0) 496 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:09:49.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:09:49 smithi144 bash[23522]: audit 2024-04-07T16:09:48.656621+0000 mon.a (mon.0) 497 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:09:49.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:09:49 smithi144 bash[23522]: audit 2024-04-07T16:09:48.658460+0000 mon.a (mon.0) 498 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:09:49.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:09:49 smithi144 bash[23522]: audit 2024-04-07T16:09:48.668782+0000 mon.a (mon.0) 499 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:09:50.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:09:50 smithi144 bash[23522]: cluster 2024-04-07T16:09:48.244792+0000 mgr.y (mgr.14152) 245 : cluster 0 pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-07T16:09:50.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:09:50 smithi144 bash[23522]: audit 2024-04-07T16:09:49.187906+0000 mon.b (mon.1) 15 : audit 1 from='osd.5 [v2:172.21.15.144:6808/2783162058,v1:172.21.15.144:6809/2783162058]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-07T16:09:50.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:09:50 smithi144 bash[23522]: audit 2024-04-07T16:09:49.190510+0000 mon.a (mon.0) 500 : audit 1 from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-07T16:09:50.443 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:09:50 smithi100 bash[21346]: cluster 2024-04-07T16:09:48.244792+0000 mgr.y (mgr.14152) 245 : cluster 0 pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-07T16:09:50.443 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:09:50 smithi100 bash[21346]: audit 2024-04-07T16:09:49.187906+0000 mon.b (mon.1) 15 : audit 1 from='osd.5 [v2:172.21.15.144:6808/2783162058,v1:172.21.15.144:6809/2783162058]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-07T16:09:50.443 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:09:50 smithi100 bash[21346]: audit 2024-04-07T16:09:49.190510+0000 mon.a (mon.0) 500 : audit 1 from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-07T16:09:50.443 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:09:50 smithi100 bash[31660]: cluster 2024-04-07T16:09:48.244792+0000 mgr.y (mgr.14152) 245 : cluster 0 pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-07T16:09:50.443 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:09:50 smithi100 bash[31660]: audit 2024-04-07T16:09:49.187906+0000 mon.b (mon.1) 15 : audit 1 from='osd.5 [v2:172.21.15.144:6808/2783162058,v1:172.21.15.144:6809/2783162058]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-07T16:09:50.443 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:09:50 smithi100 bash[31660]: audit 2024-04-07T16:09:49.190510+0000 mon.a (mon.0) 500 : audit 1 from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-07T16:09:51.151 INFO:teuthology.orchestra.run.smithi144.stdout:Created osd(s) 5 on host 'smithi144' 2024-04-07T16:09:51.443 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:09:51 smithi100 bash[21346]: audit 2024-04-07T16:09:50.134037+0000 mon.b (mon.1) 16 : audit 1 from='osd.5 [v2:172.21.15.144:6808/2783162058,v1:172.21.15.144:6809/2783162058]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-04-07T16:09:51.443 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:09:51 smithi100 bash[21346]: audit 2024-04-07T16:09:50.134804+0000 mon.a (mon.0) 501 : audit 1 from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-07T16:09:51.443 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:09:51 smithi100 bash[21346]: cluster 2024-04-07T16:09:50.135044+0000 mon.a (mon.0) 502 : cluster 0 osdmap e36: 6 total, 5 up, 6 in 2024-04-07T16:09:51.443 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:09:51 smithi100 bash[21346]: audit 2024-04-07T16:09:50.135782+0000 mon.a (mon.0) 503 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-07T16:09:51.443 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:09:51 smithi100 bash[21346]: audit 2024-04-07T16:09:50.143085+0000 mon.a (mon.0) 504 : audit 1 from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-04-07T16:09:51.444 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:09:51 smithi100 bash[21346]: audit 2024-04-07T16:09:51.096427+0000 mon.a (mon.0) 505 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:09:51.444 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:09:51 smithi100 bash[21346]: audit 2024-04-07T16:09:51.107769+0000 mon.a (mon.0) 506 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:09:51.444 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:09:51 smithi100 bash[21346]: audit 2024-04-07T16:09:51.109940+0000 mon.a (mon.0) 507 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:09:51.444 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:09:51 smithi100 bash[31660]: audit 2024-04-07T16:09:50.134037+0000 mon.b (mon.1) 16 : audit 1 from='osd.5 [v2:172.21.15.144:6808/2783162058,v1:172.21.15.144:6809/2783162058]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-04-07T16:09:51.444 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:09:51 smithi100 bash[31660]: audit 2024-04-07T16:09:50.134804+0000 mon.a (mon.0) 501 : audit 1 from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-07T16:09:51.444 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:09:51 smithi100 bash[31660]: cluster 2024-04-07T16:09:50.135044+0000 mon.a (mon.0) 502 : cluster 0 osdmap e36: 6 total, 5 up, 6 in 2024-04-07T16:09:51.444 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:09:51 smithi100 bash[31660]: audit 2024-04-07T16:09:50.135782+0000 mon.a (mon.0) 503 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-07T16:09:51.444 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:09:51 smithi100 bash[31660]: audit 2024-04-07T16:09:50.143085+0000 mon.a (mon.0) 504 : audit 1 from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-04-07T16:09:51.444 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:09:51 smithi100 bash[31660]: audit 2024-04-07T16:09:51.096427+0000 mon.a (mon.0) 505 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:09:51.445 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:09:51 smithi100 bash[31660]: audit 2024-04-07T16:09:51.107769+0000 mon.a (mon.0) 506 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:09:51.445 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:09:51 smithi100 bash[31660]: audit 2024-04-07T16:09:51.109940+0000 mon.a (mon.0) 507 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:09:51.608 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:09:51 smithi144 bash[23522]: audit 2024-04-07T16:09:50.134037+0000 mon.b (mon.1) 16 : audit 1 from='osd.5 [v2:172.21.15.144:6808/2783162058,v1:172.21.15.144:6809/2783162058]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-04-07T16:09:51.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:09:51 smithi144 bash[23522]: audit 2024-04-07T16:09:50.134804+0000 mon.a (mon.0) 501 : audit 1 from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-07T16:09:51.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:09:51 smithi144 bash[23522]: cluster 2024-04-07T16:09:50.135044+0000 mon.a (mon.0) 502 : cluster 0 osdmap e36: 6 total, 5 up, 6 in 2024-04-07T16:09:51.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:09:51 smithi144 bash[23522]: audit 2024-04-07T16:09:50.135782+0000 mon.a (mon.0) 503 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-07T16:09:51.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:09:51 smithi144 bash[23522]: audit 2024-04-07T16:09:50.143085+0000 mon.a (mon.0) 504 : audit 1 from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-04-07T16:09:51.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:09:51 smithi144 bash[23522]: audit 2024-04-07T16:09:51.096427+0000 mon.a (mon.0) 505 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:09:51.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:09:51 smithi144 bash[23522]: audit 2024-04-07T16:09:51.107769+0000 mon.a (mon.0) 506 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:09:51.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:09:51 smithi144 bash[23522]: audit 2024-04-07T16:09:51.109940+0000 mon.a (mon.0) 507 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:09:51.744 DEBUG:teuthology.orchestra.run.smithi144:osd.5> sudo journalctl -f -n 0 -u ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@osd.5.service 2024-04-07T16:09:51.747 INFO:tasks.cephadm:Deploying osd.6 on smithi144 with /dev/vg_nvme/lv_2... 2024-04-07T16:09:51.747 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0702e3a8-f4f8-11ee-b649-cb9ed24678a4 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-07T16:09:52.443 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:09:52 smithi100 bash[21346]: cluster 2024-04-07T16:09:50.245344+0000 mgr.y (mgr.14152) 246 : cluster 0 pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-07T16:09:52.443 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:09:52 smithi100 bash[21346]: audit 2024-04-07T16:09:51.160450+0000 mon.a (mon.0) 508 : audit 1 from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]': finished 2024-04-07T16:09:52.443 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:09:52 smithi100 bash[21346]: cluster 2024-04-07T16:09:51.160703+0000 mon.a (mon.0) 509 : cluster 0 osdmap e37: 6 total, 5 up, 6 in 2024-04-07T16:09:52.443 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:09:52 smithi100 bash[21346]: audit 2024-04-07T16:09:51.161308+0000 mon.a (mon.0) 510 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-07T16:09:52.443 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:09:52 smithi100 bash[21346]: audit 2024-04-07T16:09:51.168675+0000 mon.a (mon.0) 511 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-07T16:09:52.443 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:09:52 smithi100 bash[31660]: cluster 2024-04-07T16:09:50.245344+0000 mgr.y (mgr.14152) 246 : cluster 0 pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-07T16:09:52.443 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:09:52 smithi100 bash[31660]: audit 2024-04-07T16:09:51.160450+0000 mon.a (mon.0) 508 : audit 1 from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]': finished 2024-04-07T16:09:52.443 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:09:52 smithi100 bash[31660]: cluster 2024-04-07T16:09:51.160703+0000 mon.a (mon.0) 509 : cluster 0 osdmap e37: 6 total, 5 up, 6 in 2024-04-07T16:09:52.444 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:09:52 smithi100 bash[31660]: audit 2024-04-07T16:09:51.161308+0000 mon.a (mon.0) 510 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-07T16:09:52.444 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:09:52 smithi100 bash[31660]: audit 2024-04-07T16:09:51.168675+0000 mon.a (mon.0) 511 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-07T16:09:52.608 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:09:52 smithi144 bash[23522]: cluster 2024-04-07T16:09:50.245344+0000 mgr.y (mgr.14152) 246 : cluster 0 pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-07T16:09:52.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:09:52 smithi144 bash[23522]: audit 2024-04-07T16:09:51.160450+0000 mon.a (mon.0) 508 : audit 1 from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]': finished 2024-04-07T16:09:52.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:09:52 smithi144 bash[23522]: cluster 2024-04-07T16:09:51.160703+0000 mon.a (mon.0) 509 : cluster 0 osdmap e37: 6 total, 5 up, 6 in 2024-04-07T16:09:52.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:09:52 smithi144 bash[23522]: audit 2024-04-07T16:09:51.161308+0000 mon.a (mon.0) 510 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-07T16:09:52.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:09:52 smithi144 bash[23522]: audit 2024-04-07T16:09:51.168675+0000 mon.a (mon.0) 511 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-07T16:09:53.443 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:09:53 smithi100 bash[21346]: cluster 2024-04-07T16:09:50.222098+0000 osd.5 (osd.5) 1 : cluster 0 purged_snaps scrub starts 2024-04-07T16:09:53.443 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:09:53 smithi100 bash[21346]: cluster 2024-04-07T16:09:50.222181+0000 osd.5 (osd.5) 2 : cluster 0 purged_snaps scrub ok 2024-04-07T16:09:53.443 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:09:53 smithi100 bash[21346]: cluster 2024-04-07T16:09:52.161184+0000 mon.a (mon.0) 512 : cluster 1 osd.5 [v2:172.21.15.144:6808/2783162058,v1:172.21.15.144:6809/2783162058] boot 2024-04-07T16:09:53.443 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:09:53 smithi100 bash[21346]: cluster 2024-04-07T16:09:52.161327+0000 mon.a (mon.0) 513 : cluster 0 osdmap e38: 6 total, 6 up, 6 in 2024-04-07T16:09:53.444 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:09:53 smithi100 bash[21346]: audit 2024-04-07T16:09:52.162043+0000 mon.a (mon.0) 514 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-07T16:09:53.444 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:09:53 smithi100 bash[21346]: cluster 2024-04-07T16:09:52.488373+0000 mon.a (mon.0) 515 : cluster 0 osdmap e39: 6 total, 6 up, 6 in 2024-04-07T16:09:53.444 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:09:53 smithi100 bash[31660]: cluster 2024-04-07T16:09:50.222098+0000 osd.5 (osd.5) 1 : cluster 0 purged_snaps scrub starts 2024-04-07T16:09:53.444 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:09:53 smithi100 bash[31660]: cluster 2024-04-07T16:09:50.222181+0000 osd.5 (osd.5) 2 : cluster 0 purged_snaps scrub ok 2024-04-07T16:09:53.444 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:09:53 smithi100 bash[31660]: cluster 2024-04-07T16:09:52.161184+0000 mon.a (mon.0) 512 : cluster 1 osd.5 [v2:172.21.15.144:6808/2783162058,v1:172.21.15.144:6809/2783162058] boot 2024-04-07T16:09:53.444 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:09:53 smithi100 bash[31660]: cluster 2024-04-07T16:09:52.161327+0000 mon.a (mon.0) 513 : cluster 0 osdmap e38: 6 total, 6 up, 6 in 2024-04-07T16:09:53.444 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:09:53 smithi100 bash[31660]: audit 2024-04-07T16:09:52.162043+0000 mon.a (mon.0) 514 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-07T16:09:53.444 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:09:53 smithi100 bash[31660]: cluster 2024-04-07T16:09:52.488373+0000 mon.a (mon.0) 515 : cluster 0 osdmap e39: 6 total, 6 up, 6 in 2024-04-07T16:09:53.608 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:09:53 smithi144 bash[23522]: cluster 2024-04-07T16:09:50.222098+0000 osd.5 (osd.5) 1 : cluster 0 purged_snaps scrub starts 2024-04-07T16:09:53.608 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:09:53 smithi144 bash[23522]: cluster 2024-04-07T16:09:50.222181+0000 osd.5 (osd.5) 2 : cluster 0 purged_snaps scrub ok 2024-04-07T16:09:53.608 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:09:53 smithi144 bash[23522]: cluster 2024-04-07T16:09:52.161184+0000 mon.a (mon.0) 512 : cluster 1 osd.5 [v2:172.21.15.144:6808/2783162058,v1:172.21.15.144:6809/2783162058] boot 2024-04-07T16:09:53.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:09:53 smithi144 bash[23522]: cluster 2024-04-07T16:09:52.161327+0000 mon.a (mon.0) 513 : cluster 0 osdmap e38: 6 total, 6 up, 6 in 2024-04-07T16:09:53.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:09:53 smithi144 bash[23522]: audit 2024-04-07T16:09:52.162043+0000 mon.a (mon.0) 514 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-07T16:09:53.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:09:53 smithi144 bash[23522]: cluster 2024-04-07T16:09:52.488373+0000 mon.a (mon.0) 515 : cluster 0 osdmap e39: 6 total, 6 up, 6 in 2024-04-07T16:09:54.442 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:09:54 smithi100 bash[21346]: cluster 2024-04-07T16:09:52.245897+0000 mgr.y (mgr.14152) 247 : cluster 0 pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-07T16:09:54.443 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:09:54 smithi100 bash[21346]: cluster 2024-04-07T16:09:53.491033+0000 mon.a (mon.0) 516 : cluster 0 osdmap e40: 6 total, 6 up, 6 in 2024-04-07T16:09:54.443 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:09:54 smithi100 bash[31660]: cluster 2024-04-07T16:09:52.245897+0000 mgr.y (mgr.14152) 247 : cluster 0 pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-07T16:09:54.443 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:09:54 smithi100 bash[31660]: cluster 2024-04-07T16:09:53.491033+0000 mon.a (mon.0) 516 : cluster 0 osdmap e40: 6 total, 6 up, 6 in 2024-04-07T16:09:54.608 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:09:54 smithi144 bash[23522]: cluster 2024-04-07T16:09:52.245897+0000 mgr.y (mgr.14152) 247 : cluster 0 pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-07T16:09:54.608 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:09:54 smithi144 bash[23522]: cluster 2024-04-07T16:09:53.491033+0000 mon.a (mon.0) 516 : cluster 0 osdmap e40: 6 total, 6 up, 6 in 2024-04-07T16:09:55.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:09:55 smithi144 bash[23522]: cluster 2024-04-07T16:09:54.246496+0000 mgr.y (mgr.14152) 248 : cluster 0 pgmap v225: 1 pgs: 1 remapped+peering; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-07T16:09:55.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:09:55 smithi144 bash[23522]: cluster 2024-04-07T16:09:54.493429+0000 mon.a (mon.0) 517 : cluster 0 osdmap e41: 6 total, 6 up, 6 in 2024-04-07T16:09:55.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:09:55 smithi144 bash[23522]: cluster 2024-04-07T16:09:55.173957+0000 mon.a (mon.0) 518 : cluster 3 Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-04-07T16:09:55.943 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:09:55 smithi100 bash[21346]: cluster 2024-04-07T16:09:54.246496+0000 mgr.y (mgr.14152) 248 : cluster 0 pgmap v225: 1 pgs: 1 remapped+peering; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-07T16:09:55.943 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:09:55 smithi100 bash[21346]: cluster 2024-04-07T16:09:54.493429+0000 mon.a (mon.0) 517 : cluster 0 osdmap e41: 6 total, 6 up, 6 in 2024-04-07T16:09:55.943 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:09:55 smithi100 bash[21346]: cluster 2024-04-07T16:09:55.173957+0000 mon.a (mon.0) 518 : cluster 3 Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-04-07T16:09:55.944 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:09:55 smithi100 bash[31660]: cluster 2024-04-07T16:09:54.246496+0000 mgr.y (mgr.14152) 248 : cluster 0 pgmap v225: 1 pgs: 1 remapped+peering; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-07T16:09:55.944 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:09:55 smithi100 bash[31660]: cluster 2024-04-07T16:09:54.493429+0000 mon.a (mon.0) 517 : cluster 0 osdmap e41: 6 total, 6 up, 6 in 2024-04-07T16:09:55.944 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:09:55 smithi100 bash[31660]: cluster 2024-04-07T16:09:55.173957+0000 mon.a (mon.0) 518 : cluster 3 Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-04-07T16:09:55.953 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/mon.b/config 2024-04-07T16:09:56.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:09:56 smithi144 bash[23522]: cluster 2024-04-07T16:09:55.505112+0000 mon.a (mon.0) 519 : cluster 0 osdmap e42: 6 total, 6 up, 6 in 2024-04-07T16:09:56.942 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:09:56 smithi100 bash[21346]: cluster 2024-04-07T16:09:55.505112+0000 mon.a (mon.0) 519 : cluster 0 osdmap e42: 6 total, 6 up, 6 in 2024-04-07T16:09:56.943 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:09:56 smithi100 bash[31660]: cluster 2024-04-07T16:09:55.505112+0000 mon.a (mon.0) 519 : cluster 0 osdmap e42: 6 total, 6 up, 6 in 2024-04-07T16:09:57.823 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:09:57 smithi144 bash[23522]: cluster 2024-04-07T16:09:56.247089+0000 mgr.y (mgr.14152) 249 : cluster 0 pgmap v228: 1 pgs: 1 remapped+peering; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-07T16:09:57.942 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:09:57 smithi100 bash[21346]: cluster 2024-04-07T16:09:56.247089+0000 mgr.y (mgr.14152) 249 : cluster 0 pgmap v228: 1 pgs: 1 remapped+peering; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-07T16:09:57.943 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:09:57 smithi100 bash[31660]: cluster 2024-04-07T16:09:56.247089+0000 mgr.y (mgr.14152) 249 : cluster 0 pgmap v228: 1 pgs: 1 remapped+peering; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-07T16:09:58.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:09:58 smithi144 bash[23522]: cluster 2024-04-07T16:09:58.512862+0000 mon.a (mon.0) 520 : cluster 1 Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-04-07T16:09:58.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:09:58 smithi144 bash[23522]: cluster 2024-04-07T16:09:58.512918+0000 mon.a (mon.0) 521 : cluster 1 Cluster is now healthy 2024-04-07T16:09:58.942 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:09:58 smithi100 bash[21346]: cluster 2024-04-07T16:09:58.512862+0000 mon.a (mon.0) 520 : cluster 1 Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-04-07T16:09:58.943 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:09:58 smithi100 bash[21346]: cluster 2024-04-07T16:09:58.512918+0000 mon.a (mon.0) 521 : cluster 1 Cluster is now healthy 2024-04-07T16:09:58.943 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:09:58 smithi100 bash[31660]: cluster 2024-04-07T16:09:58.512862+0000 mon.a (mon.0) 520 : cluster 1 Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-04-07T16:09:58.943 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:09:58 smithi100 bash[31660]: cluster 2024-04-07T16:09:58.512918+0000 mon.a (mon.0) 521 : cluster 1 Cluster is now healthy 2024-04-07T16:09:59.293 INFO:teuthology.orchestra.run.smithi144.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-07T16:09:59.294 INFO:teuthology.orchestra.run.smithi144.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-07T16:09:59.294 INFO:teuthology.orchestra.run.smithi144.stderr: stderr: 10+0 records in 2024-04-07T16:09:59.294 INFO:teuthology.orchestra.run.smithi144.stderr:10+0 records out 2024-04-07T16:09:59.294 INFO:teuthology.orchestra.run.smithi144.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0137721 s, 761 MB/s 2024-04-07T16:09:59.294 INFO:teuthology.orchestra.run.smithi144.stderr:--> Zapping successful for: 2024-04-07T16:09:59.590 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:09:59 smithi144 bash[23522]: cluster 2024-04-07T16:09:58.247686+0000 mgr.y (mgr.14152) 250 : cluster 0 pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-07T16:09:59.942 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:09:59 smithi100 bash[21346]: cluster 2024-04-07T16:09:58.247686+0000 mgr.y (mgr.14152) 250 : cluster 0 pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-07T16:09:59.943 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:09:59 smithi100 bash[31660]: cluster 2024-04-07T16:09:58.247686+0000 mgr.y (mgr.14152) 250 : cluster 0 pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-07T16:10:00.155 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0702e3a8-f4f8-11ee-b649-cb9ed24678a4 -- ceph orch daemon add osd smithi144:vg_nvme/lv_2 2024-04-07T16:10:00.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:10:00 smithi144 bash[23522]: cluster 2024-04-07T16:10:00.000174+0000 mon.a (mon.0) 522 : cluster 1 overall HEALTH_OK 2024-04-07T16:10:00.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:10:00 smithi144 bash[23522]: audit 2024-04-07T16:10:00.463368+0000 mon.a (mon.0) 523 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:10:00.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:10:00 smithi144 bash[23522]: audit 2024-04-07T16:10:00.472713+0000 mon.a (mon.0) 524 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:10:00.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:10:00 smithi144 bash[23522]: audit 2024-04-07T16:10:00.475073+0000 mon.a (mon.0) 525 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-07T16:10:00.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:10:00 smithi144 bash[23522]: audit 2024-04-07T16:10:00.476982+0000 mon.a (mon.0) 526 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-07T16:10:00.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:10:00 smithi144 bash[23522]: audit 2024-04-07T16:10:00.486416+0000 mon.a (mon.0) 527 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:10:00.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:10:00 smithi144 bash[23522]: audit 2024-04-07T16:10:00.488783+0000 mon.a (mon.0) 528 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:10:00.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:10:00 smithi144 bash[23522]: audit 2024-04-07T16:10:00.490552+0000 mon.a (mon.0) 529 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:10:00.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:10:00 smithi144 bash[23522]: audit 2024-04-07T16:10:00.500194+0000 mon.a (mon.0) 530 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:10:00.943 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:10:00 smithi100 bash[21346]: cluster 2024-04-07T16:10:00.000174+0000 mon.a (mon.0) 522 : cluster 1 overall HEALTH_OK 2024-04-07T16:10:00.943 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:10:00 smithi100 bash[21346]: audit 2024-04-07T16:10:00.463368+0000 mon.a (mon.0) 523 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:10:00.943 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:10:00 smithi100 bash[21346]: audit 2024-04-07T16:10:00.472713+0000 mon.a (mon.0) 524 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:10:00.943 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:10:00 smithi100 bash[21346]: audit 2024-04-07T16:10:00.475073+0000 mon.a (mon.0) 525 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-07T16:10:00.943 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:10:00 smithi100 bash[21346]: audit 2024-04-07T16:10:00.476982+0000 mon.a (mon.0) 526 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-07T16:10:00.943 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:10:00 smithi100 bash[21346]: audit 2024-04-07T16:10:00.486416+0000 mon.a (mon.0) 527 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:10:00.943 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:10:00 smithi100 bash[21346]: audit 2024-04-07T16:10:00.488783+0000 mon.a (mon.0) 528 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:10:00.943 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:10:00 smithi100 bash[21346]: audit 2024-04-07T16:10:00.490552+0000 mon.a (mon.0) 529 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:10:00.943 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:10:00 smithi100 bash[21346]: audit 2024-04-07T16:10:00.500194+0000 mon.a (mon.0) 530 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:10:00.944 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:10:00 smithi100 bash[31660]: cluster 2024-04-07T16:10:00.000174+0000 mon.a (mon.0) 522 : cluster 1 overall HEALTH_OK 2024-04-07T16:10:00.944 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:10:00 smithi100 bash[31660]: audit 2024-04-07T16:10:00.463368+0000 mon.a (mon.0) 523 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:10:00.944 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:10:00 smithi100 bash[31660]: audit 2024-04-07T16:10:00.472713+0000 mon.a (mon.0) 524 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:10:00.944 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:10:00 smithi100 bash[31660]: audit 2024-04-07T16:10:00.475073+0000 mon.a (mon.0) 525 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-07T16:10:00.944 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:10:00 smithi100 bash[31660]: audit 2024-04-07T16:10:00.476982+0000 mon.a (mon.0) 526 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-07T16:10:00.944 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:10:00 smithi100 bash[31660]: audit 2024-04-07T16:10:00.486416+0000 mon.a (mon.0) 527 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:10:00.944 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:10:00 smithi100 bash[31660]: audit 2024-04-07T16:10:00.488783+0000 mon.a (mon.0) 528 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:10:00.944 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:10:00 smithi100 bash[31660]: audit 2024-04-07T16:10:00.490552+0000 mon.a (mon.0) 529 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:10:00.944 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:10:00 smithi100 bash[31660]: audit 2024-04-07T16:10:00.500194+0000 mon.a (mon.0) 530 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:10:01.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:10:01 smithi144 bash[23522]: cluster 2024-04-07T16:10:00.248186+0000 mgr.y (mgr.14152) 251 : cluster 0 pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-04-07T16:10:01.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:10:01 smithi144 bash[23522]: cephadm 2024-04-07T16:10:00.454242+0000 mgr.y (mgr.14152) 252 : cephadm 1 Detected new or changed devices on smithi144 2024-04-07T16:10:01.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:10:01 smithi144 bash[23522]: cephadm 2024-04-07T16:10:00.478207+0000 mgr.y (mgr.14152) 253 : cephadm 1 Adjusting osd_memory_target on smithi144 to 8632M 2024-04-07T16:10:01.942 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:10:01 smithi100 bash[21346]: cluster 2024-04-07T16:10:00.248186+0000 mgr.y (mgr.14152) 251 : cluster 0 pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-04-07T16:10:01.943 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:10:01 smithi100 bash[21346]: cephadm 2024-04-07T16:10:00.454242+0000 mgr.y (mgr.14152) 252 : cephadm 1 Detected new or changed devices on smithi144 2024-04-07T16:10:01.943 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:10:01 smithi100 bash[21346]: cephadm 2024-04-07T16:10:00.478207+0000 mgr.y (mgr.14152) 253 : cephadm 1 Adjusting osd_memory_target on smithi144 to 8632M 2024-04-07T16:10:01.943 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:10:01 smithi100 bash[31660]: cluster 2024-04-07T16:10:00.248186+0000 mgr.y (mgr.14152) 251 : cluster 0 pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-04-07T16:10:01.943 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:10:01 smithi100 bash[31660]: cephadm 2024-04-07T16:10:00.454242+0000 mgr.y (mgr.14152) 252 : cephadm 1 Detected new or changed devices on smithi144 2024-04-07T16:10:01.943 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:10:01 smithi100 bash[31660]: cephadm 2024-04-07T16:10:00.478207+0000 mgr.y (mgr.14152) 253 : cephadm 1 Adjusting osd_memory_target on smithi144 to 8632M 2024-04-07T16:10:03.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:10:03 smithi144 bash[23522]: cluster 2024-04-07T16:10:02.248751+0000 mgr.y (mgr.14152) 254 : cluster 0 pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-07T16:10:03.942 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:10:03 smithi100 bash[31660]: cluster 2024-04-07T16:10:02.248751+0000 mgr.y (mgr.14152) 254 : cluster 0 pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-07T16:10:03.943 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:10:03 smithi100 bash[21346]: cluster 2024-04-07T16:10:02.248751+0000 mgr.y (mgr.14152) 254 : cluster 0 pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-07T16:10:04.837 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/mon.b/config 2024-04-07T16:10:05.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:10:05 smithi144 bash[23522]: cluster 2024-04-07T16:10:04.249343+0000 mgr.y (mgr.14152) 255 : cluster 0 pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-04-07T16:10:05.942 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:10:05 smithi100 bash[21346]: cluster 2024-04-07T16:10:04.249343+0000 mgr.y (mgr.14152) 255 : cluster 0 pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-04-07T16:10:05.943 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:10:05 smithi100 bash[31660]: cluster 2024-04-07T16:10:04.249343+0000 mgr.y (mgr.14152) 255 : cluster 0 pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-04-07T16:10:07.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:10:07 smithi144 bash[23522]: cluster 2024-04-07T16:10:06.249960+0000 mgr.y (mgr.14152) 256 : cluster 0 pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-04-07T16:10:07.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:10:07 smithi144 bash[23522]: audit 2024-04-07T16:10:07.319517+0000 mon.a (mon.0) 531 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-07T16:10:07.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:10:07 smithi144 bash[23522]: audit 2024-04-07T16:10:07.325845+0000 mon.a (mon.0) 532 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-07T16:10:07.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:10:07 smithi144 bash[23522]: audit 2024-04-07T16:10:07.327564+0000 mon.a (mon.0) 533 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:10:07.942 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:10:07 smithi100 bash[21346]: cluster 2024-04-07T16:10:06.249960+0000 mgr.y (mgr.14152) 256 : cluster 0 pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-04-07T16:10:07.943 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:10:07 smithi100 bash[21346]: audit 2024-04-07T16:10:07.319517+0000 mon.a (mon.0) 531 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-07T16:10:07.943 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:10:07 smithi100 bash[21346]: audit 2024-04-07T16:10:07.325845+0000 mon.a (mon.0) 532 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-07T16:10:07.943 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:10:07 smithi100 bash[21346]: audit 2024-04-07T16:10:07.327564+0000 mon.a (mon.0) 533 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:10:07.943 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:10:07 smithi100 bash[31660]: cluster 2024-04-07T16:10:06.249960+0000 mgr.y (mgr.14152) 256 : cluster 0 pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-04-07T16:10:07.943 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:10:07 smithi100 bash[31660]: audit 2024-04-07T16:10:07.319517+0000 mon.a (mon.0) 531 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-07T16:10:07.943 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:10:07 smithi100 bash[31660]: audit 2024-04-07T16:10:07.325845+0000 mon.a (mon.0) 532 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-07T16:10:07.943 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:10:07 smithi100 bash[31660]: audit 2024-04-07T16:10:07.327564+0000 mon.a (mon.0) 533 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:10:08.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:10:08 smithi144 bash[23522]: audit 2024-04-07T16:10:07.312612+0000 mgr.y (mgr.14152) 257 : audit 0 from='client.24232 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:10:08.943 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:10:08 smithi100 bash[21346]: audit 2024-04-07T16:10:07.312612+0000 mgr.y (mgr.14152) 257 : audit 0 from='client.24232 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:10:08.943 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:10:08 smithi100 bash[31660]: audit 2024-04-07T16:10:07.312612+0000 mgr.y (mgr.14152) 257 : audit 0 from='client.24232 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:10:09.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:10:09 smithi144 bash[23522]: cluster 2024-04-07T16:10:08.250566+0000 mgr.y (mgr.14152) 258 : cluster 0 pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-04-07T16:10:09.942 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:10:09 smithi100 bash[21346]: cluster 2024-04-07T16:10:08.250566+0000 mgr.y (mgr.14152) 258 : cluster 0 pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-04-07T16:10:09.943 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:10:09 smithi100 bash[31660]: cluster 2024-04-07T16:10:08.250566+0000 mgr.y (mgr.14152) 258 : cluster 0 pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-04-07T16:10:11.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:10:11 smithi144 bash[23522]: cluster 2024-04-07T16:10:10.251125+0000 mgr.y (mgr.14152) 259 : cluster 0 pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-07T16:10:11.942 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:10:11 smithi100 bash[21346]: cluster 2024-04-07T16:10:10.251125+0000 mgr.y (mgr.14152) 259 : cluster 0 pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-07T16:10:11.942 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:10:11 smithi100 bash[31660]: cluster 2024-04-07T16:10:10.251125+0000 mgr.y (mgr.14152) 259 : cluster 0 pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-07T16:10:13.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:10:13 smithi144 bash[23522]: cluster 2024-04-07T16:10:12.251763+0000 mgr.y (mgr.14152) 260 : cluster 0 pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-07T16:10:13.942 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:10:13 smithi100 bash[21346]: cluster 2024-04-07T16:10:12.251763+0000 mgr.y (mgr.14152) 260 : cluster 0 pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-07T16:10:13.942 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:10:13 smithi100 bash[31660]: cluster 2024-04-07T16:10:12.251763+0000 mgr.y (mgr.14152) 260 : cluster 0 pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-07T16:10:15.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:10:15 smithi144 bash[23522]: cluster 2024-04-07T16:10:14.252303+0000 mgr.y (mgr.14152) 261 : cluster 0 pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-07T16:10:15.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:10:15 smithi144 bash[23522]: audit 2024-04-07T16:10:15.160047+0000 mon.b (mon.1) 17 : audit 1 from='client.? 172.21.15.144:0/2722067260' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ac619497-7605-4ca6-84bf-3abb3f6f32d7"}]: dispatch 2024-04-07T16:10:15.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:10:15 smithi144 bash[23522]: audit 2024-04-07T16:10:15.163116+0000 mon.a (mon.0) 534 : audit 1 from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ac619497-7605-4ca6-84bf-3abb3f6f32d7"}]: dispatch 2024-04-07T16:10:15.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:10:15 smithi144 bash[23522]: audit 2024-04-07T16:10:15.175665+0000 mon.a (mon.0) 535 : audit 1 from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ac619497-7605-4ca6-84bf-3abb3f6f32d7"}]': finished 2024-04-07T16:10:15.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:10:15 smithi144 bash[23522]: cluster 2024-04-07T16:10:15.175843+0000 mon.a (mon.0) 536 : cluster 0 osdmap e43: 7 total, 6 up, 7 in 2024-04-07T16:10:15.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:10:15 smithi144 bash[23522]: audit 2024-04-07T16:10:15.176178+0000 mon.a (mon.0) 537 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-07T16:10:15.942 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:10:15 smithi100 bash[21346]: cluster 2024-04-07T16:10:14.252303+0000 mgr.y (mgr.14152) 261 : cluster 0 pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-07T16:10:15.942 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:10:15 smithi100 bash[21346]: audit 2024-04-07T16:10:15.160047+0000 mon.b (mon.1) 17 : audit 1 from='client.? 172.21.15.144:0/2722067260' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ac619497-7605-4ca6-84bf-3abb3f6f32d7"}]: dispatch 2024-04-07T16:10:15.943 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:10:15 smithi100 bash[21346]: audit 2024-04-07T16:10:15.163116+0000 mon.a (mon.0) 534 : audit 1 from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ac619497-7605-4ca6-84bf-3abb3f6f32d7"}]: dispatch 2024-04-07T16:10:15.943 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:10:15 smithi100 bash[21346]: audit 2024-04-07T16:10:15.175665+0000 mon.a (mon.0) 535 : audit 1 from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ac619497-7605-4ca6-84bf-3abb3f6f32d7"}]': finished 2024-04-07T16:10:15.943 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:10:15 smithi100 bash[21346]: cluster 2024-04-07T16:10:15.175843+0000 mon.a (mon.0) 536 : cluster 0 osdmap e43: 7 total, 6 up, 7 in 2024-04-07T16:10:15.943 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:10:15 smithi100 bash[21346]: audit 2024-04-07T16:10:15.176178+0000 mon.a (mon.0) 537 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-07T16:10:15.943 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:10:15 smithi100 bash[31660]: cluster 2024-04-07T16:10:14.252303+0000 mgr.y (mgr.14152) 261 : cluster 0 pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-07T16:10:15.943 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:10:15 smithi100 bash[31660]: audit 2024-04-07T16:10:15.160047+0000 mon.b (mon.1) 17 : audit 1 from='client.? 172.21.15.144:0/2722067260' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ac619497-7605-4ca6-84bf-3abb3f6f32d7"}]: dispatch 2024-04-07T16:10:15.943 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:10:15 smithi100 bash[31660]: audit 2024-04-07T16:10:15.163116+0000 mon.a (mon.0) 534 : audit 1 from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ac619497-7605-4ca6-84bf-3abb3f6f32d7"}]: dispatch 2024-04-07T16:10:15.943 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:10:15 smithi100 bash[31660]: audit 2024-04-07T16:10:15.175665+0000 mon.a (mon.0) 535 : audit 1 from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ac619497-7605-4ca6-84bf-3abb3f6f32d7"}]': finished 2024-04-07T16:10:15.943 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:10:15 smithi100 bash[31660]: cluster 2024-04-07T16:10:15.175843+0000 mon.a (mon.0) 536 : cluster 0 osdmap e43: 7 total, 6 up, 7 in 2024-04-07T16:10:15.943 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:10:15 smithi100 bash[31660]: audit 2024-04-07T16:10:15.176178+0000 mon.a (mon.0) 537 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-07T16:10:16.942 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:10:16 smithi100 bash[31660]: audit 2024-04-07T16:10:15.868711+0000 mon.b (mon.1) 18 : audit 0 from='client.? 172.21.15.144:0/855970606' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-07T16:10:16.943 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:10:16 smithi100 bash[21346]: audit 2024-04-07T16:10:15.868711+0000 mon.b (mon.1) 18 : audit 0 from='client.? 172.21.15.144:0/855970606' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-07T16:10:17.108 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:10:16 smithi144 bash[23522]: audit 2024-04-07T16:10:15.868711+0000 mon.b (mon.1) 18 : audit 0 from='client.? 172.21.15.144:0/855970606' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-07T16:10:17.942 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:10:17 smithi100 bash[31660]: cluster 2024-04-07T16:10:16.252878+0000 mgr.y (mgr.14152) 262 : cluster 0 pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-07T16:10:17.942 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:10:17 smithi100 bash[21346]: cluster 2024-04-07T16:10:16.252878+0000 mgr.y (mgr.14152) 262 : cluster 0 pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-07T16:10:18.108 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:10:17 smithi144 bash[23522]: cluster 2024-04-07T16:10:16.252878+0000 mgr.y (mgr.14152) 262 : cluster 0 pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-07T16:10:19.942 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:10:19 smithi100 bash[31660]: cluster 2024-04-07T16:10:18.253436+0000 mgr.y (mgr.14152) 263 : cluster 0 pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-07T16:10:19.942 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:10:19 smithi100 bash[21346]: cluster 2024-04-07T16:10:18.253436+0000 mgr.y (mgr.14152) 263 : cluster 0 pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-07T16:10:20.108 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:10:19 smithi144 bash[23522]: cluster 2024-04-07T16:10:18.253436+0000 mgr.y (mgr.14152) 263 : cluster 0 pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-07T16:10:21.942 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:10:21 smithi100 bash[31660]: cluster 2024-04-07T16:10:20.253972+0000 mgr.y (mgr.14152) 264 : cluster 0 pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-07T16:10:21.942 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:10:21 smithi100 bash[21346]: cluster 2024-04-07T16:10:20.253972+0000 mgr.y (mgr.14152) 264 : cluster 0 pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-07T16:10:22.108 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:10:21 smithi144 bash[23522]: cluster 2024-04-07T16:10:20.253972+0000 mgr.y (mgr.14152) 264 : cluster 0 pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-07T16:10:23.903 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:10:23 smithi144 bash[23522]: cluster 2024-04-07T16:10:22.254534+0000 mgr.y (mgr.14152) 265 : cluster 0 pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-07T16:10:23.942 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:10:23 smithi100 bash[21346]: cluster 2024-04-07T16:10:22.254534+0000 mgr.y (mgr.14152) 265 : cluster 0 pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-07T16:10:23.942 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:10:23 smithi100 bash[31660]: cluster 2024-04-07T16:10:22.254534+0000 mgr.y (mgr.14152) 265 : cluster 0 pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-07T16:10:25.942 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:10:25 smithi100 bash[31660]: cluster 2024-04-07T16:10:24.255124+0000 mgr.y (mgr.14152) 266 : cluster 0 pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-07T16:10:25.942 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:10:25 smithi100 bash[21346]: cluster 2024-04-07T16:10:24.255124+0000 mgr.y (mgr.14152) 266 : cluster 0 pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-07T16:10:25.968 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:10:25 smithi144 bash[23522]: cluster 2024-04-07T16:10:24.255124+0000 mgr.y (mgr.14152) 266 : cluster 0 pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-07T16:10:27.942 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:10:27 smithi100 bash[21346]: cluster 2024-04-07T16:10:26.255693+0000 mgr.y (mgr.14152) 267 : cluster 0 pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-07T16:10:27.942 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:10:27 smithi100 bash[21346]: audit 2024-04-07T16:10:27.098866+0000 mon.a (mon.0) 538 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-07T16:10:27.942 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:10:27 smithi100 bash[21346]: audit 2024-04-07T16:10:27.100818+0000 mon.a (mon.0) 539 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:10:27.942 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:10:27 smithi100 bash[21346]: cephadm 2024-04-07T16:10:27.102605+0000 mgr.y (mgr.14152) 268 : cephadm 1 Deploying daemon osd.6 on smithi144 2024-04-07T16:10:27.943 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:10:27 smithi100 bash[31660]: cluster 2024-04-07T16:10:26.255693+0000 mgr.y (mgr.14152) 267 : cluster 0 pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-07T16:10:27.943 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:10:27 smithi100 bash[31660]: audit 2024-04-07T16:10:27.098866+0000 mon.a (mon.0) 538 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-07T16:10:27.943 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:10:27 smithi100 bash[31660]: audit 2024-04-07T16:10:27.100818+0000 mon.a (mon.0) 539 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:10:27.943 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:10:27 smithi100 bash[31660]: cephadm 2024-04-07T16:10:27.102605+0000 mgr.y (mgr.14152) 268 : cephadm 1 Deploying daemon osd.6 on smithi144 2024-04-07T16:10:28.065 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:10:27 smithi144 bash[23522]: cluster 2024-04-07T16:10:26.255693+0000 mgr.y (mgr.14152) 267 : cluster 0 pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-07T16:10:28.065 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:10:27 smithi144 bash[23522]: audit 2024-04-07T16:10:27.098866+0000 mon.a (mon.0) 538 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-07T16:10:28.065 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:10:27 smithi144 bash[23522]: audit 2024-04-07T16:10:27.100818+0000 mon.a (mon.0) 539 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:10:28.065 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:10:27 smithi144 bash[23522]: cephadm 2024-04-07T16:10:27.102605+0000 mgr.y (mgr.14152) 268 : cephadm 1 Deploying daemon osd.6 on smithi144 2024-04-07T16:10:29.694 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:10:29 smithi144 bash[23522]: cluster 2024-04-07T16:10:28.256204+0000 mgr.y (mgr.14152) 269 : cluster 0 pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-07T16:10:29.942 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:10:29 smithi100 bash[21346]: cluster 2024-04-07T16:10:28.256204+0000 mgr.y (mgr.14152) 269 : cluster 0 pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-07T16:10:29.942 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:10:29 smithi100 bash[31660]: cluster 2024-04-07T16:10:28.256204+0000 mgr.y (mgr.14152) 269 : cluster 0 pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-07T16:10:30.633 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:10:30 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:10:30.634 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:10:30 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:10:30.634 INFO:journalctl@ceph.osd.4.smithi144.stdout:Apr 07 16:10:30 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:10:30.634 INFO:journalctl@ceph.osd.5.smithi144.stdout:Apr 07 16:10:30 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:10:31.108 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:10:30 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:10:31.108 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:10:30 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:10:31.108 INFO:journalctl@ceph.osd.4.smithi144.stdout:Apr 07 16:10:30 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:10:31.108 INFO:journalctl@ceph.osd.5.smithi144.stdout:Apr 07 16:10:30 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:10:32.192 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:10:31 smithi100 bash[21346]: cluster 2024-04-07T16:10:30.256728+0000 mgr.y (mgr.14152) 270 : cluster 0 pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-07T16:10:32.192 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:10:31 smithi100 bash[21346]: audit 2024-04-07T16:10:30.926819+0000 mon.a (mon.0) 540 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:10:32.192 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:10:31 smithi100 bash[21346]: audit 2024-04-07T16:10:30.927453+0000 mon.a (mon.0) 541 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:10:32.192 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:10:31 smithi100 bash[21346]: audit 2024-04-07T16:10:30.938760+0000 mon.a (mon.0) 542 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:10:32.192 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:10:31 smithi100 bash[31660]: cluster 2024-04-07T16:10:30.256728+0000 mgr.y (mgr.14152) 270 : cluster 0 pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-07T16:10:32.193 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:10:31 smithi100 bash[31660]: audit 2024-04-07T16:10:30.926819+0000 mon.a (mon.0) 540 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:10:32.193 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:10:31 smithi100 bash[31660]: audit 2024-04-07T16:10:30.927453+0000 mon.a (mon.0) 541 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:10:32.193 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:10:31 smithi100 bash[31660]: audit 2024-04-07T16:10:30.938760+0000 mon.a (mon.0) 542 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:10:32.196 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:10:31 smithi144 bash[23522]: cluster 2024-04-07T16:10:30.256728+0000 mgr.y (mgr.14152) 270 : cluster 0 pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-07T16:10:32.196 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:10:31 smithi144 bash[23522]: audit 2024-04-07T16:10:30.926819+0000 mon.a (mon.0) 540 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:10:32.196 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:10:31 smithi144 bash[23522]: audit 2024-04-07T16:10:30.927453+0000 mon.a (mon.0) 541 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:10:32.196 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:10:31 smithi144 bash[23522]: audit 2024-04-07T16:10:30.938760+0000 mon.a (mon.0) 542 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:10:34.192 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:10:33 smithi100 bash[21346]: cluster 2024-04-07T16:10:32.257271+0000 mgr.y (mgr.14152) 271 : cluster 0 pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-07T16:10:34.192 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:10:33 smithi100 bash[31660]: cluster 2024-04-07T16:10:32.257271+0000 mgr.y (mgr.14152) 271 : cluster 0 pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-07T16:10:34.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:10:33 smithi144 bash[23522]: cluster 2024-04-07T16:10:32.257271+0000 mgr.y (mgr.14152) 271 : cluster 0 pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-07T16:10:36.192 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:10:35 smithi100 bash[21346]: cluster 2024-04-07T16:10:34.257857+0000 mgr.y (mgr.14152) 272 : cluster 0 pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-07T16:10:36.192 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:10:35 smithi100 bash[31660]: cluster 2024-04-07T16:10:34.257857+0000 mgr.y (mgr.14152) 272 : cluster 0 pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-07T16:10:36.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:10:35 smithi144 bash[23522]: cluster 2024-04-07T16:10:34.257857+0000 mgr.y (mgr.14152) 272 : cluster 0 pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-07T16:10:37.192 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:10:36 smithi100 bash[21346]: audit 2024-04-07T16:10:36.882075+0000 mon.b (mon.1) 19 : audit 1 from='osd.6 [v2:172.21.15.144:6816/1183840595,v1:172.21.15.144:6817/1183840595]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-07T16:10:37.192 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:10:36 smithi100 bash[21346]: audit 2024-04-07T16:10:36.885145+0000 mon.a (mon.0) 543 : audit 1 from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-07T16:10:37.192 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:10:36 smithi100 bash[31660]: audit 2024-04-07T16:10:36.882075+0000 mon.b (mon.1) 19 : audit 1 from='osd.6 [v2:172.21.15.144:6816/1183840595,v1:172.21.15.144:6817/1183840595]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-07T16:10:37.192 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:10:36 smithi100 bash[31660]: audit 2024-04-07T16:10:36.885145+0000 mon.a (mon.0) 543 : audit 1 from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-07T16:10:37.286 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:10:36 smithi144 bash[23522]: audit 2024-04-07T16:10:36.882075+0000 mon.b (mon.1) 19 : audit 1 from='osd.6 [v2:172.21.15.144:6816/1183840595,v1:172.21.15.144:6817/1183840595]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-07T16:10:37.286 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:10:36 smithi144 bash[23522]: audit 2024-04-07T16:10:36.885145+0000 mon.a (mon.0) 543 : audit 1 from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-07T16:10:38.193 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:10:37 smithi100 bash[21346]: cluster 2024-04-07T16:10:36.258426+0000 mgr.y (mgr.14152) 273 : cluster 0 pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-07T16:10:38.193 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:10:37 smithi100 bash[21346]: audit 2024-04-07T16:10:36.946089+0000 mon.a (mon.0) 544 : audit 1 from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-07T16:10:38.193 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:10:37 smithi100 bash[21346]: cluster 2024-04-07T16:10:36.946351+0000 mon.a (mon.0) 545 : cluster 0 osdmap e44: 7 total, 6 up, 7 in 2024-04-07T16:10:38.193 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:10:37 smithi100 bash[21346]: audit 2024-04-07T16:10:36.946606+0000 mon.a (mon.0) 546 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-07T16:10:38.193 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:10:37 smithi100 bash[21346]: audit 2024-04-07T16:10:36.946688+0000 mon.b (mon.1) 20 : audit 1 from='osd.6 [v2:172.21.15.144:6816/1183840595,v1:172.21.15.144:6817/1183840595]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-04-07T16:10:38.193 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:10:37 smithi100 bash[21346]: audit 2024-04-07T16:10:36.949731+0000 mon.a (mon.0) 547 : audit 1 from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-04-07T16:10:38.193 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:10:37 smithi100 bash[21346]: audit 2024-04-07T16:10:37.590927+0000 mon.a (mon.0) 548 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:10:38.193 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:10:37 smithi100 bash[21346]: audit 2024-04-07T16:10:37.600888+0000 mon.a (mon.0) 549 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:10:38.193 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:10:37 smithi100 bash[21346]: audit 2024-04-07T16:10:37.603156+0000 mon.a (mon.0) 550 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:10:38.193 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:10:37 smithi100 bash[21346]: audit 2024-04-07T16:10:37.604992+0000 mon.a (mon.0) 551 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:10:38.193 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:10:37 smithi100 bash[21346]: audit 2024-04-07T16:10:37.614806+0000 mon.a (mon.0) 552 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:10:38.193 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:10:37 smithi100 bash[31660]: cluster 2024-04-07T16:10:36.258426+0000 mgr.y (mgr.14152) 273 : cluster 0 pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-07T16:10:38.194 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:10:37 smithi100 bash[31660]: audit 2024-04-07T16:10:36.946089+0000 mon.a (mon.0) 544 : audit 1 from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-07T16:10:38.194 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:10:37 smithi100 bash[31660]: cluster 2024-04-07T16:10:36.946351+0000 mon.a (mon.0) 545 : cluster 0 osdmap e44: 7 total, 6 up, 7 in 2024-04-07T16:10:38.194 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:10:37 smithi100 bash[31660]: audit 2024-04-07T16:10:36.946606+0000 mon.a (mon.0) 546 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-07T16:10:38.194 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:10:37 smithi100 bash[31660]: audit 2024-04-07T16:10:36.946688+0000 mon.b (mon.1) 20 : audit 1 from='osd.6 [v2:172.21.15.144:6816/1183840595,v1:172.21.15.144:6817/1183840595]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-04-07T16:10:38.194 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:10:37 smithi100 bash[31660]: audit 2024-04-07T16:10:36.949731+0000 mon.a (mon.0) 547 : audit 1 from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-04-07T16:10:38.194 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:10:37 smithi100 bash[31660]: audit 2024-04-07T16:10:37.590927+0000 mon.a (mon.0) 548 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:10:38.194 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:10:37 smithi100 bash[31660]: audit 2024-04-07T16:10:37.600888+0000 mon.a (mon.0) 549 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:10:38.194 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:10:37 smithi100 bash[31660]: audit 2024-04-07T16:10:37.603156+0000 mon.a (mon.0) 550 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:10:38.194 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:10:37 smithi100 bash[31660]: audit 2024-04-07T16:10:37.604992+0000 mon.a (mon.0) 551 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:10:38.194 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:10:37 smithi100 bash[31660]: audit 2024-04-07T16:10:37.614806+0000 mon.a (mon.0) 552 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:10:38.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:10:37 smithi144 bash[23522]: cluster 2024-04-07T16:10:36.258426+0000 mgr.y (mgr.14152) 273 : cluster 0 pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-07T16:10:38.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:10:37 smithi144 bash[23522]: audit 2024-04-07T16:10:36.946089+0000 mon.a (mon.0) 544 : audit 1 from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-07T16:10:38.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:10:37 smithi144 bash[23522]: cluster 2024-04-07T16:10:36.946351+0000 mon.a (mon.0) 545 : cluster 0 osdmap e44: 7 total, 6 up, 7 in 2024-04-07T16:10:38.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:10:37 smithi144 bash[23522]: audit 2024-04-07T16:10:36.946606+0000 mon.a (mon.0) 546 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-07T16:10:38.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:10:37 smithi144 bash[23522]: audit 2024-04-07T16:10:36.946688+0000 mon.b (mon.1) 20 : audit 1 from='osd.6 [v2:172.21.15.144:6816/1183840595,v1:172.21.15.144:6817/1183840595]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-04-07T16:10:38.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:10:37 smithi144 bash[23522]: audit 2024-04-07T16:10:36.949731+0000 mon.a (mon.0) 547 : audit 1 from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-04-07T16:10:38.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:10:37 smithi144 bash[23522]: audit 2024-04-07T16:10:37.590927+0000 mon.a (mon.0) 548 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:10:38.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:10:37 smithi144 bash[23522]: audit 2024-04-07T16:10:37.600888+0000 mon.a (mon.0) 549 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:10:38.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:10:37 smithi144 bash[23522]: audit 2024-04-07T16:10:37.603156+0000 mon.a (mon.0) 550 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:10:38.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:10:37 smithi144 bash[23522]: audit 2024-04-07T16:10:37.604992+0000 mon.a (mon.0) 551 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:10:38.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:10:37 smithi144 bash[23522]: audit 2024-04-07T16:10:37.614806+0000 mon.a (mon.0) 552 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:10:39.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:10:38 smithi144 bash[23522]: audit 2024-04-07T16:10:37.949646+0000 mon.a (mon.0) 553 : audit 1 from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]': finished 2024-04-07T16:10:39.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:10:38 smithi144 bash[23522]: cluster 2024-04-07T16:10:37.949840+0000 mon.a (mon.0) 554 : cluster 0 osdmap e45: 7 total, 6 up, 7 in 2024-04-07T16:10:39.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:10:38 smithi144 bash[23522]: audit 2024-04-07T16:10:37.950577+0000 mon.a (mon.0) 555 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-07T16:10:39.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:10:38 smithi144 bash[23522]: audit 2024-04-07T16:10:37.960471+0000 mon.a (mon.0) 556 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-07T16:10:39.442 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:10:38 smithi100 bash[21346]: audit 2024-04-07T16:10:37.949646+0000 mon.a (mon.0) 553 : audit 1 from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]': finished 2024-04-07T16:10:39.442 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:10:38 smithi100 bash[21346]: cluster 2024-04-07T16:10:37.949840+0000 mon.a (mon.0) 554 : cluster 0 osdmap e45: 7 total, 6 up, 7 in 2024-04-07T16:10:39.442 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:10:38 smithi100 bash[21346]: audit 2024-04-07T16:10:37.950577+0000 mon.a (mon.0) 555 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-07T16:10:39.442 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:10:38 smithi100 bash[21346]: audit 2024-04-07T16:10:37.960471+0000 mon.a (mon.0) 556 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-07T16:10:39.442 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:10:38 smithi100 bash[31660]: audit 2024-04-07T16:10:37.949646+0000 mon.a (mon.0) 553 : audit 1 from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]': finished 2024-04-07T16:10:39.443 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:10:38 smithi100 bash[31660]: cluster 2024-04-07T16:10:37.949840+0000 mon.a (mon.0) 554 : cluster 0 osdmap e45: 7 total, 6 up, 7 in 2024-04-07T16:10:39.443 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:10:38 smithi100 bash[31660]: audit 2024-04-07T16:10:37.950577+0000 mon.a (mon.0) 555 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-07T16:10:39.443 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:10:38 smithi100 bash[31660]: audit 2024-04-07T16:10:37.960471+0000 mon.a (mon.0) 556 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-07T16:10:40.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:10:39 smithi144 bash[23522]: cluster 2024-04-07T16:10:37.856013+0000 osd.6 (osd.6) 1 : cluster 0 purged_snaps scrub starts 2024-04-07T16:10:40.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:10:39 smithi144 bash[23522]: cluster 2024-04-07T16:10:37.856139+0000 osd.6 (osd.6) 2 : cluster 0 purged_snaps scrub ok 2024-04-07T16:10:40.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:10:39 smithi144 bash[23522]: cluster 2024-04-07T16:10:38.258940+0000 mgr.y (mgr.14152) 274 : cluster 0 pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-07T16:10:40.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:10:39 smithi144 bash[23522]: cluster 2024-04-07T16:10:38.961313+0000 mon.a (mon.0) 557 : cluster 1 osd.6 [v2:172.21.15.144:6816/1183840595,v1:172.21.15.144:6817/1183840595] boot 2024-04-07T16:10:40.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:10:39 smithi144 bash[23522]: cluster 2024-04-07T16:10:38.961540+0000 mon.a (mon.0) 558 : cluster 0 osdmap e46: 7 total, 7 up, 7 in 2024-04-07T16:10:40.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:10:39 smithi144 bash[23522]: audit 2024-04-07T16:10:38.962447+0000 mon.a (mon.0) 559 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-07T16:10:40.442 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:10:39 smithi100 bash[21346]: cluster 2024-04-07T16:10:37.856013+0000 osd.6 (osd.6) 1 : cluster 0 purged_snaps scrub starts 2024-04-07T16:10:40.442 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:10:39 smithi100 bash[21346]: cluster 2024-04-07T16:10:37.856139+0000 osd.6 (osd.6) 2 : cluster 0 purged_snaps scrub ok 2024-04-07T16:10:40.442 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:10:39 smithi100 bash[21346]: cluster 2024-04-07T16:10:38.258940+0000 mgr.y (mgr.14152) 274 : cluster 0 pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-07T16:10:40.442 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:10:39 smithi100 bash[21346]: cluster 2024-04-07T16:10:38.961313+0000 mon.a (mon.0) 557 : cluster 1 osd.6 [v2:172.21.15.144:6816/1183840595,v1:172.21.15.144:6817/1183840595] boot 2024-04-07T16:10:40.442 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:10:39 smithi100 bash[21346]: cluster 2024-04-07T16:10:38.961540+0000 mon.a (mon.0) 558 : cluster 0 osdmap e46: 7 total, 7 up, 7 in 2024-04-07T16:10:40.442 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:10:39 smithi100 bash[21346]: audit 2024-04-07T16:10:38.962447+0000 mon.a (mon.0) 559 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-07T16:10:40.443 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:10:39 smithi100 bash[31660]: cluster 2024-04-07T16:10:37.856013+0000 osd.6 (osd.6) 1 : cluster 0 purged_snaps scrub starts 2024-04-07T16:10:40.443 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:10:39 smithi100 bash[31660]: cluster 2024-04-07T16:10:37.856139+0000 osd.6 (osd.6) 2 : cluster 0 purged_snaps scrub ok 2024-04-07T16:10:40.443 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:10:39 smithi100 bash[31660]: cluster 2024-04-07T16:10:38.258940+0000 mgr.y (mgr.14152) 274 : cluster 0 pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-07T16:10:40.443 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:10:39 smithi100 bash[31660]: cluster 2024-04-07T16:10:38.961313+0000 mon.a (mon.0) 557 : cluster 1 osd.6 [v2:172.21.15.144:6816/1183840595,v1:172.21.15.144:6817/1183840595] boot 2024-04-07T16:10:40.443 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:10:39 smithi100 bash[31660]: cluster 2024-04-07T16:10:38.961540+0000 mon.a (mon.0) 558 : cluster 0 osdmap e46: 7 total, 7 up, 7 in 2024-04-07T16:10:40.443 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:10:39 smithi100 bash[31660]: audit 2024-04-07T16:10:38.962447+0000 mon.a (mon.0) 559 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-07T16:10:40.531 INFO:teuthology.orchestra.run.smithi144.stdout:Created osd(s) 6 on host 'smithi144' 2024-04-07T16:10:41.084 DEBUG:teuthology.orchestra.run.smithi144:osd.6> sudo journalctl -f -n 0 -u ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@osd.6.service 2024-04-07T16:10:41.087 INFO:tasks.cephadm:Deploying osd.7 on smithi144 with /dev/vg_nvme/lv_1... 2024-04-07T16:10:41.087 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0702e3a8-f4f8-11ee-b649-cb9ed24678a4 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-07T16:10:41.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:10:40 smithi144 bash[23522]: cluster 2024-04-07T16:10:39.979567+0000 mon.a (mon.0) 560 : cluster 0 osdmap e47: 7 total, 7 up, 7 in 2024-04-07T16:10:41.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:10:40 smithi144 bash[23522]: audit 2024-04-07T16:10:40.522337+0000 mon.a (mon.0) 561 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:10:41.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:10:40 smithi144 bash[23522]: audit 2024-04-07T16:10:40.533591+0000 mon.a (mon.0) 562 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:10:41.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:10:40 smithi144 bash[23522]: audit 2024-04-07T16:10:40.559322+0000 mon.a (mon.0) 563 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:10:41.442 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:10:40 smithi100 bash[31660]: cluster 2024-04-07T16:10:39.979567+0000 mon.a (mon.0) 560 : cluster 0 osdmap e47: 7 total, 7 up, 7 in 2024-04-07T16:10:41.442 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:10:40 smithi100 bash[31660]: audit 2024-04-07T16:10:40.522337+0000 mon.a (mon.0) 561 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:10:41.442 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:10:40 smithi100 bash[31660]: audit 2024-04-07T16:10:40.533591+0000 mon.a (mon.0) 562 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:10:41.442 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:10:40 smithi100 bash[31660]: audit 2024-04-07T16:10:40.559322+0000 mon.a (mon.0) 563 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:10:41.442 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:10:40 smithi100 bash[21346]: cluster 2024-04-07T16:10:39.979567+0000 mon.a (mon.0) 560 : cluster 0 osdmap e47: 7 total, 7 up, 7 in 2024-04-07T16:10:41.442 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:10:40 smithi100 bash[21346]: audit 2024-04-07T16:10:40.522337+0000 mon.a (mon.0) 561 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:10:41.442 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:10:40 smithi100 bash[21346]: audit 2024-04-07T16:10:40.533591+0000 mon.a (mon.0) 562 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:10:41.442 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:10:40 smithi100 bash[21346]: audit 2024-04-07T16:10:40.559322+0000 mon.a (mon.0) 563 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:10:42.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:10:42 smithi144 bash[23522]: cluster 2024-04-07T16:10:40.259515+0000 mgr.y (mgr.14152) 275 : cluster 0 pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-04-07T16:10:42.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:10:42 smithi144 bash[23522]: cluster 2024-04-07T16:10:41.001928+0000 mon.a (mon.0) 564 : cluster 0 osdmap e48: 7 total, 7 up, 7 in 2024-04-07T16:10:42.442 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:10:42 smithi100 bash[21346]: cluster 2024-04-07T16:10:40.259515+0000 mgr.y (mgr.14152) 275 : cluster 0 pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-04-07T16:10:42.442 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:10:42 smithi100 bash[21346]: cluster 2024-04-07T16:10:41.001928+0000 mon.a (mon.0) 564 : cluster 0 osdmap e48: 7 total, 7 up, 7 in 2024-04-07T16:10:42.442 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:10:42 smithi100 bash[31660]: cluster 2024-04-07T16:10:40.259515+0000 mgr.y (mgr.14152) 275 : cluster 0 pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-04-07T16:10:42.442 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:10:42 smithi100 bash[31660]: cluster 2024-04-07T16:10:41.001928+0000 mon.a (mon.0) 564 : cluster 0 osdmap e48: 7 total, 7 up, 7 in 2024-04-07T16:10:43.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:10:43 smithi144 bash[23522]: cluster 2024-04-07T16:10:42.004602+0000 mon.a (mon.0) 565 : cluster 0 osdmap e49: 7 total, 7 up, 7 in 2024-04-07T16:10:43.441 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:10:43 smithi100 bash[21346]: cluster 2024-04-07T16:10:42.004602+0000 mon.a (mon.0) 565 : cluster 0 osdmap e49: 7 total, 7 up, 7 in 2024-04-07T16:10:43.442 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:10:43 smithi100 bash[31660]: cluster 2024-04-07T16:10:42.004602+0000 mon.a (mon.0) 565 : cluster 0 osdmap e49: 7 total, 7 up, 7 in 2024-04-07T16:10:44.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:10:44 smithi144 bash[23522]: cluster 2024-04-07T16:10:42.260007+0000 mgr.y (mgr.14152) 276 : cluster 0 pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-04-07T16:10:44.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:10:44 smithi144 bash[23522]: cluster 2024-04-07T16:10:43.026392+0000 mon.a (mon.0) 566 : cluster 0 osdmap e50: 7 total, 7 up, 7 in 2024-04-07T16:10:44.442 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:10:44 smithi100 bash[21346]: cluster 2024-04-07T16:10:42.260007+0000 mgr.y (mgr.14152) 276 : cluster 0 pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-04-07T16:10:44.442 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:10:44 smithi100 bash[21346]: cluster 2024-04-07T16:10:43.026392+0000 mon.a (mon.0) 566 : cluster 0 osdmap e50: 7 total, 7 up, 7 in 2024-04-07T16:10:44.442 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:10:44 smithi100 bash[31660]: cluster 2024-04-07T16:10:42.260007+0000 mgr.y (mgr.14152) 276 : cluster 0 pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-04-07T16:10:44.442 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:10:44 smithi100 bash[31660]: cluster 2024-04-07T16:10:43.026392+0000 mon.a (mon.0) 566 : cluster 0 osdmap e50: 7 total, 7 up, 7 in 2024-04-07T16:10:45.392 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/mon.b/config 2024-04-07T16:10:46.328 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:10:46 smithi144 bash[23522]: cluster 2024-04-07T16:10:44.260558+0000 mgr.y (mgr.14152) 277 : cluster 0 pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-04-07T16:10:46.441 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:10:46 smithi100 bash[21346]: cluster 2024-04-07T16:10:44.260558+0000 mgr.y (mgr.14152) 277 : cluster 0 pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-04-07T16:10:46.442 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:10:46 smithi100 bash[31660]: cluster 2024-04-07T16:10:44.260558+0000 mgr.y (mgr.14152) 277 : cluster 0 pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-04-07T16:10:48.272 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:10:48 smithi144 bash[23522]: cluster 2024-04-07T16:10:46.261208+0000 mgr.y (mgr.14152) 278 : cluster 0 pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-07T16:10:48.442 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:10:48 smithi100 bash[31660]: cluster 2024-04-07T16:10:46.261208+0000 mgr.y (mgr.14152) 278 : cluster 0 pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-07T16:10:48.442 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:10:48 smithi100 bash[21346]: cluster 2024-04-07T16:10:46.261208+0000 mgr.y (mgr.14152) 278 : cluster 0 pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-07T16:10:49.017 INFO:teuthology.orchestra.run.smithi144.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-07T16:10:49.017 INFO:teuthology.orchestra.run.smithi144.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-07T16:10:49.017 INFO:teuthology.orchestra.run.smithi144.stderr: stderr: 10+0 records in 2024-04-07T16:10:49.017 INFO:teuthology.orchestra.run.smithi144.stderr:10+0 records out 2024-04-07T16:10:49.018 INFO:teuthology.orchestra.run.smithi144.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0142275 s, 737 MB/s 2024-04-07T16:10:49.018 INFO:teuthology.orchestra.run.smithi144.stderr:--> Zapping successful for: 2024-04-07T16:10:49.728 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0702e3a8-f4f8-11ee-b649-cb9ed24678a4 -- ceph orch daemon add osd smithi144:vg_nvme/lv_1 2024-04-07T16:10:50.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:10:50 smithi144 bash[23522]: cluster 2024-04-07T16:10:48.261802+0000 mgr.y (mgr.14152) 279 : cluster 0 pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-04-07T16:10:50.441 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:10:50 smithi100 bash[21346]: cluster 2024-04-07T16:10:48.261802+0000 mgr.y (mgr.14152) 279 : cluster 0 pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-04-07T16:10:50.442 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:10:50 smithi100 bash[31660]: cluster 2024-04-07T16:10:48.261802+0000 mgr.y (mgr.14152) 279 : cluster 0 pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-04-07T16:10:51.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:10:51 smithi144 bash[23522]: cephadm 2024-04-07T16:10:50.053885+0000 mgr.y (mgr.14152) 280 : cephadm 1 Detected new or changed devices on smithi144 2024-04-07T16:10:51.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:10:51 smithi144 bash[23522]: audit 2024-04-07T16:10:50.063328+0000 mon.a (mon.0) 567 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:10:51.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:10:51 smithi144 bash[23522]: audit 2024-04-07T16:10:50.074963+0000 mon.a (mon.0) 568 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:10:51.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:10:51 smithi144 bash[23522]: audit 2024-04-07T16:10:50.077474+0000 mon.a (mon.0) 569 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-07T16:10:51.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:10:51 smithi144 bash[23522]: audit 2024-04-07T16:10:50.079392+0000 mon.a (mon.0) 570 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-07T16:10:51.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:10:51 smithi144 bash[23522]: audit 2024-04-07T16:10:50.086014+0000 mon.a (mon.0) 571 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-07T16:10:51.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:10:51 smithi144 bash[23522]: cephadm 2024-04-07T16:10:50.087483+0000 mgr.y (mgr.14152) 281 : cephadm 1 Adjusting osd_memory_target on smithi144 to 5755M 2024-04-07T16:10:51.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:10:51 smithi144 bash[23522]: audit 2024-04-07T16:10:50.098569+0000 mon.a (mon.0) 572 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:10:51.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:10:51 smithi144 bash[23522]: audit 2024-04-07T16:10:50.100359+0000 mon.a (mon.0) 573 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:10:51.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:10:51 smithi144 bash[23522]: audit 2024-04-07T16:10:50.102707+0000 mon.a (mon.0) 574 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:10:51.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:10:51 smithi144 bash[23522]: audit 2024-04-07T16:10:50.110774+0000 mon.a (mon.0) 575 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:10:51.442 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:10:51 smithi100 bash[21346]: cephadm 2024-04-07T16:10:50.053885+0000 mgr.y (mgr.14152) 280 : cephadm 1 Detected new or changed devices on smithi144 2024-04-07T16:10:51.442 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:10:51 smithi100 bash[21346]: audit 2024-04-07T16:10:50.063328+0000 mon.a (mon.0) 567 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:10:51.442 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:10:51 smithi100 bash[21346]: audit 2024-04-07T16:10:50.074963+0000 mon.a (mon.0) 568 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:10:51.442 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:10:51 smithi100 bash[21346]: audit 2024-04-07T16:10:50.077474+0000 mon.a (mon.0) 569 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-07T16:10:51.443 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:10:51 smithi100 bash[21346]: audit 2024-04-07T16:10:50.079392+0000 mon.a (mon.0) 570 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-07T16:10:51.443 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:10:51 smithi100 bash[21346]: audit 2024-04-07T16:10:50.086014+0000 mon.a (mon.0) 571 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-07T16:10:51.443 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:10:51 smithi100 bash[21346]: cephadm 2024-04-07T16:10:50.087483+0000 mgr.y (mgr.14152) 281 : cephadm 1 Adjusting osd_memory_target on smithi144 to 5755M 2024-04-07T16:10:51.443 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:10:51 smithi100 bash[21346]: audit 2024-04-07T16:10:50.098569+0000 mon.a (mon.0) 572 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:10:51.443 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:10:51 smithi100 bash[21346]: audit 2024-04-07T16:10:50.100359+0000 mon.a (mon.0) 573 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:10:51.443 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:10:51 smithi100 bash[21346]: audit 2024-04-07T16:10:50.102707+0000 mon.a (mon.0) 574 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:10:51.443 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:10:51 smithi100 bash[21346]: audit 2024-04-07T16:10:50.110774+0000 mon.a (mon.0) 575 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:10:51.443 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:10:51 smithi100 bash[31660]: cephadm 2024-04-07T16:10:50.053885+0000 mgr.y (mgr.14152) 280 : cephadm 1 Detected new or changed devices on smithi144 2024-04-07T16:10:51.443 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:10:51 smithi100 bash[31660]: audit 2024-04-07T16:10:50.063328+0000 mon.a (mon.0) 567 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:10:51.443 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:10:51 smithi100 bash[31660]: audit 2024-04-07T16:10:50.074963+0000 mon.a (mon.0) 568 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:10:51.443 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:10:51 smithi100 bash[31660]: audit 2024-04-07T16:10:50.077474+0000 mon.a (mon.0) 569 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-07T16:10:51.443 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:10:51 smithi100 bash[31660]: audit 2024-04-07T16:10:50.079392+0000 mon.a (mon.0) 570 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-07T16:10:51.443 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:10:51 smithi100 bash[31660]: audit 2024-04-07T16:10:50.086014+0000 mon.a (mon.0) 571 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-07T16:10:51.444 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:10:51 smithi100 bash[31660]: cephadm 2024-04-07T16:10:50.087483+0000 mgr.y (mgr.14152) 281 : cephadm 1 Adjusting osd_memory_target on smithi144 to 5755M 2024-04-07T16:10:51.444 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:10:51 smithi100 bash[31660]: audit 2024-04-07T16:10:50.098569+0000 mon.a (mon.0) 572 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:10:51.444 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:10:51 smithi100 bash[31660]: audit 2024-04-07T16:10:50.100359+0000 mon.a (mon.0) 573 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:10:51.444 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:10:51 smithi100 bash[31660]: audit 2024-04-07T16:10:50.102707+0000 mon.a (mon.0) 574 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:10:51.444 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:10:51 smithi100 bash[31660]: audit 2024-04-07T16:10:50.110774+0000 mon.a (mon.0) 575 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:10:52.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:10:52 smithi144 bash[23522]: cluster 2024-04-07T16:10:50.262509+0000 mgr.y (mgr.14152) 282 : cluster 0 pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-04-07T16:10:52.441 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:10:52 smithi100 bash[21346]: cluster 2024-04-07T16:10:50.262509+0000 mgr.y (mgr.14152) 282 : cluster 0 pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-04-07T16:10:52.442 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:10:52 smithi100 bash[31660]: cluster 2024-04-07T16:10:50.262509+0000 mgr.y (mgr.14152) 282 : cluster 0 pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-04-07T16:10:54.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:10:54 smithi144 bash[23522]: cluster 2024-04-07T16:10:52.263123+0000 mgr.y (mgr.14152) 283 : cluster 0 pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-07T16:10:54.425 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/mon.b/config 2024-04-07T16:10:54.441 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:10:54 smithi100 bash[21346]: cluster 2024-04-07T16:10:52.263123+0000 mgr.y (mgr.14152) 283 : cluster 0 pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-07T16:10:54.441 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:10:54 smithi100 bash[31660]: cluster 2024-04-07T16:10:52.263123+0000 mgr.y (mgr.14152) 283 : cluster 0 pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-07T16:10:56.441 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:10:56 smithi100 bash[21346]: cluster 2024-04-07T16:10:54.263691+0000 mgr.y (mgr.14152) 284 : cluster 0 pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-04-07T16:10:56.442 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:10:56 smithi100 bash[31660]: cluster 2024-04-07T16:10:54.263691+0000 mgr.y (mgr.14152) 284 : cluster 0 pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-04-07T16:10:56.445 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:10:56 smithi144 bash[23522]: cluster 2024-04-07T16:10:54.263691+0000 mgr.y (mgr.14152) 284 : cluster 0 pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-04-07T16:10:57.441 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:10:57 smithi100 bash[21346]: audit 2024-04-07T16:10:56.818288+0000 mon.a (mon.0) 576 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-07T16:10:57.442 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:10:57 smithi100 bash[21346]: audit 2024-04-07T16:10:56.824730+0000 mon.a (mon.0) 577 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-07T16:10:57.442 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:10:57 smithi100 bash[21346]: audit 2024-04-07T16:10:56.826370+0000 mon.a (mon.0) 578 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:10:57.442 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:10:57 smithi100 bash[31660]: audit 2024-04-07T16:10:56.818288+0000 mon.a (mon.0) 576 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-07T16:10:57.442 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:10:57 smithi100 bash[31660]: audit 2024-04-07T16:10:56.824730+0000 mon.a (mon.0) 577 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-07T16:10:57.442 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:10:57 smithi100 bash[31660]: audit 2024-04-07T16:10:56.826370+0000 mon.a (mon.0) 578 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:10:57.608 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:10:57 smithi144 bash[23522]: audit 2024-04-07T16:10:56.818288+0000 mon.a (mon.0) 576 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-07T16:10:57.608 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:10:57 smithi144 bash[23522]: audit 2024-04-07T16:10:56.824730+0000 mon.a (mon.0) 577 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-07T16:10:57.608 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:10:57 smithi144 bash[23522]: audit 2024-04-07T16:10:56.826370+0000 mon.a (mon.0) 578 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:10:58.441 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:10:58 smithi100 bash[21346]: cluster 2024-04-07T16:10:56.264261+0000 mgr.y (mgr.14152) 285 : cluster 0 pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-04-07T16:10:58.442 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:10:58 smithi100 bash[21346]: audit 2024-04-07T16:10:56.811168+0000 mgr.y (mgr.14152) 286 : audit 0 from='client.24259 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:10:58.442 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:10:58 smithi100 bash[31660]: cluster 2024-04-07T16:10:56.264261+0000 mgr.y (mgr.14152) 285 : cluster 0 pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-04-07T16:10:58.442 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:10:58 smithi100 bash[31660]: audit 2024-04-07T16:10:56.811168+0000 mgr.y (mgr.14152) 286 : audit 0 from='client.24259 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:10:58.608 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:10:58 smithi144 bash[23522]: cluster 2024-04-07T16:10:56.264261+0000 mgr.y (mgr.14152) 285 : cluster 0 pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-04-07T16:10:58.608 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:10:58 smithi144 bash[23522]: audit 2024-04-07T16:10:56.811168+0000 mgr.y (mgr.14152) 286 : audit 0 from='client.24259 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:11:00.441 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:11:00 smithi100 bash[21346]: cluster 2024-04-07T16:10:58.264888+0000 mgr.y (mgr.14152) 287 : cluster 0 pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail 2024-04-07T16:11:00.442 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:11:00 smithi100 bash[31660]: cluster 2024-04-07T16:10:58.264888+0000 mgr.y (mgr.14152) 287 : cluster 0 pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail 2024-04-07T16:11:00.608 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:11:00 smithi144 bash[23522]: cluster 2024-04-07T16:10:58.264888+0000 mgr.y (mgr.14152) 287 : cluster 0 pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail 2024-04-07T16:11:02.417 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:11:02 smithi144 bash[23522]: cluster 2024-04-07T16:11:00.265571+0000 mgr.y (mgr.14152) 288 : cluster 0 pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail 2024-04-07T16:11:02.441 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:11:02 smithi100 bash[31660]: cluster 2024-04-07T16:11:00.265571+0000 mgr.y (mgr.14152) 288 : cluster 0 pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail 2024-04-07T16:11:02.441 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:11:02 smithi100 bash[21346]: cluster 2024-04-07T16:11:00.265571+0000 mgr.y (mgr.14152) 288 : cluster 0 pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail 2024-04-07T16:11:04.441 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:11:04 smithi100 bash[21346]: cluster 2024-04-07T16:11:02.266233+0000 mgr.y (mgr.14152) 289 : cluster 0 pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail 2024-04-07T16:11:04.441 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:11:04 smithi100 bash[31660]: cluster 2024-04-07T16:11:02.266233+0000 mgr.y (mgr.14152) 289 : cluster 0 pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail 2024-04-07T16:11:04.608 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:11:04 smithi144 bash[23522]: cluster 2024-04-07T16:11:02.266233+0000 mgr.y (mgr.14152) 289 : cluster 0 pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail 2024-04-07T16:11:05.441 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:11:05 smithi100 bash[21346]: audit 2024-04-07T16:11:04.788608+0000 mon.b (mon.1) 21 : audit 1 from='client.? 172.21.15.144:0/2386152033' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b85b3422-9da2-4605-88e7-fe9c3b2020cb"}]: dispatch 2024-04-07T16:11:05.442 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:11:05 smithi100 bash[21346]: audit 2024-04-07T16:11:04.792275+0000 mon.a (mon.0) 579 : audit 1 from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b85b3422-9da2-4605-88e7-fe9c3b2020cb"}]: dispatch 2024-04-07T16:11:05.442 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:11:05 smithi100 bash[21346]: audit 2024-04-07T16:11:04.801318+0000 mon.a (mon.0) 580 : audit 1 from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b85b3422-9da2-4605-88e7-fe9c3b2020cb"}]': finished 2024-04-07T16:11:05.442 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:11:05 smithi100 bash[21346]: cluster 2024-04-07T16:11:04.801527+0000 mon.a (mon.0) 581 : cluster 0 osdmap e51: 8 total, 7 up, 8 in 2024-04-07T16:11:05.442 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:11:05 smithi100 bash[21346]: audit 2024-04-07T16:11:04.801786+0000 mon.a (mon.0) 582 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-07T16:11:05.442 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:11:05 smithi100 bash[31660]: audit 2024-04-07T16:11:04.788608+0000 mon.b (mon.1) 21 : audit 1 from='client.? 172.21.15.144:0/2386152033' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b85b3422-9da2-4605-88e7-fe9c3b2020cb"}]: dispatch 2024-04-07T16:11:05.442 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:11:05 smithi100 bash[31660]: audit 2024-04-07T16:11:04.792275+0000 mon.a (mon.0) 579 : audit 1 from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b85b3422-9da2-4605-88e7-fe9c3b2020cb"}]: dispatch 2024-04-07T16:11:05.442 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:11:05 smithi100 bash[31660]: audit 2024-04-07T16:11:04.801318+0000 mon.a (mon.0) 580 : audit 1 from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b85b3422-9da2-4605-88e7-fe9c3b2020cb"}]': finished 2024-04-07T16:11:05.442 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:11:05 smithi100 bash[31660]: cluster 2024-04-07T16:11:04.801527+0000 mon.a (mon.0) 581 : cluster 0 osdmap e51: 8 total, 7 up, 8 in 2024-04-07T16:11:05.442 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:11:05 smithi100 bash[31660]: audit 2024-04-07T16:11:04.801786+0000 mon.a (mon.0) 582 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-07T16:11:05.608 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:11:05 smithi144 bash[23522]: audit 2024-04-07T16:11:04.788608+0000 mon.b (mon.1) 21 : audit 1 from='client.? 172.21.15.144:0/2386152033' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b85b3422-9da2-4605-88e7-fe9c3b2020cb"}]: dispatch 2024-04-07T16:11:05.608 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:11:05 smithi144 bash[23522]: audit 2024-04-07T16:11:04.792275+0000 mon.a (mon.0) 579 : audit 1 from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b85b3422-9da2-4605-88e7-fe9c3b2020cb"}]: dispatch 2024-04-07T16:11:05.608 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:11:05 smithi144 bash[23522]: audit 2024-04-07T16:11:04.801318+0000 mon.a (mon.0) 580 : audit 1 from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b85b3422-9da2-4605-88e7-fe9c3b2020cb"}]': finished 2024-04-07T16:11:05.608 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:11:05 smithi144 bash[23522]: cluster 2024-04-07T16:11:04.801527+0000 mon.a (mon.0) 581 : cluster 0 osdmap e51: 8 total, 7 up, 8 in 2024-04-07T16:11:05.608 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:11:05 smithi144 bash[23522]: audit 2024-04-07T16:11:04.801786+0000 mon.a (mon.0) 582 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-07T16:11:06.441 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:11:06 smithi100 bash[21346]: cluster 2024-04-07T16:11:04.266941+0000 mgr.y (mgr.14152) 290 : cluster 0 pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail 2024-04-07T16:11:06.441 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:11:06 smithi100 bash[21346]: audit 2024-04-07T16:11:05.500914+0000 mon.b (mon.1) 22 : audit 0 from='client.? 172.21.15.144:0/2948495085' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-07T16:11:06.442 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:11:06 smithi100 bash[31660]: cluster 2024-04-07T16:11:04.266941+0000 mgr.y (mgr.14152) 290 : cluster 0 pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail 2024-04-07T16:11:06.442 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:11:06 smithi100 bash[31660]: audit 2024-04-07T16:11:05.500914+0000 mon.b (mon.1) 22 : audit 0 from='client.? 172.21.15.144:0/2948495085' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-07T16:11:06.608 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:11:06 smithi144 bash[23522]: cluster 2024-04-07T16:11:04.266941+0000 mgr.y (mgr.14152) 290 : cluster 0 pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail 2024-04-07T16:11:06.608 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:11:06 smithi144 bash[23522]: audit 2024-04-07T16:11:05.500914+0000 mon.b (mon.1) 22 : audit 0 from='client.? 172.21.15.144:0/2948495085' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-07T16:11:08.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:11:08 smithi144 bash[23522]: cluster 2024-04-07T16:11:06.267669+0000 mgr.y (mgr.14152) 291 : cluster 0 pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail 2024-04-07T16:11:08.441 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:11:08 smithi100 bash[31660]: cluster 2024-04-07T16:11:06.267669+0000 mgr.y (mgr.14152) 291 : cluster 0 pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail 2024-04-07T16:11:08.441 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:11:08 smithi100 bash[21346]: cluster 2024-04-07T16:11:06.267669+0000 mgr.y (mgr.14152) 291 : cluster 0 pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail 2024-04-07T16:11:10.441 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:11:10 smithi100 bash[21346]: cluster 2024-04-07T16:11:08.268238+0000 mgr.y (mgr.14152) 292 : cluster 0 pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail 2024-04-07T16:11:10.442 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:11:10 smithi100 bash[31660]: cluster 2024-04-07T16:11:08.268238+0000 mgr.y (mgr.14152) 292 : cluster 0 pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail 2024-04-07T16:11:10.608 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:11:10 smithi144 bash[23522]: cluster 2024-04-07T16:11:08.268238+0000 mgr.y (mgr.14152) 292 : cluster 0 pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail 2024-04-07T16:11:12.441 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:11:12 smithi100 bash[21346]: cluster 2024-04-07T16:11:10.268911+0000 mgr.y (mgr.14152) 293 : cluster 0 pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail 2024-04-07T16:11:12.441 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:11:12 smithi100 bash[31660]: cluster 2024-04-07T16:11:10.268911+0000 mgr.y (mgr.14152) 293 : cluster 0 pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail 2024-04-07T16:11:12.608 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:11:12 smithi144 bash[23522]: cluster 2024-04-07T16:11:10.268911+0000 mgr.y (mgr.14152) 293 : cluster 0 pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail 2024-04-07T16:11:14.441 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:11:14 smithi100 bash[21346]: cluster 2024-04-07T16:11:12.269629+0000 mgr.y (mgr.14152) 294 : cluster 0 pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail 2024-04-07T16:11:14.441 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:11:14 smithi100 bash[31660]: cluster 2024-04-07T16:11:12.269629+0000 mgr.y (mgr.14152) 294 : cluster 0 pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail 2024-04-07T16:11:14.525 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:11:14 smithi144 bash[23522]: cluster 2024-04-07T16:11:12.269629+0000 mgr.y (mgr.14152) 294 : cluster 0 pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail 2024-04-07T16:11:16.441 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:11:16 smithi100 bash[21346]: cluster 2024-04-07T16:11:14.270235+0000 mgr.y (mgr.14152) 295 : cluster 0 pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail 2024-04-07T16:11:16.441 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:11:16 smithi100 bash[31660]: cluster 2024-04-07T16:11:14.270235+0000 mgr.y (mgr.14152) 295 : cluster 0 pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail 2024-04-07T16:11:16.608 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:11:16 smithi144 bash[23522]: cluster 2024-04-07T16:11:14.270235+0000 mgr.y (mgr.14152) 295 : cluster 0 pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail 2024-04-07T16:11:17.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:11:17 smithi144 bash[23522]: audit 2024-04-07T16:11:16.729631+0000 mon.a (mon.0) 583 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-07T16:11:17.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:11:17 smithi144 bash[23522]: audit 2024-04-07T16:11:16.731402+0000 mon.a (mon.0) 584 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:11:17.441 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:11:17 smithi100 bash[21346]: audit 2024-04-07T16:11:16.729631+0000 mon.a (mon.0) 583 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-07T16:11:17.441 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:11:17 smithi100 bash[21346]: audit 2024-04-07T16:11:16.731402+0000 mon.a (mon.0) 584 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:11:17.441 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:11:17 smithi100 bash[31660]: audit 2024-04-07T16:11:16.729631+0000 mon.a (mon.0) 583 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-07T16:11:17.441 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:11:17 smithi100 bash[31660]: audit 2024-04-07T16:11:16.731402+0000 mon.a (mon.0) 584 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:11:18.441 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:11:18 smithi100 bash[21346]: cluster 2024-04-07T16:11:16.270869+0000 mgr.y (mgr.14152) 296 : cluster 0 pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail 2024-04-07T16:11:18.501 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:11:18 smithi100 bash[21346]: cephadm 2024-04-07T16:11:16.732971+0000 mgr.y (mgr.14152) 297 : cephadm 1 Deploying daemon osd.7 on smithi144 2024-04-07T16:11:18.501 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:11:18 smithi100 bash[31660]: cluster 2024-04-07T16:11:16.270869+0000 mgr.y (mgr.14152) 296 : cluster 0 pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail 2024-04-07T16:11:18.501 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:11:18 smithi100 bash[31660]: cephadm 2024-04-07T16:11:16.732971+0000 mgr.y (mgr.14152) 297 : cephadm 1 Deploying daemon osd.7 on smithi144 2024-04-07T16:11:18.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:11:18 smithi144 bash[23522]: cluster 2024-04-07T16:11:16.270869+0000 mgr.y (mgr.14152) 296 : cluster 0 pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail 2024-04-07T16:11:18.608 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:11:18 smithi144 bash[23522]: cephadm 2024-04-07T16:11:16.732971+0000 mgr.y (mgr.14152) 297 : cephadm 1 Deploying daemon osd.7 on smithi144 2024-04-07T16:11:20.441 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:11:20 smithi100 bash[21346]: cluster 2024-04-07T16:11:18.271414+0000 mgr.y (mgr.14152) 298 : cluster 0 pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail 2024-04-07T16:11:20.441 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:11:20 smithi100 bash[31660]: cluster 2024-04-07T16:11:18.271414+0000 mgr.y (mgr.14152) 298 : cluster 0 pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail 2024-04-07T16:11:20.534 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:11:20 smithi144 bash[23522]: cluster 2024-04-07T16:11:18.271414+0000 mgr.y (mgr.14152) 298 : cluster 0 pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail 2024-04-07T16:11:20.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:11:20 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:11:20.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:11:20 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:11:20.858 INFO:journalctl@ceph.osd.4.smithi144.stdout:Apr 07 16:11:20 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:11:20.858 INFO:journalctl@ceph.osd.4.smithi144.stdout:Apr 07 16:11:20 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:11:20.858 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:11:20 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:11:20.858 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:11:20 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:11:20.859 INFO:journalctl@ceph.osd.5.smithi144.stdout:Apr 07 16:11:20 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:11:20.859 INFO:journalctl@ceph.osd.5.smithi144.stdout:Apr 07 16:11:20 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:11:20.859 INFO:journalctl@ceph.osd.6.smithi144.stdout:Apr 07 16:11:20 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:11:20.859 INFO:journalctl@ceph.osd.6.smithi144.stdout:Apr 07 16:11:20 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:11:22.108 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:11:21 smithi144 bash[23522]: cluster 2024-04-07T16:11:20.271965+0000 mgr.y (mgr.14152) 299 : cluster 0 pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail 2024-04-07T16:11:22.108 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:11:21 smithi144 bash[23522]: audit 2024-04-07T16:11:20.929991+0000 mon.a (mon.0) 585 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:11:22.108 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:11:21 smithi144 bash[23522]: audit 2024-04-07T16:11:20.930456+0000 mon.a (mon.0) 586 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:11:22.108 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:11:21 smithi144 bash[23522]: audit 2024-04-07T16:11:20.943014+0000 mon.a (mon.0) 587 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:11:22.191 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:11:21 smithi100 bash[21346]: cluster 2024-04-07T16:11:20.271965+0000 mgr.y (mgr.14152) 299 : cluster 0 pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail 2024-04-07T16:11:22.191 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:11:21 smithi100 bash[21346]: audit 2024-04-07T16:11:20.929991+0000 mon.a (mon.0) 585 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:11:22.191 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:11:21 smithi100 bash[21346]: audit 2024-04-07T16:11:20.930456+0000 mon.a (mon.0) 586 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:11:22.191 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:11:21 smithi100 bash[21346]: audit 2024-04-07T16:11:20.943014+0000 mon.a (mon.0) 587 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:11:22.191 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:11:21 smithi100 bash[31660]: cluster 2024-04-07T16:11:20.271965+0000 mgr.y (mgr.14152) 299 : cluster 0 pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail 2024-04-07T16:11:22.192 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:11:21 smithi100 bash[31660]: audit 2024-04-07T16:11:20.929991+0000 mon.a (mon.0) 585 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:11:22.192 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:11:21 smithi100 bash[31660]: audit 2024-04-07T16:11:20.930456+0000 mon.a (mon.0) 586 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:11:22.192 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:11:21 smithi100 bash[31660]: audit 2024-04-07T16:11:20.943014+0000 mon.a (mon.0) 587 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:11:24.191 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:11:23 smithi100 bash[21346]: cluster 2024-04-07T16:11:22.272523+0000 mgr.y (mgr.14152) 300 : cluster 0 pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail 2024-04-07T16:11:24.191 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:11:23 smithi100 bash[31660]: cluster 2024-04-07T16:11:22.272523+0000 mgr.y (mgr.14152) 300 : cluster 0 pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail 2024-04-07T16:11:24.205 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:11:23 smithi144 bash[23522]: cluster 2024-04-07T16:11:22.272523+0000 mgr.y (mgr.14152) 300 : cluster 0 pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail 2024-04-07T16:11:26.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:11:25 smithi144 bash[23522]: cluster 2024-04-07T16:11:24.273195+0000 mgr.y (mgr.14152) 301 : cluster 0 pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail 2024-04-07T16:11:26.191 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:11:25 smithi100 bash[21346]: cluster 2024-04-07T16:11:24.273195+0000 mgr.y (mgr.14152) 301 : cluster 0 pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail 2024-04-07T16:11:26.191 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:11:25 smithi100 bash[31660]: cluster 2024-04-07T16:11:24.273195+0000 mgr.y (mgr.14152) 301 : cluster 0 pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail 2024-04-07T16:11:27.248 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:11:27 smithi144 bash[23522]: audit 2024-04-07T16:11:26.173694+0000 mon.a (mon.0) 588 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:11:27.248 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:11:27 smithi144 bash[23522]: audit 2024-04-07T16:11:26.183854+0000 mon.a (mon.0) 589 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:11:27.248 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:11:27 smithi144 bash[23522]: audit 2024-04-07T16:11:26.756707+0000 mon.a (mon.0) 590 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:11:27.248 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:11:27 smithi144 bash[23522]: audit 2024-04-07T16:11:26.758760+0000 mon.a (mon.0) 591 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:11:27.248 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:11:27 smithi144 bash[23522]: audit 2024-04-07T16:11:26.769313+0000 mon.a (mon.0) 592 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:11:27.441 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:11:27 smithi100 bash[21346]: audit 2024-04-07T16:11:26.173694+0000 mon.a (mon.0) 588 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:11:27.441 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:11:27 smithi100 bash[21346]: audit 2024-04-07T16:11:26.183854+0000 mon.a (mon.0) 589 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:11:27.441 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:11:27 smithi100 bash[21346]: audit 2024-04-07T16:11:26.756707+0000 mon.a (mon.0) 590 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:11:27.441 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:11:27 smithi100 bash[21346]: audit 2024-04-07T16:11:26.758760+0000 mon.a (mon.0) 591 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:11:27.441 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:11:27 smithi100 bash[21346]: audit 2024-04-07T16:11:26.769313+0000 mon.a (mon.0) 592 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:11:27.442 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:11:27 smithi100 bash[31660]: audit 2024-04-07T16:11:26.173694+0000 mon.a (mon.0) 588 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:11:27.442 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:11:27 smithi100 bash[31660]: audit 2024-04-07T16:11:26.183854+0000 mon.a (mon.0) 589 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:11:27.442 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:11:27 smithi100 bash[31660]: audit 2024-04-07T16:11:26.756707+0000 mon.a (mon.0) 590 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:11:27.442 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:11:27 smithi100 bash[31660]: audit 2024-04-07T16:11:26.758760+0000 mon.a (mon.0) 591 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:11:27.442 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:11:27 smithi100 bash[31660]: audit 2024-04-07T16:11:26.769313+0000 mon.a (mon.0) 592 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:11:28.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:11:28 smithi144 bash[23522]: cluster 2024-04-07T16:11:26.274042+0000 mgr.y (mgr.14152) 302 : cluster 0 pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail 2024-04-07T16:11:28.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:11:28 smithi144 bash[23522]: audit 2024-04-07T16:11:27.260241+0000 mon.b (mon.1) 23 : audit 1 from='osd.7 [v2:172.21.15.144:6824/2248617613,v1:172.21.15.144:6825/2248617613]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-07T16:11:28.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:11:28 smithi144 bash[23522]: audit 2024-04-07T16:11:27.264108+0000 mon.a (mon.0) 593 : audit 1 from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-07T16:11:28.441 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:11:28 smithi100 bash[21346]: cluster 2024-04-07T16:11:26.274042+0000 mgr.y (mgr.14152) 302 : cluster 0 pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail 2024-04-07T16:11:28.441 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:11:28 smithi100 bash[21346]: audit 2024-04-07T16:11:27.260241+0000 mon.b (mon.1) 23 : audit 1 from='osd.7 [v2:172.21.15.144:6824/2248617613,v1:172.21.15.144:6825/2248617613]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-07T16:11:28.441 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:11:28 smithi100 bash[21346]: audit 2024-04-07T16:11:27.264108+0000 mon.a (mon.0) 593 : audit 1 from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-07T16:11:28.441 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:11:28 smithi100 bash[31660]: cluster 2024-04-07T16:11:26.274042+0000 mgr.y (mgr.14152) 302 : cluster 0 pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail 2024-04-07T16:11:28.441 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:11:28 smithi100 bash[31660]: audit 2024-04-07T16:11:27.260241+0000 mon.b (mon.1) 23 : audit 1 from='osd.7 [v2:172.21.15.144:6824/2248617613,v1:172.21.15.144:6825/2248617613]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-07T16:11:28.441 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:11:28 smithi100 bash[31660]: audit 2024-04-07T16:11:27.264108+0000 mon.a (mon.0) 593 : audit 1 from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-07T16:11:29.324 INFO:teuthology.orchestra.run.smithi144.stdout:Created osd(s) 7 on host 'smithi144' 2024-04-07T16:11:29.545 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:11:29 smithi144 bash[23522]: audit 2024-04-07T16:11:28.205726+0000 mon.b (mon.1) 24 : audit 1 from='osd.7 [v2:172.21.15.144:6824/2248617613,v1:172.21.15.144:6825/2248617613]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-04-07T16:11:29.545 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:11:29 smithi144 bash[23522]: audit 2024-04-07T16:11:28.207879+0000 mon.a (mon.0) 594 : audit 1 from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-07T16:11:29.546 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:11:29 smithi144 bash[23522]: cluster 2024-04-07T16:11:28.208126+0000 mon.a (mon.0) 595 : cluster 0 osdmap e52: 8 total, 7 up, 8 in 2024-04-07T16:11:29.546 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:11:29 smithi144 bash[23522]: audit 2024-04-07T16:11:28.208430+0000 mon.a (mon.0) 596 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-07T16:11:29.546 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:11:29 smithi144 bash[23522]: audit 2024-04-07T16:11:28.209519+0000 mon.a (mon.0) 597 : audit 1 from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-04-07T16:11:29.546 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:11:29 smithi144 bash[23522]: cluster 2024-04-07T16:11:28.274578+0000 mgr.y (mgr.14152) 303 : cluster 0 pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail 2024-04-07T16:11:29.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:11:29 smithi100 bash[21346]: audit 2024-04-07T16:11:28.205726+0000 mon.b (mon.1) 24 : audit 1 from='osd.7 [v2:172.21.15.144:6824/2248617613,v1:172.21.15.144:6825/2248617613]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-04-07T16:11:29.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:11:29 smithi100 bash[21346]: audit 2024-04-07T16:11:28.207879+0000 mon.a (mon.0) 594 : audit 1 from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-07T16:11:29.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:11:29 smithi100 bash[21346]: cluster 2024-04-07T16:11:28.208126+0000 mon.a (mon.0) 595 : cluster 0 osdmap e52: 8 total, 7 up, 8 in 2024-04-07T16:11:29.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:11:29 smithi100 bash[21346]: audit 2024-04-07T16:11:28.208430+0000 mon.a (mon.0) 596 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-07T16:11:29.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:11:29 smithi100 bash[21346]: audit 2024-04-07T16:11:28.209519+0000 mon.a (mon.0) 597 : audit 1 from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-04-07T16:11:29.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:11:29 smithi100 bash[21346]: cluster 2024-04-07T16:11:28.274578+0000 mgr.y (mgr.14152) 303 : cluster 0 pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail 2024-04-07T16:11:29.692 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:11:29 smithi100 bash[31660]: audit 2024-04-07T16:11:28.205726+0000 mon.b (mon.1) 24 : audit 1 from='osd.7 [v2:172.21.15.144:6824/2248617613,v1:172.21.15.144:6825/2248617613]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-04-07T16:11:29.692 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:11:29 smithi100 bash[31660]: audit 2024-04-07T16:11:28.207879+0000 mon.a (mon.0) 594 : audit 1 from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-07T16:11:29.692 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:11:29 smithi100 bash[31660]: cluster 2024-04-07T16:11:28.208126+0000 mon.a (mon.0) 595 : cluster 0 osdmap e52: 8 total, 7 up, 8 in 2024-04-07T16:11:29.692 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:11:29 smithi100 bash[31660]: audit 2024-04-07T16:11:28.208430+0000 mon.a (mon.0) 596 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-07T16:11:29.692 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:11:29 smithi100 bash[31660]: audit 2024-04-07T16:11:28.209519+0000 mon.a (mon.0) 597 : audit 1 from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-04-07T16:11:29.692 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:11:29 smithi100 bash[31660]: cluster 2024-04-07T16:11:28.274578+0000 mgr.y (mgr.14152) 303 : cluster 0 pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail 2024-04-07T16:11:29.904 DEBUG:teuthology.orchestra.run.smithi144:osd.7> sudo journalctl -f -n 0 -u ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@osd.7.service 2024-04-07T16:11:29.907 INFO:tasks.cephadm:Waiting for 8 OSDs to come up... 2024-04-07T16:11:29.907 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0702e3a8-f4f8-11ee-b649-cb9ed24678a4 -- ceph osd stat -f json 2024-04-07T16:11:30.608 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:11:30 smithi144 bash[23522]: audit 2024-04-07T16:11:29.204663+0000 mon.a (mon.0) 598 : audit 1 from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]': finished 2024-04-07T16:11:30.608 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:11:30 smithi144 bash[23522]: cluster 2024-04-07T16:11:29.204895+0000 mon.a (mon.0) 599 : cluster 0 osdmap e53: 8 total, 7 up, 8 in 2024-04-07T16:11:30.608 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:11:30 smithi144 bash[23522]: audit 2024-04-07T16:11:29.205711+0000 mon.a (mon.0) 600 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-07T16:11:30.608 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:11:30 smithi144 bash[23522]: audit 2024-04-07T16:11:29.218887+0000 mon.a (mon.0) 601 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:11:30.608 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:11:30 smithi144 bash[23522]: audit 2024-04-07T16:11:29.220029+0000 mon.a (mon.0) 602 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-07T16:11:30.608 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:11:30 smithi144 bash[23522]: audit 2024-04-07T16:11:29.274567+0000 mon.a (mon.0) 603 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:11:30.608 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:11:30 smithi144 bash[23522]: audit 2024-04-07T16:11:29.321357+0000 mon.a (mon.0) 604 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:11:30.608 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:11:30 smithi144 bash[23522]: audit 2024-04-07T16:11:29.668353+0000 mon.a (mon.0) 605 : audit 1 from='osd.7 ' entity='osd.7' 2024-04-07T16:11:30.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:11:30 smithi100 bash[21346]: audit 2024-04-07T16:11:29.204663+0000 mon.a (mon.0) 598 : audit 1 from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]': finished 2024-04-07T16:11:30.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:11:30 smithi100 bash[21346]: cluster 2024-04-07T16:11:29.204895+0000 mon.a (mon.0) 599 : cluster 0 osdmap e53: 8 total, 7 up, 8 in 2024-04-07T16:11:30.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:11:30 smithi100 bash[21346]: audit 2024-04-07T16:11:29.205711+0000 mon.a (mon.0) 600 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-07T16:11:30.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:11:30 smithi100 bash[21346]: audit 2024-04-07T16:11:29.218887+0000 mon.a (mon.0) 601 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:11:30.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:11:30 smithi100 bash[21346]: audit 2024-04-07T16:11:29.220029+0000 mon.a (mon.0) 602 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-07T16:11:30.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:11:30 smithi100 bash[21346]: audit 2024-04-07T16:11:29.274567+0000 mon.a (mon.0) 603 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:11:30.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:11:30 smithi100 bash[21346]: audit 2024-04-07T16:11:29.321357+0000 mon.a (mon.0) 604 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:11:30.692 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:11:30 smithi100 bash[21346]: audit 2024-04-07T16:11:29.668353+0000 mon.a (mon.0) 605 : audit 1 from='osd.7 ' entity='osd.7' 2024-04-07T16:11:30.692 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:11:30 smithi100 bash[31660]: audit 2024-04-07T16:11:29.204663+0000 mon.a (mon.0) 598 : audit 1 from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]': finished 2024-04-07T16:11:30.692 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:11:30 smithi100 bash[31660]: cluster 2024-04-07T16:11:29.204895+0000 mon.a (mon.0) 599 : cluster 0 osdmap e53: 8 total, 7 up, 8 in 2024-04-07T16:11:30.692 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:11:30 smithi100 bash[31660]: audit 2024-04-07T16:11:29.205711+0000 mon.a (mon.0) 600 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-07T16:11:30.692 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:11:30 smithi100 bash[31660]: audit 2024-04-07T16:11:29.218887+0000 mon.a (mon.0) 601 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:11:30.692 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:11:30 smithi100 bash[31660]: audit 2024-04-07T16:11:29.220029+0000 mon.a (mon.0) 602 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-07T16:11:30.692 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:11:30 smithi100 bash[31660]: audit 2024-04-07T16:11:29.274567+0000 mon.a (mon.0) 603 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:11:30.692 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:11:30 smithi100 bash[31660]: audit 2024-04-07T16:11:29.321357+0000 mon.a (mon.0) 604 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:11:30.692 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:11:30 smithi100 bash[31660]: audit 2024-04-07T16:11:29.668353+0000 mon.a (mon.0) 605 : audit 1 from='osd.7 ' entity='osd.7' 2024-04-07T16:11:31.608 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:11:31 smithi144 bash[23522]: cluster 2024-04-07T16:11:28.267770+0000 osd.7 (osd.7) 1 : cluster 0 purged_snaps scrub starts 2024-04-07T16:11:31.608 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:11:31 smithi144 bash[23522]: cluster 2024-04-07T16:11:28.267874+0000 osd.7 (osd.7) 2 : cluster 0 purged_snaps scrub ok 2024-04-07T16:11:31.608 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:11:31 smithi144 bash[23522]: cluster 2024-04-07T16:11:30.219765+0000 mon.a (mon.0) 606 : cluster 1 osd.7 [v2:172.21.15.144:6824/2248617613,v1:172.21.15.144:6825/2248617613] boot 2024-04-07T16:11:31.608 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:11:31 smithi144 bash[23522]: cluster 2024-04-07T16:11:30.219922+0000 mon.a (mon.0) 607 : cluster 0 osdmap e54: 8 total, 8 up, 8 in 2024-04-07T16:11:31.608 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:11:31 smithi144 bash[23522]: audit 2024-04-07T16:11:30.220850+0000 mon.a (mon.0) 608 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-07T16:11:31.608 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:11:31 smithi144 bash[23522]: cluster 2024-04-07T16:11:30.275222+0000 mgr.y (mgr.14152) 304 : cluster 0 pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail 2024-04-07T16:11:31.608 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:11:31 smithi144 bash[23522]: cluster 2024-04-07T16:11:31.220966+0000 mon.a (mon.0) 609 : cluster 0 osdmap e55: 8 total, 8 up, 8 in 2024-04-07T16:11:31.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:11:31 smithi100 bash[21346]: cluster 2024-04-07T16:11:28.267770+0000 osd.7 (osd.7) 1 : cluster 0 purged_snaps scrub starts 2024-04-07T16:11:31.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:11:31 smithi100 bash[21346]: cluster 2024-04-07T16:11:28.267874+0000 osd.7 (osd.7) 2 : cluster 0 purged_snaps scrub ok 2024-04-07T16:11:31.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:11:31 smithi100 bash[21346]: cluster 2024-04-07T16:11:30.219765+0000 mon.a (mon.0) 606 : cluster 1 osd.7 [v2:172.21.15.144:6824/2248617613,v1:172.21.15.144:6825/2248617613] boot 2024-04-07T16:11:31.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:11:31 smithi100 bash[21346]: cluster 2024-04-07T16:11:30.219922+0000 mon.a (mon.0) 607 : cluster 0 osdmap e54: 8 total, 8 up, 8 in 2024-04-07T16:11:31.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:11:31 smithi100 bash[21346]: audit 2024-04-07T16:11:30.220850+0000 mon.a (mon.0) 608 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-07T16:11:31.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:11:31 smithi100 bash[21346]: cluster 2024-04-07T16:11:30.275222+0000 mgr.y (mgr.14152) 304 : cluster 0 pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail 2024-04-07T16:11:31.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:11:31 smithi100 bash[21346]: cluster 2024-04-07T16:11:31.220966+0000 mon.a (mon.0) 609 : cluster 0 osdmap e55: 8 total, 8 up, 8 in 2024-04-07T16:11:31.692 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:11:31 smithi100 bash[31660]: cluster 2024-04-07T16:11:28.267770+0000 osd.7 (osd.7) 1 : cluster 0 purged_snaps scrub starts 2024-04-07T16:11:31.692 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:11:31 smithi100 bash[31660]: cluster 2024-04-07T16:11:28.267874+0000 osd.7 (osd.7) 2 : cluster 0 purged_snaps scrub ok 2024-04-07T16:11:31.692 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:11:31 smithi100 bash[31660]: cluster 2024-04-07T16:11:30.219765+0000 mon.a (mon.0) 606 : cluster 1 osd.7 [v2:172.21.15.144:6824/2248617613,v1:172.21.15.144:6825/2248617613] boot 2024-04-07T16:11:31.692 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:11:31 smithi100 bash[31660]: cluster 2024-04-07T16:11:30.219922+0000 mon.a (mon.0) 607 : cluster 0 osdmap e54: 8 total, 8 up, 8 in 2024-04-07T16:11:31.692 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:11:31 smithi100 bash[31660]: audit 2024-04-07T16:11:30.220850+0000 mon.a (mon.0) 608 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-07T16:11:31.692 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:11:31 smithi100 bash[31660]: cluster 2024-04-07T16:11:30.275222+0000 mgr.y (mgr.14152) 304 : cluster 0 pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail 2024-04-07T16:11:31.692 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:11:31 smithi100 bash[31660]: cluster 2024-04-07T16:11:31.220966+0000 mon.a (mon.0) 609 : cluster 0 osdmap e55: 8 total, 8 up, 8 in 2024-04-07T16:11:33.608 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:11:33 smithi144 bash[23522]: cluster 2024-04-07T16:11:32.233018+0000 mon.a (mon.0) 610 : cluster 0 osdmap e56: 8 total, 8 up, 8 in 2024-04-07T16:11:33.608 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:11:33 smithi144 bash[23522]: cluster 2024-04-07T16:11:32.275854+0000 mgr.y (mgr.14152) 305 : cluster 0 pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:11:33.608 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:11:33 smithi144 bash[23522]: cluster 2024-04-07T16:11:32.512231+0000 mon.a (mon.0) 611 : cluster 0 osdmap e57: 8 total, 8 up, 8 in 2024-04-07T16:11:33.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:11:33 smithi100 bash[21346]: cluster 2024-04-07T16:11:32.233018+0000 mon.a (mon.0) 610 : cluster 0 osdmap e56: 8 total, 8 up, 8 in 2024-04-07T16:11:33.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:11:33 smithi100 bash[21346]: cluster 2024-04-07T16:11:32.275854+0000 mgr.y (mgr.14152) 305 : cluster 0 pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:11:33.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:11:33 smithi100 bash[21346]: cluster 2024-04-07T16:11:32.512231+0000 mon.a (mon.0) 611 : cluster 0 osdmap e57: 8 total, 8 up, 8 in 2024-04-07T16:11:33.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:11:33 smithi100 bash[31660]: cluster 2024-04-07T16:11:32.233018+0000 mon.a (mon.0) 610 : cluster 0 osdmap e56: 8 total, 8 up, 8 in 2024-04-07T16:11:33.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:11:33 smithi100 bash[31660]: cluster 2024-04-07T16:11:32.275854+0000 mgr.y (mgr.14152) 305 : cluster 0 pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:11:33.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:11:33 smithi100 bash[31660]: cluster 2024-04-07T16:11:32.512231+0000 mon.a (mon.0) 611 : cluster 0 osdmap e57: 8 total, 8 up, 8 in 2024-04-07T16:11:34.619 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/mon.a/config 2024-04-07T16:11:34.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:11:34 smithi144 bash[23522]: cluster 2024-04-07T16:11:33.514355+0000 mon.a (mon.0) 612 : cluster 0 osdmap e58: 8 total, 8 up, 8 in 2024-04-07T16:11:34.941 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:11:34 smithi100 bash[31660]: cluster 2024-04-07T16:11:33.514355+0000 mon.a (mon.0) 612 : cluster 0 osdmap e58: 8 total, 8 up, 8 in 2024-04-07T16:11:34.941 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:11:34 smithi100 bash[21346]: cluster 2024-04-07T16:11:33.514355+0000 mon.a (mon.0) 612 : cluster 0 osdmap e58: 8 total, 8 up, 8 in 2024-04-07T16:11:35.818 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:11:35 smithi100 bash[21346]: cluster 2024-04-07T16:11:34.276396+0000 mgr.y (mgr.14152) 306 : cluster 0 pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:11:35.818 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:11:35 smithi100 bash[31660]: cluster 2024-04-07T16:11:34.276396+0000 mgr.y (mgr.14152) 306 : cluster 0 pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:11:35.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:11:35 smithi144 bash[23522]: cluster 2024-04-07T16:11:34.276396+0000 mgr.y (mgr.14152) 306 : cluster 0 pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:11:37.324 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-07T16:11:37.608 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:11:37 smithi144 bash[23522]: cluster 2024-04-07T16:11:36.276877+0000 mgr.y (mgr.14152) 307 : cluster 0 pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 715 GiB / 715 GiB avail; 114 KiB/s, 0 objects/s recovering 2024-04-07T16:11:37.608 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:11:37 smithi144 bash[23522]: audit 2024-04-07T16:11:37.336574+0000 mon.c (mon.2) 16 : audit 0 from='client.? 172.21.15.100:0/3898577175' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-07T16:11:37.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:11:37 smithi100 bash[21346]: cluster 2024-04-07T16:11:36.276877+0000 mgr.y (mgr.14152) 307 : cluster 0 pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 715 GiB / 715 GiB avail; 114 KiB/s, 0 objects/s recovering 2024-04-07T16:11:37.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:11:37 smithi100 bash[21346]: audit 2024-04-07T16:11:37.336574+0000 mon.c (mon.2) 16 : audit 0 from='client.? 172.21.15.100:0/3898577175' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-07T16:11:37.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:11:37 smithi100 bash[31660]: cluster 2024-04-07T16:11:36.276877+0000 mgr.y (mgr.14152) 307 : cluster 0 pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 715 GiB / 715 GiB avail; 114 KiB/s, 0 objects/s recovering 2024-04-07T16:11:37.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:11:37 smithi100 bash[31660]: audit 2024-04-07T16:11:37.336574+0000 mon.c (mon.2) 16 : audit 0 from='client.? 172.21.15.100:0/3898577175' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-07T16:11:37.994 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":58,"num_osds":8,"num_up_osds":8,"osd_up_since":1712506290,"num_in_osds":8,"osd_in_since":1712506264,"num_remapped_pgs":0} 2024-04-07T16:11:37.994 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 0702e3a8-f4f8-11ee-b649-cb9ed24678a4 -- ceph osd dump --format=json 2024-04-07T16:11:39.108 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:11:38 smithi144 bash[23522]: cephadm 2024-04-07T16:11:37.858459+0000 mgr.y (mgr.14152) 308 : cephadm 1 Detected new or changed devices on smithi144 2024-04-07T16:11:39.108 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:11:38 smithi144 bash[23522]: audit 2024-04-07T16:11:37.869486+0000 mon.a (mon.0) 613 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:11:39.108 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:11:38 smithi144 bash[23522]: audit 2024-04-07T16:11:37.880793+0000 mon.a (mon.0) 614 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:11:39.108 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:11:38 smithi144 bash[23522]: audit 2024-04-07T16:11:37.883296+0000 mon.a (mon.0) 615 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-07T16:11:39.108 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:11:38 smithi144 bash[23522]: audit 2024-04-07T16:11:37.885021+0000 mon.a (mon.0) 616 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-07T16:11:39.108 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:11:38 smithi144 bash[23522]: audit 2024-04-07T16:11:37.886761+0000 mon.a (mon.0) 617 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-07T16:11:39.108 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:11:38 smithi144 bash[23522]: audit 2024-04-07T16:11:37.888500+0000 mon.a (mon.0) 618 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-07T16:11:39.108 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:11:38 smithi144 bash[23522]: cephadm 2024-04-07T16:11:37.889824+0000 mgr.y (mgr.14152) 309 : cephadm 1 Adjusting osd_memory_target on smithi144 to 4316M 2024-04-07T16:11:39.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:11:38 smithi144 bash[23522]: audit 2024-04-07T16:11:37.899752+0000 mon.a (mon.0) 619 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:11:39.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:11:38 smithi144 bash[23522]: audit 2024-04-07T16:11:37.901835+0000 mon.a (mon.0) 620 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:11:39.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:11:38 smithi144 bash[23522]: audit 2024-04-07T16:11:37.903524+0000 mon.a (mon.0) 621 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:11:39.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:11:38 smithi144 bash[23522]: audit 2024-04-07T16:11:37.914920+0000 mon.a (mon.0) 622 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:11:39.191 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:11:38 smithi100 bash[21346]: cephadm 2024-04-07T16:11:37.858459+0000 mgr.y (mgr.14152) 308 : cephadm 1 Detected new or changed devices on smithi144 2024-04-07T16:11:39.192 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:11:38 smithi100 bash[21346]: audit 2024-04-07T16:11:37.869486+0000 mon.a (mon.0) 613 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:11:39.192 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:11:38 smithi100 bash[21346]: audit 2024-04-07T16:11:37.880793+0000 mon.a (mon.0) 614 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:11:39.192 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:11:38 smithi100 bash[21346]: audit 2024-04-07T16:11:37.883296+0000 mon.a (mon.0) 615 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-07T16:11:39.192 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:11:38 smithi100 bash[21346]: audit 2024-04-07T16:11:37.885021+0000 mon.a (mon.0) 616 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-07T16:11:39.192 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:11:38 smithi100 bash[21346]: audit 2024-04-07T16:11:37.886761+0000 mon.a (mon.0) 617 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-07T16:11:39.192 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:11:38 smithi100 bash[21346]: audit 2024-04-07T16:11:37.888500+0000 mon.a (mon.0) 618 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-07T16:11:39.192 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:11:38 smithi100 bash[21346]: cephadm 2024-04-07T16:11:37.889824+0000 mgr.y (mgr.14152) 309 : cephadm 1 Adjusting osd_memory_target on smithi144 to 4316M 2024-04-07T16:11:39.192 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:11:38 smithi100 bash[21346]: audit 2024-04-07T16:11:37.899752+0000 mon.a (mon.0) 619 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:11:39.192 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:11:38 smithi100 bash[21346]: audit 2024-04-07T16:11:37.901835+0000 mon.a (mon.0) 620 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:11:39.193 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:11:38 smithi100 bash[21346]: audit 2024-04-07T16:11:37.903524+0000 mon.a (mon.0) 621 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:11:39.193 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:11:38 smithi100 bash[21346]: audit 2024-04-07T16:11:37.914920+0000 mon.a (mon.0) 622 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:11:39.193 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:11:38 smithi100 bash[31660]: cephadm 2024-04-07T16:11:37.858459+0000 mgr.y (mgr.14152) 308 : cephadm 1 Detected new or changed devices on smithi144 2024-04-07T16:11:39.193 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:11:38 smithi100 bash[31660]: audit 2024-04-07T16:11:37.869486+0000 mon.a (mon.0) 613 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:11:39.193 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:11:38 smithi100 bash[31660]: audit 2024-04-07T16:11:37.880793+0000 mon.a (mon.0) 614 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:11:39.193 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:11:38 smithi100 bash[31660]: audit 2024-04-07T16:11:37.883296+0000 mon.a (mon.0) 615 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-07T16:11:39.193 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:11:38 smithi100 bash[31660]: audit 2024-04-07T16:11:37.885021+0000 mon.a (mon.0) 616 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-07T16:11:39.193 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:11:38 smithi100 bash[31660]: audit 2024-04-07T16:11:37.886761+0000 mon.a (mon.0) 617 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-07T16:11:39.193 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:11:38 smithi100 bash[31660]: audit 2024-04-07T16:11:37.888500+0000 mon.a (mon.0) 618 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-07T16:11:39.193 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:11:38 smithi100 bash[31660]: cephadm 2024-04-07T16:11:37.889824+0000 mgr.y (mgr.14152) 309 : cephadm 1 Adjusting osd_memory_target on smithi144 to 4316M 2024-04-07T16:11:39.193 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:11:38 smithi100 bash[31660]: audit 2024-04-07T16:11:37.899752+0000 mon.a (mon.0) 619 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:11:39.193 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:11:38 smithi100 bash[31660]: audit 2024-04-07T16:11:37.901835+0000 mon.a (mon.0) 620 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:11:39.194 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:11:38 smithi100 bash[31660]: audit 2024-04-07T16:11:37.903524+0000 mon.a (mon.0) 621 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:11:39.194 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:11:38 smithi100 bash[31660]: audit 2024-04-07T16:11:37.914920+0000 mon.a (mon.0) 622 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:11:40.191 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:11:39 smithi100 bash[21346]: cluster 2024-04-07T16:11:38.277415+0000 mgr.y (mgr.14152) 310 : cluster 0 pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-04-07T16:11:40.191 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:11:39 smithi100 bash[31660]: cluster 2024-04-07T16:11:38.277415+0000 mgr.y (mgr.14152) 310 : cluster 0 pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-04-07T16:11:40.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:11:39 smithi144 bash[23522]: cluster 2024-04-07T16:11:38.277415+0000 mgr.y (mgr.14152) 310 : cluster 0 pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-04-07T16:11:42.190 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:11:41 smithi100 bash[21346]: cluster 2024-04-07T16:11:40.277972+0000 mgr.y (mgr.14152) 311 : cluster 0 pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-07T16:11:42.191 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:11:41 smithi100 bash[31660]: cluster 2024-04-07T16:11:40.277972+0000 mgr.y (mgr.14152) 311 : cluster 0 pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-07T16:11:42.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:11:41 smithi144 bash[23522]: cluster 2024-04-07T16:11:40.277972+0000 mgr.y (mgr.14152) 311 : cluster 0 pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-07T16:11:42.709 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/mon.a/config 2024-04-07T16:11:43.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:11:43 smithi100 bash[21346]: cluster 2024-04-07T16:11:42.278585+0000 mgr.y (mgr.14152) 312 : cluster 0 pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-04-07T16:11:43.941 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:11:43 smithi100 bash[31660]: cluster 2024-04-07T16:11:42.278585+0000 mgr.y (mgr.14152) 312 : cluster 0 pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-04-07T16:11:44.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:11:43 smithi144 bash[23522]: cluster 2024-04-07T16:11:42.278585+0000 mgr.y (mgr.14152) 312 : cluster 0 pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-04-07T16:11:45.500 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-07T16:11:45.500 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":58,"fsid":"0702e3a8-f4f8-11ee-b649-cb9ed24678a4","created":"2024-04-07T16:02:21.629692+0000","modified":"2024-04-07T16:11:33.506830+0000","last_up_change":"2024-04-07T16:11:30.210001+0000","last_in_change":"2024-04-07T16:11:04.793434+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-07T16:07:26.784071+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":7.8899998664855957,"score_stable":7.8899998664855957,"optimal_score":0.37999999523162842,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8facd48e-0a0c-4fda-aee2-f001b60a3811","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":1636950303},{"type":"v1","addr":"172.21.15.100:6803","nonce":1636950303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":1636950303},{"type":"v1","addr":"172.21.15.100:6805","nonce":1636950303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1636950303},{"type":"v1","addr":"172.21.15.100:6809","nonce":1636950303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":1636950303},{"type":"v1","addr":"172.21.15.100:6807","nonce":1636950303}]},"public_addr":"172.21.15.100:6803/1636950303","cluster_addr":"172.21.15.100:6805/1636950303","heartbeat_back_addr":"172.21.15.100:6809/1636950303","heartbeat_front_addr":"172.21.15.100:6807/1636950303","state":["exists","up"]},{"osd":1,"uuid":"cb5f6c23-6764-48b6-8b21-97a36b004786","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":470179506},{"type":"v1","addr":"172.21.15.100:6811","nonce":470179506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":470179506},{"type":"v1","addr":"172.21.15.100:6813","nonce":470179506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":470179506},{"type":"v1","addr":"172.21.15.100:6817","nonce":470179506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":470179506},{"type":"v1","addr":"172.21.15.100:6815","nonce":470179506}]},"public_addr":"172.21.15.100:6811/470179506","cluster_addr":"172.21.15.100:6813/470179506","heartbeat_back_addr":"172.21.15.100:6817/470179506","heartbeat_front_addr":"172.21.15.100:6815/470179506","state":["exists","up"]},{"osd":2,"uuid":"1650aedb-9ce9-4c14-a059-e1f089e5e63f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":2235490011},{"type":"v1","addr":"172.21.15.100:6819","nonce":2235490011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2235490011},{"type":"v1","addr":"172.21.15.100:6821","nonce":2235490011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":2235490011},{"type":"v1","addr":"172.21.15.100:6825","nonce":2235490011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":2235490011},{"type":"v1","addr":"172.21.15.100:6823","nonce":2235490011}]},"public_addr":"172.21.15.100:6819/2235490011","cluster_addr":"172.21.15.100:6821/2235490011","heartbeat_back_addr":"172.21.15.100:6825/2235490011","heartbeat_front_addr":"172.21.15.100:6823/2235490011","state":["exists","up"]},{"osd":3,"uuid":"17591c6b-862b-43b2-858b-02f4c3199247","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":3019212393},{"type":"v1","addr":"172.21.15.100:6827","nonce":3019212393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":3019212393},{"type":"v1","addr":"172.21.15.100:6829","nonce":3019212393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":3019212393},{"type":"v1","addr":"172.21.15.100:6833","nonce":3019212393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":3019212393},{"type":"v1","addr":"172.21.15.100:6831","nonce":3019212393}]},"public_addr":"172.21.15.100:6827/3019212393","cluster_addr":"172.21.15.100:6829/3019212393","heartbeat_back_addr":"172.21.15.100:6833/3019212393","heartbeat_front_addr":"172.21.15.100:6831/3019212393","state":["exists","up"]},{"osd":4,"uuid":"6ecc1205-7c78-461e-8037-640136cb3658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":974906744},{"type":"v1","addr":"172.21.15.144:6801","nonce":974906744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":974906744},{"type":"v1","addr":"172.21.15.144:6803","nonce":974906744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":974906744},{"type":"v1","addr":"172.21.15.144:6807","nonce":974906744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":974906744},{"type":"v1","addr":"172.21.15.144:6805","nonce":974906744}]},"public_addr":"172.21.15.144:6801/974906744","cluster_addr":"172.21.15.144:6803/974906744","heartbeat_back_addr":"172.21.15.144:6807/974906744","heartbeat_front_addr":"172.21.15.144:6805/974906744","state":["exists","up"]},{"osd":5,"uuid":"54789cc8-1c3c-46a4-a92e-8881564338db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":2783162058},{"type":"v1","addr":"172.21.15.144:6809","nonce":2783162058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":2783162058},{"type":"v1","addr":"172.21.15.144:6811","nonce":2783162058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":2783162058},{"type":"v1","addr":"172.21.15.144:6815","nonce":2783162058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":2783162058},{"type":"v1","addr":"172.21.15.144:6813","nonce":2783162058}]},"public_addr":"172.21.15.144:6809/2783162058","cluster_addr":"172.21.15.144:6811/2783162058","heartbeat_back_addr":"172.21.15.144:6815/2783162058","heartbeat_front_addr":"172.21.15.144:6813/2783162058","state":["exists","up"]},{"osd":6,"uuid":"ac619497-7605-4ca6-84bf-3abb3f6f32d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":1183840595},{"type":"v1","addr":"172.21.15.144:6817","nonce":1183840595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":1183840595},{"type":"v1","addr":"172.21.15.144:6819","nonce":1183840595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":1183840595},{"type":"v1","addr":"172.21.15.144:6823","nonce":1183840595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":1183840595},{"type":"v1","addr":"172.21.15.144:6821","nonce":1183840595}]},"public_addr":"172.21.15.144:6817/1183840595","cluster_addr":"172.21.15.144:6819/1183840595","heartbeat_back_addr":"172.21.15.144:6823/1183840595","heartbeat_front_addr":"172.21.15.144:6821/1183840595","state":["exists","up"]},{"osd":7,"uuid":"b85b3422-9da2-4605-88e7-fe9c3b2020cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":2248617613},{"type":"v1","addr":"172.21.15.144:6825","nonce":2248617613}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":2248617613},{"type":"v1","addr":"172.21.15.144:6827","nonce":2248617613}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":2248617613},{"type":"v1","addr":"172.21.15.144:6831","nonce":2248617613}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":2248617613},{"type":"v1","addr":"172.21.15.144:6829","nonce":2248617613}]},"public_addr":"172.21.15.144:6825/2248617613","cluster_addr":"172.21.15.144:6827/2248617613","heartbeat_back_addr":"172.21.15.144:6831/2248617613","heartbeat_front_addr":"172.21.15.144:6829/2248617613","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-07T16:05:44.696671+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-07T16:06:33.787321+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-07T16:07:22.983158+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-07T16:08:12.935606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-07T16:09:00.218619+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-07T16:09:50.222185+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-07T16:10:37.856158+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-07T16:11:28.267877+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.100:0/1472671368":"2024-04-08T16:03:24.146096+0000","172.21.15.100:0/639120693":"2024-04-08T16:03:24.146096+0000","172.21.15.100:0/2474331937":"2024-04-08T16:03:24.146096+0000","172.21.15.100:6801/2710911757":"2024-04-08T16:03:24.146096+0000","172.21.15.100:6800/3440287045":"2024-04-08T16:02:53.269514+0000","172.21.15.100:0/109071285":"2024-04-08T16:02:53.269514+0000","172.21.15.100:6801/3440287045":"2024-04-08T16:02:53.269514+0000","172.21.15.100:0/4162615562":"2024-04-08T16:02:53.269514+0000","172.21.15.100:0/4121992264":"2024-04-08T16:02:53.269514+0000","172.21.15.100:6800/2710911757":"2024-04-08T16:03:24.146096+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-07T16:11:46.123 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-07T16:07:26.784071+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 7.889999866485596, 'score_stable': 7.889999866485596, 'optimal_score': 0.3799999952316284, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-07T16:11:46.123 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 0702e3a8-f4f8-11ee-b649-cb9ed24678a4 -- ceph osd pool get .mgr pg_num 2024-04-07T16:11:46.190 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:11:45 smithi100 bash[21346]: cluster 2024-04-07T16:11:44.279212+0000 mgr.y (mgr.14152) 313 : cluster 0 pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-04-07T16:11:46.191 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:11:45 smithi100 bash[21346]: audit 2024-04-07T16:11:45.510394+0000 mon.c (mon.2) 17 : audit 0 from='client.? 172.21.15.100:0/3697485062' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-07T16:11:46.191 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:11:45 smithi100 bash[31660]: cluster 2024-04-07T16:11:44.279212+0000 mgr.y (mgr.14152) 313 : cluster 0 pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-04-07T16:11:46.191 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:11:45 smithi100 bash[31660]: audit 2024-04-07T16:11:45.510394+0000 mon.c (mon.2) 17 : audit 0 from='client.? 172.21.15.100:0/3697485062' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-07T16:11:46.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:11:45 smithi144 bash[23522]: cluster 2024-04-07T16:11:44.279212+0000 mgr.y (mgr.14152) 313 : cluster 0 pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-04-07T16:11:46.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:11:45 smithi144 bash[23522]: audit 2024-04-07T16:11:45.510394+0000 mon.c (mon.2) 17 : audit 0 from='client.? 172.21.15.100:0/3697485062' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-07T16:11:48.190 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:11:47 smithi100 bash[21346]: cluster 2024-04-07T16:11:46.279617+0000 mgr.y (mgr.14152) 314 : cluster 0 pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-04-07T16:11:48.191 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:11:47 smithi100 bash[31660]: cluster 2024-04-07T16:11:46.279617+0000 mgr.y (mgr.14152) 314 : cluster 0 pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-04-07T16:11:48.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:11:47 smithi144 bash[23522]: cluster 2024-04-07T16:11:46.279617+0000 mgr.y (mgr.14152) 314 : cluster 0 pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-04-07T16:11:50.190 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:11:49 smithi100 bash[21346]: cluster 2024-04-07T16:11:48.280185+0000 mgr.y (mgr.14152) 315 : cluster 0 pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:11:50.191 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:11:49 smithi100 bash[31660]: cluster 2024-04-07T16:11:48.280185+0000 mgr.y (mgr.14152) 315 : cluster 0 pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:11:50.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:11:49 smithi144 bash[23522]: cluster 2024-04-07T16:11:48.280185+0000 mgr.y (mgr.14152) 315 : cluster 0 pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:11:50.832 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/mon.a/config 2024-04-07T16:11:52.057 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:11:51 smithi100 bash[21346]: cluster 2024-04-07T16:11:50.280882+0000 mgr.y (mgr.14152) 316 : cluster 0 pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:11:52.057 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:11:51 smithi100 bash[31660]: cluster 2024-04-07T16:11:50.280882+0000 mgr.y (mgr.14152) 316 : cluster 0 pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:11:52.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:11:51 smithi144 bash[23522]: cluster 2024-04-07T16:11:50.280882+0000 mgr.y (mgr.14152) 316 : cluster 0 pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:11:53.585 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 1 2024-04-07T16:11:54.190 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:11:53 smithi100 bash[21346]: cluster 2024-04-07T16:11:52.281466+0000 mgr.y (mgr.14152) 317 : cluster 0 pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:11:54.191 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:11:53 smithi100 bash[21346]: audit 2024-04-07T16:11:53.597223+0000 mon.c (mon.2) 18 : audit 0 from='client.? 172.21.15.100:0/1493802322' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-07T16:11:54.191 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:11:53 smithi100 bash[31660]: cluster 2024-04-07T16:11:52.281466+0000 mgr.y (mgr.14152) 317 : cluster 0 pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:11:54.191 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:11:53 smithi100 bash[31660]: audit 2024-04-07T16:11:53.597223+0000 mon.c (mon.2) 18 : audit 0 from='client.? 172.21.15.100:0/1493802322' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-07T16:11:54.211 INFO:tasks.cephadm:Adding mds.a on smithi100 2024-04-07T16:11:54.211 INFO:tasks.cephadm:Adding mds.b on smithi144 2024-04-07T16:11:54.211 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0702e3a8-f4f8-11ee-b649-cb9ed24678a4 -- ceph orch apply mds all '2;smithi100=a;smithi144=b' 2024-04-07T16:11:54.222 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:11:53 smithi144 bash[23522]: cluster 2024-04-07T16:11:52.281466+0000 mgr.y (mgr.14152) 317 : cluster 0 pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:11:54.223 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:11:53 smithi144 bash[23522]: audit 2024-04-07T16:11:53.597223+0000 mon.c (mon.2) 18 : audit 0 from='client.? 172.21.15.100:0/1493802322' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-07T16:11:56.190 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:11:55 smithi100 bash[21346]: cluster 2024-04-07T16:11:54.282049+0000 mgr.y (mgr.14152) 318 : cluster 0 pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:11:56.190 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:11:55 smithi100 bash[31660]: cluster 2024-04-07T16:11:54.282049+0000 mgr.y (mgr.14152) 318 : cluster 0 pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:11:56.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:11:55 smithi144 bash[23522]: cluster 2024-04-07T16:11:54.282049+0000 mgr.y (mgr.14152) 318 : cluster 0 pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:11:58.190 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:11:57 smithi100 bash[21346]: cluster 2024-04-07T16:11:56.282640+0000 mgr.y (mgr.14152) 319 : cluster 0 pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:11:58.190 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:11:57 smithi100 bash[31660]: cluster 2024-04-07T16:11:56.282640+0000 mgr.y (mgr.14152) 319 : cluster 0 pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:11:58.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:11:57 smithi144 bash[23522]: cluster 2024-04-07T16:11:56.282640+0000 mgr.y (mgr.14152) 319 : cluster 0 pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:11:58.917 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/mon.b/config 2024-04-07T16:12:00.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:11:59 smithi144 bash[23522]: cluster 2024-04-07T16:11:58.283221+0000 mgr.y (mgr.14152) 320 : cluster 0 pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:12:00.440 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:11:59 smithi100 bash[21346]: cluster 2024-04-07T16:11:58.283221+0000 mgr.y (mgr.14152) 320 : cluster 0 pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:12:00.440 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:11:59 smithi100 bash[31660]: cluster 2024-04-07T16:11:58.283221+0000 mgr.y (mgr.14152) 320 : cluster 0 pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:12:01.482 INFO:teuthology.orchestra.run.smithi144.stdout:Scheduled mds.all update... 2024-04-07T16:12:02.107 DEBUG:teuthology.orchestra.run.smithi100:mds.a> sudo journalctl -f -n 0 -u ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@mds.a.service 2024-04-07T16:12:02.110 DEBUG:teuthology.orchestra.run.smithi144:mds.b> sudo journalctl -f -n 0 -u ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@mds.b.service 2024-04-07T16:12:02.152 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-04-07T16:12:02.153 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-07T16:12:02.291 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:01 smithi100 bash[21346]: cluster 2024-04-07T16:12:00.283797+0000 mgr.y (mgr.14152) 321 : cluster 0 pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:12:02.292 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:01 smithi100 bash[21346]: audit 2024-04-07T16:12:01.493231+0000 mon.a (mon.0) 623 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:12:02.292 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:01 smithi100 bash[21346]: audit 2024-04-07T16:12:01.544445+0000 mon.a (mon.0) 624 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:12:02.292 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:01 smithi100 bash[31660]: cluster 2024-04-07T16:12:00.283797+0000 mgr.y (mgr.14152) 321 : cluster 0 pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:12:02.292 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:01 smithi100 bash[31660]: audit 2024-04-07T16:12:01.493231+0000 mon.a (mon.0) 623 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:12:02.292 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:01 smithi100 bash[31660]: audit 2024-04-07T16:12:01.544445+0000 mon.a (mon.0) 624 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:12:02.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:01 smithi144 bash[23522]: cluster 2024-04-07T16:12:00.283797+0000 mgr.y (mgr.14152) 321 : cluster 0 pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:12:02.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:01 smithi144 bash[23522]: audit 2024-04-07T16:12:01.493231+0000 mon.a (mon.0) 623 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:12:02.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:01 smithi144 bash[23522]: audit 2024-04-07T16:12:01.544445+0000 mon.a (mon.0) 624 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:12:02.470 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-07T16:12:02.470 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":58,"fsid":"0702e3a8-f4f8-11ee-b649-cb9ed24678a4","created":"2024-04-07T16:02:21.629692+0000","modified":"2024-04-07T16:11:33.506830+0000","last_up_change":"2024-04-07T16:11:30.210001+0000","last_in_change":"2024-04-07T16:11:04.793434+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-07T16:07:26.784071+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":7.8899998664855957,"score_stable":7.8899998664855957,"optimal_score":0.37999999523162842,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8facd48e-0a0c-4fda-aee2-f001b60a3811","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":1636950303},{"type":"v1","addr":"172.21.15.100:6803","nonce":1636950303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":1636950303},{"type":"v1","addr":"172.21.15.100:6805","nonce":1636950303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1636950303},{"type":"v1","addr":"172.21.15.100:6809","nonce":1636950303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":1636950303},{"type":"v1","addr":"172.21.15.100:6807","nonce":1636950303}]},"public_addr":"172.21.15.100:6803/1636950303","cluster_addr":"172.21.15.100:6805/1636950303","heartbeat_back_addr":"172.21.15.100:6809/1636950303","heartbeat_front_addr":"172.21.15.100:6807/1636950303","state":["exists","up"]},{"osd":1,"uuid":"cb5f6c23-6764-48b6-8b21-97a36b004786","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":470179506},{"type":"v1","addr":"172.21.15.100:6811","nonce":470179506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":470179506},{"type":"v1","addr":"172.21.15.100:6813","nonce":470179506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":470179506},{"type":"v1","addr":"172.21.15.100:6817","nonce":470179506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":470179506},{"type":"v1","addr":"172.21.15.100:6815","nonce":470179506}]},"public_addr":"172.21.15.100:6811/470179506","cluster_addr":"172.21.15.100:6813/470179506","heartbeat_back_addr":"172.21.15.100:6817/470179506","heartbeat_front_addr":"172.21.15.100:6815/470179506","state":["exists","up"]},{"osd":2,"uuid":"1650aedb-9ce9-4c14-a059-e1f089e5e63f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":2235490011},{"type":"v1","addr":"172.21.15.100:6819","nonce":2235490011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2235490011},{"type":"v1","addr":"172.21.15.100:6821","nonce":2235490011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":2235490011},{"type":"v1","addr":"172.21.15.100:6825","nonce":2235490011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":2235490011},{"type":"v1","addr":"172.21.15.100:6823","nonce":2235490011}]},"public_addr":"172.21.15.100:6819/2235490011","cluster_addr":"172.21.15.100:6821/2235490011","heartbeat_back_addr":"172.21.15.100:6825/2235490011","heartbeat_front_addr":"172.21.15.100:6823/2235490011","state":["exists","up"]},{"osd":3,"uuid":"17591c6b-862b-43b2-858b-02f4c3199247","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":3019212393},{"type":"v1","addr":"172.21.15.100:6827","nonce":3019212393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":3019212393},{"type":"v1","addr":"172.21.15.100:6829","nonce":3019212393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":3019212393},{"type":"v1","addr":"172.21.15.100:6833","nonce":3019212393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":3019212393},{"type":"v1","addr":"172.21.15.100:6831","nonce":3019212393}]},"public_addr":"172.21.15.100:6827/3019212393","cluster_addr":"172.21.15.100:6829/3019212393","heartbeat_back_addr":"172.21.15.100:6833/3019212393","heartbeat_front_addr":"172.21.15.100:6831/3019212393","state":["exists","up"]},{"osd":4,"uuid":"6ecc1205-7c78-461e-8037-640136cb3658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":974906744},{"type":"v1","addr":"172.21.15.144:6801","nonce":974906744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":974906744},{"type":"v1","addr":"172.21.15.144:6803","nonce":974906744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":974906744},{"type":"v1","addr":"172.21.15.144:6807","nonce":974906744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":974906744},{"type":"v1","addr":"172.21.15.144:6805","nonce":974906744}]},"public_addr":"172.21.15.144:6801/974906744","cluster_addr":"172.21.15.144:6803/974906744","heartbeat_back_addr":"172.21.15.144:6807/974906744","heartbeat_front_addr":"172.21.15.144:6805/974906744","state":["exists","up"]},{"osd":5,"uuid":"54789cc8-1c3c-46a4-a92e-8881564338db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":2783162058},{"type":"v1","addr":"172.21.15.144:6809","nonce":2783162058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":2783162058},{"type":"v1","addr":"172.21.15.144:6811","nonce":2783162058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":2783162058},{"type":"v1","addr":"172.21.15.144:6815","nonce":2783162058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":2783162058},{"type":"v1","addr":"172.21.15.144:6813","nonce":2783162058}]},"public_addr":"172.21.15.144:6809/2783162058","cluster_addr":"172.21.15.144:6811/2783162058","heartbeat_back_addr":"172.21.15.144:6815/2783162058","heartbeat_front_addr":"172.21.15.144:6813/2783162058","state":["exists","up"]},{"osd":6,"uuid":"ac619497-7605-4ca6-84bf-3abb3f6f32d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":1183840595},{"type":"v1","addr":"172.21.15.144:6817","nonce":1183840595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":1183840595},{"type":"v1","addr":"172.21.15.144:6819","nonce":1183840595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":1183840595},{"type":"v1","addr":"172.21.15.144:6823","nonce":1183840595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":1183840595},{"type":"v1","addr":"172.21.15.144:6821","nonce":1183840595}]},"public_addr":"172.21.15.144:6817/1183840595","cluster_addr":"172.21.15.144:6819/1183840595","heartbeat_back_addr":"172.21.15.144:6823/1183840595","heartbeat_front_addr":"172.21.15.144:6821/1183840595","state":["exists","up"]},{"osd":7,"uuid":"b85b3422-9da2-4605-88e7-fe9c3b2020cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":2248617613},{"type":"v1","addr":"172.21.15.144:6825","nonce":2248617613}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":2248617613},{"type":"v1","addr":"172.21.15.144:6827","nonce":2248617613}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":2248617613},{"type":"v1","addr":"172.21.15.144:6831","nonce":2248617613}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":2248617613},{"type":"v1","addr":"172.21.15.144:6829","nonce":2248617613}]},"public_addr":"172.21.15.144:6825/2248617613","cluster_addr":"172.21.15.144:6827/2248617613","heartbeat_back_addr":"172.21.15.144:6831/2248617613","heartbeat_front_addr":"172.21.15.144:6829/2248617613","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-07T16:05:44.696671+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-07T16:06:33.787321+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-07T16:07:22.983158+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-07T16:08:12.935606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-07T16:09:00.218619+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-07T16:09:50.222185+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-07T16:10:37.856158+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-07T16:11:28.267877+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.100:0/1472671368":"2024-04-08T16:03:24.146096+0000","172.21.15.100:0/639120693":"2024-04-08T16:03:24.146096+0000","172.21.15.100:0/2474331937":"2024-04-08T16:03:24.146096+0000","172.21.15.100:6801/2710911757":"2024-04-08T16:03:24.146096+0000","172.21.15.100:6800/3440287045":"2024-04-08T16:02:53.269514+0000","172.21.15.100:0/109071285":"2024-04-08T16:02:53.269514+0000","172.21.15.100:6801/3440287045":"2024-04-08T16:02:53.269514+0000","172.21.15.100:0/4162615562":"2024-04-08T16:02:53.269514+0000","172.21.15.100:0/4121992264":"2024-04-08T16:02:53.269514+0000","172.21.15.100:6800/2710911757":"2024-04-08T16:03:24.146096+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-07T16:12:02.489 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-07T16:07:26.784071+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 7.889999866485596, 'score_stable': 7.889999866485596, 'optimal_score': 0.3799999952316284, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-07T16:12:02.490 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-07T16:12:02.776 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 1 2024-04-07T16:12:02.794 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-07T16:12:03.118 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-07T16:12:03.118 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-04-07T16:12:03.119 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 1 2024-04-07T16:12:03.135 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:02 smithi100 bash[21346]: audit 2024-04-07T16:12:01.481257+0000 mgr.y (mgr.14152) 322 : audit 0 from='client.24295 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "2;smithi100=a;smithi144=b", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:12:03.135 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:02 smithi100 bash[21346]: cephadm 2024-04-07T16:12:01.485323+0000 mgr.y (mgr.14152) 323 : cephadm 1 Saving service mds.all spec with placement smithi100=a;smithi144=b;count:2 2024-04-07T16:12:03.135 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:02 smithi100 bash[21346]: audit 2024-04-07T16:12:02.018194+0000 mon.a (mon.0) 625 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:12:03.135 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:02 smithi100 bash[21346]: audit 2024-04-07T16:12:02.019490+0000 mon.a (mon.0) 626 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:12:03.135 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:02 smithi100 bash[21346]: audit 2024-04-07T16:12:02.028515+0000 mon.a (mon.0) 627 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:12:03.136 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:02 smithi100 bash[21346]: audit 2024-04-07T16:12:02.032087+0000 mon.a (mon.0) 628 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-07T16:12:03.136 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:02 smithi100 bash[21346]: audit 2024-04-07T16:12:02.038597+0000 mon.a (mon.0) 629 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-07T16:12:03.136 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:02 smithi100 bash[21346]: audit 2024-04-07T16:12:02.040045+0000 mon.a (mon.0) 630 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:12:03.136 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:02 smithi100 bash[21346]: cephadm 2024-04-07T16:12:02.041613+0000 mgr.y (mgr.14152) 324 : cephadm 1 Deploying daemon mds.a on smithi100 2024-04-07T16:12:03.136 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:02 smithi100 bash[21346]: audit 2024-04-07T16:12:02.481898+0000 mon.a (mon.0) 631 : audit 0 from='client.? 172.21.15.100:0/2535342497' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-07T16:12:03.136 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:02 smithi100 bash[21346]: audit 2024-04-07T16:12:02.789197+0000 mon.c (mon.2) 19 : audit 0 from='client.? 172.21.15.100:0/1524051718' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-07T16:12:03.137 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:02 smithi100 bash[31660]: audit 2024-04-07T16:12:01.481257+0000 mgr.y (mgr.14152) 322 : audit 0 from='client.24295 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "2;smithi100=a;smithi144=b", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:12:03.137 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:02 smithi100 bash[31660]: cephadm 2024-04-07T16:12:01.485323+0000 mgr.y (mgr.14152) 323 : cephadm 1 Saving service mds.all spec with placement smithi100=a;smithi144=b;count:2 2024-04-07T16:12:03.137 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:02 smithi100 bash[31660]: audit 2024-04-07T16:12:02.018194+0000 mon.a (mon.0) 625 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:12:03.137 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:02 smithi100 bash[31660]: audit 2024-04-07T16:12:02.019490+0000 mon.a (mon.0) 626 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:12:03.137 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:02 smithi100 bash[31660]: audit 2024-04-07T16:12:02.028515+0000 mon.a (mon.0) 627 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:12:03.137 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:02 smithi100 bash[31660]: audit 2024-04-07T16:12:02.032087+0000 mon.a (mon.0) 628 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-07T16:12:03.137 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:02 smithi100 bash[31660]: audit 2024-04-07T16:12:02.038597+0000 mon.a (mon.0) 629 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-07T16:12:03.137 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:02 smithi100 bash[31660]: audit 2024-04-07T16:12:02.040045+0000 mon.a (mon.0) 630 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:12:03.137 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:02 smithi100 bash[31660]: cephadm 2024-04-07T16:12:02.041613+0000 mgr.y (mgr.14152) 324 : cephadm 1 Deploying daemon mds.a on smithi100 2024-04-07T16:12:03.138 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:02 smithi100 bash[31660]: audit 2024-04-07T16:12:02.481898+0000 mon.a (mon.0) 631 : audit 0 from='client.? 172.21.15.100:0/2535342497' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-07T16:12:03.138 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:02 smithi100 bash[31660]: audit 2024-04-07T16:12:02.789197+0000 mon.c (mon.2) 19 : audit 0 from='client.? 172.21.15.100:0/1524051718' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-07T16:12:03.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:02 smithi144 bash[23522]: audit 2024-04-07T16:12:01.481257+0000 mgr.y (mgr.14152) 322 : audit 0 from='client.24295 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "2;smithi100=a;smithi144=b", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:12:03.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:02 smithi144 bash[23522]: cephadm 2024-04-07T16:12:01.485323+0000 mgr.y (mgr.14152) 323 : cephadm 1 Saving service mds.all spec with placement smithi100=a;smithi144=b;count:2 2024-04-07T16:12:03.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:02 smithi144 bash[23522]: audit 2024-04-07T16:12:02.018194+0000 mon.a (mon.0) 625 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:12:03.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:02 smithi144 bash[23522]: audit 2024-04-07T16:12:02.019490+0000 mon.a (mon.0) 626 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:12:03.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:02 smithi144 bash[23522]: audit 2024-04-07T16:12:02.028515+0000 mon.a (mon.0) 627 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:12:03.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:02 smithi144 bash[23522]: audit 2024-04-07T16:12:02.032087+0000 mon.a (mon.0) 628 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-07T16:12:03.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:02 smithi144 bash[23522]: audit 2024-04-07T16:12:02.038597+0000 mon.a (mon.0) 629 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-07T16:12:03.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:02 smithi144 bash[23522]: audit 2024-04-07T16:12:02.040045+0000 mon.a (mon.0) 630 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:12:03.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:02 smithi144 bash[23522]: cephadm 2024-04-07T16:12:02.041613+0000 mgr.y (mgr.14152) 324 : cephadm 1 Deploying daemon mds.a on smithi100 2024-04-07T16:12:03.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:02 smithi144 bash[23522]: audit 2024-04-07T16:12:02.481898+0000 mon.a (mon.0) 631 : audit 0 from='client.? 172.21.15.100:0/2535342497' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-07T16:12:03.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:02 smithi144 bash[23522]: audit 2024-04-07T16:12:02.789197+0000 mon.c (mon.2) 19 : audit 0 from='client.? 172.21.15.100:0/1524051718' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-07T16:12:04.225 INFO:journalctl@ceph.osd.1.smithi100.stdout:Apr 07 16:12:04 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:12:04.225 INFO:journalctl@ceph.osd.0.smithi100.stdout:Apr 07 16:12:04 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:12:04.226 INFO:journalctl@ceph.osd.2.smithi100.stdout:Apr 07 16:12:04 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:12:04.226 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:04 smithi100 bash[21346]: cluster 2024-04-07T16:12:02.284183+0000 mgr.y (mgr.14152) 325 : cluster 0 pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:12:04.226 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:04 smithi100 bash[21346]: audit 2024-04-07T16:12:03.130766+0000 mon.c (mon.2) 20 : audit 0 from='client.? 172.21.15.100:0/4000278539' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-07T16:12:04.226 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:04 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:12:04.226 INFO:journalctl@ceph.mds.a.smithi100.stdout:Apr 07 16:12:04 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:12:04.227 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:12:04 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:12:04.227 INFO:journalctl@ceph.osd.3.smithi100.stdout:Apr 07 16:12:04 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:12:04.227 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:04 smithi100 bash[31660]: cluster 2024-04-07T16:12:02.284183+0000 mgr.y (mgr.14152) 325 : cluster 0 pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:12:04.227 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:04 smithi100 bash[31660]: audit 2024-04-07T16:12:03.130766+0000 mon.c (mon.2) 20 : audit 0 from='client.? 172.21.15.100:0/4000278539' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-07T16:12:04.228 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:04 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:12:04.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:04 smithi144 bash[23522]: cluster 2024-04-07T16:12:02.284183+0000 mgr.y (mgr.14152) 325 : cluster 0 pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:12:04.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:04 smithi144 bash[23522]: audit 2024-04-07T16:12:03.130766+0000 mon.c (mon.2) 20 : audit 0 from='client.? 172.21.15.100:0/4000278539' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-07T16:12:04.497 INFO:journalctl@ceph.osd.1.smithi100.stdout:Apr 07 16:12:04 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:12:04.497 INFO:journalctl@ceph.osd.2.smithi100.stdout:Apr 07 16:12:04 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:12:04.498 INFO:journalctl@ceph.mds.a.smithi100.stdout:Apr 07 16:12:04 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:12:04.498 INFO:journalctl@ceph.mds.a.smithi100.stdout:Apr 07 16:12:04 smithi100 systemd[1]: Started Ceph mds.a for 0702e3a8-f4f8-11ee-b649-cb9ed24678a4. 2024-04-07T16:12:04.498 INFO:journalctl@ceph.osd.3.smithi100.stdout:Apr 07 16:12:04 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:12:04.498 INFO:journalctl@ceph.osd.0.smithi100.stdout:Apr 07 16:12:04 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:12:04.499 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:04 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:12:04.499 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:04 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:12:04.499 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:12:04 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:12:05.138 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-07T16:12:05.491 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-07T16:12:05.491 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14424,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/3487006602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3487006602},{"type":"v1","addr":"172.21.15.100:6835","nonce":3487006602}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[]} 2024-04-07T16:12:05.491 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 2 2024-04-07T16:12:05.690 INFO:journalctl@ceph.mds.a.smithi100.stdout:Apr 07 16:12:05 smithi100 bash[56138]: starting mds.a at 2024-04-07T16:12:05.690 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:05 smithi100 bash[21346]: cluster 2024-04-07T16:12:04.284515+0000 mgr.y (mgr.14152) 326 : cluster 0 pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:12:05.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:05 smithi100 bash[21346]: audit 2024-04-07T16:12:04.497371+0000 mon.a (mon.0) 632 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:12:05.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:05 smithi100 bash[21346]: audit 2024-04-07T16:12:04.504631+0000 mon.a (mon.0) 633 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:12:05.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:05 smithi100 bash[21346]: audit 2024-04-07T16:12:04.513994+0000 mon.a (mon.0) 634 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:12:05.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:05 smithi100 bash[21346]: audit 2024-04-07T16:12:04.514618+0000 mon.a (mon.0) 635 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-07T16:12:05.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:05 smithi100 bash[21346]: audit 2024-04-07T16:12:04.520259+0000 mon.a (mon.0) 636 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-07T16:12:05.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:05 smithi100 bash[21346]: audit 2024-04-07T16:12:04.520949+0000 mon.a (mon.0) 637 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:12:05.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:05 smithi100 bash[21346]: cephadm 2024-04-07T16:12:04.521719+0000 mgr.y (mgr.14152) 327 : cephadm 1 Deploying daemon mds.b on smithi144 2024-04-07T16:12:05.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:05 smithi100 bash[21346]: cluster 2024-04-07T16:12:05.439144+0000 mon.a (mon.0) 638 : cluster 0 mds.? [v2:172.21.15.100:6834/3487006602,v1:172.21.15.100:6835/3487006602] up:boot 2024-04-07T16:12:05.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:05 smithi100 bash[21346]: cluster 2024-04-07T16:12:05.439178+0000 mon.a (mon.0) 639 : cluster 0 fsmap 1 up:standby 2024-04-07T16:12:05.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:05 smithi100 bash[21346]: audit 2024-04-07T16:12:05.439286+0000 mon.a (mon.0) 640 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-07T16:12:05.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:05 smithi100 bash[31660]: cluster 2024-04-07T16:12:04.284515+0000 mgr.y (mgr.14152) 326 : cluster 0 pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:12:05.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:05 smithi100 bash[31660]: audit 2024-04-07T16:12:04.497371+0000 mon.a (mon.0) 632 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:12:05.692 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:05 smithi100 bash[31660]: audit 2024-04-07T16:12:04.504631+0000 mon.a (mon.0) 633 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:12:05.692 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:05 smithi100 bash[31660]: audit 2024-04-07T16:12:04.513994+0000 mon.a (mon.0) 634 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:12:05.692 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:05 smithi100 bash[31660]: audit 2024-04-07T16:12:04.514618+0000 mon.a (mon.0) 635 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-07T16:12:05.692 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:05 smithi100 bash[31660]: audit 2024-04-07T16:12:04.520259+0000 mon.a (mon.0) 636 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-07T16:12:05.692 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:05 smithi100 bash[31660]: audit 2024-04-07T16:12:04.520949+0000 mon.a (mon.0) 637 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:12:05.692 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:05 smithi100 bash[31660]: cephadm 2024-04-07T16:12:04.521719+0000 mgr.y (mgr.14152) 327 : cephadm 1 Deploying daemon mds.b on smithi144 2024-04-07T16:12:05.692 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:05 smithi100 bash[31660]: cluster 2024-04-07T16:12:05.439144+0000 mon.a (mon.0) 638 : cluster 0 mds.? [v2:172.21.15.100:6834/3487006602,v1:172.21.15.100:6835/3487006602] up:boot 2024-04-07T16:12:05.692 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:05 smithi100 bash[31660]: cluster 2024-04-07T16:12:05.439178+0000 mon.a (mon.0) 639 : cluster 0 fsmap 1 up:standby 2024-04-07T16:12:05.692 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:05 smithi100 bash[31660]: audit 2024-04-07T16:12:05.439286+0000 mon.a (mon.0) 640 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-07T16:12:05.828 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:05 smithi144 bash[23522]: cluster 2024-04-07T16:12:04.284515+0000 mgr.y (mgr.14152) 326 : cluster 0 pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:12:05.828 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:05 smithi144 bash[23522]: audit 2024-04-07T16:12:04.497371+0000 mon.a (mon.0) 632 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:12:05.828 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:05 smithi144 bash[23522]: audit 2024-04-07T16:12:04.504631+0000 mon.a (mon.0) 633 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:12:05.828 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:05 smithi144 bash[23522]: audit 2024-04-07T16:12:04.513994+0000 mon.a (mon.0) 634 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:12:05.828 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:05 smithi144 bash[23522]: audit 2024-04-07T16:12:04.514618+0000 mon.a (mon.0) 635 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-07T16:12:05.828 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:05 smithi144 bash[23522]: audit 2024-04-07T16:12:04.520259+0000 mon.a (mon.0) 636 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-07T16:12:05.829 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:05 smithi144 bash[23522]: audit 2024-04-07T16:12:04.520949+0000 mon.a (mon.0) 637 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:12:05.829 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:05 smithi144 bash[23522]: cephadm 2024-04-07T16:12:04.521719+0000 mgr.y (mgr.14152) 327 : cephadm 1 Deploying daemon mds.b on smithi144 2024-04-07T16:12:05.829 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:05 smithi144 bash[23522]: cluster 2024-04-07T16:12:05.439144+0000 mon.a (mon.0) 638 : cluster 0 mds.? [v2:172.21.15.100:6834/3487006602,v1:172.21.15.100:6835/3487006602] up:boot 2024-04-07T16:12:05.829 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:05 smithi144 bash[23522]: cluster 2024-04-07T16:12:05.439178+0000 mon.a (mon.0) 639 : cluster 0 fsmap 1 up:standby 2024-04-07T16:12:05.829 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:05 smithi144 bash[23522]: audit 2024-04-07T16:12:05.439286+0000 mon.a (mon.0) 640 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-07T16:12:06.630 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:12:06 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:12:06.630 INFO:journalctl@ceph.osd.4.smithi144.stdout:Apr 07 16:12:06 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:12:06.631 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:06 smithi144 bash[23522]: audit 2024-04-07T16:12:05.502717+0000 mon.c (mon.2) 21 : audit 0 from='client.? 172.21.15.100:0/2350130947' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-07T16:12:06.631 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:06 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:12:06.631 INFO:journalctl@ceph.osd.6.smithi144.stdout:Apr 07 16:12:06 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:12:06.631 INFO:journalctl@ceph.osd.5.smithi144.stdout:Apr 07 16:12:06 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:12:06.632 INFO:journalctl@ceph.osd.7.smithi144.stdout:Apr 07 16:12:06 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:12:06.898 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:06 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:12:06.898 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:12:06 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:12:06.899 INFO:journalctl@ceph.osd.4.smithi144.stdout:Apr 07 16:12:06 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:12:06.899 INFO:journalctl@ceph.osd.6.smithi144.stdout:Apr 07 16:12:06 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:12:06.899 INFO:journalctl@ceph.osd.7.smithi144.stdout:Apr 07 16:12:06 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:12:06.900 INFO:journalctl@ceph.mds.b.smithi144.stdout:Apr 07 16:12:06 smithi144 systemd[1]: Started Ceph mds.b for 0702e3a8-f4f8-11ee-b649-cb9ed24678a4. 2024-04-07T16:12:06.900 INFO:journalctl@ceph.osd.5.smithi144.stdout:Apr 07 16:12:06 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:12:06.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:06 smithi100 bash[21346]: audit 2024-04-07T16:12:05.502717+0000 mon.c (mon.2) 21 : audit 0 from='client.? 172.21.15.100:0/2350130947' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-07T16:12:06.940 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:06 smithi100 bash[31660]: audit 2024-04-07T16:12:05.502717+0000 mon.c (mon.2) 21 : audit 0 from='client.? 172.21.15.100:0/2350130947' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-07T16:12:07.510 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-07T16:12:07.855 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-07T16:12:07.855 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14424,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/3487006602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3487006602},{"type":"v1","addr":"172.21.15.100:6835","nonce":3487006602}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[]} 2024-04-07T16:12:07.855 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 2 2024-04-07T16:12:07.894 INFO:journalctl@ceph.mds.b.smithi144.stdout:Apr 07 16:12:07 smithi144 bash[47566]: starting mds.b at 2024-04-07T16:12:08.191 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:07 smithi100 bash[21346]: cluster 2024-04-07T16:12:06.285049+0000 mgr.y (mgr.14152) 328 : cluster 0 pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:12:08.191 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:07 smithi100 bash[21346]: audit 2024-04-07T16:12:06.910649+0000 mon.a (mon.0) 641 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:12:08.191 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:07 smithi100 bash[21346]: audit 2024-04-07T16:12:06.922838+0000 mon.a (mon.0) 642 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:12:08.191 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:07 smithi100 bash[21346]: audit 2024-04-07T16:12:06.951709+0000 mon.a (mon.0) 643 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:12:08.191 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:07 smithi100 bash[21346]: audit 2024-04-07T16:12:06.966322+0000 mon.a (mon.0) 644 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:12:08.191 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:07 smithi100 bash[21346]: audit 2024-04-07T16:12:06.978151+0000 mon.a (mon.0) 645 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:12:08.192 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:07 smithi100 bash[21346]: audit 2024-04-07T16:12:07.044505+0000 mon.a (mon.0) 646 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:12:08.192 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:07 smithi100 bash[21346]: audit 2024-04-07T16:12:07.867386+0000 mon.a (mon.0) 647 : audit 0 from='client.? 172.21.15.100:0/4102447640' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-07T16:12:08.192 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:07 smithi100 bash[31660]: cluster 2024-04-07T16:12:06.285049+0000 mgr.y (mgr.14152) 328 : cluster 0 pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:12:08.192 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:07 smithi100 bash[31660]: audit 2024-04-07T16:12:06.910649+0000 mon.a (mon.0) 641 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:12:08.192 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:07 smithi100 bash[31660]: audit 2024-04-07T16:12:06.922838+0000 mon.a (mon.0) 642 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:12:08.192 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:07 smithi100 bash[31660]: audit 2024-04-07T16:12:06.951709+0000 mon.a (mon.0) 643 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:12:08.192 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:07 smithi100 bash[31660]: audit 2024-04-07T16:12:06.966322+0000 mon.a (mon.0) 644 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:12:08.192 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:07 smithi100 bash[31660]: audit 2024-04-07T16:12:06.978151+0000 mon.a (mon.0) 645 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:12:08.192 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:07 smithi100 bash[31660]: audit 2024-04-07T16:12:07.044505+0000 mon.a (mon.0) 646 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:12:08.192 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:07 smithi100 bash[31660]: audit 2024-04-07T16:12:07.867386+0000 mon.a (mon.0) 647 : audit 0 from='client.? 172.21.15.100:0/4102447640' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-07T16:12:08.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:07 smithi144 bash[23522]: cluster 2024-04-07T16:12:06.285049+0000 mgr.y (mgr.14152) 328 : cluster 0 pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:12:08.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:07 smithi144 bash[23522]: audit 2024-04-07T16:12:06.910649+0000 mon.a (mon.0) 641 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:12:08.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:07 smithi144 bash[23522]: audit 2024-04-07T16:12:06.922838+0000 mon.a (mon.0) 642 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:12:08.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:07 smithi144 bash[23522]: audit 2024-04-07T16:12:06.951709+0000 mon.a (mon.0) 643 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:12:08.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:07 smithi144 bash[23522]: audit 2024-04-07T16:12:06.966322+0000 mon.a (mon.0) 644 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:12:08.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:07 smithi144 bash[23522]: audit 2024-04-07T16:12:06.978151+0000 mon.a (mon.0) 645 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:12:08.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:07 smithi144 bash[23522]: audit 2024-04-07T16:12:07.044505+0000 mon.a (mon.0) 646 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:12:08.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:07 smithi144 bash[23522]: audit 2024-04-07T16:12:07.867386+0000 mon.a (mon.0) 647 : audit 0 from='client.? 172.21.15.100:0/4102447640' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-07T16:12:09.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:08 smithi144 bash[23522]: cluster 2024-04-07T16:12:07.980115+0000 mon.a (mon.0) 648 : cluster 0 mds.? [v2:172.21.15.144:6832/226244023,v1:172.21.15.144:6833/226244023] up:boot 2024-04-07T16:12:09.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:08 smithi144 bash[23522]: cluster 2024-04-07T16:12:07.980238+0000 mon.a (mon.0) 649 : cluster 0 fsmap 2 up:standby 2024-04-07T16:12:09.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:08 smithi144 bash[23522]: audit 2024-04-07T16:12:07.980428+0000 mon.a (mon.0) 650 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-07T16:12:09.441 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:08 smithi100 bash[21346]: cluster 2024-04-07T16:12:07.980115+0000 mon.a (mon.0) 648 : cluster 0 mds.? [v2:172.21.15.144:6832/226244023,v1:172.21.15.144:6833/226244023] up:boot 2024-04-07T16:12:09.442 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:08 smithi100 bash[21346]: cluster 2024-04-07T16:12:07.980238+0000 mon.a (mon.0) 649 : cluster 0 fsmap 2 up:standby 2024-04-07T16:12:09.442 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:08 smithi100 bash[21346]: audit 2024-04-07T16:12:07.980428+0000 mon.a (mon.0) 650 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-07T16:12:09.442 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:08 smithi100 bash[31660]: cluster 2024-04-07T16:12:07.980115+0000 mon.a (mon.0) 648 : cluster 0 mds.? [v2:172.21.15.144:6832/226244023,v1:172.21.15.144:6833/226244023] up:boot 2024-04-07T16:12:09.442 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:08 smithi100 bash[31660]: cluster 2024-04-07T16:12:07.980238+0000 mon.a (mon.0) 649 : cluster 0 fsmap 2 up:standby 2024-04-07T16:12:09.442 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:08 smithi100 bash[31660]: audit 2024-04-07T16:12:07.980428+0000 mon.a (mon.0) 650 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-07T16:12:09.875 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-07T16:12:10.190 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:09 smithi100 bash[21346]: cluster 2024-04-07T16:12:08.285714+0000 mgr.y (mgr.14152) 329 : cluster 0 pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:12:10.190 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:09 smithi100 bash[21346]: audit 2024-04-07T16:12:09.919360+0000 mon.a (mon.0) 651 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:12:10.190 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:09 smithi100 bash[31660]: cluster 2024-04-07T16:12:08.285714+0000 mgr.y (mgr.14152) 329 : cluster 0 pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:12:10.190 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:09 smithi100 bash[31660]: audit 2024-04-07T16:12:09.919360+0000 mon.a (mon.0) 651 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:12:10.207 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-07T16:12:10.207 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14424,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/3487006602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3487006602},{"type":"v1","addr":"172.21.15.100:6835","nonce":3487006602}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24322,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/226244023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":226244023},{"type":"v1","addr":"172.21.15.144:6833","nonce":226244023}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2024-04-07T16:12:10.207 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 3 2024-04-07T16:12:10.224 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-07T16:12:10.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:09 smithi144 bash[23522]: cluster 2024-04-07T16:12:08.285714+0000 mgr.y (mgr.14152) 329 : cluster 0 pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:12:10.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:09 smithi144 bash[23522]: audit 2024-04-07T16:12:09.919360+0000 mon.a (mon.0) 651 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:12:10.505 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-07T16:12:10.505 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":58,"fsid":"0702e3a8-f4f8-11ee-b649-cb9ed24678a4","created":"2024-04-07T16:02:21.629692+0000","modified":"2024-04-07T16:11:33.506830+0000","last_up_change":"2024-04-07T16:11:30.210001+0000","last_in_change":"2024-04-07T16:11:04.793434+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-07T16:07:26.784071+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":7.8899998664855957,"score_stable":7.8899998664855957,"optimal_score":0.37999999523162842,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8facd48e-0a0c-4fda-aee2-f001b60a3811","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":1636950303},{"type":"v1","addr":"172.21.15.100:6803","nonce":1636950303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":1636950303},{"type":"v1","addr":"172.21.15.100:6805","nonce":1636950303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1636950303},{"type":"v1","addr":"172.21.15.100:6809","nonce":1636950303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":1636950303},{"type":"v1","addr":"172.21.15.100:6807","nonce":1636950303}]},"public_addr":"172.21.15.100:6803/1636950303","cluster_addr":"172.21.15.100:6805/1636950303","heartbeat_back_addr":"172.21.15.100:6809/1636950303","heartbeat_front_addr":"172.21.15.100:6807/1636950303","state":["exists","up"]},{"osd":1,"uuid":"cb5f6c23-6764-48b6-8b21-97a36b004786","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":470179506},{"type":"v1","addr":"172.21.15.100:6811","nonce":470179506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":470179506},{"type":"v1","addr":"172.21.15.100:6813","nonce":470179506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":470179506},{"type":"v1","addr":"172.21.15.100:6817","nonce":470179506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":470179506},{"type":"v1","addr":"172.21.15.100:6815","nonce":470179506}]},"public_addr":"172.21.15.100:6811/470179506","cluster_addr":"172.21.15.100:6813/470179506","heartbeat_back_addr":"172.21.15.100:6817/470179506","heartbeat_front_addr":"172.21.15.100:6815/470179506","state":["exists","up"]},{"osd":2,"uuid":"1650aedb-9ce9-4c14-a059-e1f089e5e63f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":2235490011},{"type":"v1","addr":"172.21.15.100:6819","nonce":2235490011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2235490011},{"type":"v1","addr":"172.21.15.100:6821","nonce":2235490011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":2235490011},{"type":"v1","addr":"172.21.15.100:6825","nonce":2235490011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":2235490011},{"type":"v1","addr":"172.21.15.100:6823","nonce":2235490011}]},"public_addr":"172.21.15.100:6819/2235490011","cluster_addr":"172.21.15.100:6821/2235490011","heartbeat_back_addr":"172.21.15.100:6825/2235490011","heartbeat_front_addr":"172.21.15.100:6823/2235490011","state":["exists","up"]},{"osd":3,"uuid":"17591c6b-862b-43b2-858b-02f4c3199247","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":3019212393},{"type":"v1","addr":"172.21.15.100:6827","nonce":3019212393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":3019212393},{"type":"v1","addr":"172.21.15.100:6829","nonce":3019212393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":3019212393},{"type":"v1","addr":"172.21.15.100:6833","nonce":3019212393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":3019212393},{"type":"v1","addr":"172.21.15.100:6831","nonce":3019212393}]},"public_addr":"172.21.15.100:6827/3019212393","cluster_addr":"172.21.15.100:6829/3019212393","heartbeat_back_addr":"172.21.15.100:6833/3019212393","heartbeat_front_addr":"172.21.15.100:6831/3019212393","state":["exists","up"]},{"osd":4,"uuid":"6ecc1205-7c78-461e-8037-640136cb3658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":974906744},{"type":"v1","addr":"172.21.15.144:6801","nonce":974906744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":974906744},{"type":"v1","addr":"172.21.15.144:6803","nonce":974906744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":974906744},{"type":"v1","addr":"172.21.15.144:6807","nonce":974906744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":974906744},{"type":"v1","addr":"172.21.15.144:6805","nonce":974906744}]},"public_addr":"172.21.15.144:6801/974906744","cluster_addr":"172.21.15.144:6803/974906744","heartbeat_back_addr":"172.21.15.144:6807/974906744","heartbeat_front_addr":"172.21.15.144:6805/974906744","state":["exists","up"]},{"osd":5,"uuid":"54789cc8-1c3c-46a4-a92e-8881564338db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":2783162058},{"type":"v1","addr":"172.21.15.144:6809","nonce":2783162058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":2783162058},{"type":"v1","addr":"172.21.15.144:6811","nonce":2783162058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":2783162058},{"type":"v1","addr":"172.21.15.144:6815","nonce":2783162058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":2783162058},{"type":"v1","addr":"172.21.15.144:6813","nonce":2783162058}]},"public_addr":"172.21.15.144:6809/2783162058","cluster_addr":"172.21.15.144:6811/2783162058","heartbeat_back_addr":"172.21.15.144:6815/2783162058","heartbeat_front_addr":"172.21.15.144:6813/2783162058","state":["exists","up"]},{"osd":6,"uuid":"ac619497-7605-4ca6-84bf-3abb3f6f32d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":1183840595},{"type":"v1","addr":"172.21.15.144:6817","nonce":1183840595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":1183840595},{"type":"v1","addr":"172.21.15.144:6819","nonce":1183840595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":1183840595},{"type":"v1","addr":"172.21.15.144:6823","nonce":1183840595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":1183840595},{"type":"v1","addr":"172.21.15.144:6821","nonce":1183840595}]},"public_addr":"172.21.15.144:6817/1183840595","cluster_addr":"172.21.15.144:6819/1183840595","heartbeat_back_addr":"172.21.15.144:6823/1183840595","heartbeat_front_addr":"172.21.15.144:6821/1183840595","state":["exists","up"]},{"osd":7,"uuid":"b85b3422-9da2-4605-88e7-fe9c3b2020cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":2248617613},{"type":"v1","addr":"172.21.15.144:6825","nonce":2248617613}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":2248617613},{"type":"v1","addr":"172.21.15.144:6827","nonce":2248617613}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":2248617613},{"type":"v1","addr":"172.21.15.144:6831","nonce":2248617613}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":2248617613},{"type":"v1","addr":"172.21.15.144:6829","nonce":2248617613}]},"public_addr":"172.21.15.144:6825/2248617613","cluster_addr":"172.21.15.144:6827/2248617613","heartbeat_back_addr":"172.21.15.144:6831/2248617613","heartbeat_front_addr":"172.21.15.144:6829/2248617613","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-07T16:05:44.696671+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-07T16:06:33.787321+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-07T16:07:22.983158+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-07T16:08:12.935606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-07T16:09:00.218619+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-07T16:09:50.222185+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-07T16:10:37.856158+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-07T16:11:28.267877+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.100:0/1472671368":"2024-04-08T16:03:24.146096+0000","172.21.15.100:0/639120693":"2024-04-08T16:03:24.146096+0000","172.21.15.100:0/2474331937":"2024-04-08T16:03:24.146096+0000","172.21.15.100:6801/2710911757":"2024-04-08T16:03:24.146096+0000","172.21.15.100:6800/3440287045":"2024-04-08T16:02:53.269514+0000","172.21.15.100:0/109071285":"2024-04-08T16:02:53.269514+0000","172.21.15.100:6801/3440287045":"2024-04-08T16:02:53.269514+0000","172.21.15.100:0/4162615562":"2024-04-08T16:02:53.269514+0000","172.21.15.100:0/4121992264":"2024-04-08T16:02:53.269514+0000","172.21.15.100:6800/2710911757":"2024-04-08T16:03:24.146096+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-07T16:12:10.523 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-07T16:07:26.784071+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 7.889999866485596, 'score_stable': 7.889999866485596, 'optimal_score': 0.3799999952316284, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-07T16:12:10.523 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-07T16:12:10.801 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 1 2024-04-07T16:12:10.819 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-07T16:12:11.110 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-07T16:12:11.110 INFO:teuthology.orchestra.run.smithi100.stdout:[ 2024-04-07T16:12:11.110 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-04-07T16:12:11.110 INFO:teuthology.orchestra.run.smithi100.stdout: "poolnum": 1, 2024-04-07T16:12:11.110 INFO:teuthology.orchestra.run.smithi100.stdout: "poolname": ".mgr" 2024-04-07T16:12:11.110 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-04-07T16:12:11.110 INFO:teuthology.orchestra.run.smithi100.stdout:] 2024-04-07T16:12:11.126 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:10 smithi100 bash[31660]: audit 2024-04-07T16:12:10.219343+0000 mon.a (mon.0) 652 : audit 0 from='client.? 172.21.15.100:0/3942189366' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-07T16:12:11.126 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:10 smithi100 bash[31660]: audit 2024-04-07T16:12:10.516495+0000 mon.a (mon.0) 653 : audit 0 from='client.? 172.21.15.100:0/1245368311' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-07T16:12:11.126 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:10 smithi100 bash[31660]: audit 2024-04-07T16:12:10.813507+0000 mon.a (mon.0) 654 : audit 0 from='client.? 172.21.15.100:0/3484834534' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-07T16:12:11.127 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:10 smithi100 bash[21346]: audit 2024-04-07T16:12:10.219343+0000 mon.a (mon.0) 652 : audit 0 from='client.? 172.21.15.100:0/3942189366' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-07T16:12:11.127 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:10 smithi100 bash[21346]: audit 2024-04-07T16:12:10.516495+0000 mon.a (mon.0) 653 : audit 0 from='client.? 172.21.15.100:0/1245368311' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-07T16:12:11.127 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:10 smithi100 bash[21346]: audit 2024-04-07T16:12:10.813507+0000 mon.a (mon.0) 654 : audit 0 from='client.? 172.21.15.100:0/3484834534' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-07T16:12:11.128 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-07T16:12:11.128 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-07T16:12:11.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:10 smithi144 bash[23522]: audit 2024-04-07T16:12:10.219343+0000 mon.a (mon.0) 652 : audit 0 from='client.? 172.21.15.100:0/3942189366' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-07T16:12:11.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:10 smithi144 bash[23522]: audit 2024-04-07T16:12:10.516495+0000 mon.a (mon.0) 653 : audit 0 from='client.? 172.21.15.100:0/1245368311' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-07T16:12:11.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:10 smithi144 bash[23522]: audit 2024-04-07T16:12:10.813507+0000 mon.a (mon.0) 654 : audit 0 from='client.? 172.21.15.100:0/3484834534' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-07T16:12:11.994 INFO:teuthology.orchestra.run.smithi100.stderr:pool 'cephfs_metadata' created 2024-04-07T16:12:12.013 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-07T16:12:12.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:12 smithi144 bash[23522]: cluster 2024-04-07T16:12:10.286155+0000 mgr.y (mgr.14152) 330 : cluster 0 pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:12:12.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:12 smithi144 bash[23522]: audit 2024-04-07T16:12:11.122469+0000 mon.c (mon.2) 22 : audit 0 from='client.? 172.21.15.100:0/1036742970' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-07T16:12:12.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:12 smithi144 bash[23522]: audit 2024-04-07T16:12:11.446093+0000 mon.c (mon.2) 23 : audit 1 from='client.? 172.21.15.100:0/1586980879' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-07T16:12:12.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:12 smithi144 bash[23522]: audit 2024-04-07T16:12:11.446681+0000 mon.a (mon.0) 655 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-07T16:12:12.440 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:12 smithi100 bash[21346]: cluster 2024-04-07T16:12:10.286155+0000 mgr.y (mgr.14152) 330 : cluster 0 pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:12:12.440 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:12 smithi100 bash[21346]: audit 2024-04-07T16:12:11.122469+0000 mon.c (mon.2) 22 : audit 0 from='client.? 172.21.15.100:0/1036742970' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-07T16:12:12.440 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:12 smithi100 bash[21346]: audit 2024-04-07T16:12:11.446093+0000 mon.c (mon.2) 23 : audit 1 from='client.? 172.21.15.100:0/1586980879' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-07T16:12:12.440 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:12 smithi100 bash[21346]: audit 2024-04-07T16:12:11.446681+0000 mon.a (mon.0) 655 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-07T16:12:12.441 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:12 smithi100 bash[31660]: cluster 2024-04-07T16:12:10.286155+0000 mgr.y (mgr.14152) 330 : cluster 0 pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:12:12.441 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:12 smithi100 bash[31660]: audit 2024-04-07T16:12:11.122469+0000 mon.c (mon.2) 22 : audit 0 from='client.? 172.21.15.100:0/1036742970' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-07T16:12:12.441 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:12 smithi100 bash[31660]: audit 2024-04-07T16:12:11.446093+0000 mon.c (mon.2) 23 : audit 1 from='client.? 172.21.15.100:0/1586980879' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-07T16:12:12.441 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:12 smithi100 bash[31660]: audit 2024-04-07T16:12:11.446681+0000 mon.a (mon.0) 655 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-07T16:12:12.996 INFO:teuthology.orchestra.run.smithi100.stderr:pool 'cephfs_data' created 2024-04-07T16:12:13.034 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-07T16:12:13.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:13 smithi144 bash[23522]: audit 2024-04-07T16:12:12.000203+0000 mon.a (mon.0) 656 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-07T16:12:13.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:13 smithi144 bash[23522]: cluster 2024-04-07T16:12:12.000283+0000 mon.a (mon.0) 657 : cluster 0 osdmap e59: 8 total, 8 up, 8 in 2024-04-07T16:12:13.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:13 smithi144 bash[23522]: audit 2024-04-07T16:12:12.278990+0000 mon.a (mon.0) 658 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:12:13.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:13 smithi144 bash[23522]: audit 2024-04-07T16:12:12.282358+0000 mon.a (mon.0) 659 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:12:13.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:13 smithi144 bash[23522]: audit 2024-04-07T16:12:12.286229+0000 mon.a (mon.0) 660 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:12:13.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:13 smithi144 bash[23522]: audit 2024-04-07T16:12:12.290532+0000 mon.a (mon.0) 661 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:12:13.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:13 smithi144 bash[23522]: audit 2024-04-07T16:12:12.291491+0000 mon.a (mon.0) 662 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:12:13.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:13 smithi144 bash[23522]: audit 2024-04-07T16:12:12.292216+0000 mon.a (mon.0) 663 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:12:13.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:13 smithi144 bash[23522]: audit 2024-04-07T16:12:12.297214+0000 mon.a (mon.0) 664 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:12:13.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:13 smithi144 bash[23522]: audit 2024-04-07T16:12:12.332403+0000 mon.a (mon.0) 665 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:12:13.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:13 smithi144 bash[23522]: audit 2024-04-07T16:12:12.333677+0000 mon.a (mon.0) 666 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:12:13.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:13 smithi144 bash[23522]: audit 2024-04-07T16:12:12.334248+0000 mon.a (mon.0) 667 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:12:13.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:13 smithi144 bash[23522]: audit 2024-04-07T16:12:12.335257+0000 mon.c (mon.2) 24 : audit 1 from='client.? 172.21.15.100:0/1176424596' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-07T16:12:13.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:13 smithi144 bash[23522]: audit 2024-04-07T16:12:12.335591+0000 mon.a (mon.0) 668 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-07T16:12:13.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:13 smithi144 bash[23522]: audit 2024-04-07T16:12:12.744133+0000 mon.a (mon.0) 669 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:12:13.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:13 smithi144 bash[23522]: audit 2024-04-07T16:12:13.007260+0000 mon.a (mon.0) 670 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-07T16:12:13.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:13 smithi144 bash[23522]: cluster 2024-04-07T16:12:13.007515+0000 mon.a (mon.0) 671 : cluster 0 osdmap e60: 8 total, 8 up, 8 in 2024-04-07T16:12:13.362 INFO:teuthology.orchestra.run.smithi100.stderr:new fs with metadata pool 2 and data pool 3 2024-04-07T16:12:13.402 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:13 smithi100 bash[31660]: audit 2024-04-07T16:12:12.000203+0000 mon.a (mon.0) 656 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-07T16:12:13.402 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:13 smithi100 bash[31660]: cluster 2024-04-07T16:12:12.000283+0000 mon.a (mon.0) 657 : cluster 0 osdmap e59: 8 total, 8 up, 8 in 2024-04-07T16:12:13.402 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:13 smithi100 bash[31660]: audit 2024-04-07T16:12:12.278990+0000 mon.a (mon.0) 658 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:12:13.402 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:13 smithi100 bash[31660]: audit 2024-04-07T16:12:12.282358+0000 mon.a (mon.0) 659 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:12:13.402 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:13 smithi100 bash[31660]: audit 2024-04-07T16:12:12.286229+0000 mon.a (mon.0) 660 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:12:13.402 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:13 smithi100 bash[31660]: audit 2024-04-07T16:12:12.290532+0000 mon.a (mon.0) 661 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:12:13.402 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:13 smithi100 bash[31660]: audit 2024-04-07T16:12:12.291491+0000 mon.a (mon.0) 662 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:12:13.402 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:13 smithi100 bash[31660]: audit 2024-04-07T16:12:12.292216+0000 mon.a (mon.0) 663 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:12:13.402 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:13 smithi100 bash[31660]: audit 2024-04-07T16:12:12.297214+0000 mon.a (mon.0) 664 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:12:13.403 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:13 smithi100 bash[31660]: audit 2024-04-07T16:12:12.332403+0000 mon.a (mon.0) 665 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:12:13.403 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:13 smithi100 bash[31660]: audit 2024-04-07T16:12:12.333677+0000 mon.a (mon.0) 666 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:12:13.403 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:13 smithi100 bash[31660]: audit 2024-04-07T16:12:12.334248+0000 mon.a (mon.0) 667 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:12:13.403 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:13 smithi100 bash[31660]: audit 2024-04-07T16:12:12.335257+0000 mon.c (mon.2) 24 : audit 1 from='client.? 172.21.15.100:0/1176424596' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-07T16:12:13.403 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:13 smithi100 bash[31660]: audit 2024-04-07T16:12:12.335591+0000 mon.a (mon.0) 668 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-07T16:12:13.403 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:13 smithi100 bash[31660]: audit 2024-04-07T16:12:12.744133+0000 mon.a (mon.0) 669 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:12:13.403 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:13 smithi100 bash[31660]: audit 2024-04-07T16:12:13.007260+0000 mon.a (mon.0) 670 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-07T16:12:13.403 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:13 smithi100 bash[31660]: cluster 2024-04-07T16:12:13.007515+0000 mon.a (mon.0) 671 : cluster 0 osdmap e60: 8 total, 8 up, 8 in 2024-04-07T16:12:13.403 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:13 smithi100 bash[21346]: audit 2024-04-07T16:12:12.000203+0000 mon.a (mon.0) 656 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-07T16:12:13.403 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:13 smithi100 bash[21346]: cluster 2024-04-07T16:12:12.000283+0000 mon.a (mon.0) 657 : cluster 0 osdmap e59: 8 total, 8 up, 8 in 2024-04-07T16:12:13.404 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:13 smithi100 bash[21346]: audit 2024-04-07T16:12:12.278990+0000 mon.a (mon.0) 658 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:12:13.404 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:13 smithi100 bash[21346]: audit 2024-04-07T16:12:12.282358+0000 mon.a (mon.0) 659 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:12:13.404 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:13 smithi100 bash[21346]: audit 2024-04-07T16:12:12.286229+0000 mon.a (mon.0) 660 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:12:13.404 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:13 smithi100 bash[21346]: audit 2024-04-07T16:12:12.290532+0000 mon.a (mon.0) 661 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:12:13.404 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:13 smithi100 bash[21346]: audit 2024-04-07T16:12:12.291491+0000 mon.a (mon.0) 662 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:12:13.404 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:13 smithi100 bash[21346]: audit 2024-04-07T16:12:12.292216+0000 mon.a (mon.0) 663 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:12:13.404 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:13 smithi100 bash[21346]: audit 2024-04-07T16:12:12.297214+0000 mon.a (mon.0) 664 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:12:13.404 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:13 smithi100 bash[21346]: audit 2024-04-07T16:12:12.332403+0000 mon.a (mon.0) 665 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:12:13.404 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:13 smithi100 bash[21346]: audit 2024-04-07T16:12:12.333677+0000 mon.a (mon.0) 666 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:12:13.404 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:13 smithi100 bash[21346]: audit 2024-04-07T16:12:12.334248+0000 mon.a (mon.0) 667 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:12:13.404 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:13 smithi100 bash[21346]: audit 2024-04-07T16:12:12.335257+0000 mon.c (mon.2) 24 : audit 1 from='client.? 172.21.15.100:0/1176424596' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-07T16:12:13.404 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:13 smithi100 bash[21346]: audit 2024-04-07T16:12:12.335591+0000 mon.a (mon.0) 668 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-07T16:12:13.404 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:13 smithi100 bash[21346]: audit 2024-04-07T16:12:12.744133+0000 mon.a (mon.0) 669 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:12:13.404 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:13 smithi100 bash[21346]: audit 2024-04-07T16:12:13.007260+0000 mon.a (mon.0) 670 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-07T16:12:13.405 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:13 smithi100 bash[21346]: cluster 2024-04-07T16:12:13.007515+0000 mon.a (mon.0) 671 : cluster 0 osdmap e60: 8 total, 8 up, 8 in 2024-04-07T16:12:13.405 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-07T16:12:13.680 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-07T16:12:13.680 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":61,"fsid":"0702e3a8-f4f8-11ee-b649-cb9ed24678a4","created":"2024-04-07T16:02:21.629692+0000","modified":"2024-04-07T16:12:13.364605+0000","last_up_change":"2024-04-07T16:11:30.210001+0000","last_in_change":"2024-04-07T16:11:04.793434+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-07T16:07:26.784071+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":7.8899998664855957,"score_stable":7.8899998664855957,"optimal_score":0.37999999523162842,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-07T16:12:11.856505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-07T16:12:12.737709+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8facd48e-0a0c-4fda-aee2-f001b60a3811","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":1636950303},{"type":"v1","addr":"172.21.15.100:6803","nonce":1636950303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":1636950303},{"type":"v1","addr":"172.21.15.100:6805","nonce":1636950303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1636950303},{"type":"v1","addr":"172.21.15.100:6809","nonce":1636950303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":1636950303},{"type":"v1","addr":"172.21.15.100:6807","nonce":1636950303}]},"public_addr":"172.21.15.100:6803/1636950303","cluster_addr":"172.21.15.100:6805/1636950303","heartbeat_back_addr":"172.21.15.100:6809/1636950303","heartbeat_front_addr":"172.21.15.100:6807/1636950303","state":["exists","up"]},{"osd":1,"uuid":"cb5f6c23-6764-48b6-8b21-97a36b004786","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":470179506},{"type":"v1","addr":"172.21.15.100:6811","nonce":470179506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":470179506},{"type":"v1","addr":"172.21.15.100:6813","nonce":470179506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":470179506},{"type":"v1","addr":"172.21.15.100:6817","nonce":470179506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":470179506},{"type":"v1","addr":"172.21.15.100:6815","nonce":470179506}]},"public_addr":"172.21.15.100:6811/470179506","cluster_addr":"172.21.15.100:6813/470179506","heartbeat_back_addr":"172.21.15.100:6817/470179506","heartbeat_front_addr":"172.21.15.100:6815/470179506","state":["exists","up"]},{"osd":2,"uuid":"1650aedb-9ce9-4c14-a059-e1f089e5e63f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":2235490011},{"type":"v1","addr":"172.21.15.100:6819","nonce":2235490011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2235490011},{"type":"v1","addr":"172.21.15.100:6821","nonce":2235490011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":2235490011},{"type":"v1","addr":"172.21.15.100:6825","nonce":2235490011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":2235490011},{"type":"v1","addr":"172.21.15.100:6823","nonce":2235490011}]},"public_addr":"172.21.15.100:6819/2235490011","cluster_addr":"172.21.15.100:6821/2235490011","heartbeat_back_addr":"172.21.15.100:6825/2235490011","heartbeat_front_addr":"172.21.15.100:6823/2235490011","state":["exists","up"]},{"osd":3,"uuid":"17591c6b-862b-43b2-858b-02f4c3199247","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":3019212393},{"type":"v1","addr":"172.21.15.100:6827","nonce":3019212393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":3019212393},{"type":"v1","addr":"172.21.15.100:6829","nonce":3019212393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":3019212393},{"type":"v1","addr":"172.21.15.100:6833","nonce":3019212393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":3019212393},{"type":"v1","addr":"172.21.15.100:6831","nonce":3019212393}]},"public_addr":"172.21.15.100:6827/3019212393","cluster_addr":"172.21.15.100:6829/3019212393","heartbeat_back_addr":"172.21.15.100:6833/3019212393","heartbeat_front_addr":"172.21.15.100:6831/3019212393","state":["exists","up"]},{"osd":4,"uuid":"6ecc1205-7c78-461e-8037-640136cb3658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":974906744},{"type":"v1","addr":"172.21.15.144:6801","nonce":974906744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":974906744},{"type":"v1","addr":"172.21.15.144:6803","nonce":974906744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":974906744},{"type":"v1","addr":"172.21.15.144:6807","nonce":974906744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":974906744},{"type":"v1","addr":"172.21.15.144:6805","nonce":974906744}]},"public_addr":"172.21.15.144:6801/974906744","cluster_addr":"172.21.15.144:6803/974906744","heartbeat_back_addr":"172.21.15.144:6807/974906744","heartbeat_front_addr":"172.21.15.144:6805/974906744","state":["exists","up"]},{"osd":5,"uuid":"54789cc8-1c3c-46a4-a92e-8881564338db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":2783162058},{"type":"v1","addr":"172.21.15.144:6809","nonce":2783162058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":2783162058},{"type":"v1","addr":"172.21.15.144:6811","nonce":2783162058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":2783162058},{"type":"v1","addr":"172.21.15.144:6815","nonce":2783162058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":2783162058},{"type":"v1","addr":"172.21.15.144:6813","nonce":2783162058}]},"public_addr":"172.21.15.144:6809/2783162058","cluster_addr":"172.21.15.144:6811/2783162058","heartbeat_back_addr":"172.21.15.144:6815/2783162058","heartbeat_front_addr":"172.21.15.144:6813/2783162058","state":["exists","up"]},{"osd":6,"uuid":"ac619497-7605-4ca6-84bf-3abb3f6f32d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":1183840595},{"type":"v1","addr":"172.21.15.144:6817","nonce":1183840595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":1183840595},{"type":"v1","addr":"172.21.15.144:6819","nonce":1183840595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":1183840595},{"type":"v1","addr":"172.21.15.144:6823","nonce":1183840595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":1183840595},{"type":"v1","addr":"172.21.15.144:6821","nonce":1183840595}]},"public_addr":"172.21.15.144:6817/1183840595","cluster_addr":"172.21.15.144:6819/1183840595","heartbeat_back_addr":"172.21.15.144:6823/1183840595","heartbeat_front_addr":"172.21.15.144:6821/1183840595","state":["exists","up"]},{"osd":7,"uuid":"b85b3422-9da2-4605-88e7-fe9c3b2020cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":2248617613},{"type":"v1","addr":"172.21.15.144:6825","nonce":2248617613}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":2248617613},{"type":"v1","addr":"172.21.15.144:6827","nonce":2248617613}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":2248617613},{"type":"v1","addr":"172.21.15.144:6831","nonce":2248617613}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":2248617613},{"type":"v1","addr":"172.21.15.144:6829","nonce":2248617613}]},"public_addr":"172.21.15.144:6825/2248617613","cluster_addr":"172.21.15.144:6827/2248617613","heartbeat_back_addr":"172.21.15.144:6831/2248617613","heartbeat_front_addr":"172.21.15.144:6829/2248617613","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-07T16:05:44.696671+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-07T16:06:33.787321+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-07T16:07:22.983158+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-07T16:08:12.935606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-07T16:09:00.218619+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-07T16:09:50.222185+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-07T16:10:37.856158+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-07T16:11:28.267877+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.100:0/1472671368":"2024-04-08T16:03:24.146096+0000","172.21.15.100:0/639120693":"2024-04-08T16:03:24.146096+0000","172.21.15.100:0/2474331937":"2024-04-08T16:03:24.146096+0000","172.21.15.100:6801/2710911757":"2024-04-08T16:03:24.146096+0000","172.21.15.100:6800/3440287045":"2024-04-08T16:02:53.269514+0000","172.21.15.100:0/109071285":"2024-04-08T16:02:53.269514+0000","172.21.15.100:6801/3440287045":"2024-04-08T16:02:53.269514+0000","172.21.15.100:0/4162615562":"2024-04-08T16:02:53.269514+0000","172.21.15.100:0/4121992264":"2024-04-08T16:02:53.269514+0000","172.21.15.100:6800/2710911757":"2024-04-08T16:03:24.146096+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-07T16:12:13.698 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-07T16:12:13.983 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-07T16:12:13.983 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":61,"fsid":"0702e3a8-f4f8-11ee-b649-cb9ed24678a4","created":"2024-04-07T16:02:21.629692+0000","modified":"2024-04-07T16:12:13.364605+0000","last_up_change":"2024-04-07T16:11:30.210001+0000","last_in_change":"2024-04-07T16:11:04.793434+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-07T16:07:26.784071+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":7.8899998664855957,"score_stable":7.8899998664855957,"optimal_score":0.37999999523162842,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-07T16:12:11.856505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-07T16:12:12.737709+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8facd48e-0a0c-4fda-aee2-f001b60a3811","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":1636950303},{"type":"v1","addr":"172.21.15.100:6803","nonce":1636950303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":1636950303},{"type":"v1","addr":"172.21.15.100:6805","nonce":1636950303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1636950303},{"type":"v1","addr":"172.21.15.100:6809","nonce":1636950303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":1636950303},{"type":"v1","addr":"172.21.15.100:6807","nonce":1636950303}]},"public_addr":"172.21.15.100:6803/1636950303","cluster_addr":"172.21.15.100:6805/1636950303","heartbeat_back_addr":"172.21.15.100:6809/1636950303","heartbeat_front_addr":"172.21.15.100:6807/1636950303","state":["exists","up"]},{"osd":1,"uuid":"cb5f6c23-6764-48b6-8b21-97a36b004786","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":470179506},{"type":"v1","addr":"172.21.15.100:6811","nonce":470179506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":470179506},{"type":"v1","addr":"172.21.15.100:6813","nonce":470179506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":470179506},{"type":"v1","addr":"172.21.15.100:6817","nonce":470179506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":470179506},{"type":"v1","addr":"172.21.15.100:6815","nonce":470179506}]},"public_addr":"172.21.15.100:6811/470179506","cluster_addr":"172.21.15.100:6813/470179506","heartbeat_back_addr":"172.21.15.100:6817/470179506","heartbeat_front_addr":"172.21.15.100:6815/470179506","state":["exists","up"]},{"osd":2,"uuid":"1650aedb-9ce9-4c14-a059-e1f089e5e63f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":2235490011},{"type":"v1","addr":"172.21.15.100:6819","nonce":2235490011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2235490011},{"type":"v1","addr":"172.21.15.100:6821","nonce":2235490011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":2235490011},{"type":"v1","addr":"172.21.15.100:6825","nonce":2235490011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":2235490011},{"type":"v1","addr":"172.21.15.100:6823","nonce":2235490011}]},"public_addr":"172.21.15.100:6819/2235490011","cluster_addr":"172.21.15.100:6821/2235490011","heartbeat_back_addr":"172.21.15.100:6825/2235490011","heartbeat_front_addr":"172.21.15.100:6823/2235490011","state":["exists","up"]},{"osd":3,"uuid":"17591c6b-862b-43b2-858b-02f4c3199247","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":3019212393},{"type":"v1","addr":"172.21.15.100:6827","nonce":3019212393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":3019212393},{"type":"v1","addr":"172.21.15.100:6829","nonce":3019212393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":3019212393},{"type":"v1","addr":"172.21.15.100:6833","nonce":3019212393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":3019212393},{"type":"v1","addr":"172.21.15.100:6831","nonce":3019212393}]},"public_addr":"172.21.15.100:6827/3019212393","cluster_addr":"172.21.15.100:6829/3019212393","heartbeat_back_addr":"172.21.15.100:6833/3019212393","heartbeat_front_addr":"172.21.15.100:6831/3019212393","state":["exists","up"]},{"osd":4,"uuid":"6ecc1205-7c78-461e-8037-640136cb3658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":974906744},{"type":"v1","addr":"172.21.15.144:6801","nonce":974906744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":974906744},{"type":"v1","addr":"172.21.15.144:6803","nonce":974906744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":974906744},{"type":"v1","addr":"172.21.15.144:6807","nonce":974906744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":974906744},{"type":"v1","addr":"172.21.15.144:6805","nonce":974906744}]},"public_addr":"172.21.15.144:6801/974906744","cluster_addr":"172.21.15.144:6803/974906744","heartbeat_back_addr":"172.21.15.144:6807/974906744","heartbeat_front_addr":"172.21.15.144:6805/974906744","state":["exists","up"]},{"osd":5,"uuid":"54789cc8-1c3c-46a4-a92e-8881564338db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":2783162058},{"type":"v1","addr":"172.21.15.144:6809","nonce":2783162058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":2783162058},{"type":"v1","addr":"172.21.15.144:6811","nonce":2783162058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":2783162058},{"type":"v1","addr":"172.21.15.144:6815","nonce":2783162058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":2783162058},{"type":"v1","addr":"172.21.15.144:6813","nonce":2783162058}]},"public_addr":"172.21.15.144:6809/2783162058","cluster_addr":"172.21.15.144:6811/2783162058","heartbeat_back_addr":"172.21.15.144:6815/2783162058","heartbeat_front_addr":"172.21.15.144:6813/2783162058","state":["exists","up"]},{"osd":6,"uuid":"ac619497-7605-4ca6-84bf-3abb3f6f32d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":1183840595},{"type":"v1","addr":"172.21.15.144:6817","nonce":1183840595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":1183840595},{"type":"v1","addr":"172.21.15.144:6819","nonce":1183840595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":1183840595},{"type":"v1","addr":"172.21.15.144:6823","nonce":1183840595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":1183840595},{"type":"v1","addr":"172.21.15.144:6821","nonce":1183840595}]},"public_addr":"172.21.15.144:6817/1183840595","cluster_addr":"172.21.15.144:6819/1183840595","heartbeat_back_addr":"172.21.15.144:6823/1183840595","heartbeat_front_addr":"172.21.15.144:6821/1183840595","state":["exists","up"]},{"osd":7,"uuid":"b85b3422-9da2-4605-88e7-fe9c3b2020cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":2248617613},{"type":"v1","addr":"172.21.15.144:6825","nonce":2248617613}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":2248617613},{"type":"v1","addr":"172.21.15.144:6827","nonce":2248617613}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":2248617613},{"type":"v1","addr":"172.21.15.144:6831","nonce":2248617613}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":2248617613},{"type":"v1","addr":"172.21.15.144:6829","nonce":2248617613}]},"public_addr":"172.21.15.144:6825/2248617613","cluster_addr":"172.21.15.144:6827/2248617613","heartbeat_back_addr":"172.21.15.144:6831/2248617613","heartbeat_front_addr":"172.21.15.144:6829/2248617613","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-07T16:05:44.696671+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-07T16:06:33.787321+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-07T16:07:22.983158+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-07T16:08:12.935606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-07T16:09:00.218619+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-07T16:09:50.222185+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-07T16:10:37.856158+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-07T16:11:28.267877+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.100:0/1472671368":"2024-04-08T16:03:24.146096+0000","172.21.15.100:0/639120693":"2024-04-08T16:03:24.146096+0000","172.21.15.100:0/2474331937":"2024-04-08T16:03:24.146096+0000","172.21.15.100:6801/2710911757":"2024-04-08T16:03:24.146096+0000","172.21.15.100:6800/3440287045":"2024-04-08T16:02:53.269514+0000","172.21.15.100:0/109071285":"2024-04-08T16:02:53.269514+0000","172.21.15.100:6801/3440287045":"2024-04-08T16:02:53.269514+0000","172.21.15.100:0/4162615562":"2024-04-08T16:02:53.269514+0000","172.21.15.100:0/4121992264":"2024-04-08T16:02:53.269514+0000","172.21.15.100:6800/2710911757":"2024-04-08T16:03:24.146096+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-07T16:12:14.001 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-07T16:12:14.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:14 smithi144 bash[23522]: cluster 2024-04-07T16:12:12.286504+0000 mgr.y (mgr.14152) 331 : cluster 0 pgmap v313: 33 pgs: 5 creating+peering, 1 active+clean, 27 unknown; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:12:14.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:14 smithi144 bash[23522]: cluster 2024-04-07T16:12:13.284594+0000 mon.a (mon.0) 672 : cluster 3 Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T16:12:14.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:14 smithi144 bash[23522]: audit 2024-04-07T16:12:13.363541+0000 mon.c (mon.2) 25 : audit 1 from='client.? 172.21.15.100:0/3839088905' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-07T16:12:14.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:14 smithi144 bash[23522]: audit 2024-04-07T16:12:13.364279+0000 mon.a (mon.0) 673 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-07T16:12:14.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:14 smithi144 bash[23522]: cluster 2024-04-07T16:12:13.365202+0000 mon.a (mon.0) 674 : cluster 1 daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-07T16:12:14.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:14 smithi144 bash[23522]: cluster 2024-04-07T16:12:13.369833+0000 mon.a (mon.0) 675 : cluster 0 osdmap e61: 8 total, 8 up, 8 in 2024-04-07T16:12:14.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:14 smithi144 bash[23522]: audit 2024-04-07T16:12:13.370392+0000 mon.a (mon.0) 676 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-07T16:12:14.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:14 smithi144 bash[23522]: cluster 2024-04-07T16:12:13.370473+0000 mon.a (mon.0) 677 : cluster 0 fsmap cephfs:1 {0=b=up:creating} 1 up:standby 2024-04-07T16:12:14.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:14 smithi144 bash[23522]: cluster 2024-04-07T16:12:13.419682+0000 mon.a (mon.0) 678 : cluster 1 daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-07T16:12:14.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:14 smithi144 bash[23522]: audit 2024-04-07T16:12:13.691069+0000 mon.c (mon.2) 26 : audit 0 from='client.? 172.21.15.100:0/2605682933' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-07T16:12:14.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:14 smithi144 bash[23522]: audit 2024-04-07T16:12:13.992672+0000 mon.a (mon.0) 679 : audit 0 from='client.? 172.21.15.100:0/1890297963' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-07T16:12:14.382 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:14 smithi100 bash[31660]: cluster 2024-04-07T16:12:12.286504+0000 mgr.y (mgr.14152) 331 : cluster 0 pgmap v313: 33 pgs: 5 creating+peering, 1 active+clean, 27 unknown; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:12:14.382 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:14 smithi100 bash[31660]: cluster 2024-04-07T16:12:13.284594+0000 mon.a (mon.0) 672 : cluster 3 Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T16:12:14.382 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:14 smithi100 bash[31660]: audit 2024-04-07T16:12:13.363541+0000 mon.c (mon.2) 25 : audit 1 from='client.? 172.21.15.100:0/3839088905' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-07T16:12:14.382 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:14 smithi100 bash[31660]: audit 2024-04-07T16:12:13.364279+0000 mon.a (mon.0) 673 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-07T16:12:14.382 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:14 smithi100 bash[31660]: cluster 2024-04-07T16:12:13.365202+0000 mon.a (mon.0) 674 : cluster 1 daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-07T16:12:14.382 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:14 smithi100 bash[31660]: cluster 2024-04-07T16:12:13.369833+0000 mon.a (mon.0) 675 : cluster 0 osdmap e61: 8 total, 8 up, 8 in 2024-04-07T16:12:14.382 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:14 smithi100 bash[31660]: audit 2024-04-07T16:12:13.370392+0000 mon.a (mon.0) 676 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-07T16:12:14.382 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:14 smithi100 bash[31660]: cluster 2024-04-07T16:12:13.370473+0000 mon.a (mon.0) 677 : cluster 0 fsmap cephfs:1 {0=b=up:creating} 1 up:standby 2024-04-07T16:12:14.382 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:14 smithi100 bash[31660]: cluster 2024-04-07T16:12:13.419682+0000 mon.a (mon.0) 678 : cluster 1 daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-07T16:12:14.383 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:14 smithi100 bash[31660]: audit 2024-04-07T16:12:13.691069+0000 mon.c (mon.2) 26 : audit 0 from='client.? 172.21.15.100:0/2605682933' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-07T16:12:14.383 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:14 smithi100 bash[31660]: audit 2024-04-07T16:12:13.992672+0000 mon.a (mon.0) 679 : audit 0 from='client.? 172.21.15.100:0/1890297963' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-07T16:12:14.383 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:14 smithi100 bash[21346]: cluster 2024-04-07T16:12:12.286504+0000 mgr.y (mgr.14152) 331 : cluster 0 pgmap v313: 33 pgs: 5 creating+peering, 1 active+clean, 27 unknown; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:12:14.383 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:14 smithi100 bash[21346]: cluster 2024-04-07T16:12:13.284594+0000 mon.a (mon.0) 672 : cluster 3 Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T16:12:14.383 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:14 smithi100 bash[21346]: audit 2024-04-07T16:12:13.363541+0000 mon.c (mon.2) 25 : audit 1 from='client.? 172.21.15.100:0/3839088905' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-07T16:12:14.383 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:14 smithi100 bash[21346]: audit 2024-04-07T16:12:13.364279+0000 mon.a (mon.0) 673 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-07T16:12:14.383 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:14 smithi100 bash[21346]: cluster 2024-04-07T16:12:13.365202+0000 mon.a (mon.0) 674 : cluster 1 daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-07T16:12:14.383 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:14 smithi100 bash[21346]: cluster 2024-04-07T16:12:13.369833+0000 mon.a (mon.0) 675 : cluster 0 osdmap e61: 8 total, 8 up, 8 in 2024-04-07T16:12:14.383 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:14 smithi100 bash[21346]: audit 2024-04-07T16:12:13.370392+0000 mon.a (mon.0) 676 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-07T16:12:14.383 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:14 smithi100 bash[21346]: cluster 2024-04-07T16:12:13.370473+0000 mon.a (mon.0) 677 : cluster 0 fsmap cephfs:1 {0=b=up:creating} 1 up:standby 2024-04-07T16:12:14.383 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:14 smithi100 bash[21346]: cluster 2024-04-07T16:12:13.419682+0000 mon.a (mon.0) 678 : cluster 1 daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-07T16:12:14.384 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:14 smithi100 bash[21346]: audit 2024-04-07T16:12:13.691069+0000 mon.c (mon.2) 26 : audit 0 from='client.? 172.21.15.100:0/2605682933' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-07T16:12:14.384 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:14 smithi100 bash[21346]: audit 2024-04-07T16:12:13.992672+0000 mon.a (mon.0) 679 : audit 0 from='client.? 172.21.15.100:0/1890297963' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-07T16:12:14.384 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-07T16:12:14.384 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-07T16:12:15.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:15 smithi144 bash[23522]: audit 2024-04-07T16:12:14.352699+0000 mon.c (mon.2) 27 : audit 1 from='client.? 172.21.15.100:0/2708429651' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-07T16:12:15.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:15 smithi144 bash[23522]: audit 2024-04-07T16:12:14.353268+0000 mon.a (mon.0) 680 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-07T16:12:15.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:15 smithi144 bash[23522]: cluster 2024-04-07T16:12:14.371610+0000 mon.a (mon.0) 681 : cluster 0 mds.? [v2:172.21.15.144:6832/226244023,v1:172.21.15.144:6833/226244023] up:active 2024-04-07T16:12:15.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:15 smithi144 bash[23522]: audit 2024-04-07T16:12:14.371741+0000 mon.a (mon.0) 682 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-07T16:12:15.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:15 smithi144 bash[23522]: cluster 2024-04-07T16:12:14.371905+0000 mon.a (mon.0) 683 : cluster 0 fsmap cephfs:1 {0=b=up:active} 1 up:standby 2024-04-07T16:12:15.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:15 smithi144 bash[23522]: cluster 2024-04-07T16:12:14.380292+0000 mon.a (mon.0) 684 : cluster 0 osdmap e62: 8 total, 8 up, 8 in 2024-04-07T16:12:15.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:15 smithi144 bash[23522]: audit 2024-04-07T16:12:14.738670+0000 mon.a (mon.0) 685 : audit 1 from='client.? 172.21.15.100:0/2042265022' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]: dispatch 2024-04-07T16:12:15.391 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:15 smithi100 bash[21346]: audit 2024-04-07T16:12:14.352699+0000 mon.c (mon.2) 27 : audit 1 from='client.? 172.21.15.100:0/2708429651' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-07T16:12:15.391 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:15 smithi100 bash[21346]: audit 2024-04-07T16:12:14.353268+0000 mon.a (mon.0) 680 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-07T16:12:15.391 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:15 smithi100 bash[21346]: cluster 2024-04-07T16:12:14.371610+0000 mon.a (mon.0) 681 : cluster 0 mds.? [v2:172.21.15.144:6832/226244023,v1:172.21.15.144:6833/226244023] up:active 2024-04-07T16:12:15.391 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:15 smithi100 bash[21346]: audit 2024-04-07T16:12:14.371741+0000 mon.a (mon.0) 682 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-07T16:12:15.391 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:15 smithi100 bash[21346]: cluster 2024-04-07T16:12:14.371905+0000 mon.a (mon.0) 683 : cluster 0 fsmap cephfs:1 {0=b=up:active} 1 up:standby 2024-04-07T16:12:15.391 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:15 smithi100 bash[21346]: cluster 2024-04-07T16:12:14.380292+0000 mon.a (mon.0) 684 : cluster 0 osdmap e62: 8 total, 8 up, 8 in 2024-04-07T16:12:15.391 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:15 smithi100 bash[21346]: audit 2024-04-07T16:12:14.738670+0000 mon.a (mon.0) 685 : audit 1 from='client.? 172.21.15.100:0/2042265022' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]: dispatch 2024-04-07T16:12:15.392 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:15 smithi100 bash[31660]: audit 2024-04-07T16:12:14.352699+0000 mon.c (mon.2) 27 : audit 1 from='client.? 172.21.15.100:0/2708429651' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-07T16:12:15.392 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:15 smithi100 bash[31660]: audit 2024-04-07T16:12:14.353268+0000 mon.a (mon.0) 680 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-07T16:12:15.392 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:15 smithi100 bash[31660]: cluster 2024-04-07T16:12:14.371610+0000 mon.a (mon.0) 681 : cluster 0 mds.? [v2:172.21.15.144:6832/226244023,v1:172.21.15.144:6833/226244023] up:active 2024-04-07T16:12:15.392 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:15 smithi100 bash[31660]: audit 2024-04-07T16:12:14.371741+0000 mon.a (mon.0) 682 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-07T16:12:15.392 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:15 smithi100 bash[31660]: cluster 2024-04-07T16:12:14.371905+0000 mon.a (mon.0) 683 : cluster 0 fsmap cephfs:1 {0=b=up:active} 1 up:standby 2024-04-07T16:12:15.392 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:15 smithi100 bash[31660]: cluster 2024-04-07T16:12:14.380292+0000 mon.a (mon.0) 684 : cluster 0 osdmap e62: 8 total, 8 up, 8 in 2024-04-07T16:12:15.392 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:15 smithi100 bash[31660]: audit 2024-04-07T16:12:14.738670+0000 mon.a (mon.0) 685 : audit 1 from='client.? 172.21.15.100:0/2042265022' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]: dispatch 2024-04-07T16:12:15.393 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-07T16:12:15.393 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-07T16:12:15.724 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-07T16:12:15.724 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14424,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/3487006602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3487006602},{"type":"v1","addr":"172.21.15.100:6835","nonce":3487006602}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-07T16:12:13.365189+0000","modified":"2024-04-07T16:12:15.375075+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24322},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24322":{"gid":24322,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.144:6833/226244023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":226244023},{"type":"v1","addr":"172.21.15.144:6833","nonce":226244023}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-07T16:12:15.725 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 6 2024-04-07T16:12:15.743 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-07T16:12:16.025 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-07T16:12:16.025 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":62,"fsid":"0702e3a8-f4f8-11ee-b649-cb9ed24678a4","created":"2024-04-07T16:02:21.629692+0000","modified":"2024-04-07T16:12:14.367390+0000","last_up_change":"2024-04-07T16:11:30.210001+0000","last_in_change":"2024-04-07T16:11:04.793434+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-07T16:07:26.784071+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":7.8899998664855957,"score_stable":7.8899998664855957,"optimal_score":0.37999999523162842,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-07T16:12:11.856505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-07T16:12:12.737709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8facd48e-0a0c-4fda-aee2-f001b60a3811","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":1636950303},{"type":"v1","addr":"172.21.15.100:6803","nonce":1636950303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":1636950303},{"type":"v1","addr":"172.21.15.100:6805","nonce":1636950303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1636950303},{"type":"v1","addr":"172.21.15.100:6809","nonce":1636950303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":1636950303},{"type":"v1","addr":"172.21.15.100:6807","nonce":1636950303}]},"public_addr":"172.21.15.100:6803/1636950303","cluster_addr":"172.21.15.100:6805/1636950303","heartbeat_back_addr":"172.21.15.100:6809/1636950303","heartbeat_front_addr":"172.21.15.100:6807/1636950303","state":["exists","up"]},{"osd":1,"uuid":"cb5f6c23-6764-48b6-8b21-97a36b004786","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":470179506},{"type":"v1","addr":"172.21.15.100:6811","nonce":470179506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":470179506},{"type":"v1","addr":"172.21.15.100:6813","nonce":470179506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":470179506},{"type":"v1","addr":"172.21.15.100:6817","nonce":470179506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":470179506},{"type":"v1","addr":"172.21.15.100:6815","nonce":470179506}]},"public_addr":"172.21.15.100:6811/470179506","cluster_addr":"172.21.15.100:6813/470179506","heartbeat_back_addr":"172.21.15.100:6817/470179506","heartbeat_front_addr":"172.21.15.100:6815/470179506","state":["exists","up"]},{"osd":2,"uuid":"1650aedb-9ce9-4c14-a059-e1f089e5e63f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":2235490011},{"type":"v1","addr":"172.21.15.100:6819","nonce":2235490011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2235490011},{"type":"v1","addr":"172.21.15.100:6821","nonce":2235490011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":2235490011},{"type":"v1","addr":"172.21.15.100:6825","nonce":2235490011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":2235490011},{"type":"v1","addr":"172.21.15.100:6823","nonce":2235490011}]},"public_addr":"172.21.15.100:6819/2235490011","cluster_addr":"172.21.15.100:6821/2235490011","heartbeat_back_addr":"172.21.15.100:6825/2235490011","heartbeat_front_addr":"172.21.15.100:6823/2235490011","state":["exists","up"]},{"osd":3,"uuid":"17591c6b-862b-43b2-858b-02f4c3199247","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":3019212393},{"type":"v1","addr":"172.21.15.100:6827","nonce":3019212393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":3019212393},{"type":"v1","addr":"172.21.15.100:6829","nonce":3019212393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":3019212393},{"type":"v1","addr":"172.21.15.100:6833","nonce":3019212393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":3019212393},{"type":"v1","addr":"172.21.15.100:6831","nonce":3019212393}]},"public_addr":"172.21.15.100:6827/3019212393","cluster_addr":"172.21.15.100:6829/3019212393","heartbeat_back_addr":"172.21.15.100:6833/3019212393","heartbeat_front_addr":"172.21.15.100:6831/3019212393","state":["exists","up"]},{"osd":4,"uuid":"6ecc1205-7c78-461e-8037-640136cb3658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":974906744},{"type":"v1","addr":"172.21.15.144:6801","nonce":974906744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":974906744},{"type":"v1","addr":"172.21.15.144:6803","nonce":974906744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":974906744},{"type":"v1","addr":"172.21.15.144:6807","nonce":974906744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":974906744},{"type":"v1","addr":"172.21.15.144:6805","nonce":974906744}]},"public_addr":"172.21.15.144:6801/974906744","cluster_addr":"172.21.15.144:6803/974906744","heartbeat_back_addr":"172.21.15.144:6807/974906744","heartbeat_front_addr":"172.21.15.144:6805/974906744","state":["exists","up"]},{"osd":5,"uuid":"54789cc8-1c3c-46a4-a92e-8881564338db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":2783162058},{"type":"v1","addr":"172.21.15.144:6809","nonce":2783162058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":2783162058},{"type":"v1","addr":"172.21.15.144:6811","nonce":2783162058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":2783162058},{"type":"v1","addr":"172.21.15.144:6815","nonce":2783162058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":2783162058},{"type":"v1","addr":"172.21.15.144:6813","nonce":2783162058}]},"public_addr":"172.21.15.144:6809/2783162058","cluster_addr":"172.21.15.144:6811/2783162058","heartbeat_back_addr":"172.21.15.144:6815/2783162058","heartbeat_front_addr":"172.21.15.144:6813/2783162058","state":["exists","up"]},{"osd":6,"uuid":"ac619497-7605-4ca6-84bf-3abb3f6f32d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":1183840595},{"type":"v1","addr":"172.21.15.144:6817","nonce":1183840595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":1183840595},{"type":"v1","addr":"172.21.15.144:6819","nonce":1183840595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":1183840595},{"type":"v1","addr":"172.21.15.144:6823","nonce":1183840595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":1183840595},{"type":"v1","addr":"172.21.15.144:6821","nonce":1183840595}]},"public_addr":"172.21.15.144:6817/1183840595","cluster_addr":"172.21.15.144:6819/1183840595","heartbeat_back_addr":"172.21.15.144:6823/1183840595","heartbeat_front_addr":"172.21.15.144:6821/1183840595","state":["exists","up"]},{"osd":7,"uuid":"b85b3422-9da2-4605-88e7-fe9c3b2020cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":2248617613},{"type":"v1","addr":"172.21.15.144:6825","nonce":2248617613}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":2248617613},{"type":"v1","addr":"172.21.15.144:6827","nonce":2248617613}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":2248617613},{"type":"v1","addr":"172.21.15.144:6831","nonce":2248617613}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":2248617613},{"type":"v1","addr":"172.21.15.144:6829","nonce":2248617613}]},"public_addr":"172.21.15.144:6825/2248617613","cluster_addr":"172.21.15.144:6827/2248617613","heartbeat_back_addr":"172.21.15.144:6831/2248617613","heartbeat_front_addr":"172.21.15.144:6829/2248617613","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-07T16:05:44.696671+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-07T16:06:33.787321+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-07T16:07:22.983158+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-07T16:08:12.935606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-07T16:09:00.218619+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-07T16:09:50.222185+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-07T16:10:37.856158+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-07T16:11:28.267877+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.100:0/1472671368":"2024-04-08T16:03:24.146096+0000","172.21.15.100:0/639120693":"2024-04-08T16:03:24.146096+0000","172.21.15.100:0/2474331937":"2024-04-08T16:03:24.146096+0000","172.21.15.100:6801/2710911757":"2024-04-08T16:03:24.146096+0000","172.21.15.100:6800/3440287045":"2024-04-08T16:02:53.269514+0000","172.21.15.100:0/109071285":"2024-04-08T16:02:53.269514+0000","172.21.15.100:6801/3440287045":"2024-04-08T16:02:53.269514+0000","172.21.15.100:0/4162615562":"2024-04-08T16:02:53.269514+0000","172.21.15.100:0/4121992264":"2024-04-08T16:02:53.269514+0000","172.21.15.100:6800/2710911757":"2024-04-08T16:03:24.146096+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-07T16:12:16.042 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-07T16:12:16.042 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-07T16:12:16.344 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-07T16:12:16.346 INFO:teuthology.orchestra.run.smithi100.stderr:dumped all 2024-04-07T16:12:16.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:16 smithi144 bash[23522]: cluster 2024-04-07T16:12:14.287062+0000 mgr.y (mgr.14152) 332 : cluster 0 pgmap v316: 97 pgs: 1 creating+activating, 12 creating+peering, 1 active+clean, 83 unknown; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:12:16.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:16 smithi144 bash[23522]: cluster 2024-04-07T16:12:15.016940+0000 mon.a (mon.0) 686 : cluster 1 Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T16:12:16.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:16 smithi144 bash[23522]: cluster 2024-04-07T16:12:15.017006+0000 mon.a (mon.0) 687 : cluster 1 Cluster is now healthy 2024-04-07T16:12:16.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:16 smithi144 bash[23522]: audit 2024-04-07T16:12:15.382218+0000 mon.a (mon.0) 688 : audit 1 from='client.? 172.21.15.100:0/2042265022' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]': finished 2024-04-07T16:12:16.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:16 smithi144 bash[23522]: cluster 2024-04-07T16:12:15.382450+0000 mon.a (mon.0) 689 : cluster 0 fsmap cephfs:1 {0=b=up:active} 1 up:standby 2024-04-07T16:12:16.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:16 smithi144 bash[23522]: audit 2024-04-07T16:12:15.736616+0000 mon.c (mon.2) 28 : audit 0 from='client.? 172.21.15.100:0/874616174' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-07T16:12:16.361 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:16 smithi100 bash[31660]: cluster 2024-04-07T16:12:14.287062+0000 mgr.y (mgr.14152) 332 : cluster 0 pgmap v316: 97 pgs: 1 creating+activating, 12 creating+peering, 1 active+clean, 83 unknown; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:12:16.362 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:16 smithi100 bash[31660]: cluster 2024-04-07T16:12:15.016940+0000 mon.a (mon.0) 686 : cluster 1 Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T16:12:16.362 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:16 smithi100 bash[31660]: cluster 2024-04-07T16:12:15.017006+0000 mon.a (mon.0) 687 : cluster 1 Cluster is now healthy 2024-04-07T16:12:16.362 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:16 smithi100 bash[31660]: audit 2024-04-07T16:12:15.382218+0000 mon.a (mon.0) 688 : audit 1 from='client.? 172.21.15.100:0/2042265022' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]': finished 2024-04-07T16:12:16.362 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:16 smithi100 bash[31660]: cluster 2024-04-07T16:12:15.382450+0000 mon.a (mon.0) 689 : cluster 0 fsmap cephfs:1 {0=b=up:active} 1 up:standby 2024-04-07T16:12:16.362 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:16 smithi100 bash[31660]: audit 2024-04-07T16:12:15.736616+0000 mon.c (mon.2) 28 : audit 0 from='client.? 172.21.15.100:0/874616174' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-07T16:12:16.362 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:16 smithi100 bash[21346]: cluster 2024-04-07T16:12:14.287062+0000 mgr.y (mgr.14152) 332 : cluster 0 pgmap v316: 97 pgs: 1 creating+activating, 12 creating+peering, 1 active+clean, 83 unknown; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:12:16.362 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:16 smithi100 bash[21346]: cluster 2024-04-07T16:12:15.016940+0000 mon.a (mon.0) 686 : cluster 1 Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T16:12:16.362 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:16 smithi100 bash[21346]: cluster 2024-04-07T16:12:15.017006+0000 mon.a (mon.0) 687 : cluster 1 Cluster is now healthy 2024-04-07T16:12:16.362 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:16 smithi100 bash[21346]: audit 2024-04-07T16:12:15.382218+0000 mon.a (mon.0) 688 : audit 1 from='client.? 172.21.15.100:0/2042265022' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]': finished 2024-04-07T16:12:16.362 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:16 smithi100 bash[21346]: cluster 2024-04-07T16:12:15.382450+0000 mon.a (mon.0) 689 : cluster 0 fsmap cephfs:1 {0=b=up:active} 1 up:standby 2024-04-07T16:12:16.363 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:16 smithi100 bash[21346]: audit 2024-04-07T16:12:15.736616+0000 mon.c (mon.2) 28 : audit 0 from='client.? 172.21.15.100:0/874616174' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-07T16:12:16.363 INFO:teuthology.orchestra.run.smithi100.stdout:{"pg_ready":true,"pg_map":{"version":318,"stamp":"2024-04-07T16:12:16.287343+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591932,"num_objects":15,"num_object_clones":0,"num_object_copies":45,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":190,"num_write_kb":2419,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":144,"ondisk_log_size":144,"up":189,"acting":189,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":76,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":749961216,"kb_used":230856,"kb_used_data":6472,"kb_used_omap":12,"kb_used_meta":224243,"kb_avail":749730360,"statfs":{"total":767960285184,"available":767723888640,"internally_reserved":0,"allocated":6627328,"data_stored":3755083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":229625401},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1564,"num_objects":13,"num_object_clones":0,"num_object_copies":39,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.290644"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.376089+0000","last_change":"2024-04-07T16:12:13.395367+0000","last_active":"2024-04-07T16:12:14.376089+0000","last_peered":"2024-04-07T16:12:14.376089+0000","last_clean":"2024-04-07T16:12:14.376089+0000","last_became_active":"2024-04-07T16:12:13.395023+0000","last_became_peered":"2024-04-07T16:12:13.395023+0000","last_unstale":"2024-04-07T16:12:14.376089+0000","last_undegraded":"2024-04-07T16:12:14.376089+0000","last_fullsized":"2024-04-07T16:12:14.376089+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T23:05:09.752696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-07T16:12:12.996208+0000","last_change":"2024-04-07T16:12:12.996208+0000","last_active":"2024-04-07T16:12:12.996208+0000","last_peered":"2024-04-07T16:12:12.996208+0000","last_clean":"2024-04-07T16:12:12.996208+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:12.996208+0000","last_undegraded":"2024-04-07T16:12:12.996208+0000","last_fullsized":"2024-04-07T16:12:12.996208+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-07T16:12:12.996208+0000","last_change":"2024-04-07T16:12:12.996208+0000","last_active":"2024-04-07T16:12:12.996208+0000","last_peered":"2024-04-07T16:12:12.996208+0000","last_clean":"2024-04-07T16:12:12.996208+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:12.996208+0000","last_undegraded":"2024-04-07T16:12:12.996208+0000","last_fullsized":"2024-04-07T16:12:12.996208+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-07T16:12:12.996208+0000","last_change":"2024-04-07T16:12:12.996208+0000","last_active":"2024-04-07T16:12:12.996208+0000","last_peered":"2024-04-07T16:12:12.996208+0000","last_clean":"2024-04-07T16:12:12.996208+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:12.996208+0000","last_undegraded":"2024-04-07T16:12:12.996208+0000","last_fullsized":"2024-04-07T16:12:12.996208+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.375284+0000","last_change":"2024-04-07T16:12:13.413487+0000","last_active":"2024-04-07T16:12:14.375284+0000","last_peered":"2024-04-07T16:12:14.375284+0000","last_clean":"2024-04-07T16:12:14.375284+0000","last_became_active":"2024-04-07T16:12:13.413281+0000","last_became_peered":"2024-04-07T16:12:13.413281+0000","last_unstale":"2024-04-07T16:12:14.375284+0000","last_undegraded":"2024-04-07T16:12:14.375284+0000","last_fullsized":"2024-04-07T16:12:14.375284+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T23:59:23.302803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-07T16:12:12.996208+0000","last_change":"2024-04-07T16:12:12.996208+0000","last_active":"2024-04-07T16:12:12.996208+0000","last_peered":"2024-04-07T16:12:12.996208+0000","last_clean":"2024-04-07T16:12:12.996208+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:12.996208+0000","last_undegraded":"2024-04-07T16:12:12.996208+0000","last_fullsized":"2024-04-07T16:12:12.996208+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.374561+0000","last_change":"2024-04-07T16:12:13.393518+0000","last_active":"2024-04-07T16:12:14.374561+0000","last_peered":"2024-04-07T16:12:14.374561+0000","last_clean":"2024-04-07T16:12:14.374561+0000","last_became_active":"2024-04-07T16:12:13.393187+0000","last_became_peered":"2024-04-07T16:12:13.393187+0000","last_unstale":"2024-04-07T16:12:14.374561+0000","last_undegraded":"2024-04-07T16:12:14.374561+0000","last_fullsized":"2024-04-07T16:12:14.374561+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T19:41:28.702905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.475028+0000","last_change":"2024-04-07T16:12:13.394702+0000","last_active":"2024-04-07T16:12:14.475028+0000","last_peered":"2024-04-07T16:12:14.475028+0000","last_clean":"2024-04-07T16:12:14.475028+0000","last_became_active":"2024-04-07T16:12:13.394422+0000","last_became_peered":"2024-04-07T16:12:13.394422+0000","last_unstale":"2024-04-07T16:12:14.475028+0000","last_undegraded":"2024-04-07T16:12:14.475028+0000","last_fullsized":"2024-04-07T16:12:14.475028+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T00:37:44.512471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.375990+0000","last_change":"2024-04-07T16:12:13.401766+0000","last_active":"2024-04-07T16:12:14.375990+0000","last_peered":"2024-04-07T16:12:14.375990+0000","last_clean":"2024-04-07T16:12:14.375990+0000","last_became_active":"2024-04-07T16:12:13.401411+0000","last_became_peered":"2024-04-07T16:12:13.401411+0000","last_unstale":"2024-04-07T16:12:14.375990+0000","last_undegraded":"2024-04-07T16:12:14.375990+0000","last_fullsized":"2024-04-07T16:12:14.375990+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T23:32:52.283410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,0],"acting":[7,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.475526+0000","last_change":"2024-04-07T16:12:13.396409+0000","last_active":"2024-04-07T16:12:14.475526+0000","last_peered":"2024-04-07T16:12:14.475526+0000","last_clean":"2024-04-07T16:12:14.475526+0000","last_became_active":"2024-04-07T16:12:13.396094+0000","last_became_peered":"2024-04-07T16:12:13.396094+0000","last_unstale":"2024-04-07T16:12:14.475526+0000","last_undegraded":"2024-04-07T16:12:14.475526+0000","last_fullsized":"2024-04-07T16:12:14.475526+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T18:54:36.072837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.474549+0000","last_change":"2024-04-07T16:12:13.401973+0000","last_active":"2024-04-07T16:12:14.474549+0000","last_peered":"2024-04-07T16:12:14.474549+0000","last_clean":"2024-04-07T16:12:14.474549+0000","last_became_active":"2024-04-07T16:12:13.401724+0000","last_became_peered":"2024-04-07T16:12:13.401724+0000","last_unstale":"2024-04-07T16:12:14.474549+0000","last_undegraded":"2024-04-07T16:12:14.474549+0000","last_fullsized":"2024-04-07T16:12:14.474549+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T21:26:29.208223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,0],"acting":[6,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-04-07T16:12:13.005867+0000","last_change":"2024-04-07T16:12:13.005867+0000","last_active":"2024-04-07T16:12:12.996208+0000","last_peered":"2024-04-07T16:12:12.996208+0000","last_clean":"2024-04-07T16:12:12.996208+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:13.005867+0000","last_undegraded":"2024-04-07T16:12:13.005867+0000","last_fullsized":"2024-04-07T16:12:13.005867+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T22:29:22.509564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,0],"acting":[4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.474547+0000","last_change":"2024-04-07T16:12:13.394760+0000","last_active":"2024-04-07T16:12:14.474547+0000","last_peered":"2024-04-07T16:12:14.474547+0000","last_clean":"2024-04-07T16:12:14.474547+0000","last_became_active":"2024-04-07T16:12:13.394402+0000","last_became_peered":"2024-04-07T16:12:13.394402+0000","last_unstale":"2024-04-07T16:12:14.474547+0000","last_undegraded":"2024-04-07T16:12:14.474547+0000","last_fullsized":"2024-04-07T16:12:14.474547+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T18:47:13.891972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-07T16:12:12.996208+0000","last_change":"2024-04-07T16:12:12.996208+0000","last_active":"2024-04-07T16:12:12.996208+0000","last_peered":"2024-04-07T16:12:12.996208+0000","last_clean":"2024-04-07T16:12:12.996208+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:12.996208+0000","last_undegraded":"2024-04-07T16:12:12.996208+0000","last_fullsized":"2024-04-07T16:12:12.996208+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-07T16:12:12.996208+0000","last_change":"2024-04-07T16:12:12.996208+0000","last_active":"2024-04-07T16:12:12.996208+0000","last_peered":"2024-04-07T16:12:12.996208+0000","last_clean":"2024-04-07T16:12:12.996208+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:12.996208+0000","last_undegraded":"2024-04-07T16:12:12.996208+0000","last_fullsized":"2024-04-07T16:12:12.996208+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-07T16:12:12.996208+0000","last_change":"2024-04-07T16:12:12.996208+0000","last_active":"2024-04-07T16:12:12.996208+0000","last_peered":"2024-04-07T16:12:12.996208+0000","last_clean":"2024-04-07T16:12:12.996208+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:12.996208+0000","last_undegraded":"2024-04-07T16:12:12.996208+0000","last_fullsized":"2024-04-07T16:12:12.996208+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-07T16:12:12.996208+0000","last_change":"2024-04-07T16:12:12.996208+0000","last_active":"2024-04-07T16:12:12.996208+0000","last_peered":"2024-04-07T16:12:12.996208+0000","last_clean":"2024-04-07T16:12:12.996208+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:12.996208+0000","last_undegraded":"2024-04-07T16:12:12.996208+0000","last_fullsized":"2024-04-07T16:12:12.996208+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.474396+0000","last_change":"2024-04-07T16:12:13.416210+0000","last_active":"2024-04-07T16:12:14.474396+0000","last_peered":"2024-04-07T16:12:14.474396+0000","last_clean":"2024-04-07T16:12:14.474396+0000","last_became_active":"2024-04-07T16:12:13.415972+0000","last_became_peered":"2024-04-07T16:12:13.415972+0000","last_unstale":"2024-04-07T16:12:14.474396+0000","last_undegraded":"2024-04-07T16:12:14.474396+0000","last_fullsized":"2024-04-07T16:12:14.474396+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T23:40:11.606978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,7],"acting":[2,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.375192+0000","last_change":"2024-04-07T16:12:13.414203+0000","last_active":"2024-04-07T16:12:14.375192+0000","last_peered":"2024-04-07T16:12:14.375192+0000","last_clean":"2024-04-07T16:12:14.375192+0000","last_became_active":"2024-04-07T16:12:13.413869+0000","last_became_peered":"2024-04-07T16:12:13.413869+0000","last_unstale":"2024-04-07T16:12:14.375192+0000","last_undegraded":"2024-04-07T16:12:14.375192+0000","last_fullsized":"2024-04-07T16:12:14.375192+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T17:11:47.895568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,6],"acting":[7,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-04-07T16:12:12.000271+0000","last_change":"2024-04-07T16:12:12.000271+0000","last_active":"2024-04-07T16:12:11.994648+0000","last_peered":"2024-04-07T16:12:11.994648+0000","last_clean":"2024-04-07T16:12:11.994648+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:12.000271+0000","last_undegraded":"2024-04-07T16:12:12.000271+0000","last_fullsized":"2024-04-07T16:12:12.000271+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T19:01:44.798719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.474373+0000","last_change":"2024-04-07T16:12:13.407852+0000","last_active":"2024-04-07T16:12:14.474373+0000","last_peered":"2024-04-07T16:12:14.474373+0000","last_clean":"2024-04-07T16:12:14.474373+0000","last_became_active":"2024-04-07T16:12:13.407538+0000","last_became_peered":"2024-04-07T16:12:13.407538+0000","last_unstale":"2024-04-07T16:12:14.474373+0000","last_undegraded":"2024-04-07T16:12:14.474373+0000","last_fullsized":"2024-04-07T16:12:14.474373+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T20:27:52.664149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"61'1","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.474471+0000","last_change":"2024-04-07T16:12:13.038807+0000","last_active":"2024-04-07T16:12:14.474471+0000","last_peered":"2024-04-07T16:12:14.474471+0000","last_clean":"2024-04-07T16:12:14.474471+0000","last_became_active":"2024-04-07T16:12:13.038492+0000","last_became_peered":"2024-04-07T16:12:13.038492+0000","last_unstale":"2024-04-07T16:12:14.474471+0000","last_undegraded":"2024-04-07T16:12:14.474471+0000","last_fullsized":"2024-04-07T16:12:14.474471+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T02:11:03.746017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-07T16:12:12.996208+0000","last_change":"2024-04-07T16:12:12.996208+0000","last_active":"2024-04-07T16:12:12.996208+0000","last_peered":"2024-04-07T16:12:12.996208+0000","last_clean":"2024-04-07T16:12:12.996208+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:12.996208+0000","last_undegraded":"2024-04-07T16:12:12.996208+0000","last_fullsized":"2024-04-07T16:12:12.996208+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"61'2","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.375664+0000","last_change":"2024-04-07T16:12:13.042401+0000","last_active":"2024-04-07T16:12:14.375664+0000","last_peered":"2024-04-07T16:12:14.375664+0000","last_clean":"2024-04-07T16:12:14.375664+0000","last_became_active":"2024-04-07T16:12:13.041970+0000","last_became_peered":"2024-04-07T16:12:13.041970+0000","last_unstale":"2024-04-07T16:12:14.375664+0000","last_undegraded":"2024-04-07T16:12:14.375664+0000","last_fullsized":"2024-04-07T16:12:14.375664+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T16:36:29.235347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-07T16:12:12.996208+0000","last_change":"2024-04-07T16:12:12.996208+0000","last_active":"2024-04-07T16:12:12.996208+0000","last_peered":"2024-04-07T16:12:12.996208+0000","last_clean":"2024-04-07T16:12:12.996208+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:12.996208+0000","last_undegraded":"2024-04-07T16:12:12.996208+0000","last_fullsized":"2024-04-07T16:12:12.996208+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"61'2","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.375447+0000","last_change":"2024-04-07T16:12:13.042391+0000","last_active":"2024-04-07T16:12:14.375447+0000","last_peered":"2024-04-07T16:12:14.375447+0000","last_clean":"2024-04-07T16:12:14.375447+0000","last_became_active":"2024-04-07T16:12:13.041919+0000","last_became_peered":"2024-04-07T16:12:13.041919+0000","last_unstale":"2024-04-07T16:12:14.375447+0000","last_undegraded":"2024-04-07T16:12:14.375447+0000","last_fullsized":"2024-04-07T16:12:14.375447+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T21:50:59.576615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-07T16:12:12.996208+0000","last_change":"2024-04-07T16:12:12.996208+0000","last_active":"2024-04-07T16:12:12.996208+0000","last_peered":"2024-04-07T16:12:12.996208+0000","last_clean":"2024-04-07T16:12:12.996208+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:12.996208+0000","last_undegraded":"2024-04-07T16:12:12.996208+0000","last_fullsized":"2024-04-07T16:12:12.996208+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"61'1","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.474818+0000","last_change":"2024-04-07T16:12:13.041880+0000","last_active":"2024-04-07T16:12:14.474818+0000","last_peered":"2024-04-07T16:12:14.474818+0000","last_clean":"2024-04-07T16:12:14.474818+0000","last_became_active":"2024-04-07T16:12:13.041409+0000","last_became_peered":"2024-04-07T16:12:13.041409+0000","last_unstale":"2024-04-07T16:12:14.474818+0000","last_undegraded":"2024-04-07T16:12:14.474818+0000","last_fullsized":"2024-04-07T16:12:14.474818+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T18:32:23.206985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-04-07T16:12:13.002508+0000","last_change":"2024-04-07T16:12:13.002508+0000","last_active":"2024-04-07T16:12:12.996208+0000","last_peered":"2024-04-07T16:12:12.996208+0000","last_clean":"2024-04-07T16:12:12.996208+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:13.002508+0000","last_undegraded":"2024-04-07T16:12:13.002508+0000","last_fullsized":"2024-04-07T16:12:13.002508+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T23:20:40.949255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"61'1","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.476584+0000","last_change":"2024-04-07T16:12:13.038960+0000","last_active":"2024-04-07T16:12:14.476584+0000","last_peered":"2024-04-07T16:12:14.476584+0000","last_clean":"2024-04-07T16:12:14.476584+0000","last_became_active":"2024-04-07T16:12:13.038410+0000","last_became_peered":"2024-04-07T16:12:13.038410+0000","last_unstale":"2024-04-07T16:12:14.476584+0000","last_undegraded":"2024-04-07T16:12:14.476584+0000","last_fullsized":"2024-04-07T16:12:14.476584+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T03:41:38.418312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-07T16:12:12.996208+0000","last_change":"2024-04-07T16:12:12.996208+0000","last_active":"2024-04-07T16:12:12.996208+0000","last_peered":"2024-04-07T16:12:12.996208+0000","last_clean":"2024-04-07T16:12:12.996208+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:12.996208+0000","last_undegraded":"2024-04-07T16:12:12.996208+0000","last_fullsized":"2024-04-07T16:12:12.996208+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.475758+0000","last_change":"2024-04-07T16:12:13.034420+0000","last_active":"2024-04-07T16:12:14.475758+0000","last_peered":"2024-04-07T16:12:14.475758+0000","last_clean":"2024-04-07T16:12:14.475758+0000","last_became_active":"2024-04-07T16:12:13.034013+0000","last_became_peered":"2024-04-07T16:12:13.034013+0000","last_unstale":"2024-04-07T16:12:14.475758+0000","last_undegraded":"2024-04-07T16:12:14.475758+0000","last_fullsized":"2024-04-07T16:12:14.475758+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T19:51:21.525865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.474201+0000","last_change":"2024-04-07T16:12:13.398531+0000","last_active":"2024-04-07T16:12:14.474201+0000","last_peered":"2024-04-07T16:12:14.474201+0000","last_clean":"2024-04-07T16:12:14.474201+0000","last_became_active":"2024-04-07T16:12:13.398017+0000","last_became_peered":"2024-04-07T16:12:13.398017+0000","last_unstale":"2024-04-07T16:12:14.474201+0000","last_undegraded":"2024-04-07T16:12:14.474201+0000","last_fullsized":"2024-04-07T16:12:14.474201+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T17:33:54.306902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-04-07T16:12:12.000806+0000","last_change":"2024-04-07T16:12:12.000806+0000","last_active":"2024-04-07T16:12:11.994648+0000","last_peered":"2024-04-07T16:12:11.994648+0000","last_clean":"2024-04-07T16:12:11.994648+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:12.000806+0000","last_undegraded":"2024-04-07T16:12:12.000806+0000","last_fullsized":"2024-04-07T16:12:12.000806+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T22:44:14.623116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-07T16:12:12.996208+0000","last_change":"2024-04-07T16:12:12.996208+0000","last_active":"2024-04-07T16:12:12.996208+0000","last_peered":"2024-04-07T16:12:12.996208+0000","last_clean":"2024-04-07T16:12:12.996208+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:12.996208+0000","last_undegraded":"2024-04-07T16:12:12.996208+0000","last_fullsized":"2024-04-07T16:12:12.996208+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.474717+0000","last_change":"2024-04-07T16:12:13.030149+0000","last_active":"2024-04-07T16:12:14.474717+0000","last_peered":"2024-04-07T16:12:14.474717+0000","last_clean":"2024-04-07T16:12:14.474717+0000","last_became_active":"2024-04-07T16:12:13.029525+0000","last_became_peered":"2024-04-07T16:12:13.029525+0000","last_unstale":"2024-04-07T16:12:14.474717+0000","last_undegraded":"2024-04-07T16:12:14.474717+0000","last_fullsized":"2024-04-07T16:12:14.474717+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T20:09:35.180264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-07T16:12:12.996208+0000","last_change":"2024-04-07T16:12:12.996208+0000","last_active":"2024-04-07T16:12:12.996208+0000","last_peered":"2024-04-07T16:12:12.996208+0000","last_clean":"2024-04-07T16:12:12.996208+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:12.996208+0000","last_undegraded":"2024-04-07T16:12:12.996208+0000","last_fullsized":"2024-04-07T16:12:12.996208+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-04-07T16:12:12.001015+0000","last_change":"2024-04-07T16:12:12.001015+0000","last_active":"2024-04-07T16:12:11.994648+0000","last_peered":"2024-04-07T16:12:11.994648+0000","last_clean":"2024-04-07T16:12:11.994648+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:12.001015+0000","last_undegraded":"2024-04-07T16:12:12.001015+0000","last_fullsized":"2024-04-07T16:12:12.001015+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T18:26:16.873044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-07T16:12:12.996208+0000","last_change":"2024-04-07T16:12:12.996208+0000","last_active":"2024-04-07T16:12:12.996208+0000","last_peered":"2024-04-07T16:12:12.996208+0000","last_clean":"2024-04-07T16:12:12.996208+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:12.996208+0000","last_undegraded":"2024-04-07T16:12:12.996208+0000","last_fullsized":"2024-04-07T16:12:12.996208+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-07T16:12:11.994648+0000","last_change":"2024-04-07T16:12:11.994648+0000","last_active":"2024-04-07T16:12:11.994648+0000","last_peered":"2024-04-07T16:12:11.994648+0000","last_clean":"2024-04-07T16:12:11.994648+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:11.994648+0000","last_undegraded":"2024-04-07T16:12:11.994648+0000","last_fullsized":"2024-04-07T16:12:11.994648+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-04-07T16:12:13.004673+0000","last_change":"2024-04-07T16:12:13.004673+0000","last_active":"2024-04-07T16:12:12.996208+0000","last_peered":"2024-04-07T16:12:12.996208+0000","last_clean":"2024-04-07T16:12:12.996208+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:13.004673+0000","last_undegraded":"2024-04-07T16:12:13.004673+0000","last_fullsized":"2024-04-07T16:12:13.004673+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T16:55:37.429799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"61'6","reported_seq":18,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.473934+0000","last_change":"2024-04-07T16:12:13.031673+0000","last_active":"2024-04-07T16:12:14.473934+0000","last_peered":"2024-04-07T16:12:14.473934+0000","last_clean":"2024-04-07T16:12:14.473934+0000","last_became_active":"2024-04-07T16:12:13.031092+0000","last_became_peered":"2024-04-07T16:12:13.031092+0000","last_unstale":"2024-04-07T16:12:14.473934+0000","last_undegraded":"2024-04-07T16:12:14.473934+0000","last_fullsized":"2024-04-07T16:12:14.473934+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T20:17:48.593862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.375355+0000","last_change":"2024-04-07T16:12:13.390083+0000","last_active":"2024-04-07T16:12:14.375355+0000","last_peered":"2024-04-07T16:12:14.375355+0000","last_clean":"2024-04-07T16:12:14.375355+0000","last_became_active":"2024-04-07T16:12:13.389755+0000","last_became_peered":"2024-04-07T16:12:13.389755+0000","last_unstale":"2024-04-07T16:12:14.375355+0000","last_undegraded":"2024-04-07T16:12:14.375355+0000","last_fullsized":"2024-04-07T16:12:14.375355+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T02:31:09.155569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-04-07T16:12:12.001190+0000","last_change":"2024-04-07T16:12:12.001190+0000","last_active":"2024-04-07T16:12:11.994648+0000","last_peered":"2024-04-07T16:12:11.994648+0000","last_clean":"2024-04-07T16:12:11.994648+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:12.001190+0000","last_undegraded":"2024-04-07T16:12:12.001190+0000","last_fullsized":"2024-04-07T16:12:12.001190+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T01:50:02.195507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-07T16:12:12.996208+0000","last_change":"2024-04-07T16:12:12.996208+0000","last_active":"2024-04-07T16:12:12.996208+0000","last_peered":"2024-04-07T16:12:12.996208+0000","last_clean":"2024-04-07T16:12:12.996208+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:12.996208+0000","last_undegraded":"2024-04-07T16:12:12.996208+0000","last_fullsized":"2024-04-07T16:12:12.996208+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-04-07T16:12:12.002099+0000","last_change":"2024-04-07T16:12:12.002099+0000","last_active":"2024-04-07T16:12:11.994648+0000","last_peered":"2024-04-07T16:12:11.994648+0000","last_clean":"2024-04-07T16:12:11.994648+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:12.002099+0000","last_undegraded":"2024-04-07T16:12:12.002099+0000","last_fullsized":"2024-04-07T16:12:12.002099+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T23:15:11.494437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.375013+0000","last_change":"2024-04-07T16:12:13.413928+0000","last_active":"2024-04-07T16:12:14.375013+0000","last_peered":"2024-04-07T16:12:14.375013+0000","last_clean":"2024-04-07T16:12:14.375013+0000","last_became_active":"2024-04-07T16:12:13.413447+0000","last_became_peered":"2024-04-07T16:12:13.413447+0000","last_unstale":"2024-04-07T16:12:14.375013+0000","last_undegraded":"2024-04-07T16:12:14.375013+0000","last_fullsized":"2024-04-07T16:12:14.375013+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T03:26:42.875228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.473099+0000","last_change":"2024-04-07T16:12:13.032942+0000","last_active":"2024-04-07T16:12:14.473099+0000","last_peered":"2024-04-07T16:12:14.473099+0000","last_clean":"2024-04-07T16:12:14.473099+0000","last_became_active":"2024-04-07T16:12:13.032108+0000","last_became_peered":"2024-04-07T16:12:13.032108+0000","last_unstale":"2024-04-07T16:12:14.473099+0000","last_undegraded":"2024-04-07T16:12:14.473099+0000","last_fullsized":"2024-04-07T16:12:14.473099+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T18:53:06.864412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.473331+0000","last_change":"2024-04-07T16:12:13.400141+0000","last_active":"2024-04-07T16:12:14.473331+0000","last_peered":"2024-04-07T16:12:14.473331+0000","last_clean":"2024-04-07T16:12:14.473331+0000","last_became_active":"2024-04-07T16:12:13.399823+0000","last_became_peered":"2024-04-07T16:12:13.399823+0000","last_unstale":"2024-04-07T16:12:14.473331+0000","last_undegraded":"2024-04-07T16:12:14.473331+0000","last_fullsized":"2024-04-07T16:12:14.473331+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T21:27:25.248595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"61'8","reported_seq":20,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.474350+0000","last_change":"2024-04-07T16:12:13.034154+0000","last_active":"2024-04-07T16:12:14.474350+0000","last_peered":"2024-04-07T16:12:14.474350+0000","last_clean":"2024-04-07T16:12:14.474350+0000","last_became_active":"2024-04-07T16:12:13.033838+0000","last_became_peered":"2024-04-07T16:12:13.033838+0000","last_unstale":"2024-04-07T16:12:14.474350+0000","last_undegraded":"2024-04-07T16:12:14.474350+0000","last_fullsized":"2024-04-07T16:12:14.474350+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T23:13:07.216314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.374716+0000","last_change":"2024-04-07T16:12:13.385883+0000","last_active":"2024-04-07T16:12:14.374716+0000","last_peered":"2024-04-07T16:12:14.374716+0000","last_clean":"2024-04-07T16:12:14.374716+0000","last_became_active":"2024-04-07T16:12:13.385485+0000","last_became_peered":"2024-04-07T16:12:13.385485+0000","last_unstale":"2024-04-07T16:12:14.374716+0000","last_undegraded":"2024-04-07T16:12:14.374716+0000","last_fullsized":"2024-04-07T16:12:14.374716+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T21:01:09.262433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":4,"reported_epoch":60,"state":"creating+activating","last_fresh":"2024-04-07T16:12:13.024777+0000","last_change":"2024-04-07T16:12:13.002417+0000","last_active":"2024-04-07T16:12:11.994648+0000","last_peered":"2024-04-07T16:12:11.994648+0000","last_clean":"2024-04-07T16:12:11.994648+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:13.024777+0000","last_undegraded":"2024-04-07T16:12:13.024777+0000","last_fullsized":"2024-04-07T16:12:13.024777+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T17:55:06.727142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":["4","0","7"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.375770+0000","last_change":"2024-04-07T16:12:13.386584+0000","last_active":"2024-04-07T16:12:14.375770+0000","last_peered":"2024-04-07T16:12:14.375770+0000","last_clean":"2024-04-07T16:12:14.375770+0000","last_became_active":"2024-04-07T16:12:13.386057+0000","last_became_peered":"2024-04-07T16:12:13.386057+0000","last_unstale":"2024-04-07T16:12:14.375770+0000","last_undegraded":"2024-04-07T16:12:14.375770+0000","last_fullsized":"2024-04-07T16:12:14.375770+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T17:57:57.548179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-07T16:12:11.994648+0000","last_change":"2024-04-07T16:12:11.994648+0000","last_active":"2024-04-07T16:12:11.994648+0000","last_peered":"2024-04-07T16:12:11.994648+0000","last_clean":"2024-04-07T16:12:11.994648+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:11.994648+0000","last_undegraded":"2024-04-07T16:12:11.994648+0000","last_fullsized":"2024-04-07T16:12:11.994648+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-07T16:12:12.996208+0000","last_change":"2024-04-07T16:12:12.996208+0000","last_active":"2024-04-07T16:12:12.996208+0000","last_peered":"2024-04-07T16:12:12.996208+0000","last_clean":"2024-04-07T16:12:12.996208+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:12.996208+0000","last_undegraded":"2024-04-07T16:12:12.996208+0000","last_fullsized":"2024-04-07T16:12:12.996208+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-04-07T16:12:12.001190+0000","last_change":"2024-04-07T16:12:12.001190+0000","last_active":"2024-04-07T16:12:11.994648+0000","last_peered":"2024-04-07T16:12:11.994648+0000","last_clean":"2024-04-07T16:12:11.994648+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:12.001190+0000","last_undegraded":"2024-04-07T16:12:12.001190+0000","last_fullsized":"2024-04-07T16:12:12.001190+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T18:15:00.506827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-04-07T16:12:13.003585+0000","last_change":"2024-04-07T16:12:13.003585+0000","last_active":"2024-04-07T16:12:12.996208+0000","last_peered":"2024-04-07T16:12:12.996208+0000","last_clean":"2024-04-07T16:12:12.996208+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:13.003585+0000","last_undegraded":"2024-04-07T16:12:13.003585+0000","last_fullsized":"2024-04-07T16:12:13.003585+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T03:36:24.354110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-07T16:12:12.996208+0000","last_change":"2024-04-07T16:12:12.996208+0000","last_active":"2024-04-07T16:12:12.996208+0000","last_peered":"2024-04-07T16:12:12.996208+0000","last_clean":"2024-04-07T16:12:12.996208+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:12.996208+0000","last_undegraded":"2024-04-07T16:12:12.996208+0000","last_fullsized":"2024-04-07T16:12:12.996208+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.474169+0000","last_change":"2024-04-07T16:12:13.035071+0000","last_active":"2024-04-07T16:12:14.474169+0000","last_peered":"2024-04-07T16:12:14.474169+0000","last_clean":"2024-04-07T16:12:14.474169+0000","last_became_active":"2024-04-07T16:12:13.034795+0000","last_became_peered":"2024-04-07T16:12:13.034795+0000","last_unstale":"2024-04-07T16:12:14.474169+0000","last_undegraded":"2024-04-07T16:12:14.474169+0000","last_fullsized":"2024-04-07T16:12:14.474169+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T17:51:30.599527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.476063+0000","last_change":"2024-04-07T16:12:13.394635+0000","last_active":"2024-04-07T16:12:14.476063+0000","last_peered":"2024-04-07T16:12:14.476063+0000","last_clean":"2024-04-07T16:12:14.476063+0000","last_became_active":"2024-04-07T16:12:13.394392+0000","last_became_peered":"2024-04-07T16:12:13.394392+0000","last_unstale":"2024-04-07T16:12:14.476063+0000","last_undegraded":"2024-04-07T16:12:14.476063+0000","last_fullsized":"2024-04-07T16:12:14.476063+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T17:52:10.950481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-04-07T16:12:12.001243+0000","last_change":"2024-04-07T16:12:12.001243+0000","last_active":"2024-04-07T16:12:11.994648+0000","last_peered":"2024-04-07T16:12:11.994648+0000","last_clean":"2024-04-07T16:12:11.994648+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:12.001243+0000","last_undegraded":"2024-04-07T16:12:12.001243+0000","last_fullsized":"2024-04-07T16:12:12.001243+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T02:48:50.652491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"21'117","reported_seq":219,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-07T16:12:13.375677+0000","last_change":"2024-04-07T16:11:33.518868+0000","last_active":"2024-04-07T16:12:13.375677+0000","last_peered":"2024-04-07T16:12:13.375677+0000","last_clean":"2024-04-07T16:12:13.375677+0000","last_became_active":"2024-04-07T16:11:33.518060+0000","last_became_peered":"2024-04-07T16:11:33.518060+0000","last_unstale":"2024-04-07T16:12:13.375677+0000","last_undegraded":"2024-04-07T16:12:13.375677+0000","last_fullsized":"2024-04-07T16:12:13.375677+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:07:26.866003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:07:26.866003+0000","last_clean_scrub_stamp":"2024-04-07T16:07:26.866003+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T22:22:51.043941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-07T16:12:12.996208+0000","last_change":"2024-04-07T16:12:12.996208+0000","last_active":"2024-04-07T16:12:12.996208+0000","last_peered":"2024-04-07T16:12:12.996208+0000","last_clean":"2024-04-07T16:12:12.996208+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:12.996208+0000","last_undegraded":"2024-04-07T16:12:12.996208+0000","last_fullsized":"2024-04-07T16:12:12.996208+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.375081+0000","last_change":"2024-04-07T16:12:13.035536+0000","last_active":"2024-04-07T16:12:14.375081+0000","last_peered":"2024-04-07T16:12:14.375081+0000","last_clean":"2024-04-07T16:12:14.375081+0000","last_became_active":"2024-04-07T16:12:13.034825+0000","last_became_peered":"2024-04-07T16:12:13.034825+0000","last_unstale":"2024-04-07T16:12:14.375081+0000","last_undegraded":"2024-04-07T16:12:14.375081+0000","last_fullsized":"2024-04-07T16:12:14.375081+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T20:11:57.033306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.375586+0000","last_change":"2024-04-07T16:12:13.035747+0000","last_active":"2024-04-07T16:12:14.375586+0000","last_peered":"2024-04-07T16:12:14.375586+0000","last_clean":"2024-04-07T16:12:14.375586+0000","last_became_active":"2024-04-07T16:12:13.035201+0000","last_became_peered":"2024-04-07T16:12:13.035201+0000","last_unstale":"2024-04-07T16:12:14.375586+0000","last_undegraded":"2024-04-07T16:12:14.375586+0000","last_fullsized":"2024-04-07T16:12:14.375586+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T21:03:25.449728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-07T16:12:12.996208+0000","last_change":"2024-04-07T16:12:12.996208+0000","last_active":"2024-04-07T16:12:12.996208+0000","last_peered":"2024-04-07T16:12:12.996208+0000","last_clean":"2024-04-07T16:12:12.996208+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:12.996208+0000","last_undegraded":"2024-04-07T16:12:12.996208+0000","last_fullsized":"2024-04-07T16:12:12.996208+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-07T16:12:11.994648+0000","last_change":"2024-04-07T16:12:11.994648+0000","last_active":"2024-04-07T16:12:11.994648+0000","last_peered":"2024-04-07T16:12:11.994648+0000","last_clean":"2024-04-07T16:12:11.994648+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:11.994648+0000","last_undegraded":"2024-04-07T16:12:11.994648+0000","last_fullsized":"2024-04-07T16:12:11.994648+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.474648+0000","last_change":"2024-04-07T16:12:13.399486+0000","last_active":"2024-04-07T16:12:14.474648+0000","last_peered":"2024-04-07T16:12:14.474648+0000","last_clean":"2024-04-07T16:12:14.474648+0000","last_became_active":"2024-04-07T16:12:13.399184+0000","last_became_peered":"2024-04-07T16:12:13.399184+0000","last_unstale":"2024-04-07T16:12:14.474648+0000","last_undegraded":"2024-04-07T16:12:14.474648+0000","last_fullsized":"2024-04-07T16:12:14.474648+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T21:07:25.104309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"61'2","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.474473+0000","last_change":"2024-04-07T16:12:13.038351+0000","last_active":"2024-04-07T16:12:14.474473+0000","last_peered":"2024-04-07T16:12:14.474473+0000","last_clean":"2024-04-07T16:12:14.474473+0000","last_became_active":"2024-04-07T16:12:13.038026+0000","last_became_peered":"2024-04-07T16:12:13.038026+0000","last_unstale":"2024-04-07T16:12:14.474473+0000","last_undegraded":"2024-04-07T16:12:14.474473+0000","last_fullsized":"2024-04-07T16:12:14.474473+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T19:39:42.796342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-07T16:12:12.996208+0000","last_change":"2024-04-07T16:12:12.996208+0000","last_active":"2024-04-07T16:12:12.996208+0000","last_peered":"2024-04-07T16:12:12.996208+0000","last_clean":"2024-04-07T16:12:12.996208+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:12.996208+0000","last_undegraded":"2024-04-07T16:12:12.996208+0000","last_fullsized":"2024-04-07T16:12:12.996208+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.374594+0000","last_change":"2024-04-07T16:12:13.033067+0000","last_active":"2024-04-07T16:12:14.374594+0000","last_peered":"2024-04-07T16:12:14.374594+0000","last_clean":"2024-04-07T16:12:14.374594+0000","last_became_active":"2024-04-07T16:12:13.032456+0000","last_became_peered":"2024-04-07T16:12:13.032456+0000","last_unstale":"2024-04-07T16:12:14.374594+0000","last_undegraded":"2024-04-07T16:12:14.374594+0000","last_fullsized":"2024-04-07T16:12:14.374594+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T23:02:09.025617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-04-07T16:12:13.001861+0000","last_change":"2024-04-07T16:12:13.001861+0000","last_active":"2024-04-07T16:12:12.996208+0000","last_peered":"2024-04-07T16:12:12.996208+0000","last_clean":"2024-04-07T16:12:12.996208+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:13.001861+0000","last_undegraded":"2024-04-07T16:12:13.001861+0000","last_fullsized":"2024-04-07T16:12:13.001861+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T23:48:08.846232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-07T16:12:11.994648+0000","last_change":"2024-04-07T16:12:11.994648+0000","last_active":"2024-04-07T16:12:11.994648+0000","last_peered":"2024-04-07T16:12:11.994648+0000","last_clean":"2024-04-07T16:12:11.994648+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:11.994648+0000","last_undegraded":"2024-04-07T16:12:11.994648+0000","last_fullsized":"2024-04-07T16:12:11.994648+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.474100+0000","last_change":"2024-04-07T16:12:13.392396+0000","last_active":"2024-04-07T16:12:14.474100+0000","last_peered":"2024-04-07T16:12:14.474100+0000","last_clean":"2024-04-07T16:12:14.474100+0000","last_became_active":"2024-04-07T16:12:13.392016+0000","last_became_peered":"2024-04-07T16:12:13.392016+0000","last_unstale":"2024-04-07T16:12:14.474100+0000","last_undegraded":"2024-04-07T16:12:14.474100+0000","last_fullsized":"2024-04-07T16:12:14.474100+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T02:35:50.790051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-07T16:12:11.994648+0000","last_change":"2024-04-07T16:12:11.994648+0000","last_active":"2024-04-07T16:12:11.994648+0000","last_peered":"2024-04-07T16:12:11.994648+0000","last_clean":"2024-04-07T16:12:11.994648+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:11.994648+0000","last_undegraded":"2024-04-07T16:12:11.994648+0000","last_fullsized":"2024-04-07T16:12:11.994648+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.474697+0000","last_change":"2024-04-07T16:12:13.397819+0000","last_active":"2024-04-07T16:12:14.474697+0000","last_peered":"2024-04-07T16:12:14.474697+0000","last_clean":"2024-04-07T16:12:14.474697+0000","last_became_active":"2024-04-07T16:12:13.397593+0000","last_became_peered":"2024-04-07T16:12:13.397593+0000","last_unstale":"2024-04-07T16:12:14.474697+0000","last_undegraded":"2024-04-07T16:12:14.474697+0000","last_fullsized":"2024-04-07T16:12:14.474697+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T02:30:01.472030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"61'2","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.375041+0000","last_change":"2024-04-07T16:12:13.043982+0000","last_active":"2024-04-07T16:12:14.375041+0000","last_peered":"2024-04-07T16:12:14.375041+0000","last_clean":"2024-04-07T16:12:14.375041+0000","last_became_active":"2024-04-07T16:12:13.041873+0000","last_became_peered":"2024-04-07T16:12:13.041873+0000","last_unstale":"2024-04-07T16:12:14.375041+0000","last_undegraded":"2024-04-07T16:12:14.375041+0000","last_fullsized":"2024-04-07T16:12:14.375041+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T20:03:35.575133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.476449+0000","last_change":"2024-04-07T16:12:13.381810+0000","last_active":"2024-04-07T16:12:14.476449+0000","last_peered":"2024-04-07T16:12:14.476449+0000","last_clean":"2024-04-07T16:12:14.476449+0000","last_became_active":"2024-04-07T16:12:13.381470+0000","last_became_peered":"2024-04-07T16:12:13.381470+0000","last_unstale":"2024-04-07T16:12:14.476449+0000","last_undegraded":"2024-04-07T16:12:14.476449+0000","last_fullsized":"2024-04-07T16:12:14.476449+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T17:26:21.450939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.474605+0000","last_change":"2024-04-07T16:12:13.038191+0000","last_active":"2024-04-07T16:12:14.474605+0000","last_peered":"2024-04-07T16:12:14.474605+0000","last_clean":"2024-04-07T16:12:14.474605+0000","last_became_active":"2024-04-07T16:12:13.037652+0000","last_became_peered":"2024-04-07T16:12:13.037652+0000","last_unstale":"2024-04-07T16:12:14.474605+0000","last_undegraded":"2024-04-07T16:12:14.474605+0000","last_fullsized":"2024-04-07T16:12:14.474605+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T17:04:59.525025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.375473+0000","last_change":"2024-04-07T16:12:13.400660+0000","last_active":"2024-04-07T16:12:14.375473+0000","last_peered":"2024-04-07T16:12:14.375473+0000","last_clean":"2024-04-07T16:12:14.375473+0000","last_became_active":"2024-04-07T16:12:13.400353+0000","last_became_peered":"2024-04-07T16:12:13.400353+0000","last_unstale":"2024-04-07T16:12:14.375473+0000","last_undegraded":"2024-04-07T16:12:14.375473+0000","last_fullsized":"2024-04-07T16:12:14.375473+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T00:47:57.573097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-07T16:12:11.994648+0000","last_change":"2024-04-07T16:12:11.994648+0000","last_active":"2024-04-07T16:12:11.994648+0000","last_peered":"2024-04-07T16:12:11.994648+0000","last_clean":"2024-04-07T16:12:11.994648+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:11.994648+0000","last_undegraded":"2024-04-07T16:12:11.994648+0000","last_fullsized":"2024-04-07T16:12:11.994648+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-07T16:12:12.996208+0000","last_change":"2024-04-07T16:12:12.996208+0000","last_active":"2024-04-07T16:12:12.996208+0000","last_peered":"2024-04-07T16:12:12.996208+0000","last_clean":"2024-04-07T16:12:12.996208+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:12.996208+0000","last_undegraded":"2024-04-07T16:12:12.996208+0000","last_fullsized":"2024-04-07T16:12:12.996208+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"61'2","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.474501+0000","last_change":"2024-04-07T16:12:13.040101+0000","last_active":"2024-04-07T16:12:14.474501+0000","last_peered":"2024-04-07T16:12:14.474501+0000","last_clean":"2024-04-07T16:12:14.474501+0000","last_became_active":"2024-04-07T16:12:13.039797+0000","last_became_peered":"2024-04-07T16:12:13.039797+0000","last_unstale":"2024-04-07T16:12:14.474501+0000","last_undegraded":"2024-04-07T16:12:14.474501+0000","last_fullsized":"2024-04-07T16:12:14.474501+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T03:27:06.016325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.376139+0000","last_change":"2024-04-07T16:12:13.394720+0000","last_active":"2024-04-07T16:12:14.376139+0000","last_peered":"2024-04-07T16:12:14.376139+0000","last_clean":"2024-04-07T16:12:14.376139+0000","last_became_active":"2024-04-07T16:12:13.394090+0000","last_became_peered":"2024-04-07T16:12:13.394090+0000","last_unstale":"2024-04-07T16:12:14.376139+0000","last_undegraded":"2024-04-07T16:12:14.376139+0000","last_fullsized":"2024-04-07T16:12:14.376139+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T00:47:54.579168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.473133+0000","last_change":"2024-04-07T16:12:13.396725+0000","last_active":"2024-04-07T16:12:14.473133+0000","last_peered":"2024-04-07T16:12:14.473133+0000","last_clean":"2024-04-07T16:12:14.473133+0000","last_became_active":"2024-04-07T16:12:13.395978+0000","last_became_peered":"2024-04-07T16:12:13.395978+0000","last_unstale":"2024-04-07T16:12:14.473133+0000","last_undegraded":"2024-04-07T16:12:14.473133+0000","last_fullsized":"2024-04-07T16:12:14.473133+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T21:30:43.276569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.474831+0000","last_change":"2024-04-07T16:12:13.412091+0000","last_active":"2024-04-07T16:12:14.474831+0000","last_peered":"2024-04-07T16:12:14.474831+0000","last_clean":"2024-04-07T16:12:14.474831+0000","last_became_active":"2024-04-07T16:12:13.410814+0000","last_became_peered":"2024-04-07T16:12:13.410814+0000","last_unstale":"2024-04-07T16:12:14.474831+0000","last_undegraded":"2024-04-07T16:12:14.474831+0000","last_fullsized":"2024-04-07T16:12:14.474831+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T01:48:29.034291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-07T16:12:12.996208+0000","last_change":"2024-04-07T16:12:12.996208+0000","last_active":"2024-04-07T16:12:12.996208+0000","last_peered":"2024-04-07T16:12:12.996208+0000","last_clean":"2024-04-07T16:12:12.996208+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:12.996208+0000","last_undegraded":"2024-04-07T16:12:12.996208+0000","last_fullsized":"2024-04-07T16:12:12.996208+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-07T16:12:12.996208+0000","last_change":"2024-04-07T16:12:12.996208+0000","last_active":"2024-04-07T16:12:12.996208+0000","last_peered":"2024-04-07T16:12:12.996208+0000","last_clean":"2024-04-07T16:12:12.996208+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:12.996208+0000","last_undegraded":"2024-04-07T16:12:12.996208+0000","last_fullsized":"2024-04-07T16:12:12.996208+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.475244+0000","last_change":"2024-04-07T16:12:13.398730+0000","last_active":"2024-04-07T16:12:14.475244+0000","last_peered":"2024-04-07T16:12:14.475244+0000","last_clean":"2024-04-07T16:12:14.475244+0000","last_became_active":"2024-04-07T16:12:13.398259+0000","last_became_peered":"2024-04-07T16:12:13.398259+0000","last_unstale":"2024-04-07T16:12:14.475244+0000","last_undegraded":"2024-04-07T16:12:14.475244+0000","last_fullsized":"2024-04-07T16:12:14.475244+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T02:14:28.915631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,0],"acting":[3,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-07T16:12:12.996208+0000","last_change":"2024-04-07T16:12:12.996208+0000","last_active":"2024-04-07T16:12:12.996208+0000","last_peered":"2024-04-07T16:12:12.996208+0000","last_clean":"2024-04-07T16:12:12.996208+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:12.996208+0000","last_undegraded":"2024-04-07T16:12:12.996208+0000","last_fullsized":"2024-04-07T16:12:12.996208+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.475770+0000","last_change":"2024-04-07T16:12:13.382605+0000","last_active":"2024-04-07T16:12:14.475770+0000","last_peered":"2024-04-07T16:12:14.475770+0000","last_clean":"2024-04-07T16:12:14.475770+0000","last_became_active":"2024-04-07T16:12:13.382267+0000","last_became_peered":"2024-04-07T16:12:13.382267+0000","last_unstale":"2024-04-07T16:12:14.475770+0000","last_undegraded":"2024-04-07T16:12:14.475770+0000","last_fullsized":"2024-04-07T16:12:14.475770+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T20:14:03.942753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,4],"acting":[6,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.474734+0000","last_change":"2024-04-07T16:12:13.405175+0000","last_active":"2024-04-07T16:12:14.474734+0000","last_peered":"2024-04-07T16:12:14.474734+0000","last_clean":"2024-04-07T16:12:14.474734+0000","last_became_active":"2024-04-07T16:12:13.404734+0000","last_became_peered":"2024-04-07T16:12:13.404734+0000","last_unstale":"2024-04-07T16:12:14.474734+0000","last_undegraded":"2024-04-07T16:12:14.474734+0000","last_fullsized":"2024-04-07T16:12:14.474734+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T17:55:23.469795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-07T16:12:12.996208+0000","last_change":"2024-04-07T16:12:12.996208+0000","last_active":"2024-04-07T16:12:12.996208+0000","last_peered":"2024-04-07T16:12:12.996208+0000","last_clean":"2024-04-07T16:12:12.996208+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:12.996208+0000","last_undegraded":"2024-04-07T16:12:12.996208+0000","last_fullsized":"2024-04-07T16:12:12.996208+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.475578+0000","last_change":"2024-04-07T16:12:13.397638+0000","last_active":"2024-04-07T16:12:14.475578+0000","last_peered":"2024-04-07T16:12:14.475578+0000","last_clean":"2024-04-07T16:12:14.475578+0000","last_became_active":"2024-04-07T16:12:13.396766+0000","last_became_peered":"2024-04-07T16:12:13.396766+0000","last_unstale":"2024-04-07T16:12:14.475578+0000","last_undegraded":"2024-04-07T16:12:14.475578+0000","last_fullsized":"2024-04-07T16:12:14.475578+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T22:51:06.841718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-07T16:12:12.996208+0000","last_change":"2024-04-07T16:12:12.996208+0000","last_active":"2024-04-07T16:12:12.996208+0000","last_peered":"2024-04-07T16:12:12.996208+0000","last_clean":"2024-04-07T16:12:12.996208+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:12.996208+0000","last_undegraded":"2024-04-07T16:12:12.996208+0000","last_fullsized":"2024-04-07T16:12:12.996208+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-07T16:12:12.996208+0000","last_change":"2024-04-07T16:12:12.996208+0000","last_active":"2024-04-07T16:12:12.996208+0000","last_peered":"2024-04-07T16:12:12.996208+0000","last_clean":"2024-04-07T16:12:12.996208+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:12.996208+0000","last_undegraded":"2024-04-07T16:12:12.996208+0000","last_fullsized":"2024-04-07T16:12:12.996208+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.474174+0000","last_change":"2024-04-07T16:12:13.380077+0000","last_active":"2024-04-07T16:12:14.474174+0000","last_peered":"2024-04-07T16:12:14.474174+0000","last_clean":"2024-04-07T16:12:14.474174+0000","last_became_active":"2024-04-07T16:12:13.379758+0000","last_became_peered":"2024-04-07T16:12:13.379758+0000","last_unstale":"2024-04-07T16:12:14.474174+0000","last_undegraded":"2024-04-07T16:12:14.474174+0000","last_fullsized":"2024-04-07T16:12:14.474174+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T22:47:34.836189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":108,"acting":108,"num_store_stats":5},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":7},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1564,"num_objects":13,"num_object_clones":0,"num_object_copies":39,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":27,"ondisk_log_size":27,"up":78,"acting":78,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":54,"seq":231928233996,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28224,"kb_used_data":1196,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716928,"statfs":{"total":95995035648,"available":95966134272,"internally_reserved":0,"allocated":1224704,"data_stored":850413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.212}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.55}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79600000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0740000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1339999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7290000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4510000000000001}]}]},{"osd":6,"up_from":46,"seq":197568495637,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28224,"kb_used_data":1200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716928,"statfs":{"total":95995035648,"available":95966134272,"internally_reserved":0,"allocated":1228800,"data_stored":851373,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Apr 7 16:11:42 2024","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.27,"5min":1.27,"15min":1.27},"last":3.4079999999999999},{"interface":"front","average":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":1.2849999999999999,"5min":1.2849999999999999,"15min":1.2849999999999999},"last":4.165}]},{"osd":1,"last update":"Sun Apr 7 16:11:42 2024","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.427,"5min":1.427,"15min":1.427},"last":1.8799999999999999},{"interface":"front","average":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.2470000000000001,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":3.4889999999999999}]},{"osd":2,"last update":"Sun Apr 7 16:11:42 2024","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.359,"5min":1.359,"15min":1.359},"last":4.218},{"interface":"front","average":{"1min":0.999,"5min":0.999,"15min":0.999},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.5029999999999999,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":1.0529999999999999}]},{"osd":3,"last update":"Sun Apr 7 16:11:42 2024","interfaces":[{"interface":"back","average":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.363,"5min":1.363,"15min":1.363},"last":4.2809999999999997},{"interface":"front","average":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.2589999999999999,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":3.3079999999999998}]},{"osd":4,"last update":"Sun Apr 7 16:11:42 2024","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.5609999999999999,"5min":1.5609999999999999,"15min":1.5609999999999999},"last":1.216},{"interface":"front","average":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.2809999999999999,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":4.8209999999999997}]},{"osd":5,"last update":"Sun Apr 7 16:11:42 2024","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.3680000000000001,"5min":1.3680000000000001,"15min":1.3680000000000001},"last":1.974},{"interface":"front","average":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.6279999999999999,"5min":1.6279999999999999,"15min":1.6279999999999999},"last":1.1419999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":5.2350000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574916,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27588,"kb_used_data":560,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717564,"statfs":{"total":95995035648,"available":95966785536,"internally_reserved":0,"allocated":573440,"data_stored":232554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Apr 7 16:11:48 2024","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.66300000000000003,"15min":0.57899999999999996},"min":{"1min":0.49299999999999999,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":1.518,"5min":1.518,"15min":1.518},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.871,"5min":0.74299999999999999,"15min":0.629},"min":{"1min":0.59899999999999998,"5min":0.315,"15min":0.315},"max":{"1min":1.2969999999999999,"5min":2.8620000000000001,"15min":2.8620000000000001},"last":0.66600000000000004}]},{"osd":2,"last update":"Sun Apr 7 16:11:31 2024","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.73399999999999999,"15min":0.66000000000000003},"min":{"1min":0.56599999999999995,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.345,"5min":1.345,"15min":1.345},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.71099999999999997,"15min":0.65700000000000003},"min":{"1min":0.60399999999999998,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.5369999999999999,"5min":1.5369999999999999,"15min":1.5369999999999999},"last":0.623}]},{"osd":3,"last update":"Sun Apr 7 16:11:23 2024","interfaces":[{"interface":"back","average":{"1min":1.032,"5min":0.86699999999999999,"15min":0.82299999999999995},"min":{"1min":0.64400000000000002,"5min":0.628,"15min":0.628},"max":{"1min":1.343,"5min":1.343,"15min":1.343},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.91700000000000004,"5min":0.79800000000000004,"15min":0.76200000000000001},"min":{"1min":0.66700000000000004,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.4770000000000001,"5min":1.4770000000000001,"15min":1.4770000000000001},"last":0.56299999999999994}]},{"osd":4,"last update":"Sun Apr 7 16:11:11 2024","interfaces":[{"interface":"back","average":{"1min":0.94999999999999996,"5min":0.93200000000000005,"15min":0.92900000000000005},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.5149999999999999,"5min":1.5149999999999999,"15min":1.5149999999999999},"last":1.099},{"interface":"front","average":{"1min":1.119,"5min":1.056,"15min":1.046},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.9299999999999999,"5min":1.9299999999999999,"15min":1.9299999999999999},"last":0.877}]},{"osd":5,"last update":"Sun Apr 7 16:11:55 2024","interfaces":[{"interface":"back","average":{"1min":1.151,"5min":1.23,"15min":1.2430000000000001},"min":{"1min":0.85199999999999998,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":1.6870000000000001,"5min":2.0510000000000002,"15min":2.0510000000000002},"last":0.93600000000000005},{"interface":"front","average":{"1min":1.099,"5min":1.1339999999999999,"15min":1.1399999999999999},"min":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"max":{"1min":1.577,"5min":1.8859999999999999,"15min":1.8859999999999999},"last":0.96199999999999997}]},{"osd":6,"last update":"Sun Apr 7 16:11:43 2024","interfaces":[{"interface":"back","average":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"min":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"max":{"1min":1.466,"5min":1.466,"15min":1.466},"last":1.1799999999999999},{"interface":"front","average":{"1min":1.274,"5min":1.274,"15min":1.274},"min":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"max":{"1min":1.6040000000000001,"5min":1.6040000000000001,"15min":1.6040000000000001},"last":1.198}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0269999999999999}]}]},{"osd":0,"up_from":8,"seq":34359738447,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28172,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716980,"statfs":{"total":95995035648,"available":95966187520,"internally_reserved":0,"allocated":1179648,"data_stored":829169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Apr 7 16:11:47 2024","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.63600000000000001,"15min":0.56999999999999995},"min":{"1min":0.56000000000000005,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":1.353,"5min":1.353,"15min":1.353},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.70899999999999996,"15min":0.57299999999999995},"min":{"1min":0.58399999999999996,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.1439999999999999,"5min":1.214,"15min":1.214},"last":0.77800000000000002}]},{"osd":2,"last update":"Sun Apr 7 16:11:34 2024","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.70899999999999996,"15min":0.66700000000000004},"min":{"1min":0.61799999999999999,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.89000000000000001,"5min":0.77200000000000002,"15min":0.73699999999999999},"min":{"1min":0.626,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.2749999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.81299999999999994}]},{"osd":3,"last update":"Sun Apr 7 16:11:19 2024","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.78200000000000003,"15min":0.75700000000000001},"min":{"1min":0.67800000000000005,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.89000000000000001,"5min":0.81899999999999995,"15min":0.79900000000000004},"min":{"1min":0.68200000000000005,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.143,"5min":1.143,"15min":1.143},"last":0.95099999999999996}]},{"osd":4,"last update":"Sun Apr 7 16:12:08 2024","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":0.93600000000000005,"15min":0.92300000000000004},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.306,"5min":1.306,"15min":1.306},"last":0.98599999999999999},{"interface":"front","average":{"1min":1.0109999999999999,"5min":0.98799999999999999,"15min":0.96399999999999997},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":1.2150000000000001,"5min":1.7809999999999999,"15min":1.7809999999999999},"last":0.88700000000000001}]},{"osd":5,"last update":"Sun Apr 7 16:11:56 2024","interfaces":[{"interface":"back","average":{"1min":1.0349999999999999,"5min":0.98899999999999999,"15min":0.98099999999999998},"min":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"max":{"1min":1.196,"5min":1.196,"15min":1.196},"last":1.1699999999999999},{"interface":"front","average":{"1min":1.0409999999999999,"5min":1.0249999999999999,"15min":1.0229999999999999},"min":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":1.48,"5min":1.48,"15min":1.48},"last":1.268}]},{"osd":6,"last update":"Sun Apr 7 16:11:41 2024","interfaces":[{"interface":"back","average":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"min":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":1.3600000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":1.0209999999999999},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"min":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"max":{"1min":1.3580000000000001,"5min":1.3580000000000001,"15min":1.3580000000000001},"last":1.2969999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1539999999999999}]}]},{"osd":2,"up_from":18,"seq":77309411388,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31724,"kb_used_data":608,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713428,"statfs":{"total":95995035648,"available":95962550272,"internally_reserved":0,"allocated":622592,"data_stored":259441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Apr 7 16:11:30 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.63700000000000001,"15min":0.58599999999999997},"min":{"1min":0.56499999999999995,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.96199999999999997,"5min":1.103,"15min":1.103},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.67600000000000005,"15min":0.60599999999999998},"min":{"1min":0.57299999999999995,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.75800000000000001}]},{"osd":1,"last update":"Sun Apr 7 16:11:30 2024","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.69399999999999995,"15min":0.64100000000000001},"min":{"1min":0.59099999999999997,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":1.143},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.71699999999999997,"15min":0.64900000000000002},"min":{"1min":0.54200000000000004,"5min":0.313,"15min":0.313},"max":{"1min":1.0609999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.53400000000000003}]},{"osd":3,"last update":"Sun Apr 7 16:11:18 2024","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.81699999999999995,"15min":0.81100000000000005},"min":{"1min":0.67000000000000004,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.81200000000000006,"15min":0.80200000000000005},"min":{"1min":0.66100000000000003,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.002,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.88600000000000001}]},{"osd":4,"last update":"Sun Apr 7 16:12:09 2024","interfaces":[{"interface":"back","average":{"1min":0.96699999999999997,"5min":0.93400000000000005,"15min":0.92100000000000004},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.4299999999999999,"5min":1.4299999999999999,"15min":1.4299999999999999},"last":0.96999999999999997},{"interface":"front","average":{"1min":1.018,"5min":0.95799999999999996,"15min":0.93999999999999995},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.4490000000000001,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.92400000000000004}]},{"osd":5,"last update":"Sun Apr 7 16:11:54 2024","interfaces":[{"interface":"back","average":{"1min":1.05,"5min":0.98699999999999999,"15min":0.97699999999999998},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":1.6970000000000001,"5min":1.6970000000000001,"15min":1.6970000000000001},"last":0.83299999999999996},{"interface":"front","average":{"1min":1.0780000000000001,"5min":1.046,"15min":1.04},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.6000000000000001,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":1.069}]},{"osd":6,"last update":"Sun Apr 7 16:11:39 2024","interfaces":[{"interface":"back","average":{"1min":1.05,"5min":1.05,"15min":1.05},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.397,"5min":1.397,"15min":1.397},"last":1.0009999999999999},{"interface":"front","average":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.355,"5min":1.355,"15min":1.355},"last":1.1040000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80500000000000005}]}]},{"osd":3,"up_from":25,"seq":107374182451,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27656,"kb_used_data":628,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717496,"statfs":{"total":95995035648,"available":95966715904,"internally_reserved":0,"allocated":643072,"data_stored":261109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Apr 7 16:11:24 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.67000000000000004,"15min":0.66900000000000004},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.97399999999999998,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.54400000000000004,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.327,"5min":1.327,"15min":1.327},"last":0.39100000000000001}]},{"osd":1,"last update":"Sun Apr 7 16:11:24 2024","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.77200000000000002,"15min":0.75600000000000001},"min":{"1min":0.57999999999999996,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.544,"5min":1.544,"15min":1.544},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.76600000000000001,"15min":0.75800000000000001},"min":{"1min":0.60599999999999998,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.4510000000000001,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.375}]},{"osd":2,"last update":"Sun Apr 7 16:11:24 2024","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.82499999999999996,"15min":0.80900000000000005},"min":{"1min":0.70699999999999996,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.256,"5min":1.256,"15min":1.256},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.81599999999999995,"15min":0.81399999999999995},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.286,"5min":1.286,"15min":1.286},"last":0.35999999999999999}]},{"osd":4,"last update":"Sun Apr 7 16:12:08 2024","interfaces":[{"interface":"back","average":{"1min":1.036,"5min":0.97699999999999998,"15min":0.95699999999999996},"min":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.28,"5min":1.552,"15min":1.552},"last":0.81000000000000005},{"interface":"front","average":{"1min":1.0569999999999999,"5min":0.999,"15min":0.97699999999999998},"min":{"1min":0.81999999999999995,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.3640000000000001,"5min":1.3640000000000001,"15min":1.3640000000000001},"last":0.65100000000000002}]},{"osd":5,"last update":"Sun Apr 7 16:11:54 2024","interfaces":[{"interface":"back","average":{"1min":1.081,"5min":1.0609999999999999,"15min":1.0580000000000001},"min":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":1.3180000000000001,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":0.70999999999999996},{"interface":"front","average":{"1min":1.0109999999999999,"5min":1.0349999999999999,"15min":1.0389999999999999},"min":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.2410000000000001,"5min":1.571,"15min":1.571},"last":0.66800000000000004}]},{"osd":6,"last update":"Sun Apr 7 16:11:39 2024","interfaces":[{"interface":"back","average":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"min":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"max":{"1min":1.341,"5min":1.341,"15min":1.341},"last":0.84299999999999997},{"interface":"front","average":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"min":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"max":{"1min":1.4610000000000001,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.61399999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64000000000000001}]}]},{"osd":4,"up_from":30,"seq":128849018921,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31688,"kb_used_data":572,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713464,"statfs":{"total":95995035648,"available":95962587136,"internally_reserved":0,"allocated":585728,"data_stored":238801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Apr 7 16:11:10 2024","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.82899999999999996,"15min":0.81699999999999995},"min":{"1min":0.63800000000000001,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.90300000000000002,"15min":0.90500000000000003},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.1719999999999999,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.873}]},{"osd":1,"last update":"Sun Apr 7 16:11:10 2024","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.81299999999999994,"15min":0.80700000000000005},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.91100000000000003,"5min":0.84499999999999997,"15min":0.83399999999999996},"min":{"1min":0.64600000000000002,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.67700000000000005}]},{"osd":2,"last update":"Sun Apr 7 16:11:10 2024","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":0.874,"15min":0.86199999999999999},"min":{"1min":0.75700000000000001,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.2649999999999999,"5min":1.387,"15min":1.387},"last":1.046},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.88600000000000001,"15min":0.88300000000000001},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.175,"5min":1.175,"15min":1.175},"last":0.63}]},{"osd":3,"last update":"Sun Apr 7 16:11:10 2024","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.93100000000000005,"15min":0.93400000000000005},"min":{"1min":0.76700000000000002,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":1.069,"5min":1.458,"15min":1.458},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.94199999999999995,"5min":0.92600000000000005,"15min":0.92300000000000004},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.2390000000000001,"5min":1.323,"15min":1.323},"last":0.83399999999999996}]},{"osd":5,"last update":"Sun Apr 7 16:11:55 2024","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":0.78200000000000003,"15min":0.75600000000000001},"min":{"1min":0.60999999999999999,"5min":0.315,"15min":0.315},"max":{"1min":1.403,"5min":1.403,"15min":1.403},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.92500000000000004,"5min":0.87,"15min":0.86099999999999999},"min":{"1min":0.57299999999999995,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.3220000000000001,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":1.139}]},{"osd":6,"last update":"Sun Apr 7 16:11:42 2024","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.4139999999999999,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":0.98599999999999999},{"interface":"front","average":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.4370000000000001,"5min":1.4370000000000001,"15min":1.4370000000000001},"last":1.2150000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0129999999999999}]}]},{"osd":5,"up_from":38,"seq":163208757278,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27580,"kb_used_data":556,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717572,"statfs":{"total":95995035648,"available":95966793728,"internally_reserved":0,"allocated":569344,"data_stored":232223,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Apr 7 16:11:57 2024","interfaces":[{"interface":"back","average":{"1min":0.97499999999999998,"5min":0.92800000000000005,"15min":0.92000000000000004},"min":{"1min":0.68200000000000005,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.2609999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.94399999999999995,"15min":0.95699999999999996},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.153,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":1.0069999999999999}]},{"osd":1,"last update":"Sun Apr 7 16:11:57 2024","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.91500000000000004,"15min":0.92200000000000004},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.1619999999999999,"5min":1.331,"15min":1.331},"last":0.95399999999999996},{"interface":"front","average":{"1min":0.97899999999999998,"5min":0.98899999999999999,"15min":0.99099999999999999},"min":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.204,"5min":1.45,"15min":1.45},"last":1.1699999999999999}]},{"osd":2,"last update":"Sun Apr 7 16:11:57 2024","interfaces":[{"interface":"back","average":{"1min":0.91300000000000003,"5min":0.96999999999999997,"15min":0.97999999999999998},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.3140000000000001,"5min":1.462,"15min":1.462},"last":1.0449999999999999},{"interface":"front","average":{"1min":0.93500000000000005,"5min":0.96299999999999997,"15min":0.96699999999999997},"min":{"1min":0.67400000000000004,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.3879999999999999,"5min":1.472,"15min":1.472},"last":0.89000000000000001}]},{"osd":3,"last update":"Sun Apr 7 16:11:57 2024","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":1.173,"15min":1.218},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.4219999999999999,"5min":6.3860000000000001,"15min":6.3860000000000001},"last":1.2},{"interface":"front","average":{"1min":0.96199999999999997,"5min":1.0009999999999999,"15min":1.0069999999999999},"min":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.448,"5min":1.556,"15min":1.556},"last":0.79300000000000004}]},{"osd":4,"last update":"Sun Apr 7 16:11:57 2024","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.80900000000000005,"15min":0.80700000000000005},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.91500000000000004},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.90700000000000003,"15min":0.90900000000000003},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.22,"5min":1.4119999999999999,"15min":1.4119999999999999},"last":1.252}]},{"osd":6,"last update":"Sun Apr 7 16:11:39 2024","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":0.75},{"interface":"front","average":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":1.544,"5min":1.544,"15min":1.544},"last":1.125}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0940000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-07T16:12:16.369 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-07T16:12:16.699 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-07T16:12:16.701 INFO:teuthology.orchestra.run.smithi100.stderr:dumped all 2024-04-07T16:12:16.717 INFO:teuthology.orchestra.run.smithi100.stdout:{"pg_ready":true,"pg_map":{"version":318,"stamp":"2024-04-07T16:12:16.287343+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591932,"num_objects":15,"num_object_clones":0,"num_object_copies":45,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":190,"num_write_kb":2419,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":144,"ondisk_log_size":144,"up":189,"acting":189,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":76,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":749961216,"kb_used":230856,"kb_used_data":6472,"kb_used_omap":12,"kb_used_meta":224243,"kb_avail":749730360,"statfs":{"total":767960285184,"available":767723888640,"internally_reserved":0,"allocated":6627328,"data_stored":3755083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":229625401},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1564,"num_objects":13,"num_object_clones":0,"num_object_copies":39,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.290644"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.376089+0000","last_change":"2024-04-07T16:12:13.395367+0000","last_active":"2024-04-07T16:12:14.376089+0000","last_peered":"2024-04-07T16:12:14.376089+0000","last_clean":"2024-04-07T16:12:14.376089+0000","last_became_active":"2024-04-07T16:12:13.395023+0000","last_became_peered":"2024-04-07T16:12:13.395023+0000","last_unstale":"2024-04-07T16:12:14.376089+0000","last_undegraded":"2024-04-07T16:12:14.376089+0000","last_fullsized":"2024-04-07T16:12:14.376089+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T23:05:09.752696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-07T16:12:12.996208+0000","last_change":"2024-04-07T16:12:12.996208+0000","last_active":"2024-04-07T16:12:12.996208+0000","last_peered":"2024-04-07T16:12:12.996208+0000","last_clean":"2024-04-07T16:12:12.996208+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:12.996208+0000","last_undegraded":"2024-04-07T16:12:12.996208+0000","last_fullsized":"2024-04-07T16:12:12.996208+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-07T16:12:12.996208+0000","last_change":"2024-04-07T16:12:12.996208+0000","last_active":"2024-04-07T16:12:12.996208+0000","last_peered":"2024-04-07T16:12:12.996208+0000","last_clean":"2024-04-07T16:12:12.996208+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:12.996208+0000","last_undegraded":"2024-04-07T16:12:12.996208+0000","last_fullsized":"2024-04-07T16:12:12.996208+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-07T16:12:12.996208+0000","last_change":"2024-04-07T16:12:12.996208+0000","last_active":"2024-04-07T16:12:12.996208+0000","last_peered":"2024-04-07T16:12:12.996208+0000","last_clean":"2024-04-07T16:12:12.996208+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:12.996208+0000","last_undegraded":"2024-04-07T16:12:12.996208+0000","last_fullsized":"2024-04-07T16:12:12.996208+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.375284+0000","last_change":"2024-04-07T16:12:13.413487+0000","last_active":"2024-04-07T16:12:14.375284+0000","last_peered":"2024-04-07T16:12:14.375284+0000","last_clean":"2024-04-07T16:12:14.375284+0000","last_became_active":"2024-04-07T16:12:13.413281+0000","last_became_peered":"2024-04-07T16:12:13.413281+0000","last_unstale":"2024-04-07T16:12:14.375284+0000","last_undegraded":"2024-04-07T16:12:14.375284+0000","last_fullsized":"2024-04-07T16:12:14.375284+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T23:59:23.302803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-07T16:12:12.996208+0000","last_change":"2024-04-07T16:12:12.996208+0000","last_active":"2024-04-07T16:12:12.996208+0000","last_peered":"2024-04-07T16:12:12.996208+0000","last_clean":"2024-04-07T16:12:12.996208+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:12.996208+0000","last_undegraded":"2024-04-07T16:12:12.996208+0000","last_fullsized":"2024-04-07T16:12:12.996208+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.374561+0000","last_change":"2024-04-07T16:12:13.393518+0000","last_active":"2024-04-07T16:12:14.374561+0000","last_peered":"2024-04-07T16:12:14.374561+0000","last_clean":"2024-04-07T16:12:14.374561+0000","last_became_active":"2024-04-07T16:12:13.393187+0000","last_became_peered":"2024-04-07T16:12:13.393187+0000","last_unstale":"2024-04-07T16:12:14.374561+0000","last_undegraded":"2024-04-07T16:12:14.374561+0000","last_fullsized":"2024-04-07T16:12:14.374561+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T19:41:28.702905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.475028+0000","last_change":"2024-04-07T16:12:13.394702+0000","last_active":"2024-04-07T16:12:14.475028+0000","last_peered":"2024-04-07T16:12:14.475028+0000","last_clean":"2024-04-07T16:12:14.475028+0000","last_became_active":"2024-04-07T16:12:13.394422+0000","last_became_peered":"2024-04-07T16:12:13.394422+0000","last_unstale":"2024-04-07T16:12:14.475028+0000","last_undegraded":"2024-04-07T16:12:14.475028+0000","last_fullsized":"2024-04-07T16:12:14.475028+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T00:37:44.512471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.375990+0000","last_change":"2024-04-07T16:12:13.401766+0000","last_active":"2024-04-07T16:12:14.375990+0000","last_peered":"2024-04-07T16:12:14.375990+0000","last_clean":"2024-04-07T16:12:14.375990+0000","last_became_active":"2024-04-07T16:12:13.401411+0000","last_became_peered":"2024-04-07T16:12:13.401411+0000","last_unstale":"2024-04-07T16:12:14.375990+0000","last_undegraded":"2024-04-07T16:12:14.375990+0000","last_fullsized":"2024-04-07T16:12:14.375990+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T23:32:52.283410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,0],"acting":[7,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.475526+0000","last_change":"2024-04-07T16:12:13.396409+0000","last_active":"2024-04-07T16:12:14.475526+0000","last_peered":"2024-04-07T16:12:14.475526+0000","last_clean":"2024-04-07T16:12:14.475526+0000","last_became_active":"2024-04-07T16:12:13.396094+0000","last_became_peered":"2024-04-07T16:12:13.396094+0000","last_unstale":"2024-04-07T16:12:14.475526+0000","last_undegraded":"2024-04-07T16:12:14.475526+0000","last_fullsized":"2024-04-07T16:12:14.475526+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T18:54:36.072837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.474549+0000","last_change":"2024-04-07T16:12:13.401973+0000","last_active":"2024-04-07T16:12:14.474549+0000","last_peered":"2024-04-07T16:12:14.474549+0000","last_clean":"2024-04-07T16:12:14.474549+0000","last_became_active":"2024-04-07T16:12:13.401724+0000","last_became_peered":"2024-04-07T16:12:13.401724+0000","last_unstale":"2024-04-07T16:12:14.474549+0000","last_undegraded":"2024-04-07T16:12:14.474549+0000","last_fullsized":"2024-04-07T16:12:14.474549+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T21:26:29.208223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,0],"acting":[6,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-04-07T16:12:13.005867+0000","last_change":"2024-04-07T16:12:13.005867+0000","last_active":"2024-04-07T16:12:12.996208+0000","last_peered":"2024-04-07T16:12:12.996208+0000","last_clean":"2024-04-07T16:12:12.996208+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:13.005867+0000","last_undegraded":"2024-04-07T16:12:13.005867+0000","last_fullsized":"2024-04-07T16:12:13.005867+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T22:29:22.509564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,0],"acting":[4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.474547+0000","last_change":"2024-04-07T16:12:13.394760+0000","last_active":"2024-04-07T16:12:14.474547+0000","last_peered":"2024-04-07T16:12:14.474547+0000","last_clean":"2024-04-07T16:12:14.474547+0000","last_became_active":"2024-04-07T16:12:13.394402+0000","last_became_peered":"2024-04-07T16:12:13.394402+0000","last_unstale":"2024-04-07T16:12:14.474547+0000","last_undegraded":"2024-04-07T16:12:14.474547+0000","last_fullsized":"2024-04-07T16:12:14.474547+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T18:47:13.891972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-07T16:12:12.996208+0000","last_change":"2024-04-07T16:12:12.996208+0000","last_active":"2024-04-07T16:12:12.996208+0000","last_peered":"2024-04-07T16:12:12.996208+0000","last_clean":"2024-04-07T16:12:12.996208+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:12.996208+0000","last_undegraded":"2024-04-07T16:12:12.996208+0000","last_fullsized":"2024-04-07T16:12:12.996208+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-07T16:12:12.996208+0000","last_change":"2024-04-07T16:12:12.996208+0000","last_active":"2024-04-07T16:12:12.996208+0000","last_peered":"2024-04-07T16:12:12.996208+0000","last_clean":"2024-04-07T16:12:12.996208+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:12.996208+0000","last_undegraded":"2024-04-07T16:12:12.996208+0000","last_fullsized":"2024-04-07T16:12:12.996208+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-07T16:12:12.996208+0000","last_change":"2024-04-07T16:12:12.996208+0000","last_active":"2024-04-07T16:12:12.996208+0000","last_peered":"2024-04-07T16:12:12.996208+0000","last_clean":"2024-04-07T16:12:12.996208+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:12.996208+0000","last_undegraded":"2024-04-07T16:12:12.996208+0000","last_fullsized":"2024-04-07T16:12:12.996208+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-07T16:12:12.996208+0000","last_change":"2024-04-07T16:12:12.996208+0000","last_active":"2024-04-07T16:12:12.996208+0000","last_peered":"2024-04-07T16:12:12.996208+0000","last_clean":"2024-04-07T16:12:12.996208+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:12.996208+0000","last_undegraded":"2024-04-07T16:12:12.996208+0000","last_fullsized":"2024-04-07T16:12:12.996208+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.474396+0000","last_change":"2024-04-07T16:12:13.416210+0000","last_active":"2024-04-07T16:12:14.474396+0000","last_peered":"2024-04-07T16:12:14.474396+0000","last_clean":"2024-04-07T16:12:14.474396+0000","last_became_active":"2024-04-07T16:12:13.415972+0000","last_became_peered":"2024-04-07T16:12:13.415972+0000","last_unstale":"2024-04-07T16:12:14.474396+0000","last_undegraded":"2024-04-07T16:12:14.474396+0000","last_fullsized":"2024-04-07T16:12:14.474396+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T23:40:11.606978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,7],"acting":[2,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.375192+0000","last_change":"2024-04-07T16:12:13.414203+0000","last_active":"2024-04-07T16:12:14.375192+0000","last_peered":"2024-04-07T16:12:14.375192+0000","last_clean":"2024-04-07T16:12:14.375192+0000","last_became_active":"2024-04-07T16:12:13.413869+0000","last_became_peered":"2024-04-07T16:12:13.413869+0000","last_unstale":"2024-04-07T16:12:14.375192+0000","last_undegraded":"2024-04-07T16:12:14.375192+0000","last_fullsized":"2024-04-07T16:12:14.375192+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T17:11:47.895568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,6],"acting":[7,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-04-07T16:12:12.000271+0000","last_change":"2024-04-07T16:12:12.000271+0000","last_active":"2024-04-07T16:12:11.994648+0000","last_peered":"2024-04-07T16:12:11.994648+0000","last_clean":"2024-04-07T16:12:11.994648+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:12.000271+0000","last_undegraded":"2024-04-07T16:12:12.000271+0000","last_fullsized":"2024-04-07T16:12:12.000271+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T19:01:44.798719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.474373+0000","last_change":"2024-04-07T16:12:13.407852+0000","last_active":"2024-04-07T16:12:14.474373+0000","last_peered":"2024-04-07T16:12:14.474373+0000","last_clean":"2024-04-07T16:12:14.474373+0000","last_became_active":"2024-04-07T16:12:13.407538+0000","last_became_peered":"2024-04-07T16:12:13.407538+0000","last_unstale":"2024-04-07T16:12:14.474373+0000","last_undegraded":"2024-04-07T16:12:14.474373+0000","last_fullsized":"2024-04-07T16:12:14.474373+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T20:27:52.664149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"61'1","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.474471+0000","last_change":"2024-04-07T16:12:13.038807+0000","last_active":"2024-04-07T16:12:14.474471+0000","last_peered":"2024-04-07T16:12:14.474471+0000","last_clean":"2024-04-07T16:12:14.474471+0000","last_became_active":"2024-04-07T16:12:13.038492+0000","last_became_peered":"2024-04-07T16:12:13.038492+0000","last_unstale":"2024-04-07T16:12:14.474471+0000","last_undegraded":"2024-04-07T16:12:14.474471+0000","last_fullsized":"2024-04-07T16:12:14.474471+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T02:11:03.746017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-07T16:12:12.996208+0000","last_change":"2024-04-07T16:12:12.996208+0000","last_active":"2024-04-07T16:12:12.996208+0000","last_peered":"2024-04-07T16:12:12.996208+0000","last_clean":"2024-04-07T16:12:12.996208+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:12.996208+0000","last_undegraded":"2024-04-07T16:12:12.996208+0000","last_fullsized":"2024-04-07T16:12:12.996208+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"61'2","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.375664+0000","last_change":"2024-04-07T16:12:13.042401+0000","last_active":"2024-04-07T16:12:14.375664+0000","last_peered":"2024-04-07T16:12:14.375664+0000","last_clean":"2024-04-07T16:12:14.375664+0000","last_became_active":"2024-04-07T16:12:13.041970+0000","last_became_peered":"2024-04-07T16:12:13.041970+0000","last_unstale":"2024-04-07T16:12:14.375664+0000","last_undegraded":"2024-04-07T16:12:14.375664+0000","last_fullsized":"2024-04-07T16:12:14.375664+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T16:36:29.235347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-07T16:12:12.996208+0000","last_change":"2024-04-07T16:12:12.996208+0000","last_active":"2024-04-07T16:12:12.996208+0000","last_peered":"2024-04-07T16:12:12.996208+0000","last_clean":"2024-04-07T16:12:12.996208+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:12.996208+0000","last_undegraded":"2024-04-07T16:12:12.996208+0000","last_fullsized":"2024-04-07T16:12:12.996208+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"61'2","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.375447+0000","last_change":"2024-04-07T16:12:13.042391+0000","last_active":"2024-04-07T16:12:14.375447+0000","last_peered":"2024-04-07T16:12:14.375447+0000","last_clean":"2024-04-07T16:12:14.375447+0000","last_became_active":"2024-04-07T16:12:13.041919+0000","last_became_peered":"2024-04-07T16:12:13.041919+0000","last_unstale":"2024-04-07T16:12:14.375447+0000","last_undegraded":"2024-04-07T16:12:14.375447+0000","last_fullsized":"2024-04-07T16:12:14.375447+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T21:50:59.576615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-07T16:12:12.996208+0000","last_change":"2024-04-07T16:12:12.996208+0000","last_active":"2024-04-07T16:12:12.996208+0000","last_peered":"2024-04-07T16:12:12.996208+0000","last_clean":"2024-04-07T16:12:12.996208+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:12.996208+0000","last_undegraded":"2024-04-07T16:12:12.996208+0000","last_fullsized":"2024-04-07T16:12:12.996208+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"61'1","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.474818+0000","last_change":"2024-04-07T16:12:13.041880+0000","last_active":"2024-04-07T16:12:14.474818+0000","last_peered":"2024-04-07T16:12:14.474818+0000","last_clean":"2024-04-07T16:12:14.474818+0000","last_became_active":"2024-04-07T16:12:13.041409+0000","last_became_peered":"2024-04-07T16:12:13.041409+0000","last_unstale":"2024-04-07T16:12:14.474818+0000","last_undegraded":"2024-04-07T16:12:14.474818+0000","last_fullsized":"2024-04-07T16:12:14.474818+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T18:32:23.206985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-04-07T16:12:13.002508+0000","last_change":"2024-04-07T16:12:13.002508+0000","last_active":"2024-04-07T16:12:12.996208+0000","last_peered":"2024-04-07T16:12:12.996208+0000","last_clean":"2024-04-07T16:12:12.996208+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:13.002508+0000","last_undegraded":"2024-04-07T16:12:13.002508+0000","last_fullsized":"2024-04-07T16:12:13.002508+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T23:20:40.949255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"61'1","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.476584+0000","last_change":"2024-04-07T16:12:13.038960+0000","last_active":"2024-04-07T16:12:14.476584+0000","last_peered":"2024-04-07T16:12:14.476584+0000","last_clean":"2024-04-07T16:12:14.476584+0000","last_became_active":"2024-04-07T16:12:13.038410+0000","last_became_peered":"2024-04-07T16:12:13.038410+0000","last_unstale":"2024-04-07T16:12:14.476584+0000","last_undegraded":"2024-04-07T16:12:14.476584+0000","last_fullsized":"2024-04-07T16:12:14.476584+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T03:41:38.418312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-07T16:12:12.996208+0000","last_change":"2024-04-07T16:12:12.996208+0000","last_active":"2024-04-07T16:12:12.996208+0000","last_peered":"2024-04-07T16:12:12.996208+0000","last_clean":"2024-04-07T16:12:12.996208+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:12.996208+0000","last_undegraded":"2024-04-07T16:12:12.996208+0000","last_fullsized":"2024-04-07T16:12:12.996208+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.475758+0000","last_change":"2024-04-07T16:12:13.034420+0000","last_active":"2024-04-07T16:12:14.475758+0000","last_peered":"2024-04-07T16:12:14.475758+0000","last_clean":"2024-04-07T16:12:14.475758+0000","last_became_active":"2024-04-07T16:12:13.034013+0000","last_became_peered":"2024-04-07T16:12:13.034013+0000","last_unstale":"2024-04-07T16:12:14.475758+0000","last_undegraded":"2024-04-07T16:12:14.475758+0000","last_fullsized":"2024-04-07T16:12:14.475758+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T19:51:21.525865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.474201+0000","last_change":"2024-04-07T16:12:13.398531+0000","last_active":"2024-04-07T16:12:14.474201+0000","last_peered":"2024-04-07T16:12:14.474201+0000","last_clean":"2024-04-07T16:12:14.474201+0000","last_became_active":"2024-04-07T16:12:13.398017+0000","last_became_peered":"2024-04-07T16:12:13.398017+0000","last_unstale":"2024-04-07T16:12:14.474201+0000","last_undegraded":"2024-04-07T16:12:14.474201+0000","last_fullsized":"2024-04-07T16:12:14.474201+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T17:33:54.306902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-04-07T16:12:12.000806+0000","last_change":"2024-04-07T16:12:12.000806+0000","last_active":"2024-04-07T16:12:11.994648+0000","last_peered":"2024-04-07T16:12:11.994648+0000","last_clean":"2024-04-07T16:12:11.994648+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:12.000806+0000","last_undegraded":"2024-04-07T16:12:12.000806+0000","last_fullsized":"2024-04-07T16:12:12.000806+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T22:44:14.623116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-07T16:12:12.996208+0000","last_change":"2024-04-07T16:12:12.996208+0000","last_active":"2024-04-07T16:12:12.996208+0000","last_peered":"2024-04-07T16:12:12.996208+0000","last_clean":"2024-04-07T16:12:12.996208+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:12.996208+0000","last_undegraded":"2024-04-07T16:12:12.996208+0000","last_fullsized":"2024-04-07T16:12:12.996208+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.474717+0000","last_change":"2024-04-07T16:12:13.030149+0000","last_active":"2024-04-07T16:12:14.474717+0000","last_peered":"2024-04-07T16:12:14.474717+0000","last_clean":"2024-04-07T16:12:14.474717+0000","last_became_active":"2024-04-07T16:12:13.029525+0000","last_became_peered":"2024-04-07T16:12:13.029525+0000","last_unstale":"2024-04-07T16:12:14.474717+0000","last_undegraded":"2024-04-07T16:12:14.474717+0000","last_fullsized":"2024-04-07T16:12:14.474717+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T20:09:35.180264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-07T16:12:12.996208+0000","last_change":"2024-04-07T16:12:12.996208+0000","last_active":"2024-04-07T16:12:12.996208+0000","last_peered":"2024-04-07T16:12:12.996208+0000","last_clean":"2024-04-07T16:12:12.996208+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:12.996208+0000","last_undegraded":"2024-04-07T16:12:12.996208+0000","last_fullsized":"2024-04-07T16:12:12.996208+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-04-07T16:12:12.001015+0000","last_change":"2024-04-07T16:12:12.001015+0000","last_active":"2024-04-07T16:12:11.994648+0000","last_peered":"2024-04-07T16:12:11.994648+0000","last_clean":"2024-04-07T16:12:11.994648+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:12.001015+0000","last_undegraded":"2024-04-07T16:12:12.001015+0000","last_fullsized":"2024-04-07T16:12:12.001015+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T18:26:16.873044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-07T16:12:12.996208+0000","last_change":"2024-04-07T16:12:12.996208+0000","last_active":"2024-04-07T16:12:12.996208+0000","last_peered":"2024-04-07T16:12:12.996208+0000","last_clean":"2024-04-07T16:12:12.996208+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:12.996208+0000","last_undegraded":"2024-04-07T16:12:12.996208+0000","last_fullsized":"2024-04-07T16:12:12.996208+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-07T16:12:11.994648+0000","last_change":"2024-04-07T16:12:11.994648+0000","last_active":"2024-04-07T16:12:11.994648+0000","last_peered":"2024-04-07T16:12:11.994648+0000","last_clean":"2024-04-07T16:12:11.994648+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:11.994648+0000","last_undegraded":"2024-04-07T16:12:11.994648+0000","last_fullsized":"2024-04-07T16:12:11.994648+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-04-07T16:12:13.004673+0000","last_change":"2024-04-07T16:12:13.004673+0000","last_active":"2024-04-07T16:12:12.996208+0000","last_peered":"2024-04-07T16:12:12.996208+0000","last_clean":"2024-04-07T16:12:12.996208+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:13.004673+0000","last_undegraded":"2024-04-07T16:12:13.004673+0000","last_fullsized":"2024-04-07T16:12:13.004673+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T16:55:37.429799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"61'6","reported_seq":18,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.473934+0000","last_change":"2024-04-07T16:12:13.031673+0000","last_active":"2024-04-07T16:12:14.473934+0000","last_peered":"2024-04-07T16:12:14.473934+0000","last_clean":"2024-04-07T16:12:14.473934+0000","last_became_active":"2024-04-07T16:12:13.031092+0000","last_became_peered":"2024-04-07T16:12:13.031092+0000","last_unstale":"2024-04-07T16:12:14.473934+0000","last_undegraded":"2024-04-07T16:12:14.473934+0000","last_fullsized":"2024-04-07T16:12:14.473934+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T20:17:48.593862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.375355+0000","last_change":"2024-04-07T16:12:13.390083+0000","last_active":"2024-04-07T16:12:14.375355+0000","last_peered":"2024-04-07T16:12:14.375355+0000","last_clean":"2024-04-07T16:12:14.375355+0000","last_became_active":"2024-04-07T16:12:13.389755+0000","last_became_peered":"2024-04-07T16:12:13.389755+0000","last_unstale":"2024-04-07T16:12:14.375355+0000","last_undegraded":"2024-04-07T16:12:14.375355+0000","last_fullsized":"2024-04-07T16:12:14.375355+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T02:31:09.155569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-04-07T16:12:12.001190+0000","last_change":"2024-04-07T16:12:12.001190+0000","last_active":"2024-04-07T16:12:11.994648+0000","last_peered":"2024-04-07T16:12:11.994648+0000","last_clean":"2024-04-07T16:12:11.994648+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:12.001190+0000","last_undegraded":"2024-04-07T16:12:12.001190+0000","last_fullsized":"2024-04-07T16:12:12.001190+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T01:50:02.195507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-07T16:12:12.996208+0000","last_change":"2024-04-07T16:12:12.996208+0000","last_active":"2024-04-07T16:12:12.996208+0000","last_peered":"2024-04-07T16:12:12.996208+0000","last_clean":"2024-04-07T16:12:12.996208+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:12.996208+0000","last_undegraded":"2024-04-07T16:12:12.996208+0000","last_fullsized":"2024-04-07T16:12:12.996208+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-04-07T16:12:12.002099+0000","last_change":"2024-04-07T16:12:12.002099+0000","last_active":"2024-04-07T16:12:11.994648+0000","last_peered":"2024-04-07T16:12:11.994648+0000","last_clean":"2024-04-07T16:12:11.994648+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:12.002099+0000","last_undegraded":"2024-04-07T16:12:12.002099+0000","last_fullsized":"2024-04-07T16:12:12.002099+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T23:15:11.494437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.375013+0000","last_change":"2024-04-07T16:12:13.413928+0000","last_active":"2024-04-07T16:12:14.375013+0000","last_peered":"2024-04-07T16:12:14.375013+0000","last_clean":"2024-04-07T16:12:14.375013+0000","last_became_active":"2024-04-07T16:12:13.413447+0000","last_became_peered":"2024-04-07T16:12:13.413447+0000","last_unstale":"2024-04-07T16:12:14.375013+0000","last_undegraded":"2024-04-07T16:12:14.375013+0000","last_fullsized":"2024-04-07T16:12:14.375013+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T03:26:42.875228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.473099+0000","last_change":"2024-04-07T16:12:13.032942+0000","last_active":"2024-04-07T16:12:14.473099+0000","last_peered":"2024-04-07T16:12:14.473099+0000","last_clean":"2024-04-07T16:12:14.473099+0000","last_became_active":"2024-04-07T16:12:13.032108+0000","last_became_peered":"2024-04-07T16:12:13.032108+0000","last_unstale":"2024-04-07T16:12:14.473099+0000","last_undegraded":"2024-04-07T16:12:14.473099+0000","last_fullsized":"2024-04-07T16:12:14.473099+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T18:53:06.864412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.473331+0000","last_change":"2024-04-07T16:12:13.400141+0000","last_active":"2024-04-07T16:12:14.473331+0000","last_peered":"2024-04-07T16:12:14.473331+0000","last_clean":"2024-04-07T16:12:14.473331+0000","last_became_active":"2024-04-07T16:12:13.399823+0000","last_became_peered":"2024-04-07T16:12:13.399823+0000","last_unstale":"2024-04-07T16:12:14.473331+0000","last_undegraded":"2024-04-07T16:12:14.473331+0000","last_fullsized":"2024-04-07T16:12:14.473331+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T21:27:25.248595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"61'8","reported_seq":20,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.474350+0000","last_change":"2024-04-07T16:12:13.034154+0000","last_active":"2024-04-07T16:12:14.474350+0000","last_peered":"2024-04-07T16:12:14.474350+0000","last_clean":"2024-04-07T16:12:14.474350+0000","last_became_active":"2024-04-07T16:12:13.033838+0000","last_became_peered":"2024-04-07T16:12:13.033838+0000","last_unstale":"2024-04-07T16:12:14.474350+0000","last_undegraded":"2024-04-07T16:12:14.474350+0000","last_fullsized":"2024-04-07T16:12:14.474350+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T23:13:07.216314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.374716+0000","last_change":"2024-04-07T16:12:13.385883+0000","last_active":"2024-04-07T16:12:14.374716+0000","last_peered":"2024-04-07T16:12:14.374716+0000","last_clean":"2024-04-07T16:12:14.374716+0000","last_became_active":"2024-04-07T16:12:13.385485+0000","last_became_peered":"2024-04-07T16:12:13.385485+0000","last_unstale":"2024-04-07T16:12:14.374716+0000","last_undegraded":"2024-04-07T16:12:14.374716+0000","last_fullsized":"2024-04-07T16:12:14.374716+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T21:01:09.262433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":4,"reported_epoch":60,"state":"creating+activating","last_fresh":"2024-04-07T16:12:13.024777+0000","last_change":"2024-04-07T16:12:13.002417+0000","last_active":"2024-04-07T16:12:11.994648+0000","last_peered":"2024-04-07T16:12:11.994648+0000","last_clean":"2024-04-07T16:12:11.994648+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:13.024777+0000","last_undegraded":"2024-04-07T16:12:13.024777+0000","last_fullsized":"2024-04-07T16:12:13.024777+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T17:55:06.727142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":["4","0","7"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.375770+0000","last_change":"2024-04-07T16:12:13.386584+0000","last_active":"2024-04-07T16:12:14.375770+0000","last_peered":"2024-04-07T16:12:14.375770+0000","last_clean":"2024-04-07T16:12:14.375770+0000","last_became_active":"2024-04-07T16:12:13.386057+0000","last_became_peered":"2024-04-07T16:12:13.386057+0000","last_unstale":"2024-04-07T16:12:14.375770+0000","last_undegraded":"2024-04-07T16:12:14.375770+0000","last_fullsized":"2024-04-07T16:12:14.375770+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T17:57:57.548179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-07T16:12:11.994648+0000","last_change":"2024-04-07T16:12:11.994648+0000","last_active":"2024-04-07T16:12:11.994648+0000","last_peered":"2024-04-07T16:12:11.994648+0000","last_clean":"2024-04-07T16:12:11.994648+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:11.994648+0000","last_undegraded":"2024-04-07T16:12:11.994648+0000","last_fullsized":"2024-04-07T16:12:11.994648+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-07T16:12:12.996208+0000","last_change":"2024-04-07T16:12:12.996208+0000","last_active":"2024-04-07T16:12:12.996208+0000","last_peered":"2024-04-07T16:12:12.996208+0000","last_clean":"2024-04-07T16:12:12.996208+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:12.996208+0000","last_undegraded":"2024-04-07T16:12:12.996208+0000","last_fullsized":"2024-04-07T16:12:12.996208+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-04-07T16:12:12.001190+0000","last_change":"2024-04-07T16:12:12.001190+0000","last_active":"2024-04-07T16:12:11.994648+0000","last_peered":"2024-04-07T16:12:11.994648+0000","last_clean":"2024-04-07T16:12:11.994648+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:12.001190+0000","last_undegraded":"2024-04-07T16:12:12.001190+0000","last_fullsized":"2024-04-07T16:12:12.001190+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T18:15:00.506827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-04-07T16:12:13.003585+0000","last_change":"2024-04-07T16:12:13.003585+0000","last_active":"2024-04-07T16:12:12.996208+0000","last_peered":"2024-04-07T16:12:12.996208+0000","last_clean":"2024-04-07T16:12:12.996208+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:13.003585+0000","last_undegraded":"2024-04-07T16:12:13.003585+0000","last_fullsized":"2024-04-07T16:12:13.003585+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T03:36:24.354110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-07T16:12:12.996208+0000","last_change":"2024-04-07T16:12:12.996208+0000","last_active":"2024-04-07T16:12:12.996208+0000","last_peered":"2024-04-07T16:12:12.996208+0000","last_clean":"2024-04-07T16:12:12.996208+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:12.996208+0000","last_undegraded":"2024-04-07T16:12:12.996208+0000","last_fullsized":"2024-04-07T16:12:12.996208+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.474169+0000","last_change":"2024-04-07T16:12:13.035071+0000","last_active":"2024-04-07T16:12:14.474169+0000","last_peered":"2024-04-07T16:12:14.474169+0000","last_clean":"2024-04-07T16:12:14.474169+0000","last_became_active":"2024-04-07T16:12:13.034795+0000","last_became_peered":"2024-04-07T16:12:13.034795+0000","last_unstale":"2024-04-07T16:12:14.474169+0000","last_undegraded":"2024-04-07T16:12:14.474169+0000","last_fullsized":"2024-04-07T16:12:14.474169+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T17:51:30.599527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.476063+0000","last_change":"2024-04-07T16:12:13.394635+0000","last_active":"2024-04-07T16:12:14.476063+0000","last_peered":"2024-04-07T16:12:14.476063+0000","last_clean":"2024-04-07T16:12:14.476063+0000","last_became_active":"2024-04-07T16:12:13.394392+0000","last_became_peered":"2024-04-07T16:12:13.394392+0000","last_unstale":"2024-04-07T16:12:14.476063+0000","last_undegraded":"2024-04-07T16:12:14.476063+0000","last_fullsized":"2024-04-07T16:12:14.476063+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T17:52:10.950481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-04-07T16:12:12.001243+0000","last_change":"2024-04-07T16:12:12.001243+0000","last_active":"2024-04-07T16:12:11.994648+0000","last_peered":"2024-04-07T16:12:11.994648+0000","last_clean":"2024-04-07T16:12:11.994648+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:12.001243+0000","last_undegraded":"2024-04-07T16:12:12.001243+0000","last_fullsized":"2024-04-07T16:12:12.001243+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T02:48:50.652491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"21'117","reported_seq":219,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-07T16:12:13.375677+0000","last_change":"2024-04-07T16:11:33.518868+0000","last_active":"2024-04-07T16:12:13.375677+0000","last_peered":"2024-04-07T16:12:13.375677+0000","last_clean":"2024-04-07T16:12:13.375677+0000","last_became_active":"2024-04-07T16:11:33.518060+0000","last_became_peered":"2024-04-07T16:11:33.518060+0000","last_unstale":"2024-04-07T16:12:13.375677+0000","last_undegraded":"2024-04-07T16:12:13.375677+0000","last_fullsized":"2024-04-07T16:12:13.375677+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:07:26.866003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:07:26.866003+0000","last_clean_scrub_stamp":"2024-04-07T16:07:26.866003+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T22:22:51.043941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-07T16:12:12.996208+0000","last_change":"2024-04-07T16:12:12.996208+0000","last_active":"2024-04-07T16:12:12.996208+0000","last_peered":"2024-04-07T16:12:12.996208+0000","last_clean":"2024-04-07T16:12:12.996208+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:12.996208+0000","last_undegraded":"2024-04-07T16:12:12.996208+0000","last_fullsized":"2024-04-07T16:12:12.996208+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.375081+0000","last_change":"2024-04-07T16:12:13.035536+0000","last_active":"2024-04-07T16:12:14.375081+0000","last_peered":"2024-04-07T16:12:14.375081+0000","last_clean":"2024-04-07T16:12:14.375081+0000","last_became_active":"2024-04-07T16:12:13.034825+0000","last_became_peered":"2024-04-07T16:12:13.034825+0000","last_unstale":"2024-04-07T16:12:14.375081+0000","last_undegraded":"2024-04-07T16:12:14.375081+0000","last_fullsized":"2024-04-07T16:12:14.375081+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T20:11:57.033306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.375586+0000","last_change":"2024-04-07T16:12:13.035747+0000","last_active":"2024-04-07T16:12:14.375586+0000","last_peered":"2024-04-07T16:12:14.375586+0000","last_clean":"2024-04-07T16:12:14.375586+0000","last_became_active":"2024-04-07T16:12:13.035201+0000","last_became_peered":"2024-04-07T16:12:13.035201+0000","last_unstale":"2024-04-07T16:12:14.375586+0000","last_undegraded":"2024-04-07T16:12:14.375586+0000","last_fullsized":"2024-04-07T16:12:14.375586+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T21:03:25.449728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-07T16:12:12.996208+0000","last_change":"2024-04-07T16:12:12.996208+0000","last_active":"2024-04-07T16:12:12.996208+0000","last_peered":"2024-04-07T16:12:12.996208+0000","last_clean":"2024-04-07T16:12:12.996208+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:12.996208+0000","last_undegraded":"2024-04-07T16:12:12.996208+0000","last_fullsized":"2024-04-07T16:12:12.996208+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-07T16:12:11.994648+0000","last_change":"2024-04-07T16:12:11.994648+0000","last_active":"2024-04-07T16:12:11.994648+0000","last_peered":"2024-04-07T16:12:11.994648+0000","last_clean":"2024-04-07T16:12:11.994648+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:11.994648+0000","last_undegraded":"2024-04-07T16:12:11.994648+0000","last_fullsized":"2024-04-07T16:12:11.994648+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.474648+0000","last_change":"2024-04-07T16:12:13.399486+0000","last_active":"2024-04-07T16:12:14.474648+0000","last_peered":"2024-04-07T16:12:14.474648+0000","last_clean":"2024-04-07T16:12:14.474648+0000","last_became_active":"2024-04-07T16:12:13.399184+0000","last_became_peered":"2024-04-07T16:12:13.399184+0000","last_unstale":"2024-04-07T16:12:14.474648+0000","last_undegraded":"2024-04-07T16:12:14.474648+0000","last_fullsized":"2024-04-07T16:12:14.474648+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T21:07:25.104309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"61'2","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.474473+0000","last_change":"2024-04-07T16:12:13.038351+0000","last_active":"2024-04-07T16:12:14.474473+0000","last_peered":"2024-04-07T16:12:14.474473+0000","last_clean":"2024-04-07T16:12:14.474473+0000","last_became_active":"2024-04-07T16:12:13.038026+0000","last_became_peered":"2024-04-07T16:12:13.038026+0000","last_unstale":"2024-04-07T16:12:14.474473+0000","last_undegraded":"2024-04-07T16:12:14.474473+0000","last_fullsized":"2024-04-07T16:12:14.474473+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T19:39:42.796342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-07T16:12:12.996208+0000","last_change":"2024-04-07T16:12:12.996208+0000","last_active":"2024-04-07T16:12:12.996208+0000","last_peered":"2024-04-07T16:12:12.996208+0000","last_clean":"2024-04-07T16:12:12.996208+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:12.996208+0000","last_undegraded":"2024-04-07T16:12:12.996208+0000","last_fullsized":"2024-04-07T16:12:12.996208+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.374594+0000","last_change":"2024-04-07T16:12:13.033067+0000","last_active":"2024-04-07T16:12:14.374594+0000","last_peered":"2024-04-07T16:12:14.374594+0000","last_clean":"2024-04-07T16:12:14.374594+0000","last_became_active":"2024-04-07T16:12:13.032456+0000","last_became_peered":"2024-04-07T16:12:13.032456+0000","last_unstale":"2024-04-07T16:12:14.374594+0000","last_undegraded":"2024-04-07T16:12:14.374594+0000","last_fullsized":"2024-04-07T16:12:14.374594+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T23:02:09.025617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-04-07T16:12:13.001861+0000","last_change":"2024-04-07T16:12:13.001861+0000","last_active":"2024-04-07T16:12:12.996208+0000","last_peered":"2024-04-07T16:12:12.996208+0000","last_clean":"2024-04-07T16:12:12.996208+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:13.001861+0000","last_undegraded":"2024-04-07T16:12:13.001861+0000","last_fullsized":"2024-04-07T16:12:13.001861+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T23:48:08.846232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-07T16:12:11.994648+0000","last_change":"2024-04-07T16:12:11.994648+0000","last_active":"2024-04-07T16:12:11.994648+0000","last_peered":"2024-04-07T16:12:11.994648+0000","last_clean":"2024-04-07T16:12:11.994648+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:11.994648+0000","last_undegraded":"2024-04-07T16:12:11.994648+0000","last_fullsized":"2024-04-07T16:12:11.994648+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.474100+0000","last_change":"2024-04-07T16:12:13.392396+0000","last_active":"2024-04-07T16:12:14.474100+0000","last_peered":"2024-04-07T16:12:14.474100+0000","last_clean":"2024-04-07T16:12:14.474100+0000","last_became_active":"2024-04-07T16:12:13.392016+0000","last_became_peered":"2024-04-07T16:12:13.392016+0000","last_unstale":"2024-04-07T16:12:14.474100+0000","last_undegraded":"2024-04-07T16:12:14.474100+0000","last_fullsized":"2024-04-07T16:12:14.474100+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T02:35:50.790051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-07T16:12:11.994648+0000","last_change":"2024-04-07T16:12:11.994648+0000","last_active":"2024-04-07T16:12:11.994648+0000","last_peered":"2024-04-07T16:12:11.994648+0000","last_clean":"2024-04-07T16:12:11.994648+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:11.994648+0000","last_undegraded":"2024-04-07T16:12:11.994648+0000","last_fullsized":"2024-04-07T16:12:11.994648+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.474697+0000","last_change":"2024-04-07T16:12:13.397819+0000","last_active":"2024-04-07T16:12:14.474697+0000","last_peered":"2024-04-07T16:12:14.474697+0000","last_clean":"2024-04-07T16:12:14.474697+0000","last_became_active":"2024-04-07T16:12:13.397593+0000","last_became_peered":"2024-04-07T16:12:13.397593+0000","last_unstale":"2024-04-07T16:12:14.474697+0000","last_undegraded":"2024-04-07T16:12:14.474697+0000","last_fullsized":"2024-04-07T16:12:14.474697+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T02:30:01.472030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"61'2","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.375041+0000","last_change":"2024-04-07T16:12:13.043982+0000","last_active":"2024-04-07T16:12:14.375041+0000","last_peered":"2024-04-07T16:12:14.375041+0000","last_clean":"2024-04-07T16:12:14.375041+0000","last_became_active":"2024-04-07T16:12:13.041873+0000","last_became_peered":"2024-04-07T16:12:13.041873+0000","last_unstale":"2024-04-07T16:12:14.375041+0000","last_undegraded":"2024-04-07T16:12:14.375041+0000","last_fullsized":"2024-04-07T16:12:14.375041+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T20:03:35.575133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.476449+0000","last_change":"2024-04-07T16:12:13.381810+0000","last_active":"2024-04-07T16:12:14.476449+0000","last_peered":"2024-04-07T16:12:14.476449+0000","last_clean":"2024-04-07T16:12:14.476449+0000","last_became_active":"2024-04-07T16:12:13.381470+0000","last_became_peered":"2024-04-07T16:12:13.381470+0000","last_unstale":"2024-04-07T16:12:14.476449+0000","last_undegraded":"2024-04-07T16:12:14.476449+0000","last_fullsized":"2024-04-07T16:12:14.476449+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T17:26:21.450939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.474605+0000","last_change":"2024-04-07T16:12:13.038191+0000","last_active":"2024-04-07T16:12:14.474605+0000","last_peered":"2024-04-07T16:12:14.474605+0000","last_clean":"2024-04-07T16:12:14.474605+0000","last_became_active":"2024-04-07T16:12:13.037652+0000","last_became_peered":"2024-04-07T16:12:13.037652+0000","last_unstale":"2024-04-07T16:12:14.474605+0000","last_undegraded":"2024-04-07T16:12:14.474605+0000","last_fullsized":"2024-04-07T16:12:14.474605+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T17:04:59.525025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.375473+0000","last_change":"2024-04-07T16:12:13.400660+0000","last_active":"2024-04-07T16:12:14.375473+0000","last_peered":"2024-04-07T16:12:14.375473+0000","last_clean":"2024-04-07T16:12:14.375473+0000","last_became_active":"2024-04-07T16:12:13.400353+0000","last_became_peered":"2024-04-07T16:12:13.400353+0000","last_unstale":"2024-04-07T16:12:14.375473+0000","last_undegraded":"2024-04-07T16:12:14.375473+0000","last_fullsized":"2024-04-07T16:12:14.375473+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T00:47:57.573097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-07T16:12:11.994648+0000","last_change":"2024-04-07T16:12:11.994648+0000","last_active":"2024-04-07T16:12:11.994648+0000","last_peered":"2024-04-07T16:12:11.994648+0000","last_clean":"2024-04-07T16:12:11.994648+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:11.994648+0000","last_undegraded":"2024-04-07T16:12:11.994648+0000","last_fullsized":"2024-04-07T16:12:11.994648+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-07T16:12:12.996208+0000","last_change":"2024-04-07T16:12:12.996208+0000","last_active":"2024-04-07T16:12:12.996208+0000","last_peered":"2024-04-07T16:12:12.996208+0000","last_clean":"2024-04-07T16:12:12.996208+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:12.996208+0000","last_undegraded":"2024-04-07T16:12:12.996208+0000","last_fullsized":"2024-04-07T16:12:12.996208+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"61'2","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.474501+0000","last_change":"2024-04-07T16:12:13.040101+0000","last_active":"2024-04-07T16:12:14.474501+0000","last_peered":"2024-04-07T16:12:14.474501+0000","last_clean":"2024-04-07T16:12:14.474501+0000","last_became_active":"2024-04-07T16:12:13.039797+0000","last_became_peered":"2024-04-07T16:12:13.039797+0000","last_unstale":"2024-04-07T16:12:14.474501+0000","last_undegraded":"2024-04-07T16:12:14.474501+0000","last_fullsized":"2024-04-07T16:12:14.474501+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T03:27:06.016325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.376139+0000","last_change":"2024-04-07T16:12:13.394720+0000","last_active":"2024-04-07T16:12:14.376139+0000","last_peered":"2024-04-07T16:12:14.376139+0000","last_clean":"2024-04-07T16:12:14.376139+0000","last_became_active":"2024-04-07T16:12:13.394090+0000","last_became_peered":"2024-04-07T16:12:13.394090+0000","last_unstale":"2024-04-07T16:12:14.376139+0000","last_undegraded":"2024-04-07T16:12:14.376139+0000","last_fullsized":"2024-04-07T16:12:14.376139+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T00:47:54.579168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.473133+0000","last_change":"2024-04-07T16:12:13.396725+0000","last_active":"2024-04-07T16:12:14.473133+0000","last_peered":"2024-04-07T16:12:14.473133+0000","last_clean":"2024-04-07T16:12:14.473133+0000","last_became_active":"2024-04-07T16:12:13.395978+0000","last_became_peered":"2024-04-07T16:12:13.395978+0000","last_unstale":"2024-04-07T16:12:14.473133+0000","last_undegraded":"2024-04-07T16:12:14.473133+0000","last_fullsized":"2024-04-07T16:12:14.473133+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T21:30:43.276569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.474831+0000","last_change":"2024-04-07T16:12:13.412091+0000","last_active":"2024-04-07T16:12:14.474831+0000","last_peered":"2024-04-07T16:12:14.474831+0000","last_clean":"2024-04-07T16:12:14.474831+0000","last_became_active":"2024-04-07T16:12:13.410814+0000","last_became_peered":"2024-04-07T16:12:13.410814+0000","last_unstale":"2024-04-07T16:12:14.474831+0000","last_undegraded":"2024-04-07T16:12:14.474831+0000","last_fullsized":"2024-04-07T16:12:14.474831+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T01:48:29.034291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-07T16:12:12.996208+0000","last_change":"2024-04-07T16:12:12.996208+0000","last_active":"2024-04-07T16:12:12.996208+0000","last_peered":"2024-04-07T16:12:12.996208+0000","last_clean":"2024-04-07T16:12:12.996208+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:12.996208+0000","last_undegraded":"2024-04-07T16:12:12.996208+0000","last_fullsized":"2024-04-07T16:12:12.996208+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-07T16:12:12.996208+0000","last_change":"2024-04-07T16:12:12.996208+0000","last_active":"2024-04-07T16:12:12.996208+0000","last_peered":"2024-04-07T16:12:12.996208+0000","last_clean":"2024-04-07T16:12:12.996208+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:12.996208+0000","last_undegraded":"2024-04-07T16:12:12.996208+0000","last_fullsized":"2024-04-07T16:12:12.996208+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.475244+0000","last_change":"2024-04-07T16:12:13.398730+0000","last_active":"2024-04-07T16:12:14.475244+0000","last_peered":"2024-04-07T16:12:14.475244+0000","last_clean":"2024-04-07T16:12:14.475244+0000","last_became_active":"2024-04-07T16:12:13.398259+0000","last_became_peered":"2024-04-07T16:12:13.398259+0000","last_unstale":"2024-04-07T16:12:14.475244+0000","last_undegraded":"2024-04-07T16:12:14.475244+0000","last_fullsized":"2024-04-07T16:12:14.475244+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T02:14:28.915631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,0],"acting":[3,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-07T16:12:12.996208+0000","last_change":"2024-04-07T16:12:12.996208+0000","last_active":"2024-04-07T16:12:12.996208+0000","last_peered":"2024-04-07T16:12:12.996208+0000","last_clean":"2024-04-07T16:12:12.996208+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:12.996208+0000","last_undegraded":"2024-04-07T16:12:12.996208+0000","last_fullsized":"2024-04-07T16:12:12.996208+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.475770+0000","last_change":"2024-04-07T16:12:13.382605+0000","last_active":"2024-04-07T16:12:14.475770+0000","last_peered":"2024-04-07T16:12:14.475770+0000","last_clean":"2024-04-07T16:12:14.475770+0000","last_became_active":"2024-04-07T16:12:13.382267+0000","last_became_peered":"2024-04-07T16:12:13.382267+0000","last_unstale":"2024-04-07T16:12:14.475770+0000","last_undegraded":"2024-04-07T16:12:14.475770+0000","last_fullsized":"2024-04-07T16:12:14.475770+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T20:14:03.942753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,4],"acting":[6,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.474734+0000","last_change":"2024-04-07T16:12:13.405175+0000","last_active":"2024-04-07T16:12:14.474734+0000","last_peered":"2024-04-07T16:12:14.474734+0000","last_clean":"2024-04-07T16:12:14.474734+0000","last_became_active":"2024-04-07T16:12:13.404734+0000","last_became_peered":"2024-04-07T16:12:13.404734+0000","last_unstale":"2024-04-07T16:12:14.474734+0000","last_undegraded":"2024-04-07T16:12:14.474734+0000","last_fullsized":"2024-04-07T16:12:14.474734+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T17:55:23.469795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-07T16:12:12.996208+0000","last_change":"2024-04-07T16:12:12.996208+0000","last_active":"2024-04-07T16:12:12.996208+0000","last_peered":"2024-04-07T16:12:12.996208+0000","last_clean":"2024-04-07T16:12:12.996208+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:12.996208+0000","last_undegraded":"2024-04-07T16:12:12.996208+0000","last_fullsized":"2024-04-07T16:12:12.996208+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.475578+0000","last_change":"2024-04-07T16:12:13.397638+0000","last_active":"2024-04-07T16:12:14.475578+0000","last_peered":"2024-04-07T16:12:14.475578+0000","last_clean":"2024-04-07T16:12:14.475578+0000","last_became_active":"2024-04-07T16:12:13.396766+0000","last_became_peered":"2024-04-07T16:12:13.396766+0000","last_unstale":"2024-04-07T16:12:14.475578+0000","last_undegraded":"2024-04-07T16:12:14.475578+0000","last_fullsized":"2024-04-07T16:12:14.475578+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T22:51:06.841718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-07T16:12:12.996208+0000","last_change":"2024-04-07T16:12:12.996208+0000","last_active":"2024-04-07T16:12:12.996208+0000","last_peered":"2024-04-07T16:12:12.996208+0000","last_clean":"2024-04-07T16:12:12.996208+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:12.996208+0000","last_undegraded":"2024-04-07T16:12:12.996208+0000","last_fullsized":"2024-04-07T16:12:12.996208+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-07T16:12:12.996208+0000","last_change":"2024-04-07T16:12:12.996208+0000","last_active":"2024-04-07T16:12:12.996208+0000","last_peered":"2024-04-07T16:12:12.996208+0000","last_clean":"2024-04-07T16:12:12.996208+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:12.996208+0000","last_undegraded":"2024-04-07T16:12:12.996208+0000","last_fullsized":"2024-04-07T16:12:12.996208+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.474174+0000","last_change":"2024-04-07T16:12:13.380077+0000","last_active":"2024-04-07T16:12:14.474174+0000","last_peered":"2024-04-07T16:12:14.474174+0000","last_clean":"2024-04-07T16:12:14.474174+0000","last_became_active":"2024-04-07T16:12:13.379758+0000","last_became_peered":"2024-04-07T16:12:13.379758+0000","last_unstale":"2024-04-07T16:12:14.474174+0000","last_undegraded":"2024-04-07T16:12:14.474174+0000","last_fullsized":"2024-04-07T16:12:14.474174+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T22:47:34.836189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":108,"acting":108,"num_store_stats":5},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":7},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1564,"num_objects":13,"num_object_clones":0,"num_object_copies":39,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":27,"ondisk_log_size":27,"up":78,"acting":78,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":54,"seq":231928233996,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28224,"kb_used_data":1196,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716928,"statfs":{"total":95995035648,"available":95966134272,"internally_reserved":0,"allocated":1224704,"data_stored":850413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.212}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.55}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79600000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0740000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1339999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7290000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4510000000000001}]}]},{"osd":6,"up_from":46,"seq":197568495637,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28224,"kb_used_data":1200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716928,"statfs":{"total":95995035648,"available":95966134272,"internally_reserved":0,"allocated":1228800,"data_stored":851373,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Apr 7 16:11:42 2024","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.27,"5min":1.27,"15min":1.27},"last":3.4079999999999999},{"interface":"front","average":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":1.2849999999999999,"5min":1.2849999999999999,"15min":1.2849999999999999},"last":4.165}]},{"osd":1,"last update":"Sun Apr 7 16:11:42 2024","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.427,"5min":1.427,"15min":1.427},"last":1.8799999999999999},{"interface":"front","average":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.2470000000000001,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":3.4889999999999999}]},{"osd":2,"last update":"Sun Apr 7 16:11:42 2024","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.359,"5min":1.359,"15min":1.359},"last":4.218},{"interface":"front","average":{"1min":0.999,"5min":0.999,"15min":0.999},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.5029999999999999,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":1.0529999999999999}]},{"osd":3,"last update":"Sun Apr 7 16:11:42 2024","interfaces":[{"interface":"back","average":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.363,"5min":1.363,"15min":1.363},"last":4.2809999999999997},{"interface":"front","average":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.2589999999999999,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":3.3079999999999998}]},{"osd":4,"last update":"Sun Apr 7 16:11:42 2024","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.5609999999999999,"5min":1.5609999999999999,"15min":1.5609999999999999},"last":1.216},{"interface":"front","average":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.2809999999999999,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":4.8209999999999997}]},{"osd":5,"last update":"Sun Apr 7 16:11:42 2024","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.3680000000000001,"5min":1.3680000000000001,"15min":1.3680000000000001},"last":1.974},{"interface":"front","average":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.6279999999999999,"5min":1.6279999999999999,"15min":1.6279999999999999},"last":1.1419999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":5.2350000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574916,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27588,"kb_used_data":560,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717564,"statfs":{"total":95995035648,"available":95966785536,"internally_reserved":0,"allocated":573440,"data_stored":232554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Apr 7 16:11:48 2024","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.66300000000000003,"15min":0.57899999999999996},"min":{"1min":0.49299999999999999,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":1.518,"5min":1.518,"15min":1.518},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.871,"5min":0.74299999999999999,"15min":0.629},"min":{"1min":0.59899999999999998,"5min":0.315,"15min":0.315},"max":{"1min":1.2969999999999999,"5min":2.8620000000000001,"15min":2.8620000000000001},"last":0.66600000000000004}]},{"osd":2,"last update":"Sun Apr 7 16:11:31 2024","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.73399999999999999,"15min":0.66000000000000003},"min":{"1min":0.56599999999999995,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.345,"5min":1.345,"15min":1.345},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.71099999999999997,"15min":0.65700000000000003},"min":{"1min":0.60399999999999998,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.5369999999999999,"5min":1.5369999999999999,"15min":1.5369999999999999},"last":0.623}]},{"osd":3,"last update":"Sun Apr 7 16:11:23 2024","interfaces":[{"interface":"back","average":{"1min":1.032,"5min":0.86699999999999999,"15min":0.82299999999999995},"min":{"1min":0.64400000000000002,"5min":0.628,"15min":0.628},"max":{"1min":1.343,"5min":1.343,"15min":1.343},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.91700000000000004,"5min":0.79800000000000004,"15min":0.76200000000000001},"min":{"1min":0.66700000000000004,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.4770000000000001,"5min":1.4770000000000001,"15min":1.4770000000000001},"last":0.56299999999999994}]},{"osd":4,"last update":"Sun Apr 7 16:11:11 2024","interfaces":[{"interface":"back","average":{"1min":0.94999999999999996,"5min":0.93200000000000005,"15min":0.92900000000000005},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.5149999999999999,"5min":1.5149999999999999,"15min":1.5149999999999999},"last":1.099},{"interface":"front","average":{"1min":1.119,"5min":1.056,"15min":1.046},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.9299999999999999,"5min":1.9299999999999999,"15min":1.9299999999999999},"last":0.877}]},{"osd":5,"last update":"Sun Apr 7 16:11:55 2024","interfaces":[{"interface":"back","average":{"1min":1.151,"5min":1.23,"15min":1.2430000000000001},"min":{"1min":0.85199999999999998,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":1.6870000000000001,"5min":2.0510000000000002,"15min":2.0510000000000002},"last":0.93600000000000005},{"interface":"front","average":{"1min":1.099,"5min":1.1339999999999999,"15min":1.1399999999999999},"min":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"max":{"1min":1.577,"5min":1.8859999999999999,"15min":1.8859999999999999},"last":0.96199999999999997}]},{"osd":6,"last update":"Sun Apr 7 16:11:43 2024","interfaces":[{"interface":"back","average":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"min":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"max":{"1min":1.466,"5min":1.466,"15min":1.466},"last":1.1799999999999999},{"interface":"front","average":{"1min":1.274,"5min":1.274,"15min":1.274},"min":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"max":{"1min":1.6040000000000001,"5min":1.6040000000000001,"15min":1.6040000000000001},"last":1.198}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0269999999999999}]}]},{"osd":0,"up_from":8,"seq":34359738447,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28172,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716980,"statfs":{"total":95995035648,"available":95966187520,"internally_reserved":0,"allocated":1179648,"data_stored":829169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Apr 7 16:11:47 2024","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.63600000000000001,"15min":0.56999999999999995},"min":{"1min":0.56000000000000005,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":1.353,"5min":1.353,"15min":1.353},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.70899999999999996,"15min":0.57299999999999995},"min":{"1min":0.58399999999999996,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.1439999999999999,"5min":1.214,"15min":1.214},"last":0.77800000000000002}]},{"osd":2,"last update":"Sun Apr 7 16:11:34 2024","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.70899999999999996,"15min":0.66700000000000004},"min":{"1min":0.61799999999999999,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.89000000000000001,"5min":0.77200000000000002,"15min":0.73699999999999999},"min":{"1min":0.626,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.2749999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.81299999999999994}]},{"osd":3,"last update":"Sun Apr 7 16:11:19 2024","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.78200000000000003,"15min":0.75700000000000001},"min":{"1min":0.67800000000000005,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.89000000000000001,"5min":0.81899999999999995,"15min":0.79900000000000004},"min":{"1min":0.68200000000000005,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.143,"5min":1.143,"15min":1.143},"last":0.95099999999999996}]},{"osd":4,"last update":"Sun Apr 7 16:12:08 2024","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":0.93600000000000005,"15min":0.92300000000000004},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.306,"5min":1.306,"15min":1.306},"last":0.98599999999999999},{"interface":"front","average":{"1min":1.0109999999999999,"5min":0.98799999999999999,"15min":0.96399999999999997},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":1.2150000000000001,"5min":1.7809999999999999,"15min":1.7809999999999999},"last":0.88700000000000001}]},{"osd":5,"last update":"Sun Apr 7 16:11:56 2024","interfaces":[{"interface":"back","average":{"1min":1.0349999999999999,"5min":0.98899999999999999,"15min":0.98099999999999998},"min":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"max":{"1min":1.196,"5min":1.196,"15min":1.196},"last":1.1699999999999999},{"interface":"front","average":{"1min":1.0409999999999999,"5min":1.0249999999999999,"15min":1.0229999999999999},"min":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":1.48,"5min":1.48,"15min":1.48},"last":1.268}]},{"osd":6,"last update":"Sun Apr 7 16:11:41 2024","interfaces":[{"interface":"back","average":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"min":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":1.3600000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":1.0209999999999999},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"min":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"max":{"1min":1.3580000000000001,"5min":1.3580000000000001,"15min":1.3580000000000001},"last":1.2969999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1539999999999999}]}]},{"osd":2,"up_from":18,"seq":77309411388,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31724,"kb_used_data":608,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713428,"statfs":{"total":95995035648,"available":95962550272,"internally_reserved":0,"allocated":622592,"data_stored":259441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Apr 7 16:11:30 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.63700000000000001,"15min":0.58599999999999997},"min":{"1min":0.56499999999999995,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.96199999999999997,"5min":1.103,"15min":1.103},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.67600000000000005,"15min":0.60599999999999998},"min":{"1min":0.57299999999999995,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.75800000000000001}]},{"osd":1,"last update":"Sun Apr 7 16:11:30 2024","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.69399999999999995,"15min":0.64100000000000001},"min":{"1min":0.59099999999999997,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":1.143},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.71699999999999997,"15min":0.64900000000000002},"min":{"1min":0.54200000000000004,"5min":0.313,"15min":0.313},"max":{"1min":1.0609999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.53400000000000003}]},{"osd":3,"last update":"Sun Apr 7 16:11:18 2024","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.81699999999999995,"15min":0.81100000000000005},"min":{"1min":0.67000000000000004,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.81200000000000006,"15min":0.80200000000000005},"min":{"1min":0.66100000000000003,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.002,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.88600000000000001}]},{"osd":4,"last update":"Sun Apr 7 16:12:09 2024","interfaces":[{"interface":"back","average":{"1min":0.96699999999999997,"5min":0.93400000000000005,"15min":0.92100000000000004},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.4299999999999999,"5min":1.4299999999999999,"15min":1.4299999999999999},"last":0.96999999999999997},{"interface":"front","average":{"1min":1.018,"5min":0.95799999999999996,"15min":0.93999999999999995},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.4490000000000001,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.92400000000000004}]},{"osd":5,"last update":"Sun Apr 7 16:11:54 2024","interfaces":[{"interface":"back","average":{"1min":1.05,"5min":0.98699999999999999,"15min":0.97699999999999998},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":1.6970000000000001,"5min":1.6970000000000001,"15min":1.6970000000000001},"last":0.83299999999999996},{"interface":"front","average":{"1min":1.0780000000000001,"5min":1.046,"15min":1.04},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.6000000000000001,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":1.069}]},{"osd":6,"last update":"Sun Apr 7 16:11:39 2024","interfaces":[{"interface":"back","average":{"1min":1.05,"5min":1.05,"15min":1.05},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.397,"5min":1.397,"15min":1.397},"last":1.0009999999999999},{"interface":"front","average":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.355,"5min":1.355,"15min":1.355},"last":1.1040000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80500000000000005}]}]},{"osd":3,"up_from":25,"seq":107374182451,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27656,"kb_used_data":628,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717496,"statfs":{"total":95995035648,"available":95966715904,"internally_reserved":0,"allocated":643072,"data_stored":261109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Apr 7 16:11:24 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.67000000000000004,"15min":0.66900000000000004},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.97399999999999998,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.54400000000000004,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.327,"5min":1.327,"15min":1.327},"last":0.39100000000000001}]},{"osd":1,"last update":"Sun Apr 7 16:11:24 2024","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.77200000000000002,"15min":0.75600000000000001},"min":{"1min":0.57999999999999996,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.544,"5min":1.544,"15min":1.544},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.76600000000000001,"15min":0.75800000000000001},"min":{"1min":0.60599999999999998,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.4510000000000001,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.375}]},{"osd":2,"last update":"Sun Apr 7 16:11:24 2024","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.82499999999999996,"15min":0.80900000000000005},"min":{"1min":0.70699999999999996,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.256,"5min":1.256,"15min":1.256},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.81599999999999995,"15min":0.81399999999999995},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.286,"5min":1.286,"15min":1.286},"last":0.35999999999999999}]},{"osd":4,"last update":"Sun Apr 7 16:12:08 2024","interfaces":[{"interface":"back","average":{"1min":1.036,"5min":0.97699999999999998,"15min":0.95699999999999996},"min":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.28,"5min":1.552,"15min":1.552},"last":0.81000000000000005},{"interface":"front","average":{"1min":1.0569999999999999,"5min":0.999,"15min":0.97699999999999998},"min":{"1min":0.81999999999999995,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.3640000000000001,"5min":1.3640000000000001,"15min":1.3640000000000001},"last":0.65100000000000002}]},{"osd":5,"last update":"Sun Apr 7 16:11:54 2024","interfaces":[{"interface":"back","average":{"1min":1.081,"5min":1.0609999999999999,"15min":1.0580000000000001},"min":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":1.3180000000000001,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":0.70999999999999996},{"interface":"front","average":{"1min":1.0109999999999999,"5min":1.0349999999999999,"15min":1.0389999999999999},"min":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.2410000000000001,"5min":1.571,"15min":1.571},"last":0.66800000000000004}]},{"osd":6,"last update":"Sun Apr 7 16:11:39 2024","interfaces":[{"interface":"back","average":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"min":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"max":{"1min":1.341,"5min":1.341,"15min":1.341},"last":0.84299999999999997},{"interface":"front","average":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"min":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"max":{"1min":1.4610000000000001,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.61399999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64000000000000001}]}]},{"osd":4,"up_from":30,"seq":128849018921,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31688,"kb_used_data":572,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713464,"statfs":{"total":95995035648,"available":95962587136,"internally_reserved":0,"allocated":585728,"data_stored":238801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Apr 7 16:11:10 2024","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.82899999999999996,"15min":0.81699999999999995},"min":{"1min":0.63800000000000001,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.90300000000000002,"15min":0.90500000000000003},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.1719999999999999,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.873}]},{"osd":1,"last update":"Sun Apr 7 16:11:10 2024","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.81299999999999994,"15min":0.80700000000000005},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.91100000000000003,"5min":0.84499999999999997,"15min":0.83399999999999996},"min":{"1min":0.64600000000000002,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.67700000000000005}]},{"osd":2,"last update":"Sun Apr 7 16:11:10 2024","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":0.874,"15min":0.86199999999999999},"min":{"1min":0.75700000000000001,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.2649999999999999,"5min":1.387,"15min":1.387},"last":1.046},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.88600000000000001,"15min":0.88300000000000001},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.175,"5min":1.175,"15min":1.175},"last":0.63}]},{"osd":3,"last update":"Sun Apr 7 16:11:10 2024","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.93100000000000005,"15min":0.93400000000000005},"min":{"1min":0.76700000000000002,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":1.069,"5min":1.458,"15min":1.458},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.94199999999999995,"5min":0.92600000000000005,"15min":0.92300000000000004},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.2390000000000001,"5min":1.323,"15min":1.323},"last":0.83399999999999996}]},{"osd":5,"last update":"Sun Apr 7 16:11:55 2024","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":0.78200000000000003,"15min":0.75600000000000001},"min":{"1min":0.60999999999999999,"5min":0.315,"15min":0.315},"max":{"1min":1.403,"5min":1.403,"15min":1.403},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.92500000000000004,"5min":0.87,"15min":0.86099999999999999},"min":{"1min":0.57299999999999995,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.3220000000000001,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":1.139}]},{"osd":6,"last update":"Sun Apr 7 16:11:42 2024","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.4139999999999999,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":0.98599999999999999},{"interface":"front","average":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.4370000000000001,"5min":1.4370000000000001,"15min":1.4370000000000001},"last":1.2150000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0129999999999999}]}]},{"osd":5,"up_from":38,"seq":163208757278,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27580,"kb_used_data":556,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717572,"statfs":{"total":95995035648,"available":95966793728,"internally_reserved":0,"allocated":569344,"data_stored":232223,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Apr 7 16:11:57 2024","interfaces":[{"interface":"back","average":{"1min":0.97499999999999998,"5min":0.92800000000000005,"15min":0.92000000000000004},"min":{"1min":0.68200000000000005,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.2609999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.94399999999999995,"15min":0.95699999999999996},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.153,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":1.0069999999999999}]},{"osd":1,"last update":"Sun Apr 7 16:11:57 2024","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.91500000000000004,"15min":0.92200000000000004},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.1619999999999999,"5min":1.331,"15min":1.331},"last":0.95399999999999996},{"interface":"front","average":{"1min":0.97899999999999998,"5min":0.98899999999999999,"15min":0.99099999999999999},"min":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.204,"5min":1.45,"15min":1.45},"last":1.1699999999999999}]},{"osd":2,"last update":"Sun Apr 7 16:11:57 2024","interfaces":[{"interface":"back","average":{"1min":0.91300000000000003,"5min":0.96999999999999997,"15min":0.97999999999999998},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.3140000000000001,"5min":1.462,"15min":1.462},"last":1.0449999999999999},{"interface":"front","average":{"1min":0.93500000000000005,"5min":0.96299999999999997,"15min":0.96699999999999997},"min":{"1min":0.67400000000000004,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.3879999999999999,"5min":1.472,"15min":1.472},"last":0.89000000000000001}]},{"osd":3,"last update":"Sun Apr 7 16:11:57 2024","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":1.173,"15min":1.218},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.4219999999999999,"5min":6.3860000000000001,"15min":6.3860000000000001},"last":1.2},{"interface":"front","average":{"1min":0.96199999999999997,"5min":1.0009999999999999,"15min":1.0069999999999999},"min":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.448,"5min":1.556,"15min":1.556},"last":0.79300000000000004}]},{"osd":4,"last update":"Sun Apr 7 16:11:57 2024","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.80900000000000005,"15min":0.80700000000000005},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.91500000000000004},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.90700000000000003,"15min":0.90900000000000003},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.22,"5min":1.4119999999999999,"15min":1.4119999999999999},"last":1.252}]},{"osd":6,"last update":"Sun Apr 7 16:11:39 2024","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":0.75},{"interface":"front","average":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":1.544,"5min":1.544,"15min":1.544},"last":1.125}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0940000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-07T16:12:16.721 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-07T16:12:17.089 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-07T16:12:17.091 INFO:teuthology.orchestra.run.smithi100.stderr:dumped all 2024-04-07T16:12:17.109 INFO:teuthology.orchestra.run.smithi100.stdout:{"pg_ready":true,"pg_map":{"version":318,"stamp":"2024-04-07T16:12:16.287343+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591932,"num_objects":15,"num_object_clones":0,"num_object_copies":45,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":190,"num_write_kb":2419,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":144,"ondisk_log_size":144,"up":189,"acting":189,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":76,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":749961216,"kb_used":230856,"kb_used_data":6472,"kb_used_omap":12,"kb_used_meta":224243,"kb_avail":749730360,"statfs":{"total":767960285184,"available":767723888640,"internally_reserved":0,"allocated":6627328,"data_stored":3755083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":229625401},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1564,"num_objects":13,"num_object_clones":0,"num_object_copies":39,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.290644"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.376089+0000","last_change":"2024-04-07T16:12:13.395367+0000","last_active":"2024-04-07T16:12:14.376089+0000","last_peered":"2024-04-07T16:12:14.376089+0000","last_clean":"2024-04-07T16:12:14.376089+0000","last_became_active":"2024-04-07T16:12:13.395023+0000","last_became_peered":"2024-04-07T16:12:13.395023+0000","last_unstale":"2024-04-07T16:12:14.376089+0000","last_undegraded":"2024-04-07T16:12:14.376089+0000","last_fullsized":"2024-04-07T16:12:14.376089+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T23:05:09.752696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-07T16:12:12.996208+0000","last_change":"2024-04-07T16:12:12.996208+0000","last_active":"2024-04-07T16:12:12.996208+0000","last_peered":"2024-04-07T16:12:12.996208+0000","last_clean":"2024-04-07T16:12:12.996208+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:12.996208+0000","last_undegraded":"2024-04-07T16:12:12.996208+0000","last_fullsized":"2024-04-07T16:12:12.996208+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-07T16:12:12.996208+0000","last_change":"2024-04-07T16:12:12.996208+0000","last_active":"2024-04-07T16:12:12.996208+0000","last_peered":"2024-04-07T16:12:12.996208+0000","last_clean":"2024-04-07T16:12:12.996208+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:12.996208+0000","last_undegraded":"2024-04-07T16:12:12.996208+0000","last_fullsized":"2024-04-07T16:12:12.996208+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-07T16:12:12.996208+0000","last_change":"2024-04-07T16:12:12.996208+0000","last_active":"2024-04-07T16:12:12.996208+0000","last_peered":"2024-04-07T16:12:12.996208+0000","last_clean":"2024-04-07T16:12:12.996208+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:12.996208+0000","last_undegraded":"2024-04-07T16:12:12.996208+0000","last_fullsized":"2024-04-07T16:12:12.996208+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.375284+0000","last_change":"2024-04-07T16:12:13.413487+0000","last_active":"2024-04-07T16:12:14.375284+0000","last_peered":"2024-04-07T16:12:14.375284+0000","last_clean":"2024-04-07T16:12:14.375284+0000","last_became_active":"2024-04-07T16:12:13.413281+0000","last_became_peered":"2024-04-07T16:12:13.413281+0000","last_unstale":"2024-04-07T16:12:14.375284+0000","last_undegraded":"2024-04-07T16:12:14.375284+0000","last_fullsized":"2024-04-07T16:12:14.375284+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T23:59:23.302803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-07T16:12:12.996208+0000","last_change":"2024-04-07T16:12:12.996208+0000","last_active":"2024-04-07T16:12:12.996208+0000","last_peered":"2024-04-07T16:12:12.996208+0000","last_clean":"2024-04-07T16:12:12.996208+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:12.996208+0000","last_undegraded":"2024-04-07T16:12:12.996208+0000","last_fullsized":"2024-04-07T16:12:12.996208+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.374561+0000","last_change":"2024-04-07T16:12:13.393518+0000","last_active":"2024-04-07T16:12:14.374561+0000","last_peered":"2024-04-07T16:12:14.374561+0000","last_clean":"2024-04-07T16:12:14.374561+0000","last_became_active":"2024-04-07T16:12:13.393187+0000","last_became_peered":"2024-04-07T16:12:13.393187+0000","last_unstale":"2024-04-07T16:12:14.374561+0000","last_undegraded":"2024-04-07T16:12:14.374561+0000","last_fullsized":"2024-04-07T16:12:14.374561+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T19:41:28.702905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.475028+0000","last_change":"2024-04-07T16:12:13.394702+0000","last_active":"2024-04-07T16:12:14.475028+0000","last_peered":"2024-04-07T16:12:14.475028+0000","last_clean":"2024-04-07T16:12:14.475028+0000","last_became_active":"2024-04-07T16:12:13.394422+0000","last_became_peered":"2024-04-07T16:12:13.394422+0000","last_unstale":"2024-04-07T16:12:14.475028+0000","last_undegraded":"2024-04-07T16:12:14.475028+0000","last_fullsized":"2024-04-07T16:12:14.475028+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T00:37:44.512471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.375990+0000","last_change":"2024-04-07T16:12:13.401766+0000","last_active":"2024-04-07T16:12:14.375990+0000","last_peered":"2024-04-07T16:12:14.375990+0000","last_clean":"2024-04-07T16:12:14.375990+0000","last_became_active":"2024-04-07T16:12:13.401411+0000","last_became_peered":"2024-04-07T16:12:13.401411+0000","last_unstale":"2024-04-07T16:12:14.375990+0000","last_undegraded":"2024-04-07T16:12:14.375990+0000","last_fullsized":"2024-04-07T16:12:14.375990+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T23:32:52.283410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,0],"acting":[7,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.475526+0000","last_change":"2024-04-07T16:12:13.396409+0000","last_active":"2024-04-07T16:12:14.475526+0000","last_peered":"2024-04-07T16:12:14.475526+0000","last_clean":"2024-04-07T16:12:14.475526+0000","last_became_active":"2024-04-07T16:12:13.396094+0000","last_became_peered":"2024-04-07T16:12:13.396094+0000","last_unstale":"2024-04-07T16:12:14.475526+0000","last_undegraded":"2024-04-07T16:12:14.475526+0000","last_fullsized":"2024-04-07T16:12:14.475526+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T18:54:36.072837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.474549+0000","last_change":"2024-04-07T16:12:13.401973+0000","last_active":"2024-04-07T16:12:14.474549+0000","last_peered":"2024-04-07T16:12:14.474549+0000","last_clean":"2024-04-07T16:12:14.474549+0000","last_became_active":"2024-04-07T16:12:13.401724+0000","last_became_peered":"2024-04-07T16:12:13.401724+0000","last_unstale":"2024-04-07T16:12:14.474549+0000","last_undegraded":"2024-04-07T16:12:14.474549+0000","last_fullsized":"2024-04-07T16:12:14.474549+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T21:26:29.208223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,0],"acting":[6,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-04-07T16:12:13.005867+0000","last_change":"2024-04-07T16:12:13.005867+0000","last_active":"2024-04-07T16:12:12.996208+0000","last_peered":"2024-04-07T16:12:12.996208+0000","last_clean":"2024-04-07T16:12:12.996208+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:13.005867+0000","last_undegraded":"2024-04-07T16:12:13.005867+0000","last_fullsized":"2024-04-07T16:12:13.005867+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T22:29:22.509564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,0],"acting":[4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.474547+0000","last_change":"2024-04-07T16:12:13.394760+0000","last_active":"2024-04-07T16:12:14.474547+0000","last_peered":"2024-04-07T16:12:14.474547+0000","last_clean":"2024-04-07T16:12:14.474547+0000","last_became_active":"2024-04-07T16:12:13.394402+0000","last_became_peered":"2024-04-07T16:12:13.394402+0000","last_unstale":"2024-04-07T16:12:14.474547+0000","last_undegraded":"2024-04-07T16:12:14.474547+0000","last_fullsized":"2024-04-07T16:12:14.474547+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T18:47:13.891972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-07T16:12:12.996208+0000","last_change":"2024-04-07T16:12:12.996208+0000","last_active":"2024-04-07T16:12:12.996208+0000","last_peered":"2024-04-07T16:12:12.996208+0000","last_clean":"2024-04-07T16:12:12.996208+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:12.996208+0000","last_undegraded":"2024-04-07T16:12:12.996208+0000","last_fullsized":"2024-04-07T16:12:12.996208+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-07T16:12:12.996208+0000","last_change":"2024-04-07T16:12:12.996208+0000","last_active":"2024-04-07T16:12:12.996208+0000","last_peered":"2024-04-07T16:12:12.996208+0000","last_clean":"2024-04-07T16:12:12.996208+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:12.996208+0000","last_undegraded":"2024-04-07T16:12:12.996208+0000","last_fullsized":"2024-04-07T16:12:12.996208+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-07T16:12:12.996208+0000","last_change":"2024-04-07T16:12:12.996208+0000","last_active":"2024-04-07T16:12:12.996208+0000","last_peered":"2024-04-07T16:12:12.996208+0000","last_clean":"2024-04-07T16:12:12.996208+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:12.996208+0000","last_undegraded":"2024-04-07T16:12:12.996208+0000","last_fullsized":"2024-04-07T16:12:12.996208+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-07T16:12:12.996208+0000","last_change":"2024-04-07T16:12:12.996208+0000","last_active":"2024-04-07T16:12:12.996208+0000","last_peered":"2024-04-07T16:12:12.996208+0000","last_clean":"2024-04-07T16:12:12.996208+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:12.996208+0000","last_undegraded":"2024-04-07T16:12:12.996208+0000","last_fullsized":"2024-04-07T16:12:12.996208+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.474396+0000","last_change":"2024-04-07T16:12:13.416210+0000","last_active":"2024-04-07T16:12:14.474396+0000","last_peered":"2024-04-07T16:12:14.474396+0000","last_clean":"2024-04-07T16:12:14.474396+0000","last_became_active":"2024-04-07T16:12:13.415972+0000","last_became_peered":"2024-04-07T16:12:13.415972+0000","last_unstale":"2024-04-07T16:12:14.474396+0000","last_undegraded":"2024-04-07T16:12:14.474396+0000","last_fullsized":"2024-04-07T16:12:14.474396+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T23:40:11.606978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,7],"acting":[2,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.375192+0000","last_change":"2024-04-07T16:12:13.414203+0000","last_active":"2024-04-07T16:12:14.375192+0000","last_peered":"2024-04-07T16:12:14.375192+0000","last_clean":"2024-04-07T16:12:14.375192+0000","last_became_active":"2024-04-07T16:12:13.413869+0000","last_became_peered":"2024-04-07T16:12:13.413869+0000","last_unstale":"2024-04-07T16:12:14.375192+0000","last_undegraded":"2024-04-07T16:12:14.375192+0000","last_fullsized":"2024-04-07T16:12:14.375192+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T17:11:47.895568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,6],"acting":[7,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-04-07T16:12:12.000271+0000","last_change":"2024-04-07T16:12:12.000271+0000","last_active":"2024-04-07T16:12:11.994648+0000","last_peered":"2024-04-07T16:12:11.994648+0000","last_clean":"2024-04-07T16:12:11.994648+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:12.000271+0000","last_undegraded":"2024-04-07T16:12:12.000271+0000","last_fullsized":"2024-04-07T16:12:12.000271+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T19:01:44.798719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.474373+0000","last_change":"2024-04-07T16:12:13.407852+0000","last_active":"2024-04-07T16:12:14.474373+0000","last_peered":"2024-04-07T16:12:14.474373+0000","last_clean":"2024-04-07T16:12:14.474373+0000","last_became_active":"2024-04-07T16:12:13.407538+0000","last_became_peered":"2024-04-07T16:12:13.407538+0000","last_unstale":"2024-04-07T16:12:14.474373+0000","last_undegraded":"2024-04-07T16:12:14.474373+0000","last_fullsized":"2024-04-07T16:12:14.474373+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T20:27:52.664149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"61'1","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.474471+0000","last_change":"2024-04-07T16:12:13.038807+0000","last_active":"2024-04-07T16:12:14.474471+0000","last_peered":"2024-04-07T16:12:14.474471+0000","last_clean":"2024-04-07T16:12:14.474471+0000","last_became_active":"2024-04-07T16:12:13.038492+0000","last_became_peered":"2024-04-07T16:12:13.038492+0000","last_unstale":"2024-04-07T16:12:14.474471+0000","last_undegraded":"2024-04-07T16:12:14.474471+0000","last_fullsized":"2024-04-07T16:12:14.474471+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T02:11:03.746017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-07T16:12:12.996208+0000","last_change":"2024-04-07T16:12:12.996208+0000","last_active":"2024-04-07T16:12:12.996208+0000","last_peered":"2024-04-07T16:12:12.996208+0000","last_clean":"2024-04-07T16:12:12.996208+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:12.996208+0000","last_undegraded":"2024-04-07T16:12:12.996208+0000","last_fullsized":"2024-04-07T16:12:12.996208+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"61'2","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.375664+0000","last_change":"2024-04-07T16:12:13.042401+0000","last_active":"2024-04-07T16:12:14.375664+0000","last_peered":"2024-04-07T16:12:14.375664+0000","last_clean":"2024-04-07T16:12:14.375664+0000","last_became_active":"2024-04-07T16:12:13.041970+0000","last_became_peered":"2024-04-07T16:12:13.041970+0000","last_unstale":"2024-04-07T16:12:14.375664+0000","last_undegraded":"2024-04-07T16:12:14.375664+0000","last_fullsized":"2024-04-07T16:12:14.375664+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T16:36:29.235347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-07T16:12:12.996208+0000","last_change":"2024-04-07T16:12:12.996208+0000","last_active":"2024-04-07T16:12:12.996208+0000","last_peered":"2024-04-07T16:12:12.996208+0000","last_clean":"2024-04-07T16:12:12.996208+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:12.996208+0000","last_undegraded":"2024-04-07T16:12:12.996208+0000","last_fullsized":"2024-04-07T16:12:12.996208+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"61'2","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.375447+0000","last_change":"2024-04-07T16:12:13.042391+0000","last_active":"2024-04-07T16:12:14.375447+0000","last_peered":"2024-04-07T16:12:14.375447+0000","last_clean":"2024-04-07T16:12:14.375447+0000","last_became_active":"2024-04-07T16:12:13.041919+0000","last_became_peered":"2024-04-07T16:12:13.041919+0000","last_unstale":"2024-04-07T16:12:14.375447+0000","last_undegraded":"2024-04-07T16:12:14.375447+0000","last_fullsized":"2024-04-07T16:12:14.375447+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T21:50:59.576615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-07T16:12:12.996208+0000","last_change":"2024-04-07T16:12:12.996208+0000","last_active":"2024-04-07T16:12:12.996208+0000","last_peered":"2024-04-07T16:12:12.996208+0000","last_clean":"2024-04-07T16:12:12.996208+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:12.996208+0000","last_undegraded":"2024-04-07T16:12:12.996208+0000","last_fullsized":"2024-04-07T16:12:12.996208+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"61'1","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.474818+0000","last_change":"2024-04-07T16:12:13.041880+0000","last_active":"2024-04-07T16:12:14.474818+0000","last_peered":"2024-04-07T16:12:14.474818+0000","last_clean":"2024-04-07T16:12:14.474818+0000","last_became_active":"2024-04-07T16:12:13.041409+0000","last_became_peered":"2024-04-07T16:12:13.041409+0000","last_unstale":"2024-04-07T16:12:14.474818+0000","last_undegraded":"2024-04-07T16:12:14.474818+0000","last_fullsized":"2024-04-07T16:12:14.474818+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T18:32:23.206985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-04-07T16:12:13.002508+0000","last_change":"2024-04-07T16:12:13.002508+0000","last_active":"2024-04-07T16:12:12.996208+0000","last_peered":"2024-04-07T16:12:12.996208+0000","last_clean":"2024-04-07T16:12:12.996208+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:13.002508+0000","last_undegraded":"2024-04-07T16:12:13.002508+0000","last_fullsized":"2024-04-07T16:12:13.002508+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T23:20:40.949255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"61'1","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.476584+0000","last_change":"2024-04-07T16:12:13.038960+0000","last_active":"2024-04-07T16:12:14.476584+0000","last_peered":"2024-04-07T16:12:14.476584+0000","last_clean":"2024-04-07T16:12:14.476584+0000","last_became_active":"2024-04-07T16:12:13.038410+0000","last_became_peered":"2024-04-07T16:12:13.038410+0000","last_unstale":"2024-04-07T16:12:14.476584+0000","last_undegraded":"2024-04-07T16:12:14.476584+0000","last_fullsized":"2024-04-07T16:12:14.476584+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T03:41:38.418312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-07T16:12:12.996208+0000","last_change":"2024-04-07T16:12:12.996208+0000","last_active":"2024-04-07T16:12:12.996208+0000","last_peered":"2024-04-07T16:12:12.996208+0000","last_clean":"2024-04-07T16:12:12.996208+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:12.996208+0000","last_undegraded":"2024-04-07T16:12:12.996208+0000","last_fullsized":"2024-04-07T16:12:12.996208+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.475758+0000","last_change":"2024-04-07T16:12:13.034420+0000","last_active":"2024-04-07T16:12:14.475758+0000","last_peered":"2024-04-07T16:12:14.475758+0000","last_clean":"2024-04-07T16:12:14.475758+0000","last_became_active":"2024-04-07T16:12:13.034013+0000","last_became_peered":"2024-04-07T16:12:13.034013+0000","last_unstale":"2024-04-07T16:12:14.475758+0000","last_undegraded":"2024-04-07T16:12:14.475758+0000","last_fullsized":"2024-04-07T16:12:14.475758+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T19:51:21.525865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.474201+0000","last_change":"2024-04-07T16:12:13.398531+0000","last_active":"2024-04-07T16:12:14.474201+0000","last_peered":"2024-04-07T16:12:14.474201+0000","last_clean":"2024-04-07T16:12:14.474201+0000","last_became_active":"2024-04-07T16:12:13.398017+0000","last_became_peered":"2024-04-07T16:12:13.398017+0000","last_unstale":"2024-04-07T16:12:14.474201+0000","last_undegraded":"2024-04-07T16:12:14.474201+0000","last_fullsized":"2024-04-07T16:12:14.474201+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T17:33:54.306902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-04-07T16:12:12.000806+0000","last_change":"2024-04-07T16:12:12.000806+0000","last_active":"2024-04-07T16:12:11.994648+0000","last_peered":"2024-04-07T16:12:11.994648+0000","last_clean":"2024-04-07T16:12:11.994648+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:12.000806+0000","last_undegraded":"2024-04-07T16:12:12.000806+0000","last_fullsized":"2024-04-07T16:12:12.000806+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T22:44:14.623116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-07T16:12:12.996208+0000","last_change":"2024-04-07T16:12:12.996208+0000","last_active":"2024-04-07T16:12:12.996208+0000","last_peered":"2024-04-07T16:12:12.996208+0000","last_clean":"2024-04-07T16:12:12.996208+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:12.996208+0000","last_undegraded":"2024-04-07T16:12:12.996208+0000","last_fullsized":"2024-04-07T16:12:12.996208+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.474717+0000","last_change":"2024-04-07T16:12:13.030149+0000","last_active":"2024-04-07T16:12:14.474717+0000","last_peered":"2024-04-07T16:12:14.474717+0000","last_clean":"2024-04-07T16:12:14.474717+0000","last_became_active":"2024-04-07T16:12:13.029525+0000","last_became_peered":"2024-04-07T16:12:13.029525+0000","last_unstale":"2024-04-07T16:12:14.474717+0000","last_undegraded":"2024-04-07T16:12:14.474717+0000","last_fullsized":"2024-04-07T16:12:14.474717+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T20:09:35.180264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-07T16:12:12.996208+0000","last_change":"2024-04-07T16:12:12.996208+0000","last_active":"2024-04-07T16:12:12.996208+0000","last_peered":"2024-04-07T16:12:12.996208+0000","last_clean":"2024-04-07T16:12:12.996208+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:12.996208+0000","last_undegraded":"2024-04-07T16:12:12.996208+0000","last_fullsized":"2024-04-07T16:12:12.996208+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-04-07T16:12:12.001015+0000","last_change":"2024-04-07T16:12:12.001015+0000","last_active":"2024-04-07T16:12:11.994648+0000","last_peered":"2024-04-07T16:12:11.994648+0000","last_clean":"2024-04-07T16:12:11.994648+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:12.001015+0000","last_undegraded":"2024-04-07T16:12:12.001015+0000","last_fullsized":"2024-04-07T16:12:12.001015+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T18:26:16.873044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-07T16:12:12.996208+0000","last_change":"2024-04-07T16:12:12.996208+0000","last_active":"2024-04-07T16:12:12.996208+0000","last_peered":"2024-04-07T16:12:12.996208+0000","last_clean":"2024-04-07T16:12:12.996208+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:12.996208+0000","last_undegraded":"2024-04-07T16:12:12.996208+0000","last_fullsized":"2024-04-07T16:12:12.996208+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-07T16:12:11.994648+0000","last_change":"2024-04-07T16:12:11.994648+0000","last_active":"2024-04-07T16:12:11.994648+0000","last_peered":"2024-04-07T16:12:11.994648+0000","last_clean":"2024-04-07T16:12:11.994648+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:11.994648+0000","last_undegraded":"2024-04-07T16:12:11.994648+0000","last_fullsized":"2024-04-07T16:12:11.994648+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-04-07T16:12:13.004673+0000","last_change":"2024-04-07T16:12:13.004673+0000","last_active":"2024-04-07T16:12:12.996208+0000","last_peered":"2024-04-07T16:12:12.996208+0000","last_clean":"2024-04-07T16:12:12.996208+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:13.004673+0000","last_undegraded":"2024-04-07T16:12:13.004673+0000","last_fullsized":"2024-04-07T16:12:13.004673+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T16:55:37.429799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"61'6","reported_seq":18,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.473934+0000","last_change":"2024-04-07T16:12:13.031673+0000","last_active":"2024-04-07T16:12:14.473934+0000","last_peered":"2024-04-07T16:12:14.473934+0000","last_clean":"2024-04-07T16:12:14.473934+0000","last_became_active":"2024-04-07T16:12:13.031092+0000","last_became_peered":"2024-04-07T16:12:13.031092+0000","last_unstale":"2024-04-07T16:12:14.473934+0000","last_undegraded":"2024-04-07T16:12:14.473934+0000","last_fullsized":"2024-04-07T16:12:14.473934+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T20:17:48.593862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.375355+0000","last_change":"2024-04-07T16:12:13.390083+0000","last_active":"2024-04-07T16:12:14.375355+0000","last_peered":"2024-04-07T16:12:14.375355+0000","last_clean":"2024-04-07T16:12:14.375355+0000","last_became_active":"2024-04-07T16:12:13.389755+0000","last_became_peered":"2024-04-07T16:12:13.389755+0000","last_unstale":"2024-04-07T16:12:14.375355+0000","last_undegraded":"2024-04-07T16:12:14.375355+0000","last_fullsized":"2024-04-07T16:12:14.375355+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T02:31:09.155569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-04-07T16:12:12.001190+0000","last_change":"2024-04-07T16:12:12.001190+0000","last_active":"2024-04-07T16:12:11.994648+0000","last_peered":"2024-04-07T16:12:11.994648+0000","last_clean":"2024-04-07T16:12:11.994648+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:12.001190+0000","last_undegraded":"2024-04-07T16:12:12.001190+0000","last_fullsized":"2024-04-07T16:12:12.001190+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T01:50:02.195507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-07T16:12:12.996208+0000","last_change":"2024-04-07T16:12:12.996208+0000","last_active":"2024-04-07T16:12:12.996208+0000","last_peered":"2024-04-07T16:12:12.996208+0000","last_clean":"2024-04-07T16:12:12.996208+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:12.996208+0000","last_undegraded":"2024-04-07T16:12:12.996208+0000","last_fullsized":"2024-04-07T16:12:12.996208+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-04-07T16:12:12.002099+0000","last_change":"2024-04-07T16:12:12.002099+0000","last_active":"2024-04-07T16:12:11.994648+0000","last_peered":"2024-04-07T16:12:11.994648+0000","last_clean":"2024-04-07T16:12:11.994648+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:12.002099+0000","last_undegraded":"2024-04-07T16:12:12.002099+0000","last_fullsized":"2024-04-07T16:12:12.002099+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T23:15:11.494437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.375013+0000","last_change":"2024-04-07T16:12:13.413928+0000","last_active":"2024-04-07T16:12:14.375013+0000","last_peered":"2024-04-07T16:12:14.375013+0000","last_clean":"2024-04-07T16:12:14.375013+0000","last_became_active":"2024-04-07T16:12:13.413447+0000","last_became_peered":"2024-04-07T16:12:13.413447+0000","last_unstale":"2024-04-07T16:12:14.375013+0000","last_undegraded":"2024-04-07T16:12:14.375013+0000","last_fullsized":"2024-04-07T16:12:14.375013+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T03:26:42.875228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.473099+0000","last_change":"2024-04-07T16:12:13.032942+0000","last_active":"2024-04-07T16:12:14.473099+0000","last_peered":"2024-04-07T16:12:14.473099+0000","last_clean":"2024-04-07T16:12:14.473099+0000","last_became_active":"2024-04-07T16:12:13.032108+0000","last_became_peered":"2024-04-07T16:12:13.032108+0000","last_unstale":"2024-04-07T16:12:14.473099+0000","last_undegraded":"2024-04-07T16:12:14.473099+0000","last_fullsized":"2024-04-07T16:12:14.473099+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T18:53:06.864412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.473331+0000","last_change":"2024-04-07T16:12:13.400141+0000","last_active":"2024-04-07T16:12:14.473331+0000","last_peered":"2024-04-07T16:12:14.473331+0000","last_clean":"2024-04-07T16:12:14.473331+0000","last_became_active":"2024-04-07T16:12:13.399823+0000","last_became_peered":"2024-04-07T16:12:13.399823+0000","last_unstale":"2024-04-07T16:12:14.473331+0000","last_undegraded":"2024-04-07T16:12:14.473331+0000","last_fullsized":"2024-04-07T16:12:14.473331+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T21:27:25.248595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"61'8","reported_seq":20,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.474350+0000","last_change":"2024-04-07T16:12:13.034154+0000","last_active":"2024-04-07T16:12:14.474350+0000","last_peered":"2024-04-07T16:12:14.474350+0000","last_clean":"2024-04-07T16:12:14.474350+0000","last_became_active":"2024-04-07T16:12:13.033838+0000","last_became_peered":"2024-04-07T16:12:13.033838+0000","last_unstale":"2024-04-07T16:12:14.474350+0000","last_undegraded":"2024-04-07T16:12:14.474350+0000","last_fullsized":"2024-04-07T16:12:14.474350+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T23:13:07.216314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.374716+0000","last_change":"2024-04-07T16:12:13.385883+0000","last_active":"2024-04-07T16:12:14.374716+0000","last_peered":"2024-04-07T16:12:14.374716+0000","last_clean":"2024-04-07T16:12:14.374716+0000","last_became_active":"2024-04-07T16:12:13.385485+0000","last_became_peered":"2024-04-07T16:12:13.385485+0000","last_unstale":"2024-04-07T16:12:14.374716+0000","last_undegraded":"2024-04-07T16:12:14.374716+0000","last_fullsized":"2024-04-07T16:12:14.374716+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T21:01:09.262433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":4,"reported_epoch":60,"state":"creating+activating","last_fresh":"2024-04-07T16:12:13.024777+0000","last_change":"2024-04-07T16:12:13.002417+0000","last_active":"2024-04-07T16:12:11.994648+0000","last_peered":"2024-04-07T16:12:11.994648+0000","last_clean":"2024-04-07T16:12:11.994648+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:13.024777+0000","last_undegraded":"2024-04-07T16:12:13.024777+0000","last_fullsized":"2024-04-07T16:12:13.024777+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T17:55:06.727142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":["4","0","7"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.375770+0000","last_change":"2024-04-07T16:12:13.386584+0000","last_active":"2024-04-07T16:12:14.375770+0000","last_peered":"2024-04-07T16:12:14.375770+0000","last_clean":"2024-04-07T16:12:14.375770+0000","last_became_active":"2024-04-07T16:12:13.386057+0000","last_became_peered":"2024-04-07T16:12:13.386057+0000","last_unstale":"2024-04-07T16:12:14.375770+0000","last_undegraded":"2024-04-07T16:12:14.375770+0000","last_fullsized":"2024-04-07T16:12:14.375770+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T17:57:57.548179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-07T16:12:11.994648+0000","last_change":"2024-04-07T16:12:11.994648+0000","last_active":"2024-04-07T16:12:11.994648+0000","last_peered":"2024-04-07T16:12:11.994648+0000","last_clean":"2024-04-07T16:12:11.994648+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:11.994648+0000","last_undegraded":"2024-04-07T16:12:11.994648+0000","last_fullsized":"2024-04-07T16:12:11.994648+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-07T16:12:12.996208+0000","last_change":"2024-04-07T16:12:12.996208+0000","last_active":"2024-04-07T16:12:12.996208+0000","last_peered":"2024-04-07T16:12:12.996208+0000","last_clean":"2024-04-07T16:12:12.996208+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:12.996208+0000","last_undegraded":"2024-04-07T16:12:12.996208+0000","last_fullsized":"2024-04-07T16:12:12.996208+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-04-07T16:12:12.001190+0000","last_change":"2024-04-07T16:12:12.001190+0000","last_active":"2024-04-07T16:12:11.994648+0000","last_peered":"2024-04-07T16:12:11.994648+0000","last_clean":"2024-04-07T16:12:11.994648+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:12.001190+0000","last_undegraded":"2024-04-07T16:12:12.001190+0000","last_fullsized":"2024-04-07T16:12:12.001190+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T18:15:00.506827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-04-07T16:12:13.003585+0000","last_change":"2024-04-07T16:12:13.003585+0000","last_active":"2024-04-07T16:12:12.996208+0000","last_peered":"2024-04-07T16:12:12.996208+0000","last_clean":"2024-04-07T16:12:12.996208+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:13.003585+0000","last_undegraded":"2024-04-07T16:12:13.003585+0000","last_fullsized":"2024-04-07T16:12:13.003585+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T03:36:24.354110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-07T16:12:12.996208+0000","last_change":"2024-04-07T16:12:12.996208+0000","last_active":"2024-04-07T16:12:12.996208+0000","last_peered":"2024-04-07T16:12:12.996208+0000","last_clean":"2024-04-07T16:12:12.996208+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:12.996208+0000","last_undegraded":"2024-04-07T16:12:12.996208+0000","last_fullsized":"2024-04-07T16:12:12.996208+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.474169+0000","last_change":"2024-04-07T16:12:13.035071+0000","last_active":"2024-04-07T16:12:14.474169+0000","last_peered":"2024-04-07T16:12:14.474169+0000","last_clean":"2024-04-07T16:12:14.474169+0000","last_became_active":"2024-04-07T16:12:13.034795+0000","last_became_peered":"2024-04-07T16:12:13.034795+0000","last_unstale":"2024-04-07T16:12:14.474169+0000","last_undegraded":"2024-04-07T16:12:14.474169+0000","last_fullsized":"2024-04-07T16:12:14.474169+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T17:51:30.599527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.476063+0000","last_change":"2024-04-07T16:12:13.394635+0000","last_active":"2024-04-07T16:12:14.476063+0000","last_peered":"2024-04-07T16:12:14.476063+0000","last_clean":"2024-04-07T16:12:14.476063+0000","last_became_active":"2024-04-07T16:12:13.394392+0000","last_became_peered":"2024-04-07T16:12:13.394392+0000","last_unstale":"2024-04-07T16:12:14.476063+0000","last_undegraded":"2024-04-07T16:12:14.476063+0000","last_fullsized":"2024-04-07T16:12:14.476063+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T17:52:10.950481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-04-07T16:12:12.001243+0000","last_change":"2024-04-07T16:12:12.001243+0000","last_active":"2024-04-07T16:12:11.994648+0000","last_peered":"2024-04-07T16:12:11.994648+0000","last_clean":"2024-04-07T16:12:11.994648+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:12.001243+0000","last_undegraded":"2024-04-07T16:12:12.001243+0000","last_fullsized":"2024-04-07T16:12:12.001243+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T02:48:50.652491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"21'117","reported_seq":219,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-07T16:12:13.375677+0000","last_change":"2024-04-07T16:11:33.518868+0000","last_active":"2024-04-07T16:12:13.375677+0000","last_peered":"2024-04-07T16:12:13.375677+0000","last_clean":"2024-04-07T16:12:13.375677+0000","last_became_active":"2024-04-07T16:11:33.518060+0000","last_became_peered":"2024-04-07T16:11:33.518060+0000","last_unstale":"2024-04-07T16:12:13.375677+0000","last_undegraded":"2024-04-07T16:12:13.375677+0000","last_fullsized":"2024-04-07T16:12:13.375677+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:07:26.866003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:07:26.866003+0000","last_clean_scrub_stamp":"2024-04-07T16:07:26.866003+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T22:22:51.043941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-07T16:12:12.996208+0000","last_change":"2024-04-07T16:12:12.996208+0000","last_active":"2024-04-07T16:12:12.996208+0000","last_peered":"2024-04-07T16:12:12.996208+0000","last_clean":"2024-04-07T16:12:12.996208+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:12.996208+0000","last_undegraded":"2024-04-07T16:12:12.996208+0000","last_fullsized":"2024-04-07T16:12:12.996208+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.375081+0000","last_change":"2024-04-07T16:12:13.035536+0000","last_active":"2024-04-07T16:12:14.375081+0000","last_peered":"2024-04-07T16:12:14.375081+0000","last_clean":"2024-04-07T16:12:14.375081+0000","last_became_active":"2024-04-07T16:12:13.034825+0000","last_became_peered":"2024-04-07T16:12:13.034825+0000","last_unstale":"2024-04-07T16:12:14.375081+0000","last_undegraded":"2024-04-07T16:12:14.375081+0000","last_fullsized":"2024-04-07T16:12:14.375081+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T20:11:57.033306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.375586+0000","last_change":"2024-04-07T16:12:13.035747+0000","last_active":"2024-04-07T16:12:14.375586+0000","last_peered":"2024-04-07T16:12:14.375586+0000","last_clean":"2024-04-07T16:12:14.375586+0000","last_became_active":"2024-04-07T16:12:13.035201+0000","last_became_peered":"2024-04-07T16:12:13.035201+0000","last_unstale":"2024-04-07T16:12:14.375586+0000","last_undegraded":"2024-04-07T16:12:14.375586+0000","last_fullsized":"2024-04-07T16:12:14.375586+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T21:03:25.449728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-07T16:12:12.996208+0000","last_change":"2024-04-07T16:12:12.996208+0000","last_active":"2024-04-07T16:12:12.996208+0000","last_peered":"2024-04-07T16:12:12.996208+0000","last_clean":"2024-04-07T16:12:12.996208+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:12.996208+0000","last_undegraded":"2024-04-07T16:12:12.996208+0000","last_fullsized":"2024-04-07T16:12:12.996208+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-07T16:12:11.994648+0000","last_change":"2024-04-07T16:12:11.994648+0000","last_active":"2024-04-07T16:12:11.994648+0000","last_peered":"2024-04-07T16:12:11.994648+0000","last_clean":"2024-04-07T16:12:11.994648+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:11.994648+0000","last_undegraded":"2024-04-07T16:12:11.994648+0000","last_fullsized":"2024-04-07T16:12:11.994648+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.474648+0000","last_change":"2024-04-07T16:12:13.399486+0000","last_active":"2024-04-07T16:12:14.474648+0000","last_peered":"2024-04-07T16:12:14.474648+0000","last_clean":"2024-04-07T16:12:14.474648+0000","last_became_active":"2024-04-07T16:12:13.399184+0000","last_became_peered":"2024-04-07T16:12:13.399184+0000","last_unstale":"2024-04-07T16:12:14.474648+0000","last_undegraded":"2024-04-07T16:12:14.474648+0000","last_fullsized":"2024-04-07T16:12:14.474648+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T21:07:25.104309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"61'2","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.474473+0000","last_change":"2024-04-07T16:12:13.038351+0000","last_active":"2024-04-07T16:12:14.474473+0000","last_peered":"2024-04-07T16:12:14.474473+0000","last_clean":"2024-04-07T16:12:14.474473+0000","last_became_active":"2024-04-07T16:12:13.038026+0000","last_became_peered":"2024-04-07T16:12:13.038026+0000","last_unstale":"2024-04-07T16:12:14.474473+0000","last_undegraded":"2024-04-07T16:12:14.474473+0000","last_fullsized":"2024-04-07T16:12:14.474473+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T19:39:42.796342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-07T16:12:12.996208+0000","last_change":"2024-04-07T16:12:12.996208+0000","last_active":"2024-04-07T16:12:12.996208+0000","last_peered":"2024-04-07T16:12:12.996208+0000","last_clean":"2024-04-07T16:12:12.996208+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:12.996208+0000","last_undegraded":"2024-04-07T16:12:12.996208+0000","last_fullsized":"2024-04-07T16:12:12.996208+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.374594+0000","last_change":"2024-04-07T16:12:13.033067+0000","last_active":"2024-04-07T16:12:14.374594+0000","last_peered":"2024-04-07T16:12:14.374594+0000","last_clean":"2024-04-07T16:12:14.374594+0000","last_became_active":"2024-04-07T16:12:13.032456+0000","last_became_peered":"2024-04-07T16:12:13.032456+0000","last_unstale":"2024-04-07T16:12:14.374594+0000","last_undegraded":"2024-04-07T16:12:14.374594+0000","last_fullsized":"2024-04-07T16:12:14.374594+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T23:02:09.025617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-04-07T16:12:13.001861+0000","last_change":"2024-04-07T16:12:13.001861+0000","last_active":"2024-04-07T16:12:12.996208+0000","last_peered":"2024-04-07T16:12:12.996208+0000","last_clean":"2024-04-07T16:12:12.996208+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:13.001861+0000","last_undegraded":"2024-04-07T16:12:13.001861+0000","last_fullsized":"2024-04-07T16:12:13.001861+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T23:48:08.846232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-07T16:12:11.994648+0000","last_change":"2024-04-07T16:12:11.994648+0000","last_active":"2024-04-07T16:12:11.994648+0000","last_peered":"2024-04-07T16:12:11.994648+0000","last_clean":"2024-04-07T16:12:11.994648+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:11.994648+0000","last_undegraded":"2024-04-07T16:12:11.994648+0000","last_fullsized":"2024-04-07T16:12:11.994648+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.474100+0000","last_change":"2024-04-07T16:12:13.392396+0000","last_active":"2024-04-07T16:12:14.474100+0000","last_peered":"2024-04-07T16:12:14.474100+0000","last_clean":"2024-04-07T16:12:14.474100+0000","last_became_active":"2024-04-07T16:12:13.392016+0000","last_became_peered":"2024-04-07T16:12:13.392016+0000","last_unstale":"2024-04-07T16:12:14.474100+0000","last_undegraded":"2024-04-07T16:12:14.474100+0000","last_fullsized":"2024-04-07T16:12:14.474100+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T02:35:50.790051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-07T16:12:11.994648+0000","last_change":"2024-04-07T16:12:11.994648+0000","last_active":"2024-04-07T16:12:11.994648+0000","last_peered":"2024-04-07T16:12:11.994648+0000","last_clean":"2024-04-07T16:12:11.994648+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:11.994648+0000","last_undegraded":"2024-04-07T16:12:11.994648+0000","last_fullsized":"2024-04-07T16:12:11.994648+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.474697+0000","last_change":"2024-04-07T16:12:13.397819+0000","last_active":"2024-04-07T16:12:14.474697+0000","last_peered":"2024-04-07T16:12:14.474697+0000","last_clean":"2024-04-07T16:12:14.474697+0000","last_became_active":"2024-04-07T16:12:13.397593+0000","last_became_peered":"2024-04-07T16:12:13.397593+0000","last_unstale":"2024-04-07T16:12:14.474697+0000","last_undegraded":"2024-04-07T16:12:14.474697+0000","last_fullsized":"2024-04-07T16:12:14.474697+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T02:30:01.472030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"61'2","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.375041+0000","last_change":"2024-04-07T16:12:13.043982+0000","last_active":"2024-04-07T16:12:14.375041+0000","last_peered":"2024-04-07T16:12:14.375041+0000","last_clean":"2024-04-07T16:12:14.375041+0000","last_became_active":"2024-04-07T16:12:13.041873+0000","last_became_peered":"2024-04-07T16:12:13.041873+0000","last_unstale":"2024-04-07T16:12:14.375041+0000","last_undegraded":"2024-04-07T16:12:14.375041+0000","last_fullsized":"2024-04-07T16:12:14.375041+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T20:03:35.575133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.476449+0000","last_change":"2024-04-07T16:12:13.381810+0000","last_active":"2024-04-07T16:12:14.476449+0000","last_peered":"2024-04-07T16:12:14.476449+0000","last_clean":"2024-04-07T16:12:14.476449+0000","last_became_active":"2024-04-07T16:12:13.381470+0000","last_became_peered":"2024-04-07T16:12:13.381470+0000","last_unstale":"2024-04-07T16:12:14.476449+0000","last_undegraded":"2024-04-07T16:12:14.476449+0000","last_fullsized":"2024-04-07T16:12:14.476449+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T17:26:21.450939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.474605+0000","last_change":"2024-04-07T16:12:13.038191+0000","last_active":"2024-04-07T16:12:14.474605+0000","last_peered":"2024-04-07T16:12:14.474605+0000","last_clean":"2024-04-07T16:12:14.474605+0000","last_became_active":"2024-04-07T16:12:13.037652+0000","last_became_peered":"2024-04-07T16:12:13.037652+0000","last_unstale":"2024-04-07T16:12:14.474605+0000","last_undegraded":"2024-04-07T16:12:14.474605+0000","last_fullsized":"2024-04-07T16:12:14.474605+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T17:04:59.525025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.375473+0000","last_change":"2024-04-07T16:12:13.400660+0000","last_active":"2024-04-07T16:12:14.375473+0000","last_peered":"2024-04-07T16:12:14.375473+0000","last_clean":"2024-04-07T16:12:14.375473+0000","last_became_active":"2024-04-07T16:12:13.400353+0000","last_became_peered":"2024-04-07T16:12:13.400353+0000","last_unstale":"2024-04-07T16:12:14.375473+0000","last_undegraded":"2024-04-07T16:12:14.375473+0000","last_fullsized":"2024-04-07T16:12:14.375473+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T00:47:57.573097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-07T16:12:11.994648+0000","last_change":"2024-04-07T16:12:11.994648+0000","last_active":"2024-04-07T16:12:11.994648+0000","last_peered":"2024-04-07T16:12:11.994648+0000","last_clean":"2024-04-07T16:12:11.994648+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:11.994648+0000","last_undegraded":"2024-04-07T16:12:11.994648+0000","last_fullsized":"2024-04-07T16:12:11.994648+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-07T16:12:12.996208+0000","last_change":"2024-04-07T16:12:12.996208+0000","last_active":"2024-04-07T16:12:12.996208+0000","last_peered":"2024-04-07T16:12:12.996208+0000","last_clean":"2024-04-07T16:12:12.996208+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:12.996208+0000","last_undegraded":"2024-04-07T16:12:12.996208+0000","last_fullsized":"2024-04-07T16:12:12.996208+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"61'2","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.474501+0000","last_change":"2024-04-07T16:12:13.040101+0000","last_active":"2024-04-07T16:12:14.474501+0000","last_peered":"2024-04-07T16:12:14.474501+0000","last_clean":"2024-04-07T16:12:14.474501+0000","last_became_active":"2024-04-07T16:12:13.039797+0000","last_became_peered":"2024-04-07T16:12:13.039797+0000","last_unstale":"2024-04-07T16:12:14.474501+0000","last_undegraded":"2024-04-07T16:12:14.474501+0000","last_fullsized":"2024-04-07T16:12:14.474501+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T03:27:06.016325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.376139+0000","last_change":"2024-04-07T16:12:13.394720+0000","last_active":"2024-04-07T16:12:14.376139+0000","last_peered":"2024-04-07T16:12:14.376139+0000","last_clean":"2024-04-07T16:12:14.376139+0000","last_became_active":"2024-04-07T16:12:13.394090+0000","last_became_peered":"2024-04-07T16:12:13.394090+0000","last_unstale":"2024-04-07T16:12:14.376139+0000","last_undegraded":"2024-04-07T16:12:14.376139+0000","last_fullsized":"2024-04-07T16:12:14.376139+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T00:47:54.579168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.473133+0000","last_change":"2024-04-07T16:12:13.396725+0000","last_active":"2024-04-07T16:12:14.473133+0000","last_peered":"2024-04-07T16:12:14.473133+0000","last_clean":"2024-04-07T16:12:14.473133+0000","last_became_active":"2024-04-07T16:12:13.395978+0000","last_became_peered":"2024-04-07T16:12:13.395978+0000","last_unstale":"2024-04-07T16:12:14.473133+0000","last_undegraded":"2024-04-07T16:12:14.473133+0000","last_fullsized":"2024-04-07T16:12:14.473133+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T21:30:43.276569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.474831+0000","last_change":"2024-04-07T16:12:13.412091+0000","last_active":"2024-04-07T16:12:14.474831+0000","last_peered":"2024-04-07T16:12:14.474831+0000","last_clean":"2024-04-07T16:12:14.474831+0000","last_became_active":"2024-04-07T16:12:13.410814+0000","last_became_peered":"2024-04-07T16:12:13.410814+0000","last_unstale":"2024-04-07T16:12:14.474831+0000","last_undegraded":"2024-04-07T16:12:14.474831+0000","last_fullsized":"2024-04-07T16:12:14.474831+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T01:48:29.034291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-07T16:12:12.996208+0000","last_change":"2024-04-07T16:12:12.996208+0000","last_active":"2024-04-07T16:12:12.996208+0000","last_peered":"2024-04-07T16:12:12.996208+0000","last_clean":"2024-04-07T16:12:12.996208+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:12.996208+0000","last_undegraded":"2024-04-07T16:12:12.996208+0000","last_fullsized":"2024-04-07T16:12:12.996208+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-07T16:12:12.996208+0000","last_change":"2024-04-07T16:12:12.996208+0000","last_active":"2024-04-07T16:12:12.996208+0000","last_peered":"2024-04-07T16:12:12.996208+0000","last_clean":"2024-04-07T16:12:12.996208+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:12.996208+0000","last_undegraded":"2024-04-07T16:12:12.996208+0000","last_fullsized":"2024-04-07T16:12:12.996208+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.475244+0000","last_change":"2024-04-07T16:12:13.398730+0000","last_active":"2024-04-07T16:12:14.475244+0000","last_peered":"2024-04-07T16:12:14.475244+0000","last_clean":"2024-04-07T16:12:14.475244+0000","last_became_active":"2024-04-07T16:12:13.398259+0000","last_became_peered":"2024-04-07T16:12:13.398259+0000","last_unstale":"2024-04-07T16:12:14.475244+0000","last_undegraded":"2024-04-07T16:12:14.475244+0000","last_fullsized":"2024-04-07T16:12:14.475244+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T02:14:28.915631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,0],"acting":[3,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-07T16:12:12.996208+0000","last_change":"2024-04-07T16:12:12.996208+0000","last_active":"2024-04-07T16:12:12.996208+0000","last_peered":"2024-04-07T16:12:12.996208+0000","last_clean":"2024-04-07T16:12:12.996208+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:12.996208+0000","last_undegraded":"2024-04-07T16:12:12.996208+0000","last_fullsized":"2024-04-07T16:12:12.996208+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.475770+0000","last_change":"2024-04-07T16:12:13.382605+0000","last_active":"2024-04-07T16:12:14.475770+0000","last_peered":"2024-04-07T16:12:14.475770+0000","last_clean":"2024-04-07T16:12:14.475770+0000","last_became_active":"2024-04-07T16:12:13.382267+0000","last_became_peered":"2024-04-07T16:12:13.382267+0000","last_unstale":"2024-04-07T16:12:14.475770+0000","last_undegraded":"2024-04-07T16:12:14.475770+0000","last_fullsized":"2024-04-07T16:12:14.475770+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T20:14:03.942753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,4],"acting":[6,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.474734+0000","last_change":"2024-04-07T16:12:13.405175+0000","last_active":"2024-04-07T16:12:14.474734+0000","last_peered":"2024-04-07T16:12:14.474734+0000","last_clean":"2024-04-07T16:12:14.474734+0000","last_became_active":"2024-04-07T16:12:13.404734+0000","last_became_peered":"2024-04-07T16:12:13.404734+0000","last_unstale":"2024-04-07T16:12:14.474734+0000","last_undegraded":"2024-04-07T16:12:14.474734+0000","last_fullsized":"2024-04-07T16:12:14.474734+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T17:55:23.469795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-07T16:12:12.996208+0000","last_change":"2024-04-07T16:12:12.996208+0000","last_active":"2024-04-07T16:12:12.996208+0000","last_peered":"2024-04-07T16:12:12.996208+0000","last_clean":"2024-04-07T16:12:12.996208+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:12.996208+0000","last_undegraded":"2024-04-07T16:12:12.996208+0000","last_fullsized":"2024-04-07T16:12:12.996208+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.475578+0000","last_change":"2024-04-07T16:12:13.397638+0000","last_active":"2024-04-07T16:12:14.475578+0000","last_peered":"2024-04-07T16:12:14.475578+0000","last_clean":"2024-04-07T16:12:14.475578+0000","last_became_active":"2024-04-07T16:12:13.396766+0000","last_became_peered":"2024-04-07T16:12:13.396766+0000","last_unstale":"2024-04-07T16:12:14.475578+0000","last_undegraded":"2024-04-07T16:12:14.475578+0000","last_fullsized":"2024-04-07T16:12:14.475578+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T22:51:06.841718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-07T16:12:12.996208+0000","last_change":"2024-04-07T16:12:12.996208+0000","last_active":"2024-04-07T16:12:12.996208+0000","last_peered":"2024-04-07T16:12:12.996208+0000","last_clean":"2024-04-07T16:12:12.996208+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:12.996208+0000","last_undegraded":"2024-04-07T16:12:12.996208+0000","last_fullsized":"2024-04-07T16:12:12.996208+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-07T16:12:12.996208+0000","last_change":"2024-04-07T16:12:12.996208+0000","last_active":"2024-04-07T16:12:12.996208+0000","last_peered":"2024-04-07T16:12:12.996208+0000","last_clean":"2024-04-07T16:12:12.996208+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:12.996208+0000","last_undegraded":"2024-04-07T16:12:12.996208+0000","last_fullsized":"2024-04-07T16:12:12.996208+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.474174+0000","last_change":"2024-04-07T16:12:13.380077+0000","last_active":"2024-04-07T16:12:14.474174+0000","last_peered":"2024-04-07T16:12:14.474174+0000","last_clean":"2024-04-07T16:12:14.474174+0000","last_became_active":"2024-04-07T16:12:13.379758+0000","last_became_peered":"2024-04-07T16:12:13.379758+0000","last_unstale":"2024-04-07T16:12:14.474174+0000","last_undegraded":"2024-04-07T16:12:14.474174+0000","last_fullsized":"2024-04-07T16:12:14.474174+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T22:47:34.836189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":108,"acting":108,"num_store_stats":5},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":7},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1564,"num_objects":13,"num_object_clones":0,"num_object_copies":39,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":27,"ondisk_log_size":27,"up":78,"acting":78,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":54,"seq":231928233996,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28224,"kb_used_data":1196,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716928,"statfs":{"total":95995035648,"available":95966134272,"internally_reserved":0,"allocated":1224704,"data_stored":850413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.212}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.55}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79600000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0740000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1339999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7290000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4510000000000001}]}]},{"osd":6,"up_from":46,"seq":197568495637,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28224,"kb_used_data":1200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716928,"statfs":{"total":95995035648,"available":95966134272,"internally_reserved":0,"allocated":1228800,"data_stored":851373,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Apr 7 16:11:42 2024","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.27,"5min":1.27,"15min":1.27},"last":3.4079999999999999},{"interface":"front","average":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":1.2849999999999999,"5min":1.2849999999999999,"15min":1.2849999999999999},"last":4.165}]},{"osd":1,"last update":"Sun Apr 7 16:11:42 2024","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.427,"5min":1.427,"15min":1.427},"last":1.8799999999999999},{"interface":"front","average":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.2470000000000001,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":3.4889999999999999}]},{"osd":2,"last update":"Sun Apr 7 16:11:42 2024","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.359,"5min":1.359,"15min":1.359},"last":4.218},{"interface":"front","average":{"1min":0.999,"5min":0.999,"15min":0.999},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.5029999999999999,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":1.0529999999999999}]},{"osd":3,"last update":"Sun Apr 7 16:11:42 2024","interfaces":[{"interface":"back","average":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.363,"5min":1.363,"15min":1.363},"last":4.2809999999999997},{"interface":"front","average":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.2589999999999999,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":3.3079999999999998}]},{"osd":4,"last update":"Sun Apr 7 16:11:42 2024","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.5609999999999999,"5min":1.5609999999999999,"15min":1.5609999999999999},"last":1.216},{"interface":"front","average":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.2809999999999999,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":4.8209999999999997}]},{"osd":5,"last update":"Sun Apr 7 16:11:42 2024","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.3680000000000001,"5min":1.3680000000000001,"15min":1.3680000000000001},"last":1.974},{"interface":"front","average":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.6279999999999999,"5min":1.6279999999999999,"15min":1.6279999999999999},"last":1.1419999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":5.2350000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574916,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27588,"kb_used_data":560,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717564,"statfs":{"total":95995035648,"available":95966785536,"internally_reserved":0,"allocated":573440,"data_stored":232554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Apr 7 16:11:48 2024","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.66300000000000003,"15min":0.57899999999999996},"min":{"1min":0.49299999999999999,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":1.518,"5min":1.518,"15min":1.518},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.871,"5min":0.74299999999999999,"15min":0.629},"min":{"1min":0.59899999999999998,"5min":0.315,"15min":0.315},"max":{"1min":1.2969999999999999,"5min":2.8620000000000001,"15min":2.8620000000000001},"last":0.66600000000000004}]},{"osd":2,"last update":"Sun Apr 7 16:11:31 2024","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.73399999999999999,"15min":0.66000000000000003},"min":{"1min":0.56599999999999995,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.345,"5min":1.345,"15min":1.345},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.71099999999999997,"15min":0.65700000000000003},"min":{"1min":0.60399999999999998,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.5369999999999999,"5min":1.5369999999999999,"15min":1.5369999999999999},"last":0.623}]},{"osd":3,"last update":"Sun Apr 7 16:11:23 2024","interfaces":[{"interface":"back","average":{"1min":1.032,"5min":0.86699999999999999,"15min":0.82299999999999995},"min":{"1min":0.64400000000000002,"5min":0.628,"15min":0.628},"max":{"1min":1.343,"5min":1.343,"15min":1.343},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.91700000000000004,"5min":0.79800000000000004,"15min":0.76200000000000001},"min":{"1min":0.66700000000000004,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.4770000000000001,"5min":1.4770000000000001,"15min":1.4770000000000001},"last":0.56299999999999994}]},{"osd":4,"last update":"Sun Apr 7 16:11:11 2024","interfaces":[{"interface":"back","average":{"1min":0.94999999999999996,"5min":0.93200000000000005,"15min":0.92900000000000005},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.5149999999999999,"5min":1.5149999999999999,"15min":1.5149999999999999},"last":1.099},{"interface":"front","average":{"1min":1.119,"5min":1.056,"15min":1.046},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.9299999999999999,"5min":1.9299999999999999,"15min":1.9299999999999999},"last":0.877}]},{"osd":5,"last update":"Sun Apr 7 16:11:55 2024","interfaces":[{"interface":"back","average":{"1min":1.151,"5min":1.23,"15min":1.2430000000000001},"min":{"1min":0.85199999999999998,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":1.6870000000000001,"5min":2.0510000000000002,"15min":2.0510000000000002},"last":0.93600000000000005},{"interface":"front","average":{"1min":1.099,"5min":1.1339999999999999,"15min":1.1399999999999999},"min":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"max":{"1min":1.577,"5min":1.8859999999999999,"15min":1.8859999999999999},"last":0.96199999999999997}]},{"osd":6,"last update":"Sun Apr 7 16:11:43 2024","interfaces":[{"interface":"back","average":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"min":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"max":{"1min":1.466,"5min":1.466,"15min":1.466},"last":1.1799999999999999},{"interface":"front","average":{"1min":1.274,"5min":1.274,"15min":1.274},"min":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"max":{"1min":1.6040000000000001,"5min":1.6040000000000001,"15min":1.6040000000000001},"last":1.198}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0269999999999999}]}]},{"osd":0,"up_from":8,"seq":34359738447,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28172,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716980,"statfs":{"total":95995035648,"available":95966187520,"internally_reserved":0,"allocated":1179648,"data_stored":829169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Apr 7 16:11:47 2024","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.63600000000000001,"15min":0.56999999999999995},"min":{"1min":0.56000000000000005,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":1.353,"5min":1.353,"15min":1.353},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.70899999999999996,"15min":0.57299999999999995},"min":{"1min":0.58399999999999996,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.1439999999999999,"5min":1.214,"15min":1.214},"last":0.77800000000000002}]},{"osd":2,"last update":"Sun Apr 7 16:11:34 2024","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.70899999999999996,"15min":0.66700000000000004},"min":{"1min":0.61799999999999999,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.89000000000000001,"5min":0.77200000000000002,"15min":0.73699999999999999},"min":{"1min":0.626,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.2749999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.81299999999999994}]},{"osd":3,"last update":"Sun Apr 7 16:11:19 2024","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.78200000000000003,"15min":0.75700000000000001},"min":{"1min":0.67800000000000005,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.89000000000000001,"5min":0.81899999999999995,"15min":0.79900000000000004},"min":{"1min":0.68200000000000005,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.143,"5min":1.143,"15min":1.143},"last":0.95099999999999996}]},{"osd":4,"last update":"Sun Apr 7 16:12:08 2024","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":0.93600000000000005,"15min":0.92300000000000004},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.306,"5min":1.306,"15min":1.306},"last":0.98599999999999999},{"interface":"front","average":{"1min":1.0109999999999999,"5min":0.98799999999999999,"15min":0.96399999999999997},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":1.2150000000000001,"5min":1.7809999999999999,"15min":1.7809999999999999},"last":0.88700000000000001}]},{"osd":5,"last update":"Sun Apr 7 16:11:56 2024","interfaces":[{"interface":"back","average":{"1min":1.0349999999999999,"5min":0.98899999999999999,"15min":0.98099999999999998},"min":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"max":{"1min":1.196,"5min":1.196,"15min":1.196},"last":1.1699999999999999},{"interface":"front","average":{"1min":1.0409999999999999,"5min":1.0249999999999999,"15min":1.0229999999999999},"min":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":1.48,"5min":1.48,"15min":1.48},"last":1.268}]},{"osd":6,"last update":"Sun Apr 7 16:11:41 2024","interfaces":[{"interface":"back","average":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"min":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":1.3600000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":1.0209999999999999},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"min":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"max":{"1min":1.3580000000000001,"5min":1.3580000000000001,"15min":1.3580000000000001},"last":1.2969999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1539999999999999}]}]},{"osd":2,"up_from":18,"seq":77309411388,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31724,"kb_used_data":608,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713428,"statfs":{"total":95995035648,"available":95962550272,"internally_reserved":0,"allocated":622592,"data_stored":259441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Apr 7 16:11:30 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.63700000000000001,"15min":0.58599999999999997},"min":{"1min":0.56499999999999995,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.96199999999999997,"5min":1.103,"15min":1.103},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.67600000000000005,"15min":0.60599999999999998},"min":{"1min":0.57299999999999995,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.75800000000000001}]},{"osd":1,"last update":"Sun Apr 7 16:11:30 2024","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.69399999999999995,"15min":0.64100000000000001},"min":{"1min":0.59099999999999997,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":1.143},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.71699999999999997,"15min":0.64900000000000002},"min":{"1min":0.54200000000000004,"5min":0.313,"15min":0.313},"max":{"1min":1.0609999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.53400000000000003}]},{"osd":3,"last update":"Sun Apr 7 16:11:18 2024","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.81699999999999995,"15min":0.81100000000000005},"min":{"1min":0.67000000000000004,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.81200000000000006,"15min":0.80200000000000005},"min":{"1min":0.66100000000000003,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.002,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.88600000000000001}]},{"osd":4,"last update":"Sun Apr 7 16:12:09 2024","interfaces":[{"interface":"back","average":{"1min":0.96699999999999997,"5min":0.93400000000000005,"15min":0.92100000000000004},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.4299999999999999,"5min":1.4299999999999999,"15min":1.4299999999999999},"last":0.96999999999999997},{"interface":"front","average":{"1min":1.018,"5min":0.95799999999999996,"15min":0.93999999999999995},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.4490000000000001,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.92400000000000004}]},{"osd":5,"last update":"Sun Apr 7 16:11:54 2024","interfaces":[{"interface":"back","average":{"1min":1.05,"5min":0.98699999999999999,"15min":0.97699999999999998},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":1.6970000000000001,"5min":1.6970000000000001,"15min":1.6970000000000001},"last":0.83299999999999996},{"interface":"front","average":{"1min":1.0780000000000001,"5min":1.046,"15min":1.04},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.6000000000000001,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":1.069}]},{"osd":6,"last update":"Sun Apr 7 16:11:39 2024","interfaces":[{"interface":"back","average":{"1min":1.05,"5min":1.05,"15min":1.05},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.397,"5min":1.397,"15min":1.397},"last":1.0009999999999999},{"interface":"front","average":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.355,"5min":1.355,"15min":1.355},"last":1.1040000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80500000000000005}]}]},{"osd":3,"up_from":25,"seq":107374182451,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27656,"kb_used_data":628,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717496,"statfs":{"total":95995035648,"available":95966715904,"internally_reserved":0,"allocated":643072,"data_stored":261109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Apr 7 16:11:24 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.67000000000000004,"15min":0.66900000000000004},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.97399999999999998,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.54400000000000004,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.327,"5min":1.327,"15min":1.327},"last":0.39100000000000001}]},{"osd":1,"last update":"Sun Apr 7 16:11:24 2024","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.77200000000000002,"15min":0.75600000000000001},"min":{"1min":0.57999999999999996,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.544,"5min":1.544,"15min":1.544},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.76600000000000001,"15min":0.75800000000000001},"min":{"1min":0.60599999999999998,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.4510000000000001,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.375}]},{"osd":2,"last update":"Sun Apr 7 16:11:24 2024","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.82499999999999996,"15min":0.80900000000000005},"min":{"1min":0.70699999999999996,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.256,"5min":1.256,"15min":1.256},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.81599999999999995,"15min":0.81399999999999995},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.286,"5min":1.286,"15min":1.286},"last":0.35999999999999999}]},{"osd":4,"last update":"Sun Apr 7 16:12:08 2024","interfaces":[{"interface":"back","average":{"1min":1.036,"5min":0.97699999999999998,"15min":0.95699999999999996},"min":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.28,"5min":1.552,"15min":1.552},"last":0.81000000000000005},{"interface":"front","average":{"1min":1.0569999999999999,"5min":0.999,"15min":0.97699999999999998},"min":{"1min":0.81999999999999995,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.3640000000000001,"5min":1.3640000000000001,"15min":1.3640000000000001},"last":0.65100000000000002}]},{"osd":5,"last update":"Sun Apr 7 16:11:54 2024","interfaces":[{"interface":"back","average":{"1min":1.081,"5min":1.0609999999999999,"15min":1.0580000000000001},"min":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":1.3180000000000001,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":0.70999999999999996},{"interface":"front","average":{"1min":1.0109999999999999,"5min":1.0349999999999999,"15min":1.0389999999999999},"min":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.2410000000000001,"5min":1.571,"15min":1.571},"last":0.66800000000000004}]},{"osd":6,"last update":"Sun Apr 7 16:11:39 2024","interfaces":[{"interface":"back","average":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"min":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"max":{"1min":1.341,"5min":1.341,"15min":1.341},"last":0.84299999999999997},{"interface":"front","average":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"min":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"max":{"1min":1.4610000000000001,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.61399999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64000000000000001}]}]},{"osd":4,"up_from":30,"seq":128849018921,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31688,"kb_used_data":572,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713464,"statfs":{"total":95995035648,"available":95962587136,"internally_reserved":0,"allocated":585728,"data_stored":238801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Apr 7 16:11:10 2024","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.82899999999999996,"15min":0.81699999999999995},"min":{"1min":0.63800000000000001,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.90300000000000002,"15min":0.90500000000000003},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.1719999999999999,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.873}]},{"osd":1,"last update":"Sun Apr 7 16:11:10 2024","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.81299999999999994,"15min":0.80700000000000005},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.91100000000000003,"5min":0.84499999999999997,"15min":0.83399999999999996},"min":{"1min":0.64600000000000002,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.67700000000000005}]},{"osd":2,"last update":"Sun Apr 7 16:11:10 2024","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":0.874,"15min":0.86199999999999999},"min":{"1min":0.75700000000000001,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.2649999999999999,"5min":1.387,"15min":1.387},"last":1.046},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.88600000000000001,"15min":0.88300000000000001},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.175,"5min":1.175,"15min":1.175},"last":0.63}]},{"osd":3,"last update":"Sun Apr 7 16:11:10 2024","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.93100000000000005,"15min":0.93400000000000005},"min":{"1min":0.76700000000000002,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":1.069,"5min":1.458,"15min":1.458},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.94199999999999995,"5min":0.92600000000000005,"15min":0.92300000000000004},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.2390000000000001,"5min":1.323,"15min":1.323},"last":0.83399999999999996}]},{"osd":5,"last update":"Sun Apr 7 16:11:55 2024","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":0.78200000000000003,"15min":0.75600000000000001},"min":{"1min":0.60999999999999999,"5min":0.315,"15min":0.315},"max":{"1min":1.403,"5min":1.403,"15min":1.403},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.92500000000000004,"5min":0.87,"15min":0.86099999999999999},"min":{"1min":0.57299999999999995,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.3220000000000001,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":1.139}]},{"osd":6,"last update":"Sun Apr 7 16:11:42 2024","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.4139999999999999,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":0.98599999999999999},{"interface":"front","average":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.4370000000000001,"5min":1.4370000000000001,"15min":1.4370000000000001},"last":1.2150000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0129999999999999}]}]},{"osd":5,"up_from":38,"seq":163208757278,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27580,"kb_used_data":556,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717572,"statfs":{"total":95995035648,"available":95966793728,"internally_reserved":0,"allocated":569344,"data_stored":232223,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Apr 7 16:11:57 2024","interfaces":[{"interface":"back","average":{"1min":0.97499999999999998,"5min":0.92800000000000005,"15min":0.92000000000000004},"min":{"1min":0.68200000000000005,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.2609999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.94399999999999995,"15min":0.95699999999999996},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.153,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":1.0069999999999999}]},{"osd":1,"last update":"Sun Apr 7 16:11:57 2024","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.91500000000000004,"15min":0.92200000000000004},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.1619999999999999,"5min":1.331,"15min":1.331},"last":0.95399999999999996},{"interface":"front","average":{"1min":0.97899999999999998,"5min":0.98899999999999999,"15min":0.99099999999999999},"min":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.204,"5min":1.45,"15min":1.45},"last":1.1699999999999999}]},{"osd":2,"last update":"Sun Apr 7 16:11:57 2024","interfaces":[{"interface":"back","average":{"1min":0.91300000000000003,"5min":0.96999999999999997,"15min":0.97999999999999998},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.3140000000000001,"5min":1.462,"15min":1.462},"last":1.0449999999999999},{"interface":"front","average":{"1min":0.93500000000000005,"5min":0.96299999999999997,"15min":0.96699999999999997},"min":{"1min":0.67400000000000004,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.3879999999999999,"5min":1.472,"15min":1.472},"last":0.89000000000000001}]},{"osd":3,"last update":"Sun Apr 7 16:11:57 2024","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":1.173,"15min":1.218},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.4219999999999999,"5min":6.3860000000000001,"15min":6.3860000000000001},"last":1.2},{"interface":"front","average":{"1min":0.96199999999999997,"5min":1.0009999999999999,"15min":1.0069999999999999},"min":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.448,"5min":1.556,"15min":1.556},"last":0.79300000000000004}]},{"osd":4,"last update":"Sun Apr 7 16:11:57 2024","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.80900000000000005,"15min":0.80700000000000005},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.91500000000000004},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.90700000000000003,"15min":0.90900000000000003},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.22,"5min":1.4119999999999999,"15min":1.4119999999999999},"last":1.252}]},{"osd":6,"last update":"Sun Apr 7 16:11:39 2024","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":0.75},{"interface":"front","average":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":1.544,"5min":1.544,"15min":1.544},"last":1.125}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0940000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-07T16:12:17.114 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.3e is not active+clean 2024-04-07T16:12:17.114 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-07T16:12:12.996208+0000', 'last_change': '2024-04-07T16:12:12.996208+0000', 'last_active': '2024-04-07T16:12:12.996208+0000', 'last_peered': '2024-04-07T16:12:12.996208+0000', 'last_clean': '2024-04-07T16:12:12.996208+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-07T16:12:12.996208+0000', 'last_undegraded': '2024-04-07T16:12:12.996208+0000', 'last_fullsized': '2024-04-07T16:12:12.996208+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-07T16:12:12.996208+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-07T16:12:12.996208+0000', 'last_clean_scrub_stamp': '2024-04-07T16:12:12.996208+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-07T16:12:17.114 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.3d is not active+clean 2024-04-07T16:12:17.114 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-07T16:12:12.996208+0000', 'last_change': '2024-04-07T16:12:12.996208+0000', 'last_active': '2024-04-07T16:12:12.996208+0000', 'last_peered': '2024-04-07T16:12:12.996208+0000', 'last_clean': '2024-04-07T16:12:12.996208+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-07T16:12:12.996208+0000', 'last_undegraded': '2024-04-07T16:12:12.996208+0000', 'last_fullsized': '2024-04-07T16:12:12.996208+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-07T16:12:12.996208+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-07T16:12:12.996208+0000', 'last_clean_scrub_stamp': '2024-04-07T16:12:12.996208+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-07T16:12:17.114 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.3c is not active+clean 2024-04-07T16:12:17.114 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-07T16:12:12.996208+0000', 'last_change': '2024-04-07T16:12:12.996208+0000', 'last_active': '2024-04-07T16:12:12.996208+0000', 'last_peered': '2024-04-07T16:12:12.996208+0000', 'last_clean': '2024-04-07T16:12:12.996208+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-07T16:12:12.996208+0000', 'last_undegraded': '2024-04-07T16:12:12.996208+0000', 'last_fullsized': '2024-04-07T16:12:12.996208+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-07T16:12:12.996208+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-07T16:12:12.996208+0000', 'last_clean_scrub_stamp': '2024-04-07T16:12:12.996208+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-07T16:12:17.114 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.3a is not active+clean 2024-04-07T16:12:17.114 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-07T16:12:12.996208+0000', 'last_change': '2024-04-07T16:12:12.996208+0000', 'last_active': '2024-04-07T16:12:12.996208+0000', 'last_peered': '2024-04-07T16:12:12.996208+0000', 'last_clean': '2024-04-07T16:12:12.996208+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-07T16:12:12.996208+0000', 'last_undegraded': '2024-04-07T16:12:12.996208+0000', 'last_fullsized': '2024-04-07T16:12:12.996208+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-07T16:12:12.996208+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-07T16:12:12.996208+0000', 'last_clean_scrub_stamp': '2024-04-07T16:12:12.996208+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-07T16:12:17.115 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.34 is not active+clean 2024-04-07T16:12:17.115 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2024-04-07T16:12:13.005867+0000', 'last_change': '2024-04-07T16:12:13.005867+0000', 'last_active': '2024-04-07T16:12:12.996208+0000', 'last_peered': '2024-04-07T16:12:12.996208+0000', 'last_clean': '2024-04-07T16:12:12.996208+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-07T16:12:13.005867+0000', 'last_undegraded': '2024-04-07T16:12:13.005867+0000', 'last_fullsized': '2024-04-07T16:12:13.005867+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-07T16:12:12.996208+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-07T16:12:12.996208+0000', 'last_clean_scrub_stamp': '2024-04-07T16:12:12.996208+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-04-08T22:29:22.509564+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6, 0], 'acting': [4, 6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-07T16:12:17.115 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.32 is not active+clean 2024-04-07T16:12:17.115 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-07T16:12:12.996208+0000', 'last_change': '2024-04-07T16:12:12.996208+0000', 'last_active': '2024-04-07T16:12:12.996208+0000', 'last_peered': '2024-04-07T16:12:12.996208+0000', 'last_clean': '2024-04-07T16:12:12.996208+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-07T16:12:12.996208+0000', 'last_undegraded': '2024-04-07T16:12:12.996208+0000', 'last_fullsized': '2024-04-07T16:12:12.996208+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-07T16:12:12.996208+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-07T16:12:12.996208+0000', 'last_clean_scrub_stamp': '2024-04-07T16:12:12.996208+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-07T16:12:17.115 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.31 is not active+clean 2024-04-07T16:12:17.115 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-07T16:12:12.996208+0000', 'last_change': '2024-04-07T16:12:12.996208+0000', 'last_active': '2024-04-07T16:12:12.996208+0000', 'last_peered': '2024-04-07T16:12:12.996208+0000', 'last_clean': '2024-04-07T16:12:12.996208+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-07T16:12:12.996208+0000', 'last_undegraded': '2024-04-07T16:12:12.996208+0000', 'last_fullsized': '2024-04-07T16:12:12.996208+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-07T16:12:12.996208+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-07T16:12:12.996208+0000', 'last_clean_scrub_stamp': '2024-04-07T16:12:12.996208+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-07T16:12:17.115 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.30 is not active+clean 2024-04-07T16:12:17.115 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-07T16:12:12.996208+0000', 'last_change': '2024-04-07T16:12:12.996208+0000', 'last_active': '2024-04-07T16:12:12.996208+0000', 'last_peered': '2024-04-07T16:12:12.996208+0000', 'last_clean': '2024-04-07T16:12:12.996208+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-07T16:12:12.996208+0000', 'last_undegraded': '2024-04-07T16:12:12.996208+0000', 'last_fullsized': '2024-04-07T16:12:12.996208+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-07T16:12:12.996208+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-07T16:12:12.996208+0000', 'last_clean_scrub_stamp': '2024-04-07T16:12:12.996208+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-07T16:12:17.115 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.2f is not active+clean 2024-04-07T16:12:17.115 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-07T16:12:12.996208+0000', 'last_change': '2024-04-07T16:12:12.996208+0000', 'last_active': '2024-04-07T16:12:12.996208+0000', 'last_peered': '2024-04-07T16:12:12.996208+0000', 'last_clean': '2024-04-07T16:12:12.996208+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-07T16:12:12.996208+0000', 'last_undegraded': '2024-04-07T16:12:12.996208+0000', 'last_fullsized': '2024-04-07T16:12:12.996208+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-07T16:12:12.996208+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-07T16:12:12.996208+0000', 'last_clean_scrub_stamp': '2024-04-07T16:12:12.996208+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-07T16:12:17.116 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.1f is not active+clean 2024-04-07T16:12:17.116 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 59, 'state': 'creating+peering', 'last_fresh': '2024-04-07T16:12:12.000271+0000', 'last_change': '2024-04-07T16:12:12.000271+0000', 'last_active': '2024-04-07T16:12:11.994648+0000', 'last_peered': '2024-04-07T16:12:11.994648+0000', 'last_clean': '2024-04-07T16:12:11.994648+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-07T16:12:12.000271+0000', 'last_undegraded': '2024-04-07T16:12:12.000271+0000', 'last_fullsized': '2024-04-07T16:12:12.000271+0000', 'mapping_epoch': 59, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 59, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-07T16:12:11.994648+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-07T16:12:11.994648+0000', 'last_clean_scrub_stamp': '2024-04-07T16:12:11.994648+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-04-08T19:01:44.798719+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7, 4], 'acting': [0, 7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-07T16:12:17.116 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1f is not active+clean 2024-04-07T16:12:17.116 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-07T16:12:12.996208+0000', 'last_change': '2024-04-07T16:12:12.996208+0000', 'last_active': '2024-04-07T16:12:12.996208+0000', 'last_peered': '2024-04-07T16:12:12.996208+0000', 'last_clean': '2024-04-07T16:12:12.996208+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-07T16:12:12.996208+0000', 'last_undegraded': '2024-04-07T16:12:12.996208+0000', 'last_fullsized': '2024-04-07T16:12:12.996208+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-07T16:12:12.996208+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-07T16:12:12.996208+0000', 'last_clean_scrub_stamp': '2024-04-07T16:12:12.996208+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-07T16:12:17.116 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1c is not active+clean 2024-04-07T16:12:17.116 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-07T16:12:12.996208+0000', 'last_change': '2024-04-07T16:12:12.996208+0000', 'last_active': '2024-04-07T16:12:12.996208+0000', 'last_peered': '2024-04-07T16:12:12.996208+0000', 'last_clean': '2024-04-07T16:12:12.996208+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-07T16:12:12.996208+0000', 'last_undegraded': '2024-04-07T16:12:12.996208+0000', 'last_fullsized': '2024-04-07T16:12:12.996208+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-07T16:12:12.996208+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-07T16:12:12.996208+0000', 'last_clean_scrub_stamp': '2024-04-07T16:12:12.996208+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-07T16:12:17.116 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1d is not active+clean 2024-04-07T16:12:17.116 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-07T16:12:12.996208+0000', 'last_change': '2024-04-07T16:12:12.996208+0000', 'last_active': '2024-04-07T16:12:12.996208+0000', 'last_peered': '2024-04-07T16:12:12.996208+0000', 'last_clean': '2024-04-07T16:12:12.996208+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-07T16:12:12.996208+0000', 'last_undegraded': '2024-04-07T16:12:12.996208+0000', 'last_fullsized': '2024-04-07T16:12:12.996208+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-07T16:12:12.996208+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-07T16:12:12.996208+0000', 'last_clean_scrub_stamp': '2024-04-07T16:12:12.996208+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-07T16:12:17.116 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1a is not active+clean 2024-04-07T16:12:17.116 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2024-04-07T16:12:13.002508+0000', 'last_change': '2024-04-07T16:12:13.002508+0000', 'last_active': '2024-04-07T16:12:12.996208+0000', 'last_peered': '2024-04-07T16:12:12.996208+0000', 'last_clean': '2024-04-07T16:12:12.996208+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-07T16:12:13.002508+0000', 'last_undegraded': '2024-04-07T16:12:13.002508+0000', 'last_fullsized': '2024-04-07T16:12:13.002508+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-07T16:12:12.996208+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-07T16:12:12.996208+0000', 'last_clean_scrub_stamp': '2024-04-07T16:12:12.996208+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-04-08T23:20:40.949255+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1, 2], 'acting': [4, 1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-07T16:12:17.117 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1b is not active+clean 2024-04-07T16:12:17.117 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-07T16:12:12.996208+0000', 'last_change': '2024-04-07T16:12:12.996208+0000', 'last_active': '2024-04-07T16:12:12.996208+0000', 'last_peered': '2024-04-07T16:12:12.996208+0000', 'last_clean': '2024-04-07T16:12:12.996208+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-07T16:12:12.996208+0000', 'last_undegraded': '2024-04-07T16:12:12.996208+0000', 'last_fullsized': '2024-04-07T16:12:12.996208+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-07T16:12:12.996208+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-07T16:12:12.996208+0000', 'last_clean_scrub_stamp': '2024-04-07T16:12:12.996208+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-07T16:12:17.117 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.18 is not active+clean 2024-04-07T16:12:17.117 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 59, 'state': 'creating+peering', 'last_fresh': '2024-04-07T16:12:12.000806+0000', 'last_change': '2024-04-07T16:12:12.000806+0000', 'last_active': '2024-04-07T16:12:11.994648+0000', 'last_peered': '2024-04-07T16:12:11.994648+0000', 'last_clean': '2024-04-07T16:12:11.994648+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-07T16:12:12.000806+0000', 'last_undegraded': '2024-04-07T16:12:12.000806+0000', 'last_fullsized': '2024-04-07T16:12:12.000806+0000', 'mapping_epoch': 59, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 59, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-07T16:12:11.994648+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-07T16:12:11.994648+0000', 'last_clean_scrub_stamp': '2024-04-07T16:12:11.994648+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-04-08T22:44:14.623116+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4, 7], 'acting': [5, 4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-07T16:12:17.117 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.19 is not active+clean 2024-04-07T16:12:17.117 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-07T16:12:12.996208+0000', 'last_change': '2024-04-07T16:12:12.996208+0000', 'last_active': '2024-04-07T16:12:12.996208+0000', 'last_peered': '2024-04-07T16:12:12.996208+0000', 'last_clean': '2024-04-07T16:12:12.996208+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-07T16:12:12.996208+0000', 'last_undegraded': '2024-04-07T16:12:12.996208+0000', 'last_fullsized': '2024-04-07T16:12:12.996208+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-07T16:12:12.996208+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-07T16:12:12.996208+0000', 'last_clean_scrub_stamp': '2024-04-07T16:12:12.996208+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-07T16:12:17.117 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.16 is not active+clean 2024-04-07T16:12:17.117 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-07T16:12:12.996208+0000', 'last_change': '2024-04-07T16:12:12.996208+0000', 'last_active': '2024-04-07T16:12:12.996208+0000', 'last_peered': '2024-04-07T16:12:12.996208+0000', 'last_clean': '2024-04-07T16:12:12.996208+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-07T16:12:12.996208+0000', 'last_undegraded': '2024-04-07T16:12:12.996208+0000', 'last_fullsized': '2024-04-07T16:12:12.996208+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-07T16:12:12.996208+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-07T16:12:12.996208+0000', 'last_clean_scrub_stamp': '2024-04-07T16:12:12.996208+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-07T16:12:17.117 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.16 is not active+clean 2024-04-07T16:12:17.117 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 59, 'state': 'creating+peering', 'last_fresh': '2024-04-07T16:12:12.001015+0000', 'last_change': '2024-04-07T16:12:12.001015+0000', 'last_active': '2024-04-07T16:12:11.994648+0000', 'last_peered': '2024-04-07T16:12:11.994648+0000', 'last_clean': '2024-04-07T16:12:11.994648+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-07T16:12:12.001015+0000', 'last_undegraded': '2024-04-07T16:12:12.001015+0000', 'last_fullsized': '2024-04-07T16:12:12.001015+0000', 'mapping_epoch': 59, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 59, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-07T16:12:11.994648+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-07T16:12:11.994648+0000', 'last_clean_scrub_stamp': '2024-04-07T16:12:11.994648+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-04-08T18:26:16.873044+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6, 2], 'acting': [5, 6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-07T16:12:17.118 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.17 is not active+clean 2024-04-07T16:12:17.118 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-07T16:12:12.996208+0000', 'last_change': '2024-04-07T16:12:12.996208+0000', 'last_active': '2024-04-07T16:12:12.996208+0000', 'last_peered': '2024-04-07T16:12:12.996208+0000', 'last_clean': '2024-04-07T16:12:12.996208+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-07T16:12:12.996208+0000', 'last_undegraded': '2024-04-07T16:12:12.996208+0000', 'last_fullsized': '2024-04-07T16:12:12.996208+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-07T16:12:12.996208+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-07T16:12:12.996208+0000', 'last_clean_scrub_stamp': '2024-04-07T16:12:12.996208+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-07T16:12:17.118 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.15 is not active+clean 2024-04-07T16:12:17.118 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-07T16:12:11.994648+0000', 'last_change': '2024-04-07T16:12:11.994648+0000', 'last_active': '2024-04-07T16:12:11.994648+0000', 'last_peered': '2024-04-07T16:12:11.994648+0000', 'last_clean': '2024-04-07T16:12:11.994648+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-07T16:12:11.994648+0000', 'last_undegraded': '2024-04-07T16:12:11.994648+0000', 'last_fullsized': '2024-04-07T16:12:11.994648+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-07T16:12:11.994648+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-07T16:12:11.994648+0000', 'last_clean_scrub_stamp': '2024-04-07T16:12:11.994648+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-07T16:12:17.118 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.14 is not active+clean 2024-04-07T16:12:17.118 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2024-04-07T16:12:13.004673+0000', 'last_change': '2024-04-07T16:12:13.004673+0000', 'last_active': '2024-04-07T16:12:12.996208+0000', 'last_peered': '2024-04-07T16:12:12.996208+0000', 'last_clean': '2024-04-07T16:12:12.996208+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-07T16:12:13.004673+0000', 'last_undegraded': '2024-04-07T16:12:13.004673+0000', 'last_fullsized': '2024-04-07T16:12:13.004673+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-07T16:12:12.996208+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-07T16:12:12.996208+0000', 'last_clean_scrub_stamp': '2024-04-07T16:12:12.996208+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-04-08T16:55:37.429799+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7, 6], 'acting': [4, 7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-07T16:12:17.118 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.13 is not active+clean 2024-04-07T16:12:17.118 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 59, 'state': 'creating+peering', 'last_fresh': '2024-04-07T16:12:12.001190+0000', 'last_change': '2024-04-07T16:12:12.001190+0000', 'last_active': '2024-04-07T16:12:11.994648+0000', 'last_peered': '2024-04-07T16:12:11.994648+0000', 'last_clean': '2024-04-07T16:12:11.994648+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-07T16:12:12.001190+0000', 'last_undegraded': '2024-04-07T16:12:12.001190+0000', 'last_fullsized': '2024-04-07T16:12:12.001190+0000', 'mapping_epoch': 59, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 59, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-07T16:12:11.994648+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-07T16:12:11.994648+0000', 'last_clean_scrub_stamp': '2024-04-07T16:12:11.994648+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-04-09T01:50:02.195507+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4, 3], 'acting': [0, 4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-07T16:12:17.118 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.12 is not active+clean 2024-04-07T16:12:17.118 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-07T16:12:12.996208+0000', 'last_change': '2024-04-07T16:12:12.996208+0000', 'last_active': '2024-04-07T16:12:12.996208+0000', 'last_peered': '2024-04-07T16:12:12.996208+0000', 'last_clean': '2024-04-07T16:12:12.996208+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-07T16:12:12.996208+0000', 'last_undegraded': '2024-04-07T16:12:12.996208+0000', 'last_fullsized': '2024-04-07T16:12:12.996208+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-07T16:12:12.996208+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-07T16:12:12.996208+0000', 'last_clean_scrub_stamp': '2024-04-07T16:12:12.996208+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-07T16:12:17.119 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.12 is not active+clean 2024-04-07T16:12:17.119 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 59, 'state': 'creating+peering', 'last_fresh': '2024-04-07T16:12:12.002099+0000', 'last_change': '2024-04-07T16:12:12.002099+0000', 'last_active': '2024-04-07T16:12:11.994648+0000', 'last_peered': '2024-04-07T16:12:11.994648+0000', 'last_clean': '2024-04-07T16:12:11.994648+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-07T16:12:12.002099+0000', 'last_undegraded': '2024-04-07T16:12:12.002099+0000', 'last_fullsized': '2024-04-07T16:12:12.002099+0000', 'mapping_epoch': 59, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 59, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-07T16:12:11.994648+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-07T16:12:11.994648+0000', 'last_clean_scrub_stamp': '2024-04-07T16:12:11.994648+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-04-08T23:15:11.494437+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3, 7], 'acting': [5, 3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-07T16:12:17.119 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.f is not active+clean 2024-04-07T16:12:17.119 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.f', 'version': "0'0", 'reported_seq': 4, 'reported_epoch': 60, 'state': 'creating+activating', 'last_fresh': '2024-04-07T16:12:13.024777+0000', 'last_change': '2024-04-07T16:12:13.002417+0000', 'last_active': '2024-04-07T16:12:11.994648+0000', 'last_peered': '2024-04-07T16:12:11.994648+0000', 'last_clean': '2024-04-07T16:12:11.994648+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-07T16:12:13.024777+0000', 'last_undegraded': '2024-04-07T16:12:13.024777+0000', 'last_fullsized': '2024-04-07T16:12:13.024777+0000', 'mapping_epoch': 59, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 59, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-07T16:12:11.994648+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-07T16:12:11.994648+0000', 'last_clean_scrub_stamp': '2024-04-07T16:12:11.994648+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-04-08T17:55:06.727142+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0, 7], 'acting': [4, 0, 7], 'avail_no_missing': ['4', '0', '7'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-07T16:12:17.119 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.4 is not active+clean 2024-04-07T16:12:17.119 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-07T16:12:11.994648+0000', 'last_change': '2024-04-07T16:12:11.994648+0000', 'last_active': '2024-04-07T16:12:11.994648+0000', 'last_peered': '2024-04-07T16:12:11.994648+0000', 'last_clean': '2024-04-07T16:12:11.994648+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-07T16:12:11.994648+0000', 'last_undegraded': '2024-04-07T16:12:11.994648+0000', 'last_fullsized': '2024-04-07T16:12:11.994648+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-07T16:12:11.994648+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-07T16:12:11.994648+0000', 'last_clean_scrub_stamp': '2024-04-07T16:12:11.994648+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-07T16:12:17.119 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.5 is not active+clean 2024-04-07T16:12:17.119 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-07T16:12:12.996208+0000', 'last_change': '2024-04-07T16:12:12.996208+0000', 'last_active': '2024-04-07T16:12:12.996208+0000', 'last_peered': '2024-04-07T16:12:12.996208+0000', 'last_clean': '2024-04-07T16:12:12.996208+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-07T16:12:12.996208+0000', 'last_undegraded': '2024-04-07T16:12:12.996208+0000', 'last_fullsized': '2024-04-07T16:12:12.996208+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-07T16:12:12.996208+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-07T16:12:12.996208+0000', 'last_clean_scrub_stamp': '2024-04-07T16:12:12.996208+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-07T16:12:17.119 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.2 is not active+clean 2024-04-07T16:12:17.119 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 59, 'state': 'creating+peering', 'last_fresh': '2024-04-07T16:12:12.001190+0000', 'last_change': '2024-04-07T16:12:12.001190+0000', 'last_active': '2024-04-07T16:12:11.994648+0000', 'last_peered': '2024-04-07T16:12:11.994648+0000', 'last_clean': '2024-04-07T16:12:11.994648+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-07T16:12:12.001190+0000', 'last_undegraded': '2024-04-07T16:12:12.001190+0000', 'last_fullsized': '2024-04-07T16:12:12.001190+0000', 'mapping_epoch': 59, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 59, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-07T16:12:11.994648+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-07T16:12:11.994648+0000', 'last_clean_scrub_stamp': '2024-04-07T16:12:11.994648+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-04-08T18:15:00.506827+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1, 6], 'acting': [5, 1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-07T16:12:17.120 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.3 is not active+clean 2024-04-07T16:12:17.120 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2024-04-07T16:12:13.003585+0000', 'last_change': '2024-04-07T16:12:13.003585+0000', 'last_active': '2024-04-07T16:12:12.996208+0000', 'last_peered': '2024-04-07T16:12:12.996208+0000', 'last_clean': '2024-04-07T16:12:12.996208+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-07T16:12:13.003585+0000', 'last_undegraded': '2024-04-07T16:12:13.003585+0000', 'last_fullsized': '2024-04-07T16:12:13.003585+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-07T16:12:12.996208+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-07T16:12:12.996208+0000', 'last_clean_scrub_stamp': '2024-04-07T16:12:12.996208+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-04-09T03:36:24.354110+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0, 6], 'acting': [4, 0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-07T16:12:17.120 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.0 is not active+clean 2024-04-07T16:12:17.120 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-07T16:12:12.996208+0000', 'last_change': '2024-04-07T16:12:12.996208+0000', 'last_active': '2024-04-07T16:12:12.996208+0000', 'last_peered': '2024-04-07T16:12:12.996208+0000', 'last_clean': '2024-04-07T16:12:12.996208+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-07T16:12:12.996208+0000', 'last_undegraded': '2024-04-07T16:12:12.996208+0000', 'last_fullsized': '2024-04-07T16:12:12.996208+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-07T16:12:12.996208+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-07T16:12:12.996208+0000', 'last_clean_scrub_stamp': '2024-04-07T16:12:12.996208+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-07T16:12:17.120 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.3 is not active+clean 2024-04-07T16:12:17.120 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 59, 'state': 'creating+peering', 'last_fresh': '2024-04-07T16:12:12.001243+0000', 'last_change': '2024-04-07T16:12:12.001243+0000', 'last_active': '2024-04-07T16:12:11.994648+0000', 'last_peered': '2024-04-07T16:12:11.994648+0000', 'last_clean': '2024-04-07T16:12:11.994648+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-07T16:12:12.001243+0000', 'last_undegraded': '2024-04-07T16:12:12.001243+0000', 'last_fullsized': '2024-04-07T16:12:12.001243+0000', 'mapping_epoch': 59, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 59, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-07T16:12:11.994648+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-07T16:12:11.994648+0000', 'last_clean_scrub_stamp': '2024-04-07T16:12:11.994648+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-04-09T02:48:50.652491+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2, 7], 'acting': [5, 2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-07T16:12:17.120 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1 is not active+clean 2024-04-07T16:12:17.120 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-07T16:12:12.996208+0000', 'last_change': '2024-04-07T16:12:12.996208+0000', 'last_active': '2024-04-07T16:12:12.996208+0000', 'last_peered': '2024-04-07T16:12:12.996208+0000', 'last_clean': '2024-04-07T16:12:12.996208+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-07T16:12:12.996208+0000', 'last_undegraded': '2024-04-07T16:12:12.996208+0000', 'last_fullsized': '2024-04-07T16:12:12.996208+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-07T16:12:12.996208+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-07T16:12:12.996208+0000', 'last_clean_scrub_stamp': '2024-04-07T16:12:12.996208+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-07T16:12:17.120 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.4 is not active+clean 2024-04-07T16:12:17.120 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-07T16:12:12.996208+0000', 'last_change': '2024-04-07T16:12:12.996208+0000', 'last_active': '2024-04-07T16:12:12.996208+0000', 'last_peered': '2024-04-07T16:12:12.996208+0000', 'last_clean': '2024-04-07T16:12:12.996208+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-07T16:12:12.996208+0000', 'last_undegraded': '2024-04-07T16:12:12.996208+0000', 'last_fullsized': '2024-04-07T16:12:12.996208+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-07T16:12:12.996208+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-07T16:12:12.996208+0000', 'last_clean_scrub_stamp': '2024-04-07T16:12:12.996208+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-07T16:12:17.120 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.6 is not active+clean 2024-04-07T16:12:17.121 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-07T16:12:11.994648+0000', 'last_change': '2024-04-07T16:12:11.994648+0000', 'last_active': '2024-04-07T16:12:11.994648+0000', 'last_peered': '2024-04-07T16:12:11.994648+0000', 'last_clean': '2024-04-07T16:12:11.994648+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-07T16:12:11.994648+0000', 'last_undegraded': '2024-04-07T16:12:11.994648+0000', 'last_fullsized': '2024-04-07T16:12:11.994648+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-07T16:12:11.994648+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-07T16:12:11.994648+0000', 'last_clean_scrub_stamp': '2024-04-07T16:12:11.994648+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-07T16:12:17.121 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.6 is not active+clean 2024-04-07T16:12:17.121 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-07T16:12:12.996208+0000', 'last_change': '2024-04-07T16:12:12.996208+0000', 'last_active': '2024-04-07T16:12:12.996208+0000', 'last_peered': '2024-04-07T16:12:12.996208+0000', 'last_clean': '2024-04-07T16:12:12.996208+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-07T16:12:12.996208+0000', 'last_undegraded': '2024-04-07T16:12:12.996208+0000', 'last_fullsized': '2024-04-07T16:12:12.996208+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-07T16:12:12.996208+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-07T16:12:12.996208+0000', 'last_clean_scrub_stamp': '2024-04-07T16:12:12.996208+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-07T16:12:17.121 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.9 is not active+clean 2024-04-07T16:12:17.121 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2024-04-07T16:12:13.001861+0000', 'last_change': '2024-04-07T16:12:13.001861+0000', 'last_active': '2024-04-07T16:12:12.996208+0000', 'last_peered': '2024-04-07T16:12:12.996208+0000', 'last_clean': '2024-04-07T16:12:12.996208+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-07T16:12:13.001861+0000', 'last_undegraded': '2024-04-07T16:12:13.001861+0000', 'last_fullsized': '2024-04-07T16:12:13.001861+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-07T16:12:12.996208+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-07T16:12:12.996208+0000', 'last_clean_scrub_stamp': '2024-04-07T16:12:12.996208+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-04-08T23:48:08.846232+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2, 7], 'acting': [4, 2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-07T16:12:17.121 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.9 is not active+clean 2024-04-07T16:12:17.121 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-07T16:12:11.994648+0000', 'last_change': '2024-04-07T16:12:11.994648+0000', 'last_active': '2024-04-07T16:12:11.994648+0000', 'last_peered': '2024-04-07T16:12:11.994648+0000', 'last_clean': '2024-04-07T16:12:11.994648+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-07T16:12:11.994648+0000', 'last_undegraded': '2024-04-07T16:12:11.994648+0000', 'last_fullsized': '2024-04-07T16:12:11.994648+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-07T16:12:11.994648+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-07T16:12:11.994648+0000', 'last_clean_scrub_stamp': '2024-04-07T16:12:11.994648+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-07T16:12:17.121 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.a is not active+clean 2024-04-07T16:12:17.121 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-07T16:12:11.994648+0000', 'last_change': '2024-04-07T16:12:11.994648+0000', 'last_active': '2024-04-07T16:12:11.994648+0000', 'last_peered': '2024-04-07T16:12:11.994648+0000', 'last_clean': '2024-04-07T16:12:11.994648+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-07T16:12:11.994648+0000', 'last_undegraded': '2024-04-07T16:12:11.994648+0000', 'last_fullsized': '2024-04-07T16:12:11.994648+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-07T16:12:11.994648+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-07T16:12:11.994648+0000', 'last_clean_scrub_stamp': '2024-04-07T16:12:11.994648+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-07T16:12:17.121 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.d is not active+clean 2024-04-07T16:12:17.122 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-07T16:12:11.994648+0000', 'last_change': '2024-04-07T16:12:11.994648+0000', 'last_active': '2024-04-07T16:12:11.994648+0000', 'last_peered': '2024-04-07T16:12:11.994648+0000', 'last_clean': '2024-04-07T16:12:11.994648+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-07T16:12:11.994648+0000', 'last_undegraded': '2024-04-07T16:12:11.994648+0000', 'last_fullsized': '2024-04-07T16:12:11.994648+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-07T16:12:11.994648+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-07T16:12:11.994648+0000', 'last_clean_scrub_stamp': '2024-04-07T16:12:11.994648+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-07T16:12:17.122 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.c is not active+clean 2024-04-07T16:12:17.122 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-07T16:12:12.996208+0000', 'last_change': '2024-04-07T16:12:12.996208+0000', 'last_active': '2024-04-07T16:12:12.996208+0000', 'last_peered': '2024-04-07T16:12:12.996208+0000', 'last_clean': '2024-04-07T16:12:12.996208+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-07T16:12:12.996208+0000', 'last_undegraded': '2024-04-07T16:12:12.996208+0000', 'last_fullsized': '2024-04-07T16:12:12.996208+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-07T16:12:12.996208+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-07T16:12:12.996208+0000', 'last_clean_scrub_stamp': '2024-04-07T16:12:12.996208+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-07T16:12:17.122 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.22 is not active+clean 2024-04-07T16:12:17.122 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-07T16:12:12.996208+0000', 'last_change': '2024-04-07T16:12:12.996208+0000', 'last_active': '2024-04-07T16:12:12.996208+0000', 'last_peered': '2024-04-07T16:12:12.996208+0000', 'last_clean': '2024-04-07T16:12:12.996208+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-07T16:12:12.996208+0000', 'last_undegraded': '2024-04-07T16:12:12.996208+0000', 'last_fullsized': '2024-04-07T16:12:12.996208+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-07T16:12:12.996208+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-07T16:12:12.996208+0000', 'last_clean_scrub_stamp': '2024-04-07T16:12:12.996208+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-07T16:12:17.122 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.23 is not active+clean 2024-04-07T16:12:17.122 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-07T16:12:12.996208+0000', 'last_change': '2024-04-07T16:12:12.996208+0000', 'last_active': '2024-04-07T16:12:12.996208+0000', 'last_peered': '2024-04-07T16:12:12.996208+0000', 'last_clean': '2024-04-07T16:12:12.996208+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-07T16:12:12.996208+0000', 'last_undegraded': '2024-04-07T16:12:12.996208+0000', 'last_fullsized': '2024-04-07T16:12:12.996208+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-07T16:12:12.996208+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-07T16:12:12.996208+0000', 'last_clean_scrub_stamp': '2024-04-07T16:12:12.996208+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-07T16:12:17.122 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.25 is not active+clean 2024-04-07T16:12:17.122 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-07T16:12:12.996208+0000', 'last_change': '2024-04-07T16:12:12.996208+0000', 'last_active': '2024-04-07T16:12:12.996208+0000', 'last_peered': '2024-04-07T16:12:12.996208+0000', 'last_clean': '2024-04-07T16:12:12.996208+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-07T16:12:12.996208+0000', 'last_undegraded': '2024-04-07T16:12:12.996208+0000', 'last_fullsized': '2024-04-07T16:12:12.996208+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-07T16:12:12.996208+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-07T16:12:12.996208+0000', 'last_clean_scrub_stamp': '2024-04-07T16:12:12.996208+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-07T16:12:17.122 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.28 is not active+clean 2024-04-07T16:12:17.123 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-07T16:12:12.996208+0000', 'last_change': '2024-04-07T16:12:12.996208+0000', 'last_active': '2024-04-07T16:12:12.996208+0000', 'last_peered': '2024-04-07T16:12:12.996208+0000', 'last_clean': '2024-04-07T16:12:12.996208+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-07T16:12:12.996208+0000', 'last_undegraded': '2024-04-07T16:12:12.996208+0000', 'last_fullsized': '2024-04-07T16:12:12.996208+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-07T16:12:12.996208+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-07T16:12:12.996208+0000', 'last_clean_scrub_stamp': '2024-04-07T16:12:12.996208+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-07T16:12:17.123 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.2a is not active+clean 2024-04-07T16:12:17.123 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-07T16:12:12.996208+0000', 'last_change': '2024-04-07T16:12:12.996208+0000', 'last_active': '2024-04-07T16:12:12.996208+0000', 'last_peered': '2024-04-07T16:12:12.996208+0000', 'last_clean': '2024-04-07T16:12:12.996208+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-07T16:12:12.996208+0000', 'last_undegraded': '2024-04-07T16:12:12.996208+0000', 'last_fullsized': '2024-04-07T16:12:12.996208+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-07T16:12:12.996208+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-07T16:12:12.996208+0000', 'last_clean_scrub_stamp': '2024-04-07T16:12:12.996208+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-07T16:12:17.123 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.2b is not active+clean 2024-04-07T16:12:17.123 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-07T16:12:12.996208+0000', 'last_change': '2024-04-07T16:12:12.996208+0000', 'last_active': '2024-04-07T16:12:12.996208+0000', 'last_peered': '2024-04-07T16:12:12.996208+0000', 'last_clean': '2024-04-07T16:12:12.996208+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-07T16:12:12.996208+0000', 'last_undegraded': '2024-04-07T16:12:12.996208+0000', 'last_fullsized': '2024-04-07T16:12:12.996208+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-07T16:12:12.996208+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-07T16:12:12.996208+0000', 'last_clean_scrub_stamp': '2024-04-07T16:12:12.996208+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-07T16:12:17.123 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-07T16:12:17.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:17 smithi144 bash[23522]: audit 2024-04-07T16:12:16.034580+0000 mon.c (mon.2) 29 : audit 0 from='client.? 172.21.15.100:0/4169123980' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-07T16:12:17.440 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:17 smithi100 bash[21346]: audit 2024-04-07T16:12:16.034580+0000 mon.c (mon.2) 29 : audit 0 from='client.? 172.21.15.100:0/4169123980' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-07T16:12:17.440 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:17 smithi100 bash[31660]: audit 2024-04-07T16:12:16.034580+0000 mon.c (mon.2) 29 : audit 0 from='client.? 172.21.15.100:0/4169123980' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-07T16:12:17.521 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-07T16:12:17.521 INFO:teuthology.orchestra.run.smithi100.stdout:{"fsid":"0702e3a8-f4f8-11ee-b649-cb9ed24678a4","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":14,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":457,"monmap":{"epoch":3,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":62,"num_osds":8,"num_up_osds":8,"osd_up_since":1712506290,"num_in_osds":8,"osd_in_since":1712506264,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":50},{"state_name":"unknown","count":34},{"state_name":"creating+peering","count":12},{"state_name":"creating+activating","count":1}],"num_pgs":97,"num_pools":3,"num_objects":15,"data_bytes":591932,"bytes_used":236396544,"bytes_avail":767723888640,"bytes_total":767960285184,"unknown_pgs_ratio":0.3505154550075531,"inactive_pgs_ratio":0.1340206116437912,"write_bytes_sec":715,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":6,"id":1,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":1,"rank":0,"name":"b","status":"up:active","gid":24322}],"up:standby":1},"mgrmap":{"available":true,"num_standbys":1,"modules":["cephadm","dashboard","iostat","nfs","restful"],"services":{"dashboard":"https://172.21.15.100:8443/"}},"servicemap":{"epoch":12,"modified":"2024-04-07T16:11:40.277722+0000","services":{"osd":{"daemons":{"summary":"","7":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-07T16:12:17.540 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-07T16:12:17.540 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-07T16:12:17.830 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-07T16:12:17.833 INFO:teuthology.orchestra.run.smithi100.stderr:dumped all 2024-04-07T16:12:17.850 INFO:teuthology.orchestra.run.smithi100.stdout:{"pg_ready":true,"pg_map":{"version":318,"stamp":"2024-04-07T16:12:16.287343+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591932,"num_objects":15,"num_object_clones":0,"num_object_copies":45,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":190,"num_write_kb":2419,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":144,"ondisk_log_size":144,"up":189,"acting":189,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":76,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":749961216,"kb_used":230856,"kb_used_data":6472,"kb_used_omap":12,"kb_used_meta":224243,"kb_avail":749730360,"statfs":{"total":767960285184,"available":767723888640,"internally_reserved":0,"allocated":6627328,"data_stored":3755083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":229625401},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1564,"num_objects":13,"num_object_clones":0,"num_object_copies":39,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.290644"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.376089+0000","last_change":"2024-04-07T16:12:13.395367+0000","last_active":"2024-04-07T16:12:14.376089+0000","last_peered":"2024-04-07T16:12:14.376089+0000","last_clean":"2024-04-07T16:12:14.376089+0000","last_became_active":"2024-04-07T16:12:13.395023+0000","last_became_peered":"2024-04-07T16:12:13.395023+0000","last_unstale":"2024-04-07T16:12:14.376089+0000","last_undegraded":"2024-04-07T16:12:14.376089+0000","last_fullsized":"2024-04-07T16:12:14.376089+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T23:05:09.752696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-07T16:12:12.996208+0000","last_change":"2024-04-07T16:12:12.996208+0000","last_active":"2024-04-07T16:12:12.996208+0000","last_peered":"2024-04-07T16:12:12.996208+0000","last_clean":"2024-04-07T16:12:12.996208+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:12.996208+0000","last_undegraded":"2024-04-07T16:12:12.996208+0000","last_fullsized":"2024-04-07T16:12:12.996208+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-07T16:12:12.996208+0000","last_change":"2024-04-07T16:12:12.996208+0000","last_active":"2024-04-07T16:12:12.996208+0000","last_peered":"2024-04-07T16:12:12.996208+0000","last_clean":"2024-04-07T16:12:12.996208+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:12.996208+0000","last_undegraded":"2024-04-07T16:12:12.996208+0000","last_fullsized":"2024-04-07T16:12:12.996208+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-07T16:12:12.996208+0000","last_change":"2024-04-07T16:12:12.996208+0000","last_active":"2024-04-07T16:12:12.996208+0000","last_peered":"2024-04-07T16:12:12.996208+0000","last_clean":"2024-04-07T16:12:12.996208+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:12.996208+0000","last_undegraded":"2024-04-07T16:12:12.996208+0000","last_fullsized":"2024-04-07T16:12:12.996208+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.375284+0000","last_change":"2024-04-07T16:12:13.413487+0000","last_active":"2024-04-07T16:12:14.375284+0000","last_peered":"2024-04-07T16:12:14.375284+0000","last_clean":"2024-04-07T16:12:14.375284+0000","last_became_active":"2024-04-07T16:12:13.413281+0000","last_became_peered":"2024-04-07T16:12:13.413281+0000","last_unstale":"2024-04-07T16:12:14.375284+0000","last_undegraded":"2024-04-07T16:12:14.375284+0000","last_fullsized":"2024-04-07T16:12:14.375284+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T23:59:23.302803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-07T16:12:12.996208+0000","last_change":"2024-04-07T16:12:12.996208+0000","last_active":"2024-04-07T16:12:12.996208+0000","last_peered":"2024-04-07T16:12:12.996208+0000","last_clean":"2024-04-07T16:12:12.996208+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:12.996208+0000","last_undegraded":"2024-04-07T16:12:12.996208+0000","last_fullsized":"2024-04-07T16:12:12.996208+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.374561+0000","last_change":"2024-04-07T16:12:13.393518+0000","last_active":"2024-04-07T16:12:14.374561+0000","last_peered":"2024-04-07T16:12:14.374561+0000","last_clean":"2024-04-07T16:12:14.374561+0000","last_became_active":"2024-04-07T16:12:13.393187+0000","last_became_peered":"2024-04-07T16:12:13.393187+0000","last_unstale":"2024-04-07T16:12:14.374561+0000","last_undegraded":"2024-04-07T16:12:14.374561+0000","last_fullsized":"2024-04-07T16:12:14.374561+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T19:41:28.702905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.475028+0000","last_change":"2024-04-07T16:12:13.394702+0000","last_active":"2024-04-07T16:12:14.475028+0000","last_peered":"2024-04-07T16:12:14.475028+0000","last_clean":"2024-04-07T16:12:14.475028+0000","last_became_active":"2024-04-07T16:12:13.394422+0000","last_became_peered":"2024-04-07T16:12:13.394422+0000","last_unstale":"2024-04-07T16:12:14.475028+0000","last_undegraded":"2024-04-07T16:12:14.475028+0000","last_fullsized":"2024-04-07T16:12:14.475028+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T00:37:44.512471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.375990+0000","last_change":"2024-04-07T16:12:13.401766+0000","last_active":"2024-04-07T16:12:14.375990+0000","last_peered":"2024-04-07T16:12:14.375990+0000","last_clean":"2024-04-07T16:12:14.375990+0000","last_became_active":"2024-04-07T16:12:13.401411+0000","last_became_peered":"2024-04-07T16:12:13.401411+0000","last_unstale":"2024-04-07T16:12:14.375990+0000","last_undegraded":"2024-04-07T16:12:14.375990+0000","last_fullsized":"2024-04-07T16:12:14.375990+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T23:32:52.283410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,0],"acting":[7,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.475526+0000","last_change":"2024-04-07T16:12:13.396409+0000","last_active":"2024-04-07T16:12:14.475526+0000","last_peered":"2024-04-07T16:12:14.475526+0000","last_clean":"2024-04-07T16:12:14.475526+0000","last_became_active":"2024-04-07T16:12:13.396094+0000","last_became_peered":"2024-04-07T16:12:13.396094+0000","last_unstale":"2024-04-07T16:12:14.475526+0000","last_undegraded":"2024-04-07T16:12:14.475526+0000","last_fullsized":"2024-04-07T16:12:14.475526+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T18:54:36.072837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.474549+0000","last_change":"2024-04-07T16:12:13.401973+0000","last_active":"2024-04-07T16:12:14.474549+0000","last_peered":"2024-04-07T16:12:14.474549+0000","last_clean":"2024-04-07T16:12:14.474549+0000","last_became_active":"2024-04-07T16:12:13.401724+0000","last_became_peered":"2024-04-07T16:12:13.401724+0000","last_unstale":"2024-04-07T16:12:14.474549+0000","last_undegraded":"2024-04-07T16:12:14.474549+0000","last_fullsized":"2024-04-07T16:12:14.474549+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T21:26:29.208223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,0],"acting":[6,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-04-07T16:12:13.005867+0000","last_change":"2024-04-07T16:12:13.005867+0000","last_active":"2024-04-07T16:12:12.996208+0000","last_peered":"2024-04-07T16:12:12.996208+0000","last_clean":"2024-04-07T16:12:12.996208+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:13.005867+0000","last_undegraded":"2024-04-07T16:12:13.005867+0000","last_fullsized":"2024-04-07T16:12:13.005867+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T22:29:22.509564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,0],"acting":[4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.474547+0000","last_change":"2024-04-07T16:12:13.394760+0000","last_active":"2024-04-07T16:12:14.474547+0000","last_peered":"2024-04-07T16:12:14.474547+0000","last_clean":"2024-04-07T16:12:14.474547+0000","last_became_active":"2024-04-07T16:12:13.394402+0000","last_became_peered":"2024-04-07T16:12:13.394402+0000","last_unstale":"2024-04-07T16:12:14.474547+0000","last_undegraded":"2024-04-07T16:12:14.474547+0000","last_fullsized":"2024-04-07T16:12:14.474547+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T18:47:13.891972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-07T16:12:12.996208+0000","last_change":"2024-04-07T16:12:12.996208+0000","last_active":"2024-04-07T16:12:12.996208+0000","last_peered":"2024-04-07T16:12:12.996208+0000","last_clean":"2024-04-07T16:12:12.996208+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:12.996208+0000","last_undegraded":"2024-04-07T16:12:12.996208+0000","last_fullsized":"2024-04-07T16:12:12.996208+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-07T16:12:12.996208+0000","last_change":"2024-04-07T16:12:12.996208+0000","last_active":"2024-04-07T16:12:12.996208+0000","last_peered":"2024-04-07T16:12:12.996208+0000","last_clean":"2024-04-07T16:12:12.996208+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:12.996208+0000","last_undegraded":"2024-04-07T16:12:12.996208+0000","last_fullsized":"2024-04-07T16:12:12.996208+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-07T16:12:12.996208+0000","last_change":"2024-04-07T16:12:12.996208+0000","last_active":"2024-04-07T16:12:12.996208+0000","last_peered":"2024-04-07T16:12:12.996208+0000","last_clean":"2024-04-07T16:12:12.996208+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:12.996208+0000","last_undegraded":"2024-04-07T16:12:12.996208+0000","last_fullsized":"2024-04-07T16:12:12.996208+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-07T16:12:12.996208+0000","last_change":"2024-04-07T16:12:12.996208+0000","last_active":"2024-04-07T16:12:12.996208+0000","last_peered":"2024-04-07T16:12:12.996208+0000","last_clean":"2024-04-07T16:12:12.996208+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:12.996208+0000","last_undegraded":"2024-04-07T16:12:12.996208+0000","last_fullsized":"2024-04-07T16:12:12.996208+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.474396+0000","last_change":"2024-04-07T16:12:13.416210+0000","last_active":"2024-04-07T16:12:14.474396+0000","last_peered":"2024-04-07T16:12:14.474396+0000","last_clean":"2024-04-07T16:12:14.474396+0000","last_became_active":"2024-04-07T16:12:13.415972+0000","last_became_peered":"2024-04-07T16:12:13.415972+0000","last_unstale":"2024-04-07T16:12:14.474396+0000","last_undegraded":"2024-04-07T16:12:14.474396+0000","last_fullsized":"2024-04-07T16:12:14.474396+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T23:40:11.606978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,7],"acting":[2,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.375192+0000","last_change":"2024-04-07T16:12:13.414203+0000","last_active":"2024-04-07T16:12:14.375192+0000","last_peered":"2024-04-07T16:12:14.375192+0000","last_clean":"2024-04-07T16:12:14.375192+0000","last_became_active":"2024-04-07T16:12:13.413869+0000","last_became_peered":"2024-04-07T16:12:13.413869+0000","last_unstale":"2024-04-07T16:12:14.375192+0000","last_undegraded":"2024-04-07T16:12:14.375192+0000","last_fullsized":"2024-04-07T16:12:14.375192+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T17:11:47.895568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,6],"acting":[7,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-04-07T16:12:12.000271+0000","last_change":"2024-04-07T16:12:12.000271+0000","last_active":"2024-04-07T16:12:11.994648+0000","last_peered":"2024-04-07T16:12:11.994648+0000","last_clean":"2024-04-07T16:12:11.994648+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:12.000271+0000","last_undegraded":"2024-04-07T16:12:12.000271+0000","last_fullsized":"2024-04-07T16:12:12.000271+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T19:01:44.798719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.474373+0000","last_change":"2024-04-07T16:12:13.407852+0000","last_active":"2024-04-07T16:12:14.474373+0000","last_peered":"2024-04-07T16:12:14.474373+0000","last_clean":"2024-04-07T16:12:14.474373+0000","last_became_active":"2024-04-07T16:12:13.407538+0000","last_became_peered":"2024-04-07T16:12:13.407538+0000","last_unstale":"2024-04-07T16:12:14.474373+0000","last_undegraded":"2024-04-07T16:12:14.474373+0000","last_fullsized":"2024-04-07T16:12:14.474373+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T20:27:52.664149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"61'1","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.474471+0000","last_change":"2024-04-07T16:12:13.038807+0000","last_active":"2024-04-07T16:12:14.474471+0000","last_peered":"2024-04-07T16:12:14.474471+0000","last_clean":"2024-04-07T16:12:14.474471+0000","last_became_active":"2024-04-07T16:12:13.038492+0000","last_became_peered":"2024-04-07T16:12:13.038492+0000","last_unstale":"2024-04-07T16:12:14.474471+0000","last_undegraded":"2024-04-07T16:12:14.474471+0000","last_fullsized":"2024-04-07T16:12:14.474471+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T02:11:03.746017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-07T16:12:12.996208+0000","last_change":"2024-04-07T16:12:12.996208+0000","last_active":"2024-04-07T16:12:12.996208+0000","last_peered":"2024-04-07T16:12:12.996208+0000","last_clean":"2024-04-07T16:12:12.996208+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:12.996208+0000","last_undegraded":"2024-04-07T16:12:12.996208+0000","last_fullsized":"2024-04-07T16:12:12.996208+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"61'2","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.375664+0000","last_change":"2024-04-07T16:12:13.042401+0000","last_active":"2024-04-07T16:12:14.375664+0000","last_peered":"2024-04-07T16:12:14.375664+0000","last_clean":"2024-04-07T16:12:14.375664+0000","last_became_active":"2024-04-07T16:12:13.041970+0000","last_became_peered":"2024-04-07T16:12:13.041970+0000","last_unstale":"2024-04-07T16:12:14.375664+0000","last_undegraded":"2024-04-07T16:12:14.375664+0000","last_fullsized":"2024-04-07T16:12:14.375664+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T16:36:29.235347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-07T16:12:12.996208+0000","last_change":"2024-04-07T16:12:12.996208+0000","last_active":"2024-04-07T16:12:12.996208+0000","last_peered":"2024-04-07T16:12:12.996208+0000","last_clean":"2024-04-07T16:12:12.996208+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:12.996208+0000","last_undegraded":"2024-04-07T16:12:12.996208+0000","last_fullsized":"2024-04-07T16:12:12.996208+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"61'2","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.375447+0000","last_change":"2024-04-07T16:12:13.042391+0000","last_active":"2024-04-07T16:12:14.375447+0000","last_peered":"2024-04-07T16:12:14.375447+0000","last_clean":"2024-04-07T16:12:14.375447+0000","last_became_active":"2024-04-07T16:12:13.041919+0000","last_became_peered":"2024-04-07T16:12:13.041919+0000","last_unstale":"2024-04-07T16:12:14.375447+0000","last_undegraded":"2024-04-07T16:12:14.375447+0000","last_fullsized":"2024-04-07T16:12:14.375447+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T21:50:59.576615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-07T16:12:12.996208+0000","last_change":"2024-04-07T16:12:12.996208+0000","last_active":"2024-04-07T16:12:12.996208+0000","last_peered":"2024-04-07T16:12:12.996208+0000","last_clean":"2024-04-07T16:12:12.996208+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:12.996208+0000","last_undegraded":"2024-04-07T16:12:12.996208+0000","last_fullsized":"2024-04-07T16:12:12.996208+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"61'1","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.474818+0000","last_change":"2024-04-07T16:12:13.041880+0000","last_active":"2024-04-07T16:12:14.474818+0000","last_peered":"2024-04-07T16:12:14.474818+0000","last_clean":"2024-04-07T16:12:14.474818+0000","last_became_active":"2024-04-07T16:12:13.041409+0000","last_became_peered":"2024-04-07T16:12:13.041409+0000","last_unstale":"2024-04-07T16:12:14.474818+0000","last_undegraded":"2024-04-07T16:12:14.474818+0000","last_fullsized":"2024-04-07T16:12:14.474818+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T18:32:23.206985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-04-07T16:12:13.002508+0000","last_change":"2024-04-07T16:12:13.002508+0000","last_active":"2024-04-07T16:12:12.996208+0000","last_peered":"2024-04-07T16:12:12.996208+0000","last_clean":"2024-04-07T16:12:12.996208+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:13.002508+0000","last_undegraded":"2024-04-07T16:12:13.002508+0000","last_fullsized":"2024-04-07T16:12:13.002508+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T23:20:40.949255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"61'1","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.476584+0000","last_change":"2024-04-07T16:12:13.038960+0000","last_active":"2024-04-07T16:12:14.476584+0000","last_peered":"2024-04-07T16:12:14.476584+0000","last_clean":"2024-04-07T16:12:14.476584+0000","last_became_active":"2024-04-07T16:12:13.038410+0000","last_became_peered":"2024-04-07T16:12:13.038410+0000","last_unstale":"2024-04-07T16:12:14.476584+0000","last_undegraded":"2024-04-07T16:12:14.476584+0000","last_fullsized":"2024-04-07T16:12:14.476584+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T03:41:38.418312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-07T16:12:12.996208+0000","last_change":"2024-04-07T16:12:12.996208+0000","last_active":"2024-04-07T16:12:12.996208+0000","last_peered":"2024-04-07T16:12:12.996208+0000","last_clean":"2024-04-07T16:12:12.996208+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:12.996208+0000","last_undegraded":"2024-04-07T16:12:12.996208+0000","last_fullsized":"2024-04-07T16:12:12.996208+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.475758+0000","last_change":"2024-04-07T16:12:13.034420+0000","last_active":"2024-04-07T16:12:14.475758+0000","last_peered":"2024-04-07T16:12:14.475758+0000","last_clean":"2024-04-07T16:12:14.475758+0000","last_became_active":"2024-04-07T16:12:13.034013+0000","last_became_peered":"2024-04-07T16:12:13.034013+0000","last_unstale":"2024-04-07T16:12:14.475758+0000","last_undegraded":"2024-04-07T16:12:14.475758+0000","last_fullsized":"2024-04-07T16:12:14.475758+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T19:51:21.525865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.474201+0000","last_change":"2024-04-07T16:12:13.398531+0000","last_active":"2024-04-07T16:12:14.474201+0000","last_peered":"2024-04-07T16:12:14.474201+0000","last_clean":"2024-04-07T16:12:14.474201+0000","last_became_active":"2024-04-07T16:12:13.398017+0000","last_became_peered":"2024-04-07T16:12:13.398017+0000","last_unstale":"2024-04-07T16:12:14.474201+0000","last_undegraded":"2024-04-07T16:12:14.474201+0000","last_fullsized":"2024-04-07T16:12:14.474201+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T17:33:54.306902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-04-07T16:12:12.000806+0000","last_change":"2024-04-07T16:12:12.000806+0000","last_active":"2024-04-07T16:12:11.994648+0000","last_peered":"2024-04-07T16:12:11.994648+0000","last_clean":"2024-04-07T16:12:11.994648+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:12.000806+0000","last_undegraded":"2024-04-07T16:12:12.000806+0000","last_fullsized":"2024-04-07T16:12:12.000806+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T22:44:14.623116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-07T16:12:12.996208+0000","last_change":"2024-04-07T16:12:12.996208+0000","last_active":"2024-04-07T16:12:12.996208+0000","last_peered":"2024-04-07T16:12:12.996208+0000","last_clean":"2024-04-07T16:12:12.996208+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:12.996208+0000","last_undegraded":"2024-04-07T16:12:12.996208+0000","last_fullsized":"2024-04-07T16:12:12.996208+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.474717+0000","last_change":"2024-04-07T16:12:13.030149+0000","last_active":"2024-04-07T16:12:14.474717+0000","last_peered":"2024-04-07T16:12:14.474717+0000","last_clean":"2024-04-07T16:12:14.474717+0000","last_became_active":"2024-04-07T16:12:13.029525+0000","last_became_peered":"2024-04-07T16:12:13.029525+0000","last_unstale":"2024-04-07T16:12:14.474717+0000","last_undegraded":"2024-04-07T16:12:14.474717+0000","last_fullsized":"2024-04-07T16:12:14.474717+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T20:09:35.180264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-07T16:12:12.996208+0000","last_change":"2024-04-07T16:12:12.996208+0000","last_active":"2024-04-07T16:12:12.996208+0000","last_peered":"2024-04-07T16:12:12.996208+0000","last_clean":"2024-04-07T16:12:12.996208+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:12.996208+0000","last_undegraded":"2024-04-07T16:12:12.996208+0000","last_fullsized":"2024-04-07T16:12:12.996208+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-04-07T16:12:12.001015+0000","last_change":"2024-04-07T16:12:12.001015+0000","last_active":"2024-04-07T16:12:11.994648+0000","last_peered":"2024-04-07T16:12:11.994648+0000","last_clean":"2024-04-07T16:12:11.994648+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:12.001015+0000","last_undegraded":"2024-04-07T16:12:12.001015+0000","last_fullsized":"2024-04-07T16:12:12.001015+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T18:26:16.873044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-07T16:12:12.996208+0000","last_change":"2024-04-07T16:12:12.996208+0000","last_active":"2024-04-07T16:12:12.996208+0000","last_peered":"2024-04-07T16:12:12.996208+0000","last_clean":"2024-04-07T16:12:12.996208+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:12.996208+0000","last_undegraded":"2024-04-07T16:12:12.996208+0000","last_fullsized":"2024-04-07T16:12:12.996208+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-07T16:12:11.994648+0000","last_change":"2024-04-07T16:12:11.994648+0000","last_active":"2024-04-07T16:12:11.994648+0000","last_peered":"2024-04-07T16:12:11.994648+0000","last_clean":"2024-04-07T16:12:11.994648+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:11.994648+0000","last_undegraded":"2024-04-07T16:12:11.994648+0000","last_fullsized":"2024-04-07T16:12:11.994648+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-04-07T16:12:13.004673+0000","last_change":"2024-04-07T16:12:13.004673+0000","last_active":"2024-04-07T16:12:12.996208+0000","last_peered":"2024-04-07T16:12:12.996208+0000","last_clean":"2024-04-07T16:12:12.996208+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:13.004673+0000","last_undegraded":"2024-04-07T16:12:13.004673+0000","last_fullsized":"2024-04-07T16:12:13.004673+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T16:55:37.429799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"61'6","reported_seq":18,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.473934+0000","last_change":"2024-04-07T16:12:13.031673+0000","last_active":"2024-04-07T16:12:14.473934+0000","last_peered":"2024-04-07T16:12:14.473934+0000","last_clean":"2024-04-07T16:12:14.473934+0000","last_became_active":"2024-04-07T16:12:13.031092+0000","last_became_peered":"2024-04-07T16:12:13.031092+0000","last_unstale":"2024-04-07T16:12:14.473934+0000","last_undegraded":"2024-04-07T16:12:14.473934+0000","last_fullsized":"2024-04-07T16:12:14.473934+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T20:17:48.593862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.375355+0000","last_change":"2024-04-07T16:12:13.390083+0000","last_active":"2024-04-07T16:12:14.375355+0000","last_peered":"2024-04-07T16:12:14.375355+0000","last_clean":"2024-04-07T16:12:14.375355+0000","last_became_active":"2024-04-07T16:12:13.389755+0000","last_became_peered":"2024-04-07T16:12:13.389755+0000","last_unstale":"2024-04-07T16:12:14.375355+0000","last_undegraded":"2024-04-07T16:12:14.375355+0000","last_fullsized":"2024-04-07T16:12:14.375355+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T02:31:09.155569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-04-07T16:12:12.001190+0000","last_change":"2024-04-07T16:12:12.001190+0000","last_active":"2024-04-07T16:12:11.994648+0000","last_peered":"2024-04-07T16:12:11.994648+0000","last_clean":"2024-04-07T16:12:11.994648+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:12.001190+0000","last_undegraded":"2024-04-07T16:12:12.001190+0000","last_fullsized":"2024-04-07T16:12:12.001190+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T01:50:02.195507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-07T16:12:12.996208+0000","last_change":"2024-04-07T16:12:12.996208+0000","last_active":"2024-04-07T16:12:12.996208+0000","last_peered":"2024-04-07T16:12:12.996208+0000","last_clean":"2024-04-07T16:12:12.996208+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:12.996208+0000","last_undegraded":"2024-04-07T16:12:12.996208+0000","last_fullsized":"2024-04-07T16:12:12.996208+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-04-07T16:12:12.002099+0000","last_change":"2024-04-07T16:12:12.002099+0000","last_active":"2024-04-07T16:12:11.994648+0000","last_peered":"2024-04-07T16:12:11.994648+0000","last_clean":"2024-04-07T16:12:11.994648+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:12.002099+0000","last_undegraded":"2024-04-07T16:12:12.002099+0000","last_fullsized":"2024-04-07T16:12:12.002099+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T23:15:11.494437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.375013+0000","last_change":"2024-04-07T16:12:13.413928+0000","last_active":"2024-04-07T16:12:14.375013+0000","last_peered":"2024-04-07T16:12:14.375013+0000","last_clean":"2024-04-07T16:12:14.375013+0000","last_became_active":"2024-04-07T16:12:13.413447+0000","last_became_peered":"2024-04-07T16:12:13.413447+0000","last_unstale":"2024-04-07T16:12:14.375013+0000","last_undegraded":"2024-04-07T16:12:14.375013+0000","last_fullsized":"2024-04-07T16:12:14.375013+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T03:26:42.875228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.473099+0000","last_change":"2024-04-07T16:12:13.032942+0000","last_active":"2024-04-07T16:12:14.473099+0000","last_peered":"2024-04-07T16:12:14.473099+0000","last_clean":"2024-04-07T16:12:14.473099+0000","last_became_active":"2024-04-07T16:12:13.032108+0000","last_became_peered":"2024-04-07T16:12:13.032108+0000","last_unstale":"2024-04-07T16:12:14.473099+0000","last_undegraded":"2024-04-07T16:12:14.473099+0000","last_fullsized":"2024-04-07T16:12:14.473099+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T18:53:06.864412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.473331+0000","last_change":"2024-04-07T16:12:13.400141+0000","last_active":"2024-04-07T16:12:14.473331+0000","last_peered":"2024-04-07T16:12:14.473331+0000","last_clean":"2024-04-07T16:12:14.473331+0000","last_became_active":"2024-04-07T16:12:13.399823+0000","last_became_peered":"2024-04-07T16:12:13.399823+0000","last_unstale":"2024-04-07T16:12:14.473331+0000","last_undegraded":"2024-04-07T16:12:14.473331+0000","last_fullsized":"2024-04-07T16:12:14.473331+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T21:27:25.248595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"61'8","reported_seq":20,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.474350+0000","last_change":"2024-04-07T16:12:13.034154+0000","last_active":"2024-04-07T16:12:14.474350+0000","last_peered":"2024-04-07T16:12:14.474350+0000","last_clean":"2024-04-07T16:12:14.474350+0000","last_became_active":"2024-04-07T16:12:13.033838+0000","last_became_peered":"2024-04-07T16:12:13.033838+0000","last_unstale":"2024-04-07T16:12:14.474350+0000","last_undegraded":"2024-04-07T16:12:14.474350+0000","last_fullsized":"2024-04-07T16:12:14.474350+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T23:13:07.216314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.374716+0000","last_change":"2024-04-07T16:12:13.385883+0000","last_active":"2024-04-07T16:12:14.374716+0000","last_peered":"2024-04-07T16:12:14.374716+0000","last_clean":"2024-04-07T16:12:14.374716+0000","last_became_active":"2024-04-07T16:12:13.385485+0000","last_became_peered":"2024-04-07T16:12:13.385485+0000","last_unstale":"2024-04-07T16:12:14.374716+0000","last_undegraded":"2024-04-07T16:12:14.374716+0000","last_fullsized":"2024-04-07T16:12:14.374716+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T21:01:09.262433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":4,"reported_epoch":60,"state":"creating+activating","last_fresh":"2024-04-07T16:12:13.024777+0000","last_change":"2024-04-07T16:12:13.002417+0000","last_active":"2024-04-07T16:12:11.994648+0000","last_peered":"2024-04-07T16:12:11.994648+0000","last_clean":"2024-04-07T16:12:11.994648+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:13.024777+0000","last_undegraded":"2024-04-07T16:12:13.024777+0000","last_fullsized":"2024-04-07T16:12:13.024777+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T17:55:06.727142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":["4","0","7"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.375770+0000","last_change":"2024-04-07T16:12:13.386584+0000","last_active":"2024-04-07T16:12:14.375770+0000","last_peered":"2024-04-07T16:12:14.375770+0000","last_clean":"2024-04-07T16:12:14.375770+0000","last_became_active":"2024-04-07T16:12:13.386057+0000","last_became_peered":"2024-04-07T16:12:13.386057+0000","last_unstale":"2024-04-07T16:12:14.375770+0000","last_undegraded":"2024-04-07T16:12:14.375770+0000","last_fullsized":"2024-04-07T16:12:14.375770+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T17:57:57.548179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-07T16:12:11.994648+0000","last_change":"2024-04-07T16:12:11.994648+0000","last_active":"2024-04-07T16:12:11.994648+0000","last_peered":"2024-04-07T16:12:11.994648+0000","last_clean":"2024-04-07T16:12:11.994648+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:11.994648+0000","last_undegraded":"2024-04-07T16:12:11.994648+0000","last_fullsized":"2024-04-07T16:12:11.994648+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-07T16:12:12.996208+0000","last_change":"2024-04-07T16:12:12.996208+0000","last_active":"2024-04-07T16:12:12.996208+0000","last_peered":"2024-04-07T16:12:12.996208+0000","last_clean":"2024-04-07T16:12:12.996208+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:12.996208+0000","last_undegraded":"2024-04-07T16:12:12.996208+0000","last_fullsized":"2024-04-07T16:12:12.996208+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-04-07T16:12:12.001190+0000","last_change":"2024-04-07T16:12:12.001190+0000","last_active":"2024-04-07T16:12:11.994648+0000","last_peered":"2024-04-07T16:12:11.994648+0000","last_clean":"2024-04-07T16:12:11.994648+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:12.001190+0000","last_undegraded":"2024-04-07T16:12:12.001190+0000","last_fullsized":"2024-04-07T16:12:12.001190+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T18:15:00.506827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-04-07T16:12:13.003585+0000","last_change":"2024-04-07T16:12:13.003585+0000","last_active":"2024-04-07T16:12:12.996208+0000","last_peered":"2024-04-07T16:12:12.996208+0000","last_clean":"2024-04-07T16:12:12.996208+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:13.003585+0000","last_undegraded":"2024-04-07T16:12:13.003585+0000","last_fullsized":"2024-04-07T16:12:13.003585+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T03:36:24.354110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-07T16:12:12.996208+0000","last_change":"2024-04-07T16:12:12.996208+0000","last_active":"2024-04-07T16:12:12.996208+0000","last_peered":"2024-04-07T16:12:12.996208+0000","last_clean":"2024-04-07T16:12:12.996208+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:12.996208+0000","last_undegraded":"2024-04-07T16:12:12.996208+0000","last_fullsized":"2024-04-07T16:12:12.996208+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.474169+0000","last_change":"2024-04-07T16:12:13.035071+0000","last_active":"2024-04-07T16:12:14.474169+0000","last_peered":"2024-04-07T16:12:14.474169+0000","last_clean":"2024-04-07T16:12:14.474169+0000","last_became_active":"2024-04-07T16:12:13.034795+0000","last_became_peered":"2024-04-07T16:12:13.034795+0000","last_unstale":"2024-04-07T16:12:14.474169+0000","last_undegraded":"2024-04-07T16:12:14.474169+0000","last_fullsized":"2024-04-07T16:12:14.474169+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T17:51:30.599527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.476063+0000","last_change":"2024-04-07T16:12:13.394635+0000","last_active":"2024-04-07T16:12:14.476063+0000","last_peered":"2024-04-07T16:12:14.476063+0000","last_clean":"2024-04-07T16:12:14.476063+0000","last_became_active":"2024-04-07T16:12:13.394392+0000","last_became_peered":"2024-04-07T16:12:13.394392+0000","last_unstale":"2024-04-07T16:12:14.476063+0000","last_undegraded":"2024-04-07T16:12:14.476063+0000","last_fullsized":"2024-04-07T16:12:14.476063+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T17:52:10.950481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-04-07T16:12:12.001243+0000","last_change":"2024-04-07T16:12:12.001243+0000","last_active":"2024-04-07T16:12:11.994648+0000","last_peered":"2024-04-07T16:12:11.994648+0000","last_clean":"2024-04-07T16:12:11.994648+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:12.001243+0000","last_undegraded":"2024-04-07T16:12:12.001243+0000","last_fullsized":"2024-04-07T16:12:12.001243+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T02:48:50.652491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"21'117","reported_seq":219,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-07T16:12:13.375677+0000","last_change":"2024-04-07T16:11:33.518868+0000","last_active":"2024-04-07T16:12:13.375677+0000","last_peered":"2024-04-07T16:12:13.375677+0000","last_clean":"2024-04-07T16:12:13.375677+0000","last_became_active":"2024-04-07T16:11:33.518060+0000","last_became_peered":"2024-04-07T16:11:33.518060+0000","last_unstale":"2024-04-07T16:12:13.375677+0000","last_undegraded":"2024-04-07T16:12:13.375677+0000","last_fullsized":"2024-04-07T16:12:13.375677+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:07:26.866003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:07:26.866003+0000","last_clean_scrub_stamp":"2024-04-07T16:07:26.866003+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T22:22:51.043941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-07T16:12:12.996208+0000","last_change":"2024-04-07T16:12:12.996208+0000","last_active":"2024-04-07T16:12:12.996208+0000","last_peered":"2024-04-07T16:12:12.996208+0000","last_clean":"2024-04-07T16:12:12.996208+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:12.996208+0000","last_undegraded":"2024-04-07T16:12:12.996208+0000","last_fullsized":"2024-04-07T16:12:12.996208+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.375081+0000","last_change":"2024-04-07T16:12:13.035536+0000","last_active":"2024-04-07T16:12:14.375081+0000","last_peered":"2024-04-07T16:12:14.375081+0000","last_clean":"2024-04-07T16:12:14.375081+0000","last_became_active":"2024-04-07T16:12:13.034825+0000","last_became_peered":"2024-04-07T16:12:13.034825+0000","last_unstale":"2024-04-07T16:12:14.375081+0000","last_undegraded":"2024-04-07T16:12:14.375081+0000","last_fullsized":"2024-04-07T16:12:14.375081+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T20:11:57.033306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.375586+0000","last_change":"2024-04-07T16:12:13.035747+0000","last_active":"2024-04-07T16:12:14.375586+0000","last_peered":"2024-04-07T16:12:14.375586+0000","last_clean":"2024-04-07T16:12:14.375586+0000","last_became_active":"2024-04-07T16:12:13.035201+0000","last_became_peered":"2024-04-07T16:12:13.035201+0000","last_unstale":"2024-04-07T16:12:14.375586+0000","last_undegraded":"2024-04-07T16:12:14.375586+0000","last_fullsized":"2024-04-07T16:12:14.375586+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T21:03:25.449728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-07T16:12:12.996208+0000","last_change":"2024-04-07T16:12:12.996208+0000","last_active":"2024-04-07T16:12:12.996208+0000","last_peered":"2024-04-07T16:12:12.996208+0000","last_clean":"2024-04-07T16:12:12.996208+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:12.996208+0000","last_undegraded":"2024-04-07T16:12:12.996208+0000","last_fullsized":"2024-04-07T16:12:12.996208+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-07T16:12:11.994648+0000","last_change":"2024-04-07T16:12:11.994648+0000","last_active":"2024-04-07T16:12:11.994648+0000","last_peered":"2024-04-07T16:12:11.994648+0000","last_clean":"2024-04-07T16:12:11.994648+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:11.994648+0000","last_undegraded":"2024-04-07T16:12:11.994648+0000","last_fullsized":"2024-04-07T16:12:11.994648+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.474648+0000","last_change":"2024-04-07T16:12:13.399486+0000","last_active":"2024-04-07T16:12:14.474648+0000","last_peered":"2024-04-07T16:12:14.474648+0000","last_clean":"2024-04-07T16:12:14.474648+0000","last_became_active":"2024-04-07T16:12:13.399184+0000","last_became_peered":"2024-04-07T16:12:13.399184+0000","last_unstale":"2024-04-07T16:12:14.474648+0000","last_undegraded":"2024-04-07T16:12:14.474648+0000","last_fullsized":"2024-04-07T16:12:14.474648+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T21:07:25.104309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"61'2","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.474473+0000","last_change":"2024-04-07T16:12:13.038351+0000","last_active":"2024-04-07T16:12:14.474473+0000","last_peered":"2024-04-07T16:12:14.474473+0000","last_clean":"2024-04-07T16:12:14.474473+0000","last_became_active":"2024-04-07T16:12:13.038026+0000","last_became_peered":"2024-04-07T16:12:13.038026+0000","last_unstale":"2024-04-07T16:12:14.474473+0000","last_undegraded":"2024-04-07T16:12:14.474473+0000","last_fullsized":"2024-04-07T16:12:14.474473+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T19:39:42.796342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-07T16:12:12.996208+0000","last_change":"2024-04-07T16:12:12.996208+0000","last_active":"2024-04-07T16:12:12.996208+0000","last_peered":"2024-04-07T16:12:12.996208+0000","last_clean":"2024-04-07T16:12:12.996208+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:12.996208+0000","last_undegraded":"2024-04-07T16:12:12.996208+0000","last_fullsized":"2024-04-07T16:12:12.996208+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.374594+0000","last_change":"2024-04-07T16:12:13.033067+0000","last_active":"2024-04-07T16:12:14.374594+0000","last_peered":"2024-04-07T16:12:14.374594+0000","last_clean":"2024-04-07T16:12:14.374594+0000","last_became_active":"2024-04-07T16:12:13.032456+0000","last_became_peered":"2024-04-07T16:12:13.032456+0000","last_unstale":"2024-04-07T16:12:14.374594+0000","last_undegraded":"2024-04-07T16:12:14.374594+0000","last_fullsized":"2024-04-07T16:12:14.374594+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T23:02:09.025617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-04-07T16:12:13.001861+0000","last_change":"2024-04-07T16:12:13.001861+0000","last_active":"2024-04-07T16:12:12.996208+0000","last_peered":"2024-04-07T16:12:12.996208+0000","last_clean":"2024-04-07T16:12:12.996208+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:13.001861+0000","last_undegraded":"2024-04-07T16:12:13.001861+0000","last_fullsized":"2024-04-07T16:12:13.001861+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T23:48:08.846232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-07T16:12:11.994648+0000","last_change":"2024-04-07T16:12:11.994648+0000","last_active":"2024-04-07T16:12:11.994648+0000","last_peered":"2024-04-07T16:12:11.994648+0000","last_clean":"2024-04-07T16:12:11.994648+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:11.994648+0000","last_undegraded":"2024-04-07T16:12:11.994648+0000","last_fullsized":"2024-04-07T16:12:11.994648+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.474100+0000","last_change":"2024-04-07T16:12:13.392396+0000","last_active":"2024-04-07T16:12:14.474100+0000","last_peered":"2024-04-07T16:12:14.474100+0000","last_clean":"2024-04-07T16:12:14.474100+0000","last_became_active":"2024-04-07T16:12:13.392016+0000","last_became_peered":"2024-04-07T16:12:13.392016+0000","last_unstale":"2024-04-07T16:12:14.474100+0000","last_undegraded":"2024-04-07T16:12:14.474100+0000","last_fullsized":"2024-04-07T16:12:14.474100+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T02:35:50.790051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-07T16:12:11.994648+0000","last_change":"2024-04-07T16:12:11.994648+0000","last_active":"2024-04-07T16:12:11.994648+0000","last_peered":"2024-04-07T16:12:11.994648+0000","last_clean":"2024-04-07T16:12:11.994648+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:11.994648+0000","last_undegraded":"2024-04-07T16:12:11.994648+0000","last_fullsized":"2024-04-07T16:12:11.994648+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.474697+0000","last_change":"2024-04-07T16:12:13.397819+0000","last_active":"2024-04-07T16:12:14.474697+0000","last_peered":"2024-04-07T16:12:14.474697+0000","last_clean":"2024-04-07T16:12:14.474697+0000","last_became_active":"2024-04-07T16:12:13.397593+0000","last_became_peered":"2024-04-07T16:12:13.397593+0000","last_unstale":"2024-04-07T16:12:14.474697+0000","last_undegraded":"2024-04-07T16:12:14.474697+0000","last_fullsized":"2024-04-07T16:12:14.474697+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T02:30:01.472030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"61'2","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.375041+0000","last_change":"2024-04-07T16:12:13.043982+0000","last_active":"2024-04-07T16:12:14.375041+0000","last_peered":"2024-04-07T16:12:14.375041+0000","last_clean":"2024-04-07T16:12:14.375041+0000","last_became_active":"2024-04-07T16:12:13.041873+0000","last_became_peered":"2024-04-07T16:12:13.041873+0000","last_unstale":"2024-04-07T16:12:14.375041+0000","last_undegraded":"2024-04-07T16:12:14.375041+0000","last_fullsized":"2024-04-07T16:12:14.375041+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T20:03:35.575133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.476449+0000","last_change":"2024-04-07T16:12:13.381810+0000","last_active":"2024-04-07T16:12:14.476449+0000","last_peered":"2024-04-07T16:12:14.476449+0000","last_clean":"2024-04-07T16:12:14.476449+0000","last_became_active":"2024-04-07T16:12:13.381470+0000","last_became_peered":"2024-04-07T16:12:13.381470+0000","last_unstale":"2024-04-07T16:12:14.476449+0000","last_undegraded":"2024-04-07T16:12:14.476449+0000","last_fullsized":"2024-04-07T16:12:14.476449+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T17:26:21.450939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.474605+0000","last_change":"2024-04-07T16:12:13.038191+0000","last_active":"2024-04-07T16:12:14.474605+0000","last_peered":"2024-04-07T16:12:14.474605+0000","last_clean":"2024-04-07T16:12:14.474605+0000","last_became_active":"2024-04-07T16:12:13.037652+0000","last_became_peered":"2024-04-07T16:12:13.037652+0000","last_unstale":"2024-04-07T16:12:14.474605+0000","last_undegraded":"2024-04-07T16:12:14.474605+0000","last_fullsized":"2024-04-07T16:12:14.474605+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T17:04:59.525025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.375473+0000","last_change":"2024-04-07T16:12:13.400660+0000","last_active":"2024-04-07T16:12:14.375473+0000","last_peered":"2024-04-07T16:12:14.375473+0000","last_clean":"2024-04-07T16:12:14.375473+0000","last_became_active":"2024-04-07T16:12:13.400353+0000","last_became_peered":"2024-04-07T16:12:13.400353+0000","last_unstale":"2024-04-07T16:12:14.375473+0000","last_undegraded":"2024-04-07T16:12:14.375473+0000","last_fullsized":"2024-04-07T16:12:14.375473+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T00:47:57.573097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-07T16:12:11.994648+0000","last_change":"2024-04-07T16:12:11.994648+0000","last_active":"2024-04-07T16:12:11.994648+0000","last_peered":"2024-04-07T16:12:11.994648+0000","last_clean":"2024-04-07T16:12:11.994648+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:11.994648+0000","last_undegraded":"2024-04-07T16:12:11.994648+0000","last_fullsized":"2024-04-07T16:12:11.994648+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-07T16:12:12.996208+0000","last_change":"2024-04-07T16:12:12.996208+0000","last_active":"2024-04-07T16:12:12.996208+0000","last_peered":"2024-04-07T16:12:12.996208+0000","last_clean":"2024-04-07T16:12:12.996208+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:12.996208+0000","last_undegraded":"2024-04-07T16:12:12.996208+0000","last_fullsized":"2024-04-07T16:12:12.996208+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"61'2","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.474501+0000","last_change":"2024-04-07T16:12:13.040101+0000","last_active":"2024-04-07T16:12:14.474501+0000","last_peered":"2024-04-07T16:12:14.474501+0000","last_clean":"2024-04-07T16:12:14.474501+0000","last_became_active":"2024-04-07T16:12:13.039797+0000","last_became_peered":"2024-04-07T16:12:13.039797+0000","last_unstale":"2024-04-07T16:12:14.474501+0000","last_undegraded":"2024-04-07T16:12:14.474501+0000","last_fullsized":"2024-04-07T16:12:14.474501+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T03:27:06.016325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.376139+0000","last_change":"2024-04-07T16:12:13.394720+0000","last_active":"2024-04-07T16:12:14.376139+0000","last_peered":"2024-04-07T16:12:14.376139+0000","last_clean":"2024-04-07T16:12:14.376139+0000","last_became_active":"2024-04-07T16:12:13.394090+0000","last_became_peered":"2024-04-07T16:12:13.394090+0000","last_unstale":"2024-04-07T16:12:14.376139+0000","last_undegraded":"2024-04-07T16:12:14.376139+0000","last_fullsized":"2024-04-07T16:12:14.376139+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T00:47:54.579168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.473133+0000","last_change":"2024-04-07T16:12:13.396725+0000","last_active":"2024-04-07T16:12:14.473133+0000","last_peered":"2024-04-07T16:12:14.473133+0000","last_clean":"2024-04-07T16:12:14.473133+0000","last_became_active":"2024-04-07T16:12:13.395978+0000","last_became_peered":"2024-04-07T16:12:13.395978+0000","last_unstale":"2024-04-07T16:12:14.473133+0000","last_undegraded":"2024-04-07T16:12:14.473133+0000","last_fullsized":"2024-04-07T16:12:14.473133+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T21:30:43.276569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.474831+0000","last_change":"2024-04-07T16:12:13.412091+0000","last_active":"2024-04-07T16:12:14.474831+0000","last_peered":"2024-04-07T16:12:14.474831+0000","last_clean":"2024-04-07T16:12:14.474831+0000","last_became_active":"2024-04-07T16:12:13.410814+0000","last_became_peered":"2024-04-07T16:12:13.410814+0000","last_unstale":"2024-04-07T16:12:14.474831+0000","last_undegraded":"2024-04-07T16:12:14.474831+0000","last_fullsized":"2024-04-07T16:12:14.474831+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T01:48:29.034291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-07T16:12:12.996208+0000","last_change":"2024-04-07T16:12:12.996208+0000","last_active":"2024-04-07T16:12:12.996208+0000","last_peered":"2024-04-07T16:12:12.996208+0000","last_clean":"2024-04-07T16:12:12.996208+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:12.996208+0000","last_undegraded":"2024-04-07T16:12:12.996208+0000","last_fullsized":"2024-04-07T16:12:12.996208+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-07T16:12:12.996208+0000","last_change":"2024-04-07T16:12:12.996208+0000","last_active":"2024-04-07T16:12:12.996208+0000","last_peered":"2024-04-07T16:12:12.996208+0000","last_clean":"2024-04-07T16:12:12.996208+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:12.996208+0000","last_undegraded":"2024-04-07T16:12:12.996208+0000","last_fullsized":"2024-04-07T16:12:12.996208+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.475244+0000","last_change":"2024-04-07T16:12:13.398730+0000","last_active":"2024-04-07T16:12:14.475244+0000","last_peered":"2024-04-07T16:12:14.475244+0000","last_clean":"2024-04-07T16:12:14.475244+0000","last_became_active":"2024-04-07T16:12:13.398259+0000","last_became_peered":"2024-04-07T16:12:13.398259+0000","last_unstale":"2024-04-07T16:12:14.475244+0000","last_undegraded":"2024-04-07T16:12:14.475244+0000","last_fullsized":"2024-04-07T16:12:14.475244+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T02:14:28.915631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,0],"acting":[3,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-07T16:12:12.996208+0000","last_change":"2024-04-07T16:12:12.996208+0000","last_active":"2024-04-07T16:12:12.996208+0000","last_peered":"2024-04-07T16:12:12.996208+0000","last_clean":"2024-04-07T16:12:12.996208+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:12.996208+0000","last_undegraded":"2024-04-07T16:12:12.996208+0000","last_fullsized":"2024-04-07T16:12:12.996208+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.475770+0000","last_change":"2024-04-07T16:12:13.382605+0000","last_active":"2024-04-07T16:12:14.475770+0000","last_peered":"2024-04-07T16:12:14.475770+0000","last_clean":"2024-04-07T16:12:14.475770+0000","last_became_active":"2024-04-07T16:12:13.382267+0000","last_became_peered":"2024-04-07T16:12:13.382267+0000","last_unstale":"2024-04-07T16:12:14.475770+0000","last_undegraded":"2024-04-07T16:12:14.475770+0000","last_fullsized":"2024-04-07T16:12:14.475770+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T20:14:03.942753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,4],"acting":[6,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.474734+0000","last_change":"2024-04-07T16:12:13.405175+0000","last_active":"2024-04-07T16:12:14.474734+0000","last_peered":"2024-04-07T16:12:14.474734+0000","last_clean":"2024-04-07T16:12:14.474734+0000","last_became_active":"2024-04-07T16:12:13.404734+0000","last_became_peered":"2024-04-07T16:12:13.404734+0000","last_unstale":"2024-04-07T16:12:14.474734+0000","last_undegraded":"2024-04-07T16:12:14.474734+0000","last_fullsized":"2024-04-07T16:12:14.474734+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T17:55:23.469795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-07T16:12:12.996208+0000","last_change":"2024-04-07T16:12:12.996208+0000","last_active":"2024-04-07T16:12:12.996208+0000","last_peered":"2024-04-07T16:12:12.996208+0000","last_clean":"2024-04-07T16:12:12.996208+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:12.996208+0000","last_undegraded":"2024-04-07T16:12:12.996208+0000","last_fullsized":"2024-04-07T16:12:12.996208+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.475578+0000","last_change":"2024-04-07T16:12:13.397638+0000","last_active":"2024-04-07T16:12:14.475578+0000","last_peered":"2024-04-07T16:12:14.475578+0000","last_clean":"2024-04-07T16:12:14.475578+0000","last_became_active":"2024-04-07T16:12:13.396766+0000","last_became_peered":"2024-04-07T16:12:13.396766+0000","last_unstale":"2024-04-07T16:12:14.475578+0000","last_undegraded":"2024-04-07T16:12:14.475578+0000","last_fullsized":"2024-04-07T16:12:14.475578+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T22:51:06.841718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-07T16:12:12.996208+0000","last_change":"2024-04-07T16:12:12.996208+0000","last_active":"2024-04-07T16:12:12.996208+0000","last_peered":"2024-04-07T16:12:12.996208+0000","last_clean":"2024-04-07T16:12:12.996208+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:12.996208+0000","last_undegraded":"2024-04-07T16:12:12.996208+0000","last_fullsized":"2024-04-07T16:12:12.996208+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-07T16:12:12.996208+0000","last_change":"2024-04-07T16:12:12.996208+0000","last_active":"2024-04-07T16:12:12.996208+0000","last_peered":"2024-04-07T16:12:12.996208+0000","last_clean":"2024-04-07T16:12:12.996208+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-07T16:12:12.996208+0000","last_undegraded":"2024-04-07T16:12:12.996208+0000","last_fullsized":"2024-04-07T16:12:12.996208+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.474174+0000","last_change":"2024-04-07T16:12:13.380077+0000","last_active":"2024-04-07T16:12:14.474174+0000","last_peered":"2024-04-07T16:12:14.474174+0000","last_clean":"2024-04-07T16:12:14.474174+0000","last_became_active":"2024-04-07T16:12:13.379758+0000","last_became_peered":"2024-04-07T16:12:13.379758+0000","last_unstale":"2024-04-07T16:12:14.474174+0000","last_undegraded":"2024-04-07T16:12:14.474174+0000","last_fullsized":"2024-04-07T16:12:14.474174+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T22:47:34.836189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":108,"acting":108,"num_store_stats":5},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":7},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1564,"num_objects":13,"num_object_clones":0,"num_object_copies":39,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":27,"ondisk_log_size":27,"up":78,"acting":78,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":54,"seq":231928233996,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28224,"kb_used_data":1196,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716928,"statfs":{"total":95995035648,"available":95966134272,"internally_reserved":0,"allocated":1224704,"data_stored":850413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.212}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.55}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79600000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0740000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1339999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7290000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4510000000000001}]}]},{"osd":6,"up_from":46,"seq":197568495637,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28224,"kb_used_data":1200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716928,"statfs":{"total":95995035648,"available":95966134272,"internally_reserved":0,"allocated":1228800,"data_stored":851373,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Apr 7 16:11:42 2024","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.27,"5min":1.27,"15min":1.27},"last":3.4079999999999999},{"interface":"front","average":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":1.2849999999999999,"5min":1.2849999999999999,"15min":1.2849999999999999},"last":4.165}]},{"osd":1,"last update":"Sun Apr 7 16:11:42 2024","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.427,"5min":1.427,"15min":1.427},"last":1.8799999999999999},{"interface":"front","average":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.2470000000000001,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":3.4889999999999999}]},{"osd":2,"last update":"Sun Apr 7 16:11:42 2024","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.359,"5min":1.359,"15min":1.359},"last":4.218},{"interface":"front","average":{"1min":0.999,"5min":0.999,"15min":0.999},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.5029999999999999,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":1.0529999999999999}]},{"osd":3,"last update":"Sun Apr 7 16:11:42 2024","interfaces":[{"interface":"back","average":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.363,"5min":1.363,"15min":1.363},"last":4.2809999999999997},{"interface":"front","average":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.2589999999999999,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":3.3079999999999998}]},{"osd":4,"last update":"Sun Apr 7 16:11:42 2024","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.5609999999999999,"5min":1.5609999999999999,"15min":1.5609999999999999},"last":1.216},{"interface":"front","average":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.2809999999999999,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":4.8209999999999997}]},{"osd":5,"last update":"Sun Apr 7 16:11:42 2024","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.3680000000000001,"5min":1.3680000000000001,"15min":1.3680000000000001},"last":1.974},{"interface":"front","average":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.6279999999999999,"5min":1.6279999999999999,"15min":1.6279999999999999},"last":1.1419999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":5.2350000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574916,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27588,"kb_used_data":560,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717564,"statfs":{"total":95995035648,"available":95966785536,"internally_reserved":0,"allocated":573440,"data_stored":232554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Apr 7 16:11:48 2024","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.66300000000000003,"15min":0.57899999999999996},"min":{"1min":0.49299999999999999,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":1.518,"5min":1.518,"15min":1.518},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.871,"5min":0.74299999999999999,"15min":0.629},"min":{"1min":0.59899999999999998,"5min":0.315,"15min":0.315},"max":{"1min":1.2969999999999999,"5min":2.8620000000000001,"15min":2.8620000000000001},"last":0.66600000000000004}]},{"osd":2,"last update":"Sun Apr 7 16:11:31 2024","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.73399999999999999,"15min":0.66000000000000003},"min":{"1min":0.56599999999999995,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.345,"5min":1.345,"15min":1.345},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.71099999999999997,"15min":0.65700000000000003},"min":{"1min":0.60399999999999998,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.5369999999999999,"5min":1.5369999999999999,"15min":1.5369999999999999},"last":0.623}]},{"osd":3,"last update":"Sun Apr 7 16:11:23 2024","interfaces":[{"interface":"back","average":{"1min":1.032,"5min":0.86699999999999999,"15min":0.82299999999999995},"min":{"1min":0.64400000000000002,"5min":0.628,"15min":0.628},"max":{"1min":1.343,"5min":1.343,"15min":1.343},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.91700000000000004,"5min":0.79800000000000004,"15min":0.76200000000000001},"min":{"1min":0.66700000000000004,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.4770000000000001,"5min":1.4770000000000001,"15min":1.4770000000000001},"last":0.56299999999999994}]},{"osd":4,"last update":"Sun Apr 7 16:11:11 2024","interfaces":[{"interface":"back","average":{"1min":0.94999999999999996,"5min":0.93200000000000005,"15min":0.92900000000000005},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.5149999999999999,"5min":1.5149999999999999,"15min":1.5149999999999999},"last":1.099},{"interface":"front","average":{"1min":1.119,"5min":1.056,"15min":1.046},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.9299999999999999,"5min":1.9299999999999999,"15min":1.9299999999999999},"last":0.877}]},{"osd":5,"last update":"Sun Apr 7 16:11:55 2024","interfaces":[{"interface":"back","average":{"1min":1.151,"5min":1.23,"15min":1.2430000000000001},"min":{"1min":0.85199999999999998,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":1.6870000000000001,"5min":2.0510000000000002,"15min":2.0510000000000002},"last":0.93600000000000005},{"interface":"front","average":{"1min":1.099,"5min":1.1339999999999999,"15min":1.1399999999999999},"min":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"max":{"1min":1.577,"5min":1.8859999999999999,"15min":1.8859999999999999},"last":0.96199999999999997}]},{"osd":6,"last update":"Sun Apr 7 16:11:43 2024","interfaces":[{"interface":"back","average":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"min":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"max":{"1min":1.466,"5min":1.466,"15min":1.466},"last":1.1799999999999999},{"interface":"front","average":{"1min":1.274,"5min":1.274,"15min":1.274},"min":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"max":{"1min":1.6040000000000001,"5min":1.6040000000000001,"15min":1.6040000000000001},"last":1.198}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0269999999999999}]}]},{"osd":0,"up_from":8,"seq":34359738447,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28172,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716980,"statfs":{"total":95995035648,"available":95966187520,"internally_reserved":0,"allocated":1179648,"data_stored":829169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Apr 7 16:11:47 2024","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.63600000000000001,"15min":0.56999999999999995},"min":{"1min":0.56000000000000005,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":1.353,"5min":1.353,"15min":1.353},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.70899999999999996,"15min":0.57299999999999995},"min":{"1min":0.58399999999999996,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.1439999999999999,"5min":1.214,"15min":1.214},"last":0.77800000000000002}]},{"osd":2,"last update":"Sun Apr 7 16:11:34 2024","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.70899999999999996,"15min":0.66700000000000004},"min":{"1min":0.61799999999999999,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.89000000000000001,"5min":0.77200000000000002,"15min":0.73699999999999999},"min":{"1min":0.626,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.2749999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.81299999999999994}]},{"osd":3,"last update":"Sun Apr 7 16:11:19 2024","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.78200000000000003,"15min":0.75700000000000001},"min":{"1min":0.67800000000000005,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.89000000000000001,"5min":0.81899999999999995,"15min":0.79900000000000004},"min":{"1min":0.68200000000000005,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.143,"5min":1.143,"15min":1.143},"last":0.95099999999999996}]},{"osd":4,"last update":"Sun Apr 7 16:12:08 2024","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":0.93600000000000005,"15min":0.92300000000000004},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.306,"5min":1.306,"15min":1.306},"last":0.98599999999999999},{"interface":"front","average":{"1min":1.0109999999999999,"5min":0.98799999999999999,"15min":0.96399999999999997},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":1.2150000000000001,"5min":1.7809999999999999,"15min":1.7809999999999999},"last":0.88700000000000001}]},{"osd":5,"last update":"Sun Apr 7 16:11:56 2024","interfaces":[{"interface":"back","average":{"1min":1.0349999999999999,"5min":0.98899999999999999,"15min":0.98099999999999998},"min":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"max":{"1min":1.196,"5min":1.196,"15min":1.196},"last":1.1699999999999999},{"interface":"front","average":{"1min":1.0409999999999999,"5min":1.0249999999999999,"15min":1.0229999999999999},"min":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":1.48,"5min":1.48,"15min":1.48},"last":1.268}]},{"osd":6,"last update":"Sun Apr 7 16:11:41 2024","interfaces":[{"interface":"back","average":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"min":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":1.3600000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":1.0209999999999999},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"min":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"max":{"1min":1.3580000000000001,"5min":1.3580000000000001,"15min":1.3580000000000001},"last":1.2969999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1539999999999999}]}]},{"osd":2,"up_from":18,"seq":77309411388,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31724,"kb_used_data":608,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713428,"statfs":{"total":95995035648,"available":95962550272,"internally_reserved":0,"allocated":622592,"data_stored":259441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Apr 7 16:11:30 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.63700000000000001,"15min":0.58599999999999997},"min":{"1min":0.56499999999999995,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.96199999999999997,"5min":1.103,"15min":1.103},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.67600000000000005,"15min":0.60599999999999998},"min":{"1min":0.57299999999999995,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.75800000000000001}]},{"osd":1,"last update":"Sun Apr 7 16:11:30 2024","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.69399999999999995,"15min":0.64100000000000001},"min":{"1min":0.59099999999999997,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":1.143},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.71699999999999997,"15min":0.64900000000000002},"min":{"1min":0.54200000000000004,"5min":0.313,"15min":0.313},"max":{"1min":1.0609999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.53400000000000003}]},{"osd":3,"last update":"Sun Apr 7 16:11:18 2024","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.81699999999999995,"15min":0.81100000000000005},"min":{"1min":0.67000000000000004,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.81200000000000006,"15min":0.80200000000000005},"min":{"1min":0.66100000000000003,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.002,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.88600000000000001}]},{"osd":4,"last update":"Sun Apr 7 16:12:09 2024","interfaces":[{"interface":"back","average":{"1min":0.96699999999999997,"5min":0.93400000000000005,"15min":0.92100000000000004},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.4299999999999999,"5min":1.4299999999999999,"15min":1.4299999999999999},"last":0.96999999999999997},{"interface":"front","average":{"1min":1.018,"5min":0.95799999999999996,"15min":0.93999999999999995},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.4490000000000001,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.92400000000000004}]},{"osd":5,"last update":"Sun Apr 7 16:11:54 2024","interfaces":[{"interface":"back","average":{"1min":1.05,"5min":0.98699999999999999,"15min":0.97699999999999998},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":1.6970000000000001,"5min":1.6970000000000001,"15min":1.6970000000000001},"last":0.83299999999999996},{"interface":"front","average":{"1min":1.0780000000000001,"5min":1.046,"15min":1.04},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.6000000000000001,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":1.069}]},{"osd":6,"last update":"Sun Apr 7 16:11:39 2024","interfaces":[{"interface":"back","average":{"1min":1.05,"5min":1.05,"15min":1.05},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.397,"5min":1.397,"15min":1.397},"last":1.0009999999999999},{"interface":"front","average":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.355,"5min":1.355,"15min":1.355},"last":1.1040000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80500000000000005}]}]},{"osd":3,"up_from":25,"seq":107374182451,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27656,"kb_used_data":628,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717496,"statfs":{"total":95995035648,"available":95966715904,"internally_reserved":0,"allocated":643072,"data_stored":261109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Apr 7 16:11:24 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.67000000000000004,"15min":0.66900000000000004},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.97399999999999998,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.54400000000000004,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.327,"5min":1.327,"15min":1.327},"last":0.39100000000000001}]},{"osd":1,"last update":"Sun Apr 7 16:11:24 2024","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.77200000000000002,"15min":0.75600000000000001},"min":{"1min":0.57999999999999996,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.544,"5min":1.544,"15min":1.544},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.76600000000000001,"15min":0.75800000000000001},"min":{"1min":0.60599999999999998,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.4510000000000001,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.375}]},{"osd":2,"last update":"Sun Apr 7 16:11:24 2024","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.82499999999999996,"15min":0.80900000000000005},"min":{"1min":0.70699999999999996,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.256,"5min":1.256,"15min":1.256},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.81599999999999995,"15min":0.81399999999999995},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.286,"5min":1.286,"15min":1.286},"last":0.35999999999999999}]},{"osd":4,"last update":"Sun Apr 7 16:12:08 2024","interfaces":[{"interface":"back","average":{"1min":1.036,"5min":0.97699999999999998,"15min":0.95699999999999996},"min":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.28,"5min":1.552,"15min":1.552},"last":0.81000000000000005},{"interface":"front","average":{"1min":1.0569999999999999,"5min":0.999,"15min":0.97699999999999998},"min":{"1min":0.81999999999999995,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.3640000000000001,"5min":1.3640000000000001,"15min":1.3640000000000001},"last":0.65100000000000002}]},{"osd":5,"last update":"Sun Apr 7 16:11:54 2024","interfaces":[{"interface":"back","average":{"1min":1.081,"5min":1.0609999999999999,"15min":1.0580000000000001},"min":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":1.3180000000000001,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":0.70999999999999996},{"interface":"front","average":{"1min":1.0109999999999999,"5min":1.0349999999999999,"15min":1.0389999999999999},"min":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.2410000000000001,"5min":1.571,"15min":1.571},"last":0.66800000000000004}]},{"osd":6,"last update":"Sun Apr 7 16:11:39 2024","interfaces":[{"interface":"back","average":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"min":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"max":{"1min":1.341,"5min":1.341,"15min":1.341},"last":0.84299999999999997},{"interface":"front","average":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"min":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"max":{"1min":1.4610000000000001,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.61399999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64000000000000001}]}]},{"osd":4,"up_from":30,"seq":128849018921,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31688,"kb_used_data":572,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713464,"statfs":{"total":95995035648,"available":95962587136,"internally_reserved":0,"allocated":585728,"data_stored":238801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Apr 7 16:11:10 2024","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.82899999999999996,"15min":0.81699999999999995},"min":{"1min":0.63800000000000001,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.90300000000000002,"15min":0.90500000000000003},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.1719999999999999,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.873}]},{"osd":1,"last update":"Sun Apr 7 16:11:10 2024","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.81299999999999994,"15min":0.80700000000000005},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.91100000000000003,"5min":0.84499999999999997,"15min":0.83399999999999996},"min":{"1min":0.64600000000000002,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.67700000000000005}]},{"osd":2,"last update":"Sun Apr 7 16:11:10 2024","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":0.874,"15min":0.86199999999999999},"min":{"1min":0.75700000000000001,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.2649999999999999,"5min":1.387,"15min":1.387},"last":1.046},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.88600000000000001,"15min":0.88300000000000001},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.175,"5min":1.175,"15min":1.175},"last":0.63}]},{"osd":3,"last update":"Sun Apr 7 16:11:10 2024","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.93100000000000005,"15min":0.93400000000000005},"min":{"1min":0.76700000000000002,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":1.069,"5min":1.458,"15min":1.458},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.94199999999999995,"5min":0.92600000000000005,"15min":0.92300000000000004},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.2390000000000001,"5min":1.323,"15min":1.323},"last":0.83399999999999996}]},{"osd":5,"last update":"Sun Apr 7 16:11:55 2024","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":0.78200000000000003,"15min":0.75600000000000001},"min":{"1min":0.60999999999999999,"5min":0.315,"15min":0.315},"max":{"1min":1.403,"5min":1.403,"15min":1.403},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.92500000000000004,"5min":0.87,"15min":0.86099999999999999},"min":{"1min":0.57299999999999995,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.3220000000000001,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":1.139}]},{"osd":6,"last update":"Sun Apr 7 16:11:42 2024","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.4139999999999999,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":0.98599999999999999},{"interface":"front","average":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.4370000000000001,"5min":1.4370000000000001,"15min":1.4370000000000001},"last":1.2150000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0129999999999999}]}]},{"osd":5,"up_from":38,"seq":163208757278,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27580,"kb_used_data":556,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717572,"statfs":{"total":95995035648,"available":95966793728,"internally_reserved":0,"allocated":569344,"data_stored":232223,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Apr 7 16:11:57 2024","interfaces":[{"interface":"back","average":{"1min":0.97499999999999998,"5min":0.92800000000000005,"15min":0.92000000000000004},"min":{"1min":0.68200000000000005,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.2609999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.94399999999999995,"15min":0.95699999999999996},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.153,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":1.0069999999999999}]},{"osd":1,"last update":"Sun Apr 7 16:11:57 2024","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.91500000000000004,"15min":0.92200000000000004},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.1619999999999999,"5min":1.331,"15min":1.331},"last":0.95399999999999996},{"interface":"front","average":{"1min":0.97899999999999998,"5min":0.98899999999999999,"15min":0.99099999999999999},"min":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.204,"5min":1.45,"15min":1.45},"last":1.1699999999999999}]},{"osd":2,"last update":"Sun Apr 7 16:11:57 2024","interfaces":[{"interface":"back","average":{"1min":0.91300000000000003,"5min":0.96999999999999997,"15min":0.97999999999999998},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.3140000000000001,"5min":1.462,"15min":1.462},"last":1.0449999999999999},{"interface":"front","average":{"1min":0.93500000000000005,"5min":0.96299999999999997,"15min":0.96699999999999997},"min":{"1min":0.67400000000000004,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.3879999999999999,"5min":1.472,"15min":1.472},"last":0.89000000000000001}]},{"osd":3,"last update":"Sun Apr 7 16:11:57 2024","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":1.173,"15min":1.218},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.4219999999999999,"5min":6.3860000000000001,"15min":6.3860000000000001},"last":1.2},{"interface":"front","average":{"1min":0.96199999999999997,"5min":1.0009999999999999,"15min":1.0069999999999999},"min":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.448,"5min":1.556,"15min":1.556},"last":0.79300000000000004}]},{"osd":4,"last update":"Sun Apr 7 16:11:57 2024","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.80900000000000005,"15min":0.80700000000000005},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.91500000000000004},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.90700000000000003,"15min":0.90900000000000003},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.22,"5min":1.4119999999999999,"15min":1.4119999999999999},"last":1.252}]},{"osd":6,"last update":"Sun Apr 7 16:11:39 2024","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":0.75},{"interface":"front","average":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":1.544,"5min":1.544,"15min":1.544},"last":1.125}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0940000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-07T16:12:18.190 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:18 smithi100 bash[21346]: cluster 2024-04-07T16:12:16.287907+0000 mgr.y (mgr.14152) 333 : cluster 0 pgmap v318: 97 pgs: 50 active+clean, 1 creating+activating, 12 creating+peering, 34 unknown; 578 KiB data, 225 MiB used, 715 GiB / 715 GiB avail; 715 B/s wr, 7 op/s 2024-04-07T16:12:18.190 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:18 smithi100 bash[21346]: audit 2024-04-07T16:12:16.339539+0000 mgr.y (mgr.14152) 334 : audit 0 from='client.24419 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-07T16:12:18.190 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:18 smithi100 bash[21346]: audit 2024-04-07T16:12:16.685374+0000 mgr.y (mgr.14152) 335 : audit 0 from='client.14520 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-07T16:12:18.190 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:18 smithi100 bash[21346]: audit 2024-04-07T16:12:17.073050+0000 mgr.y (mgr.14152) 336 : audit 0 from='client.14526 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-07T16:12:18.190 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:18 smithi100 bash[21346]: audit 2024-04-07T16:12:17.533825+0000 mon.c (mon.2) 30 : audit 0 from='client.? 172.21.15.100:0/2780407233' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2024-04-07T16:12:18.191 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:18 smithi100 bash[31660]: cluster 2024-04-07T16:12:16.287907+0000 mgr.y (mgr.14152) 333 : cluster 0 pgmap v318: 97 pgs: 50 active+clean, 1 creating+activating, 12 creating+peering, 34 unknown; 578 KiB data, 225 MiB used, 715 GiB / 715 GiB avail; 715 B/s wr, 7 op/s 2024-04-07T16:12:18.191 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:18 smithi100 bash[31660]: audit 2024-04-07T16:12:16.339539+0000 mgr.y (mgr.14152) 334 : audit 0 from='client.24419 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-07T16:12:18.191 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:18 smithi100 bash[31660]: audit 2024-04-07T16:12:16.685374+0000 mgr.y (mgr.14152) 335 : audit 0 from='client.14520 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-07T16:12:18.191 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:18 smithi100 bash[31660]: audit 2024-04-07T16:12:17.073050+0000 mgr.y (mgr.14152) 336 : audit 0 from='client.14526 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-07T16:12:18.191 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:18 smithi100 bash[31660]: audit 2024-04-07T16:12:17.533825+0000 mon.c (mon.2) 30 : audit 0 from='client.? 172.21.15.100:0/2780407233' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2024-04-07T16:12:18.357 INFO:journalctl@ceph.mds.b.smithi144.stdout:Apr 07 16:12:17 smithi144 bash[47566]: debug 2024-04-07T16:12:17.978+0000 7fd16a3c3700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-07T16:12:18.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:18 smithi144 bash[23522]: cluster 2024-04-07T16:12:16.287907+0000 mgr.y (mgr.14152) 333 : cluster 0 pgmap v318: 97 pgs: 50 active+clean, 1 creating+activating, 12 creating+peering, 34 unknown; 578 KiB data, 225 MiB used, 715 GiB / 715 GiB avail; 715 B/s wr, 7 op/s 2024-04-07T16:12:18.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:18 smithi144 bash[23522]: audit 2024-04-07T16:12:16.339539+0000 mgr.y (mgr.14152) 334 : audit 0 from='client.24419 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-07T16:12:18.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:18 smithi144 bash[23522]: audit 2024-04-07T16:12:16.685374+0000 mgr.y (mgr.14152) 335 : audit 0 from='client.14520 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-07T16:12:18.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:18 smithi144 bash[23522]: audit 2024-04-07T16:12:17.073050+0000 mgr.y (mgr.14152) 336 : audit 0 from='client.14526 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-07T16:12:18.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:18 smithi144 bash[23522]: audit 2024-04-07T16:12:17.533825+0000 mon.c (mon.2) 30 : audit 0 from='client.? 172.21.15.100:0/2780407233' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2024-04-07T16:12:19.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:19 smithi144 bash[23522]: audit 2024-04-07T16:12:17.827720+0000 mgr.y (mgr.14152) 337 : audit 0 from='client.14538 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-07T16:12:19.440 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:19 smithi100 bash[21346]: audit 2024-04-07T16:12:17.827720+0000 mgr.y (mgr.14152) 337 : audit 0 from='client.14538 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-07T16:12:19.440 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:19 smithi100 bash[31660]: audit 2024-04-07T16:12:17.827720+0000 mgr.y (mgr.14152) 337 : audit 0 from='client.14538 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-07T16:12:20.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:20 smithi144 bash[23522]: cluster 2024-04-07T16:12:18.288820+0000 mgr.y (mgr.14152) 338 : cluster 0 pgmap v319: 97 pgs: 97 active+clean; 579 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 2.3 KiB/s wr, 7 op/s 2024-04-07T16:12:20.440 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:20 smithi100 bash[21346]: cluster 2024-04-07T16:12:18.288820+0000 mgr.y (mgr.14152) 338 : cluster 0 pgmap v319: 97 pgs: 97 active+clean; 579 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 2.3 KiB/s wr, 7 op/s 2024-04-07T16:12:20.440 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:20 smithi100 bash[31660]: cluster 2024-04-07T16:12:18.288820+0000 mgr.y (mgr.14152) 338 : cluster 0 pgmap v319: 97 pgs: 97 active+clean; 579 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 2.3 KiB/s wr, 7 op/s 2024-04-07T16:12:20.856 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-07T16:12:21.201 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-07T16:12:21.202 INFO:teuthology.orchestra.run.smithi100.stderr:dumped all 2024-04-07T16:12:21.222 INFO:teuthology.orchestra.run.smithi100.stdout:{"pg_ready":true,"pg_map":{"version":320,"stamp":"2024-04-07T16:12:20.289096+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":69,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":239,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":231068,"kb_used_data":6696,"kb_used_omap":12,"kb_used_meta":224243,"kb_avail":749730148,"statfs":{"total":767960285184,"available":767723671552,"internally_reserved":0,"allocated":6856704,"data_stored":3853387,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":229625401},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":63,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.287594"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.376089+0000","last_change":"2024-04-07T16:12:13.395367+0000","last_active":"2024-04-07T16:12:14.376089+0000","last_peered":"2024-04-07T16:12:14.376089+0000","last_clean":"2024-04-07T16:12:14.376089+0000","last_became_active":"2024-04-07T16:12:13.395023+0000","last_became_peered":"2024-04-07T16:12:13.395023+0000","last_unstale":"2024-04-07T16:12:14.376089+0000","last_undegraded":"2024-04-07T16:12:14.376089+0000","last_fullsized":"2024-04-07T16:12:14.376089+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T23:05:09.752696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.379727+0000","last_change":"2024-04-07T16:12:13.385549+0000","last_active":"2024-04-07T16:12:14.379727+0000","last_peered":"2024-04-07T16:12:14.379727+0000","last_clean":"2024-04-07T16:12:14.379727+0000","last_became_active":"2024-04-07T16:12:13.385175+0000","last_became_peered":"2024-04-07T16:12:13.385175+0000","last_unstale":"2024-04-07T16:12:14.379727+0000","last_undegraded":"2024-04-07T16:12:14.379727+0000","last_fullsized":"2024-04-07T16:12:14.379727+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T16:28:22.788092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,6],"acting":[1,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.476666+0000","last_change":"2024-04-07T16:12:13.410431+0000","last_active":"2024-04-07T16:12:14.476666+0000","last_peered":"2024-04-07T16:12:14.476666+0000","last_clean":"2024-04-07T16:12:14.476666+0000","last_became_active":"2024-04-07T16:12:13.410088+0000","last_became_peered":"2024-04-07T16:12:13.410088+0000","last_unstale":"2024-04-07T16:12:14.476666+0000","last_undegraded":"2024-04-07T16:12:14.476666+0000","last_fullsized":"2024-04-07T16:12:14.476666+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T19:47:57.716144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.475366+0000","last_change":"2024-04-07T16:12:13.402328+0000","last_active":"2024-04-07T16:12:14.475366+0000","last_peered":"2024-04-07T16:12:14.475366+0000","last_clean":"2024-04-07T16:12:14.475366+0000","last_became_active":"2024-04-07T16:12:13.401819+0000","last_became_peered":"2024-04-07T16:12:13.401819+0000","last_unstale":"2024-04-07T16:12:14.475366+0000","last_undegraded":"2024-04-07T16:12:14.475366+0000","last_fullsized":"2024-04-07T16:12:14.475366+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T21:43:13.292859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.375284+0000","last_change":"2024-04-07T16:12:13.413487+0000","last_active":"2024-04-07T16:12:14.375284+0000","last_peered":"2024-04-07T16:12:14.375284+0000","last_clean":"2024-04-07T16:12:14.375284+0000","last_became_active":"2024-04-07T16:12:13.413281+0000","last_became_peered":"2024-04-07T16:12:13.413281+0000","last_unstale":"2024-04-07T16:12:14.375284+0000","last_undegraded":"2024-04-07T16:12:14.375284+0000","last_fullsized":"2024-04-07T16:12:14.375284+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T23:59:23.302803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.475356+0000","last_change":"2024-04-07T16:12:13.401561+0000","last_active":"2024-04-07T16:12:14.475356+0000","last_peered":"2024-04-07T16:12:14.475356+0000","last_clean":"2024-04-07T16:12:14.475356+0000","last_became_active":"2024-04-07T16:12:13.400365+0000","last_became_peered":"2024-04-07T16:12:13.400365+0000","last_unstale":"2024-04-07T16:12:14.475356+0000","last_undegraded":"2024-04-07T16:12:14.475356+0000","last_fullsized":"2024-04-07T16:12:14.475356+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T20:14:49.110462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.374561+0000","last_change":"2024-04-07T16:12:13.393518+0000","last_active":"2024-04-07T16:12:14.374561+0000","last_peered":"2024-04-07T16:12:14.374561+0000","last_clean":"2024-04-07T16:12:14.374561+0000","last_became_active":"2024-04-07T16:12:13.393187+0000","last_became_peered":"2024-04-07T16:12:13.393187+0000","last_unstale":"2024-04-07T16:12:14.374561+0000","last_undegraded":"2024-04-07T16:12:14.374561+0000","last_fullsized":"2024-04-07T16:12:14.374561+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T19:41:28.702905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.475028+0000","last_change":"2024-04-07T16:12:13.394702+0000","last_active":"2024-04-07T16:12:14.475028+0000","last_peered":"2024-04-07T16:12:14.475028+0000","last_clean":"2024-04-07T16:12:14.475028+0000","last_became_active":"2024-04-07T16:12:13.394422+0000","last_became_peered":"2024-04-07T16:12:13.394422+0000","last_unstale":"2024-04-07T16:12:14.475028+0000","last_undegraded":"2024-04-07T16:12:14.475028+0000","last_fullsized":"2024-04-07T16:12:14.475028+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T00:37:44.512471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.375990+0000","last_change":"2024-04-07T16:12:13.401766+0000","last_active":"2024-04-07T16:12:14.375990+0000","last_peered":"2024-04-07T16:12:14.375990+0000","last_clean":"2024-04-07T16:12:14.375990+0000","last_became_active":"2024-04-07T16:12:13.401411+0000","last_became_peered":"2024-04-07T16:12:13.401411+0000","last_unstale":"2024-04-07T16:12:14.375990+0000","last_undegraded":"2024-04-07T16:12:14.375990+0000","last_fullsized":"2024-04-07T16:12:14.375990+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T23:32:52.283410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,0],"acting":[7,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.475526+0000","last_change":"2024-04-07T16:12:13.396409+0000","last_active":"2024-04-07T16:12:14.475526+0000","last_peered":"2024-04-07T16:12:14.475526+0000","last_clean":"2024-04-07T16:12:14.475526+0000","last_became_active":"2024-04-07T16:12:13.396094+0000","last_became_peered":"2024-04-07T16:12:13.396094+0000","last_unstale":"2024-04-07T16:12:14.475526+0000","last_undegraded":"2024-04-07T16:12:14.475526+0000","last_fullsized":"2024-04-07T16:12:14.475526+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T18:54:36.072837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.474549+0000","last_change":"2024-04-07T16:12:13.401973+0000","last_active":"2024-04-07T16:12:14.474549+0000","last_peered":"2024-04-07T16:12:14.474549+0000","last_clean":"2024-04-07T16:12:14.474549+0000","last_became_active":"2024-04-07T16:12:13.401724+0000","last_became_peered":"2024-04-07T16:12:13.401724+0000","last_unstale":"2024-04-07T16:12:14.474549+0000","last_undegraded":"2024-04-07T16:12:14.474549+0000","last_fullsized":"2024-04-07T16:12:14.474549+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T21:26:29.208223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,0],"acting":[6,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-07T16:12:13.408938+0000","last_change":"2024-04-07T16:12:13.404858+0000","last_active":"2024-04-07T16:12:13.408938+0000","last_peered":"2024-04-07T16:12:13.408938+0000","last_clean":"2024-04-07T16:12:13.408938+0000","last_became_active":"2024-04-07T16:12:13.404529+0000","last_became_peered":"2024-04-07T16:12:13.404529+0000","last_unstale":"2024-04-07T16:12:13.408938+0000","last_undegraded":"2024-04-07T16:12:13.408938+0000","last_fullsized":"2024-04-07T16:12:13.408938+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T19:57:22.893626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,0],"acting":[4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.474547+0000","last_change":"2024-04-07T16:12:13.394760+0000","last_active":"2024-04-07T16:12:14.474547+0000","last_peered":"2024-04-07T16:12:14.474547+0000","last_clean":"2024-04-07T16:12:14.474547+0000","last_became_active":"2024-04-07T16:12:13.394402+0000","last_became_peered":"2024-04-07T16:12:13.394402+0000","last_unstale":"2024-04-07T16:12:14.474547+0000","last_undegraded":"2024-04-07T16:12:14.474547+0000","last_fullsized":"2024-04-07T16:12:14.474547+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T18:47:13.891972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.476854+0000","last_change":"2024-04-07T16:12:13.400292+0000","last_active":"2024-04-07T16:12:14.476854+0000","last_peered":"2024-04-07T16:12:14.476854+0000","last_clean":"2024-04-07T16:12:14.476854+0000","last_became_active":"2024-04-07T16:12:13.400028+0000","last_became_peered":"2024-04-07T16:12:13.400028+0000","last_unstale":"2024-04-07T16:12:14.476854+0000","last_undegraded":"2024-04-07T16:12:14.476854+0000","last_fullsized":"2024-04-07T16:12:14.476854+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T18:14:33.693852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.378098+0000","last_change":"2024-04-07T16:12:13.398535+0000","last_active":"2024-04-07T16:12:14.378098+0000","last_peered":"2024-04-07T16:12:14.378098+0000","last_clean":"2024-04-07T16:12:14.378098+0000","last_became_active":"2024-04-07T16:12:13.398287+0000","last_became_peered":"2024-04-07T16:12:13.398287+0000","last_unstale":"2024-04-07T16:12:14.378098+0000","last_undegraded":"2024-04-07T16:12:14.378098+0000","last_fullsized":"2024-04-07T16:12:14.378098+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T23:58:53.081885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,5],"acting":[1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.471600+0000","last_change":"2024-04-07T16:12:13.395243+0000","last_active":"2024-04-07T16:12:14.471600+0000","last_peered":"2024-04-07T16:12:14.471600+0000","last_clean":"2024-04-07T16:12:14.471600+0000","last_became_active":"2024-04-07T16:12:13.394956+0000","last_became_peered":"2024-04-07T16:12:13.394956+0000","last_unstale":"2024-04-07T16:12:14.471600+0000","last_undegraded":"2024-04-07T16:12:14.471600+0000","last_fullsized":"2024-04-07T16:12:14.471600+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T03:04:58.804429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.477023+0000","last_change":"2024-04-07T16:12:13.398543+0000","last_active":"2024-04-07T16:12:14.477023+0000","last_peered":"2024-04-07T16:12:14.477023+0000","last_clean":"2024-04-07T16:12:14.477023+0000","last_became_active":"2024-04-07T16:12:13.398198+0000","last_became_peered":"2024-04-07T16:12:13.398198+0000","last_unstale":"2024-04-07T16:12:14.477023+0000","last_undegraded":"2024-04-07T16:12:14.477023+0000","last_fullsized":"2024-04-07T16:12:14.477023+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T22:57:24.895712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,4],"acting":[5,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.474396+0000","last_change":"2024-04-07T16:12:13.416210+0000","last_active":"2024-04-07T16:12:14.474396+0000","last_peered":"2024-04-07T16:12:14.474396+0000","last_clean":"2024-04-07T16:12:14.474396+0000","last_became_active":"2024-04-07T16:12:13.415972+0000","last_became_peered":"2024-04-07T16:12:13.415972+0000","last_unstale":"2024-04-07T16:12:14.474396+0000","last_undegraded":"2024-04-07T16:12:14.474396+0000","last_fullsized":"2024-04-07T16:12:14.474396+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T23:40:11.606978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,7],"acting":[2,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.375192+0000","last_change":"2024-04-07T16:12:13.414203+0000","last_active":"2024-04-07T16:12:14.375192+0000","last_peered":"2024-04-07T16:12:14.375192+0000","last_clean":"2024-04-07T16:12:14.375192+0000","last_became_active":"2024-04-07T16:12:13.413869+0000","last_became_peered":"2024-04-07T16:12:13.413869+0000","last_unstale":"2024-04-07T16:12:14.375192+0000","last_undegraded":"2024-04-07T16:12:14.375192+0000","last_fullsized":"2024-04-07T16:12:14.375192+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T17:11:47.895568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,6],"acting":[7,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1f","version":"61'4","reported_seq":16,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-07T16:12:13.413156+0000","last_change":"2024-04-07T16:12:13.032601+0000","last_active":"2024-04-07T16:12:13.413156+0000","last_peered":"2024-04-07T16:12:13.413156+0000","last_clean":"2024-04-07T16:12:13.413156+0000","last_became_active":"2024-04-07T16:12:13.032199+0000","last_became_peered":"2024-04-07T16:12:13.032199+0000","last_unstale":"2024-04-07T16:12:13.413156+0000","last_undegraded":"2024-04-07T16:12:13.413156+0000","last_fullsized":"2024-04-07T16:12:13.413156+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T19:01:44.798719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.474373+0000","last_change":"2024-04-07T16:12:13.407852+0000","last_active":"2024-04-07T16:12:14.474373+0000","last_peered":"2024-04-07T16:12:14.474373+0000","last_clean":"2024-04-07T16:12:14.474373+0000","last_became_active":"2024-04-07T16:12:13.407538+0000","last_became_peered":"2024-04-07T16:12:13.407538+0000","last_unstale":"2024-04-07T16:12:14.474373+0000","last_undegraded":"2024-04-07T16:12:14.474373+0000","last_fullsized":"2024-04-07T16:12:14.474373+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T20:27:52.664149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"61'1","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.474471+0000","last_change":"2024-04-07T16:12:13.038807+0000","last_active":"2024-04-07T16:12:14.474471+0000","last_peered":"2024-04-07T16:12:14.474471+0000","last_clean":"2024-04-07T16:12:14.474471+0000","last_became_active":"2024-04-07T16:12:13.038492+0000","last_became_peered":"2024-04-07T16:12:13.038492+0000","last_unstale":"2024-04-07T16:12:14.474471+0000","last_undegraded":"2024-04-07T16:12:14.474471+0000","last_fullsized":"2024-04-07T16:12:14.474471+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T02:11:03.746017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.476900+0000","last_change":"2024-04-07T16:12:13.410031+0000","last_active":"2024-04-07T16:12:14.476900+0000","last_peered":"2024-04-07T16:12:14.476900+0000","last_clean":"2024-04-07T16:12:14.476900+0000","last_became_active":"2024-04-07T16:12:13.409049+0000","last_became_peered":"2024-04-07T16:12:13.409049+0000","last_unstale":"2024-04-07T16:12:14.476900+0000","last_undegraded":"2024-04-07T16:12:14.476900+0000","last_fullsized":"2024-04-07T16:12:14.476900+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T21:35:09.308017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"61'2","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.375664+0000","last_change":"2024-04-07T16:12:13.042401+0000","last_active":"2024-04-07T16:12:14.375664+0000","last_peered":"2024-04-07T16:12:14.375664+0000","last_clean":"2024-04-07T16:12:14.375664+0000","last_became_active":"2024-04-07T16:12:13.041970+0000","last_became_peered":"2024-04-07T16:12:13.041970+0000","last_unstale":"2024-04-07T16:12:14.375664+0000","last_undegraded":"2024-04-07T16:12:14.375664+0000","last_fullsized":"2024-04-07T16:12:14.375664+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T16:36:29.235347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.476235+0000","last_change":"2024-04-07T16:12:13.394718+0000","last_active":"2024-04-07T16:12:14.476235+0000","last_peered":"2024-04-07T16:12:14.476235+0000","last_clean":"2024-04-07T16:12:14.476235+0000","last_became_active":"2024-04-07T16:12:13.394271+0000","last_became_peered":"2024-04-07T16:12:13.394271+0000","last_unstale":"2024-04-07T16:12:14.476235+0000","last_undegraded":"2024-04-07T16:12:14.476235+0000","last_fullsized":"2024-04-07T16:12:14.476235+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T20:10:29.096403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"61'2","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.375447+0000","last_change":"2024-04-07T16:12:13.042391+0000","last_active":"2024-04-07T16:12:14.375447+0000","last_peered":"2024-04-07T16:12:14.375447+0000","last_clean":"2024-04-07T16:12:14.375447+0000","last_became_active":"2024-04-07T16:12:13.041919+0000","last_became_peered":"2024-04-07T16:12:13.041919+0000","last_unstale":"2024-04-07T16:12:14.375447+0000","last_undegraded":"2024-04-07T16:12:14.375447+0000","last_fullsized":"2024-04-07T16:12:14.375447+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T21:50:59.576615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.475966+0000","last_change":"2024-04-07T16:12:13.398361+0000","last_active":"2024-04-07T16:12:14.475966+0000","last_peered":"2024-04-07T16:12:14.475966+0000","last_clean":"2024-04-07T16:12:14.475966+0000","last_became_active":"2024-04-07T16:12:13.397979+0000","last_became_peered":"2024-04-07T16:12:13.397979+0000","last_unstale":"2024-04-07T16:12:14.475966+0000","last_undegraded":"2024-04-07T16:12:14.475966+0000","last_fullsized":"2024-04-07T16:12:14.475966+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T23:16:23.843327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"61'1","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.474818+0000","last_change":"2024-04-07T16:12:13.041880+0000","last_active":"2024-04-07T16:12:14.474818+0000","last_peered":"2024-04-07T16:12:14.474818+0000","last_clean":"2024-04-07T16:12:14.474818+0000","last_became_active":"2024-04-07T16:12:13.041409+0000","last_became_peered":"2024-04-07T16:12:13.041409+0000","last_unstale":"2024-04-07T16:12:14.474818+0000","last_undegraded":"2024-04-07T16:12:14.474818+0000","last_fullsized":"2024-04-07T16:12:14.474818+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T18:32:23.206985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-07T16:12:13.414714+0000","last_change":"2024-04-07T16:12:13.413664+0000","last_active":"2024-04-07T16:12:13.414714+0000","last_peered":"2024-04-07T16:12:13.414714+0000","last_clean":"2024-04-07T16:12:13.414714+0000","last_became_active":"2024-04-07T16:12:13.413311+0000","last_became_peered":"2024-04-07T16:12:13.413311+0000","last_unstale":"2024-04-07T16:12:13.414714+0000","last_undegraded":"2024-04-07T16:12:13.414714+0000","last_fullsized":"2024-04-07T16:12:13.414714+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T02:43:21.836307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"61'1","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.476584+0000","last_change":"2024-04-07T16:12:13.038960+0000","last_active":"2024-04-07T16:12:14.476584+0000","last_peered":"2024-04-07T16:12:14.476584+0000","last_clean":"2024-04-07T16:12:14.476584+0000","last_became_active":"2024-04-07T16:12:13.038410+0000","last_became_peered":"2024-04-07T16:12:13.038410+0000","last_unstale":"2024-04-07T16:12:14.476584+0000","last_undegraded":"2024-04-07T16:12:14.476584+0000","last_fullsized":"2024-04-07T16:12:14.476584+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T03:41:38.418312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.477108+0000","last_change":"2024-04-07T16:12:13.405634+0000","last_active":"2024-04-07T16:12:14.477108+0000","last_peered":"2024-04-07T16:12:14.477108+0000","last_clean":"2024-04-07T16:12:14.477108+0000","last_became_active":"2024-04-07T16:12:13.405069+0000","last_became_peered":"2024-04-07T16:12:13.405069+0000","last_unstale":"2024-04-07T16:12:14.477108+0000","last_undegraded":"2024-04-07T16:12:14.477108+0000","last_fullsized":"2024-04-07T16:12:14.477108+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T23:50:18.973629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.475758+0000","last_change":"2024-04-07T16:12:13.034420+0000","last_active":"2024-04-07T16:12:14.475758+0000","last_peered":"2024-04-07T16:12:14.475758+0000","last_clean":"2024-04-07T16:12:14.475758+0000","last_became_active":"2024-04-07T16:12:13.034013+0000","last_became_peered":"2024-04-07T16:12:13.034013+0000","last_unstale":"2024-04-07T16:12:14.475758+0000","last_undegraded":"2024-04-07T16:12:14.475758+0000","last_fullsized":"2024-04-07T16:12:14.475758+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T19:51:21.525865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.474201+0000","last_change":"2024-04-07T16:12:13.398531+0000","last_active":"2024-04-07T16:12:14.474201+0000","last_peered":"2024-04-07T16:12:14.474201+0000","last_clean":"2024-04-07T16:12:14.474201+0000","last_became_active":"2024-04-07T16:12:13.398017+0000","last_became_peered":"2024-04-07T16:12:13.398017+0000","last_unstale":"2024-04-07T16:12:14.474201+0000","last_undegraded":"2024-04-07T16:12:14.474201+0000","last_fullsized":"2024-04-07T16:12:14.474201+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T17:33:54.306902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-07T16:12:13.366972+0000","last_change":"2024-04-07T16:12:13.045443+0000","last_active":"2024-04-07T16:12:13.366972+0000","last_peered":"2024-04-07T16:12:13.366972+0000","last_clean":"2024-04-07T16:12:13.366972+0000","last_became_active":"2024-04-07T16:12:13.044808+0000","last_became_peered":"2024-04-07T16:12:13.044808+0000","last_unstale":"2024-04-07T16:12:13.366972+0000","last_undegraded":"2024-04-07T16:12:13.366972+0000","last_fullsized":"2024-04-07T16:12:13.366972+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T22:44:14.623116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.378519+0000","last_change":"2024-04-07T16:12:13.391166+0000","last_active":"2024-04-07T16:12:14.378519+0000","last_peered":"2024-04-07T16:12:14.378519+0000","last_clean":"2024-04-07T16:12:14.378519+0000","last_became_active":"2024-04-07T16:12:13.390643+0000","last_became_peered":"2024-04-07T16:12:13.390643+0000","last_unstale":"2024-04-07T16:12:14.378519+0000","last_undegraded":"2024-04-07T16:12:14.378519+0000","last_fullsized":"2024-04-07T16:12:14.378519+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T21:22:33.110408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.474717+0000","last_change":"2024-04-07T16:12:13.030149+0000","last_active":"2024-04-07T16:12:14.474717+0000","last_peered":"2024-04-07T16:12:14.474717+0000","last_clean":"2024-04-07T16:12:14.474717+0000","last_became_active":"2024-04-07T16:12:13.029525+0000","last_became_peered":"2024-04-07T16:12:13.029525+0000","last_unstale":"2024-04-07T16:12:14.474717+0000","last_undegraded":"2024-04-07T16:12:14.474717+0000","last_fullsized":"2024-04-07T16:12:14.474717+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T20:09:35.180264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.476647+0000","last_change":"2024-04-07T16:12:13.398411+0000","last_active":"2024-04-07T16:12:14.476647+0000","last_peered":"2024-04-07T16:12:14.476647+0000","last_clean":"2024-04-07T16:12:14.476647+0000","last_became_active":"2024-04-07T16:12:13.398065+0000","last_became_peered":"2024-04-07T16:12:13.398065+0000","last_unstale":"2024-04-07T16:12:14.476647+0000","last_undegraded":"2024-04-07T16:12:14.476647+0000","last_fullsized":"2024-04-07T16:12:14.476647+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T21:13:20.448127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-07T16:12:13.369418+0000","last_change":"2024-04-07T16:12:13.036777+0000","last_active":"2024-04-07T16:12:13.369418+0000","last_peered":"2024-04-07T16:12:13.369418+0000","last_clean":"2024-04-07T16:12:13.369418+0000","last_became_active":"2024-04-07T16:12:13.036322+0000","last_became_peered":"2024-04-07T16:12:13.036322+0000","last_unstale":"2024-04-07T16:12:13.369418+0000","last_undegraded":"2024-04-07T16:12:13.369418+0000","last_fullsized":"2024-04-07T16:12:13.369418+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T18:26:16.873044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.476762+0000","last_change":"2024-04-07T16:12:13.389782+0000","last_active":"2024-04-07T16:12:14.476762+0000","last_peered":"2024-04-07T16:12:14.476762+0000","last_clean":"2024-04-07T16:12:14.476762+0000","last_became_active":"2024-04-07T16:12:13.389507+0000","last_became_peered":"2024-04-07T16:12:13.389507+0000","last_unstale":"2024-04-07T16:12:14.476762+0000","last_undegraded":"2024-04-07T16:12:14.476762+0000","last_fullsized":"2024-04-07T16:12:14.476762+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T19:07:54.825354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"61'3","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.379300+0000","last_change":"2024-04-07T16:12:13.039273+0000","last_active":"2024-04-07T16:12:14.379300+0000","last_peered":"2024-04-07T16:12:14.379300+0000","last_clean":"2024-04-07T16:12:14.379300+0000","last_became_active":"2024-04-07T16:12:13.038911+0000","last_became_peered":"2024-04-07T16:12:13.038911+0000","last_unstale":"2024-04-07T16:12:14.379300+0000","last_undegraded":"2024-04-07T16:12:14.379300+0000","last_fullsized":"2024-04-07T16:12:14.379300+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T02:11:53.262707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-07T16:12:13.392272+0000","last_change":"2024-04-07T16:12:13.391349+0000","last_active":"2024-04-07T16:12:13.392272+0000","last_peered":"2024-04-07T16:12:13.392272+0000","last_clean":"2024-04-07T16:12:13.392272+0000","last_became_active":"2024-04-07T16:12:13.390960+0000","last_became_peered":"2024-04-07T16:12:13.390960+0000","last_unstale":"2024-04-07T16:12:13.392272+0000","last_undegraded":"2024-04-07T16:12:13.392272+0000","last_fullsized":"2024-04-07T16:12:13.392272+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T16:55:37.429799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"61'6","reported_seq":18,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.473934+0000","last_change":"2024-04-07T16:12:13.031673+0000","last_active":"2024-04-07T16:12:14.473934+0000","last_peered":"2024-04-07T16:12:14.473934+0000","last_clean":"2024-04-07T16:12:14.473934+0000","last_became_active":"2024-04-07T16:12:13.031092+0000","last_became_peered":"2024-04-07T16:12:13.031092+0000","last_unstale":"2024-04-07T16:12:14.473934+0000","last_undegraded":"2024-04-07T16:12:14.473934+0000","last_fullsized":"2024-04-07T16:12:14.473934+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T20:17:48.593862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.375355+0000","last_change":"2024-04-07T16:12:13.390083+0000","last_active":"2024-04-07T16:12:14.375355+0000","last_peered":"2024-04-07T16:12:14.375355+0000","last_clean":"2024-04-07T16:12:14.375355+0000","last_became_active":"2024-04-07T16:12:13.389755+0000","last_became_peered":"2024-04-07T16:12:13.389755+0000","last_unstale":"2024-04-07T16:12:14.375355+0000","last_undegraded":"2024-04-07T16:12:14.375355+0000","last_fullsized":"2024-04-07T16:12:14.375355+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T02:31:09.155569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"61'2","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-07T16:12:13.412619+0000","last_change":"2024-04-07T16:12:13.033352+0000","last_active":"2024-04-07T16:12:13.412619+0000","last_peered":"2024-04-07T16:12:13.412619+0000","last_clean":"2024-04-07T16:12:13.412619+0000","last_became_active":"2024-04-07T16:12:13.032811+0000","last_became_peered":"2024-04-07T16:12:13.032811+0000","last_unstale":"2024-04-07T16:12:13.412619+0000","last_undegraded":"2024-04-07T16:12:13.412619+0000","last_fullsized":"2024-04-07T16:12:13.412619+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T01:50:02.195507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.476518+0000","last_change":"2024-04-07T16:12:13.396838+0000","last_active":"2024-04-07T16:12:14.476518+0000","last_peered":"2024-04-07T16:12:14.476518+0000","last_clean":"2024-04-07T16:12:14.476518+0000","last_became_active":"2024-04-07T16:12:13.396555+0000","last_became_peered":"2024-04-07T16:12:13.396555+0000","last_unstale":"2024-04-07T16:12:14.476518+0000","last_undegraded":"2024-04-07T16:12:14.476518+0000","last_fullsized":"2024-04-07T16:12:14.476518+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T21:00:22.453534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-07T16:12:13.367081+0000","last_change":"2024-04-07T16:12:13.045494+0000","last_active":"2024-04-07T16:12:13.367081+0000","last_peered":"2024-04-07T16:12:13.367081+0000","last_clean":"2024-04-07T16:12:13.367081+0000","last_became_active":"2024-04-07T16:12:13.045086+0000","last_became_peered":"2024-04-07T16:12:13.045086+0000","last_unstale":"2024-04-07T16:12:13.367081+0000","last_undegraded":"2024-04-07T16:12:13.367081+0000","last_fullsized":"2024-04-07T16:12:13.367081+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T23:15:11.494437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.375013+0000","last_change":"2024-04-07T16:12:13.413928+0000","last_active":"2024-04-07T16:12:14.375013+0000","last_peered":"2024-04-07T16:12:14.375013+0000","last_clean":"2024-04-07T16:12:14.375013+0000","last_became_active":"2024-04-07T16:12:13.413447+0000","last_became_peered":"2024-04-07T16:12:13.413447+0000","last_unstale":"2024-04-07T16:12:14.375013+0000","last_undegraded":"2024-04-07T16:12:14.375013+0000","last_fullsized":"2024-04-07T16:12:14.375013+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T03:26:42.875228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.473099+0000","last_change":"2024-04-07T16:12:13.032942+0000","last_active":"2024-04-07T16:12:14.473099+0000","last_peered":"2024-04-07T16:12:14.473099+0000","last_clean":"2024-04-07T16:12:14.473099+0000","last_became_active":"2024-04-07T16:12:13.032108+0000","last_became_peered":"2024-04-07T16:12:13.032108+0000","last_unstale":"2024-04-07T16:12:14.473099+0000","last_undegraded":"2024-04-07T16:12:14.473099+0000","last_fullsized":"2024-04-07T16:12:14.473099+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T18:53:06.864412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.473331+0000","last_change":"2024-04-07T16:12:13.400141+0000","last_active":"2024-04-07T16:12:14.473331+0000","last_peered":"2024-04-07T16:12:14.473331+0000","last_clean":"2024-04-07T16:12:14.473331+0000","last_became_active":"2024-04-07T16:12:13.399823+0000","last_became_peered":"2024-04-07T16:12:13.399823+0000","last_unstale":"2024-04-07T16:12:14.473331+0000","last_undegraded":"2024-04-07T16:12:14.473331+0000","last_fullsized":"2024-04-07T16:12:14.473331+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T21:27:25.248595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"61'8","reported_seq":20,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.474350+0000","last_change":"2024-04-07T16:12:13.034154+0000","last_active":"2024-04-07T16:12:14.474350+0000","last_peered":"2024-04-07T16:12:14.474350+0000","last_clean":"2024-04-07T16:12:14.474350+0000","last_became_active":"2024-04-07T16:12:13.033838+0000","last_became_peered":"2024-04-07T16:12:13.033838+0000","last_unstale":"2024-04-07T16:12:14.474350+0000","last_undegraded":"2024-04-07T16:12:14.474350+0000","last_fullsized":"2024-04-07T16:12:14.474350+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T23:13:07.216314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.374716+0000","last_change":"2024-04-07T16:12:13.385883+0000","last_active":"2024-04-07T16:12:14.374716+0000","last_peered":"2024-04-07T16:12:14.374716+0000","last_clean":"2024-04-07T16:12:14.374716+0000","last_became_active":"2024-04-07T16:12:13.385485+0000","last_became_peered":"2024-04-07T16:12:13.385485+0000","last_unstale":"2024-04-07T16:12:14.374716+0000","last_undegraded":"2024-04-07T16:12:14.374716+0000","last_fullsized":"2024-04-07T16:12:14.374716+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T21:01:09.262433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"61'1","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-07T16:12:13.403779+0000","last_change":"2024-04-07T16:12:13.028004+0000","last_active":"2024-04-07T16:12:13.403779+0000","last_peered":"2024-04-07T16:12:13.403779+0000","last_clean":"2024-04-07T16:12:13.403779+0000","last_became_active":"2024-04-07T16:12:13.027445+0000","last_became_peered":"2024-04-07T16:12:13.027445+0000","last_unstale":"2024-04-07T16:12:13.403779+0000","last_undegraded":"2024-04-07T16:12:13.403779+0000","last_fullsized":"2024-04-07T16:12:13.403779+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T17:55:06.727142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.375770+0000","last_change":"2024-04-07T16:12:13.386584+0000","last_active":"2024-04-07T16:12:14.375770+0000","last_peered":"2024-04-07T16:12:14.375770+0000","last_clean":"2024-04-07T16:12:14.375770+0000","last_became_active":"2024-04-07T16:12:13.386057+0000","last_became_peered":"2024-04-07T16:12:13.386057+0000","last_unstale":"2024-04-07T16:12:14.375770+0000","last_undegraded":"2024-04-07T16:12:14.375770+0000","last_fullsized":"2024-04-07T16:12:14.375770+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T17:57:57.548179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.379860+0000","last_change":"2024-04-07T16:12:13.037974+0000","last_active":"2024-04-07T16:12:14.379860+0000","last_peered":"2024-04-07T16:12:14.379860+0000","last_clean":"2024-04-07T16:12:14.379860+0000","last_became_active":"2024-04-07T16:12:13.037627+0000","last_became_peered":"2024-04-07T16:12:13.037627+0000","last_unstale":"2024-04-07T16:12:14.379860+0000","last_undegraded":"2024-04-07T16:12:14.379860+0000","last_fullsized":"2024-04-07T16:12:14.379860+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T19:28:12.174861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.476249+0000","last_change":"2024-04-07T16:12:13.395073+0000","last_active":"2024-04-07T16:12:14.476249+0000","last_peered":"2024-04-07T16:12:14.476249+0000","last_clean":"2024-04-07T16:12:14.476249+0000","last_became_active":"2024-04-07T16:12:13.394113+0000","last_became_peered":"2024-04-07T16:12:13.394113+0000","last_unstale":"2024-04-07T16:12:14.476249+0000","last_undegraded":"2024-04-07T16:12:14.476249+0000","last_fullsized":"2024-04-07T16:12:14.476249+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T23:28:32.978580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-07T16:12:13.366953+0000","last_change":"2024-04-07T16:12:13.036336+0000","last_active":"2024-04-07T16:12:13.366953+0000","last_peered":"2024-04-07T16:12:13.366953+0000","last_clean":"2024-04-07T16:12:13.366953+0000","last_became_active":"2024-04-07T16:12:13.035625+0000","last_became_peered":"2024-04-07T16:12:13.035625+0000","last_unstale":"2024-04-07T16:12:13.366953+0000","last_undegraded":"2024-04-07T16:12:13.366953+0000","last_fullsized":"2024-04-07T16:12:13.366953+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T18:15:00.506827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-07T16:12:13.408927+0000","last_change":"2024-04-07T16:12:13.404406+0000","last_active":"2024-04-07T16:12:13.408927+0000","last_peered":"2024-04-07T16:12:13.408927+0000","last_clean":"2024-04-07T16:12:13.408927+0000","last_became_active":"2024-04-07T16:12:13.404029+0000","last_became_peered":"2024-04-07T16:12:13.404029+0000","last_unstale":"2024-04-07T16:12:13.408927+0000","last_undegraded":"2024-04-07T16:12:13.408927+0000","last_fullsized":"2024-04-07T16:12:13.408927+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T16:41:25.469343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.377741+0000","last_change":"2024-04-07T16:12:13.393979+0000","last_active":"2024-04-07T16:12:14.377741+0000","last_peered":"2024-04-07T16:12:14.377741+0000","last_clean":"2024-04-07T16:12:14.377741+0000","last_became_active":"2024-04-07T16:12:13.393472+0000","last_became_peered":"2024-04-07T16:12:13.393472+0000","last_unstale":"2024-04-07T16:12:14.377741+0000","last_undegraded":"2024-04-07T16:12:14.377741+0000","last_fullsized":"2024-04-07T16:12:14.377741+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T21:51:13.834107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.474169+0000","last_change":"2024-04-07T16:12:13.035071+0000","last_active":"2024-04-07T16:12:14.474169+0000","last_peered":"2024-04-07T16:12:14.474169+0000","last_clean":"2024-04-07T16:12:14.474169+0000","last_became_active":"2024-04-07T16:12:13.034795+0000","last_became_peered":"2024-04-07T16:12:13.034795+0000","last_unstale":"2024-04-07T16:12:14.474169+0000","last_undegraded":"2024-04-07T16:12:14.474169+0000","last_fullsized":"2024-04-07T16:12:14.474169+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T17:51:30.599527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.476063+0000","last_change":"2024-04-07T16:12:13.394635+0000","last_active":"2024-04-07T16:12:14.476063+0000","last_peered":"2024-04-07T16:12:14.476063+0000","last_clean":"2024-04-07T16:12:14.476063+0000","last_became_active":"2024-04-07T16:12:13.394392+0000","last_became_peered":"2024-04-07T16:12:13.394392+0000","last_unstale":"2024-04-07T16:12:14.476063+0000","last_undegraded":"2024-04-07T16:12:14.476063+0000","last_fullsized":"2024-04-07T16:12:14.476063+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T17:52:10.950481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-07T16:12:13.368891+0000","last_change":"2024-04-07T16:12:13.045131+0000","last_active":"2024-04-07T16:12:13.368891+0000","last_peered":"2024-04-07T16:12:13.368891+0000","last_clean":"2024-04-07T16:12:13.368891+0000","last_became_active":"2024-04-07T16:12:13.044830+0000","last_became_peered":"2024-04-07T16:12:13.044830+0000","last_unstale":"2024-04-07T16:12:13.368891+0000","last_undegraded":"2024-04-07T16:12:13.368891+0000","last_fullsized":"2024-04-07T16:12:13.368891+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T02:48:50.652491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"21'117","reported_seq":219,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-07T16:12:13.375677+0000","last_change":"2024-04-07T16:11:33.518868+0000","last_active":"2024-04-07T16:12:13.375677+0000","last_peered":"2024-04-07T16:12:13.375677+0000","last_clean":"2024-04-07T16:12:13.375677+0000","last_became_active":"2024-04-07T16:11:33.518060+0000","last_became_peered":"2024-04-07T16:11:33.518060+0000","last_unstale":"2024-04-07T16:12:13.375677+0000","last_undegraded":"2024-04-07T16:12:13.375677+0000","last_fullsized":"2024-04-07T16:12:13.375677+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:07:26.866003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:07:26.866003+0000","last_clean_scrub_stamp":"2024-04-07T16:07:26.866003+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T22:22:51.043941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.476302+0000","last_change":"2024-04-07T16:12:13.406575+0000","last_active":"2024-04-07T16:12:14.476302+0000","last_peered":"2024-04-07T16:12:14.476302+0000","last_clean":"2024-04-07T16:12:14.476302+0000","last_became_active":"2024-04-07T16:12:13.404488+0000","last_became_peered":"2024-04-07T16:12:13.404488+0000","last_unstale":"2024-04-07T16:12:14.476302+0000","last_undegraded":"2024-04-07T16:12:14.476302+0000","last_fullsized":"2024-04-07T16:12:14.476302+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T00:15:22.293056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.375081+0000","last_change":"2024-04-07T16:12:13.035536+0000","last_active":"2024-04-07T16:12:14.375081+0000","last_peered":"2024-04-07T16:12:14.375081+0000","last_clean":"2024-04-07T16:12:14.375081+0000","last_became_active":"2024-04-07T16:12:13.034825+0000","last_became_peered":"2024-04-07T16:12:13.034825+0000","last_unstale":"2024-04-07T16:12:14.375081+0000","last_undegraded":"2024-04-07T16:12:14.375081+0000","last_fullsized":"2024-04-07T16:12:14.375081+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T20:11:57.033306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.375586+0000","last_change":"2024-04-07T16:12:13.035747+0000","last_active":"2024-04-07T16:12:14.375586+0000","last_peered":"2024-04-07T16:12:14.375586+0000","last_clean":"2024-04-07T16:12:14.375586+0000","last_became_active":"2024-04-07T16:12:13.035201+0000","last_became_peered":"2024-04-07T16:12:13.035201+0000","last_unstale":"2024-04-07T16:12:14.375586+0000","last_undegraded":"2024-04-07T16:12:14.375586+0000","last_fullsized":"2024-04-07T16:12:14.375586+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T21:03:25.449728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.379094+0000","last_change":"2024-04-07T16:12:13.398785+0000","last_active":"2024-04-07T16:12:14.379094+0000","last_peered":"2024-04-07T16:12:14.379094+0000","last_clean":"2024-04-07T16:12:14.379094+0000","last_became_active":"2024-04-07T16:12:13.398210+0000","last_became_peered":"2024-04-07T16:12:13.398210+0000","last_unstale":"2024-04-07T16:12:14.379094+0000","last_undegraded":"2024-04-07T16:12:14.379094+0000","last_fullsized":"2024-04-07T16:12:14.379094+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T02:11:50.752075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.379438+0000","last_change":"2024-04-07T16:12:13.039980+0000","last_active":"2024-04-07T16:12:14.379438+0000","last_peered":"2024-04-07T16:12:14.379438+0000","last_clean":"2024-04-07T16:12:14.379438+0000","last_became_active":"2024-04-07T16:12:13.039676+0000","last_became_peered":"2024-04-07T16:12:13.039676+0000","last_unstale":"2024-04-07T16:12:14.379438+0000","last_undegraded":"2024-04-07T16:12:14.379438+0000","last_fullsized":"2024-04-07T16:12:14.379438+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T02:37:05.297496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.474648+0000","last_change":"2024-04-07T16:12:13.399486+0000","last_active":"2024-04-07T16:12:14.474648+0000","last_peered":"2024-04-07T16:12:14.474648+0000","last_clean":"2024-04-07T16:12:14.474648+0000","last_became_active":"2024-04-07T16:12:13.399184+0000","last_became_peered":"2024-04-07T16:12:13.399184+0000","last_unstale":"2024-04-07T16:12:14.474648+0000","last_undegraded":"2024-04-07T16:12:14.474648+0000","last_fullsized":"2024-04-07T16:12:14.474648+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T21:07:25.104309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"61'2","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.474473+0000","last_change":"2024-04-07T16:12:13.038351+0000","last_active":"2024-04-07T16:12:14.474473+0000","last_peered":"2024-04-07T16:12:14.474473+0000","last_clean":"2024-04-07T16:12:14.474473+0000","last_became_active":"2024-04-07T16:12:13.038026+0000","last_became_peered":"2024-04-07T16:12:13.038026+0000","last_unstale":"2024-04-07T16:12:14.474473+0000","last_undegraded":"2024-04-07T16:12:14.474473+0000","last_fullsized":"2024-04-07T16:12:14.474473+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T19:39:42.796342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.476461+0000","last_change":"2024-04-07T16:12:13.406541+0000","last_active":"2024-04-07T16:12:14.476461+0000","last_peered":"2024-04-07T16:12:14.476461+0000","last_clean":"2024-04-07T16:12:14.476461+0000","last_became_active":"2024-04-07T16:12:13.406018+0000","last_became_peered":"2024-04-07T16:12:13.406018+0000","last_unstale":"2024-04-07T16:12:14.476461+0000","last_undegraded":"2024-04-07T16:12:14.476461+0000","last_fullsized":"2024-04-07T16:12:14.476461+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T00:24:42.112088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.374594+0000","last_change":"2024-04-07T16:12:13.033067+0000","last_active":"2024-04-07T16:12:14.374594+0000","last_peered":"2024-04-07T16:12:14.374594+0000","last_clean":"2024-04-07T16:12:14.374594+0000","last_became_active":"2024-04-07T16:12:13.032456+0000","last_became_peered":"2024-04-07T16:12:13.032456+0000","last_unstale":"2024-04-07T16:12:14.374594+0000","last_undegraded":"2024-04-07T16:12:14.374594+0000","last_fullsized":"2024-04-07T16:12:14.374594+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T23:02:09.025617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-07T16:12:13.408914+0000","last_change":"2024-04-07T16:12:13.404632+0000","last_active":"2024-04-07T16:12:13.408914+0000","last_peered":"2024-04-07T16:12:13.408914+0000","last_clean":"2024-04-07T16:12:13.408914+0000","last_became_active":"2024-04-07T16:12:13.404321+0000","last_became_peered":"2024-04-07T16:12:13.404321+0000","last_unstale":"2024-04-07T16:12:13.408914+0000","last_undegraded":"2024-04-07T16:12:13.408914+0000","last_fullsized":"2024-04-07T16:12:13.408914+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T19:53:50.339257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.378298+0000","last_change":"2024-04-07T16:12:13.038113+0000","last_active":"2024-04-07T16:12:14.378298+0000","last_peered":"2024-04-07T16:12:14.378298+0000","last_clean":"2024-04-07T16:12:14.378298+0000","last_became_active":"2024-04-07T16:12:13.037783+0000","last_became_peered":"2024-04-07T16:12:13.037783+0000","last_unstale":"2024-04-07T16:12:14.378298+0000","last_undegraded":"2024-04-07T16:12:14.378298+0000","last_fullsized":"2024-04-07T16:12:14.378298+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T00:58:32.085451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.474100+0000","last_change":"2024-04-07T16:12:13.392396+0000","last_active":"2024-04-07T16:12:14.474100+0000","last_peered":"2024-04-07T16:12:14.474100+0000","last_clean":"2024-04-07T16:12:14.474100+0000","last_became_active":"2024-04-07T16:12:13.392016+0000","last_became_peered":"2024-04-07T16:12:13.392016+0000","last_unstale":"2024-04-07T16:12:14.474100+0000","last_undegraded":"2024-04-07T16:12:14.474100+0000","last_fullsized":"2024-04-07T16:12:14.474100+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T02:35:50.790051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.377986+0000","last_change":"2024-04-07T16:12:13.038401+0000","last_active":"2024-04-07T16:12:14.377986+0000","last_peered":"2024-04-07T16:12:14.377986+0000","last_clean":"2024-04-07T16:12:14.377986+0000","last_became_active":"2024-04-07T16:12:13.038110+0000","last_became_peered":"2024-04-07T16:12:13.038110+0000","last_unstale":"2024-04-07T16:12:14.377986+0000","last_undegraded":"2024-04-07T16:12:14.377986+0000","last_fullsized":"2024-04-07T16:12:14.377986+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T01:16:50.295588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.474697+0000","last_change":"2024-04-07T16:12:13.397819+0000","last_active":"2024-04-07T16:12:14.474697+0000","last_peered":"2024-04-07T16:12:14.474697+0000","last_clean":"2024-04-07T16:12:14.474697+0000","last_became_active":"2024-04-07T16:12:13.397593+0000","last_became_peered":"2024-04-07T16:12:13.397593+0000","last_unstale":"2024-04-07T16:12:14.474697+0000","last_undegraded":"2024-04-07T16:12:14.474697+0000","last_fullsized":"2024-04-07T16:12:14.474697+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T02:30:01.472030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"61'2","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.375041+0000","last_change":"2024-04-07T16:12:13.043982+0000","last_active":"2024-04-07T16:12:14.375041+0000","last_peered":"2024-04-07T16:12:14.375041+0000","last_clean":"2024-04-07T16:12:14.375041+0000","last_became_active":"2024-04-07T16:12:13.041873+0000","last_became_peered":"2024-04-07T16:12:13.041873+0000","last_unstale":"2024-04-07T16:12:14.375041+0000","last_undegraded":"2024-04-07T16:12:14.375041+0000","last_fullsized":"2024-04-07T16:12:14.375041+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T20:03:35.575133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.476449+0000","last_change":"2024-04-07T16:12:13.381810+0000","last_active":"2024-04-07T16:12:14.476449+0000","last_peered":"2024-04-07T16:12:14.476449+0000","last_clean":"2024-04-07T16:12:14.476449+0000","last_became_active":"2024-04-07T16:12:13.381470+0000","last_became_peered":"2024-04-07T16:12:13.381470+0000","last_unstale":"2024-04-07T16:12:14.476449+0000","last_undegraded":"2024-04-07T16:12:14.476449+0000","last_fullsized":"2024-04-07T16:12:14.476449+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T17:26:21.450939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.474605+0000","last_change":"2024-04-07T16:12:13.038191+0000","last_active":"2024-04-07T16:12:14.474605+0000","last_peered":"2024-04-07T16:12:14.474605+0000","last_clean":"2024-04-07T16:12:14.474605+0000","last_became_active":"2024-04-07T16:12:13.037652+0000","last_became_peered":"2024-04-07T16:12:13.037652+0000","last_unstale":"2024-04-07T16:12:14.474605+0000","last_undegraded":"2024-04-07T16:12:14.474605+0000","last_fullsized":"2024-04-07T16:12:14.474605+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T17:04:59.525025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.375473+0000","last_change":"2024-04-07T16:12:13.400660+0000","last_active":"2024-04-07T16:12:14.375473+0000","last_peered":"2024-04-07T16:12:14.375473+0000","last_clean":"2024-04-07T16:12:14.375473+0000","last_became_active":"2024-04-07T16:12:13.400353+0000","last_became_peered":"2024-04-07T16:12:13.400353+0000","last_unstale":"2024-04-07T16:12:14.375473+0000","last_undegraded":"2024-04-07T16:12:14.375473+0000","last_fullsized":"2024-04-07T16:12:14.375473+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T00:47:57.573097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"61'1","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.379193+0000","last_change":"2024-04-07T16:12:13.034772+0000","last_active":"2024-04-07T16:12:14.379193+0000","last_peered":"2024-04-07T16:12:14.379193+0000","last_clean":"2024-04-07T16:12:14.379193+0000","last_became_active":"2024-04-07T16:12:13.034455+0000","last_became_peered":"2024-04-07T16:12:13.034455+0000","last_unstale":"2024-04-07T16:12:14.379193+0000","last_undegraded":"2024-04-07T16:12:14.379193+0000","last_fullsized":"2024-04-07T16:12:14.379193+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T23:10:30.557880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.475808+0000","last_change":"2024-04-07T16:12:13.400762+0000","last_active":"2024-04-07T16:12:14.475808+0000","last_peered":"2024-04-07T16:12:14.475808+0000","last_clean":"2024-04-07T16:12:14.475808+0000","last_became_active":"2024-04-07T16:12:13.398120+0000","last_became_peered":"2024-04-07T16:12:13.398120+0000","last_unstale":"2024-04-07T16:12:14.475808+0000","last_undegraded":"2024-04-07T16:12:14.475808+0000","last_fullsized":"2024-04-07T16:12:14.475808+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T22:11:21.789998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"61'2","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.474501+0000","last_change":"2024-04-07T16:12:13.040101+0000","last_active":"2024-04-07T16:12:14.474501+0000","last_peered":"2024-04-07T16:12:14.474501+0000","last_clean":"2024-04-07T16:12:14.474501+0000","last_became_active":"2024-04-07T16:12:13.039797+0000","last_became_peered":"2024-04-07T16:12:13.039797+0000","last_unstale":"2024-04-07T16:12:14.474501+0000","last_undegraded":"2024-04-07T16:12:14.474501+0000","last_fullsized":"2024-04-07T16:12:14.474501+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T03:27:06.016325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.376139+0000","last_change":"2024-04-07T16:12:13.394720+0000","last_active":"2024-04-07T16:12:14.376139+0000","last_peered":"2024-04-07T16:12:14.376139+0000","last_clean":"2024-04-07T16:12:14.376139+0000","last_became_active":"2024-04-07T16:12:13.394090+0000","last_became_peered":"2024-04-07T16:12:13.394090+0000","last_unstale":"2024-04-07T16:12:14.376139+0000","last_undegraded":"2024-04-07T16:12:14.376139+0000","last_fullsized":"2024-04-07T16:12:14.376139+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T00:47:54.579168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.473133+0000","last_change":"2024-04-07T16:12:13.396725+0000","last_active":"2024-04-07T16:12:14.473133+0000","last_peered":"2024-04-07T16:12:14.473133+0000","last_clean":"2024-04-07T16:12:14.473133+0000","last_became_active":"2024-04-07T16:12:13.395978+0000","last_became_peered":"2024-04-07T16:12:13.395978+0000","last_unstale":"2024-04-07T16:12:14.473133+0000","last_undegraded":"2024-04-07T16:12:14.473133+0000","last_fullsized":"2024-04-07T16:12:14.473133+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T21:30:43.276569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.474831+0000","last_change":"2024-04-07T16:12:13.412091+0000","last_active":"2024-04-07T16:12:14.474831+0000","last_peered":"2024-04-07T16:12:14.474831+0000","last_clean":"2024-04-07T16:12:14.474831+0000","last_became_active":"2024-04-07T16:12:13.410814+0000","last_became_peered":"2024-04-07T16:12:13.410814+0000","last_unstale":"2024-04-07T16:12:14.474831+0000","last_undegraded":"2024-04-07T16:12:14.474831+0000","last_fullsized":"2024-04-07T16:12:14.474831+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T01:48:29.034291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.475012+0000","last_change":"2024-04-07T16:12:13.400651+0000","last_active":"2024-04-07T16:12:14.475012+0000","last_peered":"2024-04-07T16:12:14.475012+0000","last_clean":"2024-04-07T16:12:14.475012+0000","last_became_active":"2024-04-07T16:12:13.399206+0000","last_became_peered":"2024-04-07T16:12:13.399206+0000","last_unstale":"2024-04-07T16:12:14.475012+0000","last_undegraded":"2024-04-07T16:12:14.475012+0000","last_fullsized":"2024-04-07T16:12:14.475012+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T22:16:50.256754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,0],"acting":[5,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.476748+0000","last_change":"2024-04-07T16:12:13.405888+0000","last_active":"2024-04-07T16:12:14.476748+0000","last_peered":"2024-04-07T16:12:14.476748+0000","last_clean":"2024-04-07T16:12:14.476748+0000","last_became_active":"2024-04-07T16:12:13.405320+0000","last_became_peered":"2024-04-07T16:12:13.405320+0000","last_unstale":"2024-04-07T16:12:14.476748+0000","last_undegraded":"2024-04-07T16:12:14.476748+0000","last_fullsized":"2024-04-07T16:12:14.476748+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T22:07:50.421903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.475244+0000","last_change":"2024-04-07T16:12:13.398730+0000","last_active":"2024-04-07T16:12:14.475244+0000","last_peered":"2024-04-07T16:12:14.475244+0000","last_clean":"2024-04-07T16:12:14.475244+0000","last_became_active":"2024-04-07T16:12:13.398259+0000","last_became_peered":"2024-04-07T16:12:13.398259+0000","last_unstale":"2024-04-07T16:12:14.475244+0000","last_undegraded":"2024-04-07T16:12:14.475244+0000","last_fullsized":"2024-04-07T16:12:14.475244+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T02:14:28.915631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,0],"acting":[3,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.379085+0000","last_change":"2024-04-07T16:12:13.391140+0000","last_active":"2024-04-07T16:12:14.379085+0000","last_peered":"2024-04-07T16:12:14.379085+0000","last_clean":"2024-04-07T16:12:14.379085+0000","last_became_active":"2024-04-07T16:12:13.390858+0000","last_became_peered":"2024-04-07T16:12:13.390858+0000","last_unstale":"2024-04-07T16:12:14.379085+0000","last_undegraded":"2024-04-07T16:12:14.379085+0000","last_fullsized":"2024-04-07T16:12:14.379085+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T22:44:20.569791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.475770+0000","last_change":"2024-04-07T16:12:13.382605+0000","last_active":"2024-04-07T16:12:14.475770+0000","last_peered":"2024-04-07T16:12:14.475770+0000","last_clean":"2024-04-07T16:12:14.475770+0000","last_became_active":"2024-04-07T16:12:13.382267+0000","last_became_peered":"2024-04-07T16:12:13.382267+0000","last_unstale":"2024-04-07T16:12:14.475770+0000","last_undegraded":"2024-04-07T16:12:14.475770+0000","last_fullsized":"2024-04-07T16:12:14.475770+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T20:14:03.942753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,4],"acting":[6,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.474734+0000","last_change":"2024-04-07T16:12:13.405175+0000","last_active":"2024-04-07T16:12:14.474734+0000","last_peered":"2024-04-07T16:12:14.474734+0000","last_clean":"2024-04-07T16:12:14.474734+0000","last_became_active":"2024-04-07T16:12:13.404734+0000","last_became_peered":"2024-04-07T16:12:13.404734+0000","last_unstale":"2024-04-07T16:12:14.474734+0000","last_undegraded":"2024-04-07T16:12:14.474734+0000","last_fullsized":"2024-04-07T16:12:14.474734+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T17:55:23.469795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.472083+0000","last_change":"2024-04-07T16:12:13.397863+0000","last_active":"2024-04-07T16:12:14.472083+0000","last_peered":"2024-04-07T16:12:14.472083+0000","last_clean":"2024-04-07T16:12:14.472083+0000","last_became_active":"2024-04-07T16:12:13.395430+0000","last_became_peered":"2024-04-07T16:12:13.395430+0000","last_unstale":"2024-04-07T16:12:14.472083+0000","last_undegraded":"2024-04-07T16:12:14.472083+0000","last_fullsized":"2024-04-07T16:12:14.472083+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T18:20:22.678341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.475578+0000","last_change":"2024-04-07T16:12:13.397638+0000","last_active":"2024-04-07T16:12:14.475578+0000","last_peered":"2024-04-07T16:12:14.475578+0000","last_clean":"2024-04-07T16:12:14.475578+0000","last_became_active":"2024-04-07T16:12:13.396766+0000","last_became_peered":"2024-04-07T16:12:13.396766+0000","last_unstale":"2024-04-07T16:12:14.475578+0000","last_undegraded":"2024-04-07T16:12:14.475578+0000","last_fullsized":"2024-04-07T16:12:14.475578+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T22:51:06.841718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.475062+0000","last_change":"2024-04-07T16:12:13.403603+0000","last_active":"2024-04-07T16:12:14.475062+0000","last_peered":"2024-04-07T16:12:14.475062+0000","last_clean":"2024-04-07T16:12:14.475062+0000","last_became_active":"2024-04-07T16:12:13.400042+0000","last_became_peered":"2024-04-07T16:12:13.400042+0000","last_unstale":"2024-04-07T16:12:14.475062+0000","last_undegraded":"2024-04-07T16:12:14.475062+0000","last_fullsized":"2024-04-07T16:12:14.475062+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T20:49:17.570678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.476432+0000","last_change":"2024-04-07T16:12:13.386718+0000","last_active":"2024-04-07T16:12:14.476432+0000","last_peered":"2024-04-07T16:12:14.476432+0000","last_clean":"2024-04-07T16:12:14.476432+0000","last_became_active":"2024-04-07T16:12:13.386422+0000","last_became_peered":"2024-04-07T16:12:13.386422+0000","last_unstale":"2024-04-07T16:12:14.476432+0000","last_undegraded":"2024-04-07T16:12:14.476432+0000","last_fullsized":"2024-04-07T16:12:14.476432+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T21:28:09.174468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1],"acting":[0,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-07T16:12:14.474174+0000","last_change":"2024-04-07T16:12:13.380077+0000","last_active":"2024-04-07T16:12:14.474174+0000","last_peered":"2024-04-07T16:12:14.474174+0000","last_clean":"2024-04-07T16:12:14.474174+0000","last_became_active":"2024-04-07T16:12:13.379758+0000","last_became_peered":"2024-04-07T16:12:13.379758+0000","last_unstale":"2024-04-07T16:12:14.474174+0000","last_undegraded":"2024-04-07T16:12:14.474174+0000","last_fullsized":"2024-04-07T16:12:14.474174+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T22:47:34.836189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":7},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":63,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":7086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":96,"acting":96,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":54,"seq":231928233996,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28224,"kb_used_data":1196,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716928,"statfs":{"total":95995035648,"available":95966134272,"internally_reserved":0,"allocated":1224704,"data_stored":850413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.212}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.55}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79600000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0740000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1339999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7290000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4510000000000001}]}]},{"osd":6,"up_from":46,"seq":197568495638,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28224,"kb_used_data":1200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716928,"statfs":{"total":95995035648,"available":95966134272,"internally_reserved":0,"allocated":1228800,"data_stored":851373,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Apr 7 16:11:42 2024","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.27,"5min":1.27,"15min":1.27},"last":0.878},{"interface":"front","average":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":1.2849999999999999,"5min":1.2849999999999999,"15min":1.2849999999999999},"last":0.67500000000000004}]},{"osd":1,"last update":"Sun Apr 7 16:11:42 2024","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.427,"5min":1.427,"15min":1.427},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.2470000000000001,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":1.173}]},{"osd":2,"last update":"Sun Apr 7 16:11:42 2024","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.359,"5min":1.359,"15min":1.359},"last":0.91900000000000004},{"interface":"front","average":{"1min":0.999,"5min":0.999,"15min":0.999},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.5029999999999999,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":1.274}]},{"osd":3,"last update":"Sun Apr 7 16:11:42 2024","interfaces":[{"interface":"back","average":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.363,"5min":1.363,"15min":1.363},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.2589999999999999,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":0.95099999999999996}]},{"osd":4,"last update":"Sun Apr 7 16:11:42 2024","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.5609999999999999,"5min":1.5609999999999999,"15min":1.5609999999999999},"last":1.026},{"interface":"front","average":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.2809999999999999,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":0.997}]},{"osd":5,"last update":"Sun Apr 7 16:11:42 2024","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.3680000000000001,"5min":1.3680000000000001,"15min":1.3680000000000001},"last":1.1499999999999999},{"interface":"front","average":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.6279999999999999,"5min":1.6279999999999999,"15min":1.6279999999999999},"last":1.347}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.087}]}]},{"osd":1,"up_from":13,"seq":55834574917,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27636,"kb_used_data":616,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717516,"statfs":{"total":95995035648,"available":95966736384,"internally_reserved":0,"allocated":630784,"data_stored":259601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Apr 7 16:11:48 2024","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.66300000000000003,"15min":0.57899999999999996},"min":{"1min":0.49299999999999999,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":1.518,"5min":1.518,"15min":1.518},"last":0.624},{"interface":"front","average":{"1min":0.871,"5min":0.74299999999999999,"15min":0.629},"min":{"1min":0.59899999999999998,"5min":0.315,"15min":0.315},"max":{"1min":1.2969999999999999,"5min":2.8620000000000001,"15min":2.8620000000000001},"last":0.876}]},{"osd":2,"last update":"Sun Apr 7 16:11:31 2024","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.73399999999999999,"15min":0.66000000000000003},"min":{"1min":0.56599999999999995,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.345,"5min":1.345,"15min":1.345},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.71099999999999997,"15min":0.65700000000000003},"min":{"1min":0.60399999999999998,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.5369999999999999,"5min":1.5369999999999999,"15min":1.5369999999999999},"last":0.73199999999999998}]},{"osd":3,"last update":"Sun Apr 7 16:11:23 2024","interfaces":[{"interface":"back","average":{"1min":1.032,"5min":0.86699999999999999,"15min":0.82299999999999995},"min":{"1min":0.64400000000000002,"5min":0.628,"15min":0.628},"max":{"1min":1.343,"5min":1.343,"15min":1.343},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.91700000000000004,"5min":0.79800000000000004,"15min":0.76200000000000001},"min":{"1min":0.66700000000000004,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.4770000000000001,"5min":1.4770000000000001,"15min":1.4770000000000001},"last":0.67300000000000004}]},{"osd":4,"last update":"Sun Apr 7 16:12:15 2024","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":0.93600000000000005,"15min":0.93000000000000005},"min":{"1min":0.71799999999999997,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.2729999999999999,"5min":1.5149999999999999,"15min":1.5149999999999999},"last":0.90300000000000002},{"interface":"front","average":{"1min":1.052,"5min":1.0580000000000001,"15min":1.046},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.3480000000000001,"5min":1.9299999999999999,"15min":1.9299999999999999},"last":1.0609999999999999}]},{"osd":5,"last update":"Sun Apr 7 16:11:55 2024","interfaces":[{"interface":"back","average":{"1min":1.151,"5min":1.23,"15min":1.2430000000000001},"min":{"1min":0.85199999999999998,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":1.6870000000000001,"5min":2.0510000000000002,"15min":2.0510000000000002},"last":1.0940000000000001},{"interface":"front","average":{"1min":1.099,"5min":1.1339999999999999,"15min":1.1399999999999999},"min":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"max":{"1min":1.577,"5min":1.8859999999999999,"15min":1.8859999999999999},"last":0.95099999999999996}]},{"osd":6,"last update":"Sun Apr 7 16:11:43 2024","interfaces":[{"interface":"back","average":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"min":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"max":{"1min":1.466,"5min":1.466,"15min":1.466},"last":0.97599999999999998},{"interface":"front","average":{"1min":1.274,"5min":1.274,"15min":1.274},"min":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"max":{"1min":1.6040000000000001,"5min":1.6040000000000001,"15min":1.6040000000000001},"last":1.024}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.008}]}]},{"osd":0,"up_from":8,"seq":34359738448,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28224,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716928,"statfs":{"total":95995035648,"available":95966134272,"internally_reserved":0,"allocated":1232896,"data_stored":850537,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Apr 7 16:11:47 2024","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.63600000000000001,"15min":0.56999999999999995},"min":{"1min":0.56000000000000005,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":1.353,"5min":1.353,"15min":1.353},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.70899999999999996,"15min":0.57299999999999995},"min":{"1min":0.58399999999999996,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.1439999999999999,"5min":1.214,"15min":1.214},"last":0.85199999999999998}]},{"osd":2,"last update":"Sun Apr 7 16:11:34 2024","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.70899999999999996,"15min":0.66700000000000004},"min":{"1min":0.61799999999999999,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.89000000000000001,"5min":0.77200000000000002,"15min":0.73699999999999999},"min":{"1min":0.626,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.2749999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.61499999999999999}]},{"osd":3,"last update":"Sun Apr 7 16:11:19 2024","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.78200000000000003,"15min":0.75700000000000001},"min":{"1min":0.67800000000000005,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.89000000000000001,"5min":0.81899999999999995,"15min":0.79900000000000004},"min":{"1min":0.68200000000000005,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.143,"5min":1.143,"15min":1.143},"last":1.0609999999999999}]},{"osd":4,"last update":"Sun Apr 7 16:12:08 2024","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":0.93600000000000005,"15min":0.92300000000000004},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.306,"5min":1.306,"15min":1.306},"last":0.94499999999999995},{"interface":"front","average":{"1min":1.0109999999999999,"5min":0.98799999999999999,"15min":0.96399999999999997},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":1.2150000000000001,"5min":1.7809999999999999,"15min":1.7809999999999999},"last":1.016}]},{"osd":5,"last update":"Sun Apr 7 16:11:56 2024","interfaces":[{"interface":"back","average":{"1min":1.0349999999999999,"5min":0.98899999999999999,"15min":0.98099999999999998},"min":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"max":{"1min":1.196,"5min":1.196,"15min":1.196},"last":0.91600000000000004},{"interface":"front","average":{"1min":1.0409999999999999,"5min":1.0249999999999999,"15min":1.0229999999999999},"min":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":1.48,"5min":1.48,"15min":1.48},"last":1.179}]},{"osd":6,"last update":"Sun Apr 7 16:11:41 2024","interfaces":[{"interface":"back","average":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"min":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":1.3600000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":1.0960000000000001},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"min":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"max":{"1min":1.3580000000000001,"5min":1.3580000000000001,"15min":1.3580000000000001},"last":0.99299999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96499999999999997}]}]},{"osd":2,"up_from":18,"seq":77309411388,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31724,"kb_used_data":608,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713428,"statfs":{"total":95995035648,"available":95962550272,"internally_reserved":0,"allocated":622592,"data_stored":259441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Apr 7 16:11:30 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.63700000000000001,"15min":0.58599999999999997},"min":{"1min":0.56499999999999995,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.96199999999999997,"5min":1.103,"15min":1.103},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.67600000000000005,"15min":0.60599999999999998},"min":{"1min":0.57299999999999995,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.75800000000000001}]},{"osd":1,"last update":"Sun Apr 7 16:11:30 2024","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.69399999999999995,"15min":0.64100000000000001},"min":{"1min":0.59099999999999997,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":1.143},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.71699999999999997,"15min":0.64900000000000002},"min":{"1min":0.54200000000000004,"5min":0.313,"15min":0.313},"max":{"1min":1.0609999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.53400000000000003}]},{"osd":3,"last update":"Sun Apr 7 16:11:18 2024","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.81699999999999995,"15min":0.81100000000000005},"min":{"1min":0.67000000000000004,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.81200000000000006,"15min":0.80200000000000005},"min":{"1min":0.66100000000000003,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.002,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.88600000000000001}]},{"osd":4,"last update":"Sun Apr 7 16:12:09 2024","interfaces":[{"interface":"back","average":{"1min":0.96699999999999997,"5min":0.93400000000000005,"15min":0.92100000000000004},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.4299999999999999,"5min":1.4299999999999999,"15min":1.4299999999999999},"last":0.96999999999999997},{"interface":"front","average":{"1min":1.018,"5min":0.95799999999999996,"15min":0.93999999999999995},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.4490000000000001,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.92400000000000004}]},{"osd":5,"last update":"Sun Apr 7 16:11:54 2024","interfaces":[{"interface":"back","average":{"1min":1.05,"5min":0.98699999999999999,"15min":0.97699999999999998},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":1.6970000000000001,"5min":1.6970000000000001,"15min":1.6970000000000001},"last":0.83299999999999996},{"interface":"front","average":{"1min":1.0780000000000001,"5min":1.046,"15min":1.04},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.6000000000000001,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":1.069}]},{"osd":6,"last update":"Sun Apr 7 16:11:39 2024","interfaces":[{"interface":"back","average":{"1min":1.05,"5min":1.05,"15min":1.05},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.397,"5min":1.397,"15min":1.397},"last":1.0009999999999999},{"interface":"front","average":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.355,"5min":1.355,"15min":1.355},"last":1.1040000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80500000000000005}]}]},{"osd":3,"up_from":25,"seq":107374182451,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27656,"kb_used_data":628,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717496,"statfs":{"total":95995035648,"available":95966715904,"internally_reserved":0,"allocated":643072,"data_stored":261109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Apr 7 16:11:24 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.67000000000000004,"15min":0.66900000000000004},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.97399999999999998,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.54400000000000004,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.327,"5min":1.327,"15min":1.327},"last":0.39100000000000001}]},{"osd":1,"last update":"Sun Apr 7 16:11:24 2024","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.77200000000000002,"15min":0.75600000000000001},"min":{"1min":0.57999999999999996,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.544,"5min":1.544,"15min":1.544},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.76600000000000001,"15min":0.75800000000000001},"min":{"1min":0.60599999999999998,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.4510000000000001,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.375}]},{"osd":2,"last update":"Sun Apr 7 16:11:24 2024","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.82499999999999996,"15min":0.80900000000000005},"min":{"1min":0.70699999999999996,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.256,"5min":1.256,"15min":1.256},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.81599999999999995,"15min":0.81399999999999995},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.286,"5min":1.286,"15min":1.286},"last":0.35999999999999999}]},{"osd":4,"last update":"Sun Apr 7 16:12:08 2024","interfaces":[{"interface":"back","average":{"1min":1.036,"5min":0.97699999999999998,"15min":0.95699999999999996},"min":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.28,"5min":1.552,"15min":1.552},"last":0.81000000000000005},{"interface":"front","average":{"1min":1.0569999999999999,"5min":0.999,"15min":0.97699999999999998},"min":{"1min":0.81999999999999995,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.3640000000000001,"5min":1.3640000000000001,"15min":1.3640000000000001},"last":0.65100000000000002}]},{"osd":5,"last update":"Sun Apr 7 16:11:54 2024","interfaces":[{"interface":"back","average":{"1min":1.081,"5min":1.0609999999999999,"15min":1.0580000000000001},"min":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":1.3180000000000001,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":0.70999999999999996},{"interface":"front","average":{"1min":1.0109999999999999,"5min":1.0349999999999999,"15min":1.0389999999999999},"min":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.2410000000000001,"5min":1.571,"15min":1.571},"last":0.66800000000000004}]},{"osd":6,"last update":"Sun Apr 7 16:11:39 2024","interfaces":[{"interface":"back","average":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"min":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"max":{"1min":1.341,"5min":1.341,"15min":1.341},"last":0.84299999999999997},{"interface":"front","average":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"min":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"max":{"1min":1.4610000000000001,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.61399999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64000000000000001}]}]},{"osd":4,"up_from":30,"seq":128849018922,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31740,"kb_used_data":624,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713412,"statfs":{"total":95995035648,"available":95962533888,"internally_reserved":0,"allocated":638976,"data_stored":260149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Apr 7 16:12:13 2024","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.84599999999999997,"15min":0.82199999999999995},"min":{"1min":0.60299999999999998,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.22,"5min":1.22,"15min":1.22},"last":1.1399999999999999},{"interface":"front","average":{"1min":0.877,"5min":0.89700000000000002,"15min":0.90300000000000002},"min":{"1min":0.60899999999999999,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.119,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.83499999999999996}]},{"osd":1,"last update":"Sun Apr 7 16:12:13 2024","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":0.84999999999999998,"15min":0.81999999999999995},"min":{"1min":0.69199999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.3260000000000001,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.623},{"interface":"front","average":{"1min":0.92300000000000004,"5min":0.86399999999999999,"15min":0.83999999999999997},"min":{"1min":0.56399999999999995,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.196,"5min":1.196,"15min":1.196},"last":0.70199999999999996}]},{"osd":2,"last update":"Sun Apr 7 16:12:13 2024","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":0.89000000000000001,"15min":0.86799999999999999},"min":{"1min":0.65300000000000002,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.1890000000000001,"5min":1.387,"15min":1.387},"last":1.0580000000000001},{"interface":"front","average":{"1min":0.871,"5min":0.88300000000000001,"15min":0.88200000000000001},"min":{"1min":0.63,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.3320000000000001,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":1.1859999999999999}]},{"osd":3,"last update":"Sun Apr 7 16:12:13 2024","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.92800000000000005,"15min":0.93300000000000005},"min":{"1min":0.79100000000000004,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":1.151,"5min":1.458,"15min":1.458},"last":0.78400000000000003},{"interface":"front","average":{"1min":1.034,"5min":0.94799999999999995,"15min":0.93000000000000005},"min":{"1min":0.71399999999999997,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.472,"5min":1.472,"15min":1.472},"last":0.93799999999999994}]},{"osd":5,"last update":"Sun Apr 7 16:11:55 2024","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":0.78200000000000003,"15min":0.75600000000000001},"min":{"1min":0.60999999999999999,"5min":0.315,"15min":0.315},"max":{"1min":1.403,"5min":1.403,"15min":1.403},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.92500000000000004,"5min":0.87,"15min":0.86099999999999999},"min":{"1min":0.57299999999999995,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.3220000000000001,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.90100000000000002}]},{"osd":6,"last update":"Sun Apr 7 16:11:42 2024","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.4139999999999999,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.4370000000000001,"5min":1.4370000000000001,"15min":1.4370000000000001},"last":0.97699999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.097}]}]},{"osd":5,"up_from":38,"seq":163208757279,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27640,"kb_used_data":620,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717512,"statfs":{"total":95995035648,"available":95966732288,"internally_reserved":0,"allocated":634880,"data_stored":260764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Apr 7 16:11:57 2024","interfaces":[{"interface":"back","average":{"1min":0.97499999999999998,"5min":0.92800000000000005,"15min":0.92000000000000004},"min":{"1min":0.68200000000000005,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.2609999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.94399999999999995,"15min":0.95699999999999996},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.153,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":0.60999999999999999}]},{"osd":1,"last update":"Sun Apr 7 16:11:57 2024","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.91500000000000004,"15min":0.92200000000000004},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.1619999999999999,"5min":1.331,"15min":1.331},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.97899999999999998,"5min":0.98899999999999999,"15min":0.99099999999999999},"min":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.204,"5min":1.45,"15min":1.45},"last":0.72199999999999998}]},{"osd":2,"last update":"Sun Apr 7 16:11:57 2024","interfaces":[{"interface":"back","average":{"1min":0.91300000000000003,"5min":0.96999999999999997,"15min":0.97999999999999998},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.3140000000000001,"5min":1.462,"15min":1.462},"last":0.753},{"interface":"front","average":{"1min":0.93500000000000005,"5min":0.96299999999999997,"15min":0.96699999999999997},"min":{"1min":0.67400000000000004,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.3879999999999999,"5min":1.472,"15min":1.472},"last":0.67000000000000004}]},{"osd":3,"last update":"Sun Apr 7 16:11:57 2024","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":1.173,"15min":1.218},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.4219999999999999,"5min":6.3860000000000001,"15min":6.3860000000000001},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.96199999999999997,"5min":1.0009999999999999,"15min":1.0069999999999999},"min":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.448,"5min":1.556,"15min":1.556},"last":0.79100000000000004}]},{"osd":4,"last update":"Sun Apr 7 16:11:57 2024","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.80900000000000005,"15min":0.80700000000000005},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.93999999999999995},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.90700000000000003,"15min":0.90900000000000003},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.22,"5min":1.4119999999999999,"15min":1.4119999999999999},"last":0.91900000000000004}]},{"osd":6,"last update":"Sun Apr 7 16:11:39 2024","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":1.155},{"interface":"front","average":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":1.544,"5min":1.544,"15min":1.544},"last":0.98899999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.256}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-07T16:12:21.226 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-07T16:12:21.226 INFO:tasks.cephadm:Adding prometheus.a on smithi144 2024-04-07T16:12:21.227 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0702e3a8-f4f8-11ee-b649-cb9ed24678a4 -- ceph orch apply prometheus '1;smithi144=a' 2024-04-07T16:12:22.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:22 smithi144 bash[23522]: cluster 2024-04-07T16:12:20.289557+0000 mgr.y (mgr.14152) 339 : cluster 0 pgmap v320: 97 pgs: 97 active+clean; 579 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 1.9 KiB/s wr, 6 op/s 2024-04-07T16:12:22.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:22 smithi144 bash[23522]: audit 2024-04-07T16:12:21.182897+0000 mgr.y (mgr.14152) 340 : audit 0 from='client.24449 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-07T16:12:22.440 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:22 smithi100 bash[21346]: cluster 2024-04-07T16:12:20.289557+0000 mgr.y (mgr.14152) 339 : cluster 0 pgmap v320: 97 pgs: 97 active+clean; 579 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 1.9 KiB/s wr, 6 op/s 2024-04-07T16:12:22.440 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:22 smithi100 bash[21346]: audit 2024-04-07T16:12:21.182897+0000 mgr.y (mgr.14152) 340 : audit 0 from='client.24449 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-07T16:12:22.440 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:22 smithi100 bash[31660]: cluster 2024-04-07T16:12:20.289557+0000 mgr.y (mgr.14152) 339 : cluster 0 pgmap v320: 97 pgs: 97 active+clean; 579 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 1.9 KiB/s wr, 6 op/s 2024-04-07T16:12:22.440 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:22 smithi100 bash[31660]: audit 2024-04-07T16:12:21.182897+0000 mgr.y (mgr.14152) 340 : audit 0 from='client.24449 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-07T16:12:24.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:24 smithi144 bash[23522]: cluster 2024-04-07T16:12:22.290537+0000 mgr.y (mgr.14152) 341 : cluster 0 pgmap v321: 97 pgs: 97 active+clean; 579 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s wr, 5 op/s 2024-04-07T16:12:24.440 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:24 smithi100 bash[21346]: cluster 2024-04-07T16:12:22.290537+0000 mgr.y (mgr.14152) 341 : cluster 0 pgmap v321: 97 pgs: 97 active+clean; 579 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s wr, 5 op/s 2024-04-07T16:12:24.440 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:24 smithi100 bash[31660]: cluster 2024-04-07T16:12:22.290537+0000 mgr.y (mgr.14152) 341 : cluster 0 pgmap v321: 97 pgs: 97 active+clean; 579 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s wr, 5 op/s 2024-04-07T16:12:25.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:25 smithi144 bash[23522]: audit 2024-04-07T16:12:24.220154+0000 mon.a (mon.0) 690 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.7", "id": [7, 4]}]: dispatch 2024-04-07T16:12:25.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:25 smithi144 bash[23522]: audit 2024-04-07T16:12:24.220888+0000 mon.a (mon.0) 691 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [0, 2]}]: dispatch 2024-04-07T16:12:25.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:25 smithi144 bash[23522]: audit 2024-04-07T16:12:24.221534+0000 mon.a (mon.0) 692 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.12", "id": [0, 2]}]: dispatch 2024-04-07T16:12:25.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:25 smithi144 bash[23522]: audit 2024-04-07T16:12:24.222157+0000 mon.a (mon.0) 693 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.15", "id": [4, 1]}]: dispatch 2024-04-07T16:12:25.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:25 smithi144 bash[23522]: audit 2024-04-07T16:12:24.222781+0000 mon.a (mon.0) 694 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [0, 2]}]: dispatch 2024-04-07T16:12:25.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:25 smithi144 bash[23522]: audit 2024-04-07T16:12:24.223406+0000 mon.a (mon.0) 695 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 1]}]: dispatch 2024-04-07T16:12:25.440 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:25 smithi100 bash[21346]: audit 2024-04-07T16:12:24.220154+0000 mon.a (mon.0) 690 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.7", "id": [7, 4]}]: dispatch 2024-04-07T16:12:25.440 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:25 smithi100 bash[21346]: audit 2024-04-07T16:12:24.220888+0000 mon.a (mon.0) 691 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [0, 2]}]: dispatch 2024-04-07T16:12:25.440 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:25 smithi100 bash[21346]: audit 2024-04-07T16:12:24.221534+0000 mon.a (mon.0) 692 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.12", "id": [0, 2]}]: dispatch 2024-04-07T16:12:25.440 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:25 smithi100 bash[21346]: audit 2024-04-07T16:12:24.222157+0000 mon.a (mon.0) 693 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.15", "id": [4, 1]}]: dispatch 2024-04-07T16:12:25.440 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:25 smithi100 bash[21346]: audit 2024-04-07T16:12:24.222781+0000 mon.a (mon.0) 694 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [0, 2]}]: dispatch 2024-04-07T16:12:25.440 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:25 smithi100 bash[21346]: audit 2024-04-07T16:12:24.223406+0000 mon.a (mon.0) 695 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 1]}]: dispatch 2024-04-07T16:12:25.441 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:25 smithi100 bash[31660]: audit 2024-04-07T16:12:24.220154+0000 mon.a (mon.0) 690 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.7", "id": [7, 4]}]: dispatch 2024-04-07T16:12:25.441 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:25 smithi100 bash[31660]: audit 2024-04-07T16:12:24.220888+0000 mon.a (mon.0) 691 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [0, 2]}]: dispatch 2024-04-07T16:12:25.441 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:25 smithi100 bash[31660]: audit 2024-04-07T16:12:24.221534+0000 mon.a (mon.0) 692 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.12", "id": [0, 2]}]: dispatch 2024-04-07T16:12:25.441 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:25 smithi100 bash[31660]: audit 2024-04-07T16:12:24.222157+0000 mon.a (mon.0) 693 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.15", "id": [4, 1]}]: dispatch 2024-04-07T16:12:25.441 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:25 smithi100 bash[31660]: audit 2024-04-07T16:12:24.222781+0000 mon.a (mon.0) 694 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [0, 2]}]: dispatch 2024-04-07T16:12:25.441 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:25 smithi100 bash[31660]: audit 2024-04-07T16:12:24.223406+0000 mon.a (mon.0) 695 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 1]}]: dispatch 2024-04-07T16:12:25.937 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/mon.b/config 2024-04-07T16:12:26.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:26 smithi144 bash[23522]: cluster 2024-04-07T16:12:24.291310+0000 mgr.y (mgr.14152) 342 : cluster 0 pgmap v322: 97 pgs: 97 active+clean; 579 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s wr, 4 op/s 2024-04-07T16:12:26.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:26 smithi144 bash[23522]: audit 2024-04-07T16:12:25.073779+0000 mon.a (mon.0) 696 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.7", "id": [7, 4]}]': finished 2024-04-07T16:12:26.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:26 smithi144 bash[23522]: audit 2024-04-07T16:12:25.073960+0000 mon.a (mon.0) 697 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [0, 2]}]': finished 2024-04-07T16:12:26.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:26 smithi144 bash[23522]: audit 2024-04-07T16:12:25.074092+0000 mon.a (mon.0) 698 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.12", "id": [0, 2]}]': finished 2024-04-07T16:12:26.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:26 smithi144 bash[23522]: audit 2024-04-07T16:12:25.074334+0000 mon.a (mon.0) 699 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.15", "id": [4, 1]}]': finished 2024-04-07T16:12:26.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:26 smithi144 bash[23522]: audit 2024-04-07T16:12:25.074538+0000 mon.a (mon.0) 700 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [0, 2]}]': finished 2024-04-07T16:12:26.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:26 smithi144 bash[23522]: audit 2024-04-07T16:12:25.074779+0000 mon.a (mon.0) 701 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 1]}]': finished 2024-04-07T16:12:26.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:26 smithi144 bash[23522]: cluster 2024-04-07T16:12:25.075001+0000 mon.a (mon.0) 702 : cluster 0 osdmap e63: 8 total, 8 up, 8 in 2024-04-07T16:12:26.440 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:26 smithi100 bash[21346]: cluster 2024-04-07T16:12:24.291310+0000 mgr.y (mgr.14152) 342 : cluster 0 pgmap v322: 97 pgs: 97 active+clean; 579 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s wr, 4 op/s 2024-04-07T16:12:26.440 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:26 smithi100 bash[21346]: audit 2024-04-07T16:12:25.073779+0000 mon.a (mon.0) 696 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.7", "id": [7, 4]}]': finished 2024-04-07T16:12:26.440 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:26 smithi100 bash[21346]: audit 2024-04-07T16:12:25.073960+0000 mon.a (mon.0) 697 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [0, 2]}]': finished 2024-04-07T16:12:26.440 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:26 smithi100 bash[21346]: audit 2024-04-07T16:12:25.074092+0000 mon.a (mon.0) 698 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.12", "id": [0, 2]}]': finished 2024-04-07T16:12:26.440 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:26 smithi100 bash[21346]: audit 2024-04-07T16:12:25.074334+0000 mon.a (mon.0) 699 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.15", "id": [4, 1]}]': finished 2024-04-07T16:12:26.440 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:26 smithi100 bash[21346]: audit 2024-04-07T16:12:25.074538+0000 mon.a (mon.0) 700 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [0, 2]}]': finished 2024-04-07T16:12:26.441 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:26 smithi100 bash[21346]: audit 2024-04-07T16:12:25.074779+0000 mon.a (mon.0) 701 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 1]}]': finished 2024-04-07T16:12:26.441 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:26 smithi100 bash[21346]: cluster 2024-04-07T16:12:25.075001+0000 mon.a (mon.0) 702 : cluster 0 osdmap e63: 8 total, 8 up, 8 in 2024-04-07T16:12:26.441 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:26 smithi100 bash[31660]: cluster 2024-04-07T16:12:24.291310+0000 mgr.y (mgr.14152) 342 : cluster 0 pgmap v322: 97 pgs: 97 active+clean; 579 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s wr, 4 op/s 2024-04-07T16:12:26.441 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:26 smithi100 bash[31660]: audit 2024-04-07T16:12:25.073779+0000 mon.a (mon.0) 696 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.7", "id": [7, 4]}]': finished 2024-04-07T16:12:26.441 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:26 smithi100 bash[31660]: audit 2024-04-07T16:12:25.073960+0000 mon.a (mon.0) 697 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [0, 2]}]': finished 2024-04-07T16:12:26.441 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:26 smithi100 bash[31660]: audit 2024-04-07T16:12:25.074092+0000 mon.a (mon.0) 698 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.12", "id": [0, 2]}]': finished 2024-04-07T16:12:26.441 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:26 smithi100 bash[31660]: audit 2024-04-07T16:12:25.074334+0000 mon.a (mon.0) 699 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.15", "id": [4, 1]}]': finished 2024-04-07T16:12:26.441 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:26 smithi100 bash[31660]: audit 2024-04-07T16:12:25.074538+0000 mon.a (mon.0) 700 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [0, 2]}]': finished 2024-04-07T16:12:26.441 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:26 smithi100 bash[31660]: audit 2024-04-07T16:12:25.074779+0000 mon.a (mon.0) 701 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 1]}]': finished 2024-04-07T16:12:26.441 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:26 smithi100 bash[31660]: cluster 2024-04-07T16:12:25.075001+0000 mon.a (mon.0) 702 : cluster 0 osdmap e63: 8 total, 8 up, 8 in 2024-04-07T16:12:27.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:27 smithi144 bash[23522]: cluster 2024-04-07T16:12:26.077058+0000 mon.a (mon.0) 703 : cluster 0 osdmap e64: 8 total, 8 up, 8 in 2024-04-07T16:12:27.440 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:27 smithi100 bash[21346]: cluster 2024-04-07T16:12:26.077058+0000 mon.a (mon.0) 703 : cluster 0 osdmap e64: 8 total, 8 up, 8 in 2024-04-07T16:12:27.440 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:27 smithi100 bash[31660]: cluster 2024-04-07T16:12:26.077058+0000 mon.a (mon.0) 703 : cluster 0 osdmap e64: 8 total, 8 up, 8 in 2024-04-07T16:12:28.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:28 smithi144 bash[23522]: cluster 2024-04-07T16:12:26.292103+0000 mgr.y (mgr.14152) 343 : cluster 0 pgmap v325: 97 pgs: 1 activating, 3 peering, 93 active+clean; 579 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 0 B/s wr, 0 op/s 2024-04-07T16:12:28.440 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:28 smithi100 bash[21346]: cluster 2024-04-07T16:12:26.292103+0000 mgr.y (mgr.14152) 343 : cluster 0 pgmap v325: 97 pgs: 1 activating, 3 peering, 93 active+clean; 579 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 0 B/s wr, 0 op/s 2024-04-07T16:12:28.440 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:28 smithi100 bash[31660]: cluster 2024-04-07T16:12:26.292103+0000 mgr.y (mgr.14152) 343 : cluster 0 pgmap v325: 97 pgs: 1 activating, 3 peering, 93 active+clean; 579 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 0 B/s wr, 0 op/s 2024-04-07T16:12:28.646 INFO:teuthology.orchestra.run.smithi144.stdout:Scheduled prometheus update... 2024-04-07T16:12:29.261 DEBUG:teuthology.orchestra.run.smithi144:prometheus.a> sudo journalctl -f -n 0 -u ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@prometheus.a.service 2024-04-07T16:12:29.263 INFO:tasks.cephadm:Adding node-exporter.a on smithi100 2024-04-07T16:12:29.263 INFO:tasks.cephadm:Adding node-exporter.b on smithi144 2024-04-07T16:12:29.264 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0702e3a8-f4f8-11ee-b649-cb9ed24678a4 -- ceph orch apply node-exporter '2;smithi100=a;smithi144=b' 2024-04-07T16:12:29.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:29 smithi100 bash[21346]: cluster 2024-04-07T16:12:28.293050+0000 mgr.y (mgr.14152) 344 : cluster 0 pgmap v326: 97 pgs: 1 activating, 3 peering, 93 active+clean; 579 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 0 B/s wr, 0 op/s 2024-04-07T16:12:29.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:29 smithi100 bash[21346]: audit 2024-04-07T16:12:28.644734+0000 mgr.y (mgr.14152) 345 : audit 0 from='client.24358 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi144=a", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:12:29.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:29 smithi100 bash[21346]: cephadm 2024-04-07T16:12:28.648200+0000 mgr.y (mgr.14152) 346 : cephadm 1 Saving service prometheus spec with placement smithi144=a;count:1 2024-04-07T16:12:29.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:29 smithi100 bash[21346]: audit 2024-04-07T16:12:28.657947+0000 mon.a (mon.0) 704 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:12:29.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:29 smithi100 bash[21346]: audit 2024-04-07T16:12:28.667561+0000 mon.a (mon.0) 705 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:12:29.941 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:29 smithi100 bash[21346]: audit 2024-04-07T16:12:29.199273+0000 mon.a (mon.0) 706 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:12:29.941 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:29 smithi100 bash[21346]: audit 2024-04-07T16:12:29.201292+0000 mon.a (mon.0) 707 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:12:29.941 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:29 smithi100 bash[21346]: audit 2024-04-07T16:12:29.214027+0000 mon.a (mon.0) 708 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:12:29.941 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:29 smithi100 bash[31660]: cluster 2024-04-07T16:12:28.293050+0000 mgr.y (mgr.14152) 344 : cluster 0 pgmap v326: 97 pgs: 1 activating, 3 peering, 93 active+clean; 579 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 0 B/s wr, 0 op/s 2024-04-07T16:12:29.941 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:29 smithi100 bash[31660]: audit 2024-04-07T16:12:28.644734+0000 mgr.y (mgr.14152) 345 : audit 0 from='client.24358 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi144=a", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:12:29.941 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:29 smithi100 bash[31660]: cephadm 2024-04-07T16:12:28.648200+0000 mgr.y (mgr.14152) 346 : cephadm 1 Saving service prometheus spec with placement smithi144=a;count:1 2024-04-07T16:12:29.941 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:29 smithi100 bash[31660]: audit 2024-04-07T16:12:28.657947+0000 mon.a (mon.0) 704 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:12:29.941 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:29 smithi100 bash[31660]: audit 2024-04-07T16:12:28.667561+0000 mon.a (mon.0) 705 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:12:29.941 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:29 smithi100 bash[31660]: audit 2024-04-07T16:12:29.199273+0000 mon.a (mon.0) 706 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:12:29.941 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:29 smithi100 bash[31660]: audit 2024-04-07T16:12:29.201292+0000 mon.a (mon.0) 707 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:12:29.941 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:29 smithi100 bash[31660]: audit 2024-04-07T16:12:29.214027+0000 mon.a (mon.0) 708 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:12:30.108 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:12:29 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:12:30.108 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:29 smithi144 bash[23522]: cluster 2024-04-07T16:12:28.293050+0000 mgr.y (mgr.14152) 344 : cluster 0 pgmap v326: 97 pgs: 1 activating, 3 peering, 93 active+clean; 579 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 0 B/s wr, 0 op/s 2024-04-07T16:12:30.108 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:29 smithi144 bash[23522]: audit 2024-04-07T16:12:28.644734+0000 mgr.y (mgr.14152) 345 : audit 0 from='client.24358 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi144=a", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:12:30.108 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:29 smithi144 bash[23522]: cephadm 2024-04-07T16:12:28.648200+0000 mgr.y (mgr.14152) 346 : cephadm 1 Saving service prometheus spec with placement smithi144=a;count:1 2024-04-07T16:12:30.108 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:29 smithi144 bash[23522]: audit 2024-04-07T16:12:28.657947+0000 mon.a (mon.0) 704 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:12:30.108 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:29 smithi144 bash[23522]: audit 2024-04-07T16:12:28.667561+0000 mon.a (mon.0) 705 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:12:30.108 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:29 smithi144 bash[23522]: audit 2024-04-07T16:12:29.199273+0000 mon.a (mon.0) 706 : audit 0 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:12:30.108 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:29 smithi144 bash[23522]: audit 2024-04-07T16:12:29.201292+0000 mon.a (mon.0) 707 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:12:30.108 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:29 smithi144 bash[23522]: audit 2024-04-07T16:12:29.214027+0000 mon.a (mon.0) 708 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:12:30.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:30 smithi100 bash[21346]: cephadm 2024-04-07T16:12:29.509863+0000 mgr.y (mgr.14152) 347 : cephadm 1 Deploying daemon prometheus.a on smithi144 2024-04-07T16:12:30.940 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:30 smithi100 bash[31660]: cephadm 2024-04-07T16:12:29.509863+0000 mgr.y (mgr.14152) 347 : cephadm 1 Deploying daemon prometheus.a on smithi144 2024-04-07T16:12:31.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:30 smithi144 bash[23522]: cephadm 2024-04-07T16:12:29.509863+0000 mgr.y (mgr.14152) 347 : cephadm 1 Deploying daemon prometheus.a on smithi144 2024-04-07T16:12:31.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:31 smithi100 bash[21346]: cluster 2024-04-07T16:12:30.293784+0000 mgr.y (mgr.14152) 348 : cluster 0 pgmap v327: 97 pgs: 1 activating, 3 peering, 93 active+clean; 579 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 0 B/s, 0 objects/s recovering 2024-04-07T16:12:31.940 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:31 smithi100 bash[31660]: cluster 2024-04-07T16:12:30.293784+0000 mgr.y (mgr.14152) 348 : cluster 0 pgmap v327: 97 pgs: 1 activating, 3 peering, 93 active+clean; 579 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 0 B/s, 0 objects/s recovering 2024-04-07T16:12:32.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:31 smithi144 bash[23522]: cluster 2024-04-07T16:12:30.293784+0000 mgr.y (mgr.14152) 348 : cluster 0 pgmap v327: 97 pgs: 1 activating, 3 peering, 93 active+clean; 579 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 0 B/s, 0 objects/s recovering 2024-04-07T16:12:33.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:33 smithi100 bash[21346]: cluster 2024-04-07T16:12:32.294865+0000 mgr.y (mgr.14152) 349 : cluster 0 pgmap v328: 97 pgs: 97 active+clean; 579 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 0 B/s, 0 objects/s recovering 2024-04-07T16:12:33.940 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:33 smithi100 bash[31660]: cluster 2024-04-07T16:12:32.294865+0000 mgr.y (mgr.14152) 349 : cluster 0 pgmap v328: 97 pgs: 97 active+clean; 579 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 0 B/s, 0 objects/s recovering 2024-04-07T16:12:34.020 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/mon.b/config 2024-04-07T16:12:34.041 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:33 smithi144 bash[23522]: cluster 2024-04-07T16:12:32.294865+0000 mgr.y (mgr.14152) 349 : cluster 0 pgmap v328: 97 pgs: 97 active+clean; 579 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 0 B/s, 0 objects/s recovering 2024-04-07T16:12:35.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:35 smithi100 bash[21346]: cluster 2024-04-07T16:12:34.295660+0000 mgr.y (mgr.14152) 350 : cluster 0 pgmap v329: 97 pgs: 97 active+clean; 579 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 0 B/s, 0 objects/s recovering 2024-04-07T16:12:35.940 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:35 smithi100 bash[31660]: cluster 2024-04-07T16:12:34.295660+0000 mgr.y (mgr.14152) 350 : cluster 0 pgmap v329: 97 pgs: 97 active+clean; 579 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 0 B/s, 0 objects/s recovering 2024-04-07T16:12:36.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:35 smithi144 bash[23522]: cluster 2024-04-07T16:12:34.295660+0000 mgr.y (mgr.14152) 350 : cluster 0 pgmap v329: 97 pgs: 97 active+clean; 579 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 0 B/s, 0 objects/s recovering 2024-04-07T16:12:37.939 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:37 smithi100 bash[21346]: cluster 2024-04-07T16:12:36.296726+0000 mgr.y (mgr.14152) 351 : cluster 0 pgmap v330: 97 pgs: 97 active+clean; 579 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 0 B/s, 0 objects/s recovering 2024-04-07T16:12:37.940 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:37 smithi100 bash[31660]: cluster 2024-04-07T16:12:36.296726+0000 mgr.y (mgr.14152) 351 : cluster 0 pgmap v330: 97 pgs: 97 active+clean; 579 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 0 B/s, 0 objects/s recovering 2024-04-07T16:12:38.060 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:37 smithi144 bash[23522]: cluster 2024-04-07T16:12:36.296726+0000 mgr.y (mgr.14152) 351 : cluster 0 pgmap v330: 97 pgs: 97 active+clean; 579 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 0 B/s, 0 objects/s recovering 2024-04-07T16:12:39.615 INFO:teuthology.orchestra.run.smithi144.stdout:Scheduled node-exporter update... 2024-04-07T16:12:39.939 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:39 smithi100 bash[21346]: cluster 2024-04-07T16:12:38.297772+0000 mgr.y (mgr.14152) 352 : cluster 0 pgmap v331: 97 pgs: 97 active+clean; 579 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 0 B/s, 0 objects/s recovering 2024-04-07T16:12:39.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:39 smithi100 bash[21346]: audit 2024-04-07T16:12:39.627029+0000 mon.a (mon.0) 709 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:12:39.940 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:39 smithi100 bash[31660]: cluster 2024-04-07T16:12:38.297772+0000 mgr.y (mgr.14152) 352 : cluster 0 pgmap v331: 97 pgs: 97 active+clean; 579 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 0 B/s, 0 objects/s recovering 2024-04-07T16:12:39.940 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:39 smithi100 bash[31660]: audit 2024-04-07T16:12:39.627029+0000 mon.a (mon.0) 709 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:12:40.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:39 smithi144 bash[23522]: cluster 2024-04-07T16:12:38.297772+0000 mgr.y (mgr.14152) 352 : cluster 0 pgmap v331: 97 pgs: 97 active+clean; 579 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 0 B/s, 0 objects/s recovering 2024-04-07T16:12:40.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:39 smithi144 bash[23522]: audit 2024-04-07T16:12:39.627029+0000 mon.a (mon.0) 709 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:12:40.528 DEBUG:teuthology.orchestra.run.smithi100:node-exporter.a> sudo journalctl -f -n 0 -u ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@node-exporter.a.service 2024-04-07T16:12:40.531 DEBUG:teuthology.orchestra.run.smithi144:node-exporter.b> sudo journalctl -f -n 0 -u ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@node-exporter.b.service 2024-04-07T16:12:40.534 INFO:tasks.cephadm:Adding alertmanager.a on smithi100 2024-04-07T16:12:40.534 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0702e3a8-f4f8-11ee-b649-cb9ed24678a4 -- ceph orch apply alertmanager '1;smithi100=a' 2024-04-07T16:12:41.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:40 smithi144 bash[23522]: audit 2024-04-07T16:12:39.615445+0000 mgr.y (mgr.14152) 353 : audit 0 from='client.24364 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi100=a;smithi144=b", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:12:41.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:40 smithi144 bash[23522]: cephadm 2024-04-07T16:12:39.619127+0000 mgr.y (mgr.14152) 354 : cephadm 1 Saving service node-exporter spec with placement smithi100=a;smithi144=b;count:2 2024-04-07T16:12:41.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:40 smithi144 bash[23522]: audit 2024-04-07T16:12:39.948723+0000 mon.a (mon.0) 710 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:12:41.439 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:40 smithi100 bash[21346]: audit 2024-04-07T16:12:39.615445+0000 mgr.y (mgr.14152) 353 : audit 0 from='client.24364 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi100=a;smithi144=b", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:12:41.440 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:40 smithi100 bash[21346]: cephadm 2024-04-07T16:12:39.619127+0000 mgr.y (mgr.14152) 354 : cephadm 1 Saving service node-exporter spec with placement smithi100=a;smithi144=b;count:2 2024-04-07T16:12:41.440 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:40 smithi100 bash[21346]: audit 2024-04-07T16:12:39.948723+0000 mon.a (mon.0) 710 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:12:41.440 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:40 smithi100 bash[31660]: audit 2024-04-07T16:12:39.615445+0000 mgr.y (mgr.14152) 353 : audit 0 from='client.24364 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi100=a;smithi144=b", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:12:41.440 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:40 smithi100 bash[31660]: cephadm 2024-04-07T16:12:39.619127+0000 mgr.y (mgr.14152) 354 : cephadm 1 Saving service node-exporter spec with placement smithi100=a;smithi144=b;count:2 2024-04-07T16:12:41.440 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:40 smithi100 bash[31660]: audit 2024-04-07T16:12:39.948723+0000 mon.a (mon.0) 710 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:12:42.238 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:41 smithi144 bash[23522]: cluster 2024-04-07T16:12:40.298662+0000 mgr.y (mgr.14152) 355 : cluster 0 pgmap v332: 97 pgs: 97 active+clean; 579 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 0 B/s, 0 objects/s recovering 2024-04-07T16:12:42.440 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:41 smithi100 bash[21346]: cluster 2024-04-07T16:12:40.298662+0000 mgr.y (mgr.14152) 355 : cluster 0 pgmap v332: 97 pgs: 97 active+clean; 579 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 0 B/s, 0 objects/s recovering 2024-04-07T16:12:42.440 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:41 smithi100 bash[31660]: cluster 2024-04-07T16:12:40.298662+0000 mgr.y (mgr.14152) 355 : cluster 0 pgmap v332: 97 pgs: 97 active+clean; 579 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 0 B/s, 0 objects/s recovering 2024-04-07T16:12:43.180 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:12:43 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:12:43.181 INFO:journalctl@ceph.osd.6.smithi144.stdout:Apr 07 16:12:43 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:12:43.181 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:43 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:12:43.181 INFO:journalctl@ceph.osd.5.smithi144.stdout:Apr 07 16:12:43 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:12:43.182 INFO:journalctl@ceph.osd.7.smithi144.stdout:Apr 07 16:12:43 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:12:43.182 INFO:journalctl@ceph.mds.b.smithi144.stdout:Apr 07 16:12:43 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:12:43.182 INFO:journalctl@ceph.osd.4.smithi144.stdout:Apr 07 16:12:43 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:12:43.440 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:43 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:12:43.440 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:12:43 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:12:43.441 INFO:journalctl@ceph.osd.5.smithi144.stdout:Apr 07 16:12:43 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:12:43.441 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:12:43 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:12:43.441 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:12:43 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:12:43.441 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:12:43 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:12:43.441 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:12:43 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:12:43.441 INFO:journalctl@ceph.osd.6.smithi144.stdout:Apr 07 16:12:43 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:12:43.442 INFO:journalctl@ceph.mds.b.smithi144.stdout:Apr 07 16:12:43 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:12:43.442 INFO:journalctl@ceph.osd.4.smithi144.stdout:Apr 07 16:12:43 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:12:43.442 INFO:journalctl@ceph.osd.7.smithi144.stdout:Apr 07 16:12:43 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:12:43.857 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:12:43 smithi144 systemd[1]: Started Ceph prometheus.a for 0702e3a8-f4f8-11ee-b649-cb9ed24678a4. 2024-04-07T16:12:44.332 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:43 smithi144 bash[23522]: cluster 2024-04-07T16:12:42.299793+0000 mgr.y (mgr.14152) 356 : cluster 0 pgmap v333: 97 pgs: 97 active+clean; 579 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:12:44.333 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:43 smithi144 bash[23522]: audit 2024-04-07T16:12:43.497594+0000 mon.a (mon.0) 711 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:12:44.333 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:43 smithi144 bash[23522]: audit 2024-04-07T16:12:43.510561+0000 mon.a (mon.0) 712 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:12:44.333 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:43 smithi144 bash[23522]: audit 2024-04-07T16:12:43.525875+0000 mon.a (mon.0) 713 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:12:44.333 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:43 smithi144 bash[23522]: audit 2024-04-07T16:12:43.534225+0000 mon.a (mon.0) 714 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-04-07T16:12:44.440 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:43 smithi100 bash[31660]: cluster 2024-04-07T16:12:42.299793+0000 mgr.y (mgr.14152) 356 : cluster 0 pgmap v333: 97 pgs: 97 active+clean; 579 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:12:44.440 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:43 smithi100 bash[31660]: audit 2024-04-07T16:12:43.497594+0000 mon.a (mon.0) 711 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:12:44.440 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:43 smithi100 bash[31660]: audit 2024-04-07T16:12:43.510561+0000 mon.a (mon.0) 712 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:12:44.440 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:43 smithi100 bash[31660]: audit 2024-04-07T16:12:43.525875+0000 mon.a (mon.0) 713 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:12:44.440 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:43 smithi100 bash[31660]: audit 2024-04-07T16:12:43.534225+0000 mon.a (mon.0) 714 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-04-07T16:12:44.441 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:43 smithi100 bash[21346]: cluster 2024-04-07T16:12:42.299793+0000 mgr.y (mgr.14152) 356 : cluster 0 pgmap v333: 97 pgs: 97 active+clean; 579 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:12:44.441 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:43 smithi100 bash[21346]: audit 2024-04-07T16:12:43.497594+0000 mon.a (mon.0) 711 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:12:44.441 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:43 smithi100 bash[21346]: audit 2024-04-07T16:12:43.510561+0000 mon.a (mon.0) 712 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:12:44.441 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:43 smithi100 bash[21346]: audit 2024-04-07T16:12:43.525875+0000 mon.a (mon.0) 713 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' 2024-04-07T16:12:44.441 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:43 smithi100 bash[21346]: audit 2024-04-07T16:12:43.534225+0000 mon.a (mon.0) 714 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-04-07T16:12:44.585 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:12:44 smithi144 bash[48586]: ts=2024-04-07T16:12:44.431Z caller=main.go:564 level=info msg="Starting Prometheus Server" mode=server version="(version=2.43.0, branch=HEAD, revision=edfc3bcd025dd6fe296c167a14a216cab1e552ee)" 2024-04-07T16:12:44.585 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:12:44 smithi144 bash[48586]: ts=2024-04-07T16:12:44.431Z caller=main.go:569 level=info build_context="(go=go1.19.7, platform=linux/amd64, user=root@8a0ee342e522, date=20230321-12:56:07, tags=netgo,builtinassets)" 2024-04-07T16:12:44.586 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:12:44 smithi144 bash[48586]: ts=2024-04-07T16:12:44.431Z caller=main.go:570 level=info host_details="(Linux 5.15.0-101-generic #111-Ubuntu SMP Tue Mar 5 20:16:58 UTC 2024 x86_64 smithi144 (none))" 2024-04-07T16:12:44.586 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:12:44 smithi144 bash[48586]: ts=2024-04-07T16:12:44.431Z caller=main.go:571 level=info fd_limits="(soft=1048576, hard=1048576)" 2024-04-07T16:12:44.586 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:12:44 smithi144 bash[48586]: ts=2024-04-07T16:12:44.431Z caller=main.go:572 level=info vm_limits="(soft=unlimited, hard=unlimited)" 2024-04-07T16:12:44.586 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:12:44 smithi144 bash[48586]: ts=2024-04-07T16:12:44.433Z caller=web.go:561 level=info component=web msg="Start listening for connections" address=:9095 2024-04-07T16:12:44.586 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:12:44 smithi144 bash[48586]: ts=2024-04-07T16:12:44.433Z caller=main.go:1005 level=info msg="Starting TSDB ..." 2024-04-07T16:12:44.586 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:12:44 smithi144 bash[48586]: ts=2024-04-07T16:12:44.434Z caller=tls_config.go:232 level=info component=web msg="Listening on" address=[::]:9095 2024-04-07T16:12:44.586 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:12:44 smithi144 bash[48586]: ts=2024-04-07T16:12:44.434Z caller=tls_config.go:235 level=info component=web msg="TLS is disabled." http2=false address=[::]:9095 2024-04-07T16:12:44.586 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:12:44 smithi144 bash[48586]: ts=2024-04-07T16:12:44.436Z caller=head.go:587 level=info component=tsdb msg="Replaying on-disk memory mappable chunks if any" 2024-04-07T16:12:44.586 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:12:44 smithi144 bash[48586]: ts=2024-04-07T16:12:44.436Z caller=head.go:658 level=info component=tsdb msg="On-disk memory mappable chunks replay completed" duration=1.915µs 2024-04-07T16:12:44.586 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:12:44 smithi144 bash[48586]: ts=2024-04-07T16:12:44.436Z caller=head.go:664 level=info component=tsdb msg="Replaying WAL, this may take a while" 2024-04-07T16:12:44.587 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:12:44 smithi144 bash[48586]: ts=2024-04-07T16:12:44.437Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=0 maxSegment=0 2024-04-07T16:12:44.587 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:12:44 smithi144 bash[48586]: ts=2024-04-07T16:12:44.437Z caller=head.go:772 level=info component=tsdb msg="WAL replay completed" checkpoint_replay_duration=35.761µs wal_replay_duration=1.298914ms wbl_replay_duration=177ns total_replay_duration=1.358656ms 2024-04-07T16:12:44.587 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:12:44 smithi144 bash[48586]: ts=2024-04-07T16:12:44.439Z caller=main.go:1026 level=info fs_type=XFS_SUPER_MAGIC 2024-04-07T16:12:44.587 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:12:44 smithi144 bash[48586]: ts=2024-04-07T16:12:44.439Z caller=main.go:1029 level=info msg="TSDB started" 2024-04-07T16:12:44.587 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:12:44 smithi144 bash[48586]: ts=2024-04-07T16:12:44.439Z caller=main.go:1209 level=info msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml 2024-04-07T16:12:44.587 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:12:44 smithi144 bash[48586]: ts=2024-04-07T16:12:44.458Z caller=main.go:1246 level=info msg="Completed loading of configuration file" filename=/etc/prometheus/prometheus.yml totalDuration=19.13367ms db_storage=924ns remote_storage=1.305µs web_handler=455ns query_engine=800ns scrape=326.573µs scrape_sd=84.239µs notify=1.121µs notify_sd=4.953µs rules=18.463523ms tracing=8.624µs 2024-04-07T16:12:44.587 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:12:44 smithi144 bash[48586]: ts=2024-04-07T16:12:44.458Z caller=main.go:990 level=info msg="Server is ready to receive web requests." 2024-04-07T16:12:44.587 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:12:44 smithi144 bash[48586]: ts=2024-04-07T16:12:44.458Z caller=manager.go:974 level=info component="rule manager" msg="Starting rule manager..." 2024-04-07T16:12:44.858 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:12:44 smithi144 bash[24472]: ignoring --setuser ceph since I am not root 2024-04-07T16:12:44.858 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:12:44 smithi144 bash[24472]: ignoring --setgroup ceph since I am not root 2024-04-07T16:12:44.859 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:12:44 smithi144 bash[24472]: debug 2024-04-07T16:12:44.593+0000 7f15b2103700 1 -- 172.21.15.144:0/1420147613 <== mon.1 v2:172.21.15.144:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56264a0841a0 con 0x56264ae06800 2024-04-07T16:12:44.859 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:12:44 smithi144 bash[24472]: debug 2024-04-07T16:12:44.737+0000 7f15bbe69200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-07T16:12:44.939 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:12:44 smithi100 bash[21601]: ignoring --setuser ceph since I am not root 2024-04-07T16:12:44.939 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:12:44 smithi100 bash[21601]: ignoring --setgroup ceph since I am not root 2024-04-07T16:12:44.940 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:12:44 smithi100 bash[21601]: debug 2024-04-07T16:12:44.734+0000 7fe780b6d200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-07T16:12:45.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:45 smithi144 bash[23522]: audit 2024-04-07T16:12:44.538284+0000 mon.a (mon.0) 715 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-04-07T16:12:45.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:45 smithi144 bash[23522]: cluster 2024-04-07T16:12:44.538345+0000 mon.a (mon.0) 716 : cluster 0 mgrmap e16: y(active, since 9m), standbys: x 2024-04-07T16:12:45.940 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:45 smithi100 bash[31660]: audit 2024-04-07T16:12:44.538284+0000 mon.a (mon.0) 715 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-04-07T16:12:45.940 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:45 smithi100 bash[31660]: cluster 2024-04-07T16:12:44.538345+0000 mon.a (mon.0) 716 : cluster 0 mgrmap e16: y(active, since 9m), standbys: x 2024-04-07T16:12:45.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:45 smithi100 bash[21346]: audit 2024-04-07T16:12:44.538284+0000 mon.a (mon.0) 715 : audit 1 from='mgr.14152 172.21.15.100:0/2215562150' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-04-07T16:12:45.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:45 smithi100 bash[21346]: cluster 2024-04-07T16:12:44.538345+0000 mon.a (mon.0) 716 : cluster 0 mgrmap e16: y(active, since 9m), standbys: x 2024-04-07T16:12:46.304 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/mon.b/config 2024-04-07T16:12:46.607 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:12:46 smithi144 bash[24472]: debug 2024-04-07T16:12:46.321+0000 7f15bbe69200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-07T16:12:46.607 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:12:46 smithi144 bash[24472]: debug 2024-04-07T16:12:46.421+0000 7f15bbe69200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-07T16:12:46.627 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:12:46 smithi100 bash[21601]: debug 2024-04-07T16:12:46.294+0000 7fe780b6d200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-07T16:12:46.627 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:12:46 smithi100 bash[21601]: debug 2024-04-07T16:12:46.390+0000 7fe780b6d200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-07T16:12:46.939 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:12:46 smithi100 bash[21601]: debug 2024-04-07T16:12:46.638+0000 7fe780b6d200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-07T16:12:47.107 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:12:46 smithi144 bash[24472]: debug 2024-04-07T16:12:46.669+0000 7f15bbe69200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-07T16:12:47.538 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:12:47 smithi144 bash[24472]: debug 2024-04-07T16:12:47.473+0000 7f15bbe69200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-07T16:12:47.689 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:12:47 smithi100 bash[21601]: debug 2024-04-07T16:12:47.418+0000 7fe780b6d200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-07T16:12:47.689 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:12:47 smithi100 bash[21601]: debug 2024-04-07T16:12:47.530+0000 7fe780b6d200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-07T16:12:47.689 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:12:47 smithi100 bash[21601]: debug 2024-04-07T16:12:47.634+0000 7fe780b6d200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-07T16:12:47.857 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:12:47 smithi144 bash[24472]: debug 2024-04-07T16:12:47.585+0000 7f15bbe69200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-07T16:12:47.857 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:12:47 smithi144 bash[24472]: debug 2024-04-07T16:12:47.689+0000 7f15bbe69200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-07T16:12:48.258 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:12:48 smithi144 bash[24472]: debug 2024-04-07T16:12:48.045+0000 7f15bbe69200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-07T16:12:48.258 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:12:48 smithi144 bash[24472]: debug 2024-04-07T16:12:48.157+0000 7f15bbe69200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-07T16:12:48.439 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:12:47 smithi100 bash[21601]: debug 2024-04-07T16:12:47.986+0000 7fe780b6d200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-07T16:12:48.439 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:12:48 smithi100 bash[21601]: debug 2024-04-07T16:12:48.102+0000 7fe780b6d200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-07T16:12:48.439 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:12:48 smithi100 bash[21601]: debug 2024-04-07T16:12:48.206+0000 7fe780b6d200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-07T16:12:48.607 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:12:48 smithi144 bash[24472]: debug 2024-04-07T16:12:48.261+0000 7f15bbe69200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-07T16:12:48.939 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:12:48 smithi100 bash[21601]: debug 2024-04-07T16:12:48.558+0000 7fe780b6d200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-07T16:12:48.939 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:12:48 smithi100 bash[21601]: debug 2024-04-07T16:12:48.798+0000 7fe780b6d200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-07T16:12:49.107 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:12:48 smithi144 bash[24472]: debug 2024-04-07T16:12:48.617+0000 7f15bbe69200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-07T16:12:49.107 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:12:48 smithi144 bash[24472]: debug 2024-04-07T16:12:48.869+0000 7f15bbe69200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-07T16:12:49.436 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:12:49 smithi100 bash[21601]: debug 2024-04-07T16:12:49.058+0000 7fe780b6d200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-07T16:12:49.436 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:12:49 smithi100 bash[21601]: debug 2024-04-07T16:12:49.170+0000 7fe780b6d200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-07T16:12:49.514 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:12:49 smithi144 bash[24472]: debug 2024-04-07T16:12:49.133+0000 7f15bbe69200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-07T16:12:49.514 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:12:49 smithi144 bash[24472]: debug 2024-04-07T16:12:49.245+0000 7f15bbe69200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-07T16:12:49.689 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:12:49 smithi100 bash[21601]: debug 2024-04-07T16:12:49.446+0000 7fe780b6d200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-07T16:12:49.689 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:12:49 smithi100 bash[21601]: debug 2024-04-07T16:12:49.690+0000 7fe780b6d200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-07T16:12:49.857 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:12:49 smithi144 bash[24472]: debug 2024-04-07T16:12:49.517+0000 7f15bbe69200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-07T16:12:49.857 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:12:49 smithi144 bash[24472]: debug 2024-04-07T16:12:49.757+0000 7f15bbe69200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-07T16:12:50.439 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:12:50 smithi100 bash[21601]: debug 2024-04-07T16:12:50.098+0000 7fe780b6d200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-07T16:12:50.439 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:12:50 smithi100 bash[21601]: debug 2024-04-07T16:12:50.218+0000 7fe780b6d200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-07T16:12:50.529 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:12:50 smithi144 bash[24472]: debug 2024-04-07T16:12:50.169+0000 7f15bbe69200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-07T16:12:50.529 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:12:50 smithi144 bash[24472]: debug 2024-04-07T16:12:50.289+0000 7f15bbe69200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-07T16:12:50.857 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:12:50 smithi144 bash[24472]: debug 2024-04-07T16:12:50.533+0000 7f15bbe69200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-07T16:12:50.939 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:12:50 smithi100 bash[21601]: debug 2024-04-07T16:12:50.466+0000 7fe780b6d200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-07T16:12:51.857 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:12:51 smithi144 bash[24472]: debug 2024-04-07T16:12:51.533+0000 7f15bbe69200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-07T16:12:51.857 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:12:51 smithi144 bash[24472]: debug 2024-04-07T16:12:51.645+0000 7f15bbe69200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-07T16:12:51.862 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:12:51 smithi100 bash[21601]: debug 2024-04-07T16:12:51.462+0000 7fe780b6d200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-07T16:12:51.862 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:12:51 smithi100 bash[21601]: debug 2024-04-07T16:12:51.578+0000 7fe780b6d200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-07T16:12:52.189 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:12:51 smithi100 bash[21601]: debug 2024-04-07T16:12:51.870+0000 7fe780b6d200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-07T16:12:52.189 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:12:51 smithi100 bash[21601]: debug 2024-04-07T16:12:51.986+0000 7fe780b6d200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-07T16:12:52.357 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:12:51 smithi144 bash[24472]: debug 2024-04-07T16:12:51.937+0000 7f15bbe69200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-07T16:12:52.357 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:12:52 smithi144 bash[24472]: debug 2024-04-07T16:12:52.049+0000 7f15bbe69200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-07T16:12:52.682 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:12:52 smithi100 bash[21601]: debug 2024-04-07T16:12:52.434+0000 7fe780b6d200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-07T16:12:52.682 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:12:52 smithi100 bash[21601]: debug 2024-04-07T16:12:52.554+0000 7fe780b6d200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-07T16:12:52.682 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:52 smithi100 bash[21346]: cluster 2024-04-07T16:12:52.577512+0000 mon.a (mon.0) 717 : cluster 1 Active manager daemon y restarted 2024-04-07T16:12:52.682 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:52 smithi100 bash[21346]: cluster 2024-04-07T16:12:52.577968+0000 mon.a (mon.0) 718 : cluster 1 Activating manager daemon y 2024-04-07T16:12:52.683 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:52 smithi100 bash[21346]: cluster 2024-04-07T16:12:52.589977+0000 mon.a (mon.0) 719 : cluster 0 osdmap e65: 8 total, 8 up, 8 in 2024-04-07T16:12:52.683 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:52 smithi100 bash[21346]: cluster 2024-04-07T16:12:52.591270+0000 mon.a (mon.0) 720 : cluster 0 mgrmap e17: y(active, starting, since 0.0134684s), standbys: x 2024-04-07T16:12:52.683 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:52 smithi100 bash[21346]: audit 2024-04-07T16:12:52.592063+0000 mon.b (mon.1) 25 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:12:52.683 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:52 smithi100 bash[21346]: audit 2024-04-07T16:12:52.592534+0000 mon.b (mon.1) 26 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:12:52.683 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:52 smithi100 bash[21346]: audit 2024-04-07T16:12:52.592962+0000 mon.b (mon.1) 27 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:12:52.683 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:52 smithi100 bash[21346]: audit 2024-04-07T16:12:52.593989+0000 mon.b (mon.1) 28 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-07T16:12:52.683 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:52 smithi100 bash[21346]: audit 2024-04-07T16:12:52.594541+0000 mon.b (mon.1) 29 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-07T16:12:52.683 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:52 smithi100 bash[21346]: audit 2024-04-07T16:12:52.595037+0000 mon.b (mon.1) 30 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-07T16:12:52.683 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:52 smithi100 bash[21346]: audit 2024-04-07T16:12:52.595447+0000 mon.b (mon.1) 31 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-07T16:12:52.683 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:52 smithi100 bash[21346]: audit 2024-04-07T16:12:52.595890+0000 mon.b (mon.1) 32 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-07T16:12:52.683 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:52 smithi100 bash[21346]: audit 2024-04-07T16:12:52.596442+0000 mon.b (mon.1) 33 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-07T16:12:52.683 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:52 smithi100 bash[21346]: audit 2024-04-07T16:12:52.596929+0000 mon.b (mon.1) 34 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-07T16:12:52.683 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:52 smithi100 bash[21346]: audit 2024-04-07T16:12:52.597443+0000 mon.b (mon.1) 35 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-07T16:12:52.683 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:52 smithi100 bash[21346]: audit 2024-04-07T16:12:52.597923+0000 mon.b (mon.1) 36 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-07T16:12:52.684 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:52 smithi100 bash[21346]: audit 2024-04-07T16:12:52.598427+0000 mon.b (mon.1) 37 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-07T16:12:52.684 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:52 smithi100 bash[21346]: audit 2024-04-07T16:12:52.598905+0000 mon.b (mon.1) 38 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-07T16:12:52.684 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:52 smithi100 bash[21346]: audit 2024-04-07T16:12:52.599430+0000 mon.b (mon.1) 39 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-07T16:12:52.684 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:52 smithi100 bash[21346]: audit 2024-04-07T16:12:52.600021+0000 mon.b (mon.1) 40 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-07T16:12:52.684 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:52 smithi100 bash[21346]: audit 2024-04-07T16:12:52.600779+0000 mon.b (mon.1) 41 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-07T16:12:52.684 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:52 smithi100 bash[21346]: audit 2024-04-07T16:12:52.602087+0000 mon.b (mon.1) 42 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-07T16:12:52.684 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:52 smithi100 bash[31660]: cluster 2024-04-07T16:12:52.577512+0000 mon.a (mon.0) 717 : cluster 1 Active manager daemon y restarted 2024-04-07T16:12:52.684 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:52 smithi100 bash[31660]: cluster 2024-04-07T16:12:52.577968+0000 mon.a (mon.0) 718 : cluster 1 Activating manager daemon y 2024-04-07T16:12:52.684 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:52 smithi100 bash[31660]: cluster 2024-04-07T16:12:52.589977+0000 mon.a (mon.0) 719 : cluster 0 osdmap e65: 8 total, 8 up, 8 in 2024-04-07T16:12:52.684 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:52 smithi100 bash[31660]: cluster 2024-04-07T16:12:52.591270+0000 mon.a (mon.0) 720 : cluster 0 mgrmap e17: y(active, starting, since 0.0134684s), standbys: x 2024-04-07T16:12:52.684 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:52 smithi100 bash[31660]: audit 2024-04-07T16:12:52.592063+0000 mon.b (mon.1) 25 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:12:52.684 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:52 smithi100 bash[31660]: audit 2024-04-07T16:12:52.592534+0000 mon.b (mon.1) 26 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:12:52.685 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:52 smithi100 bash[31660]: audit 2024-04-07T16:12:52.592962+0000 mon.b (mon.1) 27 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:12:52.685 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:52 smithi100 bash[31660]: audit 2024-04-07T16:12:52.593989+0000 mon.b (mon.1) 28 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-07T16:12:52.685 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:52 smithi100 bash[31660]: audit 2024-04-07T16:12:52.594541+0000 mon.b (mon.1) 29 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-07T16:12:52.685 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:52 smithi100 bash[31660]: audit 2024-04-07T16:12:52.595037+0000 mon.b (mon.1) 30 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-07T16:12:52.685 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:52 smithi100 bash[31660]: audit 2024-04-07T16:12:52.595447+0000 mon.b (mon.1) 31 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-07T16:12:52.685 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:52 smithi100 bash[31660]: audit 2024-04-07T16:12:52.595890+0000 mon.b (mon.1) 32 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-07T16:12:52.685 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:52 smithi100 bash[31660]: audit 2024-04-07T16:12:52.596442+0000 mon.b (mon.1) 33 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-07T16:12:52.685 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:52 smithi100 bash[31660]: audit 2024-04-07T16:12:52.596929+0000 mon.b (mon.1) 34 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-07T16:12:52.685 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:52 smithi100 bash[31660]: audit 2024-04-07T16:12:52.597443+0000 mon.b (mon.1) 35 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-07T16:12:52.685 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:52 smithi100 bash[31660]: audit 2024-04-07T16:12:52.597923+0000 mon.b (mon.1) 36 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-07T16:12:52.685 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:52 smithi100 bash[31660]: audit 2024-04-07T16:12:52.598427+0000 mon.b (mon.1) 37 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-07T16:12:52.685 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:52 smithi100 bash[31660]: audit 2024-04-07T16:12:52.598905+0000 mon.b (mon.1) 38 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-07T16:12:52.685 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:52 smithi100 bash[31660]: audit 2024-04-07T16:12:52.599430+0000 mon.b (mon.1) 39 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-07T16:12:52.685 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:52 smithi100 bash[31660]: audit 2024-04-07T16:12:52.600021+0000 mon.b (mon.1) 40 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-07T16:12:52.686 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:52 smithi100 bash[31660]: audit 2024-04-07T16:12:52.600779+0000 mon.b (mon.1) 41 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-07T16:12:52.686 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:52 smithi100 bash[31660]: audit 2024-04-07T16:12:52.602087+0000 mon.b (mon.1) 42 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-07T16:12:52.752 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:52 smithi144 bash[23522]: cluster 2024-04-07T16:12:52.577512+0000 mon.a (mon.0) 717 : cluster 1 Active manager daemon y restarted 2024-04-07T16:12:52.752 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:52 smithi144 bash[23522]: cluster 2024-04-07T16:12:52.577968+0000 mon.a (mon.0) 718 : cluster 1 Activating manager daemon y 2024-04-07T16:12:52.752 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:52 smithi144 bash[23522]: cluster 2024-04-07T16:12:52.589977+0000 mon.a (mon.0) 719 : cluster 0 osdmap e65: 8 total, 8 up, 8 in 2024-04-07T16:12:52.752 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:52 smithi144 bash[23522]: cluster 2024-04-07T16:12:52.591270+0000 mon.a (mon.0) 720 : cluster 0 mgrmap e17: y(active, starting, since 0.0134684s), standbys: x 2024-04-07T16:12:52.752 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:52 smithi144 bash[23522]: audit 2024-04-07T16:12:52.592063+0000 mon.b (mon.1) 25 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:12:52.753 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:52 smithi144 bash[23522]: audit 2024-04-07T16:12:52.592534+0000 mon.b (mon.1) 26 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:12:52.753 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:52 smithi144 bash[23522]: audit 2024-04-07T16:12:52.592962+0000 mon.b (mon.1) 27 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:12:52.753 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:52 smithi144 bash[23522]: audit 2024-04-07T16:12:52.593989+0000 mon.b (mon.1) 28 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-07T16:12:52.753 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:52 smithi144 bash[23522]: audit 2024-04-07T16:12:52.594541+0000 mon.b (mon.1) 29 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-07T16:12:52.753 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:52 smithi144 bash[23522]: audit 2024-04-07T16:12:52.595037+0000 mon.b (mon.1) 30 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-07T16:12:52.753 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:52 smithi144 bash[23522]: audit 2024-04-07T16:12:52.595447+0000 mon.b (mon.1) 31 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-07T16:12:52.753 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:52 smithi144 bash[23522]: audit 2024-04-07T16:12:52.595890+0000 mon.b (mon.1) 32 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-07T16:12:52.753 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:52 smithi144 bash[23522]: audit 2024-04-07T16:12:52.596442+0000 mon.b (mon.1) 33 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-07T16:12:52.753 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:52 smithi144 bash[23522]: audit 2024-04-07T16:12:52.596929+0000 mon.b (mon.1) 34 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-07T16:12:52.753 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:52 smithi144 bash[23522]: audit 2024-04-07T16:12:52.597443+0000 mon.b (mon.1) 35 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-07T16:12:52.753 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:52 smithi144 bash[23522]: audit 2024-04-07T16:12:52.597923+0000 mon.b (mon.1) 36 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-07T16:12:52.753 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:52 smithi144 bash[23522]: audit 2024-04-07T16:12:52.598427+0000 mon.b (mon.1) 37 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-07T16:12:52.753 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:52 smithi144 bash[23522]: audit 2024-04-07T16:12:52.598905+0000 mon.b (mon.1) 38 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-07T16:12:52.754 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:52 smithi144 bash[23522]: audit 2024-04-07T16:12:52.599430+0000 mon.b (mon.1) 39 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-07T16:12:52.754 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:52 smithi144 bash[23522]: audit 2024-04-07T16:12:52.600021+0000 mon.b (mon.1) 40 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-07T16:12:52.754 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:52 smithi144 bash[23522]: audit 2024-04-07T16:12:52.600779+0000 mon.b (mon.1) 41 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-07T16:12:52.754 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:52 smithi144 bash[23522]: audit 2024-04-07T16:12:52.602087+0000 mon.b (mon.1) 42 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-07T16:12:52.754 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:12:52 smithi144 bash[24472]: debug 2024-04-07T16:12:52.497+0000 7f15bbe69200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-07T16:12:52.754 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:12:52 smithi144 bash[24472]: debug 2024-04-07T16:12:52.621+0000 7f15bbe69200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-07T16:12:52.754 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:12:52 smithi144 bash[24472]: [07/Apr/2024:16:12:52] ENGINE Bus STARTING 2024-04-07T16:12:52.754 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:12:52 smithi144 bash[24472]: CherryPy Checker: 2024-04-07T16:12:52.754 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:12:52 smithi144 bash[24472]: The Application mounted at '' has an empty config. 2024-04-07T16:12:52.754 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:12:52 smithi144 bash[24472]: [07/Apr/2024:16:12:52] ENGINE Serving on http://:::9283 2024-04-07T16:12:52.754 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:12:52 smithi144 bash[24472]: [07/Apr/2024:16:12:52] ENGINE Bus STARTED 2024-04-07T16:12:52.939 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:12:52 smithi100 bash[21601]: [07/Apr/2024:16:12:52] ENGINE Bus STARTING 2024-04-07T16:12:52.939 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:12:52 smithi100 bash[21601]: debug 2024-04-07T16:12:52.750+0000 7fe6e38e0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-07T16:12:52.939 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:12:52 smithi100 bash[21601]: debug 2024-04-07T16:12:52.750+0000 7fe6e38e0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-07T16:12:52.939 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:12:52 smithi100 bash[21601]: debug 2024-04-07T16:12:52.750+0000 7fe6e38e0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-07T16:12:52.939 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:12:52 smithi100 bash[21601]: debug 2024-04-07T16:12:52.750+0000 7fe6e38e0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-07T16:12:52.939 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:12:52 smithi100 bash[21601]: debug 2024-04-07T16:12:52.750+0000 7fe6e38e0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-07T16:12:52.940 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:12:52 smithi100 bash[21601]: debug 2024-04-07T16:12:52.750+0000 7fe6df8d8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-07T16:12:52.940 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:12:52 smithi100 bash[21601]: debug 2024-04-07T16:12:52.754+0000 7fe6df8d8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-07T16:12:52.940 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:12:52 smithi100 bash[21601]: debug 2024-04-07T16:12:52.754+0000 7fe6df8d8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-07T16:12:52.940 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:12:52 smithi100 bash[21601]: debug 2024-04-07T16:12:52.754+0000 7fe6df8d8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-07T16:12:52.940 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:12:52 smithi100 bash[21601]: debug 2024-04-07T16:12:52.754+0000 7fe6df8d8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-07T16:12:52.940 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:12:52 smithi100 bash[21601]: CherryPy Checker: 2024-04-07T16:12:52.940 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:12:52 smithi100 bash[21601]: The Application mounted at '' has an empty config. 2024-04-07T16:12:52.940 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:12:52 smithi100 bash[21601]: [07/Apr/2024:16:12:52] ENGINE Serving on http://:::9283 2024-04-07T16:12:52.940 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:12:52 smithi100 bash[21601]: [07/Apr/2024:16:12:52] ENGINE Bus STARTED 2024-04-07T16:12:53.675 INFO:teuthology.orchestra.run.smithi144.stdout:Scheduled alertmanager update... 2024-04-07T16:12:53.888 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:53 smithi144 bash[23522]: cluster 2024-04-07T16:12:52.640894+0000 mon.a (mon.0) 721 : cluster 1 Manager daemon y is now available 2024-04-07T16:12:53.888 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:53 smithi144 bash[23522]: cluster 2024-04-07T16:12:52.655926+0000 mon.a (mon.0) 722 : cluster 0 Standby manager daemon x restarted 2024-04-07T16:12:53.889 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:53 smithi144 bash[23522]: audit 2024-04-07T16:12:52.655982+0000 mon.b (mon.1) 43 : audit 0 from='mgr.? 172.21.15.144:0/2473343730' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-04-07T16:12:53.889 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:53 smithi144 bash[23522]: cluster 2024-04-07T16:12:52.656102+0000 mon.a (mon.0) 723 : cluster 0 Standby manager daemon x started 2024-04-07T16:12:53.889 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:53 smithi144 bash[23522]: audit 2024-04-07T16:12:52.657498+0000 mon.b (mon.1) 44 : audit 0 from='mgr.? 172.21.15.144:0/2473343730' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-07T16:12:53.889 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:53 smithi144 bash[23522]: audit 2024-04-07T16:12:52.658159+0000 mon.b (mon.1) 45 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:12:53.889 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:53 smithi144 bash[23522]: audit 2024-04-07T16:12:52.660969+0000 mon.b (mon.1) 46 : audit 0 from='mgr.? 172.21.15.144:0/2473343730' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-04-07T16:12:53.889 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:53 smithi144 bash[23522]: audit 2024-04-07T16:12:52.661395+0000 mon.b (mon.1) 47 : audit 0 from='mgr.? 172.21.15.144:0/2473343730' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-07T16:12:53.889 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:53 smithi144 bash[23522]: audit 2024-04-07T16:12:52.685907+0000 mon.a (mon.0) 724 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:12:53.889 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:53 smithi144 bash[23522]: audit 2024-04-07T16:12:52.689549+0000 mon.b (mon.1) 48 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:12:53.889 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:53 smithi144 bash[23522]: audit 2024-04-07T16:12:52.698893+0000 mon.b (mon.1) 49 : audit 1 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-04-07T16:12:53.889 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:53 smithi144 bash[23522]: audit 2024-04-07T16:12:52.703605+0000 mon.a (mon.0) 725 : audit 1 from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-04-07T16:12:53.889 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:53 smithi144 bash[23522]: audit 2024-04-07T16:12:52.720797+0000 mon.b (mon.1) 50 : audit 1 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-04-07T16:12:53.889 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:53 smithi144 bash[23522]: audit 2024-04-07T16:12:52.725615+0000 mon.a (mon.0) 726 : audit 1 from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-04-07T16:12:53.889 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:53 smithi144 bash[23522]: cephadm 2024-04-07T16:12:53.203149+0000 mgr.y (mgr.14553) 1 : cephadm 1 [07/Apr/2024:16:12:53] ENGINE Bus STARTING 2024-04-07T16:12:53.889 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:53 smithi144 bash[23522]: cephadm 2024-04-07T16:12:53.305116+0000 mgr.y (mgr.14553) 2 : cephadm 1 [07/Apr/2024:16:12:53] ENGINE Serving on http://172.21.15.100:8765 2024-04-07T16:12:53.889 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:53 smithi144 bash[23522]: cephadm 2024-04-07T16:12:53.422301+0000 mgr.y (mgr.14553) 3 : cephadm 1 [07/Apr/2024:16:12:53] ENGINE Serving on https://172.21.15.100:7150 2024-04-07T16:12:53.889 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:53 smithi144 bash[23522]: cephadm 2024-04-07T16:12:53.422486+0000 mgr.y (mgr.14553) 4 : cephadm 1 [07/Apr/2024:16:12:53] ENGINE Bus STARTED 2024-04-07T16:12:53.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:53 smithi100 bash[21346]: cluster 2024-04-07T16:12:52.640894+0000 mon.a (mon.0) 721 : cluster 1 Manager daemon y is now available 2024-04-07T16:12:53.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:53 smithi100 bash[21346]: cluster 2024-04-07T16:12:52.655926+0000 mon.a (mon.0) 722 : cluster 0 Standby manager daemon x restarted 2024-04-07T16:12:53.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:53 smithi100 bash[21346]: audit 2024-04-07T16:12:52.655982+0000 mon.b (mon.1) 43 : audit 0 from='mgr.? 172.21.15.144:0/2473343730' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-04-07T16:12:53.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:53 smithi100 bash[21346]: cluster 2024-04-07T16:12:52.656102+0000 mon.a (mon.0) 723 : cluster 0 Standby manager daemon x started 2024-04-07T16:12:53.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:53 smithi100 bash[21346]: audit 2024-04-07T16:12:52.657498+0000 mon.b (mon.1) 44 : audit 0 from='mgr.? 172.21.15.144:0/2473343730' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-07T16:12:53.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:53 smithi100 bash[21346]: audit 2024-04-07T16:12:52.658159+0000 mon.b (mon.1) 45 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:12:53.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:53 smithi100 bash[21346]: audit 2024-04-07T16:12:52.660969+0000 mon.b (mon.1) 46 : audit 0 from='mgr.? 172.21.15.144:0/2473343730' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-04-07T16:12:53.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:53 smithi100 bash[21346]: audit 2024-04-07T16:12:52.661395+0000 mon.b (mon.1) 47 : audit 0 from='mgr.? 172.21.15.144:0/2473343730' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-07T16:12:53.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:53 smithi100 bash[21346]: audit 2024-04-07T16:12:52.685907+0000 mon.a (mon.0) 724 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:12:53.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:53 smithi100 bash[21346]: audit 2024-04-07T16:12:52.689549+0000 mon.b (mon.1) 48 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:12:53.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:53 smithi100 bash[21346]: audit 2024-04-07T16:12:52.698893+0000 mon.b (mon.1) 49 : audit 1 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-04-07T16:12:53.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:53 smithi100 bash[21346]: audit 2024-04-07T16:12:52.703605+0000 mon.a (mon.0) 725 : audit 1 from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-04-07T16:12:53.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:53 smithi100 bash[21346]: audit 2024-04-07T16:12:52.720797+0000 mon.b (mon.1) 50 : audit 1 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-04-07T16:12:53.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:53 smithi100 bash[21346]: audit 2024-04-07T16:12:52.725615+0000 mon.a (mon.0) 726 : audit 1 from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-04-07T16:12:53.941 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:53 smithi100 bash[21346]: cephadm 2024-04-07T16:12:53.203149+0000 mgr.y (mgr.14553) 1 : cephadm 1 [07/Apr/2024:16:12:53] ENGINE Bus STARTING 2024-04-07T16:12:53.941 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:53 smithi100 bash[21346]: cephadm 2024-04-07T16:12:53.305116+0000 mgr.y (mgr.14553) 2 : cephadm 1 [07/Apr/2024:16:12:53] ENGINE Serving on http://172.21.15.100:8765 2024-04-07T16:12:53.941 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:53 smithi100 bash[21346]: cephadm 2024-04-07T16:12:53.422301+0000 mgr.y (mgr.14553) 3 : cephadm 1 [07/Apr/2024:16:12:53] ENGINE Serving on https://172.21.15.100:7150 2024-04-07T16:12:53.941 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:53 smithi100 bash[21346]: cephadm 2024-04-07T16:12:53.422486+0000 mgr.y (mgr.14553) 4 : cephadm 1 [07/Apr/2024:16:12:53] ENGINE Bus STARTED 2024-04-07T16:12:53.941 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:53 smithi100 bash[31660]: cluster 2024-04-07T16:12:52.640894+0000 mon.a (mon.0) 721 : cluster 1 Manager daemon y is now available 2024-04-07T16:12:53.941 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:53 smithi100 bash[31660]: cluster 2024-04-07T16:12:52.655926+0000 mon.a (mon.0) 722 : cluster 0 Standby manager daemon x restarted 2024-04-07T16:12:53.941 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:53 smithi100 bash[31660]: audit 2024-04-07T16:12:52.655982+0000 mon.b (mon.1) 43 : audit 0 from='mgr.? 172.21.15.144:0/2473343730' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-04-07T16:12:53.941 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:53 smithi100 bash[31660]: cluster 2024-04-07T16:12:52.656102+0000 mon.a (mon.0) 723 : cluster 0 Standby manager daemon x started 2024-04-07T16:12:53.941 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:53 smithi100 bash[31660]: audit 2024-04-07T16:12:52.657498+0000 mon.b (mon.1) 44 : audit 0 from='mgr.? 172.21.15.144:0/2473343730' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-07T16:12:53.941 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:53 smithi100 bash[31660]: audit 2024-04-07T16:12:52.658159+0000 mon.b (mon.1) 45 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:12:53.941 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:53 smithi100 bash[31660]: audit 2024-04-07T16:12:52.660969+0000 mon.b (mon.1) 46 : audit 0 from='mgr.? 172.21.15.144:0/2473343730' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-04-07T16:12:53.941 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:53 smithi100 bash[31660]: audit 2024-04-07T16:12:52.661395+0000 mon.b (mon.1) 47 : audit 0 from='mgr.? 172.21.15.144:0/2473343730' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-07T16:12:53.942 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:53 smithi100 bash[31660]: audit 2024-04-07T16:12:52.685907+0000 mon.a (mon.0) 724 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:12:53.942 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:53 smithi100 bash[31660]: audit 2024-04-07T16:12:52.689549+0000 mon.b (mon.1) 48 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:12:53.942 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:53 smithi100 bash[31660]: audit 2024-04-07T16:12:52.698893+0000 mon.b (mon.1) 49 : audit 1 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-04-07T16:12:53.942 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:53 smithi100 bash[31660]: audit 2024-04-07T16:12:52.703605+0000 mon.a (mon.0) 725 : audit 1 from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-04-07T16:12:53.942 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:53 smithi100 bash[31660]: audit 2024-04-07T16:12:52.720797+0000 mon.b (mon.1) 50 : audit 1 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-04-07T16:12:53.942 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:53 smithi100 bash[31660]: audit 2024-04-07T16:12:52.725615+0000 mon.a (mon.0) 726 : audit 1 from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-04-07T16:12:53.942 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:53 smithi100 bash[31660]: cephadm 2024-04-07T16:12:53.203149+0000 mgr.y (mgr.14553) 1 : cephadm 1 [07/Apr/2024:16:12:53] ENGINE Bus STARTING 2024-04-07T16:12:53.942 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:53 smithi100 bash[31660]: cephadm 2024-04-07T16:12:53.305116+0000 mgr.y (mgr.14553) 2 : cephadm 1 [07/Apr/2024:16:12:53] ENGINE Serving on http://172.21.15.100:8765 2024-04-07T16:12:53.942 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:53 smithi100 bash[31660]: cephadm 2024-04-07T16:12:53.422301+0000 mgr.y (mgr.14553) 3 : cephadm 1 [07/Apr/2024:16:12:53] ENGINE Serving on https://172.21.15.100:7150 2024-04-07T16:12:53.942 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:53 smithi100 bash[31660]: cephadm 2024-04-07T16:12:53.422486+0000 mgr.y (mgr.14553) 4 : cephadm 1 [07/Apr/2024:16:12:53] ENGINE Bus STARTED 2024-04-07T16:12:54.416 DEBUG:teuthology.orchestra.run.smithi100:alertmanager.a> sudo journalctl -f -n 0 -u ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@alertmanager.a.service 2024-04-07T16:12:54.419 INFO:tasks.cephadm:Adding grafana.a on smithi144 2024-04-07T16:12:54.419 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0702e3a8-f4f8-11ee-b649-cb9ed24678a4 -- ceph orch apply grafana '1;smithi144=a' 2024-04-07T16:12:54.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:54 smithi100 bash[31660]: cluster 2024-04-07T16:12:53.661240+0000 mon.a (mon.0) 727 : cluster 0 mgrmap e18: y(active, since 1.08341s), standbys: x 2024-04-07T16:12:54.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:54 smithi100 bash[31660]: audit 2024-04-07T16:12:53.672661+0000 mgr.y (mgr.14553) 5 : audit 0 from='client.24379 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi100=a", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:12:54.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:54 smithi100 bash[31660]: cephadm 2024-04-07T16:12:53.678772+0000 mgr.y (mgr.14553) 6 : cephadm 1 Saving service alertmanager spec with placement smithi100=a;count:1 2024-04-07T16:12:54.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:54 smithi100 bash[31660]: cluster 2024-04-07T16:12:53.679198+0000 mgr.y (mgr.14553) 7 : cluster 0 pgmap v3: 97 pgs: 97 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:12:54.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:54 smithi100 bash[31660]: audit 2024-04-07T16:12:53.686694+0000 mon.a (mon.0) 728 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:12:54.690 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:54 smithi100 bash[21346]: cluster 2024-04-07T16:12:53.661240+0000 mon.a (mon.0) 727 : cluster 0 mgrmap e18: y(active, since 1.08341s), standbys: x 2024-04-07T16:12:54.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:54 smithi100 bash[21346]: audit 2024-04-07T16:12:53.672661+0000 mgr.y (mgr.14553) 5 : audit 0 from='client.24379 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi100=a", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:12:54.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:54 smithi100 bash[21346]: cephadm 2024-04-07T16:12:53.678772+0000 mgr.y (mgr.14553) 6 : cephadm 1 Saving service alertmanager spec with placement smithi100=a;count:1 2024-04-07T16:12:54.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:54 smithi100 bash[21346]: cluster 2024-04-07T16:12:53.679198+0000 mgr.y (mgr.14553) 7 : cluster 0 pgmap v3: 97 pgs: 97 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:12:54.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:54 smithi100 bash[21346]: audit 2024-04-07T16:12:53.686694+0000 mon.a (mon.0) 728 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:12:54.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:54 smithi144 bash[23522]: cluster 2024-04-07T16:12:53.661240+0000 mon.a (mon.0) 727 : cluster 0 mgrmap e18: y(active, since 1.08341s), standbys: x 2024-04-07T16:12:54.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:54 smithi144 bash[23522]: audit 2024-04-07T16:12:53.672661+0000 mgr.y (mgr.14553) 5 : audit 0 from='client.24379 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi100=a", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:12:54.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:54 smithi144 bash[23522]: cephadm 2024-04-07T16:12:53.678772+0000 mgr.y (mgr.14553) 6 : cephadm 1 Saving service alertmanager spec with placement smithi100=a;count:1 2024-04-07T16:12:54.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:54 smithi144 bash[23522]: cluster 2024-04-07T16:12:53.679198+0000 mgr.y (mgr.14553) 7 : cluster 0 pgmap v3: 97 pgs: 97 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:12:54.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:54 smithi144 bash[23522]: audit 2024-04-07T16:12:53.686694+0000 mon.a (mon.0) 728 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:12:55.439 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:12:55 smithi100 bash[21601]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:12:55] "GET /metrics HTTP/1.1" 200 28773 "" "Prometheus/2.43.0" 2024-04-07T16:12:55.939 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:55 smithi100 bash[21346]: cluster 2024-04-07T16:12:54.600692+0000 mgr.y (mgr.14553) 8 : cluster 0 pgmap v4: 97 pgs: 97 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:12:55.939 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:55 smithi100 bash[21346]: cluster 2024-04-07T16:12:54.696598+0000 mon.a (mon.0) 729 : cluster 0 mgrmap e19: y(active, since 2s), standbys: x 2024-04-07T16:12:55.940 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:55 smithi100 bash[31660]: cluster 2024-04-07T16:12:54.600692+0000 mgr.y (mgr.14553) 8 : cluster 0 pgmap v4: 97 pgs: 97 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:12:55.940 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:55 smithi100 bash[31660]: cluster 2024-04-07T16:12:54.696598+0000 mon.a (mon.0) 729 : cluster 0 mgrmap e19: y(active, since 2s), standbys: x 2024-04-07T16:12:56.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:55 smithi144 bash[23522]: cluster 2024-04-07T16:12:54.600692+0000 mgr.y (mgr.14553) 8 : cluster 0 pgmap v4: 97 pgs: 97 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:12:56.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:55 smithi144 bash[23522]: cluster 2024-04-07T16:12:54.696598+0000 mon.a (mon.0) 729 : cluster 0 mgrmap e19: y(active, since 2s), standbys: x 2024-04-07T16:12:57.939 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:57 smithi100 bash[21346]: cluster 2024-04-07T16:12:56.601344+0000 mgr.y (mgr.14553) 9 : cluster 0 pgmap v5: 97 pgs: 97 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:12:57.939 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:57 smithi100 bash[21346]: cluster 2024-04-07T16:12:56.703918+0000 mon.a (mon.0) 730 : cluster 0 mgrmap e20: y(active, since 4s), standbys: x 2024-04-07T16:12:57.939 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:57 smithi100 bash[31660]: cluster 2024-04-07T16:12:56.601344+0000 mgr.y (mgr.14553) 9 : cluster 0 pgmap v5: 97 pgs: 97 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:12:57.940 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:57 smithi100 bash[31660]: cluster 2024-04-07T16:12:56.703918+0000 mon.a (mon.0) 730 : cluster 0 mgrmap e20: y(active, since 4s), standbys: x 2024-04-07T16:12:58.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:57 smithi144 bash[23522]: cluster 2024-04-07T16:12:56.601344+0000 mgr.y (mgr.14553) 9 : cluster 0 pgmap v5: 97 pgs: 97 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:12:58.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:57 smithi144 bash[23522]: cluster 2024-04-07T16:12:56.703918+0000 mon.a (mon.0) 730 : cluster 0 mgrmap e20: y(active, since 4s), standbys: x 2024-04-07T16:12:58.937 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/mon.b/config 2024-04-07T16:12:59.608 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:59 smithi144 bash[23522]: audit 2024-04-07T16:12:58.289640+0000 mon.a (mon.0) 731 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:12:59.608 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:59 smithi144 bash[23522]: audit 2024-04-07T16:12:58.296484+0000 mon.a (mon.0) 732 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:12:59.608 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:59 smithi144 bash[23522]: audit 2024-04-07T16:12:58.450193+0000 mon.a (mon.0) 733 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:12:59.608 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:59 smithi144 bash[23522]: audit 2024-04-07T16:12:58.466129+0000 mon.a (mon.0) 734 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:12:59.608 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:59 smithi144 bash[23522]: audit 2024-04-07T16:12:59.061703+0000 mon.a (mon.0) 735 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:12:59.608 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:59 smithi144 bash[23522]: audit 2024-04-07T16:12:59.070147+0000 mon.a (mon.0) 736 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:12:59.608 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:59 smithi144 bash[23522]: audit 2024-04-07T16:12:59.070854+0000 mon.b (mon.1) 51 : audit 1 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-07T16:12:59.608 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:59 smithi144 bash[23522]: audit 2024-04-07T16:12:59.073465+0000 mon.b (mon.1) 52 : audit 1 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-07T16:12:59.608 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:59 smithi144 bash[23522]: audit 2024-04-07T16:12:59.075675+0000 mon.b (mon.1) 53 : audit 1 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-07T16:12:59.608 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:59 smithi144 bash[23522]: audit 2024-04-07T16:12:59.075733+0000 mon.a (mon.0) 737 : audit 1 from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-07T16:12:59.608 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:59 smithi144 bash[23522]: audit 2024-04-07T16:12:59.077836+0000 mon.b (mon.1) 54 : audit 1 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-07T16:12:59.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:59 smithi144 bash[23522]: audit 2024-04-07T16:12:59.078313+0000 mon.a (mon.0) 738 : audit 1 from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-07T16:12:59.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:59 smithi144 bash[23522]: audit 2024-04-07T16:12:59.080480+0000 mon.a (mon.0) 739 : audit 1 from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-07T16:12:59.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:59 smithi144 bash[23522]: audit 2024-04-07T16:12:59.082629+0000 mon.a (mon.0) 740 : audit 1 from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-07T16:12:59.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:59 smithi144 bash[23522]: audit 2024-04-07T16:12:59.090748+0000 mon.a (mon.0) 741 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:12:59.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:12:59 smithi144 bash[23522]: audit 2024-04-07T16:12:59.272721+0000 mon.a (mon.0) 742 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:12:59.689 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:59 smithi100 bash[21346]: audit 2024-04-07T16:12:58.289640+0000 mon.a (mon.0) 731 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:12:59.689 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:59 smithi100 bash[21346]: audit 2024-04-07T16:12:58.296484+0000 mon.a (mon.0) 732 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:12:59.690 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:59 smithi100 bash[21346]: audit 2024-04-07T16:12:58.450193+0000 mon.a (mon.0) 733 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:12:59.690 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:59 smithi100 bash[21346]: audit 2024-04-07T16:12:58.466129+0000 mon.a (mon.0) 734 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:12:59.690 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:59 smithi100 bash[21346]: audit 2024-04-07T16:12:59.061703+0000 mon.a (mon.0) 735 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:12:59.690 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:59 smithi100 bash[21346]: audit 2024-04-07T16:12:59.070147+0000 mon.a (mon.0) 736 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:12:59.690 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:59 smithi100 bash[21346]: audit 2024-04-07T16:12:59.070854+0000 mon.b (mon.1) 51 : audit 1 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-07T16:12:59.690 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:59 smithi100 bash[21346]: audit 2024-04-07T16:12:59.073465+0000 mon.b (mon.1) 52 : audit 1 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-07T16:12:59.690 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:59 smithi100 bash[21346]: audit 2024-04-07T16:12:59.075675+0000 mon.b (mon.1) 53 : audit 1 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-07T16:12:59.690 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:59 smithi100 bash[21346]: audit 2024-04-07T16:12:59.075733+0000 mon.a (mon.0) 737 : audit 1 from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-07T16:12:59.690 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:59 smithi100 bash[21346]: audit 2024-04-07T16:12:59.077836+0000 mon.b (mon.1) 54 : audit 1 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-07T16:12:59.690 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:59 smithi100 bash[21346]: audit 2024-04-07T16:12:59.078313+0000 mon.a (mon.0) 738 : audit 1 from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-07T16:12:59.690 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:59 smithi100 bash[21346]: audit 2024-04-07T16:12:59.080480+0000 mon.a (mon.0) 739 : audit 1 from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-07T16:12:59.690 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:59 smithi100 bash[21346]: audit 2024-04-07T16:12:59.082629+0000 mon.a (mon.0) 740 : audit 1 from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-07T16:12:59.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:59 smithi100 bash[21346]: audit 2024-04-07T16:12:59.090748+0000 mon.a (mon.0) 741 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:12:59.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:12:59 smithi100 bash[21346]: audit 2024-04-07T16:12:59.272721+0000 mon.a (mon.0) 742 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:12:59.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:59 smithi100 bash[31660]: audit 2024-04-07T16:12:58.289640+0000 mon.a (mon.0) 731 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:12:59.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:59 smithi100 bash[31660]: audit 2024-04-07T16:12:58.296484+0000 mon.a (mon.0) 732 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:12:59.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:59 smithi100 bash[31660]: audit 2024-04-07T16:12:58.450193+0000 mon.a (mon.0) 733 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:12:59.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:59 smithi100 bash[31660]: audit 2024-04-07T16:12:58.466129+0000 mon.a (mon.0) 734 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:12:59.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:59 smithi100 bash[31660]: audit 2024-04-07T16:12:59.061703+0000 mon.a (mon.0) 735 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:12:59.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:59 smithi100 bash[31660]: audit 2024-04-07T16:12:59.070147+0000 mon.a (mon.0) 736 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:12:59.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:59 smithi100 bash[31660]: audit 2024-04-07T16:12:59.070854+0000 mon.b (mon.1) 51 : audit 1 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-07T16:12:59.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:59 smithi100 bash[31660]: audit 2024-04-07T16:12:59.073465+0000 mon.b (mon.1) 52 : audit 1 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-07T16:12:59.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:59 smithi100 bash[31660]: audit 2024-04-07T16:12:59.075675+0000 mon.b (mon.1) 53 : audit 1 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-07T16:12:59.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:59 smithi100 bash[31660]: audit 2024-04-07T16:12:59.075733+0000 mon.a (mon.0) 737 : audit 1 from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-07T16:12:59.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:59 smithi100 bash[31660]: audit 2024-04-07T16:12:59.077836+0000 mon.b (mon.1) 54 : audit 1 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-07T16:12:59.692 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:59 smithi100 bash[31660]: audit 2024-04-07T16:12:59.078313+0000 mon.a (mon.0) 738 : audit 1 from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-07T16:12:59.692 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:59 smithi100 bash[31660]: audit 2024-04-07T16:12:59.080480+0000 mon.a (mon.0) 739 : audit 1 from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-07T16:12:59.692 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:59 smithi100 bash[31660]: audit 2024-04-07T16:12:59.082629+0000 mon.a (mon.0) 740 : audit 1 from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-07T16:12:59.692 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:59 smithi100 bash[31660]: audit 2024-04-07T16:12:59.090748+0000 mon.a (mon.0) 741 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:12:59.692 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:12:59 smithi100 bash[31660]: audit 2024-04-07T16:12:59.272721+0000 mon.a (mon.0) 742 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:13:00.549 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:13:00 smithi144 bash[23522]: cluster 2024-04-07T16:12:58.601994+0000 mgr.y (mgr.14553) 10 : cluster 0 pgmap v6: 97 pgs: 97 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:13:00.549 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:13:00 smithi144 bash[23522]: cephadm 2024-04-07T16:12:59.083800+0000 mgr.y (mgr.14553) 11 : cephadm 1 Adjusting osd_memory_target on smithi100 to 3036M 2024-04-07T16:13:00.549 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:13:00 smithi144 bash[23522]: audit 2024-04-07T16:12:59.288153+0000 mon.a (mon.0) 743 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:13:00.549 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:13:00 smithi144 bash[23522]: audit 2024-04-07T16:12:59.291633+0000 mon.b (mon.1) 55 : audit 1 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-07T16:13:00.549 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:13:00 smithi144 bash[23522]: audit 2024-04-07T16:12:59.297929+0000 mon.b (mon.1) 56 : audit 1 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-07T16:13:00.549 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:13:00 smithi144 bash[23522]: audit 2024-04-07T16:12:59.300052+0000 mon.a (mon.0) 744 : audit 1 from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-07T16:13:00.550 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:13:00 smithi144 bash[23522]: audit 2024-04-07T16:12:59.300446+0000 mon.b (mon.1) 57 : audit 1 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-07T16:13:00.550 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:13:00 smithi144 bash[23522]: audit 2024-04-07T16:12:59.302910+0000 mon.a (mon.0) 745 : audit 1 from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-07T16:13:00.550 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:13:00 smithi144 bash[23522]: audit 2024-04-07T16:12:59.303038+0000 mon.b (mon.1) 58 : audit 1 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-07T16:13:00.550 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:13:00 smithi144 bash[23522]: audit 2024-04-07T16:12:59.305271+0000 mon.a (mon.0) 746 : audit 1 from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-07T16:13:00.550 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:13:00 smithi144 bash[23522]: audit 2024-04-07T16:12:59.307994+0000 mon.a (mon.0) 747 : audit 1 from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-07T16:13:00.550 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:13:00 smithi144 bash[23522]: cephadm 2024-04-07T16:12:59.309696+0000 mgr.y (mgr.14553) 12 : cephadm 1 Adjusting osd_memory_target on smithi144 to 3036M 2024-04-07T16:13:00.550 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:13:00 smithi144 bash[23522]: audit 2024-04-07T16:12:59.317507+0000 mon.b (mon.1) 59 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:13:00.550 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:13:00 smithi144 bash[23522]: audit 2024-04-07T16:12:59.319135+0000 mon.a (mon.0) 748 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:13:00.550 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:13:00 smithi144 bash[23522]: audit 2024-04-07T16:12:59.319258+0000 mon.b (mon.1) 60 : audit 1 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:13:00.550 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:13:00 smithi144 bash[23522]: cephadm 2024-04-07T16:12:59.326005+0000 mgr.y (mgr.14553) 13 : cephadm 1 Updating smithi100:/etc/ceph/ceph.conf 2024-04-07T16:13:00.550 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:13:00 smithi144 bash[23522]: cephadm 2024-04-07T16:12:59.326474+0000 mgr.y (mgr.14553) 14 : cephadm 1 Updating smithi144:/etc/ceph/ceph.conf 2024-04-07T16:13:00.551 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:13:00 smithi144 bash[23522]: cephadm 2024-04-07T16:12:59.592449+0000 mgr.y (mgr.14553) 15 : cephadm 1 Updating smithi100:/var/lib/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/config/ceph.conf 2024-04-07T16:13:00.551 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:13:00 smithi144 bash[23522]: cephadm 2024-04-07T16:12:59.592726+0000 mgr.y (mgr.14553) 16 : cephadm 1 Updating smithi144:/var/lib/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/config/ceph.conf 2024-04-07T16:13:00.551 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:13:00 smithi144 bash[23522]: audit 2024-04-07T16:13:00.094389+0000 mon.a (mon.0) 749 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:13:00.551 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:13:00 smithi144 bash[23522]: audit 2024-04-07T16:13:00.098627+0000 mon.a (mon.0) 750 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:13:00.551 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:13:00 smithi144 bash[23522]: audit 2024-04-07T16:13:00.105320+0000 mon.a (mon.0) 751 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:13:00.551 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:13:00 smithi144 bash[23522]: audit 2024-04-07T16:13:00.114378+0000 mon.a (mon.0) 752 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:13:00.551 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:13:00 smithi144 bash[23522]: audit 2024-04-07T16:13:00.125335+0000 mon.a (mon.0) 753 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:13:00.669 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:13:00 smithi100 bash[21346]: cluster 2024-04-07T16:12:58.601994+0000 mgr.y (mgr.14553) 10 : cluster 0 pgmap v6: 97 pgs: 97 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:13:00.670 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:13:00 smithi100 bash[21346]: cephadm 2024-04-07T16:12:59.083800+0000 mgr.y (mgr.14553) 11 : cephadm 1 Adjusting osd_memory_target on smithi100 to 3036M 2024-04-07T16:13:00.670 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:13:00 smithi100 bash[21346]: audit 2024-04-07T16:12:59.288153+0000 mon.a (mon.0) 743 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:13:00.670 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:13:00 smithi100 bash[21346]: audit 2024-04-07T16:12:59.291633+0000 mon.b (mon.1) 55 : audit 1 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-07T16:13:00.670 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:13:00 smithi100 bash[21346]: audit 2024-04-07T16:12:59.297929+0000 mon.b (mon.1) 56 : audit 1 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-07T16:13:00.670 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:13:00 smithi100 bash[21346]: audit 2024-04-07T16:12:59.300052+0000 mon.a (mon.0) 744 : audit 1 from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-07T16:13:00.670 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:13:00 smithi100 bash[21346]: audit 2024-04-07T16:12:59.300446+0000 mon.b (mon.1) 57 : audit 1 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-07T16:13:00.670 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:13:00 smithi100 bash[21346]: audit 2024-04-07T16:12:59.302910+0000 mon.a (mon.0) 745 : audit 1 from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-07T16:13:00.670 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:13:00 smithi100 bash[21346]: audit 2024-04-07T16:12:59.303038+0000 mon.b (mon.1) 58 : audit 1 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-07T16:13:00.670 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:13:00 smithi100 bash[21346]: audit 2024-04-07T16:12:59.305271+0000 mon.a (mon.0) 746 : audit 1 from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-07T16:13:00.670 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:13:00 smithi100 bash[21346]: audit 2024-04-07T16:12:59.307994+0000 mon.a (mon.0) 747 : audit 1 from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-07T16:13:00.670 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:13:00 smithi100 bash[21346]: cephadm 2024-04-07T16:12:59.309696+0000 mgr.y (mgr.14553) 12 : cephadm 1 Adjusting osd_memory_target on smithi144 to 3036M 2024-04-07T16:13:00.671 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:13:00 smithi100 bash[21346]: audit 2024-04-07T16:12:59.317507+0000 mon.b (mon.1) 59 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:13:00.671 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:13:00 smithi100 bash[21346]: audit 2024-04-07T16:12:59.319135+0000 mon.a (mon.0) 748 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:13:00.671 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:13:00 smithi100 bash[21346]: audit 2024-04-07T16:12:59.319258+0000 mon.b (mon.1) 60 : audit 1 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:13:00.671 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:13:00 smithi100 bash[21346]: cephadm 2024-04-07T16:12:59.326005+0000 mgr.y (mgr.14553) 13 : cephadm 1 Updating smithi100:/etc/ceph/ceph.conf 2024-04-07T16:13:00.671 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:13:00 smithi100 bash[21346]: cephadm 2024-04-07T16:12:59.326474+0000 mgr.y (mgr.14553) 14 : cephadm 1 Updating smithi144:/etc/ceph/ceph.conf 2024-04-07T16:13:00.671 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:13:00 smithi100 bash[21346]: cephadm 2024-04-07T16:12:59.592449+0000 mgr.y (mgr.14553) 15 : cephadm 1 Updating smithi100:/var/lib/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/config/ceph.conf 2024-04-07T16:13:00.671 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:13:00 smithi100 bash[21346]: cephadm 2024-04-07T16:12:59.592726+0000 mgr.y (mgr.14553) 16 : cephadm 1 Updating smithi144:/var/lib/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/config/ceph.conf 2024-04-07T16:13:00.671 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:13:00 smithi100 bash[21346]: audit 2024-04-07T16:13:00.094389+0000 mon.a (mon.0) 749 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:13:00.671 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:13:00 smithi100 bash[21346]: audit 2024-04-07T16:13:00.098627+0000 mon.a (mon.0) 750 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:13:00.671 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:13:00 smithi100 bash[21346]: audit 2024-04-07T16:13:00.105320+0000 mon.a (mon.0) 751 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:13:00.671 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:13:00 smithi100 bash[21346]: audit 2024-04-07T16:13:00.114378+0000 mon.a (mon.0) 752 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:13:00.672 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:13:00 smithi100 bash[21346]: audit 2024-04-07T16:13:00.125335+0000 mon.a (mon.0) 753 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:13:00.672 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:13:00 smithi100 bash[31660]: cluster 2024-04-07T16:12:58.601994+0000 mgr.y (mgr.14553) 10 : cluster 0 pgmap v6: 97 pgs: 97 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:13:00.672 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:13:00 smithi100 bash[31660]: cephadm 2024-04-07T16:12:59.083800+0000 mgr.y (mgr.14553) 11 : cephadm 1 Adjusting osd_memory_target on smithi100 to 3036M 2024-04-07T16:13:00.672 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:13:00 smithi100 bash[31660]: audit 2024-04-07T16:12:59.288153+0000 mon.a (mon.0) 743 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:13:00.672 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:13:00 smithi100 bash[31660]: audit 2024-04-07T16:12:59.291633+0000 mon.b (mon.1) 55 : audit 1 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-07T16:13:00.672 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:13:00 smithi100 bash[31660]: audit 2024-04-07T16:12:59.297929+0000 mon.b (mon.1) 56 : audit 1 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-07T16:13:00.672 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:13:00 smithi100 bash[31660]: audit 2024-04-07T16:12:59.300052+0000 mon.a (mon.0) 744 : audit 1 from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-07T16:13:00.673 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:13:00 smithi100 bash[31660]: audit 2024-04-07T16:12:59.300446+0000 mon.b (mon.1) 57 : audit 1 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-07T16:13:00.673 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:13:00 smithi100 bash[31660]: audit 2024-04-07T16:12:59.302910+0000 mon.a (mon.0) 745 : audit 1 from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-07T16:13:00.673 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:13:00 smithi100 bash[31660]: audit 2024-04-07T16:12:59.303038+0000 mon.b (mon.1) 58 : audit 1 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-07T16:13:00.673 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:13:00 smithi100 bash[31660]: audit 2024-04-07T16:12:59.305271+0000 mon.a (mon.0) 746 : audit 1 from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-07T16:13:00.673 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:13:00 smithi100 bash[31660]: audit 2024-04-07T16:12:59.307994+0000 mon.a (mon.0) 747 : audit 1 from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-07T16:13:00.673 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:13:00 smithi100 bash[31660]: cephadm 2024-04-07T16:12:59.309696+0000 mgr.y (mgr.14553) 12 : cephadm 1 Adjusting osd_memory_target on smithi144 to 3036M 2024-04-07T16:13:00.673 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:13:00 smithi100 bash[31660]: audit 2024-04-07T16:12:59.317507+0000 mon.b (mon.1) 59 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:13:00.673 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:13:00 smithi100 bash[31660]: audit 2024-04-07T16:12:59.319135+0000 mon.a (mon.0) 748 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:13:00.673 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:13:00 smithi100 bash[31660]: audit 2024-04-07T16:12:59.319258+0000 mon.b (mon.1) 60 : audit 1 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:13:00.673 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:13:00 smithi100 bash[31660]: cephadm 2024-04-07T16:12:59.326005+0000 mgr.y (mgr.14553) 13 : cephadm 1 Updating smithi100:/etc/ceph/ceph.conf 2024-04-07T16:13:00.673 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:13:00 smithi100 bash[31660]: cephadm 2024-04-07T16:12:59.326474+0000 mgr.y (mgr.14553) 14 : cephadm 1 Updating smithi144:/etc/ceph/ceph.conf 2024-04-07T16:13:00.674 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:13:00 smithi100 bash[31660]: cephadm 2024-04-07T16:12:59.592449+0000 mgr.y (mgr.14553) 15 : cephadm 1 Updating smithi100:/var/lib/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/config/ceph.conf 2024-04-07T16:13:00.674 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:13:00 smithi100 bash[31660]: cephadm 2024-04-07T16:12:59.592726+0000 mgr.y (mgr.14553) 16 : cephadm 1 Updating smithi144:/var/lib/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/config/ceph.conf 2024-04-07T16:13:00.674 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:13:00 smithi100 bash[31660]: audit 2024-04-07T16:13:00.094389+0000 mon.a (mon.0) 749 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:13:00.674 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:13:00 smithi100 bash[31660]: audit 2024-04-07T16:13:00.098627+0000 mon.a (mon.0) 750 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:13:00.674 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:13:00 smithi100 bash[31660]: audit 2024-04-07T16:13:00.105320+0000 mon.a (mon.0) 751 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:13:00.674 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:13:00 smithi100 bash[31660]: audit 2024-04-07T16:13:00.114378+0000 mon.a (mon.0) 752 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:13:00.674 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:13:00 smithi100 bash[31660]: audit 2024-04-07T16:13:00.125335+0000 mon.a (mon.0) 753 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:13:00.920 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:13:00 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:13:00.920 INFO:journalctl@ceph.osd.1.smithi100.stdout:Apr 07 16:13:00 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:13:00.920 INFO:journalctl@ceph.osd.0.smithi100.stdout:Apr 07 16:13:00 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:13:00.921 INFO:journalctl@ceph.osd.2.smithi100.stdout:Apr 07 16:13:00 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:13:00.921 INFO:journalctl@ceph.osd.2.smithi100.stdout:Apr 07 16:13:00 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:13:00.921 INFO:journalctl@ceph.osd.3.smithi100.stdout:Apr 07 16:13:00 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:13:00.921 INFO:journalctl@ceph.osd.3.smithi100.stdout:Apr 07 16:13:00 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:13:00.921 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:13:00 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:13:00.922 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:13:00 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:13:00.922 INFO:journalctl@ceph.mds.a.smithi100.stdout:Apr 07 16:13:00 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:13:01.189 INFO:journalctl@ceph.mds.a.smithi100.stdout:Apr 07 16:13:00 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:13:01.189 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:13:00 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:13:01.190 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:13:00 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:13:01.190 INFO:journalctl@ceph.osd.0.smithi100.stdout:Apr 07 16:13:00 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:13:01.190 INFO:journalctl@ceph.osd.1.smithi100.stdout:Apr 07 16:13:00 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:13:01.191 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:13:00 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:13:01.191 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 16:13:01 smithi100 systemd[1]: Started Ceph node-exporter.a for 0702e3a8-f4f8-11ee-b649-cb9ed24678a4. 2024-04-07T16:13:01.191 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 16:13:01 smithi100 bash[57604]: Unable to find image 'quay.io/prometheus/node-exporter:v1.5.0' locally 2024-04-07T16:13:01.559 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 16:13:01 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:13:01.560 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:13:01 smithi144 bash[23522]: cephadm 2024-04-07T16:12:59.757359+0000 mgr.y (mgr.14553) 17 : cephadm 1 Updating smithi100:/etc/ceph/ceph.client.admin.keyring 2024-04-07T16:13:01.560 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:13:01 smithi144 bash[23522]: cephadm 2024-04-07T16:12:59.758018+0000 mgr.y (mgr.14553) 18 : cephadm 1 Updating smithi144:/etc/ceph/ceph.client.admin.keyring 2024-04-07T16:13:01.560 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:13:01 smithi144 bash[23522]: cephadm 2024-04-07T16:12:59.922512+0000 mgr.y (mgr.14553) 19 : cephadm 1 Updating smithi144:/var/lib/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/config/ceph.client.admin.keyring 2024-04-07T16:13:01.560 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:13:01 smithi144 bash[23522]: cephadm 2024-04-07T16:12:59.922858+0000 mgr.y (mgr.14553) 20 : cephadm 1 Updating smithi100:/var/lib/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/config/ceph.client.admin.keyring 2024-04-07T16:13:01.560 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:13:01 smithi144 bash[23522]: cephadm 2024-04-07T16:13:00.132939+0000 mgr.y (mgr.14553) 21 : cephadm 1 Deploying daemon node-exporter.a on smithi100 2024-04-07T16:13:01.560 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:13:01 smithi144 bash[23522]: audit 2024-04-07T16:13:01.082628+0000 mon.a (mon.0) 754 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:13:01.560 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:13:01 smithi144 bash[23522]: audit 2024-04-07T16:13:01.096041+0000 mon.a (mon.0) 755 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:13:01.560 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:13:01 smithi144 bash[23522]: audit 2024-04-07T16:13:01.156446+0000 mon.a (mon.0) 756 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:13:01.689 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:13:01 smithi100 bash[21346]: cephadm 2024-04-07T16:12:59.757359+0000 mgr.y (mgr.14553) 17 : cephadm 1 Updating smithi100:/etc/ceph/ceph.client.admin.keyring 2024-04-07T16:13:01.689 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:13:01 smithi100 bash[21346]: cephadm 2024-04-07T16:12:59.758018+0000 mgr.y (mgr.14553) 18 : cephadm 1 Updating smithi144:/etc/ceph/ceph.client.admin.keyring 2024-04-07T16:13:01.690 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:13:01 smithi100 bash[21346]: cephadm 2024-04-07T16:12:59.922512+0000 mgr.y (mgr.14553) 19 : cephadm 1 Updating smithi144:/var/lib/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/config/ceph.client.admin.keyring 2024-04-07T16:13:01.690 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:13:01 smithi100 bash[21346]: cephadm 2024-04-07T16:12:59.922858+0000 mgr.y (mgr.14553) 20 : cephadm 1 Updating smithi100:/var/lib/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/config/ceph.client.admin.keyring 2024-04-07T16:13:01.690 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:13:01 smithi100 bash[21346]: cephadm 2024-04-07T16:13:00.132939+0000 mgr.y (mgr.14553) 21 : cephadm 1 Deploying daemon node-exporter.a on smithi100 2024-04-07T16:13:01.690 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:13:01 smithi100 bash[21346]: audit 2024-04-07T16:13:01.082628+0000 mon.a (mon.0) 754 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:13:01.690 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:13:01 smithi100 bash[21346]: audit 2024-04-07T16:13:01.096041+0000 mon.a (mon.0) 755 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:13:01.690 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:13:01 smithi100 bash[21346]: audit 2024-04-07T16:13:01.156446+0000 mon.a (mon.0) 756 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:13:01.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:13:01 smithi100 bash[31660]: cephadm 2024-04-07T16:12:59.757359+0000 mgr.y (mgr.14553) 17 : cephadm 1 Updating smithi100:/etc/ceph/ceph.client.admin.keyring 2024-04-07T16:13:01.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:13:01 smithi100 bash[31660]: cephadm 2024-04-07T16:12:59.758018+0000 mgr.y (mgr.14553) 18 : cephadm 1 Updating smithi144:/etc/ceph/ceph.client.admin.keyring 2024-04-07T16:13:01.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:13:01 smithi100 bash[31660]: cephadm 2024-04-07T16:12:59.922512+0000 mgr.y (mgr.14553) 19 : cephadm 1 Updating smithi144:/var/lib/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/config/ceph.client.admin.keyring 2024-04-07T16:13:01.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:13:01 smithi100 bash[31660]: cephadm 2024-04-07T16:12:59.922858+0000 mgr.y (mgr.14553) 20 : cephadm 1 Updating smithi100:/var/lib/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/config/ceph.client.admin.keyring 2024-04-07T16:13:01.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:13:01 smithi100 bash[31660]: cephadm 2024-04-07T16:13:00.132939+0000 mgr.y (mgr.14553) 21 : cephadm 1 Deploying daemon node-exporter.a on smithi100 2024-04-07T16:13:01.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:13:01 smithi100 bash[31660]: audit 2024-04-07T16:13:01.082628+0000 mon.a (mon.0) 754 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:13:01.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:13:01 smithi100 bash[31660]: audit 2024-04-07T16:13:01.096041+0000 mon.a (mon.0) 755 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:13:01.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:13:01 smithi100 bash[31660]: audit 2024-04-07T16:13:01.156446+0000 mon.a (mon.0) 756 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:13:01.796 INFO:teuthology.orchestra.run.smithi144.stdout:Scheduled grafana update... 2024-04-07T16:13:01.810 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:13:01 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:13:01.810 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:13:01 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:13:01.810 INFO:journalctl@ceph.osd.6.smithi144.stdout:Apr 07 16:13:01 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:13:01.811 INFO:journalctl@ceph.osd.7.smithi144.stdout:Apr 07 16:13:01 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:13:01.811 INFO:journalctl@ceph.mds.b.smithi144.stdout:Apr 07 16:13:01 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:13:01.811 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:13:01 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:13:01.812 INFO:journalctl@ceph.osd.4.smithi144.stdout:Apr 07 16:13:01 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:13:01.812 INFO:journalctl@ceph.osd.5.smithi144.stdout:Apr 07 16:13:01 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:13:02.081 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:13:01 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:13:02.081 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:13:01 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:13:02.081 INFO:journalctl@ceph.mds.b.smithi144.stdout:Apr 07 16:13:01 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:13:02.082 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:13:01 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:13:02.082 INFO:journalctl@ceph.osd.7.smithi144.stdout:Apr 07 16:13:01 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:13:02.082 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 16:13:01 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:13:02.082 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 16:13:01 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:13:02.083 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 16:13:01 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:13:02.083 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 16:13:01 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:13:02.083 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 16:13:02 smithi144 systemd[1]: Started Ceph node-exporter.b for 0702e3a8-f4f8-11ee-b649-cb9ed24678a4. 2024-04-07T16:13:02.083 INFO:journalctl@ceph.osd.6.smithi144.stdout:Apr 07 16:13:01 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:13:02.083 INFO:journalctl@ceph.osd.4.smithi144.stdout:Apr 07 16:13:01 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:13:02.083 INFO:journalctl@ceph.osd.5.smithi144.stdout:Apr 07 16:13:01 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:13:02.357 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 16:13:02 smithi144 bash[49566]: Unable to find image 'quay.io/prometheus/node-exporter:v1.5.0' locally 2024-04-07T16:13:02.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:13:02 smithi144 bash[23522]: cluster 2024-04-07T16:13:00.603291+0000 mgr.y (mgr.14553) 22 : cluster 0 pgmap v7: 97 pgs: 97 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 30 KiB/s rd, 0 B/s wr, 16 op/s 2024-04-07T16:13:02.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:13:02 smithi144 bash[23522]: cephadm 2024-04-07T16:13:01.160636+0000 mgr.y (mgr.14553) 23 : cephadm 1 Deploying daemon node-exporter.b on smithi144 2024-04-07T16:13:02.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:13:02 smithi144 bash[23522]: audit 2024-04-07T16:13:01.808731+0000 mon.a (mon.0) 757 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:13:02.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:13:02 smithi144 bash[23522]: audit 2024-04-07T16:13:02.112753+0000 mon.a (mon.0) 758 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:13:02.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:13:02 smithi144 bash[23522]: audit 2024-04-07T16:13:02.126044+0000 mon.a (mon.0) 759 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:13:02.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:13:02 smithi144 bash[23522]: audit 2024-04-07T16:13:02.185885+0000 mon.a (mon.0) 760 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:13:02.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:13:02 smithi144 bash[23522]: audit 2024-04-07T16:13:02.196918+0000 mon.a (mon.0) 761 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:13:02.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:13:02 smithi144 bash[23522]: audit 2024-04-07T16:13:02.216310+0000 mon.a (mon.0) 762 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:13:02.429 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:13:02 smithi100 bash[21346]: cluster 2024-04-07T16:13:00.603291+0000 mgr.y (mgr.14553) 22 : cluster 0 pgmap v7: 97 pgs: 97 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 30 KiB/s rd, 0 B/s wr, 16 op/s 2024-04-07T16:13:02.430 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:13:02 smithi100 bash[21346]: cephadm 2024-04-07T16:13:01.160636+0000 mgr.y (mgr.14553) 23 : cephadm 1 Deploying daemon node-exporter.b on smithi144 2024-04-07T16:13:02.430 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:13:02 smithi100 bash[21346]: audit 2024-04-07T16:13:01.808731+0000 mon.a (mon.0) 757 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:13:02.430 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:13:02 smithi100 bash[21346]: audit 2024-04-07T16:13:02.112753+0000 mon.a (mon.0) 758 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:13:02.430 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:13:02 smithi100 bash[21346]: audit 2024-04-07T16:13:02.126044+0000 mon.a (mon.0) 759 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:13:02.430 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:13:02 smithi100 bash[21346]: audit 2024-04-07T16:13:02.185885+0000 mon.a (mon.0) 760 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:13:02.430 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:13:02 smithi100 bash[21346]: audit 2024-04-07T16:13:02.196918+0000 mon.a (mon.0) 761 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:13:02.430 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:13:02 smithi100 bash[21346]: audit 2024-04-07T16:13:02.216310+0000 mon.a (mon.0) 762 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:13:02.430 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 16:13:02 smithi100 bash[57604]: v1.5.0: Pulling from prometheus/node-exporter 2024-04-07T16:13:02.431 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 16:13:02 smithi100 bash[57604]: 22b70bddd3ac: Pulling fs layer 2024-04-07T16:13:02.431 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 16:13:02 smithi100 bash[57604]: 5c12815fee55: Pulling fs layer 2024-04-07T16:13:02.431 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 16:13:02 smithi100 bash[57604]: c0e87333d380: Pulling fs layer 2024-04-07T16:13:02.431 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 16:13:02 smithi100 bash[57604]: 5c12815fee55: Verifying Checksum 2024-04-07T16:13:02.431 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 16:13:02 smithi100 bash[57604]: 5c12815fee55: Download complete 2024-04-07T16:13:02.431 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:13:02 smithi100 bash[31660]: cluster 2024-04-07T16:13:00.603291+0000 mgr.y (mgr.14553) 22 : cluster 0 pgmap v7: 97 pgs: 97 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 30 KiB/s rd, 0 B/s wr, 16 op/s 2024-04-07T16:13:02.431 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:13:02 smithi100 bash[31660]: cephadm 2024-04-07T16:13:01.160636+0000 mgr.y (mgr.14553) 23 : cephadm 1 Deploying daemon node-exporter.b on smithi144 2024-04-07T16:13:02.431 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:13:02 smithi100 bash[31660]: audit 2024-04-07T16:13:01.808731+0000 mon.a (mon.0) 757 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:13:02.431 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:13:02 smithi100 bash[31660]: audit 2024-04-07T16:13:02.112753+0000 mon.a (mon.0) 758 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:13:02.431 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:13:02 smithi100 bash[31660]: audit 2024-04-07T16:13:02.126044+0000 mon.a (mon.0) 759 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:13:02.431 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:13:02 smithi100 bash[31660]: audit 2024-04-07T16:13:02.185885+0000 mon.a (mon.0) 760 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:13:02.431 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:13:02 smithi100 bash[31660]: audit 2024-04-07T16:13:02.196918+0000 mon.a (mon.0) 761 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:13:02.432 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:13:02 smithi100 bash[31660]: audit 2024-04-07T16:13:02.216310+0000 mon.a (mon.0) 762 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:13:02.689 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 16:13:02 smithi100 bash[57604]: 22b70bddd3ac: Verifying Checksum 2024-04-07T16:13:02.689 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 16:13:02 smithi100 bash[57604]: 22b70bddd3ac: Download complete 2024-04-07T16:13:02.689 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 16:13:02 smithi100 bash[57604]: c0e87333d380: Verifying Checksum 2024-04-07T16:13:02.689 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 16:13:02 smithi100 bash[57604]: c0e87333d380: Download complete 2024-04-07T16:13:02.689 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 16:13:02 smithi100 bash[57604]: 22b70bddd3ac: Pull complete 2024-04-07T16:13:02.690 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Apr 07 16:13:02 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:13:02.831 DEBUG:teuthology.orchestra.run.smithi144:grafana.a> sudo journalctl -f -n 0 -u ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@grafana.a.service 2024-04-07T16:13:02.833 INFO:tasks.cephadm:Setting up client nodes... 2024-04-07T16:13:02.834 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0702e3a8-f4f8-11ee-b649-cb9ed24678a4 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-04-07T16:13:03.107 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 16:13:02 smithi144 bash[49566]: v1.5.0: Pulling from prometheus/node-exporter 2024-04-07T16:13:03.189 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 16:13:03 smithi100 bash[57604]: 5c12815fee55: Pull complete 2024-04-07T16:13:03.439 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:13:03 smithi100 bash[21346]: audit 2024-04-07T16:13:01.796371+0000 mgr.y (mgr.14553) 24 : audit 0 from='client.24418 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi144=a", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:13:03.439 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:13:03 smithi100 bash[21346]: cephadm 2024-04-07T16:13:01.800446+0000 mgr.y (mgr.14553) 25 : cephadm 1 Saving service grafana spec with placement smithi144=a;count:1 2024-04-07T16:13:03.439 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:13:03 smithi100 bash[21346]: cephadm 2024-04-07T16:13:02.252937+0000 mgr.y (mgr.14553) 26 : cephadm 1 Deploying daemon alertmanager.a on smithi100 2024-04-07T16:13:03.440 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:13:03 smithi100 bash[21346]: audit 2024-04-07T16:13:02.689800+0000 mon.a (mon.0) 763 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:13:03.440 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:13:03 smithi100 bash[31660]: audit 2024-04-07T16:13:01.796371+0000 mgr.y (mgr.14553) 24 : audit 0 from='client.24418 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi144=a", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:13:03.440 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:13:03 smithi100 bash[31660]: cephadm 2024-04-07T16:13:01.800446+0000 mgr.y (mgr.14553) 25 : cephadm 1 Saving service grafana spec with placement smithi144=a;count:1 2024-04-07T16:13:03.440 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:13:03 smithi100 bash[31660]: cephadm 2024-04-07T16:13:02.252937+0000 mgr.y (mgr.14553) 26 : cephadm 1 Deploying daemon alertmanager.a on smithi100 2024-04-07T16:13:03.440 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:13:03 smithi100 bash[31660]: audit 2024-04-07T16:13:02.689800+0000 mon.a (mon.0) 763 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:13:03.496 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:13:03 smithi144 bash[23522]: audit 2024-04-07T16:13:01.796371+0000 mgr.y (mgr.14553) 24 : audit 0 from='client.24418 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi144=a", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:13:03.496 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:13:03 smithi144 bash[23522]: cephadm 2024-04-07T16:13:01.800446+0000 mgr.y (mgr.14553) 25 : cephadm 1 Saving service grafana spec with placement smithi144=a;count:1 2024-04-07T16:13:03.496 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:13:03 smithi144 bash[23522]: cephadm 2024-04-07T16:13:02.252937+0000 mgr.y (mgr.14553) 26 : cephadm 1 Deploying daemon alertmanager.a on smithi100 2024-04-07T16:13:03.496 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:13:03 smithi144 bash[23522]: audit 2024-04-07T16:13:02.689800+0000 mon.a (mon.0) 763 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:13:03.497 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 16:13:03 smithi144 bash[49566]: 22b70bddd3ac: Pulling fs layer 2024-04-07T16:13:03.497 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 16:13:03 smithi144 bash[49566]: 5c12815fee55: Pulling fs layer 2024-04-07T16:13:03.497 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 16:13:03 smithi144 bash[49566]: c0e87333d380: Pulling fs layer 2024-04-07T16:13:03.497 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 16:13:03 smithi144 bash[49566]: 5c12815fee55: Verifying Checksum 2024-04-07T16:13:03.497 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 16:13:03 smithi144 bash[49566]: 5c12815fee55: Download complete 2024-04-07T16:13:03.497 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 16:13:03 smithi144 bash[49566]: 22b70bddd3ac: Verifying Checksum 2024-04-07T16:13:03.497 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 16:13:03 smithi144 bash[49566]: 22b70bddd3ac: Download complete 2024-04-07T16:13:03.497 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 16:13:03 smithi144 bash[49566]: 22b70bddd3ac: Pull complete 2024-04-07T16:13:03.730 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 16:13:03 smithi100 bash[57604]: c0e87333d380: Pull complete 2024-04-07T16:13:03.730 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 16:13:03 smithi100 bash[57604]: Digest: sha256:39c642b2b337e38c18e80266fb14383754178202f40103646337722a594d984c 2024-04-07T16:13:03.857 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 16:13:03 smithi144 bash[49566]: c0e87333d380: Verifying Checksum 2024-04-07T16:13:03.857 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 16:13:03 smithi144 bash[49566]: c0e87333d380: Download complete 2024-04-07T16:13:04.189 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 16:13:03 smithi100 bash[57604]: Status: Downloaded newer image for quay.io/prometheus/node-exporter:v1.5.0 2024-04-07T16:13:04.255 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 16:13:03 smithi144 bash[49566]: 5c12815fee55: Pull complete 2024-04-07T16:13:04.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:13:04 smithi144 bash[23522]: cluster 2024-04-07T16:13:02.603724+0000 mgr.y (mgr.14553) 27 : cluster 0 pgmap v8: 97 pgs: 97 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 23 KiB/s rd, 0 B/s wr, 12 op/s 2024-04-07T16:13:04.610 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 16:13:04 smithi144 bash[49566]: c0e87333d380: Pull complete 2024-04-07T16:13:04.610 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 16:13:04 smithi144 bash[49566]: Digest: sha256:39c642b2b337e38c18e80266fb14383754178202f40103646337722a594d984c 2024-04-07T16:13:04.610 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 16:13:04 smithi144 bash[49566]: Status: Downloaded newer image for quay.io/prometheus/node-exporter:v1.5.0 2024-04-07T16:13:04.689 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:13:04 smithi100 bash[31660]: cluster 2024-04-07T16:13:02.603724+0000 mgr.y (mgr.14553) 27 : cluster 0 pgmap v8: 97 pgs: 97 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 23 KiB/s rd, 0 B/s wr, 12 op/s 2024-04-07T16:13:04.689 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:13:04 smithi100 bash[21346]: cluster 2024-04-07T16:13:02.603724+0000 mgr.y (mgr.14553) 27 : cluster 0 pgmap v8: 97 pgs: 97 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 23 KiB/s rd, 0 B/s wr, 12 op/s 2024-04-07T16:13:05.439 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:13:05 smithi100 bash[21601]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:13:05] "GET /metrics HTTP/1.1" 200 28773 "" "Prometheus/2.43.0" 2024-04-07T16:13:05.858 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 16:13:05 smithi144 bash[49566]: ts=2024-04-07T16:13:05.626Z caller=node_exporter.go:180 level=info msg="Starting node_exporter" version="(version=1.5.0, branch=HEAD, revision=1b48970ffcf5630534fb00bb0687d73c66d1c959)" 2024-04-07T16:13:05.858 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 16:13:05 smithi144 bash[49566]: ts=2024-04-07T16:13:05.627Z caller=node_exporter.go:181 level=info msg="Build context" build_context="(go=go1.19.3, user=root@6e7732a7b81b, date=20221129-18:59:09)" 2024-04-07T16:13:05.858 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 16:13:05 smithi144 bash[49566]: ts=2024-04-07T16:13:05.627Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+|var/lib/containers/storage/.+)($|/) 2024-04-07T16:13:05.858 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 16:13:05 smithi144 bash[49566]: ts=2024-04-07T16:13:05.627Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2024-04-07T16:13:05.858 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 16:13:05 smithi144 bash[49566]: ts=2024-04-07T16:13:05.627Z caller=diskstats_common.go:111 level=info collector=diskstats msg="Parsed flag --collector.diskstats.device-exclude" flag=^(ram|loop|fd|(h|s|v|xv)d[a-z]|nvme\d+n\d+p)\d+$ 2024-04-07T16:13:05.858 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 16:13:05 smithi144 bash[49566]: ts=2024-04-07T16:13:05.628Z caller=diskstats_linux.go:264 level=error collector=diskstats msg="Failed to open directory, disabling udev device properties" path=/run/udev/data 2024-04-07T16:13:05.859 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 16:13:05 smithi144 bash[49566]: ts=2024-04-07T16:13:05.628Z caller=node_exporter.go:110 level=info msg="Enabled collectors" 2024-04-07T16:13:05.859 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 16:13:05 smithi144 bash[49566]: ts=2024-04-07T16:13:05.628Z caller=node_exporter.go:117 level=info collector=arp 2024-04-07T16:13:05.859 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 16:13:05 smithi144 bash[49566]: ts=2024-04-07T16:13:05.628Z caller=node_exporter.go:117 level=info collector=bcache 2024-04-07T16:13:05.859 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 16:13:05 smithi144 bash[49566]: ts=2024-04-07T16:13:05.628Z caller=node_exporter.go:117 level=info collector=bonding 2024-04-07T16:13:05.859 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 16:13:05 smithi144 bash[49566]: ts=2024-04-07T16:13:05.628Z caller=node_exporter.go:117 level=info collector=btrfs 2024-04-07T16:13:05.859 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 16:13:05 smithi144 bash[49566]: ts=2024-04-07T16:13:05.628Z caller=node_exporter.go:117 level=info collector=conntrack 2024-04-07T16:13:05.859 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 16:13:05 smithi144 bash[49566]: ts=2024-04-07T16:13:05.628Z caller=node_exporter.go:117 level=info collector=cpu 2024-04-07T16:13:05.859 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 16:13:05 smithi144 bash[49566]: ts=2024-04-07T16:13:05.628Z caller=node_exporter.go:117 level=info collector=cpufreq 2024-04-07T16:13:05.859 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 16:13:05 smithi144 bash[49566]: ts=2024-04-07T16:13:05.628Z caller=node_exporter.go:117 level=info collector=diskstats 2024-04-07T16:13:05.859 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 16:13:05 smithi144 bash[49566]: ts=2024-04-07T16:13:05.628Z caller=node_exporter.go:117 level=info collector=dmi 2024-04-07T16:13:05.859 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 16:13:05 smithi144 bash[49566]: ts=2024-04-07T16:13:05.628Z caller=node_exporter.go:117 level=info collector=edac 2024-04-07T16:13:05.859 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 16:13:05 smithi144 bash[49566]: ts=2024-04-07T16:13:05.628Z caller=node_exporter.go:117 level=info collector=entropy 2024-04-07T16:13:05.859 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 16:13:05 smithi144 bash[49566]: ts=2024-04-07T16:13:05.628Z caller=node_exporter.go:117 level=info collector=fibrechannel 2024-04-07T16:13:05.860 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 16:13:05 smithi144 bash[49566]: ts=2024-04-07T16:13:05.628Z caller=node_exporter.go:117 level=info collector=filefd 2024-04-07T16:13:05.860 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 16:13:05 smithi144 bash[49566]: ts=2024-04-07T16:13:05.628Z caller=node_exporter.go:117 level=info collector=filesystem 2024-04-07T16:13:05.860 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 16:13:05 smithi144 bash[49566]: ts=2024-04-07T16:13:05.628Z caller=node_exporter.go:117 level=info collector=hwmon 2024-04-07T16:13:05.860 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 16:13:05 smithi144 bash[49566]: ts=2024-04-07T16:13:05.628Z caller=node_exporter.go:117 level=info collector=infiniband 2024-04-07T16:13:05.860 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 16:13:05 smithi144 bash[49566]: ts=2024-04-07T16:13:05.628Z caller=node_exporter.go:117 level=info collector=ipvs 2024-04-07T16:13:05.860 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 16:13:05 smithi144 bash[49566]: ts=2024-04-07T16:13:05.628Z caller=node_exporter.go:117 level=info collector=loadavg 2024-04-07T16:13:05.860 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 16:13:05 smithi144 bash[49566]: ts=2024-04-07T16:13:05.628Z caller=node_exporter.go:117 level=info collector=mdadm 2024-04-07T16:13:05.860 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 16:13:05 smithi144 bash[49566]: ts=2024-04-07T16:13:05.628Z caller=node_exporter.go:117 level=info collector=meminfo 2024-04-07T16:13:05.861 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 16:13:05 smithi144 bash[49566]: ts=2024-04-07T16:13:05.628Z caller=node_exporter.go:117 level=info collector=netclass 2024-04-07T16:13:05.861 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 16:13:05 smithi144 bash[49566]: ts=2024-04-07T16:13:05.628Z caller=node_exporter.go:117 level=info collector=netdev 2024-04-07T16:13:05.861 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 16:13:05 smithi144 bash[49566]: ts=2024-04-07T16:13:05.628Z caller=node_exporter.go:117 level=info collector=netstat 2024-04-07T16:13:05.861 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 16:13:05 smithi144 bash[49566]: ts=2024-04-07T16:13:05.628Z caller=node_exporter.go:117 level=info collector=nfs 2024-04-07T16:13:05.861 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 16:13:05 smithi144 bash[49566]: ts=2024-04-07T16:13:05.628Z caller=node_exporter.go:117 level=info collector=nfsd 2024-04-07T16:13:05.861 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 16:13:05 smithi144 bash[49566]: ts=2024-04-07T16:13:05.628Z caller=node_exporter.go:117 level=info collector=nvme 2024-04-07T16:13:05.861 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 16:13:05 smithi144 bash[49566]: ts=2024-04-07T16:13:05.628Z caller=node_exporter.go:117 level=info collector=os 2024-04-07T16:13:05.861 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 16:13:05 smithi144 bash[49566]: ts=2024-04-07T16:13:05.628Z caller=node_exporter.go:117 level=info collector=powersupplyclass 2024-04-07T16:13:05.861 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 16:13:05 smithi144 bash[49566]: ts=2024-04-07T16:13:05.628Z caller=node_exporter.go:117 level=info collector=pressure 2024-04-07T16:13:05.861 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 16:13:05 smithi144 bash[49566]: ts=2024-04-07T16:13:05.628Z caller=node_exporter.go:117 level=info collector=rapl 2024-04-07T16:13:05.861 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 16:13:05 smithi144 bash[49566]: ts=2024-04-07T16:13:05.628Z caller=node_exporter.go:117 level=info collector=schedstat 2024-04-07T16:13:05.861 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 16:13:05 smithi144 bash[49566]: ts=2024-04-07T16:13:05.628Z caller=node_exporter.go:117 level=info collector=selinux 2024-04-07T16:13:05.862 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 16:13:05 smithi144 bash[49566]: ts=2024-04-07T16:13:05.628Z caller=node_exporter.go:117 level=info collector=sockstat 2024-04-07T16:13:05.862 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 16:13:05 smithi144 bash[49566]: ts=2024-04-07T16:13:05.628Z caller=node_exporter.go:117 level=info collector=softnet 2024-04-07T16:13:05.862 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 16:13:05 smithi144 bash[49566]: ts=2024-04-07T16:13:05.628Z caller=node_exporter.go:117 level=info collector=stat 2024-04-07T16:13:05.862 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 16:13:05 smithi144 bash[49566]: ts=2024-04-07T16:13:05.628Z caller=node_exporter.go:117 level=info collector=tapestats 2024-04-07T16:13:05.862 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 16:13:05 smithi144 bash[49566]: ts=2024-04-07T16:13:05.628Z caller=node_exporter.go:117 level=info collector=textfile 2024-04-07T16:13:05.862 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 16:13:05 smithi144 bash[49566]: ts=2024-04-07T16:13:05.628Z caller=node_exporter.go:117 level=info collector=thermal_zone 2024-04-07T16:13:05.862 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 16:13:05 smithi144 bash[49566]: ts=2024-04-07T16:13:05.628Z caller=node_exporter.go:117 level=info collector=time 2024-04-07T16:13:05.862 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 16:13:05 smithi144 bash[49566]: ts=2024-04-07T16:13:05.628Z caller=node_exporter.go:117 level=info collector=udp_queues 2024-04-07T16:13:05.862 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 16:13:05 smithi144 bash[49566]: ts=2024-04-07T16:13:05.628Z caller=node_exporter.go:117 level=info collector=uname 2024-04-07T16:13:05.862 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 16:13:05 smithi144 bash[49566]: ts=2024-04-07T16:13:05.628Z caller=node_exporter.go:117 level=info collector=vmstat 2024-04-07T16:13:05.862 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 16:13:05 smithi144 bash[49566]: ts=2024-04-07T16:13:05.628Z caller=node_exporter.go:117 level=info collector=xfs 2024-04-07T16:13:05.862 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 16:13:05 smithi144 bash[49566]: ts=2024-04-07T16:13:05.628Z caller=node_exporter.go:117 level=info collector=zfs 2024-04-07T16:13:05.863 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 16:13:05 smithi144 bash[49566]: ts=2024-04-07T16:13:05.629Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9100 2024-04-07T16:13:05.863 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 16:13:05 smithi144 bash[49566]: ts=2024-04-07T16:13:05.629Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9100 2024-04-07T16:13:05.940 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 16:13:05 smithi100 bash[57604]: ts=2024-04-07T16:13:05.703Z caller=node_exporter.go:180 level=info msg="Starting node_exporter" version="(version=1.5.0, branch=HEAD, revision=1b48970ffcf5630534fb00bb0687d73c66d1c959)" 2024-04-07T16:13:05.940 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 16:13:05 smithi100 bash[57604]: ts=2024-04-07T16:13:05.703Z caller=node_exporter.go:181 level=info msg="Build context" build_context="(go=go1.19.3, user=root@6e7732a7b81b, date=20221129-18:59:09)" 2024-04-07T16:13:05.940 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 16:13:05 smithi100 bash[57604]: ts=2024-04-07T16:13:05.704Z caller=diskstats_common.go:111 level=info collector=diskstats msg="Parsed flag --collector.diskstats.device-exclude" flag=^(ram|loop|fd|(h|s|v|xv)d[a-z]|nvme\d+n\d+p)\d+$ 2024-04-07T16:13:05.940 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 16:13:05 smithi100 bash[57604]: ts=2024-04-07T16:13:05.704Z caller=diskstats_linux.go:264 level=error collector=diskstats msg="Failed to open directory, disabling udev device properties" path=/run/udev/data 2024-04-07T16:13:05.940 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 16:13:05 smithi100 bash[57604]: ts=2024-04-07T16:13:05.705Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+|var/lib/containers/storage/.+)($|/) 2024-04-07T16:13:05.940 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 16:13:05 smithi100 bash[57604]: ts=2024-04-07T16:13:05.705Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2024-04-07T16:13:05.940 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 16:13:05 smithi100 bash[57604]: ts=2024-04-07T16:13:05.705Z caller=node_exporter.go:110 level=info msg="Enabled collectors" 2024-04-07T16:13:05.941 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 16:13:05 smithi100 bash[57604]: ts=2024-04-07T16:13:05.705Z caller=node_exporter.go:117 level=info collector=arp 2024-04-07T16:13:05.941 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 16:13:05 smithi100 bash[57604]: ts=2024-04-07T16:13:05.705Z caller=node_exporter.go:117 level=info collector=bcache 2024-04-07T16:13:05.941 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 16:13:05 smithi100 bash[57604]: ts=2024-04-07T16:13:05.705Z caller=node_exporter.go:117 level=info collector=bonding 2024-04-07T16:13:05.941 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 16:13:05 smithi100 bash[57604]: ts=2024-04-07T16:13:05.705Z caller=node_exporter.go:117 level=info collector=btrfs 2024-04-07T16:13:05.941 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 16:13:05 smithi100 bash[57604]: ts=2024-04-07T16:13:05.705Z caller=node_exporter.go:117 level=info collector=conntrack 2024-04-07T16:13:05.941 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 16:13:05 smithi100 bash[57604]: ts=2024-04-07T16:13:05.705Z caller=node_exporter.go:117 level=info collector=cpu 2024-04-07T16:13:05.941 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 16:13:05 smithi100 bash[57604]: ts=2024-04-07T16:13:05.705Z caller=node_exporter.go:117 level=info collector=cpufreq 2024-04-07T16:13:05.941 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 16:13:05 smithi100 bash[57604]: ts=2024-04-07T16:13:05.705Z caller=node_exporter.go:117 level=info collector=diskstats 2024-04-07T16:13:05.941 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 16:13:05 smithi100 bash[57604]: ts=2024-04-07T16:13:05.705Z caller=node_exporter.go:117 level=info collector=dmi 2024-04-07T16:13:05.941 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 16:13:05 smithi100 bash[57604]: ts=2024-04-07T16:13:05.705Z caller=node_exporter.go:117 level=info collector=edac 2024-04-07T16:13:05.941 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 16:13:05 smithi100 bash[57604]: ts=2024-04-07T16:13:05.705Z caller=node_exporter.go:117 level=info collector=entropy 2024-04-07T16:13:05.941 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 16:13:05 smithi100 bash[57604]: ts=2024-04-07T16:13:05.705Z caller=node_exporter.go:117 level=info collector=fibrechannel 2024-04-07T16:13:05.941 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 16:13:05 smithi100 bash[57604]: ts=2024-04-07T16:13:05.705Z caller=node_exporter.go:117 level=info collector=filefd 2024-04-07T16:13:05.942 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 16:13:05 smithi100 bash[57604]: ts=2024-04-07T16:13:05.705Z caller=node_exporter.go:117 level=info collector=filesystem 2024-04-07T16:13:05.942 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 16:13:05 smithi100 bash[57604]: ts=2024-04-07T16:13:05.705Z caller=node_exporter.go:117 level=info collector=hwmon 2024-04-07T16:13:05.942 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 16:13:05 smithi100 bash[57604]: ts=2024-04-07T16:13:05.705Z caller=node_exporter.go:117 level=info collector=infiniband 2024-04-07T16:13:05.942 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 16:13:05 smithi100 bash[57604]: ts=2024-04-07T16:13:05.705Z caller=node_exporter.go:117 level=info collector=ipvs 2024-04-07T16:13:05.942 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 16:13:05 smithi100 bash[57604]: ts=2024-04-07T16:13:05.705Z caller=node_exporter.go:117 level=info collector=loadavg 2024-04-07T16:13:05.942 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 16:13:05 smithi100 bash[57604]: ts=2024-04-07T16:13:05.705Z caller=node_exporter.go:117 level=info collector=mdadm 2024-04-07T16:13:05.942 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 16:13:05 smithi100 bash[57604]: ts=2024-04-07T16:13:05.705Z caller=node_exporter.go:117 level=info collector=meminfo 2024-04-07T16:13:05.942 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 16:13:05 smithi100 bash[57604]: ts=2024-04-07T16:13:05.705Z caller=node_exporter.go:117 level=info collector=netclass 2024-04-07T16:13:05.942 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 16:13:05 smithi100 bash[57604]: ts=2024-04-07T16:13:05.705Z caller=node_exporter.go:117 level=info collector=netdev 2024-04-07T16:13:05.943 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 16:13:05 smithi100 bash[57604]: ts=2024-04-07T16:13:05.705Z caller=node_exporter.go:117 level=info collector=netstat 2024-04-07T16:13:05.943 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 16:13:05 smithi100 bash[57604]: ts=2024-04-07T16:13:05.705Z caller=node_exporter.go:117 level=info collector=nfs 2024-04-07T16:13:05.943 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 16:13:05 smithi100 bash[57604]: ts=2024-04-07T16:13:05.705Z caller=node_exporter.go:117 level=info collector=nfsd 2024-04-07T16:13:05.943 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 16:13:05 smithi100 bash[57604]: ts=2024-04-07T16:13:05.705Z caller=node_exporter.go:117 level=info collector=nvme 2024-04-07T16:13:05.943 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 16:13:05 smithi100 bash[57604]: ts=2024-04-07T16:13:05.705Z caller=node_exporter.go:117 level=info collector=os 2024-04-07T16:13:05.943 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 16:13:05 smithi100 bash[57604]: ts=2024-04-07T16:13:05.705Z caller=node_exporter.go:117 level=info collector=powersupplyclass 2024-04-07T16:13:05.943 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 16:13:05 smithi100 bash[57604]: ts=2024-04-07T16:13:05.705Z caller=node_exporter.go:117 level=info collector=pressure 2024-04-07T16:13:05.943 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 16:13:05 smithi100 bash[57604]: ts=2024-04-07T16:13:05.705Z caller=node_exporter.go:117 level=info collector=rapl 2024-04-07T16:13:05.943 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 16:13:05 smithi100 bash[57604]: ts=2024-04-07T16:13:05.705Z caller=node_exporter.go:117 level=info collector=schedstat 2024-04-07T16:13:05.943 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 16:13:05 smithi100 bash[57604]: ts=2024-04-07T16:13:05.705Z caller=node_exporter.go:117 level=info collector=selinux 2024-04-07T16:13:05.943 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 16:13:05 smithi100 bash[57604]: ts=2024-04-07T16:13:05.705Z caller=node_exporter.go:117 level=info collector=sockstat 2024-04-07T16:13:05.943 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 16:13:05 smithi100 bash[57604]: ts=2024-04-07T16:13:05.705Z caller=node_exporter.go:117 level=info collector=softnet 2024-04-07T16:13:05.943 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 16:13:05 smithi100 bash[57604]: ts=2024-04-07T16:13:05.705Z caller=node_exporter.go:117 level=info collector=stat 2024-04-07T16:13:05.943 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 16:13:05 smithi100 bash[57604]: ts=2024-04-07T16:13:05.705Z caller=node_exporter.go:117 level=info collector=tapestats 2024-04-07T16:13:05.944 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 16:13:05 smithi100 bash[57604]: ts=2024-04-07T16:13:05.705Z caller=node_exporter.go:117 level=info collector=textfile 2024-04-07T16:13:05.944 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 16:13:05 smithi100 bash[57604]: ts=2024-04-07T16:13:05.705Z caller=node_exporter.go:117 level=info collector=thermal_zone 2024-04-07T16:13:05.944 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 16:13:05 smithi100 bash[57604]: ts=2024-04-07T16:13:05.705Z caller=node_exporter.go:117 level=info collector=time 2024-04-07T16:13:05.944 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 16:13:05 smithi100 bash[57604]: ts=2024-04-07T16:13:05.705Z caller=node_exporter.go:117 level=info collector=udp_queues 2024-04-07T16:13:05.944 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 16:13:05 smithi100 bash[57604]: ts=2024-04-07T16:13:05.706Z caller=node_exporter.go:117 level=info collector=uname 2024-04-07T16:13:05.944 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 16:13:05 smithi100 bash[57604]: ts=2024-04-07T16:13:05.706Z caller=node_exporter.go:117 level=info collector=vmstat 2024-04-07T16:13:05.944 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 16:13:05 smithi100 bash[57604]: ts=2024-04-07T16:13:05.706Z caller=node_exporter.go:117 level=info collector=xfs 2024-04-07T16:13:05.944 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 16:13:05 smithi100 bash[57604]: ts=2024-04-07T16:13:05.706Z caller=node_exporter.go:117 level=info collector=zfs 2024-04-07T16:13:05.944 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 16:13:05 smithi100 bash[57604]: ts=2024-04-07T16:13:05.706Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9100 2024-04-07T16:13:05.944 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 16:13:05 smithi100 bash[57604]: ts=2024-04-07T16:13:05.706Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9100 2024-04-07T16:13:06.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:13:06 smithi144 bash[23522]: cluster 2024-04-07T16:13:04.604897+0000 mgr.y (mgr.14553) 28 : cluster 0 pgmap v9: 97 pgs: 97 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 19 KiB/s rd, 0 B/s wr, 10 op/s 2024-04-07T16:13:06.689 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:13:06 smithi100 bash[21346]: cluster 2024-04-07T16:13:04.604897+0000 mgr.y (mgr.14553) 28 : cluster 0 pgmap v9: 97 pgs: 97 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 19 KiB/s rd, 0 B/s wr, 10 op/s 2024-04-07T16:13:06.689 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:13:06 smithi100 bash[31660]: cluster 2024-04-07T16:13:04.604897+0000 mgr.y (mgr.14553) 28 : cluster 0 pgmap v9: 97 pgs: 97 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 19 KiB/s rd, 0 B/s wr, 10 op/s 2024-04-07T16:13:07.547 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/mon.a/config 2024-04-07T16:13:08.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:13:08 smithi144 bash[23522]: cluster 2024-04-07T16:13:06.605620+0000 mgr.y (mgr.14553) 29 : cluster 0 pgmap v10: 97 pgs: 97 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-07T16:13:08.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:13:08 smithi144 bash[23522]: audit 2024-04-07T16:13:07.691135+0000 mon.b (mon.1) 61 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:13:08.627 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:13:08 smithi100 bash[21346]: cluster 2024-04-07T16:13:06.605620+0000 mgr.y (mgr.14553) 29 : cluster 0 pgmap v10: 97 pgs: 97 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-07T16:13:08.627 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:13:08 smithi100 bash[21346]: audit 2024-04-07T16:13:07.691135+0000 mon.b (mon.1) 61 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:13:08.627 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:13:08 smithi100 bash[31660]: cluster 2024-04-07T16:13:06.605620+0000 mgr.y (mgr.14553) 29 : cluster 0 pgmap v10: 97 pgs: 97 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-07T16:13:08.628 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:13:08 smithi100 bash[31660]: audit 2024-04-07T16:13:07.691135+0000 mon.b (mon.1) 61 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:13:10.439 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:13:10 smithi100 bash[21346]: cluster 2024-04-07T16:13:08.606302+0000 mgr.y (mgr.14553) 30 : cluster 0 pgmap v11: 97 pgs: 97 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-07T16:13:10.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:13:10 smithi100 bash[31660]: cluster 2024-04-07T16:13:08.606302+0000 mgr.y (mgr.14553) 30 : cluster 0 pgmap v11: 97 pgs: 97 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-07T16:13:10.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:13:10 smithi144 bash[23522]: cluster 2024-04-07T16:13:08.606302+0000 mgr.y (mgr.14553) 30 : cluster 0 pgmap v11: 97 pgs: 97 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-07T16:13:11.356 INFO:teuthology.orchestra.run.smithi100.stdout:[client.0] 2024-04-07T16:13:11.356 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQAXxhJmJwOrFRAA1P0K54ZhulGvDBXTatZ6kw== 2024-04-07T16:13:12.240 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-07T16:13:12.240 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-04-07T16:13:12.240 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-07T16:13:12.257 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0702e3a8-f4f8-11ee-b649-cb9ed24678a4 -- ceph auth get-or-create client.1 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-04-07T16:13:12.395 INFO:journalctl@ceph.osd.1.smithi100.stdout:Apr 07 16:13:12 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:13:12.395 INFO:journalctl@ceph.osd.2.smithi100.stdout:Apr 07 16:13:12 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:13:12.396 INFO:journalctl@ceph.osd.3.smithi100.stdout:Apr 07 16:13:12 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:13:12.396 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:13:12 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:13:12.396 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:13:12 smithi100 bash[21346]: cluster 2024-04-07T16:13:10.607602+0000 mgr.y (mgr.14553) 31 : cluster 0 pgmap v12: 97 pgs: 97 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-07T16:13:12.396 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:13:12 smithi100 bash[21346]: audit 2024-04-07T16:13:11.363378+0000 mon.a (mon.0) 764 : audit 1 from='client.? 172.21.15.100:0/578238720' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-07T16:13:12.396 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:13:12 smithi100 bash[21346]: audit 2024-04-07T16:13:11.369732+0000 mon.a (mon.0) 765 : audit 1 from='client.? 172.21.15.100:0/578238720' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-07T16:13:12.397 INFO:journalctl@ceph.osd.0.smithi100.stdout:Apr 07 16:13:12 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:13:12.397 INFO:journalctl@ceph.mds.a.smithi100.stdout:Apr 07 16:13:12 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:13:12.397 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:13:12 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:13:12.397 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:13:12 smithi100 bash[31660]: cluster 2024-04-07T16:13:10.607602+0000 mgr.y (mgr.14553) 31 : cluster 0 pgmap v12: 97 pgs: 97 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-07T16:13:12.397 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:13:12 smithi100 bash[31660]: audit 2024-04-07T16:13:11.363378+0000 mon.a (mon.0) 764 : audit 1 from='client.? 172.21.15.100:0/578238720' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-07T16:13:12.397 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:13:12 smithi100 bash[31660]: audit 2024-04-07T16:13:11.369732+0000 mon.a (mon.0) 765 : audit 1 from='client.? 172.21.15.100:0/578238720' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-07T16:13:12.398 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:13:12 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:13:12.398 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 16:13:12 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:13:12.537 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Apr 07 16:13:12 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:13:12.537 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Apr 07 16:13:12 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:13:12.537 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Apr 07 16:13:12 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:13:12.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:13:12 smithi144 bash[23522]: cluster 2024-04-07T16:13:10.607602+0000 mgr.y (mgr.14553) 31 : cluster 0 pgmap v12: 97 pgs: 97 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-07T16:13:12.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:13:12 smithi144 bash[23522]: audit 2024-04-07T16:13:11.363378+0000 mon.a (mon.0) 764 : audit 1 from='client.? 172.21.15.100:0/578238720' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-07T16:13:12.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:13:12 smithi144 bash[23522]: audit 2024-04-07T16:13:11.369732+0000 mon.a (mon.0) 765 : audit 1 from='client.? 172.21.15.100:0/578238720' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-07T16:13:12.653 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:13:12 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:13:12.653 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:13:12 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:13:12.653 INFO:journalctl@ceph.osd.2.smithi100.stdout:Apr 07 16:13:12 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:13:12.653 INFO:journalctl@ceph.osd.3.smithi100.stdout:Apr 07 16:13:12 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:13:12.654 INFO:journalctl@ceph.osd.1.smithi100.stdout:Apr 07 16:13:12 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:13:12.654 INFO:journalctl@ceph.osd.0.smithi100.stdout:Apr 07 16:13:12 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:13:12.654 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 16:13:12 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:13:12.655 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:13:12 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:13:12.655 INFO:journalctl@ceph.mds.a.smithi100.stdout:Apr 07 16:13:12 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:13:12.655 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Apr 07 16:13:12 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:13:12.655 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Apr 07 16:13:12 smithi100 systemd[1]: Started Ceph alertmanager.a for 0702e3a8-f4f8-11ee-b649-cb9ed24678a4. 2024-04-07T16:13:13.939 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:13:13 smithi100 bash[21346]: cluster 2024-04-07T16:13:12.608235+0000 mgr.y (mgr.14553) 32 : cluster 0 pgmap v13: 97 pgs: 97 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:13:13.939 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:13:13 smithi100 bash[21346]: audit 2024-04-07T16:13:12.701361+0000 mon.a (mon.0) 766 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:13:13.939 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:13:13 smithi100 bash[21346]: audit 2024-04-07T16:13:12.713782+0000 mon.a (mon.0) 767 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:13:13.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:13:13 smithi100 bash[21346]: audit 2024-04-07T16:13:12.755522+0000 mon.a (mon.0) 768 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:13:13.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:13:13 smithi100 bash[21346]: audit 2024-04-07T16:13:12.763216+0000 mon.a (mon.0) 769 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:13:13.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:13:13 smithi100 bash[21346]: audit 2024-04-07T16:13:12.809794+0000 mon.b (mon.1) 62 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:13:13.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:13:13 smithi100 bash[21346]: audit 2024-04-07T16:13:13.139771+0000 mon.a (mon.0) 770 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:13:13.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:13:13 smithi100 bash[21346]: audit 2024-04-07T16:13:13.146164+0000 mon.a (mon.0) 771 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:13:13.940 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:13:13 smithi100 bash[31660]: cluster 2024-04-07T16:13:12.608235+0000 mgr.y (mgr.14553) 32 : cluster 0 pgmap v13: 97 pgs: 97 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:13:13.940 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:13:13 smithi100 bash[31660]: audit 2024-04-07T16:13:12.701361+0000 mon.a (mon.0) 766 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:13:13.940 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:13:13 smithi100 bash[31660]: audit 2024-04-07T16:13:12.713782+0000 mon.a (mon.0) 767 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:13:13.940 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:13:13 smithi100 bash[31660]: audit 2024-04-07T16:13:12.755522+0000 mon.a (mon.0) 768 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:13:13.940 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:13:13 smithi100 bash[31660]: audit 2024-04-07T16:13:12.763216+0000 mon.a (mon.0) 769 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:13:13.941 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:13:13 smithi100 bash[31660]: audit 2024-04-07T16:13:12.809794+0000 mon.b (mon.1) 62 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:13:13.941 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:13:13 smithi100 bash[31660]: audit 2024-04-07T16:13:13.139771+0000 mon.a (mon.0) 770 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:13:13.941 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:13:13 smithi100 bash[31660]: audit 2024-04-07T16:13:13.146164+0000 mon.a (mon.0) 771 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:13:13.941 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Apr 07 16:13:13 smithi100 bash[58158]: ts=2024-04-07T16:13:13.677Z caller=main.go:240 level=info msg="Starting Alertmanager" version="(version=0.25.0, branch=HEAD, revision=258fab7cdd551f2cf251ed0348f0ad7289aee789)" 2024-04-07T16:13:13.941 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Apr 07 16:13:13 smithi100 bash[58158]: ts=2024-04-07T16:13:13.677Z caller=main.go:241 level=info build_context="(go=go1.19.4, user=root@abe866dd5717, date=20221222-14:51:36)" 2024-04-07T16:13:13.941 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Apr 07 16:13:13 smithi100 bash[58158]: ts=2024-04-07T16:13:13.679Z caller=cluster.go:185 level=info component=cluster msg="setting advertise address explicitly" addr=172.21.15.100 port=9094 2024-04-07T16:13:13.941 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Apr 07 16:13:13 smithi100 bash[58158]: ts=2024-04-07T16:13:13.680Z caller=cluster.go:681 level=info component=cluster msg="Waiting for gossip to settle..." interval=2s 2024-04-07T16:13:13.941 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Apr 07 16:13:13 smithi100 bash[58158]: ts=2024-04-07T16:13:13.712Z caller=coordinator.go:113 level=info component=configuration msg="Loading configuration file" file=/etc/alertmanager/alertmanager.yml 2024-04-07T16:13:13.941 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Apr 07 16:13:13 smithi100 bash[58158]: ts=2024-04-07T16:13:13.712Z caller=coordinator.go:126 level=info component=configuration msg="Completed loading of configuration file" file=/etc/alertmanager/alertmanager.yml 2024-04-07T16:13:13.941 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Apr 07 16:13:13 smithi100 bash[58158]: ts=2024-04-07T16:13:13.715Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9093 2024-04-07T16:13:13.941 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Apr 07 16:13:13 smithi100 bash[58158]: ts=2024-04-07T16:13:13.715Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9093 2024-04-07T16:13:14.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:13:13 smithi144 bash[23522]: cluster 2024-04-07T16:13:12.608235+0000 mgr.y (mgr.14553) 32 : cluster 0 pgmap v13: 97 pgs: 97 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:13:14.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:13:13 smithi144 bash[23522]: audit 2024-04-07T16:13:12.701361+0000 mon.a (mon.0) 766 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:13:14.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:13:13 smithi144 bash[23522]: audit 2024-04-07T16:13:12.713782+0000 mon.a (mon.0) 767 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:13:14.108 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:13:13 smithi144 bash[23522]: audit 2024-04-07T16:13:12.755522+0000 mon.a (mon.0) 768 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:13:14.108 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:13:13 smithi144 bash[23522]: audit 2024-04-07T16:13:12.763216+0000 mon.a (mon.0) 769 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:13:14.108 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:13:13 smithi144 bash[23522]: audit 2024-04-07T16:13:12.809794+0000 mon.b (mon.1) 62 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:13:14.108 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:13:13 smithi144 bash[23522]: audit 2024-04-07T16:13:13.139771+0000 mon.a (mon.0) 770 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:13:14.108 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:13:13 smithi144 bash[23522]: audit 2024-04-07T16:13:13.146164+0000 mon.a (mon.0) 771 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:13:15.439 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:13:15 smithi100 bash[21601]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:13:15] "GET /metrics HTTP/1.1" 200 31386 "" "Prometheus/2.43.0" 2024-04-07T16:13:15.939 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:13:15 smithi100 bash[21346]: cluster 2024-04-07T16:13:14.609360+0000 mgr.y (mgr.14553) 33 : cluster 0 pgmap v14: 97 pgs: 97 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:13:15.939 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:13:15 smithi100 bash[31660]: cluster 2024-04-07T16:13:14.609360+0000 mgr.y (mgr.14553) 33 : cluster 0 pgmap v14: 97 pgs: 97 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:13:15.939 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Apr 07 16:13:15 smithi100 bash[58158]: ts=2024-04-07T16:13:15.680Z caller=cluster.go:706 level=info component=cluster msg="gossip not settled" polls=0 before=0 now=1 elapsed=2.00052725s 2024-04-07T16:13:16.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:13:15 smithi144 bash[23522]: cluster 2024-04-07T16:13:14.609360+0000 mgr.y (mgr.14553) 33 : cluster 0 pgmap v14: 97 pgs: 97 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:13:16.990 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/mon.b/config 2024-04-07T16:13:18.840 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:13:18 smithi100 bash[21346]: cluster 2024-04-07T16:13:16.609978+0000 mgr.y (mgr.14553) 34 : cluster 0 pgmap v15: 97 pgs: 97 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:13:18.840 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:13:18 smithi100 bash[21346]: audit 2024-04-07T16:13:17.556030+0000 mon.a (mon.0) 772 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:13:18.840 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:13:18 smithi100 bash[21346]: audit 2024-04-07T16:13:17.568915+0000 mon.a (mon.0) 773 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:13:18.840 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:13:18 smithi100 bash[21346]: audit 2024-04-07T16:13:17.702999+0000 mon.a (mon.0) 774 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:13:18.840 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:13:18 smithi100 bash[31660]: cluster 2024-04-07T16:13:16.609978+0000 mgr.y (mgr.14553) 34 : cluster 0 pgmap v15: 97 pgs: 97 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:13:18.840 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:13:18 smithi100 bash[31660]: audit 2024-04-07T16:13:17.556030+0000 mon.a (mon.0) 772 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:13:18.840 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:13:18 smithi100 bash[31660]: audit 2024-04-07T16:13:17.568915+0000 mon.a (mon.0) 773 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:13:18.840 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:13:18 smithi100 bash[31660]: audit 2024-04-07T16:13:17.702999+0000 mon.a (mon.0) 774 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:13:18.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:13:18 smithi144 bash[23522]: cluster 2024-04-07T16:13:16.609978+0000 mgr.y (mgr.14553) 34 : cluster 0 pgmap v15: 97 pgs: 97 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:13:18.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:13:18 smithi144 bash[23522]: audit 2024-04-07T16:13:17.556030+0000 mon.a (mon.0) 772 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:13:18.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:13:18 smithi144 bash[23522]: audit 2024-04-07T16:13:17.568915+0000 mon.a (mon.0) 773 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:13:18.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:13:18 smithi144 bash[23522]: audit 2024-04-07T16:13:17.702999+0000 mon.a (mon.0) 774 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:13:19.189 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:13:18 smithi100 bash[21601]: [07/Apr/2024:16:13:18] ENGINE Bus STOPPING 2024-04-07T16:13:19.190 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:13:18 smithi100 bash[21601]: [07/Apr/2024:16:13:18] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-04-07T16:13:19.190 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:13:18 smithi100 bash[21601]: [07/Apr/2024:16:13:18] ENGINE Bus STOPPED 2024-04-07T16:13:19.190 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:13:18 smithi100 bash[21601]: [07/Apr/2024:16:13:18] ENGINE Bus STARTING 2024-04-07T16:13:19.190 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:13:18 smithi100 bash[21601]: [07/Apr/2024:16:13:18] ENGINE Serving on http://:::9283 2024-04-07T16:13:19.190 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:13:18 smithi100 bash[21601]: [07/Apr/2024:16:13:18] ENGINE Bus STARTED 2024-04-07T16:13:19.588 INFO:teuthology.orchestra.run.smithi144.stdout:[client.1] 2024-04-07T16:13:19.588 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQAfxhJmPKN1IxAATo1ZMeHFlY5uEaEC6GOEjQ== 2024-04-07T16:13:19.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:13:19 smithi144 bash[23522]: cluster 2024-04-07T16:13:18.610449+0000 mgr.y (mgr.14553) 35 : cluster 0 pgmap v16: 97 pgs: 97 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:13:19.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:13:19 smithi144 bash[23522]: audit 2024-04-07T16:13:18.640700+0000 mon.a (mon.0) 775 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:13:19.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:13:19 smithi144 bash[23522]: audit 2024-04-07T16:13:18.657262+0000 mon.b (mon.1) 63 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:13:19.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:13:19 smithi144 bash[23522]: audit 2024-04-07T16:13:18.657624+0000 mon.a (mon.0) 776 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:13:19.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:13:19 smithi144 bash[23522]: audit 2024-04-07T16:13:18.659370+0000 mon.b (mon.1) 64 : audit 1 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:13:19.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:13:19 smithi144 bash[23522]: audit 2024-04-07T16:13:18.678324+0000 mon.a (mon.0) 777 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:13:19.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:13:19 smithi144 bash[23522]: cephadm 2024-04-07T16:13:18.716104+0000 mgr.y (mgr.14553) 36 : cephadm 1 Regenerating cephadm self-signed grafana TLS certificates 2024-04-07T16:13:19.861 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:13:19 smithi144 bash[23522]: audit 2024-04-07T16:13:18.810166+0000 mon.a (mon.0) 778 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:13:19.861 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:13:19 smithi144 bash[23522]: audit 2024-04-07T16:13:18.822038+0000 mon.a (mon.0) 779 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:13:19.861 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:13:19 smithi144 bash[23522]: audit 2024-04-07T16:13:18.822898+0000 mon.b (mon.1) 65 : audit 1 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-04-07T16:13:19.861 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:13:19 smithi144 bash[23522]: audit 2024-04-07T16:13:18.828283+0000 mgr.y (mgr.14553) 37 : audit 0 from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-04-07T16:13:19.861 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:13:19 smithi144 bash[23522]: audit 2024-04-07T16:13:18.837907+0000 mon.a (mon.0) 780 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:13:19.862 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:13:19 smithi144 bash[23522]: cephadm 2024-04-07T16:13:18.852543+0000 mgr.y (mgr.14553) 38 : cephadm 1 Deploying daemon grafana.a on smithi144 2024-04-07T16:13:19.862 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:13:19 smithi144 bash[23522]: audit 2024-04-07T16:13:19.588670+0000 mon.b (mon.1) 66 : audit 1 from='client.? 172.21.15.144:0/3178124117' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-07T16:13:19.862 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:13:19 smithi144 bash[23522]: audit 2024-04-07T16:13:19.594486+0000 mon.a (mon.0) 781 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-07T16:13:19.862 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:13:19 smithi144 bash[23522]: audit 2024-04-07T16:13:19.601793+0000 mon.a (mon.0) 782 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-07T16:13:19.939 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:13:19 smithi100 bash[21346]: cluster 2024-04-07T16:13:18.610449+0000 mgr.y (mgr.14553) 35 : cluster 0 pgmap v16: 97 pgs: 97 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:13:19.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:13:19 smithi100 bash[21346]: audit 2024-04-07T16:13:18.640700+0000 mon.a (mon.0) 775 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:13:19.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:13:19 smithi100 bash[21346]: audit 2024-04-07T16:13:18.657262+0000 mon.b (mon.1) 63 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:13:19.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:13:19 smithi100 bash[21346]: audit 2024-04-07T16:13:18.657624+0000 mon.a (mon.0) 776 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:13:19.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:13:19 smithi100 bash[21346]: audit 2024-04-07T16:13:18.659370+0000 mon.b (mon.1) 64 : audit 1 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:13:19.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:13:19 smithi100 bash[21346]: audit 2024-04-07T16:13:18.678324+0000 mon.a (mon.0) 777 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:13:19.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:13:19 smithi100 bash[21346]: cephadm 2024-04-07T16:13:18.716104+0000 mgr.y (mgr.14553) 36 : cephadm 1 Regenerating cephadm self-signed grafana TLS certificates 2024-04-07T16:13:19.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:13:19 smithi100 bash[21346]: audit 2024-04-07T16:13:18.810166+0000 mon.a (mon.0) 778 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:13:19.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:13:19 smithi100 bash[21346]: audit 2024-04-07T16:13:18.822038+0000 mon.a (mon.0) 779 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:13:19.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:13:19 smithi100 bash[21346]: audit 2024-04-07T16:13:18.822898+0000 mon.b (mon.1) 65 : audit 1 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-04-07T16:13:19.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:13:19 smithi100 bash[21346]: audit 2024-04-07T16:13:18.828283+0000 mgr.y (mgr.14553) 37 : audit 0 from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-04-07T16:13:19.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:13:19 smithi100 bash[21346]: audit 2024-04-07T16:13:18.837907+0000 mon.a (mon.0) 780 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:13:19.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:13:19 smithi100 bash[21346]: cephadm 2024-04-07T16:13:18.852543+0000 mgr.y (mgr.14553) 38 : cephadm 1 Deploying daemon grafana.a on smithi144 2024-04-07T16:13:19.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:13:19 smithi100 bash[21346]: audit 2024-04-07T16:13:19.588670+0000 mon.b (mon.1) 66 : audit 1 from='client.? 172.21.15.144:0/3178124117' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-07T16:13:19.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:13:19 smithi100 bash[21346]: audit 2024-04-07T16:13:19.594486+0000 mon.a (mon.0) 781 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-07T16:13:19.941 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:13:19 smithi100 bash[21346]: audit 2024-04-07T16:13:19.601793+0000 mon.a (mon.0) 782 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-07T16:13:19.941 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:13:19 smithi100 bash[31660]: cluster 2024-04-07T16:13:18.610449+0000 mgr.y (mgr.14553) 35 : cluster 0 pgmap v16: 97 pgs: 97 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:13:19.941 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:13:19 smithi100 bash[31660]: audit 2024-04-07T16:13:18.640700+0000 mon.a (mon.0) 775 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:13:19.941 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:13:19 smithi100 bash[31660]: audit 2024-04-07T16:13:18.657262+0000 mon.b (mon.1) 63 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:13:19.941 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:13:19 smithi100 bash[31660]: audit 2024-04-07T16:13:18.657624+0000 mon.a (mon.0) 776 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:13:19.941 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:13:19 smithi100 bash[31660]: audit 2024-04-07T16:13:18.659370+0000 mon.b (mon.1) 64 : audit 1 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:13:19.941 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:13:19 smithi100 bash[31660]: audit 2024-04-07T16:13:18.678324+0000 mon.a (mon.0) 777 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:13:19.941 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:13:19 smithi100 bash[31660]: cephadm 2024-04-07T16:13:18.716104+0000 mgr.y (mgr.14553) 36 : cephadm 1 Regenerating cephadm self-signed grafana TLS certificates 2024-04-07T16:13:19.941 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:13:19 smithi100 bash[31660]: audit 2024-04-07T16:13:18.810166+0000 mon.a (mon.0) 778 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:13:19.941 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:13:19 smithi100 bash[31660]: audit 2024-04-07T16:13:18.822038+0000 mon.a (mon.0) 779 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:13:19.941 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:13:19 smithi100 bash[31660]: audit 2024-04-07T16:13:18.822898+0000 mon.b (mon.1) 65 : audit 1 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-04-07T16:13:19.941 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:13:19 smithi100 bash[31660]: audit 2024-04-07T16:13:18.828283+0000 mgr.y (mgr.14553) 37 : audit 0 from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-04-07T16:13:19.942 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:13:19 smithi100 bash[31660]: audit 2024-04-07T16:13:18.837907+0000 mon.a (mon.0) 780 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:13:19.942 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:13:19 smithi100 bash[31660]: cephadm 2024-04-07T16:13:18.852543+0000 mgr.y (mgr.14553) 38 : cephadm 1 Deploying daemon grafana.a on smithi144 2024-04-07T16:13:19.942 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:13:19 smithi100 bash[31660]: audit 2024-04-07T16:13:19.588670+0000 mon.b (mon.1) 66 : audit 1 from='client.? 172.21.15.144:0/3178124117' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-07T16:13:19.942 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:13:19 smithi100 bash[31660]: audit 2024-04-07T16:13:19.594486+0000 mon.a (mon.0) 781 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-07T16:13:19.942 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:13:19 smithi100 bash[31660]: audit 2024-04-07T16:13:19.601793+0000 mon.a (mon.0) 782 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-07T16:13:20.431 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-04-07T16:13:20.432 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/etc/ceph/ceph.client.1.keyring 2024-04-07T16:13:20.432 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-04-07T16:13:20.454 INFO:tasks.cephadm:Waiting for OSDs to come up 2024-04-07T16:13:20.455 DEBUG:teuthology.orchestra.run.smithi100:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-07T16:13:20.765 INFO:teuthology.misc.health.smithi100.stdout: 2024-04-07T16:13:20.765 INFO:teuthology.misc.health.smithi100.stdout:{"epoch":65,"fsid":"0702e3a8-f4f8-11ee-b649-cb9ed24678a4","created":"2024-04-07T16:02:21.629692+0000","modified":"2024-04-07T16:12:52.577573+0000","last_up_change":"2024-04-07T16:11:30.210001+0000","last_in_change":"2024-04-07T16:11:04.793434+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-07T16:07:26.784071+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":7.8899998664855957,"score_stable":7.8899998664855957,"optimal_score":0.37999999523162842,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-07T16:12:11.856505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-07T16:12:12.737709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8facd48e-0a0c-4fda-aee2-f001b60a3811","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":1636950303},{"type":"v1","addr":"172.21.15.100:6803","nonce":1636950303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":1636950303},{"type":"v1","addr":"172.21.15.100:6805","nonce":1636950303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1636950303},{"type":"v1","addr":"172.21.15.100:6809","nonce":1636950303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":1636950303},{"type":"v1","addr":"172.21.15.100:6807","nonce":1636950303}]},"public_addr":"172.21.15.100:6803/1636950303","cluster_addr":"172.21.15.100:6805/1636950303","heartbeat_back_addr":"172.21.15.100:6809/1636950303","heartbeat_front_addr":"172.21.15.100:6807/1636950303","state":["exists","up"]},{"osd":1,"uuid":"cb5f6c23-6764-48b6-8b21-97a36b004786","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":470179506},{"type":"v1","addr":"172.21.15.100:6811","nonce":470179506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":470179506},{"type":"v1","addr":"172.21.15.100:6813","nonce":470179506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":470179506},{"type":"v1","addr":"172.21.15.100:6817","nonce":470179506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":470179506},{"type":"v1","addr":"172.21.15.100:6815","nonce":470179506}]},"public_addr":"172.21.15.100:6811/470179506","cluster_addr":"172.21.15.100:6813/470179506","heartbeat_back_addr":"172.21.15.100:6817/470179506","heartbeat_front_addr":"172.21.15.100:6815/470179506","state":["exists","up"]},{"osd":2,"uuid":"1650aedb-9ce9-4c14-a059-e1f089e5e63f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":2235490011},{"type":"v1","addr":"172.21.15.100:6819","nonce":2235490011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2235490011},{"type":"v1","addr":"172.21.15.100:6821","nonce":2235490011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":2235490011},{"type":"v1","addr":"172.21.15.100:6825","nonce":2235490011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":2235490011},{"type":"v1","addr":"172.21.15.100:6823","nonce":2235490011}]},"public_addr":"172.21.15.100:6819/2235490011","cluster_addr":"172.21.15.100:6821/2235490011","heartbeat_back_addr":"172.21.15.100:6825/2235490011","heartbeat_front_addr":"172.21.15.100:6823/2235490011","state":["exists","up"]},{"osd":3,"uuid":"17591c6b-862b-43b2-858b-02f4c3199247","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":3019212393},{"type":"v1","addr":"172.21.15.100:6827","nonce":3019212393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":3019212393},{"type":"v1","addr":"172.21.15.100:6829","nonce":3019212393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":3019212393},{"type":"v1","addr":"172.21.15.100:6833","nonce":3019212393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":3019212393},{"type":"v1","addr":"172.21.15.100:6831","nonce":3019212393}]},"public_addr":"172.21.15.100:6827/3019212393","cluster_addr":"172.21.15.100:6829/3019212393","heartbeat_back_addr":"172.21.15.100:6833/3019212393","heartbeat_front_addr":"172.21.15.100:6831/3019212393","state":["exists","up"]},{"osd":4,"uuid":"6ecc1205-7c78-461e-8037-640136cb3658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":974906744},{"type":"v1","addr":"172.21.15.144:6801","nonce":974906744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":974906744},{"type":"v1","addr":"172.21.15.144:6803","nonce":974906744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":974906744},{"type":"v1","addr":"172.21.15.144:6807","nonce":974906744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":974906744},{"type":"v1","addr":"172.21.15.144:6805","nonce":974906744}]},"public_addr":"172.21.15.144:6801/974906744","cluster_addr":"172.21.15.144:6803/974906744","heartbeat_back_addr":"172.21.15.144:6807/974906744","heartbeat_front_addr":"172.21.15.144:6805/974906744","state":["exists","up"]},{"osd":5,"uuid":"54789cc8-1c3c-46a4-a92e-8881564338db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":2783162058},{"type":"v1","addr":"172.21.15.144:6809","nonce":2783162058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":2783162058},{"type":"v1","addr":"172.21.15.144:6811","nonce":2783162058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":2783162058},{"type":"v1","addr":"172.21.15.144:6815","nonce":2783162058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":2783162058},{"type":"v1","addr":"172.21.15.144:6813","nonce":2783162058}]},"public_addr":"172.21.15.144:6809/2783162058","cluster_addr":"172.21.15.144:6811/2783162058","heartbeat_back_addr":"172.21.15.144:6815/2783162058","heartbeat_front_addr":"172.21.15.144:6813/2783162058","state":["exists","up"]},{"osd":6,"uuid":"ac619497-7605-4ca6-84bf-3abb3f6f32d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":1183840595},{"type":"v1","addr":"172.21.15.144:6817","nonce":1183840595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":1183840595},{"type":"v1","addr":"172.21.15.144:6819","nonce":1183840595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":1183840595},{"type":"v1","addr":"172.21.15.144:6823","nonce":1183840595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":1183840595},{"type":"v1","addr":"172.21.15.144:6821","nonce":1183840595}]},"public_addr":"172.21.15.144:6817/1183840595","cluster_addr":"172.21.15.144:6819/1183840595","heartbeat_back_addr":"172.21.15.144:6823/1183840595","heartbeat_front_addr":"172.21.15.144:6821/1183840595","state":["exists","up"]},{"osd":7,"uuid":"b85b3422-9da2-4605-88e7-fe9c3b2020cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":2248617613},{"type":"v1","addr":"172.21.15.144:6825","nonce":2248617613}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":2248617613},{"type":"v1","addr":"172.21.15.144:6827","nonce":2248617613}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":2248617613},{"type":"v1","addr":"172.21.15.144:6831","nonce":2248617613}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":2248617613},{"type":"v1","addr":"172.21.15.144:6829","nonce":2248617613}]},"public_addr":"172.21.15.144:6825/2248617613","cluster_addr":"172.21.15.144:6827/2248617613","heartbeat_back_addr":"172.21.15.144:6831/2248617613","heartbeat_front_addr":"172.21.15.144:6829/2248617613","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-07T16:05:44.696671+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-07T16:06:33.787321+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-07T16:07:22.983158+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-07T16:08:12.935606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-07T16:09:00.218619+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-07T16:09:50.222185+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-07T16:10:37.856158+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-07T16:11:28.267877+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.7","mappings":[{"from":7,"to":4}]},{"pgid":"3.1","mappings":[{"from":0,"to":2}]},{"pgid":"3.12","mappings":[{"from":0,"to":2}]},{"pgid":"3.15","mappings":[{"from":4,"to":1}]},{"pgid":"3.17","mappings":[{"from":0,"to":2}]},{"pgid":"3.1d","mappings":[{"from":4,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.100:0/74983397":"2024-04-08T16:12:52.577534+0000","172.21.15.100:6800/190788518":"2024-04-08T16:12:52.577534+0000","172.21.15.100:0/1309290877":"2024-04-08T16:12:52.577534+0000","172.21.15.100:0/1472671368":"2024-04-08T16:03:24.146096+0000","172.21.15.100:6801/190788518":"2024-04-08T16:12:52.577534+0000","172.21.15.100:0/639120693":"2024-04-08T16:03:24.146096+0000","172.21.15.100:0/2474331937":"2024-04-08T16:03:24.146096+0000","172.21.15.100:0/3273267396":"2024-04-08T16:12:52.577534+0000","172.21.15.100:0/2535336108":"2024-04-08T16:12:52.577534+0000","172.21.15.100:6801/2710911757":"2024-04-08T16:03:24.146096+0000","172.21.15.100:6800/3440287045":"2024-04-08T16:02:53.269514+0000","172.21.15.100:0/109071285":"2024-04-08T16:02:53.269514+0000","172.21.15.100:6801/3440287045":"2024-04-08T16:02:53.269514+0000","172.21.15.100:0/4162615562":"2024-04-08T16:02:53.269514+0000","172.21.15.100:0/4121992264":"2024-04-08T16:02:53.269514+0000","172.21.15.100:6800/2710911757":"2024-04-08T16:03:24.146096+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-07T16:13:20.784 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-04-07T16:13:20.784 INFO:tasks.cephadm:Creating RBD pool 2024-04-07T16:13:20.784 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0702e3a8-f4f8-11ee-b649-cb9ed24678a4 -- sudo ceph --cluster ceph osd pool create rbd 8 2024-04-07T16:13:21.939 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:13:21 smithi100 bash[21346]: cluster 2024-04-07T16:13:20.611046+0000 mgr.y (mgr.14553) 39 : cluster 0 pgmap v17: 97 pgs: 97 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:13:21.939 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:13:21 smithi100 bash[21346]: audit 2024-04-07T16:13:20.776248+0000 mon.a (mon.0) 783 : audit 0 from='client.? 172.21.15.100:0/2665887245' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-07T16:13:21.939 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:13:21 smithi100 bash[31660]: cluster 2024-04-07T16:13:20.611046+0000 mgr.y (mgr.14553) 39 : cluster 0 pgmap v17: 97 pgs: 97 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:13:21.939 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:13:21 smithi100 bash[31660]: audit 2024-04-07T16:13:20.776248+0000 mon.a (mon.0) 783 : audit 0 from='client.? 172.21.15.100:0/2665887245' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-07T16:13:22.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:13:21 smithi144 bash[23522]: cluster 2024-04-07T16:13:20.611046+0000 mgr.y (mgr.14553) 39 : cluster 0 pgmap v17: 97 pgs: 97 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:13:22.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:13:21 smithi144 bash[23522]: audit 2024-04-07T16:13:20.776248+0000 mon.a (mon.0) 783 : audit 0 from='client.? 172.21.15.100:0/2665887245' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-07T16:13:23.939 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Apr 07 16:13:23 smithi100 bash[58158]: ts=2024-04-07T16:13:23.684Z caller=cluster.go:698 level=info component=cluster msg="gossip settled; proceeding" elapsed=10.003873964s 2024-04-07T16:13:23.939 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:13:23 smithi100 bash[21346]: cluster 2024-04-07T16:13:22.611688+0000 mgr.y (mgr.14553) 40 : cluster 0 pgmap v18: 97 pgs: 97 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:13:23.939 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:13:23 smithi100 bash[21346]: audit 2024-04-07T16:13:22.691400+0000 mon.b (mon.1) 67 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:13:23.939 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:13:23 smithi100 bash[31660]: cluster 2024-04-07T16:13:22.611688+0000 mgr.y (mgr.14553) 40 : cluster 0 pgmap v18: 97 pgs: 97 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:13:23.940 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:13:23 smithi100 bash[31660]: audit 2024-04-07T16:13:22.691400+0000 mon.b (mon.1) 67 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:13:24.108 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:13:23 smithi144 bash[23522]: cluster 2024-04-07T16:13:22.611688+0000 mgr.y (mgr.14553) 40 : cluster 0 pgmap v18: 97 pgs: 97 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:13:24.108 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:13:23 smithi144 bash[23522]: audit 2024-04-07T16:13:22.691400+0000 mon.b (mon.1) 67 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:13:25.439 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:13:25 smithi100 bash[21601]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:13:25] "GET /metrics HTTP/1.1" 200 31381 "" "Prometheus/2.43.0" 2024-04-07T16:13:25.507 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/mon.a/config 2024-04-07T16:13:25.939 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:13:25 smithi100 bash[21346]: cluster 2024-04-07T16:13:24.612870+0000 mgr.y (mgr.14553) 41 : cluster 0 pgmap v19: 97 pgs: 97 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:13:25.939 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:13:25 smithi100 bash[31660]: cluster 2024-04-07T16:13:24.612870+0000 mgr.y (mgr.14553) 41 : cluster 0 pgmap v19: 97 pgs: 97 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:13:26.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:13:25 smithi144 bash[23522]: cluster 2024-04-07T16:13:24.612870+0000 mgr.y (mgr.14553) 41 : cluster 0 pgmap v19: 97 pgs: 97 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:13:27.939 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:13:27 smithi100 bash[21346]: cluster 2024-04-07T16:13:26.613263+0000 mgr.y (mgr.14553) 42 : cluster 0 pgmap v20: 97 pgs: 97 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:13:27.939 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:13:27 smithi100 bash[31660]: cluster 2024-04-07T16:13:26.613263+0000 mgr.y (mgr.14553) 42 : cluster 0 pgmap v20: 97 pgs: 97 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:13:28.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:13:27 smithi144 bash[23522]: cluster 2024-04-07T16:13:26.613263+0000 mgr.y (mgr.14553) 42 : cluster 0 pgmap v20: 97 pgs: 97 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:13:28.968 INFO:teuthology.orchestra.run.smithi100.stderr:pool 'rbd' created 2024-04-07T16:13:29.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:13:28 smithi144 bash[23522]: audit 2024-04-07T16:13:28.477435+0000 mon.c (mon.2) 31 : audit 1 from='client.? 172.21.15.100:0/3310299812' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]: dispatch 2024-04-07T16:13:29.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:13:28 smithi144 bash[23522]: audit 2024-04-07T16:13:28.478053+0000 mon.a (mon.0) 784 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]: dispatch 2024-04-07T16:13:29.439 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:13:28 smithi100 bash[21346]: audit 2024-04-07T16:13:28.477435+0000 mon.c (mon.2) 31 : audit 1 from='client.? 172.21.15.100:0/3310299812' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]: dispatch 2024-04-07T16:13:29.439 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:13:28 smithi100 bash[21346]: audit 2024-04-07T16:13:28.478053+0000 mon.a (mon.0) 784 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]: dispatch 2024-04-07T16:13:29.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:13:28 smithi100 bash[31660]: audit 2024-04-07T16:13:28.477435+0000 mon.c (mon.2) 31 : audit 1 from='client.? 172.21.15.100:0/3310299812' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]: dispatch 2024-04-07T16:13:29.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:13:28 smithi100 bash[31660]: audit 2024-04-07T16:13:28.478053+0000 mon.a (mon.0) 784 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]: dispatch 2024-04-07T16:13:29.594 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0702e3a8-f4f8-11ee-b649-cb9ed24678a4 -- sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2024-04-07T16:13:30.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:13:29 smithi144 bash[23522]: cluster 2024-04-07T16:13:28.613811+0000 mgr.y (mgr.14553) 43 : cluster 0 pgmap v21: 97 pgs: 97 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:13:30.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:13:29 smithi144 bash[23522]: audit 2024-04-07T16:13:28.971636+0000 mon.a (mon.0) 785 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]': finished 2024-04-07T16:13:30.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:13:29 smithi144 bash[23522]: cluster 2024-04-07T16:13:28.971853+0000 mon.a (mon.0) 786 : cluster 0 osdmap e66: 8 total, 8 up, 8 in 2024-04-07T16:13:30.439 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:13:29 smithi100 bash[21346]: cluster 2024-04-07T16:13:28.613811+0000 mgr.y (mgr.14553) 43 : cluster 0 pgmap v21: 97 pgs: 97 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:13:30.439 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:13:29 smithi100 bash[21346]: audit 2024-04-07T16:13:28.971636+0000 mon.a (mon.0) 785 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]': finished 2024-04-07T16:13:30.439 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:13:29 smithi100 bash[21346]: cluster 2024-04-07T16:13:28.971853+0000 mon.a (mon.0) 786 : cluster 0 osdmap e66: 8 total, 8 up, 8 in 2024-04-07T16:13:30.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:13:29 smithi100 bash[31660]: cluster 2024-04-07T16:13:28.613811+0000 mgr.y (mgr.14553) 43 : cluster 0 pgmap v21: 97 pgs: 97 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:13:30.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:13:29 smithi100 bash[31660]: audit 2024-04-07T16:13:28.971636+0000 mon.a (mon.0) 785 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]': finished 2024-04-07T16:13:30.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:13:29 smithi100 bash[31660]: cluster 2024-04-07T16:13:28.971853+0000 mon.a (mon.0) 786 : cluster 0 osdmap e66: 8 total, 8 up, 8 in 2024-04-07T16:13:31.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:13:30 smithi144 bash[23522]: cluster 2024-04-07T16:13:30.000107+0000 mon.a (mon.0) 787 : cluster 0 osdmap e67: 8 total, 8 up, 8 in 2024-04-07T16:13:31.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:13:31 smithi100 bash[21346]: cluster 2024-04-07T16:13:30.000107+0000 mon.a (mon.0) 787 : cluster 0 osdmap e67: 8 total, 8 up, 8 in 2024-04-07T16:13:31.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:13:31 smithi100 bash[31660]: cluster 2024-04-07T16:13:30.000107+0000 mon.a (mon.0) 787 : cluster 0 osdmap e67: 8 total, 8 up, 8 in 2024-04-07T16:13:32.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:13:32 smithi144 bash[23522]: cluster 2024-04-07T16:13:30.614538+0000 mgr.y (mgr.14553) 44 : cluster 0 pgmap v24: 105 pgs: 8 unknown, 97 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:13:32.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:13:32 smithi144 bash[23522]: cluster 2024-04-07T16:13:30.993875+0000 mon.a (mon.0) 788 : cluster 3 Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T16:13:32.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:13:32 smithi144 bash[23522]: cluster 2024-04-07T16:13:31.010569+0000 mon.a (mon.0) 789 : cluster 0 osdmap e68: 8 total, 8 up, 8 in 2024-04-07T16:13:32.439 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:13:32 smithi100 bash[21346]: cluster 2024-04-07T16:13:30.614538+0000 mgr.y (mgr.14553) 44 : cluster 0 pgmap v24: 105 pgs: 8 unknown, 97 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:13:32.439 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:13:32 smithi100 bash[21346]: cluster 2024-04-07T16:13:30.993875+0000 mon.a (mon.0) 788 : cluster 3 Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T16:13:32.439 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:13:32 smithi100 bash[21346]: cluster 2024-04-07T16:13:31.010569+0000 mon.a (mon.0) 789 : cluster 0 osdmap e68: 8 total, 8 up, 8 in 2024-04-07T16:13:32.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:13:32 smithi100 bash[31660]: cluster 2024-04-07T16:13:30.614538+0000 mgr.y (mgr.14553) 44 : cluster 0 pgmap v24: 105 pgs: 8 unknown, 97 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:13:32.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:13:32 smithi100 bash[31660]: cluster 2024-04-07T16:13:30.993875+0000 mon.a (mon.0) 788 : cluster 3 Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T16:13:32.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:13:32 smithi100 bash[31660]: cluster 2024-04-07T16:13:31.010569+0000 mon.a (mon.0) 789 : cluster 0 osdmap e68: 8 total, 8 up, 8 in 2024-04-07T16:13:34.336 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/mon.a/config 2024-04-07T16:13:34.357 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:13:34 smithi100 bash[21346]: cluster 2024-04-07T16:13:32.615276+0000 mgr.y (mgr.14553) 45 : cluster 0 pgmap v26: 105 pgs: 8 unknown, 97 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:13:34.357 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:13:34 smithi100 bash[31660]: cluster 2024-04-07T16:13:32.615276+0000 mgr.y (mgr.14553) 45 : cluster 0 pgmap v26: 105 pgs: 8 unknown, 97 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:13:34.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:13:34 smithi144 bash[23522]: cluster 2024-04-07T16:13:32.615276+0000 mgr.y (mgr.14553) 45 : cluster 0 pgmap v26: 105 pgs: 8 unknown, 97 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:13:35.438 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:13:35 smithi100 bash[21601]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:13:35] "GET /metrics HTTP/1.1" 200 31381 "" "Prometheus/2.43.0" 2024-04-07T16:13:36.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:13:36 smithi144 bash[23522]: cluster 2024-04-07T16:13:34.616588+0000 mgr.y (mgr.14553) 46 : cluster 0 pgmap v27: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:13:36.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:13:36 smithi100 bash[21346]: cluster 2024-04-07T16:13:34.616588+0000 mgr.y (mgr.14553) 46 : cluster 0 pgmap v27: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:13:36.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:13:36 smithi100 bash[31660]: cluster 2024-04-07T16:13:34.616588+0000 mgr.y (mgr.14553) 46 : cluster 0 pgmap v27: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:13:38.021 INFO:teuthology.orchestra.run.smithi100.stderr:enabled application 'rbd' on pool 'rbd' 2024-04-07T16:13:38.256 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:13:38 smithi100 bash[21346]: cluster 2024-04-07T16:13:36.617284+0000 mgr.y (mgr.14553) 47 : cluster 0 pgmap v28: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:13:38.256 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:13:38 smithi100 bash[21346]: audit 2024-04-07T16:13:37.081672+0000 mon.a (mon.0) 790 : audit 1 from='client.? 172.21.15.100:0/799702122' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:13:38.257 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:13:38 smithi100 bash[21346]: audit 2024-04-07T16:13:37.700818+0000 mon.b (mon.1) 68 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:13:38.257 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:13:38 smithi100 bash[21346]: audit 2024-04-07T16:13:37.703100+0000 mon.a (mon.0) 791 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:13:38.257 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:13:38 smithi100 bash[31660]: cluster 2024-04-07T16:13:36.617284+0000 mgr.y (mgr.14553) 47 : cluster 0 pgmap v28: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:13:38.257 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:13:38 smithi100 bash[31660]: audit 2024-04-07T16:13:37.081672+0000 mon.a (mon.0) 790 : audit 1 from='client.? 172.21.15.100:0/799702122' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:13:38.257 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:13:38 smithi100 bash[31660]: audit 2024-04-07T16:13:37.700818+0000 mon.b (mon.1) 68 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:13:38.257 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:13:38 smithi100 bash[31660]: audit 2024-04-07T16:13:37.703100+0000 mon.a (mon.0) 791 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:13:38.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:13:38 smithi144 bash[23522]: cluster 2024-04-07T16:13:36.617284+0000 mgr.y (mgr.14553) 47 : cluster 0 pgmap v28: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:13:38.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:13:38 smithi144 bash[23522]: audit 2024-04-07T16:13:37.081672+0000 mon.a (mon.0) 790 : audit 1 from='client.? 172.21.15.100:0/799702122' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:13:38.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:13:38 smithi144 bash[23522]: audit 2024-04-07T16:13:37.700818+0000 mon.b (mon.1) 68 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:13:38.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:13:38 smithi144 bash[23522]: audit 2024-04-07T16:13:37.703100+0000 mon.a (mon.0) 791 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:13:38.703 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-07T16:13:38.703 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-04-07T16:13:38.703 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 0702e3a8-f4f8-11ee-b649-cb9ed24678a4 -- ceph mgr dump --format=json 2024-04-07T16:13:39.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:13:39 smithi144 bash[23522]: audit 2024-04-07T16:13:38.030656+0000 mon.a (mon.0) 792 : audit 1 from='client.? 172.21.15.100:0/799702122' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]': finished 2024-04-07T16:13:39.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:13:39 smithi144 bash[23522]: cluster 2024-04-07T16:13:38.030831+0000 mon.a (mon.0) 793 : cluster 0 osdmap e69: 8 total, 8 up, 8 in 2024-04-07T16:13:39.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:13:39 smithi100 bash[21346]: audit 2024-04-07T16:13:38.030656+0000 mon.a (mon.0) 792 : audit 1 from='client.? 172.21.15.100:0/799702122' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]': finished 2024-04-07T16:13:39.439 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:13:39 smithi100 bash[21346]: cluster 2024-04-07T16:13:38.030831+0000 mon.a (mon.0) 793 : cluster 0 osdmap e69: 8 total, 8 up, 8 in 2024-04-07T16:13:39.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:13:39 smithi100 bash[31660]: audit 2024-04-07T16:13:38.030656+0000 mon.a (mon.0) 792 : audit 1 from='client.? 172.21.15.100:0/799702122' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]': finished 2024-04-07T16:13:39.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:13:39 smithi100 bash[31660]: cluster 2024-04-07T16:13:38.030831+0000 mon.a (mon.0) 793 : cluster 0 osdmap e69: 8 total, 8 up, 8 in 2024-04-07T16:13:40.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:13:40 smithi144 bash[23522]: cluster 2024-04-07T16:13:38.617965+0000 mgr.y (mgr.14553) 48 : cluster 0 pgmap v30: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:13:40.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:13:40 smithi144 bash[23522]: cluster 2024-04-07T16:13:39.024252+0000 mon.a (mon.0) 794 : cluster 1 Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T16:13:40.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:13:40 smithi144 bash[23522]: cluster 2024-04-07T16:13:39.024304+0000 mon.a (mon.0) 795 : cluster 1 Cluster is now healthy 2024-04-07T16:13:40.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:13:40 smithi100 bash[21346]: cluster 2024-04-07T16:13:38.617965+0000 mgr.y (mgr.14553) 48 : cluster 0 pgmap v30: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:13:40.439 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:13:40 smithi100 bash[21346]: cluster 2024-04-07T16:13:39.024252+0000 mon.a (mon.0) 794 : cluster 1 Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T16:13:40.439 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:13:40 smithi100 bash[21346]: cluster 2024-04-07T16:13:39.024304+0000 mon.a (mon.0) 795 : cluster 1 Cluster is now healthy 2024-04-07T16:13:40.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:13:40 smithi100 bash[31660]: cluster 2024-04-07T16:13:38.617965+0000 mgr.y (mgr.14553) 48 : cluster 0 pgmap v30: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:13:40.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:13:40 smithi100 bash[31660]: cluster 2024-04-07T16:13:39.024252+0000 mon.a (mon.0) 794 : cluster 1 Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T16:13:40.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:13:40 smithi100 bash[31660]: cluster 2024-04-07T16:13:39.024304+0000 mon.a (mon.0) 795 : cluster 1 Cluster is now healthy 2024-04-07T16:13:42.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:13:42 smithi144 bash[23522]: cluster 2024-04-07T16:13:40.619197+0000 mgr.y (mgr.14553) 49 : cluster 0 pgmap v31: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:13:42.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:13:42 smithi100 bash[21346]: cluster 2024-04-07T16:13:40.619197+0000 mgr.y (mgr.14553) 49 : cluster 0 pgmap v31: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:13:42.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:13:42 smithi100 bash[31660]: cluster 2024-04-07T16:13:40.619197+0000 mgr.y (mgr.14553) 49 : cluster 0 pgmap v31: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:13:43.443 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/mon.a/config 2024-04-07T16:13:44.310 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:13:44 smithi100 bash[21346]: cluster 2024-04-07T16:13:42.619891+0000 mgr.y (mgr.14553) 50 : cluster 0 pgmap v32: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:13:44.310 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:13:44 smithi100 bash[21346]: audit 2024-04-07T16:13:43.064307+0000 mon.a (mon.0) 796 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:13:44.310 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:13:44 smithi100 bash[31660]: cluster 2024-04-07T16:13:42.619891+0000 mgr.y (mgr.14553) 50 : cluster 0 pgmap v32: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:13:44.310 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:13:44 smithi100 bash[31660]: audit 2024-04-07T16:13:43.064307+0000 mon.a (mon.0) 796 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:13:44.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:13:44 smithi144 bash[23522]: cluster 2024-04-07T16:13:42.619891+0000 mgr.y (mgr.14553) 50 : cluster 0 pgmap v32: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:13:44.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:13:44 smithi144 bash[23522]: audit 2024-04-07T16:13:43.064307+0000 mon.a (mon.0) 796 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:13:45.438 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:13:45 smithi100 bash[21601]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:13:45] "GET /metrics HTTP/1.1" 200 33731 "" "Prometheus/2.43.0" 2024-04-07T16:13:46.214 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-07T16:13:46.313 INFO:journalctl@ceph.osd.6.smithi144.stdout:Apr 07 16:13:46 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:13:46.313 INFO:journalctl@ceph.mds.b.smithi144.stdout:Apr 07 16:13:46 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:13:46.313 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:13:46 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:13:46.313 INFO:journalctl@ceph.osd.7.smithi144.stdout:Apr 07 16:13:46 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:13:46.314 INFO:journalctl@ceph.osd.4.smithi144.stdout:Apr 07 16:13:46 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:13:46.314 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:13:46 smithi144 bash[23522]: cluster 2024-04-07T16:13:44.621173+0000 mgr.y (mgr.14553) 51 : cluster 0 pgmap v33: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:13:46.314 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:13:46 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:13:46.314 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 16:13:46 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:13:46.315 INFO:journalctl@ceph.osd.5.smithi144.stdout:Apr 07 16:13:46 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:13:46.315 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:13:46 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:13:46.420 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:13:46 smithi100 bash[21346]: cluster 2024-04-07T16:13:44.621173+0000 mgr.y (mgr.14553) 51 : cluster 0 pgmap v33: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:13:46.421 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:13:46 smithi100 bash[31660]: cluster 2024-04-07T16:13:44.621173+0000 mgr.y (mgr.14553) 51 : cluster 0 pgmap v33: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:13:46.564 INFO:journalctl@ceph.mds.b.smithi144.stdout:Apr 07 16:13:46 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:13:46.565 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:13:46 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:13:46.565 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:13:46 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:13:46.565 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 16:13:46 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:13:46.566 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:46 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:13:46.566 INFO:journalctl@ceph.osd.5.smithi144.stdout:Apr 07 16:13:46 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:13:46.566 INFO:journalctl@ceph.osd.6.smithi144.stdout:Apr 07 16:13:46 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:13:46.566 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:13:46 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:13:46.567 INFO:journalctl@ceph.osd.4.smithi144.stdout:Apr 07 16:13:46 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:13:46.567 INFO:journalctl@ceph.osd.7.smithi144.stdout:Apr 07 16:13:46 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:13:46.832 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":20,"flags":0,"active_gid":14553,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":3429225708},{"type":"v1","addr":"172.21.15.100:6801","nonce":3429225708}]},"active_addr":"172.21.15.100:6801/3429225708","active_change":"2024-04-07T16:12:52.577784+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":24373,"name":"x","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","prometheus","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.100:8443/","prometheus":"http://172.21.15.100:9283/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":65,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.100:0","nonce":2558802424}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.100:0","nonce":1631130639}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.100:0","nonce":218414173}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.100:0","nonce":353657034}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.100:0","nonce":4208004927}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.100:0","nonce":912272369}]}]} 2024-04-07T16:13:46.837 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-04-07T16:13:46.837 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-04-07T16:13:46.837 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 0702e3a8-f4f8-11ee-b649-cb9ed24678a4 -- ceph osd dump --format=json 2024-04-07T16:13:46.856 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:46 smithi144 systemd[1]: Started Ceph grafana.a for 0702e3a8-f4f8-11ee-b649-cb9ed24678a4. 2024-04-07T16:13:47.188 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:13:47 smithi100 bash[21346]: audit 2024-04-07T16:13:46.217507+0000 mon.c (mon.2) 32 : audit 0 from='client.? 172.21.15.100:0/2773385644' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-07T16:13:47.189 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:13:47 smithi100 bash[21346]: audit 2024-04-07T16:13:46.625839+0000 mon.a (mon.0) 797 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:13:47.189 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:13:47 smithi100 bash[21346]: audit 2024-04-07T16:13:46.639625+0000 mon.a (mon.0) 798 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:13:47.189 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:13:47 smithi100 bash[21346]: audit 2024-04-07T16:13:46.700447+0000 mon.a (mon.0) 799 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:13:47.189 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:13:47 smithi100 bash[21346]: audit 2024-04-07T16:13:46.709511+0000 mon.a (mon.0) 800 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:13:47.189 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:13:47 smithi100 bash[21346]: audit 2024-04-07T16:13:46.772308+0000 mon.b (mon.1) 69 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:13:47.189 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:13:47 smithi100 bash[31660]: audit 2024-04-07T16:13:46.217507+0000 mon.c (mon.2) 32 : audit 0 from='client.? 172.21.15.100:0/2773385644' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-07T16:13:47.189 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:13:47 smithi100 bash[31660]: audit 2024-04-07T16:13:46.625839+0000 mon.a (mon.0) 797 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:13:47.190 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:13:47 smithi100 bash[31660]: audit 2024-04-07T16:13:46.639625+0000 mon.a (mon.0) 798 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:13:47.190 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:13:47 smithi100 bash[31660]: audit 2024-04-07T16:13:46.700447+0000 mon.a (mon.0) 799 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:13:47.190 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:13:47 smithi100 bash[31660]: audit 2024-04-07T16:13:46.709511+0000 mon.a (mon.0) 800 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:13:47.190 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:13:47 smithi100 bash[31660]: audit 2024-04-07T16:13:46.772308+0000 mon.b (mon.1) 69 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:13:47.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:13:47 smithi144 bash[23522]: audit 2024-04-07T16:13:46.217507+0000 mon.c (mon.2) 32 : audit 0 from='client.? 172.21.15.100:0/2773385644' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-07T16:13:47.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:13:47 smithi144 bash[23522]: audit 2024-04-07T16:13:46.625839+0000 mon.a (mon.0) 797 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:13:47.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:13:47 smithi144 bash[23522]: audit 2024-04-07T16:13:46.639625+0000 mon.a (mon.0) 798 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:13:47.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:13:47 smithi144 bash[23522]: audit 2024-04-07T16:13:46.700447+0000 mon.a (mon.0) 799 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:13:47.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:13:47 smithi144 bash[23522]: audit 2024-04-07T16:13:46.709511+0000 mon.a (mon.0) 800 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:13:47.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:13:47 smithi144 bash[23522]: audit 2024-04-07T16:13:46.772308+0000 mon.b (mon.1) 69 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:13:47.605 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:47 smithi144 bash[50386]: logger=settings t=2024-04-07T16:13:47.613977793Z level=info msg="Starting Grafana" version=9.4.7 commit=4add91f03d branch=HEAD compiled=2023-03-16T23:56:52Z 2024-04-07T16:13:47.606 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:47 smithi144 bash[50386]: logger=settings t=2024-04-07T16:13:47.614205049Z level=warn msg="\"sentry\" frontend logging provider is deprecated and will be removed in the next major version. Use \"grafana\" provider instead." 2024-04-07T16:13:47.606 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:47 smithi144 bash[50386]: logger=settings t=2024-04-07T16:13:47.614231076Z level=info msg="Config loaded from" file=/usr/share/grafana/conf/defaults.ini 2024-04-07T16:13:47.606 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:47 smithi144 bash[50386]: logger=settings t=2024-04-07T16:13:47.614239269Z level=info msg="Config loaded from" file=/etc/grafana/grafana.ini 2024-04-07T16:13:47.858 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:47 smithi144 bash[50386]: logger=settings t=2024-04-07T16:13:47.614246328Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_DATA=/var/lib/grafana" 2024-04-07T16:13:47.858 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:47 smithi144 bash[50386]: logger=settings t=2024-04-07T16:13:47.614253035Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_LOGS=/var/log/grafana" 2024-04-07T16:13:47.858 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:47 smithi144 bash[50386]: logger=settings t=2024-04-07T16:13:47.61425986Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_PLUGINS=/var/lib/grafana/plugins" 2024-04-07T16:13:47.858 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:47 smithi144 bash[50386]: logger=settings t=2024-04-07T16:13:47.614266504Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_PROVISIONING=/etc/grafana/provisioning" 2024-04-07T16:13:47.858 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:47 smithi144 bash[50386]: logger=settings t=2024-04-07T16:13:47.614273544Z level=info msg="Path Home" path=/usr/share/grafana 2024-04-07T16:13:47.858 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:47 smithi144 bash[50386]: logger=settings t=2024-04-07T16:13:47.614280383Z level=info msg="Path Data" path=/var/lib/grafana 2024-04-07T16:13:47.859 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:47 smithi144 bash[50386]: logger=settings t=2024-04-07T16:13:47.61428731Z level=info msg="Path Logs" path=/var/log/grafana 2024-04-07T16:13:47.859 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:47 smithi144 bash[50386]: logger=settings t=2024-04-07T16:13:47.614293849Z level=info msg="Path Plugins" path=/var/lib/grafana/plugins 2024-04-07T16:13:47.859 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:47 smithi144 bash[50386]: logger=settings t=2024-04-07T16:13:47.614300721Z level=info msg="Path Provisioning" path=/etc/grafana/provisioning 2024-04-07T16:13:47.859 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:47 smithi144 bash[50386]: logger=settings t=2024-04-07T16:13:47.614310295Z level=info msg="App mode production" 2024-04-07T16:13:47.859 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:47 smithi144 bash[50386]: logger=sqlstore t=2024-04-07T16:13:47.614378963Z level=info msg="Connecting to DB" dbtype=sqlite3 2024-04-07T16:13:47.859 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:47 smithi144 bash[50386]: logger=sqlstore t=2024-04-07T16:13:47.614409021Z level=warn msg="SQLite database file has broader permissions than it should" path=/var/lib/grafana/grafana.db mode=-rw-r--r-- expected=-rw-r----- 2024-04-07T16:13:47.859 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:47 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:47.615510852Z level=info msg="Starting DB migrations" 2024-04-07T16:13:47.859 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:47 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:47.615630021Z level=info msg="Executing migration" id="create migration_log table" 2024-04-07T16:13:47.859 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:47 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:47.656377666Z level=info msg="Executing migration" id="create user table" 2024-04-07T16:13:47.859 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:47 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:47.68136469Z level=info msg="Executing migration" id="add unique index user.login" 2024-04-07T16:13:47.859 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:47 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:47.706763111Z level=info msg="Executing migration" id="add unique index user.email" 2024-04-07T16:13:47.859 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:47 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:47.748870826Z level=info msg="Executing migration" id="drop index UQE_user_login - v1" 2024-04-07T16:13:47.859 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:47 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:47.790822162Z level=info msg="Executing migration" id="drop index UQE_user_email - v1" 2024-04-07T16:13:47.860 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:47 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:47.832798514Z level=info msg="Executing migration" id="Rename table user to user_v1 - v1" 2024-04-07T16:13:47.860 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:47 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:47.858179481Z level=info msg="Executing migration" id="create user table v2" 2024-04-07T16:13:48.152 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:13:48 smithi144 bash[23522]: cluster 2024-04-07T16:13:46.621802+0000 mgr.y (mgr.14553) 52 : cluster 0 pgmap v34: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:13:48.152 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:47 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:47.900073654Z level=info msg="Executing migration" id="create index UQE_user_login - v2" 2024-04-07T16:13:48.152 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:47 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:47.925392277Z level=info msg="Executing migration" id="create index UQE_user_email - v2" 2024-04-07T16:13:48.152 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:47 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:47.976410778Z level=info msg="Executing migration" id="copy data_source v1 to v2" 2024-04-07T16:13:48.152 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:48 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:48.026051154Z level=info msg="Executing migration" id="Drop old table user_v1" 2024-04-07T16:13:48.152 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:48 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:48.043318586Z level=info msg="Executing migration" id="Add column help_flags1 to user table" 2024-04-07T16:13:48.152 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:48 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:48.060295807Z level=info msg="Executing migration" id="Update user table charset" 2024-04-07T16:13:48.152 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:48 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:48.09402027Z level=info msg="Executing migration" id="Add last_seen_at column to user" 2024-04-07T16:13:48.153 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:48 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:48.135866491Z level=info msg="Executing migration" id="Add missing user data" 2024-04-07T16:13:48.153 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:48 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:48.161219268Z level=info msg="Executing migration" id="Add is_disabled column to user" 2024-04-07T16:13:48.405 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:48 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:48.186575781Z level=info msg="Executing migration" id="Add index user.login/user.email" 2024-04-07T16:13:48.405 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:48 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:48.211854811Z level=info msg="Executing migration" id="Add is_service_account column to user" 2024-04-07T16:13:48.405 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:48 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:48.237259073Z level=info msg="Executing migration" id="Update is_service_account column to nullable" 2024-04-07T16:13:48.405 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:48 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:48.262559197Z level=info msg="Executing migration" id="create temp user table v1-7" 2024-04-07T16:13:48.405 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:48 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:48.30455151Z level=info msg="Executing migration" id="create index IDX_temp_user_email - v1-7" 2024-04-07T16:13:48.405 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:48 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:48.346672555Z level=info msg="Executing migration" id="create index IDX_temp_user_org_id - v1-7" 2024-04-07T16:13:48.405 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:48 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:48.371901186Z level=info msg="Executing migration" id="create index IDX_temp_user_code - v1-7" 2024-04-07T16:13:48.405 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:48 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:48.388829685Z level=info msg="Executing migration" id="create index IDX_temp_user_status - v1-7" 2024-04-07T16:13:48.406 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:48 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:48.414069064Z level=info msg="Executing migration" id="Update temp_user table charset" 2024-04-07T16:13:48.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:13:48 smithi100 bash[21346]: cluster 2024-04-07T16:13:46.621802+0000 mgr.y (mgr.14553) 52 : cluster 0 pgmap v34: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:13:48.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:13:48 smithi100 bash[31660]: cluster 2024-04-07T16:13:46.621802+0000 mgr.y (mgr.14553) 52 : cluster 0 pgmap v34: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:13:48.684 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:48 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:48.43092674Z level=info msg="Executing migration" id="drop index IDX_temp_user_email - v1" 2024-04-07T16:13:48.684 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:48 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:48.45632286Z level=info msg="Executing migration" id="drop index IDX_temp_user_org_id - v1" 2024-04-07T16:13:48.684 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:48 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:48.473185303Z level=info msg="Executing migration" id="drop index IDX_temp_user_code - v1" 2024-04-07T16:13:48.684 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:48 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:48.490054676Z level=info msg="Executing migration" id="drop index IDX_temp_user_status - v1" 2024-04-07T16:13:48.684 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:48 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:48.507069243Z level=info msg="Executing migration" id="Rename table temp_user to temp_user_tmp_qwerty - v1" 2024-04-07T16:13:48.684 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:48 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:48.523975623Z level=info msg="Executing migration" id="create temp_user v2" 2024-04-07T16:13:48.684 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:48 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:48.540848069Z level=info msg="Executing migration" id="create index IDX_temp_user_email - v2" 2024-04-07T16:13:48.684 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:48 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:48.557830398Z level=info msg="Executing migration" id="create index IDX_temp_user_org_id - v2" 2024-04-07T16:13:48.684 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:48 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:48.574802879Z level=info msg="Executing migration" id="create index IDX_temp_user_code - v2" 2024-04-07T16:13:48.684 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:48 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:48.591706878Z level=info msg="Executing migration" id="create index IDX_temp_user_status - v2" 2024-04-07T16:13:48.685 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:48 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:48.608638581Z level=info msg="Executing migration" id="copy temp_user v1 to v2" 2024-04-07T16:13:48.685 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:48 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:48.642222159Z level=info msg="Executing migration" id="drop temp_user_tmp_qwerty" 2024-04-07T16:13:48.685 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:48 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:48.659274753Z level=info msg="Executing migration" id="Set created for temp users that will otherwise prematurely expire" 2024-04-07T16:13:48.685 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:48 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:48.692879029Z level=info msg="Executing migration" id="create star table" 2024-04-07T16:13:48.937 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:48 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:48.743554318Z level=info msg="Executing migration" id="add unique index star.user_id_dashboard_id" 2024-04-07T16:13:48.937 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:48 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:48.768899002Z level=info msg="Executing migration" id="create org table v1" 2024-04-07T16:13:48.937 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:48 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:48.794219832Z level=info msg="Executing migration" id="create index UQE_org_name - v1" 2024-04-07T16:13:48.937 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:48 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:48.836146019Z level=info msg="Executing migration" id="create org_user table v1" 2024-04-07T16:13:48.937 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:48 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:48.878107595Z level=info msg="Executing migration" id="create index IDX_org_user_org_id - v1" 2024-04-07T16:13:48.937 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:48 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:48.903515011Z level=info msg="Executing migration" id="create index UQE_org_user_org_id_user_id - v1" 2024-04-07T16:13:48.937 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:48 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:48.945655319Z level=info msg="Executing migration" id="create index IDX_org_user_user_id - v1" 2024-04-07T16:13:49.197 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:13:49 smithi144 bash[23522]: audit 2024-04-07T16:13:48.081301+0000 mon.a (mon.0) 801 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:13:49.197 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:48 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:48.987479297Z level=info msg="Executing migration" id="Update org table charset" 2024-04-07T16:13:49.197 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:49 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:49.02948435Z level=info msg="Executing migration" id="Update org_user table charset" 2024-04-07T16:13:49.197 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:49 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:49.071564108Z level=info msg="Executing migration" id="Migrate all Read Only Viewers to Viewers" 2024-04-07T16:13:49.197 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:49 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:49.1137037Z level=info msg="Executing migration" id="create dashboard table" 2024-04-07T16:13:49.197 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:49 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:49.138962578Z level=info msg="Executing migration" id="add index dashboard.account_id" 2024-04-07T16:13:49.197 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:49 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:49.180849239Z level=info msg="Executing migration" id="add unique index dashboard_account_id_slug" 2024-04-07T16:13:49.197 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:49 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:49.206245155Z level=info msg="Executing migration" id="create dashboard_tag table" 2024-04-07T16:13:49.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:13:49 smithi100 bash[21346]: audit 2024-04-07T16:13:48.081301+0000 mon.a (mon.0) 801 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:13:49.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:13:49 smithi100 bash[31660]: audit 2024-04-07T16:13:48.081301+0000 mon.a (mon.0) 801 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:13:49.466 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:49 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:49.231381512Z level=info msg="Executing migration" id="add unique index dashboard_tag.dasboard_id_term" 2024-04-07T16:13:49.466 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:49 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:49.256754369Z level=info msg="Executing migration" id="drop index UQE_dashboard_tag_dashboard_id_term - v1" 2024-04-07T16:13:49.466 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:49 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:49.29867511Z level=info msg="Executing migration" id="Rename table dashboard to dashboard_v1 - v1" 2024-04-07T16:13:49.466 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:49 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:49.340666953Z level=info msg="Executing migration" id="create dashboard v2" 2024-04-07T16:13:49.466 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:49 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:49.382563743Z level=info msg="Executing migration" id="create index IDX_dashboard_org_id - v2" 2024-04-07T16:13:49.466 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:49 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:49.424717494Z level=info msg="Executing migration" id="create index UQE_dashboard_org_id_slug - v2" 2024-04-07T16:13:49.467 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:49 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:49.450021866Z level=info msg="Executing migration" id="copy dashboard v1 to v2" 2024-04-07T16:13:49.467 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:49 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:49.47532801Z level=info msg="Executing migration" id="drop table dashboard_v1" 2024-04-07T16:13:49.752 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:49 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:49.500621645Z level=info msg="Executing migration" id="alter dashboard.data to mediumtext v1" 2024-04-07T16:13:49.752 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:49 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:49.542571578Z level=info msg="Executing migration" id="Add column updated_by in dashboard - v2" 2024-04-07T16:13:49.753 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:49 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:49.567818767Z level=info msg="Executing migration" id="Add column created_by in dashboard - v2" 2024-04-07T16:13:49.753 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:49 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:49.618478915Z level=info msg="Executing migration" id="Add column gnetId in dashboard" 2024-04-07T16:13:49.753 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:49 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:49.643708037Z level=info msg="Executing migration" id="Add index for gnetId in dashboard" 2024-04-07T16:13:49.753 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:49 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:49.668846389Z level=info msg="Executing migration" id="Add column plugin_id in dashboard" 2024-04-07T16:13:49.753 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:49 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:49.694203695Z level=info msg="Executing migration" id="Add index for plugin_id in dashboard" 2024-04-07T16:13:49.753 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:49 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:49.719357441Z level=info msg="Executing migration" id="Add index for dashboard_id in dashboard_tag" 2024-04-07T16:13:49.753 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:49 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:49.761517595Z level=info msg="Executing migration" id="Update dashboard table charset" 2024-04-07T16:13:50.021 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:49 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:49.803471095Z level=info msg="Executing migration" id="Update dashboard_tag table charset" 2024-04-07T16:13:50.022 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:49 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:49.845495964Z level=info msg="Executing migration" id="Add column folder_id in dashboard" 2024-04-07T16:13:50.022 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:49 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:49.870845998Z level=info msg="Executing migration" id="Add column isFolder in dashboard" 2024-04-07T16:13:50.022 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:49 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:49.895985884Z level=info msg="Executing migration" id="Add column has_acl in dashboard" 2024-04-07T16:13:50.022 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:49 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:49.921228513Z level=info msg="Executing migration" id="Add column uid in dashboard" 2024-04-07T16:13:50.022 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:49 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:49.963156606Z level=info msg="Executing migration" id="Update uid column values in dashboard" 2024-04-07T16:13:50.022 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:49 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:49.988471124Z level=info msg="Executing migration" id="Add unique index dashboard_org_id_uid" 2024-04-07T16:13:50.022 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:50 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:50.030418304Z level=info msg="Executing migration" id="Remove unique index org_id_slug" 2024-04-07T16:13:50.282 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:13:50 smithi144 bash[23522]: cluster 2024-04-07T16:13:48.622482+0000 mgr.y (mgr.14553) 53 : cluster 0 pgmap v35: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:13:50.282 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:50 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:50.055656919Z level=info msg="Executing migration" id="Update dashboard title length" 2024-04-07T16:13:50.282 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:50 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:50.080850969Z level=info msg="Executing migration" id="Add unique index for dashboard_org_id_title_folder_id" 2024-04-07T16:13:50.282 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:50 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:50.123006927Z level=info msg="Executing migration" id="create dashboard_provisioning" 2024-04-07T16:13:50.282 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:50 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:50.148258599Z level=info msg="Executing migration" id="Rename table dashboard_provisioning to dashboard_provisioning_tmp_qwerty - v1" 2024-04-07T16:13:50.282 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:50 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:50.173504381Z level=info msg="Executing migration" id="create dashboard_provisioning v2" 2024-04-07T16:13:50.283 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:50 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:50.198771606Z level=info msg="Executing migration" id="create index IDX_dashboard_provisioning_dashboard_id - v2" 2024-04-07T16:13:50.283 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:50 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:50.224023285Z level=info msg="Executing migration" id="create index IDX_dashboard_provisioning_dashboard_id_name - v2" 2024-04-07T16:13:50.283 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:50 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:50.249259267Z level=info msg="Executing migration" id="copy dashboard_provisioning v1 to v2" 2024-04-07T16:13:50.283 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:50 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:50.291184691Z level=info msg="Executing migration" id="drop dashboard_provisioning_tmp_qwerty" 2024-04-07T16:13:50.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:13:50 smithi100 bash[21346]: cluster 2024-04-07T16:13:48.622482+0000 mgr.y (mgr.14553) 53 : cluster 0 pgmap v35: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:13:50.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:13:50 smithi100 bash[31660]: cluster 2024-04-07T16:13:48.622482+0000 mgr.y (mgr.14553) 53 : cluster 0 pgmap v35: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:13:50.543 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:50 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:50.316488239Z level=info msg="Executing migration" id="Add check_sum column" 2024-04-07T16:13:50.543 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:50 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:50.358435196Z level=info msg="Executing migration" id="Add index for dashboard_title" 2024-04-07T16:13:50.543 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:50 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:50.383639305Z level=info msg="Executing migration" id="delete tags for deleted dashboards" 2024-04-07T16:13:50.543 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:50 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:50.408805881Z level=info msg="Executing migration" id="delete stars for deleted dashboards" 2024-04-07T16:13:50.543 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:50 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:50.434070336Z level=info msg="Executing migration" id="Add index for dashboard_is_folder" 2024-04-07T16:13:50.543 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:50 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:50.467722561Z level=info msg="Executing migration" id="Add isPublic for dashboard" 2024-04-07T16:13:50.543 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:50 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:50.492948999Z level=info msg="Executing migration" id="create data_source table" 2024-04-07T16:13:50.543 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:50 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:50.518203017Z level=info msg="Executing migration" id="add index data_source.account_id" 2024-04-07T16:13:50.543 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:50 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:50.552060137Z level=info msg="Executing migration" id="add unique index data_source.account_id_name" 2024-04-07T16:13:50.804 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:50 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:50.577295872Z level=info msg="Executing migration" id="drop index IDX_data_source_account_id - v1" 2024-04-07T16:13:50.804 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:50 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:50.602492708Z level=info msg="Executing migration" id="drop index UQE_data_source_account_id_name - v1" 2024-04-07T16:13:50.804 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:50 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:50.644462538Z level=info msg="Executing migration" id="Rename table data_source to data_source_v1 - v1" 2024-04-07T16:13:50.804 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:50 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:50.669787239Z level=info msg="Executing migration" id="create data_source table v2" 2024-04-07T16:13:50.804 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:50 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:50.695029624Z level=info msg="Executing migration" id="create index IDX_data_source_org_id - v2" 2024-04-07T16:13:50.804 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:50 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:50.711900947Z level=info msg="Executing migration" id="create index UQE_data_source_org_id_name - v2" 2024-04-07T16:13:50.804 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:50 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:50.728807541Z level=info msg="Executing migration" id="Drop old table data_source_v1 #2" 2024-04-07T16:13:50.805 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:50 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:50.74570729Z level=info msg="Executing migration" id="Add column with_credentials" 2024-04-07T16:13:50.805 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:50 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:50.76256864Z level=info msg="Executing migration" id="Add secure json data column" 2024-04-07T16:13:50.805 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:50 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:50.796152901Z level=info msg="Executing migration" id="Update data_source table charset" 2024-04-07T16:13:50.805 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:50 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:50.813156923Z level=info msg="Executing migration" id="Update initial version to 1" 2024-04-07T16:13:51.064 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:50 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:50.846886696Z level=info msg="Executing migration" id="Add read_only data column" 2024-04-07T16:13:51.065 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:50 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:50.888787093Z level=info msg="Executing migration" id="Migrate logging ds to loki ds" 2024-04-07T16:13:51.065 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:50 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:50.914092468Z level=info msg="Executing migration" id="Update json_data with nulls" 2024-04-07T16:13:51.065 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:50 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:50.956071692Z level=info msg="Executing migration" id="Add uid column" 2024-04-07T16:13:51.065 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:50 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:50.981321313Z level=info msg="Executing migration" id="Update uid value" 2024-04-07T16:13:51.065 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:51 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:51.023273878Z level=info msg="Executing migration" id="Add unique index datasource_org_id_uid" 2024-04-07T16:13:51.065 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:51 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:51.048554052Z level=info msg="Executing migration" id="add unique index datasource_org_id_is_default" 2024-04-07T16:13:51.065 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:51 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:51.073796918Z level=info msg="Executing migration" id="create api_key table" 2024-04-07T16:13:51.326 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:51 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:51.115955867Z level=info msg="Executing migration" id="add index api_key.account_id" 2024-04-07T16:13:51.326 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:51 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:51.149555765Z level=info msg="Executing migration" id="add index api_key.key" 2024-04-07T16:13:51.326 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:51 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:51.166432053Z level=info msg="Executing migration" id="add index api_key.account_id_name" 2024-04-07T16:13:51.326 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:51 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:51.208514911Z level=info msg="Executing migration" id="drop index IDX_api_key_account_id - v1" 2024-04-07T16:13:51.327 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:51 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:51.22538486Z level=info msg="Executing migration" id="drop index UQE_api_key_key - v1" 2024-04-07T16:13:51.327 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:51 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:51.259228843Z level=info msg="Executing migration" id="drop index UQE_api_key_account_id_name - v1" 2024-04-07T16:13:51.327 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:51 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:51.276278828Z level=info msg="Executing migration" id="Rename table api_key to api_key_v1 - v1" 2024-04-07T16:13:51.327 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:51 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:51.293126645Z level=info msg="Executing migration" id="create api_key table v2" 2024-04-07T16:13:51.327 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:51 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:51.335444421Z level=info msg="Executing migration" id="create index IDX_api_key_org_id - v2" 2024-04-07T16:13:51.569 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/mon.a/config 2024-04-07T16:13:51.579 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:51 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:51.360639034Z level=info msg="Executing migration" id="create index UQE_api_key_key - v2" 2024-04-07T16:13:51.579 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:51 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:51.385999999Z level=info msg="Executing migration" id="create index UQE_api_key_org_id_name - v2" 2024-04-07T16:13:51.579 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:51 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:51.411270137Z level=info msg="Executing migration" id="copy api_key v1 to v2" 2024-04-07T16:13:51.579 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:51 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:51.436455924Z level=info msg="Executing migration" id="Drop old table api_key_v1" 2024-04-07T16:13:51.579 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:51 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:51.461737385Z level=info msg="Executing migration" id="Update api_key table charset" 2024-04-07T16:13:51.579 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:51 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:51.486921864Z level=info msg="Executing migration" id="Add expires to api_key table" 2024-04-07T16:13:51.579 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:51 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:51.512310069Z level=info msg="Executing migration" id="Add service account foreign key" 2024-04-07T16:13:51.579 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:51 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:51.537456919Z level=info msg="Executing migration" id="set service account foreign key to nil if 0" 2024-04-07T16:13:51.579 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:51 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:51.562795277Z level=info msg="Executing migration" id="Add last_used_at to api_key table" 2024-04-07T16:13:51.579 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:51 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:51.588075866Z level=info msg="Executing migration" id="Add is_revoked column to api_key table" 2024-04-07T16:13:51.831 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:51 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:51.630132872Z level=info msg="Executing migration" id="create dashboard_snapshot table v4" 2024-04-07T16:13:51.831 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:51 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:51.655398963Z level=info msg="Executing migration" id="drop table dashboard_snapshot_v4 #1" 2024-04-07T16:13:51.831 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:51 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:51.680801875Z level=info msg="Executing migration" id="create dashboard_snapshot table v5 #2" 2024-04-07T16:13:51.831 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:51 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:51.705775023Z level=info msg="Executing migration" id="create index UQE_dashboard_snapshot_key - v5" 2024-04-07T16:13:51.831 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:51 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:51.747747589Z level=info msg="Executing migration" id="create index UQE_dashboard_snapshot_delete_key - v5" 2024-04-07T16:13:51.831 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:51 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:51.789891824Z level=info msg="Executing migration" id="create index IDX_dashboard_snapshot_user_id - v5" 2024-04-07T16:13:51.831 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:51 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:51.815304665Z level=info msg="Executing migration" id="alter dashboard_snapshot to mediumtext v2" 2024-04-07T16:13:52.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:13:52 smithi144 bash[23522]: cluster 2024-04-07T16:13:50.623798+0000 mgr.y (mgr.14553) 54 : cluster 0 pgmap v36: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:13:52.107 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:51 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:51.840596218Z level=info msg="Executing migration" id="Update dashboard_snapshot table charset" 2024-04-07T16:13:52.107 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:51 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:51.924355185Z level=info msg="Executing migration" id="Add column external_delete_url to dashboard_snapshots table" 2024-04-07T16:13:52.107 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:51 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:51.97458429Z level=info msg="Executing migration" id="Add encrypted dashboard json column" 2024-04-07T16:13:52.107 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:52 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:51.99989902Z level=info msg="Executing migration" id="Change dashboard_encrypted column to MEDIUMBLOB" 2024-04-07T16:13:52.107 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:52 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:52.041870228Z level=info msg="Executing migration" id="create quota table v1" 2024-04-07T16:13:52.107 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:52 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:52.083922883Z level=info msg="Executing migration" id="create index UQE_quota_org_id_user_id_target - v1" 2024-04-07T16:13:52.437 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:52 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:52.177597603Z level=info msg="Executing migration" id="Update quota table charset" 2024-04-07T16:13:52.437 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:52 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:52.218754738Z level=info msg="Executing migration" id="create plugin_setting table" 2024-04-07T16:13:52.437 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:52 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:52.244150219Z level=info msg="Executing migration" id="create index UQE_plugin_setting_org_id_plugin_id - v1" 2024-04-07T16:13:52.437 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:52 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:52.286170867Z level=info msg="Executing migration" id="Add column plugin_version to plugin_settings" 2024-04-07T16:13:52.437 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:52 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:52.328467567Z level=info msg="Executing migration" id="Update plugin_setting table charset" 2024-04-07T16:13:52.437 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:52 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:52.370447061Z level=info msg="Executing migration" id="create session table" 2024-04-07T16:13:52.437 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:52 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:52.395739723Z level=info msg="Executing migration" id="Drop old table playlist table" 2024-04-07T16:13:52.437 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:52 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:52.420991989Z level=info msg="Executing migration" id="Drop old table playlist_item table" 2024-04-07T16:13:52.438 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:52 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:52.446269382Z level=info msg="Executing migration" id="create playlist table v2" 2024-04-07T16:13:52.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:13:52 smithi100 bash[21346]: cluster 2024-04-07T16:13:50.623798+0000 mgr.y (mgr.14553) 54 : cluster 0 pgmap v36: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:13:52.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:13:52 smithi100 bash[31660]: cluster 2024-04-07T16:13:50.623798+0000 mgr.y (mgr.14553) 54 : cluster 0 pgmap v36: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:13:52.715 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:52 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:52.471681721Z level=info msg="Executing migration" id="create playlist item table v2" 2024-04-07T16:13:52.715 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:52 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:52.49695765Z level=info msg="Executing migration" id="Update playlist table charset" 2024-04-07T16:13:52.715 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:52 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:52.522221591Z level=info msg="Executing migration" id="Update playlist_item table charset" 2024-04-07T16:13:52.715 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:52 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:52.547361682Z level=info msg="Executing migration" id="drop preferences table v2" 2024-04-07T16:13:52.715 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:52 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:52.589374118Z level=info msg="Executing migration" id="drop preferences table v3" 2024-04-07T16:13:52.715 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:52 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:52.614751561Z level=info msg="Executing migration" id="create preferences table v3" 2024-04-07T16:13:52.715 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:52 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:52.65663518Z level=info msg="Executing migration" id="Update preferences table charset" 2024-04-07T16:13:52.715 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:52 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:52.681899167Z level=info msg="Executing migration" id="Add column team_id in preferences" 2024-04-07T16:13:52.715 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:52 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:52.724002534Z level=info msg="Executing migration" id="Update team_id column values in preferences" 2024-04-07T16:13:52.967 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:52 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:52.765991782Z level=info msg="Executing migration" id="Add column week_start in preferences" 2024-04-07T16:13:52.967 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:52 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:52.791391139Z level=info msg="Executing migration" id="Add column preferences.json_data" 2024-04-07T16:13:52.968 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:52 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:52.816637822Z level=info msg="Executing migration" id="alter preferences.json_data to mediumtext v1" 2024-04-07T16:13:52.968 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:52 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:52.841846797Z level=info msg="Executing migration" id="Add preferences index org_id" 2024-04-07T16:13:52.968 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:52 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:52.883737726Z level=info msg="Executing migration" id="Add preferences index user_id" 2024-04-07T16:13:52.968 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:52 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:52.909204173Z level=info msg="Executing migration" id="create alert table v1" 2024-04-07T16:13:52.968 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:52 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:52.951284163Z level=info msg="Executing migration" id="add index alert org_id & id " 2024-04-07T16:13:52.968 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:52 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:52.97641554Z level=info msg="Executing migration" id="add index alert state" 2024-04-07T16:13:53.228 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:53 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:53.001516134Z level=info msg="Executing migration" id="add index alert dashboard_id" 2024-04-07T16:13:53.228 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:53 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:53.026894458Z level=info msg="Executing migration" id="Create alert_rule_tag table v1" 2024-04-07T16:13:53.229 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:53 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:53.077553858Z level=info msg="Executing migration" id="Add unique index alert_rule_tag.alert_id_tag_id" 2024-04-07T16:13:53.229 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:53 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:53.102860288Z level=info msg="Executing migration" id="drop index UQE_alert_rule_tag_alert_id_tag_id - v1" 2024-04-07T16:13:53.229 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:53 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:53.144917654Z level=info msg="Executing migration" id="Rename table alert_rule_tag to alert_rule_tag_v1 - v1" 2024-04-07T16:13:53.229 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:53 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:53.19542946Z level=info msg="Executing migration" id="Create alert_rule_tag table v2" 2024-04-07T16:13:53.229 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:53 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:53.237391694Z level=info msg="Executing migration" id="create index UQE_alert_rule_tag_alert_id_tag_id - Add unique index alert_rule_tag.alert_id_tag_id V2" 2024-04-07T16:13:53.229 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:13:53 smithi144 bash[23522]: audit 2024-04-07T16:13:52.114627+0000 mon.a (mon.0) 802 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:13:53.229 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:13:53 smithi144 bash[23522]: audit 2024-04-07T16:13:52.121341+0000 mon.a (mon.0) 803 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:13:53.229 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:13:53 smithi144 bash[23522]: audit 2024-04-07T16:13:52.569815+0000 mon.a (mon.0) 804 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:13:53.229 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:13:53 smithi144 bash[23522]: audit 2024-04-07T16:13:52.573687+0000 mon.b (mon.1) 70 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:13:53.229 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:13:53 smithi144 bash[23522]: audit 2024-04-07T16:13:52.574539+0000 mon.b (mon.1) 71 : audit 1 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:13:53.229 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:13:53 smithi144 bash[23522]: audit 2024-04-07T16:13:52.576877+0000 mon.a (mon.0) 805 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:13:53.230 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:13:53 smithi144 bash[23522]: audit 2024-04-07T16:13:52.584546+0000 mon.a (mon.0) 806 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:13:53.230 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:13:53 smithi144 bash[23522]: audit 2024-04-07T16:13:52.704704+0000 mon.b (mon.1) 72 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:13:53.230 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:13:53 smithi144 bash[23522]: audit 2024-04-07T16:13:52.706657+0000 mon.a (mon.0) 807 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:13:53.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:13:53 smithi100 bash[21346]: audit 2024-04-07T16:13:52.114627+0000 mon.a (mon.0) 802 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:13:53.439 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:13:53 smithi100 bash[21346]: audit 2024-04-07T16:13:52.121341+0000 mon.a (mon.0) 803 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:13:53.439 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:13:53 smithi100 bash[21346]: audit 2024-04-07T16:13:52.569815+0000 mon.a (mon.0) 804 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:13:53.439 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:13:53 smithi100 bash[21346]: audit 2024-04-07T16:13:52.573687+0000 mon.b (mon.1) 70 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:13:53.439 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:13:53 smithi100 bash[21346]: audit 2024-04-07T16:13:52.574539+0000 mon.b (mon.1) 71 : audit 1 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:13:53.439 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:13:53 smithi100 bash[21346]: audit 2024-04-07T16:13:52.576877+0000 mon.a (mon.0) 805 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:13:53.439 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:13:53 smithi100 bash[21346]: audit 2024-04-07T16:13:52.584546+0000 mon.a (mon.0) 806 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:13:53.439 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:13:53 smithi100 bash[21346]: audit 2024-04-07T16:13:52.704704+0000 mon.b (mon.1) 72 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:13:53.439 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:13:53 smithi100 bash[21346]: audit 2024-04-07T16:13:52.706657+0000 mon.a (mon.0) 807 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:13:53.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:13:53 smithi100 bash[31660]: audit 2024-04-07T16:13:52.114627+0000 mon.a (mon.0) 802 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:13:53.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:13:53 smithi100 bash[31660]: audit 2024-04-07T16:13:52.121341+0000 mon.a (mon.0) 803 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:13:53.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:13:53 smithi100 bash[31660]: audit 2024-04-07T16:13:52.569815+0000 mon.a (mon.0) 804 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:13:53.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:13:53 smithi100 bash[31660]: audit 2024-04-07T16:13:52.573687+0000 mon.b (mon.1) 70 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:13:53.440 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:13:53 smithi100 bash[31660]: audit 2024-04-07T16:13:52.574539+0000 mon.b (mon.1) 71 : audit 1 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:13:53.440 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:13:53 smithi100 bash[31660]: audit 2024-04-07T16:13:52.576877+0000 mon.a (mon.0) 805 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:13:53.440 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:13:53 smithi100 bash[31660]: audit 2024-04-07T16:13:52.584546+0000 mon.a (mon.0) 806 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:13:53.440 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:13:53 smithi100 bash[31660]: audit 2024-04-07T16:13:52.704704+0000 mon.b (mon.1) 72 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:13:53.440 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:13:53 smithi100 bash[31660]: audit 2024-04-07T16:13:52.706657+0000 mon.a (mon.0) 807 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:13:53.490 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:53 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:53.262739619Z level=info msg="Executing migration" id="copy alert_rule_tag v1 to v2" 2024-04-07T16:13:53.490 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:53 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:53.287935109Z level=info msg="Executing migration" id="drop table alert_rule_tag_v1" 2024-04-07T16:13:53.490 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:53 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:53.304758779Z level=info msg="Executing migration" id="create alert_notification table v1" 2024-04-07T16:13:53.490 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:53 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:53.33850448Z level=info msg="Executing migration" id="Add column is_default" 2024-04-07T16:13:53.490 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:53 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:53.355485429Z level=info msg="Executing migration" id="Add column frequency" 2024-04-07T16:13:53.490 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:53 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:53.372467882Z level=info msg="Executing migration" id="Add column send_reminder" 2024-04-07T16:13:53.491 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:53 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:53.389359935Z level=info msg="Executing migration" id="Add column disable_resolve_message" 2024-04-07T16:13:53.491 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:53 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:53.406283161Z level=info msg="Executing migration" id="add index alert_notification org_id & name" 2024-04-07T16:13:53.491 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:53 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:53.44818783Z level=info msg="Executing migration" id="Update alert table charset" 2024-04-07T16:13:53.491 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:53 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:53.473622837Z level=info msg="Executing migration" id="Update alert_notification table charset" 2024-04-07T16:13:53.491 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:53 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:53.498896529Z level=info msg="Executing migration" id="create notification_journal table v1" 2024-04-07T16:13:53.767 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:53 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:53.524168718Z level=info msg="Executing migration" id="add index notification_journal org_id & alert_id & notifier_id" 2024-04-07T16:13:53.768 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:53 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:53.549467951Z level=info msg="Executing migration" id="drop alert_notification_journal" 2024-04-07T16:13:53.768 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:53 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:53.574705573Z level=info msg="Executing migration" id="create alert_notification_state table v1" 2024-04-07T16:13:53.768 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:53 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:53.616724557Z level=info msg="Executing migration" id="add index alert_notification_state org_id & alert_id & notifier_id" 2024-04-07T16:13:53.768 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:53 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:53.641876574Z level=info msg="Executing migration" id="Add for to alert table" 2024-04-07T16:13:53.768 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:53 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:53.683906568Z level=info msg="Executing migration" id="Add column uid in alert_notification" 2024-04-07T16:13:53.768 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:53 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:53.709261423Z level=info msg="Executing migration" id="Update uid column values in alert_notification" 2024-04-07T16:13:53.768 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:53 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:53.734558539Z level=info msg="Executing migration" id="Add unique index alert_notification_org_id_uid" 2024-04-07T16:13:53.768 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:53 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:53.776624858Z level=info msg="Executing migration" id="Remove unique index org_id_name" 2024-04-07T16:13:54.020 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:53 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:53.801888924Z level=info msg="Executing migration" id="Add column secure_settings in alert_notification" 2024-04-07T16:13:54.020 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:53 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:53.827328842Z level=info msg="Executing migration" id="alter alert.settings to mediumtext" 2024-04-07T16:13:54.020 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:53 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:53.852651257Z level=info msg="Executing migration" id="Add non-unique index alert_notification_state_alert_id" 2024-04-07T16:13:54.020 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:53 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:53.894638568Z level=info msg="Executing migration" id="Add non-unique index alert_rule_tag_alert_id" 2024-04-07T16:13:54.020 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:53 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:53.919935784Z level=info msg="Executing migration" id="Drop old annotation table v4" 2024-04-07T16:13:54.020 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:53 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:53.961941311Z level=info msg="Executing migration" id="create annotation table v5" 2024-04-07T16:13:54.020 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:53 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:53.98722145Z level=info msg="Executing migration" id="add index annotation 0 v3" 2024-04-07T16:13:54.020 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:54 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:54.029217548Z level=info msg="Executing migration" id="add index annotation 1 v3" 2024-04-07T16:13:54.289 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:54 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:54.071332837Z level=info msg="Executing migration" id="add index annotation 2 v3" 2024-04-07T16:13:54.290 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:54 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:54.113262193Z level=info msg="Executing migration" id="add index annotation 3 v3" 2024-04-07T16:13:54.290 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:54 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:54.155483642Z level=info msg="Executing migration" id="add index annotation 4 v3" 2024-04-07T16:13:54.290 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:54 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:54.180759211Z level=info msg="Executing migration" id="Update annotation table charset" 2024-04-07T16:13:54.290 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:54 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:54.205890765Z level=info msg="Executing migration" id="Add column region_id to annotation table" 2024-04-07T16:13:54.290 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:54 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:54.247992345Z level=info msg="Executing migration" id="Drop category_id index" 2024-04-07T16:13:54.290 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:54 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:54.273331027Z level=info msg="Executing migration" id="Add column tags to annotation table" 2024-04-07T16:13:54.290 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:54 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:54.298609707Z level=info msg="Executing migration" id="Create annotation_tag table v2" 2024-04-07T16:13:54.559 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:54 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:54.340759053Z level=info msg="Executing migration" id="Add unique index annotation_tag.annotation_id_tag_id" 2024-04-07T16:13:54.559 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:54 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:54.365869724Z level=info msg="Executing migration" id="drop index UQE_annotation_tag_annotation_id_tag_id - v2" 2024-04-07T16:13:54.559 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:54 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:54.399520812Z level=info msg="Executing migration" id="Rename table annotation_tag to annotation_tag_v2 - v2" 2024-04-07T16:13:54.560 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:54 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:54.424735879Z level=info msg="Executing migration" id="Create annotation_tag table v3" 2024-04-07T16:13:54.560 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:54 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:54.441772384Z level=info msg="Executing migration" id="create index UQE_annotation_tag_annotation_id_tag_id - Add unique index annotation_tag.annotation_id_tag_id V3" 2024-04-07T16:13:54.560 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:54 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:54.458673424Z level=info msg="Executing migration" id="copy annotation_tag v2 to v3" 2024-04-07T16:13:54.560 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:54 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:54.475548587Z level=info msg="Executing migration" id="drop table annotation_tag_v2" 2024-04-07T16:13:54.560 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:54 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:54.509287845Z level=info msg="Executing migration" id="Update alert annotations and set TEXT to empty" 2024-04-07T16:13:54.560 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:54 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:54.526177085Z level=info msg="Executing migration" id="Add created time to annotation table" 2024-04-07T16:13:54.560 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:54 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:54.543135695Z level=info msg="Executing migration" id="Add updated time to annotation table" 2024-04-07T16:13:54.560 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:54 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:54.56849978Z level=info msg="Executing migration" id="Add index for created in annotation table" 2024-04-07T16:13:54.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:13:54 smithi100 bash[21346]: cephadm 2024-04-07T16:13:52.621147+0000 mgr.y (mgr.14553) 55 : cephadm 1 Reconfiguring alertmanager.a (dependencies changed)... 2024-04-07T16:13:54.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:13:54 smithi100 bash[21346]: cluster 2024-04-07T16:13:52.624192+0000 mgr.y (mgr.14553) 56 : cluster 0 pgmap v37: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:13:54.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:13:54 smithi100 bash[21346]: cephadm 2024-04-07T16:13:52.625067+0000 mgr.y (mgr.14553) 57 : cephadm 1 Reconfiguring daemon alertmanager.a on smithi100 2024-04-07T16:13:54.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:13:54 smithi100 bash[21346]: cluster 2024-04-07T16:13:54.134107+0000 mon.a (mon.0) 808 : cluster 0 mgrmap e21: y(active, since 61s), standbys: x 2024-04-07T16:13:54.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:13:54 smithi100 bash[31660]: cephadm 2024-04-07T16:13:52.621147+0000 mgr.y (mgr.14553) 55 : cephadm 1 Reconfiguring alertmanager.a (dependencies changed)... 2024-04-07T16:13:54.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:13:54 smithi100 bash[31660]: cluster 2024-04-07T16:13:52.624192+0000 mgr.y (mgr.14553) 56 : cluster 0 pgmap v37: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:13:54.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:13:54 smithi100 bash[31660]: cephadm 2024-04-07T16:13:52.625067+0000 mgr.y (mgr.14553) 57 : cephadm 1 Reconfiguring daemon alertmanager.a on smithi100 2024-04-07T16:13:54.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:13:54 smithi100 bash[31660]: cluster 2024-04-07T16:13:54.134107+0000 mon.a (mon.0) 808 : cluster 0 mgrmap e21: y(active, since 61s), standbys: x 2024-04-07T16:13:54.812 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:54 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:54.610965309Z level=info msg="Executing migration" id="Add index for updated in annotation table" 2024-04-07T16:13:54.812 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:54 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:54.652767166Z level=info msg="Executing migration" id="Convert existing annotations from seconds to milliseconds" 2024-04-07T16:13:54.812 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:54 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:54.678030169Z level=info msg="Executing migration" id="Add epoch_end column" 2024-04-07T16:13:54.812 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:54 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:54.703375337Z level=info msg="Executing migration" id="Add index for epoch_end" 2024-04-07T16:13:54.812 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:54 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:54.728570514Z level=info msg="Executing migration" id="Make epoch_end the same as epoch" 2024-04-07T16:13:54.812 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:54 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:54.753860083Z level=info msg="Executing migration" id="Move region to single row" 2024-04-07T16:13:54.812 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:54 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:54.779083946Z level=info msg="Executing migration" id="Remove index org_id_epoch from annotation table" 2024-04-07T16:13:54.813 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:54 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:54.82115572Z level=info msg="Executing migration" id="Remove index org_id_dashboard_id_panel_id_epoch from annotation table" 2024-04-07T16:13:54.813 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:13:54 smithi144 bash[23522]: cephadm 2024-04-07T16:13:52.621147+0000 mgr.y (mgr.14553) 55 : cephadm 1 Reconfiguring alertmanager.a (dependencies changed)... 2024-04-07T16:13:54.813 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:13:54 smithi144 bash[23522]: cluster 2024-04-07T16:13:52.624192+0000 mgr.y (mgr.14553) 56 : cluster 0 pgmap v37: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:13:54.813 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:13:54 smithi144 bash[23522]: cephadm 2024-04-07T16:13:52.625067+0000 mgr.y (mgr.14553) 57 : cephadm 1 Reconfiguring daemon alertmanager.a on smithi100 2024-04-07T16:13:54.813 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:13:54 smithi144 bash[23522]: cluster 2024-04-07T16:13:54.134107+0000 mon.a (mon.0) 808 : cluster 0 mgrmap e21: y(active, since 61s), standbys: x 2024-04-07T16:13:55.020 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-07T16:13:55.021 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":69,"fsid":"0702e3a8-f4f8-11ee-b649-cb9ed24678a4","created":"2024-04-07T16:02:21.629692+0000","modified":"2024-04-07T16:13:38.014991+0000","last_up_change":"2024-04-07T16:11:30.210001+0000","last_in_change":"2024-04-07T16:11:04.793434+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-07T16:07:26.784071+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":7.8899998664855957,"score_stable":7.8899998664855957,"optimal_score":0.37999999523162842,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-07T16:12:11.856505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-07T16:12:12.737709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"rbd","create_time":"2024-04-07T16:13:28.913394+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":3,"score_stable":3,"optimal_score":1,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8facd48e-0a0c-4fda-aee2-f001b60a3811","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":1636950303},{"type":"v1","addr":"172.21.15.100:6803","nonce":1636950303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":1636950303},{"type":"v1","addr":"172.21.15.100:6805","nonce":1636950303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1636950303},{"type":"v1","addr":"172.21.15.100:6809","nonce":1636950303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":1636950303},{"type":"v1","addr":"172.21.15.100:6807","nonce":1636950303}]},"public_addr":"172.21.15.100:6803/1636950303","cluster_addr":"172.21.15.100:6805/1636950303","heartbeat_back_addr":"172.21.15.100:6809/1636950303","heartbeat_front_addr":"172.21.15.100:6807/1636950303","state":["exists","up"]},{"osd":1,"uuid":"cb5f6c23-6764-48b6-8b21-97a36b004786","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":470179506},{"type":"v1","addr":"172.21.15.100:6811","nonce":470179506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":470179506},{"type":"v1","addr":"172.21.15.100:6813","nonce":470179506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":470179506},{"type":"v1","addr":"172.21.15.100:6817","nonce":470179506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":470179506},{"type":"v1","addr":"172.21.15.100:6815","nonce":470179506}]},"public_addr":"172.21.15.100:6811/470179506","cluster_addr":"172.21.15.100:6813/470179506","heartbeat_back_addr":"172.21.15.100:6817/470179506","heartbeat_front_addr":"172.21.15.100:6815/470179506","state":["exists","up"]},{"osd":2,"uuid":"1650aedb-9ce9-4c14-a059-e1f089e5e63f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":2235490011},{"type":"v1","addr":"172.21.15.100:6819","nonce":2235490011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2235490011},{"type":"v1","addr":"172.21.15.100:6821","nonce":2235490011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":2235490011},{"type":"v1","addr":"172.21.15.100:6825","nonce":2235490011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":2235490011},{"type":"v1","addr":"172.21.15.100:6823","nonce":2235490011}]},"public_addr":"172.21.15.100:6819/2235490011","cluster_addr":"172.21.15.100:6821/2235490011","heartbeat_back_addr":"172.21.15.100:6825/2235490011","heartbeat_front_addr":"172.21.15.100:6823/2235490011","state":["exists","up"]},{"osd":3,"uuid":"17591c6b-862b-43b2-858b-02f4c3199247","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":3019212393},{"type":"v1","addr":"172.21.15.100:6827","nonce":3019212393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":3019212393},{"type":"v1","addr":"172.21.15.100:6829","nonce":3019212393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":3019212393},{"type":"v1","addr":"172.21.15.100:6833","nonce":3019212393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":3019212393},{"type":"v1","addr":"172.21.15.100:6831","nonce":3019212393}]},"public_addr":"172.21.15.100:6827/3019212393","cluster_addr":"172.21.15.100:6829/3019212393","heartbeat_back_addr":"172.21.15.100:6833/3019212393","heartbeat_front_addr":"172.21.15.100:6831/3019212393","state":["exists","up"]},{"osd":4,"uuid":"6ecc1205-7c78-461e-8037-640136cb3658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":974906744},{"type":"v1","addr":"172.21.15.144:6801","nonce":974906744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":974906744},{"type":"v1","addr":"172.21.15.144:6803","nonce":974906744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":974906744},{"type":"v1","addr":"172.21.15.144:6807","nonce":974906744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":974906744},{"type":"v1","addr":"172.21.15.144:6805","nonce":974906744}]},"public_addr":"172.21.15.144:6801/974906744","cluster_addr":"172.21.15.144:6803/974906744","heartbeat_back_addr":"172.21.15.144:6807/974906744","heartbeat_front_addr":"172.21.15.144:6805/974906744","state":["exists","up"]},{"osd":5,"uuid":"54789cc8-1c3c-46a4-a92e-8881564338db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":2783162058},{"type":"v1","addr":"172.21.15.144:6809","nonce":2783162058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":2783162058},{"type":"v1","addr":"172.21.15.144:6811","nonce":2783162058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":2783162058},{"type":"v1","addr":"172.21.15.144:6815","nonce":2783162058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":2783162058},{"type":"v1","addr":"172.21.15.144:6813","nonce":2783162058}]},"public_addr":"172.21.15.144:6809/2783162058","cluster_addr":"172.21.15.144:6811/2783162058","heartbeat_back_addr":"172.21.15.144:6815/2783162058","heartbeat_front_addr":"172.21.15.144:6813/2783162058","state":["exists","up"]},{"osd":6,"uuid":"ac619497-7605-4ca6-84bf-3abb3f6f32d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":1183840595},{"type":"v1","addr":"172.21.15.144:6817","nonce":1183840595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":1183840595},{"type":"v1","addr":"172.21.15.144:6819","nonce":1183840595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":1183840595},{"type":"v1","addr":"172.21.15.144:6823","nonce":1183840595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":1183840595},{"type":"v1","addr":"172.21.15.144:6821","nonce":1183840595}]},"public_addr":"172.21.15.144:6817/1183840595","cluster_addr":"172.21.15.144:6819/1183840595","heartbeat_back_addr":"172.21.15.144:6823/1183840595","heartbeat_front_addr":"172.21.15.144:6821/1183840595","state":["exists","up"]},{"osd":7,"uuid":"b85b3422-9da2-4605-88e7-fe9c3b2020cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":2248617613},{"type":"v1","addr":"172.21.15.144:6825","nonce":2248617613}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":2248617613},{"type":"v1","addr":"172.21.15.144:6827","nonce":2248617613}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":2248617613},{"type":"v1","addr":"172.21.15.144:6831","nonce":2248617613}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":2248617613},{"type":"v1","addr":"172.21.15.144:6829","nonce":2248617613}]},"public_addr":"172.21.15.144:6825/2248617613","cluster_addr":"172.21.15.144:6827/2248617613","heartbeat_back_addr":"172.21.15.144:6831/2248617613","heartbeat_front_addr":"172.21.15.144:6829/2248617613","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-07T16:05:44.696671+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-07T16:06:33.787321+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-07T16:07:22.983158+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-07T16:08:12.935606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-07T16:09:00.218619+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-07T16:09:50.222185+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-07T16:10:37.856158+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-07T16:11:28.267877+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.7","mappings":[{"from":7,"to":4}]},{"pgid":"3.1","mappings":[{"from":0,"to":2}]},{"pgid":"3.12","mappings":[{"from":0,"to":2}]},{"pgid":"3.15","mappings":[{"from":4,"to":1}]},{"pgid":"3.17","mappings":[{"from":0,"to":2}]},{"pgid":"3.1d","mappings":[{"from":4,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.100:0/74983397":"2024-04-08T16:12:52.577534+0000","172.21.15.100:6800/190788518":"2024-04-08T16:12:52.577534+0000","172.21.15.100:0/1309290877":"2024-04-08T16:12:52.577534+0000","172.21.15.100:0/1472671368":"2024-04-08T16:03:24.146096+0000","172.21.15.100:6801/190788518":"2024-04-08T16:12:52.577534+0000","172.21.15.100:0/639120693":"2024-04-08T16:03:24.146096+0000","172.21.15.100:0/2474331937":"2024-04-08T16:03:24.146096+0000","172.21.15.100:0/3273267396":"2024-04-08T16:12:52.577534+0000","172.21.15.100:0/2535336108":"2024-04-08T16:12:52.577534+0000","172.21.15.100:6801/2710911757":"2024-04-08T16:03:24.146096+0000","172.21.15.100:6800/3440287045":"2024-04-08T16:02:53.269514+0000","172.21.15.100:0/109071285":"2024-04-08T16:02:53.269514+0000","172.21.15.100:6801/3440287045":"2024-04-08T16:02:53.269514+0000","172.21.15.100:0/4162615562":"2024-04-08T16:02:53.269514+0000","172.21.15.100:0/4121992264":"2024-04-08T16:02:53.269514+0000","172.21.15.100:6800/2710911757":"2024-04-08T16:03:24.146096+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-07T16:13:55.097 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:54 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:54.863217033Z level=info msg="Executing migration" id="Add index for org_id_dashboard_id_epoch_end_epoch on annotation table" 2024-04-07T16:13:55.098 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:54 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:54.905206548Z level=info msg="Executing migration" id="Add index for org_id_epoch_end_epoch on annotation table" 2024-04-07T16:13:55.098 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:54 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:54.972249015Z level=info msg="Executing migration" id="Remove index org_id_epoch_epoch_end from annotation table" 2024-04-07T16:13:55.098 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:54 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:54.997523028Z level=info msg="Executing migration" id="Add index for alert_id on annotation table" 2024-04-07T16:13:55.098 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:55 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:55.039511286Z level=info msg="Executing migration" id="Increase tags column to length 4096" 2024-04-07T16:13:55.098 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:55 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:55.064760042Z level=info msg="Executing migration" id="create test_data table" 2024-04-07T16:13:55.098 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:55 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:55.10675714Z level=info msg="Executing migration" id="create dashboard_version table v1" 2024-04-07T16:13:55.350 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:55 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:55.148807078Z level=info msg="Executing migration" id="add index dashboard_version.dashboard_id" 2024-04-07T16:13:55.350 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:55 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:55.190870956Z level=info msg="Executing migration" id="add unique index dashboard_version.dashboard_id and dashboard_version.version" 2024-04-07T16:13:55.350 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:55 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:55.216080756Z level=info msg="Executing migration" id="Set dashboard version to 1 where 0" 2024-04-07T16:13:55.350 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:55 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:55.258161353Z level=info msg="Executing migration" id="save existing dashboard data in dashboard_version table v1" 2024-04-07T16:13:55.350 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:55 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:55.291838612Z level=info msg="Executing migration" id="alter dashboard_version.data to mediumtext v1" 2024-04-07T16:13:55.350 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:55 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:55.317022742Z level=info msg="Executing migration" id="create team table" 2024-04-07T16:13:55.361 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:13:55 smithi100 bash[21601]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:13:55] "GET /metrics HTTP/1.1" 200 33727 "" "Prometheus/2.43.0" 2024-04-07T16:13:55.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:13:55 smithi144 bash[23522]: audit 2024-04-07T16:13:55.033680+0000 mon.a (mon.0) 809 : audit 0 from='client.? 172.21.15.100:0/784172237' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-07T16:13:55.607 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:55 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:55.359084076Z level=info msg="Executing migration" id="add index team.org_id" 2024-04-07T16:13:55.607 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:55 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:55.384396361Z level=info msg="Executing migration" id="add unique index team_org_id_name" 2024-04-07T16:13:55.607 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:55 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:55.409533493Z level=info msg="Executing migration" id="create team member table" 2024-04-07T16:13:55.607 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:55 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:55.434816979Z level=info msg="Executing migration" id="add index team_member.org_id" 2024-04-07T16:13:55.607 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:55 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:55.460061779Z level=info msg="Executing migration" id="add unique index team_member_org_id_team_id_user_id" 2024-04-07T16:13:55.607 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:55 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:55.485326448Z level=info msg="Executing migration" id="add index team_member.team_id" 2024-04-07T16:13:55.607 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:55 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:55.510545258Z level=info msg="Executing migration" id="Add column email to team table" 2024-04-07T16:13:55.608 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:55 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:55.552708428Z level=info msg="Executing migration" id="Add column external to team_member table" 2024-04-07T16:13:55.608 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:55 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:55.569700154Z level=info msg="Executing migration" id="Add column permission to team_member table" 2024-04-07T16:13:55.608 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:55 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:55.603446501Z level=info msg="Executing migration" id="create dashboard acl table" 2024-04-07T16:13:55.689 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:13:55 smithi100 bash[21346]: audit 2024-04-07T16:13:55.033680+0000 mon.a (mon.0) 809 : audit 0 from='client.? 172.21.15.100:0/784172237' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-07T16:13:55.689 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:13:55 smithi100 bash[31660]: audit 2024-04-07T16:13:55.033680+0000 mon.a (mon.0) 809 : audit 0 from='client.? 172.21.15.100:0/784172237' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-07T16:13:55.889 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:55 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:55.645540125Z level=info msg="Executing migration" id="add index dashboard_acl_dashboard_id" 2024-04-07T16:13:55.889 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:55 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:55.670771932Z level=info msg="Executing migration" id="add unique index dashboard_acl_dashboard_id_user_id" 2024-04-07T16:13:55.889 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:55 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:55.696119254Z level=info msg="Executing migration" id="add unique index dashboard_acl_dashboard_id_team_id" 2024-04-07T16:13:55.889 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:55 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:55.72134604Z level=info msg="Executing migration" id="add index dashboard_acl_user_id" 2024-04-07T16:13:55.889 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:55 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:55.763315406Z level=info msg="Executing migration" id="add index dashboard_acl_team_id" 2024-04-07T16:13:55.889 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:55 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:55.805248675Z level=info msg="Executing migration" id="add index dashboard_acl_org_id_role" 2024-04-07T16:13:55.889 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:55 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:55.830522408Z level=info msg="Executing migration" id="add index dashboard_permission" 2024-04-07T16:13:55.889 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:55 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:55.847444221Z level=info msg="Executing migration" id="save default acl rules in dashboard_acl table" 2024-04-07T16:13:55.889 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:55 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:55.864320428Z level=info msg="Executing migration" id="delete acl rules for deleted dashboards and folders" 2024-04-07T16:13:55.889 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:55 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:55.897903427Z level=info msg="Executing migration" id="create tag table" 2024-04-07T16:13:56.063 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-04-07T16:13:56.063 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 0702e3a8-f4f8-11ee-b649-cb9ed24678a4 -- ceph osd dump --format=json 2024-04-07T16:13:56.141 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:55 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:55.914874277Z level=info msg="Executing migration" id="add index tag.key_value" 2024-04-07T16:13:56.141 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:55 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:55.948474702Z level=info msg="Executing migration" id="create login attempt table" 2024-04-07T16:13:56.141 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:55 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:55.965411042Z level=info msg="Executing migration" id="add index login_attempt.username" 2024-04-07T16:13:56.141 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:55 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:55.999101504Z level=info msg="Executing migration" id="drop index IDX_login_attempt_username - v1" 2024-04-07T16:13:56.141 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:56 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:56.041031153Z level=info msg="Executing migration" id="Rename table login_attempt to login_attempt_tmp_qwerty - v1" 2024-04-07T16:13:56.141 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:56 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:56.074720115Z level=info msg="Executing migration" id="create login_attempt v2" 2024-04-07T16:13:56.142 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:56 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:56.100012891Z level=info msg="Executing migration" id="create index IDX_login_attempt_username - v2" 2024-04-07T16:13:56.142 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:56 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:56.125255278Z level=info msg="Executing migration" id="copy login_attempt v1 to v2" 2024-04-07T16:13:56.142 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:56 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:56.150425069Z level=info msg="Executing migration" id="drop login_attempt_tmp_qwerty" 2024-04-07T16:13:56.410 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:56 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:56.192441541Z level=info msg="Executing migration" id="create user auth table" 2024-04-07T16:13:56.410 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:56 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:56.23432276Z level=info msg="Executing migration" id="create index IDX_user_auth_auth_module_auth_id - v1" 2024-04-07T16:13:56.411 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:56 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:56.259712189Z level=info msg="Executing migration" id="alter user_auth.auth_id to length 190" 2024-04-07T16:13:56.411 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:56 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:56.284850127Z level=info msg="Executing migration" id="Add OAuth access token to user_auth" 2024-04-07T16:13:56.411 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:56 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:56.326871674Z level=info msg="Executing migration" id="Add OAuth refresh token to user_auth" 2024-04-07T16:13:56.411 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:56 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:56.352087769Z level=info msg="Executing migration" id="Add OAuth token type to user_auth" 2024-04-07T16:13:56.411 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:56 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:56.394397131Z level=info msg="Executing migration" id="Add OAuth expiry to user_auth" 2024-04-07T16:13:56.411 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:56 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:56.41973614Z level=info msg="Executing migration" id="Add index to user_id column in user_auth" 2024-04-07T16:13:56.645 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:13:56 smithi100 bash[21346]: cluster 2024-04-07T16:13:54.625573+0000 mgr.y (mgr.14553) 58 : cluster 0 pgmap v38: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 255 B/s wr, 0 op/s 2024-04-07T16:13:56.645 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:13:56 smithi100 bash[31660]: cluster 2024-04-07T16:13:54.625573+0000 mgr.y (mgr.14553) 58 : cluster 0 pgmap v38: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 255 B/s wr, 0 op/s 2024-04-07T16:13:56.671 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:13:56 smithi144 bash[23522]: cluster 2024-04-07T16:13:54.625573+0000 mgr.y (mgr.14553) 58 : cluster 0 pgmap v38: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 255 B/s wr, 0 op/s 2024-04-07T16:13:56.671 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:56 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:56.444906021Z level=info msg="Executing migration" id="Add OAuth ID token to user_auth" 2024-04-07T16:13:56.672 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:56 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:56.470228806Z level=info msg="Executing migration" id="create server_lock table" 2024-04-07T16:13:56.672 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:56 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:56.495499853Z level=info msg="Executing migration" id="add index server_lock.operation_uid" 2024-04-07T16:13:56.672 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:56 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:56.520758756Z level=info msg="Executing migration" id="create user auth token table" 2024-04-07T16:13:56.672 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:56 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:56.546132608Z level=info msg="Executing migration" id="add unique index user_auth_token.auth_token" 2024-04-07T16:13:56.672 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:56 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:56.571314082Z level=info msg="Executing migration" id="add unique index user_auth_token.prev_auth_token" 2024-04-07T16:13:56.672 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:56 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:56.613436156Z level=info msg="Executing migration" id="add index user_auth_token.user_id" 2024-04-07T16:13:56.672 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:56 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:56.655335392Z level=info msg="Executing migration" id="Add revoked_at to the user auth token" 2024-04-07T16:13:56.672 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:56 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:56.680666515Z level=info msg="Executing migration" id="create cache_data table" 2024-04-07T16:13:56.923 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:56 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:56.705902119Z level=info msg="Executing migration" id="add unique index cache_data.cache_key" 2024-04-07T16:13:56.924 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:56 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:56.747891064Z level=info msg="Executing migration" id="create short_url table v1" 2024-04-07T16:13:56.924 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:56 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:56.773176354Z level=info msg="Executing migration" id="add index short_url.org_id-uid" 2024-04-07T16:13:56.924 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:56 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:56.798401287Z level=info msg="Executing migration" id="alter table short_url alter column created_by type to bigint" 2024-04-07T16:13:56.924 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:56 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:56.823663447Z level=info msg="Executing migration" id="delete alert_definition table" 2024-04-07T16:13:56.924 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:56 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:56.848853847Z level=info msg="Executing migration" id="recreate alert_definition table" 2024-04-07T16:13:56.924 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:56 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:56.890985538Z level=info msg="Executing migration" id="add index in alert_definition on org_id and title columns" 2024-04-07T16:13:56.924 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:56 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:56.932879294Z level=info msg="Executing migration" id="add index in alert_definition on org_id and uid columns" 2024-04-07T16:13:57.201 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:56 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:56.958260219Z level=info msg="Executing migration" id="alter alert_definition table data column to mediumtext in mysql" 2024-04-07T16:13:57.201 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:57 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:57.033578984Z level=info msg="Executing migration" id="drop index in alert_definition on org_id and title columns" 2024-04-07T16:13:57.201 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:57 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:57.075780041Z level=info msg="Executing migration" id="drop index in alert_definition on org_id and uid columns" 2024-04-07T16:13:57.201 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:57 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:57.10111421Z level=info msg="Executing migration" id="add unique index in alert_definition on org_id and title columns" 2024-04-07T16:13:57.201 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:57 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:57.12635747Z level=info msg="Executing migration" id="add unique index in alert_definition on org_id and uid columns" 2024-04-07T16:13:57.202 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:57 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:57.168383589Z level=info msg="Executing migration" id="Add column paused in alert_definition" 2024-04-07T16:13:57.202 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:57 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:57.210329805Z level=info msg="Executing migration" id="drop alert_definition table" 2024-04-07T16:13:57.478 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:57 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:57.227288482Z level=info msg="Executing migration" id="delete alert_definition_version table" 2024-04-07T16:13:57.478 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:57 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:57.329289425Z level=info msg="Executing migration" id="recreate alert_definition_version table" 2024-04-07T16:13:57.478 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:57 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:57.403688066Z level=info msg="Executing migration" id="add index in alert_definition_version table on alert_definition_id and version columns" 2024-04-07T16:13:57.545 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Apr 07 16:13:57 smithi100 systemd[1]: Stopping Ceph alertmanager.a for 0702e3a8-f4f8-11ee-b649-cb9ed24678a4... 2024-04-07T16:13:57.545 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Apr 07 16:13:57 smithi100 bash[58158]: ts=2024-04-07T16:13:57.357Z caller=main.go:583 level=info msg="Received SIGTERM, exiting gracefully..." 2024-04-07T16:13:57.780 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:57 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:57.48718658Z level=info msg="Executing migration" id="add index in alert_definition_version table on alert_definition_uid and version columns" 2024-04-07T16:13:57.780 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:57 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:57.562660496Z level=info msg="Executing migration" id="alter alert_definition_version table data column to mediumtext in mysql" 2024-04-07T16:13:57.781 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:57 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:57.587950689Z level=info msg="Executing migration" id="drop alert_definition_version table" 2024-04-07T16:13:57.781 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:57 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:57.629994837Z level=info msg="Executing migration" id="create alert_instance table" 2024-04-07T16:13:57.781 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:57 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:57.655244157Z level=info msg="Executing migration" id="add index in alert_instance table on def_org_id, def_uid and current_state columns" 2024-04-07T16:13:57.781 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:57 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:57.680503294Z level=info msg="Executing migration" id="add index in alert_instance table on def_org_id, current_state columns" 2024-04-07T16:13:57.781 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:57 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:57.705778733Z level=info msg="Executing migration" id="add column current_state_end to alert_instance" 2024-04-07T16:13:57.781 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:57 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:57.73100595Z level=info msg="Executing migration" id="remove index def_org_id, def_uid, current_state on alert_instance" 2024-04-07T16:13:57.938 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Apr 07 16:13:57 smithi100 bash[59929]: ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4-alertmanager-a 2024-04-07T16:13:58.041 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:57 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:57.789702109Z level=info msg="Executing migration" id="remove index def_org_id, current_state on alert_instance" 2024-04-07T16:13:58.041 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:57 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:57.814921536Z level=info msg="Executing migration" id="rename def_org_id to rule_org_id in alert_instance" 2024-04-07T16:13:58.041 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:57 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:57.856868188Z level=info msg="Executing migration" id="rename def_uid to rule_uid in alert_instance" 2024-04-07T16:13:58.041 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:57 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:57.8989308Z level=info msg="Executing migration" id="add index rule_org_id, rule_uid, current_state on alert_instance" 2024-04-07T16:13:58.041 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:57 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:57.94084079Z level=info msg="Executing migration" id="add index rule_org_id, current_state on alert_instance" 2024-04-07T16:13:58.041 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:57 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:57.966208249Z level=info msg="Executing migration" id="add current_reason column related to current_state" 2024-04-07T16:13:58.042 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:58 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:58.008282197Z level=info msg="Executing migration" id="create alert_rule table" 2024-04-07T16:13:58.042 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:58 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:58.050363289Z level=info msg="Executing migration" id="add index in alert_rule on org_id and title columns" 2024-04-07T16:13:58.313 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:58 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:58.075579069Z level=info msg="Executing migration" id="add index in alert_rule on org_id and uid columns" 2024-04-07T16:13:58.313 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:58 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:58.100817Z level=info msg="Executing migration" id="add index in alert_rule on org_id, namespace_uid, group_uid columns" 2024-04-07T16:13:58.313 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:58 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:58.12608769Z level=info msg="Executing migration" id="alter alert_rule table data column to mediumtext in mysql" 2024-04-07T16:13:58.313 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:58 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:58.168046856Z level=info msg="Executing migration" id="add column for to alert_rule" 2024-04-07T16:13:58.313 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:58 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:58.193461542Z level=info msg="Executing migration" id="add column annotations to alert_rule" 2024-04-07T16:13:58.313 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:58 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:58.243776994Z level=info msg="Executing migration" id="add column labels to alert_rule" 2024-04-07T16:13:58.313 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:58 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:58.268995175Z level=info msg="Executing migration" id="remove unique index from alert_rule on org_id, title columns" 2024-04-07T16:13:58.313 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:58 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:58.296518111Z level=info msg="Executing migration" id="add index in alert_rule on org_id, namespase_uid and title columns" 2024-04-07T16:13:58.319 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Apr 07 16:13:57 smithi100 bash[59987]: Error response from daemon: No such container: ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4-alertmanager-a 2024-04-07T16:13:58.319 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Apr 07 16:13:57 smithi100 systemd[1]: ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@alertmanager.a.service: Deactivated successfully. 2024-04-07T16:13:58.319 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Apr 07 16:13:57 smithi100 systemd[1]: Stopped Ceph alertmanager.a for 0702e3a8-f4f8-11ee-b649-cb9ed24678a4. 2024-04-07T16:13:58.319 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Apr 07 16:13:57 smithi100 systemd[1]: Started Ceph alertmanager.a for 0702e3a8-f4f8-11ee-b649-cb9ed24678a4. 2024-04-07T16:13:58.573 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:58 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:58.322194021Z level=info msg="Executing migration" id="add dashboard_uid column to alert_rule" 2024-04-07T16:13:58.573 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:58 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:58.347111674Z level=info msg="Executing migration" id="add panel_id column to alert_rule" 2024-04-07T16:13:58.573 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:58 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:58.389010438Z level=info msg="Executing migration" id="add index in alert_rule on org_id, dashboard_uid and panel_id columns" 2024-04-07T16:13:58.573 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:58 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:58.414397121Z level=info msg="Executing migration" id="add rule_group_idx column to alert_rule" 2024-04-07T16:13:58.573 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:58 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:58.439538115Z level=info msg="Executing migration" id="add is_paused column to alert_rule table" 2024-04-07T16:13:58.573 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:58 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:58.464846215Z level=info msg="Executing migration" id="fix is_paused column for alert_rule table" 2024-04-07T16:13:58.573 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:58 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:58.490067482Z level=info msg="Executing migration" id="create alert_rule_version table" 2024-04-07T16:13:58.573 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:58 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:58.515400725Z level=info msg="Executing migration" id="add index in alert_rule_version table on rule_org_id, rule_uid and version columns" 2024-04-07T16:13:58.573 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:58 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:58.540590843Z level=info msg="Executing migration" id="add index in alert_rule_version table on rule_org_id, rule_namespace_uid and rule_group columns" 2024-04-07T16:13:58.573 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:58 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:58.58231898Z level=info msg="Executing migration" id="alter alert_rule_version table data column to mediumtext in mysql" 2024-04-07T16:13:58.857 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:58 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:58.792317234Z level=info msg="Executing migration" id="add column for to alert_rule_version" 2024-04-07T16:13:58.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:13:58 smithi144 bash[23522]: cluster 2024-04-07T16:13:56.626067+0000 mgr.y (mgr.14553) 59 : cluster 0 pgmap v39: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 255 B/s wr, 0 op/s 2024-04-07T16:13:58.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:13:58 smithi144 bash[23522]: audit 2024-04-07T16:13:58.029959+0000 mon.a (mon.0) 810 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:13:58.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:13:58 smithi144 bash[23522]: audit 2024-04-07T16:13:58.042007+0000 mon.a (mon.0) 811 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:13:58.869 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:13:58 smithi100 bash[21346]: cluster 2024-04-07T16:13:56.626067+0000 mgr.y (mgr.14553) 59 : cluster 0 pgmap v39: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 255 B/s wr, 0 op/s 2024-04-07T16:13:58.869 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:13:58 smithi100 bash[21346]: audit 2024-04-07T16:13:58.029959+0000 mon.a (mon.0) 810 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:13:58.869 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:13:58 smithi100 bash[21346]: audit 2024-04-07T16:13:58.042007+0000 mon.a (mon.0) 811 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:13:58.869 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:13:58 smithi100 bash[31660]: cluster 2024-04-07T16:13:56.626067+0000 mgr.y (mgr.14553) 59 : cluster 0 pgmap v39: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 255 B/s wr, 0 op/s 2024-04-07T16:13:58.869 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:13:58 smithi100 bash[31660]: audit 2024-04-07T16:13:58.029959+0000 mon.a (mon.0) 810 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:13:58.869 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:13:58 smithi100 bash[31660]: audit 2024-04-07T16:13:58.042007+0000 mon.a (mon.0) 811 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:13:59.188 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Apr 07 16:13:58 smithi100 bash[60014]: ts=2024-04-07T16:13:58.883Z caller=main.go:240 level=info msg="Starting Alertmanager" version="(version=0.25.0, branch=HEAD, revision=258fab7cdd551f2cf251ed0348f0ad7289aee789)" 2024-04-07T16:13:59.189 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Apr 07 16:13:58 smithi100 bash[60014]: ts=2024-04-07T16:13:58.883Z caller=main.go:241 level=info build_context="(go=go1.19.4, user=root@abe866dd5717, date=20221222-14:51:36)" 2024-04-07T16:13:59.189 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Apr 07 16:13:58 smithi100 bash[60014]: ts=2024-04-07T16:13:58.885Z caller=cluster.go:185 level=info component=cluster msg="setting advertise address explicitly" addr=172.21.15.100 port=9094 2024-04-07T16:13:59.189 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Apr 07 16:13:58 smithi100 bash[60014]: ts=2024-04-07T16:13:58.885Z caller=cluster.go:681 level=info component=cluster msg="Waiting for gossip to settle..." interval=2s 2024-04-07T16:13:59.189 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Apr 07 16:13:58 smithi100 bash[60014]: ts=2024-04-07T16:13:58.915Z caller=coordinator.go:113 level=info component=configuration msg="Loading configuration file" file=/etc/alertmanager/alertmanager.yml 2024-04-07T16:13:59.189 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Apr 07 16:13:58 smithi100 bash[60014]: ts=2024-04-07T16:13:58.916Z caller=coordinator.go:126 level=info component=configuration msg="Completed loading of configuration file" file=/etc/alertmanager/alertmanager.yml 2024-04-07T16:13:59.189 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Apr 07 16:13:58 smithi100 bash[60014]: ts=2024-04-07T16:13:58.919Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9093 2024-04-07T16:13:59.189 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Apr 07 16:13:58 smithi100 bash[60014]: ts=2024-04-07T16:13:58.919Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9093 2024-04-07T16:13:59.357 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:59 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:59.063609091Z level=info msg="Executing migration" id="add column annotations to alert_rule_version" 2024-04-07T16:13:59.698 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:13:59 smithi144 bash[23522]: cephadm 2024-04-07T16:13:58.050929+0000 mgr.y (mgr.14553) 60 : cephadm 1 Reconfiguring prometheus.a (dependencies changed)... 2024-04-07T16:13:59.699 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:13:59 smithi144 bash[23522]: cephadm 2024-04-07T16:13:58.286262+0000 mgr.y (mgr.14553) 61 : cephadm 1 Reconfiguring daemon prometheus.a on smithi144 2024-04-07T16:13:59.699 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:59 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:59.38077112Z level=info msg="Executing migration" id="add column labels to alert_rule_version" 2024-04-07T16:13:59.699 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:59 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:59.448015877Z level=info msg="Executing migration" id="add rule_group_idx column to alert_rule_version" 2024-04-07T16:13:59.699 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:59 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:59.570522876Z level=info msg="Executing migration" id="add is_paused column to alert_rule_versions table" 2024-04-07T16:13:59.699 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:59 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:59.621198328Z level=info msg="Executing migration" id="fix is_paused column for alert_rule_version table" 2024-04-07T16:13:59.699 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:59 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:59.707160507Z level=info msg="Executing migration" id=create_alert_configuration_table 2024-04-07T16:13:59.938 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:13:59 smithi100 bash[21346]: cephadm 2024-04-07T16:13:58.050929+0000 mgr.y (mgr.14553) 60 : cephadm 1 Reconfiguring prometheus.a (dependencies changed)... 2024-04-07T16:13:59.938 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:13:59 smithi100 bash[21346]: cephadm 2024-04-07T16:13:58.286262+0000 mgr.y (mgr.14553) 61 : cephadm 1 Reconfiguring daemon prometheus.a on smithi144 2024-04-07T16:13:59.939 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:13:59 smithi100 bash[31660]: cephadm 2024-04-07T16:13:58.050929+0000 mgr.y (mgr.14553) 60 : cephadm 1 Reconfiguring prometheus.a (dependencies changed)... 2024-04-07T16:13:59.939 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:13:59 smithi100 bash[31660]: cephadm 2024-04-07T16:13:58.286262+0000 mgr.y (mgr.14553) 61 : cephadm 1 Reconfiguring daemon prometheus.a on smithi144 2024-04-07T16:13:59.990 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:59 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:59.775810183Z level=info msg="Executing migration" id="Add column default in alert_configuration" 2024-04-07T16:13:59.990 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:59 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:59.847742301Z level=info msg="Executing migration" id="alert alert_configuration alertmanager_configuration column from TEXT to MEDIUMTEXT if mysql" 2024-04-07T16:13:59.991 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:13:59 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:59.906603649Z level=info msg="Executing migration" id="add column org_id in alert_configuration" 2024-04-07T16:14:00.268 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:14:00 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:13:59.999728963Z level=info msg="Executing migration" id="add index in alert_configuration table on org_id column" 2024-04-07T16:14:00.269 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:14:00 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:14:00.049364773Z level=info msg="Executing migration" id="add configuration_hash column to alert_configuration" 2024-04-07T16:14:00.269 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:14:00 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:14:00.091390889Z level=info msg="Executing migration" id=create_ngalert_configuration_table 2024-04-07T16:14:00.269 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:14:00 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:14:00.167402798Z level=info msg="Executing migration" id="add index in ngalert_configuration on org_id column" 2024-04-07T16:14:00.269 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:14:00 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:14:00.20974957Z level=info msg="Executing migration" id="add column send_alerts_to in ngalert_configuration" 2024-04-07T16:14:00.600 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:14:00 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:14:00.278335382Z level=info msg="Executing migration" id="create provenance_type table" 2024-04-07T16:14:00.600 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:14:00 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:14:00.373666178Z level=info msg="Executing migration" id="add index to uniquify (record_key, record_type, org_id) columns" 2024-04-07T16:14:00.600 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:14:00 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:14:00.464989988Z level=info msg="Executing migration" id="create alert_image table" 2024-04-07T16:14:00.791 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/mon.a/config 2024-04-07T16:14:00.871 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:14:00 smithi144 bash[23522]: cluster 2024-04-07T16:13:58.626772+0000 mgr.y (mgr.14553) 62 : cluster 0 pgmap v40: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 255 B/s wr, 0 op/s 2024-04-07T16:14:00.871 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:14:00 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:14:00.682221291Z level=info msg="Executing migration" id="add unique index on token to alert_image table" 2024-04-07T16:14:00.872 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:14:00 smithi100 bash[31660]: cluster 2024-04-07T16:13:58.626772+0000 mgr.y (mgr.14553) 62 : cluster 0 pgmap v40: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 255 B/s wr, 0 op/s 2024-04-07T16:14:00.872 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:14:00 smithi100 bash[21346]: cluster 2024-04-07T16:13:58.626772+0000 mgr.y (mgr.14553) 62 : cluster 0 pgmap v40: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 255 B/s wr, 0 op/s 2024-04-07T16:14:01.175 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:14:00 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:14:00.932501878Z level=info msg="Executing migration" id="support longer URLs in alert_image table" 2024-04-07T16:14:01.175 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:14:01 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:14:01.055529331Z level=info msg="Executing migration" id=create_alert_configuration_history_table 2024-04-07T16:14:01.175 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:14:01 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:14:01.131185373Z level=info msg="Executing migration" id="drop non-unique orgID index on alert_configuration" 2024-04-07T16:14:01.188 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Apr 07 16:14:00 smithi100 bash[60014]: ts=2024-04-07T16:14:00.886Z caller=cluster.go:706 level=info component=cluster msg="gossip not settled" polls=0 before=0 now=1 elapsed=2.000340184s 2024-04-07T16:14:01.461 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:14:01 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:14:01.184494125Z level=info msg="Executing migration" id="drop unique orgID index on alert_configuration if exists" 2024-04-07T16:14:01.461 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:14:01 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:14:01.184968094Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop unique orgID index on alert_configuration if exists" 2024-04-07T16:14:01.461 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:14:01 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:14:01.256386221Z level=info msg="Executing migration" id="extract alertmanager configuration history to separate table" 2024-04-07T16:14:01.837 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:14:01 smithi144 bash[23522]: cluster 2024-04-07T16:14:00.628071+0000 mgr.y (mgr.14553) 63 : cluster 0 pgmap v41: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 255 B/s wr, 0 op/s 2024-04-07T16:14:01.837 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:14:01 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:14:01.470449497Z level=info msg="Executing migration" id="add unique index on orgID to alert_configuration" 2024-04-07T16:14:01.837 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:14:01 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:14:01.643838168Z level=info msg="Executing migration" id="clear migration entry \"remove unified alerting data\"" 2024-04-07T16:14:01.837 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:14:01 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:14:01.644370192Z level=info msg="Executing migration" id="move dashboard alerts to unified alerting" 2024-04-07T16:14:01.837 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:14:01 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:14:01.644829157Z level=info msg="alerts found to migrate" alerts=0 2024-04-07T16:14:01.837 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:14:01 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:14:01.846320466Z level=info msg="Executing migration" id="create library_element table v1" 2024-04-07T16:14:01.893 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:14:01 smithi100 bash[31660]: cluster 2024-04-07T16:14:00.628071+0000 mgr.y (mgr.14553) 63 : cluster 0 pgmap v41: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 255 B/s wr, 0 op/s 2024-04-07T16:14:01.894 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:14:01 smithi100 bash[21346]: cluster 2024-04-07T16:14:00.628071+0000 mgr.y (mgr.14553) 63 : cluster 0 pgmap v41: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 255 B/s wr, 0 op/s 2024-04-07T16:14:02.107 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:14:01 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:14:01.977732157Z level=info msg="Executing migration" id="add index library_element org_id-folder_id-name-kind" 2024-04-07T16:14:02.107 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:14:02 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:14:02.028481783Z level=info msg="Executing migration" id="create library_element_connection table v1" 2024-04-07T16:14:02.415 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:14:02 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:14:02.120563304Z level=info msg="Executing migration" id="add index library_element_connection element_id-kind-connection_id" 2024-04-07T16:14:02.415 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:14:02 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:14:02.179422173Z level=info msg="Executing migration" id="add unique index library_element org_id_uid" 2024-04-07T16:14:02.415 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:14:02 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:14:02.255447964Z level=info msg="Executing migration" id="increase max description length to 2048" 2024-04-07T16:14:02.415 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:14:02 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:14:02.305865333Z level=info msg="Executing migration" id="clone move dashboard alerts to unified alerting" 2024-04-07T16:14:02.415 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:14:02 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:14:02.356376252Z level=info msg="Executing migration" id="create data_keys table" 2024-04-07T16:14:02.415 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:14:02 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:14:02.424326983Z level=info msg="Executing migration" id="create secrets table" 2024-04-07T16:14:02.693 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:14:02 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:14:02.473914656Z level=info msg="Executing migration" id="rename data_keys name column to id" 2024-04-07T16:14:02.693 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:14:02 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:14:02.507538813Z level=info msg="Executing migration" id="add name column into data_keys" 2024-04-07T16:14:02.693 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:14:02 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:14:02.558192251Z level=info msg="Executing migration" id="copy data_keys id column values into name" 2024-04-07T16:14:02.693 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:14:02 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:14:02.625460315Z level=info msg="Executing migration" id="rename data_keys name column to label" 2024-04-07T16:14:03.057 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:14:02 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:14:02.702739269Z level=info msg="Executing migration" id="rename data_keys id column back to name" 2024-04-07T16:14:03.058 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:14:02 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:14:02.800364905Z level=info msg="Executing migration" id="create kv_store table v1" 2024-04-07T16:14:03.058 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:14:02 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:14:02.893846192Z level=info msg="Executing migration" id="add index kv_store.org_id-namespace-key" 2024-04-07T16:14:03.353 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:14:03 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:14:03.064577577Z level=info msg="Executing migration" id="update dashboard_uid and panel_id from existing annotations" 2024-04-07T16:14:03.353 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:14:03 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:14:03.306561353Z level=info msg="Executing migration" id="create permission table" 2024-04-07T16:14:03.606 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:14:03 smithi144 systemd[1]: Stopping Ceph prometheus.a for 0702e3a8-f4f8-11ee-b649-cb9ed24678a4... 2024-04-07T16:14:03.607 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:14:03 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:14:03.362781117Z level=info msg="Executing migration" id="add unique index permission.role_id" 2024-04-07T16:14:03.607 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:14:03 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:14:03.463679903Z level=info msg="Executing migration" id="add unique index role_id_action_scope" 2024-04-07T16:14:03.607 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:14:03 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:14:03.517471092Z level=info msg="Executing migration" id="create role table" 2024-04-07T16:14:03.607 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:14:03 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:14:03.556613785Z level=info msg="Executing migration" id="add column display_name" 2024-04-07T16:14:03.607 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:14:03 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:14:03.573533805Z level=info msg="Executing migration" id="add column group_name" 2024-04-07T16:14:03.607 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:14:03 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:14:03.590483344Z level=info msg="Executing migration" id="add index role.org_id" 2024-04-07T16:14:03.740 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-07T16:14:03.740 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":69,"fsid":"0702e3a8-f4f8-11ee-b649-cb9ed24678a4","created":"2024-04-07T16:02:21.629692+0000","modified":"2024-04-07T16:13:38.014991+0000","last_up_change":"2024-04-07T16:11:30.210001+0000","last_in_change":"2024-04-07T16:11:04.793434+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-07T16:07:26.784071+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":7.8899998664855957,"score_stable":7.8899998664855957,"optimal_score":0.37999999523162842,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-07T16:12:11.856505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-07T16:12:12.737709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"rbd","create_time":"2024-04-07T16:13:28.913394+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":3,"score_stable":3,"optimal_score":1,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8facd48e-0a0c-4fda-aee2-f001b60a3811","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":1636950303},{"type":"v1","addr":"172.21.15.100:6803","nonce":1636950303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":1636950303},{"type":"v1","addr":"172.21.15.100:6805","nonce":1636950303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1636950303},{"type":"v1","addr":"172.21.15.100:6809","nonce":1636950303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":1636950303},{"type":"v1","addr":"172.21.15.100:6807","nonce":1636950303}]},"public_addr":"172.21.15.100:6803/1636950303","cluster_addr":"172.21.15.100:6805/1636950303","heartbeat_back_addr":"172.21.15.100:6809/1636950303","heartbeat_front_addr":"172.21.15.100:6807/1636950303","state":["exists","up"]},{"osd":1,"uuid":"cb5f6c23-6764-48b6-8b21-97a36b004786","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":470179506},{"type":"v1","addr":"172.21.15.100:6811","nonce":470179506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":470179506},{"type":"v1","addr":"172.21.15.100:6813","nonce":470179506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":470179506},{"type":"v1","addr":"172.21.15.100:6817","nonce":470179506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":470179506},{"type":"v1","addr":"172.21.15.100:6815","nonce":470179506}]},"public_addr":"172.21.15.100:6811/470179506","cluster_addr":"172.21.15.100:6813/470179506","heartbeat_back_addr":"172.21.15.100:6817/470179506","heartbeat_front_addr":"172.21.15.100:6815/470179506","state":["exists","up"]},{"osd":2,"uuid":"1650aedb-9ce9-4c14-a059-e1f089e5e63f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":2235490011},{"type":"v1","addr":"172.21.15.100:6819","nonce":2235490011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2235490011},{"type":"v1","addr":"172.21.15.100:6821","nonce":2235490011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":2235490011},{"type":"v1","addr":"172.21.15.100:6825","nonce":2235490011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":2235490011},{"type":"v1","addr":"172.21.15.100:6823","nonce":2235490011}]},"public_addr":"172.21.15.100:6819/2235490011","cluster_addr":"172.21.15.100:6821/2235490011","heartbeat_back_addr":"172.21.15.100:6825/2235490011","heartbeat_front_addr":"172.21.15.100:6823/2235490011","state":["exists","up"]},{"osd":3,"uuid":"17591c6b-862b-43b2-858b-02f4c3199247","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":3019212393},{"type":"v1","addr":"172.21.15.100:6827","nonce":3019212393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":3019212393},{"type":"v1","addr":"172.21.15.100:6829","nonce":3019212393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":3019212393},{"type":"v1","addr":"172.21.15.100:6833","nonce":3019212393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":3019212393},{"type":"v1","addr":"172.21.15.100:6831","nonce":3019212393}]},"public_addr":"172.21.15.100:6827/3019212393","cluster_addr":"172.21.15.100:6829/3019212393","heartbeat_back_addr":"172.21.15.100:6833/3019212393","heartbeat_front_addr":"172.21.15.100:6831/3019212393","state":["exists","up"]},{"osd":4,"uuid":"6ecc1205-7c78-461e-8037-640136cb3658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":974906744},{"type":"v1","addr":"172.21.15.144:6801","nonce":974906744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":974906744},{"type":"v1","addr":"172.21.15.144:6803","nonce":974906744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":974906744},{"type":"v1","addr":"172.21.15.144:6807","nonce":974906744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":974906744},{"type":"v1","addr":"172.21.15.144:6805","nonce":974906744}]},"public_addr":"172.21.15.144:6801/974906744","cluster_addr":"172.21.15.144:6803/974906744","heartbeat_back_addr":"172.21.15.144:6807/974906744","heartbeat_front_addr":"172.21.15.144:6805/974906744","state":["exists","up"]},{"osd":5,"uuid":"54789cc8-1c3c-46a4-a92e-8881564338db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":2783162058},{"type":"v1","addr":"172.21.15.144:6809","nonce":2783162058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":2783162058},{"type":"v1","addr":"172.21.15.144:6811","nonce":2783162058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":2783162058},{"type":"v1","addr":"172.21.15.144:6815","nonce":2783162058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":2783162058},{"type":"v1","addr":"172.21.15.144:6813","nonce":2783162058}]},"public_addr":"172.21.15.144:6809/2783162058","cluster_addr":"172.21.15.144:6811/2783162058","heartbeat_back_addr":"172.21.15.144:6815/2783162058","heartbeat_front_addr":"172.21.15.144:6813/2783162058","state":["exists","up"]},{"osd":6,"uuid":"ac619497-7605-4ca6-84bf-3abb3f6f32d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":1183840595},{"type":"v1","addr":"172.21.15.144:6817","nonce":1183840595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":1183840595},{"type":"v1","addr":"172.21.15.144:6819","nonce":1183840595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":1183840595},{"type":"v1","addr":"172.21.15.144:6823","nonce":1183840595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":1183840595},{"type":"v1","addr":"172.21.15.144:6821","nonce":1183840595}]},"public_addr":"172.21.15.144:6817/1183840595","cluster_addr":"172.21.15.144:6819/1183840595","heartbeat_back_addr":"172.21.15.144:6823/1183840595","heartbeat_front_addr":"172.21.15.144:6821/1183840595","state":["exists","up"]},{"osd":7,"uuid":"b85b3422-9da2-4605-88e7-fe9c3b2020cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":2248617613},{"type":"v1","addr":"172.21.15.144:6825","nonce":2248617613}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":2248617613},{"type":"v1","addr":"172.21.15.144:6827","nonce":2248617613}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":2248617613},{"type":"v1","addr":"172.21.15.144:6831","nonce":2248617613}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":2248617613},{"type":"v1","addr":"172.21.15.144:6829","nonce":2248617613}]},"public_addr":"172.21.15.144:6825/2248617613","cluster_addr":"172.21.15.144:6827/2248617613","heartbeat_back_addr":"172.21.15.144:6831/2248617613","heartbeat_front_addr":"172.21.15.144:6829/2248617613","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-07T16:05:44.696671+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-07T16:06:33.787321+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-07T16:07:22.983158+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-07T16:08:12.935606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-07T16:09:00.218619+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-07T16:09:50.222185+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-07T16:10:37.856158+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-07T16:11:28.267877+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.7","mappings":[{"from":7,"to":4}]},{"pgid":"3.1","mappings":[{"from":0,"to":2}]},{"pgid":"3.12","mappings":[{"from":0,"to":2}]},{"pgid":"3.15","mappings":[{"from":4,"to":1}]},{"pgid":"3.17","mappings":[{"from":0,"to":2}]},{"pgid":"3.1d","mappings":[{"from":4,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.100:0/74983397":"2024-04-08T16:12:52.577534+0000","172.21.15.100:6800/190788518":"2024-04-08T16:12:52.577534+0000","172.21.15.100:0/1309290877":"2024-04-08T16:12:52.577534+0000","172.21.15.100:0/1472671368":"2024-04-08T16:03:24.146096+0000","172.21.15.100:6801/190788518":"2024-04-08T16:12:52.577534+0000","172.21.15.100:0/639120693":"2024-04-08T16:03:24.146096+0000","172.21.15.100:0/2474331937":"2024-04-08T16:03:24.146096+0000","172.21.15.100:0/3273267396":"2024-04-08T16:12:52.577534+0000","172.21.15.100:0/2535336108":"2024-04-08T16:12:52.577534+0000","172.21.15.100:6801/2710911757":"2024-04-08T16:03:24.146096+0000","172.21.15.100:6800/3440287045":"2024-04-08T16:02:53.269514+0000","172.21.15.100:0/109071285":"2024-04-08T16:02:53.269514+0000","172.21.15.100:6801/3440287045":"2024-04-08T16:02:53.269514+0000","172.21.15.100:0/4162615562":"2024-04-08T16:02:53.269514+0000","172.21.15.100:0/4121992264":"2024-04-08T16:02:53.269514+0000","172.21.15.100:6800/2710911757":"2024-04-08T16:03:24.146096+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-07T16:14:03.893 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:14:03 smithi144 bash[23522]: cluster 2024-04-07T16:14:02.628714+0000 mgr.y (mgr.14553) 64 : cluster 0 pgmap v42: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 255 B/s wr, 0 op/s 2024-04-07T16:14:03.893 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:14:03 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:14:03.63250181Z level=info msg="Executing migration" id="add unique index role_org_id_name" 2024-04-07T16:14:03.893 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:14:03 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:14:03.70036403Z level=info msg="Executing migration" id="add index role_org_id_uid" 2024-04-07T16:14:03.893 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:14:03 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:14:03.742441939Z level=info msg="Executing migration" id="create team role table" 2024-04-07T16:14:03.893 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:14:03 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:14:03.826608484Z level=info msg="Executing migration" id="add index team_role.org_id" 2024-04-07T16:14:03.893 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:14:03 smithi144 bash[48586]: ts=2024-04-07T16:14:03.694Z caller=main.go:840 level=warn msg="Received SIGTERM, exiting gracefully..." 2024-04-07T16:14:03.894 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:14:03 smithi144 bash[48586]: ts=2024-04-07T16:14:03.694Z caller=main.go:864 level=info msg="Stopping scrape discovery manager..." 2024-04-07T16:14:03.894 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:14:03 smithi144 bash[48586]: ts=2024-04-07T16:14:03.694Z caller=main.go:878 level=info msg="Stopping notify discovery manager..." 2024-04-07T16:14:03.894 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:14:03 smithi144 bash[48586]: ts=2024-04-07T16:14:03.694Z caller=manager.go:988 level=info component="rule manager" msg="Stopping rule manager..." 2024-04-07T16:14:03.894 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:14:03 smithi144 bash[48586]: ts=2024-04-07T16:14:03.694Z caller=main.go:860 level=info msg="Scrape discovery manager stopped" 2024-04-07T16:14:03.894 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:14:03 smithi144 bash[48586]: ts=2024-04-07T16:14:03.694Z caller=main.go:874 level=info msg="Notify discovery manager stopped" 2024-04-07T16:14:03.894 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:14:03 smithi144 bash[48586]: ts=2024-04-07T16:14:03.694Z caller=manager.go:998 level=info component="rule manager" msg="Rule manager stopped" 2024-04-07T16:14:03.894 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:14:03 smithi144 bash[48586]: ts=2024-04-07T16:14:03.694Z caller=main.go:915 level=info msg="Stopping scrape manager..." 2024-04-07T16:14:03.894 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:14:03 smithi144 bash[48586]: ts=2024-04-07T16:14:03.694Z caller=main.go:907 level=info msg="Scrape manager stopped" 2024-04-07T16:14:03.894 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:14:03 smithi144 bash[48586]: ts=2024-04-07T16:14:03.694Z caller=notifier.go:606 level=info component=notifier msg="Stopping notification manager..." 2024-04-07T16:14:03.894 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:14:03 smithi144 bash[48586]: ts=2024-04-07T16:14:03.694Z caller=main.go:1135 level=info msg="Notifier manager stopped" 2024-04-07T16:14:03.894 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:14:03 smithi144 bash[48586]: ts=2024-04-07T16:14:03.695Z caller=main.go:1147 level=info msg="See you next time!" 2024-04-07T16:14:03.938 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:14:03 smithi100 bash[21346]: cluster 2024-04-07T16:14:02.628714+0000 mgr.y (mgr.14553) 64 : cluster 0 pgmap v42: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 255 B/s wr, 0 op/s 2024-04-07T16:14:03.938 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:14:03 smithi100 bash[31660]: cluster 2024-04-07T16:14:02.628714+0000 mgr.y (mgr.14553) 64 : cluster 0 pgmap v42: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 255 B/s wr, 0 op/s 2024-04-07T16:14:04.143 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:14:03 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:14:03.902047805Z level=info msg="Executing migration" id="add unique index team_role_org_id_team_id_role_id" 2024-04-07T16:14:04.143 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:14:03 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:14:03.994048491Z level=info msg="Executing migration" id="add index team_role.team_id" 2024-04-07T16:14:04.351 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 0702e3a8-f4f8-11ee-b649-cb9ed24678a4 -- ceph tell osd.0 flush_pg_stats 2024-04-07T16:14:04.352 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 0702e3a8-f4f8-11ee-b649-cb9ed24678a4 -- ceph tell osd.1 flush_pg_stats 2024-04-07T16:14:04.352 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 0702e3a8-f4f8-11ee-b649-cb9ed24678a4 -- ceph tell osd.2 flush_pg_stats 2024-04-07T16:14:04.353 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 0702e3a8-f4f8-11ee-b649-cb9ed24678a4 -- ceph tell osd.3 flush_pg_stats 2024-04-07T16:14:04.353 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 0702e3a8-f4f8-11ee-b649-cb9ed24678a4 -- ceph tell osd.4 flush_pg_stats 2024-04-07T16:14:04.354 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 0702e3a8-f4f8-11ee-b649-cb9ed24678a4 -- ceph tell osd.5 flush_pg_stats 2024-04-07T16:14:04.354 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 0702e3a8-f4f8-11ee-b649-cb9ed24678a4 -- ceph tell osd.6 flush_pg_stats 2024-04-07T16:14:04.354 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 0702e3a8-f4f8-11ee-b649-cb9ed24678a4 -- ceph tell osd.7 flush_pg_stats 2024-04-07T16:14:04.495 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:14:04 smithi144 bash[50985]: ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4-prometheus-a 2024-04-07T16:14:04.497 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:14:04 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:14:04.152953631Z level=info msg="Executing migration" id="create user role table" 2024-04-07T16:14:04.497 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:14:04 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:14:04.381574571Z level=info msg="Executing migration" id="add index user_role.org_id" 2024-04-07T16:14:04.497 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:14:04 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:14:04.504713728Z level=info msg="Executing migration" id="add unique index user_role_org_id_user_id_role_id" 2024-04-07T16:14:04.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:14:04 smithi144 bash[23522]: audit 2024-04-07T16:14:03.752972+0000 mon.a (mon.0) 812 : audit 0 from='client.? 172.21.15.100:0/3061835631' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-07T16:14:04.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:14:04 smithi144 bash[23522]: audit 2024-04-07T16:14:04.644483+0000 mon.a (mon.0) 813 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:14:04.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:14:04 smithi144 bash[23522]: audit 2024-04-07T16:14:04.650754+0000 mon.a (mon.0) 814 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:14:04.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:14:04 smithi144 bash[23522]: audit 2024-04-07T16:14:04.667647+0000 mon.b (mon.1) 73 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-07T16:14:04.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:14:04 smithi144 bash[23522]: audit 2024-04-07T16:14:04.669944+0000 mon.b (mon.1) 74 : audit 1 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi100.front.sepia.ceph.com:9093"}]: dispatch 2024-04-07T16:14:04.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:14:04 smithi144 bash[23522]: audit 2024-04-07T16:14:04.680025+0000 mon.a (mon.0) 815 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:14:04.857 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:14:04 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:14:04.580719324Z level=info msg="Executing migration" id="add index user_role.user_id" 2024-04-07T16:14:04.858 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:14:04 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:14:04.623640984Z level=info msg="Executing migration" id="create builtin role table" 2024-04-07T16:14:04.858 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:14:04 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:14:04.750067582Z level=info msg="Executing migration" id="add index builtin_role.role_id" 2024-04-07T16:14:04.858 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:14:04 smithi144 bash[51047]: Error response from daemon: No such container: ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4-prometheus-a 2024-04-07T16:14:04.858 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:14:04 smithi144 systemd[1]: ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@prometheus.a.service: Deactivated successfully. 2024-04-07T16:14:04.858 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:14:04 smithi144 systemd[1]: Stopped Ceph prometheus.a for 0702e3a8-f4f8-11ee-b649-cb9ed24678a4. 2024-04-07T16:14:04.858 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:14:04 smithi144 systemd[1]: Started Ceph prometheus.a for 0702e3a8-f4f8-11ee-b649-cb9ed24678a4. 2024-04-07T16:14:04.938 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:14:04 smithi100 bash[21346]: audit 2024-04-07T16:14:03.752972+0000 mon.a (mon.0) 812 : audit 0 from='client.? 172.21.15.100:0/3061835631' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-07T16:14:04.938 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:14:04 smithi100 bash[21346]: audit 2024-04-07T16:14:04.644483+0000 mon.a (mon.0) 813 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:14:04.938 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:14:04 smithi100 bash[21346]: audit 2024-04-07T16:14:04.650754+0000 mon.a (mon.0) 814 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:14:04.938 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:14:04 smithi100 bash[21346]: audit 2024-04-07T16:14:04.667647+0000 mon.b (mon.1) 73 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-07T16:14:04.938 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:14:04 smithi100 bash[21346]: audit 2024-04-07T16:14:04.669944+0000 mon.b (mon.1) 74 : audit 1 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi100.front.sepia.ceph.com:9093"}]: dispatch 2024-04-07T16:14:04.938 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:14:04 smithi100 bash[21346]: audit 2024-04-07T16:14:04.680025+0000 mon.a (mon.0) 815 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:14:04.939 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:14:04 smithi100 bash[31660]: audit 2024-04-07T16:14:03.752972+0000 mon.a (mon.0) 812 : audit 0 from='client.? 172.21.15.100:0/3061835631' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-07T16:14:04.939 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:14:04 smithi100 bash[31660]: audit 2024-04-07T16:14:04.644483+0000 mon.a (mon.0) 813 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:14:04.939 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:14:04 smithi100 bash[31660]: audit 2024-04-07T16:14:04.650754+0000 mon.a (mon.0) 814 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:14:04.939 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:14:04 smithi100 bash[31660]: audit 2024-04-07T16:14:04.667647+0000 mon.b (mon.1) 73 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-07T16:14:04.939 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:14:04 smithi100 bash[31660]: audit 2024-04-07T16:14:04.669944+0000 mon.b (mon.1) 74 : audit 1 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi100.front.sepia.ceph.com:9093"}]: dispatch 2024-04-07T16:14:04.939 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:14:04 smithi100 bash[31660]: audit 2024-04-07T16:14:04.680025+0000 mon.a (mon.0) 815 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:14:04.939 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:14:04 smithi100 bash[21601]: [07/Apr/2024:16:14:04] ENGINE Bus STOPPING 2024-04-07T16:14:04.939 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:14:04 smithi100 bash[21601]: [07/Apr/2024:16:14:04] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-04-07T16:14:04.939 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:14:04 smithi100 bash[21601]: [07/Apr/2024:16:14:04] ENGINE Bus STOPPED 2024-04-07T16:14:04.939 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:14:04 smithi100 bash[21601]: [07/Apr/2024:16:14:04] ENGINE Bus STARTING 2024-04-07T16:14:04.940 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:14:04 smithi100 bash[21601]: [07/Apr/2024:16:14:04] ENGINE Serving on http://:::9283 2024-04-07T16:14:04.940 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:14:04 smithi100 bash[21601]: [07/Apr/2024:16:14:04] ENGINE Bus STARTED 2024-04-07T16:14:04.940 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:14:04 smithi100 bash[21601]: [07/Apr/2024:16:14:04] ENGINE Bus STOPPING 2024-04-07T16:14:04.940 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:14:04 smithi100 bash[21601]: [07/Apr/2024:16:14:04] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-04-07T16:14:04.940 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:14:04 smithi100 bash[21601]: [07/Apr/2024:16:14:04] ENGINE Bus STOPPED 2024-04-07T16:14:04.940 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:14:04 smithi100 bash[21601]: [07/Apr/2024:16:14:04] ENGINE Bus STARTING 2024-04-07T16:14:04.940 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:14:04 smithi100 bash[21601]: [07/Apr/2024:16:14:04] ENGINE Serving on http://:::9283 2024-04-07T16:14:04.940 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:14:04 smithi100 bash[21601]: [07/Apr/2024:16:14:04] ENGINE Bus STARTED 2024-04-07T16:14:04.940 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:14:04 smithi100 bash[21601]: [07/Apr/2024:16:14:04] ENGINE Bus STOPPING 2024-04-07T16:14:04.940 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:14:04 smithi100 bash[21601]: [07/Apr/2024:16:14:04] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-04-07T16:14:04.940 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:14:04 smithi100 bash[21601]: [07/Apr/2024:16:14:04] ENGINE Bus STOPPED 2024-04-07T16:14:04.940 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:14:04 smithi100 bash[21601]: [07/Apr/2024:16:14:04] ENGINE Bus STARTING 2024-04-07T16:14:05.357 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:14:05 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:14:05.017429392Z level=info msg="Executing migration" id="add index builtin_role.name" 2024-04-07T16:14:05.357 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:14:05 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:14:05.253634379Z level=info msg="Executing migration" id="Add column org_id to builtin_role table" 2024-04-07T16:14:05.438 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:14:05 smithi100 bash[21601]: [07/Apr/2024:16:14:05] ENGINE Serving on http://:::9283 2024-04-07T16:14:05.438 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:14:05 smithi100 bash[21601]: [07/Apr/2024:16:14:05] ENGINE Bus STARTED 2024-04-07T16:14:05.667 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:14:05 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:14:05.395253119Z level=info msg="Executing migration" id="add index builtin_role.org_id" 2024-04-07T16:14:05.668 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:14:05 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:14:05.487898612Z level=info msg="Executing migration" id="add unique index builtin_role_org_id_role_id_role" 2024-04-07T16:14:05.668 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:14:05 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:14:05.580278285Z level=info msg="Executing migration" id="Remove unique index role_org_id_uid" 2024-04-07T16:14:05.668 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:14:05 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:14:05.676991826Z level=info msg="Executing migration" id="add unique index role.uid" 2024-04-07T16:14:05.939 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:14:05 smithi100 bash[21346]: cluster 2024-04-07T16:14:04.629348+0000 mgr.y (mgr.14553) 65 : cluster 0 pgmap v43: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 255 B/s wr, 0 op/s 2024-04-07T16:14:05.939 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:14:05 smithi100 bash[21346]: audit 2024-04-07T16:14:04.673244+0000 mgr.y (mgr.14553) 66 : audit 0 from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-07T16:14:05.939 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:14:05 smithi100 bash[21346]: audit 2024-04-07T16:14:04.675668+0000 mgr.y (mgr.14553) 67 : audit 0 from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi100.front.sepia.ceph.com:9093"}]: dispatch 2024-04-07T16:14:05.939 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:14:05 smithi100 bash[21346]: audit 2024-04-07T16:14:04.689025+0000 mon.b (mon.1) 75 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-07T16:14:05.939 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:14:05 smithi100 bash[21346]: audit 2024-04-07T16:14:04.690999+0000 mon.b (mon.1) 76 : audit 1 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi144.front.sepia.ceph.com:9095"}]: dispatch 2024-04-07T16:14:05.939 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:14:05 smithi100 bash[21346]: audit 2024-04-07T16:14:04.694721+0000 mgr.y (mgr.14553) 68 : audit 0 from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-07T16:14:05.939 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:14:05 smithi100 bash[21346]: audit 2024-04-07T16:14:04.696853+0000 mgr.y (mgr.14553) 69 : audit 0 from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi144.front.sepia.ceph.com:9095"}]: dispatch 2024-04-07T16:14:05.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:14:05 smithi100 bash[21346]: audit 2024-04-07T16:14:04.701664+0000 mon.a (mon.0) 816 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:14:05.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:14:05 smithi100 bash[21346]: audit 2024-04-07T16:14:04.706088+0000 mon.b (mon.1) 77 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-07T16:14:05.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:14:05 smithi100 bash[21346]: audit 2024-04-07T16:14:04.708274+0000 mon.b (mon.1) 78 : audit 1 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi144.front.sepia.ceph.com:3000"}]: dispatch 2024-04-07T16:14:05.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:14:05 smithi100 bash[21346]: audit 2024-04-07T16:14:04.711891+0000 mgr.y (mgr.14553) 70 : audit 0 from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-07T16:14:05.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:14:05 smithi100 bash[21346]: audit 2024-04-07T16:14:04.714110+0000 mgr.y (mgr.14553) 71 : audit 0 from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi144.front.sepia.ceph.com:3000"}]: dispatch 2024-04-07T16:14:05.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:14:05 smithi100 bash[21346]: audit 2024-04-07T16:14:04.718241+0000 mon.a (mon.0) 817 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:14:05.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:14:05 smithi100 bash[21346]: audit 2024-04-07T16:14:04.725196+0000 mon.b (mon.1) 79 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:14:05.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:14:05 smithi100 bash[21346]: audit 2024-04-07T16:14:05.107689+0000 mon.a (mon.0) 818 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:14:05.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:14:05 smithi100 bash[21346]: audit 2024-04-07T16:14:05.122484+0000 mon.a (mon.0) 819 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:14:05.940 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:14:05 smithi100 bash[31660]: cluster 2024-04-07T16:14:04.629348+0000 mgr.y (mgr.14553) 65 : cluster 0 pgmap v43: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 255 B/s wr, 0 op/s 2024-04-07T16:14:05.940 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:14:05 smithi100 bash[31660]: audit 2024-04-07T16:14:04.673244+0000 mgr.y (mgr.14553) 66 : audit 0 from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-07T16:14:05.940 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:14:05 smithi100 bash[31660]: audit 2024-04-07T16:14:04.675668+0000 mgr.y (mgr.14553) 67 : audit 0 from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi100.front.sepia.ceph.com:9093"}]: dispatch 2024-04-07T16:14:05.941 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:14:05 smithi100 bash[31660]: audit 2024-04-07T16:14:04.689025+0000 mon.b (mon.1) 75 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-07T16:14:05.941 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:14:05 smithi100 bash[31660]: audit 2024-04-07T16:14:04.690999+0000 mon.b (mon.1) 76 : audit 1 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi144.front.sepia.ceph.com:9095"}]: dispatch 2024-04-07T16:14:05.941 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:14:05 smithi100 bash[31660]: audit 2024-04-07T16:14:04.694721+0000 mgr.y (mgr.14553) 68 : audit 0 from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-07T16:14:05.941 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:14:05 smithi100 bash[31660]: audit 2024-04-07T16:14:04.696853+0000 mgr.y (mgr.14553) 69 : audit 0 from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi144.front.sepia.ceph.com:9095"}]: dispatch 2024-04-07T16:14:05.941 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:14:05 smithi100 bash[31660]: audit 2024-04-07T16:14:04.701664+0000 mon.a (mon.0) 816 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:14:05.941 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:14:05 smithi100 bash[31660]: audit 2024-04-07T16:14:04.706088+0000 mon.b (mon.1) 77 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-07T16:14:05.941 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:14:05 smithi100 bash[31660]: audit 2024-04-07T16:14:04.708274+0000 mon.b (mon.1) 78 : audit 1 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi144.front.sepia.ceph.com:3000"}]: dispatch 2024-04-07T16:14:05.941 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:14:05 smithi100 bash[31660]: audit 2024-04-07T16:14:04.711891+0000 mgr.y (mgr.14553) 70 : audit 0 from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-07T16:14:05.941 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:14:05 smithi100 bash[31660]: audit 2024-04-07T16:14:04.714110+0000 mgr.y (mgr.14553) 71 : audit 0 from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi144.front.sepia.ceph.com:3000"}]: dispatch 2024-04-07T16:14:05.941 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:14:05 smithi100 bash[31660]: audit 2024-04-07T16:14:04.718241+0000 mon.a (mon.0) 817 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:14:05.941 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:14:05 smithi100 bash[31660]: audit 2024-04-07T16:14:04.725196+0000 mon.b (mon.1) 79 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:14:05.941 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:14:05 smithi100 bash[31660]: audit 2024-04-07T16:14:05.107689+0000 mon.a (mon.0) 818 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:14:05.941 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:14:05 smithi100 bash[31660]: audit 2024-04-07T16:14:05.122484+0000 mon.a (mon.0) 819 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:14:05.950 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:14:05 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:14:05.722789907Z level=info msg="Executing migration" id="create seed assignment table" 2024-04-07T16:14:05.950 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:14:05 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:14:05.773172576Z level=info msg="Executing migration" id="add unique index builtin_role_role_name" 2024-04-07T16:14:05.950 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:14:05 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:14:05.831683433Z level=info msg="Executing migration" id="add column hidden to role table" 2024-04-07T16:14:05.950 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:14:05 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:14:05.882230503Z level=info msg="Executing migration" id="create query_history table v1" 2024-04-07T16:14:05.950 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:14:05 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:14:05.924471783Z level=info msg="Executing migration" id="add index query_history.org_id-created_by-datasource_uid" 2024-04-07T16:14:05.950 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:14:05 smithi144 bash[23522]: cluster 2024-04-07T16:14:04.629348+0000 mgr.y (mgr.14553) 65 : cluster 0 pgmap v43: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 255 B/s wr, 0 op/s 2024-04-07T16:14:05.951 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:14:05 smithi144 bash[23522]: audit 2024-04-07T16:14:04.673244+0000 mgr.y (mgr.14553) 66 : audit 0 from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-07T16:14:05.951 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:14:05 smithi144 bash[23522]: audit 2024-04-07T16:14:04.675668+0000 mgr.y (mgr.14553) 67 : audit 0 from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi100.front.sepia.ceph.com:9093"}]: dispatch 2024-04-07T16:14:05.951 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:14:05 smithi144 bash[23522]: audit 2024-04-07T16:14:04.689025+0000 mon.b (mon.1) 75 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-07T16:14:05.951 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:14:05 smithi144 bash[23522]: audit 2024-04-07T16:14:04.690999+0000 mon.b (mon.1) 76 : audit 1 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi144.front.sepia.ceph.com:9095"}]: dispatch 2024-04-07T16:14:05.951 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:14:05 smithi144 bash[23522]: audit 2024-04-07T16:14:04.694721+0000 mgr.y (mgr.14553) 68 : audit 0 from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-07T16:14:05.951 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:14:05 smithi144 bash[23522]: audit 2024-04-07T16:14:04.696853+0000 mgr.y (mgr.14553) 69 : audit 0 from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi144.front.sepia.ceph.com:9095"}]: dispatch 2024-04-07T16:14:05.951 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:14:05 smithi144 bash[23522]: audit 2024-04-07T16:14:04.701664+0000 mon.a (mon.0) 816 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:14:05.951 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:14:05 smithi144 bash[23522]: audit 2024-04-07T16:14:04.706088+0000 mon.b (mon.1) 77 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-07T16:14:05.951 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:14:05 smithi144 bash[23522]: audit 2024-04-07T16:14:04.708274+0000 mon.b (mon.1) 78 : audit 1 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi144.front.sepia.ceph.com:3000"}]: dispatch 2024-04-07T16:14:05.951 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:14:05 smithi144 bash[23522]: audit 2024-04-07T16:14:04.711891+0000 mgr.y (mgr.14553) 70 : audit 0 from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-07T16:14:05.951 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:14:05 smithi144 bash[23522]: audit 2024-04-07T16:14:04.714110+0000 mgr.y (mgr.14553) 71 : audit 0 from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi144.front.sepia.ceph.com:3000"}]: dispatch 2024-04-07T16:14:05.951 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:14:05 smithi144 bash[23522]: audit 2024-04-07T16:14:04.718241+0000 mon.a (mon.0) 817 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:14:05.951 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:14:05 smithi144 bash[23522]: audit 2024-04-07T16:14:04.725196+0000 mon.b (mon.1) 79 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:14:05.951 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:14:05 smithi144 bash[23522]: audit 2024-04-07T16:14:05.107689+0000 mon.a (mon.0) 818 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:14:05.951 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:14:05 smithi144 bash[23522]: audit 2024-04-07T16:14:05.122484+0000 mon.a (mon.0) 819 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:14:06.227 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:14:05 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:14:05.974928222Z level=info msg="Executing migration" id="alter table query_history alter column created_by type to bigint" 2024-04-07T16:14:06.227 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:14:06 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:14:06.042491302Z level=info msg="Executing migration" id="teams permissions migration" 2024-04-07T16:14:06.227 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:14:06 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:14:06.084475381Z level=info msg="Executing migration" id="dashboard permissions" 2024-04-07T16:14:06.227 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:14:06 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:14:06.109867107Z level=info msg="Executing migration" id="dashboard permissions uid scopes" 2024-04-07T16:14:06.227 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:14:06 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:14:06.135167976Z level=info msg="Executing migration" id="drop managed folder create actions" 2024-04-07T16:14:06.227 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:14:06 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:14:06.177174498Z level=info msg="Executing migration" id="alerting notification permissions" 2024-04-07T16:14:06.227 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:14:06 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:14:06.23588704Z level=info msg="Executing migration" id="create query_history_star table v1" 2024-04-07T16:14:06.228 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:14:05 smithi144 bash[51076]: ts=2024-04-07T16:14:05.959Z caller=main.go:564 level=info msg="Starting Prometheus Server" mode=server version="(version=2.43.0, branch=HEAD, revision=edfc3bcd025dd6fe296c167a14a216cab1e552ee)" 2024-04-07T16:14:06.228 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:14:05 smithi144 bash[51076]: ts=2024-04-07T16:14:05.959Z caller=main.go:569 level=info build_context="(go=go1.19.7, platform=linux/amd64, user=root@8a0ee342e522, date=20230321-12:56:07, tags=netgo,builtinassets)" 2024-04-07T16:14:06.228 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:14:05 smithi144 bash[51076]: ts=2024-04-07T16:14:05.959Z caller=main.go:570 level=info host_details="(Linux 5.15.0-101-generic #111-Ubuntu SMP Tue Mar 5 20:16:58 UTC 2024 x86_64 smithi144 (none))" 2024-04-07T16:14:06.228 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:14:05 smithi144 bash[51076]: ts=2024-04-07T16:14:05.959Z caller=main.go:571 level=info fd_limits="(soft=1048576, hard=1048576)" 2024-04-07T16:14:06.228 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:14:05 smithi144 bash[51076]: ts=2024-04-07T16:14:05.959Z caller=main.go:572 level=info vm_limits="(soft=unlimited, hard=unlimited)" 2024-04-07T16:14:06.228 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:14:05 smithi144 bash[51076]: ts=2024-04-07T16:14:05.961Z caller=web.go:561 level=info component=web msg="Start listening for connections" address=:9095 2024-04-07T16:14:06.228 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:14:05 smithi144 bash[51076]: ts=2024-04-07T16:14:05.962Z caller=main.go:1005 level=info msg="Starting TSDB ..." 2024-04-07T16:14:06.228 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:14:05 smithi144 bash[51076]: ts=2024-04-07T16:14:05.964Z caller=tls_config.go:232 level=info component=web msg="Listening on" address=[::]:9095 2024-04-07T16:14:06.228 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:14:05 smithi144 bash[51076]: ts=2024-04-07T16:14:05.964Z caller=tls_config.go:235 level=info component=web msg="TLS is disabled." http2=false address=[::]:9095 2024-04-07T16:14:06.228 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:14:05 smithi144 bash[51076]: ts=2024-04-07T16:14:05.966Z caller=head.go:587 level=info component=tsdb msg="Replaying on-disk memory mappable chunks if any" 2024-04-07T16:14:06.228 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:14:05 smithi144 bash[51076]: ts=2024-04-07T16:14:05.966Z caller=head.go:658 level=info component=tsdb msg="On-disk memory mappable chunks replay completed" duration=1.972µs 2024-04-07T16:14:06.228 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:14:05 smithi144 bash[51076]: ts=2024-04-07T16:14:05.966Z caller=head.go:664 level=info component=tsdb msg="Replaying WAL, this may take a while" 2024-04-07T16:14:06.228 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:14:05 smithi144 bash[51076]: ts=2024-04-07T16:14:05.968Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=0 maxSegment=1 2024-04-07T16:14:06.229 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:14:05 smithi144 bash[51076]: ts=2024-04-07T16:14:05.968Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=1 maxSegment=1 2024-04-07T16:14:06.229 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:14:05 smithi144 bash[51076]: ts=2024-04-07T16:14:05.968Z caller=head.go:772 level=info component=tsdb msg="WAL replay completed" checkpoint_replay_duration=34.941µs wal_replay_duration=2.552855ms wbl_replay_duration=153ns total_replay_duration=2.608175ms 2024-04-07T16:14:06.229 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:14:05 smithi144 bash[51076]: ts=2024-04-07T16:14:05.969Z caller=main.go:1026 level=info fs_type=XFS_SUPER_MAGIC 2024-04-07T16:14:06.229 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:14:05 smithi144 bash[51076]: ts=2024-04-07T16:14:05.969Z caller=main.go:1029 level=info msg="TSDB started" 2024-04-07T16:14:06.229 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:14:05 smithi144 bash[51076]: ts=2024-04-07T16:14:05.969Z caller=main.go:1209 level=info msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml 2024-04-07T16:14:06.229 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:14:05 smithi144 bash[51076]: ts=2024-04-07T16:14:05.990Z caller=main.go:1246 level=info msg="Completed loading of configuration file" filename=/etc/prometheus/prometheus.yml totalDuration=20.933973ms db_storage=889ns remote_storage=1.662µs web_handler=404ns query_engine=616ns scrape=348.926µs scrape_sd=179.243µs notify=14.757µs notify_sd=27.407µs rules=20.072715ms tracing=6.305µs 2024-04-07T16:14:06.229 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:14:05 smithi144 bash[51076]: ts=2024-04-07T16:14:05.990Z caller=main.go:990 level=info msg="Server is ready to receive web requests." 2024-04-07T16:14:06.229 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:14:05 smithi144 bash[51076]: ts=2024-04-07T16:14:05.990Z caller=manager.go:974 level=info component="rule manager" msg="Starting rule manager..." 2024-04-07T16:14:06.505 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:14:06 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:14:06.261248635Z level=info msg="Executing migration" id="add index query_history.user_id-query_uid" 2024-04-07T16:14:06.506 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:14:06 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:14:06.286529416Z level=info msg="Executing migration" id="add column org_id in query_history_star" 2024-04-07T16:14:06.506 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:14:06 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:14:06.328510348Z level=info msg="Executing migration" id="alter table query_history_star_mig column user_id type to bigint" 2024-04-07T16:14:06.506 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:14:06 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:14:06.353829632Z level=info msg="Executing migration" id="create correlation table v1" 2024-04-07T16:14:06.506 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:14:06 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:14:06.37916941Z level=info msg="Executing migration" id="add index correlations.uid" 2024-04-07T16:14:06.506 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:14:06 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:14:06.404424973Z level=info msg="Executing migration" id="add index correlations.source_uid" 2024-04-07T16:14:06.506 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:14:06 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:14:06.421302378Z level=info msg="Executing migration" id="add correlation config column" 2024-04-07T16:14:06.506 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:14:06 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:14:06.446889099Z level=info msg="Executing migration" id="create entity_events table" 2024-04-07T16:14:06.506 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:14:06 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:14:06.4639227Z level=info msg="Executing migration" id="create dashboard public config v1" 2024-04-07T16:14:06.506 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:14:06 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:14:06.480825385Z level=info msg="Executing migration" id="drop index UQE_dashboard_public_config_uid - v1" 2024-04-07T16:14:06.506 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:14:06 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:14:06.481191258Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop index UQE_dashboard_public_config_uid - v1" 2024-04-07T16:14:06.506 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:14:06 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:14:06.514511217Z level=info msg="Executing migration" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-04-07T16:14:06.758 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:14:06 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:14:06.514912679Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-04-07T16:14:06.758 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:14:06 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:14:06.539654243Z level=info msg="Executing migration" id="Drop old dashboard public config table" 2024-04-07T16:14:06.758 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:14:06 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:14:06.564997197Z level=info msg="Executing migration" id="recreate dashboard public config v1" 2024-04-07T16:14:06.758 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:14:06 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:14:06.607081738Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_uid - v1" 2024-04-07T16:14:06.758 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:14:06 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:14:06.649097397Z level=info msg="Executing migration" id="create index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-04-07T16:14:06.758 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:14:06 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:14:06.674494067Z level=info msg="Executing migration" id="drop index UQE_dashboard_public_config_uid - v2" 2024-04-07T16:14:06.758 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:14:06 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:14:06.716469156Z level=info msg="Executing migration" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v2" 2024-04-07T16:14:06.759 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:14:06 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:14:06.741700195Z level=info msg="Executing migration" id="Drop public config table" 2024-04-07T16:14:06.759 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:14:06 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:14:06.767058772Z level=info msg="Executing migration" id="Recreate dashboard public config v2" 2024-04-07T16:14:07.018 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:14:06 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:14:06.792245005Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_uid - v2" 2024-04-07T16:14:07.019 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:14:06 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:14:06.834216374Z level=info msg="Executing migration" id="create index IDX_dashboard_public_config_org_id_dashboard_uid - v2" 2024-04-07T16:14:07.019 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:14:06 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:14:06.859528825Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_access_token - v2" 2024-04-07T16:14:07.019 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:14:06 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:14:06.90158443Z level=info msg="Executing migration" id="Rename table dashboard_public_config to dashboard_public - v2" 2024-04-07T16:14:07.019 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:14:06 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:14:06.951890155Z level=info msg="Executing migration" id="add annotations_enabled column" 2024-04-07T16:14:07.019 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:14:06 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:14:06.977262753Z level=info msg="Executing migration" id="add time_selection_enabled column" 2024-04-07T16:14:07.019 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:14:07 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:14:07.002550154Z level=info msg="Executing migration" id="delete orphaned public dashboards" 2024-04-07T16:14:07.019 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:14:07 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:14:07.02780526Z level=info msg="Executing migration" id="add share column" 2024-04-07T16:14:07.296 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:14:07 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:14:07.053022279Z level=info msg="Executing migration" id="create default alerting folders" 2024-04-07T16:14:07.296 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:14:07 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:14:07.095102894Z level=info msg="Executing migration" id="create file table" 2024-04-07T16:14:07.296 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:14:07 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:14:07.120402918Z level=info msg="Executing migration" id="file table idx: path natural pk" 2024-04-07T16:14:07.297 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:14:07 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:14:07.145674727Z level=info msg="Executing migration" id="file table idx: parent_folder_path_hash fast folder retrieval" 2024-04-07T16:14:07.297 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:14:07 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:14:07.171009139Z level=info msg="Executing migration" id="create file_meta table" 2024-04-07T16:14:07.297 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:14:07 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:14:07.212881688Z level=info msg="Executing migration" id="file table idx: path key" 2024-04-07T16:14:07.297 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:14:07 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:14:07.254979926Z level=info msg="Executing migration" id="set path collation in file table" 2024-04-07T16:14:07.549 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:14:07 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:14:07.305549043Z level=info msg="Executing migration" id="managed permissions migration" 2024-04-07T16:14:07.549 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:14:07 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:14:07.347611192Z level=info msg="Executing migration" id="managed folder permissions alert actions migration" 2024-04-07T16:14:07.549 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:14:07 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:14:07.372797182Z level=info msg="Executing migration" id="RBAC action name migrator" 2024-04-07T16:14:07.549 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:14:07 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:14:07.389962284Z level=info msg="Executing migration" id="Add UID column to playlist" 2024-04-07T16:14:07.549 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:14:07 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:14:07.415228249Z level=info msg="Executing migration" id="Update uid column values in playlist" 2024-04-07T16:14:07.549 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:14:07 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:14:07.457197229Z level=info msg="Executing migration" id="Add index for uid in playlist" 2024-04-07T16:14:07.549 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:14:07 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:14:07.482607689Z level=info msg="Executing migration" id="update group index for alert rules" 2024-04-07T16:14:07.549 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:14:07 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:14:07.507717927Z level=info msg="Executing migration" id="managed folder permissions alert actions repeated migration" 2024-04-07T16:14:07.550 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:14:07 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:14:07.532989111Z level=info msg="Executing migration" id="admin only folder/dashboard permission" 2024-04-07T16:14:07.550 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:14:07 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:14:07.558326859Z level=info msg="Executing migration" id="add action column to seed_assignment" 2024-04-07T16:14:07.835 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:14:07 smithi144 bash[23522]: cluster 2024-04-07T16:14:06.629777+0000 mgr.y (mgr.14553) 72 : cluster 0 pgmap v44: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:14:07.835 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:14:07 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:14:07.58360315Z level=info msg="Executing migration" id="add scope column to seed_assignment" 2024-04-07T16:14:07.835 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:14:07 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:14:07.625717932Z level=info msg="Executing migration" id="remove unique index builtin_role_role_name before nullable update" 2024-04-07T16:14:07.835 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:14:07 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:14:07.65100502Z level=info msg="Executing migration" id="update seed_assignment role_name column to nullable" 2024-04-07T16:14:07.835 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:14:07 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:14:07.7519985Z level=info msg="Executing migration" id="add unique index builtin_role_name back" 2024-04-07T16:14:07.835 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:14:07 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:14:07.77675889Z level=info msg="Executing migration" id="add unique index builtin_role_action_scope" 2024-04-07T16:14:07.835 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:14:07 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:14:07.8021622Z level=info msg="Executing migration" id="add primary key to seed_assigment" 2024-04-07T16:14:07.835 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:14:07 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:14:07.844131713Z level=info msg="Executing migration" id="managed folder permissions alert actions repeated fixed migration" 2024-04-07T16:14:07.938 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:14:07 smithi100 bash[21346]: cluster 2024-04-07T16:14:06.629777+0000 mgr.y (mgr.14553) 72 : cluster 0 pgmap v44: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:14:07.938 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:14:07 smithi100 bash[31660]: cluster 2024-04-07T16:14:06.629777+0000 mgr.y (mgr.14553) 72 : cluster 0 pgmap v44: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:14:08.107 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:14:07 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:14:07.869309422Z level=info msg="Executing migration" id="migrate external alertmanagers to datsourcse" 2024-04-07T16:14:08.107 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:14:07 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:14:07.911335668Z level=info msg="Executing migration" id="create folder table" 2024-04-07T16:14:08.107 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:14:07 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:14:07.953446389Z level=info msg="Executing migration" id="Add index for parent_uid" 2024-04-07T16:14:08.107 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:14:07 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:14:07.978564271Z level=info msg="Executing migration" id="Add unique index for folder.uid and folder.org_id" 2024-04-07T16:14:08.107 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:14:08 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:14:08.003899822Z level=info msg="Executing migration" id="Update folder title length" 2024-04-07T16:14:08.107 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:14:08 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:14:08.029213635Z level=info msg="Executing migration" id="Add unique index for folder.title and folder.parent_uid" 2024-04-07T16:14:08.107 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:14:08 smithi144 bash[50386]: logger=migrator t=2024-04-07T16:14:08.054410871Z level=info msg="migrations completed" performed=485 skipped=0 duration=20.438818614s 2024-04-07T16:14:08.107 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:14:08 smithi144 bash[50386]: logger=sqlstore t=2024-04-07T16:14:08.055163746Z level=info msg="Created default organization" 2024-04-07T16:14:08.607 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:14:08 smithi144 bash[50386]: logger=plugin.loader t=2024-04-07T16:14:08.188635246Z level=info msg="Plugin registered" pluginID=input 2024-04-07T16:14:08.607 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:14:08 smithi144 bash[50386]: logger=plugin.loader t=2024-04-07T16:14:08.222259177Z level=info msg="Plugin registered" pluginID=grafana-piechart-panel 2024-04-07T16:14:08.607 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:14:08 smithi144 bash[50386]: logger=plugin.loader t=2024-04-07T16:14:08.222289548Z level=info msg="Plugin registered" pluginID=vonage-status-panel 2024-04-07T16:14:08.607 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:14:08 smithi144 bash[50386]: logger=secrets t=2024-04-07T16:14:08.222341261Z level=info msg="Envelope encryption state" enabled=true currentprovider=secretKey.v1 2024-04-07T16:14:08.607 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:14:08 smithi144 bash[50386]: logger=query_data t=2024-04-07T16:14:08.224438487Z level=info msg="Query Service initialization" 2024-04-07T16:14:08.607 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:14:08 smithi144 bash[50386]: logger=live.push_http t=2024-04-07T16:14:08.228802052Z level=info msg="Live Push Gateway initialization" 2024-04-07T16:14:08.938 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:14:08 smithi100 bash[21346]: audit 2024-04-07T16:14:07.698140+0000 mon.b (mon.1) 80 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:14:08.938 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:14:08 smithi100 bash[31660]: audit 2024-04-07T16:14:07.698140+0000 mon.b (mon.1) 80 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:14:08.939 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Apr 07 16:14:08 smithi100 bash[60014]: ts=2024-04-07T16:14:08.889Z caller=cluster.go:698 level=info component=cluster msg="gossip settled; proceeding" elapsed=10.003632724s 2024-04-07T16:14:09.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:14:08 smithi144 bash[23522]: audit 2024-04-07T16:14:07.698140+0000 mon.b (mon.1) 80 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:14:09.176 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/mon.a/config 2024-04-07T16:14:09.176 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/mon.a/config 2024-04-07T16:14:09.176 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/mon.a/config 2024-04-07T16:14:09.177 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/mon.a/config 2024-04-07T16:14:09.180 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/mon.a/config 2024-04-07T16:14:09.180 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/mon.a/config 2024-04-07T16:14:09.180 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/mon.a/config 2024-04-07T16:14:09.181 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/mon.a/config 2024-04-07T16:14:09.695 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:14:09 smithi144 bash[50386]: logger=infra.usagestats.collector t=2024-04-07T16:14:09.424020335Z level=info msg="registering usage stat providers" usageStatsProvidersLen=2 2024-04-07T16:14:09.695 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:14:09 smithi144 bash[50386]: logger=provisioning.datasources t=2024-04-07T16:14:09.48010896Z level=info msg="inserting datasource from configuration " name=Dashboard1 uid=P43CA22E17D0F9596 2024-04-07T16:14:09.695 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:14:09 smithi144 bash[50386]: logger=provisioning.datasources t=2024-04-07T16:14:09.575287366Z level=info msg="inserting datasource from configuration " name=Loki uid=P8E80F9AEF21F6940 2024-04-07T16:14:10.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:14:09 smithi144 bash[23522]: cluster 2024-04-07T16:14:08.630407+0000 mgr.y (mgr.14553) 73 : cluster 0 pgmap v45: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:14:10.107 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:14:09 smithi144 bash[50386]: logger=provisioning.alerting t=2024-04-07T16:14:09.741149078Z level=info msg="starting to provision alerting" 2024-04-07T16:14:10.107 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:14:09 smithi144 bash[50386]: logger=provisioning.alerting t=2024-04-07T16:14:09.741238981Z level=info msg="finished to provision alerting" 2024-04-07T16:14:10.107 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:14:09 smithi144 bash[50386]: logger=ngalert.state.manager t=2024-04-07T16:14:09.741629877Z level=info msg="Warming state cache for startup" 2024-04-07T16:14:10.107 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:14:09 smithi144 bash[50386]: logger=grafanaStorageLogger t=2024-04-07T16:14:09.741736736Z level=info msg="storage starting" 2024-04-07T16:14:10.107 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:14:09 smithi144 bash[50386]: logger=http.server t=2024-04-07T16:14:09.744035058Z level=info msg="HTTP Server Listen" address=[::]:3000 protocol=https subUrl= socket= 2024-04-07T16:14:10.107 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:14:09 smithi144 bash[50386]: logger=ngalert.state.manager t=2024-04-07T16:14:09.846913876Z level=info msg="State cache has been initialized" states=0 duration=105.281509ms 2024-04-07T16:14:10.107 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:14:09 smithi144 bash[50386]: logger=ticker t=2024-04-07T16:14:09.847045105Z level=info msg=starting first_tick=2024-04-07T16:14:10Z 2024-04-07T16:14:10.107 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:14:09 smithi144 bash[50386]: logger=ngalert.multiorg.alertmanager t=2024-04-07T16:14:09.847030465Z level=info msg="starting MultiOrg Alertmanager" 2024-04-07T16:14:10.107 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:14:09 smithi144 bash[50386]: logger=sqlstore.transactions t=2024-04-07T16:14:09.933910602Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-04-07T16:14:10.188 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:14:09 smithi100 bash[31660]: cluster 2024-04-07T16:14:08.630407+0000 mgr.y (mgr.14553) 73 : cluster 0 pgmap v45: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:14:10.188 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:14:09 smithi100 bash[21346]: cluster 2024-04-07T16:14:08.630407+0000 mgr.y (mgr.14553) 73 : cluster 0 pgmap v45: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:14:10.802 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:14:10 smithi100 bash[21346]: audit 2024-04-07T16:14:09.766250+0000 mon.a (mon.0) 820 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:14:10.802 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:14:10 smithi100 bash[21346]: audit 2024-04-07T16:14:09.775122+0000 mon.a (mon.0) 821 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:14:10.802 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:14:10 smithi100 bash[31660]: audit 2024-04-07T16:14:09.766250+0000 mon.a (mon.0) 820 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:14:10.802 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:14:10 smithi100 bash[31660]: audit 2024-04-07T16:14:09.775122+0000 mon.a (mon.0) 821 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:14:11.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:14:10 smithi144 bash[23522]: audit 2024-04-07T16:14:09.766250+0000 mon.a (mon.0) 820 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:14:11.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:14:10 smithi144 bash[23522]: audit 2024-04-07T16:14:09.775122+0000 mon.a (mon.0) 821 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:14:12.188 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:14:11 smithi100 bash[21346]: cluster 2024-04-07T16:14:10.631174+0000 mgr.y (mgr.14553) 74 : cluster 0 pgmap v46: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:14:12.188 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:14:11 smithi100 bash[21346]: audit 2024-04-07T16:14:10.874672+0000 mon.a (mon.0) 822 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:14:12.188 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:14:11 smithi100 bash[21346]: audit 2024-04-07T16:14:10.884055+0000 mon.a (mon.0) 823 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:14:12.188 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:14:11 smithi100 bash[21346]: audit 2024-04-07T16:14:11.351262+0000 mon.b (mon.1) 81 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:14:12.189 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:14:11 smithi100 bash[21346]: audit 2024-04-07T16:14:11.353149+0000 mon.b (mon.1) 82 : audit 1 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:14:12.189 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:14:11 smithi100 bash[21346]: audit 2024-04-07T16:14:11.370824+0000 mon.a (mon.0) 824 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:14:12.189 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:14:11 smithi100 bash[31660]: cluster 2024-04-07T16:14:10.631174+0000 mgr.y (mgr.14553) 74 : cluster 0 pgmap v46: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:14:12.189 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:14:11 smithi100 bash[31660]: audit 2024-04-07T16:14:10.874672+0000 mon.a (mon.0) 822 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:14:12.189 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:14:11 smithi100 bash[31660]: audit 2024-04-07T16:14:10.884055+0000 mon.a (mon.0) 823 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:14:12.189 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:14:11 smithi100 bash[31660]: audit 2024-04-07T16:14:11.351262+0000 mon.b (mon.1) 81 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:14:12.189 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:14:11 smithi100 bash[31660]: audit 2024-04-07T16:14:11.353149+0000 mon.b (mon.1) 82 : audit 1 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:14:12.189 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:14:11 smithi100 bash[31660]: audit 2024-04-07T16:14:11.370824+0000 mon.a (mon.0) 824 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:14:12.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:14:11 smithi144 bash[23522]: cluster 2024-04-07T16:14:10.631174+0000 mgr.y (mgr.14553) 74 : cluster 0 pgmap v46: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:14:12.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:14:11 smithi144 bash[23522]: audit 2024-04-07T16:14:10.874672+0000 mon.a (mon.0) 822 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:14:12.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:14:11 smithi144 bash[23522]: audit 2024-04-07T16:14:10.884055+0000 mon.a (mon.0) 823 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:14:12.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:14:11 smithi144 bash[23522]: audit 2024-04-07T16:14:11.351262+0000 mon.b (mon.1) 81 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:14:12.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:14:11 smithi144 bash[23522]: audit 2024-04-07T16:14:11.353149+0000 mon.b (mon.1) 82 : audit 1 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:14:12.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:14:11 smithi144 bash[23522]: audit 2024-04-07T16:14:11.370824+0000 mon.a (mon.0) 824 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:14:13.984 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:14:13 smithi100 bash[21346]: cluster 2024-04-07T16:14:12.631653+0000 mgr.y (mgr.14553) 75 : cluster 0 pgmap v47: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:14:13.985 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:14:13 smithi100 bash[31660]: cluster 2024-04-07T16:14:12.631653+0000 mgr.y (mgr.14553) 75 : cluster 0 pgmap v47: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:14:14.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:14:13 smithi144 bash[23522]: cluster 2024-04-07T16:14:12.631653+0000 mgr.y (mgr.14553) 75 : cluster 0 pgmap v47: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:14:15.405 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:14:15 smithi100 bash[21601]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:14:15] "GET /metrics HTTP/1.1" 200 33728 "" "Prometheus/2.43.0" 2024-04-07T16:14:16.010 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:14:15 smithi100 bash[21346]: cluster 2024-04-07T16:14:14.632276+0000 mgr.y (mgr.14553) 76 : cluster 0 pgmap v48: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:14:16.010 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:14:15 smithi100 bash[31660]: cluster 2024-04-07T16:14:14.632276+0000 mgr.y (mgr.14553) 76 : cluster 0 pgmap v48: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:14:16.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:14:15 smithi144 bash[23522]: cluster 2024-04-07T16:14:14.632276+0000 mgr.y (mgr.14553) 76 : cluster 0 pgmap v48: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:14:17.938 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:14:17 smithi100 bash[21346]: cluster 2024-04-07T16:14:16.632626+0000 mgr.y (mgr.14553) 77 : cluster 0 pgmap v49: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:14:17.938 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:14:17 smithi100 bash[31660]: cluster 2024-04-07T16:14:16.632626+0000 mgr.y (mgr.14553) 77 : cluster 0 pgmap v49: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:14:18.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:14:17 smithi144 bash[23522]: cluster 2024-04-07T16:14:16.632626+0000 mgr.y (mgr.14553) 77 : cluster 0 pgmap v49: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:14:20.188 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:14:19 smithi100 bash[31660]: cluster 2024-04-07T16:14:18.633414+0000 mgr.y (mgr.14553) 78 : cluster 0 pgmap v50: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:14:20.188 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:14:19 smithi100 bash[21346]: cluster 2024-04-07T16:14:18.633414+0000 mgr.y (mgr.14553) 78 : cluster 0 pgmap v50: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:14:20.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:14:19 smithi144 bash[23522]: cluster 2024-04-07T16:14:18.633414+0000 mgr.y (mgr.14553) 78 : cluster 0 pgmap v50: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:14:21.228 INFO:teuthology.orchestra.run.smithi100.stdout:107374182474 2024-04-07T16:14:21.228 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 0702e3a8-f4f8-11ee-b649-cb9ed24678a4 -- ceph osd last-stat-seq osd.3 2024-04-07T16:14:21.962 INFO:teuthology.orchestra.run.smithi100.stdout:128849018945 2024-04-07T16:14:21.963 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 0702e3a8-f4f8-11ee-b649-cb9ed24678a4 -- ceph osd last-stat-seq osd.4 2024-04-07T16:14:22.188 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:14:21 smithi100 bash[21346]: cluster 2024-04-07T16:14:20.634647+0000 mgr.y (mgr.14553) 79 : cluster 0 pgmap v51: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:14:22.188 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:14:21 smithi100 bash[31660]: cluster 2024-04-07T16:14:20.634647+0000 mgr.y (mgr.14553) 79 : cluster 0 pgmap v51: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:14:22.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:14:21 smithi144 bash[23522]: cluster 2024-04-07T16:14:20.634647+0000 mgr.y (mgr.14553) 79 : cluster 0 pgmap v51: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:14:22.532 INFO:teuthology.orchestra.run.smithi100.stdout:55834574941 2024-04-07T16:14:22.533 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 0702e3a8-f4f8-11ee-b649-cb9ed24678a4 -- ceph osd last-stat-seq osd.1 2024-04-07T16:14:23.143 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:14:22 smithi100 bash[21346]: audit 2024-04-07T16:14:22.697390+0000 mon.b (mon.1) 83 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:14:23.143 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:14:22 smithi100 bash[31660]: audit 2024-04-07T16:14:22.697390+0000 mon.b (mon.1) 83 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:14:23.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:14:22 smithi144 bash[23522]: audit 2024-04-07T16:14:22.697390+0000 mon.b (mon.1) 83 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:14:24.188 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:14:23 smithi100 bash[21346]: cluster 2024-04-07T16:14:22.635383+0000 mgr.y (mgr.14553) 80 : cluster 0 pgmap v52: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:14:24.188 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:14:23 smithi100 bash[31660]: cluster 2024-04-07T16:14:22.635383+0000 mgr.y (mgr.14553) 80 : cluster 0 pgmap v52: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:14:24.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:14:23 smithi144 bash[23522]: cluster 2024-04-07T16:14:22.635383+0000 mgr.y (mgr.14553) 80 : cluster 0 pgmap v52: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:14:24.641 INFO:teuthology.orchestra.run.smithi100.stdout:231928234020 2024-04-07T16:14:24.641 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 0702e3a8-f4f8-11ee-b649-cb9ed24678a4 -- ceph osd last-stat-seq osd.7 2024-04-07T16:14:25.239 INFO:teuthology.orchestra.run.smithi100.stdout:34359738472 2024-04-07T16:14:25.239 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 0702e3a8-f4f8-11ee-b649-cb9ed24678a4 -- ceph osd last-stat-seq osd.0 2024-04-07T16:14:25.382 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:14:25 smithi100 bash[21601]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:14:25] "GET /metrics HTTP/1.1" 200 33731 "" "Prometheus/2.43.0" 2024-04-07T16:14:25.830 INFO:teuthology.orchestra.run.smithi100.stdout:197568495661 2024-04-07T16:14:25.830 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 0702e3a8-f4f8-11ee-b649-cb9ed24678a4 -- ceph osd last-stat-seq osd.6 2024-04-07T16:14:25.836 INFO:teuthology.orchestra.run.smithi100.stdout:163208757303 2024-04-07T16:14:25.836 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 0702e3a8-f4f8-11ee-b649-cb9ed24678a4 -- ceph osd last-stat-seq osd.5 2024-04-07T16:14:25.982 INFO:teuthology.orchestra.run.smithi100.stdout:77309411412 2024-04-07T16:14:25.982 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 0702e3a8-f4f8-11ee-b649-cb9ed24678a4 -- ceph osd last-stat-seq osd.2 2024-04-07T16:14:26.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:14:25 smithi100 bash[31660]: cluster 2024-04-07T16:14:24.636718+0000 mgr.y (mgr.14553) 81 : cluster 0 pgmap v53: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:14:26.188 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:14:25 smithi100 bash[21346]: cluster 2024-04-07T16:14:24.636718+0000 mgr.y (mgr.14553) 81 : cluster 0 pgmap v53: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:14:26.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:14:25 smithi144 bash[23522]: cluster 2024-04-07T16:14:24.636718+0000 mgr.y (mgr.14553) 81 : cluster 0 pgmap v53: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:14:28.188 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:14:27 smithi100 bash[21346]: cluster 2024-04-07T16:14:26.637366+0000 mgr.y (mgr.14553) 82 : cluster 0 pgmap v54: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:14:28.188 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:14:27 smithi100 bash[31660]: cluster 2024-04-07T16:14:26.637366+0000 mgr.y (mgr.14553) 82 : cluster 0 pgmap v54: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:14:28.356 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/mon.a/config 2024-04-07T16:14:28.357 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/mon.a/config 2024-04-07T16:14:28.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:14:27 smithi144 bash[23522]: cluster 2024-04-07T16:14:26.637366+0000 mgr.y (mgr.14553) 82 : cluster 0 pgmap v54: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:14:29.374 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/mon.a/config 2024-04-07T16:14:30.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:14:29 smithi100 bash[21346]: cluster 2024-04-07T16:14:28.637986+0000 mgr.y (mgr.14553) 83 : cluster 0 pgmap v55: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:14:30.188 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:14:29 smithi100 bash[31660]: cluster 2024-04-07T16:14:28.637986+0000 mgr.y (mgr.14553) 83 : cluster 0 pgmap v55: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:14:30.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:14:29 smithi144 bash[23522]: cluster 2024-04-07T16:14:28.637986+0000 mgr.y (mgr.14553) 83 : cluster 0 pgmap v55: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:14:31.995 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/mon.a/config 2024-04-07T16:14:31.995 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/mon.a/config 2024-04-07T16:14:31.996 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/mon.a/config 2024-04-07T16:14:31.996 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/mon.a/config 2024-04-07T16:14:31.996 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/mon.a/config 2024-04-07T16:14:32.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:14:31 smithi100 bash[21346]: cluster 2024-04-07T16:14:30.639196+0000 mgr.y (mgr.14553) 84 : cluster 0 pgmap v56: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:14:32.188 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:14:31 smithi100 bash[31660]: cluster 2024-04-07T16:14:30.639196+0000 mgr.y (mgr.14553) 84 : cluster 0 pgmap v56: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:14:32.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:14:31 smithi144 bash[23522]: cluster 2024-04-07T16:14:30.639196+0000 mgr.y (mgr.14553) 84 : cluster 0 pgmap v56: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:14:33.502 INFO:teuthology.orchestra.run.smithi100.stdout:107374182477 2024-04-07T16:14:34.188 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:14:33 smithi100 bash[21346]: cluster 2024-04-07T16:14:32.639815+0000 mgr.y (mgr.14553) 85 : cluster 0 pgmap v57: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:14:34.188 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:14:33 smithi100 bash[21346]: audit 2024-04-07T16:14:33.517633+0000 mon.a (mon.0) 825 : audit 0 from='client.? 172.21.15.100:0/3775843996' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-07T16:14:34.188 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:14:33 smithi100 bash[31660]: cluster 2024-04-07T16:14:32.639815+0000 mgr.y (mgr.14553) 85 : cluster 0 pgmap v57: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:14:34.188 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:14:33 smithi100 bash[31660]: audit 2024-04-07T16:14:33.517633+0000 mon.a (mon.0) 825 : audit 0 from='client.? 172.21.15.100:0/3775843996' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-07T16:14:34.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:14:33 smithi144 bash[23522]: cluster 2024-04-07T16:14:32.639815+0000 mgr.y (mgr.14553) 85 : cluster 0 pgmap v57: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:14:34.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:14:33 smithi144 bash[23522]: audit 2024-04-07T16:14:33.517633+0000 mon.a (mon.0) 825 : audit 0 from='client.? 172.21.15.100:0/3775843996' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-07T16:14:34.617 INFO:teuthology.orchestra.run.smithi100.stdout:128849018948 2024-04-07T16:14:35.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:14:34 smithi100 bash[21346]: audit 2024-04-07T16:14:34.632056+0000 mon.a (mon.0) 826 : audit 0 from='client.? 172.21.15.100:0/1604311610' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-07T16:14:35.188 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:14:35 smithi100 bash[21601]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:14:35] "GET /metrics HTTP/1.1" 200 33731 "" "Prometheus/2.43.0" 2024-04-07T16:14:35.188 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:14:34 smithi100 bash[31660]: audit 2024-04-07T16:14:34.632056+0000 mon.a (mon.0) 826 : audit 0 from='client.? 172.21.15.100:0/1604311610' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-07T16:14:35.268 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182474 got 107374182477 for osd.3 2024-04-07T16:14:35.268 DEBUG:teuthology.parallel:result is None 2024-04-07T16:14:35.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:14:34 smithi144 bash[23522]: audit 2024-04-07T16:14:34.632056+0000 mon.a (mon.0) 826 : audit 0 from='client.? 172.21.15.100:0/1604311610' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-07T16:14:36.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:14:35 smithi100 bash[21346]: cluster 2024-04-07T16:14:34.640531+0000 mgr.y (mgr.14553) 86 : cluster 0 pgmap v58: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:14:36.188 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:14:35 smithi100 bash[31660]: cluster 2024-04-07T16:14:34.640531+0000 mgr.y (mgr.14553) 86 : cluster 0 pgmap v58: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:14:36.205 INFO:tasks.cephadm.ceph_manager.ceph:need seq 128849018945 got 128849018948 for osd.4 2024-04-07T16:14:36.205 DEBUG:teuthology.parallel:result is None 2024-04-07T16:14:36.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:14:35 smithi144 bash[23522]: cluster 2024-04-07T16:14:34.640531+0000 mgr.y (mgr.14553) 86 : cluster 0 pgmap v58: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:14:37.782 INFO:teuthology.orchestra.run.smithi100.stdout:55834574944 2024-04-07T16:14:38.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:14:37 smithi100 bash[21346]: cluster 2024-04-07T16:14:36.640984+0000 mgr.y (mgr.14553) 87 : cluster 0 pgmap v59: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:14:38.037 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:14:37 smithi100 bash[21346]: audit 2024-04-07T16:14:37.698180+0000 mon.b (mon.1) 84 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:14:38.038 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:14:37 smithi100 bash[21346]: audit 2024-04-07T16:14:37.797867+0000 mon.a (mon.0) 827 : audit 0 from='client.? 172.21.15.100:0/1234761484' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-07T16:14:38.038 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:14:37 smithi100 bash[31660]: cluster 2024-04-07T16:14:36.640984+0000 mgr.y (mgr.14553) 87 : cluster 0 pgmap v59: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:14:38.038 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:14:37 smithi100 bash[31660]: audit 2024-04-07T16:14:37.698180+0000 mon.b (mon.1) 84 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:14:38.038 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:14:37 smithi100 bash[31660]: audit 2024-04-07T16:14:37.797867+0000 mon.a (mon.0) 827 : audit 0 from='client.? 172.21.15.100:0/1234761484' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-07T16:14:38.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:14:37 smithi144 bash[23522]: cluster 2024-04-07T16:14:36.640984+0000 mgr.y (mgr.14553) 87 : cluster 0 pgmap v59: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:14:38.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:14:37 smithi144 bash[23522]: audit 2024-04-07T16:14:37.698180+0000 mon.b (mon.1) 84 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:14:38.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:14:37 smithi144 bash[23522]: audit 2024-04-07T16:14:37.797867+0000 mon.a (mon.0) 827 : audit 0 from='client.? 172.21.15.100:0/1234761484' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-07T16:14:40.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:14:39 smithi100 bash[31660]: cluster 2024-04-07T16:14:38.641652+0000 mgr.y (mgr.14553) 88 : cluster 0 pgmap v60: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:14:40.188 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:14:39 smithi100 bash[21346]: cluster 2024-04-07T16:14:38.641652+0000 mgr.y (mgr.14553) 88 : cluster 0 pgmap v60: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:14:40.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:14:39 smithi144 bash[23522]: cluster 2024-04-07T16:14:38.641652+0000 mgr.y (mgr.14553) 88 : cluster 0 pgmap v60: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:14:40.939 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574941 got 55834574944 for osd.1 2024-04-07T16:14:40.939 DEBUG:teuthology.parallel:result is None 2024-04-07T16:14:42.007 INFO:teuthology.orchestra.run.smithi100.stdout:77309411416 2024-04-07T16:14:42.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:14:41 smithi100 bash[21346]: cluster 2024-04-07T16:14:40.642920+0000 mgr.y (mgr.14553) 89 : cluster 0 pgmap v61: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:14:42.188 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:14:41 smithi100 bash[31660]: cluster 2024-04-07T16:14:40.642920+0000 mgr.y (mgr.14553) 89 : cluster 0 pgmap v61: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:14:42.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:14:41 smithi144 bash[23522]: cluster 2024-04-07T16:14:40.642920+0000 mgr.y (mgr.14553) 89 : cluster 0 pgmap v61: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:14:43.039 INFO:teuthology.orchestra.run.smithi100.stdout:163208757307 2024-04-07T16:14:43.048 INFO:teuthology.orchestra.run.smithi100.stdout:34359738476 2024-04-07T16:14:43.112 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411412 got 77309411416 for osd.2 2024-04-07T16:14:43.112 DEBUG:teuthology.parallel:result is None 2024-04-07T16:14:43.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:14:42 smithi100 bash[21346]: audit 2024-04-07T16:14:42.022713+0000 mon.c (mon.2) 33 : audit 0 from='client.? 172.21.15.100:0/2735217877' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-07T16:14:43.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:14:42 smithi100 bash[31660]: audit 2024-04-07T16:14:42.022713+0000 mon.c (mon.2) 33 : audit 0 from='client.? 172.21.15.100:0/2735217877' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-07T16:14:43.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:14:42 smithi144 bash[23522]: audit 2024-04-07T16:14:42.022713+0000 mon.c (mon.2) 33 : audit 0 from='client.? 172.21.15.100:0/2735217877' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-07T16:14:43.428 INFO:teuthology.orchestra.run.smithi100.stdout:231928234024 2024-04-07T16:14:43.446 INFO:teuthology.orchestra.run.smithi100.stdout:197568495665 2024-04-07T16:14:44.188 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:14:43 smithi100 bash[21346]: cluster 2024-04-07T16:14:42.643361+0000 mgr.y (mgr.14553) 90 : cluster 0 pgmap v62: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:14:44.188 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:14:43 smithi100 bash[21346]: audit 2024-04-07T16:14:43.054189+0000 mon.a (mon.0) 828 : audit 0 from='client.? 172.21.15.100:0/157700684' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-07T16:14:44.188 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:14:43 smithi100 bash[21346]: audit 2024-04-07T16:14:43.064048+0000 mon.a (mon.0) 829 : audit 0 from='client.? 172.21.15.100:0/335718286' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-07T16:14:44.188 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:14:43 smithi100 bash[21346]: audit 2024-04-07T16:14:43.443279+0000 mon.c (mon.2) 34 : audit 0 from='client.? 172.21.15.100:0/3568958110' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-07T16:14:44.188 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:14:43 smithi100 bash[21346]: audit 2024-04-07T16:14:43.461275+0000 mon.a (mon.0) 830 : audit 0 from='client.? 172.21.15.100:0/3062087619' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-07T16:14:44.188 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:14:43 smithi100 bash[31660]: cluster 2024-04-07T16:14:42.643361+0000 mgr.y (mgr.14553) 90 : cluster 0 pgmap v62: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:14:44.188 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:14:43 smithi100 bash[31660]: audit 2024-04-07T16:14:43.054189+0000 mon.a (mon.0) 828 : audit 0 from='client.? 172.21.15.100:0/157700684' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-07T16:14:44.188 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:14:43 smithi100 bash[31660]: audit 2024-04-07T16:14:43.064048+0000 mon.a (mon.0) 829 : audit 0 from='client.? 172.21.15.100:0/335718286' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-07T16:14:44.189 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:14:43 smithi100 bash[31660]: audit 2024-04-07T16:14:43.443279+0000 mon.c (mon.2) 34 : audit 0 from='client.? 172.21.15.100:0/3568958110' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-07T16:14:44.189 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:14:43 smithi100 bash[31660]: audit 2024-04-07T16:14:43.461275+0000 mon.a (mon.0) 830 : audit 0 from='client.? 172.21.15.100:0/3062087619' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-07T16:14:44.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:14:43 smithi144 bash[23522]: cluster 2024-04-07T16:14:42.643361+0000 mgr.y (mgr.14553) 90 : cluster 0 pgmap v62: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:14:44.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:14:43 smithi144 bash[23522]: audit 2024-04-07T16:14:43.054189+0000 mon.a (mon.0) 828 : audit 0 from='client.? 172.21.15.100:0/157700684' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-07T16:14:44.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:14:43 smithi144 bash[23522]: audit 2024-04-07T16:14:43.064048+0000 mon.a (mon.0) 829 : audit 0 from='client.? 172.21.15.100:0/335718286' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-07T16:14:44.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:14:43 smithi144 bash[23522]: audit 2024-04-07T16:14:43.443279+0000 mon.c (mon.2) 34 : audit 0 from='client.? 172.21.15.100:0/3568958110' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-07T16:14:44.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:14:43 smithi144 bash[23522]: audit 2024-04-07T16:14:43.461275+0000 mon.a (mon.0) 830 : audit 0 from='client.? 172.21.15.100:0/3062087619' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-07T16:14:44.821 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738472 got 34359738476 for osd.0 2024-04-07T16:14:44.822 DEBUG:teuthology.parallel:result is None 2024-04-07T16:14:45.072 INFO:tasks.cephadm.ceph_manager.ceph:need seq 163208757303 got 163208757307 for osd.5 2024-04-07T16:14:45.072 DEBUG:teuthology.parallel:result is None 2024-04-07T16:14:45.238 INFO:tasks.cephadm.ceph_manager.ceph:need seq 231928234020 got 231928234024 for osd.7 2024-04-07T16:14:45.239 DEBUG:teuthology.parallel:result is None 2024-04-07T16:14:45.303 INFO:tasks.cephadm.ceph_manager.ceph:need seq 197568495661 got 197568495665 for osd.6 2024-04-07T16:14:45.303 DEBUG:teuthology.parallel:result is None 2024-04-07T16:14:45.303 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-04-07T16:14:45.304 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 0702e3a8-f4f8-11ee-b649-cb9ed24678a4 -- ceph pg dump --format=json 2024-04-07T16:14:45.437 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:14:45 smithi100 bash[21601]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:14:45] "GET /metrics HTTP/1.1" 200 33730 "" "Prometheus/2.43.0" 2024-04-07T16:14:46.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:14:45 smithi144 bash[23522]: cluster 2024-04-07T16:14:44.644589+0000 mgr.y (mgr.14553) 91 : cluster 0 pgmap v63: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:14:46.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:14:45 smithi100 bash[21346]: cluster 2024-04-07T16:14:44.644589+0000 mgr.y (mgr.14553) 91 : cluster 0 pgmap v63: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:14:46.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:14:45 smithi100 bash[31660]: cluster 2024-04-07T16:14:44.644589+0000 mgr.y (mgr.14553) 91 : cluster 0 pgmap v63: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:14:48.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:14:47 smithi144 bash[23522]: cluster 2024-04-07T16:14:46.645260+0000 mgr.y (mgr.14553) 92 : cluster 0 pgmap v64: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:14:48.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:14:47 smithi100 bash[21346]: cluster 2024-04-07T16:14:46.645260+0000 mgr.y (mgr.14553) 92 : cluster 0 pgmap v64: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:14:48.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:14:47 smithi100 bash[31660]: cluster 2024-04-07T16:14:46.645260+0000 mgr.y (mgr.14553) 92 : cluster 0 pgmap v64: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:14:50.045 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/mon.a/config 2024-04-07T16:14:50.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:14:49 smithi144 bash[23522]: cluster 2024-04-07T16:14:48.646005+0000 mgr.y (mgr.14553) 93 : cluster 0 pgmap v65: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:14:50.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:14:49 smithi100 bash[21346]: cluster 2024-04-07T16:14:48.646005+0000 mgr.y (mgr.14553) 93 : cluster 0 pgmap v65: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:14:50.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:14:49 smithi100 bash[31660]: cluster 2024-04-07T16:14:48.646005+0000 mgr.y (mgr.14553) 93 : cluster 0 pgmap v65: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:14:52.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:14:51 smithi144 bash[23522]: cluster 2024-04-07T16:14:50.647292+0000 mgr.y (mgr.14553) 94 : cluster 0 pgmap v66: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:14:52.395 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:14:51 smithi100 bash[21346]: cluster 2024-04-07T16:14:50.647292+0000 mgr.y (mgr.14553) 94 : cluster 0 pgmap v66: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:14:52.396 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:14:51 smithi100 bash[31660]: cluster 2024-04-07T16:14:50.647292+0000 mgr.y (mgr.14553) 94 : cluster 0 pgmap v66: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:14:52.823 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-07T16:14:52.825 INFO:teuthology.orchestra.run.smithi100.stderr:dumped all 2024-04-07T16:14:53.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:14:52 smithi100 bash[21346]: audit 2024-04-07T16:14:52.698694+0000 mon.b (mon.1) 85 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:14:53.188 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:14:52 smithi100 bash[31660]: audit 2024-04-07T16:14:52.698694+0000 mon.b (mon.1) 85 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:14:53.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:14:52 smithi144 bash[23522]: audit 2024-04-07T16:14:52.698694+0000 mon.b (mon.1) 85 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:14:53.516 INFO:teuthology.orchestra.run.smithi100.stdout:{"pg_ready":true,"pg_map":{"version":67,"stamp":"2024-04-07T16:14:52.647493+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":593798,"num_objects":24,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":230,"num_write_kb":2436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":173,"ondisk_log_size":173,"up":315,"acting":315,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":315,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":235804,"kb_used_data":7368,"kb_used_omap":12,"kb_used_meta":228339,"kb_avail":749725412,"statfs":{"total":767960285184,"available":767718821888,"internally_reserved":0,"allocated":7544832,"data_stored":4254919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":233819705},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005508"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.637813+0000","last_change":"2024-04-07T16:12:13.395367+0000","last_active":"2024-04-07T16:13:38.637813+0000","last_peered":"2024-04-07T16:13:38.637813+0000","last_clean":"2024-04-07T16:13:38.637813+0000","last_became_active":"2024-04-07T16:12:13.395023+0000","last_became_peered":"2024-04-07T16:12:13.395023+0000","last_unstale":"2024-04-07T16:13:38.637813+0000","last_undegraded":"2024-04-07T16:13:38.637813+0000","last_fullsized":"2024-04-07T16:13:38.637813+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T23:05:09.752696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.237070+0000","last_change":"2024-04-07T16:12:13.385549+0000","last_active":"2024-04-07T16:13:38.237070+0000","last_peered":"2024-04-07T16:13:38.237070+0000","last_clean":"2024-04-07T16:13:38.237070+0000","last_became_active":"2024-04-07T16:12:13.385175+0000","last_became_peered":"2024-04-07T16:12:13.385175+0000","last_unstale":"2024-04-07T16:13:38.237070+0000","last_undegraded":"2024-04-07T16:13:38.237070+0000","last_fullsized":"2024-04-07T16:13:38.237070+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T16:28:22.788092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,6],"acting":[1,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.030532+0000","last_change":"2024-04-07T16:12:13.410431+0000","last_active":"2024-04-07T16:13:38.030532+0000","last_peered":"2024-04-07T16:13:38.030532+0000","last_clean":"2024-04-07T16:13:38.030532+0000","last_became_active":"2024-04-07T16:12:13.410088+0000","last_became_peered":"2024-04-07T16:12:13.410088+0000","last_unstale":"2024-04-07T16:13:38.030532+0000","last_undegraded":"2024-04-07T16:13:38.030532+0000","last_fullsized":"2024-04-07T16:13:38.030532+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T19:47:57.716144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.025110+0000","last_change":"2024-04-07T16:12:13.402328+0000","last_active":"2024-04-07T16:13:38.025110+0000","last_peered":"2024-04-07T16:13:38.025110+0000","last_clean":"2024-04-07T16:13:38.025110+0000","last_became_active":"2024-04-07T16:12:13.401819+0000","last_became_peered":"2024-04-07T16:12:13.401819+0000","last_unstale":"2024-04-07T16:13:38.025110+0000","last_undegraded":"2024-04-07T16:13:38.025110+0000","last_fullsized":"2024-04-07T16:13:38.025110+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T21:43:13.292859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.637270+0000","last_change":"2024-04-07T16:12:13.413487+0000","last_active":"2024-04-07T16:13:38.637270+0000","last_peered":"2024-04-07T16:13:38.637270+0000","last_clean":"2024-04-07T16:13:38.637270+0000","last_became_active":"2024-04-07T16:12:13.413281+0000","last_became_peered":"2024-04-07T16:12:13.413281+0000","last_unstale":"2024-04-07T16:13:38.637270+0000","last_undegraded":"2024-04-07T16:13:38.637270+0000","last_fullsized":"2024-04-07T16:13:38.637270+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T23:59:23.302803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.024848+0000","last_change":"2024-04-07T16:12:13.401561+0000","last_active":"2024-04-07T16:13:38.024848+0000","last_peered":"2024-04-07T16:13:38.024848+0000","last_clean":"2024-04-07T16:13:38.024848+0000","last_became_active":"2024-04-07T16:12:13.400365+0000","last_became_peered":"2024-04-07T16:12:13.400365+0000","last_unstale":"2024-04-07T16:13:38.024848+0000","last_undegraded":"2024-04-07T16:13:38.024848+0000","last_fullsized":"2024-04-07T16:13:38.024848+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T20:14:49.110462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.636229+0000","last_change":"2024-04-07T16:12:13.393518+0000","last_active":"2024-04-07T16:13:38.636229+0000","last_peered":"2024-04-07T16:13:38.636229+0000","last_clean":"2024-04-07T16:13:38.636229+0000","last_became_active":"2024-04-07T16:12:13.393187+0000","last_became_peered":"2024-04-07T16:12:13.393187+0000","last_unstale":"2024-04-07T16:13:38.636229+0000","last_undegraded":"2024-04-07T16:13:38.636229+0000","last_fullsized":"2024-04-07T16:13:38.636229+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T19:41:28.702905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.187095+0000","last_change":"2024-04-07T16:12:13.394702+0000","last_active":"2024-04-07T16:13:38.187095+0000","last_peered":"2024-04-07T16:13:38.187095+0000","last_clean":"2024-04-07T16:13:38.187095+0000","last_became_active":"2024-04-07T16:12:13.394422+0000","last_became_peered":"2024-04-07T16:12:13.394422+0000","last_unstale":"2024-04-07T16:13:38.187095+0000","last_undegraded":"2024-04-07T16:13:38.187095+0000","last_fullsized":"2024-04-07T16:13:38.187095+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T00:37:44.512471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.637842+0000","last_change":"2024-04-07T16:12:13.401766+0000","last_active":"2024-04-07T16:13:38.637842+0000","last_peered":"2024-04-07T16:13:38.637842+0000","last_clean":"2024-04-07T16:13:38.637842+0000","last_became_active":"2024-04-07T16:12:13.401411+0000","last_became_peered":"2024-04-07T16:12:13.401411+0000","last_unstale":"2024-04-07T16:13:38.637842+0000","last_undegraded":"2024-04-07T16:13:38.637842+0000","last_fullsized":"2024-04-07T16:13:38.637842+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T23:32:52.283410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,0],"acting":[7,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.553143+0000","last_change":"2024-04-07T16:12:13.396409+0000","last_active":"2024-04-07T16:13:38.553143+0000","last_peered":"2024-04-07T16:13:38.553143+0000","last_clean":"2024-04-07T16:13:38.553143+0000","last_became_active":"2024-04-07T16:12:13.396094+0000","last_became_peered":"2024-04-07T16:12:13.396094+0000","last_unstale":"2024-04-07T16:13:38.553143+0000","last_undegraded":"2024-04-07T16:13:38.553143+0000","last_fullsized":"2024-04-07T16:13:38.553143+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T18:54:36.072837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.552867+0000","last_change":"2024-04-07T16:12:13.401973+0000","last_active":"2024-04-07T16:13:38.552867+0000","last_peered":"2024-04-07T16:13:38.552867+0000","last_clean":"2024-04-07T16:13:38.552867+0000","last_became_active":"2024-04-07T16:12:13.401724+0000","last_became_peered":"2024-04-07T16:12:13.401724+0000","last_unstale":"2024-04-07T16:13:38.552867+0000","last_undegraded":"2024-04-07T16:13:38.552867+0000","last_fullsized":"2024-04-07T16:13:38.552867+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T21:26:29.208223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,0],"acting":[6,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:39.824604+0000","last_change":"2024-04-07T16:12:13.404858+0000","last_active":"2024-04-07T16:13:39.824604+0000","last_peered":"2024-04-07T16:13:39.824604+0000","last_clean":"2024-04-07T16:13:39.824604+0000","last_became_active":"2024-04-07T16:12:13.404529+0000","last_became_peered":"2024-04-07T16:12:13.404529+0000","last_unstale":"2024-04-07T16:13:39.824604+0000","last_undegraded":"2024-04-07T16:13:39.824604+0000","last_fullsized":"2024-04-07T16:13:39.824604+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T19:57:22.893626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,0],"acting":[4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.186817+0000","last_change":"2024-04-07T16:12:13.394760+0000","last_active":"2024-04-07T16:13:38.186817+0000","last_peered":"2024-04-07T16:13:38.186817+0000","last_clean":"2024-04-07T16:13:38.186817+0000","last_became_active":"2024-04-07T16:12:13.394402+0000","last_became_peered":"2024-04-07T16:12:13.394402+0000","last_unstale":"2024-04-07T16:13:38.186817+0000","last_undegraded":"2024-04-07T16:13:38.186817+0000","last_fullsized":"2024-04-07T16:13:38.186817+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T18:47:13.891972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.029316+0000","last_change":"2024-04-07T16:12:13.400292+0000","last_active":"2024-04-07T16:13:38.029316+0000","last_peered":"2024-04-07T16:13:38.029316+0000","last_clean":"2024-04-07T16:13:38.029316+0000","last_became_active":"2024-04-07T16:12:13.400028+0000","last_became_peered":"2024-04-07T16:12:13.400028+0000","last_unstale":"2024-04-07T16:13:38.029316+0000","last_undegraded":"2024-04-07T16:13:38.029316+0000","last_fullsized":"2024-04-07T16:13:38.029316+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T18:14:33.693852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.235899+0000","last_change":"2024-04-07T16:12:13.398535+0000","last_active":"2024-04-07T16:13:38.235899+0000","last_peered":"2024-04-07T16:13:38.235899+0000","last_clean":"2024-04-07T16:13:38.235899+0000","last_became_active":"2024-04-07T16:12:13.398287+0000","last_became_peered":"2024-04-07T16:12:13.398287+0000","last_unstale":"2024-04-07T16:13:38.235899+0000","last_undegraded":"2024-04-07T16:13:38.235899+0000","last_fullsized":"2024-04-07T16:13:38.235899+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T23:58:53.081885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,5],"acting":[1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.023905+0000","last_change":"2024-04-07T16:12:13.395243+0000","last_active":"2024-04-07T16:13:38.023905+0000","last_peered":"2024-04-07T16:13:38.023905+0000","last_clean":"2024-04-07T16:13:38.023905+0000","last_became_active":"2024-04-07T16:12:13.394956+0000","last_became_peered":"2024-04-07T16:12:13.394956+0000","last_unstale":"2024-04-07T16:13:38.023905+0000","last_undegraded":"2024-04-07T16:13:38.023905+0000","last_fullsized":"2024-04-07T16:13:38.023905+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T03:04:58.804429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.026420+0000","last_change":"2024-04-07T16:12:13.398543+0000","last_active":"2024-04-07T16:13:38.026420+0000","last_peered":"2024-04-07T16:13:38.026420+0000","last_clean":"2024-04-07T16:13:38.026420+0000","last_became_active":"2024-04-07T16:12:13.398198+0000","last_became_peered":"2024-04-07T16:12:13.398198+0000","last_unstale":"2024-04-07T16:13:38.026420+0000","last_undegraded":"2024-04-07T16:13:38.026420+0000","last_fullsized":"2024-04-07T16:13:38.026420+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T22:57:24.895712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,4],"acting":[5,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.764814+0000","last_change":"2024-04-07T16:12:13.416210+0000","last_active":"2024-04-07T16:13:38.764814+0000","last_peered":"2024-04-07T16:13:38.764814+0000","last_clean":"2024-04-07T16:13:38.764814+0000","last_became_active":"2024-04-07T16:12:13.415972+0000","last_became_peered":"2024-04-07T16:12:13.415972+0000","last_unstale":"2024-04-07T16:13:38.764814+0000","last_undegraded":"2024-04-07T16:13:38.764814+0000","last_fullsized":"2024-04-07T16:13:38.764814+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T23:40:11.606978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,7],"acting":[2,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.636945+0000","last_change":"2024-04-07T16:12:13.414203+0000","last_active":"2024-04-07T16:13:38.636945+0000","last_peered":"2024-04-07T16:13:38.636945+0000","last_clean":"2024-04-07T16:13:38.636945+0000","last_became_active":"2024-04-07T16:12:13.413869+0000","last_became_peered":"2024-04-07T16:12:13.413869+0000","last_unstale":"2024-04-07T16:13:38.636945+0000","last_undegraded":"2024-04-07T16:13:38.636945+0000","last_fullsized":"2024-04-07T16:13:38.636945+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T17:11:47.895568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,6],"acting":[7,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1f","version":"61'4","reported_seq":24,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.031549+0000","last_change":"2024-04-07T16:12:13.032601+0000","last_active":"2024-04-07T16:13:38.031549+0000","last_peered":"2024-04-07T16:13:38.031549+0000","last_clean":"2024-04-07T16:13:38.031549+0000","last_became_active":"2024-04-07T16:12:13.032199+0000","last_became_peered":"2024-04-07T16:12:13.032199+0000","last_unstale":"2024-04-07T16:13:38.031549+0000","last_undegraded":"2024-04-07T16:13:38.031549+0000","last_fullsized":"2024-04-07T16:13:38.031549+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T19:01:44.798719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.187256+0000","last_change":"2024-04-07T16:12:13.407852+0000","last_active":"2024-04-07T16:13:38.187256+0000","last_peered":"2024-04-07T16:13:38.187256+0000","last_clean":"2024-04-07T16:13:38.187256+0000","last_became_active":"2024-04-07T16:12:13.407538+0000","last_became_peered":"2024-04-07T16:12:13.407538+0000","last_unstale":"2024-04-07T16:13:38.187256+0000","last_undegraded":"2024-04-07T16:13:38.187256+0000","last_fullsized":"2024-04-07T16:13:38.187256+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T20:27:52.664149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"61'1","reported_seq":21,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.187484+0000","last_change":"2024-04-07T16:12:13.038807+0000","last_active":"2024-04-07T16:13:38.187484+0000","last_peered":"2024-04-07T16:13:38.187484+0000","last_clean":"2024-04-07T16:13:38.187484+0000","last_became_active":"2024-04-07T16:12:13.038492+0000","last_became_peered":"2024-04-07T16:12:13.038492+0000","last_unstale":"2024-04-07T16:13:38.187484+0000","last_undegraded":"2024-04-07T16:13:38.187484+0000","last_fullsized":"2024-04-07T16:13:38.187484+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T02:11:03.746017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.031268+0000","last_change":"2024-04-07T16:12:13.410031+0000","last_active":"2024-04-07T16:13:38.031268+0000","last_peered":"2024-04-07T16:13:38.031268+0000","last_clean":"2024-04-07T16:13:38.031268+0000","last_became_active":"2024-04-07T16:12:13.409049+0000","last_became_peered":"2024-04-07T16:12:13.409049+0000","last_unstale":"2024-04-07T16:13:38.031268+0000","last_undegraded":"2024-04-07T16:13:38.031268+0000","last_fullsized":"2024-04-07T16:13:38.031268+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T21:35:09.308017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"61'2","reported_seq":22,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.637577+0000","last_change":"2024-04-07T16:12:13.042401+0000","last_active":"2024-04-07T16:13:38.637577+0000","last_peered":"2024-04-07T16:13:38.637577+0000","last_clean":"2024-04-07T16:13:38.637577+0000","last_became_active":"2024-04-07T16:12:13.041970+0000","last_became_peered":"2024-04-07T16:12:13.041970+0000","last_unstale":"2024-04-07T16:13:38.637577+0000","last_undegraded":"2024-04-07T16:13:38.637577+0000","last_fullsized":"2024-04-07T16:13:38.637577+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T16:36:29.235347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.025650+0000","last_change":"2024-04-07T16:12:13.394718+0000","last_active":"2024-04-07T16:13:38.025650+0000","last_peered":"2024-04-07T16:13:38.025650+0000","last_clean":"2024-04-07T16:13:38.025650+0000","last_became_active":"2024-04-07T16:12:13.394271+0000","last_became_peered":"2024-04-07T16:12:13.394271+0000","last_unstale":"2024-04-07T16:13:38.025650+0000","last_undegraded":"2024-04-07T16:13:38.025650+0000","last_fullsized":"2024-04-07T16:13:38.025650+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T20:10:29.096403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"61'2","reported_seq":22,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.637065+0000","last_change":"2024-04-07T16:12:13.042391+0000","last_active":"2024-04-07T16:13:38.637065+0000","last_peered":"2024-04-07T16:13:38.637065+0000","last_clean":"2024-04-07T16:13:38.637065+0000","last_became_active":"2024-04-07T16:12:13.041919+0000","last_became_peered":"2024-04-07T16:12:13.041919+0000","last_unstale":"2024-04-07T16:13:38.637065+0000","last_undegraded":"2024-04-07T16:13:38.637065+0000","last_fullsized":"2024-04-07T16:13:38.637065+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T21:50:59.576615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":28,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.025658+0000","last_change":"2024-04-07T16:12:26.222601+0000","last_active":"2024-04-07T16:13:38.025658+0000","last_peered":"2024-04-07T16:13:38.025658+0000","last_clean":"2024-04-07T16:13:38.025658+0000","last_became_active":"2024-04-07T16:12:26.222249+0000","last_became_peered":"2024-04-07T16:12:26.222249+0000","last_unstale":"2024-04-07T16:13:38.025658+0000","last_undegraded":"2024-04-07T16:13:38.025658+0000","last_fullsized":"2024-04-07T16:13:38.025658+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T16:28:47.005643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"61'1","reported_seq":21,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.187224+0000","last_change":"2024-04-07T16:12:13.041880+0000","last_active":"2024-04-07T16:13:38.187224+0000","last_peered":"2024-04-07T16:13:38.187224+0000","last_clean":"2024-04-07T16:13:38.187224+0000","last_became_active":"2024-04-07T16:12:13.041409+0000","last_became_peered":"2024-04-07T16:12:13.041409+0000","last_unstale":"2024-04-07T16:13:38.187224+0000","last_undegraded":"2024-04-07T16:13:38.187224+0000","last_fullsized":"2024-04-07T16:13:38.187224+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T18:32:23.206985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:39.824041+0000","last_change":"2024-04-07T16:12:13.413664+0000","last_active":"2024-04-07T16:13:39.824041+0000","last_peered":"2024-04-07T16:13:39.824041+0000","last_clean":"2024-04-07T16:13:39.824041+0000","last_became_active":"2024-04-07T16:12:13.413311+0000","last_became_peered":"2024-04-07T16:12:13.413311+0000","last_unstale":"2024-04-07T16:13:39.824041+0000","last_undegraded":"2024-04-07T16:13:39.824041+0000","last_fullsized":"2024-04-07T16:13:39.824041+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T02:43:21.836307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"61'1","reported_seq":21,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.526629+0000","last_change":"2024-04-07T16:12:13.038960+0000","last_active":"2024-04-07T16:13:38.526629+0000","last_peered":"2024-04-07T16:13:38.526629+0000","last_clean":"2024-04-07T16:13:38.526629+0000","last_became_active":"2024-04-07T16:12:13.038410+0000","last_became_peered":"2024-04-07T16:12:13.038410+0000","last_unstale":"2024-04-07T16:13:38.526629+0000","last_undegraded":"2024-04-07T16:13:38.526629+0000","last_fullsized":"2024-04-07T16:13:38.526629+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T03:41:38.418312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.029545+0000","last_change":"2024-04-07T16:12:13.405634+0000","last_active":"2024-04-07T16:13:38.029545+0000","last_peered":"2024-04-07T16:13:38.029545+0000","last_clean":"2024-04-07T16:13:38.029545+0000","last_became_active":"2024-04-07T16:12:13.405069+0000","last_became_peered":"2024-04-07T16:12:13.405069+0000","last_unstale":"2024-04-07T16:13:38.029545+0000","last_undegraded":"2024-04-07T16:13:38.029545+0000","last_fullsized":"2024-04-07T16:13:38.029545+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T23:50:18.973629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":20,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.186860+0000","last_change":"2024-04-07T16:12:13.034420+0000","last_active":"2024-04-07T16:13:38.186860+0000","last_peered":"2024-04-07T16:13:38.186860+0000","last_clean":"2024-04-07T16:13:38.186860+0000","last_became_active":"2024-04-07T16:12:13.034013+0000","last_became_peered":"2024-04-07T16:12:13.034013+0000","last_unstale":"2024-04-07T16:13:38.186860+0000","last_undegraded":"2024-04-07T16:13:38.186860+0000","last_fullsized":"2024-04-07T16:13:38.186860+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T19:51:21.525865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.187269+0000","last_change":"2024-04-07T16:12:13.398531+0000","last_active":"2024-04-07T16:13:38.187269+0000","last_peered":"2024-04-07T16:13:38.187269+0000","last_clean":"2024-04-07T16:13:38.187269+0000","last_became_active":"2024-04-07T16:12:13.398017+0000","last_became_peered":"2024-04-07T16:12:13.398017+0000","last_unstale":"2024-04-07T16:13:38.187269+0000","last_undegraded":"2024-04-07T16:13:38.187269+0000","last_fullsized":"2024-04-07T16:13:38.187269+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T17:33:54.306902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":20,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.024099+0000","last_change":"2024-04-07T16:12:13.045443+0000","last_active":"2024-04-07T16:13:38.024099+0000","last_peered":"2024-04-07T16:13:38.024099+0000","last_clean":"2024-04-07T16:13:38.024099+0000","last_became_active":"2024-04-07T16:12:13.044808+0000","last_became_peered":"2024-04-07T16:12:13.044808+0000","last_unstale":"2024-04-07T16:13:38.024099+0000","last_undegraded":"2024-04-07T16:13:38.024099+0000","last_fullsized":"2024-04-07T16:13:38.024099+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T22:44:14.623116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.236151+0000","last_change":"2024-04-07T16:12:13.391166+0000","last_active":"2024-04-07T16:13:38.236151+0000","last_peered":"2024-04-07T16:13:38.236151+0000","last_clean":"2024-04-07T16:13:38.236151+0000","last_became_active":"2024-04-07T16:12:13.390643+0000","last_became_peered":"2024-04-07T16:12:13.390643+0000","last_unstale":"2024-04-07T16:13:38.236151+0000","last_undegraded":"2024-04-07T16:13:38.236151+0000","last_fullsized":"2024-04-07T16:13:38.236151+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T21:22:33.110408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":20,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.553074+0000","last_change":"2024-04-07T16:12:13.030149+0000","last_active":"2024-04-07T16:13:38.553074+0000","last_peered":"2024-04-07T16:13:38.553074+0000","last_clean":"2024-04-07T16:13:38.553074+0000","last_became_active":"2024-04-07T16:12:13.029525+0000","last_became_peered":"2024-04-07T16:12:13.029525+0000","last_unstale":"2024-04-07T16:13:38.553074+0000","last_undegraded":"2024-04-07T16:13:38.553074+0000","last_fullsized":"2024-04-07T16:13:38.553074+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T20:09:35.180264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.025897+0000","last_change":"2024-04-07T16:12:13.398411+0000","last_active":"2024-04-07T16:13:38.025897+0000","last_peered":"2024-04-07T16:13:38.025897+0000","last_clean":"2024-04-07T16:13:38.025897+0000","last_became_active":"2024-04-07T16:12:13.398065+0000","last_became_peered":"2024-04-07T16:12:13.398065+0000","last_unstale":"2024-04-07T16:13:38.025897+0000","last_undegraded":"2024-04-07T16:13:38.025897+0000","last_fullsized":"2024-04-07T16:13:38.025897+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T21:13:20.448127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":20,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.025656+0000","last_change":"2024-04-07T16:12:13.036777+0000","last_active":"2024-04-07T16:13:38.025656+0000","last_peered":"2024-04-07T16:13:38.025656+0000","last_clean":"2024-04-07T16:13:38.025656+0000","last_became_active":"2024-04-07T16:12:13.036322+0000","last_became_peered":"2024-04-07T16:12:13.036322+0000","last_unstale":"2024-04-07T16:13:38.025656+0000","last_undegraded":"2024-04-07T16:13:38.025656+0000","last_fullsized":"2024-04-07T16:13:38.025656+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T18:26:16.873044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":16,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-07T16:13:31.435811+0000","last_change":"2024-04-07T16:12:26.226313+0000","last_active":"2024-04-07T16:13:31.435811+0000","last_peered":"2024-04-07T16:13:31.435811+0000","last_clean":"2024-04-07T16:13:31.435811+0000","last_became_active":"2024-04-07T16:12:26.225343+0000","last_became_peered":"2024-04-07T16:12:26.225343+0000","last_unstale":"2024-04-07T16:13:31.435811+0000","last_undegraded":"2024-04-07T16:13:31.435811+0000","last_fullsized":"2024-04-07T16:13:31.435811+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T17:42:38.440812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3],"acting":[2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.15","version":"61'3","reported_seq":23,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.237334+0000","last_change":"2024-04-07T16:12:13.039273+0000","last_active":"2024-04-07T16:13:38.237334+0000","last_peered":"2024-04-07T16:13:38.237334+0000","last_clean":"2024-04-07T16:13:38.237334+0000","last_became_active":"2024-04-07T16:12:13.038911+0000","last_became_peered":"2024-04-07T16:12:13.038911+0000","last_unstale":"2024-04-07T16:13:38.237334+0000","last_undegraded":"2024-04-07T16:13:38.237334+0000","last_fullsized":"2024-04-07T16:13:38.237334+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T02:11:53.262707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:39.824449+0000","last_change":"2024-04-07T16:12:13.391349+0000","last_active":"2024-04-07T16:13:39.824449+0000","last_peered":"2024-04-07T16:13:39.824449+0000","last_clean":"2024-04-07T16:13:39.824449+0000","last_became_active":"2024-04-07T16:12:13.390960+0000","last_became_peered":"2024-04-07T16:12:13.390960+0000","last_unstale":"2024-04-07T16:13:39.824449+0000","last_undegraded":"2024-04-07T16:13:39.824449+0000","last_fullsized":"2024-04-07T16:13:39.824449+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T16:55:37.429799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"69'12","reported_seq":32,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:53.034681+0000","last_change":"2024-04-07T16:12:13.031673+0000","last_active":"2024-04-07T16:13:53.034681+0000","last_peered":"2024-04-07T16:13:53.034681+0000","last_clean":"2024-04-07T16:13:53.034681+0000","last_became_active":"2024-04-07T16:12:13.031092+0000","last_became_peered":"2024-04-07T16:12:13.031092+0000","last_unstale":"2024-04-07T16:13:53.034681+0000","last_undegraded":"2024-04-07T16:13:53.034681+0000","last_fullsized":"2024-04-07T16:13:53.034681+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T20:17:48.593862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2632,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":28,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.635967+0000","last_change":"2024-04-07T16:12:26.223406+0000","last_active":"2024-04-07T16:13:38.635967+0000","last_peered":"2024-04-07T16:13:38.635967+0000","last_clean":"2024-04-07T16:13:38.635967+0000","last_became_active":"2024-04-07T16:12:26.222663+0000","last_became_peered":"2024-04-07T16:12:26.222663+0000","last_unstale":"2024-04-07T16:13:38.635967+0000","last_undegraded":"2024-04-07T16:13:38.635967+0000","last_fullsized":"2024-04-07T16:13:38.635967+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T19:54:33.089377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"61'2","reported_seq":22,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.029978+0000","last_change":"2024-04-07T16:12:13.033352+0000","last_active":"2024-04-07T16:13:38.029978+0000","last_peered":"2024-04-07T16:13:38.029978+0000","last_clean":"2024-04-07T16:13:38.029978+0000","last_became_active":"2024-04-07T16:12:13.032811+0000","last_became_peered":"2024-04-07T16:12:13.032811+0000","last_unstale":"2024-04-07T16:13:38.029978+0000","last_undegraded":"2024-04-07T16:13:38.029978+0000","last_fullsized":"2024-04-07T16:13:38.029978+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T01:50:02.195507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.763887+0000","last_change":"2024-04-07T16:12:26.226163+0000","last_active":"2024-04-07T16:13:38.763887+0000","last_peered":"2024-04-07T16:13:38.763887+0000","last_clean":"2024-04-07T16:13:38.763887+0000","last_became_active":"2024-04-07T16:12:26.224225+0000","last_became_peered":"2024-04-07T16:12:26.224225+0000","last_unstale":"2024-04-07T16:13:38.763887+0000","last_undegraded":"2024-04-07T16:13:38.763887+0000","last_fullsized":"2024-04-07T16:13:38.763887+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T23:54:31.223096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,3],"acting":[2,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":20,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.024421+0000","last_change":"2024-04-07T16:12:13.045494+0000","last_active":"2024-04-07T16:13:38.024421+0000","last_peered":"2024-04-07T16:13:38.024421+0000","last_clean":"2024-04-07T16:13:38.024421+0000","last_became_active":"2024-04-07T16:12:13.045086+0000","last_became_peered":"2024-04-07T16:12:13.045086+0000","last_unstale":"2024-04-07T16:13:38.024421+0000","last_undegraded":"2024-04-07T16:13:38.024421+0000","last_fullsized":"2024-04-07T16:13:38.024421+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T23:15:11.494437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.637043+0000","last_change":"2024-04-07T16:12:13.413928+0000","last_active":"2024-04-07T16:13:38.637043+0000","last_peered":"2024-04-07T16:13:38.637043+0000","last_clean":"2024-04-07T16:13:38.637043+0000","last_became_active":"2024-04-07T16:12:13.413447+0000","last_became_peered":"2024-04-07T16:12:13.413447+0000","last_unstale":"2024-04-07T16:13:38.637043+0000","last_undegraded":"2024-04-07T16:13:38.637043+0000","last_fullsized":"2024-04-07T16:13:38.637043+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T03:26:42.875228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":20,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.526216+0000","last_change":"2024-04-07T16:12:13.032942+0000","last_active":"2024-04-07T16:13:38.526216+0000","last_peered":"2024-04-07T16:13:38.526216+0000","last_clean":"2024-04-07T16:13:38.526216+0000","last_became_active":"2024-04-07T16:12:13.032108+0000","last_became_peered":"2024-04-07T16:12:13.032108+0000","last_unstale":"2024-04-07T16:13:38.526216+0000","last_undegraded":"2024-04-07T16:13:38.526216+0000","last_fullsized":"2024-04-07T16:13:38.526216+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T18:53:06.864412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.525876+0000","last_change":"2024-04-07T16:12:13.400141+0000","last_active":"2024-04-07T16:13:38.525876+0000","last_peered":"2024-04-07T16:13:38.525876+0000","last_clean":"2024-04-07T16:13:38.525876+0000","last_became_active":"2024-04-07T16:12:13.399823+0000","last_became_peered":"2024-04-07T16:12:13.399823+0000","last_unstale":"2024-04-07T16:13:38.525876+0000","last_undegraded":"2024-04-07T16:13:38.525876+0000","last_fullsized":"2024-04-07T16:13:38.525876+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T21:27:25.248595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"61'8","reported_seq":28,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.764170+0000","last_change":"2024-04-07T16:12:13.034154+0000","last_active":"2024-04-07T16:13:38.764170+0000","last_peered":"2024-04-07T16:13:38.764170+0000","last_clean":"2024-04-07T16:13:38.764170+0000","last_became_active":"2024-04-07T16:12:13.033838+0000","last_became_peered":"2024-04-07T16:12:13.033838+0000","last_unstale":"2024-04-07T16:13:38.764170+0000","last_undegraded":"2024-04-07T16:13:38.764170+0000","last_fullsized":"2024-04-07T16:13:38.764170+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T23:13:07.216314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.636593+0000","last_change":"2024-04-07T16:12:13.385883+0000","last_active":"2024-04-07T16:13:38.636593+0000","last_peered":"2024-04-07T16:13:38.636593+0000","last_clean":"2024-04-07T16:13:38.636593+0000","last_became_active":"2024-04-07T16:12:13.385485+0000","last_became_peered":"2024-04-07T16:12:13.385485+0000","last_unstale":"2024-04-07T16:13:38.636593+0000","last_undegraded":"2024-04-07T16:13:38.636593+0000","last_fullsized":"2024-04-07T16:13:38.636593+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T21:01:09.262433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"61'1","reported_seq":21,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:39.825314+0000","last_change":"2024-04-07T16:12:13.028004+0000","last_active":"2024-04-07T16:13:39.825314+0000","last_peered":"2024-04-07T16:13:39.825314+0000","last_clean":"2024-04-07T16:13:39.825314+0000","last_became_active":"2024-04-07T16:12:13.027445+0000","last_became_peered":"2024-04-07T16:12:13.027445+0000","last_unstale":"2024-04-07T16:13:39.825314+0000","last_undegraded":"2024-04-07T16:13:39.825314+0000","last_fullsized":"2024-04-07T16:13:39.825314+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T17:55:06.727142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.637723+0000","last_change":"2024-04-07T16:12:13.386584+0000","last_active":"2024-04-07T16:13:38.637723+0000","last_peered":"2024-04-07T16:13:38.637723+0000","last_clean":"2024-04-07T16:13:38.637723+0000","last_became_active":"2024-04-07T16:12:13.386057+0000","last_became_peered":"2024-04-07T16:12:13.386057+0000","last_unstale":"2024-04-07T16:13:38.637723+0000","last_undegraded":"2024-04-07T16:13:38.637723+0000","last_fullsized":"2024-04-07T16:13:38.637723+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T17:57:57.548179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-07T16:13:33.673641+0000","last_change":"2024-04-07T16:13:30.009561+0000","last_active":"2024-04-07T16:13:33.673641+0000","last_peered":"2024-04-07T16:13:33.673641+0000","last_clean":"2024-04-07T16:13:33.673641+0000","last_became_active":"2024-04-07T16:13:30.008706+0000","last_became_peered":"2024-04-07T16:13:30.008706+0000","last_unstale":"2024-04-07T16:13:33.673641+0000","last_undegraded":"2024-04-07T16:13:33.673641+0000","last_fullsized":"2024-04-07T16:13:33.673641+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:13:28.963534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:13:28.963534+0000","last_clean_scrub_stamp":"2024-04-07T16:13:28.963534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T00:36:26.958742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":20,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.236743+0000","last_change":"2024-04-07T16:12:13.037974+0000","last_active":"2024-04-07T16:13:38.236743+0000","last_peered":"2024-04-07T16:13:38.236743+0000","last_clean":"2024-04-07T16:13:38.236743+0000","last_became_active":"2024-04-07T16:12:13.037627+0000","last_became_peered":"2024-04-07T16:12:13.037627+0000","last_unstale":"2024-04-07T16:13:38.236743+0000","last_undegraded":"2024-04-07T16:13:38.236743+0000","last_fullsized":"2024-04-07T16:13:38.236743+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T19:28:12.174861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.025595+0000","last_change":"2024-04-07T16:12:13.395073+0000","last_active":"2024-04-07T16:13:38.025595+0000","last_peered":"2024-04-07T16:13:38.025595+0000","last_clean":"2024-04-07T16:13:38.025595+0000","last_became_active":"2024-04-07T16:12:13.394113+0000","last_became_peered":"2024-04-07T16:12:13.394113+0000","last_unstale":"2024-04-07T16:13:38.025595+0000","last_undegraded":"2024-04-07T16:13:38.025595+0000","last_fullsized":"2024-04-07T16:13:38.025595+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T23:28:32.978580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-07T16:13:33.673705+0000","last_change":"2024-04-07T16:13:30.041147+0000","last_active":"2024-04-07T16:13:33.673705+0000","last_peered":"2024-04-07T16:13:33.673705+0000","last_clean":"2024-04-07T16:13:33.673705+0000","last_became_active":"2024-04-07T16:13:30.040318+0000","last_became_peered":"2024-04-07T16:13:30.040318+0000","last_unstale":"2024-04-07T16:13:33.673705+0000","last_undegraded":"2024-04-07T16:13:33.673705+0000","last_fullsized":"2024-04-07T16:13:33.673705+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:13:28.963534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:13:28.963534+0000","last_clean_scrub_stamp":"2024-04-07T16:13:28.963534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T20:45:42.413572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":20,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.024311+0000","last_change":"2024-04-07T16:12:13.036336+0000","last_active":"2024-04-07T16:13:38.024311+0000","last_peered":"2024-04-07T16:13:38.024311+0000","last_clean":"2024-04-07T16:13:38.024311+0000","last_became_active":"2024-04-07T16:12:13.035625+0000","last_became_peered":"2024-04-07T16:12:13.035625+0000","last_unstale":"2024-04-07T16:13:38.024311+0000","last_undegraded":"2024-04-07T16:13:38.024311+0000","last_fullsized":"2024-04-07T16:13:38.024311+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T18:15:00.506827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:39.824462+0000","last_change":"2024-04-07T16:12:13.404406+0000","last_active":"2024-04-07T16:13:39.824462+0000","last_peered":"2024-04-07T16:13:39.824462+0000","last_clean":"2024-04-07T16:13:39.824462+0000","last_became_active":"2024-04-07T16:12:13.404029+0000","last_became_peered":"2024-04-07T16:12:13.404029+0000","last_unstale":"2024-04-07T16:13:39.824462+0000","last_undegraded":"2024-04-07T16:13:39.824462+0000","last_fullsized":"2024-04-07T16:13:39.824462+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T16:41:25.469343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-07T16:13:33.673820+0000","last_change":"2024-04-07T16:13:30.012072+0000","last_active":"2024-04-07T16:13:33.673820+0000","last_peered":"2024-04-07T16:13:33.673820+0000","last_clean":"2024-04-07T16:13:33.673820+0000","last_became_active":"2024-04-07T16:13:30.011181+0000","last_became_peered":"2024-04-07T16:13:30.011181+0000","last_unstale":"2024-04-07T16:13:33.673820+0000","last_undegraded":"2024-04-07T16:13:33.673820+0000","last_fullsized":"2024-04-07T16:13:33.673820+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:13:28.963534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:13:28.963534+0000","last_clean_scrub_stamp":"2024-04-07T16:13:28.963534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T21:19:09.530105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.235831+0000","last_change":"2024-04-07T16:12:13.393979+0000","last_active":"2024-04-07T16:13:38.235831+0000","last_peered":"2024-04-07T16:13:38.235831+0000","last_clean":"2024-04-07T16:13:38.235831+0000","last_became_active":"2024-04-07T16:12:13.393472+0000","last_became_peered":"2024-04-07T16:12:13.393472+0000","last_unstale":"2024-04-07T16:13:38.235831+0000","last_undegraded":"2024-04-07T16:13:38.235831+0000","last_fullsized":"2024-04-07T16:13:38.235831+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T21:51:13.834107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":20,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.764382+0000","last_change":"2024-04-07T16:12:13.035071+0000","last_active":"2024-04-07T16:13:38.764382+0000","last_peered":"2024-04-07T16:13:38.764382+0000","last_clean":"2024-04-07T16:13:38.764382+0000","last_became_active":"2024-04-07T16:12:13.034795+0000","last_became_peered":"2024-04-07T16:12:13.034795+0000","last_unstale":"2024-04-07T16:13:38.764382+0000","last_undegraded":"2024-04-07T16:13:38.764382+0000","last_fullsized":"2024-04-07T16:13:38.764382+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T17:51:30.599527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-07T16:13:33.674725+0000","last_change":"2024-04-07T16:13:30.035513+0000","last_active":"2024-04-07T16:13:33.674725+0000","last_peered":"2024-04-07T16:13:33.674725+0000","last_clean":"2024-04-07T16:13:33.674725+0000","last_became_active":"2024-04-07T16:13:30.035058+0000","last_became_peered":"2024-04-07T16:13:30.035058+0000","last_unstale":"2024-04-07T16:13:33.674725+0000","last_undegraded":"2024-04-07T16:13:33.674725+0000","last_fullsized":"2024-04-07T16:13:33.674725+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:13:28.963534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:13:28.963534+0000","last_clean_scrub_stamp":"2024-04-07T16:13:28.963534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T22:41:30.257805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.186788+0000","last_change":"2024-04-07T16:12:13.394635+0000","last_active":"2024-04-07T16:13:38.186788+0000","last_peered":"2024-04-07T16:13:38.186788+0000","last_clean":"2024-04-07T16:13:38.186788+0000","last_became_active":"2024-04-07T16:12:13.394392+0000","last_became_peered":"2024-04-07T16:12:13.394392+0000","last_unstale":"2024-04-07T16:13:38.186788+0000","last_undegraded":"2024-04-07T16:13:38.186788+0000","last_fullsized":"2024-04-07T16:13:38.186788+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T17:52:10.950481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":20,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.024445+0000","last_change":"2024-04-07T16:12:13.045131+0000","last_active":"2024-04-07T16:13:38.024445+0000","last_peered":"2024-04-07T16:13:38.024445+0000","last_clean":"2024-04-07T16:13:38.024445+0000","last_became_active":"2024-04-07T16:12:13.044830+0000","last_became_peered":"2024-04-07T16:12:13.044830+0000","last_unstale":"2024-04-07T16:13:38.024445+0000","last_undegraded":"2024-04-07T16:13:38.024445+0000","last_fullsized":"2024-04-07T16:13:38.024445+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T02:48:50.652491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"65'128","reported_seq":266,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.636317+0000","last_change":"2024-04-07T16:11:33.518868+0000","last_active":"2024-04-07T16:13:38.636317+0000","last_peered":"2024-04-07T16:13:38.636317+0000","last_clean":"2024-04-07T16:13:38.636317+0000","last_became_active":"2024-04-07T16:11:33.518060+0000","last_became_peered":"2024-04-07T16:11:33.518060+0000","last_unstale":"2024-04-07T16:13:38.636317+0000","last_undegraded":"2024-04-07T16:13:38.636317+0000","last_fullsized":"2024-04-07T16:13:38.636317+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:07:26.866003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:07:26.866003+0000","last_clean_scrub_stamp":"2024-04-07T16:07:26.866003+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T22:22:51.043941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-07T16:13:33.672600+0000","last_change":"2024-04-07T16:13:30.041000+0000","last_active":"2024-04-07T16:13:33.672600+0000","last_peered":"2024-04-07T16:13:33.672600+0000","last_clean":"2024-04-07T16:13:33.672600+0000","last_became_active":"2024-04-07T16:13:30.040114+0000","last_became_peered":"2024-04-07T16:13:30.040114+0000","last_unstale":"2024-04-07T16:13:33.672600+0000","last_undegraded":"2024-04-07T16:13:33.672600+0000","last_fullsized":"2024-04-07T16:13:33.672600+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:13:28.963534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:13:28.963534+0000","last_clean_scrub_stamp":"2024-04-07T16:13:28.963534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T03:03:13.375776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.763809+0000","last_change":"2024-04-07T16:12:26.226945+0000","last_active":"2024-04-07T16:13:38.763809+0000","last_peered":"2024-04-07T16:13:38.763809+0000","last_clean":"2024-04-07T16:13:38.763809+0000","last_became_active":"2024-04-07T16:12:26.225518+0000","last_became_peered":"2024-04-07T16:12:26.225518+0000","last_unstale":"2024-04-07T16:13:38.763809+0000","last_undegraded":"2024-04-07T16:13:38.763809+0000","last_fullsized":"2024-04-07T16:13:38.763809+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T22:14:39.830063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":20,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.636018+0000","last_change":"2024-04-07T16:12:13.035536+0000","last_active":"2024-04-07T16:13:38.636018+0000","last_peered":"2024-04-07T16:13:38.636018+0000","last_clean":"2024-04-07T16:13:38.636018+0000","last_became_active":"2024-04-07T16:12:13.034825+0000","last_became_peered":"2024-04-07T16:12:13.034825+0000","last_unstale":"2024-04-07T16:13:38.636018+0000","last_undegraded":"2024-04-07T16:13:38.636018+0000","last_fullsized":"2024-04-07T16:13:38.636018+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T20:11:57.033306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-07T16:13:33.672484+0000","last_change":"2024-04-07T16:13:30.041281+0000","last_active":"2024-04-07T16:13:33.672484+0000","last_peered":"2024-04-07T16:13:33.672484+0000","last_clean":"2024-04-07T16:13:33.672484+0000","last_became_active":"2024-04-07T16:13:30.040143+0000","last_became_peered":"2024-04-07T16:13:30.040143+0000","last_unstale":"2024-04-07T16:13:33.672484+0000","last_undegraded":"2024-04-07T16:13:33.672484+0000","last_fullsized":"2024-04-07T16:13:33.672484+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:13:28.963534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:13:28.963534+0000","last_clean_scrub_stamp":"2024-04-07T16:13:28.963534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T01:04:02.813753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":20,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.637358+0000","last_change":"2024-04-07T16:12:13.035747+0000","last_active":"2024-04-07T16:13:38.637358+0000","last_peered":"2024-04-07T16:13:38.637358+0000","last_clean":"2024-04-07T16:13:38.637358+0000","last_became_active":"2024-04-07T16:12:13.035201+0000","last_became_peered":"2024-04-07T16:12:13.035201+0000","last_unstale":"2024-04-07T16:13:38.637358+0000","last_undegraded":"2024-04-07T16:13:38.637358+0000","last_fullsized":"2024-04-07T16:13:38.637358+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T21:03:25.449728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.237019+0000","last_change":"2024-04-07T16:12:13.398785+0000","last_active":"2024-04-07T16:13:38.237019+0000","last_peered":"2024-04-07T16:13:38.237019+0000","last_clean":"2024-04-07T16:13:38.237019+0000","last_became_active":"2024-04-07T16:12:13.398210+0000","last_became_peered":"2024-04-07T16:12:13.398210+0000","last_unstale":"2024-04-07T16:13:38.237019+0000","last_undegraded":"2024-04-07T16:13:38.237019+0000","last_fullsized":"2024-04-07T16:13:38.237019+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T02:11:50.752075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-07T16:13:33.680261+0000","last_change":"2024-04-07T16:13:30.040851+0000","last_active":"2024-04-07T16:13:33.680261+0000","last_peered":"2024-04-07T16:13:33.680261+0000","last_clean":"2024-04-07T16:13:33.680261+0000","last_became_active":"2024-04-07T16:13:30.040026+0000","last_became_peered":"2024-04-07T16:13:30.040026+0000","last_unstale":"2024-04-07T16:13:33.680261+0000","last_undegraded":"2024-04-07T16:13:33.680261+0000","last_fullsized":"2024-04-07T16:13:33.680261+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:13:28.963534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:13:28.963534+0000","last_clean_scrub_stamp":"2024-04-07T16:13:28.963534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T18:29:22.422209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"62'1","reported_seq":21,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.237606+0000","last_change":"2024-04-07T16:12:13.039980+0000","last_active":"2024-04-07T16:13:38.237606+0000","last_peered":"2024-04-07T16:13:38.237606+0000","last_clean":"2024-04-07T16:13:38.237606+0000","last_became_active":"2024-04-07T16:12:13.039676+0000","last_became_peered":"2024-04-07T16:12:13.039676+0000","last_unstale":"2024-04-07T16:13:38.237606+0000","last_undegraded":"2024-04-07T16:13:38.237606+0000","last_fullsized":"2024-04-07T16:13:38.237606+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T02:37:05.297496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.187224+0000","last_change":"2024-04-07T16:12:13.399486+0000","last_active":"2024-04-07T16:13:38.187224+0000","last_peered":"2024-04-07T16:13:38.187224+0000","last_clean":"2024-04-07T16:13:38.187224+0000","last_became_active":"2024-04-07T16:12:13.399184+0000","last_became_peered":"2024-04-07T16:12:13.399184+0000","last_unstale":"2024-04-07T16:13:38.187224+0000","last_undegraded":"2024-04-07T16:13:38.187224+0000","last_fullsized":"2024-04-07T16:13:38.187224+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T21:07:25.104309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-07T16:13:33.668617+0000","last_change":"2024-04-07T16:13:30.003468+0000","last_active":"2024-04-07T16:13:33.668617+0000","last_peered":"2024-04-07T16:13:33.668617+0000","last_clean":"2024-04-07T16:13:33.668617+0000","last_became_active":"2024-04-07T16:13:30.002945+0000","last_became_peered":"2024-04-07T16:13:30.002945+0000","last_unstale":"2024-04-07T16:13:33.668617+0000","last_undegraded":"2024-04-07T16:13:33.668617+0000","last_fullsized":"2024-04-07T16:13:33.668617+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:13:28.963534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:13:28.963534+0000","last_clean_scrub_stamp":"2024-04-07T16:13:28.963534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T03:30:42.844279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"61'2","reported_seq":35,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.552993+0000","last_change":"2024-04-07T16:12:26.091787+0000","last_active":"2024-04-07T16:13:38.552993+0000","last_peered":"2024-04-07T16:13:38.552993+0000","last_clean":"2024-04-07T16:13:38.552993+0000","last_became_active":"2024-04-07T16:12:26.084985+0000","last_became_peered":"2024-04-07T16:12:26.084985+0000","last_unstale":"2024-04-07T16:13:38.552993+0000","last_undegraded":"2024-04-07T16:13:38.552993+0000","last_fullsized":"2024-04-07T16:13:38.552993+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T02:39:40.077566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.031201+0000","last_change":"2024-04-07T16:12:13.406541+0000","last_active":"2024-04-07T16:13:38.031201+0000","last_peered":"2024-04-07T16:13:38.031201+0000","last_clean":"2024-04-07T16:13:38.031201+0000","last_became_active":"2024-04-07T16:12:13.406018+0000","last_became_peered":"2024-04-07T16:12:13.406018+0000","last_unstale":"2024-04-07T16:13:38.031201+0000","last_undegraded":"2024-04-07T16:13:38.031201+0000","last_fullsized":"2024-04-07T16:13:38.031201+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T00:24:42.112088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":20,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.636513+0000","last_change":"2024-04-07T16:12:13.033067+0000","last_active":"2024-04-07T16:13:38.636513+0000","last_peered":"2024-04-07T16:13:38.636513+0000","last_clean":"2024-04-07T16:13:38.636513+0000","last_became_active":"2024-04-07T16:12:13.032456+0000","last_became_peered":"2024-04-07T16:12:13.032456+0000","last_unstale":"2024-04-07T16:13:38.636513+0000","last_undegraded":"2024-04-07T16:13:38.636513+0000","last_fullsized":"2024-04-07T16:13:38.636513+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T23:02:09.025617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:39.824230+0000","last_change":"2024-04-07T16:12:13.404632+0000","last_active":"2024-04-07T16:13:39.824230+0000","last_peered":"2024-04-07T16:13:39.824230+0000","last_clean":"2024-04-07T16:13:39.824230+0000","last_became_active":"2024-04-07T16:12:13.404321+0000","last_became_peered":"2024-04-07T16:12:13.404321+0000","last_unstale":"2024-04-07T16:13:39.824230+0000","last_undegraded":"2024-04-07T16:13:39.824230+0000","last_fullsized":"2024-04-07T16:13:39.824230+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T19:53:50.339257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":20,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.236054+0000","last_change":"2024-04-07T16:12:13.038113+0000","last_active":"2024-04-07T16:13:38.236054+0000","last_peered":"2024-04-07T16:13:38.236054+0000","last_clean":"2024-04-07T16:13:38.236054+0000","last_became_active":"2024-04-07T16:12:13.037783+0000","last_became_peered":"2024-04-07T16:12:13.037783+0000","last_unstale":"2024-04-07T16:13:38.236054+0000","last_undegraded":"2024-04-07T16:13:38.236054+0000","last_fullsized":"2024-04-07T16:13:38.236054+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T00:58:32.085451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.187183+0000","last_change":"2024-04-07T16:12:13.392396+0000","last_active":"2024-04-07T16:13:38.187183+0000","last_peered":"2024-04-07T16:13:38.187183+0000","last_clean":"2024-04-07T16:13:38.187183+0000","last_became_active":"2024-04-07T16:12:13.392016+0000","last_became_peered":"2024-04-07T16:12:13.392016+0000","last_unstale":"2024-04-07T16:13:38.187183+0000","last_undegraded":"2024-04-07T16:13:38.187183+0000","last_fullsized":"2024-04-07T16:13:38.187183+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T02:35:50.790051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":20,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.237136+0000","last_change":"2024-04-07T16:12:13.038401+0000","last_active":"2024-04-07T16:13:38.237136+0000","last_peered":"2024-04-07T16:13:38.237136+0000","last_clean":"2024-04-07T16:13:38.237136+0000","last_became_active":"2024-04-07T16:12:13.038110+0000","last_became_peered":"2024-04-07T16:12:13.038110+0000","last_unstale":"2024-04-07T16:13:38.237136+0000","last_undegraded":"2024-04-07T16:13:38.237136+0000","last_fullsized":"2024-04-07T16:13:38.237136+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T01:16:50.295588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.186955+0000","last_change":"2024-04-07T16:12:13.397819+0000","last_active":"2024-04-07T16:13:38.186955+0000","last_peered":"2024-04-07T16:13:38.186955+0000","last_clean":"2024-04-07T16:13:38.186955+0000","last_became_active":"2024-04-07T16:12:13.397593+0000","last_became_peered":"2024-04-07T16:12:13.397593+0000","last_unstale":"2024-04-07T16:13:38.186955+0000","last_undegraded":"2024-04-07T16:13:38.186955+0000","last_fullsized":"2024-04-07T16:13:38.186955+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T02:30:01.472030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"61'2","reported_seq":22,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.636990+0000","last_change":"2024-04-07T16:12:13.043982+0000","last_active":"2024-04-07T16:13:38.636990+0000","last_peered":"2024-04-07T16:13:38.636990+0000","last_clean":"2024-04-07T16:13:38.636990+0000","last_became_active":"2024-04-07T16:12:13.041873+0000","last_became_peered":"2024-04-07T16:12:13.041873+0000","last_unstale":"2024-04-07T16:13:38.636990+0000","last_undegraded":"2024-04-07T16:13:38.636990+0000","last_fullsized":"2024-04-07T16:13:38.636990+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T20:03:35.575133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.526346+0000","last_change":"2024-04-07T16:12:13.381810+0000","last_active":"2024-04-07T16:13:38.526346+0000","last_peered":"2024-04-07T16:13:38.526346+0000","last_clean":"2024-04-07T16:13:38.526346+0000","last_became_active":"2024-04-07T16:12:13.381470+0000","last_became_peered":"2024-04-07T16:12:13.381470+0000","last_unstale":"2024-04-07T16:13:38.526346+0000","last_undegraded":"2024-04-07T16:13:38.526346+0000","last_fullsized":"2024-04-07T16:13:38.526346+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T17:26:21.450939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":20,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.764853+0000","last_change":"2024-04-07T16:12:13.038191+0000","last_active":"2024-04-07T16:13:38.764853+0000","last_peered":"2024-04-07T16:13:38.764853+0000","last_clean":"2024-04-07T16:13:38.764853+0000","last_became_active":"2024-04-07T16:12:13.037652+0000","last_became_peered":"2024-04-07T16:12:13.037652+0000","last_unstale":"2024-04-07T16:13:38.764853+0000","last_undegraded":"2024-04-07T16:13:38.764853+0000","last_fullsized":"2024-04-07T16:13:38.764853+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T17:04:59.525025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.637269+0000","last_change":"2024-04-07T16:12:13.400660+0000","last_active":"2024-04-07T16:13:38.637269+0000","last_peered":"2024-04-07T16:13:38.637269+0000","last_clean":"2024-04-07T16:13:38.637269+0000","last_became_active":"2024-04-07T16:12:13.400353+0000","last_became_peered":"2024-04-07T16:12:13.400353+0000","last_unstale":"2024-04-07T16:13:38.637269+0000","last_undegraded":"2024-04-07T16:13:38.637269+0000","last_fullsized":"2024-04-07T16:13:38.637269+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T00:47:57.573097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"61'1","reported_seq":21,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.237152+0000","last_change":"2024-04-07T16:12:13.034772+0000","last_active":"2024-04-07T16:13:38.237152+0000","last_peered":"2024-04-07T16:13:38.237152+0000","last_clean":"2024-04-07T16:13:38.237152+0000","last_became_active":"2024-04-07T16:12:13.034455+0000","last_became_peered":"2024-04-07T16:12:13.034455+0000","last_unstale":"2024-04-07T16:13:38.237152+0000","last_undegraded":"2024-04-07T16:13:38.237152+0000","last_fullsized":"2024-04-07T16:13:38.237152+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T23:10:30.557880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.025357+0000","last_change":"2024-04-07T16:12:13.400762+0000","last_active":"2024-04-07T16:13:38.025357+0000","last_peered":"2024-04-07T16:13:38.025357+0000","last_clean":"2024-04-07T16:13:38.025357+0000","last_became_active":"2024-04-07T16:12:13.398120+0000","last_became_peered":"2024-04-07T16:12:13.398120+0000","last_unstale":"2024-04-07T16:13:38.025357+0000","last_undegraded":"2024-04-07T16:13:38.025357+0000","last_fullsized":"2024-04-07T16:13:38.025357+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T22:11:21.789998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"61'2","reported_seq":22,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.764862+0000","last_change":"2024-04-07T16:12:13.040101+0000","last_active":"2024-04-07T16:13:38.764862+0000","last_peered":"2024-04-07T16:13:38.764862+0000","last_clean":"2024-04-07T16:13:38.764862+0000","last_became_active":"2024-04-07T16:12:13.039797+0000","last_became_peered":"2024-04-07T16:12:13.039797+0000","last_unstale":"2024-04-07T16:13:38.764862+0000","last_undegraded":"2024-04-07T16:13:38.764862+0000","last_fullsized":"2024-04-07T16:13:38.764862+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T03:27:06.016325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.637921+0000","last_change":"2024-04-07T16:12:13.394720+0000","last_active":"2024-04-07T16:13:38.637921+0000","last_peered":"2024-04-07T16:13:38.637921+0000","last_clean":"2024-04-07T16:13:38.637921+0000","last_became_active":"2024-04-07T16:12:13.394090+0000","last_became_peered":"2024-04-07T16:12:13.394090+0000","last_unstale":"2024-04-07T16:13:38.637921+0000","last_undegraded":"2024-04-07T16:13:38.637921+0000","last_fullsized":"2024-04-07T16:13:38.637921+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T00:47:54.579168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.525708+0000","last_change":"2024-04-07T16:12:13.396725+0000","last_active":"2024-04-07T16:13:38.525708+0000","last_peered":"2024-04-07T16:13:38.525708+0000","last_clean":"2024-04-07T16:13:38.525708+0000","last_became_active":"2024-04-07T16:12:13.395978+0000","last_became_peered":"2024-04-07T16:12:13.395978+0000","last_unstale":"2024-04-07T16:13:38.525708+0000","last_undegraded":"2024-04-07T16:13:38.525708+0000","last_fullsized":"2024-04-07T16:13:38.525708+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T21:30:43.276569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.764525+0000","last_change":"2024-04-07T16:12:13.412091+0000","last_active":"2024-04-07T16:13:38.764525+0000","last_peered":"2024-04-07T16:13:38.764525+0000","last_clean":"2024-04-07T16:13:38.764525+0000","last_became_active":"2024-04-07T16:12:13.410814+0000","last_became_peered":"2024-04-07T16:12:13.410814+0000","last_unstale":"2024-04-07T16:13:38.764525+0000","last_undegraded":"2024-04-07T16:13:38.764525+0000","last_fullsized":"2024-04-07T16:13:38.764525+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T01:48:29.034291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.024540+0000","last_change":"2024-04-07T16:12:13.400651+0000","last_active":"2024-04-07T16:13:38.024540+0000","last_peered":"2024-04-07T16:13:38.024540+0000","last_clean":"2024-04-07T16:13:38.024540+0000","last_became_active":"2024-04-07T16:12:13.399206+0000","last_became_peered":"2024-04-07T16:12:13.399206+0000","last_unstale":"2024-04-07T16:13:38.024540+0000","last_undegraded":"2024-04-07T16:13:38.024540+0000","last_fullsized":"2024-04-07T16:13:38.024540+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T22:16:50.256754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,0],"acting":[5,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.029689+0000","last_change":"2024-04-07T16:12:13.405888+0000","last_active":"2024-04-07T16:13:38.029689+0000","last_peered":"2024-04-07T16:13:38.029689+0000","last_clean":"2024-04-07T16:13:38.029689+0000","last_became_active":"2024-04-07T16:12:13.405320+0000","last_became_peered":"2024-04-07T16:12:13.405320+0000","last_unstale":"2024-04-07T16:13:38.029689+0000","last_undegraded":"2024-04-07T16:13:38.029689+0000","last_fullsized":"2024-04-07T16:13:38.029689+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T22:07:50.421903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.186832+0000","last_change":"2024-04-07T16:12:13.398730+0000","last_active":"2024-04-07T16:13:38.186832+0000","last_peered":"2024-04-07T16:13:38.186832+0000","last_clean":"2024-04-07T16:13:38.186832+0000","last_became_active":"2024-04-07T16:12:13.398259+0000","last_became_peered":"2024-04-07T16:12:13.398259+0000","last_unstale":"2024-04-07T16:13:38.186832+0000","last_undegraded":"2024-04-07T16:13:38.186832+0000","last_fullsized":"2024-04-07T16:13:38.186832+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T02:14:28.915631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,0],"acting":[3,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.236896+0000","last_change":"2024-04-07T16:12:13.391140+0000","last_active":"2024-04-07T16:13:38.236896+0000","last_peered":"2024-04-07T16:13:38.236896+0000","last_clean":"2024-04-07T16:13:38.236896+0000","last_became_active":"2024-04-07T16:12:13.390858+0000","last_became_peered":"2024-04-07T16:12:13.390858+0000","last_unstale":"2024-04-07T16:13:38.236896+0000","last_undegraded":"2024-04-07T16:13:38.236896+0000","last_fullsized":"2024-04-07T16:13:38.236896+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T22:44:20.569791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.553144+0000","last_change":"2024-04-07T16:12:13.382605+0000","last_active":"2024-04-07T16:13:38.553144+0000","last_peered":"2024-04-07T16:13:38.553144+0000","last_clean":"2024-04-07T16:13:38.553144+0000","last_became_active":"2024-04-07T16:12:13.382267+0000","last_became_peered":"2024-04-07T16:12:13.382267+0000","last_unstale":"2024-04-07T16:13:38.553144+0000","last_undegraded":"2024-04-07T16:13:38.553144+0000","last_fullsized":"2024-04-07T16:13:38.553144+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T20:14:03.942753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,4],"acting":[6,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.187449+0000","last_change":"2024-04-07T16:12:13.405175+0000","last_active":"2024-04-07T16:13:38.187449+0000","last_peered":"2024-04-07T16:13:38.187449+0000","last_clean":"2024-04-07T16:13:38.187449+0000","last_became_active":"2024-04-07T16:12:13.404734+0000","last_became_peered":"2024-04-07T16:12:13.404734+0000","last_unstale":"2024-04-07T16:13:38.187449+0000","last_undegraded":"2024-04-07T16:13:38.187449+0000","last_fullsized":"2024-04-07T16:13:38.187449+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T17:55:23.469795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.024146+0000","last_change":"2024-04-07T16:12:13.397863+0000","last_active":"2024-04-07T16:13:38.024146+0000","last_peered":"2024-04-07T16:13:38.024146+0000","last_clean":"2024-04-07T16:13:38.024146+0000","last_became_active":"2024-04-07T16:12:13.395430+0000","last_became_peered":"2024-04-07T16:12:13.395430+0000","last_unstale":"2024-04-07T16:13:38.024146+0000","last_undegraded":"2024-04-07T16:13:38.024146+0000","last_fullsized":"2024-04-07T16:13:38.024146+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T18:20:22.678341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.186791+0000","last_change":"2024-04-07T16:12:13.397638+0000","last_active":"2024-04-07T16:13:38.186791+0000","last_peered":"2024-04-07T16:13:38.186791+0000","last_clean":"2024-04-07T16:13:38.186791+0000","last_became_active":"2024-04-07T16:12:13.396766+0000","last_became_peered":"2024-04-07T16:12:13.396766+0000","last_unstale":"2024-04-07T16:13:38.186791+0000","last_undegraded":"2024-04-07T16:13:38.186791+0000","last_fullsized":"2024-04-07T16:13:38.186791+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T22:51:06.841718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.024630+0000","last_change":"2024-04-07T16:12:13.403603+0000","last_active":"2024-04-07T16:13:38.024630+0000","last_peered":"2024-04-07T16:13:38.024630+0000","last_clean":"2024-04-07T16:13:38.024630+0000","last_became_active":"2024-04-07T16:12:13.400042+0000","last_became_peered":"2024-04-07T16:12:13.400042+0000","last_unstale":"2024-04-07T16:13:38.024630+0000","last_undegraded":"2024-04-07T16:13:38.024630+0000","last_fullsized":"2024-04-07T16:13:38.024630+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T20:49:17.570678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.029710+0000","last_change":"2024-04-07T16:12:13.386718+0000","last_active":"2024-04-07T16:13:38.029710+0000","last_peered":"2024-04-07T16:13:38.029710+0000","last_clean":"2024-04-07T16:13:38.029710+0000","last_became_active":"2024-04-07T16:12:13.386422+0000","last_became_peered":"2024-04-07T16:12:13.386422+0000","last_unstale":"2024-04-07T16:13:38.029710+0000","last_undegraded":"2024-04-07T16:13:38.029710+0000","last_fullsized":"2024-04-07T16:13:38.029710+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T21:28:09.174468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1],"acting":[0,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.526600+0000","last_change":"2024-04-07T16:12:13.380077+0000","last_active":"2024-04-07T16:13:38.526600+0000","last_peered":"2024-04-07T16:13:38.526600+0000","last_clean":"2024-04-07T16:13:38.526600+0000","last_became_active":"2024-04-07T16:12:13.379758+0000","last_became_peered":"2024-04-07T16:12:13.379758+0000","last_unstale":"2024-04-07T16:13:38.526600+0000","last_undegraded":"2024-04-07T16:13:38.526600+0000","last_fullsized":"2024-04-07T16:13:38.526600+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T22:47:34.836189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":3430,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":51,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":10290,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":45,"ondisk_log_size":45,"up":96,"acting":96,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":54,"seq":231928234026,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32396,"kb_used_data":1280,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93712756,"statfs":{"total":95995035648,"available":95961862144,"internally_reserved":0,"allocated":1310720,"data_stored":900204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Apr 7 16:14:37 2024","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":0.97399999999999998,"15min":1.004},"min":{"1min":0.65300000000000002,"5min":0.625,"15min":0.625},"max":{"1min":1.3700000000000001,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":1.012},{"interface":"front","average":{"1min":0.874,"5min":0.95799999999999996,"15min":0.98499999999999999},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.2090000000000001,"5min":1.363,"15min":1.363},"last":0.88300000000000001}]},{"osd":1,"last update":"Sun Apr 7 16:14:37 2024","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.97899999999999998,"15min":0.999},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.1419999999999999,"5min":1.55,"15min":1.55},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.92000000000000004,"5min":1.002,"15min":1.026},"min":{"1min":0.63100000000000001,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.3620000000000001,"5min":1.5069999999999999,"15min":1.5069999999999999},"last":0.95599999999999996}]},{"osd":2,"last update":"Sun Apr 7 16:14:37 2024","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.93899999999999995,"15min":0.95499999999999996},"min":{"1min":0.71799999999999997,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.115,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.99199999999999999},{"interface":"front","average":{"1min":0.96899999999999997,"5min":1.016,"15min":1.048},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.2649999999999999,"5min":1.288,"15min":1.288},"last":1.103}]},{"osd":3,"last update":"Sun Apr 7 16:14:37 2024","interfaces":[{"interface":"back","average":{"1min":0.95999999999999996,"5min":0.99399999999999999,"15min":1.018},"min":{"1min":0.67400000000000004,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.323,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":0.93300000000000005},{"interface":"front","average":{"1min":1.036,"5min":1.0800000000000001,"15min":1.113},"min":{"1min":0.63900000000000001,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.4099999999999999,"5min":1.4099999999999999,"15min":1.4099999999999999},"last":1.1679999999999999}]},{"osd":4,"last update":"Sun Apr 7 16:14:37 2024","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":0.90900000000000003,"15min":0.90700000000000003},"min":{"1min":0.45100000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.514,"5min":1.514,"15min":1.514},"last":0.69499999999999995},{"interface":"front","average":{"1min":1.0269999999999999,"5min":1.0029999999999999,"15min":1.0169999999999999},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.3220000000000001,"5min":1.468,"15min":1.468},"last":0.76600000000000001}]},{"osd":5,"last update":"Sun Apr 7 16:14:37 2024","interfaces":[{"interface":"back","average":{"1min":1.085,"5min":1.0980000000000001,"15min":1.1299999999999999},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.4810000000000001,"5min":1.7290000000000001,"15min":1.7290000000000001},"last":0.83699999999999997},{"interface":"front","average":{"1min":1.123,"5min":1.0469999999999999,"15min":1.0509999999999999},"min":{"1min":0.58199999999999996,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.6160000000000001,"5min":1.6160000000000001,"15min":1.6160000000000001},"last":1.036}]},{"osd":6,"last update":"Sun Apr 7 16:14:37 2024","interfaces":[{"interface":"back","average":{"1min":1.1639999999999999,"5min":1.135,"15min":1.1539999999999999},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.6699999999999999,"5min":1.6919999999999999,"15min":1.6919999999999999},"last":0.80500000000000005},{"interface":"front","average":{"1min":1.113,"5min":1.1120000000000001,"15min":1.125},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.5860000000000001,"5min":1.6060000000000001,"15min":1.6060000000000001},"last":1.073}]}]},{"osd":6,"up_from":46,"seq":197568495667,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28300,"kb_used_data":1284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716852,"statfs":{"total":95995035648,"available":95966056448,"internally_reserved":0,"allocated":1314816,"data_stored":902232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Apr 7 16:13:45 2024","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.93100000000000005,"15min":0.92100000000000004},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.2430000000000001,"5min":3.4079999999999999,"15min":3.4079999999999999},"last":1.105},{"interface":"front","average":{"1min":0.96899999999999997,"5min":0.96199999999999997,"15min":0.94499999999999995},"min":{"1min":0.61399999999999999,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.304,"5min":4.165,"15min":4.165},"last":0.61799999999999999}]},{"osd":1,"last update":"Sun Apr 7 16:13:45 2024","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.90400000000000003,"15min":0.90500000000000003},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.454,"5min":1.8799999999999999,"15min":1.8799999999999999},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.91200000000000003,"5min":0.96999999999999997,"15min":0.96399999999999997},"min":{"1min":0.56000000000000005,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.3160000000000001,"5min":3.4889999999999999,"15min":3.4889999999999999},"last":0.80300000000000005}]},{"osd":2,"last update":"Sun Apr 7 16:13:45 2024","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.94799999999999995,"15min":0.92900000000000005},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.286,"5min":4.218,"15min":4.218},"last":1.1419999999999999},{"interface":"front","average":{"1min":0.98099999999999998,"5min":0.99299999999999999,"15min":0.997},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.244,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":1.1539999999999999}]},{"osd":3,"last update":"Sun Apr 7 16:13:45 2024","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.97499999999999998,"15min":0.95799999999999996},"min":{"1min":0.56599999999999995,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.3120000000000001,"5min":4.2809999999999997,"15min":4.2809999999999997},"last":0.97699999999999998},{"interface":"front","average":{"1min":0.97999999999999998,"5min":0.96599999999999997,"15min":0.94799999999999995},"min":{"1min":0.55900000000000005,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.3680000000000001,"5min":3.3079999999999998,"15min":3.3079999999999998},"last":0.57399999999999995}]},{"osd":4,"last update":"Sun Apr 7 16:13:45 2024","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.88500000000000001,"15min":0.88600000000000001},"min":{"1min":0.48999999999999999,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.228,"5min":1.5609999999999999,"15min":1.5609999999999999},"last":1.056},{"interface":"front","average":{"1min":1.0269999999999999,"5min":0.97599999999999998,"15min":0.93899999999999995},"min":{"1min":0.52700000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.3360000000000001,"5min":4.8209999999999997,"15min":4.8209999999999997},"last":1.0780000000000001}]},{"osd":5,"last update":"Sun Apr 7 16:13:45 2024","interfaces":[{"interface":"back","average":{"1min":0.98899999999999999,"5min":0.92100000000000004,"15min":0.90100000000000002},"min":{"1min":0.52300000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.554,"5min":1.974,"15min":1.974},"last":1.0209999999999999},{"interface":"front","average":{"1min":1.0209999999999999,"5min":1.0129999999999999,"15min":0.999},"min":{"1min":0.59299999999999997,"5min":0.375,"15min":0.375},"max":{"1min":1.3979999999999999,"5min":1.6279999999999999,"15min":1.6279999999999999},"last":0.95299999999999996}]},{"osd":7,"last update":"Sun Apr 7 16:14:34 2024","interfaces":[{"interface":"back","average":{"1min":1.1000000000000001,"5min":1.1599999999999999,"15min":1.2050000000000001},"min":{"1min":0.51300000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.512,"5min":5.2350000000000003,"15min":5.2350000000000003},"last":0.89400000000000002},{"interface":"front","average":{"1min":1.159,"5min":1.2310000000000001,"15min":1.2749999999999999},"min":{"1min":0.47099999999999997,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.4810000000000001,"5min":4.0940000000000003,"15min":4.0940000000000003},"last":1.1890000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574947,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27720,"kb_used_data":700,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717432,"statfs":{"total":95995035648,"available":95966650368,"internally_reserved":0,"allocated":716800,"data_stored":309392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Apr 7 16:13:54 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.70999999999999996,"15min":0.59999999999999998},"min":{"1min":0.375,"5min":0.375,"15min":0.29499999999999998},"max":{"1min":1.0660000000000001,"5min":1.518,"15min":1.518},"last":0.94899999999999995},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.753,"15min":0.64400000000000002},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.315},"max":{"1min":1.04,"5min":1.2969999999999999,"15min":2.8620000000000001},"last":0.69799999999999995}]},{"osd":2,"last update":"Sun Apr 7 16:14:34 2024","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.75900000000000001,"15min":0.67700000000000005},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34000000000000002},"max":{"1min":1.159,"5min":1.345,"15min":1.345},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.73699999999999999,"15min":0.67200000000000004},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":1.2050000000000001,"5min":1.5369999999999999,"15min":1.5369999999999999},"last":0.77400000000000002}]},{"osd":3,"last update":"Sun Apr 7 16:14:29 2024","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.88200000000000001,"15min":0.82799999999999996},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.0609999999999999,"5min":1.343,"15min":1.343},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.81200000000000006,"15min":0.76600000000000001},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":1.1080000000000001,"5min":1.4770000000000001,"15min":1.4770000000000001},"last":0.82099999999999995}]},{"osd":4,"last update":"Sun Apr 7 16:14:17 2024","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":0.94099999999999995,"15min":0.93200000000000005},"min":{"1min":0.59999999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.248,"5min":1.5149999999999999,"15min":1.5149999999999999},"last":0.98399999999999999},{"interface":"front","average":{"1min":1.0369999999999999,"5min":1.0820000000000001,"15min":1.054},"min":{"1min":0.59999999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.393,"5min":1.9299999999999999,"15min":1.9299999999999999},"last":1.0600000000000001}]},{"osd":5,"last update":"Sun Apr 7 16:14:02 2024","interfaces":[{"interface":"back","average":{"1min":0.996,"5min":1.1459999999999999,"15min":1.2150000000000001},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.3620000000000001,"5min":2.0510000000000002,"15min":2.0510000000000002},"last":1.0129999999999999},{"interface":"front","average":{"1min":0.96799999999999997,"5min":1.0660000000000001,"15min":1.117},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.5740000000000001,"5min":1.8859999999999999,"15min":1.8859999999999999},"last":1.29}]},{"osd":6,"last update":"Sun Apr 7 16:13:51 2024","interfaces":[{"interface":"back","average":{"1min":0.97099999999999997,"5min":1.0640000000000001,"15min":1.0860000000000001},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.4890000000000001,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":1.339},{"interface":"front","average":{"1min":1.1679999999999999,"5min":1.258,"15min":1.268},"min":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"max":{"1min":1.538,"5min":2.5609999999999999,"15min":2.5609999999999999},"last":1.218}]},{"osd":7,"last update":"Sun Apr 7 16:14:34 2024","interfaces":[{"interface":"back","average":{"1min":0.97999999999999998,"5min":1.083,"15min":1.109},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.3919999999999999,"5min":1.702,"15min":1.702},"last":1.411},{"interface":"front","average":{"1min":1.123,"5min":1.179,"15min":1.181},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.698,"5min":1.698,"15min":1.698},"last":1.1699999999999999}]}]},{"osd":0,"up_from":8,"seq":34359738478,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28308,"kb_used_data":1288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716844,"statfs":{"total":95995035648,"available":95966048256,"internally_reserved":0,"allocated":1318912,"data_stored":900328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Apr 7 16:13:49 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.67200000000000004,"15min":0.58699999999999997},"min":{"1min":0.372,"5min":0.372,"15min":0.25600000000000001},"max":{"1min":0.96699999999999997,"5min":1.353,"15min":1.353},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.76300000000000001,"15min":0.60199999999999998},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.0129999999999999,"5min":1.214,"15min":1.214},"last":0.96199999999999997}]},{"osd":2,"last update":"Sun Apr 7 16:14:40 2024","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.73999999999999999,"15min":0.68200000000000005},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":1.2,"5min":1.2,"15min":1.2},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.77100000000000002,"15min":0.73999999999999999},"min":{"1min":0.40600000000000003,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.96499999999999997,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.80500000000000005}]},{"osd":3,"last update":"Sun Apr 7 16:14:26 2024","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.78800000000000003,"15min":0.75900000000000001},"min":{"1min":0.36399999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.84399999999999997,"15min":0.80700000000000005},"min":{"1min":0.39400000000000002,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.0589999999999999,"5min":1.143,"15min":1.143},"last":0.74199999999999999}]},{"osd":4,"last update":"Sun Apr 7 16:14:13 2024","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":0.95399999999999996,"15min":0.92900000000000005},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.4239999999999999,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.89900000000000002},{"interface":"front","average":{"1min":1.0269999999999999,"5min":1.0169999999999999,"15min":0.97399999999999998},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.6930000000000001,"5min":1.7809999999999999,"15min":1.7809999999999999},"last":1.0820000000000001}]},{"osd":5,"last update":"Sun Apr 7 16:13:58 2024","interfaces":[{"interface":"back","average":{"1min":1.022,"5min":0.999,"15min":0.98499999999999999},"min":{"1min":0.745,"5min":0.745,"15min":0.745},"max":{"1min":1.4450000000000001,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":1.0429999999999999},{"interface":"front","average":{"1min":1.0429999999999999,"5min":1.0329999999999999,"15min":1.0249999999999999},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.488,"5min":1.488,"15min":1.488},"last":0.98299999999999998}]},{"osd":6,"last update":"Sun Apr 7 16:14:44 2024","interfaces":[{"interface":"back","average":{"1min":0.998,"5min":1.0649999999999999,"15min":1.0680000000000001},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.335,"5min":1.7629999999999999,"15min":1.7629999999999999},"last":1.0249999999999999},{"interface":"front","average":{"1min":1.0389999999999999,"5min":1.109,"15min":1.119},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.272,"5min":1.5429999999999999,"15min":1.5429999999999999},"last":0.93000000000000005}]},{"osd":7,"last update":"Sun Apr 7 16:14:35 2024","interfaces":[{"interface":"back","average":{"1min":1.052,"5min":1.139,"15min":1.165},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.421,"5min":1.6100000000000001,"15min":1.6100000000000001},"last":1.1299999999999999},{"interface":"front","average":{"1min":1.0980000000000001,"5min":1.1579999999999999,"15min":1.1759999999999999},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.385,"5min":1.464,"15min":1.464},"last":1.111}]}]},{"osd":2,"up_from":18,"seq":77309411418,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31808,"kb_used_data":692,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713344,"statfs":{"total":95995035648,"available":95962464256,"internally_reserved":0,"allocated":708608,"data_stored":309232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Apr 7 16:14:36 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.70199999999999996,"15min":0.61499999999999999},"min":{"1min":0.38100000000000001,"5min":0.16800000000000001,"15min":0.16800000000000001},"max":{"1min":1.0720000000000001,"5min":1.204,"15min":1.204},"last":1.1910000000000001},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.747,"15min":0.63900000000000001},"min":{"1min":0.36599999999999999,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.97299999999999998,"5min":1.109,"15min":1.109},"last":0.91400000000000003}]},{"osd":1,"last update":"Sun Apr 7 16:14:36 2024","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.76500000000000001,"15min":0.67200000000000004},"min":{"1min":0.38600000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":1.204,"5min":1.204,"15min":1.204},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.749,"15min":0.66900000000000004},"min":{"1min":0.33400000000000002,"5min":0.318,"15min":0.313},"max":{"1min":1.0109999999999999,"5min":1.3120000000000001,"15min":1.3120000000000001},"last":0.83999999999999997}]},{"osd":3,"last update":"Sun Apr 7 16:14:24 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.80100000000000005,"15min":0.80500000000000005},"min":{"1min":0.39400000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":1.01,"5min":1.099,"15min":1.099},"last":0.93200000000000005},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.80100000000000005,"15min":0.79800000000000004},"min":{"1min":0.40899999999999997,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.046,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":0.79600000000000004}]},{"osd":4,"last update":"Sun Apr 7 16:14:16 2024","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.94699999999999995,"15min":0.92500000000000004},"min":{"1min":0.78700000000000003,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.1899999999999999,"5min":1.4299999999999999,"15min":1.4299999999999999},"last":0.97999999999999998},{"interface":"front","average":{"1min":1.0209999999999999,"5min":0.995,"15min":0.95199999999999996},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.411,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":1.421}]},{"osd":5,"last update":"Sun Apr 7 16:13:57 2024","interfaces":[{"interface":"back","average":{"1min":1.0429999999999999,"5min":1.008,"15min":0.98399999999999999},"min":{"1min":0.68100000000000005,"5min":0.374,"15min":0.374},"max":{"1min":1.2030000000000001,"5min":1.786,"15min":1.786},"last":1.1000000000000001},{"interface":"front","average":{"1min":1.101,"5min":1.0760000000000001,"15min":1.05},"min":{"1min":0.85699999999999998,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.3540000000000001,"5min":1.6759999999999999,"15min":1.6759999999999999},"last":1.806}]},{"osd":6,"last update":"Sun Apr 7 16:14:44 2024","interfaces":[{"interface":"back","average":{"1min":1.0129999999999999,"5min":1.069,"15min":1.056},"min":{"1min":0.66000000000000003,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.409,"5min":1.5449999999999999,"15min":1.5449999999999999},"last":1.976},{"interface":"front","average":{"1min":0.99099999999999999,"5min":1.073,"15min":1.0649999999999999},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.254,"5min":1.839,"15min":1.839},"last":1.387}]},{"osd":7,"last update":"Sun Apr 7 16:14:36 2024","interfaces":[{"interface":"back","average":{"1min":1.0629999999999999,"5min":1.117,"15min":1.1100000000000001},"min":{"1min":0.56000000000000005,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.377,"5min":1.6259999999999999,"15min":1.6259999999999999},"last":1.726},{"interface":"front","average":{"1min":1.054,"5min":1.111,"15min":1.1100000000000001},"min":{"1min":0.66200000000000003,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.3720000000000001,"5min":1.579,"15min":1.579},"last":1.46}]}]},{"osd":3,"up_from":25,"seq":107374182481,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27728,"kb_used_data":712,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717424,"statfs":{"total":95995035648,"available":95966642176,"internally_reserved":0,"allocated":729088,"data_stored":311968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Apr 7 16:14:32 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.68600000000000005,"15min":0.67400000000000004},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":1.0389999999999999,"5min":1.345,"15min":1.345},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.76700000000000002,"15min":0.78300000000000003},"min":{"1min":0.372,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.94499999999999995,"5min":1.327,"15min":1.327},"last":0.70899999999999996}]},{"osd":1,"last update":"Sun Apr 7 16:14:32 2024","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.78600000000000003,"15min":0.76000000000000001},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":1.1140000000000001,"5min":1.544,"15min":1.544},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.79300000000000004,"15min":0.76700000000000002},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.0209999999999999,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.88800000000000001}]},{"osd":2,"last update":"Sun Apr 7 16:14:32 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.85099999999999998,"15min":0.81699999999999995},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.258,"5min":1.27,"15min":1.27},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.79100000000000004,"15min":0.80600000000000005},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.999,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.77900000000000003}]},{"osd":4,"last update":"Sun Apr 7 16:14:10 2024","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":0.97399999999999998,"15min":0.95599999999999996},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.218,"5min":1.552,"15min":1.552},"last":1.526},{"interface":"front","average":{"1min":0.96599999999999997,"5min":1.014,"15min":0.98199999999999998},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.3779999999999999,"5min":1.403,"15min":1.403},"last":0.97799999999999998}]},{"osd":5,"last update":"Sun Apr 7 16:13:59 2024","interfaces":[{"interface":"back","average":{"1min":1.0669999999999999,"5min":1.0600000000000001,"15min":1.0580000000000001},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.5589999999999999,"5min":1.5589999999999999,"15min":1.5589999999999999},"last":1.05},{"interface":"front","average":{"1min":0.97499999999999998,"5min":1.012,"15min":1.0309999999999999},"min":{"1min":0.68600000000000005,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.2470000000000001,"5min":1.571,"15min":1.571},"last":1.202}]},{"osd":6,"last update":"Sun Apr 7 16:14:47 2024","interfaces":[{"interface":"back","average":{"1min":0.97199999999999998,"5min":1.0640000000000001,"15min":1.0860000000000001},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.3600000000000001,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":1.3600000000000001},{"interface":"front","average":{"1min":1.0600000000000001,"5min":1.1240000000000001,"15min":1.1319999999999999},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.4139999999999999,"5min":1.5660000000000001,"15min":1.5660000000000001},"last":1.2450000000000001}]},{"osd":7,"last update":"Sun Apr 7 16:14:36 2024","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":1.069,"15min":1.089},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.373,"5min":1.387,"15min":1.387},"last":1.008},{"interface":"front","average":{"1min":0.98799999999999999,"5min":1.1559999999999999,"15min":1.194},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.3169999999999999,"5min":1.52,"15min":1.52},"last":1.4490000000000001}]}]},{"osd":4,"up_from":30,"seq":128849018952,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31824,"kb_used_data":708,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713328,"statfs":{"total":95995035648,"available":95962447872,"internally_reserved":0,"allocated":724992,"data_stored":309940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Apr 7 16:14:14 2024","interfaces":[{"interface":"back","average":{"1min":0.995,"5min":0.90700000000000003,"15min":0.84299999999999997},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.319,"5min":1.3839999999999999,"15min":1.3839999999999999},"last":1.012},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.89300000000000002,"15min":0.90100000000000002},"min":{"1min":0.67100000000000004,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.204,"5min":1.345,"15min":1.345},"last":1.046}]},{"osd":1,"last update":"Sun Apr 7 16:14:14 2024","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":0.90300000000000002,"15min":0.83699999999999997},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.389,"5min":1.389,"15min":1.389},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.96999999999999997,"5min":0.91400000000000003,"15min":0.85699999999999998},"min":{"1min":0.64800000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.264,"5min":1.486,"15min":1.486},"last":1.2689999999999999}]},{"osd":2,"last update":"Sun Apr 7 16:14:14 2024","interfaces":[{"interface":"back","average":{"1min":0.94899999999999995,"5min":0.92100000000000004,"15min":0.878},"min":{"1min":0.58099999999999996,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.355,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":1.1419999999999999},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.88900000000000001,"15min":0.88400000000000001},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.407,"5min":1.407,"15min":1.407},"last":0.96699999999999997}]},{"osd":3,"last update":"Sun Apr 7 16:14:14 2024","interfaces":[{"interface":"back","average":{"1min":0.92600000000000005,"5min":0.92100000000000004,"15min":0.93100000000000005},"min":{"1min":0.69299999999999995,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.4450000000000001,"5min":1.458,"15min":1.458},"last":0.89500000000000002},{"interface":"front","average":{"1min":1.0589999999999999,"5min":1.0009999999999999,"15min":0.94799999999999995},"min":{"1min":0.68400000000000005,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.3660000000000001,"5min":1.472,"15min":1.472},"last":1.1799999999999999}]},{"osd":5,"last update":"Sun Apr 7 16:13:58 2024","interfaces":[{"interface":"back","average":{"1min":1.0169999999999999,"5min":0.875,"15min":0.78700000000000003},"min":{"1min":0.42899999999999999,"5min":0.315,"15min":0.315},"max":{"1min":1.2949999999999999,"5min":1.403,"15min":1.403},"last":0.85599999999999998},{"interface":"front","average":{"1min":1.0529999999999999,"5min":0.94899999999999995,"15min":0.88700000000000001},"min":{"1min":0.40999999999999998,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.4530000000000001,"5min":1.546,"15min":1.546},"last":1.2190000000000001}]},{"osd":6,"last update":"Sun Apr 7 16:14:47 2024","interfaces":[{"interface":"back","average":{"1min":1.1879999999999999,"5min":1.046,"15min":0.99399999999999999},"min":{"1min":0.88300000000000001,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.5,"5min":1.5,"15min":1.5},"last":0.93799999999999994},{"interface":"front","average":{"1min":1.169,"5min":1.044,"15min":0.98599999999999999},"min":{"1min":0.79200000000000004,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.472,"5min":1.5309999999999999,"15min":1.5309999999999999},"last":1.3129999999999999}]},{"osd":7,"last update":"Sun Apr 7 16:14:36 2024","interfaces":[{"interface":"back","average":{"1min":1.109,"5min":1.044,"15min":1.048},"min":{"1min":0.77500000000000002,"5min":0.433,"15min":0.433},"max":{"1min":1.5029999999999999,"5min":1.5449999999999999,"15min":1.5449999999999999},"last":1.117},{"interface":"front","average":{"1min":1.319,"5min":1.2210000000000001,"15min":1.206},"min":{"1min":1.113,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.5940000000000001,"5min":1.734,"15min":1.734},"last":1.085}]}]},{"osd":5,"up_from":38,"seq":163208757309,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27720,"kb_used_data":704,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717432,"statfs":{"total":95995035648,"available":95966650368,"internally_reserved":0,"allocated":720896,"data_stored":311623,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Apr 7 16:14:00 2024","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.92700000000000005,"15min":0.92000000000000004},"min":{"1min":0.57499999999999996,"5min":0.443,"15min":0.443},"max":{"1min":1.355,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":1.1020000000000001},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.89800000000000002,"15min":0.94199999999999995},"min":{"1min":0.55000000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.196,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":1.0509999999999999}]},{"osd":1,"last update":"Sun Apr 7 16:14:00 2024","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":0.89800000000000002,"15min":0.91600000000000004},"min":{"1min":0.53000000000000003,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.474,"5min":1.474,"15min":1.474},"last":0.89200000000000002},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.95899999999999996,"15min":0.98099999999999998},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.2150000000000001,"5min":1.45,"15min":1.45},"last":0.86099999999999999}]},{"osd":2,"last update":"Sun Apr 7 16:14:00 2024","interfaces":[{"interface":"back","average":{"1min":0.97599999999999998,"5min":0.95599999999999996,"15min":0.97499999999999998},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.3740000000000001,"5min":1.462,"15min":1.462},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.93600000000000005,"5min":0.94699999999999995,"15min":0.96199999999999997},"min":{"1min":0.63700000000000001,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.1479999999999999,"5min":1.472,"15min":1.472},"last":1.0720000000000001}]},{"osd":3,"last update":"Sun Apr 7 16:14:00 2024","interfaces":[{"interface":"back","average":{"1min":0.95399999999999996,"5min":1.05,"15min":1.177},"min":{"1min":0.58199999999999996,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.5880000000000001,"5min":6.3860000000000001,"15min":6.3860000000000001},"last":1.0229999999999999},{"interface":"front","average":{"1min":0.96699999999999997,"5min":0.97499999999999998,"15min":0.999},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.284,"5min":1.556,"15min":1.556},"last":0.99399999999999999}]},{"osd":4,"last update":"Sun Apr 7 16:14:00 2024","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.84099999999999997,"15min":0.81699999999999995},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":1.3280000000000001,"5min":1.403,"15min":1.403},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.96099999999999997,"5min":0.91900000000000004,"15min":0.91300000000000003},"min":{"1min":0.372,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.333,"5min":1.4119999999999999,"15min":1.4119999999999999},"last":1.1579999999999999}]},{"osd":6,"last update":"Sun Apr 7 16:14:44 2024","interfaces":[{"interface":"back","average":{"1min":1.0629999999999999,"5min":0.93200000000000005,"15min":0.879},"min":{"1min":0.42499999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":2.6459999999999999,"5min":2.6459999999999999,"15min":2.6459999999999999},"last":0.96199999999999997},{"interface":"front","average":{"1min":1.1200000000000001,"5min":1.036,"15min":0.98999999999999999},"min":{"1min":0.48799999999999999,"5min":0.31,"15min":0.31},"max":{"1min":2.109,"5min":2.109,"15min":2.109},"last":0.93600000000000005}]},{"osd":7,"last update":"Sun Apr 7 16:14:34 2024","interfaces":[{"interface":"back","average":{"1min":1.1220000000000001,"5min":1.0900000000000001,"15min":1.1000000000000001},"min":{"1min":0.54500000000000004,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":2.5830000000000002,"5min":2.5830000000000002,"15min":2.5830000000000002},"last":1.129},{"interface":"front","average":{"1min":1.2210000000000001,"5min":1.163,"15min":1.169},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":2.7509999999999999,"5min":2.7509999999999999,"15min":2.7509999999999999},"last":1.2050000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-07T16:14:53.523 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 0702e3a8-f4f8-11ee-b649-cb9ed24678a4 -- ceph pg dump --format=json 2024-04-07T16:14:54.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:14:53 smithi144 bash[23522]: cluster 2024-04-07T16:14:52.647636+0000 mgr.y (mgr.14553) 95 : cluster 0 pgmap v67: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:14:54.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:14:53 smithi144 bash[23522]: audit 2024-04-07T16:14:52.802270+0000 mgr.y (mgr.14553) 96 : audit 0 from='client.14745 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-07T16:14:54.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:14:53 smithi100 bash[31660]: cluster 2024-04-07T16:14:52.647636+0000 mgr.y (mgr.14553) 95 : cluster 0 pgmap v67: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:14:54.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:14:53 smithi100 bash[31660]: audit 2024-04-07T16:14:52.802270+0000 mgr.y (mgr.14553) 96 : audit 0 from='client.14745 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-07T16:14:54.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:14:53 smithi100 bash[21346]: cluster 2024-04-07T16:14:52.647636+0000 mgr.y (mgr.14553) 95 : cluster 0 pgmap v67: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:14:54.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:14:53 smithi100 bash[21346]: audit 2024-04-07T16:14:52.802270+0000 mgr.y (mgr.14553) 96 : audit 0 from='client.14745 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-07T16:14:55.437 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:14:55 smithi100 bash[21601]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:14:55] "GET /metrics HTTP/1.1" 200 33732 "" "Prometheus/2.43.0" 2024-04-07T16:14:56.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:14:55 smithi144 bash[23522]: cluster 2024-04-07T16:14:54.648766+0000 mgr.y (mgr.14553) 97 : cluster 0 pgmap v68: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:14:56.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:14:55 smithi100 bash[21346]: cluster 2024-04-07T16:14:54.648766+0000 mgr.y (mgr.14553) 97 : cluster 0 pgmap v68: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:14:56.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:14:55 smithi100 bash[31660]: cluster 2024-04-07T16:14:54.648766+0000 mgr.y (mgr.14553) 97 : cluster 0 pgmap v68: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:14:58.252 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/mon.a/config 2024-04-07T16:14:58.272 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:14:57 smithi100 bash[21346]: cluster 2024-04-07T16:14:56.649417+0000 mgr.y (mgr.14553) 98 : cluster 0 pgmap v69: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:14:58.273 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:14:57 smithi100 bash[31660]: cluster 2024-04-07T16:14:56.649417+0000 mgr.y (mgr.14553) 98 : cluster 0 pgmap v69: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:14:58.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:14:57 smithi144 bash[23522]: cluster 2024-04-07T16:14:56.649417+0000 mgr.y (mgr.14553) 98 : cluster 0 pgmap v69: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:15:00.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:14:59 smithi100 bash[21346]: cluster 2024-04-07T16:14:58.650199+0000 mgr.y (mgr.14553) 99 : cluster 0 pgmap v70: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:15:00.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:14:59 smithi100 bash[31660]: cluster 2024-04-07T16:14:58.650199+0000 mgr.y (mgr.14553) 99 : cluster 0 pgmap v70: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:15:00.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:14:59 smithi144 bash[23522]: cluster 2024-04-07T16:14:58.650199+0000 mgr.y (mgr.14553) 99 : cluster 0 pgmap v70: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:15:00.806 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-07T16:15:00.810 INFO:teuthology.orchestra.run.smithi100.stderr:dumped all 2024-04-07T16:15:01.465 INFO:teuthology.orchestra.run.smithi100.stdout:{"pg_ready":true,"pg_map":{"version":71,"stamp":"2024-04-07T16:15:00.650495+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":593798,"num_objects":24,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":230,"num_write_kb":2436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":173,"ondisk_log_size":173,"up":315,"acting":315,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":315,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":235804,"kb_used_data":7368,"kb_used_omap":12,"kb_used_meta":228339,"kb_avail":749725412,"statfs":{"total":767960285184,"available":767718821888,"internally_reserved":0,"allocated":7544832,"data_stored":4254919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":233819705},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004926"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.637813+0000","last_change":"2024-04-07T16:12:13.395367+0000","last_active":"2024-04-07T16:13:38.637813+0000","last_peered":"2024-04-07T16:13:38.637813+0000","last_clean":"2024-04-07T16:13:38.637813+0000","last_became_active":"2024-04-07T16:12:13.395023+0000","last_became_peered":"2024-04-07T16:12:13.395023+0000","last_unstale":"2024-04-07T16:13:38.637813+0000","last_undegraded":"2024-04-07T16:13:38.637813+0000","last_fullsized":"2024-04-07T16:13:38.637813+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T23:05:09.752696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.237070+0000","last_change":"2024-04-07T16:12:13.385549+0000","last_active":"2024-04-07T16:13:38.237070+0000","last_peered":"2024-04-07T16:13:38.237070+0000","last_clean":"2024-04-07T16:13:38.237070+0000","last_became_active":"2024-04-07T16:12:13.385175+0000","last_became_peered":"2024-04-07T16:12:13.385175+0000","last_unstale":"2024-04-07T16:13:38.237070+0000","last_undegraded":"2024-04-07T16:13:38.237070+0000","last_fullsized":"2024-04-07T16:13:38.237070+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T16:28:22.788092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,6],"acting":[1,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.030532+0000","last_change":"2024-04-07T16:12:13.410431+0000","last_active":"2024-04-07T16:13:38.030532+0000","last_peered":"2024-04-07T16:13:38.030532+0000","last_clean":"2024-04-07T16:13:38.030532+0000","last_became_active":"2024-04-07T16:12:13.410088+0000","last_became_peered":"2024-04-07T16:12:13.410088+0000","last_unstale":"2024-04-07T16:13:38.030532+0000","last_undegraded":"2024-04-07T16:13:38.030532+0000","last_fullsized":"2024-04-07T16:13:38.030532+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T19:47:57.716144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.025110+0000","last_change":"2024-04-07T16:12:13.402328+0000","last_active":"2024-04-07T16:13:38.025110+0000","last_peered":"2024-04-07T16:13:38.025110+0000","last_clean":"2024-04-07T16:13:38.025110+0000","last_became_active":"2024-04-07T16:12:13.401819+0000","last_became_peered":"2024-04-07T16:12:13.401819+0000","last_unstale":"2024-04-07T16:13:38.025110+0000","last_undegraded":"2024-04-07T16:13:38.025110+0000","last_fullsized":"2024-04-07T16:13:38.025110+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T21:43:13.292859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.637270+0000","last_change":"2024-04-07T16:12:13.413487+0000","last_active":"2024-04-07T16:13:38.637270+0000","last_peered":"2024-04-07T16:13:38.637270+0000","last_clean":"2024-04-07T16:13:38.637270+0000","last_became_active":"2024-04-07T16:12:13.413281+0000","last_became_peered":"2024-04-07T16:12:13.413281+0000","last_unstale":"2024-04-07T16:13:38.637270+0000","last_undegraded":"2024-04-07T16:13:38.637270+0000","last_fullsized":"2024-04-07T16:13:38.637270+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T23:59:23.302803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.024848+0000","last_change":"2024-04-07T16:12:13.401561+0000","last_active":"2024-04-07T16:13:38.024848+0000","last_peered":"2024-04-07T16:13:38.024848+0000","last_clean":"2024-04-07T16:13:38.024848+0000","last_became_active":"2024-04-07T16:12:13.400365+0000","last_became_peered":"2024-04-07T16:12:13.400365+0000","last_unstale":"2024-04-07T16:13:38.024848+0000","last_undegraded":"2024-04-07T16:13:38.024848+0000","last_fullsized":"2024-04-07T16:13:38.024848+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T20:14:49.110462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.636229+0000","last_change":"2024-04-07T16:12:13.393518+0000","last_active":"2024-04-07T16:13:38.636229+0000","last_peered":"2024-04-07T16:13:38.636229+0000","last_clean":"2024-04-07T16:13:38.636229+0000","last_became_active":"2024-04-07T16:12:13.393187+0000","last_became_peered":"2024-04-07T16:12:13.393187+0000","last_unstale":"2024-04-07T16:13:38.636229+0000","last_undegraded":"2024-04-07T16:13:38.636229+0000","last_fullsized":"2024-04-07T16:13:38.636229+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T19:41:28.702905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.187095+0000","last_change":"2024-04-07T16:12:13.394702+0000","last_active":"2024-04-07T16:13:38.187095+0000","last_peered":"2024-04-07T16:13:38.187095+0000","last_clean":"2024-04-07T16:13:38.187095+0000","last_became_active":"2024-04-07T16:12:13.394422+0000","last_became_peered":"2024-04-07T16:12:13.394422+0000","last_unstale":"2024-04-07T16:13:38.187095+0000","last_undegraded":"2024-04-07T16:13:38.187095+0000","last_fullsized":"2024-04-07T16:13:38.187095+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T00:37:44.512471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.637842+0000","last_change":"2024-04-07T16:12:13.401766+0000","last_active":"2024-04-07T16:13:38.637842+0000","last_peered":"2024-04-07T16:13:38.637842+0000","last_clean":"2024-04-07T16:13:38.637842+0000","last_became_active":"2024-04-07T16:12:13.401411+0000","last_became_peered":"2024-04-07T16:12:13.401411+0000","last_unstale":"2024-04-07T16:13:38.637842+0000","last_undegraded":"2024-04-07T16:13:38.637842+0000","last_fullsized":"2024-04-07T16:13:38.637842+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T23:32:52.283410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,0],"acting":[7,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.553143+0000","last_change":"2024-04-07T16:12:13.396409+0000","last_active":"2024-04-07T16:13:38.553143+0000","last_peered":"2024-04-07T16:13:38.553143+0000","last_clean":"2024-04-07T16:13:38.553143+0000","last_became_active":"2024-04-07T16:12:13.396094+0000","last_became_peered":"2024-04-07T16:12:13.396094+0000","last_unstale":"2024-04-07T16:13:38.553143+0000","last_undegraded":"2024-04-07T16:13:38.553143+0000","last_fullsized":"2024-04-07T16:13:38.553143+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T18:54:36.072837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.552867+0000","last_change":"2024-04-07T16:12:13.401973+0000","last_active":"2024-04-07T16:13:38.552867+0000","last_peered":"2024-04-07T16:13:38.552867+0000","last_clean":"2024-04-07T16:13:38.552867+0000","last_became_active":"2024-04-07T16:12:13.401724+0000","last_became_peered":"2024-04-07T16:12:13.401724+0000","last_unstale":"2024-04-07T16:13:38.552867+0000","last_undegraded":"2024-04-07T16:13:38.552867+0000","last_fullsized":"2024-04-07T16:13:38.552867+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T21:26:29.208223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,0],"acting":[6,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:39.824604+0000","last_change":"2024-04-07T16:12:13.404858+0000","last_active":"2024-04-07T16:13:39.824604+0000","last_peered":"2024-04-07T16:13:39.824604+0000","last_clean":"2024-04-07T16:13:39.824604+0000","last_became_active":"2024-04-07T16:12:13.404529+0000","last_became_peered":"2024-04-07T16:12:13.404529+0000","last_unstale":"2024-04-07T16:13:39.824604+0000","last_undegraded":"2024-04-07T16:13:39.824604+0000","last_fullsized":"2024-04-07T16:13:39.824604+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T19:57:22.893626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,0],"acting":[4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.186817+0000","last_change":"2024-04-07T16:12:13.394760+0000","last_active":"2024-04-07T16:13:38.186817+0000","last_peered":"2024-04-07T16:13:38.186817+0000","last_clean":"2024-04-07T16:13:38.186817+0000","last_became_active":"2024-04-07T16:12:13.394402+0000","last_became_peered":"2024-04-07T16:12:13.394402+0000","last_unstale":"2024-04-07T16:13:38.186817+0000","last_undegraded":"2024-04-07T16:13:38.186817+0000","last_fullsized":"2024-04-07T16:13:38.186817+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T18:47:13.891972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.029316+0000","last_change":"2024-04-07T16:12:13.400292+0000","last_active":"2024-04-07T16:13:38.029316+0000","last_peered":"2024-04-07T16:13:38.029316+0000","last_clean":"2024-04-07T16:13:38.029316+0000","last_became_active":"2024-04-07T16:12:13.400028+0000","last_became_peered":"2024-04-07T16:12:13.400028+0000","last_unstale":"2024-04-07T16:13:38.029316+0000","last_undegraded":"2024-04-07T16:13:38.029316+0000","last_fullsized":"2024-04-07T16:13:38.029316+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T18:14:33.693852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.235899+0000","last_change":"2024-04-07T16:12:13.398535+0000","last_active":"2024-04-07T16:13:38.235899+0000","last_peered":"2024-04-07T16:13:38.235899+0000","last_clean":"2024-04-07T16:13:38.235899+0000","last_became_active":"2024-04-07T16:12:13.398287+0000","last_became_peered":"2024-04-07T16:12:13.398287+0000","last_unstale":"2024-04-07T16:13:38.235899+0000","last_undegraded":"2024-04-07T16:13:38.235899+0000","last_fullsized":"2024-04-07T16:13:38.235899+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T23:58:53.081885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,5],"acting":[1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.023905+0000","last_change":"2024-04-07T16:12:13.395243+0000","last_active":"2024-04-07T16:13:38.023905+0000","last_peered":"2024-04-07T16:13:38.023905+0000","last_clean":"2024-04-07T16:13:38.023905+0000","last_became_active":"2024-04-07T16:12:13.394956+0000","last_became_peered":"2024-04-07T16:12:13.394956+0000","last_unstale":"2024-04-07T16:13:38.023905+0000","last_undegraded":"2024-04-07T16:13:38.023905+0000","last_fullsized":"2024-04-07T16:13:38.023905+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T03:04:58.804429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.026420+0000","last_change":"2024-04-07T16:12:13.398543+0000","last_active":"2024-04-07T16:13:38.026420+0000","last_peered":"2024-04-07T16:13:38.026420+0000","last_clean":"2024-04-07T16:13:38.026420+0000","last_became_active":"2024-04-07T16:12:13.398198+0000","last_became_peered":"2024-04-07T16:12:13.398198+0000","last_unstale":"2024-04-07T16:13:38.026420+0000","last_undegraded":"2024-04-07T16:13:38.026420+0000","last_fullsized":"2024-04-07T16:13:38.026420+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T22:57:24.895712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,4],"acting":[5,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.764814+0000","last_change":"2024-04-07T16:12:13.416210+0000","last_active":"2024-04-07T16:13:38.764814+0000","last_peered":"2024-04-07T16:13:38.764814+0000","last_clean":"2024-04-07T16:13:38.764814+0000","last_became_active":"2024-04-07T16:12:13.415972+0000","last_became_peered":"2024-04-07T16:12:13.415972+0000","last_unstale":"2024-04-07T16:13:38.764814+0000","last_undegraded":"2024-04-07T16:13:38.764814+0000","last_fullsized":"2024-04-07T16:13:38.764814+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T23:40:11.606978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,7],"acting":[2,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.636945+0000","last_change":"2024-04-07T16:12:13.414203+0000","last_active":"2024-04-07T16:13:38.636945+0000","last_peered":"2024-04-07T16:13:38.636945+0000","last_clean":"2024-04-07T16:13:38.636945+0000","last_became_active":"2024-04-07T16:12:13.413869+0000","last_became_peered":"2024-04-07T16:12:13.413869+0000","last_unstale":"2024-04-07T16:13:38.636945+0000","last_undegraded":"2024-04-07T16:13:38.636945+0000","last_fullsized":"2024-04-07T16:13:38.636945+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T17:11:47.895568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,6],"acting":[7,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1f","version":"61'4","reported_seq":24,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.031549+0000","last_change":"2024-04-07T16:12:13.032601+0000","last_active":"2024-04-07T16:13:38.031549+0000","last_peered":"2024-04-07T16:13:38.031549+0000","last_clean":"2024-04-07T16:13:38.031549+0000","last_became_active":"2024-04-07T16:12:13.032199+0000","last_became_peered":"2024-04-07T16:12:13.032199+0000","last_unstale":"2024-04-07T16:13:38.031549+0000","last_undegraded":"2024-04-07T16:13:38.031549+0000","last_fullsized":"2024-04-07T16:13:38.031549+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T19:01:44.798719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.187256+0000","last_change":"2024-04-07T16:12:13.407852+0000","last_active":"2024-04-07T16:13:38.187256+0000","last_peered":"2024-04-07T16:13:38.187256+0000","last_clean":"2024-04-07T16:13:38.187256+0000","last_became_active":"2024-04-07T16:12:13.407538+0000","last_became_peered":"2024-04-07T16:12:13.407538+0000","last_unstale":"2024-04-07T16:13:38.187256+0000","last_undegraded":"2024-04-07T16:13:38.187256+0000","last_fullsized":"2024-04-07T16:13:38.187256+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T20:27:52.664149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"61'1","reported_seq":21,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.187484+0000","last_change":"2024-04-07T16:12:13.038807+0000","last_active":"2024-04-07T16:13:38.187484+0000","last_peered":"2024-04-07T16:13:38.187484+0000","last_clean":"2024-04-07T16:13:38.187484+0000","last_became_active":"2024-04-07T16:12:13.038492+0000","last_became_peered":"2024-04-07T16:12:13.038492+0000","last_unstale":"2024-04-07T16:13:38.187484+0000","last_undegraded":"2024-04-07T16:13:38.187484+0000","last_fullsized":"2024-04-07T16:13:38.187484+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T02:11:03.746017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.031268+0000","last_change":"2024-04-07T16:12:13.410031+0000","last_active":"2024-04-07T16:13:38.031268+0000","last_peered":"2024-04-07T16:13:38.031268+0000","last_clean":"2024-04-07T16:13:38.031268+0000","last_became_active":"2024-04-07T16:12:13.409049+0000","last_became_peered":"2024-04-07T16:12:13.409049+0000","last_unstale":"2024-04-07T16:13:38.031268+0000","last_undegraded":"2024-04-07T16:13:38.031268+0000","last_fullsized":"2024-04-07T16:13:38.031268+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T21:35:09.308017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"61'2","reported_seq":22,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.637577+0000","last_change":"2024-04-07T16:12:13.042401+0000","last_active":"2024-04-07T16:13:38.637577+0000","last_peered":"2024-04-07T16:13:38.637577+0000","last_clean":"2024-04-07T16:13:38.637577+0000","last_became_active":"2024-04-07T16:12:13.041970+0000","last_became_peered":"2024-04-07T16:12:13.041970+0000","last_unstale":"2024-04-07T16:13:38.637577+0000","last_undegraded":"2024-04-07T16:13:38.637577+0000","last_fullsized":"2024-04-07T16:13:38.637577+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T16:36:29.235347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.025650+0000","last_change":"2024-04-07T16:12:13.394718+0000","last_active":"2024-04-07T16:13:38.025650+0000","last_peered":"2024-04-07T16:13:38.025650+0000","last_clean":"2024-04-07T16:13:38.025650+0000","last_became_active":"2024-04-07T16:12:13.394271+0000","last_became_peered":"2024-04-07T16:12:13.394271+0000","last_unstale":"2024-04-07T16:13:38.025650+0000","last_undegraded":"2024-04-07T16:13:38.025650+0000","last_fullsized":"2024-04-07T16:13:38.025650+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T20:10:29.096403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"61'2","reported_seq":22,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.637065+0000","last_change":"2024-04-07T16:12:13.042391+0000","last_active":"2024-04-07T16:13:38.637065+0000","last_peered":"2024-04-07T16:13:38.637065+0000","last_clean":"2024-04-07T16:13:38.637065+0000","last_became_active":"2024-04-07T16:12:13.041919+0000","last_became_peered":"2024-04-07T16:12:13.041919+0000","last_unstale":"2024-04-07T16:13:38.637065+0000","last_undegraded":"2024-04-07T16:13:38.637065+0000","last_fullsized":"2024-04-07T16:13:38.637065+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T21:50:59.576615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":28,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.025658+0000","last_change":"2024-04-07T16:12:26.222601+0000","last_active":"2024-04-07T16:13:38.025658+0000","last_peered":"2024-04-07T16:13:38.025658+0000","last_clean":"2024-04-07T16:13:38.025658+0000","last_became_active":"2024-04-07T16:12:26.222249+0000","last_became_peered":"2024-04-07T16:12:26.222249+0000","last_unstale":"2024-04-07T16:13:38.025658+0000","last_undegraded":"2024-04-07T16:13:38.025658+0000","last_fullsized":"2024-04-07T16:13:38.025658+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T16:28:47.005643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"61'1","reported_seq":21,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.187224+0000","last_change":"2024-04-07T16:12:13.041880+0000","last_active":"2024-04-07T16:13:38.187224+0000","last_peered":"2024-04-07T16:13:38.187224+0000","last_clean":"2024-04-07T16:13:38.187224+0000","last_became_active":"2024-04-07T16:12:13.041409+0000","last_became_peered":"2024-04-07T16:12:13.041409+0000","last_unstale":"2024-04-07T16:13:38.187224+0000","last_undegraded":"2024-04-07T16:13:38.187224+0000","last_fullsized":"2024-04-07T16:13:38.187224+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T18:32:23.206985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:39.824041+0000","last_change":"2024-04-07T16:12:13.413664+0000","last_active":"2024-04-07T16:13:39.824041+0000","last_peered":"2024-04-07T16:13:39.824041+0000","last_clean":"2024-04-07T16:13:39.824041+0000","last_became_active":"2024-04-07T16:12:13.413311+0000","last_became_peered":"2024-04-07T16:12:13.413311+0000","last_unstale":"2024-04-07T16:13:39.824041+0000","last_undegraded":"2024-04-07T16:13:39.824041+0000","last_fullsized":"2024-04-07T16:13:39.824041+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T02:43:21.836307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"61'1","reported_seq":21,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.526629+0000","last_change":"2024-04-07T16:12:13.038960+0000","last_active":"2024-04-07T16:13:38.526629+0000","last_peered":"2024-04-07T16:13:38.526629+0000","last_clean":"2024-04-07T16:13:38.526629+0000","last_became_active":"2024-04-07T16:12:13.038410+0000","last_became_peered":"2024-04-07T16:12:13.038410+0000","last_unstale":"2024-04-07T16:13:38.526629+0000","last_undegraded":"2024-04-07T16:13:38.526629+0000","last_fullsized":"2024-04-07T16:13:38.526629+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T03:41:38.418312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.029545+0000","last_change":"2024-04-07T16:12:13.405634+0000","last_active":"2024-04-07T16:13:38.029545+0000","last_peered":"2024-04-07T16:13:38.029545+0000","last_clean":"2024-04-07T16:13:38.029545+0000","last_became_active":"2024-04-07T16:12:13.405069+0000","last_became_peered":"2024-04-07T16:12:13.405069+0000","last_unstale":"2024-04-07T16:13:38.029545+0000","last_undegraded":"2024-04-07T16:13:38.029545+0000","last_fullsized":"2024-04-07T16:13:38.029545+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T23:50:18.973629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":20,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.186860+0000","last_change":"2024-04-07T16:12:13.034420+0000","last_active":"2024-04-07T16:13:38.186860+0000","last_peered":"2024-04-07T16:13:38.186860+0000","last_clean":"2024-04-07T16:13:38.186860+0000","last_became_active":"2024-04-07T16:12:13.034013+0000","last_became_peered":"2024-04-07T16:12:13.034013+0000","last_unstale":"2024-04-07T16:13:38.186860+0000","last_undegraded":"2024-04-07T16:13:38.186860+0000","last_fullsized":"2024-04-07T16:13:38.186860+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T19:51:21.525865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.187269+0000","last_change":"2024-04-07T16:12:13.398531+0000","last_active":"2024-04-07T16:13:38.187269+0000","last_peered":"2024-04-07T16:13:38.187269+0000","last_clean":"2024-04-07T16:13:38.187269+0000","last_became_active":"2024-04-07T16:12:13.398017+0000","last_became_peered":"2024-04-07T16:12:13.398017+0000","last_unstale":"2024-04-07T16:13:38.187269+0000","last_undegraded":"2024-04-07T16:13:38.187269+0000","last_fullsized":"2024-04-07T16:13:38.187269+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T17:33:54.306902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":20,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.024099+0000","last_change":"2024-04-07T16:12:13.045443+0000","last_active":"2024-04-07T16:13:38.024099+0000","last_peered":"2024-04-07T16:13:38.024099+0000","last_clean":"2024-04-07T16:13:38.024099+0000","last_became_active":"2024-04-07T16:12:13.044808+0000","last_became_peered":"2024-04-07T16:12:13.044808+0000","last_unstale":"2024-04-07T16:13:38.024099+0000","last_undegraded":"2024-04-07T16:13:38.024099+0000","last_fullsized":"2024-04-07T16:13:38.024099+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T22:44:14.623116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.236151+0000","last_change":"2024-04-07T16:12:13.391166+0000","last_active":"2024-04-07T16:13:38.236151+0000","last_peered":"2024-04-07T16:13:38.236151+0000","last_clean":"2024-04-07T16:13:38.236151+0000","last_became_active":"2024-04-07T16:12:13.390643+0000","last_became_peered":"2024-04-07T16:12:13.390643+0000","last_unstale":"2024-04-07T16:13:38.236151+0000","last_undegraded":"2024-04-07T16:13:38.236151+0000","last_fullsized":"2024-04-07T16:13:38.236151+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T21:22:33.110408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":20,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.553074+0000","last_change":"2024-04-07T16:12:13.030149+0000","last_active":"2024-04-07T16:13:38.553074+0000","last_peered":"2024-04-07T16:13:38.553074+0000","last_clean":"2024-04-07T16:13:38.553074+0000","last_became_active":"2024-04-07T16:12:13.029525+0000","last_became_peered":"2024-04-07T16:12:13.029525+0000","last_unstale":"2024-04-07T16:13:38.553074+0000","last_undegraded":"2024-04-07T16:13:38.553074+0000","last_fullsized":"2024-04-07T16:13:38.553074+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T20:09:35.180264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.025897+0000","last_change":"2024-04-07T16:12:13.398411+0000","last_active":"2024-04-07T16:13:38.025897+0000","last_peered":"2024-04-07T16:13:38.025897+0000","last_clean":"2024-04-07T16:13:38.025897+0000","last_became_active":"2024-04-07T16:12:13.398065+0000","last_became_peered":"2024-04-07T16:12:13.398065+0000","last_unstale":"2024-04-07T16:13:38.025897+0000","last_undegraded":"2024-04-07T16:13:38.025897+0000","last_fullsized":"2024-04-07T16:13:38.025897+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T21:13:20.448127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":20,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.025656+0000","last_change":"2024-04-07T16:12:13.036777+0000","last_active":"2024-04-07T16:13:38.025656+0000","last_peered":"2024-04-07T16:13:38.025656+0000","last_clean":"2024-04-07T16:13:38.025656+0000","last_became_active":"2024-04-07T16:12:13.036322+0000","last_became_peered":"2024-04-07T16:12:13.036322+0000","last_unstale":"2024-04-07T16:13:38.025656+0000","last_undegraded":"2024-04-07T16:13:38.025656+0000","last_fullsized":"2024-04-07T16:13:38.025656+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T18:26:16.873044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.763837+0000","last_change":"2024-04-07T16:12:26.226313+0000","last_active":"2024-04-07T16:13:38.763837+0000","last_peered":"2024-04-07T16:13:38.763837+0000","last_clean":"2024-04-07T16:13:38.763837+0000","last_became_active":"2024-04-07T16:12:26.225343+0000","last_became_peered":"2024-04-07T16:12:26.225343+0000","last_unstale":"2024-04-07T16:13:38.763837+0000","last_undegraded":"2024-04-07T16:13:38.763837+0000","last_fullsized":"2024-04-07T16:13:38.763837+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T17:42:38.440812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3],"acting":[2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.15","version":"61'3","reported_seq":23,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.237334+0000","last_change":"2024-04-07T16:12:13.039273+0000","last_active":"2024-04-07T16:13:38.237334+0000","last_peered":"2024-04-07T16:13:38.237334+0000","last_clean":"2024-04-07T16:13:38.237334+0000","last_became_active":"2024-04-07T16:12:13.038911+0000","last_became_peered":"2024-04-07T16:12:13.038911+0000","last_unstale":"2024-04-07T16:13:38.237334+0000","last_undegraded":"2024-04-07T16:13:38.237334+0000","last_fullsized":"2024-04-07T16:13:38.237334+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T02:11:53.262707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:39.824449+0000","last_change":"2024-04-07T16:12:13.391349+0000","last_active":"2024-04-07T16:13:39.824449+0000","last_peered":"2024-04-07T16:13:39.824449+0000","last_clean":"2024-04-07T16:13:39.824449+0000","last_became_active":"2024-04-07T16:12:13.390960+0000","last_became_peered":"2024-04-07T16:12:13.390960+0000","last_unstale":"2024-04-07T16:13:39.824449+0000","last_undegraded":"2024-04-07T16:13:39.824449+0000","last_fullsized":"2024-04-07T16:13:39.824449+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T16:55:37.429799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"69'12","reported_seq":32,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:53.034681+0000","last_change":"2024-04-07T16:12:13.031673+0000","last_active":"2024-04-07T16:13:53.034681+0000","last_peered":"2024-04-07T16:13:53.034681+0000","last_clean":"2024-04-07T16:13:53.034681+0000","last_became_active":"2024-04-07T16:12:13.031092+0000","last_became_peered":"2024-04-07T16:12:13.031092+0000","last_unstale":"2024-04-07T16:13:53.034681+0000","last_undegraded":"2024-04-07T16:13:53.034681+0000","last_fullsized":"2024-04-07T16:13:53.034681+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T20:17:48.593862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2632,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":28,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.635967+0000","last_change":"2024-04-07T16:12:26.223406+0000","last_active":"2024-04-07T16:13:38.635967+0000","last_peered":"2024-04-07T16:13:38.635967+0000","last_clean":"2024-04-07T16:13:38.635967+0000","last_became_active":"2024-04-07T16:12:26.222663+0000","last_became_peered":"2024-04-07T16:12:26.222663+0000","last_unstale":"2024-04-07T16:13:38.635967+0000","last_undegraded":"2024-04-07T16:13:38.635967+0000","last_fullsized":"2024-04-07T16:13:38.635967+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T19:54:33.089377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"61'2","reported_seq":22,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.029978+0000","last_change":"2024-04-07T16:12:13.033352+0000","last_active":"2024-04-07T16:13:38.029978+0000","last_peered":"2024-04-07T16:13:38.029978+0000","last_clean":"2024-04-07T16:13:38.029978+0000","last_became_active":"2024-04-07T16:12:13.032811+0000","last_became_peered":"2024-04-07T16:12:13.032811+0000","last_unstale":"2024-04-07T16:13:38.029978+0000","last_undegraded":"2024-04-07T16:13:38.029978+0000","last_fullsized":"2024-04-07T16:13:38.029978+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T01:50:02.195507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.763887+0000","last_change":"2024-04-07T16:12:26.226163+0000","last_active":"2024-04-07T16:13:38.763887+0000","last_peered":"2024-04-07T16:13:38.763887+0000","last_clean":"2024-04-07T16:13:38.763887+0000","last_became_active":"2024-04-07T16:12:26.224225+0000","last_became_peered":"2024-04-07T16:12:26.224225+0000","last_unstale":"2024-04-07T16:13:38.763887+0000","last_undegraded":"2024-04-07T16:13:38.763887+0000","last_fullsized":"2024-04-07T16:13:38.763887+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T23:54:31.223096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,3],"acting":[2,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":20,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.024421+0000","last_change":"2024-04-07T16:12:13.045494+0000","last_active":"2024-04-07T16:13:38.024421+0000","last_peered":"2024-04-07T16:13:38.024421+0000","last_clean":"2024-04-07T16:13:38.024421+0000","last_became_active":"2024-04-07T16:12:13.045086+0000","last_became_peered":"2024-04-07T16:12:13.045086+0000","last_unstale":"2024-04-07T16:13:38.024421+0000","last_undegraded":"2024-04-07T16:13:38.024421+0000","last_fullsized":"2024-04-07T16:13:38.024421+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T23:15:11.494437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.637043+0000","last_change":"2024-04-07T16:12:13.413928+0000","last_active":"2024-04-07T16:13:38.637043+0000","last_peered":"2024-04-07T16:13:38.637043+0000","last_clean":"2024-04-07T16:13:38.637043+0000","last_became_active":"2024-04-07T16:12:13.413447+0000","last_became_peered":"2024-04-07T16:12:13.413447+0000","last_unstale":"2024-04-07T16:13:38.637043+0000","last_undegraded":"2024-04-07T16:13:38.637043+0000","last_fullsized":"2024-04-07T16:13:38.637043+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T03:26:42.875228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":20,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.526216+0000","last_change":"2024-04-07T16:12:13.032942+0000","last_active":"2024-04-07T16:13:38.526216+0000","last_peered":"2024-04-07T16:13:38.526216+0000","last_clean":"2024-04-07T16:13:38.526216+0000","last_became_active":"2024-04-07T16:12:13.032108+0000","last_became_peered":"2024-04-07T16:12:13.032108+0000","last_unstale":"2024-04-07T16:13:38.526216+0000","last_undegraded":"2024-04-07T16:13:38.526216+0000","last_fullsized":"2024-04-07T16:13:38.526216+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T18:53:06.864412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.525876+0000","last_change":"2024-04-07T16:12:13.400141+0000","last_active":"2024-04-07T16:13:38.525876+0000","last_peered":"2024-04-07T16:13:38.525876+0000","last_clean":"2024-04-07T16:13:38.525876+0000","last_became_active":"2024-04-07T16:12:13.399823+0000","last_became_peered":"2024-04-07T16:12:13.399823+0000","last_unstale":"2024-04-07T16:13:38.525876+0000","last_undegraded":"2024-04-07T16:13:38.525876+0000","last_fullsized":"2024-04-07T16:13:38.525876+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T21:27:25.248595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"61'8","reported_seq":28,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.764170+0000","last_change":"2024-04-07T16:12:13.034154+0000","last_active":"2024-04-07T16:13:38.764170+0000","last_peered":"2024-04-07T16:13:38.764170+0000","last_clean":"2024-04-07T16:13:38.764170+0000","last_became_active":"2024-04-07T16:12:13.033838+0000","last_became_peered":"2024-04-07T16:12:13.033838+0000","last_unstale":"2024-04-07T16:13:38.764170+0000","last_undegraded":"2024-04-07T16:13:38.764170+0000","last_fullsized":"2024-04-07T16:13:38.764170+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T23:13:07.216314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.636593+0000","last_change":"2024-04-07T16:12:13.385883+0000","last_active":"2024-04-07T16:13:38.636593+0000","last_peered":"2024-04-07T16:13:38.636593+0000","last_clean":"2024-04-07T16:13:38.636593+0000","last_became_active":"2024-04-07T16:12:13.385485+0000","last_became_peered":"2024-04-07T16:12:13.385485+0000","last_unstale":"2024-04-07T16:13:38.636593+0000","last_undegraded":"2024-04-07T16:13:38.636593+0000","last_fullsized":"2024-04-07T16:13:38.636593+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T21:01:09.262433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"61'1","reported_seq":21,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:39.825314+0000","last_change":"2024-04-07T16:12:13.028004+0000","last_active":"2024-04-07T16:13:39.825314+0000","last_peered":"2024-04-07T16:13:39.825314+0000","last_clean":"2024-04-07T16:13:39.825314+0000","last_became_active":"2024-04-07T16:12:13.027445+0000","last_became_peered":"2024-04-07T16:12:13.027445+0000","last_unstale":"2024-04-07T16:13:39.825314+0000","last_undegraded":"2024-04-07T16:13:39.825314+0000","last_fullsized":"2024-04-07T16:13:39.825314+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T17:55:06.727142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.637723+0000","last_change":"2024-04-07T16:12:13.386584+0000","last_active":"2024-04-07T16:13:38.637723+0000","last_peered":"2024-04-07T16:13:38.637723+0000","last_clean":"2024-04-07T16:13:38.637723+0000","last_became_active":"2024-04-07T16:12:13.386057+0000","last_became_peered":"2024-04-07T16:12:13.386057+0000","last_unstale":"2024-04-07T16:13:38.637723+0000","last_undegraded":"2024-04-07T16:13:38.637723+0000","last_fullsized":"2024-04-07T16:13:38.637723+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T17:57:57.548179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-07T16:13:33.673641+0000","last_change":"2024-04-07T16:13:30.009561+0000","last_active":"2024-04-07T16:13:33.673641+0000","last_peered":"2024-04-07T16:13:33.673641+0000","last_clean":"2024-04-07T16:13:33.673641+0000","last_became_active":"2024-04-07T16:13:30.008706+0000","last_became_peered":"2024-04-07T16:13:30.008706+0000","last_unstale":"2024-04-07T16:13:33.673641+0000","last_undegraded":"2024-04-07T16:13:33.673641+0000","last_fullsized":"2024-04-07T16:13:33.673641+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:13:28.963534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:13:28.963534+0000","last_clean_scrub_stamp":"2024-04-07T16:13:28.963534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T00:36:26.958742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":20,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.236743+0000","last_change":"2024-04-07T16:12:13.037974+0000","last_active":"2024-04-07T16:13:38.236743+0000","last_peered":"2024-04-07T16:13:38.236743+0000","last_clean":"2024-04-07T16:13:38.236743+0000","last_became_active":"2024-04-07T16:12:13.037627+0000","last_became_peered":"2024-04-07T16:12:13.037627+0000","last_unstale":"2024-04-07T16:13:38.236743+0000","last_undegraded":"2024-04-07T16:13:38.236743+0000","last_fullsized":"2024-04-07T16:13:38.236743+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T19:28:12.174861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.025595+0000","last_change":"2024-04-07T16:12:13.395073+0000","last_active":"2024-04-07T16:13:38.025595+0000","last_peered":"2024-04-07T16:13:38.025595+0000","last_clean":"2024-04-07T16:13:38.025595+0000","last_became_active":"2024-04-07T16:12:13.394113+0000","last_became_peered":"2024-04-07T16:12:13.394113+0000","last_unstale":"2024-04-07T16:13:38.025595+0000","last_undegraded":"2024-04-07T16:13:38.025595+0000","last_fullsized":"2024-04-07T16:13:38.025595+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T23:28:32.978580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-07T16:13:33.673705+0000","last_change":"2024-04-07T16:13:30.041147+0000","last_active":"2024-04-07T16:13:33.673705+0000","last_peered":"2024-04-07T16:13:33.673705+0000","last_clean":"2024-04-07T16:13:33.673705+0000","last_became_active":"2024-04-07T16:13:30.040318+0000","last_became_peered":"2024-04-07T16:13:30.040318+0000","last_unstale":"2024-04-07T16:13:33.673705+0000","last_undegraded":"2024-04-07T16:13:33.673705+0000","last_fullsized":"2024-04-07T16:13:33.673705+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:13:28.963534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:13:28.963534+0000","last_clean_scrub_stamp":"2024-04-07T16:13:28.963534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T20:45:42.413572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":20,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.024311+0000","last_change":"2024-04-07T16:12:13.036336+0000","last_active":"2024-04-07T16:13:38.024311+0000","last_peered":"2024-04-07T16:13:38.024311+0000","last_clean":"2024-04-07T16:13:38.024311+0000","last_became_active":"2024-04-07T16:12:13.035625+0000","last_became_peered":"2024-04-07T16:12:13.035625+0000","last_unstale":"2024-04-07T16:13:38.024311+0000","last_undegraded":"2024-04-07T16:13:38.024311+0000","last_fullsized":"2024-04-07T16:13:38.024311+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T18:15:00.506827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:39.824462+0000","last_change":"2024-04-07T16:12:13.404406+0000","last_active":"2024-04-07T16:13:39.824462+0000","last_peered":"2024-04-07T16:13:39.824462+0000","last_clean":"2024-04-07T16:13:39.824462+0000","last_became_active":"2024-04-07T16:12:13.404029+0000","last_became_peered":"2024-04-07T16:12:13.404029+0000","last_unstale":"2024-04-07T16:13:39.824462+0000","last_undegraded":"2024-04-07T16:13:39.824462+0000","last_fullsized":"2024-04-07T16:13:39.824462+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T16:41:25.469343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-07T16:13:33.673820+0000","last_change":"2024-04-07T16:13:30.012072+0000","last_active":"2024-04-07T16:13:33.673820+0000","last_peered":"2024-04-07T16:13:33.673820+0000","last_clean":"2024-04-07T16:13:33.673820+0000","last_became_active":"2024-04-07T16:13:30.011181+0000","last_became_peered":"2024-04-07T16:13:30.011181+0000","last_unstale":"2024-04-07T16:13:33.673820+0000","last_undegraded":"2024-04-07T16:13:33.673820+0000","last_fullsized":"2024-04-07T16:13:33.673820+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:13:28.963534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:13:28.963534+0000","last_clean_scrub_stamp":"2024-04-07T16:13:28.963534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T21:19:09.530105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.235831+0000","last_change":"2024-04-07T16:12:13.393979+0000","last_active":"2024-04-07T16:13:38.235831+0000","last_peered":"2024-04-07T16:13:38.235831+0000","last_clean":"2024-04-07T16:13:38.235831+0000","last_became_active":"2024-04-07T16:12:13.393472+0000","last_became_peered":"2024-04-07T16:12:13.393472+0000","last_unstale":"2024-04-07T16:13:38.235831+0000","last_undegraded":"2024-04-07T16:13:38.235831+0000","last_fullsized":"2024-04-07T16:13:38.235831+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T21:51:13.834107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":20,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.764382+0000","last_change":"2024-04-07T16:12:13.035071+0000","last_active":"2024-04-07T16:13:38.764382+0000","last_peered":"2024-04-07T16:13:38.764382+0000","last_clean":"2024-04-07T16:13:38.764382+0000","last_became_active":"2024-04-07T16:12:13.034795+0000","last_became_peered":"2024-04-07T16:12:13.034795+0000","last_unstale":"2024-04-07T16:13:38.764382+0000","last_undegraded":"2024-04-07T16:13:38.764382+0000","last_fullsized":"2024-04-07T16:13:38.764382+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T17:51:30.599527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-07T16:13:33.674725+0000","last_change":"2024-04-07T16:13:30.035513+0000","last_active":"2024-04-07T16:13:33.674725+0000","last_peered":"2024-04-07T16:13:33.674725+0000","last_clean":"2024-04-07T16:13:33.674725+0000","last_became_active":"2024-04-07T16:13:30.035058+0000","last_became_peered":"2024-04-07T16:13:30.035058+0000","last_unstale":"2024-04-07T16:13:33.674725+0000","last_undegraded":"2024-04-07T16:13:33.674725+0000","last_fullsized":"2024-04-07T16:13:33.674725+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:13:28.963534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:13:28.963534+0000","last_clean_scrub_stamp":"2024-04-07T16:13:28.963534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T22:41:30.257805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.186788+0000","last_change":"2024-04-07T16:12:13.394635+0000","last_active":"2024-04-07T16:13:38.186788+0000","last_peered":"2024-04-07T16:13:38.186788+0000","last_clean":"2024-04-07T16:13:38.186788+0000","last_became_active":"2024-04-07T16:12:13.394392+0000","last_became_peered":"2024-04-07T16:12:13.394392+0000","last_unstale":"2024-04-07T16:13:38.186788+0000","last_undegraded":"2024-04-07T16:13:38.186788+0000","last_fullsized":"2024-04-07T16:13:38.186788+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T17:52:10.950481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":20,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.024445+0000","last_change":"2024-04-07T16:12:13.045131+0000","last_active":"2024-04-07T16:13:38.024445+0000","last_peered":"2024-04-07T16:13:38.024445+0000","last_clean":"2024-04-07T16:13:38.024445+0000","last_became_active":"2024-04-07T16:12:13.044830+0000","last_became_peered":"2024-04-07T16:12:13.044830+0000","last_unstale":"2024-04-07T16:13:38.024445+0000","last_undegraded":"2024-04-07T16:13:38.024445+0000","last_fullsized":"2024-04-07T16:13:38.024445+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T02:48:50.652491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"65'128","reported_seq":266,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.636317+0000","last_change":"2024-04-07T16:11:33.518868+0000","last_active":"2024-04-07T16:13:38.636317+0000","last_peered":"2024-04-07T16:13:38.636317+0000","last_clean":"2024-04-07T16:13:38.636317+0000","last_became_active":"2024-04-07T16:11:33.518060+0000","last_became_peered":"2024-04-07T16:11:33.518060+0000","last_unstale":"2024-04-07T16:13:38.636317+0000","last_undegraded":"2024-04-07T16:13:38.636317+0000","last_fullsized":"2024-04-07T16:13:38.636317+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:07:26.866003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:07:26.866003+0000","last_clean_scrub_stamp":"2024-04-07T16:07:26.866003+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T22:22:51.043941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-07T16:13:33.672600+0000","last_change":"2024-04-07T16:13:30.041000+0000","last_active":"2024-04-07T16:13:33.672600+0000","last_peered":"2024-04-07T16:13:33.672600+0000","last_clean":"2024-04-07T16:13:33.672600+0000","last_became_active":"2024-04-07T16:13:30.040114+0000","last_became_peered":"2024-04-07T16:13:30.040114+0000","last_unstale":"2024-04-07T16:13:33.672600+0000","last_undegraded":"2024-04-07T16:13:33.672600+0000","last_fullsized":"2024-04-07T16:13:33.672600+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:13:28.963534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:13:28.963534+0000","last_clean_scrub_stamp":"2024-04-07T16:13:28.963534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T03:03:13.375776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.763809+0000","last_change":"2024-04-07T16:12:26.226945+0000","last_active":"2024-04-07T16:13:38.763809+0000","last_peered":"2024-04-07T16:13:38.763809+0000","last_clean":"2024-04-07T16:13:38.763809+0000","last_became_active":"2024-04-07T16:12:26.225518+0000","last_became_peered":"2024-04-07T16:12:26.225518+0000","last_unstale":"2024-04-07T16:13:38.763809+0000","last_undegraded":"2024-04-07T16:13:38.763809+0000","last_fullsized":"2024-04-07T16:13:38.763809+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T22:14:39.830063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":20,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.636018+0000","last_change":"2024-04-07T16:12:13.035536+0000","last_active":"2024-04-07T16:13:38.636018+0000","last_peered":"2024-04-07T16:13:38.636018+0000","last_clean":"2024-04-07T16:13:38.636018+0000","last_became_active":"2024-04-07T16:12:13.034825+0000","last_became_peered":"2024-04-07T16:12:13.034825+0000","last_unstale":"2024-04-07T16:13:38.636018+0000","last_undegraded":"2024-04-07T16:13:38.636018+0000","last_fullsized":"2024-04-07T16:13:38.636018+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T20:11:57.033306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-07T16:13:33.672484+0000","last_change":"2024-04-07T16:13:30.041281+0000","last_active":"2024-04-07T16:13:33.672484+0000","last_peered":"2024-04-07T16:13:33.672484+0000","last_clean":"2024-04-07T16:13:33.672484+0000","last_became_active":"2024-04-07T16:13:30.040143+0000","last_became_peered":"2024-04-07T16:13:30.040143+0000","last_unstale":"2024-04-07T16:13:33.672484+0000","last_undegraded":"2024-04-07T16:13:33.672484+0000","last_fullsized":"2024-04-07T16:13:33.672484+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:13:28.963534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:13:28.963534+0000","last_clean_scrub_stamp":"2024-04-07T16:13:28.963534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T01:04:02.813753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":20,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.637358+0000","last_change":"2024-04-07T16:12:13.035747+0000","last_active":"2024-04-07T16:13:38.637358+0000","last_peered":"2024-04-07T16:13:38.637358+0000","last_clean":"2024-04-07T16:13:38.637358+0000","last_became_active":"2024-04-07T16:12:13.035201+0000","last_became_peered":"2024-04-07T16:12:13.035201+0000","last_unstale":"2024-04-07T16:13:38.637358+0000","last_undegraded":"2024-04-07T16:13:38.637358+0000","last_fullsized":"2024-04-07T16:13:38.637358+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T21:03:25.449728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.237019+0000","last_change":"2024-04-07T16:12:13.398785+0000","last_active":"2024-04-07T16:13:38.237019+0000","last_peered":"2024-04-07T16:13:38.237019+0000","last_clean":"2024-04-07T16:13:38.237019+0000","last_became_active":"2024-04-07T16:12:13.398210+0000","last_became_peered":"2024-04-07T16:12:13.398210+0000","last_unstale":"2024-04-07T16:13:38.237019+0000","last_undegraded":"2024-04-07T16:13:38.237019+0000","last_fullsized":"2024-04-07T16:13:38.237019+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T02:11:50.752075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-07T16:13:33.680261+0000","last_change":"2024-04-07T16:13:30.040851+0000","last_active":"2024-04-07T16:13:33.680261+0000","last_peered":"2024-04-07T16:13:33.680261+0000","last_clean":"2024-04-07T16:13:33.680261+0000","last_became_active":"2024-04-07T16:13:30.040026+0000","last_became_peered":"2024-04-07T16:13:30.040026+0000","last_unstale":"2024-04-07T16:13:33.680261+0000","last_undegraded":"2024-04-07T16:13:33.680261+0000","last_fullsized":"2024-04-07T16:13:33.680261+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:13:28.963534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:13:28.963534+0000","last_clean_scrub_stamp":"2024-04-07T16:13:28.963534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T18:29:22.422209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"62'1","reported_seq":21,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.237606+0000","last_change":"2024-04-07T16:12:13.039980+0000","last_active":"2024-04-07T16:13:38.237606+0000","last_peered":"2024-04-07T16:13:38.237606+0000","last_clean":"2024-04-07T16:13:38.237606+0000","last_became_active":"2024-04-07T16:12:13.039676+0000","last_became_peered":"2024-04-07T16:12:13.039676+0000","last_unstale":"2024-04-07T16:13:38.237606+0000","last_undegraded":"2024-04-07T16:13:38.237606+0000","last_fullsized":"2024-04-07T16:13:38.237606+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T02:37:05.297496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.187224+0000","last_change":"2024-04-07T16:12:13.399486+0000","last_active":"2024-04-07T16:13:38.187224+0000","last_peered":"2024-04-07T16:13:38.187224+0000","last_clean":"2024-04-07T16:13:38.187224+0000","last_became_active":"2024-04-07T16:12:13.399184+0000","last_became_peered":"2024-04-07T16:12:13.399184+0000","last_unstale":"2024-04-07T16:13:38.187224+0000","last_undegraded":"2024-04-07T16:13:38.187224+0000","last_fullsized":"2024-04-07T16:13:38.187224+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T21:07:25.104309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-07T16:13:33.668617+0000","last_change":"2024-04-07T16:13:30.003468+0000","last_active":"2024-04-07T16:13:33.668617+0000","last_peered":"2024-04-07T16:13:33.668617+0000","last_clean":"2024-04-07T16:13:33.668617+0000","last_became_active":"2024-04-07T16:13:30.002945+0000","last_became_peered":"2024-04-07T16:13:30.002945+0000","last_unstale":"2024-04-07T16:13:33.668617+0000","last_undegraded":"2024-04-07T16:13:33.668617+0000","last_fullsized":"2024-04-07T16:13:33.668617+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:13:28.963534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:13:28.963534+0000","last_clean_scrub_stamp":"2024-04-07T16:13:28.963534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T03:30:42.844279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"61'2","reported_seq":35,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.552993+0000","last_change":"2024-04-07T16:12:26.091787+0000","last_active":"2024-04-07T16:13:38.552993+0000","last_peered":"2024-04-07T16:13:38.552993+0000","last_clean":"2024-04-07T16:13:38.552993+0000","last_became_active":"2024-04-07T16:12:26.084985+0000","last_became_peered":"2024-04-07T16:12:26.084985+0000","last_unstale":"2024-04-07T16:13:38.552993+0000","last_undegraded":"2024-04-07T16:13:38.552993+0000","last_fullsized":"2024-04-07T16:13:38.552993+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T02:39:40.077566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.031201+0000","last_change":"2024-04-07T16:12:13.406541+0000","last_active":"2024-04-07T16:13:38.031201+0000","last_peered":"2024-04-07T16:13:38.031201+0000","last_clean":"2024-04-07T16:13:38.031201+0000","last_became_active":"2024-04-07T16:12:13.406018+0000","last_became_peered":"2024-04-07T16:12:13.406018+0000","last_unstale":"2024-04-07T16:13:38.031201+0000","last_undegraded":"2024-04-07T16:13:38.031201+0000","last_fullsized":"2024-04-07T16:13:38.031201+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T00:24:42.112088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":20,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.636513+0000","last_change":"2024-04-07T16:12:13.033067+0000","last_active":"2024-04-07T16:13:38.636513+0000","last_peered":"2024-04-07T16:13:38.636513+0000","last_clean":"2024-04-07T16:13:38.636513+0000","last_became_active":"2024-04-07T16:12:13.032456+0000","last_became_peered":"2024-04-07T16:12:13.032456+0000","last_unstale":"2024-04-07T16:13:38.636513+0000","last_undegraded":"2024-04-07T16:13:38.636513+0000","last_fullsized":"2024-04-07T16:13:38.636513+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T23:02:09.025617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:39.824230+0000","last_change":"2024-04-07T16:12:13.404632+0000","last_active":"2024-04-07T16:13:39.824230+0000","last_peered":"2024-04-07T16:13:39.824230+0000","last_clean":"2024-04-07T16:13:39.824230+0000","last_became_active":"2024-04-07T16:12:13.404321+0000","last_became_peered":"2024-04-07T16:12:13.404321+0000","last_unstale":"2024-04-07T16:13:39.824230+0000","last_undegraded":"2024-04-07T16:13:39.824230+0000","last_fullsized":"2024-04-07T16:13:39.824230+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T19:53:50.339257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":20,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.236054+0000","last_change":"2024-04-07T16:12:13.038113+0000","last_active":"2024-04-07T16:13:38.236054+0000","last_peered":"2024-04-07T16:13:38.236054+0000","last_clean":"2024-04-07T16:13:38.236054+0000","last_became_active":"2024-04-07T16:12:13.037783+0000","last_became_peered":"2024-04-07T16:12:13.037783+0000","last_unstale":"2024-04-07T16:13:38.236054+0000","last_undegraded":"2024-04-07T16:13:38.236054+0000","last_fullsized":"2024-04-07T16:13:38.236054+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T00:58:32.085451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.187183+0000","last_change":"2024-04-07T16:12:13.392396+0000","last_active":"2024-04-07T16:13:38.187183+0000","last_peered":"2024-04-07T16:13:38.187183+0000","last_clean":"2024-04-07T16:13:38.187183+0000","last_became_active":"2024-04-07T16:12:13.392016+0000","last_became_peered":"2024-04-07T16:12:13.392016+0000","last_unstale":"2024-04-07T16:13:38.187183+0000","last_undegraded":"2024-04-07T16:13:38.187183+0000","last_fullsized":"2024-04-07T16:13:38.187183+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T02:35:50.790051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":20,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.237136+0000","last_change":"2024-04-07T16:12:13.038401+0000","last_active":"2024-04-07T16:13:38.237136+0000","last_peered":"2024-04-07T16:13:38.237136+0000","last_clean":"2024-04-07T16:13:38.237136+0000","last_became_active":"2024-04-07T16:12:13.038110+0000","last_became_peered":"2024-04-07T16:12:13.038110+0000","last_unstale":"2024-04-07T16:13:38.237136+0000","last_undegraded":"2024-04-07T16:13:38.237136+0000","last_fullsized":"2024-04-07T16:13:38.237136+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T01:16:50.295588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.186955+0000","last_change":"2024-04-07T16:12:13.397819+0000","last_active":"2024-04-07T16:13:38.186955+0000","last_peered":"2024-04-07T16:13:38.186955+0000","last_clean":"2024-04-07T16:13:38.186955+0000","last_became_active":"2024-04-07T16:12:13.397593+0000","last_became_peered":"2024-04-07T16:12:13.397593+0000","last_unstale":"2024-04-07T16:13:38.186955+0000","last_undegraded":"2024-04-07T16:13:38.186955+0000","last_fullsized":"2024-04-07T16:13:38.186955+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T02:30:01.472030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"61'2","reported_seq":22,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.636990+0000","last_change":"2024-04-07T16:12:13.043982+0000","last_active":"2024-04-07T16:13:38.636990+0000","last_peered":"2024-04-07T16:13:38.636990+0000","last_clean":"2024-04-07T16:13:38.636990+0000","last_became_active":"2024-04-07T16:12:13.041873+0000","last_became_peered":"2024-04-07T16:12:13.041873+0000","last_unstale":"2024-04-07T16:13:38.636990+0000","last_undegraded":"2024-04-07T16:13:38.636990+0000","last_fullsized":"2024-04-07T16:13:38.636990+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T20:03:35.575133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.526346+0000","last_change":"2024-04-07T16:12:13.381810+0000","last_active":"2024-04-07T16:13:38.526346+0000","last_peered":"2024-04-07T16:13:38.526346+0000","last_clean":"2024-04-07T16:13:38.526346+0000","last_became_active":"2024-04-07T16:12:13.381470+0000","last_became_peered":"2024-04-07T16:12:13.381470+0000","last_unstale":"2024-04-07T16:13:38.526346+0000","last_undegraded":"2024-04-07T16:13:38.526346+0000","last_fullsized":"2024-04-07T16:13:38.526346+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T17:26:21.450939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":20,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.764853+0000","last_change":"2024-04-07T16:12:13.038191+0000","last_active":"2024-04-07T16:13:38.764853+0000","last_peered":"2024-04-07T16:13:38.764853+0000","last_clean":"2024-04-07T16:13:38.764853+0000","last_became_active":"2024-04-07T16:12:13.037652+0000","last_became_peered":"2024-04-07T16:12:13.037652+0000","last_unstale":"2024-04-07T16:13:38.764853+0000","last_undegraded":"2024-04-07T16:13:38.764853+0000","last_fullsized":"2024-04-07T16:13:38.764853+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T17:04:59.525025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.637269+0000","last_change":"2024-04-07T16:12:13.400660+0000","last_active":"2024-04-07T16:13:38.637269+0000","last_peered":"2024-04-07T16:13:38.637269+0000","last_clean":"2024-04-07T16:13:38.637269+0000","last_became_active":"2024-04-07T16:12:13.400353+0000","last_became_peered":"2024-04-07T16:12:13.400353+0000","last_unstale":"2024-04-07T16:13:38.637269+0000","last_undegraded":"2024-04-07T16:13:38.637269+0000","last_fullsized":"2024-04-07T16:13:38.637269+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T00:47:57.573097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"61'1","reported_seq":21,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.237152+0000","last_change":"2024-04-07T16:12:13.034772+0000","last_active":"2024-04-07T16:13:38.237152+0000","last_peered":"2024-04-07T16:13:38.237152+0000","last_clean":"2024-04-07T16:13:38.237152+0000","last_became_active":"2024-04-07T16:12:13.034455+0000","last_became_peered":"2024-04-07T16:12:13.034455+0000","last_unstale":"2024-04-07T16:13:38.237152+0000","last_undegraded":"2024-04-07T16:13:38.237152+0000","last_fullsized":"2024-04-07T16:13:38.237152+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T23:10:30.557880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.025357+0000","last_change":"2024-04-07T16:12:13.400762+0000","last_active":"2024-04-07T16:13:38.025357+0000","last_peered":"2024-04-07T16:13:38.025357+0000","last_clean":"2024-04-07T16:13:38.025357+0000","last_became_active":"2024-04-07T16:12:13.398120+0000","last_became_peered":"2024-04-07T16:12:13.398120+0000","last_unstale":"2024-04-07T16:13:38.025357+0000","last_undegraded":"2024-04-07T16:13:38.025357+0000","last_fullsized":"2024-04-07T16:13:38.025357+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T22:11:21.789998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"61'2","reported_seq":22,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.764862+0000","last_change":"2024-04-07T16:12:13.040101+0000","last_active":"2024-04-07T16:13:38.764862+0000","last_peered":"2024-04-07T16:13:38.764862+0000","last_clean":"2024-04-07T16:13:38.764862+0000","last_became_active":"2024-04-07T16:12:13.039797+0000","last_became_peered":"2024-04-07T16:12:13.039797+0000","last_unstale":"2024-04-07T16:13:38.764862+0000","last_undegraded":"2024-04-07T16:13:38.764862+0000","last_fullsized":"2024-04-07T16:13:38.764862+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T03:27:06.016325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.637921+0000","last_change":"2024-04-07T16:12:13.394720+0000","last_active":"2024-04-07T16:13:38.637921+0000","last_peered":"2024-04-07T16:13:38.637921+0000","last_clean":"2024-04-07T16:13:38.637921+0000","last_became_active":"2024-04-07T16:12:13.394090+0000","last_became_peered":"2024-04-07T16:12:13.394090+0000","last_unstale":"2024-04-07T16:13:38.637921+0000","last_undegraded":"2024-04-07T16:13:38.637921+0000","last_fullsized":"2024-04-07T16:13:38.637921+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T00:47:54.579168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.525708+0000","last_change":"2024-04-07T16:12:13.396725+0000","last_active":"2024-04-07T16:13:38.525708+0000","last_peered":"2024-04-07T16:13:38.525708+0000","last_clean":"2024-04-07T16:13:38.525708+0000","last_became_active":"2024-04-07T16:12:13.395978+0000","last_became_peered":"2024-04-07T16:12:13.395978+0000","last_unstale":"2024-04-07T16:13:38.525708+0000","last_undegraded":"2024-04-07T16:13:38.525708+0000","last_fullsized":"2024-04-07T16:13:38.525708+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T21:30:43.276569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.764525+0000","last_change":"2024-04-07T16:12:13.412091+0000","last_active":"2024-04-07T16:13:38.764525+0000","last_peered":"2024-04-07T16:13:38.764525+0000","last_clean":"2024-04-07T16:13:38.764525+0000","last_became_active":"2024-04-07T16:12:13.410814+0000","last_became_peered":"2024-04-07T16:12:13.410814+0000","last_unstale":"2024-04-07T16:13:38.764525+0000","last_undegraded":"2024-04-07T16:13:38.764525+0000","last_fullsized":"2024-04-07T16:13:38.764525+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T01:48:29.034291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.024540+0000","last_change":"2024-04-07T16:12:13.400651+0000","last_active":"2024-04-07T16:13:38.024540+0000","last_peered":"2024-04-07T16:13:38.024540+0000","last_clean":"2024-04-07T16:13:38.024540+0000","last_became_active":"2024-04-07T16:12:13.399206+0000","last_became_peered":"2024-04-07T16:12:13.399206+0000","last_unstale":"2024-04-07T16:13:38.024540+0000","last_undegraded":"2024-04-07T16:13:38.024540+0000","last_fullsized":"2024-04-07T16:13:38.024540+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T22:16:50.256754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,0],"acting":[5,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.029689+0000","last_change":"2024-04-07T16:12:13.405888+0000","last_active":"2024-04-07T16:13:38.029689+0000","last_peered":"2024-04-07T16:13:38.029689+0000","last_clean":"2024-04-07T16:13:38.029689+0000","last_became_active":"2024-04-07T16:12:13.405320+0000","last_became_peered":"2024-04-07T16:12:13.405320+0000","last_unstale":"2024-04-07T16:13:38.029689+0000","last_undegraded":"2024-04-07T16:13:38.029689+0000","last_fullsized":"2024-04-07T16:13:38.029689+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T22:07:50.421903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.186832+0000","last_change":"2024-04-07T16:12:13.398730+0000","last_active":"2024-04-07T16:13:38.186832+0000","last_peered":"2024-04-07T16:13:38.186832+0000","last_clean":"2024-04-07T16:13:38.186832+0000","last_became_active":"2024-04-07T16:12:13.398259+0000","last_became_peered":"2024-04-07T16:12:13.398259+0000","last_unstale":"2024-04-07T16:13:38.186832+0000","last_undegraded":"2024-04-07T16:13:38.186832+0000","last_fullsized":"2024-04-07T16:13:38.186832+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T02:14:28.915631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,0],"acting":[3,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.236896+0000","last_change":"2024-04-07T16:12:13.391140+0000","last_active":"2024-04-07T16:13:38.236896+0000","last_peered":"2024-04-07T16:13:38.236896+0000","last_clean":"2024-04-07T16:13:38.236896+0000","last_became_active":"2024-04-07T16:12:13.390858+0000","last_became_peered":"2024-04-07T16:12:13.390858+0000","last_unstale":"2024-04-07T16:13:38.236896+0000","last_undegraded":"2024-04-07T16:13:38.236896+0000","last_fullsized":"2024-04-07T16:13:38.236896+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T22:44:20.569791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.553144+0000","last_change":"2024-04-07T16:12:13.382605+0000","last_active":"2024-04-07T16:13:38.553144+0000","last_peered":"2024-04-07T16:13:38.553144+0000","last_clean":"2024-04-07T16:13:38.553144+0000","last_became_active":"2024-04-07T16:12:13.382267+0000","last_became_peered":"2024-04-07T16:12:13.382267+0000","last_unstale":"2024-04-07T16:13:38.553144+0000","last_undegraded":"2024-04-07T16:13:38.553144+0000","last_fullsized":"2024-04-07T16:13:38.553144+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T20:14:03.942753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,4],"acting":[6,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.187449+0000","last_change":"2024-04-07T16:12:13.405175+0000","last_active":"2024-04-07T16:13:38.187449+0000","last_peered":"2024-04-07T16:13:38.187449+0000","last_clean":"2024-04-07T16:13:38.187449+0000","last_became_active":"2024-04-07T16:12:13.404734+0000","last_became_peered":"2024-04-07T16:12:13.404734+0000","last_unstale":"2024-04-07T16:13:38.187449+0000","last_undegraded":"2024-04-07T16:13:38.187449+0000","last_fullsized":"2024-04-07T16:13:38.187449+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T17:55:23.469795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.024146+0000","last_change":"2024-04-07T16:12:13.397863+0000","last_active":"2024-04-07T16:13:38.024146+0000","last_peered":"2024-04-07T16:13:38.024146+0000","last_clean":"2024-04-07T16:13:38.024146+0000","last_became_active":"2024-04-07T16:12:13.395430+0000","last_became_peered":"2024-04-07T16:12:13.395430+0000","last_unstale":"2024-04-07T16:13:38.024146+0000","last_undegraded":"2024-04-07T16:13:38.024146+0000","last_fullsized":"2024-04-07T16:13:38.024146+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T18:20:22.678341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.186791+0000","last_change":"2024-04-07T16:12:13.397638+0000","last_active":"2024-04-07T16:13:38.186791+0000","last_peered":"2024-04-07T16:13:38.186791+0000","last_clean":"2024-04-07T16:13:38.186791+0000","last_became_active":"2024-04-07T16:12:13.396766+0000","last_became_peered":"2024-04-07T16:12:13.396766+0000","last_unstale":"2024-04-07T16:13:38.186791+0000","last_undegraded":"2024-04-07T16:13:38.186791+0000","last_fullsized":"2024-04-07T16:13:38.186791+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T22:51:06.841718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.024630+0000","last_change":"2024-04-07T16:12:13.403603+0000","last_active":"2024-04-07T16:13:38.024630+0000","last_peered":"2024-04-07T16:13:38.024630+0000","last_clean":"2024-04-07T16:13:38.024630+0000","last_became_active":"2024-04-07T16:12:13.400042+0000","last_became_peered":"2024-04-07T16:12:13.400042+0000","last_unstale":"2024-04-07T16:13:38.024630+0000","last_undegraded":"2024-04-07T16:13:38.024630+0000","last_fullsized":"2024-04-07T16:13:38.024630+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T20:49:17.570678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.029710+0000","last_change":"2024-04-07T16:12:13.386718+0000","last_active":"2024-04-07T16:13:38.029710+0000","last_peered":"2024-04-07T16:13:38.029710+0000","last_clean":"2024-04-07T16:13:38.029710+0000","last_became_active":"2024-04-07T16:12:13.386422+0000","last_became_peered":"2024-04-07T16:12:13.386422+0000","last_unstale":"2024-04-07T16:13:38.029710+0000","last_undegraded":"2024-04-07T16:13:38.029710+0000","last_fullsized":"2024-04-07T16:13:38.029710+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T21:28:09.174468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1],"acting":[0,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-07T16:13:38.526600+0000","last_change":"2024-04-07T16:12:13.380077+0000","last_active":"2024-04-07T16:13:38.526600+0000","last_peered":"2024-04-07T16:13:38.526600+0000","last_clean":"2024-04-07T16:13:38.526600+0000","last_became_active":"2024-04-07T16:12:13.379758+0000","last_became_peered":"2024-04-07T16:12:13.379758+0000","last_unstale":"2024-04-07T16:13:38.526600+0000","last_undegraded":"2024-04-07T16:13:38.526600+0000","last_fullsized":"2024-04-07T16:13:38.526600+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T22:47:34.836189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":3430,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":51,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":10290,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":45,"ondisk_log_size":45,"up":96,"acting":96,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":54,"seq":231928234028,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32396,"kb_used_data":1280,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93712756,"statfs":{"total":95995035648,"available":95961862144,"internally_reserved":0,"allocated":1310720,"data_stored":900204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Apr 7 16:14:37 2024","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":0.97399999999999998,"15min":1.004},"min":{"1min":0.65300000000000002,"5min":0.625,"15min":0.625},"max":{"1min":1.3700000000000001,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.874,"5min":0.95799999999999996,"15min":0.98499999999999999},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.2090000000000001,"5min":1.363,"15min":1.363},"last":1.208}]},{"osd":1,"last update":"Sun Apr 7 16:14:37 2024","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.97899999999999998,"15min":0.999},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.1419999999999999,"5min":1.55,"15min":1.55},"last":1.2310000000000001},{"interface":"front","average":{"1min":0.92000000000000004,"5min":1.002,"15min":1.026},"min":{"1min":0.63100000000000001,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.3620000000000001,"5min":1.5069999999999999,"15min":1.5069999999999999},"last":0.97399999999999998}]},{"osd":2,"last update":"Sun Apr 7 16:14:37 2024","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.93899999999999995,"15min":0.95499999999999996},"min":{"1min":0.71799999999999997,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.115,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.92400000000000004},{"interface":"front","average":{"1min":0.96899999999999997,"5min":1.016,"15min":1.048},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.2649999999999999,"5min":1.288,"15min":1.288},"last":1.0760000000000001}]},{"osd":3,"last update":"Sun Apr 7 16:14:37 2024","interfaces":[{"interface":"back","average":{"1min":0.95999999999999996,"5min":0.99399999999999999,"15min":1.018},"min":{"1min":0.67400000000000004,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.323,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":1.2709999999999999},{"interface":"front","average":{"1min":1.036,"5min":1.0800000000000001,"15min":1.113},"min":{"1min":0.63900000000000001,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.4099999999999999,"5min":1.4099999999999999,"15min":1.4099999999999999},"last":0.877}]},{"osd":4,"last update":"Sun Apr 7 16:14:37 2024","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":0.90900000000000003,"15min":0.90700000000000003},"min":{"1min":0.45100000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.514,"5min":1.514,"15min":1.514},"last":1.131},{"interface":"front","average":{"1min":1.0269999999999999,"5min":1.0029999999999999,"15min":1.0169999999999999},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.3220000000000001,"5min":1.468,"15min":1.468},"last":1.038}]},{"osd":5,"last update":"Sun Apr 7 16:14:37 2024","interfaces":[{"interface":"back","average":{"1min":1.085,"5min":1.0980000000000001,"15min":1.1299999999999999},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.4810000000000001,"5min":1.7290000000000001,"15min":1.7290000000000001},"last":0.79800000000000004},{"interface":"front","average":{"1min":1.123,"5min":1.0469999999999999,"15min":1.0509999999999999},"min":{"1min":0.58199999999999996,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.6160000000000001,"5min":1.6160000000000001,"15min":1.6160000000000001},"last":1.3}]},{"osd":6,"last update":"Sun Apr 7 16:14:37 2024","interfaces":[{"interface":"back","average":{"1min":1.1639999999999999,"5min":1.135,"15min":1.1539999999999999},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.6699999999999999,"5min":1.6919999999999999,"15min":1.6919999999999999},"last":1.4119999999999999},{"interface":"front","average":{"1min":1.113,"5min":1.1120000000000001,"15min":1.125},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.5860000000000001,"5min":1.6060000000000001,"15min":1.6060000000000001},"last":1.1719999999999999}]}]},{"osd":6,"up_from":46,"seq":197568495669,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28300,"kb_used_data":1284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716852,"statfs":{"total":95995035648,"available":95966056448,"internally_reserved":0,"allocated":1314816,"data_stored":902232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Apr 7 16:14:48 2024","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.92100000000000004,"15min":0.91700000000000004},"min":{"1min":0.48599999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.216,"5min":3.4079999999999999,"15min":3.4079999999999999},"last":1.0429999999999999},{"interface":"front","average":{"1min":0.89900000000000002,"5min":0.95399999999999996,"15min":0.94199999999999995},"min":{"1min":0.56499999999999995,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.4039999999999999,"5min":4.165,"15min":4.165},"last":0.98499999999999999}]},{"osd":1,"last update":"Sun Apr 7 16:14:48 2024","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.89400000000000002,"15min":0.90200000000000002},"min":{"1min":0.54800000000000004,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.306,"5min":1.8799999999999999,"15min":1.8799999999999999},"last":1.016},{"interface":"front","average":{"1min":0.92500000000000004,"5min":0.96299999999999997,"15min":0.96199999999999997},"min":{"1min":0.57299999999999995,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.4199999999999999,"5min":3.4889999999999999,"15min":3.4889999999999999},"last":1.1240000000000001}]},{"osd":2,"last update":"Sun Apr 7 16:14:48 2024","interfaces":[{"interface":"back","average":{"1min":1.018,"5min":0.96799999999999997,"15min":0.93600000000000005},"min":{"1min":0.58599999999999997,"5min":0.502,"15min":0.502},"max":{"1min":1.5429999999999999,"5min":4.218,"15min":4.218},"last":1.0720000000000001},{"interface":"front","average":{"1min":0.95199999999999996,"5min":0.98299999999999998,"15min":0.99299999999999999},"min":{"1min":0.54700000000000004,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.3440000000000001,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":1.2490000000000001}]},{"osd":3,"last update":"Sun Apr 7 16:14:48 2024","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.96899999999999997,"15min":0.95599999999999996},"min":{"1min":0.54300000000000004,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.2130000000000001,"5min":4.2809999999999997,"15min":4.2809999999999997},"last":1.147},{"interface":"front","average":{"1min":1.0649999999999999,"5min":0.99099999999999999,"15min":0.95699999999999996},"min":{"1min":0.57399999999999995,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":2.581,"5min":3.3079999999999998,"15min":3.3079999999999998},"last":1.103}]},{"osd":4,"last update":"Sun Apr 7 16:14:48 2024","interfaces":[{"interface":"back","average":{"1min":1.018,"5min":0.91100000000000003,"15min":0.89500000000000002},"min":{"1min":0.42899999999999999,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.446,"5min":1.5609999999999999,"15min":1.5609999999999999},"last":0.95199999999999996},{"interface":"front","average":{"1min":1.034,"5min":0.998,"15min":0.94599999999999995},"min":{"1min":0.39700000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.304,"5min":4.8209999999999997,"15min":4.8209999999999997},"last":1.2270000000000001}]},{"osd":5,"last update":"Sun Apr 7 16:14:48 2024","interfaces":[{"interface":"back","average":{"1min":1.0029999999999999,"5min":0.94299999999999995,"15min":0.90900000000000003},"min":{"1min":0.44700000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.393,"5min":1.974,"15min":1.974},"last":0.90200000000000002},{"interface":"front","average":{"1min":1.0780000000000001,"5min":1.03,"15min":1.0049999999999999},"min":{"1min":0.53100000000000003,"5min":0.375,"15min":0.375},"max":{"1min":1.498,"5min":1.6279999999999999,"15min":1.6279999999999999},"last":1.198}]},{"osd":7,"last update":"Sun Apr 7 16:14:34 2024","interfaces":[{"interface":"back","average":{"1min":1.1000000000000001,"5min":1.1599999999999999,"15min":1.2050000000000001},"min":{"1min":0.51300000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.512,"5min":5.2350000000000003,"15min":5.2350000000000003},"last":1.1779999999999999},{"interface":"front","average":{"1min":1.159,"5min":1.2310000000000001,"15min":1.2749999999999999},"min":{"1min":0.47099999999999997,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.4810000000000001,"5min":4.0940000000000003,"15min":4.0940000000000003},"last":1.286}]}]},{"osd":1,"up_from":13,"seq":55834574949,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27720,"kb_used_data":700,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717432,"statfs":{"total":95995035648,"available":95966650368,"internally_reserved":0,"allocated":716800,"data_stored":309392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Apr 7 16:14:56 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.70799999999999996,"15min":0.60699999999999998},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":1.3009999999999999,"5min":1.518,"15min":1.518},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.73799999999999999,"15min":0.65000000000000002},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1,"5min":1.2969999999999999,"15min":2.8620000000000001},"last":0.68500000000000005}]},{"osd":2,"last update":"Sun Apr 7 16:14:34 2024","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.75900000000000001,"15min":0.67700000000000005},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34000000000000002},"max":{"1min":1.159,"5min":1.345,"15min":1.345},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.73699999999999999,"15min":0.67200000000000004},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":1.2050000000000001,"5min":1.5369999999999999,"15min":1.5369999999999999},"last":0.747}]},{"osd":3,"last update":"Sun Apr 7 16:14:29 2024","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.88200000000000001,"15min":0.82799999999999996},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.0609999999999999,"5min":1.343,"15min":1.343},"last":0.95899999999999996},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.81200000000000006,"15min":0.76600000000000001},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":1.1080000000000001,"5min":1.4770000000000001,"15min":1.4770000000000001},"last":0.88500000000000001}]},{"osd":4,"last update":"Sun Apr 7 16:14:17 2024","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":0.94099999999999995,"15min":0.93200000000000005},"min":{"1min":0.59999999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.248,"5min":1.5149999999999999,"15min":1.5149999999999999},"last":1.022},{"interface":"front","average":{"1min":1.0369999999999999,"5min":1.0820000000000001,"15min":1.054},"min":{"1min":0.59999999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.393,"5min":1.9299999999999999,"15min":1.9299999999999999},"last":1.2010000000000001}]},{"osd":5,"last update":"Sun Apr 7 16:14:02 2024","interfaces":[{"interface":"back","average":{"1min":0.996,"5min":1.1459999999999999,"15min":1.2150000000000001},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.3620000000000001,"5min":2.0510000000000002,"15min":2.0510000000000002},"last":1.0409999999999999},{"interface":"front","average":{"1min":0.96799999999999997,"5min":1.0660000000000001,"15min":1.117},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.5740000000000001,"5min":1.8859999999999999,"15min":1.8859999999999999},"last":1.2370000000000001}]},{"osd":6,"last update":"Sun Apr 7 16:14:52 2024","interfaces":[{"interface":"back","average":{"1min":0.98399999999999999,"5min":1.042,"15min":1.079},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.353,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":0.93600000000000005},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.2130000000000001,"15min":1.2529999999999999},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.4399999999999999,"5min":2.5609999999999999,"15min":2.5609999999999999},"last":1.2889999999999999}]},{"osd":7,"last update":"Sun Apr 7 16:14:34 2024","interfaces":[{"interface":"back","average":{"1min":0.97999999999999998,"5min":1.083,"15min":1.109},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.3919999999999999,"5min":1.702,"15min":1.702},"last":1.1719999999999999},{"interface":"front","average":{"1min":1.123,"5min":1.179,"15min":1.181},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.698,"5min":1.698,"15min":1.698},"last":1.359}]}]},{"osd":0,"up_from":8,"seq":34359738480,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28308,"kb_used_data":1288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716844,"statfs":{"total":95995035648,"available":95966048256,"internally_reserved":0,"allocated":1318912,"data_stored":900328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Apr 7 16:14:50 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.66700000000000004,"15min":0.59099999999999997},"min":{"1min":0.315,"5min":0.315,"15min":0.25600000000000001},"max":{"1min":0.80300000000000005,"5min":1.353,"15min":1.353},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77000000000000002,"15min":0.62},"min":{"1min":0.34200000000000003,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.377,"5min":1.377,"15min":1.377},"last":0.85599999999999998}]},{"osd":2,"last update":"Sun Apr 7 16:14:40 2024","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.73999999999999999,"15min":0.68200000000000005},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":1.2,"5min":1.2,"15min":1.2},"last":0.97799999999999998},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.77100000000000002,"15min":0.73999999999999999},"min":{"1min":0.40600000000000003,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.96499999999999997,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":1.008}]},{"osd":3,"last update":"Sun Apr 7 16:14:26 2024","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.78800000000000003,"15min":0.75900000000000001},"min":{"1min":0.36399999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":1.05},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.84399999999999997,"15min":0.80700000000000005},"min":{"1min":0.39400000000000002,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.0589999999999999,"5min":1.143,"15min":1.143},"last":0.72399999999999998}]},{"osd":4,"last update":"Sun Apr 7 16:14:13 2024","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":0.95399999999999996,"15min":0.92900000000000005},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.4239999999999999,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":1.26},{"interface":"front","average":{"1min":1.0269999999999999,"5min":1.0169999999999999,"15min":0.97399999999999998},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.6930000000000001,"5min":1.7809999999999999,"15min":1.7809999999999999},"last":1.1359999999999999}]},{"osd":5,"last update":"Sun Apr 7 16:13:58 2024","interfaces":[{"interface":"back","average":{"1min":1.022,"5min":0.999,"15min":0.98499999999999999},"min":{"1min":0.745,"5min":0.745,"15min":0.745},"max":{"1min":1.4450000000000001,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":1.341},{"interface":"front","average":{"1min":1.0429999999999999,"5min":1.0329999999999999,"15min":1.0249999999999999},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.488,"5min":1.488,"15min":1.488},"last":1.1000000000000001}]},{"osd":6,"last update":"Sun Apr 7 16:14:44 2024","interfaces":[{"interface":"back","average":{"1min":0.998,"5min":1.0649999999999999,"15min":1.0680000000000001},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.335,"5min":1.7629999999999999,"15min":1.7629999999999999},"last":1.294},{"interface":"front","average":{"1min":1.0389999999999999,"5min":1.109,"15min":1.119},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.272,"5min":1.5429999999999999,"15min":1.5429999999999999},"last":1.222}]},{"osd":7,"last update":"Sun Apr 7 16:14:35 2024","interfaces":[{"interface":"back","average":{"1min":1.052,"5min":1.139,"15min":1.165},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.421,"5min":1.6100000000000001,"15min":1.6100000000000001},"last":1.3779999999999999},{"interface":"front","average":{"1min":1.0980000000000001,"5min":1.1579999999999999,"15min":1.1759999999999999},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.385,"5min":1.464,"15min":1.464},"last":1.4239999999999999}]}]},{"osd":2,"up_from":18,"seq":77309411420,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31808,"kb_used_data":692,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713344,"statfs":{"total":95995035648,"available":95962464256,"internally_reserved":0,"allocated":708608,"data_stored":309232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Apr 7 16:14:36 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.70199999999999996,"15min":0.61499999999999999},"min":{"1min":0.38100000000000001,"5min":0.16800000000000001,"15min":0.16800000000000001},"max":{"1min":1.0720000000000001,"5min":1.204,"15min":1.204},"last":0.622},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.747,"15min":0.63900000000000001},"min":{"1min":0.36599999999999999,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.97299999999999998,"5min":1.109,"15min":1.109},"last":0.79500000000000004}]},{"osd":1,"last update":"Sun Apr 7 16:14:36 2024","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.76500000000000001,"15min":0.67200000000000004},"min":{"1min":0.38600000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":1.204,"5min":1.204,"15min":1.204},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.749,"15min":0.66900000000000004},"min":{"1min":0.33400000000000002,"5min":0.318,"15min":0.313},"max":{"1min":1.0109999999999999,"5min":1.3120000000000001,"15min":1.3120000000000001},"last":0.91000000000000003}]},{"osd":3,"last update":"Sun Apr 7 16:14:24 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.80100000000000005,"15min":0.80500000000000005},"min":{"1min":0.39400000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":1.01,"5min":1.099,"15min":1.099},"last":0.749},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.80100000000000005,"15min":0.79800000000000004},"min":{"1min":0.40899999999999997,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.046,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":0.94299999999999995}]},{"osd":4,"last update":"Sun Apr 7 16:14:16 2024","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.94699999999999995,"15min":0.92500000000000004},"min":{"1min":0.78700000000000003,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.1899999999999999,"5min":1.4299999999999999,"15min":1.4299999999999999},"last":1.1120000000000001},{"interface":"front","average":{"1min":1.0209999999999999,"5min":0.995,"15min":0.95199999999999996},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.411,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":1.1399999999999999}]},{"osd":5,"last update":"Sun Apr 7 16:13:57 2024","interfaces":[{"interface":"back","average":{"1min":1.0429999999999999,"5min":1.008,"15min":0.98399999999999999},"min":{"1min":0.68100000000000005,"5min":0.374,"15min":0.374},"max":{"1min":1.2030000000000001,"5min":1.786,"15min":1.786},"last":1.2410000000000001},{"interface":"front","average":{"1min":1.101,"5min":1.0760000000000001,"15min":1.05},"min":{"1min":0.85699999999999998,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.3540000000000001,"5min":1.6759999999999999,"15min":1.6759999999999999},"last":1.0569999999999999}]},{"osd":6,"last update":"Sun Apr 7 16:14:44 2024","interfaces":[{"interface":"back","average":{"1min":1.0129999999999999,"5min":1.069,"15min":1.056},"min":{"1min":0.66000000000000003,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.409,"5min":1.5449999999999999,"15min":1.5449999999999999},"last":1.006},{"interface":"front","average":{"1min":0.99099999999999999,"5min":1.073,"15min":1.0649999999999999},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.254,"5min":1.839,"15min":1.839},"last":1.173}]},{"osd":7,"last update":"Sun Apr 7 16:14:36 2024","interfaces":[{"interface":"back","average":{"1min":1.0629999999999999,"5min":1.117,"15min":1.1100000000000001},"min":{"1min":0.56000000000000005,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.377,"5min":1.6259999999999999,"15min":1.6259999999999999},"last":1.2},{"interface":"front","average":{"1min":1.054,"5min":1.111,"15min":1.1100000000000001},"min":{"1min":0.66200000000000003,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.3720000000000001,"5min":1.579,"15min":1.579},"last":1.0780000000000001}]}]},{"osd":3,"up_from":25,"seq":107374182483,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27728,"kb_used_data":712,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717424,"statfs":{"total":95995035648,"available":95966642176,"internally_reserved":0,"allocated":729088,"data_stored":311968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Apr 7 16:14:32 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.68600000000000005,"15min":0.67400000000000004},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":1.0389999999999999,"5min":1.345,"15min":1.345},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.76700000000000002,"15min":0.78300000000000003},"min":{"1min":0.372,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.94499999999999995,"5min":1.327,"15min":1.327},"last":0.86199999999999999}]},{"osd":1,"last update":"Sun Apr 7 16:14:32 2024","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.78600000000000003,"15min":0.76000000000000001},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":1.1140000000000001,"5min":1.544,"15min":1.544},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.79300000000000004,"15min":0.76700000000000002},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.0209999999999999,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.65900000000000003}]},{"osd":2,"last update":"Sun Apr 7 16:14:32 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.85099999999999998,"15min":0.81699999999999995},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.258,"5min":1.27,"15min":1.27},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.79100000000000004,"15min":0.80600000000000005},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.999,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.68300000000000005}]},{"osd":4,"last update":"Sun Apr 7 16:14:10 2024","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":0.97399999999999998,"15min":0.95599999999999996},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.218,"5min":1.552,"15min":1.552},"last":1.1339999999999999},{"interface":"front","average":{"1min":0.96599999999999997,"5min":1.014,"15min":0.98199999999999998},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.3779999999999999,"5min":1.403,"15min":1.403},"last":1.115}]},{"osd":5,"last update":"Sun Apr 7 16:13:59 2024","interfaces":[{"interface":"back","average":{"1min":1.0669999999999999,"5min":1.0600000000000001,"15min":1.0580000000000001},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.5589999999999999,"5min":1.5589999999999999,"15min":1.5589999999999999},"last":1.0389999999999999},{"interface":"front","average":{"1min":0.97499999999999998,"5min":1.012,"15min":1.0309999999999999},"min":{"1min":0.68600000000000005,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.2470000000000001,"5min":1.571,"15min":1.571},"last":0.84299999999999997}]},{"osd":6,"last update":"Sun Apr 7 16:14:47 2024","interfaces":[{"interface":"back","average":{"1min":0.97199999999999998,"5min":1.0640000000000001,"15min":1.0860000000000001},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.3600000000000001,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":0.98499999999999999},{"interface":"front","average":{"1min":1.0600000000000001,"5min":1.1240000000000001,"15min":1.1319999999999999},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.4139999999999999,"5min":1.5660000000000001,"15min":1.5660000000000001},"last":0.95499999999999996}]},{"osd":7,"last update":"Sun Apr 7 16:14:36 2024","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":1.069,"15min":1.089},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.373,"5min":1.387,"15min":1.387},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.98799999999999999,"5min":1.1559999999999999,"15min":1.194},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.3169999999999999,"5min":1.52,"15min":1.52},"last":1.0740000000000001}]}]},{"osd":4,"up_from":30,"seq":128849018954,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31824,"kb_used_data":708,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713328,"statfs":{"total":95995035648,"available":95962447872,"internally_reserved":0,"allocated":724992,"data_stored":309940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Apr 7 16:14:14 2024","interfaces":[{"interface":"back","average":{"1min":0.995,"5min":0.90700000000000003,"15min":0.84299999999999997},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.319,"5min":1.3839999999999999,"15min":1.3839999999999999},"last":0.91100000000000003},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.89300000000000002,"15min":0.90100000000000002},"min":{"1min":0.67100000000000004,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.204,"5min":1.345,"15min":1.345},"last":1.198}]},{"osd":1,"last update":"Sun Apr 7 16:14:14 2024","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":0.90300000000000002,"15min":0.83699999999999997},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.389,"5min":1.389,"15min":1.389},"last":1.0189999999999999},{"interface":"front","average":{"1min":0.96999999999999997,"5min":0.91400000000000003,"15min":0.85699999999999998},"min":{"1min":0.64800000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.264,"5min":1.486,"15min":1.486},"last":1.081}]},{"osd":2,"last update":"Sun Apr 7 16:14:14 2024","interfaces":[{"interface":"back","average":{"1min":0.94899999999999995,"5min":0.92100000000000004,"15min":0.878},"min":{"1min":0.58099999999999996,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.355,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.88900000000000001,"15min":0.88400000000000001},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.407,"5min":1.407,"15min":1.407},"last":1.04}]},{"osd":3,"last update":"Sun Apr 7 16:14:14 2024","interfaces":[{"interface":"back","average":{"1min":0.92600000000000005,"5min":0.92100000000000004,"15min":0.93100000000000005},"min":{"1min":0.69299999999999995,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.4450000000000001,"5min":1.458,"15min":1.458},"last":0.95299999999999996},{"interface":"front","average":{"1min":1.0589999999999999,"5min":1.0009999999999999,"15min":0.94799999999999995},"min":{"1min":0.68400000000000005,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.3660000000000001,"5min":1.472,"15min":1.472},"last":1.165}]},{"osd":5,"last update":"Sun Apr 7 16:13:58 2024","interfaces":[{"interface":"back","average":{"1min":1.0169999999999999,"5min":0.875,"15min":0.78700000000000003},"min":{"1min":0.42899999999999999,"5min":0.315,"15min":0.315},"max":{"1min":1.2949999999999999,"5min":1.403,"15min":1.403},"last":1.228},{"interface":"front","average":{"1min":1.0529999999999999,"5min":0.94899999999999995,"15min":0.88700000000000001},"min":{"1min":0.40999999999999998,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.4530000000000001,"5min":1.546,"15min":1.546},"last":1.2769999999999999}]},{"osd":6,"last update":"Sun Apr 7 16:14:47 2024","interfaces":[{"interface":"back","average":{"1min":1.1879999999999999,"5min":1.046,"15min":0.99399999999999999},"min":{"1min":0.88300000000000001,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.5,"5min":1.5,"15min":1.5},"last":1.2649999999999999},{"interface":"front","average":{"1min":1.169,"5min":1.044,"15min":0.98599999999999999},"min":{"1min":0.79200000000000004,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.472,"5min":1.5309999999999999,"15min":1.5309999999999999},"last":1.3560000000000001}]},{"osd":7,"last update":"Sun Apr 7 16:14:36 2024","interfaces":[{"interface":"back","average":{"1min":1.109,"5min":1.044,"15min":1.048},"min":{"1min":0.77500000000000002,"5min":0.433,"15min":0.433},"max":{"1min":1.5029999999999999,"5min":1.5449999999999999,"15min":1.5449999999999999},"last":1.1359999999999999},{"interface":"front","average":{"1min":1.319,"5min":1.2210000000000001,"15min":1.206},"min":{"1min":1.113,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.5940000000000001,"5min":1.734,"15min":1.734},"last":1.306}]}]},{"osd":5,"up_from":38,"seq":163208757311,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27720,"kb_used_data":704,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717432,"statfs":{"total":95995035648,"available":95966650368,"internally_reserved":0,"allocated":720896,"data_stored":311623,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Apr 7 16:14:00 2024","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.92700000000000005,"15min":0.92000000000000004},"min":{"1min":0.57499999999999996,"5min":0.443,"15min":0.443},"max":{"1min":1.355,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":1.1140000000000001},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.89800000000000002,"15min":0.94199999999999995},"min":{"1min":0.55000000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.196,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":0.69399999999999995}]},{"osd":1,"last update":"Sun Apr 7 16:14:00 2024","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":0.89800000000000002,"15min":0.91600000000000004},"min":{"1min":0.53000000000000003,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.474,"5min":1.474,"15min":1.474},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.95899999999999996,"15min":0.98099999999999998},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.2150000000000001,"5min":1.45,"15min":1.45},"last":1.157}]},{"osd":2,"last update":"Sun Apr 7 16:14:00 2024","interfaces":[{"interface":"back","average":{"1min":0.97599999999999998,"5min":0.95599999999999996,"15min":0.97499999999999998},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.3740000000000001,"5min":1.462,"15min":1.462},"last":0.875},{"interface":"front","average":{"1min":0.93600000000000005,"5min":0.94699999999999995,"15min":0.96199999999999997},"min":{"1min":0.63700000000000001,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.1479999999999999,"5min":1.472,"15min":1.472},"last":1.212}]},{"osd":3,"last update":"Sun Apr 7 16:14:00 2024","interfaces":[{"interface":"back","average":{"1min":0.95399999999999996,"5min":1.05,"15min":1.177},"min":{"1min":0.58199999999999996,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.5880000000000001,"5min":6.3860000000000001,"15min":6.3860000000000001},"last":1.24},{"interface":"front","average":{"1min":0.96699999999999997,"5min":0.97499999999999998,"15min":0.999},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.284,"5min":1.556,"15min":1.556},"last":1.0860000000000001}]},{"osd":4,"last update":"Sun Apr 7 16:14:00 2024","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.84099999999999997,"15min":0.81699999999999995},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":1.3280000000000001,"5min":1.403,"15min":1.403},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.96099999999999997,"5min":0.91900000000000004,"15min":0.91300000000000003},"min":{"1min":0.372,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.333,"5min":1.4119999999999999,"15min":1.4119999999999999},"last":1.1890000000000001}]},{"osd":6,"last update":"Sun Apr 7 16:14:44 2024","interfaces":[{"interface":"back","average":{"1min":1.0629999999999999,"5min":0.93200000000000005,"15min":0.879},"min":{"1min":0.42499999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":2.6459999999999999,"5min":2.6459999999999999,"15min":2.6459999999999999},"last":1.0389999999999999},{"interface":"front","average":{"1min":1.1200000000000001,"5min":1.036,"15min":0.98999999999999999},"min":{"1min":0.48799999999999999,"5min":0.31,"15min":0.31},"max":{"1min":2.109,"5min":2.109,"15min":2.109},"last":1.2869999999999999}]},{"osd":7,"last update":"Sun Apr 7 16:14:34 2024","interfaces":[{"interface":"back","average":{"1min":1.1220000000000001,"5min":1.0900000000000001,"15min":1.1000000000000001},"min":{"1min":0.54500000000000004,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":2.5830000000000002,"5min":2.5830000000000002,"15min":2.5830000000000002},"last":1.2669999999999999},{"interface":"front","average":{"1min":1.2210000000000001,"5min":1.163,"15min":1.169},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":2.7509999999999999,"5min":2.7509999999999999,"15min":2.7509999999999999},"last":1.3300000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-07T16:15:01.470 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-04-07T16:15:01.471 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-07T16:15:01.471 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-04-07T16:15:01.471 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 0702e3a8-f4f8-11ee-b649-cb9ed24678a4 -- ceph health --format=json 2024-04-07T16:15:02.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:15:01 smithi144 bash[23522]: cluster 2024-04-07T16:15:00.651381+0000 mgr.y (mgr.14553) 100 : cluster 0 pgmap v71: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:15:02.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:15:01 smithi144 bash[23522]: audit 2024-04-07T16:15:00.809333+0000 mgr.y (mgr.14553) 101 : audit 0 from='client.24659 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-07T16:15:02.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:15:01 smithi100 bash[21346]: cluster 2024-04-07T16:15:00.651381+0000 mgr.y (mgr.14553) 100 : cluster 0 pgmap v71: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:15:02.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:15:01 smithi100 bash[21346]: audit 2024-04-07T16:15:00.809333+0000 mgr.y (mgr.14553) 101 : audit 0 from='client.24659 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-07T16:15:02.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:15:01 smithi100 bash[31660]: cluster 2024-04-07T16:15:00.651381+0000 mgr.y (mgr.14553) 100 : cluster 0 pgmap v71: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:15:02.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:15:01 smithi100 bash[31660]: audit 2024-04-07T16:15:00.809333+0000 mgr.y (mgr.14553) 101 : audit 0 from='client.24659 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-07T16:15:04.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:15:03 smithi144 bash[23522]: cluster 2024-04-07T16:15:02.652091+0000 mgr.y (mgr.14553) 102 : cluster 0 pgmap v72: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:15:04.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:15:03 smithi100 bash[21346]: cluster 2024-04-07T16:15:02.652091+0000 mgr.y (mgr.14553) 102 : cluster 0 pgmap v72: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:15:04.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:15:03 smithi100 bash[31660]: cluster 2024-04-07T16:15:02.652091+0000 mgr.y (mgr.14553) 102 : cluster 0 pgmap v72: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:15:05.437 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:15:05 smithi100 bash[21601]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:15:05] "GET /metrics HTTP/1.1" 200 33732 "" "Prometheus/2.43.0" 2024-04-07T16:15:06.257 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/mon.a/config 2024-04-07T16:15:06.278 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:15:06 smithi100 bash[21346]: cluster 2024-04-07T16:15:04.653415+0000 mgr.y (mgr.14553) 103 : cluster 0 pgmap v73: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:15:06.279 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:15:06 smithi100 bash[31660]: cluster 2024-04-07T16:15:04.653415+0000 mgr.y (mgr.14553) 103 : cluster 0 pgmap v73: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:15:06.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:15:05 smithi144 bash[23522]: cluster 2024-04-07T16:15:04.653415+0000 mgr.y (mgr.14553) 103 : cluster 0 pgmap v73: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:15:08.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:15:08 smithi100 bash[21346]: cluster 2024-04-07T16:15:06.654251+0000 mgr.y (mgr.14553) 104 : cluster 0 pgmap v74: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:15:08.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:15:08 smithi100 bash[21346]: audit 2024-04-07T16:15:07.699561+0000 mon.b (mon.1) 86 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:15:08.188 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:15:08 smithi100 bash[31660]: cluster 2024-04-07T16:15:06.654251+0000 mgr.y (mgr.14553) 104 : cluster 0 pgmap v74: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:15:08.188 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:15:08 smithi100 bash[31660]: audit 2024-04-07T16:15:07.699561+0000 mon.b (mon.1) 86 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:15:08.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:15:08 smithi144 bash[23522]: cluster 2024-04-07T16:15:06.654251+0000 mgr.y (mgr.14553) 104 : cluster 0 pgmap v74: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:15:08.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:15:08 smithi144 bash[23522]: audit 2024-04-07T16:15:07.699561+0000 mon.b (mon.1) 86 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:15:08.836 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-07T16:15:08.836 INFO:teuthology.orchestra.run.smithi100.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-07T16:15:09.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:15:09 smithi100 bash[21346]: audit 2024-04-07T16:15:08.851331+0000 mon.c (mon.2) 35 : audit 0 from='client.? 172.21.15.100:0/69947173' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-07T16:15:09.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:15:09 smithi100 bash[31660]: audit 2024-04-07T16:15:08.851331+0000 mon.c (mon.2) 35 : audit 0 from='client.? 172.21.15.100:0/69947173' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-07T16:15:09.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:15:09 smithi144 bash[23522]: audit 2024-04-07T16:15:08.851331+0000 mon.c (mon.2) 35 : audit 0 from='client.? 172.21.15.100:0/69947173' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-07T16:15:09.478 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-04-07T16:15:09.479 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-07T16:15:09.799 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-07T16:15:09.799 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":69,"fsid":"0702e3a8-f4f8-11ee-b649-cb9ed24678a4","created":"2024-04-07T16:02:21.629692+0000","modified":"2024-04-07T16:13:38.014991+0000","last_up_change":"2024-04-07T16:11:30.210001+0000","last_in_change":"2024-04-07T16:11:04.793434+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-07T16:07:26.784071+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":7.8899998664855957,"score_stable":7.8899998664855957,"optimal_score":0.37999999523162842,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-07T16:12:11.856505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-07T16:12:12.737709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"rbd","create_time":"2024-04-07T16:13:28.913394+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":3,"score_stable":3,"optimal_score":1,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8facd48e-0a0c-4fda-aee2-f001b60a3811","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":1636950303},{"type":"v1","addr":"172.21.15.100:6803","nonce":1636950303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":1636950303},{"type":"v1","addr":"172.21.15.100:6805","nonce":1636950303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1636950303},{"type":"v1","addr":"172.21.15.100:6809","nonce":1636950303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":1636950303},{"type":"v1","addr":"172.21.15.100:6807","nonce":1636950303}]},"public_addr":"172.21.15.100:6803/1636950303","cluster_addr":"172.21.15.100:6805/1636950303","heartbeat_back_addr":"172.21.15.100:6809/1636950303","heartbeat_front_addr":"172.21.15.100:6807/1636950303","state":["exists","up"]},{"osd":1,"uuid":"cb5f6c23-6764-48b6-8b21-97a36b004786","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":470179506},{"type":"v1","addr":"172.21.15.100:6811","nonce":470179506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":470179506},{"type":"v1","addr":"172.21.15.100:6813","nonce":470179506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":470179506},{"type":"v1","addr":"172.21.15.100:6817","nonce":470179506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":470179506},{"type":"v1","addr":"172.21.15.100:6815","nonce":470179506}]},"public_addr":"172.21.15.100:6811/470179506","cluster_addr":"172.21.15.100:6813/470179506","heartbeat_back_addr":"172.21.15.100:6817/470179506","heartbeat_front_addr":"172.21.15.100:6815/470179506","state":["exists","up"]},{"osd":2,"uuid":"1650aedb-9ce9-4c14-a059-e1f089e5e63f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":2235490011},{"type":"v1","addr":"172.21.15.100:6819","nonce":2235490011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2235490011},{"type":"v1","addr":"172.21.15.100:6821","nonce":2235490011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":2235490011},{"type":"v1","addr":"172.21.15.100:6825","nonce":2235490011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":2235490011},{"type":"v1","addr":"172.21.15.100:6823","nonce":2235490011}]},"public_addr":"172.21.15.100:6819/2235490011","cluster_addr":"172.21.15.100:6821/2235490011","heartbeat_back_addr":"172.21.15.100:6825/2235490011","heartbeat_front_addr":"172.21.15.100:6823/2235490011","state":["exists","up"]},{"osd":3,"uuid":"17591c6b-862b-43b2-858b-02f4c3199247","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":3019212393},{"type":"v1","addr":"172.21.15.100:6827","nonce":3019212393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":3019212393},{"type":"v1","addr":"172.21.15.100:6829","nonce":3019212393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":3019212393},{"type":"v1","addr":"172.21.15.100:6833","nonce":3019212393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":3019212393},{"type":"v1","addr":"172.21.15.100:6831","nonce":3019212393}]},"public_addr":"172.21.15.100:6827/3019212393","cluster_addr":"172.21.15.100:6829/3019212393","heartbeat_back_addr":"172.21.15.100:6833/3019212393","heartbeat_front_addr":"172.21.15.100:6831/3019212393","state":["exists","up"]},{"osd":4,"uuid":"6ecc1205-7c78-461e-8037-640136cb3658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":974906744},{"type":"v1","addr":"172.21.15.144:6801","nonce":974906744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":974906744},{"type":"v1","addr":"172.21.15.144:6803","nonce":974906744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":974906744},{"type":"v1","addr":"172.21.15.144:6807","nonce":974906744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":974906744},{"type":"v1","addr":"172.21.15.144:6805","nonce":974906744}]},"public_addr":"172.21.15.144:6801/974906744","cluster_addr":"172.21.15.144:6803/974906744","heartbeat_back_addr":"172.21.15.144:6807/974906744","heartbeat_front_addr":"172.21.15.144:6805/974906744","state":["exists","up"]},{"osd":5,"uuid":"54789cc8-1c3c-46a4-a92e-8881564338db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":2783162058},{"type":"v1","addr":"172.21.15.144:6809","nonce":2783162058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":2783162058},{"type":"v1","addr":"172.21.15.144:6811","nonce":2783162058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":2783162058},{"type":"v1","addr":"172.21.15.144:6815","nonce":2783162058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":2783162058},{"type":"v1","addr":"172.21.15.144:6813","nonce":2783162058}]},"public_addr":"172.21.15.144:6809/2783162058","cluster_addr":"172.21.15.144:6811/2783162058","heartbeat_back_addr":"172.21.15.144:6815/2783162058","heartbeat_front_addr":"172.21.15.144:6813/2783162058","state":["exists","up"]},{"osd":6,"uuid":"ac619497-7605-4ca6-84bf-3abb3f6f32d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":1183840595},{"type":"v1","addr":"172.21.15.144:6817","nonce":1183840595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":1183840595},{"type":"v1","addr":"172.21.15.144:6819","nonce":1183840595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":1183840595},{"type":"v1","addr":"172.21.15.144:6823","nonce":1183840595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":1183840595},{"type":"v1","addr":"172.21.15.144:6821","nonce":1183840595}]},"public_addr":"172.21.15.144:6817/1183840595","cluster_addr":"172.21.15.144:6819/1183840595","heartbeat_back_addr":"172.21.15.144:6823/1183840595","heartbeat_front_addr":"172.21.15.144:6821/1183840595","state":["exists","up"]},{"osd":7,"uuid":"b85b3422-9da2-4605-88e7-fe9c3b2020cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":2248617613},{"type":"v1","addr":"172.21.15.144:6825","nonce":2248617613}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":2248617613},{"type":"v1","addr":"172.21.15.144:6827","nonce":2248617613}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":2248617613},{"type":"v1","addr":"172.21.15.144:6831","nonce":2248617613}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":2248617613},{"type":"v1","addr":"172.21.15.144:6829","nonce":2248617613}]},"public_addr":"172.21.15.144:6825/2248617613","cluster_addr":"172.21.15.144:6827/2248617613","heartbeat_back_addr":"172.21.15.144:6831/2248617613","heartbeat_front_addr":"172.21.15.144:6829/2248617613","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-07T16:05:44.696671+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-07T16:06:33.787321+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-07T16:07:22.983158+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-07T16:08:12.935606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-07T16:09:00.218619+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-07T16:09:50.222185+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-07T16:10:37.856158+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-07T16:11:28.267877+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.7","mappings":[{"from":7,"to":4}]},{"pgid":"3.1","mappings":[{"from":0,"to":2}]},{"pgid":"3.12","mappings":[{"from":0,"to":2}]},{"pgid":"3.15","mappings":[{"from":4,"to":1}]},{"pgid":"3.17","mappings":[{"from":0,"to":2}]},{"pgid":"3.1d","mappings":[{"from":4,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.100:0/74983397":"2024-04-08T16:12:52.577534+0000","172.21.15.100:6800/190788518":"2024-04-08T16:12:52.577534+0000","172.21.15.100:0/1309290877":"2024-04-08T16:12:52.577534+0000","172.21.15.100:0/1472671368":"2024-04-08T16:03:24.146096+0000","172.21.15.100:6801/190788518":"2024-04-08T16:12:52.577534+0000","172.21.15.100:0/639120693":"2024-04-08T16:03:24.146096+0000","172.21.15.100:0/2474331937":"2024-04-08T16:03:24.146096+0000","172.21.15.100:0/3273267396":"2024-04-08T16:12:52.577534+0000","172.21.15.100:0/2535336108":"2024-04-08T16:12:52.577534+0000","172.21.15.100:6801/2710911757":"2024-04-08T16:03:24.146096+0000","172.21.15.100:6800/3440287045":"2024-04-08T16:02:53.269514+0000","172.21.15.100:0/109071285":"2024-04-08T16:02:53.269514+0000","172.21.15.100:6801/3440287045":"2024-04-08T16:02:53.269514+0000","172.21.15.100:0/4162615562":"2024-04-08T16:02:53.269514+0000","172.21.15.100:0/4121992264":"2024-04-08T16:02:53.269514+0000","172.21.15.100:6800/2710911757":"2024-04-08T16:03:24.146096+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-07T16:15:09.819 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-07T16:07:26.784071+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 7.889999866485596, 'score_stable': 7.889999866485596, 'optimal_score': 0.3799999952316284, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-07T16:12:11.856505+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '61', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-07T16:12:12.737709+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '62', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'rbd', 'create_time': '2024-04-07T16:13:28.913394+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '69', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 3, 'score_stable': 3, 'optimal_score': 1, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-07T16:15:09.820 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-07T16:15:10.126 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 1 2024-04-07T16:15:10.145 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:15:10 smithi100 bash[21346]: cluster 2024-04-07T16:15:08.655003+0000 mgr.y (mgr.14553) 105 : cluster 0 pgmap v75: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:15:10.145 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:15:10 smithi100 bash[21346]: audit 2024-04-07T16:15:09.811303+0000 mon.a (mon.0) 831 : audit 0 from='client.? 172.21.15.100:0/379578191' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-07T16:15:10.145 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:15:10 smithi100 bash[31660]: cluster 2024-04-07T16:15:08.655003+0000 mgr.y (mgr.14553) 105 : cluster 0 pgmap v75: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:15:10.145 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:15:10 smithi100 bash[31660]: audit 2024-04-07T16:15:09.811303+0000 mon.a (mon.0) 831 : audit 0 from='client.? 172.21.15.100:0/379578191' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-07T16:15:10.146 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-07T16:15:10.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:15:10 smithi144 bash[23522]: cluster 2024-04-07T16:15:08.655003+0000 mgr.y (mgr.14553) 105 : cluster 0 pgmap v75: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:15:10.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:15:10 smithi144 bash[23522]: audit 2024-04-07T16:15:09.811303+0000 mon.a (mon.0) 831 : audit 0 from='client.? 172.21.15.100:0/379578191' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-07T16:15:10.448 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 32 2024-04-07T16:15:10.465 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-07T16:15:10.772 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 64 2024-04-07T16:15:10.791 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2024-04-07T16:15:11.093 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 8 2024-04-07T16:15:11.110 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:15:11 smithi100 bash[21346]: audit 2024-04-07T16:15:10.141679+0000 mon.a (mon.0) 832 : audit 0 from='client.? 172.21.15.100:0/3571814016' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-07T16:15:11.110 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:15:11 smithi100 bash[21346]: audit 2024-04-07T16:15:10.463243+0000 mon.a (mon.0) 833 : audit 0 from='client.? 172.21.15.100:0/3717790617' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-07T16:15:11.110 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:15:11 smithi100 bash[21346]: audit 2024-04-07T16:15:10.787481+0000 mon.a (mon.0) 834 : audit 0 from='client.? 172.21.15.100:0/3493991550' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-07T16:15:11.110 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:15:11 smithi100 bash[31660]: audit 2024-04-07T16:15:10.141679+0000 mon.a (mon.0) 832 : audit 0 from='client.? 172.21.15.100:0/3571814016' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-07T16:15:11.110 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:15:11 smithi100 bash[31660]: audit 2024-04-07T16:15:10.463243+0000 mon.a (mon.0) 833 : audit 0 from='client.? 172.21.15.100:0/3717790617' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-07T16:15:11.110 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:15:11 smithi100 bash[31660]: audit 2024-04-07T16:15:10.787481+0000 mon.a (mon.0) 834 : audit 0 from='client.? 172.21.15.100:0/3493991550' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-07T16:15:11.111 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-07T16:15:11.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:15:11 smithi144 bash[23522]: audit 2024-04-07T16:15:10.141679+0000 mon.a (mon.0) 832 : audit 0 from='client.? 172.21.15.100:0/3571814016' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-07T16:15:11.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:15:11 smithi144 bash[23522]: audit 2024-04-07T16:15:10.463243+0000 mon.a (mon.0) 833 : audit 0 from='client.? 172.21.15.100:0/3717790617' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-07T16:15:11.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:15:11 smithi144 bash[23522]: audit 2024-04-07T16:15:10.787481+0000 mon.a (mon.0) 834 : audit 0 from='client.? 172.21.15.100:0/3493991550' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-07T16:15:11.419 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-07T16:15:11.419 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14424,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/3487006602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3487006602},{"type":"v1","addr":"172.21.15.100:6835","nonce":3487006602}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-07T16:12:13.365189+0000","modified":"2024-04-07T16:12:15.375075+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24322},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24322":{"gid":24322,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.144:6833/226244023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":226244023},{"type":"v1","addr":"172.21.15.144:6833","nonce":226244023}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-07T16:15:11.419 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 6 2024-04-07T16:15:11.437 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-07T16:15:11.751 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-07T16:15:11.751 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":69,"fsid":"0702e3a8-f4f8-11ee-b649-cb9ed24678a4","created":"2024-04-07T16:02:21.629692+0000","modified":"2024-04-07T16:13:38.014991+0000","last_up_change":"2024-04-07T16:11:30.210001+0000","last_in_change":"2024-04-07T16:11:04.793434+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-07T16:07:26.784071+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":7.8899998664855957,"score_stable":7.8899998664855957,"optimal_score":0.37999999523162842,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-07T16:12:11.856505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-07T16:12:12.737709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"rbd","create_time":"2024-04-07T16:13:28.913394+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":3,"score_stable":3,"optimal_score":1,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8facd48e-0a0c-4fda-aee2-f001b60a3811","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":1636950303},{"type":"v1","addr":"172.21.15.100:6803","nonce":1636950303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":1636950303},{"type":"v1","addr":"172.21.15.100:6805","nonce":1636950303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1636950303},{"type":"v1","addr":"172.21.15.100:6809","nonce":1636950303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":1636950303},{"type":"v1","addr":"172.21.15.100:6807","nonce":1636950303}]},"public_addr":"172.21.15.100:6803/1636950303","cluster_addr":"172.21.15.100:6805/1636950303","heartbeat_back_addr":"172.21.15.100:6809/1636950303","heartbeat_front_addr":"172.21.15.100:6807/1636950303","state":["exists","up"]},{"osd":1,"uuid":"cb5f6c23-6764-48b6-8b21-97a36b004786","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":470179506},{"type":"v1","addr":"172.21.15.100:6811","nonce":470179506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":470179506},{"type":"v1","addr":"172.21.15.100:6813","nonce":470179506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":470179506},{"type":"v1","addr":"172.21.15.100:6817","nonce":470179506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":470179506},{"type":"v1","addr":"172.21.15.100:6815","nonce":470179506}]},"public_addr":"172.21.15.100:6811/470179506","cluster_addr":"172.21.15.100:6813/470179506","heartbeat_back_addr":"172.21.15.100:6817/470179506","heartbeat_front_addr":"172.21.15.100:6815/470179506","state":["exists","up"]},{"osd":2,"uuid":"1650aedb-9ce9-4c14-a059-e1f089e5e63f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":2235490011},{"type":"v1","addr":"172.21.15.100:6819","nonce":2235490011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2235490011},{"type":"v1","addr":"172.21.15.100:6821","nonce":2235490011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":2235490011},{"type":"v1","addr":"172.21.15.100:6825","nonce":2235490011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":2235490011},{"type":"v1","addr":"172.21.15.100:6823","nonce":2235490011}]},"public_addr":"172.21.15.100:6819/2235490011","cluster_addr":"172.21.15.100:6821/2235490011","heartbeat_back_addr":"172.21.15.100:6825/2235490011","heartbeat_front_addr":"172.21.15.100:6823/2235490011","state":["exists","up"]},{"osd":3,"uuid":"17591c6b-862b-43b2-858b-02f4c3199247","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":3019212393},{"type":"v1","addr":"172.21.15.100:6827","nonce":3019212393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":3019212393},{"type":"v1","addr":"172.21.15.100:6829","nonce":3019212393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":3019212393},{"type":"v1","addr":"172.21.15.100:6833","nonce":3019212393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":3019212393},{"type":"v1","addr":"172.21.15.100:6831","nonce":3019212393}]},"public_addr":"172.21.15.100:6827/3019212393","cluster_addr":"172.21.15.100:6829/3019212393","heartbeat_back_addr":"172.21.15.100:6833/3019212393","heartbeat_front_addr":"172.21.15.100:6831/3019212393","state":["exists","up"]},{"osd":4,"uuid":"6ecc1205-7c78-461e-8037-640136cb3658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":974906744},{"type":"v1","addr":"172.21.15.144:6801","nonce":974906744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":974906744},{"type":"v1","addr":"172.21.15.144:6803","nonce":974906744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":974906744},{"type":"v1","addr":"172.21.15.144:6807","nonce":974906744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":974906744},{"type":"v1","addr":"172.21.15.144:6805","nonce":974906744}]},"public_addr":"172.21.15.144:6801/974906744","cluster_addr":"172.21.15.144:6803/974906744","heartbeat_back_addr":"172.21.15.144:6807/974906744","heartbeat_front_addr":"172.21.15.144:6805/974906744","state":["exists","up"]},{"osd":5,"uuid":"54789cc8-1c3c-46a4-a92e-8881564338db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":2783162058},{"type":"v1","addr":"172.21.15.144:6809","nonce":2783162058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":2783162058},{"type":"v1","addr":"172.21.15.144:6811","nonce":2783162058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":2783162058},{"type":"v1","addr":"172.21.15.144:6815","nonce":2783162058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":2783162058},{"type":"v1","addr":"172.21.15.144:6813","nonce":2783162058}]},"public_addr":"172.21.15.144:6809/2783162058","cluster_addr":"172.21.15.144:6811/2783162058","heartbeat_back_addr":"172.21.15.144:6815/2783162058","heartbeat_front_addr":"172.21.15.144:6813/2783162058","state":["exists","up"]},{"osd":6,"uuid":"ac619497-7605-4ca6-84bf-3abb3f6f32d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":1183840595},{"type":"v1","addr":"172.21.15.144:6817","nonce":1183840595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":1183840595},{"type":"v1","addr":"172.21.15.144:6819","nonce":1183840595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":1183840595},{"type":"v1","addr":"172.21.15.144:6823","nonce":1183840595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":1183840595},{"type":"v1","addr":"172.21.15.144:6821","nonce":1183840595}]},"public_addr":"172.21.15.144:6817/1183840595","cluster_addr":"172.21.15.144:6819/1183840595","heartbeat_back_addr":"172.21.15.144:6823/1183840595","heartbeat_front_addr":"172.21.15.144:6821/1183840595","state":["exists","up"]},{"osd":7,"uuid":"b85b3422-9da2-4605-88e7-fe9c3b2020cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":2248617613},{"type":"v1","addr":"172.21.15.144:6825","nonce":2248617613}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":2248617613},{"type":"v1","addr":"172.21.15.144:6827","nonce":2248617613}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":2248617613},{"type":"v1","addr":"172.21.15.144:6831","nonce":2248617613}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":2248617613},{"type":"v1","addr":"172.21.15.144:6829","nonce":2248617613}]},"public_addr":"172.21.15.144:6825/2248617613","cluster_addr":"172.21.15.144:6827/2248617613","heartbeat_back_addr":"172.21.15.144:6831/2248617613","heartbeat_front_addr":"172.21.15.144:6829/2248617613","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-07T16:05:44.696671+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-07T16:06:33.787321+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-07T16:07:22.983158+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-07T16:08:12.935606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-07T16:09:00.218619+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-07T16:09:50.222185+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-07T16:10:37.856158+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-07T16:11:28.267877+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.7","mappings":[{"from":7,"to":4}]},{"pgid":"3.1","mappings":[{"from":0,"to":2}]},{"pgid":"3.12","mappings":[{"from":0,"to":2}]},{"pgid":"3.15","mappings":[{"from":4,"to":1}]},{"pgid":"3.17","mappings":[{"from":0,"to":2}]},{"pgid":"3.1d","mappings":[{"from":4,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.100:0/74983397":"2024-04-08T16:12:52.577534+0000","172.21.15.100:6800/190788518":"2024-04-08T16:12:52.577534+0000","172.21.15.100:0/1309290877":"2024-04-08T16:12:52.577534+0000","172.21.15.100:0/1472671368":"2024-04-08T16:03:24.146096+0000","172.21.15.100:6801/190788518":"2024-04-08T16:12:52.577534+0000","172.21.15.100:0/639120693":"2024-04-08T16:03:24.146096+0000","172.21.15.100:0/2474331937":"2024-04-08T16:03:24.146096+0000","172.21.15.100:0/3273267396":"2024-04-08T16:12:52.577534+0000","172.21.15.100:0/2535336108":"2024-04-08T16:12:52.577534+0000","172.21.15.100:6801/2710911757":"2024-04-08T16:03:24.146096+0000","172.21.15.100:6800/3440287045":"2024-04-08T16:02:53.269514+0000","172.21.15.100:0/109071285":"2024-04-08T16:02:53.269514+0000","172.21.15.100:6801/3440287045":"2024-04-08T16:02:53.269514+0000","172.21.15.100:0/4162615562":"2024-04-08T16:02:53.269514+0000","172.21.15.100:0/4121992264":"2024-04-08T16:02:53.269514+0000","172.21.15.100:6800/2710911757":"2024-04-08T16:03:24.146096+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-07T16:15:11.769 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-07T16:07:26.784071+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 7.889999866485596, 'score_stable': 7.889999866485596, 'optimal_score': 0.3799999952316284, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-07T16:12:11.856505+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '61', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-07T16:12:12.737709+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '62', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'rbd', 'create_time': '2024-04-07T16:13:28.913394+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '69', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 3, 'score_stable': 3, 'optimal_score': 1, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-07T16:15:11.770 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-07T16:15:12.051 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 1 2024-04-07T16:15:12.068 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:15:12 smithi100 bash[21346]: cluster 2024-04-07T16:15:10.655801+0000 mgr.y (mgr.14553) 106 : cluster 0 pgmap v76: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:15:12.068 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:15:12 smithi100 bash[21346]: audit 2024-04-07T16:15:11.108358+0000 mon.c (mon.2) 36 : audit 0 from='client.? 172.21.15.100:0/2425728965' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]: dispatch 2024-04-07T16:15:12.068 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:15:12 smithi100 bash[21346]: audit 2024-04-07T16:15:11.434318+0000 mon.a (mon.0) 835 : audit 0 from='client.? 172.21.15.100:0/3422061778' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-07T16:15:12.068 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:15:12 smithi100 bash[21346]: audit 2024-04-07T16:15:11.460701+0000 mon.b (mon.1) 87 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:15:12.068 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:15:12 smithi100 bash[21346]: audit 2024-04-07T16:15:11.765470+0000 mon.a (mon.0) 836 : audit 0 from='client.? 172.21.15.100:0/3082337599' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-07T16:15:12.068 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:15:12 smithi100 bash[21346]: audit 2024-04-07T16:15:11.987735+0000 mon.b (mon.1) 88 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:15:12.068 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:15:12 smithi100 bash[21346]: audit 2024-04-07T16:15:11.989688+0000 mon.b (mon.1) 89 : audit 1 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:15:12.068 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:15:12 smithi100 bash[21346]: audit 2024-04-07T16:15:12.005176+0000 mon.a (mon.0) 837 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:15:12.069 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:15:12 smithi100 bash[31660]: cluster 2024-04-07T16:15:10.655801+0000 mgr.y (mgr.14553) 106 : cluster 0 pgmap v76: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:15:12.069 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:15:12 smithi100 bash[31660]: audit 2024-04-07T16:15:11.108358+0000 mon.c (mon.2) 36 : audit 0 from='client.? 172.21.15.100:0/2425728965' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]: dispatch 2024-04-07T16:15:12.069 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:15:12 smithi100 bash[31660]: audit 2024-04-07T16:15:11.434318+0000 mon.a (mon.0) 835 : audit 0 from='client.? 172.21.15.100:0/3422061778' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-07T16:15:12.069 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:15:12 smithi100 bash[31660]: audit 2024-04-07T16:15:11.460701+0000 mon.b (mon.1) 87 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:15:12.069 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:15:12 smithi100 bash[31660]: audit 2024-04-07T16:15:11.765470+0000 mon.a (mon.0) 836 : audit 0 from='client.? 172.21.15.100:0/3082337599' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-07T16:15:12.069 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:15:12 smithi100 bash[31660]: audit 2024-04-07T16:15:11.987735+0000 mon.b (mon.1) 88 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:15:12.069 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:15:12 smithi100 bash[31660]: audit 2024-04-07T16:15:11.989688+0000 mon.b (mon.1) 89 : audit 1 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:15:12.069 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:15:12 smithi100 bash[31660]: audit 2024-04-07T16:15:12.005176+0000 mon.a (mon.0) 837 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:15:12.070 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-07T16:15:12.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:15:12 smithi144 bash[23522]: cluster 2024-04-07T16:15:10.655801+0000 mgr.y (mgr.14553) 106 : cluster 0 pgmap v76: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:15:12.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:15:12 smithi144 bash[23522]: audit 2024-04-07T16:15:11.108358+0000 mon.c (mon.2) 36 : audit 0 from='client.? 172.21.15.100:0/2425728965' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]: dispatch 2024-04-07T16:15:12.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:15:12 smithi144 bash[23522]: audit 2024-04-07T16:15:11.434318+0000 mon.a (mon.0) 835 : audit 0 from='client.? 172.21.15.100:0/3422061778' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-07T16:15:12.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:15:12 smithi144 bash[23522]: audit 2024-04-07T16:15:11.460701+0000 mon.b (mon.1) 87 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:15:12.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:15:12 smithi144 bash[23522]: audit 2024-04-07T16:15:11.765470+0000 mon.a (mon.0) 836 : audit 0 from='client.? 172.21.15.100:0/3082337599' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-07T16:15:12.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:15:12 smithi144 bash[23522]: audit 2024-04-07T16:15:11.987735+0000 mon.b (mon.1) 88 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:15:12.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:15:12 smithi144 bash[23522]: audit 2024-04-07T16:15:11.989688+0000 mon.b (mon.1) 89 : audit 1 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:15:12.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:15:12 smithi144 bash[23522]: audit 2024-04-07T16:15:12.005176+0000 mon.a (mon.0) 837 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:15:12.360 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 32 2024-04-07T16:15:12.379 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-07T16:15:12.661 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 64 2024-04-07T16:15:12.677 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2024-04-07T16:15:12.977 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 8 2024-04-07T16:15:12.997 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-07T16:15:13.313 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-07T16:15:13.313 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14424,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/3487006602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3487006602},{"type":"v1","addr":"172.21.15.100:6835","nonce":3487006602}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-07T16:12:13.365189+0000","modified":"2024-04-07T16:12:15.375075+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24322},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24322":{"gid":24322,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.144:6833/226244023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":226244023},{"type":"v1","addr":"172.21.15.144:6833","nonce":226244023}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-07T16:15:13.313 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 6 2024-04-07T16:15:13.330 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:15:13 smithi100 bash[21346]: audit 2024-04-07T16:15:12.067079+0000 mon.c (mon.2) 37 : audit 0 from='client.? 172.21.15.100:0/4075938311' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-07T16:15:13.330 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:15:13 smithi100 bash[21346]: audit 2024-04-07T16:15:12.375450+0000 mon.a (mon.0) 838 : audit 0 from='client.? 172.21.15.100:0/4014253115' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-07T16:15:13.330 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:15:13 smithi100 bash[21346]: audit 2024-04-07T16:15:12.675297+0000 mon.a (mon.0) 839 : audit 0 from='client.? 172.21.15.100:0/4081063795' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-07T16:15:13.330 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:15:13 smithi100 bash[21346]: audit 2024-04-07T16:15:12.993246+0000 mon.c (mon.2) 38 : audit 0 from='client.? 172.21.15.100:0/692979912' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]: dispatch 2024-04-07T16:15:13.331 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:15:13 smithi100 bash[31660]: audit 2024-04-07T16:15:12.067079+0000 mon.c (mon.2) 37 : audit 0 from='client.? 172.21.15.100:0/4075938311' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-07T16:15:13.331 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:15:13 smithi100 bash[31660]: audit 2024-04-07T16:15:12.375450+0000 mon.a (mon.0) 838 : audit 0 from='client.? 172.21.15.100:0/4014253115' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-07T16:15:13.331 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:15:13 smithi100 bash[31660]: audit 2024-04-07T16:15:12.675297+0000 mon.a (mon.0) 839 : audit 0 from='client.? 172.21.15.100:0/4081063795' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-07T16:15:13.331 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:15:13 smithi100 bash[31660]: audit 2024-04-07T16:15:12.993246+0000 mon.c (mon.2) 38 : audit 0 from='client.? 172.21.15.100:0/692979912' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]: dispatch 2024-04-07T16:15:13.332 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-07T16:15:13.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:15:13 smithi144 bash[23522]: audit 2024-04-07T16:15:12.067079+0000 mon.c (mon.2) 37 : audit 0 from='client.? 172.21.15.100:0/4075938311' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-07T16:15:13.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:15:13 smithi144 bash[23522]: audit 2024-04-07T16:15:12.375450+0000 mon.a (mon.0) 838 : audit 0 from='client.? 172.21.15.100:0/4014253115' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-07T16:15:13.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:15:13 smithi144 bash[23522]: audit 2024-04-07T16:15:12.675297+0000 mon.a (mon.0) 839 : audit 0 from='client.? 172.21.15.100:0/4081063795' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-07T16:15:13.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:15:13 smithi144 bash[23522]: audit 2024-04-07T16:15:12.993246+0000 mon.c (mon.2) 38 : audit 0 from='client.? 172.21.15.100:0/692979912' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]: dispatch 2024-04-07T16:15:13.621 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-07T16:15:13.622 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":69,"fsid":"0702e3a8-f4f8-11ee-b649-cb9ed24678a4","created":"2024-04-07T16:02:21.629692+0000","modified":"2024-04-07T16:13:38.014991+0000","last_up_change":"2024-04-07T16:11:30.210001+0000","last_in_change":"2024-04-07T16:11:04.793434+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-07T16:07:26.784071+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":7.8899998664855957,"score_stable":7.8899998664855957,"optimal_score":0.37999999523162842,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-07T16:12:11.856505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-07T16:12:12.737709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"rbd","create_time":"2024-04-07T16:13:28.913394+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":3,"score_stable":3,"optimal_score":1,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8facd48e-0a0c-4fda-aee2-f001b60a3811","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":1636950303},{"type":"v1","addr":"172.21.15.100:6803","nonce":1636950303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":1636950303},{"type":"v1","addr":"172.21.15.100:6805","nonce":1636950303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1636950303},{"type":"v1","addr":"172.21.15.100:6809","nonce":1636950303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":1636950303},{"type":"v1","addr":"172.21.15.100:6807","nonce":1636950303}]},"public_addr":"172.21.15.100:6803/1636950303","cluster_addr":"172.21.15.100:6805/1636950303","heartbeat_back_addr":"172.21.15.100:6809/1636950303","heartbeat_front_addr":"172.21.15.100:6807/1636950303","state":["exists","up"]},{"osd":1,"uuid":"cb5f6c23-6764-48b6-8b21-97a36b004786","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":470179506},{"type":"v1","addr":"172.21.15.100:6811","nonce":470179506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":470179506},{"type":"v1","addr":"172.21.15.100:6813","nonce":470179506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":470179506},{"type":"v1","addr":"172.21.15.100:6817","nonce":470179506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":470179506},{"type":"v1","addr":"172.21.15.100:6815","nonce":470179506}]},"public_addr":"172.21.15.100:6811/470179506","cluster_addr":"172.21.15.100:6813/470179506","heartbeat_back_addr":"172.21.15.100:6817/470179506","heartbeat_front_addr":"172.21.15.100:6815/470179506","state":["exists","up"]},{"osd":2,"uuid":"1650aedb-9ce9-4c14-a059-e1f089e5e63f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":2235490011},{"type":"v1","addr":"172.21.15.100:6819","nonce":2235490011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2235490011},{"type":"v1","addr":"172.21.15.100:6821","nonce":2235490011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":2235490011},{"type":"v1","addr":"172.21.15.100:6825","nonce":2235490011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":2235490011},{"type":"v1","addr":"172.21.15.100:6823","nonce":2235490011}]},"public_addr":"172.21.15.100:6819/2235490011","cluster_addr":"172.21.15.100:6821/2235490011","heartbeat_back_addr":"172.21.15.100:6825/2235490011","heartbeat_front_addr":"172.21.15.100:6823/2235490011","state":["exists","up"]},{"osd":3,"uuid":"17591c6b-862b-43b2-858b-02f4c3199247","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":3019212393},{"type":"v1","addr":"172.21.15.100:6827","nonce":3019212393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":3019212393},{"type":"v1","addr":"172.21.15.100:6829","nonce":3019212393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":3019212393},{"type":"v1","addr":"172.21.15.100:6833","nonce":3019212393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":3019212393},{"type":"v1","addr":"172.21.15.100:6831","nonce":3019212393}]},"public_addr":"172.21.15.100:6827/3019212393","cluster_addr":"172.21.15.100:6829/3019212393","heartbeat_back_addr":"172.21.15.100:6833/3019212393","heartbeat_front_addr":"172.21.15.100:6831/3019212393","state":["exists","up"]},{"osd":4,"uuid":"6ecc1205-7c78-461e-8037-640136cb3658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":974906744},{"type":"v1","addr":"172.21.15.144:6801","nonce":974906744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":974906744},{"type":"v1","addr":"172.21.15.144:6803","nonce":974906744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":974906744},{"type":"v1","addr":"172.21.15.144:6807","nonce":974906744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":974906744},{"type":"v1","addr":"172.21.15.144:6805","nonce":974906744}]},"public_addr":"172.21.15.144:6801/974906744","cluster_addr":"172.21.15.144:6803/974906744","heartbeat_back_addr":"172.21.15.144:6807/974906744","heartbeat_front_addr":"172.21.15.144:6805/974906744","state":["exists","up"]},{"osd":5,"uuid":"54789cc8-1c3c-46a4-a92e-8881564338db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":2783162058},{"type":"v1","addr":"172.21.15.144:6809","nonce":2783162058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":2783162058},{"type":"v1","addr":"172.21.15.144:6811","nonce":2783162058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":2783162058},{"type":"v1","addr":"172.21.15.144:6815","nonce":2783162058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":2783162058},{"type":"v1","addr":"172.21.15.144:6813","nonce":2783162058}]},"public_addr":"172.21.15.144:6809/2783162058","cluster_addr":"172.21.15.144:6811/2783162058","heartbeat_back_addr":"172.21.15.144:6815/2783162058","heartbeat_front_addr":"172.21.15.144:6813/2783162058","state":["exists","up"]},{"osd":6,"uuid":"ac619497-7605-4ca6-84bf-3abb3f6f32d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":1183840595},{"type":"v1","addr":"172.21.15.144:6817","nonce":1183840595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":1183840595},{"type":"v1","addr":"172.21.15.144:6819","nonce":1183840595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":1183840595},{"type":"v1","addr":"172.21.15.144:6823","nonce":1183840595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":1183840595},{"type":"v1","addr":"172.21.15.144:6821","nonce":1183840595}]},"public_addr":"172.21.15.144:6817/1183840595","cluster_addr":"172.21.15.144:6819/1183840595","heartbeat_back_addr":"172.21.15.144:6823/1183840595","heartbeat_front_addr":"172.21.15.144:6821/1183840595","state":["exists","up"]},{"osd":7,"uuid":"b85b3422-9da2-4605-88e7-fe9c3b2020cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":2248617613},{"type":"v1","addr":"172.21.15.144:6825","nonce":2248617613}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":2248617613},{"type":"v1","addr":"172.21.15.144:6827","nonce":2248617613}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":2248617613},{"type":"v1","addr":"172.21.15.144:6831","nonce":2248617613}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":2248617613},{"type":"v1","addr":"172.21.15.144:6829","nonce":2248617613}]},"public_addr":"172.21.15.144:6825/2248617613","cluster_addr":"172.21.15.144:6827/2248617613","heartbeat_back_addr":"172.21.15.144:6831/2248617613","heartbeat_front_addr":"172.21.15.144:6829/2248617613","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-07T16:05:44.696671+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-07T16:06:33.787321+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-07T16:07:22.983158+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-07T16:08:12.935606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-07T16:09:00.218619+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-07T16:09:50.222185+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-07T16:10:37.856158+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-07T16:11:28.267877+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.7","mappings":[{"from":7,"to":4}]},{"pgid":"3.1","mappings":[{"from":0,"to":2}]},{"pgid":"3.12","mappings":[{"from":0,"to":2}]},{"pgid":"3.15","mappings":[{"from":4,"to":1}]},{"pgid":"3.17","mappings":[{"from":0,"to":2}]},{"pgid":"3.1d","mappings":[{"from":4,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.100:0/74983397":"2024-04-08T16:12:52.577534+0000","172.21.15.100:6800/190788518":"2024-04-08T16:12:52.577534+0000","172.21.15.100:0/1309290877":"2024-04-08T16:12:52.577534+0000","172.21.15.100:0/1472671368":"2024-04-08T16:03:24.146096+0000","172.21.15.100:6801/190788518":"2024-04-08T16:12:52.577534+0000","172.21.15.100:0/639120693":"2024-04-08T16:03:24.146096+0000","172.21.15.100:0/2474331937":"2024-04-08T16:03:24.146096+0000","172.21.15.100:0/3273267396":"2024-04-08T16:12:52.577534+0000","172.21.15.100:0/2535336108":"2024-04-08T16:12:52.577534+0000","172.21.15.100:6801/2710911757":"2024-04-08T16:03:24.146096+0000","172.21.15.100:6800/3440287045":"2024-04-08T16:02:53.269514+0000","172.21.15.100:0/109071285":"2024-04-08T16:02:53.269514+0000","172.21.15.100:6801/3440287045":"2024-04-08T16:02:53.269514+0000","172.21.15.100:0/4162615562":"2024-04-08T16:02:53.269514+0000","172.21.15.100:0/4121992264":"2024-04-08T16:02:53.269514+0000","172.21.15.100:6800/2710911757":"2024-04-08T16:03:24.146096+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-07T16:15:13.640 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-07T16:15:13.962 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-07T16:15:13.962 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14424,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/3487006602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3487006602},{"type":"v1","addr":"172.21.15.100:6835","nonce":3487006602}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-07T16:12:13.365189+0000","modified":"2024-04-07T16:12:15.375075+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24322},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24322":{"gid":24322,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.144:6833/226244023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":226244023},{"type":"v1","addr":"172.21.15.144:6833","nonce":226244023}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-07T16:15:13.962 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 6 2024-04-07T16:15:13.980 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 6, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-07T16:12:13.365189+0000', 'modified': '2024-04-07T16:12:15.375075+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24322}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24322': {'gid': 24322, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.144:6833/226244023', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6832', 'nonce': 226244023}, {'type': 'v1', 'addr': '172.21.15.144:6833', 'nonce': 226244023}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-07T16:15:13.980 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-07T16:15:13.980 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-07T16:15:14.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-07T16:15:14.117+0000 7fb15dffb640 0 client.24761 ms_handle_reset on v2:172.21.15.144:6832/226244023 2024-04-07T16:15:14.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-07T16:15:14.137+0000 7fb15dffb640 0 client.24767 ms_handle_reset on v2:172.21.15.144:6832/226244023 2024-04-07T16:15:14.125 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-04-07T16:15:14.125 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_fsid": "0702e3a8-f4f8-11ee-b649-cb9ed24678a4", 2024-04-07T16:15:14.125 INFO:teuthology.orchestra.run.smithi100.stdout: "whoami": 0, 2024-04-07T16:15:14.125 INFO:teuthology.orchestra.run.smithi100.stdout: "id": 24322, 2024-04-07T16:15:14.125 INFO:teuthology.orchestra.run.smithi100.stdout: "want_state": "up:active", 2024-04-07T16:15:14.125 INFO:teuthology.orchestra.run.smithi100.stdout: "state": "up:active", 2024-04-07T16:15:14.125 INFO:teuthology.orchestra.run.smithi100.stdout: "fs_name": "cephfs", 2024-04-07T16:15:14.125 INFO:teuthology.orchestra.run.smithi100.stdout: "rank_uptime": 180.769916739, 2024-04-07T16:15:14.125 INFO:teuthology.orchestra.run.smithi100.stdout: "mdsmap_epoch": 6, 2024-04-07T16:15:14.125 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_epoch": 69, 2024-04-07T16:15:14.125 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_epoch_barrier": 62, 2024-04-07T16:15:14.125 INFO:teuthology.orchestra.run.smithi100.stdout: "uptime": 186.41574048800001 2024-04-07T16:15:14.125 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-04-07T16:15:14.140 INFO:tasks.cephadm:Setup complete, yielding 2024-04-07T16:15:14.140 INFO:teuthology.run_tasks:Running task print... 2024-04-07T16:15:14.150 INFO:teuthology.task.print:**** done end installing reef cephadm ... 2024-04-07T16:15:14.150 INFO:teuthology.run_tasks:Running task print... 2024-04-07T16:15:14.158 INFO:teuthology.task.print:**** done start cephadm.shell ceph config set mgr... 2024-04-07T16:15:14.158 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-07T16:15:14.168 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi100.front.sepia.ceph.com 2024-04-07T16:15:14.168 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0702e3a8-f4f8-11ee-b649-cb9ed24678a4 -- bash -c 'ceph config set mgr mgr/cephadm/use_repo_digest true --force' 2024-04-07T16:15:14.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:15:14 smithi144 bash[23522]: cluster 2024-04-07T16:15:12.656333+0000 mgr.y (mgr.14553) 107 : cluster 0 pgmap v77: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:15:14.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:15:14 smithi144 bash[23522]: audit 2024-04-07T16:15:13.328131+0000 mon.a (mon.0) 840 : audit 0 from='client.? 172.21.15.100:0/590210072' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-07T16:15:14.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:15:14 smithi144 bash[23522]: audit 2024-04-07T16:15:13.633421+0000 mon.c (mon.2) 39 : audit 0 from='client.? 172.21.15.100:0/4269255161' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-07T16:15:14.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:15:14 smithi144 bash[23522]: audit 2024-04-07T16:15:13.977341+0000 mon.c (mon.2) 40 : audit 0 from='client.? 172.21.15.100:0/3624916027' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-07T16:15:14.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:15:14 smithi100 bash[21346]: cluster 2024-04-07T16:15:12.656333+0000 mgr.y (mgr.14553) 107 : cluster 0 pgmap v77: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:15:14.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:15:14 smithi100 bash[21346]: audit 2024-04-07T16:15:13.328131+0000 mon.a (mon.0) 840 : audit 0 from='client.? 172.21.15.100:0/590210072' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-07T16:15:14.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:15:14 smithi100 bash[21346]: audit 2024-04-07T16:15:13.633421+0000 mon.c (mon.2) 39 : audit 0 from='client.? 172.21.15.100:0/4269255161' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-07T16:15:14.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:15:14 smithi100 bash[21346]: audit 2024-04-07T16:15:13.977341+0000 mon.c (mon.2) 40 : audit 0 from='client.? 172.21.15.100:0/3624916027' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-07T16:15:14.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:15:14 smithi100 bash[31660]: cluster 2024-04-07T16:15:12.656333+0000 mgr.y (mgr.14553) 107 : cluster 0 pgmap v77: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:15:14.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:15:14 smithi100 bash[31660]: audit 2024-04-07T16:15:13.328131+0000 mon.a (mon.0) 840 : audit 0 from='client.? 172.21.15.100:0/590210072' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-07T16:15:14.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:15:14 smithi100 bash[31660]: audit 2024-04-07T16:15:13.633421+0000 mon.c (mon.2) 39 : audit 0 from='client.? 172.21.15.100:0/4269255161' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-07T16:15:14.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:15:14 smithi100 bash[31660]: audit 2024-04-07T16:15:13.977341+0000 mon.c (mon.2) 40 : audit 0 from='client.? 172.21.15.100:0/3624916027' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-07T16:15:15.437 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:15:15 smithi100 bash[21601]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:15:15] "GET /metrics HTTP/1.1" 200 33740 "" "Prometheus/2.43.0" 2024-04-07T16:15:16.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:15:16 smithi144 bash[23522]: cluster 2024-04-07T16:15:14.657586+0000 mgr.y (mgr.14553) 108 : cluster 0 pgmap v78: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:15:16.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:15:16 smithi100 bash[21346]: cluster 2024-04-07T16:15:14.657586+0000 mgr.y (mgr.14553) 108 : cluster 0 pgmap v78: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:15:16.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:15:16 smithi100 bash[31660]: cluster 2024-04-07T16:15:14.657586+0000 mgr.y (mgr.14553) 108 : cluster 0 pgmap v78: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:15:18.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:15:18 smithi144 bash[23522]: cluster 2024-04-07T16:15:16.658254+0000 mgr.y (mgr.14553) 109 : cluster 0 pgmap v79: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:15:18.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:15:18 smithi100 bash[21346]: cluster 2024-04-07T16:15:16.658254+0000 mgr.y (mgr.14553) 109 : cluster 0 pgmap v79: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:15:18.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:15:18 smithi100 bash[31660]: cluster 2024-04-07T16:15:16.658254+0000 mgr.y (mgr.14553) 109 : cluster 0 pgmap v79: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:15:18.888 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/mon.a/config 2024-04-07T16:15:20.131 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:15:20 smithi100 bash[21346]: cluster 2024-04-07T16:15:18.658905+0000 mgr.y (mgr.14553) 110 : cluster 0 pgmap v80: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:15:20.131 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:15:20 smithi100 bash[31660]: cluster 2024-04-07T16:15:18.658905+0000 mgr.y (mgr.14553) 110 : cluster 0 pgmap v80: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:15:20.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:15:20 smithi144 bash[23522]: cluster 2024-04-07T16:15:18.658905+0000 mgr.y (mgr.14553) 110 : cluster 0 pgmap v80: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:15:21.937 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:15:21 smithi100 bash[21601]: [07/Apr/2024:16:15:21] ENGINE Bus STOPPING 2024-04-07T16:15:21.937 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:15:21 smithi100 bash[21601]: [07/Apr/2024:16:15:21] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-04-07T16:15:21.937 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:15:21 smithi100 bash[21601]: [07/Apr/2024:16:15:21] ENGINE Bus STOPPED 2024-04-07T16:15:21.937 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:15:21 smithi100 bash[21601]: [07/Apr/2024:16:15:21] ENGINE Bus STARTING 2024-04-07T16:15:21.937 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:15:21 smithi100 bash[21601]: [07/Apr/2024:16:15:21] ENGINE Serving on http://:::9283 2024-04-07T16:15:21.938 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:15:21 smithi100 bash[21601]: [07/Apr/2024:16:15:21] ENGINE Bus STARTED 2024-04-07T16:15:22.290 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:15:22 smithi100 bash[31660]: cluster 2024-04-07T16:15:20.659939+0000 mgr.y (mgr.14553) 111 : cluster 0 pgmap v81: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:15:22.290 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:15:22 smithi100 bash[31660]: audit 2024-04-07T16:15:21.670793+0000 mon.a (mon.0) 841 : audit 1 from='client.? 172.21.15.100:0/3489764091' entity='client.admin' 2024-04-07T16:15:22.290 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:15:22 smithi100 bash[31660]: audit 2024-04-07T16:15:21.720691+0000 mon.b (mon.1) 90 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:15:22.290 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:15:22 smithi100 bash[31660]: audit 2024-04-07T16:15:21.723320+0000 mon.b (mon.1) 91 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:15:22.290 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:15:22 smithi100 bash[31660]: audit 2024-04-07T16:15:21.724608+0000 mon.b (mon.1) 92 : audit 1 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:15:22.290 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:15:22 smithi100 bash[31660]: audit 2024-04-07T16:15:21.736095+0000 mon.a (mon.0) 842 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:15:22.291 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:15:22 smithi100 bash[21346]: cluster 2024-04-07T16:15:20.659939+0000 mgr.y (mgr.14553) 111 : cluster 0 pgmap v81: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:15:22.291 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:15:22 smithi100 bash[21346]: audit 2024-04-07T16:15:21.670793+0000 mon.a (mon.0) 841 : audit 1 from='client.? 172.21.15.100:0/3489764091' entity='client.admin' 2024-04-07T16:15:22.291 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:15:22 smithi100 bash[21346]: audit 2024-04-07T16:15:21.720691+0000 mon.b (mon.1) 90 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:15:22.291 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:15:22 smithi100 bash[21346]: audit 2024-04-07T16:15:21.723320+0000 mon.b (mon.1) 91 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:15:22.291 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:15:22 smithi100 bash[21346]: audit 2024-04-07T16:15:21.724608+0000 mon.b (mon.1) 92 : audit 1 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:15:22.291 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:15:22 smithi100 bash[21346]: audit 2024-04-07T16:15:21.736095+0000 mon.a (mon.0) 842 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:15:22.292 INFO:teuthology.run_tasks:Running task print... 2024-04-07T16:15:22.301 INFO:teuthology.task.print:**** done cephadm.shell ceph config set mgr... 2024-04-07T16:15:22.301 INFO:teuthology.run_tasks:Running task print... 2024-04-07T16:15:22.329 INFO:teuthology.task.print:**** done start telemetry reef... 2024-04-07T16:15:22.329 INFO:teuthology.run_tasks:Running task workunit... 2024-04-07T16:15:22.342 INFO:tasks.workunit:Pulling workunits from ref adce92d056533f22dc1ee37e3141d098b046ec1b 2024-04-07T16:15:22.342 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-04-07T16:15:22.342 DEBUG:teuthology.orchestra.run.smithi100:> stat -- /home/ubuntu/cephtest/mnt.0 2024-04-07T16:15:22.348 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-07T16:15:22.348 INFO:teuthology.orchestra.run.smithi100.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-04-07T16:15:22.349 DEBUG:teuthology.orchestra.run.smithi100:> mkdir -- /home/ubuntu/cephtest/mnt.0 2024-04-07T16:15:22.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:15:22 smithi144 bash[23522]: cluster 2024-04-07T16:15:20.659939+0000 mgr.y (mgr.14553) 111 : cluster 0 pgmap v81: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:15:22.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:15:22 smithi144 bash[23522]: audit 2024-04-07T16:15:21.670793+0000 mon.a (mon.0) 841 : audit 1 from='client.? 172.21.15.100:0/3489764091' entity='client.admin' 2024-04-07T16:15:22.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:15:22 smithi144 bash[23522]: audit 2024-04-07T16:15:21.720691+0000 mon.b (mon.1) 90 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:15:22.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:15:22 smithi144 bash[23522]: audit 2024-04-07T16:15:21.723320+0000 mon.b (mon.1) 91 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:15:22.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:15:22 smithi144 bash[23522]: audit 2024-04-07T16:15:21.724608+0000 mon.b (mon.1) 92 : audit 1 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:15:22.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:15:22 smithi144 bash[23522]: audit 2024-04-07T16:15:21.736095+0000 mon.a (mon.0) 842 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:15:22.396 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2024-04-07T16:15:22.396 DEBUG:teuthology.orchestra.run.smithi100:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2024-04-07T16:15:22.444 INFO:tasks.workunit:timeout=3h 2024-04-07T16:15:22.444 INFO:tasks.workunit:cleanup=True 2024-04-07T16:15:22.445 DEBUG:teuthology.orchestra.run.smithi100:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/ceph/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout adce92d056533f22dc1ee37e3141d098b046ec1b 2024-04-07T16:15:22.494 INFO:tasks.workunit.client.0.smithi100.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-04-07T16:15:23.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:15:23 smithi144 bash[23522]: audit 2024-04-07T16:15:22.700295+0000 mon.b (mon.1) 93 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:15:23.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:15:23 smithi100 bash[21346]: audit 2024-04-07T16:15:22.700295+0000 mon.b (mon.1) 93 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:15:23.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:15:23 smithi100 bash[31660]: audit 2024-04-07T16:15:22.700295+0000 mon.b (mon.1) 93 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:15:24.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:15:24 smithi144 bash[23522]: cluster 2024-04-07T16:15:22.660508+0000 mgr.y (mgr.14553) 112 : cluster 0 pgmap v82: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:15:24.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:15:24 smithi100 bash[21346]: cluster 2024-04-07T16:15:22.660508+0000 mgr.y (mgr.14553) 112 : cluster 0 pgmap v82: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:15:24.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:15:24 smithi100 bash[31660]: cluster 2024-04-07T16:15:22.660508+0000 mgr.y (mgr.14553) 112 : cluster 0 pgmap v82: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:15:25.437 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:15:25 smithi100 bash[21601]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:15:25] "GET /metrics HTTP/1.1" 200 33740 "" "Prometheus/2.43.0" 2024-04-07T16:15:26.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:15:26 smithi144 bash[23522]: cluster 2024-04-07T16:15:24.661680+0000 mgr.y (mgr.14553) 113 : cluster 0 pgmap v83: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:15:26.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:15:26 smithi100 bash[21346]: cluster 2024-04-07T16:15:24.661680+0000 mgr.y (mgr.14553) 113 : cluster 0 pgmap v83: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:15:26.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:15:26 smithi100 bash[31660]: cluster 2024-04-07T16:15:24.661680+0000 mgr.y (mgr.14553) 113 : cluster 0 pgmap v83: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:15:28.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:15:28 smithi144 bash[23522]: cluster 2024-04-07T16:15:26.662105+0000 mgr.y (mgr.14553) 114 : cluster 0 pgmap v84: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:15:28.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:15:28 smithi100 bash[21346]: cluster 2024-04-07T16:15:26.662105+0000 mgr.y (mgr.14553) 114 : cluster 0 pgmap v84: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:15:28.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:15:28 smithi100 bash[31660]: cluster 2024-04-07T16:15:26.662105+0000 mgr.y (mgr.14553) 114 : cluster 0 pgmap v84: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:15:30.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:15:30 smithi144 bash[23522]: cluster 2024-04-07T16:15:28.662489+0000 mgr.y (mgr.14553) 115 : cluster 0 pgmap v85: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:15:30.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:15:30 smithi100 bash[21346]: cluster 2024-04-07T16:15:28.662489+0000 mgr.y (mgr.14553) 115 : cluster 0 pgmap v85: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:15:30.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:15:30 smithi100 bash[31660]: cluster 2024-04-07T16:15:28.662489+0000 mgr.y (mgr.14553) 115 : cluster 0 pgmap v85: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:15:32.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:15:32 smithi144 bash[23522]: cluster 2024-04-07T16:15:30.663201+0000 mgr.y (mgr.14553) 116 : cluster 0 pgmap v86: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:15:32.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:15:32 smithi100 bash[21346]: cluster 2024-04-07T16:15:30.663201+0000 mgr.y (mgr.14553) 116 : cluster 0 pgmap v86: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:15:32.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:15:32 smithi100 bash[31660]: cluster 2024-04-07T16:15:30.663201+0000 mgr.y (mgr.14553) 116 : cluster 0 pgmap v86: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:15:34.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:15:34 smithi144 bash[23522]: cluster 2024-04-07T16:15:32.663931+0000 mgr.y (mgr.14553) 117 : cluster 0 pgmap v87: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:15:34.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:15:34 smithi100 bash[31660]: cluster 2024-04-07T16:15:32.663931+0000 mgr.y (mgr.14553) 117 : cluster 0 pgmap v87: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:15:34.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:15:34 smithi100 bash[21346]: cluster 2024-04-07T16:15:32.663931+0000 mgr.y (mgr.14553) 117 : cluster 0 pgmap v87: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:15:35.437 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:15:35 smithi100 bash[21601]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:15:35] "GET /metrics HTTP/1.1" 200 33740 "" "Prometheus/2.43.0" 2024-04-07T16:15:36.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:15:36 smithi144 bash[23522]: cluster 2024-04-07T16:15:34.664559+0000 mgr.y (mgr.14553) 118 : cluster 0 pgmap v88: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:15:36.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:15:36 smithi100 bash[31660]: cluster 2024-04-07T16:15:34.664559+0000 mgr.y (mgr.14553) 118 : cluster 0 pgmap v88: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:15:36.440 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:15:36 smithi100 bash[21346]: cluster 2024-04-07T16:15:34.664559+0000 mgr.y (mgr.14553) 118 : cluster 0 pgmap v88: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:15:38.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:15:38 smithi144 bash[23522]: cluster 2024-04-07T16:15:36.665161+0000 mgr.y (mgr.14553) 119 : cluster 0 pgmap v89: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:15:38.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:15:38 smithi144 bash[23522]: audit 2024-04-07T16:15:37.700138+0000 mon.b (mon.1) 94 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:15:38.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:15:38 smithi100 bash[21346]: cluster 2024-04-07T16:15:36.665161+0000 mgr.y (mgr.14553) 119 : cluster 0 pgmap v89: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:15:38.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:15:38 smithi100 bash[21346]: audit 2024-04-07T16:15:37.700138+0000 mon.b (mon.1) 94 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:15:38.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:15:38 smithi100 bash[31660]: cluster 2024-04-07T16:15:36.665161+0000 mgr.y (mgr.14553) 119 : cluster 0 pgmap v89: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:15:38.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:15:38 smithi100 bash[31660]: audit 2024-04-07T16:15:37.700138+0000 mon.b (mon.1) 94 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:15:40.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:15:40 smithi144 bash[23522]: cluster 2024-04-07T16:15:38.665503+0000 mgr.y (mgr.14553) 120 : cluster 0 pgmap v90: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:15:40.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:15:40 smithi100 bash[21346]: cluster 2024-04-07T16:15:38.665503+0000 mgr.y (mgr.14553) 120 : cluster 0 pgmap v90: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:15:40.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:15:40 smithi100 bash[31660]: cluster 2024-04-07T16:15:38.665503+0000 mgr.y (mgr.14553) 120 : cluster 0 pgmap v90: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:15:42.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:15:42 smithi144 bash[23522]: cluster 2024-04-07T16:15:40.666155+0000 mgr.y (mgr.14553) 121 : cluster 0 pgmap v91: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:15:42.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:15:42 smithi100 bash[21346]: cluster 2024-04-07T16:15:40.666155+0000 mgr.y (mgr.14553) 121 : cluster 0 pgmap v91: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:15:42.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:15:42 smithi100 bash[31660]: cluster 2024-04-07T16:15:40.666155+0000 mgr.y (mgr.14553) 121 : cluster 0 pgmap v91: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:15:44.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:15:44 smithi144 bash[23522]: cluster 2024-04-07T16:15:42.666542+0000 mgr.y (mgr.14553) 122 : cluster 0 pgmap v92: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:15:44.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:15:44 smithi100 bash[21346]: cluster 2024-04-07T16:15:42.666542+0000 mgr.y (mgr.14553) 122 : cluster 0 pgmap v92: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:15:44.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:15:44 smithi100 bash[31660]: cluster 2024-04-07T16:15:42.666542+0000 mgr.y (mgr.14553) 122 : cluster 0 pgmap v92: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:15:45.437 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:15:45 smithi100 bash[21601]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:15:45] "GET /metrics HTTP/1.1" 200 33741 "" "Prometheus/2.43.0" 2024-04-07T16:15:46.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:15:46 smithi144 bash[23522]: cluster 2024-04-07T16:15:44.667250+0000 mgr.y (mgr.14553) 123 : cluster 0 pgmap v93: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:15:46.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:15:46 smithi100 bash[21346]: cluster 2024-04-07T16:15:44.667250+0000 mgr.y (mgr.14553) 123 : cluster 0 pgmap v93: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:15:46.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:15:46 smithi100 bash[31660]: cluster 2024-04-07T16:15:44.667250+0000 mgr.y (mgr.14553) 123 : cluster 0 pgmap v93: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:15:48.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:15:48 smithi144 bash[23522]: cluster 2024-04-07T16:15:46.667651+0000 mgr.y (mgr.14553) 124 : cluster 0 pgmap v94: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:15:48.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:15:48 smithi100 bash[21346]: cluster 2024-04-07T16:15:46.667651+0000 mgr.y (mgr.14553) 124 : cluster 0 pgmap v94: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:15:48.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:15:48 smithi100 bash[31660]: cluster 2024-04-07T16:15:46.667651+0000 mgr.y (mgr.14553) 124 : cluster 0 pgmap v94: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:15:50.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:15:50 smithi144 bash[23522]: cluster 2024-04-07T16:15:48.668028+0000 mgr.y (mgr.14553) 125 : cluster 0 pgmap v95: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:15:50.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:15:50 smithi100 bash[21346]: cluster 2024-04-07T16:15:48.668028+0000 mgr.y (mgr.14553) 125 : cluster 0 pgmap v95: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:15:50.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:15:50 smithi100 bash[31660]: cluster 2024-04-07T16:15:48.668028+0000 mgr.y (mgr.14553) 125 : cluster 0 pgmap v95: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:15:52.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:15:52 smithi144 bash[23522]: cluster 2024-04-07T16:15:50.668707+0000 mgr.y (mgr.14553) 126 : cluster 0 pgmap v96: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:15:52.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:15:52 smithi100 bash[21346]: cluster 2024-04-07T16:15:50.668707+0000 mgr.y (mgr.14553) 126 : cluster 0 pgmap v96: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:15:52.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:15:52 smithi100 bash[31660]: cluster 2024-04-07T16:15:50.668707+0000 mgr.y (mgr.14553) 126 : cluster 0 pgmap v96: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:15:53.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:15:53 smithi144 bash[23522]: audit 2024-04-07T16:15:52.700104+0000 mon.b (mon.1) 95 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:15:53.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:15:53 smithi100 bash[21346]: audit 2024-04-07T16:15:52.700104+0000 mon.b (mon.1) 95 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:15:53.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:15:53 smithi100 bash[31660]: audit 2024-04-07T16:15:52.700104+0000 mon.b (mon.1) 95 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:15:54.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:15:54 smithi100 bash[21346]: cluster 2024-04-07T16:15:52.669087+0000 mgr.y (mgr.14553) 127 : cluster 0 pgmap v97: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:15:54.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:15:54 smithi100 bash[31660]: cluster 2024-04-07T16:15:52.669087+0000 mgr.y (mgr.14553) 127 : cluster 0 pgmap v97: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:15:54.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:15:54 smithi144 bash[23522]: cluster 2024-04-07T16:15:52.669087+0000 mgr.y (mgr.14553) 127 : cluster 0 pgmap v97: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:15:55.437 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:15:55 smithi100 bash[21601]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:15:55] "GET /metrics HTTP/1.1" 200 33742 "" "Prometheus/2.43.0" 2024-04-07T16:15:56.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:15:56 smithi100 bash[21346]: cluster 2024-04-07T16:15:54.670283+0000 mgr.y (mgr.14553) 128 : cluster 0 pgmap v98: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:15:56.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:15:56 smithi100 bash[31660]: cluster 2024-04-07T16:15:54.670283+0000 mgr.y (mgr.14553) 128 : cluster 0 pgmap v98: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:15:56.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:15:56 smithi144 bash[23522]: cluster 2024-04-07T16:15:54.670283+0000 mgr.y (mgr.14553) 128 : cluster 0 pgmap v98: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:15:58.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:15:58 smithi100 bash[21346]: cluster 2024-04-07T16:15:56.670737+0000 mgr.y (mgr.14553) 129 : cluster 0 pgmap v99: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:15:58.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:15:58 smithi100 bash[31660]: cluster 2024-04-07T16:15:56.670737+0000 mgr.y (mgr.14553) 129 : cluster 0 pgmap v99: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:15:58.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:15:58 smithi144 bash[23522]: cluster 2024-04-07T16:15:56.670737+0000 mgr.y (mgr.14553) 129 : cluster 0 pgmap v99: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:16:00.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:00 smithi100 bash[21346]: cluster 2024-04-07T16:15:58.671113+0000 mgr.y (mgr.14553) 130 : cluster 0 pgmap v100: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:16:00.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:00 smithi100 bash[31660]: cluster 2024-04-07T16:15:58.671113+0000 mgr.y (mgr.14553) 130 : cluster 0 pgmap v100: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:16:00.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:00 smithi144 bash[23522]: cluster 2024-04-07T16:15:58.671113+0000 mgr.y (mgr.14553) 130 : cluster 0 pgmap v100: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:16:02.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:02 smithi100 bash[21346]: cluster 2024-04-07T16:16:00.671824+0000 mgr.y (mgr.14553) 131 : cluster 0 pgmap v101: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:16:02.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:02 smithi100 bash[31660]: cluster 2024-04-07T16:16:00.671824+0000 mgr.y (mgr.14553) 131 : cluster 0 pgmap v101: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:16:02.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:02 smithi144 bash[23522]: cluster 2024-04-07T16:16:00.671824+0000 mgr.y (mgr.14553) 131 : cluster 0 pgmap v101: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:16:04.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:04 smithi100 bash[21346]: cluster 2024-04-07T16:16:02.672139+0000 mgr.y (mgr.14553) 132 : cluster 0 pgmap v102: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:16:04.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:04 smithi100 bash[31660]: cluster 2024-04-07T16:16:02.672139+0000 mgr.y (mgr.14553) 132 : cluster 0 pgmap v102: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:16:04.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:04 smithi144 bash[23522]: cluster 2024-04-07T16:16:02.672139+0000 mgr.y (mgr.14553) 132 : cluster 0 pgmap v102: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:16:05.437 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:16:05 smithi100 bash[21601]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:16:05] "GET /metrics HTTP/1.1" 200 33742 "" "Prometheus/2.43.0" 2024-04-07T16:16:06.378 INFO:tasks.workunit.client.0.smithi100.stderr:Note: switching to 'adce92d056533f22dc1ee37e3141d098b046ec1b'. 2024-04-07T16:16:06.378 INFO:tasks.workunit.client.0.smithi100.stderr: 2024-04-07T16:16:06.378 INFO:tasks.workunit.client.0.smithi100.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-04-07T16:16:06.378 INFO:tasks.workunit.client.0.smithi100.stderr:changes and commit them, and you can discard any commits you make in this 2024-04-07T16:16:06.378 INFO:tasks.workunit.client.0.smithi100.stderr:state without impacting any branches by switching back to a branch. 2024-04-07T16:16:06.378 INFO:tasks.workunit.client.0.smithi100.stderr: 2024-04-07T16:16:06.378 INFO:tasks.workunit.client.0.smithi100.stderr:If you want to create a new branch to retain commits you create, you may 2024-04-07T16:16:06.378 INFO:tasks.workunit.client.0.smithi100.stderr:do so (now or later) by using -c with the switch command. Example: 2024-04-07T16:16:06.378 INFO:tasks.workunit.client.0.smithi100.stderr: 2024-04-07T16:16:06.378 INFO:tasks.workunit.client.0.smithi100.stderr: git switch -c 2024-04-07T16:16:06.379 INFO:tasks.workunit.client.0.smithi100.stderr: 2024-04-07T16:16:06.379 INFO:tasks.workunit.client.0.smithi100.stderr:Or undo this operation with: 2024-04-07T16:16:06.379 INFO:tasks.workunit.client.0.smithi100.stderr: 2024-04-07T16:16:06.379 INFO:tasks.workunit.client.0.smithi100.stderr: git switch - 2024-04-07T16:16:06.379 INFO:tasks.workunit.client.0.smithi100.stderr: 2024-04-07T16:16:06.379 INFO:tasks.workunit.client.0.smithi100.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-04-07T16:16:06.379 INFO:tasks.workunit.client.0.smithi100.stderr: 2024-04-07T16:16:06.379 INFO:tasks.workunit.client.0.smithi100.stderr:HEAD is now at adce92d0565 Merge branch 'wlk-dev' of https://github.com/wanglinke521/ceph into wip-yuri3-testing-2024-04-05-0825 2024-04-07T16:16:06.385 DEBUG:teuthology.orchestra.run.smithi100:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-04-07T16:16:06.432 INFO:tasks.workunit.client.0.smithi100.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-04-07T16:16:06.434 INFO:tasks.workunit.client.0.smithi100.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-07T16:16:06.434 INFO:tasks.workunit.client.0.smithi100.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-04-07T16:16:06.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:06 smithi100 bash[21346]: cluster 2024-04-07T16:16:04.673352+0000 mgr.y (mgr.14553) 133 : cluster 0 pgmap v103: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:16:06.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:06 smithi100 bash[31660]: cluster 2024-04-07T16:16:04.673352+0000 mgr.y (mgr.14553) 133 : cluster 0 pgmap v103: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:16:06.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:06 smithi144 bash[23522]: cluster 2024-04-07T16:16:04.673352+0000 mgr.y (mgr.14553) 133 : cluster 0 pgmap v103: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:16:07.236 INFO:tasks.workunit.client.0.smithi100.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-04-07T16:16:07.309 INFO:tasks.workunit.client.0.smithi100.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-04-07T16:16:07.372 INFO:tasks.workunit.client.0.smithi100.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-07T16:16:07.373 INFO:tasks.workunit.client.0.smithi100.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-07T16:16:07.373 INFO:tasks.workunit.client.0.smithi100.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-04-07T16:16:07.417 INFO:tasks.workunit.client.0.smithi100.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-07T16:16:07.420 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-07T16:16:07.420 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-04-07T16:16:07.475 INFO:tasks.workunit:Running workunits matching test_telemetry_reef.sh on client.0... 2024-04-07T16:16:07.476 INFO:tasks.workunit:Running workunit test_telemetry_reef.sh... 2024-04-07T16:16:07.476 DEBUG:teuthology.orchestra.run.smithi100:workunit test test_telemetry_reef.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=adce92d056533f22dc1ee37e3141d098b046ec1b TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/test_telemetry_reef.sh 2024-04-07T16:16:07.528 INFO:tasks.workunit.client.0.smithi100.stderr:+ ceph config set mgr mgr/telemetry/channel_ident true 2024-04-07T16:16:07.958 INFO:tasks.workunit.client.0.smithi100.stderr:+ ceph config set mgr mgr/telemetry/organization ceph-qa 2024-04-07T16:16:08.133 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:16:07 smithi100 bash[21601]: [07/Apr/2024:16:16:07] ENGINE Bus STOPPING 2024-04-07T16:16:08.133 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:16:07 smithi100 bash[21601]: [07/Apr/2024:16:16:07] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-04-07T16:16:08.133 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:16:07 smithi100 bash[21601]: [07/Apr/2024:16:16:07] ENGINE Bus STOPPED 2024-04-07T16:16:08.133 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:16:07 smithi100 bash[21601]: [07/Apr/2024:16:16:07] ENGINE Bus STARTING 2024-04-07T16:16:08.133 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:16:07 smithi100 bash[21601]: [07/Apr/2024:16:16:07] ENGINE Serving on http://:::9283 2024-04-07T16:16:08.133 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:16:07 smithi100 bash[21601]: [07/Apr/2024:16:16:07] ENGINE Bus STARTED 2024-04-07T16:16:08.367 INFO:tasks.workunit.client.0.smithi100.stderr:+ ceph config set mgr mgr/telemetry/description 'upgrade test cluster' 2024-04-07T16:16:08.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:08 smithi100 bash[31660]: cluster 2024-04-07T16:16:06.674028+0000 mgr.y (mgr.14553) 134 : cluster 0 pgmap v104: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:16:08.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:08 smithi100 bash[31660]: audit 2024-04-07T16:16:07.700515+0000 mon.b (mon.1) 96 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:16:08.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:08 smithi100 bash[31660]: audit 2024-04-07T16:16:07.852089+0000 mon.a (mon.0) 843 : audit 1 from='client.? ' entity='client.admin' 2024-04-07T16:16:08.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:08 smithi100 bash[31660]: audit 2024-04-07T16:16:07.908009+0000 mon.b (mon.1) 97 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:16:08.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:08 smithi100 bash[31660]: audit 2024-04-07T16:16:07.911550+0000 mon.b (mon.1) 98 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:16:08.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:08 smithi100 bash[31660]: audit 2024-04-07T16:16:07.913181+0000 mon.b (mon.1) 99 : audit 1 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:16:08.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:08 smithi100 bash[31660]: audit 2024-04-07T16:16:07.928611+0000 mon.a (mon.0) 844 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:16:08.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:08 smithi100 bash[21346]: cluster 2024-04-07T16:16:06.674028+0000 mgr.y (mgr.14553) 134 : cluster 0 pgmap v104: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:16:08.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:08 smithi100 bash[21346]: audit 2024-04-07T16:16:07.700515+0000 mon.b (mon.1) 96 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:16:08.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:08 smithi100 bash[21346]: audit 2024-04-07T16:16:07.852089+0000 mon.a (mon.0) 843 : audit 1 from='client.? ' entity='client.admin' 2024-04-07T16:16:08.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:08 smithi100 bash[21346]: audit 2024-04-07T16:16:07.908009+0000 mon.b (mon.1) 97 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:16:08.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:08 smithi100 bash[21346]: audit 2024-04-07T16:16:07.911550+0000 mon.b (mon.1) 98 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:16:08.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:08 smithi100 bash[21346]: audit 2024-04-07T16:16:07.913181+0000 mon.b (mon.1) 99 : audit 1 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:16:08.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:08 smithi100 bash[21346]: audit 2024-04-07T16:16:07.928611+0000 mon.a (mon.0) 844 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:16:08.439 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:16:08 smithi100 bash[21601]: [07/Apr/2024:16:16:08] ENGINE Bus STOPPING 2024-04-07T16:16:08.439 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:16:08 smithi100 bash[21601]: [07/Apr/2024:16:16:08] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-04-07T16:16:08.439 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:16:08 smithi100 bash[21601]: [07/Apr/2024:16:16:08] ENGINE Bus STOPPED 2024-04-07T16:16:08.439 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:16:08 smithi100 bash[21601]: [07/Apr/2024:16:16:08] ENGINE Bus STARTING 2024-04-07T16:16:08.439 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:16:08 smithi100 bash[21601]: [07/Apr/2024:16:16:08] ENGINE Serving on http://:::9283 2024-04-07T16:16:08.439 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:16:08 smithi100 bash[21601]: [07/Apr/2024:16:16:08] ENGINE Bus STARTED 2024-04-07T16:16:08.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:08 smithi144 bash[23522]: cluster 2024-04-07T16:16:06.674028+0000 mgr.y (mgr.14553) 134 : cluster 0 pgmap v104: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:16:08.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:08 smithi144 bash[23522]: audit 2024-04-07T16:16:07.700515+0000 mon.b (mon.1) 96 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:16:08.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:08 smithi144 bash[23522]: audit 2024-04-07T16:16:07.852089+0000 mon.a (mon.0) 843 : audit 1 from='client.? ' entity='client.admin' 2024-04-07T16:16:08.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:08 smithi144 bash[23522]: audit 2024-04-07T16:16:07.908009+0000 mon.b (mon.1) 97 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:16:08.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:08 smithi144 bash[23522]: audit 2024-04-07T16:16:07.911550+0000 mon.b (mon.1) 98 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:16:08.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:08 smithi144 bash[23522]: audit 2024-04-07T16:16:07.913181+0000 mon.b (mon.1) 99 : audit 1 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:16:08.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:08 smithi144 bash[23522]: audit 2024-04-07T16:16:07.928611+0000 mon.a (mon.0) 844 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:16:08.776 INFO:tasks.workunit.client.0.smithi100.stderr:+ ceph telemetry preview 2024-04-07T16:16:08.937 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:16:08 smithi100 bash[21601]: [07/Apr/2024:16:16:08] ENGINE Bus STOPPING 2024-04-07T16:16:08.937 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:16:08 smithi100 bash[21601]: [07/Apr/2024:16:16:08] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-04-07T16:16:08.937 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:16:08 smithi100 bash[21601]: [07/Apr/2024:16:16:08] ENGINE Bus STOPPED 2024-04-07T16:16:08.937 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:16:08 smithi100 bash[21601]: [07/Apr/2024:16:16:08] ENGINE Bus STARTING 2024-04-07T16:16:08.937 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:16:08 smithi100 bash[21601]: [07/Apr/2024:16:16:08] ENGINE Serving on http://:::9283 2024-04-07T16:16:08.937 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:16:08 smithi100 bash[21601]: [07/Apr/2024:16:16:08] ENGINE Bus STARTED 2024-04-07T16:16:09.298 INFO:tasks.workunit.client.0.smithi100.stdout:{ 2024-04-07T16:16:09.298 INFO:tasks.workunit.client.0.smithi100.stdout: "balancer": { 2024-04-07T16:16:09.298 INFO:tasks.workunit.client.0.smithi100.stdout: "active": true, 2024-04-07T16:16:09.298 INFO:tasks.workunit.client.0.smithi100.stdout: "mode": "" 2024-04-07T16:16:09.298 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:09.298 INFO:tasks.workunit.client.0.smithi100.stdout: "channels": [ 2024-04-07T16:16:09.298 INFO:tasks.workunit.client.0.smithi100.stdout: "basic", 2024-04-07T16:16:09.298 INFO:tasks.workunit.client.0.smithi100.stdout: "crash", 2024-04-07T16:16:09.298 INFO:tasks.workunit.client.0.smithi100.stdout: "device", 2024-04-07T16:16:09.299 INFO:tasks.workunit.client.0.smithi100.stdout: "ident" 2024-04-07T16:16:09.299 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:09.299 INFO:tasks.workunit.client.0.smithi100.stdout: "channels_available": [ 2024-04-07T16:16:09.299 INFO:tasks.workunit.client.0.smithi100.stdout: "basic", 2024-04-07T16:16:09.299 INFO:tasks.workunit.client.0.smithi100.stdout: "ident", 2024-04-07T16:16:09.299 INFO:tasks.workunit.client.0.smithi100.stdout: "crash", 2024-04-07T16:16:09.299 INFO:tasks.workunit.client.0.smithi100.stdout: "device", 2024-04-07T16:16:09.299 INFO:tasks.workunit.client.0.smithi100.stdout: "perf" 2024-04-07T16:16:09.299 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:09.299 INFO:tasks.workunit.client.0.smithi100.stdout: "collections_available": [ 2024-04-07T16:16:09.299 INFO:tasks.workunit.client.0.smithi100.stdout: "basic_base", 2024-04-07T16:16:09.299 INFO:tasks.workunit.client.0.smithi100.stdout: "device_base", 2024-04-07T16:16:09.299 INFO:tasks.workunit.client.0.smithi100.stdout: "crash_base", 2024-04-07T16:16:09.299 INFO:tasks.workunit.client.0.smithi100.stdout: "ident_base", 2024-04-07T16:16:09.299 INFO:tasks.workunit.client.0.smithi100.stdout: "perf_perf", 2024-04-07T16:16:09.300 INFO:tasks.workunit.client.0.smithi100.stdout: "basic_mds_metadata", 2024-04-07T16:16:09.300 INFO:tasks.workunit.client.0.smithi100.stdout: "basic_pool_usage", 2024-04-07T16:16:09.300 INFO:tasks.workunit.client.0.smithi100.stdout: "basic_usage_by_class", 2024-04-07T16:16:09.300 INFO:tasks.workunit.client.0.smithi100.stdout: "basic_rook_v01", 2024-04-07T16:16:09.300 INFO:tasks.workunit.client.0.smithi100.stdout: "perf_memory_metrics", 2024-04-07T16:16:09.300 INFO:tasks.workunit.client.0.smithi100.stdout: "basic_pool_options_bluestore" 2024-04-07T16:16:09.300 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:09.300 INFO:tasks.workunit.client.0.smithi100.stdout: "collections_opted_in": [ 2024-04-07T16:16:09.300 INFO:tasks.workunit.client.0.smithi100.stdout: "basic_base", 2024-04-07T16:16:09.300 INFO:tasks.workunit.client.0.smithi100.stdout: "device_base", 2024-04-07T16:16:09.300 INFO:tasks.workunit.client.0.smithi100.stdout: "crash_base", 2024-04-07T16:16:09.300 INFO:tasks.workunit.client.0.smithi100.stdout: "ident_base", 2024-04-07T16:16:09.300 INFO:tasks.workunit.client.0.smithi100.stdout: "perf_perf", 2024-04-07T16:16:09.300 INFO:tasks.workunit.client.0.smithi100.stdout: "basic_mds_metadata", 2024-04-07T16:16:09.301 INFO:tasks.workunit.client.0.smithi100.stdout: "basic_pool_usage", 2024-04-07T16:16:09.301 INFO:tasks.workunit.client.0.smithi100.stdout: "basic_usage_by_class", 2024-04-07T16:16:09.301 INFO:tasks.workunit.client.0.smithi100.stdout: "basic_rook_v01", 2024-04-07T16:16:09.301 INFO:tasks.workunit.client.0.smithi100.stdout: "perf_memory_metrics", 2024-04-07T16:16:09.301 INFO:tasks.workunit.client.0.smithi100.stdout: "basic_pool_options_bluestore" 2024-04-07T16:16:09.301 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:09.301 INFO:tasks.workunit.client.0.smithi100.stdout: "config": { 2024-04-07T16:16:09.301 INFO:tasks.workunit.client.0.smithi100.stdout: "active_changed": [ 2024-04-07T16:16:09.301 INFO:tasks.workunit.client.0.smithi100.stdout: "auth_allow_insecure_global_id_reclaim", 2024-04-07T16:16:09.301 INFO:tasks.workunit.client.0.smithi100.stdout: "auth_debug", 2024-04-07T16:16:09.301 INFO:tasks.workunit.client.0.smithi100.stdout: "auth_mon_ticket_ttl", 2024-04-07T16:16:09.301 INFO:tasks.workunit.client.0.smithi100.stdout: "auth_service_ticket_ttl", 2024-04-07T16:16:09.301 INFO:tasks.workunit.client.0.smithi100.stdout: "bdev_debug_aio", 2024-04-07T16:16:09.301 INFO:tasks.workunit.client.0.smithi100.stdout: "client_check_pool_perm", 2024-04-07T16:16:09.302 INFO:tasks.workunit.client.0.smithi100.stdout: "client_mount_gid", 2024-04-07T16:16:09.302 INFO:tasks.workunit.client.0.smithi100.stdout: "client_mount_uid", 2024-04-07T16:16:09.302 INFO:tasks.workunit.client.0.smithi100.stdout: "client_quota", 2024-04-07T16:16:09.302 INFO:tasks.workunit.client.0.smithi100.stdout: "container_image", 2024-04-07T16:16:09.302 INFO:tasks.workunit.client.0.smithi100.stdout: "daemonize", 2024-04-07T16:16:09.302 INFO:tasks.workunit.client.0.smithi100.stdout: "debug_asserts_on_shutdown", 2024-04-07T16:16:09.302 INFO:tasks.workunit.client.0.smithi100.stdout: "debug_mgr", 2024-04-07T16:16:09.302 INFO:tasks.workunit.client.0.smithi100.stdout: "debug_mon", 2024-04-07T16:16:09.302 INFO:tasks.workunit.client.0.smithi100.stdout: "debug_ms", 2024-04-07T16:16:09.302 INFO:tasks.workunit.client.0.smithi100.stdout: "debug_osd", 2024-04-07T16:16:09.302 INFO:tasks.workunit.client.0.smithi100.stdout: "debug_paxos", 2024-04-07T16:16:09.302 INFO:tasks.workunit.client.0.smithi100.stdout: "keyring", 2024-04-07T16:16:09.302 INFO:tasks.workunit.client.0.smithi100.stdout: "leveldb_block_size", 2024-04-07T16:16:09.302 INFO:tasks.workunit.client.0.smithi100.stdout: "leveldb_cache_size", 2024-04-07T16:16:09.303 INFO:tasks.workunit.client.0.smithi100.stdout: "leveldb_compression", 2024-04-07T16:16:09.303 INFO:tasks.workunit.client.0.smithi100.stdout: "leveldb_log", 2024-04-07T16:16:09.303 INFO:tasks.workunit.client.0.smithi100.stdout: "leveldb_write_buffer_size", 2024-04-07T16:16:09.303 INFO:tasks.workunit.client.0.smithi100.stdout: "log_stderr_prefix", 2024-04-07T16:16:09.303 INFO:tasks.workunit.client.0.smithi100.stdout: "log_to_file", 2024-04-07T16:16:09.303 INFO:tasks.workunit.client.0.smithi100.stdout: "log_to_journald", 2024-04-07T16:16:09.303 INFO:tasks.workunit.client.0.smithi100.stdout: "log_to_stderr", 2024-04-07T16:16:09.303 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_allow_pool_delete", 2024-04-07T16:16:09.303 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_clock_drift_allowed", 2024-04-07T16:16:09.303 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_cluster_log_file_level", 2024-04-07T16:16:09.303 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_cluster_log_to_file", 2024-04-07T16:16:09.303 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_cluster_log_to_stderr", 2024-04-07T16:16:09.303 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_data_avail_warn", 2024-04-07T16:16:09.303 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_election_default_strategy", 2024-04-07T16:16:09.304 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_host", 2024-04-07T16:16:09.304 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_max_pg_per_osd", 2024-04-07T16:16:09.304 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_mgr_mkfs_grace", 2024-04-07T16:16:09.304 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_osd_prime_pg_temp", 2024-04-07T16:16:09.304 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_osd_reporter_subtree_level", 2024-04-07T16:16:09.304 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_pg_warn_max_object_skew", 2024-04-07T16:16:09.304 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_reweight_min_bytes_per_osd", 2024-04-07T16:16:09.304 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_reweight_min_pgs_per_osd", 2024-04-07T16:16:09.304 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_warn_on_crush_straw_calc_version_zero", 2024-04-07T16:16:09.304 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_warn_on_insecure_global_id_reclaim", 2024-04-07T16:16:09.304 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_warn_on_insecure_global_id_reclaim_allowed", 2024-04-07T16:16:09.304 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_warn_on_legacy_crush_tunables", 2024-04-07T16:16:09.304 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_warn_on_osd_down_out_interval_zero", 2024-04-07T16:16:09.304 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_warn_on_pool_pg_num_not_power_of_two", 2024-04-07T16:16:09.304 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_warn_on_too_few_osds", 2024-04-07T16:16:09.305 INFO:tasks.workunit.client.0.smithi100.stdout: "ms_die_on_bug", 2024-04-07T16:16:09.305 INFO:tasks.workunit.client.0.smithi100.stdout: "ms_die_on_old_message", 2024-04-07T16:16:09.305 INFO:tasks.workunit.client.0.smithi100.stdout: "no_config_file", 2024-04-07T16:16:09.305 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_class_default_list", 2024-04-07T16:16:09.305 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_class_load_list", 2024-04-07T16:16:09.305 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_crush_chooseleaf_type", 2024-04-07T16:16:09.305 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_debug_misdirected_ops", 2024-04-07T16:16:09.305 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_debug_op_order", 2024-04-07T16:16:09.305 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_debug_pg_log_writeout", 2024-04-07T16:16:09.305 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_debug_shutdown", 2024-04-07T16:16:09.305 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_debug_verify_cached_snaps", 2024-04-07T16:16:09.305 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_debug_verify_missing_on_start", 2024-04-07T16:16:09.306 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_debug_verify_stray_on_activate", 2024-04-07T16:16:09.306 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_deep_scrub_update_digest_min_age", 2024-04-07T16:16:09.306 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_delete_sleep", 2024-04-07T16:16:09.306 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_delete_sleep_hdd", 2024-04-07T16:16:09.306 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_delete_sleep_hybrid", 2024-04-07T16:16:09.306 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_delete_sleep_ssd", 2024-04-07T16:16:09.306 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_max_backfills", 2024-04-07T16:16:09.306 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mclock_max_capacity_iops_ssd", 2024-04-07T16:16:09.306 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mclock_profile", 2024-04-07T16:16:09.306 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mclock_scheduler_background_best_effort_lim", 2024-04-07T16:16:09.306 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mclock_scheduler_background_best_effort_res", 2024-04-07T16:16:09.306 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mclock_scheduler_background_best_effort_wgt", 2024-04-07T16:16:09.306 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mclock_scheduler_background_recovery_lim", 2024-04-07T16:16:09.307 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mclock_scheduler_background_recovery_res", 2024-04-07T16:16:09.307 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mclock_scheduler_background_recovery_wgt", 2024-04-07T16:16:09.307 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mclock_scheduler_client_lim", 2024-04-07T16:16:09.307 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mclock_scheduler_client_res", 2024-04-07T16:16:09.307 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mclock_scheduler_client_wgt", 2024-04-07T16:16:09.307 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_memory_target", 2024-04-07T16:16:09.307 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_memory_target_autotune", 2024-04-07T16:16:09.307 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_op_queue", 2024-04-07T16:16:09.307 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_op_queue_cut_off", 2024-04-07T16:16:09.307 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_pg_delete_cost", 2024-04-07T16:16:09.307 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_pool_default_erasure_code_profile", 2024-04-07T16:16:09.307 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_pool_default_pg_autoscale_mode", 2024-04-07T16:16:09.307 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_recover_clone_overlap", 2024-04-07T16:16:09.307 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_recovery_max_active", 2024-04-07T16:16:09.307 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_recovery_max_active_hdd", 2024-04-07T16:16:09.308 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_recovery_max_active_ssd", 2024-04-07T16:16:09.308 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_recovery_max_chunk", 2024-04-07T16:16:09.308 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_recovery_sleep", 2024-04-07T16:16:09.308 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_recovery_sleep_hdd", 2024-04-07T16:16:09.308 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_recovery_sleep_hybrid", 2024-04-07T16:16:09.308 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_recovery_sleep_ssd", 2024-04-07T16:16:09.308 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_scrub_load_threshold", 2024-04-07T16:16:09.308 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_scrub_max_interval", 2024-04-07T16:16:09.308 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_scrub_sleep", 2024-04-07T16:16:09.308 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_shutdown_pgref_assert", 2024-04-07T16:16:09.308 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_snap_trim_sleep", 2024-04-07T16:16:09.308 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_snap_trim_sleep_hdd", 2024-04-07T16:16:09.308 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_snap_trim_sleep_hybrid", 2024-04-07T16:16:09.308 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_snap_trim_sleep_ssd", 2024-04-07T16:16:09.309 INFO:tasks.workunit.client.0.smithi100.stdout: "public_addr", 2024-04-07T16:16:09.309 INFO:tasks.workunit.client.0.smithi100.stdout: "public_addrv", 2024-04-07T16:16:09.309 INFO:tasks.workunit.client.0.smithi100.stdout: "public_network", 2024-04-07T16:16:09.309 INFO:tasks.workunit.client.0.smithi100.stdout: "rbd_default_features", 2024-04-07T16:16:09.309 INFO:tasks.workunit.client.0.smithi100.stdout: "rbd_qos_exclude_ops", 2024-04-07T16:16:09.309 INFO:tasks.workunit.client.0.smithi100.stdout: "setgroup", 2024-04-07T16:16:09.309 INFO:tasks.workunit.client.0.smithi100.stdout: "setuser" 2024-04-07T16:16:09.309 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:09.309 INFO:tasks.workunit.client.0.smithi100.stdout: "cluster_changed": [ 2024-04-07T16:16:09.309 INFO:tasks.workunit.client.0.smithi100.stdout: "auth_allow_insecure_global_id_reclaim", 2024-04-07T16:16:09.309 INFO:tasks.workunit.client.0.smithi100.stdout: "auth_debug", 2024-04-07T16:16:09.309 INFO:tasks.workunit.client.0.smithi100.stdout: "auth_mon_ticket_ttl", 2024-04-07T16:16:09.309 INFO:tasks.workunit.client.0.smithi100.stdout: "auth_service_ticket_ttl", 2024-04-07T16:16:09.309 INFO:tasks.workunit.client.0.smithi100.stdout: "bdev_debug_aio", 2024-04-07T16:16:09.310 INFO:tasks.workunit.client.0.smithi100.stdout: "container_image", 2024-04-07T16:16:09.310 INFO:tasks.workunit.client.0.smithi100.stdout: "debug_asserts_on_shutdown", 2024-04-07T16:16:09.310 INFO:tasks.workunit.client.0.smithi100.stdout: "debug_mgr", 2024-04-07T16:16:09.310 INFO:tasks.workunit.client.0.smithi100.stdout: "debug_mon", 2024-04-07T16:16:09.310 INFO:tasks.workunit.client.0.smithi100.stdout: "debug_ms", 2024-04-07T16:16:09.310 INFO:tasks.workunit.client.0.smithi100.stdout: "debug_osd", 2024-04-07T16:16:09.310 INFO:tasks.workunit.client.0.smithi100.stdout: "debug_paxos", 2024-04-07T16:16:09.310 INFO:tasks.workunit.client.0.smithi100.stdout: "log_to_file", 2024-04-07T16:16:09.310 INFO:tasks.workunit.client.0.smithi100.stdout: "log_to_journald", 2024-04-07T16:16:09.310 INFO:tasks.workunit.client.0.smithi100.stdout: "log_to_stderr", 2024-04-07T16:16:09.310 INFO:tasks.workunit.client.0.smithi100.stdout: "mds_join_fs", 2024-04-07T16:16:09.310 INFO:tasks.workunit.client.0.smithi100.stdout: "mgr/cephadm/allow_ptrace", 2024-04-07T16:16:09.310 INFO:tasks.workunit.client.0.smithi100.stdout: "mgr/cephadm/container_init", 2024-04-07T16:16:09.310 INFO:tasks.workunit.client.0.smithi100.stdout: "mgr/cephadm/migration_current", 2024-04-07T16:16:09.311 INFO:tasks.workunit.client.0.smithi100.stdout: "mgr/cephadm/use_repo_digest", 2024-04-07T16:16:09.311 INFO:tasks.workunit.client.0.smithi100.stdout: "mgr/dashboard/ALERTMANAGER_API_HOST", 2024-04-07T16:16:09.311 INFO:tasks.workunit.client.0.smithi100.stdout: "mgr/dashboard/GRAFANA_API_SSL_VERIFY", 2024-04-07T16:16:09.311 INFO:tasks.workunit.client.0.smithi100.stdout: "mgr/dashboard/GRAFANA_API_URL", 2024-04-07T16:16:09.311 INFO:tasks.workunit.client.0.smithi100.stdout: "mgr/dashboard/PROMETHEUS_API_HOST", 2024-04-07T16:16:09.311 INFO:tasks.workunit.client.0.smithi100.stdout: "mgr/dashboard/ssl_server_port", 2024-04-07T16:16:09.311 INFO:tasks.workunit.client.0.smithi100.stdout: "mgr/orchestrator/orchestrator", 2024-04-07T16:16:09.311 INFO:tasks.workunit.client.0.smithi100.stdout: "mgr/telemetry/channel_ident", 2024-04-07T16:16:09.311 INFO:tasks.workunit.client.0.smithi100.stdout: "mgr/telemetry/description", 2024-04-07T16:16:09.311 INFO:tasks.workunit.client.0.smithi100.stdout: "mgr/telemetry/organization", 2024-04-07T16:16:09.311 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_allow_pool_delete", 2024-04-07T16:16:09.311 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_clock_drift_allowed", 2024-04-07T16:16:09.311 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_cluster_log_file_level", 2024-04-07T16:16:09.311 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_cluster_log_to_file", 2024-04-07T16:16:09.311 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_data_avail_warn", 2024-04-07T16:16:09.312 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_election_default_strategy", 2024-04-07T16:16:09.312 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_max_pg_per_osd", 2024-04-07T16:16:09.312 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_mgr_mkfs_grace", 2024-04-07T16:16:09.312 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_osd_prime_pg_temp", 2024-04-07T16:16:09.312 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_osd_reporter_subtree_level", 2024-04-07T16:16:09.312 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_pg_warn_max_object_skew", 2024-04-07T16:16:09.312 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_reweight_min_bytes_per_osd", 2024-04-07T16:16:09.312 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_reweight_min_pgs_per_osd", 2024-04-07T16:16:09.312 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_warn_on_crush_straw_calc_version_zero", 2024-04-07T16:16:09.312 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_warn_on_insecure_global_id_reclaim", 2024-04-07T16:16:09.312 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_warn_on_insecure_global_id_reclaim_allowed", 2024-04-07T16:16:09.312 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_warn_on_legacy_crush_tunables", 2024-04-07T16:16:09.312 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_warn_on_osd_down_out_interval_zero", 2024-04-07T16:16:09.312 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_warn_on_pool_pg_num_not_power_of_two", 2024-04-07T16:16:09.312 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_warn_on_too_few_osds", 2024-04-07T16:16:09.312 INFO:tasks.workunit.client.0.smithi100.stdout: "ms_die_on_bug", 2024-04-07T16:16:09.313 INFO:tasks.workunit.client.0.smithi100.stdout: "ms_die_on_old_message", 2024-04-07T16:16:09.313 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_class_default_list", 2024-04-07T16:16:09.313 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_class_load_list", 2024-04-07T16:16:09.313 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_debug_misdirected_ops", 2024-04-07T16:16:09.313 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_debug_op_order", 2024-04-07T16:16:09.313 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_debug_pg_log_writeout", 2024-04-07T16:16:09.313 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_debug_shutdown", 2024-04-07T16:16:09.313 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_debug_verify_cached_snaps", 2024-04-07T16:16:09.313 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_debug_verify_missing_on_start", 2024-04-07T16:16:09.313 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_debug_verify_stray_on_activate", 2024-04-07T16:16:09.313 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_deep_scrub_update_digest_min_age", 2024-04-07T16:16:09.313 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mclock_max_capacity_iops_ssd", 2024-04-07T16:16:09.313 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mclock_profile", 2024-04-07T16:16:09.313 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_memory_target", 2024-04-07T16:16:09.313 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_memory_target_autotune", 2024-04-07T16:16:09.314 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_op_queue", 2024-04-07T16:16:09.314 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_op_queue_cut_off", 2024-04-07T16:16:09.314 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_pool_default_erasure_code_profile", 2024-04-07T16:16:09.314 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_pool_default_pg_autoscale_mode", 2024-04-07T16:16:09.314 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_recover_clone_overlap", 2024-04-07T16:16:09.314 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_recovery_max_chunk", 2024-04-07T16:16:09.314 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_scrub_load_threshold", 2024-04-07T16:16:09.314 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_scrub_max_interval", 2024-04-07T16:16:09.314 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_shutdown_pgref_assert", 2024-04-07T16:16:09.314 INFO:tasks.workunit.client.0.smithi100.stdout: "public_network", 2024-04-07T16:16:09.314 INFO:tasks.workunit.client.0.smithi100.stdout: "rgw_cache_enabled", 2024-04-07T16:16:09.314 INFO:tasks.workunit.client.0.smithi100.stdout: "rgw_enable_ops_log", 2024-04-07T16:16:09.314 INFO:tasks.workunit.client.0.smithi100.stdout: "rgw_enable_usage_log" 2024-04-07T16:16:09.314 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T16:16:09.314 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:09.315 INFO:tasks.workunit.client.0.smithi100.stdout: "contact": null, 2024-04-07T16:16:09.315 INFO:tasks.workunit.client.0.smithi100.stdout: "crashes": [], 2024-04-07T16:16:09.315 INFO:tasks.workunit.client.0.smithi100.stdout: "created": "2024-04-07T16:02:17.280630Z", 2024-04-07T16:16:09.315 INFO:tasks.workunit.client.0.smithi100.stdout: "crush": { 2024-04-07T16:16:09.315 INFO:tasks.workunit.client.0.smithi100.stdout: "bucket_algs": { 2024-04-07T16:16:09.315 INFO:tasks.workunit.client.0.smithi100.stdout: "straw2": 3 2024-04-07T16:16:09.315 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:09.315 INFO:tasks.workunit.client.0.smithi100.stdout: "bucket_sizes": { 2024-04-07T16:16:09.315 INFO:tasks.workunit.client.0.smithi100.stdout: "2": 1, 2024-04-07T16:16:09.315 INFO:tasks.workunit.client.0.smithi100.stdout: "4": 2 2024-04-07T16:16:09.315 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:09.315 INFO:tasks.workunit.client.0.smithi100.stdout: "bucket_types": { 2024-04-07T16:16:09.315 INFO:tasks.workunit.client.0.smithi100.stdout: "1": 2, 2024-04-07T16:16:09.316 INFO:tasks.workunit.client.0.smithi100.stdout: "11": 1 2024-04-07T16:16:09.316 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:09.316 INFO:tasks.workunit.client.0.smithi100.stdout: "compat_weight_set": false, 2024-04-07T16:16:09.316 INFO:tasks.workunit.client.0.smithi100.stdout: "device_classes": [ 2024-04-07T16:16:09.316 INFO:tasks.workunit.client.0.smithi100.stdout: 8 2024-04-07T16:16:09.316 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:09.316 INFO:tasks.workunit.client.0.smithi100.stdout: "num_buckets": 6, 2024-04-07T16:16:09.316 INFO:tasks.workunit.client.0.smithi100.stdout: "num_devices": 8, 2024-04-07T16:16:09.316 INFO:tasks.workunit.client.0.smithi100.stdout: "num_rules": 1, 2024-04-07T16:16:09.316 INFO:tasks.workunit.client.0.smithi100.stdout: "num_types": 12, 2024-04-07T16:16:09.316 INFO:tasks.workunit.client.0.smithi100.stdout: "num_weight_sets": 0, 2024-04-07T16:16:09.317 INFO:tasks.workunit.client.0.smithi100.stdout: "tunables": { 2024-04-07T16:16:09.317 INFO:tasks.workunit.client.0.smithi100.stdout: "allowed_bucket_algs": 54, 2024-04-07T16:16:09.317 INFO:tasks.workunit.client.0.smithi100.stdout: "choose_local_fallback_tries": 0, 2024-04-07T16:16:09.317 INFO:tasks.workunit.client.0.smithi100.stdout: "choose_local_tries": 0, 2024-04-07T16:16:09.317 INFO:tasks.workunit.client.0.smithi100.stdout: "choose_total_tries": 50, 2024-04-07T16:16:09.317 INFO:tasks.workunit.client.0.smithi100.stdout: "chooseleaf_descend_once": 1, 2024-04-07T16:16:09.317 INFO:tasks.workunit.client.0.smithi100.stdout: "chooseleaf_stable": 1, 2024-04-07T16:16:09.317 INFO:tasks.workunit.client.0.smithi100.stdout: "chooseleaf_vary_r": 1, 2024-04-07T16:16:09.317 INFO:tasks.workunit.client.0.smithi100.stdout: "has_v2_rules": 0, 2024-04-07T16:16:09.317 INFO:tasks.workunit.client.0.smithi100.stdout: "has_v3_rules": 0, 2024-04-07T16:16:09.317 INFO:tasks.workunit.client.0.smithi100.stdout: "has_v4_buckets": 1, 2024-04-07T16:16:09.317 INFO:tasks.workunit.client.0.smithi100.stdout: "has_v5_rules": 0, 2024-04-07T16:16:09.317 INFO:tasks.workunit.client.0.smithi100.stdout: "legacy_tunables": 0, 2024-04-07T16:16:09.317 INFO:tasks.workunit.client.0.smithi100.stdout: "minimum_required_version": "jewel", 2024-04-07T16:16:09.317 INFO:tasks.workunit.client.0.smithi100.stdout: "optimal_tunables": 1, 2024-04-07T16:16:09.318 INFO:tasks.workunit.client.0.smithi100.stdout: "profile": "jewel", 2024-04-07T16:16:09.318 INFO:tasks.workunit.client.0.smithi100.stdout: "require_feature_tunables": 1, 2024-04-07T16:16:09.318 INFO:tasks.workunit.client.0.smithi100.stdout: "require_feature_tunables2": 1, 2024-04-07T16:16:09.318 INFO:tasks.workunit.client.0.smithi100.stdout: "require_feature_tunables3": 1, 2024-04-07T16:16:09.318 INFO:tasks.workunit.client.0.smithi100.stdout: "require_feature_tunables5": 1, 2024-04-07T16:16:09.318 INFO:tasks.workunit.client.0.smithi100.stdout: "straw_calc_version": 1 2024-04-07T16:16:09.318 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:09.318 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:09.318 INFO:tasks.workunit.client.0.smithi100.stdout: "description": "upgrade test cluster", 2024-04-07T16:16:09.318 INFO:tasks.workunit.client.0.smithi100.stdout: "fs": { 2024-04-07T16:16:09.318 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1, 2024-04-07T16:16:09.319 INFO:tasks.workunit.client.0.smithi100.stdout: "feature_flags": { 2024-04-07T16:16:09.319 INFO:tasks.workunit.client.0.smithi100.stdout: "enable_multiple": true, 2024-04-07T16:16:09.319 INFO:tasks.workunit.client.0.smithi100.stdout: "ever_enabled_multiple": true 2024-04-07T16:16:09.319 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:09.319 INFO:tasks.workunit.client.0.smithi100.stdout: "filesystems": [ 2024-04-07T16:16:09.319 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:09.319 INFO:tasks.workunit.client.0.smithi100.stdout: "approx_ctime": "2024-04", 2024-04-07T16:16:09.319 INFO:tasks.workunit.client.0.smithi100.stdout: "balancer_enabled": false, 2024-04-07T16:16:09.319 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:09.319 INFO:tasks.workunit.client.0.smithi100.stdout: "cached_caps": 0, 2024-04-07T16:16:09.319 INFO:tasks.workunit.client.0.smithi100.stdout: "cached_dns": 10, 2024-04-07T16:16:09.319 INFO:tasks.workunit.client.0.smithi100.stdout: "cached_inos": 13, 2024-04-07T16:16:09.319 INFO:tasks.workunit.client.0.smithi100.stdout: "cached_subtrees": 2, 2024-04-07T16:16:09.319 INFO:tasks.workunit.client.0.smithi100.stdout: "ever_allowed_features": 0, 2024-04-07T16:16:09.320 INFO:tasks.workunit.client.0.smithi100.stdout: "explicitly_allowed_features": 0, 2024-04-07T16:16:09.320 INFO:tasks.workunit.client.0.smithi100.stdout: "files": 0, 2024-04-07T16:16:09.320 INFO:tasks.workunit.client.0.smithi100.stdout: "max_mds": 1, 2024-04-07T16:16:09.320 INFO:tasks.workunit.client.0.smithi100.stdout: "num_data_pools": 1, 2024-04-07T16:16:09.320 INFO:tasks.workunit.client.0.smithi100.stdout: "num_in": 1, 2024-04-07T16:16:09.320 INFO:tasks.workunit.client.0.smithi100.stdout: "num_mds": 1, 2024-04-07T16:16:09.320 INFO:tasks.workunit.client.0.smithi100.stdout: "num_sessions": 0, 2024-04-07T16:16:09.320 INFO:tasks.workunit.client.0.smithi100.stdout: "num_standby_replay": 0, 2024-04-07T16:16:09.320 INFO:tasks.workunit.client.0.smithi100.stdout: "num_up": 1, 2024-04-07T16:16:09.320 INFO:tasks.workunit.client.0.smithi100.stdout: "snaps": 0, 2024-04-07T16:16:09.320 INFO:tasks.workunit.client.0.smithi100.stdout: "standby_count_wanted": 0 2024-04-07T16:16:09.320 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:09.321 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:09.321 INFO:tasks.workunit.client.0.smithi100.stdout: "num_standby_mds": 1, 2024-04-07T16:16:09.321 INFO:tasks.workunit.client.0.smithi100.stdout: "total_num_mds": 2 2024-04-07T16:16:09.321 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:09.321 INFO:tasks.workunit.client.0.smithi100.stdout: "hosts": { 2024-04-07T16:16:09.321 INFO:tasks.workunit.client.0.smithi100.stdout: "num": 2, 2024-04-07T16:16:09.321 INFO:tasks.workunit.client.0.smithi100.stdout: "num_with_mds": 2, 2024-04-07T16:16:09.321 INFO:tasks.workunit.client.0.smithi100.stdout: "num_with_mgr": 2, 2024-04-07T16:16:09.321 INFO:tasks.workunit.client.0.smithi100.stdout: "num_with_mon": 2, 2024-04-07T16:16:09.321 INFO:tasks.workunit.client.0.smithi100.stdout: "num_with_osd": 2 2024-04-07T16:16:09.321 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:09.321 INFO:tasks.workunit.client.0.smithi100.stdout: "leaderboard": false, 2024-04-07T16:16:09.321 INFO:tasks.workunit.client.0.smithi100.stdout: "leaderboard_description": null, 2024-04-07T16:16:09.321 INFO:tasks.workunit.client.0.smithi100.stdout: "license": "sharing-1-0", 2024-04-07T16:16:09.321 INFO:tasks.workunit.client.0.smithi100.stdout: "metadata": { 2024-04-07T16:16:09.322 INFO:tasks.workunit.client.0.smithi100.stdout: "mds": { 2024-04-07T16:16:09.322 INFO:tasks.workunit.client.0.smithi100.stdout: "arch": { 2024-04-07T16:16:09.322 INFO:tasks.workunit.client.0.smithi100.stdout: "x86_64": 2 2024-04-07T16:16:09.322 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:09.322 INFO:tasks.workunit.client.0.smithi100.stdout: "ceph_version": { 2024-04-07T16:16:09.322 INFO:tasks.workunit.client.0.smithi100.stdout: "ceph version 18.2.2-1171-g267c8ff5 (267c8ff5b1ff5815581b3b2381d19360e081b37b) reef (stable)": 2 2024-04-07T16:16:09.322 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:09.322 INFO:tasks.workunit.client.0.smithi100.stdout: "cpu": { 2024-04-07T16:16:09.322 INFO:tasks.workunit.client.0.smithi100.stdout: "Intel(R) Xeon(R) CPU E5-1620 v4 @ 3.50GHz": 2 2024-04-07T16:16:09.322 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:09.323 INFO:tasks.workunit.client.0.smithi100.stdout: "distro": { 2024-04-07T16:16:09.323 INFO:tasks.workunit.client.0.smithi100.stdout: "centos": 2 2024-04-07T16:16:09.323 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:09.323 INFO:tasks.workunit.client.0.smithi100.stdout: "distro_description": { 2024-04-07T16:16:09.323 INFO:tasks.workunit.client.0.smithi100.stdout: "CentOS Stream 8": 2 2024-04-07T16:16:09.323 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:09.323 INFO:tasks.workunit.client.0.smithi100.stdout: "kernel_description": { 2024-04-07T16:16:09.323 INFO:tasks.workunit.client.0.smithi100.stdout: "#111-Ubuntu SMP Tue Mar 5 20:16:58 UTC 2024": 2 2024-04-07T16:16:09.323 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:09.323 INFO:tasks.workunit.client.0.smithi100.stdout: "kernel_version": { 2024-04-07T16:16:09.323 INFO:tasks.workunit.client.0.smithi100.stdout: "5.15.0-101-generic": 2 2024-04-07T16:16:09.323 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:09.324 INFO:tasks.workunit.client.0.smithi100.stdout: "os": { 2024-04-07T16:16:09.324 INFO:tasks.workunit.client.0.smithi100.stdout: "Linux": 2 2024-04-07T16:16:09.324 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:09.324 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:09.324 INFO:tasks.workunit.client.0.smithi100.stdout: "mon": { 2024-04-07T16:16:09.324 INFO:tasks.workunit.client.0.smithi100.stdout: "arch": { 2024-04-07T16:16:09.324 INFO:tasks.workunit.client.0.smithi100.stdout: "x86_64": 3 2024-04-07T16:16:09.324 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:09.324 INFO:tasks.workunit.client.0.smithi100.stdout: "ceph_version": { 2024-04-07T16:16:09.324 INFO:tasks.workunit.client.0.smithi100.stdout: "ceph version 18.2.2-1171-g267c8ff5 (267c8ff5b1ff5815581b3b2381d19360e081b37b) reef (stable)": 3 2024-04-07T16:16:09.324 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:09.324 INFO:tasks.workunit.client.0.smithi100.stdout: "cpu": { 2024-04-07T16:16:09.324 INFO:tasks.workunit.client.0.smithi100.stdout: "Intel(R) Xeon(R) CPU E5-1620 v4 @ 3.50GHz": 3 2024-04-07T16:16:09.325 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:09.325 INFO:tasks.workunit.client.0.smithi100.stdout: "distro": { 2024-04-07T16:16:09.325 INFO:tasks.workunit.client.0.smithi100.stdout: "centos": 3 2024-04-07T16:16:09.325 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:09.325 INFO:tasks.workunit.client.0.smithi100.stdout: "distro_description": { 2024-04-07T16:16:09.325 INFO:tasks.workunit.client.0.smithi100.stdout: "CentOS Stream 8": 3 2024-04-07T16:16:09.325 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:09.325 INFO:tasks.workunit.client.0.smithi100.stdout: "kernel_description": { 2024-04-07T16:16:09.325 INFO:tasks.workunit.client.0.smithi100.stdout: "#111-Ubuntu SMP Tue Mar 5 20:16:58 UTC 2024": 3 2024-04-07T16:16:09.325 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:09.325 INFO:tasks.workunit.client.0.smithi100.stdout: "kernel_version": { 2024-04-07T16:16:09.326 INFO:tasks.workunit.client.0.smithi100.stdout: "5.15.0-101-generic": 3 2024-04-07T16:16:09.326 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:09.326 INFO:tasks.workunit.client.0.smithi100.stdout: "os": { 2024-04-07T16:16:09.326 INFO:tasks.workunit.client.0.smithi100.stdout: "Linux": 3 2024-04-07T16:16:09.326 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:09.326 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:09.326 INFO:tasks.workunit.client.0.smithi100.stdout: "osd": { 2024-04-07T16:16:09.326 INFO:tasks.workunit.client.0.smithi100.stdout: "arch": { 2024-04-07T16:16:09.326 INFO:tasks.workunit.client.0.smithi100.stdout: "x86_64": 8 2024-04-07T16:16:09.326 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:09.326 INFO:tasks.workunit.client.0.smithi100.stdout: "ceph_version": { 2024-04-07T16:16:09.327 INFO:tasks.workunit.client.0.smithi100.stdout: "ceph version 18.2.2-1171-g267c8ff5 (267c8ff5b1ff5815581b3b2381d19360e081b37b) reef (stable)": 8 2024-04-07T16:16:09.327 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:09.327 INFO:tasks.workunit.client.0.smithi100.stdout: "cpu": { 2024-04-07T16:16:09.327 INFO:tasks.workunit.client.0.smithi100.stdout: "Intel(R) Xeon(R) CPU E5-1620 v4 @ 3.50GHz": 8 2024-04-07T16:16:09.327 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:09.327 INFO:tasks.workunit.client.0.smithi100.stdout: "distro": { 2024-04-07T16:16:09.327 INFO:tasks.workunit.client.0.smithi100.stdout: "centos": 8 2024-04-07T16:16:09.327 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:09.327 INFO:tasks.workunit.client.0.smithi100.stdout: "distro_description": { 2024-04-07T16:16:09.327 INFO:tasks.workunit.client.0.smithi100.stdout: "CentOS Stream 8": 8 2024-04-07T16:16:09.327 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:09.328 INFO:tasks.workunit.client.0.smithi100.stdout: "kernel_description": { 2024-04-07T16:16:09.328 INFO:tasks.workunit.client.0.smithi100.stdout: "#111-Ubuntu SMP Tue Mar 5 20:16:58 UTC 2024": 8 2024-04-07T16:16:09.328 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:09.328 INFO:tasks.workunit.client.0.smithi100.stdout: "kernel_version": { 2024-04-07T16:16:09.328 INFO:tasks.workunit.client.0.smithi100.stdout: "5.15.0-101-generic": 8 2024-04-07T16:16:09.328 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:09.328 INFO:tasks.workunit.client.0.smithi100.stdout: "os": { 2024-04-07T16:16:09.328 INFO:tasks.workunit.client.0.smithi100.stdout: "Linux": 8 2024-04-07T16:16:09.328 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:09.328 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_objectstore": { 2024-04-07T16:16:09.328 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore": 8 2024-04-07T16:16:09.329 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:09.329 INFO:tasks.workunit.client.0.smithi100.stdout: "rotational": { 2024-04-07T16:16:09.329 INFO:tasks.workunit.client.0.smithi100.stdout: "0": 8 2024-04-07T16:16:09.329 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:09.329 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:09.329 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:09.329 INFO:tasks.workunit.client.0.smithi100.stdout: "mon": { 2024-04-07T16:16:09.329 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 3, 2024-04-07T16:16:09.329 INFO:tasks.workunit.client.0.smithi100.stdout: "features": { 2024-04-07T16:16:09.329 INFO:tasks.workunit.client.0.smithi100.stdout: "optional": [], 2024-04-07T16:16:09.329 INFO:tasks.workunit.client.0.smithi100.stdout: "persistent": [ 2024-04-07T16:16:09.329 INFO:tasks.workunit.client.0.smithi100.stdout: "kraken", 2024-04-07T16:16:09.329 INFO:tasks.workunit.client.0.smithi100.stdout: "luminous", 2024-04-07T16:16:09.329 INFO:tasks.workunit.client.0.smithi100.stdout: "mimic", 2024-04-07T16:16:09.330 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap-prune", 2024-04-07T16:16:09.330 INFO:tasks.workunit.client.0.smithi100.stdout: "nautilus", 2024-04-07T16:16:09.330 INFO:tasks.workunit.client.0.smithi100.stdout: "octopus", 2024-04-07T16:16:09.330 INFO:tasks.workunit.client.0.smithi100.stdout: "pacific", 2024-04-07T16:16:09.330 INFO:tasks.workunit.client.0.smithi100.stdout: "elector-pinging", 2024-04-07T16:16:09.330 INFO:tasks.workunit.client.0.smithi100.stdout: "quincy", 2024-04-07T16:16:09.330 INFO:tasks.workunit.client.0.smithi100.stdout: "reef" 2024-04-07T16:16:09.330 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T16:16:09.330 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:09.330 INFO:tasks.workunit.client.0.smithi100.stdout: "ipv4_addr_mons": 6, 2024-04-07T16:16:09.330 INFO:tasks.workunit.client.0.smithi100.stdout: "ipv6_addr_mons": 0, 2024-04-07T16:16:09.330 INFO:tasks.workunit.client.0.smithi100.stdout: "min_mon_release": 18, 2024-04-07T16:16:09.330 INFO:tasks.workunit.client.0.smithi100.stdout: "v1_addr_mons": 3, 2024-04-07T16:16:09.330 INFO:tasks.workunit.client.0.smithi100.stdout: "v2_addr_mons": 3 2024-04-07T16:16:09.330 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:09.331 INFO:tasks.workunit.client.0.smithi100.stdout: "organization": "ceph-qa", 2024-04-07T16:16:09.331 INFO:tasks.workunit.client.0.smithi100.stdout: "osd": { 2024-04-07T16:16:09.331 INFO:tasks.workunit.client.0.smithi100.stdout: "cluster_network": false, 2024-04-07T16:16:09.331 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 8, 2024-04-07T16:16:09.331 INFO:tasks.workunit.client.0.smithi100.stdout: "require_min_compat_client": "luminous", 2024-04-07T16:16:09.331 INFO:tasks.workunit.client.0.smithi100.stdout: "require_osd_release": "reef" 2024-04-07T16:16:09.331 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:09.331 INFO:tasks.workunit.client.0.smithi100.stdout: "pools": [ 2024-04-07T16:16:09.331 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:09.331 INFO:tasks.workunit.client.0.smithi100.stdout: "application": [ 2024-04-07T16:16:09.331 INFO:tasks.workunit.client.0.smithi100.stdout: "mgr" 2024-04-07T16:16:09.331 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:09.331 INFO:tasks.workunit.client.0.smithi100.stdout: "cache_mode": "none", 2024-04-07T16:16:09.331 INFO:tasks.workunit.client.0.smithi100.stdout: "erasure_code_profile": {}, 2024-04-07T16:16:09.331 INFO:tasks.workunit.client.0.smithi100.stdout: "min_size": 2, 2024-04-07T16:16:09.331 INFO:tasks.workunit.client.0.smithi100.stdout: "options": {}, 2024-04-07T16:16:09.332 INFO:tasks.workunit.client.0.smithi100.stdout: "pg_autoscale_mode": "off", 2024-04-07T16:16:09.332 INFO:tasks.workunit.client.0.smithi100.stdout: "pg_num": 1, 2024-04-07T16:16:09.332 INFO:tasks.workunit.client.0.smithi100.stdout: "pgp_num": 1, 2024-04-07T16:16:09.332 INFO:tasks.workunit.client.0.smithi100.stdout: "pool": 1, 2024-04-07T16:16:09.332 INFO:tasks.workunit.client.0.smithi100.stdout: "size": 3, 2024-04-07T16:16:09.332 INFO:tasks.workunit.client.0.smithi100.stdout: "stats": { 2024-04-07T16:16:09.332 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_raw": 729296873738, 2024-04-07T16:16:09.332 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_used": 1781760, 2024-04-07T16:16:09.332 INFO:tasks.workunit.client.0.smithi100.stdout: "compress_bytes_used": 0, 2024-04-07T16:16:09.332 INFO:tasks.workunit.client.0.smithi100.stdout: "compress_under_bytes": 0, 2024-04-07T16:16:09.332 INFO:tasks.workunit.client.0.smithi100.stdout: "data_bytes_used": 1781760, 2024-04-07T16:16:09.332 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty": 0, 2024-04-07T16:16:09.332 INFO:tasks.workunit.client.0.smithi100.stdout: "max_avail": 243098959872, 2024-04-07T16:16:09.332 INFO:tasks.workunit.client.0.smithi100.stdout: "objects": 2, 2024-04-07T16:16:09.333 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_bytes_used": 0, 2024-04-07T16:16:09.333 INFO:tasks.workunit.client.0.smithi100.stdout: "percent_used": 2.443114453853923e-06, 2024-04-07T16:16:09.333 INFO:tasks.workunit.client.0.smithi100.stdout: "quota_bytes": 0, 2024-04-07T16:16:09.333 INFO:tasks.workunit.client.0.smithi100.stdout: "quota_objects": 0, 2024-04-07T16:16:09.333 INFO:tasks.workunit.client.0.smithi100.stdout: "rd": 212, 2024-04-07T16:16:09.333 INFO:tasks.workunit.client.0.smithi100.stdout: "rd_bytes": 313344, 2024-04-07T16:16:09.333 INFO:tasks.workunit.client.0.smithi100.stdout: "stored": 590368, 2024-04-07T16:16:09.333 INFO:tasks.workunit.client.0.smithi100.stdout: "stored_data": 590368, 2024-04-07T16:16:09.333 INFO:tasks.workunit.client.0.smithi100.stdout: "stored_omap": 0, 2024-04-07T16:16:09.333 INFO:tasks.workunit.client.0.smithi100.stdout: "stored_raw": 1771104, 2024-04-07T16:16:09.333 INFO:tasks.workunit.client.0.smithi100.stdout: "wr": 179, 2024-04-07T16:16:09.333 INFO:tasks.workunit.client.0.smithi100.stdout: "wr_bytes": 2473984 2024-04-07T16:16:09.333 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:09.333 INFO:tasks.workunit.client.0.smithi100.stdout: "target_max_bytes": 0, 2024-04-07T16:16:09.334 INFO:tasks.workunit.client.0.smithi100.stdout: "target_max_objects": 0, 2024-04-07T16:16:09.334 INFO:tasks.workunit.client.0.smithi100.stdout: "type": "replicated" 2024-04-07T16:16:09.334 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:09.334 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:09.334 INFO:tasks.workunit.client.0.smithi100.stdout: "application": [ 2024-04-07T16:16:09.334 INFO:tasks.workunit.client.0.smithi100.stdout: "cephfs" 2024-04-07T16:16:09.334 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:09.334 INFO:tasks.workunit.client.0.smithi100.stdout: "cache_mode": "none", 2024-04-07T16:16:09.334 INFO:tasks.workunit.client.0.smithi100.stdout: "erasure_code_profile": {}, 2024-04-07T16:16:09.334 INFO:tasks.workunit.client.0.smithi100.stdout: "min_size": 2, 2024-04-07T16:16:09.334 INFO:tasks.workunit.client.0.smithi100.stdout: "options": {}, 2024-04-07T16:16:09.334 INFO:tasks.workunit.client.0.smithi100.stdout: "pg_autoscale_mode": "off", 2024-04-07T16:16:09.334 INFO:tasks.workunit.client.0.smithi100.stdout: "pg_num": 32, 2024-04-07T16:16:09.334 INFO:tasks.workunit.client.0.smithi100.stdout: "pgp_num": 32, 2024-04-07T16:16:09.335 INFO:tasks.workunit.client.0.smithi100.stdout: "pool": 2, 2024-04-07T16:16:09.335 INFO:tasks.workunit.client.0.smithi100.stdout: "size": 3, 2024-04-07T16:16:09.335 INFO:tasks.workunit.client.0.smithi100.stdout: "stats": { 2024-04-07T16:16:09.335 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_raw": 729296873738, 2024-04-07T16:16:09.335 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_used": 98304, 2024-04-07T16:16:09.335 INFO:tasks.workunit.client.0.smithi100.stdout: "compress_bytes_used": 0, 2024-04-07T16:16:09.335 INFO:tasks.workunit.client.0.smithi100.stdout: "compress_under_bytes": 0, 2024-04-07T16:16:09.335 INFO:tasks.workunit.client.0.smithi100.stdout: "data_bytes_used": 98304, 2024-04-07T16:16:09.335 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty": 0, 2024-04-07T16:16:09.335 INFO:tasks.workunit.client.0.smithi100.stdout: "max_avail": 243098959872, 2024-04-07T16:16:09.335 INFO:tasks.workunit.client.0.smithi100.stdout: "objects": 22, 2024-04-07T16:16:09.335 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_bytes_used": 0, 2024-04-07T16:16:09.335 INFO:tasks.workunit.client.0.smithi100.stdout: "percent_used": 1.3479282756634348e-07, 2024-04-07T16:16:09.335 INFO:tasks.workunit.client.0.smithi100.stdout: "quota_bytes": 0, 2024-04-07T16:16:09.335 INFO:tasks.workunit.client.0.smithi100.stdout: "quota_objects": 0, 2024-04-07T16:16:09.335 INFO:tasks.workunit.client.0.smithi100.stdout: "rd": 0, 2024-04-07T16:16:09.336 INFO:tasks.workunit.client.0.smithi100.stdout: "rd_bytes": 0, 2024-04-07T16:16:09.336 INFO:tasks.workunit.client.0.smithi100.stdout: "stored": 3430, 2024-04-07T16:16:09.336 INFO:tasks.workunit.client.0.smithi100.stdout: "stored_data": 3430, 2024-04-07T16:16:09.336 INFO:tasks.workunit.client.0.smithi100.stdout: "stored_omap": 0, 2024-04-07T16:16:09.336 INFO:tasks.workunit.client.0.smithi100.stdout: "stored_raw": 10290, 2024-04-07T16:16:09.336 INFO:tasks.workunit.client.0.smithi100.stdout: "wr": 51, 2024-04-07T16:16:09.336 INFO:tasks.workunit.client.0.smithi100.stdout: "wr_bytes": 20480 2024-04-07T16:16:09.336 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:09.336 INFO:tasks.workunit.client.0.smithi100.stdout: "target_max_bytes": 0, 2024-04-07T16:16:09.336 INFO:tasks.workunit.client.0.smithi100.stdout: "target_max_objects": 0, 2024-04-07T16:16:09.336 INFO:tasks.workunit.client.0.smithi100.stdout: "type": "replicated" 2024-04-07T16:16:09.336 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:09.336 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:09.336 INFO:tasks.workunit.client.0.smithi100.stdout: "application": [ 2024-04-07T16:16:09.337 INFO:tasks.workunit.client.0.smithi100.stdout: "cephfs" 2024-04-07T16:16:09.337 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:09.337 INFO:tasks.workunit.client.0.smithi100.stdout: "cache_mode": "none", 2024-04-07T16:16:09.337 INFO:tasks.workunit.client.0.smithi100.stdout: "erasure_code_profile": {}, 2024-04-07T16:16:09.337 INFO:tasks.workunit.client.0.smithi100.stdout: "min_size": 2, 2024-04-07T16:16:09.337 INFO:tasks.workunit.client.0.smithi100.stdout: "options": {}, 2024-04-07T16:16:09.337 INFO:tasks.workunit.client.0.smithi100.stdout: "pg_autoscale_mode": "off", 2024-04-07T16:16:09.337 INFO:tasks.workunit.client.0.smithi100.stdout: "pg_num": 64, 2024-04-07T16:16:09.337 INFO:tasks.workunit.client.0.smithi100.stdout: "pgp_num": 64, 2024-04-07T16:16:09.337 INFO:tasks.workunit.client.0.smithi100.stdout: "pool": 3, 2024-04-07T16:16:09.337 INFO:tasks.workunit.client.0.smithi100.stdout: "size": 3, 2024-04-07T16:16:09.337 INFO:tasks.workunit.client.0.smithi100.stdout: "stats": { 2024-04-07T16:16:09.337 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_raw": 729296873738, 2024-04-07T16:16:09.337 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_used": 0, 2024-04-07T16:16:09.337 INFO:tasks.workunit.client.0.smithi100.stdout: "compress_bytes_used": 0, 2024-04-07T16:16:09.338 INFO:tasks.workunit.client.0.smithi100.stdout: "compress_under_bytes": 0, 2024-04-07T16:16:09.338 INFO:tasks.workunit.client.0.smithi100.stdout: "data_bytes_used": 0, 2024-04-07T16:16:09.338 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty": 0, 2024-04-07T16:16:09.338 INFO:tasks.workunit.client.0.smithi100.stdout: "max_avail": 243098959872, 2024-04-07T16:16:09.338 INFO:tasks.workunit.client.0.smithi100.stdout: "objects": 0, 2024-04-07T16:16:09.338 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_bytes_used": 0, 2024-04-07T16:16:09.338 INFO:tasks.workunit.client.0.smithi100.stdout: "percent_used": 0.0, 2024-04-07T16:16:09.338 INFO:tasks.workunit.client.0.smithi100.stdout: "quota_bytes": 0, 2024-04-07T16:16:09.338 INFO:tasks.workunit.client.0.smithi100.stdout: "quota_objects": 0, 2024-04-07T16:16:09.338 INFO:tasks.workunit.client.0.smithi100.stdout: "rd": 0, 2024-04-07T16:16:09.338 INFO:tasks.workunit.client.0.smithi100.stdout: "rd_bytes": 0, 2024-04-07T16:16:09.338 INFO:tasks.workunit.client.0.smithi100.stdout: "stored": 0, 2024-04-07T16:16:09.338 INFO:tasks.workunit.client.0.smithi100.stdout: "stored_data": 0, 2024-04-07T16:16:09.338 INFO:tasks.workunit.client.0.smithi100.stdout: "stored_omap": 0, 2024-04-07T16:16:09.339 INFO:tasks.workunit.client.0.smithi100.stdout: "stored_raw": 0, 2024-04-07T16:16:09.339 INFO:tasks.workunit.client.0.smithi100.stdout: "wr": 0, 2024-04-07T16:16:09.339 INFO:tasks.workunit.client.0.smithi100.stdout: "wr_bytes": 0 2024-04-07T16:16:09.339 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:09.339 INFO:tasks.workunit.client.0.smithi100.stdout: "target_max_bytes": 0, 2024-04-07T16:16:09.339 INFO:tasks.workunit.client.0.smithi100.stdout: "target_max_objects": 0, 2024-04-07T16:16:09.339 INFO:tasks.workunit.client.0.smithi100.stdout: "type": "replicated" 2024-04-07T16:16:09.339 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:09.339 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:09.339 INFO:tasks.workunit.client.0.smithi100.stdout: "application": [ 2024-04-07T16:16:09.339 INFO:tasks.workunit.client.0.smithi100.stdout: "rbd" 2024-04-07T16:16:09.339 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:09.339 INFO:tasks.workunit.client.0.smithi100.stdout: "cache_mode": "none", 2024-04-07T16:16:09.339 INFO:tasks.workunit.client.0.smithi100.stdout: "erasure_code_profile": {}, 2024-04-07T16:16:09.339 INFO:tasks.workunit.client.0.smithi100.stdout: "min_size": 2, 2024-04-07T16:16:09.340 INFO:tasks.workunit.client.0.smithi100.stdout: "options": {}, 2024-04-07T16:16:09.340 INFO:tasks.workunit.client.0.smithi100.stdout: "pg_autoscale_mode": "off", 2024-04-07T16:16:09.340 INFO:tasks.workunit.client.0.smithi100.stdout: "pg_num": 8, 2024-04-07T16:16:09.340 INFO:tasks.workunit.client.0.smithi100.stdout: "pgp_num": 8, 2024-04-07T16:16:09.340 INFO:tasks.workunit.client.0.smithi100.stdout: "pool": 4, 2024-04-07T16:16:09.340 INFO:tasks.workunit.client.0.smithi100.stdout: "size": 3, 2024-04-07T16:16:09.340 INFO:tasks.workunit.client.0.smithi100.stdout: "stats": { 2024-04-07T16:16:09.340 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_raw": 729296873738, 2024-04-07T16:16:09.340 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_used": 0, 2024-04-07T16:16:09.340 INFO:tasks.workunit.client.0.smithi100.stdout: "compress_bytes_used": 0, 2024-04-07T16:16:09.340 INFO:tasks.workunit.client.0.smithi100.stdout: "compress_under_bytes": 0, 2024-04-07T16:16:09.340 INFO:tasks.workunit.client.0.smithi100.stdout: "data_bytes_used": 0, 2024-04-07T16:16:09.340 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty": 0, 2024-04-07T16:16:09.340 INFO:tasks.workunit.client.0.smithi100.stdout: "max_avail": 243098959872, 2024-04-07T16:16:09.340 INFO:tasks.workunit.client.0.smithi100.stdout: "objects": 0, 2024-04-07T16:16:09.341 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_bytes_used": 0, 2024-04-07T16:16:09.341 INFO:tasks.workunit.client.0.smithi100.stdout: "percent_used": 0.0, 2024-04-07T16:16:09.341 INFO:tasks.workunit.client.0.smithi100.stdout: "quota_bytes": 0, 2024-04-07T16:16:09.341 INFO:tasks.workunit.client.0.smithi100.stdout: "quota_objects": 0, 2024-04-07T16:16:09.341 INFO:tasks.workunit.client.0.smithi100.stdout: "rd": 0, 2024-04-07T16:16:09.341 INFO:tasks.workunit.client.0.smithi100.stdout: "rd_bytes": 0, 2024-04-07T16:16:09.341 INFO:tasks.workunit.client.0.smithi100.stdout: "stored": 0, 2024-04-07T16:16:09.341 INFO:tasks.workunit.client.0.smithi100.stdout: "stored_data": 0, 2024-04-07T16:16:09.341 INFO:tasks.workunit.client.0.smithi100.stdout: "stored_omap": 0, 2024-04-07T16:16:09.341 INFO:tasks.workunit.client.0.smithi100.stdout: "stored_raw": 0, 2024-04-07T16:16:09.341 INFO:tasks.workunit.client.0.smithi100.stdout: "wr": 0, 2024-04-07T16:16:09.341 INFO:tasks.workunit.client.0.smithi100.stdout: "wr_bytes": 0 2024-04-07T16:16:09.341 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:09.341 INFO:tasks.workunit.client.0.smithi100.stdout: "target_max_bytes": 0, 2024-04-07T16:16:09.342 INFO:tasks.workunit.client.0.smithi100.stdout: "target_max_objects": 0, 2024-04-07T16:16:09.342 INFO:tasks.workunit.client.0.smithi100.stdout: "type": "replicated" 2024-04-07T16:16:09.342 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:09.342 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:09.342 INFO:tasks.workunit.client.0.smithi100.stdout: "rbd": { 2024-04-07T16:16:09.342 INFO:tasks.workunit.client.0.smithi100.stdout: "mirroring_by_pool": [ 2024-04-07T16:16:09.342 INFO:tasks.workunit.client.0.smithi100.stdout: false 2024-04-07T16:16:09.342 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:09.342 INFO:tasks.workunit.client.0.smithi100.stdout: "num_images_by_pool": [ 2024-04-07T16:16:09.342 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T16:16:09.342 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:09.342 INFO:tasks.workunit.client.0.smithi100.stdout: "num_pools": 1 2024-04-07T16:16:09.342 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:09.342 INFO:tasks.workunit.client.0.smithi100.stdout: "report_id": "ee983b25-10fa-4633-b6cf-1c3036b91b52", 2024-04-07T16:16:09.342 INFO:tasks.workunit.client.0.smithi100.stdout: "report_timestamp": "2024-04-07T16:16:09.292505", 2024-04-07T16:16:09.343 INFO:tasks.workunit.client.0.smithi100.stdout: "report_version": 1, 2024-04-07T16:16:09.343 INFO:tasks.workunit.client.0.smithi100.stdout: "rook": { 2024-04-07T16:16:09.343 INFO:tasks.workunit.client.0.smithi100.stdout: "cluster": { 2024-04-07T16:16:09.343 INFO:tasks.workunit.client.0.smithi100.stdout: "external-mode": null, 2024-04-07T16:16:09.343 INFO:tasks.workunit.client.0.smithi100.stdout: "mon": { 2024-04-07T16:16:09.343 INFO:tasks.workunit.client.0.smithi100.stdout: "allow-multiple-per-node": null, 2024-04-07T16:16:09.343 INFO:tasks.workunit.client.0.smithi100.stdout: "count": null, 2024-04-07T16:16:09.343 INFO:tasks.workunit.client.0.smithi100.stdout: "max-id": null, 2024-04-07T16:16:09.343 INFO:tasks.workunit.client.0.smithi100.stdout: "pvc": { 2024-04-07T16:16:09.343 INFO:tasks.workunit.client.0.smithi100.stdout: "enabled": null 2024-04-07T16:16:09.343 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:09.343 INFO:tasks.workunit.client.0.smithi100.stdout: "stretch": { 2024-04-07T16:16:09.343 INFO:tasks.workunit.client.0.smithi100.stdout: "enabled": null 2024-04-07T16:16:09.343 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:09.343 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:09.343 INFO:tasks.workunit.client.0.smithi100.stdout: "network": { 2024-04-07T16:16:09.344 INFO:tasks.workunit.client.0.smithi100.stdout: "provider": null 2024-04-07T16:16:09.344 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:09.344 INFO:tasks.workunit.client.0.smithi100.stdout: "storage": { 2024-04-07T16:16:09.344 INFO:tasks.workunit.client.0.smithi100.stdout: "device-set": { 2024-04-07T16:16:09.344 INFO:tasks.workunit.client.0.smithi100.stdout: "count": { 2024-04-07T16:16:09.344 INFO:tasks.workunit.client.0.smithi100.stdout: "non-portable": null, 2024-04-07T16:16:09.344 INFO:tasks.workunit.client.0.smithi100.stdout: "portable": null, 2024-04-07T16:16:09.344 INFO:tasks.workunit.client.0.smithi100.stdout: "total": null 2024-04-07T16:16:09.344 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:09.344 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:09.344 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:09.344 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:09.344 INFO:tasks.workunit.client.0.smithi100.stdout: "csi": { 2024-04-07T16:16:09.344 INFO:tasks.workunit.client.0.smithi100.stdout: "version": null 2024-04-07T16:16:09.344 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:09.344 INFO:tasks.workunit.client.0.smithi100.stdout: "kubernetes": { 2024-04-07T16:16:09.345 INFO:tasks.workunit.client.0.smithi100.stdout: "version": null 2024-04-07T16:16:09.345 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:09.345 INFO:tasks.workunit.client.0.smithi100.stdout: "node": { 2024-04-07T16:16:09.345 INFO:tasks.workunit.client.0.smithi100.stdout: "count": { 2024-04-07T16:16:09.345 INFO:tasks.workunit.client.0.smithi100.stdout: "kubernetes-total": null, 2024-04-07T16:16:09.345 INFO:tasks.workunit.client.0.smithi100.stdout: "with-ceph-daemons": null, 2024-04-07T16:16:09.345 INFO:tasks.workunit.client.0.smithi100.stdout: "with-csi-cephfs-plugin": null, 2024-04-07T16:16:09.345 INFO:tasks.workunit.client.0.smithi100.stdout: "with-csi-nfs-plugin": null, 2024-04-07T16:16:09.345 INFO:tasks.workunit.client.0.smithi100.stdout: "with-csi-rbd-plugin": null 2024-04-07T16:16:09.345 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:09.345 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:09.345 INFO:tasks.workunit.client.0.smithi100.stdout: "usage": { 2024-04-07T16:16:09.345 INFO:tasks.workunit.client.0.smithi100.stdout: "storage-class": { 2024-04-07T16:16:09.345 INFO:tasks.workunit.client.0.smithi100.stdout: "count": { 2024-04-07T16:16:09.345 INFO:tasks.workunit.client.0.smithi100.stdout: "bucket": null, 2024-04-07T16:16:09.346 INFO:tasks.workunit.client.0.smithi100.stdout: "cephfs": null, 2024-04-07T16:16:09.346 INFO:tasks.workunit.client.0.smithi100.stdout: "nfs": null, 2024-04-07T16:16:09.346 INFO:tasks.workunit.client.0.smithi100.stdout: "rbd": null, 2024-04-07T16:16:09.346 INFO:tasks.workunit.client.0.smithi100.stdout: "total": null 2024-04-07T16:16:09.346 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:09.346 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:09.346 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:09.346 INFO:tasks.workunit.client.0.smithi100.stdout: "version": null 2024-04-07T16:16:09.346 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:09.346 INFO:tasks.workunit.client.0.smithi100.stdout: "services": {}, 2024-04-07T16:16:09.346 INFO:tasks.workunit.client.0.smithi100.stdout: "usage": { 2024-04-07T16:16:09.346 INFO:tasks.workunit.client.0.smithi100.stdout: "pg_num": 105, 2024-04-07T16:16:09.346 INFO:tasks.workunit.client.0.smithi100.stdout: "pools": 4, 2024-04-07T16:16:09.346 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_by_class": { 2024-04-07T16:16:09.346 INFO:tasks.workunit.client.0.smithi100.stdout: "ssd": { 2024-04-07T16:16:09.347 INFO:tasks.workunit.client.0.smithi100.stdout: "total_avail_bytes": 767718821888, 2024-04-07T16:16:09.347 INFO:tasks.workunit.client.0.smithi100.stdout: "total_bytes": 767960285184, 2024-04-07T16:16:09.347 INFO:tasks.workunit.client.0.smithi100.stdout: "total_used_bytes": 241463296, 2024-04-07T16:16:09.347 INFO:tasks.workunit.client.0.smithi100.stdout: "total_used_raw_bytes": 241463296, 2024-04-07T16:16:09.347 INFO:tasks.workunit.client.0.smithi100.stdout: "total_used_raw_ratio": 0.000314421602524817 2024-04-07T16:16:09.347 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:09.347 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:09.347 INFO:tasks.workunit.client.0.smithi100.stdout: "total_avail_bytes": 767718821888, 2024-04-07T16:16:09.347 INFO:tasks.workunit.client.0.smithi100.stdout: "total_bytes": 767960285184, 2024-04-07T16:16:09.347 INFO:tasks.workunit.client.0.smithi100.stdout: "total_used_bytes": 241463296 2024-04-07T16:16:09.347 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:09.347 INFO:tasks.workunit.client.0.smithi100.stdout:} 2024-04-07T16:16:09.347 INFO:tasks.workunit.client.0.smithi100.stdout:Device report is generated separately. To see it run 'ceph telemetry preview-device'. 2024-04-07T16:16:09.348 INFO:tasks.workunit.client.0.smithi100.stderr:+ ceph telemetry preview-device 2024-04-07T16:16:09.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:09 smithi144 bash[23522]: audit 2024-04-07T16:16:08.261884+0000 mon.a (mon.0) 845 : audit 1 from='client.? ' entity='client.admin' 2024-04-07T16:16:09.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:09 smithi144 bash[23522]: audit 2024-04-07T16:16:08.320924+0000 mon.b (mon.1) 100 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:16:09.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:09 smithi144 bash[23522]: audit 2024-04-07T16:16:08.324025+0000 mon.b (mon.1) 101 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:16:09.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:09 smithi144 bash[23522]: audit 2024-04-07T16:16:08.325396+0000 mon.b (mon.1) 102 : audit 1 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:16:09.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:09 smithi144 bash[23522]: audit 2024-04-07T16:16:08.340748+0000 mon.a (mon.0) 846 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:16:09.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:09 smithi144 bash[23522]: audit 2024-04-07T16:16:08.664115+0000 mon.a (mon.0) 847 : audit 1 from='client.? ' entity='client.admin' 2024-04-07T16:16:09.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:09 smithi144 bash[23522]: audit 2024-04-07T16:16:08.715340+0000 mon.b (mon.1) 103 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:16:09.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:09 smithi144 bash[23522]: audit 2024-04-07T16:16:08.717895+0000 mon.b (mon.1) 104 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:16:09.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:09 smithi144 bash[23522]: audit 2024-04-07T16:16:08.719191+0000 mon.b (mon.1) 105 : audit 1 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:16:09.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:09 smithi144 bash[23522]: audit 2024-04-07T16:16:08.731860+0000 mon.a (mon.0) 848 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:16:09.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:09 smithi144 bash[23522]: audit 2024-04-07T16:16:09.122209+0000 mon.b (mon.1) 106 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:16:09.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:09 smithi144 bash[23522]: audit 2024-04-07T16:16:09.135306+0000 mon.b (mon.1) 107 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config-key dump", "format": "json"}]: dispatch 2024-04-07T16:16:09.687 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:09 smithi100 bash[21346]: audit 2024-04-07T16:16:08.261884+0000 mon.a (mon.0) 845 : audit 1 from='client.? ' entity='client.admin' 2024-04-07T16:16:09.687 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:09 smithi100 bash[21346]: audit 2024-04-07T16:16:08.320924+0000 mon.b (mon.1) 100 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:16:09.687 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:09 smithi100 bash[21346]: audit 2024-04-07T16:16:08.324025+0000 mon.b (mon.1) 101 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:16:09.687 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:09 smithi100 bash[21346]: audit 2024-04-07T16:16:08.325396+0000 mon.b (mon.1) 102 : audit 1 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:16:09.687 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:09 smithi100 bash[21346]: audit 2024-04-07T16:16:08.340748+0000 mon.a (mon.0) 846 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:16:09.688 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:09 smithi100 bash[21346]: audit 2024-04-07T16:16:08.664115+0000 mon.a (mon.0) 847 : audit 1 from='client.? ' entity='client.admin' 2024-04-07T16:16:09.688 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:09 smithi100 bash[21346]: audit 2024-04-07T16:16:08.715340+0000 mon.b (mon.1) 103 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:16:09.688 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:09 smithi100 bash[21346]: audit 2024-04-07T16:16:08.717895+0000 mon.b (mon.1) 104 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:16:09.688 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:09 smithi100 bash[21346]: audit 2024-04-07T16:16:08.719191+0000 mon.b (mon.1) 105 : audit 1 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:16:09.688 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:09 smithi100 bash[21346]: audit 2024-04-07T16:16:08.731860+0000 mon.a (mon.0) 848 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:16:09.688 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:09 smithi100 bash[21346]: audit 2024-04-07T16:16:09.122209+0000 mon.b (mon.1) 106 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:16:09.688 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:09 smithi100 bash[21346]: audit 2024-04-07T16:16:09.135306+0000 mon.b (mon.1) 107 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config-key dump", "format": "json"}]: dispatch 2024-04-07T16:16:09.688 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:09 smithi100 bash[31660]: audit 2024-04-07T16:16:08.261884+0000 mon.a (mon.0) 845 : audit 1 from='client.? ' entity='client.admin' 2024-04-07T16:16:09.688 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:09 smithi100 bash[31660]: audit 2024-04-07T16:16:08.320924+0000 mon.b (mon.1) 100 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:16:09.689 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:09 smithi100 bash[31660]: audit 2024-04-07T16:16:08.324025+0000 mon.b (mon.1) 101 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:16:09.689 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:09 smithi100 bash[31660]: audit 2024-04-07T16:16:08.325396+0000 mon.b (mon.1) 102 : audit 1 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:16:09.689 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:09 smithi100 bash[31660]: audit 2024-04-07T16:16:08.340748+0000 mon.a (mon.0) 846 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:16:09.689 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:09 smithi100 bash[31660]: audit 2024-04-07T16:16:08.664115+0000 mon.a (mon.0) 847 : audit 1 from='client.? ' entity='client.admin' 2024-04-07T16:16:09.689 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:09 smithi100 bash[31660]: audit 2024-04-07T16:16:08.715340+0000 mon.b (mon.1) 103 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:16:09.689 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:09 smithi100 bash[31660]: audit 2024-04-07T16:16:08.717895+0000 mon.b (mon.1) 104 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:16:09.689 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:09 smithi100 bash[31660]: audit 2024-04-07T16:16:08.719191+0000 mon.b (mon.1) 105 : audit 1 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:16:09.689 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:09 smithi100 bash[31660]: audit 2024-04-07T16:16:08.731860+0000 mon.a (mon.0) 848 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:16:09.689 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:09 smithi100 bash[31660]: audit 2024-04-07T16:16:09.122209+0000 mon.b (mon.1) 106 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:16:09.689 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:09 smithi100 bash[31660]: audit 2024-04-07T16:16:09.135306+0000 mon.b (mon.1) 107 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config-key dump", "format": "json"}]: dispatch 2024-04-07T16:16:09.890 INFO:tasks.workunit.client.0.smithi100.stdout:{ 2024-04-07T16:16:09.890 INFO:tasks.workunit.client.0.smithi100.stdout: "251e7b66-f4fa-11ee-8447-0cc47a8fd733": { 2024-04-07T16:16:09.890 INFO:tasks.workunit.client.0.smithi100.stdout: "INTEL_SSDPEDMD400G4_251ffd38-f4fa-11ee-8447-0cc47a8fd733": { 2024-04-07T16:16:09.891 INFO:tasks.workunit.client.0.smithi100.stdout: "20240407-160729": { 2024-04-07T16:16:09.891 INFO:tasks.workunit.client.0.smithi100.stdout: "device": { 2024-04-07T16:16:09.891 INFO:tasks.workunit.client.0.smithi100.stdout: "info_name": "/dev/nvme0n1", 2024-04-07T16:16:09.891 INFO:tasks.workunit.client.0.smithi100.stdout: "name": "/dev/nvme0n1", 2024-04-07T16:16:09.891 INFO:tasks.workunit.client.0.smithi100.stdout: "protocol": "NVMe", 2024-04-07T16:16:09.891 INFO:tasks.workunit.client.0.smithi100.stdout: "type": "nvme" 2024-04-07T16:16:09.891 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:09.891 INFO:tasks.workunit.client.0.smithi100.stdout: "firmware_version": "8DV101H0", 2024-04-07T16:16:09.892 INFO:tasks.workunit.client.0.smithi100.stdout: "host_id": "251e7b66-f4fa-11ee-8447-0cc47a8fd733", 2024-04-07T16:16:09.892 INFO:tasks.workunit.client.0.smithi100.stdout: "json_format_version": [ 2024-04-07T16:16:09.892 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T16:16:09.892 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T16:16:09.892 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:09.892 INFO:tasks.workunit.client.0.smithi100.stdout: "local_time": { 2024-04-07T16:16:09.892 INFO:tasks.workunit.client.0.smithi100.stdout: "asctime": "Sun Apr 7 16:07:28 2024 UTC", 2024-04-07T16:16:09.892 INFO:tasks.workunit.client.0.smithi100.stdout: "time_t": 1712506048 2024-04-07T16:16:09.892 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:09.892 INFO:tasks.workunit.client.0.smithi100.stdout: "logical_block_size": 512, 2024-04-07T16:16:09.892 INFO:tasks.workunit.client.0.smithi100.stdout: "model_name": "INTEL SSDPEDMD400G4", 2024-04-07T16:16:09.892 INFO:tasks.workunit.client.0.smithi100.stdout: "nvme_controller_id": 0, 2024-04-07T16:16:09.892 INFO:tasks.workunit.client.0.smithi100.stdout: "nvme_ieee_oui_identifier": 6083300, 2024-04-07T16:16:09.892 INFO:tasks.workunit.client.0.smithi100.stdout: "nvme_namespaces": [ 2024-04-07T16:16:09.893 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:09.893 INFO:tasks.workunit.client.0.smithi100.stdout: "capacity": { 2024-04-07T16:16:09.893 INFO:tasks.workunit.client.0.smithi100.stdout: "blocks": 781422768, 2024-04-07T16:16:09.893 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 400088457216 2024-04-07T16:16:09.893 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:09.893 INFO:tasks.workunit.client.0.smithi100.stdout: "formatted_lba_size": 512, 2024-04-07T16:16:09.893 INFO:tasks.workunit.client.0.smithi100.stdout: "id": 1, 2024-04-07T16:16:09.893 INFO:tasks.workunit.client.0.smithi100.stdout: "size": { 2024-04-07T16:16:09.893 INFO:tasks.workunit.client.0.smithi100.stdout: "blocks": 781422768, 2024-04-07T16:16:09.893 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 400088457216 2024-04-07T16:16:09.893 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:09.893 INFO:tasks.workunit.client.0.smithi100.stdout: "utilization": { 2024-04-07T16:16:09.893 INFO:tasks.workunit.client.0.smithi100.stdout: "blocks": 781422768, 2024-04-07T16:16:09.893 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 400088457216 2024-04-07T16:16:09.894 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:09.894 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:09.894 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:09.894 INFO:tasks.workunit.client.0.smithi100.stdout: "nvme_number_of_namespaces": 1, 2024-04-07T16:16:09.894 INFO:tasks.workunit.client.0.smithi100.stdout: "nvme_pci_vendor": { 2024-04-07T16:16:09.894 INFO:tasks.workunit.client.0.smithi100.stdout: "id": 32902, 2024-04-07T16:16:09.894 INFO:tasks.workunit.client.0.smithi100.stdout: "subsystem_id": 32902 2024-04-07T16:16:09.894 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:09.894 INFO:tasks.workunit.client.0.smithi100.stdout: "nvme_smart_health_information_add_log": { 2024-04-07T16:16:09.894 INFO:tasks.workunit.client.0.smithi100.stdout: "Device stats": { 2024-04-07T16:16:09.894 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_fw_downgrades": { 2024-04-07T16:16:09.894 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 0, 2024-04-07T16:16:09.894 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 0 2024-04-07T16:16:09.894 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:09.895 INFO:tasks.workunit.client.0.smithi100.stdout: "crc_error_count": { 2024-04-07T16:16:09.895 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 100, 2024-04-07T16:16:09.895 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 0 2024-04-07T16:16:09.895 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:09.895 INFO:tasks.workunit.client.0.smithi100.stdout: "end_to_end_error_detection_count": { 2024-04-07T16:16:09.895 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 100, 2024-04-07T16:16:09.895 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 0 2024-04-07T16:16:09.895 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:09.895 INFO:tasks.workunit.client.0.smithi100.stdout: "erase_fail_count": { 2024-04-07T16:16:09.895 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 100, 2024-04-07T16:16:09.895 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 0 2024-04-07T16:16:09.896 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:09.896 INFO:tasks.workunit.client.0.smithi100.stdout: "host_bytes_written": { 2024-04-07T16:16:09.896 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 100, 2024-04-07T16:16:09.896 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 29568720 2024-04-07T16:16:09.896 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:09.896 INFO:tasks.workunit.client.0.smithi100.stdout: "host_ctx_wear_used": { 2024-04-07T16:16:09.896 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 100, 2024-04-07T16:16:09.896 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 1 2024-04-07T16:16:09.896 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:09.896 INFO:tasks.workunit.client.0.smithi100.stdout: "media_bytes_read": { 2024-04-07T16:16:09.896 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 0, 2024-04-07T16:16:09.896 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 0 2024-04-07T16:16:09.896 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:09.896 INFO:tasks.workunit.client.0.smithi100.stdout: "nand_bytes_written": { 2024-04-07T16:16:09.897 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 100, 2024-04-07T16:16:09.897 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 47802746 2024-04-07T16:16:09.897 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:09.897 INFO:tasks.workunit.client.0.smithi100.stdout: "perf_stat_indicator": { 2024-04-07T16:16:09.897 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 0, 2024-04-07T16:16:09.897 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 0 2024-04-07T16:16:09.897 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:09.897 INFO:tasks.workunit.client.0.smithi100.stdout: "pll_lock_loss_count": { 2024-04-07T16:16:09.897 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 100, 2024-04-07T16:16:09.897 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 0 2024-04-07T16:16:09.897 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:09.897 INFO:tasks.workunit.client.0.smithi100.stdout: "program_fail_count": { 2024-04-07T16:16:09.897 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 100, 2024-04-07T16:16:09.897 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 0 2024-04-07T16:16:09.898 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:09.898 INFO:tasks.workunit.client.0.smithi100.stdout: "re_alloc_sectr_cnt": { 2024-04-07T16:16:09.898 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 0, 2024-04-07T16:16:09.898 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 0 2024-04-07T16:16:09.898 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:09.898 INFO:tasks.workunit.client.0.smithi100.stdout: "retry_buffer_overflow_count": { 2024-04-07T16:16:09.898 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 100, 2024-04-07T16:16:09.898 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 0 2024-04-07T16:16:09.898 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:09.898 INFO:tasks.workunit.client.0.smithi100.stdout: "soft_ecc_err_rate": { 2024-04-07T16:16:09.898 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 0, 2024-04-07T16:16:09.898 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 0 2024-04-07T16:16:09.898 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:09.898 INFO:tasks.workunit.client.0.smithi100.stdout: "thermal_throttle_status": { 2024-04-07T16:16:09.899 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 100, 2024-04-07T16:16:09.899 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": { 2024-04-07T16:16:09.899 INFO:tasks.workunit.client.0.smithi100.stdout: "cnt": 0, 2024-04-07T16:16:09.899 INFO:tasks.workunit.client.0.smithi100.stdout: "pct": 0 2024-04-07T16:16:09.899 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:09.899 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:09.899 INFO:tasks.workunit.client.0.smithi100.stdout: "timed_workload_host_reads": { 2024-04-07T16:16:09.899 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 100, 2024-04-07T16:16:09.899 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 65535 2024-04-07T16:16:09.899 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:09.899 INFO:tasks.workunit.client.0.smithi100.stdout: "timed_workload_media_wear": { 2024-04-07T16:16:09.899 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 100, 2024-04-07T16:16:09.899 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 64 2024-04-07T16:16:09.899 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:09.899 INFO:tasks.workunit.client.0.smithi100.stdout: "timed_workload_timer": { 2024-04-07T16:16:09.900 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 100, 2024-04-07T16:16:09.900 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 65535 2024-04-07T16:16:09.900 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:09.900 INFO:tasks.workunit.client.0.smithi100.stdout: "unexp_power_loss": { 2024-04-07T16:16:09.900 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 0, 2024-04-07T16:16:09.900 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 0 2024-04-07T16:16:09.900 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:09.900 INFO:tasks.workunit.client.0.smithi100.stdout: "wear_leveling": { 2024-04-07T16:16:09.900 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 90, 2024-04-07T16:16:09.900 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": { 2024-04-07T16:16:09.900 INFO:tasks.workunit.client.0.smithi100.stdout: "avg": 3209, 2024-04-07T16:16:09.900 INFO:tasks.workunit.client.0.smithi100.stdout: "max": 3225, 2024-04-07T16:16:09.900 INFO:tasks.workunit.client.0.smithi100.stdout: "min": 3194 2024-04-07T16:16:09.900 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:09.901 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:09.901 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:09.901 INFO:tasks.workunit.client.0.smithi100.stdout: "Intel Smart log": "nvme0n1" 2024-04-07T16:16:09.901 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:09.901 INFO:tasks.workunit.client.0.smithi100.stdout: "nvme_smart_health_information_log": { 2024-04-07T16:16:09.901 INFO:tasks.workunit.client.0.smithi100.stdout: "available_spare": 100, 2024-04-07T16:16:09.901 INFO:tasks.workunit.client.0.smithi100.stdout: "available_spare_threshold": 10, 2024-04-07T16:16:09.901 INFO:tasks.workunit.client.0.smithi100.stdout: "controller_busy_time": 4, 2024-04-07T16:16:09.901 INFO:tasks.workunit.client.0.smithi100.stdout: "critical_warning": 0, 2024-04-07T16:16:09.901 INFO:tasks.workunit.client.0.smithi100.stdout: "data_units_read": 385784931, 2024-04-07T16:16:09.901 INFO:tasks.workunit.client.0.smithi100.stdout: "data_units_written": 1937815661, 2024-04-07T16:16:09.901 INFO:tasks.workunit.client.0.smithi100.stdout: "host_reads": 5771442418, 2024-04-07T16:16:09.901 INFO:tasks.workunit.client.0.smithi100.stdout: "host_writes": 31722089956, 2024-04-07T16:16:09.901 INFO:tasks.workunit.client.0.smithi100.stdout: "media_errors": 0, 2024-04-07T16:16:09.902 INFO:tasks.workunit.client.0.smithi100.stdout: "num_err_log_entries": 0, 2024-04-07T16:16:09.902 INFO:tasks.workunit.client.0.smithi100.stdout: "percentage_used": 10, 2024-04-07T16:16:09.902 INFO:tasks.workunit.client.0.smithi100.stdout: "power_cycles": 45774, 2024-04-07T16:16:09.902 INFO:tasks.workunit.client.0.smithi100.stdout: "power_on_hours": 57978, 2024-04-07T16:16:09.902 INFO:tasks.workunit.client.0.smithi100.stdout: "temperature": 24, 2024-04-07T16:16:09.902 INFO:tasks.workunit.client.0.smithi100.stdout: "unsafe_shutdowns": 210 2024-04-07T16:16:09.902 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:09.902 INFO:tasks.workunit.client.0.smithi100.stdout: "nvme_vendor": "intel", 2024-04-07T16:16:09.902 INFO:tasks.workunit.client.0.smithi100.stdout: "power_cycle_count": 45774, 2024-04-07T16:16:09.902 INFO:tasks.workunit.client.0.smithi100.stdout: "power_on_time": { 2024-04-07T16:16:09.902 INFO:tasks.workunit.client.0.smithi100.stdout: "hours": 57978 2024-04-07T16:16:09.902 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:09.902 INFO:tasks.workunit.client.0.smithi100.stdout: "serial_number": "deleted", 2024-04-07T16:16:09.902 INFO:tasks.workunit.client.0.smithi100.stdout: "smart_status": { 2024-04-07T16:16:09.902 INFO:tasks.workunit.client.0.smithi100.stdout: "nvme": { 2024-04-07T16:16:09.903 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:09.903 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:09.903 INFO:tasks.workunit.client.0.smithi100.stdout: "passed": true 2024-04-07T16:16:09.903 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:09.903 INFO:tasks.workunit.client.0.smithi100.stdout: "smartctl": { 2024-04-07T16:16:09.903 INFO:tasks.workunit.client.0.smithi100.stdout: "argv": [ 2024-04-07T16:16:09.903 INFO:tasks.workunit.client.0.smithi100.stdout: "smartctl", 2024-04-07T16:16:09.903 INFO:tasks.workunit.client.0.smithi100.stdout: "-x", 2024-04-07T16:16:09.903 INFO:tasks.workunit.client.0.smithi100.stdout: "--json=o", 2024-04-07T16:16:09.903 INFO:tasks.workunit.client.0.smithi100.stdout: "/dev/nvme0n1" 2024-04-07T16:16:09.903 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:09.903 INFO:tasks.workunit.client.0.smithi100.stdout: "build_info": "(local build)", 2024-04-07T16:16:09.903 INFO:tasks.workunit.client.0.smithi100.stdout: "exit_status": 0, 2024-04-07T16:16:09.903 INFO:tasks.workunit.client.0.smithi100.stdout: "output": [ 2024-04-07T16:16:09.903 INFO:tasks.workunit.client.0.smithi100.stdout: "smartctl 7.1 2019-12-30 r5022 [x86_64-linux-5.15.0-101-generic] (local build)", 2024-04-07T16:16:09.904 INFO:tasks.workunit.client.0.smithi100.stdout: "Copyright (C) 2002-19, Bruce Allen, Christian Franke, www.smartmontools.org", 2024-04-07T16:16:09.904 INFO:tasks.workunit.client.0.smithi100.stdout: "", 2024-04-07T16:16:09.904 INFO:tasks.workunit.client.0.smithi100.stdout: "=== START OF INFORMATION SECTION ===", 2024-04-07T16:16:09.904 INFO:tasks.workunit.client.0.smithi100.stdout: "Model Number: INTEL SSDPEDMD400G4", 2024-04-07T16:16:09.904 INFO:tasks.workunit.client.0.smithi100.stdout: "Serial Number: deleted", 2024-04-07T16:16:09.904 INFO:tasks.workunit.client.0.smithi100.stdout: "Firmware Version: 8DV101H0", 2024-04-07T16:16:09.904 INFO:tasks.workunit.client.0.smithi100.stdout: "PCI Vendor/Subsystem ID: 0x8086", 2024-04-07T16:16:09.904 INFO:tasks.workunit.client.0.smithi100.stdout: "IEEE OUI Identifier: 0x5cd2e4", 2024-04-07T16:16:09.904 INFO:tasks.workunit.client.0.smithi100.stdout: "Controller ID: 0", 2024-04-07T16:16:09.904 INFO:tasks.workunit.client.0.smithi100.stdout: "Number of Namespaces: 1", 2024-04-07T16:16:09.904 INFO:tasks.workunit.client.0.smithi100.stdout: "Namespace 1 Size/Capacity: 400,088,457,216 [400 GB]", 2024-04-07T16:16:09.904 INFO:tasks.workunit.client.0.smithi100.stdout: "Namespace 1 Formatted LBA Size: 512", 2024-04-07T16:16:09.904 INFO:tasks.workunit.client.0.smithi100.stdout: "Local Time is: Sun Apr 7 16:07:28 2024 UTC", 2024-04-07T16:16:09.904 INFO:tasks.workunit.client.0.smithi100.stdout: "Firmware Updates (0x02): 1 Slot", 2024-04-07T16:16:09.905 INFO:tasks.workunit.client.0.smithi100.stdout: "Optional Admin Commands (0x0006): Format Frmw_DL", 2024-04-07T16:16:09.905 INFO:tasks.workunit.client.0.smithi100.stdout: "Optional NVM Commands (0x0006): Wr_Unc DS_Mngmt", 2024-04-07T16:16:09.905 INFO:tasks.workunit.client.0.smithi100.stdout: "Maximum Data Transfer Size: 32 Pages", 2024-04-07T16:16:09.905 INFO:tasks.workunit.client.0.smithi100.stdout: "", 2024-04-07T16:16:09.905 INFO:tasks.workunit.client.0.smithi100.stdout: "Supported Power States", 2024-04-07T16:16:09.905 INFO:tasks.workunit.client.0.smithi100.stdout: "St Op Max Active Idle RL RT WL WT Ent_Lat Ex_Lat", 2024-04-07T16:16:09.905 INFO:tasks.workunit.client.0.smithi100.stdout: " 0 + 25.00W - - 0 0 0 0 0 0", 2024-04-07T16:16:09.905 INFO:tasks.workunit.client.0.smithi100.stdout: "", 2024-04-07T16:16:09.905 INFO:tasks.workunit.client.0.smithi100.stdout: "Supported LBA Sizes (NSID 0x1)", 2024-04-07T16:16:09.905 INFO:tasks.workunit.client.0.smithi100.stdout: "Id Fmt Data Metadt Rel_Perf", 2024-04-07T16:16:09.905 INFO:tasks.workunit.client.0.smithi100.stdout: " 0 + 512 0 2", 2024-04-07T16:16:09.905 INFO:tasks.workunit.client.0.smithi100.stdout: " 1 - 512 8 2", 2024-04-07T16:16:09.905 INFO:tasks.workunit.client.0.smithi100.stdout: " 2 - 512 16 2", 2024-04-07T16:16:09.905 INFO:tasks.workunit.client.0.smithi100.stdout: " 3 - 4096 0 0", 2024-04-07T16:16:09.906 INFO:tasks.workunit.client.0.smithi100.stdout: " 4 - 4096 8 0", 2024-04-07T16:16:09.906 INFO:tasks.workunit.client.0.smithi100.stdout: " 5 - 4096 64 0", 2024-04-07T16:16:09.906 INFO:tasks.workunit.client.0.smithi100.stdout: " 6 - 4096 128 0", 2024-04-07T16:16:09.906 INFO:tasks.workunit.client.0.smithi100.stdout: "", 2024-04-07T16:16:09.906 INFO:tasks.workunit.client.0.smithi100.stdout: "=== START OF SMART DATA SECTION ===", 2024-04-07T16:16:09.906 INFO:tasks.workunit.client.0.smithi100.stdout: "SMART overall-health self-assessment test result: PASSED", 2024-04-07T16:16:09.906 INFO:tasks.workunit.client.0.smithi100.stdout: "", 2024-04-07T16:16:09.906 INFO:tasks.workunit.client.0.smithi100.stdout: "SMART/Health Information (NVMe Log 0x02)", 2024-04-07T16:16:09.906 INFO:tasks.workunit.client.0.smithi100.stdout: "Critical Warning: 0x00", 2024-04-07T16:16:09.906 INFO:tasks.workunit.client.0.smithi100.stdout: "Temperature: 24 Celsius", 2024-04-07T16:16:09.906 INFO:tasks.workunit.client.0.smithi100.stdout: "Available Spare: 100%", 2024-04-07T16:16:09.906 INFO:tasks.workunit.client.0.smithi100.stdout: "Available Spare Threshold: 10%", 2024-04-07T16:16:09.906 INFO:tasks.workunit.client.0.smithi100.stdout: "Percentage Used: 10%", 2024-04-07T16:16:09.907 INFO:tasks.workunit.client.0.smithi100.stdout: "Data Units Read: 385,784,931 [197 TB]", 2024-04-07T16:16:09.907 INFO:tasks.workunit.client.0.smithi100.stdout: "Data Units Written: 1,937,815,661 [992 TB]", 2024-04-07T16:16:09.907 INFO:tasks.workunit.client.0.smithi100.stdout: "Host Read Commands: 5,771,442,418", 2024-04-07T16:16:09.907 INFO:tasks.workunit.client.0.smithi100.stdout: "Host Write Commands: 31,722,089,956", 2024-04-07T16:16:09.907 INFO:tasks.workunit.client.0.smithi100.stdout: "Controller Busy Time: 4", 2024-04-07T16:16:09.907 INFO:tasks.workunit.client.0.smithi100.stdout: "Power Cycles: 45,774", 2024-04-07T16:16:09.907 INFO:tasks.workunit.client.0.smithi100.stdout: "Power On Hours: 57,978", 2024-04-07T16:16:09.907 INFO:tasks.workunit.client.0.smithi100.stdout: "Unsafe Shutdowns: 210", 2024-04-07T16:16:09.907 INFO:tasks.workunit.client.0.smithi100.stdout: "Media and Data Integrity Errors: 0", 2024-04-07T16:16:09.907 INFO:tasks.workunit.client.0.smithi100.stdout: "Error Information Log Entries: 0", 2024-04-07T16:16:09.907 INFO:tasks.workunit.client.0.smithi100.stdout: "", 2024-04-07T16:16:09.907 INFO:tasks.workunit.client.0.smithi100.stdout: "Error Information (NVMe Log 0x01, max 64 entries)", 2024-04-07T16:16:09.908 INFO:tasks.workunit.client.0.smithi100.stdout: "No Errors Logged", 2024-04-07T16:16:09.908 INFO:tasks.workunit.client.0.smithi100.stdout: "" 2024-04-07T16:16:09.908 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:09.908 INFO:tasks.workunit.client.0.smithi100.stdout: "platform_info": "x86_64-linux-5.15.0-101-generic", 2024-04-07T16:16:09.908 INFO:tasks.workunit.client.0.smithi100.stdout: "svn_revision": "5022", 2024-04-07T16:16:09.908 INFO:tasks.workunit.client.0.smithi100.stdout: "version": [ 2024-04-07T16:16:09.908 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T16:16:09.908 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T16:16:09.908 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T16:16:09.908 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:09.908 INFO:tasks.workunit.client.0.smithi100.stdout: "temperature": { 2024-04-07T16:16:09.908 INFO:tasks.workunit.client.0.smithi100.stdout: "current": 24 2024-04-07T16:16:09.908 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:09.908 INFO:tasks.workunit.client.0.smithi100.stdout: "user_capacity": { 2024-04-07T16:16:09.909 INFO:tasks.workunit.client.0.smithi100.stdout: "blocks": 781422768, 2024-04-07T16:16:09.909 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 400088457216 2024-04-07T16:16:09.909 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:09.909 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:09.909 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:09.909 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:09.909 INFO:tasks.workunit.client.0.smithi100.stdout: "25233610-f4fa-11ee-8447-0cc47a8fd733": { 2024-04-07T16:16:09.909 INFO:tasks.workunit.client.0.smithi100.stdout: "INTEL_SSDPEDMD400G4_2524b6c0-f4fa-11ee-8447-0cc47a8fd733": { 2024-04-07T16:16:09.909 INFO:tasks.workunit.client.0.smithi100.stdout: "20240407-160732": { 2024-04-07T16:16:09.909 INFO:tasks.workunit.client.0.smithi100.stdout: "device": { 2024-04-07T16:16:09.909 INFO:tasks.workunit.client.0.smithi100.stdout: "info_name": "/dev/nvme0n1", 2024-04-07T16:16:09.909 INFO:tasks.workunit.client.0.smithi100.stdout: "name": "/dev/nvme0n1", 2024-04-07T16:16:09.909 INFO:tasks.workunit.client.0.smithi100.stdout: "protocol": "NVMe", 2024-04-07T16:16:09.909 INFO:tasks.workunit.client.0.smithi100.stdout: "type": "nvme" 2024-04-07T16:16:09.910 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:09.910 INFO:tasks.workunit.client.0.smithi100.stdout: "firmware_version": "8DV101H0", 2024-04-07T16:16:09.910 INFO:tasks.workunit.client.0.smithi100.stdout: "host_id": "25233610-f4fa-11ee-8447-0cc47a8fd733", 2024-04-07T16:16:09.910 INFO:tasks.workunit.client.0.smithi100.stdout: "json_format_version": [ 2024-04-07T16:16:09.910 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T16:16:09.910 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T16:16:09.910 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:09.910 INFO:tasks.workunit.client.0.smithi100.stdout: "local_time": { 2024-04-07T16:16:09.910 INFO:tasks.workunit.client.0.smithi100.stdout: "asctime": "Sun Apr 7 16:07:32 2024 UTC", 2024-04-07T16:16:09.910 INFO:tasks.workunit.client.0.smithi100.stdout: "time_t": 1712506052 2024-04-07T16:16:09.910 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:09.911 INFO:tasks.workunit.client.0.smithi100.stdout: "logical_block_size": 512, 2024-04-07T16:16:09.911 INFO:tasks.workunit.client.0.smithi100.stdout: "model_name": "INTEL SSDPEDMD400G4", 2024-04-07T16:16:09.911 INFO:tasks.workunit.client.0.smithi100.stdout: "nvme_controller_id": 0, 2024-04-07T16:16:09.911 INFO:tasks.workunit.client.0.smithi100.stdout: "nvme_ieee_oui_identifier": 6083300, 2024-04-07T16:16:09.911 INFO:tasks.workunit.client.0.smithi100.stdout: "nvme_namespaces": [ 2024-04-07T16:16:09.911 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:09.911 INFO:tasks.workunit.client.0.smithi100.stdout: "capacity": { 2024-04-07T16:16:09.911 INFO:tasks.workunit.client.0.smithi100.stdout: "blocks": 781422768, 2024-04-07T16:16:09.911 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 400088457216 2024-04-07T16:16:09.911 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:09.911 INFO:tasks.workunit.client.0.smithi100.stdout: "formatted_lba_size": 512, 2024-04-07T16:16:09.911 INFO:tasks.workunit.client.0.smithi100.stdout: "id": 1, 2024-04-07T16:16:09.912 INFO:tasks.workunit.client.0.smithi100.stdout: "size": { 2024-04-07T16:16:09.912 INFO:tasks.workunit.client.0.smithi100.stdout: "blocks": 781422768, 2024-04-07T16:16:09.912 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 400088457216 2024-04-07T16:16:09.912 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:09.912 INFO:tasks.workunit.client.0.smithi100.stdout: "utilization": { 2024-04-07T16:16:09.912 INFO:tasks.workunit.client.0.smithi100.stdout: "blocks": 781422768, 2024-04-07T16:16:09.912 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 400088457216 2024-04-07T16:16:09.912 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:09.912 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:09.912 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:09.912 INFO:tasks.workunit.client.0.smithi100.stdout: "nvme_number_of_namespaces": 1, 2024-04-07T16:16:09.912 INFO:tasks.workunit.client.0.smithi100.stdout: "nvme_pci_vendor": { 2024-04-07T16:16:09.912 INFO:tasks.workunit.client.0.smithi100.stdout: "id": 32902, 2024-04-07T16:16:09.912 INFO:tasks.workunit.client.0.smithi100.stdout: "subsystem_id": 32902 2024-04-07T16:16:09.913 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:09.913 INFO:tasks.workunit.client.0.smithi100.stdout: "nvme_smart_health_information_add_log": { 2024-04-07T16:16:09.913 INFO:tasks.workunit.client.0.smithi100.stdout: "Device stats": { 2024-04-07T16:16:09.913 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_fw_downgrades": { 2024-04-07T16:16:09.913 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 0, 2024-04-07T16:16:09.913 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 0 2024-04-07T16:16:09.913 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:09.913 INFO:tasks.workunit.client.0.smithi100.stdout: "crc_error_count": { 2024-04-07T16:16:09.913 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 100, 2024-04-07T16:16:09.913 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 0 2024-04-07T16:16:09.913 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:09.913 INFO:tasks.workunit.client.0.smithi100.stdout: "end_to_end_error_detection_count": { 2024-04-07T16:16:09.913 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 100, 2024-04-07T16:16:09.913 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 0 2024-04-07T16:16:09.914 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:09.914 INFO:tasks.workunit.client.0.smithi100.stdout: "erase_fail_count": { 2024-04-07T16:16:09.914 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 100, 2024-04-07T16:16:09.914 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 0 2024-04-07T16:16:09.914 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:09.914 INFO:tasks.workunit.client.0.smithi100.stdout: "host_bytes_written": { 2024-04-07T16:16:09.914 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 100, 2024-04-07T16:16:09.914 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 25060599 2024-04-07T16:16:09.914 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:09.914 INFO:tasks.workunit.client.0.smithi100.stdout: "host_ctx_wear_used": { 2024-04-07T16:16:09.914 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 100, 2024-04-07T16:16:09.914 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 1 2024-04-07T16:16:09.914 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:09.914 INFO:tasks.workunit.client.0.smithi100.stdout: "media_bytes_read": { 2024-04-07T16:16:09.915 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 0, 2024-04-07T16:16:09.915 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 0 2024-04-07T16:16:09.915 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:09.915 INFO:tasks.workunit.client.0.smithi100.stdout: "nand_bytes_written": { 2024-04-07T16:16:09.915 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 100, 2024-04-07T16:16:09.915 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 42751216 2024-04-07T16:16:09.915 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:09.915 INFO:tasks.workunit.client.0.smithi100.stdout: "perf_stat_indicator": { 2024-04-07T16:16:09.915 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 0, 2024-04-07T16:16:09.915 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 0 2024-04-07T16:16:09.915 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:09.915 INFO:tasks.workunit.client.0.smithi100.stdout: "pll_lock_loss_count": { 2024-04-07T16:16:09.915 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 100, 2024-04-07T16:16:09.915 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 0 2024-04-07T16:16:09.916 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:09.916 INFO:tasks.workunit.client.0.smithi100.stdout: "program_fail_count": { 2024-04-07T16:16:09.916 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 100, 2024-04-07T16:16:09.916 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 0 2024-04-07T16:16:09.916 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:09.916 INFO:tasks.workunit.client.0.smithi100.stdout: "re_alloc_sectr_cnt": { 2024-04-07T16:16:09.916 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 0, 2024-04-07T16:16:09.916 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 0 2024-04-07T16:16:09.916 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:09.917 INFO:tasks.workunit.client.0.smithi100.stdout: "retry_buffer_overflow_count": { 2024-04-07T16:16:09.917 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 100, 2024-04-07T16:16:09.917 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 0 2024-04-07T16:16:09.917 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:09.917 INFO:tasks.workunit.client.0.smithi100.stdout: "soft_ecc_err_rate": { 2024-04-07T16:16:09.917 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 0, 2024-04-07T16:16:09.917 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 0 2024-04-07T16:16:09.917 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:09.917 INFO:tasks.workunit.client.0.smithi100.stdout: "thermal_throttle_status": { 2024-04-07T16:16:09.917 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 100, 2024-04-07T16:16:09.917 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": { 2024-04-07T16:16:09.917 INFO:tasks.workunit.client.0.smithi100.stdout: "cnt": 0, 2024-04-07T16:16:09.918 INFO:tasks.workunit.client.0.smithi100.stdout: "pct": 0 2024-04-07T16:16:09.918 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:09.918 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:09.918 INFO:tasks.workunit.client.0.smithi100.stdout: "timed_workload_host_reads": { 2024-04-07T16:16:09.918 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 100, 2024-04-07T16:16:09.918 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 65535 2024-04-07T16:16:09.918 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:09.918 INFO:tasks.workunit.client.0.smithi100.stdout: "timed_workload_media_wear": { 2024-04-07T16:16:09.918 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 100, 2024-04-07T16:16:09.918 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 64 2024-04-07T16:16:09.918 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:09.918 INFO:tasks.workunit.client.0.smithi100.stdout: "timed_workload_timer": { 2024-04-07T16:16:09.919 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 100, 2024-04-07T16:16:09.919 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 65535 2024-04-07T16:16:09.919 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:09.919 INFO:tasks.workunit.client.0.smithi100.stdout: "unexp_power_loss": { 2024-04-07T16:16:09.919 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 0, 2024-04-07T16:16:09.919 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 0 2024-04-07T16:16:09.919 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:09.919 INFO:tasks.workunit.client.0.smithi100.stdout: "wear_leveling": { 2024-04-07T16:16:09.919 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 91, 2024-04-07T16:16:09.919 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": { 2024-04-07T16:16:09.919 INFO:tasks.workunit.client.0.smithi100.stdout: "avg": 2895, 2024-04-07T16:16:09.919 INFO:tasks.workunit.client.0.smithi100.stdout: "max": 2907, 2024-04-07T16:16:09.919 INFO:tasks.workunit.client.0.smithi100.stdout: "min": 2884 2024-04-07T16:16:09.920 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:09.920 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:09.920 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:09.920 INFO:tasks.workunit.client.0.smithi100.stdout: "Intel Smart log": "nvme0n1" 2024-04-07T16:16:09.920 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:09.920 INFO:tasks.workunit.client.0.smithi100.stdout: "nvme_smart_health_information_log": { 2024-04-07T16:16:09.920 INFO:tasks.workunit.client.0.smithi100.stdout: "available_spare": 100, 2024-04-07T16:16:09.920 INFO:tasks.workunit.client.0.smithi100.stdout: "available_spare_threshold": 10, 2024-04-07T16:16:09.920 INFO:tasks.workunit.client.0.smithi100.stdout: "controller_busy_time": 0, 2024-04-07T16:16:09.920 INFO:tasks.workunit.client.0.smithi100.stdout: "critical_warning": 0, 2024-04-07T16:16:09.920 INFO:tasks.workunit.client.0.smithi100.stdout: "data_units_read": 359011585, 2024-04-07T16:16:09.921 INFO:tasks.workunit.client.0.smithi100.stdout: "data_units_written": 1642371481, 2024-04-07T16:16:09.921 INFO:tasks.workunit.client.0.smithi100.stdout: "host_reads": 5750712954, 2024-04-07T16:16:09.921 INFO:tasks.workunit.client.0.smithi100.stdout: "host_writes": 28276536650, 2024-04-07T16:16:09.921 INFO:tasks.workunit.client.0.smithi100.stdout: "media_errors": 0, 2024-04-07T16:16:09.921 INFO:tasks.workunit.client.0.smithi100.stdout: "num_err_log_entries": 0, 2024-04-07T16:16:09.921 INFO:tasks.workunit.client.0.smithi100.stdout: "percentage_used": 9, 2024-04-07T16:16:09.921 INFO:tasks.workunit.client.0.smithi100.stdout: "power_cycles": 47059, 2024-04-07T16:16:09.921 INFO:tasks.workunit.client.0.smithi100.stdout: "power_on_hours": 54821, 2024-04-07T16:16:09.921 INFO:tasks.workunit.client.0.smithi100.stdout: "temperature": 25, 2024-04-07T16:16:09.921 INFO:tasks.workunit.client.0.smithi100.stdout: "unsafe_shutdowns": 12 2024-04-07T16:16:09.921 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:09.921 INFO:tasks.workunit.client.0.smithi100.stdout: "nvme_vendor": "intel", 2024-04-07T16:16:09.922 INFO:tasks.workunit.client.0.smithi100.stdout: "power_cycle_count": 47059, 2024-04-07T16:16:09.922 INFO:tasks.workunit.client.0.smithi100.stdout: "power_on_time": { 2024-04-07T16:16:09.922 INFO:tasks.workunit.client.0.smithi100.stdout: "hours": 54821 2024-04-07T16:16:09.922 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:09.922 INFO:tasks.workunit.client.0.smithi100.stdout: "serial_number": "deleted", 2024-04-07T16:16:09.922 INFO:tasks.workunit.client.0.smithi100.stdout: "smart_status": { 2024-04-07T16:16:09.922 INFO:tasks.workunit.client.0.smithi100.stdout: "nvme": { 2024-04-07T16:16:09.922 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:09.922 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:09.922 INFO:tasks.workunit.client.0.smithi100.stdout: "passed": true 2024-04-07T16:16:09.922 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:09.922 INFO:tasks.workunit.client.0.smithi100.stdout: "smartctl": { 2024-04-07T16:16:09.923 INFO:tasks.workunit.client.0.smithi100.stdout: "argv": [ 2024-04-07T16:16:09.923 INFO:tasks.workunit.client.0.smithi100.stdout: "smartctl", 2024-04-07T16:16:09.923 INFO:tasks.workunit.client.0.smithi100.stdout: "-x", 2024-04-07T16:16:09.923 INFO:tasks.workunit.client.0.smithi100.stdout: "--json=o", 2024-04-07T16:16:09.923 INFO:tasks.workunit.client.0.smithi100.stdout: "/dev/nvme0n1" 2024-04-07T16:16:09.923 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:09.923 INFO:tasks.workunit.client.0.smithi100.stdout: "build_info": "(local build)", 2024-04-07T16:16:09.923 INFO:tasks.workunit.client.0.smithi100.stdout: "exit_status": 0, 2024-04-07T16:16:09.923 INFO:tasks.workunit.client.0.smithi100.stdout: "output": [ 2024-04-07T16:16:09.923 INFO:tasks.workunit.client.0.smithi100.stdout: "smartctl 7.1 2019-12-30 r5022 [x86_64-linux-5.15.0-101-generic] (local build)", 2024-04-07T16:16:09.923 INFO:tasks.workunit.client.0.smithi100.stdout: "Copyright (C) 2002-19, Bruce Allen, Christian Franke, www.smartmontools.org", 2024-04-07T16:16:09.923 INFO:tasks.workunit.client.0.smithi100.stdout: "", 2024-04-07T16:16:09.923 INFO:tasks.workunit.client.0.smithi100.stdout: "=== START OF INFORMATION SECTION ===", 2024-04-07T16:16:09.923 INFO:tasks.workunit.client.0.smithi100.stdout: "Model Number: INTEL SSDPEDMD400G4", 2024-04-07T16:16:09.924 INFO:tasks.workunit.client.0.smithi100.stdout: "Serial Number: deleted", 2024-04-07T16:16:09.924 INFO:tasks.workunit.client.0.smithi100.stdout: "Firmware Version: 8DV101H0", 2024-04-07T16:16:09.924 INFO:tasks.workunit.client.0.smithi100.stdout: "PCI Vendor/Subsystem ID: 0x8086", 2024-04-07T16:16:09.924 INFO:tasks.workunit.client.0.smithi100.stdout: "IEEE OUI Identifier: 0x5cd2e4", 2024-04-07T16:16:09.924 INFO:tasks.workunit.client.0.smithi100.stdout: "Controller ID: 0", 2024-04-07T16:16:09.924 INFO:tasks.workunit.client.0.smithi100.stdout: "Number of Namespaces: 1", 2024-04-07T16:16:09.924 INFO:tasks.workunit.client.0.smithi100.stdout: "Namespace 1 Size/Capacity: 400,088,457,216 [400 GB]", 2024-04-07T16:16:09.924 INFO:tasks.workunit.client.0.smithi100.stdout: "Namespace 1 Formatted LBA Size: 512", 2024-04-07T16:16:09.924 INFO:tasks.workunit.client.0.smithi100.stdout: "Local Time is: Sun Apr 7 16:07:32 2024 UTC", 2024-04-07T16:16:09.924 INFO:tasks.workunit.client.0.smithi100.stdout: "Firmware Updates (0x02): 1 Slot", 2024-04-07T16:16:09.924 INFO:tasks.workunit.client.0.smithi100.stdout: "Optional Admin Commands (0x0006): Format Frmw_DL", 2024-04-07T16:16:09.924 INFO:tasks.workunit.client.0.smithi100.stdout: "Optional NVM Commands (0x0006): Wr_Unc DS_Mngmt", 2024-04-07T16:16:09.924 INFO:tasks.workunit.client.0.smithi100.stdout: "Maximum Data Transfer Size: 32 Pages", 2024-04-07T16:16:09.924 INFO:tasks.workunit.client.0.smithi100.stdout: "", 2024-04-07T16:16:09.925 INFO:tasks.workunit.client.0.smithi100.stdout: "Supported Power States", 2024-04-07T16:16:09.925 INFO:tasks.workunit.client.0.smithi100.stdout: "St Op Max Active Idle RL RT WL WT Ent_Lat Ex_Lat", 2024-04-07T16:16:09.925 INFO:tasks.workunit.client.0.smithi100.stdout: " 0 + 25.00W - - 0 0 0 0 0 0", 2024-04-07T16:16:09.925 INFO:tasks.workunit.client.0.smithi100.stdout: "", 2024-04-07T16:16:09.925 INFO:tasks.workunit.client.0.smithi100.stdout: "Supported LBA Sizes (NSID 0x1)", 2024-04-07T16:16:09.925 INFO:tasks.workunit.client.0.smithi100.stdout: "Id Fmt Data Metadt Rel_Perf", 2024-04-07T16:16:09.925 INFO:tasks.workunit.client.0.smithi100.stdout: " 0 + 512 0 2", 2024-04-07T16:16:09.925 INFO:tasks.workunit.client.0.smithi100.stdout: " 1 - 512 8 2", 2024-04-07T16:16:09.925 INFO:tasks.workunit.client.0.smithi100.stdout: " 2 - 512 16 2", 2024-04-07T16:16:09.925 INFO:tasks.workunit.client.0.smithi100.stdout: " 3 - 4096 0 0", 2024-04-07T16:16:09.925 INFO:tasks.workunit.client.0.smithi100.stdout: " 4 - 4096 8 0", 2024-04-07T16:16:09.925 INFO:tasks.workunit.client.0.smithi100.stdout: " 5 - 4096 64 0", 2024-04-07T16:16:09.925 INFO:tasks.workunit.client.0.smithi100.stdout: " 6 - 4096 128 0", 2024-04-07T16:16:09.925 INFO:tasks.workunit.client.0.smithi100.stdout: "", 2024-04-07T16:16:09.926 INFO:tasks.workunit.client.0.smithi100.stdout: "=== START OF SMART DATA SECTION ===", 2024-04-07T16:16:09.926 INFO:tasks.workunit.client.0.smithi100.stdout: "SMART overall-health self-assessment test result: PASSED", 2024-04-07T16:16:09.926 INFO:tasks.workunit.client.0.smithi100.stdout: "", 2024-04-07T16:16:09.926 INFO:tasks.workunit.client.0.smithi100.stdout: "SMART/Health Information (NVMe Log 0x02)", 2024-04-07T16:16:09.926 INFO:tasks.workunit.client.0.smithi100.stdout: "Critical Warning: 0x00", 2024-04-07T16:16:09.926 INFO:tasks.workunit.client.0.smithi100.stdout: "Temperature: 25 Celsius", 2024-04-07T16:16:09.926 INFO:tasks.workunit.client.0.smithi100.stdout: "Available Spare: 100%", 2024-04-07T16:16:09.926 INFO:tasks.workunit.client.0.smithi100.stdout: "Available Spare Threshold: 10%", 2024-04-07T16:16:09.926 INFO:tasks.workunit.client.0.smithi100.stdout: "Percentage Used: 9%", 2024-04-07T16:16:09.926 INFO:tasks.workunit.client.0.smithi100.stdout: "Data Units Read: 359,011,585 [183 TB]", 2024-04-07T16:16:09.926 INFO:tasks.workunit.client.0.smithi100.stdout: "Data Units Written: 1,642,371,481 [840 TB]", 2024-04-07T16:16:09.926 INFO:tasks.workunit.client.0.smithi100.stdout: "Host Read Commands: 5,750,712,954", 2024-04-07T16:16:09.926 INFO:tasks.workunit.client.0.smithi100.stdout: "Host Write Commands: 28,276,536,650", 2024-04-07T16:16:09.926 INFO:tasks.workunit.client.0.smithi100.stdout: "Controller Busy Time: 0", 2024-04-07T16:16:09.927 INFO:tasks.workunit.client.0.smithi100.stdout: "Power Cycles: 47,059", 2024-04-07T16:16:09.927 INFO:tasks.workunit.client.0.smithi100.stdout: "Power On Hours: 54,821", 2024-04-07T16:16:09.927 INFO:tasks.workunit.client.0.smithi100.stdout: "Unsafe Shutdowns: 12", 2024-04-07T16:16:09.927 INFO:tasks.workunit.client.0.smithi100.stdout: "Media and Data Integrity Errors: 0", 2024-04-07T16:16:09.927 INFO:tasks.workunit.client.0.smithi100.stdout: "Error Information Log Entries: 0", 2024-04-07T16:16:09.927 INFO:tasks.workunit.client.0.smithi100.stdout: "", 2024-04-07T16:16:09.927 INFO:tasks.workunit.client.0.smithi100.stdout: "Error Information (NVMe Log 0x01, max 64 entries)", 2024-04-07T16:16:09.927 INFO:tasks.workunit.client.0.smithi100.stdout: "No Errors Logged", 2024-04-07T16:16:09.927 INFO:tasks.workunit.client.0.smithi100.stdout: "" 2024-04-07T16:16:09.927 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:09.927 INFO:tasks.workunit.client.0.smithi100.stdout: "platform_info": "x86_64-linux-5.15.0-101-generic", 2024-04-07T16:16:09.927 INFO:tasks.workunit.client.0.smithi100.stdout: "svn_revision": "5022", 2024-04-07T16:16:09.927 INFO:tasks.workunit.client.0.smithi100.stdout: "version": [ 2024-04-07T16:16:09.927 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T16:16:09.928 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T16:16:09.928 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T16:16:09.928 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:09.928 INFO:tasks.workunit.client.0.smithi100.stdout: "temperature": { 2024-04-07T16:16:09.928 INFO:tasks.workunit.client.0.smithi100.stdout: "current": 25 2024-04-07T16:16:09.928 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:09.928 INFO:tasks.workunit.client.0.smithi100.stdout: "user_capacity": { 2024-04-07T16:16:09.928 INFO:tasks.workunit.client.0.smithi100.stdout: "blocks": 781422768, 2024-04-07T16:16:09.928 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 400088457216 2024-04-07T16:16:09.928 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:09.928 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:09.928 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:09.928 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:09.928 INFO:tasks.workunit.client.0.smithi100.stdout:} 2024-04-07T16:16:09.929 INFO:tasks.workunit.client.0.smithi100.stderr:+ ceph telemetry preview-all 2024-04-07T16:16:10.494 INFO:tasks.workunit.client.0.smithi100.stdout:{ 2024-04-07T16:16:10.494 INFO:tasks.workunit.client.0.smithi100.stdout: "device_report": { 2024-04-07T16:16:10.494 INFO:tasks.workunit.client.0.smithi100.stdout: "251e7b66-f4fa-11ee-8447-0cc47a8fd733": { 2024-04-07T16:16:10.494 INFO:tasks.workunit.client.0.smithi100.stdout: "INTEL_SSDPEDMD400G4_251ffd38-f4fa-11ee-8447-0cc47a8fd733": { 2024-04-07T16:16:10.494 INFO:tasks.workunit.client.0.smithi100.stdout: "20240407-160729": { 2024-04-07T16:16:10.495 INFO:tasks.workunit.client.0.smithi100.stdout: "device": { 2024-04-07T16:16:10.495 INFO:tasks.workunit.client.0.smithi100.stdout: "info_name": "/dev/nvme0n1", 2024-04-07T16:16:10.495 INFO:tasks.workunit.client.0.smithi100.stdout: "name": "/dev/nvme0n1", 2024-04-07T16:16:10.495 INFO:tasks.workunit.client.0.smithi100.stdout: "protocol": "NVMe", 2024-04-07T16:16:10.495 INFO:tasks.workunit.client.0.smithi100.stdout: "type": "nvme" 2024-04-07T16:16:10.495 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:10.495 INFO:tasks.workunit.client.0.smithi100.stdout: "firmware_version": "8DV101H0", 2024-04-07T16:16:10.495 INFO:tasks.workunit.client.0.smithi100.stdout: "host_id": "251e7b66-f4fa-11ee-8447-0cc47a8fd733", 2024-04-07T16:16:10.495 INFO:tasks.workunit.client.0.smithi100.stdout: "json_format_version": [ 2024-04-07T16:16:10.495 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T16:16:10.496 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T16:16:10.496 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:10.496 INFO:tasks.workunit.client.0.smithi100.stdout: "local_time": { 2024-04-07T16:16:10.496 INFO:tasks.workunit.client.0.smithi100.stdout: "asctime": "Sun Apr 7 16:07:28 2024 UTC", 2024-04-07T16:16:10.496 INFO:tasks.workunit.client.0.smithi100.stdout: "time_t": 1712506048 2024-04-07T16:16:10.496 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:10.497 INFO:tasks.workunit.client.0.smithi100.stdout: "logical_block_size": 512, 2024-04-07T16:16:10.497 INFO:tasks.workunit.client.0.smithi100.stdout: "model_name": "INTEL SSDPEDMD400G4", 2024-04-07T16:16:10.497 INFO:tasks.workunit.client.0.smithi100.stdout: "nvme_controller_id": 0, 2024-04-07T16:16:10.497 INFO:tasks.workunit.client.0.smithi100.stdout: "nvme_ieee_oui_identifier": 6083300, 2024-04-07T16:16:10.497 INFO:tasks.workunit.client.0.smithi100.stdout: "nvme_namespaces": [ 2024-04-07T16:16:10.497 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:10.497 INFO:tasks.workunit.client.0.smithi100.stdout: "capacity": { 2024-04-07T16:16:10.497 INFO:tasks.workunit.client.0.smithi100.stdout: "blocks": 781422768, 2024-04-07T16:16:10.497 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 400088457216 2024-04-07T16:16:10.497 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:10.497 INFO:tasks.workunit.client.0.smithi100.stdout: "formatted_lba_size": 512, 2024-04-07T16:16:10.497 INFO:tasks.workunit.client.0.smithi100.stdout: "id": 1, 2024-04-07T16:16:10.498 INFO:tasks.workunit.client.0.smithi100.stdout: "size": { 2024-04-07T16:16:10.498 INFO:tasks.workunit.client.0.smithi100.stdout: "blocks": 781422768, 2024-04-07T16:16:10.498 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 400088457216 2024-04-07T16:16:10.498 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:10.498 INFO:tasks.workunit.client.0.smithi100.stdout: "utilization": { 2024-04-07T16:16:10.498 INFO:tasks.workunit.client.0.smithi100.stdout: "blocks": 781422768, 2024-04-07T16:16:10.498 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 400088457216 2024-04-07T16:16:10.498 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:10.498 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:10.498 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:10.498 INFO:tasks.workunit.client.0.smithi100.stdout: "nvme_number_of_namespaces": 1, 2024-04-07T16:16:10.498 INFO:tasks.workunit.client.0.smithi100.stdout: "nvme_pci_vendor": { 2024-04-07T16:16:10.498 INFO:tasks.workunit.client.0.smithi100.stdout: "id": 32902, 2024-04-07T16:16:10.499 INFO:tasks.workunit.client.0.smithi100.stdout: "subsystem_id": 32902 2024-04-07T16:16:10.499 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:10.499 INFO:tasks.workunit.client.0.smithi100.stdout: "nvme_smart_health_information_add_log": { 2024-04-07T16:16:10.499 INFO:tasks.workunit.client.0.smithi100.stdout: "Device stats": { 2024-04-07T16:16:10.499 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_fw_downgrades": { 2024-04-07T16:16:10.499 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 0, 2024-04-07T16:16:10.500 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 0 2024-04-07T16:16:10.500 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:10.500 INFO:tasks.workunit.client.0.smithi100.stdout: "crc_error_count": { 2024-04-07T16:16:10.500 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 100, 2024-04-07T16:16:10.500 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 0 2024-04-07T16:16:10.500 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:10.500 INFO:tasks.workunit.client.0.smithi100.stdout: "end_to_end_error_detection_count": { 2024-04-07T16:16:10.501 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 100, 2024-04-07T16:16:10.501 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 0 2024-04-07T16:16:10.501 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:10.501 INFO:tasks.workunit.client.0.smithi100.stdout: "erase_fail_count": { 2024-04-07T16:16:10.501 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 100, 2024-04-07T16:16:10.501 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 0 2024-04-07T16:16:10.501 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:10.502 INFO:tasks.workunit.client.0.smithi100.stdout: "host_bytes_written": { 2024-04-07T16:16:10.502 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 100, 2024-04-07T16:16:10.502 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 29568720 2024-04-07T16:16:10.502 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:10.502 INFO:tasks.workunit.client.0.smithi100.stdout: "host_ctx_wear_used": { 2024-04-07T16:16:10.502 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 100, 2024-04-07T16:16:10.503 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 1 2024-04-07T16:16:10.503 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:10.503 INFO:tasks.workunit.client.0.smithi100.stdout: "media_bytes_read": { 2024-04-07T16:16:10.503 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 0, 2024-04-07T16:16:10.503 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 0 2024-04-07T16:16:10.503 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:10.504 INFO:tasks.workunit.client.0.smithi100.stdout: "nand_bytes_written": { 2024-04-07T16:16:10.504 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 100, 2024-04-07T16:16:10.504 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 47802746 2024-04-07T16:16:10.504 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:10.504 INFO:tasks.workunit.client.0.smithi100.stdout: "perf_stat_indicator": { 2024-04-07T16:16:10.504 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 0, 2024-04-07T16:16:10.504 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 0 2024-04-07T16:16:10.505 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:10.505 INFO:tasks.workunit.client.0.smithi100.stdout: "pll_lock_loss_count": { 2024-04-07T16:16:10.505 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 100, 2024-04-07T16:16:10.505 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 0 2024-04-07T16:16:10.505 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:10.505 INFO:tasks.workunit.client.0.smithi100.stdout: "program_fail_count": { 2024-04-07T16:16:10.506 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 100, 2024-04-07T16:16:10.506 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 0 2024-04-07T16:16:10.506 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:10.506 INFO:tasks.workunit.client.0.smithi100.stdout: "re_alloc_sectr_cnt": { 2024-04-07T16:16:10.506 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 0, 2024-04-07T16:16:10.506 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 0 2024-04-07T16:16:10.507 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:10.507 INFO:tasks.workunit.client.0.smithi100.stdout: "retry_buffer_overflow_count": { 2024-04-07T16:16:10.507 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 100, 2024-04-07T16:16:10.507 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 0 2024-04-07T16:16:10.507 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:10.507 INFO:tasks.workunit.client.0.smithi100.stdout: "soft_ecc_err_rate": { 2024-04-07T16:16:10.508 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 0, 2024-04-07T16:16:10.508 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 0 2024-04-07T16:16:10.508 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:10.508 INFO:tasks.workunit.client.0.smithi100.stdout: "thermal_throttle_status": { 2024-04-07T16:16:10.508 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 100, 2024-04-07T16:16:10.508 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": { 2024-04-07T16:16:10.509 INFO:tasks.workunit.client.0.smithi100.stdout: "cnt": 0, 2024-04-07T16:16:10.509 INFO:tasks.workunit.client.0.smithi100.stdout: "pct": 0 2024-04-07T16:16:10.509 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:10.509 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:10.509 INFO:tasks.workunit.client.0.smithi100.stdout: "timed_workload_host_reads": { 2024-04-07T16:16:10.509 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 100, 2024-04-07T16:16:10.509 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 65535 2024-04-07T16:16:10.510 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:10.510 INFO:tasks.workunit.client.0.smithi100.stdout: "timed_workload_media_wear": { 2024-04-07T16:16:10.510 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 100, 2024-04-07T16:16:10.510 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 64 2024-04-07T16:16:10.510 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:10.510 INFO:tasks.workunit.client.0.smithi100.stdout: "timed_workload_timer": { 2024-04-07T16:16:10.511 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 100, 2024-04-07T16:16:10.511 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 65535 2024-04-07T16:16:10.511 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:10.511 INFO:tasks.workunit.client.0.smithi100.stdout: "unexp_power_loss": { 2024-04-07T16:16:10.511 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 0, 2024-04-07T16:16:10.511 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 0 2024-04-07T16:16:10.512 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:10.512 INFO:tasks.workunit.client.0.smithi100.stdout: "wear_leveling": { 2024-04-07T16:16:10.512 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 90, 2024-04-07T16:16:10.512 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": { 2024-04-07T16:16:10.512 INFO:tasks.workunit.client.0.smithi100.stdout: "avg": 3209, 2024-04-07T16:16:10.512 INFO:tasks.workunit.client.0.smithi100.stdout: "max": 3225, 2024-04-07T16:16:10.513 INFO:tasks.workunit.client.0.smithi100.stdout: "min": 3194 2024-04-07T16:16:10.513 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:10.513 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:10.513 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:10.513 INFO:tasks.workunit.client.0.smithi100.stdout: "Intel Smart log": "nvme0n1" 2024-04-07T16:16:10.513 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:10.513 INFO:tasks.workunit.client.0.smithi100.stdout: "nvme_smart_health_information_log": { 2024-04-07T16:16:10.514 INFO:tasks.workunit.client.0.smithi100.stdout: "available_spare": 100, 2024-04-07T16:16:10.514 INFO:tasks.workunit.client.0.smithi100.stdout: "available_spare_threshold": 10, 2024-04-07T16:16:10.514 INFO:tasks.workunit.client.0.smithi100.stdout: "controller_busy_time": 4, 2024-04-07T16:16:10.514 INFO:tasks.workunit.client.0.smithi100.stdout: "critical_warning": 0, 2024-04-07T16:16:10.514 INFO:tasks.workunit.client.0.smithi100.stdout: "data_units_read": 385784931, 2024-04-07T16:16:10.514 INFO:tasks.workunit.client.0.smithi100.stdout: "data_units_written": 1937815661, 2024-04-07T16:16:10.515 INFO:tasks.workunit.client.0.smithi100.stdout: "host_reads": 5771442418, 2024-04-07T16:16:10.515 INFO:tasks.workunit.client.0.smithi100.stdout: "host_writes": 31722089956, 2024-04-07T16:16:10.515 INFO:tasks.workunit.client.0.smithi100.stdout: "media_errors": 0, 2024-04-07T16:16:10.515 INFO:tasks.workunit.client.0.smithi100.stdout: "num_err_log_entries": 0, 2024-04-07T16:16:10.515 INFO:tasks.workunit.client.0.smithi100.stdout: "percentage_used": 10, 2024-04-07T16:16:10.515 INFO:tasks.workunit.client.0.smithi100.stdout: "power_cycles": 45774, 2024-04-07T16:16:10.516 INFO:tasks.workunit.client.0.smithi100.stdout: "power_on_hours": 57978, 2024-04-07T16:16:10.516 INFO:tasks.workunit.client.0.smithi100.stdout: "temperature": 24, 2024-04-07T16:16:10.516 INFO:tasks.workunit.client.0.smithi100.stdout: "unsafe_shutdowns": 210 2024-04-07T16:16:10.516 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:10.516 INFO:tasks.workunit.client.0.smithi100.stdout: "nvme_vendor": "intel", 2024-04-07T16:16:10.516 INFO:tasks.workunit.client.0.smithi100.stdout: "power_cycle_count": 45774, 2024-04-07T16:16:10.517 INFO:tasks.workunit.client.0.smithi100.stdout: "power_on_time": { 2024-04-07T16:16:10.517 INFO:tasks.workunit.client.0.smithi100.stdout: "hours": 57978 2024-04-07T16:16:10.517 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:10.517 INFO:tasks.workunit.client.0.smithi100.stdout: "serial_number": "deleted", 2024-04-07T16:16:10.517 INFO:tasks.workunit.client.0.smithi100.stdout: "smart_status": { 2024-04-07T16:16:10.517 INFO:tasks.workunit.client.0.smithi100.stdout: "nvme": { 2024-04-07T16:16:10.518 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:10.518 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:10.518 INFO:tasks.workunit.client.0.smithi100.stdout: "passed": true 2024-04-07T16:16:10.518 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:10.518 INFO:tasks.workunit.client.0.smithi100.stdout: "smartctl": { 2024-04-07T16:16:10.518 INFO:tasks.workunit.client.0.smithi100.stdout: "argv": [ 2024-04-07T16:16:10.519 INFO:tasks.workunit.client.0.smithi100.stdout: "smartctl", 2024-04-07T16:16:10.519 INFO:tasks.workunit.client.0.smithi100.stdout: "-x", 2024-04-07T16:16:10.519 INFO:tasks.workunit.client.0.smithi100.stdout: "--json=o", 2024-04-07T16:16:10.519 INFO:tasks.workunit.client.0.smithi100.stdout: "/dev/nvme0n1" 2024-04-07T16:16:10.519 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:10.519 INFO:tasks.workunit.client.0.smithi100.stdout: "build_info": "(local build)", 2024-04-07T16:16:10.520 INFO:tasks.workunit.client.0.smithi100.stdout: "exit_status": 0, 2024-04-07T16:16:10.520 INFO:tasks.workunit.client.0.smithi100.stdout: "output": [ 2024-04-07T16:16:10.520 INFO:tasks.workunit.client.0.smithi100.stdout: "smartctl 7.1 2019-12-30 r5022 [x86_64-linux-5.15.0-101-generic] (local build)", 2024-04-07T16:16:10.520 INFO:tasks.workunit.client.0.smithi100.stdout: "Copyright (C) 2002-19, Bruce Allen, Christian Franke, www.smartmontools.org", 2024-04-07T16:16:10.520 INFO:tasks.workunit.client.0.smithi100.stdout: "", 2024-04-07T16:16:10.520 INFO:tasks.workunit.client.0.smithi100.stdout: "=== START OF INFORMATION SECTION ===", 2024-04-07T16:16:10.521 INFO:tasks.workunit.client.0.smithi100.stdout: "Model Number: INTEL SSDPEDMD400G4", 2024-04-07T16:16:10.521 INFO:tasks.workunit.client.0.smithi100.stdout: "Serial Number: deleted", 2024-04-07T16:16:10.521 INFO:tasks.workunit.client.0.smithi100.stdout: "Firmware Version: 8DV101H0", 2024-04-07T16:16:10.521 INFO:tasks.workunit.client.0.smithi100.stdout: "PCI Vendor/Subsystem ID: 0x8086", 2024-04-07T16:16:10.521 INFO:tasks.workunit.client.0.smithi100.stdout: "IEEE OUI Identifier: 0x5cd2e4", 2024-04-07T16:16:10.522 INFO:tasks.workunit.client.0.smithi100.stdout: "Controller ID: 0", 2024-04-07T16:16:10.522 INFO:tasks.workunit.client.0.smithi100.stdout: "Number of Namespaces: 1", 2024-04-07T16:16:10.522 INFO:tasks.workunit.client.0.smithi100.stdout: "Namespace 1 Size/Capacity: 400,088,457,216 [400 GB]", 2024-04-07T16:16:10.522 INFO:tasks.workunit.client.0.smithi100.stdout: "Namespace 1 Formatted LBA Size: 512", 2024-04-07T16:16:10.522 INFO:tasks.workunit.client.0.smithi100.stdout: "Local Time is: Sun Apr 7 16:07:28 2024 UTC", 2024-04-07T16:16:10.522 INFO:tasks.workunit.client.0.smithi100.stdout: "Firmware Updates (0x02): 1 Slot", 2024-04-07T16:16:10.523 INFO:tasks.workunit.client.0.smithi100.stdout: "Optional Admin Commands (0x0006): Format Frmw_DL", 2024-04-07T16:16:10.523 INFO:tasks.workunit.client.0.smithi100.stdout: "Optional NVM Commands (0x0006): Wr_Unc DS_Mngmt", 2024-04-07T16:16:10.523 INFO:tasks.workunit.client.0.smithi100.stdout: "Maximum Data Transfer Size: 32 Pages", 2024-04-07T16:16:10.523 INFO:tasks.workunit.client.0.smithi100.stdout: "", 2024-04-07T16:16:10.523 INFO:tasks.workunit.client.0.smithi100.stdout: "Supported Power States", 2024-04-07T16:16:10.524 INFO:tasks.workunit.client.0.smithi100.stdout: "St Op Max Active Idle RL RT WL WT Ent_Lat Ex_Lat", 2024-04-07T16:16:10.524 INFO:tasks.workunit.client.0.smithi100.stdout: " 0 + 25.00W - - 0 0 0 0 0 0", 2024-04-07T16:16:10.524 INFO:tasks.workunit.client.0.smithi100.stdout: "", 2024-04-07T16:16:10.524 INFO:tasks.workunit.client.0.smithi100.stdout: "Supported LBA Sizes (NSID 0x1)", 2024-04-07T16:16:10.524 INFO:tasks.workunit.client.0.smithi100.stdout: "Id Fmt Data Metadt Rel_Perf", 2024-04-07T16:16:10.524 INFO:tasks.workunit.client.0.smithi100.stdout: " 0 + 512 0 2", 2024-04-07T16:16:10.524 INFO:tasks.workunit.client.0.smithi100.stdout: " 1 - 512 8 2", 2024-04-07T16:16:10.525 INFO:tasks.workunit.client.0.smithi100.stdout: " 2 - 512 16 2", 2024-04-07T16:16:10.525 INFO:tasks.workunit.client.0.smithi100.stdout: " 3 - 4096 0 0", 2024-04-07T16:16:10.525 INFO:tasks.workunit.client.0.smithi100.stdout: " 4 - 4096 8 0", 2024-04-07T16:16:10.525 INFO:tasks.workunit.client.0.smithi100.stdout: " 5 - 4096 64 0", 2024-04-07T16:16:10.525 INFO:tasks.workunit.client.0.smithi100.stdout: " 6 - 4096 128 0", 2024-04-07T16:16:10.525 INFO:tasks.workunit.client.0.smithi100.stdout: "", 2024-04-07T16:16:10.526 INFO:tasks.workunit.client.0.smithi100.stdout: "=== START OF SMART DATA SECTION ===", 2024-04-07T16:16:10.526 INFO:tasks.workunit.client.0.smithi100.stdout: "SMART overall-health self-assessment test result: PASSED", 2024-04-07T16:16:10.526 INFO:tasks.workunit.client.0.smithi100.stdout: "", 2024-04-07T16:16:10.526 INFO:tasks.workunit.client.0.smithi100.stdout: "SMART/Health Information (NVMe Log 0x02)", 2024-04-07T16:16:10.526 INFO:tasks.workunit.client.0.smithi100.stdout: "Critical Warning: 0x00", 2024-04-07T16:16:10.526 INFO:tasks.workunit.client.0.smithi100.stdout: "Temperature: 24 Celsius", 2024-04-07T16:16:10.527 INFO:tasks.workunit.client.0.smithi100.stdout: "Available Spare: 100%", 2024-04-07T16:16:10.527 INFO:tasks.workunit.client.0.smithi100.stdout: "Available Spare Threshold: 10%", 2024-04-07T16:16:10.527 INFO:tasks.workunit.client.0.smithi100.stdout: "Percentage Used: 10%", 2024-04-07T16:16:10.527 INFO:tasks.workunit.client.0.smithi100.stdout: "Data Units Read: 385,784,931 [197 TB]", 2024-04-07T16:16:10.527 INFO:tasks.workunit.client.0.smithi100.stdout: "Data Units Written: 1,937,815,661 [992 TB]", 2024-04-07T16:16:10.527 INFO:tasks.workunit.client.0.smithi100.stdout: "Host Read Commands: 5,771,442,418", 2024-04-07T16:16:10.528 INFO:tasks.workunit.client.0.smithi100.stdout: "Host Write Commands: 31,722,089,956", 2024-04-07T16:16:10.528 INFO:tasks.workunit.client.0.smithi100.stdout: "Controller Busy Time: 4", 2024-04-07T16:16:10.528 INFO:tasks.workunit.client.0.smithi100.stdout: "Power Cycles: 45,774", 2024-04-07T16:16:10.528 INFO:tasks.workunit.client.0.smithi100.stdout: "Power On Hours: 57,978", 2024-04-07T16:16:10.528 INFO:tasks.workunit.client.0.smithi100.stdout: "Unsafe Shutdowns: 210", 2024-04-07T16:16:10.528 INFO:tasks.workunit.client.0.smithi100.stdout: "Media and Data Integrity Errors: 0", 2024-04-07T16:16:10.529 INFO:tasks.workunit.client.0.smithi100.stdout: "Error Information Log Entries: 0", 2024-04-07T16:16:10.529 INFO:tasks.workunit.client.0.smithi100.stdout: "", 2024-04-07T16:16:10.529 INFO:tasks.workunit.client.0.smithi100.stdout: "Error Information (NVMe Log 0x01, max 64 entries)", 2024-04-07T16:16:10.529 INFO:tasks.workunit.client.0.smithi100.stdout: "No Errors Logged", 2024-04-07T16:16:10.529 INFO:tasks.workunit.client.0.smithi100.stdout: "" 2024-04-07T16:16:10.529 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:10.530 INFO:tasks.workunit.client.0.smithi100.stdout: "platform_info": "x86_64-linux-5.15.0-101-generic", 2024-04-07T16:16:10.530 INFO:tasks.workunit.client.0.smithi100.stdout: "svn_revision": "5022", 2024-04-07T16:16:10.530 INFO:tasks.workunit.client.0.smithi100.stdout: "version": [ 2024-04-07T16:16:10.530 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T16:16:10.530 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T16:16:10.530 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T16:16:10.531 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:10.531 INFO:tasks.workunit.client.0.smithi100.stdout: "temperature": { 2024-04-07T16:16:10.531 INFO:tasks.workunit.client.0.smithi100.stdout: "current": 24 2024-04-07T16:16:10.531 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:10.531 INFO:tasks.workunit.client.0.smithi100.stdout: "user_capacity": { 2024-04-07T16:16:10.531 INFO:tasks.workunit.client.0.smithi100.stdout: "blocks": 781422768, 2024-04-07T16:16:10.532 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 400088457216 2024-04-07T16:16:10.533 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:10.534 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:10.534 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:10.534 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:10.534 INFO:tasks.workunit.client.0.smithi100.stdout: "25233610-f4fa-11ee-8447-0cc47a8fd733": { 2024-04-07T16:16:10.534 INFO:tasks.workunit.client.0.smithi100.stdout: "INTEL_SSDPEDMD400G4_2524b6c0-f4fa-11ee-8447-0cc47a8fd733": { 2024-04-07T16:16:10.534 INFO:tasks.workunit.client.0.smithi100.stdout: "20240407-160732": { 2024-04-07T16:16:10.535 INFO:tasks.workunit.client.0.smithi100.stdout: "device": { 2024-04-07T16:16:10.535 INFO:tasks.workunit.client.0.smithi100.stdout: "info_name": "/dev/nvme0n1", 2024-04-07T16:16:10.535 INFO:tasks.workunit.client.0.smithi100.stdout: "name": "/dev/nvme0n1", 2024-04-07T16:16:10.535 INFO:tasks.workunit.client.0.smithi100.stdout: "protocol": "NVMe", 2024-04-07T16:16:10.535 INFO:tasks.workunit.client.0.smithi100.stdout: "type": "nvme" 2024-04-07T16:16:10.536 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:10.536 INFO:tasks.workunit.client.0.smithi100.stdout: "firmware_version": "8DV101H0", 2024-04-07T16:16:10.536 INFO:tasks.workunit.client.0.smithi100.stdout: "host_id": "25233610-f4fa-11ee-8447-0cc47a8fd733", 2024-04-07T16:16:10.536 INFO:tasks.workunit.client.0.smithi100.stdout: "json_format_version": [ 2024-04-07T16:16:10.536 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T16:16:10.536 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T16:16:10.537 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:10.537 INFO:tasks.workunit.client.0.smithi100.stdout: "local_time": { 2024-04-07T16:16:10.537 INFO:tasks.workunit.client.0.smithi100.stdout: "asctime": "Sun Apr 7 16:07:32 2024 UTC", 2024-04-07T16:16:10.537 INFO:tasks.workunit.client.0.smithi100.stdout: "time_t": 1712506052 2024-04-07T16:16:10.537 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:10.537 INFO:tasks.workunit.client.0.smithi100.stdout: "logical_block_size": 512, 2024-04-07T16:16:10.538 INFO:tasks.workunit.client.0.smithi100.stdout: "model_name": "INTEL SSDPEDMD400G4", 2024-04-07T16:16:10.538 INFO:tasks.workunit.client.0.smithi100.stdout: "nvme_controller_id": 0, 2024-04-07T16:16:10.538 INFO:tasks.workunit.client.0.smithi100.stdout: "nvme_ieee_oui_identifier": 6083300, 2024-04-07T16:16:10.538 INFO:tasks.workunit.client.0.smithi100.stdout: "nvme_namespaces": [ 2024-04-07T16:16:10.538 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:10.538 INFO:tasks.workunit.client.0.smithi100.stdout: "capacity": { 2024-04-07T16:16:10.539 INFO:tasks.workunit.client.0.smithi100.stdout: "blocks": 781422768, 2024-04-07T16:16:10.539 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 400088457216 2024-04-07T16:16:10.539 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:10.539 INFO:tasks.workunit.client.0.smithi100.stdout: "formatted_lba_size": 512, 2024-04-07T16:16:10.539 INFO:tasks.workunit.client.0.smithi100.stdout: "id": 1, 2024-04-07T16:16:10.539 INFO:tasks.workunit.client.0.smithi100.stdout: "size": { 2024-04-07T16:16:10.540 INFO:tasks.workunit.client.0.smithi100.stdout: "blocks": 781422768, 2024-04-07T16:16:10.540 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 400088457216 2024-04-07T16:16:10.540 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:10.540 INFO:tasks.workunit.client.0.smithi100.stdout: "utilization": { 2024-04-07T16:16:10.540 INFO:tasks.workunit.client.0.smithi100.stdout: "blocks": 781422768, 2024-04-07T16:16:10.541 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 400088457216 2024-04-07T16:16:10.541 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:10.541 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:10.541 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:10.541 INFO:tasks.workunit.client.0.smithi100.stdout: "nvme_number_of_namespaces": 1, 2024-04-07T16:16:10.541 INFO:tasks.workunit.client.0.smithi100.stdout: "nvme_pci_vendor": { 2024-04-07T16:16:10.542 INFO:tasks.workunit.client.0.smithi100.stdout: "id": 32902, 2024-04-07T16:16:10.542 INFO:tasks.workunit.client.0.smithi100.stdout: "subsystem_id": 32902 2024-04-07T16:16:10.542 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:10.542 INFO:tasks.workunit.client.0.smithi100.stdout: "nvme_smart_health_information_add_log": { 2024-04-07T16:16:10.542 INFO:tasks.workunit.client.0.smithi100.stdout: "Device stats": { 2024-04-07T16:16:10.542 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_fw_downgrades": { 2024-04-07T16:16:10.543 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 0, 2024-04-07T16:16:10.543 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 0 2024-04-07T16:16:10.543 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:10.543 INFO:tasks.workunit.client.0.smithi100.stdout: "crc_error_count": { 2024-04-07T16:16:10.543 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 100, 2024-04-07T16:16:10.543 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 0 2024-04-07T16:16:10.544 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:10.544 INFO:tasks.workunit.client.0.smithi100.stdout: "end_to_end_error_detection_count": { 2024-04-07T16:16:10.544 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 100, 2024-04-07T16:16:10.544 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 0 2024-04-07T16:16:10.544 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:10.544 INFO:tasks.workunit.client.0.smithi100.stdout: "erase_fail_count": { 2024-04-07T16:16:10.544 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 100, 2024-04-07T16:16:10.545 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 0 2024-04-07T16:16:10.545 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:10.545 INFO:tasks.workunit.client.0.smithi100.stdout: "host_bytes_written": { 2024-04-07T16:16:10.545 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 100, 2024-04-07T16:16:10.545 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 25060599 2024-04-07T16:16:10.545 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:10.546 INFO:tasks.workunit.client.0.smithi100.stdout: "host_ctx_wear_used": { 2024-04-07T16:16:10.546 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 100, 2024-04-07T16:16:10.546 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 1 2024-04-07T16:16:10.546 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:10.546 INFO:tasks.workunit.client.0.smithi100.stdout: "media_bytes_read": { 2024-04-07T16:16:10.547 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 0, 2024-04-07T16:16:10.547 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 0 2024-04-07T16:16:10.547 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:10.547 INFO:tasks.workunit.client.0.smithi100.stdout: "nand_bytes_written": { 2024-04-07T16:16:10.547 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 100, 2024-04-07T16:16:10.547 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 42751216 2024-04-07T16:16:10.547 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:10.547 INFO:tasks.workunit.client.0.smithi100.stdout: "perf_stat_indicator": { 2024-04-07T16:16:10.548 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 0, 2024-04-07T16:16:10.548 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 0 2024-04-07T16:16:10.548 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:10.548 INFO:tasks.workunit.client.0.smithi100.stdout: "pll_lock_loss_count": { 2024-04-07T16:16:10.548 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 100, 2024-04-07T16:16:10.548 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 0 2024-04-07T16:16:10.548 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:10.548 INFO:tasks.workunit.client.0.smithi100.stdout: "program_fail_count": { 2024-04-07T16:16:10.548 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 100, 2024-04-07T16:16:10.549 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 0 2024-04-07T16:16:10.549 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:10.549 INFO:tasks.workunit.client.0.smithi100.stdout: "re_alloc_sectr_cnt": { 2024-04-07T16:16:10.549 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 0, 2024-04-07T16:16:10.549 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 0 2024-04-07T16:16:10.549 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:10.549 INFO:tasks.workunit.client.0.smithi100.stdout: "retry_buffer_overflow_count": { 2024-04-07T16:16:10.549 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 100, 2024-04-07T16:16:10.550 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 0 2024-04-07T16:16:10.550 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:10.550 INFO:tasks.workunit.client.0.smithi100.stdout: "soft_ecc_err_rate": { 2024-04-07T16:16:10.550 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 0, 2024-04-07T16:16:10.550 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 0 2024-04-07T16:16:10.550 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:10.550 INFO:tasks.workunit.client.0.smithi100.stdout: "thermal_throttle_status": { 2024-04-07T16:16:10.550 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 100, 2024-04-07T16:16:10.551 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": { 2024-04-07T16:16:10.551 INFO:tasks.workunit.client.0.smithi100.stdout: "cnt": 0, 2024-04-07T16:16:10.551 INFO:tasks.workunit.client.0.smithi100.stdout: "pct": 0 2024-04-07T16:16:10.551 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:10.551 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:10.551 INFO:tasks.workunit.client.0.smithi100.stdout: "timed_workload_host_reads": { 2024-04-07T16:16:10.552 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 100, 2024-04-07T16:16:10.552 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 65535 2024-04-07T16:16:10.552 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:10.552 INFO:tasks.workunit.client.0.smithi100.stdout: "timed_workload_media_wear": { 2024-04-07T16:16:10.552 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 100, 2024-04-07T16:16:10.552 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 64 2024-04-07T16:16:10.552 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:10.552 INFO:tasks.workunit.client.0.smithi100.stdout: "timed_workload_timer": { 2024-04-07T16:16:10.552 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 100, 2024-04-07T16:16:10.552 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 65535 2024-04-07T16:16:10.552 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:10.552 INFO:tasks.workunit.client.0.smithi100.stdout: "unexp_power_loss": { 2024-04-07T16:16:10.552 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 0, 2024-04-07T16:16:10.552 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 0 2024-04-07T16:16:10.553 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:10.553 INFO:tasks.workunit.client.0.smithi100.stdout: "wear_leveling": { 2024-04-07T16:16:10.553 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 91, 2024-04-07T16:16:10.553 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": { 2024-04-07T16:16:10.553 INFO:tasks.workunit.client.0.smithi100.stdout: "avg": 2895, 2024-04-07T16:16:10.553 INFO:tasks.workunit.client.0.smithi100.stdout: "max": 2907, 2024-04-07T16:16:10.553 INFO:tasks.workunit.client.0.smithi100.stdout: "min": 2884 2024-04-07T16:16:10.553 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:10.553 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:10.553 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:10.553 INFO:tasks.workunit.client.0.smithi100.stdout: "Intel Smart log": "nvme0n1" 2024-04-07T16:16:10.553 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:10.553 INFO:tasks.workunit.client.0.smithi100.stdout: "nvme_smart_health_information_log": { 2024-04-07T16:16:10.553 INFO:tasks.workunit.client.0.smithi100.stdout: "available_spare": 100, 2024-04-07T16:16:10.554 INFO:tasks.workunit.client.0.smithi100.stdout: "available_spare_threshold": 10, 2024-04-07T16:16:10.554 INFO:tasks.workunit.client.0.smithi100.stdout: "controller_busy_time": 0, 2024-04-07T16:16:10.554 INFO:tasks.workunit.client.0.smithi100.stdout: "critical_warning": 0, 2024-04-07T16:16:10.554 INFO:tasks.workunit.client.0.smithi100.stdout: "data_units_read": 359011585, 2024-04-07T16:16:10.554 INFO:tasks.workunit.client.0.smithi100.stdout: "data_units_written": 1642371481, 2024-04-07T16:16:10.554 INFO:tasks.workunit.client.0.smithi100.stdout: "host_reads": 5750712954, 2024-04-07T16:16:10.554 INFO:tasks.workunit.client.0.smithi100.stdout: "host_writes": 28276536650, 2024-04-07T16:16:10.554 INFO:tasks.workunit.client.0.smithi100.stdout: "media_errors": 0, 2024-04-07T16:16:10.554 INFO:tasks.workunit.client.0.smithi100.stdout: "num_err_log_entries": 0, 2024-04-07T16:16:10.554 INFO:tasks.workunit.client.0.smithi100.stdout: "percentage_used": 9, 2024-04-07T16:16:10.554 INFO:tasks.workunit.client.0.smithi100.stdout: "power_cycles": 47059, 2024-04-07T16:16:10.554 INFO:tasks.workunit.client.0.smithi100.stdout: "power_on_hours": 54821, 2024-04-07T16:16:10.554 INFO:tasks.workunit.client.0.smithi100.stdout: "temperature": 25, 2024-04-07T16:16:10.554 INFO:tasks.workunit.client.0.smithi100.stdout: "unsafe_shutdowns": 12 2024-04-07T16:16:10.555 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:10.555 INFO:tasks.workunit.client.0.smithi100.stdout: "nvme_vendor": "intel", 2024-04-07T16:16:10.555 INFO:tasks.workunit.client.0.smithi100.stdout: "power_cycle_count": 47059, 2024-04-07T16:16:10.555 INFO:tasks.workunit.client.0.smithi100.stdout: "power_on_time": { 2024-04-07T16:16:10.555 INFO:tasks.workunit.client.0.smithi100.stdout: "hours": 54821 2024-04-07T16:16:10.555 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:10.555 INFO:tasks.workunit.client.0.smithi100.stdout: "serial_number": "deleted", 2024-04-07T16:16:10.555 INFO:tasks.workunit.client.0.smithi100.stdout: "smart_status": { 2024-04-07T16:16:10.555 INFO:tasks.workunit.client.0.smithi100.stdout: "nvme": { 2024-04-07T16:16:10.555 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:10.555 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:10.555 INFO:tasks.workunit.client.0.smithi100.stdout: "passed": true 2024-04-07T16:16:10.555 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:10.555 INFO:tasks.workunit.client.0.smithi100.stdout: "smartctl": { 2024-04-07T16:16:10.556 INFO:tasks.workunit.client.0.smithi100.stdout: "argv": [ 2024-04-07T16:16:10.556 INFO:tasks.workunit.client.0.smithi100.stdout: "smartctl", 2024-04-07T16:16:10.556 INFO:tasks.workunit.client.0.smithi100.stdout: "-x", 2024-04-07T16:16:10.556 INFO:tasks.workunit.client.0.smithi100.stdout: "--json=o", 2024-04-07T16:16:10.556 INFO:tasks.workunit.client.0.smithi100.stdout: "/dev/nvme0n1" 2024-04-07T16:16:10.556 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:10.556 INFO:tasks.workunit.client.0.smithi100.stdout: "build_info": "(local build)", 2024-04-07T16:16:10.556 INFO:tasks.workunit.client.0.smithi100.stdout: "exit_status": 0, 2024-04-07T16:16:10.556 INFO:tasks.workunit.client.0.smithi100.stdout: "output": [ 2024-04-07T16:16:10.556 INFO:tasks.workunit.client.0.smithi100.stdout: "smartctl 7.1 2019-12-30 r5022 [x86_64-linux-5.15.0-101-generic] (local build)", 2024-04-07T16:16:10.556 INFO:tasks.workunit.client.0.smithi100.stdout: "Copyright (C) 2002-19, Bruce Allen, Christian Franke, www.smartmontools.org", 2024-04-07T16:16:10.556 INFO:tasks.workunit.client.0.smithi100.stdout: "", 2024-04-07T16:16:10.556 INFO:tasks.workunit.client.0.smithi100.stdout: "=== START OF INFORMATION SECTION ===", 2024-04-07T16:16:10.556 INFO:tasks.workunit.client.0.smithi100.stdout: "Model Number: INTEL SSDPEDMD400G4", 2024-04-07T16:16:10.557 INFO:tasks.workunit.client.0.smithi100.stdout: "Serial Number: deleted", 2024-04-07T16:16:10.557 INFO:tasks.workunit.client.0.smithi100.stdout: "Firmware Version: 8DV101H0", 2024-04-07T16:16:10.557 INFO:tasks.workunit.client.0.smithi100.stdout: "PCI Vendor/Subsystem ID: 0x8086", 2024-04-07T16:16:10.557 INFO:tasks.workunit.client.0.smithi100.stdout: "IEEE OUI Identifier: 0x5cd2e4", 2024-04-07T16:16:10.557 INFO:tasks.workunit.client.0.smithi100.stdout: "Controller ID: 0", 2024-04-07T16:16:10.557 INFO:tasks.workunit.client.0.smithi100.stdout: "Number of Namespaces: 1", 2024-04-07T16:16:10.557 INFO:tasks.workunit.client.0.smithi100.stdout: "Namespace 1 Size/Capacity: 400,088,457,216 [400 GB]", 2024-04-07T16:16:10.557 INFO:tasks.workunit.client.0.smithi100.stdout: "Namespace 1 Formatted LBA Size: 512", 2024-04-07T16:16:10.557 INFO:tasks.workunit.client.0.smithi100.stdout: "Local Time is: Sun Apr 7 16:07:32 2024 UTC", 2024-04-07T16:16:10.557 INFO:tasks.workunit.client.0.smithi100.stdout: "Firmware Updates (0x02): 1 Slot", 2024-04-07T16:16:10.557 INFO:tasks.workunit.client.0.smithi100.stdout: "Optional Admin Commands (0x0006): Format Frmw_DL", 2024-04-07T16:16:10.557 INFO:tasks.workunit.client.0.smithi100.stdout: "Optional NVM Commands (0x0006): Wr_Unc DS_Mngmt", 2024-04-07T16:16:10.558 INFO:tasks.workunit.client.0.smithi100.stdout: "Maximum Data Transfer Size: 32 Pages", 2024-04-07T16:16:10.558 INFO:tasks.workunit.client.0.smithi100.stdout: "", 2024-04-07T16:16:10.558 INFO:tasks.workunit.client.0.smithi100.stdout: "Supported Power States", 2024-04-07T16:16:10.558 INFO:tasks.workunit.client.0.smithi100.stdout: "St Op Max Active Idle RL RT WL WT Ent_Lat Ex_Lat", 2024-04-07T16:16:10.558 INFO:tasks.workunit.client.0.smithi100.stdout: " 0 + 25.00W - - 0 0 0 0 0 0", 2024-04-07T16:16:10.558 INFO:tasks.workunit.client.0.smithi100.stdout: "", 2024-04-07T16:16:10.558 INFO:tasks.workunit.client.0.smithi100.stdout: "Supported LBA Sizes (NSID 0x1)", 2024-04-07T16:16:10.558 INFO:tasks.workunit.client.0.smithi100.stdout: "Id Fmt Data Metadt Rel_Perf", 2024-04-07T16:16:10.558 INFO:tasks.workunit.client.0.smithi100.stdout: " 0 + 512 0 2", 2024-04-07T16:16:10.558 INFO:tasks.workunit.client.0.smithi100.stdout: " 1 - 512 8 2", 2024-04-07T16:16:10.558 INFO:tasks.workunit.client.0.smithi100.stdout: " 2 - 512 16 2", 2024-04-07T16:16:10.558 INFO:tasks.workunit.client.0.smithi100.stdout: " 3 - 4096 0 0", 2024-04-07T16:16:10.559 INFO:tasks.workunit.client.0.smithi100.stdout: " 4 - 4096 8 0", 2024-04-07T16:16:10.559 INFO:tasks.workunit.client.0.smithi100.stdout: " 5 - 4096 64 0", 2024-04-07T16:16:10.559 INFO:tasks.workunit.client.0.smithi100.stdout: " 6 - 4096 128 0", 2024-04-07T16:16:10.559 INFO:tasks.workunit.client.0.smithi100.stdout: "", 2024-04-07T16:16:10.559 INFO:tasks.workunit.client.0.smithi100.stdout: "=== START OF SMART DATA SECTION ===", 2024-04-07T16:16:10.559 INFO:tasks.workunit.client.0.smithi100.stdout: "SMART overall-health self-assessment test result: PASSED", 2024-04-07T16:16:10.559 INFO:tasks.workunit.client.0.smithi100.stdout: "", 2024-04-07T16:16:10.559 INFO:tasks.workunit.client.0.smithi100.stdout: "SMART/Health Information (NVMe Log 0x02)", 2024-04-07T16:16:10.559 INFO:tasks.workunit.client.0.smithi100.stdout: "Critical Warning: 0x00", 2024-04-07T16:16:10.559 INFO:tasks.workunit.client.0.smithi100.stdout: "Temperature: 25 Celsius", 2024-04-07T16:16:10.559 INFO:tasks.workunit.client.0.smithi100.stdout: "Available Spare: 100%", 2024-04-07T16:16:10.559 INFO:tasks.workunit.client.0.smithi100.stdout: "Available Spare Threshold: 10%", 2024-04-07T16:16:10.560 INFO:tasks.workunit.client.0.smithi100.stdout: "Percentage Used: 9%", 2024-04-07T16:16:10.560 INFO:tasks.workunit.client.0.smithi100.stdout: "Data Units Read: 359,011,585 [183 TB]", 2024-04-07T16:16:10.560 INFO:tasks.workunit.client.0.smithi100.stdout: "Data Units Written: 1,642,371,481 [840 TB]", 2024-04-07T16:16:10.560 INFO:tasks.workunit.client.0.smithi100.stdout: "Host Read Commands: 5,750,712,954", 2024-04-07T16:16:10.560 INFO:tasks.workunit.client.0.smithi100.stdout: "Host Write Commands: 28,276,536,650", 2024-04-07T16:16:10.560 INFO:tasks.workunit.client.0.smithi100.stdout: "Controller Busy Time: 0", 2024-04-07T16:16:10.560 INFO:tasks.workunit.client.0.smithi100.stdout: "Power Cycles: 47,059", 2024-04-07T16:16:10.560 INFO:tasks.workunit.client.0.smithi100.stdout: "Power On Hours: 54,821", 2024-04-07T16:16:10.560 INFO:tasks.workunit.client.0.smithi100.stdout: "Unsafe Shutdowns: 12", 2024-04-07T16:16:10.560 INFO:tasks.workunit.client.0.smithi100.stdout: "Media and Data Integrity Errors: 0", 2024-04-07T16:16:10.560 INFO:tasks.workunit.client.0.smithi100.stdout: "Error Information Log Entries: 0", 2024-04-07T16:16:10.560 INFO:tasks.workunit.client.0.smithi100.stdout: "", 2024-04-07T16:16:10.560 INFO:tasks.workunit.client.0.smithi100.stdout: "Error Information (NVMe Log 0x01, max 64 entries)", 2024-04-07T16:16:10.560 INFO:tasks.workunit.client.0.smithi100.stdout: "No Errors Logged", 2024-04-07T16:16:10.560 INFO:tasks.workunit.client.0.smithi100.stdout: "" 2024-04-07T16:16:10.561 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:10.561 INFO:tasks.workunit.client.0.smithi100.stdout: "platform_info": "x86_64-linux-5.15.0-101-generic", 2024-04-07T16:16:10.561 INFO:tasks.workunit.client.0.smithi100.stdout: "svn_revision": "5022", 2024-04-07T16:16:10.561 INFO:tasks.workunit.client.0.smithi100.stdout: "version": [ 2024-04-07T16:16:10.561 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T16:16:10.561 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T16:16:10.561 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T16:16:10.561 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:10.561 INFO:tasks.workunit.client.0.smithi100.stdout: "temperature": { 2024-04-07T16:16:10.561 INFO:tasks.workunit.client.0.smithi100.stdout: "current": 25 2024-04-07T16:16:10.561 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:10.562 INFO:tasks.workunit.client.0.smithi100.stdout: "user_capacity": { 2024-04-07T16:16:10.562 INFO:tasks.workunit.client.0.smithi100.stdout: "blocks": 781422768, 2024-04-07T16:16:10.562 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 400088457216 2024-04-07T16:16:10.562 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:10.562 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:10.562 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:10.562 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:10.562 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:10.562 INFO:tasks.workunit.client.0.smithi100.stdout: "report": { 2024-04-07T16:16:10.563 INFO:tasks.workunit.client.0.smithi100.stdout: "balancer": { 2024-04-07T16:16:10.563 INFO:tasks.workunit.client.0.smithi100.stdout: "active": true, 2024-04-07T16:16:10.563 INFO:tasks.workunit.client.0.smithi100.stdout: "mode": "" 2024-04-07T16:16:10.563 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:10.563 INFO:tasks.workunit.client.0.smithi100.stdout: "channels": [ 2024-04-07T16:16:10.563 INFO:tasks.workunit.client.0.smithi100.stdout: "basic", 2024-04-07T16:16:10.563 INFO:tasks.workunit.client.0.smithi100.stdout: "crash", 2024-04-07T16:16:10.563 INFO:tasks.workunit.client.0.smithi100.stdout: "device", 2024-04-07T16:16:10.563 INFO:tasks.workunit.client.0.smithi100.stdout: "ident" 2024-04-07T16:16:10.564 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:10.564 INFO:tasks.workunit.client.0.smithi100.stdout: "channels_available": [ 2024-04-07T16:16:10.564 INFO:tasks.workunit.client.0.smithi100.stdout: "basic", 2024-04-07T16:16:10.564 INFO:tasks.workunit.client.0.smithi100.stdout: "ident", 2024-04-07T16:16:10.564 INFO:tasks.workunit.client.0.smithi100.stdout: "crash", 2024-04-07T16:16:10.564 INFO:tasks.workunit.client.0.smithi100.stdout: "device", 2024-04-07T16:16:10.564 INFO:tasks.workunit.client.0.smithi100.stdout: "perf" 2024-04-07T16:16:10.564 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:10.565 INFO:tasks.workunit.client.0.smithi100.stdout: "collections_available": [ 2024-04-07T16:16:10.565 INFO:tasks.workunit.client.0.smithi100.stdout: "basic_base", 2024-04-07T16:16:10.565 INFO:tasks.workunit.client.0.smithi100.stdout: "device_base", 2024-04-07T16:16:10.565 INFO:tasks.workunit.client.0.smithi100.stdout: "crash_base", 2024-04-07T16:16:10.565 INFO:tasks.workunit.client.0.smithi100.stdout: "ident_base", 2024-04-07T16:16:10.565 INFO:tasks.workunit.client.0.smithi100.stdout: "perf_perf", 2024-04-07T16:16:10.565 INFO:tasks.workunit.client.0.smithi100.stdout: "basic_mds_metadata", 2024-04-07T16:16:10.565 INFO:tasks.workunit.client.0.smithi100.stdout: "basic_pool_usage", 2024-04-07T16:16:10.565 INFO:tasks.workunit.client.0.smithi100.stdout: "basic_usage_by_class", 2024-04-07T16:16:10.565 INFO:tasks.workunit.client.0.smithi100.stdout: "basic_rook_v01", 2024-04-07T16:16:10.565 INFO:tasks.workunit.client.0.smithi100.stdout: "perf_memory_metrics", 2024-04-07T16:16:10.565 INFO:tasks.workunit.client.0.smithi100.stdout: "basic_pool_options_bluestore" 2024-04-07T16:16:10.565 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:10.565 INFO:tasks.workunit.client.0.smithi100.stdout: "collections_opted_in": [ 2024-04-07T16:16:10.566 INFO:tasks.workunit.client.0.smithi100.stdout: "basic_base", 2024-04-07T16:16:10.566 INFO:tasks.workunit.client.0.smithi100.stdout: "device_base", 2024-04-07T16:16:10.566 INFO:tasks.workunit.client.0.smithi100.stdout: "crash_base", 2024-04-07T16:16:10.566 INFO:tasks.workunit.client.0.smithi100.stdout: "ident_base", 2024-04-07T16:16:10.566 INFO:tasks.workunit.client.0.smithi100.stdout: "perf_perf", 2024-04-07T16:16:10.566 INFO:tasks.workunit.client.0.smithi100.stdout: "basic_mds_metadata", 2024-04-07T16:16:10.566 INFO:tasks.workunit.client.0.smithi100.stdout: "basic_pool_usage", 2024-04-07T16:16:10.566 INFO:tasks.workunit.client.0.smithi100.stdout: "basic_usage_by_class", 2024-04-07T16:16:10.566 INFO:tasks.workunit.client.0.smithi100.stdout: "basic_rook_v01", 2024-04-07T16:16:10.566 INFO:tasks.workunit.client.0.smithi100.stdout: "perf_memory_metrics", 2024-04-07T16:16:10.566 INFO:tasks.workunit.client.0.smithi100.stdout: "basic_pool_options_bluestore" 2024-04-07T16:16:10.567 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:10.567 INFO:tasks.workunit.client.0.smithi100.stdout: "config": { 2024-04-07T16:16:10.567 INFO:tasks.workunit.client.0.smithi100.stdout: "active_changed": [ 2024-04-07T16:16:10.567 INFO:tasks.workunit.client.0.smithi100.stdout: "auth_allow_insecure_global_id_reclaim", 2024-04-07T16:16:10.567 INFO:tasks.workunit.client.0.smithi100.stdout: "auth_debug", 2024-04-07T16:16:10.567 INFO:tasks.workunit.client.0.smithi100.stdout: "auth_mon_ticket_ttl", 2024-04-07T16:16:10.567 INFO:tasks.workunit.client.0.smithi100.stdout: "auth_service_ticket_ttl", 2024-04-07T16:16:10.567 INFO:tasks.workunit.client.0.smithi100.stdout: "bdev_debug_aio", 2024-04-07T16:16:10.567 INFO:tasks.workunit.client.0.smithi100.stdout: "client_check_pool_perm", 2024-04-07T16:16:10.567 INFO:tasks.workunit.client.0.smithi100.stdout: "client_mount_gid", 2024-04-07T16:16:10.567 INFO:tasks.workunit.client.0.smithi100.stdout: "client_mount_uid", 2024-04-07T16:16:10.567 INFO:tasks.workunit.client.0.smithi100.stdout: "client_quota", 2024-04-07T16:16:10.567 INFO:tasks.workunit.client.0.smithi100.stdout: "container_image", 2024-04-07T16:16:10.567 INFO:tasks.workunit.client.0.smithi100.stdout: "daemonize", 2024-04-07T16:16:10.568 INFO:tasks.workunit.client.0.smithi100.stdout: "debug_asserts_on_shutdown", 2024-04-07T16:16:10.568 INFO:tasks.workunit.client.0.smithi100.stdout: "debug_mgr", 2024-04-07T16:16:10.568 INFO:tasks.workunit.client.0.smithi100.stdout: "debug_mon", 2024-04-07T16:16:10.568 INFO:tasks.workunit.client.0.smithi100.stdout: "debug_ms", 2024-04-07T16:16:10.568 INFO:tasks.workunit.client.0.smithi100.stdout: "debug_osd", 2024-04-07T16:16:10.568 INFO:tasks.workunit.client.0.smithi100.stdout: "debug_paxos", 2024-04-07T16:16:10.568 INFO:tasks.workunit.client.0.smithi100.stdout: "keyring", 2024-04-07T16:16:10.568 INFO:tasks.workunit.client.0.smithi100.stdout: "leveldb_block_size", 2024-04-07T16:16:10.568 INFO:tasks.workunit.client.0.smithi100.stdout: "leveldb_cache_size", 2024-04-07T16:16:10.568 INFO:tasks.workunit.client.0.smithi100.stdout: "leveldb_compression", 2024-04-07T16:16:10.568 INFO:tasks.workunit.client.0.smithi100.stdout: "leveldb_log", 2024-04-07T16:16:10.568 INFO:tasks.workunit.client.0.smithi100.stdout: "leveldb_write_buffer_size", 2024-04-07T16:16:10.568 INFO:tasks.workunit.client.0.smithi100.stdout: "log_stderr_prefix", 2024-04-07T16:16:10.568 INFO:tasks.workunit.client.0.smithi100.stdout: "log_to_file", 2024-04-07T16:16:10.569 INFO:tasks.workunit.client.0.smithi100.stdout: "log_to_journald", 2024-04-07T16:16:10.569 INFO:tasks.workunit.client.0.smithi100.stdout: "log_to_stderr", 2024-04-07T16:16:10.569 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_allow_pool_delete", 2024-04-07T16:16:10.569 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_clock_drift_allowed", 2024-04-07T16:16:10.569 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_cluster_log_file_level", 2024-04-07T16:16:10.569 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_cluster_log_to_file", 2024-04-07T16:16:10.569 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_cluster_log_to_stderr", 2024-04-07T16:16:10.569 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_data_avail_warn", 2024-04-07T16:16:10.569 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_election_default_strategy", 2024-04-07T16:16:10.569 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_host", 2024-04-07T16:16:10.569 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_max_pg_per_osd", 2024-04-07T16:16:10.569 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_mgr_mkfs_grace", 2024-04-07T16:16:10.569 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_osd_prime_pg_temp", 2024-04-07T16:16:10.569 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_osd_reporter_subtree_level", 2024-04-07T16:16:10.570 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_pg_warn_max_object_skew", 2024-04-07T16:16:10.570 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_reweight_min_bytes_per_osd", 2024-04-07T16:16:10.570 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_reweight_min_pgs_per_osd", 2024-04-07T16:16:10.570 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_warn_on_crush_straw_calc_version_zero", 2024-04-07T16:16:10.570 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_warn_on_insecure_global_id_reclaim", 2024-04-07T16:16:10.570 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_warn_on_insecure_global_id_reclaim_allowed", 2024-04-07T16:16:10.570 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_warn_on_legacy_crush_tunables", 2024-04-07T16:16:10.570 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_warn_on_osd_down_out_interval_zero", 2024-04-07T16:16:10.570 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_warn_on_pool_pg_num_not_power_of_two", 2024-04-07T16:16:10.570 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_warn_on_too_few_osds", 2024-04-07T16:16:10.570 INFO:tasks.workunit.client.0.smithi100.stdout: "ms_die_on_bug", 2024-04-07T16:16:10.570 INFO:tasks.workunit.client.0.smithi100.stdout: "ms_die_on_old_message", 2024-04-07T16:16:10.570 INFO:tasks.workunit.client.0.smithi100.stdout: "no_config_file", 2024-04-07T16:16:10.570 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_class_default_list", 2024-04-07T16:16:10.571 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_class_load_list", 2024-04-07T16:16:10.571 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_crush_chooseleaf_type", 2024-04-07T16:16:10.571 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_debug_misdirected_ops", 2024-04-07T16:16:10.571 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_debug_op_order", 2024-04-07T16:16:10.571 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_debug_pg_log_writeout", 2024-04-07T16:16:10.571 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_debug_shutdown", 2024-04-07T16:16:10.571 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_debug_verify_cached_snaps", 2024-04-07T16:16:10.571 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_debug_verify_missing_on_start", 2024-04-07T16:16:10.571 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_debug_verify_stray_on_activate", 2024-04-07T16:16:10.571 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_deep_scrub_update_digest_min_age", 2024-04-07T16:16:10.571 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_delete_sleep", 2024-04-07T16:16:10.571 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_delete_sleep_hdd", 2024-04-07T16:16:10.571 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_delete_sleep_hybrid", 2024-04-07T16:16:10.571 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_delete_sleep_ssd", 2024-04-07T16:16:10.572 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_max_backfills", 2024-04-07T16:16:10.572 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mclock_max_capacity_iops_ssd", 2024-04-07T16:16:10.572 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mclock_profile", 2024-04-07T16:16:10.572 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mclock_scheduler_background_best_effort_lim", 2024-04-07T16:16:10.572 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mclock_scheduler_background_best_effort_res", 2024-04-07T16:16:10.572 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mclock_scheduler_background_best_effort_wgt", 2024-04-07T16:16:10.572 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mclock_scheduler_background_recovery_lim", 2024-04-07T16:16:10.572 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mclock_scheduler_background_recovery_res", 2024-04-07T16:16:10.572 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mclock_scheduler_background_recovery_wgt", 2024-04-07T16:16:10.572 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mclock_scheduler_client_lim", 2024-04-07T16:16:10.572 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mclock_scheduler_client_res", 2024-04-07T16:16:10.572 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mclock_scheduler_client_wgt", 2024-04-07T16:16:10.572 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_memory_target", 2024-04-07T16:16:10.572 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_memory_target_autotune", 2024-04-07T16:16:10.573 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_op_queue", 2024-04-07T16:16:10.573 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_op_queue_cut_off", 2024-04-07T16:16:10.573 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_pg_delete_cost", 2024-04-07T16:16:10.573 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_pool_default_erasure_code_profile", 2024-04-07T16:16:10.573 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_pool_default_pg_autoscale_mode", 2024-04-07T16:16:10.573 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_recover_clone_overlap", 2024-04-07T16:16:10.573 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_recovery_max_active", 2024-04-07T16:16:10.573 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_recovery_max_active_hdd", 2024-04-07T16:16:10.573 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_recovery_max_active_ssd", 2024-04-07T16:16:10.573 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_recovery_max_chunk", 2024-04-07T16:16:10.573 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_recovery_sleep", 2024-04-07T16:16:10.573 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_recovery_sleep_hdd", 2024-04-07T16:16:10.573 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_recovery_sleep_hybrid", 2024-04-07T16:16:10.573 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_recovery_sleep_ssd", 2024-04-07T16:16:10.573 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_scrub_load_threshold", 2024-04-07T16:16:10.574 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_scrub_max_interval", 2024-04-07T16:16:10.574 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_scrub_sleep", 2024-04-07T16:16:10.574 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_shutdown_pgref_assert", 2024-04-07T16:16:10.574 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_snap_trim_sleep", 2024-04-07T16:16:10.574 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_snap_trim_sleep_hdd", 2024-04-07T16:16:10.574 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_snap_trim_sleep_hybrid", 2024-04-07T16:16:10.574 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_snap_trim_sleep_ssd", 2024-04-07T16:16:10.574 INFO:tasks.workunit.client.0.smithi100.stdout: "public_addr", 2024-04-07T16:16:10.574 INFO:tasks.workunit.client.0.smithi100.stdout: "public_addrv", 2024-04-07T16:16:10.574 INFO:tasks.workunit.client.0.smithi100.stdout: "public_network", 2024-04-07T16:16:10.574 INFO:tasks.workunit.client.0.smithi100.stdout: "rbd_default_features", 2024-04-07T16:16:10.574 INFO:tasks.workunit.client.0.smithi100.stdout: "rbd_qos_exclude_ops", 2024-04-07T16:16:10.574 INFO:tasks.workunit.client.0.smithi100.stdout: "setgroup", 2024-04-07T16:16:10.575 INFO:tasks.workunit.client.0.smithi100.stdout: "setuser" 2024-04-07T16:16:10.575 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:10.575 INFO:tasks.workunit.client.0.smithi100.stdout: "cluster_changed": [ 2024-04-07T16:16:10.575 INFO:tasks.workunit.client.0.smithi100.stdout: "auth_allow_insecure_global_id_reclaim", 2024-04-07T16:16:10.575 INFO:tasks.workunit.client.0.smithi100.stdout: "auth_debug", 2024-04-07T16:16:10.575 INFO:tasks.workunit.client.0.smithi100.stdout: "auth_mon_ticket_ttl", 2024-04-07T16:16:10.575 INFO:tasks.workunit.client.0.smithi100.stdout: "auth_service_ticket_ttl", 2024-04-07T16:16:10.575 INFO:tasks.workunit.client.0.smithi100.stdout: "bdev_debug_aio", 2024-04-07T16:16:10.575 INFO:tasks.workunit.client.0.smithi100.stdout: "container_image", 2024-04-07T16:16:10.575 INFO:tasks.workunit.client.0.smithi100.stdout: "debug_asserts_on_shutdown", 2024-04-07T16:16:10.575 INFO:tasks.workunit.client.0.smithi100.stdout: "debug_mgr", 2024-04-07T16:16:10.575 INFO:tasks.workunit.client.0.smithi100.stdout: "debug_mon", 2024-04-07T16:16:10.576 INFO:tasks.workunit.client.0.smithi100.stdout: "debug_ms", 2024-04-07T16:16:10.576 INFO:tasks.workunit.client.0.smithi100.stdout: "debug_osd", 2024-04-07T16:16:10.576 INFO:tasks.workunit.client.0.smithi100.stdout: "debug_paxos", 2024-04-07T16:16:10.576 INFO:tasks.workunit.client.0.smithi100.stdout: "log_to_file", 2024-04-07T16:16:10.576 INFO:tasks.workunit.client.0.smithi100.stdout: "log_to_journald", 2024-04-07T16:16:10.576 INFO:tasks.workunit.client.0.smithi100.stdout: "log_to_stderr", 2024-04-07T16:16:10.576 INFO:tasks.workunit.client.0.smithi100.stdout: "mds_join_fs", 2024-04-07T16:16:10.576 INFO:tasks.workunit.client.0.smithi100.stdout: "mgr/cephadm/allow_ptrace", 2024-04-07T16:16:10.576 INFO:tasks.workunit.client.0.smithi100.stdout: "mgr/cephadm/container_init", 2024-04-07T16:16:10.576 INFO:tasks.workunit.client.0.smithi100.stdout: "mgr/cephadm/migration_current", 2024-04-07T16:16:10.576 INFO:tasks.workunit.client.0.smithi100.stdout: "mgr/cephadm/use_repo_digest", 2024-04-07T16:16:10.576 INFO:tasks.workunit.client.0.smithi100.stdout: "mgr/dashboard/ALERTMANAGER_API_HOST", 2024-04-07T16:16:10.576 INFO:tasks.workunit.client.0.smithi100.stdout: "mgr/dashboard/GRAFANA_API_SSL_VERIFY", 2024-04-07T16:16:10.576 INFO:tasks.workunit.client.0.smithi100.stdout: "mgr/dashboard/GRAFANA_API_URL", 2024-04-07T16:16:10.576 INFO:tasks.workunit.client.0.smithi100.stdout: "mgr/dashboard/PROMETHEUS_API_HOST", 2024-04-07T16:16:10.577 INFO:tasks.workunit.client.0.smithi100.stdout: "mgr/dashboard/ssl_server_port", 2024-04-07T16:16:10.577 INFO:tasks.workunit.client.0.smithi100.stdout: "mgr/orchestrator/orchestrator", 2024-04-07T16:16:10.577 INFO:tasks.workunit.client.0.smithi100.stdout: "mgr/telemetry/channel_ident", 2024-04-07T16:16:10.577 INFO:tasks.workunit.client.0.smithi100.stdout: "mgr/telemetry/description", 2024-04-07T16:16:10.577 INFO:tasks.workunit.client.0.smithi100.stdout: "mgr/telemetry/organization", 2024-04-07T16:16:10.577 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_allow_pool_delete", 2024-04-07T16:16:10.577 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_clock_drift_allowed", 2024-04-07T16:16:10.577 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_cluster_log_file_level", 2024-04-07T16:16:10.577 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_cluster_log_to_file", 2024-04-07T16:16:10.577 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_data_avail_warn", 2024-04-07T16:16:10.577 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_election_default_strategy", 2024-04-07T16:16:10.577 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_max_pg_per_osd", 2024-04-07T16:16:10.577 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_mgr_mkfs_grace", 2024-04-07T16:16:10.577 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_osd_prime_pg_temp", 2024-04-07T16:16:10.578 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_osd_reporter_subtree_level", 2024-04-07T16:16:10.578 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_pg_warn_max_object_skew", 2024-04-07T16:16:10.578 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_reweight_min_bytes_per_osd", 2024-04-07T16:16:10.578 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_reweight_min_pgs_per_osd", 2024-04-07T16:16:10.578 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_warn_on_crush_straw_calc_version_zero", 2024-04-07T16:16:10.578 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_warn_on_insecure_global_id_reclaim", 2024-04-07T16:16:10.578 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_warn_on_insecure_global_id_reclaim_allowed", 2024-04-07T16:16:10.578 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_warn_on_legacy_crush_tunables", 2024-04-07T16:16:10.578 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_warn_on_osd_down_out_interval_zero", 2024-04-07T16:16:10.578 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_warn_on_pool_pg_num_not_power_of_two", 2024-04-07T16:16:10.578 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_warn_on_too_few_osds", 2024-04-07T16:16:10.578 INFO:tasks.workunit.client.0.smithi100.stdout: "ms_die_on_bug", 2024-04-07T16:16:10.578 INFO:tasks.workunit.client.0.smithi100.stdout: "ms_die_on_old_message", 2024-04-07T16:16:10.578 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_class_default_list", 2024-04-07T16:16:10.578 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_class_load_list", 2024-04-07T16:16:10.579 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_debug_misdirected_ops", 2024-04-07T16:16:10.579 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_debug_op_order", 2024-04-07T16:16:10.579 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_debug_pg_log_writeout", 2024-04-07T16:16:10.579 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_debug_shutdown", 2024-04-07T16:16:10.579 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_debug_verify_cached_snaps", 2024-04-07T16:16:10.579 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_debug_verify_missing_on_start", 2024-04-07T16:16:10.579 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_debug_verify_stray_on_activate", 2024-04-07T16:16:10.579 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_deep_scrub_update_digest_min_age", 2024-04-07T16:16:10.579 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mclock_max_capacity_iops_ssd", 2024-04-07T16:16:10.579 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mclock_profile", 2024-04-07T16:16:10.579 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_memory_target", 2024-04-07T16:16:10.579 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_memory_target_autotune", 2024-04-07T16:16:10.579 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_op_queue", 2024-04-07T16:16:10.579 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_op_queue_cut_off", 2024-04-07T16:16:10.580 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_pool_default_erasure_code_profile", 2024-04-07T16:16:10.580 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_pool_default_pg_autoscale_mode", 2024-04-07T16:16:10.580 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_recover_clone_overlap", 2024-04-07T16:16:10.580 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_recovery_max_chunk", 2024-04-07T16:16:10.580 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_scrub_load_threshold", 2024-04-07T16:16:10.580 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_scrub_max_interval", 2024-04-07T16:16:10.580 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_shutdown_pgref_assert", 2024-04-07T16:16:10.580 INFO:tasks.workunit.client.0.smithi100.stdout: "public_network", 2024-04-07T16:16:10.580 INFO:tasks.workunit.client.0.smithi100.stdout: "rgw_cache_enabled", 2024-04-07T16:16:10.580 INFO:tasks.workunit.client.0.smithi100.stdout: "rgw_enable_ops_log", 2024-04-07T16:16:10.580 INFO:tasks.workunit.client.0.smithi100.stdout: "rgw_enable_usage_log" 2024-04-07T16:16:10.580 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T16:16:10.580 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:10.580 INFO:tasks.workunit.client.0.smithi100.stdout: "contact": null, 2024-04-07T16:16:10.581 INFO:tasks.workunit.client.0.smithi100.stdout: "crashes": [], 2024-04-07T16:16:10.581 INFO:tasks.workunit.client.0.smithi100.stdout: "created": "2024-04-07T16:02:17.280630Z", 2024-04-07T16:16:10.581 INFO:tasks.workunit.client.0.smithi100.stdout: "crush": { 2024-04-07T16:16:10.581 INFO:tasks.workunit.client.0.smithi100.stdout: "bucket_algs": { 2024-04-07T16:16:10.581 INFO:tasks.workunit.client.0.smithi100.stdout: "straw2": 3 2024-04-07T16:16:10.581 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:10.581 INFO:tasks.workunit.client.0.smithi100.stdout: "bucket_sizes": { 2024-04-07T16:16:10.581 INFO:tasks.workunit.client.0.smithi100.stdout: "2": 1, 2024-04-07T16:16:10.581 INFO:tasks.workunit.client.0.smithi100.stdout: "4": 2 2024-04-07T16:16:10.581 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:10.581 INFO:tasks.workunit.client.0.smithi100.stdout: "bucket_types": { 2024-04-07T16:16:10.581 INFO:tasks.workunit.client.0.smithi100.stdout: "1": 2, 2024-04-07T16:16:10.581 INFO:tasks.workunit.client.0.smithi100.stdout: "11": 1 2024-04-07T16:16:10.581 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:10.581 INFO:tasks.workunit.client.0.smithi100.stdout: "compat_weight_set": false, 2024-04-07T16:16:10.582 INFO:tasks.workunit.client.0.smithi100.stdout: "device_classes": [ 2024-04-07T16:16:10.582 INFO:tasks.workunit.client.0.smithi100.stdout: 8 2024-04-07T16:16:10.582 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:10.582 INFO:tasks.workunit.client.0.smithi100.stdout: "num_buckets": 6, 2024-04-07T16:16:10.582 INFO:tasks.workunit.client.0.smithi100.stdout: "num_devices": 8, 2024-04-07T16:16:10.582 INFO:tasks.workunit.client.0.smithi100.stdout: "num_rules": 1, 2024-04-07T16:16:10.582 INFO:tasks.workunit.client.0.smithi100.stdout: "num_types": 12, 2024-04-07T16:16:10.582 INFO:tasks.workunit.client.0.smithi100.stdout: "num_weight_sets": 0, 2024-04-07T16:16:10.582 INFO:tasks.workunit.client.0.smithi100.stdout: "tunables": { 2024-04-07T16:16:10.582 INFO:tasks.workunit.client.0.smithi100.stdout: "allowed_bucket_algs": 54, 2024-04-07T16:16:10.582 INFO:tasks.workunit.client.0.smithi100.stdout: "choose_local_fallback_tries": 0, 2024-04-07T16:16:10.582 INFO:tasks.workunit.client.0.smithi100.stdout: "choose_local_tries": 0, 2024-04-07T16:16:10.582 INFO:tasks.workunit.client.0.smithi100.stdout: "choose_total_tries": 50, 2024-04-07T16:16:10.583 INFO:tasks.workunit.client.0.smithi100.stdout: "chooseleaf_descend_once": 1, 2024-04-07T16:16:10.583 INFO:tasks.workunit.client.0.smithi100.stdout: "chooseleaf_stable": 1, 2024-04-07T16:16:10.583 INFO:tasks.workunit.client.0.smithi100.stdout: "chooseleaf_vary_r": 1, 2024-04-07T16:16:10.583 INFO:tasks.workunit.client.0.smithi100.stdout: "has_v2_rules": 0, 2024-04-07T16:16:10.583 INFO:tasks.workunit.client.0.smithi100.stdout: "has_v3_rules": 0, 2024-04-07T16:16:10.583 INFO:tasks.workunit.client.0.smithi100.stdout: "has_v4_buckets": 1, 2024-04-07T16:16:10.583 INFO:tasks.workunit.client.0.smithi100.stdout: "has_v5_rules": 0, 2024-04-07T16:16:10.583 INFO:tasks.workunit.client.0.smithi100.stdout: "legacy_tunables": 0, 2024-04-07T16:16:10.583 INFO:tasks.workunit.client.0.smithi100.stdout: "minimum_required_version": "jewel", 2024-04-07T16:16:10.583 INFO:tasks.workunit.client.0.smithi100.stdout: "optimal_tunables": 1, 2024-04-07T16:16:10.583 INFO:tasks.workunit.client.0.smithi100.stdout: "profile": "jewel", 2024-04-07T16:16:10.583 INFO:tasks.workunit.client.0.smithi100.stdout: "require_feature_tunables": 1, 2024-04-07T16:16:10.583 INFO:tasks.workunit.client.0.smithi100.stdout: "require_feature_tunables2": 1, 2024-04-07T16:16:10.583 INFO:tasks.workunit.client.0.smithi100.stdout: "require_feature_tunables3": 1, 2024-04-07T16:16:10.583 INFO:tasks.workunit.client.0.smithi100.stdout: "require_feature_tunables5": 1, 2024-04-07T16:16:10.584 INFO:tasks.workunit.client.0.smithi100.stdout: "straw_calc_version": 1 2024-04-07T16:16:10.584 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:10.584 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:10.584 INFO:tasks.workunit.client.0.smithi100.stdout: "description": "upgrade test cluster", 2024-04-07T16:16:10.584 INFO:tasks.workunit.client.0.smithi100.stdout: "fs": { 2024-04-07T16:16:10.584 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1, 2024-04-07T16:16:10.584 INFO:tasks.workunit.client.0.smithi100.stdout: "feature_flags": { 2024-04-07T16:16:10.584 INFO:tasks.workunit.client.0.smithi100.stdout: "enable_multiple": true, 2024-04-07T16:16:10.584 INFO:tasks.workunit.client.0.smithi100.stdout: "ever_enabled_multiple": true 2024-04-07T16:16:10.584 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:10.584 INFO:tasks.workunit.client.0.smithi100.stdout: "filesystems": [ 2024-04-07T16:16:10.584 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:10.584 INFO:tasks.workunit.client.0.smithi100.stdout: "approx_ctime": "2024-04", 2024-04-07T16:16:10.584 INFO:tasks.workunit.client.0.smithi100.stdout: "balancer_enabled": false, 2024-04-07T16:16:10.585 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:10.585 INFO:tasks.workunit.client.0.smithi100.stdout: "cached_caps": 0, 2024-04-07T16:16:10.585 INFO:tasks.workunit.client.0.smithi100.stdout: "cached_dns": 10, 2024-04-07T16:16:10.585 INFO:tasks.workunit.client.0.smithi100.stdout: "cached_inos": 13, 2024-04-07T16:16:10.585 INFO:tasks.workunit.client.0.smithi100.stdout: "cached_subtrees": 2, 2024-04-07T16:16:10.585 INFO:tasks.workunit.client.0.smithi100.stdout: "ever_allowed_features": 0, 2024-04-07T16:16:10.585 INFO:tasks.workunit.client.0.smithi100.stdout: "explicitly_allowed_features": 0, 2024-04-07T16:16:10.585 INFO:tasks.workunit.client.0.smithi100.stdout: "files": 0, 2024-04-07T16:16:10.585 INFO:tasks.workunit.client.0.smithi100.stdout: "max_mds": 1, 2024-04-07T16:16:10.585 INFO:tasks.workunit.client.0.smithi100.stdout: "num_data_pools": 1, 2024-04-07T16:16:10.585 INFO:tasks.workunit.client.0.smithi100.stdout: "num_in": 1, 2024-04-07T16:16:10.585 INFO:tasks.workunit.client.0.smithi100.stdout: "num_mds": 1, 2024-04-07T16:16:10.585 INFO:tasks.workunit.client.0.smithi100.stdout: "num_sessions": 0, 2024-04-07T16:16:10.585 INFO:tasks.workunit.client.0.smithi100.stdout: "num_standby_replay": 0, 2024-04-07T16:16:10.586 INFO:tasks.workunit.client.0.smithi100.stdout: "num_up": 1, 2024-04-07T16:16:10.586 INFO:tasks.workunit.client.0.smithi100.stdout: "snaps": 0, 2024-04-07T16:16:10.586 INFO:tasks.workunit.client.0.smithi100.stdout: "standby_count_wanted": 0 2024-04-07T16:16:10.586 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:10.586 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:10.586 INFO:tasks.workunit.client.0.smithi100.stdout: "num_standby_mds": 1, 2024-04-07T16:16:10.586 INFO:tasks.workunit.client.0.smithi100.stdout: "total_num_mds": 2 2024-04-07T16:16:10.586 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:10.586 INFO:tasks.workunit.client.0.smithi100.stdout: "hosts": { 2024-04-07T16:16:10.586 INFO:tasks.workunit.client.0.smithi100.stdout: "num": 2, 2024-04-07T16:16:10.586 INFO:tasks.workunit.client.0.smithi100.stdout: "num_with_mds": 2, 2024-04-07T16:16:10.586 INFO:tasks.workunit.client.0.smithi100.stdout: "num_with_mgr": 2, 2024-04-07T16:16:10.586 INFO:tasks.workunit.client.0.smithi100.stdout: "num_with_mon": 2, 2024-04-07T16:16:10.587 INFO:tasks.workunit.client.0.smithi100.stdout: "num_with_osd": 2 2024-04-07T16:16:10.587 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:10.587 INFO:tasks.workunit.client.0.smithi100.stdout: "leaderboard": false, 2024-04-07T16:16:10.587 INFO:tasks.workunit.client.0.smithi100.stdout: "leaderboard_description": null, 2024-04-07T16:16:10.587 INFO:tasks.workunit.client.0.smithi100.stdout: "license": "sharing-1-0", 2024-04-07T16:16:10.587 INFO:tasks.workunit.client.0.smithi100.stdout: "metadata": { 2024-04-07T16:16:10.587 INFO:tasks.workunit.client.0.smithi100.stdout: "mds": { 2024-04-07T16:16:10.587 INFO:tasks.workunit.client.0.smithi100.stdout: "arch": { 2024-04-07T16:16:10.587 INFO:tasks.workunit.client.0.smithi100.stdout: "x86_64": 2 2024-04-07T16:16:10.587 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:10.587 INFO:tasks.workunit.client.0.smithi100.stdout: "ceph_version": { 2024-04-07T16:16:10.587 INFO:tasks.workunit.client.0.smithi100.stdout: "ceph version 18.2.2-1171-g267c8ff5 (267c8ff5b1ff5815581b3b2381d19360e081b37b) reef (stable)": 2 2024-04-07T16:16:10.587 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:10.588 INFO:tasks.workunit.client.0.smithi100.stdout: "cpu": { 2024-04-07T16:16:10.588 INFO:tasks.workunit.client.0.smithi100.stdout: "Intel(R) Xeon(R) CPU E5-1620 v4 @ 3.50GHz": 2 2024-04-07T16:16:10.588 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:10.588 INFO:tasks.workunit.client.0.smithi100.stdout: "distro": { 2024-04-07T16:16:10.588 INFO:tasks.workunit.client.0.smithi100.stdout: "centos": 2 2024-04-07T16:16:10.588 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:10.588 INFO:tasks.workunit.client.0.smithi100.stdout: "distro_description": { 2024-04-07T16:16:10.588 INFO:tasks.workunit.client.0.smithi100.stdout: "CentOS Stream 8": 2 2024-04-07T16:16:10.588 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:10.588 INFO:tasks.workunit.client.0.smithi100.stdout: "kernel_description": { 2024-04-07T16:16:10.588 INFO:tasks.workunit.client.0.smithi100.stdout: "#111-Ubuntu SMP Tue Mar 5 20:16:58 UTC 2024": 2 2024-04-07T16:16:10.588 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:10.588 INFO:tasks.workunit.client.0.smithi100.stdout: "kernel_version": { 2024-04-07T16:16:10.588 INFO:tasks.workunit.client.0.smithi100.stdout: "5.15.0-101-generic": 2 2024-04-07T16:16:10.589 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:10.589 INFO:tasks.workunit.client.0.smithi100.stdout: "os": { 2024-04-07T16:16:10.589 INFO:tasks.workunit.client.0.smithi100.stdout: "Linux": 2 2024-04-07T16:16:10.589 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:10.589 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:10.589 INFO:tasks.workunit.client.0.smithi100.stdout: "mon": { 2024-04-07T16:16:10.589 INFO:tasks.workunit.client.0.smithi100.stdout: "arch": { 2024-04-07T16:16:10.589 INFO:tasks.workunit.client.0.smithi100.stdout: "x86_64": 3 2024-04-07T16:16:10.589 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:10.589 INFO:tasks.workunit.client.0.smithi100.stdout: "ceph_version": { 2024-04-07T16:16:10.589 INFO:tasks.workunit.client.0.smithi100.stdout: "ceph version 18.2.2-1171-g267c8ff5 (267c8ff5b1ff5815581b3b2381d19360e081b37b) reef (stable)": 3 2024-04-07T16:16:10.589 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:10.590 INFO:tasks.workunit.client.0.smithi100.stdout: "cpu": { 2024-04-07T16:16:10.590 INFO:tasks.workunit.client.0.smithi100.stdout: "Intel(R) Xeon(R) CPU E5-1620 v4 @ 3.50GHz": 3 2024-04-07T16:16:10.590 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:10.590 INFO:tasks.workunit.client.0.smithi100.stdout: "distro": { 2024-04-07T16:16:10.590 INFO:tasks.workunit.client.0.smithi100.stdout: "centos": 3 2024-04-07T16:16:10.590 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:10.590 INFO:tasks.workunit.client.0.smithi100.stdout: "distro_description": { 2024-04-07T16:16:10.590 INFO:tasks.workunit.client.0.smithi100.stdout: "CentOS Stream 8": 3 2024-04-07T16:16:10.590 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:10.590 INFO:tasks.workunit.client.0.smithi100.stdout: "kernel_description": { 2024-04-07T16:16:10.590 INFO:tasks.workunit.client.0.smithi100.stdout: "#111-Ubuntu SMP Tue Mar 5 20:16:58 UTC 2024": 3 2024-04-07T16:16:10.590 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:10.590 INFO:tasks.workunit.client.0.smithi100.stdout: "kernel_version": { 2024-04-07T16:16:10.591 INFO:tasks.workunit.client.0.smithi100.stdout: "5.15.0-101-generic": 3 2024-04-07T16:16:10.591 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:10.591 INFO:tasks.workunit.client.0.smithi100.stdout: "os": { 2024-04-07T16:16:10.591 INFO:tasks.workunit.client.0.smithi100.stdout: "Linux": 3 2024-04-07T16:16:10.591 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:10.591 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:10.591 INFO:tasks.workunit.client.0.smithi100.stdout: "osd": { 2024-04-07T16:16:10.591 INFO:tasks.workunit.client.0.smithi100.stdout: "arch": { 2024-04-07T16:16:10.591 INFO:tasks.workunit.client.0.smithi100.stdout: "x86_64": 8 2024-04-07T16:16:10.591 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:10.591 INFO:tasks.workunit.client.0.smithi100.stdout: "ceph_version": { 2024-04-07T16:16:10.591 INFO:tasks.workunit.client.0.smithi100.stdout: "ceph version 18.2.2-1171-g267c8ff5 (267c8ff5b1ff5815581b3b2381d19360e081b37b) reef (stable)": 8 2024-04-07T16:16:10.591 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:10.592 INFO:tasks.workunit.client.0.smithi100.stdout: "cpu": { 2024-04-07T16:16:10.592 INFO:tasks.workunit.client.0.smithi100.stdout: "Intel(R) Xeon(R) CPU E5-1620 v4 @ 3.50GHz": 8 2024-04-07T16:16:10.592 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:10.592 INFO:tasks.workunit.client.0.smithi100.stdout: "distro": { 2024-04-07T16:16:10.592 INFO:tasks.workunit.client.0.smithi100.stdout: "centos": 8 2024-04-07T16:16:10.592 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:10.592 INFO:tasks.workunit.client.0.smithi100.stdout: "distro_description": { 2024-04-07T16:16:10.592 INFO:tasks.workunit.client.0.smithi100.stdout: "CentOS Stream 8": 8 2024-04-07T16:16:10.592 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:10.592 INFO:tasks.workunit.client.0.smithi100.stdout: "kernel_description": { 2024-04-07T16:16:10.592 INFO:tasks.workunit.client.0.smithi100.stdout: "#111-Ubuntu SMP Tue Mar 5 20:16:58 UTC 2024": 8 2024-04-07T16:16:10.592 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:10.592 INFO:tasks.workunit.client.0.smithi100.stdout: "kernel_version": { 2024-04-07T16:16:10.592 INFO:tasks.workunit.client.0.smithi100.stdout: "5.15.0-101-generic": 8 2024-04-07T16:16:10.593 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:10.593 INFO:tasks.workunit.client.0.smithi100.stdout: "os": { 2024-04-07T16:16:10.593 INFO:tasks.workunit.client.0.smithi100.stdout: "Linux": 8 2024-04-07T16:16:10.593 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:10.593 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_objectstore": { 2024-04-07T16:16:10.593 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore": 8 2024-04-07T16:16:10.593 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:10.593 INFO:tasks.workunit.client.0.smithi100.stdout: "rotational": { 2024-04-07T16:16:10.593 INFO:tasks.workunit.client.0.smithi100.stdout: "0": 8 2024-04-07T16:16:10.593 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:10.593 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:10.593 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:10.594 INFO:tasks.workunit.client.0.smithi100.stdout: "mon": { 2024-04-07T16:16:10.594 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 3, 2024-04-07T16:16:10.594 INFO:tasks.workunit.client.0.smithi100.stdout: "features": { 2024-04-07T16:16:10.594 INFO:tasks.workunit.client.0.smithi100.stdout: "optional": [], 2024-04-07T16:16:10.594 INFO:tasks.workunit.client.0.smithi100.stdout: "persistent": [ 2024-04-07T16:16:10.594 INFO:tasks.workunit.client.0.smithi100.stdout: "kraken", 2024-04-07T16:16:10.594 INFO:tasks.workunit.client.0.smithi100.stdout: "luminous", 2024-04-07T16:16:10.594 INFO:tasks.workunit.client.0.smithi100.stdout: "mimic", 2024-04-07T16:16:10.594 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap-prune", 2024-04-07T16:16:10.594 INFO:tasks.workunit.client.0.smithi100.stdout: "nautilus", 2024-04-07T16:16:10.594 INFO:tasks.workunit.client.0.smithi100.stdout: "octopus", 2024-04-07T16:16:10.594 INFO:tasks.workunit.client.0.smithi100.stdout: "pacific", 2024-04-07T16:16:10.594 INFO:tasks.workunit.client.0.smithi100.stdout: "elector-pinging", 2024-04-07T16:16:10.594 INFO:tasks.workunit.client.0.smithi100.stdout: "quincy", 2024-04-07T16:16:10.595 INFO:tasks.workunit.client.0.smithi100.stdout: "reef" 2024-04-07T16:16:10.595 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T16:16:10.595 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:10.595 INFO:tasks.workunit.client.0.smithi100.stdout: "ipv4_addr_mons": 6, 2024-04-07T16:16:10.595 INFO:tasks.workunit.client.0.smithi100.stdout: "ipv6_addr_mons": 0, 2024-04-07T16:16:10.595 INFO:tasks.workunit.client.0.smithi100.stdout: "min_mon_release": 18, 2024-04-07T16:16:10.595 INFO:tasks.workunit.client.0.smithi100.stdout: "v1_addr_mons": 3, 2024-04-07T16:16:10.595 INFO:tasks.workunit.client.0.smithi100.stdout: "v2_addr_mons": 3 2024-04-07T16:16:10.595 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:10.595 INFO:tasks.workunit.client.0.smithi100.stdout: "organization": "ceph-qa", 2024-04-07T16:16:10.595 INFO:tasks.workunit.client.0.smithi100.stdout: "osd": { 2024-04-07T16:16:10.595 INFO:tasks.workunit.client.0.smithi100.stdout: "cluster_network": false, 2024-04-07T16:16:10.595 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 8, 2024-04-07T16:16:10.595 INFO:tasks.workunit.client.0.smithi100.stdout: "require_min_compat_client": "luminous", 2024-04-07T16:16:10.596 INFO:tasks.workunit.client.0.smithi100.stdout: "require_osd_release": "reef" 2024-04-07T16:16:10.596 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:10.596 INFO:tasks.workunit.client.0.smithi100.stdout: "pools": [ 2024-04-07T16:16:10.596 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:10.596 INFO:tasks.workunit.client.0.smithi100.stdout: "application": [ 2024-04-07T16:16:10.596 INFO:tasks.workunit.client.0.smithi100.stdout: "mgr" 2024-04-07T16:16:10.596 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:10.596 INFO:tasks.workunit.client.0.smithi100.stdout: "cache_mode": "none", 2024-04-07T16:16:10.596 INFO:tasks.workunit.client.0.smithi100.stdout: "erasure_code_profile": {}, 2024-04-07T16:16:10.596 INFO:tasks.workunit.client.0.smithi100.stdout: "min_size": 2, 2024-04-07T16:16:10.596 INFO:tasks.workunit.client.0.smithi100.stdout: "options": {}, 2024-04-07T16:16:10.596 INFO:tasks.workunit.client.0.smithi100.stdout: "pg_autoscale_mode": "off", 2024-04-07T16:16:10.596 INFO:tasks.workunit.client.0.smithi100.stdout: "pg_num": 1, 2024-04-07T16:16:10.596 INFO:tasks.workunit.client.0.smithi100.stdout: "pgp_num": 1, 2024-04-07T16:16:10.596 INFO:tasks.workunit.client.0.smithi100.stdout: "pool": 1, 2024-04-07T16:16:10.597 INFO:tasks.workunit.client.0.smithi100.stdout: "size": 3, 2024-04-07T16:16:10.597 INFO:tasks.workunit.client.0.smithi100.stdout: "stats": { 2024-04-07T16:16:10.597 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_raw": 729296873738, 2024-04-07T16:16:10.597 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_used": 1781760, 2024-04-07T16:16:10.597 INFO:tasks.workunit.client.0.smithi100.stdout: "compress_bytes_used": 0, 2024-04-07T16:16:10.597 INFO:tasks.workunit.client.0.smithi100.stdout: "compress_under_bytes": 0, 2024-04-07T16:16:10.597 INFO:tasks.workunit.client.0.smithi100.stdout: "data_bytes_used": 1781760, 2024-04-07T16:16:10.597 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty": 0, 2024-04-07T16:16:10.597 INFO:tasks.workunit.client.0.smithi100.stdout: "max_avail": 243098959872, 2024-04-07T16:16:10.597 INFO:tasks.workunit.client.0.smithi100.stdout: "objects": 2, 2024-04-07T16:16:10.597 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_bytes_used": 0, 2024-04-07T16:16:10.597 INFO:tasks.workunit.client.0.smithi100.stdout: "percent_used": 2.443114453853923e-06, 2024-04-07T16:16:10.597 INFO:tasks.workunit.client.0.smithi100.stdout: "quota_bytes": 0, 2024-04-07T16:16:10.597 INFO:tasks.workunit.client.0.smithi100.stdout: "quota_objects": 0, 2024-04-07T16:16:10.598 INFO:tasks.workunit.client.0.smithi100.stdout: "rd": 212, 2024-04-07T16:16:10.598 INFO:tasks.workunit.client.0.smithi100.stdout: "rd_bytes": 313344, 2024-04-07T16:16:10.598 INFO:tasks.workunit.client.0.smithi100.stdout: "stored": 590368, 2024-04-07T16:16:10.598 INFO:tasks.workunit.client.0.smithi100.stdout: "stored_data": 590368, 2024-04-07T16:16:10.598 INFO:tasks.workunit.client.0.smithi100.stdout: "stored_omap": 0, 2024-04-07T16:16:10.598 INFO:tasks.workunit.client.0.smithi100.stdout: "stored_raw": 1771104, 2024-04-07T16:16:10.598 INFO:tasks.workunit.client.0.smithi100.stdout: "wr": 179, 2024-04-07T16:16:10.598 INFO:tasks.workunit.client.0.smithi100.stdout: "wr_bytes": 2473984 2024-04-07T16:16:10.598 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:10.598 INFO:tasks.workunit.client.0.smithi100.stdout: "target_max_bytes": 0, 2024-04-07T16:16:10.598 INFO:tasks.workunit.client.0.smithi100.stdout: "target_max_objects": 0, 2024-04-07T16:16:10.598 INFO:tasks.workunit.client.0.smithi100.stdout: "type": "replicated" 2024-04-07T16:16:10.598 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:10.598 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:10.598 INFO:tasks.workunit.client.0.smithi100.stdout: "application": [ 2024-04-07T16:16:10.599 INFO:tasks.workunit.client.0.smithi100.stdout: "cephfs" 2024-04-07T16:16:10.599 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:10.599 INFO:tasks.workunit.client.0.smithi100.stdout: "cache_mode": "none", 2024-04-07T16:16:10.599 INFO:tasks.workunit.client.0.smithi100.stdout: "erasure_code_profile": {}, 2024-04-07T16:16:10.599 INFO:tasks.workunit.client.0.smithi100.stdout: "min_size": 2, 2024-04-07T16:16:10.599 INFO:tasks.workunit.client.0.smithi100.stdout: "options": {}, 2024-04-07T16:16:10.599 INFO:tasks.workunit.client.0.smithi100.stdout: "pg_autoscale_mode": "off", 2024-04-07T16:16:10.599 INFO:tasks.workunit.client.0.smithi100.stdout: "pg_num": 32, 2024-04-07T16:16:10.599 INFO:tasks.workunit.client.0.smithi100.stdout: "pgp_num": 32, 2024-04-07T16:16:10.599 INFO:tasks.workunit.client.0.smithi100.stdout: "pool": 2, 2024-04-07T16:16:10.599 INFO:tasks.workunit.client.0.smithi100.stdout: "size": 3, 2024-04-07T16:16:10.599 INFO:tasks.workunit.client.0.smithi100.stdout: "stats": { 2024-04-07T16:16:10.599 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_raw": 729296873738, 2024-04-07T16:16:10.599 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_used": 98304, 2024-04-07T16:16:10.600 INFO:tasks.workunit.client.0.smithi100.stdout: "compress_bytes_used": 0, 2024-04-07T16:16:10.600 INFO:tasks.workunit.client.0.smithi100.stdout: "compress_under_bytes": 0, 2024-04-07T16:16:10.600 INFO:tasks.workunit.client.0.smithi100.stdout: "data_bytes_used": 98304, 2024-04-07T16:16:10.600 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty": 0, 2024-04-07T16:16:10.600 INFO:tasks.workunit.client.0.smithi100.stdout: "max_avail": 243098959872, 2024-04-07T16:16:10.600 INFO:tasks.workunit.client.0.smithi100.stdout: "objects": 22, 2024-04-07T16:16:10.600 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_bytes_used": 0, 2024-04-07T16:16:10.600 INFO:tasks.workunit.client.0.smithi100.stdout: "percent_used": 1.3479282756634348e-07, 2024-04-07T16:16:10.600 INFO:tasks.workunit.client.0.smithi100.stdout: "quota_bytes": 0, 2024-04-07T16:16:10.600 INFO:tasks.workunit.client.0.smithi100.stdout: "quota_objects": 0, 2024-04-07T16:16:10.600 INFO:tasks.workunit.client.0.smithi100.stdout: "rd": 0, 2024-04-07T16:16:10.600 INFO:tasks.workunit.client.0.smithi100.stdout: "rd_bytes": 0, 2024-04-07T16:16:10.600 INFO:tasks.workunit.client.0.smithi100.stdout: "stored": 3430, 2024-04-07T16:16:10.600 INFO:tasks.workunit.client.0.smithi100.stdout: "stored_data": 3430, 2024-04-07T16:16:10.600 INFO:tasks.workunit.client.0.smithi100.stdout: "stored_omap": 0, 2024-04-07T16:16:10.601 INFO:tasks.workunit.client.0.smithi100.stdout: "stored_raw": 10290, 2024-04-07T16:16:10.601 INFO:tasks.workunit.client.0.smithi100.stdout: "wr": 51, 2024-04-07T16:16:10.601 INFO:tasks.workunit.client.0.smithi100.stdout: "wr_bytes": 20480 2024-04-07T16:16:10.601 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:10.601 INFO:tasks.workunit.client.0.smithi100.stdout: "target_max_bytes": 0, 2024-04-07T16:16:10.601 INFO:tasks.workunit.client.0.smithi100.stdout: "target_max_objects": 0, 2024-04-07T16:16:10.601 INFO:tasks.workunit.client.0.smithi100.stdout: "type": "replicated" 2024-04-07T16:16:10.601 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:10.601 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:10.601 INFO:tasks.workunit.client.0.smithi100.stdout: "application": [ 2024-04-07T16:16:10.601 INFO:tasks.workunit.client.0.smithi100.stdout: "cephfs" 2024-04-07T16:16:10.601 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:10.601 INFO:tasks.workunit.client.0.smithi100.stdout: "cache_mode": "none", 2024-04-07T16:16:10.601 INFO:tasks.workunit.client.0.smithi100.stdout: "erasure_code_profile": {}, 2024-04-07T16:16:10.602 INFO:tasks.workunit.client.0.smithi100.stdout: "min_size": 2, 2024-04-07T16:16:10.602 INFO:tasks.workunit.client.0.smithi100.stdout: "options": {}, 2024-04-07T16:16:10.602 INFO:tasks.workunit.client.0.smithi100.stdout: "pg_autoscale_mode": "off", 2024-04-07T16:16:10.602 INFO:tasks.workunit.client.0.smithi100.stdout: "pg_num": 64, 2024-04-07T16:16:10.602 INFO:tasks.workunit.client.0.smithi100.stdout: "pgp_num": 64, 2024-04-07T16:16:10.602 INFO:tasks.workunit.client.0.smithi100.stdout: "pool": 3, 2024-04-07T16:16:10.602 INFO:tasks.workunit.client.0.smithi100.stdout: "size": 3, 2024-04-07T16:16:10.602 INFO:tasks.workunit.client.0.smithi100.stdout: "stats": { 2024-04-07T16:16:10.602 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_raw": 729296873738, 2024-04-07T16:16:10.602 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_used": 0, 2024-04-07T16:16:10.602 INFO:tasks.workunit.client.0.smithi100.stdout: "compress_bytes_used": 0, 2024-04-07T16:16:10.602 INFO:tasks.workunit.client.0.smithi100.stdout: "compress_under_bytes": 0, 2024-04-07T16:16:10.602 INFO:tasks.workunit.client.0.smithi100.stdout: "data_bytes_used": 0, 2024-04-07T16:16:10.602 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty": 0, 2024-04-07T16:16:10.603 INFO:tasks.workunit.client.0.smithi100.stdout: "max_avail": 243098959872, 2024-04-07T16:16:10.603 INFO:tasks.workunit.client.0.smithi100.stdout: "objects": 0, 2024-04-07T16:16:10.603 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_bytes_used": 0, 2024-04-07T16:16:10.603 INFO:tasks.workunit.client.0.smithi100.stdout: "percent_used": 0.0, 2024-04-07T16:16:10.603 INFO:tasks.workunit.client.0.smithi100.stdout: "quota_bytes": 0, 2024-04-07T16:16:10.603 INFO:tasks.workunit.client.0.smithi100.stdout: "quota_objects": 0, 2024-04-07T16:16:10.603 INFO:tasks.workunit.client.0.smithi100.stdout: "rd": 0, 2024-04-07T16:16:10.603 INFO:tasks.workunit.client.0.smithi100.stdout: "rd_bytes": 0, 2024-04-07T16:16:10.603 INFO:tasks.workunit.client.0.smithi100.stdout: "stored": 0, 2024-04-07T16:16:10.603 INFO:tasks.workunit.client.0.smithi100.stdout: "stored_data": 0, 2024-04-07T16:16:10.603 INFO:tasks.workunit.client.0.smithi100.stdout: "stored_omap": 0, 2024-04-07T16:16:10.603 INFO:tasks.workunit.client.0.smithi100.stdout: "stored_raw": 0, 2024-04-07T16:16:10.603 INFO:tasks.workunit.client.0.smithi100.stdout: "wr": 0, 2024-04-07T16:16:10.603 INFO:tasks.workunit.client.0.smithi100.stdout: "wr_bytes": 0 2024-04-07T16:16:10.603 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:10.604 INFO:tasks.workunit.client.0.smithi100.stdout: "target_max_bytes": 0, 2024-04-07T16:16:10.604 INFO:tasks.workunit.client.0.smithi100.stdout: "target_max_objects": 0, 2024-04-07T16:16:10.604 INFO:tasks.workunit.client.0.smithi100.stdout: "type": "replicated" 2024-04-07T16:16:10.604 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:10.604 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:10.604 INFO:tasks.workunit.client.0.smithi100.stdout: "application": [ 2024-04-07T16:16:10.604 INFO:tasks.workunit.client.0.smithi100.stdout: "rbd" 2024-04-07T16:16:10.604 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:10.604 INFO:tasks.workunit.client.0.smithi100.stdout: "cache_mode": "none", 2024-04-07T16:16:10.604 INFO:tasks.workunit.client.0.smithi100.stdout: "erasure_code_profile": {}, 2024-04-07T16:16:10.604 INFO:tasks.workunit.client.0.smithi100.stdout: "min_size": 2, 2024-04-07T16:16:10.604 INFO:tasks.workunit.client.0.smithi100.stdout: "options": {}, 2024-04-07T16:16:10.604 INFO:tasks.workunit.client.0.smithi100.stdout: "pg_autoscale_mode": "off", 2024-04-07T16:16:10.604 INFO:tasks.workunit.client.0.smithi100.stdout: "pg_num": 8, 2024-04-07T16:16:10.605 INFO:tasks.workunit.client.0.smithi100.stdout: "pgp_num": 8, 2024-04-07T16:16:10.605 INFO:tasks.workunit.client.0.smithi100.stdout: "pool": 4, 2024-04-07T16:16:10.605 INFO:tasks.workunit.client.0.smithi100.stdout: "size": 3, 2024-04-07T16:16:10.605 INFO:tasks.workunit.client.0.smithi100.stdout: "stats": { 2024-04-07T16:16:10.605 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_raw": 729296873738, 2024-04-07T16:16:10.605 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_used": 0, 2024-04-07T16:16:10.605 INFO:tasks.workunit.client.0.smithi100.stdout: "compress_bytes_used": 0, 2024-04-07T16:16:10.605 INFO:tasks.workunit.client.0.smithi100.stdout: "compress_under_bytes": 0, 2024-04-07T16:16:10.605 INFO:tasks.workunit.client.0.smithi100.stdout: "data_bytes_used": 0, 2024-04-07T16:16:10.605 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty": 0, 2024-04-07T16:16:10.605 INFO:tasks.workunit.client.0.smithi100.stdout: "max_avail": 243098959872, 2024-04-07T16:16:10.605 INFO:tasks.workunit.client.0.smithi100.stdout: "objects": 0, 2024-04-07T16:16:10.605 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_bytes_used": 0, 2024-04-07T16:16:10.605 INFO:tasks.workunit.client.0.smithi100.stdout: "percent_used": 0.0, 2024-04-07T16:16:10.605 INFO:tasks.workunit.client.0.smithi100.stdout: "quota_bytes": 0, 2024-04-07T16:16:10.605 INFO:tasks.workunit.client.0.smithi100.stdout: "quota_objects": 0, 2024-04-07T16:16:10.606 INFO:tasks.workunit.client.0.smithi100.stdout: "rd": 0, 2024-04-07T16:16:10.606 INFO:tasks.workunit.client.0.smithi100.stdout: "rd_bytes": 0, 2024-04-07T16:16:10.606 INFO:tasks.workunit.client.0.smithi100.stdout: "stored": 0, 2024-04-07T16:16:10.606 INFO:tasks.workunit.client.0.smithi100.stdout: "stored_data": 0, 2024-04-07T16:16:10.606 INFO:tasks.workunit.client.0.smithi100.stdout: "stored_omap": 0, 2024-04-07T16:16:10.606 INFO:tasks.workunit.client.0.smithi100.stdout: "stored_raw": 0, 2024-04-07T16:16:10.606 INFO:tasks.workunit.client.0.smithi100.stdout: "wr": 0, 2024-04-07T16:16:10.606 INFO:tasks.workunit.client.0.smithi100.stdout: "wr_bytes": 0 2024-04-07T16:16:10.606 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:10.606 INFO:tasks.workunit.client.0.smithi100.stdout: "target_max_bytes": 0, 2024-04-07T16:16:10.606 INFO:tasks.workunit.client.0.smithi100.stdout: "target_max_objects": 0, 2024-04-07T16:16:10.606 INFO:tasks.workunit.client.0.smithi100.stdout: "type": "replicated" 2024-04-07T16:16:10.606 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:10.606 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:10.607 INFO:tasks.workunit.client.0.smithi100.stdout: "rbd": { 2024-04-07T16:16:10.607 INFO:tasks.workunit.client.0.smithi100.stdout: "mirroring_by_pool": [ 2024-04-07T16:16:10.607 INFO:tasks.workunit.client.0.smithi100.stdout: false 2024-04-07T16:16:10.607 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:10.607 INFO:tasks.workunit.client.0.smithi100.stdout: "num_images_by_pool": [ 2024-04-07T16:16:10.607 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T16:16:10.607 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:10.607 INFO:tasks.workunit.client.0.smithi100.stdout: "num_pools": 1 2024-04-07T16:16:10.607 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:10.607 INFO:tasks.workunit.client.0.smithi100.stdout: "report_id": "ee983b25-10fa-4633-b6cf-1c3036b91b52", 2024-04-07T16:16:10.607 INFO:tasks.workunit.client.0.smithi100.stdout: "report_timestamp": "2024-04-07T16:16:10.460588", 2024-04-07T16:16:10.607 INFO:tasks.workunit.client.0.smithi100.stdout: "report_version": 1, 2024-04-07T16:16:10.607 INFO:tasks.workunit.client.0.smithi100.stdout: "rook": { 2024-04-07T16:16:10.607 INFO:tasks.workunit.client.0.smithi100.stdout: "cluster": { 2024-04-07T16:16:10.607 INFO:tasks.workunit.client.0.smithi100.stdout: "external-mode": null, 2024-04-07T16:16:10.608 INFO:tasks.workunit.client.0.smithi100.stdout: "mon": { 2024-04-07T16:16:10.608 INFO:tasks.workunit.client.0.smithi100.stdout: "allow-multiple-per-node": null, 2024-04-07T16:16:10.608 INFO:tasks.workunit.client.0.smithi100.stdout: "count": null, 2024-04-07T16:16:10.608 INFO:tasks.workunit.client.0.smithi100.stdout: "max-id": null, 2024-04-07T16:16:10.608 INFO:tasks.workunit.client.0.smithi100.stdout: "pvc": { 2024-04-07T16:16:10.608 INFO:tasks.workunit.client.0.smithi100.stdout: "enabled": null 2024-04-07T16:16:10.608 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:10.608 INFO:tasks.workunit.client.0.smithi100.stdout: "stretch": { 2024-04-07T16:16:10.608 INFO:tasks.workunit.client.0.smithi100.stdout: "enabled": null 2024-04-07T16:16:10.608 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:10.608 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:10.608 INFO:tasks.workunit.client.0.smithi100.stdout: "network": { 2024-04-07T16:16:10.608 INFO:tasks.workunit.client.0.smithi100.stdout: "provider": null 2024-04-07T16:16:10.608 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:10.608 INFO:tasks.workunit.client.0.smithi100.stdout: "storage": { 2024-04-07T16:16:10.609 INFO:tasks.workunit.client.0.smithi100.stdout: "device-set": { 2024-04-07T16:16:10.609 INFO:tasks.workunit.client.0.smithi100.stdout: "count": { 2024-04-07T16:16:10.609 INFO:tasks.workunit.client.0.smithi100.stdout: "non-portable": null, 2024-04-07T16:16:10.609 INFO:tasks.workunit.client.0.smithi100.stdout: "portable": null, 2024-04-07T16:16:10.609 INFO:tasks.workunit.client.0.smithi100.stdout: "total": null 2024-04-07T16:16:10.609 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:10.609 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:10.609 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:10.609 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:10.609 INFO:tasks.workunit.client.0.smithi100.stdout: "csi": { 2024-04-07T16:16:10.609 INFO:tasks.workunit.client.0.smithi100.stdout: "version": null 2024-04-07T16:16:10.609 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:10.609 INFO:tasks.workunit.client.0.smithi100.stdout: "kubernetes": { 2024-04-07T16:16:10.609 INFO:tasks.workunit.client.0.smithi100.stdout: "version": null 2024-04-07T16:16:10.610 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:10.610 INFO:tasks.workunit.client.0.smithi100.stdout: "node": { 2024-04-07T16:16:10.610 INFO:tasks.workunit.client.0.smithi100.stdout: "count": { 2024-04-07T16:16:10.610 INFO:tasks.workunit.client.0.smithi100.stdout: "kubernetes-total": null, 2024-04-07T16:16:10.610 INFO:tasks.workunit.client.0.smithi100.stdout: "with-ceph-daemons": null, 2024-04-07T16:16:10.610 INFO:tasks.workunit.client.0.smithi100.stdout: "with-csi-cephfs-plugin": null, 2024-04-07T16:16:10.610 INFO:tasks.workunit.client.0.smithi100.stdout: "with-csi-nfs-plugin": null, 2024-04-07T16:16:10.610 INFO:tasks.workunit.client.0.smithi100.stdout: "with-csi-rbd-plugin": null 2024-04-07T16:16:10.610 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:10.610 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:10.610 INFO:tasks.workunit.client.0.smithi100.stdout: "usage": { 2024-04-07T16:16:10.610 INFO:tasks.workunit.client.0.smithi100.stdout: "storage-class": { 2024-04-07T16:16:10.610 INFO:tasks.workunit.client.0.smithi100.stdout: "count": { 2024-04-07T16:16:10.610 INFO:tasks.workunit.client.0.smithi100.stdout: "bucket": null, 2024-04-07T16:16:10.610 INFO:tasks.workunit.client.0.smithi100.stdout: "cephfs": null, 2024-04-07T16:16:10.611 INFO:tasks.workunit.client.0.smithi100.stdout: "nfs": null, 2024-04-07T16:16:10.611 INFO:tasks.workunit.client.0.smithi100.stdout: "rbd": null, 2024-04-07T16:16:10.611 INFO:tasks.workunit.client.0.smithi100.stdout: "total": null 2024-04-07T16:16:10.611 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:10.611 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:10.611 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:10.611 INFO:tasks.workunit.client.0.smithi100.stdout: "version": null 2024-04-07T16:16:10.611 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:10.611 INFO:tasks.workunit.client.0.smithi100.stdout: "services": {}, 2024-04-07T16:16:10.611 INFO:tasks.workunit.client.0.smithi100.stdout: "usage": { 2024-04-07T16:16:10.611 INFO:tasks.workunit.client.0.smithi100.stdout: "pg_num": 105, 2024-04-07T16:16:10.611 INFO:tasks.workunit.client.0.smithi100.stdout: "pools": 4, 2024-04-07T16:16:10.611 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_by_class": { 2024-04-07T16:16:10.611 INFO:tasks.workunit.client.0.smithi100.stdout: "ssd": { 2024-04-07T16:16:10.611 INFO:tasks.workunit.client.0.smithi100.stdout: "total_avail_bytes": 767718821888, 2024-04-07T16:16:10.612 INFO:tasks.workunit.client.0.smithi100.stdout: "total_bytes": 767960285184, 2024-04-07T16:16:10.612 INFO:tasks.workunit.client.0.smithi100.stdout: "total_used_bytes": 241463296, 2024-04-07T16:16:10.612 INFO:tasks.workunit.client.0.smithi100.stdout: "total_used_raw_bytes": 241463296, 2024-04-07T16:16:10.612 INFO:tasks.workunit.client.0.smithi100.stdout: "total_used_raw_ratio": 0.000314421602524817 2024-04-07T16:16:10.612 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:10.612 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:10.612 INFO:tasks.workunit.client.0.smithi100.stdout: "total_avail_bytes": 767718821888, 2024-04-07T16:16:10.612 INFO:tasks.workunit.client.0.smithi100.stdout: "total_bytes": 767960285184, 2024-04-07T16:16:10.612 INFO:tasks.workunit.client.0.smithi100.stdout: "total_used_bytes": 241463296 2024-04-07T16:16:10.612 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:10.612 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:10.612 INFO:tasks.workunit.client.0.smithi100.stdout:} 2024-04-07T16:16:10.613 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:10 smithi144 bash[23522]: cluster 2024-04-07T16:16:08.674726+0000 mgr.y (mgr.14553) 135 : cluster 0 pgmap v105: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:16:10.613 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:10 smithi144 bash[23522]: audit 2024-04-07T16:16:09.122201+0000 mgr.y (mgr.14553) 136 : audit 0 from='client.24797 -' entity='client.admin' cmd=[{"prefix": "telemetry preview", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:16:10.613 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:10 smithi144 bash[23522]: audit 2024-04-07T16:16:09.291573+0000 mon.b (mon.1) 108 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:16:10.613 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:10 smithi144 bash[23522]: audit 2024-04-07T16:16:09.292272+0000 mgr.y (mgr.14553) 137 : audit 0 from='client.24797 -' entity='client.admin' cmd=[{"prefix": "telemetry preview", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:16:10.613 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:10 smithi144 bash[23522]: audit 2024-04-07T16:16:09.298809+0000 mon.b (mon.1) 109 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config-key dump", "format": "json"}]: dispatch 2024-04-07T16:16:10.613 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:10 smithi144 bash[23522]: audit 2024-04-07T16:16:09.691037+0000 mon.a (mon.0) 849 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:16:10.613 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:10 smithi144 bash[23522]: audit 2024-04-07T16:16:09.700558+0000 mon.a (mon.0) 850 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:16:10.613 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:10 smithi144 bash[23522]: audit 2024-04-07T16:16:09.721969+0000 mon.a (mon.0) 851 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:16:10.614 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:10 smithi144 bash[23522]: audit 2024-04-07T16:16:09.731525+0000 mon.a (mon.0) 852 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:16:10.614 INFO:tasks.workunit.client.0.smithi100.stderr:++ ceph telemetry collection ls 2024-04-07T16:16:10.687 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:10 smithi100 bash[31660]: cluster 2024-04-07T16:16:08.674726+0000 mgr.y (mgr.14553) 135 : cluster 0 pgmap v105: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:16:10.688 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:10 smithi100 bash[31660]: audit 2024-04-07T16:16:09.122201+0000 mgr.y (mgr.14553) 136 : audit 0 from='client.24797 -' entity='client.admin' cmd=[{"prefix": "telemetry preview", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:16:10.688 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:10 smithi100 bash[31660]: audit 2024-04-07T16:16:09.291573+0000 mon.b (mon.1) 108 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:16:10.688 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:10 smithi100 bash[31660]: audit 2024-04-07T16:16:09.292272+0000 mgr.y (mgr.14553) 137 : audit 0 from='client.24797 -' entity='client.admin' cmd=[{"prefix": "telemetry preview", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:16:10.688 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:10 smithi100 bash[31660]: audit 2024-04-07T16:16:09.298809+0000 mon.b (mon.1) 109 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config-key dump", "format": "json"}]: dispatch 2024-04-07T16:16:10.688 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:10 smithi100 bash[31660]: audit 2024-04-07T16:16:09.691037+0000 mon.a (mon.0) 849 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:16:10.689 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:10 smithi100 bash[31660]: audit 2024-04-07T16:16:09.700558+0000 mon.a (mon.0) 850 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:16:10.689 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:10 smithi100 bash[31660]: audit 2024-04-07T16:16:09.721969+0000 mon.a (mon.0) 851 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:16:10.689 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:10 smithi100 bash[31660]: audit 2024-04-07T16:16:09.731525+0000 mon.a (mon.0) 852 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:16:10.689 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:10 smithi100 bash[21346]: cluster 2024-04-07T16:16:08.674726+0000 mgr.y (mgr.14553) 135 : cluster 0 pgmap v105: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:16:10.689 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:10 smithi100 bash[21346]: audit 2024-04-07T16:16:09.122201+0000 mgr.y (mgr.14553) 136 : audit 0 from='client.24797 -' entity='client.admin' cmd=[{"prefix": "telemetry preview", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:16:10.689 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:10 smithi100 bash[21346]: audit 2024-04-07T16:16:09.291573+0000 mon.b (mon.1) 108 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:16:10.689 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:10 smithi100 bash[21346]: audit 2024-04-07T16:16:09.292272+0000 mgr.y (mgr.14553) 137 : audit 0 from='client.24797 -' entity='client.admin' cmd=[{"prefix": "telemetry preview", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:16:10.690 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:10 smithi100 bash[21346]: audit 2024-04-07T16:16:09.298809+0000 mon.b (mon.1) 109 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config-key dump", "format": "json"}]: dispatch 2024-04-07T16:16:10.690 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:10 smithi100 bash[21346]: audit 2024-04-07T16:16:09.691037+0000 mon.a (mon.0) 849 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:16:10.690 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:10 smithi100 bash[21346]: audit 2024-04-07T16:16:09.700558+0000 mon.a (mon.0) 850 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:16:10.690 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:10 smithi100 bash[21346]: audit 2024-04-07T16:16:09.721969+0000 mon.a (mon.0) 851 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:16:10.690 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:10 smithi100 bash[21346]: audit 2024-04-07T16:16:09.731525+0000 mon.a (mon.0) 852 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:16:11.040 INFO:tasks.workunit.client.0.smithi100.stderr:+ COLLECTIONS='New collections are available: 2024-04-07T16:16:11.040 INFO:tasks.workunit.client.0.smithi100.stderr:['\''basic_base'\'', '\''basic_mds_metadata'\'', '\''basic_pool_options_bluestore'\'', '\''basic_pool_usage'\'', '\''basic_rook_v01'\'', '\''basic_usage_by_class'\'', '\''crash_base'\'', '\''device_base'\'', '\''ident_base'\'', '\''perf_memory_metrics'\'', '\''perf_perf'\''] 2024-04-07T16:16:11.040 INFO:tasks.workunit.client.0.smithi100.stderr:Run `ceph telemetry on` to opt-in to these collections. 2024-04-07T16:16:11.040 INFO:tasks.workunit.client.0.smithi100.stderr: 2024-04-07T16:16:11.041 INFO:tasks.workunit.client.0.smithi100.stderr:NAME STATUS DESC 2024-04-07T16:16:11.041 INFO:tasks.workunit.client.0.smithi100.stderr:basic_base NOT REPORTING: NOT OPTED-IN Basic information about the cluster (capacity, number and type of daemons, version, etc.) 2024-04-07T16:16:11.041 INFO:tasks.workunit.client.0.smithi100.stderr:basic_mds_metadata NOT REPORTING: NOT OPTED-IN MDS metadata 2024-04-07T16:16:11.041 INFO:tasks.workunit.client.0.smithi100.stderr:basic_pool_options_bluestore NOT REPORTING: NOT OPTED-IN Per-pool bluestore config options 2024-04-07T16:16:11.041 INFO:tasks.workunit.client.0.smithi100.stderr:basic_pool_usage NOT REPORTING: NOT OPTED-IN Default pool application and usage statistics 2024-04-07T16:16:11.041 INFO:tasks.workunit.client.0.smithi100.stderr:basic_rook_v01 NOT REPORTING: NOT OPTED-IN Basic Rook deployment data 2024-04-07T16:16:11.041 INFO:tasks.workunit.client.0.smithi100.stderr:basic_usage_by_class NOT REPORTING: NOT OPTED-IN Default device class usage statistics 2024-04-07T16:16:11.041 INFO:tasks.workunit.client.0.smithi100.stderr:crash_base NOT REPORTING: NOT OPTED-IN Information about daemon crashes (daemon type and version, backtrace, etc.) 2024-04-07T16:16:11.041 INFO:tasks.workunit.client.0.smithi100.stderr:device_base NOT REPORTING: NOT OPTED-IN Information about device health metrics 2024-04-07T16:16:11.041 INFO:tasks.workunit.client.0.smithi100.stderr:ident_base NOT REPORTING: NOT OPTED-IN User-provided identifying information about the cluster 2024-04-07T16:16:11.041 INFO:tasks.workunit.client.0.smithi100.stderr:perf_memory_metrics NOT REPORTING: NOT OPTED-IN, CHANNEL perf IS OFF Heap stats and mempools for mon and mds 2024-04-07T16:16:11.041 INFO:tasks.workunit.client.0.smithi100.stderr:perf_perf NOT REPORTING: NOT OPTED-IN, CHANNEL perf IS OFF Information about performance counters of the cluster ' 2024-04-07T16:16:11.041 INFO:tasks.workunit.client.0.smithi100.stderr:+ NEW_COLLECTIONS=("perf_perf" "basic_mds_metadata" "basic_pool_usage" "basic_rook_v01" "perf_memory_metrics") 2024-04-07T16:16:11.041 INFO:tasks.workunit.client.0.smithi100.stderr:+ for col in ${NEW_COLLECTIONS[@]} 2024-04-07T16:16:11.042 INFO:tasks.workunit.client.0.smithi100.stderr:+ [[ New collections are available: 2024-04-07T16:16:11.042 INFO:tasks.workunit.client.0.smithi100.stderr:['basic_base', 'basic_mds_metadata', 'basic_pool_options_bluestore', 'basic_pool_usage', 'basic_rook_v01', 'basic_usage_by_class', 'crash_base', 'device_base', 'ident_base', 'perf_memory_metrics', 'perf_perf'] 2024-04-07T16:16:11.042 INFO:tasks.workunit.client.0.smithi100.stderr:Run `ceph telemetry on` to opt-in to these collections. 2024-04-07T16:16:11.042 INFO:tasks.workunit.client.0.smithi100.stderr: 2024-04-07T16:16:11.042 INFO:tasks.workunit.client.0.smithi100.stderr:NAME STATUS DESC 2024-04-07T16:16:11.042 INFO:tasks.workunit.client.0.smithi100.stderr:basic_base NOT REPORTING: NOT OPTED-IN Basic information about the cluster (capacity, number and type of daemons, version, etc.) 2024-04-07T16:16:11.042 INFO:tasks.workunit.client.0.smithi100.stderr:basic_mds_metadata NOT REPORTING: NOT OPTED-IN MDS metadata 2024-04-07T16:16:11.042 INFO:tasks.workunit.client.0.smithi100.stderr:basic_pool_options_bluestore NOT REPORTING: NOT OPTED-IN Per-pool bluestore config options 2024-04-07T16:16:11.042 INFO:tasks.workunit.client.0.smithi100.stderr:basic_pool_usage NOT REPORTING: NOT OPTED-IN Default pool application and usage statistics 2024-04-07T16:16:11.042 INFO:tasks.workunit.client.0.smithi100.stderr:basic_rook_v01 NOT REPORTING: NOT OPTED-IN Basic Rook deployment data 2024-04-07T16:16:11.042 INFO:tasks.workunit.client.0.smithi100.stderr:basic_usage_by_class NOT REPORTING: NOT OPTED-IN Default device class usage statistics 2024-04-07T16:16:11.042 INFO:tasks.workunit.client.0.smithi100.stderr:crash_base NOT REPORTING: NOT OPTED-IN Information about daemon crashes (daemon type and version, backtrace, etc.) 2024-04-07T16:16:11.042 INFO:tasks.workunit.client.0.smithi100.stderr:device_base NOT REPORTING: NOT OPTED-IN Information about device health metrics 2024-04-07T16:16:11.043 INFO:tasks.workunit.client.0.smithi100.stderr:ident_base NOT REPORTING: NOT OPTED-IN User-provided identifying information about the cluster 2024-04-07T16:16:11.043 INFO:tasks.workunit.client.0.smithi100.stderr:perf_memory_metrics NOT REPORTING: NOT OPTED-IN, CHANNEL perf IS OFF Heap stats and mempools for mon and mds 2024-04-07T16:16:11.043 INFO:tasks.workunit.client.0.smithi100.stderr:perf_perf NOT REPORTING: NOT OPTED-IN, CHANNEL perf IS OFF Information about performance counters of the cluster == *perf_perf* ]] 2024-04-07T16:16:11.043 INFO:tasks.workunit.client.0.smithi100.stderr:+ for col in ${NEW_COLLECTIONS[@]} 2024-04-07T16:16:11.043 INFO:tasks.workunit.client.0.smithi100.stderr:+ [[ New collections are available: 2024-04-07T16:16:11.043 INFO:tasks.workunit.client.0.smithi100.stderr:['basic_base', 'basic_mds_metadata', 'basic_pool_options_bluestore', 'basic_pool_usage', 'basic_rook_v01', 'basic_usage_by_class', 'crash_base', 'device_base', 'ident_base', 'perf_memory_metrics', 'perf_perf'] 2024-04-07T16:16:11.043 INFO:tasks.workunit.client.0.smithi100.stderr:Run `ceph telemetry on` to opt-in to these collections. 2024-04-07T16:16:11.043 INFO:tasks.workunit.client.0.smithi100.stderr: 2024-04-07T16:16:11.043 INFO:tasks.workunit.client.0.smithi100.stderr:NAME STATUS DESC 2024-04-07T16:16:11.043 INFO:tasks.workunit.client.0.smithi100.stderr:basic_base NOT REPORTING: NOT OPTED-IN Basic information about the cluster (capacity, number and type of daemons, version, etc.) 2024-04-07T16:16:11.043 INFO:tasks.workunit.client.0.smithi100.stderr:basic_mds_metadata NOT REPORTING: NOT OPTED-IN MDS metadata 2024-04-07T16:16:11.043 INFO:tasks.workunit.client.0.smithi100.stderr:basic_pool_options_bluestore NOT REPORTING: NOT OPTED-IN Per-pool bluestore config options 2024-04-07T16:16:11.043 INFO:tasks.workunit.client.0.smithi100.stderr:basic_pool_usage NOT REPORTING: NOT OPTED-IN Default pool application and usage statistics 2024-04-07T16:16:11.043 INFO:tasks.workunit.client.0.smithi100.stderr:basic_rook_v01 NOT REPORTING: NOT OPTED-IN Basic Rook deployment data 2024-04-07T16:16:11.044 INFO:tasks.workunit.client.0.smithi100.stderr:basic_usage_by_class NOT REPORTING: NOT OPTED-IN Default device class usage statistics 2024-04-07T16:16:11.044 INFO:tasks.workunit.client.0.smithi100.stderr:crash_base NOT REPORTING: NOT OPTED-IN Information about daemon crashes (daemon type and version, backtrace, etc.) 2024-04-07T16:16:11.044 INFO:tasks.workunit.client.0.smithi100.stderr:device_base NOT REPORTING: NOT OPTED-IN Information about device health metrics 2024-04-07T16:16:11.044 INFO:tasks.workunit.client.0.smithi100.stderr:ident_base NOT REPORTING: NOT OPTED-IN User-provided identifying information about the cluster 2024-04-07T16:16:11.044 INFO:tasks.workunit.client.0.smithi100.stderr:perf_memory_metrics NOT REPORTING: NOT OPTED-IN, CHANNEL perf IS OFF Heap stats and mempools for mon and mds 2024-04-07T16:16:11.044 INFO:tasks.workunit.client.0.smithi100.stderr:perf_perf NOT REPORTING: NOT OPTED-IN, CHANNEL perf IS OFF Information about performance counters of the cluster == *basic_mds_metadata* ]] 2024-04-07T16:16:11.044 INFO:tasks.workunit.client.0.smithi100.stderr:+ for col in ${NEW_COLLECTIONS[@]} 2024-04-07T16:16:11.044 INFO:tasks.workunit.client.0.smithi100.stderr:+ [[ New collections are available: 2024-04-07T16:16:11.044 INFO:tasks.workunit.client.0.smithi100.stderr:['basic_base', 'basic_mds_metadata', 'basic_pool_options_bluestore', 'basic_pool_usage', 'basic_rook_v01', 'basic_usage_by_class', 'crash_base', 'device_base', 'ident_base', 'perf_memory_metrics', 'perf_perf'] 2024-04-07T16:16:11.044 INFO:tasks.workunit.client.0.smithi100.stderr:Run `ceph telemetry on` to opt-in to these collections. 2024-04-07T16:16:11.044 INFO:tasks.workunit.client.0.smithi100.stderr: 2024-04-07T16:16:11.044 INFO:tasks.workunit.client.0.smithi100.stderr:NAME STATUS DESC 2024-04-07T16:16:11.044 INFO:tasks.workunit.client.0.smithi100.stderr:basic_base NOT REPORTING: NOT OPTED-IN Basic information about the cluster (capacity, number and type of daemons, version, etc.) 2024-04-07T16:16:11.044 INFO:tasks.workunit.client.0.smithi100.stderr:basic_mds_metadata NOT REPORTING: NOT OPTED-IN MDS metadata 2024-04-07T16:16:11.045 INFO:tasks.workunit.client.0.smithi100.stderr:basic_pool_options_bluestore NOT REPORTING: NOT OPTED-IN Per-pool bluestore config options 2024-04-07T16:16:11.045 INFO:tasks.workunit.client.0.smithi100.stderr:basic_pool_usage NOT REPORTING: NOT OPTED-IN Default pool application and usage statistics 2024-04-07T16:16:11.045 INFO:tasks.workunit.client.0.smithi100.stderr:basic_rook_v01 NOT REPORTING: NOT OPTED-IN Basic Rook deployment data 2024-04-07T16:16:11.045 INFO:tasks.workunit.client.0.smithi100.stderr:basic_usage_by_class NOT REPORTING: NOT OPTED-IN Default device class usage statistics 2024-04-07T16:16:11.045 INFO:tasks.workunit.client.0.smithi100.stderr:crash_base NOT REPORTING: NOT OPTED-IN Information about daemon crashes (daemon type and version, backtrace, etc.) 2024-04-07T16:16:11.045 INFO:tasks.workunit.client.0.smithi100.stderr:device_base NOT REPORTING: NOT OPTED-IN Information about device health metrics 2024-04-07T16:16:11.045 INFO:tasks.workunit.client.0.smithi100.stderr:ident_base NOT REPORTING: NOT OPTED-IN User-provided identifying information about the cluster 2024-04-07T16:16:11.045 INFO:tasks.workunit.client.0.smithi100.stderr:perf_memory_metrics NOT REPORTING: NOT OPTED-IN, CHANNEL perf IS OFF Heap stats and mempools for mon and mds 2024-04-07T16:16:11.045 INFO:tasks.workunit.client.0.smithi100.stderr:perf_perf NOT REPORTING: NOT OPTED-IN, CHANNEL perf IS OFF Information about performance counters of the cluster == *basic_pool_usage* ]] 2024-04-07T16:16:11.045 INFO:tasks.workunit.client.0.smithi100.stderr:+ for col in ${NEW_COLLECTIONS[@]} 2024-04-07T16:16:11.045 INFO:tasks.workunit.client.0.smithi100.stderr:+ [[ New collections are available: 2024-04-07T16:16:11.045 INFO:tasks.workunit.client.0.smithi100.stderr:['basic_base', 'basic_mds_metadata', 'basic_pool_options_bluestore', 'basic_pool_usage', 'basic_rook_v01', 'basic_usage_by_class', 'crash_base', 'device_base', 'ident_base', 'perf_memory_metrics', 'perf_perf'] 2024-04-07T16:16:11.045 INFO:tasks.workunit.client.0.smithi100.stderr:Run `ceph telemetry on` to opt-in to these collections. 2024-04-07T16:16:11.045 INFO:tasks.workunit.client.0.smithi100.stderr: 2024-04-07T16:16:11.045 INFO:tasks.workunit.client.0.smithi100.stderr:NAME STATUS DESC 2024-04-07T16:16:11.046 INFO:tasks.workunit.client.0.smithi100.stderr:basic_base NOT REPORTING: NOT OPTED-IN Basic information about the cluster (capacity, number and type of daemons, version, etc.) 2024-04-07T16:16:11.046 INFO:tasks.workunit.client.0.smithi100.stderr:basic_mds_metadata NOT REPORTING: NOT OPTED-IN MDS metadata 2024-04-07T16:16:11.046 INFO:tasks.workunit.client.0.smithi100.stderr:basic_pool_options_bluestore NOT REPORTING: NOT OPTED-IN Per-pool bluestore config options 2024-04-07T16:16:11.046 INFO:tasks.workunit.client.0.smithi100.stderr:basic_pool_usage NOT REPORTING: NOT OPTED-IN Default pool application and usage statistics 2024-04-07T16:16:11.046 INFO:tasks.workunit.client.0.smithi100.stderr:basic_rook_v01 NOT REPORTING: NOT OPTED-IN Basic Rook deployment data 2024-04-07T16:16:11.046 INFO:tasks.workunit.client.0.smithi100.stderr:basic_usage_by_class NOT REPORTING: NOT OPTED-IN Default device class usage statistics 2024-04-07T16:16:11.046 INFO:tasks.workunit.client.0.smithi100.stderr:crash_base NOT REPORTING: NOT OPTED-IN Information about daemon crashes (daemon type and version, backtrace, etc.) 2024-04-07T16:16:11.046 INFO:tasks.workunit.client.0.smithi100.stderr:device_base NOT REPORTING: NOT OPTED-IN Information about device health metrics 2024-04-07T16:16:11.046 INFO:tasks.workunit.client.0.smithi100.stderr:ident_base NOT REPORTING: NOT OPTED-IN User-provided identifying information about the cluster 2024-04-07T16:16:11.046 INFO:tasks.workunit.client.0.smithi100.stderr:perf_memory_metrics NOT REPORTING: NOT OPTED-IN, CHANNEL perf IS OFF Heap stats and mempools for mon and mds 2024-04-07T16:16:11.046 INFO:tasks.workunit.client.0.smithi100.stderr:perf_perf NOT REPORTING: NOT OPTED-IN, CHANNEL perf IS OFF Information about performance counters of the cluster == *basic_rook_v01* ]] 2024-04-07T16:16:11.046 INFO:tasks.workunit.client.0.smithi100.stderr:+ for col in ${NEW_COLLECTIONS[@]} 2024-04-07T16:16:11.046 INFO:tasks.workunit.client.0.smithi100.stderr:+ [[ New collections are available: 2024-04-07T16:16:11.046 INFO:tasks.workunit.client.0.smithi100.stderr:['basic_base', 'basic_mds_metadata', 'basic_pool_options_bluestore', 'basic_pool_usage', 'basic_rook_v01', 'basic_usage_by_class', 'crash_base', 'device_base', 'ident_base', 'perf_memory_metrics', 'perf_perf'] 2024-04-07T16:16:11.047 INFO:tasks.workunit.client.0.smithi100.stderr:Run `ceph telemetry on` to opt-in to these collections. 2024-04-07T16:16:11.047 INFO:tasks.workunit.client.0.smithi100.stderr: 2024-04-07T16:16:11.047 INFO:tasks.workunit.client.0.smithi100.stderr:NAME STATUS DESC 2024-04-07T16:16:11.047 INFO:tasks.workunit.client.0.smithi100.stderr:basic_base NOT REPORTING: NOT OPTED-IN Basic information about the cluster (capacity, number and type of daemons, version, etc.) 2024-04-07T16:16:11.047 INFO:tasks.workunit.client.0.smithi100.stderr:basic_mds_metadata NOT REPORTING: NOT OPTED-IN MDS metadata 2024-04-07T16:16:11.047 INFO:tasks.workunit.client.0.smithi100.stderr:basic_pool_options_bluestore NOT REPORTING: NOT OPTED-IN Per-pool bluestore config options 2024-04-07T16:16:11.047 INFO:tasks.workunit.client.0.smithi100.stderr:basic_pool_usage NOT REPORTING: NOT OPTED-IN Default pool application and usage statistics 2024-04-07T16:16:11.047 INFO:tasks.workunit.client.0.smithi100.stderr:basic_rook_v01 NOT REPORTING: NOT OPTED-IN Basic Rook deployment data 2024-04-07T16:16:11.047 INFO:tasks.workunit.client.0.smithi100.stderr:basic_usage_by_class NOT REPORTING: NOT OPTED-IN Default device class usage statistics 2024-04-07T16:16:11.047 INFO:tasks.workunit.client.0.smithi100.stderr:crash_base NOT REPORTING: NOT OPTED-IN Information about daemon crashes (daemon type and version, backtrace, etc.) 2024-04-07T16:16:11.047 INFO:tasks.workunit.client.0.smithi100.stderr:device_base NOT REPORTING: NOT OPTED-IN Information about device health metrics 2024-04-07T16:16:11.047 INFO:tasks.workunit.client.0.smithi100.stderr:ident_base NOT REPORTING: NOT OPTED-IN User-provided identifying information about the cluster 2024-04-07T16:16:11.047 INFO:tasks.workunit.client.0.smithi100.stderr:perf_memory_metrics NOT REPORTING: NOT OPTED-IN, CHANNEL perf IS OFF Heap stats and mempools for mon and mds 2024-04-07T16:16:11.048 INFO:tasks.workunit.client.0.smithi100.stderr:perf_perf NOT REPORTING: NOT OPTED-IN, CHANNEL perf IS OFF Information about performance counters of the cluster == *perf_memory_metrics* ]] 2024-04-07T16:16:11.048 INFO:tasks.workunit.client.0.smithi100.stderr:+ ceph telemetry on --license sharing-1-0 2024-04-07T16:16:11.509 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:11 smithi100 bash[21346]: audit 2024-04-07T16:16:09.666793+0000 mgr.y (mgr.14553) 138 : audit 0 from='client.14901 -' entity='client.admin' cmd=[{"prefix": "telemetry preview-device", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:16:11.509 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:11 smithi100 bash[21346]: audit 2024-04-07T16:16:09.876555+0000 mgr.y (mgr.14553) 139 : audit 0 from='client.14901 -' entity='client.admin' cmd=[{"prefix": "telemetry preview-device", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:16:11.510 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:11 smithi100 bash[21346]: audit 2024-04-07T16:16:10.267533+0000 mon.b (mon.1) 110 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:16:11.510 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:11 smithi100 bash[21346]: audit 2024-04-07T16:16:10.268126+0000 mgr.y (mgr.14553) 140 : audit 0 from='client.14907 -' entity='client.admin' cmd=[{"prefix": "telemetry preview-all", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:16:11.510 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:11 smithi100 bash[21346]: audit 2024-04-07T16:16:10.274397+0000 mon.b (mon.1) 111 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config-key dump", "format": "json"}]: dispatch 2024-04-07T16:16:11.510 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:11 smithi100 bash[21346]: audit 2024-04-07T16:16:10.459480+0000 mon.b (mon.1) 112 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:16:11.510 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:11 smithi100 bash[21346]: audit 2024-04-07T16:16:10.460356+0000 mgr.y (mgr.14553) 141 : audit 0 from='client.14907 -' entity='client.admin' cmd=[{"prefix": "telemetry preview-all", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:16:11.510 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:11 smithi100 bash[21346]: audit 2024-04-07T16:16:10.466653+0000 mon.b (mon.1) 113 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config-key dump", "format": "json"}]: dispatch 2024-04-07T16:16:11.510 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:11 smithi100 bash[21346]: cluster 2024-04-07T16:16:10.740250+0000 mon.a (mon.0) 853 : cluster 0 mgrmap e22: y(active, since 3m), standbys: x 2024-04-07T16:16:11.510 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:16:11 smithi100 bash[21601]: [07/Apr/2024:16:16:11] ENGINE Bus STOPPING 2024-04-07T16:16:11.510 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:16:11 smithi100 bash[21601]: [07/Apr/2024:16:16:11] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-04-07T16:16:11.510 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:16:11 smithi100 bash[21601]: [07/Apr/2024:16:16:11] ENGINE Bus STOPPED 2024-04-07T16:16:11.511 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:16:11 smithi100 bash[21601]: [07/Apr/2024:16:16:11] ENGINE Bus STARTING 2024-04-07T16:16:11.511 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:11 smithi100 bash[31660]: audit 2024-04-07T16:16:09.666793+0000 mgr.y (mgr.14553) 138 : audit 0 from='client.14901 -' entity='client.admin' cmd=[{"prefix": "telemetry preview-device", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:16:11.511 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:11 smithi100 bash[31660]: audit 2024-04-07T16:16:09.876555+0000 mgr.y (mgr.14553) 139 : audit 0 from='client.14901 -' entity='client.admin' cmd=[{"prefix": "telemetry preview-device", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:16:11.511 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:11 smithi100 bash[31660]: audit 2024-04-07T16:16:10.267533+0000 mon.b (mon.1) 110 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:16:11.511 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:11 smithi100 bash[31660]: audit 2024-04-07T16:16:10.268126+0000 mgr.y (mgr.14553) 140 : audit 0 from='client.14907 -' entity='client.admin' cmd=[{"prefix": "telemetry preview-all", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:16:11.511 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:11 smithi100 bash[31660]: audit 2024-04-07T16:16:10.274397+0000 mon.b (mon.1) 111 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config-key dump", "format": "json"}]: dispatch 2024-04-07T16:16:11.511 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:11 smithi100 bash[31660]: audit 2024-04-07T16:16:10.459480+0000 mon.b (mon.1) 112 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:16:11.511 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:11 smithi100 bash[31660]: audit 2024-04-07T16:16:10.460356+0000 mgr.y (mgr.14553) 141 : audit 0 from='client.14907 -' entity='client.admin' cmd=[{"prefix": "telemetry preview-all", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:16:11.511 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:11 smithi100 bash[31660]: audit 2024-04-07T16:16:10.466653+0000 mon.b (mon.1) 113 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config-key dump", "format": "json"}]: dispatch 2024-04-07T16:16:11.511 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:11 smithi100 bash[31660]: cluster 2024-04-07T16:16:10.740250+0000 mon.a (mon.0) 853 : cluster 0 mgrmap e22: y(active, since 3m), standbys: x 2024-04-07T16:16:11.586 INFO:tasks.workunit.client.0.smithi100.stdout:Telemetry is on. 2024-04-07T16:16:11.587 INFO:tasks.workunit.client.0.smithi100.stdout:Some channels are disabled, please enable with: 2024-04-07T16:16:11.587 INFO:tasks.workunit.client.0.smithi100.stdout:`ceph telemetry enable channel perf` 2024-04-07T16:16:11.604 INFO:tasks.workunit.client.0.smithi100.stderr:+ ceph telemetry enable channel perf 2024-04-07T16:16:11.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:11 smithi144 bash[23522]: audit 2024-04-07T16:16:09.666793+0000 mgr.y (mgr.14553) 138 : audit 0 from='client.14901 -' entity='client.admin' cmd=[{"prefix": "telemetry preview-device", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:16:11.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:11 smithi144 bash[23522]: audit 2024-04-07T16:16:09.876555+0000 mgr.y (mgr.14553) 139 : audit 0 from='client.14901 -' entity='client.admin' cmd=[{"prefix": "telemetry preview-device", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:16:11.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:11 smithi144 bash[23522]: audit 2024-04-07T16:16:10.267533+0000 mon.b (mon.1) 110 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:16:11.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:11 smithi144 bash[23522]: audit 2024-04-07T16:16:10.268126+0000 mgr.y (mgr.14553) 140 : audit 0 from='client.14907 -' entity='client.admin' cmd=[{"prefix": "telemetry preview-all", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:16:11.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:11 smithi144 bash[23522]: audit 2024-04-07T16:16:10.274397+0000 mon.b (mon.1) 111 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config-key dump", "format": "json"}]: dispatch 2024-04-07T16:16:11.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:11 smithi144 bash[23522]: audit 2024-04-07T16:16:10.459480+0000 mon.b (mon.1) 112 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:16:11.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:11 smithi144 bash[23522]: audit 2024-04-07T16:16:10.460356+0000 mgr.y (mgr.14553) 141 : audit 0 from='client.14907 -' entity='client.admin' cmd=[{"prefix": "telemetry preview-all", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:16:11.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:11 smithi144 bash[23522]: audit 2024-04-07T16:16:10.466653+0000 mon.b (mon.1) 113 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config-key dump", "format": "json"}]: dispatch 2024-04-07T16:16:11.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:11 smithi144 bash[23522]: cluster 2024-04-07T16:16:10.740250+0000 mon.a (mon.0) 853 : cluster 0 mgrmap e22: y(active, since 3m), standbys: x 2024-04-07T16:16:11.937 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:16:11 smithi100 bash[21601]: [07/Apr/2024:16:16:11] ENGINE Serving on http://:::9283 2024-04-07T16:16:11.937 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:16:11 smithi100 bash[21601]: [07/Apr/2024:16:16:11] ENGINE Bus STARTED 2024-04-07T16:16:12.144 INFO:tasks.workunit.client.0.smithi100.stdout:channel_perf is enabled 2024-04-07T16:16:12.171 INFO:tasks.workunit.client.0.smithi100.stderr:++ ceph config get mgr mgr/telemetry/last_opt_revision 2024-04-07T16:16:12.386 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:16:11 smithi100 bash[21601]: [07/Apr/2024:16:16:11] ENGINE Bus STOPPING 2024-04-07T16:16:12.386 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:16:11 smithi100 bash[21601]: [07/Apr/2024:16:16:11] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-04-07T16:16:12.386 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:16:11 smithi100 bash[21601]: [07/Apr/2024:16:16:11] ENGINE Bus STOPPED 2024-04-07T16:16:12.386 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:16:11 smithi100 bash[21601]: [07/Apr/2024:16:16:11] ENGINE Bus STARTING 2024-04-07T16:16:12.386 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:16:12 smithi100 bash[21601]: [07/Apr/2024:16:16:12] ENGINE Serving on http://:::9283 2024-04-07T16:16:12.386 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:16:12 smithi100 bash[21601]: [07/Apr/2024:16:16:12] ENGINE Bus STARTED 2024-04-07T16:16:12.607 INFO:tasks.workunit.client.0.smithi100.stderr:+ LAST_OPT_REVISION=1 2024-04-07T16:16:12.608 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' 1 -ne 1 ']' 2024-04-07T16:16:12.608 INFO:tasks.workunit.client.0.smithi100.stderr:+ ceph telemetry show 2024-04-07T16:16:12.687 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:12 smithi100 bash[31660]: cluster 2024-04-07T16:16:10.675666+0000 mgr.y (mgr.14553) 142 : cluster 0 pgmap v106: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:16:12.688 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:12 smithi100 bash[31660]: audit 2024-04-07T16:16:10.876771+0000 mgr.y (mgr.14553) 143 : audit 0 from='client.14913 -' entity='client.admin' cmd=[{"prefix": "telemetry collection ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:16:12.688 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:12 smithi100 bash[31660]: audit 2024-04-07T16:16:11.033661+0000 mgr.y (mgr.14553) 144 : audit 0 from='client.14913 -' entity='client.admin' cmd=[{"prefix": "telemetry collection ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:16:12.688 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:12 smithi100 bash[31660]: audit 2024-04-07T16:16:11.393996+0000 mgr.y (mgr.14553) 145 : audit 0 from='client.24827 -' entity='client.admin' cmd=[{"prefix": "telemetry on", "license": "sharing-1-0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:16:12.688 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:12 smithi100 bash[31660]: audit 2024-04-07T16:16:11.408256+0000 mon.a (mon.0) 854 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:16:12.688 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:12 smithi100 bash[31660]: audit 2024-04-07T16:16:11.422248+0000 mon.a (mon.0) 855 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:16:12.688 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:12 smithi100 bash[31660]: audit 2024-04-07T16:16:11.429332+0000 mon.a (mon.0) 856 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:16:12.688 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:12 smithi100 bash[31660]: audit 2024-04-07T16:16:11.470039+0000 mon.b (mon.1) 114 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:16:12.688 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:12 smithi100 bash[31660]: audit 2024-04-07T16:16:11.471511+0000 mon.b (mon.1) 115 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:16:12.688 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:12 smithi100 bash[31660]: audit 2024-04-07T16:16:11.472880+0000 mon.b (mon.1) 116 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:16:12.688 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:12 smithi100 bash[31660]: audit 2024-04-07T16:16:11.474151+0000 mon.b (mon.1) 117 : audit 1 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:16:12.688 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:12 smithi100 bash[31660]: audit 2024-04-07T16:16:11.479824+0000 mon.b (mon.1) 118 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config-key dump", "format": "json"}]: dispatch 2024-04-07T16:16:12.688 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:12 smithi100 bash[31660]: audit 2024-04-07T16:16:11.493610+0000 mon.a (mon.0) 857 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:16:12.689 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:12 smithi100 bash[31660]: audit 2024-04-07T16:16:11.585741+0000 mgr.y (mgr.14553) 146 : audit 0 from='client.24827 -' entity='client.admin' cmd=[{"prefix": "telemetry on", "license": "sharing-1-0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:16:12.689 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:12 smithi100 bash[31660]: audit 2024-04-07T16:16:11.594867+0000 mon.a (mon.0) 858 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:16:12.689 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:12 smithi100 bash[31660]: audit 2024-04-07T16:16:11.600914+0000 mon.a (mon.0) 859 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:16:12.689 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:12 smithi100 bash[31660]: audit 2024-04-07T16:16:11.602412+0000 mon.b (mon.1) 119 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:16:12.689 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:12 smithi100 bash[31660]: audit 2024-04-07T16:16:11.606541+0000 mon.b (mon.1) 120 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config-key dump", "format": "json"}]: dispatch 2024-04-07T16:16:12.689 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:12 smithi100 bash[31660]: audit 2024-04-07T16:16:11.973858+0000 mon.a (mon.0) 860 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:16:12.689 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:12 smithi100 bash[31660]: audit 2024-04-07T16:16:11.986919+0000 mon.b (mon.1) 121 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:16:12.689 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:12 smithi100 bash[21346]: cluster 2024-04-07T16:16:10.675666+0000 mgr.y (mgr.14553) 142 : cluster 0 pgmap v106: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:16:12.689 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:12 smithi100 bash[21346]: audit 2024-04-07T16:16:10.876771+0000 mgr.y (mgr.14553) 143 : audit 0 from='client.14913 -' entity='client.admin' cmd=[{"prefix": "telemetry collection ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:16:12.689 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:12 smithi100 bash[21346]: audit 2024-04-07T16:16:11.033661+0000 mgr.y (mgr.14553) 144 : audit 0 from='client.14913 -' entity='client.admin' cmd=[{"prefix": "telemetry collection ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:16:12.690 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:12 smithi100 bash[21346]: audit 2024-04-07T16:16:11.393996+0000 mgr.y (mgr.14553) 145 : audit 0 from='client.24827 -' entity='client.admin' cmd=[{"prefix": "telemetry on", "license": "sharing-1-0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:16:12.690 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:12 smithi100 bash[21346]: audit 2024-04-07T16:16:11.408256+0000 mon.a (mon.0) 854 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:16:12.690 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:12 smithi100 bash[21346]: audit 2024-04-07T16:16:11.422248+0000 mon.a (mon.0) 855 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:16:12.690 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:12 smithi100 bash[21346]: audit 2024-04-07T16:16:11.429332+0000 mon.a (mon.0) 856 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:16:12.690 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:12 smithi100 bash[21346]: audit 2024-04-07T16:16:11.470039+0000 mon.b (mon.1) 114 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:16:12.690 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:12 smithi100 bash[21346]: audit 2024-04-07T16:16:11.471511+0000 mon.b (mon.1) 115 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:16:12.690 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:12 smithi100 bash[21346]: audit 2024-04-07T16:16:11.472880+0000 mon.b (mon.1) 116 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:16:12.690 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:12 smithi100 bash[21346]: audit 2024-04-07T16:16:11.474151+0000 mon.b (mon.1) 117 : audit 1 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:16:12.690 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:12 smithi100 bash[21346]: audit 2024-04-07T16:16:11.479824+0000 mon.b (mon.1) 118 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config-key dump", "format": "json"}]: dispatch 2024-04-07T16:16:12.690 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:12 smithi100 bash[21346]: audit 2024-04-07T16:16:11.493610+0000 mon.a (mon.0) 857 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:16:12.690 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:12 smithi100 bash[21346]: audit 2024-04-07T16:16:11.585741+0000 mgr.y (mgr.14553) 146 : audit 0 from='client.24827 -' entity='client.admin' cmd=[{"prefix": "telemetry on", "license": "sharing-1-0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:16:12.690 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:12 smithi100 bash[21346]: audit 2024-04-07T16:16:11.594867+0000 mon.a (mon.0) 858 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:16:12.690 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:12 smithi100 bash[21346]: audit 2024-04-07T16:16:11.600914+0000 mon.a (mon.0) 859 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:16:12.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:12 smithi100 bash[21346]: audit 2024-04-07T16:16:11.602412+0000 mon.b (mon.1) 119 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:16:12.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:12 smithi100 bash[21346]: audit 2024-04-07T16:16:11.606541+0000 mon.b (mon.1) 120 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config-key dump", "format": "json"}]: dispatch 2024-04-07T16:16:12.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:12 smithi100 bash[21346]: audit 2024-04-07T16:16:11.973858+0000 mon.a (mon.0) 860 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:16:12.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:12 smithi100 bash[21346]: audit 2024-04-07T16:16:11.986919+0000 mon.b (mon.1) 121 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:16:12.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:12 smithi100 bash[31660]: audit 2024-04-07T16:16:11.995208+0000 mon.b (mon.1) 122 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:16:12.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:12 smithi100 bash[31660]: audit 2024-04-07T16:16:12.002222+0000 mon.b (mon.1) 123 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config-key dump", "format": "json"}]: dispatch 2024-04-07T16:16:12.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:12 smithi100 bash[31660]: audit 2024-04-07T16:16:12.157728+0000 mon.b (mon.1) 124 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:16:12.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:12 smithi100 bash[31660]: audit 2024-04-07T16:16:12.158088+0000 mon.b (mon.1) 125 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:16:12.692 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:12 smithi100 bash[31660]: audit 2024-04-07T16:16:12.158478+0000 mon.b (mon.1) 126 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:16:12.692 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:12 smithi100 bash[31660]: audit 2024-04-07T16:16:12.160899+0000 mon.b (mon.1) 127 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:16:12.692 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:12 smithi100 bash[31660]: audit 2024-04-07T16:16:12.161240+0000 mon.b (mon.1) 128 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:16:12.692 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:12 smithi100 bash[31660]: audit 2024-04-07T16:16:12.161548+0000 mon.b (mon.1) 129 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:16:12.692 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:12 smithi100 bash[31660]: audit 2024-04-07T16:16:12.163387+0000 mon.b (mon.1) 130 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:16:12.692 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:12 smithi100 bash[31660]: audit 2024-04-07T16:16:12.163706+0000 mon.b (mon.1) 131 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:16:12.692 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:12 smithi100 bash[31660]: audit 2024-04-07T16:16:12.164004+0000 mon.b (mon.1) 132 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:16:12.692 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:12 smithi100 bash[31660]: audit 2024-04-07T16:16:12.175019+0000 mon.b (mon.1) 133 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:16:12.692 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:12 smithi100 bash[31660]: audit 2024-04-07T16:16:12.175426+0000 mon.b (mon.1) 134 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:16:12.692 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:12 smithi100 bash[31660]: audit 2024-04-07T16:16:12.175750+0000 mon.b (mon.1) 135 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:16:12.692 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:12 smithi100 bash[31660]: audit 2024-04-07T16:16:12.177572+0000 mon.b (mon.1) 136 : audit 1 from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T16:16:12.692 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:12 smithi100 bash[31660]: audit 2024-04-07T16:16:12.177649+0000 mon.b (mon.1) 137 : audit 1 from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T16:16:12.692 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:12 smithi100 bash[31660]: audit 2024-04-07T16:16:12.178197+0000 mon.b (mon.1) 138 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:16:12.692 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:12 smithi100 bash[31660]: audit 2024-04-07T16:16:12.178592+0000 mon.b (mon.1) 139 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:16:12.693 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:12 smithi100 bash[31660]: audit 2024-04-07T16:16:12.179078+0000 mon.b (mon.1) 140 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:16:12.693 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:12 smithi100 bash[31660]: audit 2024-04-07T16:16:12.179612+0000 mon.a (mon.0) 861 : audit 1 from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T16:16:12.693 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:12 smithi100 bash[31660]: audit 2024-04-07T16:16:12.179683+0000 mon.a (mon.0) 862 : audit 1 from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T16:16:12.693 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:12 smithi100 bash[31660]: audit 2024-04-07T16:16:12.181097+0000 mon.b (mon.1) 141 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:16:12.693 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:12 smithi100 bash[31660]: audit 2024-04-07T16:16:12.181456+0000 mon.b (mon.1) 142 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:16:12.693 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:12 smithi100 bash[31660]: audit 2024-04-07T16:16:12.181864+0000 mon.b (mon.1) 143 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:16:12.693 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:12 smithi100 bash[31660]: audit 2024-04-07T16:16:12.185741+0000 mon.c (mon.2) 41 : audit 1 from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T16:16:12.693 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:12 smithi100 bash[31660]: audit 2024-04-07T16:16:12.185801+0000 mon.c (mon.2) 42 : audit 1 from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T16:16:12.693 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:12 smithi100 bash[21346]: audit 2024-04-07T16:16:11.995208+0000 mon.b (mon.1) 122 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:16:12.694 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:12 smithi100 bash[21346]: audit 2024-04-07T16:16:12.002222+0000 mon.b (mon.1) 123 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config-key dump", "format": "json"}]: dispatch 2024-04-07T16:16:12.694 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:12 smithi100 bash[21346]: audit 2024-04-07T16:16:12.157728+0000 mon.b (mon.1) 124 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:16:12.694 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:12 smithi100 bash[21346]: audit 2024-04-07T16:16:12.158088+0000 mon.b (mon.1) 125 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:16:12.694 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:12 smithi100 bash[21346]: audit 2024-04-07T16:16:12.158478+0000 mon.b (mon.1) 126 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:16:12.694 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:12 smithi100 bash[21346]: audit 2024-04-07T16:16:12.160899+0000 mon.b (mon.1) 127 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:16:12.694 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:12 smithi100 bash[21346]: audit 2024-04-07T16:16:12.161240+0000 mon.b (mon.1) 128 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:16:12.694 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:12 smithi100 bash[21346]: audit 2024-04-07T16:16:12.161548+0000 mon.b (mon.1) 129 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:16:12.694 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:12 smithi100 bash[21346]: audit 2024-04-07T16:16:12.163387+0000 mon.b (mon.1) 130 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:16:12.694 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:12 smithi100 bash[21346]: audit 2024-04-07T16:16:12.163706+0000 mon.b (mon.1) 131 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:16:12.694 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:12 smithi100 bash[21346]: audit 2024-04-07T16:16:12.164004+0000 mon.b (mon.1) 132 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:16:12.694 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:12 smithi100 bash[21346]: audit 2024-04-07T16:16:12.175019+0000 mon.b (mon.1) 133 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:16:12.694 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:12 smithi100 bash[21346]: audit 2024-04-07T16:16:12.175426+0000 mon.b (mon.1) 134 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:16:12.694 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:12 smithi100 bash[21346]: audit 2024-04-07T16:16:12.175750+0000 mon.b (mon.1) 135 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:16:12.694 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:12 smithi100 bash[21346]: audit 2024-04-07T16:16:12.177572+0000 mon.b (mon.1) 136 : audit 1 from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T16:16:12.695 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:12 smithi100 bash[21346]: audit 2024-04-07T16:16:12.177649+0000 mon.b (mon.1) 137 : audit 1 from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T16:16:12.695 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:12 smithi100 bash[21346]: audit 2024-04-07T16:16:12.178197+0000 mon.b (mon.1) 138 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:16:12.695 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:12 smithi100 bash[21346]: audit 2024-04-07T16:16:12.178592+0000 mon.b (mon.1) 139 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:16:12.695 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:12 smithi100 bash[21346]: audit 2024-04-07T16:16:12.179078+0000 mon.b (mon.1) 140 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:16:12.695 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:12 smithi100 bash[21346]: audit 2024-04-07T16:16:12.179612+0000 mon.a (mon.0) 861 : audit 1 from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T16:16:12.695 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:12 smithi100 bash[21346]: audit 2024-04-07T16:16:12.179683+0000 mon.a (mon.0) 862 : audit 1 from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T16:16:12.695 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:12 smithi100 bash[21346]: audit 2024-04-07T16:16:12.181097+0000 mon.b (mon.1) 141 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:16:12.695 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:12 smithi100 bash[21346]: audit 2024-04-07T16:16:12.181456+0000 mon.b (mon.1) 142 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:16:12.695 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:12 smithi100 bash[21346]: audit 2024-04-07T16:16:12.181864+0000 mon.b (mon.1) 143 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:16:12.695 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:12 smithi100 bash[21346]: audit 2024-04-07T16:16:12.185741+0000 mon.c (mon.2) 41 : audit 1 from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T16:16:12.695 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:12 smithi100 bash[21346]: audit 2024-04-07T16:16:12.185801+0000 mon.c (mon.2) 42 : audit 1 from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T16:16:12.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:12 smithi144 bash[23522]: cluster 2024-04-07T16:16:10.675666+0000 mgr.y (mgr.14553) 142 : cluster 0 pgmap v106: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:16:12.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:12 smithi144 bash[23522]: audit 2024-04-07T16:16:10.876771+0000 mgr.y (mgr.14553) 143 : audit 0 from='client.14913 -' entity='client.admin' cmd=[{"prefix": "telemetry collection ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:16:12.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:12 smithi144 bash[23522]: audit 2024-04-07T16:16:11.033661+0000 mgr.y (mgr.14553) 144 : audit 0 from='client.14913 -' entity='client.admin' cmd=[{"prefix": "telemetry collection ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:16:12.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:12 smithi144 bash[23522]: audit 2024-04-07T16:16:11.393996+0000 mgr.y (mgr.14553) 145 : audit 0 from='client.24827 -' entity='client.admin' cmd=[{"prefix": "telemetry on", "license": "sharing-1-0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:16:12.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:12 smithi144 bash[23522]: audit 2024-04-07T16:16:11.408256+0000 mon.a (mon.0) 854 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:16:12.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:12 smithi144 bash[23522]: audit 2024-04-07T16:16:11.422248+0000 mon.a (mon.0) 855 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:16:12.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:12 smithi144 bash[23522]: audit 2024-04-07T16:16:11.429332+0000 mon.a (mon.0) 856 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:16:12.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:12 smithi144 bash[23522]: audit 2024-04-07T16:16:11.470039+0000 mon.b (mon.1) 114 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:16:12.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:12 smithi144 bash[23522]: audit 2024-04-07T16:16:11.471511+0000 mon.b (mon.1) 115 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:16:12.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:12 smithi144 bash[23522]: audit 2024-04-07T16:16:11.472880+0000 mon.b (mon.1) 116 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:16:12.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:12 smithi144 bash[23522]: audit 2024-04-07T16:16:11.474151+0000 mon.b (mon.1) 117 : audit 1 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:16:12.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:12 smithi144 bash[23522]: audit 2024-04-07T16:16:11.479824+0000 mon.b (mon.1) 118 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config-key dump", "format": "json"}]: dispatch 2024-04-07T16:16:12.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:12 smithi144 bash[23522]: audit 2024-04-07T16:16:11.493610+0000 mon.a (mon.0) 857 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:16:12.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:12 smithi144 bash[23522]: audit 2024-04-07T16:16:11.585741+0000 mgr.y (mgr.14553) 146 : audit 0 from='client.24827 -' entity='client.admin' cmd=[{"prefix": "telemetry on", "license": "sharing-1-0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:16:12.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:12 smithi144 bash[23522]: audit 2024-04-07T16:16:11.594867+0000 mon.a (mon.0) 858 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:16:12.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:12 smithi144 bash[23522]: audit 2024-04-07T16:16:11.600914+0000 mon.a (mon.0) 859 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:16:12.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:12 smithi144 bash[23522]: audit 2024-04-07T16:16:11.602412+0000 mon.b (mon.1) 119 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:16:12.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:12 smithi144 bash[23522]: audit 2024-04-07T16:16:11.606541+0000 mon.b (mon.1) 120 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config-key dump", "format": "json"}]: dispatch 2024-04-07T16:16:12.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:12 smithi144 bash[23522]: audit 2024-04-07T16:16:11.973858+0000 mon.a (mon.0) 860 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:16:12.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:12 smithi144 bash[23522]: audit 2024-04-07T16:16:11.986919+0000 mon.b (mon.1) 121 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:16:12.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:12 smithi144 bash[23522]: audit 2024-04-07T16:16:11.995208+0000 mon.b (mon.1) 122 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:16:12.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:12 smithi144 bash[23522]: audit 2024-04-07T16:16:12.002222+0000 mon.b (mon.1) 123 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config-key dump", "format": "json"}]: dispatch 2024-04-07T16:16:12.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:12 smithi144 bash[23522]: audit 2024-04-07T16:16:12.157728+0000 mon.b (mon.1) 124 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:16:12.860 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:12 smithi144 bash[23522]: audit 2024-04-07T16:16:12.158088+0000 mon.b (mon.1) 125 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:16:12.860 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:12 smithi144 bash[23522]: audit 2024-04-07T16:16:12.158478+0000 mon.b (mon.1) 126 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:16:12.860 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:12 smithi144 bash[23522]: audit 2024-04-07T16:16:12.160899+0000 mon.b (mon.1) 127 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:16:12.860 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:12 smithi144 bash[23522]: audit 2024-04-07T16:16:12.161240+0000 mon.b (mon.1) 128 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:16:12.860 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:12 smithi144 bash[23522]: audit 2024-04-07T16:16:12.161548+0000 mon.b (mon.1) 129 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:16:12.860 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:12 smithi144 bash[23522]: audit 2024-04-07T16:16:12.163387+0000 mon.b (mon.1) 130 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:16:12.860 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:12 smithi144 bash[23522]: audit 2024-04-07T16:16:12.163706+0000 mon.b (mon.1) 131 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:16:12.860 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:12 smithi144 bash[23522]: audit 2024-04-07T16:16:12.164004+0000 mon.b (mon.1) 132 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:16:12.860 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:12 smithi144 bash[23522]: audit 2024-04-07T16:16:12.175019+0000 mon.b (mon.1) 133 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:16:12.860 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:12 smithi144 bash[23522]: audit 2024-04-07T16:16:12.175426+0000 mon.b (mon.1) 134 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:16:12.860 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:12 smithi144 bash[23522]: audit 2024-04-07T16:16:12.175750+0000 mon.b (mon.1) 135 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:16:12.860 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:12 smithi144 bash[23522]: audit 2024-04-07T16:16:12.177572+0000 mon.b (mon.1) 136 : audit 1 from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T16:16:12.860 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:12 smithi144 bash[23522]: audit 2024-04-07T16:16:12.177649+0000 mon.b (mon.1) 137 : audit 1 from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T16:16:12.860 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:12 smithi144 bash[23522]: audit 2024-04-07T16:16:12.178197+0000 mon.b (mon.1) 138 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:16:12.861 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:12 smithi144 bash[23522]: audit 2024-04-07T16:16:12.178592+0000 mon.b (mon.1) 139 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:16:12.861 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:12 smithi144 bash[23522]: audit 2024-04-07T16:16:12.179078+0000 mon.b (mon.1) 140 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:16:12.861 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:12 smithi144 bash[23522]: audit 2024-04-07T16:16:12.179612+0000 mon.a (mon.0) 861 : audit 1 from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T16:16:12.861 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:12 smithi144 bash[23522]: audit 2024-04-07T16:16:12.179683+0000 mon.a (mon.0) 862 : audit 1 from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T16:16:12.861 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:12 smithi144 bash[23522]: audit 2024-04-07T16:16:12.181097+0000 mon.b (mon.1) 141 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:16:12.861 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:12 smithi144 bash[23522]: audit 2024-04-07T16:16:12.181456+0000 mon.b (mon.1) 142 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:16:12.861 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:12 smithi144 bash[23522]: audit 2024-04-07T16:16:12.181864+0000 mon.b (mon.1) 143 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:16:12.861 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:12 smithi144 bash[23522]: audit 2024-04-07T16:16:12.185741+0000 mon.c (mon.2) 41 : audit 1 from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T16:16:12.861 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:12 smithi144 bash[23522]: audit 2024-04-07T16:16:12.185801+0000 mon.c (mon.2) 42 : audit 1 from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T16:16:13.688 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:13 smithi100 bash[21346]: audit 2024-04-07T16:16:11.964224+0000 mgr.y (mgr.14553) 147 : audit 0 from='client.24833 -' entity='client.admin' cmd=[{"prefix": "telemetry enable channel", "channels": ["perf"], "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:16:13.688 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:13 smithi100 bash[21346]: audit 2024-04-07T16:16:12.158204+0000 mgr.y (mgr.14553) 148 : audit 0 from='client.24833 -' entity='client.admin' cmd=[{"prefix": "telemetry enable channel", "channels": ["perf"], "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:16:13.689 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:13 smithi100 bash[21346]: audit 2024-04-07T16:16:12.506052+0000 mon.a (mon.0) 863 : audit 0 from='client.? 172.21.15.100:0/1004373990' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/telemetry/last_opt_revision"}]: dispatch 2024-04-07T16:16:13.689 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:13 smithi100 bash[21346]: audit 2024-04-07T16:16:12.515678+0000 mon.b (mon.1) 144 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:16:13.689 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:13 smithi100 bash[21346]: audit 2024-04-07T16:16:12.517498+0000 mon.b (mon.1) 145 : audit 1 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:16:13.689 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:13 smithi100 bash[21346]: audit 2024-04-07T16:16:12.529266+0000 mon.a (mon.0) 864 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:16:13.689 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:13 smithi100 bash[21346]: audit 2024-04-07T16:16:12.606371+0000 mon.a (mon.0) 865 : audit 0 from='client.? 172.21.15.100:0/1004373990' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/telemetry/last_opt_revision"}]: dispatch 2024-04-07T16:16:13.689 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:13 smithi100 bash[21346]: audit 2024-04-07T16:16:12.960776+0000 mon.b (mon.1) 146 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:16:13.690 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:13 smithi100 bash[21346]: audit 2024-04-07T16:16:12.968390+0000 mon.b (mon.1) 147 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config-key dump", "format": "json"}]: dispatch 2024-04-07T16:16:13.690 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:13 smithi100 bash[21346]: audit 2024-04-07T16:16:13.175350+0000 mon.b (mon.1) 148 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:16:13.690 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:13 smithi100 bash[21346]: audit 2024-04-07T16:16:13.175795+0000 mon.b (mon.1) 149 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:16:13.690 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:13 smithi100 bash[21346]: audit 2024-04-07T16:16:13.176259+0000 mon.b (mon.1) 150 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:16:13.690 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:13 smithi100 bash[21346]: audit 2024-04-07T16:16:13.179628+0000 mon.b (mon.1) 151 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:16:13.690 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:13 smithi100 bash[21346]: audit 2024-04-07T16:16:13.180052+0000 mon.b (mon.1) 152 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:16:13.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:13 smithi100 bash[21346]: audit 2024-04-07T16:16:13.180591+0000 mon.b (mon.1) 153 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:16:13.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:13 smithi100 bash[21346]: audit 2024-04-07T16:16:13.183655+0000 mon.b (mon.1) 154 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:16:13.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:13 smithi100 bash[21346]: audit 2024-04-07T16:16:13.184023+0000 mon.b (mon.1) 155 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:16:13.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:13 smithi100 bash[21346]: audit 2024-04-07T16:16:13.184475+0000 mon.b (mon.1) 156 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:16:13.692 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:13 smithi100 bash[31660]: audit 2024-04-07T16:16:11.964224+0000 mgr.y (mgr.14553) 147 : audit 0 from='client.24833 -' entity='client.admin' cmd=[{"prefix": "telemetry enable channel", "channels": ["perf"], "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:16:13.692 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:13 smithi100 bash[31660]: audit 2024-04-07T16:16:12.158204+0000 mgr.y (mgr.14553) 148 : audit 0 from='client.24833 -' entity='client.admin' cmd=[{"prefix": "telemetry enable channel", "channels": ["perf"], "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:16:13.692 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:13 smithi100 bash[31660]: audit 2024-04-07T16:16:12.506052+0000 mon.a (mon.0) 863 : audit 0 from='client.? 172.21.15.100:0/1004373990' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/telemetry/last_opt_revision"}]: dispatch 2024-04-07T16:16:13.692 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:13 smithi100 bash[31660]: audit 2024-04-07T16:16:12.515678+0000 mon.b (mon.1) 144 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:16:13.692 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:13 smithi100 bash[31660]: audit 2024-04-07T16:16:12.517498+0000 mon.b (mon.1) 145 : audit 1 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:16:13.693 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:13 smithi100 bash[31660]: audit 2024-04-07T16:16:12.529266+0000 mon.a (mon.0) 864 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:16:13.693 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:13 smithi100 bash[31660]: audit 2024-04-07T16:16:12.606371+0000 mon.a (mon.0) 865 : audit 0 from='client.? 172.21.15.100:0/1004373990' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/telemetry/last_opt_revision"}]: dispatch 2024-04-07T16:16:13.693 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:13 smithi100 bash[31660]: audit 2024-04-07T16:16:12.960776+0000 mon.b (mon.1) 146 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:16:13.693 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:13 smithi100 bash[31660]: audit 2024-04-07T16:16:12.968390+0000 mon.b (mon.1) 147 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config-key dump", "format": "json"}]: dispatch 2024-04-07T16:16:13.693 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:13 smithi100 bash[31660]: audit 2024-04-07T16:16:13.175350+0000 mon.b (mon.1) 148 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:16:13.693 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:13 smithi100 bash[31660]: audit 2024-04-07T16:16:13.175795+0000 mon.b (mon.1) 149 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:16:13.694 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:13 smithi100 bash[31660]: audit 2024-04-07T16:16:13.176259+0000 mon.b (mon.1) 150 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:16:13.694 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:13 smithi100 bash[31660]: audit 2024-04-07T16:16:13.179628+0000 mon.b (mon.1) 151 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:16:13.694 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:13 smithi100 bash[31660]: audit 2024-04-07T16:16:13.180052+0000 mon.b (mon.1) 152 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:16:13.694 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:13 smithi100 bash[31660]: audit 2024-04-07T16:16:13.180591+0000 mon.b (mon.1) 153 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:16:13.694 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:13 smithi100 bash[31660]: audit 2024-04-07T16:16:13.183655+0000 mon.b (mon.1) 154 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:16:13.695 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:13 smithi100 bash[31660]: audit 2024-04-07T16:16:13.184023+0000 mon.b (mon.1) 155 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:16:13.695 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:13 smithi100 bash[31660]: audit 2024-04-07T16:16:13.184475+0000 mon.b (mon.1) 156 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:16:13.695 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:13 smithi100 bash[21346]: audit 2024-04-07T16:16:13.203018+0000 mon.b (mon.1) 157 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:16:13.695 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:13 smithi100 bash[21346]: audit 2024-04-07T16:16:13.203474+0000 mon.b (mon.1) 158 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:16:13.696 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:13 smithi100 bash[21346]: audit 2024-04-07T16:16:13.203908+0000 mon.b (mon.1) 159 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:16:13.696 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:13 smithi100 bash[21346]: audit 2024-04-07T16:16:13.206707+0000 mon.b (mon.1) 160 : audit 1 from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T16:16:13.696 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:13 smithi100 bash[21346]: audit 2024-04-07T16:16:13.206803+0000 mon.b (mon.1) 161 : audit 1 from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T16:16:13.696 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:13 smithi100 bash[21346]: audit 2024-04-07T16:16:13.207372+0000 mon.b (mon.1) 162 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:16:13.696 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:13 smithi100 bash[21346]: audit 2024-04-07T16:16:13.207495+0000 mon.a (mon.0) 866 : audit 1 from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T16:16:13.696 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:13 smithi100 bash[21346]: audit 2024-04-07T16:16:13.207632+0000 mon.a (mon.0) 867 : audit 1 from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T16:16:13.697 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:13 smithi100 bash[21346]: audit 2024-04-07T16:16:13.207844+0000 mon.b (mon.1) 163 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:16:13.697 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:13 smithi100 bash[21346]: audit 2024-04-07T16:16:13.208311+0000 mon.b (mon.1) 164 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:16:13.697 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:13 smithi100 bash[21346]: audit 2024-04-07T16:16:13.211658+0000 mon.b (mon.1) 165 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:16:13.697 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:13 smithi100 bash[21346]: audit 2024-04-07T16:16:13.212075+0000 mon.b (mon.1) 166 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:16:13.697 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:13 smithi100 bash[21346]: audit 2024-04-07T16:16:13.212540+0000 mon.b (mon.1) 167 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:16:13.698 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:13 smithi100 bash[21346]: audit 2024-04-07T16:16:13.216108+0000 mon.c (mon.2) 43 : audit 1 from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T16:16:13.698 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:13 smithi100 bash[21346]: audit 2024-04-07T16:16:13.216265+0000 mon.c (mon.2) 44 : audit 1 from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T16:16:13.698 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:13 smithi100 bash[31660]: audit 2024-04-07T16:16:13.203018+0000 mon.b (mon.1) 157 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:16:13.698 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:13 smithi100 bash[31660]: audit 2024-04-07T16:16:13.203474+0000 mon.b (mon.1) 158 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:16:13.698 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:13 smithi100 bash[31660]: audit 2024-04-07T16:16:13.203908+0000 mon.b (mon.1) 159 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:16:13.699 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:13 smithi100 bash[31660]: audit 2024-04-07T16:16:13.206707+0000 mon.b (mon.1) 160 : audit 1 from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T16:16:13.699 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:13 smithi100 bash[31660]: audit 2024-04-07T16:16:13.206803+0000 mon.b (mon.1) 161 : audit 1 from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T16:16:13.699 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:13 smithi100 bash[31660]: audit 2024-04-07T16:16:13.207372+0000 mon.b (mon.1) 162 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:16:13.699 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:13 smithi100 bash[31660]: audit 2024-04-07T16:16:13.207495+0000 mon.a (mon.0) 866 : audit 1 from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T16:16:13.699 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:13 smithi100 bash[31660]: audit 2024-04-07T16:16:13.207632+0000 mon.a (mon.0) 867 : audit 1 from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T16:16:13.699 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:13 smithi100 bash[31660]: audit 2024-04-07T16:16:13.207844+0000 mon.b (mon.1) 163 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:16:13.700 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:13 smithi100 bash[31660]: audit 2024-04-07T16:16:13.208311+0000 mon.b (mon.1) 164 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:16:13.700 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:13 smithi100 bash[31660]: audit 2024-04-07T16:16:13.211658+0000 mon.b (mon.1) 165 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:16:13.700 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:13 smithi100 bash[31660]: audit 2024-04-07T16:16:13.212075+0000 mon.b (mon.1) 166 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:16:13.700 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:13 smithi100 bash[31660]: audit 2024-04-07T16:16:13.212540+0000 mon.b (mon.1) 167 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:16:13.700 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:13 smithi100 bash[31660]: audit 2024-04-07T16:16:13.216108+0000 mon.c (mon.2) 43 : audit 1 from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T16:16:13.700 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:13 smithi100 bash[31660]: audit 2024-04-07T16:16:13.216265+0000 mon.c (mon.2) 44 : audit 1 from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T16:16:13.856 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:13 smithi144 bash[23522]: audit 2024-04-07T16:16:11.964224+0000 mgr.y (mgr.14553) 147 : audit 0 from='client.24833 -' entity='client.admin' cmd=[{"prefix": "telemetry enable channel", "channels": ["perf"], "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:16:13.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:13 smithi144 bash[23522]: audit 2024-04-07T16:16:12.158204+0000 mgr.y (mgr.14553) 148 : audit 0 from='client.24833 -' entity='client.admin' cmd=[{"prefix": "telemetry enable channel", "channels": ["perf"], "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:16:13.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:13 smithi144 bash[23522]: audit 2024-04-07T16:16:12.506052+0000 mon.a (mon.0) 863 : audit 0 from='client.? 172.21.15.100:0/1004373990' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/telemetry/last_opt_revision"}]: dispatch 2024-04-07T16:16:13.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:13 smithi144 bash[23522]: audit 2024-04-07T16:16:12.515678+0000 mon.b (mon.1) 144 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:16:13.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:13 smithi144 bash[23522]: audit 2024-04-07T16:16:12.517498+0000 mon.b (mon.1) 145 : audit 1 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:16:13.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:13 smithi144 bash[23522]: audit 2024-04-07T16:16:12.529266+0000 mon.a (mon.0) 864 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:16:13.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:13 smithi144 bash[23522]: audit 2024-04-07T16:16:12.606371+0000 mon.a (mon.0) 865 : audit 0 from='client.? 172.21.15.100:0/1004373990' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/telemetry/last_opt_revision"}]: dispatch 2024-04-07T16:16:13.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:13 smithi144 bash[23522]: audit 2024-04-07T16:16:12.960776+0000 mon.b (mon.1) 146 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:16:13.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:13 smithi144 bash[23522]: audit 2024-04-07T16:16:12.968390+0000 mon.b (mon.1) 147 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config-key dump", "format": "json"}]: dispatch 2024-04-07T16:16:13.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:13 smithi144 bash[23522]: audit 2024-04-07T16:16:13.175350+0000 mon.b (mon.1) 148 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:16:13.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:13 smithi144 bash[23522]: audit 2024-04-07T16:16:13.175795+0000 mon.b (mon.1) 149 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:16:13.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:13 smithi144 bash[23522]: audit 2024-04-07T16:16:13.176259+0000 mon.b (mon.1) 150 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:16:13.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:13 smithi144 bash[23522]: audit 2024-04-07T16:16:13.179628+0000 mon.b (mon.1) 151 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:16:13.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:13 smithi144 bash[23522]: audit 2024-04-07T16:16:13.180052+0000 mon.b (mon.1) 152 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:16:13.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:13 smithi144 bash[23522]: audit 2024-04-07T16:16:13.180591+0000 mon.b (mon.1) 153 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:16:13.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:13 smithi144 bash[23522]: audit 2024-04-07T16:16:13.183655+0000 mon.b (mon.1) 154 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:16:13.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:13 smithi144 bash[23522]: audit 2024-04-07T16:16:13.184023+0000 mon.b (mon.1) 155 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:16:13.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:13 smithi144 bash[23522]: audit 2024-04-07T16:16:13.184475+0000 mon.b (mon.1) 156 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:16:13.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:13 smithi144 bash[23522]: audit 2024-04-07T16:16:13.203018+0000 mon.b (mon.1) 157 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:16:13.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:13 smithi144 bash[23522]: audit 2024-04-07T16:16:13.203474+0000 mon.b (mon.1) 158 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:16:13.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:13 smithi144 bash[23522]: audit 2024-04-07T16:16:13.203908+0000 mon.b (mon.1) 159 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:16:13.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:13 smithi144 bash[23522]: audit 2024-04-07T16:16:13.206707+0000 mon.b (mon.1) 160 : audit 1 from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T16:16:13.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:13 smithi144 bash[23522]: audit 2024-04-07T16:16:13.206803+0000 mon.b (mon.1) 161 : audit 1 from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T16:16:13.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:13 smithi144 bash[23522]: audit 2024-04-07T16:16:13.207372+0000 mon.b (mon.1) 162 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:16:13.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:13 smithi144 bash[23522]: audit 2024-04-07T16:16:13.207495+0000 mon.a (mon.0) 866 : audit 1 from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T16:16:13.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:13 smithi144 bash[23522]: audit 2024-04-07T16:16:13.207632+0000 mon.a (mon.0) 867 : audit 1 from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T16:16:13.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:13 smithi144 bash[23522]: audit 2024-04-07T16:16:13.207844+0000 mon.b (mon.1) 163 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:16:13.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:13 smithi144 bash[23522]: audit 2024-04-07T16:16:13.208311+0000 mon.b (mon.1) 164 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:16:13.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:13 smithi144 bash[23522]: audit 2024-04-07T16:16:13.211658+0000 mon.b (mon.1) 165 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:16:13.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:13 smithi144 bash[23522]: audit 2024-04-07T16:16:13.212075+0000 mon.b (mon.1) 166 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:16:13.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:13 smithi144 bash[23522]: audit 2024-04-07T16:16:13.212540+0000 mon.b (mon.1) 167 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:16:13.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:13 smithi144 bash[23522]: audit 2024-04-07T16:16:13.216108+0000 mon.c (mon.2) 43 : audit 1 from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T16:16:13.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:13 smithi144 bash[23522]: audit 2024-04-07T16:16:13.216265+0000 mon.c (mon.2) 44 : audit 1 from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T16:16:13.891 INFO:tasks.workunit.client.0.smithi100.stdout:{ 2024-04-07T16:16:13.892 INFO:tasks.workunit.client.0.smithi100.stdout: "balancer": { 2024-04-07T16:16:13.892 INFO:tasks.workunit.client.0.smithi100.stdout: "active": true, 2024-04-07T16:16:13.892 INFO:tasks.workunit.client.0.smithi100.stdout: "mode": "" 2024-04-07T16:16:13.892 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.892 INFO:tasks.workunit.client.0.smithi100.stdout: "channels": [ 2024-04-07T16:16:13.892 INFO:tasks.workunit.client.0.smithi100.stdout: "basic", 2024-04-07T16:16:13.892 INFO:tasks.workunit.client.0.smithi100.stdout: "crash", 2024-04-07T16:16:13.892 INFO:tasks.workunit.client.0.smithi100.stdout: "device", 2024-04-07T16:16:13.892 INFO:tasks.workunit.client.0.smithi100.stdout: "ident", 2024-04-07T16:16:13.892 INFO:tasks.workunit.client.0.smithi100.stdout: "perf" 2024-04-07T16:16:13.892 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:13.893 INFO:tasks.workunit.client.0.smithi100.stdout: "channels_available": [ 2024-04-07T16:16:13.893 INFO:tasks.workunit.client.0.smithi100.stdout: "basic", 2024-04-07T16:16:13.893 INFO:tasks.workunit.client.0.smithi100.stdout: "ident", 2024-04-07T16:16:13.893 INFO:tasks.workunit.client.0.smithi100.stdout: "crash", 2024-04-07T16:16:13.893 INFO:tasks.workunit.client.0.smithi100.stdout: "device", 2024-04-07T16:16:13.893 INFO:tasks.workunit.client.0.smithi100.stdout: "perf" 2024-04-07T16:16:13.893 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:13.893 INFO:tasks.workunit.client.0.smithi100.stdout: "collections_available": [ 2024-04-07T16:16:13.893 INFO:tasks.workunit.client.0.smithi100.stdout: "basic_base", 2024-04-07T16:16:13.893 INFO:tasks.workunit.client.0.smithi100.stdout: "device_base", 2024-04-07T16:16:13.893 INFO:tasks.workunit.client.0.smithi100.stdout: "crash_base", 2024-04-07T16:16:13.893 INFO:tasks.workunit.client.0.smithi100.stdout: "ident_base", 2024-04-07T16:16:13.893 INFO:tasks.workunit.client.0.smithi100.stdout: "perf_perf", 2024-04-07T16:16:13.893 INFO:tasks.workunit.client.0.smithi100.stdout: "basic_mds_metadata", 2024-04-07T16:16:13.894 INFO:tasks.workunit.client.0.smithi100.stdout: "basic_pool_usage", 2024-04-07T16:16:13.894 INFO:tasks.workunit.client.0.smithi100.stdout: "basic_usage_by_class", 2024-04-07T16:16:13.894 INFO:tasks.workunit.client.0.smithi100.stdout: "basic_rook_v01", 2024-04-07T16:16:13.894 INFO:tasks.workunit.client.0.smithi100.stdout: "perf_memory_metrics", 2024-04-07T16:16:13.894 INFO:tasks.workunit.client.0.smithi100.stdout: "basic_pool_options_bluestore" 2024-04-07T16:16:13.894 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:13.894 INFO:tasks.workunit.client.0.smithi100.stdout: "collections_opted_in": [ 2024-04-07T16:16:13.894 INFO:tasks.workunit.client.0.smithi100.stdout: "basic_base", 2024-04-07T16:16:13.894 INFO:tasks.workunit.client.0.smithi100.stdout: "device_base", 2024-04-07T16:16:13.894 INFO:tasks.workunit.client.0.smithi100.stdout: "crash_base", 2024-04-07T16:16:13.894 INFO:tasks.workunit.client.0.smithi100.stdout: "ident_base", 2024-04-07T16:16:13.895 INFO:tasks.workunit.client.0.smithi100.stdout: "perf_perf", 2024-04-07T16:16:13.895 INFO:tasks.workunit.client.0.smithi100.stdout: "basic_mds_metadata", 2024-04-07T16:16:13.895 INFO:tasks.workunit.client.0.smithi100.stdout: "basic_pool_usage", 2024-04-07T16:16:13.895 INFO:tasks.workunit.client.0.smithi100.stdout: "basic_usage_by_class", 2024-04-07T16:16:13.895 INFO:tasks.workunit.client.0.smithi100.stdout: "basic_rook_v01", 2024-04-07T16:16:13.895 INFO:tasks.workunit.client.0.smithi100.stdout: "perf_memory_metrics", 2024-04-07T16:16:13.895 INFO:tasks.workunit.client.0.smithi100.stdout: "basic_pool_options_bluestore" 2024-04-07T16:16:13.895 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:13.895 INFO:tasks.workunit.client.0.smithi100.stdout: "config": { 2024-04-07T16:16:13.895 INFO:tasks.workunit.client.0.smithi100.stdout: "active_changed": [ 2024-04-07T16:16:13.895 INFO:tasks.workunit.client.0.smithi100.stdout: "auth_allow_insecure_global_id_reclaim", 2024-04-07T16:16:13.895 INFO:tasks.workunit.client.0.smithi100.stdout: "auth_debug", 2024-04-07T16:16:13.895 INFO:tasks.workunit.client.0.smithi100.stdout: "auth_mon_ticket_ttl", 2024-04-07T16:16:13.895 INFO:tasks.workunit.client.0.smithi100.stdout: "auth_service_ticket_ttl", 2024-04-07T16:16:13.896 INFO:tasks.workunit.client.0.smithi100.stdout: "bdev_debug_aio", 2024-04-07T16:16:13.896 INFO:tasks.workunit.client.0.smithi100.stdout: "client_check_pool_perm", 2024-04-07T16:16:13.896 INFO:tasks.workunit.client.0.smithi100.stdout: "client_mount_gid", 2024-04-07T16:16:13.896 INFO:tasks.workunit.client.0.smithi100.stdout: "client_mount_uid", 2024-04-07T16:16:13.896 INFO:tasks.workunit.client.0.smithi100.stdout: "client_quota", 2024-04-07T16:16:13.896 INFO:tasks.workunit.client.0.smithi100.stdout: "container_image", 2024-04-07T16:16:13.896 INFO:tasks.workunit.client.0.smithi100.stdout: "daemonize", 2024-04-07T16:16:13.896 INFO:tasks.workunit.client.0.smithi100.stdout: "debug_asserts_on_shutdown", 2024-04-07T16:16:13.896 INFO:tasks.workunit.client.0.smithi100.stdout: "debug_mgr", 2024-04-07T16:16:13.896 INFO:tasks.workunit.client.0.smithi100.stdout: "debug_mon", 2024-04-07T16:16:13.896 INFO:tasks.workunit.client.0.smithi100.stdout: "debug_ms", 2024-04-07T16:16:13.896 INFO:tasks.workunit.client.0.smithi100.stdout: "debug_osd", 2024-04-07T16:16:13.896 INFO:tasks.workunit.client.0.smithi100.stdout: "debug_paxos", 2024-04-07T16:16:13.896 INFO:tasks.workunit.client.0.smithi100.stdout: "keyring", 2024-04-07T16:16:13.896 INFO:tasks.workunit.client.0.smithi100.stdout: "leveldb_block_size", 2024-04-07T16:16:13.897 INFO:tasks.workunit.client.0.smithi100.stdout: "leveldb_cache_size", 2024-04-07T16:16:13.897 INFO:tasks.workunit.client.0.smithi100.stdout: "leveldb_compression", 2024-04-07T16:16:13.897 INFO:tasks.workunit.client.0.smithi100.stdout: "leveldb_log", 2024-04-07T16:16:13.897 INFO:tasks.workunit.client.0.smithi100.stdout: "leveldb_write_buffer_size", 2024-04-07T16:16:13.897 INFO:tasks.workunit.client.0.smithi100.stdout: "log_stderr_prefix", 2024-04-07T16:16:13.897 INFO:tasks.workunit.client.0.smithi100.stdout: "log_to_file", 2024-04-07T16:16:13.897 INFO:tasks.workunit.client.0.smithi100.stdout: "log_to_journald", 2024-04-07T16:16:13.897 INFO:tasks.workunit.client.0.smithi100.stdout: "log_to_stderr", 2024-04-07T16:16:13.897 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_allow_pool_delete", 2024-04-07T16:16:13.897 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_clock_drift_allowed", 2024-04-07T16:16:13.897 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_cluster_log_file_level", 2024-04-07T16:16:13.897 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_cluster_log_to_file", 2024-04-07T16:16:13.897 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_cluster_log_to_stderr", 2024-04-07T16:16:13.897 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_data_avail_warn", 2024-04-07T16:16:13.898 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_election_default_strategy", 2024-04-07T16:16:13.898 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_host", 2024-04-07T16:16:13.898 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_max_pg_per_osd", 2024-04-07T16:16:13.898 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_mgr_mkfs_grace", 2024-04-07T16:16:13.898 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_osd_prime_pg_temp", 2024-04-07T16:16:13.898 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_osd_reporter_subtree_level", 2024-04-07T16:16:13.898 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_pg_warn_max_object_skew", 2024-04-07T16:16:13.898 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_reweight_min_bytes_per_osd", 2024-04-07T16:16:13.898 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_reweight_min_pgs_per_osd", 2024-04-07T16:16:13.898 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_warn_on_crush_straw_calc_version_zero", 2024-04-07T16:16:13.898 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_warn_on_insecure_global_id_reclaim", 2024-04-07T16:16:13.898 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_warn_on_insecure_global_id_reclaim_allowed", 2024-04-07T16:16:13.898 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_warn_on_legacy_crush_tunables", 2024-04-07T16:16:13.898 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_warn_on_osd_down_out_interval_zero", 2024-04-07T16:16:13.899 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_warn_on_pool_pg_num_not_power_of_two", 2024-04-07T16:16:13.899 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_warn_on_too_few_osds", 2024-04-07T16:16:13.899 INFO:tasks.workunit.client.0.smithi100.stdout: "ms_die_on_bug", 2024-04-07T16:16:13.899 INFO:tasks.workunit.client.0.smithi100.stdout: "ms_die_on_old_message", 2024-04-07T16:16:13.899 INFO:tasks.workunit.client.0.smithi100.stdout: "no_config_file", 2024-04-07T16:16:13.899 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_class_default_list", 2024-04-07T16:16:13.899 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_class_load_list", 2024-04-07T16:16:13.899 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_crush_chooseleaf_type", 2024-04-07T16:16:13.899 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_debug_misdirected_ops", 2024-04-07T16:16:13.899 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_debug_op_order", 2024-04-07T16:16:13.899 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_debug_pg_log_writeout", 2024-04-07T16:16:13.899 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_debug_shutdown", 2024-04-07T16:16:13.899 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_debug_verify_cached_snaps", 2024-04-07T16:16:13.899 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_debug_verify_missing_on_start", 2024-04-07T16:16:13.899 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_debug_verify_stray_on_activate", 2024-04-07T16:16:13.900 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_deep_scrub_update_digest_min_age", 2024-04-07T16:16:13.900 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_delete_sleep", 2024-04-07T16:16:13.900 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_delete_sleep_hdd", 2024-04-07T16:16:13.900 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_delete_sleep_hybrid", 2024-04-07T16:16:13.900 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_delete_sleep_ssd", 2024-04-07T16:16:13.900 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_max_backfills", 2024-04-07T16:16:13.900 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mclock_max_capacity_iops_ssd", 2024-04-07T16:16:13.900 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mclock_profile", 2024-04-07T16:16:13.900 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mclock_scheduler_background_best_effort_lim", 2024-04-07T16:16:13.900 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mclock_scheduler_background_best_effort_res", 2024-04-07T16:16:13.900 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mclock_scheduler_background_best_effort_wgt", 2024-04-07T16:16:13.901 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mclock_scheduler_background_recovery_lim", 2024-04-07T16:16:13.901 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mclock_scheduler_background_recovery_res", 2024-04-07T16:16:13.901 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mclock_scheduler_background_recovery_wgt", 2024-04-07T16:16:13.901 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mclock_scheduler_client_lim", 2024-04-07T16:16:13.901 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mclock_scheduler_client_res", 2024-04-07T16:16:13.901 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mclock_scheduler_client_wgt", 2024-04-07T16:16:13.901 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_memory_target", 2024-04-07T16:16:13.901 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_memory_target_autotune", 2024-04-07T16:16:13.901 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_op_queue", 2024-04-07T16:16:13.901 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_op_queue_cut_off", 2024-04-07T16:16:13.901 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_pg_delete_cost", 2024-04-07T16:16:13.901 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_pool_default_erasure_code_profile", 2024-04-07T16:16:13.901 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_pool_default_pg_autoscale_mode", 2024-04-07T16:16:13.901 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_recover_clone_overlap", 2024-04-07T16:16:13.901 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_recovery_max_active", 2024-04-07T16:16:13.902 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_recovery_max_active_hdd", 2024-04-07T16:16:13.902 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_recovery_max_active_ssd", 2024-04-07T16:16:13.902 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_recovery_max_chunk", 2024-04-07T16:16:13.902 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_recovery_sleep", 2024-04-07T16:16:13.902 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_recovery_sleep_hdd", 2024-04-07T16:16:13.902 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_recovery_sleep_hybrid", 2024-04-07T16:16:13.902 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_recovery_sleep_ssd", 2024-04-07T16:16:13.902 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_scrub_load_threshold", 2024-04-07T16:16:13.902 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_scrub_max_interval", 2024-04-07T16:16:13.902 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_scrub_sleep", 2024-04-07T16:16:13.902 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_shutdown_pgref_assert", 2024-04-07T16:16:13.902 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_snap_trim_sleep", 2024-04-07T16:16:13.902 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_snap_trim_sleep_hdd", 2024-04-07T16:16:13.903 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_snap_trim_sleep_hybrid", 2024-04-07T16:16:13.903 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_snap_trim_sleep_ssd", 2024-04-07T16:16:13.903 INFO:tasks.workunit.client.0.smithi100.stdout: "public_addr", 2024-04-07T16:16:13.903 INFO:tasks.workunit.client.0.smithi100.stdout: "public_addrv", 2024-04-07T16:16:13.903 INFO:tasks.workunit.client.0.smithi100.stdout: "public_network", 2024-04-07T16:16:13.903 INFO:tasks.workunit.client.0.smithi100.stdout: "rbd_default_features", 2024-04-07T16:16:13.903 INFO:tasks.workunit.client.0.smithi100.stdout: "rbd_qos_exclude_ops", 2024-04-07T16:16:13.903 INFO:tasks.workunit.client.0.smithi100.stdout: "setgroup", 2024-04-07T16:16:13.903 INFO:tasks.workunit.client.0.smithi100.stdout: "setuser" 2024-04-07T16:16:13.903 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:13.903 INFO:tasks.workunit.client.0.smithi100.stdout: "cluster_changed": [ 2024-04-07T16:16:13.903 INFO:tasks.workunit.client.0.smithi100.stdout: "auth_allow_insecure_global_id_reclaim", 2024-04-07T16:16:13.903 INFO:tasks.workunit.client.0.smithi100.stdout: "auth_debug", 2024-04-07T16:16:13.903 INFO:tasks.workunit.client.0.smithi100.stdout: "auth_mon_ticket_ttl", 2024-04-07T16:16:13.904 INFO:tasks.workunit.client.0.smithi100.stdout: "auth_service_ticket_ttl", 2024-04-07T16:16:13.904 INFO:tasks.workunit.client.0.smithi100.stdout: "bdev_debug_aio", 2024-04-07T16:16:13.904 INFO:tasks.workunit.client.0.smithi100.stdout: "container_image", 2024-04-07T16:16:13.904 INFO:tasks.workunit.client.0.smithi100.stdout: "debug_asserts_on_shutdown", 2024-04-07T16:16:13.904 INFO:tasks.workunit.client.0.smithi100.stdout: "debug_mgr", 2024-04-07T16:16:13.904 INFO:tasks.workunit.client.0.smithi100.stdout: "debug_mon", 2024-04-07T16:16:13.904 INFO:tasks.workunit.client.0.smithi100.stdout: "debug_ms", 2024-04-07T16:16:13.904 INFO:tasks.workunit.client.0.smithi100.stdout: "debug_osd", 2024-04-07T16:16:13.904 INFO:tasks.workunit.client.0.smithi100.stdout: "debug_paxos", 2024-04-07T16:16:13.904 INFO:tasks.workunit.client.0.smithi100.stdout: "log_to_file", 2024-04-07T16:16:13.904 INFO:tasks.workunit.client.0.smithi100.stdout: "log_to_journald", 2024-04-07T16:16:13.904 INFO:tasks.workunit.client.0.smithi100.stdout: "log_to_stderr", 2024-04-07T16:16:13.904 INFO:tasks.workunit.client.0.smithi100.stdout: "mds_join_fs", 2024-04-07T16:16:13.904 INFO:tasks.workunit.client.0.smithi100.stdout: "mgr/cephadm/allow_ptrace", 2024-04-07T16:16:13.905 INFO:tasks.workunit.client.0.smithi100.stdout: "mgr/cephadm/container_init", 2024-04-07T16:16:13.905 INFO:tasks.workunit.client.0.smithi100.stdout: "mgr/cephadm/migration_current", 2024-04-07T16:16:13.905 INFO:tasks.workunit.client.0.smithi100.stdout: "mgr/cephadm/use_repo_digest", 2024-04-07T16:16:13.905 INFO:tasks.workunit.client.0.smithi100.stdout: "mgr/dashboard/ALERTMANAGER_API_HOST", 2024-04-07T16:16:13.905 INFO:tasks.workunit.client.0.smithi100.stdout: "mgr/dashboard/GRAFANA_API_SSL_VERIFY", 2024-04-07T16:16:13.905 INFO:tasks.workunit.client.0.smithi100.stdout: "mgr/dashboard/GRAFANA_API_URL", 2024-04-07T16:16:13.905 INFO:tasks.workunit.client.0.smithi100.stdout: "mgr/dashboard/PROMETHEUS_API_HOST", 2024-04-07T16:16:13.905 INFO:tasks.workunit.client.0.smithi100.stdout: "mgr/dashboard/ssl_server_port", 2024-04-07T16:16:13.905 INFO:tasks.workunit.client.0.smithi100.stdout: "mgr/orchestrator/orchestrator", 2024-04-07T16:16:13.905 INFO:tasks.workunit.client.0.smithi100.stdout: "mgr/telemetry/channel_ident", 2024-04-07T16:16:13.905 INFO:tasks.workunit.client.0.smithi100.stdout: "mgr/telemetry/channel_perf", 2024-04-07T16:16:13.905 INFO:tasks.workunit.client.0.smithi100.stdout: "mgr/telemetry/description", 2024-04-07T16:16:13.905 INFO:tasks.workunit.client.0.smithi100.stdout: "mgr/telemetry/enabled", 2024-04-07T16:16:13.905 INFO:tasks.workunit.client.0.smithi100.stdout: "mgr/telemetry/organization", 2024-04-07T16:16:13.905 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_allow_pool_delete", 2024-04-07T16:16:13.906 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_clock_drift_allowed", 2024-04-07T16:16:13.906 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_cluster_log_file_level", 2024-04-07T16:16:13.906 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_cluster_log_to_file", 2024-04-07T16:16:13.906 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_data_avail_warn", 2024-04-07T16:16:13.906 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_election_default_strategy", 2024-04-07T16:16:13.906 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_max_pg_per_osd", 2024-04-07T16:16:13.906 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_mgr_mkfs_grace", 2024-04-07T16:16:13.906 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_osd_prime_pg_temp", 2024-04-07T16:16:13.906 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_osd_reporter_subtree_level", 2024-04-07T16:16:13.906 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_pg_warn_max_object_skew", 2024-04-07T16:16:13.906 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_reweight_min_bytes_per_osd", 2024-04-07T16:16:13.906 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_reweight_min_pgs_per_osd", 2024-04-07T16:16:13.907 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_warn_on_crush_straw_calc_version_zero", 2024-04-07T16:16:13.907 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_warn_on_insecure_global_id_reclaim", 2024-04-07T16:16:13.907 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_warn_on_insecure_global_id_reclaim_allowed", 2024-04-07T16:16:13.907 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_warn_on_legacy_crush_tunables", 2024-04-07T16:16:13.907 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_warn_on_osd_down_out_interval_zero", 2024-04-07T16:16:13.907 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_warn_on_pool_pg_num_not_power_of_two", 2024-04-07T16:16:13.907 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_warn_on_too_few_osds", 2024-04-07T16:16:13.907 INFO:tasks.workunit.client.0.smithi100.stdout: "ms_die_on_bug", 2024-04-07T16:16:13.907 INFO:tasks.workunit.client.0.smithi100.stdout: "ms_die_on_old_message", 2024-04-07T16:16:13.907 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_class_default_list", 2024-04-07T16:16:13.907 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_class_load_list", 2024-04-07T16:16:13.907 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_debug_misdirected_ops", 2024-04-07T16:16:13.907 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_debug_op_order", 2024-04-07T16:16:13.907 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_debug_pg_log_writeout", 2024-04-07T16:16:13.907 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_debug_shutdown", 2024-04-07T16:16:13.908 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_debug_verify_cached_snaps", 2024-04-07T16:16:13.908 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_debug_verify_missing_on_start", 2024-04-07T16:16:13.908 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_debug_verify_stray_on_activate", 2024-04-07T16:16:13.908 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_deep_scrub_update_digest_min_age", 2024-04-07T16:16:13.908 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mclock_max_capacity_iops_ssd", 2024-04-07T16:16:13.908 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mclock_profile", 2024-04-07T16:16:13.908 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_memory_target", 2024-04-07T16:16:13.908 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_memory_target_autotune", 2024-04-07T16:16:13.908 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_op_queue", 2024-04-07T16:16:13.908 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_op_queue_cut_off", 2024-04-07T16:16:13.908 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_pool_default_erasure_code_profile", 2024-04-07T16:16:13.908 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_pool_default_pg_autoscale_mode", 2024-04-07T16:16:13.908 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_recover_clone_overlap", 2024-04-07T16:16:13.909 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_recovery_max_chunk", 2024-04-07T16:16:13.909 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_scrub_load_threshold", 2024-04-07T16:16:13.909 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_scrub_max_interval", 2024-04-07T16:16:13.909 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_shutdown_pgref_assert", 2024-04-07T16:16:13.909 INFO:tasks.workunit.client.0.smithi100.stdout: "public_network", 2024-04-07T16:16:13.909 INFO:tasks.workunit.client.0.smithi100.stdout: "rgw_cache_enabled", 2024-04-07T16:16:13.909 INFO:tasks.workunit.client.0.smithi100.stdout: "rgw_enable_ops_log", 2024-04-07T16:16:13.909 INFO:tasks.workunit.client.0.smithi100.stdout: "rgw_enable_usage_log" 2024-04-07T16:16:13.909 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T16:16:13.909 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.909 INFO:tasks.workunit.client.0.smithi100.stdout: "contact": null, 2024-04-07T16:16:13.909 INFO:tasks.workunit.client.0.smithi100.stdout: "crashes": [], 2024-04-07T16:16:13.909 INFO:tasks.workunit.client.0.smithi100.stdout: "created": "2024-04-07T16:02:17.280630Z", 2024-04-07T16:16:13.909 INFO:tasks.workunit.client.0.smithi100.stdout: "crush": { 2024-04-07T16:16:13.910 INFO:tasks.workunit.client.0.smithi100.stdout: "bucket_algs": { 2024-04-07T16:16:13.910 INFO:tasks.workunit.client.0.smithi100.stdout: "straw2": 3 2024-04-07T16:16:13.910 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.910 INFO:tasks.workunit.client.0.smithi100.stdout: "bucket_sizes": { 2024-04-07T16:16:13.910 INFO:tasks.workunit.client.0.smithi100.stdout: "2": 1, 2024-04-07T16:16:13.910 INFO:tasks.workunit.client.0.smithi100.stdout: "4": 2 2024-04-07T16:16:13.910 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.910 INFO:tasks.workunit.client.0.smithi100.stdout: "bucket_types": { 2024-04-07T16:16:13.910 INFO:tasks.workunit.client.0.smithi100.stdout: "1": 2, 2024-04-07T16:16:13.910 INFO:tasks.workunit.client.0.smithi100.stdout: "11": 1 2024-04-07T16:16:13.910 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.910 INFO:tasks.workunit.client.0.smithi100.stdout: "compat_weight_set": false, 2024-04-07T16:16:13.910 INFO:tasks.workunit.client.0.smithi100.stdout: "device_classes": [ 2024-04-07T16:16:13.910 INFO:tasks.workunit.client.0.smithi100.stdout: 8 2024-04-07T16:16:13.911 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:13.911 INFO:tasks.workunit.client.0.smithi100.stdout: "num_buckets": 6, 2024-04-07T16:16:13.911 INFO:tasks.workunit.client.0.smithi100.stdout: "num_devices": 8, 2024-04-07T16:16:13.911 INFO:tasks.workunit.client.0.smithi100.stdout: "num_rules": 1, 2024-04-07T16:16:13.911 INFO:tasks.workunit.client.0.smithi100.stdout: "num_types": 12, 2024-04-07T16:16:13.911 INFO:tasks.workunit.client.0.smithi100.stdout: "num_weight_sets": 0, 2024-04-07T16:16:13.911 INFO:tasks.workunit.client.0.smithi100.stdout: "tunables": { 2024-04-07T16:16:13.911 INFO:tasks.workunit.client.0.smithi100.stdout: "allowed_bucket_algs": 54, 2024-04-07T16:16:13.911 INFO:tasks.workunit.client.0.smithi100.stdout: "choose_local_fallback_tries": 0, 2024-04-07T16:16:13.911 INFO:tasks.workunit.client.0.smithi100.stdout: "choose_local_tries": 0, 2024-04-07T16:16:13.911 INFO:tasks.workunit.client.0.smithi100.stdout: "choose_total_tries": 50, 2024-04-07T16:16:13.911 INFO:tasks.workunit.client.0.smithi100.stdout: "chooseleaf_descend_once": 1, 2024-04-07T16:16:13.911 INFO:tasks.workunit.client.0.smithi100.stdout: "chooseleaf_stable": 1, 2024-04-07T16:16:13.911 INFO:tasks.workunit.client.0.smithi100.stdout: "chooseleaf_vary_r": 1, 2024-04-07T16:16:13.911 INFO:tasks.workunit.client.0.smithi100.stdout: "has_v2_rules": 0, 2024-04-07T16:16:13.912 INFO:tasks.workunit.client.0.smithi100.stdout: "has_v3_rules": 0, 2024-04-07T16:16:13.912 INFO:tasks.workunit.client.0.smithi100.stdout: "has_v4_buckets": 1, 2024-04-07T16:16:13.912 INFO:tasks.workunit.client.0.smithi100.stdout: "has_v5_rules": 0, 2024-04-07T16:16:13.912 INFO:tasks.workunit.client.0.smithi100.stdout: "legacy_tunables": 0, 2024-04-07T16:16:13.912 INFO:tasks.workunit.client.0.smithi100.stdout: "minimum_required_version": "jewel", 2024-04-07T16:16:13.912 INFO:tasks.workunit.client.0.smithi100.stdout: "optimal_tunables": 1, 2024-04-07T16:16:13.912 INFO:tasks.workunit.client.0.smithi100.stdout: "profile": "jewel", 2024-04-07T16:16:13.912 INFO:tasks.workunit.client.0.smithi100.stdout: "require_feature_tunables": 1, 2024-04-07T16:16:13.912 INFO:tasks.workunit.client.0.smithi100.stdout: "require_feature_tunables2": 1, 2024-04-07T16:16:13.912 INFO:tasks.workunit.client.0.smithi100.stdout: "require_feature_tunables3": 1, 2024-04-07T16:16:13.912 INFO:tasks.workunit.client.0.smithi100.stdout: "require_feature_tunables5": 1, 2024-04-07T16:16:13.912 INFO:tasks.workunit.client.0.smithi100.stdout: "straw_calc_version": 1 2024-04-07T16:16:13.912 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:13.912 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.913 INFO:tasks.workunit.client.0.smithi100.stdout: "description": "upgrade test cluster", 2024-04-07T16:16:13.913 INFO:tasks.workunit.client.0.smithi100.stdout: "fs": { 2024-04-07T16:16:13.913 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1, 2024-04-07T16:16:13.913 INFO:tasks.workunit.client.0.smithi100.stdout: "feature_flags": { 2024-04-07T16:16:13.913 INFO:tasks.workunit.client.0.smithi100.stdout: "enable_multiple": true, 2024-04-07T16:16:13.913 INFO:tasks.workunit.client.0.smithi100.stdout: "ever_enabled_multiple": true 2024-04-07T16:16:13.913 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.913 INFO:tasks.workunit.client.0.smithi100.stdout: "filesystems": [ 2024-04-07T16:16:13.913 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:13.913 INFO:tasks.workunit.client.0.smithi100.stdout: "approx_ctime": "2024-04", 2024-04-07T16:16:13.913 INFO:tasks.workunit.client.0.smithi100.stdout: "balancer_enabled": false, 2024-04-07T16:16:13.913 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:13.914 INFO:tasks.workunit.client.0.smithi100.stdout: "cached_caps": 0, 2024-04-07T16:16:13.914 INFO:tasks.workunit.client.0.smithi100.stdout: "cached_dns": 10, 2024-04-07T16:16:13.914 INFO:tasks.workunit.client.0.smithi100.stdout: "cached_inos": 13, 2024-04-07T16:16:13.914 INFO:tasks.workunit.client.0.smithi100.stdout: "cached_subtrees": 2, 2024-04-07T16:16:13.914 INFO:tasks.workunit.client.0.smithi100.stdout: "ever_allowed_features": 0, 2024-04-07T16:16:13.914 INFO:tasks.workunit.client.0.smithi100.stdout: "explicitly_allowed_features": 0, 2024-04-07T16:16:13.914 INFO:tasks.workunit.client.0.smithi100.stdout: "files": 0, 2024-04-07T16:16:13.914 INFO:tasks.workunit.client.0.smithi100.stdout: "max_mds": 1, 2024-04-07T16:16:13.914 INFO:tasks.workunit.client.0.smithi100.stdout: "num_data_pools": 1, 2024-04-07T16:16:13.914 INFO:tasks.workunit.client.0.smithi100.stdout: "num_in": 1, 2024-04-07T16:16:13.914 INFO:tasks.workunit.client.0.smithi100.stdout: "num_mds": 1, 2024-04-07T16:16:13.914 INFO:tasks.workunit.client.0.smithi100.stdout: "num_sessions": 0, 2024-04-07T16:16:13.914 INFO:tasks.workunit.client.0.smithi100.stdout: "num_standby_replay": 0, 2024-04-07T16:16:13.914 INFO:tasks.workunit.client.0.smithi100.stdout: "num_up": 1, 2024-04-07T16:16:13.915 INFO:tasks.workunit.client.0.smithi100.stdout: "snaps": 0, 2024-04-07T16:16:13.915 INFO:tasks.workunit.client.0.smithi100.stdout: "standby_count_wanted": 0 2024-04-07T16:16:13.915 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:13.915 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:13.915 INFO:tasks.workunit.client.0.smithi100.stdout: "num_standby_mds": 1, 2024-04-07T16:16:13.915 INFO:tasks.workunit.client.0.smithi100.stdout: "total_num_mds": 2 2024-04-07T16:16:13.915 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.915 INFO:tasks.workunit.client.0.smithi100.stdout: "heap_stats": { 2024-04-07T16:16:13.915 INFO:tasks.workunit.client.0.smithi100.stdout: "mds": { 2024-04-07T16:16:13.915 INFO:tasks.workunit.client.0.smithi100.stdout: "mds.a52fc0d8acb515431879057649ff294cfe14c620": { 2024-04-07T16:16:13.915 INFO:tasks.workunit.client.0.smithi100.stdout: "actual_memory_used": 20447232, 2024-04-07T16:16:13.915 INFO:tasks.workunit.client.0.smithi100.stdout: "central_cache_freelist": 502696, 2024-04-07T16:16:13.915 INFO:tasks.workunit.client.0.smithi100.stdout: "malloc_metadata": 2752512, 2024-04-07T16:16:13.915 INFO:tasks.workunit.client.0.smithi100.stdout: "page_heap_freelist": 0, 2024-04-07T16:16:13.915 INFO:tasks.workunit.client.0.smithi100.stdout: "released_to_os": 868352, 2024-04-07T16:16:13.916 INFO:tasks.workunit.client.0.smithi100.stdout: "spans_in_use": 405, 2024-04-07T16:16:13.916 INFO:tasks.workunit.client.0.smithi100.stdout: "tcmalloc_page_size": 8192, 2024-04-07T16:16:13.916 INFO:tasks.workunit.client.0.smithi100.stdout: "thread_cache_freelists": 1329432, 2024-04-07T16:16:13.916 INFO:tasks.workunit.client.0.smithi100.stdout: "thread_heaps_in_use": 21, 2024-04-07T16:16:13.916 INFO:tasks.workunit.client.0.smithi100.stdout: "transfer_cache_freelist": 982528, 2024-04-07T16:16:13.916 INFO:tasks.workunit.client.0.smithi100.stdout: "use_by_application": 14880064, 2024-04-07T16:16:13.916 INFO:tasks.workunit.client.0.smithi100.stdout: "virtual_address_space_used": 21315584 2024-04-07T16:16:13.916 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.916 INFO:tasks.workunit.client.0.smithi100.stdout: "mds.f47a4ff15a45950a63160265564a123fc5da5e5e": { 2024-04-07T16:16:13.916 INFO:tasks.workunit.client.0.smithi100.stdout: "actual_memory_used": 20267008, 2024-04-07T16:16:13.916 INFO:tasks.workunit.client.0.smithi100.stdout: "central_cache_freelist": 340008, 2024-04-07T16:16:13.916 INFO:tasks.workunit.client.0.smithi100.stdout: "malloc_metadata": 2752512, 2024-04-07T16:16:13.916 INFO:tasks.workunit.client.0.smithi100.stdout: "page_heap_freelist": 974848, 2024-04-07T16:16:13.916 INFO:tasks.workunit.client.0.smithi100.stdout: "released_to_os": 0, 2024-04-07T16:16:13.917 INFO:tasks.workunit.client.0.smithi100.stdout: "spans_in_use": 323, 2024-04-07T16:16:13.917 INFO:tasks.workunit.client.0.smithi100.stdout: "tcmalloc_page_size": 8192, 2024-04-07T16:16:13.917 INFO:tasks.workunit.client.0.smithi100.stdout: "thread_cache_freelists": 1191432, 2024-04-07T16:16:13.917 INFO:tasks.workunit.client.0.smithi100.stdout: "thread_heaps_in_use": 12, 2024-04-07T16:16:13.917 INFO:tasks.workunit.client.0.smithi100.stdout: "transfer_cache_freelist": 545792, 2024-04-07T16:16:13.917 INFO:tasks.workunit.client.0.smithi100.stdout: "use_by_application": 14462416, 2024-04-07T16:16:13.917 INFO:tasks.workunit.client.0.smithi100.stdout: "virtual_address_space_used": 20267008 2024-04-07T16:16:13.917 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:13.917 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.917 INFO:tasks.workunit.client.0.smithi100.stdout: "mon": { 2024-04-07T16:16:13.917 INFO:tasks.workunit.client.0.smithi100.stdout: "mon.8dd0baa036f0a7e6d1f3bdce86136fe527836c3d": { 2024-04-07T16:16:13.917 INFO:tasks.workunit.client.0.smithi100.stdout: "actual_memory_used": 34676736, 2024-04-07T16:16:13.917 INFO:tasks.workunit.client.0.smithi100.stdout: "central_cache_freelist": 1113848, 2024-04-07T16:16:13.917 INFO:tasks.workunit.client.0.smithi100.stdout: "malloc_metadata": 2752512, 2024-04-07T16:16:13.918 INFO:tasks.workunit.client.0.smithi100.stdout: "page_heap_freelist": 0, 2024-04-07T16:16:13.918 INFO:tasks.workunit.client.0.smithi100.stdout: "released_to_os": 5644288, 2024-04-07T16:16:13.918 INFO:tasks.workunit.client.0.smithi100.stdout: "spans_in_use": 1050, 2024-04-07T16:16:13.918 INFO:tasks.workunit.client.0.smithi100.stdout: "tcmalloc_page_size": 8192, 2024-04-07T16:16:13.918 INFO:tasks.workunit.client.0.smithi100.stdout: "thread_cache_freelists": 8225816, 2024-04-07T16:16:13.918 INFO:tasks.workunit.client.0.smithi100.stdout: "thread_heaps_in_use": 21, 2024-04-07T16:16:13.918 INFO:tasks.workunit.client.0.smithi100.stdout: "transfer_cache_freelist": 1355776, 2024-04-07T16:16:13.918 INFO:tasks.workunit.client.0.smithi100.stdout: "use_by_application": 21228784, 2024-04-07T16:16:13.918 INFO:tasks.workunit.client.0.smithi100.stdout: "virtual_address_space_used": 40321024 2024-04-07T16:16:13.918 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.918 INFO:tasks.workunit.client.0.smithi100.stdout: "mon.a52fc0d8acb515431879057649ff294cfe14c620": { 2024-04-07T16:16:13.918 INFO:tasks.workunit.client.0.smithi100.stdout: "actual_memory_used": 36831232, 2024-04-07T16:16:13.918 INFO:tasks.workunit.client.0.smithi100.stdout: "central_cache_freelist": 935360, 2024-04-07T16:16:13.918 INFO:tasks.workunit.client.0.smithi100.stdout: "malloc_metadata": 2752512, 2024-04-07T16:16:13.918 INFO:tasks.workunit.client.0.smithi100.stdout: "page_heap_freelist": 704512, 2024-04-07T16:16:13.919 INFO:tasks.workunit.client.0.smithi100.stdout: "released_to_os": 4538368, 2024-04-07T16:16:13.919 INFO:tasks.workunit.client.0.smithi100.stdout: "spans_in_use": 1252, 2024-04-07T16:16:13.919 INFO:tasks.workunit.client.0.smithi100.stdout: "tcmalloc_page_size": 8192, 2024-04-07T16:16:13.919 INFO:tasks.workunit.client.0.smithi100.stdout: "thread_cache_freelists": 9614856, 2024-04-07T16:16:13.919 INFO:tasks.workunit.client.0.smithi100.stdout: "thread_heaps_in_use": 22, 2024-04-07T16:16:13.919 INFO:tasks.workunit.client.0.smithi100.stdout: "transfer_cache_freelist": 1432576, 2024-04-07T16:16:13.919 INFO:tasks.workunit.client.0.smithi100.stdout: "use_by_application": 21391416, 2024-04-07T16:16:13.919 INFO:tasks.workunit.client.0.smithi100.stdout: "virtual_address_space_used": 41369600 2024-04-07T16:16:13.919 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.919 INFO:tasks.workunit.client.0.smithi100.stdout: "mon.f47a4ff15a45950a63160265564a123fc5da5e5e": { 2024-04-07T16:16:13.919 INFO:tasks.workunit.client.0.smithi100.stdout: "actual_memory_used": 44195840, 2024-04-07T16:16:13.919 INFO:tasks.workunit.client.0.smithi100.stdout: "central_cache_freelist": 1239944, 2024-04-07T16:16:13.919 INFO:tasks.workunit.client.0.smithi100.stdout: "malloc_metadata": 2752512, 2024-04-07T16:16:13.919 INFO:tasks.workunit.client.0.smithi100.stdout: "page_heap_freelist": 0, 2024-04-07T16:16:13.919 INFO:tasks.workunit.client.0.smithi100.stdout: "released_to_os": 3366912, 2024-04-07T16:16:13.920 INFO:tasks.workunit.client.0.smithi100.stdout: "spans_in_use": 1651, 2024-04-07T16:16:13.920 INFO:tasks.workunit.client.0.smithi100.stdout: "tcmalloc_page_size": 8192, 2024-04-07T16:16:13.920 INFO:tasks.workunit.client.0.smithi100.stdout: "thread_cache_freelists": 12500560, 2024-04-07T16:16:13.920 INFO:tasks.workunit.client.0.smithi100.stdout: "thread_heaps_in_use": 22, 2024-04-07T16:16:13.920 INFO:tasks.workunit.client.0.smithi100.stdout: "transfer_cache_freelist": 2158848, 2024-04-07T16:16:13.920 INFO:tasks.workunit.client.0.smithi100.stdout: "use_by_application": 25543976, 2024-04-07T16:16:13.920 INFO:tasks.workunit.client.0.smithi100.stdout: "virtual_address_space_used": 47562752 2024-04-07T16:16:13.920 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:13.920 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.920 INFO:tasks.workunit.client.0.smithi100.stdout: "osd": { 2024-04-07T16:16:13.920 INFO:tasks.workunit.client.0.smithi100.stdout: "osd.0": { 2024-04-07T16:16:13.920 INFO:tasks.workunit.client.0.smithi100.stdout: "actual_memory_used": 47538176, 2024-04-07T16:16:13.920 INFO:tasks.workunit.client.0.smithi100.stdout: "central_cache_freelist": 913648, 2024-04-07T16:16:13.920 INFO:tasks.workunit.client.0.smithi100.stdout: "malloc_metadata": 2883584, 2024-04-07T16:16:13.921 INFO:tasks.workunit.client.0.smithi100.stdout: "page_heap_freelist": 0, 2024-04-07T16:16:13.921 INFO:tasks.workunit.client.0.smithi100.stdout: "released_to_os": 876544, 2024-04-07T16:16:13.921 INFO:tasks.workunit.client.0.smithi100.stdout: "spans_in_use": 1471, 2024-04-07T16:16:13.921 INFO:tasks.workunit.client.0.smithi100.stdout: "tcmalloc_page_size": 8192, 2024-04-07T16:16:13.921 INFO:tasks.workunit.client.0.smithi100.stdout: "thread_cache_freelists": 5106408, 2024-04-07T16:16:13.921 INFO:tasks.workunit.client.0.smithi100.stdout: "thread_heaps_in_use": 46, 2024-04-07T16:16:13.921 INFO:tasks.workunit.client.0.smithi100.stdout: "transfer_cache_freelist": 2300672, 2024-04-07T16:16:13.921 INFO:tasks.workunit.client.0.smithi100.stdout: "use_by_application": 36333864, 2024-04-07T16:16:13.921 INFO:tasks.workunit.client.0.smithi100.stdout: "virtual_address_space_used": 48414720 2024-04-07T16:16:13.921 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.921 INFO:tasks.workunit.client.0.smithi100.stdout: "osd.1": { 2024-04-07T16:16:13.921 INFO:tasks.workunit.client.0.smithi100.stdout: "actual_memory_used": 47022080, 2024-04-07T16:16:13.921 INFO:tasks.workunit.client.0.smithi100.stdout: "central_cache_freelist": 914336, 2024-04-07T16:16:13.921 INFO:tasks.workunit.client.0.smithi100.stdout: "malloc_metadata": 2883584, 2024-04-07T16:16:13.921 INFO:tasks.workunit.client.0.smithi100.stdout: "page_heap_freelist": 0, 2024-04-07T16:16:13.922 INFO:tasks.workunit.client.0.smithi100.stdout: "released_to_os": 344064, 2024-04-07T16:16:13.922 INFO:tasks.workunit.client.0.smithi100.stdout: "spans_in_use": 1442, 2024-04-07T16:16:13.922 INFO:tasks.workunit.client.0.smithi100.stdout: "tcmalloc_page_size": 8192, 2024-04-07T16:16:13.922 INFO:tasks.workunit.client.0.smithi100.stdout: "thread_cache_freelists": 5214888, 2024-04-07T16:16:13.922 INFO:tasks.workunit.client.0.smithi100.stdout: "thread_heaps_in_use": 47, 2024-04-07T16:16:13.922 INFO:tasks.workunit.client.0.smithi100.stdout: "transfer_cache_freelist": 2809088, 2024-04-07T16:16:13.922 INFO:tasks.workunit.client.0.smithi100.stdout: "use_by_application": 35200184, 2024-04-07T16:16:13.922 INFO:tasks.workunit.client.0.smithi100.stdout: "virtual_address_space_used": 47366144 2024-04-07T16:16:13.922 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.922 INFO:tasks.workunit.client.0.smithi100.stdout: "osd.2": { 2024-04-07T16:16:13.922 INFO:tasks.workunit.client.0.smithi100.stdout: "actual_memory_used": 51060736, 2024-04-07T16:16:13.922 INFO:tasks.workunit.client.0.smithi100.stdout: "central_cache_freelist": 1779128, 2024-04-07T16:16:13.922 INFO:tasks.workunit.client.0.smithi100.stdout: "malloc_metadata": 2883584, 2024-04-07T16:16:13.922 INFO:tasks.workunit.client.0.smithi100.stdout: "page_heap_freelist": 8192, 2024-04-07T16:16:13.923 INFO:tasks.workunit.client.0.smithi100.stdout: "released_to_os": 499712, 2024-04-07T16:16:13.923 INFO:tasks.workunit.client.0.smithi100.stdout: "spans_in_use": 1776, 2024-04-07T16:16:13.923 INFO:tasks.workunit.client.0.smithi100.stdout: "tcmalloc_page_size": 8192, 2024-04-07T16:16:13.923 INFO:tasks.workunit.client.0.smithi100.stdout: "thread_cache_freelists": 5597448, 2024-04-07T16:16:13.923 INFO:tasks.workunit.client.0.smithi100.stdout: "thread_heaps_in_use": 47, 2024-04-07T16:16:13.923 INFO:tasks.workunit.client.0.smithi100.stdout: "transfer_cache_freelist": 4880384, 2024-04-07T16:16:13.923 INFO:tasks.workunit.client.0.smithi100.stdout: "use_by_application": 35912000, 2024-04-07T16:16:13.923 INFO:tasks.workunit.client.0.smithi100.stdout: "virtual_address_space_used": 51560448 2024-04-07T16:16:13.923 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.923 INFO:tasks.workunit.client.0.smithi100.stdout: "osd.3": { 2024-04-07T16:16:13.923 INFO:tasks.workunit.client.0.smithi100.stdout: "actual_memory_used": 44515328, 2024-04-07T16:16:13.923 INFO:tasks.workunit.client.0.smithi100.stdout: "central_cache_freelist": 621424, 2024-04-07T16:16:13.923 INFO:tasks.workunit.client.0.smithi100.stdout: "malloc_metadata": 2883584, 2024-04-07T16:16:13.923 INFO:tasks.workunit.client.0.smithi100.stdout: "page_heap_freelist": 0, 2024-04-07T16:16:13.923 INFO:tasks.workunit.client.0.smithi100.stdout: "released_to_os": 688128, 2024-04-07T16:16:13.924 INFO:tasks.workunit.client.0.smithi100.stdout: "spans_in_use": 1280, 2024-04-07T16:16:13.924 INFO:tasks.workunit.client.0.smithi100.stdout: "tcmalloc_page_size": 8192, 2024-04-07T16:16:13.924 INFO:tasks.workunit.client.0.smithi100.stdout: "thread_cache_freelists": 4438664, 2024-04-07T16:16:13.924 INFO:tasks.workunit.client.0.smithi100.stdout: "thread_heaps_in_use": 45, 2024-04-07T16:16:13.924 INFO:tasks.workunit.client.0.smithi100.stdout: "transfer_cache_freelist": 1818880, 2024-04-07T16:16:13.924 INFO:tasks.workunit.client.0.smithi100.stdout: "use_by_application": 34752776, 2024-04-07T16:16:13.924 INFO:tasks.workunit.client.0.smithi100.stdout: "virtual_address_space_used": 45203456 2024-04-07T16:16:13.924 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.924 INFO:tasks.workunit.client.0.smithi100.stdout: "osd.4": { 2024-04-07T16:16:13.924 INFO:tasks.workunit.client.0.smithi100.stdout: "actual_memory_used": 48398336, 2024-04-07T16:16:13.924 INFO:tasks.workunit.client.0.smithi100.stdout: "central_cache_freelist": 1937864, 2024-04-07T16:16:13.924 INFO:tasks.workunit.client.0.smithi100.stdout: "malloc_metadata": 2883584, 2024-04-07T16:16:13.925 INFO:tasks.workunit.client.0.smithi100.stdout: "page_heap_freelist": 0, 2024-04-07T16:16:13.925 INFO:tasks.workunit.client.0.smithi100.stdout: "released_to_os": 933888, 2024-04-07T16:16:13.925 INFO:tasks.workunit.client.0.smithi100.stdout: "spans_in_use": 1652, 2024-04-07T16:16:13.925 INFO:tasks.workunit.client.0.smithi100.stdout: "tcmalloc_page_size": 8192, 2024-04-07T16:16:13.925 INFO:tasks.workunit.client.0.smithi100.stdout: "thread_cache_freelists": 5674376, 2024-04-07T16:16:13.925 INFO:tasks.workunit.client.0.smithi100.stdout: "thread_heaps_in_use": 47, 2024-04-07T16:16:13.925 INFO:tasks.workunit.client.0.smithi100.stdout: "transfer_cache_freelist": 3033344, 2024-04-07T16:16:13.925 INFO:tasks.workunit.client.0.smithi100.stdout: "use_by_application": 34869168, 2024-04-07T16:16:13.925 INFO:tasks.workunit.client.0.smithi100.stdout: "virtual_address_space_used": 49332224 2024-04-07T16:16:13.925 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.925 INFO:tasks.workunit.client.0.smithi100.stdout: "osd.5": { 2024-04-07T16:16:13.925 INFO:tasks.workunit.client.0.smithi100.stdout: "actual_memory_used": 45432832, 2024-04-07T16:16:13.925 INFO:tasks.workunit.client.0.smithi100.stdout: "central_cache_freelist": 949104, 2024-04-07T16:16:13.925 INFO:tasks.workunit.client.0.smithi100.stdout: "malloc_metadata": 2883584, 2024-04-07T16:16:13.926 INFO:tasks.workunit.client.0.smithi100.stdout: "page_heap_freelist": 16384, 2024-04-07T16:16:13.926 INFO:tasks.workunit.client.0.smithi100.stdout: "released_to_os": 884736, 2024-04-07T16:16:13.926 INFO:tasks.workunit.client.0.smithi100.stdout: "spans_in_use": 1290, 2024-04-07T16:16:13.926 INFO:tasks.workunit.client.0.smithi100.stdout: "tcmalloc_page_size": 8192, 2024-04-07T16:16:13.926 INFO:tasks.workunit.client.0.smithi100.stdout: "thread_cache_freelists": 4586480, 2024-04-07T16:16:13.926 INFO:tasks.workunit.client.0.smithi100.stdout: "thread_heaps_in_use": 48, 2024-04-07T16:16:13.926 INFO:tasks.workunit.client.0.smithi100.stdout: "transfer_cache_freelist": 1884928, 2024-04-07T16:16:13.926 INFO:tasks.workunit.client.0.smithi100.stdout: "use_by_application": 35112352, 2024-04-07T16:16:13.926 INFO:tasks.workunit.client.0.smithi100.stdout: "virtual_address_space_used": 46317568 2024-04-07T16:16:13.926 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.926 INFO:tasks.workunit.client.0.smithi100.stdout: "osd.6": { 2024-04-07T16:16:13.926 INFO:tasks.workunit.client.0.smithi100.stdout: "actual_memory_used": 45875200, 2024-04-07T16:16:13.926 INFO:tasks.workunit.client.0.smithi100.stdout: "central_cache_freelist": 1087984, 2024-04-07T16:16:13.926 INFO:tasks.workunit.client.0.smithi100.stdout: "malloc_metadata": 2883584, 2024-04-07T16:16:13.927 INFO:tasks.workunit.client.0.smithi100.stdout: "page_heap_freelist": 0, 2024-04-07T16:16:13.927 INFO:tasks.workunit.client.0.smithi100.stdout: "released_to_os": 589824, 2024-04-07T16:16:13.927 INFO:tasks.workunit.client.0.smithi100.stdout: "spans_in_use": 1295, 2024-04-07T16:16:13.927 INFO:tasks.workunit.client.0.smithi100.stdout: "tcmalloc_page_size": 8192, 2024-04-07T16:16:13.927 INFO:tasks.workunit.client.0.smithi100.stdout: "thread_cache_freelists": 4714016, 2024-04-07T16:16:13.927 INFO:tasks.workunit.client.0.smithi100.stdout: "thread_heaps_in_use": 46, 2024-04-07T16:16:13.927 INFO:tasks.workunit.client.0.smithi100.stdout: "transfer_cache_freelist": 2215680, 2024-04-07T16:16:13.927 INFO:tasks.workunit.client.0.smithi100.stdout: "use_by_application": 34973936, 2024-04-07T16:16:13.927 INFO:tasks.workunit.client.0.smithi100.stdout: "virtual_address_space_used": 46465024 2024-04-07T16:16:13.927 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.927 INFO:tasks.workunit.client.0.smithi100.stdout: "osd.7": { 2024-04-07T16:16:13.927 INFO:tasks.workunit.client.0.smithi100.stdout: "actual_memory_used": 48955392, 2024-04-07T16:16:13.927 INFO:tasks.workunit.client.0.smithi100.stdout: "central_cache_freelist": 2094392, 2024-04-07T16:16:13.927 INFO:tasks.workunit.client.0.smithi100.stdout: "malloc_metadata": 2883584, 2024-04-07T16:16:13.928 INFO:tasks.workunit.client.0.smithi100.stdout: "page_heap_freelist": 0, 2024-04-07T16:16:13.928 INFO:tasks.workunit.client.0.smithi100.stdout: "released_to_os": 720896, 2024-04-07T16:16:13.928 INFO:tasks.workunit.client.0.smithi100.stdout: "spans_in_use": 1717, 2024-04-07T16:16:13.928 INFO:tasks.workunit.client.0.smithi100.stdout: "tcmalloc_page_size": 8192, 2024-04-07T16:16:13.928 INFO:tasks.workunit.client.0.smithi100.stdout: "thread_cache_freelists": 5694984, 2024-04-07T16:16:13.928 INFO:tasks.workunit.client.0.smithi100.stdout: "thread_heaps_in_use": 47, 2024-04-07T16:16:13.928 INFO:tasks.workunit.client.0.smithi100.stdout: "transfer_cache_freelist": 3430656, 2024-04-07T16:16:13.928 INFO:tasks.workunit.client.0.smithi100.stdout: "use_by_application": 34851776, 2024-04-07T16:16:13.928 INFO:tasks.workunit.client.0.smithi100.stdout: "virtual_address_space_used": 49676288 2024-04-07T16:16:13.928 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:13.928 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:13.928 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.928 INFO:tasks.workunit.client.0.smithi100.stdout: "hosts": { 2024-04-07T16:16:13.929 INFO:tasks.workunit.client.0.smithi100.stdout: "num": 2, 2024-04-07T16:16:13.929 INFO:tasks.workunit.client.0.smithi100.stdout: "num_with_mds": 2, 2024-04-07T16:16:13.929 INFO:tasks.workunit.client.0.smithi100.stdout: "num_with_mgr": 2, 2024-04-07T16:16:13.929 INFO:tasks.workunit.client.0.smithi100.stdout: "num_with_mon": 2, 2024-04-07T16:16:13.929 INFO:tasks.workunit.client.0.smithi100.stdout: "num_with_osd": 2 2024-04-07T16:16:13.929 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.929 INFO:tasks.workunit.client.0.smithi100.stdout: "io_rate": { 2024-04-07T16:16:13.929 INFO:tasks.workunit.client.0.smithi100.stdout: "pg_stats_delta": { 2024-04-07T16:16:13.929 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": 0, 2024-04-07T16:16:13.929 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:13.929 INFO:tasks.workunit.client.0.smithi100.stdout: "num_store_stats": 0, 2024-04-07T16:16:13.929 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:13.929 INFO:tasks.workunit.client.0.smithi100.stdout: "stamp_delta": "12.004608", 2024-04-07T16:16:13.930 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:13.930 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:13.930 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:13.930 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:13.930 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:13.930 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:13.930 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:13.930 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:13.930 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:13.931 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:13.931 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:13.931 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:13.931 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:13.931 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:13.931 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:13.931 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:13.931 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:13.931 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:13.931 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:13.931 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:13.931 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:13.931 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:13.931 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:13.932 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:13.932 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:13.932 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:13.932 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:13.932 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:13.932 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:13.932 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:13.932 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:13.932 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:13.932 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:13.932 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:13.932 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:13.932 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:13.932 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:13.933 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:13.933 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:13.933 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:13.933 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:13.933 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.933 INFO:tasks.workunit.client.0.smithi100.stdout: "store_stats": { 2024-04-07T16:16:13.933 INFO:tasks.workunit.client.0.smithi100.stdout: "allocated": 0, 2024-04-07T16:16:13.933 INFO:tasks.workunit.client.0.smithi100.stdout: "available": 0, 2024-04-07T16:16:13.933 INFO:tasks.workunit.client.0.smithi100.stdout: "data_compressed": 0, 2024-04-07T16:16:13.933 INFO:tasks.workunit.client.0.smithi100.stdout: "data_compressed_allocated": 0, 2024-04-07T16:16:13.933 INFO:tasks.workunit.client.0.smithi100.stdout: "data_compressed_original": 0, 2024-04-07T16:16:13.933 INFO:tasks.workunit.client.0.smithi100.stdout: "data_stored": 0, 2024-04-07T16:16:13.933 INFO:tasks.workunit.client.0.smithi100.stdout: "internal_metadata": 0, 2024-04-07T16:16:13.933 INFO:tasks.workunit.client.0.smithi100.stdout: "internally_reserved": 0, 2024-04-07T16:16:13.933 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_allocated": 0, 2024-04-07T16:16:13.934 INFO:tasks.workunit.client.0.smithi100.stdout: "total": 0 2024-04-07T16:16:13.934 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.934 INFO:tasks.workunit.client.0.smithi100.stdout: "up": 0 2024-04-07T16:16:13.934 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:13.934 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.934 INFO:tasks.workunit.client.0.smithi100.stdout: "leaderboard": false, 2024-04-07T16:16:13.934 INFO:tasks.workunit.client.0.smithi100.stdout: "leaderboard_description": null, 2024-04-07T16:16:13.934 INFO:tasks.workunit.client.0.smithi100.stdout: "license": "sharing-1-0", 2024-04-07T16:16:13.934 INFO:tasks.workunit.client.0.smithi100.stdout: "mempool": { 2024-04-07T16:16:13.934 INFO:tasks.workunit.client.0.smithi100.stdout: "mds": { 2024-04-07T16:16:13.934 INFO:tasks.workunit.client.0.smithi100.stdout: "mds.a52fc0d8acb515431879057649ff294cfe14c620": { 2024-04-07T16:16:13.934 INFO:tasks.workunit.client.0.smithi100.stdout: "bloom_filter": { 2024-04-07T16:16:13.934 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:13.935 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:13.935 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.935 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs": { 2024-04-07T16:16:13.935 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:13.935 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:13.935 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.935 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs_file_reader": { 2024-04-07T16:16:13.935 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:13.935 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:13.935 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.935 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs_file_writer": { 2024-04-07T16:16:13.935 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:13.935 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:13.935 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.935 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_alloc": { 2024-04-07T16:16:13.936 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:13.936 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:13.936 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.936 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_blob": { 2024-04-07T16:16:13.936 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:13.936 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:13.936 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.936 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_buffer": { 2024-04-07T16:16:13.936 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:13.936 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:13.936 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.936 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_data": { 2024-04-07T16:16:13.936 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:13.936 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:13.936 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.937 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_meta": { 2024-04-07T16:16:13.937 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:13.937 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:13.937 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.937 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_onode": { 2024-04-07T16:16:13.937 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:13.937 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:13.937 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.937 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_other": { 2024-04-07T16:16:13.937 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:13.937 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:13.937 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.937 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_extent": { 2024-04-07T16:16:13.937 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:13.938 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:13.938 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.938 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_fsck": { 2024-04-07T16:16:13.938 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:13.938 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:13.938 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.938 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_inline_bl": { 2024-04-07T16:16:13.938 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:13.938 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:13.938 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.938 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_shared_blob": { 2024-04-07T16:16:13.938 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:13.938 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:13.938 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.938 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_txc": { 2024-04-07T16:16:13.939 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:13.939 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:13.939 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.939 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_writing": { 2024-04-07T16:16:13.939 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:13.939 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:13.939 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.939 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_writing_deferred": { 2024-04-07T16:16:13.939 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:13.939 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:13.939 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.939 INFO:tasks.workunit.client.0.smithi100.stdout: "buffer_anon": { 2024-04-07T16:16:13.939 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 222727, 2024-04-07T16:16:13.939 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 72 2024-04-07T16:16:13.940 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.940 INFO:tasks.workunit.client.0.smithi100.stdout: "buffer_meta": { 2024-04-07T16:16:13.940 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:13.940 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:13.940 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.940 INFO:tasks.workunit.client.0.smithi100.stdout: "mds_co": { 2024-04-07T16:16:13.940 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 51496, 2024-04-07T16:16:13.940 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 208 2024-04-07T16:16:13.940 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.940 INFO:tasks.workunit.client.0.smithi100.stdout: "osd": { 2024-04-07T16:16:13.940 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:13.940 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:13.940 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.940 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mapbl": { 2024-04-07T16:16:13.940 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:13.941 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:13.941 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.941 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_pglog": { 2024-04-07T16:16:13.941 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:13.941 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:13.941 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.941 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap": { 2024-04-07T16:16:13.941 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 7912, 2024-04-07T16:16:13.941 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 123 2024-04-07T16:16:13.941 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.941 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap_mapping": { 2024-04-07T16:16:13.941 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:13.941 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:13.942 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.942 INFO:tasks.workunit.client.0.smithi100.stdout: "pgmap": { 2024-04-07T16:16:13.942 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:13.942 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:13.942 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.942 INFO:tasks.workunit.client.0.smithi100.stdout: "unittest_1": { 2024-04-07T16:16:13.942 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:13.942 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:13.942 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.942 INFO:tasks.workunit.client.0.smithi100.stdout: "unittest_2": { 2024-04-07T16:16:13.942 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:13.942 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:13.942 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:13.943 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.943 INFO:tasks.workunit.client.0.smithi100.stdout: "mds.f47a4ff15a45950a63160265564a123fc5da5e5e": { 2024-04-07T16:16:13.943 INFO:tasks.workunit.client.0.smithi100.stdout: "bloom_filter": { 2024-04-07T16:16:13.943 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:13.943 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:13.943 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.943 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs": { 2024-04-07T16:16:13.943 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:13.943 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:13.943 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.943 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs_file_reader": { 2024-04-07T16:16:13.943 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:13.943 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:13.943 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.943 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs_file_writer": { 2024-04-07T16:16:13.944 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:13.944 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:13.944 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.944 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_alloc": { 2024-04-07T16:16:13.944 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:13.944 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:13.944 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.944 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_blob": { 2024-04-07T16:16:13.944 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:13.944 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:13.944 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.944 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_buffer": { 2024-04-07T16:16:13.944 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:13.944 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:13.945 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.945 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_data": { 2024-04-07T16:16:13.945 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:13.945 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:13.945 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.945 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_meta": { 2024-04-07T16:16:13.945 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:13.945 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:13.945 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.945 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_onode": { 2024-04-07T16:16:13.945 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:13.945 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:13.945 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.945 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_other": { 2024-04-07T16:16:13.946 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:13.946 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:13.946 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.946 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_extent": { 2024-04-07T16:16:13.946 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:13.946 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:13.946 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.946 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_fsck": { 2024-04-07T16:16:13.946 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:13.946 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:13.946 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.946 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_inline_bl": { 2024-04-07T16:16:13.946 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:13.947 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:13.947 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.947 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_shared_blob": { 2024-04-07T16:16:13.947 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:13.947 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:13.947 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.947 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_txc": { 2024-04-07T16:16:13.947 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:13.947 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:13.947 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.947 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_writing": { 2024-04-07T16:16:13.947 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:13.947 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:13.947 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.948 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_writing_deferred": { 2024-04-07T16:16:13.948 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:13.948 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:13.948 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.948 INFO:tasks.workunit.client.0.smithi100.stdout: "buffer_anon": { 2024-04-07T16:16:13.948 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 150208, 2024-04-07T16:16:13.948 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 34 2024-04-07T16:16:13.948 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.948 INFO:tasks.workunit.client.0.smithi100.stdout: "buffer_meta": { 2024-04-07T16:16:13.948 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:13.948 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:13.948 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.948 INFO:tasks.workunit.client.0.smithi100.stdout: "mds_co": { 2024-04-07T16:16:13.948 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 928, 2024-04-07T16:16:13.949 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 2 2024-04-07T16:16:13.949 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.949 INFO:tasks.workunit.client.0.smithi100.stdout: "osd": { 2024-04-07T16:16:13.949 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:13.949 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:13.949 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.949 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mapbl": { 2024-04-07T16:16:13.949 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:13.949 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:13.949 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.949 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_pglog": { 2024-04-07T16:16:13.949 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:13.949 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:13.949 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.949 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap": { 2024-04-07T16:16:13.950 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:13.950 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:13.950 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.950 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap_mapping": { 2024-04-07T16:16:13.950 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:13.950 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:13.950 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.950 INFO:tasks.workunit.client.0.smithi100.stdout: "pgmap": { 2024-04-07T16:16:13.950 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:13.951 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:13.951 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.951 INFO:tasks.workunit.client.0.smithi100.stdout: "unittest_1": { 2024-04-07T16:16:13.951 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:13.951 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:13.951 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.951 INFO:tasks.workunit.client.0.smithi100.stdout: "unittest_2": { 2024-04-07T16:16:13.951 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:13.951 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:13.951 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:13.952 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:13.952 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.952 INFO:tasks.workunit.client.0.smithi100.stdout: "mon": { 2024-04-07T16:16:13.952 INFO:tasks.workunit.client.0.smithi100.stdout: "mon.8dd0baa036f0a7e6d1f3bdce86136fe527836c3d": { 2024-04-07T16:16:13.952 INFO:tasks.workunit.client.0.smithi100.stdout: "bloom_filter": { 2024-04-07T16:16:13.952 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:13.952 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:13.952 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.952 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs": { 2024-04-07T16:16:13.952 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:13.952 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:13.952 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.952 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs_file_reader": { 2024-04-07T16:16:13.952 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:13.953 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:13.953 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.953 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs_file_writer": { 2024-04-07T16:16:13.953 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:13.953 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:13.953 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.953 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_alloc": { 2024-04-07T16:16:13.953 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:13.953 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:13.953 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.953 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_blob": { 2024-04-07T16:16:13.953 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:13.953 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:13.953 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.953 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_buffer": { 2024-04-07T16:16:13.954 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:13.954 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:13.954 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.954 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_data": { 2024-04-07T16:16:13.954 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:13.954 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:13.954 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.954 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_meta": { 2024-04-07T16:16:13.954 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:13.954 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:13.954 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.954 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_onode": { 2024-04-07T16:16:13.955 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:13.955 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:13.955 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.955 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_other": { 2024-04-07T16:16:13.955 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:13.955 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:13.955 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.955 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_extent": { 2024-04-07T16:16:13.955 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:13.955 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:13.955 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.955 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_fsck": { 2024-04-07T16:16:13.955 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:13.955 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:13.956 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.956 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_inline_bl": { 2024-04-07T16:16:13.956 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:13.956 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:13.956 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.956 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_shared_blob": { 2024-04-07T16:16:13.956 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:13.956 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:13.956 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.956 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_txc": { 2024-04-07T16:16:13.956 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:13.956 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:13.956 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.956 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_writing": { 2024-04-07T16:16:13.956 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:13.957 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:13.957 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.957 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_writing_deferred": { 2024-04-07T16:16:13.957 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:13.957 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:13.957 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.957 INFO:tasks.workunit.client.0.smithi100.stdout: "buffer_anon": { 2024-04-07T16:16:13.957 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 1117132, 2024-04-07T16:16:13.957 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 362 2024-04-07T16:16:13.957 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.957 INFO:tasks.workunit.client.0.smithi100.stdout: "buffer_meta": { 2024-04-07T16:16:13.957 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:13.957 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:13.957 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.958 INFO:tasks.workunit.client.0.smithi100.stdout: "mds_co": { 2024-04-07T16:16:13.958 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:13.958 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:13.958 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.958 INFO:tasks.workunit.client.0.smithi100.stdout: "osd": { 2024-04-07T16:16:13.958 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:13.958 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:13.958 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.958 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mapbl": { 2024-04-07T16:16:13.958 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:13.958 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:13.958 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.958 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_pglog": { 2024-04-07T16:16:13.958 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:13.958 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:13.959 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.959 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap": { 2024-04-07T16:16:13.959 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 6488, 2024-04-07T16:16:13.959 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 122 2024-04-07T16:16:13.959 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.959 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap_mapping": { 2024-04-07T16:16:13.959 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 12744, 2024-04-07T16:16:13.959 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 1546 2024-04-07T16:16:13.959 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.959 INFO:tasks.workunit.client.0.smithi100.stdout: "pgmap": { 2024-04-07T16:16:13.959 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 5152, 2024-04-07T16:16:13.959 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 83 2024-04-07T16:16:13.959 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.959 INFO:tasks.workunit.client.0.smithi100.stdout: "unittest_1": { 2024-04-07T16:16:13.959 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:13.960 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:13.960 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.960 INFO:tasks.workunit.client.0.smithi100.stdout: "unittest_2": { 2024-04-07T16:16:13.960 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:13.960 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:13.960 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:13.960 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.960 INFO:tasks.workunit.client.0.smithi100.stdout: "mon.a52fc0d8acb515431879057649ff294cfe14c620": { 2024-04-07T16:16:13.960 INFO:tasks.workunit.client.0.smithi100.stdout: "bloom_filter": { 2024-04-07T16:16:13.960 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:13.960 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:13.960 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.960 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs": { 2024-04-07T16:16:13.960 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:13.961 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:13.961 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.961 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs_file_reader": { 2024-04-07T16:16:13.961 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:13.961 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:13.961 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.961 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs_file_writer": { 2024-04-07T16:16:13.961 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:13.961 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:13.961 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.961 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_alloc": { 2024-04-07T16:16:13.961 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:13.961 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:13.962 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.962 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_blob": { 2024-04-07T16:16:13.962 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:13.962 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:13.962 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.962 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_buffer": { 2024-04-07T16:16:13.962 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:13.962 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:13.962 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.962 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_data": { 2024-04-07T16:16:13.962 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:13.962 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:13.963 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.963 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_meta": { 2024-04-07T16:16:13.963 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:13.963 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:13.963 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.963 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_onode": { 2024-04-07T16:16:13.963 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:13.963 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:13.963 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.963 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_other": { 2024-04-07T16:16:13.963 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:13.963 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:13.963 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.963 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_extent": { 2024-04-07T16:16:13.964 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:13.964 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:13.964 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.964 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_fsck": { 2024-04-07T16:16:13.964 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:13.964 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:13.964 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.964 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_inline_bl": { 2024-04-07T16:16:13.964 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:13.964 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:13.964 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.964 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_shared_blob": { 2024-04-07T16:16:13.964 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:13.964 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:13.965 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.965 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_txc": { 2024-04-07T16:16:13.965 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:13.965 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:13.965 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.965 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_writing": { 2024-04-07T16:16:13.965 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:13.965 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:13.965 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.965 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_writing_deferred": { 2024-04-07T16:16:13.965 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:13.965 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:13.965 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.965 INFO:tasks.workunit.client.0.smithi100.stdout: "buffer_anon": { 2024-04-07T16:16:13.965 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 1154909, 2024-04-07T16:16:13.966 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 372 2024-04-07T16:16:13.966 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.966 INFO:tasks.workunit.client.0.smithi100.stdout: "buffer_meta": { 2024-04-07T16:16:13.966 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 88, 2024-04-07T16:16:13.966 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 1 2024-04-07T16:16:13.966 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.966 INFO:tasks.workunit.client.0.smithi100.stdout: "mds_co": { 2024-04-07T16:16:13.966 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:13.966 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:13.966 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.966 INFO:tasks.workunit.client.0.smithi100.stdout: "osd": { 2024-04-07T16:16:13.966 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:13.966 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:13.966 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.967 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mapbl": { 2024-04-07T16:16:13.967 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:13.967 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:13.967 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.967 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_pglog": { 2024-04-07T16:16:13.967 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:13.967 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:13.967 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.967 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap": { 2024-04-07T16:16:13.967 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 6488, 2024-04-07T16:16:13.967 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 122 2024-04-07T16:16:13.967 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.967 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap_mapping": { 2024-04-07T16:16:13.967 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 12744, 2024-04-07T16:16:13.967 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 1546 2024-04-07T16:16:13.968 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.968 INFO:tasks.workunit.client.0.smithi100.stdout: "pgmap": { 2024-04-07T16:16:13.968 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 5152, 2024-04-07T16:16:13.968 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 83 2024-04-07T16:16:13.968 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.968 INFO:tasks.workunit.client.0.smithi100.stdout: "unittest_1": { 2024-04-07T16:16:13.968 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:13.968 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:13.968 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.968 INFO:tasks.workunit.client.0.smithi100.stdout: "unittest_2": { 2024-04-07T16:16:13.968 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:13.968 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:13.968 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:13.969 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.969 INFO:tasks.workunit.client.0.smithi100.stdout: "mon.f47a4ff15a45950a63160265564a123fc5da5e5e": { 2024-04-07T16:16:13.969 INFO:tasks.workunit.client.0.smithi100.stdout: "bloom_filter": { 2024-04-07T16:16:13.969 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:13.969 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:13.969 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.969 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs": { 2024-04-07T16:16:13.969 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:13.969 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:13.969 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.969 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs_file_reader": { 2024-04-07T16:16:13.969 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:13.969 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:13.969 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.969 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs_file_writer": { 2024-04-07T16:16:13.970 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:13.970 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:13.970 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.970 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_alloc": { 2024-04-07T16:16:13.970 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:13.970 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:13.970 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.970 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_blob": { 2024-04-07T16:16:13.970 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:13.970 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:13.970 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.970 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_buffer": { 2024-04-07T16:16:13.970 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:13.970 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:13.971 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.971 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_data": { 2024-04-07T16:16:13.971 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:13.971 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:13.971 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.971 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_meta": { 2024-04-07T16:16:13.971 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:13.971 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:13.971 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.971 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_onode": { 2024-04-07T16:16:13.971 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:13.971 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:13.971 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.971 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_other": { 2024-04-07T16:16:13.971 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:13.972 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:13.972 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.972 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_extent": { 2024-04-07T16:16:13.972 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:13.972 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:13.972 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.972 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_fsck": { 2024-04-07T16:16:13.972 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:13.972 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:13.972 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.972 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_inline_bl": { 2024-04-07T16:16:13.972 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:13.972 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:13.972 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.973 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_shared_blob": { 2024-04-07T16:16:13.973 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:13.973 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:13.973 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.973 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_txc": { 2024-04-07T16:16:13.973 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:13.973 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:13.973 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.973 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_writing": { 2024-04-07T16:16:13.973 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:13.973 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:13.973 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.973 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_writing_deferred": { 2024-04-07T16:16:13.973 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:13.973 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:13.974 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.974 INFO:tasks.workunit.client.0.smithi100.stdout: "buffer_anon": { 2024-04-07T16:16:13.974 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 1049692, 2024-04-07T16:16:13.974 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 336 2024-04-07T16:16:13.974 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.974 INFO:tasks.workunit.client.0.smithi100.stdout: "buffer_meta": { 2024-04-07T16:16:13.974 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:13.974 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:13.974 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.974 INFO:tasks.workunit.client.0.smithi100.stdout: "mds_co": { 2024-04-07T16:16:13.974 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:13.974 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:13.974 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.975 INFO:tasks.workunit.client.0.smithi100.stdout: "osd": { 2024-04-07T16:16:13.975 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:13.975 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:13.975 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.975 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mapbl": { 2024-04-07T16:16:13.975 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:13.975 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:13.975 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.975 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_pglog": { 2024-04-07T16:16:13.976 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:13.976 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:13.976 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.976 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap": { 2024-04-07T16:16:13.976 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 6488, 2024-04-07T16:16:13.976 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 122 2024-04-07T16:16:13.977 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.977 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap_mapping": { 2024-04-07T16:16:13.977 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 12744, 2024-04-07T16:16:13.977 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 1546 2024-04-07T16:16:13.977 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.977 INFO:tasks.workunit.client.0.smithi100.stdout: "pgmap": { 2024-04-07T16:16:13.977 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 10304, 2024-04-07T16:16:13.978 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 166 2024-04-07T16:16:13.978 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.978 INFO:tasks.workunit.client.0.smithi100.stdout: "unittest_1": { 2024-04-07T16:16:13.978 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:13.978 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:13.978 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.978 INFO:tasks.workunit.client.0.smithi100.stdout: "unittest_2": { 2024-04-07T16:16:13.979 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:13.979 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:13.979 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:13.979 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:13.979 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.979 INFO:tasks.workunit.client.0.smithi100.stdout: "osd": { 2024-04-07T16:16:13.979 INFO:tasks.workunit.client.0.smithi100.stdout: "osd.0": { 2024-04-07T16:16:13.979 INFO:tasks.workunit.client.0.smithi100.stdout: "bloom_filter": { 2024-04-07T16:16:13.980 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:13.980 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:13.980 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.980 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs": { 2024-04-07T16:16:13.980 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 6368, 2024-04-07T16:16:13.980 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 123 2024-04-07T16:16:13.980 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.980 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs_file_reader": { 2024-04-07T16:16:13.981 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 16896, 2024-04-07T16:16:13.981 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 8 2024-04-07T16:16:13.981 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.981 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs_file_writer": { 2024-04-07T16:16:13.981 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 576, 2024-04-07T16:16:13.981 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 3 2024-04-07T16:16:13.981 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.982 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_alloc": { 2024-04-07T16:16:13.982 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 5600, 2024-04-07T16:16:13.982 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 70 2024-04-07T16:16:13.982 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.982 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_blob": { 2024-04-07T16:16:13.982 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 20944, 2024-04-07T16:16:13.982 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 187 2024-04-07T16:16:13.982 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.983 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_buffer": { 2024-04-07T16:16:13.983 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 1936, 2024-04-07T16:16:13.983 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 47 2024-04-07T16:16:13.983 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.983 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_data": { 2024-04-07T16:16:13.983 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 620927, 2024-04-07T16:16:13.983 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 20 2024-04-07T16:16:13.984 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.984 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_meta": { 2024-04-07T16:16:13.984 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 50844, 2024-04-07T16:16:13.984 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 9431 2024-04-07T16:16:13.984 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.984 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_onode": { 2024-04-07T16:16:13.984 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 136256, 2024-04-07T16:16:13.984 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 232 2024-04-07T16:16:13.985 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.985 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_other": { 2024-04-07T16:16:13.985 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 11260, 2024-04-07T16:16:13.985 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 741 2024-04-07T16:16:13.985 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.985 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_extent": { 2024-04-07T16:16:13.985 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 8976, 2024-04-07T16:16:13.986 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 187 2024-04-07T16:16:13.986 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.986 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_fsck": { 2024-04-07T16:16:13.986 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:13.986 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:13.986 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.986 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_inline_bl": { 2024-04-07T16:16:13.987 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 31446, 2024-04-07T16:16:13.987 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 200 2024-04-07T16:16:13.987 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.987 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_shared_blob": { 2024-04-07T16:16:13.987 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 20944, 2024-04-07T16:16:13.987 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 187 2024-04-07T16:16:13.988 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.988 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_txc": { 2024-04-07T16:16:13.988 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:13.988 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:13.988 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.988 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_writing": { 2024-04-07T16:16:13.988 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 407616, 2024-04-07T16:16:13.989 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 91 2024-04-07T16:16:13.989 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.989 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_writing_deferred": { 2024-04-07T16:16:13.989 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:13.989 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:13.989 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.989 INFO:tasks.workunit.client.0.smithi100.stdout: "buffer_anon": { 2024-04-07T16:16:13.989 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 2460800, 2024-04-07T16:16:13.990 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 202 2024-04-07T16:16:13.990 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.990 INFO:tasks.workunit.client.0.smithi100.stdout: "buffer_meta": { 2024-04-07T16:16:13.990 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 6952, 2024-04-07T16:16:13.990 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 79 2024-04-07T16:16:13.990 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.990 INFO:tasks.workunit.client.0.smithi100.stdout: "mds_co": { 2024-04-07T16:16:13.991 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:13.991 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:13.991 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.991 INFO:tasks.workunit.client.0.smithi100.stdout: "osd": { 2024-04-07T16:16:13.991 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 510048, 2024-04-07T16:16:13.991 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 44 2024-04-07T16:16:13.991 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.991 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mapbl": { 2024-04-07T16:16:13.991 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:13.991 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:13.992 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.992 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_pglog": { 2024-04-07T16:16:13.992 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 92560, 2024-04-07T16:16:13.992 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 182 2024-04-07T16:16:13.992 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.992 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap": { 2024-04-07T16:16:13.992 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 225512, 2024-04-07T16:16:13.992 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 3460 2024-04-07T16:16:13.993 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.993 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap_mapping": { 2024-04-07T16:16:13.993 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:13.993 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:13.993 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.993 INFO:tasks.workunit.client.0.smithi100.stdout: "pgmap": { 2024-04-07T16:16:13.994 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:13.994 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:13.994 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.994 INFO:tasks.workunit.client.0.smithi100.stdout: "unittest_1": { 2024-04-07T16:16:13.994 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:13.994 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:13.994 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.994 INFO:tasks.workunit.client.0.smithi100.stdout: "unittest_2": { 2024-04-07T16:16:13.994 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:13.995 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:13.995 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:13.995 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.995 INFO:tasks.workunit.client.0.smithi100.stdout: "osd.1": { 2024-04-07T16:16:13.995 INFO:tasks.workunit.client.0.smithi100.stdout: "bloom_filter": { 2024-04-07T16:16:13.995 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:13.995 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:13.995 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.995 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs": { 2024-04-07T16:16:13.995 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 6368, 2024-04-07T16:16:13.995 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 123 2024-04-07T16:16:13.995 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.995 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs_file_reader": { 2024-04-07T16:16:13.995 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 16896, 2024-04-07T16:16:13.995 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 8 2024-04-07T16:16:13.996 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.996 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs_file_writer": { 2024-04-07T16:16:13.996 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 576, 2024-04-07T16:16:13.996 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 3 2024-04-07T16:16:13.996 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.996 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_alloc": { 2024-04-07T16:16:13.996 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 4720, 2024-04-07T16:16:13.996 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 59 2024-04-07T16:16:13.996 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.996 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_blob": { 2024-04-07T16:16:13.996 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 19600, 2024-04-07T16:16:13.996 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 175 2024-04-07T16:16:13.996 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.996 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_buffer": { 2024-04-07T16:16:13.996 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 1264, 2024-04-07T16:16:13.997 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 41 2024-04-07T16:16:13.997 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.997 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_data": { 2024-04-07T16:16:13.997 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 16384, 2024-04-07T16:16:13.997 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 4 2024-04-07T16:16:13.997 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.997 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_meta": { 2024-04-07T16:16:13.997 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 45252, 2024-04-07T16:16:13.997 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 8555 2024-04-07T16:16:13.997 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.997 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_onode": { 2024-04-07T16:16:13.997 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 128096, 2024-04-07T16:16:13.997 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 220 2024-04-07T16:16:13.997 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.998 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_other": { 2024-04-07T16:16:13.998 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 9440, 2024-04-07T16:16:13.998 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 568 2024-04-07T16:16:13.998 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.998 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_extent": { 2024-04-07T16:16:13.998 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 8400, 2024-04-07T16:16:13.998 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 175 2024-04-07T16:16:13.998 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.998 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_fsck": { 2024-04-07T16:16:13.998 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:13.998 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:13.998 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.998 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_inline_bl": { 2024-04-07T16:16:13.999 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 27770, 2024-04-07T16:16:13.999 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 188 2024-04-07T16:16:13.999 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.999 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_shared_blob": { 2024-04-07T16:16:13.999 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 19600, 2024-04-07T16:16:13.999 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 175 2024-04-07T16:16:13.999 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.999 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_txc": { 2024-04-07T16:16:13.999 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 736, 2024-04-07T16:16:13.999 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 1 2024-04-07T16:16:13.999 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:13.999 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_writing": { 2024-04-07T16:16:13.999 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 434028, 2024-04-07T16:16:13.999 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 97 2024-04-07T16:16:13.999 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.000 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_writing_deferred": { 2024-04-07T16:16:14.000 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 12112, 2024-04-07T16:16:14.000 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 3 2024-04-07T16:16:14.000 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.000 INFO:tasks.workunit.client.0.smithi100.stdout: "buffer_anon": { 2024-04-07T16:16:14.000 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 2420101, 2024-04-07T16:16:14.000 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 185 2024-04-07T16:16:14.000 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.000 INFO:tasks.workunit.client.0.smithi100.stdout: "buffer_meta": { 2024-04-07T16:16:14.000 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 5016, 2024-04-07T16:16:14.000 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 57 2024-04-07T16:16:14.000 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.000 INFO:tasks.workunit.client.0.smithi100.stdout: "mds_co": { 2024-04-07T16:16:14.000 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:14.000 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:14.000 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.001 INFO:tasks.workunit.client.0.smithi100.stdout: "osd": { 2024-04-07T16:16:14.001 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 440496, 2024-04-07T16:16:14.001 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 38 2024-04-07T16:16:14.001 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.001 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mapbl": { 2024-04-07T16:16:14.001 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:14.001 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:14.001 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.001 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_pglog": { 2024-04-07T16:16:14.001 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 7280, 2024-04-07T16:16:14.001 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 18 2024-04-07T16:16:14.001 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.002 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap": { 2024-04-07T16:16:14.002 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 225512, 2024-04-07T16:16:14.002 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 3460 2024-04-07T16:16:14.002 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.002 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap_mapping": { 2024-04-07T16:16:14.002 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:14.002 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:14.002 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.002 INFO:tasks.workunit.client.0.smithi100.stdout: "pgmap": { 2024-04-07T16:16:14.002 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:14.002 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:14.002 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.002 INFO:tasks.workunit.client.0.smithi100.stdout: "unittest_1": { 2024-04-07T16:16:14.003 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:14.003 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:14.003 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.003 INFO:tasks.workunit.client.0.smithi100.stdout: "unittest_2": { 2024-04-07T16:16:14.003 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:14.003 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:14.003 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:14.003 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.003 INFO:tasks.workunit.client.0.smithi100.stdout: "osd.2": { 2024-04-07T16:16:14.003 INFO:tasks.workunit.client.0.smithi100.stdout: "bloom_filter": { 2024-04-07T16:16:14.003 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:14.003 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:14.003 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.003 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs": { 2024-04-07T16:16:14.003 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 6368, 2024-04-07T16:16:14.003 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 123 2024-04-07T16:16:14.004 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.004 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs_file_reader": { 2024-04-07T16:16:14.004 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 16896, 2024-04-07T16:16:14.004 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 8 2024-04-07T16:16:14.004 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.004 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs_file_writer": { 2024-04-07T16:16:14.004 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 576, 2024-04-07T16:16:14.004 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 3 2024-04-07T16:16:14.004 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.004 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_alloc": { 2024-04-07T16:16:14.004 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 4160, 2024-04-07T16:16:14.004 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 52 2024-04-07T16:16:14.004 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.004 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_blob": { 2024-04-07T16:16:14.004 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 19376, 2024-04-07T16:16:14.005 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 173 2024-04-07T16:16:14.005 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.005 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_buffer": { 2024-04-07T16:16:14.005 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 1040, 2024-04-07T16:16:14.005 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 39 2024-04-07T16:16:14.005 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.005 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_data": { 2024-04-07T16:16:14.005 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 8192, 2024-04-07T16:16:14.005 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 2 2024-04-07T16:16:14.005 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.005 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_meta": { 2024-04-07T16:16:14.005 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 83624, 2024-04-07T16:16:14.005 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 14455 2024-04-07T16:16:14.005 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.005 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_onode": { 2024-04-07T16:16:14.005 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 189976, 2024-04-07T16:16:14.006 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 311 2024-04-07T16:16:14.006 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.006 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_other": { 2024-04-07T16:16:14.006 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 8168, 2024-04-07T16:16:14.006 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 514 2024-04-07T16:16:14.006 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.006 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_extent": { 2024-04-07T16:16:14.006 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 8304, 2024-04-07T16:16:14.006 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 173 2024-04-07T16:16:14.006 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.006 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_fsck": { 2024-04-07T16:16:14.006 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:14.006 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:14.006 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.007 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_inline_bl": { 2024-04-07T16:16:14.007 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 25802, 2024-04-07T16:16:14.007 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 179 2024-04-07T16:16:14.007 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.007 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_shared_blob": { 2024-04-07T16:16:14.007 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 19376, 2024-04-07T16:16:14.007 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 173 2024-04-07T16:16:14.007 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.007 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_txc": { 2024-04-07T16:16:14.007 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 736, 2024-04-07T16:16:14.007 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 1 2024-04-07T16:16:14.007 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.007 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_writing": { 2024-04-07T16:16:14.007 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 350932, 2024-04-07T16:16:14.008 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 77 2024-04-07T16:16:14.008 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.008 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_writing_deferred": { 2024-04-07T16:16:14.008 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 12112, 2024-04-07T16:16:14.008 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 3 2024-04-07T16:16:14.008 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.008 INFO:tasks.workunit.client.0.smithi100.stdout: "buffer_anon": { 2024-04-07T16:16:14.008 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 2422924, 2024-04-07T16:16:14.008 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 191 2024-04-07T16:16:14.008 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.008 INFO:tasks.workunit.client.0.smithi100.stdout: "buffer_meta": { 2024-04-07T16:16:14.008 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 4224, 2024-04-07T16:16:14.008 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 48 2024-04-07T16:16:14.008 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.008 INFO:tasks.workunit.client.0.smithi100.stdout: "mds_co": { 2024-04-07T16:16:14.009 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:14.009 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:14.009 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.009 INFO:tasks.workunit.client.0.smithi100.stdout: "osd": { 2024-04-07T16:16:14.009 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 347760, 2024-04-07T16:16:14.009 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 30 2024-04-07T16:16:14.009 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.009 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mapbl": { 2024-04-07T16:16:14.009 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:14.009 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:14.009 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.009 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_pglog": { 2024-04-07T16:16:14.010 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 8320, 2024-04-07T16:16:14.010 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 24 2024-04-07T16:16:14.010 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.010 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap": { 2024-04-07T16:16:14.010 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 226024, 2024-04-07T16:16:14.010 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 3484 2024-04-07T16:16:14.010 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.010 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap_mapping": { 2024-04-07T16:16:14.010 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:14.010 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:14.010 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.010 INFO:tasks.workunit.client.0.smithi100.stdout: "pgmap": { 2024-04-07T16:16:14.011 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:14.011 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:14.011 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.011 INFO:tasks.workunit.client.0.smithi100.stdout: "unittest_1": { 2024-04-07T16:16:14.011 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:14.011 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:14.011 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.012 INFO:tasks.workunit.client.0.smithi100.stdout: "unittest_2": { 2024-04-07T16:16:14.012 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:14.012 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:14.012 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:14.012 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.013 INFO:tasks.workunit.client.0.smithi100.stdout: "osd.3": { 2024-04-07T16:16:14.013 INFO:tasks.workunit.client.0.smithi100.stdout: "bloom_filter": { 2024-04-07T16:16:14.013 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:14.013 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:14.013 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.013 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs": { 2024-04-07T16:16:14.013 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 6368, 2024-04-07T16:16:14.014 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 123 2024-04-07T16:16:14.014 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.014 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs_file_reader": { 2024-04-07T16:16:14.014 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 16896, 2024-04-07T16:16:14.014 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 8 2024-04-07T16:16:14.015 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.015 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs_file_writer": { 2024-04-07T16:16:14.015 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 576, 2024-04-07T16:16:14.015 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 3 2024-04-07T16:16:14.015 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.015 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_alloc": { 2024-04-07T16:16:14.015 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 3840, 2024-04-07T16:16:14.015 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 48 2024-04-07T16:16:14.015 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.015 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_blob": { 2024-04-07T16:16:14.015 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 19936, 2024-04-07T16:16:14.015 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 178 2024-04-07T16:16:14.015 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.015 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_buffer": { 2024-04-07T16:16:14.016 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 480, 2024-04-07T16:16:14.016 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 34 2024-04-07T16:16:14.016 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.016 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_data": { 2024-04-07T16:16:14.016 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:14.016 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:14.016 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.016 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_meta": { 2024-04-07T16:16:14.016 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 44190, 2024-04-07T16:16:14.016 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 8467 2024-04-07T16:16:14.016 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.016 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_onode": { 2024-04-07T16:16:14.016 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 126736, 2024-04-07T16:16:14.016 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 218 2024-04-07T16:16:14.016 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.017 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_other": { 2024-04-07T16:16:14.017 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 9308, 2024-04-07T16:16:14.017 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 574 2024-04-07T16:16:14.017 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.017 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_extent": { 2024-04-07T16:16:14.017 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 8544, 2024-04-07T16:16:14.017 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 178 2024-04-07T16:16:14.017 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.017 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_fsck": { 2024-04-07T16:16:14.017 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:14.017 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:14.017 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.017 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_inline_bl": { 2024-04-07T16:16:14.017 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 26754, 2024-04-07T16:16:14.018 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 186 2024-04-07T16:16:14.018 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.018 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_shared_blob": { 2024-04-07T16:16:14.018 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 19936, 2024-04-07T16:16:14.018 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 178 2024-04-07T16:16:14.018 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.018 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_txc": { 2024-04-07T16:16:14.018 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 2208, 2024-04-07T16:16:14.018 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 3 2024-04-07T16:16:14.018 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.018 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_writing": { 2024-04-07T16:16:14.018 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 277799, 2024-04-07T16:16:14.018 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 59 2024-04-07T16:16:14.019 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.019 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_writing_deferred": { 2024-04-07T16:16:14.019 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 26565, 2024-04-07T16:16:14.019 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 9 2024-04-07T16:16:14.019 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.019 INFO:tasks.workunit.client.0.smithi100.stdout: "buffer_anon": { 2024-04-07T16:16:14.019 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 2446914, 2024-04-07T16:16:14.019 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 197 2024-04-07T16:16:14.019 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.019 INFO:tasks.workunit.client.0.smithi100.stdout: "buffer_meta": { 2024-04-07T16:16:14.019 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 4664, 2024-04-07T16:16:14.019 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 53 2024-04-07T16:16:14.019 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.019 INFO:tasks.workunit.client.0.smithi100.stdout: "mds_co": { 2024-04-07T16:16:14.020 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:14.020 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:14.020 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.020 INFO:tasks.workunit.client.0.smithi100.stdout: "osd": { 2024-04-07T16:16:14.020 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 417312, 2024-04-07T16:16:14.020 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 36 2024-04-07T16:16:14.020 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.020 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mapbl": { 2024-04-07T16:16:14.020 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:14.020 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:14.020 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.020 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_pglog": { 2024-04-07T16:16:14.020 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 10400, 2024-04-07T16:16:14.020 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 20 2024-04-07T16:16:14.021 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.021 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap": { 2024-04-07T16:16:14.021 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 225512, 2024-04-07T16:16:14.021 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 3460 2024-04-07T16:16:14.021 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.021 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap_mapping": { 2024-04-07T16:16:14.021 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:14.021 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:14.021 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.021 INFO:tasks.workunit.client.0.smithi100.stdout: "pgmap": { 2024-04-07T16:16:14.021 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:14.021 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:14.021 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.021 INFO:tasks.workunit.client.0.smithi100.stdout: "unittest_1": { 2024-04-07T16:16:14.021 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:14.022 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:14.022 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.022 INFO:tasks.workunit.client.0.smithi100.stdout: "unittest_2": { 2024-04-07T16:16:14.022 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:14.022 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:14.022 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:14.022 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.022 INFO:tasks.workunit.client.0.smithi100.stdout: "osd.4": { 2024-04-07T16:16:14.022 INFO:tasks.workunit.client.0.smithi100.stdout: "bloom_filter": { 2024-04-07T16:16:14.022 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:14.022 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:14.022 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.022 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs": { 2024-04-07T16:16:14.022 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 6368, 2024-04-07T16:16:14.022 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 123 2024-04-07T16:16:14.023 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.023 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs_file_reader": { 2024-04-07T16:16:14.023 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 16896, 2024-04-07T16:16:14.023 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 8 2024-04-07T16:16:14.023 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.023 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs_file_writer": { 2024-04-07T16:16:14.023 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 576, 2024-04-07T16:16:14.023 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 3 2024-04-07T16:16:14.023 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.023 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_alloc": { 2024-04-07T16:16:14.023 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 3600, 2024-04-07T16:16:14.023 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 45 2024-04-07T16:16:14.023 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.023 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_blob": { 2024-04-07T16:16:14.023 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 19824, 2024-04-07T16:16:14.024 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 177 2024-04-07T16:16:14.024 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.024 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_buffer": { 2024-04-07T16:16:14.024 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 816, 2024-04-07T16:16:14.024 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 37 2024-04-07T16:16:14.024 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.024 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_data": { 2024-04-07T16:16:14.024 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:14.024 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:14.024 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.024 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_meta": { 2024-04-07T16:16:14.024 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 86386, 2024-04-07T16:16:14.024 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 14938 2024-04-07T16:16:14.024 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.025 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_onode": { 2024-04-07T16:16:14.025 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 198136, 2024-04-07T16:16:14.025 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 323 2024-04-07T16:16:14.025 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.025 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_other": { 2024-04-07T16:16:14.025 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 9704, 2024-04-07T16:16:14.025 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 574 2024-04-07T16:16:14.025 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.025 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_extent": { 2024-04-07T16:16:14.025 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 8496, 2024-04-07T16:16:14.025 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 177 2024-04-07T16:16:14.025 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.025 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_fsck": { 2024-04-07T16:16:14.025 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:14.026 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:14.026 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.026 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_inline_bl": { 2024-04-07T16:16:14.026 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 28202, 2024-04-07T16:16:14.026 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 191 2024-04-07T16:16:14.026 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.026 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_shared_blob": { 2024-04-07T16:16:14.026 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 19824, 2024-04-07T16:16:14.026 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 177 2024-04-07T16:16:14.026 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.026 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_txc": { 2024-04-07T16:16:14.026 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 736, 2024-04-07T16:16:14.026 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 1 2024-04-07T16:16:14.026 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.027 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_writing": { 2024-04-07T16:16:14.027 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 915802, 2024-04-07T16:16:14.027 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 72 2024-04-07T16:16:14.027 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.027 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_writing_deferred": { 2024-04-07T16:16:14.027 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 12112, 2024-04-07T16:16:14.027 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 3 2024-04-07T16:16:14.027 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.027 INFO:tasks.workunit.client.0.smithi100.stdout: "buffer_anon": { 2024-04-07T16:16:14.027 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 2445590, 2024-04-07T16:16:14.027 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 181 2024-04-07T16:16:14.027 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.027 INFO:tasks.workunit.client.0.smithi100.stdout: "buffer_meta": { 2024-04-07T16:16:14.028 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 4928, 2024-04-07T16:16:14.028 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 56 2024-04-07T16:16:14.028 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.028 INFO:tasks.workunit.client.0.smithi100.stdout: "mds_co": { 2024-04-07T16:16:14.028 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:14.028 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:14.028 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.028 INFO:tasks.workunit.client.0.smithi100.stdout: "osd": { 2024-04-07T16:16:14.028 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 463680, 2024-04-07T16:16:14.028 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 40 2024-04-07T16:16:14.028 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.028 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mapbl": { 2024-04-07T16:16:14.028 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:14.028 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:14.029 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.029 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_pglog": { 2024-04-07T16:16:14.029 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 8944, 2024-04-07T16:16:14.029 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 18 2024-04-07T16:16:14.029 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.029 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap": { 2024-04-07T16:16:14.029 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 225768, 2024-04-07T16:16:14.029 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 3472 2024-04-07T16:16:14.029 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.029 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap_mapping": { 2024-04-07T16:16:14.029 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:14.029 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:14.029 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.029 INFO:tasks.workunit.client.0.smithi100.stdout: "pgmap": { 2024-04-07T16:16:14.030 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:14.030 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:14.030 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.030 INFO:tasks.workunit.client.0.smithi100.stdout: "unittest_1": { 2024-04-07T16:16:14.030 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:14.030 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:14.030 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.030 INFO:tasks.workunit.client.0.smithi100.stdout: "unittest_2": { 2024-04-07T16:16:14.030 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:14.030 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:14.030 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:14.031 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.031 INFO:tasks.workunit.client.0.smithi100.stdout: "osd.5": { 2024-04-07T16:16:14.031 INFO:tasks.workunit.client.0.smithi100.stdout: "bloom_filter": { 2024-04-07T16:16:14.031 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:14.031 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:14.031 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.031 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs": { 2024-04-07T16:16:14.031 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 6368, 2024-04-07T16:16:14.031 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 123 2024-04-07T16:16:14.031 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.031 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs_file_reader": { 2024-04-07T16:16:14.031 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 16896, 2024-04-07T16:16:14.032 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 8 2024-04-07T16:16:14.032 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.032 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs_file_writer": { 2024-04-07T16:16:14.032 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 576, 2024-04-07T16:16:14.032 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 3 2024-04-07T16:16:14.032 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.032 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_alloc": { 2024-04-07T16:16:14.032 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 2720, 2024-04-07T16:16:14.032 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 34 2024-04-07T16:16:14.032 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.032 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_blob": { 2024-04-07T16:16:14.032 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 19712, 2024-04-07T16:16:14.032 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 176 2024-04-07T16:16:14.033 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.033 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_buffer": { 2024-04-07T16:16:14.033 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 480, 2024-04-07T16:16:14.033 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 34 2024-04-07T16:16:14.033 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.033 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_data": { 2024-04-07T16:16:14.033 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:14.033 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:14.033 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.033 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_meta": { 2024-04-07T16:16:14.033 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 45425, 2024-04-07T16:16:14.033 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 8650 2024-04-07T16:16:14.033 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.033 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_onode": { 2024-04-07T16:16:14.034 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 130136, 2024-04-07T16:16:14.034 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 223 2024-04-07T16:16:14.034 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.034 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_other": { 2024-04-07T16:16:14.034 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 9940, 2024-04-07T16:16:14.034 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 576 2024-04-07T16:16:14.034 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.034 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_extent": { 2024-04-07T16:16:14.034 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 8448, 2024-04-07T16:16:14.034 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 176 2024-04-07T16:16:14.034 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.034 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_fsck": { 2024-04-07T16:16:14.034 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:14.034 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:14.034 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.035 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_inline_bl": { 2024-04-07T16:16:14.035 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 28370, 2024-04-07T16:16:14.035 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 191 2024-04-07T16:16:14.035 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.035 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_shared_blob": { 2024-04-07T16:16:14.035 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 19712, 2024-04-07T16:16:14.035 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 176 2024-04-07T16:16:14.035 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.035 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_txc": { 2024-04-07T16:16:14.035 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 2208, 2024-04-07T16:16:14.035 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 3 2024-04-07T16:16:14.035 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.035 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_writing": { 2024-04-07T16:16:14.035 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 832819, 2024-04-07T16:16:14.035 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 52 2024-04-07T16:16:14.036 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.036 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_writing_deferred": { 2024-04-07T16:16:14.036 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 26565, 2024-04-07T16:16:14.036 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 9 2024-04-07T16:16:14.036 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.036 INFO:tasks.workunit.client.0.smithi100.stdout: "buffer_anon": { 2024-04-07T16:16:14.036 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 2415758, 2024-04-07T16:16:14.036 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 171 2024-04-07T16:16:14.036 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.036 INFO:tasks.workunit.client.0.smithi100.stdout: "buffer_meta": { 2024-04-07T16:16:14.036 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 5456, 2024-04-07T16:16:14.036 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 62 2024-04-07T16:16:14.036 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.036 INFO:tasks.workunit.client.0.smithi100.stdout: "mds_co": { 2024-04-07T16:16:14.036 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:14.037 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:14.037 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.037 INFO:tasks.workunit.client.0.smithi100.stdout: "osd": { 2024-04-07T16:16:14.037 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 486864, 2024-04-07T16:16:14.037 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 42 2024-04-07T16:16:14.037 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.037 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mapbl": { 2024-04-07T16:16:14.037 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:14.037 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:14.037 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.037 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_pglog": { 2024-04-07T16:16:14.037 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 11960, 2024-04-07T16:16:14.037 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 23 2024-04-07T16:16:14.037 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.038 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap": { 2024-04-07T16:16:14.038 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 226024, 2024-04-07T16:16:14.038 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 3484 2024-04-07T16:16:14.038 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.038 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap_mapping": { 2024-04-07T16:16:14.038 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:14.038 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:14.038 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.038 INFO:tasks.workunit.client.0.smithi100.stdout: "pgmap": { 2024-04-07T16:16:14.038 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:14.038 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:14.038 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.038 INFO:tasks.workunit.client.0.smithi100.stdout: "unittest_1": { 2024-04-07T16:16:14.038 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:14.039 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:14.039 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.039 INFO:tasks.workunit.client.0.smithi100.stdout: "unittest_2": { 2024-04-07T16:16:14.039 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:14.039 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:14.039 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:14.039 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.039 INFO:tasks.workunit.client.0.smithi100.stdout: "osd.6": { 2024-04-07T16:16:14.039 INFO:tasks.workunit.client.0.smithi100.stdout: "bloom_filter": { 2024-04-07T16:16:14.039 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:14.039 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:14.039 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.039 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs": { 2024-04-07T16:16:14.040 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 6368, 2024-04-07T16:16:14.040 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 123 2024-04-07T16:16:14.040 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.040 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs_file_reader": { 2024-04-07T16:16:14.040 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 16896, 2024-04-07T16:16:14.040 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 8 2024-04-07T16:16:14.040 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.040 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs_file_writer": { 2024-04-07T16:16:14.040 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 576, 2024-04-07T16:16:14.040 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 3 2024-04-07T16:16:14.040 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.040 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_alloc": { 2024-04-07T16:16:14.040 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 2400, 2024-04-07T16:16:14.040 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 30 2024-04-07T16:16:14.040 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.040 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_blob": { 2024-04-07T16:16:14.041 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 20832, 2024-04-07T16:16:14.041 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 186 2024-04-07T16:16:14.041 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.041 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_buffer": { 2024-04-07T16:16:14.041 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 256, 2024-04-07T16:16:14.041 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 32 2024-04-07T16:16:14.041 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.041 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_data": { 2024-04-07T16:16:14.041 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:14.041 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:14.041 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.041 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_meta": { 2024-04-07T16:16:14.041 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 45988, 2024-04-07T16:16:14.041 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 8964 2024-04-07T16:16:14.041 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.042 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_onode": { 2024-04-07T16:16:14.042 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 129456, 2024-04-07T16:16:14.042 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 222 2024-04-07T16:16:14.042 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.042 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_other": { 2024-04-07T16:16:14.042 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 10872, 2024-04-07T16:16:14.042 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 734 2024-04-07T16:16:14.042 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.042 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_extent": { 2024-04-07T16:16:14.042 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 8928, 2024-04-07T16:16:14.042 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 186 2024-04-07T16:16:14.042 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.042 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_fsck": { 2024-04-07T16:16:14.042 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:14.043 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:14.043 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.043 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_inline_bl": { 2024-04-07T16:16:14.043 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 28974, 2024-04-07T16:16:14.043 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 190 2024-04-07T16:16:14.043 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.043 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_shared_blob": { 2024-04-07T16:16:14.043 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 20832, 2024-04-07T16:16:14.043 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 186 2024-04-07T16:16:14.043 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.043 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_txc": { 2024-04-07T16:16:14.043 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:14.043 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:14.043 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.043 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_writing": { 2024-04-07T16:16:14.044 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 804548, 2024-04-07T16:16:14.044 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 45 2024-04-07T16:16:14.044 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.044 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_writing_deferred": { 2024-04-07T16:16:14.044 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:14.044 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:14.044 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.044 INFO:tasks.workunit.client.0.smithi100.stdout: "buffer_anon": { 2024-04-07T16:16:14.044 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 2445979, 2024-04-07T16:16:14.044 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 183 2024-04-07T16:16:14.044 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.044 INFO:tasks.workunit.client.0.smithi100.stdout: "buffer_meta": { 2024-04-07T16:16:14.044 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 4664, 2024-04-07T16:16:14.044 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 53 2024-04-07T16:16:14.044 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.045 INFO:tasks.workunit.client.0.smithi100.stdout: "mds_co": { 2024-04-07T16:16:14.045 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:14.045 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:14.045 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.045 INFO:tasks.workunit.client.0.smithi100.stdout: "osd": { 2024-04-07T16:16:14.045 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 452088, 2024-04-07T16:16:14.045 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 39 2024-04-07T16:16:14.045 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.045 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mapbl": { 2024-04-07T16:16:14.045 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:14.045 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:14.045 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.045 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_pglog": { 2024-04-07T16:16:14.045 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 92144, 2024-04-07T16:16:14.045 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 186 2024-04-07T16:16:14.046 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.046 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap": { 2024-04-07T16:16:14.046 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 226280, 2024-04-07T16:16:14.046 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 3496 2024-04-07T16:16:14.046 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.046 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap_mapping": { 2024-04-07T16:16:14.046 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:14.046 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:14.046 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.046 INFO:tasks.workunit.client.0.smithi100.stdout: "pgmap": { 2024-04-07T16:16:14.046 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:14.046 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:14.046 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.047 INFO:tasks.workunit.client.0.smithi100.stdout: "unittest_1": { 2024-04-07T16:16:14.047 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:14.047 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:14.047 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.047 INFO:tasks.workunit.client.0.smithi100.stdout: "unittest_2": { 2024-04-07T16:16:14.047 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:14.047 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:14.047 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:14.047 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.047 INFO:tasks.workunit.client.0.smithi100.stdout: "osd.7": { 2024-04-07T16:16:14.047 INFO:tasks.workunit.client.0.smithi100.stdout: "bloom_filter": { 2024-04-07T16:16:14.047 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:14.047 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:14.047 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.048 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs": { 2024-04-07T16:16:14.048 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 6368, 2024-04-07T16:16:14.048 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 123 2024-04-07T16:16:14.048 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.048 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs_file_reader": { 2024-04-07T16:16:14.048 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 16896, 2024-04-07T16:16:14.048 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 8 2024-04-07T16:16:14.048 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.048 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs_file_writer": { 2024-04-07T16:16:14.048 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 576, 2024-04-07T16:16:14.048 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 3 2024-04-07T16:16:14.048 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.048 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_alloc": { 2024-04-07T16:16:14.048 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 1760, 2024-04-07T16:16:14.048 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 22 2024-04-07T16:16:14.049 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.049 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_blob": { 2024-04-07T16:16:14.049 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 20720, 2024-04-07T16:16:14.049 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 185 2024-04-07T16:16:14.049 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.049 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_buffer": { 2024-04-07T16:16:14.049 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 1264, 2024-04-07T16:16:14.049 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 41 2024-04-07T16:16:14.049 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.049 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_data": { 2024-04-07T16:16:14.049 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 285055, 2024-04-07T16:16:14.049 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 10 2024-04-07T16:16:14.049 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.049 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_meta": { 2024-04-07T16:16:14.049 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 88710, 2024-04-07T16:16:14.050 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 15500 2024-04-07T16:16:14.050 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.050 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_onode": { 2024-04-07T16:16:14.050 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 202216, 2024-04-07T16:16:14.050 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 329 2024-04-07T16:16:14.050 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.050 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_other": { 2024-04-07T16:16:14.050 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 11636, 2024-04-07T16:16:14.050 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 739 2024-04-07T16:16:14.050 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.050 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_extent": { 2024-04-07T16:16:14.050 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 8880, 2024-04-07T16:16:14.050 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 185 2024-04-07T16:16:14.051 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.051 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_fsck": { 2024-04-07T16:16:14.051 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:14.051 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:14.051 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.051 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_inline_bl": { 2024-04-07T16:16:14.051 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 30934, 2024-04-07T16:16:14.051 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 197 2024-04-07T16:16:14.051 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.051 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_shared_blob": { 2024-04-07T16:16:14.051 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 20720, 2024-04-07T16:16:14.051 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 185 2024-04-07T16:16:14.051 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.051 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_txc": { 2024-04-07T16:16:14.051 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:14.052 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:14.052 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.052 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_writing": { 2024-04-07T16:16:14.052 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 145868, 2024-04-07T16:16:14.052 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 29 2024-04-07T16:16:14.052 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.052 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_writing_deferred": { 2024-04-07T16:16:14.052 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:14.052 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:14.052 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.052 INFO:tasks.workunit.client.0.smithi100.stdout: "buffer_anon": { 2024-04-07T16:16:14.052 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 2431591, 2024-04-07T16:16:14.052 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 220 2024-04-07T16:16:14.052 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.053 INFO:tasks.workunit.client.0.smithi100.stdout: "buffer_meta": { 2024-04-07T16:16:14.053 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 5984, 2024-04-07T16:16:14.053 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 68 2024-04-07T16:16:14.053 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.053 INFO:tasks.workunit.client.0.smithi100.stdout: "mds_co": { 2024-04-07T16:16:14.053 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:14.053 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:14.053 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.053 INFO:tasks.workunit.client.0.smithi100.stdout: "osd": { 2024-04-07T16:16:14.053 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 533232, 2024-04-07T16:16:14.053 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 46 2024-04-07T16:16:14.053 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.053 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mapbl": { 2024-04-07T16:16:14.053 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:14.053 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:14.054 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.054 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_pglog": { 2024-04-07T16:16:14.054 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 93912, 2024-04-07T16:16:14.054 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 215 2024-04-07T16:16:14.054 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.054 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap": { 2024-04-07T16:16:14.054 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 225512, 2024-04-07T16:16:14.054 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 3460 2024-04-07T16:16:14.054 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.054 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap_mapping": { 2024-04-07T16:16:14.054 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:14.054 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:14.054 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.054 INFO:tasks.workunit.client.0.smithi100.stdout: "pgmap": { 2024-04-07T16:16:14.055 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:14.055 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:14.055 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.055 INFO:tasks.workunit.client.0.smithi100.stdout: "unittest_1": { 2024-04-07T16:16:14.055 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:14.055 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:14.055 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.055 INFO:tasks.workunit.client.0.smithi100.stdout: "unittest_2": { 2024-04-07T16:16:14.055 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:14.055 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:14.055 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:14.055 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:14.056 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:14.056 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.056 INFO:tasks.workunit.client.0.smithi100.stdout: "metadata": { 2024-04-07T16:16:14.056 INFO:tasks.workunit.client.0.smithi100.stdout: "mds": { 2024-04-07T16:16:14.056 INFO:tasks.workunit.client.0.smithi100.stdout: "arch": { 2024-04-07T16:16:14.056 INFO:tasks.workunit.client.0.smithi100.stdout: "x86_64": 2 2024-04-07T16:16:14.056 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.057 INFO:tasks.workunit.client.0.smithi100.stdout: "ceph_version": { 2024-04-07T16:16:14.057 INFO:tasks.workunit.client.0.smithi100.stdout: "ceph version 18.2.2-1171-g267c8ff5 (267c8ff5b1ff5815581b3b2381d19360e081b37b) reef (stable)": 2 2024-04-07T16:16:14.057 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.057 INFO:tasks.workunit.client.0.smithi100.stdout: "cpu": { 2024-04-07T16:16:14.057 INFO:tasks.workunit.client.0.smithi100.stdout: "Intel(R) Xeon(R) CPU E5-1620 v4 @ 3.50GHz": 2 2024-04-07T16:16:14.057 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.057 INFO:tasks.workunit.client.0.smithi100.stdout: "distro": { 2024-04-07T16:16:14.058 INFO:tasks.workunit.client.0.smithi100.stdout: "centos": 2 2024-04-07T16:16:14.058 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.058 INFO:tasks.workunit.client.0.smithi100.stdout: "distro_description": { 2024-04-07T16:16:14.058 INFO:tasks.workunit.client.0.smithi100.stdout: "CentOS Stream 8": 2 2024-04-07T16:16:14.058 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.058 INFO:tasks.workunit.client.0.smithi100.stdout: "kernel_description": { 2024-04-07T16:16:14.058 INFO:tasks.workunit.client.0.smithi100.stdout: "#111-Ubuntu SMP Tue Mar 5 20:16:58 UTC 2024": 2 2024-04-07T16:16:14.058 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.059 INFO:tasks.workunit.client.0.smithi100.stdout: "kernel_version": { 2024-04-07T16:16:14.059 INFO:tasks.workunit.client.0.smithi100.stdout: "5.15.0-101-generic": 2 2024-04-07T16:16:14.059 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.059 INFO:tasks.workunit.client.0.smithi100.stdout: "os": { 2024-04-07T16:16:14.059 INFO:tasks.workunit.client.0.smithi100.stdout: "Linux": 2 2024-04-07T16:16:14.059 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:14.059 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.059 INFO:tasks.workunit.client.0.smithi100.stdout: "mon": { 2024-04-07T16:16:14.060 INFO:tasks.workunit.client.0.smithi100.stdout: "arch": { 2024-04-07T16:16:14.060 INFO:tasks.workunit.client.0.smithi100.stdout: "x86_64": 3 2024-04-07T16:16:14.060 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.060 INFO:tasks.workunit.client.0.smithi100.stdout: "ceph_version": { 2024-04-07T16:16:14.060 INFO:tasks.workunit.client.0.smithi100.stdout: "ceph version 18.2.2-1171-g267c8ff5 (267c8ff5b1ff5815581b3b2381d19360e081b37b) reef (stable)": 3 2024-04-07T16:16:14.060 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.060 INFO:tasks.workunit.client.0.smithi100.stdout: "cpu": { 2024-04-07T16:16:14.060 INFO:tasks.workunit.client.0.smithi100.stdout: "Intel(R) Xeon(R) CPU E5-1620 v4 @ 3.50GHz": 3 2024-04-07T16:16:14.060 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.060 INFO:tasks.workunit.client.0.smithi100.stdout: "distro": { 2024-04-07T16:16:14.060 INFO:tasks.workunit.client.0.smithi100.stdout: "centos": 3 2024-04-07T16:16:14.060 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.060 INFO:tasks.workunit.client.0.smithi100.stdout: "distro_description": { 2024-04-07T16:16:14.060 INFO:tasks.workunit.client.0.smithi100.stdout: "CentOS Stream 8": 3 2024-04-07T16:16:14.061 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.061 INFO:tasks.workunit.client.0.smithi100.stdout: "kernel_description": { 2024-04-07T16:16:14.061 INFO:tasks.workunit.client.0.smithi100.stdout: "#111-Ubuntu SMP Tue Mar 5 20:16:58 UTC 2024": 3 2024-04-07T16:16:14.061 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.061 INFO:tasks.workunit.client.0.smithi100.stdout: "kernel_version": { 2024-04-07T16:16:14.061 INFO:tasks.workunit.client.0.smithi100.stdout: "5.15.0-101-generic": 3 2024-04-07T16:16:14.061 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.061 INFO:tasks.workunit.client.0.smithi100.stdout: "os": { 2024-04-07T16:16:14.061 INFO:tasks.workunit.client.0.smithi100.stdout: "Linux": 3 2024-04-07T16:16:14.061 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:14.062 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.062 INFO:tasks.workunit.client.0.smithi100.stdout: "osd": { 2024-04-07T16:16:14.062 INFO:tasks.workunit.client.0.smithi100.stdout: "arch": { 2024-04-07T16:16:14.062 INFO:tasks.workunit.client.0.smithi100.stdout: "x86_64": 8 2024-04-07T16:16:14.062 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.062 INFO:tasks.workunit.client.0.smithi100.stdout: "ceph_version": { 2024-04-07T16:16:14.062 INFO:tasks.workunit.client.0.smithi100.stdout: "ceph version 18.2.2-1171-g267c8ff5 (267c8ff5b1ff5815581b3b2381d19360e081b37b) reef (stable)": 8 2024-04-07T16:16:14.062 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.062 INFO:tasks.workunit.client.0.smithi100.stdout: "cpu": { 2024-04-07T16:16:14.062 INFO:tasks.workunit.client.0.smithi100.stdout: "Intel(R) Xeon(R) CPU E5-1620 v4 @ 3.50GHz": 8 2024-04-07T16:16:14.062 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.062 INFO:tasks.workunit.client.0.smithi100.stdout: "distro": { 2024-04-07T16:16:14.063 INFO:tasks.workunit.client.0.smithi100.stdout: "centos": 8 2024-04-07T16:16:14.063 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.063 INFO:tasks.workunit.client.0.smithi100.stdout: "distro_description": { 2024-04-07T16:16:14.063 INFO:tasks.workunit.client.0.smithi100.stdout: "CentOS Stream 8": 8 2024-04-07T16:16:14.063 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.063 INFO:tasks.workunit.client.0.smithi100.stdout: "kernel_description": { 2024-04-07T16:16:14.063 INFO:tasks.workunit.client.0.smithi100.stdout: "#111-Ubuntu SMP Tue Mar 5 20:16:58 UTC 2024": 8 2024-04-07T16:16:14.063 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.063 INFO:tasks.workunit.client.0.smithi100.stdout: "kernel_version": { 2024-04-07T16:16:14.063 INFO:tasks.workunit.client.0.smithi100.stdout: "5.15.0-101-generic": 8 2024-04-07T16:16:14.063 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.063 INFO:tasks.workunit.client.0.smithi100.stdout: "os": { 2024-04-07T16:16:14.063 INFO:tasks.workunit.client.0.smithi100.stdout: "Linux": 8 2024-04-07T16:16:14.063 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.064 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_objectstore": { 2024-04-07T16:16:14.064 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore": 8 2024-04-07T16:16:14.064 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.064 INFO:tasks.workunit.client.0.smithi100.stdout: "rotational": { 2024-04-07T16:16:14.064 INFO:tasks.workunit.client.0.smithi100.stdout: "0": 8 2024-04-07T16:16:14.064 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:14.064 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:14.064 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.064 INFO:tasks.workunit.client.0.smithi100.stdout: "mon": { 2024-04-07T16:16:14.064 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 3, 2024-04-07T16:16:14.064 INFO:tasks.workunit.client.0.smithi100.stdout: "features": { 2024-04-07T16:16:14.064 INFO:tasks.workunit.client.0.smithi100.stdout: "optional": [], 2024-04-07T16:16:14.064 INFO:tasks.workunit.client.0.smithi100.stdout: "persistent": [ 2024-04-07T16:16:14.064 INFO:tasks.workunit.client.0.smithi100.stdout: "kraken", 2024-04-07T16:16:14.065 INFO:tasks.workunit.client.0.smithi100.stdout: "luminous", 2024-04-07T16:16:14.065 INFO:tasks.workunit.client.0.smithi100.stdout: "mimic", 2024-04-07T16:16:14.065 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap-prune", 2024-04-07T16:16:14.065 INFO:tasks.workunit.client.0.smithi100.stdout: "nautilus", 2024-04-07T16:16:14.065 INFO:tasks.workunit.client.0.smithi100.stdout: "octopus", 2024-04-07T16:16:14.065 INFO:tasks.workunit.client.0.smithi100.stdout: "pacific", 2024-04-07T16:16:14.065 INFO:tasks.workunit.client.0.smithi100.stdout: "elector-pinging", 2024-04-07T16:16:14.065 INFO:tasks.workunit.client.0.smithi100.stdout: "quincy", 2024-04-07T16:16:14.065 INFO:tasks.workunit.client.0.smithi100.stdout: "reef" 2024-04-07T16:16:14.065 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T16:16:14.065 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.065 INFO:tasks.workunit.client.0.smithi100.stdout: "ipv4_addr_mons": 6, 2024-04-07T16:16:14.065 INFO:tasks.workunit.client.0.smithi100.stdout: "ipv6_addr_mons": 0, 2024-04-07T16:16:14.065 INFO:tasks.workunit.client.0.smithi100.stdout: "min_mon_release": 18, 2024-04-07T16:16:14.065 INFO:tasks.workunit.client.0.smithi100.stdout: "v1_addr_mons": 3, 2024-04-07T16:16:14.066 INFO:tasks.workunit.client.0.smithi100.stdout: "v2_addr_mons": 3 2024-04-07T16:16:14.066 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.066 INFO:tasks.workunit.client.0.smithi100.stdout: "organization": "ceph-qa", 2024-04-07T16:16:14.066 INFO:tasks.workunit.client.0.smithi100.stdout: "osd": { 2024-04-07T16:16:14.066 INFO:tasks.workunit.client.0.smithi100.stdout: "cluster_network": false, 2024-04-07T16:16:14.066 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 8, 2024-04-07T16:16:14.066 INFO:tasks.workunit.client.0.smithi100.stdout: "require_min_compat_client": "luminous", 2024-04-07T16:16:14.066 INFO:tasks.workunit.client.0.smithi100.stdout: "require_osd_release": "reef" 2024-04-07T16:16:14.066 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.066 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_perf_histograms": [ 2024-04-07T16:16:14.066 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:14.066 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_latency_out_bytes_histogram": { 2024-04-07T16:16:14.066 INFO:tasks.workunit.client.0.smithi100.stdout: "axes": [ 2024-04-07T16:16:14.066 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:14.067 INFO:tasks.workunit.client.0.smithi100.stdout: "buckets": 32, 2024-04-07T16:16:14.067 INFO:tasks.workunit.client.0.smithi100.stdout: "min": 0, 2024-04-07T16:16:14.067 INFO:tasks.workunit.client.0.smithi100.stdout: "name": "Latency (usec)", 2024-04-07T16:16:14.067 INFO:tasks.workunit.client.0.smithi100.stdout: "quant_size": 100000, 2024-04-07T16:16:14.067 INFO:tasks.workunit.client.0.smithi100.stdout: "ranges": [ 2024-04-07T16:16:14.067 INFO:tasks.workunit.client.0.smithi100.stdout: "[None, -1]", 2024-04-07T16:16:14.067 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 99999]", 2024-04-07T16:16:14.067 INFO:tasks.workunit.client.0.smithi100.stdout: "[100000, 199999]", 2024-04-07T16:16:14.067 INFO:tasks.workunit.client.0.smithi100.stdout: "[200000, 399999]", 2024-04-07T16:16:14.067 INFO:tasks.workunit.client.0.smithi100.stdout: "[400000, 799999]", 2024-04-07T16:16:14.067 INFO:tasks.workunit.client.0.smithi100.stdout: "[800000, 1599999]", 2024-04-07T16:16:14.067 INFO:tasks.workunit.client.0.smithi100.stdout: "[1600000, 3199999]", 2024-04-07T16:16:14.067 INFO:tasks.workunit.client.0.smithi100.stdout: "[3200000, 6399999]", 2024-04-07T16:16:14.067 INFO:tasks.workunit.client.0.smithi100.stdout: "[6400000, 12799999]", 2024-04-07T16:16:14.068 INFO:tasks.workunit.client.0.smithi100.stdout: "[12800000, 25599999]", 2024-04-07T16:16:14.068 INFO:tasks.workunit.client.0.smithi100.stdout: "[25600000, 51199999]", 2024-04-07T16:16:14.068 INFO:tasks.workunit.client.0.smithi100.stdout: "[51200000, 102399999]", 2024-04-07T16:16:14.068 INFO:tasks.workunit.client.0.smithi100.stdout: "[102400000, 204799999]", 2024-04-07T16:16:14.068 INFO:tasks.workunit.client.0.smithi100.stdout: "[204800000, 409599999]", 2024-04-07T16:16:14.068 INFO:tasks.workunit.client.0.smithi100.stdout: "[409600000, 819199999]", 2024-04-07T16:16:14.068 INFO:tasks.workunit.client.0.smithi100.stdout: "[819200000, 1638399999]", 2024-04-07T16:16:14.068 INFO:tasks.workunit.client.0.smithi100.stdout: "[1638400000, 3276799999]", 2024-04-07T16:16:14.068 INFO:tasks.workunit.client.0.smithi100.stdout: "[3276800000, 6553599999]", 2024-04-07T16:16:14.068 INFO:tasks.workunit.client.0.smithi100.stdout: "[6553600000, 13107199999]", 2024-04-07T16:16:14.068 INFO:tasks.workunit.client.0.smithi100.stdout: "[13107200000, 26214399999]", 2024-04-07T16:16:14.068 INFO:tasks.workunit.client.0.smithi100.stdout: "[26214400000, 52428799999]", 2024-04-07T16:16:14.068 INFO:tasks.workunit.client.0.smithi100.stdout: "[52428800000, 104857599999]", 2024-04-07T16:16:14.068 INFO:tasks.workunit.client.0.smithi100.stdout: "[104857600000, 209715199999]", 2024-04-07T16:16:14.068 INFO:tasks.workunit.client.0.smithi100.stdout: "[209715200000, 419430399999]", 2024-04-07T16:16:14.069 INFO:tasks.workunit.client.0.smithi100.stdout: "[419430400000, 838860799999]", 2024-04-07T16:16:14.069 INFO:tasks.workunit.client.0.smithi100.stdout: "[838860800000, 1677721599999]", 2024-04-07T16:16:14.069 INFO:tasks.workunit.client.0.smithi100.stdout: "[1677721600000, 3355443199999]", 2024-04-07T16:16:14.069 INFO:tasks.workunit.client.0.smithi100.stdout: "[3355443200000, 6710886399999]", 2024-04-07T16:16:14.069 INFO:tasks.workunit.client.0.smithi100.stdout: "[6710886400000, 13421772799999]", 2024-04-07T16:16:14.069 INFO:tasks.workunit.client.0.smithi100.stdout: "[13421772800000, 26843545599999]", 2024-04-07T16:16:14.069 INFO:tasks.workunit.client.0.smithi100.stdout: "[26843545600000, 53687091199999]", 2024-04-07T16:16:14.069 INFO:tasks.workunit.client.0.smithi100.stdout: "[53687091200000, None]" 2024-04-07T16:16:14.069 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:14.069 INFO:tasks.workunit.client.0.smithi100.stdout: "scale_type": "log2" 2024-04-07T16:16:14.069 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.069 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:14.069 INFO:tasks.workunit.client.0.smithi100.stdout: "buckets": 32, 2024-04-07T16:16:14.069 INFO:tasks.workunit.client.0.smithi100.stdout: "min": 0, 2024-04-07T16:16:14.069 INFO:tasks.workunit.client.0.smithi100.stdout: "name": "Request size (bytes)", 2024-04-07T16:16:14.069 INFO:tasks.workunit.client.0.smithi100.stdout: "quant_size": 512, 2024-04-07T16:16:14.070 INFO:tasks.workunit.client.0.smithi100.stdout: "ranges": [ 2024-04-07T16:16:14.070 INFO:tasks.workunit.client.0.smithi100.stdout: "[None, -1]", 2024-04-07T16:16:14.070 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 511]", 2024-04-07T16:16:14.070 INFO:tasks.workunit.client.0.smithi100.stdout: "[512, 1023]", 2024-04-07T16:16:14.070 INFO:tasks.workunit.client.0.smithi100.stdout: "[1024, 2047]", 2024-04-07T16:16:14.070 INFO:tasks.workunit.client.0.smithi100.stdout: "[2048, 4095]", 2024-04-07T16:16:14.070 INFO:tasks.workunit.client.0.smithi100.stdout: "[4096, 8191]", 2024-04-07T16:16:14.070 INFO:tasks.workunit.client.0.smithi100.stdout: "[8192, 16383]", 2024-04-07T16:16:14.070 INFO:tasks.workunit.client.0.smithi100.stdout: "[16384, 32767]", 2024-04-07T16:16:14.070 INFO:tasks.workunit.client.0.smithi100.stdout: "[32768, 65535]", 2024-04-07T16:16:14.070 INFO:tasks.workunit.client.0.smithi100.stdout: "[65536, 131071]", 2024-04-07T16:16:14.070 INFO:tasks.workunit.client.0.smithi100.stdout: "[131072, 262143]", 2024-04-07T16:16:14.071 INFO:tasks.workunit.client.0.smithi100.stdout: "[262144, 524287]", 2024-04-07T16:16:14.071 INFO:tasks.workunit.client.0.smithi100.stdout: "[524288, 1048575]", 2024-04-07T16:16:14.071 INFO:tasks.workunit.client.0.smithi100.stdout: "[1048576, 2097151]", 2024-04-07T16:16:14.071 INFO:tasks.workunit.client.0.smithi100.stdout: "[2097152, 4194303]", 2024-04-07T16:16:14.071 INFO:tasks.workunit.client.0.smithi100.stdout: "[4194304, 8388607]", 2024-04-07T16:16:14.071 INFO:tasks.workunit.client.0.smithi100.stdout: "[8388608, 16777215]", 2024-04-07T16:16:14.071 INFO:tasks.workunit.client.0.smithi100.stdout: "[16777216, 33554431]", 2024-04-07T16:16:14.071 INFO:tasks.workunit.client.0.smithi100.stdout: "[33554432, 67108863]", 2024-04-07T16:16:14.071 INFO:tasks.workunit.client.0.smithi100.stdout: "[67108864, 134217727]", 2024-04-07T16:16:14.071 INFO:tasks.workunit.client.0.smithi100.stdout: "[134217728, 268435455]", 2024-04-07T16:16:14.071 INFO:tasks.workunit.client.0.smithi100.stdout: "[268435456, 536870911]", 2024-04-07T16:16:14.071 INFO:tasks.workunit.client.0.smithi100.stdout: "[536870912, 1073741823]", 2024-04-07T16:16:14.071 INFO:tasks.workunit.client.0.smithi100.stdout: "[1073741824, 2147483647]", 2024-04-07T16:16:14.071 INFO:tasks.workunit.client.0.smithi100.stdout: "[2147483648, 4294967295]", 2024-04-07T16:16:14.072 INFO:tasks.workunit.client.0.smithi100.stdout: "[4294967296, 8589934591]", 2024-04-07T16:16:14.072 INFO:tasks.workunit.client.0.smithi100.stdout: "[8589934592, 17179869183]", 2024-04-07T16:16:14.072 INFO:tasks.workunit.client.0.smithi100.stdout: "[17179869184, 34359738367]", 2024-04-07T16:16:14.072 INFO:tasks.workunit.client.0.smithi100.stdout: "[34359738368, 68719476735]", 2024-04-07T16:16:14.072 INFO:tasks.workunit.client.0.smithi100.stdout: "[68719476736, 137438953471]", 2024-04-07T16:16:14.072 INFO:tasks.workunit.client.0.smithi100.stdout: "[137438953472, 274877906943]", 2024-04-07T16:16:14.072 INFO:tasks.workunit.client.0.smithi100.stdout: "[274877906944, None]" 2024-04-07T16:16:14.072 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:14.072 INFO:tasks.workunit.client.0.smithi100.stdout: "scale_type": "log2" 2024-04-07T16:16:14.072 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:14.072 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:14.072 INFO:tasks.workunit.client.0.smithi100.stdout: "osds": [ 2024-04-07T16:16:14.072 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:14.072 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 0, 2024-04-07T16:16:14.072 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T16:16:14.072 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.073 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.073 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.073 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.073 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.073 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.073 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.073 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.073 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.073 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.073 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.073 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.073 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.073 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.073 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.073 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.074 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.074 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.074 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.074 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.074 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.074 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.074 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.074 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.074 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.074 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.074 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.074 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.074 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.074 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.075 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.075 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T16:16:14.075 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T16:16:14.075 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.075 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:14.075 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 1, 2024-04-07T16:16:14.075 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T16:16:14.075 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.075 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.075 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 9, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.075 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 17, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.075 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 6, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.075 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.075 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.075 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.076 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.076 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.076 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.076 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.076 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.076 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.076 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.076 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.076 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.076 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.076 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.076 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.076 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.076 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.076 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.077 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.077 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.077 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.077 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.077 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.077 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.077 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.077 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.077 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T16:16:14.077 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T16:16:14.077 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.077 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:14.077 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 2, 2024-04-07T16:16:14.078 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T16:16:14.078 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.078 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.078 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.078 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.078 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.078 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.078 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.078 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.078 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.078 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.078 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.078 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.079 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.079 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.079 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.079 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.079 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.079 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.079 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.079 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.079 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.081 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.081 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.081 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.081 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.081 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.082 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.082 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.082 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.082 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.082 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.082 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T16:16:14.082 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T16:16:14.082 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.082 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:14.082 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 3, 2024-04-07T16:16:14.082 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T16:16:14.082 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.082 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.083 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.083 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.083 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.083 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.083 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.083 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.083 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.083 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.083 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.083 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.083 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.083 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.083 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.083 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.083 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.084 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.084 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.084 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.084 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.084 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.084 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.084 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.084 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.084 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.084 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.084 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.084 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.085 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.085 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.085 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T16:16:14.085 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T16:16:14.085 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.085 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:14.085 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 4, 2024-04-07T16:16:14.085 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T16:16:14.085 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.085 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.085 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.085 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.086 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.086 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.086 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.086 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.086 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.086 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.086 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.086 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.086 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.086 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.086 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.086 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.086 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.087 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.087 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.087 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.087 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.087 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.087 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.087 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.087 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.087 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.087 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.087 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.087 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.087 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.087 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.088 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T16:16:14.088 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T16:16:14.088 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.088 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:14.088 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 5, 2024-04-07T16:16:14.088 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T16:16:14.088 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.088 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.088 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.088 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.088 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.088 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.088 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.088 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.089 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.089 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.089 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.089 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.089 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.089 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.089 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.089 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.089 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.090 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.090 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.090 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.090 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.090 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.090 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.090 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.090 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.090 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.090 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.090 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.090 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.090 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.090 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.091 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T16:16:14.091 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T16:16:14.091 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.091 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:14.091 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 6, 2024-04-07T16:16:14.091 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T16:16:14.091 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.091 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.091 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.091 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.091 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.091 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.091 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.092 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.092 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.092 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.092 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.092 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.092 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.092 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.092 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.092 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.092 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.092 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.092 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.092 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.092 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.092 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.093 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.093 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.093 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.093 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.093 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.093 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.093 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.093 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.093 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.093 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T16:16:14.093 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T16:16:14.093 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.093 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:14.093 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 7, 2024-04-07T16:16:14.094 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T16:16:14.094 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.094 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.094 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.094 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 43, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.094 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 51, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.094 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.094 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.094 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.094 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.094 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.094 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.094 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.094 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.095 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.095 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.095 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.095 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.095 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.095 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.095 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.095 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.095 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.095 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.095 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.095 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.095 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.095 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.096 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.096 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.096 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.096 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.096 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T16:16:14.096 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T16:16:14.096 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:14.096 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T16:16:14.096 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.096 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_latency_in_bytes_histogram": { 2024-04-07T16:16:14.096 INFO:tasks.workunit.client.0.smithi100.stdout: "axes": [ 2024-04-07T16:16:14.096 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:14.096 INFO:tasks.workunit.client.0.smithi100.stdout: "buckets": 32, 2024-04-07T16:16:14.096 INFO:tasks.workunit.client.0.smithi100.stdout: "min": 0, 2024-04-07T16:16:14.096 INFO:tasks.workunit.client.0.smithi100.stdout: "name": "Latency (usec)", 2024-04-07T16:16:14.096 INFO:tasks.workunit.client.0.smithi100.stdout: "quant_size": 100000, 2024-04-07T16:16:14.097 INFO:tasks.workunit.client.0.smithi100.stdout: "ranges": [ 2024-04-07T16:16:14.097 INFO:tasks.workunit.client.0.smithi100.stdout: "[None, -1]", 2024-04-07T16:16:14.097 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 99999]", 2024-04-07T16:16:14.097 INFO:tasks.workunit.client.0.smithi100.stdout: "[100000, 199999]", 2024-04-07T16:16:14.097 INFO:tasks.workunit.client.0.smithi100.stdout: "[200000, 399999]", 2024-04-07T16:16:14.097 INFO:tasks.workunit.client.0.smithi100.stdout: "[400000, 799999]", 2024-04-07T16:16:14.097 INFO:tasks.workunit.client.0.smithi100.stdout: "[800000, 1599999]", 2024-04-07T16:16:14.097 INFO:tasks.workunit.client.0.smithi100.stdout: "[1600000, 3199999]", 2024-04-07T16:16:14.097 INFO:tasks.workunit.client.0.smithi100.stdout: "[3200000, 6399999]", 2024-04-07T16:16:14.097 INFO:tasks.workunit.client.0.smithi100.stdout: "[6400000, 12799999]", 2024-04-07T16:16:14.097 INFO:tasks.workunit.client.0.smithi100.stdout: "[12800000, 25599999]", 2024-04-07T16:16:14.097 INFO:tasks.workunit.client.0.smithi100.stdout: "[25600000, 51199999]", 2024-04-07T16:16:14.097 INFO:tasks.workunit.client.0.smithi100.stdout: "[51200000, 102399999]", 2024-04-07T16:16:14.097 INFO:tasks.workunit.client.0.smithi100.stdout: "[102400000, 204799999]", 2024-04-07T16:16:14.097 INFO:tasks.workunit.client.0.smithi100.stdout: "[204800000, 409599999]", 2024-04-07T16:16:14.098 INFO:tasks.workunit.client.0.smithi100.stdout: "[409600000, 819199999]", 2024-04-07T16:16:14.098 INFO:tasks.workunit.client.0.smithi100.stdout: "[819200000, 1638399999]", 2024-04-07T16:16:14.098 INFO:tasks.workunit.client.0.smithi100.stdout: "[1638400000, 3276799999]", 2024-04-07T16:16:14.098 INFO:tasks.workunit.client.0.smithi100.stdout: "[3276800000, 6553599999]", 2024-04-07T16:16:14.098 INFO:tasks.workunit.client.0.smithi100.stdout: "[6553600000, 13107199999]", 2024-04-07T16:16:14.098 INFO:tasks.workunit.client.0.smithi100.stdout: "[13107200000, 26214399999]", 2024-04-07T16:16:14.098 INFO:tasks.workunit.client.0.smithi100.stdout: "[26214400000, 52428799999]", 2024-04-07T16:16:14.098 INFO:tasks.workunit.client.0.smithi100.stdout: "[52428800000, 104857599999]", 2024-04-07T16:16:14.098 INFO:tasks.workunit.client.0.smithi100.stdout: "[104857600000, 209715199999]", 2024-04-07T16:16:14.098 INFO:tasks.workunit.client.0.smithi100.stdout: "[209715200000, 419430399999]", 2024-04-07T16:16:14.098 INFO:tasks.workunit.client.0.smithi100.stdout: "[419430400000, 838860799999]", 2024-04-07T16:16:14.098 INFO:tasks.workunit.client.0.smithi100.stdout: "[838860800000, 1677721599999]", 2024-04-07T16:16:14.098 INFO:tasks.workunit.client.0.smithi100.stdout: "[1677721600000, 3355443199999]", 2024-04-07T16:16:14.098 INFO:tasks.workunit.client.0.smithi100.stdout: "[3355443200000, 6710886399999]", 2024-04-07T16:16:14.098 INFO:tasks.workunit.client.0.smithi100.stdout: "[6710886400000, 13421772799999]", 2024-04-07T16:16:14.099 INFO:tasks.workunit.client.0.smithi100.stdout: "[13421772800000, 26843545599999]", 2024-04-07T16:16:14.099 INFO:tasks.workunit.client.0.smithi100.stdout: "[26843545600000, 53687091199999]", 2024-04-07T16:16:14.099 INFO:tasks.workunit.client.0.smithi100.stdout: "[53687091200000, None]" 2024-04-07T16:16:14.099 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:14.099 INFO:tasks.workunit.client.0.smithi100.stdout: "scale_type": "log2" 2024-04-07T16:16:14.099 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.099 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:14.100 INFO:tasks.workunit.client.0.smithi100.stdout: "buckets": 32, 2024-04-07T16:16:14.100 INFO:tasks.workunit.client.0.smithi100.stdout: "min": 0, 2024-04-07T16:16:14.100 INFO:tasks.workunit.client.0.smithi100.stdout: "name": "Request size (bytes)", 2024-04-07T16:16:14.100 INFO:tasks.workunit.client.0.smithi100.stdout: "quant_size": 512, 2024-04-07T16:16:14.100 INFO:tasks.workunit.client.0.smithi100.stdout: "ranges": [ 2024-04-07T16:16:14.100 INFO:tasks.workunit.client.0.smithi100.stdout: "[None, -1]", 2024-04-07T16:16:14.100 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 511]", 2024-04-07T16:16:14.101 INFO:tasks.workunit.client.0.smithi100.stdout: "[512, 1023]", 2024-04-07T16:16:14.101 INFO:tasks.workunit.client.0.smithi100.stdout: "[1024, 2047]", 2024-04-07T16:16:14.101 INFO:tasks.workunit.client.0.smithi100.stdout: "[2048, 4095]", 2024-04-07T16:16:14.101 INFO:tasks.workunit.client.0.smithi100.stdout: "[4096, 8191]", 2024-04-07T16:16:14.101 INFO:tasks.workunit.client.0.smithi100.stdout: "[8192, 16383]", 2024-04-07T16:16:14.101 INFO:tasks.workunit.client.0.smithi100.stdout: "[16384, 32767]", 2024-04-07T16:16:14.101 INFO:tasks.workunit.client.0.smithi100.stdout: "[32768, 65535]", 2024-04-07T16:16:14.101 INFO:tasks.workunit.client.0.smithi100.stdout: "[65536, 131071]", 2024-04-07T16:16:14.101 INFO:tasks.workunit.client.0.smithi100.stdout: "[131072, 262143]", 2024-04-07T16:16:14.101 INFO:tasks.workunit.client.0.smithi100.stdout: "[262144, 524287]", 2024-04-07T16:16:14.101 INFO:tasks.workunit.client.0.smithi100.stdout: "[524288, 1048575]", 2024-04-07T16:16:14.101 INFO:tasks.workunit.client.0.smithi100.stdout: "[1048576, 2097151]", 2024-04-07T16:16:14.101 INFO:tasks.workunit.client.0.smithi100.stdout: "[2097152, 4194303]", 2024-04-07T16:16:14.101 INFO:tasks.workunit.client.0.smithi100.stdout: "[4194304, 8388607]", 2024-04-07T16:16:14.102 INFO:tasks.workunit.client.0.smithi100.stdout: "[8388608, 16777215]", 2024-04-07T16:16:14.102 INFO:tasks.workunit.client.0.smithi100.stdout: "[16777216, 33554431]", 2024-04-07T16:16:14.102 INFO:tasks.workunit.client.0.smithi100.stdout: "[33554432, 67108863]", 2024-04-07T16:16:14.102 INFO:tasks.workunit.client.0.smithi100.stdout: "[67108864, 134217727]", 2024-04-07T16:16:14.102 INFO:tasks.workunit.client.0.smithi100.stdout: "[134217728, 268435455]", 2024-04-07T16:16:14.102 INFO:tasks.workunit.client.0.smithi100.stdout: "[268435456, 536870911]", 2024-04-07T16:16:14.102 INFO:tasks.workunit.client.0.smithi100.stdout: "[536870912, 1073741823]", 2024-04-07T16:16:14.102 INFO:tasks.workunit.client.0.smithi100.stdout: "[1073741824, 2147483647]", 2024-04-07T16:16:14.102 INFO:tasks.workunit.client.0.smithi100.stdout: "[2147483648, 4294967295]", 2024-04-07T16:16:14.102 INFO:tasks.workunit.client.0.smithi100.stdout: "[4294967296, 8589934591]", 2024-04-07T16:16:14.102 INFO:tasks.workunit.client.0.smithi100.stdout: "[8589934592, 17179869183]", 2024-04-07T16:16:14.102 INFO:tasks.workunit.client.0.smithi100.stdout: "[17179869184, 34359738367]", 2024-04-07T16:16:14.102 INFO:tasks.workunit.client.0.smithi100.stdout: "[34359738368, 68719476735]", 2024-04-07T16:16:14.102 INFO:tasks.workunit.client.0.smithi100.stdout: "[68719476736, 137438953471]", 2024-04-07T16:16:14.103 INFO:tasks.workunit.client.0.smithi100.stdout: "[137438953472, 274877906943]", 2024-04-07T16:16:14.103 INFO:tasks.workunit.client.0.smithi100.stdout: "[274877906944, None]" 2024-04-07T16:16:14.103 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:14.103 INFO:tasks.workunit.client.0.smithi100.stdout: "scale_type": "log2" 2024-04-07T16:16:14.103 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:14.103 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:14.103 INFO:tasks.workunit.client.0.smithi100.stdout: "osds": [ 2024-04-07T16:16:14.103 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:14.103 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 0, 2024-04-07T16:16:14.103 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T16:16:14.103 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.103 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.103 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.103 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.104 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.104 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.104 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.104 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.104 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.104 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.104 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.104 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.104 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.104 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.104 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.104 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.104 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.104 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.105 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.105 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.105 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.105 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.105 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.105 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.105 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.105 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.105 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.105 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.105 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.105 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.105 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.105 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T16:16:14.105 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T16:16:14.106 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.106 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:14.106 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 1, 2024-04-07T16:16:14.106 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T16:16:14.106 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.106 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.106 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.106 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.106 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.106 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 11, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.106 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 10, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.106 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.106 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.106 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.107 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.107 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.107 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.107 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.107 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.107 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.107 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.107 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.107 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.107 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.107 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.107 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.107 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.107 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.108 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.108 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.108 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.108 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.108 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.108 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.108 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.108 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T16:16:14.108 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T16:16:14.108 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.108 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:14.108 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 2, 2024-04-07T16:16:14.108 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T16:16:14.108 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.108 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.109 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.109 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.109 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.109 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.109 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.109 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.109 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.109 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.109 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.109 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.109 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.109 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.109 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.109 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.110 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.110 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.110 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.110 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.110 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.110 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.110 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.110 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.110 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.110 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.110 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.110 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.110 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.111 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.111 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.111 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T16:16:14.111 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T16:16:14.111 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.111 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:14.111 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 3, 2024-04-07T16:16:14.111 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T16:16:14.111 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.111 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.111 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.111 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.111 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.111 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.112 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.112 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.112 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.112 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.112 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.112 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.112 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.112 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.112 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.112 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.112 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.112 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.112 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.112 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.113 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.113 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.113 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.113 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.113 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.113 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.113 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.113 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.113 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.113 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.113 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.113 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T16:16:14.113 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T16:16:14.113 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.113 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:14.114 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 4, 2024-04-07T16:16:14.114 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T16:16:14.114 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.114 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.114 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.114 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.114 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.114 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.114 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.114 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.114 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.114 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.115 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.115 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.115 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.115 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.115 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.115 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.115 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.115 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.115 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.115 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.115 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.115 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.115 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.115 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.115 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.116 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.116 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.116 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.116 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.116 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.116 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.116 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T16:16:14.116 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T16:16:14.116 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.116 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:14.116 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 5, 2024-04-07T16:16:14.116 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T16:16:14.116 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.116 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.116 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.117 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.117 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.117 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.117 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.117 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.117 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.117 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.117 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.117 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.117 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.117 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.117 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.117 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.117 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.117 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.118 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.118 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.118 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.118 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.118 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.118 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.118 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.118 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.118 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.118 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.118 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.118 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.118 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.119 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T16:16:14.119 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T16:16:14.119 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.119 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:14.119 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 6, 2024-04-07T16:16:14.119 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T16:16:14.119 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.119 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.119 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.119 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.119 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.119 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.119 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.119 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.120 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.120 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.120 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.120 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.120 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.120 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.120 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.120 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.120 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.120 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.120 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.120 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.120 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.120 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.121 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.121 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.121 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.121 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.121 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.121 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.121 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.121 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.121 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.121 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T16:16:14.121 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T16:16:14.121 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.121 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:14.121 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 7, 2024-04-07T16:16:14.121 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T16:16:14.122 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.122 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.122 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.122 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.122 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.122 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.122 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 36, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.122 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.122 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.122 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.122 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.122 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.122 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.123 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.123 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.123 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.123 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.123 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.123 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.123 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.123 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.123 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.123 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.123 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.123 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.123 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.123 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.124 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.124 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.124 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.124 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.124 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T16:16:14.124 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T16:16:14.124 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:14.124 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T16:16:14.124 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.124 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_latency_out_bytes_histogram": { 2024-04-07T16:16:14.124 INFO:tasks.workunit.client.0.smithi100.stdout: "axes": [ 2024-04-07T16:16:14.124 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:14.124 INFO:tasks.workunit.client.0.smithi100.stdout: "buckets": 32, 2024-04-07T16:16:14.124 INFO:tasks.workunit.client.0.smithi100.stdout: "min": 0, 2024-04-07T16:16:14.124 INFO:tasks.workunit.client.0.smithi100.stdout: "name": "Latency (usec)", 2024-04-07T16:16:14.125 INFO:tasks.workunit.client.0.smithi100.stdout: "quant_size": 100000, 2024-04-07T16:16:14.125 INFO:tasks.workunit.client.0.smithi100.stdout: "ranges": [ 2024-04-07T16:16:14.125 INFO:tasks.workunit.client.0.smithi100.stdout: "[None, -1]", 2024-04-07T16:16:14.125 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 99999]", 2024-04-07T16:16:14.125 INFO:tasks.workunit.client.0.smithi100.stdout: "[100000, 199999]", 2024-04-07T16:16:14.125 INFO:tasks.workunit.client.0.smithi100.stdout: "[200000, 399999]", 2024-04-07T16:16:14.125 INFO:tasks.workunit.client.0.smithi100.stdout: "[400000, 799999]", 2024-04-07T16:16:14.125 INFO:tasks.workunit.client.0.smithi100.stdout: "[800000, 1599999]", 2024-04-07T16:16:14.125 INFO:tasks.workunit.client.0.smithi100.stdout: "[1600000, 3199999]", 2024-04-07T16:16:14.125 INFO:tasks.workunit.client.0.smithi100.stdout: "[3200000, 6399999]", 2024-04-07T16:16:14.125 INFO:tasks.workunit.client.0.smithi100.stdout: "[6400000, 12799999]", 2024-04-07T16:16:14.125 INFO:tasks.workunit.client.0.smithi100.stdout: "[12800000, 25599999]", 2024-04-07T16:16:14.126 INFO:tasks.workunit.client.0.smithi100.stdout: "[25600000, 51199999]", 2024-04-07T16:16:14.126 INFO:tasks.workunit.client.0.smithi100.stdout: "[51200000, 102399999]", 2024-04-07T16:16:14.126 INFO:tasks.workunit.client.0.smithi100.stdout: "[102400000, 204799999]", 2024-04-07T16:16:14.126 INFO:tasks.workunit.client.0.smithi100.stdout: "[204800000, 409599999]", 2024-04-07T16:16:14.126 INFO:tasks.workunit.client.0.smithi100.stdout: "[409600000, 819199999]", 2024-04-07T16:16:14.126 INFO:tasks.workunit.client.0.smithi100.stdout: "[819200000, 1638399999]", 2024-04-07T16:16:14.126 INFO:tasks.workunit.client.0.smithi100.stdout: "[1638400000, 3276799999]", 2024-04-07T16:16:14.126 INFO:tasks.workunit.client.0.smithi100.stdout: "[3276800000, 6553599999]", 2024-04-07T16:16:14.126 INFO:tasks.workunit.client.0.smithi100.stdout: "[6553600000, 13107199999]", 2024-04-07T16:16:14.126 INFO:tasks.workunit.client.0.smithi100.stdout: "[13107200000, 26214399999]", 2024-04-07T16:16:14.126 INFO:tasks.workunit.client.0.smithi100.stdout: "[26214400000, 52428799999]", 2024-04-07T16:16:14.126 INFO:tasks.workunit.client.0.smithi100.stdout: "[52428800000, 104857599999]", 2024-04-07T16:16:14.126 INFO:tasks.workunit.client.0.smithi100.stdout: "[104857600000, 209715199999]", 2024-04-07T16:16:14.127 INFO:tasks.workunit.client.0.smithi100.stdout: "[209715200000, 419430399999]", 2024-04-07T16:16:14.127 INFO:tasks.workunit.client.0.smithi100.stdout: "[419430400000, 838860799999]", 2024-04-07T16:16:14.127 INFO:tasks.workunit.client.0.smithi100.stdout: "[838860800000, 1677721599999]", 2024-04-07T16:16:14.127 INFO:tasks.workunit.client.0.smithi100.stdout: "[1677721600000, 3355443199999]", 2024-04-07T16:16:14.127 INFO:tasks.workunit.client.0.smithi100.stdout: "[3355443200000, 6710886399999]", 2024-04-07T16:16:14.127 INFO:tasks.workunit.client.0.smithi100.stdout: "[6710886400000, 13421772799999]", 2024-04-07T16:16:14.127 INFO:tasks.workunit.client.0.smithi100.stdout: "[13421772800000, 26843545599999]", 2024-04-07T16:16:14.127 INFO:tasks.workunit.client.0.smithi100.stdout: "[26843545600000, 53687091199999]", 2024-04-07T16:16:14.127 INFO:tasks.workunit.client.0.smithi100.stdout: "[53687091200000, None]" 2024-04-07T16:16:14.127 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:14.127 INFO:tasks.workunit.client.0.smithi100.stdout: "scale_type": "log2" 2024-04-07T16:16:14.127 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.127 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:14.127 INFO:tasks.workunit.client.0.smithi100.stdout: "buckets": 32, 2024-04-07T16:16:14.128 INFO:tasks.workunit.client.0.smithi100.stdout: "min": 0, 2024-04-07T16:16:14.128 INFO:tasks.workunit.client.0.smithi100.stdout: "name": "Request size (bytes)", 2024-04-07T16:16:14.128 INFO:tasks.workunit.client.0.smithi100.stdout: "quant_size": 512, 2024-04-07T16:16:14.128 INFO:tasks.workunit.client.0.smithi100.stdout: "ranges": [ 2024-04-07T16:16:14.128 INFO:tasks.workunit.client.0.smithi100.stdout: "[None, -1]", 2024-04-07T16:16:14.128 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 511]", 2024-04-07T16:16:14.128 INFO:tasks.workunit.client.0.smithi100.stdout: "[512, 1023]", 2024-04-07T16:16:14.128 INFO:tasks.workunit.client.0.smithi100.stdout: "[1024, 2047]", 2024-04-07T16:16:14.128 INFO:tasks.workunit.client.0.smithi100.stdout: "[2048, 4095]", 2024-04-07T16:16:14.128 INFO:tasks.workunit.client.0.smithi100.stdout: "[4096, 8191]", 2024-04-07T16:16:14.128 INFO:tasks.workunit.client.0.smithi100.stdout: "[8192, 16383]", 2024-04-07T16:16:14.128 INFO:tasks.workunit.client.0.smithi100.stdout: "[16384, 32767]", 2024-04-07T16:16:14.128 INFO:tasks.workunit.client.0.smithi100.stdout: "[32768, 65535]", 2024-04-07T16:16:14.128 INFO:tasks.workunit.client.0.smithi100.stdout: "[65536, 131071]", 2024-04-07T16:16:14.129 INFO:tasks.workunit.client.0.smithi100.stdout: "[131072, 262143]", 2024-04-07T16:16:14.129 INFO:tasks.workunit.client.0.smithi100.stdout: "[262144, 524287]", 2024-04-07T16:16:14.129 INFO:tasks.workunit.client.0.smithi100.stdout: "[524288, 1048575]", 2024-04-07T16:16:14.129 INFO:tasks.workunit.client.0.smithi100.stdout: "[1048576, 2097151]", 2024-04-07T16:16:14.129 INFO:tasks.workunit.client.0.smithi100.stdout: "[2097152, 4194303]", 2024-04-07T16:16:14.129 INFO:tasks.workunit.client.0.smithi100.stdout: "[4194304, 8388607]", 2024-04-07T16:16:14.129 INFO:tasks.workunit.client.0.smithi100.stdout: "[8388608, 16777215]", 2024-04-07T16:16:14.129 INFO:tasks.workunit.client.0.smithi100.stdout: "[16777216, 33554431]", 2024-04-07T16:16:14.129 INFO:tasks.workunit.client.0.smithi100.stdout: "[33554432, 67108863]", 2024-04-07T16:16:14.129 INFO:tasks.workunit.client.0.smithi100.stdout: "[67108864, 134217727]", 2024-04-07T16:16:14.129 INFO:tasks.workunit.client.0.smithi100.stdout: "[134217728, 268435455]", 2024-04-07T16:16:14.129 INFO:tasks.workunit.client.0.smithi100.stdout: "[268435456, 536870911]", 2024-04-07T16:16:14.129 INFO:tasks.workunit.client.0.smithi100.stdout: "[536870912, 1073741823]", 2024-04-07T16:16:14.129 INFO:tasks.workunit.client.0.smithi100.stdout: "[1073741824, 2147483647]", 2024-04-07T16:16:14.130 INFO:tasks.workunit.client.0.smithi100.stdout: "[2147483648, 4294967295]", 2024-04-07T16:16:14.130 INFO:tasks.workunit.client.0.smithi100.stdout: "[4294967296, 8589934591]", 2024-04-07T16:16:14.130 INFO:tasks.workunit.client.0.smithi100.stdout: "[8589934592, 17179869183]", 2024-04-07T16:16:14.130 INFO:tasks.workunit.client.0.smithi100.stdout: "[17179869184, 34359738367]", 2024-04-07T16:16:14.130 INFO:tasks.workunit.client.0.smithi100.stdout: "[34359738368, 68719476735]", 2024-04-07T16:16:14.130 INFO:tasks.workunit.client.0.smithi100.stdout: "[68719476736, 137438953471]", 2024-04-07T16:16:14.130 INFO:tasks.workunit.client.0.smithi100.stdout: "[137438953472, 274877906943]", 2024-04-07T16:16:14.130 INFO:tasks.workunit.client.0.smithi100.stdout: "[274877906944, None]" 2024-04-07T16:16:14.130 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:14.130 INFO:tasks.workunit.client.0.smithi100.stdout: "scale_type": "log2" 2024-04-07T16:16:14.130 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:14.130 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:14.131 INFO:tasks.workunit.client.0.smithi100.stdout: "osds": [ 2024-04-07T16:16:14.131 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:14.131 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 0, 2024-04-07T16:16:14.131 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T16:16:14.131 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.131 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.131 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.131 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.131 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.131 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.131 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.132 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.132 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.132 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.132 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.132 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.132 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.132 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.132 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.132 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.132 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.132 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.132 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.132 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.132 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.133 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.133 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.133 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.133 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.133 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.133 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.133 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.133 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.133 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.133 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.133 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T16:16:14.133 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T16:16:14.133 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.133 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:14.134 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 1, 2024-04-07T16:16:14.134 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T16:16:14.134 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.134 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.134 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.134 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.134 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.134 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 11, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.134 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 10, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.134 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.134 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.134 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.135 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.135 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.135 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.135 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.135 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.135 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.135 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.135 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.135 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.135 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.135 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.135 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.135 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.135 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.136 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.136 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.136 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.136 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.136 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.136 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.136 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.136 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T16:16:14.136 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T16:16:14.136 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.136 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:14.136 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 2, 2024-04-07T16:16:14.136 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T16:16:14.136 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.137 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.137 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.137 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.137 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.137 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.137 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.137 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.137 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.137 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.137 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.137 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.138 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.138 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.138 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.138 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.138 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.138 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.138 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.138 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.138 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.138 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.138 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.138 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.138 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.139 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.139 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.139 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.139 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.139 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.139 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.139 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T16:16:14.139 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T16:16:14.139 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.139 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:14.139 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 3, 2024-04-07T16:16:14.139 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T16:16:14.139 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.139 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.140 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.140 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.140 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.140 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.140 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.140 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.140 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.140 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.140 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.140 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.140 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.140 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.140 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.140 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.141 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.141 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.141 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.141 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.141 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.141 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.141 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.141 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.141 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.141 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.141 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.141 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.141 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.141 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.141 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.142 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T16:16:14.142 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T16:16:14.142 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.142 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:14.142 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 4, 2024-04-07T16:16:14.142 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T16:16:14.142 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.143 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.143 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.143 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.143 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.143 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.143 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.143 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.143 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.144 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.144 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.144 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.144 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.144 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.144 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.144 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.144 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.144 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.144 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.144 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.144 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.144 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.144 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.145 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.145 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.145 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.145 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.145 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.145 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.145 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.145 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.145 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T16:16:14.145 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T16:16:14.145 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.145 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:14.145 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 5, 2024-04-07T16:16:14.146 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T16:16:14.146 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.146 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.146 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.146 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.146 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.146 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.146 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.146 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.146 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.146 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.146 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.146 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.146 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.147 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.147 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.147 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.147 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.147 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.147 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.147 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.147 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.147 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.147 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.147 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.147 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.147 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.148 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.148 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.148 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.148 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.148 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.148 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T16:16:14.148 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T16:16:14.148 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.148 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:14.148 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 6, 2024-04-07T16:16:14.148 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T16:16:14.148 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.148 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.148 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.149 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.149 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.149 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.149 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.149 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.149 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.149 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.149 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.149 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.149 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.149 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.149 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.149 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.149 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.150 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.150 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.150 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.150 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.150 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.150 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.150 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.150 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.150 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.150 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.150 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.150 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.150 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.151 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.151 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T16:16:14.151 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T16:16:14.151 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.151 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:14.151 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 7, 2024-04-07T16:16:14.151 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T16:16:14.151 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.151 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.151 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.151 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.151 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.151 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.151 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 36, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.152 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.152 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.152 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.152 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.152 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.152 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.152 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.153 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.153 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.153 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.153 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.153 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.153 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.154 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.154 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.154 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.154 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.154 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.154 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.154 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.154 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.154 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.155 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.155 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.155 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T16:16:14.155 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T16:16:14.155 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:14.155 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T16:16:14.155 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.155 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_latency_in_bytes_histogram": { 2024-04-07T16:16:14.155 INFO:tasks.workunit.client.0.smithi100.stdout: "axes": [ 2024-04-07T16:16:14.155 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:14.155 INFO:tasks.workunit.client.0.smithi100.stdout: "buckets": 32, 2024-04-07T16:16:14.156 INFO:tasks.workunit.client.0.smithi100.stdout: "min": 0, 2024-04-07T16:16:14.156 INFO:tasks.workunit.client.0.smithi100.stdout: "name": "Latency (usec)", 2024-04-07T16:16:14.156 INFO:tasks.workunit.client.0.smithi100.stdout: "quant_size": 100000, 2024-04-07T16:16:14.156 INFO:tasks.workunit.client.0.smithi100.stdout: "ranges": [ 2024-04-07T16:16:14.156 INFO:tasks.workunit.client.0.smithi100.stdout: "[None, -1]", 2024-04-07T16:16:14.156 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 99999]", 2024-04-07T16:16:14.156 INFO:tasks.workunit.client.0.smithi100.stdout: "[100000, 199999]", 2024-04-07T16:16:14.156 INFO:tasks.workunit.client.0.smithi100.stdout: "[200000, 399999]", 2024-04-07T16:16:14.156 INFO:tasks.workunit.client.0.smithi100.stdout: "[400000, 799999]", 2024-04-07T16:16:14.156 INFO:tasks.workunit.client.0.smithi100.stdout: "[800000, 1599999]", 2024-04-07T16:16:14.156 INFO:tasks.workunit.client.0.smithi100.stdout: "[1600000, 3199999]", 2024-04-07T16:16:14.156 INFO:tasks.workunit.client.0.smithi100.stdout: "[3200000, 6399999]", 2024-04-07T16:16:14.156 INFO:tasks.workunit.client.0.smithi100.stdout: "[6400000, 12799999]", 2024-04-07T16:16:14.156 INFO:tasks.workunit.client.0.smithi100.stdout: "[12800000, 25599999]", 2024-04-07T16:16:14.157 INFO:tasks.workunit.client.0.smithi100.stdout: "[25600000, 51199999]", 2024-04-07T16:16:14.157 INFO:tasks.workunit.client.0.smithi100.stdout: "[51200000, 102399999]", 2024-04-07T16:16:14.157 INFO:tasks.workunit.client.0.smithi100.stdout: "[102400000, 204799999]", 2024-04-07T16:16:14.157 INFO:tasks.workunit.client.0.smithi100.stdout: "[204800000, 409599999]", 2024-04-07T16:16:14.157 INFO:tasks.workunit.client.0.smithi100.stdout: "[409600000, 819199999]", 2024-04-07T16:16:14.157 INFO:tasks.workunit.client.0.smithi100.stdout: "[819200000, 1638399999]", 2024-04-07T16:16:14.157 INFO:tasks.workunit.client.0.smithi100.stdout: "[1638400000, 3276799999]", 2024-04-07T16:16:14.157 INFO:tasks.workunit.client.0.smithi100.stdout: "[3276800000, 6553599999]", 2024-04-07T16:16:14.157 INFO:tasks.workunit.client.0.smithi100.stdout: "[6553600000, 13107199999]", 2024-04-07T16:16:14.157 INFO:tasks.workunit.client.0.smithi100.stdout: "[13107200000, 26214399999]", 2024-04-07T16:16:14.157 INFO:tasks.workunit.client.0.smithi100.stdout: "[26214400000, 52428799999]", 2024-04-07T16:16:14.157 INFO:tasks.workunit.client.0.smithi100.stdout: "[52428800000, 104857599999]", 2024-04-07T16:16:14.157 INFO:tasks.workunit.client.0.smithi100.stdout: "[104857600000, 209715199999]", 2024-04-07T16:16:14.157 INFO:tasks.workunit.client.0.smithi100.stdout: "[209715200000, 419430399999]", 2024-04-07T16:16:14.157 INFO:tasks.workunit.client.0.smithi100.stdout: "[419430400000, 838860799999]", 2024-04-07T16:16:14.158 INFO:tasks.workunit.client.0.smithi100.stdout: "[838860800000, 1677721599999]", 2024-04-07T16:16:14.158 INFO:tasks.workunit.client.0.smithi100.stdout: "[1677721600000, 3355443199999]", 2024-04-07T16:16:14.158 INFO:tasks.workunit.client.0.smithi100.stdout: "[3355443200000, 6710886399999]", 2024-04-07T16:16:14.158 INFO:tasks.workunit.client.0.smithi100.stdout: "[6710886400000, 13421772799999]", 2024-04-07T16:16:14.158 INFO:tasks.workunit.client.0.smithi100.stdout: "[13421772800000, 26843545599999]", 2024-04-07T16:16:14.158 INFO:tasks.workunit.client.0.smithi100.stdout: "[26843545600000, 53687091199999]", 2024-04-07T16:16:14.158 INFO:tasks.workunit.client.0.smithi100.stdout: "[53687091200000, None]" 2024-04-07T16:16:14.158 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:14.158 INFO:tasks.workunit.client.0.smithi100.stdout: "scale_type": "log2" 2024-04-07T16:16:14.158 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.158 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:14.158 INFO:tasks.workunit.client.0.smithi100.stdout: "buckets": 32, 2024-04-07T16:16:14.158 INFO:tasks.workunit.client.0.smithi100.stdout: "min": 0, 2024-04-07T16:16:14.159 INFO:tasks.workunit.client.0.smithi100.stdout: "name": "Request size (bytes)", 2024-04-07T16:16:14.159 INFO:tasks.workunit.client.0.smithi100.stdout: "quant_size": 512, 2024-04-07T16:16:14.159 INFO:tasks.workunit.client.0.smithi100.stdout: "ranges": [ 2024-04-07T16:16:14.159 INFO:tasks.workunit.client.0.smithi100.stdout: "[None, -1]", 2024-04-07T16:16:14.159 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 511]", 2024-04-07T16:16:14.159 INFO:tasks.workunit.client.0.smithi100.stdout: "[512, 1023]", 2024-04-07T16:16:14.159 INFO:tasks.workunit.client.0.smithi100.stdout: "[1024, 2047]", 2024-04-07T16:16:14.159 INFO:tasks.workunit.client.0.smithi100.stdout: "[2048, 4095]", 2024-04-07T16:16:14.159 INFO:tasks.workunit.client.0.smithi100.stdout: "[4096, 8191]", 2024-04-07T16:16:14.159 INFO:tasks.workunit.client.0.smithi100.stdout: "[8192, 16383]", 2024-04-07T16:16:14.159 INFO:tasks.workunit.client.0.smithi100.stdout: "[16384, 32767]", 2024-04-07T16:16:14.159 INFO:tasks.workunit.client.0.smithi100.stdout: "[32768, 65535]", 2024-04-07T16:16:14.159 INFO:tasks.workunit.client.0.smithi100.stdout: "[65536, 131071]", 2024-04-07T16:16:14.159 INFO:tasks.workunit.client.0.smithi100.stdout: "[131072, 262143]", 2024-04-07T16:16:14.159 INFO:tasks.workunit.client.0.smithi100.stdout: "[262144, 524287]", 2024-04-07T16:16:14.159 INFO:tasks.workunit.client.0.smithi100.stdout: "[524288, 1048575]", 2024-04-07T16:16:14.160 INFO:tasks.workunit.client.0.smithi100.stdout: "[1048576, 2097151]", 2024-04-07T16:16:14.160 INFO:tasks.workunit.client.0.smithi100.stdout: "[2097152, 4194303]", 2024-04-07T16:16:14.160 INFO:tasks.workunit.client.0.smithi100.stdout: "[4194304, 8388607]", 2024-04-07T16:16:14.160 INFO:tasks.workunit.client.0.smithi100.stdout: "[8388608, 16777215]", 2024-04-07T16:16:14.160 INFO:tasks.workunit.client.0.smithi100.stdout: "[16777216, 33554431]", 2024-04-07T16:16:14.160 INFO:tasks.workunit.client.0.smithi100.stdout: "[33554432, 67108863]", 2024-04-07T16:16:14.160 INFO:tasks.workunit.client.0.smithi100.stdout: "[67108864, 134217727]", 2024-04-07T16:16:14.160 INFO:tasks.workunit.client.0.smithi100.stdout: "[134217728, 268435455]", 2024-04-07T16:16:14.160 INFO:tasks.workunit.client.0.smithi100.stdout: "[268435456, 536870911]", 2024-04-07T16:16:14.160 INFO:tasks.workunit.client.0.smithi100.stdout: "[536870912, 1073741823]", 2024-04-07T16:16:14.160 INFO:tasks.workunit.client.0.smithi100.stdout: "[1073741824, 2147483647]", 2024-04-07T16:16:14.160 INFO:tasks.workunit.client.0.smithi100.stdout: "[2147483648, 4294967295]", 2024-04-07T16:16:14.160 INFO:tasks.workunit.client.0.smithi100.stdout: "[4294967296, 8589934591]", 2024-04-07T16:16:14.161 INFO:tasks.workunit.client.0.smithi100.stdout: "[8589934592, 17179869183]", 2024-04-07T16:16:14.161 INFO:tasks.workunit.client.0.smithi100.stdout: "[17179869184, 34359738367]", 2024-04-07T16:16:14.161 INFO:tasks.workunit.client.0.smithi100.stdout: "[34359738368, 68719476735]", 2024-04-07T16:16:14.161 INFO:tasks.workunit.client.0.smithi100.stdout: "[68719476736, 137438953471]", 2024-04-07T16:16:14.161 INFO:tasks.workunit.client.0.smithi100.stdout: "[137438953472, 274877906943]", 2024-04-07T16:16:14.161 INFO:tasks.workunit.client.0.smithi100.stdout: "[274877906944, None]" 2024-04-07T16:16:14.161 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:14.161 INFO:tasks.workunit.client.0.smithi100.stdout: "scale_type": "log2" 2024-04-07T16:16:14.161 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:14.161 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:14.161 INFO:tasks.workunit.client.0.smithi100.stdout: "osds": [ 2024-04-07T16:16:14.161 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:14.161 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 0, 2024-04-07T16:16:14.161 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T16:16:14.162 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.162 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.162 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.162 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.162 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.162 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.162 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.162 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.162 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.162 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 4, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.162 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.163 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.163 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.163 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.163 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.163 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.163 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.163 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.163 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.163 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.163 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.163 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.163 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.163 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.164 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.164 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.164 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.164 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.164 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.164 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.164 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.164 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T16:16:14.164 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T16:16:14.164 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.164 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:14.164 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 1, 2024-04-07T16:16:14.164 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T16:16:14.164 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.164 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.165 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.165 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.165 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.165 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 6, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.165 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 9, 3, 0, 0, 0, 0, 0, 0, 17, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.165 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 24, 5, 0, 0, 0, 0, 0, 0, 15, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.165 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 14, 0, 0, 0, 0, 0, 0, 0, 5, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.165 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.165 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.165 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.165 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.165 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.165 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.165 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.166 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.166 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.166 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.166 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.166 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.166 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.166 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.166 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.166 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.166 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.166 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.166 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.166 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.167 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.167 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.167 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T16:16:14.167 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T16:16:14.167 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.167 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:14.167 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 2, 2024-04-07T16:16:14.167 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T16:16:14.167 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.167 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.167 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.167 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.167 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.168 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.168 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.168 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.168 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 5, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.168 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.168 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 5, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.168 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.168 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.168 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.168 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.168 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.168 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.168 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.168 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.169 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.169 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.169 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.169 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.169 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.169 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.169 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.169 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.169 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.169 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.169 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.169 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.169 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T16:16:14.169 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T16:16:14.170 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.170 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:14.170 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 3, 2024-04-07T16:16:14.170 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T16:16:14.170 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.170 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.170 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.170 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.170 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.170 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.170 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.170 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.170 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.171 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.171 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.171 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.171 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.171 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.171 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.171 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.171 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.171 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.171 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.171 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.171 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.171 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.172 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.172 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.172 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.172 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.172 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.172 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.172 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.172 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.172 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.172 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T16:16:14.172 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T16:16:14.172 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.172 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:14.172 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 4, 2024-04-07T16:16:14.173 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T16:16:14.173 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.173 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.173 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.173 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.173 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.173 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.173 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.173 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.173 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.173 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.173 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.173 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.173 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.174 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.174 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.174 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.174 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.174 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.174 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.174 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.174 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.174 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.174 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.174 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.174 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.174 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.175 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.175 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.175 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.175 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.175 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.175 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T16:16:14.175 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T16:16:14.175 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.175 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:14.175 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 5, 2024-04-07T16:16:14.175 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T16:16:14.175 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.175 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.176 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.176 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.176 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.176 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.176 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.176 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.176 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.176 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.176 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.176 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.176 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.176 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.176 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.176 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.176 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.176 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.177 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.177 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.177 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.177 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.177 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.177 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.177 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.177 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.177 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.177 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.178 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.178 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.178 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.178 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T16:16:14.178 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T16:16:14.178 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.178 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:14.179 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 6, 2024-04-07T16:16:14.179 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T16:16:14.179 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.179 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.179 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.179 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.179 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.180 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.180 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.180 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 6, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.180 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.180 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 2, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.180 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.180 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.180 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.180 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.180 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.181 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.181 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.181 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.181 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.181 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.181 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.181 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.181 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.181 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.181 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.181 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.181 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.182 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.182 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.182 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.182 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.182 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T16:16:14.182 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T16:16:14.182 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.182 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:14.182 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 7, 2024-04-07T16:16:14.182 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T16:16:14.182 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.183 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.183 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.183 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.183 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.183 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.183 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.184 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.184 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.184 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.184 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.184 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.184 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.184 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.185 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.185 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.185 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.185 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.185 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.185 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.185 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.185 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.186 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.186 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.186 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.186 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.186 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.186 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.186 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.186 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.187 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:14.187 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T16:16:14.187 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T16:16:14.187 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:14.187 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T16:16:14.187 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:14.187 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:14.188 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:14.188 INFO:tasks.workunit.client.0.smithi100.stdout: "perf_counters": { 2024-04-07T16:16:14.188 INFO:tasks.workunit.client.0.smithi100.stdout: "mds.a52fc0d8acb515431879057649ff294cfe14c620": { 2024-04-07T16:16:14.188 INFO:tasks.workunit.client.0.smithi100.stdout: "mds": { 2024-04-07T16:16:14.188 INFO:tasks.workunit.client.0.smithi100.stdout: "caps": { 2024-04-07T16:16:14.188 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.188 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.189 INFO:tasks.workunit.client.0.smithi100.stdout: "ceph_cap_op_flush_ack": { 2024-04-07T16:16:14.189 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.189 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.189 INFO:tasks.workunit.client.0.smithi100.stdout: "ceph_cap_op_flushsnap_ack": { 2024-04-07T16:16:14.189 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.189 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.190 INFO:tasks.workunit.client.0.smithi100.stdout: "ceph_cap_op_grant": { 2024-04-07T16:16:14.190 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.190 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.190 INFO:tasks.workunit.client.0.smithi100.stdout: "ceph_cap_op_revoke": { 2024-04-07T16:16:14.190 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.190 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.191 INFO:tasks.workunit.client.0.smithi100.stdout: "ceph_cap_op_trunc": { 2024-04-07T16:16:14.191 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.191 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.191 INFO:tasks.workunit.client.0.smithi100.stdout: "dir_commit": { 2024-04-07T16:16:14.191 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 12 2024-04-07T16:16:14.191 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.191 INFO:tasks.workunit.client.0.smithi100.stdout: "dir_fetch_complete": { 2024-04-07T16:16:14.191 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.191 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.191 INFO:tasks.workunit.client.0.smithi100.stdout: "dir_fetch_keys": { 2024-04-07T16:16:14.191 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.191 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.191 INFO:tasks.workunit.client.0.smithi100.stdout: "dir_merge": { 2024-04-07T16:16:14.192 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.192 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.192 INFO:tasks.workunit.client.0.smithi100.stdout: "dir_split": { 2024-04-07T16:16:14.192 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.192 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.192 INFO:tasks.workunit.client.0.smithi100.stdout: "exported_inodes": { 2024-04-07T16:16:14.192 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.192 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.192 INFO:tasks.workunit.client.0.smithi100.stdout: "forward": { 2024-04-07T16:16:14.192 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.192 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.192 INFO:tasks.workunit.client.0.smithi100.stdout: "handle_client_cap_release": { 2024-04-07T16:16:14.193 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.193 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.193 INFO:tasks.workunit.client.0.smithi100.stdout: "handle_client_caps": { 2024-04-07T16:16:14.193 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.193 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.193 INFO:tasks.workunit.client.0.smithi100.stdout: "handle_client_caps_dirty": { 2024-04-07T16:16:14.193 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.193 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.193 INFO:tasks.workunit.client.0.smithi100.stdout: "handle_inode_file_caps": { 2024-04-07T16:16:14.193 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.193 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.193 INFO:tasks.workunit.client.0.smithi100.stdout: "imported_inodes": { 2024-04-07T16:16:14.193 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.193 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.194 INFO:tasks.workunit.client.0.smithi100.stdout: "inodes": { 2024-04-07T16:16:14.194 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 10 2024-04-07T16:16:14.194 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.194 INFO:tasks.workunit.client.0.smithi100.stdout: "inodes_expired": { 2024-04-07T16:16:14.194 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.194 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.194 INFO:tasks.workunit.client.0.smithi100.stdout: "inodes_pinned": { 2024-04-07T16:16:14.194 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 10 2024-04-07T16:16:14.194 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.194 INFO:tasks.workunit.client.0.smithi100.stdout: "inodes_with_caps": { 2024-04-07T16:16:14.194 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.194 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.194 INFO:tasks.workunit.client.0.smithi100.stdout: "load_cent": { 2024-04-07T16:16:14.195 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.195 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.195 INFO:tasks.workunit.client.0.smithi100.stdout: "openino_dir_fetch": { 2024-04-07T16:16:14.195 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.195 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.195 INFO:tasks.workunit.client.0.smithi100.stdout: "process_request_cap_release": { 2024-04-07T16:16:14.195 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.195 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.195 INFO:tasks.workunit.client.0.smithi100.stdout: "reply_latency": { 2024-04-07T16:16:14.195 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 2, 2024-04-07T16:16:14.195 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 131939 2024-04-07T16:16:14.195 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.195 INFO:tasks.workunit.client.0.smithi100.stdout: "request": { 2024-04-07T16:16:14.196 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2 2024-04-07T16:16:14.196 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.196 INFO:tasks.workunit.client.0.smithi100.stdout: "root_rbytes": { 2024-04-07T16:16:14.196 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.196 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.196 INFO:tasks.workunit.client.0.smithi100.stdout: "root_rfiles": { 2024-04-07T16:16:14.196 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.196 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.196 INFO:tasks.workunit.client.0.smithi100.stdout: "root_rsnaps": { 2024-04-07T16:16:14.196 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.196 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.196 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_reply": { 2024-04-07T16:16:14.196 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.196 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.197 INFO:tasks.workunit.client.0.smithi100.stdout: "subtrees": { 2024-04-07T16:16:14.197 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2 2024-04-07T16:16:14.197 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:14.197 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.197 INFO:tasks.workunit.client.0.smithi100.stdout: "mds_cache": { 2024-04-07T16:16:14.197 INFO:tasks.workunit.client.0.smithi100.stdout: "ireq_enqueue_scrub": { 2024-04-07T16:16:14.197 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.197 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.197 INFO:tasks.workunit.client.0.smithi100.stdout: "ireq_exportdir": { 2024-04-07T16:16:14.197 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.197 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.197 INFO:tasks.workunit.client.0.smithi100.stdout: "ireq_flush": { 2024-04-07T16:16:14.197 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.197 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.198 INFO:tasks.workunit.client.0.smithi100.stdout: "ireq_fragmentdir": { 2024-04-07T16:16:14.198 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.198 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.198 INFO:tasks.workunit.client.0.smithi100.stdout: "ireq_fragstats": { 2024-04-07T16:16:14.198 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.198 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.198 INFO:tasks.workunit.client.0.smithi100.stdout: "ireq_inodestats": { 2024-04-07T16:16:14.198 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.198 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.198 INFO:tasks.workunit.client.0.smithi100.stdout: "num_recovering_enqueued": { 2024-04-07T16:16:14.198 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.198 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.198 INFO:tasks.workunit.client.0.smithi100.stdout: "num_recovering_prioritized": { 2024-04-07T16:16:14.199 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.199 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.199 INFO:tasks.workunit.client.0.smithi100.stdout: "num_recovering_processing": { 2024-04-07T16:16:14.199 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.199 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.199 INFO:tasks.workunit.client.0.smithi100.stdout: "num_strays": { 2024-04-07T16:16:14.199 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.199 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.199 INFO:tasks.workunit.client.0.smithi100.stdout: "num_strays_delayed": { 2024-04-07T16:16:14.199 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.199 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.199 INFO:tasks.workunit.client.0.smithi100.stdout: "num_strays_enqueuing": { 2024-04-07T16:16:14.199 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.199 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.200 INFO:tasks.workunit.client.0.smithi100.stdout: "recovery_completed": { 2024-04-07T16:16:14.200 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.200 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.200 INFO:tasks.workunit.client.0.smithi100.stdout: "recovery_started": { 2024-04-07T16:16:14.200 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.200 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.200 INFO:tasks.workunit.client.0.smithi100.stdout: "strays_created": { 2024-04-07T16:16:14.200 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.200 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.200 INFO:tasks.workunit.client.0.smithi100.stdout: "strays_enqueued": { 2024-04-07T16:16:14.200 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.200 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.200 INFO:tasks.workunit.client.0.smithi100.stdout: "strays_migrated": { 2024-04-07T16:16:14.200 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.200 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.201 INFO:tasks.workunit.client.0.smithi100.stdout: "strays_reintegrated": { 2024-04-07T16:16:14.201 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.201 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:14.201 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.201 INFO:tasks.workunit.client.0.smithi100.stdout: "mds_log": { 2024-04-07T16:16:14.201 INFO:tasks.workunit.client.0.smithi100.stdout: "ev": { 2024-04-07T16:16:14.201 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5 2024-04-07T16:16:14.202 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.202 INFO:tasks.workunit.client.0.smithi100.stdout: "evadd": { 2024-04-07T16:16:14.202 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5 2024-04-07T16:16:14.202 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.202 INFO:tasks.workunit.client.0.smithi100.stdout: "evex": { 2024-04-07T16:16:14.202 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.202 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.203 INFO:tasks.workunit.client.0.smithi100.stdout: "evexd": { 2024-04-07T16:16:14.203 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.203 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.203 INFO:tasks.workunit.client.0.smithi100.stdout: "evexg": { 2024-04-07T16:16:14.203 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.203 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.203 INFO:tasks.workunit.client.0.smithi100.stdout: "evtrm": { 2024-04-07T16:16:14.204 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.204 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.204 INFO:tasks.workunit.client.0.smithi100.stdout: "jlat": { 2024-04-07T16:16:14.204 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 5, 2024-04-07T16:16:14.204 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 20108532 2024-04-07T16:16:14.204 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.204 INFO:tasks.workunit.client.0.smithi100.stdout: "replayed": { 2024-04-07T16:16:14.204 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.204 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.204 INFO:tasks.workunit.client.0.smithi100.stdout: "seg": { 2024-04-07T16:16:14.204 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1 2024-04-07T16:16:14.204 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.205 INFO:tasks.workunit.client.0.smithi100.stdout: "segadd": { 2024-04-07T16:16:14.205 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1 2024-04-07T16:16:14.205 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.205 INFO:tasks.workunit.client.0.smithi100.stdout: "segex": { 2024-04-07T16:16:14.205 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.205 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.205 INFO:tasks.workunit.client.0.smithi100.stdout: "segexd": { 2024-04-07T16:16:14.205 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.205 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.205 INFO:tasks.workunit.client.0.smithi100.stdout: "segexg": { 2024-04-07T16:16:14.205 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.205 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.205 INFO:tasks.workunit.client.0.smithi100.stdout: "segtrm": { 2024-04-07T16:16:14.206 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.206 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:14.206 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.206 INFO:tasks.workunit.client.0.smithi100.stdout: "mds_mem": { 2024-04-07T16:16:14.206 INFO:tasks.workunit.client.0.smithi100.stdout: "cap": { 2024-04-07T16:16:14.206 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.206 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.206 INFO:tasks.workunit.client.0.smithi100.stdout: "cap+": { 2024-04-07T16:16:14.206 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2 2024-04-07T16:16:14.206 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.206 INFO:tasks.workunit.client.0.smithi100.stdout: "cap-": { 2024-04-07T16:16:14.206 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2 2024-04-07T16:16:14.206 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.207 INFO:tasks.workunit.client.0.smithi100.stdout: "dir": { 2024-04-07T16:16:14.207 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 12 2024-04-07T16:16:14.207 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.207 INFO:tasks.workunit.client.0.smithi100.stdout: "dir+": { 2024-04-07T16:16:14.207 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 12 2024-04-07T16:16:14.207 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.207 INFO:tasks.workunit.client.0.smithi100.stdout: "dir-": { 2024-04-07T16:16:14.207 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.207 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.207 INFO:tasks.workunit.client.0.smithi100.stdout: "dn": { 2024-04-07T16:16:14.207 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 10 2024-04-07T16:16:14.207 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.208 INFO:tasks.workunit.client.0.smithi100.stdout: "dn+": { 2024-04-07T16:16:14.208 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 10 2024-04-07T16:16:14.208 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.208 INFO:tasks.workunit.client.0.smithi100.stdout: "dn-": { 2024-04-07T16:16:14.208 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.208 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.208 INFO:tasks.workunit.client.0.smithi100.stdout: "heap": { 2024-04-07T16:16:14.208 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 207156 2024-04-07T16:16:14.208 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.208 INFO:tasks.workunit.client.0.smithi100.stdout: "ino": { 2024-04-07T16:16:14.208 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 13 2024-04-07T16:16:14.208 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.208 INFO:tasks.workunit.client.0.smithi100.stdout: "ino+": { 2024-04-07T16:16:14.209 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 13 2024-04-07T16:16:14.209 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.209 INFO:tasks.workunit.client.0.smithi100.stdout: "ino-": { 2024-04-07T16:16:14.209 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.209 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.209 INFO:tasks.workunit.client.0.smithi100.stdout: "rss": { 2024-04-07T16:16:14.209 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 35308 2024-04-07T16:16:14.209 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:14.209 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.209 INFO:tasks.workunit.client.0.smithi100.stdout: "mds_server": { 2024-04-07T16:16:14.209 INFO:tasks.workunit.client.0.smithi100.stdout: "cap_acquisition_throttle": { 2024-04-07T16:16:14.209 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.209 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.210 INFO:tasks.workunit.client.0.smithi100.stdout: "cap_revoke_eviction": { 2024-04-07T16:16:14.210 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.210 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.210 INFO:tasks.workunit.client.0.smithi100.stdout: "handle_client_request": { 2024-04-07T16:16:14.210 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2 2024-04-07T16:16:14.210 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.210 INFO:tasks.workunit.client.0.smithi100.stdout: "handle_client_session": { 2024-04-07T16:16:14.210 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 12 2024-04-07T16:16:14.210 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.210 INFO:tasks.workunit.client.0.smithi100.stdout: "handle_peer_request": { 2024-04-07T16:16:14.210 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.210 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.211 INFO:tasks.workunit.client.0.smithi100.stdout: "req_create_latency": { 2024-04-07T16:16:14.211 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.211 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.211 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.211 INFO:tasks.workunit.client.0.smithi100.stdout: "req_getattr_latency": { 2024-04-07T16:16:14.211 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 2, 2024-04-07T16:16:14.211 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 131939 2024-04-07T16:16:14.211 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.211 INFO:tasks.workunit.client.0.smithi100.stdout: "req_getfilelock_latency": { 2024-04-07T16:16:14.211 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.211 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.211 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.211 INFO:tasks.workunit.client.0.smithi100.stdout: "req_getvxattr_latency": { 2024-04-07T16:16:14.211 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.212 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.212 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.212 INFO:tasks.workunit.client.0.smithi100.stdout: "req_link_latency": { 2024-04-07T16:16:14.212 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.212 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.212 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.212 INFO:tasks.workunit.client.0.smithi100.stdout: "req_lookup_latency": { 2024-04-07T16:16:14.212 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.212 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.212 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.212 INFO:tasks.workunit.client.0.smithi100.stdout: "req_lookuphash_latency": { 2024-04-07T16:16:14.212 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.213 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.213 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.213 INFO:tasks.workunit.client.0.smithi100.stdout: "req_lookupino_latency": { 2024-04-07T16:16:14.213 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.213 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.213 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.213 INFO:tasks.workunit.client.0.smithi100.stdout: "req_lookupname_latency": { 2024-04-07T16:16:14.213 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.213 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.213 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.213 INFO:tasks.workunit.client.0.smithi100.stdout: "req_lookupparent_latency": { 2024-04-07T16:16:14.213 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.213 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.213 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.214 INFO:tasks.workunit.client.0.smithi100.stdout: "req_lookupsnap_latency": { 2024-04-07T16:16:14.214 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.214 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.214 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.214 INFO:tasks.workunit.client.0.smithi100.stdout: "req_lssnap_latency": { 2024-04-07T16:16:14.214 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.214 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.214 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.214 INFO:tasks.workunit.client.0.smithi100.stdout: "req_mkdir_latency": { 2024-04-07T16:16:14.214 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.214 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.214 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.215 INFO:tasks.workunit.client.0.smithi100.stdout: "req_mknod_latency": { 2024-04-07T16:16:14.215 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.215 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.215 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.215 INFO:tasks.workunit.client.0.smithi100.stdout: "req_mksnap_latency": { 2024-04-07T16:16:14.215 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.215 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.215 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.215 INFO:tasks.workunit.client.0.smithi100.stdout: "req_open_latency": { 2024-04-07T16:16:14.215 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.215 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.215 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.215 INFO:tasks.workunit.client.0.smithi100.stdout: "req_readdir_latency": { 2024-04-07T16:16:14.215 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.216 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.216 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.216 INFO:tasks.workunit.client.0.smithi100.stdout: "req_rename_latency": { 2024-04-07T16:16:14.216 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.216 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.216 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.216 INFO:tasks.workunit.client.0.smithi100.stdout: "req_renamesnap_latency": { 2024-04-07T16:16:14.216 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.216 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.216 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.216 INFO:tasks.workunit.client.0.smithi100.stdout: "req_rmdir_latency": { 2024-04-07T16:16:14.216 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.216 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.216 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.217 INFO:tasks.workunit.client.0.smithi100.stdout: "req_rmsnap_latency": { 2024-04-07T16:16:14.217 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.217 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.217 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.217 INFO:tasks.workunit.client.0.smithi100.stdout: "req_rmxattr_latency": { 2024-04-07T16:16:14.217 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.217 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.217 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.217 INFO:tasks.workunit.client.0.smithi100.stdout: "req_setattr_latency": { 2024-04-07T16:16:14.217 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.217 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.217 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.217 INFO:tasks.workunit.client.0.smithi100.stdout: "req_setdirlayout_latency": { 2024-04-07T16:16:14.217 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.218 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.218 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.218 INFO:tasks.workunit.client.0.smithi100.stdout: "req_setfilelock_latency": { 2024-04-07T16:16:14.218 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.218 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.218 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.218 INFO:tasks.workunit.client.0.smithi100.stdout: "req_setlayout_latency": { 2024-04-07T16:16:14.218 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.218 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.218 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.218 INFO:tasks.workunit.client.0.smithi100.stdout: "req_setxattr_latency": { 2024-04-07T16:16:14.218 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.218 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.219 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.219 INFO:tasks.workunit.client.0.smithi100.stdout: "req_snapdiff_latency": { 2024-04-07T16:16:14.219 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.219 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.219 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.219 INFO:tasks.workunit.client.0.smithi100.stdout: "req_symlink_latency": { 2024-04-07T16:16:14.219 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.219 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.219 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.219 INFO:tasks.workunit.client.0.smithi100.stdout: "req_unlink_latency": { 2024-04-07T16:16:14.219 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.219 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.219 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:14.219 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.220 INFO:tasks.workunit.client.0.smithi100.stdout: "mds_sessions": { 2024-04-07T16:16:14.220 INFO:tasks.workunit.client.0.smithi100.stdout: "average_load": { 2024-04-07T16:16:14.220 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1 2024-04-07T16:16:14.220 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.220 INFO:tasks.workunit.client.0.smithi100.stdout: "avg_session_uptime": { 2024-04-07T16:16:14.220 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 59 2024-04-07T16:16:14.220 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.220 INFO:tasks.workunit.client.0.smithi100.stdout: "mdthresh_evicted": { 2024-04-07T16:16:14.220 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.220 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.220 INFO:tasks.workunit.client.0.smithi100.stdout: "session_add": { 2024-04-07T16:16:14.220 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2 2024-04-07T16:16:14.220 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.220 INFO:tasks.workunit.client.0.smithi100.stdout: "session_count": { 2024-04-07T16:16:14.221 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.221 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.221 INFO:tasks.workunit.client.0.smithi100.stdout: "session_remove": { 2024-04-07T16:16:14.221 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2 2024-04-07T16:16:14.221 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.221 INFO:tasks.workunit.client.0.smithi100.stdout: "sessions_open": { 2024-04-07T16:16:14.221 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.221 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.221 INFO:tasks.workunit.client.0.smithi100.stdout: "sessions_stale": { 2024-04-07T16:16:14.221 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.221 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.221 INFO:tasks.workunit.client.0.smithi100.stdout: "total_load": { 2024-04-07T16:16:14.221 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2 2024-04-07T16:16:14.221 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:14.221 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.222 INFO:tasks.workunit.client.0.smithi100.stdout: "objecter": { 2024-04-07T16:16:14.222 INFO:tasks.workunit.client.0.smithi100.stdout: "op_active": { 2024-04-07T16:16:14.222 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.222 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.222 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r": { 2024-04-07T16:16:14.222 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.222 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.222 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rmw": { 2024-04-07T16:16:14.222 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.222 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.222 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w": { 2024-04-07T16:16:14.222 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 45 2024-04-07T16:16:14.222 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:14.222 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.223 INFO:tasks.workunit.client.0.smithi100.stdout: "purge_queue": { 2024-04-07T16:16:14.223 INFO:tasks.workunit.client.0.smithi100.stdout: "pq_executed": { 2024-04-07T16:16:14.223 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.223 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.223 INFO:tasks.workunit.client.0.smithi100.stdout: "pq_executing": { 2024-04-07T16:16:14.223 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.223 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.223 INFO:tasks.workunit.client.0.smithi100.stdout: "pq_executing_high_water": { 2024-04-07T16:16:14.223 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.223 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.223 INFO:tasks.workunit.client.0.smithi100.stdout: "pq_executing_ops": { 2024-04-07T16:16:14.223 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.223 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.223 INFO:tasks.workunit.client.0.smithi100.stdout: "pq_executing_ops_high_water": { 2024-04-07T16:16:14.223 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.224 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.224 INFO:tasks.workunit.client.0.smithi100.stdout: "pq_item_in_journal": { 2024-04-07T16:16:14.224 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.224 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:14.224 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:14.224 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.224 INFO:tasks.workunit.client.0.smithi100.stdout: "mon.8dd0baa036f0a7e6d1f3bdce86136fe527836c3d": { 2024-04-07T16:16:14.224 INFO:tasks.workunit.client.0.smithi100.stdout: "mon": { 2024-04-07T16:16:14.224 INFO:tasks.workunit.client.0.smithi100.stdout: "election_call": { 2024-04-07T16:16:14.224 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1 2024-04-07T16:16:14.224 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.224 INFO:tasks.workunit.client.0.smithi100.stdout: "election_lose": { 2024-04-07T16:16:14.224 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1 2024-04-07T16:16:14.224 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.224 INFO:tasks.workunit.client.0.smithi100.stdout: "election_win": { 2024-04-07T16:16:14.224 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.225 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.225 INFO:tasks.workunit.client.0.smithi100.stdout: "num_elections": { 2024-04-07T16:16:14.225 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3 2024-04-07T16:16:14.225 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.225 INFO:tasks.workunit.client.0.smithi100.stdout: "num_sessions": { 2024-04-07T16:16:14.225 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 11 2024-04-07T16:16:14.225 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.225 INFO:tasks.workunit.client.0.smithi100.stdout: "session_add": { 2024-04-07T16:16:14.225 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5 2024-04-07T16:16:14.225 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.225 INFO:tasks.workunit.client.0.smithi100.stdout: "session_rm": { 2024-04-07T16:16:14.225 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 107 2024-04-07T16:16:14.225 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.225 INFO:tasks.workunit.client.0.smithi100.stdout: "session_trim": { 2024-04-07T16:16:14.226 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.226 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:14.226 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.226 INFO:tasks.workunit.client.0.smithi100.stdout: "paxos": { 2024-04-07T16:16:14.226 INFO:tasks.workunit.client.0.smithi100.stdout: "accept_timeout": { 2024-04-07T16:16:14.226 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.226 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.226 INFO:tasks.workunit.client.0.smithi100.stdout: "begin": { 2024-04-07T16:16:14.226 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1084 2024-04-07T16:16:14.226 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.226 INFO:tasks.workunit.client.0.smithi100.stdout: "begin_bytes": { 2024-04-07T16:16:14.226 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1084, 2024-04-07T16:16:14.226 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4403831 2024-04-07T16:16:14.226 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.227 INFO:tasks.workunit.client.0.smithi100.stdout: "begin_keys": { 2024-04-07T16:16:14.227 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.227 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.227 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.227 INFO:tasks.workunit.client.0.smithi100.stdout: "begin_latency": { 2024-04-07T16:16:14.227 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1084, 2024-04-07T16:16:14.227 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 423993548 2024-04-07T16:16:14.227 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.227 INFO:tasks.workunit.client.0.smithi100.stdout: "collect": { 2024-04-07T16:16:14.227 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1 2024-04-07T16:16:14.227 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.227 INFO:tasks.workunit.client.0.smithi100.stdout: "collect_bytes": { 2024-04-07T16:16:14.227 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1, 2024-04-07T16:16:14.228 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 73 2024-04-07T16:16:14.228 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.228 INFO:tasks.workunit.client.0.smithi100.stdout: "collect_keys": { 2024-04-07T16:16:14.228 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1, 2024-04-07T16:16:14.228 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1 2024-04-07T16:16:14.228 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.229 INFO:tasks.workunit.client.0.smithi100.stdout: "collect_latency": { 2024-04-07T16:16:14.229 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1, 2024-04-07T16:16:14.229 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.229 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.229 INFO:tasks.workunit.client.0.smithi100.stdout: "collect_timeout": { 2024-04-07T16:16:14.229 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.229 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.229 INFO:tasks.workunit.client.0.smithi100.stdout: "collect_uncommitted": { 2024-04-07T16:16:14.229 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.229 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.229 INFO:tasks.workunit.client.0.smithi100.stdout: "commit": { 2024-04-07T16:16:14.229 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1085 2024-04-07T16:16:14.230 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.230 INFO:tasks.workunit.client.0.smithi100.stdout: "commit_bytes": { 2024-04-07T16:16:14.230 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.230 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.230 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.230 INFO:tasks.workunit.client.0.smithi100.stdout: "commit_keys": { 2024-04-07T16:16:14.230 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.230 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.230 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.230 INFO:tasks.workunit.client.0.smithi100.stdout: "commit_latency": { 2024-04-07T16:16:14.230 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.230 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.231 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.231 INFO:tasks.workunit.client.0.smithi100.stdout: "lease_ack_timeout": { 2024-04-07T16:16:14.231 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.231 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.231 INFO:tasks.workunit.client.0.smithi100.stdout: "lease_timeout": { 2024-04-07T16:16:14.231 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.231 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.231 INFO:tasks.workunit.client.0.smithi100.stdout: "new_pn": { 2024-04-07T16:16:14.231 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.232 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.232 INFO:tasks.workunit.client.0.smithi100.stdout: "new_pn_latency": { 2024-04-07T16:16:14.232 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.232 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.232 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.232 INFO:tasks.workunit.client.0.smithi100.stdout: "refresh": { 2024-04-07T16:16:14.233 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1085 2024-04-07T16:16:14.233 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.233 INFO:tasks.workunit.client.0.smithi100.stdout: "refresh_latency": { 2024-04-07T16:16:14.233 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1085, 2024-04-07T16:16:14.233 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4775931470 2024-04-07T16:16:14.233 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.234 INFO:tasks.workunit.client.0.smithi100.stdout: "restart": { 2024-04-07T16:16:14.234 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 9 2024-04-07T16:16:14.234 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.234 INFO:tasks.workunit.client.0.smithi100.stdout: "share_state": { 2024-04-07T16:16:14.234 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.234 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.234 INFO:tasks.workunit.client.0.smithi100.stdout: "share_state_bytes": { 2024-04-07T16:16:14.235 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.235 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.235 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.235 INFO:tasks.workunit.client.0.smithi100.stdout: "share_state_keys": { 2024-04-07T16:16:14.235 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.235 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.236 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.236 INFO:tasks.workunit.client.0.smithi100.stdout: "start_leader": { 2024-04-07T16:16:14.236 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.236 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.236 INFO:tasks.workunit.client.0.smithi100.stdout: "start_peon": { 2024-04-07T16:16:14.236 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1 2024-04-07T16:16:14.236 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.237 INFO:tasks.workunit.client.0.smithi100.stdout: "store_state": { 2024-04-07T16:16:14.237 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1085 2024-04-07T16:16:14.237 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.237 INFO:tasks.workunit.client.0.smithi100.stdout: "store_state_bytes": { 2024-04-07T16:16:14.237 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1085, 2024-04-07T16:16:14.237 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 8651910 2024-04-07T16:16:14.238 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.238 INFO:tasks.workunit.client.0.smithi100.stdout: "store_state_keys": { 2024-04-07T16:16:14.238 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1085, 2024-04-07T16:16:14.238 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 7649 2024-04-07T16:16:14.238 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.238 INFO:tasks.workunit.client.0.smithi100.stdout: "store_state_latency": { 2024-04-07T16:16:14.239 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1085, 2024-04-07T16:16:14.239 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 747988349 2024-04-07T16:16:14.239 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:14.239 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.239 INFO:tasks.workunit.client.0.smithi100.stdout: "prioritycache": { 2024-04-07T16:16:14.239 INFO:tasks.workunit.client.0.smithi100.stdout: "cache_bytes": { 2024-04-07T16:16:14.239 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1020054731 2024-04-07T16:16:14.240 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.240 INFO:tasks.workunit.client.0.smithi100.stdout: "heap_bytes": { 2024-04-07T16:16:14.240 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 36511744 2024-04-07T16:16:14.240 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.240 INFO:tasks.workunit.client.0.smithi100.stdout: "mapped_bytes": { 2024-04-07T16:16:14.240 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 34045952 2024-04-07T16:16:14.240 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.241 INFO:tasks.workunit.client.0.smithi100.stdout: "target_bytes": { 2024-04-07T16:16:14.241 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2147483648 2024-04-07T16:16:14.241 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.241 INFO:tasks.workunit.client.0.smithi100.stdout: "unmapped_bytes": { 2024-04-07T16:16:14.241 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2465792 2024-04-07T16:16:14.241 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:14.241 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.242 INFO:tasks.workunit.client.0.smithi100.stdout: "prioritycache:full": { 2024-04-07T16:16:14.242 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T16:16:14.242 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 348127232 2024-04-07T16:16:14.242 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.242 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T16:16:14.242 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.243 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.243 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T16:16:14.243 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.243 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.243 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T16:16:14.243 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 277021602 2024-04-07T16:16:14.243 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.244 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T16:16:14.244 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 270681 2024-04-07T16:16:14.244 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.244 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T16:16:14.244 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.244 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.245 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T16:16:14.245 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.245 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.245 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T16:16:14.245 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.245 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.246 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T16:16:14.246 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.246 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.246 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T16:16:14.246 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.246 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.247 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T16:16:14.247 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.247 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.247 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T16:16:14.247 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.247 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.248 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T16:16:14.248 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.248 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.248 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T16:16:14.248 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 70834949 2024-04-07T16:16:14.248 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:14.249 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.249 INFO:tasks.workunit.client.0.smithi100.stdout: "prioritycache:inc": { 2024-04-07T16:16:14.249 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T16:16:14.249 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 348127232 2024-04-07T16:16:14.249 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.249 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T16:16:14.250 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.250 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.250 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T16:16:14.250 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.250 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.250 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T16:16:14.251 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 277021602 2024-04-07T16:16:14.251 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.251 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T16:16:14.251 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 41409 2024-04-07T16:16:14.251 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.251 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T16:16:14.252 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.252 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.252 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T16:16:14.252 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.252 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.252 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T16:16:14.253 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.253 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.253 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T16:16:14.253 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.253 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.253 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T16:16:14.254 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.254 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.254 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T16:16:14.254 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.254 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.254 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T16:16:14.255 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.255 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.255 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T16:16:14.255 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.255 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.255 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T16:16:14.256 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 71064221 2024-04-07T16:16:14.256 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:14.256 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.256 INFO:tasks.workunit.client.0.smithi100.stdout: "prioritycache:kv": { 2024-04-07T16:16:14.256 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T16:16:14.256 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 322961408 2024-04-07T16:16:14.257 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.257 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T16:16:14.257 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 67108864 2024-04-07T16:16:14.257 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.257 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T16:16:14.257 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.258 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.258 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T16:16:14.258 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 184681068 2024-04-07T16:16:14.258 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.258 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T16:16:14.259 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.259 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.259 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T16:16:14.259 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.259 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.259 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T16:16:14.260 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.260 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.260 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T16:16:14.260 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.260 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.260 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T16:16:14.261 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.261 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.261 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T16:16:14.261 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.261 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.262 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T16:16:14.262 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.262 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.262 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T16:16:14.262 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.262 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.263 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T16:16:14.263 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.263 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.263 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T16:16:14.263 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 71171476 2024-04-07T16:16:14.263 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:14.263 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.263 INFO:tasks.workunit.client.0.smithi100.stdout: "rocksdb": { 2024-04-07T16:16:14.263 INFO:tasks.workunit.client.0.smithi100.stdout: "compact": { 2024-04-07T16:16:14.264 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.264 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.264 INFO:tasks.workunit.client.0.smithi100.stdout: "compact_queue_len": { 2024-04-07T16:16:14.264 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.264 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.264 INFO:tasks.workunit.client.0.smithi100.stdout: "compact_queue_merge": { 2024-04-07T16:16:14.264 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.265 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.265 INFO:tasks.workunit.client.0.smithi100.stdout: "compact_range": { 2024-04-07T16:16:14.265 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5 2024-04-07T16:16:14.265 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.265 INFO:tasks.workunit.client.0.smithi100.stdout: "get_latency": { 2024-04-07T16:16:14.265 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 46090, 2024-04-07T16:16:14.265 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 304003357 2024-04-07T16:16:14.265 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.266 INFO:tasks.workunit.client.0.smithi100.stdout: "rocksdb_write_delay_time": { 2024-04-07T16:16:14.266 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.266 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.266 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.266 INFO:tasks.workunit.client.0.smithi100.stdout: "rocksdb_write_memtable_time": { 2024-04-07T16:16:14.266 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.266 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.267 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.267 INFO:tasks.workunit.client.0.smithi100.stdout: "rocksdb_write_pre_and_post_time": { 2024-04-07T16:16:14.267 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.267 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.267 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.267 INFO:tasks.workunit.client.0.smithi100.stdout: "rocksdb_write_wal_time": { 2024-04-07T16:16:14.267 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.267 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.268 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.268 INFO:tasks.workunit.client.0.smithi100.stdout: "submit_latency": { 2024-04-07T16:16:14.268 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.268 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.268 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.268 INFO:tasks.workunit.client.0.smithi100.stdout: "submit_sync_latency": { 2024-04-07T16:16:14.268 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 2315, 2024-04-07T16:16:14.268 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1264177928 2024-04-07T16:16:14.268 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:14.268 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:14.268 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.268 INFO:tasks.workunit.client.0.smithi100.stdout: "mon.a52fc0d8acb515431879057649ff294cfe14c620": { 2024-04-07T16:16:14.268 INFO:tasks.workunit.client.0.smithi100.stdout: "mon": { 2024-04-07T16:16:14.269 INFO:tasks.workunit.client.0.smithi100.stdout: "election_call": { 2024-04-07T16:16:14.269 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2 2024-04-07T16:16:14.269 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.269 INFO:tasks.workunit.client.0.smithi100.stdout: "election_lose": { 2024-04-07T16:16:14.269 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2 2024-04-07T16:16:14.269 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.269 INFO:tasks.workunit.client.0.smithi100.stdout: "election_win": { 2024-04-07T16:16:14.269 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.269 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.269 INFO:tasks.workunit.client.0.smithi100.stdout: "num_elections": { 2024-04-07T16:16:14.269 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 7 2024-04-07T16:16:14.269 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.269 INFO:tasks.workunit.client.0.smithi100.stdout: "num_sessions": { 2024-04-07T16:16:14.270 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 11 2024-04-07T16:16:14.270 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.270 INFO:tasks.workunit.client.0.smithi100.stdout: "session_add": { 2024-04-07T16:16:14.270 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4 2024-04-07T16:16:14.270 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.270 INFO:tasks.workunit.client.0.smithi100.stdout: "session_rm": { 2024-04-07T16:16:14.270 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 52 2024-04-07T16:16:14.270 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.270 INFO:tasks.workunit.client.0.smithi100.stdout: "session_trim": { 2024-04-07T16:16:14.270 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.270 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:14.270 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.270 INFO:tasks.workunit.client.0.smithi100.stdout: "paxos": { 2024-04-07T16:16:14.270 INFO:tasks.workunit.client.0.smithi100.stdout: "accept_timeout": { 2024-04-07T16:16:14.271 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.271 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.271 INFO:tasks.workunit.client.0.smithi100.stdout: "begin": { 2024-04-07T16:16:14.271 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1091 2024-04-07T16:16:14.271 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.271 INFO:tasks.workunit.client.0.smithi100.stdout: "begin_bytes": { 2024-04-07T16:16:14.271 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1091, 2024-04-07T16:16:14.271 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4429500 2024-04-07T16:16:14.271 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.271 INFO:tasks.workunit.client.0.smithi100.stdout: "begin_keys": { 2024-04-07T16:16:14.271 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.271 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.271 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.271 INFO:tasks.workunit.client.0.smithi100.stdout: "begin_latency": { 2024-04-07T16:16:14.271 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1091, 2024-04-07T16:16:14.271 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 315995887 2024-04-07T16:16:14.272 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.272 INFO:tasks.workunit.client.0.smithi100.stdout: "collect": { 2024-04-07T16:16:14.272 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2 2024-04-07T16:16:14.272 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.272 INFO:tasks.workunit.client.0.smithi100.stdout: "collect_bytes": { 2024-04-07T16:16:14.272 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 2, 2024-04-07T16:16:14.272 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 146 2024-04-07T16:16:14.272 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.272 INFO:tasks.workunit.client.0.smithi100.stdout: "collect_keys": { 2024-04-07T16:16:14.272 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 2, 2024-04-07T16:16:14.272 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2 2024-04-07T16:16:14.272 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.272 INFO:tasks.workunit.client.0.smithi100.stdout: "collect_latency": { 2024-04-07T16:16:14.272 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 2, 2024-04-07T16:16:14.272 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3999939 2024-04-07T16:16:14.273 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.273 INFO:tasks.workunit.client.0.smithi100.stdout: "collect_timeout": { 2024-04-07T16:16:14.273 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.273 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.273 INFO:tasks.workunit.client.0.smithi100.stdout: "collect_uncommitted": { 2024-04-07T16:16:14.273 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.273 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.273 INFO:tasks.workunit.client.0.smithi100.stdout: "commit": { 2024-04-07T16:16:14.273 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1092 2024-04-07T16:16:14.273 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.273 INFO:tasks.workunit.client.0.smithi100.stdout: "commit_bytes": { 2024-04-07T16:16:14.273 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.273 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.273 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.273 INFO:tasks.workunit.client.0.smithi100.stdout: "commit_keys": { 2024-04-07T16:16:14.273 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.274 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.274 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.274 INFO:tasks.workunit.client.0.smithi100.stdout: "commit_latency": { 2024-04-07T16:16:14.274 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.274 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.274 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.274 INFO:tasks.workunit.client.0.smithi100.stdout: "lease_ack_timeout": { 2024-04-07T16:16:14.274 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.274 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.274 INFO:tasks.workunit.client.0.smithi100.stdout: "lease_timeout": { 2024-04-07T16:16:14.274 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.274 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.274 INFO:tasks.workunit.client.0.smithi100.stdout: "new_pn": { 2024-04-07T16:16:14.274 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.275 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.275 INFO:tasks.workunit.client.0.smithi100.stdout: "new_pn_latency": { 2024-04-07T16:16:14.275 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.275 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.275 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.275 INFO:tasks.workunit.client.0.smithi100.stdout: "refresh": { 2024-04-07T16:16:14.275 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1092 2024-04-07T16:16:14.275 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.275 INFO:tasks.workunit.client.0.smithi100.stdout: "refresh_latency": { 2024-04-07T16:16:14.275 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1092, 2024-04-07T16:16:14.275 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4191947086 2024-04-07T16:16:14.275 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.275 INFO:tasks.workunit.client.0.smithi100.stdout: "restart": { 2024-04-07T16:16:14.275 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 13 2024-04-07T16:16:14.276 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.276 INFO:tasks.workunit.client.0.smithi100.stdout: "share_state": { 2024-04-07T16:16:14.276 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.276 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.276 INFO:tasks.workunit.client.0.smithi100.stdout: "share_state_bytes": { 2024-04-07T16:16:14.276 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.276 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.276 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.276 INFO:tasks.workunit.client.0.smithi100.stdout: "share_state_keys": { 2024-04-07T16:16:14.276 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.276 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.276 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.276 INFO:tasks.workunit.client.0.smithi100.stdout: "start_leader": { 2024-04-07T16:16:14.276 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.277 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.277 INFO:tasks.workunit.client.0.smithi100.stdout: "start_peon": { 2024-04-07T16:16:14.277 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2 2024-04-07T16:16:14.277 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.277 INFO:tasks.workunit.client.0.smithi100.stdout: "store_state": { 2024-04-07T16:16:14.277 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1092 2024-04-07T16:16:14.277 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.277 INFO:tasks.workunit.client.0.smithi100.stdout: "store_state_bytes": { 2024-04-07T16:16:14.277 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1092, 2024-04-07T16:16:14.277 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 8653387 2024-04-07T16:16:14.277 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.277 INFO:tasks.workunit.client.0.smithi100.stdout: "store_state_keys": { 2024-04-07T16:16:14.277 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1092, 2024-04-07T16:16:14.277 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 7660 2024-04-07T16:16:14.278 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.278 INFO:tasks.workunit.client.0.smithi100.stdout: "store_state_latency": { 2024-04-07T16:16:14.278 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1092, 2024-04-07T16:16:14.278 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 419994480 2024-04-07T16:16:14.278 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:14.278 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.278 INFO:tasks.workunit.client.0.smithi100.stdout: "prioritycache": { 2024-04-07T16:16:14.278 INFO:tasks.workunit.client.0.smithi100.stdout: "cache_bytes": { 2024-04-07T16:16:14.278 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1020054731 2024-04-07T16:16:14.278 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.278 INFO:tasks.workunit.client.0.smithi100.stdout: "heap_bytes": { 2024-04-07T16:16:14.279 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 37560320 2024-04-07T16:16:14.279 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.279 INFO:tasks.workunit.client.0.smithi100.stdout: "mapped_bytes": { 2024-04-07T16:16:14.279 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 35463168 2024-04-07T16:16:14.279 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.279 INFO:tasks.workunit.client.0.smithi100.stdout: "target_bytes": { 2024-04-07T16:16:14.279 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2147483648 2024-04-07T16:16:14.279 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.279 INFO:tasks.workunit.client.0.smithi100.stdout: "unmapped_bytes": { 2024-04-07T16:16:14.279 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2097152 2024-04-07T16:16:14.279 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:14.279 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.279 INFO:tasks.workunit.client.0.smithi100.stdout: "prioritycache:full": { 2024-04-07T16:16:14.279 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T16:16:14.280 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 348127232 2024-04-07T16:16:14.280 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.280 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T16:16:14.280 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.280 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.280 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T16:16:14.280 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.281 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.281 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T16:16:14.281 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 277021602 2024-04-07T16:16:14.281 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.281 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T16:16:14.281 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 270681 2024-04-07T16:16:14.281 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.281 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T16:16:14.281 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.282 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.282 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T16:16:14.282 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.282 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.282 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T16:16:14.282 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.282 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.282 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T16:16:14.282 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.282 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.282 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T16:16:14.282 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.282 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.282 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T16:16:14.283 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.283 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.283 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T16:16:14.283 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.283 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.283 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T16:16:14.283 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.283 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.283 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T16:16:14.283 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 70834949 2024-04-07T16:16:14.283 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:14.283 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.283 INFO:tasks.workunit.client.0.smithi100.stdout: "prioritycache:inc": { 2024-04-07T16:16:14.283 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T16:16:14.283 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 348127232 2024-04-07T16:16:14.284 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.284 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T16:16:14.284 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.284 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.284 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T16:16:14.284 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.284 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.284 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T16:16:14.284 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 277021602 2024-04-07T16:16:14.284 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.284 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T16:16:14.284 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 41409 2024-04-07T16:16:14.284 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.284 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T16:16:14.284 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.285 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.285 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T16:16:14.285 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.285 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.285 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T16:16:14.285 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.285 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.285 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T16:16:14.285 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.285 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.285 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T16:16:14.285 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.285 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.285 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T16:16:14.285 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.285 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.286 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T16:16:14.286 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.286 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.286 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T16:16:14.286 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.286 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.286 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T16:16:14.286 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 71064221 2024-04-07T16:16:14.286 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:14.286 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.286 INFO:tasks.workunit.client.0.smithi100.stdout: "prioritycache:kv": { 2024-04-07T16:16:14.286 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T16:16:14.286 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 322961408 2024-04-07T16:16:14.286 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.287 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T16:16:14.287 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 67108864 2024-04-07T16:16:14.287 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.287 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T16:16:14.287 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.287 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.287 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T16:16:14.287 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 184681068 2024-04-07T16:16:14.287 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.287 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T16:16:14.287 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.287 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.287 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T16:16:14.287 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.288 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.288 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T16:16:14.288 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.288 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.288 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T16:16:14.288 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.288 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.288 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T16:16:14.288 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.288 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.288 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T16:16:14.288 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.288 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.288 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T16:16:14.288 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.289 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.289 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T16:16:14.289 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.289 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.289 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T16:16:14.289 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.289 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.289 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T16:16:14.289 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 71171476 2024-04-07T16:16:14.289 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:14.289 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.289 INFO:tasks.workunit.client.0.smithi100.stdout: "rocksdb": { 2024-04-07T16:16:14.289 INFO:tasks.workunit.client.0.smithi100.stdout: "compact": { 2024-04-07T16:16:14.289 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.289 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.290 INFO:tasks.workunit.client.0.smithi100.stdout: "compact_queue_len": { 2024-04-07T16:16:14.290 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.290 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.290 INFO:tasks.workunit.client.0.smithi100.stdout: "compact_queue_merge": { 2024-04-07T16:16:14.290 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.290 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.290 INFO:tasks.workunit.client.0.smithi100.stdout: "compact_range": { 2024-04-07T16:16:14.290 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5 2024-04-07T16:16:14.290 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.290 INFO:tasks.workunit.client.0.smithi100.stdout: "get_latency": { 2024-04-07T16:16:14.290 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 46763, 2024-04-07T16:16:14.290 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 265530695 2024-04-07T16:16:14.291 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.291 INFO:tasks.workunit.client.0.smithi100.stdout: "rocksdb_write_delay_time": { 2024-04-07T16:16:14.291 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.291 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.291 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.291 INFO:tasks.workunit.client.0.smithi100.stdout: "rocksdb_write_memtable_time": { 2024-04-07T16:16:14.291 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.291 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.291 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.291 INFO:tasks.workunit.client.0.smithi100.stdout: "rocksdb_write_pre_and_post_time": { 2024-04-07T16:16:14.291 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.291 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.291 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.291 INFO:tasks.workunit.client.0.smithi100.stdout: "rocksdb_write_wal_time": { 2024-04-07T16:16:14.291 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.292 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.292 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.292 INFO:tasks.workunit.client.0.smithi100.stdout: "submit_latency": { 2024-04-07T16:16:14.292 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.292 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.292 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.292 INFO:tasks.workunit.client.0.smithi100.stdout: "submit_sync_latency": { 2024-04-07T16:16:14.292 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 2369, 2024-04-07T16:16:14.292 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 837174549 2024-04-07T16:16:14.292 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:14.292 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:14.292 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.292 INFO:tasks.workunit.client.0.smithi100.stdout: "mon.f47a4ff15a45950a63160265564a123fc5da5e5e": { 2024-04-07T16:16:14.292 INFO:tasks.workunit.client.0.smithi100.stdout: "mon": { 2024-04-07T16:16:14.292 INFO:tasks.workunit.client.0.smithi100.stdout: "election_call": { 2024-04-07T16:16:14.293 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2 2024-04-07T16:16:14.293 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.293 INFO:tasks.workunit.client.0.smithi100.stdout: "election_lose": { 2024-04-07T16:16:14.293 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.293 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.293 INFO:tasks.workunit.client.0.smithi100.stdout: "election_win": { 2024-04-07T16:16:14.293 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3 2024-04-07T16:16:14.293 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.293 INFO:tasks.workunit.client.0.smithi100.stdout: "num_elections": { 2024-04-07T16:16:14.293 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 9 2024-04-07T16:16:14.293 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.293 INFO:tasks.workunit.client.0.smithi100.stdout: "num_sessions": { 2024-04-07T16:16:14.293 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 6 2024-04-07T16:16:14.293 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.293 INFO:tasks.workunit.client.0.smithi100.stdout: "session_add": { 2024-04-07T16:16:14.294 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3 2024-04-07T16:16:14.294 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.294 INFO:tasks.workunit.client.0.smithi100.stdout: "session_rm": { 2024-04-07T16:16:14.294 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 183 2024-04-07T16:16:14.294 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.294 INFO:tasks.workunit.client.0.smithi100.stdout: "session_trim": { 2024-04-07T16:16:14.294 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.294 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:14.294 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.294 INFO:tasks.workunit.client.0.smithi100.stdout: "paxos": { 2024-04-07T16:16:14.294 INFO:tasks.workunit.client.0.smithi100.stdout: "accept_timeout": { 2024-04-07T16:16:14.294 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.294 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.294 INFO:tasks.workunit.client.0.smithi100.stdout: "begin": { 2024-04-07T16:16:14.295 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1247 2024-04-07T16:16:14.295 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.295 INFO:tasks.workunit.client.0.smithi100.stdout: "begin_bytes": { 2024-04-07T16:16:14.295 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1247, 2024-04-07T16:16:14.295 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5523788 2024-04-07T16:16:14.295 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.295 INFO:tasks.workunit.client.0.smithi100.stdout: "begin_keys": { 2024-04-07T16:16:14.295 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1247, 2024-04-07T16:16:14.295 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3741 2024-04-07T16:16:14.295 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.295 INFO:tasks.workunit.client.0.smithi100.stdout: "begin_latency": { 2024-04-07T16:16:14.295 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1247, 2024-04-07T16:16:14.295 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 395992072 2024-04-07T16:16:14.296 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.296 INFO:tasks.workunit.client.0.smithi100.stdout: "collect": { 2024-04-07T16:16:14.296 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.296 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.296 INFO:tasks.workunit.client.0.smithi100.stdout: "collect_bytes": { 2024-04-07T16:16:14.296 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.296 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.296 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.296 INFO:tasks.workunit.client.0.smithi100.stdout: "collect_keys": { 2024-04-07T16:16:14.296 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.296 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.296 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.296 INFO:tasks.workunit.client.0.smithi100.stdout: "collect_latency": { 2024-04-07T16:16:14.296 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.296 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.297 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.297 INFO:tasks.workunit.client.0.smithi100.stdout: "collect_timeout": { 2024-04-07T16:16:14.297 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.297 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.297 INFO:tasks.workunit.client.0.smithi100.stdout: "collect_uncommitted": { 2024-04-07T16:16:14.297 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.297 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.297 INFO:tasks.workunit.client.0.smithi100.stdout: "commit": { 2024-04-07T16:16:14.297 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1247 2024-04-07T16:16:14.297 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.297 INFO:tasks.workunit.client.0.smithi100.stdout: "commit_bytes": { 2024-04-07T16:16:14.297 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1247, 2024-04-07T16:16:14.297 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5433659 2024-04-07T16:16:14.297 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.298 INFO:tasks.workunit.client.0.smithi100.stdout: "commit_keys": { 2024-04-07T16:16:14.298 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1247, 2024-04-07T16:16:14.298 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 7385 2024-04-07T16:16:14.298 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.298 INFO:tasks.workunit.client.0.smithi100.stdout: "commit_latency": { 2024-04-07T16:16:14.298 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1247, 2024-04-07T16:16:14.298 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 588170271 2024-04-07T16:16:14.298 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.298 INFO:tasks.workunit.client.0.smithi100.stdout: "lease_ack_timeout": { 2024-04-07T16:16:14.298 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.298 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.298 INFO:tasks.workunit.client.0.smithi100.stdout: "lease_timeout": { 2024-04-07T16:16:14.298 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.298 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.299 INFO:tasks.workunit.client.0.smithi100.stdout: "new_pn": { 2024-04-07T16:16:14.299 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2 2024-04-07T16:16:14.299 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.299 INFO:tasks.workunit.client.0.smithi100.stdout: "new_pn_latency": { 2024-04-07T16:16:14.299 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 2, 2024-04-07T16:16:14.299 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.299 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.299 INFO:tasks.workunit.client.0.smithi100.stdout: "refresh": { 2024-04-07T16:16:14.299 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1249 2024-04-07T16:16:14.299 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.299 INFO:tasks.workunit.client.0.smithi100.stdout: "refresh_latency": { 2024-04-07T16:16:14.299 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1249, 2024-04-07T16:16:14.299 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5695905187 2024-04-07T16:16:14.299 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.300 INFO:tasks.workunit.client.0.smithi100.stdout: "restart": { 2024-04-07T16:16:14.300 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 12 2024-04-07T16:16:14.300 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.300 INFO:tasks.workunit.client.0.smithi100.stdout: "share_state": { 2024-04-07T16:16:14.300 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2 2024-04-07T16:16:14.300 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.300 INFO:tasks.workunit.client.0.smithi100.stdout: "share_state_bytes": { 2024-04-07T16:16:14.300 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 2, 2024-04-07T16:16:14.300 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 25218 2024-04-07T16:16:14.300 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.300 INFO:tasks.workunit.client.0.smithi100.stdout: "share_state_keys": { 2024-04-07T16:16:14.300 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 2, 2024-04-07T16:16:14.300 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 8 2024-04-07T16:16:14.300 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.301 INFO:tasks.workunit.client.0.smithi100.stdout: "start_leader": { 2024-04-07T16:16:14.301 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3 2024-04-07T16:16:14.301 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.301 INFO:tasks.workunit.client.0.smithi100.stdout: "start_peon": { 2024-04-07T16:16:14.301 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.301 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.301 INFO:tasks.workunit.client.0.smithi100.stdout: "store_state": { 2024-04-07T16:16:14.301 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.301 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.301 INFO:tasks.workunit.client.0.smithi100.stdout: "store_state_bytes": { 2024-04-07T16:16:14.301 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.301 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.301 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.302 INFO:tasks.workunit.client.0.smithi100.stdout: "store_state_keys": { 2024-04-07T16:16:14.302 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.302 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.302 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.302 INFO:tasks.workunit.client.0.smithi100.stdout: "store_state_latency": { 2024-04-07T16:16:14.302 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.302 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.302 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:14.302 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.302 INFO:tasks.workunit.client.0.smithi100.stdout: "prioritycache": { 2024-04-07T16:16:14.302 INFO:tasks.workunit.client.0.smithi100.stdout: "cache_bytes": { 2024-04-07T16:16:14.302 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1020054731 2024-04-07T16:16:14.302 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.302 INFO:tasks.workunit.client.0.smithi100.stdout: "heap_bytes": { 2024-04-07T16:16:14.303 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 43753472 2024-04-07T16:16:14.303 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.303 INFO:tasks.workunit.client.0.smithi100.stdout: "mapped_bytes": { 2024-04-07T16:16:14.303 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 42385408 2024-04-07T16:16:14.303 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.303 INFO:tasks.workunit.client.0.smithi100.stdout: "target_bytes": { 2024-04-07T16:16:14.303 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2147483648 2024-04-07T16:16:14.303 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.303 INFO:tasks.workunit.client.0.smithi100.stdout: "unmapped_bytes": { 2024-04-07T16:16:14.303 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1368064 2024-04-07T16:16:14.303 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:14.303 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.303 INFO:tasks.workunit.client.0.smithi100.stdout: "prioritycache:full": { 2024-04-07T16:16:14.303 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T16:16:14.304 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 348127232 2024-04-07T16:16:14.304 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.304 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T16:16:14.304 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.304 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.304 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T16:16:14.304 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.304 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.304 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T16:16:14.304 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 277023025 2024-04-07T16:16:14.304 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.304 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T16:16:14.304 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 270251 2024-04-07T16:16:14.304 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.305 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T16:16:14.305 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.305 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.305 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T16:16:14.305 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.305 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.305 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T16:16:14.305 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.305 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.305 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T16:16:14.305 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.305 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.305 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T16:16:14.305 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.306 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.306 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T16:16:14.306 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.306 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.306 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T16:16:14.306 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.306 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.306 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T16:16:14.306 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.306 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.306 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T16:16:14.306 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 70833956 2024-04-07T16:16:14.306 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:14.306 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.307 INFO:tasks.workunit.client.0.smithi100.stdout: "prioritycache:inc": { 2024-04-07T16:16:14.307 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T16:16:14.307 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 348127232 2024-04-07T16:16:14.307 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.307 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T16:16:14.307 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.307 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.307 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T16:16:14.307 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.307 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.307 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T16:16:14.307 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 277023025 2024-04-07T16:16:14.307 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.307 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T16:16:14.308 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 38045 2024-04-07T16:16:14.308 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.308 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T16:16:14.308 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.308 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.308 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T16:16:14.308 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.308 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.308 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T16:16:14.308 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.308 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.308 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T16:16:14.308 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.308 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.308 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T16:16:14.308 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.309 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.309 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T16:16:14.309 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.309 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.309 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T16:16:14.309 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.309 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.309 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T16:16:14.309 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.309 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.309 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T16:16:14.309 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 71066162 2024-04-07T16:16:14.309 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:14.309 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.309 INFO:tasks.workunit.client.0.smithi100.stdout: "prioritycache:kv": { 2024-04-07T16:16:14.310 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T16:16:14.310 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 322961408 2024-04-07T16:16:14.310 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.310 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T16:16:14.310 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 67108864 2024-04-07T16:16:14.310 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.310 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T16:16:14.310 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.310 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.310 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T16:16:14.310 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 184682016 2024-04-07T16:16:14.311 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.311 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T16:16:14.311 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.311 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.311 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T16:16:14.311 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.311 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.311 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T16:16:14.311 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.311 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.311 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T16:16:14.311 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.311 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.311 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T16:16:14.312 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.312 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.312 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T16:16:14.312 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.312 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.312 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T16:16:14.312 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.312 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.312 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T16:16:14.312 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.312 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.312 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T16:16:14.312 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.312 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.313 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T16:16:14.313 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 71170528 2024-04-07T16:16:14.313 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:14.313 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.313 INFO:tasks.workunit.client.0.smithi100.stdout: "rocksdb": { 2024-04-07T16:16:14.313 INFO:tasks.workunit.client.0.smithi100.stdout: "compact": { 2024-04-07T16:16:14.313 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.313 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.313 INFO:tasks.workunit.client.0.smithi100.stdout: "compact_queue_len": { 2024-04-07T16:16:14.313 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.313 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.313 INFO:tasks.workunit.client.0.smithi100.stdout: "compact_queue_merge": { 2024-04-07T16:16:14.313 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.313 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.314 INFO:tasks.workunit.client.0.smithi100.stdout: "compact_range": { 2024-04-07T16:16:14.314 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5 2024-04-07T16:16:14.314 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.314 INFO:tasks.workunit.client.0.smithi100.stdout: "get_latency": { 2024-04-07T16:16:14.314 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 55404, 2024-04-07T16:16:14.314 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 482881550 2024-04-07T16:16:14.314 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.314 INFO:tasks.workunit.client.0.smithi100.stdout: "rocksdb_write_delay_time": { 2024-04-07T16:16:14.314 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.314 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.314 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.314 INFO:tasks.workunit.client.0.smithi100.stdout: "rocksdb_write_memtable_time": { 2024-04-07T16:16:14.314 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.314 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.315 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.315 INFO:tasks.workunit.client.0.smithi100.stdout: "rocksdb_write_pre_and_post_time": { 2024-04-07T16:16:14.315 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.315 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.315 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.315 INFO:tasks.workunit.client.0.smithi100.stdout: "rocksdb_write_wal_time": { 2024-04-07T16:16:14.315 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.315 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.315 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.315 INFO:tasks.workunit.client.0.smithi100.stdout: "submit_latency": { 2024-04-07T16:16:14.315 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.315 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.315 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.315 INFO:tasks.workunit.client.0.smithi100.stdout: "submit_sync_latency": { 2024-04-07T16:16:14.316 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 2712, 2024-04-07T16:16:14.316 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1014341376 2024-04-07T16:16:14.316 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:14.316 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:14.316 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.316 INFO:tasks.workunit.client.0.smithi100.stdout: "osd.0": { 2024-04-07T16:16:14.316 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs": { 2024-04-07T16:16:14.316 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_slow_fallback": { 2024-04-07T16:16:14.316 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.316 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.316 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_slow_size_fallback": { 2024-04-07T16:16:14.316 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.316 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.316 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit_db": { 2024-04-07T16:16:14.317 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 65536 2024-04-07T16:16:14.317 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.317 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit_main": { 2024-04-07T16:16:14.317 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.317 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.317 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit_wal": { 2024-04-07T16:16:14.317 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.317 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.317 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_written_slow": { 2024-04-07T16:16:14.317 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.317 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.317 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_written_sst": { 2024-04-07T16:16:14.317 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 12288 2024-04-07T16:16:14.317 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.317 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_written_wal": { 2024-04-07T16:16:14.318 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2666496 2024-04-07T16:16:14.318 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.318 INFO:tasks.workunit.client.0.smithi100.stdout: "compact_lat": { 2024-04-07T16:16:14.318 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.318 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.318 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.318 INFO:tasks.workunit.client.0.smithi100.stdout: "compact_lock_lat": { 2024-04-07T16:16:14.318 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.318 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.318 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.318 INFO:tasks.workunit.client.0.smithi100.stdout: "db_total_bytes": { 2024-04-07T16:16:14.318 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 95995035648 2024-04-07T16:16:14.318 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.318 INFO:tasks.workunit.client.0.smithi100.stdout: "db_used_bytes": { 2024-04-07T16:16:14.319 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 27656192 2024-04-07T16:16:14.319 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.319 INFO:tasks.workunit.client.0.smithi100.stdout: "log_bytes": { 2024-04-07T16:16:14.319 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1687552 2024-04-07T16:16:14.319 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.319 INFO:tasks.workunit.client.0.smithi100.stdout: "logged_bytes": { 2024-04-07T16:16:14.319 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1548288 2024-04-07T16:16:14.319 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.319 INFO:tasks.workunit.client.0.smithi100.stdout: "max_bytes_db": { 2024-04-07T16:16:14.319 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 31850496 2024-04-07T16:16:14.319 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.319 INFO:tasks.workunit.client.0.smithi100.stdout: "max_bytes_slow": { 2024-04-07T16:16:14.319 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.320 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.320 INFO:tasks.workunit.client.0.smithi100.stdout: "max_bytes_wal": { 2024-04-07T16:16:14.320 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.320 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.320 INFO:tasks.workunit.client.0.smithi100.stdout: "num_files": { 2024-04-07T16:16:14.320 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 14 2024-04-07T16:16:14.320 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.320 INFO:tasks.workunit.client.0.smithi100.stdout: "read_bytes": { 2024-04-07T16:16:14.320 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 206272 2024-04-07T16:16:14.320 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.320 INFO:tasks.workunit.client.0.smithi100.stdout: "read_count": { 2024-04-07T16:16:14.320 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 23 2024-04-07T16:16:14.320 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.320 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes": { 2024-04-07T16:16:14.320 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1282048 2024-04-07T16:16:14.321 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.321 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes_db": { 2024-04-07T16:16:14.321 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1286144 2024-04-07T16:16:14.321 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.321 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes_slow": { 2024-04-07T16:16:14.321 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.321 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.321 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes_wal": { 2024-04-07T16:16:14.321 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.321 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.321 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_count": { 2024-04-07T16:16:14.321 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 15 2024-04-07T16:16:14.321 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.321 INFO:tasks.workunit.client.0.smithi100.stdout: "read_prefetch_bytes": { 2024-04-07T16:16:14.322 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4913 2024-04-07T16:16:14.322 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.322 INFO:tasks.workunit.client.0.smithi100.stdout: "read_prefetch_count": { 2024-04-07T16:16:14.322 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4 2024-04-07T16:16:14.322 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.322 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_buffer_bytes": { 2024-04-07T16:16:14.322 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5504 2024-04-07T16:16:14.322 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.322 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_buffer_count": { 2024-04-07T16:16:14.322 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 24 2024-04-07T16:16:14.322 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.322 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_bytes": { 2024-04-07T16:16:14.322 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5504 2024-04-07T16:16:14.323 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.323 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_count": { 2024-04-07T16:16:14.323 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 24 2024-04-07T16:16:14.323 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.323 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes": { 2024-04-07T16:16:14.323 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.323 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.323 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes_db": { 2024-04-07T16:16:14.323 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.323 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.323 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes_slow": { 2024-04-07T16:16:14.323 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.323 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.323 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes_wal": { 2024-04-07T16:16:14.324 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.324 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.324 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_count": { 2024-04-07T16:16:14.324 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.324 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.324 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_total_bytes": { 2024-04-07T16:16:14.324 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.324 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.324 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_used_bytes": { 2024-04-07T16:16:14.324 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.324 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.324 INFO:tasks.workunit.client.0.smithi100.stdout: "wal_total_bytes": { 2024-04-07T16:16:14.324 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.324 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.324 INFO:tasks.workunit.client.0.smithi100.stdout: "wal_used_bytes": { 2024-04-07T16:16:14.325 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.325 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.325 INFO:tasks.workunit.client.0.smithi100.stdout: "write_bytes": { 2024-04-07T16:16:14.325 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4292608 2024-04-07T16:16:14.325 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:14.325 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.325 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore": { 2024-04-07T16:16:14.325 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit": { 2024-04-07T16:16:14.325 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4096 2024-04-07T16:16:14.325 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.325 INFO:tasks.workunit.client.0.smithi100.stdout: "allocated": { 2024-04-07T16:16:14.325 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1318912 2024-04-07T16:16:14.325 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.325 INFO:tasks.workunit.client.0.smithi100.stdout: "clist_lat": { 2024-04-07T16:16:14.326 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 6, 2024-04-07T16:16:14.326 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 377454 2024-04-07T16:16:14.326 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.326 INFO:tasks.workunit.client.0.smithi100.stdout: "compress_lat": { 2024-04-07T16:16:14.326 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.326 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.326 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.326 INFO:tasks.workunit.client.0.smithi100.stdout: "compressed": { 2024-04-07T16:16:14.326 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.326 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.326 INFO:tasks.workunit.client.0.smithi100.stdout: "compressed_allocated": { 2024-04-07T16:16:14.326 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.326 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.326 INFO:tasks.workunit.client.0.smithi100.stdout: "compressed_original": { 2024-04-07T16:16:14.327 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.327 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.327 INFO:tasks.workunit.client.0.smithi100.stdout: "csum_lat": { 2024-04-07T16:16:14.327 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 25, 2024-04-07T16:16:14.327 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 111336 2024-04-07T16:16:14.327 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.327 INFO:tasks.workunit.client.0.smithi100.stdout: "decompress_lat": { 2024-04-07T16:16:14.327 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.327 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.327 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.327 INFO:tasks.workunit.client.0.smithi100.stdout: "fragmentation_micros": { 2024-04-07T16:16:14.327 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.327 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.327 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_commit_lat": { 2024-04-07T16:16:14.328 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 365, 2024-04-07T16:16:14.328 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 124990572 2024-04-07T16:16:14.328 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.328 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_final_lat": { 2024-04-07T16:16:14.328 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 348, 2024-04-07T16:16:14.328 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 27679682 2024-04-07T16:16:14.328 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.328 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_flush_lat": { 2024-04-07T16:16:14.328 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 365, 2024-04-07T16:16:14.328 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 705709 2024-04-07T16:16:14.328 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.328 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_sync_lat": { 2024-04-07T16:16:14.328 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 365, 2024-04-07T16:16:14.328 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 125696281 2024-04-07T16:16:14.329 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.329 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_get_keys_lat": { 2024-04-07T16:16:14.329 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.329 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.329 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.329 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_get_values_lat": { 2024-04-07T16:16:14.329 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1, 2024-04-07T16:16:14.329 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 53150 2024-04-07T16:16:14.329 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.329 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_lower_bound_lat": { 2024-04-07T16:16:14.329 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 6, 2024-04-07T16:16:14.329 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 32820 2024-04-07T16:16:14.329 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.330 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_next_lat": { 2024-04-07T16:16:14.330 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.330 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.330 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.330 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_seek_to_first_lat": { 2024-04-07T16:16:14.330 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.330 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.330 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.330 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_upper_bound_lat": { 2024-04-07T16:16:14.330 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 2, 2024-04-07T16:16:14.330 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 6225 2024-04-07T16:16:14.330 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.330 INFO:tasks.workunit.client.0.smithi100.stdout: "onode_hits": { 2024-04-07T16:16:14.331 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 802 2024-04-07T16:16:14.331 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.331 INFO:tasks.workunit.client.0.smithi100.stdout: "onode_misses": { 2024-04-07T16:16:14.331 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 222 2024-04-07T16:16:14.331 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.331 INFO:tasks.workunit.client.0.smithi100.stdout: "read_lat": { 2024-04-07T16:16:14.331 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 121, 2024-04-07T16:16:14.331 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3967663 2024-04-07T16:16:14.331 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.331 INFO:tasks.workunit.client.0.smithi100.stdout: "read_onode_meta_lat": { 2024-04-07T16:16:14.331 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 327, 2024-04-07T16:16:14.331 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 449756 2024-04-07T16:16:14.332 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.332 INFO:tasks.workunit.client.0.smithi100.stdout: "read_wait_aio_lat": { 2024-04-07T16:16:14.332 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 206, 2024-04-07T16:16:14.332 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3412921 2024-04-07T16:16:14.332 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.332 INFO:tasks.workunit.client.0.smithi100.stdout: "reads_with_retries": { 2024-04-07T16:16:14.332 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.332 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.332 INFO:tasks.workunit.client.0.smithi100.stdout: "remove_lat": { 2024-04-07T16:16:14.333 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 3, 2024-04-07T16:16:14.333 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 138272 2024-04-07T16:16:14.333 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.333 INFO:tasks.workunit.client.0.smithi100.stdout: "state_aio_wait_lat": { 2024-04-07T16:16:14.333 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 465, 2024-04-07T16:16:14.333 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 12011436 2024-04-07T16:16:14.333 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.333 INFO:tasks.workunit.client.0.smithi100.stdout: "state_deferred_aio_wait_lat": { 2024-04-07T16:16:14.333 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 125, 2024-04-07T16:16:14.333 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 9953550 2024-04-07T16:16:14.333 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.333 INFO:tasks.workunit.client.0.smithi100.stdout: "state_deferred_cleanup_lat": { 2024-04-07T16:16:14.333 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 124, 2024-04-07T16:16:14.333 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 621573355212 2024-04-07T16:16:14.333 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.334 INFO:tasks.workunit.client.0.smithi100.stdout: "state_deferred_queued_lat": { 2024-04-07T16:16:14.334 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 125, 2024-04-07T16:16:14.334 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 150872178514 2024-04-07T16:16:14.334 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.334 INFO:tasks.workunit.client.0.smithi100.stdout: "state_done_lat": { 2024-04-07T16:16:14.334 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 464, 2024-04-07T16:16:14.334 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 88578242285 2024-04-07T16:16:14.334 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.334 INFO:tasks.workunit.client.0.smithi100.stdout: "state_finishing_lat": { 2024-04-07T16:16:14.334 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 464, 2024-04-07T16:16:14.334 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 73968 2024-04-07T16:16:14.334 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.334 INFO:tasks.workunit.client.0.smithi100.stdout: "state_io_done_lat": { 2024-04-07T16:16:14.334 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 465, 2024-04-07T16:16:14.335 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 847905 2024-04-07T16:16:14.335 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.335 INFO:tasks.workunit.client.0.smithi100.stdout: "state_kv_commiting_lat": { 2024-04-07T16:16:14.335 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 465, 2024-04-07T16:16:14.335 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 433490520 2024-04-07T16:16:14.335 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.335 INFO:tasks.workunit.client.0.smithi100.stdout: "state_kv_done_lat": { 2024-04-07T16:16:14.335 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 465, 2024-04-07T16:16:14.335 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 285844 2024-04-07T16:16:14.335 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.335 INFO:tasks.workunit.client.0.smithi100.stdout: "state_kv_queued_lat": { 2024-04-07T16:16:14.335 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 465, 2024-04-07T16:16:14.335 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 139202565 2024-04-07T16:16:14.335 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.335 INFO:tasks.workunit.client.0.smithi100.stdout: "state_prepare_lat": { 2024-04-07T16:16:14.335 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 465, 2024-04-07T16:16:14.336 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 36530204 2024-04-07T16:16:14.336 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.336 INFO:tasks.workunit.client.0.smithi100.stdout: "stored": { 2024-04-07T16:16:14.336 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 900328 2024-04-07T16:16:14.336 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.336 INFO:tasks.workunit.client.0.smithi100.stdout: "truncate_lat": { 2024-04-07T16:16:14.336 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 59, 2024-04-07T16:16:14.336 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 497022 2024-04-07T16:16:14.336 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.336 INFO:tasks.workunit.client.0.smithi100.stdout: "txc_commit_lat": { 2024-04-07T16:16:14.336 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 465, 2024-04-07T16:16:14.336 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 622206329 2024-04-07T16:16:14.336 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.336 INFO:tasks.workunit.client.0.smithi100.stdout: "txc_submit_lat": { 2024-04-07T16:16:14.337 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 465, 2024-04-07T16:16:14.337 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 44152219 2024-04-07T16:16:14.337 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.337 INFO:tasks.workunit.client.0.smithi100.stdout: "txc_throttle_lat": { 2024-04-07T16:16:14.337 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 465, 2024-04-07T16:16:14.337 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5209121 2024-04-07T16:16:14.337 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:14.337 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.337 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache": { 2024-04-07T16:16:14.337 INFO:tasks.workunit.client.0.smithi100.stdout: "cache_bytes": { 2024-04-07T16:16:14.337 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1900954368 2024-04-07T16:16:14.337 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.337 INFO:tasks.workunit.client.0.smithi100.stdout: "heap_bytes": { 2024-04-07T16:16:14.337 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 44482560 2024-04-07T16:16:14.338 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.338 INFO:tasks.workunit.client.0.smithi100.stdout: "mapped_bytes": { 2024-04-07T16:16:14.338 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 44417024 2024-04-07T16:16:14.338 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.338 INFO:tasks.workunit.client.0.smithi100.stdout: "target_bytes": { 2024-04-07T16:16:14.338 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3183836160 2024-04-07T16:16:14.338 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.338 INFO:tasks.workunit.client.0.smithi100.stdout: "unmapped_bytes": { 2024-04-07T16:16:14.338 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 65536 2024-04-07T16:16:14.338 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:14.338 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.338 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:data": { 2024-04-07T16:16:14.338 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T16:16:14.338 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 159383552 2024-04-07T16:16:14.339 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.339 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T16:16:14.339 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.339 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.339 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T16:16:14.339 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.339 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.339 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T16:16:14.339 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 87830780 2024-04-07T16:16:14.339 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.339 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T16:16:14.339 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.339 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.339 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T16:16:14.339 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.340 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.340 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T16:16:14.340 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.340 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.340 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T16:16:14.340 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.340 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.340 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T16:16:14.340 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.340 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.340 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T16:16:14.340 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 614400 2024-04-07T16:16:14.340 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.340 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T16:16:14.340 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.341 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.341 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T16:16:14.341 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.341 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.341 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T16:16:14.341 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.341 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.341 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T16:16:14.341 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 70938372 2024-04-07T16:16:14.341 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:14.341 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.341 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:kv": { 2024-04-07T16:16:14.341 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T16:16:14.341 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 796917760 2024-04-07T16:16:14.342 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.342 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T16:16:14.342 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 67108864 2024-04-07T16:16:14.342 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.342 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T16:16:14.342 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.342 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.342 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T16:16:14.342 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 658730852 2024-04-07T16:16:14.342 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.342 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T16:16:14.342 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.342 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.342 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T16:16:14.342 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.343 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.343 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T16:16:14.343 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.343 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.343 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T16:16:14.343 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.343 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.343 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T16:16:14.343 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.343 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.343 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T16:16:14.343 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2128 2024-04-07T16:16:14.343 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.343 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T16:16:14.343 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.344 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.344 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T16:16:14.344 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.344 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.344 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T16:16:14.344 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.344 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.344 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T16:16:14.344 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 71075916 2024-04-07T16:16:14.344 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:14.344 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.344 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:kv_onode": { 2024-04-07T16:16:14.344 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T16:16:14.344 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 192937984 2024-04-07T16:16:14.344 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.344 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T16:16:14.345 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 67108864 2024-04-07T16:16:14.345 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.345 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T16:16:14.345 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 464 2024-04-07T16:16:14.345 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.345 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T16:16:14.345 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 58553853 2024-04-07T16:16:14.345 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.345 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T16:16:14.345 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.345 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.345 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T16:16:14.345 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.345 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.345 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T16:16:14.346 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.346 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.346 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T16:16:14.346 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.346 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.346 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T16:16:14.346 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.346 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.346 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T16:16:14.346 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.346 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.346 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T16:16:14.346 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.346 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.347 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T16:16:14.347 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.347 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.347 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T16:16:14.347 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.347 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.347 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T16:16:14.347 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 67274803 2024-04-07T16:16:14.347 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:14.347 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.347 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:meta": { 2024-04-07T16:16:14.347 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T16:16:14.347 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 729808896 2024-04-07T16:16:14.347 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.348 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T16:16:14.348 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.348 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.348 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T16:16:14.348 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.348 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.348 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T16:16:14.348 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 658730852 2024-04-07T16:16:14.348 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.348 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T16:16:14.348 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.348 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.348 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T16:16:14.348 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.348 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.349 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T16:16:14.349 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.349 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.349 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T16:16:14.349 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.349 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.349 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T16:16:14.349 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.349 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.349 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T16:16:14.349 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 241886 2024-04-07T16:16:14.349 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.350 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T16:16:14.350 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.350 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.350 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T16:16:14.350 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.350 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.350 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T16:16:14.350 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.350 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.350 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T16:16:14.350 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 70836158 2024-04-07T16:16:14.350 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:14.350 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.350 INFO:tasks.workunit.client.0.smithi100.stdout: "objecter": { 2024-04-07T16:16:14.351 INFO:tasks.workunit.client.0.smithi100.stdout: "op_active": { 2024-04-07T16:16:14.351 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.351 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.351 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r": { 2024-04-07T16:16:14.351 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.351 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.351 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rmw": { 2024-04-07T16:16:14.351 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.351 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.351 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w": { 2024-04-07T16:16:14.351 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.351 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:14.351 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.351 INFO:tasks.workunit.client.0.smithi100.stdout: "osd": { 2024-04-07T16:16:14.351 INFO:tasks.workunit.client.0.smithi100.stdout: "numpg": { 2024-04-07T16:16:14.352 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 44 2024-04-07T16:16:14.352 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.352 INFO:tasks.workunit.client.0.smithi100.stdout: "numpg_removing": { 2024-04-07T16:16:14.352 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.352 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.352 INFO:tasks.workunit.client.0.smithi100.stdout: "op": { 2024-04-07T16:16:14.352 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5 2024-04-07T16:16:14.352 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.352 INFO:tasks.workunit.client.0.smithi100.stdout: "op_delayed_degraded": { 2024-04-07T16:16:14.352 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.352 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.352 INFO:tasks.workunit.client.0.smithi100.stdout: "op_delayed_unreadable": { 2024-04-07T16:16:14.352 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.352 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.353 INFO:tasks.workunit.client.0.smithi100.stdout: "op_in_bytes": { 2024-04-07T16:16:14.353 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 616 2024-04-07T16:16:14.353 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.353 INFO:tasks.workunit.client.0.smithi100.stdout: "op_latency": { 2024-04-07T16:16:14.353 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 5, 2024-04-07T16:16:14.353 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 77374518 2024-04-07T16:16:14.353 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.353 INFO:tasks.workunit.client.0.smithi100.stdout: "op_out_bytes": { 2024-04-07T16:16:14.353 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.353 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.353 INFO:tasks.workunit.client.0.smithi100.stdout: "op_prepare_latency": { 2024-04-07T16:16:14.353 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 6, 2024-04-07T16:16:14.353 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 9316980 2024-04-07T16:16:14.353 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.354 INFO:tasks.workunit.client.0.smithi100.stdout: "op_process_latency": { 2024-04-07T16:16:14.354 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 5, 2024-04-07T16:16:14.354 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 57164984 2024-04-07T16:16:14.354 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.354 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r": { 2024-04-07T16:16:14.354 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.354 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.354 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_latency": { 2024-04-07T16:16:14.354 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.354 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.354 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.354 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_latency_out_bytes_histogram": { 2024-04-07T16:16:14.354 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.354 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.355 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_out_bytes": { 2024-04-07T16:16:14.355 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.355 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.355 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_prepare_latency": { 2024-04-07T16:16:14.355 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.355 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.355 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.355 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_process_latency": { 2024-04-07T16:16:14.355 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.355 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.355 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.355 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw": { 2024-04-07T16:16:14.355 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.355 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.356 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_in_bytes": { 2024-04-07T16:16:14.356 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.356 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.356 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_latency": { 2024-04-07T16:16:14.356 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.356 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.356 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.356 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_latency_in_bytes_histogram": { 2024-04-07T16:16:14.356 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.356 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.356 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_latency_out_bytes_histogram": { 2024-04-07T16:16:14.356 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.356 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.356 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_out_bytes": { 2024-04-07T16:16:14.357 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.357 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.357 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_prepare_latency": { 2024-04-07T16:16:14.357 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.357 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.357 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.357 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_process_latency": { 2024-04-07T16:16:14.357 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.357 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.357 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.357 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w": { 2024-04-07T16:16:14.357 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5 2024-04-07T16:16:14.357 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.357 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_in_bytes": { 2024-04-07T16:16:14.358 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 616 2024-04-07T16:16:14.358 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.358 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_latency": { 2024-04-07T16:16:14.358 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 5, 2024-04-07T16:16:14.358 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 77374518 2024-04-07T16:16:14.358 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.358 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_latency_in_bytes_histogram": { 2024-04-07T16:16:14.358 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.358 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.358 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_prepare_latency": { 2024-04-07T16:16:14.358 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 6, 2024-04-07T16:16:14.358 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 9316980 2024-04-07T16:16:14.358 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.358 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_process_latency": { 2024-04-07T16:16:14.359 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 5, 2024-04-07T16:16:14.359 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 57164984 2024-04-07T16:16:14.359 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.359 INFO:tasks.workunit.client.0.smithi100.stdout: "op_wip": { 2024-04-07T16:16:14.359 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.359 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.359 INFO:tasks.workunit.client.0.smithi100.stdout: "recovery_bytes": { 2024-04-07T16:16:14.359 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1771104 2024-04-07T16:16:14.359 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.359 INFO:tasks.workunit.client.0.smithi100.stdout: "recovery_ops": { 2024-04-07T16:16:14.359 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 6 2024-04-07T16:16:14.359 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.359 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_bytes": { 2024-04-07T16:16:14.359 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 95995035648 2024-04-07T16:16:14.360 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.360 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_bytes_used": { 2024-04-07T16:16:14.360 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 28987392 2024-04-07T16:16:14.360 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:14.360 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:14.360 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.360 INFO:tasks.workunit.client.0.smithi100.stdout: "osd.1": { 2024-04-07T16:16:14.360 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs": { 2024-04-07T16:16:14.360 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_slow_fallback": { 2024-04-07T16:16:14.360 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.360 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.360 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_slow_size_fallback": { 2024-04-07T16:16:14.360 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.360 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.361 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit_db": { 2024-04-07T16:16:14.361 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 65536 2024-04-07T16:16:14.361 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.361 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit_main": { 2024-04-07T16:16:14.361 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.361 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.361 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit_wal": { 2024-04-07T16:16:14.361 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.361 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.361 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_written_slow": { 2024-04-07T16:16:14.361 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.361 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.361 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_written_sst": { 2024-04-07T16:16:14.362 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 12288 2024-04-07T16:16:14.362 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.362 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_written_wal": { 2024-04-07T16:16:14.362 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2367488 2024-04-07T16:16:14.362 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.362 INFO:tasks.workunit.client.0.smithi100.stdout: "compact_lat": { 2024-04-07T16:16:14.362 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.362 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.362 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.362 INFO:tasks.workunit.client.0.smithi100.stdout: "compact_lock_lat": { 2024-04-07T16:16:14.362 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.363 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.363 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.363 INFO:tasks.workunit.client.0.smithi100.stdout: "db_total_bytes": { 2024-04-07T16:16:14.363 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 95995035648 2024-04-07T16:16:14.363 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.363 INFO:tasks.workunit.client.0.smithi100.stdout: "db_used_bytes": { 2024-04-07T16:16:14.363 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 27656192 2024-04-07T16:16:14.363 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.363 INFO:tasks.workunit.client.0.smithi100.stdout: "log_bytes": { 2024-04-07T16:16:14.363 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1499136 2024-04-07T16:16:14.363 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.363 INFO:tasks.workunit.client.0.smithi100.stdout: "logged_bytes": { 2024-04-07T16:16:14.363 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1359872 2024-04-07T16:16:14.364 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.364 INFO:tasks.workunit.client.0.smithi100.stdout: "max_bytes_db": { 2024-04-07T16:16:14.364 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 31850496 2024-04-07T16:16:14.364 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.364 INFO:tasks.workunit.client.0.smithi100.stdout: "max_bytes_slow": { 2024-04-07T16:16:14.364 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.364 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.364 INFO:tasks.workunit.client.0.smithi100.stdout: "max_bytes_wal": { 2024-04-07T16:16:14.364 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.364 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.364 INFO:tasks.workunit.client.0.smithi100.stdout: "num_files": { 2024-04-07T16:16:14.364 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 14 2024-04-07T16:16:14.365 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.365 INFO:tasks.workunit.client.0.smithi100.stdout: "read_bytes": { 2024-04-07T16:16:14.365 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 206272 2024-04-07T16:16:14.365 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.365 INFO:tasks.workunit.client.0.smithi100.stdout: "read_count": { 2024-04-07T16:16:14.365 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 23 2024-04-07T16:16:14.365 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.365 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes": { 2024-04-07T16:16:14.365 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1282048 2024-04-07T16:16:14.365 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.365 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes_db": { 2024-04-07T16:16:14.365 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1286144 2024-04-07T16:16:14.365 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.365 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes_slow": { 2024-04-07T16:16:14.366 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.366 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.366 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes_wal": { 2024-04-07T16:16:14.366 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.366 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.366 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_count": { 2024-04-07T16:16:14.366 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 15 2024-04-07T16:16:14.366 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.366 INFO:tasks.workunit.client.0.smithi100.stdout: "read_prefetch_bytes": { 2024-04-07T16:16:14.366 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4913 2024-04-07T16:16:14.366 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.366 INFO:tasks.workunit.client.0.smithi100.stdout: "read_prefetch_count": { 2024-04-07T16:16:14.366 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4 2024-04-07T16:16:14.366 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.366 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_buffer_bytes": { 2024-04-07T16:16:14.367 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5505 2024-04-07T16:16:14.367 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.367 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_buffer_count": { 2024-04-07T16:16:14.367 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 24 2024-04-07T16:16:14.367 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.367 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_bytes": { 2024-04-07T16:16:14.367 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5505 2024-04-07T16:16:14.367 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.367 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_count": { 2024-04-07T16:16:14.367 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 24 2024-04-07T16:16:14.367 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.367 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes": { 2024-04-07T16:16:14.367 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.367 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.368 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes_db": { 2024-04-07T16:16:14.368 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.368 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.368 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes_slow": { 2024-04-07T16:16:14.368 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.368 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.368 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes_wal": { 2024-04-07T16:16:14.368 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.368 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.368 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_count": { 2024-04-07T16:16:14.368 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.368 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.368 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_total_bytes": { 2024-04-07T16:16:14.368 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.368 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.369 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_used_bytes": { 2024-04-07T16:16:14.369 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.369 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.369 INFO:tasks.workunit.client.0.smithi100.stdout: "wal_total_bytes": { 2024-04-07T16:16:14.369 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.369 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.369 INFO:tasks.workunit.client.0.smithi100.stdout: "wal_used_bytes": { 2024-04-07T16:16:14.369 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.369 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.369 INFO:tasks.workunit.client.0.smithi100.stdout: "write_bytes": { 2024-04-07T16:16:14.369 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3805184 2024-04-07T16:16:14.369 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:14.370 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.370 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore": { 2024-04-07T16:16:14.370 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit": { 2024-04-07T16:16:14.370 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4096 2024-04-07T16:16:14.370 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.370 INFO:tasks.workunit.client.0.smithi100.stdout: "allocated": { 2024-04-07T16:16:14.370 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 716800 2024-04-07T16:16:14.370 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.370 INFO:tasks.workunit.client.0.smithi100.stdout: "clist_lat": { 2024-04-07T16:16:14.370 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 5, 2024-04-07T16:16:14.370 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 341315 2024-04-07T16:16:14.370 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.370 INFO:tasks.workunit.client.0.smithi100.stdout: "compress_lat": { 2024-04-07T16:16:14.370 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.371 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.371 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.371 INFO:tasks.workunit.client.0.smithi100.stdout: "compressed": { 2024-04-07T16:16:14.371 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.371 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.371 INFO:tasks.workunit.client.0.smithi100.stdout: "compressed_allocated": { 2024-04-07T16:16:14.371 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.371 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.371 INFO:tasks.workunit.client.0.smithi100.stdout: "compressed_original": { 2024-04-07T16:16:14.371 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.371 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.371 INFO:tasks.workunit.client.0.smithi100.stdout: "csum_lat": { 2024-04-07T16:16:14.371 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 31, 2024-04-07T16:16:14.371 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 131637 2024-04-07T16:16:14.371 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.372 INFO:tasks.workunit.client.0.smithi100.stdout: "decompress_lat": { 2024-04-07T16:16:14.372 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.372 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.372 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.372 INFO:tasks.workunit.client.0.smithi100.stdout: "fragmentation_micros": { 2024-04-07T16:16:14.372 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.372 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.372 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_commit_lat": { 2024-04-07T16:16:14.372 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 319, 2024-04-07T16:16:14.372 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 102343784 2024-04-07T16:16:14.372 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.372 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_final_lat": { 2024-04-07T16:16:14.372 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 318, 2024-04-07T16:16:14.373 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 9983561 2024-04-07T16:16:14.373 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.373 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_flush_lat": { 2024-04-07T16:16:14.373 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 319, 2024-04-07T16:16:14.373 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 701088 2024-04-07T16:16:14.373 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.373 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_sync_lat": { 2024-04-07T16:16:14.373 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 319, 2024-04-07T16:16:14.373 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 103044872 2024-04-07T16:16:14.373 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.373 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_get_keys_lat": { 2024-04-07T16:16:14.373 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.373 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.373 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.374 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_get_values_lat": { 2024-04-07T16:16:14.374 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 3, 2024-04-07T16:16:14.374 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 57225 2024-04-07T16:16:14.374 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.374 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_lower_bound_lat": { 2024-04-07T16:16:14.374 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 2, 2024-04-07T16:16:14.374 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 11753 2024-04-07T16:16:14.374 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.374 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_next_lat": { 2024-04-07T16:16:14.374 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.374 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.374 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.374 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_seek_to_first_lat": { 2024-04-07T16:16:14.374 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.375 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.375 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.375 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_upper_bound_lat": { 2024-04-07T16:16:14.375 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 2, 2024-04-07T16:16:14.375 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 7756 2024-04-07T16:16:14.375 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.375 INFO:tasks.workunit.client.0.smithi100.stdout: "onode_hits": { 2024-04-07T16:16:14.375 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 807 2024-04-07T16:16:14.375 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.375 INFO:tasks.workunit.client.0.smithi100.stdout: "onode_misses": { 2024-04-07T16:16:14.375 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 232 2024-04-07T16:16:14.375 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.375 INFO:tasks.workunit.client.0.smithi100.stdout: "read_lat": { 2024-04-07T16:16:14.375 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 137, 2024-04-07T16:16:14.376 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 7601666 2024-04-07T16:16:14.376 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.376 INFO:tasks.workunit.client.0.smithi100.stdout: "read_onode_meta_lat": { 2024-04-07T16:16:14.376 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 354, 2024-04-07T16:16:14.376 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 588807 2024-04-07T16:16:14.376 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.376 INFO:tasks.workunit.client.0.smithi100.stdout: "read_wait_aio_lat": { 2024-04-07T16:16:14.376 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 217, 2024-04-07T16:16:14.376 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 6809857 2024-04-07T16:16:14.376 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.376 INFO:tasks.workunit.client.0.smithi100.stdout: "reads_with_retries": { 2024-04-07T16:16:14.376 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.377 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.377 INFO:tasks.workunit.client.0.smithi100.stdout: "remove_lat": { 2024-04-07T16:16:14.377 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 3, 2024-04-07T16:16:14.377 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 227318 2024-04-07T16:16:14.377 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.377 INFO:tasks.workunit.client.0.smithi100.stdout: "state_aio_wait_lat": { 2024-04-07T16:16:14.377 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 402, 2024-04-07T16:16:14.377 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 8680922 2024-04-07T16:16:14.377 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.377 INFO:tasks.workunit.client.0.smithi100.stdout: "state_deferred_aio_wait_lat": { 2024-04-07T16:16:14.377 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 120, 2024-04-07T16:16:14.378 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 26785405 2024-04-07T16:16:14.378 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.378 INFO:tasks.workunit.client.0.smithi100.stdout: "state_deferred_cleanup_lat": { 2024-04-07T16:16:14.378 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 119, 2024-04-07T16:16:14.378 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 571827153179 2024-04-07T16:16:14.378 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.378 INFO:tasks.workunit.client.0.smithi100.stdout: "state_deferred_queued_lat": { 2024-04-07T16:16:14.378 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 120, 2024-04-07T16:16:14.378 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 161506863344 2024-04-07T16:16:14.378 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.378 INFO:tasks.workunit.client.0.smithi100.stdout: "state_done_lat": { 2024-04-07T16:16:14.379 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 401, 2024-04-07T16:16:14.379 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 82819121573 2024-04-07T16:16:14.379 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.379 INFO:tasks.workunit.client.0.smithi100.stdout: "state_finishing_lat": { 2024-04-07T16:16:14.379 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 401, 2024-04-07T16:16:14.379 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 91444 2024-04-07T16:16:14.379 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.379 INFO:tasks.workunit.client.0.smithi100.stdout: "state_io_done_lat": { 2024-04-07T16:16:14.379 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 402, 2024-04-07T16:16:14.379 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 718729 2024-04-07T16:16:14.380 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.380 INFO:tasks.workunit.client.0.smithi100.stdout: "state_kv_commiting_lat": { 2024-04-07T16:16:14.380 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 402, 2024-04-07T16:16:14.380 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 184174016 2024-04-07T16:16:14.380 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.380 INFO:tasks.workunit.client.0.smithi100.stdout: "state_kv_done_lat": { 2024-04-07T16:16:14.380 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 402, 2024-04-07T16:16:14.380 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 225055 2024-04-07T16:16:14.380 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.381 INFO:tasks.workunit.client.0.smithi100.stdout: "state_kv_queued_lat": { 2024-04-07T16:16:14.381 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 402, 2024-04-07T16:16:14.381 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 106892522 2024-04-07T16:16:14.381 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.381 INFO:tasks.workunit.client.0.smithi100.stdout: "state_prepare_lat": { 2024-04-07T16:16:14.381 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 402, 2024-04-07T16:16:14.382 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 32108553 2024-04-07T16:16:14.382 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.382 INFO:tasks.workunit.client.0.smithi100.stdout: "stored": { 2024-04-07T16:16:14.382 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 309392 2024-04-07T16:16:14.382 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.382 INFO:tasks.workunit.client.0.smithi100.stdout: "truncate_lat": { 2024-04-07T16:16:14.382 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 54, 2024-04-07T16:16:14.382 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 542793 2024-04-07T16:16:14.382 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.382 INFO:tasks.workunit.client.0.smithi100.stdout: "txc_commit_lat": { 2024-04-07T16:16:14.383 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 402, 2024-04-07T16:16:14.383 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 332673277 2024-04-07T16:16:14.383 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.383 INFO:tasks.workunit.client.0.smithi100.stdout: "txc_submit_lat": { 2024-04-07T16:16:14.383 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 402, 2024-04-07T16:16:14.383 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 38729789 2024-04-07T16:16:14.383 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.383 INFO:tasks.workunit.client.0.smithi100.stdout: "txc_throttle_lat": { 2024-04-07T16:16:14.383 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 402, 2024-04-07T16:16:14.383 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3872843 2024-04-07T16:16:14.383 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:14.383 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.383 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache": { 2024-04-07T16:16:14.383 INFO:tasks.workunit.client.0.smithi100.stdout: "cache_bytes": { 2024-04-07T16:16:14.384 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1900954368 2024-04-07T16:16:14.384 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.384 INFO:tasks.workunit.client.0.smithi100.stdout: "heap_bytes": { 2024-04-07T16:16:14.384 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 44482560 2024-04-07T16:16:14.384 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.384 INFO:tasks.workunit.client.0.smithi100.stdout: "mapped_bytes": { 2024-04-07T16:16:14.384 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 44007424 2024-04-07T16:16:14.384 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.384 INFO:tasks.workunit.client.0.smithi100.stdout: "target_bytes": { 2024-04-07T16:16:14.384 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3183836160 2024-04-07T16:16:14.384 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.384 INFO:tasks.workunit.client.0.smithi100.stdout: "unmapped_bytes": { 2024-04-07T16:16:14.384 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 475136 2024-04-07T16:16:14.384 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:14.385 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.385 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:data": { 2024-04-07T16:16:14.385 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T16:16:14.385 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 159383552 2024-04-07T16:16:14.385 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.385 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T16:16:14.385 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.385 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.385 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T16:16:14.385 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.385 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.385 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T16:16:14.385 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 87868101 2024-04-07T16:16:14.385 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.385 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T16:16:14.385 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.386 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.386 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T16:16:14.386 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.386 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.386 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T16:16:14.386 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.386 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.386 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T16:16:14.386 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.386 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.386 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T16:16:14.386 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.386 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.387 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T16:16:14.387 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 16384 2024-04-07T16:16:14.387 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.387 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T16:16:14.387 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.387 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.387 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T16:16:14.387 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.387 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.387 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T16:16:14.387 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.387 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.387 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T16:16:14.387 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 71499067 2024-04-07T16:16:14.387 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:14.388 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.388 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:kv": { 2024-04-07T16:16:14.388 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T16:16:14.388 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 796917760 2024-04-07T16:16:14.388 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.388 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T16:16:14.388 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 67108864 2024-04-07T16:16:14.388 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.388 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T16:16:14.388 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.388 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.388 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T16:16:14.388 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 659010759 2024-04-07T16:16:14.388 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.389 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T16:16:14.389 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.389 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.389 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T16:16:14.389 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.389 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.389 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T16:16:14.389 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.389 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.389 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T16:16:14.389 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.389 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.389 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T16:16:14.389 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.389 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.389 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T16:16:14.390 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2128 2024-04-07T16:16:14.390 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.390 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T16:16:14.390 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.390 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.390 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T16:16:14.390 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.390 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.390 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T16:16:14.390 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.390 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.390 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T16:16:14.390 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 70796009 2024-04-07T16:16:14.391 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:14.391 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.391 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:kv_onode": { 2024-04-07T16:16:14.391 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T16:16:14.391 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 192937984 2024-04-07T16:16:14.391 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.391 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T16:16:14.391 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 67108864 2024-04-07T16:16:14.391 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.391 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T16:16:14.391 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 464 2024-04-07T16:16:14.391 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.391 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T16:16:14.392 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 58578734 2024-04-07T16:16:14.392 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.392 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T16:16:14.392 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.392 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.392 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T16:16:14.392 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.392 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.392 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T16:16:14.392 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.392 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.392 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T16:16:14.392 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.392 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.393 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T16:16:14.393 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.393 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.393 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T16:16:14.393 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.393 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.393 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T16:16:14.393 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.393 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.393 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T16:16:14.393 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.393 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.393 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T16:16:14.393 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.393 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.394 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T16:16:14.394 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 67249922 2024-04-07T16:16:14.394 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:14.394 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.394 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:meta": { 2024-04-07T16:16:14.394 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T16:16:14.394 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 729808896 2024-04-07T16:16:14.394 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.394 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T16:16:14.394 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.394 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.394 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T16:16:14.394 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.395 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.395 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T16:16:14.395 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 659010759 2024-04-07T16:16:14.395 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.395 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T16:16:14.395 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.395 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.396 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T16:16:14.396 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.396 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.396 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T16:16:14.396 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.396 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.396 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T16:16:14.396 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.396 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.396 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T16:16:14.396 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.396 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.396 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T16:16:14.397 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 219329 2024-04-07T16:16:14.397 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.397 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T16:16:14.397 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.397 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.397 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T16:16:14.397 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.397 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.397 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T16:16:14.397 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.397 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.397 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T16:16:14.398 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 70578808 2024-04-07T16:16:14.398 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:14.398 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.398 INFO:tasks.workunit.client.0.smithi100.stdout: "objecter": { 2024-04-07T16:16:14.398 INFO:tasks.workunit.client.0.smithi100.stdout: "op_active": { 2024-04-07T16:16:14.398 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.398 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.398 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r": { 2024-04-07T16:16:14.398 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.398 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.398 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rmw": { 2024-04-07T16:16:14.398 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.398 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.398 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w": { 2024-04-07T16:16:14.399 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.399 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:14.399 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.399 INFO:tasks.workunit.client.0.smithi100.stdout: "osd": { 2024-04-07T16:16:14.399 INFO:tasks.workunit.client.0.smithi100.stdout: "numpg": { 2024-04-07T16:16:14.399 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 38 2024-04-07T16:16:14.399 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.399 INFO:tasks.workunit.client.0.smithi100.stdout: "numpg_removing": { 2024-04-07T16:16:14.399 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.399 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.399 INFO:tasks.workunit.client.0.smithi100.stdout: "op": { 2024-04-07T16:16:14.400 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 155 2024-04-07T16:16:14.400 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.400 INFO:tasks.workunit.client.0.smithi100.stdout: "op_delayed_degraded": { 2024-04-07T16:16:14.400 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.400 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.400 INFO:tasks.workunit.client.0.smithi100.stdout: "op_delayed_unreadable": { 2024-04-07T16:16:14.400 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.400 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.400 INFO:tasks.workunit.client.0.smithi100.stdout: "op_in_bytes": { 2024-04-07T16:16:14.400 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2429440 2024-04-07T16:16:14.400 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.400 INFO:tasks.workunit.client.0.smithi100.stdout: "op_latency": { 2024-04-07T16:16:14.400 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 155, 2024-04-07T16:16:14.400 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 823968569 2024-04-07T16:16:14.400 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.401 INFO:tasks.workunit.client.0.smithi100.stdout: "op_out_bytes": { 2024-04-07T16:16:14.401 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1063 2024-04-07T16:16:14.401 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.401 INFO:tasks.workunit.client.0.smithi100.stdout: "op_prepare_latency": { 2024-04-07T16:16:14.401 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 155, 2024-04-07T16:16:14.401 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 89196798 2024-04-07T16:16:14.401 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.401 INFO:tasks.workunit.client.0.smithi100.stdout: "op_process_latency": { 2024-04-07T16:16:14.401 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 155, 2024-04-07T16:16:14.401 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 307014042 2024-04-07T16:16:14.401 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.401 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r": { 2024-04-07T16:16:14.401 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 33 2024-04-07T16:16:14.401 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.402 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_latency": { 2024-04-07T16:16:14.402 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 33, 2024-04-07T16:16:14.402 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 12714834 2024-04-07T16:16:14.402 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.402 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_latency_out_bytes_histogram": { 2024-04-07T16:16:14.402 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.402 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.402 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_out_bytes": { 2024-04-07T16:16:14.402 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1063 2024-04-07T16:16:14.402 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.402 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_prepare_latency": { 2024-04-07T16:16:14.402 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 33, 2024-04-07T16:16:14.402 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 11164911 2024-04-07T16:16:14.402 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.403 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_process_latency": { 2024-04-07T16:16:14.403 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 33, 2024-04-07T16:16:14.403 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 9726155 2024-04-07T16:16:14.403 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.403 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw": { 2024-04-07T16:16:14.403 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 22 2024-04-07T16:16:14.403 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.403 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_in_bytes": { 2024-04-07T16:16:14.403 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.403 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.403 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_latency": { 2024-04-07T16:16:14.403 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 22, 2024-04-07T16:16:14.403 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 36605744 2024-04-07T16:16:14.403 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.404 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_latency_in_bytes_histogram": { 2024-04-07T16:16:14.404 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.404 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.404 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_latency_out_bytes_histogram": { 2024-04-07T16:16:14.404 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.404 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.404 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_out_bytes": { 2024-04-07T16:16:14.404 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.404 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.404 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_prepare_latency": { 2024-04-07T16:16:14.404 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 22, 2024-04-07T16:16:14.404 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 15805848 2024-04-07T16:16:14.404 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.404 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_process_latency": { 2024-04-07T16:16:14.405 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 22, 2024-04-07T16:16:14.405 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 34480353 2024-04-07T16:16:14.405 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.405 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w": { 2024-04-07T16:16:14.405 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 100 2024-04-07T16:16:14.405 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.405 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_in_bytes": { 2024-04-07T16:16:14.405 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2429440 2024-04-07T16:16:14.405 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.405 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_latency": { 2024-04-07T16:16:14.405 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 100, 2024-04-07T16:16:14.405 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 774647991 2024-04-07T16:16:14.405 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.405 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_latency_in_bytes_histogram": { 2024-04-07T16:16:14.405 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.406 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.406 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_prepare_latency": { 2024-04-07T16:16:14.406 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 100, 2024-04-07T16:16:14.406 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 62226039 2024-04-07T16:16:14.406 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.406 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_process_latency": { 2024-04-07T16:16:14.406 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 100, 2024-04-07T16:16:14.406 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 262807534 2024-04-07T16:16:14.406 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.406 INFO:tasks.workunit.client.0.smithi100.stdout: "op_wip": { 2024-04-07T16:16:14.406 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.406 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.406 INFO:tasks.workunit.client.0.smithi100.stdout: "recovery_bytes": { 2024-04-07T16:16:14.407 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 590368 2024-04-07T16:16:14.407 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.407 INFO:tasks.workunit.client.0.smithi100.stdout: "recovery_ops": { 2024-04-07T16:16:14.407 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2 2024-04-07T16:16:14.407 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.407 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_bytes": { 2024-04-07T16:16:14.407 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 95995035648 2024-04-07T16:16:14.407 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.407 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_bytes_used": { 2024-04-07T16:16:14.407 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 28385280 2024-04-07T16:16:14.407 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:14.407 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:14.407 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.407 INFO:tasks.workunit.client.0.smithi100.stdout: "osd.2": { 2024-04-07T16:16:14.407 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs": { 2024-04-07T16:16:14.408 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_slow_fallback": { 2024-04-07T16:16:14.408 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.408 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.408 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_slow_size_fallback": { 2024-04-07T16:16:14.408 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.408 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.408 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit_db": { 2024-04-07T16:16:14.408 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 65536 2024-04-07T16:16:14.408 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.408 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit_main": { 2024-04-07T16:16:14.408 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.408 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.408 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit_wal": { 2024-04-07T16:16:14.408 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.409 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.409 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_written_slow": { 2024-04-07T16:16:14.409 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.409 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.409 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_written_sst": { 2024-04-07T16:16:14.409 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 12288 2024-04-07T16:16:14.409 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.409 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_written_wal": { 2024-04-07T16:16:14.409 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 7000064 2024-04-07T16:16:14.409 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.409 INFO:tasks.workunit.client.0.smithi100.stdout: "compact_lat": { 2024-04-07T16:16:14.409 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.409 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.409 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.409 INFO:tasks.workunit.client.0.smithi100.stdout: "compact_lock_lat": { 2024-04-07T16:16:14.410 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.410 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.410 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.410 INFO:tasks.workunit.client.0.smithi100.stdout: "db_total_bytes": { 2024-04-07T16:16:14.410 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 95995035648 2024-04-07T16:16:14.410 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.410 INFO:tasks.workunit.client.0.smithi100.stdout: "db_used_bytes": { 2024-04-07T16:16:14.410 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 31850496 2024-04-07T16:16:14.410 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.410 INFO:tasks.workunit.client.0.smithi100.stdout: "log_bytes": { 2024-04-07T16:16:14.410 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3526656 2024-04-07T16:16:14.410 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.410 INFO:tasks.workunit.client.0.smithi100.stdout: "logged_bytes": { 2024-04-07T16:16:14.411 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3387392 2024-04-07T16:16:14.411 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.411 INFO:tasks.workunit.client.0.smithi100.stdout: "max_bytes_db": { 2024-04-07T16:16:14.411 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 31850496 2024-04-07T16:16:14.411 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.411 INFO:tasks.workunit.client.0.smithi100.stdout: "max_bytes_slow": { 2024-04-07T16:16:14.411 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.411 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.411 INFO:tasks.workunit.client.0.smithi100.stdout: "max_bytes_wal": { 2024-04-07T16:16:14.411 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.411 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.411 INFO:tasks.workunit.client.0.smithi100.stdout: "num_files": { 2024-04-07T16:16:14.411 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 14 2024-04-07T16:16:14.411 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.412 INFO:tasks.workunit.client.0.smithi100.stdout: "read_bytes": { 2024-04-07T16:16:14.412 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 206272 2024-04-07T16:16:14.412 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.412 INFO:tasks.workunit.client.0.smithi100.stdout: "read_count": { 2024-04-07T16:16:14.412 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 23 2024-04-07T16:16:14.412 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.412 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes": { 2024-04-07T16:16:14.412 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1282048 2024-04-07T16:16:14.412 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.412 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes_db": { 2024-04-07T16:16:14.412 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1286144 2024-04-07T16:16:14.412 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.412 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes_slow": { 2024-04-07T16:16:14.413 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.413 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.413 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes_wal": { 2024-04-07T16:16:14.413 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.413 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.413 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_count": { 2024-04-07T16:16:14.413 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 15 2024-04-07T16:16:14.413 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.413 INFO:tasks.workunit.client.0.smithi100.stdout: "read_prefetch_bytes": { 2024-04-07T16:16:14.413 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4913 2024-04-07T16:16:14.413 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.413 INFO:tasks.workunit.client.0.smithi100.stdout: "read_prefetch_count": { 2024-04-07T16:16:14.413 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4 2024-04-07T16:16:14.414 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.414 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_buffer_bytes": { 2024-04-07T16:16:14.414 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5505 2024-04-07T16:16:14.414 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.414 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_buffer_count": { 2024-04-07T16:16:14.414 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 24 2024-04-07T16:16:14.414 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.414 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_bytes": { 2024-04-07T16:16:14.414 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5505 2024-04-07T16:16:14.414 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.414 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_count": { 2024-04-07T16:16:14.414 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 24 2024-04-07T16:16:14.414 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.415 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes": { 2024-04-07T16:16:14.415 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.415 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.415 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes_db": { 2024-04-07T16:16:14.415 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.415 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.415 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes_slow": { 2024-04-07T16:16:14.415 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.415 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.415 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes_wal": { 2024-04-07T16:16:14.415 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.415 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.415 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_count": { 2024-04-07T16:16:14.415 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.415 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.416 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_total_bytes": { 2024-04-07T16:16:14.416 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.416 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.416 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_used_bytes": { 2024-04-07T16:16:14.416 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.416 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.416 INFO:tasks.workunit.client.0.smithi100.stdout: "wal_total_bytes": { 2024-04-07T16:16:14.416 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.416 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.416 INFO:tasks.workunit.client.0.smithi100.stdout: "wal_used_bytes": { 2024-04-07T16:16:14.416 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.416 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.416 INFO:tasks.workunit.client.0.smithi100.stdout: "write_bytes": { 2024-04-07T16:16:14.416 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 10465280 2024-04-07T16:16:14.417 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:14.417 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.417 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore": { 2024-04-07T16:16:14.417 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit": { 2024-04-07T16:16:14.417 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4096 2024-04-07T16:16:14.417 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.417 INFO:tasks.workunit.client.0.smithi100.stdout: "allocated": { 2024-04-07T16:16:14.417 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 708608 2024-04-07T16:16:14.417 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.417 INFO:tasks.workunit.client.0.smithi100.stdout: "clist_lat": { 2024-04-07T16:16:14.417 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 4, 2024-04-07T16:16:14.417 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 541920 2024-04-07T16:16:14.417 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.417 INFO:tasks.workunit.client.0.smithi100.stdout: "compress_lat": { 2024-04-07T16:16:14.418 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.418 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.418 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.418 INFO:tasks.workunit.client.0.smithi100.stdout: "compressed": { 2024-04-07T16:16:14.418 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.418 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.418 INFO:tasks.workunit.client.0.smithi100.stdout: "compressed_allocated": { 2024-04-07T16:16:14.418 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.418 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.418 INFO:tasks.workunit.client.0.smithi100.stdout: "compressed_original": { 2024-04-07T16:16:14.418 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.418 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.418 INFO:tasks.workunit.client.0.smithi100.stdout: "csum_lat": { 2024-04-07T16:16:14.419 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 10, 2024-04-07T16:16:14.419 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 16574 2024-04-07T16:16:14.419 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.419 INFO:tasks.workunit.client.0.smithi100.stdout: "decompress_lat": { 2024-04-07T16:16:14.419 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.419 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.419 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.419 INFO:tasks.workunit.client.0.smithi100.stdout: "fragmentation_micros": { 2024-04-07T16:16:14.419 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.419 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.419 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_commit_lat": { 2024-04-07T16:16:14.419 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 814, 2024-04-07T16:16:14.419 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 532460835 2024-04-07T16:16:14.419 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.420 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_final_lat": { 2024-04-07T16:16:14.420 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 809, 2024-04-07T16:16:14.420 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 29164356 2024-04-07T16:16:14.420 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.420 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_flush_lat": { 2024-04-07T16:16:14.420 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 814, 2024-04-07T16:16:14.420 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1352905 2024-04-07T16:16:14.420 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.420 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_sync_lat": { 2024-04-07T16:16:14.420 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 814, 2024-04-07T16:16:14.420 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 533813740 2024-04-07T16:16:14.420 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.420 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_get_keys_lat": { 2024-04-07T16:16:14.421 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.421 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.421 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.421 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_get_values_lat": { 2024-04-07T16:16:14.421 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 3, 2024-04-07T16:16:14.421 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 66261 2024-04-07T16:16:14.421 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.421 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_lower_bound_lat": { 2024-04-07T16:16:14.421 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.421 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.421 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.421 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_next_lat": { 2024-04-07T16:16:14.421 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.422 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.422 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.422 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_seek_to_first_lat": { 2024-04-07T16:16:14.422 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.422 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.423 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.423 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_upper_bound_lat": { 2024-04-07T16:16:14.423 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 2, 2024-04-07T16:16:14.423 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 10313 2024-04-07T16:16:14.423 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.423 INFO:tasks.workunit.client.0.smithi100.stdout: "onode_hits": { 2024-04-07T16:16:14.423 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3692 2024-04-07T16:16:14.424 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.424 INFO:tasks.workunit.client.0.smithi100.stdout: "onode_misses": { 2024-04-07T16:16:14.424 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 312 2024-04-07T16:16:14.424 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.424 INFO:tasks.workunit.client.0.smithi100.stdout: "read_lat": { 2024-04-07T16:16:14.424 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 97, 2024-04-07T16:16:14.424 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1919506 2024-04-07T16:16:14.425 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.425 INFO:tasks.workunit.client.0.smithi100.stdout: "read_onode_meta_lat": { 2024-04-07T16:16:14.425 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 268, 2024-04-07T16:16:14.425 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 467195 2024-04-07T16:16:14.425 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.425 INFO:tasks.workunit.client.0.smithi100.stdout: "read_wait_aio_lat": { 2024-04-07T16:16:14.425 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 171, 2024-04-07T16:16:14.426 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4487537 2024-04-07T16:16:14.426 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.426 INFO:tasks.workunit.client.0.smithi100.stdout: "reads_with_retries": { 2024-04-07T16:16:14.426 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.426 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.426 INFO:tasks.workunit.client.0.smithi100.stdout: "remove_lat": { 2024-04-07T16:16:14.426 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 103, 2024-04-07T16:16:14.426 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 6376664 2024-04-07T16:16:14.426 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.426 INFO:tasks.workunit.client.0.smithi100.stdout: "state_aio_wait_lat": { 2024-04-07T16:16:14.426 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 3432, 2024-04-07T16:16:14.426 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4037452882 2024-04-07T16:16:14.426 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.427 INFO:tasks.workunit.client.0.smithi100.stdout: "state_deferred_aio_wait_lat": { 2024-04-07T16:16:14.427 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 114, 2024-04-07T16:16:14.427 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 28436992 2024-04-07T16:16:14.427 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.427 INFO:tasks.workunit.client.0.smithi100.stdout: "state_deferred_cleanup_lat": { 2024-04-07T16:16:14.427 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 113, 2024-04-07T16:16:14.427 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 784322434793 2024-04-07T16:16:14.427 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.427 INFO:tasks.workunit.client.0.smithi100.stdout: "state_deferred_queued_lat": { 2024-04-07T16:16:14.427 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 114, 2024-04-07T16:16:14.427 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 128648915339 2024-04-07T16:16:14.427 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.427 INFO:tasks.workunit.client.0.smithi100.stdout: "state_done_lat": { 2024-04-07T16:16:14.427 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 3431, 2024-04-07T16:16:14.428 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 192843779440 2024-04-07T16:16:14.428 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.428 INFO:tasks.workunit.client.0.smithi100.stdout: "state_finishing_lat": { 2024-04-07T16:16:14.428 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 3431, 2024-04-07T16:16:14.428 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 277568 2024-04-07T16:16:14.428 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.428 INFO:tasks.workunit.client.0.smithi100.stdout: "state_io_done_lat": { 2024-04-07T16:16:14.428 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 3432, 2024-04-07T16:16:14.428 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 485004891 2024-04-07T16:16:14.428 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.428 INFO:tasks.workunit.client.0.smithi100.stdout: "state_kv_commiting_lat": { 2024-04-07T16:16:14.428 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 3432, 2024-04-07T16:16:14.428 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3506235439 2024-04-07T16:16:14.428 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.429 INFO:tasks.workunit.client.0.smithi100.stdout: "state_kv_done_lat": { 2024-04-07T16:16:14.429 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 3432, 2024-04-07T16:16:14.429 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 657511 2024-04-07T16:16:14.429 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.429 INFO:tasks.workunit.client.0.smithi100.stdout: "state_kv_queued_lat": { 2024-04-07T16:16:14.429 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 3432, 2024-04-07T16:16:14.429 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2061285108 2024-04-07T16:16:14.429 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.429 INFO:tasks.workunit.client.0.smithi100.stdout: "state_prepare_lat": { 2024-04-07T16:16:14.429 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 3432, 2024-04-07T16:16:14.429 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 367118666 2024-04-07T16:16:14.429 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.429 INFO:tasks.workunit.client.0.smithi100.stdout: "stored": { 2024-04-07T16:16:14.429 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 309232 2024-04-07T16:16:14.429 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.430 INFO:tasks.workunit.client.0.smithi100.stdout: "truncate_lat": { 2024-04-07T16:16:14.430 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 48, 2024-04-07T16:16:14.430 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 387879 2024-04-07T16:16:14.430 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.430 INFO:tasks.workunit.client.0.smithi100.stdout: "txc_commit_lat": { 2024-04-07T16:16:14.430 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 3432, 2024-04-07T16:16:14.430 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 10457408332 2024-04-07T16:16:14.430 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.430 INFO:tasks.workunit.client.0.smithi100.stdout: "txc_submit_lat": { 2024-04-07T16:16:14.430 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 3432, 2024-04-07T16:16:14.430 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 439673087 2024-04-07T16:16:14.430 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.430 INFO:tasks.workunit.client.0.smithi100.stdout: "txc_throttle_lat": { 2024-04-07T16:16:14.431 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 3432, 2024-04-07T16:16:14.431 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 197209404 2024-04-07T16:16:14.431 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:14.431 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.431 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache": { 2024-04-07T16:16:14.431 INFO:tasks.workunit.client.0.smithi100.stdout: "cache_bytes": { 2024-04-07T16:16:14.431 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1900954368 2024-04-07T16:16:14.431 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.431 INFO:tasks.workunit.client.0.smithi100.stdout: "heap_bytes": { 2024-04-07T16:16:14.431 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 48676864 2024-04-07T16:16:14.431 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.432 INFO:tasks.workunit.client.0.smithi100.stdout: "mapped_bytes": { 2024-04-07T16:16:14.432 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 48054272 2024-04-07T16:16:14.432 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.432 INFO:tasks.workunit.client.0.smithi100.stdout: "target_bytes": { 2024-04-07T16:16:14.432 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3183836160 2024-04-07T16:16:14.432 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.432 INFO:tasks.workunit.client.0.smithi100.stdout: "unmapped_bytes": { 2024-04-07T16:16:14.432 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 622592 2024-04-07T16:16:14.432 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:14.432 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.432 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:data": { 2024-04-07T16:16:14.433 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T16:16:14.433 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 159383552 2024-04-07T16:16:14.433 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.433 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T16:16:14.433 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.433 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.433 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T16:16:14.433 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.433 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.433 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T16:16:14.433 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 87862818 2024-04-07T16:16:14.433 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.433 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T16:16:14.434 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.434 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.434 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T16:16:14.434 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.434 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.434 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T16:16:14.434 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.434 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.434 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T16:16:14.435 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.435 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.435 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T16:16:14.435 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.435 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.435 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T16:16:14.435 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 8192 2024-04-07T16:16:14.435 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.435 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T16:16:14.435 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.435 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.435 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T16:16:14.435 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.435 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.436 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T16:16:14.436 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.436 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.436 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T16:16:14.436 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 71512542 2024-04-07T16:16:14.436 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:14.436 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.436 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:kv": { 2024-04-07T16:16:14.436 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T16:16:14.436 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 796917760 2024-04-07T16:16:14.436 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.436 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T16:16:14.436 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 67108864 2024-04-07T16:16:14.436 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.437 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T16:16:14.437 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.437 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.437 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T16:16:14.437 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 658971135 2024-04-07T16:16:14.437 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.437 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T16:16:14.437 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.437 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.437 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T16:16:14.437 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.437 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.437 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T16:16:14.438 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.438 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.438 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T16:16:14.438 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.438 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.438 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T16:16:14.438 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.438 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.438 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T16:16:14.438 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2128 2024-04-07T16:16:14.438 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.438 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T16:16:14.438 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.438 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.439 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T16:16:14.439 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.439 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.439 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T16:16:14.439 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.439 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.439 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T16:16:14.439 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 70835633 2024-04-07T16:16:14.439 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:14.439 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.440 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:kv_onode": { 2024-04-07T16:16:14.440 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T16:16:14.440 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 192937984 2024-04-07T16:16:14.440 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.440 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T16:16:14.440 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 67108864 2024-04-07T16:16:14.440 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.441 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T16:16:14.441 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 464 2024-04-07T16:16:14.441 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.441 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T16:16:14.441 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 58575212 2024-04-07T16:16:14.441 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.441 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T16:16:14.441 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.442 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.442 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T16:16:14.442 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.442 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.442 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T16:16:14.442 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.443 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.443 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T16:16:14.443 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.443 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.443 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T16:16:14.443 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.443 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.443 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T16:16:14.443 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.443 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.443 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T16:16:14.443 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.443 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.443 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T16:16:14.443 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.443 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.444 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T16:16:14.444 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.444 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.444 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T16:16:14.444 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 67253444 2024-04-07T16:16:14.444 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:14.444 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.444 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:meta": { 2024-04-07T16:16:14.444 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T16:16:14.445 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 729808896 2024-04-07T16:16:14.445 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.445 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T16:16:14.445 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.445 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.445 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T16:16:14.445 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.445 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.445 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T16:16:14.445 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 658971135 2024-04-07T16:16:14.445 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.446 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T16:16:14.446 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.446 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.446 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T16:16:14.446 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.446 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.446 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T16:16:14.446 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.446 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.446 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T16:16:14.446 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.446 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.446 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T16:16:14.447 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.447 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.447 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T16:16:14.447 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 316782 2024-04-07T16:16:14.447 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.447 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T16:16:14.447 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.447 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.447 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T16:16:14.447 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.447 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.448 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T16:16:14.448 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.448 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.448 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T16:16:14.448 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 70520979 2024-04-07T16:16:14.449 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:14.449 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.449 INFO:tasks.workunit.client.0.smithi100.stdout: "objecter": { 2024-04-07T16:16:14.449 INFO:tasks.workunit.client.0.smithi100.stdout: "op_active": { 2024-04-07T16:16:14.449 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.449 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.450 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r": { 2024-04-07T16:16:14.450 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.450 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.450 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rmw": { 2024-04-07T16:16:14.450 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.450 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.451 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w": { 2024-04-07T16:16:14.451 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.451 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:14.451 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.451 INFO:tasks.workunit.client.0.smithi100.stdout: "osd": { 2024-04-07T16:16:14.451 INFO:tasks.workunit.client.0.smithi100.stdout: "numpg": { 2024-04-07T16:16:14.451 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 30 2024-04-07T16:16:14.451 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.451 INFO:tasks.workunit.client.0.smithi100.stdout: "numpg_removing": { 2024-04-07T16:16:14.451 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.451 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.451 INFO:tasks.workunit.client.0.smithi100.stdout: "op": { 2024-04-07T16:16:14.451 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 10 2024-04-07T16:16:14.451 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.452 INFO:tasks.workunit.client.0.smithi100.stdout: "op_delayed_degraded": { 2024-04-07T16:16:14.452 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.452 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.452 INFO:tasks.workunit.client.0.smithi100.stdout: "op_delayed_unreadable": { 2024-04-07T16:16:14.452 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.452 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.452 INFO:tasks.workunit.client.0.smithi100.stdout: "op_in_bytes": { 2024-04-07T16:16:14.452 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.453 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.453 INFO:tasks.workunit.client.0.smithi100.stdout: "op_latency": { 2024-04-07T16:16:14.453 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 10, 2024-04-07T16:16:14.453 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 238086038 2024-04-07T16:16:14.453 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.453 INFO:tasks.workunit.client.0.smithi100.stdout: "op_out_bytes": { 2024-04-07T16:16:14.453 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.453 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.453 INFO:tasks.workunit.client.0.smithi100.stdout: "op_prepare_latency": { 2024-04-07T16:16:14.453 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 10, 2024-04-07T16:16:14.453 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 21363181 2024-04-07T16:16:14.453 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.453 INFO:tasks.workunit.client.0.smithi100.stdout: "op_process_latency": { 2024-04-07T16:16:14.454 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 10, 2024-04-07T16:16:14.454 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 163225415 2024-04-07T16:16:14.454 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.454 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r": { 2024-04-07T16:16:14.454 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.454 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.454 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_latency": { 2024-04-07T16:16:14.454 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.454 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.454 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.454 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_latency_out_bytes_histogram": { 2024-04-07T16:16:14.455 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.455 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.455 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_out_bytes": { 2024-04-07T16:16:14.455 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.455 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.455 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_prepare_latency": { 2024-04-07T16:16:14.455 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.455 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.455 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.455 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_process_latency": { 2024-04-07T16:16:14.455 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.455 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.455 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.455 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw": { 2024-04-07T16:16:14.455 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.456 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.456 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_in_bytes": { 2024-04-07T16:16:14.456 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.456 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.456 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_latency": { 2024-04-07T16:16:14.456 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.456 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.456 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.456 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_latency_in_bytes_histogram": { 2024-04-07T16:16:14.456 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.456 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.456 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_latency_out_bytes_histogram": { 2024-04-07T16:16:14.456 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.456 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.456 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_out_bytes": { 2024-04-07T16:16:14.457 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.457 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.457 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_prepare_latency": { 2024-04-07T16:16:14.457 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.457 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.457 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.457 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_process_latency": { 2024-04-07T16:16:14.457 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.457 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.457 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.457 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w": { 2024-04-07T16:16:14.457 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 10 2024-04-07T16:16:14.458 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.458 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_in_bytes": { 2024-04-07T16:16:14.458 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.458 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.458 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_latency": { 2024-04-07T16:16:14.458 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 10, 2024-04-07T16:16:14.458 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 238086038 2024-04-07T16:16:14.458 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.458 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_latency_in_bytes_histogram": { 2024-04-07T16:16:14.458 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.458 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.458 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_prepare_latency": { 2024-04-07T16:16:14.458 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 10, 2024-04-07T16:16:14.459 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 21363181 2024-04-07T16:16:14.459 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.459 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_process_latency": { 2024-04-07T16:16:14.459 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 10, 2024-04-07T16:16:14.459 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 163225415 2024-04-07T16:16:14.459 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.459 INFO:tasks.workunit.client.0.smithi100.stdout: "op_wip": { 2024-04-07T16:16:14.459 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.459 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.459 INFO:tasks.workunit.client.0.smithi100.stdout: "recovery_bytes": { 2024-04-07T16:16:14.459 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.459 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.459 INFO:tasks.workunit.client.0.smithi100.stdout: "recovery_ops": { 2024-04-07T16:16:14.460 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.460 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.460 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_bytes": { 2024-04-07T16:16:14.460 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 95995035648 2024-04-07T16:16:14.460 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.460 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_bytes_used": { 2024-04-07T16:16:14.460 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 32571392 2024-04-07T16:16:14.460 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:14.460 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:14.460 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.460 INFO:tasks.workunit.client.0.smithi100.stdout: "osd.3": { 2024-04-07T16:16:14.460 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs": { 2024-04-07T16:16:14.460 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_slow_fallback": { 2024-04-07T16:16:14.460 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.460 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.461 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_slow_size_fallback": { 2024-04-07T16:16:14.461 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.461 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.461 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit_db": { 2024-04-07T16:16:14.461 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 65536 2024-04-07T16:16:14.461 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.461 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit_main": { 2024-04-07T16:16:14.461 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.461 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.461 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit_wal": { 2024-04-07T16:16:14.461 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.461 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.462 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_written_slow": { 2024-04-07T16:16:14.462 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.462 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.462 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_written_sst": { 2024-04-07T16:16:14.462 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 12288 2024-04-07T16:16:14.462 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.462 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_written_wal": { 2024-04-07T16:16:14.462 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1110016 2024-04-07T16:16:14.462 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.462 INFO:tasks.workunit.client.0.smithi100.stdout: "compact_lat": { 2024-04-07T16:16:14.462 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.462 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.463 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.463 INFO:tasks.workunit.client.0.smithi100.stdout: "compact_lock_lat": { 2024-04-07T16:16:14.463 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.463 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.463 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.463 INFO:tasks.workunit.client.0.smithi100.stdout: "db_total_bytes": { 2024-04-07T16:16:14.463 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 95995035648 2024-04-07T16:16:14.463 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.463 INFO:tasks.workunit.client.0.smithi100.stdout: "db_used_bytes": { 2024-04-07T16:16:14.463 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 27656192 2024-04-07T16:16:14.463 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.463 INFO:tasks.workunit.client.0.smithi100.stdout: "log_bytes": { 2024-04-07T16:16:14.463 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 802816 2024-04-07T16:16:14.463 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.464 INFO:tasks.workunit.client.0.smithi100.stdout: "logged_bytes": { 2024-04-07T16:16:14.464 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 663552 2024-04-07T16:16:14.464 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.464 INFO:tasks.workunit.client.0.smithi100.stdout: "max_bytes_db": { 2024-04-07T16:16:14.464 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 31850496 2024-04-07T16:16:14.464 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.464 INFO:tasks.workunit.client.0.smithi100.stdout: "max_bytes_slow": { 2024-04-07T16:16:14.464 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.464 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.464 INFO:tasks.workunit.client.0.smithi100.stdout: "max_bytes_wal": { 2024-04-07T16:16:14.464 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.464 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.464 INFO:tasks.workunit.client.0.smithi100.stdout: "num_files": { 2024-04-07T16:16:14.464 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 14 2024-04-07T16:16:14.465 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.465 INFO:tasks.workunit.client.0.smithi100.stdout: "read_bytes": { 2024-04-07T16:16:14.465 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 206272 2024-04-07T16:16:14.465 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.465 INFO:tasks.workunit.client.0.smithi100.stdout: "read_count": { 2024-04-07T16:16:14.465 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 23 2024-04-07T16:16:14.465 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.465 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes": { 2024-04-07T16:16:14.465 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1282048 2024-04-07T16:16:14.465 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.465 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes_db": { 2024-04-07T16:16:14.465 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1286144 2024-04-07T16:16:14.465 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.465 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes_slow": { 2024-04-07T16:16:14.466 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.466 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.466 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes_wal": { 2024-04-07T16:16:14.466 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.466 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.466 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_count": { 2024-04-07T16:16:14.466 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 15 2024-04-07T16:16:14.466 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.466 INFO:tasks.workunit.client.0.smithi100.stdout: "read_prefetch_bytes": { 2024-04-07T16:16:14.466 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4913 2024-04-07T16:16:14.466 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.466 INFO:tasks.workunit.client.0.smithi100.stdout: "read_prefetch_count": { 2024-04-07T16:16:14.466 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4 2024-04-07T16:16:14.466 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.467 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_buffer_bytes": { 2024-04-07T16:16:14.467 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5503 2024-04-07T16:16:14.467 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.467 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_buffer_count": { 2024-04-07T16:16:14.467 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 24 2024-04-07T16:16:14.467 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.467 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_bytes": { 2024-04-07T16:16:14.467 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5503 2024-04-07T16:16:14.467 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.467 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_count": { 2024-04-07T16:16:14.467 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 24 2024-04-07T16:16:14.467 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.467 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes": { 2024-04-07T16:16:14.467 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.467 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.468 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes_db": { 2024-04-07T16:16:14.468 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.468 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.468 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes_slow": { 2024-04-07T16:16:14.468 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.468 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.468 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes_wal": { 2024-04-07T16:16:14.468 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.468 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.468 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_count": { 2024-04-07T16:16:14.468 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.468 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.468 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_total_bytes": { 2024-04-07T16:16:14.468 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.469 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.469 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_used_bytes": { 2024-04-07T16:16:14.469 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.469 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.469 INFO:tasks.workunit.client.0.smithi100.stdout: "wal_total_bytes": { 2024-04-07T16:16:14.469 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.469 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.469 INFO:tasks.workunit.client.0.smithi100.stdout: "wal_used_bytes": { 2024-04-07T16:16:14.469 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.469 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.469 INFO:tasks.workunit.client.0.smithi100.stdout: "write_bytes": { 2024-04-07T16:16:14.469 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1851392 2024-04-07T16:16:14.469 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:14.469 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.470 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore": { 2024-04-07T16:16:14.470 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit": { 2024-04-07T16:16:14.470 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4096 2024-04-07T16:16:14.470 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.470 INFO:tasks.workunit.client.0.smithi100.stdout: "allocated": { 2024-04-07T16:16:14.470 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 729088 2024-04-07T16:16:14.470 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.470 INFO:tasks.workunit.client.0.smithi100.stdout: "clist_lat": { 2024-04-07T16:16:14.470 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.470 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.470 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.470 INFO:tasks.workunit.client.0.smithi100.stdout: "compress_lat": { 2024-04-07T16:16:14.470 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.470 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.470 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.471 INFO:tasks.workunit.client.0.smithi100.stdout: "compressed": { 2024-04-07T16:16:14.471 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.471 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.471 INFO:tasks.workunit.client.0.smithi100.stdout: "compressed_allocated": { 2024-04-07T16:16:14.471 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.471 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.471 INFO:tasks.workunit.client.0.smithi100.stdout: "compressed_original": { 2024-04-07T16:16:14.471 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.471 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.471 INFO:tasks.workunit.client.0.smithi100.stdout: "csum_lat": { 2024-04-07T16:16:14.471 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 3, 2024-04-07T16:16:14.471 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5389 2024-04-07T16:16:14.471 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.471 INFO:tasks.workunit.client.0.smithi100.stdout: "decompress_lat": { 2024-04-07T16:16:14.471 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.471 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.472 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.472 INFO:tasks.workunit.client.0.smithi100.stdout: "fragmentation_micros": { 2024-04-07T16:16:14.472 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.472 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.472 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_commit_lat": { 2024-04-07T16:16:14.472 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 149, 2024-04-07T16:16:14.472 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 61891663 2024-04-07T16:16:14.472 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.472 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_final_lat": { 2024-04-07T16:16:14.472 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 137, 2024-04-07T16:16:14.472 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 21858858 2024-04-07T16:16:14.472 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.472 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_flush_lat": { 2024-04-07T16:16:14.472 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 149, 2024-04-07T16:16:14.472 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 396948 2024-04-07T16:16:14.473 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.473 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_sync_lat": { 2024-04-07T16:16:14.473 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 149, 2024-04-07T16:16:14.473 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 62288611 2024-04-07T16:16:14.473 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.473 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_get_keys_lat": { 2024-04-07T16:16:14.473 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.473 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.473 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.473 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_get_values_lat": { 2024-04-07T16:16:14.473 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1, 2024-04-07T16:16:14.473 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 62841 2024-04-07T16:16:14.473 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.473 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_lower_bound_lat": { 2024-04-07T16:16:14.473 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.474 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.474 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.474 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_next_lat": { 2024-04-07T16:16:14.474 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.474 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.474 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.474 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_seek_to_first_lat": { 2024-04-07T16:16:14.474 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.474 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.474 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.474 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_upper_bound_lat": { 2024-04-07T16:16:14.474 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 2, 2024-04-07T16:16:14.474 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4744 2024-04-07T16:16:14.474 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.474 INFO:tasks.workunit.client.0.smithi100.stdout: "onode_hits": { 2024-04-07T16:16:14.475 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 347 2024-04-07T16:16:14.475 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.475 INFO:tasks.workunit.client.0.smithi100.stdout: "onode_misses": { 2024-04-07T16:16:14.475 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 218 2024-04-07T16:16:14.475 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.475 INFO:tasks.workunit.client.0.smithi100.stdout: "read_lat": { 2024-04-07T16:16:14.475 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 86, 2024-04-07T16:16:14.475 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1300527 2024-04-07T16:16:14.475 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.475 INFO:tasks.workunit.client.0.smithi100.stdout: "read_onode_meta_lat": { 2024-04-07T16:16:14.475 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 150, 2024-04-07T16:16:14.475 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 440282 2024-04-07T16:16:14.475 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.475 INFO:tasks.workunit.client.0.smithi100.stdout: "read_wait_aio_lat": { 2024-04-07T16:16:14.475 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 64, 2024-04-07T16:16:14.476 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 474532 2024-04-07T16:16:14.476 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.476 INFO:tasks.workunit.client.0.smithi100.stdout: "reads_with_retries": { 2024-04-07T16:16:14.476 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.476 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.476 INFO:tasks.workunit.client.0.smithi100.stdout: "remove_lat": { 2024-04-07T16:16:14.476 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.476 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.476 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.476 INFO:tasks.workunit.client.0.smithi100.stdout: "state_aio_wait_lat": { 2024-04-07T16:16:14.476 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 244, 2024-04-07T16:16:14.476 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 8191326 2024-04-07T16:16:14.476 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.476 INFO:tasks.workunit.client.0.smithi100.stdout: "state_deferred_aio_wait_lat": { 2024-04-07T16:16:14.476 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 52, 2024-04-07T16:16:14.476 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5014766 2024-04-07T16:16:14.477 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.477 INFO:tasks.workunit.client.0.smithi100.stdout: "state_deferred_cleanup_lat": { 2024-04-07T16:16:14.477 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 49, 2024-04-07T16:16:14.477 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 474728018464 2024-04-07T16:16:14.477 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.477 INFO:tasks.workunit.client.0.smithi100.stdout: "state_deferred_queued_lat": { 2024-04-07T16:16:14.477 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 52, 2024-04-07T16:16:14.477 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 83299056736 2024-04-07T16:16:14.477 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.477 INFO:tasks.workunit.client.0.smithi100.stdout: "state_done_lat": { 2024-04-07T16:16:14.477 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 241, 2024-04-07T16:16:14.477 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 828537 2024-04-07T16:16:14.477 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.477 INFO:tasks.workunit.client.0.smithi100.stdout: "state_finishing_lat": { 2024-04-07T16:16:14.477 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 241, 2024-04-07T16:16:14.477 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 34493 2024-04-07T16:16:14.478 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.478 INFO:tasks.workunit.client.0.smithi100.stdout: "state_io_done_lat": { 2024-04-07T16:16:14.478 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 244, 2024-04-07T16:16:14.478 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 468909 2024-04-07T16:16:14.478 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.478 INFO:tasks.workunit.client.0.smithi100.stdout: "state_kv_commiting_lat": { 2024-04-07T16:16:14.478 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 244, 2024-04-07T16:16:14.478 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 305726439 2024-04-07T16:16:14.478 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.478 INFO:tasks.workunit.client.0.smithi100.stdout: "state_kv_done_lat": { 2024-04-07T16:16:14.478 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 244, 2024-04-07T16:16:14.478 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 103204 2024-04-07T16:16:14.478 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.479 INFO:tasks.workunit.client.0.smithi100.stdout: "state_kv_queued_lat": { 2024-04-07T16:16:14.479 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 244, 2024-04-07T16:16:14.479 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 136509628 2024-04-07T16:16:14.479 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.479 INFO:tasks.workunit.client.0.smithi100.stdout: "state_prepare_lat": { 2024-04-07T16:16:14.479 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 244, 2024-04-07T16:16:14.479 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 22445973 2024-04-07T16:16:14.479 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.479 INFO:tasks.workunit.client.0.smithi100.stdout: "stored": { 2024-04-07T16:16:14.479 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 311968 2024-04-07T16:16:14.479 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.479 INFO:tasks.workunit.client.0.smithi100.stdout: "truncate_lat": { 2024-04-07T16:16:14.479 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 43, 2024-04-07T16:16:14.479 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 379725 2024-04-07T16:16:14.480 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.480 INFO:tasks.workunit.client.0.smithi100.stdout: "txc_commit_lat": { 2024-04-07T16:16:14.480 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 244, 2024-04-07T16:16:14.480 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 473385875 2024-04-07T16:16:14.480 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.480 INFO:tasks.workunit.client.0.smithi100.stdout: "txc_submit_lat": { 2024-04-07T16:16:14.480 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 244, 2024-04-07T16:16:14.480 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 27691111 2024-04-07T16:16:14.480 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.480 INFO:tasks.workunit.client.0.smithi100.stdout: "txc_throttle_lat": { 2024-04-07T16:16:14.480 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 244, 2024-04-07T16:16:14.480 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2138648 2024-04-07T16:16:14.480 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:14.480 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.480 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache": { 2024-04-07T16:16:14.481 INFO:tasks.workunit.client.0.smithi100.stdout: "cache_bytes": { 2024-04-07T16:16:14.481 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1900954368 2024-04-07T16:16:14.481 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.481 INFO:tasks.workunit.client.0.smithi100.stdout: "heap_bytes": { 2024-04-07T16:16:14.481 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 42319872 2024-04-07T16:16:14.481 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.481 INFO:tasks.workunit.client.0.smithi100.stdout: "mapped_bytes": { 2024-04-07T16:16:14.481 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 41533440 2024-04-07T16:16:14.481 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.481 INFO:tasks.workunit.client.0.smithi100.stdout: "target_bytes": { 2024-04-07T16:16:14.481 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3183836160 2024-04-07T16:16:14.481 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.481 INFO:tasks.workunit.client.0.smithi100.stdout: "unmapped_bytes": { 2024-04-07T16:16:14.481 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 786432 2024-04-07T16:16:14.481 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:14.482 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.482 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:data": { 2024-04-07T16:16:14.482 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T16:16:14.482 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 159383552 2024-04-07T16:16:14.482 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.482 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T16:16:14.482 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.482 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.482 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T16:16:14.482 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.482 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.482 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T16:16:14.482 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 87869296 2024-04-07T16:16:14.482 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.483 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T16:16:14.483 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.483 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.483 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T16:16:14.483 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.483 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.483 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T16:16:14.483 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.483 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.483 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T16:16:14.483 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.483 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.483 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T16:16:14.483 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.483 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.484 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T16:16:14.484 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.484 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.484 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T16:16:14.484 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.484 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.484 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T16:16:14.484 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.484 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.484 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T16:16:14.484 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.484 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.484 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T16:16:14.484 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 71514256 2024-04-07T16:16:14.484 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:14.485 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.485 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:kv": { 2024-04-07T16:16:14.485 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T16:16:14.485 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 796917760 2024-04-07T16:16:14.485 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.485 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T16:16:14.485 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 67108864 2024-04-07T16:16:14.485 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.485 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T16:16:14.485 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.485 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.485 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T16:16:14.485 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 659019724 2024-04-07T16:16:14.485 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.485 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T16:16:14.485 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.486 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.486 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T16:16:14.486 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.486 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.486 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T16:16:14.486 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.486 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.486 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T16:16:14.486 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.486 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.486 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T16:16:14.486 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.486 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.486 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T16:16:14.487 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2128 2024-04-07T16:16:14.487 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.487 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T16:16:14.487 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.487 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.487 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T16:16:14.487 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.487 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.487 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T16:16:14.487 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.487 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.487 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T16:16:14.487 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 70787044 2024-04-07T16:16:14.487 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:14.487 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.488 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:kv_onode": { 2024-04-07T16:16:14.488 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T16:16:14.488 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 192937984 2024-04-07T16:16:14.488 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.488 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T16:16:14.488 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 67108864 2024-04-07T16:16:14.488 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.488 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T16:16:14.488 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 464 2024-04-07T16:16:14.488 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.488 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T16:16:14.488 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 58579531 2024-04-07T16:16:14.488 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.488 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T16:16:14.488 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.489 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.489 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T16:16:14.489 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.489 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.489 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T16:16:14.489 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.489 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.489 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T16:16:14.489 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.489 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.489 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T16:16:14.489 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.489 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.490 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T16:16:14.490 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.490 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.490 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T16:16:14.490 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.490 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.490 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T16:16:14.490 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.490 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.490 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T16:16:14.490 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.490 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.490 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T16:16:14.490 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 67249125 2024-04-07T16:16:14.490 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:14.491 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.491 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:meta": { 2024-04-07T16:16:14.491 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T16:16:14.491 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 729808896 2024-04-07T16:16:14.491 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.491 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T16:16:14.491 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.491 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.491 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T16:16:14.491 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.491 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.491 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T16:16:14.491 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 659019724 2024-04-07T16:16:14.492 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.492 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T16:16:14.492 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.492 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.492 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T16:16:14.492 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.492 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.492 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T16:16:14.492 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.492 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.492 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T16:16:14.492 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.492 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.492 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T16:16:14.493 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.493 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.493 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T16:16:14.493 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 218323 2024-04-07T16:16:14.493 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.493 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T16:16:14.493 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.493 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.493 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T16:16:14.493 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.493 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.493 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T16:16:14.493 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.493 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.493 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T16:16:14.493 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 70570849 2024-04-07T16:16:14.494 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:14.494 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.494 INFO:tasks.workunit.client.0.smithi100.stdout: "objecter": { 2024-04-07T16:16:14.494 INFO:tasks.workunit.client.0.smithi100.stdout: "op_active": { 2024-04-07T16:16:14.494 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.494 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.494 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r": { 2024-04-07T16:16:14.494 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.494 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.494 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rmw": { 2024-04-07T16:16:14.494 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.494 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.494 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w": { 2024-04-07T16:16:14.494 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.495 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:14.495 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.495 INFO:tasks.workunit.client.0.smithi100.stdout: "osd": { 2024-04-07T16:16:14.495 INFO:tasks.workunit.client.0.smithi100.stdout: "numpg": { 2024-04-07T16:16:14.495 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 36 2024-04-07T16:16:14.495 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.495 INFO:tasks.workunit.client.0.smithi100.stdout: "numpg_removing": { 2024-04-07T16:16:14.495 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.495 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.495 INFO:tasks.workunit.client.0.smithi100.stdout: "op": { 2024-04-07T16:16:14.495 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.495 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.495 INFO:tasks.workunit.client.0.smithi100.stdout: "op_delayed_degraded": { 2024-04-07T16:16:14.495 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.495 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.495 INFO:tasks.workunit.client.0.smithi100.stdout: "op_delayed_unreadable": { 2024-04-07T16:16:14.496 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.496 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.496 INFO:tasks.workunit.client.0.smithi100.stdout: "op_in_bytes": { 2024-04-07T16:16:14.496 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.496 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.496 INFO:tasks.workunit.client.0.smithi100.stdout: "op_latency": { 2024-04-07T16:16:14.496 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.496 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.496 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.496 INFO:tasks.workunit.client.0.smithi100.stdout: "op_out_bytes": { 2024-04-07T16:16:14.496 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.496 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.496 INFO:tasks.workunit.client.0.smithi100.stdout: "op_prepare_latency": { 2024-04-07T16:16:14.496 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 2, 2024-04-07T16:16:14.496 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1073024 2024-04-07T16:16:14.497 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.497 INFO:tasks.workunit.client.0.smithi100.stdout: "op_process_latency": { 2024-04-07T16:16:14.497 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.497 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.497 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.497 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r": { 2024-04-07T16:16:14.497 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.497 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.497 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_latency": { 2024-04-07T16:16:14.497 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.497 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.497 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.497 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_latency_out_bytes_histogram": { 2024-04-07T16:16:14.497 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.497 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.498 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_out_bytes": { 2024-04-07T16:16:14.498 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.498 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.498 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_prepare_latency": { 2024-04-07T16:16:14.498 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.498 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.498 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.498 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_process_latency": { 2024-04-07T16:16:14.498 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.498 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.498 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.498 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw": { 2024-04-07T16:16:14.498 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.499 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.499 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_in_bytes": { 2024-04-07T16:16:14.499 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.499 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.499 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_latency": { 2024-04-07T16:16:14.499 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.499 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.499 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.499 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_latency_in_bytes_histogram": { 2024-04-07T16:16:14.499 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.499 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.499 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_latency_out_bytes_histogram": { 2024-04-07T16:16:14.499 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.499 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.500 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_out_bytes": { 2024-04-07T16:16:14.500 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.500 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.500 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_prepare_latency": { 2024-04-07T16:16:14.500 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.500 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.500 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.500 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_process_latency": { 2024-04-07T16:16:14.500 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.500 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.500 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.500 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w": { 2024-04-07T16:16:14.500 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.500 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.500 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_in_bytes": { 2024-04-07T16:16:14.501 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.501 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.501 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_latency": { 2024-04-07T16:16:14.501 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.501 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.501 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.501 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_latency_in_bytes_histogram": { 2024-04-07T16:16:14.501 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.501 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.501 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_prepare_latency": { 2024-04-07T16:16:14.501 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 2, 2024-04-07T16:16:14.501 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1073024 2024-04-07T16:16:14.501 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.501 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_process_latency": { 2024-04-07T16:16:14.501 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.501 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.502 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.502 INFO:tasks.workunit.client.0.smithi100.stdout: "op_wip": { 2024-04-07T16:16:14.502 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.502 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.502 INFO:tasks.workunit.client.0.smithi100.stdout: "recovery_bytes": { 2024-04-07T16:16:14.502 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.502 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.502 INFO:tasks.workunit.client.0.smithi100.stdout: "recovery_ops": { 2024-04-07T16:16:14.502 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.502 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.502 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_bytes": { 2024-04-07T16:16:14.502 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 95995035648 2024-04-07T16:16:14.502 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.502 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_bytes_used": { 2024-04-07T16:16:14.503 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 28393472 2024-04-07T16:16:14.503 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:14.503 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:14.503 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.503 INFO:tasks.workunit.client.0.smithi100.stdout: "osd.4": { 2024-04-07T16:16:14.503 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs": { 2024-04-07T16:16:14.503 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_slow_fallback": { 2024-04-07T16:16:14.503 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.503 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.503 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_slow_size_fallback": { 2024-04-07T16:16:14.503 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.503 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.503 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit_db": { 2024-04-07T16:16:14.503 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 65536 2024-04-07T16:16:14.503 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.503 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit_main": { 2024-04-07T16:16:14.504 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.504 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.504 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit_wal": { 2024-04-07T16:16:14.504 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.504 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.504 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_written_slow": { 2024-04-07T16:16:14.504 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.504 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.504 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_written_sst": { 2024-04-07T16:16:14.504 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 12288 2024-04-07T16:16:14.504 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.504 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_written_wal": { 2024-04-07T16:16:14.504 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 8314880 2024-04-07T16:16:14.504 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.504 INFO:tasks.workunit.client.0.smithi100.stdout: "compact_lat": { 2024-04-07T16:16:14.505 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.505 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.505 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.505 INFO:tasks.workunit.client.0.smithi100.stdout: "compact_lock_lat": { 2024-04-07T16:16:14.505 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.505 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.505 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.505 INFO:tasks.workunit.client.0.smithi100.stdout: "db_total_bytes": { 2024-04-07T16:16:14.505 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 95995035648 2024-04-07T16:16:14.505 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.505 INFO:tasks.workunit.client.0.smithi100.stdout: "db_used_bytes": { 2024-04-07T16:16:14.505 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 31850496 2024-04-07T16:16:14.505 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.505 INFO:tasks.workunit.client.0.smithi100.stdout: "log_bytes": { 2024-04-07T16:16:14.505 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5283840 2024-04-07T16:16:14.505 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.506 INFO:tasks.workunit.client.0.smithi100.stdout: "logged_bytes": { 2024-04-07T16:16:14.506 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5144576 2024-04-07T16:16:14.506 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.506 INFO:tasks.workunit.client.0.smithi100.stdout: "max_bytes_db": { 2024-04-07T16:16:14.506 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 31850496 2024-04-07T16:16:14.506 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.506 INFO:tasks.workunit.client.0.smithi100.stdout: "max_bytes_slow": { 2024-04-07T16:16:14.506 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.506 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.506 INFO:tasks.workunit.client.0.smithi100.stdout: "max_bytes_wal": { 2024-04-07T16:16:14.506 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.506 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.506 INFO:tasks.workunit.client.0.smithi100.stdout: "num_files": { 2024-04-07T16:16:14.506 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 14 2024-04-07T16:16:14.507 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.507 INFO:tasks.workunit.client.0.smithi100.stdout: "read_bytes": { 2024-04-07T16:16:14.507 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 206272 2024-04-07T16:16:14.507 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.507 INFO:tasks.workunit.client.0.smithi100.stdout: "read_count": { 2024-04-07T16:16:14.507 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 23 2024-04-07T16:16:14.507 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.507 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes": { 2024-04-07T16:16:14.507 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1282048 2024-04-07T16:16:14.507 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.507 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes_db": { 2024-04-07T16:16:14.507 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1286144 2024-04-07T16:16:14.507 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.507 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes_slow": { 2024-04-07T16:16:14.507 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.507 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.508 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes_wal": { 2024-04-07T16:16:14.508 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.508 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.508 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_count": { 2024-04-07T16:16:14.508 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 15 2024-04-07T16:16:14.508 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.508 INFO:tasks.workunit.client.0.smithi100.stdout: "read_prefetch_bytes": { 2024-04-07T16:16:14.508 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4913 2024-04-07T16:16:14.508 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.508 INFO:tasks.workunit.client.0.smithi100.stdout: "read_prefetch_count": { 2024-04-07T16:16:14.508 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4 2024-04-07T16:16:14.508 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.508 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_buffer_bytes": { 2024-04-07T16:16:14.508 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5505 2024-04-07T16:16:14.508 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.508 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_buffer_count": { 2024-04-07T16:16:14.509 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 24 2024-04-07T16:16:14.509 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.509 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_bytes": { 2024-04-07T16:16:14.509 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5505 2024-04-07T16:16:14.509 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.509 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_count": { 2024-04-07T16:16:14.509 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 24 2024-04-07T16:16:14.509 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.509 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes": { 2024-04-07T16:16:14.509 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.509 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.509 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes_db": { 2024-04-07T16:16:14.509 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.509 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.509 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes_slow": { 2024-04-07T16:16:14.509 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.510 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.510 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes_wal": { 2024-04-07T16:16:14.510 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.510 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.510 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_count": { 2024-04-07T16:16:14.510 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.510 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.510 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_total_bytes": { 2024-04-07T16:16:14.510 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.510 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.510 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_used_bytes": { 2024-04-07T16:16:14.510 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.510 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.510 INFO:tasks.workunit.client.0.smithi100.stdout: "wal_total_bytes": { 2024-04-07T16:16:14.510 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.511 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.511 INFO:tasks.workunit.client.0.smithi100.stdout: "wal_used_bytes": { 2024-04-07T16:16:14.511 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.511 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.511 INFO:tasks.workunit.client.0.smithi100.stdout: "write_bytes": { 2024-04-07T16:16:14.511 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 13537280 2024-04-07T16:16:14.511 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:14.511 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.511 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore": { 2024-04-07T16:16:14.511 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit": { 2024-04-07T16:16:14.511 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4096 2024-04-07T16:16:14.511 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.511 INFO:tasks.workunit.client.0.smithi100.stdout: "allocated": { 2024-04-07T16:16:14.511 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 724992 2024-04-07T16:16:14.511 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.512 INFO:tasks.workunit.client.0.smithi100.stdout: "clist_lat": { 2024-04-07T16:16:14.512 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 8, 2024-04-07T16:16:14.512 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 725717 2024-04-07T16:16:14.512 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.512 INFO:tasks.workunit.client.0.smithi100.stdout: "compress_lat": { 2024-04-07T16:16:14.512 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.512 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.512 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.512 INFO:tasks.workunit.client.0.smithi100.stdout: "compressed": { 2024-04-07T16:16:14.512 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.512 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.512 INFO:tasks.workunit.client.0.smithi100.stdout: "compressed_allocated": { 2024-04-07T16:16:14.512 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.512 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.512 INFO:tasks.workunit.client.0.smithi100.stdout: "compressed_original": { 2024-04-07T16:16:14.513 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.513 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.513 INFO:tasks.workunit.client.0.smithi100.stdout: "csum_lat": { 2024-04-07T16:16:14.513 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1, 2024-04-07T16:16:14.513 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2633 2024-04-07T16:16:14.513 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.513 INFO:tasks.workunit.client.0.smithi100.stdout: "decompress_lat": { 2024-04-07T16:16:14.513 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.513 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.513 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.513 INFO:tasks.workunit.client.0.smithi100.stdout: "fragmentation_micros": { 2024-04-07T16:16:14.513 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.513 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.513 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_commit_lat": { 2024-04-07T16:16:14.513 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1243, 2024-04-07T16:16:14.514 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 591333721 2024-04-07T16:16:14.514 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.514 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_final_lat": { 2024-04-07T16:16:14.514 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1238, 2024-04-07T16:16:14.514 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 32704757 2024-04-07T16:16:14.514 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.514 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_flush_lat": { 2024-04-07T16:16:14.514 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1243, 2024-04-07T16:16:14.514 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1896043 2024-04-07T16:16:14.514 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.514 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_sync_lat": { 2024-04-07T16:16:14.514 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1243, 2024-04-07T16:16:14.514 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 593229764 2024-04-07T16:16:14.514 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.514 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_get_keys_lat": { 2024-04-07T16:16:14.515 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.515 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.515 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.515 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_get_values_lat": { 2024-04-07T16:16:14.515 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 3, 2024-04-07T16:16:14.515 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 66079 2024-04-07T16:16:14.515 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.515 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_lower_bound_lat": { 2024-04-07T16:16:14.515 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.515 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.515 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.515 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_next_lat": { 2024-04-07T16:16:14.515 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.515 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.515 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.515 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_seek_to_first_lat": { 2024-04-07T16:16:14.516 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.516 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.516 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.516 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_upper_bound_lat": { 2024-04-07T16:16:14.516 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 2, 2024-04-07T16:16:14.516 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 7970 2024-04-07T16:16:14.516 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.516 INFO:tasks.workunit.client.0.smithi100.stdout: "onode_hits": { 2024-04-07T16:16:14.516 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3462 2024-04-07T16:16:14.516 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.516 INFO:tasks.workunit.client.0.smithi100.stdout: "onode_misses": { 2024-04-07T16:16:14.516 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 340 2024-04-07T16:16:14.516 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.517 INFO:tasks.workunit.client.0.smithi100.stdout: "read_lat": { 2024-04-07T16:16:14.517 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 96, 2024-04-07T16:16:14.517 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1472078 2024-04-07T16:16:14.517 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.517 INFO:tasks.workunit.client.0.smithi100.stdout: "read_onode_meta_lat": { 2024-04-07T16:16:14.517 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 162, 2024-04-07T16:16:14.517 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 507967 2024-04-07T16:16:14.517 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.517 INFO:tasks.workunit.client.0.smithi100.stdout: "read_wait_aio_lat": { 2024-04-07T16:16:14.517 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 66, 2024-04-07T16:16:14.517 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 172698 2024-04-07T16:16:14.517 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.517 INFO:tasks.workunit.client.0.smithi100.stdout: "reads_with_retries": { 2024-04-07T16:16:14.517 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.517 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.517 INFO:tasks.workunit.client.0.smithi100.stdout: "remove_lat": { 2024-04-07T16:16:14.518 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 105, 2024-04-07T16:16:14.518 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 9372306 2024-04-07T16:16:14.518 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.518 INFO:tasks.workunit.client.0.smithi100.stdout: "state_aio_wait_lat": { 2024-04-07T16:16:14.518 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 3362, 2024-04-07T16:16:14.518 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3286501480 2024-04-07T16:16:14.518 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.518 INFO:tasks.workunit.client.0.smithi100.stdout: "state_deferred_aio_wait_lat": { 2024-04-07T16:16:14.518 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 42, 2024-04-07T16:16:14.518 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 15424969 2024-04-07T16:16:14.518 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.518 INFO:tasks.workunit.client.0.smithi100.stdout: "state_deferred_cleanup_lat": { 2024-04-07T16:16:14.518 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 41, 2024-04-07T16:16:14.518 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 382467755091 2024-04-07T16:16:14.519 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.519 INFO:tasks.workunit.client.0.smithi100.stdout: "state_deferred_queued_lat": { 2024-04-07T16:16:14.519 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 42, 2024-04-07T16:16:14.519 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 60268645703 2024-04-07T16:16:14.519 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.519 INFO:tasks.workunit.client.0.smithi100.stdout: "state_done_lat": { 2024-04-07T16:16:14.519 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 3361, 2024-04-07T16:16:14.519 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1208218205 2024-04-07T16:16:14.519 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.519 INFO:tasks.workunit.client.0.smithi100.stdout: "state_finishing_lat": { 2024-04-07T16:16:14.519 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 3361, 2024-04-07T16:16:14.519 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 306962 2024-04-07T16:16:14.519 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.519 INFO:tasks.workunit.client.0.smithi100.stdout: "state_io_done_lat": { 2024-04-07T16:16:14.519 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 3362, 2024-04-07T16:16:14.520 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4984398 2024-04-07T16:16:14.520 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.520 INFO:tasks.workunit.client.0.smithi100.stdout: "state_kv_commiting_lat": { 2024-04-07T16:16:14.520 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 3362, 2024-04-07T16:16:14.520 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2870095345 2024-04-07T16:16:14.520 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.520 INFO:tasks.workunit.client.0.smithi100.stdout: "state_kv_done_lat": { 2024-04-07T16:16:14.520 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 3362, 2024-04-07T16:16:14.520 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 697077 2024-04-07T16:16:14.520 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.520 INFO:tasks.workunit.client.0.smithi100.stdout: "state_kv_queued_lat": { 2024-04-07T16:16:14.520 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 3362, 2024-04-07T16:16:14.520 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1718221786 2024-04-07T16:16:14.520 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.520 INFO:tasks.workunit.client.0.smithi100.stdout: "state_prepare_lat": { 2024-04-07T16:16:14.520 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 3362, 2024-04-07T16:16:14.521 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 371037756 2024-04-07T16:16:14.521 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.521 INFO:tasks.workunit.client.0.smithi100.stdout: "stored": { 2024-04-07T16:16:14.521 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 309940 2024-04-07T16:16:14.521 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.521 INFO:tasks.workunit.client.0.smithi100.stdout: "truncate_lat": { 2024-04-07T16:16:14.521 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 42, 2024-04-07T16:16:14.521 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 302711 2024-04-07T16:16:14.521 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.521 INFO:tasks.workunit.client.0.smithi100.stdout: "txc_commit_lat": { 2024-04-07T16:16:14.521 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 3362, 2024-04-07T16:16:14.521 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 8251174891 2024-04-07T16:16:14.521 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.521 INFO:tasks.workunit.client.0.smithi100.stdout: "txc_submit_lat": { 2024-04-07T16:16:14.521 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 3362, 2024-04-07T16:16:14.522 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 476238588 2024-04-07T16:16:14.522 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.522 INFO:tasks.workunit.client.0.smithi100.stdout: "txc_throttle_lat": { 2024-04-07T16:16:14.522 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 3362, 2024-04-07T16:16:14.522 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 165941416 2024-04-07T16:16:14.522 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:14.522 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.522 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache": { 2024-04-07T16:16:14.522 INFO:tasks.workunit.client.0.smithi100.stdout: "cache_bytes": { 2024-04-07T16:16:14.522 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1900990924 2024-04-07T16:16:14.522 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.522 INFO:tasks.workunit.client.0.smithi100.stdout: "heap_bytes": { 2024-04-07T16:16:14.522 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 45400064 2024-04-07T16:16:14.522 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.522 INFO:tasks.workunit.client.0.smithi100.stdout: "mapped_bytes": { 2024-04-07T16:16:14.523 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 45400064 2024-04-07T16:16:14.523 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.523 INFO:tasks.workunit.client.0.smithi100.stdout: "target_bytes": { 2024-04-07T16:16:14.523 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3183879168 2024-04-07T16:16:14.523 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.523 INFO:tasks.workunit.client.0.smithi100.stdout: "unmapped_bytes": { 2024-04-07T16:16:14.523 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.523 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:14.523 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.523 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:data": { 2024-04-07T16:16:14.523 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T16:16:14.523 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 159383552 2024-04-07T16:16:14.523 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.523 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T16:16:14.523 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.524 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.524 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T16:16:14.524 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.524 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.524 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T16:16:14.524 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 87864559 2024-04-07T16:16:14.524 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.524 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T16:16:14.524 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.524 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.524 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T16:16:14.524 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.524 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.524 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T16:16:14.524 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.525 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.525 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T16:16:14.525 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.525 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.525 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T16:16:14.525 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.525 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.525 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T16:16:14.525 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.525 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.525 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T16:16:14.525 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.525 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.525 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T16:16:14.525 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.526 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.526 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T16:16:14.526 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.526 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.526 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T16:16:14.526 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 71518993 2024-04-07T16:16:14.526 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:14.526 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.526 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:kv": { 2024-04-07T16:16:14.526 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T16:16:14.526 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 796917760 2024-04-07T16:16:14.526 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.526 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T16:16:14.526 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 67108864 2024-04-07T16:16:14.526 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.527 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T16:16:14.527 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.527 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.527 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T16:16:14.527 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 658984197 2024-04-07T16:16:14.527 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.527 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T16:16:14.527 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.527 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.527 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T16:16:14.527 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.527 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.527 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T16:16:14.527 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.527 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.528 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T16:16:14.528 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.528 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.528 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T16:16:14.528 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.528 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.528 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T16:16:14.528 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2128 2024-04-07T16:16:14.528 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.528 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T16:16:14.528 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.528 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.528 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T16:16:14.528 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.528 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.529 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T16:16:14.529 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.529 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.529 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T16:16:14.529 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 70822571 2024-04-07T16:16:14.529 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:14.529 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.529 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:kv_onode": { 2024-04-07T16:16:14.529 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T16:16:14.529 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 192937984 2024-04-07T16:16:14.529 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.529 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T16:16:14.529 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 67108864 2024-04-07T16:16:14.529 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.529 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T16:16:14.530 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 464 2024-04-07T16:16:14.530 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.530 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T16:16:14.530 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 58576373 2024-04-07T16:16:14.530 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.530 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T16:16:14.530 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.530 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.530 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T16:16:14.530 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.530 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.530 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T16:16:14.530 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.530 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.531 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T16:16:14.531 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.531 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.531 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T16:16:14.531 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.531 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.531 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T16:16:14.531 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.531 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.531 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T16:16:14.531 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.531 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.531 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T16:16:14.531 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.531 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.531 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T16:16:14.532 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.532 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.532 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T16:16:14.532 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 67252283 2024-04-07T16:16:14.532 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:14.532 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.532 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:meta": { 2024-04-07T16:16:14.532 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T16:16:14.532 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 729808896 2024-04-07T16:16:14.532 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.532 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T16:16:14.532 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.532 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.532 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T16:16:14.532 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.533 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.533 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T16:16:14.533 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 658984197 2024-04-07T16:16:14.533 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.533 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T16:16:14.533 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.533 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.533 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T16:16:14.533 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.533 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.533 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T16:16:14.533 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.533 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.533 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T16:16:14.534 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.534 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.534 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T16:16:14.534 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.534 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.534 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T16:16:14.534 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 331144 2024-04-07T16:16:14.534 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.534 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T16:16:14.534 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.534 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.534 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T16:16:14.534 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.534 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.535 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T16:16:14.535 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.535 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.535 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T16:16:14.535 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 70493555 2024-04-07T16:16:14.535 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:14.535 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.535 INFO:tasks.workunit.client.0.smithi100.stdout: "objecter": { 2024-04-07T16:16:14.535 INFO:tasks.workunit.client.0.smithi100.stdout: "op_active": { 2024-04-07T16:16:14.535 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.535 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.535 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r": { 2024-04-07T16:16:14.535 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.535 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.535 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rmw": { 2024-04-07T16:16:14.536 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.536 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.536 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w": { 2024-04-07T16:16:14.536 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.536 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:14.536 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.536 INFO:tasks.workunit.client.0.smithi100.stdout: "osd": { 2024-04-07T16:16:14.536 INFO:tasks.workunit.client.0.smithi100.stdout: "numpg": { 2024-04-07T16:16:14.536 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 40 2024-04-07T16:16:14.536 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.536 INFO:tasks.workunit.client.0.smithi100.stdout: "numpg_removing": { 2024-04-07T16:16:14.536 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.536 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.536 INFO:tasks.workunit.client.0.smithi100.stdout: "op": { 2024-04-07T16:16:14.536 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1 2024-04-07T16:16:14.537 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.537 INFO:tasks.workunit.client.0.smithi100.stdout: "op_delayed_degraded": { 2024-04-07T16:16:14.537 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.537 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.537 INFO:tasks.workunit.client.0.smithi100.stdout: "op_delayed_unreadable": { 2024-04-07T16:16:14.537 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.537 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.537 INFO:tasks.workunit.client.0.smithi100.stdout: "op_in_bytes": { 2024-04-07T16:16:14.537 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 22 2024-04-07T16:16:14.537 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.537 INFO:tasks.workunit.client.0.smithi100.stdout: "op_latency": { 2024-04-07T16:16:14.537 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1, 2024-04-07T16:16:14.537 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 10592946 2024-04-07T16:16:14.537 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.538 INFO:tasks.workunit.client.0.smithi100.stdout: "op_out_bytes": { 2024-04-07T16:16:14.538 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.538 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.538 INFO:tasks.workunit.client.0.smithi100.stdout: "op_prepare_latency": { 2024-04-07T16:16:14.538 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1, 2024-04-07T16:16:14.538 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 622928 2024-04-07T16:16:14.538 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.538 INFO:tasks.workunit.client.0.smithi100.stdout: "op_process_latency": { 2024-04-07T16:16:14.538 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1, 2024-04-07T16:16:14.538 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 10505965 2024-04-07T16:16:14.538 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.538 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r": { 2024-04-07T16:16:14.538 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.539 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.539 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_latency": { 2024-04-07T16:16:14.539 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.539 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.539 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.539 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_latency_out_bytes_histogram": { 2024-04-07T16:16:14.539 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.539 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.539 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_out_bytes": { 2024-04-07T16:16:14.539 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.539 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.539 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_prepare_latency": { 2024-04-07T16:16:14.539 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.539 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.539 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.540 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_process_latency": { 2024-04-07T16:16:14.540 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.540 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.540 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.540 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw": { 2024-04-07T16:16:14.540 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.540 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.540 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_in_bytes": { 2024-04-07T16:16:14.540 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.540 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.540 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_latency": { 2024-04-07T16:16:14.540 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.540 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.540 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.540 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_latency_in_bytes_histogram": { 2024-04-07T16:16:14.541 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.541 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.541 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_latency_out_bytes_histogram": { 2024-04-07T16:16:14.541 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.541 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.541 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_out_bytes": { 2024-04-07T16:16:14.541 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.541 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.541 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_prepare_latency": { 2024-04-07T16:16:14.541 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.541 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.541 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.541 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_process_latency": { 2024-04-07T16:16:14.541 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.541 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.542 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.542 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w": { 2024-04-07T16:16:14.542 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1 2024-04-07T16:16:14.542 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.542 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_in_bytes": { 2024-04-07T16:16:14.542 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 22 2024-04-07T16:16:14.542 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.542 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_latency": { 2024-04-07T16:16:14.542 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1, 2024-04-07T16:16:14.542 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 10592946 2024-04-07T16:16:14.542 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.542 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_latency_in_bytes_histogram": { 2024-04-07T16:16:14.542 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.542 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.543 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_prepare_latency": { 2024-04-07T16:16:14.543 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1, 2024-04-07T16:16:14.543 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 622928 2024-04-07T16:16:14.543 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.543 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_process_latency": { 2024-04-07T16:16:14.543 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1, 2024-04-07T16:16:14.543 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 10505965 2024-04-07T16:16:14.543 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.543 INFO:tasks.workunit.client.0.smithi100.stdout: "op_wip": { 2024-04-07T16:16:14.543 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.543 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.543 INFO:tasks.workunit.client.0.smithi100.stdout: "recovery_bytes": { 2024-04-07T16:16:14.543 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.543 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.544 INFO:tasks.workunit.client.0.smithi100.stdout: "recovery_ops": { 2024-04-07T16:16:14.544 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.544 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.544 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_bytes": { 2024-04-07T16:16:14.544 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 95995035648 2024-04-07T16:16:14.544 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.544 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_bytes_used": { 2024-04-07T16:16:14.544 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 32587776 2024-04-07T16:16:14.544 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:14.544 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:14.544 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.544 INFO:tasks.workunit.client.0.smithi100.stdout: "osd.5": { 2024-04-07T16:16:14.544 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs": { 2024-04-07T16:16:14.544 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_slow_fallback": { 2024-04-07T16:16:14.544 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.544 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.545 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_slow_size_fallback": { 2024-04-07T16:16:14.545 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.545 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.545 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit_db": { 2024-04-07T16:16:14.545 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 65536 2024-04-07T16:16:14.545 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.545 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit_main": { 2024-04-07T16:16:14.545 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.545 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.545 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit_wal": { 2024-04-07T16:16:14.545 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.545 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.545 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_written_slow": { 2024-04-07T16:16:14.545 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.545 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.546 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_written_sst": { 2024-04-07T16:16:14.546 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 12288 2024-04-07T16:16:14.546 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.546 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_written_wal": { 2024-04-07T16:16:14.546 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1191936 2024-04-07T16:16:14.546 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.546 INFO:tasks.workunit.client.0.smithi100.stdout: "compact_lat": { 2024-04-07T16:16:14.546 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.546 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.546 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.546 INFO:tasks.workunit.client.0.smithi100.stdout: "compact_lock_lat": { 2024-04-07T16:16:14.546 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.546 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.546 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.547 INFO:tasks.workunit.client.0.smithi100.stdout: "db_total_bytes": { 2024-04-07T16:16:14.547 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 95995035648 2024-04-07T16:16:14.547 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.547 INFO:tasks.workunit.client.0.smithi100.stdout: "db_used_bytes": { 2024-04-07T16:16:14.547 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 27656192 2024-04-07T16:16:14.547 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.547 INFO:tasks.workunit.client.0.smithi100.stdout: "log_bytes": { 2024-04-07T16:16:14.547 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 880640 2024-04-07T16:16:14.547 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.547 INFO:tasks.workunit.client.0.smithi100.stdout: "logged_bytes": { 2024-04-07T16:16:14.547 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 741376 2024-04-07T16:16:14.547 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.547 INFO:tasks.workunit.client.0.smithi100.stdout: "max_bytes_db": { 2024-04-07T16:16:14.547 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 31850496 2024-04-07T16:16:14.547 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.548 INFO:tasks.workunit.client.0.smithi100.stdout: "max_bytes_slow": { 2024-04-07T16:16:14.548 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.548 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.548 INFO:tasks.workunit.client.0.smithi100.stdout: "max_bytes_wal": { 2024-04-07T16:16:14.548 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.548 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.548 INFO:tasks.workunit.client.0.smithi100.stdout: "num_files": { 2024-04-07T16:16:14.548 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 14 2024-04-07T16:16:14.548 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.548 INFO:tasks.workunit.client.0.smithi100.stdout: "read_bytes": { 2024-04-07T16:16:14.548 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 206272 2024-04-07T16:16:14.548 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.548 INFO:tasks.workunit.client.0.smithi100.stdout: "read_count": { 2024-04-07T16:16:14.548 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 23 2024-04-07T16:16:14.548 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.549 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes": { 2024-04-07T16:16:14.549 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1282048 2024-04-07T16:16:14.549 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.549 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes_db": { 2024-04-07T16:16:14.549 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1286144 2024-04-07T16:16:14.549 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.549 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes_slow": { 2024-04-07T16:16:14.549 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.549 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.549 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes_wal": { 2024-04-07T16:16:14.549 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.549 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.549 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_count": { 2024-04-07T16:16:14.549 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 15 2024-04-07T16:16:14.549 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.550 INFO:tasks.workunit.client.0.smithi100.stdout: "read_prefetch_bytes": { 2024-04-07T16:16:14.550 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4913 2024-04-07T16:16:14.550 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.550 INFO:tasks.workunit.client.0.smithi100.stdout: "read_prefetch_count": { 2024-04-07T16:16:14.550 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4 2024-04-07T16:16:14.550 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.550 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_buffer_bytes": { 2024-04-07T16:16:14.550 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5505 2024-04-07T16:16:14.550 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.550 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_buffer_count": { 2024-04-07T16:16:14.550 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 24 2024-04-07T16:16:14.550 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.550 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_bytes": { 2024-04-07T16:16:14.550 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5505 2024-04-07T16:16:14.550 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.551 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_count": { 2024-04-07T16:16:14.551 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 24 2024-04-07T16:16:14.551 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.551 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes": { 2024-04-07T16:16:14.551 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.551 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.551 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes_db": { 2024-04-07T16:16:14.551 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.551 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.551 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes_slow": { 2024-04-07T16:16:14.551 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.551 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.551 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes_wal": { 2024-04-07T16:16:14.551 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.551 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.552 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_count": { 2024-04-07T16:16:14.552 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.552 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.552 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_total_bytes": { 2024-04-07T16:16:14.552 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.552 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.552 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_used_bytes": { 2024-04-07T16:16:14.552 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.552 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.552 INFO:tasks.workunit.client.0.smithi100.stdout: "wal_total_bytes": { 2024-04-07T16:16:14.552 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.552 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.552 INFO:tasks.workunit.client.0.smithi100.stdout: "wal_used_bytes": { 2024-04-07T16:16:14.552 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.552 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.552 INFO:tasks.workunit.client.0.smithi100.stdout: "write_bytes": { 2024-04-07T16:16:14.553 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2011136 2024-04-07T16:16:14.553 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:14.553 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.553 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore": { 2024-04-07T16:16:14.553 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit": { 2024-04-07T16:16:14.553 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4096 2024-04-07T16:16:14.553 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.553 INFO:tasks.workunit.client.0.smithi100.stdout: "allocated": { 2024-04-07T16:16:14.553 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 720896 2024-04-07T16:16:14.553 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.553 INFO:tasks.workunit.client.0.smithi100.stdout: "clist_lat": { 2024-04-07T16:16:14.553 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 4, 2024-04-07T16:16:14.553 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 270895 2024-04-07T16:16:14.553 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.554 INFO:tasks.workunit.client.0.smithi100.stdout: "compress_lat": { 2024-04-07T16:16:14.554 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.554 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.554 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.554 INFO:tasks.workunit.client.0.smithi100.stdout: "compressed": { 2024-04-07T16:16:14.554 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.554 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.554 INFO:tasks.workunit.client.0.smithi100.stdout: "compressed_allocated": { 2024-04-07T16:16:14.554 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.554 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.554 INFO:tasks.workunit.client.0.smithi100.stdout: "compressed_original": { 2024-04-07T16:16:14.554 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.554 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.554 INFO:tasks.workunit.client.0.smithi100.stdout: "csum_lat": { 2024-04-07T16:16:14.555 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 3, 2024-04-07T16:16:14.555 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 6907 2024-04-07T16:16:14.555 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.555 INFO:tasks.workunit.client.0.smithi100.stdout: "decompress_lat": { 2024-04-07T16:16:14.555 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.555 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.555 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.555 INFO:tasks.workunit.client.0.smithi100.stdout: "fragmentation_micros": { 2024-04-07T16:16:14.555 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.555 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.555 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_commit_lat": { 2024-04-07T16:16:14.555 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 168, 2024-04-07T16:16:14.555 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 81377208 2024-04-07T16:16:14.555 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.555 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_final_lat": { 2024-04-07T16:16:14.556 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 160, 2024-04-07T16:16:14.556 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 11809538 2024-04-07T16:16:14.556 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.556 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_flush_lat": { 2024-04-07T16:16:14.556 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 168, 2024-04-07T16:16:14.556 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 269652 2024-04-07T16:16:14.556 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.556 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_sync_lat": { 2024-04-07T16:16:14.556 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 168, 2024-04-07T16:16:14.556 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 81646860 2024-04-07T16:16:14.556 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.556 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_get_keys_lat": { 2024-04-07T16:16:14.556 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.556 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.557 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.557 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_get_values_lat": { 2024-04-07T16:16:14.557 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 3, 2024-04-07T16:16:14.557 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 59409 2024-04-07T16:16:14.557 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.557 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_lower_bound_lat": { 2024-04-07T16:16:14.557 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.557 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.557 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.557 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_next_lat": { 2024-04-07T16:16:14.557 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.557 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.557 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.557 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_seek_to_first_lat": { 2024-04-07T16:16:14.557 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.557 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.558 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.558 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_upper_bound_lat": { 2024-04-07T16:16:14.558 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 2, 2024-04-07T16:16:14.558 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5918 2024-04-07T16:16:14.558 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.558 INFO:tasks.workunit.client.0.smithi100.stdout: "onode_hits": { 2024-04-07T16:16:14.558 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 329 2024-04-07T16:16:14.558 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.558 INFO:tasks.workunit.client.0.smithi100.stdout: "onode_misses": { 2024-04-07T16:16:14.562 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 243 2024-04-07T16:16:14.562 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.562 INFO:tasks.workunit.client.0.smithi100.stdout: "read_lat": { 2024-04-07T16:16:14.562 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 91, 2024-04-07T16:16:14.562 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1181894 2024-04-07T16:16:14.562 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.562 INFO:tasks.workunit.client.0.smithi100.stdout: "read_onode_meta_lat": { 2024-04-07T16:16:14.562 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 145, 2024-04-07T16:16:14.562 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 469150 2024-04-07T16:16:14.562 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.562 INFO:tasks.workunit.client.0.smithi100.stdout: "read_wait_aio_lat": { 2024-04-07T16:16:14.563 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 54, 2024-04-07T16:16:14.563 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 430530 2024-04-07T16:16:14.563 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.563 INFO:tasks.workunit.client.0.smithi100.stdout: "reads_with_retries": { 2024-04-07T16:16:14.563 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.563 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.563 INFO:tasks.workunit.client.0.smithi100.stdout: "remove_lat": { 2024-04-07T16:16:14.563 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 3, 2024-04-07T16:16:14.563 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 191068 2024-04-07T16:16:14.563 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.563 INFO:tasks.workunit.client.0.smithi100.stdout: "state_aio_wait_lat": { 2024-04-07T16:16:14.563 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 269, 2024-04-07T16:16:14.563 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 8414866 2024-04-07T16:16:14.563 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.563 INFO:tasks.workunit.client.0.smithi100.stdout: "state_deferred_aio_wait_lat": { 2024-04-07T16:16:14.564 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 38, 2024-04-07T16:16:14.564 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3864381 2024-04-07T16:16:14.564 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.564 INFO:tasks.workunit.client.0.smithi100.stdout: "state_deferred_cleanup_lat": { 2024-04-07T16:16:14.564 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 35, 2024-04-07T16:16:14.564 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 411804893436 2024-04-07T16:16:14.564 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.564 INFO:tasks.workunit.client.0.smithi100.stdout: "state_deferred_queued_lat": { 2024-04-07T16:16:14.564 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 38, 2024-04-07T16:16:14.564 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 62590955187 2024-04-07T16:16:14.564 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.564 INFO:tasks.workunit.client.0.smithi100.stdout: "state_done_lat": { 2024-04-07T16:16:14.564 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 266, 2024-04-07T16:16:14.564 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 793682 2024-04-07T16:16:14.564 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.565 INFO:tasks.workunit.client.0.smithi100.stdout: "state_finishing_lat": { 2024-04-07T16:16:14.565 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 266, 2024-04-07T16:16:14.565 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 73704 2024-04-07T16:16:14.565 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.565 INFO:tasks.workunit.client.0.smithi100.stdout: "state_io_done_lat": { 2024-04-07T16:16:14.565 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 269, 2024-04-07T16:16:14.565 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 376944 2024-04-07T16:16:14.565 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.565 INFO:tasks.workunit.client.0.smithi100.stdout: "state_kv_commiting_lat": { 2024-04-07T16:16:14.565 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 269, 2024-04-07T16:16:14.565 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 169873885 2024-04-07T16:16:14.565 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.565 INFO:tasks.workunit.client.0.smithi100.stdout: "state_kv_done_lat": { 2024-04-07T16:16:14.565 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 269, 2024-04-07T16:16:14.565 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 119204 2024-04-07T16:16:14.566 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.566 INFO:tasks.workunit.client.0.smithi100.stdout: "state_kv_queued_lat": { 2024-04-07T16:16:14.566 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 269, 2024-04-07T16:16:14.566 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 89218554 2024-04-07T16:16:14.566 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.566 INFO:tasks.workunit.client.0.smithi100.stdout: "state_prepare_lat": { 2024-04-07T16:16:14.566 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 269, 2024-04-07T16:16:14.566 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 19868622 2024-04-07T16:16:14.566 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.566 INFO:tasks.workunit.client.0.smithi100.stdout: "stored": { 2024-04-07T16:16:14.566 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 311623 2024-04-07T16:16:14.566 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.566 INFO:tasks.workunit.client.0.smithi100.stdout: "truncate_lat": { 2024-04-07T16:16:14.566 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 31, 2024-04-07T16:16:14.567 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 218105 2024-04-07T16:16:14.567 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.567 INFO:tasks.workunit.client.0.smithi100.stdout: "txc_commit_lat": { 2024-04-07T16:16:14.567 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 269, 2024-04-07T16:16:14.567 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 287805482 2024-04-07T16:16:14.567 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.567 INFO:tasks.workunit.client.0.smithi100.stdout: "txc_submit_lat": { 2024-04-07T16:16:14.567 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 269, 2024-04-07T16:16:14.567 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 23169788 2024-04-07T16:16:14.567 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.567 INFO:tasks.workunit.client.0.smithi100.stdout: "txc_throttle_lat": { 2024-04-07T16:16:14.567 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 269, 2024-04-07T16:16:14.567 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3200638 2024-04-07T16:16:14.567 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:14.567 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.568 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache": { 2024-04-07T16:16:14.568 INFO:tasks.workunit.client.0.smithi100.stdout: "cache_bytes": { 2024-04-07T16:16:14.568 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1900990924 2024-04-07T16:16:14.568 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.568 INFO:tasks.workunit.client.0.smithi100.stdout: "heap_bytes": { 2024-04-07T16:16:14.568 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 43433984 2024-04-07T16:16:14.568 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.568 INFO:tasks.workunit.client.0.smithi100.stdout: "mapped_bytes": { 2024-04-07T16:16:14.568 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 42459136 2024-04-07T16:16:14.568 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.568 INFO:tasks.workunit.client.0.smithi100.stdout: "target_bytes": { 2024-04-07T16:16:14.568 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3183879168 2024-04-07T16:16:14.568 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.568 INFO:tasks.workunit.client.0.smithi100.stdout: "unmapped_bytes": { 2024-04-07T16:16:14.568 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 974848 2024-04-07T16:16:14.569 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:14.569 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.569 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:data": { 2024-04-07T16:16:14.569 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T16:16:14.569 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 159383552 2024-04-07T16:16:14.569 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.569 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T16:16:14.569 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.569 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.569 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T16:16:14.569 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.569 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.569 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T16:16:14.569 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 87871109 2024-04-07T16:16:14.569 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.569 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T16:16:14.570 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.570 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.570 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T16:16:14.570 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.570 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.570 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T16:16:14.570 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.570 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.570 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T16:16:14.570 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.570 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.570 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T16:16:14.570 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.570 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.570 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T16:16:14.571 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.571 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.571 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T16:16:14.571 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.571 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.571 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T16:16:14.571 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.571 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.571 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T16:16:14.571 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.571 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.571 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T16:16:14.571 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 71512443 2024-04-07T16:16:14.571 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:14.571 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.572 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:kv": { 2024-04-07T16:16:14.572 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T16:16:14.572 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 796917760 2024-04-07T16:16:14.572 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.572 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T16:16:14.572 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 67108864 2024-04-07T16:16:14.572 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.572 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T16:16:14.572 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.572 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.572 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T16:16:14.572 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 659033321 2024-04-07T16:16:14.572 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.572 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T16:16:14.572 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.573 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.573 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T16:16:14.573 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.573 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.573 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T16:16:14.573 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.573 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.573 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T16:16:14.573 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.573 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.573 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T16:16:14.573 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.573 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.573 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T16:16:14.573 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2128 2024-04-07T16:16:14.573 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.574 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T16:16:14.574 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.574 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.574 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T16:16:14.574 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.574 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.574 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T16:16:14.574 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.574 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.574 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T16:16:14.574 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 70773447 2024-04-07T16:16:14.574 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:14.574 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.574 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:kv_onode": { 2024-04-07T16:16:14.575 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T16:16:14.575 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 192937984 2024-04-07T16:16:14.575 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.575 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T16:16:14.575 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 67108864 2024-04-07T16:16:14.575 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.575 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T16:16:14.575 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 464 2024-04-07T16:16:14.575 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.575 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T16:16:14.575 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 58580739 2024-04-07T16:16:14.575 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.575 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T16:16:14.575 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.576 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.576 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T16:16:14.576 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.576 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.576 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T16:16:14.576 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.576 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.576 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T16:16:14.576 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.576 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.576 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T16:16:14.576 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.576 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.576 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T16:16:14.576 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.577 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.577 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T16:16:14.577 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.577 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.577 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T16:16:14.577 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.577 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.577 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T16:16:14.577 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.577 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.577 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T16:16:14.577 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 67247917 2024-04-07T16:16:14.577 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:14.577 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.577 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:meta": { 2024-04-07T16:16:14.578 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T16:16:14.578 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 729808896 2024-04-07T16:16:14.578 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.578 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T16:16:14.578 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.578 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.578 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T16:16:14.578 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.578 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.578 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T16:16:14.578 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 659033321 2024-04-07T16:16:14.578 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.578 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T16:16:14.578 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.579 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.579 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T16:16:14.579 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.579 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.579 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T16:16:14.579 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.579 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.579 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T16:16:14.579 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.579 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.579 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T16:16:14.579 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.579 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.579 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T16:16:14.579 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 224594 2024-04-07T16:16:14.579 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.580 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T16:16:14.580 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.580 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.580 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T16:16:14.580 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.580 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.580 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T16:16:14.580 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.580 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.580 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T16:16:14.580 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 70550981 2024-04-07T16:16:14.580 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:14.580 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.580 INFO:tasks.workunit.client.0.smithi100.stdout: "objecter": { 2024-04-07T16:16:14.580 INFO:tasks.workunit.client.0.smithi100.stdout: "op_active": { 2024-04-07T16:16:14.581 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.581 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.581 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r": { 2024-04-07T16:16:14.581 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.581 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.581 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rmw": { 2024-04-07T16:16:14.581 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.581 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.581 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w": { 2024-04-07T16:16:14.581 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.581 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:14.581 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.581 INFO:tasks.workunit.client.0.smithi100.stdout: "osd": { 2024-04-07T16:16:14.581 INFO:tasks.workunit.client.0.smithi100.stdout: "numpg": { 2024-04-07T16:16:14.582 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 42 2024-04-07T16:16:14.582 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.582 INFO:tasks.workunit.client.0.smithi100.stdout: "numpg_removing": { 2024-04-07T16:16:14.582 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.582 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.582 INFO:tasks.workunit.client.0.smithi100.stdout: "op": { 2024-04-07T16:16:14.582 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.582 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.582 INFO:tasks.workunit.client.0.smithi100.stdout: "op_delayed_degraded": { 2024-04-07T16:16:14.582 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.582 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.582 INFO:tasks.workunit.client.0.smithi100.stdout: "op_delayed_unreadable": { 2024-04-07T16:16:14.582 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.582 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.583 INFO:tasks.workunit.client.0.smithi100.stdout: "op_in_bytes": { 2024-04-07T16:16:14.583 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.583 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.583 INFO:tasks.workunit.client.0.smithi100.stdout: "op_latency": { 2024-04-07T16:16:14.583 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.583 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.583 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.583 INFO:tasks.workunit.client.0.smithi100.stdout: "op_out_bytes": { 2024-04-07T16:16:14.583 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.583 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.583 INFO:tasks.workunit.client.0.smithi100.stdout: "op_prepare_latency": { 2024-04-07T16:16:14.583 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.583 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.583 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.583 INFO:tasks.workunit.client.0.smithi100.stdout: "op_process_latency": { 2024-04-07T16:16:14.584 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.584 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.584 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.584 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r": { 2024-04-07T16:16:14.584 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.584 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.584 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_latency": { 2024-04-07T16:16:14.584 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.584 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.584 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.584 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_latency_out_bytes_histogram": { 2024-04-07T16:16:14.584 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.584 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.585 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_out_bytes": { 2024-04-07T16:16:14.585 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.585 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.585 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_prepare_latency": { 2024-04-07T16:16:14.585 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.585 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.585 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.585 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_process_latency": { 2024-04-07T16:16:14.585 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.585 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.585 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.585 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw": { 2024-04-07T16:16:14.585 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.585 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.585 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_in_bytes": { 2024-04-07T16:16:14.586 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.586 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.586 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_latency": { 2024-04-07T16:16:14.586 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.586 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.586 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.586 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_latency_in_bytes_histogram": { 2024-04-07T16:16:14.586 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.586 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.586 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_latency_out_bytes_histogram": { 2024-04-07T16:16:14.586 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.586 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.586 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_out_bytes": { 2024-04-07T16:16:14.587 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.587 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.587 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_prepare_latency": { 2024-04-07T16:16:14.587 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.587 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.587 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.587 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_process_latency": { 2024-04-07T16:16:14.587 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.587 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.587 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.587 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w": { 2024-04-07T16:16:14.587 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.587 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.587 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_in_bytes": { 2024-04-07T16:16:14.588 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.588 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.588 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_latency": { 2024-04-07T16:16:14.588 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.588 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.588 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.588 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_latency_in_bytes_histogram": { 2024-04-07T16:16:14.588 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.588 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.588 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_prepare_latency": { 2024-04-07T16:16:14.588 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.588 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.588 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.588 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_process_latency": { 2024-04-07T16:16:14.589 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.589 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.589 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.589 INFO:tasks.workunit.client.0.smithi100.stdout: "op_wip": { 2024-04-07T16:16:14.589 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.589 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.589 INFO:tasks.workunit.client.0.smithi100.stdout: "recovery_bytes": { 2024-04-07T16:16:14.589 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.589 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.589 INFO:tasks.workunit.client.0.smithi100.stdout: "recovery_ops": { 2024-04-07T16:16:14.589 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.589 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.589 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_bytes": { 2024-04-07T16:16:14.589 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 95995035648 2024-04-07T16:16:14.589 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.590 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_bytes_used": { 2024-04-07T16:16:14.590 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 28385280 2024-04-07T16:16:14.590 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:14.590 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:14.590 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.590 INFO:tasks.workunit.client.0.smithi100.stdout: "osd.6": { 2024-04-07T16:16:14.590 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs": { 2024-04-07T16:16:14.590 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_slow_fallback": { 2024-04-07T16:16:14.590 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.590 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.590 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_slow_size_fallback": { 2024-04-07T16:16:14.590 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.590 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.590 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit_db": { 2024-04-07T16:16:14.591 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 65536 2024-04-07T16:16:14.591 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.591 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit_main": { 2024-04-07T16:16:14.591 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.591 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.591 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit_wal": { 2024-04-07T16:16:14.591 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.591 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.591 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_written_slow": { 2024-04-07T16:16:14.591 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.591 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.591 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_written_sst": { 2024-04-07T16:16:14.591 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 12288 2024-04-07T16:16:14.591 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.591 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_written_wal": { 2024-04-07T16:16:14.592 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1208320 2024-04-07T16:16:14.592 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.592 INFO:tasks.workunit.client.0.smithi100.stdout: "compact_lat": { 2024-04-07T16:16:14.592 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.592 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.592 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.592 INFO:tasks.workunit.client.0.smithi100.stdout: "compact_lock_lat": { 2024-04-07T16:16:14.592 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.592 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.592 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.592 INFO:tasks.workunit.client.0.smithi100.stdout: "db_total_bytes": { 2024-04-07T16:16:14.592 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 95995035648 2024-04-07T16:16:14.592 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.592 INFO:tasks.workunit.client.0.smithi100.stdout: "db_used_bytes": { 2024-04-07T16:16:14.592 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 27656192 2024-04-07T16:16:14.593 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.593 INFO:tasks.workunit.client.0.smithi100.stdout: "log_bytes": { 2024-04-07T16:16:14.593 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 921600 2024-04-07T16:16:14.593 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.593 INFO:tasks.workunit.client.0.smithi100.stdout: "logged_bytes": { 2024-04-07T16:16:14.593 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 782336 2024-04-07T16:16:14.593 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.593 INFO:tasks.workunit.client.0.smithi100.stdout: "max_bytes_db": { 2024-04-07T16:16:14.593 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 31850496 2024-04-07T16:16:14.593 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.593 INFO:tasks.workunit.client.0.smithi100.stdout: "max_bytes_slow": { 2024-04-07T16:16:14.593 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.593 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.593 INFO:tasks.workunit.client.0.smithi100.stdout: "max_bytes_wal": { 2024-04-07T16:16:14.594 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.594 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.594 INFO:tasks.workunit.client.0.smithi100.stdout: "num_files": { 2024-04-07T16:16:14.594 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 14 2024-04-07T16:16:14.594 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.594 INFO:tasks.workunit.client.0.smithi100.stdout: "read_bytes": { 2024-04-07T16:16:14.594 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 206272 2024-04-07T16:16:14.594 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.594 INFO:tasks.workunit.client.0.smithi100.stdout: "read_count": { 2024-04-07T16:16:14.594 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 23 2024-04-07T16:16:14.594 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.594 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes": { 2024-04-07T16:16:14.594 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1282048 2024-04-07T16:16:14.594 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.595 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes_db": { 2024-04-07T16:16:14.595 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1286144 2024-04-07T16:16:14.595 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.595 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes_slow": { 2024-04-07T16:16:14.595 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.595 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.595 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes_wal": { 2024-04-07T16:16:14.595 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.595 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.595 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_count": { 2024-04-07T16:16:14.595 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 15 2024-04-07T16:16:14.595 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.595 INFO:tasks.workunit.client.0.smithi100.stdout: "read_prefetch_bytes": { 2024-04-07T16:16:14.596 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4913 2024-04-07T16:16:14.596 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.596 INFO:tasks.workunit.client.0.smithi100.stdout: "read_prefetch_count": { 2024-04-07T16:16:14.596 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4 2024-04-07T16:16:14.596 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.596 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_buffer_bytes": { 2024-04-07T16:16:14.596 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5504 2024-04-07T16:16:14.596 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.596 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_buffer_count": { 2024-04-07T16:16:14.596 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 24 2024-04-07T16:16:14.596 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.596 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_bytes": { 2024-04-07T16:16:14.596 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5504 2024-04-07T16:16:14.596 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.596 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_count": { 2024-04-07T16:16:14.597 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 24 2024-04-07T16:16:14.597 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.597 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes": { 2024-04-07T16:16:14.597 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.597 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.597 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes_db": { 2024-04-07T16:16:14.597 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.597 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.597 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes_slow": { 2024-04-07T16:16:14.597 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.597 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.597 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes_wal": { 2024-04-07T16:16:14.597 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.597 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.597 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_count": { 2024-04-07T16:16:14.598 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.598 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.598 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_total_bytes": { 2024-04-07T16:16:14.598 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.598 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.598 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_used_bytes": { 2024-04-07T16:16:14.598 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.598 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.598 INFO:tasks.workunit.client.0.smithi100.stdout: "wal_total_bytes": { 2024-04-07T16:16:14.598 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.598 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.598 INFO:tasks.workunit.client.0.smithi100.stdout: "wal_used_bytes": { 2024-04-07T16:16:14.598 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.598 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.599 INFO:tasks.workunit.client.0.smithi100.stdout: "write_bytes": { 2024-04-07T16:16:14.599 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2068480 2024-04-07T16:16:14.599 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:14.599 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.599 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore": { 2024-04-07T16:16:14.599 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit": { 2024-04-07T16:16:14.599 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4096 2024-04-07T16:16:14.599 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.599 INFO:tasks.workunit.client.0.smithi100.stdout: "allocated": { 2024-04-07T16:16:14.599 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1314816 2024-04-07T16:16:14.599 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.599 INFO:tasks.workunit.client.0.smithi100.stdout: "clist_lat": { 2024-04-07T16:16:14.599 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.599 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.599 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.600 INFO:tasks.workunit.client.0.smithi100.stdout: "compress_lat": { 2024-04-07T16:16:14.600 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.600 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.600 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.600 INFO:tasks.workunit.client.0.smithi100.stdout: "compressed": { 2024-04-07T16:16:14.600 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.600 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.600 INFO:tasks.workunit.client.0.smithi100.stdout: "compressed_allocated": { 2024-04-07T16:16:14.600 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.600 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.600 INFO:tasks.workunit.client.0.smithi100.stdout: "compressed_original": { 2024-04-07T16:16:14.600 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.600 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.600 INFO:tasks.workunit.client.0.smithi100.stdout: "csum_lat": { 2024-04-07T16:16:14.600 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 3, 2024-04-07T16:16:14.601 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 7548 2024-04-07T16:16:14.601 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.601 INFO:tasks.workunit.client.0.smithi100.stdout: "decompress_lat": { 2024-04-07T16:16:14.601 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.601 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.601 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.601 INFO:tasks.workunit.client.0.smithi100.stdout: "fragmentation_micros": { 2024-04-07T16:16:14.601 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.601 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.601 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_commit_lat": { 2024-04-07T16:16:14.601 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 178, 2024-04-07T16:16:14.601 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 59820077 2024-04-07T16:16:14.601 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.601 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_final_lat": { 2024-04-07T16:16:14.602 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 159, 2024-04-07T16:16:14.602 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 8681071 2024-04-07T16:16:14.602 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.602 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_flush_lat": { 2024-04-07T16:16:14.602 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 178, 2024-04-07T16:16:14.602 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 236301 2024-04-07T16:16:14.602 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.602 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_sync_lat": { 2024-04-07T16:16:14.602 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 178, 2024-04-07T16:16:14.602 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 60056378 2024-04-07T16:16:14.602 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.602 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_get_keys_lat": { 2024-04-07T16:16:14.602 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.602 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.603 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.603 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_get_values_lat": { 2024-04-07T16:16:14.603 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1, 2024-04-07T16:16:14.603 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 54597 2024-04-07T16:16:14.603 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.603 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_lower_bound_lat": { 2024-04-07T16:16:14.603 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1, 2024-04-07T16:16:14.603 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2371 2024-04-07T16:16:14.603 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.603 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_next_lat": { 2024-04-07T16:16:14.603 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.603 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.603 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.603 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_seek_to_first_lat": { 2024-04-07T16:16:14.604 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.604 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.604 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.604 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_upper_bound_lat": { 2024-04-07T16:16:14.604 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 2, 2024-04-07T16:16:14.604 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 7421 2024-04-07T16:16:14.604 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.604 INFO:tasks.workunit.client.0.smithi100.stdout: "onode_hits": { 2024-04-07T16:16:14.604 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 319 2024-04-07T16:16:14.604 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.604 INFO:tasks.workunit.client.0.smithi100.stdout: "onode_misses": { 2024-04-07T16:16:14.604 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 248 2024-04-07T16:16:14.604 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.604 INFO:tasks.workunit.client.0.smithi100.stdout: "read_lat": { 2024-04-07T16:16:14.605 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 89, 2024-04-07T16:16:14.605 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 947386 2024-04-07T16:16:14.605 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.605 INFO:tasks.workunit.client.0.smithi100.stdout: "read_onode_meta_lat": { 2024-04-07T16:16:14.605 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 137, 2024-04-07T16:16:14.605 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 326760 2024-04-07T16:16:14.605 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.605 INFO:tasks.workunit.client.0.smithi100.stdout: "read_wait_aio_lat": { 2024-04-07T16:16:14.605 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 48, 2024-04-07T16:16:14.605 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 454246 2024-04-07T16:16:14.605 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.605 INFO:tasks.workunit.client.0.smithi100.stdout: "reads_with_retries": { 2024-04-07T16:16:14.605 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.605 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.605 INFO:tasks.workunit.client.0.smithi100.stdout: "remove_lat": { 2024-04-07T16:16:14.606 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.606 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.606 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.606 INFO:tasks.workunit.client.0.smithi100.stdout: "state_aio_wait_lat": { 2024-04-07T16:16:14.606 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 253, 2024-04-07T16:16:14.606 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5669588 2024-04-07T16:16:14.606 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.606 INFO:tasks.workunit.client.0.smithi100.stdout: "state_deferred_aio_wait_lat": { 2024-04-07T16:16:14.606 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 34, 2024-04-07T16:16:14.606 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3229934 2024-04-07T16:16:14.606 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.606 INFO:tasks.workunit.client.0.smithi100.stdout: "state_deferred_cleanup_lat": { 2024-04-07T16:16:14.606 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 31, 2024-04-07T16:16:14.606 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 390577146387 2024-04-07T16:16:14.607 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.607 INFO:tasks.workunit.client.0.smithi100.stdout: "state_deferred_queued_lat": { 2024-04-07T16:16:14.607 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 34, 2024-04-07T16:16:14.607 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 64017169880 2024-04-07T16:16:14.607 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.607 INFO:tasks.workunit.client.0.smithi100.stdout: "state_done_lat": { 2024-04-07T16:16:14.607 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 250, 2024-04-07T16:16:14.607 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 678466 2024-04-07T16:16:14.607 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.607 INFO:tasks.workunit.client.0.smithi100.stdout: "state_finishing_lat": { 2024-04-07T16:16:14.607 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 250, 2024-04-07T16:16:14.607 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 40036 2024-04-07T16:16:14.607 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.607 INFO:tasks.workunit.client.0.smithi100.stdout: "state_io_done_lat": { 2024-04-07T16:16:14.607 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 253, 2024-04-07T16:16:14.608 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 421689 2024-04-07T16:16:14.608 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.608 INFO:tasks.workunit.client.0.smithi100.stdout: "state_kv_commiting_lat": { 2024-04-07T16:16:14.608 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 253, 2024-04-07T16:16:14.608 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 136263078 2024-04-07T16:16:14.608 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.608 INFO:tasks.workunit.client.0.smithi100.stdout: "state_kv_done_lat": { 2024-04-07T16:16:14.608 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 253, 2024-04-07T16:16:14.608 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 135906 2024-04-07T16:16:14.608 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.608 INFO:tasks.workunit.client.0.smithi100.stdout: "state_kv_queued_lat": { 2024-04-07T16:16:14.608 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 253, 2024-04-07T16:16:14.608 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 97488277 2024-04-07T16:16:14.608 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.608 INFO:tasks.workunit.client.0.smithi100.stdout: "state_prepare_lat": { 2024-04-07T16:16:14.609 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 253, 2024-04-07T16:16:14.609 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 17188337 2024-04-07T16:16:14.609 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.609 INFO:tasks.workunit.client.0.smithi100.stdout: "stored": { 2024-04-07T16:16:14.609 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 902232 2024-04-07T16:16:14.609 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.609 INFO:tasks.workunit.client.0.smithi100.stdout: "truncate_lat": { 2024-04-07T16:16:14.609 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 27, 2024-04-07T16:16:14.609 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 191944 2024-04-07T16:16:14.609 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.609 INFO:tasks.workunit.client.0.smithi100.stdout: "txc_commit_lat": { 2024-04-07T16:16:14.609 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 253, 2024-04-07T16:16:14.609 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 257088473 2024-04-07T16:16:14.609 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.609 INFO:tasks.workunit.client.0.smithi100.stdout: "txc_submit_lat": { 2024-04-07T16:16:14.610 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 253, 2024-04-07T16:16:14.610 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 20384955 2024-04-07T16:16:14.610 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.610 INFO:tasks.workunit.client.0.smithi100.stdout: "txc_throttle_lat": { 2024-04-07T16:16:14.610 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 253, 2024-04-07T16:16:14.610 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2299697 2024-04-07T16:16:14.610 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:14.610 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.610 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache": { 2024-04-07T16:16:14.610 INFO:tasks.workunit.client.0.smithi100.stdout: "cache_bytes": { 2024-04-07T16:16:14.610 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1900990924 2024-04-07T16:16:14.610 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.610 INFO:tasks.workunit.client.0.smithi100.stdout: "heap_bytes": { 2024-04-07T16:16:14.610 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 43581440 2024-04-07T16:16:14.610 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.611 INFO:tasks.workunit.client.0.smithi100.stdout: "mapped_bytes": { 2024-04-07T16:16:14.611 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 42827776 2024-04-07T16:16:14.611 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.611 INFO:tasks.workunit.client.0.smithi100.stdout: "target_bytes": { 2024-04-07T16:16:14.611 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3183879168 2024-04-07T16:16:14.611 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.611 INFO:tasks.workunit.client.0.smithi100.stdout: "unmapped_bytes": { 2024-04-07T16:16:14.611 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 753664 2024-04-07T16:16:14.611 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:14.611 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.611 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:data": { 2024-04-07T16:16:14.611 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T16:16:14.611 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 159383552 2024-04-07T16:16:14.611 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.611 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T16:16:14.612 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.612 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.612 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T16:16:14.612 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.612 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.612 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T16:16:14.612 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 87870852 2024-04-07T16:16:14.612 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.612 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T16:16:14.612 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.612 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.612 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T16:16:14.612 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.612 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.612 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T16:16:14.613 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.613 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.613 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T16:16:14.613 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.613 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.613 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T16:16:14.613 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.613 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.613 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T16:16:14.613 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.613 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.613 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T16:16:14.613 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.613 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.613 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T16:16:14.614 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.614 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.614 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T16:16:14.614 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.614 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.614 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T16:16:14.614 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 71512700 2024-04-07T16:16:14.614 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:14.614 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.614 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:kv": { 2024-04-07T16:16:14.614 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T16:16:14.614 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 796917760 2024-04-07T16:16:14.615 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.615 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T16:16:14.615 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 67108864 2024-04-07T16:16:14.615 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.615 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T16:16:14.615 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.615 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.615 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T16:16:14.615 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 659031394 2024-04-07T16:16:14.615 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.615 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T16:16:14.615 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.615 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.615 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T16:16:14.616 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.616 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.616 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T16:16:14.616 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.616 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.616 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T16:16:14.616 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.616 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.616 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T16:16:14.616 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.616 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.616 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T16:16:14.616 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2128 2024-04-07T16:16:14.616 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.616 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T16:16:14.617 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.617 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.617 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T16:16:14.617 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.617 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.617 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T16:16:14.617 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.617 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.617 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T16:16:14.617 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 70775374 2024-04-07T16:16:14.617 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:14.617 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.617 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:kv_onode": { 2024-04-07T16:16:14.617 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T16:16:14.617 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 192937984 2024-04-07T16:16:14.618 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.618 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T16:16:14.618 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 67108864 2024-04-07T16:16:14.618 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.618 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T16:16:14.618 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 464 2024-04-07T16:16:14.618 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.618 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T16:16:14.618 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 58580568 2024-04-07T16:16:14.618 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.618 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T16:16:14.618 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.618 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.618 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T16:16:14.619 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.619 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.619 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T16:16:14.619 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.619 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.619 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T16:16:14.619 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.619 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.619 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T16:16:14.619 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.619 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.619 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T16:16:14.619 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.619 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.620 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T16:16:14.620 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.620 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.620 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T16:16:14.620 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.620 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.620 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T16:16:14.620 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.620 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.620 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T16:16:14.620 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 67248088 2024-04-07T16:16:14.620 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:14.620 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.620 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:meta": { 2024-04-07T16:16:14.620 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T16:16:14.621 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 729808896 2024-04-07T16:16:14.621 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.621 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T16:16:14.621 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.621 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.621 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T16:16:14.621 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.621 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.621 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T16:16:14.621 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 659031394 2024-04-07T16:16:14.621 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.621 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T16:16:14.621 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.621 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.621 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T16:16:14.622 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.622 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.622 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T16:16:14.622 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.622 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.622 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T16:16:14.622 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.622 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.622 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T16:16:14.622 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.622 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.622 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T16:16:14.622 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 228090 2024-04-07T16:16:14.622 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.623 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T16:16:14.623 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.623 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.623 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T16:16:14.623 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.623 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.623 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T16:16:14.623 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.623 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.623 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T16:16:14.623 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 70549412 2024-04-07T16:16:14.623 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:14.623 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.623 INFO:tasks.workunit.client.0.smithi100.stdout: "objecter": { 2024-04-07T16:16:14.623 INFO:tasks.workunit.client.0.smithi100.stdout: "op_active": { 2024-04-07T16:16:14.624 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.624 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.624 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r": { 2024-04-07T16:16:14.624 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.624 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.624 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rmw": { 2024-04-07T16:16:14.624 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.624 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.624 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w": { 2024-04-07T16:16:14.624 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.624 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:14.624 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.624 INFO:tasks.workunit.client.0.smithi100.stdout: "osd": { 2024-04-07T16:16:14.624 INFO:tasks.workunit.client.0.smithi100.stdout: "numpg": { 2024-04-07T16:16:14.625 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 39 2024-04-07T16:16:14.625 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.625 INFO:tasks.workunit.client.0.smithi100.stdout: "numpg_removing": { 2024-04-07T16:16:14.625 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.625 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.625 INFO:tasks.workunit.client.0.smithi100.stdout: "op": { 2024-04-07T16:16:14.625 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 13 2024-04-07T16:16:14.625 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.625 INFO:tasks.workunit.client.0.smithi100.stdout: "op_delayed_degraded": { 2024-04-07T16:16:14.625 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.625 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.625 INFO:tasks.workunit.client.0.smithi100.stdout: "op_delayed_unreadable": { 2024-04-07T16:16:14.625 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.625 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.625 INFO:tasks.workunit.client.0.smithi100.stdout: "op_in_bytes": { 2024-04-07T16:16:14.625 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2812 2024-04-07T16:16:14.626 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.626 INFO:tasks.workunit.client.0.smithi100.stdout: "op_latency": { 2024-04-07T16:16:14.626 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 13, 2024-04-07T16:16:14.626 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 180979041 2024-04-07T16:16:14.626 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.626 INFO:tasks.workunit.client.0.smithi100.stdout: "op_out_bytes": { 2024-04-07T16:16:14.626 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.626 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.626 INFO:tasks.workunit.client.0.smithi100.stdout: "op_prepare_latency": { 2024-04-07T16:16:14.626 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 15, 2024-04-07T16:16:14.626 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 10137974 2024-04-07T16:16:14.626 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.626 INFO:tasks.workunit.client.0.smithi100.stdout: "op_process_latency": { 2024-04-07T16:16:14.626 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 13, 2024-04-07T16:16:14.626 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 165152222 2024-04-07T16:16:14.627 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.627 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r": { 2024-04-07T16:16:14.627 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.627 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.627 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_latency": { 2024-04-07T16:16:14.627 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.627 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.627 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.627 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_latency_out_bytes_histogram": { 2024-04-07T16:16:14.627 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.627 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.627 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_out_bytes": { 2024-04-07T16:16:14.627 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.627 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.627 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_prepare_latency": { 2024-04-07T16:16:14.628 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.628 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.628 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.628 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_process_latency": { 2024-04-07T16:16:14.628 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.628 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.628 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.628 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw": { 2024-04-07T16:16:14.628 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.628 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.628 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_in_bytes": { 2024-04-07T16:16:14.628 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.628 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.628 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_latency": { 2024-04-07T16:16:14.628 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.629 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.629 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.629 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_latency_in_bytes_histogram": { 2024-04-07T16:16:14.629 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.629 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.629 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_latency_out_bytes_histogram": { 2024-04-07T16:16:14.629 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.629 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.629 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_out_bytes": { 2024-04-07T16:16:14.629 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.629 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.629 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_prepare_latency": { 2024-04-07T16:16:14.629 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.629 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.630 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.630 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_process_latency": { 2024-04-07T16:16:14.630 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.630 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.630 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.630 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w": { 2024-04-07T16:16:14.630 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 13 2024-04-07T16:16:14.630 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.630 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_in_bytes": { 2024-04-07T16:16:14.630 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2812 2024-04-07T16:16:14.630 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.630 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_latency": { 2024-04-07T16:16:14.630 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 13, 2024-04-07T16:16:14.630 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 180979041 2024-04-07T16:16:14.630 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.630 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_latency_in_bytes_histogram": { 2024-04-07T16:16:14.631 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.631 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.631 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_prepare_latency": { 2024-04-07T16:16:14.631 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 15, 2024-04-07T16:16:14.631 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 10137974 2024-04-07T16:16:14.631 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.631 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_process_latency": { 2024-04-07T16:16:14.631 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 13, 2024-04-07T16:16:14.631 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 165152222 2024-04-07T16:16:14.631 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.631 INFO:tasks.workunit.client.0.smithi100.stdout: "op_wip": { 2024-04-07T16:16:14.631 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.631 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.631 INFO:tasks.workunit.client.0.smithi100.stdout: "recovery_bytes": { 2024-04-07T16:16:14.632 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.632 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.632 INFO:tasks.workunit.client.0.smithi100.stdout: "recovery_ops": { 2024-04-07T16:16:14.632 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1 2024-04-07T16:16:14.632 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.632 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_bytes": { 2024-04-07T16:16:14.632 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 95995035648 2024-04-07T16:16:14.632 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.632 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_bytes_used": { 2024-04-07T16:16:14.632 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 28979200 2024-04-07T16:16:14.632 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:14.632 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:14.632 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.632 INFO:tasks.workunit.client.0.smithi100.stdout: "osd.7": { 2024-04-07T16:16:14.633 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs": { 2024-04-07T16:16:14.633 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_slow_fallback": { 2024-04-07T16:16:14.633 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.633 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.633 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_slow_size_fallback": { 2024-04-07T16:16:14.633 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.633 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.633 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit_db": { 2024-04-07T16:16:14.633 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 65536 2024-04-07T16:16:14.633 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.633 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit_main": { 2024-04-07T16:16:14.633 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.633 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.633 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit_wal": { 2024-04-07T16:16:14.633 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.634 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.634 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_written_slow": { 2024-04-07T16:16:14.634 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.634 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.634 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_written_sst": { 2024-04-07T16:16:14.634 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 12288 2024-04-07T16:16:14.634 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.634 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_written_wal": { 2024-04-07T16:16:14.634 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 6361088 2024-04-07T16:16:14.634 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.634 INFO:tasks.workunit.client.0.smithi100.stdout: "compact_lat": { 2024-04-07T16:16:14.634 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.634 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.634 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.635 INFO:tasks.workunit.client.0.smithi100.stdout: "compact_lock_lat": { 2024-04-07T16:16:14.635 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.635 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.635 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.635 INFO:tasks.workunit.client.0.smithi100.stdout: "db_total_bytes": { 2024-04-07T16:16:14.635 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 95995035648 2024-04-07T16:16:14.635 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.635 INFO:tasks.workunit.client.0.smithi100.stdout: "db_used_bytes": { 2024-04-07T16:16:14.635 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 31850496 2024-04-07T16:16:14.635 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.635 INFO:tasks.workunit.client.0.smithi100.stdout: "log_bytes": { 2024-04-07T16:16:14.635 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3411968 2024-04-07T16:16:14.635 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.636 INFO:tasks.workunit.client.0.smithi100.stdout: "logged_bytes": { 2024-04-07T16:16:14.636 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3272704 2024-04-07T16:16:14.636 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.636 INFO:tasks.workunit.client.0.smithi100.stdout: "max_bytes_db": { 2024-04-07T16:16:14.636 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 31850496 2024-04-07T16:16:14.636 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.636 INFO:tasks.workunit.client.0.smithi100.stdout: "max_bytes_slow": { 2024-04-07T16:16:14.636 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.636 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.636 INFO:tasks.workunit.client.0.smithi100.stdout: "max_bytes_wal": { 2024-04-07T16:16:14.636 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.636 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.636 INFO:tasks.workunit.client.0.smithi100.stdout: "num_files": { 2024-04-07T16:16:14.636 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 14 2024-04-07T16:16:14.636 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.637 INFO:tasks.workunit.client.0.smithi100.stdout: "read_bytes": { 2024-04-07T16:16:14.637 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 206272 2024-04-07T16:16:14.637 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.637 INFO:tasks.workunit.client.0.smithi100.stdout: "read_count": { 2024-04-07T16:16:14.637 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 23 2024-04-07T16:16:14.637 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.637 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes": { 2024-04-07T16:16:14.637 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1282048 2024-04-07T16:16:14.637 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.637 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes_db": { 2024-04-07T16:16:14.637 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1286144 2024-04-07T16:16:14.637 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.637 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes_slow": { 2024-04-07T16:16:14.637 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.637 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.638 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes_wal": { 2024-04-07T16:16:14.638 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.638 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.638 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_count": { 2024-04-07T16:16:14.638 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 15 2024-04-07T16:16:14.638 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.638 INFO:tasks.workunit.client.0.smithi100.stdout: "read_prefetch_bytes": { 2024-04-07T16:16:14.638 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4913 2024-04-07T16:16:14.638 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.638 INFO:tasks.workunit.client.0.smithi100.stdout: "read_prefetch_count": { 2024-04-07T16:16:14.638 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4 2024-04-07T16:16:14.638 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.638 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_buffer_bytes": { 2024-04-07T16:16:14.638 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5500 2024-04-07T16:16:14.639 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.639 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_buffer_count": { 2024-04-07T16:16:14.639 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 24 2024-04-07T16:16:14.639 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.639 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_bytes": { 2024-04-07T16:16:14.639 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5500 2024-04-07T16:16:14.639 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.639 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_count": { 2024-04-07T16:16:14.639 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 24 2024-04-07T16:16:14.639 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.639 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes": { 2024-04-07T16:16:14.639 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.639 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.639 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes_db": { 2024-04-07T16:16:14.640 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.640 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.640 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes_slow": { 2024-04-07T16:16:14.640 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.640 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.640 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes_wal": { 2024-04-07T16:16:14.640 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.640 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.640 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_count": { 2024-04-07T16:16:14.640 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.640 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.640 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_total_bytes": { 2024-04-07T16:16:14.640 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.641 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.641 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_used_bytes": { 2024-04-07T16:16:14.641 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.641 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.641 INFO:tasks.workunit.client.0.smithi100.stdout: "wal_total_bytes": { 2024-04-07T16:16:14.641 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.641 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.641 INFO:tasks.workunit.client.0.smithi100.stdout: "wal_used_bytes": { 2024-04-07T16:16:14.641 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.641 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.641 INFO:tasks.workunit.client.0.smithi100.stdout: "write_bytes": { 2024-04-07T16:16:14.641 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 9711616 2024-04-07T16:16:14.641 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:14.641 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.642 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore": { 2024-04-07T16:16:14.642 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit": { 2024-04-07T16:16:14.642 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4096 2024-04-07T16:16:14.642 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.642 INFO:tasks.workunit.client.0.smithi100.stdout: "allocated": { 2024-04-07T16:16:14.642 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1310720 2024-04-07T16:16:14.642 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.642 INFO:tasks.workunit.client.0.smithi100.stdout: "clist_lat": { 2024-04-07T16:16:14.642 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 5, 2024-04-07T16:16:14.642 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 540485 2024-04-07T16:16:14.642 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.642 INFO:tasks.workunit.client.0.smithi100.stdout: "compress_lat": { 2024-04-07T16:16:14.642 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.642 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.642 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.643 INFO:tasks.workunit.client.0.smithi100.stdout: "compressed": { 2024-04-07T16:16:14.643 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.643 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.643 INFO:tasks.workunit.client.0.smithi100.stdout: "compressed_allocated": { 2024-04-07T16:16:14.643 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.643 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.643 INFO:tasks.workunit.client.0.smithi100.stdout: "compressed_original": { 2024-04-07T16:16:14.643 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.643 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.643 INFO:tasks.workunit.client.0.smithi100.stdout: "csum_lat": { 2024-04-07T16:16:14.643 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 7, 2024-04-07T16:16:14.643 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 38263 2024-04-07T16:16:14.643 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.643 INFO:tasks.workunit.client.0.smithi100.stdout: "decompress_lat": { 2024-04-07T16:16:14.643 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.644 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.644 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.644 INFO:tasks.workunit.client.0.smithi100.stdout: "fragmentation_micros": { 2024-04-07T16:16:14.644 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.644 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.644 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_commit_lat": { 2024-04-07T16:16:14.644 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 786, 2024-04-07T16:16:14.644 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 505301819 2024-04-07T16:16:14.644 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.644 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_final_lat": { 2024-04-07T16:16:14.644 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 772, 2024-04-07T16:16:14.644 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 32971405 2024-04-07T16:16:14.644 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.644 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_flush_lat": { 2024-04-07T16:16:14.645 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 786, 2024-04-07T16:16:14.645 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 936075 2024-04-07T16:16:14.645 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.645 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_sync_lat": { 2024-04-07T16:16:14.645 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 786, 2024-04-07T16:16:14.645 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 506237894 2024-04-07T16:16:14.645 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.645 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_get_keys_lat": { 2024-04-07T16:16:14.645 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.645 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.645 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.645 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_get_values_lat": { 2024-04-07T16:16:14.645 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 2, 2024-04-07T16:16:14.645 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 125983 2024-04-07T16:16:14.646 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.646 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_lower_bound_lat": { 2024-04-07T16:16:14.646 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.646 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.646 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.646 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_next_lat": { 2024-04-07T16:16:14.646 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.646 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.646 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.646 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_seek_to_first_lat": { 2024-04-07T16:16:14.646 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:14.646 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.646 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.646 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_upper_bound_lat": { 2024-04-07T16:16:14.647 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 2, 2024-04-07T16:16:14.647 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 6034 2024-04-07T16:16:14.647 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.647 INFO:tasks.workunit.client.0.smithi100.stdout: "onode_hits": { 2024-04-07T16:16:14.647 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3493 2024-04-07T16:16:14.647 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.647 INFO:tasks.workunit.client.0.smithi100.stdout: "onode_misses": { 2024-04-07T16:16:14.647 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 368 2024-04-07T16:16:14.647 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.647 INFO:tasks.workunit.client.0.smithi100.stdout: "read_lat": { 2024-04-07T16:16:14.647 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 94, 2024-04-07T16:16:14.647 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2646182 2024-04-07T16:16:14.647 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.647 INFO:tasks.workunit.client.0.smithi100.stdout: "read_onode_meta_lat": { 2024-04-07T16:16:14.647 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 136, 2024-04-07T16:16:14.648 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 338226 2024-04-07T16:16:14.648 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.648 INFO:tasks.workunit.client.0.smithi100.stdout: "read_wait_aio_lat": { 2024-04-07T16:16:14.648 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 42, 2024-04-07T16:16:14.648 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1770471 2024-04-07T16:16:14.648 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.648 INFO:tasks.workunit.client.0.smithi100.stdout: "reads_with_retries": { 2024-04-07T16:16:14.648 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.648 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.648 INFO:tasks.workunit.client.0.smithi100.stdout: "remove_lat": { 2024-04-07T16:16:14.648 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 102, 2024-04-07T16:16:14.648 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5813187 2024-04-07T16:16:14.648 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.648 INFO:tasks.workunit.client.0.smithi100.stdout: "state_aio_wait_lat": { 2024-04-07T16:16:14.649 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 3378, 2024-04-07T16:16:14.649 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3775556449 2024-04-07T16:16:14.649 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.649 INFO:tasks.workunit.client.0.smithi100.stdout: "state_deferred_aio_wait_lat": { 2024-04-07T16:16:14.649 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 20, 2024-04-07T16:16:14.649 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 22546871 2024-04-07T16:16:14.649 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.649 INFO:tasks.workunit.client.0.smithi100.stdout: "state_deferred_cleanup_lat": { 2024-04-07T16:16:14.649 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 19, 2024-04-07T16:16:14.649 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 124933302941 2024-04-07T16:16:14.649 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.649 INFO:tasks.workunit.client.0.smithi100.stdout: "state_deferred_queued_lat": { 2024-04-07T16:16:14.649 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 20, 2024-04-07T16:16:14.649 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 36047466444 2024-04-07T16:16:14.650 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.650 INFO:tasks.workunit.client.0.smithi100.stdout: "state_done_lat": { 2024-04-07T16:16:14.650 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 3377, 2024-04-07T16:16:14.650 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1052881861 2024-04-07T16:16:14.650 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.650 INFO:tasks.workunit.client.0.smithi100.stdout: "state_finishing_lat": { 2024-04-07T16:16:14.650 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 3377, 2024-04-07T16:16:14.650 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 279338 2024-04-07T16:16:14.650 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.650 INFO:tasks.workunit.client.0.smithi100.stdout: "state_io_done_lat": { 2024-04-07T16:16:14.650 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 3378, 2024-04-07T16:16:14.650 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 124893327 2024-04-07T16:16:14.650 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.650 INFO:tasks.workunit.client.0.smithi100.stdout: "state_kv_commiting_lat": { 2024-04-07T16:16:14.650 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 3378, 2024-04-07T16:16:14.651 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2913250367 2024-04-07T16:16:14.651 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.651 INFO:tasks.workunit.client.0.smithi100.stdout: "state_kv_done_lat": { 2024-04-07T16:16:14.651 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 3378, 2024-04-07T16:16:14.651 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 674161 2024-04-07T16:16:14.651 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.651 INFO:tasks.workunit.client.0.smithi100.stdout: "state_kv_queued_lat": { 2024-04-07T16:16:14.651 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 3378, 2024-04-07T16:16:14.651 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1742584173 2024-04-07T16:16:14.651 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.651 INFO:tasks.workunit.client.0.smithi100.stdout: "state_prepare_lat": { 2024-04-07T16:16:14.651 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 3378, 2024-04-07T16:16:14.651 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 362538307 2024-04-07T16:16:14.651 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.651 INFO:tasks.workunit.client.0.smithi100.stdout: "stored": { 2024-04-07T16:16:14.652 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 900204 2024-04-07T16:16:14.652 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.652 INFO:tasks.workunit.client.0.smithi100.stdout: "truncate_lat": { 2024-04-07T16:16:14.652 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 19, 2024-04-07T16:16:14.652 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 125892 2024-04-07T16:16:14.652 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.652 INFO:tasks.workunit.client.0.smithi100.stdout: "txc_commit_lat": { 2024-04-07T16:16:14.652 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 3378, 2024-04-07T16:16:14.652 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 8919154076 2024-04-07T16:16:14.652 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.652 INFO:tasks.workunit.client.0.smithi100.stdout: "txc_submit_lat": { 2024-04-07T16:16:14.652 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 3378, 2024-04-07T16:16:14.652 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 420768261 2024-04-07T16:16:14.653 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.653 INFO:tasks.workunit.client.0.smithi100.stdout: "txc_throttle_lat": { 2024-04-07T16:16:14.653 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 3378, 2024-04-07T16:16:14.653 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 232599958 2024-04-07T16:16:14.653 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:14.653 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.653 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache": { 2024-04-07T16:16:14.653 INFO:tasks.workunit.client.0.smithi100.stdout: "cache_bytes": { 2024-04-07T16:16:14.653 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1900990924 2024-04-07T16:16:14.653 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.653 INFO:tasks.workunit.client.0.smithi100.stdout: "heap_bytes": { 2024-04-07T16:16:14.653 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 46792704 2024-04-07T16:16:14.653 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.653 INFO:tasks.workunit.client.0.smithi100.stdout: "mapped_bytes": { 2024-04-07T16:16:14.653 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 46129152 2024-04-07T16:16:14.654 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.654 INFO:tasks.workunit.client.0.smithi100.stdout: "target_bytes": { 2024-04-07T16:16:14.654 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3183879168 2024-04-07T16:16:14.654 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.654 INFO:tasks.workunit.client.0.smithi100.stdout: "unmapped_bytes": { 2024-04-07T16:16:14.654 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 663552 2024-04-07T16:16:14.654 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:14.654 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.654 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:data": { 2024-04-07T16:16:14.654 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T16:16:14.654 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 159383552 2024-04-07T16:16:14.654 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.654 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T16:16:14.654 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.655 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.655 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T16:16:14.655 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.655 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.655 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T16:16:14.655 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 87855116 2024-04-07T16:16:14.655 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.655 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T16:16:14.655 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.655 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.655 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T16:16:14.655 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.655 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.655 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T16:16:14.655 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.655 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.656 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T16:16:14.656 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.656 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.656 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T16:16:14.656 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.656 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.656 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T16:16:14.656 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 143360 2024-04-07T16:16:14.656 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.656 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T16:16:14.656 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.656 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.656 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T16:16:14.656 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.657 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.657 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T16:16:14.657 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.657 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.657 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T16:16:14.657 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 71385076 2024-04-07T16:16:14.657 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:14.657 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.657 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:kv": { 2024-04-07T16:16:14.657 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T16:16:14.657 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 796917760 2024-04-07T16:16:14.657 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.657 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T16:16:14.657 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 67108864 2024-04-07T16:16:14.657 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.657 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T16:16:14.658 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.658 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.658 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T16:16:14.658 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 658913373 2024-04-07T16:16:14.658 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.658 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T16:16:14.658 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.658 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.658 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T16:16:14.658 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.658 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.658 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T16:16:14.658 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.658 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.659 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T16:16:14.659 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.659 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.659 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T16:16:14.659 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.659 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.659 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T16:16:14.659 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2128 2024-04-07T16:16:14.659 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.659 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T16:16:14.659 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.659 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.659 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T16:16:14.659 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.659 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.660 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T16:16:14.660 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.660 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.660 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T16:16:14.660 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 70893395 2024-04-07T16:16:14.660 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:14.660 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.660 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:kv_onode": { 2024-04-07T16:16:14.660 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T16:16:14.660 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 192937984 2024-04-07T16:16:14.660 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.660 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T16:16:14.660 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 67108864 2024-04-07T16:16:14.660 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.660 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T16:16:14.661 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 464 2024-04-07T16:16:14.661 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.661 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T16:16:14.661 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 58570077 2024-04-07T16:16:14.661 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.661 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T16:16:14.661 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.661 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.661 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T16:16:14.661 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.661 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.661 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T16:16:14.661 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.661 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.661 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T16:16:14.661 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.662 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.662 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T16:16:14.662 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.662 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.662 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T16:16:14.662 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.662 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.662 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T16:16:14.662 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.662 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.662 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T16:16:14.662 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.662 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.662 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T16:16:14.663 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.663 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.663 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T16:16:14.663 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 67258579 2024-04-07T16:16:14.663 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:14.663 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.663 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:meta": { 2024-04-07T16:16:14.663 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T16:16:14.663 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 729808896 2024-04-07T16:16:14.663 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.663 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T16:16:14.663 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.663 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.663 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T16:16:14.663 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.664 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.664 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T16:16:14.664 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 658913373 2024-04-07T16:16:14.664 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.664 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T16:16:14.664 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.664 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.664 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T16:16:14.664 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.664 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.664 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T16:16:14.664 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.664 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.664 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T16:16:14.664 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.665 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.665 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T16:16:14.665 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.665 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.665 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T16:16:14.665 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 345585 2024-04-07T16:16:14.665 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.665 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T16:16:14.665 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.665 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.665 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T16:16:14.665 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.665 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.665 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T16:16:14.665 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.665 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.666 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T16:16:14.666 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 70549938 2024-04-07T16:16:14.666 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:14.666 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.666 INFO:tasks.workunit.client.0.smithi100.stdout: "objecter": { 2024-04-07T16:16:14.666 INFO:tasks.workunit.client.0.smithi100.stdout: "op_active": { 2024-04-07T16:16:14.666 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.666 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.666 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r": { 2024-04-07T16:16:14.666 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.666 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.666 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rmw": { 2024-04-07T16:16:14.666 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.666 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.667 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w": { 2024-04-07T16:16:14.667 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.667 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:14.667 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.667 INFO:tasks.workunit.client.0.smithi100.stdout: "osd": { 2024-04-07T16:16:14.667 INFO:tasks.workunit.client.0.smithi100.stdout: "numpg": { 2024-04-07T16:16:14.667 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 46 2024-04-07T16:16:14.667 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.667 INFO:tasks.workunit.client.0.smithi100.stdout: "numpg_removing": { 2024-04-07T16:16:14.667 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.667 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.667 INFO:tasks.workunit.client.0.smithi100.stdout: "op": { 2024-04-07T16:16:14.667 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 43 2024-04-07T16:16:14.667 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.667 INFO:tasks.workunit.client.0.smithi100.stdout: "op_delayed_degraded": { 2024-04-07T16:16:14.668 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.668 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.668 INFO:tasks.workunit.client.0.smithi100.stdout: "op_delayed_unreadable": { 2024-04-07T16:16:14.668 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.668 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.668 INFO:tasks.workunit.client.0.smithi100.stdout: "op_in_bytes": { 2024-04-07T16:16:14.668 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.668 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.668 INFO:tasks.workunit.client.0.smithi100.stdout: "op_latency": { 2024-04-07T16:16:14.668 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 43, 2024-04-07T16:16:14.668 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 162822355 2024-04-07T16:16:14.668 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.668 INFO:tasks.workunit.client.0.smithi100.stdout: "op_out_bytes": { 2024-04-07T16:16:14.668 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 132139 2024-04-07T16:16:14.668 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.669 INFO:tasks.workunit.client.0.smithi100.stdout: "op_prepare_latency": { 2024-04-07T16:16:14.669 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 45, 2024-04-07T16:16:14.669 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 31285016 2024-04-07T16:16:14.669 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.669 INFO:tasks.workunit.client.0.smithi100.stdout: "op_process_latency": { 2024-04-07T16:16:14.669 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 43, 2024-04-07T16:16:14.669 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 154213994 2024-04-07T16:16:14.669 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.669 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r": { 2024-04-07T16:16:14.669 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 28 2024-04-07T16:16:14.669 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.669 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_latency": { 2024-04-07T16:16:14.669 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 28, 2024-04-07T16:16:14.669 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 15710610 2024-04-07T16:16:14.669 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.670 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_latency_out_bytes_histogram": { 2024-04-07T16:16:14.670 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.670 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.670 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_out_bytes": { 2024-04-07T16:16:14.670 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 132139 2024-04-07T16:16:14.670 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.670 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_prepare_latency": { 2024-04-07T16:16:14.670 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 28, 2024-04-07T16:16:14.670 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 14653970 2024-04-07T16:16:14.670 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.670 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_process_latency": { 2024-04-07T16:16:14.670 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 28, 2024-04-07T16:16:14.670 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 12056616 2024-04-07T16:16:14.670 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.670 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw": { 2024-04-07T16:16:14.671 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 10 2024-04-07T16:16:14.671 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.671 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_in_bytes": { 2024-04-07T16:16:14.671 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.671 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.671 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_latency": { 2024-04-07T16:16:14.671 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 10, 2024-04-07T16:16:14.671 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 25806492 2024-04-07T16:16:14.671 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.671 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_latency_in_bytes_histogram": { 2024-04-07T16:16:14.671 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.671 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.671 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_latency_out_bytes_histogram": { 2024-04-07T16:16:14.671 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.672 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.672 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_out_bytes": { 2024-04-07T16:16:14.672 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.672 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.672 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_prepare_latency": { 2024-04-07T16:16:14.672 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 10, 2024-04-07T16:16:14.672 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 13165753 2024-04-07T16:16:14.672 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.672 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_process_latency": { 2024-04-07T16:16:14.672 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 10, 2024-04-07T16:16:14.672 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 24387544 2024-04-07T16:16:14.672 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.672 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w": { 2024-04-07T16:16:14.672 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5 2024-04-07T16:16:14.673 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.673 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_in_bytes": { 2024-04-07T16:16:14.673 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.673 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.673 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_latency": { 2024-04-07T16:16:14.673 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 5, 2024-04-07T16:16:14.673 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 121305253 2024-04-07T16:16:14.673 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.673 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_latency_in_bytes_histogram": { 2024-04-07T16:16:14.673 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.673 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.673 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_prepare_latency": { 2024-04-07T16:16:14.673 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 7, 2024-04-07T16:16:14.673 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3465293 2024-04-07T16:16:14.673 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.674 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_process_latency": { 2024-04-07T16:16:14.674 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 5, 2024-04-07T16:16:14.674 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 117769834 2024-04-07T16:16:14.674 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.674 INFO:tasks.workunit.client.0.smithi100.stdout: "op_wip": { 2024-04-07T16:16:14.674 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.674 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.674 INFO:tasks.workunit.client.0.smithi100.stdout: "recovery_bytes": { 2024-04-07T16:16:14.674 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.674 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.674 INFO:tasks.workunit.client.0.smithi100.stdout: "recovery_ops": { 2024-04-07T16:16:14.674 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:14.674 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.674 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_bytes": { 2024-04-07T16:16:14.675 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 95995035648 2024-04-07T16:16:14.675 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.675 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_bytes_used": { 2024-04-07T16:16:14.675 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 33173504 2024-04-07T16:16:14.675 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:14.675 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:14.675 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:14.675 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.675 INFO:tasks.workunit.client.0.smithi100.stdout: "pools": [ 2024-04-07T16:16:14.675 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:14.675 INFO:tasks.workunit.client.0.smithi100.stdout: "application": [ 2024-04-07T16:16:14.675 INFO:tasks.workunit.client.0.smithi100.stdout: "mgr" 2024-04-07T16:16:14.675 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:14.675 INFO:tasks.workunit.client.0.smithi100.stdout: "cache_mode": "none", 2024-04-07T16:16:14.675 INFO:tasks.workunit.client.0.smithi100.stdout: "erasure_code_profile": {}, 2024-04-07T16:16:14.676 INFO:tasks.workunit.client.0.smithi100.stdout: "min_size": 2, 2024-04-07T16:16:14.676 INFO:tasks.workunit.client.0.smithi100.stdout: "options": {}, 2024-04-07T16:16:14.676 INFO:tasks.workunit.client.0.smithi100.stdout: "pg_autoscale_mode": "off", 2024-04-07T16:16:14.676 INFO:tasks.workunit.client.0.smithi100.stdout: "pg_num": 1, 2024-04-07T16:16:14.676 INFO:tasks.workunit.client.0.smithi100.stdout: "pgp_num": 1, 2024-04-07T16:16:14.676 INFO:tasks.workunit.client.0.smithi100.stdout: "pool": 1, 2024-04-07T16:16:14.676 INFO:tasks.workunit.client.0.smithi100.stdout: "size": 3, 2024-04-07T16:16:14.676 INFO:tasks.workunit.client.0.smithi100.stdout: "stats": { 2024-04-07T16:16:14.676 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_raw": 729296873738, 2024-04-07T16:16:14.676 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_used": 1781760, 2024-04-07T16:16:14.676 INFO:tasks.workunit.client.0.smithi100.stdout: "compress_bytes_used": 0, 2024-04-07T16:16:14.676 INFO:tasks.workunit.client.0.smithi100.stdout: "compress_under_bytes": 0, 2024-04-07T16:16:14.676 INFO:tasks.workunit.client.0.smithi100.stdout: "data_bytes_used": 1781760, 2024-04-07T16:16:14.676 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty": 0, 2024-04-07T16:16:14.676 INFO:tasks.workunit.client.0.smithi100.stdout: "max_avail": 243098959872, 2024-04-07T16:16:14.677 INFO:tasks.workunit.client.0.smithi100.stdout: "objects": 2, 2024-04-07T16:16:14.677 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_bytes_used": 0, 2024-04-07T16:16:14.677 INFO:tasks.workunit.client.0.smithi100.stdout: "percent_used": 2.443114453853923e-06, 2024-04-07T16:16:14.677 INFO:tasks.workunit.client.0.smithi100.stdout: "quota_bytes": 0, 2024-04-07T16:16:14.677 INFO:tasks.workunit.client.0.smithi100.stdout: "quota_objects": 0, 2024-04-07T16:16:14.677 INFO:tasks.workunit.client.0.smithi100.stdout: "rd": 212, 2024-04-07T16:16:14.677 INFO:tasks.workunit.client.0.smithi100.stdout: "rd_bytes": 313344, 2024-04-07T16:16:14.677 INFO:tasks.workunit.client.0.smithi100.stdout: "stored": 590368, 2024-04-07T16:16:14.677 INFO:tasks.workunit.client.0.smithi100.stdout: "stored_data": 590368, 2024-04-07T16:16:14.677 INFO:tasks.workunit.client.0.smithi100.stdout: "stored_omap": 0, 2024-04-07T16:16:14.677 INFO:tasks.workunit.client.0.smithi100.stdout: "stored_raw": 1771104, 2024-04-07T16:16:14.677 INFO:tasks.workunit.client.0.smithi100.stdout: "wr": 179, 2024-04-07T16:16:14.677 INFO:tasks.workunit.client.0.smithi100.stdout: "wr_bytes": 2473984 2024-04-07T16:16:14.677 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.677 INFO:tasks.workunit.client.0.smithi100.stdout: "target_max_bytes": 0, 2024-04-07T16:16:14.677 INFO:tasks.workunit.client.0.smithi100.stdout: "target_max_objects": 0, 2024-04-07T16:16:14.678 INFO:tasks.workunit.client.0.smithi100.stdout: "type": "replicated" 2024-04-07T16:16:14.678 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.678 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:14.678 INFO:tasks.workunit.client.0.smithi100.stdout: "application": [ 2024-04-07T16:16:14.678 INFO:tasks.workunit.client.0.smithi100.stdout: "cephfs" 2024-04-07T16:16:14.678 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:14.678 INFO:tasks.workunit.client.0.smithi100.stdout: "cache_mode": "none", 2024-04-07T16:16:14.678 INFO:tasks.workunit.client.0.smithi100.stdout: "erasure_code_profile": {}, 2024-04-07T16:16:14.678 INFO:tasks.workunit.client.0.smithi100.stdout: "min_size": 2, 2024-04-07T16:16:14.678 INFO:tasks.workunit.client.0.smithi100.stdout: "options": {}, 2024-04-07T16:16:14.678 INFO:tasks.workunit.client.0.smithi100.stdout: "pg_autoscale_mode": "off", 2024-04-07T16:16:14.678 INFO:tasks.workunit.client.0.smithi100.stdout: "pg_num": 32, 2024-04-07T16:16:14.678 INFO:tasks.workunit.client.0.smithi100.stdout: "pgp_num": 32, 2024-04-07T16:16:14.678 INFO:tasks.workunit.client.0.smithi100.stdout: "pool": 2, 2024-04-07T16:16:14.678 INFO:tasks.workunit.client.0.smithi100.stdout: "size": 3, 2024-04-07T16:16:14.679 INFO:tasks.workunit.client.0.smithi100.stdout: "stats": { 2024-04-07T16:16:14.679 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_raw": 729296873738, 2024-04-07T16:16:14.679 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_used": 98304, 2024-04-07T16:16:14.679 INFO:tasks.workunit.client.0.smithi100.stdout: "compress_bytes_used": 0, 2024-04-07T16:16:14.679 INFO:tasks.workunit.client.0.smithi100.stdout: "compress_under_bytes": 0, 2024-04-07T16:16:14.679 INFO:tasks.workunit.client.0.smithi100.stdout: "data_bytes_used": 98304, 2024-04-07T16:16:14.679 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty": 0, 2024-04-07T16:16:14.679 INFO:tasks.workunit.client.0.smithi100.stdout: "max_avail": 243098959872, 2024-04-07T16:16:14.679 INFO:tasks.workunit.client.0.smithi100.stdout: "objects": 22, 2024-04-07T16:16:14.679 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_bytes_used": 0, 2024-04-07T16:16:14.679 INFO:tasks.workunit.client.0.smithi100.stdout: "percent_used": 1.3479282756634348e-07, 2024-04-07T16:16:14.679 INFO:tasks.workunit.client.0.smithi100.stdout: "quota_bytes": 0, 2024-04-07T16:16:14.679 INFO:tasks.workunit.client.0.smithi100.stdout: "quota_objects": 0, 2024-04-07T16:16:14.679 INFO:tasks.workunit.client.0.smithi100.stdout: "rd": 0, 2024-04-07T16:16:14.679 INFO:tasks.workunit.client.0.smithi100.stdout: "rd_bytes": 0, 2024-04-07T16:16:14.680 INFO:tasks.workunit.client.0.smithi100.stdout: "stored": 3430, 2024-04-07T16:16:14.680 INFO:tasks.workunit.client.0.smithi100.stdout: "stored_data": 3430, 2024-04-07T16:16:14.680 INFO:tasks.workunit.client.0.smithi100.stdout: "stored_omap": 0, 2024-04-07T16:16:14.680 INFO:tasks.workunit.client.0.smithi100.stdout: "stored_raw": 10290, 2024-04-07T16:16:14.680 INFO:tasks.workunit.client.0.smithi100.stdout: "wr": 51, 2024-04-07T16:16:14.680 INFO:tasks.workunit.client.0.smithi100.stdout: "wr_bytes": 20480 2024-04-07T16:16:14.680 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.680 INFO:tasks.workunit.client.0.smithi100.stdout: "target_max_bytes": 0, 2024-04-07T16:16:14.680 INFO:tasks.workunit.client.0.smithi100.stdout: "target_max_objects": 0, 2024-04-07T16:16:14.680 INFO:tasks.workunit.client.0.smithi100.stdout: "type": "replicated" 2024-04-07T16:16:14.680 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.680 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:14.680 INFO:tasks.workunit.client.0.smithi100.stdout: "application": [ 2024-04-07T16:16:14.680 INFO:tasks.workunit.client.0.smithi100.stdout: "cephfs" 2024-04-07T16:16:14.680 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:14.680 INFO:tasks.workunit.client.0.smithi100.stdout: "cache_mode": "none", 2024-04-07T16:16:14.681 INFO:tasks.workunit.client.0.smithi100.stdout: "erasure_code_profile": {}, 2024-04-07T16:16:14.681 INFO:tasks.workunit.client.0.smithi100.stdout: "min_size": 2, 2024-04-07T16:16:14.681 INFO:tasks.workunit.client.0.smithi100.stdout: "options": {}, 2024-04-07T16:16:14.681 INFO:tasks.workunit.client.0.smithi100.stdout: "pg_autoscale_mode": "off", 2024-04-07T16:16:14.681 INFO:tasks.workunit.client.0.smithi100.stdout: "pg_num": 64, 2024-04-07T16:16:14.681 INFO:tasks.workunit.client.0.smithi100.stdout: "pgp_num": 64, 2024-04-07T16:16:14.681 INFO:tasks.workunit.client.0.smithi100.stdout: "pool": 3, 2024-04-07T16:16:14.681 INFO:tasks.workunit.client.0.smithi100.stdout: "size": 3, 2024-04-07T16:16:14.681 INFO:tasks.workunit.client.0.smithi100.stdout: "stats": { 2024-04-07T16:16:14.681 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_raw": 729296873738, 2024-04-07T16:16:14.681 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_used": 0, 2024-04-07T16:16:14.681 INFO:tasks.workunit.client.0.smithi100.stdout: "compress_bytes_used": 0, 2024-04-07T16:16:14.681 INFO:tasks.workunit.client.0.smithi100.stdout: "compress_under_bytes": 0, 2024-04-07T16:16:14.681 INFO:tasks.workunit.client.0.smithi100.stdout: "data_bytes_used": 0, 2024-04-07T16:16:14.681 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty": 0, 2024-04-07T16:16:14.681 INFO:tasks.workunit.client.0.smithi100.stdout: "max_avail": 243098959872, 2024-04-07T16:16:14.682 INFO:tasks.workunit.client.0.smithi100.stdout: "objects": 0, 2024-04-07T16:16:14.682 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_bytes_used": 0, 2024-04-07T16:16:14.682 INFO:tasks.workunit.client.0.smithi100.stdout: "percent_used": 0.0, 2024-04-07T16:16:14.682 INFO:tasks.workunit.client.0.smithi100.stdout: "quota_bytes": 0, 2024-04-07T16:16:14.682 INFO:tasks.workunit.client.0.smithi100.stdout: "quota_objects": 0, 2024-04-07T16:16:14.682 INFO:tasks.workunit.client.0.smithi100.stdout: "rd": 0, 2024-04-07T16:16:14.682 INFO:tasks.workunit.client.0.smithi100.stdout: "rd_bytes": 0, 2024-04-07T16:16:14.682 INFO:tasks.workunit.client.0.smithi100.stdout: "stored": 0, 2024-04-07T16:16:14.682 INFO:tasks.workunit.client.0.smithi100.stdout: "stored_data": 0, 2024-04-07T16:16:14.682 INFO:tasks.workunit.client.0.smithi100.stdout: "stored_omap": 0, 2024-04-07T16:16:14.682 INFO:tasks.workunit.client.0.smithi100.stdout: "stored_raw": 0, 2024-04-07T16:16:14.682 INFO:tasks.workunit.client.0.smithi100.stdout: "wr": 0, 2024-04-07T16:16:14.682 INFO:tasks.workunit.client.0.smithi100.stdout: "wr_bytes": 0 2024-04-07T16:16:14.682 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.682 INFO:tasks.workunit.client.0.smithi100.stdout: "target_max_bytes": 0, 2024-04-07T16:16:14.683 INFO:tasks.workunit.client.0.smithi100.stdout: "target_max_objects": 0, 2024-04-07T16:16:14.683 INFO:tasks.workunit.client.0.smithi100.stdout: "type": "replicated" 2024-04-07T16:16:14.683 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.683 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:14.683 INFO:tasks.workunit.client.0.smithi100.stdout: "application": [ 2024-04-07T16:16:14.683 INFO:tasks.workunit.client.0.smithi100.stdout: "rbd" 2024-04-07T16:16:14.683 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:14.683 INFO:tasks.workunit.client.0.smithi100.stdout: "cache_mode": "none", 2024-04-07T16:16:14.683 INFO:tasks.workunit.client.0.smithi100.stdout: "erasure_code_profile": {}, 2024-04-07T16:16:14.683 INFO:tasks.workunit.client.0.smithi100.stdout: "min_size": 2, 2024-04-07T16:16:14.683 INFO:tasks.workunit.client.0.smithi100.stdout: "options": {}, 2024-04-07T16:16:14.683 INFO:tasks.workunit.client.0.smithi100.stdout: "pg_autoscale_mode": "off", 2024-04-07T16:16:14.683 INFO:tasks.workunit.client.0.smithi100.stdout: "pg_num": 8, 2024-04-07T16:16:14.684 INFO:tasks.workunit.client.0.smithi100.stdout: "pgp_num": 8, 2024-04-07T16:16:14.684 INFO:tasks.workunit.client.0.smithi100.stdout: "pool": 4, 2024-04-07T16:16:14.684 INFO:tasks.workunit.client.0.smithi100.stdout: "size": 3, 2024-04-07T16:16:14.684 INFO:tasks.workunit.client.0.smithi100.stdout: "stats": { 2024-04-07T16:16:14.684 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_raw": 729296873738, 2024-04-07T16:16:14.684 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_used": 0, 2024-04-07T16:16:14.684 INFO:tasks.workunit.client.0.smithi100.stdout: "compress_bytes_used": 0, 2024-04-07T16:16:14.684 INFO:tasks.workunit.client.0.smithi100.stdout: "compress_under_bytes": 0, 2024-04-07T16:16:14.684 INFO:tasks.workunit.client.0.smithi100.stdout: "data_bytes_used": 0, 2024-04-07T16:16:14.684 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty": 0, 2024-04-07T16:16:14.684 INFO:tasks.workunit.client.0.smithi100.stdout: "max_avail": 243098959872, 2024-04-07T16:16:14.684 INFO:tasks.workunit.client.0.smithi100.stdout: "objects": 0, 2024-04-07T16:16:14.684 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_bytes_used": 0, 2024-04-07T16:16:14.684 INFO:tasks.workunit.client.0.smithi100.stdout: "percent_used": 0.0, 2024-04-07T16:16:14.685 INFO:tasks.workunit.client.0.smithi100.stdout: "quota_bytes": 0, 2024-04-07T16:16:14.685 INFO:tasks.workunit.client.0.smithi100.stdout: "quota_objects": 0, 2024-04-07T16:16:14.685 INFO:tasks.workunit.client.0.smithi100.stdout: "rd": 0, 2024-04-07T16:16:14.685 INFO:tasks.workunit.client.0.smithi100.stdout: "rd_bytes": 0, 2024-04-07T16:16:14.685 INFO:tasks.workunit.client.0.smithi100.stdout: "stored": 0, 2024-04-07T16:16:14.685 INFO:tasks.workunit.client.0.smithi100.stdout: "stored_data": 0, 2024-04-07T16:16:14.685 INFO:tasks.workunit.client.0.smithi100.stdout: "stored_omap": 0, 2024-04-07T16:16:14.685 INFO:tasks.workunit.client.0.smithi100.stdout: "stored_raw": 0, 2024-04-07T16:16:14.685 INFO:tasks.workunit.client.0.smithi100.stdout: "wr": 0, 2024-04-07T16:16:14.685 INFO:tasks.workunit.client.0.smithi100.stdout: "wr_bytes": 0 2024-04-07T16:16:14.685 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.685 INFO:tasks.workunit.client.0.smithi100.stdout: "target_max_bytes": 0, 2024-04-07T16:16:14.685 INFO:tasks.workunit.client.0.smithi100.stdout: "target_max_objects": 0, 2024-04-07T16:16:14.685 INFO:tasks.workunit.client.0.smithi100.stdout: "type": "replicated" 2024-04-07T16:16:14.685 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:14.686 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:14.686 INFO:tasks.workunit.client.0.smithi100.stdout: "rbd": { 2024-04-07T16:16:14.686 INFO:tasks.workunit.client.0.smithi100.stdout: "mirroring_by_pool": [ 2024-04-07T16:16:14.686 INFO:tasks.workunit.client.0.smithi100.stdout: false 2024-04-07T16:16:14.686 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:14.686 INFO:tasks.workunit.client.0.smithi100.stdout: "num_images_by_pool": [ 2024-04-07T16:16:14.686 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T16:16:14.686 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:14.686 INFO:tasks.workunit.client.0.smithi100.stdout: "num_pools": 1 2024-04-07T16:16:14.686 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.686 INFO:tasks.workunit.client.0.smithi100.stdout: "report_id": "ee983b25-10fa-4633-b6cf-1c3036b91b52", 2024-04-07T16:16:14.686 INFO:tasks.workunit.client.0.smithi100.stdout: "report_timestamp": "2024-04-07T16:16:13.510047", 2024-04-07T16:16:14.686 INFO:tasks.workunit.client.0.smithi100.stdout: "report_version": 1, 2024-04-07T16:16:14.686 INFO:tasks.workunit.client.0.smithi100.stdout: "rocksdb_stats": { 2024-04-07T16:16:14.687 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "7.9.2" 2024-04-07T16:16:14.687 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.687 INFO:tasks.workunit.client.0.smithi100.stdout: "rook": { 2024-04-07T16:16:14.687 INFO:tasks.workunit.client.0.smithi100.stdout: "cluster": { 2024-04-07T16:16:14.687 INFO:tasks.workunit.client.0.smithi100.stdout: "external-mode": null, 2024-04-07T16:16:14.687 INFO:tasks.workunit.client.0.smithi100.stdout: "mon": { 2024-04-07T16:16:14.687 INFO:tasks.workunit.client.0.smithi100.stdout: "allow-multiple-per-node": null, 2024-04-07T16:16:14.687 INFO:tasks.workunit.client.0.smithi100.stdout: "count": null, 2024-04-07T16:16:14.687 INFO:tasks.workunit.client.0.smithi100.stdout: "max-id": null, 2024-04-07T16:16:14.687 INFO:tasks.workunit.client.0.smithi100.stdout: "pvc": { 2024-04-07T16:16:14.687 INFO:tasks.workunit.client.0.smithi100.stdout: "enabled": null 2024-04-07T16:16:14.687 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.687 INFO:tasks.workunit.client.0.smithi100.stdout: "stretch": { 2024-04-07T16:16:14.688 INFO:tasks.workunit.client.0.smithi100.stdout: "enabled": null 2024-04-07T16:16:14.688 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:14.688 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.688 INFO:tasks.workunit.client.0.smithi100.stdout: "network": { 2024-04-07T16:16:14.688 INFO:tasks.workunit.client.0.smithi100.stdout: "provider": null 2024-04-07T16:16:14.688 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.688 INFO:tasks.workunit.client.0.smithi100.stdout: "storage": { 2024-04-07T16:16:14.688 INFO:tasks.workunit.client.0.smithi100.stdout: "device-set": { 2024-04-07T16:16:14.688 INFO:tasks.workunit.client.0.smithi100.stdout: "count": { 2024-04-07T16:16:14.688 INFO:tasks.workunit.client.0.smithi100.stdout: "non-portable": null, 2024-04-07T16:16:14.688 INFO:tasks.workunit.client.0.smithi100.stdout: "portable": null, 2024-04-07T16:16:14.688 INFO:tasks.workunit.client.0.smithi100.stdout: "total": null 2024-04-07T16:16:14.688 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:14.688 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:14.688 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:14.689 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.689 INFO:tasks.workunit.client.0.smithi100.stdout: "csi": { 2024-04-07T16:16:14.689 INFO:tasks.workunit.client.0.smithi100.stdout: "version": null 2024-04-07T16:16:14.689 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.689 INFO:tasks.workunit.client.0.smithi100.stdout: "kubernetes": { 2024-04-07T16:16:14.689 INFO:tasks.workunit.client.0.smithi100.stdout: "version": null 2024-04-07T16:16:14.689 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.689 INFO:tasks.workunit.client.0.smithi100.stdout: "node": { 2024-04-07T16:16:14.689 INFO:tasks.workunit.client.0.smithi100.stdout: "count": { 2024-04-07T16:16:14.689 INFO:tasks.workunit.client.0.smithi100.stdout: "kubernetes-total": null, 2024-04-07T16:16:14.689 INFO:tasks.workunit.client.0.smithi100.stdout: "with-ceph-daemons": null, 2024-04-07T16:16:14.689 INFO:tasks.workunit.client.0.smithi100.stdout: "with-csi-cephfs-plugin": null, 2024-04-07T16:16:14.689 INFO:tasks.workunit.client.0.smithi100.stdout: "with-csi-nfs-plugin": null, 2024-04-07T16:16:14.689 INFO:tasks.workunit.client.0.smithi100.stdout: "with-csi-rbd-plugin": null 2024-04-07T16:16:14.689 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:14.690 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.690 INFO:tasks.workunit.client.0.smithi100.stdout: "usage": { 2024-04-07T16:16:14.690 INFO:tasks.workunit.client.0.smithi100.stdout: "storage-class": { 2024-04-07T16:16:14.690 INFO:tasks.workunit.client.0.smithi100.stdout: "count": { 2024-04-07T16:16:14.690 INFO:tasks.workunit.client.0.smithi100.stdout: "bucket": null, 2024-04-07T16:16:14.690 INFO:tasks.workunit.client.0.smithi100.stdout: "cephfs": null, 2024-04-07T16:16:14.690 INFO:tasks.workunit.client.0.smithi100.stdout: "nfs": null, 2024-04-07T16:16:14.690 INFO:tasks.workunit.client.0.smithi100.stdout: "rbd": null, 2024-04-07T16:16:14.690 INFO:tasks.workunit.client.0.smithi100.stdout: "total": null 2024-04-07T16:16:14.690 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:14.690 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:14.690 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.690 INFO:tasks.workunit.client.0.smithi100.stdout: "version": null 2024-04-07T16:16:14.691 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.691 INFO:tasks.workunit.client.0.smithi100.stdout: "services": {}, 2024-04-07T16:16:14.691 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_per_pg": [ 2024-04-07T16:16:14.691 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:14.691 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:14.691 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T16:16:14.691 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T16:16:14.691 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T16:16:14.691 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:14.691 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 7, 2024-04-07T16:16:14.691 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:14.691 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:14.691 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T16:16:14.691 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:14.691 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:14.692 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:14.692 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.637813+0000", 2024-04-07T16:16:14.692 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.395023+0000", 2024-04-07T16:16:14.692 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.395023+0000", 2024-04-07T16:16:14.692 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.395367+0000", 2024-04-07T16:16:14.692 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.637813+0000", 2024-04-07T16:16:14.692 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:14.692 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:14.692 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:14.692 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 61, 2024-04-07T16:16:14.692 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.637813+0000", 2024-04-07T16:16:14.692 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.637813+0000", 2024-04-07T16:16:14.692 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.637813+0000", 2024-04-07T16:16:14.692 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:14.693 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:14.693 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:14.693 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.637813+0000", 2024-04-07T16:16:14.693 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.637813+0000", 2024-04-07T16:16:14.693 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:14.693 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:14.693 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:14.693 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:14.693 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 60, 2024-04-07T16:16:14.693 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:14.693 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:14.693 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:14.693 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:14.693 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:14.693 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:14.694 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:14.694 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:14.694 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.3f", 2024-04-07T16:16:14.694 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:14.694 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:14.694 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:14.694 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 19, 2024-04-07T16:16:14.694 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:14.694 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T23:05:09.752696+0000", 2024-04-07T16:16:14.694 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:14.694 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:14.694 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:14.694 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:14.694 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:14.695 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:14.695 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:14.695 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:14.695 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:14.695 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:14.695 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:14.695 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:14.695 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:14.695 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:14.695 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:14.695 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:14.695 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:14.695 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:14.695 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:14.695 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:14.696 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:14.696 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:14.696 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:14.696 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:14.696 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:14.696 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:14.696 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:14.696 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:14.696 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:14.696 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:14.696 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:14.696 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:14.696 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:14.696 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:14.696 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:14.697 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:14.697 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:14.697 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:14.697 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:14.697 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:14.697 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:14.697 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:14.697 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:14.697 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.697 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:14.697 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:14.697 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:14.697 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T16:16:14.697 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T16:16:14.697 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T16:16:14.698 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:14.698 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 7, 2024-04-07T16:16:14.698 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:14.698 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.698 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:14.698 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:14.698 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T16:16:14.698 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T16:16:14.698 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T16:16:14.698 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:14.698 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 1, 2024-04-07T16:16:14.698 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:14.698 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:14.698 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T16:16:14.699 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:14.699 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:14.699 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:14.699 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.237070+0000", 2024-04-07T16:16:14.699 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.385175+0000", 2024-04-07T16:16:14.699 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.385175+0000", 2024-04-07T16:16:14.699 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.385549+0000", 2024-04-07T16:16:14.699 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.237070+0000", 2024-04-07T16:16:14.699 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:14.699 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:14.699 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:14.699 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 61, 2024-04-07T16:16:14.699 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.237070+0000", 2024-04-07T16:16:14.699 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.237070+0000", 2024-04-07T16:16:14.699 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.237070+0000", 2024-04-07T16:16:14.700 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:14.700 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:14.700 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:14.700 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.237070+0000", 2024-04-07T16:16:14.700 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.237070+0000", 2024-04-07T16:16:14.700 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:14.700 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:14.700 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:14.700 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:14.700 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 60, 2024-04-07T16:16:14.700 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:14.700 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:14.700 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:14.700 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:14.700 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:14.700 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:14.701 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:14.701 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:14.701 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.3e", 2024-04-07T16:16:14.701 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:14.701 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:14.701 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:14.701 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 19, 2024-04-07T16:16:14.701 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:14.701 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T16:28:22.788092+0000", 2024-04-07T16:16:14.701 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:14.701 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:14.701 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:14.701 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:14.701 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:14.701 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:14.702 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:14.702 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:14.702 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:14.702 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:14.702 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:14.702 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:14.702 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:14.702 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:14.702 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:14.702 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:14.702 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:14.702 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:14.702 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:14.702 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:14.702 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:14.703 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:14.703 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:14.703 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:14.703 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:14.703 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:14.703 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:14.703 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:14.703 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:14.703 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:14.703 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:14.703 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:14.703 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:14.703 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:14.703 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:14.703 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:14.704 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:14.704 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:14.704 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:14.704 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:14.704 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:14.704 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:14.704 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:14.704 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.704 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:14.704 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:14.704 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:14.704 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T16:16:14.704 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T16:16:14.704 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T16:16:14.705 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:14.705 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 1, 2024-04-07T16:16:14.705 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:14.705 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.705 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:14.705 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:14.705 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T16:16:14.705 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T16:16:14.705 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T16:16:14.705 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:14.705 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 0, 2024-04-07T16:16:14.705 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:14.705 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:14.705 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T16:16:14.705 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:14.706 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:14.706 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:14.706 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.030532+0000", 2024-04-07T16:16:14.706 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.410088+0000", 2024-04-07T16:16:14.706 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.410088+0000", 2024-04-07T16:16:14.706 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.410431+0000", 2024-04-07T16:16:14.706 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.030532+0000", 2024-04-07T16:16:14.706 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:14.706 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:14.706 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:14.706 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 61, 2024-04-07T16:16:14.706 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.030532+0000", 2024-04-07T16:16:14.706 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.030532+0000", 2024-04-07T16:16:14.706 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.030532+0000", 2024-04-07T16:16:14.706 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:14.707 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:14.707 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:14.707 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.030532+0000", 2024-04-07T16:16:14.707 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.030532+0000", 2024-04-07T16:16:14.707 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:14.707 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:14.707 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:14.707 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:14.707 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 60, 2024-04-07T16:16:14.707 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:14.707 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:14.707 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:14.707 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:14.708 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:14.708 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:14.708 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:14.708 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:14.708 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.3d", 2024-04-07T16:16:14.708 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:14.708 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:14.708 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:14.708 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 19, 2024-04-07T16:16:14.708 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:14.708 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T19:47:57.716144+0000", 2024-04-07T16:16:14.708 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:14.708 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:14.708 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:14.709 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:14.709 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:14.709 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:14.709 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:14.709 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:14.709 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:14.709 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:14.709 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:14.709 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:14.709 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:14.709 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:14.709 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:14.709 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:14.709 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:14.710 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:14.710 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:14.710 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:14.710 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:14.710 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:14.710 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:14.710 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:14.710 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:14.710 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:14.710 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:14.710 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:14.710 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:14.710 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:14.711 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:14.711 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:14.711 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:14.711 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:14.711 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:14.711 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:14.711 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:14.711 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:14.711 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:14.711 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:14.711 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:14.711 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:14.711 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:14.711 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.712 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:14.712 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:14.712 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:14.712 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T16:16:14.712 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T16:16:14.712 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T16:16:14.712 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:14.712 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 0, 2024-04-07T16:16:14.712 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:14.712 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.712 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:14.712 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:14.712 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T16:16:14.713 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T16:16:14.713 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T16:16:14.713 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:14.713 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 5, 2024-04-07T16:16:14.713 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:14.713 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:14.713 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T16:16:14.713 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:14.713 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:14.713 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:14.713 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.025110+0000", 2024-04-07T16:16:14.713 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.401819+0000", 2024-04-07T16:16:14.713 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.401819+0000", 2024-04-07T16:16:14.714 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.402328+0000", 2024-04-07T16:16:14.714 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.025110+0000", 2024-04-07T16:16:14.714 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:14.714 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:14.714 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:14.714 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 61, 2024-04-07T16:16:14.714 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.025110+0000", 2024-04-07T16:16:14.714 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.025110+0000", 2024-04-07T16:16:14.714 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.025110+0000", 2024-04-07T16:16:14.714 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:14.714 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:14.714 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:14.714 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.025110+0000", 2024-04-07T16:16:14.714 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.025110+0000", 2024-04-07T16:16:14.715 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:14.715 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:14.715 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:14.715 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:14.715 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 60, 2024-04-07T16:16:14.715 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:14.715 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:14.715 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:14.715 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:14.715 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:14.715 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:14.715 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:14.715 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:14.716 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.3c", 2024-04-07T16:16:14.716 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:14.716 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:14.716 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:14.716 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 19, 2024-04-07T16:16:14.716 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:14.716 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T21:43:13.292859+0000", 2024-04-07T16:16:14.716 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:14.716 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:14.716 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:14.716 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:14.716 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:14.716 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:14.717 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:14.717 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:14.717 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:14.717 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:14.717 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:14.717 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:14.717 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:14.717 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:14.717 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:14.717 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:14.717 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:14.717 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:14.717 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:14.717 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:14.717 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:14.717 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:14.718 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:14.718 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:14.718 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:14.718 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:14.718 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:14.718 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:14.718 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:14.718 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:14.718 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:14.718 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:14.718 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:14.718 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:14.718 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:14.718 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:14.718 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:14.718 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:14.719 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:14.719 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:14.719 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:14.719 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:14.719 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:14.719 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.719 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:14.719 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:14.719 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:14.719 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T16:16:14.719 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T16:16:14.719 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T16:16:14.719 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:14.719 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 5, 2024-04-07T16:16:14.719 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:14.720 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.720 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:14.720 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:14.720 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T16:16:14.720 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T16:16:14.720 INFO:tasks.workunit.client.0.smithi100.stdout: 4 2024-04-07T16:16:14.720 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:14.720 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 7, 2024-04-07T16:16:14.720 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:14.720 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:14.720 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T16:16:14.720 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:14.720 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:14.720 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:14.720 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.637270+0000", 2024-04-07T16:16:14.720 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.413281+0000", 2024-04-07T16:16:14.721 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.413281+0000", 2024-04-07T16:16:14.721 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.413487+0000", 2024-04-07T16:16:14.721 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.637270+0000", 2024-04-07T16:16:14.721 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:14.721 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:14.721 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:14.721 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 61, 2024-04-07T16:16:14.721 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.637270+0000", 2024-04-07T16:16:14.721 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.637270+0000", 2024-04-07T16:16:14.721 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.637270+0000", 2024-04-07T16:16:14.721 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:14.721 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:14.721 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:14.721 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.637270+0000", 2024-04-07T16:16:14.721 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.637270+0000", 2024-04-07T16:16:14.722 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:14.722 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:14.722 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:14.722 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:14.722 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 60, 2024-04-07T16:16:14.722 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:14.722 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:14.722 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:14.722 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:14.722 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:14.722 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:14.722 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:14.722 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:14.722 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.3b", 2024-04-07T16:16:14.723 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:14.723 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:14.723 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:14.723 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 19, 2024-04-07T16:16:14.723 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:14.723 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T23:59:23.302803+0000", 2024-04-07T16:16:14.723 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:14.723 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:14.723 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:14.723 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:14.723 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:14.723 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:14.723 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:14.723 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:14.723 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:14.724 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:14.724 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:14.724 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:14.724 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:14.724 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:14.724 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:14.724 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:14.724 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:14.724 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:14.724 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:14.724 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:14.724 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:14.724 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:14.724 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:14.724 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:14.724 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:14.725 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:14.725 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:14.725 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:14.725 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:14.725 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:14.725 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:14.725 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:14.725 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:14.725 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:14.725 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:14.725 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:14.725 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:14.725 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:14.725 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:14.725 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:14.726 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:14.726 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:14.726 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:14.726 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.726 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:14.726 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:14.726 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:14.726 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T16:16:14.726 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T16:16:14.726 INFO:tasks.workunit.client.0.smithi100.stdout: 4 2024-04-07T16:16:14.726 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:14.726 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 7, 2024-04-07T16:16:14.726 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:14.726 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.727 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:14.727 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:14.727 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T16:16:14.727 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T16:16:14.727 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T16:16:14.727 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:14.727 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 5, 2024-04-07T16:16:14.727 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:14.727 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:14.727 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T16:16:14.727 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:14.727 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:14.727 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:14.727 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.024848+0000", 2024-04-07T16:16:14.727 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.400365+0000", 2024-04-07T16:16:14.728 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.400365+0000", 2024-04-07T16:16:14.728 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.401561+0000", 2024-04-07T16:16:14.728 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.024848+0000", 2024-04-07T16:16:14.728 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:14.728 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:14.728 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:14.728 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 61, 2024-04-07T16:16:14.728 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.024848+0000", 2024-04-07T16:16:14.728 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.024848+0000", 2024-04-07T16:16:14.728 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.024848+0000", 2024-04-07T16:16:14.728 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:14.728 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:14.728 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:14.728 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.024848+0000", 2024-04-07T16:16:14.728 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.024848+0000", 2024-04-07T16:16:14.729 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:14.729 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:14.729 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:14.729 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:14.729 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 60, 2024-04-07T16:16:14.729 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:14.729 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:14.729 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:14.729 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:14.729 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:14.729 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:14.729 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:14.729 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:14.729 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.3a", 2024-04-07T16:16:14.729 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:14.730 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:14.730 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:14.730 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 19, 2024-04-07T16:16:14.730 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:14.730 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T20:14:49.110462+0000", 2024-04-07T16:16:14.730 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:14.730 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:14.730 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:14.730 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:14.730 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:14.730 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:14.730 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:14.730 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:14.730 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:14.730 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:14.731 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:14.731 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:14.731 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:14.731 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:14.731 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:14.731 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:14.731 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:14.731 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:14.731 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:14.731 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:14.731 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:14.731 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:14.731 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:14.731 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:14.731 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:14.732 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:14.732 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:14.732 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:14.732 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:14.732 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:14.732 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:14.732 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:14.732 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:14.732 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:14.732 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:14.732 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:14.732 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:14.732 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:14.732 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:14.733 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:14.733 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:14.733 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:14.733 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:14.733 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.733 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:14.733 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:14.733 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:14.733 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T16:16:14.733 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T16:16:14.733 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T16:16:14.733 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:14.733 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 5, 2024-04-07T16:16:14.733 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:14.733 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.733 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:14.734 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:14.734 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T16:16:14.734 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T16:16:14.734 INFO:tasks.workunit.client.0.smithi100.stdout: 4 2024-04-07T16:16:14.734 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:14.734 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 7, 2024-04-07T16:16:14.734 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:14.734 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:14.734 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T16:16:14.734 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:14.734 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:14.734 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:14.734 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.636229+0000", 2024-04-07T16:16:14.735 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.393187+0000", 2024-04-07T16:16:14.735 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.393187+0000", 2024-04-07T16:16:14.735 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.393518+0000", 2024-04-07T16:16:14.735 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.636229+0000", 2024-04-07T16:16:14.735 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:14.735 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:14.735 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:14.735 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 61, 2024-04-07T16:16:14.735 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.636229+0000", 2024-04-07T16:16:14.735 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.636229+0000", 2024-04-07T16:16:14.735 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.636229+0000", 2024-04-07T16:16:14.735 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:14.735 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:14.735 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:14.736 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.636229+0000", 2024-04-07T16:16:14.736 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.636229+0000", 2024-04-07T16:16:14.736 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:14.736 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:14.736 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:14.736 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:14.736 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 60, 2024-04-07T16:16:14.736 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:14.736 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:14.736 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:14.736 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:14.736 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:14.736 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:14.736 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:14.736 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:14.737 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.39", 2024-04-07T16:16:14.737 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:14.737 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:14.737 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:14.737 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 19, 2024-04-07T16:16:14.737 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:14.737 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T19:41:28.702905+0000", 2024-04-07T16:16:14.737 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:14.737 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:14.737 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:14.737 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:14.737 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:14.737 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:14.737 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:14.737 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:14.738 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:14.738 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:14.738 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:14.738 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:14.738 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:14.738 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:14.738 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:14.738 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:14.738 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:14.738 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:14.738 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:14.738 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:14.738 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:14.738 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:14.739 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:14.739 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:14.739 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:14.739 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:14.739 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:14.739 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:14.739 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:14.739 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:14.739 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:14.739 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:14.739 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:14.739 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:14.739 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:14.739 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:14.739 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:14.739 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:14.740 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:14.740 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:14.740 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:14.740 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:14.740 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:14.740 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.740 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:14.740 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:14.740 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:14.740 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T16:16:14.740 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T16:16:14.740 INFO:tasks.workunit.client.0.smithi100.stdout: 4 2024-04-07T16:16:14.740 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:14.740 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 7, 2024-04-07T16:16:14.740 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:14.741 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.741 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:14.741 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:14.741 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T16:16:14.741 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T16:16:14.741 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T16:16:14.741 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:14.741 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 3, 2024-04-07T16:16:14.741 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:14.741 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:14.741 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T16:16:14.741 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:14.741 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:14.741 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:14.742 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.187095+0000", 2024-04-07T16:16:14.742 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.394422+0000", 2024-04-07T16:16:14.742 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.394422+0000", 2024-04-07T16:16:14.742 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.394702+0000", 2024-04-07T16:16:14.742 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.187095+0000", 2024-04-07T16:16:14.742 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:14.742 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:14.742 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:14.742 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 61, 2024-04-07T16:16:14.742 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.187095+0000", 2024-04-07T16:16:14.742 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.187095+0000", 2024-04-07T16:16:14.742 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.187095+0000", 2024-04-07T16:16:14.742 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:14.742 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:14.743 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:14.743 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.187095+0000", 2024-04-07T16:16:14.743 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.187095+0000", 2024-04-07T16:16:14.743 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:14.743 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:14.743 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:14.743 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:14.743 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 60, 2024-04-07T16:16:14.743 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:14.743 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:14.743 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:14.743 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:14.743 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:14.743 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:14.743 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:14.744 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:14.744 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.38", 2024-04-07T16:16:14.744 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:14.744 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:14.744 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:14.744 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 19, 2024-04-07T16:16:14.744 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:14.744 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T00:37:44.512471+0000", 2024-04-07T16:16:14.744 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:14.744 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:14.744 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:14.744 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:14.744 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:14.744 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:14.744 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:14.745 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:14.745 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:14.745 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:14.745 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:14.745 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:14.745 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:14.745 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:14.745 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:14.745 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:14.745 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:14.745 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:14.745 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:14.745 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:14.745 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:14.745 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:14.745 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:14.746 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:14.746 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:14.746 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:14.746 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:14.746 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:14.746 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:14.746 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:14.746 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:14.746 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:14.746 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:14.746 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:14.746 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:14.746 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:14.746 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:14.746 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:14.747 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:14.747 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:14.747 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:14.747 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:14.747 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:14.747 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.747 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:14.747 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:14.747 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:14.747 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T16:16:14.747 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T16:16:14.747 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T16:16:14.747 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:14.747 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 3, 2024-04-07T16:16:14.747 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:14.747 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.748 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:14.748 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:14.748 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T16:16:14.748 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T16:16:14.748 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T16:16:14.748 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:14.748 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 7, 2024-04-07T16:16:14.748 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:14.748 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:14.748 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T16:16:14.748 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:14.748 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:14.748 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:14.749 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.637842+0000", 2024-04-07T16:16:14.749 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.401411+0000", 2024-04-07T16:16:14.749 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.401411+0000", 2024-04-07T16:16:14.749 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.401766+0000", 2024-04-07T16:16:14.749 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.637842+0000", 2024-04-07T16:16:14.749 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:14.749 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:14.749 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:14.749 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 61, 2024-04-07T16:16:14.749 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.637842+0000", 2024-04-07T16:16:14.749 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.637842+0000", 2024-04-07T16:16:14.749 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.637842+0000", 2024-04-07T16:16:14.749 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:14.749 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:14.750 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:14.750 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.637842+0000", 2024-04-07T16:16:14.750 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.637842+0000", 2024-04-07T16:16:14.750 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:14.750 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:14.750 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:14.750 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:14.750 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 60, 2024-04-07T16:16:14.750 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:14.750 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:14.750 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:14.750 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:14.750 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:14.750 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:14.751 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:14.751 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:14.751 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.37", 2024-04-07T16:16:14.751 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:14.751 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:14.751 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:14.751 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 19, 2024-04-07T16:16:14.751 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:14.751 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T23:32:52.283410+0000", 2024-04-07T16:16:14.751 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:14.751 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:14.751 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:14.751 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:14.751 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:14.751 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:14.751 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:14.752 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:14.752 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:14.752 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:14.752 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:14.752 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:14.752 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:14.752 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:14.752 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:14.752 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:14.752 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:14.752 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:14.752 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:14.752 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:14.752 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:14.752 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:14.753 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:14.753 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:14.753 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:14.753 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:14.753 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:14.753 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:14.753 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:14.753 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:14.753 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:14.753 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:14.753 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:14.753 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:14.753 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:14.753 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:14.753 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:14.754 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:14.754 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:14.754 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:14.754 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:14.754 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:14.754 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:14.754 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.754 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:14.754 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:14.754 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:14.754 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T16:16:14.754 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T16:16:14.754 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T16:16:14.754 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:14.755 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 7, 2024-04-07T16:16:14.755 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:14.755 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.755 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:14.755 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:14.755 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T16:16:14.755 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T16:16:14.755 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T16:16:14.755 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:14.755 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 6, 2024-04-07T16:16:14.755 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:14.755 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:14.755 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T16:16:14.755 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:14.755 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:14.756 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:14.756 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.553143+0000", 2024-04-07T16:16:14.756 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.396094+0000", 2024-04-07T16:16:14.756 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.396094+0000", 2024-04-07T16:16:14.756 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.396409+0000", 2024-04-07T16:16:14.756 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.553143+0000", 2024-04-07T16:16:14.756 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:14.756 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:14.756 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:14.756 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 61, 2024-04-07T16:16:14.756 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.553143+0000", 2024-04-07T16:16:14.756 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.553143+0000", 2024-04-07T16:16:14.756 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.553143+0000", 2024-04-07T16:16:14.756 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:14.757 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:14.757 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:14.757 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.553143+0000", 2024-04-07T16:16:14.757 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.553143+0000", 2024-04-07T16:16:14.757 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:14.757 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:14.757 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:14.757 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:14.757 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 60, 2024-04-07T16:16:14.757 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:14.757 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:14.757 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:14.757 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:14.757 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:14.757 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:14.757 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:14.758 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:14.758 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.36", 2024-04-07T16:16:14.758 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:14.758 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:14.758 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:14.758 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 19, 2024-04-07T16:16:14.758 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:14.758 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T18:54:36.072837+0000", 2024-04-07T16:16:14.758 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:14.758 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:14.758 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:14.758 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:14.758 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:14.758 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:14.759 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:14.759 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:14.759 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:14.759 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:14.759 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:14.759 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:14.759 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:14.759 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:14.759 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:14.759 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:14.759 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:14.759 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:14.759 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:14.759 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:14.759 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:14.760 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:14.760 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:14.760 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:14.760 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:14.760 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:14.760 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:14.760 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:14.760 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:14.760 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:14.760 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:14.760 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:14.760 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:14.760 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:14.760 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:14.761 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:14.761 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:14.761 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:14.761 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:14.761 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:14.761 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:14.761 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:14.761 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:14.761 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.761 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:14.761 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:14.761 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:14.761 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T16:16:14.761 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T16:16:14.761 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T16:16:14.762 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:14.762 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 6, 2024-04-07T16:16:14.762 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:14.762 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.762 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:14.762 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:14.762 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T16:16:14.762 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T16:16:14.762 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T16:16:14.762 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:14.762 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 6, 2024-04-07T16:16:14.762 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:14.762 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:14.762 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T16:16:14.763 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:14.763 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:14.763 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:14.763 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.552867+0000", 2024-04-07T16:16:14.763 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.401724+0000", 2024-04-07T16:16:14.763 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.401724+0000", 2024-04-07T16:16:14.763 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.401973+0000", 2024-04-07T16:16:14.763 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.552867+0000", 2024-04-07T16:16:14.763 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:14.763 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:14.763 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:14.763 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 61, 2024-04-07T16:16:14.763 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.552867+0000", 2024-04-07T16:16:14.763 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.552867+0000", 2024-04-07T16:16:14.764 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.552867+0000", 2024-04-07T16:16:14.764 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:14.764 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:14.764 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:14.764 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.552867+0000", 2024-04-07T16:16:14.764 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.552867+0000", 2024-04-07T16:16:14.764 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:14.764 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:14.764 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:14.764 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:14.764 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 60, 2024-04-07T16:16:14.764 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:14.764 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:14.764 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:14.764 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:14.765 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:14.765 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:14.765 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:14.765 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:14.765 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.35", 2024-04-07T16:16:14.765 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:14.765 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:14.765 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:14.765 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 19, 2024-04-07T16:16:14.765 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:14.765 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T21:26:29.208223+0000", 2024-04-07T16:16:14.765 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:14.765 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:14.765 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:14.765 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:14.766 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:14.766 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:14.766 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:14.766 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:14.766 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:14.766 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:14.766 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:14.766 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:14.766 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:14.766 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:14.766 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:14.766 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:14.766 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:14.766 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:14.766 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:14.767 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:14.767 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:14.767 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:14.767 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:14.767 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:14.767 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:14.767 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:14.767 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:14.767 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:14.767 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:14.767 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:14.767 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:14.767 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:14.767 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:14.767 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:14.768 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:14.768 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:14.768 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:14.768 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:14.768 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:14.768 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:14.768 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:14.768 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:14.768 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:14.768 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.768 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:14.768 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:14.768 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:14.768 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T16:16:14.768 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T16:16:14.769 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T16:16:14.769 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:14.769 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 6, 2024-04-07T16:16:14.769 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:14.769 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.769 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:14.769 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:14.769 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T16:16:14.769 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T16:16:14.769 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T16:16:14.769 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:14.769 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 4, 2024-04-07T16:16:14.769 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:14.769 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:14.769 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T16:16:14.769 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:14.770 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:14.770 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:14.770 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:39.824604+0000", 2024-04-07T16:16:14.770 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.404529+0000", 2024-04-07T16:16:14.770 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.404529+0000", 2024-04-07T16:16:14.770 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.404858+0000", 2024-04-07T16:16:14.770 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:39.824604+0000", 2024-04-07T16:16:14.770 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:14.770 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:14.770 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:14.770 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 61, 2024-04-07T16:16:14.770 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:39.824604+0000", 2024-04-07T16:16:14.770 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:39.824604+0000", 2024-04-07T16:16:14.770 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:39.824604+0000", 2024-04-07T16:16:14.770 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:14.771 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:14.771 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:14.771 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:39.824604+0000", 2024-04-07T16:16:14.771 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:39.824604+0000", 2024-04-07T16:16:14.771 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:14.771 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:14.771 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:14.771 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:14.771 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 60, 2024-04-07T16:16:14.771 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:14.771 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:14.771 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:14.771 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:14.771 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:14.771 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:14.772 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:14.772 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:14.772 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.34", 2024-04-07T16:16:14.772 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:14.772 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:14.772 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:14.772 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 19, 2024-04-07T16:16:14.772 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:14.772 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T19:57:22.893626+0000", 2024-04-07T16:16:14.772 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:14.772 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:14.772 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:14.772 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:14.772 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:14.772 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:14.773 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:14.773 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:14.773 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:14.773 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:14.773 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:14.773 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:14.773 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:14.773 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:14.773 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:14.773 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:14.773 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:14.773 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:14.773 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:14.773 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:14.773 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:14.774 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:14.774 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:14.774 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:14.774 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:14.774 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:14.774 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:14.774 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:14.774 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:14.774 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:14.774 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:14.774 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:14.774 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:14.774 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:14.774 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:14.775 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:14.775 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:14.775 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:14.775 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:14.775 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:14.775 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:14.775 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:14.775 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:14.775 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.775 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:14.775 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:14.775 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:14.775 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T16:16:14.775 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T16:16:14.775 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T16:16:14.775 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:14.776 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 4, 2024-04-07T16:16:14.776 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:14.776 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.776 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:14.776 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:14.776 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T16:16:14.776 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T16:16:14.776 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T16:16:14.776 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:14.776 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 3, 2024-04-07T16:16:14.776 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:14.776 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:14.776 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T16:16:14.776 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:14.777 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:14.777 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:14.777 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.186817+0000", 2024-04-07T16:16:14.777 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.394402+0000", 2024-04-07T16:16:14.777 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.394402+0000", 2024-04-07T16:16:14.777 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.394760+0000", 2024-04-07T16:16:14.777 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.186817+0000", 2024-04-07T16:16:14.777 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:14.777 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:14.777 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:14.777 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 61, 2024-04-07T16:16:14.777 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.186817+0000", 2024-04-07T16:16:14.777 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.186817+0000", 2024-04-07T16:16:14.777 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.186817+0000", 2024-04-07T16:16:14.777 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:14.778 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:14.778 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:14.778 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.186817+0000", 2024-04-07T16:16:14.778 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.186817+0000", 2024-04-07T16:16:14.778 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:14.778 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:14.778 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:14.778 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:14.778 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 60, 2024-04-07T16:16:14.778 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:14.778 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:14.778 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:14.778 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:14.779 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:14.779 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:14.779 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:14.779 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:14.779 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.33", 2024-04-07T16:16:14.779 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:14.779 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:14.779 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:14.779 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 19, 2024-04-07T16:16:14.779 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:14.779 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T18:47:13.891972+0000", 2024-04-07T16:16:14.779 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:14.779 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:14.779 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:14.779 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:14.780 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:14.780 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:14.780 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:14.780 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:14.780 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:14.780 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:14.780 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:14.780 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:14.780 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:14.780 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:14.780 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:14.780 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:14.780 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:14.780 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:14.781 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:14.781 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:14.781 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:14.781 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:14.781 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:14.781 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:14.781 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:14.781 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:14.781 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:14.781 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:14.781 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:14.781 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:14.781 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:14.781 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:14.781 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:14.782 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:14.782 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:14.782 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:14.782 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:14.782 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:14.782 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:14.782 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:14.782 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:14.782 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:14.782 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:14.782 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.782 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:14.782 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:14.783 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:14.783 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T16:16:14.783 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T16:16:14.783 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T16:16:14.783 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:14.783 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 3, 2024-04-07T16:16:14.783 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:14.783 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.783 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:14.783 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:14.783 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T16:16:14.783 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T16:16:14.783 INFO:tasks.workunit.client.0.smithi100.stdout: 5 2024-04-07T16:16:14.783 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:14.783 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 0, 2024-04-07T16:16:14.784 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:14.784 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:14.784 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T16:16:14.784 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:14.784 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:14.784 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:14.784 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.029316+0000", 2024-04-07T16:16:14.784 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.400028+0000", 2024-04-07T16:16:14.784 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.400028+0000", 2024-04-07T16:16:14.784 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.400292+0000", 2024-04-07T16:16:14.784 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.029316+0000", 2024-04-07T16:16:14.784 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:14.784 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:14.784 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:14.785 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 61, 2024-04-07T16:16:14.785 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.029316+0000", 2024-04-07T16:16:14.785 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.029316+0000", 2024-04-07T16:16:14.785 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.029316+0000", 2024-04-07T16:16:14.785 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:14.785 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:14.785 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:14.785 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.029316+0000", 2024-04-07T16:16:14.785 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.029316+0000", 2024-04-07T16:16:14.785 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:14.785 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:14.785 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:14.785 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:14.785 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 60, 2024-04-07T16:16:14.786 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:14.786 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:14.786 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:14.786 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:14.786 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:14.786 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:14.786 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:14.786 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:14.786 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.32", 2024-04-07T16:16:14.786 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:14.786 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:14.786 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:14.786 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 19, 2024-04-07T16:16:14.787 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:14.787 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T18:14:33.693852+0000", 2024-04-07T16:16:14.787 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:14.787 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:14.787 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:14.787 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:14.787 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:14.787 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:14.787 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:14.787 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:14.787 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:14.787 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:14.787 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:14.787 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:14.788 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:14.788 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:14.788 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:14.788 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:14.788 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:14.788 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:14.788 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:14.788 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:14.788 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:14.788 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:14.788 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:14.788 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:14.788 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:14.788 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:14.788 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:14.789 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:14.789 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:14.789 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:14.789 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:14.789 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:14.789 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:14.789 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:14.789 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:14.789 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:14.789 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:14.789 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:14.789 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:14.789 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:14.789 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:14.789 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:14.790 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:14.790 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.790 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:14.790 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:14.790 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:14.790 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T16:16:14.790 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T16:16:14.790 INFO:tasks.workunit.client.0.smithi100.stdout: 5 2024-04-07T16:16:14.790 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:14.790 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 0, 2024-04-07T16:16:14.790 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:14.790 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.790 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:14.790 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:14.790 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T16:16:14.791 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T16:16:14.791 INFO:tasks.workunit.client.0.smithi100.stdout: 5 2024-04-07T16:16:14.791 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:14.791 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 1, 2024-04-07T16:16:14.791 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:14.791 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:14.791 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T16:16:14.791 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:14.791 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:14.791 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:14.791 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.235899+0000", 2024-04-07T16:16:14.791 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.398287+0000", 2024-04-07T16:16:14.791 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.398287+0000", 2024-04-07T16:16:14.791 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.398535+0000", 2024-04-07T16:16:14.792 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.235899+0000", 2024-04-07T16:16:14.792 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:14.792 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:14.792 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:14.792 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 61, 2024-04-07T16:16:14.792 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.235899+0000", 2024-04-07T16:16:14.792 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.235899+0000", 2024-04-07T16:16:14.792 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.235899+0000", 2024-04-07T16:16:14.792 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:14.792 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:14.792 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:14.792 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.235899+0000", 2024-04-07T16:16:14.792 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.235899+0000", 2024-04-07T16:16:14.792 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:14.793 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:14.793 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:14.793 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:14.793 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 60, 2024-04-07T16:16:14.793 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:14.793 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:14.793 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:14.793 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:14.793 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:14.793 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:14.793 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:14.793 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:14.793 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.31", 2024-04-07T16:16:14.793 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:14.793 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:14.794 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:14.794 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 19, 2024-04-07T16:16:14.794 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:14.794 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T23:58:53.081885+0000", 2024-04-07T16:16:14.794 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:14.794 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:14.794 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:14.794 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:14.794 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:14.794 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:14.794 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:14.794 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:14.794 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:14.794 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:14.794 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:14.795 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:14.795 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:14.795 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:14.795 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:14.795 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:14.795 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:14.795 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:14.795 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:14.795 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:14.795 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:14.795 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:14.795 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:14.795 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:14.795 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:14.795 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:14.796 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:14.796 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:14.796 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:14.796 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:14.796 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:14.796 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:14.796 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:14.796 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:14.796 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:14.796 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:14.796 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:14.796 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:14.796 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:14.796 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:14.796 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:14.797 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:14.797 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:14.797 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.797 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:14.797 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:14.797 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:14.797 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T16:16:14.797 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T16:16:14.797 INFO:tasks.workunit.client.0.smithi100.stdout: 5 2024-04-07T16:16:14.797 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:14.797 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 1, 2024-04-07T16:16:14.797 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:14.797 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.797 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:14.797 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:14.798 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T16:16:14.798 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T16:16:14.798 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T16:16:14.798 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:14.798 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 5, 2024-04-07T16:16:14.798 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:14.798 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:14.798 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T16:16:14.798 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:14.798 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:14.798 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:14.798 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.023905+0000", 2024-04-07T16:16:14.798 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.394956+0000", 2024-04-07T16:16:14.799 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.394956+0000", 2024-04-07T16:16:14.799 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.395243+0000", 2024-04-07T16:16:14.799 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.023905+0000", 2024-04-07T16:16:14.799 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:14.799 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:14.799 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:14.799 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 61, 2024-04-07T16:16:14.799 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.023905+0000", 2024-04-07T16:16:14.799 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.023905+0000", 2024-04-07T16:16:14.799 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.023905+0000", 2024-04-07T16:16:14.799 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:14.799 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:14.799 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:14.799 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.023905+0000", 2024-04-07T16:16:14.799 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.023905+0000", 2024-04-07T16:16:14.800 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:14.800 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:14.800 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:14.800 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:14.800 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 60, 2024-04-07T16:16:14.800 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:14.800 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:14.800 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:14.800 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:14.800 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:14.800 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:14.800 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:14.800 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:14.800 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.30", 2024-04-07T16:16:14.800 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:14.801 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:14.801 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:14.801 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 19, 2024-04-07T16:16:14.801 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:14.801 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T03:04:58.804429+0000", 2024-04-07T16:16:14.801 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:14.801 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:14.801 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:14.801 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:14.801 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:14.801 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:14.801 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:14.801 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:14.801 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:14.801 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:14.802 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:14.802 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:14.802 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:14.802 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:14.802 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:14.802 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:14.802 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:14.802 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:14.802 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:14.802 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:14.802 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:14.802 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:14.802 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:14.802 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:14.802 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:14.803 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:14.803 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:14.803 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:14.803 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:14.803 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:14.803 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:14.803 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:14.803 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:14.803 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:14.803 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:14.803 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:14.803 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:14.803 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:14.803 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:14.804 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:14.804 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:14.804 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:14.804 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:14.804 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.804 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:14.804 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:14.804 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:14.804 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T16:16:14.804 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T16:16:14.804 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T16:16:14.804 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:14.804 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 5, 2024-04-07T16:16:14.804 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:14.804 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.805 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:14.805 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:14.805 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T16:16:14.805 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T16:16:14.805 INFO:tasks.workunit.client.0.smithi100.stdout: 4 2024-04-07T16:16:14.805 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:14.805 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 5, 2024-04-07T16:16:14.805 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:14.805 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:14.805 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T16:16:14.805 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:14.805 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:14.805 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:14.805 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.026420+0000", 2024-04-07T16:16:14.806 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.398198+0000", 2024-04-07T16:16:14.806 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.398198+0000", 2024-04-07T16:16:14.806 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.398543+0000", 2024-04-07T16:16:14.806 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.026420+0000", 2024-04-07T16:16:14.806 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:14.806 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:14.806 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:14.806 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 61, 2024-04-07T16:16:14.806 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.026420+0000", 2024-04-07T16:16:14.806 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.026420+0000", 2024-04-07T16:16:14.806 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.026420+0000", 2024-04-07T16:16:14.806 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:14.806 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:14.806 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:14.807 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.026420+0000", 2024-04-07T16:16:14.807 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.026420+0000", 2024-04-07T16:16:14.807 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:14.807 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:14.807 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:14.807 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:14.807 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 60, 2024-04-07T16:16:14.807 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:14.807 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:14.807 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:14.807 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:14.807 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:14.808 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:14.808 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:14.808 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:14.808 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.2f", 2024-04-07T16:16:14.808 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:14.808 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:14.808 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:14.808 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 19, 2024-04-07T16:16:14.808 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:14.808 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T22:57:24.895712+0000", 2024-04-07T16:16:14.808 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:14.808 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:14.808 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:14.808 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:14.809 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:14.809 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:14.809 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:14.809 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:14.809 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:14.809 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:14.809 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:14.809 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:14.809 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:14.809 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:14.809 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:14.809 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:14.809 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:14.809 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:14.810 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:14.810 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:14.810 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:14.810 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:14.810 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:14.810 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:14.810 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:14.810 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:14.810 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:14.810 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:14.810 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:14.810 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:14.810 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:14.810 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:14.810 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:14.811 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:14.811 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:14.811 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:14.811 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:14.811 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:14.811 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:14.811 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:14.811 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:14.811 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:14.811 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:14.811 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.811 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:14.811 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:14.811 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:14.811 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T16:16:14.812 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T16:16:14.812 INFO:tasks.workunit.client.0.smithi100.stdout: 4 2024-04-07T16:16:14.812 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:14.812 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 5, 2024-04-07T16:16:14.812 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:14.812 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.812 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:14.812 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:14.812 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T16:16:14.812 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T16:16:14.812 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T16:16:14.812 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:14.812 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 2, 2024-04-07T16:16:14.812 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:14.812 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:14.812 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T16:16:14.813 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:14.813 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:14.813 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:14.813 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.764814+0000", 2024-04-07T16:16:14.813 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.415972+0000", 2024-04-07T16:16:14.813 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.415972+0000", 2024-04-07T16:16:14.813 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.416210+0000", 2024-04-07T16:16:14.813 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.764814+0000", 2024-04-07T16:16:14.813 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:14.813 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:14.813 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:14.813 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 61, 2024-04-07T16:16:14.813 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.764814+0000", 2024-04-07T16:16:14.813 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.764814+0000", 2024-04-07T16:16:14.813 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.764814+0000", 2024-04-07T16:16:14.814 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:14.814 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:14.814 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:14.814 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.764814+0000", 2024-04-07T16:16:14.814 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.764814+0000", 2024-04-07T16:16:14.814 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:14.814 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:14.814 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:14.814 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:14.814 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 60, 2024-04-07T16:16:14.814 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:14.814 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:14.814 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:14.814 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:14.814 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:14.815 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:14.815 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:14.815 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:14.815 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.2e", 2024-04-07T16:16:14.815 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:14.815 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:14.815 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:14.815 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 19, 2024-04-07T16:16:14.815 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:14.815 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T23:40:11.606978+0000", 2024-04-07T16:16:14.815 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:14.815 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:14.815 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:14.815 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:14.815 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:14.815 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:14.816 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:14.816 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:14.816 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:14.816 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:14.816 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:14.816 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:14.816 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:14.816 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:14.816 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:14.816 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:14.816 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:14.816 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:14.816 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:14.816 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:14.816 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:14.816 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:14.817 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:14.817 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:14.817 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:14.817 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:14.817 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:14.817 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:14.817 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:14.817 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:14.817 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:14.817 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:14.817 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:14.817 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:14.817 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:14.817 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:14.817 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:14.817 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:14.818 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:14.818 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:14.818 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:14.818 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:14.818 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:14.818 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.818 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:14.818 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:14.818 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:14.818 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T16:16:14.818 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T16:16:14.818 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T16:16:14.818 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:14.818 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 2, 2024-04-07T16:16:14.818 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:14.819 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.819 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:14.819 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:14.819 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T16:16:14.819 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T16:16:14.819 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T16:16:14.819 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:14.819 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 7, 2024-04-07T16:16:14.819 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:14.819 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:14.819 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T16:16:14.819 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:14.819 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:14.819 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:14.819 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.636945+0000", 2024-04-07T16:16:14.819 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.413869+0000", 2024-04-07T16:16:14.819 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.413869+0000", 2024-04-07T16:16:14.820 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.414203+0000", 2024-04-07T16:16:14.820 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.636945+0000", 2024-04-07T16:16:14.820 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:14.820 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:14.820 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:14.820 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 61, 2024-04-07T16:16:14.820 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.636945+0000", 2024-04-07T16:16:14.820 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.636945+0000", 2024-04-07T16:16:14.820 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.636945+0000", 2024-04-07T16:16:14.820 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:14.820 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:14.820 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:14.820 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.636945+0000", 2024-04-07T16:16:14.820 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.636945+0000", 2024-04-07T16:16:14.820 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:14.821 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:14.821 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:14.821 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:14.821 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 60, 2024-04-07T16:16:14.821 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:14.821 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:14.821 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:14.821 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:14.821 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:14.821 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:14.821 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:14.821 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:14.821 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.2d", 2024-04-07T16:16:14.821 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:14.821 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:14.821 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:14.822 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 19, 2024-04-07T16:16:14.822 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:14.822 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T17:11:47.895568+0000", 2024-04-07T16:16:14.822 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:14.822 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:14.822 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:14.822 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:14.822 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:14.822 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:14.822 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:14.822 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:14.822 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:14.822 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:14.822 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:14.823 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:14.823 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:14.823 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:14.823 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:14.823 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:14.823 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:14.823 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:14.823 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:14.823 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:14.823 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:14.823 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:14.823 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:14.823 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:14.823 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:14.823 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:14.824 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:14.824 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:14.824 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:14.824 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:14.824 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:14.824 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:14.824 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:14.824 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:14.824 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:14.824 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:14.824 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:14.824 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:14.824 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:14.824 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:14.825 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:14.825 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:14.825 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:14.825 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.825 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:14.825 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:14.825 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:14.825 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T16:16:14.825 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T16:16:14.825 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T16:16:14.825 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:14.825 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 7, 2024-04-07T16:16:14.825 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:14.826 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.826 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:14.826 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:14.826 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T16:16:14.826 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T16:16:14.826 INFO:tasks.workunit.client.0.smithi100.stdout: 4 2024-04-07T16:16:14.826 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:14.826 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 0, 2024-04-07T16:16:14.826 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:14.826 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:14.826 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T16:16:14.826 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:14.826 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:14.826 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:14.827 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.031549+0000", 2024-04-07T16:16:14.827 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.032199+0000", 2024-04-07T16:16:14.827 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.032199+0000", 2024-04-07T16:16:14.827 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.032601+0000", 2024-04-07T16:16:14.827 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.031549+0000", 2024-04-07T16:16:14.827 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:14.827 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:14.827 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:14.827 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 60, 2024-04-07T16:16:14.827 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.031549+0000", 2024-04-07T16:16:14.827 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.031549+0000", 2024-04-07T16:16:14.827 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.031549+0000", 2024-04-07T16:16:14.827 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:14.827 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:14.827 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:14.828 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.031549+0000", 2024-04-07T16:16:14.828 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.031549+0000", 2024-04-07T16:16:14.828 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:14.828 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 4, 2024-04-07T16:16:14.828 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:14.828 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:14.828 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 59, 2024-04-07T16:16:14.828 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:14.828 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:14.828 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:14.828 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:14.828 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 4, 2024-04-07T16:16:14.828 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:14.828 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:14.829 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:14.829 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.1f", 2024-04-07T16:16:14.829 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:14.829 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:14.829 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:14.829 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 24, 2024-04-07T16:16:14.829 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:14.829 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T19:01:44.798719+0000", 2024-04-07T16:16:14.829 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:14.829 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:14.829 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:14.829 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 582, 2024-04-07T16:16:14.829 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:14.829 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:14.829 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:14.829 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:14.830 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:14.830 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:14.830 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:14.830 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:14.830 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:14.830 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:14.830 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:14.830 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:14.830 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:14.830 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:14.830 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:14.830 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 6, 2024-04-07T16:16:14.830 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 2, 2024-04-07T16:16:14.830 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:14.830 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 2, 2024-04-07T16:16:14.831 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:14.831 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:14.831 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:14.831 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:14.831 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:14.831 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 1, 2024-04-07T16:16:14.831 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:14.831 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:14.831 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:14.831 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:14.831 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:14.831 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:14.831 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:14.831 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:14.831 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:14.831 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:14.832 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:14.832 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:14.832 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 4, 2024-04-07T16:16:14.832 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 1 2024-04-07T16:16:14.832 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.832 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:14.832 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:14.832 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:14.832 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T16:16:14.832 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T16:16:14.832 INFO:tasks.workunit.client.0.smithi100.stdout: 4 2024-04-07T16:16:14.832 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:14.832 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 0, 2024-04-07T16:16:14.832 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "61'4" 2024-04-07T16:16:14.832 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.832 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:14.833 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:14.833 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T16:16:14.833 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T16:16:14.833 INFO:tasks.workunit.client.0.smithi100.stdout: 2 2024-04-07T16:16:14.833 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:14.833 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 3, 2024-04-07T16:16:14.833 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:14.833 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:14.833 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T16:16:14.833 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:14.833 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:14.833 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:14.833 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.187256+0000", 2024-04-07T16:16:14.833 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.407538+0000", 2024-04-07T16:16:14.833 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.407538+0000", 2024-04-07T16:16:14.833 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.407852+0000", 2024-04-07T16:16:14.834 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.187256+0000", 2024-04-07T16:16:14.834 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:14.834 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:14.834 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:14.834 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 61, 2024-04-07T16:16:14.834 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.187256+0000", 2024-04-07T16:16:14.834 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.187256+0000", 2024-04-07T16:16:14.834 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.187256+0000", 2024-04-07T16:16:14.834 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:14.834 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:14.834 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:14.834 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.187256+0000", 2024-04-07T16:16:14.834 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.187256+0000", 2024-04-07T16:16:14.835 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:14.835 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:14.835 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:14.835 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:14.835 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 60, 2024-04-07T16:16:14.835 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:14.835 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:14.835 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:14.835 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:14.835 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:14.835 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:14.835 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:14.835 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:14.835 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.1e", 2024-04-07T16:16:14.835 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:14.836 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:14.836 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:14.836 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 19, 2024-04-07T16:16:14.836 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:14.836 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T20:27:52.664149+0000", 2024-04-07T16:16:14.836 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:14.836 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:14.836 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:14.836 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:14.836 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:14.836 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:14.836 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:14.836 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:14.836 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:14.836 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:14.837 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:14.837 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:14.837 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:14.837 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:14.837 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:14.837 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:14.837 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:14.837 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:14.837 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:14.837 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:14.837 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:14.837 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:14.837 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:14.837 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:14.837 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:14.837 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:14.838 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:14.838 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:14.838 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:14.838 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:14.838 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:14.838 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:14.838 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:14.838 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:14.838 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:14.838 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:14.838 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:14.838 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:14.838 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:14.838 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:14.838 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:14.839 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:14.839 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:14.839 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.839 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:14.839 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:14.839 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:14.839 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T16:16:14.839 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T16:16:14.839 INFO:tasks.workunit.client.0.smithi100.stdout: 2 2024-04-07T16:16:14.839 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:14.839 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 3, 2024-04-07T16:16:14.839 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:14.839 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.839 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:14.839 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:14.840 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T16:16:14.840 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T16:16:14.840 INFO:tasks.workunit.client.0.smithi100.stdout: 5 2024-04-07T16:16:14.840 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:14.840 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 3, 2024-04-07T16:16:14.840 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:14.840 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:14.840 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T16:16:14.840 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:14.840 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:14.840 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:14.840 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.187484+0000", 2024-04-07T16:16:14.840 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.038492+0000", 2024-04-07T16:16:14.840 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.038492+0000", 2024-04-07T16:16:14.840 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.038807+0000", 2024-04-07T16:16:14.841 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.187484+0000", 2024-04-07T16:16:14.841 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:14.841 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:14.841 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:14.841 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 60, 2024-04-07T16:16:14.841 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.187484+0000", 2024-04-07T16:16:14.841 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.187484+0000", 2024-04-07T16:16:14.841 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.187484+0000", 2024-04-07T16:16:14.841 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:14.841 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:14.841 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:14.841 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.187484+0000", 2024-04-07T16:16:14.841 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.187484+0000", 2024-04-07T16:16:14.841 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:14.841 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 1, 2024-04-07T16:16:14.842 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:14.842 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:14.842 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 59, 2024-04-07T16:16:14.842 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:14.842 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:14.842 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:14.842 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:14.842 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 1, 2024-04-07T16:16:14.842 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:14.842 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:14.842 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:14.842 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.1e", 2024-04-07T16:16:14.842 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:14.842 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:14.843 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:14.843 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 21, 2024-04-07T16:16:14.843 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:14.843 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T02:11:03.746017+0000", 2024-04-07T16:16:14.843 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:14.843 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:14.843 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:14.843 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:14.843 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:14.843 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:14.843 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:14.843 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:14.843 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:14.843 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:14.843 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:14.844 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:14.844 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:14.844 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:14.844 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:14.844 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:14.844 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:14.844 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:14.844 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:14.844 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:14.844 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:14.844 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:14.844 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:14.844 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:14.844 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:14.844 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:14.844 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:14.845 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:14.845 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:14.845 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:14.845 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:14.845 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:14.845 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:14.845 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:14.845 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:14.845 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:14.845 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:14.845 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:14.845 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:14.845 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:14.845 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:14.846 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:14.846 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:14.846 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.846 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:14.846 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:14.846 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:14.846 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T16:16:14.846 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T16:16:14.846 INFO:tasks.workunit.client.0.smithi100.stdout: 5 2024-04-07T16:16:14.846 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:14.846 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 3, 2024-04-07T16:16:14.846 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "61'1" 2024-04-07T16:16:14.846 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.846 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:14.846 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:14.847 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T16:16:14.847 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T16:16:14.847 INFO:tasks.workunit.client.0.smithi100.stdout: 2 2024-04-07T16:16:14.847 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:14.847 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 0, 2024-04-07T16:16:14.847 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:14.847 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:14.847 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T16:16:14.847 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:14.847 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:14.847 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:14.847 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.031268+0000", 2024-04-07T16:16:14.847 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.409049+0000", 2024-04-07T16:16:14.848 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.409049+0000", 2024-04-07T16:16:14.848 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.410031+0000", 2024-04-07T16:16:14.848 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.031268+0000", 2024-04-07T16:16:14.848 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:14.848 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:14.848 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:14.848 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 61, 2024-04-07T16:16:14.848 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.031268+0000", 2024-04-07T16:16:14.848 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.031268+0000", 2024-04-07T16:16:14.848 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.031268+0000", 2024-04-07T16:16:14.848 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:14.848 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:14.848 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:14.849 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.031268+0000", 2024-04-07T16:16:14.849 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.031268+0000", 2024-04-07T16:16:14.849 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:14.849 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:14.849 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:14.849 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:14.849 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 60, 2024-04-07T16:16:14.849 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:14.849 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:14.849 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:14.849 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:14.849 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:14.849 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:14.849 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:14.850 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:14.850 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.1f", 2024-04-07T16:16:14.850 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:14.850 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:14.850 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:14.850 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 19, 2024-04-07T16:16:14.850 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:14.850 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T21:35:09.308017+0000", 2024-04-07T16:16:14.850 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:14.850 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:14.850 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:14.850 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:14.850 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:14.850 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:14.851 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:14.851 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:14.851 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:14.851 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:14.851 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:14.851 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:14.851 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:14.851 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:14.851 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:14.851 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:14.851 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:14.851 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:14.851 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:14.851 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:14.852 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:14.852 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:14.852 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:14.852 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:14.852 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:14.852 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:14.852 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:14.852 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:14.852 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:14.852 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:14.852 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:14.852 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:14.852 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:14.852 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:14.852 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:14.853 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:14.853 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:14.853 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:14.853 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:14.853 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:14.853 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:14.853 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:14.853 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:14.853 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.853 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:14.853 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:14.853 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:14.853 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T16:16:14.853 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T16:16:14.854 INFO:tasks.workunit.client.0.smithi100.stdout: 2 2024-04-07T16:16:14.854 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:14.854 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 0, 2024-04-07T16:16:14.854 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:14.854 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.854 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:14.854 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:14.854 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T16:16:14.854 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T16:16:14.854 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T16:16:14.854 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:14.854 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 7, 2024-04-07T16:16:14.854 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:14.854 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:14.855 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T16:16:14.855 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:14.855 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:14.855 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:14.855 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.637577+0000", 2024-04-07T16:16:14.855 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.041970+0000", 2024-04-07T16:16:14.855 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.041970+0000", 2024-04-07T16:16:14.855 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.042401+0000", 2024-04-07T16:16:14.855 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.637577+0000", 2024-04-07T16:16:14.855 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:14.855 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:14.855 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:14.855 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 60, 2024-04-07T16:16:14.855 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.637577+0000", 2024-04-07T16:16:14.856 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.637577+0000", 2024-04-07T16:16:14.856 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.637577+0000", 2024-04-07T16:16:14.856 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:14.856 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:14.856 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:14.856 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.637577+0000", 2024-04-07T16:16:14.856 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.637577+0000", 2024-04-07T16:16:14.856 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:14.856 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 2, 2024-04-07T16:16:14.856 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:14.856 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:14.856 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 59, 2024-04-07T16:16:14.856 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:14.856 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:14.857 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:14.857 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:14.857 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 2, 2024-04-07T16:16:14.857 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:14.857 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:14.857 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:14.857 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.1d", 2024-04-07T16:16:14.857 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:14.857 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:14.857 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:14.857 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 22, 2024-04-07T16:16:14.857 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:14.857 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T16:36:29.235347+0000", 2024-04-07T16:16:14.857 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:14.857 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:14.857 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:14.858 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:14.858 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:14.858 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:14.858 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:14.858 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:14.858 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:14.858 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:14.858 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:14.858 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:14.858 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:14.858 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:14.858 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:14.858 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:14.858 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:14.858 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:14.859 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:14.859 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 3, 2024-04-07T16:16:14.859 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 1, 2024-04-07T16:16:14.859 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:14.859 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 1, 2024-04-07T16:16:14.859 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:14.859 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:14.859 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:14.859 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:14.859 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:14.859 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 1, 2024-04-07T16:16:14.859 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:14.859 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:14.859 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:14.859 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:14.859 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:14.860 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:14.860 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:14.860 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:14.860 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:14.860 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:14.860 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:14.860 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:14.860 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 3, 2024-04-07T16:16:14.860 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:14.860 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.860 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:14.860 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:14.860 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:14.860 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T16:16:14.860 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T16:16:14.861 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T16:16:14.861 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:14.861 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 7, 2024-04-07T16:16:14.861 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "61'2" 2024-04-07T16:16:14.861 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.861 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:14.861 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:14.861 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T16:16:14.861 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T16:16:14.861 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T16:16:14.861 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:14.861 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 5, 2024-04-07T16:16:14.861 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:14.861 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:14.861 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T16:16:14.862 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:14.862 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:14.862 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:14.862 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.025650+0000", 2024-04-07T16:16:14.862 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.394271+0000", 2024-04-07T16:16:14.862 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.394271+0000", 2024-04-07T16:16:14.862 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.394718+0000", 2024-04-07T16:16:14.862 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.025650+0000", 2024-04-07T16:16:14.862 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:14.862 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:14.862 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:14.862 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 61, 2024-04-07T16:16:14.862 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.025650+0000", 2024-04-07T16:16:14.863 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.025650+0000", 2024-04-07T16:16:14.863 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.025650+0000", 2024-04-07T16:16:14.863 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:14.863 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:14.863 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:14.863 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.025650+0000", 2024-04-07T16:16:14.863 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.025650+0000", 2024-04-07T16:16:14.863 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:14.863 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:14.863 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:14.863 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:14.863 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 60, 2024-04-07T16:16:14.863 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:14.863 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:14.864 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:14.864 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:14.864 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:14.864 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:14.864 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:14.864 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:14.864 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.1c", 2024-04-07T16:16:14.864 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:14.864 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:14.864 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:14.864 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 19, 2024-04-07T16:16:14.864 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:14.864 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T20:10:29.096403+0000", 2024-04-07T16:16:14.864 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:14.864 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:14.865 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:14.865 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:14.865 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:14.865 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:14.865 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:14.865 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:14.865 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:14.865 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:14.865 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:14.865 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:14.865 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:14.865 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:14.865 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:14.866 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:14.866 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:14.866 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:14.866 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:14.866 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:14.866 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:14.866 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:14.866 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:14.866 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:14.866 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:14.866 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:14.866 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:14.866 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:14.867 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:14.867 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:14.867 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:14.867 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:14.867 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:14.867 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:14.867 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:14.867 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:14.867 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:14.867 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:14.867 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:14.867 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:14.867 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:14.867 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:14.867 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:14.868 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.868 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:14.868 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:14.868 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:14.868 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T16:16:14.868 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T16:16:14.868 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T16:16:14.868 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:14.868 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 5, 2024-04-07T16:16:14.868 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:14.868 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.868 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:14.868 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:14.868 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T16:16:14.869 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T16:16:14.869 INFO:tasks.workunit.client.0.smithi100.stdout: 2 2024-04-07T16:16:14.869 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:14.869 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 7, 2024-04-07T16:16:14.869 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:14.869 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:14.869 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T16:16:14.869 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:14.869 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:14.869 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:14.869 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.637065+0000", 2024-04-07T16:16:14.869 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.041919+0000", 2024-04-07T16:16:14.869 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.041919+0000", 2024-04-07T16:16:14.869 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.042391+0000", 2024-04-07T16:16:14.870 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.637065+0000", 2024-04-07T16:16:14.870 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:14.870 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:14.870 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:14.870 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 60, 2024-04-07T16:16:14.870 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.637065+0000", 2024-04-07T16:16:14.870 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.637065+0000", 2024-04-07T16:16:14.870 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.637065+0000", 2024-04-07T16:16:14.870 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:14.870 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:14.870 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:14.870 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.637065+0000", 2024-04-07T16:16:14.870 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.637065+0000", 2024-04-07T16:16:14.870 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:14.870 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 2, 2024-04-07T16:16:14.871 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:14.871 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:14.871 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 59, 2024-04-07T16:16:14.871 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:14.871 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:14.871 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:14.871 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:14.871 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 2, 2024-04-07T16:16:14.871 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:14.871 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:14.871 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:14.871 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.1c", 2024-04-07T16:16:14.871 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:14.871 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:14.871 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:14.872 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 22, 2024-04-07T16:16:14.872 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:14.872 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T21:50:59.576615+0000", 2024-04-07T16:16:14.872 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:14.872 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:14.872 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:14.872 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:14.872 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:14.872 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:14.872 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:14.872 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:14.872 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:14.872 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:14.872 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:14.872 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:14.873 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:14.873 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:14.873 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:14.873 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:14.873 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:14.873 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:14.873 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:14.873 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 3, 2024-04-07T16:16:14.873 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 1, 2024-04-07T16:16:14.873 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:14.873 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 1, 2024-04-07T16:16:14.873 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:14.873 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:14.873 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:14.873 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:14.874 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:14.874 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 1, 2024-04-07T16:16:14.874 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:14.874 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:14.874 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:14.874 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:14.874 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:14.874 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:14.874 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:14.874 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:14.874 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:14.874 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:14.874 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:14.874 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:14.874 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 3, 2024-04-07T16:16:14.875 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:14.875 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.875 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:14.875 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:14.875 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:14.875 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T16:16:14.875 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T16:16:14.875 INFO:tasks.workunit.client.0.smithi100.stdout: 2 2024-04-07T16:16:14.875 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:14.875 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 7, 2024-04-07T16:16:14.875 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "61'2" 2024-04-07T16:16:14.875 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.875 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:14.875 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:14.875 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T16:16:14.876 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T16:16:14.876 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T16:16:14.876 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:14.876 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 5, 2024-04-07T16:16:14.876 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:14.876 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:14.876 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T16:16:14.876 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:14.876 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:14.876 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:14.876 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.025658+0000", 2024-04-07T16:16:14.876 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:26.222249+0000", 2024-04-07T16:16:14.876 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:26.222249+0000", 2024-04-07T16:16:14.876 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:26.222601+0000", 2024-04-07T16:16:14.876 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.025658+0000", 2024-04-07T16:16:14.877 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:14.877 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:14.877 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:14.877 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 64, 2024-04-07T16:16:14.877 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.025658+0000", 2024-04-07T16:16:14.877 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.025658+0000", 2024-04-07T16:16:14.877 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.025658+0000", 2024-04-07T16:16:14.877 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:14.877 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:14.877 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:14.878 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.025658+0000", 2024-04-07T16:16:14.878 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.025658+0000", 2024-04-07T16:16:14.878 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:14.878 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:14.878 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:14.878 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:14.878 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 63, 2024-04-07T16:16:14.878 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:14.878 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:14.878 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:14.878 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:14.878 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:14.878 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:14.878 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:14.879 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:14.879 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.1d", 2024-04-07T16:16:14.879 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:14.879 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:14.879 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:14.879 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 28, 2024-04-07T16:16:14.879 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:14.879 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T16:28:47.005643+0000", 2024-04-07T16:16:14.879 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:14.879 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:14.879 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:14.879 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:14.879 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:14.879 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:14.879 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:14.880 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:14.880 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:14.880 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:14.880 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:14.880 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:14.880 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:14.880 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:14.880 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:14.880 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:14.880 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:14.880 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:14.880 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:14.880 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:14.880 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:14.881 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:14.881 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:14.881 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:14.881 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:14.881 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:14.881 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:14.881 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:14.881 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:14.881 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:14.881 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:14.881 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:14.881 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:14.881 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:14.881 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:14.881 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:14.882 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:14.882 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:14.882 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:14.882 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:14.882 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:14.882 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:14.882 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:14.882 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.882 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:14.882 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:14.882 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:14.882 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T16:16:14.882 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T16:16:14.883 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T16:16:14.883 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:14.883 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 5, 2024-04-07T16:16:14.883 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:14.883 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.883 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:14.883 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:14.883 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T16:16:14.883 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T16:16:14.883 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T16:16:14.883 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:14.883 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 3, 2024-04-07T16:16:14.883 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:14.883 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:14.883 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T16:16:14.884 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:14.884 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:14.884 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:14.884 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.187224+0000", 2024-04-07T16:16:14.884 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.041409+0000", 2024-04-07T16:16:14.884 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.041409+0000", 2024-04-07T16:16:14.884 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.041880+0000", 2024-04-07T16:16:14.884 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.187224+0000", 2024-04-07T16:16:14.884 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:14.884 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:14.884 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:14.884 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 60, 2024-04-07T16:16:14.884 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.187224+0000", 2024-04-07T16:16:14.884 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.187224+0000", 2024-04-07T16:16:14.884 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.187224+0000", 2024-04-07T16:16:14.885 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:14.885 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:14.885 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:14.885 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.187224+0000", 2024-04-07T16:16:14.885 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.187224+0000", 2024-04-07T16:16:14.885 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:14.885 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 1, 2024-04-07T16:16:14.885 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:14.885 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:14.885 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 59, 2024-04-07T16:16:14.885 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:14.885 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:14.885 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:14.885 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:14.885 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 1, 2024-04-07T16:16:14.886 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:14.886 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:14.886 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:14.886 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.1b", 2024-04-07T16:16:14.886 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:14.886 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:14.886 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:14.886 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 21, 2024-04-07T16:16:14.886 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:14.886 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T18:32:23.206985+0000", 2024-04-07T16:16:14.886 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:14.886 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:14.886 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:14.886 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:14.887 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:14.887 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:14.887 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:14.887 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:14.887 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:14.887 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:14.887 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:14.887 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:14.887 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:14.887 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:14.887 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:14.887 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:14.887 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:14.888 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:14.888 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:14.888 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:14.888 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:14.888 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:14.888 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:14.888 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:14.888 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:14.888 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:14.888 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:14.888 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:14.888 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:14.888 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:14.888 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:14.889 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:14.889 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:14.889 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:14.889 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:14.889 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:14.889 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:14.889 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:14.889 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:14.889 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:14.889 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:14.889 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:14.889 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:14.889 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.889 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:14.889 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:14.890 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:14.890 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T16:16:14.890 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T16:16:14.890 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T16:16:14.890 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:14.890 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 3, 2024-04-07T16:16:14.890 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "61'1" 2024-04-07T16:16:14.890 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.890 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:14.890 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:14.890 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T16:16:14.890 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T16:16:14.890 INFO:tasks.workunit.client.0.smithi100.stdout: 2 2024-04-07T16:16:14.891 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:14.891 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 4, 2024-04-07T16:16:14.891 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:14.891 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:14.891 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T16:16:14.891 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:14.891 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:14.891 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:14.891 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:39.824041+0000", 2024-04-07T16:16:14.891 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.413311+0000", 2024-04-07T16:16:14.891 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.413311+0000", 2024-04-07T16:16:14.891 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.413664+0000", 2024-04-07T16:16:14.891 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:39.824041+0000", 2024-04-07T16:16:14.891 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:14.891 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:14.891 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:14.892 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 61, 2024-04-07T16:16:14.892 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:39.824041+0000", 2024-04-07T16:16:14.892 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:39.824041+0000", 2024-04-07T16:16:14.892 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:39.824041+0000", 2024-04-07T16:16:14.892 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:14.892 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:14.892 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:14.892 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:39.824041+0000", 2024-04-07T16:16:14.892 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:39.824041+0000", 2024-04-07T16:16:14.892 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:14.892 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:14.892 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:14.893 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:14.893 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 60, 2024-04-07T16:16:14.893 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:14.893 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:14.893 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:14.893 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:14.893 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:14.893 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:14.893 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:14.893 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:14.893 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.1a", 2024-04-07T16:16:14.893 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:14.893 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:14.893 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:14.893 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 19, 2024-04-07T16:16:14.894 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:14.894 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T02:43:21.836307+0000", 2024-04-07T16:16:14.894 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:14.894 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:14.894 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:14.894 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:14.894 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:14.894 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:14.894 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:14.894 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:14.894 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:14.894 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:14.894 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:14.894 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:14.895 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:14.895 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:14.895 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:14.895 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:14.895 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:14.895 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:14.895 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:14.895 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:14.895 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:14.895 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:14.895 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:14.895 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:14.895 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:14.895 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:14.896 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:14.896 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:14.896 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:14.896 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:14.896 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:14.896 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:14.896 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:14.896 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:14.896 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:14.896 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:14.896 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:14.896 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:14.896 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:14.896 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:14.897 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:14.897 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:14.897 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:14.897 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.897 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:14.897 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:14.897 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:14.897 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T16:16:14.897 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T16:16:14.897 INFO:tasks.workunit.client.0.smithi100.stdout: 2 2024-04-07T16:16:14.897 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:14.897 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 4, 2024-04-07T16:16:14.897 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:14.897 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.897 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:14.898 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:14.898 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T16:16:14.898 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T16:16:14.898 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T16:16:14.898 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:14.898 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 6, 2024-04-07T16:16:14.898 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:14.898 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:14.898 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T16:16:14.898 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:14.898 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:14.898 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:14.898 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.526629+0000", 2024-04-07T16:16:14.898 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.038410+0000", 2024-04-07T16:16:14.899 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.038410+0000", 2024-04-07T16:16:14.899 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.038960+0000", 2024-04-07T16:16:14.899 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.526629+0000", 2024-04-07T16:16:14.899 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:14.899 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:14.899 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:14.899 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 60, 2024-04-07T16:16:14.899 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.526629+0000", 2024-04-07T16:16:14.899 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.526629+0000", 2024-04-07T16:16:14.899 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.526629+0000", 2024-04-07T16:16:14.899 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:14.899 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:14.899 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:14.899 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.526629+0000", 2024-04-07T16:16:14.899 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.526629+0000", 2024-04-07T16:16:14.900 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:14.900 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 1, 2024-04-07T16:16:14.900 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:14.900 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:14.900 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 59, 2024-04-07T16:16:14.900 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:14.900 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:14.902 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:14.902 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:14.902 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 1, 2024-04-07T16:16:14.902 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:14.902 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:14.902 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:14.902 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.1a", 2024-04-07T16:16:14.902 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:14.902 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:14.902 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:14.903 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 21, 2024-04-07T16:16:14.903 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:14.903 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T03:41:38.418312+0000", 2024-04-07T16:16:14.903 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:14.903 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:14.903 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:14.903 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:14.903 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:14.903 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:14.903 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:14.903 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:14.903 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:14.903 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:14.903 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:14.904 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:14.904 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:14.904 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:14.904 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:14.904 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:14.904 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:14.904 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:14.904 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:14.904 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:14.904 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:14.904 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:14.904 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:14.904 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:14.904 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:14.904 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:14.905 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:14.905 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:14.905 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:14.905 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:14.905 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:14.905 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:14.905 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:14.905 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:14.905 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:14.905 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:14.905 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:14.905 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:14.905 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:14.905 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:14.906 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:14.906 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:14.906 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:14.906 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.906 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:14.906 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:14.906 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:14.906 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T16:16:14.906 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T16:16:14.906 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T16:16:14.906 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:14.906 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 6, 2024-04-07T16:16:14.906 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "61'1" 2024-04-07T16:16:14.906 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.906 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:14.907 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:14.907 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T16:16:14.907 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T16:16:14.907 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T16:16:14.907 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:14.907 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 0, 2024-04-07T16:16:14.907 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:14.907 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:14.907 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T16:16:14.907 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:14.907 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:14.907 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:14.907 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.029545+0000", 2024-04-07T16:16:14.907 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.405069+0000", 2024-04-07T16:16:14.907 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.405069+0000", 2024-04-07T16:16:14.908 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.405634+0000", 2024-04-07T16:16:14.908 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.029545+0000", 2024-04-07T16:16:14.908 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:14.908 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:14.908 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:14.908 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 61, 2024-04-07T16:16:14.908 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.029545+0000", 2024-04-07T16:16:14.908 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.029545+0000", 2024-04-07T16:16:14.908 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.029545+0000", 2024-04-07T16:16:14.908 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:14.908 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:14.908 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:14.909 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.029545+0000", 2024-04-07T16:16:14.909 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.029545+0000", 2024-04-07T16:16:14.909 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:14.909 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:14.909 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:14.909 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:14.909 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 60, 2024-04-07T16:16:14.909 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:14.909 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:14.909 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:14.909 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:14.909 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:14.909 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:14.910 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:14.910 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:14.910 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.1b", 2024-04-07T16:16:14.910 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:14.910 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:14.910 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:14.910 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 19, 2024-04-07T16:16:14.910 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:14.910 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T23:50:18.973629+0000", 2024-04-07T16:16:14.910 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:14.910 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:14.910 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:14.911 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:14.911 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:14.911 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:14.911 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:14.911 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:14.911 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:14.911 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:14.911 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:14.911 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:14.911 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:14.911 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:14.911 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:14.911 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:14.911 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:14.912 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:14.912 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:14.912 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:14.912 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:14.912 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:14.912 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:14.912 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:14.912 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:14.912 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:14.912 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:14.912 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:14.912 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:14.912 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:14.912 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:14.912 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:14.913 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:14.913 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:14.913 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:14.913 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:14.913 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:14.913 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:14.913 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:14.913 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:14.913 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:14.913 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:14.913 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:14.913 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.913 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:14.913 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:14.913 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:14.914 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T16:16:14.914 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T16:16:14.914 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T16:16:14.914 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:14.914 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 0, 2024-04-07T16:16:14.914 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:14.914 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.914 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:14.914 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:14.914 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T16:16:14.914 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T16:16:14.914 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T16:16:14.914 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:14.914 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 3, 2024-04-07T16:16:14.914 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:14.915 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:14.915 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T16:16:14.915 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:14.915 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:14.915 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:14.915 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.186860+0000", 2024-04-07T16:16:14.915 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.034013+0000", 2024-04-07T16:16:14.915 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.034013+0000", 2024-04-07T16:16:14.915 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.034420+0000", 2024-04-07T16:16:14.915 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.186860+0000", 2024-04-07T16:16:14.915 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:14.915 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:14.915 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:14.916 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 60, 2024-04-07T16:16:14.916 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.186860+0000", 2024-04-07T16:16:14.916 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.186860+0000", 2024-04-07T16:16:14.916 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.186860+0000", 2024-04-07T16:16:14.916 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:14.916 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:14.916 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:14.916 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.186860+0000", 2024-04-07T16:16:14.916 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.186860+0000", 2024-04-07T16:16:14.916 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:14.916 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:14.916 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:14.916 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:14.916 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 59, 2024-04-07T16:16:14.916 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:14.917 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:14.917 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:14.917 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:14.917 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:14.917 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:14.917 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:14.917 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:14.917 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.19", 2024-04-07T16:16:14.917 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:14.917 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:14.917 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:14.917 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 20, 2024-04-07T16:16:14.917 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:14.917 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T19:51:21.525865+0000", 2024-04-07T16:16:14.917 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:14.918 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:14.918 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:14.918 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:14.918 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:14.918 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:14.918 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:14.918 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:14.918 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:14.918 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:14.918 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:14.918 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:14.918 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:14.918 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:14.918 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:14.918 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:14.919 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:14.919 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:14.919 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:14.919 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:14.919 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:14.919 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:14.919 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:14.919 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:14.919 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:14.919 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:14.919 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:14.919 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:14.919 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:14.919 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:14.919 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:14.919 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:14.920 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:14.920 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:14.920 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:14.920 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:14.920 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:14.920 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:14.920 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:14.920 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:14.920 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:14.920 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:14.920 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:14.920 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.920 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:14.920 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:14.920 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:14.921 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T16:16:14.921 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T16:16:14.921 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T16:16:14.921 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:14.921 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 3, 2024-04-07T16:16:14.921 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:14.921 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.921 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:14.921 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:14.921 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T16:16:14.921 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T16:16:14.921 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T16:16:14.921 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:14.921 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 3, 2024-04-07T16:16:14.921 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:14.922 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:14.922 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T16:16:14.922 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:14.922 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:14.922 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:14.922 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.187269+0000", 2024-04-07T16:16:14.922 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.398017+0000", 2024-04-07T16:16:14.922 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.398017+0000", 2024-04-07T16:16:14.922 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.398531+0000", 2024-04-07T16:16:14.922 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.187269+0000", 2024-04-07T16:16:14.922 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:14.922 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:14.922 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:14.922 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 61, 2024-04-07T16:16:14.922 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.187269+0000", 2024-04-07T16:16:14.923 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.187269+0000", 2024-04-07T16:16:14.923 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.187269+0000", 2024-04-07T16:16:14.923 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:14.923 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:14.923 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:14.923 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.187269+0000", 2024-04-07T16:16:14.923 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.187269+0000", 2024-04-07T16:16:14.923 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:14.923 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:14.923 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:14.923 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:14.923 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 60, 2024-04-07T16:16:14.923 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:14.924 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:14.924 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:14.924 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:14.924 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:14.924 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:14.924 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:14.924 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:14.924 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.18", 2024-04-07T16:16:14.924 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:14.924 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:14.924 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:14.924 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 19, 2024-04-07T16:16:14.924 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:14.924 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T17:33:54.306902+0000", 2024-04-07T16:16:14.924 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:14.925 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:14.925 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:14.925 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:14.925 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:14.925 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:14.925 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:14.925 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:14.925 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:14.925 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:14.925 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:14.925 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:14.925 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:14.925 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:14.925 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:14.926 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:14.926 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:14.926 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:14.926 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:14.926 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:14.926 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:14.926 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:14.926 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:14.926 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:14.926 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:14.926 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:14.926 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:14.926 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:14.926 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:14.927 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:14.927 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:14.927 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:14.927 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:14.927 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:14.927 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:14.927 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:14.927 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:14.927 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:14.927 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:14.927 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:14.927 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:14.927 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:14.927 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:14.927 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.928 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:14.928 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:14.928 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:14.928 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T16:16:14.928 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T16:16:14.928 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T16:16:14.928 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:14.928 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 3, 2024-04-07T16:16:14.928 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:14.928 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.928 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:14.928 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:14.928 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T16:16:14.928 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T16:16:14.929 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T16:16:14.929 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:14.929 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 5, 2024-04-07T16:16:14.929 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:14.929 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:14.929 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T16:16:14.929 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:14.929 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:14.929 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:14.929 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.024099+0000", 2024-04-07T16:16:14.929 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.044808+0000", 2024-04-07T16:16:14.929 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.044808+0000", 2024-04-07T16:16:14.929 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.045443+0000", 2024-04-07T16:16:14.929 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.024099+0000", 2024-04-07T16:16:14.929 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:14.930 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:14.930 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:14.930 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 60, 2024-04-07T16:16:14.930 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.024099+0000", 2024-04-07T16:16:14.930 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.024099+0000", 2024-04-07T16:16:14.930 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.024099+0000", 2024-04-07T16:16:14.930 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:14.930 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:14.930 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:14.930 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.024099+0000", 2024-04-07T16:16:14.930 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.024099+0000", 2024-04-07T16:16:14.930 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:14.930 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:14.931 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:14.931 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:14.931 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 59, 2024-04-07T16:16:14.931 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:14.931 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:14.931 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:14.931 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:14.931 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:14.931 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:14.931 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:14.931 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:14.931 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.18", 2024-04-07T16:16:14.931 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:14.932 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:14.932 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:14.932 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 20, 2024-04-07T16:16:14.932 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:14.932 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T22:44:14.623116+0000", 2024-04-07T16:16:14.932 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:14.932 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:14.932 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:14.932 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:14.932 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:14.932 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:14.932 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:14.932 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:14.932 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:14.932 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:14.933 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:14.933 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:14.933 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:14.933 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:14.933 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:14.933 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:14.933 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:14.933 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:14.933 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:14.933 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:14.933 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:14.933 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:14.933 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:14.933 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:14.933 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:14.934 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:14.934 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:14.934 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:14.934 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:14.934 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:14.934 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:14.934 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:14.934 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:14.934 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:14.934 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:14.934 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:14.934 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:14.934 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:14.934 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:14.934 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:14.935 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:14.935 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:14.935 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:14.935 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.935 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:14.935 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:14.935 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:14.935 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T16:16:14.935 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T16:16:14.935 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T16:16:14.935 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:14.935 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 5, 2024-04-07T16:16:14.935 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:14.935 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.935 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:14.936 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:14.936 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T16:16:14.936 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T16:16:14.936 INFO:tasks.workunit.client.0.smithi100.stdout: 4 2024-04-07T16:16:14.936 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:14.936 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 1, 2024-04-07T16:16:14.936 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:14.936 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:14.936 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T16:16:14.936 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:14.936 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:14.936 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:14.936 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.236151+0000", 2024-04-07T16:16:14.936 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.390643+0000", 2024-04-07T16:16:14.937 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.390643+0000", 2024-04-07T16:16:14.937 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.391166+0000", 2024-04-07T16:16:14.937 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.236151+0000", 2024-04-07T16:16:14.937 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:14.937 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:14.937 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:14.937 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 61, 2024-04-07T16:16:14.937 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.236151+0000", 2024-04-07T16:16:14.937 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.236151+0000", 2024-04-07T16:16:14.937 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.236151+0000", 2024-04-07T16:16:14.937 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:14.937 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:14.937 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:14.937 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.236151+0000", 2024-04-07T16:16:14.937 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.236151+0000", 2024-04-07T16:16:14.938 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:14.938 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:14.938 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:14.938 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:14.938 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 60, 2024-04-07T16:16:14.938 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:14.938 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:14.938 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:14.938 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:14.938 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:14.938 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:14.938 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:14.938 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:14.938 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.19", 2024-04-07T16:16:14.939 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:14.939 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:14.939 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:14.939 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 19, 2024-04-07T16:16:14.939 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:14.939 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T21:22:33.110408+0000", 2024-04-07T16:16:14.939 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:14.939 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:14.939 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:14.939 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:14.939 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:14.939 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:14.939 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:14.939 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:14.940 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:14.940 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:14.940 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:14.940 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:14.940 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:14.940 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:14.940 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:14.940 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:14.940 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:14.940 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:14.940 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:14.940 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:14.940 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:14.940 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:14.940 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:14.941 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:14.941 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:14.941 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:14.941 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:14.941 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:14.941 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:14.941 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:14.941 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:14.941 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:14.941 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:14.941 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:14.941 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:14.941 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:14.941 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:14.941 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:14.942 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:14.942 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:14.942 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:14.942 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:14.942 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:14.942 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.942 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:14.942 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:14.942 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:14.942 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T16:16:14.942 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T16:16:14.942 INFO:tasks.workunit.client.0.smithi100.stdout: 4 2024-04-07T16:16:14.942 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:14.942 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 1, 2024-04-07T16:16:14.942 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:14.943 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.943 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:14.943 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:14.943 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T16:16:14.943 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T16:16:14.943 INFO:tasks.workunit.client.0.smithi100.stdout: 2 2024-04-07T16:16:14.943 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:14.943 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 6, 2024-04-07T16:16:14.943 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:14.943 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:14.943 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T16:16:14.943 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:14.943 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:14.943 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:14.943 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.553074+0000", 2024-04-07T16:16:14.944 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.029525+0000", 2024-04-07T16:16:14.944 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.029525+0000", 2024-04-07T16:16:14.944 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.030149+0000", 2024-04-07T16:16:14.944 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.553074+0000", 2024-04-07T16:16:14.944 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:14.944 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:14.944 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:14.944 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 60, 2024-04-07T16:16:14.944 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.553074+0000", 2024-04-07T16:16:14.944 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.553074+0000", 2024-04-07T16:16:14.944 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.553074+0000", 2024-04-07T16:16:14.944 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:14.944 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:14.944 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:14.945 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.553074+0000", 2024-04-07T16:16:14.945 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.553074+0000", 2024-04-07T16:16:14.945 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:14.945 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:14.945 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:14.945 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:14.945 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 59, 2024-04-07T16:16:14.945 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:14.945 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:14.945 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:14.945 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:14.945 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:14.945 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:14.945 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:14.946 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:14.946 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.17", 2024-04-07T16:16:14.946 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:14.946 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:14.946 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:14.946 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 20, 2024-04-07T16:16:14.946 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:14.946 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T20:09:35.180264+0000", 2024-04-07T16:16:14.946 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:14.946 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:14.946 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:14.946 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:14.946 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:14.946 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:14.947 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:14.947 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:14.947 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:14.947 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:14.947 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:14.947 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:14.947 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:14.947 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:14.947 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:14.947 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:14.947 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:14.947 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:14.947 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:14.947 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:14.948 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:14.948 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:14.948 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:14.948 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:14.948 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:14.948 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:14.948 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:14.948 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:14.948 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:14.948 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:14.948 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:14.948 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:14.948 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:14.948 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:14.948 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:14.949 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:14.949 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:14.949 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:14.949 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:14.949 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:14.949 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:14.949 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:14.949 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:14.949 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.949 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:14.949 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:14.949 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:14.949 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T16:16:14.949 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T16:16:14.949 INFO:tasks.workunit.client.0.smithi100.stdout: 2 2024-04-07T16:16:14.950 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:14.950 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 6, 2024-04-07T16:16:14.950 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:14.950 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.950 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:14.950 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:14.950 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T16:16:14.950 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T16:16:14.950 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T16:16:14.950 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:14.950 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 5, 2024-04-07T16:16:14.950 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:14.950 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:14.950 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T16:16:14.950 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:14.951 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:14.951 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:14.951 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.025897+0000", 2024-04-07T16:16:14.951 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.398065+0000", 2024-04-07T16:16:14.951 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.398065+0000", 2024-04-07T16:16:14.951 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.398411+0000", 2024-04-07T16:16:14.951 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.025897+0000", 2024-04-07T16:16:14.951 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:14.951 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:14.951 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:14.951 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 61, 2024-04-07T16:16:14.951 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.025897+0000", 2024-04-07T16:16:14.951 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.025897+0000", 2024-04-07T16:16:14.952 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.025897+0000", 2024-04-07T16:16:14.952 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:14.952 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:14.952 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:14.952 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.025897+0000", 2024-04-07T16:16:14.952 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.025897+0000", 2024-04-07T16:16:14.952 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:14.952 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:14.952 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:14.952 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:14.952 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 60, 2024-04-07T16:16:14.952 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:14.952 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:14.953 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:14.953 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:14.953 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:14.953 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:14.953 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:14.953 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:14.953 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.16", 2024-04-07T16:16:14.953 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:14.953 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:14.953 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:14.953 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 19, 2024-04-07T16:16:14.953 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:14.953 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T21:13:20.448127+0000", 2024-04-07T16:16:14.953 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:14.953 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:14.954 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:14.954 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:14.954 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:14.954 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:14.954 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:14.954 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:14.954 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:14.954 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:14.954 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:14.954 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:14.954 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:14.954 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:14.954 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:14.954 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:14.955 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:14.955 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:14.955 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:14.955 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:14.955 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:14.955 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:14.955 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:14.955 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:14.955 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:14.955 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:14.955 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:14.955 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:14.955 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:14.955 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:14.956 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:14.956 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:14.956 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:14.956 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:14.956 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:14.956 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:14.956 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:14.956 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:14.956 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:14.956 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:14.956 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:14.956 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:14.956 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:14.956 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.956 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:14.957 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:14.957 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:14.957 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T16:16:14.957 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T16:16:14.957 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T16:16:14.957 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:14.957 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 5, 2024-04-07T16:16:14.957 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:14.957 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.957 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:14.957 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:14.957 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T16:16:14.957 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T16:16:14.957 INFO:tasks.workunit.client.0.smithi100.stdout: 2 2024-04-07T16:16:14.958 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:14.958 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 5, 2024-04-07T16:16:14.958 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:14.958 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:14.958 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T16:16:14.958 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:14.958 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:14.958 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:14.958 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.025656+0000", 2024-04-07T16:16:14.958 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.036322+0000", 2024-04-07T16:16:14.958 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.036322+0000", 2024-04-07T16:16:14.958 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.036777+0000", 2024-04-07T16:16:14.958 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.025656+0000", 2024-04-07T16:16:14.958 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:14.958 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:14.959 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:14.959 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 60, 2024-04-07T16:16:14.959 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.025656+0000", 2024-04-07T16:16:14.959 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.025656+0000", 2024-04-07T16:16:14.959 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.025656+0000", 2024-04-07T16:16:14.959 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:14.959 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:14.959 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:14.959 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.025656+0000", 2024-04-07T16:16:14.959 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.025656+0000", 2024-04-07T16:16:14.959 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:14.959 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:14.959 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:14.959 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:14.959 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 59, 2024-04-07T16:16:14.959 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:14.960 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:14.960 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:14.960 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:14.960 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:14.960 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:14.960 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:14.960 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:14.960 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.16", 2024-04-07T16:16:14.960 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:14.960 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:14.960 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:14.960 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 20, 2024-04-07T16:16:14.960 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:14.960 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T18:26:16.873044+0000", 2024-04-07T16:16:14.961 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:14.961 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:14.961 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:14.961 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:14.961 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:14.961 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:14.961 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:14.961 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:14.961 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:14.961 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:14.961 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:14.961 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:14.961 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:14.961 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:14.962 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:14.962 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:14.962 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:14.962 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:14.962 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:14.962 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:14.962 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:14.962 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:14.962 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:14.962 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:14.962 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:14.962 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:14.962 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:14.962 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:14.963 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:14.963 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:14.963 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:14.963 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:14.963 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:14.963 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:14.963 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:14.963 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:14.963 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:14.963 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:14.963 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:14.963 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:14.963 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:14.963 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:14.963 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:14.964 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.964 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:14.964 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:14.964 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:14.964 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T16:16:14.964 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T16:16:14.964 INFO:tasks.workunit.client.0.smithi100.stdout: 2 2024-04-07T16:16:14.964 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:14.964 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 5, 2024-04-07T16:16:14.964 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:14.964 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.964 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:14.964 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:14.964 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T16:16:14.965 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T16:16:14.965 INFO:tasks.workunit.client.0.smithi100.stdout: 3 2024-04-07T16:16:14.965 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:14.965 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 2, 2024-04-07T16:16:14.965 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:14.965 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:14.965 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T16:16:14.965 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:14.965 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:14.965 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:14.965 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.763837+0000", 2024-04-07T16:16:14.965 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:26.225343+0000", 2024-04-07T16:16:14.965 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:26.225343+0000", 2024-04-07T16:16:14.965 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:26.226313+0000", 2024-04-07T16:16:14.965 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.763837+0000", 2024-04-07T16:16:14.966 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:14.966 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:14.966 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:14.966 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 64, 2024-04-07T16:16:14.966 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.763837+0000", 2024-04-07T16:16:14.966 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.763837+0000", 2024-04-07T16:16:14.966 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.763837+0000", 2024-04-07T16:16:14.966 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:14.966 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:14.966 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:14.966 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.763837+0000", 2024-04-07T16:16:14.966 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.763837+0000", 2024-04-07T16:16:14.966 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:14.966 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:14.967 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:14.967 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:14.967 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 63, 2024-04-07T16:16:14.967 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:14.967 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:14.967 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:14.967 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:14.967 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:14.967 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:14.967 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:14.967 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:14.967 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.17", 2024-04-07T16:16:14.967 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:14.967 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:14.968 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:14.968 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 17, 2024-04-07T16:16:14.968 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:14.968 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T17:42:38.440812+0000", 2024-04-07T16:16:14.968 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:14.968 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:14.968 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:14.968 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:14.968 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:14.968 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:14.968 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:14.968 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:14.968 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:14.968 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:14.968 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:14.969 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:14.969 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:14.969 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:14.969 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:14.969 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:14.969 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:14.969 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:14.969 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:14.969 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:14.969 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:14.969 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:14.969 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:14.969 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:14.969 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:14.970 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:14.970 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:14.970 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:14.970 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:14.970 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:14.970 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:14.970 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:14.970 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:14.970 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:14.970 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:14.970 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:14.970 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:14.970 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:14.971 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:14.971 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:14.971 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:14.971 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:14.971 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:14.971 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.971 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:14.971 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:14.971 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:14.971 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T16:16:14.971 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T16:16:14.971 INFO:tasks.workunit.client.0.smithi100.stdout: 3 2024-04-07T16:16:14.971 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:14.971 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 2, 2024-04-07T16:16:14.972 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:14.972 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.972 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:14.972 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:14.972 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T16:16:14.972 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T16:16:14.972 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T16:16:14.972 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:14.972 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 1, 2024-04-07T16:16:14.972 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:14.972 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:14.972 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T16:16:14.972 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:14.972 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:14.972 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:14.973 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.237334+0000", 2024-04-07T16:16:14.973 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.038911+0000", 2024-04-07T16:16:14.973 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.038911+0000", 2024-04-07T16:16:14.973 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.039273+0000", 2024-04-07T16:16:14.973 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.237334+0000", 2024-04-07T16:16:14.973 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:14.973 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:14.973 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:14.973 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 60, 2024-04-07T16:16:14.973 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.237334+0000", 2024-04-07T16:16:14.973 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.237334+0000", 2024-04-07T16:16:14.973 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.237334+0000", 2024-04-07T16:16:14.973 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:14.973 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:14.973 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:14.974 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.237334+0000", 2024-04-07T16:16:14.974 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.237334+0000", 2024-04-07T16:16:14.974 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:14.974 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 3, 2024-04-07T16:16:14.974 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:14.974 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:14.974 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 59, 2024-04-07T16:16:14.974 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:14.974 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:14.974 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:14.974 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:14.974 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 3, 2024-04-07T16:16:14.974 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:14.975 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:14.975 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:14.975 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.15", 2024-04-07T16:16:14.975 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:14.975 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:14.975 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:14.975 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 23, 2024-04-07T16:16:14.975 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:14.975 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T02:11:53.262707+0000", 2024-04-07T16:16:14.975 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:14.975 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:14.975 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:14.975 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 90, 2024-04-07T16:16:14.976 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:14.976 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:14.976 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:14.976 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:14.976 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:14.976 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:14.976 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:14.976 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:14.976 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:14.976 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:14.976 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:14.976 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:14.976 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:14.977 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:14.977 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:14.977 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 6, 2024-04-07T16:16:14.977 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 2, 2024-04-07T16:16:14.977 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:14.977 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 2, 2024-04-07T16:16:14.977 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:14.977 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:14.977 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:14.977 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:14.977 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:14.977 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 1, 2024-04-07T16:16:14.977 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:14.977 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:14.978 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:14.978 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:14.978 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:14.978 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:14.978 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:14.978 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:14.978 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:14.978 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:14.978 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:14.978 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:14.978 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 4, 2024-04-07T16:16:14.978 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 1 2024-04-07T16:16:14.978 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.978 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:14.979 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:14.979 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:14.979 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T16:16:14.979 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T16:16:14.979 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T16:16:14.979 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:14.979 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 1, 2024-04-07T16:16:14.979 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "61'3" 2024-04-07T16:16:14.979 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.979 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:14.979 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:14.979 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T16:16:14.979 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T16:16:14.979 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T16:16:14.979 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:14.980 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 4, 2024-04-07T16:16:14.980 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:14.980 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:14.980 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T16:16:14.980 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:14.980 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:14.980 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:14.980 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:39.824449+0000", 2024-04-07T16:16:14.980 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.390960+0000", 2024-04-07T16:16:14.980 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.390960+0000", 2024-04-07T16:16:14.980 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.391349+0000", 2024-04-07T16:16:14.980 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:39.824449+0000", 2024-04-07T16:16:14.980 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:14.980 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:14.981 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:14.981 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 61, 2024-04-07T16:16:14.981 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:39.824449+0000", 2024-04-07T16:16:14.981 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:39.824449+0000", 2024-04-07T16:16:14.981 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:39.824449+0000", 2024-04-07T16:16:14.981 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:14.981 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:14.981 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:14.981 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:39.824449+0000", 2024-04-07T16:16:14.981 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:39.824449+0000", 2024-04-07T16:16:14.981 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:14.981 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:14.981 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:14.982 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:14.982 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 60, 2024-04-07T16:16:14.982 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:14.982 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:14.982 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:14.982 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:14.982 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:14.982 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:14.982 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:14.982 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:14.982 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.14", 2024-04-07T16:16:14.982 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:14.983 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:14.983 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:14.983 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 19, 2024-04-07T16:16:14.983 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:14.983 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T16:55:37.429799+0000", 2024-04-07T16:16:14.983 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:14.983 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:14.983 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:14.983 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:14.983 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:14.983 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:14.983 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:14.983 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:14.983 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:14.984 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:14.984 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:14.984 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:14.984 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:14.984 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:14.984 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:14.984 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:14.984 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:14.984 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:14.984 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:14.984 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:14.984 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:14.984 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:14.984 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:14.984 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:14.985 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:14.985 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:14.985 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:14.985 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:14.985 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:14.985 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:14.985 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:14.985 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:14.985 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:14.985 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:14.985 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:14.985 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:14.985 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:14.985 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:14.985 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:14.986 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:14.986 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:14.986 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:14.986 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:14.986 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.986 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:14.986 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:14.986 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:14.986 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T16:16:14.986 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T16:16:14.986 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T16:16:14.986 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:14.986 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 4, 2024-04-07T16:16:14.986 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:14.987 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.987 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:14.987 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:14.987 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T16:16:14.987 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T16:16:14.987 INFO:tasks.workunit.client.0.smithi100.stdout: 5 2024-04-07T16:16:14.987 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:14.987 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 6, 2024-04-07T16:16:14.987 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:14.987 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:14.987 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T16:16:14.987 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:14.987 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:14.987 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:14.987 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:53.034681+0000", 2024-04-07T16:16:14.988 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.031092+0000", 2024-04-07T16:16:14.988 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.031092+0000", 2024-04-07T16:16:14.988 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.031673+0000", 2024-04-07T16:16:14.988 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:53.034681+0000", 2024-04-07T16:16:14.988 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:14.988 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:14.988 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:14.988 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 60, 2024-04-07T16:16:14.988 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:53.034681+0000", 2024-04-07T16:16:14.988 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:53.034681+0000", 2024-04-07T16:16:14.988 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:53.034681+0000", 2024-04-07T16:16:14.988 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:14.988 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:14.988 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:14.989 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:53.034681+0000", 2024-04-07T16:16:14.989 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:53.034681+0000", 2024-04-07T16:16:14.989 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:14.989 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 12, 2024-04-07T16:16:14.989 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:14.989 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:14.989 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 59, 2024-04-07T16:16:14.989 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:14.989 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:14.989 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:14.989 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:14.989 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 12, 2024-04-07T16:16:14.989 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:14.989 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:14.989 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:14.990 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.14", 2024-04-07T16:16:14.990 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:14.990 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:14.990 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:14.990 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 32, 2024-04-07T16:16:14.990 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:14.990 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T20:17:48.593862+0000", 2024-04-07T16:16:14.990 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:14.990 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:14.990 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:14.990 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 2632, 2024-04-07T16:16:14.990 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:14.990 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:14.990 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:14.990 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:14.991 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:14.991 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:14.991 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:14.991 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:14.991 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:14.991 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:14.991 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:14.991 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:14.991 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:14.991 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:14.991 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:14.991 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 12, 2024-04-07T16:16:14.991 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 4, 2024-04-07T16:16:14.991 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:14.991 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 4, 2024-04-07T16:16:14.992 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:14.992 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:14.992 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:14.992 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:14.992 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:14.992 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 1, 2024-04-07T16:16:14.992 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:14.992 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:14.992 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:14.992 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:14.992 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:14.992 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:14.992 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:14.992 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:14.992 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:14.993 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:14.993 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:14.993 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:14.993 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 12, 2024-04-07T16:16:14.993 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 9 2024-04-07T16:16:14.993 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.993 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:14.993 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:14.993 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:14.993 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T16:16:14.993 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T16:16:14.993 INFO:tasks.workunit.client.0.smithi100.stdout: 5 2024-04-07T16:16:14.993 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:14.993 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 6, 2024-04-07T16:16:14.993 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "69'12" 2024-04-07T16:16:14.994 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:14.994 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:14.994 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:14.994 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T16:16:14.994 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T16:16:14.994 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T16:16:14.994 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:14.994 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 7, 2024-04-07T16:16:14.994 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:14.994 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:14.994 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T16:16:14.994 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:14.994 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:14.994 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:14.995 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.635967+0000", 2024-04-07T16:16:14.995 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:26.222663+0000", 2024-04-07T16:16:14.995 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:26.222663+0000", 2024-04-07T16:16:14.995 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:26.223406+0000", 2024-04-07T16:16:14.995 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.635967+0000", 2024-04-07T16:16:14.995 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:14.995 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:14.995 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:14.995 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 64, 2024-04-07T16:16:14.995 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.635967+0000", 2024-04-07T16:16:14.995 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.635967+0000", 2024-04-07T16:16:14.995 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.635967+0000", 2024-04-07T16:16:14.995 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:14.995 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:14.995 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:14.996 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.635967+0000", 2024-04-07T16:16:14.996 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.635967+0000", 2024-04-07T16:16:14.996 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:14.996 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:14.996 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:14.996 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:14.996 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 63, 2024-04-07T16:16:14.996 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:14.996 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:14.996 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:14.996 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:14.996 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:14.996 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:14.997 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:14.997 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:14.997 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.15", 2024-04-07T16:16:14.997 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:14.997 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:14.997 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:14.997 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 28, 2024-04-07T16:16:14.997 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:14.997 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T19:54:33.089377+0000", 2024-04-07T16:16:14.997 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:14.997 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:14.997 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:14.997 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:14.998 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:14.998 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:14.998 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:14.998 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:14.998 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:14.998 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:14.998 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:14.998 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:14.998 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:14.998 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:14.998 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:14.998 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:14.998 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:14.998 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:14.999 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:14.999 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:14.999 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:14.999 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:14.999 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:14.999 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:14.999 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:14.999 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:14.999 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:14.999 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:14.999 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:14.999 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:14.999 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:14.999 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:14.999 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:15.000 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:15.000 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:15.000 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:15.000 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:15.000 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:15.000 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:15.000 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:15.000 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:15.000 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:15.000 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:15.000 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.000 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:15.000 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:15.000 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:15.001 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T16:16:15.001 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T16:16:15.001 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T16:16:15.001 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:15.001 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 7, 2024-04-07T16:16:15.001 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:15.001 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.001 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:15.001 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:15.001 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T16:16:15.001 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T16:16:15.001 INFO:tasks.workunit.client.0.smithi100.stdout: 3 2024-04-07T16:16:15.001 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:15.001 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 0, 2024-04-07T16:16:15.001 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:15.002 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:15.002 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T16:16:15.002 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:15.002 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:15.002 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:15.002 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.029978+0000", 2024-04-07T16:16:15.002 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.032811+0000", 2024-04-07T16:16:15.002 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.032811+0000", 2024-04-07T16:16:15.002 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.033352+0000", 2024-04-07T16:16:15.002 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.029978+0000", 2024-04-07T16:16:15.002 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:15.002 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:15.002 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:15.003 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 60, 2024-04-07T16:16:15.003 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.029978+0000", 2024-04-07T16:16:15.003 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.029978+0000", 2024-04-07T16:16:15.003 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.029978+0000", 2024-04-07T16:16:15.003 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:15.003 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:15.003 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:15.003 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.029978+0000", 2024-04-07T16:16:15.003 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.029978+0000", 2024-04-07T16:16:15.003 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:15.003 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 2, 2024-04-07T16:16:15.003 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:15.003 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:15.003 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 59, 2024-04-07T16:16:15.003 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:15.004 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:15.004 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:15.004 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:15.004 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 2, 2024-04-07T16:16:15.004 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:15.004 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:15.004 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:15.004 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.13", 2024-04-07T16:16:15.004 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:15.004 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:15.004 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:15.004 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 22, 2024-04-07T16:16:15.004 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:15.004 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T01:50:02.195507+0000", 2024-04-07T16:16:15.004 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:15.004 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:15.005 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:15.005 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 34, 2024-04-07T16:16:15.005 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:15.005 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:15.005 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:15.005 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:15.005 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:15.005 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:15.005 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:15.005 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:15.005 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:15.005 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:15.005 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:15.005 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:15.006 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:15.006 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:15.006 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:15.006 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 6, 2024-04-07T16:16:15.006 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 2, 2024-04-07T16:16:15.006 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:15.006 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 2, 2024-04-07T16:16:15.006 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:15.006 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:15.006 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:15.006 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:15.006 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:15.006 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 1, 2024-04-07T16:16:15.006 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:15.006 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:15.007 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:15.007 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:15.007 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:15.007 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:15.007 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:15.007 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:15.007 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:15.007 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:15.007 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:15.007 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:15.007 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 3, 2024-04-07T16:16:15.007 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 7 2024-04-07T16:16:15.007 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.007 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:15.007 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:15.007 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:15.008 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T16:16:15.008 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T16:16:15.008 INFO:tasks.workunit.client.0.smithi100.stdout: 3 2024-04-07T16:16:15.008 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:15.008 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 0, 2024-04-07T16:16:15.008 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "61'2" 2024-04-07T16:16:15.008 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.008 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:15.008 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:15.008 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T16:16:15.008 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T16:16:15.008 INFO:tasks.workunit.client.0.smithi100.stdout: 3 2024-04-07T16:16:15.008 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:15.008 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 2, 2024-04-07T16:16:15.008 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:15.008 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:15.009 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T16:16:15.009 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:15.009 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:15.009 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:15.009 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.763887+0000", 2024-04-07T16:16:15.009 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:26.224225+0000", 2024-04-07T16:16:15.009 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:26.224225+0000", 2024-04-07T16:16:15.009 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:26.226163+0000", 2024-04-07T16:16:15.009 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.763887+0000", 2024-04-07T16:16:15.009 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:15.009 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:15.009 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:15.009 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 64, 2024-04-07T16:16:15.009 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.763887+0000", 2024-04-07T16:16:15.010 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.763887+0000", 2024-04-07T16:16:15.010 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.763887+0000", 2024-04-07T16:16:15.010 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:15.010 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:15.010 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:15.010 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.763887+0000", 2024-04-07T16:16:15.010 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.763887+0000", 2024-04-07T16:16:15.010 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:15.010 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:15.010 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:15.010 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:15.010 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 63, 2024-04-07T16:16:15.010 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:15.010 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:15.010 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:15.011 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:15.011 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:15.011 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:15.011 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:15.011 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:15.011 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.12", 2024-04-07T16:16:15.011 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:15.011 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:15.011 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:15.011 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 17, 2024-04-07T16:16:15.011 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:15.011 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T23:54:31.223096+0000", 2024-04-07T16:16:15.011 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:15.011 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:15.011 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:15.012 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:15.012 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:15.012 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:15.012 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:15.012 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:15.012 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:15.012 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:15.012 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:15.012 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:15.012 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:15.012 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:15.012 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:15.012 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:15.012 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:15.012 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:15.012 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:15.013 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:15.013 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:15.013 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:15.013 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:15.013 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:15.013 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:15.013 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:15.013 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:15.013 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:15.013 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:15.013 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:15.013 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:15.013 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:15.013 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:15.013 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:15.013 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:15.014 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:15.014 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:15.014 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:15.014 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:15.014 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:15.014 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:15.014 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:15.014 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:15.014 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.014 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:15.014 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:15.014 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:15.014 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T16:16:15.014 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T16:16:15.014 INFO:tasks.workunit.client.0.smithi100.stdout: 3 2024-04-07T16:16:15.015 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:15.015 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 2, 2024-04-07T16:16:15.015 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:15.015 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.015 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:15.015 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:15.015 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T16:16:15.015 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T16:16:15.015 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T16:16:15.015 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:15.015 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 5, 2024-04-07T16:16:15.015 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:15.015 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:15.015 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T16:16:15.015 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:15.016 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:15.016 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:15.016 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.024421+0000", 2024-04-07T16:16:15.016 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.045086+0000", 2024-04-07T16:16:15.016 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.045086+0000", 2024-04-07T16:16:15.016 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.045494+0000", 2024-04-07T16:16:15.016 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.024421+0000", 2024-04-07T16:16:15.016 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:15.016 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:15.016 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:15.016 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 60, 2024-04-07T16:16:15.016 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.024421+0000", 2024-04-07T16:16:15.016 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.024421+0000", 2024-04-07T16:16:15.016 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.024421+0000", 2024-04-07T16:16:15.017 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:15.017 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:15.017 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:15.017 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.024421+0000", 2024-04-07T16:16:15.017 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.024421+0000", 2024-04-07T16:16:15.017 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:15.017 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:15.017 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:15.017 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:15.017 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 59, 2024-04-07T16:16:15.017 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:15.017 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:15.017 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:15.017 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:15.017 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:15.018 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:15.018 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:15.018 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:15.018 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.12", 2024-04-07T16:16:15.018 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:15.018 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:15.018 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:15.018 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 20, 2024-04-07T16:16:15.018 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:15.018 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T23:15:11.494437+0000", 2024-04-07T16:16:15.018 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:15.018 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:15.018 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:15.018 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:15.019 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:15.019 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:15.019 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:15.019 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:15.019 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:15.019 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:15.019 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:15.019 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:15.019 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:15.019 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:15.019 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:15.019 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:15.019 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:15.019 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:15.019 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:15.020 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:15.020 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:15.020 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:15.020 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:15.020 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:15.020 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:15.020 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:15.020 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:15.020 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:15.020 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:15.020 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:15.020 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:15.020 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:15.020 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:15.020 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:15.021 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:15.021 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:15.021 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:15.021 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:15.021 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:15.021 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:15.021 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:15.021 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:15.021 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:15.021 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.021 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:15.021 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:15.021 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:15.022 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T16:16:15.022 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T16:16:15.022 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T16:16:15.022 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:15.022 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 5, 2024-04-07T16:16:15.022 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:15.022 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.022 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:15.022 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:15.022 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T16:16:15.022 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T16:16:15.022 INFO:tasks.workunit.client.0.smithi100.stdout: 2 2024-04-07T16:16:15.022 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:15.022 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 7, 2024-04-07T16:16:15.023 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:15.023 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:15.023 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T16:16:15.023 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:15.023 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:15.023 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:15.023 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.637043+0000", 2024-04-07T16:16:15.023 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.413447+0000", 2024-04-07T16:16:15.023 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.413447+0000", 2024-04-07T16:16:15.023 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.413928+0000", 2024-04-07T16:16:15.023 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.637043+0000", 2024-04-07T16:16:15.023 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:15.023 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:15.023 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:15.024 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 61, 2024-04-07T16:16:15.024 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.637043+0000", 2024-04-07T16:16:15.024 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.637043+0000", 2024-04-07T16:16:15.024 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.637043+0000", 2024-04-07T16:16:15.024 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:15.024 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:15.024 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:15.024 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.637043+0000", 2024-04-07T16:16:15.024 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.637043+0000", 2024-04-07T16:16:15.024 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:15.024 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:15.024 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:15.024 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:15.024 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 60, 2024-04-07T16:16:15.025 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:15.025 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:15.025 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:15.025 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:15.025 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:15.025 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:15.025 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:15.025 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:15.025 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.13", 2024-04-07T16:16:15.025 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:15.025 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:15.025 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:15.026 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 19, 2024-04-07T16:16:15.026 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:15.026 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T03:26:42.875228+0000", 2024-04-07T16:16:15.026 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:15.026 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:15.026 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:15.026 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:15.026 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:15.026 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:15.026 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:15.026 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:15.026 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:15.026 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:15.026 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:15.027 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:15.027 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:15.027 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:15.027 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:15.027 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:15.027 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:15.027 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:15.027 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:15.027 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:15.027 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:15.027 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:15.027 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:15.027 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:15.027 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:15.028 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:15.028 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:15.028 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:15.028 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:15.028 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:15.028 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:15.028 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:15.028 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:15.028 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:15.028 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:15.028 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:15.028 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:15.028 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:15.028 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:15.028 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:15.029 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:15.029 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:15.029 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:15.029 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.029 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:15.029 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:15.029 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:15.029 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T16:16:15.029 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T16:16:15.029 INFO:tasks.workunit.client.0.smithi100.stdout: 2 2024-04-07T16:16:15.029 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:15.029 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 7, 2024-04-07T16:16:15.029 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:15.029 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.029 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:15.030 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:15.030 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T16:16:15.030 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T16:16:15.030 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T16:16:15.030 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:15.030 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 6, 2024-04-07T16:16:15.030 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:15.030 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:15.030 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T16:16:15.030 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:15.030 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:15.030 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:15.030 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.526216+0000", 2024-04-07T16:16:15.030 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.032108+0000", 2024-04-07T16:16:15.030 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.032108+0000", 2024-04-07T16:16:15.031 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.032942+0000", 2024-04-07T16:16:15.031 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.526216+0000", 2024-04-07T16:16:15.031 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:15.031 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:15.031 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:15.031 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 60, 2024-04-07T16:16:15.031 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.526216+0000", 2024-04-07T16:16:15.031 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.526216+0000", 2024-04-07T16:16:15.031 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.526216+0000", 2024-04-07T16:16:15.031 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:15.031 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:15.031 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:15.031 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.526216+0000", 2024-04-07T16:16:15.031 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.526216+0000", 2024-04-07T16:16:15.032 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:15.032 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:15.032 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:15.032 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:15.032 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 59, 2024-04-07T16:16:15.032 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:15.032 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:15.032 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:15.032 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:15.032 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:15.032 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:15.032 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:15.032 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:15.032 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.11", 2024-04-07T16:16:15.032 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:15.033 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:15.033 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:15.033 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 20, 2024-04-07T16:16:15.033 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:15.033 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T18:53:06.864412+0000", 2024-04-07T16:16:15.033 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:15.033 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:15.033 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:15.033 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:15.033 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:15.033 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:15.033 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:15.033 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:15.033 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:15.033 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:15.034 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:15.034 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:15.034 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:15.034 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:15.034 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:15.034 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:15.034 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:15.034 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:15.034 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:15.034 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:15.034 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:15.034 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:15.034 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:15.034 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:15.034 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:15.035 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:15.035 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:15.035 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:15.035 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:15.035 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:15.035 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:15.035 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:15.035 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:15.035 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:15.035 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:15.035 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:15.035 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:15.035 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:15.035 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:15.035 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:15.036 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:15.036 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:15.036 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:15.036 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.036 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:15.036 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:15.036 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:15.036 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T16:16:15.036 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T16:16:15.036 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T16:16:15.036 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:15.036 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 6, 2024-04-07T16:16:15.036 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:15.036 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.036 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:15.037 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:15.037 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T16:16:15.037 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T16:16:15.037 INFO:tasks.workunit.client.0.smithi100.stdout: 5 2024-04-07T16:16:15.037 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:15.037 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 6, 2024-04-07T16:16:15.037 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:15.037 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:15.037 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T16:16:15.037 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:15.037 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:15.037 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:15.037 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.525876+0000", 2024-04-07T16:16:15.037 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.399823+0000", 2024-04-07T16:16:15.037 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.399823+0000", 2024-04-07T16:16:15.038 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.400141+0000", 2024-04-07T16:16:15.038 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.525876+0000", 2024-04-07T16:16:15.038 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:15.038 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:15.038 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:15.038 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 61, 2024-04-07T16:16:15.038 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.525876+0000", 2024-04-07T16:16:15.038 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.525876+0000", 2024-04-07T16:16:15.038 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.525876+0000", 2024-04-07T16:16:15.038 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:15.038 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:15.038 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:15.038 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.525876+0000", 2024-04-07T16:16:15.038 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.525876+0000", 2024-04-07T16:16:15.038 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:15.038 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:15.039 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:15.039 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:15.039 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 60, 2024-04-07T16:16:15.039 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:15.039 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:15.039 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:15.039 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:15.039 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:15.039 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:15.039 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:15.039 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:15.039 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.10", 2024-04-07T16:16:15.039 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:15.039 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:15.040 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:15.040 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 19, 2024-04-07T16:16:15.040 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:15.040 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T21:27:25.248595+0000", 2024-04-07T16:16:15.040 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:15.040 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:15.040 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:15.040 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:15.040 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:15.040 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:15.040 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:15.040 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:15.040 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:15.041 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:15.041 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:15.041 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:15.041 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:15.041 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:15.041 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:15.041 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:15.041 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:15.041 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:15.041 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:15.041 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:15.041 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:15.041 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:15.041 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:15.041 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:15.041 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:15.042 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:15.042 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:15.042 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:15.042 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:15.042 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:15.042 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:15.042 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:15.042 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:15.042 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:15.042 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:15.042 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:15.042 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:15.042 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:15.042 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:15.043 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:15.043 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:15.043 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:15.043 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:15.043 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.043 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:15.043 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:15.043 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:15.043 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T16:16:15.043 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T16:16:15.043 INFO:tasks.workunit.client.0.smithi100.stdout: 5 2024-04-07T16:16:15.043 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:15.043 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 6, 2024-04-07T16:16:15.044 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:15.044 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.044 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:15.044 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:15.044 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T16:16:15.044 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T16:16:15.044 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T16:16:15.044 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:15.044 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 2, 2024-04-07T16:16:15.044 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:15.044 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:15.044 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T16:16:15.044 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:15.044 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:15.045 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:15.045 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.764170+0000", 2024-04-07T16:16:15.045 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.033838+0000", 2024-04-07T16:16:15.045 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.033838+0000", 2024-04-07T16:16:15.045 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.034154+0000", 2024-04-07T16:16:15.045 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.764170+0000", 2024-04-07T16:16:15.045 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:15.045 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:15.045 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:15.045 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 60, 2024-04-07T16:16:15.045 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.764170+0000", 2024-04-07T16:16:15.045 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.764170+0000", 2024-04-07T16:16:15.045 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.764170+0000", 2024-04-07T16:16:15.046 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:15.046 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:15.046 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:15.046 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.764170+0000", 2024-04-07T16:16:15.046 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.764170+0000", 2024-04-07T16:16:15.046 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:15.046 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 8, 2024-04-07T16:16:15.046 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:15.046 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:15.046 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 59, 2024-04-07T16:16:15.046 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:15.046 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:15.046 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:15.046 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:15.046 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 8, 2024-04-07T16:16:15.047 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:15.047 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:15.047 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:15.047 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.10", 2024-04-07T16:16:15.047 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:15.047 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:15.047 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:15.047 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 28, 2024-04-07T16:16:15.047 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:15.047 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T23:13:07.216314+0000", 2024-04-07T16:16:15.047 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:15.047 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:15.047 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:15.047 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:15.048 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:15.048 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:15.048 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:15.048 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:15.048 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:15.048 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:15.048 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:15.048 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:15.048 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:15.048 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:15.048 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:15.048 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:15.048 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:15.048 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:15.049 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:15.049 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 12, 2024-04-07T16:16:15.049 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 4, 2024-04-07T16:16:15.049 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:15.049 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 4, 2024-04-07T16:16:15.049 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:15.049 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:15.049 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:15.049 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:15.049 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:15.049 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 4, 2024-04-07T16:16:15.049 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:15.049 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:15.049 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:15.050 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:15.050 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:15.050 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:15.050 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:15.050 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:15.050 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:15.050 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:15.050 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:15.050 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:15.050 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 12, 2024-04-07T16:16:15.050 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:15.050 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.050 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:15.050 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:15.050 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:15.051 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T16:16:15.051 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T16:16:15.051 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T16:16:15.051 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:15.051 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 2, 2024-04-07T16:16:15.051 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "61'8" 2024-04-07T16:16:15.051 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.051 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:15.051 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:15.051 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T16:16:15.051 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T16:16:15.051 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T16:16:15.051 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:15.051 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 7, 2024-04-07T16:16:15.051 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:15.052 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:15.052 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T16:16:15.052 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:15.052 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:15.052 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:15.052 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.636593+0000", 2024-04-07T16:16:15.052 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.385485+0000", 2024-04-07T16:16:15.052 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.385485+0000", 2024-04-07T16:16:15.052 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.385883+0000", 2024-04-07T16:16:15.052 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.636593+0000", 2024-04-07T16:16:15.052 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:15.052 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:15.052 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:15.052 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 61, 2024-04-07T16:16:15.052 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.636593+0000", 2024-04-07T16:16:15.053 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.636593+0000", 2024-04-07T16:16:15.053 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.636593+0000", 2024-04-07T16:16:15.053 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:15.053 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:15.053 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:15.053 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.636593+0000", 2024-04-07T16:16:15.053 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.636593+0000", 2024-04-07T16:16:15.053 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:15.053 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:15.053 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:15.053 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:15.053 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 60, 2024-04-07T16:16:15.053 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:15.053 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:15.053 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:15.054 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:15.054 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:15.054 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:15.054 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:15.054 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:15.054 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.11", 2024-04-07T16:16:15.054 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:15.054 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:15.054 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:15.054 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 19, 2024-04-07T16:16:15.054 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:15.054 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T21:01:09.262433+0000", 2024-04-07T16:16:15.054 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:15.055 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:15.055 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:15.055 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:15.055 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:15.055 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:15.055 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:15.055 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:15.055 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:15.055 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:15.055 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:15.055 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:15.055 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:15.055 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:15.055 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:15.055 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:15.056 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:15.056 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:15.056 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:15.056 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:15.056 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:15.056 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:15.056 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:15.056 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:15.056 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:15.056 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:15.056 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:15.056 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:15.056 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:15.056 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:15.057 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:15.057 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:15.057 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:15.057 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:15.057 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:15.057 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:15.057 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:15.057 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:15.057 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:15.057 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:15.057 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:15.057 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:15.057 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:15.057 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.057 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:15.057 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:15.058 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:15.058 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T16:16:15.058 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T16:16:15.058 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T16:16:15.058 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:15.058 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 7, 2024-04-07T16:16:15.058 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:15.058 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.058 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:15.058 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:15.058 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T16:16:15.058 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T16:16:15.058 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T16:16:15.058 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:15.058 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 4, 2024-04-07T16:16:15.059 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:15.059 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:15.059 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T16:16:15.059 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:15.059 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:15.059 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:15.059 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:39.825314+0000", 2024-04-07T16:16:15.059 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.027445+0000", 2024-04-07T16:16:15.059 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.027445+0000", 2024-04-07T16:16:15.059 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.028004+0000", 2024-04-07T16:16:15.059 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:39.825314+0000", 2024-04-07T16:16:15.059 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:15.059 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:15.059 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:15.060 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 60, 2024-04-07T16:16:15.060 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:39.825314+0000", 2024-04-07T16:16:15.060 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:39.825314+0000", 2024-04-07T16:16:15.060 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:39.825314+0000", 2024-04-07T16:16:15.060 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:15.060 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:15.060 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:15.060 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:39.825314+0000", 2024-04-07T16:16:15.060 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:39.825314+0000", 2024-04-07T16:16:15.060 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:15.060 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 1, 2024-04-07T16:16:15.060 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:15.060 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:15.060 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 59, 2024-04-07T16:16:15.060 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:15.061 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:15.061 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:15.061 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:15.061 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 1, 2024-04-07T16:16:15.061 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:15.061 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:15.061 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:15.061 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.f", 2024-04-07T16:16:15.061 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:15.061 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:15.061 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:15.061 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 21, 2024-04-07T16:16:15.061 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:15.061 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T17:55:06.727142+0000", 2024-04-07T16:16:15.061 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:15.061 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:15.062 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:15.062 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 22, 2024-04-07T16:16:15.062 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:15.062 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:15.062 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:15.062 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:15.062 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:15.062 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:15.062 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:15.062 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:15.062 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:15.062 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:15.062 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:15.062 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:15.063 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:15.063 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:15.063 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:15.063 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 3, 2024-04-07T16:16:15.063 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 1, 2024-04-07T16:16:15.063 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:15.063 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 1, 2024-04-07T16:16:15.063 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:15.063 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:15.063 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:15.063 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:15.063 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:15.063 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:15.063 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:15.063 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:15.064 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:15.064 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:15.064 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:15.064 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:15.064 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:15.064 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:15.064 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:15.064 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:15.064 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:15.064 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:15.064 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 1, 2024-04-07T16:16:15.064 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 1 2024-04-07T16:16:15.064 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.064 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:15.064 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:15.065 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:15.065 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T16:16:15.065 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T16:16:15.065 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T16:16:15.065 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:15.065 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 4, 2024-04-07T16:16:15.065 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "61'1" 2024-04-07T16:16:15.065 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.065 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:15.065 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:15.065 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T16:16:15.065 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T16:16:15.065 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T16:16:15.065 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:15.065 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 7, 2024-04-07T16:16:15.066 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:15.066 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:15.066 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T16:16:15.066 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:15.066 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:15.066 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:15.066 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.637723+0000", 2024-04-07T16:16:15.066 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.386057+0000", 2024-04-07T16:16:15.066 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.386057+0000", 2024-04-07T16:16:15.066 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.386584+0000", 2024-04-07T16:16:15.066 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.637723+0000", 2024-04-07T16:16:15.066 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:15.066 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:15.066 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:15.067 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 61, 2024-04-07T16:16:15.067 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.637723+0000", 2024-04-07T16:16:15.067 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.637723+0000", 2024-04-07T16:16:15.067 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.637723+0000", 2024-04-07T16:16:15.067 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:15.067 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:15.067 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:15.067 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.637723+0000", 2024-04-07T16:16:15.067 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.637723+0000", 2024-04-07T16:16:15.067 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:15.067 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:15.067 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:15.067 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:15.067 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 60, 2024-04-07T16:16:15.067 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:15.067 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:15.068 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:15.068 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:15.068 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:15.068 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:15.068 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:15.068 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:15.068 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.e", 2024-04-07T16:16:15.068 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:15.068 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:15.068 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:15.068 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 19, 2024-04-07T16:16:15.068 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:15.068 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T17:57:57.548179+0000", 2024-04-07T16:16:15.069 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:15.069 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:15.069 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:15.069 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:15.069 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:15.069 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:15.069 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:15.069 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:15.069 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:15.069 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:15.069 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:15.069 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:15.069 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:15.070 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:15.070 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:15.070 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:15.070 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:15.070 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:15.070 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:15.070 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:15.070 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:15.070 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:15.070 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:15.070 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:15.070 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:15.070 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:15.071 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:15.071 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:15.071 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:15.071 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:15.071 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:15.071 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:15.071 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:15.071 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:15.071 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:15.071 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:15.071 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:15.071 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:15.071 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:15.071 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:15.071 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:15.072 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:15.072 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:15.072 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.072 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:15.072 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:15.072 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:15.072 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T16:16:15.072 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T16:16:15.072 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T16:16:15.072 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:15.072 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 7, 2024-04-07T16:16:15.072 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:15.072 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.072 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:15.073 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:15.073 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T16:16:15.073 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T16:16:15.073 INFO:tasks.workunit.client.0.smithi100.stdout: 4 2024-04-07T16:16:15.073 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:15.073 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 1, 2024-04-07T16:16:15.073 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:15.073 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:15.073 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 66, 2024-04-07T16:16:15.073 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:15.073 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:15.073 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:15.073 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.236643+0000", 2024-04-07T16:16:15.073 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:13:30.008706+0000", 2024-04-07T16:16:15.073 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:13:30.008706+0000", 2024-04-07T16:16:15.073 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:13:30.009561+0000", 2024-04-07T16:16:15.074 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.236643+0000", 2024-04-07T16:16:15.074 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:13:28.963534+0000", 2024-04-07T16:16:15.074 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:15.074 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:13:28.963534+0000", 2024-04-07T16:16:15.074 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 67, 2024-04-07T16:16:15.074 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.236643+0000", 2024-04-07T16:16:15.074 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.236643+0000", 2024-04-07T16:16:15.074 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.236643+0000", 2024-04-07T16:16:15.074 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:15.074 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:15.074 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:13:28.963534+0000", 2024-04-07T16:16:15.074 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.236643+0000", 2024-04-07T16:16:15.074 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.236643+0000", 2024-04-07T16:16:15.074 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:15.075 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:15.075 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:15.075 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:15.075 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 66, 2024-04-07T16:16:15.075 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:15.075 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:15.075 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:15.075 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:15.075 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:15.075 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:15.075 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:15.075 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:15.075 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "4.2", 2024-04-07T16:16:15.075 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:15.075 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:15.076 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:15.076 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 13, 2024-04-07T16:16:15.076 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:15.076 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T00:36:26.958742+0000", 2024-04-07T16:16:15.076 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:15.076 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:15.076 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:15.076 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:15.076 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:15.076 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:15.076 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:15.076 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:15.076 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:15.076 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:15.076 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:15.077 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:15.077 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:15.077 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:15.077 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:15.077 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:15.077 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:15.077 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:15.077 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:15.077 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:15.077 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:15.077 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:15.077 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:15.077 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:15.077 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:15.077 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:15.078 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:15.078 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:15.078 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:15.078 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:15.078 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:15.078 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:15.078 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:15.078 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:15.078 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:15.078 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:15.078 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:15.078 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:15.078 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:15.078 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:15.079 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:15.079 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:15.079 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:15.079 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.079 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:15.079 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:15.079 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:15.079 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T16:16:15.079 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T16:16:15.079 INFO:tasks.workunit.client.0.smithi100.stdout: 4 2024-04-07T16:16:15.079 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:15.079 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 1, 2024-04-07T16:16:15.079 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:15.079 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.079 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:15.080 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:15.080 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T16:16:15.080 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T16:16:15.080 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T16:16:15.080 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:15.080 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 1, 2024-04-07T16:16:15.080 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:15.080 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:15.080 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T16:16:15.080 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:15.080 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:15.080 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:15.080 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.236743+0000", 2024-04-07T16:16:15.080 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.037627+0000", 2024-04-07T16:16:15.080 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.037627+0000", 2024-04-07T16:16:15.081 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.037974+0000", 2024-04-07T16:16:15.081 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.236743+0000", 2024-04-07T16:16:15.081 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:15.081 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:15.081 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:15.081 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 60, 2024-04-07T16:16:15.081 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.236743+0000", 2024-04-07T16:16:15.081 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.236743+0000", 2024-04-07T16:16:15.081 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.236743+0000", 2024-04-07T16:16:15.081 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:15.081 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:15.081 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:15.081 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.236743+0000", 2024-04-07T16:16:15.081 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.236743+0000", 2024-04-07T16:16:15.081 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:15.082 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:15.082 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:15.082 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:15.082 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 59, 2024-04-07T16:16:15.082 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:15.082 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:15.082 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:15.082 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:15.082 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:15.082 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:15.082 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:15.082 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:15.082 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.4", 2024-04-07T16:16:15.082 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:15.082 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:15.083 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:15.083 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 20, 2024-04-07T16:16:15.083 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:15.083 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T19:28:12.174861+0000", 2024-04-07T16:16:15.083 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:15.083 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:15.083 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:15.083 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:15.083 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:15.083 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:15.083 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:15.083 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:15.083 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:15.083 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:15.084 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:15.084 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:15.084 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:15.084 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:15.084 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:15.084 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:15.084 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:15.084 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:15.084 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:15.084 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:15.084 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:15.084 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:15.084 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:15.084 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:15.085 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:15.085 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:15.085 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:15.085 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:15.085 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:15.085 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:15.085 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:15.085 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:15.085 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:15.085 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:15.085 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:15.085 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:15.085 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:15.085 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:15.085 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:15.086 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:15.086 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:15.086 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:15.086 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:15.086 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.086 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:15.086 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:15.086 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:15.086 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T16:16:15.086 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T16:16:15.086 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T16:16:15.086 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:15.086 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 1, 2024-04-07T16:16:15.087 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:15.087 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.087 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:15.087 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:15.087 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T16:16:15.087 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T16:16:15.087 INFO:tasks.workunit.client.0.smithi100.stdout: 2 2024-04-07T16:16:15.087 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:15.087 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 5, 2024-04-07T16:16:15.087 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:15.087 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:15.087 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T16:16:15.087 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:15.087 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:15.088 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:15.088 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.025595+0000", 2024-04-07T16:16:15.088 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.394113+0000", 2024-04-07T16:16:15.088 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.394113+0000", 2024-04-07T16:16:15.088 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.395073+0000", 2024-04-07T16:16:15.088 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.025595+0000", 2024-04-07T16:16:15.088 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:15.088 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:15.088 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:15.088 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 61, 2024-04-07T16:16:15.088 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.025595+0000", 2024-04-07T16:16:15.088 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.025595+0000", 2024-04-07T16:16:15.088 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.025595+0000", 2024-04-07T16:16:15.088 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:15.088 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:15.089 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:15.089 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.025595+0000", 2024-04-07T16:16:15.089 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.025595+0000", 2024-04-07T16:16:15.089 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:15.089 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:15.089 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:15.089 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:15.089 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 60, 2024-04-07T16:16:15.089 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:15.089 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:15.089 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:15.089 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:15.089 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:15.089 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:15.090 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:15.090 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:15.090 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.5", 2024-04-07T16:16:15.090 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:15.090 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:15.090 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:15.090 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 19, 2024-04-07T16:16:15.090 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:15.090 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T23:28:32.978580+0000", 2024-04-07T16:16:15.090 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:15.090 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:15.090 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:15.090 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:15.090 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:15.090 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:15.091 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:15.091 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:15.091 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:15.091 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:15.091 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:15.091 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:15.091 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:15.091 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:15.091 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:15.091 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:15.091 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:15.091 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:15.091 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:15.092 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:15.092 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:15.092 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:15.092 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:15.092 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:15.092 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:15.092 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:15.092 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:15.092 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:15.092 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:15.092 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:15.092 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:15.092 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:15.093 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:15.093 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:15.093 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:15.093 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:15.093 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:15.093 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:15.093 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:15.093 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:15.093 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:15.093 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:15.093 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:15.093 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.093 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:15.093 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:15.094 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:15.094 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T16:16:15.094 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T16:16:15.094 INFO:tasks.workunit.client.0.smithi100.stdout: 2 2024-04-07T16:16:15.094 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:15.094 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 5, 2024-04-07T16:16:15.094 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:15.094 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.094 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:15.094 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:15.094 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T16:16:15.094 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T16:16:15.094 INFO:tasks.workunit.client.0.smithi100.stdout: 3 2024-04-07T16:16:15.094 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:15.094 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 1, 2024-04-07T16:16:15.095 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:15.095 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:15.095 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 66, 2024-04-07T16:16:15.095 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:15.095 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:15.095 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:15.095 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.236861+0000", 2024-04-07T16:16:15.095 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:13:30.040318+0000", 2024-04-07T16:16:15.095 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:13:30.040318+0000", 2024-04-07T16:16:15.095 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:13:30.041147+0000", 2024-04-07T16:16:15.095 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.236861+0000", 2024-04-07T16:16:15.095 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:13:28.963534+0000", 2024-04-07T16:16:15.095 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:15.095 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:13:28.963534+0000", 2024-04-07T16:16:15.096 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 67, 2024-04-07T16:16:15.096 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.236861+0000", 2024-04-07T16:16:15.096 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.236861+0000", 2024-04-07T16:16:15.096 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.236861+0000", 2024-04-07T16:16:15.096 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:15.096 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:15.096 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:13:28.963534+0000", 2024-04-07T16:16:15.096 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.236861+0000", 2024-04-07T16:16:15.096 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.236861+0000", 2024-04-07T16:16:15.096 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:15.096 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:15.096 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:15.096 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:15.096 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 66, 2024-04-07T16:16:15.096 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:15.097 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:15.097 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:15.097 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:15.097 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:15.097 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:15.097 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:15.097 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:15.097 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "4.4", 2024-04-07T16:16:15.097 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:15.097 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:15.097 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:15.097 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 13, 2024-04-07T16:16:15.097 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:15.097 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T20:45:42.413572+0000", 2024-04-07T16:16:15.097 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:15.098 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:15.098 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:15.098 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:15.098 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:15.098 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:15.098 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:15.098 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:15.098 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:15.098 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:15.098 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:15.098 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:15.098 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:15.098 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:15.098 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:15.099 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:15.099 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:15.099 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:15.099 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:15.099 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:15.099 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:15.099 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:15.099 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:15.099 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:15.099 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:15.099 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:15.099 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:15.099 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:15.099 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:15.099 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:15.099 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:15.100 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:15.100 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:15.100 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:15.100 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:15.100 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:15.100 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:15.100 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:15.100 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:15.100 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:15.100 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:15.100 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:15.100 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:15.100 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.100 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:15.100 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:15.101 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:15.101 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T16:16:15.101 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T16:16:15.101 INFO:tasks.workunit.client.0.smithi100.stdout: 3 2024-04-07T16:16:15.101 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:15.101 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 1, 2024-04-07T16:16:15.101 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:15.101 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.101 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:15.101 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:15.101 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T16:16:15.101 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T16:16:15.101 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T16:16:15.102 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:15.102 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 5, 2024-04-07T16:16:15.102 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:15.102 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:15.102 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T16:16:15.102 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:15.102 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:15.102 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:15.102 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.024311+0000", 2024-04-07T16:16:15.102 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.035625+0000", 2024-04-07T16:16:15.102 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.035625+0000", 2024-04-07T16:16:15.102 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.036336+0000", 2024-04-07T16:16:15.102 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.024311+0000", 2024-04-07T16:16:15.102 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:15.103 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:15.103 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:15.103 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 60, 2024-04-07T16:16:15.103 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.024311+0000", 2024-04-07T16:16:15.103 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.024311+0000", 2024-04-07T16:16:15.103 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.024311+0000", 2024-04-07T16:16:15.103 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:15.103 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:15.103 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:15.103 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.024311+0000", 2024-04-07T16:16:15.103 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.024311+0000", 2024-04-07T16:16:15.103 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:15.103 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:15.103 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:15.103 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:15.104 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 59, 2024-04-07T16:16:15.104 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:15.104 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:15.104 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:15.104 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:15.104 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:15.104 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:15.104 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:15.104 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:15.104 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.2", 2024-04-07T16:16:15.104 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:15.104 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:15.104 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:15.104 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 20, 2024-04-07T16:16:15.104 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:15.105 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T18:15:00.506827+0000", 2024-04-07T16:16:15.105 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:15.105 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:15.105 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:15.105 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:15.105 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:15.105 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:15.105 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:15.105 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:15.105 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:15.105 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:15.105 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:15.105 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:15.105 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:15.106 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:15.106 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:15.106 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:15.106 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:15.106 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:15.106 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:15.106 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:15.106 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:15.106 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:15.106 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:15.106 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:15.106 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:15.106 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:15.106 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:15.106 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:15.107 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:15.107 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:15.107 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:15.107 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:15.107 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:15.107 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:15.107 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:15.107 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:15.107 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:15.107 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:15.107 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:15.107 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:15.107 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:15.107 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:15.107 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:15.108 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.108 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:15.108 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:15.108 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:15.108 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T16:16:15.108 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T16:16:15.108 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T16:16:15.108 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:15.108 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 5, 2024-04-07T16:16:15.108 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:15.108 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.108 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:15.108 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:15.108 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T16:16:15.108 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T16:16:15.108 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T16:16:15.109 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:15.109 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 4, 2024-04-07T16:16:15.109 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:15.109 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:15.109 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T16:16:15.109 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:15.109 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:15.109 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:15.109 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:39.824462+0000", 2024-04-07T16:16:15.109 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.404029+0000", 2024-04-07T16:16:15.109 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.404029+0000", 2024-04-07T16:16:15.109 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.404406+0000", 2024-04-07T16:16:15.109 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:39.824462+0000", 2024-04-07T16:16:15.109 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:15.109 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:15.110 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:15.110 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 61, 2024-04-07T16:16:15.110 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:39.824462+0000", 2024-04-07T16:16:15.110 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:39.824462+0000", 2024-04-07T16:16:15.110 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:39.824462+0000", 2024-04-07T16:16:15.110 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:15.110 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:15.110 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:15.110 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:39.824462+0000", 2024-04-07T16:16:15.110 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:39.824462+0000", 2024-04-07T16:16:15.110 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:15.110 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:15.110 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:15.110 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:15.110 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 60, 2024-04-07T16:16:15.111 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:15.111 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:15.111 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:15.111 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:15.111 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:15.111 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:15.111 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:15.111 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:15.111 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.3", 2024-04-07T16:16:15.111 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:15.111 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:15.111 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:15.111 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 19, 2024-04-07T16:16:15.111 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:15.111 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T16:41:25.469343+0000", 2024-04-07T16:16:15.112 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:15.112 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:15.112 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:15.112 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:15.112 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:15.112 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:15.112 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:15.112 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:15.112 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:15.112 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:15.112 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:15.112 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:15.112 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:15.112 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:15.112 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:15.113 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:15.113 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:15.113 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:15.113 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:15.113 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:15.113 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:15.113 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:15.113 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:15.113 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:15.113 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:15.113 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:15.113 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:15.113 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:15.113 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:15.114 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:15.114 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:15.114 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:15.114 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:15.114 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:15.114 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:15.114 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:15.114 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:15.114 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:15.114 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:15.114 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:15.114 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:15.114 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:15.114 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:15.115 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.115 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:15.115 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:15.115 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:15.115 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T16:16:15.115 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T16:16:15.115 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T16:16:15.115 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:15.115 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 4, 2024-04-07T16:16:15.115 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:15.115 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.115 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:15.115 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:15.115 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T16:16:15.115 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T16:16:15.116 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T16:16:15.116 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:15.116 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 1, 2024-04-07T16:16:15.116 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:15.116 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:15.116 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 66, 2024-04-07T16:16:15.116 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:15.116 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:15.116 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:15.116 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.237236+0000", 2024-04-07T16:16:15.116 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:13:30.011181+0000", 2024-04-07T16:16:15.116 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:13:30.011181+0000", 2024-04-07T16:16:15.116 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:13:30.012072+0000", 2024-04-07T16:16:15.116 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.237236+0000", 2024-04-07T16:16:15.116 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:13:28.963534+0000", 2024-04-07T16:16:15.117 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:15.117 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:13:28.963534+0000", 2024-04-07T16:16:15.117 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 67, 2024-04-07T16:16:15.117 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.237236+0000", 2024-04-07T16:16:15.117 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.237236+0000", 2024-04-07T16:16:15.117 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.237236+0000", 2024-04-07T16:16:15.117 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:15.117 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:15.117 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:13:28.963534+0000", 2024-04-07T16:16:15.117 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.237236+0000", 2024-04-07T16:16:15.117 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.237236+0000", 2024-04-07T16:16:15.117 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:15.118 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:15.118 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:15.118 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:15.118 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 66, 2024-04-07T16:16:15.118 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:15.118 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:15.118 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:15.118 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:15.118 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:15.118 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:15.118 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:15.119 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:15.119 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "4.7", 2024-04-07T16:16:15.119 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:15.119 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:15.119 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:15.119 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 13, 2024-04-07T16:16:15.119 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:15.119 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T21:19:09.530105+0000", 2024-04-07T16:16:15.119 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:15.119 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:15.119 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:15.119 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:15.119 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:15.119 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:15.120 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:15.120 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:15.120 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:15.120 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:15.120 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:15.120 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:15.120 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:15.120 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:15.120 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:15.120 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:15.120 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:15.120 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:15.120 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:15.120 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:15.120 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:15.121 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:15.121 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:15.121 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:15.121 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:15.121 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:15.121 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:15.121 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:15.121 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:15.121 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:15.121 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:15.121 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:15.121 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:15.121 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:15.121 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:15.121 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:15.122 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:15.122 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:15.122 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:15.122 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:15.122 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:15.122 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:15.122 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:15.122 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.122 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:15.122 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:15.122 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:15.122 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T16:16:15.122 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T16:16:15.122 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T16:16:15.122 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:15.123 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 1, 2024-04-07T16:16:15.123 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:15.123 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.123 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:15.123 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:15.123 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T16:16:15.123 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T16:16:15.123 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T16:16:15.123 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:15.123 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 1, 2024-04-07T16:16:15.123 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:15.123 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:15.123 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T16:16:15.123 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:15.123 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:15.124 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:15.124 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.235831+0000", 2024-04-07T16:16:15.124 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.393472+0000", 2024-04-07T16:16:15.124 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.393472+0000", 2024-04-07T16:16:15.124 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.393979+0000", 2024-04-07T16:16:15.124 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.235831+0000", 2024-04-07T16:16:15.124 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:15.124 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:15.124 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:15.124 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 61, 2024-04-07T16:16:15.124 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.235831+0000", 2024-04-07T16:16:15.124 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.235831+0000", 2024-04-07T16:16:15.124 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.235831+0000", 2024-04-07T16:16:15.124 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:15.124 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:15.125 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:15.125 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.235831+0000", 2024-04-07T16:16:15.125 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.235831+0000", 2024-04-07T16:16:15.125 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:15.125 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:15.125 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:15.125 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:15.125 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 60, 2024-04-07T16:16:15.125 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:15.125 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:15.125 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:15.125 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:15.125 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:15.125 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:15.125 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:15.125 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:15.126 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.0", 2024-04-07T16:16:15.126 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:15.126 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:15.126 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:15.126 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 19, 2024-04-07T16:16:15.126 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:15.126 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T21:51:13.834107+0000", 2024-04-07T16:16:15.126 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:15.126 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:15.126 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:15.126 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:15.126 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:15.126 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:15.126 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:15.127 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:15.127 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:15.127 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:15.127 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:15.127 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:15.127 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:15.127 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:15.127 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:15.127 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:15.127 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:15.127 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:15.127 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:15.127 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:15.127 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:15.128 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:15.128 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:15.128 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:15.128 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:15.128 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:15.128 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:15.128 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:15.128 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:15.128 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:15.128 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:15.128 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:15.128 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:15.128 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:15.128 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:15.129 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:15.129 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:15.129 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:15.129 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:15.129 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:15.129 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:15.129 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:15.129 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:15.129 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.129 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:15.129 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:15.129 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:15.129 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T16:16:15.129 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T16:16:15.129 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T16:16:15.130 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:15.130 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 1, 2024-04-07T16:16:15.130 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:15.130 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.130 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:15.130 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:15.130 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T16:16:15.130 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T16:16:15.130 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T16:16:15.130 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:15.130 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 2, 2024-04-07T16:16:15.130 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:15.130 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:15.130 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T16:16:15.130 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:15.131 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:15.131 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:15.131 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.764382+0000", 2024-04-07T16:16:15.131 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.034795+0000", 2024-04-07T16:16:15.131 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.034795+0000", 2024-04-07T16:16:15.131 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.035071+0000", 2024-04-07T16:16:15.131 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.764382+0000", 2024-04-07T16:16:15.131 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:15.131 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:15.131 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:15.131 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 60, 2024-04-07T16:16:15.131 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.764382+0000", 2024-04-07T16:16:15.131 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.764382+0000", 2024-04-07T16:16:15.131 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.764382+0000", 2024-04-07T16:16:15.131 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:15.132 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:15.132 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:15.132 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.764382+0000", 2024-04-07T16:16:15.132 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.764382+0000", 2024-04-07T16:16:15.132 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:15.132 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:15.132 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:15.132 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:15.132 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 59, 2024-04-07T16:16:15.132 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:15.132 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:15.132 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:15.132 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:15.132 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:15.132 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:15.133 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:15.133 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:15.133 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.1", 2024-04-07T16:16:15.133 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:15.133 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:15.133 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:15.133 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 20, 2024-04-07T16:16:15.133 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:15.133 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T17:51:30.599527+0000", 2024-04-07T16:16:15.133 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:15.133 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:15.133 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:15.133 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:15.133 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:15.134 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:15.134 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:15.134 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:15.134 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:15.134 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:15.134 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:15.134 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:15.134 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:15.134 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:15.134 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:15.134 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:15.134 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:15.134 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:15.134 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:15.134 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:15.135 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:15.135 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:15.135 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:15.135 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:15.135 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:15.135 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:15.135 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:15.135 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:15.135 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:15.135 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:15.135 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:15.135 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:15.135 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:15.135 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:15.136 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:15.136 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:15.136 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:15.136 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:15.136 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:15.136 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:15.136 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:15.136 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:15.136 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:15.136 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.136 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:15.136 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:15.136 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:15.136 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T16:16:15.136 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T16:16:15.137 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T16:16:15.137 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:15.137 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 2, 2024-04-07T16:16:15.137 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:15.137 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.137 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:15.137 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:15.137 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T16:16:15.137 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T16:16:15.137 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T16:16:15.137 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:15.137 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 6, 2024-04-07T16:16:15.137 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:15.137 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:15.137 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 66, 2024-04-07T16:16:15.138 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:15.138 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:15.138 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:15.138 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.526254+0000", 2024-04-07T16:16:15.138 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:13:30.035058+0000", 2024-04-07T16:16:15.138 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:13:30.035058+0000", 2024-04-07T16:16:15.138 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:13:30.035513+0000", 2024-04-07T16:16:15.138 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.526254+0000", 2024-04-07T16:16:15.138 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:13:28.963534+0000", 2024-04-07T16:16:15.138 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:15.138 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:13:28.963534+0000", 2024-04-07T16:16:15.138 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 67, 2024-04-07T16:16:15.138 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.526254+0000", 2024-04-07T16:16:15.138 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.526254+0000", 2024-04-07T16:16:15.139 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.526254+0000", 2024-04-07T16:16:15.139 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:15.139 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:15.139 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:13:28.963534+0000", 2024-04-07T16:16:15.139 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.526254+0000", 2024-04-07T16:16:15.139 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.526254+0000", 2024-04-07T16:16:15.139 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:15.139 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:15.139 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:15.139 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:15.139 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 66, 2024-04-07T16:16:15.139 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:15.139 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:15.139 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:15.140 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:15.140 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:15.140 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:15.140 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:15.140 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:15.140 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "4.5", 2024-04-07T16:16:15.140 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:15.140 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:15.140 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:15.140 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 13, 2024-04-07T16:16:15.140 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:15.140 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T22:41:30.257805+0000", 2024-04-07T16:16:15.140 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:15.140 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:15.140 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:15.140 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:15.141 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:15.141 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:15.141 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:15.141 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:15.141 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:15.141 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:15.141 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:15.141 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:15.141 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:15.141 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:15.141 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:15.141 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:15.141 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:15.141 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:15.141 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:15.141 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:15.142 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:15.142 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:15.142 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:15.142 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:15.142 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:15.142 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:15.142 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:15.142 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:15.142 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:15.142 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:15.142 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:15.142 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:15.142 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:15.142 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:15.143 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:15.143 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:15.143 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:15.143 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:15.143 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:15.143 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:15.143 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:15.143 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:15.143 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:15.143 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.143 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:15.143 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:15.143 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:15.143 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T16:16:15.143 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T16:16:15.143 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T16:16:15.144 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:15.144 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 6, 2024-04-07T16:16:15.144 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:15.144 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.144 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:15.144 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:15.144 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T16:16:15.144 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T16:16:15.144 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T16:16:15.144 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:15.144 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 3, 2024-04-07T16:16:15.144 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:15.144 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:15.144 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T16:16:15.144 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:15.144 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:15.145 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:15.145 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.186788+0000", 2024-04-07T16:16:15.145 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.394392+0000", 2024-04-07T16:16:15.145 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.394392+0000", 2024-04-07T16:16:15.145 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.394635+0000", 2024-04-07T16:16:15.145 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.186788+0000", 2024-04-07T16:16:15.145 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:15.145 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:15.145 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:15.145 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 61, 2024-04-07T16:16:15.145 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.186788+0000", 2024-04-07T16:16:15.145 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.186788+0000", 2024-04-07T16:16:15.145 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.186788+0000", 2024-04-07T16:16:15.145 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:15.145 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:15.145 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:15.146 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.186788+0000", 2024-04-07T16:16:15.146 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.186788+0000", 2024-04-07T16:16:15.146 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:15.146 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:15.146 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:15.146 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:15.146 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 60, 2024-04-07T16:16:15.146 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:15.146 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:15.146 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:15.146 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:15.146 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:15.146 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:15.146 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:15.146 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:15.146 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.2", 2024-04-07T16:16:15.147 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:15.147 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:15.147 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:15.147 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 19, 2024-04-07T16:16:15.147 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:15.147 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T17:52:10.950481+0000", 2024-04-07T16:16:15.147 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:15.147 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:15.147 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:15.147 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:15.147 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:15.147 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:15.147 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:15.147 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:15.147 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:15.147 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:15.148 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:15.148 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:15.148 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:15.148 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:15.148 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:15.148 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:15.148 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:15.148 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:15.148 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:15.148 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:15.148 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:15.148 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:15.148 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:15.148 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:15.148 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:15.148 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:15.149 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:15.149 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:15.149 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:15.149 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:15.149 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:15.149 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:15.149 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:15.149 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:15.149 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:15.149 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:15.149 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:15.149 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:15.149 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:15.149 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:15.150 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:15.150 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:15.150 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:15.150 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.150 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:15.150 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:15.150 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:15.150 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T16:16:15.150 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T16:16:15.150 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T16:16:15.150 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:15.150 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 3, 2024-04-07T16:16:15.150 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:15.150 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.151 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:15.151 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:15.151 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T16:16:15.151 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T16:16:15.151 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T16:16:15.151 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:15.151 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 5, 2024-04-07T16:16:15.151 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:15.151 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:15.151 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T16:16:15.151 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:15.151 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:15.151 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:15.151 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.024445+0000", 2024-04-07T16:16:15.152 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.044830+0000", 2024-04-07T16:16:15.152 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.044830+0000", 2024-04-07T16:16:15.152 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.045131+0000", 2024-04-07T16:16:15.152 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.024445+0000", 2024-04-07T16:16:15.152 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:15.152 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:15.152 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:15.152 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 60, 2024-04-07T16:16:15.152 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.024445+0000", 2024-04-07T16:16:15.152 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.024445+0000", 2024-04-07T16:16:15.152 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.024445+0000", 2024-04-07T16:16:15.152 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:15.152 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:15.152 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:15.152 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.024445+0000", 2024-04-07T16:16:15.153 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.024445+0000", 2024-04-07T16:16:15.153 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:15.153 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:15.153 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:15.153 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:15.153 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 59, 2024-04-07T16:16:15.153 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:15.153 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:15.153 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:15.153 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:15.153 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:15.153 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:15.153 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:15.153 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:15.154 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.3", 2024-04-07T16:16:15.154 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:15.154 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:15.154 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:15.154 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 20, 2024-04-07T16:16:15.154 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:15.154 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T02:48:50.652491+0000", 2024-04-07T16:16:15.154 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:15.154 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:15.154 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:15.154 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:15.154 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:15.154 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:15.154 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:15.154 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:15.154 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:15.155 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:15.155 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:15.155 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:15.155 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:15.155 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:15.155 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:15.155 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:15.155 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:15.155 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:15.155 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:15.155 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:15.155 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:15.155 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:15.155 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:15.155 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:15.156 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:15.156 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:15.156 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:15.156 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:15.156 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:15.156 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:15.156 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:15.156 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:15.156 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:15.156 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:15.156 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:15.156 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:15.156 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:15.156 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:15.156 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:15.157 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:15.157 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:15.157 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:15.157 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:15.157 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.157 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:15.157 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:15.157 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:15.157 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T16:16:15.157 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T16:16:15.157 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T16:16:15.157 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:15.157 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 5, 2024-04-07T16:16:15.157 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:15.157 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.157 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:15.158 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:15.158 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T16:16:15.158 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T16:16:15.158 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T16:16:15.158 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:15.158 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 7, 2024-04-07T16:16:15.158 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:15.158 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:15.158 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 19, 2024-04-07T16:16:15.158 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:15.158 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:15.158 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:15.158 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.672058+0000", 2024-04-07T16:16:15.158 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:11:33.518060+0000", 2024-04-07T16:16:15.159 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:11:33.518060+0000", 2024-04-07T16:16:15.159 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:11:33.518868+0000", 2024-04-07T16:16:15.159 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.672058+0000", 2024-04-07T16:16:15.159 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:07:26.866003+0000", 2024-04-07T16:16:15.159 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:15.159 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:07:26.866003+0000", 2024-04-07T16:16:15.159 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 58, 2024-04-07T16:16:15.159 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.672058+0000", 2024-04-07T16:16:15.159 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.672058+0000", 2024-04-07T16:16:15.159 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.672058+0000", 2024-04-07T16:16:15.159 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:15.159 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:15.159 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:07:26.866003+0000", 2024-04-07T16:16:15.159 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.672058+0000", 2024-04-07T16:16:15.159 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.672058+0000", 2024-04-07T16:16:15.160 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:15.160 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 128, 2024-04-07T16:16:15.160 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:15.160 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:15.160 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 57, 2024-04-07T16:16:15.160 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:15.160 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:15.160 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:15.160 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:15.160 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 128, 2024-04-07T16:16:15.160 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:15.160 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:15.160 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:15.160 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "1.0", 2024-04-07T16:16:15.160 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:15.161 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:15.161 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:15.161 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 267, 2024-04-07T16:16:15.161 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:15.161 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T22:22:51.043941+0000", 2024-04-07T16:16:15.161 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:15.161 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:15.161 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:15.161 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 590368, 2024-04-07T16:16:15.161 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:15.161 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 1771104, 2024-04-07T16:16:15.161 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:15.161 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:15.161 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:15.161 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:15.161 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:15.162 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:15.162 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:15.162 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:15.162 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:15.162 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:15.162 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:15.162 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:15.162 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:15.162 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 6, 2024-04-07T16:16:15.162 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 2, 2024-04-07T16:16:15.162 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:15.162 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 2, 2024-04-07T16:16:15.162 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:15.162 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:15.163 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:15.163 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:15.163 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:15.163 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:15.163 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:15.163 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 6, 2024-04-07T16:16:15.163 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:15.163 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:15.163 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:15.163 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:15.163 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:15.163 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 212, 2024-04-07T16:16:15.163 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 306, 2024-04-07T16:16:15.163 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:15.163 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:15.163 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:15.164 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 179, 2024-04-07T16:16:15.164 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 2416 2024-04-07T16:16:15.164 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.164 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:15.164 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:15.164 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:15.164 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T16:16:15.164 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T16:16:15.164 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T16:16:15.164 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:15.164 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 7, 2024-04-07T16:16:15.164 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "65'128" 2024-04-07T16:16:15.164 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.164 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:15.164 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:15.165 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T16:16:15.165 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T16:16:15.165 INFO:tasks.workunit.client.0.smithi100.stdout: 2 2024-04-07T16:16:15.165 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:15.165 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 0, 2024-04-07T16:16:15.165 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:15.165 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:15.165 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 66, 2024-04-07T16:16:15.165 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:15.165 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:15.165 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:15.165 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.030611+0000", 2024-04-07T16:16:15.165 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:13:30.040114+0000", 2024-04-07T16:16:15.165 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:13:30.040114+0000", 2024-04-07T16:16:15.165 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:13:30.041000+0000", 2024-04-07T16:16:15.166 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.030611+0000", 2024-04-07T16:16:15.166 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:13:28.963534+0000", 2024-04-07T16:16:15.166 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:15.166 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:13:28.963534+0000", 2024-04-07T16:16:15.166 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 67, 2024-04-07T16:16:15.166 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.030611+0000", 2024-04-07T16:16:15.166 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.030611+0000", 2024-04-07T16:16:15.166 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.030611+0000", 2024-04-07T16:16:15.166 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:15.166 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:15.166 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:13:28.963534+0000", 2024-04-07T16:16:15.166 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.030611+0000", 2024-04-07T16:16:15.166 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.030611+0000", 2024-04-07T16:16:15.166 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:15.167 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:15.167 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:15.167 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:15.167 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 66, 2024-04-07T16:16:15.167 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:15.167 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:15.167 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:15.167 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:15.167 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:15.167 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:15.167 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:15.167 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:15.167 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "4.6", 2024-04-07T16:16:15.167 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:15.168 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:15.168 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:15.168 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 13, 2024-04-07T16:16:15.168 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:15.168 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T03:03:13.375776+0000", 2024-04-07T16:16:15.168 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:15.168 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:15.168 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:15.168 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:15.168 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:15.168 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:15.168 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:15.168 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:15.168 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:15.168 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:15.169 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:15.169 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:15.169 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:15.169 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:15.169 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:15.169 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:15.169 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:15.169 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:15.169 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:15.169 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:15.169 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:15.169 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:15.169 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:15.170 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:15.170 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:15.170 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:15.170 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:15.170 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:15.170 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:15.170 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:15.170 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:15.170 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:15.170 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:15.170 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:15.170 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:15.170 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:15.171 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:15.171 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:15.171 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:15.171 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:15.171 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:15.171 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:15.171 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:15.171 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.171 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:15.171 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:15.171 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:15.171 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T16:16:15.171 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T16:16:15.171 INFO:tasks.workunit.client.0.smithi100.stdout: 2 2024-04-07T16:16:15.172 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:15.172 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 0, 2024-04-07T16:16:15.172 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:15.172 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.172 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:15.172 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:15.172 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T16:16:15.172 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T16:16:15.172 INFO:tasks.workunit.client.0.smithi100.stdout: 3 2024-04-07T16:16:15.172 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:15.172 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 2, 2024-04-07T16:16:15.172 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:15.172 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:15.172 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T16:16:15.172 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:15.173 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:15.173 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:15.173 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.763809+0000", 2024-04-07T16:16:15.173 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:26.225518+0000", 2024-04-07T16:16:15.173 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:26.225518+0000", 2024-04-07T16:16:15.173 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:26.226945+0000", 2024-04-07T16:16:15.173 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.763809+0000", 2024-04-07T16:16:15.173 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:15.173 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:15.173 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:15.173 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 64, 2024-04-07T16:16:15.173 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.763809+0000", 2024-04-07T16:16:15.173 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.763809+0000", 2024-04-07T16:16:15.173 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.763809+0000", 2024-04-07T16:16:15.173 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:15.174 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:15.174 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:15.174 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.763809+0000", 2024-04-07T16:16:15.174 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.763809+0000", 2024-04-07T16:16:15.174 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:15.174 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:15.174 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:15.174 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:15.174 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 63, 2024-04-07T16:16:15.174 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:15.174 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:15.174 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:15.174 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:15.174 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:15.175 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:15.175 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:15.175 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:15.175 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.1", 2024-04-07T16:16:15.175 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:15.175 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:15.175 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:15.175 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 17, 2024-04-07T16:16:15.175 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:15.175 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T22:14:39.830063+0000", 2024-04-07T16:16:15.175 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:15.175 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:15.175 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:15.175 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:15.176 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:15.176 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:15.176 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:15.176 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:15.176 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:15.176 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:15.176 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:15.176 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:15.176 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:15.176 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:15.176 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:15.176 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:15.176 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:15.176 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:15.176 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:15.176 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:15.177 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:15.177 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:15.177 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:15.177 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:15.177 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:15.177 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:15.177 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:15.177 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:15.177 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:15.177 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:15.177 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:15.177 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:15.177 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:15.177 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:15.177 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:15.178 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:15.178 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:15.178 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:15.178 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:15.178 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:15.178 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:15.178 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:15.178 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:15.178 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.178 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:15.178 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:15.178 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:15.178 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T16:16:15.178 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T16:16:15.178 INFO:tasks.workunit.client.0.smithi100.stdout: 3 2024-04-07T16:16:15.179 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:15.179 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 2, 2024-04-07T16:16:15.179 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:15.179 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.179 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:15.179 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:15.179 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T16:16:15.179 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T16:16:15.179 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T16:16:15.179 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:15.179 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 7, 2024-04-07T16:16:15.179 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:15.179 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:15.179 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T16:16:15.179 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:15.180 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:15.180 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:15.180 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.636018+0000", 2024-04-07T16:16:15.180 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.034825+0000", 2024-04-07T16:16:15.180 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.034825+0000", 2024-04-07T16:16:15.180 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.035536+0000", 2024-04-07T16:16:15.180 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.636018+0000", 2024-04-07T16:16:15.180 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:15.180 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:15.180 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:15.180 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 60, 2024-04-07T16:16:15.180 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.636018+0000", 2024-04-07T16:16:15.180 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.636018+0000", 2024-04-07T16:16:15.180 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.636018+0000", 2024-04-07T16:16:15.180 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:15.180 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:15.181 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:15.181 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.636018+0000", 2024-04-07T16:16:15.181 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.636018+0000", 2024-04-07T16:16:15.181 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:15.181 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:15.181 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:15.181 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:15.181 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 59, 2024-04-07T16:16:15.181 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:15.181 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:15.181 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:15.181 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:15.181 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:15.181 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:15.181 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:15.181 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:15.182 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.0", 2024-04-07T16:16:15.182 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:15.182 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:15.182 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:15.182 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 20, 2024-04-07T16:16:15.182 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:15.182 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T20:11:57.033306+0000", 2024-04-07T16:16:15.182 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:15.182 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:15.182 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:15.182 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:15.182 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:15.182 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:15.182 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:15.182 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:15.183 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:15.183 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:15.183 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:15.183 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:15.183 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:15.183 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:15.183 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:15.183 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:15.183 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:15.183 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:15.183 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:15.183 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:15.183 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:15.183 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:15.184 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:15.184 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:15.184 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:15.184 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:15.184 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:15.184 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:15.184 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:15.184 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:15.184 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:15.184 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:15.184 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:15.184 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:15.184 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:15.184 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:15.184 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:15.185 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:15.185 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:15.185 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:15.185 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:15.185 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:15.185 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:15.185 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.185 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:15.185 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:15.185 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:15.185 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T16:16:15.185 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T16:16:15.185 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T16:16:15.185 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:15.185 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 7, 2024-04-07T16:16:15.186 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:15.186 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.186 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:15.186 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:15.186 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T16:16:15.186 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T16:16:15.186 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T16:16:15.186 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:15.186 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 0, 2024-04-07T16:16:15.186 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:15.186 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:15.186 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 66, 2024-04-07T16:16:15.186 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:15.186 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:15.186 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:15.187 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.029002+0000", 2024-04-07T16:16:15.187 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:13:30.040143+0000", 2024-04-07T16:16:15.187 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:13:30.040143+0000", 2024-04-07T16:16:15.187 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:13:30.041281+0000", 2024-04-07T16:16:15.187 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.029002+0000", 2024-04-07T16:16:15.187 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:13:28.963534+0000", 2024-04-07T16:16:15.187 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:15.187 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:13:28.963534+0000", 2024-04-07T16:16:15.187 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 67, 2024-04-07T16:16:15.187 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.029002+0000", 2024-04-07T16:16:15.187 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.029002+0000", 2024-04-07T16:16:15.187 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.029002+0000", 2024-04-07T16:16:15.187 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:15.187 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:15.188 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:13:28.963534+0000", 2024-04-07T16:16:15.188 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.029002+0000", 2024-04-07T16:16:15.188 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.029002+0000", 2024-04-07T16:16:15.188 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:15.188 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:15.188 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:15.188 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:15.188 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 66, 2024-04-07T16:16:15.188 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:15.188 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:15.188 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:15.188 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:15.188 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:15.188 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:15.188 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:15.189 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:15.189 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "4.3", 2024-04-07T16:16:15.189 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:15.189 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:15.189 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:15.189 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 13, 2024-04-07T16:16:15.189 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:15.189 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T01:04:02.813753+0000", 2024-04-07T16:16:15.189 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:15.189 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:15.189 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:15.189 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:15.189 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:15.190 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:15.190 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:15.190 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:15.190 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:15.190 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:15.190 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:15.190 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:15.190 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:15.190 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:15.190 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:15.190 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:15.190 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:15.190 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:15.190 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:15.191 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:15.191 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:15.191 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:15.191 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:15.191 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:15.191 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:15.191 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:15.191 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:15.191 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:15.191 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:15.191 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:15.191 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:15.191 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:15.191 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:15.191 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:15.192 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:15.192 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:15.192 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:15.192 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:15.192 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:15.192 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:15.192 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:15.192 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:15.192 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:15.192 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.192 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:15.192 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:15.192 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:15.192 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T16:16:15.193 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T16:16:15.193 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T16:16:15.193 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:15.193 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 0, 2024-04-07T16:16:15.193 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:15.193 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.193 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:15.193 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:15.193 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T16:16:15.193 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T16:16:15.193 INFO:tasks.workunit.client.0.smithi100.stdout: 4 2024-04-07T16:16:15.193 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:15.193 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 7, 2024-04-07T16:16:15.193 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:15.193 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:15.193 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T16:16:15.194 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:15.194 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:15.194 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:15.194 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.637358+0000", 2024-04-07T16:16:15.194 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.035201+0000", 2024-04-07T16:16:15.194 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.035201+0000", 2024-04-07T16:16:15.194 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.035747+0000", 2024-04-07T16:16:15.194 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.637358+0000", 2024-04-07T16:16:15.194 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:15.194 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:15.194 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:15.194 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 60, 2024-04-07T16:16:15.194 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.637358+0000", 2024-04-07T16:16:15.194 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.637358+0000", 2024-04-07T16:16:15.195 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.637358+0000", 2024-04-07T16:16:15.195 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:15.195 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:15.195 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:15.195 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.637358+0000", 2024-04-07T16:16:15.195 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.637358+0000", 2024-04-07T16:16:15.195 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:15.195 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:15.195 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:15.195 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:15.195 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 59, 2024-04-07T16:16:15.195 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:15.195 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:15.195 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:15.195 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:15.196 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:15.196 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:15.196 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:15.196 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:15.196 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.5", 2024-04-07T16:16:15.196 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:15.196 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:15.196 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:15.196 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 20, 2024-04-07T16:16:15.196 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:15.196 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T21:03:25.449728+0000", 2024-04-07T16:16:15.196 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:15.196 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:15.196 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:15.196 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:15.196 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:15.197 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:15.197 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:15.197 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:15.197 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:15.197 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:15.197 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:15.197 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:15.197 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:15.197 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:15.197 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:15.197 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:15.197 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:15.197 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:15.197 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:15.197 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:15.198 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:15.198 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:15.198 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:15.198 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:15.198 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:15.198 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:15.198 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:15.198 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:15.198 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:15.198 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:15.198 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:15.198 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:15.198 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:15.198 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:15.199 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:15.199 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:15.199 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:15.199 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:15.199 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:15.199 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:15.199 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:15.199 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:15.199 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:15.199 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.199 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:15.199 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:15.199 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:15.199 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T16:16:15.199 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T16:16:15.200 INFO:tasks.workunit.client.0.smithi100.stdout: 4 2024-04-07T16:16:15.200 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:15.200 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 7, 2024-04-07T16:16:15.200 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:15.200 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.200 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:15.200 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:15.200 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T16:16:15.200 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T16:16:15.200 INFO:tasks.workunit.client.0.smithi100.stdout: 5 2024-04-07T16:16:15.200 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:15.200 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 1, 2024-04-07T16:16:15.200 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:15.200 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:15.200 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T16:16:15.201 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:15.201 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:15.201 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:15.201 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.237019+0000", 2024-04-07T16:16:15.201 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.398210+0000", 2024-04-07T16:16:15.201 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.398210+0000", 2024-04-07T16:16:15.201 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.398785+0000", 2024-04-07T16:16:15.201 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.237019+0000", 2024-04-07T16:16:15.201 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:15.201 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:15.201 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:15.201 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 61, 2024-04-07T16:16:15.201 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.237019+0000", 2024-04-07T16:16:15.201 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.237019+0000", 2024-04-07T16:16:15.201 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.237019+0000", 2024-04-07T16:16:15.202 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:15.202 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:15.202 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:15.202 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.237019+0000", 2024-04-07T16:16:15.202 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.237019+0000", 2024-04-07T16:16:15.202 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:15.202 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:15.202 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:15.202 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:15.202 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 60, 2024-04-07T16:16:15.202 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:15.202 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:15.202 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:15.202 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:15.203 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:15.203 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:15.203 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:15.203 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:15.203 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.4", 2024-04-07T16:16:15.203 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:15.203 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:15.203 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:15.203 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 19, 2024-04-07T16:16:15.203 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:15.203 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T02:11:50.752075+0000", 2024-04-07T16:16:15.203 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:15.203 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:15.203 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:15.203 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:15.204 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:15.204 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:15.204 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:15.204 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:15.204 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:15.204 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:15.204 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:15.204 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:15.204 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:15.204 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:15.204 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:15.204 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:15.204 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:15.204 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:15.204 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:15.205 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:15.205 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:15.205 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:15.205 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:15.205 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:15.205 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:15.205 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:15.205 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:15.205 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:15.205 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:15.205 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:15.205 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:15.205 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:15.205 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:15.205 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:15.206 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:15.206 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:15.206 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:15.206 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:15.206 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:15.206 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:15.206 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:15.206 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:15.206 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:15.206 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.206 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:15.206 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:15.206 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:15.206 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T16:16:15.206 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T16:16:15.207 INFO:tasks.workunit.client.0.smithi100.stdout: 5 2024-04-07T16:16:15.207 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:15.207 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 1, 2024-04-07T16:16:15.207 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:15.207 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.207 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:15.207 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:15.207 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T16:16:15.207 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T16:16:15.207 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T16:16:15.207 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:15.207 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 3, 2024-04-07T16:16:15.207 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:15.207 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:15.207 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 66, 2024-04-07T16:16:15.208 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:15.208 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:15.208 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:15.208 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.186582+0000", 2024-04-07T16:16:15.208 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:13:30.040026+0000", 2024-04-07T16:16:15.208 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:13:30.040026+0000", 2024-04-07T16:16:15.208 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:13:30.040851+0000", 2024-04-07T16:16:15.208 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.186582+0000", 2024-04-07T16:16:15.208 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:13:28.963534+0000", 2024-04-07T16:16:15.208 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:15.208 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:13:28.963534+0000", 2024-04-07T16:16:15.208 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 67, 2024-04-07T16:16:15.208 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.186582+0000", 2024-04-07T16:16:15.208 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.186582+0000", 2024-04-07T16:16:15.208 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.186582+0000", 2024-04-07T16:16:15.208 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:15.209 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:15.209 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:13:28.963534+0000", 2024-04-07T16:16:15.209 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.186582+0000", 2024-04-07T16:16:15.209 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.186582+0000", 2024-04-07T16:16:15.209 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:15.209 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:15.209 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:15.209 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:15.209 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 66, 2024-04-07T16:16:15.209 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:15.209 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:15.209 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:15.209 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:15.209 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:15.209 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:15.210 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:15.210 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:15.210 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "4.0", 2024-04-07T16:16:15.210 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:15.210 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:15.210 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:15.210 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 13, 2024-04-07T16:16:15.210 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:15.210 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T18:29:22.422209+0000", 2024-04-07T16:16:15.210 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:15.210 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:15.210 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:15.210 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:15.210 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:15.211 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:15.211 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:15.211 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:15.211 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:15.211 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:15.211 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:15.211 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:15.211 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:15.211 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:15.211 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:15.211 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:15.211 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:15.211 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:15.211 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:15.211 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:15.211 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:15.212 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:15.212 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:15.212 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:15.212 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:15.212 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:15.212 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:15.212 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:15.212 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:15.212 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:15.212 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:15.212 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:15.212 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:15.212 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:15.213 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:15.213 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:15.213 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:15.213 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:15.213 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:15.213 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:15.213 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:15.213 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:15.213 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:15.213 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.213 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:15.213 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:15.213 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:15.213 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T16:16:15.213 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T16:16:15.214 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T16:16:15.214 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:15.214 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 3, 2024-04-07T16:16:15.214 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:15.214 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.214 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:15.214 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:15.214 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T16:16:15.214 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T16:16:15.214 INFO:tasks.workunit.client.0.smithi100.stdout: 4 2024-04-07T16:16:15.214 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:15.214 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 1, 2024-04-07T16:16:15.214 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:15.214 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:15.215 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T16:16:15.215 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:15.215 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:15.215 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:15.215 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.237606+0000", 2024-04-07T16:16:15.215 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.039676+0000", 2024-04-07T16:16:15.215 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.039676+0000", 2024-04-07T16:16:15.215 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.039980+0000", 2024-04-07T16:16:15.215 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.237606+0000", 2024-04-07T16:16:15.215 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:15.215 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:15.215 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:15.215 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 60, 2024-04-07T16:16:15.215 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.237606+0000", 2024-04-07T16:16:15.215 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.237606+0000", 2024-04-07T16:16:15.215 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.237606+0000", 2024-04-07T16:16:15.216 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:15.216 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:15.216 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:15.216 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.237606+0000", 2024-04-07T16:16:15.216 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.237606+0000", 2024-04-07T16:16:15.216 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:15.216 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 1, 2024-04-07T16:16:15.216 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:15.216 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:15.216 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 59, 2024-04-07T16:16:15.216 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:15.216 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:15.216 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:15.216 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:15.216 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 1, 2024-04-07T16:16:15.217 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:15.217 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:15.217 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:15.217 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.6", 2024-04-07T16:16:15.217 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:15.217 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:15.217 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:15.217 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 21, 2024-04-07T16:16:15.217 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:15.217 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T02:37:05.297496+0000", 2024-04-07T16:16:15.217 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:15.217 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:15.217 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:15.217 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:15.217 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:15.217 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:15.218 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:15.218 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:15.218 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:15.218 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:15.218 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:15.218 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:15.218 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:15.218 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:15.218 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:15.218 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:15.218 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:15.218 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:15.218 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:15.218 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 3, 2024-04-07T16:16:15.218 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 1, 2024-04-07T16:16:15.219 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:15.219 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 1, 2024-04-07T16:16:15.219 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:15.219 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:15.219 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:15.219 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:15.219 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:15.219 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 1, 2024-04-07T16:16:15.219 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:15.219 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:15.219 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:15.219 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:15.219 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:15.219 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:15.219 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:15.219 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:15.220 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:15.220 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:15.220 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:15.220 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:15.220 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 1, 2024-04-07T16:16:15.220 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:15.220 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.220 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:15.220 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:15.220 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:15.220 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T16:16:15.220 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T16:16:15.220 INFO:tasks.workunit.client.0.smithi100.stdout: 4 2024-04-07T16:16:15.220 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:15.220 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 1, 2024-04-07T16:16:15.221 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "62'1" 2024-04-07T16:16:15.221 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.221 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:15.221 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:15.221 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T16:16:15.221 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T16:16:15.221 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T16:16:15.221 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:15.221 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 3, 2024-04-07T16:16:15.221 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:15.221 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:15.221 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T16:16:15.221 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:15.221 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:15.221 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:15.222 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.187224+0000", 2024-04-07T16:16:15.222 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.399184+0000", 2024-04-07T16:16:15.222 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.399184+0000", 2024-04-07T16:16:15.222 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.399486+0000", 2024-04-07T16:16:15.222 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.187224+0000", 2024-04-07T16:16:15.222 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:15.222 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:15.222 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:15.222 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 61, 2024-04-07T16:16:15.222 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.187224+0000", 2024-04-07T16:16:15.222 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.187224+0000", 2024-04-07T16:16:15.222 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.187224+0000", 2024-04-07T16:16:15.222 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:15.222 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:15.223 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:15.223 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.187224+0000", 2024-04-07T16:16:15.223 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.187224+0000", 2024-04-07T16:16:15.223 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:15.223 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:15.223 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:15.223 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:15.223 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 60, 2024-04-07T16:16:15.223 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:15.223 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:15.223 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:15.223 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:15.223 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:15.223 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:15.223 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:15.224 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:15.224 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.7", 2024-04-07T16:16:15.224 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:15.224 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:15.224 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:15.224 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 19, 2024-04-07T16:16:15.224 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:15.224 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T21:07:25.104309+0000", 2024-04-07T16:16:15.224 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:15.224 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:15.224 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:15.224 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:15.224 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:15.224 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:15.224 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:15.225 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:15.225 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:15.225 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:15.225 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:15.225 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:15.225 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:15.225 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:15.225 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:15.225 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:15.225 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:15.225 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:15.225 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:15.225 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:15.225 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:15.225 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:15.225 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:15.226 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:15.226 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:15.226 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:15.226 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:15.226 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:15.226 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:15.226 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:15.226 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:15.226 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:15.226 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:15.226 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:15.226 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:15.226 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:15.226 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:15.226 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:15.227 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:15.227 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:15.227 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:15.227 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:15.227 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:15.227 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.227 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:15.227 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:15.227 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:15.227 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T16:16:15.227 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T16:16:15.227 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T16:16:15.227 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:15.227 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 3, 2024-04-07T16:16:15.227 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:15.228 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.228 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:15.228 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:15.228 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T16:16:15.228 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T16:16:15.228 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T16:16:15.228 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:15.228 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 4, 2024-04-07T16:16:15.228 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:15.228 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:15.228 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 66, 2024-04-07T16:16:15.228 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:15.228 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:15.228 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:15.228 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:39.823554+0000", 2024-04-07T16:16:15.228 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:13:30.002945+0000", 2024-04-07T16:16:15.229 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:13:30.002945+0000", 2024-04-07T16:16:15.229 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:13:30.003468+0000", 2024-04-07T16:16:15.229 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:39.823554+0000", 2024-04-07T16:16:15.229 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:13:28.963534+0000", 2024-04-07T16:16:15.229 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:15.229 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:13:28.963534+0000", 2024-04-07T16:16:15.229 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 67, 2024-04-07T16:16:15.229 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:39.823554+0000", 2024-04-07T16:16:15.229 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:39.823554+0000", 2024-04-07T16:16:15.229 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:39.823554+0000", 2024-04-07T16:16:15.229 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:15.229 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:15.229 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:13:28.963534+0000", 2024-04-07T16:16:15.229 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:39.823554+0000", 2024-04-07T16:16:15.229 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:39.823554+0000", 2024-04-07T16:16:15.230 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:15.230 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:15.230 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:15.230 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:15.230 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 66, 2024-04-07T16:16:15.230 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:15.230 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:15.230 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:15.230 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:15.230 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:15.230 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:15.230 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:15.230 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:15.230 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "4.1", 2024-04-07T16:16:15.231 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:15.231 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:15.231 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:15.231 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 13, 2024-04-07T16:16:15.231 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:15.231 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T03:30:42.844279+0000", 2024-04-07T16:16:15.231 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:15.231 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:15.231 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:15.231 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:15.231 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:15.231 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:15.231 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:15.231 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:15.232 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:15.232 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:15.232 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:15.232 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:15.232 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:15.232 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:15.232 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:15.232 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:15.232 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:15.232 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:15.232 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:15.232 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:15.232 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:15.232 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:15.232 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:15.233 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:15.233 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:15.233 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:15.233 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:15.233 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:15.233 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:15.233 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:15.233 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:15.233 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:15.233 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:15.233 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:15.233 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:15.233 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:15.233 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:15.233 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:15.234 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:15.234 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:15.234 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:15.234 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:15.234 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:15.234 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.234 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:15.234 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:15.234 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:15.234 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T16:16:15.234 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T16:16:15.234 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T16:16:15.234 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:15.234 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 4, 2024-04-07T16:16:15.234 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:15.235 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.235 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:15.235 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:15.235 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T16:16:15.235 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T16:16:15.235 INFO:tasks.workunit.client.0.smithi100.stdout: 2 2024-04-07T16:16:15.235 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:15.235 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 6, 2024-04-07T16:16:15.235 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:15.235 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:15.235 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T16:16:15.235 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:15.235 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:15.235 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:15.235 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.552993+0000", 2024-04-07T16:16:15.236 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:26.084985+0000", 2024-04-07T16:16:15.236 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:26.084985+0000", 2024-04-07T16:16:15.236 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:26.091787+0000", 2024-04-07T16:16:15.236 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.552993+0000", 2024-04-07T16:16:15.236 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:15.236 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:15.236 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:15.236 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 64, 2024-04-07T16:16:15.236 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.552993+0000", 2024-04-07T16:16:15.236 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.552993+0000", 2024-04-07T16:16:15.236 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.552993+0000", 2024-04-07T16:16:15.236 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:15.236 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:15.236 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:15.236 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.552993+0000", 2024-04-07T16:16:15.237 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.552993+0000", 2024-04-07T16:16:15.237 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:15.237 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 2, 2024-04-07T16:16:15.237 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:15.237 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:15.237 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 63, 2024-04-07T16:16:15.237 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:15.237 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:15.237 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:15.237 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:15.237 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 2, 2024-04-07T16:16:15.237 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:15.237 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:15.237 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:15.237 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.7", 2024-04-07T16:16:15.238 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:15.238 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:15.238 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:15.238 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 35, 2024-04-07T16:16:15.238 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:15.238 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T02:39:40.077566+0000", 2024-04-07T16:16:15.238 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:15.238 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:15.238 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:15.238 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:15.238 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:15.238 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:15.238 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:15.238 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:15.238 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:15.239 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:15.239 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:15.239 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:15.239 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:15.239 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:15.239 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:15.239 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:15.239 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:15.239 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:15.239 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:15.239 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 3, 2024-04-07T16:16:15.239 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 1, 2024-04-07T16:16:15.239 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:15.239 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 1, 2024-04-07T16:16:15.239 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:15.240 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:15.240 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:15.240 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:15.240 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:15.240 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 1, 2024-04-07T16:16:15.240 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:15.240 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 1, 2024-04-07T16:16:15.240 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:15.240 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:15.240 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:15.240 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:15.240 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:15.240 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:15.240 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:15.240 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:15.241 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:15.241 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:15.241 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 3, 2024-04-07T16:16:15.241 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:15.241 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.241 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:15.241 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:15.241 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:15.241 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T16:16:15.241 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T16:16:15.241 INFO:tasks.workunit.client.0.smithi100.stdout: 2 2024-04-07T16:16:15.241 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:15.241 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 6, 2024-04-07T16:16:15.241 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "61'2" 2024-04-07T16:16:15.242 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.242 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:15.242 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:15.242 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T16:16:15.242 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T16:16:15.242 INFO:tasks.workunit.client.0.smithi100.stdout: 4 2024-04-07T16:16:15.242 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:15.242 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 0, 2024-04-07T16:16:15.242 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:15.242 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:15.242 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T16:16:15.242 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:15.242 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:15.243 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:15.243 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.031201+0000", 2024-04-07T16:16:15.243 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.406018+0000", 2024-04-07T16:16:15.243 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.406018+0000", 2024-04-07T16:16:15.243 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.406541+0000", 2024-04-07T16:16:15.243 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.031201+0000", 2024-04-07T16:16:15.243 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:15.243 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:15.243 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:15.243 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 61, 2024-04-07T16:16:15.243 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.031201+0000", 2024-04-07T16:16:15.243 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.031201+0000", 2024-04-07T16:16:15.243 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.031201+0000", 2024-04-07T16:16:15.243 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:15.243 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:15.244 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:15.244 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.031201+0000", 2024-04-07T16:16:15.244 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.031201+0000", 2024-04-07T16:16:15.244 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:15.244 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:15.244 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:15.244 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:15.244 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 60, 2024-04-07T16:16:15.244 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:15.244 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:15.244 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:15.244 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:15.244 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:15.244 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:15.244 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:15.245 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:15.245 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.6", 2024-04-07T16:16:15.245 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:15.245 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:15.245 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:15.245 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 19, 2024-04-07T16:16:15.245 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:15.245 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T00:24:42.112088+0000", 2024-04-07T16:16:15.245 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:15.245 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:15.245 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:15.245 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:15.245 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:15.245 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:15.245 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:15.246 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:15.246 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:15.246 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:15.246 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:15.246 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:15.246 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:15.246 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:15.246 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:15.246 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:15.246 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:15.246 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:15.246 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:15.246 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:15.246 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:15.246 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:15.247 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:15.247 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:15.247 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:15.247 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:15.247 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:15.247 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:15.247 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:15.247 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:15.247 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:15.247 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:15.247 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:15.247 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:15.247 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:15.247 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:15.247 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:15.248 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:15.248 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:15.248 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:15.248 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:15.248 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:15.248 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:15.248 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.248 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:15.248 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:15.248 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:15.248 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T16:16:15.248 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T16:16:15.248 INFO:tasks.workunit.client.0.smithi100.stdout: 4 2024-04-07T16:16:15.248 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:15.248 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 0, 2024-04-07T16:16:15.249 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:15.249 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.249 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:15.249 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:15.249 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T16:16:15.249 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T16:16:15.249 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T16:16:15.249 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:15.249 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 7, 2024-04-07T16:16:15.249 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:15.249 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:15.249 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T16:16:15.249 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:15.249 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:15.249 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:15.250 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.636513+0000", 2024-04-07T16:16:15.250 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.032456+0000", 2024-04-07T16:16:15.250 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.032456+0000", 2024-04-07T16:16:15.250 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.033067+0000", 2024-04-07T16:16:15.250 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.636513+0000", 2024-04-07T16:16:15.250 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:15.250 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:15.250 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:15.250 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 60, 2024-04-07T16:16:15.250 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.636513+0000", 2024-04-07T16:16:15.250 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.636513+0000", 2024-04-07T16:16:15.250 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.636513+0000", 2024-04-07T16:16:15.250 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:15.250 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:15.251 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:15.251 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.636513+0000", 2024-04-07T16:16:15.251 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.636513+0000", 2024-04-07T16:16:15.251 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:15.251 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:15.251 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:15.251 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:15.251 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 59, 2024-04-07T16:16:15.251 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:15.251 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:15.251 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:15.251 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:15.251 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:15.251 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:15.251 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:15.252 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:15.252 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.8", 2024-04-07T16:16:15.252 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:15.252 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:15.252 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:15.252 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 20, 2024-04-07T16:16:15.252 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:15.252 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T23:02:09.025617+0000", 2024-04-07T16:16:15.252 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:15.252 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:15.252 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:15.252 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:15.252 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:15.252 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:15.252 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:15.253 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:15.253 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:15.253 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:15.253 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:15.253 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:15.253 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:15.253 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:15.253 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:15.253 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:15.253 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:15.253 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:15.253 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:15.253 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:15.253 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:15.253 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:15.254 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:15.254 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:15.254 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:15.254 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:15.254 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:15.254 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:15.254 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:15.254 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:15.254 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:15.254 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:15.255 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:15.255 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:15.255 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:15.255 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:15.255 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:15.255 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:15.255 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:15.255 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:15.255 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:15.255 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:15.255 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:15.255 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.255 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:15.255 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:15.256 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:15.256 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T16:16:15.256 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T16:16:15.256 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T16:16:15.256 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:15.256 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 7, 2024-04-07T16:16:15.256 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:15.256 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.256 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:15.256 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:15.256 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T16:16:15.256 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T16:16:15.256 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T16:16:15.256 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:15.257 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 4, 2024-04-07T16:16:15.257 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:15.257 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:15.257 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T16:16:15.257 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:15.257 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:15.257 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:15.257 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:39.824230+0000", 2024-04-07T16:16:15.257 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.404321+0000", 2024-04-07T16:16:15.257 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.404321+0000", 2024-04-07T16:16:15.257 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.404632+0000", 2024-04-07T16:16:15.257 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:39.824230+0000", 2024-04-07T16:16:15.257 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:15.257 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:15.257 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:15.258 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 61, 2024-04-07T16:16:15.258 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:39.824230+0000", 2024-04-07T16:16:15.258 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:39.824230+0000", 2024-04-07T16:16:15.258 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:39.824230+0000", 2024-04-07T16:16:15.258 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:15.258 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:15.258 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:15.258 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:39.824230+0000", 2024-04-07T16:16:15.258 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:39.824230+0000", 2024-04-07T16:16:15.258 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:15.258 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:15.258 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:15.258 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:15.258 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 60, 2024-04-07T16:16:15.259 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:15.259 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:15.259 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:15.259 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:15.259 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:15.259 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:15.259 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:15.259 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:15.259 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.9", 2024-04-07T16:16:15.259 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:15.259 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:15.259 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:15.259 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 19, 2024-04-07T16:16:15.259 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:15.259 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T19:53:50.339257+0000", 2024-04-07T16:16:15.260 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:15.260 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:15.260 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:15.260 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:15.260 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:15.260 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:15.260 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:15.260 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:15.260 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:15.260 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:15.260 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:15.260 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:15.260 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:15.260 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:15.260 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:15.261 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:15.261 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:15.261 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:15.261 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:15.261 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:15.261 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:15.261 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:15.261 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:15.261 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:15.261 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:15.261 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:15.261 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:15.261 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:15.261 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:15.261 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:15.261 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:15.262 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:15.262 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:15.262 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:15.262 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:15.262 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:15.262 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:15.262 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:15.262 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:15.262 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:15.262 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:15.262 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:15.262 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:15.262 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.262 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:15.262 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:15.263 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:15.263 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T16:16:15.263 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T16:16:15.263 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T16:16:15.263 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:15.263 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 4, 2024-04-07T16:16:15.263 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:15.263 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.263 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:15.263 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:15.263 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T16:16:15.263 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T16:16:15.263 INFO:tasks.workunit.client.0.smithi100.stdout: 3 2024-04-07T16:16:15.263 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:15.263 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 1, 2024-04-07T16:16:15.264 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:15.264 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:15.264 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T16:16:15.264 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:15.264 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:15.264 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:15.264 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.236054+0000", 2024-04-07T16:16:15.264 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.037783+0000", 2024-04-07T16:16:15.264 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.037783+0000", 2024-04-07T16:16:15.264 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.038113+0000", 2024-04-07T16:16:15.264 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.236054+0000", 2024-04-07T16:16:15.264 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:15.264 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:15.264 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:15.264 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 60, 2024-04-07T16:16:15.265 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.236054+0000", 2024-04-07T16:16:15.265 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.236054+0000", 2024-04-07T16:16:15.265 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.236054+0000", 2024-04-07T16:16:15.265 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:15.265 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:15.265 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:15.265 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.236054+0000", 2024-04-07T16:16:15.265 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.236054+0000", 2024-04-07T16:16:15.265 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:15.265 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:15.265 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:15.265 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:15.265 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 59, 2024-04-07T16:16:15.265 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:15.265 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:15.266 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:15.266 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:15.266 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:15.266 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:15.266 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:15.266 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:15.266 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.9", 2024-04-07T16:16:15.266 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:15.266 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:15.266 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:15.266 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 20, 2024-04-07T16:16:15.266 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:15.266 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T00:58:32.085451+0000", 2024-04-07T16:16:15.266 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:15.266 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:15.267 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:15.267 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:15.267 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:15.267 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:15.267 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:15.267 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:15.267 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:15.267 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:15.267 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:15.267 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:15.267 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:15.267 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:15.267 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:15.267 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:15.267 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:15.268 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:15.268 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:15.268 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:15.268 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:15.268 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:15.268 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:15.268 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:15.268 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:15.268 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:15.268 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:15.268 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:15.268 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:15.268 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:15.268 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:15.268 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:15.269 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:15.269 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:15.269 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:15.269 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:15.269 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:15.269 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:15.269 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:15.269 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:15.269 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:15.269 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:15.269 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:15.269 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.269 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:15.269 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:15.269 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:15.270 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T16:16:15.270 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T16:16:15.270 INFO:tasks.workunit.client.0.smithi100.stdout: 3 2024-04-07T16:16:15.270 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:15.270 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 1, 2024-04-07T16:16:15.270 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:15.270 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.270 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:15.270 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:15.270 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T16:16:15.270 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T16:16:15.270 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T16:16:15.270 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:15.270 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 3, 2024-04-07T16:16:15.271 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:15.271 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:15.271 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T16:16:15.271 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:15.271 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:15.271 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:15.271 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.187183+0000", 2024-04-07T16:16:15.271 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.392016+0000", 2024-04-07T16:16:15.271 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.392016+0000", 2024-04-07T16:16:15.271 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.392396+0000", 2024-04-07T16:16:15.271 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.187183+0000", 2024-04-07T16:16:15.271 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:15.271 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:15.271 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:15.271 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 61, 2024-04-07T16:16:15.272 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.187183+0000", 2024-04-07T16:16:15.272 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.187183+0000", 2024-04-07T16:16:15.272 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.187183+0000", 2024-04-07T16:16:15.272 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:15.272 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:15.272 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:15.272 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.187183+0000", 2024-04-07T16:16:15.272 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.187183+0000", 2024-04-07T16:16:15.272 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:15.272 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:15.272 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:15.272 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:15.272 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 60, 2024-04-07T16:16:15.272 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:15.272 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:15.273 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:15.273 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:15.273 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:15.273 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:15.273 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:15.273 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:15.273 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.8", 2024-04-07T16:16:15.273 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:15.273 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:15.273 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:15.273 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 19, 2024-04-07T16:16:15.273 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:15.273 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T02:35:50.790051+0000", 2024-04-07T16:16:15.273 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:15.273 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:15.274 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:15.274 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:15.274 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:15.274 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:15.274 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:15.274 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:15.274 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:15.274 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:15.274 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:15.274 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:15.274 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:15.274 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:15.274 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:15.274 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:15.275 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:15.275 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:15.275 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:15.275 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:15.275 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:15.275 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:15.275 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:15.275 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:15.275 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:15.275 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:15.275 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:15.275 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:15.275 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:15.275 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:15.275 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:15.276 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:15.276 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:15.276 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:15.276 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:15.276 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:15.276 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:15.276 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:15.276 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:15.276 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:15.276 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:15.276 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:15.276 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:15.276 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.276 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:15.276 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:15.277 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:15.277 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T16:16:15.277 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T16:16:15.277 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T16:16:15.277 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:15.277 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 3, 2024-04-07T16:16:15.277 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:15.277 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.277 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:15.277 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:15.277 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T16:16:15.277 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T16:16:15.277 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T16:16:15.277 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:15.277 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 1, 2024-04-07T16:16:15.278 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:15.278 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:15.278 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T16:16:15.278 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:15.278 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:15.278 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:15.278 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.237136+0000", 2024-04-07T16:16:15.278 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.038110+0000", 2024-04-07T16:16:15.278 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.038110+0000", 2024-04-07T16:16:15.278 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.038401+0000", 2024-04-07T16:16:15.278 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.237136+0000", 2024-04-07T16:16:15.278 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:15.278 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:15.278 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:15.278 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 60, 2024-04-07T16:16:15.279 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.237136+0000", 2024-04-07T16:16:15.279 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.237136+0000", 2024-04-07T16:16:15.279 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.237136+0000", 2024-04-07T16:16:15.279 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:15.279 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:15.279 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:15.279 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.237136+0000", 2024-04-07T16:16:15.279 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.237136+0000", 2024-04-07T16:16:15.279 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:15.279 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:15.279 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:15.279 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:15.279 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 59, 2024-04-07T16:16:15.279 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:15.280 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:15.280 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:15.280 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:15.280 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:15.280 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:15.280 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:15.280 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:15.280 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.a", 2024-04-07T16:16:15.280 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:15.280 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:15.280 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:15.280 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 20, 2024-04-07T16:16:15.280 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:15.280 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T01:16:50.295588+0000", 2024-04-07T16:16:15.280 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:15.280 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:15.281 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:15.281 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:15.281 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:15.281 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:15.281 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:15.281 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:15.281 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:15.281 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:15.281 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:15.281 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:15.281 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:15.281 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:15.281 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:15.281 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:15.281 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:15.281 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:15.282 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:15.282 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:15.282 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:15.282 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:15.282 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:15.282 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:15.282 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:15.282 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:15.282 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:15.282 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:15.282 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:15.282 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:15.282 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:15.282 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:15.282 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:15.283 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:15.283 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:15.283 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:15.283 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:15.283 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:15.283 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:15.283 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:15.283 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:15.283 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:15.283 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:15.283 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.283 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:15.283 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:15.283 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:15.283 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T16:16:15.284 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T16:16:15.284 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T16:16:15.284 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:15.284 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 1, 2024-04-07T16:16:15.284 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:15.284 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.284 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:15.284 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:15.284 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T16:16:15.284 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T16:16:15.284 INFO:tasks.workunit.client.0.smithi100.stdout: 4 2024-04-07T16:16:15.284 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:15.284 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 3, 2024-04-07T16:16:15.284 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:15.284 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:15.284 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T16:16:15.285 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:15.285 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:15.285 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:15.285 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.186955+0000", 2024-04-07T16:16:15.285 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.397593+0000", 2024-04-07T16:16:15.285 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.397593+0000", 2024-04-07T16:16:15.285 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.397819+0000", 2024-04-07T16:16:15.285 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.186955+0000", 2024-04-07T16:16:15.285 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:15.285 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:15.285 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:15.285 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 61, 2024-04-07T16:16:15.285 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.186955+0000", 2024-04-07T16:16:15.285 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.186955+0000", 2024-04-07T16:16:15.285 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.186955+0000", 2024-04-07T16:16:15.285 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:15.286 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:15.286 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:15.286 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.186955+0000", 2024-04-07T16:16:15.286 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.186955+0000", 2024-04-07T16:16:15.286 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:15.286 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:15.286 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:15.286 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:15.286 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 60, 2024-04-07T16:16:15.286 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:15.286 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:15.286 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:15.286 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:15.286 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:15.286 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:15.287 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:15.287 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:15.287 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.b", 2024-04-07T16:16:15.287 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:15.287 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:15.287 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:15.287 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 19, 2024-04-07T16:16:15.287 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:15.287 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T02:30:01.472030+0000", 2024-04-07T16:16:15.287 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:15.287 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:15.287 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:15.287 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:15.287 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:15.287 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:15.288 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:15.288 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:15.288 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:15.288 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:15.288 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:15.288 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:15.288 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:15.288 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:15.288 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:15.288 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:15.288 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:15.288 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:15.288 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:15.288 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:15.288 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:15.289 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:15.289 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:15.289 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:15.289 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:15.289 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:15.289 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:15.289 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:15.289 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:15.289 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:15.289 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:15.289 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:15.289 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:15.289 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:15.289 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:15.289 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:15.290 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:15.290 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:15.290 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:15.290 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:15.290 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:15.290 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:15.290 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:15.290 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.290 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:15.290 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:15.290 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:15.290 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T16:16:15.290 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T16:16:15.290 INFO:tasks.workunit.client.0.smithi100.stdout: 4 2024-04-07T16:16:15.290 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:15.290 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 3, 2024-04-07T16:16:15.291 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:15.291 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.291 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:15.291 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:15.291 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T16:16:15.291 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T16:16:15.291 INFO:tasks.workunit.client.0.smithi100.stdout: 5 2024-04-07T16:16:15.291 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:15.291 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 7, 2024-04-07T16:16:15.291 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:15.291 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:15.291 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T16:16:15.291 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:15.291 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:15.291 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:15.291 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.636990+0000", 2024-04-07T16:16:15.291 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.041873+0000", 2024-04-07T16:16:15.292 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.041873+0000", 2024-04-07T16:16:15.292 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.043982+0000", 2024-04-07T16:16:15.292 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.636990+0000", 2024-04-07T16:16:15.292 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:15.292 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:15.292 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:15.292 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 60, 2024-04-07T16:16:15.292 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.636990+0000", 2024-04-07T16:16:15.292 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.636990+0000", 2024-04-07T16:16:15.292 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.636990+0000", 2024-04-07T16:16:15.292 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:15.292 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:15.292 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:15.292 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.636990+0000", 2024-04-07T16:16:15.293 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.636990+0000", 2024-04-07T16:16:15.293 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:15.293 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 2, 2024-04-07T16:16:15.293 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:15.293 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:15.293 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 59, 2024-04-07T16:16:15.293 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:15.293 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:15.293 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:15.293 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:15.293 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 2, 2024-04-07T16:16:15.293 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:15.293 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:15.293 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:15.293 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.b", 2024-04-07T16:16:15.293 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:15.294 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:15.294 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:15.294 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 22, 2024-04-07T16:16:15.294 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:15.294 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T20:03:35.575133+0000", 2024-04-07T16:16:15.294 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:15.294 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:15.294 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:15.294 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:15.294 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:15.294 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:15.294 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:15.294 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:15.294 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:15.294 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:15.294 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:15.295 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:15.295 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:15.295 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:15.295 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:15.295 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:15.295 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:15.295 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:15.295 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:15.295 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 3, 2024-04-07T16:16:15.295 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 1, 2024-04-07T16:16:15.295 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:15.295 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 1, 2024-04-07T16:16:15.295 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:15.295 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:15.295 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:15.295 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:15.296 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:15.296 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 1, 2024-04-07T16:16:15.296 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:15.296 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:15.296 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:15.296 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:15.296 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:15.296 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:15.296 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:15.296 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:15.296 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:15.296 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:15.296 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:15.296 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:15.296 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 1, 2024-04-07T16:16:15.297 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:15.297 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.297 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:15.297 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:15.297 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:15.297 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T16:16:15.297 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T16:16:15.297 INFO:tasks.workunit.client.0.smithi100.stdout: 5 2024-04-07T16:16:15.297 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:15.297 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 7, 2024-04-07T16:16:15.297 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "61'2" 2024-04-07T16:16:15.297 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.297 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:15.297 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:15.297 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T16:16:15.298 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T16:16:15.298 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T16:16:15.298 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:15.298 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 6, 2024-04-07T16:16:15.298 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:15.298 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:15.298 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T16:16:15.298 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:15.298 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:15.298 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:15.298 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.526346+0000", 2024-04-07T16:16:15.298 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.381470+0000", 2024-04-07T16:16:15.298 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.381470+0000", 2024-04-07T16:16:15.298 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.381810+0000", 2024-04-07T16:16:15.298 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.526346+0000", 2024-04-07T16:16:15.299 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:15.299 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:15.299 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:15.299 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 61, 2024-04-07T16:16:15.299 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.526346+0000", 2024-04-07T16:16:15.299 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.526346+0000", 2024-04-07T16:16:15.299 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.526346+0000", 2024-04-07T16:16:15.299 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:15.299 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:15.299 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:15.299 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.526346+0000", 2024-04-07T16:16:15.299 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.526346+0000", 2024-04-07T16:16:15.299 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:15.299 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:15.299 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:15.299 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:15.300 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 60, 2024-04-07T16:16:15.300 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:15.300 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:15.300 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:15.300 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:15.300 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:15.300 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:15.300 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:15.300 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:15.300 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.a", 2024-04-07T16:16:15.300 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:15.300 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:15.300 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:15.300 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 19, 2024-04-07T16:16:15.300 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:15.301 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T17:26:21.450939+0000", 2024-04-07T16:16:15.301 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:15.301 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:15.301 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:15.301 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:15.301 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:15.301 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:15.301 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:15.301 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:15.301 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:15.301 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:15.301 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:15.301 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:15.301 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:15.301 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:15.302 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:15.302 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:15.302 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:15.302 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:15.302 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:15.302 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:15.302 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:15.302 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:15.302 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:15.302 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:15.302 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:15.302 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:15.302 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:15.302 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:15.303 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:15.303 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:15.303 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:15.303 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:15.303 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:15.303 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:15.303 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:15.303 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:15.303 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:15.303 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:15.303 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:15.303 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:15.303 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:15.303 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:15.304 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:15.304 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.304 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:15.304 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:15.304 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:15.304 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T16:16:15.304 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T16:16:15.304 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T16:16:15.304 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:15.304 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 6, 2024-04-07T16:16:15.304 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:15.304 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.304 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:15.304 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:15.304 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T16:16:15.305 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T16:16:15.305 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T16:16:15.305 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:15.305 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 2, 2024-04-07T16:16:15.305 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:15.305 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:15.305 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T16:16:15.305 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:15.305 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:15.305 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:15.305 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.764853+0000", 2024-04-07T16:16:15.305 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.037652+0000", 2024-04-07T16:16:15.305 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.037652+0000", 2024-04-07T16:16:15.305 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.038191+0000", 2024-04-07T16:16:15.305 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.764853+0000", 2024-04-07T16:16:15.306 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:15.306 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:15.306 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:15.306 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 60, 2024-04-07T16:16:15.306 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.764853+0000", 2024-04-07T16:16:15.306 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.764853+0000", 2024-04-07T16:16:15.306 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.764853+0000", 2024-04-07T16:16:15.306 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:15.306 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:15.306 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:15.306 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.764853+0000", 2024-04-07T16:16:15.306 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.764853+0000", 2024-04-07T16:16:15.306 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:15.306 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:15.307 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:15.307 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:15.307 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 59, 2024-04-07T16:16:15.307 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:15.307 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:15.307 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:15.307 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:15.307 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:15.307 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:15.307 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:15.307 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:15.307 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.c", 2024-04-07T16:16:15.307 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:15.307 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:15.307 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:15.308 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 20, 2024-04-07T16:16:15.308 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:15.308 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T17:04:59.525025+0000", 2024-04-07T16:16:15.308 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:15.308 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:15.308 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:15.308 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:15.308 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:15.308 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:15.308 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:15.308 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:15.308 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:15.308 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:15.309 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:15.309 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:15.309 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:15.309 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:15.309 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:15.309 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:15.309 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:15.309 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:15.309 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:15.309 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:15.309 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:15.309 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:15.309 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:15.309 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:15.310 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:15.310 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:15.310 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:15.310 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:15.310 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:15.310 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:15.310 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:15.310 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:15.310 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:15.310 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:15.310 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:15.310 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:15.310 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:15.310 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:15.311 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:15.311 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:15.311 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:15.311 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:15.311 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:15.311 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.311 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:15.311 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:15.311 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:15.311 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T16:16:15.311 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T16:16:15.311 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T16:16:15.311 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:15.311 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 2, 2024-04-07T16:16:15.312 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:15.312 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.312 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:15.312 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:15.312 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T16:16:15.312 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T16:16:15.312 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T16:16:15.312 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:15.312 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 7, 2024-04-07T16:16:15.312 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:15.312 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:15.312 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T16:16:15.312 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:15.312 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:15.313 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:15.313 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.637269+0000", 2024-04-07T16:16:15.313 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.400353+0000", 2024-04-07T16:16:15.313 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.400353+0000", 2024-04-07T16:16:15.313 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.400660+0000", 2024-04-07T16:16:15.313 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.637269+0000", 2024-04-07T16:16:15.313 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:15.313 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:15.313 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:15.313 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 61, 2024-04-07T16:16:15.313 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.637269+0000", 2024-04-07T16:16:15.313 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.637269+0000", 2024-04-07T16:16:15.313 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.637269+0000", 2024-04-07T16:16:15.313 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:15.313 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:15.314 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:15.314 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.637269+0000", 2024-04-07T16:16:15.314 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.637269+0000", 2024-04-07T16:16:15.314 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:15.314 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:15.314 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:15.314 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:15.314 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 60, 2024-04-07T16:16:15.314 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:15.314 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:15.314 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:15.314 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:15.314 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:15.314 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:15.315 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:15.315 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:15.315 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.d", 2024-04-07T16:16:15.315 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:15.315 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:15.315 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:15.315 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 19, 2024-04-07T16:16:15.315 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:15.315 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T00:47:57.573097+0000", 2024-04-07T16:16:15.315 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:15.315 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:15.315 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:15.315 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:15.315 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:15.315 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:15.315 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:15.316 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:15.316 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:15.316 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:15.316 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:15.316 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:15.316 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:15.316 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:15.316 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:15.316 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:15.316 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:15.316 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:15.316 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:15.316 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:15.316 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:15.316 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:15.317 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:15.317 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:15.317 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:15.317 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:15.317 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:15.317 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:15.317 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:15.317 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:15.317 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:15.317 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:15.317 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:15.317 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:15.317 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:15.317 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:15.317 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:15.317 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:15.318 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:15.318 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:15.318 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:15.318 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:15.318 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:15.318 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.318 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:15.318 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:15.318 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:15.318 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T16:16:15.318 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T16:16:15.318 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T16:16:15.318 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:15.318 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 7, 2024-04-07T16:16:15.319 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:15.319 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.319 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:15.319 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:15.319 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T16:16:15.319 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T16:16:15.319 INFO:tasks.workunit.client.0.smithi100.stdout: 3 2024-04-07T16:16:15.319 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:15.319 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 1, 2024-04-07T16:16:15.319 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:15.319 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:15.319 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T16:16:15.319 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:15.319 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:15.319 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:15.320 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.237152+0000", 2024-04-07T16:16:15.320 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.034455+0000", 2024-04-07T16:16:15.320 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.034455+0000", 2024-04-07T16:16:15.320 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.034772+0000", 2024-04-07T16:16:15.320 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.237152+0000", 2024-04-07T16:16:15.320 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:15.320 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:15.320 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:15.320 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 60, 2024-04-07T16:16:15.320 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.237152+0000", 2024-04-07T16:16:15.320 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.237152+0000", 2024-04-07T16:16:15.320 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.237152+0000", 2024-04-07T16:16:15.320 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:15.320 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:15.320 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:15.321 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.237152+0000", 2024-04-07T16:16:15.321 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.237152+0000", 2024-04-07T16:16:15.321 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:15.321 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 1, 2024-04-07T16:16:15.321 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:15.321 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:15.321 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 59, 2024-04-07T16:16:15.321 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:15.321 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:15.321 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:15.321 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:15.321 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 1, 2024-04-07T16:16:15.321 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:15.321 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:15.321 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:15.322 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.d", 2024-04-07T16:16:15.322 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:15.322 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:15.322 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:15.322 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 21, 2024-04-07T16:16:15.322 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:15.322 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T23:10:30.557880+0000", 2024-04-07T16:16:15.322 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:15.322 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:15.322 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:15.322 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 70, 2024-04-07T16:16:15.322 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:15.322 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:15.322 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:15.322 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:15.323 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:15.323 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:15.323 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:15.323 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:15.323 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:15.323 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:15.323 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:15.323 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:15.323 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:15.323 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:15.323 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:15.323 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 3, 2024-04-07T16:16:15.323 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 1, 2024-04-07T16:16:15.323 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:15.323 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 1, 2024-04-07T16:16:15.323 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:15.324 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:15.324 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:15.324 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:15.324 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:15.324 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:15.324 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:15.324 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:15.324 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:15.324 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:15.324 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:15.324 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:15.324 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:15.324 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:15.324 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:15.324 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:15.325 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:15.325 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:15.325 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 1, 2024-04-07T16:16:15.325 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 1 2024-04-07T16:16:15.325 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.325 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:15.325 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:15.325 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:15.325 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T16:16:15.325 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T16:16:15.325 INFO:tasks.workunit.client.0.smithi100.stdout: 3 2024-04-07T16:16:15.325 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:15.325 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 1, 2024-04-07T16:16:15.325 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "61'1" 2024-04-07T16:16:15.326 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.326 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:15.326 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:15.326 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T16:16:15.326 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T16:16:15.326 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T16:16:15.326 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:15.326 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 5, 2024-04-07T16:16:15.326 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:15.326 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:15.326 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T16:16:15.326 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:15.326 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:15.326 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:15.326 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.025357+0000", 2024-04-07T16:16:15.327 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.398120+0000", 2024-04-07T16:16:15.327 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.398120+0000", 2024-04-07T16:16:15.327 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.400762+0000", 2024-04-07T16:16:15.327 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.025357+0000", 2024-04-07T16:16:15.327 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:15.327 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:15.327 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:15.327 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 61, 2024-04-07T16:16:15.327 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.025357+0000", 2024-04-07T16:16:15.327 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.025357+0000", 2024-04-07T16:16:15.327 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.025357+0000", 2024-04-07T16:16:15.327 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:15.327 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:15.327 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:15.327 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.025357+0000", 2024-04-07T16:16:15.328 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.025357+0000", 2024-04-07T16:16:15.328 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:15.328 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:15.328 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:15.328 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:15.328 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 60, 2024-04-07T16:16:15.328 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:15.328 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:15.328 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:15.328 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:15.328 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:15.328 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:15.328 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:15.328 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:15.328 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.c", 2024-04-07T16:16:15.329 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:15.329 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:15.329 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:15.329 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 19, 2024-04-07T16:16:15.329 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:15.329 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T22:11:21.789998+0000", 2024-04-07T16:16:15.329 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:15.329 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:15.329 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:15.329 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:15.329 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:15.329 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:15.329 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:15.329 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:15.330 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:15.330 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:15.330 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:15.330 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:15.330 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:15.330 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:15.330 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:15.330 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:15.330 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:15.330 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:15.330 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:15.330 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:15.330 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:15.330 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:15.331 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:15.331 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:15.331 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:15.331 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:15.331 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:15.331 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:15.331 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:15.331 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:15.331 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:15.331 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:15.331 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:15.331 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:15.331 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:15.331 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:15.331 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:15.332 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:15.332 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:15.332 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:15.332 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:15.332 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:15.332 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:15.332 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.332 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:15.332 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:15.332 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:15.332 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T16:16:15.332 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T16:16:15.332 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T16:16:15.332 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:15.332 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 5, 2024-04-07T16:16:15.333 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:15.333 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.333 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:15.333 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:15.333 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T16:16:15.333 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T16:16:15.333 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T16:16:15.333 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:15.333 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 2, 2024-04-07T16:16:15.333 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:15.333 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:15.333 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T16:16:15.333 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:15.333 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:15.334 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:15.334 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.764862+0000", 2024-04-07T16:16:15.334 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.039797+0000", 2024-04-07T16:16:15.334 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.039797+0000", 2024-04-07T16:16:15.334 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.040101+0000", 2024-04-07T16:16:15.334 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.764862+0000", 2024-04-07T16:16:15.334 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:15.334 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:15.334 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:15.334 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 60, 2024-04-07T16:16:15.334 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.764862+0000", 2024-04-07T16:16:15.334 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.764862+0000", 2024-04-07T16:16:15.334 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.764862+0000", 2024-04-07T16:16:15.334 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:15.335 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:15.335 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:15.335 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.764862+0000", 2024-04-07T16:16:15.335 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.764862+0000", 2024-04-07T16:16:15.335 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:15.335 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 2, 2024-04-07T16:16:15.335 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:15.335 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:15.335 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 59, 2024-04-07T16:16:15.335 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:15.335 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:15.335 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:15.335 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:15.335 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 2, 2024-04-07T16:16:15.336 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:15.336 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:15.336 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:15.336 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.e", 2024-04-07T16:16:15.336 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:15.336 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:15.336 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:15.336 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 22, 2024-04-07T16:16:15.336 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:15.336 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T03:27:06.016325+0000", 2024-04-07T16:16:15.336 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:15.336 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:15.336 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:15.336 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:15.336 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:15.337 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:15.337 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:15.337 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:15.337 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:15.337 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:15.337 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:15.337 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:15.337 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:15.337 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:15.337 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:15.337 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:15.337 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:15.337 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:15.337 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:15.337 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 3, 2024-04-07T16:16:15.338 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 1, 2024-04-07T16:16:15.338 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:15.338 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 1, 2024-04-07T16:16:15.338 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:15.338 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:15.338 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:15.338 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:15.338 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:15.338 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 1, 2024-04-07T16:16:15.338 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:15.338 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:15.338 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:15.338 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:15.338 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:15.338 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:15.338 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:15.339 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:15.339 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:15.339 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:15.339 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:15.339 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:15.339 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 3, 2024-04-07T16:16:15.339 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:15.339 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.339 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:15.339 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:15.339 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:15.339 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T16:16:15.339 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T16:16:15.340 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T16:16:15.340 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:15.340 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 2, 2024-04-07T16:16:15.340 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "61'2" 2024-04-07T16:16:15.340 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.340 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:15.340 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:15.340 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T16:16:15.340 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T16:16:15.340 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T16:16:15.340 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:15.340 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 7, 2024-04-07T16:16:15.340 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:15.340 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:15.340 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T16:16:15.341 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:15.341 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:15.341 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:15.341 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.637921+0000", 2024-04-07T16:16:15.341 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.394090+0000", 2024-04-07T16:16:15.341 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.394090+0000", 2024-04-07T16:16:15.341 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.394720+0000", 2024-04-07T16:16:15.341 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.637921+0000", 2024-04-07T16:16:15.341 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:15.341 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:15.341 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:15.341 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 61, 2024-04-07T16:16:15.341 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.637921+0000", 2024-04-07T16:16:15.341 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.637921+0000", 2024-04-07T16:16:15.341 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.637921+0000", 2024-04-07T16:16:15.342 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:15.342 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:15.342 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:15.342 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.637921+0000", 2024-04-07T16:16:15.342 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.637921+0000", 2024-04-07T16:16:15.342 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:15.342 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:15.342 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:15.342 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:15.342 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 60, 2024-04-07T16:16:15.342 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:15.342 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:15.342 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:15.342 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:15.342 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:15.343 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:15.343 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:15.343 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:15.343 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.f", 2024-04-07T16:16:15.343 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:15.343 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:15.343 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:15.343 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 19, 2024-04-07T16:16:15.343 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:15.343 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T00:47:54.579168+0000", 2024-04-07T16:16:15.343 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:15.343 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:15.343 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:15.343 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:15.343 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:15.344 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:15.344 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:15.344 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:15.344 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:15.344 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:15.344 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:15.344 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:15.344 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:15.344 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:15.344 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:15.344 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:15.344 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:15.344 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:15.344 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:15.344 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:15.345 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:15.345 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:15.345 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:15.345 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:15.345 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:15.345 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:15.345 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:15.345 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:15.345 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:15.345 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:15.345 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:15.345 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:15.345 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:15.345 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:15.345 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:15.346 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:15.346 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:15.346 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:15.346 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:15.346 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:15.346 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:15.346 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:15.346 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:15.346 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.346 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:15.346 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:15.346 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:15.346 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T16:16:15.346 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T16:16:15.346 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T16:16:15.347 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:15.347 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 7, 2024-04-07T16:16:15.347 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:15.347 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.347 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:15.347 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:15.347 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T16:16:15.347 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T16:16:15.347 INFO:tasks.workunit.client.0.smithi100.stdout: 5 2024-04-07T16:16:15.347 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:15.347 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 6, 2024-04-07T16:16:15.347 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:15.347 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:15.347 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T16:16:15.347 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:15.348 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:15.348 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:15.348 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.525708+0000", 2024-04-07T16:16:15.348 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.395978+0000", 2024-04-07T16:16:15.348 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.395978+0000", 2024-04-07T16:16:15.348 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.396725+0000", 2024-04-07T16:16:15.348 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.525708+0000", 2024-04-07T16:16:15.348 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:15.348 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:15.348 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:15.348 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 61, 2024-04-07T16:16:15.348 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.525708+0000", 2024-04-07T16:16:15.348 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.525708+0000", 2024-04-07T16:16:15.348 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.525708+0000", 2024-04-07T16:16:15.348 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:15.349 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:15.349 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:15.349 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.525708+0000", 2024-04-07T16:16:15.349 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.525708+0000", 2024-04-07T16:16:15.349 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:15.349 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:15.349 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:15.349 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:15.349 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 60, 2024-04-07T16:16:15.349 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:15.349 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:15.349 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:15.349 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:15.349 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:15.349 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:15.349 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:15.350 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:15.350 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.20", 2024-04-07T16:16:15.350 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:15.350 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:15.350 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:15.350 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 19, 2024-04-07T16:16:15.350 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:15.350 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T21:30:43.276569+0000", 2024-04-07T16:16:15.350 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:15.350 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:15.350 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:15.350 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:15.350 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:15.350 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:15.350 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:15.351 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:15.351 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:15.351 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:15.351 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:15.351 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:15.351 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:15.351 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:15.351 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:15.351 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:15.351 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:15.351 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:15.351 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:15.351 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:15.351 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:15.351 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:15.352 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:15.352 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:15.352 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:15.352 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:15.352 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:15.352 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:15.352 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:15.352 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:15.352 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:15.352 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:15.352 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:15.352 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:15.352 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:15.352 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:15.352 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:15.352 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:15.353 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:15.353 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:15.353 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:15.353 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:15.353 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:15.353 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.353 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:15.353 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:15.353 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:15.353 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T16:16:15.353 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T16:16:15.353 INFO:tasks.workunit.client.0.smithi100.stdout: 5 2024-04-07T16:16:15.353 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:15.353 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 6, 2024-04-07T16:16:15.353 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:15.354 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.354 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:15.354 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:15.354 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T16:16:15.354 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T16:16:15.354 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T16:16:15.354 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:15.354 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 2, 2024-04-07T16:16:15.354 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:15.354 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:15.354 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T16:16:15.354 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:15.354 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:15.354 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:15.355 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.764525+0000", 2024-04-07T16:16:15.355 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.410814+0000", 2024-04-07T16:16:15.355 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.410814+0000", 2024-04-07T16:16:15.355 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.412091+0000", 2024-04-07T16:16:15.355 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.764525+0000", 2024-04-07T16:16:15.355 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:15.355 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:15.355 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:15.355 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 61, 2024-04-07T16:16:15.355 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.764525+0000", 2024-04-07T16:16:15.355 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.764525+0000", 2024-04-07T16:16:15.355 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.764525+0000", 2024-04-07T16:16:15.355 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:15.355 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:15.355 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:15.355 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.764525+0000", 2024-04-07T16:16:15.356 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.764525+0000", 2024-04-07T16:16:15.356 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:15.356 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:15.356 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:15.356 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:15.356 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 60, 2024-04-07T16:16:15.356 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:15.356 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:15.356 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:15.356 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:15.356 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:15.356 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:15.356 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:15.356 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:15.356 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.21", 2024-04-07T16:16:15.357 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:15.357 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:15.357 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:15.357 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 19, 2024-04-07T16:16:15.357 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:15.357 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T01:48:29.034291+0000", 2024-04-07T16:16:15.357 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:15.357 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:15.357 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:15.357 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:15.357 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:15.357 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:15.357 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:15.357 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:15.357 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:15.357 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:15.358 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:15.358 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:15.358 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:15.358 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:15.358 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:15.358 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:15.358 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:15.358 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:15.358 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:15.358 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:15.358 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:15.358 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:15.358 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:15.358 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:15.358 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:15.359 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:15.359 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:15.359 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:15.359 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:15.359 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:15.359 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:15.359 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:15.359 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:15.359 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:15.359 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:15.359 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:15.359 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:15.359 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:15.359 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:15.359 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:15.359 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:15.360 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:15.360 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:15.360 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.360 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:15.360 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:15.360 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:15.360 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T16:16:15.360 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T16:16:15.360 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T16:16:15.360 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:15.360 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 2, 2024-04-07T16:16:15.360 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:15.360 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.360 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:15.361 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:15.361 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T16:16:15.361 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T16:16:15.361 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T16:16:15.361 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:15.361 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 5, 2024-04-07T16:16:15.361 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:15.361 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:15.361 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T16:16:15.361 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:15.361 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:15.361 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:15.361 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.024540+0000", 2024-04-07T16:16:15.361 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.399206+0000", 2024-04-07T16:16:15.361 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.399206+0000", 2024-04-07T16:16:15.361 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.400651+0000", 2024-04-07T16:16:15.362 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.024540+0000", 2024-04-07T16:16:15.362 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:15.362 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:15.362 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:15.362 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 61, 2024-04-07T16:16:15.362 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.024540+0000", 2024-04-07T16:16:15.362 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.024540+0000", 2024-04-07T16:16:15.362 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.024540+0000", 2024-04-07T16:16:15.362 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:15.362 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:15.362 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:15.362 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.024540+0000", 2024-04-07T16:16:15.362 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.024540+0000", 2024-04-07T16:16:15.362 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:15.362 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:15.363 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:15.363 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:15.363 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 60, 2024-04-07T16:16:15.363 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:15.363 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:15.363 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:15.363 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:15.363 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:15.363 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:15.363 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:15.363 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:15.363 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.22", 2024-04-07T16:16:15.363 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:15.363 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:15.363 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:15.364 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 19, 2024-04-07T16:16:15.364 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:15.364 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T22:16:50.256754+0000", 2024-04-07T16:16:15.364 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:15.364 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:15.364 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:15.364 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:15.364 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:15.364 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:15.364 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:15.364 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:15.364 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:15.364 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:15.364 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:15.364 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:15.364 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:15.365 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:15.365 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:15.365 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:15.365 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:15.365 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:15.365 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:15.365 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:15.365 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:15.365 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:15.365 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:15.365 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:15.365 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:15.365 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:15.365 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:15.365 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:15.365 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:15.365 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:15.366 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:15.366 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:15.366 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:15.366 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:15.366 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:15.366 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:15.366 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:15.366 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:15.366 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:15.366 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:15.366 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:15.366 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:15.366 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:15.366 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.366 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:15.367 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:15.367 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:15.367 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T16:16:15.367 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T16:16:15.367 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T16:16:15.367 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:15.367 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 5, 2024-04-07T16:16:15.367 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:15.367 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.367 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:15.367 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:15.367 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T16:16:15.367 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T16:16:15.367 INFO:tasks.workunit.client.0.smithi100.stdout: 4 2024-04-07T16:16:15.367 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:15.368 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 0, 2024-04-07T16:16:15.368 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:15.368 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:15.368 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T16:16:15.368 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:15.368 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:15.368 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:15.368 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.029689+0000", 2024-04-07T16:16:15.368 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.405320+0000", 2024-04-07T16:16:15.368 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.405320+0000", 2024-04-07T16:16:15.368 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.405888+0000", 2024-04-07T16:16:15.368 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.029689+0000", 2024-04-07T16:16:15.368 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:15.368 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:15.368 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:15.368 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 61, 2024-04-07T16:16:15.369 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.029689+0000", 2024-04-07T16:16:15.369 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.029689+0000", 2024-04-07T16:16:15.369 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.029689+0000", 2024-04-07T16:16:15.369 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:15.369 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:15.369 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:15.369 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.029689+0000", 2024-04-07T16:16:15.369 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.029689+0000", 2024-04-07T16:16:15.369 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:15.369 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:15.369 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:15.369 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:15.369 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 60, 2024-04-07T16:16:15.369 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:15.369 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:15.369 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:15.370 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:15.370 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:15.370 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:15.370 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:15.370 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:15.370 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.23", 2024-04-07T16:16:15.370 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:15.370 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:15.370 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:15.370 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 19, 2024-04-07T16:16:15.370 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:15.370 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T22:07:50.421903+0000", 2024-04-07T16:16:15.370 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:15.370 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:15.371 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:15.371 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:15.371 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:15.371 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:15.371 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:15.371 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:15.371 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:15.371 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:15.371 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:15.371 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:15.371 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:15.371 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:15.371 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:15.371 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:15.371 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:15.372 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:15.372 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:15.372 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:15.372 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:15.372 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:15.372 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:15.372 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:15.372 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:15.372 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:15.372 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:15.372 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:15.372 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:15.372 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:15.372 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:15.372 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:15.373 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:15.373 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:15.373 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:15.373 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:15.373 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:15.373 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:15.373 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:15.373 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:15.373 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:15.373 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:15.373 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:15.373 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.373 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:15.373 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:15.374 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:15.374 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T16:16:15.374 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T16:16:15.374 INFO:tasks.workunit.client.0.smithi100.stdout: 4 2024-04-07T16:16:15.374 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:15.374 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 0, 2024-04-07T16:16:15.374 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:15.374 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.374 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:15.374 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:15.374 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T16:16:15.374 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T16:16:15.374 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T16:16:15.374 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:15.374 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 3, 2024-04-07T16:16:15.375 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:15.375 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:15.375 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T16:16:15.375 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:15.375 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:15.375 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:15.375 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.186832+0000", 2024-04-07T16:16:15.375 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.398259+0000", 2024-04-07T16:16:15.375 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.398259+0000", 2024-04-07T16:16:15.375 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.398730+0000", 2024-04-07T16:16:15.375 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.186832+0000", 2024-04-07T16:16:15.375 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:15.375 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:15.375 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:15.375 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 61, 2024-04-07T16:16:15.376 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.186832+0000", 2024-04-07T16:16:15.376 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.186832+0000", 2024-04-07T16:16:15.376 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.186832+0000", 2024-04-07T16:16:15.376 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:15.376 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:15.376 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:15.376 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.186832+0000", 2024-04-07T16:16:15.376 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.186832+0000", 2024-04-07T16:16:15.376 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:15.376 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:15.376 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:15.376 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:15.376 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 60, 2024-04-07T16:16:15.376 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:15.376 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:15.377 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:15.377 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:15.377 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:15.377 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:15.377 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:15.377 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:15.377 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.24", 2024-04-07T16:16:15.377 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:15.377 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:15.377 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:15.377 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 19, 2024-04-07T16:16:15.377 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:15.377 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T02:14:28.915631+0000", 2024-04-07T16:16:15.377 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:15.377 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:15.378 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:15.378 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:15.378 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:15.378 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:15.378 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:15.378 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:15.378 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:15.378 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:15.378 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:15.378 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:15.378 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:15.378 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:15.378 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:15.378 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:15.378 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:15.379 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:15.379 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:15.379 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:15.379 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:15.379 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:15.379 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:15.379 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:15.379 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:15.379 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:15.379 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:15.379 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:15.379 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:15.379 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:15.379 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:15.379 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:15.379 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:15.380 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:15.380 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:15.380 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:15.380 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:15.380 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:15.380 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:15.380 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:15.380 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:15.380 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:15.380 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:15.380 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.380 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:15.380 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:15.380 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:15.380 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T16:16:15.380 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T16:16:15.381 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T16:16:15.381 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:15.381 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 3, 2024-04-07T16:16:15.381 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:15.381 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.381 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:15.381 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:15.381 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T16:16:15.381 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T16:16:15.381 INFO:tasks.workunit.client.0.smithi100.stdout: 2 2024-04-07T16:16:15.381 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:15.381 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 1, 2024-04-07T16:16:15.381 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:15.381 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:15.382 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T16:16:15.382 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:15.382 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:15.382 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:15.382 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.236896+0000", 2024-04-07T16:16:15.382 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.390858+0000", 2024-04-07T16:16:15.382 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.390858+0000", 2024-04-07T16:16:15.382 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.391140+0000", 2024-04-07T16:16:15.382 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.236896+0000", 2024-04-07T16:16:15.382 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:15.382 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:15.382 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:15.382 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 61, 2024-04-07T16:16:15.382 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.236896+0000", 2024-04-07T16:16:15.382 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.236896+0000", 2024-04-07T16:16:15.383 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.236896+0000", 2024-04-07T16:16:15.383 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:15.383 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:15.383 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:15.383 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.236896+0000", 2024-04-07T16:16:15.383 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.236896+0000", 2024-04-07T16:16:15.383 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:15.383 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:15.383 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:15.383 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:15.383 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 60, 2024-04-07T16:16:15.383 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:15.384 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:15.384 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:15.384 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:15.384 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:15.384 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:15.384 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:15.384 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:15.384 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.25", 2024-04-07T16:16:15.384 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:15.384 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:15.384 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:15.384 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 19, 2024-04-07T16:16:15.384 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:15.384 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T22:44:20.569791+0000", 2024-04-07T16:16:15.384 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:15.384 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:15.385 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:15.385 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:15.385 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:15.385 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:15.385 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:15.385 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:15.385 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:15.385 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:15.385 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:15.385 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:15.385 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:15.385 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:15.385 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:15.385 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:15.385 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:15.385 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:15.386 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:15.386 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:15.386 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:15.386 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:15.386 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:15.386 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:15.386 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:15.386 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:15.386 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:15.386 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:15.386 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:15.386 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:15.386 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:15.386 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:15.387 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:15.387 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:15.387 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:15.387 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:15.387 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:15.387 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:15.387 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:15.387 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:15.387 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:15.387 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:15.387 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:15.387 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.387 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:15.387 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:15.387 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:15.387 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T16:16:15.388 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T16:16:15.388 INFO:tasks.workunit.client.0.smithi100.stdout: 2 2024-04-07T16:16:15.388 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:15.388 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 1, 2024-04-07T16:16:15.388 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:15.388 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.388 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:15.388 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:15.388 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T16:16:15.388 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T16:16:15.388 INFO:tasks.workunit.client.0.smithi100.stdout: 4 2024-04-07T16:16:15.388 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:15.388 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 6, 2024-04-07T16:16:15.388 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:15.388 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:15.389 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T16:16:15.389 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:15.389 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:15.389 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:15.389 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.553144+0000", 2024-04-07T16:16:15.389 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.382267+0000", 2024-04-07T16:16:15.389 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.382267+0000", 2024-04-07T16:16:15.389 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.382605+0000", 2024-04-07T16:16:15.389 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.553144+0000", 2024-04-07T16:16:15.389 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:15.389 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:15.389 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:15.389 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 61, 2024-04-07T16:16:15.389 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.553144+0000", 2024-04-07T16:16:15.390 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.553144+0000", 2024-04-07T16:16:15.390 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.553144+0000", 2024-04-07T16:16:15.390 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:15.390 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:15.390 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:15.390 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.553144+0000", 2024-04-07T16:16:15.390 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.553144+0000", 2024-04-07T16:16:15.390 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:15.390 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:15.390 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:15.390 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:15.390 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 60, 2024-04-07T16:16:15.390 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:15.390 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:15.391 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:15.391 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:15.391 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:15.391 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:15.391 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:15.391 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:15.391 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.26", 2024-04-07T16:16:15.391 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:15.391 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:15.391 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:15.391 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 19, 2024-04-07T16:16:15.391 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:15.391 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T20:14:03.942753+0000", 2024-04-07T16:16:15.391 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:15.391 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:15.391 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:15.392 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:15.392 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:15.392 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:15.392 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:15.392 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:15.392 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:15.392 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:15.392 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:15.392 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:15.392 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:15.392 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:15.392 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:15.392 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:15.392 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:15.392 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:15.393 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:15.393 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:15.393 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:15.393 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:15.393 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:15.393 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:15.393 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:15.393 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:15.393 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:15.393 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:15.393 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:15.393 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:15.393 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:15.393 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:15.393 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:15.394 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:15.394 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:15.394 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:15.394 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:15.394 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:15.394 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:15.394 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:15.394 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:15.394 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:15.394 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:15.394 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.394 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:15.394 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:15.394 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:15.394 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T16:16:15.394 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T16:16:15.395 INFO:tasks.workunit.client.0.smithi100.stdout: 4 2024-04-07T16:16:15.395 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:15.395 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 6, 2024-04-07T16:16:15.395 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:15.395 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.395 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:15.395 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:15.395 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T16:16:15.395 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T16:16:15.395 INFO:tasks.workunit.client.0.smithi100.stdout: 2 2024-04-07T16:16:15.395 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:15.395 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 3, 2024-04-07T16:16:15.395 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:15.395 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:15.395 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T16:16:15.395 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:15.396 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:15.396 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:15.396 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.187449+0000", 2024-04-07T16:16:15.396 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.404734+0000", 2024-04-07T16:16:15.396 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.404734+0000", 2024-04-07T16:16:15.396 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.405175+0000", 2024-04-07T16:16:15.396 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.187449+0000", 2024-04-07T16:16:15.396 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:15.396 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:15.396 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:15.396 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 61, 2024-04-07T16:16:15.396 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.187449+0000", 2024-04-07T16:16:15.396 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.187449+0000", 2024-04-07T16:16:15.396 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.187449+0000", 2024-04-07T16:16:15.396 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:15.396 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:15.397 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:15.397 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.187449+0000", 2024-04-07T16:16:15.397 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.187449+0000", 2024-04-07T16:16:15.397 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:15.397 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:15.397 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:15.397 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:15.397 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 60, 2024-04-07T16:16:15.397 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:15.397 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:15.397 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:15.397 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:15.397 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:15.397 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:15.397 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:15.397 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:15.398 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.27", 2024-04-07T16:16:15.398 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:15.398 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:15.398 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:15.398 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 19, 2024-04-07T16:16:15.398 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:15.398 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T17:55:23.469795+0000", 2024-04-07T16:16:15.398 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:15.398 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:15.398 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:15.398 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:15.398 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:15.398 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:15.398 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:15.399 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:15.399 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:15.399 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:15.399 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:15.399 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:15.399 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:15.399 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:15.399 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:15.399 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:15.399 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:15.399 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:15.399 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:15.399 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:15.399 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:15.399 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:15.400 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:15.400 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:15.400 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:15.400 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:15.400 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:15.400 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:15.400 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:15.400 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:15.400 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:15.400 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:15.400 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:15.400 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:15.400 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:15.400 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:15.400 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:15.400 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:15.401 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:15.401 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:15.401 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:15.401 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:15.401 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:15.401 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.401 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:15.401 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:15.401 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:15.401 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T16:16:15.401 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T16:16:15.401 INFO:tasks.workunit.client.0.smithi100.stdout: 2 2024-04-07T16:16:15.401 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:15.401 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 3, 2024-04-07T16:16:15.401 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:15.401 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.402 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:15.402 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:15.402 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T16:16:15.402 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T16:16:15.402 INFO:tasks.workunit.client.0.smithi100.stdout: 2 2024-04-07T16:16:15.402 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:15.402 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 5, 2024-04-07T16:16:15.402 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:15.402 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:15.402 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T16:16:15.402 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:15.402 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:15.402 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:15.402 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.024146+0000", 2024-04-07T16:16:15.402 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.395430+0000", 2024-04-07T16:16:15.403 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.395430+0000", 2024-04-07T16:16:15.403 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.397863+0000", 2024-04-07T16:16:15.403 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.024146+0000", 2024-04-07T16:16:15.403 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:15.403 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:15.403 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:15.403 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 61, 2024-04-07T16:16:15.403 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.024146+0000", 2024-04-07T16:16:15.403 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.024146+0000", 2024-04-07T16:16:15.403 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.024146+0000", 2024-04-07T16:16:15.403 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:15.403 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:15.403 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:15.403 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.024146+0000", 2024-04-07T16:16:15.404 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.024146+0000", 2024-04-07T16:16:15.404 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:15.404 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:15.404 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:15.404 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:15.404 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 60, 2024-04-07T16:16:15.404 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:15.404 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:15.404 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:15.404 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:15.404 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:15.404 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:15.404 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:15.405 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:15.405 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.28", 2024-04-07T16:16:15.405 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:15.405 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:15.405 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:15.405 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 19, 2024-04-07T16:16:15.405 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:15.405 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T18:20:22.678341+0000", 2024-04-07T16:16:15.405 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:15.405 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:15.405 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:15.405 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:15.405 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:15.405 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:15.406 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:15.406 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:15.406 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:15.406 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:15.406 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:15.406 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:15.406 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:15.406 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:15.406 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:15.406 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:15.406 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:15.406 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:15.406 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:15.406 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:15.407 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:15.407 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:15.407 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:15.407 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:15.407 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:15.407 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:15.407 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:15.407 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:15.407 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:15.407 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:15.407 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:15.407 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:15.407 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:15.407 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:15.407 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:15.408 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:15.408 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:15.408 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:15.408 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:15.408 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:15.408 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:15.408 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:15.408 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:15.408 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.408 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:15.408 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:15.408 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:15.408 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T16:16:15.408 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T16:16:15.408 INFO:tasks.workunit.client.0.smithi100.stdout: 2 2024-04-07T16:16:15.409 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:15.409 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 5, 2024-04-07T16:16:15.409 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:15.409 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.409 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:15.409 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:15.409 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T16:16:15.409 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T16:16:15.409 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T16:16:15.409 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:15.409 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 3, 2024-04-07T16:16:15.409 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:15.409 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:15.409 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T16:16:15.409 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:15.410 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:15.410 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:15.410 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.186791+0000", 2024-04-07T16:16:15.410 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.396766+0000", 2024-04-07T16:16:15.410 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.396766+0000", 2024-04-07T16:16:15.410 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.397638+0000", 2024-04-07T16:16:15.410 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.186791+0000", 2024-04-07T16:16:15.410 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:15.410 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:15.410 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:15.410 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 61, 2024-04-07T16:16:15.410 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.186791+0000", 2024-04-07T16:16:15.410 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.186791+0000", 2024-04-07T16:16:15.411 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.186791+0000", 2024-04-07T16:16:15.411 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:15.411 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:15.411 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:15.411 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.186791+0000", 2024-04-07T16:16:15.411 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.186791+0000", 2024-04-07T16:16:15.411 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:15.411 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:15.411 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:15.411 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:15.411 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 60, 2024-04-07T16:16:15.411 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:15.411 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:15.411 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:15.412 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:15.412 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:15.412 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:15.412 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:15.412 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:15.412 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.29", 2024-04-07T16:16:15.412 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:15.412 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:15.412 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:15.412 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 19, 2024-04-07T16:16:15.412 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:15.412 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T22:51:06.841718+0000", 2024-04-07T16:16:15.412 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:15.412 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:15.412 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:15.413 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:15.413 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:15.413 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:15.413 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:15.413 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:15.413 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:15.413 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:15.413 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:15.413 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:15.413 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:15.413 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:15.413 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:15.413 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:15.413 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:15.413 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:15.414 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:15.414 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:15.414 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:15.414 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:15.414 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:15.414 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:15.414 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:15.414 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:15.414 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:15.414 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:15.414 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:15.414 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:15.414 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:15.414 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:15.415 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:15.415 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:15.415 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:15.415 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:15.415 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:15.415 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:15.415 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:15.415 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:15.415 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:15.415 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:15.415 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:15.415 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.415 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:15.415 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:15.415 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:15.416 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T16:16:15.416 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T16:16:15.416 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T16:16:15.416 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:15.416 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 3, 2024-04-07T16:16:15.416 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:15.416 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.416 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:15.416 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:15.416 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T16:16:15.416 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T16:16:15.416 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T16:16:15.416 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:15.416 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 5, 2024-04-07T16:16:15.416 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:15.417 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:15.417 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T16:16:15.417 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:15.417 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:15.417 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:15.417 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.024630+0000", 2024-04-07T16:16:15.417 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.400042+0000", 2024-04-07T16:16:15.417 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.400042+0000", 2024-04-07T16:16:15.417 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.403603+0000", 2024-04-07T16:16:15.417 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.024630+0000", 2024-04-07T16:16:15.417 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:15.417 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:15.417 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:15.417 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 61, 2024-04-07T16:16:15.417 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.024630+0000", 2024-04-07T16:16:15.418 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.024630+0000", 2024-04-07T16:16:15.418 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.024630+0000", 2024-04-07T16:16:15.418 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:15.418 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:15.418 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:15.418 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.024630+0000", 2024-04-07T16:16:15.418 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.024630+0000", 2024-04-07T16:16:15.418 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:15.418 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:15.418 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:15.418 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:15.418 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 60, 2024-04-07T16:16:15.418 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:15.418 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:15.418 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:15.419 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:15.419 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:15.419 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:15.419 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:15.419 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:15.419 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.2a", 2024-04-07T16:16:15.419 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:15.419 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:15.419 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:15.419 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 19, 2024-04-07T16:16:15.419 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:15.419 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T20:49:17.570678+0000", 2024-04-07T16:16:15.419 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:15.419 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:15.419 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:15.420 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:15.420 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:15.420 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:15.420 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:15.420 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:15.420 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:15.420 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:15.420 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:15.420 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:15.420 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:15.420 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:15.420 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:15.420 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:15.420 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:15.420 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:15.420 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:15.421 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:15.421 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:15.421 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:15.421 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:15.421 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:15.421 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:15.421 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:15.421 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:15.421 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:15.421 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:15.421 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:15.421 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:15.421 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:15.421 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:15.422 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:15.422 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:15.422 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:15.422 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:15.422 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:15.422 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:15.422 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:15.422 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:15.422 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:15.422 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:15.422 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.422 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:15.422 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:15.422 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:15.422 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T16:16:15.423 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T16:16:15.423 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T16:16:15.423 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:15.423 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 5, 2024-04-07T16:16:15.423 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:15.423 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.423 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:15.423 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:15.423 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T16:16:15.423 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T16:16:15.423 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T16:16:15.423 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:15.423 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 0, 2024-04-07T16:16:15.423 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:15.423 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:15.423 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T16:16:15.424 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:15.424 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:15.424 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:15.424 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.029710+0000", 2024-04-07T16:16:15.424 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.386422+0000", 2024-04-07T16:16:15.424 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.386422+0000", 2024-04-07T16:16:15.424 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.386718+0000", 2024-04-07T16:16:15.424 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.029710+0000", 2024-04-07T16:16:15.424 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:15.424 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:15.424 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:15.424 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 61, 2024-04-07T16:16:15.424 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.029710+0000", 2024-04-07T16:16:15.424 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.029710+0000", 2024-04-07T16:16:15.424 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.029710+0000", 2024-04-07T16:16:15.425 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:15.425 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:15.425 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:15.425 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.029710+0000", 2024-04-07T16:16:15.425 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.029710+0000", 2024-04-07T16:16:15.425 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:15.425 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:15.425 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:15.425 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:15.425 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 60, 2024-04-07T16:16:15.425 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:15.425 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:15.425 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:15.425 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:15.425 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:15.426 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:15.426 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:15.426 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:15.426 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.2b", 2024-04-07T16:16:15.426 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:15.426 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:15.426 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:15.426 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 19, 2024-04-07T16:16:15.426 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:15.426 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T21:28:09.174468+0000", 2024-04-07T16:16:15.426 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:15.426 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:15.426 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:15.426 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:15.426 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:15.426 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:15.427 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:15.427 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:15.427 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:15.427 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:15.427 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:15.427 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:15.427 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:15.427 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:15.427 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:15.427 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:15.427 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:15.427 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:15.427 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:15.427 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:15.428 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:15.428 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:15.428 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:15.428 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:15.428 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:15.428 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:15.428 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:15.428 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:15.428 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:15.428 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:15.428 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:15.428 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:15.428 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:15.428 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:15.429 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:15.429 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:15.429 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:15.429 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:15.429 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:15.429 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:15.429 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:15.429 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:15.429 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:15.429 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.429 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:15.429 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:15.429 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:15.429 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T16:16:15.430 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T16:16:15.430 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T16:16:15.430 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:15.430 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 0, 2024-04-07T16:16:15.430 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:15.430 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.430 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:15.430 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:15.430 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T16:16:15.430 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T16:16:15.430 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T16:16:15.430 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:15.430 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 6, 2024-04-07T16:16:15.430 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:15.430 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:15.431 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T16:16:15.431 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:15.431 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:15.431 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:15.431 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.526600+0000", 2024-04-07T16:16:15.431 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.379758+0000", 2024-04-07T16:16:15.431 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.379758+0000", 2024-04-07T16:16:15.431 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.380077+0000", 2024-04-07T16:16:15.431 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.526600+0000", 2024-04-07T16:16:15.431 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:15.431 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:15.431 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:15.431 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 61, 2024-04-07T16:16:15.431 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.526600+0000", 2024-04-07T16:16:15.431 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.526600+0000", 2024-04-07T16:16:15.432 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.526600+0000", 2024-04-07T16:16:15.432 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:15.432 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:15.432 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:15.432 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.526600+0000", 2024-04-07T16:16:15.432 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.526600+0000", 2024-04-07T16:16:15.432 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:15.432 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:15.432 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:15.432 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:15.432 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 60, 2024-04-07T16:16:15.432 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:15.432 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:15.432 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:15.432 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:15.433 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:15.433 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:15.433 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:15.433 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:15.433 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.2c", 2024-04-07T16:16:15.433 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:15.433 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:15.433 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:15.433 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 19, 2024-04-07T16:16:15.433 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:15.433 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T22:47:34.836189+0000", 2024-04-07T16:16:15.433 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:15.433 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:15.433 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:15.433 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:15.434 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:15.434 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:15.434 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:15.434 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:15.434 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:15.434 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:15.434 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:15.434 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:15.434 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:15.434 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:15.434 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:15.434 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:15.434 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:15.434 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:15.434 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:15.435 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:15.435 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:15.435 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:15.435 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:15.435 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:15.435 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:15.435 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:15.435 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:15.435 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:15.435 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:15.435 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:15.435 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:15.435 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:15.435 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:15.435 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:15.435 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:15.436 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:15.436 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:15.436 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:15.436 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:15.436 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:15.436 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:15.436 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:15.436 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:15.436 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.436 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:15.436 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:15.436 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:15.436 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T16:16:15.436 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T16:16:15.436 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T16:16:15.437 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:15.437 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 6, 2024-04-07T16:16:15.437 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:15.437 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:15.437 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:15.437 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_per_pool": [ 2024-04-07T16:16:15.437 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:15.437 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": 24, 2024-04-07T16:16:15.437 INFO:tasks.workunit.client.0.smithi100.stdout: "application": [ 2024-04-07T16:16:15.437 INFO:tasks.workunit.client.0.smithi100.stdout: "rbd" 2024-04-07T16:16:15.437 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:15.437 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:15.437 INFO:tasks.workunit.client.0.smithi100.stdout: "num_pg": 8, 2024-04-07T16:16:15.437 INFO:tasks.workunit.client.0.smithi100.stdout: "num_store_stats": 8, 2024-04-07T16:16:15.437 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:15.438 INFO:tasks.workunit.client.0.smithi100.stdout: "poolid": 4, 2024-04-07T16:16:15.438 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:15.438 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:15.438 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:15.438 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:15.438 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:15.438 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:15.438 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:15.438 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:15.438 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:15.438 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:15.438 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:15.438 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:15.438 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:15.439 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:15.439 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:15.439 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:15.439 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:15.439 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:15.439 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:15.439 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:15.439 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:15.439 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:15.439 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:15.439 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:15.439 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:15.439 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:15.439 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:15.439 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:15.440 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:15.440 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:15.440 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:15.440 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:15.440 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:15.440 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:15.440 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:15.440 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:15.440 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:15.440 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:15.440 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:15.440 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:15.440 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:15.440 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.440 INFO:tasks.workunit.client.0.smithi100.stdout: "store_stats": { 2024-04-07T16:16:15.440 INFO:tasks.workunit.client.0.smithi100.stdout: "allocated": 0, 2024-04-07T16:16:15.441 INFO:tasks.workunit.client.0.smithi100.stdout: "available": 0, 2024-04-07T16:16:15.441 INFO:tasks.workunit.client.0.smithi100.stdout: "data_compressed": 0, 2024-04-07T16:16:15.441 INFO:tasks.workunit.client.0.smithi100.stdout: "data_compressed_allocated": 0, 2024-04-07T16:16:15.441 INFO:tasks.workunit.client.0.smithi100.stdout: "data_compressed_original": 0, 2024-04-07T16:16:15.441 INFO:tasks.workunit.client.0.smithi100.stdout: "data_stored": 0, 2024-04-07T16:16:15.441 INFO:tasks.workunit.client.0.smithi100.stdout: "internal_metadata": 0, 2024-04-07T16:16:15.441 INFO:tasks.workunit.client.0.smithi100.stdout: "internally_reserved": 0, 2024-04-07T16:16:15.441 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_allocated": 0, 2024-04-07T16:16:15.441 INFO:tasks.workunit.client.0.smithi100.stdout: "total": 0 2024-04-07T16:16:15.441 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.441 INFO:tasks.workunit.client.0.smithi100.stdout: "up": 24 2024-04-07T16:16:15.441 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.441 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:15.441 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": 192, 2024-04-07T16:16:15.442 INFO:tasks.workunit.client.0.smithi100.stdout: "application": [ 2024-04-07T16:16:15.442 INFO:tasks.workunit.client.0.smithi100.stdout: "cephfs" 2024-04-07T16:16:15.442 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:15.442 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:15.442 INFO:tasks.workunit.client.0.smithi100.stdout: "num_pg": 64, 2024-04-07T16:16:15.442 INFO:tasks.workunit.client.0.smithi100.stdout: "num_store_stats": 8, 2024-04-07T16:16:15.442 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:15.442 INFO:tasks.workunit.client.0.smithi100.stdout: "poolid": 3, 2024-04-07T16:16:15.442 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:15.442 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:15.442 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:15.442 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:15.442 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:15.442 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:15.442 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:15.443 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:15.443 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:15.443 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:15.443 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:15.443 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:15.443 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:15.443 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:15.443 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:15.443 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:15.443 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:15.443 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:15.443 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:15.443 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:15.443 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:15.444 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:15.444 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:15.444 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:15.444 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:15.444 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:15.444 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:15.444 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:15.444 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:15.444 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:15.444 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:15.444 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:15.444 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:15.444 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:15.445 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:15.445 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:15.445 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:15.445 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:15.445 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:15.445 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:15.445 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:15.445 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.445 INFO:tasks.workunit.client.0.smithi100.stdout: "store_stats": { 2024-04-07T16:16:15.445 INFO:tasks.workunit.client.0.smithi100.stdout: "allocated": 0, 2024-04-07T16:16:15.445 INFO:tasks.workunit.client.0.smithi100.stdout: "available": 0, 2024-04-07T16:16:15.445 INFO:tasks.workunit.client.0.smithi100.stdout: "data_compressed": 0, 2024-04-07T16:16:15.445 INFO:tasks.workunit.client.0.smithi100.stdout: "data_compressed_allocated": 0, 2024-04-07T16:16:15.445 INFO:tasks.workunit.client.0.smithi100.stdout: "data_compressed_original": 0, 2024-04-07T16:16:15.446 INFO:tasks.workunit.client.0.smithi100.stdout: "data_stored": 0, 2024-04-07T16:16:15.446 INFO:tasks.workunit.client.0.smithi100.stdout: "internal_metadata": 0, 2024-04-07T16:16:15.446 INFO:tasks.workunit.client.0.smithi100.stdout: "internally_reserved": 0, 2024-04-07T16:16:15.446 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_allocated": 0, 2024-04-07T16:16:15.446 INFO:tasks.workunit.client.0.smithi100.stdout: "total": 0 2024-04-07T16:16:15.446 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.446 INFO:tasks.workunit.client.0.smithi100.stdout: "up": 192 2024-04-07T16:16:15.446 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.446 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:15.446 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": 3, 2024-04-07T16:16:15.446 INFO:tasks.workunit.client.0.smithi100.stdout: "application": [ 2024-04-07T16:16:15.446 INFO:tasks.workunit.client.0.smithi100.stdout: "mgr" 2024-04-07T16:16:15.446 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:15.446 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 128, 2024-04-07T16:16:15.447 INFO:tasks.workunit.client.0.smithi100.stdout: "num_pg": 1, 2024-04-07T16:16:15.447 INFO:tasks.workunit.client.0.smithi100.stdout: "num_store_stats": 3, 2024-04-07T16:16:15.447 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 128, 2024-04-07T16:16:15.447 INFO:tasks.workunit.client.0.smithi100.stdout: "poolid": 1, 2024-04-07T16:16:15.447 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:15.447 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 590368, 2024-04-07T16:16:15.447 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:15.447 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 1771104, 2024-04-07T16:16:15.447 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:15.447 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:15.447 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:15.447 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:15.447 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:15.447 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:15.447 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:15.448 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:15.448 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:15.448 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:15.448 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:15.448 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:15.448 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:15.448 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 6, 2024-04-07T16:16:15.448 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 2, 2024-04-07T16:16:15.448 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:15.448 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 2, 2024-04-07T16:16:15.448 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:15.448 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:15.448 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:15.448 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:15.448 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:15.449 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:15.449 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:15.449 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 6, 2024-04-07T16:16:15.449 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:15.449 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:15.449 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:15.449 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:15.449 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:15.449 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 212, 2024-04-07T16:16:15.449 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 306, 2024-04-07T16:16:15.449 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:15.449 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:15.449 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:15.449 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 179, 2024-04-07T16:16:15.449 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 2416 2024-04-07T16:16:15.450 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.450 INFO:tasks.workunit.client.0.smithi100.stdout: "store_stats": { 2024-04-07T16:16:15.450 INFO:tasks.workunit.client.0.smithi100.stdout: "allocated": 1781760, 2024-04-07T16:16:15.450 INFO:tasks.workunit.client.0.smithi100.stdout: "available": 0, 2024-04-07T16:16:15.450 INFO:tasks.workunit.client.0.smithi100.stdout: "data_compressed": 0, 2024-04-07T16:16:15.450 INFO:tasks.workunit.client.0.smithi100.stdout: "data_compressed_allocated": 0, 2024-04-07T16:16:15.450 INFO:tasks.workunit.client.0.smithi100.stdout: "data_compressed_original": 0, 2024-04-07T16:16:15.450 INFO:tasks.workunit.client.0.smithi100.stdout: "data_stored": 1771104, 2024-04-07T16:16:15.450 INFO:tasks.workunit.client.0.smithi100.stdout: "internal_metadata": 0, 2024-04-07T16:16:15.450 INFO:tasks.workunit.client.0.smithi100.stdout: "internally_reserved": 0, 2024-04-07T16:16:15.450 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_allocated": 0, 2024-04-07T16:16:15.450 INFO:tasks.workunit.client.0.smithi100.stdout: "total": 0 2024-04-07T16:16:15.450 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.450 INFO:tasks.workunit.client.0.smithi100.stdout: "up": 3 2024-04-07T16:16:15.450 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.451 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:15.451 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": 96, 2024-04-07T16:16:15.451 INFO:tasks.workunit.client.0.smithi100.stdout: "application": [ 2024-04-07T16:16:15.451 INFO:tasks.workunit.client.0.smithi100.stdout: "cephfs" 2024-04-07T16:16:15.451 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:15.451 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 45, 2024-04-07T16:16:15.451 INFO:tasks.workunit.client.0.smithi100.stdout: "num_pg": 32, 2024-04-07T16:16:15.451 INFO:tasks.workunit.client.0.smithi100.stdout: "num_store_stats": 8, 2024-04-07T16:16:15.451 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 45, 2024-04-07T16:16:15.451 INFO:tasks.workunit.client.0.smithi100.stdout: "poolid": 2, 2024-04-07T16:16:15.451 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:15.451 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 3430, 2024-04-07T16:16:15.451 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:15.451 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:15.451 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:15.452 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:15.452 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:15.452 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:15.452 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:15.452 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:15.452 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:15.452 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:15.452 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:15.452 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:15.452 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:15.452 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:15.452 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:15.452 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 66, 2024-04-07T16:16:15.452 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 22, 2024-04-07T16:16:15.452 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:15.453 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 22, 2024-04-07T16:16:15.453 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:15.453 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:15.453 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:15.453 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:15.453 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:15.453 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 14, 2024-04-07T16:16:15.453 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:15.453 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 1, 2024-04-07T16:16:15.453 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:15.453 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:15.453 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:15.453 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:15.453 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:15.454 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:15.454 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:15.454 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:15.454 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:15.454 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:15.454 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 51, 2024-04-07T16:16:15.454 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 20 2024-04-07T16:16:15.454 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.454 INFO:tasks.workunit.client.0.smithi100.stdout: "store_stats": { 2024-04-07T16:16:15.454 INFO:tasks.workunit.client.0.smithi100.stdout: "allocated": 98304, 2024-04-07T16:16:15.454 INFO:tasks.workunit.client.0.smithi100.stdout: "available": 0, 2024-04-07T16:16:15.454 INFO:tasks.workunit.client.0.smithi100.stdout: "data_compressed": 0, 2024-04-07T16:16:15.454 INFO:tasks.workunit.client.0.smithi100.stdout: "data_compressed_allocated": 0, 2024-04-07T16:16:15.454 INFO:tasks.workunit.client.0.smithi100.stdout: "data_compressed_original": 0, 2024-04-07T16:16:15.454 INFO:tasks.workunit.client.0.smithi100.stdout: "data_stored": 10290, 2024-04-07T16:16:15.455 INFO:tasks.workunit.client.0.smithi100.stdout: "internal_metadata": 0, 2024-04-07T16:16:15.455 INFO:tasks.workunit.client.0.smithi100.stdout: "internally_reserved": 0, 2024-04-07T16:16:15.455 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_allocated": 0, 2024-04-07T16:16:15.455 INFO:tasks.workunit.client.0.smithi100.stdout: "total": 0 2024-04-07T16:16:15.455 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.455 INFO:tasks.workunit.client.0.smithi100.stdout: "up": 96 2024-04-07T16:16:15.455 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:15.455 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:15.455 INFO:tasks.workunit.client.0.smithi100.stdout: "usage": { 2024-04-07T16:16:15.455 INFO:tasks.workunit.client.0.smithi100.stdout: "pg_num": 105, 2024-04-07T16:16:15.455 INFO:tasks.workunit.client.0.smithi100.stdout: "pools": 4, 2024-04-07T16:16:15.455 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_by_class": { 2024-04-07T16:16:15.455 INFO:tasks.workunit.client.0.smithi100.stdout: "ssd": { 2024-04-07T16:16:15.455 INFO:tasks.workunit.client.0.smithi100.stdout: "total_avail_bytes": 767718821888, 2024-04-07T16:16:15.455 INFO:tasks.workunit.client.0.smithi100.stdout: "total_bytes": 767960285184, 2024-04-07T16:16:15.456 INFO:tasks.workunit.client.0.smithi100.stdout: "total_used_bytes": 241463296, 2024-04-07T16:16:15.456 INFO:tasks.workunit.client.0.smithi100.stdout: "total_used_raw_bytes": 241463296, 2024-04-07T16:16:15.456 INFO:tasks.workunit.client.0.smithi100.stdout: "total_used_raw_ratio": 0.000314421602524817 2024-04-07T16:16:15.456 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:15.456 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.456 INFO:tasks.workunit.client.0.smithi100.stdout: "total_avail_bytes": 767718821888, 2024-04-07T16:16:15.456 INFO:tasks.workunit.client.0.smithi100.stdout: "total_bytes": 767960285184, 2024-04-07T16:16:15.456 INFO:tasks.workunit.client.0.smithi100.stdout: "total_used_bytes": 241463296 2024-04-07T16:16:15.456 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:15.456 INFO:tasks.workunit.client.0.smithi100.stdout:} 2024-04-07T16:16:15.456 INFO:tasks.workunit.client.0.smithi100.stdout:Device report is generated separately. To see it run 'ceph telemetry show-device'. 2024-04-07T16:16:15.457 INFO:tasks.workunit.client.0.smithi100.stderr:+ ceph telemetry show-device 2024-04-07T16:16:15.457 INFO:tasks.workunit.client.0.smithi100.stdout:{ 2024-04-07T16:16:15.457 INFO:tasks.workunit.client.0.smithi100.stdout: "251e7b66-f4fa-11ee-8447-0cc47a8fd733": { 2024-04-07T16:16:15.458 INFO:tasks.workunit.client.0.smithi100.stdout: "INTEL_SSDPEDMD400G4_251ffd38-f4fa-11ee-8447-0cc47a8fd733": { 2024-04-07T16:16:15.458 INFO:tasks.workunit.client.0.smithi100.stdout: "20240407-160729": { 2024-04-07T16:16:15.458 INFO:tasks.workunit.client.0.smithi100.stdout: "device": { 2024-04-07T16:16:15.458 INFO:tasks.workunit.client.0.smithi100.stdout: "info_name": "/dev/nvme0n1", 2024-04-07T16:16:15.458 INFO:tasks.workunit.client.0.smithi100.stdout: "name": "/dev/nvme0n1", 2024-04-07T16:16:15.458 INFO:tasks.workunit.client.0.smithi100.stdout: "protocol": "NVMe", 2024-04-07T16:16:15.458 INFO:tasks.workunit.client.0.smithi100.stdout: "type": "nvme" 2024-04-07T16:16:15.458 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.458 INFO:tasks.workunit.client.0.smithi100.stdout: "firmware_version": "8DV101H0", 2024-04-07T16:16:15.458 INFO:tasks.workunit.client.0.smithi100.stdout: "host_id": "251e7b66-f4fa-11ee-8447-0cc47a8fd733", 2024-04-07T16:16:15.458 INFO:tasks.workunit.client.0.smithi100.stdout: "json_format_version": [ 2024-04-07T16:16:15.458 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T16:16:15.458 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T16:16:15.458 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:15.459 INFO:tasks.workunit.client.0.smithi100.stdout: "local_time": { 2024-04-07T16:16:15.459 INFO:tasks.workunit.client.0.smithi100.stdout: "asctime": "Sun Apr 7 16:07:28 2024 UTC", 2024-04-07T16:16:15.459 INFO:tasks.workunit.client.0.smithi100.stdout: "time_t": 1712506048 2024-04-07T16:16:15.459 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.459 INFO:tasks.workunit.client.0.smithi100.stdout: "logical_block_size": 512, 2024-04-07T16:16:15.459 INFO:tasks.workunit.client.0.smithi100.stdout: "model_name": "INTEL SSDPEDMD400G4", 2024-04-07T16:16:15.459 INFO:tasks.workunit.client.0.smithi100.stdout: "nvme_controller_id": 0, 2024-04-07T16:16:15.459 INFO:tasks.workunit.client.0.smithi100.stdout: "nvme_ieee_oui_identifier": 6083300, 2024-04-07T16:16:15.459 INFO:tasks.workunit.client.0.smithi100.stdout: "nvme_namespaces": [ 2024-04-07T16:16:15.459 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:15.459 INFO:tasks.workunit.client.0.smithi100.stdout: "capacity": { 2024-04-07T16:16:15.459 INFO:tasks.workunit.client.0.smithi100.stdout: "blocks": 781422768, 2024-04-07T16:16:15.459 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 400088457216 2024-04-07T16:16:15.459 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.459 INFO:tasks.workunit.client.0.smithi100.stdout: "formatted_lba_size": 512, 2024-04-07T16:16:15.460 INFO:tasks.workunit.client.0.smithi100.stdout: "id": 1, 2024-04-07T16:16:15.460 INFO:tasks.workunit.client.0.smithi100.stdout: "size": { 2024-04-07T16:16:15.460 INFO:tasks.workunit.client.0.smithi100.stdout: "blocks": 781422768, 2024-04-07T16:16:15.460 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 400088457216 2024-04-07T16:16:15.460 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.460 INFO:tasks.workunit.client.0.smithi100.stdout: "utilization": { 2024-04-07T16:16:15.460 INFO:tasks.workunit.client.0.smithi100.stdout: "blocks": 781422768, 2024-04-07T16:16:15.460 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 400088457216 2024-04-07T16:16:15.460 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:15.460 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:15.460 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:15.460 INFO:tasks.workunit.client.0.smithi100.stdout: "nvme_number_of_namespaces": 1, 2024-04-07T16:16:15.460 INFO:tasks.workunit.client.0.smithi100.stdout: "nvme_pci_vendor": { 2024-04-07T16:16:15.460 INFO:tasks.workunit.client.0.smithi100.stdout: "id": 32902, 2024-04-07T16:16:15.460 INFO:tasks.workunit.client.0.smithi100.stdout: "subsystem_id": 32902 2024-04-07T16:16:15.460 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.461 INFO:tasks.workunit.client.0.smithi100.stdout: "nvme_smart_health_information_add_log": { 2024-04-07T16:16:15.461 INFO:tasks.workunit.client.0.smithi100.stdout: "Device stats": { 2024-04-07T16:16:15.461 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_fw_downgrades": { 2024-04-07T16:16:15.461 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 0, 2024-04-07T16:16:15.461 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 0 2024-04-07T16:16:15.461 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.461 INFO:tasks.workunit.client.0.smithi100.stdout: "crc_error_count": { 2024-04-07T16:16:15.461 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 100, 2024-04-07T16:16:15.461 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 0 2024-04-07T16:16:15.461 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.461 INFO:tasks.workunit.client.0.smithi100.stdout: "end_to_end_error_detection_count": { 2024-04-07T16:16:15.461 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 100, 2024-04-07T16:16:15.461 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 0 2024-04-07T16:16:15.461 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.462 INFO:tasks.workunit.client.0.smithi100.stdout: "erase_fail_count": { 2024-04-07T16:16:15.462 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 100, 2024-04-07T16:16:15.462 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 0 2024-04-07T16:16:15.462 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.462 INFO:tasks.workunit.client.0.smithi100.stdout: "host_bytes_written": { 2024-04-07T16:16:15.462 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 100, 2024-04-07T16:16:15.462 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 29568720 2024-04-07T16:16:15.462 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.462 INFO:tasks.workunit.client.0.smithi100.stdout: "host_ctx_wear_used": { 2024-04-07T16:16:15.462 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 100, 2024-04-07T16:16:15.462 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 1 2024-04-07T16:16:15.462 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.462 INFO:tasks.workunit.client.0.smithi100.stdout: "media_bytes_read": { 2024-04-07T16:16:15.462 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 0, 2024-04-07T16:16:15.462 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 0 2024-04-07T16:16:15.462 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.463 INFO:tasks.workunit.client.0.smithi100.stdout: "nand_bytes_written": { 2024-04-07T16:16:15.463 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 100, 2024-04-07T16:16:15.463 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 47802746 2024-04-07T16:16:15.463 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.463 INFO:tasks.workunit.client.0.smithi100.stdout: "perf_stat_indicator": { 2024-04-07T16:16:15.463 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 0, 2024-04-07T16:16:15.463 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 0 2024-04-07T16:16:15.463 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.463 INFO:tasks.workunit.client.0.smithi100.stdout: "pll_lock_loss_count": { 2024-04-07T16:16:15.463 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 100, 2024-04-07T16:16:15.463 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 0 2024-04-07T16:16:15.463 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.463 INFO:tasks.workunit.client.0.smithi100.stdout: "program_fail_count": { 2024-04-07T16:16:15.463 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 100, 2024-04-07T16:16:15.463 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 0 2024-04-07T16:16:15.464 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.464 INFO:tasks.workunit.client.0.smithi100.stdout: "re_alloc_sectr_cnt": { 2024-04-07T16:16:15.464 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 0, 2024-04-07T16:16:15.464 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 0 2024-04-07T16:16:15.464 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.464 INFO:tasks.workunit.client.0.smithi100.stdout: "retry_buffer_overflow_count": { 2024-04-07T16:16:15.464 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 100, 2024-04-07T16:16:15.464 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 0 2024-04-07T16:16:15.464 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.464 INFO:tasks.workunit.client.0.smithi100.stdout: "soft_ecc_err_rate": { 2024-04-07T16:16:15.464 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 0, 2024-04-07T16:16:15.464 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 0 2024-04-07T16:16:15.464 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.464 INFO:tasks.workunit.client.0.smithi100.stdout: "thermal_throttle_status": { 2024-04-07T16:16:15.464 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 100, 2024-04-07T16:16:15.464 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": { 2024-04-07T16:16:15.465 INFO:tasks.workunit.client.0.smithi100.stdout: "cnt": 0, 2024-04-07T16:16:15.465 INFO:tasks.workunit.client.0.smithi100.stdout: "pct": 0 2024-04-07T16:16:15.465 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:15.465 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.465 INFO:tasks.workunit.client.0.smithi100.stdout: "timed_workload_host_reads": { 2024-04-07T16:16:15.465 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 100, 2024-04-07T16:16:15.465 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 65535 2024-04-07T16:16:15.465 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.465 INFO:tasks.workunit.client.0.smithi100.stdout: "timed_workload_media_wear": { 2024-04-07T16:16:15.465 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 100, 2024-04-07T16:16:15.465 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 64 2024-04-07T16:16:15.465 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.465 INFO:tasks.workunit.client.0.smithi100.stdout: "timed_workload_timer": { 2024-04-07T16:16:15.465 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 100, 2024-04-07T16:16:15.465 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 65535 2024-04-07T16:16:15.465 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.466 INFO:tasks.workunit.client.0.smithi100.stdout: "unexp_power_loss": { 2024-04-07T16:16:15.466 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 0, 2024-04-07T16:16:15.466 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 0 2024-04-07T16:16:15.466 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.466 INFO:tasks.workunit.client.0.smithi100.stdout: "wear_leveling": { 2024-04-07T16:16:15.466 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 90, 2024-04-07T16:16:15.466 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": { 2024-04-07T16:16:15.466 INFO:tasks.workunit.client.0.smithi100.stdout: "avg": 3209, 2024-04-07T16:16:15.466 INFO:tasks.workunit.client.0.smithi100.stdout: "max": 3225, 2024-04-07T16:16:15.466 INFO:tasks.workunit.client.0.smithi100.stdout: "min": 3194 2024-04-07T16:16:15.466 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:15.466 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:15.466 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.466 INFO:tasks.workunit.client.0.smithi100.stdout: "Intel Smart log": "nvme0n1" 2024-04-07T16:16:15.466 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.467 INFO:tasks.workunit.client.0.smithi100.stdout: "nvme_smart_health_information_log": { 2024-04-07T16:16:15.467 INFO:tasks.workunit.client.0.smithi100.stdout: "available_spare": 100, 2024-04-07T16:16:15.467 INFO:tasks.workunit.client.0.smithi100.stdout: "available_spare_threshold": 10, 2024-04-07T16:16:15.467 INFO:tasks.workunit.client.0.smithi100.stdout: "controller_busy_time": 4, 2024-04-07T16:16:15.467 INFO:tasks.workunit.client.0.smithi100.stdout: "critical_warning": 0, 2024-04-07T16:16:15.467 INFO:tasks.workunit.client.0.smithi100.stdout: "data_units_read": 385784931, 2024-04-07T16:16:15.467 INFO:tasks.workunit.client.0.smithi100.stdout: "data_units_written": 1937815661, 2024-04-07T16:16:15.467 INFO:tasks.workunit.client.0.smithi100.stdout: "host_reads": 5771442418, 2024-04-07T16:16:15.467 INFO:tasks.workunit.client.0.smithi100.stdout: "host_writes": 31722089956, 2024-04-07T16:16:15.467 INFO:tasks.workunit.client.0.smithi100.stdout: "media_errors": 0, 2024-04-07T16:16:15.467 INFO:tasks.workunit.client.0.smithi100.stdout: "num_err_log_entries": 0, 2024-04-07T16:16:15.467 INFO:tasks.workunit.client.0.smithi100.stdout: "percentage_used": 10, 2024-04-07T16:16:15.467 INFO:tasks.workunit.client.0.smithi100.stdout: "power_cycles": 45774, 2024-04-07T16:16:15.467 INFO:tasks.workunit.client.0.smithi100.stdout: "power_on_hours": 57978, 2024-04-07T16:16:15.467 INFO:tasks.workunit.client.0.smithi100.stdout: "temperature": 24, 2024-04-07T16:16:15.467 INFO:tasks.workunit.client.0.smithi100.stdout: "unsafe_shutdowns": 210 2024-04-07T16:16:15.468 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.468 INFO:tasks.workunit.client.0.smithi100.stdout: "nvme_vendor": "intel", 2024-04-07T16:16:15.468 INFO:tasks.workunit.client.0.smithi100.stdout: "power_cycle_count": 45774, 2024-04-07T16:16:15.468 INFO:tasks.workunit.client.0.smithi100.stdout: "power_on_time": { 2024-04-07T16:16:15.468 INFO:tasks.workunit.client.0.smithi100.stdout: "hours": 57978 2024-04-07T16:16:15.468 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.468 INFO:tasks.workunit.client.0.smithi100.stdout: "serial_number": "deleted", 2024-04-07T16:16:15.468 INFO:tasks.workunit.client.0.smithi100.stdout: "smart_status": { 2024-04-07T16:16:15.468 INFO:tasks.workunit.client.0.smithi100.stdout: "nvme": { 2024-04-07T16:16:15.468 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:15.468 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.468 INFO:tasks.workunit.client.0.smithi100.stdout: "passed": true 2024-04-07T16:16:15.468 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.468 INFO:tasks.workunit.client.0.smithi100.stdout: "smartctl": { 2024-04-07T16:16:15.468 INFO:tasks.workunit.client.0.smithi100.stdout: "argv": [ 2024-04-07T16:16:15.468 INFO:tasks.workunit.client.0.smithi100.stdout: "smartctl", 2024-04-07T16:16:15.469 INFO:tasks.workunit.client.0.smithi100.stdout: "-x", 2024-04-07T16:16:15.469 INFO:tasks.workunit.client.0.smithi100.stdout: "--json=o", 2024-04-07T16:16:15.469 INFO:tasks.workunit.client.0.smithi100.stdout: "/dev/nvme0n1" 2024-04-07T16:16:15.469 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:15.469 INFO:tasks.workunit.client.0.smithi100.stdout: "build_info": "(local build)", 2024-04-07T16:16:15.469 INFO:tasks.workunit.client.0.smithi100.stdout: "exit_status": 0, 2024-04-07T16:16:15.469 INFO:tasks.workunit.client.0.smithi100.stdout: "output": [ 2024-04-07T16:16:15.469 INFO:tasks.workunit.client.0.smithi100.stdout: "smartctl 7.1 2019-12-30 r5022 [x86_64-linux-5.15.0-101-generic] (local build)", 2024-04-07T16:16:15.469 INFO:tasks.workunit.client.0.smithi100.stdout: "Copyright (C) 2002-19, Bruce Allen, Christian Franke, www.smartmontools.org", 2024-04-07T16:16:15.469 INFO:tasks.workunit.client.0.smithi100.stdout: "", 2024-04-07T16:16:15.469 INFO:tasks.workunit.client.0.smithi100.stdout: "=== START OF INFORMATION SECTION ===", 2024-04-07T16:16:15.469 INFO:tasks.workunit.client.0.smithi100.stdout: "Model Number: INTEL SSDPEDMD400G4", 2024-04-07T16:16:15.469 INFO:tasks.workunit.client.0.smithi100.stdout: "Serial Number: deleted", 2024-04-07T16:16:15.469 INFO:tasks.workunit.client.0.smithi100.stdout: "Firmware Version: 8DV101H0", 2024-04-07T16:16:15.469 INFO:tasks.workunit.client.0.smithi100.stdout: "PCI Vendor/Subsystem ID: 0x8086", 2024-04-07T16:16:15.469 INFO:tasks.workunit.client.0.smithi100.stdout: "IEEE OUI Identifier: 0x5cd2e4", 2024-04-07T16:16:15.470 INFO:tasks.workunit.client.0.smithi100.stdout: "Controller ID: 0", 2024-04-07T16:16:15.470 INFO:tasks.workunit.client.0.smithi100.stdout: "Number of Namespaces: 1", 2024-04-07T16:16:15.470 INFO:tasks.workunit.client.0.smithi100.stdout: "Namespace 1 Size/Capacity: 400,088,457,216 [400 GB]", 2024-04-07T16:16:15.470 INFO:tasks.workunit.client.0.smithi100.stdout: "Namespace 1 Formatted LBA Size: 512", 2024-04-07T16:16:15.470 INFO:tasks.workunit.client.0.smithi100.stdout: "Local Time is: Sun Apr 7 16:07:28 2024 UTC", 2024-04-07T16:16:15.470 INFO:tasks.workunit.client.0.smithi100.stdout: "Firmware Updates (0x02): 1 Slot", 2024-04-07T16:16:15.470 INFO:tasks.workunit.client.0.smithi100.stdout: "Optional Admin Commands (0x0006): Format Frmw_DL", 2024-04-07T16:16:15.470 INFO:tasks.workunit.client.0.smithi100.stdout: "Optional NVM Commands (0x0006): Wr_Unc DS_Mngmt", 2024-04-07T16:16:15.470 INFO:tasks.workunit.client.0.smithi100.stdout: "Maximum Data Transfer Size: 32 Pages", 2024-04-07T16:16:15.470 INFO:tasks.workunit.client.0.smithi100.stdout: "", 2024-04-07T16:16:15.470 INFO:tasks.workunit.client.0.smithi100.stdout: "Supported Power States", 2024-04-07T16:16:15.470 INFO:tasks.workunit.client.0.smithi100.stdout: "St Op Max Active Idle RL RT WL WT Ent_Lat Ex_Lat", 2024-04-07T16:16:15.470 INFO:tasks.workunit.client.0.smithi100.stdout: " 0 + 25.00W - - 0 0 0 0 0 0", 2024-04-07T16:16:15.470 INFO:tasks.workunit.client.0.smithi100.stdout: "", 2024-04-07T16:16:15.471 INFO:tasks.workunit.client.0.smithi100.stdout: "Supported LBA Sizes (NSID 0x1)", 2024-04-07T16:16:15.471 INFO:tasks.workunit.client.0.smithi100.stdout: "Id Fmt Data Metadt Rel_Perf", 2024-04-07T16:16:15.471 INFO:tasks.workunit.client.0.smithi100.stdout: " 0 + 512 0 2", 2024-04-07T16:16:15.471 INFO:tasks.workunit.client.0.smithi100.stdout: " 1 - 512 8 2", 2024-04-07T16:16:15.471 INFO:tasks.workunit.client.0.smithi100.stdout: " 2 - 512 16 2", 2024-04-07T16:16:15.471 INFO:tasks.workunit.client.0.smithi100.stdout: " 3 - 4096 0 0", 2024-04-07T16:16:15.471 INFO:tasks.workunit.client.0.smithi100.stdout: " 4 - 4096 8 0", 2024-04-07T16:16:15.471 INFO:tasks.workunit.client.0.smithi100.stdout: " 5 - 4096 64 0", 2024-04-07T16:16:15.471 INFO:tasks.workunit.client.0.smithi100.stdout: " 6 - 4096 128 0", 2024-04-07T16:16:15.471 INFO:tasks.workunit.client.0.smithi100.stdout: "", 2024-04-07T16:16:15.471 INFO:tasks.workunit.client.0.smithi100.stdout: "=== START OF SMART DATA SECTION ===", 2024-04-07T16:16:15.471 INFO:tasks.workunit.client.0.smithi100.stdout: "SMART overall-health self-assessment test result: PASSED", 2024-04-07T16:16:15.471 INFO:tasks.workunit.client.0.smithi100.stdout: "", 2024-04-07T16:16:15.471 INFO:tasks.workunit.client.0.smithi100.stdout: "SMART/Health Information (NVMe Log 0x02)", 2024-04-07T16:16:15.471 INFO:tasks.workunit.client.0.smithi100.stdout: "Critical Warning: 0x00", 2024-04-07T16:16:15.471 INFO:tasks.workunit.client.0.smithi100.stdout: "Temperature: 24 Celsius", 2024-04-07T16:16:15.472 INFO:tasks.workunit.client.0.smithi100.stdout: "Available Spare: 100%", 2024-04-07T16:16:15.472 INFO:tasks.workunit.client.0.smithi100.stdout: "Available Spare Threshold: 10%", 2024-04-07T16:16:15.472 INFO:tasks.workunit.client.0.smithi100.stdout: "Percentage Used: 10%", 2024-04-07T16:16:15.472 INFO:tasks.workunit.client.0.smithi100.stdout: "Data Units Read: 385,784,931 [197 TB]", 2024-04-07T16:16:15.472 INFO:tasks.workunit.client.0.smithi100.stdout: "Data Units Written: 1,937,815,661 [992 TB]", 2024-04-07T16:16:15.472 INFO:tasks.workunit.client.0.smithi100.stdout: "Host Read Commands: 5,771,442,418", 2024-04-07T16:16:15.472 INFO:tasks.workunit.client.0.smithi100.stdout: "Host Write Commands: 31,722,089,956", 2024-04-07T16:16:15.472 INFO:tasks.workunit.client.0.smithi100.stdout: "Controller Busy Time: 4", 2024-04-07T16:16:15.472 INFO:tasks.workunit.client.0.smithi100.stdout: "Power Cycles: 45,774", 2024-04-07T16:16:15.472 INFO:tasks.workunit.client.0.smithi100.stdout: "Power On Hours: 57,978", 2024-04-07T16:16:15.472 INFO:tasks.workunit.client.0.smithi100.stdout: "Unsafe Shutdowns: 210", 2024-04-07T16:16:15.472 INFO:tasks.workunit.client.0.smithi100.stdout: "Media and Data Integrity Errors: 0", 2024-04-07T16:16:15.472 INFO:tasks.workunit.client.0.smithi100.stdout: "Error Information Log Entries: 0", 2024-04-07T16:16:15.472 INFO:tasks.workunit.client.0.smithi100.stdout: "", 2024-04-07T16:16:15.472 INFO:tasks.workunit.client.0.smithi100.stdout: "Error Information (NVMe Log 0x01, max 64 entries)", 2024-04-07T16:16:15.472 INFO:tasks.workunit.client.0.smithi100.stdout: "No Errors Logged", 2024-04-07T16:16:15.473 INFO:tasks.workunit.client.0.smithi100.stdout: "" 2024-04-07T16:16:15.473 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:15.473 INFO:tasks.workunit.client.0.smithi100.stdout: "platform_info": "x86_64-linux-5.15.0-101-generic", 2024-04-07T16:16:15.473 INFO:tasks.workunit.client.0.smithi100.stdout: "svn_revision": "5022", 2024-04-07T16:16:15.473 INFO:tasks.workunit.client.0.smithi100.stdout: "version": [ 2024-04-07T16:16:15.473 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T16:16:15.473 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T16:16:15.473 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T16:16:15.473 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.473 INFO:tasks.workunit.client.0.smithi100.stdout: "temperature": { 2024-04-07T16:16:15.473 INFO:tasks.workunit.client.0.smithi100.stdout: "current": 24 2024-04-07T16:16:15.473 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.473 INFO:tasks.workunit.client.0.smithi100.stdout: "user_capacity": { 2024-04-07T16:16:15.473 INFO:tasks.workunit.client.0.smithi100.stdout: "blocks": 781422768, 2024-04-07T16:16:15.473 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 400088457216 2024-04-07T16:16:15.474 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:15.474 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:15.474 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:15.474 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.474 INFO:tasks.workunit.client.0.smithi100.stdout: "25233610-f4fa-11ee-8447-0cc47a8fd733": { 2024-04-07T16:16:15.474 INFO:tasks.workunit.client.0.smithi100.stdout: "INTEL_SSDPEDMD400G4_2524b6c0-f4fa-11ee-8447-0cc47a8fd733": { 2024-04-07T16:16:15.474 INFO:tasks.workunit.client.0.smithi100.stdout: "20240407-160732": { 2024-04-07T16:16:15.474 INFO:tasks.workunit.client.0.smithi100.stdout: "device": { 2024-04-07T16:16:15.474 INFO:tasks.workunit.client.0.smithi100.stdout: "info_name": "/dev/nvme0n1", 2024-04-07T16:16:15.474 INFO:tasks.workunit.client.0.smithi100.stdout: "name": "/dev/nvme0n1", 2024-04-07T16:16:15.474 INFO:tasks.workunit.client.0.smithi100.stdout: "protocol": "NVMe", 2024-04-07T16:16:15.474 INFO:tasks.workunit.client.0.smithi100.stdout: "type": "nvme" 2024-04-07T16:16:15.474 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.474 INFO:tasks.workunit.client.0.smithi100.stdout: "firmware_version": "8DV101H0", 2024-04-07T16:16:15.474 INFO:tasks.workunit.client.0.smithi100.stdout: "host_id": "25233610-f4fa-11ee-8447-0cc47a8fd733", 2024-04-07T16:16:15.475 INFO:tasks.workunit.client.0.smithi100.stdout: "json_format_version": [ 2024-04-07T16:16:15.475 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T16:16:15.475 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T16:16:15.475 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:15.475 INFO:tasks.workunit.client.0.smithi100.stdout: "local_time": { 2024-04-07T16:16:15.475 INFO:tasks.workunit.client.0.smithi100.stdout: "asctime": "Sun Apr 7 16:07:32 2024 UTC", 2024-04-07T16:16:15.475 INFO:tasks.workunit.client.0.smithi100.stdout: "time_t": 1712506052 2024-04-07T16:16:15.475 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.475 INFO:tasks.workunit.client.0.smithi100.stdout: "logical_block_size": 512, 2024-04-07T16:16:15.475 INFO:tasks.workunit.client.0.smithi100.stdout: "model_name": "INTEL SSDPEDMD400G4", 2024-04-07T16:16:15.475 INFO:tasks.workunit.client.0.smithi100.stdout: "nvme_controller_id": 0, 2024-04-07T16:16:15.475 INFO:tasks.workunit.client.0.smithi100.stdout: "nvme_ieee_oui_identifier": 6083300, 2024-04-07T16:16:15.475 INFO:tasks.workunit.client.0.smithi100.stdout: "nvme_namespaces": [ 2024-04-07T16:16:15.475 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:15.475 INFO:tasks.workunit.client.0.smithi100.stdout: "capacity": { 2024-04-07T16:16:15.476 INFO:tasks.workunit.client.0.smithi100.stdout: "blocks": 781422768, 2024-04-07T16:16:15.476 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 400088457216 2024-04-07T16:16:15.476 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.476 INFO:tasks.workunit.client.0.smithi100.stdout: "formatted_lba_size": 512, 2024-04-07T16:16:15.476 INFO:tasks.workunit.client.0.smithi100.stdout: "id": 1, 2024-04-07T16:16:15.476 INFO:tasks.workunit.client.0.smithi100.stdout: "size": { 2024-04-07T16:16:15.476 INFO:tasks.workunit.client.0.smithi100.stdout: "blocks": 781422768, 2024-04-07T16:16:15.476 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 400088457216 2024-04-07T16:16:15.476 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.476 INFO:tasks.workunit.client.0.smithi100.stdout: "utilization": { 2024-04-07T16:16:15.476 INFO:tasks.workunit.client.0.smithi100.stdout: "blocks": 781422768, 2024-04-07T16:16:15.476 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 400088457216 2024-04-07T16:16:15.476 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:15.476 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:15.476 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:15.477 INFO:tasks.workunit.client.0.smithi100.stdout: "nvme_number_of_namespaces": 1, 2024-04-07T16:16:15.477 INFO:tasks.workunit.client.0.smithi100.stdout: "nvme_pci_vendor": { 2024-04-07T16:16:15.477 INFO:tasks.workunit.client.0.smithi100.stdout: "id": 32902, 2024-04-07T16:16:15.477 INFO:tasks.workunit.client.0.smithi100.stdout: "subsystem_id": 32902 2024-04-07T16:16:15.477 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.477 INFO:tasks.workunit.client.0.smithi100.stdout: "nvme_smart_health_information_add_log": { 2024-04-07T16:16:15.477 INFO:tasks.workunit.client.0.smithi100.stdout: "Device stats": { 2024-04-07T16:16:15.477 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_fw_downgrades": { 2024-04-07T16:16:15.477 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 0, 2024-04-07T16:16:15.477 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 0 2024-04-07T16:16:15.477 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.477 INFO:tasks.workunit.client.0.smithi100.stdout: "crc_error_count": { 2024-04-07T16:16:15.477 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 100, 2024-04-07T16:16:15.477 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 0 2024-04-07T16:16:15.477 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.478 INFO:tasks.workunit.client.0.smithi100.stdout: "end_to_end_error_detection_count": { 2024-04-07T16:16:15.478 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 100, 2024-04-07T16:16:15.478 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 0 2024-04-07T16:16:15.478 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.478 INFO:tasks.workunit.client.0.smithi100.stdout: "erase_fail_count": { 2024-04-07T16:16:15.478 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 100, 2024-04-07T16:16:15.478 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 0 2024-04-07T16:16:15.478 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.478 INFO:tasks.workunit.client.0.smithi100.stdout: "host_bytes_written": { 2024-04-07T16:16:15.478 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 100, 2024-04-07T16:16:15.478 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 25060599 2024-04-07T16:16:15.478 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.478 INFO:tasks.workunit.client.0.smithi100.stdout: "host_ctx_wear_used": { 2024-04-07T16:16:15.478 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 100, 2024-04-07T16:16:15.478 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 1 2024-04-07T16:16:15.478 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.479 INFO:tasks.workunit.client.0.smithi100.stdout: "media_bytes_read": { 2024-04-07T16:16:15.479 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 0, 2024-04-07T16:16:15.479 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 0 2024-04-07T16:16:15.479 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.479 INFO:tasks.workunit.client.0.smithi100.stdout: "nand_bytes_written": { 2024-04-07T16:16:15.479 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 100, 2024-04-07T16:16:15.479 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 42751216 2024-04-07T16:16:15.479 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.479 INFO:tasks.workunit.client.0.smithi100.stdout: "perf_stat_indicator": { 2024-04-07T16:16:15.479 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 0, 2024-04-07T16:16:15.479 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 0 2024-04-07T16:16:15.479 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.479 INFO:tasks.workunit.client.0.smithi100.stdout: "pll_lock_loss_count": { 2024-04-07T16:16:15.479 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 100, 2024-04-07T16:16:15.480 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 0 2024-04-07T16:16:15.480 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.480 INFO:tasks.workunit.client.0.smithi100.stdout: "program_fail_count": { 2024-04-07T16:16:15.480 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 100, 2024-04-07T16:16:15.480 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 0 2024-04-07T16:16:15.480 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.480 INFO:tasks.workunit.client.0.smithi100.stdout: "re_alloc_sectr_cnt": { 2024-04-07T16:16:15.480 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 0, 2024-04-07T16:16:15.480 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 0 2024-04-07T16:16:15.480 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.480 INFO:tasks.workunit.client.0.smithi100.stdout: "retry_buffer_overflow_count": { 2024-04-07T16:16:15.480 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 100, 2024-04-07T16:16:15.480 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 0 2024-04-07T16:16:15.480 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.481 INFO:tasks.workunit.client.0.smithi100.stdout: "soft_ecc_err_rate": { 2024-04-07T16:16:15.481 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 0, 2024-04-07T16:16:15.481 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 0 2024-04-07T16:16:15.481 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.481 INFO:tasks.workunit.client.0.smithi100.stdout: "thermal_throttle_status": { 2024-04-07T16:16:15.481 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 100, 2024-04-07T16:16:15.481 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": { 2024-04-07T16:16:15.481 INFO:tasks.workunit.client.0.smithi100.stdout: "cnt": 0, 2024-04-07T16:16:15.481 INFO:tasks.workunit.client.0.smithi100.stdout: "pct": 0 2024-04-07T16:16:15.481 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:15.481 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.481 INFO:tasks.workunit.client.0.smithi100.stdout: "timed_workload_host_reads": { 2024-04-07T16:16:15.481 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 100, 2024-04-07T16:16:15.481 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 65535 2024-04-07T16:16:15.481 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.482 INFO:tasks.workunit.client.0.smithi100.stdout: "timed_workload_media_wear": { 2024-04-07T16:16:15.482 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 100, 2024-04-07T16:16:15.482 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 64 2024-04-07T16:16:15.482 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.482 INFO:tasks.workunit.client.0.smithi100.stdout: "timed_workload_timer": { 2024-04-07T16:16:15.482 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 100, 2024-04-07T16:16:15.482 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 65535 2024-04-07T16:16:15.482 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.482 INFO:tasks.workunit.client.0.smithi100.stdout: "unexp_power_loss": { 2024-04-07T16:16:15.482 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 0, 2024-04-07T16:16:15.482 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 0 2024-04-07T16:16:15.482 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.482 INFO:tasks.workunit.client.0.smithi100.stdout: "wear_leveling": { 2024-04-07T16:16:15.482 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 91, 2024-04-07T16:16:15.482 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": { 2024-04-07T16:16:15.483 INFO:tasks.workunit.client.0.smithi100.stdout: "avg": 2895, 2024-04-07T16:16:15.483 INFO:tasks.workunit.client.0.smithi100.stdout: "max": 2907, 2024-04-07T16:16:15.483 INFO:tasks.workunit.client.0.smithi100.stdout: "min": 2884 2024-04-07T16:16:15.483 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:15.483 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:15.483 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.483 INFO:tasks.workunit.client.0.smithi100.stdout: "Intel Smart log": "nvme0n1" 2024-04-07T16:16:15.483 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.483 INFO:tasks.workunit.client.0.smithi100.stdout: "nvme_smart_health_information_log": { 2024-04-07T16:16:15.483 INFO:tasks.workunit.client.0.smithi100.stdout: "available_spare": 100, 2024-04-07T16:16:15.483 INFO:tasks.workunit.client.0.smithi100.stdout: "available_spare_threshold": 10, 2024-04-07T16:16:15.484 INFO:tasks.workunit.client.0.smithi100.stdout: "controller_busy_time": 0, 2024-04-07T16:16:15.484 INFO:tasks.workunit.client.0.smithi100.stdout: "critical_warning": 0, 2024-04-07T16:16:15.484 INFO:tasks.workunit.client.0.smithi100.stdout: "data_units_read": 359011585, 2024-04-07T16:16:15.484 INFO:tasks.workunit.client.0.smithi100.stdout: "data_units_written": 1642371481, 2024-04-07T16:16:15.484 INFO:tasks.workunit.client.0.smithi100.stdout: "host_reads": 5750712954, 2024-04-07T16:16:15.484 INFO:tasks.workunit.client.0.smithi100.stdout: "host_writes": 28276536650, 2024-04-07T16:16:15.484 INFO:tasks.workunit.client.0.smithi100.stdout: "media_errors": 0, 2024-04-07T16:16:15.484 INFO:tasks.workunit.client.0.smithi100.stdout: "num_err_log_entries": 0, 2024-04-07T16:16:15.484 INFO:tasks.workunit.client.0.smithi100.stdout: "percentage_used": 9, 2024-04-07T16:16:15.484 INFO:tasks.workunit.client.0.smithi100.stdout: "power_cycles": 47059, 2024-04-07T16:16:15.484 INFO:tasks.workunit.client.0.smithi100.stdout: "power_on_hours": 54821, 2024-04-07T16:16:15.484 INFO:tasks.workunit.client.0.smithi100.stdout: "temperature": 25, 2024-04-07T16:16:15.484 INFO:tasks.workunit.client.0.smithi100.stdout: "unsafe_shutdowns": 12 2024-04-07T16:16:15.484 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.485 INFO:tasks.workunit.client.0.smithi100.stdout: "nvme_vendor": "intel", 2024-04-07T16:16:15.485 INFO:tasks.workunit.client.0.smithi100.stdout: "power_cycle_count": 47059, 2024-04-07T16:16:15.485 INFO:tasks.workunit.client.0.smithi100.stdout: "power_on_time": { 2024-04-07T16:16:15.485 INFO:tasks.workunit.client.0.smithi100.stdout: "hours": 54821 2024-04-07T16:16:15.485 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.485 INFO:tasks.workunit.client.0.smithi100.stdout: "serial_number": "deleted", 2024-04-07T16:16:15.485 INFO:tasks.workunit.client.0.smithi100.stdout: "smart_status": { 2024-04-07T16:16:15.485 INFO:tasks.workunit.client.0.smithi100.stdout: "nvme": { 2024-04-07T16:16:15.485 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:15.485 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.485 INFO:tasks.workunit.client.0.smithi100.stdout: "passed": true 2024-04-07T16:16:15.485 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.485 INFO:tasks.workunit.client.0.smithi100.stdout: "smartctl": { 2024-04-07T16:16:15.485 INFO:tasks.workunit.client.0.smithi100.stdout: "argv": [ 2024-04-07T16:16:15.485 INFO:tasks.workunit.client.0.smithi100.stdout: "smartctl", 2024-04-07T16:16:15.486 INFO:tasks.workunit.client.0.smithi100.stdout: "-x", 2024-04-07T16:16:15.486 INFO:tasks.workunit.client.0.smithi100.stdout: "--json=o", 2024-04-07T16:16:15.486 INFO:tasks.workunit.client.0.smithi100.stdout: "/dev/nvme0n1" 2024-04-07T16:16:15.486 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:15.486 INFO:tasks.workunit.client.0.smithi100.stdout: "build_info": "(local build)", 2024-04-07T16:16:15.486 INFO:tasks.workunit.client.0.smithi100.stdout: "exit_status": 0, 2024-04-07T16:16:15.486 INFO:tasks.workunit.client.0.smithi100.stdout: "output": [ 2024-04-07T16:16:15.486 INFO:tasks.workunit.client.0.smithi100.stdout: "smartctl 7.1 2019-12-30 r5022 [x86_64-linux-5.15.0-101-generic] (local build)", 2024-04-07T16:16:15.486 INFO:tasks.workunit.client.0.smithi100.stdout: "Copyright (C) 2002-19, Bruce Allen, Christian Franke, www.smartmontools.org", 2024-04-07T16:16:15.486 INFO:tasks.workunit.client.0.smithi100.stdout: "", 2024-04-07T16:16:15.486 INFO:tasks.workunit.client.0.smithi100.stdout: "=== START OF INFORMATION SECTION ===", 2024-04-07T16:16:15.486 INFO:tasks.workunit.client.0.smithi100.stdout: "Model Number: INTEL SSDPEDMD400G4", 2024-04-07T16:16:15.486 INFO:tasks.workunit.client.0.smithi100.stdout: "Serial Number: deleted", 2024-04-07T16:16:15.486 INFO:tasks.workunit.client.0.smithi100.stdout: "Firmware Version: 8DV101H0", 2024-04-07T16:16:15.487 INFO:tasks.workunit.client.0.smithi100.stdout: "PCI Vendor/Subsystem ID: 0x8086", 2024-04-07T16:16:15.487 INFO:tasks.workunit.client.0.smithi100.stdout: "IEEE OUI Identifier: 0x5cd2e4", 2024-04-07T16:16:15.487 INFO:tasks.workunit.client.0.smithi100.stdout: "Controller ID: 0", 2024-04-07T16:16:15.487 INFO:tasks.workunit.client.0.smithi100.stdout: "Number of Namespaces: 1", 2024-04-07T16:16:15.487 INFO:tasks.workunit.client.0.smithi100.stdout: "Namespace 1 Size/Capacity: 400,088,457,216 [400 GB]", 2024-04-07T16:16:15.487 INFO:tasks.workunit.client.0.smithi100.stdout: "Namespace 1 Formatted LBA Size: 512", 2024-04-07T16:16:15.487 INFO:tasks.workunit.client.0.smithi100.stdout: "Local Time is: Sun Apr 7 16:07:32 2024 UTC", 2024-04-07T16:16:15.487 INFO:tasks.workunit.client.0.smithi100.stdout: "Firmware Updates (0x02): 1 Slot", 2024-04-07T16:16:15.487 INFO:tasks.workunit.client.0.smithi100.stdout: "Optional Admin Commands (0x0006): Format Frmw_DL", 2024-04-07T16:16:15.487 INFO:tasks.workunit.client.0.smithi100.stdout: "Optional NVM Commands (0x0006): Wr_Unc DS_Mngmt", 2024-04-07T16:16:15.487 INFO:tasks.workunit.client.0.smithi100.stdout: "Maximum Data Transfer Size: 32 Pages", 2024-04-07T16:16:15.487 INFO:tasks.workunit.client.0.smithi100.stdout: "", 2024-04-07T16:16:15.487 INFO:tasks.workunit.client.0.smithi100.stdout: "Supported Power States", 2024-04-07T16:16:15.487 INFO:tasks.workunit.client.0.smithi100.stdout: "St Op Max Active Idle RL RT WL WT Ent_Lat Ex_Lat", 2024-04-07T16:16:15.487 INFO:tasks.workunit.client.0.smithi100.stdout: " 0 + 25.00W - - 0 0 0 0 0 0", 2024-04-07T16:16:15.488 INFO:tasks.workunit.client.0.smithi100.stdout: "", 2024-04-07T16:16:15.488 INFO:tasks.workunit.client.0.smithi100.stdout: "Supported LBA Sizes (NSID 0x1)", 2024-04-07T16:16:15.488 INFO:tasks.workunit.client.0.smithi100.stdout: "Id Fmt Data Metadt Rel_Perf", 2024-04-07T16:16:15.488 INFO:tasks.workunit.client.0.smithi100.stdout: " 0 + 512 0 2", 2024-04-07T16:16:15.488 INFO:tasks.workunit.client.0.smithi100.stdout: " 1 - 512 8 2", 2024-04-07T16:16:15.488 INFO:tasks.workunit.client.0.smithi100.stdout: " 2 - 512 16 2", 2024-04-07T16:16:15.488 INFO:tasks.workunit.client.0.smithi100.stdout: " 3 - 4096 0 0", 2024-04-07T16:16:15.488 INFO:tasks.workunit.client.0.smithi100.stdout: " 4 - 4096 8 0", 2024-04-07T16:16:15.488 INFO:tasks.workunit.client.0.smithi100.stdout: " 5 - 4096 64 0", 2024-04-07T16:16:15.488 INFO:tasks.workunit.client.0.smithi100.stdout: " 6 - 4096 128 0", 2024-04-07T16:16:15.488 INFO:tasks.workunit.client.0.smithi100.stdout: "", 2024-04-07T16:16:15.488 INFO:tasks.workunit.client.0.smithi100.stdout: "=== START OF SMART DATA SECTION ===", 2024-04-07T16:16:15.488 INFO:tasks.workunit.client.0.smithi100.stdout: "SMART overall-health self-assessment test result: PASSED", 2024-04-07T16:16:15.488 INFO:tasks.workunit.client.0.smithi100.stdout: "", 2024-04-07T16:16:15.488 INFO:tasks.workunit.client.0.smithi100.stdout: "SMART/Health Information (NVMe Log 0x02)", 2024-04-07T16:16:15.489 INFO:tasks.workunit.client.0.smithi100.stdout: "Critical Warning: 0x00", 2024-04-07T16:16:15.489 INFO:tasks.workunit.client.0.smithi100.stdout: "Temperature: 25 Celsius", 2024-04-07T16:16:15.489 INFO:tasks.workunit.client.0.smithi100.stdout: "Available Spare: 100%", 2024-04-07T16:16:15.489 INFO:tasks.workunit.client.0.smithi100.stdout: "Available Spare Threshold: 10%", 2024-04-07T16:16:15.489 INFO:tasks.workunit.client.0.smithi100.stdout: "Percentage Used: 9%", 2024-04-07T16:16:15.489 INFO:tasks.workunit.client.0.smithi100.stdout: "Data Units Read: 359,011,585 [183 TB]", 2024-04-07T16:16:15.489 INFO:tasks.workunit.client.0.smithi100.stdout: "Data Units Written: 1,642,371,481 [840 TB]", 2024-04-07T16:16:15.489 INFO:tasks.workunit.client.0.smithi100.stdout: "Host Read Commands: 5,750,712,954", 2024-04-07T16:16:15.489 INFO:tasks.workunit.client.0.smithi100.stdout: "Host Write Commands: 28,276,536,650", 2024-04-07T16:16:15.489 INFO:tasks.workunit.client.0.smithi100.stdout: "Controller Busy Time: 0", 2024-04-07T16:16:15.489 INFO:tasks.workunit.client.0.smithi100.stdout: "Power Cycles: 47,059", 2024-04-07T16:16:15.489 INFO:tasks.workunit.client.0.smithi100.stdout: "Power On Hours: 54,821", 2024-04-07T16:16:15.489 INFO:tasks.workunit.client.0.smithi100.stdout: "Unsafe Shutdowns: 12", 2024-04-07T16:16:15.489 INFO:tasks.workunit.client.0.smithi100.stdout: "Media and Data Integrity Errors: 0", 2024-04-07T16:16:15.489 INFO:tasks.workunit.client.0.smithi100.stdout: "Error Information Log Entries: 0", 2024-04-07T16:16:15.490 INFO:tasks.workunit.client.0.smithi100.stdout: "", 2024-04-07T16:16:15.490 INFO:tasks.workunit.client.0.smithi100.stdout: "Error Information (NVMe Log 0x01, max 64 entries)", 2024-04-07T16:16:15.490 INFO:tasks.workunit.client.0.smithi100.stdout: "No Errors Logged", 2024-04-07T16:16:15.490 INFO:tasks.workunit.client.0.smithi100.stdout: "" 2024-04-07T16:16:15.490 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:15.490 INFO:tasks.workunit.client.0.smithi100.stdout: "platform_info": "x86_64-linux-5.15.0-101-generic", 2024-04-07T16:16:15.490 INFO:tasks.workunit.client.0.smithi100.stdout: "svn_revision": "5022", 2024-04-07T16:16:15.490 INFO:tasks.workunit.client.0.smithi100.stdout: "version": [ 2024-04-07T16:16:15.490 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T16:16:15.490 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T16:16:15.490 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T16:16:15.490 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.490 INFO:tasks.workunit.client.0.smithi100.stdout: "temperature": { 2024-04-07T16:16:15.490 INFO:tasks.workunit.client.0.smithi100.stdout: "current": 25 2024-04-07T16:16:15.490 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.490 INFO:tasks.workunit.client.0.smithi100.stdout: "user_capacity": { 2024-04-07T16:16:15.491 INFO:tasks.workunit.client.0.smithi100.stdout: "blocks": 781422768, 2024-04-07T16:16:15.491 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 400088457216 2024-04-07T16:16:15.491 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:15.491 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:15.491 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:15.491 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:15.491 INFO:tasks.workunit.client.0.smithi100.stdout:} 2024-04-07T16:16:15.491 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:14 smithi144 bash[23522]: cluster 2024-04-07T16:16:12.676160+0000 mgr.y (mgr.14553) 149 : cluster 0 pgmap v107: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:16:15.491 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:14 smithi144 bash[23522]: audit 2024-04-07T16:16:12.961581+0000 mgr.y (mgr.14553) 150 : audit 0 from='client.14943 -' entity='client.admin' cmd=[{"prefix": "telemetry show", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:16:15.491 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:14 smithi144 bash[23522]: audit 2024-04-07T16:16:13.507778+0000 mon.b (mon.1) 168 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:16:15.491 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:14 smithi144 bash[23522]: audit 2024-04-07T16:16:13.509858+0000 mgr.y (mgr.14553) 151 : audit 0 from='client.14943 -' entity='client.admin' cmd=[{"prefix": "telemetry show", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:16:15.491 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:14 smithi144 bash[23522]: audit 2024-04-07T16:16:13.515750+0000 mon.b (mon.1) 169 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config-key dump", "format": "json"}]: dispatch 2024-04-07T16:16:15.491 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:14 smithi144 bash[23522]: audit 2024-04-07T16:16:13.726455+0000 mon.b (mon.1) 170 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:16:15.491 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:14 smithi144 bash[23522]: audit 2024-04-07T16:16:13.726843+0000 mon.b (mon.1) 171 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:16:15.492 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:14 smithi144 bash[23522]: audit 2024-04-07T16:16:13.727179+0000 mon.b (mon.1) 172 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:16:15.492 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:14 smithi144 bash[23522]: audit 2024-04-07T16:16:13.730422+0000 mon.b (mon.1) 173 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:16:15.492 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:14 smithi144 bash[23522]: audit 2024-04-07T16:16:13.730799+0000 mon.b (mon.1) 174 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:16:15.492 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:14 smithi144 bash[23522]: audit 2024-04-07T16:16:13.731131+0000 mon.b (mon.1) 175 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:16:15.492 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:14 smithi144 bash[23522]: audit 2024-04-07T16:16:13.734302+0000 mon.b (mon.1) 176 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:16:15.492 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:14 smithi144 bash[23522]: audit 2024-04-07T16:16:13.734673+0000 mon.b (mon.1) 177 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:16:15.492 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:14 smithi144 bash[23522]: audit 2024-04-07T16:16:13.735015+0000 mon.b (mon.1) 178 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:16:15.492 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:14 smithi144 bash[23522]: audit 2024-04-07T16:16:13.754342+0000 mon.b (mon.1) 179 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:16:15.492 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:14 smithi144 bash[23522]: audit 2024-04-07T16:16:13.754719+0000 mon.b (mon.1) 180 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:16:15.492 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:14 smithi144 bash[23522]: audit 2024-04-07T16:16:13.755052+0000 mon.b (mon.1) 181 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:16:15.492 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:14 smithi144 bash[23522]: audit 2024-04-07T16:16:13.757687+0000 mon.b (mon.1) 182 : audit 1 from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T16:16:15.492 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:14 smithi144 bash[23522]: audit 2024-04-07T16:16:13.757813+0000 mon.b (mon.1) 183 : audit 1 from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T16:16:15.492 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:14 smithi144 bash[23522]: audit 2024-04-07T16:16:13.758204+0000 mon.b (mon.1) 184 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:16:15.492 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:14 smithi144 bash[23522]: audit 2024-04-07T16:16:13.758608+0000 mon.b (mon.1) 185 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:16:15.492 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:14 smithi144 bash[23522]: audit 2024-04-07T16:16:13.758706+0000 mon.a (mon.0) 868 : audit 1 from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T16:16:15.493 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:14 smithi144 bash[23522]: audit 2024-04-07T16:16:13.758858+0000 mon.a (mon.0) 869 : audit 1 from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T16:16:15.493 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:14 smithi144 bash[23522]: audit 2024-04-07T16:16:13.758942+0000 mon.b (mon.1) 186 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:16:15.493 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:14 smithi144 bash[23522]: audit 2024-04-07T16:16:13.762176+0000 mon.b (mon.1) 187 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:16:15.493 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:14 smithi144 bash[23522]: audit 2024-04-07T16:16:13.762547+0000 mon.b (mon.1) 188 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:16:15.493 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:14 smithi144 bash[23522]: audit 2024-04-07T16:16:13.762887+0000 mon.b (mon.1) 189 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:16:15.493 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:14 smithi144 bash[23522]: audit 2024-04-07T16:16:13.766574+0000 mon.c (mon.2) 45 : audit 1 from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T16:16:15.493 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:14 smithi144 bash[23522]: audit 2024-04-07T16:16:13.766753+0000 mon.c (mon.2) 46 : audit 1 from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T16:16:15.493 INFO:tasks.workunit.client.0.smithi100.stderr:+ ceph telemetry show-all 2024-04-07T16:16:15.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:14 smithi100 bash[21346]: cluster 2024-04-07T16:16:12.676160+0000 mgr.y (mgr.14553) 149 : cluster 0 pgmap v107: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:16:15.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:14 smithi100 bash[21346]: audit 2024-04-07T16:16:12.961581+0000 mgr.y (mgr.14553) 150 : audit 0 from='client.14943 -' entity='client.admin' cmd=[{"prefix": "telemetry show", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:16:15.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:14 smithi100 bash[21346]: audit 2024-04-07T16:16:13.507778+0000 mon.b (mon.1) 168 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:16:15.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:14 smithi100 bash[21346]: audit 2024-04-07T16:16:13.509858+0000 mgr.y (mgr.14553) 151 : audit 0 from='client.14943 -' entity='client.admin' cmd=[{"prefix": "telemetry show", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:16:15.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:14 smithi100 bash[21346]: audit 2024-04-07T16:16:13.515750+0000 mon.b (mon.1) 169 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config-key dump", "format": "json"}]: dispatch 2024-04-07T16:16:15.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:14 smithi100 bash[21346]: audit 2024-04-07T16:16:13.726455+0000 mon.b (mon.1) 170 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:16:15.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:14 smithi100 bash[21346]: audit 2024-04-07T16:16:13.726843+0000 mon.b (mon.1) 171 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:16:15.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:14 smithi100 bash[21346]: audit 2024-04-07T16:16:13.727179+0000 mon.b (mon.1) 172 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:16:15.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:14 smithi100 bash[21346]: audit 2024-04-07T16:16:13.730422+0000 mon.b (mon.1) 173 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:16:15.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:14 smithi100 bash[21346]: audit 2024-04-07T16:16:13.730799+0000 mon.b (mon.1) 174 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:16:15.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:14 smithi100 bash[21346]: audit 2024-04-07T16:16:13.731131+0000 mon.b (mon.1) 175 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:16:15.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:14 smithi100 bash[21346]: audit 2024-04-07T16:16:13.734302+0000 mon.b (mon.1) 176 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:16:15.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:14 smithi100 bash[21346]: audit 2024-04-07T16:16:13.734673+0000 mon.b (mon.1) 177 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:16:15.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:14 smithi100 bash[21346]: audit 2024-04-07T16:16:13.735015+0000 mon.b (mon.1) 178 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:16:15.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:14 smithi100 bash[21346]: audit 2024-04-07T16:16:13.754342+0000 mon.b (mon.1) 179 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:16:15.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:14 smithi100 bash[21346]: audit 2024-04-07T16:16:13.754719+0000 mon.b (mon.1) 180 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:16:15.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:14 smithi100 bash[21346]: audit 2024-04-07T16:16:13.755052+0000 mon.b (mon.1) 181 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:16:15.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:14 smithi100 bash[21346]: audit 2024-04-07T16:16:13.757687+0000 mon.b (mon.1) 182 : audit 1 from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T16:16:15.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:14 smithi100 bash[21346]: audit 2024-04-07T16:16:13.757813+0000 mon.b (mon.1) 183 : audit 1 from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T16:16:15.495 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:14 smithi100 bash[31660]: cluster 2024-04-07T16:16:12.676160+0000 mgr.y (mgr.14553) 149 : cluster 0 pgmap v107: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:16:15.495 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:14 smithi100 bash[31660]: audit 2024-04-07T16:16:12.961581+0000 mgr.y (mgr.14553) 150 : audit 0 from='client.14943 -' entity='client.admin' cmd=[{"prefix": "telemetry show", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:16:15.495 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:14 smithi100 bash[31660]: audit 2024-04-07T16:16:13.507778+0000 mon.b (mon.1) 168 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:16:15.495 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:14 smithi100 bash[31660]: audit 2024-04-07T16:16:13.509858+0000 mgr.y (mgr.14553) 151 : audit 0 from='client.14943 -' entity='client.admin' cmd=[{"prefix": "telemetry show", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:16:15.495 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:14 smithi100 bash[31660]: audit 2024-04-07T16:16:13.515750+0000 mon.b (mon.1) 169 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config-key dump", "format": "json"}]: dispatch 2024-04-07T16:16:15.496 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:14 smithi100 bash[31660]: audit 2024-04-07T16:16:13.726455+0000 mon.b (mon.1) 170 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:16:15.496 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:14 smithi100 bash[31660]: audit 2024-04-07T16:16:13.726843+0000 mon.b (mon.1) 171 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:16:15.496 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:14 smithi100 bash[31660]: audit 2024-04-07T16:16:13.727179+0000 mon.b (mon.1) 172 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:16:15.496 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:14 smithi100 bash[31660]: audit 2024-04-07T16:16:13.730422+0000 mon.b (mon.1) 173 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:16:15.496 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:14 smithi100 bash[31660]: audit 2024-04-07T16:16:13.730799+0000 mon.b (mon.1) 174 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:16:15.496 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:14 smithi100 bash[31660]: audit 2024-04-07T16:16:13.731131+0000 mon.b (mon.1) 175 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:16:15.496 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:14 smithi100 bash[31660]: audit 2024-04-07T16:16:13.734302+0000 mon.b (mon.1) 176 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:16:15.496 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:14 smithi100 bash[31660]: audit 2024-04-07T16:16:13.734673+0000 mon.b (mon.1) 177 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:16:15.496 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:14 smithi100 bash[31660]: audit 2024-04-07T16:16:13.735015+0000 mon.b (mon.1) 178 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:16:15.496 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:14 smithi100 bash[31660]: audit 2024-04-07T16:16:13.754342+0000 mon.b (mon.1) 179 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:16:15.496 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:14 smithi100 bash[31660]: audit 2024-04-07T16:16:13.754719+0000 mon.b (mon.1) 180 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:16:15.496 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:14 smithi100 bash[31660]: audit 2024-04-07T16:16:13.755052+0000 mon.b (mon.1) 181 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:16:15.496 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:14 smithi100 bash[31660]: audit 2024-04-07T16:16:13.757687+0000 mon.b (mon.1) 182 : audit 1 from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T16:16:15.496 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:14 smithi100 bash[31660]: audit 2024-04-07T16:16:13.757813+0000 mon.b (mon.1) 183 : audit 1 from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T16:16:15.497 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:14 smithi100 bash[21346]: audit 2024-04-07T16:16:13.758204+0000 mon.b (mon.1) 184 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:16:15.497 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:14 smithi100 bash[21346]: audit 2024-04-07T16:16:13.758608+0000 mon.b (mon.1) 185 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:16:15.497 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:14 smithi100 bash[21346]: audit 2024-04-07T16:16:13.758706+0000 mon.a (mon.0) 868 : audit 1 from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T16:16:15.497 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:14 smithi100 bash[21346]: audit 2024-04-07T16:16:13.758858+0000 mon.a (mon.0) 869 : audit 1 from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T16:16:15.497 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:14 smithi100 bash[21346]: audit 2024-04-07T16:16:13.758942+0000 mon.b (mon.1) 186 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:16:15.497 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:14 smithi100 bash[21346]: audit 2024-04-07T16:16:13.762176+0000 mon.b (mon.1) 187 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:16:15.497 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:14 smithi100 bash[21346]: audit 2024-04-07T16:16:13.762547+0000 mon.b (mon.1) 188 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:16:15.497 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:14 smithi100 bash[21346]: audit 2024-04-07T16:16:13.762887+0000 mon.b (mon.1) 189 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:16:15.497 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:14 smithi100 bash[21346]: audit 2024-04-07T16:16:13.766574+0000 mon.c (mon.2) 45 : audit 1 from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T16:16:15.497 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:14 smithi100 bash[21346]: audit 2024-04-07T16:16:13.766753+0000 mon.c (mon.2) 46 : audit 1 from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T16:16:15.498 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:14 smithi100 bash[31660]: audit 2024-04-07T16:16:13.758204+0000 mon.b (mon.1) 184 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:16:15.498 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:14 smithi100 bash[31660]: audit 2024-04-07T16:16:13.758608+0000 mon.b (mon.1) 185 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:16:15.498 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:14 smithi100 bash[31660]: audit 2024-04-07T16:16:13.758706+0000 mon.a (mon.0) 868 : audit 1 from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T16:16:15.498 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:14 smithi100 bash[31660]: audit 2024-04-07T16:16:13.758858+0000 mon.a (mon.0) 869 : audit 1 from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T16:16:15.498 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:14 smithi100 bash[31660]: audit 2024-04-07T16:16:13.758942+0000 mon.b (mon.1) 186 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:16:15.498 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:14 smithi100 bash[31660]: audit 2024-04-07T16:16:13.762176+0000 mon.b (mon.1) 187 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:16:15.498 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:14 smithi100 bash[31660]: audit 2024-04-07T16:16:13.762547+0000 mon.b (mon.1) 188 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:16:15.498 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:14 smithi100 bash[31660]: audit 2024-04-07T16:16:13.762887+0000 mon.b (mon.1) 189 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:16:15.498 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:14 smithi100 bash[31660]: audit 2024-04-07T16:16:13.766574+0000 mon.c (mon.2) 45 : audit 1 from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T16:16:15.498 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:14 smithi100 bash[31660]: audit 2024-04-07T16:16:13.766753+0000 mon.c (mon.2) 46 : audit 1 from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T16:16:15.499 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:16:15 smithi100 bash[21601]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:16:15] "GET /metrics HTTP/1.1" 200 33734 "" "Prometheus/2.43.0" 2024-04-07T16:16:15.499 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:15 smithi100 bash[31660]: audit 2024-04-07T16:16:14.278598+0000 mgr.y (mgr.14553) 152 : audit 0 from='client.24869 -' entity='client.admin' cmd=[{"prefix": "telemetry show-device", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:16:15.499 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:15 smithi100 bash[31660]: audit 2024-04-07T16:16:14.460054+0000 mgr.y (mgr.14553) 153 : audit 0 from='client.24869 -' entity='client.admin' cmd=[{"prefix": "telemetry show-device", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:16:15.499 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:15 smithi100 bash[31660]: audit 2024-04-07T16:16:14.855670+0000 mon.b (mon.1) 190 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:16:15.499 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:15 smithi100 bash[31660]: audit 2024-04-07T16:16:14.863466+0000 mon.b (mon.1) 191 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config-key dump", "format": "json"}]: dispatch 2024-04-07T16:16:15.499 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:15 smithi100 bash[31660]: audit 2024-04-07T16:16:15.075383+0000 mon.b (mon.1) 192 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:16:15.499 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:15 smithi100 bash[31660]: audit 2024-04-07T16:16:15.075782+0000 mon.b (mon.1) 193 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:16:15.499 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:15 smithi100 bash[31660]: audit 2024-04-07T16:16:15.076281+0000 mon.b (mon.1) 194 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:16:15.499 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:15 smithi100 bash[31660]: audit 2024-04-07T16:16:15.079739+0000 mon.b (mon.1) 195 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:16:15.499 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:15 smithi100 bash[31660]: audit 2024-04-07T16:16:15.080241+0000 mon.b (mon.1) 196 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:16:15.500 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:15 smithi100 bash[31660]: audit 2024-04-07T16:16:15.080638+0000 mon.b (mon.1) 197 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:16:15.500 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:15 smithi100 bash[31660]: audit 2024-04-07T16:16:15.083921+0000 mon.b (mon.1) 198 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:16:15.500 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:15 smithi100 bash[31660]: audit 2024-04-07T16:16:15.084302+0000 mon.b (mon.1) 199 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:16:15.500 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:15 smithi100 bash[31660]: audit 2024-04-07T16:16:15.084691+0000 mon.b (mon.1) 200 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:16:15.500 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:15 smithi100 bash[31660]: audit 2024-04-07T16:16:15.104410+0000 mon.b (mon.1) 201 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:16:15.500 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:15 smithi100 bash[31660]: audit 2024-04-07T16:16:15.104787+0000 mon.b (mon.1) 202 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:16:15.500 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:15 smithi100 bash[31660]: audit 2024-04-07T16:16:15.105183+0000 mon.b (mon.1) 203 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:16:15.500 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:15 smithi100 bash[31660]: audit 2024-04-07T16:16:15.107945+0000 mon.b (mon.1) 204 : audit 1 from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T16:16:15.500 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:15 smithi100 bash[31660]: audit 2024-04-07T16:16:15.108046+0000 mon.b (mon.1) 205 : audit 1 from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T16:16:15.500 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:15 smithi100 bash[21346]: audit 2024-04-07T16:16:14.278598+0000 mgr.y (mgr.14553) 152 : audit 0 from='client.24869 -' entity='client.admin' cmd=[{"prefix": "telemetry show-device", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:16:15.500 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:15 smithi100 bash[21346]: audit 2024-04-07T16:16:14.460054+0000 mgr.y (mgr.14553) 153 : audit 0 from='client.24869 -' entity='client.admin' cmd=[{"prefix": "telemetry show-device", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:16:15.500 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:15 smithi100 bash[21346]: audit 2024-04-07T16:16:14.855670+0000 mon.b (mon.1) 190 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:16:15.501 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:15 smithi100 bash[21346]: audit 2024-04-07T16:16:14.863466+0000 mon.b (mon.1) 191 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config-key dump", "format": "json"}]: dispatch 2024-04-07T16:16:15.501 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:15 smithi100 bash[21346]: audit 2024-04-07T16:16:15.075383+0000 mon.b (mon.1) 192 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:16:15.501 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:15 smithi100 bash[21346]: audit 2024-04-07T16:16:15.075782+0000 mon.b (mon.1) 193 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:16:15.501 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:15 smithi100 bash[21346]: audit 2024-04-07T16:16:15.076281+0000 mon.b (mon.1) 194 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:16:15.501 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:15 smithi100 bash[21346]: audit 2024-04-07T16:16:15.079739+0000 mon.b (mon.1) 195 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:16:15.501 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:15 smithi100 bash[21346]: audit 2024-04-07T16:16:15.080241+0000 mon.b (mon.1) 196 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:16:15.501 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:15 smithi100 bash[21346]: audit 2024-04-07T16:16:15.080638+0000 mon.b (mon.1) 197 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:16:15.501 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:15 smithi100 bash[21346]: audit 2024-04-07T16:16:15.083921+0000 mon.b (mon.1) 198 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:16:15.501 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:15 smithi100 bash[21346]: audit 2024-04-07T16:16:15.084302+0000 mon.b (mon.1) 199 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:16:15.501 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:15 smithi100 bash[21346]: audit 2024-04-07T16:16:15.084691+0000 mon.b (mon.1) 200 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:16:15.501 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:15 smithi100 bash[21346]: audit 2024-04-07T16:16:15.104410+0000 mon.b (mon.1) 201 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:16:15.501 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:15 smithi100 bash[21346]: audit 2024-04-07T16:16:15.104787+0000 mon.b (mon.1) 202 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:16:15.501 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:15 smithi100 bash[21346]: audit 2024-04-07T16:16:15.105183+0000 mon.b (mon.1) 203 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:16:15.501 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:15 smithi100 bash[21346]: audit 2024-04-07T16:16:15.107945+0000 mon.b (mon.1) 204 : audit 1 from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T16:16:15.501 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:15 smithi100 bash[21346]: audit 2024-04-07T16:16:15.108046+0000 mon.b (mon.1) 205 : audit 1 from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T16:16:15.502 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:15 smithi100 bash[31660]: audit 2024-04-07T16:16:15.108552+0000 mon.b (mon.1) 206 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:16:15.502 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:15 smithi100 bash[31660]: audit 2024-04-07T16:16:15.108719+0000 mon.a (mon.0) 870 : audit 1 from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T16:16:15.502 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:15 smithi100 bash[31660]: audit 2024-04-07T16:16:15.108866+0000 mon.a (mon.0) 871 : audit 1 from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T16:16:15.502 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:15 smithi100 bash[31660]: audit 2024-04-07T16:16:15.109002+0000 mon.b (mon.1) 207 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:16:15.502 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:15 smithi100 bash[31660]: audit 2024-04-07T16:16:15.109369+0000 mon.b (mon.1) 208 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:16:15.502 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:15 smithi100 bash[31660]: audit 2024-04-07T16:16:15.112687+0000 mon.b (mon.1) 209 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:16:15.502 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:15 smithi100 bash[31660]: audit 2024-04-07T16:16:15.113079+0000 mon.b (mon.1) 210 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:16:15.502 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:15 smithi100 bash[31660]: audit 2024-04-07T16:16:15.113454+0000 mon.b (mon.1) 211 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:16:15.502 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:15 smithi100 bash[31660]: audit 2024-04-07T16:16:15.117175+0000 mon.c (mon.2) 47 : audit 1 from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T16:16:15.502 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:15 smithi100 bash[31660]: audit 2024-04-07T16:16:15.117316+0000 mon.c (mon.2) 48 : audit 1 from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T16:16:15.503 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:15 smithi100 bash[21346]: audit 2024-04-07T16:16:15.108552+0000 mon.b (mon.1) 206 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:16:15.503 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:15 smithi100 bash[21346]: audit 2024-04-07T16:16:15.108719+0000 mon.a (mon.0) 870 : audit 1 from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T16:16:15.503 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:15 smithi100 bash[21346]: audit 2024-04-07T16:16:15.108866+0000 mon.a (mon.0) 871 : audit 1 from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T16:16:15.503 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:15 smithi100 bash[21346]: audit 2024-04-07T16:16:15.109002+0000 mon.b (mon.1) 207 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:16:15.503 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:15 smithi100 bash[21346]: audit 2024-04-07T16:16:15.109369+0000 mon.b (mon.1) 208 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:16:15.503 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:15 smithi100 bash[21346]: audit 2024-04-07T16:16:15.112687+0000 mon.b (mon.1) 209 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:16:15.503 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:15 smithi100 bash[21346]: audit 2024-04-07T16:16:15.113079+0000 mon.b (mon.1) 210 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:16:15.503 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:15 smithi100 bash[21346]: audit 2024-04-07T16:16:15.113454+0000 mon.b (mon.1) 211 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:16:15.503 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:15 smithi100 bash[21346]: audit 2024-04-07T16:16:15.117175+0000 mon.c (mon.2) 47 : audit 1 from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T16:16:15.503 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:15 smithi100 bash[21346]: audit 2024-04-07T16:16:15.117316+0000 mon.c (mon.2) 48 : audit 1 from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T16:16:15.861 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:15 smithi144 bash[23522]: audit 2024-04-07T16:16:14.278598+0000 mgr.y (mgr.14553) 152 : audit 0 from='client.24869 -' entity='client.admin' cmd=[{"prefix": "telemetry show-device", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:16:15.861 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:15 smithi144 bash[23522]: audit 2024-04-07T16:16:14.460054+0000 mgr.y (mgr.14553) 153 : audit 0 from='client.24869 -' entity='client.admin' cmd=[{"prefix": "telemetry show-device", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:16:15.861 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:15 smithi144 bash[23522]: audit 2024-04-07T16:16:14.855670+0000 mon.b (mon.1) 190 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:16:15.861 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:15 smithi144 bash[23522]: audit 2024-04-07T16:16:14.863466+0000 mon.b (mon.1) 191 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config-key dump", "format": "json"}]: dispatch 2024-04-07T16:16:15.861 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:15 smithi144 bash[23522]: audit 2024-04-07T16:16:15.075383+0000 mon.b (mon.1) 192 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:16:15.861 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:15 smithi144 bash[23522]: audit 2024-04-07T16:16:15.075782+0000 mon.b (mon.1) 193 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:16:15.862 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:15 smithi144 bash[23522]: audit 2024-04-07T16:16:15.076281+0000 mon.b (mon.1) 194 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:16:15.862 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:15 smithi144 bash[23522]: audit 2024-04-07T16:16:15.079739+0000 mon.b (mon.1) 195 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:16:15.862 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:15 smithi144 bash[23522]: audit 2024-04-07T16:16:15.080241+0000 mon.b (mon.1) 196 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:16:15.862 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:15 smithi144 bash[23522]: audit 2024-04-07T16:16:15.080638+0000 mon.b (mon.1) 197 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:16:15.862 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:15 smithi144 bash[23522]: audit 2024-04-07T16:16:15.083921+0000 mon.b (mon.1) 198 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:16:15.862 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:15 smithi144 bash[23522]: audit 2024-04-07T16:16:15.084302+0000 mon.b (mon.1) 199 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:16:15.862 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:15 smithi144 bash[23522]: audit 2024-04-07T16:16:15.084691+0000 mon.b (mon.1) 200 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:16:15.862 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:15 smithi144 bash[23522]: audit 2024-04-07T16:16:15.104410+0000 mon.b (mon.1) 201 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:16:15.862 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:15 smithi144 bash[23522]: audit 2024-04-07T16:16:15.104787+0000 mon.b (mon.1) 202 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:16:15.862 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:15 smithi144 bash[23522]: audit 2024-04-07T16:16:15.105183+0000 mon.b (mon.1) 203 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:16:15.862 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:15 smithi144 bash[23522]: audit 2024-04-07T16:16:15.107945+0000 mon.b (mon.1) 204 : audit 1 from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T16:16:15.862 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:15 smithi144 bash[23522]: audit 2024-04-07T16:16:15.108046+0000 mon.b (mon.1) 205 : audit 1 from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T16:16:15.862 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:15 smithi144 bash[23522]: audit 2024-04-07T16:16:15.108552+0000 mon.b (mon.1) 206 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:16:15.863 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:15 smithi144 bash[23522]: audit 2024-04-07T16:16:15.108719+0000 mon.a (mon.0) 870 : audit 1 from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T16:16:15.863 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:15 smithi144 bash[23522]: audit 2024-04-07T16:16:15.108866+0000 mon.a (mon.0) 871 : audit 1 from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T16:16:15.863 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:15 smithi144 bash[23522]: audit 2024-04-07T16:16:15.109002+0000 mon.b (mon.1) 207 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:16:15.863 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:15 smithi144 bash[23522]: audit 2024-04-07T16:16:15.109369+0000 mon.b (mon.1) 208 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:16:15.863 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:15 smithi144 bash[23522]: audit 2024-04-07T16:16:15.112687+0000 mon.b (mon.1) 209 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:16:15.863 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:15 smithi144 bash[23522]: audit 2024-04-07T16:16:15.113079+0000 mon.b (mon.1) 210 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:16:15.863 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:15 smithi144 bash[23522]: audit 2024-04-07T16:16:15.113454+0000 mon.b (mon.1) 211 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:16:15.863 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:15 smithi144 bash[23522]: audit 2024-04-07T16:16:15.117175+0000 mon.c (mon.2) 47 : audit 1 from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T16:16:15.863 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:15 smithi144 bash[23522]: audit 2024-04-07T16:16:15.117316+0000 mon.c (mon.2) 48 : audit 1 from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T16:16:15.864 INFO:tasks.workunit.client.0.smithi100.stdout:{ 2024-04-07T16:16:15.864 INFO:tasks.workunit.client.0.smithi100.stdout: "device_report": { 2024-04-07T16:16:15.864 INFO:tasks.workunit.client.0.smithi100.stdout: "251e7b66-f4fa-11ee-8447-0cc47a8fd733": { 2024-04-07T16:16:15.864 INFO:tasks.workunit.client.0.smithi100.stdout: "INTEL_SSDPEDMD400G4_251ffd38-f4fa-11ee-8447-0cc47a8fd733": { 2024-04-07T16:16:15.864 INFO:tasks.workunit.client.0.smithi100.stdout: "20240407-160729": { 2024-04-07T16:16:15.864 INFO:tasks.workunit.client.0.smithi100.stdout: "device": { 2024-04-07T16:16:15.864 INFO:tasks.workunit.client.0.smithi100.stdout: "info_name": "/dev/nvme0n1", 2024-04-07T16:16:15.864 INFO:tasks.workunit.client.0.smithi100.stdout: "name": "/dev/nvme0n1", 2024-04-07T16:16:15.864 INFO:tasks.workunit.client.0.smithi100.stdout: "protocol": "NVMe", 2024-04-07T16:16:15.864 INFO:tasks.workunit.client.0.smithi100.stdout: "type": "nvme" 2024-04-07T16:16:15.864 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.864 INFO:tasks.workunit.client.0.smithi100.stdout: "firmware_version": "8DV101H0", 2024-04-07T16:16:15.864 INFO:tasks.workunit.client.0.smithi100.stdout: "host_id": "251e7b66-f4fa-11ee-8447-0cc47a8fd733", 2024-04-07T16:16:15.865 INFO:tasks.workunit.client.0.smithi100.stdout: "json_format_version": [ 2024-04-07T16:16:15.865 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T16:16:15.865 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T16:16:15.865 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:15.865 INFO:tasks.workunit.client.0.smithi100.stdout: "local_time": { 2024-04-07T16:16:15.865 INFO:tasks.workunit.client.0.smithi100.stdout: "asctime": "Sun Apr 7 16:07:28 2024 UTC", 2024-04-07T16:16:15.865 INFO:tasks.workunit.client.0.smithi100.stdout: "time_t": 1712506048 2024-04-07T16:16:15.865 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.865 INFO:tasks.workunit.client.0.smithi100.stdout: "logical_block_size": 512, 2024-04-07T16:16:15.865 INFO:tasks.workunit.client.0.smithi100.stdout: "model_name": "INTEL SSDPEDMD400G4", 2024-04-07T16:16:15.865 INFO:tasks.workunit.client.0.smithi100.stdout: "nvme_controller_id": 0, 2024-04-07T16:16:15.865 INFO:tasks.workunit.client.0.smithi100.stdout: "nvme_ieee_oui_identifier": 6083300, 2024-04-07T16:16:15.865 INFO:tasks.workunit.client.0.smithi100.stdout: "nvme_namespaces": [ 2024-04-07T16:16:15.865 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:15.866 INFO:tasks.workunit.client.0.smithi100.stdout: "capacity": { 2024-04-07T16:16:15.866 INFO:tasks.workunit.client.0.smithi100.stdout: "blocks": 781422768, 2024-04-07T16:16:15.866 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 400088457216 2024-04-07T16:16:15.866 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.866 INFO:tasks.workunit.client.0.smithi100.stdout: "formatted_lba_size": 512, 2024-04-07T16:16:15.866 INFO:tasks.workunit.client.0.smithi100.stdout: "id": 1, 2024-04-07T16:16:15.866 INFO:tasks.workunit.client.0.smithi100.stdout: "size": { 2024-04-07T16:16:15.866 INFO:tasks.workunit.client.0.smithi100.stdout: "blocks": 781422768, 2024-04-07T16:16:15.866 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 400088457216 2024-04-07T16:16:15.866 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.866 INFO:tasks.workunit.client.0.smithi100.stdout: "utilization": { 2024-04-07T16:16:15.866 INFO:tasks.workunit.client.0.smithi100.stdout: "blocks": 781422768, 2024-04-07T16:16:15.866 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 400088457216 2024-04-07T16:16:15.866 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:15.866 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:15.867 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:15.867 INFO:tasks.workunit.client.0.smithi100.stdout: "nvme_number_of_namespaces": 1, 2024-04-07T16:16:15.867 INFO:tasks.workunit.client.0.smithi100.stdout: "nvme_pci_vendor": { 2024-04-07T16:16:15.867 INFO:tasks.workunit.client.0.smithi100.stdout: "id": 32902, 2024-04-07T16:16:15.867 INFO:tasks.workunit.client.0.smithi100.stdout: "subsystem_id": 32902 2024-04-07T16:16:15.867 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.867 INFO:tasks.workunit.client.0.smithi100.stdout: "nvme_smart_health_information_add_log": { 2024-04-07T16:16:15.867 INFO:tasks.workunit.client.0.smithi100.stdout: "Device stats": { 2024-04-07T16:16:15.867 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_fw_downgrades": { 2024-04-07T16:16:15.867 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 0, 2024-04-07T16:16:15.867 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 0 2024-04-07T16:16:15.867 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.867 INFO:tasks.workunit.client.0.smithi100.stdout: "crc_error_count": { 2024-04-07T16:16:15.867 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 100, 2024-04-07T16:16:15.867 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 0 2024-04-07T16:16:15.868 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.868 INFO:tasks.workunit.client.0.smithi100.stdout: "end_to_end_error_detection_count": { 2024-04-07T16:16:15.868 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 100, 2024-04-07T16:16:15.868 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 0 2024-04-07T16:16:15.868 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.868 INFO:tasks.workunit.client.0.smithi100.stdout: "erase_fail_count": { 2024-04-07T16:16:15.868 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 100, 2024-04-07T16:16:15.868 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 0 2024-04-07T16:16:15.868 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.868 INFO:tasks.workunit.client.0.smithi100.stdout: "host_bytes_written": { 2024-04-07T16:16:15.868 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 100, 2024-04-07T16:16:15.868 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 29568720 2024-04-07T16:16:15.868 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.868 INFO:tasks.workunit.client.0.smithi100.stdout: "host_ctx_wear_used": { 2024-04-07T16:16:15.868 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 100, 2024-04-07T16:16:15.869 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 1 2024-04-07T16:16:15.869 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.869 INFO:tasks.workunit.client.0.smithi100.stdout: "media_bytes_read": { 2024-04-07T16:16:15.869 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 0, 2024-04-07T16:16:15.869 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 0 2024-04-07T16:16:15.869 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.869 INFO:tasks.workunit.client.0.smithi100.stdout: "nand_bytes_written": { 2024-04-07T16:16:15.869 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 100, 2024-04-07T16:16:15.869 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 47802746 2024-04-07T16:16:15.869 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.869 INFO:tasks.workunit.client.0.smithi100.stdout: "perf_stat_indicator": { 2024-04-07T16:16:15.869 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 0, 2024-04-07T16:16:15.869 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 0 2024-04-07T16:16:15.869 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.869 INFO:tasks.workunit.client.0.smithi100.stdout: "pll_lock_loss_count": { 2024-04-07T16:16:15.870 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 100, 2024-04-07T16:16:15.870 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 0 2024-04-07T16:16:15.870 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.870 INFO:tasks.workunit.client.0.smithi100.stdout: "program_fail_count": { 2024-04-07T16:16:15.870 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 100, 2024-04-07T16:16:15.870 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 0 2024-04-07T16:16:15.870 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.870 INFO:tasks.workunit.client.0.smithi100.stdout: "re_alloc_sectr_cnt": { 2024-04-07T16:16:15.870 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 0, 2024-04-07T16:16:15.870 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 0 2024-04-07T16:16:15.870 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.870 INFO:tasks.workunit.client.0.smithi100.stdout: "retry_buffer_overflow_count": { 2024-04-07T16:16:15.870 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 100, 2024-04-07T16:16:15.870 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 0 2024-04-07T16:16:15.870 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.871 INFO:tasks.workunit.client.0.smithi100.stdout: "soft_ecc_err_rate": { 2024-04-07T16:16:15.871 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 0, 2024-04-07T16:16:15.871 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 0 2024-04-07T16:16:15.871 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.871 INFO:tasks.workunit.client.0.smithi100.stdout: "thermal_throttle_status": { 2024-04-07T16:16:15.871 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 100, 2024-04-07T16:16:15.871 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": { 2024-04-07T16:16:15.871 INFO:tasks.workunit.client.0.smithi100.stdout: "cnt": 0, 2024-04-07T16:16:15.871 INFO:tasks.workunit.client.0.smithi100.stdout: "pct": 0 2024-04-07T16:16:15.871 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:15.871 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.871 INFO:tasks.workunit.client.0.smithi100.stdout: "timed_workload_host_reads": { 2024-04-07T16:16:15.871 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 100, 2024-04-07T16:16:15.871 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 65535 2024-04-07T16:16:15.871 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.872 INFO:tasks.workunit.client.0.smithi100.stdout: "timed_workload_media_wear": { 2024-04-07T16:16:15.872 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 100, 2024-04-07T16:16:15.872 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 64 2024-04-07T16:16:15.872 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.872 INFO:tasks.workunit.client.0.smithi100.stdout: "timed_workload_timer": { 2024-04-07T16:16:15.872 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 100, 2024-04-07T16:16:15.872 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 65535 2024-04-07T16:16:15.872 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.872 INFO:tasks.workunit.client.0.smithi100.stdout: "unexp_power_loss": { 2024-04-07T16:16:15.872 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 0, 2024-04-07T16:16:15.872 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 0 2024-04-07T16:16:15.872 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.872 INFO:tasks.workunit.client.0.smithi100.stdout: "wear_leveling": { 2024-04-07T16:16:15.872 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 90, 2024-04-07T16:16:15.872 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": { 2024-04-07T16:16:15.872 INFO:tasks.workunit.client.0.smithi100.stdout: "avg": 3209, 2024-04-07T16:16:15.873 INFO:tasks.workunit.client.0.smithi100.stdout: "max": 3225, 2024-04-07T16:16:15.873 INFO:tasks.workunit.client.0.smithi100.stdout: "min": 3194 2024-04-07T16:16:15.873 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:15.873 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:15.873 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.873 INFO:tasks.workunit.client.0.smithi100.stdout: "Intel Smart log": "nvme0n1" 2024-04-07T16:16:15.873 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.873 INFO:tasks.workunit.client.0.smithi100.stdout: "nvme_smart_health_information_log": { 2024-04-07T16:16:15.873 INFO:tasks.workunit.client.0.smithi100.stdout: "available_spare": 100, 2024-04-07T16:16:15.873 INFO:tasks.workunit.client.0.smithi100.stdout: "available_spare_threshold": 10, 2024-04-07T16:16:15.873 INFO:tasks.workunit.client.0.smithi100.stdout: "controller_busy_time": 4, 2024-04-07T16:16:15.873 INFO:tasks.workunit.client.0.smithi100.stdout: "critical_warning": 0, 2024-04-07T16:16:15.873 INFO:tasks.workunit.client.0.smithi100.stdout: "data_units_read": 385784931, 2024-04-07T16:16:15.873 INFO:tasks.workunit.client.0.smithi100.stdout: "data_units_written": 1937815661, 2024-04-07T16:16:15.873 INFO:tasks.workunit.client.0.smithi100.stdout: "host_reads": 5771442418, 2024-04-07T16:16:15.874 INFO:tasks.workunit.client.0.smithi100.stdout: "host_writes": 31722089956, 2024-04-07T16:16:15.874 INFO:tasks.workunit.client.0.smithi100.stdout: "media_errors": 0, 2024-04-07T16:16:15.874 INFO:tasks.workunit.client.0.smithi100.stdout: "num_err_log_entries": 0, 2024-04-07T16:16:15.874 INFO:tasks.workunit.client.0.smithi100.stdout: "percentage_used": 10, 2024-04-07T16:16:15.874 INFO:tasks.workunit.client.0.smithi100.stdout: "power_cycles": 45774, 2024-04-07T16:16:15.874 INFO:tasks.workunit.client.0.smithi100.stdout: "power_on_hours": 57978, 2024-04-07T16:16:15.874 INFO:tasks.workunit.client.0.smithi100.stdout: "temperature": 24, 2024-04-07T16:16:15.874 INFO:tasks.workunit.client.0.smithi100.stdout: "unsafe_shutdowns": 210 2024-04-07T16:16:15.874 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.874 INFO:tasks.workunit.client.0.smithi100.stdout: "nvme_vendor": "intel", 2024-04-07T16:16:15.874 INFO:tasks.workunit.client.0.smithi100.stdout: "power_cycle_count": 45774, 2024-04-07T16:16:15.874 INFO:tasks.workunit.client.0.smithi100.stdout: "power_on_time": { 2024-04-07T16:16:15.874 INFO:tasks.workunit.client.0.smithi100.stdout: "hours": 57978 2024-04-07T16:16:15.874 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.874 INFO:tasks.workunit.client.0.smithi100.stdout: "serial_number": "deleted", 2024-04-07T16:16:15.875 INFO:tasks.workunit.client.0.smithi100.stdout: "smart_status": { 2024-04-07T16:16:15.875 INFO:tasks.workunit.client.0.smithi100.stdout: "nvme": { 2024-04-07T16:16:15.875 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:15.875 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.875 INFO:tasks.workunit.client.0.smithi100.stdout: "passed": true 2024-04-07T16:16:15.875 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.875 INFO:tasks.workunit.client.0.smithi100.stdout: "smartctl": { 2024-04-07T16:16:15.875 INFO:tasks.workunit.client.0.smithi100.stdout: "argv": [ 2024-04-07T16:16:15.875 INFO:tasks.workunit.client.0.smithi100.stdout: "smartctl", 2024-04-07T16:16:15.875 INFO:tasks.workunit.client.0.smithi100.stdout: "-x", 2024-04-07T16:16:15.875 INFO:tasks.workunit.client.0.smithi100.stdout: "--json=o", 2024-04-07T16:16:15.875 INFO:tasks.workunit.client.0.smithi100.stdout: "/dev/nvme0n1" 2024-04-07T16:16:15.875 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:15.875 INFO:tasks.workunit.client.0.smithi100.stdout: "build_info": "(local build)", 2024-04-07T16:16:15.875 INFO:tasks.workunit.client.0.smithi100.stdout: "exit_status": 0, 2024-04-07T16:16:15.875 INFO:tasks.workunit.client.0.smithi100.stdout: "output": [ 2024-04-07T16:16:15.876 INFO:tasks.workunit.client.0.smithi100.stdout: "smartctl 7.1 2019-12-30 r5022 [x86_64-linux-5.15.0-101-generic] (local build)", 2024-04-07T16:16:15.876 INFO:tasks.workunit.client.0.smithi100.stdout: "Copyright (C) 2002-19, Bruce Allen, Christian Franke, www.smartmontools.org", 2024-04-07T16:16:15.876 INFO:tasks.workunit.client.0.smithi100.stdout: "", 2024-04-07T16:16:15.876 INFO:tasks.workunit.client.0.smithi100.stdout: "=== START OF INFORMATION SECTION ===", 2024-04-07T16:16:15.876 INFO:tasks.workunit.client.0.smithi100.stdout: "Model Number: INTEL SSDPEDMD400G4", 2024-04-07T16:16:15.876 INFO:tasks.workunit.client.0.smithi100.stdout: "Serial Number: deleted", 2024-04-07T16:16:15.876 INFO:tasks.workunit.client.0.smithi100.stdout: "Firmware Version: 8DV101H0", 2024-04-07T16:16:15.876 INFO:tasks.workunit.client.0.smithi100.stdout: "PCI Vendor/Subsystem ID: 0x8086", 2024-04-07T16:16:15.876 INFO:tasks.workunit.client.0.smithi100.stdout: "IEEE OUI Identifier: 0x5cd2e4", 2024-04-07T16:16:15.876 INFO:tasks.workunit.client.0.smithi100.stdout: "Controller ID: 0", 2024-04-07T16:16:15.876 INFO:tasks.workunit.client.0.smithi100.stdout: "Number of Namespaces: 1", 2024-04-07T16:16:15.876 INFO:tasks.workunit.client.0.smithi100.stdout: "Namespace 1 Size/Capacity: 400,088,457,216 [400 GB]", 2024-04-07T16:16:15.876 INFO:tasks.workunit.client.0.smithi100.stdout: "Namespace 1 Formatted LBA Size: 512", 2024-04-07T16:16:15.877 INFO:tasks.workunit.client.0.smithi100.stdout: "Local Time is: Sun Apr 7 16:07:28 2024 UTC", 2024-04-07T16:16:15.877 INFO:tasks.workunit.client.0.smithi100.stdout: "Firmware Updates (0x02): 1 Slot", 2024-04-07T16:16:15.877 INFO:tasks.workunit.client.0.smithi100.stdout: "Optional Admin Commands (0x0006): Format Frmw_DL", 2024-04-07T16:16:15.877 INFO:tasks.workunit.client.0.smithi100.stdout: "Optional NVM Commands (0x0006): Wr_Unc DS_Mngmt", 2024-04-07T16:16:15.877 INFO:tasks.workunit.client.0.smithi100.stdout: "Maximum Data Transfer Size: 32 Pages", 2024-04-07T16:16:15.877 INFO:tasks.workunit.client.0.smithi100.stdout: "", 2024-04-07T16:16:15.877 INFO:tasks.workunit.client.0.smithi100.stdout: "Supported Power States", 2024-04-07T16:16:15.877 INFO:tasks.workunit.client.0.smithi100.stdout: "St Op Max Active Idle RL RT WL WT Ent_Lat Ex_Lat", 2024-04-07T16:16:15.877 INFO:tasks.workunit.client.0.smithi100.stdout: " 0 + 25.00W - - 0 0 0 0 0 0", 2024-04-07T16:16:15.877 INFO:tasks.workunit.client.0.smithi100.stdout: "", 2024-04-07T16:16:15.877 INFO:tasks.workunit.client.0.smithi100.stdout: "Supported LBA Sizes (NSID 0x1)", 2024-04-07T16:16:15.877 INFO:tasks.workunit.client.0.smithi100.stdout: "Id Fmt Data Metadt Rel_Perf", 2024-04-07T16:16:15.877 INFO:tasks.workunit.client.0.smithi100.stdout: " 0 + 512 0 2", 2024-04-07T16:16:15.877 INFO:tasks.workunit.client.0.smithi100.stdout: " 1 - 512 8 2", 2024-04-07T16:16:15.877 INFO:tasks.workunit.client.0.smithi100.stdout: " 2 - 512 16 2", 2024-04-07T16:16:15.878 INFO:tasks.workunit.client.0.smithi100.stdout: " 3 - 4096 0 0", 2024-04-07T16:16:15.878 INFO:tasks.workunit.client.0.smithi100.stdout: " 4 - 4096 8 0", 2024-04-07T16:16:15.878 INFO:tasks.workunit.client.0.smithi100.stdout: " 5 - 4096 64 0", 2024-04-07T16:16:15.878 INFO:tasks.workunit.client.0.smithi100.stdout: " 6 - 4096 128 0", 2024-04-07T16:16:15.878 INFO:tasks.workunit.client.0.smithi100.stdout: "", 2024-04-07T16:16:15.878 INFO:tasks.workunit.client.0.smithi100.stdout: "=== START OF SMART DATA SECTION ===", 2024-04-07T16:16:15.878 INFO:tasks.workunit.client.0.smithi100.stdout: "SMART overall-health self-assessment test result: PASSED", 2024-04-07T16:16:15.878 INFO:tasks.workunit.client.0.smithi100.stdout: "", 2024-04-07T16:16:15.878 INFO:tasks.workunit.client.0.smithi100.stdout: "SMART/Health Information (NVMe Log 0x02)", 2024-04-07T16:16:15.878 INFO:tasks.workunit.client.0.smithi100.stdout: "Critical Warning: 0x00", 2024-04-07T16:16:15.878 INFO:tasks.workunit.client.0.smithi100.stdout: "Temperature: 24 Celsius", 2024-04-07T16:16:15.878 INFO:tasks.workunit.client.0.smithi100.stdout: "Available Spare: 100%", 2024-04-07T16:16:15.878 INFO:tasks.workunit.client.0.smithi100.stdout: "Available Spare Threshold: 10%", 2024-04-07T16:16:15.878 INFO:tasks.workunit.client.0.smithi100.stdout: "Percentage Used: 10%", 2024-04-07T16:16:15.878 INFO:tasks.workunit.client.0.smithi100.stdout: "Data Units Read: 385,784,931 [197 TB]", 2024-04-07T16:16:15.879 INFO:tasks.workunit.client.0.smithi100.stdout: "Data Units Written: 1,937,815,661 [992 TB]", 2024-04-07T16:16:15.879 INFO:tasks.workunit.client.0.smithi100.stdout: "Host Read Commands: 5,771,442,418", 2024-04-07T16:16:15.879 INFO:tasks.workunit.client.0.smithi100.stdout: "Host Write Commands: 31,722,089,956", 2024-04-07T16:16:15.879 INFO:tasks.workunit.client.0.smithi100.stdout: "Controller Busy Time: 4", 2024-04-07T16:16:15.879 INFO:tasks.workunit.client.0.smithi100.stdout: "Power Cycles: 45,774", 2024-04-07T16:16:15.879 INFO:tasks.workunit.client.0.smithi100.stdout: "Power On Hours: 57,978", 2024-04-07T16:16:15.879 INFO:tasks.workunit.client.0.smithi100.stdout: "Unsafe Shutdowns: 210", 2024-04-07T16:16:15.879 INFO:tasks.workunit.client.0.smithi100.stdout: "Media and Data Integrity Errors: 0", 2024-04-07T16:16:15.879 INFO:tasks.workunit.client.0.smithi100.stdout: "Error Information Log Entries: 0", 2024-04-07T16:16:15.879 INFO:tasks.workunit.client.0.smithi100.stdout: "", 2024-04-07T16:16:15.879 INFO:tasks.workunit.client.0.smithi100.stdout: "Error Information (NVMe Log 0x01, max 64 entries)", 2024-04-07T16:16:15.879 INFO:tasks.workunit.client.0.smithi100.stdout: "No Errors Logged", 2024-04-07T16:16:15.879 INFO:tasks.workunit.client.0.smithi100.stdout: "" 2024-04-07T16:16:15.879 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:15.879 INFO:tasks.workunit.client.0.smithi100.stdout: "platform_info": "x86_64-linux-5.15.0-101-generic", 2024-04-07T16:16:15.880 INFO:tasks.workunit.client.0.smithi100.stdout: "svn_revision": "5022", 2024-04-07T16:16:15.880 INFO:tasks.workunit.client.0.smithi100.stdout: "version": [ 2024-04-07T16:16:15.880 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T16:16:15.880 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T16:16:15.880 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T16:16:15.880 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.880 INFO:tasks.workunit.client.0.smithi100.stdout: "temperature": { 2024-04-07T16:16:15.880 INFO:tasks.workunit.client.0.smithi100.stdout: "current": 24 2024-04-07T16:16:15.880 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.880 INFO:tasks.workunit.client.0.smithi100.stdout: "user_capacity": { 2024-04-07T16:16:15.880 INFO:tasks.workunit.client.0.smithi100.stdout: "blocks": 781422768, 2024-04-07T16:16:15.880 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 400088457216 2024-04-07T16:16:15.880 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:15.880 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:15.880 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:15.880 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.881 INFO:tasks.workunit.client.0.smithi100.stdout: "25233610-f4fa-11ee-8447-0cc47a8fd733": { 2024-04-07T16:16:15.881 INFO:tasks.workunit.client.0.smithi100.stdout: "INTEL_SSDPEDMD400G4_2524b6c0-f4fa-11ee-8447-0cc47a8fd733": { 2024-04-07T16:16:15.881 INFO:tasks.workunit.client.0.smithi100.stdout: "20240407-160732": { 2024-04-07T16:16:15.881 INFO:tasks.workunit.client.0.smithi100.stdout: "device": { 2024-04-07T16:16:15.881 INFO:tasks.workunit.client.0.smithi100.stdout: "info_name": "/dev/nvme0n1", 2024-04-07T16:16:15.881 INFO:tasks.workunit.client.0.smithi100.stdout: "name": "/dev/nvme0n1", 2024-04-07T16:16:15.881 INFO:tasks.workunit.client.0.smithi100.stdout: "protocol": "NVMe", 2024-04-07T16:16:15.881 INFO:tasks.workunit.client.0.smithi100.stdout: "type": "nvme" 2024-04-07T16:16:15.881 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.881 INFO:tasks.workunit.client.0.smithi100.stdout: "firmware_version": "8DV101H0", 2024-04-07T16:16:15.881 INFO:tasks.workunit.client.0.smithi100.stdout: "host_id": "25233610-f4fa-11ee-8447-0cc47a8fd733", 2024-04-07T16:16:15.881 INFO:tasks.workunit.client.0.smithi100.stdout: "json_format_version": [ 2024-04-07T16:16:15.881 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T16:16:15.881 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T16:16:15.881 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:15.882 INFO:tasks.workunit.client.0.smithi100.stdout: "local_time": { 2024-04-07T16:16:15.882 INFO:tasks.workunit.client.0.smithi100.stdout: "asctime": "Sun Apr 7 16:07:32 2024 UTC", 2024-04-07T16:16:15.882 INFO:tasks.workunit.client.0.smithi100.stdout: "time_t": 1712506052 2024-04-07T16:16:15.882 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.882 INFO:tasks.workunit.client.0.smithi100.stdout: "logical_block_size": 512, 2024-04-07T16:16:15.882 INFO:tasks.workunit.client.0.smithi100.stdout: "model_name": "INTEL SSDPEDMD400G4", 2024-04-07T16:16:15.882 INFO:tasks.workunit.client.0.smithi100.stdout: "nvme_controller_id": 0, 2024-04-07T16:16:15.882 INFO:tasks.workunit.client.0.smithi100.stdout: "nvme_ieee_oui_identifier": 6083300, 2024-04-07T16:16:15.882 INFO:tasks.workunit.client.0.smithi100.stdout: "nvme_namespaces": [ 2024-04-07T16:16:15.882 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:15.882 INFO:tasks.workunit.client.0.smithi100.stdout: "capacity": { 2024-04-07T16:16:15.882 INFO:tasks.workunit.client.0.smithi100.stdout: "blocks": 781422768, 2024-04-07T16:16:15.882 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 400088457216 2024-04-07T16:16:15.882 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.882 INFO:tasks.workunit.client.0.smithi100.stdout: "formatted_lba_size": 512, 2024-04-07T16:16:15.883 INFO:tasks.workunit.client.0.smithi100.stdout: "id": 1, 2024-04-07T16:16:15.883 INFO:tasks.workunit.client.0.smithi100.stdout: "size": { 2024-04-07T16:16:15.883 INFO:tasks.workunit.client.0.smithi100.stdout: "blocks": 781422768, 2024-04-07T16:16:15.883 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 400088457216 2024-04-07T16:16:15.883 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.883 INFO:tasks.workunit.client.0.smithi100.stdout: "utilization": { 2024-04-07T16:16:15.883 INFO:tasks.workunit.client.0.smithi100.stdout: "blocks": 781422768, 2024-04-07T16:16:15.883 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 400088457216 2024-04-07T16:16:15.883 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:15.883 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:15.883 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:15.883 INFO:tasks.workunit.client.0.smithi100.stdout: "nvme_number_of_namespaces": 1, 2024-04-07T16:16:15.883 INFO:tasks.workunit.client.0.smithi100.stdout: "nvme_pci_vendor": { 2024-04-07T16:16:15.883 INFO:tasks.workunit.client.0.smithi100.stdout: "id": 32902, 2024-04-07T16:16:15.883 INFO:tasks.workunit.client.0.smithi100.stdout: "subsystem_id": 32902 2024-04-07T16:16:15.884 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.884 INFO:tasks.workunit.client.0.smithi100.stdout: "nvme_smart_health_information_add_log": { 2024-04-07T16:16:15.884 INFO:tasks.workunit.client.0.smithi100.stdout: "Device stats": { 2024-04-07T16:16:15.884 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_fw_downgrades": { 2024-04-07T16:16:15.884 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 0, 2024-04-07T16:16:15.884 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 0 2024-04-07T16:16:15.884 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.884 INFO:tasks.workunit.client.0.smithi100.stdout: "crc_error_count": { 2024-04-07T16:16:15.884 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 100, 2024-04-07T16:16:15.885 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 0 2024-04-07T16:16:15.885 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.885 INFO:tasks.workunit.client.0.smithi100.stdout: "end_to_end_error_detection_count": { 2024-04-07T16:16:15.885 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 100, 2024-04-07T16:16:15.885 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 0 2024-04-07T16:16:15.885 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.885 INFO:tasks.workunit.client.0.smithi100.stdout: "erase_fail_count": { 2024-04-07T16:16:15.886 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 100, 2024-04-07T16:16:15.886 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 0 2024-04-07T16:16:15.886 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.886 INFO:tasks.workunit.client.0.smithi100.stdout: "host_bytes_written": { 2024-04-07T16:16:15.886 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 100, 2024-04-07T16:16:15.886 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 25060599 2024-04-07T16:16:15.886 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.886 INFO:tasks.workunit.client.0.smithi100.stdout: "host_ctx_wear_used": { 2024-04-07T16:16:15.886 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 100, 2024-04-07T16:16:15.886 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 1 2024-04-07T16:16:15.886 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.886 INFO:tasks.workunit.client.0.smithi100.stdout: "media_bytes_read": { 2024-04-07T16:16:15.887 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 0, 2024-04-07T16:16:15.887 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 0 2024-04-07T16:16:15.887 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.887 INFO:tasks.workunit.client.0.smithi100.stdout: "nand_bytes_written": { 2024-04-07T16:16:15.887 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 100, 2024-04-07T16:16:15.887 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 42751216 2024-04-07T16:16:15.887 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.887 INFO:tasks.workunit.client.0.smithi100.stdout: "perf_stat_indicator": { 2024-04-07T16:16:15.887 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 0, 2024-04-07T16:16:15.887 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 0 2024-04-07T16:16:15.887 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.887 INFO:tasks.workunit.client.0.smithi100.stdout: "pll_lock_loss_count": { 2024-04-07T16:16:15.887 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 100, 2024-04-07T16:16:15.887 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 0 2024-04-07T16:16:15.888 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.888 INFO:tasks.workunit.client.0.smithi100.stdout: "program_fail_count": { 2024-04-07T16:16:15.888 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 100, 2024-04-07T16:16:15.888 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 0 2024-04-07T16:16:15.888 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.888 INFO:tasks.workunit.client.0.smithi100.stdout: "re_alloc_sectr_cnt": { 2024-04-07T16:16:15.888 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 0, 2024-04-07T16:16:15.888 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 0 2024-04-07T16:16:15.888 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.888 INFO:tasks.workunit.client.0.smithi100.stdout: "retry_buffer_overflow_count": { 2024-04-07T16:16:15.889 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 100, 2024-04-07T16:16:15.889 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 0 2024-04-07T16:16:15.889 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.889 INFO:tasks.workunit.client.0.smithi100.stdout: "soft_ecc_err_rate": { 2024-04-07T16:16:15.889 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 0, 2024-04-07T16:16:15.889 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 0 2024-04-07T16:16:15.889 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.889 INFO:tasks.workunit.client.0.smithi100.stdout: "thermal_throttle_status": { 2024-04-07T16:16:15.889 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 100, 2024-04-07T16:16:15.889 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": { 2024-04-07T16:16:15.889 INFO:tasks.workunit.client.0.smithi100.stdout: "cnt": 0, 2024-04-07T16:16:15.889 INFO:tasks.workunit.client.0.smithi100.stdout: "pct": 0 2024-04-07T16:16:15.889 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:15.889 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.889 INFO:tasks.workunit.client.0.smithi100.stdout: "timed_workload_host_reads": { 2024-04-07T16:16:15.890 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 100, 2024-04-07T16:16:15.890 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 65535 2024-04-07T16:16:15.890 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.890 INFO:tasks.workunit.client.0.smithi100.stdout: "timed_workload_media_wear": { 2024-04-07T16:16:15.890 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 100, 2024-04-07T16:16:15.890 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 64 2024-04-07T16:16:15.890 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.890 INFO:tasks.workunit.client.0.smithi100.stdout: "timed_workload_timer": { 2024-04-07T16:16:15.890 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 100, 2024-04-07T16:16:15.890 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 65535 2024-04-07T16:16:15.890 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.890 INFO:tasks.workunit.client.0.smithi100.stdout: "unexp_power_loss": { 2024-04-07T16:16:15.890 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 0, 2024-04-07T16:16:15.890 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 0 2024-04-07T16:16:15.891 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.891 INFO:tasks.workunit.client.0.smithi100.stdout: "wear_leveling": { 2024-04-07T16:16:15.891 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 91, 2024-04-07T16:16:15.891 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": { 2024-04-07T16:16:15.891 INFO:tasks.workunit.client.0.smithi100.stdout: "avg": 2895, 2024-04-07T16:16:15.891 INFO:tasks.workunit.client.0.smithi100.stdout: "max": 2907, 2024-04-07T16:16:15.891 INFO:tasks.workunit.client.0.smithi100.stdout: "min": 2884 2024-04-07T16:16:15.891 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:15.891 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:15.891 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.891 INFO:tasks.workunit.client.0.smithi100.stdout: "Intel Smart log": "nvme0n1" 2024-04-07T16:16:15.891 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.891 INFO:tasks.workunit.client.0.smithi100.stdout: "nvme_smart_health_information_log": { 2024-04-07T16:16:15.891 INFO:tasks.workunit.client.0.smithi100.stdout: "available_spare": 100, 2024-04-07T16:16:15.891 INFO:tasks.workunit.client.0.smithi100.stdout: "available_spare_threshold": 10, 2024-04-07T16:16:15.892 INFO:tasks.workunit.client.0.smithi100.stdout: "controller_busy_time": 0, 2024-04-07T16:16:15.892 INFO:tasks.workunit.client.0.smithi100.stdout: "critical_warning": 0, 2024-04-07T16:16:15.892 INFO:tasks.workunit.client.0.smithi100.stdout: "data_units_read": 359011585, 2024-04-07T16:16:15.892 INFO:tasks.workunit.client.0.smithi100.stdout: "data_units_written": 1642371481, 2024-04-07T16:16:15.892 INFO:tasks.workunit.client.0.smithi100.stdout: "host_reads": 5750712954, 2024-04-07T16:16:15.892 INFO:tasks.workunit.client.0.smithi100.stdout: "host_writes": 28276536650, 2024-04-07T16:16:15.892 INFO:tasks.workunit.client.0.smithi100.stdout: "media_errors": 0, 2024-04-07T16:16:15.892 INFO:tasks.workunit.client.0.smithi100.stdout: "num_err_log_entries": 0, 2024-04-07T16:16:15.892 INFO:tasks.workunit.client.0.smithi100.stdout: "percentage_used": 9, 2024-04-07T16:16:15.892 INFO:tasks.workunit.client.0.smithi100.stdout: "power_cycles": 47059, 2024-04-07T16:16:15.892 INFO:tasks.workunit.client.0.smithi100.stdout: "power_on_hours": 54821, 2024-04-07T16:16:15.892 INFO:tasks.workunit.client.0.smithi100.stdout: "temperature": 25, 2024-04-07T16:16:15.892 INFO:tasks.workunit.client.0.smithi100.stdout: "unsafe_shutdowns": 12 2024-04-07T16:16:15.892 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.892 INFO:tasks.workunit.client.0.smithi100.stdout: "nvme_vendor": "intel", 2024-04-07T16:16:15.893 INFO:tasks.workunit.client.0.smithi100.stdout: "power_cycle_count": 47059, 2024-04-07T16:16:15.893 INFO:tasks.workunit.client.0.smithi100.stdout: "power_on_time": { 2024-04-07T16:16:15.893 INFO:tasks.workunit.client.0.smithi100.stdout: "hours": 54821 2024-04-07T16:16:15.893 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.893 INFO:tasks.workunit.client.0.smithi100.stdout: "serial_number": "deleted", 2024-04-07T16:16:15.893 INFO:tasks.workunit.client.0.smithi100.stdout: "smart_status": { 2024-04-07T16:16:15.893 INFO:tasks.workunit.client.0.smithi100.stdout: "nvme": { 2024-04-07T16:16:15.893 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:15.893 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.893 INFO:tasks.workunit.client.0.smithi100.stdout: "passed": true 2024-04-07T16:16:15.893 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.893 INFO:tasks.workunit.client.0.smithi100.stdout: "smartctl": { 2024-04-07T16:16:15.894 INFO:tasks.workunit.client.0.smithi100.stdout: "argv": [ 2024-04-07T16:16:15.894 INFO:tasks.workunit.client.0.smithi100.stdout: "smartctl", 2024-04-07T16:16:15.894 INFO:tasks.workunit.client.0.smithi100.stdout: "-x", 2024-04-07T16:16:15.894 INFO:tasks.workunit.client.0.smithi100.stdout: "--json=o", 2024-04-07T16:16:15.894 INFO:tasks.workunit.client.0.smithi100.stdout: "/dev/nvme0n1" 2024-04-07T16:16:15.894 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:15.894 INFO:tasks.workunit.client.0.smithi100.stdout: "build_info": "(local build)", 2024-04-07T16:16:15.894 INFO:tasks.workunit.client.0.smithi100.stdout: "exit_status": 0, 2024-04-07T16:16:15.894 INFO:tasks.workunit.client.0.smithi100.stdout: "output": [ 2024-04-07T16:16:15.894 INFO:tasks.workunit.client.0.smithi100.stdout: "smartctl 7.1 2019-12-30 r5022 [x86_64-linux-5.15.0-101-generic] (local build)", 2024-04-07T16:16:15.894 INFO:tasks.workunit.client.0.smithi100.stdout: "Copyright (C) 2002-19, Bruce Allen, Christian Franke, www.smartmontools.org", 2024-04-07T16:16:15.894 INFO:tasks.workunit.client.0.smithi100.stdout: "", 2024-04-07T16:16:15.894 INFO:tasks.workunit.client.0.smithi100.stdout: "=== START OF INFORMATION SECTION ===", 2024-04-07T16:16:15.894 INFO:tasks.workunit.client.0.smithi100.stdout: "Model Number: INTEL SSDPEDMD400G4", 2024-04-07T16:16:15.895 INFO:tasks.workunit.client.0.smithi100.stdout: "Serial Number: deleted", 2024-04-07T16:16:15.895 INFO:tasks.workunit.client.0.smithi100.stdout: "Firmware Version: 8DV101H0", 2024-04-07T16:16:15.895 INFO:tasks.workunit.client.0.smithi100.stdout: "PCI Vendor/Subsystem ID: 0x8086", 2024-04-07T16:16:15.895 INFO:tasks.workunit.client.0.smithi100.stdout: "IEEE OUI Identifier: 0x5cd2e4", 2024-04-07T16:16:15.895 INFO:tasks.workunit.client.0.smithi100.stdout: "Controller ID: 0", 2024-04-07T16:16:15.895 INFO:tasks.workunit.client.0.smithi100.stdout: "Number of Namespaces: 1", 2024-04-07T16:16:15.895 INFO:tasks.workunit.client.0.smithi100.stdout: "Namespace 1 Size/Capacity: 400,088,457,216 [400 GB]", 2024-04-07T16:16:15.895 INFO:tasks.workunit.client.0.smithi100.stdout: "Namespace 1 Formatted LBA Size: 512", 2024-04-07T16:16:15.895 INFO:tasks.workunit.client.0.smithi100.stdout: "Local Time is: Sun Apr 7 16:07:32 2024 UTC", 2024-04-07T16:16:15.895 INFO:tasks.workunit.client.0.smithi100.stdout: "Firmware Updates (0x02): 1 Slot", 2024-04-07T16:16:15.895 INFO:tasks.workunit.client.0.smithi100.stdout: "Optional Admin Commands (0x0006): Format Frmw_DL", 2024-04-07T16:16:15.895 INFO:tasks.workunit.client.0.smithi100.stdout: "Optional NVM Commands (0x0006): Wr_Unc DS_Mngmt", 2024-04-07T16:16:15.895 INFO:tasks.workunit.client.0.smithi100.stdout: "Maximum Data Transfer Size: 32 Pages", 2024-04-07T16:16:15.895 INFO:tasks.workunit.client.0.smithi100.stdout: "", 2024-04-07T16:16:15.895 INFO:tasks.workunit.client.0.smithi100.stdout: "Supported Power States", 2024-04-07T16:16:15.896 INFO:tasks.workunit.client.0.smithi100.stdout: "St Op Max Active Idle RL RT WL WT Ent_Lat Ex_Lat", 2024-04-07T16:16:15.896 INFO:tasks.workunit.client.0.smithi100.stdout: " 0 + 25.00W - - 0 0 0 0 0 0", 2024-04-07T16:16:15.896 INFO:tasks.workunit.client.0.smithi100.stdout: "", 2024-04-07T16:16:15.896 INFO:tasks.workunit.client.0.smithi100.stdout: "Supported LBA Sizes (NSID 0x1)", 2024-04-07T16:16:15.896 INFO:tasks.workunit.client.0.smithi100.stdout: "Id Fmt Data Metadt Rel_Perf", 2024-04-07T16:16:15.896 INFO:tasks.workunit.client.0.smithi100.stdout: " 0 + 512 0 2", 2024-04-07T16:16:15.896 INFO:tasks.workunit.client.0.smithi100.stdout: " 1 - 512 8 2", 2024-04-07T16:16:15.896 INFO:tasks.workunit.client.0.smithi100.stdout: " 2 - 512 16 2", 2024-04-07T16:16:15.896 INFO:tasks.workunit.client.0.smithi100.stdout: " 3 - 4096 0 0", 2024-04-07T16:16:15.896 INFO:tasks.workunit.client.0.smithi100.stdout: " 4 - 4096 8 0", 2024-04-07T16:16:15.896 INFO:tasks.workunit.client.0.smithi100.stdout: " 5 - 4096 64 0", 2024-04-07T16:16:15.896 INFO:tasks.workunit.client.0.smithi100.stdout: " 6 - 4096 128 0", 2024-04-07T16:16:15.896 INFO:tasks.workunit.client.0.smithi100.stdout: "", 2024-04-07T16:16:15.896 INFO:tasks.workunit.client.0.smithi100.stdout: "=== START OF SMART DATA SECTION ===", 2024-04-07T16:16:15.896 INFO:tasks.workunit.client.0.smithi100.stdout: "SMART overall-health self-assessment test result: PASSED", 2024-04-07T16:16:15.897 INFO:tasks.workunit.client.0.smithi100.stdout: "", 2024-04-07T16:16:15.897 INFO:tasks.workunit.client.0.smithi100.stdout: "SMART/Health Information (NVMe Log 0x02)", 2024-04-07T16:16:15.897 INFO:tasks.workunit.client.0.smithi100.stdout: "Critical Warning: 0x00", 2024-04-07T16:16:15.897 INFO:tasks.workunit.client.0.smithi100.stdout: "Temperature: 25 Celsius", 2024-04-07T16:16:15.897 INFO:tasks.workunit.client.0.smithi100.stdout: "Available Spare: 100%", 2024-04-07T16:16:15.897 INFO:tasks.workunit.client.0.smithi100.stdout: "Available Spare Threshold: 10%", 2024-04-07T16:16:15.897 INFO:tasks.workunit.client.0.smithi100.stdout: "Percentage Used: 9%", 2024-04-07T16:16:15.897 INFO:tasks.workunit.client.0.smithi100.stdout: "Data Units Read: 359,011,585 [183 TB]", 2024-04-07T16:16:15.897 INFO:tasks.workunit.client.0.smithi100.stdout: "Data Units Written: 1,642,371,481 [840 TB]", 2024-04-07T16:16:15.897 INFO:tasks.workunit.client.0.smithi100.stdout: "Host Read Commands: 5,750,712,954", 2024-04-07T16:16:15.897 INFO:tasks.workunit.client.0.smithi100.stdout: "Host Write Commands: 28,276,536,650", 2024-04-07T16:16:15.897 INFO:tasks.workunit.client.0.smithi100.stdout: "Controller Busy Time: 0", 2024-04-07T16:16:15.897 INFO:tasks.workunit.client.0.smithi100.stdout: "Power Cycles: 47,059", 2024-04-07T16:16:15.897 INFO:tasks.workunit.client.0.smithi100.stdout: "Power On Hours: 54,821", 2024-04-07T16:16:15.897 INFO:tasks.workunit.client.0.smithi100.stdout: "Unsafe Shutdowns: 12", 2024-04-07T16:16:15.898 INFO:tasks.workunit.client.0.smithi100.stdout: "Media and Data Integrity Errors: 0", 2024-04-07T16:16:15.898 INFO:tasks.workunit.client.0.smithi100.stdout: "Error Information Log Entries: 0", 2024-04-07T16:16:15.898 INFO:tasks.workunit.client.0.smithi100.stdout: "", 2024-04-07T16:16:15.898 INFO:tasks.workunit.client.0.smithi100.stdout: "Error Information (NVMe Log 0x01, max 64 entries)", 2024-04-07T16:16:15.898 INFO:tasks.workunit.client.0.smithi100.stdout: "No Errors Logged", 2024-04-07T16:16:15.898 INFO:tasks.workunit.client.0.smithi100.stdout: "" 2024-04-07T16:16:15.898 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:15.898 INFO:tasks.workunit.client.0.smithi100.stdout: "platform_info": "x86_64-linux-5.15.0-101-generic", 2024-04-07T16:16:15.898 INFO:tasks.workunit.client.0.smithi100.stdout: "svn_revision": "5022", 2024-04-07T16:16:15.898 INFO:tasks.workunit.client.0.smithi100.stdout: "version": [ 2024-04-07T16:16:15.898 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T16:16:15.898 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T16:16:15.898 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T16:16:15.898 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.898 INFO:tasks.workunit.client.0.smithi100.stdout: "temperature": { 2024-04-07T16:16:15.899 INFO:tasks.workunit.client.0.smithi100.stdout: "current": 25 2024-04-07T16:16:15.899 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.899 INFO:tasks.workunit.client.0.smithi100.stdout: "user_capacity": { 2024-04-07T16:16:15.899 INFO:tasks.workunit.client.0.smithi100.stdout: "blocks": 781422768, 2024-04-07T16:16:15.899 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 400088457216 2024-04-07T16:16:15.899 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:15.899 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:15.899 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:15.899 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:15.899 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.899 INFO:tasks.workunit.client.0.smithi100.stdout: "report": { 2024-04-07T16:16:15.899 INFO:tasks.workunit.client.0.smithi100.stdout: "balancer": { 2024-04-07T16:16:15.899 INFO:tasks.workunit.client.0.smithi100.stdout: "active": true, 2024-04-07T16:16:15.899 INFO:tasks.workunit.client.0.smithi100.stdout: "mode": "" 2024-04-07T16:16:15.899 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.900 INFO:tasks.workunit.client.0.smithi100.stdout: "channels": [ 2024-04-07T16:16:15.900 INFO:tasks.workunit.client.0.smithi100.stdout: "basic", 2024-04-07T16:16:15.900 INFO:tasks.workunit.client.0.smithi100.stdout: "crash", 2024-04-07T16:16:15.900 INFO:tasks.workunit.client.0.smithi100.stdout: "device", 2024-04-07T16:16:15.900 INFO:tasks.workunit.client.0.smithi100.stdout: "ident", 2024-04-07T16:16:15.900 INFO:tasks.workunit.client.0.smithi100.stdout: "perf" 2024-04-07T16:16:15.900 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:15.900 INFO:tasks.workunit.client.0.smithi100.stdout: "channels_available": [ 2024-04-07T16:16:15.900 INFO:tasks.workunit.client.0.smithi100.stdout: "basic", 2024-04-07T16:16:15.900 INFO:tasks.workunit.client.0.smithi100.stdout: "ident", 2024-04-07T16:16:15.900 INFO:tasks.workunit.client.0.smithi100.stdout: "crash", 2024-04-07T16:16:15.900 INFO:tasks.workunit.client.0.smithi100.stdout: "device", 2024-04-07T16:16:15.900 INFO:tasks.workunit.client.0.smithi100.stdout: "perf" 2024-04-07T16:16:15.901 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:15.901 INFO:tasks.workunit.client.0.smithi100.stdout: "collections_available": [ 2024-04-07T16:16:15.901 INFO:tasks.workunit.client.0.smithi100.stdout: "basic_base", 2024-04-07T16:16:15.901 INFO:tasks.workunit.client.0.smithi100.stdout: "device_base", 2024-04-07T16:16:15.901 INFO:tasks.workunit.client.0.smithi100.stdout: "crash_base", 2024-04-07T16:16:15.901 INFO:tasks.workunit.client.0.smithi100.stdout: "ident_base", 2024-04-07T16:16:15.901 INFO:tasks.workunit.client.0.smithi100.stdout: "perf_perf", 2024-04-07T16:16:15.901 INFO:tasks.workunit.client.0.smithi100.stdout: "basic_mds_metadata", 2024-04-07T16:16:15.901 INFO:tasks.workunit.client.0.smithi100.stdout: "basic_pool_usage", 2024-04-07T16:16:15.901 INFO:tasks.workunit.client.0.smithi100.stdout: "basic_usage_by_class", 2024-04-07T16:16:15.901 INFO:tasks.workunit.client.0.smithi100.stdout: "basic_rook_v01", 2024-04-07T16:16:15.901 INFO:tasks.workunit.client.0.smithi100.stdout: "perf_memory_metrics", 2024-04-07T16:16:15.901 INFO:tasks.workunit.client.0.smithi100.stdout: "basic_pool_options_bluestore" 2024-04-07T16:16:15.902 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:15.902 INFO:tasks.workunit.client.0.smithi100.stdout: "collections_opted_in": [ 2024-04-07T16:16:15.902 INFO:tasks.workunit.client.0.smithi100.stdout: "basic_base", 2024-04-07T16:16:15.902 INFO:tasks.workunit.client.0.smithi100.stdout: "device_base", 2024-04-07T16:16:15.902 INFO:tasks.workunit.client.0.smithi100.stdout: "crash_base", 2024-04-07T16:16:15.902 INFO:tasks.workunit.client.0.smithi100.stdout: "ident_base", 2024-04-07T16:16:15.902 INFO:tasks.workunit.client.0.smithi100.stdout: "perf_perf", 2024-04-07T16:16:15.902 INFO:tasks.workunit.client.0.smithi100.stdout: "basic_mds_metadata", 2024-04-07T16:16:15.902 INFO:tasks.workunit.client.0.smithi100.stdout: "basic_pool_usage", 2024-04-07T16:16:15.902 INFO:tasks.workunit.client.0.smithi100.stdout: "basic_usage_by_class", 2024-04-07T16:16:15.902 INFO:tasks.workunit.client.0.smithi100.stdout: "basic_rook_v01", 2024-04-07T16:16:15.902 INFO:tasks.workunit.client.0.smithi100.stdout: "perf_memory_metrics", 2024-04-07T16:16:15.902 INFO:tasks.workunit.client.0.smithi100.stdout: "basic_pool_options_bluestore" 2024-04-07T16:16:15.903 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:15.903 INFO:tasks.workunit.client.0.smithi100.stdout: "config": { 2024-04-07T16:16:15.903 INFO:tasks.workunit.client.0.smithi100.stdout: "active_changed": [ 2024-04-07T16:16:15.903 INFO:tasks.workunit.client.0.smithi100.stdout: "auth_allow_insecure_global_id_reclaim", 2024-04-07T16:16:15.903 INFO:tasks.workunit.client.0.smithi100.stdout: "auth_debug", 2024-04-07T16:16:15.903 INFO:tasks.workunit.client.0.smithi100.stdout: "auth_mon_ticket_ttl", 2024-04-07T16:16:15.903 INFO:tasks.workunit.client.0.smithi100.stdout: "auth_service_ticket_ttl", 2024-04-07T16:16:15.903 INFO:tasks.workunit.client.0.smithi100.stdout: "bdev_debug_aio", 2024-04-07T16:16:15.903 INFO:tasks.workunit.client.0.smithi100.stdout: "client_check_pool_perm", 2024-04-07T16:16:15.903 INFO:tasks.workunit.client.0.smithi100.stdout: "client_mount_gid", 2024-04-07T16:16:15.903 INFO:tasks.workunit.client.0.smithi100.stdout: "client_mount_uid", 2024-04-07T16:16:15.903 INFO:tasks.workunit.client.0.smithi100.stdout: "client_quota", 2024-04-07T16:16:15.903 INFO:tasks.workunit.client.0.smithi100.stdout: "container_image", 2024-04-07T16:16:15.903 INFO:tasks.workunit.client.0.smithi100.stdout: "daemonize", 2024-04-07T16:16:15.903 INFO:tasks.workunit.client.0.smithi100.stdout: "debug_asserts_on_shutdown", 2024-04-07T16:16:15.904 INFO:tasks.workunit.client.0.smithi100.stdout: "debug_mgr", 2024-04-07T16:16:15.904 INFO:tasks.workunit.client.0.smithi100.stdout: "debug_mon", 2024-04-07T16:16:15.904 INFO:tasks.workunit.client.0.smithi100.stdout: "debug_ms", 2024-04-07T16:16:15.904 INFO:tasks.workunit.client.0.smithi100.stdout: "debug_osd", 2024-04-07T16:16:15.904 INFO:tasks.workunit.client.0.smithi100.stdout: "debug_paxos", 2024-04-07T16:16:15.904 INFO:tasks.workunit.client.0.smithi100.stdout: "keyring", 2024-04-07T16:16:15.904 INFO:tasks.workunit.client.0.smithi100.stdout: "leveldb_block_size", 2024-04-07T16:16:15.904 INFO:tasks.workunit.client.0.smithi100.stdout: "leveldb_cache_size", 2024-04-07T16:16:15.904 INFO:tasks.workunit.client.0.smithi100.stdout: "leveldb_compression", 2024-04-07T16:16:15.904 INFO:tasks.workunit.client.0.smithi100.stdout: "leveldb_log", 2024-04-07T16:16:15.904 INFO:tasks.workunit.client.0.smithi100.stdout: "leveldb_write_buffer_size", 2024-04-07T16:16:15.904 INFO:tasks.workunit.client.0.smithi100.stdout: "log_stderr_prefix", 2024-04-07T16:16:15.905 INFO:tasks.workunit.client.0.smithi100.stdout: "log_to_file", 2024-04-07T16:16:15.905 INFO:tasks.workunit.client.0.smithi100.stdout: "log_to_journald", 2024-04-07T16:16:15.905 INFO:tasks.workunit.client.0.smithi100.stdout: "log_to_stderr", 2024-04-07T16:16:15.905 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_allow_pool_delete", 2024-04-07T16:16:15.905 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_clock_drift_allowed", 2024-04-07T16:16:15.905 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_cluster_log_file_level", 2024-04-07T16:16:15.905 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_cluster_log_to_file", 2024-04-07T16:16:15.905 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_cluster_log_to_stderr", 2024-04-07T16:16:15.905 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_data_avail_warn", 2024-04-07T16:16:15.905 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_election_default_strategy", 2024-04-07T16:16:15.905 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_host", 2024-04-07T16:16:15.905 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_max_pg_per_osd", 2024-04-07T16:16:15.905 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_mgr_mkfs_grace", 2024-04-07T16:16:15.905 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_osd_prime_pg_temp", 2024-04-07T16:16:15.906 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_osd_reporter_subtree_level", 2024-04-07T16:16:15.906 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_pg_warn_max_object_skew", 2024-04-07T16:16:15.906 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_reweight_min_bytes_per_osd", 2024-04-07T16:16:15.906 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_reweight_min_pgs_per_osd", 2024-04-07T16:16:15.906 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_warn_on_crush_straw_calc_version_zero", 2024-04-07T16:16:15.906 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_warn_on_insecure_global_id_reclaim", 2024-04-07T16:16:15.906 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_warn_on_insecure_global_id_reclaim_allowed", 2024-04-07T16:16:15.906 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_warn_on_legacy_crush_tunables", 2024-04-07T16:16:15.906 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_warn_on_osd_down_out_interval_zero", 2024-04-07T16:16:15.906 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_warn_on_pool_pg_num_not_power_of_two", 2024-04-07T16:16:15.906 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_warn_on_too_few_osds", 2024-04-07T16:16:15.906 INFO:tasks.workunit.client.0.smithi100.stdout: "ms_die_on_bug", 2024-04-07T16:16:15.906 INFO:tasks.workunit.client.0.smithi100.stdout: "ms_die_on_old_message", 2024-04-07T16:16:15.906 INFO:tasks.workunit.client.0.smithi100.stdout: "no_config_file", 2024-04-07T16:16:15.906 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_class_default_list", 2024-04-07T16:16:15.907 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_class_load_list", 2024-04-07T16:16:15.907 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_crush_chooseleaf_type", 2024-04-07T16:16:15.907 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_debug_misdirected_ops", 2024-04-07T16:16:15.907 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_debug_op_order", 2024-04-07T16:16:15.907 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_debug_pg_log_writeout", 2024-04-07T16:16:15.907 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_debug_shutdown", 2024-04-07T16:16:15.907 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_debug_verify_cached_snaps", 2024-04-07T16:16:15.907 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_debug_verify_missing_on_start", 2024-04-07T16:16:15.907 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_debug_verify_stray_on_activate", 2024-04-07T16:16:15.907 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_deep_scrub_update_digest_min_age", 2024-04-07T16:16:15.907 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_delete_sleep", 2024-04-07T16:16:15.907 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_delete_sleep_hdd", 2024-04-07T16:16:15.907 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_delete_sleep_hybrid", 2024-04-07T16:16:15.907 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_delete_sleep_ssd", 2024-04-07T16:16:15.907 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_max_backfills", 2024-04-07T16:16:15.908 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mclock_max_capacity_iops_ssd", 2024-04-07T16:16:15.908 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mclock_profile", 2024-04-07T16:16:15.908 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mclock_scheduler_background_best_effort_lim", 2024-04-07T16:16:15.908 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mclock_scheduler_background_best_effort_res", 2024-04-07T16:16:15.908 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mclock_scheduler_background_best_effort_wgt", 2024-04-07T16:16:15.908 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mclock_scheduler_background_recovery_lim", 2024-04-07T16:16:15.908 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mclock_scheduler_background_recovery_res", 2024-04-07T16:16:15.908 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mclock_scheduler_background_recovery_wgt", 2024-04-07T16:16:15.908 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mclock_scheduler_client_lim", 2024-04-07T16:16:15.908 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mclock_scheduler_client_res", 2024-04-07T16:16:15.908 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mclock_scheduler_client_wgt", 2024-04-07T16:16:15.908 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_memory_target", 2024-04-07T16:16:15.908 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_memory_target_autotune", 2024-04-07T16:16:15.908 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_op_queue", 2024-04-07T16:16:15.908 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_op_queue_cut_off", 2024-04-07T16:16:15.909 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_pg_delete_cost", 2024-04-07T16:16:15.909 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_pool_default_erasure_code_profile", 2024-04-07T16:16:15.909 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_pool_default_pg_autoscale_mode", 2024-04-07T16:16:15.909 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_recover_clone_overlap", 2024-04-07T16:16:15.909 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_recovery_max_active", 2024-04-07T16:16:15.909 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_recovery_max_active_hdd", 2024-04-07T16:16:15.909 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_recovery_max_active_ssd", 2024-04-07T16:16:15.909 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_recovery_max_chunk", 2024-04-07T16:16:15.909 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_recovery_sleep", 2024-04-07T16:16:15.909 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_recovery_sleep_hdd", 2024-04-07T16:16:15.909 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_recovery_sleep_hybrid", 2024-04-07T16:16:15.909 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_recovery_sleep_ssd", 2024-04-07T16:16:15.909 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_scrub_load_threshold", 2024-04-07T16:16:15.909 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_scrub_max_interval", 2024-04-07T16:16:15.909 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_scrub_sleep", 2024-04-07T16:16:15.910 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_shutdown_pgref_assert", 2024-04-07T16:16:15.910 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_snap_trim_sleep", 2024-04-07T16:16:15.910 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_snap_trim_sleep_hdd", 2024-04-07T16:16:15.910 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_snap_trim_sleep_hybrid", 2024-04-07T16:16:15.910 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_snap_trim_sleep_ssd", 2024-04-07T16:16:15.910 INFO:tasks.workunit.client.0.smithi100.stdout: "public_addr", 2024-04-07T16:16:15.910 INFO:tasks.workunit.client.0.smithi100.stdout: "public_addrv", 2024-04-07T16:16:15.910 INFO:tasks.workunit.client.0.smithi100.stdout: "public_network", 2024-04-07T16:16:15.910 INFO:tasks.workunit.client.0.smithi100.stdout: "rbd_default_features", 2024-04-07T16:16:15.910 INFO:tasks.workunit.client.0.smithi100.stdout: "rbd_qos_exclude_ops", 2024-04-07T16:16:15.910 INFO:tasks.workunit.client.0.smithi100.stdout: "setgroup", 2024-04-07T16:16:15.910 INFO:tasks.workunit.client.0.smithi100.stdout: "setuser" 2024-04-07T16:16:15.910 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:15.910 INFO:tasks.workunit.client.0.smithi100.stdout: "cluster_changed": [ 2024-04-07T16:16:15.910 INFO:tasks.workunit.client.0.smithi100.stdout: "auth_allow_insecure_global_id_reclaim", 2024-04-07T16:16:15.911 INFO:tasks.workunit.client.0.smithi100.stdout: "auth_debug", 2024-04-07T16:16:15.911 INFO:tasks.workunit.client.0.smithi100.stdout: "auth_mon_ticket_ttl", 2024-04-07T16:16:15.911 INFO:tasks.workunit.client.0.smithi100.stdout: "auth_service_ticket_ttl", 2024-04-07T16:16:15.911 INFO:tasks.workunit.client.0.smithi100.stdout: "bdev_debug_aio", 2024-04-07T16:16:15.911 INFO:tasks.workunit.client.0.smithi100.stdout: "container_image", 2024-04-07T16:16:15.911 INFO:tasks.workunit.client.0.smithi100.stdout: "debug_asserts_on_shutdown", 2024-04-07T16:16:15.911 INFO:tasks.workunit.client.0.smithi100.stdout: "debug_mgr", 2024-04-07T16:16:15.911 INFO:tasks.workunit.client.0.smithi100.stdout: "debug_mon", 2024-04-07T16:16:15.911 INFO:tasks.workunit.client.0.smithi100.stdout: "debug_ms", 2024-04-07T16:16:15.911 INFO:tasks.workunit.client.0.smithi100.stdout: "debug_osd", 2024-04-07T16:16:15.911 INFO:tasks.workunit.client.0.smithi100.stdout: "debug_paxos", 2024-04-07T16:16:15.911 INFO:tasks.workunit.client.0.smithi100.stdout: "log_to_file", 2024-04-07T16:16:15.911 INFO:tasks.workunit.client.0.smithi100.stdout: "log_to_journald", 2024-04-07T16:16:15.911 INFO:tasks.workunit.client.0.smithi100.stdout: "log_to_stderr", 2024-04-07T16:16:15.911 INFO:tasks.workunit.client.0.smithi100.stdout: "mds_join_fs", 2024-04-07T16:16:15.911 INFO:tasks.workunit.client.0.smithi100.stdout: "mgr/cephadm/allow_ptrace", 2024-04-07T16:16:15.912 INFO:tasks.workunit.client.0.smithi100.stdout: "mgr/cephadm/container_init", 2024-04-07T16:16:15.912 INFO:tasks.workunit.client.0.smithi100.stdout: "mgr/cephadm/migration_current", 2024-04-07T16:16:15.912 INFO:tasks.workunit.client.0.smithi100.stdout: "mgr/cephadm/use_repo_digest", 2024-04-07T16:16:15.912 INFO:tasks.workunit.client.0.smithi100.stdout: "mgr/dashboard/ALERTMANAGER_API_HOST", 2024-04-07T16:16:15.912 INFO:tasks.workunit.client.0.smithi100.stdout: "mgr/dashboard/GRAFANA_API_SSL_VERIFY", 2024-04-07T16:16:15.912 INFO:tasks.workunit.client.0.smithi100.stdout: "mgr/dashboard/GRAFANA_API_URL", 2024-04-07T16:16:15.912 INFO:tasks.workunit.client.0.smithi100.stdout: "mgr/dashboard/PROMETHEUS_API_HOST", 2024-04-07T16:16:15.912 INFO:tasks.workunit.client.0.smithi100.stdout: "mgr/dashboard/ssl_server_port", 2024-04-07T16:16:15.912 INFO:tasks.workunit.client.0.smithi100.stdout: "mgr/orchestrator/orchestrator", 2024-04-07T16:16:15.912 INFO:tasks.workunit.client.0.smithi100.stdout: "mgr/telemetry/channel_ident", 2024-04-07T16:16:15.912 INFO:tasks.workunit.client.0.smithi100.stdout: "mgr/telemetry/channel_perf", 2024-04-07T16:16:15.912 INFO:tasks.workunit.client.0.smithi100.stdout: "mgr/telemetry/description", 2024-04-07T16:16:15.912 INFO:tasks.workunit.client.0.smithi100.stdout: "mgr/telemetry/enabled", 2024-04-07T16:16:15.912 INFO:tasks.workunit.client.0.smithi100.stdout: "mgr/telemetry/organization", 2024-04-07T16:16:15.912 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_allow_pool_delete", 2024-04-07T16:16:15.913 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_clock_drift_allowed", 2024-04-07T16:16:15.913 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_cluster_log_file_level", 2024-04-07T16:16:15.913 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_cluster_log_to_file", 2024-04-07T16:16:15.913 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_data_avail_warn", 2024-04-07T16:16:15.913 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_election_default_strategy", 2024-04-07T16:16:15.913 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_max_pg_per_osd", 2024-04-07T16:16:15.913 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_mgr_mkfs_grace", 2024-04-07T16:16:15.913 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_osd_prime_pg_temp", 2024-04-07T16:16:15.913 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_osd_reporter_subtree_level", 2024-04-07T16:16:15.913 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_pg_warn_max_object_skew", 2024-04-07T16:16:15.913 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_reweight_min_bytes_per_osd", 2024-04-07T16:16:15.913 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_reweight_min_pgs_per_osd", 2024-04-07T16:16:15.913 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_warn_on_crush_straw_calc_version_zero", 2024-04-07T16:16:15.913 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_warn_on_insecure_global_id_reclaim", 2024-04-07T16:16:15.913 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_warn_on_insecure_global_id_reclaim_allowed", 2024-04-07T16:16:15.914 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_warn_on_legacy_crush_tunables", 2024-04-07T16:16:15.914 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_warn_on_osd_down_out_interval_zero", 2024-04-07T16:16:15.914 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_warn_on_pool_pg_num_not_power_of_two", 2024-04-07T16:16:15.914 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_warn_on_too_few_osds", 2024-04-07T16:16:15.914 INFO:tasks.workunit.client.0.smithi100.stdout: "ms_die_on_bug", 2024-04-07T16:16:15.914 INFO:tasks.workunit.client.0.smithi100.stdout: "ms_die_on_old_message", 2024-04-07T16:16:15.914 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_class_default_list", 2024-04-07T16:16:15.914 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_class_load_list", 2024-04-07T16:16:15.914 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_debug_misdirected_ops", 2024-04-07T16:16:15.914 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_debug_op_order", 2024-04-07T16:16:15.914 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_debug_pg_log_writeout", 2024-04-07T16:16:15.914 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_debug_shutdown", 2024-04-07T16:16:15.914 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_debug_verify_cached_snaps", 2024-04-07T16:16:15.914 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_debug_verify_missing_on_start", 2024-04-07T16:16:15.915 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_debug_verify_stray_on_activate", 2024-04-07T16:16:15.915 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_deep_scrub_update_digest_min_age", 2024-04-07T16:16:15.915 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mclock_max_capacity_iops_ssd", 2024-04-07T16:16:15.915 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mclock_profile", 2024-04-07T16:16:15.915 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_memory_target", 2024-04-07T16:16:15.915 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_memory_target_autotune", 2024-04-07T16:16:15.915 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_op_queue", 2024-04-07T16:16:15.915 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_op_queue_cut_off", 2024-04-07T16:16:15.915 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_pool_default_erasure_code_profile", 2024-04-07T16:16:15.915 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_pool_default_pg_autoscale_mode", 2024-04-07T16:16:15.915 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_recover_clone_overlap", 2024-04-07T16:16:15.915 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_recovery_max_chunk", 2024-04-07T16:16:15.915 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_scrub_load_threshold", 2024-04-07T16:16:15.915 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_scrub_max_interval", 2024-04-07T16:16:15.916 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_shutdown_pgref_assert", 2024-04-07T16:16:15.916 INFO:tasks.workunit.client.0.smithi100.stdout: "public_network", 2024-04-07T16:16:15.916 INFO:tasks.workunit.client.0.smithi100.stdout: "rgw_cache_enabled", 2024-04-07T16:16:15.916 INFO:tasks.workunit.client.0.smithi100.stdout: "rgw_enable_ops_log", 2024-04-07T16:16:15.916 INFO:tasks.workunit.client.0.smithi100.stdout: "rgw_enable_usage_log" 2024-04-07T16:16:15.916 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T16:16:15.916 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.916 INFO:tasks.workunit.client.0.smithi100.stdout: "contact": null, 2024-04-07T16:16:15.916 INFO:tasks.workunit.client.0.smithi100.stdout: "crashes": [], 2024-04-07T16:16:15.916 INFO:tasks.workunit.client.0.smithi100.stdout: "created": "2024-04-07T16:02:17.280630Z", 2024-04-07T16:16:15.916 INFO:tasks.workunit.client.0.smithi100.stdout: "crush": { 2024-04-07T16:16:15.916 INFO:tasks.workunit.client.0.smithi100.stdout: "bucket_algs": { 2024-04-07T16:16:15.916 INFO:tasks.workunit.client.0.smithi100.stdout: "straw2": 3 2024-04-07T16:16:15.916 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.916 INFO:tasks.workunit.client.0.smithi100.stdout: "bucket_sizes": { 2024-04-07T16:16:15.917 INFO:tasks.workunit.client.0.smithi100.stdout: "2": 1, 2024-04-07T16:16:15.917 INFO:tasks.workunit.client.0.smithi100.stdout: "4": 2 2024-04-07T16:16:15.917 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.917 INFO:tasks.workunit.client.0.smithi100.stdout: "bucket_types": { 2024-04-07T16:16:15.917 INFO:tasks.workunit.client.0.smithi100.stdout: "1": 2, 2024-04-07T16:16:15.917 INFO:tasks.workunit.client.0.smithi100.stdout: "11": 1 2024-04-07T16:16:15.917 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.917 INFO:tasks.workunit.client.0.smithi100.stdout: "compat_weight_set": false, 2024-04-07T16:16:15.917 INFO:tasks.workunit.client.0.smithi100.stdout: "device_classes": [ 2024-04-07T16:16:15.917 INFO:tasks.workunit.client.0.smithi100.stdout: 8 2024-04-07T16:16:15.917 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:15.917 INFO:tasks.workunit.client.0.smithi100.stdout: "num_buckets": 6, 2024-04-07T16:16:15.917 INFO:tasks.workunit.client.0.smithi100.stdout: "num_devices": 8, 2024-04-07T16:16:15.917 INFO:tasks.workunit.client.0.smithi100.stdout: "num_rules": 1, 2024-04-07T16:16:15.917 INFO:tasks.workunit.client.0.smithi100.stdout: "num_types": 12, 2024-04-07T16:16:15.918 INFO:tasks.workunit.client.0.smithi100.stdout: "num_weight_sets": 0, 2024-04-07T16:16:15.918 INFO:tasks.workunit.client.0.smithi100.stdout: "tunables": { 2024-04-07T16:16:15.918 INFO:tasks.workunit.client.0.smithi100.stdout: "allowed_bucket_algs": 54, 2024-04-07T16:16:15.918 INFO:tasks.workunit.client.0.smithi100.stdout: "choose_local_fallback_tries": 0, 2024-04-07T16:16:15.918 INFO:tasks.workunit.client.0.smithi100.stdout: "choose_local_tries": 0, 2024-04-07T16:16:15.918 INFO:tasks.workunit.client.0.smithi100.stdout: "choose_total_tries": 50, 2024-04-07T16:16:15.918 INFO:tasks.workunit.client.0.smithi100.stdout: "chooseleaf_descend_once": 1, 2024-04-07T16:16:15.918 INFO:tasks.workunit.client.0.smithi100.stdout: "chooseleaf_stable": 1, 2024-04-07T16:16:15.918 INFO:tasks.workunit.client.0.smithi100.stdout: "chooseleaf_vary_r": 1, 2024-04-07T16:16:15.918 INFO:tasks.workunit.client.0.smithi100.stdout: "has_v2_rules": 0, 2024-04-07T16:16:15.918 INFO:tasks.workunit.client.0.smithi100.stdout: "has_v3_rules": 0, 2024-04-07T16:16:15.918 INFO:tasks.workunit.client.0.smithi100.stdout: "has_v4_buckets": 1, 2024-04-07T16:16:15.918 INFO:tasks.workunit.client.0.smithi100.stdout: "has_v5_rules": 0, 2024-04-07T16:16:15.918 INFO:tasks.workunit.client.0.smithi100.stdout: "legacy_tunables": 0, 2024-04-07T16:16:15.919 INFO:tasks.workunit.client.0.smithi100.stdout: "minimum_required_version": "jewel", 2024-04-07T16:16:15.919 INFO:tasks.workunit.client.0.smithi100.stdout: "optimal_tunables": 1, 2024-04-07T16:16:15.919 INFO:tasks.workunit.client.0.smithi100.stdout: "profile": "jewel", 2024-04-07T16:16:15.919 INFO:tasks.workunit.client.0.smithi100.stdout: "require_feature_tunables": 1, 2024-04-07T16:16:15.919 INFO:tasks.workunit.client.0.smithi100.stdout: "require_feature_tunables2": 1, 2024-04-07T16:16:15.919 INFO:tasks.workunit.client.0.smithi100.stdout: "require_feature_tunables3": 1, 2024-04-07T16:16:15.919 INFO:tasks.workunit.client.0.smithi100.stdout: "require_feature_tunables5": 1, 2024-04-07T16:16:15.919 INFO:tasks.workunit.client.0.smithi100.stdout: "straw_calc_version": 1 2024-04-07T16:16:15.919 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:15.919 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.919 INFO:tasks.workunit.client.0.smithi100.stdout: "description": "upgrade test cluster", 2024-04-07T16:16:15.919 INFO:tasks.workunit.client.0.smithi100.stdout: "fs": { 2024-04-07T16:16:15.919 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1, 2024-04-07T16:16:15.919 INFO:tasks.workunit.client.0.smithi100.stdout: "feature_flags": { 2024-04-07T16:16:15.919 INFO:tasks.workunit.client.0.smithi100.stdout: "enable_multiple": true, 2024-04-07T16:16:15.919 INFO:tasks.workunit.client.0.smithi100.stdout: "ever_enabled_multiple": true 2024-04-07T16:16:15.920 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.920 INFO:tasks.workunit.client.0.smithi100.stdout: "filesystems": [ 2024-04-07T16:16:15.920 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:15.920 INFO:tasks.workunit.client.0.smithi100.stdout: "approx_ctime": "2024-04", 2024-04-07T16:16:15.920 INFO:tasks.workunit.client.0.smithi100.stdout: "balancer_enabled": false, 2024-04-07T16:16:15.920 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:15.920 INFO:tasks.workunit.client.0.smithi100.stdout: "cached_caps": 0, 2024-04-07T16:16:15.920 INFO:tasks.workunit.client.0.smithi100.stdout: "cached_dns": 10, 2024-04-07T16:16:15.920 INFO:tasks.workunit.client.0.smithi100.stdout: "cached_inos": 13, 2024-04-07T16:16:15.920 INFO:tasks.workunit.client.0.smithi100.stdout: "cached_subtrees": 2, 2024-04-07T16:16:15.920 INFO:tasks.workunit.client.0.smithi100.stdout: "ever_allowed_features": 0, 2024-04-07T16:16:15.921 INFO:tasks.workunit.client.0.smithi100.stdout: "explicitly_allowed_features": 0, 2024-04-07T16:16:15.921 INFO:tasks.workunit.client.0.smithi100.stdout: "files": 0, 2024-04-07T16:16:15.921 INFO:tasks.workunit.client.0.smithi100.stdout: "max_mds": 1, 2024-04-07T16:16:15.921 INFO:tasks.workunit.client.0.smithi100.stdout: "num_data_pools": 1, 2024-04-07T16:16:15.921 INFO:tasks.workunit.client.0.smithi100.stdout: "num_in": 1, 2024-04-07T16:16:15.921 INFO:tasks.workunit.client.0.smithi100.stdout: "num_mds": 1, 2024-04-07T16:16:15.921 INFO:tasks.workunit.client.0.smithi100.stdout: "num_sessions": 0, 2024-04-07T16:16:15.921 INFO:tasks.workunit.client.0.smithi100.stdout: "num_standby_replay": 0, 2024-04-07T16:16:15.921 INFO:tasks.workunit.client.0.smithi100.stdout: "num_up": 1, 2024-04-07T16:16:15.921 INFO:tasks.workunit.client.0.smithi100.stdout: "snaps": 0, 2024-04-07T16:16:15.921 INFO:tasks.workunit.client.0.smithi100.stdout: "standby_count_wanted": 0 2024-04-07T16:16:15.921 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:15.921 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:15.921 INFO:tasks.workunit.client.0.smithi100.stdout: "num_standby_mds": 1, 2024-04-07T16:16:15.921 INFO:tasks.workunit.client.0.smithi100.stdout: "total_num_mds": 2 2024-04-07T16:16:15.922 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.922 INFO:tasks.workunit.client.0.smithi100.stdout: "heap_stats": { 2024-04-07T16:16:15.922 INFO:tasks.workunit.client.0.smithi100.stdout: "mds": { 2024-04-07T16:16:15.922 INFO:tasks.workunit.client.0.smithi100.stdout: "mds.a52fc0d8acb515431879057649ff294cfe14c620": { 2024-04-07T16:16:15.922 INFO:tasks.workunit.client.0.smithi100.stdout: "actual_memory_used": 20463616, 2024-04-07T16:16:15.922 INFO:tasks.workunit.client.0.smithi100.stdout: "central_cache_freelist": 500808, 2024-04-07T16:16:15.922 INFO:tasks.workunit.client.0.smithi100.stdout: "malloc_metadata": 2752512, 2024-04-07T16:16:15.922 INFO:tasks.workunit.client.0.smithi100.stdout: "page_heap_freelist": 0, 2024-04-07T16:16:15.922 INFO:tasks.workunit.client.0.smithi100.stdout: "released_to_os": 851968, 2024-04-07T16:16:15.922 INFO:tasks.workunit.client.0.smithi100.stdout: "spans_in_use": 406, 2024-04-07T16:16:15.922 INFO:tasks.workunit.client.0.smithi100.stdout: "tcmalloc_page_size": 8192, 2024-04-07T16:16:15.922 INFO:tasks.workunit.client.0.smithi100.stdout: "thread_cache_freelists": 1344120, 2024-04-07T16:16:15.922 INFO:tasks.workunit.client.0.smithi100.stdout: "thread_heaps_in_use": 21, 2024-04-07T16:16:15.923 INFO:tasks.workunit.client.0.smithi100.stdout: "transfer_cache_freelist": 986112, 2024-04-07T16:16:15.923 INFO:tasks.workunit.client.0.smithi100.stdout: "use_by_application": 14880064, 2024-04-07T16:16:15.923 INFO:tasks.workunit.client.0.smithi100.stdout: "virtual_address_space_used": 21315584 2024-04-07T16:16:15.923 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.923 INFO:tasks.workunit.client.0.smithi100.stdout: "mds.f47a4ff15a45950a63160265564a123fc5da5e5e": { 2024-04-07T16:16:15.923 INFO:tasks.workunit.client.0.smithi100.stdout: "actual_memory_used": 20267008, 2024-04-07T16:16:15.923 INFO:tasks.workunit.client.0.smithi100.stdout: "central_cache_freelist": 338472, 2024-04-07T16:16:15.923 INFO:tasks.workunit.client.0.smithi100.stdout: "malloc_metadata": 2752512, 2024-04-07T16:16:15.923 INFO:tasks.workunit.client.0.smithi100.stdout: "page_heap_freelist": 958464, 2024-04-07T16:16:15.923 INFO:tasks.workunit.client.0.smithi100.stdout: "released_to_os": 0, 2024-04-07T16:16:15.923 INFO:tasks.workunit.client.0.smithi100.stdout: "spans_in_use": 323, 2024-04-07T16:16:15.923 INFO:tasks.workunit.client.0.smithi100.stdout: "tcmalloc_page_size": 8192, 2024-04-07T16:16:15.923 INFO:tasks.workunit.client.0.smithi100.stdout: "thread_cache_freelists": 1209352, 2024-04-07T16:16:15.924 INFO:tasks.workunit.client.0.smithi100.stdout: "thread_heaps_in_use": 12, 2024-04-07T16:16:15.924 INFO:tasks.workunit.client.0.smithi100.stdout: "transfer_cache_freelist": 545792, 2024-04-07T16:16:15.924 INFO:tasks.workunit.client.0.smithi100.stdout: "use_by_application": 14462416, 2024-04-07T16:16:15.924 INFO:tasks.workunit.client.0.smithi100.stdout: "virtual_address_space_used": 20267008 2024-04-07T16:16:15.924 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:15.924 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.924 INFO:tasks.workunit.client.0.smithi100.stdout: "mon": { 2024-04-07T16:16:15.924 INFO:tasks.workunit.client.0.smithi100.stdout: "mon.8dd0baa036f0a7e6d1f3bdce86136fe527836c3d": { 2024-04-07T16:16:15.924 INFO:tasks.workunit.client.0.smithi100.stdout: "actual_memory_used": 35627008, 2024-04-07T16:16:15.924 INFO:tasks.workunit.client.0.smithi100.stdout: "central_cache_freelist": 1106840, 2024-04-07T16:16:15.924 INFO:tasks.workunit.client.0.smithi100.stdout: "malloc_metadata": 2752512, 2024-04-07T16:16:15.924 INFO:tasks.workunit.client.0.smithi100.stdout: "page_heap_freelist": 811008, 2024-04-07T16:16:15.924 INFO:tasks.workunit.client.0.smithi100.stdout: "released_to_os": 4694016, 2024-04-07T16:16:15.924 INFO:tasks.workunit.client.0.smithi100.stdout: "spans_in_use": 1057, 2024-04-07T16:16:15.924 INFO:tasks.workunit.client.0.smithi100.stdout: "tcmalloc_page_size": 8192, 2024-04-07T16:16:15.925 INFO:tasks.workunit.client.0.smithi100.stdout: "thread_cache_freelists": 8454744, 2024-04-07T16:16:15.925 INFO:tasks.workunit.client.0.smithi100.stdout: "thread_heaps_in_use": 21, 2024-04-07T16:16:15.925 INFO:tasks.workunit.client.0.smithi100.stdout: "transfer_cache_freelist": 1304064, 2024-04-07T16:16:15.925 INFO:tasks.workunit.client.0.smithi100.stdout: "use_by_application": 21197840, 2024-04-07T16:16:15.925 INFO:tasks.workunit.client.0.smithi100.stdout: "virtual_address_space_used": 40321024 2024-04-07T16:16:15.925 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.925 INFO:tasks.workunit.client.0.smithi100.stdout: "mon.a52fc0d8acb515431879057649ff294cfe14c620": { 2024-04-07T16:16:15.925 INFO:tasks.workunit.client.0.smithi100.stdout: "actual_memory_used": 36208640, 2024-04-07T16:16:15.925 INFO:tasks.workunit.client.0.smithi100.stdout: "central_cache_freelist": 1022032, 2024-04-07T16:16:15.925 INFO:tasks.workunit.client.0.smithi100.stdout: "malloc_metadata": 2752512, 2024-04-07T16:16:15.925 INFO:tasks.workunit.client.0.smithi100.stdout: "page_heap_freelist": 81920, 2024-04-07T16:16:15.925 INFO:tasks.workunit.client.0.smithi100.stdout: "released_to_os": 5160960, 2024-04-07T16:16:15.925 INFO:tasks.workunit.client.0.smithi100.stdout: "spans_in_use": 1257, 2024-04-07T16:16:15.925 INFO:tasks.workunit.client.0.smithi100.stdout: "tcmalloc_page_size": 8192, 2024-04-07T16:16:15.925 INFO:tasks.workunit.client.0.smithi100.stdout: "thread_cache_freelists": 9297144, 2024-04-07T16:16:15.925 INFO:tasks.workunit.client.0.smithi100.stdout: "thread_heaps_in_use": 22, 2024-04-07T16:16:15.926 INFO:tasks.workunit.client.0.smithi100.stdout: "transfer_cache_freelist": 1702400, 2024-04-07T16:16:15.926 INFO:tasks.workunit.client.0.smithi100.stdout: "use_by_application": 21352632, 2024-04-07T16:16:15.926 INFO:tasks.workunit.client.0.smithi100.stdout: "virtual_address_space_used": 41369600 2024-04-07T16:16:15.926 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.926 INFO:tasks.workunit.client.0.smithi100.stdout: "mon.f47a4ff15a45950a63160265564a123fc5da5e5e": { 2024-04-07T16:16:15.926 INFO:tasks.workunit.client.0.smithi100.stdout: "actual_memory_used": 44269568, 2024-04-07T16:16:15.926 INFO:tasks.workunit.client.0.smithi100.stdout: "central_cache_freelist": 1092424, 2024-04-07T16:16:15.926 INFO:tasks.workunit.client.0.smithi100.stdout: "malloc_metadata": 2752512, 2024-04-07T16:16:15.926 INFO:tasks.workunit.client.0.smithi100.stdout: "page_heap_freelist": 0, 2024-04-07T16:16:15.926 INFO:tasks.workunit.client.0.smithi100.stdout: "released_to_os": 3293184, 2024-04-07T16:16:15.926 INFO:tasks.workunit.client.0.smithi100.stdout: "spans_in_use": 1655, 2024-04-07T16:16:15.926 INFO:tasks.workunit.client.0.smithi100.stdout: "tcmalloc_page_size": 8192, 2024-04-07T16:16:15.926 INFO:tasks.workunit.client.0.smithi100.stdout: "thread_cache_freelists": 12244080, 2024-04-07T16:16:15.926 INFO:tasks.workunit.client.0.smithi100.stdout: "thread_heaps_in_use": 22, 2024-04-07T16:16:15.927 INFO:tasks.workunit.client.0.smithi100.stdout: "transfer_cache_freelist": 2658560, 2024-04-07T16:16:15.927 INFO:tasks.workunit.client.0.smithi100.stdout: "use_by_application": 25521992, 2024-04-07T16:16:15.927 INFO:tasks.workunit.client.0.smithi100.stdout: "virtual_address_space_used": 47562752 2024-04-07T16:16:15.927 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:15.927 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.927 INFO:tasks.workunit.client.0.smithi100.stdout: "osd": { 2024-04-07T16:16:15.927 INFO:tasks.workunit.client.0.smithi100.stdout: "osd.0": { 2024-04-07T16:16:15.927 INFO:tasks.workunit.client.0.smithi100.stdout: "actual_memory_used": 47562752, 2024-04-07T16:16:15.927 INFO:tasks.workunit.client.0.smithi100.stdout: "central_cache_freelist": 874160, 2024-04-07T16:16:15.927 INFO:tasks.workunit.client.0.smithi100.stdout: "malloc_metadata": 2883584, 2024-04-07T16:16:15.927 INFO:tasks.workunit.client.0.smithi100.stdout: "page_heap_freelist": 0, 2024-04-07T16:16:15.927 INFO:tasks.workunit.client.0.smithi100.stdout: "released_to_os": 851968, 2024-04-07T16:16:15.927 INFO:tasks.workunit.client.0.smithi100.stdout: "spans_in_use": 1472, 2024-04-07T16:16:15.927 INFO:tasks.workunit.client.0.smithi100.stdout: "tcmalloc_page_size": 8192, 2024-04-07T16:16:15.927 INFO:tasks.workunit.client.0.smithi100.stdout: "thread_cache_freelists": 5091784, 2024-04-07T16:16:15.928 INFO:tasks.workunit.client.0.smithi100.stdout: "thread_heaps_in_use": 46, 2024-04-07T16:16:15.928 INFO:tasks.workunit.client.0.smithi100.stdout: "transfer_cache_freelist": 2364160, 2024-04-07T16:16:15.928 INFO:tasks.workunit.client.0.smithi100.stdout: "use_by_application": 36349064, 2024-04-07T16:16:15.928 INFO:tasks.workunit.client.0.smithi100.stdout: "virtual_address_space_used": 48414720 2024-04-07T16:16:15.928 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.928 INFO:tasks.workunit.client.0.smithi100.stdout: "osd.1": { 2024-04-07T16:16:15.928 INFO:tasks.workunit.client.0.smithi100.stdout: "actual_memory_used": 47030272, 2024-04-07T16:16:15.928 INFO:tasks.workunit.client.0.smithi100.stdout: "central_cache_freelist": 902000, 2024-04-07T16:16:15.928 INFO:tasks.workunit.client.0.smithi100.stdout: "malloc_metadata": 2883584, 2024-04-07T16:16:15.928 INFO:tasks.workunit.client.0.smithi100.stdout: "page_heap_freelist": 0, 2024-04-07T16:16:15.928 INFO:tasks.workunit.client.0.smithi100.stdout: "released_to_os": 335872, 2024-04-07T16:16:15.928 INFO:tasks.workunit.client.0.smithi100.stdout: "spans_in_use": 1443, 2024-04-07T16:16:15.928 INFO:tasks.workunit.client.0.smithi100.stdout: "tcmalloc_page_size": 8192, 2024-04-07T16:16:15.929 INFO:tasks.workunit.client.0.smithi100.stdout: "thread_cache_freelists": 5236920, 2024-04-07T16:16:15.929 INFO:tasks.workunit.client.0.smithi100.stdout: "thread_heaps_in_use": 47, 2024-04-07T16:16:15.929 INFO:tasks.workunit.client.0.smithi100.stdout: "transfer_cache_freelist": 2801408, 2024-04-07T16:16:15.929 INFO:tasks.workunit.client.0.smithi100.stdout: "use_by_application": 35206360, 2024-04-07T16:16:15.929 INFO:tasks.workunit.client.0.smithi100.stdout: "virtual_address_space_used": 47366144 2024-04-07T16:16:15.929 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.929 INFO:tasks.workunit.client.0.smithi100.stdout: "osd.2": { 2024-04-07T16:16:15.929 INFO:tasks.workunit.client.0.smithi100.stdout: "actual_memory_used": 51052544, 2024-04-07T16:16:15.929 INFO:tasks.workunit.client.0.smithi100.stdout: "central_cache_freelist": 1757832, 2024-04-07T16:16:15.929 INFO:tasks.workunit.client.0.smithi100.stdout: "malloc_metadata": 2883584, 2024-04-07T16:16:15.929 INFO:tasks.workunit.client.0.smithi100.stdout: "page_heap_freelist": 0, 2024-04-07T16:16:15.929 INFO:tasks.workunit.client.0.smithi100.stdout: "released_to_os": 507904, 2024-04-07T16:16:15.929 INFO:tasks.workunit.client.0.smithi100.stdout: "spans_in_use": 1776, 2024-04-07T16:16:15.929 INFO:tasks.workunit.client.0.smithi100.stdout: "tcmalloc_page_size": 8192, 2024-04-07T16:16:15.930 INFO:tasks.workunit.client.0.smithi100.stdout: "thread_cache_freelists": 5618712, 2024-04-07T16:16:15.930 INFO:tasks.workunit.client.0.smithi100.stdout: "thread_heaps_in_use": 47, 2024-04-07T16:16:15.930 INFO:tasks.workunit.client.0.smithi100.stdout: "transfer_cache_freelist": 4874240, 2024-04-07T16:16:15.930 INFO:tasks.workunit.client.0.smithi100.stdout: "use_by_application": 35918176, 2024-04-07T16:16:15.930 INFO:tasks.workunit.client.0.smithi100.stdout: "virtual_address_space_used": 51560448 2024-04-07T16:16:15.930 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.930 INFO:tasks.workunit.client.0.smithi100.stdout: "osd.3": { 2024-04-07T16:16:15.930 INFO:tasks.workunit.client.0.smithi100.stdout: "actual_memory_used": 44531712, 2024-04-07T16:16:15.930 INFO:tasks.workunit.client.0.smithi100.stdout: "central_cache_freelist": 608560, 2024-04-07T16:16:15.930 INFO:tasks.workunit.client.0.smithi100.stdout: "malloc_metadata": 2883584, 2024-04-07T16:16:15.930 INFO:tasks.workunit.client.0.smithi100.stdout: "page_heap_freelist": 0, 2024-04-07T16:16:15.930 INFO:tasks.workunit.client.0.smithi100.stdout: "released_to_os": 671744, 2024-04-07T16:16:15.931 INFO:tasks.workunit.client.0.smithi100.stdout: "spans_in_use": 1281, 2024-04-07T16:16:15.931 INFO:tasks.workunit.client.0.smithi100.stdout: "tcmalloc_page_size": 8192, 2024-04-07T16:16:15.931 INFO:tasks.workunit.client.0.smithi100.stdout: "thread_cache_freelists": 4415144, 2024-04-07T16:16:15.931 INFO:tasks.workunit.client.0.smithi100.stdout: "thread_heaps_in_use": 45, 2024-04-07T16:16:15.931 INFO:tasks.workunit.client.0.smithi100.stdout: "transfer_cache_freelist": 1865472, 2024-04-07T16:16:15.931 INFO:tasks.workunit.client.0.smithi100.stdout: "use_by_application": 34758952, 2024-04-07T16:16:15.931 INFO:tasks.workunit.client.0.smithi100.stdout: "virtual_address_space_used": 45203456 2024-04-07T16:16:15.931 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.931 INFO:tasks.workunit.client.0.smithi100.stdout: "osd.4": { 2024-04-07T16:16:15.931 INFO:tasks.workunit.client.0.smithi100.stdout: "actual_memory_used": 48398336, 2024-04-07T16:16:15.931 INFO:tasks.workunit.client.0.smithi100.stdout: "central_cache_freelist": 1920712, 2024-04-07T16:16:15.931 INFO:tasks.workunit.client.0.smithi100.stdout: "malloc_metadata": 2883584, 2024-04-07T16:16:15.931 INFO:tasks.workunit.client.0.smithi100.stdout: "page_heap_freelist": 0, 2024-04-07T16:16:15.931 INFO:tasks.workunit.client.0.smithi100.stdout: "released_to_os": 933888, 2024-04-07T16:16:15.932 INFO:tasks.workunit.client.0.smithi100.stdout: "spans_in_use": 1652, 2024-04-07T16:16:15.932 INFO:tasks.workunit.client.0.smithi100.stdout: "tcmalloc_page_size": 8192, 2024-04-07T16:16:15.932 INFO:tasks.workunit.client.0.smithi100.stdout: "thread_cache_freelists": 5689960, 2024-04-07T16:16:15.932 INFO:tasks.workunit.client.0.smithi100.stdout: "thread_heaps_in_use": 47, 2024-04-07T16:16:15.932 INFO:tasks.workunit.client.0.smithi100.stdout: "transfer_cache_freelist": 3028736, 2024-04-07T16:16:15.932 INFO:tasks.workunit.client.0.smithi100.stdout: "use_by_application": 34875344, 2024-04-07T16:16:15.932 INFO:tasks.workunit.client.0.smithi100.stdout: "virtual_address_space_used": 49332224 2024-04-07T16:16:15.932 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.932 INFO:tasks.workunit.client.0.smithi100.stdout: "osd.5": { 2024-04-07T16:16:15.932 INFO:tasks.workunit.client.0.smithi100.stdout: "actual_memory_used": 45416448, 2024-04-07T16:16:15.932 INFO:tasks.workunit.client.0.smithi100.stdout: "central_cache_freelist": 928960, 2024-04-07T16:16:15.932 INFO:tasks.workunit.client.0.smithi100.stdout: "malloc_metadata": 2883584, 2024-04-07T16:16:15.932 INFO:tasks.workunit.client.0.smithi100.stdout: "page_heap_freelist": 0, 2024-04-07T16:16:15.932 INFO:tasks.workunit.client.0.smithi100.stdout: "released_to_os": 901120, 2024-04-07T16:16:15.932 INFO:tasks.workunit.client.0.smithi100.stdout: "spans_in_use": 1290, 2024-04-07T16:16:15.932 INFO:tasks.workunit.client.0.smithi100.stdout: "tcmalloc_page_size": 8192, 2024-04-07T16:16:15.933 INFO:tasks.workunit.client.0.smithi100.stdout: "thread_cache_freelists": 4542592, 2024-04-07T16:16:15.933 INFO:tasks.workunit.client.0.smithi100.stdout: "thread_heaps_in_use": 48, 2024-04-07T16:16:15.933 INFO:tasks.workunit.client.0.smithi100.stdout: "transfer_cache_freelist": 1942784, 2024-04-07T16:16:15.933 INFO:tasks.workunit.client.0.smithi100.stdout: "use_by_application": 35118528, 2024-04-07T16:16:15.933 INFO:tasks.workunit.client.0.smithi100.stdout: "virtual_address_space_used": 46317568 2024-04-07T16:16:15.933 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.933 INFO:tasks.workunit.client.0.smithi100.stdout: "osd.6": { 2024-04-07T16:16:15.933 INFO:tasks.workunit.client.0.smithi100.stdout: "actual_memory_used": 45883392, 2024-04-07T16:16:15.933 INFO:tasks.workunit.client.0.smithi100.stdout: "central_cache_freelist": 1034064, 2024-04-07T16:16:15.933 INFO:tasks.workunit.client.0.smithi100.stdout: "malloc_metadata": 2883584, 2024-04-07T16:16:15.933 INFO:tasks.workunit.client.0.smithi100.stdout: "page_heap_freelist": 0, 2024-04-07T16:16:15.933 INFO:tasks.workunit.client.0.smithi100.stdout: "released_to_os": 581632, 2024-04-07T16:16:15.933 INFO:tasks.workunit.client.0.smithi100.stdout: "spans_in_use": 1297, 2024-04-07T16:16:15.933 INFO:tasks.workunit.client.0.smithi100.stdout: "tcmalloc_page_size": 8192, 2024-04-07T16:16:15.934 INFO:tasks.workunit.client.0.smithi100.stdout: "thread_cache_freelists": 4963552, 2024-04-07T16:16:15.934 INFO:tasks.workunit.client.0.smithi100.stdout: "thread_heaps_in_use": 46, 2024-04-07T16:16:15.934 INFO:tasks.workunit.client.0.smithi100.stdout: "transfer_cache_freelist": 2013952, 2024-04-07T16:16:15.934 INFO:tasks.workunit.client.0.smithi100.stdout: "use_by_application": 34988240, 2024-04-07T16:16:15.934 INFO:tasks.workunit.client.0.smithi100.stdout: "virtual_address_space_used": 46465024 2024-04-07T16:16:15.934 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.934 INFO:tasks.workunit.client.0.smithi100.stdout: "osd.7": { 2024-04-07T16:16:15.934 INFO:tasks.workunit.client.0.smithi100.stdout: "actual_memory_used": 49004544, 2024-04-07T16:16:15.934 INFO:tasks.workunit.client.0.smithi100.stdout: "central_cache_freelist": 2069032, 2024-04-07T16:16:15.934 INFO:tasks.workunit.client.0.smithi100.stdout: "malloc_metadata": 2883584, 2024-04-07T16:16:15.934 INFO:tasks.workunit.client.0.smithi100.stdout: "page_heap_freelist": 0, 2024-04-07T16:16:15.934 INFO:tasks.workunit.client.0.smithi100.stdout: "released_to_os": 671744, 2024-04-07T16:16:15.934 INFO:tasks.workunit.client.0.smithi100.stdout: "spans_in_use": 1721, 2024-04-07T16:16:15.934 INFO:tasks.workunit.client.0.smithi100.stdout: "tcmalloc_page_size": 8192, 2024-04-07T16:16:15.934 INFO:tasks.workunit.client.0.smithi100.stdout: "thread_cache_freelists": 5835048, 2024-04-07T16:16:15.935 INFO:tasks.workunit.client.0.smithi100.stdout: "thread_heaps_in_use": 47, 2024-04-07T16:16:15.935 INFO:tasks.workunit.client.0.smithi100.stdout: "transfer_cache_freelist": 3345664, 2024-04-07T16:16:15.935 INFO:tasks.workunit.client.0.smithi100.stdout: "use_by_application": 34871216, 2024-04-07T16:16:15.935 INFO:tasks.workunit.client.0.smithi100.stdout: "virtual_address_space_used": 49676288 2024-04-07T16:16:15.935 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:15.935 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:15.935 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.935 INFO:tasks.workunit.client.0.smithi100.stdout: "hosts": { 2024-04-07T16:16:15.935 INFO:tasks.workunit.client.0.smithi100.stdout: "num": 2, 2024-04-07T16:16:15.935 INFO:tasks.workunit.client.0.smithi100.stdout: "num_with_mds": 2, 2024-04-07T16:16:15.935 INFO:tasks.workunit.client.0.smithi100.stdout: "num_with_mgr": 2, 2024-04-07T16:16:15.935 INFO:tasks.workunit.client.0.smithi100.stdout: "num_with_mon": 2, 2024-04-07T16:16:15.935 INFO:tasks.workunit.client.0.smithi100.stdout: "num_with_osd": 2 2024-04-07T16:16:15.935 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.935 INFO:tasks.workunit.client.0.smithi100.stdout: "io_rate": { 2024-04-07T16:16:15.936 INFO:tasks.workunit.client.0.smithi100.stdout: "pg_stats_delta": { 2024-04-07T16:16:15.936 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": 0, 2024-04-07T16:16:15.936 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:15.936 INFO:tasks.workunit.client.0.smithi100.stdout: "num_store_stats": 0, 2024-04-07T16:16:15.936 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:15.936 INFO:tasks.workunit.client.0.smithi100.stdout: "stamp_delta": "12.004431", 2024-04-07T16:16:15.936 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:15.936 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:15.936 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:15.936 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:15.936 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:15.936 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:15.936 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:15.936 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:15.936 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:15.937 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:15.937 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:15.937 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:15.937 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:15.937 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:15.937 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:15.937 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:15.937 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:15.937 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:15.937 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:15.937 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:15.937 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:15.937 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:15.937 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:15.937 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:15.938 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:15.938 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:15.938 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:15.938 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:15.938 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:15.938 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:15.938 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:15.938 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:15.938 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:15.938 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:15.938 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 254, 2024-04-07T16:16:15.938 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 338, 2024-04-07T16:16:15.938 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:15.938 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:15.938 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:15.939 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 56, 2024-04-07T16:16:15.939 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:15.939 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.939 INFO:tasks.workunit.client.0.smithi100.stdout: "store_stats": { 2024-04-07T16:16:15.939 INFO:tasks.workunit.client.0.smithi100.stdout: "allocated": 0, 2024-04-07T16:16:15.939 INFO:tasks.workunit.client.0.smithi100.stdout: "available": 0, 2024-04-07T16:16:15.939 INFO:tasks.workunit.client.0.smithi100.stdout: "data_compressed": 0, 2024-04-07T16:16:15.939 INFO:tasks.workunit.client.0.smithi100.stdout: "data_compressed_allocated": 0, 2024-04-07T16:16:15.939 INFO:tasks.workunit.client.0.smithi100.stdout: "data_compressed_original": 0, 2024-04-07T16:16:15.939 INFO:tasks.workunit.client.0.smithi100.stdout: "data_stored": 0, 2024-04-07T16:16:15.939 INFO:tasks.workunit.client.0.smithi100.stdout: "internal_metadata": 0, 2024-04-07T16:16:15.939 INFO:tasks.workunit.client.0.smithi100.stdout: "internally_reserved": 0, 2024-04-07T16:16:15.939 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_allocated": 0, 2024-04-07T16:16:15.939 INFO:tasks.workunit.client.0.smithi100.stdout: "total": 0 2024-04-07T16:16:15.939 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.940 INFO:tasks.workunit.client.0.smithi100.stdout: "up": 0 2024-04-07T16:16:15.940 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:15.940 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.940 INFO:tasks.workunit.client.0.smithi100.stdout: "leaderboard": false, 2024-04-07T16:16:15.940 INFO:tasks.workunit.client.0.smithi100.stdout: "leaderboard_description": null, 2024-04-07T16:16:15.940 INFO:tasks.workunit.client.0.smithi100.stdout: "license": "sharing-1-0", 2024-04-07T16:16:15.940 INFO:tasks.workunit.client.0.smithi100.stdout: "mempool": { 2024-04-07T16:16:15.940 INFO:tasks.workunit.client.0.smithi100.stdout: "mds": { 2024-04-07T16:16:15.940 INFO:tasks.workunit.client.0.smithi100.stdout: "mds.a52fc0d8acb515431879057649ff294cfe14c620": { 2024-04-07T16:16:15.940 INFO:tasks.workunit.client.0.smithi100.stdout: "bloom_filter": { 2024-04-07T16:16:15.940 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:15.940 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:15.940 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.940 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs": { 2024-04-07T16:16:15.940 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:15.941 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:15.941 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.941 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs_file_reader": { 2024-04-07T16:16:15.941 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:15.941 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:15.941 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.941 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs_file_writer": { 2024-04-07T16:16:15.941 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:15.941 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:15.941 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.941 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_alloc": { 2024-04-07T16:16:15.941 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:15.941 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:15.941 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.942 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_blob": { 2024-04-07T16:16:15.942 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:15.942 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:15.942 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.942 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_buffer": { 2024-04-07T16:16:15.942 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:15.942 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:15.942 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.942 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_data": { 2024-04-07T16:16:15.942 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:15.942 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:15.942 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.942 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_meta": { 2024-04-07T16:16:15.943 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:15.943 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:15.943 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.943 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_onode": { 2024-04-07T16:16:15.943 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:15.943 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:15.943 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.943 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_other": { 2024-04-07T16:16:15.943 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:15.943 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:15.943 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.943 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_extent": { 2024-04-07T16:16:15.943 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:15.944 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:15.944 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.944 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_fsck": { 2024-04-07T16:16:15.944 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:15.944 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:15.944 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.944 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_inline_bl": { 2024-04-07T16:16:15.944 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:15.944 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:15.944 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.944 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_shared_blob": { 2024-04-07T16:16:15.944 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:15.945 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:15.945 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.945 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_txc": { 2024-04-07T16:16:15.945 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:15.945 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:15.945 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.945 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_writing": { 2024-04-07T16:16:15.945 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:15.945 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:15.945 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.945 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_writing_deferred": { 2024-04-07T16:16:15.945 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:15.945 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:15.945 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.945 INFO:tasks.workunit.client.0.smithi100.stdout: "buffer_anon": { 2024-04-07T16:16:15.946 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 222727, 2024-04-07T16:16:15.946 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 72 2024-04-07T16:16:15.946 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.946 INFO:tasks.workunit.client.0.smithi100.stdout: "buffer_meta": { 2024-04-07T16:16:15.946 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:15.946 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:15.946 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.946 INFO:tasks.workunit.client.0.smithi100.stdout: "mds_co": { 2024-04-07T16:16:15.946 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 51496, 2024-04-07T16:16:15.946 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 208 2024-04-07T16:16:15.946 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.946 INFO:tasks.workunit.client.0.smithi100.stdout: "osd": { 2024-04-07T16:16:15.946 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:15.946 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:15.947 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.947 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mapbl": { 2024-04-07T16:16:15.947 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:15.947 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:15.947 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.947 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_pglog": { 2024-04-07T16:16:15.947 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:15.947 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:15.947 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.947 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap": { 2024-04-07T16:16:15.947 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 7912, 2024-04-07T16:16:15.947 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 123 2024-04-07T16:16:15.947 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.947 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap_mapping": { 2024-04-07T16:16:15.947 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:15.947 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:15.948 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.948 INFO:tasks.workunit.client.0.smithi100.stdout: "pgmap": { 2024-04-07T16:16:15.948 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:15.948 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:15.948 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.948 INFO:tasks.workunit.client.0.smithi100.stdout: "unittest_1": { 2024-04-07T16:16:15.948 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:15.948 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:15.948 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.948 INFO:tasks.workunit.client.0.smithi100.stdout: "unittest_2": { 2024-04-07T16:16:15.948 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:15.948 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:15.948 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:15.948 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.948 INFO:tasks.workunit.client.0.smithi100.stdout: "mds.f47a4ff15a45950a63160265564a123fc5da5e5e": { 2024-04-07T16:16:15.949 INFO:tasks.workunit.client.0.smithi100.stdout: "bloom_filter": { 2024-04-07T16:16:15.949 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:15.949 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:15.949 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.949 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs": { 2024-04-07T16:16:15.949 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:15.949 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:15.949 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.949 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs_file_reader": { 2024-04-07T16:16:15.949 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:15.949 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:15.949 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.949 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs_file_writer": { 2024-04-07T16:16:15.949 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:15.950 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:15.950 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.950 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_alloc": { 2024-04-07T16:16:15.950 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:15.950 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:15.950 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.950 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_blob": { 2024-04-07T16:16:15.950 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:15.950 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:15.950 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.950 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_buffer": { 2024-04-07T16:16:15.950 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:15.950 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:15.950 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.950 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_data": { 2024-04-07T16:16:15.951 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:15.951 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:15.951 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.951 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_meta": { 2024-04-07T16:16:15.951 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:15.951 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:15.951 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.951 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_onode": { 2024-04-07T16:16:15.951 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:15.951 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:15.951 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.951 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_other": { 2024-04-07T16:16:15.952 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:15.952 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:15.952 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.952 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_extent": { 2024-04-07T16:16:15.952 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:15.952 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:15.952 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.952 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_fsck": { 2024-04-07T16:16:15.952 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:15.952 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:15.952 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.952 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_inline_bl": { 2024-04-07T16:16:15.953 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:15.953 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:15.953 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.953 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_shared_blob": { 2024-04-07T16:16:15.953 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:15.953 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:15.953 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.953 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_txc": { 2024-04-07T16:16:15.953 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:15.953 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:15.953 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.953 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_writing": { 2024-04-07T16:16:15.953 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:15.953 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:15.953 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.954 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_writing_deferred": { 2024-04-07T16:16:15.954 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:15.954 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:15.954 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.954 INFO:tasks.workunit.client.0.smithi100.stdout: "buffer_anon": { 2024-04-07T16:16:15.954 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 150208, 2024-04-07T16:16:15.954 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 34 2024-04-07T16:16:15.954 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.954 INFO:tasks.workunit.client.0.smithi100.stdout: "buffer_meta": { 2024-04-07T16:16:15.954 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:15.954 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:15.954 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.954 INFO:tasks.workunit.client.0.smithi100.stdout: "mds_co": { 2024-04-07T16:16:15.954 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 928, 2024-04-07T16:16:15.955 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 2 2024-04-07T16:16:15.955 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.955 INFO:tasks.workunit.client.0.smithi100.stdout: "osd": { 2024-04-07T16:16:15.955 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:15.955 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:15.955 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.955 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mapbl": { 2024-04-07T16:16:15.955 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:15.955 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:15.955 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.955 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_pglog": { 2024-04-07T16:16:15.955 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:15.955 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:15.955 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.956 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap": { 2024-04-07T16:16:15.956 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:15.956 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:15.956 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.956 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap_mapping": { 2024-04-07T16:16:15.956 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:15.956 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:15.956 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.956 INFO:tasks.workunit.client.0.smithi100.stdout: "pgmap": { 2024-04-07T16:16:15.956 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:15.956 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:15.956 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.956 INFO:tasks.workunit.client.0.smithi100.stdout: "unittest_1": { 2024-04-07T16:16:15.956 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:15.956 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:15.956 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.957 INFO:tasks.workunit.client.0.smithi100.stdout: "unittest_2": { 2024-04-07T16:16:15.957 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:15.957 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:15.957 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:15.957 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:15.957 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.957 INFO:tasks.workunit.client.0.smithi100.stdout: "mon": { 2024-04-07T16:16:15.957 INFO:tasks.workunit.client.0.smithi100.stdout: "mon.8dd0baa036f0a7e6d1f3bdce86136fe527836c3d": { 2024-04-07T16:16:15.957 INFO:tasks.workunit.client.0.smithi100.stdout: "bloom_filter": { 2024-04-07T16:16:15.957 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:15.957 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:15.957 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.957 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs": { 2024-04-07T16:16:15.958 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:15.958 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:15.958 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.958 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs_file_reader": { 2024-04-07T16:16:15.958 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:15.958 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:15.958 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.958 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs_file_writer": { 2024-04-07T16:16:15.958 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:15.958 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:15.958 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.958 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_alloc": { 2024-04-07T16:16:15.959 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:15.959 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:15.959 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.959 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_blob": { 2024-04-07T16:16:15.959 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:15.959 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:15.959 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.959 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_buffer": { 2024-04-07T16:16:15.959 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:15.959 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:15.959 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.959 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_data": { 2024-04-07T16:16:15.959 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:15.959 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:15.960 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.960 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_meta": { 2024-04-07T16:16:15.960 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:15.960 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:15.960 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.960 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_onode": { 2024-04-07T16:16:15.960 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:15.960 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:15.960 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.960 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_other": { 2024-04-07T16:16:15.960 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:15.960 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:15.961 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.961 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_extent": { 2024-04-07T16:16:15.961 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:15.961 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:15.961 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.961 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_fsck": { 2024-04-07T16:16:15.961 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:15.961 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:15.961 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.961 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_inline_bl": { 2024-04-07T16:16:15.961 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:15.961 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:15.961 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.961 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_shared_blob": { 2024-04-07T16:16:15.961 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:15.962 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:15.962 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.962 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_txc": { 2024-04-07T16:16:15.962 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:15.962 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:15.962 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.962 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_writing": { 2024-04-07T16:16:15.962 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:15.962 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:15.962 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.962 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_writing_deferred": { 2024-04-07T16:16:15.962 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:15.962 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:15.962 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.963 INFO:tasks.workunit.client.0.smithi100.stdout: "buffer_anon": { 2024-04-07T16:16:15.963 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 1097676, 2024-04-07T16:16:15.963 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 361 2024-04-07T16:16:15.963 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.963 INFO:tasks.workunit.client.0.smithi100.stdout: "buffer_meta": { 2024-04-07T16:16:15.963 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:15.963 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:15.963 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.963 INFO:tasks.workunit.client.0.smithi100.stdout: "mds_co": { 2024-04-07T16:16:15.963 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:15.963 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:15.963 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.963 INFO:tasks.workunit.client.0.smithi100.stdout: "osd": { 2024-04-07T16:16:15.964 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:15.964 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:15.964 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.964 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mapbl": { 2024-04-07T16:16:15.964 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:15.964 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:15.964 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.964 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_pglog": { 2024-04-07T16:16:15.964 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:15.964 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:15.964 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.964 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap": { 2024-04-07T16:16:15.964 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 6488, 2024-04-07T16:16:15.964 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 122 2024-04-07T16:16:15.965 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.965 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap_mapping": { 2024-04-07T16:16:15.965 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 12744, 2024-04-07T16:16:15.965 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 1546 2024-04-07T16:16:15.965 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.965 INFO:tasks.workunit.client.0.smithi100.stdout: "pgmap": { 2024-04-07T16:16:15.965 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 5152, 2024-04-07T16:16:15.965 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 83 2024-04-07T16:16:15.965 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.965 INFO:tasks.workunit.client.0.smithi100.stdout: "unittest_1": { 2024-04-07T16:16:15.965 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:15.965 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:15.965 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.965 INFO:tasks.workunit.client.0.smithi100.stdout: "unittest_2": { 2024-04-07T16:16:15.965 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:15.966 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:15.966 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:15.966 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.966 INFO:tasks.workunit.client.0.smithi100.stdout: "mon.a52fc0d8acb515431879057649ff294cfe14c620": { 2024-04-07T16:16:15.966 INFO:tasks.workunit.client.0.smithi100.stdout: "bloom_filter": { 2024-04-07T16:16:15.966 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:15.966 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:15.966 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.966 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs": { 2024-04-07T16:16:15.966 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:15.966 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:15.966 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.966 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs_file_reader": { 2024-04-07T16:16:15.966 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:15.967 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:15.967 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.967 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs_file_writer": { 2024-04-07T16:16:15.967 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:15.967 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:15.967 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.967 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_alloc": { 2024-04-07T16:16:15.967 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:15.967 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:15.967 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.967 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_blob": { 2024-04-07T16:16:15.967 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:15.967 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:15.967 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.967 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_buffer": { 2024-04-07T16:16:15.968 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:15.968 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:15.968 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.968 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_data": { 2024-04-07T16:16:15.968 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:15.968 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:15.968 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.968 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_meta": { 2024-04-07T16:16:15.968 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:15.968 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:15.968 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.968 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_onode": { 2024-04-07T16:16:15.968 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:15.968 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:15.968 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.969 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_other": { 2024-04-07T16:16:15.969 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:15.969 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:15.969 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.969 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_extent": { 2024-04-07T16:16:15.969 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:15.969 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:15.969 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.969 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_fsck": { 2024-04-07T16:16:15.969 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:15.969 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:15.969 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.969 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_inline_bl": { 2024-04-07T16:16:15.969 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:15.969 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:15.970 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.970 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_shared_blob": { 2024-04-07T16:16:15.970 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:15.970 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:15.970 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.970 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_txc": { 2024-04-07T16:16:15.970 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:15.970 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:15.970 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.970 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_writing": { 2024-04-07T16:16:15.970 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:15.970 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:15.970 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.970 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_writing_deferred": { 2024-04-07T16:16:15.971 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:15.971 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:15.971 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.971 INFO:tasks.workunit.client.0.smithi100.stdout: "buffer_anon": { 2024-04-07T16:16:15.971 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 1142772, 2024-04-07T16:16:15.971 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 367 2024-04-07T16:16:15.971 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.971 INFO:tasks.workunit.client.0.smithi100.stdout: "buffer_meta": { 2024-04-07T16:16:15.971 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:15.971 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:15.971 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.971 INFO:tasks.workunit.client.0.smithi100.stdout: "mds_co": { 2024-04-07T16:16:15.971 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:15.971 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:15.971 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.972 INFO:tasks.workunit.client.0.smithi100.stdout: "osd": { 2024-04-07T16:16:15.972 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:15.972 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:15.972 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.972 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mapbl": { 2024-04-07T16:16:15.972 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:15.972 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:15.972 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.972 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_pglog": { 2024-04-07T16:16:15.972 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:15.972 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:15.972 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.972 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap": { 2024-04-07T16:16:15.972 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 6488, 2024-04-07T16:16:15.972 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 122 2024-04-07T16:16:15.972 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.973 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap_mapping": { 2024-04-07T16:16:15.973 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 12744, 2024-04-07T16:16:15.973 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 1546 2024-04-07T16:16:15.973 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.973 INFO:tasks.workunit.client.0.smithi100.stdout: "pgmap": { 2024-04-07T16:16:15.973 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 5152, 2024-04-07T16:16:15.973 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 83 2024-04-07T16:16:15.973 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.973 INFO:tasks.workunit.client.0.smithi100.stdout: "unittest_1": { 2024-04-07T16:16:15.973 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:15.973 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:15.973 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.973 INFO:tasks.workunit.client.0.smithi100.stdout: "unittest_2": { 2024-04-07T16:16:15.973 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:15.974 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:15.974 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:15.974 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.974 INFO:tasks.workunit.client.0.smithi100.stdout: "mon.f47a4ff15a45950a63160265564a123fc5da5e5e": { 2024-04-07T16:16:15.974 INFO:tasks.workunit.client.0.smithi100.stdout: "bloom_filter": { 2024-04-07T16:16:15.974 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:15.974 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:15.974 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.974 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs": { 2024-04-07T16:16:15.974 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:15.974 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:15.974 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.974 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs_file_reader": { 2024-04-07T16:16:15.975 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:15.975 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:15.975 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.975 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs_file_writer": { 2024-04-07T16:16:15.975 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:15.975 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:15.975 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.975 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_alloc": { 2024-04-07T16:16:15.975 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:15.975 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:15.975 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.975 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_blob": { 2024-04-07T16:16:15.975 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:15.975 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:15.975 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.975 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_buffer": { 2024-04-07T16:16:15.976 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:15.976 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:15.976 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.976 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_data": { 2024-04-07T16:16:15.976 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:15.976 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:15.976 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.976 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_meta": { 2024-04-07T16:16:15.976 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:15.976 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:15.976 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.976 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_onode": { 2024-04-07T16:16:15.976 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:15.977 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:15.977 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.977 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_other": { 2024-04-07T16:16:15.977 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:15.977 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:15.977 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.977 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_extent": { 2024-04-07T16:16:15.977 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:15.977 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:15.977 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.977 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_fsck": { 2024-04-07T16:16:15.977 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:15.977 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:15.977 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.977 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_inline_bl": { 2024-04-07T16:16:15.978 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:15.978 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:15.978 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.978 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_shared_blob": { 2024-04-07T16:16:15.978 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:15.978 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:15.978 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.978 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_txc": { 2024-04-07T16:16:15.978 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:15.978 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:15.978 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.978 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_writing": { 2024-04-07T16:16:15.978 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:15.978 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:15.979 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.979 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_writing_deferred": { 2024-04-07T16:16:15.979 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:15.979 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:15.979 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.979 INFO:tasks.workunit.client.0.smithi100.stdout: "buffer_anon": { 2024-04-07T16:16:15.979 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 1041516, 2024-04-07T16:16:15.979 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 331 2024-04-07T16:16:15.979 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.979 INFO:tasks.workunit.client.0.smithi100.stdout: "buffer_meta": { 2024-04-07T16:16:15.979 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:15.979 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:15.979 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.979 INFO:tasks.workunit.client.0.smithi100.stdout: "mds_co": { 2024-04-07T16:16:15.979 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:15.980 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:15.980 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.980 INFO:tasks.workunit.client.0.smithi100.stdout: "osd": { 2024-04-07T16:16:15.980 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:15.980 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:15.980 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.980 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mapbl": { 2024-04-07T16:16:15.980 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:15.980 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:15.980 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.980 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_pglog": { 2024-04-07T16:16:15.980 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:15.980 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:15.980 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.980 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap": { 2024-04-07T16:16:15.981 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 6488, 2024-04-07T16:16:15.981 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 122 2024-04-07T16:16:15.981 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.981 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap_mapping": { 2024-04-07T16:16:15.981 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 12744, 2024-04-07T16:16:15.981 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 1546 2024-04-07T16:16:15.981 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.981 INFO:tasks.workunit.client.0.smithi100.stdout: "pgmap": { 2024-04-07T16:16:15.981 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 10304, 2024-04-07T16:16:15.981 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 166 2024-04-07T16:16:15.981 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.981 INFO:tasks.workunit.client.0.smithi100.stdout: "unittest_1": { 2024-04-07T16:16:15.981 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:15.981 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:15.981 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.982 INFO:tasks.workunit.client.0.smithi100.stdout: "unittest_2": { 2024-04-07T16:16:15.982 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:15.982 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:15.982 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:15.982 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:15.982 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.982 INFO:tasks.workunit.client.0.smithi100.stdout: "osd": { 2024-04-07T16:16:15.982 INFO:tasks.workunit.client.0.smithi100.stdout: "osd.0": { 2024-04-07T16:16:15.982 INFO:tasks.workunit.client.0.smithi100.stdout: "bloom_filter": { 2024-04-07T16:16:15.982 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:15.982 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:15.982 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.982 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs": { 2024-04-07T16:16:15.982 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 6368, 2024-04-07T16:16:15.982 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 123 2024-04-07T16:16:15.983 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.983 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs_file_reader": { 2024-04-07T16:16:15.983 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 16896, 2024-04-07T16:16:15.983 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 8 2024-04-07T16:16:15.983 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.983 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs_file_writer": { 2024-04-07T16:16:15.983 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 576, 2024-04-07T16:16:15.983 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 3 2024-04-07T16:16:15.983 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.983 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_alloc": { 2024-04-07T16:16:15.983 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 5600, 2024-04-07T16:16:15.983 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 70 2024-04-07T16:16:15.983 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.983 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_blob": { 2024-04-07T16:16:15.983 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 20944, 2024-04-07T16:16:15.984 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 187 2024-04-07T16:16:15.984 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.984 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_buffer": { 2024-04-07T16:16:15.984 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 1936, 2024-04-07T16:16:15.984 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 47 2024-04-07T16:16:15.984 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.984 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_data": { 2024-04-07T16:16:15.984 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 620927, 2024-04-07T16:16:15.984 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 20 2024-04-07T16:16:15.984 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.984 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_meta": { 2024-04-07T16:16:15.984 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 50844, 2024-04-07T16:16:15.984 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 9431 2024-04-07T16:16:15.984 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.984 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_onode": { 2024-04-07T16:16:15.985 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 136256, 2024-04-07T16:16:15.985 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 232 2024-04-07T16:16:15.985 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.985 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_other": { 2024-04-07T16:16:15.985 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 11260, 2024-04-07T16:16:15.985 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 741 2024-04-07T16:16:15.985 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.985 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_extent": { 2024-04-07T16:16:15.985 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 8976, 2024-04-07T16:16:15.985 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 187 2024-04-07T16:16:15.985 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.985 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_fsck": { 2024-04-07T16:16:15.985 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:15.985 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:15.985 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.985 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_inline_bl": { 2024-04-07T16:16:15.986 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 31446, 2024-04-07T16:16:15.986 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 200 2024-04-07T16:16:15.986 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.986 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_shared_blob": { 2024-04-07T16:16:15.986 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 20944, 2024-04-07T16:16:15.986 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 187 2024-04-07T16:16:15.986 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.986 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_txc": { 2024-04-07T16:16:15.986 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:15.986 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:15.986 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.986 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_writing": { 2024-04-07T16:16:15.986 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 407616, 2024-04-07T16:16:15.986 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 91 2024-04-07T16:16:15.986 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.986 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_writing_deferred": { 2024-04-07T16:16:15.987 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:15.987 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:15.987 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.987 INFO:tasks.workunit.client.0.smithi100.stdout: "buffer_anon": { 2024-04-07T16:16:15.987 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 2460800, 2024-04-07T16:16:15.987 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 202 2024-04-07T16:16:15.987 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.987 INFO:tasks.workunit.client.0.smithi100.stdout: "buffer_meta": { 2024-04-07T16:16:15.987 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 6952, 2024-04-07T16:16:15.987 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 79 2024-04-07T16:16:15.987 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.987 INFO:tasks.workunit.client.0.smithi100.stdout: "mds_co": { 2024-04-07T16:16:15.987 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:15.987 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:15.987 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.987 INFO:tasks.workunit.client.0.smithi100.stdout: "osd": { 2024-04-07T16:16:15.988 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 510048, 2024-04-07T16:16:15.988 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 44 2024-04-07T16:16:15.988 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.988 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mapbl": { 2024-04-07T16:16:15.988 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:15.988 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:15.988 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.988 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_pglog": { 2024-04-07T16:16:15.988 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 98800, 2024-04-07T16:16:15.988 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 194 2024-04-07T16:16:15.988 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.988 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap": { 2024-04-07T16:16:15.988 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 225512, 2024-04-07T16:16:15.988 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 3460 2024-04-07T16:16:15.988 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.988 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap_mapping": { 2024-04-07T16:16:15.988 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:15.989 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:15.989 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.989 INFO:tasks.workunit.client.0.smithi100.stdout: "pgmap": { 2024-04-07T16:16:15.989 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:15.989 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:15.989 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.989 INFO:tasks.workunit.client.0.smithi100.stdout: "unittest_1": { 2024-04-07T16:16:15.989 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:15.989 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:15.989 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.990 INFO:tasks.workunit.client.0.smithi100.stdout: "unittest_2": { 2024-04-07T16:16:15.990 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:15.990 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:15.990 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:15.990 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.990 INFO:tasks.workunit.client.0.smithi100.stdout: "osd.1": { 2024-04-07T16:16:15.990 INFO:tasks.workunit.client.0.smithi100.stdout: "bloom_filter": { 2024-04-07T16:16:15.990 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:15.990 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:15.990 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.990 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs": { 2024-04-07T16:16:15.990 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 6368, 2024-04-07T16:16:15.990 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 123 2024-04-07T16:16:15.990 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.990 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs_file_reader": { 2024-04-07T16:16:15.991 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 16896, 2024-04-07T16:16:15.991 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 8 2024-04-07T16:16:15.991 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.991 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs_file_writer": { 2024-04-07T16:16:15.991 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 576, 2024-04-07T16:16:15.991 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 3 2024-04-07T16:16:15.991 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.991 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_alloc": { 2024-04-07T16:16:15.991 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 4720, 2024-04-07T16:16:15.991 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 59 2024-04-07T16:16:15.991 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.991 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_blob": { 2024-04-07T16:16:15.991 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 19600, 2024-04-07T16:16:15.991 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 175 2024-04-07T16:16:15.992 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.992 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_buffer": { 2024-04-07T16:16:15.992 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 1264, 2024-04-07T16:16:15.992 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 41 2024-04-07T16:16:15.992 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.992 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_data": { 2024-04-07T16:16:15.992 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 16384, 2024-04-07T16:16:15.992 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 4 2024-04-07T16:16:15.992 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.992 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_meta": { 2024-04-07T16:16:15.992 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 45252, 2024-04-07T16:16:15.992 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 8555 2024-04-07T16:16:15.992 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.992 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_onode": { 2024-04-07T16:16:15.992 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 128096, 2024-04-07T16:16:15.992 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 220 2024-04-07T16:16:15.993 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.993 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_other": { 2024-04-07T16:16:15.993 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 9440, 2024-04-07T16:16:15.993 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 568 2024-04-07T16:16:15.993 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.993 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_extent": { 2024-04-07T16:16:15.993 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 8400, 2024-04-07T16:16:15.993 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 175 2024-04-07T16:16:15.994 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.994 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_fsck": { 2024-04-07T16:16:15.994 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:15.994 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:15.994 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.994 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_inline_bl": { 2024-04-07T16:16:15.994 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 27770, 2024-04-07T16:16:15.994 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 188 2024-04-07T16:16:15.994 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.994 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_shared_blob": { 2024-04-07T16:16:15.994 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 19600, 2024-04-07T16:16:15.994 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 175 2024-04-07T16:16:15.994 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.995 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_txc": { 2024-04-07T16:16:15.995 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 736, 2024-04-07T16:16:15.995 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 1 2024-04-07T16:16:15.995 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.995 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_writing": { 2024-04-07T16:16:15.995 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 434028, 2024-04-07T16:16:15.995 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 97 2024-04-07T16:16:15.995 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.995 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_writing_deferred": { 2024-04-07T16:16:15.995 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 12112, 2024-04-07T16:16:15.995 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 3 2024-04-07T16:16:15.995 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.995 INFO:tasks.workunit.client.0.smithi100.stdout: "buffer_anon": { 2024-04-07T16:16:15.995 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 2420101, 2024-04-07T16:16:15.996 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 185 2024-04-07T16:16:15.996 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.996 INFO:tasks.workunit.client.0.smithi100.stdout: "buffer_meta": { 2024-04-07T16:16:15.996 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 5016, 2024-04-07T16:16:15.996 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 57 2024-04-07T16:16:15.996 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.996 INFO:tasks.workunit.client.0.smithi100.stdout: "mds_co": { 2024-04-07T16:16:15.996 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:15.996 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:15.996 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.996 INFO:tasks.workunit.client.0.smithi100.stdout: "osd": { 2024-04-07T16:16:15.996 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 440496, 2024-04-07T16:16:15.996 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 38 2024-04-07T16:16:15.997 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.997 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mapbl": { 2024-04-07T16:16:15.997 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:15.997 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:15.997 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.997 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_pglog": { 2024-04-07T16:16:15.997 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 7280, 2024-04-07T16:16:15.997 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 18 2024-04-07T16:16:15.997 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.997 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap": { 2024-04-07T16:16:15.997 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 225512, 2024-04-07T16:16:15.997 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 3460 2024-04-07T16:16:15.997 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.997 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap_mapping": { 2024-04-07T16:16:15.997 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:15.998 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:15.998 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.998 INFO:tasks.workunit.client.0.smithi100.stdout: "pgmap": { 2024-04-07T16:16:15.998 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:15.998 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:15.998 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.998 INFO:tasks.workunit.client.0.smithi100.stdout: "unittest_1": { 2024-04-07T16:16:15.998 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:15.998 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:15.998 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.998 INFO:tasks.workunit.client.0.smithi100.stdout: "unittest_2": { 2024-04-07T16:16:15.998 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:15.998 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:15.999 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:15.999 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.999 INFO:tasks.workunit.client.0.smithi100.stdout: "osd.2": { 2024-04-07T16:16:15.999 INFO:tasks.workunit.client.0.smithi100.stdout: "bloom_filter": { 2024-04-07T16:16:15.999 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:15.999 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:15.999 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.999 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs": { 2024-04-07T16:16:15.999 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 6368, 2024-04-07T16:16:15.999 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 123 2024-04-07T16:16:15.999 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:15.999 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs_file_reader": { 2024-04-07T16:16:15.999 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 16896, 2024-04-07T16:16:15.999 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 8 2024-04-07T16:16:15.999 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.000 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs_file_writer": { 2024-04-07T16:16:16.000 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 576, 2024-04-07T16:16:16.000 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 3 2024-04-07T16:16:16.000 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.000 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_alloc": { 2024-04-07T16:16:16.000 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 4160, 2024-04-07T16:16:16.000 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 52 2024-04-07T16:16:16.000 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.000 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_blob": { 2024-04-07T16:16:16.000 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 19376, 2024-04-07T16:16:16.000 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 173 2024-04-07T16:16:16.000 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.000 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_buffer": { 2024-04-07T16:16:16.001 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 1040, 2024-04-07T16:16:16.001 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 39 2024-04-07T16:16:16.001 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.001 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_data": { 2024-04-07T16:16:16.001 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 8192, 2024-04-07T16:16:16.001 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 2 2024-04-07T16:16:16.001 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.001 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_meta": { 2024-04-07T16:16:16.001 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 83624, 2024-04-07T16:16:16.001 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 14455 2024-04-07T16:16:16.001 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.001 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_onode": { 2024-04-07T16:16:16.001 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 189976, 2024-04-07T16:16:16.001 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 311 2024-04-07T16:16:16.002 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.002 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_other": { 2024-04-07T16:16:16.002 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 8168, 2024-04-07T16:16:16.002 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 514 2024-04-07T16:16:16.002 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.002 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_extent": { 2024-04-07T16:16:16.002 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 8304, 2024-04-07T16:16:16.002 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 173 2024-04-07T16:16:16.002 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.002 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_fsck": { 2024-04-07T16:16:16.002 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:16.002 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:16.003 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.003 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_inline_bl": { 2024-04-07T16:16:16.003 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 25802, 2024-04-07T16:16:16.003 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 179 2024-04-07T16:16:16.003 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.003 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_shared_blob": { 2024-04-07T16:16:16.003 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 19376, 2024-04-07T16:16:16.003 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 173 2024-04-07T16:16:16.003 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.003 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_txc": { 2024-04-07T16:16:16.003 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 736, 2024-04-07T16:16:16.003 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 1 2024-04-07T16:16:16.003 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.003 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_writing": { 2024-04-07T16:16:16.003 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 350932, 2024-04-07T16:16:16.004 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 77 2024-04-07T16:16:16.004 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.004 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_writing_deferred": { 2024-04-07T16:16:16.004 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 12112, 2024-04-07T16:16:16.004 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 3 2024-04-07T16:16:16.004 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.004 INFO:tasks.workunit.client.0.smithi100.stdout: "buffer_anon": { 2024-04-07T16:16:16.004 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 2422924, 2024-04-07T16:16:16.004 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 191 2024-04-07T16:16:16.004 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.004 INFO:tasks.workunit.client.0.smithi100.stdout: "buffer_meta": { 2024-04-07T16:16:16.004 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 4224, 2024-04-07T16:16:16.004 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 48 2024-04-07T16:16:16.005 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.005 INFO:tasks.workunit.client.0.smithi100.stdout: "mds_co": { 2024-04-07T16:16:16.005 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:16.005 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:16.005 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.005 INFO:tasks.workunit.client.0.smithi100.stdout: "osd": { 2024-04-07T16:16:16.005 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 347760, 2024-04-07T16:16:16.005 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 30 2024-04-07T16:16:16.005 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.005 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mapbl": { 2024-04-07T16:16:16.005 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:16.005 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:16.005 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.006 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_pglog": { 2024-04-07T16:16:16.006 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 8320, 2024-04-07T16:16:16.006 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 24 2024-04-07T16:16:16.006 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.006 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap": { 2024-04-07T16:16:16.006 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 226024, 2024-04-07T16:16:16.006 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 3484 2024-04-07T16:16:16.006 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.006 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap_mapping": { 2024-04-07T16:16:16.006 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:16.006 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:16.006 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.007 INFO:tasks.workunit.client.0.smithi100.stdout: "pgmap": { 2024-04-07T16:16:16.007 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:16.007 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:16.007 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.007 INFO:tasks.workunit.client.0.smithi100.stdout: "unittest_1": { 2024-04-07T16:16:16.007 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:16.007 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:16.007 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.007 INFO:tasks.workunit.client.0.smithi100.stdout: "unittest_2": { 2024-04-07T16:16:16.007 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:16.007 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:16.007 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:16.007 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.007 INFO:tasks.workunit.client.0.smithi100.stdout: "osd.3": { 2024-04-07T16:16:16.007 INFO:tasks.workunit.client.0.smithi100.stdout: "bloom_filter": { 2024-04-07T16:16:16.008 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:16.008 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:16.008 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.008 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs": { 2024-04-07T16:16:16.008 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 6368, 2024-04-07T16:16:16.008 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 123 2024-04-07T16:16:16.008 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.008 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs_file_reader": { 2024-04-07T16:16:16.008 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 16896, 2024-04-07T16:16:16.008 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 8 2024-04-07T16:16:16.008 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.008 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs_file_writer": { 2024-04-07T16:16:16.008 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 576, 2024-04-07T16:16:16.008 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 3 2024-04-07T16:16:16.008 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.009 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_alloc": { 2024-04-07T16:16:16.009 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 3840, 2024-04-07T16:16:16.009 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 48 2024-04-07T16:16:16.009 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.009 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_blob": { 2024-04-07T16:16:16.009 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 19936, 2024-04-07T16:16:16.009 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 178 2024-04-07T16:16:16.009 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.009 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_buffer": { 2024-04-07T16:16:16.009 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 480, 2024-04-07T16:16:16.009 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 34 2024-04-07T16:16:16.009 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.009 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_data": { 2024-04-07T16:16:16.009 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:16.009 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:16.010 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.010 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_meta": { 2024-04-07T16:16:16.010 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 44190, 2024-04-07T16:16:16.010 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 8467 2024-04-07T16:16:16.010 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.010 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_onode": { 2024-04-07T16:16:16.010 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 126736, 2024-04-07T16:16:16.010 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 218 2024-04-07T16:16:16.010 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.010 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_other": { 2024-04-07T16:16:16.011 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 9308, 2024-04-07T16:16:16.011 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 574 2024-04-07T16:16:16.011 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.011 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_extent": { 2024-04-07T16:16:16.011 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 8544, 2024-04-07T16:16:16.011 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 178 2024-04-07T16:16:16.011 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.011 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_fsck": { 2024-04-07T16:16:16.011 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:16.012 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:16.012 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.012 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_inline_bl": { 2024-04-07T16:16:16.012 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 26754, 2024-04-07T16:16:16.012 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 186 2024-04-07T16:16:16.012 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.012 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_shared_blob": { 2024-04-07T16:16:16.012 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 19936, 2024-04-07T16:16:16.012 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 178 2024-04-07T16:16:16.012 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.012 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_txc": { 2024-04-07T16:16:16.012 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 2208, 2024-04-07T16:16:16.012 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 3 2024-04-07T16:16:16.012 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.012 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_writing": { 2024-04-07T16:16:16.013 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 277799, 2024-04-07T16:16:16.013 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 59 2024-04-07T16:16:16.013 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.013 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_writing_deferred": { 2024-04-07T16:16:16.013 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 26565, 2024-04-07T16:16:16.013 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 9 2024-04-07T16:16:16.013 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.013 INFO:tasks.workunit.client.0.smithi100.stdout: "buffer_anon": { 2024-04-07T16:16:16.013 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 2446914, 2024-04-07T16:16:16.013 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 197 2024-04-07T16:16:16.013 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.013 INFO:tasks.workunit.client.0.smithi100.stdout: "buffer_meta": { 2024-04-07T16:16:16.013 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 4664, 2024-04-07T16:16:16.013 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 53 2024-04-07T16:16:16.013 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.014 INFO:tasks.workunit.client.0.smithi100.stdout: "mds_co": { 2024-04-07T16:16:16.014 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:16.014 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:16.014 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.014 INFO:tasks.workunit.client.0.smithi100.stdout: "osd": { 2024-04-07T16:16:16.014 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 417312, 2024-04-07T16:16:16.014 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 36 2024-04-07T16:16:16.014 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.014 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mapbl": { 2024-04-07T16:16:16.014 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:16.014 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:16.014 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.014 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_pglog": { 2024-04-07T16:16:16.014 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 10400, 2024-04-07T16:16:16.014 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 20 2024-04-07T16:16:16.015 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.015 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap": { 2024-04-07T16:16:16.015 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 225512, 2024-04-07T16:16:16.015 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 3460 2024-04-07T16:16:16.015 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.015 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap_mapping": { 2024-04-07T16:16:16.015 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:16.015 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:16.015 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.015 INFO:tasks.workunit.client.0.smithi100.stdout: "pgmap": { 2024-04-07T16:16:16.015 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:16.015 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:16.015 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.015 INFO:tasks.workunit.client.0.smithi100.stdout: "unittest_1": { 2024-04-07T16:16:16.015 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:16.016 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:16.016 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.016 INFO:tasks.workunit.client.0.smithi100.stdout: "unittest_2": { 2024-04-07T16:16:16.016 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:16.016 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:16.016 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:16.016 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.016 INFO:tasks.workunit.client.0.smithi100.stdout: "osd.4": { 2024-04-07T16:16:16.016 INFO:tasks.workunit.client.0.smithi100.stdout: "bloom_filter": { 2024-04-07T16:16:16.016 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:16.016 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:16.016 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.016 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs": { 2024-04-07T16:16:16.016 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 6368, 2024-04-07T16:16:16.016 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 123 2024-04-07T16:16:16.017 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.017 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs_file_reader": { 2024-04-07T16:16:16.017 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 16896, 2024-04-07T16:16:16.017 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 8 2024-04-07T16:16:16.017 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.017 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs_file_writer": { 2024-04-07T16:16:16.017 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 576, 2024-04-07T16:16:16.017 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 3 2024-04-07T16:16:16.017 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.017 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_alloc": { 2024-04-07T16:16:16.017 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 3600, 2024-04-07T16:16:16.017 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 45 2024-04-07T16:16:16.017 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.017 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_blob": { 2024-04-07T16:16:16.017 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 19824, 2024-04-07T16:16:16.018 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 177 2024-04-07T16:16:16.018 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.018 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_buffer": { 2024-04-07T16:16:16.018 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 816, 2024-04-07T16:16:16.018 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 37 2024-04-07T16:16:16.018 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.018 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_data": { 2024-04-07T16:16:16.018 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:16.018 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:16.018 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.018 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_meta": { 2024-04-07T16:16:16.018 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 86386, 2024-04-07T16:16:16.018 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 14938 2024-04-07T16:16:16.018 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.018 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_onode": { 2024-04-07T16:16:16.019 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 198136, 2024-04-07T16:16:16.019 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 323 2024-04-07T16:16:16.019 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.019 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_other": { 2024-04-07T16:16:16.019 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 9704, 2024-04-07T16:16:16.019 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 574 2024-04-07T16:16:16.019 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.019 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_extent": { 2024-04-07T16:16:16.019 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 8496, 2024-04-07T16:16:16.019 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 177 2024-04-07T16:16:16.019 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.019 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_fsck": { 2024-04-07T16:16:16.019 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:16.019 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:16.019 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.020 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_inline_bl": { 2024-04-07T16:16:16.020 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 28202, 2024-04-07T16:16:16.020 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 191 2024-04-07T16:16:16.020 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.020 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_shared_blob": { 2024-04-07T16:16:16.020 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 19824, 2024-04-07T16:16:16.020 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 177 2024-04-07T16:16:16.020 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.020 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_txc": { 2024-04-07T16:16:16.020 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 736, 2024-04-07T16:16:16.020 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 1 2024-04-07T16:16:16.020 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.020 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_writing": { 2024-04-07T16:16:16.020 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 915802, 2024-04-07T16:16:16.020 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 72 2024-04-07T16:16:16.021 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.021 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_writing_deferred": { 2024-04-07T16:16:16.021 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 12112, 2024-04-07T16:16:16.021 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 3 2024-04-07T16:16:16.021 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.021 INFO:tasks.workunit.client.0.smithi100.stdout: "buffer_anon": { 2024-04-07T16:16:16.021 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 2445590, 2024-04-07T16:16:16.021 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 181 2024-04-07T16:16:16.021 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.021 INFO:tasks.workunit.client.0.smithi100.stdout: "buffer_meta": { 2024-04-07T16:16:16.021 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 4928, 2024-04-07T16:16:16.021 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 56 2024-04-07T16:16:16.022 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.022 INFO:tasks.workunit.client.0.smithi100.stdout: "mds_co": { 2024-04-07T16:16:16.022 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:16.022 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:16.022 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.022 INFO:tasks.workunit.client.0.smithi100.stdout: "osd": { 2024-04-07T16:16:16.022 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 463680, 2024-04-07T16:16:16.022 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 40 2024-04-07T16:16:16.022 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.022 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mapbl": { 2024-04-07T16:16:16.022 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:16.022 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:16.022 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.022 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_pglog": { 2024-04-07T16:16:16.023 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 8944, 2024-04-07T16:16:16.023 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 18 2024-04-07T16:16:16.023 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.023 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap": { 2024-04-07T16:16:16.023 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 225768, 2024-04-07T16:16:16.023 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 3472 2024-04-07T16:16:16.023 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.023 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap_mapping": { 2024-04-07T16:16:16.023 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:16.023 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:16.023 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.023 INFO:tasks.workunit.client.0.smithi100.stdout: "pgmap": { 2024-04-07T16:16:16.024 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:16.024 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:16.024 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.024 INFO:tasks.workunit.client.0.smithi100.stdout: "unittest_1": { 2024-04-07T16:16:16.024 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:16.024 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:16.024 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.024 INFO:tasks.workunit.client.0.smithi100.stdout: "unittest_2": { 2024-04-07T16:16:16.024 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:16.024 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:16.024 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:16.024 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.024 INFO:tasks.workunit.client.0.smithi100.stdout: "osd.5": { 2024-04-07T16:16:16.024 INFO:tasks.workunit.client.0.smithi100.stdout: "bloom_filter": { 2024-04-07T16:16:16.024 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:16.025 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:16.025 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.025 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs": { 2024-04-07T16:16:16.025 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 6368, 2024-04-07T16:16:16.025 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 123 2024-04-07T16:16:16.025 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.025 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs_file_reader": { 2024-04-07T16:16:16.025 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 16896, 2024-04-07T16:16:16.025 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 8 2024-04-07T16:16:16.025 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.025 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs_file_writer": { 2024-04-07T16:16:16.025 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 576, 2024-04-07T16:16:16.025 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 3 2024-04-07T16:16:16.025 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.025 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_alloc": { 2024-04-07T16:16:16.026 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 2720, 2024-04-07T16:16:16.026 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 34 2024-04-07T16:16:16.026 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.026 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_blob": { 2024-04-07T16:16:16.026 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 19712, 2024-04-07T16:16:16.026 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 176 2024-04-07T16:16:16.026 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.026 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_buffer": { 2024-04-07T16:16:16.027 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 480, 2024-04-07T16:16:16.027 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 34 2024-04-07T16:16:16.027 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.027 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_data": { 2024-04-07T16:16:16.027 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:16.027 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:16.027 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.027 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_meta": { 2024-04-07T16:16:16.027 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 45425, 2024-04-07T16:16:16.027 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 8650 2024-04-07T16:16:16.027 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.027 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_onode": { 2024-04-07T16:16:16.027 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 130136, 2024-04-07T16:16:16.027 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 223 2024-04-07T16:16:16.027 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.028 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_other": { 2024-04-07T16:16:16.028 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 9940, 2024-04-07T16:16:16.028 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 576 2024-04-07T16:16:16.028 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.028 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_extent": { 2024-04-07T16:16:16.028 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 8448, 2024-04-07T16:16:16.028 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 176 2024-04-07T16:16:16.028 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.028 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_fsck": { 2024-04-07T16:16:16.028 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:16.028 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:16.029 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.029 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_inline_bl": { 2024-04-07T16:16:16.029 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 28370, 2024-04-07T16:16:16.029 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 191 2024-04-07T16:16:16.029 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.029 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_shared_blob": { 2024-04-07T16:16:16.029 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 19712, 2024-04-07T16:16:16.029 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 176 2024-04-07T16:16:16.029 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.029 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_txc": { 2024-04-07T16:16:16.029 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 2208, 2024-04-07T16:16:16.029 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 3 2024-04-07T16:16:16.029 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.029 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_writing": { 2024-04-07T16:16:16.030 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 832819, 2024-04-07T16:16:16.030 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 52 2024-04-07T16:16:16.030 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.030 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_writing_deferred": { 2024-04-07T16:16:16.030 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 26565, 2024-04-07T16:16:16.030 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 9 2024-04-07T16:16:16.030 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.030 INFO:tasks.workunit.client.0.smithi100.stdout: "buffer_anon": { 2024-04-07T16:16:16.030 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 2415758, 2024-04-07T16:16:16.030 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 171 2024-04-07T16:16:16.030 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.030 INFO:tasks.workunit.client.0.smithi100.stdout: "buffer_meta": { 2024-04-07T16:16:16.030 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 5456, 2024-04-07T16:16:16.030 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 62 2024-04-07T16:16:16.030 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.031 INFO:tasks.workunit.client.0.smithi100.stdout: "mds_co": { 2024-04-07T16:16:16.031 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:16.031 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:16.031 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.031 INFO:tasks.workunit.client.0.smithi100.stdout: "osd": { 2024-04-07T16:16:16.031 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 486864, 2024-04-07T16:16:16.031 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 42 2024-04-07T16:16:16.031 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.031 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mapbl": { 2024-04-07T16:16:16.031 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:16.031 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:16.031 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.031 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_pglog": { 2024-04-07T16:16:16.032 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 11960, 2024-04-07T16:16:16.032 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 23 2024-04-07T16:16:16.032 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.032 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap": { 2024-04-07T16:16:16.032 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 226024, 2024-04-07T16:16:16.032 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 3484 2024-04-07T16:16:16.032 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.032 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap_mapping": { 2024-04-07T16:16:16.032 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:16.032 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:16.032 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.032 INFO:tasks.workunit.client.0.smithi100.stdout: "pgmap": { 2024-04-07T16:16:16.032 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:16.032 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:16.032 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.033 INFO:tasks.workunit.client.0.smithi100.stdout: "unittest_1": { 2024-04-07T16:16:16.033 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:16.033 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:16.033 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.033 INFO:tasks.workunit.client.0.smithi100.stdout: "unittest_2": { 2024-04-07T16:16:16.033 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:16.033 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:16.033 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:16.033 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.033 INFO:tasks.workunit.client.0.smithi100.stdout: "osd.6": { 2024-04-07T16:16:16.033 INFO:tasks.workunit.client.0.smithi100.stdout: "bloom_filter": { 2024-04-07T16:16:16.033 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:16.033 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:16.033 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.033 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs": { 2024-04-07T16:16:16.034 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 6368, 2024-04-07T16:16:16.034 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 123 2024-04-07T16:16:16.034 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.034 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs_file_reader": { 2024-04-07T16:16:16.034 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 16896, 2024-04-07T16:16:16.034 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 8 2024-04-07T16:16:16.034 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.034 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs_file_writer": { 2024-04-07T16:16:16.034 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 576, 2024-04-07T16:16:16.034 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 3 2024-04-07T16:16:16.034 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.034 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_alloc": { 2024-04-07T16:16:16.034 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 2400, 2024-04-07T16:16:16.034 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 30 2024-04-07T16:16:16.034 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.035 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_blob": { 2024-04-07T16:16:16.035 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 20832, 2024-04-07T16:16:16.035 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 186 2024-04-07T16:16:16.035 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.035 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_buffer": { 2024-04-07T16:16:16.035 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 256, 2024-04-07T16:16:16.035 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 32 2024-04-07T16:16:16.035 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.035 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_data": { 2024-04-07T16:16:16.035 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:16.035 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:16.035 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.035 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_meta": { 2024-04-07T16:16:16.035 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 45988, 2024-04-07T16:16:16.035 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 8964 2024-04-07T16:16:16.036 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.036 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_onode": { 2024-04-07T16:16:16.036 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 129456, 2024-04-07T16:16:16.036 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 222 2024-04-07T16:16:16.036 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.036 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_other": { 2024-04-07T16:16:16.036 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 10872, 2024-04-07T16:16:16.036 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 734 2024-04-07T16:16:16.036 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.036 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_extent": { 2024-04-07T16:16:16.036 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 8928, 2024-04-07T16:16:16.036 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 186 2024-04-07T16:16:16.036 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.036 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_fsck": { 2024-04-07T16:16:16.037 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:16.037 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:16.037 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.037 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_inline_bl": { 2024-04-07T16:16:16.037 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 28974, 2024-04-07T16:16:16.037 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 190 2024-04-07T16:16:16.037 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.037 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_shared_blob": { 2024-04-07T16:16:16.037 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 20832, 2024-04-07T16:16:16.037 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 186 2024-04-07T16:16:16.037 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.037 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_txc": { 2024-04-07T16:16:16.037 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:16.037 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:16.037 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.038 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_writing": { 2024-04-07T16:16:16.038 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 804548, 2024-04-07T16:16:16.038 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 45 2024-04-07T16:16:16.038 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.038 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_writing_deferred": { 2024-04-07T16:16:16.038 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:16.038 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:16.038 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.038 INFO:tasks.workunit.client.0.smithi100.stdout: "buffer_anon": { 2024-04-07T16:16:16.038 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 2445979, 2024-04-07T16:16:16.038 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 183 2024-04-07T16:16:16.038 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.038 INFO:tasks.workunit.client.0.smithi100.stdout: "buffer_meta": { 2024-04-07T16:16:16.038 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 4664, 2024-04-07T16:16:16.039 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 53 2024-04-07T16:16:16.039 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.039 INFO:tasks.workunit.client.0.smithi100.stdout: "mds_co": { 2024-04-07T16:16:16.039 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:16.039 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:16.039 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.039 INFO:tasks.workunit.client.0.smithi100.stdout: "osd": { 2024-04-07T16:16:16.039 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 452088, 2024-04-07T16:16:16.039 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 39 2024-04-07T16:16:16.039 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.039 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mapbl": { 2024-04-07T16:16:16.039 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:16.039 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:16.039 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.039 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_pglog": { 2024-04-07T16:16:16.040 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 98384, 2024-04-07T16:16:16.040 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 198 2024-04-07T16:16:16.040 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.040 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap": { 2024-04-07T16:16:16.040 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 226280, 2024-04-07T16:16:16.040 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 3496 2024-04-07T16:16:16.040 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.040 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap_mapping": { 2024-04-07T16:16:16.040 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:16.040 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:16.040 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.040 INFO:tasks.workunit.client.0.smithi100.stdout: "pgmap": { 2024-04-07T16:16:16.040 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:16.040 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:16.040 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.040 INFO:tasks.workunit.client.0.smithi100.stdout: "unittest_1": { 2024-04-07T16:16:16.041 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:16.041 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:16.041 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.041 INFO:tasks.workunit.client.0.smithi100.stdout: "unittest_2": { 2024-04-07T16:16:16.041 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:16.041 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:16.041 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:16.041 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.041 INFO:tasks.workunit.client.0.smithi100.stdout: "osd.7": { 2024-04-07T16:16:16.041 INFO:tasks.workunit.client.0.smithi100.stdout: "bloom_filter": { 2024-04-07T16:16:16.041 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:16.041 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:16.041 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.041 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs": { 2024-04-07T16:16:16.042 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 6368, 2024-04-07T16:16:16.042 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 123 2024-04-07T16:16:16.042 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.042 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs_file_reader": { 2024-04-07T16:16:16.042 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 16896, 2024-04-07T16:16:16.042 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 8 2024-04-07T16:16:16.042 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.042 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs_file_writer": { 2024-04-07T16:16:16.042 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 576, 2024-04-07T16:16:16.042 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 3 2024-04-07T16:16:16.042 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.042 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_alloc": { 2024-04-07T16:16:16.042 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 1760, 2024-04-07T16:16:16.042 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 22 2024-04-07T16:16:16.042 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.043 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_blob": { 2024-04-07T16:16:16.043 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 20720, 2024-04-07T16:16:16.043 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 185 2024-04-07T16:16:16.043 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.043 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_buffer": { 2024-04-07T16:16:16.043 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 1264, 2024-04-07T16:16:16.043 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 41 2024-04-07T16:16:16.043 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.043 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_data": { 2024-04-07T16:16:16.043 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 285055, 2024-04-07T16:16:16.043 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 10 2024-04-07T16:16:16.043 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.043 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_meta": { 2024-04-07T16:16:16.043 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 88710, 2024-04-07T16:16:16.043 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 15500 2024-04-07T16:16:16.044 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.044 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_onode": { 2024-04-07T16:16:16.044 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 202216, 2024-04-07T16:16:16.044 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 329 2024-04-07T16:16:16.044 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.044 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_other": { 2024-04-07T16:16:16.044 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 11636, 2024-04-07T16:16:16.044 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 739 2024-04-07T16:16:16.044 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.044 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_extent": { 2024-04-07T16:16:16.044 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 8880, 2024-04-07T16:16:16.044 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 185 2024-04-07T16:16:16.044 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.044 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_fsck": { 2024-04-07T16:16:16.044 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:16.044 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:16.045 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.045 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_inline_bl": { 2024-04-07T16:16:16.045 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 30934, 2024-04-07T16:16:16.045 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 197 2024-04-07T16:16:16.045 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.045 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_shared_blob": { 2024-04-07T16:16:16.045 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 20720, 2024-04-07T16:16:16.045 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 185 2024-04-07T16:16:16.045 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.045 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_txc": { 2024-04-07T16:16:16.046 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:16.046 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:16.046 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.046 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_writing": { 2024-04-07T16:16:16.046 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 145868, 2024-04-07T16:16:16.046 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 29 2024-04-07T16:16:16.046 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.046 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_writing_deferred": { 2024-04-07T16:16:16.046 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:16.046 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:16.046 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.046 INFO:tasks.workunit.client.0.smithi100.stdout: "buffer_anon": { 2024-04-07T16:16:16.046 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 2431589, 2024-04-07T16:16:16.046 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 220 2024-04-07T16:16:16.047 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.047 INFO:tasks.workunit.client.0.smithi100.stdout: "buffer_meta": { 2024-04-07T16:16:16.047 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 5984, 2024-04-07T16:16:16.047 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 68 2024-04-07T16:16:16.047 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.047 INFO:tasks.workunit.client.0.smithi100.stdout: "mds_co": { 2024-04-07T16:16:16.047 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:16.047 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:16.047 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.047 INFO:tasks.workunit.client.0.smithi100.stdout: "osd": { 2024-04-07T16:16:16.047 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 533232, 2024-04-07T16:16:16.047 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 46 2024-04-07T16:16:16.047 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.047 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mapbl": { 2024-04-07T16:16:16.047 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:16.048 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:16.048 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.048 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_pglog": { 2024-04-07T16:16:16.048 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 101400, 2024-04-07T16:16:16.048 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 239 2024-04-07T16:16:16.048 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.048 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap": { 2024-04-07T16:16:16.048 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 225512, 2024-04-07T16:16:16.048 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 3460 2024-04-07T16:16:16.048 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.048 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap_mapping": { 2024-04-07T16:16:16.048 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:16.048 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:16.048 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.048 INFO:tasks.workunit.client.0.smithi100.stdout: "pgmap": { 2024-04-07T16:16:16.049 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:16.049 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:16.049 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.049 INFO:tasks.workunit.client.0.smithi100.stdout: "unittest_1": { 2024-04-07T16:16:16.049 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:16.049 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:16.049 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.049 INFO:tasks.workunit.client.0.smithi100.stdout: "unittest_2": { 2024-04-07T16:16:16.049 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T16:16:16.049 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T16:16:16.049 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:16.049 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:16.049 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:16.049 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.049 INFO:tasks.workunit.client.0.smithi100.stdout: "metadata": { 2024-04-07T16:16:16.050 INFO:tasks.workunit.client.0.smithi100.stdout: "mds": { 2024-04-07T16:16:16.050 INFO:tasks.workunit.client.0.smithi100.stdout: "arch": { 2024-04-07T16:16:16.050 INFO:tasks.workunit.client.0.smithi100.stdout: "x86_64": 2 2024-04-07T16:16:16.050 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.050 INFO:tasks.workunit.client.0.smithi100.stdout: "ceph_version": { 2024-04-07T16:16:16.050 INFO:tasks.workunit.client.0.smithi100.stdout: "ceph version 18.2.2-1171-g267c8ff5 (267c8ff5b1ff5815581b3b2381d19360e081b37b) reef (stable)": 2 2024-04-07T16:16:16.050 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.050 INFO:tasks.workunit.client.0.smithi100.stdout: "cpu": { 2024-04-07T16:16:16.050 INFO:tasks.workunit.client.0.smithi100.stdout: "Intel(R) Xeon(R) CPU E5-1620 v4 @ 3.50GHz": 2 2024-04-07T16:16:16.050 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.050 INFO:tasks.workunit.client.0.smithi100.stdout: "distro": { 2024-04-07T16:16:16.050 INFO:tasks.workunit.client.0.smithi100.stdout: "centos": 2 2024-04-07T16:16:16.050 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.050 INFO:tasks.workunit.client.0.smithi100.stdout: "distro_description": { 2024-04-07T16:16:16.051 INFO:tasks.workunit.client.0.smithi100.stdout: "CentOS Stream 8": 2 2024-04-07T16:16:16.051 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.051 INFO:tasks.workunit.client.0.smithi100.stdout: "kernel_description": { 2024-04-07T16:16:16.051 INFO:tasks.workunit.client.0.smithi100.stdout: "#111-Ubuntu SMP Tue Mar 5 20:16:58 UTC 2024": 2 2024-04-07T16:16:16.051 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.051 INFO:tasks.workunit.client.0.smithi100.stdout: "kernel_version": { 2024-04-07T16:16:16.051 INFO:tasks.workunit.client.0.smithi100.stdout: "5.15.0-101-generic": 2 2024-04-07T16:16:16.051 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.051 INFO:tasks.workunit.client.0.smithi100.stdout: "os": { 2024-04-07T16:16:16.051 INFO:tasks.workunit.client.0.smithi100.stdout: "Linux": 2 2024-04-07T16:16:16.051 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:16.051 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.051 INFO:tasks.workunit.client.0.smithi100.stdout: "mon": { 2024-04-07T16:16:16.052 INFO:tasks.workunit.client.0.smithi100.stdout: "arch": { 2024-04-07T16:16:16.052 INFO:tasks.workunit.client.0.smithi100.stdout: "x86_64": 3 2024-04-07T16:16:16.052 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.052 INFO:tasks.workunit.client.0.smithi100.stdout: "ceph_version": { 2024-04-07T16:16:16.052 INFO:tasks.workunit.client.0.smithi100.stdout: "ceph version 18.2.2-1171-g267c8ff5 (267c8ff5b1ff5815581b3b2381d19360e081b37b) reef (stable)": 3 2024-04-07T16:16:16.052 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.052 INFO:tasks.workunit.client.0.smithi100.stdout: "cpu": { 2024-04-07T16:16:16.052 INFO:tasks.workunit.client.0.smithi100.stdout: "Intel(R) Xeon(R) CPU E5-1620 v4 @ 3.50GHz": 3 2024-04-07T16:16:16.052 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.052 INFO:tasks.workunit.client.0.smithi100.stdout: "distro": { 2024-04-07T16:16:16.052 INFO:tasks.workunit.client.0.smithi100.stdout: "centos": 3 2024-04-07T16:16:16.052 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.053 INFO:tasks.workunit.client.0.smithi100.stdout: "distro_description": { 2024-04-07T16:16:16.053 INFO:tasks.workunit.client.0.smithi100.stdout: "CentOS Stream 8": 3 2024-04-07T16:16:16.053 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.053 INFO:tasks.workunit.client.0.smithi100.stdout: "kernel_description": { 2024-04-07T16:16:16.053 INFO:tasks.workunit.client.0.smithi100.stdout: "#111-Ubuntu SMP Tue Mar 5 20:16:58 UTC 2024": 3 2024-04-07T16:16:16.053 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.053 INFO:tasks.workunit.client.0.smithi100.stdout: "kernel_version": { 2024-04-07T16:16:16.053 INFO:tasks.workunit.client.0.smithi100.stdout: "5.15.0-101-generic": 3 2024-04-07T16:16:16.053 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.053 INFO:tasks.workunit.client.0.smithi100.stdout: "os": { 2024-04-07T16:16:16.053 INFO:tasks.workunit.client.0.smithi100.stdout: "Linux": 3 2024-04-07T16:16:16.053 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:16.053 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.053 INFO:tasks.workunit.client.0.smithi100.stdout: "osd": { 2024-04-07T16:16:16.053 INFO:tasks.workunit.client.0.smithi100.stdout: "arch": { 2024-04-07T16:16:16.054 INFO:tasks.workunit.client.0.smithi100.stdout: "x86_64": 8 2024-04-07T16:16:16.054 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.054 INFO:tasks.workunit.client.0.smithi100.stdout: "ceph_version": { 2024-04-07T16:16:16.054 INFO:tasks.workunit.client.0.smithi100.stdout: "ceph version 18.2.2-1171-g267c8ff5 (267c8ff5b1ff5815581b3b2381d19360e081b37b) reef (stable)": 8 2024-04-07T16:16:16.054 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.054 INFO:tasks.workunit.client.0.smithi100.stdout: "cpu": { 2024-04-07T16:16:16.054 INFO:tasks.workunit.client.0.smithi100.stdout: "Intel(R) Xeon(R) CPU E5-1620 v4 @ 3.50GHz": 8 2024-04-07T16:16:16.054 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.054 INFO:tasks.workunit.client.0.smithi100.stdout: "distro": { 2024-04-07T16:16:16.054 INFO:tasks.workunit.client.0.smithi100.stdout: "centos": 8 2024-04-07T16:16:16.054 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.054 INFO:tasks.workunit.client.0.smithi100.stdout: "distro_description": { 2024-04-07T16:16:16.054 INFO:tasks.workunit.client.0.smithi100.stdout: "CentOS Stream 8": 8 2024-04-07T16:16:16.055 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.055 INFO:tasks.workunit.client.0.smithi100.stdout: "kernel_description": { 2024-04-07T16:16:16.055 INFO:tasks.workunit.client.0.smithi100.stdout: "#111-Ubuntu SMP Tue Mar 5 20:16:58 UTC 2024": 8 2024-04-07T16:16:16.055 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.055 INFO:tasks.workunit.client.0.smithi100.stdout: "kernel_version": { 2024-04-07T16:16:16.055 INFO:tasks.workunit.client.0.smithi100.stdout: "5.15.0-101-generic": 8 2024-04-07T16:16:16.055 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.055 INFO:tasks.workunit.client.0.smithi100.stdout: "os": { 2024-04-07T16:16:16.055 INFO:tasks.workunit.client.0.smithi100.stdout: "Linux": 8 2024-04-07T16:16:16.055 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.055 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_objectstore": { 2024-04-07T16:16:16.055 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore": 8 2024-04-07T16:16:16.055 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.055 INFO:tasks.workunit.client.0.smithi100.stdout: "rotational": { 2024-04-07T16:16:16.055 INFO:tasks.workunit.client.0.smithi100.stdout: "0": 8 2024-04-07T16:16:16.056 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:16.056 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:16.056 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.056 INFO:tasks.workunit.client.0.smithi100.stdout: "mon": { 2024-04-07T16:16:16.056 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 3, 2024-04-07T16:16:16.056 INFO:tasks.workunit.client.0.smithi100.stdout: "features": { 2024-04-07T16:16:16.056 INFO:tasks.workunit.client.0.smithi100.stdout: "optional": [], 2024-04-07T16:16:16.056 INFO:tasks.workunit.client.0.smithi100.stdout: "persistent": [ 2024-04-07T16:16:16.056 INFO:tasks.workunit.client.0.smithi100.stdout: "kraken", 2024-04-07T16:16:16.056 INFO:tasks.workunit.client.0.smithi100.stdout: "luminous", 2024-04-07T16:16:16.056 INFO:tasks.workunit.client.0.smithi100.stdout: "mimic", 2024-04-07T16:16:16.056 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap-prune", 2024-04-07T16:16:16.056 INFO:tasks.workunit.client.0.smithi100.stdout: "nautilus", 2024-04-07T16:16:16.056 INFO:tasks.workunit.client.0.smithi100.stdout: "octopus", 2024-04-07T16:16:16.056 INFO:tasks.workunit.client.0.smithi100.stdout: "pacific", 2024-04-07T16:16:16.056 INFO:tasks.workunit.client.0.smithi100.stdout: "elector-pinging", 2024-04-07T16:16:16.057 INFO:tasks.workunit.client.0.smithi100.stdout: "quincy", 2024-04-07T16:16:16.057 INFO:tasks.workunit.client.0.smithi100.stdout: "reef" 2024-04-07T16:16:16.057 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T16:16:16.057 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.057 INFO:tasks.workunit.client.0.smithi100.stdout: "ipv4_addr_mons": 6, 2024-04-07T16:16:16.057 INFO:tasks.workunit.client.0.smithi100.stdout: "ipv6_addr_mons": 0, 2024-04-07T16:16:16.057 INFO:tasks.workunit.client.0.smithi100.stdout: "min_mon_release": 18, 2024-04-07T16:16:16.057 INFO:tasks.workunit.client.0.smithi100.stdout: "v1_addr_mons": 3, 2024-04-07T16:16:16.057 INFO:tasks.workunit.client.0.smithi100.stdout: "v2_addr_mons": 3 2024-04-07T16:16:16.057 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.057 INFO:tasks.workunit.client.0.smithi100.stdout: "organization": "ceph-qa", 2024-04-07T16:16:16.057 INFO:tasks.workunit.client.0.smithi100.stdout: "osd": { 2024-04-07T16:16:16.057 INFO:tasks.workunit.client.0.smithi100.stdout: "cluster_network": false, 2024-04-07T16:16:16.057 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 8, 2024-04-07T16:16:16.058 INFO:tasks.workunit.client.0.smithi100.stdout: "require_min_compat_client": "luminous", 2024-04-07T16:16:16.058 INFO:tasks.workunit.client.0.smithi100.stdout: "require_osd_release": "reef" 2024-04-07T16:16:16.058 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.058 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_perf_histograms": [ 2024-04-07T16:16:16.058 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:16.058 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_latency_out_bytes_histogram": { 2024-04-07T16:16:16.058 INFO:tasks.workunit.client.0.smithi100.stdout: "axes": [ 2024-04-07T16:16:16.058 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:16.058 INFO:tasks.workunit.client.0.smithi100.stdout: "buckets": 32, 2024-04-07T16:16:16.059 INFO:tasks.workunit.client.0.smithi100.stdout: "min": 0, 2024-04-07T16:16:16.059 INFO:tasks.workunit.client.0.smithi100.stdout: "name": "Latency (usec)", 2024-04-07T16:16:16.059 INFO:tasks.workunit.client.0.smithi100.stdout: "quant_size": 100000, 2024-04-07T16:16:16.059 INFO:tasks.workunit.client.0.smithi100.stdout: "ranges": [ 2024-04-07T16:16:16.059 INFO:tasks.workunit.client.0.smithi100.stdout: "[None, -1]", 2024-04-07T16:16:16.059 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 99999]", 2024-04-07T16:16:16.059 INFO:tasks.workunit.client.0.smithi100.stdout: "[100000, 199999]", 2024-04-07T16:16:16.060 INFO:tasks.workunit.client.0.smithi100.stdout: "[200000, 399999]", 2024-04-07T16:16:16.060 INFO:tasks.workunit.client.0.smithi100.stdout: "[400000, 799999]", 2024-04-07T16:16:16.060 INFO:tasks.workunit.client.0.smithi100.stdout: "[800000, 1599999]", 2024-04-07T16:16:16.060 INFO:tasks.workunit.client.0.smithi100.stdout: "[1600000, 3199999]", 2024-04-07T16:16:16.060 INFO:tasks.workunit.client.0.smithi100.stdout: "[3200000, 6399999]", 2024-04-07T16:16:16.060 INFO:tasks.workunit.client.0.smithi100.stdout: "[6400000, 12799999]", 2024-04-07T16:16:16.060 INFO:tasks.workunit.client.0.smithi100.stdout: "[12800000, 25599999]", 2024-04-07T16:16:16.061 INFO:tasks.workunit.client.0.smithi100.stdout: "[25600000, 51199999]", 2024-04-07T16:16:16.061 INFO:tasks.workunit.client.0.smithi100.stdout: "[51200000, 102399999]", 2024-04-07T16:16:16.061 INFO:tasks.workunit.client.0.smithi100.stdout: "[102400000, 204799999]", 2024-04-07T16:16:16.061 INFO:tasks.workunit.client.0.smithi100.stdout: "[204800000, 409599999]", 2024-04-07T16:16:16.061 INFO:tasks.workunit.client.0.smithi100.stdout: "[409600000, 819199999]", 2024-04-07T16:16:16.061 INFO:tasks.workunit.client.0.smithi100.stdout: "[819200000, 1638399999]", 2024-04-07T16:16:16.061 INFO:tasks.workunit.client.0.smithi100.stdout: "[1638400000, 3276799999]", 2024-04-07T16:16:16.062 INFO:tasks.workunit.client.0.smithi100.stdout: "[3276800000, 6553599999]", 2024-04-07T16:16:16.062 INFO:tasks.workunit.client.0.smithi100.stdout: "[6553600000, 13107199999]", 2024-04-07T16:16:16.062 INFO:tasks.workunit.client.0.smithi100.stdout: "[13107200000, 26214399999]", 2024-04-07T16:16:16.062 INFO:tasks.workunit.client.0.smithi100.stdout: "[26214400000, 52428799999]", 2024-04-07T16:16:16.062 INFO:tasks.workunit.client.0.smithi100.stdout: "[52428800000, 104857599999]", 2024-04-07T16:16:16.062 INFO:tasks.workunit.client.0.smithi100.stdout: "[104857600000, 209715199999]", 2024-04-07T16:16:16.062 INFO:tasks.workunit.client.0.smithi100.stdout: "[209715200000, 419430399999]", 2024-04-07T16:16:16.062 INFO:tasks.workunit.client.0.smithi100.stdout: "[419430400000, 838860799999]", 2024-04-07T16:16:16.062 INFO:tasks.workunit.client.0.smithi100.stdout: "[838860800000, 1677721599999]", 2024-04-07T16:16:16.062 INFO:tasks.workunit.client.0.smithi100.stdout: "[1677721600000, 3355443199999]", 2024-04-07T16:16:16.062 INFO:tasks.workunit.client.0.smithi100.stdout: "[3355443200000, 6710886399999]", 2024-04-07T16:16:16.063 INFO:tasks.workunit.client.0.smithi100.stdout: "[6710886400000, 13421772799999]", 2024-04-07T16:16:16.063 INFO:tasks.workunit.client.0.smithi100.stdout: "[13421772800000, 26843545599999]", 2024-04-07T16:16:16.063 INFO:tasks.workunit.client.0.smithi100.stdout: "[26843545600000, 53687091199999]", 2024-04-07T16:16:16.063 INFO:tasks.workunit.client.0.smithi100.stdout: "[53687091200000, None]" 2024-04-07T16:16:16.063 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:16.063 INFO:tasks.workunit.client.0.smithi100.stdout: "scale_type": "log2" 2024-04-07T16:16:16.063 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.064 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:16.064 INFO:tasks.workunit.client.0.smithi100.stdout: "buckets": 32, 2024-04-07T16:16:16.064 INFO:tasks.workunit.client.0.smithi100.stdout: "min": 0, 2024-04-07T16:16:16.064 INFO:tasks.workunit.client.0.smithi100.stdout: "name": "Request size (bytes)", 2024-04-07T16:16:16.064 INFO:tasks.workunit.client.0.smithi100.stdout: "quant_size": 512, 2024-04-07T16:16:16.064 INFO:tasks.workunit.client.0.smithi100.stdout: "ranges": [ 2024-04-07T16:16:16.064 INFO:tasks.workunit.client.0.smithi100.stdout: "[None, -1]", 2024-04-07T16:16:16.064 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 511]", 2024-04-07T16:16:16.065 INFO:tasks.workunit.client.0.smithi100.stdout: "[512, 1023]", 2024-04-07T16:16:16.065 INFO:tasks.workunit.client.0.smithi100.stdout: "[1024, 2047]", 2024-04-07T16:16:16.065 INFO:tasks.workunit.client.0.smithi100.stdout: "[2048, 4095]", 2024-04-07T16:16:16.065 INFO:tasks.workunit.client.0.smithi100.stdout: "[4096, 8191]", 2024-04-07T16:16:16.065 INFO:tasks.workunit.client.0.smithi100.stdout: "[8192, 16383]", 2024-04-07T16:16:16.065 INFO:tasks.workunit.client.0.smithi100.stdout: "[16384, 32767]", 2024-04-07T16:16:16.065 INFO:tasks.workunit.client.0.smithi100.stdout: "[32768, 65535]", 2024-04-07T16:16:16.065 INFO:tasks.workunit.client.0.smithi100.stdout: "[65536, 131071]", 2024-04-07T16:16:16.066 INFO:tasks.workunit.client.0.smithi100.stdout: "[131072, 262143]", 2024-04-07T16:16:16.066 INFO:tasks.workunit.client.0.smithi100.stdout: "[262144, 524287]", 2024-04-07T16:16:16.066 INFO:tasks.workunit.client.0.smithi100.stdout: "[524288, 1048575]", 2024-04-07T16:16:16.066 INFO:tasks.workunit.client.0.smithi100.stdout: "[1048576, 2097151]", 2024-04-07T16:16:16.066 INFO:tasks.workunit.client.0.smithi100.stdout: "[2097152, 4194303]", 2024-04-07T16:16:16.066 INFO:tasks.workunit.client.0.smithi100.stdout: "[4194304, 8388607]", 2024-04-07T16:16:16.066 INFO:tasks.workunit.client.0.smithi100.stdout: "[8388608, 16777215]", 2024-04-07T16:16:16.066 INFO:tasks.workunit.client.0.smithi100.stdout: "[16777216, 33554431]", 2024-04-07T16:16:16.067 INFO:tasks.workunit.client.0.smithi100.stdout: "[33554432, 67108863]", 2024-04-07T16:16:16.067 INFO:tasks.workunit.client.0.smithi100.stdout: "[67108864, 134217727]", 2024-04-07T16:16:16.067 INFO:tasks.workunit.client.0.smithi100.stdout: "[134217728, 268435455]", 2024-04-07T16:16:16.067 INFO:tasks.workunit.client.0.smithi100.stdout: "[268435456, 536870911]", 2024-04-07T16:16:16.067 INFO:tasks.workunit.client.0.smithi100.stdout: "[536870912, 1073741823]", 2024-04-07T16:16:16.067 INFO:tasks.workunit.client.0.smithi100.stdout: "[1073741824, 2147483647]", 2024-04-07T16:16:16.067 INFO:tasks.workunit.client.0.smithi100.stdout: "[2147483648, 4294967295]", 2024-04-07T16:16:16.067 INFO:tasks.workunit.client.0.smithi100.stdout: "[4294967296, 8589934591]", 2024-04-07T16:16:16.067 INFO:tasks.workunit.client.0.smithi100.stdout: "[8589934592, 17179869183]", 2024-04-07T16:16:16.067 INFO:tasks.workunit.client.0.smithi100.stdout: "[17179869184, 34359738367]", 2024-04-07T16:16:16.068 INFO:tasks.workunit.client.0.smithi100.stdout: "[34359738368, 68719476735]", 2024-04-07T16:16:16.068 INFO:tasks.workunit.client.0.smithi100.stdout: "[68719476736, 137438953471]", 2024-04-07T16:16:16.068 INFO:tasks.workunit.client.0.smithi100.stdout: "[137438953472, 274877906943]", 2024-04-07T16:16:16.068 INFO:tasks.workunit.client.0.smithi100.stdout: "[274877906944, None]" 2024-04-07T16:16:16.068 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:16.068 INFO:tasks.workunit.client.0.smithi100.stdout: "scale_type": "log2" 2024-04-07T16:16:16.068 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:16.068 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:16.068 INFO:tasks.workunit.client.0.smithi100.stdout: "osds": [ 2024-04-07T16:16:16.068 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:16.068 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 0, 2024-04-07T16:16:16.068 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T16:16:16.068 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.068 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.068 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.068 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.069 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.069 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.069 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.069 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.069 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.069 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.069 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.069 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.069 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.069 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.069 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.069 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.069 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.069 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.069 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.069 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.070 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.070 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.070 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.070 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.070 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.070 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.070 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.070 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.070 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.070 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.070 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.070 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T16:16:16.070 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T16:16:16.070 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.071 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:16.071 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 1, 2024-04-07T16:16:16.071 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T16:16:16.071 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.071 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.071 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 9, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.071 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 17, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.071 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 6, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.071 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.071 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.071 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.071 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.071 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.071 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.071 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.072 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.072 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.072 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.072 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.072 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.072 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.072 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.072 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.072 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.072 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.072 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.072 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.072 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.072 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.072 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.073 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.073 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.073 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.073 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.073 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T16:16:16.073 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T16:16:16.073 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.073 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:16.073 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 2, 2024-04-07T16:16:16.073 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T16:16:16.073 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.073 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.073 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.073 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.073 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.074 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.074 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.074 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.074 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.074 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.074 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.074 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.074 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.074 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.074 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.074 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.075 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.075 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.075 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.075 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.075 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.075 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.075 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.075 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.075 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.075 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.075 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.075 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.075 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.075 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.075 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.076 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T16:16:16.076 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T16:16:16.076 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.076 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:16.076 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 3, 2024-04-07T16:16:16.076 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T16:16:16.076 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.076 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.076 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.076 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.076 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.076 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.076 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.076 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.077 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.077 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.077 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.077 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.077 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.077 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.077 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.077 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.077 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.077 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.077 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.077 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.077 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.077 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.078 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.078 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.078 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.078 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.078 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.078 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.078 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.078 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.078 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.078 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T16:16:16.078 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T16:16:16.078 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.079 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:16.079 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 4, 2024-04-07T16:16:16.079 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T16:16:16.079 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.079 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.079 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.079 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.079 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.079 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.079 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.079 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.079 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.079 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.079 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.079 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.080 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.080 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.080 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.080 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.080 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.080 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.080 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.080 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.080 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.080 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.080 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.080 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.080 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.080 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.081 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.081 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.081 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.081 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.081 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.081 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T16:16:16.081 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T16:16:16.081 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.081 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:16.081 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 5, 2024-04-07T16:16:16.081 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T16:16:16.081 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.081 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.082 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.082 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.082 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.082 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.082 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.082 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.082 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.082 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.082 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.082 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.082 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.082 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.082 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.082 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.083 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.083 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.083 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.083 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.083 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.083 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.083 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.083 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.083 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.083 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.083 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.083 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.083 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.084 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.084 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.084 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T16:16:16.084 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T16:16:16.084 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.084 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:16.084 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 6, 2024-04-07T16:16:16.084 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T16:16:16.084 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.084 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.084 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.084 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.084 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.084 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.085 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.085 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.085 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.085 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.085 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.085 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.085 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.085 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.085 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.085 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.085 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.085 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.085 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.085 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.085 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.086 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.086 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.086 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.086 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.086 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.086 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.086 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.086 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.086 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.086 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.086 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T16:16:16.086 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T16:16:16.087 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.087 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:16.087 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 7, 2024-04-07T16:16:16.087 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T16:16:16.087 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.087 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.087 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.087 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 62, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.087 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 62, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.087 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.087 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.087 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.087 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.087 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.087 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.088 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.088 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.088 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.088 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.088 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.088 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.088 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.088 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.088 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.088 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.088 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.088 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.088 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.088 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.089 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.089 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.089 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.089 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.089 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.089 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.089 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T16:16:16.089 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T16:16:16.089 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:16.089 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T16:16:16.089 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.089 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_latency_in_bytes_histogram": { 2024-04-07T16:16:16.090 INFO:tasks.workunit.client.0.smithi100.stdout: "axes": [ 2024-04-07T16:16:16.090 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:16.090 INFO:tasks.workunit.client.0.smithi100.stdout: "buckets": 32, 2024-04-07T16:16:16.090 INFO:tasks.workunit.client.0.smithi100.stdout: "min": 0, 2024-04-07T16:16:16.090 INFO:tasks.workunit.client.0.smithi100.stdout: "name": "Latency (usec)", 2024-04-07T16:16:16.090 INFO:tasks.workunit.client.0.smithi100.stdout: "quant_size": 100000, 2024-04-07T16:16:16.090 INFO:tasks.workunit.client.0.smithi100.stdout: "ranges": [ 2024-04-07T16:16:16.090 INFO:tasks.workunit.client.0.smithi100.stdout: "[None, -1]", 2024-04-07T16:16:16.090 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 99999]", 2024-04-07T16:16:16.090 INFO:tasks.workunit.client.0.smithi100.stdout: "[100000, 199999]", 2024-04-07T16:16:16.090 INFO:tasks.workunit.client.0.smithi100.stdout: "[200000, 399999]", 2024-04-07T16:16:16.090 INFO:tasks.workunit.client.0.smithi100.stdout: "[400000, 799999]", 2024-04-07T16:16:16.090 INFO:tasks.workunit.client.0.smithi100.stdout: "[800000, 1599999]", 2024-04-07T16:16:16.091 INFO:tasks.workunit.client.0.smithi100.stdout: "[1600000, 3199999]", 2024-04-07T16:16:16.091 INFO:tasks.workunit.client.0.smithi100.stdout: "[3200000, 6399999]", 2024-04-07T16:16:16.091 INFO:tasks.workunit.client.0.smithi100.stdout: "[6400000, 12799999]", 2024-04-07T16:16:16.091 INFO:tasks.workunit.client.0.smithi100.stdout: "[12800000, 25599999]", 2024-04-07T16:16:16.091 INFO:tasks.workunit.client.0.smithi100.stdout: "[25600000, 51199999]", 2024-04-07T16:16:16.091 INFO:tasks.workunit.client.0.smithi100.stdout: "[51200000, 102399999]", 2024-04-07T16:16:16.091 INFO:tasks.workunit.client.0.smithi100.stdout: "[102400000, 204799999]", 2024-04-07T16:16:16.091 INFO:tasks.workunit.client.0.smithi100.stdout: "[204800000, 409599999]", 2024-04-07T16:16:16.091 INFO:tasks.workunit.client.0.smithi100.stdout: "[409600000, 819199999]", 2024-04-07T16:16:16.091 INFO:tasks.workunit.client.0.smithi100.stdout: "[819200000, 1638399999]", 2024-04-07T16:16:16.091 INFO:tasks.workunit.client.0.smithi100.stdout: "[1638400000, 3276799999]", 2024-04-07T16:16:16.091 INFO:tasks.workunit.client.0.smithi100.stdout: "[3276800000, 6553599999]", 2024-04-07T16:16:16.091 INFO:tasks.workunit.client.0.smithi100.stdout: "[6553600000, 13107199999]", 2024-04-07T16:16:16.091 INFO:tasks.workunit.client.0.smithi100.stdout: "[13107200000, 26214399999]", 2024-04-07T16:16:16.091 INFO:tasks.workunit.client.0.smithi100.stdout: "[26214400000, 52428799999]", 2024-04-07T16:16:16.092 INFO:tasks.workunit.client.0.smithi100.stdout: "[52428800000, 104857599999]", 2024-04-07T16:16:16.092 INFO:tasks.workunit.client.0.smithi100.stdout: "[104857600000, 209715199999]", 2024-04-07T16:16:16.092 INFO:tasks.workunit.client.0.smithi100.stdout: "[209715200000, 419430399999]", 2024-04-07T16:16:16.092 INFO:tasks.workunit.client.0.smithi100.stdout: "[419430400000, 838860799999]", 2024-04-07T16:16:16.092 INFO:tasks.workunit.client.0.smithi100.stdout: "[838860800000, 1677721599999]", 2024-04-07T16:16:16.092 INFO:tasks.workunit.client.0.smithi100.stdout: "[1677721600000, 3355443199999]", 2024-04-07T16:16:16.092 INFO:tasks.workunit.client.0.smithi100.stdout: "[3355443200000, 6710886399999]", 2024-04-07T16:16:16.092 INFO:tasks.workunit.client.0.smithi100.stdout: "[6710886400000, 13421772799999]", 2024-04-07T16:16:16.092 INFO:tasks.workunit.client.0.smithi100.stdout: "[13421772800000, 26843545599999]", 2024-04-07T16:16:16.092 INFO:tasks.workunit.client.0.smithi100.stdout: "[26843545600000, 53687091199999]", 2024-04-07T16:16:16.092 INFO:tasks.workunit.client.0.smithi100.stdout: "[53687091200000, None]" 2024-04-07T16:16:16.092 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:16.092 INFO:tasks.workunit.client.0.smithi100.stdout: "scale_type": "log2" 2024-04-07T16:16:16.092 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.092 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:16.093 INFO:tasks.workunit.client.0.smithi100.stdout: "buckets": 32, 2024-04-07T16:16:16.093 INFO:tasks.workunit.client.0.smithi100.stdout: "min": 0, 2024-04-07T16:16:16.093 INFO:tasks.workunit.client.0.smithi100.stdout: "name": "Request size (bytes)", 2024-04-07T16:16:16.093 INFO:tasks.workunit.client.0.smithi100.stdout: "quant_size": 512, 2024-04-07T16:16:16.093 INFO:tasks.workunit.client.0.smithi100.stdout: "ranges": [ 2024-04-07T16:16:16.093 INFO:tasks.workunit.client.0.smithi100.stdout: "[None, -1]", 2024-04-07T16:16:16.093 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 511]", 2024-04-07T16:16:16.093 INFO:tasks.workunit.client.0.smithi100.stdout: "[512, 1023]", 2024-04-07T16:16:16.093 INFO:tasks.workunit.client.0.smithi100.stdout: "[1024, 2047]", 2024-04-07T16:16:16.093 INFO:tasks.workunit.client.0.smithi100.stdout: "[2048, 4095]", 2024-04-07T16:16:16.093 INFO:tasks.workunit.client.0.smithi100.stdout: "[4096, 8191]", 2024-04-07T16:16:16.093 INFO:tasks.workunit.client.0.smithi100.stdout: "[8192, 16383]", 2024-04-07T16:16:16.093 INFO:tasks.workunit.client.0.smithi100.stdout: "[16384, 32767]", 2024-04-07T16:16:16.093 INFO:tasks.workunit.client.0.smithi100.stdout: "[32768, 65535]", 2024-04-07T16:16:16.093 INFO:tasks.workunit.client.0.smithi100.stdout: "[65536, 131071]", 2024-04-07T16:16:16.093 INFO:tasks.workunit.client.0.smithi100.stdout: "[131072, 262143]", 2024-04-07T16:16:16.094 INFO:tasks.workunit.client.0.smithi100.stdout: "[262144, 524287]", 2024-04-07T16:16:16.094 INFO:tasks.workunit.client.0.smithi100.stdout: "[524288, 1048575]", 2024-04-07T16:16:16.094 INFO:tasks.workunit.client.0.smithi100.stdout: "[1048576, 2097151]", 2024-04-07T16:16:16.094 INFO:tasks.workunit.client.0.smithi100.stdout: "[2097152, 4194303]", 2024-04-07T16:16:16.094 INFO:tasks.workunit.client.0.smithi100.stdout: "[4194304, 8388607]", 2024-04-07T16:16:16.094 INFO:tasks.workunit.client.0.smithi100.stdout: "[8388608, 16777215]", 2024-04-07T16:16:16.094 INFO:tasks.workunit.client.0.smithi100.stdout: "[16777216, 33554431]", 2024-04-07T16:16:16.094 INFO:tasks.workunit.client.0.smithi100.stdout: "[33554432, 67108863]", 2024-04-07T16:16:16.094 INFO:tasks.workunit.client.0.smithi100.stdout: "[67108864, 134217727]", 2024-04-07T16:16:16.094 INFO:tasks.workunit.client.0.smithi100.stdout: "[134217728, 268435455]", 2024-04-07T16:16:16.094 INFO:tasks.workunit.client.0.smithi100.stdout: "[268435456, 536870911]", 2024-04-07T16:16:16.094 INFO:tasks.workunit.client.0.smithi100.stdout: "[536870912, 1073741823]", 2024-04-07T16:16:16.095 INFO:tasks.workunit.client.0.smithi100.stdout: "[1073741824, 2147483647]", 2024-04-07T16:16:16.095 INFO:tasks.workunit.client.0.smithi100.stdout: "[2147483648, 4294967295]", 2024-04-07T16:16:16.095 INFO:tasks.workunit.client.0.smithi100.stdout: "[4294967296, 8589934591]", 2024-04-07T16:16:16.095 INFO:tasks.workunit.client.0.smithi100.stdout: "[8589934592, 17179869183]", 2024-04-07T16:16:16.095 INFO:tasks.workunit.client.0.smithi100.stdout: "[17179869184, 34359738367]", 2024-04-07T16:16:16.095 INFO:tasks.workunit.client.0.smithi100.stdout: "[34359738368, 68719476735]", 2024-04-07T16:16:16.095 INFO:tasks.workunit.client.0.smithi100.stdout: "[68719476736, 137438953471]", 2024-04-07T16:16:16.095 INFO:tasks.workunit.client.0.smithi100.stdout: "[137438953472, 274877906943]", 2024-04-07T16:16:16.095 INFO:tasks.workunit.client.0.smithi100.stdout: "[274877906944, None]" 2024-04-07T16:16:16.095 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:16.095 INFO:tasks.workunit.client.0.smithi100.stdout: "scale_type": "log2" 2024-04-07T16:16:16.095 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:16.095 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:16.096 INFO:tasks.workunit.client.0.smithi100.stdout: "osds": [ 2024-04-07T16:16:16.096 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:16.096 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 0, 2024-04-07T16:16:16.096 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T16:16:16.096 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.096 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.096 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.096 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.096 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.096 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.096 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.096 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.096 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.096 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.096 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.097 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.097 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.097 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.097 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.097 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.097 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.097 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.097 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.097 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.097 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.097 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.097 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.097 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.097 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.097 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.098 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.098 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.098 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.098 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.098 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.098 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T16:16:16.098 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T16:16:16.098 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.098 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:16.098 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 1, 2024-04-07T16:16:16.098 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T16:16:16.098 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.098 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.099 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.099 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.099 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.099 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 11, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.099 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 10, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.099 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.099 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.099 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.099 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.099 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.099 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.099 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.099 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.099 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.099 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.099 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.100 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.100 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.100 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.100 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.100 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.100 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.100 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.100 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.100 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.100 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.100 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.100 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.100 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.100 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T16:16:16.101 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T16:16:16.101 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.101 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:16.101 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 2, 2024-04-07T16:16:16.101 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T16:16:16.101 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.101 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.101 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.101 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.101 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.101 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.101 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.101 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.101 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.101 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.102 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.102 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.102 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.102 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.102 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.102 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.102 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.102 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.102 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.102 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.102 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.102 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.102 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.103 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.103 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.103 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.103 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.103 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.103 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.103 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.103 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.103 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T16:16:16.103 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T16:16:16.103 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.103 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:16.103 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 3, 2024-04-07T16:16:16.103 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T16:16:16.104 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.104 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.104 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.104 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.104 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.104 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.104 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.104 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.104 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.104 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.104 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.104 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.104 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.105 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.105 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.105 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.105 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.105 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.105 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.106 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.106 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.106 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.106 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.106 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.106 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.107 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.107 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.107 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.107 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.107 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.107 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.108 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T16:16:16.108 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T16:16:16.108 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.108 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:16.108 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 4, 2024-04-07T16:16:16.108 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T16:16:16.109 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.109 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.109 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.109 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.109 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.109 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.109 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.110 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.110 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.110 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.110 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.110 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.110 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.111 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.111 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.111 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.111 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.111 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.111 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.111 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.112 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.112 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.112 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.112 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.112 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.112 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.113 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.113 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.113 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.113 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.113 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.113 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T16:16:16.114 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T16:16:16.114 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.114 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:16.114 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 5, 2024-04-07T16:16:16.114 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T16:16:16.114 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.114 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.115 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.115 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.115 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.115 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.115 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.115 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.116 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.116 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.116 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.116 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.117 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.117 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.117 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.117 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.117 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.117 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.118 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.118 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.118 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.118 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.118 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.118 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.118 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.118 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.118 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.118 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.119 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.119 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.119 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.119 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T16:16:16.119 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T16:16:16.119 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.119 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:16.119 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 6, 2024-04-07T16:16:16.119 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T16:16:16.119 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.119 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.119 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.119 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.119 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.119 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.120 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.120 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.120 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.120 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.120 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.120 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.120 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.120 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.120 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.120 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.120 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.120 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.120 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.120 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.121 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.121 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.121 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.121 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.121 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.121 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.121 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.121 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.121 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.121 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.121 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.121 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T16:16:16.121 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T16:16:16.121 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.122 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:16.122 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 7, 2024-04-07T16:16:16.122 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T16:16:16.122 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.122 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.122 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.122 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.122 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.122 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.122 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 47, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.122 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.122 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.122 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.123 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.123 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.123 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.123 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.123 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.123 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.123 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.123 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.123 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.123 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.123 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.123 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.123 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.123 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.123 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.124 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.124 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.124 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.124 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.124 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.125 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.125 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T16:16:16.125 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T16:16:16.125 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:16.126 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T16:16:16.126 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.126 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_latency_out_bytes_histogram": { 2024-04-07T16:16:16.126 INFO:tasks.workunit.client.0.smithi100.stdout: "axes": [ 2024-04-07T16:16:16.126 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:16.126 INFO:tasks.workunit.client.0.smithi100.stdout: "buckets": 32, 2024-04-07T16:16:16.126 INFO:tasks.workunit.client.0.smithi100.stdout: "min": 0, 2024-04-07T16:16:16.126 INFO:tasks.workunit.client.0.smithi100.stdout: "name": "Latency (usec)", 2024-04-07T16:16:16.126 INFO:tasks.workunit.client.0.smithi100.stdout: "quant_size": 100000, 2024-04-07T16:16:16.126 INFO:tasks.workunit.client.0.smithi100.stdout: "ranges": [ 2024-04-07T16:16:16.126 INFO:tasks.workunit.client.0.smithi100.stdout: "[None, -1]", 2024-04-07T16:16:16.126 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 99999]", 2024-04-07T16:16:16.126 INFO:tasks.workunit.client.0.smithi100.stdout: "[100000, 199999]", 2024-04-07T16:16:16.126 INFO:tasks.workunit.client.0.smithi100.stdout: "[200000, 399999]", 2024-04-07T16:16:16.127 INFO:tasks.workunit.client.0.smithi100.stdout: "[400000, 799999]", 2024-04-07T16:16:16.127 INFO:tasks.workunit.client.0.smithi100.stdout: "[800000, 1599999]", 2024-04-07T16:16:16.127 INFO:tasks.workunit.client.0.smithi100.stdout: "[1600000, 3199999]", 2024-04-07T16:16:16.127 INFO:tasks.workunit.client.0.smithi100.stdout: "[3200000, 6399999]", 2024-04-07T16:16:16.127 INFO:tasks.workunit.client.0.smithi100.stdout: "[6400000, 12799999]", 2024-04-07T16:16:16.127 INFO:tasks.workunit.client.0.smithi100.stdout: "[12800000, 25599999]", 2024-04-07T16:16:16.127 INFO:tasks.workunit.client.0.smithi100.stdout: "[25600000, 51199999]", 2024-04-07T16:16:16.127 INFO:tasks.workunit.client.0.smithi100.stdout: "[51200000, 102399999]", 2024-04-07T16:16:16.127 INFO:tasks.workunit.client.0.smithi100.stdout: "[102400000, 204799999]", 2024-04-07T16:16:16.127 INFO:tasks.workunit.client.0.smithi100.stdout: "[204800000, 409599999]", 2024-04-07T16:16:16.127 INFO:tasks.workunit.client.0.smithi100.stdout: "[409600000, 819199999]", 2024-04-07T16:16:16.127 INFO:tasks.workunit.client.0.smithi100.stdout: "[819200000, 1638399999]", 2024-04-07T16:16:16.128 INFO:tasks.workunit.client.0.smithi100.stdout: "[1638400000, 3276799999]", 2024-04-07T16:16:16.128 INFO:tasks.workunit.client.0.smithi100.stdout: "[3276800000, 6553599999]", 2024-04-07T16:16:16.128 INFO:tasks.workunit.client.0.smithi100.stdout: "[6553600000, 13107199999]", 2024-04-07T16:16:16.128 INFO:tasks.workunit.client.0.smithi100.stdout: "[13107200000, 26214399999]", 2024-04-07T16:16:16.128 INFO:tasks.workunit.client.0.smithi100.stdout: "[26214400000, 52428799999]", 2024-04-07T16:16:16.128 INFO:tasks.workunit.client.0.smithi100.stdout: "[52428800000, 104857599999]", 2024-04-07T16:16:16.128 INFO:tasks.workunit.client.0.smithi100.stdout: "[104857600000, 209715199999]", 2024-04-07T16:16:16.128 INFO:tasks.workunit.client.0.smithi100.stdout: "[209715200000, 419430399999]", 2024-04-07T16:16:16.128 INFO:tasks.workunit.client.0.smithi100.stdout: "[419430400000, 838860799999]", 2024-04-07T16:16:16.128 INFO:tasks.workunit.client.0.smithi100.stdout: "[838860800000, 1677721599999]", 2024-04-07T16:16:16.128 INFO:tasks.workunit.client.0.smithi100.stdout: "[1677721600000, 3355443199999]", 2024-04-07T16:16:16.128 INFO:tasks.workunit.client.0.smithi100.stdout: "[3355443200000, 6710886399999]", 2024-04-07T16:16:16.128 INFO:tasks.workunit.client.0.smithi100.stdout: "[6710886400000, 13421772799999]", 2024-04-07T16:16:16.128 INFO:tasks.workunit.client.0.smithi100.stdout: "[13421772800000, 26843545599999]", 2024-04-07T16:16:16.129 INFO:tasks.workunit.client.0.smithi100.stdout: "[26843545600000, 53687091199999]", 2024-04-07T16:16:16.129 INFO:tasks.workunit.client.0.smithi100.stdout: "[53687091200000, None]" 2024-04-07T16:16:16.129 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:16.129 INFO:tasks.workunit.client.0.smithi100.stdout: "scale_type": "log2" 2024-04-07T16:16:16.129 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.129 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:16.129 INFO:tasks.workunit.client.0.smithi100.stdout: "buckets": 32, 2024-04-07T16:16:16.129 INFO:tasks.workunit.client.0.smithi100.stdout: "min": 0, 2024-04-07T16:16:16.129 INFO:tasks.workunit.client.0.smithi100.stdout: "name": "Request size (bytes)", 2024-04-07T16:16:16.129 INFO:tasks.workunit.client.0.smithi100.stdout: "quant_size": 512, 2024-04-07T16:16:16.129 INFO:tasks.workunit.client.0.smithi100.stdout: "ranges": [ 2024-04-07T16:16:16.129 INFO:tasks.workunit.client.0.smithi100.stdout: "[None, -1]", 2024-04-07T16:16:16.129 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 511]", 2024-04-07T16:16:16.129 INFO:tasks.workunit.client.0.smithi100.stdout: "[512, 1023]", 2024-04-07T16:16:16.130 INFO:tasks.workunit.client.0.smithi100.stdout: "[1024, 2047]", 2024-04-07T16:16:16.130 INFO:tasks.workunit.client.0.smithi100.stdout: "[2048, 4095]", 2024-04-07T16:16:16.130 INFO:tasks.workunit.client.0.smithi100.stdout: "[4096, 8191]", 2024-04-07T16:16:16.130 INFO:tasks.workunit.client.0.smithi100.stdout: "[8192, 16383]", 2024-04-07T16:16:16.130 INFO:tasks.workunit.client.0.smithi100.stdout: "[16384, 32767]", 2024-04-07T16:16:16.130 INFO:tasks.workunit.client.0.smithi100.stdout: "[32768, 65535]", 2024-04-07T16:16:16.130 INFO:tasks.workunit.client.0.smithi100.stdout: "[65536, 131071]", 2024-04-07T16:16:16.130 INFO:tasks.workunit.client.0.smithi100.stdout: "[131072, 262143]", 2024-04-07T16:16:16.130 INFO:tasks.workunit.client.0.smithi100.stdout: "[262144, 524287]", 2024-04-07T16:16:16.130 INFO:tasks.workunit.client.0.smithi100.stdout: "[524288, 1048575]", 2024-04-07T16:16:16.130 INFO:tasks.workunit.client.0.smithi100.stdout: "[1048576, 2097151]", 2024-04-07T16:16:16.130 INFO:tasks.workunit.client.0.smithi100.stdout: "[2097152, 4194303]", 2024-04-07T16:16:16.130 INFO:tasks.workunit.client.0.smithi100.stdout: "[4194304, 8388607]", 2024-04-07T16:16:16.130 INFO:tasks.workunit.client.0.smithi100.stdout: "[8388608, 16777215]", 2024-04-07T16:16:16.130 INFO:tasks.workunit.client.0.smithi100.stdout: "[16777216, 33554431]", 2024-04-07T16:16:16.131 INFO:tasks.workunit.client.0.smithi100.stdout: "[33554432, 67108863]", 2024-04-07T16:16:16.131 INFO:tasks.workunit.client.0.smithi100.stdout: "[67108864, 134217727]", 2024-04-07T16:16:16.131 INFO:tasks.workunit.client.0.smithi100.stdout: "[134217728, 268435455]", 2024-04-07T16:16:16.131 INFO:tasks.workunit.client.0.smithi100.stdout: "[268435456, 536870911]", 2024-04-07T16:16:16.131 INFO:tasks.workunit.client.0.smithi100.stdout: "[536870912, 1073741823]", 2024-04-07T16:16:16.131 INFO:tasks.workunit.client.0.smithi100.stdout: "[1073741824, 2147483647]", 2024-04-07T16:16:16.131 INFO:tasks.workunit.client.0.smithi100.stdout: "[2147483648, 4294967295]", 2024-04-07T16:16:16.131 INFO:tasks.workunit.client.0.smithi100.stdout: "[4294967296, 8589934591]", 2024-04-07T16:16:16.131 INFO:tasks.workunit.client.0.smithi100.stdout: "[8589934592, 17179869183]", 2024-04-07T16:16:16.131 INFO:tasks.workunit.client.0.smithi100.stdout: "[17179869184, 34359738367]", 2024-04-07T16:16:16.131 INFO:tasks.workunit.client.0.smithi100.stdout: "[34359738368, 68719476735]", 2024-04-07T16:16:16.131 INFO:tasks.workunit.client.0.smithi100.stdout: "[68719476736, 137438953471]", 2024-04-07T16:16:16.131 INFO:tasks.workunit.client.0.smithi100.stdout: "[137438953472, 274877906943]", 2024-04-07T16:16:16.131 INFO:tasks.workunit.client.0.smithi100.stdout: "[274877906944, None]" 2024-04-07T16:16:16.131 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:16.132 INFO:tasks.workunit.client.0.smithi100.stdout: "scale_type": "log2" 2024-04-07T16:16:16.132 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:16.132 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:16.132 INFO:tasks.workunit.client.0.smithi100.stdout: "osds": [ 2024-04-07T16:16:16.132 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:16.132 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 0, 2024-04-07T16:16:16.132 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T16:16:16.132 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.132 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.132 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.132 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.132 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.133 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.133 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.133 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.133 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.133 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.133 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.133 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.133 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.133 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.133 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.133 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.133 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.133 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.133 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.133 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.134 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.134 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.134 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.134 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.134 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.134 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.134 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.134 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.134 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.134 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.134 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.134 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T16:16:16.134 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T16:16:16.134 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.134 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:16.135 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 1, 2024-04-07T16:16:16.135 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T16:16:16.135 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.135 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.135 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.135 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.135 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.135 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 11, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.135 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 10, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.135 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.135 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.135 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.135 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.135 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.135 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.136 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.136 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.136 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.136 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.136 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.136 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.136 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.136 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.136 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.136 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.136 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.136 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.136 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.136 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.136 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.137 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.137 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.137 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.137 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T16:16:16.137 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T16:16:16.137 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.137 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:16.137 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 2, 2024-04-07T16:16:16.137 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T16:16:16.137 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.137 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.137 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.137 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.138 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.138 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.138 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.138 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.138 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.138 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.138 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.138 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.138 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.138 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.138 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.138 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.138 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.139 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.139 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.139 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.139 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.139 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.139 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.139 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.139 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.139 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.139 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.139 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.139 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.139 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.139 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.139 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T16:16:16.139 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T16:16:16.140 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.140 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:16.140 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 3, 2024-04-07T16:16:16.140 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T16:16:16.140 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.140 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.140 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.140 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.140 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.140 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.140 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.140 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.140 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.140 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.140 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.140 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.141 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.141 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.141 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.141 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.141 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.141 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.141 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.141 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.141 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.141 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.141 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.141 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.141 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.141 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.141 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.142 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.142 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.142 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.142 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.142 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T16:16:16.142 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T16:16:16.142 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.142 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:16.142 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 4, 2024-04-07T16:16:16.142 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T16:16:16.142 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.142 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.142 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.142 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.143 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.143 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.143 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.143 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.143 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.143 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.143 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.143 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.143 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.143 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.143 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.143 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.143 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.143 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.143 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.144 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.144 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.144 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.144 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.144 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.144 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.144 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.144 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.144 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.144 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.144 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.144 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.144 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T16:16:16.144 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T16:16:16.144 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.145 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:16.145 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 5, 2024-04-07T16:16:16.145 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T16:16:16.145 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.145 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.145 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.145 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.145 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.145 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.145 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.145 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.145 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.145 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.145 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.145 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.145 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.146 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.146 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.146 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.146 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.146 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.146 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.146 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.146 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.146 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.146 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.146 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.146 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.146 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.147 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.147 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.147 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.147 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.147 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.147 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T16:16:16.147 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T16:16:16.147 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.147 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:16.147 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 6, 2024-04-07T16:16:16.147 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T16:16:16.147 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.147 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.147 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.147 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.148 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.148 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.148 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.148 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.148 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.148 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.148 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.148 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.148 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.148 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.148 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.148 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.148 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.148 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.148 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.149 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.149 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.149 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.149 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.149 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.149 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.149 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.149 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.149 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.149 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.149 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.149 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.149 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T16:16:16.149 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T16:16:16.149 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.149 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:16.150 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 7, 2024-04-07T16:16:16.150 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T16:16:16.150 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.150 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.150 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.150 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.150 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.150 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.150 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 47, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.150 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.150 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.150 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.150 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.150 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.151 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.151 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.151 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.151 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.151 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.151 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.151 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.151 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.151 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.151 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.151 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.151 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.151 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.152 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.152 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.152 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.152 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.152 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.152 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.152 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T16:16:16.152 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T16:16:16.152 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:16.152 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T16:16:16.152 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.152 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_latency_in_bytes_histogram": { 2024-04-07T16:16:16.152 INFO:tasks.workunit.client.0.smithi100.stdout: "axes": [ 2024-04-07T16:16:16.152 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:16.152 INFO:tasks.workunit.client.0.smithi100.stdout: "buckets": 32, 2024-04-07T16:16:16.153 INFO:tasks.workunit.client.0.smithi100.stdout: "min": 0, 2024-04-07T16:16:16.153 INFO:tasks.workunit.client.0.smithi100.stdout: "name": "Latency (usec)", 2024-04-07T16:16:16.153 INFO:tasks.workunit.client.0.smithi100.stdout: "quant_size": 100000, 2024-04-07T16:16:16.153 INFO:tasks.workunit.client.0.smithi100.stdout: "ranges": [ 2024-04-07T16:16:16.153 INFO:tasks.workunit.client.0.smithi100.stdout: "[None, -1]", 2024-04-07T16:16:16.153 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 99999]", 2024-04-07T16:16:16.153 INFO:tasks.workunit.client.0.smithi100.stdout: "[100000, 199999]", 2024-04-07T16:16:16.153 INFO:tasks.workunit.client.0.smithi100.stdout: "[200000, 399999]", 2024-04-07T16:16:16.153 INFO:tasks.workunit.client.0.smithi100.stdout: "[400000, 799999]", 2024-04-07T16:16:16.153 INFO:tasks.workunit.client.0.smithi100.stdout: "[800000, 1599999]", 2024-04-07T16:16:16.153 INFO:tasks.workunit.client.0.smithi100.stdout: "[1600000, 3199999]", 2024-04-07T16:16:16.153 INFO:tasks.workunit.client.0.smithi100.stdout: "[3200000, 6399999]", 2024-04-07T16:16:16.153 INFO:tasks.workunit.client.0.smithi100.stdout: "[6400000, 12799999]", 2024-04-07T16:16:16.153 INFO:tasks.workunit.client.0.smithi100.stdout: "[12800000, 25599999]", 2024-04-07T16:16:16.154 INFO:tasks.workunit.client.0.smithi100.stdout: "[25600000, 51199999]", 2024-04-07T16:16:16.154 INFO:tasks.workunit.client.0.smithi100.stdout: "[51200000, 102399999]", 2024-04-07T16:16:16.154 INFO:tasks.workunit.client.0.smithi100.stdout: "[102400000, 204799999]", 2024-04-07T16:16:16.154 INFO:tasks.workunit.client.0.smithi100.stdout: "[204800000, 409599999]", 2024-04-07T16:16:16.154 INFO:tasks.workunit.client.0.smithi100.stdout: "[409600000, 819199999]", 2024-04-07T16:16:16.154 INFO:tasks.workunit.client.0.smithi100.stdout: "[819200000, 1638399999]", 2024-04-07T16:16:16.154 INFO:tasks.workunit.client.0.smithi100.stdout: "[1638400000, 3276799999]", 2024-04-07T16:16:16.154 INFO:tasks.workunit.client.0.smithi100.stdout: "[3276800000, 6553599999]", 2024-04-07T16:16:16.154 INFO:tasks.workunit.client.0.smithi100.stdout: "[6553600000, 13107199999]", 2024-04-07T16:16:16.154 INFO:tasks.workunit.client.0.smithi100.stdout: "[13107200000, 26214399999]", 2024-04-07T16:16:16.154 INFO:tasks.workunit.client.0.smithi100.stdout: "[26214400000, 52428799999]", 2024-04-07T16:16:16.154 INFO:tasks.workunit.client.0.smithi100.stdout: "[52428800000, 104857599999]", 2024-04-07T16:16:16.154 INFO:tasks.workunit.client.0.smithi100.stdout: "[104857600000, 209715199999]", 2024-04-07T16:16:16.154 INFO:tasks.workunit.client.0.smithi100.stdout: "[209715200000, 419430399999]", 2024-04-07T16:16:16.155 INFO:tasks.workunit.client.0.smithi100.stdout: "[419430400000, 838860799999]", 2024-04-07T16:16:16.155 INFO:tasks.workunit.client.0.smithi100.stdout: "[838860800000, 1677721599999]", 2024-04-07T16:16:16.155 INFO:tasks.workunit.client.0.smithi100.stdout: "[1677721600000, 3355443199999]", 2024-04-07T16:16:16.155 INFO:tasks.workunit.client.0.smithi100.stdout: "[3355443200000, 6710886399999]", 2024-04-07T16:16:16.155 INFO:tasks.workunit.client.0.smithi100.stdout: "[6710886400000, 13421772799999]", 2024-04-07T16:16:16.155 INFO:tasks.workunit.client.0.smithi100.stdout: "[13421772800000, 26843545599999]", 2024-04-07T16:16:16.155 INFO:tasks.workunit.client.0.smithi100.stdout: "[26843545600000, 53687091199999]", 2024-04-07T16:16:16.155 INFO:tasks.workunit.client.0.smithi100.stdout: "[53687091200000, None]" 2024-04-07T16:16:16.155 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:16.155 INFO:tasks.workunit.client.0.smithi100.stdout: "scale_type": "log2" 2024-04-07T16:16:16.155 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.155 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:16.155 INFO:tasks.workunit.client.0.smithi100.stdout: "buckets": 32, 2024-04-07T16:16:16.155 INFO:tasks.workunit.client.0.smithi100.stdout: "min": 0, 2024-04-07T16:16:16.155 INFO:tasks.workunit.client.0.smithi100.stdout: "name": "Request size (bytes)", 2024-04-07T16:16:16.155 INFO:tasks.workunit.client.0.smithi100.stdout: "quant_size": 512, 2024-04-07T16:16:16.156 INFO:tasks.workunit.client.0.smithi100.stdout: "ranges": [ 2024-04-07T16:16:16.156 INFO:tasks.workunit.client.0.smithi100.stdout: "[None, -1]", 2024-04-07T16:16:16.156 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 511]", 2024-04-07T16:16:16.156 INFO:tasks.workunit.client.0.smithi100.stdout: "[512, 1023]", 2024-04-07T16:16:16.156 INFO:tasks.workunit.client.0.smithi100.stdout: "[1024, 2047]", 2024-04-07T16:16:16.156 INFO:tasks.workunit.client.0.smithi100.stdout: "[2048, 4095]", 2024-04-07T16:16:16.156 INFO:tasks.workunit.client.0.smithi100.stdout: "[4096, 8191]", 2024-04-07T16:16:16.156 INFO:tasks.workunit.client.0.smithi100.stdout: "[8192, 16383]", 2024-04-07T16:16:16.156 INFO:tasks.workunit.client.0.smithi100.stdout: "[16384, 32767]", 2024-04-07T16:16:16.156 INFO:tasks.workunit.client.0.smithi100.stdout: "[32768, 65535]", 2024-04-07T16:16:16.156 INFO:tasks.workunit.client.0.smithi100.stdout: "[65536, 131071]", 2024-04-07T16:16:16.156 INFO:tasks.workunit.client.0.smithi100.stdout: "[131072, 262143]", 2024-04-07T16:16:16.156 INFO:tasks.workunit.client.0.smithi100.stdout: "[262144, 524287]", 2024-04-07T16:16:16.156 INFO:tasks.workunit.client.0.smithi100.stdout: "[524288, 1048575]", 2024-04-07T16:16:16.157 INFO:tasks.workunit.client.0.smithi100.stdout: "[1048576, 2097151]", 2024-04-07T16:16:16.157 INFO:tasks.workunit.client.0.smithi100.stdout: "[2097152, 4194303]", 2024-04-07T16:16:16.157 INFO:tasks.workunit.client.0.smithi100.stdout: "[4194304, 8388607]", 2024-04-07T16:16:16.157 INFO:tasks.workunit.client.0.smithi100.stdout: "[8388608, 16777215]", 2024-04-07T16:16:16.157 INFO:tasks.workunit.client.0.smithi100.stdout: "[16777216, 33554431]", 2024-04-07T16:16:16.157 INFO:tasks.workunit.client.0.smithi100.stdout: "[33554432, 67108863]", 2024-04-07T16:16:16.157 INFO:tasks.workunit.client.0.smithi100.stdout: "[67108864, 134217727]", 2024-04-07T16:16:16.157 INFO:tasks.workunit.client.0.smithi100.stdout: "[134217728, 268435455]", 2024-04-07T16:16:16.157 INFO:tasks.workunit.client.0.smithi100.stdout: "[268435456, 536870911]", 2024-04-07T16:16:16.157 INFO:tasks.workunit.client.0.smithi100.stdout: "[536870912, 1073741823]", 2024-04-07T16:16:16.157 INFO:tasks.workunit.client.0.smithi100.stdout: "[1073741824, 2147483647]", 2024-04-07T16:16:16.157 INFO:tasks.workunit.client.0.smithi100.stdout: "[2147483648, 4294967295]", 2024-04-07T16:16:16.157 INFO:tasks.workunit.client.0.smithi100.stdout: "[4294967296, 8589934591]", 2024-04-07T16:16:16.157 INFO:tasks.workunit.client.0.smithi100.stdout: "[8589934592, 17179869183]", 2024-04-07T16:16:16.157 INFO:tasks.workunit.client.0.smithi100.stdout: "[17179869184, 34359738367]", 2024-04-07T16:16:16.158 INFO:tasks.workunit.client.0.smithi100.stdout: "[34359738368, 68719476735]", 2024-04-07T16:16:16.158 INFO:tasks.workunit.client.0.smithi100.stdout: "[68719476736, 137438953471]", 2024-04-07T16:16:16.158 INFO:tasks.workunit.client.0.smithi100.stdout: "[137438953472, 274877906943]", 2024-04-07T16:16:16.158 INFO:tasks.workunit.client.0.smithi100.stdout: "[274877906944, None]" 2024-04-07T16:16:16.158 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:16.158 INFO:tasks.workunit.client.0.smithi100.stdout: "scale_type": "log2" 2024-04-07T16:16:16.158 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:16.158 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:16.158 INFO:tasks.workunit.client.0.smithi100.stdout: "osds": [ 2024-04-07T16:16:16.158 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:16.158 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 0, 2024-04-07T16:16:16.158 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T16:16:16.158 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.158 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.158 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.159 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.159 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.159 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.159 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.159 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.159 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.159 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 4, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.159 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.159 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.159 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.159 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.159 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.159 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.159 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.160 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.160 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.160 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.160 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.160 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.160 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.160 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.160 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.160 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.160 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.160 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.160 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.160 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.160 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.161 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T16:16:16.161 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T16:16:16.161 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.161 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:16.161 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 1, 2024-04-07T16:16:16.161 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T16:16:16.161 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.161 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.161 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.161 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.161 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.161 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 6, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.161 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 9, 3, 0, 0, 0, 0, 0, 0, 17, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.161 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 24, 5, 0, 0, 0, 0, 0, 0, 15, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.161 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 14, 0, 0, 0, 0, 0, 0, 0, 5, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.161 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.162 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.162 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.162 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.162 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.162 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.162 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.162 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.162 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.162 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.162 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.162 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.162 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.162 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.162 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.163 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.163 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.163 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.163 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.163 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.163 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.163 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.163 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T16:16:16.163 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T16:16:16.163 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.163 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:16.163 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 2, 2024-04-07T16:16:16.163 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T16:16:16.163 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.163 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.163 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.164 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.164 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.164 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.164 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.164 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.164 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 5, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.164 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.164 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 5, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.164 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.164 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.164 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.164 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.164 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.165 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.165 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.165 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.165 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.165 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.165 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.165 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.165 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.165 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.165 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.165 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.165 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.165 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.165 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.165 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.166 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T16:16:16.166 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T16:16:16.166 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.166 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:16.166 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 3, 2024-04-07T16:16:16.166 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T16:16:16.166 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.166 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.166 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.166 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.166 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.166 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.166 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.166 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.167 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.167 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.167 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.167 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.167 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.167 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.167 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.167 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.167 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.167 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.167 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.167 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.167 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.167 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.167 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.168 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.168 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.168 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.168 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.168 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.168 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.168 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.168 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.168 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T16:16:16.168 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T16:16:16.168 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.168 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:16.168 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 4, 2024-04-07T16:16:16.168 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T16:16:16.168 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.168 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.169 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.169 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.169 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.169 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.169 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.169 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.169 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.169 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.169 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.169 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.169 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.169 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.169 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.170 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.170 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.170 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.170 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.170 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.170 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.170 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.170 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.170 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.170 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.170 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.170 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.170 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.170 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.171 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.171 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.171 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T16:16:16.171 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T16:16:16.171 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.171 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:16.171 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 5, 2024-04-07T16:16:16.171 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T16:16:16.171 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.171 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.171 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.171 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.171 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.171 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.171 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.172 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.172 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.172 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.172 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.172 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.172 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.172 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.172 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.172 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.172 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.172 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.172 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.172 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.172 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.172 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.173 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.173 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.173 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.173 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.173 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.173 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.173 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.173 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.173 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.173 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T16:16:16.173 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T16:16:16.173 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.173 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:16.173 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 6, 2024-04-07T16:16:16.173 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T16:16:16.174 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.174 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.174 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.174 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.174 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.174 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.174 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.174 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 6, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.174 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.174 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 2, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.174 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.174 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.174 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.174 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.175 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.175 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.175 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.175 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.175 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.175 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.175 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.175 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.175 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.175 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.175 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.175 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.175 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.175 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.175 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.176 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.176 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.176 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T16:16:16.176 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T16:16:16.176 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.176 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:16.176 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 7, 2024-04-07T16:16:16.176 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T16:16:16.176 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.176 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.176 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.176 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.176 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.176 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.177 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.177 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.177 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.177 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.177 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.177 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.177 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.177 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.177 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.177 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.177 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.177 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.177 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.177 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.177 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.177 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.178 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.178 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.178 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.178 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.178 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.178 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.178 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.178 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.178 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T16:16:16.178 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T16:16:16.178 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T16:16:16.178 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:16.178 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T16:16:16.179 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:16.179 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:16.179 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:16.179 INFO:tasks.workunit.client.0.smithi100.stdout: "perf_counters": { 2024-04-07T16:16:16.179 INFO:tasks.workunit.client.0.smithi100.stdout: "mds.a52fc0d8acb515431879057649ff294cfe14c620": { 2024-04-07T16:16:16.179 INFO:tasks.workunit.client.0.smithi100.stdout: "mds": { 2024-04-07T16:16:16.179 INFO:tasks.workunit.client.0.smithi100.stdout: "caps": { 2024-04-07T16:16:16.179 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.179 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.179 INFO:tasks.workunit.client.0.smithi100.stdout: "ceph_cap_op_flush_ack": { 2024-04-07T16:16:16.179 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.179 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.179 INFO:tasks.workunit.client.0.smithi100.stdout: "ceph_cap_op_flushsnap_ack": { 2024-04-07T16:16:16.179 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.179 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.180 INFO:tasks.workunit.client.0.smithi100.stdout: "ceph_cap_op_grant": { 2024-04-07T16:16:16.180 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.180 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.180 INFO:tasks.workunit.client.0.smithi100.stdout: "ceph_cap_op_revoke": { 2024-04-07T16:16:16.180 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.180 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.180 INFO:tasks.workunit.client.0.smithi100.stdout: "ceph_cap_op_trunc": { 2024-04-07T16:16:16.180 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.180 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.180 INFO:tasks.workunit.client.0.smithi100.stdout: "dir_commit": { 2024-04-07T16:16:16.180 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 12 2024-04-07T16:16:16.180 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.180 INFO:tasks.workunit.client.0.smithi100.stdout: "dir_fetch_complete": { 2024-04-07T16:16:16.180 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.180 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.181 INFO:tasks.workunit.client.0.smithi100.stdout: "dir_fetch_keys": { 2024-04-07T16:16:16.181 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.181 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.181 INFO:tasks.workunit.client.0.smithi100.stdout: "dir_merge": { 2024-04-07T16:16:16.181 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.181 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.181 INFO:tasks.workunit.client.0.smithi100.stdout: "dir_split": { 2024-04-07T16:16:16.181 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.181 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.181 INFO:tasks.workunit.client.0.smithi100.stdout: "exported_inodes": { 2024-04-07T16:16:16.181 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.181 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.181 INFO:tasks.workunit.client.0.smithi100.stdout: "forward": { 2024-04-07T16:16:16.181 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.181 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.182 INFO:tasks.workunit.client.0.smithi100.stdout: "handle_client_cap_release": { 2024-04-07T16:16:16.182 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.182 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.182 INFO:tasks.workunit.client.0.smithi100.stdout: "handle_client_caps": { 2024-04-07T16:16:16.182 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.182 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.182 INFO:tasks.workunit.client.0.smithi100.stdout: "handle_client_caps_dirty": { 2024-04-07T16:16:16.182 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.182 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.182 INFO:tasks.workunit.client.0.smithi100.stdout: "handle_inode_file_caps": { 2024-04-07T16:16:16.182 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.182 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.182 INFO:tasks.workunit.client.0.smithi100.stdout: "imported_inodes": { 2024-04-07T16:16:16.182 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.183 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.183 INFO:tasks.workunit.client.0.smithi100.stdout: "inodes": { 2024-04-07T16:16:16.183 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 10 2024-04-07T16:16:16.183 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.183 INFO:tasks.workunit.client.0.smithi100.stdout: "inodes_expired": { 2024-04-07T16:16:16.183 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.183 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.183 INFO:tasks.workunit.client.0.smithi100.stdout: "inodes_pinned": { 2024-04-07T16:16:16.183 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 10 2024-04-07T16:16:16.183 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.183 INFO:tasks.workunit.client.0.smithi100.stdout: "inodes_with_caps": { 2024-04-07T16:16:16.183 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.183 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.183 INFO:tasks.workunit.client.0.smithi100.stdout: "load_cent": { 2024-04-07T16:16:16.183 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.184 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.184 INFO:tasks.workunit.client.0.smithi100.stdout: "openino_dir_fetch": { 2024-04-07T16:16:16.184 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.184 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.184 INFO:tasks.workunit.client.0.smithi100.stdout: "process_request_cap_release": { 2024-04-07T16:16:16.184 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.184 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.184 INFO:tasks.workunit.client.0.smithi100.stdout: "reply_latency": { 2024-04-07T16:16:16.184 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 2, 2024-04-07T16:16:16.184 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 131939 2024-04-07T16:16:16.184 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.184 INFO:tasks.workunit.client.0.smithi100.stdout: "request": { 2024-04-07T16:16:16.184 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2 2024-04-07T16:16:16.184 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.184 INFO:tasks.workunit.client.0.smithi100.stdout: "root_rbytes": { 2024-04-07T16:16:16.185 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.185 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.185 INFO:tasks.workunit.client.0.smithi100.stdout: "root_rfiles": { 2024-04-07T16:16:16.185 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.185 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.185 INFO:tasks.workunit.client.0.smithi100.stdout: "root_rsnaps": { 2024-04-07T16:16:16.185 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.185 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.185 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_reply": { 2024-04-07T16:16:16.185 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.185 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.185 INFO:tasks.workunit.client.0.smithi100.stdout: "subtrees": { 2024-04-07T16:16:16.185 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2 2024-04-07T16:16:16.185 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:16.185 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.186 INFO:tasks.workunit.client.0.smithi100.stdout: "mds_cache": { 2024-04-07T16:16:16.186 INFO:tasks.workunit.client.0.smithi100.stdout: "ireq_enqueue_scrub": { 2024-04-07T16:16:16.186 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.186 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.186 INFO:tasks.workunit.client.0.smithi100.stdout: "ireq_exportdir": { 2024-04-07T16:16:16.186 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.186 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.186 INFO:tasks.workunit.client.0.smithi100.stdout: "ireq_flush": { 2024-04-07T16:16:16.186 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.186 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.186 INFO:tasks.workunit.client.0.smithi100.stdout: "ireq_fragmentdir": { 2024-04-07T16:16:16.186 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.186 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.186 INFO:tasks.workunit.client.0.smithi100.stdout: "ireq_fragstats": { 2024-04-07T16:16:16.187 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.187 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.187 INFO:tasks.workunit.client.0.smithi100.stdout: "ireq_inodestats": { 2024-04-07T16:16:16.187 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.187 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.187 INFO:tasks.workunit.client.0.smithi100.stdout: "num_recovering_enqueued": { 2024-04-07T16:16:16.187 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.187 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.187 INFO:tasks.workunit.client.0.smithi100.stdout: "num_recovering_prioritized": { 2024-04-07T16:16:16.187 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.187 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.187 INFO:tasks.workunit.client.0.smithi100.stdout: "num_recovering_processing": { 2024-04-07T16:16:16.187 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.187 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.187 INFO:tasks.workunit.client.0.smithi100.stdout: "num_strays": { 2024-04-07T16:16:16.188 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.188 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.188 INFO:tasks.workunit.client.0.smithi100.stdout: "num_strays_delayed": { 2024-04-07T16:16:16.188 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.188 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.188 INFO:tasks.workunit.client.0.smithi100.stdout: "num_strays_enqueuing": { 2024-04-07T16:16:16.188 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.188 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.188 INFO:tasks.workunit.client.0.smithi100.stdout: "recovery_completed": { 2024-04-07T16:16:16.188 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.188 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.188 INFO:tasks.workunit.client.0.smithi100.stdout: "recovery_started": { 2024-04-07T16:16:16.188 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.188 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.188 INFO:tasks.workunit.client.0.smithi100.stdout: "strays_created": { 2024-04-07T16:16:16.189 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.189 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.189 INFO:tasks.workunit.client.0.smithi100.stdout: "strays_enqueued": { 2024-04-07T16:16:16.189 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.189 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.189 INFO:tasks.workunit.client.0.smithi100.stdout: "strays_migrated": { 2024-04-07T16:16:16.189 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.189 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.189 INFO:tasks.workunit.client.0.smithi100.stdout: "strays_reintegrated": { 2024-04-07T16:16:16.189 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.189 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:16.189 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.189 INFO:tasks.workunit.client.0.smithi100.stdout: "mds_log": { 2024-04-07T16:16:16.189 INFO:tasks.workunit.client.0.smithi100.stdout: "ev": { 2024-04-07T16:16:16.189 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5 2024-04-07T16:16:16.190 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.190 INFO:tasks.workunit.client.0.smithi100.stdout: "evadd": { 2024-04-07T16:16:16.190 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5 2024-04-07T16:16:16.190 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.190 INFO:tasks.workunit.client.0.smithi100.stdout: "evex": { 2024-04-07T16:16:16.190 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.190 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.190 INFO:tasks.workunit.client.0.smithi100.stdout: "evexd": { 2024-04-07T16:16:16.190 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.190 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.190 INFO:tasks.workunit.client.0.smithi100.stdout: "evexg": { 2024-04-07T16:16:16.190 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.190 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.190 INFO:tasks.workunit.client.0.smithi100.stdout: "evtrm": { 2024-04-07T16:16:16.190 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.191 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.191 INFO:tasks.workunit.client.0.smithi100.stdout: "jlat": { 2024-04-07T16:16:16.191 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 5, 2024-04-07T16:16:16.191 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 20108532 2024-04-07T16:16:16.191 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.191 INFO:tasks.workunit.client.0.smithi100.stdout: "replayed": { 2024-04-07T16:16:16.191 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.191 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.191 INFO:tasks.workunit.client.0.smithi100.stdout: "seg": { 2024-04-07T16:16:16.191 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1 2024-04-07T16:16:16.191 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.191 INFO:tasks.workunit.client.0.smithi100.stdout: "segadd": { 2024-04-07T16:16:16.191 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1 2024-04-07T16:16:16.191 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.191 INFO:tasks.workunit.client.0.smithi100.stdout: "segex": { 2024-04-07T16:16:16.192 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.192 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.192 INFO:tasks.workunit.client.0.smithi100.stdout: "segexd": { 2024-04-07T16:16:16.192 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.192 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.192 INFO:tasks.workunit.client.0.smithi100.stdout: "segexg": { 2024-04-07T16:16:16.192 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.192 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.192 INFO:tasks.workunit.client.0.smithi100.stdout: "segtrm": { 2024-04-07T16:16:16.192 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.192 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:16.192 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.192 INFO:tasks.workunit.client.0.smithi100.stdout: "mds_mem": { 2024-04-07T16:16:16.192 INFO:tasks.workunit.client.0.smithi100.stdout: "cap": { 2024-04-07T16:16:16.192 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.193 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.193 INFO:tasks.workunit.client.0.smithi100.stdout: "cap+": { 2024-04-07T16:16:16.193 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2 2024-04-07T16:16:16.193 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.193 INFO:tasks.workunit.client.0.smithi100.stdout: "cap-": { 2024-04-07T16:16:16.193 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2 2024-04-07T16:16:16.193 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.193 INFO:tasks.workunit.client.0.smithi100.stdout: "dir": { 2024-04-07T16:16:16.193 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 12 2024-04-07T16:16:16.193 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.193 INFO:tasks.workunit.client.0.smithi100.stdout: "dir+": { 2024-04-07T16:16:16.193 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 12 2024-04-07T16:16:16.193 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.193 INFO:tasks.workunit.client.0.smithi100.stdout: "dir-": { 2024-04-07T16:16:16.193 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.193 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.194 INFO:tasks.workunit.client.0.smithi100.stdout: "dn": { 2024-04-07T16:16:16.194 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 10 2024-04-07T16:16:16.194 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.194 INFO:tasks.workunit.client.0.smithi100.stdout: "dn+": { 2024-04-07T16:16:16.194 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 10 2024-04-07T16:16:16.194 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.194 INFO:tasks.workunit.client.0.smithi100.stdout: "dn-": { 2024-04-07T16:16:16.194 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.194 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.194 INFO:tasks.workunit.client.0.smithi100.stdout: "heap": { 2024-04-07T16:16:16.194 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 207156 2024-04-07T16:16:16.194 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.194 INFO:tasks.workunit.client.0.smithi100.stdout: "ino": { 2024-04-07T16:16:16.194 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 13 2024-04-07T16:16:16.194 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.195 INFO:tasks.workunit.client.0.smithi100.stdout: "ino+": { 2024-04-07T16:16:16.195 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 13 2024-04-07T16:16:16.195 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.195 INFO:tasks.workunit.client.0.smithi100.stdout: "ino-": { 2024-04-07T16:16:16.195 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.195 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.195 INFO:tasks.workunit.client.0.smithi100.stdout: "rss": { 2024-04-07T16:16:16.195 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 35308 2024-04-07T16:16:16.195 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:16.195 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.195 INFO:tasks.workunit.client.0.smithi100.stdout: "mds_server": { 2024-04-07T16:16:16.195 INFO:tasks.workunit.client.0.smithi100.stdout: "cap_acquisition_throttle": { 2024-04-07T16:16:16.195 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.196 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.196 INFO:tasks.workunit.client.0.smithi100.stdout: "cap_revoke_eviction": { 2024-04-07T16:16:16.196 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.196 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.196 INFO:tasks.workunit.client.0.smithi100.stdout: "handle_client_request": { 2024-04-07T16:16:16.196 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2 2024-04-07T16:16:16.196 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.196 INFO:tasks.workunit.client.0.smithi100.stdout: "handle_client_session": { 2024-04-07T16:16:16.196 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 12 2024-04-07T16:16:16.196 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.196 INFO:tasks.workunit.client.0.smithi100.stdout: "handle_peer_request": { 2024-04-07T16:16:16.196 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.196 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.196 INFO:tasks.workunit.client.0.smithi100.stdout: "req_create_latency": { 2024-04-07T16:16:16.197 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.197 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.197 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.197 INFO:tasks.workunit.client.0.smithi100.stdout: "req_getattr_latency": { 2024-04-07T16:16:16.197 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 2, 2024-04-07T16:16:16.197 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 131939 2024-04-07T16:16:16.197 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.197 INFO:tasks.workunit.client.0.smithi100.stdout: "req_getfilelock_latency": { 2024-04-07T16:16:16.197 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.197 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.197 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.197 INFO:tasks.workunit.client.0.smithi100.stdout: "req_getvxattr_latency": { 2024-04-07T16:16:16.197 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.197 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.197 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.198 INFO:tasks.workunit.client.0.smithi100.stdout: "req_link_latency": { 2024-04-07T16:16:16.198 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.198 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.198 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.198 INFO:tasks.workunit.client.0.smithi100.stdout: "req_lookup_latency": { 2024-04-07T16:16:16.198 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.198 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.198 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.198 INFO:tasks.workunit.client.0.smithi100.stdout: "req_lookuphash_latency": { 2024-04-07T16:16:16.198 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.198 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.198 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.198 INFO:tasks.workunit.client.0.smithi100.stdout: "req_lookupino_latency": { 2024-04-07T16:16:16.199 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.199 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.199 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.199 INFO:tasks.workunit.client.0.smithi100.stdout: "req_lookupname_latency": { 2024-04-07T16:16:16.199 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.199 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.199 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.199 INFO:tasks.workunit.client.0.smithi100.stdout: "req_lookupparent_latency": { 2024-04-07T16:16:16.199 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.199 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.199 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.199 INFO:tasks.workunit.client.0.smithi100.stdout: "req_lookupsnap_latency": { 2024-04-07T16:16:16.199 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.199 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.199 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.200 INFO:tasks.workunit.client.0.smithi100.stdout: "req_lssnap_latency": { 2024-04-07T16:16:16.200 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.200 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.200 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.200 INFO:tasks.workunit.client.0.smithi100.stdout: "req_mkdir_latency": { 2024-04-07T16:16:16.200 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.200 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.200 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.200 INFO:tasks.workunit.client.0.smithi100.stdout: "req_mknod_latency": { 2024-04-07T16:16:16.200 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.200 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.200 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.200 INFO:tasks.workunit.client.0.smithi100.stdout: "req_mksnap_latency": { 2024-04-07T16:16:16.200 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.200 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.201 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.201 INFO:tasks.workunit.client.0.smithi100.stdout: "req_open_latency": { 2024-04-07T16:16:16.201 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.201 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.201 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.201 INFO:tasks.workunit.client.0.smithi100.stdout: "req_readdir_latency": { 2024-04-07T16:16:16.201 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.201 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.201 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.201 INFO:tasks.workunit.client.0.smithi100.stdout: "req_rename_latency": { 2024-04-07T16:16:16.201 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.201 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.201 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.201 INFO:tasks.workunit.client.0.smithi100.stdout: "req_renamesnap_latency": { 2024-04-07T16:16:16.202 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.202 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.202 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.202 INFO:tasks.workunit.client.0.smithi100.stdout: "req_rmdir_latency": { 2024-04-07T16:16:16.202 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.202 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.202 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.202 INFO:tasks.workunit.client.0.smithi100.stdout: "req_rmsnap_latency": { 2024-04-07T16:16:16.202 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.202 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.202 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.202 INFO:tasks.workunit.client.0.smithi100.stdout: "req_rmxattr_latency": { 2024-04-07T16:16:16.202 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.202 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.202 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.203 INFO:tasks.workunit.client.0.smithi100.stdout: "req_setattr_latency": { 2024-04-07T16:16:16.203 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.203 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.203 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.203 INFO:tasks.workunit.client.0.smithi100.stdout: "req_setdirlayout_latency": { 2024-04-07T16:16:16.203 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.203 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.203 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.203 INFO:tasks.workunit.client.0.smithi100.stdout: "req_setfilelock_latency": { 2024-04-07T16:16:16.203 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.203 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.203 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.203 INFO:tasks.workunit.client.0.smithi100.stdout: "req_setlayout_latency": { 2024-04-07T16:16:16.203 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.203 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.204 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.204 INFO:tasks.workunit.client.0.smithi100.stdout: "req_setxattr_latency": { 2024-04-07T16:16:16.204 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.204 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.204 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.204 INFO:tasks.workunit.client.0.smithi100.stdout: "req_snapdiff_latency": { 2024-04-07T16:16:16.204 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.204 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.204 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.204 INFO:tasks.workunit.client.0.smithi100.stdout: "req_symlink_latency": { 2024-04-07T16:16:16.204 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.204 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.204 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.204 INFO:tasks.workunit.client.0.smithi100.stdout: "req_unlink_latency": { 2024-04-07T16:16:16.204 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.205 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.205 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:16.205 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.205 INFO:tasks.workunit.client.0.smithi100.stdout: "mds_sessions": { 2024-04-07T16:16:16.205 INFO:tasks.workunit.client.0.smithi100.stdout: "average_load": { 2024-04-07T16:16:16.205 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1 2024-04-07T16:16:16.205 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.205 INFO:tasks.workunit.client.0.smithi100.stdout: "avg_session_uptime": { 2024-04-07T16:16:16.205 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 59 2024-04-07T16:16:16.205 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.205 INFO:tasks.workunit.client.0.smithi100.stdout: "mdthresh_evicted": { 2024-04-07T16:16:16.205 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.205 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.205 INFO:tasks.workunit.client.0.smithi100.stdout: "session_add": { 2024-04-07T16:16:16.205 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2 2024-04-07T16:16:16.206 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.206 INFO:tasks.workunit.client.0.smithi100.stdout: "session_count": { 2024-04-07T16:16:16.206 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.206 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.206 INFO:tasks.workunit.client.0.smithi100.stdout: "session_remove": { 2024-04-07T16:16:16.206 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2 2024-04-07T16:16:16.206 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.206 INFO:tasks.workunit.client.0.smithi100.stdout: "sessions_open": { 2024-04-07T16:16:16.206 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.206 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.206 INFO:tasks.workunit.client.0.smithi100.stdout: "sessions_stale": { 2024-04-07T16:16:16.206 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.206 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.206 INFO:tasks.workunit.client.0.smithi100.stdout: "total_load": { 2024-04-07T16:16:16.207 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2 2024-04-07T16:16:16.207 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:16.207 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.207 INFO:tasks.workunit.client.0.smithi100.stdout: "objecter": { 2024-04-07T16:16:16.207 INFO:tasks.workunit.client.0.smithi100.stdout: "op_active": { 2024-04-07T16:16:16.207 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.207 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.207 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r": { 2024-04-07T16:16:16.207 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.207 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.207 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rmw": { 2024-04-07T16:16:16.207 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.207 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.207 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w": { 2024-04-07T16:16:16.207 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 45 2024-04-07T16:16:16.208 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:16.208 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.208 INFO:tasks.workunit.client.0.smithi100.stdout: "purge_queue": { 2024-04-07T16:16:16.208 INFO:tasks.workunit.client.0.smithi100.stdout: "pq_executed": { 2024-04-07T16:16:16.208 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.208 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.208 INFO:tasks.workunit.client.0.smithi100.stdout: "pq_executing": { 2024-04-07T16:16:16.208 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.208 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.208 INFO:tasks.workunit.client.0.smithi100.stdout: "pq_executing_high_water": { 2024-04-07T16:16:16.208 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.208 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.208 INFO:tasks.workunit.client.0.smithi100.stdout: "pq_executing_ops": { 2024-04-07T16:16:16.208 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.208 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.209 INFO:tasks.workunit.client.0.smithi100.stdout: "pq_executing_ops_high_water": { 2024-04-07T16:16:16.209 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.209 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.209 INFO:tasks.workunit.client.0.smithi100.stdout: "pq_item_in_journal": { 2024-04-07T16:16:16.209 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.209 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:16.209 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:16.209 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.209 INFO:tasks.workunit.client.0.smithi100.stdout: "mon.8dd0baa036f0a7e6d1f3bdce86136fe527836c3d": { 2024-04-07T16:16:16.209 INFO:tasks.workunit.client.0.smithi100.stdout: "mon": { 2024-04-07T16:16:16.209 INFO:tasks.workunit.client.0.smithi100.stdout: "election_call": { 2024-04-07T16:16:16.209 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1 2024-04-07T16:16:16.209 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.209 INFO:tasks.workunit.client.0.smithi100.stdout: "election_lose": { 2024-04-07T16:16:16.209 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1 2024-04-07T16:16:16.209 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.210 INFO:tasks.workunit.client.0.smithi100.stdout: "election_win": { 2024-04-07T16:16:16.210 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.210 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.210 INFO:tasks.workunit.client.0.smithi100.stdout: "num_elections": { 2024-04-07T16:16:16.210 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3 2024-04-07T16:16:16.210 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.210 INFO:tasks.workunit.client.0.smithi100.stdout: "num_sessions": { 2024-04-07T16:16:16.210 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 12 2024-04-07T16:16:16.210 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.210 INFO:tasks.workunit.client.0.smithi100.stdout: "session_add": { 2024-04-07T16:16:16.210 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5 2024-04-07T16:16:16.210 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.210 INFO:tasks.workunit.client.0.smithi100.stdout: "session_rm": { 2024-04-07T16:16:16.210 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 122 2024-04-07T16:16:16.211 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.211 INFO:tasks.workunit.client.0.smithi100.stdout: "session_trim": { 2024-04-07T16:16:16.211 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.211 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:16.211 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.211 INFO:tasks.workunit.client.0.smithi100.stdout: "paxos": { 2024-04-07T16:16:16.211 INFO:tasks.workunit.client.0.smithi100.stdout: "accept_timeout": { 2024-04-07T16:16:16.211 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.211 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.211 INFO:tasks.workunit.client.0.smithi100.stdout: "begin": { 2024-04-07T16:16:16.211 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1106 2024-04-07T16:16:16.211 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.211 INFO:tasks.workunit.client.0.smithi100.stdout: "begin_bytes": { 2024-04-07T16:16:16.211 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1106, 2024-04-07T16:16:16.212 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4574327 2024-04-07T16:16:16.212 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.212 INFO:tasks.workunit.client.0.smithi100.stdout: "begin_keys": { 2024-04-07T16:16:16.212 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.212 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.212 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.212 INFO:tasks.workunit.client.0.smithi100.stdout: "begin_latency": { 2024-04-07T16:16:16.212 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1106, 2024-04-07T16:16:16.212 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 423993548 2024-04-07T16:16:16.212 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.212 INFO:tasks.workunit.client.0.smithi100.stdout: "collect": { 2024-04-07T16:16:16.212 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1 2024-04-07T16:16:16.212 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.212 INFO:tasks.workunit.client.0.smithi100.stdout: "collect_bytes": { 2024-04-07T16:16:16.213 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1, 2024-04-07T16:16:16.213 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 73 2024-04-07T16:16:16.213 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.213 INFO:tasks.workunit.client.0.smithi100.stdout: "collect_keys": { 2024-04-07T16:16:16.213 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1, 2024-04-07T16:16:16.213 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1 2024-04-07T16:16:16.213 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.213 INFO:tasks.workunit.client.0.smithi100.stdout: "collect_latency": { 2024-04-07T16:16:16.213 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1, 2024-04-07T16:16:16.213 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.213 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.213 INFO:tasks.workunit.client.0.smithi100.stdout: "collect_timeout": { 2024-04-07T16:16:16.213 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.213 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.214 INFO:tasks.workunit.client.0.smithi100.stdout: "collect_uncommitted": { 2024-04-07T16:16:16.214 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.214 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.214 INFO:tasks.workunit.client.0.smithi100.stdout: "commit": { 2024-04-07T16:16:16.214 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1107 2024-04-07T16:16:16.214 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.214 INFO:tasks.workunit.client.0.smithi100.stdout: "commit_bytes": { 2024-04-07T16:16:16.214 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.214 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.214 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.214 INFO:tasks.workunit.client.0.smithi100.stdout: "commit_keys": { 2024-04-07T16:16:16.214 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.214 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.214 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.215 INFO:tasks.workunit.client.0.smithi100.stdout: "commit_latency": { 2024-04-07T16:16:16.215 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.215 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.215 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.215 INFO:tasks.workunit.client.0.smithi100.stdout: "lease_ack_timeout": { 2024-04-07T16:16:16.215 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.215 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.215 INFO:tasks.workunit.client.0.smithi100.stdout: "lease_timeout": { 2024-04-07T16:16:16.215 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.215 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.215 INFO:tasks.workunit.client.0.smithi100.stdout: "new_pn": { 2024-04-07T16:16:16.215 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.215 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.215 INFO:tasks.workunit.client.0.smithi100.stdout: "new_pn_latency": { 2024-04-07T16:16:16.215 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.216 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.216 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.216 INFO:tasks.workunit.client.0.smithi100.stdout: "refresh": { 2024-04-07T16:16:16.216 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1107 2024-04-07T16:16:16.216 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.216 INFO:tasks.workunit.client.0.smithi100.stdout: "refresh_latency": { 2024-04-07T16:16:16.216 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1107, 2024-04-07T16:16:16.216 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4887928258 2024-04-07T16:16:16.216 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.216 INFO:tasks.workunit.client.0.smithi100.stdout: "restart": { 2024-04-07T16:16:16.216 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 9 2024-04-07T16:16:16.216 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.216 INFO:tasks.workunit.client.0.smithi100.stdout: "share_state": { 2024-04-07T16:16:16.216 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.216 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.217 INFO:tasks.workunit.client.0.smithi100.stdout: "share_state_bytes": { 2024-04-07T16:16:16.217 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.217 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.217 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.217 INFO:tasks.workunit.client.0.smithi100.stdout: "share_state_keys": { 2024-04-07T16:16:16.217 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.217 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.217 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.217 INFO:tasks.workunit.client.0.smithi100.stdout: "start_leader": { 2024-04-07T16:16:16.217 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.217 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.217 INFO:tasks.workunit.client.0.smithi100.stdout: "start_peon": { 2024-04-07T16:16:16.217 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1 2024-04-07T16:16:16.217 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.218 INFO:tasks.workunit.client.0.smithi100.stdout: "store_state": { 2024-04-07T16:16:16.218 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1107 2024-04-07T16:16:16.218 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.218 INFO:tasks.workunit.client.0.smithi100.stdout: "store_state_bytes": { 2024-04-07T16:16:16.218 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1107, 2024-04-07T16:16:16.218 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 8988722 2024-04-07T16:16:16.218 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.218 INFO:tasks.workunit.client.0.smithi100.stdout: "store_state_keys": { 2024-04-07T16:16:16.218 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1107, 2024-04-07T16:16:16.218 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 7870 2024-04-07T16:16:16.218 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.218 INFO:tasks.workunit.client.0.smithi100.stdout: "store_state_latency": { 2024-04-07T16:16:16.218 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1107, 2024-04-07T16:16:16.218 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 755988120 2024-04-07T16:16:16.218 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:16.219 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.219 INFO:tasks.workunit.client.0.smithi100.stdout: "prioritycache": { 2024-04-07T16:16:16.219 INFO:tasks.workunit.client.0.smithi100.stdout: "cache_bytes": { 2024-04-07T16:16:16.219 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1020054731 2024-04-07T16:16:16.219 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.219 INFO:tasks.workunit.client.0.smithi100.stdout: "heap_bytes": { 2024-04-07T16:16:16.219 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 37568512 2024-04-07T16:16:16.219 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.219 INFO:tasks.workunit.client.0.smithi100.stdout: "mapped_bytes": { 2024-04-07T16:16:16.219 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 31899648 2024-04-07T16:16:16.219 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.219 INFO:tasks.workunit.client.0.smithi100.stdout: "target_bytes": { 2024-04-07T16:16:16.219 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2147483648 2024-04-07T16:16:16.219 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.219 INFO:tasks.workunit.client.0.smithi100.stdout: "unmapped_bytes": { 2024-04-07T16:16:16.220 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5668864 2024-04-07T16:16:16.220 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:16.220 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.220 INFO:tasks.workunit.client.0.smithi100.stdout: "prioritycache:full": { 2024-04-07T16:16:16.220 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T16:16:16.220 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 348127232 2024-04-07T16:16:16.220 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.220 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T16:16:16.220 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.220 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.220 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T16:16:16.220 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.220 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.220 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T16:16:16.220 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 277021602 2024-04-07T16:16:16.220 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.221 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T16:16:16.221 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 270681 2024-04-07T16:16:16.221 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.221 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T16:16:16.221 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.221 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.221 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T16:16:16.221 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.221 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.221 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T16:16:16.221 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.221 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.221 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T16:16:16.221 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.221 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.222 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T16:16:16.222 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.222 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.222 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T16:16:16.222 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.222 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.222 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T16:16:16.222 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.222 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.222 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T16:16:16.222 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.222 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.222 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T16:16:16.222 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 70834949 2024-04-07T16:16:16.223 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:16.223 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.223 INFO:tasks.workunit.client.0.smithi100.stdout: "prioritycache:inc": { 2024-04-07T16:16:16.223 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T16:16:16.223 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 348127232 2024-04-07T16:16:16.223 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.223 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T16:16:16.223 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.223 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.223 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T16:16:16.223 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.223 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.223 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T16:16:16.223 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 277021602 2024-04-07T16:16:16.223 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.224 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T16:16:16.224 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 41409 2024-04-07T16:16:16.224 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.224 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T16:16:16.224 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.224 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.224 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T16:16:16.224 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.224 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.224 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T16:16:16.224 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.224 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.224 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T16:16:16.224 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.224 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.224 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T16:16:16.225 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.225 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.225 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T16:16:16.225 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.225 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.225 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T16:16:16.225 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.225 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.225 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T16:16:16.225 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.225 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.225 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T16:16:16.225 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 71064221 2024-04-07T16:16:16.225 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:16.225 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.226 INFO:tasks.workunit.client.0.smithi100.stdout: "prioritycache:kv": { 2024-04-07T16:16:16.226 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T16:16:16.226 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 322961408 2024-04-07T16:16:16.226 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.226 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T16:16:16.226 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 67108864 2024-04-07T16:16:16.226 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.226 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T16:16:16.226 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.226 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.226 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T16:16:16.226 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 184681068 2024-04-07T16:16:16.226 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.226 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T16:16:16.226 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.227 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.227 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T16:16:16.227 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.227 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.227 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T16:16:16.227 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.227 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.227 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T16:16:16.227 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.227 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.227 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T16:16:16.227 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.227 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.227 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T16:16:16.227 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.228 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.228 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T16:16:16.228 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.228 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.228 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T16:16:16.228 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.228 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.228 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T16:16:16.228 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.228 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.228 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T16:16:16.228 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 71171476 2024-04-07T16:16:16.228 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:16.229 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.229 INFO:tasks.workunit.client.0.smithi100.stdout: "rocksdb": { 2024-04-07T16:16:16.229 INFO:tasks.workunit.client.0.smithi100.stdout: "compact": { 2024-04-07T16:16:16.229 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.229 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.229 INFO:tasks.workunit.client.0.smithi100.stdout: "compact_queue_len": { 2024-04-07T16:16:16.229 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.229 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.229 INFO:tasks.workunit.client.0.smithi100.stdout: "compact_queue_merge": { 2024-04-07T16:16:16.229 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.229 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.229 INFO:tasks.workunit.client.0.smithi100.stdout: "compact_range": { 2024-04-07T16:16:16.229 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5 2024-04-07T16:16:16.230 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.230 INFO:tasks.workunit.client.0.smithi100.stdout: "get_latency": { 2024-04-07T16:16:16.230 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 47005, 2024-04-07T16:16:16.230 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 311784664 2024-04-07T16:16:16.230 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.230 INFO:tasks.workunit.client.0.smithi100.stdout: "rocksdb_write_delay_time": { 2024-04-07T16:16:16.230 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.230 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.230 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.230 INFO:tasks.workunit.client.0.smithi100.stdout: "rocksdb_write_memtable_time": { 2024-04-07T16:16:16.230 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.230 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.230 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.230 INFO:tasks.workunit.client.0.smithi100.stdout: "rocksdb_write_pre_and_post_time": { 2024-04-07T16:16:16.231 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.231 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.231 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.231 INFO:tasks.workunit.client.0.smithi100.stdout: "rocksdb_write_wal_time": { 2024-04-07T16:16:16.231 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.231 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.231 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.231 INFO:tasks.workunit.client.0.smithi100.stdout: "submit_latency": { 2024-04-07T16:16:16.231 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.231 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.231 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.231 INFO:tasks.workunit.client.0.smithi100.stdout: "submit_sync_latency": { 2024-04-07T16:16:16.231 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 2360, 2024-04-07T16:16:16.231 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1279516086 2024-04-07T16:16:16.232 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:16.232 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:16.232 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.232 INFO:tasks.workunit.client.0.smithi100.stdout: "mon.a52fc0d8acb515431879057649ff294cfe14c620": { 2024-04-07T16:16:16.232 INFO:tasks.workunit.client.0.smithi100.stdout: "mon": { 2024-04-07T16:16:16.232 INFO:tasks.workunit.client.0.smithi100.stdout: "election_call": { 2024-04-07T16:16:16.232 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2 2024-04-07T16:16:16.232 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.232 INFO:tasks.workunit.client.0.smithi100.stdout: "election_lose": { 2024-04-07T16:16:16.232 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2 2024-04-07T16:16:16.232 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.232 INFO:tasks.workunit.client.0.smithi100.stdout: "election_win": { 2024-04-07T16:16:16.232 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.232 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.232 INFO:tasks.workunit.client.0.smithi100.stdout: "num_elections": { 2024-04-07T16:16:16.233 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 7 2024-04-07T16:16:16.233 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.233 INFO:tasks.workunit.client.0.smithi100.stdout: "num_sessions": { 2024-04-07T16:16:16.233 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 11 2024-04-07T16:16:16.233 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.233 INFO:tasks.workunit.client.0.smithi100.stdout: "session_add": { 2024-04-07T16:16:16.233 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4 2024-04-07T16:16:16.233 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.233 INFO:tasks.workunit.client.0.smithi100.stdout: "session_rm": { 2024-04-07T16:16:16.233 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 56 2024-04-07T16:16:16.233 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.233 INFO:tasks.workunit.client.0.smithi100.stdout: "session_trim": { 2024-04-07T16:16:16.233 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.233 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:16.234 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.234 INFO:tasks.workunit.client.0.smithi100.stdout: "paxos": { 2024-04-07T16:16:16.234 INFO:tasks.workunit.client.0.smithi100.stdout: "accept_timeout": { 2024-04-07T16:16:16.234 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.234 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.234 INFO:tasks.workunit.client.0.smithi100.stdout: "begin": { 2024-04-07T16:16:16.234 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1113 2024-04-07T16:16:16.234 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.234 INFO:tasks.workunit.client.0.smithi100.stdout: "begin_bytes": { 2024-04-07T16:16:16.234 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1113, 2024-04-07T16:16:16.234 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4599996 2024-04-07T16:16:16.234 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.234 INFO:tasks.workunit.client.0.smithi100.stdout: "begin_keys": { 2024-04-07T16:16:16.234 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.235 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.235 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.235 INFO:tasks.workunit.client.0.smithi100.stdout: "begin_latency": { 2024-04-07T16:16:16.235 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1113, 2024-04-07T16:16:16.235 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 319995845 2024-04-07T16:16:16.235 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.235 INFO:tasks.workunit.client.0.smithi100.stdout: "collect": { 2024-04-07T16:16:16.235 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2 2024-04-07T16:16:16.235 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.235 INFO:tasks.workunit.client.0.smithi100.stdout: "collect_bytes": { 2024-04-07T16:16:16.235 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 2, 2024-04-07T16:16:16.235 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 146 2024-04-07T16:16:16.235 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.236 INFO:tasks.workunit.client.0.smithi100.stdout: "collect_keys": { 2024-04-07T16:16:16.236 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 2, 2024-04-07T16:16:16.236 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2 2024-04-07T16:16:16.236 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.236 INFO:tasks.workunit.client.0.smithi100.stdout: "collect_latency": { 2024-04-07T16:16:16.236 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 2, 2024-04-07T16:16:16.236 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3999939 2024-04-07T16:16:16.236 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.236 INFO:tasks.workunit.client.0.smithi100.stdout: "collect_timeout": { 2024-04-07T16:16:16.236 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.236 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.236 INFO:tasks.workunit.client.0.smithi100.stdout: "collect_uncommitted": { 2024-04-07T16:16:16.236 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.236 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.236 INFO:tasks.workunit.client.0.smithi100.stdout: "commit": { 2024-04-07T16:16:16.237 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1114 2024-04-07T16:16:16.237 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.237 INFO:tasks.workunit.client.0.smithi100.stdout: "commit_bytes": { 2024-04-07T16:16:16.237 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.237 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.237 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.237 INFO:tasks.workunit.client.0.smithi100.stdout: "commit_keys": { 2024-04-07T16:16:16.237 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.237 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.237 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.237 INFO:tasks.workunit.client.0.smithi100.stdout: "commit_latency": { 2024-04-07T16:16:16.237 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.237 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.237 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.237 INFO:tasks.workunit.client.0.smithi100.stdout: "lease_ack_timeout": { 2024-04-07T16:16:16.238 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.238 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.238 INFO:tasks.workunit.client.0.smithi100.stdout: "lease_timeout": { 2024-04-07T16:16:16.238 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.238 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.238 INFO:tasks.workunit.client.0.smithi100.stdout: "new_pn": { 2024-04-07T16:16:16.238 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.238 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.238 INFO:tasks.workunit.client.0.smithi100.stdout: "new_pn_latency": { 2024-04-07T16:16:16.238 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.238 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.238 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.238 INFO:tasks.workunit.client.0.smithi100.stdout: "refresh": { 2024-04-07T16:16:16.238 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1114 2024-04-07T16:16:16.238 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.239 INFO:tasks.workunit.client.0.smithi100.stdout: "refresh_latency": { 2024-04-07T16:16:16.239 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1114, 2024-04-07T16:16:16.239 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4299945922 2024-04-07T16:16:16.239 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.239 INFO:tasks.workunit.client.0.smithi100.stdout: "restart": { 2024-04-07T16:16:16.239 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 13 2024-04-07T16:16:16.239 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.239 INFO:tasks.workunit.client.0.smithi100.stdout: "share_state": { 2024-04-07T16:16:16.239 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.239 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.239 INFO:tasks.workunit.client.0.smithi100.stdout: "share_state_bytes": { 2024-04-07T16:16:16.239 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.239 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.239 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.239 INFO:tasks.workunit.client.0.smithi100.stdout: "share_state_keys": { 2024-04-07T16:16:16.240 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.240 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.240 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.240 INFO:tasks.workunit.client.0.smithi100.stdout: "start_leader": { 2024-04-07T16:16:16.240 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.240 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.240 INFO:tasks.workunit.client.0.smithi100.stdout: "start_peon": { 2024-04-07T16:16:16.240 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2 2024-04-07T16:16:16.240 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.240 INFO:tasks.workunit.client.0.smithi100.stdout: "store_state": { 2024-04-07T16:16:16.240 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1114 2024-04-07T16:16:16.240 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.240 INFO:tasks.workunit.client.0.smithi100.stdout: "store_state_bytes": { 2024-04-07T16:16:16.240 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1114, 2024-04-07T16:16:16.240 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 8990199 2024-04-07T16:16:16.240 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.241 INFO:tasks.workunit.client.0.smithi100.stdout: "store_state_keys": { 2024-04-07T16:16:16.241 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1114, 2024-04-07T16:16:16.241 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 7881 2024-04-07T16:16:16.241 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.241 INFO:tasks.workunit.client.0.smithi100.stdout: "store_state_latency": { 2024-04-07T16:16:16.241 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1114, 2024-04-07T16:16:16.241 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 419994480 2024-04-07T16:16:16.241 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:16.241 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.241 INFO:tasks.workunit.client.0.smithi100.stdout: "prioritycache": { 2024-04-07T16:16:16.241 INFO:tasks.workunit.client.0.smithi100.stdout: "cache_bytes": { 2024-04-07T16:16:16.241 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1020054731 2024-04-07T16:16:16.241 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.241 INFO:tasks.workunit.client.0.smithi100.stdout: "heap_bytes": { 2024-04-07T16:16:16.241 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 38617088 2024-04-07T16:16:16.242 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.242 INFO:tasks.workunit.client.0.smithi100.stdout: "mapped_bytes": { 2024-04-07T16:16:16.242 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 32612352 2024-04-07T16:16:16.242 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.242 INFO:tasks.workunit.client.0.smithi100.stdout: "target_bytes": { 2024-04-07T16:16:16.242 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2147483648 2024-04-07T16:16:16.242 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.242 INFO:tasks.workunit.client.0.smithi100.stdout: "unmapped_bytes": { 2024-04-07T16:16:16.242 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 6004736 2024-04-07T16:16:16.242 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:16.242 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.242 INFO:tasks.workunit.client.0.smithi100.stdout: "prioritycache:full": { 2024-04-07T16:16:16.242 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T16:16:16.242 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 348127232 2024-04-07T16:16:16.242 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.243 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T16:16:16.243 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.243 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.243 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T16:16:16.243 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.243 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.243 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T16:16:16.243 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 277021602 2024-04-07T16:16:16.243 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.243 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T16:16:16.243 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 270681 2024-04-07T16:16:16.243 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.243 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T16:16:16.243 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.243 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.244 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T16:16:16.244 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.244 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.244 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T16:16:16.244 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.244 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.244 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T16:16:16.244 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.244 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.244 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T16:16:16.244 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.244 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.244 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T16:16:16.244 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.245 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.245 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T16:16:16.245 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.245 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.245 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T16:16:16.245 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.245 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.245 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T16:16:16.245 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 70834949 2024-04-07T16:16:16.245 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:16.245 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.245 INFO:tasks.workunit.client.0.smithi100.stdout: "prioritycache:inc": { 2024-04-07T16:16:16.245 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T16:16:16.245 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 348127232 2024-04-07T16:16:16.245 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.246 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T16:16:16.246 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.246 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.246 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T16:16:16.246 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.246 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.246 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T16:16:16.246 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 277021602 2024-04-07T16:16:16.246 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.246 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T16:16:16.246 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 41409 2024-04-07T16:16:16.246 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.246 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T16:16:16.247 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.247 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.247 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T16:16:16.247 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.247 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.247 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T16:16:16.247 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.247 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.247 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T16:16:16.247 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.247 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.247 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T16:16:16.247 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.247 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.248 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T16:16:16.248 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.248 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.248 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T16:16:16.248 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.248 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.248 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T16:16:16.248 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.248 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.248 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T16:16:16.248 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 71064221 2024-04-07T16:16:16.248 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:16.248 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.248 INFO:tasks.workunit.client.0.smithi100.stdout: "prioritycache:kv": { 2024-04-07T16:16:16.249 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T16:16:16.249 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 322961408 2024-04-07T16:16:16.249 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.249 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T16:16:16.249 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 67108864 2024-04-07T16:16:16.249 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.249 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T16:16:16.249 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.249 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.249 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T16:16:16.249 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 184681068 2024-04-07T16:16:16.249 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.249 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T16:16:16.249 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.249 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.249 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T16:16:16.250 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.250 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.250 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T16:16:16.250 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.250 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.250 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T16:16:16.250 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.250 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.250 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T16:16:16.250 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.250 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.250 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T16:16:16.250 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.250 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.251 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T16:16:16.251 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.251 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.251 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T16:16:16.251 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.251 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.251 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T16:16:16.251 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.251 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.251 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T16:16:16.251 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 71171476 2024-04-07T16:16:16.251 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:16.251 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.251 INFO:tasks.workunit.client.0.smithi100.stdout: "rocksdb": { 2024-04-07T16:16:16.251 INFO:tasks.workunit.client.0.smithi100.stdout: "compact": { 2024-04-07T16:16:16.252 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.252 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.252 INFO:tasks.workunit.client.0.smithi100.stdout: "compact_queue_len": { 2024-04-07T16:16:16.252 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.252 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.252 INFO:tasks.workunit.client.0.smithi100.stdout: "compact_queue_merge": { 2024-04-07T16:16:16.252 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.252 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.252 INFO:tasks.workunit.client.0.smithi100.stdout: "compact_range": { 2024-04-07T16:16:16.252 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5 2024-04-07T16:16:16.252 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.252 INFO:tasks.workunit.client.0.smithi100.stdout: "get_latency": { 2024-04-07T16:16:16.252 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 47722, 2024-04-07T16:16:16.252 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 273846774 2024-04-07T16:16:16.252 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.253 INFO:tasks.workunit.client.0.smithi100.stdout: "rocksdb_write_delay_time": { 2024-04-07T16:16:16.253 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.253 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.253 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.253 INFO:tasks.workunit.client.0.smithi100.stdout: "rocksdb_write_memtable_time": { 2024-04-07T16:16:16.253 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.253 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.253 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.253 INFO:tasks.workunit.client.0.smithi100.stdout: "rocksdb_write_pre_and_post_time": { 2024-04-07T16:16:16.253 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.253 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.253 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.253 INFO:tasks.workunit.client.0.smithi100.stdout: "rocksdb_write_wal_time": { 2024-04-07T16:16:16.253 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.254 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.254 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.254 INFO:tasks.workunit.client.0.smithi100.stdout: "submit_latency": { 2024-04-07T16:16:16.254 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.254 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.254 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.254 INFO:tasks.workunit.client.0.smithi100.stdout: "submit_sync_latency": { 2024-04-07T16:16:16.254 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 2414, 2024-04-07T16:16:16.254 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 851811048 2024-04-07T16:16:16.254 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:16.254 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:16.254 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.254 INFO:tasks.workunit.client.0.smithi100.stdout: "mon.f47a4ff15a45950a63160265564a123fc5da5e5e": { 2024-04-07T16:16:16.254 INFO:tasks.workunit.client.0.smithi100.stdout: "mon": { 2024-04-07T16:16:16.255 INFO:tasks.workunit.client.0.smithi100.stdout: "election_call": { 2024-04-07T16:16:16.255 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2 2024-04-07T16:16:16.255 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.255 INFO:tasks.workunit.client.0.smithi100.stdout: "election_lose": { 2024-04-07T16:16:16.255 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.255 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.255 INFO:tasks.workunit.client.0.smithi100.stdout: "election_win": { 2024-04-07T16:16:16.255 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3 2024-04-07T16:16:16.255 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.255 INFO:tasks.workunit.client.0.smithi100.stdout: "num_elections": { 2024-04-07T16:16:16.255 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 9 2024-04-07T16:16:16.255 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.255 INFO:tasks.workunit.client.0.smithi100.stdout: "num_sessions": { 2024-04-07T16:16:16.255 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 7 2024-04-07T16:16:16.256 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.256 INFO:tasks.workunit.client.0.smithi100.stdout: "session_add": { 2024-04-07T16:16:16.256 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3 2024-04-07T16:16:16.256 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.256 INFO:tasks.workunit.client.0.smithi100.stdout: "session_rm": { 2024-04-07T16:16:16.256 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 193 2024-04-07T16:16:16.256 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.256 INFO:tasks.workunit.client.0.smithi100.stdout: "session_trim": { 2024-04-07T16:16:16.256 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.256 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:16.256 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.256 INFO:tasks.workunit.client.0.smithi100.stdout: "paxos": { 2024-04-07T16:16:16.256 INFO:tasks.workunit.client.0.smithi100.stdout: "accept_timeout": { 2024-04-07T16:16:16.256 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.256 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.257 INFO:tasks.workunit.client.0.smithi100.stdout: "begin": { 2024-04-07T16:16:16.257 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1269 2024-04-07T16:16:16.257 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.257 INFO:tasks.workunit.client.0.smithi100.stdout: "begin_bytes": { 2024-04-07T16:16:16.257 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1269, 2024-04-07T16:16:16.257 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5694284 2024-04-07T16:16:16.257 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.257 INFO:tasks.workunit.client.0.smithi100.stdout: "begin_keys": { 2024-04-07T16:16:16.257 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1269, 2024-04-07T16:16:16.257 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3807 2024-04-07T16:16:16.257 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.257 INFO:tasks.workunit.client.0.smithi100.stdout: "begin_latency": { 2024-04-07T16:16:16.257 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1269, 2024-04-07T16:16:16.257 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 403991839 2024-04-07T16:16:16.257 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.258 INFO:tasks.workunit.client.0.smithi100.stdout: "collect": { 2024-04-07T16:16:16.258 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.258 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.258 INFO:tasks.workunit.client.0.smithi100.stdout: "collect_bytes": { 2024-04-07T16:16:16.258 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.258 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.258 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.258 INFO:tasks.workunit.client.0.smithi100.stdout: "collect_keys": { 2024-04-07T16:16:16.258 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.258 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.258 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.258 INFO:tasks.workunit.client.0.smithi100.stdout: "collect_latency": { 2024-04-07T16:16:16.258 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.258 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.258 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.259 INFO:tasks.workunit.client.0.smithi100.stdout: "collect_timeout": { 2024-04-07T16:16:16.259 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.259 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.259 INFO:tasks.workunit.client.0.smithi100.stdout: "collect_uncommitted": { 2024-04-07T16:16:16.259 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.259 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.259 INFO:tasks.workunit.client.0.smithi100.stdout: "commit": { 2024-04-07T16:16:16.259 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1269 2024-04-07T16:16:16.259 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.259 INFO:tasks.workunit.client.0.smithi100.stdout: "commit_bytes": { 2024-04-07T16:16:16.259 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1269, 2024-04-07T16:16:16.259 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5602549 2024-04-07T16:16:16.259 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.259 INFO:tasks.workunit.client.0.smithi100.stdout: "commit_keys": { 2024-04-07T16:16:16.259 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1269, 2024-04-07T16:16:16.260 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 7584 2024-04-07T16:16:16.260 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.260 INFO:tasks.workunit.client.0.smithi100.stdout: "commit_latency": { 2024-04-07T16:16:16.260 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1269, 2024-04-07T16:16:16.260 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 601015819 2024-04-07T16:16:16.260 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.260 INFO:tasks.workunit.client.0.smithi100.stdout: "lease_ack_timeout": { 2024-04-07T16:16:16.260 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.260 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.260 INFO:tasks.workunit.client.0.smithi100.stdout: "lease_timeout": { 2024-04-07T16:16:16.260 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.260 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.260 INFO:tasks.workunit.client.0.smithi100.stdout: "new_pn": { 2024-04-07T16:16:16.260 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2 2024-04-07T16:16:16.260 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.260 INFO:tasks.workunit.client.0.smithi100.stdout: "new_pn_latency": { 2024-04-07T16:16:16.260 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 2, 2024-04-07T16:16:16.261 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.261 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.261 INFO:tasks.workunit.client.0.smithi100.stdout: "refresh": { 2024-04-07T16:16:16.261 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1271 2024-04-07T16:16:16.261 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.261 INFO:tasks.workunit.client.0.smithi100.stdout: "refresh_latency": { 2024-04-07T16:16:16.261 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1271, 2024-04-07T16:16:16.261 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5811901860 2024-04-07T16:16:16.261 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.261 INFO:tasks.workunit.client.0.smithi100.stdout: "restart": { 2024-04-07T16:16:16.261 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 12 2024-04-07T16:16:16.261 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.261 INFO:tasks.workunit.client.0.smithi100.stdout: "share_state": { 2024-04-07T16:16:16.261 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2 2024-04-07T16:16:16.261 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.261 INFO:tasks.workunit.client.0.smithi100.stdout: "share_state_bytes": { 2024-04-07T16:16:16.262 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 2, 2024-04-07T16:16:16.262 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 25218 2024-04-07T16:16:16.262 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.262 INFO:tasks.workunit.client.0.smithi100.stdout: "share_state_keys": { 2024-04-07T16:16:16.262 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 2, 2024-04-07T16:16:16.262 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 8 2024-04-07T16:16:16.262 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.262 INFO:tasks.workunit.client.0.smithi100.stdout: "start_leader": { 2024-04-07T16:16:16.262 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3 2024-04-07T16:16:16.262 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.262 INFO:tasks.workunit.client.0.smithi100.stdout: "start_peon": { 2024-04-07T16:16:16.262 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.262 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.262 INFO:tasks.workunit.client.0.smithi100.stdout: "store_state": { 2024-04-07T16:16:16.263 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.263 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.263 INFO:tasks.workunit.client.0.smithi100.stdout: "store_state_bytes": { 2024-04-07T16:16:16.263 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.263 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.263 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.263 INFO:tasks.workunit.client.0.smithi100.stdout: "store_state_keys": { 2024-04-07T16:16:16.263 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.263 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.263 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.263 INFO:tasks.workunit.client.0.smithi100.stdout: "store_state_latency": { 2024-04-07T16:16:16.263 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.263 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.263 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:16.264 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.264 INFO:tasks.workunit.client.0.smithi100.stdout: "prioritycache": { 2024-04-07T16:16:16.264 INFO:tasks.workunit.client.0.smithi100.stdout: "cache_bytes": { 2024-04-07T16:16:16.264 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1020054731 2024-04-07T16:16:16.264 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.264 INFO:tasks.workunit.client.0.smithi100.stdout: "heap_bytes": { 2024-04-07T16:16:16.264 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 44810240 2024-04-07T16:16:16.264 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.264 INFO:tasks.workunit.client.0.smithi100.stdout: "mapped_bytes": { 2024-04-07T16:16:16.264 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 41385984 2024-04-07T16:16:16.264 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.264 INFO:tasks.workunit.client.0.smithi100.stdout: "target_bytes": { 2024-04-07T16:16:16.264 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2147483648 2024-04-07T16:16:16.264 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.264 INFO:tasks.workunit.client.0.smithi100.stdout: "unmapped_bytes": { 2024-04-07T16:16:16.265 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3424256 2024-04-07T16:16:16.265 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:16.265 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.265 INFO:tasks.workunit.client.0.smithi100.stdout: "prioritycache:full": { 2024-04-07T16:16:16.265 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T16:16:16.265 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 348127232 2024-04-07T16:16:16.265 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.265 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T16:16:16.265 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.265 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.265 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T16:16:16.265 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.265 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.265 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T16:16:16.265 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 277023025 2024-04-07T16:16:16.265 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.266 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T16:16:16.266 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 270251 2024-04-07T16:16:16.266 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.266 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T16:16:16.266 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.266 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.266 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T16:16:16.266 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.266 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.266 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T16:16:16.266 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.266 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.266 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T16:16:16.266 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.267 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.267 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T16:16:16.267 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.267 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.267 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T16:16:16.267 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.267 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.267 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T16:16:16.267 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.267 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.267 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T16:16:16.267 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.267 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.267 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T16:16:16.267 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 70833956 2024-04-07T16:16:16.268 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:16.268 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.268 INFO:tasks.workunit.client.0.smithi100.stdout: "prioritycache:inc": { 2024-04-07T16:16:16.268 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T16:16:16.268 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 348127232 2024-04-07T16:16:16.268 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.268 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T16:16:16.268 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.268 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.268 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T16:16:16.268 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.268 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.268 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T16:16:16.268 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 277023025 2024-04-07T16:16:16.269 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.269 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T16:16:16.269 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 38045 2024-04-07T16:16:16.269 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.269 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T16:16:16.269 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.269 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.269 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T16:16:16.269 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.269 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.269 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T16:16:16.269 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.269 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.269 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T16:16:16.269 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.270 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.270 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T16:16:16.270 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.270 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.270 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T16:16:16.270 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.270 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.270 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T16:16:16.270 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.270 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.270 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T16:16:16.270 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.270 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.270 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T16:16:16.270 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 71066162 2024-04-07T16:16:16.271 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:16.271 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.271 INFO:tasks.workunit.client.0.smithi100.stdout: "prioritycache:kv": { 2024-04-07T16:16:16.271 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T16:16:16.271 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 322961408 2024-04-07T16:16:16.271 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.271 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T16:16:16.271 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 67108864 2024-04-07T16:16:16.271 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.271 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T16:16:16.271 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.271 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.271 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T16:16:16.271 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 184682016 2024-04-07T16:16:16.272 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.272 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T16:16:16.272 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.272 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.272 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T16:16:16.272 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.272 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.272 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T16:16:16.272 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.272 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.272 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T16:16:16.272 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.272 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.272 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T16:16:16.272 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.273 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.273 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T16:16:16.273 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.273 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.273 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T16:16:16.273 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.273 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.273 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T16:16:16.273 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.273 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.273 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T16:16:16.273 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.273 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.273 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T16:16:16.273 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 71170528 2024-04-07T16:16:16.274 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:16.274 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.274 INFO:tasks.workunit.client.0.smithi100.stdout: "rocksdb": { 2024-04-07T16:16:16.274 INFO:tasks.workunit.client.0.smithi100.stdout: "compact": { 2024-04-07T16:16:16.274 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.274 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.274 INFO:tasks.workunit.client.0.smithi100.stdout: "compact_queue_len": { 2024-04-07T16:16:16.274 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.274 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.274 INFO:tasks.workunit.client.0.smithi100.stdout: "compact_queue_merge": { 2024-04-07T16:16:16.274 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.274 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.274 INFO:tasks.workunit.client.0.smithi100.stdout: "compact_range": { 2024-04-07T16:16:16.274 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5 2024-04-07T16:16:16.274 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.275 INFO:tasks.workunit.client.0.smithi100.stdout: "get_latency": { 2024-04-07T16:16:16.275 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 56343, 2024-04-07T16:16:16.275 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 491963746 2024-04-07T16:16:16.275 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.275 INFO:tasks.workunit.client.0.smithi100.stdout: "rocksdb_write_delay_time": { 2024-04-07T16:16:16.275 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.275 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.275 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.275 INFO:tasks.workunit.client.0.smithi100.stdout: "rocksdb_write_memtable_time": { 2024-04-07T16:16:16.275 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.275 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.275 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.275 INFO:tasks.workunit.client.0.smithi100.stdout: "rocksdb_write_pre_and_post_time": { 2024-04-07T16:16:16.275 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.275 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.276 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.276 INFO:tasks.workunit.client.0.smithi100.stdout: "rocksdb_write_wal_time": { 2024-04-07T16:16:16.276 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.276 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.276 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.276 INFO:tasks.workunit.client.0.smithi100.stdout: "submit_latency": { 2024-04-07T16:16:16.276 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.276 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.276 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.276 INFO:tasks.workunit.client.0.smithi100.stdout: "submit_sync_latency": { 2024-04-07T16:16:16.276 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 2757, 2024-04-07T16:16:16.276 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1031496647 2024-04-07T16:16:16.276 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:16.276 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:16.276 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.277 INFO:tasks.workunit.client.0.smithi100.stdout: "osd.0": { 2024-04-07T16:16:16.277 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs": { 2024-04-07T16:16:16.277 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_slow_fallback": { 2024-04-07T16:16:16.277 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.277 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.277 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_slow_size_fallback": { 2024-04-07T16:16:16.277 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.277 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.277 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit_db": { 2024-04-07T16:16:16.277 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 65536 2024-04-07T16:16:16.277 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.277 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit_main": { 2024-04-07T16:16:16.277 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.277 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.277 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit_wal": { 2024-04-07T16:16:16.278 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.278 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.278 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_written_slow": { 2024-04-07T16:16:16.278 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.278 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.278 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_written_sst": { 2024-04-07T16:16:16.278 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 12288 2024-04-07T16:16:16.278 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.278 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_written_wal": { 2024-04-07T16:16:16.278 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2830336 2024-04-07T16:16:16.278 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.278 INFO:tasks.workunit.client.0.smithi100.stdout: "compact_lat": { 2024-04-07T16:16:16.278 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.278 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.279 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.279 INFO:tasks.workunit.client.0.smithi100.stdout: "compact_lock_lat": { 2024-04-07T16:16:16.279 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.279 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.279 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.279 INFO:tasks.workunit.client.0.smithi100.stdout: "db_total_bytes": { 2024-04-07T16:16:16.279 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 95995035648 2024-04-07T16:16:16.279 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.279 INFO:tasks.workunit.client.0.smithi100.stdout: "db_used_bytes": { 2024-04-07T16:16:16.279 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 27656192 2024-04-07T16:16:16.279 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.279 INFO:tasks.workunit.client.0.smithi100.stdout: "log_bytes": { 2024-04-07T16:16:16.279 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1802240 2024-04-07T16:16:16.279 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.279 INFO:tasks.workunit.client.0.smithi100.stdout: "logged_bytes": { 2024-04-07T16:16:16.280 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1662976 2024-04-07T16:16:16.280 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.280 INFO:tasks.workunit.client.0.smithi100.stdout: "max_bytes_db": { 2024-04-07T16:16:16.280 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 31850496 2024-04-07T16:16:16.280 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.280 INFO:tasks.workunit.client.0.smithi100.stdout: "max_bytes_slow": { 2024-04-07T16:16:16.280 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.280 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.280 INFO:tasks.workunit.client.0.smithi100.stdout: "max_bytes_wal": { 2024-04-07T16:16:16.280 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.280 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.280 INFO:tasks.workunit.client.0.smithi100.stdout: "num_files": { 2024-04-07T16:16:16.280 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 14 2024-04-07T16:16:16.280 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.280 INFO:tasks.workunit.client.0.smithi100.stdout: "read_bytes": { 2024-04-07T16:16:16.281 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 206272 2024-04-07T16:16:16.281 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.281 INFO:tasks.workunit.client.0.smithi100.stdout: "read_count": { 2024-04-07T16:16:16.281 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 23 2024-04-07T16:16:16.281 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.281 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes": { 2024-04-07T16:16:16.281 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1282048 2024-04-07T16:16:16.281 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.281 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes_db": { 2024-04-07T16:16:16.281 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1286144 2024-04-07T16:16:16.281 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.281 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes_slow": { 2024-04-07T16:16:16.281 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.281 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.281 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes_wal": { 2024-04-07T16:16:16.282 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.282 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.282 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_count": { 2024-04-07T16:16:16.282 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 15 2024-04-07T16:16:16.282 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.282 INFO:tasks.workunit.client.0.smithi100.stdout: "read_prefetch_bytes": { 2024-04-07T16:16:16.282 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4913 2024-04-07T16:16:16.282 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.282 INFO:tasks.workunit.client.0.smithi100.stdout: "read_prefetch_count": { 2024-04-07T16:16:16.282 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4 2024-04-07T16:16:16.282 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.282 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_buffer_bytes": { 2024-04-07T16:16:16.282 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5504 2024-04-07T16:16:16.282 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.282 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_buffer_count": { 2024-04-07T16:16:16.283 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 24 2024-04-07T16:16:16.283 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.283 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_bytes": { 2024-04-07T16:16:16.283 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5504 2024-04-07T16:16:16.283 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.283 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_count": { 2024-04-07T16:16:16.283 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 24 2024-04-07T16:16:16.283 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.283 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes": { 2024-04-07T16:16:16.283 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.283 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.283 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes_db": { 2024-04-07T16:16:16.283 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.284 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.284 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes_slow": { 2024-04-07T16:16:16.284 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.284 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.284 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes_wal": { 2024-04-07T16:16:16.284 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.284 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.284 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_count": { 2024-04-07T16:16:16.284 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.284 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.284 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_total_bytes": { 2024-04-07T16:16:16.284 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.284 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.284 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_used_bytes": { 2024-04-07T16:16:16.284 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.285 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.285 INFO:tasks.workunit.client.0.smithi100.stdout: "wal_total_bytes": { 2024-04-07T16:16:16.285 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.285 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.285 INFO:tasks.workunit.client.0.smithi100.stdout: "wal_used_bytes": { 2024-04-07T16:16:16.285 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.285 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.285 INFO:tasks.workunit.client.0.smithi100.stdout: "write_bytes": { 2024-04-07T16:16:16.285 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4571136 2024-04-07T16:16:16.285 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:16.285 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.285 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore": { 2024-04-07T16:16:16.285 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit": { 2024-04-07T16:16:16.285 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4096 2024-04-07T16:16:16.285 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.285 INFO:tasks.workunit.client.0.smithi100.stdout: "allocated": { 2024-04-07T16:16:16.285 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1318912 2024-04-07T16:16:16.286 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.286 INFO:tasks.workunit.client.0.smithi100.stdout: "clist_lat": { 2024-04-07T16:16:16.286 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 6, 2024-04-07T16:16:16.286 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 377454 2024-04-07T16:16:16.286 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.286 INFO:tasks.workunit.client.0.smithi100.stdout: "compress_lat": { 2024-04-07T16:16:16.286 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.286 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.286 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.286 INFO:tasks.workunit.client.0.smithi100.stdout: "compressed": { 2024-04-07T16:16:16.286 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.286 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.286 INFO:tasks.workunit.client.0.smithi100.stdout: "compressed_allocated": { 2024-04-07T16:16:16.286 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.287 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.287 INFO:tasks.workunit.client.0.smithi100.stdout: "compressed_original": { 2024-04-07T16:16:16.287 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.287 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.287 INFO:tasks.workunit.client.0.smithi100.stdout: "csum_lat": { 2024-04-07T16:16:16.287 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 25, 2024-04-07T16:16:16.287 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 111336 2024-04-07T16:16:16.287 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.287 INFO:tasks.workunit.client.0.smithi100.stdout: "decompress_lat": { 2024-04-07T16:16:16.287 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.287 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.287 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.287 INFO:tasks.workunit.client.0.smithi100.stdout: "fragmentation_micros": { 2024-04-07T16:16:16.287 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.287 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.287 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_commit_lat": { 2024-04-07T16:16:16.288 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 393, 2024-04-07T16:16:16.288 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 130576663 2024-04-07T16:16:16.288 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.288 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_final_lat": { 2024-04-07T16:16:16.288 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 376, 2024-04-07T16:16:16.288 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 28092600 2024-04-07T16:16:16.288 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.288 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_flush_lat": { 2024-04-07T16:16:16.288 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 393, 2024-04-07T16:16:16.288 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 711688 2024-04-07T16:16:16.288 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.288 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_sync_lat": { 2024-04-07T16:16:16.288 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 393, 2024-04-07T16:16:16.288 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 131288351 2024-04-07T16:16:16.289 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.289 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_get_keys_lat": { 2024-04-07T16:16:16.289 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.289 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.289 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.289 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_get_values_lat": { 2024-04-07T16:16:16.289 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1, 2024-04-07T16:16:16.289 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 53150 2024-04-07T16:16:16.289 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.289 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_lower_bound_lat": { 2024-04-07T16:16:16.289 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 6, 2024-04-07T16:16:16.289 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 32820 2024-04-07T16:16:16.289 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.289 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_next_lat": { 2024-04-07T16:16:16.289 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.290 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.290 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.290 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_seek_to_first_lat": { 2024-04-07T16:16:16.290 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.290 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.290 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.290 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_upper_bound_lat": { 2024-04-07T16:16:16.290 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 2, 2024-04-07T16:16:16.290 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 6225 2024-04-07T16:16:16.290 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.290 INFO:tasks.workunit.client.0.smithi100.stdout: "onode_hits": { 2024-04-07T16:16:16.290 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 858 2024-04-07T16:16:16.290 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.290 INFO:tasks.workunit.client.0.smithi100.stdout: "onode_misses": { 2024-04-07T16:16:16.290 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 231 2024-04-07T16:16:16.291 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.291 INFO:tasks.workunit.client.0.smithi100.stdout: "read_lat": { 2024-04-07T16:16:16.291 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 121, 2024-04-07T16:16:16.291 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3967663 2024-04-07T16:16:16.291 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.291 INFO:tasks.workunit.client.0.smithi100.stdout: "read_onode_meta_lat": { 2024-04-07T16:16:16.291 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 327, 2024-04-07T16:16:16.291 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 449756 2024-04-07T16:16:16.291 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.291 INFO:tasks.workunit.client.0.smithi100.stdout: "read_wait_aio_lat": { 2024-04-07T16:16:16.291 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 206, 2024-04-07T16:16:16.291 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3412921 2024-04-07T16:16:16.291 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.291 INFO:tasks.workunit.client.0.smithi100.stdout: "reads_with_retries": { 2024-04-07T16:16:16.291 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.292 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.292 INFO:tasks.workunit.client.0.smithi100.stdout: "remove_lat": { 2024-04-07T16:16:16.292 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 3, 2024-04-07T16:16:16.292 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 138272 2024-04-07T16:16:16.292 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.292 INFO:tasks.workunit.client.0.smithi100.stdout: "state_aio_wait_lat": { 2024-04-07T16:16:16.292 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 493, 2024-04-07T16:16:16.292 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 12016374 2024-04-07T16:16:16.292 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.292 INFO:tasks.workunit.client.0.smithi100.stdout: "state_deferred_aio_wait_lat": { 2024-04-07T16:16:16.292 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 125, 2024-04-07T16:16:16.292 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 9953550 2024-04-07T16:16:16.292 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.292 INFO:tasks.workunit.client.0.smithi100.stdout: "state_deferred_cleanup_lat": { 2024-04-07T16:16:16.292 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 125, 2024-04-07T16:16:16.292 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 770299669578 2024-04-07T16:16:16.292 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.293 INFO:tasks.workunit.client.0.smithi100.stdout: "state_deferred_queued_lat": { 2024-04-07T16:16:16.293 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 125, 2024-04-07T16:16:16.293 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 150872178514 2024-04-07T16:16:16.293 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.293 INFO:tasks.workunit.client.0.smithi100.stdout: "state_done_lat": { 2024-04-07T16:16:16.293 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 493, 2024-04-07T16:16:16.293 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 88578307500 2024-04-07T16:16:16.293 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.293 INFO:tasks.workunit.client.0.smithi100.stdout: "state_finishing_lat": { 2024-04-07T16:16:16.293 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 493, 2024-04-07T16:16:16.293 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 79813 2024-04-07T16:16:16.293 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.293 INFO:tasks.workunit.client.0.smithi100.stdout: "state_io_done_lat": { 2024-04-07T16:16:16.293 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 493, 2024-04-07T16:16:16.294 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 862909 2024-04-07T16:16:16.294 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.294 INFO:tasks.workunit.client.0.smithi100.stdout: "state_kv_commiting_lat": { 2024-04-07T16:16:16.294 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 493, 2024-04-07T16:16:16.294 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 439808267 2024-04-07T16:16:16.294 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.294 INFO:tasks.workunit.client.0.smithi100.stdout: "state_kv_done_lat": { 2024-04-07T16:16:16.294 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 493, 2024-04-07T16:16:16.294 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 307213 2024-04-07T16:16:16.294 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.294 INFO:tasks.workunit.client.0.smithi100.stdout: "state_kv_queued_lat": { 2024-04-07T16:16:16.294 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 493, 2024-04-07T16:16:16.294 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 139744494 2024-04-07T16:16:16.294 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.294 INFO:tasks.workunit.client.0.smithi100.stdout: "state_prepare_lat": { 2024-04-07T16:16:16.295 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 493, 2024-04-07T16:16:16.295 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 37698270 2024-04-07T16:16:16.295 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.295 INFO:tasks.workunit.client.0.smithi100.stdout: "stored": { 2024-04-07T16:16:16.295 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 900328 2024-04-07T16:16:16.295 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.295 INFO:tasks.workunit.client.0.smithi100.stdout: "truncate_lat": { 2024-04-07T16:16:16.295 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 59, 2024-04-07T16:16:16.295 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 497022 2024-04-07T16:16:16.295 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.295 INFO:tasks.workunit.client.0.smithi100.stdout: "txc_commit_lat": { 2024-04-07T16:16:16.295 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 493, 2024-04-07T16:16:16.295 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 630264562 2024-04-07T16:16:16.296 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.296 INFO:tasks.workunit.client.0.smithi100.stdout: "txc_submit_lat": { 2024-04-07T16:16:16.296 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 493, 2024-04-07T16:16:16.296 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 45477448 2024-04-07T16:16:16.296 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.296 INFO:tasks.workunit.client.0.smithi100.stdout: "txc_throttle_lat": { 2024-04-07T16:16:16.296 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 493, 2024-04-07T16:16:16.296 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5468903 2024-04-07T16:16:16.296 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:16.296 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.296 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache": { 2024-04-07T16:16:16.296 INFO:tasks.workunit.client.0.smithi100.stdout: "cache_bytes": { 2024-04-07T16:16:16.296 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1900954368 2024-04-07T16:16:16.296 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.296 INFO:tasks.workunit.client.0.smithi100.stdout: "heap_bytes": { 2024-04-07T16:16:16.297 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 45531136 2024-04-07T16:16:16.297 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.297 INFO:tasks.workunit.client.0.smithi100.stdout: "mapped_bytes": { 2024-04-07T16:16:16.297 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 44654592 2024-04-07T16:16:16.297 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.297 INFO:tasks.workunit.client.0.smithi100.stdout: "target_bytes": { 2024-04-07T16:16:16.297 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3183836160 2024-04-07T16:16:16.297 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.297 INFO:tasks.workunit.client.0.smithi100.stdout: "unmapped_bytes": { 2024-04-07T16:16:16.297 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 876544 2024-04-07T16:16:16.297 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:16.297 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.297 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:data": { 2024-04-07T16:16:16.297 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T16:16:16.298 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 159383552 2024-04-07T16:16:16.298 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.298 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T16:16:16.298 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.298 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.298 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T16:16:16.298 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.298 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.298 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T16:16:16.298 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 87830829 2024-04-07T16:16:16.298 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.298 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T16:16:16.298 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.298 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.299 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T16:16:16.299 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.299 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.299 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T16:16:16.299 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.299 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.299 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T16:16:16.299 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.299 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.299 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T16:16:16.299 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.299 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.299 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T16:16:16.299 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 614400 2024-04-07T16:16:16.299 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.300 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T16:16:16.300 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.300 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.300 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T16:16:16.300 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.300 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.300 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T16:16:16.300 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.300 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.300 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T16:16:16.300 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 70938323 2024-04-07T16:16:16.300 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:16.300 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.300 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:kv": { 2024-04-07T16:16:16.300 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T16:16:16.300 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 796917760 2024-04-07T16:16:16.301 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.301 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T16:16:16.301 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 67108864 2024-04-07T16:16:16.301 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.301 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T16:16:16.301 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.301 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.301 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T16:16:16.301 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 658731219 2024-04-07T16:16:16.301 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.301 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T16:16:16.301 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.301 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.301 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T16:16:16.302 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.302 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.302 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T16:16:16.302 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.302 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.302 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T16:16:16.302 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.302 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.302 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T16:16:16.302 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.302 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.302 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T16:16:16.302 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2128 2024-04-07T16:16:16.302 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.302 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T16:16:16.303 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.303 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.303 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T16:16:16.303 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.303 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.303 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T16:16:16.303 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.303 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.303 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T16:16:16.303 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 71075549 2024-04-07T16:16:16.303 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:16.303 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.303 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:kv_onode": { 2024-04-07T16:16:16.303 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T16:16:16.304 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 192937984 2024-04-07T16:16:16.304 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.304 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T16:16:16.304 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 67108864 2024-04-07T16:16:16.304 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.304 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T16:16:16.304 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 464 2024-04-07T16:16:16.304 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.304 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T16:16:16.304 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 58553886 2024-04-07T16:16:16.304 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.304 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T16:16:16.304 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.304 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.304 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T16:16:16.304 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.305 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.305 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T16:16:16.305 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.305 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.305 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T16:16:16.305 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.305 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.305 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T16:16:16.305 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.305 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.305 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T16:16:16.305 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.305 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.305 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T16:16:16.305 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.305 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.306 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T16:16:16.306 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.306 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.306 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T16:16:16.306 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.306 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.306 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T16:16:16.306 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 67274770 2024-04-07T16:16:16.306 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:16.306 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.306 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:meta": { 2024-04-07T16:16:16.306 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T16:16:16.306 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 729808896 2024-04-07T16:16:16.306 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.307 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T16:16:16.307 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.307 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.307 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T16:16:16.307 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.307 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.307 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T16:16:16.307 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 658731219 2024-04-07T16:16:16.307 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.307 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T16:16:16.307 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.307 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.307 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T16:16:16.307 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 8526 2024-04-07T16:16:16.307 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.308 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T16:16:16.308 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.308 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.308 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T16:16:16.308 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.308 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.308 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T16:16:16.308 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.308 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.308 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T16:16:16.308 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 235098 2024-04-07T16:16:16.308 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.308 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T16:16:16.308 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.308 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.309 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T16:16:16.309 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.309 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.309 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T16:16:16.309 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.309 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.309 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T16:16:16.309 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 70834053 2024-04-07T16:16:16.309 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:16.309 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.309 INFO:tasks.workunit.client.0.smithi100.stdout: "objecter": { 2024-04-07T16:16:16.309 INFO:tasks.workunit.client.0.smithi100.stdout: "op_active": { 2024-04-07T16:16:16.309 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.309 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.309 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r": { 2024-04-07T16:16:16.310 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.310 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.310 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rmw": { 2024-04-07T16:16:16.310 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.310 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.310 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w": { 2024-04-07T16:16:16.310 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.310 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:16.310 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.310 INFO:tasks.workunit.client.0.smithi100.stdout: "osd": { 2024-04-07T16:16:16.310 INFO:tasks.workunit.client.0.smithi100.stdout: "numpg": { 2024-04-07T16:16:16.310 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 44 2024-04-07T16:16:16.310 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.310 INFO:tasks.workunit.client.0.smithi100.stdout: "numpg_removing": { 2024-04-07T16:16:16.310 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.311 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.311 INFO:tasks.workunit.client.0.smithi100.stdout: "op": { 2024-04-07T16:16:16.311 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5 2024-04-07T16:16:16.311 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.311 INFO:tasks.workunit.client.0.smithi100.stdout: "op_delayed_degraded": { 2024-04-07T16:16:16.311 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.311 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.311 INFO:tasks.workunit.client.0.smithi100.stdout: "op_delayed_unreadable": { 2024-04-07T16:16:16.311 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.311 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.311 INFO:tasks.workunit.client.0.smithi100.stdout: "op_in_bytes": { 2024-04-07T16:16:16.311 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 616 2024-04-07T16:16:16.311 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.311 INFO:tasks.workunit.client.0.smithi100.stdout: "op_latency": { 2024-04-07T16:16:16.311 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 5, 2024-04-07T16:16:16.312 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 77374518 2024-04-07T16:16:16.312 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.312 INFO:tasks.workunit.client.0.smithi100.stdout: "op_out_bytes": { 2024-04-07T16:16:16.312 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.312 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.312 INFO:tasks.workunit.client.0.smithi100.stdout: "op_prepare_latency": { 2024-04-07T16:16:16.312 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 6, 2024-04-07T16:16:16.312 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 9316980 2024-04-07T16:16:16.312 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.312 INFO:tasks.workunit.client.0.smithi100.stdout: "op_process_latency": { 2024-04-07T16:16:16.312 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 5, 2024-04-07T16:16:16.312 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 57164984 2024-04-07T16:16:16.312 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.312 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r": { 2024-04-07T16:16:16.313 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.313 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.313 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_latency": { 2024-04-07T16:16:16.313 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.313 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.313 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.313 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_latency_out_bytes_histogram": { 2024-04-07T16:16:16.313 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.313 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.313 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_out_bytes": { 2024-04-07T16:16:16.313 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.313 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.313 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_prepare_latency": { 2024-04-07T16:16:16.313 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.313 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.314 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.314 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_process_latency": { 2024-04-07T16:16:16.314 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.314 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.314 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.314 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw": { 2024-04-07T16:16:16.314 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.314 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.314 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_in_bytes": { 2024-04-07T16:16:16.314 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.314 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.314 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_latency": { 2024-04-07T16:16:16.314 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.314 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.314 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.315 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_latency_in_bytes_histogram": { 2024-04-07T16:16:16.315 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.315 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.315 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_latency_out_bytes_histogram": { 2024-04-07T16:16:16.315 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.315 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.315 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_out_bytes": { 2024-04-07T16:16:16.315 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.315 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.315 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_prepare_latency": { 2024-04-07T16:16:16.315 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.315 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.315 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.315 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_process_latency": { 2024-04-07T16:16:16.315 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.315 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.316 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.316 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w": { 2024-04-07T16:16:16.316 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5 2024-04-07T16:16:16.316 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.316 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_in_bytes": { 2024-04-07T16:16:16.316 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 616 2024-04-07T16:16:16.316 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.316 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_latency": { 2024-04-07T16:16:16.316 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 5, 2024-04-07T16:16:16.316 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 77374518 2024-04-07T16:16:16.316 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.316 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_latency_in_bytes_histogram": { 2024-04-07T16:16:16.316 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.316 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.316 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_prepare_latency": { 2024-04-07T16:16:16.317 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 6, 2024-04-07T16:16:16.317 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 9316980 2024-04-07T16:16:16.317 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.317 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_process_latency": { 2024-04-07T16:16:16.317 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 5, 2024-04-07T16:16:16.317 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 57164984 2024-04-07T16:16:16.317 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.317 INFO:tasks.workunit.client.0.smithi100.stdout: "op_wip": { 2024-04-07T16:16:16.317 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.317 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.317 INFO:tasks.workunit.client.0.smithi100.stdout: "recovery_bytes": { 2024-04-07T16:16:16.317 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1771104 2024-04-07T16:16:16.317 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.317 INFO:tasks.workunit.client.0.smithi100.stdout: "recovery_ops": { 2024-04-07T16:16:16.317 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 6 2024-04-07T16:16:16.318 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.318 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_bytes": { 2024-04-07T16:16:16.318 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 95995035648 2024-04-07T16:16:16.318 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.318 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_bytes_used": { 2024-04-07T16:16:16.318 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 28983296 2024-04-07T16:16:16.318 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:16.318 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:16.318 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.318 INFO:tasks.workunit.client.0.smithi100.stdout: "osd.1": { 2024-04-07T16:16:16.318 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs": { 2024-04-07T16:16:16.318 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_slow_fallback": { 2024-04-07T16:16:16.318 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.318 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.318 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_slow_size_fallback": { 2024-04-07T16:16:16.319 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.319 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.319 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit_db": { 2024-04-07T16:16:16.319 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 65536 2024-04-07T16:16:16.319 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.319 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit_main": { 2024-04-07T16:16:16.319 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.319 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.319 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit_wal": { 2024-04-07T16:16:16.319 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.319 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.319 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_written_slow": { 2024-04-07T16:16:16.319 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.319 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.319 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_written_sst": { 2024-04-07T16:16:16.319 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 12288 2024-04-07T16:16:16.320 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.320 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_written_wal": { 2024-04-07T16:16:16.320 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2367488 2024-04-07T16:16:16.320 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.320 INFO:tasks.workunit.client.0.smithi100.stdout: "compact_lat": { 2024-04-07T16:16:16.320 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.320 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.320 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.320 INFO:tasks.workunit.client.0.smithi100.stdout: "compact_lock_lat": { 2024-04-07T16:16:16.320 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.320 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.320 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.320 INFO:tasks.workunit.client.0.smithi100.stdout: "db_total_bytes": { 2024-04-07T16:16:16.320 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 95995035648 2024-04-07T16:16:16.320 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.321 INFO:tasks.workunit.client.0.smithi100.stdout: "db_used_bytes": { 2024-04-07T16:16:16.321 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 27656192 2024-04-07T16:16:16.321 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.321 INFO:tasks.workunit.client.0.smithi100.stdout: "log_bytes": { 2024-04-07T16:16:16.321 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1499136 2024-04-07T16:16:16.321 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.321 INFO:tasks.workunit.client.0.smithi100.stdout: "logged_bytes": { 2024-04-07T16:16:16.321 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1359872 2024-04-07T16:16:16.321 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.321 INFO:tasks.workunit.client.0.smithi100.stdout: "max_bytes_db": { 2024-04-07T16:16:16.321 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 31850496 2024-04-07T16:16:16.321 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.321 INFO:tasks.workunit.client.0.smithi100.stdout: "max_bytes_slow": { 2024-04-07T16:16:16.321 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.321 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.322 INFO:tasks.workunit.client.0.smithi100.stdout: "max_bytes_wal": { 2024-04-07T16:16:16.322 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.322 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.322 INFO:tasks.workunit.client.0.smithi100.stdout: "num_files": { 2024-04-07T16:16:16.322 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 14 2024-04-07T16:16:16.322 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.322 INFO:tasks.workunit.client.0.smithi100.stdout: "read_bytes": { 2024-04-07T16:16:16.322 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 206272 2024-04-07T16:16:16.322 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.322 INFO:tasks.workunit.client.0.smithi100.stdout: "read_count": { 2024-04-07T16:16:16.322 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 23 2024-04-07T16:16:16.322 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.322 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes": { 2024-04-07T16:16:16.322 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1282048 2024-04-07T16:16:16.322 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.323 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes_db": { 2024-04-07T16:16:16.323 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1286144 2024-04-07T16:16:16.323 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.323 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes_slow": { 2024-04-07T16:16:16.323 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.323 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.323 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes_wal": { 2024-04-07T16:16:16.323 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.323 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.323 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_count": { 2024-04-07T16:16:16.323 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 15 2024-04-07T16:16:16.323 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.323 INFO:tasks.workunit.client.0.smithi100.stdout: "read_prefetch_bytes": { 2024-04-07T16:16:16.323 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4913 2024-04-07T16:16:16.323 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.323 INFO:tasks.workunit.client.0.smithi100.stdout: "read_prefetch_count": { 2024-04-07T16:16:16.324 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4 2024-04-07T16:16:16.324 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.324 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_buffer_bytes": { 2024-04-07T16:16:16.324 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5505 2024-04-07T16:16:16.324 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.324 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_buffer_count": { 2024-04-07T16:16:16.324 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 24 2024-04-07T16:16:16.324 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.324 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_bytes": { 2024-04-07T16:16:16.324 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5505 2024-04-07T16:16:16.324 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.324 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_count": { 2024-04-07T16:16:16.324 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 24 2024-04-07T16:16:16.324 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.324 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes": { 2024-04-07T16:16:16.324 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.325 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.325 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes_db": { 2024-04-07T16:16:16.325 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.325 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.325 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes_slow": { 2024-04-07T16:16:16.325 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.325 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.325 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes_wal": { 2024-04-07T16:16:16.325 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.325 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.325 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_count": { 2024-04-07T16:16:16.325 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.325 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.325 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_total_bytes": { 2024-04-07T16:16:16.325 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.325 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.326 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_used_bytes": { 2024-04-07T16:16:16.326 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.326 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.326 INFO:tasks.workunit.client.0.smithi100.stdout: "wal_total_bytes": { 2024-04-07T16:16:16.326 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.326 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.326 INFO:tasks.workunit.client.0.smithi100.stdout: "wal_used_bytes": { 2024-04-07T16:16:16.326 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.326 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.326 INFO:tasks.workunit.client.0.smithi100.stdout: "write_bytes": { 2024-04-07T16:16:16.326 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3805184 2024-04-07T16:16:16.326 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:16.326 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.326 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore": { 2024-04-07T16:16:16.326 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit": { 2024-04-07T16:16:16.327 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4096 2024-04-07T16:16:16.327 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.327 INFO:tasks.workunit.client.0.smithi100.stdout: "allocated": { 2024-04-07T16:16:16.327 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 716800 2024-04-07T16:16:16.327 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.327 INFO:tasks.workunit.client.0.smithi100.stdout: "clist_lat": { 2024-04-07T16:16:16.327 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 5, 2024-04-07T16:16:16.327 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 341315 2024-04-07T16:16:16.327 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.327 INFO:tasks.workunit.client.0.smithi100.stdout: "compress_lat": { 2024-04-07T16:16:16.327 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.327 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.327 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.327 INFO:tasks.workunit.client.0.smithi100.stdout: "compressed": { 2024-04-07T16:16:16.327 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.327 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.328 INFO:tasks.workunit.client.0.smithi100.stdout: "compressed_allocated": { 2024-04-07T16:16:16.328 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.328 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.328 INFO:tasks.workunit.client.0.smithi100.stdout: "compressed_original": { 2024-04-07T16:16:16.328 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.328 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.328 INFO:tasks.workunit.client.0.smithi100.stdout: "csum_lat": { 2024-04-07T16:16:16.328 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 31, 2024-04-07T16:16:16.328 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 131637 2024-04-07T16:16:16.328 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.328 INFO:tasks.workunit.client.0.smithi100.stdout: "decompress_lat": { 2024-04-07T16:16:16.328 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.328 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.328 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.328 INFO:tasks.workunit.client.0.smithi100.stdout: "fragmentation_micros": { 2024-04-07T16:16:16.329 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.329 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.329 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_commit_lat": { 2024-04-07T16:16:16.329 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 319, 2024-04-07T16:16:16.329 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 102343784 2024-04-07T16:16:16.329 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.329 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_final_lat": { 2024-04-07T16:16:16.329 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 318, 2024-04-07T16:16:16.329 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 9983561 2024-04-07T16:16:16.329 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.329 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_flush_lat": { 2024-04-07T16:16:16.329 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 319, 2024-04-07T16:16:16.329 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 701088 2024-04-07T16:16:16.329 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.329 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_sync_lat": { 2024-04-07T16:16:16.330 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 319, 2024-04-07T16:16:16.330 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 103044872 2024-04-07T16:16:16.330 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.330 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_get_keys_lat": { 2024-04-07T16:16:16.330 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.330 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.330 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.330 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_get_values_lat": { 2024-04-07T16:16:16.330 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 3, 2024-04-07T16:16:16.330 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 57225 2024-04-07T16:16:16.330 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.330 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_lower_bound_lat": { 2024-04-07T16:16:16.330 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 2, 2024-04-07T16:16:16.330 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 11753 2024-04-07T16:16:16.330 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.331 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_next_lat": { 2024-04-07T16:16:16.331 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.331 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.331 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.331 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_seek_to_first_lat": { 2024-04-07T16:16:16.331 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.331 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.331 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.331 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_upper_bound_lat": { 2024-04-07T16:16:16.331 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 2, 2024-04-07T16:16:16.331 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 7756 2024-04-07T16:16:16.331 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.331 INFO:tasks.workunit.client.0.smithi100.stdout: "onode_hits": { 2024-04-07T16:16:16.331 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 807 2024-04-07T16:16:16.331 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.332 INFO:tasks.workunit.client.0.smithi100.stdout: "onode_misses": { 2024-04-07T16:16:16.332 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 241 2024-04-07T16:16:16.332 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.332 INFO:tasks.workunit.client.0.smithi100.stdout: "read_lat": { 2024-04-07T16:16:16.332 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 137, 2024-04-07T16:16:16.332 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 7601666 2024-04-07T16:16:16.332 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.332 INFO:tasks.workunit.client.0.smithi100.stdout: "read_onode_meta_lat": { 2024-04-07T16:16:16.332 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 354, 2024-04-07T16:16:16.332 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 588807 2024-04-07T16:16:16.332 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.332 INFO:tasks.workunit.client.0.smithi100.stdout: "read_wait_aio_lat": { 2024-04-07T16:16:16.332 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 217, 2024-04-07T16:16:16.332 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 6809857 2024-04-07T16:16:16.332 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.333 INFO:tasks.workunit.client.0.smithi100.stdout: "reads_with_retries": { 2024-04-07T16:16:16.333 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.333 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.333 INFO:tasks.workunit.client.0.smithi100.stdout: "remove_lat": { 2024-04-07T16:16:16.333 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 3, 2024-04-07T16:16:16.333 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 227318 2024-04-07T16:16:16.333 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.333 INFO:tasks.workunit.client.0.smithi100.stdout: "state_aio_wait_lat": { 2024-04-07T16:16:16.333 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 402, 2024-04-07T16:16:16.333 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 8680922 2024-04-07T16:16:16.333 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.333 INFO:tasks.workunit.client.0.smithi100.stdout: "state_deferred_aio_wait_lat": { 2024-04-07T16:16:16.333 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 120, 2024-04-07T16:16:16.333 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 26785405 2024-04-07T16:16:16.333 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.334 INFO:tasks.workunit.client.0.smithi100.stdout: "state_deferred_cleanup_lat": { 2024-04-07T16:16:16.334 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 119, 2024-04-07T16:16:16.334 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 571827153179 2024-04-07T16:16:16.334 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.334 INFO:tasks.workunit.client.0.smithi100.stdout: "state_deferred_queued_lat": { 2024-04-07T16:16:16.334 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 120, 2024-04-07T16:16:16.334 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 161506863344 2024-04-07T16:16:16.334 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.334 INFO:tasks.workunit.client.0.smithi100.stdout: "state_done_lat": { 2024-04-07T16:16:16.334 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 401, 2024-04-07T16:16:16.334 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 82819121573 2024-04-07T16:16:16.334 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.334 INFO:tasks.workunit.client.0.smithi100.stdout: "state_finishing_lat": { 2024-04-07T16:16:16.334 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 401, 2024-04-07T16:16:16.334 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 91444 2024-04-07T16:16:16.335 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.335 INFO:tasks.workunit.client.0.smithi100.stdout: "state_io_done_lat": { 2024-04-07T16:16:16.335 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 402, 2024-04-07T16:16:16.335 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 718729 2024-04-07T16:16:16.335 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.335 INFO:tasks.workunit.client.0.smithi100.stdout: "state_kv_commiting_lat": { 2024-04-07T16:16:16.335 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 402, 2024-04-07T16:16:16.335 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 184174016 2024-04-07T16:16:16.335 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.335 INFO:tasks.workunit.client.0.smithi100.stdout: "state_kv_done_lat": { 2024-04-07T16:16:16.335 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 402, 2024-04-07T16:16:16.335 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 225055 2024-04-07T16:16:16.335 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.335 INFO:tasks.workunit.client.0.smithi100.stdout: "state_kv_queued_lat": { 2024-04-07T16:16:16.335 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 402, 2024-04-07T16:16:16.335 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 106892522 2024-04-07T16:16:16.336 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.336 INFO:tasks.workunit.client.0.smithi100.stdout: "state_prepare_lat": { 2024-04-07T16:16:16.336 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 402, 2024-04-07T16:16:16.336 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 32108553 2024-04-07T16:16:16.336 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.336 INFO:tasks.workunit.client.0.smithi100.stdout: "stored": { 2024-04-07T16:16:16.336 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 309392 2024-04-07T16:16:16.336 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.336 INFO:tasks.workunit.client.0.smithi100.stdout: "truncate_lat": { 2024-04-07T16:16:16.336 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 54, 2024-04-07T16:16:16.336 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 542793 2024-04-07T16:16:16.336 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.336 INFO:tasks.workunit.client.0.smithi100.stdout: "txc_commit_lat": { 2024-04-07T16:16:16.336 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 402, 2024-04-07T16:16:16.337 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 332673277 2024-04-07T16:16:16.337 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.337 INFO:tasks.workunit.client.0.smithi100.stdout: "txc_submit_lat": { 2024-04-07T16:16:16.337 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 402, 2024-04-07T16:16:16.337 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 38729789 2024-04-07T16:16:16.337 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.337 INFO:tasks.workunit.client.0.smithi100.stdout: "txc_throttle_lat": { 2024-04-07T16:16:16.337 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 402, 2024-04-07T16:16:16.337 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3872843 2024-04-07T16:16:16.337 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:16.337 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.337 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache": { 2024-04-07T16:16:16.337 INFO:tasks.workunit.client.0.smithi100.stdout: "cache_bytes": { 2024-04-07T16:16:16.337 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1900954368 2024-04-07T16:16:16.338 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.338 INFO:tasks.workunit.client.0.smithi100.stdout: "heap_bytes": { 2024-04-07T16:16:16.338 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 44482560 2024-04-07T16:16:16.338 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.338 INFO:tasks.workunit.client.0.smithi100.stdout: "mapped_bytes": { 2024-04-07T16:16:16.338 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 44089344 2024-04-07T16:16:16.338 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.338 INFO:tasks.workunit.client.0.smithi100.stdout: "target_bytes": { 2024-04-07T16:16:16.338 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3183836160 2024-04-07T16:16:16.338 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.338 INFO:tasks.workunit.client.0.smithi100.stdout: "unmapped_bytes": { 2024-04-07T16:16:16.338 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 393216 2024-04-07T16:16:16.338 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:16.338 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.339 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:data": { 2024-04-07T16:16:16.339 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T16:16:16.339 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 159383552 2024-04-07T16:16:16.339 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.339 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T16:16:16.339 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.339 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.339 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T16:16:16.339 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.339 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.339 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T16:16:16.339 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 87868101 2024-04-07T16:16:16.339 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.339 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T16:16:16.339 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.340 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.340 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T16:16:16.340 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.340 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.340 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T16:16:16.340 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.340 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.340 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T16:16:16.340 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.340 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.340 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T16:16:16.340 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.340 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.340 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T16:16:16.340 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 16384 2024-04-07T16:16:16.341 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.341 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T16:16:16.341 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.341 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.341 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T16:16:16.341 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.341 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.341 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T16:16:16.341 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.341 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.341 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T16:16:16.341 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 71499067 2024-04-07T16:16:16.341 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:16.341 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.341 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:kv": { 2024-04-07T16:16:16.341 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T16:16:16.342 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 796917760 2024-04-07T16:16:16.342 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.342 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T16:16:16.342 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 67108864 2024-04-07T16:16:16.342 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.342 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T16:16:16.342 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.342 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.342 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T16:16:16.342 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 659010759 2024-04-07T16:16:16.342 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.342 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T16:16:16.342 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.342 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.342 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T16:16:16.343 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.343 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.343 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T16:16:16.343 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.343 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.343 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T16:16:16.343 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.343 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.343 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T16:16:16.343 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.343 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.343 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T16:16:16.343 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2128 2024-04-07T16:16:16.343 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.343 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T16:16:16.343 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.344 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.344 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T16:16:16.344 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.344 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.344 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T16:16:16.344 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.344 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.344 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T16:16:16.344 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 70796009 2024-04-07T16:16:16.344 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:16.344 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.344 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:kv_onode": { 2024-04-07T16:16:16.344 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T16:16:16.344 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 192937984 2024-04-07T16:16:16.345 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.345 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T16:16:16.345 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 67108864 2024-04-07T16:16:16.345 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.345 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T16:16:16.345 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 464 2024-04-07T16:16:16.345 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.345 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T16:16:16.345 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 58578734 2024-04-07T16:16:16.345 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.345 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T16:16:16.345 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.345 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.345 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T16:16:16.345 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.346 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.346 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T16:16:16.346 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.346 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.346 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T16:16:16.346 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.346 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.346 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T16:16:16.346 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.346 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.346 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T16:16:16.346 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.346 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.346 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T16:16:16.346 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.347 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.347 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T16:16:16.347 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.347 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.347 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T16:16:16.347 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.347 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.347 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T16:16:16.347 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 67249922 2024-04-07T16:16:16.347 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:16.347 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.347 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:meta": { 2024-04-07T16:16:16.347 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T16:16:16.347 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 729808896 2024-04-07T16:16:16.347 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.348 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T16:16:16.348 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.348 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.348 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T16:16:16.348 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.348 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.348 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T16:16:16.348 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 659010759 2024-04-07T16:16:16.348 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.348 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T16:16:16.348 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.348 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.348 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T16:16:16.348 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.349 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.349 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T16:16:16.349 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.349 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.349 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T16:16:16.349 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.349 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.349 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T16:16:16.349 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.349 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.349 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T16:16:16.349 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 219329 2024-04-07T16:16:16.349 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.349 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T16:16:16.349 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.349 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.349 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T16:16:16.350 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.350 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.350 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T16:16:16.350 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.350 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.350 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T16:16:16.350 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 70578808 2024-04-07T16:16:16.350 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:16.350 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.350 INFO:tasks.workunit.client.0.smithi100.stdout: "objecter": { 2024-04-07T16:16:16.350 INFO:tasks.workunit.client.0.smithi100.stdout: "op_active": { 2024-04-07T16:16:16.350 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.350 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.350 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r": { 2024-04-07T16:16:16.350 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.351 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.351 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rmw": { 2024-04-07T16:16:16.351 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.351 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.351 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w": { 2024-04-07T16:16:16.351 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.351 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:16.351 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.351 INFO:tasks.workunit.client.0.smithi100.stdout: "osd": { 2024-04-07T16:16:16.351 INFO:tasks.workunit.client.0.smithi100.stdout: "numpg": { 2024-04-07T16:16:16.351 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 38 2024-04-07T16:16:16.351 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.351 INFO:tasks.workunit.client.0.smithi100.stdout: "numpg_removing": { 2024-04-07T16:16:16.351 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.351 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.352 INFO:tasks.workunit.client.0.smithi100.stdout: "op": { 2024-04-07T16:16:16.352 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 155 2024-04-07T16:16:16.352 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.352 INFO:tasks.workunit.client.0.smithi100.stdout: "op_delayed_degraded": { 2024-04-07T16:16:16.352 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.352 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.352 INFO:tasks.workunit.client.0.smithi100.stdout: "op_delayed_unreadable": { 2024-04-07T16:16:16.352 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.352 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.352 INFO:tasks.workunit.client.0.smithi100.stdout: "op_in_bytes": { 2024-04-07T16:16:16.352 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2429440 2024-04-07T16:16:16.352 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.352 INFO:tasks.workunit.client.0.smithi100.stdout: "op_latency": { 2024-04-07T16:16:16.352 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 155, 2024-04-07T16:16:16.352 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 823968569 2024-04-07T16:16:16.352 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.353 INFO:tasks.workunit.client.0.smithi100.stdout: "op_out_bytes": { 2024-04-07T16:16:16.353 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1063 2024-04-07T16:16:16.353 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.353 INFO:tasks.workunit.client.0.smithi100.stdout: "op_prepare_latency": { 2024-04-07T16:16:16.353 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 155, 2024-04-07T16:16:16.353 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 89196798 2024-04-07T16:16:16.353 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.353 INFO:tasks.workunit.client.0.smithi100.stdout: "op_process_latency": { 2024-04-07T16:16:16.353 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 155, 2024-04-07T16:16:16.353 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 307014042 2024-04-07T16:16:16.353 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.353 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r": { 2024-04-07T16:16:16.353 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 33 2024-04-07T16:16:16.353 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.353 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_latency": { 2024-04-07T16:16:16.353 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 33, 2024-04-07T16:16:16.353 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 12714834 2024-04-07T16:16:16.354 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.354 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_latency_out_bytes_histogram": { 2024-04-07T16:16:16.354 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.354 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.354 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_out_bytes": { 2024-04-07T16:16:16.354 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1063 2024-04-07T16:16:16.354 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.354 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_prepare_latency": { 2024-04-07T16:16:16.354 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 33, 2024-04-07T16:16:16.354 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 11164911 2024-04-07T16:16:16.354 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.354 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_process_latency": { 2024-04-07T16:16:16.354 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 33, 2024-04-07T16:16:16.354 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 9726155 2024-04-07T16:16:16.354 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.355 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw": { 2024-04-07T16:16:16.355 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 22 2024-04-07T16:16:16.355 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.355 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_in_bytes": { 2024-04-07T16:16:16.355 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.355 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.355 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_latency": { 2024-04-07T16:16:16.355 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 22, 2024-04-07T16:16:16.355 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 36605744 2024-04-07T16:16:16.355 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.355 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_latency_in_bytes_histogram": { 2024-04-07T16:16:16.355 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.355 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.355 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_latency_out_bytes_histogram": { 2024-04-07T16:16:16.356 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.356 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.356 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_out_bytes": { 2024-04-07T16:16:16.356 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.356 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.356 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_prepare_latency": { 2024-04-07T16:16:16.356 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 22, 2024-04-07T16:16:16.356 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 15805848 2024-04-07T16:16:16.356 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.356 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_process_latency": { 2024-04-07T16:16:16.356 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 22, 2024-04-07T16:16:16.356 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 34480353 2024-04-07T16:16:16.356 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.356 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w": { 2024-04-07T16:16:16.357 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 100 2024-04-07T16:16:16.357 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.357 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_in_bytes": { 2024-04-07T16:16:16.357 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2429440 2024-04-07T16:16:16.357 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.357 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_latency": { 2024-04-07T16:16:16.357 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 100, 2024-04-07T16:16:16.357 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 774647991 2024-04-07T16:16:16.357 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.357 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_latency_in_bytes_histogram": { 2024-04-07T16:16:16.357 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.357 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.357 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_prepare_latency": { 2024-04-07T16:16:16.357 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 100, 2024-04-07T16:16:16.357 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 62226039 2024-04-07T16:16:16.357 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.357 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_process_latency": { 2024-04-07T16:16:16.358 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 100, 2024-04-07T16:16:16.358 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 262807534 2024-04-07T16:16:16.358 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.358 INFO:tasks.workunit.client.0.smithi100.stdout: "op_wip": { 2024-04-07T16:16:16.358 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.358 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.358 INFO:tasks.workunit.client.0.smithi100.stdout: "recovery_bytes": { 2024-04-07T16:16:16.358 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 590368 2024-04-07T16:16:16.358 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.358 INFO:tasks.workunit.client.0.smithi100.stdout: "recovery_ops": { 2024-04-07T16:16:16.358 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2 2024-04-07T16:16:16.358 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.358 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_bytes": { 2024-04-07T16:16:16.358 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 95995035648 2024-04-07T16:16:16.358 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.359 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_bytes_used": { 2024-04-07T16:16:16.359 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 28385280 2024-04-07T16:16:16.359 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:16.359 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:16.359 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.359 INFO:tasks.workunit.client.0.smithi100.stdout: "osd.2": { 2024-04-07T16:16:16.359 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs": { 2024-04-07T16:16:16.359 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_slow_fallback": { 2024-04-07T16:16:16.359 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.359 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.359 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_slow_size_fallback": { 2024-04-07T16:16:16.359 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.359 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.359 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit_db": { 2024-04-07T16:16:16.360 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 65536 2024-04-07T16:16:16.360 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.360 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit_main": { 2024-04-07T16:16:16.360 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.360 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.360 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit_wal": { 2024-04-07T16:16:16.360 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.360 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.360 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_written_slow": { 2024-04-07T16:16:16.360 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.360 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.360 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_written_sst": { 2024-04-07T16:16:16.360 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 12288 2024-04-07T16:16:16.360 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.360 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_written_wal": { 2024-04-07T16:16:16.361 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 7000064 2024-04-07T16:16:16.361 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.361 INFO:tasks.workunit.client.0.smithi100.stdout: "compact_lat": { 2024-04-07T16:16:16.361 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.361 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.361 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.361 INFO:tasks.workunit.client.0.smithi100.stdout: "compact_lock_lat": { 2024-04-07T16:16:16.361 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.361 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.361 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.361 INFO:tasks.workunit.client.0.smithi100.stdout: "db_total_bytes": { 2024-04-07T16:16:16.361 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 95995035648 2024-04-07T16:16:16.361 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.361 INFO:tasks.workunit.client.0.smithi100.stdout: "db_used_bytes": { 2024-04-07T16:16:16.361 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 31850496 2024-04-07T16:16:16.362 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.362 INFO:tasks.workunit.client.0.smithi100.stdout: "log_bytes": { 2024-04-07T16:16:16.362 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3526656 2024-04-07T16:16:16.362 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.362 INFO:tasks.workunit.client.0.smithi100.stdout: "logged_bytes": { 2024-04-07T16:16:16.362 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3387392 2024-04-07T16:16:16.362 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.362 INFO:tasks.workunit.client.0.smithi100.stdout: "max_bytes_db": { 2024-04-07T16:16:16.362 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 31850496 2024-04-07T16:16:16.362 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.362 INFO:tasks.workunit.client.0.smithi100.stdout: "max_bytes_slow": { 2024-04-07T16:16:16.362 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.362 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.362 INFO:tasks.workunit.client.0.smithi100.stdout: "max_bytes_wal": { 2024-04-07T16:16:16.363 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.363 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.363 INFO:tasks.workunit.client.0.smithi100.stdout: "num_files": { 2024-04-07T16:16:16.363 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 14 2024-04-07T16:16:16.363 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.363 INFO:tasks.workunit.client.0.smithi100.stdout: "read_bytes": { 2024-04-07T16:16:16.363 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 206272 2024-04-07T16:16:16.363 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.363 INFO:tasks.workunit.client.0.smithi100.stdout: "read_count": { 2024-04-07T16:16:16.363 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 23 2024-04-07T16:16:16.363 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.363 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes": { 2024-04-07T16:16:16.363 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1282048 2024-04-07T16:16:16.363 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.363 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes_db": { 2024-04-07T16:16:16.363 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1286144 2024-04-07T16:16:16.364 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.364 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes_slow": { 2024-04-07T16:16:16.364 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.364 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.364 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes_wal": { 2024-04-07T16:16:16.364 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.364 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.364 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_count": { 2024-04-07T16:16:16.364 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 15 2024-04-07T16:16:16.364 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.364 INFO:tasks.workunit.client.0.smithi100.stdout: "read_prefetch_bytes": { 2024-04-07T16:16:16.364 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4913 2024-04-07T16:16:16.364 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.364 INFO:tasks.workunit.client.0.smithi100.stdout: "read_prefetch_count": { 2024-04-07T16:16:16.364 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4 2024-04-07T16:16:16.365 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.365 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_buffer_bytes": { 2024-04-07T16:16:16.365 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5505 2024-04-07T16:16:16.365 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.365 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_buffer_count": { 2024-04-07T16:16:16.365 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 24 2024-04-07T16:16:16.365 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.365 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_bytes": { 2024-04-07T16:16:16.365 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5505 2024-04-07T16:16:16.365 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.365 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_count": { 2024-04-07T16:16:16.365 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 24 2024-04-07T16:16:16.365 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.365 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes": { 2024-04-07T16:16:16.365 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.365 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.365 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes_db": { 2024-04-07T16:16:16.366 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.366 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.366 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes_slow": { 2024-04-07T16:16:16.366 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.366 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.366 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes_wal": { 2024-04-07T16:16:16.366 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.366 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.366 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_count": { 2024-04-07T16:16:16.366 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.366 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.366 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_total_bytes": { 2024-04-07T16:16:16.366 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.366 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.367 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_used_bytes": { 2024-04-07T16:16:16.367 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.367 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.367 INFO:tasks.workunit.client.0.smithi100.stdout: "wal_total_bytes": { 2024-04-07T16:16:16.367 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.367 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.367 INFO:tasks.workunit.client.0.smithi100.stdout: "wal_used_bytes": { 2024-04-07T16:16:16.367 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.367 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.367 INFO:tasks.workunit.client.0.smithi100.stdout: "write_bytes": { 2024-04-07T16:16:16.367 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 10465280 2024-04-07T16:16:16.367 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:16.367 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.367 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore": { 2024-04-07T16:16:16.367 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit": { 2024-04-07T16:16:16.368 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4096 2024-04-07T16:16:16.368 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.368 INFO:tasks.workunit.client.0.smithi100.stdout: "allocated": { 2024-04-07T16:16:16.368 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 708608 2024-04-07T16:16:16.368 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.368 INFO:tasks.workunit.client.0.smithi100.stdout: "clist_lat": { 2024-04-07T16:16:16.368 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 4, 2024-04-07T16:16:16.368 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 541920 2024-04-07T16:16:16.368 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.368 INFO:tasks.workunit.client.0.smithi100.stdout: "compress_lat": { 2024-04-07T16:16:16.368 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.368 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.368 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.368 INFO:tasks.workunit.client.0.smithi100.stdout: "compressed": { 2024-04-07T16:16:16.369 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.369 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.369 INFO:tasks.workunit.client.0.smithi100.stdout: "compressed_allocated": { 2024-04-07T16:16:16.369 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.369 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.369 INFO:tasks.workunit.client.0.smithi100.stdout: "compressed_original": { 2024-04-07T16:16:16.369 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.369 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.369 INFO:tasks.workunit.client.0.smithi100.stdout: "csum_lat": { 2024-04-07T16:16:16.369 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 10, 2024-04-07T16:16:16.369 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 16574 2024-04-07T16:16:16.369 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.369 INFO:tasks.workunit.client.0.smithi100.stdout: "decompress_lat": { 2024-04-07T16:16:16.369 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.370 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.370 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.370 INFO:tasks.workunit.client.0.smithi100.stdout: "fragmentation_micros": { 2024-04-07T16:16:16.370 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.370 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.370 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_commit_lat": { 2024-04-07T16:16:16.370 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 814, 2024-04-07T16:16:16.370 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 532460835 2024-04-07T16:16:16.370 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.370 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_final_lat": { 2024-04-07T16:16:16.370 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 809, 2024-04-07T16:16:16.370 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 29164356 2024-04-07T16:16:16.370 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.370 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_flush_lat": { 2024-04-07T16:16:16.370 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 814, 2024-04-07T16:16:16.371 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1352905 2024-04-07T16:16:16.371 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.371 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_sync_lat": { 2024-04-07T16:16:16.371 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 814, 2024-04-07T16:16:16.371 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 533813740 2024-04-07T16:16:16.371 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.371 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_get_keys_lat": { 2024-04-07T16:16:16.371 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.371 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.371 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.371 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_get_values_lat": { 2024-04-07T16:16:16.371 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 3, 2024-04-07T16:16:16.371 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 66261 2024-04-07T16:16:16.371 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.371 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_lower_bound_lat": { 2024-04-07T16:16:16.372 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.372 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.372 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.372 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_next_lat": { 2024-04-07T16:16:16.372 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.372 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.372 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.372 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_seek_to_first_lat": { 2024-04-07T16:16:16.372 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.372 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.372 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.372 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_upper_bound_lat": { 2024-04-07T16:16:16.372 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 2, 2024-04-07T16:16:16.372 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 10313 2024-04-07T16:16:16.373 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.373 INFO:tasks.workunit.client.0.smithi100.stdout: "onode_hits": { 2024-04-07T16:16:16.373 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3692 2024-04-07T16:16:16.373 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.373 INFO:tasks.workunit.client.0.smithi100.stdout: "onode_misses": { 2024-04-07T16:16:16.373 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 312 2024-04-07T16:16:16.373 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.373 INFO:tasks.workunit.client.0.smithi100.stdout: "read_lat": { 2024-04-07T16:16:16.373 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 97, 2024-04-07T16:16:16.373 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1919506 2024-04-07T16:16:16.373 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.373 INFO:tasks.workunit.client.0.smithi100.stdout: "read_onode_meta_lat": { 2024-04-07T16:16:16.373 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 268, 2024-04-07T16:16:16.373 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 467195 2024-04-07T16:16:16.373 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.374 INFO:tasks.workunit.client.0.smithi100.stdout: "read_wait_aio_lat": { 2024-04-07T16:16:16.374 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 171, 2024-04-07T16:16:16.374 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4487537 2024-04-07T16:16:16.374 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.374 INFO:tasks.workunit.client.0.smithi100.stdout: "reads_with_retries": { 2024-04-07T16:16:16.374 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.374 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.374 INFO:tasks.workunit.client.0.smithi100.stdout: "remove_lat": { 2024-04-07T16:16:16.374 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 103, 2024-04-07T16:16:16.374 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 6376664 2024-04-07T16:16:16.374 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.374 INFO:tasks.workunit.client.0.smithi100.stdout: "state_aio_wait_lat": { 2024-04-07T16:16:16.374 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 3432, 2024-04-07T16:16:16.374 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4037452882 2024-04-07T16:16:16.374 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.375 INFO:tasks.workunit.client.0.smithi100.stdout: "state_deferred_aio_wait_lat": { 2024-04-07T16:16:16.375 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 114, 2024-04-07T16:16:16.375 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 28436992 2024-04-07T16:16:16.375 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.375 INFO:tasks.workunit.client.0.smithi100.stdout: "state_deferred_cleanup_lat": { 2024-04-07T16:16:16.375 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 113, 2024-04-07T16:16:16.375 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 784322434793 2024-04-07T16:16:16.375 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.375 INFO:tasks.workunit.client.0.smithi100.stdout: "state_deferred_queued_lat": { 2024-04-07T16:16:16.375 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 114, 2024-04-07T16:16:16.375 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 128648915339 2024-04-07T16:16:16.375 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.375 INFO:tasks.workunit.client.0.smithi100.stdout: "state_done_lat": { 2024-04-07T16:16:16.375 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 3431, 2024-04-07T16:16:16.375 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 192843779440 2024-04-07T16:16:16.376 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.376 INFO:tasks.workunit.client.0.smithi100.stdout: "state_finishing_lat": { 2024-04-07T16:16:16.376 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 3431, 2024-04-07T16:16:16.376 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 277568 2024-04-07T16:16:16.376 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.376 INFO:tasks.workunit.client.0.smithi100.stdout: "state_io_done_lat": { 2024-04-07T16:16:16.376 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 3432, 2024-04-07T16:16:16.376 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 485004891 2024-04-07T16:16:16.376 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.376 INFO:tasks.workunit.client.0.smithi100.stdout: "state_kv_commiting_lat": { 2024-04-07T16:16:16.376 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 3432, 2024-04-07T16:16:16.376 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3506235439 2024-04-07T16:16:16.376 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.376 INFO:tasks.workunit.client.0.smithi100.stdout: "state_kv_done_lat": { 2024-04-07T16:16:16.377 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 3432, 2024-04-07T16:16:16.377 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 657511 2024-04-07T16:16:16.377 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.377 INFO:tasks.workunit.client.0.smithi100.stdout: "state_kv_queued_lat": { 2024-04-07T16:16:16.377 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 3432, 2024-04-07T16:16:16.377 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2061285108 2024-04-07T16:16:16.377 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.377 INFO:tasks.workunit.client.0.smithi100.stdout: "state_prepare_lat": { 2024-04-07T16:16:16.377 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 3432, 2024-04-07T16:16:16.377 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 367118666 2024-04-07T16:16:16.377 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.377 INFO:tasks.workunit.client.0.smithi100.stdout: "stored": { 2024-04-07T16:16:16.377 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 309232 2024-04-07T16:16:16.377 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.377 INFO:tasks.workunit.client.0.smithi100.stdout: "truncate_lat": { 2024-04-07T16:16:16.377 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 48, 2024-04-07T16:16:16.378 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 387879 2024-04-07T16:16:16.378 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.378 INFO:tasks.workunit.client.0.smithi100.stdout: "txc_commit_lat": { 2024-04-07T16:16:16.378 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 3432, 2024-04-07T16:16:16.378 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 10457408332 2024-04-07T16:16:16.378 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.378 INFO:tasks.workunit.client.0.smithi100.stdout: "txc_submit_lat": { 2024-04-07T16:16:16.378 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 3432, 2024-04-07T16:16:16.378 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 439673087 2024-04-07T16:16:16.378 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.378 INFO:tasks.workunit.client.0.smithi100.stdout: "txc_throttle_lat": { 2024-04-07T16:16:16.378 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 3432, 2024-04-07T16:16:16.378 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 197209404 2024-04-07T16:16:16.378 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:16.378 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.378 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache": { 2024-04-07T16:16:16.379 INFO:tasks.workunit.client.0.smithi100.stdout: "cache_bytes": { 2024-04-07T16:16:16.379 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1900954368 2024-04-07T16:16:16.379 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.379 INFO:tasks.workunit.client.0.smithi100.stdout: "heap_bytes": { 2024-04-07T16:16:16.379 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 48676864 2024-04-07T16:16:16.379 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.379 INFO:tasks.workunit.client.0.smithi100.stdout: "mapped_bytes": { 2024-04-07T16:16:16.379 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 48152576 2024-04-07T16:16:16.379 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.379 INFO:tasks.workunit.client.0.smithi100.stdout: "target_bytes": { 2024-04-07T16:16:16.379 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3183836160 2024-04-07T16:16:16.379 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.379 INFO:tasks.workunit.client.0.smithi100.stdout: "unmapped_bytes": { 2024-04-07T16:16:16.379 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 524288 2024-04-07T16:16:16.379 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:16.379 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.380 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:data": { 2024-04-07T16:16:16.380 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T16:16:16.380 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 159383552 2024-04-07T16:16:16.380 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.380 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T16:16:16.380 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.380 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.380 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T16:16:16.380 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.380 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.380 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T16:16:16.380 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 87862818 2024-04-07T16:16:16.380 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.380 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T16:16:16.380 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.380 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.380 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T16:16:16.381 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.381 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.381 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T16:16:16.381 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.381 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.381 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T16:16:16.381 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.381 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.381 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T16:16:16.381 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.381 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.381 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T16:16:16.381 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 8192 2024-04-07T16:16:16.381 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.381 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T16:16:16.381 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.382 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.382 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T16:16:16.382 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.382 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.382 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T16:16:16.382 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.382 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.382 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T16:16:16.382 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 71512542 2024-04-07T16:16:16.382 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:16.382 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.382 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:kv": { 2024-04-07T16:16:16.382 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T16:16:16.383 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 796917760 2024-04-07T16:16:16.383 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.383 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T16:16:16.383 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 67108864 2024-04-07T16:16:16.383 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.383 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T16:16:16.383 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.383 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.383 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T16:16:16.383 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 658971135 2024-04-07T16:16:16.383 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.383 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T16:16:16.383 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.383 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.384 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T16:16:16.384 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.384 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.384 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T16:16:16.384 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.384 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.384 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T16:16:16.384 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.384 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.384 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T16:16:16.384 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.384 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.384 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T16:16:16.384 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2128 2024-04-07T16:16:16.384 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.385 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T16:16:16.385 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.385 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.385 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T16:16:16.385 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.385 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.385 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T16:16:16.385 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.385 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.385 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T16:16:16.385 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 70835633 2024-04-07T16:16:16.385 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:16.385 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.385 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:kv_onode": { 2024-04-07T16:16:16.385 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T16:16:16.386 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 192937984 2024-04-07T16:16:16.386 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.386 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T16:16:16.386 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 67108864 2024-04-07T16:16:16.386 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.386 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T16:16:16.386 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 464 2024-04-07T16:16:16.386 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.386 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T16:16:16.386 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 58575212 2024-04-07T16:16:16.386 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.386 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T16:16:16.386 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.386 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.387 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T16:16:16.387 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.387 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.387 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T16:16:16.387 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.387 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.387 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T16:16:16.387 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.387 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.387 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T16:16:16.387 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.387 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.387 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T16:16:16.387 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.387 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.388 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T16:16:16.388 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.388 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.388 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T16:16:16.388 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.388 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.388 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T16:16:16.388 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.388 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.388 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T16:16:16.388 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 67253444 2024-04-07T16:16:16.388 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:16.388 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.388 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:meta": { 2024-04-07T16:16:16.388 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T16:16:16.388 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 729808896 2024-04-07T16:16:16.389 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.389 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T16:16:16.389 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.389 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.389 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T16:16:16.389 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.389 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.389 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T16:16:16.389 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 658971135 2024-04-07T16:16:16.389 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.389 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T16:16:16.389 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.389 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.389 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T16:16:16.389 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.389 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.389 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T16:16:16.390 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.390 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.390 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T16:16:16.390 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.390 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.390 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T16:16:16.390 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.390 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.390 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T16:16:16.390 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 316782 2024-04-07T16:16:16.390 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.390 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T16:16:16.390 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.390 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.391 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T16:16:16.391 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.391 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.391 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T16:16:16.391 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.391 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.391 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T16:16:16.391 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 70520979 2024-04-07T16:16:16.391 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:16.391 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.391 INFO:tasks.workunit.client.0.smithi100.stdout: "objecter": { 2024-04-07T16:16:16.391 INFO:tasks.workunit.client.0.smithi100.stdout: "op_active": { 2024-04-07T16:16:16.391 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.391 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.391 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r": { 2024-04-07T16:16:16.391 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.391 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.391 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rmw": { 2024-04-07T16:16:16.392 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.392 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.392 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w": { 2024-04-07T16:16:16.392 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.392 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:16.392 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.392 INFO:tasks.workunit.client.0.smithi100.stdout: "osd": { 2024-04-07T16:16:16.392 INFO:tasks.workunit.client.0.smithi100.stdout: "numpg": { 2024-04-07T16:16:16.392 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 30 2024-04-07T16:16:16.392 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.392 INFO:tasks.workunit.client.0.smithi100.stdout: "numpg_removing": { 2024-04-07T16:16:16.392 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.392 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.392 INFO:tasks.workunit.client.0.smithi100.stdout: "op": { 2024-04-07T16:16:16.393 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 10 2024-04-07T16:16:16.393 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.393 INFO:tasks.workunit.client.0.smithi100.stdout: "op_delayed_degraded": { 2024-04-07T16:16:16.393 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.393 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.393 INFO:tasks.workunit.client.0.smithi100.stdout: "op_delayed_unreadable": { 2024-04-07T16:16:16.393 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.393 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.393 INFO:tasks.workunit.client.0.smithi100.stdout: "op_in_bytes": { 2024-04-07T16:16:16.393 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.393 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.393 INFO:tasks.workunit.client.0.smithi100.stdout: "op_latency": { 2024-04-07T16:16:16.393 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 10, 2024-04-07T16:16:16.393 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 238086038 2024-04-07T16:16:16.393 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.394 INFO:tasks.workunit.client.0.smithi100.stdout: "op_out_bytes": { 2024-04-07T16:16:16.394 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.394 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.394 INFO:tasks.workunit.client.0.smithi100.stdout: "op_prepare_latency": { 2024-04-07T16:16:16.394 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 10, 2024-04-07T16:16:16.394 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 21363181 2024-04-07T16:16:16.394 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.394 INFO:tasks.workunit.client.0.smithi100.stdout: "op_process_latency": { 2024-04-07T16:16:16.394 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 10, 2024-04-07T16:16:16.394 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 163225415 2024-04-07T16:16:16.394 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.394 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r": { 2024-04-07T16:16:16.394 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.394 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.395 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_latency": { 2024-04-07T16:16:16.395 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.395 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.395 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.395 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_latency_out_bytes_histogram": { 2024-04-07T16:16:16.395 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.395 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.395 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_out_bytes": { 2024-04-07T16:16:16.395 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.395 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.395 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_prepare_latency": { 2024-04-07T16:16:16.395 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.395 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.395 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.395 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_process_latency": { 2024-04-07T16:16:16.395 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.396 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.396 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.396 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw": { 2024-04-07T16:16:16.396 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.396 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.396 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_in_bytes": { 2024-04-07T16:16:16.396 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.396 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.396 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_latency": { 2024-04-07T16:16:16.396 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.396 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.396 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.396 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_latency_in_bytes_histogram": { 2024-04-07T16:16:16.396 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.396 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.396 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_latency_out_bytes_histogram": { 2024-04-07T16:16:16.397 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.397 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.397 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_out_bytes": { 2024-04-07T16:16:16.397 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.397 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.397 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_prepare_latency": { 2024-04-07T16:16:16.397 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.397 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.397 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.397 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_process_latency": { 2024-04-07T16:16:16.397 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.397 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.397 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.397 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w": { 2024-04-07T16:16:16.397 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 10 2024-04-07T16:16:16.397 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.397 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_in_bytes": { 2024-04-07T16:16:16.398 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.398 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.398 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_latency": { 2024-04-07T16:16:16.398 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 10, 2024-04-07T16:16:16.398 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 238086038 2024-04-07T16:16:16.398 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.398 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_latency_in_bytes_histogram": { 2024-04-07T16:16:16.398 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.398 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.398 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_prepare_latency": { 2024-04-07T16:16:16.398 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 10, 2024-04-07T16:16:16.398 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 21363181 2024-04-07T16:16:16.398 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.398 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_process_latency": { 2024-04-07T16:16:16.398 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 10, 2024-04-07T16:16:16.399 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 163225415 2024-04-07T16:16:16.399 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.399 INFO:tasks.workunit.client.0.smithi100.stdout: "op_wip": { 2024-04-07T16:16:16.399 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.399 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.399 INFO:tasks.workunit.client.0.smithi100.stdout: "recovery_bytes": { 2024-04-07T16:16:16.399 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.399 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.399 INFO:tasks.workunit.client.0.smithi100.stdout: "recovery_ops": { 2024-04-07T16:16:16.399 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.399 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.399 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_bytes": { 2024-04-07T16:16:16.399 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 95995035648 2024-04-07T16:16:16.399 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.399 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_bytes_used": { 2024-04-07T16:16:16.399 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 32571392 2024-04-07T16:16:16.399 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:16.400 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:16.400 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.400 INFO:tasks.workunit.client.0.smithi100.stdout: "osd.3": { 2024-04-07T16:16:16.400 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs": { 2024-04-07T16:16:16.400 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_slow_fallback": { 2024-04-07T16:16:16.400 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.400 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.400 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_slow_size_fallback": { 2024-04-07T16:16:16.400 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.400 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.400 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit_db": { 2024-04-07T16:16:16.400 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 65536 2024-04-07T16:16:16.400 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.400 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit_main": { 2024-04-07T16:16:16.400 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.400 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.400 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit_wal": { 2024-04-07T16:16:16.401 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.401 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.401 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_written_slow": { 2024-04-07T16:16:16.401 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.401 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.401 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_written_sst": { 2024-04-07T16:16:16.401 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 12288 2024-04-07T16:16:16.401 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.401 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_written_wal": { 2024-04-07T16:16:16.401 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1110016 2024-04-07T16:16:16.401 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.401 INFO:tasks.workunit.client.0.smithi100.stdout: "compact_lat": { 2024-04-07T16:16:16.401 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.401 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.401 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.401 INFO:tasks.workunit.client.0.smithi100.stdout: "compact_lock_lat": { 2024-04-07T16:16:16.401 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.402 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.402 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.402 INFO:tasks.workunit.client.0.smithi100.stdout: "db_total_bytes": { 2024-04-07T16:16:16.402 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 95995035648 2024-04-07T16:16:16.402 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.402 INFO:tasks.workunit.client.0.smithi100.stdout: "db_used_bytes": { 2024-04-07T16:16:16.402 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 27656192 2024-04-07T16:16:16.402 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.402 INFO:tasks.workunit.client.0.smithi100.stdout: "log_bytes": { 2024-04-07T16:16:16.402 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 802816 2024-04-07T16:16:16.402 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.402 INFO:tasks.workunit.client.0.smithi100.stdout: "logged_bytes": { 2024-04-07T16:16:16.402 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 663552 2024-04-07T16:16:16.402 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.402 INFO:tasks.workunit.client.0.smithi100.stdout: "max_bytes_db": { 2024-04-07T16:16:16.403 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 31850496 2024-04-07T16:16:16.403 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.403 INFO:tasks.workunit.client.0.smithi100.stdout: "max_bytes_slow": { 2024-04-07T16:16:16.403 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.403 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.403 INFO:tasks.workunit.client.0.smithi100.stdout: "max_bytes_wal": { 2024-04-07T16:16:16.403 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.403 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.403 INFO:tasks.workunit.client.0.smithi100.stdout: "num_files": { 2024-04-07T16:16:16.403 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 14 2024-04-07T16:16:16.403 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.403 INFO:tasks.workunit.client.0.smithi100.stdout: "read_bytes": { 2024-04-07T16:16:16.403 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 206272 2024-04-07T16:16:16.403 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.403 INFO:tasks.workunit.client.0.smithi100.stdout: "read_count": { 2024-04-07T16:16:16.403 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 23 2024-04-07T16:16:16.404 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.404 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes": { 2024-04-07T16:16:16.404 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1282048 2024-04-07T16:16:16.404 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.404 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes_db": { 2024-04-07T16:16:16.404 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1286144 2024-04-07T16:16:16.404 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.404 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes_slow": { 2024-04-07T16:16:16.404 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.404 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.404 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes_wal": { 2024-04-07T16:16:16.404 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.404 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.404 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_count": { 2024-04-07T16:16:16.404 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 15 2024-04-07T16:16:16.404 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.405 INFO:tasks.workunit.client.0.smithi100.stdout: "read_prefetch_bytes": { 2024-04-07T16:16:16.405 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4913 2024-04-07T16:16:16.405 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.405 INFO:tasks.workunit.client.0.smithi100.stdout: "read_prefetch_count": { 2024-04-07T16:16:16.405 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4 2024-04-07T16:16:16.405 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.405 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_buffer_bytes": { 2024-04-07T16:16:16.405 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5503 2024-04-07T16:16:16.405 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.405 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_buffer_count": { 2024-04-07T16:16:16.405 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 24 2024-04-07T16:16:16.405 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.405 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_bytes": { 2024-04-07T16:16:16.405 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5503 2024-04-07T16:16:16.405 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.405 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_count": { 2024-04-07T16:16:16.405 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 24 2024-04-07T16:16:16.406 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.406 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes": { 2024-04-07T16:16:16.406 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.406 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.406 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes_db": { 2024-04-07T16:16:16.406 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.406 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.406 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes_slow": { 2024-04-07T16:16:16.406 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.406 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.406 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes_wal": { 2024-04-07T16:16:16.406 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.406 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.406 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_count": { 2024-04-07T16:16:16.406 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.407 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.407 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_total_bytes": { 2024-04-07T16:16:16.407 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.407 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.407 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_used_bytes": { 2024-04-07T16:16:16.407 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.407 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.407 INFO:tasks.workunit.client.0.smithi100.stdout: "wal_total_bytes": { 2024-04-07T16:16:16.407 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.407 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.407 INFO:tasks.workunit.client.0.smithi100.stdout: "wal_used_bytes": { 2024-04-07T16:16:16.407 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.407 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.407 INFO:tasks.workunit.client.0.smithi100.stdout: "write_bytes": { 2024-04-07T16:16:16.407 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1851392 2024-04-07T16:16:16.407 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:16.408 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.408 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore": { 2024-04-07T16:16:16.408 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit": { 2024-04-07T16:16:16.408 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4096 2024-04-07T16:16:16.408 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.408 INFO:tasks.workunit.client.0.smithi100.stdout: "allocated": { 2024-04-07T16:16:16.408 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 729088 2024-04-07T16:16:16.408 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.408 INFO:tasks.workunit.client.0.smithi100.stdout: "clist_lat": { 2024-04-07T16:16:16.408 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.408 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.408 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.408 INFO:tasks.workunit.client.0.smithi100.stdout: "compress_lat": { 2024-04-07T16:16:16.408 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.408 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.408 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.409 INFO:tasks.workunit.client.0.smithi100.stdout: "compressed": { 2024-04-07T16:16:16.409 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.409 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.409 INFO:tasks.workunit.client.0.smithi100.stdout: "compressed_allocated": { 2024-04-07T16:16:16.409 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.409 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.409 INFO:tasks.workunit.client.0.smithi100.stdout: "compressed_original": { 2024-04-07T16:16:16.409 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.409 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.409 INFO:tasks.workunit.client.0.smithi100.stdout: "csum_lat": { 2024-04-07T16:16:16.409 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 3, 2024-04-07T16:16:16.409 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5389 2024-04-07T16:16:16.409 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.409 INFO:tasks.workunit.client.0.smithi100.stdout: "decompress_lat": { 2024-04-07T16:16:16.409 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.409 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.410 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.410 INFO:tasks.workunit.client.0.smithi100.stdout: "fragmentation_micros": { 2024-04-07T16:16:16.410 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.410 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.410 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_commit_lat": { 2024-04-07T16:16:16.410 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 149, 2024-04-07T16:16:16.410 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 61891663 2024-04-07T16:16:16.410 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.410 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_final_lat": { 2024-04-07T16:16:16.410 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 137, 2024-04-07T16:16:16.410 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 21858858 2024-04-07T16:16:16.410 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.410 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_flush_lat": { 2024-04-07T16:16:16.410 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 149, 2024-04-07T16:16:16.410 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 396948 2024-04-07T16:16:16.411 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.411 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_sync_lat": { 2024-04-07T16:16:16.411 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 149, 2024-04-07T16:16:16.411 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 62288611 2024-04-07T16:16:16.411 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.411 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_get_keys_lat": { 2024-04-07T16:16:16.411 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.411 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.411 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.411 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_get_values_lat": { 2024-04-07T16:16:16.411 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1, 2024-04-07T16:16:16.411 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 62841 2024-04-07T16:16:16.411 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.411 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_lower_bound_lat": { 2024-04-07T16:16:16.411 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.411 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.412 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.412 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_next_lat": { 2024-04-07T16:16:16.412 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.412 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.412 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.412 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_seek_to_first_lat": { 2024-04-07T16:16:16.412 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.412 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.412 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.412 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_upper_bound_lat": { 2024-04-07T16:16:16.412 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 2, 2024-04-07T16:16:16.412 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4744 2024-04-07T16:16:16.412 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.412 INFO:tasks.workunit.client.0.smithi100.stdout: "onode_hits": { 2024-04-07T16:16:16.412 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 347 2024-04-07T16:16:16.412 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.412 INFO:tasks.workunit.client.0.smithi100.stdout: "onode_misses": { 2024-04-07T16:16:16.413 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 218 2024-04-07T16:16:16.413 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.413 INFO:tasks.workunit.client.0.smithi100.stdout: "read_lat": { 2024-04-07T16:16:16.413 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 86, 2024-04-07T16:16:16.413 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1300527 2024-04-07T16:16:16.413 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.413 INFO:tasks.workunit.client.0.smithi100.stdout: "read_onode_meta_lat": { 2024-04-07T16:16:16.413 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 150, 2024-04-07T16:16:16.413 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 440282 2024-04-07T16:16:16.413 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.413 INFO:tasks.workunit.client.0.smithi100.stdout: "read_wait_aio_lat": { 2024-04-07T16:16:16.413 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 64, 2024-04-07T16:16:16.413 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 474532 2024-04-07T16:16:16.413 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.413 INFO:tasks.workunit.client.0.smithi100.stdout: "reads_with_retries": { 2024-04-07T16:16:16.414 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.414 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.414 INFO:tasks.workunit.client.0.smithi100.stdout: "remove_lat": { 2024-04-07T16:16:16.414 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.414 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.414 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.414 INFO:tasks.workunit.client.0.smithi100.stdout: "state_aio_wait_lat": { 2024-04-07T16:16:16.414 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 244, 2024-04-07T16:16:16.414 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 8191326 2024-04-07T16:16:16.414 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.414 INFO:tasks.workunit.client.0.smithi100.stdout: "state_deferred_aio_wait_lat": { 2024-04-07T16:16:16.414 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 52, 2024-04-07T16:16:16.414 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5014766 2024-04-07T16:16:16.415 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.415 INFO:tasks.workunit.client.0.smithi100.stdout: "state_deferred_cleanup_lat": { 2024-04-07T16:16:16.415 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 49, 2024-04-07T16:16:16.415 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 474728018464 2024-04-07T16:16:16.415 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.415 INFO:tasks.workunit.client.0.smithi100.stdout: "state_deferred_queued_lat": { 2024-04-07T16:16:16.415 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 52, 2024-04-07T16:16:16.415 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 83299056736 2024-04-07T16:16:16.415 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.415 INFO:tasks.workunit.client.0.smithi100.stdout: "state_done_lat": { 2024-04-07T16:16:16.415 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 241, 2024-04-07T16:16:16.415 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 828537 2024-04-07T16:16:16.415 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.415 INFO:tasks.workunit.client.0.smithi100.stdout: "state_finishing_lat": { 2024-04-07T16:16:16.416 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 241, 2024-04-07T16:16:16.416 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 34493 2024-04-07T16:16:16.416 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.416 INFO:tasks.workunit.client.0.smithi100.stdout: "state_io_done_lat": { 2024-04-07T16:16:16.416 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 244, 2024-04-07T16:16:16.416 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 468909 2024-04-07T16:16:16.416 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.416 INFO:tasks.workunit.client.0.smithi100.stdout: "state_kv_commiting_lat": { 2024-04-07T16:16:16.416 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 244, 2024-04-07T16:16:16.416 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 305726439 2024-04-07T16:16:16.416 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.416 INFO:tasks.workunit.client.0.smithi100.stdout: "state_kv_done_lat": { 2024-04-07T16:16:16.416 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 244, 2024-04-07T16:16:16.416 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 103204 2024-04-07T16:16:16.416 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.417 INFO:tasks.workunit.client.0.smithi100.stdout: "state_kv_queued_lat": { 2024-04-07T16:16:16.417 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 244, 2024-04-07T16:16:16.417 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 136509628 2024-04-07T16:16:16.417 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.417 INFO:tasks.workunit.client.0.smithi100.stdout: "state_prepare_lat": { 2024-04-07T16:16:16.417 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 244, 2024-04-07T16:16:16.417 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 22445973 2024-04-07T16:16:16.417 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.417 INFO:tasks.workunit.client.0.smithi100.stdout: "stored": { 2024-04-07T16:16:16.417 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 311968 2024-04-07T16:16:16.417 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.417 INFO:tasks.workunit.client.0.smithi100.stdout: "truncate_lat": { 2024-04-07T16:16:16.417 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 43, 2024-04-07T16:16:16.417 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 379725 2024-04-07T16:16:16.417 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.418 INFO:tasks.workunit.client.0.smithi100.stdout: "txc_commit_lat": { 2024-04-07T16:16:16.418 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 244, 2024-04-07T16:16:16.418 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 473385875 2024-04-07T16:16:16.418 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.418 INFO:tasks.workunit.client.0.smithi100.stdout: "txc_submit_lat": { 2024-04-07T16:16:16.418 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 244, 2024-04-07T16:16:16.418 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 27691111 2024-04-07T16:16:16.418 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.418 INFO:tasks.workunit.client.0.smithi100.stdout: "txc_throttle_lat": { 2024-04-07T16:16:16.418 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 244, 2024-04-07T16:16:16.418 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2138648 2024-04-07T16:16:16.418 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:16.418 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.418 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache": { 2024-04-07T16:16:16.418 INFO:tasks.workunit.client.0.smithi100.stdout: "cache_bytes": { 2024-04-07T16:16:16.419 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1900954368 2024-04-07T16:16:16.419 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.419 INFO:tasks.workunit.client.0.smithi100.stdout: "heap_bytes": { 2024-04-07T16:16:16.419 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 42319872 2024-04-07T16:16:16.419 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.419 INFO:tasks.workunit.client.0.smithi100.stdout: "mapped_bytes": { 2024-04-07T16:16:16.419 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 41598976 2024-04-07T16:16:16.419 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.419 INFO:tasks.workunit.client.0.smithi100.stdout: "target_bytes": { 2024-04-07T16:16:16.419 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3183836160 2024-04-07T16:16:16.419 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.419 INFO:tasks.workunit.client.0.smithi100.stdout: "unmapped_bytes": { 2024-04-07T16:16:16.419 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 720896 2024-04-07T16:16:16.419 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:16.419 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.420 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:data": { 2024-04-07T16:16:16.420 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T16:16:16.420 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 159383552 2024-04-07T16:16:16.420 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.420 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T16:16:16.420 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.420 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.420 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T16:16:16.420 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.420 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.420 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T16:16:16.420 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 87869296 2024-04-07T16:16:16.420 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.420 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T16:16:16.420 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.421 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.421 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T16:16:16.421 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.421 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.421 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T16:16:16.421 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.421 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.421 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T16:16:16.421 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.421 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.421 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T16:16:16.421 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.421 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.421 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T16:16:16.422 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.422 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.422 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T16:16:16.422 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.422 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.422 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T16:16:16.422 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.422 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.422 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T16:16:16.422 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.422 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.422 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T16:16:16.422 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 71514256 2024-04-07T16:16:16.422 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:16.423 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.423 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:kv": { 2024-04-07T16:16:16.423 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T16:16:16.423 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 796917760 2024-04-07T16:16:16.423 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.423 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T16:16:16.423 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 67108864 2024-04-07T16:16:16.423 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.423 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T16:16:16.423 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.423 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.423 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T16:16:16.423 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 659019724 2024-04-07T16:16:16.423 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.423 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T16:16:16.424 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.424 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.424 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T16:16:16.424 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.424 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.424 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T16:16:16.424 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.424 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.424 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T16:16:16.424 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.424 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.424 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T16:16:16.424 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.424 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.425 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T16:16:16.425 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2128 2024-04-07T16:16:16.425 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.425 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T16:16:16.425 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.425 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.425 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T16:16:16.425 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.425 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.425 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T16:16:16.425 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.425 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.425 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T16:16:16.425 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 70787044 2024-04-07T16:16:16.425 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:16.426 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.426 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:kv_onode": { 2024-04-07T16:16:16.426 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T16:16:16.426 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 192937984 2024-04-07T16:16:16.426 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.426 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T16:16:16.426 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 67108864 2024-04-07T16:16:16.426 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.426 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T16:16:16.426 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 464 2024-04-07T16:16:16.426 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.426 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T16:16:16.426 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 58579531 2024-04-07T16:16:16.426 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.427 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T16:16:16.427 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.427 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.427 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T16:16:16.427 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.427 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.427 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T16:16:16.427 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.427 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.427 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T16:16:16.427 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.427 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.427 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T16:16:16.427 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.427 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.427 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T16:16:16.428 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.428 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.428 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T16:16:16.428 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.428 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.428 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T16:16:16.428 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.428 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.428 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T16:16:16.428 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.428 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.428 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T16:16:16.428 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 67249125 2024-04-07T16:16:16.428 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:16.428 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.429 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:meta": { 2024-04-07T16:16:16.429 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T16:16:16.429 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 729808896 2024-04-07T16:16:16.429 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.429 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T16:16:16.429 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.429 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.429 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T16:16:16.429 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.429 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.429 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T16:16:16.429 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 659019724 2024-04-07T16:16:16.429 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.429 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T16:16:16.429 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.430 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.430 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T16:16:16.430 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.430 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.430 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T16:16:16.430 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.430 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.430 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T16:16:16.430 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.430 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.430 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T16:16:16.430 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.430 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.430 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T16:16:16.430 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 218323 2024-04-07T16:16:16.431 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.431 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T16:16:16.431 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.431 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.431 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T16:16:16.431 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.431 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.431 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T16:16:16.431 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.431 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.431 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T16:16:16.431 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 70570849 2024-04-07T16:16:16.431 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:16.431 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.431 INFO:tasks.workunit.client.0.smithi100.stdout: "objecter": { 2024-04-07T16:16:16.432 INFO:tasks.workunit.client.0.smithi100.stdout: "op_active": { 2024-04-07T16:16:16.432 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.432 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.432 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r": { 2024-04-07T16:16:16.432 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.432 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.432 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rmw": { 2024-04-07T16:16:16.432 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.432 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.432 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w": { 2024-04-07T16:16:16.432 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.432 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:16.432 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.432 INFO:tasks.workunit.client.0.smithi100.stdout: "osd": { 2024-04-07T16:16:16.432 INFO:tasks.workunit.client.0.smithi100.stdout: "numpg": { 2024-04-07T16:16:16.433 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 36 2024-04-07T16:16:16.433 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.433 INFO:tasks.workunit.client.0.smithi100.stdout: "numpg_removing": { 2024-04-07T16:16:16.433 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.433 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.433 INFO:tasks.workunit.client.0.smithi100.stdout: "op": { 2024-04-07T16:16:16.433 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.433 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.433 INFO:tasks.workunit.client.0.smithi100.stdout: "op_delayed_degraded": { 2024-04-07T16:16:16.433 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.433 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.433 INFO:tasks.workunit.client.0.smithi100.stdout: "op_delayed_unreadable": { 2024-04-07T16:16:16.433 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.433 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.433 INFO:tasks.workunit.client.0.smithi100.stdout: "op_in_bytes": { 2024-04-07T16:16:16.433 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.434 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.434 INFO:tasks.workunit.client.0.smithi100.stdout: "op_latency": { 2024-04-07T16:16:16.434 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.434 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.434 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.434 INFO:tasks.workunit.client.0.smithi100.stdout: "op_out_bytes": { 2024-04-07T16:16:16.434 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.434 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.434 INFO:tasks.workunit.client.0.smithi100.stdout: "op_prepare_latency": { 2024-04-07T16:16:16.434 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 2, 2024-04-07T16:16:16.434 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1073024 2024-04-07T16:16:16.434 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.434 INFO:tasks.workunit.client.0.smithi100.stdout: "op_process_latency": { 2024-04-07T16:16:16.434 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.435 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.435 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.435 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r": { 2024-04-07T16:16:16.435 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.435 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.435 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_latency": { 2024-04-07T16:16:16.435 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.435 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.435 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.435 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_latency_out_bytes_histogram": { 2024-04-07T16:16:16.435 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.435 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.435 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_out_bytes": { 2024-04-07T16:16:16.435 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.435 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.435 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_prepare_latency": { 2024-04-07T16:16:16.436 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.436 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.436 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.436 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_process_latency": { 2024-04-07T16:16:16.436 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.436 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.436 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.436 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw": { 2024-04-07T16:16:16.436 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.436 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.436 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_in_bytes": { 2024-04-07T16:16:16.436 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.436 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.436 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_latency": { 2024-04-07T16:16:16.436 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.437 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.437 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.437 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_latency_in_bytes_histogram": { 2024-04-07T16:16:16.437 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.437 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.437 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_latency_out_bytes_histogram": { 2024-04-07T16:16:16.437 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.437 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.437 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_out_bytes": { 2024-04-07T16:16:16.437 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.437 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.437 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_prepare_latency": { 2024-04-07T16:16:16.437 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.437 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.437 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.438 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_process_latency": { 2024-04-07T16:16:16.438 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.438 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.438 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.438 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w": { 2024-04-07T16:16:16.438 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.438 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.438 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_in_bytes": { 2024-04-07T16:16:16.438 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.438 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.438 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_latency": { 2024-04-07T16:16:16.438 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.438 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.438 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.438 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_latency_in_bytes_histogram": { 2024-04-07T16:16:16.439 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.439 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.439 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_prepare_latency": { 2024-04-07T16:16:16.439 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 2, 2024-04-07T16:16:16.439 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1073024 2024-04-07T16:16:16.439 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.439 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_process_latency": { 2024-04-07T16:16:16.439 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.439 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.439 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.439 INFO:tasks.workunit.client.0.smithi100.stdout: "op_wip": { 2024-04-07T16:16:16.439 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.439 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.439 INFO:tasks.workunit.client.0.smithi100.stdout: "recovery_bytes": { 2024-04-07T16:16:16.439 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.440 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.440 INFO:tasks.workunit.client.0.smithi100.stdout: "recovery_ops": { 2024-04-07T16:16:16.440 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.440 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.440 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_bytes": { 2024-04-07T16:16:16.440 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 95995035648 2024-04-07T16:16:16.440 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.440 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_bytes_used": { 2024-04-07T16:16:16.440 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 28393472 2024-04-07T16:16:16.440 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:16.440 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:16.440 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.440 INFO:tasks.workunit.client.0.smithi100.stdout: "osd.4": { 2024-04-07T16:16:16.440 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs": { 2024-04-07T16:16:16.440 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_slow_fallback": { 2024-04-07T16:16:16.441 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.441 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.441 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_slow_size_fallback": { 2024-04-07T16:16:16.441 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.441 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.441 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit_db": { 2024-04-07T16:16:16.441 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 65536 2024-04-07T16:16:16.441 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.441 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit_main": { 2024-04-07T16:16:16.441 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.441 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.441 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit_wal": { 2024-04-07T16:16:16.441 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.441 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.441 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_written_slow": { 2024-04-07T16:16:16.442 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.442 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.442 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_written_sst": { 2024-04-07T16:16:16.442 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 12288 2024-04-07T16:16:16.442 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.442 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_written_wal": { 2024-04-07T16:16:16.442 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 8314880 2024-04-07T16:16:16.442 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.442 INFO:tasks.workunit.client.0.smithi100.stdout: "compact_lat": { 2024-04-07T16:16:16.442 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.442 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.442 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.442 INFO:tasks.workunit.client.0.smithi100.stdout: "compact_lock_lat": { 2024-04-07T16:16:16.442 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.443 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.443 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.443 INFO:tasks.workunit.client.0.smithi100.stdout: "db_total_bytes": { 2024-04-07T16:16:16.443 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 95995035648 2024-04-07T16:16:16.443 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.443 INFO:tasks.workunit.client.0.smithi100.stdout: "db_used_bytes": { 2024-04-07T16:16:16.443 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 31850496 2024-04-07T16:16:16.443 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.443 INFO:tasks.workunit.client.0.smithi100.stdout: "log_bytes": { 2024-04-07T16:16:16.443 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5283840 2024-04-07T16:16:16.443 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.443 INFO:tasks.workunit.client.0.smithi100.stdout: "logged_bytes": { 2024-04-07T16:16:16.443 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5144576 2024-04-07T16:16:16.443 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.443 INFO:tasks.workunit.client.0.smithi100.stdout: "max_bytes_db": { 2024-04-07T16:16:16.444 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 31850496 2024-04-07T16:16:16.444 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.444 INFO:tasks.workunit.client.0.smithi100.stdout: "max_bytes_slow": { 2024-04-07T16:16:16.444 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.444 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.444 INFO:tasks.workunit.client.0.smithi100.stdout: "max_bytes_wal": { 2024-04-07T16:16:16.444 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.444 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.444 INFO:tasks.workunit.client.0.smithi100.stdout: "num_files": { 2024-04-07T16:16:16.444 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 14 2024-04-07T16:16:16.444 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.444 INFO:tasks.workunit.client.0.smithi100.stdout: "read_bytes": { 2024-04-07T16:16:16.444 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 206272 2024-04-07T16:16:16.444 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.444 INFO:tasks.workunit.client.0.smithi100.stdout: "read_count": { 2024-04-07T16:16:16.445 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 23 2024-04-07T16:16:16.445 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.445 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes": { 2024-04-07T16:16:16.445 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1282048 2024-04-07T16:16:16.445 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.445 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes_db": { 2024-04-07T16:16:16.445 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1286144 2024-04-07T16:16:16.445 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.445 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes_slow": { 2024-04-07T16:16:16.445 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.445 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.445 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes_wal": { 2024-04-07T16:16:16.445 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.445 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.445 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_count": { 2024-04-07T16:16:16.446 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 15 2024-04-07T16:16:16.446 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.446 INFO:tasks.workunit.client.0.smithi100.stdout: "read_prefetch_bytes": { 2024-04-07T16:16:16.446 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4913 2024-04-07T16:16:16.446 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.446 INFO:tasks.workunit.client.0.smithi100.stdout: "read_prefetch_count": { 2024-04-07T16:16:16.446 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4 2024-04-07T16:16:16.446 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.446 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_buffer_bytes": { 2024-04-07T16:16:16.446 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5505 2024-04-07T16:16:16.446 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.446 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_buffer_count": { 2024-04-07T16:16:16.446 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 24 2024-04-07T16:16:16.446 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.447 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_bytes": { 2024-04-07T16:16:16.447 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5505 2024-04-07T16:16:16.447 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.447 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_count": { 2024-04-07T16:16:16.447 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 24 2024-04-07T16:16:16.447 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.447 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes": { 2024-04-07T16:16:16.447 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.447 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.447 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes_db": { 2024-04-07T16:16:16.447 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.447 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.447 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes_slow": { 2024-04-07T16:16:16.447 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.447 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.448 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes_wal": { 2024-04-07T16:16:16.448 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.448 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.448 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_count": { 2024-04-07T16:16:16.448 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.448 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.448 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_total_bytes": { 2024-04-07T16:16:16.448 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.448 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.448 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_used_bytes": { 2024-04-07T16:16:16.448 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.448 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.448 INFO:tasks.workunit.client.0.smithi100.stdout: "wal_total_bytes": { 2024-04-07T16:16:16.448 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.449 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.449 INFO:tasks.workunit.client.0.smithi100.stdout: "wal_used_bytes": { 2024-04-07T16:16:16.449 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.449 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.449 INFO:tasks.workunit.client.0.smithi100.stdout: "write_bytes": { 2024-04-07T16:16:16.449 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 13537280 2024-04-07T16:16:16.449 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:16.449 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.449 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore": { 2024-04-07T16:16:16.449 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit": { 2024-04-07T16:16:16.449 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4096 2024-04-07T16:16:16.449 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.449 INFO:tasks.workunit.client.0.smithi100.stdout: "allocated": { 2024-04-07T16:16:16.449 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 724992 2024-04-07T16:16:16.449 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.450 INFO:tasks.workunit.client.0.smithi100.stdout: "clist_lat": { 2024-04-07T16:16:16.450 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 8, 2024-04-07T16:16:16.450 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 725717 2024-04-07T16:16:16.450 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.450 INFO:tasks.workunit.client.0.smithi100.stdout: "compress_lat": { 2024-04-07T16:16:16.450 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.450 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.450 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.450 INFO:tasks.workunit.client.0.smithi100.stdout: "compressed": { 2024-04-07T16:16:16.450 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.450 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.450 INFO:tasks.workunit.client.0.smithi100.stdout: "compressed_allocated": { 2024-04-07T16:16:16.450 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.450 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.451 INFO:tasks.workunit.client.0.smithi100.stdout: "compressed_original": { 2024-04-07T16:16:16.451 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.451 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.451 INFO:tasks.workunit.client.0.smithi100.stdout: "csum_lat": { 2024-04-07T16:16:16.451 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1, 2024-04-07T16:16:16.451 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2633 2024-04-07T16:16:16.451 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.451 INFO:tasks.workunit.client.0.smithi100.stdout: "decompress_lat": { 2024-04-07T16:16:16.451 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.451 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.451 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.451 INFO:tasks.workunit.client.0.smithi100.stdout: "fragmentation_micros": { 2024-04-07T16:16:16.451 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.451 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.451 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_commit_lat": { 2024-04-07T16:16:16.452 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1243, 2024-04-07T16:16:16.452 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 591333721 2024-04-07T16:16:16.452 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.452 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_final_lat": { 2024-04-07T16:16:16.452 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1238, 2024-04-07T16:16:16.452 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 32704757 2024-04-07T16:16:16.452 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.452 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_flush_lat": { 2024-04-07T16:16:16.452 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1243, 2024-04-07T16:16:16.452 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1896043 2024-04-07T16:16:16.452 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.452 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_sync_lat": { 2024-04-07T16:16:16.452 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1243, 2024-04-07T16:16:16.452 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 593229764 2024-04-07T16:16:16.453 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.453 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_get_keys_lat": { 2024-04-07T16:16:16.453 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.453 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.453 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.453 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_get_values_lat": { 2024-04-07T16:16:16.453 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 3, 2024-04-07T16:16:16.453 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 66079 2024-04-07T16:16:16.453 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.453 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_lower_bound_lat": { 2024-04-07T16:16:16.453 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.453 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.453 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.453 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_next_lat": { 2024-04-07T16:16:16.453 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.453 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.454 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.454 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_seek_to_first_lat": { 2024-04-07T16:16:16.454 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.454 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.454 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.454 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_upper_bound_lat": { 2024-04-07T16:16:16.454 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 2, 2024-04-07T16:16:16.454 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 7970 2024-04-07T16:16:16.454 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.454 INFO:tasks.workunit.client.0.smithi100.stdout: "onode_hits": { 2024-04-07T16:16:16.454 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3462 2024-04-07T16:16:16.454 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.454 INFO:tasks.workunit.client.0.smithi100.stdout: "onode_misses": { 2024-04-07T16:16:16.454 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 340 2024-04-07T16:16:16.454 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.455 INFO:tasks.workunit.client.0.smithi100.stdout: "read_lat": { 2024-04-07T16:16:16.455 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 96, 2024-04-07T16:16:16.455 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1472078 2024-04-07T16:16:16.455 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.455 INFO:tasks.workunit.client.0.smithi100.stdout: "read_onode_meta_lat": { 2024-04-07T16:16:16.455 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 162, 2024-04-07T16:16:16.455 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 507967 2024-04-07T16:16:16.455 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.455 INFO:tasks.workunit.client.0.smithi100.stdout: "read_wait_aio_lat": { 2024-04-07T16:16:16.455 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 66, 2024-04-07T16:16:16.455 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 172698 2024-04-07T16:16:16.455 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.455 INFO:tasks.workunit.client.0.smithi100.stdout: "reads_with_retries": { 2024-04-07T16:16:16.455 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.455 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.456 INFO:tasks.workunit.client.0.smithi100.stdout: "remove_lat": { 2024-04-07T16:16:16.456 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 105, 2024-04-07T16:16:16.456 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 9372306 2024-04-07T16:16:16.456 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.456 INFO:tasks.workunit.client.0.smithi100.stdout: "state_aio_wait_lat": { 2024-04-07T16:16:16.456 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 3362, 2024-04-07T16:16:16.456 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3286501480 2024-04-07T16:16:16.456 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.456 INFO:tasks.workunit.client.0.smithi100.stdout: "state_deferred_aio_wait_lat": { 2024-04-07T16:16:16.456 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 42, 2024-04-07T16:16:16.456 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 15424969 2024-04-07T16:16:16.456 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.456 INFO:tasks.workunit.client.0.smithi100.stdout: "state_deferred_cleanup_lat": { 2024-04-07T16:16:16.456 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 41, 2024-04-07T16:16:16.456 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 382467755091 2024-04-07T16:16:16.457 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.457 INFO:tasks.workunit.client.0.smithi100.stdout: "state_deferred_queued_lat": { 2024-04-07T16:16:16.457 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 42, 2024-04-07T16:16:16.457 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 60268645703 2024-04-07T16:16:16.457 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.457 INFO:tasks.workunit.client.0.smithi100.stdout: "state_done_lat": { 2024-04-07T16:16:16.457 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 3361, 2024-04-07T16:16:16.457 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1208218205 2024-04-07T16:16:16.457 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.457 INFO:tasks.workunit.client.0.smithi100.stdout: "state_finishing_lat": { 2024-04-07T16:16:16.457 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 3361, 2024-04-07T16:16:16.457 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 306962 2024-04-07T16:16:16.457 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.458 INFO:tasks.workunit.client.0.smithi100.stdout: "state_io_done_lat": { 2024-04-07T16:16:16.458 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 3362, 2024-04-07T16:16:16.458 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4984398 2024-04-07T16:16:16.458 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.458 INFO:tasks.workunit.client.0.smithi100.stdout: "state_kv_commiting_lat": { 2024-04-07T16:16:16.458 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 3362, 2024-04-07T16:16:16.458 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2870095345 2024-04-07T16:16:16.458 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.458 INFO:tasks.workunit.client.0.smithi100.stdout: "state_kv_done_lat": { 2024-04-07T16:16:16.458 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 3362, 2024-04-07T16:16:16.458 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 697077 2024-04-07T16:16:16.458 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.458 INFO:tasks.workunit.client.0.smithi100.stdout: "state_kv_queued_lat": { 2024-04-07T16:16:16.458 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 3362, 2024-04-07T16:16:16.459 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1718221786 2024-04-07T16:16:16.459 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.459 INFO:tasks.workunit.client.0.smithi100.stdout: "state_prepare_lat": { 2024-04-07T16:16:16.459 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 3362, 2024-04-07T16:16:16.459 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 371037756 2024-04-07T16:16:16.459 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.459 INFO:tasks.workunit.client.0.smithi100.stdout: "stored": { 2024-04-07T16:16:16.459 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 309940 2024-04-07T16:16:16.459 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.459 INFO:tasks.workunit.client.0.smithi100.stdout: "truncate_lat": { 2024-04-07T16:16:16.459 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 42, 2024-04-07T16:16:16.459 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 302711 2024-04-07T16:16:16.459 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.459 INFO:tasks.workunit.client.0.smithi100.stdout: "txc_commit_lat": { 2024-04-07T16:16:16.460 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 3362, 2024-04-07T16:16:16.460 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 8251174891 2024-04-07T16:16:16.460 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.460 INFO:tasks.workunit.client.0.smithi100.stdout: "txc_submit_lat": { 2024-04-07T16:16:16.460 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 3362, 2024-04-07T16:16:16.460 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 476238588 2024-04-07T16:16:16.460 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.460 INFO:tasks.workunit.client.0.smithi100.stdout: "txc_throttle_lat": { 2024-04-07T16:16:16.460 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 3362, 2024-04-07T16:16:16.460 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 165941416 2024-04-07T16:16:16.460 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:16.460 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.460 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache": { 2024-04-07T16:16:16.460 INFO:tasks.workunit.client.0.smithi100.stdout: "cache_bytes": { 2024-04-07T16:16:16.460 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1900990924 2024-04-07T16:16:16.461 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.461 INFO:tasks.workunit.client.0.smithi100.stdout: "heap_bytes": { 2024-04-07T16:16:16.461 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 46448640 2024-04-07T16:16:16.461 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.461 INFO:tasks.workunit.client.0.smithi100.stdout: "mapped_bytes": { 2024-04-07T16:16:16.461 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 45514752 2024-04-07T16:16:16.461 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.461 INFO:tasks.workunit.client.0.smithi100.stdout: "target_bytes": { 2024-04-07T16:16:16.461 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3183879168 2024-04-07T16:16:16.461 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.461 INFO:tasks.workunit.client.0.smithi100.stdout: "unmapped_bytes": { 2024-04-07T16:16:16.461 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 933888 2024-04-07T16:16:16.461 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:16.461 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.461 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:data": { 2024-04-07T16:16:16.462 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T16:16:16.462 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 159383552 2024-04-07T16:16:16.462 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.462 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T16:16:16.462 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.462 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.462 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T16:16:16.462 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.462 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.462 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T16:16:16.462 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 87864559 2024-04-07T16:16:16.462 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.462 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T16:16:16.462 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.462 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.463 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T16:16:16.463 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.463 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.463 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T16:16:16.463 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.463 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.463 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T16:16:16.463 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.463 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.463 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T16:16:16.463 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.463 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.463 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T16:16:16.463 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.464 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.464 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T16:16:16.464 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.464 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.464 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T16:16:16.464 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.464 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.464 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T16:16:16.464 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.464 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.464 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T16:16:16.464 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 71518993 2024-04-07T16:16:16.464 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:16.464 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.464 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:kv": { 2024-04-07T16:16:16.464 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T16:16:16.465 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 796917760 2024-04-07T16:16:16.465 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.465 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T16:16:16.465 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 67108864 2024-04-07T16:16:16.465 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.465 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T16:16:16.465 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.465 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.465 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T16:16:16.465 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 658984197 2024-04-07T16:16:16.465 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.465 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T16:16:16.465 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.465 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.465 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T16:16:16.466 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.466 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.466 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T16:16:16.466 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.466 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.466 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T16:16:16.466 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.466 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.466 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T16:16:16.466 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.466 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.466 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T16:16:16.466 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2128 2024-04-07T16:16:16.466 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.466 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T16:16:16.467 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.467 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.467 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T16:16:16.467 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.467 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.467 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T16:16:16.467 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.467 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.467 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T16:16:16.467 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 70822571 2024-04-07T16:16:16.467 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:16.467 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.467 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:kv_onode": { 2024-04-07T16:16:16.467 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T16:16:16.467 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 192937984 2024-04-07T16:16:16.468 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.468 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T16:16:16.468 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 67108864 2024-04-07T16:16:16.468 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.468 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T16:16:16.468 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 464 2024-04-07T16:16:16.468 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.468 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T16:16:16.468 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 58576373 2024-04-07T16:16:16.468 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.468 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T16:16:16.468 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.468 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.468 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T16:16:16.468 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.468 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.469 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T16:16:16.469 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.469 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.469 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T16:16:16.469 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.469 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.469 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T16:16:16.469 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.469 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.469 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T16:16:16.469 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.469 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.469 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T16:16:16.469 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.469 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.469 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T16:16:16.470 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.470 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.470 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T16:16:16.470 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.470 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.470 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T16:16:16.470 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 67252283 2024-04-07T16:16:16.470 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:16.470 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.470 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:meta": { 2024-04-07T16:16:16.470 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T16:16:16.470 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 729808896 2024-04-07T16:16:16.470 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.470 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T16:16:16.470 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.471 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.471 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T16:16:16.471 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.471 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.471 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T16:16:16.471 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 658984197 2024-04-07T16:16:16.471 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.471 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T16:16:16.471 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.471 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.471 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T16:16:16.471 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.471 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.471 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T16:16:16.471 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.472 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.472 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T16:16:16.472 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.472 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.472 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T16:16:16.472 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.472 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.472 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T16:16:16.472 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 331144 2024-04-07T16:16:16.472 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.472 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T16:16:16.472 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.472 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.472 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T16:16:16.472 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.472 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.473 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T16:16:16.473 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.473 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.473 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T16:16:16.473 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 70493555 2024-04-07T16:16:16.473 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:16.473 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.473 INFO:tasks.workunit.client.0.smithi100.stdout: "objecter": { 2024-04-07T16:16:16.473 INFO:tasks.workunit.client.0.smithi100.stdout: "op_active": { 2024-04-07T16:16:16.473 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.473 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.473 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r": { 2024-04-07T16:16:16.473 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.473 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.473 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rmw": { 2024-04-07T16:16:16.474 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.474 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.474 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w": { 2024-04-07T16:16:16.474 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.474 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:16.474 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.474 INFO:tasks.workunit.client.0.smithi100.stdout: "osd": { 2024-04-07T16:16:16.474 INFO:tasks.workunit.client.0.smithi100.stdout: "numpg": { 2024-04-07T16:16:16.474 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 40 2024-04-07T16:16:16.474 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.474 INFO:tasks.workunit.client.0.smithi100.stdout: "numpg_removing": { 2024-04-07T16:16:16.474 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.474 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.474 INFO:tasks.workunit.client.0.smithi100.stdout: "op": { 2024-04-07T16:16:16.474 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1 2024-04-07T16:16:16.475 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.475 INFO:tasks.workunit.client.0.smithi100.stdout: "op_delayed_degraded": { 2024-04-07T16:16:16.475 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.475 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.475 INFO:tasks.workunit.client.0.smithi100.stdout: "op_delayed_unreadable": { 2024-04-07T16:16:16.475 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.475 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.475 INFO:tasks.workunit.client.0.smithi100.stdout: "op_in_bytes": { 2024-04-07T16:16:16.475 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 22 2024-04-07T16:16:16.475 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.475 INFO:tasks.workunit.client.0.smithi100.stdout: "op_latency": { 2024-04-07T16:16:16.475 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1, 2024-04-07T16:16:16.475 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 10592946 2024-04-07T16:16:16.475 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.476 INFO:tasks.workunit.client.0.smithi100.stdout: "op_out_bytes": { 2024-04-07T16:16:16.476 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.476 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.476 INFO:tasks.workunit.client.0.smithi100.stdout: "op_prepare_latency": { 2024-04-07T16:16:16.476 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1, 2024-04-07T16:16:16.476 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 622928 2024-04-07T16:16:16.476 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.476 INFO:tasks.workunit.client.0.smithi100.stdout: "op_process_latency": { 2024-04-07T16:16:16.476 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1, 2024-04-07T16:16:16.476 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 10505965 2024-04-07T16:16:16.476 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.476 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r": { 2024-04-07T16:16:16.476 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.476 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.476 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_latency": { 2024-04-07T16:16:16.476 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.477 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.477 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.477 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_latency_out_bytes_histogram": { 2024-04-07T16:16:16.477 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.477 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.477 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_out_bytes": { 2024-04-07T16:16:16.477 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.477 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.477 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_prepare_latency": { 2024-04-07T16:16:16.477 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.477 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.477 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.477 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_process_latency": { 2024-04-07T16:16:16.477 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.477 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.478 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.478 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw": { 2024-04-07T16:16:16.478 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.478 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.478 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_in_bytes": { 2024-04-07T16:16:16.478 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.478 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.478 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_latency": { 2024-04-07T16:16:16.478 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.478 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.478 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.478 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_latency_in_bytes_histogram": { 2024-04-07T16:16:16.478 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.478 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.478 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_latency_out_bytes_histogram": { 2024-04-07T16:16:16.479 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.479 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.479 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_out_bytes": { 2024-04-07T16:16:16.479 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.479 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.479 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_prepare_latency": { 2024-04-07T16:16:16.479 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.479 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.479 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.479 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_process_latency": { 2024-04-07T16:16:16.479 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.479 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.479 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.479 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w": { 2024-04-07T16:16:16.479 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1 2024-04-07T16:16:16.479 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.479 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_in_bytes": { 2024-04-07T16:16:16.480 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 22 2024-04-07T16:16:16.480 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.480 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_latency": { 2024-04-07T16:16:16.480 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1, 2024-04-07T16:16:16.480 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 10592946 2024-04-07T16:16:16.480 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.480 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_latency_in_bytes_histogram": { 2024-04-07T16:16:16.480 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.480 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.480 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_prepare_latency": { 2024-04-07T16:16:16.480 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1, 2024-04-07T16:16:16.480 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 622928 2024-04-07T16:16:16.480 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.480 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_process_latency": { 2024-04-07T16:16:16.480 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1, 2024-04-07T16:16:16.480 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 10505965 2024-04-07T16:16:16.481 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.481 INFO:tasks.workunit.client.0.smithi100.stdout: "op_wip": { 2024-04-07T16:16:16.481 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.481 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.481 INFO:tasks.workunit.client.0.smithi100.stdout: "recovery_bytes": { 2024-04-07T16:16:16.481 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.481 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.481 INFO:tasks.workunit.client.0.smithi100.stdout: "recovery_ops": { 2024-04-07T16:16:16.481 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.481 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.481 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_bytes": { 2024-04-07T16:16:16.481 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 95995035648 2024-04-07T16:16:16.481 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.481 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_bytes_used": { 2024-04-07T16:16:16.481 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 32587776 2024-04-07T16:16:16.481 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:16.482 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:16.482 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.482 INFO:tasks.workunit.client.0.smithi100.stdout: "osd.5": { 2024-04-07T16:16:16.482 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs": { 2024-04-07T16:16:16.482 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_slow_fallback": { 2024-04-07T16:16:16.482 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.482 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.482 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_slow_size_fallback": { 2024-04-07T16:16:16.482 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.482 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.482 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit_db": { 2024-04-07T16:16:16.482 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 65536 2024-04-07T16:16:16.482 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.482 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit_main": { 2024-04-07T16:16:16.483 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.483 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.483 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit_wal": { 2024-04-07T16:16:16.483 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.483 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.483 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_written_slow": { 2024-04-07T16:16:16.483 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.483 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.483 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_written_sst": { 2024-04-07T16:16:16.483 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 12288 2024-04-07T16:16:16.483 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.483 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_written_wal": { 2024-04-07T16:16:16.483 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1191936 2024-04-07T16:16:16.483 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.483 INFO:tasks.workunit.client.0.smithi100.stdout: "compact_lat": { 2024-04-07T16:16:16.483 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.484 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.484 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.484 INFO:tasks.workunit.client.0.smithi100.stdout: "compact_lock_lat": { 2024-04-07T16:16:16.484 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.484 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.484 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.484 INFO:tasks.workunit.client.0.smithi100.stdout: "db_total_bytes": { 2024-04-07T16:16:16.484 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 95995035648 2024-04-07T16:16:16.484 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.484 INFO:tasks.workunit.client.0.smithi100.stdout: "db_used_bytes": { 2024-04-07T16:16:16.484 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 27656192 2024-04-07T16:16:16.484 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.484 INFO:tasks.workunit.client.0.smithi100.stdout: "log_bytes": { 2024-04-07T16:16:16.484 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 880640 2024-04-07T16:16:16.485 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.485 INFO:tasks.workunit.client.0.smithi100.stdout: "logged_bytes": { 2024-04-07T16:16:16.485 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 741376 2024-04-07T16:16:16.485 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.485 INFO:tasks.workunit.client.0.smithi100.stdout: "max_bytes_db": { 2024-04-07T16:16:16.485 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 31850496 2024-04-07T16:16:16.485 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.485 INFO:tasks.workunit.client.0.smithi100.stdout: "max_bytes_slow": { 2024-04-07T16:16:16.485 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.485 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.485 INFO:tasks.workunit.client.0.smithi100.stdout: "max_bytes_wal": { 2024-04-07T16:16:16.485 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.485 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.485 INFO:tasks.workunit.client.0.smithi100.stdout: "num_files": { 2024-04-07T16:16:16.485 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 14 2024-04-07T16:16:16.485 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.486 INFO:tasks.workunit.client.0.smithi100.stdout: "read_bytes": { 2024-04-07T16:16:16.486 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 206272 2024-04-07T16:16:16.486 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.486 INFO:tasks.workunit.client.0.smithi100.stdout: "read_count": { 2024-04-07T16:16:16.486 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 23 2024-04-07T16:16:16.486 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.486 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes": { 2024-04-07T16:16:16.486 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1282048 2024-04-07T16:16:16.486 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.486 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes_db": { 2024-04-07T16:16:16.486 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1286144 2024-04-07T16:16:16.486 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.486 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes_slow": { 2024-04-07T16:16:16.486 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.486 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.487 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes_wal": { 2024-04-07T16:16:16.487 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.487 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.487 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_count": { 2024-04-07T16:16:16.487 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 15 2024-04-07T16:16:16.487 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.487 INFO:tasks.workunit.client.0.smithi100.stdout: "read_prefetch_bytes": { 2024-04-07T16:16:16.487 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4913 2024-04-07T16:16:16.487 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.487 INFO:tasks.workunit.client.0.smithi100.stdout: "read_prefetch_count": { 2024-04-07T16:16:16.487 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4 2024-04-07T16:16:16.487 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.487 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_buffer_bytes": { 2024-04-07T16:16:16.487 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5505 2024-04-07T16:16:16.488 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.488 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_buffer_count": { 2024-04-07T16:16:16.488 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 24 2024-04-07T16:16:16.488 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.488 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_bytes": { 2024-04-07T16:16:16.488 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5505 2024-04-07T16:16:16.488 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.488 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_count": { 2024-04-07T16:16:16.488 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 24 2024-04-07T16:16:16.488 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.488 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes": { 2024-04-07T16:16:16.488 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.488 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.488 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes_db": { 2024-04-07T16:16:16.488 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.488 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.489 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes_slow": { 2024-04-07T16:16:16.489 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.489 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.489 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes_wal": { 2024-04-07T16:16:16.489 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.489 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.489 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_count": { 2024-04-07T16:16:16.489 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.489 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.489 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_total_bytes": { 2024-04-07T16:16:16.489 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.489 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.489 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_used_bytes": { 2024-04-07T16:16:16.489 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.490 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.490 INFO:tasks.workunit.client.0.smithi100.stdout: "wal_total_bytes": { 2024-04-07T16:16:16.490 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.490 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.490 INFO:tasks.workunit.client.0.smithi100.stdout: "wal_used_bytes": { 2024-04-07T16:16:16.490 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.490 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.490 INFO:tasks.workunit.client.0.smithi100.stdout: "write_bytes": { 2024-04-07T16:16:16.490 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2011136 2024-04-07T16:16:16.490 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:16.490 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.490 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore": { 2024-04-07T16:16:16.490 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit": { 2024-04-07T16:16:16.490 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4096 2024-04-07T16:16:16.491 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.491 INFO:tasks.workunit.client.0.smithi100.stdout: "allocated": { 2024-04-07T16:16:16.491 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 720896 2024-04-07T16:16:16.491 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.491 INFO:tasks.workunit.client.0.smithi100.stdout: "clist_lat": { 2024-04-07T16:16:16.491 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 4, 2024-04-07T16:16:16.491 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 270895 2024-04-07T16:16:16.491 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.491 INFO:tasks.workunit.client.0.smithi100.stdout: "compress_lat": { 2024-04-07T16:16:16.491 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.491 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.491 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.491 INFO:tasks.workunit.client.0.smithi100.stdout: "compressed": { 2024-04-07T16:16:16.491 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.491 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.492 INFO:tasks.workunit.client.0.smithi100.stdout: "compressed_allocated": { 2024-04-07T16:16:16.492 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.492 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.492 INFO:tasks.workunit.client.0.smithi100.stdout: "compressed_original": { 2024-04-07T16:16:16.492 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.492 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.492 INFO:tasks.workunit.client.0.smithi100.stdout: "csum_lat": { 2024-04-07T16:16:16.492 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 3, 2024-04-07T16:16:16.492 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 6907 2024-04-07T16:16:16.492 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.492 INFO:tasks.workunit.client.0.smithi100.stdout: "decompress_lat": { 2024-04-07T16:16:16.492 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.492 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.492 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.492 INFO:tasks.workunit.client.0.smithi100.stdout: "fragmentation_micros": { 2024-04-07T16:16:16.493 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.493 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.493 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_commit_lat": { 2024-04-07T16:16:16.493 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 168, 2024-04-07T16:16:16.493 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 81377208 2024-04-07T16:16:16.493 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.493 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_final_lat": { 2024-04-07T16:16:16.493 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 160, 2024-04-07T16:16:16.493 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 11809538 2024-04-07T16:16:16.493 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.493 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_flush_lat": { 2024-04-07T16:16:16.493 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 168, 2024-04-07T16:16:16.493 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 269652 2024-04-07T16:16:16.493 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.493 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_sync_lat": { 2024-04-07T16:16:16.494 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 168, 2024-04-07T16:16:16.494 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 81646860 2024-04-07T16:16:16.494 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.494 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_get_keys_lat": { 2024-04-07T16:16:16.494 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.494 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.494 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.494 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_get_values_lat": { 2024-04-07T16:16:16.494 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 3, 2024-04-07T16:16:16.494 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 59409 2024-04-07T16:16:16.494 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.494 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_lower_bound_lat": { 2024-04-07T16:16:16.494 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.494 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.495 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.495 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_next_lat": { 2024-04-07T16:16:16.495 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.495 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.495 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.495 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_seek_to_first_lat": { 2024-04-07T16:16:16.495 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.495 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.495 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.495 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_upper_bound_lat": { 2024-04-07T16:16:16.495 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 2, 2024-04-07T16:16:16.495 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5918 2024-04-07T16:16:16.495 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.495 INFO:tasks.workunit.client.0.smithi100.stdout: "onode_hits": { 2024-04-07T16:16:16.495 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 329 2024-04-07T16:16:16.495 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.496 INFO:tasks.workunit.client.0.smithi100.stdout: "onode_misses": { 2024-04-07T16:16:16.496 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 243 2024-04-07T16:16:16.496 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.496 INFO:tasks.workunit.client.0.smithi100.stdout: "read_lat": { 2024-04-07T16:16:16.496 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 91, 2024-04-07T16:16:16.496 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1181894 2024-04-07T16:16:16.496 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.496 INFO:tasks.workunit.client.0.smithi100.stdout: "read_onode_meta_lat": { 2024-04-07T16:16:16.496 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 145, 2024-04-07T16:16:16.496 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 469150 2024-04-07T16:16:16.496 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.496 INFO:tasks.workunit.client.0.smithi100.stdout: "read_wait_aio_lat": { 2024-04-07T16:16:16.496 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 54, 2024-04-07T16:16:16.496 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 430530 2024-04-07T16:16:16.496 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.497 INFO:tasks.workunit.client.0.smithi100.stdout: "reads_with_retries": { 2024-04-07T16:16:16.497 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.497 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.497 INFO:tasks.workunit.client.0.smithi100.stdout: "remove_lat": { 2024-04-07T16:16:16.497 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 3, 2024-04-07T16:16:16.497 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 191068 2024-04-07T16:16:16.497 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.497 INFO:tasks.workunit.client.0.smithi100.stdout: "state_aio_wait_lat": { 2024-04-07T16:16:16.497 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 269, 2024-04-07T16:16:16.497 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 8414866 2024-04-07T16:16:16.497 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.497 INFO:tasks.workunit.client.0.smithi100.stdout: "state_deferred_aio_wait_lat": { 2024-04-07T16:16:16.497 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 38, 2024-04-07T16:16:16.497 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3864381 2024-04-07T16:16:16.498 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.498 INFO:tasks.workunit.client.0.smithi100.stdout: "state_deferred_cleanup_lat": { 2024-04-07T16:16:16.498 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 35, 2024-04-07T16:16:16.498 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 411804893436 2024-04-07T16:16:16.498 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.498 INFO:tasks.workunit.client.0.smithi100.stdout: "state_deferred_queued_lat": { 2024-04-07T16:16:16.498 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 38, 2024-04-07T16:16:16.498 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 62590955187 2024-04-07T16:16:16.498 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.498 INFO:tasks.workunit.client.0.smithi100.stdout: "state_done_lat": { 2024-04-07T16:16:16.498 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 266, 2024-04-07T16:16:16.498 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 793682 2024-04-07T16:16:16.498 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.498 INFO:tasks.workunit.client.0.smithi100.stdout: "state_finishing_lat": { 2024-04-07T16:16:16.498 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 266, 2024-04-07T16:16:16.499 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 73704 2024-04-07T16:16:16.499 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.499 INFO:tasks.workunit.client.0.smithi100.stdout: "state_io_done_lat": { 2024-04-07T16:16:16.499 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 269, 2024-04-07T16:16:16.499 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 376944 2024-04-07T16:16:16.499 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.499 INFO:tasks.workunit.client.0.smithi100.stdout: "state_kv_commiting_lat": { 2024-04-07T16:16:16.499 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 269, 2024-04-07T16:16:16.499 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 169873885 2024-04-07T16:16:16.499 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.499 INFO:tasks.workunit.client.0.smithi100.stdout: "state_kv_done_lat": { 2024-04-07T16:16:16.499 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 269, 2024-04-07T16:16:16.499 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 119204 2024-04-07T16:16:16.499 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.500 INFO:tasks.workunit.client.0.smithi100.stdout: "state_kv_queued_lat": { 2024-04-07T16:16:16.500 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 269, 2024-04-07T16:16:16.500 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 89218554 2024-04-07T16:16:16.500 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.500 INFO:tasks.workunit.client.0.smithi100.stdout: "state_prepare_lat": { 2024-04-07T16:16:16.500 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 269, 2024-04-07T16:16:16.500 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 19868622 2024-04-07T16:16:16.500 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.500 INFO:tasks.workunit.client.0.smithi100.stdout: "stored": { 2024-04-07T16:16:16.500 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 311623 2024-04-07T16:16:16.500 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.500 INFO:tasks.workunit.client.0.smithi100.stdout: "truncate_lat": { 2024-04-07T16:16:16.500 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 31, 2024-04-07T16:16:16.500 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 218105 2024-04-07T16:16:16.500 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.501 INFO:tasks.workunit.client.0.smithi100.stdout: "txc_commit_lat": { 2024-04-07T16:16:16.501 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 269, 2024-04-07T16:16:16.501 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 287805482 2024-04-07T16:16:16.501 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.501 INFO:tasks.workunit.client.0.smithi100.stdout: "txc_submit_lat": { 2024-04-07T16:16:16.501 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 269, 2024-04-07T16:16:16.501 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 23169788 2024-04-07T16:16:16.501 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.501 INFO:tasks.workunit.client.0.smithi100.stdout: "txc_throttle_lat": { 2024-04-07T16:16:16.501 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 269, 2024-04-07T16:16:16.501 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3200638 2024-04-07T16:16:16.501 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:16.501 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.501 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache": { 2024-04-07T16:16:16.501 INFO:tasks.workunit.client.0.smithi100.stdout: "cache_bytes": { 2024-04-07T16:16:16.501 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1900990924 2024-04-07T16:16:16.502 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.502 INFO:tasks.workunit.client.0.smithi100.stdout: "heap_bytes": { 2024-04-07T16:16:16.502 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 43433984 2024-04-07T16:16:16.502 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.502 INFO:tasks.workunit.client.0.smithi100.stdout: "mapped_bytes": { 2024-04-07T16:16:16.502 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 42541056 2024-04-07T16:16:16.502 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.502 INFO:tasks.workunit.client.0.smithi100.stdout: "target_bytes": { 2024-04-07T16:16:16.502 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3183879168 2024-04-07T16:16:16.502 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.502 INFO:tasks.workunit.client.0.smithi100.stdout: "unmapped_bytes": { 2024-04-07T16:16:16.502 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 892928 2024-04-07T16:16:16.502 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:16.502 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.502 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:data": { 2024-04-07T16:16:16.503 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T16:16:16.503 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 159383552 2024-04-07T16:16:16.503 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.503 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T16:16:16.503 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.503 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.503 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T16:16:16.503 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.503 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.503 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T16:16:16.503 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 87871109 2024-04-07T16:16:16.503 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.503 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T16:16:16.503 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.503 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.503 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T16:16:16.503 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.504 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.504 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T16:16:16.504 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.504 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.504 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T16:16:16.504 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.504 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.504 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T16:16:16.504 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.504 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.504 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T16:16:16.504 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.504 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.504 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T16:16:16.504 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.504 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.505 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T16:16:16.505 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.505 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.505 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T16:16:16.505 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.505 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.505 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T16:16:16.505 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 71512443 2024-04-07T16:16:16.505 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:16.505 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.505 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:kv": { 2024-04-07T16:16:16.505 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T16:16:16.505 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 796917760 2024-04-07T16:16:16.505 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.505 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T16:16:16.506 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 67108864 2024-04-07T16:16:16.506 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.506 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T16:16:16.506 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.506 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.506 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T16:16:16.506 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 659033321 2024-04-07T16:16:16.506 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.506 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T16:16:16.506 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.506 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.506 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T16:16:16.506 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.506 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.506 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T16:16:16.507 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.507 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.507 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T16:16:16.507 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.507 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.507 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T16:16:16.507 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.507 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.507 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T16:16:16.507 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2128 2024-04-07T16:16:16.507 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.507 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T16:16:16.507 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.507 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.508 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T16:16:16.508 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.508 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.508 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T16:16:16.508 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.508 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.508 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T16:16:16.508 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 70773447 2024-04-07T16:16:16.508 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:16.508 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.508 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:kv_onode": { 2024-04-07T16:16:16.508 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T16:16:16.508 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 192937984 2024-04-07T16:16:16.508 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.508 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T16:16:16.509 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 67108864 2024-04-07T16:16:16.509 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.509 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T16:16:16.509 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 464 2024-04-07T16:16:16.509 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.509 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T16:16:16.509 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 58580739 2024-04-07T16:16:16.509 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.509 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T16:16:16.509 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.509 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.509 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T16:16:16.509 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.509 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.509 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T16:16:16.509 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.510 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.510 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T16:16:16.510 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.510 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.510 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T16:16:16.510 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.510 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.510 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T16:16:16.510 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.510 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.510 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T16:16:16.510 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.510 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.510 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T16:16:16.510 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.511 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.511 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T16:16:16.511 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.511 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.511 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T16:16:16.511 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 67247917 2024-04-07T16:16:16.511 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:16.511 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.511 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:meta": { 2024-04-07T16:16:16.511 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T16:16:16.511 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 729808896 2024-04-07T16:16:16.511 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.511 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T16:16:16.511 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.511 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.511 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T16:16:16.512 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.512 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.512 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T16:16:16.512 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 659033321 2024-04-07T16:16:16.512 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.512 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T16:16:16.512 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.512 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.512 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T16:16:16.512 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.512 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.512 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T16:16:16.512 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.512 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.512 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T16:16:16.512 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.513 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.513 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T16:16:16.513 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.513 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.513 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T16:16:16.513 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 224594 2024-04-07T16:16:16.513 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.513 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T16:16:16.513 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.513 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.513 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T16:16:16.513 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.513 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.513 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T16:16:16.513 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.514 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.514 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T16:16:16.514 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 70550981 2024-04-07T16:16:16.514 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:16.514 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.514 INFO:tasks.workunit.client.0.smithi100.stdout: "objecter": { 2024-04-07T16:16:16.514 INFO:tasks.workunit.client.0.smithi100.stdout: "op_active": { 2024-04-07T16:16:16.514 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.514 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.514 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r": { 2024-04-07T16:16:16.514 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.514 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.514 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rmw": { 2024-04-07T16:16:16.514 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.515 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.515 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w": { 2024-04-07T16:16:16.515 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.515 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:16.515 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.515 INFO:tasks.workunit.client.0.smithi100.stdout: "osd": { 2024-04-07T16:16:16.515 INFO:tasks.workunit.client.0.smithi100.stdout: "numpg": { 2024-04-07T16:16:16.515 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 42 2024-04-07T16:16:16.515 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.515 INFO:tasks.workunit.client.0.smithi100.stdout: "numpg_removing": { 2024-04-07T16:16:16.515 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.515 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.515 INFO:tasks.workunit.client.0.smithi100.stdout: "op": { 2024-04-07T16:16:16.515 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.515 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.516 INFO:tasks.workunit.client.0.smithi100.stdout: "op_delayed_degraded": { 2024-04-07T16:16:16.516 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.516 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.516 INFO:tasks.workunit.client.0.smithi100.stdout: "op_delayed_unreadable": { 2024-04-07T16:16:16.516 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.516 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.516 INFO:tasks.workunit.client.0.smithi100.stdout: "op_in_bytes": { 2024-04-07T16:16:16.516 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.516 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.516 INFO:tasks.workunit.client.0.smithi100.stdout: "op_latency": { 2024-04-07T16:16:16.516 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.516 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.516 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.516 INFO:tasks.workunit.client.0.smithi100.stdout: "op_out_bytes": { 2024-04-07T16:16:16.516 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.516 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.517 INFO:tasks.workunit.client.0.smithi100.stdout: "op_prepare_latency": { 2024-04-07T16:16:16.517 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.517 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.517 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.517 INFO:tasks.workunit.client.0.smithi100.stdout: "op_process_latency": { 2024-04-07T16:16:16.517 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.517 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.517 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.517 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r": { 2024-04-07T16:16:16.517 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.517 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.517 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_latency": { 2024-04-07T16:16:16.517 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.517 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.518 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.518 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_latency_out_bytes_histogram": { 2024-04-07T16:16:16.518 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.518 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.518 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_out_bytes": { 2024-04-07T16:16:16.518 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.518 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.518 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_prepare_latency": { 2024-04-07T16:16:16.518 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.518 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.518 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.518 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_process_latency": { 2024-04-07T16:16:16.518 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.518 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.518 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.519 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw": { 2024-04-07T16:16:16.519 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.519 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.519 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_in_bytes": { 2024-04-07T16:16:16.519 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.519 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.519 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_latency": { 2024-04-07T16:16:16.519 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.519 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.519 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.519 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_latency_in_bytes_histogram": { 2024-04-07T16:16:16.519 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.519 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.519 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_latency_out_bytes_histogram": { 2024-04-07T16:16:16.519 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.519 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.520 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_out_bytes": { 2024-04-07T16:16:16.520 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.520 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.520 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_prepare_latency": { 2024-04-07T16:16:16.520 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.520 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.520 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.520 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_process_latency": { 2024-04-07T16:16:16.520 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.520 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.520 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.520 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w": { 2024-04-07T16:16:16.520 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.520 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.520 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_in_bytes": { 2024-04-07T16:16:16.520 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.521 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.521 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_latency": { 2024-04-07T16:16:16.521 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.521 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.521 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.521 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_latency_in_bytes_histogram": { 2024-04-07T16:16:16.521 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.521 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.521 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_prepare_latency": { 2024-04-07T16:16:16.521 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.521 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.521 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.521 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_process_latency": { 2024-04-07T16:16:16.521 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.521 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.521 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.522 INFO:tasks.workunit.client.0.smithi100.stdout: "op_wip": { 2024-04-07T16:16:16.522 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.522 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.522 INFO:tasks.workunit.client.0.smithi100.stdout: "recovery_bytes": { 2024-04-07T16:16:16.522 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.522 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.522 INFO:tasks.workunit.client.0.smithi100.stdout: "recovery_ops": { 2024-04-07T16:16:16.522 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.522 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.522 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_bytes": { 2024-04-07T16:16:16.522 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 95995035648 2024-04-07T16:16:16.522 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.522 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_bytes_used": { 2024-04-07T16:16:16.523 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 28385280 2024-04-07T16:16:16.523 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:16.523 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:16.523 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.523 INFO:tasks.workunit.client.0.smithi100.stdout: "osd.6": { 2024-04-07T16:16:16.523 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs": { 2024-04-07T16:16:16.523 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_slow_fallback": { 2024-04-07T16:16:16.523 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.523 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.523 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_slow_size_fallback": { 2024-04-07T16:16:16.523 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.523 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.523 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit_db": { 2024-04-07T16:16:16.523 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 65536 2024-04-07T16:16:16.523 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.523 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit_main": { 2024-04-07T16:16:16.524 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.524 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.524 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit_wal": { 2024-04-07T16:16:16.524 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.524 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.524 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_written_slow": { 2024-04-07T16:16:16.524 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.524 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.524 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_written_sst": { 2024-04-07T16:16:16.524 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 12288 2024-04-07T16:16:16.524 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.524 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_written_wal": { 2024-04-07T16:16:16.524 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1372160 2024-04-07T16:16:16.525 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.525 INFO:tasks.workunit.client.0.smithi100.stdout: "compact_lat": { 2024-04-07T16:16:16.525 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.525 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.525 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.525 INFO:tasks.workunit.client.0.smithi100.stdout: "compact_lock_lat": { 2024-04-07T16:16:16.525 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.525 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.525 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.525 INFO:tasks.workunit.client.0.smithi100.stdout: "db_total_bytes": { 2024-04-07T16:16:16.525 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 95995035648 2024-04-07T16:16:16.525 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.525 INFO:tasks.workunit.client.0.smithi100.stdout: "db_used_bytes": { 2024-04-07T16:16:16.525 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 27656192 2024-04-07T16:16:16.526 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.526 INFO:tasks.workunit.client.0.smithi100.stdout: "log_bytes": { 2024-04-07T16:16:16.526 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1036288 2024-04-07T16:16:16.526 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.526 INFO:tasks.workunit.client.0.smithi100.stdout: "logged_bytes": { 2024-04-07T16:16:16.526 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 897024 2024-04-07T16:16:16.526 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.526 INFO:tasks.workunit.client.0.smithi100.stdout: "max_bytes_db": { 2024-04-07T16:16:16.526 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 31850496 2024-04-07T16:16:16.526 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.526 INFO:tasks.workunit.client.0.smithi100.stdout: "max_bytes_slow": { 2024-04-07T16:16:16.526 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.526 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.526 INFO:tasks.workunit.client.0.smithi100.stdout: "max_bytes_wal": { 2024-04-07T16:16:16.526 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.527 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.527 INFO:tasks.workunit.client.0.smithi100.stdout: "num_files": { 2024-04-07T16:16:16.527 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 14 2024-04-07T16:16:16.527 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.527 INFO:tasks.workunit.client.0.smithi100.stdout: "read_bytes": { 2024-04-07T16:16:16.527 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 206272 2024-04-07T16:16:16.527 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.527 INFO:tasks.workunit.client.0.smithi100.stdout: "read_count": { 2024-04-07T16:16:16.527 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 23 2024-04-07T16:16:16.527 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.527 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes": { 2024-04-07T16:16:16.527 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1282048 2024-04-07T16:16:16.527 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.527 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes_db": { 2024-04-07T16:16:16.527 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1286144 2024-04-07T16:16:16.528 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.528 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes_slow": { 2024-04-07T16:16:16.528 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.528 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.528 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes_wal": { 2024-04-07T16:16:16.528 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.528 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.528 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_count": { 2024-04-07T16:16:16.528 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 15 2024-04-07T16:16:16.528 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.528 INFO:tasks.workunit.client.0.smithi100.stdout: "read_prefetch_bytes": { 2024-04-07T16:16:16.528 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4913 2024-04-07T16:16:16.528 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.529 INFO:tasks.workunit.client.0.smithi100.stdout: "read_prefetch_count": { 2024-04-07T16:16:16.529 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4 2024-04-07T16:16:16.529 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.529 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_buffer_bytes": { 2024-04-07T16:16:16.529 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5504 2024-04-07T16:16:16.529 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.529 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_buffer_count": { 2024-04-07T16:16:16.529 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 24 2024-04-07T16:16:16.529 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.529 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_bytes": { 2024-04-07T16:16:16.529 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5504 2024-04-07T16:16:16.529 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.529 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_count": { 2024-04-07T16:16:16.529 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 24 2024-04-07T16:16:16.529 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.529 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes": { 2024-04-07T16:16:16.530 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.530 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.530 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes_db": { 2024-04-07T16:16:16.530 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.530 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.530 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes_slow": { 2024-04-07T16:16:16.530 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.530 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.530 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes_wal": { 2024-04-07T16:16:16.530 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.530 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.530 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_count": { 2024-04-07T16:16:16.530 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.530 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.531 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_total_bytes": { 2024-04-07T16:16:16.531 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.531 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.531 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_used_bytes": { 2024-04-07T16:16:16.531 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.531 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.531 INFO:tasks.workunit.client.0.smithi100.stdout: "wal_total_bytes": { 2024-04-07T16:16:16.531 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.531 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.531 INFO:tasks.workunit.client.0.smithi100.stdout: "wal_used_bytes": { 2024-04-07T16:16:16.531 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.531 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.531 INFO:tasks.workunit.client.0.smithi100.stdout: "write_bytes": { 2024-04-07T16:16:16.531 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2347008 2024-04-07T16:16:16.531 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:16.531 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.532 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore": { 2024-04-07T16:16:16.532 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit": { 2024-04-07T16:16:16.532 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4096 2024-04-07T16:16:16.532 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.532 INFO:tasks.workunit.client.0.smithi100.stdout: "allocated": { 2024-04-07T16:16:16.532 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1314816 2024-04-07T16:16:16.532 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.532 INFO:tasks.workunit.client.0.smithi100.stdout: "clist_lat": { 2024-04-07T16:16:16.532 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.532 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.532 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.532 INFO:tasks.workunit.client.0.smithi100.stdout: "compress_lat": { 2024-04-07T16:16:16.532 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.532 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.532 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.532 INFO:tasks.workunit.client.0.smithi100.stdout: "compressed": { 2024-04-07T16:16:16.533 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.533 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.533 INFO:tasks.workunit.client.0.smithi100.stdout: "compressed_allocated": { 2024-04-07T16:16:16.533 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.533 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.533 INFO:tasks.workunit.client.0.smithi100.stdout: "compressed_original": { 2024-04-07T16:16:16.533 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.533 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.533 INFO:tasks.workunit.client.0.smithi100.stdout: "csum_lat": { 2024-04-07T16:16:16.533 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 3, 2024-04-07T16:16:16.533 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 7548 2024-04-07T16:16:16.533 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.533 INFO:tasks.workunit.client.0.smithi100.stdout: "decompress_lat": { 2024-04-07T16:16:16.533 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.533 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.533 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.534 INFO:tasks.workunit.client.0.smithi100.stdout: "fragmentation_micros": { 2024-04-07T16:16:16.534 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.534 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.534 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_commit_lat": { 2024-04-07T16:16:16.534 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 206, 2024-04-07T16:16:16.534 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 67027413 2024-04-07T16:16:16.534 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.534 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_final_lat": { 2024-04-07T16:16:16.534 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 187, 2024-04-07T16:16:16.534 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 9193262 2024-04-07T16:16:16.534 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.534 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_flush_lat": { 2024-04-07T16:16:16.534 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 206, 2024-04-07T16:16:16.534 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 244150 2024-04-07T16:16:16.534 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.535 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_sync_lat": { 2024-04-07T16:16:16.535 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 206, 2024-04-07T16:16:16.535 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 67271563 2024-04-07T16:16:16.535 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.535 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_get_keys_lat": { 2024-04-07T16:16:16.535 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.535 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.535 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.535 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_get_values_lat": { 2024-04-07T16:16:16.535 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1, 2024-04-07T16:16:16.535 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 54597 2024-04-07T16:16:16.535 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.535 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_lower_bound_lat": { 2024-04-07T16:16:16.535 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1, 2024-04-07T16:16:16.535 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2371 2024-04-07T16:16:16.536 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.536 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_next_lat": { 2024-04-07T16:16:16.536 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.536 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.536 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.536 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_seek_to_first_lat": { 2024-04-07T16:16:16.536 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.536 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.536 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.536 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_upper_bound_lat": { 2024-04-07T16:16:16.536 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 2, 2024-04-07T16:16:16.536 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 7421 2024-04-07T16:16:16.536 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.536 INFO:tasks.workunit.client.0.smithi100.stdout: "onode_hits": { 2024-04-07T16:16:16.536 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 375 2024-04-07T16:16:16.536 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.537 INFO:tasks.workunit.client.0.smithi100.stdout: "onode_misses": { 2024-04-07T16:16:16.537 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 248 2024-04-07T16:16:16.537 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.537 INFO:tasks.workunit.client.0.smithi100.stdout: "read_lat": { 2024-04-07T16:16:16.537 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 89, 2024-04-07T16:16:16.537 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 947386 2024-04-07T16:16:16.537 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.537 INFO:tasks.workunit.client.0.smithi100.stdout: "read_onode_meta_lat": { 2024-04-07T16:16:16.537 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 137, 2024-04-07T16:16:16.537 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 326760 2024-04-07T16:16:16.537 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.537 INFO:tasks.workunit.client.0.smithi100.stdout: "read_wait_aio_lat": { 2024-04-07T16:16:16.537 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 48, 2024-04-07T16:16:16.537 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 454246 2024-04-07T16:16:16.538 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.538 INFO:tasks.workunit.client.0.smithi100.stdout: "reads_with_retries": { 2024-04-07T16:16:16.538 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.538 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.538 INFO:tasks.workunit.client.0.smithi100.stdout: "remove_lat": { 2024-04-07T16:16:16.538 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.538 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.538 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.538 INFO:tasks.workunit.client.0.smithi100.stdout: "state_aio_wait_lat": { 2024-04-07T16:16:16.538 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 281, 2024-04-07T16:16:16.538 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5676100 2024-04-07T16:16:16.538 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.538 INFO:tasks.workunit.client.0.smithi100.stdout: "state_deferred_aio_wait_lat": { 2024-04-07T16:16:16.538 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 34, 2024-04-07T16:16:16.539 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3229934 2024-04-07T16:16:16.539 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.539 INFO:tasks.workunit.client.0.smithi100.stdout: "state_deferred_cleanup_lat": { 2024-04-07T16:16:16.539 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 34, 2024-04-07T16:16:16.539 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 793536067086 2024-04-07T16:16:16.539 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.539 INFO:tasks.workunit.client.0.smithi100.stdout: "state_deferred_queued_lat": { 2024-04-07T16:16:16.539 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 34, 2024-04-07T16:16:16.539 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 64017169880 2024-04-07T16:16:16.539 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.539 INFO:tasks.workunit.client.0.smithi100.stdout: "state_done_lat": { 2024-04-07T16:16:16.539 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 281, 2024-04-07T16:16:16.539 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 745434 2024-04-07T16:16:16.539 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.539 INFO:tasks.workunit.client.0.smithi100.stdout: "state_finishing_lat": { 2024-04-07T16:16:16.540 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 281, 2024-04-07T16:16:16.540 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 48923 2024-04-07T16:16:16.540 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.540 INFO:tasks.workunit.client.0.smithi100.stdout: "state_io_done_lat": { 2024-04-07T16:16:16.540 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 281, 2024-04-07T16:16:16.540 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 439729 2024-04-07T16:16:16.540 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.540 INFO:tasks.workunit.client.0.smithi100.stdout: "state_kv_commiting_lat": { 2024-04-07T16:16:16.540 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 281, 2024-04-07T16:16:16.540 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 143969182 2024-04-07T16:16:16.540 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.540 INFO:tasks.workunit.client.0.smithi100.stdout: "state_kv_done_lat": { 2024-04-07T16:16:16.540 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 281, 2024-04-07T16:16:16.540 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 163140 2024-04-07T16:16:16.540 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.541 INFO:tasks.workunit.client.0.smithi100.stdout: "state_kv_queued_lat": { 2024-04-07T16:16:16.541 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 281, 2024-04-07T16:16:16.541 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 98095617 2024-04-07T16:16:16.541 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.541 INFO:tasks.workunit.client.0.smithi100.stdout: "state_prepare_lat": { 2024-04-07T16:16:16.541 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 281, 2024-04-07T16:16:16.541 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 18797974 2024-04-07T16:16:16.541 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.541 INFO:tasks.workunit.client.0.smithi100.stdout: "stored": { 2024-04-07T16:16:16.541 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 902232 2024-04-07T16:16:16.541 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.541 INFO:tasks.workunit.client.0.smithi100.stdout: "truncate_lat": { 2024-04-07T16:16:16.541 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 27, 2024-04-07T16:16:16.541 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 191944 2024-04-07T16:16:16.541 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.542 INFO:tasks.workunit.client.0.smithi100.stdout: "txc_commit_lat": { 2024-04-07T16:16:16.542 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 281, 2024-04-07T16:16:16.542 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 267049517 2024-04-07T16:16:16.542 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.542 INFO:tasks.workunit.client.0.smithi100.stdout: "txc_submit_lat": { 2024-04-07T16:16:16.542 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 281, 2024-04-07T16:16:16.542 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 22200728 2024-04-07T16:16:16.542 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.542 INFO:tasks.workunit.client.0.smithi100.stdout: "txc_throttle_lat": { 2024-04-07T16:16:16.542 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 281, 2024-04-07T16:16:16.542 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2636656 2024-04-07T16:16:16.542 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:16.542 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.542 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache": { 2024-04-07T16:16:16.543 INFO:tasks.workunit.client.0.smithi100.stdout: "cache_bytes": { 2024-04-07T16:16:16.543 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1900990924 2024-04-07T16:16:16.543 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.543 INFO:tasks.workunit.client.0.smithi100.stdout: "heap_bytes": { 2024-04-07T16:16:16.543 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 43581440 2024-04-07T16:16:16.543 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.543 INFO:tasks.workunit.client.0.smithi100.stdout: "mapped_bytes": { 2024-04-07T16:16:16.543 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 42934272 2024-04-07T16:16:16.543 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.543 INFO:tasks.workunit.client.0.smithi100.stdout: "target_bytes": { 2024-04-07T16:16:16.543 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3183879168 2024-04-07T16:16:16.543 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.543 INFO:tasks.workunit.client.0.smithi100.stdout: "unmapped_bytes": { 2024-04-07T16:16:16.543 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 647168 2024-04-07T16:16:16.544 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:16.544 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.544 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:data": { 2024-04-07T16:16:16.544 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T16:16:16.544 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 159383552 2024-04-07T16:16:16.544 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.544 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T16:16:16.544 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.544 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.544 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T16:16:16.544 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.544 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.544 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T16:16:16.544 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 87870874 2024-04-07T16:16:16.544 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.545 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T16:16:16.545 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.545 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.545 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T16:16:16.545 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.545 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.545 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T16:16:16.545 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.545 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.545 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T16:16:16.545 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.545 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.545 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T16:16:16.545 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.545 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.545 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T16:16:16.546 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.546 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.546 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T16:16:16.546 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.546 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.546 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T16:16:16.546 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.546 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.546 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T16:16:16.546 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.546 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.546 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T16:16:16.546 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 71512678 2024-04-07T16:16:16.546 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:16.546 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.547 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:kv": { 2024-04-07T16:16:16.547 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T16:16:16.547 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 796917760 2024-04-07T16:16:16.547 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.547 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T16:16:16.547 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 67108864 2024-04-07T16:16:16.547 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.547 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T16:16:16.547 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.547 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.547 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T16:16:16.547 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 659031560 2024-04-07T16:16:16.547 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.547 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T16:16:16.547 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.548 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.548 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T16:16:16.548 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.548 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.548 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T16:16:16.548 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.548 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.548 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T16:16:16.548 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.548 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.548 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T16:16:16.548 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.548 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.548 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T16:16:16.548 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2128 2024-04-07T16:16:16.549 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.549 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T16:16:16.549 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.549 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.549 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T16:16:16.549 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.549 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.549 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T16:16:16.549 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.549 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.549 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T16:16:16.549 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 70775208 2024-04-07T16:16:16.549 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:16.549 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.549 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:kv_onode": { 2024-04-07T16:16:16.549 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T16:16:16.549 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 192937984 2024-04-07T16:16:16.550 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.550 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T16:16:16.550 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 67108864 2024-04-07T16:16:16.550 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.550 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T16:16:16.550 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 464 2024-04-07T16:16:16.550 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.550 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T16:16:16.550 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 58580583 2024-04-07T16:16:16.550 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.550 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T16:16:16.550 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.550 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.550 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T16:16:16.551 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.551 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.551 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T16:16:16.551 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.551 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.551 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T16:16:16.551 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.551 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.551 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T16:16:16.551 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.551 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.551 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T16:16:16.551 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.551 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.551 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T16:16:16.551 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.552 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.552 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T16:16:16.552 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.552 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.552 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T16:16:16.552 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.552 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.552 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T16:16:16.552 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 67248073 2024-04-07T16:16:16.552 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:16.552 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.552 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:meta": { 2024-04-07T16:16:16.552 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T16:16:16.552 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 729808896 2024-04-07T16:16:16.552 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.553 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T16:16:16.553 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.553 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.553 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T16:16:16.553 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.553 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.553 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T16:16:16.553 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 659031560 2024-04-07T16:16:16.553 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.553 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T16:16:16.553 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.553 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.553 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T16:16:16.553 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2397 2024-04-07T16:16:16.553 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.554 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T16:16:16.554 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3596 2024-04-07T16:16:16.554 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.554 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T16:16:16.554 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.554 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.554 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T16:16:16.554 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.554 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.554 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T16:16:16.554 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 221781 2024-04-07T16:16:16.554 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.554 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T16:16:16.554 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.555 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.555 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T16:16:16.555 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.555 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.555 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T16:16:16.555 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.555 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.555 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T16:16:16.555 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 70549562 2024-04-07T16:16:16.555 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:16.555 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.555 INFO:tasks.workunit.client.0.smithi100.stdout: "objecter": { 2024-04-07T16:16:16.555 INFO:tasks.workunit.client.0.smithi100.stdout: "op_active": { 2024-04-07T16:16:16.555 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.556 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.556 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r": { 2024-04-07T16:16:16.556 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.556 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.556 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rmw": { 2024-04-07T16:16:16.556 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.556 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.556 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w": { 2024-04-07T16:16:16.556 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.556 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:16.556 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.556 INFO:tasks.workunit.client.0.smithi100.stdout: "osd": { 2024-04-07T16:16:16.556 INFO:tasks.workunit.client.0.smithi100.stdout: "numpg": { 2024-04-07T16:16:16.556 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 39 2024-04-07T16:16:16.556 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.556 INFO:tasks.workunit.client.0.smithi100.stdout: "numpg_removing": { 2024-04-07T16:16:16.557 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.557 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.557 INFO:tasks.workunit.client.0.smithi100.stdout: "op": { 2024-04-07T16:16:16.557 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 13 2024-04-07T16:16:16.557 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.557 INFO:tasks.workunit.client.0.smithi100.stdout: "op_delayed_degraded": { 2024-04-07T16:16:16.557 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.557 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.557 INFO:tasks.workunit.client.0.smithi100.stdout: "op_delayed_unreadable": { 2024-04-07T16:16:16.557 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.557 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.557 INFO:tasks.workunit.client.0.smithi100.stdout: "op_in_bytes": { 2024-04-07T16:16:16.557 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2812 2024-04-07T16:16:16.557 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.557 INFO:tasks.workunit.client.0.smithi100.stdout: "op_latency": { 2024-04-07T16:16:16.557 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 13, 2024-04-07T16:16:16.558 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 180979041 2024-04-07T16:16:16.558 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.558 INFO:tasks.workunit.client.0.smithi100.stdout: "op_out_bytes": { 2024-04-07T16:16:16.558 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.558 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.558 INFO:tasks.workunit.client.0.smithi100.stdout: "op_prepare_latency": { 2024-04-07T16:16:16.558 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 15, 2024-04-07T16:16:16.558 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 10137974 2024-04-07T16:16:16.558 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.558 INFO:tasks.workunit.client.0.smithi100.stdout: "op_process_latency": { 2024-04-07T16:16:16.558 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 13, 2024-04-07T16:16:16.558 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 165152222 2024-04-07T16:16:16.558 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.558 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r": { 2024-04-07T16:16:16.558 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.558 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.559 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_latency": { 2024-04-07T16:16:16.559 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.559 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.559 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.559 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_latency_out_bytes_histogram": { 2024-04-07T16:16:16.559 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.559 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.559 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_out_bytes": { 2024-04-07T16:16:16.559 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.559 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.559 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_prepare_latency": { 2024-04-07T16:16:16.559 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.559 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.559 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.559 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_process_latency": { 2024-04-07T16:16:16.560 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.560 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.560 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.560 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw": { 2024-04-07T16:16:16.560 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.560 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.560 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_in_bytes": { 2024-04-07T16:16:16.560 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.560 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.560 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_latency": { 2024-04-07T16:16:16.560 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.560 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.560 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.560 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_latency_in_bytes_histogram": { 2024-04-07T16:16:16.560 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.561 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.561 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_latency_out_bytes_histogram": { 2024-04-07T16:16:16.561 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.561 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.561 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_out_bytes": { 2024-04-07T16:16:16.561 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.561 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.561 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_prepare_latency": { 2024-04-07T16:16:16.561 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.561 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.561 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.561 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_process_latency": { 2024-04-07T16:16:16.561 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.561 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.561 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.561 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w": { 2024-04-07T16:16:16.562 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 13 2024-04-07T16:16:16.562 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.562 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_in_bytes": { 2024-04-07T16:16:16.562 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2812 2024-04-07T16:16:16.562 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.562 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_latency": { 2024-04-07T16:16:16.562 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 13, 2024-04-07T16:16:16.562 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 180979041 2024-04-07T16:16:16.562 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.562 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_latency_in_bytes_histogram": { 2024-04-07T16:16:16.562 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.562 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.562 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_prepare_latency": { 2024-04-07T16:16:16.562 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 15, 2024-04-07T16:16:16.562 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 10137974 2024-04-07T16:16:16.563 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.563 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_process_latency": { 2024-04-07T16:16:16.563 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 13, 2024-04-07T16:16:16.563 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 165152222 2024-04-07T16:16:16.563 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.563 INFO:tasks.workunit.client.0.smithi100.stdout: "op_wip": { 2024-04-07T16:16:16.563 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.563 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.563 INFO:tasks.workunit.client.0.smithi100.stdout: "recovery_bytes": { 2024-04-07T16:16:16.563 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.563 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.563 INFO:tasks.workunit.client.0.smithi100.stdout: "recovery_ops": { 2024-04-07T16:16:16.563 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1 2024-04-07T16:16:16.563 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.564 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_bytes": { 2024-04-07T16:16:16.564 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 95995035648 2024-04-07T16:16:16.564 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.564 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_bytes_used": { 2024-04-07T16:16:16.564 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 28979200 2024-04-07T16:16:16.564 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:16.564 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:16.564 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.564 INFO:tasks.workunit.client.0.smithi100.stdout: "osd.7": { 2024-04-07T16:16:16.564 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs": { 2024-04-07T16:16:16.564 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_slow_fallback": { 2024-04-07T16:16:16.564 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.564 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.564 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_slow_size_fallback": { 2024-04-07T16:16:16.564 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.564 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.565 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit_db": { 2024-04-07T16:16:16.565 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 65536 2024-04-07T16:16:16.565 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.565 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit_main": { 2024-04-07T16:16:16.565 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.565 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.565 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit_wal": { 2024-04-07T16:16:16.565 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.565 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.565 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_written_slow": { 2024-04-07T16:16:16.565 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.565 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.565 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_written_sst": { 2024-04-07T16:16:16.565 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 12288 2024-04-07T16:16:16.566 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.566 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_written_wal": { 2024-04-07T16:16:16.566 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 6524928 2024-04-07T16:16:16.566 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.566 INFO:tasks.workunit.client.0.smithi100.stdout: "compact_lat": { 2024-04-07T16:16:16.566 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.566 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.566 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.566 INFO:tasks.workunit.client.0.smithi100.stdout: "compact_lock_lat": { 2024-04-07T16:16:16.566 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.566 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.566 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.566 INFO:tasks.workunit.client.0.smithi100.stdout: "db_total_bytes": { 2024-04-07T16:16:16.566 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 95995035648 2024-04-07T16:16:16.566 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.567 INFO:tasks.workunit.client.0.smithi100.stdout: "db_used_bytes": { 2024-04-07T16:16:16.567 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 31850496 2024-04-07T16:16:16.567 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.567 INFO:tasks.workunit.client.0.smithi100.stdout: "log_bytes": { 2024-04-07T16:16:16.567 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3526656 2024-04-07T16:16:16.567 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.567 INFO:tasks.workunit.client.0.smithi100.stdout: "logged_bytes": { 2024-04-07T16:16:16.567 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3387392 2024-04-07T16:16:16.567 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.567 INFO:tasks.workunit.client.0.smithi100.stdout: "max_bytes_db": { 2024-04-07T16:16:16.567 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 31850496 2024-04-07T16:16:16.567 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.567 INFO:tasks.workunit.client.0.smithi100.stdout: "max_bytes_slow": { 2024-04-07T16:16:16.567 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.567 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.568 INFO:tasks.workunit.client.0.smithi100.stdout: "max_bytes_wal": { 2024-04-07T16:16:16.568 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.568 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.568 INFO:tasks.workunit.client.0.smithi100.stdout: "num_files": { 2024-04-07T16:16:16.568 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 14 2024-04-07T16:16:16.568 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.568 INFO:tasks.workunit.client.0.smithi100.stdout: "read_bytes": { 2024-04-07T16:16:16.568 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 206272 2024-04-07T16:16:16.568 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.568 INFO:tasks.workunit.client.0.smithi100.stdout: "read_count": { 2024-04-07T16:16:16.568 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 23 2024-04-07T16:16:16.568 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.568 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes": { 2024-04-07T16:16:16.568 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1282048 2024-04-07T16:16:16.568 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.569 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes_db": { 2024-04-07T16:16:16.569 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1286144 2024-04-07T16:16:16.569 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.569 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes_slow": { 2024-04-07T16:16:16.569 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.569 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.569 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes_wal": { 2024-04-07T16:16:16.569 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.569 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.569 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_count": { 2024-04-07T16:16:16.569 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 15 2024-04-07T16:16:16.569 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.569 INFO:tasks.workunit.client.0.smithi100.stdout: "read_prefetch_bytes": { 2024-04-07T16:16:16.569 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4913 2024-04-07T16:16:16.569 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.570 INFO:tasks.workunit.client.0.smithi100.stdout: "read_prefetch_count": { 2024-04-07T16:16:16.570 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4 2024-04-07T16:16:16.570 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.570 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_buffer_bytes": { 2024-04-07T16:16:16.570 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5500 2024-04-07T16:16:16.570 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.570 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_buffer_count": { 2024-04-07T16:16:16.570 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 24 2024-04-07T16:16:16.570 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.570 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_bytes": { 2024-04-07T16:16:16.570 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5500 2024-04-07T16:16:16.570 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.570 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_count": { 2024-04-07T16:16:16.570 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 24 2024-04-07T16:16:16.571 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.571 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes": { 2024-04-07T16:16:16.571 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.571 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.571 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes_db": { 2024-04-07T16:16:16.571 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.571 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.571 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes_slow": { 2024-04-07T16:16:16.571 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.571 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.571 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes_wal": { 2024-04-07T16:16:16.571 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.571 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.571 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_count": { 2024-04-07T16:16:16.571 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.572 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.572 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_total_bytes": { 2024-04-07T16:16:16.572 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.572 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.572 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_used_bytes": { 2024-04-07T16:16:16.572 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.572 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.572 INFO:tasks.workunit.client.0.smithi100.stdout: "wal_total_bytes": { 2024-04-07T16:16:16.572 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.572 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.572 INFO:tasks.workunit.client.0.smithi100.stdout: "wal_used_bytes": { 2024-04-07T16:16:16.572 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.572 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.572 INFO:tasks.workunit.client.0.smithi100.stdout: "write_bytes": { 2024-04-07T16:16:16.572 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 9990144 2024-04-07T16:16:16.573 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:16.573 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.573 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore": { 2024-04-07T16:16:16.573 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit": { 2024-04-07T16:16:16.573 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4096 2024-04-07T16:16:16.573 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.573 INFO:tasks.workunit.client.0.smithi100.stdout: "allocated": { 2024-04-07T16:16:16.573 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1310720 2024-04-07T16:16:16.573 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.573 INFO:tasks.workunit.client.0.smithi100.stdout: "clist_lat": { 2024-04-07T16:16:16.573 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 5, 2024-04-07T16:16:16.573 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 540485 2024-04-07T16:16:16.573 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.573 INFO:tasks.workunit.client.0.smithi100.stdout: "compress_lat": { 2024-04-07T16:16:16.574 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.574 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.574 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.574 INFO:tasks.workunit.client.0.smithi100.stdout: "compressed": { 2024-04-07T16:16:16.574 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.574 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.574 INFO:tasks.workunit.client.0.smithi100.stdout: "compressed_allocated": { 2024-04-07T16:16:16.574 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.574 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.574 INFO:tasks.workunit.client.0.smithi100.stdout: "compressed_original": { 2024-04-07T16:16:16.577 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.578 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.578 INFO:tasks.workunit.client.0.smithi100.stdout: "csum_lat": { 2024-04-07T16:16:16.578 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 10, 2024-04-07T16:16:16.578 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 65865 2024-04-07T16:16:16.578 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.578 INFO:tasks.workunit.client.0.smithi100.stdout: "decompress_lat": { 2024-04-07T16:16:16.578 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.578 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.578 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.578 INFO:tasks.workunit.client.0.smithi100.stdout: "fragmentation_micros": { 2024-04-07T16:16:16.578 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.578 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.578 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_commit_lat": { 2024-04-07T16:16:16.578 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 814, 2024-04-07T16:16:16.579 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 513180052 2024-04-07T16:16:16.579 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.579 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_final_lat": { 2024-04-07T16:16:16.579 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 800, 2024-04-07T16:16:16.579 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 33570130 2024-04-07T16:16:16.579 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.579 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_flush_lat": { 2024-04-07T16:16:16.579 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 814, 2024-04-07T16:16:16.579 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 944498 2024-04-07T16:16:16.579 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.579 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_sync_lat": { 2024-04-07T16:16:16.579 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 814, 2024-04-07T16:16:16.579 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 514124550 2024-04-07T16:16:16.579 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.580 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_get_keys_lat": { 2024-04-07T16:16:16.580 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.580 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.580 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.580 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_get_values_lat": { 2024-04-07T16:16:16.580 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 2, 2024-04-07T16:16:16.580 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 125983 2024-04-07T16:16:16.580 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.580 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_lower_bound_lat": { 2024-04-07T16:16:16.580 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.580 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.580 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.580 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_next_lat": { 2024-04-07T16:16:16.580 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.580 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.581 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.581 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_seek_to_first_lat": { 2024-04-07T16:16:16.581 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T16:16:16.581 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.581 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.581 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_upper_bound_lat": { 2024-04-07T16:16:16.581 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 2, 2024-04-07T16:16:16.581 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 6034 2024-04-07T16:16:16.581 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.581 INFO:tasks.workunit.client.0.smithi100.stdout: "onode_hits": { 2024-04-07T16:16:16.581 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3805 2024-04-07T16:16:16.581 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.581 INFO:tasks.workunit.client.0.smithi100.stdout: "onode_misses": { 2024-04-07T16:16:16.581 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 382 2024-04-07T16:16:16.581 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.582 INFO:tasks.workunit.client.0.smithi100.stdout: "read_lat": { 2024-04-07T16:16:16.582 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 126, 2024-04-07T16:16:16.582 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3952043 2024-04-07T16:16:16.582 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.582 INFO:tasks.workunit.client.0.smithi100.stdout: "read_onode_meta_lat": { 2024-04-07T16:16:16.582 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 200, 2024-04-07T16:16:16.582 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 390268 2024-04-07T16:16:16.582 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.582 INFO:tasks.workunit.client.0.smithi100.stdout: "read_wait_aio_lat": { 2024-04-07T16:16:16.582 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 74, 2024-04-07T16:16:16.582 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2702716 2024-04-07T16:16:16.582 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.582 INFO:tasks.workunit.client.0.smithi100.stdout: "reads_with_retries": { 2024-04-07T16:16:16.582 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.583 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.583 INFO:tasks.workunit.client.0.smithi100.stdout: "remove_lat": { 2024-04-07T16:16:16.583 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 102, 2024-04-07T16:16:16.583 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5813187 2024-04-07T16:16:16.583 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.583 INFO:tasks.workunit.client.0.smithi100.stdout: "state_aio_wait_lat": { 2024-04-07T16:16:16.583 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 3406, 2024-04-07T16:16:16.583 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3775563427 2024-04-07T16:16:16.583 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.583 INFO:tasks.workunit.client.0.smithi100.stdout: "state_deferred_aio_wait_lat": { 2024-04-07T16:16:16.583 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 20, 2024-04-07T16:16:16.583 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 22546871 2024-04-07T16:16:16.583 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.583 INFO:tasks.workunit.client.0.smithi100.stdout: "state_deferred_cleanup_lat": { 2024-04-07T16:16:16.584 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 20, 2024-04-07T16:16:16.584 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 275039457935 2024-04-07T16:16:16.584 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.584 INFO:tasks.workunit.client.0.smithi100.stdout: "state_deferred_queued_lat": { 2024-04-07T16:16:16.584 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 20, 2024-04-07T16:16:16.584 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 36047466444 2024-04-07T16:16:16.584 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.584 INFO:tasks.workunit.client.0.smithi100.stdout: "state_done_lat": { 2024-04-07T16:16:16.584 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 3406, 2024-04-07T16:16:16.584 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1052965722 2024-04-07T16:16:16.584 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.584 INFO:tasks.workunit.client.0.smithi100.stdout: "state_finishing_lat": { 2024-04-07T16:16:16.584 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 3406, 2024-04-07T16:16:16.584 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 288488 2024-04-07T16:16:16.584 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.585 INFO:tasks.workunit.client.0.smithi100.stdout: "state_io_done_lat": { 2024-04-07T16:16:16.585 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 3406, 2024-04-07T16:16:16.585 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 124913645 2024-04-07T16:16:16.585 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.585 INFO:tasks.workunit.client.0.smithi100.stdout: "state_kv_commiting_lat": { 2024-04-07T16:16:16.585 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 3406, 2024-04-07T16:16:16.585 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2921805483 2024-04-07T16:16:16.585 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.585 INFO:tasks.workunit.client.0.smithi100.stdout: "state_kv_done_lat": { 2024-04-07T16:16:16.585 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 3406, 2024-04-07T16:16:16.585 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 701924 2024-04-07T16:16:16.585 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.585 INFO:tasks.workunit.client.0.smithi100.stdout: "state_kv_queued_lat": { 2024-04-07T16:16:16.585 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 3406, 2024-04-07T16:16:16.586 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1743273173 2024-04-07T16:16:16.586 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.586 INFO:tasks.workunit.client.0.smithi100.stdout: "state_prepare_lat": { 2024-04-07T16:16:16.586 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 3406, 2024-04-07T16:16:16.586 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 364232665 2024-04-07T16:16:16.586 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.586 INFO:tasks.workunit.client.0.smithi100.stdout: "stored": { 2024-04-07T16:16:16.586 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 900204 2024-04-07T16:16:16.586 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.586 INFO:tasks.workunit.client.0.smithi100.stdout: "truncate_lat": { 2024-04-07T16:16:16.586 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 19, 2024-04-07T16:16:16.586 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 125892 2024-04-07T16:16:16.586 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.586 INFO:tasks.workunit.client.0.smithi100.stdout: "txc_commit_lat": { 2024-04-07T16:16:16.587 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 3406, 2024-04-07T16:16:16.587 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 8930132063 2024-04-07T16:16:16.587 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.587 INFO:tasks.workunit.client.0.smithi100.stdout: "txc_submit_lat": { 2024-04-07T16:16:16.587 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 3406, 2024-04-07T16:16:16.587 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 422734310 2024-04-07T16:16:16.587 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.587 INFO:tasks.workunit.client.0.smithi100.stdout: "txc_throttle_lat": { 2024-04-07T16:16:16.587 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 3406, 2024-04-07T16:16:16.587 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 232969191 2024-04-07T16:16:16.587 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:16.587 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.587 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache": { 2024-04-07T16:16:16.587 INFO:tasks.workunit.client.0.smithi100.stdout: "cache_bytes": { 2024-04-07T16:16:16.587 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1900990924 2024-04-07T16:16:16.588 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.588 INFO:tasks.workunit.client.0.smithi100.stdout: "heap_bytes": { 2024-04-07T16:16:16.588 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 46792704 2024-04-07T16:16:16.588 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.588 INFO:tasks.workunit.client.0.smithi100.stdout: "mapped_bytes": { 2024-04-07T16:16:16.588 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 46063616 2024-04-07T16:16:16.588 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.588 INFO:tasks.workunit.client.0.smithi100.stdout: "target_bytes": { 2024-04-07T16:16:16.588 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3183879168 2024-04-07T16:16:16.588 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.588 INFO:tasks.workunit.client.0.smithi100.stdout: "unmapped_bytes": { 2024-04-07T16:16:16.588 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 729088 2024-04-07T16:16:16.588 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:16.588 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.588 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:data": { 2024-04-07T16:16:16.589 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T16:16:16.589 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 159383552 2024-04-07T16:16:16.589 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.589 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T16:16:16.589 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.589 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.589 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T16:16:16.589 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.589 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.589 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T16:16:16.589 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 87847026 2024-04-07T16:16:16.589 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.589 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T16:16:16.589 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.589 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.590 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T16:16:16.590 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 8192 2024-04-07T16:16:16.590 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.590 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T16:16:16.590 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 135168 2024-04-07T16:16:16.590 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.590 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T16:16:16.590 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.590 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.590 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T16:16:16.590 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.590 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.591 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T16:16:16.591 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 135168 2024-04-07T16:16:16.591 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.591 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T16:16:16.591 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.591 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.591 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T16:16:16.591 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.591 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.591 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T16:16:16.591 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.591 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.591 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T16:16:16.591 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 71257998 2024-04-07T16:16:16.591 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:16.592 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.592 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:kv": { 2024-04-07T16:16:16.592 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T16:16:16.592 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 796917760 2024-04-07T16:16:16.592 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.592 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T16:16:16.592 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 67108864 2024-04-07T16:16:16.592 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.592 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T16:16:16.592 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.592 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.592 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T16:16:16.592 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 658852699 2024-04-07T16:16:16.592 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.593 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T16:16:16.593 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.593 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.593 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T16:16:16.593 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.593 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.593 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T16:16:16.593 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.593 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.593 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T16:16:16.593 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.593 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.593 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T16:16:16.593 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.594 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.594 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T16:16:16.594 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2128 2024-04-07T16:16:16.594 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.594 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T16:16:16.594 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.594 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.594 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T16:16:16.594 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.594 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.594 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T16:16:16.594 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.594 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.594 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T16:16:16.594 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 70954069 2024-04-07T16:16:16.595 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:16.595 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.595 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:kv_onode": { 2024-04-07T16:16:16.595 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T16:16:16.595 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 192937984 2024-04-07T16:16:16.595 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.595 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T16:16:16.595 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 67108864 2024-04-07T16:16:16.595 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.595 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T16:16:16.595 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 464 2024-04-07T16:16:16.595 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.595 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T16:16:16.595 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 58564684 2024-04-07T16:16:16.596 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.596 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T16:16:16.596 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.596 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.596 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T16:16:16.596 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.596 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.596 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T16:16:16.596 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.596 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.596 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T16:16:16.596 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.596 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.596 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T16:16:16.597 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.597 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.597 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T16:16:16.597 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.597 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.597 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T16:16:16.597 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.597 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.597 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T16:16:16.597 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.597 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.597 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T16:16:16.597 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.597 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.597 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T16:16:16.598 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 67263972 2024-04-07T16:16:16.598 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:16.598 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.598 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:meta": { 2024-04-07T16:16:16.598 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T16:16:16.598 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 729808896 2024-04-07T16:16:16.598 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.598 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T16:16:16.598 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.598 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.598 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T16:16:16.598 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.598 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.598 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T16:16:16.599 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 658852699 2024-04-07T16:16:16.599 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.599 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T16:16:16.599 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.599 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.599 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T16:16:16.599 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3511 2024-04-07T16:16:16.599 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.599 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T16:16:16.599 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 7022 2024-04-07T16:16:16.599 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.599 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T16:16:16.599 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.599 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.599 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T16:16:16.600 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.600 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.600 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T16:16:16.600 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 337091 2024-04-07T16:16:16.600 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.600 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T16:16:16.600 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.600 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.600 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T16:16:16.600 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.600 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.600 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T16:16:16.600 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.600 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.600 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T16:16:16.601 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 70608573 2024-04-07T16:16:16.601 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:16.601 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.601 INFO:tasks.workunit.client.0.smithi100.stdout: "objecter": { 2024-04-07T16:16:16.601 INFO:tasks.workunit.client.0.smithi100.stdout: "op_active": { 2024-04-07T16:16:16.601 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.601 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.601 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r": { 2024-04-07T16:16:16.601 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.601 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.601 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rmw": { 2024-04-07T16:16:16.601 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.601 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.601 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w": { 2024-04-07T16:16:16.601 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.602 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:16.602 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.602 INFO:tasks.workunit.client.0.smithi100.stdout: "osd": { 2024-04-07T16:16:16.602 INFO:tasks.workunit.client.0.smithi100.stdout: "numpg": { 2024-04-07T16:16:16.602 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 46 2024-04-07T16:16:16.602 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.602 INFO:tasks.workunit.client.0.smithi100.stdout: "numpg_removing": { 2024-04-07T16:16:16.602 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.602 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.602 INFO:tasks.workunit.client.0.smithi100.stdout: "op": { 2024-04-07T16:16:16.602 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 143 2024-04-07T16:16:16.602 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.602 INFO:tasks.workunit.client.0.smithi100.stdout: "op_delayed_degraded": { 2024-04-07T16:16:16.602 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.602 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.603 INFO:tasks.workunit.client.0.smithi100.stdout: "op_delayed_unreadable": { 2024-04-07T16:16:16.603 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.603 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.603 INFO:tasks.workunit.client.0.smithi100.stdout: "op_in_bytes": { 2024-04-07T16:16:16.603 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.603 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.603 INFO:tasks.workunit.client.0.smithi100.stdout: "op_latency": { 2024-04-07T16:16:16.603 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 143, 2024-04-07T16:16:16.603 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 253621634 2024-04-07T16:16:16.603 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.603 INFO:tasks.workunit.client.0.smithi100.stdout: "op_out_bytes": { 2024-04-07T16:16:16.603 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 265829 2024-04-07T16:16:16.603 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.603 INFO:tasks.workunit.client.0.smithi100.stdout: "op_prepare_latency": { 2024-04-07T16:16:16.603 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 145, 2024-04-07T16:16:16.604 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 85324355 2024-04-07T16:16:16.604 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.604 INFO:tasks.workunit.client.0.smithi100.stdout: "op_process_latency": { 2024-04-07T16:16:16.604 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 143, 2024-04-07T16:16:16.604 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 231824488 2024-04-07T16:16:16.604 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.604 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r": { 2024-04-07T16:16:16.604 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 100 2024-04-07T16:16:16.604 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.604 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_latency": { 2024-04-07T16:16:16.604 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 100, 2024-04-07T16:16:16.604 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 44667141 2024-04-07T16:16:16.604 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.604 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_latency_out_bytes_histogram": { 2024-04-07T16:16:16.604 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.604 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.605 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_out_bytes": { 2024-04-07T16:16:16.605 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 265829 2024-04-07T16:16:16.605 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.605 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_prepare_latency": { 2024-04-07T16:16:16.605 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 100, 2024-04-07T16:16:16.605 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 39144457 2024-04-07T16:16:16.605 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.605 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_process_latency": { 2024-04-07T16:16:16.605 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 100, 2024-04-07T16:16:16.605 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 31896453 2024-04-07T16:16:16.605 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.605 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw": { 2024-04-07T16:16:16.605 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 38 2024-04-07T16:16:16.605 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.605 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_in_bytes": { 2024-04-07T16:16:16.605 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.606 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.606 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_latency": { 2024-04-07T16:16:16.606 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 38, 2024-04-07T16:16:16.606 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 87649240 2024-04-07T16:16:16.606 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.606 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_latency_in_bytes_histogram": { 2024-04-07T16:16:16.606 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.606 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.606 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_latency_out_bytes_histogram": { 2024-04-07T16:16:16.606 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.606 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.606 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_out_bytes": { 2024-04-07T16:16:16.606 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.606 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.606 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_prepare_latency": { 2024-04-07T16:16:16.607 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 38, 2024-04-07T16:16:16.607 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 42714605 2024-04-07T16:16:16.607 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.607 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_process_latency": { 2024-04-07T16:16:16.607 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 38, 2024-04-07T16:16:16.607 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 82158201 2024-04-07T16:16:16.607 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.607 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w": { 2024-04-07T16:16:16.607 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5 2024-04-07T16:16:16.607 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.607 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_in_bytes": { 2024-04-07T16:16:16.607 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.607 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.607 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_latency": { 2024-04-07T16:16:16.608 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 5, 2024-04-07T16:16:16.608 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 121305253 2024-04-07T16:16:16.608 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.608 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_latency_in_bytes_histogram": { 2024-04-07T16:16:16.608 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.608 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.608 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_prepare_latency": { 2024-04-07T16:16:16.608 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 7, 2024-04-07T16:16:16.608 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3465293 2024-04-07T16:16:16.608 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.608 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_process_latency": { 2024-04-07T16:16:16.608 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 5, 2024-04-07T16:16:16.608 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 117769834 2024-04-07T16:16:16.608 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.608 INFO:tasks.workunit.client.0.smithi100.stdout: "op_wip": { 2024-04-07T16:16:16.608 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.609 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.609 INFO:tasks.workunit.client.0.smithi100.stdout: "recovery_bytes": { 2024-04-07T16:16:16.609 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.609 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.609 INFO:tasks.workunit.client.0.smithi100.stdout: "recovery_ops": { 2024-04-07T16:16:16.609 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T16:16:16.609 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.609 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_bytes": { 2024-04-07T16:16:16.609 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 95995035648 2024-04-07T16:16:16.609 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.609 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_bytes_used": { 2024-04-07T16:16:16.609 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 33169408 2024-04-07T16:16:16.609 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:16.609 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:16.610 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:16.610 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.610 INFO:tasks.workunit.client.0.smithi100.stdout: "pools": [ 2024-04-07T16:16:16.610 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:16.610 INFO:tasks.workunit.client.0.smithi100.stdout: "application": [ 2024-04-07T16:16:16.610 INFO:tasks.workunit.client.0.smithi100.stdout: "mgr" 2024-04-07T16:16:16.610 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:16.610 INFO:tasks.workunit.client.0.smithi100.stdout: "cache_mode": "none", 2024-04-07T16:16:16.610 INFO:tasks.workunit.client.0.smithi100.stdout: "erasure_code_profile": {}, 2024-04-07T16:16:16.610 INFO:tasks.workunit.client.0.smithi100.stdout: "min_size": 2, 2024-04-07T16:16:16.610 INFO:tasks.workunit.client.0.smithi100.stdout: "options": {}, 2024-04-07T16:16:16.610 INFO:tasks.workunit.client.0.smithi100.stdout: "pg_autoscale_mode": "off", 2024-04-07T16:16:16.610 INFO:tasks.workunit.client.0.smithi100.stdout: "pg_num": 1, 2024-04-07T16:16:16.610 INFO:tasks.workunit.client.0.smithi100.stdout: "pgp_num": 1, 2024-04-07T16:16:16.610 INFO:tasks.workunit.client.0.smithi100.stdout: "pool": 1, 2024-04-07T16:16:16.611 INFO:tasks.workunit.client.0.smithi100.stdout: "size": 3, 2024-04-07T16:16:16.611 INFO:tasks.workunit.client.0.smithi100.stdout: "stats": { 2024-04-07T16:16:16.611 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_raw": 729296906506, 2024-04-07T16:16:16.611 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_used": 1781760, 2024-04-07T16:16:16.611 INFO:tasks.workunit.client.0.smithi100.stdout: "compress_bytes_used": 0, 2024-04-07T16:16:16.611 INFO:tasks.workunit.client.0.smithi100.stdout: "compress_under_bytes": 0, 2024-04-07T16:16:16.611 INFO:tasks.workunit.client.0.smithi100.stdout: "data_bytes_used": 1781760, 2024-04-07T16:16:16.611 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty": 0, 2024-04-07T16:16:16.611 INFO:tasks.workunit.client.0.smithi100.stdout: "max_avail": 243098976256, 2024-04-07T16:16:16.611 INFO:tasks.workunit.client.0.smithi100.stdout: "objects": 2, 2024-04-07T16:16:16.611 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_bytes_used": 0, 2024-04-07T16:16:16.611 INFO:tasks.workunit.client.0.smithi100.stdout: "percent_used": 2.4431142264802475e-06, 2024-04-07T16:16:16.611 INFO:tasks.workunit.client.0.smithi100.stdout: "quota_bytes": 0, 2024-04-07T16:16:16.611 INFO:tasks.workunit.client.0.smithi100.stdout: "quota_objects": 0, 2024-04-07T16:16:16.612 INFO:tasks.workunit.client.0.smithi100.stdout: "rd": 466, 2024-04-07T16:16:16.612 INFO:tasks.workunit.client.0.smithi100.stdout: "rd_bytes": 659456, 2024-04-07T16:16:16.612 INFO:tasks.workunit.client.0.smithi100.stdout: "stored": 590368, 2024-04-07T16:16:16.612 INFO:tasks.workunit.client.0.smithi100.stdout: "stored_data": 590368, 2024-04-07T16:16:16.612 INFO:tasks.workunit.client.0.smithi100.stdout: "stored_omap": 0, 2024-04-07T16:16:16.612 INFO:tasks.workunit.client.0.smithi100.stdout: "stored_raw": 1771104, 2024-04-07T16:16:16.612 INFO:tasks.workunit.client.0.smithi100.stdout: "wr": 235, 2024-04-07T16:16:16.612 INFO:tasks.workunit.client.0.smithi100.stdout: "wr_bytes": 2473984 2024-04-07T16:16:16.612 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.612 INFO:tasks.workunit.client.0.smithi100.stdout: "target_max_bytes": 0, 2024-04-07T16:16:16.612 INFO:tasks.workunit.client.0.smithi100.stdout: "target_max_objects": 0, 2024-04-07T16:16:16.612 INFO:tasks.workunit.client.0.smithi100.stdout: "type": "replicated" 2024-04-07T16:16:16.612 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.612 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:16.612 INFO:tasks.workunit.client.0.smithi100.stdout: "application": [ 2024-04-07T16:16:16.613 INFO:tasks.workunit.client.0.smithi100.stdout: "cephfs" 2024-04-07T16:16:16.613 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:16.613 INFO:tasks.workunit.client.0.smithi100.stdout: "cache_mode": "none", 2024-04-07T16:16:16.613 INFO:tasks.workunit.client.0.smithi100.stdout: "erasure_code_profile": {}, 2024-04-07T16:16:16.613 INFO:tasks.workunit.client.0.smithi100.stdout: "min_size": 2, 2024-04-07T16:16:16.613 INFO:tasks.workunit.client.0.smithi100.stdout: "options": {}, 2024-04-07T16:16:16.613 INFO:tasks.workunit.client.0.smithi100.stdout: "pg_autoscale_mode": "off", 2024-04-07T16:16:16.613 INFO:tasks.workunit.client.0.smithi100.stdout: "pg_num": 32, 2024-04-07T16:16:16.613 INFO:tasks.workunit.client.0.smithi100.stdout: "pgp_num": 32, 2024-04-07T16:16:16.613 INFO:tasks.workunit.client.0.smithi100.stdout: "pool": 2, 2024-04-07T16:16:16.613 INFO:tasks.workunit.client.0.smithi100.stdout: "size": 3, 2024-04-07T16:16:16.613 INFO:tasks.workunit.client.0.smithi100.stdout: "stats": { 2024-04-07T16:16:16.613 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_raw": 729296906506, 2024-04-07T16:16:16.613 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_used": 98304, 2024-04-07T16:16:16.613 INFO:tasks.workunit.client.0.smithi100.stdout: "compress_bytes_used": 0, 2024-04-07T16:16:16.614 INFO:tasks.workunit.client.0.smithi100.stdout: "compress_under_bytes": 0, 2024-04-07T16:16:16.614 INFO:tasks.workunit.client.0.smithi100.stdout: "data_bytes_used": 98304, 2024-04-07T16:16:16.614 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty": 0, 2024-04-07T16:16:16.614 INFO:tasks.workunit.client.0.smithi100.stdout: "max_avail": 243098976256, 2024-04-07T16:16:16.614 INFO:tasks.workunit.client.0.smithi100.stdout: "objects": 22, 2024-04-07T16:16:16.614 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_bytes_used": 0, 2024-04-07T16:16:16.614 INFO:tasks.workunit.client.0.smithi100.stdout: "percent_used": 1.3479281335548876e-07, 2024-04-07T16:16:16.614 INFO:tasks.workunit.client.0.smithi100.stdout: "quota_bytes": 0, 2024-04-07T16:16:16.614 INFO:tasks.workunit.client.0.smithi100.stdout: "quota_objects": 0, 2024-04-07T16:16:16.614 INFO:tasks.workunit.client.0.smithi100.stdout: "rd": 0, 2024-04-07T16:16:16.614 INFO:tasks.workunit.client.0.smithi100.stdout: "rd_bytes": 0, 2024-04-07T16:16:16.614 INFO:tasks.workunit.client.0.smithi100.stdout: "stored": 3430, 2024-04-07T16:16:16.614 INFO:tasks.workunit.client.0.smithi100.stdout: "stored_data": 3430, 2024-04-07T16:16:16.614 INFO:tasks.workunit.client.0.smithi100.stdout: "stored_omap": 0, 2024-04-07T16:16:16.615 INFO:tasks.workunit.client.0.smithi100.stdout: "stored_raw": 10290, 2024-04-07T16:16:16.615 INFO:tasks.workunit.client.0.smithi100.stdout: "wr": 51, 2024-04-07T16:16:16.615 INFO:tasks.workunit.client.0.smithi100.stdout: "wr_bytes": 20480 2024-04-07T16:16:16.615 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.615 INFO:tasks.workunit.client.0.smithi100.stdout: "target_max_bytes": 0, 2024-04-07T16:16:16.615 INFO:tasks.workunit.client.0.smithi100.stdout: "target_max_objects": 0, 2024-04-07T16:16:16.615 INFO:tasks.workunit.client.0.smithi100.stdout: "type": "replicated" 2024-04-07T16:16:16.615 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.615 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:16.615 INFO:tasks.workunit.client.0.smithi100.stdout: "application": [ 2024-04-07T16:16:16.615 INFO:tasks.workunit.client.0.smithi100.stdout: "cephfs" 2024-04-07T16:16:16.615 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:16.615 INFO:tasks.workunit.client.0.smithi100.stdout: "cache_mode": "none", 2024-04-07T16:16:16.615 INFO:tasks.workunit.client.0.smithi100.stdout: "erasure_code_profile": {}, 2024-04-07T16:16:16.615 INFO:tasks.workunit.client.0.smithi100.stdout: "min_size": 2, 2024-04-07T16:16:16.616 INFO:tasks.workunit.client.0.smithi100.stdout: "options": {}, 2024-04-07T16:16:16.616 INFO:tasks.workunit.client.0.smithi100.stdout: "pg_autoscale_mode": "off", 2024-04-07T16:16:16.616 INFO:tasks.workunit.client.0.smithi100.stdout: "pg_num": 64, 2024-04-07T16:16:16.616 INFO:tasks.workunit.client.0.smithi100.stdout: "pgp_num": 64, 2024-04-07T16:16:16.616 INFO:tasks.workunit.client.0.smithi100.stdout: "pool": 3, 2024-04-07T16:16:16.616 INFO:tasks.workunit.client.0.smithi100.stdout: "size": 3, 2024-04-07T16:16:16.616 INFO:tasks.workunit.client.0.smithi100.stdout: "stats": { 2024-04-07T16:16:16.616 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_raw": 729296906506, 2024-04-07T16:16:16.616 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_used": 0, 2024-04-07T16:16:16.616 INFO:tasks.workunit.client.0.smithi100.stdout: "compress_bytes_used": 0, 2024-04-07T16:16:16.616 INFO:tasks.workunit.client.0.smithi100.stdout: "compress_under_bytes": 0, 2024-04-07T16:16:16.616 INFO:tasks.workunit.client.0.smithi100.stdout: "data_bytes_used": 0, 2024-04-07T16:16:16.616 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty": 0, 2024-04-07T16:16:16.616 INFO:tasks.workunit.client.0.smithi100.stdout: "max_avail": 243098976256, 2024-04-07T16:16:16.616 INFO:tasks.workunit.client.0.smithi100.stdout: "objects": 0, 2024-04-07T16:16:16.617 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_bytes_used": 0, 2024-04-07T16:16:16.617 INFO:tasks.workunit.client.0.smithi100.stdout: "percent_used": 0.0, 2024-04-07T16:16:16.617 INFO:tasks.workunit.client.0.smithi100.stdout: "quota_bytes": 0, 2024-04-07T16:16:16.617 INFO:tasks.workunit.client.0.smithi100.stdout: "quota_objects": 0, 2024-04-07T16:16:16.617 INFO:tasks.workunit.client.0.smithi100.stdout: "rd": 0, 2024-04-07T16:16:16.617 INFO:tasks.workunit.client.0.smithi100.stdout: "rd_bytes": 0, 2024-04-07T16:16:16.617 INFO:tasks.workunit.client.0.smithi100.stdout: "stored": 0, 2024-04-07T16:16:16.617 INFO:tasks.workunit.client.0.smithi100.stdout: "stored_data": 0, 2024-04-07T16:16:16.617 INFO:tasks.workunit.client.0.smithi100.stdout: "stored_omap": 0, 2024-04-07T16:16:16.617 INFO:tasks.workunit.client.0.smithi100.stdout: "stored_raw": 0, 2024-04-07T16:16:16.617 INFO:tasks.workunit.client.0.smithi100.stdout: "wr": 0, 2024-04-07T16:16:16.617 INFO:tasks.workunit.client.0.smithi100.stdout: "wr_bytes": 0 2024-04-07T16:16:16.617 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.617 INFO:tasks.workunit.client.0.smithi100.stdout: "target_max_bytes": 0, 2024-04-07T16:16:16.618 INFO:tasks.workunit.client.0.smithi100.stdout: "target_max_objects": 0, 2024-04-07T16:16:16.618 INFO:tasks.workunit.client.0.smithi100.stdout: "type": "replicated" 2024-04-07T16:16:16.618 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.618 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:16.618 INFO:tasks.workunit.client.0.smithi100.stdout: "application": [ 2024-04-07T16:16:16.618 INFO:tasks.workunit.client.0.smithi100.stdout: "rbd" 2024-04-07T16:16:16.618 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:16.618 INFO:tasks.workunit.client.0.smithi100.stdout: "cache_mode": "none", 2024-04-07T16:16:16.618 INFO:tasks.workunit.client.0.smithi100.stdout: "erasure_code_profile": {}, 2024-04-07T16:16:16.618 INFO:tasks.workunit.client.0.smithi100.stdout: "min_size": 2, 2024-04-07T16:16:16.618 INFO:tasks.workunit.client.0.smithi100.stdout: "options": {}, 2024-04-07T16:16:16.618 INFO:tasks.workunit.client.0.smithi100.stdout: "pg_autoscale_mode": "off", 2024-04-07T16:16:16.618 INFO:tasks.workunit.client.0.smithi100.stdout: "pg_num": 8, 2024-04-07T16:16:16.618 INFO:tasks.workunit.client.0.smithi100.stdout: "pgp_num": 8, 2024-04-07T16:16:16.618 INFO:tasks.workunit.client.0.smithi100.stdout: "pool": 4, 2024-04-07T16:16:16.619 INFO:tasks.workunit.client.0.smithi100.stdout: "size": 3, 2024-04-07T16:16:16.619 INFO:tasks.workunit.client.0.smithi100.stdout: "stats": { 2024-04-07T16:16:16.619 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_raw": 729296906506, 2024-04-07T16:16:16.619 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_used": 0, 2024-04-07T16:16:16.619 INFO:tasks.workunit.client.0.smithi100.stdout: "compress_bytes_used": 0, 2024-04-07T16:16:16.619 INFO:tasks.workunit.client.0.smithi100.stdout: "compress_under_bytes": 0, 2024-04-07T16:16:16.619 INFO:tasks.workunit.client.0.smithi100.stdout: "data_bytes_used": 0, 2024-04-07T16:16:16.619 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty": 0, 2024-04-07T16:16:16.619 INFO:tasks.workunit.client.0.smithi100.stdout: "max_avail": 243098976256, 2024-04-07T16:16:16.619 INFO:tasks.workunit.client.0.smithi100.stdout: "objects": 0, 2024-04-07T16:16:16.619 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_bytes_used": 0, 2024-04-07T16:16:16.619 INFO:tasks.workunit.client.0.smithi100.stdout: "percent_used": 0.0, 2024-04-07T16:16:16.619 INFO:tasks.workunit.client.0.smithi100.stdout: "quota_bytes": 0, 2024-04-07T16:16:16.619 INFO:tasks.workunit.client.0.smithi100.stdout: "quota_objects": 0, 2024-04-07T16:16:16.619 INFO:tasks.workunit.client.0.smithi100.stdout: "rd": 0, 2024-04-07T16:16:16.619 INFO:tasks.workunit.client.0.smithi100.stdout: "rd_bytes": 0, 2024-04-07T16:16:16.619 INFO:tasks.workunit.client.0.smithi100.stdout: "stored": 0, 2024-04-07T16:16:16.620 INFO:tasks.workunit.client.0.smithi100.stdout: "stored_data": 0, 2024-04-07T16:16:16.620 INFO:tasks.workunit.client.0.smithi100.stdout: "stored_omap": 0, 2024-04-07T16:16:16.620 INFO:tasks.workunit.client.0.smithi100.stdout: "stored_raw": 0, 2024-04-07T16:16:16.620 INFO:tasks.workunit.client.0.smithi100.stdout: "wr": 0, 2024-04-07T16:16:16.620 INFO:tasks.workunit.client.0.smithi100.stdout: "wr_bytes": 0 2024-04-07T16:16:16.620 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.620 INFO:tasks.workunit.client.0.smithi100.stdout: "target_max_bytes": 0, 2024-04-07T16:16:16.620 INFO:tasks.workunit.client.0.smithi100.stdout: "target_max_objects": 0, 2024-04-07T16:16:16.620 INFO:tasks.workunit.client.0.smithi100.stdout: "type": "replicated" 2024-04-07T16:16:16.620 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:16.620 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:16.620 INFO:tasks.workunit.client.0.smithi100.stdout: "rbd": { 2024-04-07T16:16:16.620 INFO:tasks.workunit.client.0.smithi100.stdout: "mirroring_by_pool": [ 2024-04-07T16:16:16.620 INFO:tasks.workunit.client.0.smithi100.stdout: false 2024-04-07T16:16:16.620 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:16.620 INFO:tasks.workunit.client.0.smithi100.stdout: "num_images_by_pool": [ 2024-04-07T16:16:16.621 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T16:16:16.621 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:16.621 INFO:tasks.workunit.client.0.smithi100.stdout: "num_pools": 1 2024-04-07T16:16:16.621 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.621 INFO:tasks.workunit.client.0.smithi100.stdout: "report_id": "ee983b25-10fa-4633-b6cf-1c3036b91b52", 2024-04-07T16:16:16.621 INFO:tasks.workunit.client.0.smithi100.stdout: "report_timestamp": "2024-04-07T16:16:15.447010", 2024-04-07T16:16:16.621 INFO:tasks.workunit.client.0.smithi100.stdout: "report_version": 1, 2024-04-07T16:16:16.621 INFO:tasks.workunit.client.0.smithi100.stdout: "rocksdb_stats": { 2024-04-07T16:16:16.621 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "7.9.2" 2024-04-07T16:16:16.621 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.621 INFO:tasks.workunit.client.0.smithi100.stdout: "rook": { 2024-04-07T16:16:16.621 INFO:tasks.workunit.client.0.smithi100.stdout: "cluster": { 2024-04-07T16:16:16.621 INFO:tasks.workunit.client.0.smithi100.stdout: "external-mode": null, 2024-04-07T16:16:16.621 INFO:tasks.workunit.client.0.smithi100.stdout: "mon": { 2024-04-07T16:16:16.621 INFO:tasks.workunit.client.0.smithi100.stdout: "allow-multiple-per-node": null, 2024-04-07T16:16:16.622 INFO:tasks.workunit.client.0.smithi100.stdout: "count": null, 2024-04-07T16:16:16.622 INFO:tasks.workunit.client.0.smithi100.stdout: "max-id": null, 2024-04-07T16:16:16.622 INFO:tasks.workunit.client.0.smithi100.stdout: "pvc": { 2024-04-07T16:16:16.622 INFO:tasks.workunit.client.0.smithi100.stdout: "enabled": null 2024-04-07T16:16:16.622 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.622 INFO:tasks.workunit.client.0.smithi100.stdout: "stretch": { 2024-04-07T16:16:16.622 INFO:tasks.workunit.client.0.smithi100.stdout: "enabled": null 2024-04-07T16:16:16.622 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:16.622 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.622 INFO:tasks.workunit.client.0.smithi100.stdout: "network": { 2024-04-07T16:16:16.622 INFO:tasks.workunit.client.0.smithi100.stdout: "provider": null 2024-04-07T16:16:16.622 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.622 INFO:tasks.workunit.client.0.smithi100.stdout: "storage": { 2024-04-07T16:16:16.622 INFO:tasks.workunit.client.0.smithi100.stdout: "device-set": { 2024-04-07T16:16:16.623 INFO:tasks.workunit.client.0.smithi100.stdout: "count": { 2024-04-07T16:16:16.623 INFO:tasks.workunit.client.0.smithi100.stdout: "non-portable": null, 2024-04-07T16:16:16.623 INFO:tasks.workunit.client.0.smithi100.stdout: "portable": null, 2024-04-07T16:16:16.623 INFO:tasks.workunit.client.0.smithi100.stdout: "total": null 2024-04-07T16:16:16.623 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:16.623 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:16.623 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:16.623 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.623 INFO:tasks.workunit.client.0.smithi100.stdout: "csi": { 2024-04-07T16:16:16.623 INFO:tasks.workunit.client.0.smithi100.stdout: "version": null 2024-04-07T16:16:16.623 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.623 INFO:tasks.workunit.client.0.smithi100.stdout: "kubernetes": { 2024-04-07T16:16:16.623 INFO:tasks.workunit.client.0.smithi100.stdout: "version": null 2024-04-07T16:16:16.623 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.623 INFO:tasks.workunit.client.0.smithi100.stdout: "node": { 2024-04-07T16:16:16.624 INFO:tasks.workunit.client.0.smithi100.stdout: "count": { 2024-04-07T16:16:16.624 INFO:tasks.workunit.client.0.smithi100.stdout: "kubernetes-total": null, 2024-04-07T16:16:16.624 INFO:tasks.workunit.client.0.smithi100.stdout: "with-ceph-daemons": null, 2024-04-07T16:16:16.624 INFO:tasks.workunit.client.0.smithi100.stdout: "with-csi-cephfs-plugin": null, 2024-04-07T16:16:16.624 INFO:tasks.workunit.client.0.smithi100.stdout: "with-csi-nfs-plugin": null, 2024-04-07T16:16:16.624 INFO:tasks.workunit.client.0.smithi100.stdout: "with-csi-rbd-plugin": null 2024-04-07T16:16:16.624 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:16.624 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.624 INFO:tasks.workunit.client.0.smithi100.stdout: "usage": { 2024-04-07T16:16:16.624 INFO:tasks.workunit.client.0.smithi100.stdout: "storage-class": { 2024-04-07T16:16:16.624 INFO:tasks.workunit.client.0.smithi100.stdout: "count": { 2024-04-07T16:16:16.624 INFO:tasks.workunit.client.0.smithi100.stdout: "bucket": null, 2024-04-07T16:16:16.624 INFO:tasks.workunit.client.0.smithi100.stdout: "cephfs": null, 2024-04-07T16:16:16.624 INFO:tasks.workunit.client.0.smithi100.stdout: "nfs": null, 2024-04-07T16:16:16.625 INFO:tasks.workunit.client.0.smithi100.stdout: "rbd": null, 2024-04-07T16:16:16.625 INFO:tasks.workunit.client.0.smithi100.stdout: "total": null 2024-04-07T16:16:16.625 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:16.625 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:16.625 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.625 INFO:tasks.workunit.client.0.smithi100.stdout: "version": null 2024-04-07T16:16:16.625 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.625 INFO:tasks.workunit.client.0.smithi100.stdout: "services": {}, 2024-04-07T16:16:16.625 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_per_pg": [ 2024-04-07T16:16:16.625 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:16.625 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:16.625 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T16:16:16.625 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T16:16:16.625 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T16:16:16.625 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:16.626 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 7, 2024-04-07T16:16:16.626 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:16.626 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:16.626 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T16:16:16.626 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:16.626 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:16.626 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:16.626 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.637813+0000", 2024-04-07T16:16:16.626 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.395023+0000", 2024-04-07T16:16:16.626 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.395023+0000", 2024-04-07T16:16:16.626 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.395367+0000", 2024-04-07T16:16:16.626 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.637813+0000", 2024-04-07T16:16:16.626 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:16.626 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:16.627 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:16.627 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 61, 2024-04-07T16:16:16.627 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.637813+0000", 2024-04-07T16:16:16.627 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.637813+0000", 2024-04-07T16:16:16.627 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.637813+0000", 2024-04-07T16:16:16.627 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:16.627 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:16.627 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:16.627 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.637813+0000", 2024-04-07T16:16:16.627 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.637813+0000", 2024-04-07T16:16:16.627 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:16.627 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:16.627 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:16.627 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:16.627 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 60, 2024-04-07T16:16:16.628 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:16.628 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:16.628 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:16.628 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:16.628 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:16.628 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:16.628 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:16.628 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:16.628 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.3f", 2024-04-07T16:16:16.628 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:16.628 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:16.628 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:16.628 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 19, 2024-04-07T16:16:16.628 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:16.628 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T23:05:09.752696+0000", 2024-04-07T16:16:16.628 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:16.628 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:16.629 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:16.629 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:16.629 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:16.629 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:16.629 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:16.629 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:16.629 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:16.629 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:16.629 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:16.629 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:16.629 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:16.629 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:16.629 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:16.629 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:16.629 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:16.629 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:16.630 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:16.630 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:16.630 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:16.630 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:16.630 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:16.630 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:16.630 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:16.630 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:16.630 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:16.630 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:16.630 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:16.630 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:16.630 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:16.630 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:16.630 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:16.631 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:16.631 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:16.631 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:16.631 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:16.631 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:16.631 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:16.631 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:16.631 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:16.631 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:16.631 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:16.631 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.631 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:16.631 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:16.631 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:16.631 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T16:16:16.631 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T16:16:16.632 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T16:16:16.632 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:16.632 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 7, 2024-04-07T16:16:16.632 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:16.632 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.632 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:16.632 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:16.632 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T16:16:16.632 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T16:16:16.632 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T16:16:16.632 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:16.632 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 1, 2024-04-07T16:16:16.632 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:16.632 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:16.632 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T16:16:16.632 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:16.632 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:16.633 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:16.633 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.237070+0000", 2024-04-07T16:16:16.633 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.385175+0000", 2024-04-07T16:16:16.633 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.385175+0000", 2024-04-07T16:16:16.633 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.385549+0000", 2024-04-07T16:16:16.633 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.237070+0000", 2024-04-07T16:16:16.633 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:16.633 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:16.633 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:16.633 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 61, 2024-04-07T16:16:16.633 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.237070+0000", 2024-04-07T16:16:16.633 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.237070+0000", 2024-04-07T16:16:16.633 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.237070+0000", 2024-04-07T16:16:16.633 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:16.633 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:16.633 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:16.634 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.237070+0000", 2024-04-07T16:16:16.634 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.237070+0000", 2024-04-07T16:16:16.634 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:16.634 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:16.634 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:16.634 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:16.634 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 60, 2024-04-07T16:16:16.634 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:16.634 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:16.634 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:16.634 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:16.634 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:16.634 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:16.634 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:16.634 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:16.634 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.3e", 2024-04-07T16:16:16.635 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:16.635 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:16.635 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:16.635 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 19, 2024-04-07T16:16:16.635 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:16.635 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T16:28:22.788092+0000", 2024-04-07T16:16:16.635 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:16.635 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:16.635 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:16.635 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:16.635 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:16.635 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:16.635 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:16.635 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:16.635 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:16.635 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:16.635 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:16.636 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:16.636 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:16.636 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:16.636 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:16.636 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:16.636 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:16.636 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:16.636 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:16.636 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:16.636 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:16.636 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:16.636 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:16.636 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:16.636 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:16.636 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:16.637 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:16.637 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:16.637 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:16.637 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:16.637 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:16.637 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:16.637 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:16.637 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:16.637 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:16.637 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:16.637 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:16.637 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:16.637 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:16.637 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:16.637 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:16.638 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:16.638 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:16.638 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.638 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:16.638 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:16.638 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:16.638 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T16:16:16.638 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T16:16:16.638 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T16:16:16.638 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:16.638 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 1, 2024-04-07T16:16:16.638 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:16.638 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.638 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:16.638 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:16.639 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T16:16:16.639 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T16:16:16.639 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T16:16:16.639 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:16.639 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 0, 2024-04-07T16:16:16.639 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:16.639 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:16.639 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T16:16:16.639 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:16.639 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:16.639 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:16.639 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.030532+0000", 2024-04-07T16:16:16.639 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.410088+0000", 2024-04-07T16:16:16.639 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.410088+0000", 2024-04-07T16:16:16.639 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.410431+0000", 2024-04-07T16:16:16.639 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.030532+0000", 2024-04-07T16:16:16.640 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:16.640 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:16.640 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:16.640 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 61, 2024-04-07T16:16:16.640 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.030532+0000", 2024-04-07T16:16:16.640 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.030532+0000", 2024-04-07T16:16:16.640 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.030532+0000", 2024-04-07T16:16:16.640 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:16.640 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:16.640 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:16.640 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.030532+0000", 2024-04-07T16:16:16.640 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.030532+0000", 2024-04-07T16:16:16.640 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:16.640 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:16.640 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:16.641 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:16.641 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 60, 2024-04-07T16:16:16.641 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:16.641 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:16.641 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:16.641 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:16.641 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:16.641 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:16.641 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:16.641 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:16.641 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.3d", 2024-04-07T16:16:16.641 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:16.641 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:16.641 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:16.641 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 19, 2024-04-07T16:16:16.642 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:16.642 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T19:47:57.716144+0000", 2024-04-07T16:16:16.642 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:16.642 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:16.642 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:16.642 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:16.642 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:16.642 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:16.642 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:16.642 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:16.642 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:16.642 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:16.642 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:16.642 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:16.642 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:16.643 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:16.643 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:16.643 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:16.643 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:16.643 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:16.643 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:16.643 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:16.643 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:16.643 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:16.643 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:16.643 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:16.643 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:16.643 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:16.643 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:16.643 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:16.644 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:16.644 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:16.644 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:16.644 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:16.644 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:16.644 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:16.644 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:16.644 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:16.644 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:16.644 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:16.644 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:16.644 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:16.644 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:16.644 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:16.644 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:16.644 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.645 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:16.645 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:16.645 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:16.645 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T16:16:16.645 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T16:16:16.645 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T16:16:16.645 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:16.645 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 0, 2024-04-07T16:16:16.645 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:16.645 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.645 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:16.645 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:16.645 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T16:16:16.645 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T16:16:16.645 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T16:16:16.646 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:16.646 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 5, 2024-04-07T16:16:16.646 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:16.646 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:16.646 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T16:16:16.646 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:16.646 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:16.646 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:16.646 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.025110+0000", 2024-04-07T16:16:16.646 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.401819+0000", 2024-04-07T16:16:16.646 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.401819+0000", 2024-04-07T16:16:16.646 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.402328+0000", 2024-04-07T16:16:16.646 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.025110+0000", 2024-04-07T16:16:16.646 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:16.646 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:16.647 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:16.647 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 61, 2024-04-07T16:16:16.647 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.025110+0000", 2024-04-07T16:16:16.647 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.025110+0000", 2024-04-07T16:16:16.647 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.025110+0000", 2024-04-07T16:16:16.647 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:16.647 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:16.647 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:16.647 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.025110+0000", 2024-04-07T16:16:16.647 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.025110+0000", 2024-04-07T16:16:16.647 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:16.647 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:16.647 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:16.647 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:16.647 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 60, 2024-04-07T16:16:16.647 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:16.648 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:16.648 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:16.648 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:16.648 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:16.648 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:16.648 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:16.648 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:16.648 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.3c", 2024-04-07T16:16:16.648 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:16.648 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:16.648 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:16.648 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 19, 2024-04-07T16:16:16.648 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:16.649 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T21:43:13.292859+0000", 2024-04-07T16:16:16.649 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:16.649 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:16.649 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:16.649 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:16.649 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:16.649 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:16.649 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:16.649 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:16.649 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:16.649 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:16.649 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:16.649 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:16.650 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:16.650 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:16.650 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:16.650 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:16.650 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:16.650 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:16.650 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:16.650 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:16.650 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:16.650 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:16.650 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:16.650 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:16.650 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:16.650 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:16.651 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:16.651 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:16.651 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:16.651 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:16.651 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:16.651 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:16.651 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:16.651 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:16.651 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:16.651 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:16.651 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:16.651 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:16.651 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:16.651 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:16.651 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:16.652 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:16.652 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:16.652 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.652 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:16.652 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:16.652 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:16.652 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T16:16:16.652 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T16:16:16.652 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T16:16:16.652 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:16.652 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 5, 2024-04-07T16:16:16.652 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:16.652 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.652 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:16.652 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:16.653 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T16:16:16.653 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T16:16:16.653 INFO:tasks.workunit.client.0.smithi100.stdout: 4 2024-04-07T16:16:16.653 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:16.653 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 7, 2024-04-07T16:16:16.653 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:16.653 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:16.653 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T16:16:16.653 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:16.653 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:16.653 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:16.653 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.637270+0000", 2024-04-07T16:16:16.653 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.413281+0000", 2024-04-07T16:16:16.653 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.413281+0000", 2024-04-07T16:16:16.654 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.413487+0000", 2024-04-07T16:16:16.654 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.637270+0000", 2024-04-07T16:16:16.654 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:16.654 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:16.654 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:16.654 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 61, 2024-04-07T16:16:16.654 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.637270+0000", 2024-04-07T16:16:16.654 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.637270+0000", 2024-04-07T16:16:16.654 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.637270+0000", 2024-04-07T16:16:16.654 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:16.654 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:16.654 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:16.654 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.637270+0000", 2024-04-07T16:16:16.654 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.637270+0000", 2024-04-07T16:16:16.655 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:16.655 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:16.655 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:16.655 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:16.655 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 60, 2024-04-07T16:16:16.655 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:16.655 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:16.655 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:16.655 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:16.655 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:16.655 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:16.655 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:16.655 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:16.655 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.3b", 2024-04-07T16:16:16.655 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:16.656 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:16.656 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:16.656 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 19, 2024-04-07T16:16:16.656 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:16.656 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T23:59:23.302803+0000", 2024-04-07T16:16:16.656 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:16.656 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:16.656 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:16.656 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:16.656 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:16.656 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:16.656 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:16.656 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:16.656 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:16.656 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:16.657 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:16.657 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:16.657 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:16.657 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:16.657 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:16.657 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:16.657 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:16.657 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:16.657 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:16.657 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:16.657 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:16.657 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:16.657 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:16.657 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:16.657 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:16.658 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:16.658 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:16.658 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:16.658 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:16.658 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:16.658 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:16.658 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:16.658 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:16.658 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:16.658 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:16.658 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:16.658 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:16.658 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:16.658 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:16.658 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:16.659 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:16.659 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:16.659 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:16.659 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.659 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:16.659 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:16.659 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:16.659 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T16:16:16.659 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T16:16:16.659 INFO:tasks.workunit.client.0.smithi100.stdout: 4 2024-04-07T16:16:16.659 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:16.659 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 7, 2024-04-07T16:16:16.659 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:16.659 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.659 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:16.659 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:16.660 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T16:16:16.660 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T16:16:16.660 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T16:16:16.660 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:16.660 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 5, 2024-04-07T16:16:16.660 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:16.660 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:16.660 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T16:16:16.660 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:16.660 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:16.660 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:16.660 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.024848+0000", 2024-04-07T16:16:16.660 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.400365+0000", 2024-04-07T16:16:16.660 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.400365+0000", 2024-04-07T16:16:16.660 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.401561+0000", 2024-04-07T16:16:16.661 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.024848+0000", 2024-04-07T16:16:16.661 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:16.661 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:16.661 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:16.661 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 61, 2024-04-07T16:16:16.661 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.024848+0000", 2024-04-07T16:16:16.661 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.024848+0000", 2024-04-07T16:16:16.661 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.024848+0000", 2024-04-07T16:16:16.661 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:16.661 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:16.661 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:16.661 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.024848+0000", 2024-04-07T16:16:16.661 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.024848+0000", 2024-04-07T16:16:16.661 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:16.661 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:16.662 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:16.662 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:16.662 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 60, 2024-04-07T16:16:16.662 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:16.662 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:16.662 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:16.662 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:16.662 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:16.662 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:16.662 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:16.662 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:16.662 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.3a", 2024-04-07T16:16:16.662 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:16.662 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:16.663 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:16.663 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 19, 2024-04-07T16:16:16.663 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:16.663 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T20:14:49.110462+0000", 2024-04-07T16:16:16.663 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:16.663 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:16.663 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:16.663 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:16.663 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:16.663 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:16.663 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:16.663 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:16.663 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:16.663 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:16.663 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:16.663 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:16.664 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:16.664 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:16.664 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:16.664 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:16.664 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:16.664 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:16.664 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:16.664 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:16.664 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:16.664 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:16.664 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:16.664 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:16.664 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:16.664 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:16.664 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:16.665 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:16.665 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:16.665 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:16.665 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:16.665 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:16.665 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:16.665 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:16.665 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:16.665 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:16.665 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:16.665 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:16.665 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:16.665 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:16.665 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:16.665 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:16.666 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:16.666 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.666 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:16.666 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:16.666 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:16.666 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T16:16:16.666 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T16:16:16.666 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T16:16:16.666 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:16.666 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 5, 2024-04-07T16:16:16.666 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:16.666 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.666 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:16.666 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:16.666 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T16:16:16.667 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T16:16:16.667 INFO:tasks.workunit.client.0.smithi100.stdout: 4 2024-04-07T16:16:16.667 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:16.667 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 7, 2024-04-07T16:16:16.667 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:16.667 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:16.667 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T16:16:16.667 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:16.667 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:16.667 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:16.667 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.636229+0000", 2024-04-07T16:16:16.667 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.393187+0000", 2024-04-07T16:16:16.667 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.393187+0000", 2024-04-07T16:16:16.667 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.393518+0000", 2024-04-07T16:16:16.667 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.636229+0000", 2024-04-07T16:16:16.668 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:16.668 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:16.668 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:16.668 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 61, 2024-04-07T16:16:16.668 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.636229+0000", 2024-04-07T16:16:16.668 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.636229+0000", 2024-04-07T16:16:16.668 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.636229+0000", 2024-04-07T16:16:16.668 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:16.668 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:16.668 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:16.668 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.636229+0000", 2024-04-07T16:16:16.668 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.636229+0000", 2024-04-07T16:16:16.668 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:16.668 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:16.668 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:16.669 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:16.669 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 60, 2024-04-07T16:16:16.669 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:16.669 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:16.669 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:16.669 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:16.669 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:16.669 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:16.669 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:16.669 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:16.669 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.39", 2024-04-07T16:16:16.669 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:16.669 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:16.669 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:16.669 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 19, 2024-04-07T16:16:16.669 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:16.670 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T19:41:28.702905+0000", 2024-04-07T16:16:16.670 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:16.670 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:16.670 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:16.670 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:16.670 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:16.670 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:16.670 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:16.670 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:16.670 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:16.670 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:16.670 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:16.670 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:16.670 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:16.670 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:16.671 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:16.671 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:16.671 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:16.671 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:16.671 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:16.671 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:16.671 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:16.671 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:16.671 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:16.671 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:16.671 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:16.671 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:16.671 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:16.671 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:16.671 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:16.672 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:16.672 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:16.672 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:16.672 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:16.672 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:16.672 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:16.672 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:16.672 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:16.672 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:16.672 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:16.672 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:16.672 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:16.672 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:16.672 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:16.672 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.673 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:16.673 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:16.673 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:16.673 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T16:16:16.673 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T16:16:16.673 INFO:tasks.workunit.client.0.smithi100.stdout: 4 2024-04-07T16:16:16.673 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:16.673 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 7, 2024-04-07T16:16:16.673 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:16.673 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.673 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:16.673 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:16.673 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T16:16:16.673 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T16:16:16.673 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T16:16:16.673 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:16.674 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 3, 2024-04-07T16:16:16.674 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:16.674 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:16.674 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T16:16:16.674 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:16.674 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:16.674 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:16.674 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.187095+0000", 2024-04-07T16:16:16.674 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.394422+0000", 2024-04-07T16:16:16.674 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.394422+0000", 2024-04-07T16:16:16.674 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.394702+0000", 2024-04-07T16:16:16.674 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.187095+0000", 2024-04-07T16:16:16.674 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:16.674 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:16.674 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:16.675 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 61, 2024-04-07T16:16:16.675 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.187095+0000", 2024-04-07T16:16:16.675 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.187095+0000", 2024-04-07T16:16:16.675 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.187095+0000", 2024-04-07T16:16:16.675 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:16.675 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:16.675 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:16.675 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.187095+0000", 2024-04-07T16:16:16.675 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.187095+0000", 2024-04-07T16:16:16.675 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:16.675 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:16.675 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:16.675 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:16.675 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 60, 2024-04-07T16:16:16.676 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:16.676 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:16.676 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:16.676 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:16.676 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:16.676 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:16.676 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:16.676 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:16.676 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.38", 2024-04-07T16:16:16.676 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:16.676 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:16.676 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:16.676 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 19, 2024-04-07T16:16:16.676 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:16.676 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T00:37:44.512471+0000", 2024-04-07T16:16:16.676 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:16.677 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:16.677 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:16.677 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:16.677 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:16.677 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:16.677 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:16.677 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:16.677 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:16.677 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:16.677 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:16.677 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:16.677 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:16.677 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:16.677 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:16.678 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:16.678 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:16.678 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:16.678 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:16.678 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:16.678 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:16.678 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:16.678 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:16.678 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:16.678 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:16.678 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:16.678 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:16.678 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:16.678 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:16.678 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:16.679 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:16.679 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:16.679 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:16.679 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:16.679 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:16.679 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:16.679 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:16.679 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:16.679 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:16.679 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:16.679 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:16.679 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:16.679 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:16.679 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.679 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:16.680 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:16.680 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:16.680 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T16:16:16.680 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T16:16:16.680 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T16:16:16.680 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:16.680 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 3, 2024-04-07T16:16:16.680 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:16.680 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.680 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:16.680 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:16.680 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T16:16:16.680 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T16:16:16.680 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T16:16:16.680 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:16.680 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 7, 2024-04-07T16:16:16.681 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:16.681 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:16.681 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T16:16:16.681 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:16.681 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:16.681 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:16.681 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.637842+0000", 2024-04-07T16:16:16.681 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.401411+0000", 2024-04-07T16:16:16.681 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.401411+0000", 2024-04-07T16:16:16.681 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.401766+0000", 2024-04-07T16:16:16.681 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.637842+0000", 2024-04-07T16:16:16.681 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:16.681 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:16.681 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:16.681 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 61, 2024-04-07T16:16:16.682 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.637842+0000", 2024-04-07T16:16:16.682 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.637842+0000", 2024-04-07T16:16:16.682 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.637842+0000", 2024-04-07T16:16:16.682 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:16.682 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:16.682 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:16.682 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.637842+0000", 2024-04-07T16:16:16.682 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.637842+0000", 2024-04-07T16:16:16.682 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:16.682 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:16.682 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:16.682 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:16.682 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 60, 2024-04-07T16:16:16.682 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:16.682 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:16.683 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:16.683 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:16.683 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:16.683 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:16.683 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:16.683 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:16.683 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.37", 2024-04-07T16:16:16.683 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:16.683 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:16.683 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:16.683 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 19, 2024-04-07T16:16:16.683 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:16.683 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T23:32:52.283410+0000", 2024-04-07T16:16:16.683 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:16.684 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:16.684 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:16.684 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:16.684 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:16.684 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:16.684 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:16.684 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:16.684 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:16.684 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:16.684 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:16.684 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:16.684 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:16.684 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:16.684 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:16.684 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:16.684 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:16.685 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:16.685 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:16.685 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:16.685 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:16.685 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:16.685 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:16.685 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:16.685 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:16.685 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:16.685 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:16.685 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:16.685 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:16.685 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:16.685 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:16.686 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:16.686 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:16.686 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:16.686 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:16.686 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:16.686 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:16.686 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:16.686 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:16.686 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:16.686 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:16.686 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:16.686 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:16.686 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.686 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:16.687 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:16.687 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:16.687 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T16:16:16.687 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T16:16:16.687 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T16:16:16.687 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:16.687 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 7, 2024-04-07T16:16:16.687 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:16.687 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.687 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:16.687 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:16.687 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T16:16:16.687 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T16:16:16.687 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T16:16:16.688 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:16.688 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 6, 2024-04-07T16:16:16.688 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:16.688 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:16.688 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T16:16:16.688 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:16.688 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:16.688 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:16.688 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.553143+0000", 2024-04-07T16:16:16.688 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.396094+0000", 2024-04-07T16:16:16.688 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.396094+0000", 2024-04-07T16:16:16.688 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.396409+0000", 2024-04-07T16:16:16.688 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.553143+0000", 2024-04-07T16:16:16.688 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:16.689 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:16.689 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:16.689 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 61, 2024-04-07T16:16:16.689 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.553143+0000", 2024-04-07T16:16:16.689 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.553143+0000", 2024-04-07T16:16:16.689 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.553143+0000", 2024-04-07T16:16:16.689 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:16.689 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:16.689 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:16.689 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.553143+0000", 2024-04-07T16:16:16.689 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.553143+0000", 2024-04-07T16:16:16.689 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:16.689 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:16.689 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:16.689 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:16.690 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 60, 2024-04-07T16:16:16.690 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:16.690 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:16.690 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:16.690 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:16.690 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:16.690 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:16.690 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:16.690 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:16.690 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.36", 2024-04-07T16:16:16.690 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:16.690 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:16.690 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:16.690 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 19, 2024-04-07T16:16:16.690 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:16.691 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T18:54:36.072837+0000", 2024-04-07T16:16:16.691 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:16.691 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:16.691 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:16.691 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:16.691 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:16.691 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:16.691 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:16.691 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:16.691 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:16.691 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:16.691 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:16.691 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:16.691 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:16.691 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:16.692 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:16.692 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:16.692 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:16.692 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:16.692 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:16.692 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:16.692 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:16.692 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:16.692 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:16.692 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:16.692 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:16.692 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:16.692 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:16.692 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:16.692 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:16.693 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:16.693 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:16.693 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:16.693 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:16.693 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:16.693 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:16.693 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:16.693 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:16.693 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:16.693 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:16.693 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:16.693 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:16.693 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:16.693 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:16.693 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.694 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:16.694 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:16.694 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:16.694 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T16:16:16.694 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T16:16:16.694 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T16:16:16.694 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:16.694 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 6, 2024-04-07T16:16:16.694 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:16.694 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.694 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:16.694 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:16.694 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T16:16:16.694 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T16:16:16.694 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T16:16:16.694 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:16.695 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 6, 2024-04-07T16:16:16.695 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:16.695 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:16.695 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T16:16:16.695 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:16.695 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:16.695 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:16.695 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.552867+0000", 2024-04-07T16:16:16.695 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.401724+0000", 2024-04-07T16:16:16.695 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.401724+0000", 2024-04-07T16:16:16.695 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.401973+0000", 2024-04-07T16:16:16.695 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.552867+0000", 2024-04-07T16:16:16.695 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:16.695 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:16.695 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:16.696 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 61, 2024-04-07T16:16:16.696 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.552867+0000", 2024-04-07T16:16:16.696 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.552867+0000", 2024-04-07T16:16:16.696 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.552867+0000", 2024-04-07T16:16:16.696 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:16.696 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:16.696 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:16.696 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.552867+0000", 2024-04-07T16:16:16.696 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.552867+0000", 2024-04-07T16:16:16.696 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:16.696 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:16.696 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:16.696 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:16.696 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 60, 2024-04-07T16:16:16.696 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:16.696 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:16.697 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:16.697 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:16.697 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:16.697 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:16.697 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:16.697 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:16.697 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.35", 2024-04-07T16:16:16.697 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:16.697 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:16.697 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:16.697 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 19, 2024-04-07T16:16:16.697 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:16.697 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T21:26:29.208223+0000", 2024-04-07T16:16:16.697 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:16.697 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:16.698 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:16.698 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:16.698 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:16.698 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:16.698 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:16.698 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:16.698 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:16.698 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:16.698 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:16.698 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:16.698 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:16.698 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:16.698 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:16.698 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:16.698 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:16.698 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:16.698 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:16.699 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:16.699 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:16.699 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:16.699 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:16.699 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:16.699 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:16.699 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:16.699 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:16.699 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:16.699 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:16.699 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:16.699 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:16.699 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:16.699 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:16.699 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:16.699 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:16.700 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:16.700 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:16.700 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:16.700 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:16.700 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:16.700 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:16.700 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:16.700 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:16.700 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.700 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:16.700 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:16.700 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:16.700 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T16:16:16.700 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T16:16:16.700 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T16:16:16.700 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:16.700 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 6, 2024-04-07T16:16:16.701 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:16.701 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.701 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:16.701 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:16.701 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T16:16:16.701 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T16:16:16.701 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T16:16:16.701 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:16.701 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 4, 2024-04-07T16:16:16.701 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:16.701 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:16.701 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T16:16:16.701 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:16.701 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:16.701 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:16.702 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:39.824604+0000", 2024-04-07T16:16:16.702 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.404529+0000", 2024-04-07T16:16:16.702 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.404529+0000", 2024-04-07T16:16:16.702 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.404858+0000", 2024-04-07T16:16:16.702 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:39.824604+0000", 2024-04-07T16:16:16.702 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:16.702 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:16.702 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:16.702 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 61, 2024-04-07T16:16:16.702 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:39.824604+0000", 2024-04-07T16:16:16.702 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:39.824604+0000", 2024-04-07T16:16:16.702 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:39.824604+0000", 2024-04-07T16:16:16.702 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:16.702 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:16.702 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:16.703 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:39.824604+0000", 2024-04-07T16:16:16.703 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:39.824604+0000", 2024-04-07T16:16:16.703 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:16.703 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:16.703 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:16.703 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:16.703 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 60, 2024-04-07T16:16:16.703 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:16.703 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:16.703 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:16.703 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:16.703 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:16.703 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:16.703 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:16.703 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:16.703 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.34", 2024-04-07T16:16:16.703 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:16.704 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:16.704 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:16.704 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 19, 2024-04-07T16:16:16.704 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:16.704 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T19:57:22.893626+0000", 2024-04-07T16:16:16.704 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:16.704 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:16.704 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:16.704 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:16.704 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:16.704 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:16.704 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:16.704 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:16.704 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:16.704 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:16.704 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:16.705 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:16.705 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:16.705 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:16.705 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:16.705 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:16.705 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:16.705 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:16.705 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:16.705 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:16.705 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:16.705 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:16.705 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:16.705 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:16.705 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:16.705 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:16.705 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:16.705 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:16.706 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:16.706 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:16.706 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:16.706 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:16.706 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:16.706 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:16.706 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:16.706 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:16.706 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:16.706 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:16.706 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:16.706 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:16.706 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:16.706 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:16.706 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:16.707 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.707 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:16.707 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:16.707 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:16.707 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T16:16:16.707 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T16:16:16.707 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T16:16:16.707 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:16.707 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 4, 2024-04-07T16:16:16.707 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:16.707 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.707 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:16.707 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:16.707 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T16:16:16.707 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T16:16:16.707 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T16:16:16.708 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:16.708 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 3, 2024-04-07T16:16:16.708 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:16.708 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:16.708 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T16:16:16.708 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:16.708 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:16.708 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:16.708 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.186817+0000", 2024-04-07T16:16:16.708 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.394402+0000", 2024-04-07T16:16:16.708 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.394402+0000", 2024-04-07T16:16:16.708 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.394760+0000", 2024-04-07T16:16:16.708 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.186817+0000", 2024-04-07T16:16:16.708 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:16.708 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:16.708 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:16.709 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 61, 2024-04-07T16:16:16.709 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.186817+0000", 2024-04-07T16:16:16.709 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.186817+0000", 2024-04-07T16:16:16.709 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.186817+0000", 2024-04-07T16:16:16.709 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:16.709 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:16.709 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:16.709 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.186817+0000", 2024-04-07T16:16:16.709 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.186817+0000", 2024-04-07T16:16:16.709 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:16.709 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:16.709 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:16.709 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:16.709 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 60, 2024-04-07T16:16:16.709 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:16.709 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:16.710 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:16.710 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:16.710 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:16.710 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:16.710 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:16.710 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:16.710 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.33", 2024-04-07T16:16:16.710 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:16.710 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:16.710 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:16.710 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 19, 2024-04-07T16:16:16.710 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:16.710 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T18:47:13.891972+0000", 2024-04-07T16:16:16.710 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:16.710 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:16.710 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:16.711 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:16.711 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:16.711 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:16.711 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:16.711 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:16.711 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:16.711 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:16.711 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:16.711 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:16.711 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:16.711 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:16.711 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:16.711 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:16.711 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:16.711 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:16.711 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:16.712 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:16.712 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:16.712 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:16.712 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:16.712 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:16.712 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:16.712 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:16.712 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:16.712 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:16.712 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:16.712 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:16.712 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:16.712 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:16.712 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:16.712 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:16.712 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:16.713 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:16.713 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:16.713 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:16.713 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:16.713 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:16.713 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:16.713 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:16.713 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:16.713 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.713 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:16.713 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:16.713 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:16.713 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T16:16:16.713 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T16:16:16.713 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T16:16:16.713 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:16.714 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 3, 2024-04-07T16:16:16.714 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:16.714 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.714 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:16.714 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:16.714 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T16:16:16.714 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T16:16:16.714 INFO:tasks.workunit.client.0.smithi100.stdout: 5 2024-04-07T16:16:16.714 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:16.714 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 0, 2024-04-07T16:16:16.714 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:16.714 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:16.714 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T16:16:16.714 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:16.714 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:16.714 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:16.715 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.029316+0000", 2024-04-07T16:16:16.715 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.400028+0000", 2024-04-07T16:16:16.715 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.400028+0000", 2024-04-07T16:16:16.715 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.400292+0000", 2024-04-07T16:16:16.715 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.029316+0000", 2024-04-07T16:16:16.715 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:16.715 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:16.715 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:16.715 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 61, 2024-04-07T16:16:16.715 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.029316+0000", 2024-04-07T16:16:16.715 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.029316+0000", 2024-04-07T16:16:16.715 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.029316+0000", 2024-04-07T16:16:16.715 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:16.715 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:16.715 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:16.716 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.029316+0000", 2024-04-07T16:16:16.716 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.029316+0000", 2024-04-07T16:16:16.716 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:16.716 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:16.716 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:16.716 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:16.716 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 60, 2024-04-07T16:16:16.716 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:16.716 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:16.716 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:16.716 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:16.716 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:16.716 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:16.716 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:16.716 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:16.717 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.32", 2024-04-07T16:16:16.717 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:16.717 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:16.717 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:16.717 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 19, 2024-04-07T16:16:16.717 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:16.717 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T18:14:33.693852+0000", 2024-04-07T16:16:16.717 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:16.717 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:16.717 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:16.717 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:16.717 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:16.717 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:16.717 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:16.717 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:16.717 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:16.718 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:16.718 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:16.718 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:16.718 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:16.718 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:16.718 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:16.718 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:16.718 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:16.718 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:16.718 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:16.718 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:16.718 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:16.718 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:16.718 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:16.718 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:16.719 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:16.719 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:16.719 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:16.719 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:16.719 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:16.719 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:16.719 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:16.719 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:16.719 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:16.719 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:16.719 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:16.719 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:16.719 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:16.719 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:16.719 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:16.719 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:16.720 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:16.720 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:16.720 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:16.720 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.720 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:16.720 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:16.720 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:16.720 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T16:16:16.720 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T16:16:16.720 INFO:tasks.workunit.client.0.smithi100.stdout: 5 2024-04-07T16:16:16.720 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:16.720 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 0, 2024-04-07T16:16:16.720 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:16.720 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.720 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:16.720 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:16.721 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T16:16:16.721 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T16:16:16.721 INFO:tasks.workunit.client.0.smithi100.stdout: 5 2024-04-07T16:16:16.721 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:16.721 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 1, 2024-04-07T16:16:16.721 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:16.721 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:16.721 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T16:16:16.721 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:16.721 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:16.721 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:16.721 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.235899+0000", 2024-04-07T16:16:16.721 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.398287+0000", 2024-04-07T16:16:16.721 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.398287+0000", 2024-04-07T16:16:16.721 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.398535+0000", 2024-04-07T16:16:16.721 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.235899+0000", 2024-04-07T16:16:16.722 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:16.722 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:16.722 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:16.722 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 61, 2024-04-07T16:16:16.722 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.235899+0000", 2024-04-07T16:16:16.722 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.235899+0000", 2024-04-07T16:16:16.722 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.235899+0000", 2024-04-07T16:16:16.722 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:16.722 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:16.722 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:16.722 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.235899+0000", 2024-04-07T16:16:16.722 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.235899+0000", 2024-04-07T16:16:16.722 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:16.722 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:16.722 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:16.723 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:16.723 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 60, 2024-04-07T16:16:16.723 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:16.723 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:16.723 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:16.723 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:16.723 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:16.723 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:16.723 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:16.723 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:16.723 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.31", 2024-04-07T16:16:16.723 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:16.723 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:16.723 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:16.723 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 19, 2024-04-07T16:16:16.723 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:16.724 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T23:58:53.081885+0000", 2024-04-07T16:16:16.724 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:16.724 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:16.724 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:16.724 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:16.724 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:16.724 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:16.724 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:16.724 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:16.724 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:16.724 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:16.724 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:16.724 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:16.725 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:16.725 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:16.725 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:16.725 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:16.725 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:16.725 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:16.725 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:16.725 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:16.725 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:16.725 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:16.725 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:16.725 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:16.725 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:16.725 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:16.725 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:16.726 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:16.726 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:16.726 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:16.726 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:16.726 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:16.726 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:16.726 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:16.726 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:16.726 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:16.726 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:16.726 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:16.726 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:16.726 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:16.726 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:16.727 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:16.727 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:16.727 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.727 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:16.727 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:16.727 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:16.727 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T16:16:16.727 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T16:16:16.727 INFO:tasks.workunit.client.0.smithi100.stdout: 5 2024-04-07T16:16:16.727 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:16.727 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 1, 2024-04-07T16:16:16.727 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:16.727 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.727 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:16.727 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:16.728 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T16:16:16.728 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T16:16:16.728 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T16:16:16.728 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:16.728 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 5, 2024-04-07T16:16:16.728 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:16.728 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:16.728 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T16:16:16.728 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:16.728 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:16.728 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:16.728 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.023905+0000", 2024-04-07T16:16:16.728 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.394956+0000", 2024-04-07T16:16:16.728 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.394956+0000", 2024-04-07T16:16:16.728 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.395243+0000", 2024-04-07T16:16:16.728 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.023905+0000", 2024-04-07T16:16:16.728 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:16.729 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:16.729 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:16.729 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 61, 2024-04-07T16:16:16.729 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.023905+0000", 2024-04-07T16:16:16.729 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.023905+0000", 2024-04-07T16:16:16.729 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.023905+0000", 2024-04-07T16:16:16.729 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:16.729 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:16.729 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:16.729 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.023905+0000", 2024-04-07T16:16:16.729 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.023905+0000", 2024-04-07T16:16:16.729 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:16.729 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:16.729 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:16.729 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:16.729 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 60, 2024-04-07T16:16:16.730 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:16.730 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:16.730 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:16.730 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:16.730 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:16.730 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:16.730 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:16.730 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:16.730 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.30", 2024-04-07T16:16:16.730 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:16.730 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:16.730 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:16.730 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 19, 2024-04-07T16:16:16.730 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:16.730 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T03:04:58.804429+0000", 2024-04-07T16:16:16.730 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:16.731 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:16.731 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:16.731 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:16.731 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:16.731 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:16.731 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:16.731 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:16.731 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:16.731 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:16.731 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:16.731 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:16.731 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:16.731 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:16.731 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:16.731 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:16.731 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:16.732 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:16.732 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:16.732 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:16.732 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:16.732 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:16.732 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:16.732 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:16.732 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:16.732 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:16.732 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:16.732 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:16.732 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:16.732 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:16.732 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:16.732 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:16.732 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:16.732 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:16.733 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:16.733 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:16.733 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:16.733 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:16.733 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:16.733 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:16.733 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:16.733 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:16.733 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:16.733 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.733 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:16.733 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:16.733 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:16.733 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T16:16:16.733 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T16:16:16.733 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T16:16:16.733 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:16.734 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 5, 2024-04-07T16:16:16.734 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:16.734 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.734 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:16.734 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:16.734 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T16:16:16.734 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T16:16:16.734 INFO:tasks.workunit.client.0.smithi100.stdout: 4 2024-04-07T16:16:16.734 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:16.734 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 5, 2024-04-07T16:16:16.734 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:16.734 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:16.734 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T16:16:16.734 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:16.734 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:16.735 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:16.735 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.026420+0000", 2024-04-07T16:16:16.735 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.398198+0000", 2024-04-07T16:16:16.735 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.398198+0000", 2024-04-07T16:16:16.735 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.398543+0000", 2024-04-07T16:16:16.735 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.026420+0000", 2024-04-07T16:16:16.735 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:16.735 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:16.735 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:16.735 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 61, 2024-04-07T16:16:16.735 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.026420+0000", 2024-04-07T16:16:16.735 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.026420+0000", 2024-04-07T16:16:16.735 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.026420+0000", 2024-04-07T16:16:16.735 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:16.735 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:16.735 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:16.735 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.026420+0000", 2024-04-07T16:16:16.736 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.026420+0000", 2024-04-07T16:16:16.736 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:16.736 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:16.736 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:16.736 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:16.736 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 60, 2024-04-07T16:16:16.736 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:16.736 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:16.736 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:16.736 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:16.736 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:16.736 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:16.736 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:16.736 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:16.736 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.2f", 2024-04-07T16:16:16.737 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:16.737 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:16.737 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:16.737 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 19, 2024-04-07T16:16:16.737 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:16.737 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T22:57:24.895712+0000", 2024-04-07T16:16:16.737 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:16.737 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:16.737 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:16.737 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:16.737 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:16.737 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:16.737 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:16.737 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:16.737 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:16.738 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:16.738 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:16.738 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:16.738 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:16.738 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:16.738 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:16.738 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:16.738 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:16.738 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:16.738 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:16.738 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:16.738 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:16.738 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:16.738 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:16.738 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:16.739 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:16.739 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:16.739 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:16.739 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:16.739 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:16.739 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:16.739 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:16.739 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:16.739 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:16.739 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:16.739 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:16.739 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:16.739 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:16.739 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:16.739 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:16.739 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:16.739 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:16.740 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:16.740 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:16.740 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.740 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:16.740 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:16.740 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:16.740 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T16:16:16.740 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T16:16:16.740 INFO:tasks.workunit.client.0.smithi100.stdout: 4 2024-04-07T16:16:16.740 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:16.740 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 5, 2024-04-07T16:16:16.740 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:16.740 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.740 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:16.740 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:16.741 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T16:16:16.741 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T16:16:16.741 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T16:16:16.741 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:16.741 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 2, 2024-04-07T16:16:16.741 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:16.741 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:16.741 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T16:16:16.741 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:16.741 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:16.741 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:16.741 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.764814+0000", 2024-04-07T16:16:16.741 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.415972+0000", 2024-04-07T16:16:16.741 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.415972+0000", 2024-04-07T16:16:16.741 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.416210+0000", 2024-04-07T16:16:16.741 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.764814+0000", 2024-04-07T16:16:16.742 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:16.742 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:16.742 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:16.742 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 61, 2024-04-07T16:16:16.742 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.764814+0000", 2024-04-07T16:16:16.742 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.764814+0000", 2024-04-07T16:16:16.742 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.764814+0000", 2024-04-07T16:16:16.742 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:16.742 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:16.742 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:16.742 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.764814+0000", 2024-04-07T16:16:16.742 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.764814+0000", 2024-04-07T16:16:16.742 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:16.742 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:16.742 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:16.742 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:16.743 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 60, 2024-04-07T16:16:16.743 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:16.743 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:16.743 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:16.743 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:16.743 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:16.743 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:16.743 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:16.743 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:16.743 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.2e", 2024-04-07T16:16:16.743 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:16.743 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:16.743 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:16.743 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 19, 2024-04-07T16:16:16.743 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:16.743 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T23:40:11.606978+0000", 2024-04-07T16:16:16.743 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:16.744 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:16.744 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:16.744 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:16.744 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:16.744 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:16.744 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:16.744 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:16.744 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:16.744 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:16.744 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:16.744 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:16.744 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:16.744 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:16.744 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:16.744 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:16.744 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:16.744 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:16.745 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:16.745 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:16.745 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:16.745 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:16.745 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:16.745 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:16.745 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:16.745 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:16.745 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:16.745 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:16.745 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:16.745 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:16.745 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:16.745 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:16.745 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:16.745 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:16.746 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:16.746 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:16.746 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:16.746 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:16.746 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:16.746 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:16.746 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:16.746 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:16.746 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:16.746 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.746 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:16.746 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:16.746 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:16.746 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T16:16:16.746 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T16:16:16.746 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T16:16:16.747 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:16.747 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 2, 2024-04-07T16:16:16.747 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:16.747 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.747 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:16.747 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:16.747 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T16:16:16.747 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T16:16:16.747 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T16:16:16.747 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:16.747 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 7, 2024-04-07T16:16:16.747 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:16.747 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:16.747 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T16:16:16.747 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:16.747 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:16.747 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:16.747 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.636945+0000", 2024-04-07T16:16:16.748 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.413869+0000", 2024-04-07T16:16:16.748 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.413869+0000", 2024-04-07T16:16:16.748 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.414203+0000", 2024-04-07T16:16:16.748 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.636945+0000", 2024-04-07T16:16:16.748 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:16.748 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:16.748 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:16.748 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 61, 2024-04-07T16:16:16.748 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.636945+0000", 2024-04-07T16:16:16.748 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.636945+0000", 2024-04-07T16:16:16.748 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.636945+0000", 2024-04-07T16:16:16.748 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:16.748 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:16.748 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:16.749 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.636945+0000", 2024-04-07T16:16:16.749 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.636945+0000", 2024-04-07T16:16:16.749 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:16.749 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:16.749 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:16.749 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:16.749 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 60, 2024-04-07T16:16:16.749 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:16.749 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:16.749 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:16.749 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:16.749 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:16.749 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:16.749 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:16.749 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:16.750 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.2d", 2024-04-07T16:16:16.750 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:16.750 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:16.750 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:16.750 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 19, 2024-04-07T16:16:16.750 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:16.750 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T17:11:47.895568+0000", 2024-04-07T16:16:16.750 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:16.750 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:16.750 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:16.750 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:16.750 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:16.750 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:16.750 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:16.750 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:16.750 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:16.751 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:16.751 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:16.751 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:16.751 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:16.751 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:16.751 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:16.751 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:16.751 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:16.751 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:16.751 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:16.751 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:16.751 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:16.751 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:16.751 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:16.751 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:16.751 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:16.752 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:16.752 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:16.752 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:16.752 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:16.752 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:16.752 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:16.752 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:16.752 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:16.752 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:16.752 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:16.752 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:16.752 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:16.752 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:16.752 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:16.752 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:16.752 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:16.752 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:16.753 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:16.753 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.753 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:16.753 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:16.753 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:16.753 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T16:16:16.753 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T16:16:16.753 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T16:16:16.753 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:16.753 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 7, 2024-04-07T16:16:16.753 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:16.753 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.753 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:16.753 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:16.753 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T16:16:16.753 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T16:16:16.754 INFO:tasks.workunit.client.0.smithi100.stdout: 4 2024-04-07T16:16:16.754 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:16.754 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 0, 2024-04-07T16:16:16.754 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:16.754 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:16.754 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T16:16:16.754 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:16.754 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:16.754 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:16.754 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.031549+0000", 2024-04-07T16:16:16.754 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.032199+0000", 2024-04-07T16:16:16.754 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.032199+0000", 2024-04-07T16:16:16.754 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.032601+0000", 2024-04-07T16:16:16.754 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.031549+0000", 2024-04-07T16:16:16.754 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:16.755 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:16.755 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:16.755 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 60, 2024-04-07T16:16:16.755 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.031549+0000", 2024-04-07T16:16:16.755 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.031549+0000", 2024-04-07T16:16:16.755 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.031549+0000", 2024-04-07T16:16:16.755 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:16.755 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:16.755 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:16.755 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.031549+0000", 2024-04-07T16:16:16.755 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.031549+0000", 2024-04-07T16:16:16.755 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:16.755 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 4, 2024-04-07T16:16:16.755 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:16.755 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:16.756 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 59, 2024-04-07T16:16:16.756 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:16.756 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:16.756 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:16.756 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:16.756 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 4, 2024-04-07T16:16:16.756 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:16.756 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:16.756 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:16.756 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.1f", 2024-04-07T16:16:16.756 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:16.756 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:16.756 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:16.756 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 24, 2024-04-07T16:16:16.756 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:16.757 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T19:01:44.798719+0000", 2024-04-07T16:16:16.757 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:16.757 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:16.757 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:16.757 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 582, 2024-04-07T16:16:16.757 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:16.757 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:16.757 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:16.757 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:16.757 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:16.757 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:16.757 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:16.757 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:16.757 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:16.757 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:16.758 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:16.758 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:16.758 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:16.758 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:16.758 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:16.758 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 6, 2024-04-07T16:16:16.758 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 2, 2024-04-07T16:16:16.758 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:16.758 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 2, 2024-04-07T16:16:16.758 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:16.758 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:16.758 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:16.758 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:16.758 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:16.758 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 1, 2024-04-07T16:16:16.758 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:16.759 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:16.759 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:16.759 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:16.759 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:16.759 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:16.759 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:16.759 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:16.759 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:16.759 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:16.759 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:16.759 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:16.759 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 4, 2024-04-07T16:16:16.759 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 1 2024-04-07T16:16:16.759 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.759 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:16.760 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:16.760 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:16.760 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T16:16:16.760 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T16:16:16.760 INFO:tasks.workunit.client.0.smithi100.stdout: 4 2024-04-07T16:16:16.760 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:16.760 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 0, 2024-04-07T16:16:16.760 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "61'4" 2024-04-07T16:16:16.760 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.760 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:16.760 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:16.760 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T16:16:16.760 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T16:16:16.760 INFO:tasks.workunit.client.0.smithi100.stdout: 2 2024-04-07T16:16:16.760 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:16.761 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 3, 2024-04-07T16:16:16.761 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:16.761 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:16.761 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T16:16:16.761 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:16.761 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:16.761 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:16.761 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.187256+0000", 2024-04-07T16:16:16.761 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.407538+0000", 2024-04-07T16:16:16.761 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.407538+0000", 2024-04-07T16:16:16.761 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.407852+0000", 2024-04-07T16:16:16.761 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.187256+0000", 2024-04-07T16:16:16.761 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:16.761 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:16.761 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:16.762 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 61, 2024-04-07T16:16:16.762 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.187256+0000", 2024-04-07T16:16:16.762 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.187256+0000", 2024-04-07T16:16:16.762 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.187256+0000", 2024-04-07T16:16:16.762 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:16.762 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:16.762 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:16.762 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.187256+0000", 2024-04-07T16:16:16.762 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.187256+0000", 2024-04-07T16:16:16.762 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:16.762 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:16.762 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:16.762 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:16.762 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 60, 2024-04-07T16:16:16.762 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:16.762 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:16.763 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:16.763 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:16.763 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:16.763 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:16.763 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:16.763 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:16.763 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.1e", 2024-04-07T16:16:16.763 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:16.763 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:16.763 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:16.763 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 19, 2024-04-07T16:16:16.763 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:16.763 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T20:27:52.664149+0000", 2024-04-07T16:16:16.763 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:16.763 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:16.763 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:16.763 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:16.764 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:16.764 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:16.764 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:16.764 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:16.764 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:16.764 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:16.764 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:16.764 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:16.764 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:16.764 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:16.764 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:16.764 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:16.764 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:16.764 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:16.764 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:16.764 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:16.764 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:16.765 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:16.765 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:16.765 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:16.765 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:16.765 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:16.765 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:16.765 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:16.765 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:16.765 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:16.765 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:16.765 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:16.765 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:16.765 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:16.765 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:16.765 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:16.765 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:16.765 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:16.766 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:16.766 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:16.766 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:16.766 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:16.766 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:16.766 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.766 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:16.766 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:16.766 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:16.766 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T16:16:16.766 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T16:16:16.766 INFO:tasks.workunit.client.0.smithi100.stdout: 2 2024-04-07T16:16:16.766 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:16.766 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 3, 2024-04-07T16:16:16.766 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:16.766 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.767 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:16.767 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:16.767 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T16:16:16.767 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T16:16:16.767 INFO:tasks.workunit.client.0.smithi100.stdout: 5 2024-04-07T16:16:16.767 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:16.767 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 3, 2024-04-07T16:16:16.767 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:16.767 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:16.767 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T16:16:16.767 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:16.767 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:16.767 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:16.767 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.187484+0000", 2024-04-07T16:16:16.767 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.038492+0000", 2024-04-07T16:16:16.767 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.038492+0000", 2024-04-07T16:16:16.768 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.038807+0000", 2024-04-07T16:16:16.768 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.187484+0000", 2024-04-07T16:16:16.768 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:16.768 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:16.768 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:16.768 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 60, 2024-04-07T16:16:16.768 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.187484+0000", 2024-04-07T16:16:16.768 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.187484+0000", 2024-04-07T16:16:16.768 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.187484+0000", 2024-04-07T16:16:16.768 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:16.768 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:16.768 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:16.768 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.187484+0000", 2024-04-07T16:16:16.768 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.187484+0000", 2024-04-07T16:16:16.768 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:16.768 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 1, 2024-04-07T16:16:16.769 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:16.769 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:16.769 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 59, 2024-04-07T16:16:16.769 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:16.769 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:16.769 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:16.769 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:16.769 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 1, 2024-04-07T16:16:16.769 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:16.769 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:16.769 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:16.769 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.1e", 2024-04-07T16:16:16.769 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:16.769 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:16.769 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:16.769 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 21, 2024-04-07T16:16:16.769 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:16.770 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T02:11:03.746017+0000", 2024-04-07T16:16:16.770 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:16.770 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:16.770 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:16.770 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:16.770 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:16.770 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:16.770 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:16.770 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:16.770 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:16.770 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:16.770 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:16.770 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:16.770 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:16.770 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:16.770 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:16.771 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:16.771 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:16.771 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:16.771 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:16.771 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:16.771 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:16.771 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:16.771 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:16.771 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:16.771 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:16.771 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:16.771 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:16.771 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:16.771 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:16.771 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:16.771 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:16.771 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:16.772 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:16.772 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:16.772 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:16.772 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:16.772 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:16.772 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:16.772 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:16.772 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:16.772 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:16.772 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:16.772 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:16.772 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.772 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:16.772 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:16.772 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:16.772 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T16:16:16.773 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T16:16:16.773 INFO:tasks.workunit.client.0.smithi100.stdout: 5 2024-04-07T16:16:16.773 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:16.773 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 3, 2024-04-07T16:16:16.773 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "61'1" 2024-04-07T16:16:16.773 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.773 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:16.773 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:16.773 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T16:16:16.773 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T16:16:16.773 INFO:tasks.workunit.client.0.smithi100.stdout: 2 2024-04-07T16:16:16.773 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:16.773 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 0, 2024-04-07T16:16:16.773 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:16.773 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:16.773 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T16:16:16.774 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:16.774 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:16.774 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:16.774 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.031268+0000", 2024-04-07T16:16:16.774 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.409049+0000", 2024-04-07T16:16:16.774 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.409049+0000", 2024-04-07T16:16:16.774 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.410031+0000", 2024-04-07T16:16:16.774 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.031268+0000", 2024-04-07T16:16:16.774 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:16.774 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:16.774 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:16.774 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 61, 2024-04-07T16:16:16.774 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.031268+0000", 2024-04-07T16:16:16.774 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.031268+0000", 2024-04-07T16:16:16.774 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.031268+0000", 2024-04-07T16:16:16.774 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:16.775 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:16.775 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:16.775 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.031268+0000", 2024-04-07T16:16:16.775 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.031268+0000", 2024-04-07T16:16:16.775 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:16.775 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:16.775 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:16.775 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:16.775 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 60, 2024-04-07T16:16:16.775 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:16.775 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:16.775 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:16.775 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:16.775 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:16.775 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:16.775 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:16.776 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:16.776 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.1f", 2024-04-07T16:16:16.776 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:16.776 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:16.776 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:16.776 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 19, 2024-04-07T16:16:16.776 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:16.776 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T21:35:09.308017+0000", 2024-04-07T16:16:16.776 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:16.776 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:16.776 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:16.776 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:16.776 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:16.776 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:16.776 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:16.776 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:16.776 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:16.777 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:16.777 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:16.777 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:16.777 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:16.777 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:16.777 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:16.777 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:16.777 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:16.777 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:16.777 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:16.777 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:16.777 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:16.777 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:16.777 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:16.777 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:16.777 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:16.777 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:16.778 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:16.778 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:16.778 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:16.778 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:16.778 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:16.778 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:16.778 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:16.778 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:16.778 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:16.778 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:16.778 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:16.778 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:16.778 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:16.778 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:16.778 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:16.778 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:16.779 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:16.779 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.779 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:16.779 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:16.779 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:16.779 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T16:16:16.779 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T16:16:16.779 INFO:tasks.workunit.client.0.smithi100.stdout: 2 2024-04-07T16:16:16.779 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:16.779 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 0, 2024-04-07T16:16:16.779 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:16.779 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.779 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:16.779 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:16.779 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T16:16:16.779 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T16:16:16.780 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T16:16:16.780 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:16.780 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 7, 2024-04-07T16:16:16.780 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:16.780 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:16.780 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T16:16:16.780 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:16.780 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:16.780 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:16.780 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.637577+0000", 2024-04-07T16:16:16.780 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.041970+0000", 2024-04-07T16:16:16.780 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.041970+0000", 2024-04-07T16:16:16.780 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.042401+0000", 2024-04-07T16:16:16.780 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.637577+0000", 2024-04-07T16:16:16.780 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:16.780 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:16.780 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:16.781 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 60, 2024-04-07T16:16:16.781 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.637577+0000", 2024-04-07T16:16:16.781 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.637577+0000", 2024-04-07T16:16:16.781 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.637577+0000", 2024-04-07T16:16:16.781 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:16.781 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:16.781 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:16.781 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.637577+0000", 2024-04-07T16:16:16.781 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.637577+0000", 2024-04-07T16:16:16.781 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:16.781 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 2, 2024-04-07T16:16:16.781 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:16.781 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:16.781 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 59, 2024-04-07T16:16:16.781 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:16.781 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:16.781 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:16.782 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:16.782 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 2, 2024-04-07T16:16:16.782 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:16.782 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:16.782 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:16.782 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.1d", 2024-04-07T16:16:16.782 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:16.782 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:16.782 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:16.782 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 22, 2024-04-07T16:16:16.782 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:16.782 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T16:36:29.235347+0000", 2024-04-07T16:16:16.782 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:16.782 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:16.782 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:16.782 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:16.783 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:16.783 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:16.783 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:16.783 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:16.783 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:16.783 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:16.783 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:16.783 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:16.783 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:16.783 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:16.783 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:16.783 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:16.783 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:16.783 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:16.783 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:16.783 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 3, 2024-04-07T16:16:16.784 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 1, 2024-04-07T16:16:16.784 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:16.784 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 1, 2024-04-07T16:16:16.784 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:16.784 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:16.784 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:16.784 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:16.784 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:16.784 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 1, 2024-04-07T16:16:16.784 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:16.784 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:16.784 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:16.784 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:16.784 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:16.785 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:16.785 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:16.785 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:16.785 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:16.785 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:16.785 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:16.785 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:16.785 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 3, 2024-04-07T16:16:16.785 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:16.785 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.785 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:16.785 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:16.785 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:16.785 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T16:16:16.785 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T16:16:16.785 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T16:16:16.786 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:16.786 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 7, 2024-04-07T16:16:16.786 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "61'2" 2024-04-07T16:16:16.786 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.786 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:16.786 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:16.786 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T16:16:16.786 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T16:16:16.786 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T16:16:16.786 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:16.786 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 5, 2024-04-07T16:16:16.786 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:16.786 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:16.786 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T16:16:16.786 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:16.786 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:16.787 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:16.787 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.025650+0000", 2024-04-07T16:16:16.787 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.394271+0000", 2024-04-07T16:16:16.787 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.394271+0000", 2024-04-07T16:16:16.787 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.394718+0000", 2024-04-07T16:16:16.787 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.025650+0000", 2024-04-07T16:16:16.787 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:16.787 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:16.787 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:16.787 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 61, 2024-04-07T16:16:16.787 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.025650+0000", 2024-04-07T16:16:16.787 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.025650+0000", 2024-04-07T16:16:16.787 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.025650+0000", 2024-04-07T16:16:16.787 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:16.787 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:16.787 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:16.788 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.025650+0000", 2024-04-07T16:16:16.788 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.025650+0000", 2024-04-07T16:16:16.788 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:16.788 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:16.788 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:16.788 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:16.788 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 60, 2024-04-07T16:16:16.788 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:16.788 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:16.788 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:16.788 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:16.788 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:16.788 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:16.788 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:16.788 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:16.788 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.1c", 2024-04-07T16:16:16.789 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:16.789 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:16.789 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:16.789 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 19, 2024-04-07T16:16:16.789 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:16.789 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T20:10:29.096403+0000", 2024-04-07T16:16:16.789 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:16.789 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:16.789 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:16.789 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:16.789 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:16.789 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:16.789 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:16.789 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:16.789 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:16.790 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:16.790 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:16.790 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:16.790 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:16.790 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:16.790 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:16.790 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:16.790 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:16.790 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:16.790 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:16.790 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:16.790 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:16.790 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:16.790 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:16.790 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:16.791 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:16.791 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:16.791 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:16.791 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:16.791 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:16.791 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:16.791 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:16.791 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:16.791 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:16.791 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:16.791 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:16.791 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:16.791 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:16.791 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:16.791 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:16.791 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:16.792 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:16.792 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:16.792 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:16.792 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.792 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:16.792 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:16.792 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:16.792 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T16:16:16.792 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T16:16:16.792 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T16:16:16.792 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:16.792 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 5, 2024-04-07T16:16:16.792 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:16.792 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.792 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:16.793 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:16.793 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T16:16:16.793 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T16:16:16.793 INFO:tasks.workunit.client.0.smithi100.stdout: 2 2024-04-07T16:16:16.793 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:16.793 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 7, 2024-04-07T16:16:16.793 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:16.793 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:16.793 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T16:16:16.793 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:16.793 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:16.793 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:16.793 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.637065+0000", 2024-04-07T16:16:16.793 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.041919+0000", 2024-04-07T16:16:16.793 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.041919+0000", 2024-04-07T16:16:16.793 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.042391+0000", 2024-04-07T16:16:16.793 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.637065+0000", 2024-04-07T16:16:16.794 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:16.794 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:16.794 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:16.794 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 60, 2024-04-07T16:16:16.794 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.637065+0000", 2024-04-07T16:16:16.794 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.637065+0000", 2024-04-07T16:16:16.794 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.637065+0000", 2024-04-07T16:16:16.794 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:16.794 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:16.794 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:16.794 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.637065+0000", 2024-04-07T16:16:16.794 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.637065+0000", 2024-04-07T16:16:16.794 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:16.794 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 2, 2024-04-07T16:16:16.794 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:16.795 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:16.795 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 59, 2024-04-07T16:16:16.795 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:16.795 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:16.795 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:16.795 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:16.795 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 2, 2024-04-07T16:16:16.795 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:16.795 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:16.795 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:16.795 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.1c", 2024-04-07T16:16:16.795 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:16.795 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:16.795 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:16.795 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 22, 2024-04-07T16:16:16.795 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:16.796 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T21:50:59.576615+0000", 2024-04-07T16:16:16.796 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:16.796 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:16.796 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:16.796 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:16.796 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:16.796 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:16.796 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:16.796 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:16.796 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:16.796 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:16.796 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:16.796 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:16.797 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:16.797 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:16.797 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:16.797 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:16.797 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:16.797 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:16.797 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:16.797 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 3, 2024-04-07T16:16:16.797 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 1, 2024-04-07T16:16:16.797 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:16.797 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 1, 2024-04-07T16:16:16.797 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:16.797 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:16.797 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:16.798 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:16.798 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:16.798 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 1, 2024-04-07T16:16:16.798 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:16.798 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:16.798 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:16.798 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:16.798 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:16.798 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:16.798 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:16.798 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:16.798 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:16.798 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:16.798 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:16.799 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:16.799 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 3, 2024-04-07T16:16:16.799 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:16.799 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.799 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:16.799 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:16.799 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:16.799 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T16:16:16.799 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T16:16:16.799 INFO:tasks.workunit.client.0.smithi100.stdout: 2 2024-04-07T16:16:16.799 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:16.799 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 7, 2024-04-07T16:16:16.799 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "61'2" 2024-04-07T16:16:16.799 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.799 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:16.799 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:16.800 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T16:16:16.800 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T16:16:16.800 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T16:16:16.800 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:16.800 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 5, 2024-04-07T16:16:16.800 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:16.800 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:16.800 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T16:16:16.800 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:16.800 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:16.800 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:16.800 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.025658+0000", 2024-04-07T16:16:16.800 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:26.222249+0000", 2024-04-07T16:16:16.800 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:26.222249+0000", 2024-04-07T16:16:16.800 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:26.222601+0000", 2024-04-07T16:16:16.800 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.025658+0000", 2024-04-07T16:16:16.801 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:16.801 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:16.801 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:16.801 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 64, 2024-04-07T16:16:16.801 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.025658+0000", 2024-04-07T16:16:16.801 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.025658+0000", 2024-04-07T16:16:16.801 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.025658+0000", 2024-04-07T16:16:16.801 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:16.801 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:16.801 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:16.801 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.025658+0000", 2024-04-07T16:16:16.801 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.025658+0000", 2024-04-07T16:16:16.801 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:16.801 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:16.801 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:16.801 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:16.802 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 63, 2024-04-07T16:16:16.802 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:16.802 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:16.802 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:16.802 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:16.802 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:16.802 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:16.802 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:16.802 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:16.802 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.1d", 2024-04-07T16:16:16.802 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:16.802 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:16.802 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:16.802 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 28, 2024-04-07T16:16:16.802 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:16.803 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T16:28:47.005643+0000", 2024-04-07T16:16:16.803 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:16.803 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:16.803 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:16.803 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:16.803 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:16.803 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:16.803 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:16.803 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:16.803 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:16.803 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:16.803 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:16.803 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:16.803 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:16.803 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:16.803 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:16.803 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:16.804 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:16.804 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:16.804 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:16.804 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:16.804 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:16.804 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:16.804 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:16.804 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:16.804 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:16.804 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:16.804 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:16.804 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:16.804 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:16.804 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:16.804 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:16.804 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:16.804 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:16.805 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:16.805 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:16.805 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:16.805 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:16.805 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:16.805 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:16.805 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:16.805 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:16.805 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:16.805 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:16.805 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.805 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:16.805 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:16.805 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:16.805 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T16:16:16.805 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T16:16:16.805 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T16:16:16.806 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:16.806 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 5, 2024-04-07T16:16:16.806 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:16.806 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.806 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:16.806 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:16.806 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T16:16:16.806 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T16:16:16.806 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T16:16:16.806 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:16.806 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 3, 2024-04-07T16:16:16.806 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:16.806 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:16.806 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T16:16:16.806 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:16.807 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:16.807 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:16.807 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.187224+0000", 2024-04-07T16:16:16.807 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.041409+0000", 2024-04-07T16:16:16.807 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.041409+0000", 2024-04-07T16:16:16.807 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.041880+0000", 2024-04-07T16:16:16.807 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.187224+0000", 2024-04-07T16:16:16.807 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:16.807 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:16.807 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:16.807 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 60, 2024-04-07T16:16:16.807 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.187224+0000", 2024-04-07T16:16:16.807 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.187224+0000", 2024-04-07T16:16:16.807 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.187224+0000", 2024-04-07T16:16:16.807 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:16.808 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:16.808 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:16.808 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.187224+0000", 2024-04-07T16:16:16.808 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.187224+0000", 2024-04-07T16:16:16.808 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:16.808 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 1, 2024-04-07T16:16:16.808 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:16.808 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:16.808 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 59, 2024-04-07T16:16:16.808 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:16.808 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:16.808 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:16.808 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:16.808 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 1, 2024-04-07T16:16:16.808 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:16.809 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:16.809 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:16.809 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.1b", 2024-04-07T16:16:16.809 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:16.809 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:16.809 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:16.809 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 21, 2024-04-07T16:16:16.809 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:16.809 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T18:32:23.206985+0000", 2024-04-07T16:16:16.809 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:16.809 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:16.809 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:16.809 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:16.809 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:16.809 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:16.809 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:16.810 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:16.810 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:16.810 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:16.810 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:16.810 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:16.810 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:16.810 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:16.810 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:16.810 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:16.810 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:16.810 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:16.810 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:16.810 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:16.810 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:16.810 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:16.811 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:16.811 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:16.811 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:16.811 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:16.811 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:16.811 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:16.811 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:16.811 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:16.811 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:16.811 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:16.811 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:16.811 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:16.811 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:16.811 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:16.811 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:16.811 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:16.811 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:16.812 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:16.812 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:16.812 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:16.812 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:16.812 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.812 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:16.812 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:16.812 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:16.812 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T16:16:16.812 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T16:16:16.812 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T16:16:16.812 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:16.812 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 3, 2024-04-07T16:16:16.812 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "61'1" 2024-04-07T16:16:16.812 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.812 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:16.813 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:16.813 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T16:16:16.813 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T16:16:16.813 INFO:tasks.workunit.client.0.smithi100.stdout: 2 2024-04-07T16:16:16.813 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:16.813 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 4, 2024-04-07T16:16:16.813 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:16.813 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:16.813 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T16:16:16.813 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:16.813 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:16.813 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:16.813 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:39.824041+0000", 2024-04-07T16:16:16.813 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.413311+0000", 2024-04-07T16:16:16.813 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.413311+0000", 2024-04-07T16:16:16.814 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.413664+0000", 2024-04-07T16:16:16.814 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:39.824041+0000", 2024-04-07T16:16:16.814 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:16.814 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:16.814 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:16.814 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 61, 2024-04-07T16:16:16.814 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:39.824041+0000", 2024-04-07T16:16:16.814 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:39.824041+0000", 2024-04-07T16:16:16.814 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:39.824041+0000", 2024-04-07T16:16:16.814 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:16.814 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:16.814 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:16.814 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:39.824041+0000", 2024-04-07T16:16:16.814 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:39.824041+0000", 2024-04-07T16:16:16.814 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:16.814 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:16.815 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:16.815 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:16.815 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 60, 2024-04-07T16:16:16.815 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:16.815 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:16.815 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:16.815 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:16.815 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:16.815 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:16.815 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:16.815 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:16.815 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.1a", 2024-04-07T16:16:16.815 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:16.815 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:16.815 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:16.815 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 19, 2024-04-07T16:16:16.816 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:16.816 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T02:43:21.836307+0000", 2024-04-07T16:16:16.816 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:16.816 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:16.816 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:16.816 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:16.816 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:16.816 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:16.816 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:16.816 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:16.816 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:16.816 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:16.816 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:16.816 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:16.816 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:16.816 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:16.816 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:16.817 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:16.817 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:16.817 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:16.817 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:16.817 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:16.817 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:16.817 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:16.817 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:16.817 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:16.817 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:16.817 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:16.817 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:16.817 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:16.817 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:16.817 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:16.818 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:16.818 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:16.818 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:16.818 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:16.818 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:16.818 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:16.818 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:16.818 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:16.818 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:16.818 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:16.818 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:16.818 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:16.818 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:16.818 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.819 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:16.819 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:16.819 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:16.819 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T16:16:16.819 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T16:16:16.819 INFO:tasks.workunit.client.0.smithi100.stdout: 2 2024-04-07T16:16:16.819 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:16.819 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 4, 2024-04-07T16:16:16.819 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:16.819 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.819 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:16.819 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:16.819 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T16:16:16.819 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T16:16:16.819 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T16:16:16.819 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:16.820 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 6, 2024-04-07T16:16:16.820 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:16.820 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:16.820 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T16:16:16.820 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:16.820 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:16.820 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:16.820 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.526629+0000", 2024-04-07T16:16:16.820 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.038410+0000", 2024-04-07T16:16:16.820 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.038410+0000", 2024-04-07T16:16:16.820 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.038960+0000", 2024-04-07T16:16:16.820 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.526629+0000", 2024-04-07T16:16:16.820 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:16.820 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:16.820 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:16.821 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 60, 2024-04-07T16:16:16.821 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.526629+0000", 2024-04-07T16:16:16.821 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.526629+0000", 2024-04-07T16:16:16.821 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.526629+0000", 2024-04-07T16:16:16.821 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:16.821 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:16.821 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:16.821 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.526629+0000", 2024-04-07T16:16:16.821 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.526629+0000", 2024-04-07T16:16:16.821 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:16.821 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 1, 2024-04-07T16:16:16.821 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:16.821 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:16.821 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 59, 2024-04-07T16:16:16.821 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:16.821 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:16.822 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:16.822 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:16.822 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 1, 2024-04-07T16:16:16.822 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:16.822 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:16.822 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:16.822 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.1a", 2024-04-07T16:16:16.822 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:16.822 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:16.822 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:16.822 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 21, 2024-04-07T16:16:16.822 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:16.822 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T03:41:38.418312+0000", 2024-04-07T16:16:16.822 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:16.822 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:16.823 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:16.823 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:16.823 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:16.823 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:16.823 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:16.823 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:16.823 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:16.823 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:16.823 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:16.823 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:16.823 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:16.823 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:16.823 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:16.823 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:16.823 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:16.823 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:16.824 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:16.824 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:16.824 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:16.824 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:16.824 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:16.824 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:16.824 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:16.824 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:16.824 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:16.824 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:16.824 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:16.824 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:16.824 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:16.824 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:16.824 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:16.825 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:16.825 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:16.825 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:16.825 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:16.825 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:16.825 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:16.825 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:16.825 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:16.825 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:16.825 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:16.825 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.825 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:16.825 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:16.825 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:16.825 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T16:16:16.825 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T16:16:16.826 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T16:16:16.826 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:16.826 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 6, 2024-04-07T16:16:16.826 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "61'1" 2024-04-07T16:16:16.826 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.826 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:16.826 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:16.826 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T16:16:16.826 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T16:16:16.826 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T16:16:16.826 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:16.826 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 0, 2024-04-07T16:16:16.826 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:16.826 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:16.826 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T16:16:16.827 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:16.827 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:16.827 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:16.827 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.029545+0000", 2024-04-07T16:16:16.827 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.405069+0000", 2024-04-07T16:16:16.827 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.405069+0000", 2024-04-07T16:16:16.827 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.405634+0000", 2024-04-07T16:16:16.827 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.029545+0000", 2024-04-07T16:16:16.827 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:16.827 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:16.827 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:16.827 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 61, 2024-04-07T16:16:16.827 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.029545+0000", 2024-04-07T16:16:16.827 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.029545+0000", 2024-04-07T16:16:16.828 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.029545+0000", 2024-04-07T16:16:16.828 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:16.828 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:16.828 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:16.828 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.029545+0000", 2024-04-07T16:16:16.828 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.029545+0000", 2024-04-07T16:16:16.828 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:16.828 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:16.828 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:16.828 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:16.828 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 60, 2024-04-07T16:16:16.828 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:16.828 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:16.828 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:16.828 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:16.829 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:16.829 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:16.829 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:16.829 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:16.829 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.1b", 2024-04-07T16:16:16.829 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:16.829 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:16.829 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:16.829 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 19, 2024-04-07T16:16:16.829 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:16.829 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T23:50:18.973629+0000", 2024-04-07T16:16:16.829 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:16.829 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:16.829 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:16.829 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:16.830 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:16.830 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:16.830 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:16.830 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:16.830 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:16.830 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:16.830 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:16.830 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:16.830 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:16.830 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:16.830 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:16.830 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:16.830 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:16.830 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:16.831 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:16.831 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:16.831 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:16.831 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:16.831 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:16.831 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:16.831 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:16.831 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:16.831 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:16.831 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:16.831 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:16.831 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:16.831 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:16.831 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:16.831 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:16.832 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:16.832 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:16.832 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:16.832 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:16.832 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:16.832 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:16.832 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:16.832 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:16.832 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:16.832 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:16.832 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.832 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:16.832 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:16.832 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:16.832 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T16:16:16.832 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T16:16:16.833 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T16:16:16.833 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:16.833 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 0, 2024-04-07T16:16:16.833 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:16.833 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.833 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:16.833 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:16.833 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T16:16:16.833 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T16:16:16.833 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T16:16:16.833 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:16.833 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 3, 2024-04-07T16:16:16.833 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:16.833 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:16.834 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T16:16:16.834 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:16.834 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:16.834 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:16.834 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.186860+0000", 2024-04-07T16:16:16.834 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.034013+0000", 2024-04-07T16:16:16.834 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.034013+0000", 2024-04-07T16:16:16.834 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.034420+0000", 2024-04-07T16:16:16.834 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.186860+0000", 2024-04-07T16:16:16.834 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:16.834 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:16.834 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:16.834 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 60, 2024-04-07T16:16:16.834 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.186860+0000", 2024-04-07T16:16:16.835 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.186860+0000", 2024-04-07T16:16:16.835 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.186860+0000", 2024-04-07T16:16:16.835 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:16.835 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:16.835 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:16.835 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.186860+0000", 2024-04-07T16:16:16.835 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.186860+0000", 2024-04-07T16:16:16.835 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:16.835 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:16.835 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:16.835 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:16.835 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 59, 2024-04-07T16:16:16.835 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:16.835 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:16.836 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:16.836 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:16.836 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:16.836 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:16.836 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:16.836 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:16.836 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.19", 2024-04-07T16:16:16.836 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:16.836 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:16.836 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:16.836 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 20, 2024-04-07T16:16:16.836 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:16.836 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T19:51:21.525865+0000", 2024-04-07T16:16:16.836 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:16.836 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:16.836 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:16.836 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:16.837 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:16.837 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:16.837 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:16.837 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:16.837 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:16.837 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:16.837 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:16.837 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:16.837 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:16.837 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:16.837 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:16.837 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:16.837 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:16.837 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:16.838 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:16.838 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:16.838 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:16.838 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:16.838 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:16.838 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:16.838 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:16.838 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:16.838 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:16.838 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:16.838 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:16.838 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:16.838 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:16.838 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:16.838 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:16.839 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:16.839 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:16.839 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:16.839 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:16.839 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:16.839 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:16.839 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:16.839 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:16.839 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:16.839 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:16.839 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.839 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:16.839 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:16.839 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:16.839 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T16:16:16.840 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T16:16:16.840 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T16:16:16.840 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:16.840 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 3, 2024-04-07T16:16:16.840 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:16.840 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.840 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:16.840 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:16.840 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T16:16:16.840 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T16:16:16.840 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T16:16:16.840 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:16.840 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 3, 2024-04-07T16:16:16.840 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:16.840 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:16.840 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T16:16:16.841 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:16.841 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:16.841 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:16.841 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.187269+0000", 2024-04-07T16:16:16.841 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.398017+0000", 2024-04-07T16:16:16.841 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.398017+0000", 2024-04-07T16:16:16.841 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.398531+0000", 2024-04-07T16:16:16.841 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.187269+0000", 2024-04-07T16:16:16.841 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:16.841 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:16.841 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:16.841 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 61, 2024-04-07T16:16:16.841 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.187269+0000", 2024-04-07T16:16:16.841 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.187269+0000", 2024-04-07T16:16:16.842 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.187269+0000", 2024-04-07T16:16:16.842 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:16.842 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:16.842 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:16.842 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.187269+0000", 2024-04-07T16:16:16.842 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.187269+0000", 2024-04-07T16:16:16.842 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:16.842 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:16.842 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:16.842 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:16.842 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 60, 2024-04-07T16:16:16.842 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:16.842 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:16.842 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:16.842 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:16.843 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:16.843 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:16.843 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:16.843 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:16.843 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.18", 2024-04-07T16:16:16.843 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:16.843 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:16.843 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:16.843 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 19, 2024-04-07T16:16:16.843 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:16.843 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T17:33:54.306902+0000", 2024-04-07T16:16:16.843 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:16.843 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:16.843 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:16.843 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:16.843 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:16.843 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:16.844 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:16.844 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:16.844 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:16.844 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:16.844 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:16.844 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:16.844 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:16.844 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:16.844 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:16.844 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:16.844 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:16.844 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:16.844 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:16.844 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:16.844 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:16.844 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:16.844 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:16.845 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:16.845 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:16.845 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:16.845 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:16.845 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:16.845 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:16.845 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:16.845 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:16.845 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:16.845 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:16.845 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:16.845 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:16.845 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:16.845 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:16.845 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:16.846 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:16.846 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:16.846 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:16.846 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:16.846 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:16.846 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.846 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:16.846 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:16.846 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:16.846 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T16:16:16.846 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T16:16:16.846 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T16:16:16.846 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:16.846 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 3, 2024-04-07T16:16:16.847 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:16.847 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.847 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:16.847 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:16.847 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T16:16:16.847 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T16:16:16.847 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T16:16:16.847 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:16.847 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 5, 2024-04-07T16:16:16.847 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:16.847 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:16.847 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T16:16:16.847 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:16.847 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:16.847 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:16.847 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.024099+0000", 2024-04-07T16:16:16.848 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.044808+0000", 2024-04-07T16:16:16.848 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.044808+0000", 2024-04-07T16:16:16.848 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.045443+0000", 2024-04-07T16:16:16.848 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.024099+0000", 2024-04-07T16:16:16.848 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:16.848 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:16.848 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:16.848 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 60, 2024-04-07T16:16:16.848 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.024099+0000", 2024-04-07T16:16:16.848 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.024099+0000", 2024-04-07T16:16:16.848 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.024099+0000", 2024-04-07T16:16:16.848 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:16.848 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:16.848 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:16.848 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.024099+0000", 2024-04-07T16:16:16.849 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.024099+0000", 2024-04-07T16:16:16.849 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:16.849 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:16.849 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:16.849 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:16.849 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 59, 2024-04-07T16:16:16.849 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:16.849 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:16.849 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:16.849 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:16.849 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:16.849 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:16.849 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:16.849 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:16.849 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.18", 2024-04-07T16:16:16.850 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:16.850 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:16.850 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:16.850 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 20, 2024-04-07T16:16:16.850 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:16.850 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T22:44:14.623116+0000", 2024-04-07T16:16:16.850 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:16.850 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:16.850 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:16.850 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:16.850 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:16.850 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:16.850 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:16.850 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:16.851 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:16.851 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:16.851 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:16.851 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:16.851 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:16.851 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:16.851 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:16.851 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:16.851 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:16.851 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:16.851 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:16.851 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:16.851 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:16.851 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:16.851 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:16.852 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:16.852 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:16.852 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:16.852 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:16.852 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:16.852 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:16.852 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:16.852 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:16.852 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:16.852 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:16.852 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:16.852 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:16.852 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:16.852 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:16.852 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:16.852 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:16.853 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:16.853 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:16.853 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:16.853 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:16.853 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.853 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:16.853 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:16.853 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:16.853 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T16:16:16.853 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T16:16:16.853 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T16:16:16.853 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:16.853 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 5, 2024-04-07T16:16:16.853 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:16.853 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.853 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:16.854 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:16.854 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T16:16:16.854 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T16:16:16.854 INFO:tasks.workunit.client.0.smithi100.stdout: 4 2024-04-07T16:16:16.854 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:16.854 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 1, 2024-04-07T16:16:16.854 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:16.854 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:16.854 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T16:16:16.854 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:16.854 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:16.854 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:16.854 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.236151+0000", 2024-04-07T16:16:16.854 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.390643+0000", 2024-04-07T16:16:16.854 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.390643+0000", 2024-04-07T16:16:16.855 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.391166+0000", 2024-04-07T16:16:16.855 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.236151+0000", 2024-04-07T16:16:16.855 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:16.855 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:16.855 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:16.855 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 61, 2024-04-07T16:16:16.855 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.236151+0000", 2024-04-07T16:16:16.855 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.236151+0000", 2024-04-07T16:16:16.855 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.236151+0000", 2024-04-07T16:16:16.855 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:16.855 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:16.855 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:16.855 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.236151+0000", 2024-04-07T16:16:16.855 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.236151+0000", 2024-04-07T16:16:16.855 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:16.855 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:16.855 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:16.856 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:16.856 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 60, 2024-04-07T16:16:16.856 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:16.856 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:16.856 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:16.856 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:16.856 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:16.856 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:16.856 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:16.856 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:16.856 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.19", 2024-04-07T16:16:16.856 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:16.856 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:16.856 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:16.856 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 19, 2024-04-07T16:16:16.857 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:16.857 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T21:22:33.110408+0000", 2024-04-07T16:16:16.857 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:16.857 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:16.857 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:16.857 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:16.857 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:16.857 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:16.857 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:16.857 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:16.857 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:16.857 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:16.857 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:16.857 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:16.857 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:16.858 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:16.858 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:16.858 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:16.858 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:16.858 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:16.858 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:16.858 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:16.858 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:16.858 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:16.858 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:16.858 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:16.858 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:16.858 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:16.859 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:16.859 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:16.859 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:16.859 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:16.859 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:16.859 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:16.859 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:16.859 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:16.859 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:16.859 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:16.859 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:16.859 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:16.859 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:16.859 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:16.859 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:16.860 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:16.860 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:16.860 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.860 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:16.860 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:16.860 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:16.860 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T16:16:16.860 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T16:16:16.860 INFO:tasks.workunit.client.0.smithi100.stdout: 4 2024-04-07T16:16:16.860 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:16.860 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 1, 2024-04-07T16:16:16.860 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:16.860 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.860 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:16.860 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:16.861 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T16:16:16.861 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T16:16:16.861 INFO:tasks.workunit.client.0.smithi100.stdout: 2 2024-04-07T16:16:16.861 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:16.861 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 6, 2024-04-07T16:16:16.861 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:16.861 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:16.861 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T16:16:16.861 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:16.861 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:16.861 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:16.861 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.553074+0000", 2024-04-07T16:16:16.861 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.029525+0000", 2024-04-07T16:16:16.861 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.029525+0000", 2024-04-07T16:16:16.861 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.030149+0000", 2024-04-07T16:16:16.862 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.553074+0000", 2024-04-07T16:16:16.862 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:16.862 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:16.862 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:16.862 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 60, 2024-04-07T16:16:16.862 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.553074+0000", 2024-04-07T16:16:16.862 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.553074+0000", 2024-04-07T16:16:16.862 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.553074+0000", 2024-04-07T16:16:16.862 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:16.862 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:16.862 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:16.862 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.553074+0000", 2024-04-07T16:16:16.862 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.553074+0000", 2024-04-07T16:16:16.862 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:16.862 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:16.863 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:16.863 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:16.863 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 59, 2024-04-07T16:16:16.863 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:16.863 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:16.863 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:16.863 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:16.863 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:16.863 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:16.863 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:16.863 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:16.863 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.17", 2024-04-07T16:16:16.863 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:16.863 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:16.863 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:16.864 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 20, 2024-04-07T16:16:16.864 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:16.864 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T20:09:35.180264+0000", 2024-04-07T16:16:16.864 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:16.864 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:16.864 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:16.864 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:16.864 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:16.864 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:16.864 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:16.864 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:16.864 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:16.864 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:16.864 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:16.864 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:16.865 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:16.865 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:16.865 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:16.865 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:16.865 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:16.865 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:16.865 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:16.865 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:16.865 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:16.865 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:16.865 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:16.865 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:16.865 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:16.865 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:16.865 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:16.865 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:16.865 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:16.866 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:16.866 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:16.866 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:16.866 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:16.866 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:16.866 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:16.866 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:16.866 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:16.866 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:16.866 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:16.866 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:16.866 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:16.866 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:16.866 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:16.866 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.867 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:16.867 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:16.867 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:16.867 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T16:16:16.867 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T16:16:16.867 INFO:tasks.workunit.client.0.smithi100.stdout: 2 2024-04-07T16:16:16.867 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:16.867 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 6, 2024-04-07T16:16:16.867 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:16.867 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.867 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:16.867 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:16.867 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T16:16:16.867 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T16:16:16.867 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T16:16:16.867 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:16.868 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 5, 2024-04-07T16:16:16.868 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:16.868 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:16.868 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T16:16:16.868 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:16.868 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:16.868 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:16.868 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.025897+0000", 2024-04-07T16:16:16.868 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.398065+0000", 2024-04-07T16:16:16.868 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.398065+0000", 2024-04-07T16:16:16.868 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.398411+0000", 2024-04-07T16:16:16.868 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.025897+0000", 2024-04-07T16:16:16.868 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:16.868 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:16.868 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:16.869 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 61, 2024-04-07T16:16:16.869 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.025897+0000", 2024-04-07T16:16:16.869 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.025897+0000", 2024-04-07T16:16:16.869 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.025897+0000", 2024-04-07T16:16:16.869 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:16.869 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:16.869 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:16.869 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.025897+0000", 2024-04-07T16:16:16.869 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.025897+0000", 2024-04-07T16:16:16.869 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:16.869 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:16.869 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:16.869 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:16.869 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 60, 2024-04-07T16:16:16.869 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:16.869 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:16.870 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:16.870 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:16.870 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:16.870 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:16.870 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:16.870 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:16.870 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.16", 2024-04-07T16:16:16.870 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:16.870 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:16.870 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:16.870 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 19, 2024-04-07T16:16:16.870 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:16.870 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T21:13:20.448127+0000", 2024-04-07T16:16:16.870 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:16.871 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:16.871 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:16.871 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:16.871 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:16.871 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:16.871 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:16.871 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:16.871 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:16.871 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:16.871 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:16.871 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:16.871 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:16.871 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:16.871 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:16.871 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:16.872 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:16.872 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:16.872 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:16.872 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:16.872 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:16.872 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:16.872 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:16.872 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:16.872 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:16.872 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:16.872 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:16.872 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:16.872 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:16.872 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:16.872 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:16.873 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:16.873 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:16.873 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:16.873 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:16.873 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:16.873 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:16.873 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:16.873 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:16.873 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:16.873 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:16.873 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:16.873 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:16.873 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.873 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:16.874 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:16.874 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:16.874 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T16:16:16.874 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T16:16:16.874 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T16:16:16.874 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:16.874 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 5, 2024-04-07T16:16:16.874 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:16.874 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.874 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:16.874 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:16.874 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T16:16:16.874 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T16:16:16.874 INFO:tasks.workunit.client.0.smithi100.stdout: 2 2024-04-07T16:16:16.874 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:16.875 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 5, 2024-04-07T16:16:16.875 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:16.875 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:16.875 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T16:16:16.875 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:16.875 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:16.875 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:16.875 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.025656+0000", 2024-04-07T16:16:16.875 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.036322+0000", 2024-04-07T16:16:16.875 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.036322+0000", 2024-04-07T16:16:16.875 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.036777+0000", 2024-04-07T16:16:16.875 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.025656+0000", 2024-04-07T16:16:16.875 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:16.875 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:16.875 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:16.876 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 60, 2024-04-07T16:16:16.876 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.025656+0000", 2024-04-07T16:16:16.876 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.025656+0000", 2024-04-07T16:16:16.876 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.025656+0000", 2024-04-07T16:16:16.876 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:16.876 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:16.876 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:16.876 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.025656+0000", 2024-04-07T16:16:16.876 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.025656+0000", 2024-04-07T16:16:16.876 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:16.876 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:16.876 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:16.876 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:16.876 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 59, 2024-04-07T16:16:16.876 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:16.877 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:16.877 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:16.877 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:16.877 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:16.877 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:16.877 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:16.877 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:16.877 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.16", 2024-04-07T16:16:16.877 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:16.877 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:16.877 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:16.877 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 20, 2024-04-07T16:16:16.877 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:16.877 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T18:26:16.873044+0000", 2024-04-07T16:16:16.877 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:16.877 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:16.878 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:16.878 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:16.878 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:16.878 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:16.878 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:16.878 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:16.878 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:16.878 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:16.878 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:16.878 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:16.878 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:16.878 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:16.878 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:16.878 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:16.879 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:16.879 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:16.879 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:16.879 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:16.879 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:16.879 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:16.879 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:16.879 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:16.879 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:16.879 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:16.879 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:16.879 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:16.879 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:16.879 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:16.879 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:16.880 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:16.880 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:16.880 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:16.880 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:16.880 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:16.880 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:16.880 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:16.880 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:16.880 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:16.880 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:16.880 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:16.880 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:16.880 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.880 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:16.880 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:16.880 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:16.880 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T16:16:16.881 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T16:16:16.881 INFO:tasks.workunit.client.0.smithi100.stdout: 2 2024-04-07T16:16:16.881 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:16.881 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 5, 2024-04-07T16:16:16.881 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:16.881 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.881 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:16.881 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:16.881 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T16:16:16.881 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T16:16:16.881 INFO:tasks.workunit.client.0.smithi100.stdout: 3 2024-04-07T16:16:16.881 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:16.881 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 2, 2024-04-07T16:16:16.881 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:16.881 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:16.881 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T16:16:16.882 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:16.882 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:16.882 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:16.882 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.763837+0000", 2024-04-07T16:16:16.882 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:26.225343+0000", 2024-04-07T16:16:16.882 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:26.225343+0000", 2024-04-07T16:16:16.882 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:26.226313+0000", 2024-04-07T16:16:16.882 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.763837+0000", 2024-04-07T16:16:16.882 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:16.882 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:16.882 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:16.882 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 64, 2024-04-07T16:16:16.882 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.763837+0000", 2024-04-07T16:16:16.882 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.763837+0000", 2024-04-07T16:16:16.882 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.763837+0000", 2024-04-07T16:16:16.883 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:16.883 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:16.883 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:16.883 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.763837+0000", 2024-04-07T16:16:16.883 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.763837+0000", 2024-04-07T16:16:16.883 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:16.883 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:16.883 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:16.883 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:16.883 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 63, 2024-04-07T16:16:16.883 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:16.883 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:16.883 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:16.883 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:16.883 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:16.884 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:16.884 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:16.884 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:16.884 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.17", 2024-04-07T16:16:16.884 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:16.884 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:16.884 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:16.884 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 17, 2024-04-07T16:16:16.884 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:16.884 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T17:42:38.440812+0000", 2024-04-07T16:16:16.884 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:16.884 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:16.884 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:16.884 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:16.884 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:16.885 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:16.885 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:16.885 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:16.885 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:16.885 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:16.885 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:16.885 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:16.885 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:16.885 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:16.885 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:16.885 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:16.885 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:16.885 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:16.885 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:16.885 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:16.886 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:16.886 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:16.886 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:16.886 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:16.886 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:16.886 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:16.886 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:16.886 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:16.886 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:16.886 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:16.886 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:16.886 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:16.886 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:16.886 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:16.887 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:16.887 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:16.887 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:16.887 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:16.887 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:16.887 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:16.887 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:16.887 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:16.887 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:16.887 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.887 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:16.887 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:16.887 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:16.887 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T16:16:16.887 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T16:16:16.887 INFO:tasks.workunit.client.0.smithi100.stdout: 3 2024-04-07T16:16:16.887 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:16.888 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 2, 2024-04-07T16:16:16.888 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:16.888 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.888 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:16.888 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:16.888 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T16:16:16.888 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T16:16:16.888 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T16:16:16.888 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:16.888 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 1, 2024-04-07T16:16:16.888 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:16.888 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:16.888 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T16:16:16.888 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:16.889 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:16.889 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:16.889 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.237334+0000", 2024-04-07T16:16:16.889 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.038911+0000", 2024-04-07T16:16:16.889 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.038911+0000", 2024-04-07T16:16:16.889 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.039273+0000", 2024-04-07T16:16:16.889 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.237334+0000", 2024-04-07T16:16:16.889 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:16.889 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:16.889 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:16.889 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 60, 2024-04-07T16:16:16.889 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.237334+0000", 2024-04-07T16:16:16.889 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.237334+0000", 2024-04-07T16:16:16.889 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.237334+0000", 2024-04-07T16:16:16.889 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:16.889 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:16.889 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:16.890 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.237334+0000", 2024-04-07T16:16:16.890 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.237334+0000", 2024-04-07T16:16:16.890 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:16.890 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 3, 2024-04-07T16:16:16.890 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:16.890 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:16.890 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 59, 2024-04-07T16:16:16.890 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:16.890 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:16.890 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:16.890 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:16.890 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 3, 2024-04-07T16:16:16.890 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:16.890 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:16.890 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:16.891 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.15", 2024-04-07T16:16:16.891 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:16.891 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:16.891 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:16.891 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 23, 2024-04-07T16:16:16.891 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:16.891 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T02:11:53.262707+0000", 2024-04-07T16:16:16.891 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:16.891 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:16.891 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:16.891 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 90, 2024-04-07T16:16:16.891 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:16.891 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:16.891 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:16.891 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:16.891 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:16.891 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:16.892 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:16.892 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:16.892 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:16.892 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:16.892 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:16.892 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:16.892 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:16.892 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:16.892 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:16.892 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 6, 2024-04-07T16:16:16.892 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 2, 2024-04-07T16:16:16.892 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:16.892 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 2, 2024-04-07T16:16:16.892 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:16.892 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:16.893 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:16.893 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:16.893 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:16.893 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 1, 2024-04-07T16:16:16.893 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:16.893 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:16.893 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:16.893 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:16.893 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:16.893 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:16.893 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:16.893 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:16.893 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:16.893 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:16.893 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:16.893 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:16.893 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 4, 2024-04-07T16:16:16.894 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 1 2024-04-07T16:16:16.894 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.894 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:16.894 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:16.894 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:16.894 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T16:16:16.894 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T16:16:16.894 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T16:16:16.894 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:16.894 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 1, 2024-04-07T16:16:16.894 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "61'3" 2024-04-07T16:16:16.894 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.894 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:16.894 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:16.894 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T16:16:16.894 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T16:16:16.895 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T16:16:16.895 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:16.895 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 4, 2024-04-07T16:16:16.895 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:16.895 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:16.895 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T16:16:16.895 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:16.895 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:16.895 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:16.895 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:39.824449+0000", 2024-04-07T16:16:16.895 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.390960+0000", 2024-04-07T16:16:16.895 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.390960+0000", 2024-04-07T16:16:16.895 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.391349+0000", 2024-04-07T16:16:16.895 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:39.824449+0000", 2024-04-07T16:16:16.895 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:16.896 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:16.896 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:16.896 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 61, 2024-04-07T16:16:16.896 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:39.824449+0000", 2024-04-07T16:16:16.896 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:39.824449+0000", 2024-04-07T16:16:16.896 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:39.824449+0000", 2024-04-07T16:16:16.896 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:16.896 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:16.896 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:16.896 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:39.824449+0000", 2024-04-07T16:16:16.896 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:39.824449+0000", 2024-04-07T16:16:16.896 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:16.896 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:16.896 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:16.896 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:16.897 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 60, 2024-04-07T16:16:16.897 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:16.897 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:16.897 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:16.897 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:16.897 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:16.897 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:16.897 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:16.897 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:16.897 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.14", 2024-04-07T16:16:16.897 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:16.897 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:16.897 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:16.897 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 19, 2024-04-07T16:16:16.898 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:16.898 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T16:55:37.429799+0000", 2024-04-07T16:16:16.898 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:16.898 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:16.898 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:16.898 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:16.898 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:16.898 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:16.898 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:16.898 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:16.898 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:16.898 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:16.898 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:16.898 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:16.898 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:16.899 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:16.899 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:16.899 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:16.899 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:16.899 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:16.899 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:16.899 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:16.899 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:16.899 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:16.899 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:16.899 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:16.899 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:16.899 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:16.899 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:16.899 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:16.899 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:16.900 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:16.900 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:16.900 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:16.900 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:16.900 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:16.900 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:16.900 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:16.900 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:16.900 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:16.900 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:16.900 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:16.900 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:16.900 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:16.900 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:16.900 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.900 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:16.901 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:16.901 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:16.901 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T16:16:16.901 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T16:16:16.901 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T16:16:16.901 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:16.901 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 4, 2024-04-07T16:16:16.901 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:16.901 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.901 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:16.901 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:16.901 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T16:16:16.901 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T16:16:16.901 INFO:tasks.workunit.client.0.smithi100.stdout: 5 2024-04-07T16:16:16.901 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:16.902 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 6, 2024-04-07T16:16:16.902 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:16.902 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:16.902 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T16:16:16.902 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:16.902 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:16.902 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:16.902 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:53.034681+0000", 2024-04-07T16:16:16.902 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.031092+0000", 2024-04-07T16:16:16.902 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.031092+0000", 2024-04-07T16:16:16.902 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.031673+0000", 2024-04-07T16:16:16.902 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:53.034681+0000", 2024-04-07T16:16:16.902 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:16.902 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:16.903 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:16.903 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 60, 2024-04-07T16:16:16.903 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:53.034681+0000", 2024-04-07T16:16:16.903 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:53.034681+0000", 2024-04-07T16:16:16.903 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:53.034681+0000", 2024-04-07T16:16:16.903 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:16.903 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:16.903 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:16.903 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:53.034681+0000", 2024-04-07T16:16:16.903 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:53.034681+0000", 2024-04-07T16:16:16.903 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:16.903 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 12, 2024-04-07T16:16:16.903 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:16.903 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:16.903 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 59, 2024-04-07T16:16:16.904 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:16.904 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:16.904 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:16.904 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:16.904 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 12, 2024-04-07T16:16:16.904 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:16.904 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:16.904 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:16.904 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.14", 2024-04-07T16:16:16.904 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:16.904 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:16.904 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:16.904 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 32, 2024-04-07T16:16:16.904 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:16.904 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T20:17:48.593862+0000", 2024-04-07T16:16:16.904 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:16.905 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:16.905 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:16.905 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 2632, 2024-04-07T16:16:16.905 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:16.905 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:16.905 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:16.905 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:16.905 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:16.905 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:16.905 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:16.905 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:16.905 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:16.905 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:16.905 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:16.905 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:16.905 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:16.906 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:16.906 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:16.906 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 12, 2024-04-07T16:16:16.906 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 4, 2024-04-07T16:16:16.906 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:16.906 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 4, 2024-04-07T16:16:16.906 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:16.906 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:16.906 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:16.906 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:16.906 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:16.906 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 1, 2024-04-07T16:16:16.906 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:16.906 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:16.906 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:16.907 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:16.907 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:16.907 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:16.907 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:16.907 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:16.907 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:16.907 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:16.907 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:16.907 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:16.907 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 12, 2024-04-07T16:16:16.907 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 9 2024-04-07T16:16:16.907 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.907 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:16.907 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:16.907 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:16.908 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T16:16:16.908 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T16:16:16.908 INFO:tasks.workunit.client.0.smithi100.stdout: 5 2024-04-07T16:16:16.908 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:16.908 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 6, 2024-04-07T16:16:16.908 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "69'12" 2024-04-07T16:16:16.908 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.908 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:16.908 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:16.908 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T16:16:16.908 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T16:16:16.908 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T16:16:16.908 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:16.908 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 7, 2024-04-07T16:16:16.909 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:16.909 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:16.909 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T16:16:16.909 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:16.909 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:16.909 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:16.909 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.635967+0000", 2024-04-07T16:16:16.909 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:26.222663+0000", 2024-04-07T16:16:16.909 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:26.222663+0000", 2024-04-07T16:16:16.909 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:26.223406+0000", 2024-04-07T16:16:16.909 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.635967+0000", 2024-04-07T16:16:16.909 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:16.909 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:16.909 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:16.909 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 64, 2024-04-07T16:16:16.910 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.635967+0000", 2024-04-07T16:16:16.910 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.635967+0000", 2024-04-07T16:16:16.910 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.635967+0000", 2024-04-07T16:16:16.910 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:16.910 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:16.910 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:16.910 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.635967+0000", 2024-04-07T16:16:16.910 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.635967+0000", 2024-04-07T16:16:16.910 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:16.910 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:16.910 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:16.910 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:16.910 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 63, 2024-04-07T16:16:16.910 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:16.911 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:16.911 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:16.911 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:16.911 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:16.911 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:16.911 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:16.911 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:16.911 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.15", 2024-04-07T16:16:16.911 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:16.911 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:16.911 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:16.911 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 28, 2024-04-07T16:16:16.911 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:16.911 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T19:54:33.089377+0000", 2024-04-07T16:16:16.911 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:16.912 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:16.912 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:16.912 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:16.912 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:16.912 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:16.912 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:16.912 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:16.912 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:16.912 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:16.912 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:16.912 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:16.912 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:16.912 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:16.912 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:16.912 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:16.912 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:16.913 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:16.913 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:16.913 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:16.913 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:16.913 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:16.913 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:16.913 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:16.913 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:16.913 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:16.913 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:16.913 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:16.913 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:16.913 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:16.913 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:16.913 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:16.914 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:16.914 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:16.914 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:16.914 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:16.914 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:16.914 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:16.914 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:16.914 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:16.914 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:16.914 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:16.914 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:16.914 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.914 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:16.914 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:16.914 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:16.915 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T16:16:16.915 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T16:16:16.915 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T16:16:16.915 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:16.915 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 7, 2024-04-07T16:16:16.915 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:16.915 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.915 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:16.915 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:16.915 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T16:16:16.915 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T16:16:16.915 INFO:tasks.workunit.client.0.smithi100.stdout: 3 2024-04-07T16:16:16.915 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:16.915 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 0, 2024-04-07T16:16:16.915 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:16.915 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:16.915 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T16:16:16.916 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:16.916 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:16.916 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:16.916 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.029978+0000", 2024-04-07T16:16:16.916 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.032811+0000", 2024-04-07T16:16:16.916 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.032811+0000", 2024-04-07T16:16:16.916 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.033352+0000", 2024-04-07T16:16:16.916 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.029978+0000", 2024-04-07T16:16:16.916 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:16.916 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:16.916 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:16.916 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 60, 2024-04-07T16:16:16.916 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.029978+0000", 2024-04-07T16:16:16.916 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.029978+0000", 2024-04-07T16:16:16.916 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.029978+0000", 2024-04-07T16:16:16.917 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:16.917 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:16.917 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:16.917 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.029978+0000", 2024-04-07T16:16:16.917 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.029978+0000", 2024-04-07T16:16:16.917 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:16.917 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 2, 2024-04-07T16:16:16.917 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:16.917 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:16.917 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 59, 2024-04-07T16:16:16.917 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:16.917 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:16.917 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:16.917 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:16.917 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 2, 2024-04-07T16:16:16.918 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:16.918 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:16.918 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:16.918 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.13", 2024-04-07T16:16:16.918 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:16.918 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:16.918 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:16.918 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 22, 2024-04-07T16:16:16.918 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:16.918 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T01:50:02.195507+0000", 2024-04-07T16:16:16.918 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:16.918 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:16.918 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:16.918 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 34, 2024-04-07T16:16:16.918 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:16.919 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:16.919 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:16.919 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:16.919 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:16.919 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:16.919 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:16.919 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:16.919 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:16.919 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:16.919 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:16.919 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:16.919 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:16.919 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:16.919 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:16.919 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 6, 2024-04-07T16:16:16.919 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 2, 2024-04-07T16:16:16.919 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:16.920 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 2, 2024-04-07T16:16:16.920 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:16.920 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:16.920 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:16.920 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:16.920 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:16.920 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 1, 2024-04-07T16:16:16.920 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:16.920 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:16.920 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:16.920 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:16.920 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:16.920 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:16.920 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:16.920 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:16.920 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:16.921 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:16.921 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:16.921 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:16.921 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 3, 2024-04-07T16:16:16.921 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 7 2024-04-07T16:16:16.921 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.921 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:16.921 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:16.921 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:16.921 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T16:16:16.921 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T16:16:16.921 INFO:tasks.workunit.client.0.smithi100.stdout: 3 2024-04-07T16:16:16.921 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:16.921 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 0, 2024-04-07T16:16:16.921 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "61'2" 2024-04-07T16:16:16.922 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.922 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:16.922 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:16.922 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T16:16:16.922 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T16:16:16.922 INFO:tasks.workunit.client.0.smithi100.stdout: 3 2024-04-07T16:16:16.922 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:16.922 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 2, 2024-04-07T16:16:16.922 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:16.922 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:16.922 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T16:16:16.922 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:16.922 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:16.922 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:16.922 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.763887+0000", 2024-04-07T16:16:16.923 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:26.224225+0000", 2024-04-07T16:16:16.923 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:26.224225+0000", 2024-04-07T16:16:16.923 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:26.226163+0000", 2024-04-07T16:16:16.923 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.763887+0000", 2024-04-07T16:16:16.923 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:16.923 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:16.923 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:16.923 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 64, 2024-04-07T16:16:16.923 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.763887+0000", 2024-04-07T16:16:16.923 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.763887+0000", 2024-04-07T16:16:16.923 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.763887+0000", 2024-04-07T16:16:16.923 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:16.923 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:16.923 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:16.923 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.763887+0000", 2024-04-07T16:16:16.923 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.763887+0000", 2024-04-07T16:16:16.924 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:16.924 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:16.924 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:16.924 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:16.924 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 63, 2024-04-07T16:16:16.924 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:16.924 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:16.924 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:16.924 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:16.924 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:16.924 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:16.924 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:16.924 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:16.924 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.12", 2024-04-07T16:16:16.924 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:16.924 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:16.925 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:16.925 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 17, 2024-04-07T16:16:16.925 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:16.925 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T23:54:31.223096+0000", 2024-04-07T16:16:16.925 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:16.925 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:16.925 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:16.925 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:16.925 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:16.925 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:16.925 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:16.925 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:16.925 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:16.925 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:16.925 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:16.926 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:16.926 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:16.926 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:16.926 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:16.926 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:16.926 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:16.926 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:16.926 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:16.926 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:16.926 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:16.926 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:16.926 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:16.926 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:16.926 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:16.927 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:16.927 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:16.927 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:16.927 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:16.927 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:16.927 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:16.927 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:16.927 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:16.927 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:16.927 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:16.927 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:16.927 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:16.927 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:16.927 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:16.927 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:16.928 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:16.928 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:16.928 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:16.928 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.928 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:16.928 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:16.928 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:16.928 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T16:16:16.928 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T16:16:16.928 INFO:tasks.workunit.client.0.smithi100.stdout: 3 2024-04-07T16:16:16.928 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:16.928 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 2, 2024-04-07T16:16:16.928 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:16.928 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.928 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:16.928 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:16.929 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T16:16:16.929 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T16:16:16.929 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T16:16:16.929 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:16.929 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 5, 2024-04-07T16:16:16.929 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:16.929 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:16.929 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T16:16:16.929 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:16.929 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:16.929 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:16.929 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.024421+0000", 2024-04-07T16:16:16.929 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.045086+0000", 2024-04-07T16:16:16.929 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.045086+0000", 2024-04-07T16:16:16.929 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.045494+0000", 2024-04-07T16:16:16.929 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.024421+0000", 2024-04-07T16:16:16.930 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:16.930 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:16.930 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:16.930 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 60, 2024-04-07T16:16:16.930 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.024421+0000", 2024-04-07T16:16:16.930 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.024421+0000", 2024-04-07T16:16:16.930 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.024421+0000", 2024-04-07T16:16:16.930 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:16.930 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:16.930 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:16.930 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.024421+0000", 2024-04-07T16:16:16.930 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.024421+0000", 2024-04-07T16:16:16.930 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:16.930 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:16.931 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:16.931 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:16.931 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 59, 2024-04-07T16:16:16.931 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:16.931 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:16.931 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:16.931 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:16.931 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:16.931 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:16.931 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:16.931 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:16.931 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.12", 2024-04-07T16:16:16.931 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:16.931 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:16.932 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:16.932 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 20, 2024-04-07T16:16:16.932 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:16.932 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T23:15:11.494437+0000", 2024-04-07T16:16:16.932 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:16.932 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:16.932 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:16.932 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:16.932 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:16.932 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:16.932 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:16.932 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:16.932 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:16.932 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:16.932 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:16.932 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:16.933 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:16.933 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:16.933 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:16.933 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:16.933 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:16.933 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:16.933 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:16.933 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:16.933 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:16.933 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:16.933 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:16.933 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:16.933 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:16.933 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:16.934 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:16.934 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:16.934 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:16.934 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:16.934 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:16.934 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:16.934 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:16.934 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:16.934 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:16.934 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:16.934 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:16.934 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:16.934 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:16.934 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:16.934 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:16.935 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:16.935 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:16.935 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.935 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:16.935 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:16.935 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:16.935 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T16:16:16.935 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T16:16:16.935 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T16:16:16.935 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:16.935 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 5, 2024-04-07T16:16:16.935 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:16.935 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.935 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:16.936 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:16.936 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T16:16:16.936 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T16:16:16.936 INFO:tasks.workunit.client.0.smithi100.stdout: 2 2024-04-07T16:16:16.936 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:16.936 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 7, 2024-04-07T16:16:16.936 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:16.936 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:16.936 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T16:16:16.936 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:16.936 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:16.936 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:16.936 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.637043+0000", 2024-04-07T16:16:16.936 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.413447+0000", 2024-04-07T16:16:16.936 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.413447+0000", 2024-04-07T16:16:16.937 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.413928+0000", 2024-04-07T16:16:16.937 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.637043+0000", 2024-04-07T16:16:16.937 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:16.937 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:16.937 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:16.937 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 61, 2024-04-07T16:16:16.937 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.637043+0000", 2024-04-07T16:16:16.937 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.637043+0000", 2024-04-07T16:16:16.937 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.637043+0000", 2024-04-07T16:16:16.937 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:16.937 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:16.937 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:16.937 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.637043+0000", 2024-04-07T16:16:16.937 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.637043+0000", 2024-04-07T16:16:16.938 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:16.938 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:16.938 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:16.938 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:16.938 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 60, 2024-04-07T16:16:16.938 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:16.938 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:16.938 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:16.938 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:16.938 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:16.938 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:16.938 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:16.938 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:16.938 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.13", 2024-04-07T16:16:16.938 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:16.939 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:16.939 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:16.939 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 19, 2024-04-07T16:16:16.939 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:16.939 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T03:26:42.875228+0000", 2024-04-07T16:16:16.939 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:16.939 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:16.939 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:16.939 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:16.939 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:16.939 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:16.939 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:16.939 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:16.939 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:16.939 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:16.939 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:16.940 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:16.940 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:16.940 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:16.940 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:16.940 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:16.940 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:16.940 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:16.940 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:16.940 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:16.940 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:16.940 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:16.940 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:16.940 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:16.940 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:16.940 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:16.941 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:16.941 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:16.941 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:16.941 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:16.941 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:16.941 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:16.941 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:16.941 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:16.941 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:16.941 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:16.941 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:16.941 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:16.941 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:16.941 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:16.941 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:16.941 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:16.942 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:16.942 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.942 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:16.942 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:16.942 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:16.942 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T16:16:16.942 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T16:16:16.942 INFO:tasks.workunit.client.0.smithi100.stdout: 2 2024-04-07T16:16:16.942 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:16.942 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 7, 2024-04-07T16:16:16.942 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:16.942 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.942 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:16.942 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:16.942 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T16:16:16.943 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T16:16:16.943 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T16:16:16.943 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:16.943 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 6, 2024-04-07T16:16:16.943 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:16.943 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:16.943 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T16:16:16.943 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:16.943 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:16.943 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:16.943 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.526216+0000", 2024-04-07T16:16:16.943 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.032108+0000", 2024-04-07T16:16:16.943 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.032108+0000", 2024-04-07T16:16:16.943 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.032942+0000", 2024-04-07T16:16:16.943 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.526216+0000", 2024-04-07T16:16:16.944 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:16.944 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:16.944 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:16.944 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 60, 2024-04-07T16:16:16.944 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.526216+0000", 2024-04-07T16:16:16.944 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.526216+0000", 2024-04-07T16:16:16.944 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.526216+0000", 2024-04-07T16:16:16.944 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:16.944 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:16.944 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:16.944 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.526216+0000", 2024-04-07T16:16:16.944 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.526216+0000", 2024-04-07T16:16:16.944 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:16.944 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:16.944 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:16.945 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:16.945 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 59, 2024-04-07T16:16:16.945 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:16.945 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:16.945 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:16.945 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:16.945 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:16.945 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:16.945 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:16.945 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:16.945 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.11", 2024-04-07T16:16:16.945 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:16.945 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:16.945 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:16.945 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 20, 2024-04-07T16:16:16.946 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:16.946 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T18:53:06.864412+0000", 2024-04-07T16:16:16.946 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:16.946 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:16.946 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:16.946 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:16.946 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:16.946 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:16.946 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:16.946 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:16.946 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:16.946 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:16.946 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:16.946 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:16.947 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:16.947 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:16.947 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:16.947 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:16.947 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:16.947 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:16.947 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:16.947 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:16.947 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:16.947 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:16.947 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:16.947 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:16.947 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:16.947 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:16.947 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:16.948 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:16.948 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:16.948 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:16.948 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:16.948 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:16.948 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:16.948 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:16.948 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:16.948 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:16.948 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:16.948 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:16.948 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:16.948 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:16.948 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:16.949 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:16.949 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:16.949 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.949 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:16.949 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:16.949 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:16.949 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T16:16:16.949 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T16:16:16.949 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T16:16:16.949 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:16.949 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 6, 2024-04-07T16:16:16.949 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:16.949 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.949 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:16.949 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:16.949 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T16:16:16.950 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T16:16:16.950 INFO:tasks.workunit.client.0.smithi100.stdout: 5 2024-04-07T16:16:16.950 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:16.950 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 6, 2024-04-07T16:16:16.950 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:16.950 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:16.950 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T16:16:16.950 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:16.950 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:16.950 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:16.950 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.525876+0000", 2024-04-07T16:16:16.950 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.399823+0000", 2024-04-07T16:16:16.950 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.399823+0000", 2024-04-07T16:16:16.950 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.400141+0000", 2024-04-07T16:16:16.951 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.525876+0000", 2024-04-07T16:16:16.951 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:16.951 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:16.951 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:16.951 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 61, 2024-04-07T16:16:16.951 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.525876+0000", 2024-04-07T16:16:16.951 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.525876+0000", 2024-04-07T16:16:16.951 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.525876+0000", 2024-04-07T16:16:16.951 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:16.951 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:16.951 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:16.951 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.525876+0000", 2024-04-07T16:16:16.951 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.525876+0000", 2024-04-07T16:16:16.951 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:16.951 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:16.951 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:16.952 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:16.952 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 60, 2024-04-07T16:16:16.952 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:16.952 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:16.952 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:16.952 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:16.952 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:16.952 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:16.952 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:16.952 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:16.952 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.10", 2024-04-07T16:16:16.952 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:16.952 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:16.952 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:16.952 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 19, 2024-04-07T16:16:16.953 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:16.953 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T21:27:25.248595+0000", 2024-04-07T16:16:16.953 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:16.953 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:16.953 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:16.953 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:16.953 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:16.953 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:16.953 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:16.953 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:16.953 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:16.953 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:16.953 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:16.953 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:16.953 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:16.953 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:16.954 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:16.954 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:16.954 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:16.954 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:16.954 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:16.954 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:16.954 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:16.954 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:16.954 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:16.954 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:16.954 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:16.954 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:16.954 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:16.954 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:16.954 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:16.955 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:16.955 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:16.955 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:16.955 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:16.955 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:16.955 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:16.955 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:16.955 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:16.955 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:16.955 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:16.955 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:16.955 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:16.955 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:16.955 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:16.955 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.956 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:16.956 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:16.956 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:16.956 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T16:16:16.956 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T16:16:16.956 INFO:tasks.workunit.client.0.smithi100.stdout: 5 2024-04-07T16:16:16.956 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:16.956 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 6, 2024-04-07T16:16:16.956 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:16.956 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.956 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:16.956 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:16.956 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T16:16:16.956 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T16:16:16.956 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T16:16:16.957 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:16.957 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 2, 2024-04-07T16:16:16.957 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:16.957 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:16.957 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T16:16:16.957 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:16.957 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:16.957 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:16.957 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.764170+0000", 2024-04-07T16:16:16.957 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.033838+0000", 2024-04-07T16:16:16.957 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.033838+0000", 2024-04-07T16:16:16.957 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.034154+0000", 2024-04-07T16:16:16.957 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.764170+0000", 2024-04-07T16:16:16.957 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:16.957 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:16.958 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:16.958 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 60, 2024-04-07T16:16:16.958 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.764170+0000", 2024-04-07T16:16:16.958 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.764170+0000", 2024-04-07T16:16:16.958 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.764170+0000", 2024-04-07T16:16:16.958 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:16.958 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:16.958 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:16.958 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.764170+0000", 2024-04-07T16:16:16.958 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.764170+0000", 2024-04-07T16:16:16.958 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:16.958 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 8, 2024-04-07T16:16:16.958 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:16.958 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:16.958 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 59, 2024-04-07T16:16:16.959 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:16.959 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:16.959 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:16.959 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:16.959 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 8, 2024-04-07T16:16:16.959 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:16.959 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:16.959 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:16.959 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.10", 2024-04-07T16:16:16.959 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:16.959 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:16.959 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:16.959 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 28, 2024-04-07T16:16:16.959 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:16.959 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T23:13:07.216314+0000", 2024-04-07T16:16:16.960 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:16.960 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:16.960 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:16.960 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:16.960 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:16.960 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:16.960 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:16.960 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:16.960 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:16.960 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:16.960 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:16.960 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:16.960 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:16.960 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:16.960 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:16.961 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:16.961 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:16.961 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:16.961 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:16.961 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 12, 2024-04-07T16:16:16.961 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 4, 2024-04-07T16:16:16.961 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:16.961 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 4, 2024-04-07T16:16:16.961 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:16.961 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:16.961 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:16.961 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:16.961 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:16.961 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 4, 2024-04-07T16:16:16.961 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:16.962 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:16.962 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:16.962 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:16.962 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:16.962 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:16.962 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:16.962 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:16.962 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:16.962 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:16.962 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:16.962 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:16.962 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 12, 2024-04-07T16:16:16.962 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:16.962 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.962 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:16.963 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:16.963 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:16.963 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T16:16:16.963 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T16:16:16.963 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T16:16:16.963 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:16.963 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 2, 2024-04-07T16:16:16.963 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "61'8" 2024-04-07T16:16:16.963 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.963 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:16.963 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:16.963 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T16:16:16.963 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T16:16:16.963 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T16:16:16.963 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:16.964 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 7, 2024-04-07T16:16:16.964 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:16.964 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:16.964 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T16:16:16.964 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:16.964 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:16.964 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:16.964 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.636593+0000", 2024-04-07T16:16:16.964 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.385485+0000", 2024-04-07T16:16:16.964 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.385485+0000", 2024-04-07T16:16:16.964 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.385883+0000", 2024-04-07T16:16:16.964 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.636593+0000", 2024-04-07T16:16:16.964 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:16.964 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:16.964 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:16.965 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 61, 2024-04-07T16:16:16.965 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.636593+0000", 2024-04-07T16:16:16.965 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.636593+0000", 2024-04-07T16:16:16.965 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.636593+0000", 2024-04-07T16:16:16.965 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:16.965 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:16.965 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:16.965 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.636593+0000", 2024-04-07T16:16:16.965 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.636593+0000", 2024-04-07T16:16:16.965 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:16.965 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:16.965 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:16.965 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:16.965 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 60, 2024-04-07T16:16:16.965 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:16.966 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:16.966 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:16.966 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:16.966 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:16.966 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:16.966 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:16.966 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:16.966 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.11", 2024-04-07T16:16:16.966 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:16.966 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:16.966 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:16.966 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 19, 2024-04-07T16:16:16.966 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:16.966 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T21:01:09.262433+0000", 2024-04-07T16:16:16.966 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:16.966 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:16.967 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:16.967 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:16.967 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:16.967 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:16.967 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:16.967 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:16.967 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:16.967 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:16.967 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:16.967 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:16.967 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:16.967 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:16.967 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:16.967 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:16.967 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:16.968 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:16.968 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:16.968 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:16.968 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:16.968 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:16.968 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:16.968 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:16.968 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:16.968 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:16.968 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:16.968 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:16.968 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:16.968 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:16.968 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:16.968 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:16.968 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:16.969 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:16.969 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:16.969 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:16.969 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:16.969 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:16.969 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:16.969 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:16.969 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:16.969 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:16.969 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:16.969 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.969 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:16.969 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:16.969 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:16.969 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T16:16:16.969 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T16:16:16.970 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T16:16:16.970 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:16.970 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 7, 2024-04-07T16:16:16.970 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:16.970 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.970 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:16.970 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:16.970 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T16:16:16.970 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T16:16:16.970 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T16:16:16.970 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:16.970 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 4, 2024-04-07T16:16:16.970 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:16.970 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:16.971 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T16:16:16.971 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:16.971 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:16.971 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:16.971 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:39.825314+0000", 2024-04-07T16:16:16.971 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.027445+0000", 2024-04-07T16:16:16.971 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.027445+0000", 2024-04-07T16:16:16.971 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.028004+0000", 2024-04-07T16:16:16.971 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:39.825314+0000", 2024-04-07T16:16:16.971 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:16.971 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:16.971 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:16.972 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 60, 2024-04-07T16:16:16.972 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:39.825314+0000", 2024-04-07T16:16:16.972 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:39.825314+0000", 2024-04-07T16:16:16.972 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:39.825314+0000", 2024-04-07T16:16:16.972 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:16.972 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:16.972 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:16.972 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:39.825314+0000", 2024-04-07T16:16:16.972 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:39.825314+0000", 2024-04-07T16:16:16.972 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:16.972 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 1, 2024-04-07T16:16:16.972 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:16.972 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:16.972 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 59, 2024-04-07T16:16:16.973 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:16.973 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:16.973 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:16.973 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:16.973 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 1, 2024-04-07T16:16:16.973 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:16.973 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:16.973 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:16.973 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.f", 2024-04-07T16:16:16.973 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:16.973 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:16.973 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:16.973 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 21, 2024-04-07T16:16:16.973 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:16.973 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T17:55:06.727142+0000", 2024-04-07T16:16:16.974 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:16.974 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:16.974 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:16.974 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 22, 2024-04-07T16:16:16.974 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:16.974 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:16.974 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:16.974 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:16.974 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:16.974 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:16.974 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:16.974 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:16.974 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:16.974 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:16.974 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:16.974 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:16.975 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:16.975 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:16.975 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:16.975 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 3, 2024-04-07T16:16:16.975 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 1, 2024-04-07T16:16:16.975 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:16.975 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 1, 2024-04-07T16:16:16.975 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:16.975 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:16.975 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:16.975 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:16.975 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:16.975 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:16.975 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:16.975 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:16.976 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:16.976 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:16.976 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:16.976 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:16.976 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:16.976 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:16.976 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:16.976 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:16.976 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:16.976 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:16.976 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 1, 2024-04-07T16:16:16.976 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 1 2024-04-07T16:16:16.976 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.976 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:16.977 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:16.977 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:16.977 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T16:16:16.977 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T16:16:16.977 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T16:16:16.977 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:16.977 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 4, 2024-04-07T16:16:16.977 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "61'1" 2024-04-07T16:16:16.977 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.977 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:16.977 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:16.977 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T16:16:16.977 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T16:16:16.977 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T16:16:16.977 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:16.978 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 7, 2024-04-07T16:16:16.978 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:16.978 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:16.978 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T16:16:16.978 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:16.978 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:16.978 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:16.978 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.637723+0000", 2024-04-07T16:16:16.978 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.386057+0000", 2024-04-07T16:16:16.978 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.386057+0000", 2024-04-07T16:16:16.978 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.386584+0000", 2024-04-07T16:16:16.978 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.637723+0000", 2024-04-07T16:16:16.978 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:16.978 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:16.979 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:16.979 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 61, 2024-04-07T16:16:16.979 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.637723+0000", 2024-04-07T16:16:16.979 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.637723+0000", 2024-04-07T16:16:16.979 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.637723+0000", 2024-04-07T16:16:16.979 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:16.979 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:16.979 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:16.979 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.637723+0000", 2024-04-07T16:16:16.979 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.637723+0000", 2024-04-07T16:16:16.979 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:16.979 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:16.979 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:16.979 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:16.979 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 60, 2024-04-07T16:16:16.980 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:16.980 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:16.980 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:16.980 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:16.980 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:16.980 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:16.980 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:16.980 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:16.980 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.e", 2024-04-07T16:16:16.980 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:16.980 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:16.980 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:16.980 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 19, 2024-04-07T16:16:16.980 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:16.980 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T17:57:57.548179+0000", 2024-04-07T16:16:16.981 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:16.981 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:16.981 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:16.981 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:16.981 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:16.981 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:16.981 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:16.981 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:16.981 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:16.981 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:16.981 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:16.981 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:16.981 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:16.981 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:16.981 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:16.981 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:16.982 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:16.982 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:16.982 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:16.982 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:16.982 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:16.982 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:16.982 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:16.982 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:16.982 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:16.982 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:16.982 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:16.982 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:16.982 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:16.982 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:16.983 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:16.983 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:16.983 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:16.983 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:16.983 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:16.983 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:16.983 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:16.983 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:16.983 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:16.983 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:16.983 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:16.983 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:16.983 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:16.983 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.983 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:16.984 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:16.984 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:16.984 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T16:16:16.984 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T16:16:16.984 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T16:16:16.984 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:16.984 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 7, 2024-04-07T16:16:16.984 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:16.984 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.984 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:16.984 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:16.984 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T16:16:16.984 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T16:16:16.984 INFO:tasks.workunit.client.0.smithi100.stdout: 4 2024-04-07T16:16:16.984 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:16.985 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 1, 2024-04-07T16:16:16.985 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:16.985 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:16.985 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 66, 2024-04-07T16:16:16.985 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:16.985 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:16.985 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:16.985 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.236643+0000", 2024-04-07T16:16:16.985 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:13:30.008706+0000", 2024-04-07T16:16:16.985 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:13:30.008706+0000", 2024-04-07T16:16:16.985 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:13:30.009561+0000", 2024-04-07T16:16:16.985 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.236643+0000", 2024-04-07T16:16:16.985 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:13:28.963534+0000", 2024-04-07T16:16:16.986 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:16.986 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:13:28.963534+0000", 2024-04-07T16:16:16.986 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 67, 2024-04-07T16:16:16.986 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.236643+0000", 2024-04-07T16:16:16.986 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.236643+0000", 2024-04-07T16:16:16.986 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.236643+0000", 2024-04-07T16:16:16.986 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:16.986 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:16.986 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:13:28.963534+0000", 2024-04-07T16:16:16.986 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.236643+0000", 2024-04-07T16:16:16.986 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.236643+0000", 2024-04-07T16:16:16.986 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:16.986 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:16.986 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:16.987 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:16.987 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 66, 2024-04-07T16:16:16.987 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:16.987 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:16.987 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:16.987 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:16.987 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:16.987 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:16.987 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:16.987 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:16.987 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "4.2", 2024-04-07T16:16:16.987 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:16.987 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:16.987 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:16.988 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 13, 2024-04-07T16:16:16.988 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:16.988 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T00:36:26.958742+0000", 2024-04-07T16:16:16.988 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:16.988 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:16.988 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:16.988 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:16.988 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:16.988 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:16.988 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:16.988 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:16.988 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:16.988 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:16.988 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:16.988 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:16.989 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:16.989 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:16.989 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:16.989 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:16.989 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:16.989 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:16.989 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:16.989 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:16.989 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:16.989 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:16.989 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:16.989 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:16.989 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:16.989 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:16.990 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:16.990 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:16.990 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:16.990 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:16.990 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:16.990 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:16.990 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:16.990 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:16.990 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:16.990 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:16.990 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:16.990 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:16.990 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:16.990 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:16.990 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:16.990 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:16.991 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:16.991 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.991 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:16.991 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:16.991 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:16.991 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T16:16:16.991 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T16:16:16.991 INFO:tasks.workunit.client.0.smithi100.stdout: 4 2024-04-07T16:16:16.991 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:16.991 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 1, 2024-04-07T16:16:16.991 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:16.991 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.991 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:16.991 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:16.991 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T16:16:16.992 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T16:16:16.992 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T16:16:16.992 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:16.992 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 1, 2024-04-07T16:16:16.992 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:16.992 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:16.992 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T16:16:16.992 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:16.992 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:16.992 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:16.992 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.236743+0000", 2024-04-07T16:16:16.992 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.037627+0000", 2024-04-07T16:16:16.992 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.037627+0000", 2024-04-07T16:16:16.992 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.037974+0000", 2024-04-07T16:16:16.992 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.236743+0000", 2024-04-07T16:16:16.993 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:16.993 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:16.993 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:16.993 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 60, 2024-04-07T16:16:16.993 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.236743+0000", 2024-04-07T16:16:16.993 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.236743+0000", 2024-04-07T16:16:16.993 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.236743+0000", 2024-04-07T16:16:16.993 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:16.993 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:16.993 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:16.993 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.236743+0000", 2024-04-07T16:16:16.993 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.236743+0000", 2024-04-07T16:16:16.993 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:16.993 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:16.993 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:16.993 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:16.994 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 59, 2024-04-07T16:16:16.994 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:16.994 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:16.994 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:16.994 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:16.994 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:16.994 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:16.994 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:16.994 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:16.994 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.4", 2024-04-07T16:16:16.994 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:16.994 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:16.994 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:16.994 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 20, 2024-04-07T16:16:16.994 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:16.995 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T19:28:12.174861+0000", 2024-04-07T16:16:16.995 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:16.995 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:16.995 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:16.995 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:16.995 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:16.995 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:16.995 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:16.995 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:16.995 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:16.995 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:16.995 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:16.995 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:16.995 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:16.995 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:16.996 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:16.996 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:16.996 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:16.996 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:16.996 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:16.996 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:16.996 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:16.996 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:16.996 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:16.996 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:16.996 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:16.996 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:16.996 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:16.996 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:16.996 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:16.997 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:16.997 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:16.997 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:16.997 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:16.997 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:16.997 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:16.997 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:16.997 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:16.997 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:16.997 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:16.997 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:16.997 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:16.997 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:16.997 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:16.998 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.998 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:16.998 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:16.998 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:16.998 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T16:16:16.998 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T16:16:16.998 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T16:16:16.998 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:16.998 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 1, 2024-04-07T16:16:16.998 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:16.998 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:16.998 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:16.998 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:16.998 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T16:16:16.998 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T16:16:16.999 INFO:tasks.workunit.client.0.smithi100.stdout: 2 2024-04-07T16:16:16.999 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:16.999 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 5, 2024-04-07T16:16:16.999 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:16.999 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:16.999 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T16:16:16.999 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:16.999 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:16.999 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:16.999 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.025595+0000", 2024-04-07T16:16:16.999 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.394113+0000", 2024-04-07T16:16:16.999 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.394113+0000", 2024-04-07T16:16:16.999 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.395073+0000", 2024-04-07T16:16:16.999 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.025595+0000", 2024-04-07T16:16:16.999 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:17.000 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:17.000 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:17.000 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 61, 2024-04-07T16:16:17.000 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.025595+0000", 2024-04-07T16:16:17.000 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.025595+0000", 2024-04-07T16:16:17.000 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.025595+0000", 2024-04-07T16:16:17.000 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:17.000 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:17.000 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:17.000 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.025595+0000", 2024-04-07T16:16:17.000 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.025595+0000", 2024-04-07T16:16:17.000 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:17.000 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:17.000 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:17.000 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:17.001 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 60, 2024-04-07T16:16:17.001 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:17.001 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:17.001 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:17.001 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:17.001 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:17.001 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:17.001 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:17.001 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:17.001 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.5", 2024-04-07T16:16:17.001 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:17.001 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:17.001 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:17.001 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 19, 2024-04-07T16:16:17.001 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:17.001 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T23:28:32.978580+0000", 2024-04-07T16:16:17.002 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:17.002 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:17.002 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:17.002 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:17.002 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:17.002 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:17.002 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:17.002 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:17.002 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:17.002 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:17.002 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:17.002 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:17.002 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:17.002 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:17.003 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:17.003 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:17.003 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:17.003 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:17.003 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:17.003 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:17.003 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:17.003 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:17.003 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:17.003 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:17.003 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:17.003 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:17.003 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:17.003 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:17.003 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:17.004 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:17.004 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:17.004 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:17.004 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:17.004 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:17.004 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:17.004 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:17.004 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:17.004 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:17.004 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:17.004 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:17.004 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:17.004 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:17.004 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:17.004 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:17.004 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:17.005 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:17.005 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:17.005 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T16:16:17.005 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T16:16:17.005 INFO:tasks.workunit.client.0.smithi100.stdout: 2 2024-04-07T16:16:17.005 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:17.005 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 5, 2024-04-07T16:16:17.005 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:17.005 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:17.005 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:17.005 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:17.005 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T16:16:17.005 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T16:16:17.005 INFO:tasks.workunit.client.0.smithi100.stdout: 3 2024-04-07T16:16:17.005 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:17.006 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 1, 2024-04-07T16:16:17.006 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:17.006 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:17.006 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 66, 2024-04-07T16:16:17.006 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:17.006 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:17.006 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:17.006 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.236861+0000", 2024-04-07T16:16:17.006 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:13:30.040318+0000", 2024-04-07T16:16:17.006 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:13:30.040318+0000", 2024-04-07T16:16:17.006 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:13:30.041147+0000", 2024-04-07T16:16:17.006 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.236861+0000", 2024-04-07T16:16:17.006 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:13:28.963534+0000", 2024-04-07T16:16:17.006 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:17.006 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:13:28.963534+0000", 2024-04-07T16:16:17.007 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 67, 2024-04-07T16:16:17.007 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.236861+0000", 2024-04-07T16:16:17.007 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.236861+0000", 2024-04-07T16:16:17.007 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.236861+0000", 2024-04-07T16:16:17.007 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:17.007 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:17.007 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:13:28.963534+0000", 2024-04-07T16:16:17.007 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.236861+0000", 2024-04-07T16:16:17.007 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.236861+0000", 2024-04-07T16:16:17.007 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:17.007 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:17.007 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:17.007 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:17.007 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 66, 2024-04-07T16:16:17.007 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:17.008 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:17.008 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:17.008 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:17.008 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:17.008 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:17.008 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:17.008 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:17.008 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "4.4", 2024-04-07T16:16:17.008 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:17.008 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:17.008 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:17.008 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 13, 2024-04-07T16:16:17.008 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:17.008 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T20:45:42.413572+0000", 2024-04-07T16:16:17.008 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:17.009 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:17.009 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:17.009 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:17.009 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:17.009 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:17.009 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:17.009 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:17.009 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:17.009 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:17.009 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:17.009 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:17.009 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:17.009 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:17.009 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:17.009 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:17.010 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:17.010 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:17.010 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:17.010 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:17.010 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:17.010 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:17.010 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:17.010 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:17.010 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:17.010 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:17.010 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:17.010 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:17.010 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:17.010 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:17.011 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:17.011 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:17.011 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:17.011 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:17.011 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:17.011 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:17.011 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:17.011 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:17.011 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:17.011 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:17.011 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:17.011 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:17.011 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:17.011 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:17.011 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:17.012 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:17.012 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:17.012 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T16:16:17.012 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T16:16:17.012 INFO:tasks.workunit.client.0.smithi100.stdout: 3 2024-04-07T16:16:17.012 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:17.012 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 1, 2024-04-07T16:16:17.012 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:17.012 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:17.012 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:17.012 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:17.012 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T16:16:17.012 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T16:16:17.012 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T16:16:17.013 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:17.013 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 5, 2024-04-07T16:16:17.013 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:17.013 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:17.013 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T16:16:17.013 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:17.013 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:17.013 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:17.013 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.024311+0000", 2024-04-07T16:16:17.013 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.035625+0000", 2024-04-07T16:16:17.013 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.035625+0000", 2024-04-07T16:16:17.013 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.036336+0000", 2024-04-07T16:16:17.013 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.024311+0000", 2024-04-07T16:16:17.013 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:17.013 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:17.014 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:17.014 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 60, 2024-04-07T16:16:17.014 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.024311+0000", 2024-04-07T16:16:17.014 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.024311+0000", 2024-04-07T16:16:17.014 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.024311+0000", 2024-04-07T16:16:17.014 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:17.014 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:17.014 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:17.014 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.024311+0000", 2024-04-07T16:16:17.014 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.024311+0000", 2024-04-07T16:16:17.014 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:17.014 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:17.014 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:17.014 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:17.015 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 59, 2024-04-07T16:16:17.015 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:17.015 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:17.015 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:17.015 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:17.015 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:17.015 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:17.015 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:17.015 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:17.015 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.2", 2024-04-07T16:16:17.015 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:17.015 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:17.015 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:17.015 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 20, 2024-04-07T16:16:17.015 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:17.016 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T18:15:00.506827+0000", 2024-04-07T16:16:17.016 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:17.016 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:17.016 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:17.016 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:17.016 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:17.016 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:17.016 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:17.016 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:17.016 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:17.016 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:17.016 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:17.016 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:17.016 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:17.016 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:17.017 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:17.017 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:17.017 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:17.017 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:17.017 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:17.017 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:17.017 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:17.017 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:17.017 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:17.017 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:17.017 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:17.017 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:17.017 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:17.017 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:17.017 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:17.018 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:17.018 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:17.018 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:17.018 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:17.018 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:17.018 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:17.018 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:17.018 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:17.018 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:17.018 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:17.018 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:17.018 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:17.018 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:17.018 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:17.019 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:17.019 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:17.019 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:17.019 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:17.019 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T16:16:17.019 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T16:16:17.019 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T16:16:17.019 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:17.019 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 5, 2024-04-07T16:16:17.019 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:17.019 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:17.019 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:17.019 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:17.019 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T16:16:17.019 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T16:16:17.019 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T16:16:17.020 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:17.020 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 4, 2024-04-07T16:16:17.020 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:17.020 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:17.020 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T16:16:17.020 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:17.020 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:17.020 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:17.020 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:39.824462+0000", 2024-04-07T16:16:17.020 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.404029+0000", 2024-04-07T16:16:17.020 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.404029+0000", 2024-04-07T16:16:17.020 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.404406+0000", 2024-04-07T16:16:17.020 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:39.824462+0000", 2024-04-07T16:16:17.020 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:17.020 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:17.021 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:17.021 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 61, 2024-04-07T16:16:17.021 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:39.824462+0000", 2024-04-07T16:16:17.021 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:39.824462+0000", 2024-04-07T16:16:17.021 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:39.824462+0000", 2024-04-07T16:16:17.021 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:17.021 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:17.021 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:17.021 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:39.824462+0000", 2024-04-07T16:16:17.021 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:39.824462+0000", 2024-04-07T16:16:17.021 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:17.021 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:17.021 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:17.021 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:17.021 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 60, 2024-04-07T16:16:17.022 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:17.022 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:17.022 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:17.022 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:17.022 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:17.022 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:17.022 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:17.022 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:17.022 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.3", 2024-04-07T16:16:17.022 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:17.022 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:17.022 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:17.022 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 19, 2024-04-07T16:16:17.022 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:17.023 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T16:41:25.469343+0000", 2024-04-07T16:16:17.023 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:17.023 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:17.023 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:17.023 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:17.023 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:17.023 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:17.023 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:17.023 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:17.023 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:17.023 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:17.023 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:17.023 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:17.023 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:17.024 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:17.024 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:17.024 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:17.024 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:17.024 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:17.024 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:17.024 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:17.024 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:17.024 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:17.024 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:17.024 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:17.024 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:17.024 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:17.024 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:17.024 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:17.025 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:17.025 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:17.025 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:17.025 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:17.025 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:17.025 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:17.025 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:17.025 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:17.025 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:17.025 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:17.025 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:17.025 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:17.025 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:17.025 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:17.025 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:17.026 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:17.026 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:17.026 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:17.026 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:17.026 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T16:16:17.026 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T16:16:17.026 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T16:16:17.026 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:17.026 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 4, 2024-04-07T16:16:17.026 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:17.026 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:17.026 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:17.026 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:17.026 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T16:16:17.027 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T16:16:17.027 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T16:16:17.027 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:17.027 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 1, 2024-04-07T16:16:17.027 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:17.027 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:17.027 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 66, 2024-04-07T16:16:17.027 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:17.027 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:17.027 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:17.027 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.237236+0000", 2024-04-07T16:16:17.027 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:13:30.011181+0000", 2024-04-07T16:16:17.027 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:13:30.011181+0000", 2024-04-07T16:16:17.027 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:13:30.012072+0000", 2024-04-07T16:16:17.028 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.237236+0000", 2024-04-07T16:16:17.028 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:13:28.963534+0000", 2024-04-07T16:16:17.028 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:17.028 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:13:28.963534+0000", 2024-04-07T16:16:17.028 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 67, 2024-04-07T16:16:17.028 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.237236+0000", 2024-04-07T16:16:17.028 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.237236+0000", 2024-04-07T16:16:17.028 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.237236+0000", 2024-04-07T16:16:17.028 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:17.028 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:17.028 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:13:28.963534+0000", 2024-04-07T16:16:17.028 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.237236+0000", 2024-04-07T16:16:17.028 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.237236+0000", 2024-04-07T16:16:17.028 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:17.029 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:17.029 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:17.029 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:17.029 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 66, 2024-04-07T16:16:17.029 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:17.029 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:17.029 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:17.029 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:17.029 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:17.029 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:17.029 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:17.029 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:17.029 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "4.7", 2024-04-07T16:16:17.029 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:17.029 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:17.030 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:17.030 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 13, 2024-04-07T16:16:17.030 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:17.030 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T21:19:09.530105+0000", 2024-04-07T16:16:17.030 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:17.030 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:17.030 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:17.030 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:17.030 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:17.030 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:17.030 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:17.030 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:17.030 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:17.030 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:17.030 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:17.031 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:17.031 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:17.031 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:17.031 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:17.031 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:17.031 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:17.031 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:17.031 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:17.031 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:17.031 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:17.031 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:17.031 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:17.031 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:17.031 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:17.031 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:17.032 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:17.032 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:17.032 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:17.032 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:17.032 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:17.032 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:17.032 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:17.032 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:17.032 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:17.032 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:17.032 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:17.032 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:17.032 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:17.032 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:17.032 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:17.033 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:17.033 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:17.033 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:17.033 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:17.033 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:17.033 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:17.033 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T16:16:17.033 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T16:16:17.033 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T16:16:17.033 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:17.033 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 1, 2024-04-07T16:16:17.033 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:17.033 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:17.033 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:17.033 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:17.034 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T16:16:17.034 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T16:16:17.034 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T16:16:17.034 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:17.034 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 1, 2024-04-07T16:16:17.034 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:17.034 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:17.034 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T16:16:17.034 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:17.034 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:17.034 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:17.034 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.235831+0000", 2024-04-07T16:16:17.034 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.393472+0000", 2024-04-07T16:16:17.034 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.393472+0000", 2024-04-07T16:16:17.034 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.393979+0000", 2024-04-07T16:16:17.035 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.235831+0000", 2024-04-07T16:16:17.035 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:17.035 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:17.035 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:17.035 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 61, 2024-04-07T16:16:17.035 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.235831+0000", 2024-04-07T16:16:17.035 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.235831+0000", 2024-04-07T16:16:17.035 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.235831+0000", 2024-04-07T16:16:17.035 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:17.035 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:17.035 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:17.035 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.235831+0000", 2024-04-07T16:16:17.035 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.235831+0000", 2024-04-07T16:16:17.035 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:17.035 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:17.036 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:17.036 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:17.036 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 60, 2024-04-07T16:16:17.036 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:17.036 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:17.036 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:17.036 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:17.036 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:17.036 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:17.036 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:17.036 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:17.036 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.0", 2024-04-07T16:16:17.036 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:17.036 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:17.037 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:17.037 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 19, 2024-04-07T16:16:17.037 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:17.037 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T21:51:13.834107+0000", 2024-04-07T16:16:17.037 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:17.037 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:17.037 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:17.037 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:17.037 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:17.037 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:17.037 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:17.037 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:17.037 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:17.037 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:17.038 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:17.038 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:17.038 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:17.038 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:17.038 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:17.038 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:17.038 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:17.038 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:17.038 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:17.038 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:17.038 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:17.038 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:17.038 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:17.038 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:17.039 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:17.039 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:17.039 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:17.039 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:17.039 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:17.039 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:17.039 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:17.039 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:17.039 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:17.039 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:17.039 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:17.039 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:17.039 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:17.039 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:17.039 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:17.039 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:17.040 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:17.040 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:17.040 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:17.040 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:17.040 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:17.040 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:17.040 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:17.040 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T16:16:17.040 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T16:16:17.040 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T16:16:17.040 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:17.040 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 1, 2024-04-07T16:16:17.040 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:17.040 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:17.041 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:17.041 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:17.041 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T16:16:17.041 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T16:16:17.041 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T16:16:17.041 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:17.041 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 2, 2024-04-07T16:16:17.041 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:17.041 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:17.041 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T16:16:17.041 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:17.041 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:17.041 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:17.041 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.764382+0000", 2024-04-07T16:16:17.041 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.034795+0000", 2024-04-07T16:16:17.041 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.034795+0000", 2024-04-07T16:16:17.042 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.035071+0000", 2024-04-07T16:16:17.042 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.764382+0000", 2024-04-07T16:16:17.042 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:17.042 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:17.042 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:17.042 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 60, 2024-04-07T16:16:17.042 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.764382+0000", 2024-04-07T16:16:17.042 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.764382+0000", 2024-04-07T16:16:17.042 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.764382+0000", 2024-04-07T16:16:17.042 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:17.042 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:17.042 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:17.042 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.764382+0000", 2024-04-07T16:16:17.042 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.764382+0000", 2024-04-07T16:16:17.042 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:17.043 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:17.043 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:17.043 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:17.043 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 59, 2024-04-07T16:16:17.043 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:17.043 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:17.043 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:17.043 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:17.043 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:17.043 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:17.043 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:17.043 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:17.043 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.1", 2024-04-07T16:16:17.043 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:17.044 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:17.044 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:17.044 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 20, 2024-04-07T16:16:17.044 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:17.044 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T17:51:30.599527+0000", 2024-04-07T16:16:17.044 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:17.044 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:17.044 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:17.044 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:17.044 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:17.044 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:17.044 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:17.044 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:17.044 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:17.044 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:17.044 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:17.045 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:17.045 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:17.045 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:17.045 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:17.045 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:17.045 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:17.045 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:17.045 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:17.045 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:17.045 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:17.045 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:17.045 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:17.045 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:17.045 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:17.045 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:17.046 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:17.046 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:17.046 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:17.046 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:17.046 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:17.046 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:17.046 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:17.046 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:17.046 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:17.046 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:17.046 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:17.046 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:17.046 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:17.046 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:17.046 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:17.047 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:17.047 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:17.047 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:17.047 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:17.047 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:17.047 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:17.047 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T16:16:17.047 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T16:16:17.047 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T16:16:17.047 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:17.047 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 2, 2024-04-07T16:16:17.047 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:17.047 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:17.047 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:17.047 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:17.048 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T16:16:17.048 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T16:16:17.048 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T16:16:17.048 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:17.048 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 6, 2024-04-07T16:16:17.048 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:17.048 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:17.048 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 66, 2024-04-07T16:16:17.048 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:17.048 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:17.048 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:17.048 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.526254+0000", 2024-04-07T16:16:17.048 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:13:30.035058+0000", 2024-04-07T16:16:17.048 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:13:30.035058+0000", 2024-04-07T16:16:17.049 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:13:30.035513+0000", 2024-04-07T16:16:17.049 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.526254+0000", 2024-04-07T16:16:17.049 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:13:28.963534+0000", 2024-04-07T16:16:17.049 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:17.049 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:13:28.963534+0000", 2024-04-07T16:16:17.049 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 67, 2024-04-07T16:16:17.049 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.526254+0000", 2024-04-07T16:16:17.049 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.526254+0000", 2024-04-07T16:16:17.049 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.526254+0000", 2024-04-07T16:16:17.049 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:17.049 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:17.049 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:13:28.963534+0000", 2024-04-07T16:16:17.049 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.526254+0000", 2024-04-07T16:16:17.049 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.526254+0000", 2024-04-07T16:16:17.050 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:17.050 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:17.050 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:17.050 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:17.050 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 66, 2024-04-07T16:16:17.050 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:17.050 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:17.050 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:17.050 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:17.050 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:17.050 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:17.050 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:17.050 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:17.050 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "4.5", 2024-04-07T16:16:17.051 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:17.051 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:17.051 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:17.051 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 13, 2024-04-07T16:16:17.051 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:17.051 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T22:41:30.257805+0000", 2024-04-07T16:16:17.051 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:17.051 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:17.051 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:17.051 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:17.051 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:17.051 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:17.051 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:17.051 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:17.051 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:17.052 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:17.052 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:17.052 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:17.052 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:17.052 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:17.052 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:17.052 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:17.052 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:17.052 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:17.052 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:17.052 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:17.052 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:17.052 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:17.052 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:17.052 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:17.053 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:17.053 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:17.053 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:17.053 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:17.053 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:17.053 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:17.053 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:17.053 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:17.053 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:17.053 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:17.053 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:17.053 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:17.053 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:17.053 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:17.054 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:17.054 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:17.054 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:17.054 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:17.054 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:17.054 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:17.054 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:17.054 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:17.054 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:17.054 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T16:16:17.054 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T16:16:17.054 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T16:16:17.054 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:17.054 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 6, 2024-04-07T16:16:17.055 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:17.055 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:17.055 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:17.055 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:17.055 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T16:16:17.055 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T16:16:17.055 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T16:16:17.055 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:17.055 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 3, 2024-04-07T16:16:17.055 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:17.055 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:17.055 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T16:16:17.055 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:17.055 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:17.055 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:17.056 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.186788+0000", 2024-04-07T16:16:17.056 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.394392+0000", 2024-04-07T16:16:17.056 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.394392+0000", 2024-04-07T16:16:17.056 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.394635+0000", 2024-04-07T16:16:17.056 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.186788+0000", 2024-04-07T16:16:17.056 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:17.056 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:17.056 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:17.056 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 61, 2024-04-07T16:16:17.056 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.186788+0000", 2024-04-07T16:16:17.056 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.186788+0000", 2024-04-07T16:16:17.056 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.186788+0000", 2024-04-07T16:16:17.056 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:17.056 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:17.056 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:17.057 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.186788+0000", 2024-04-07T16:16:17.057 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.186788+0000", 2024-04-07T16:16:17.057 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:17.057 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:17.057 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:17.057 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:17.057 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 60, 2024-04-07T16:16:17.057 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:17.057 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:17.057 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:17.057 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:17.057 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:17.057 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:17.057 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:17.058 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:17.058 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.2", 2024-04-07T16:16:17.058 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:17.058 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:17.058 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:17.058 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 19, 2024-04-07T16:16:17.058 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:17.058 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T17:52:10.950481+0000", 2024-04-07T16:16:17.058 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:17.058 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:17.058 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:17.058 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:17.058 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:17.058 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:17.059 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:17.059 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:17.059 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:17.059 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:17.059 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:17.059 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:17.059 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:17.059 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:17.059 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:17.059 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:17.059 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:17.059 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:17.059 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:17.059 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:17.059 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:17.059 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:17.060 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:17.060 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:17.060 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:17.060 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:17.060 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:17.060 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:17.060 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:17.060 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:17.060 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:17.060 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:17.060 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:17.060 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:17.060 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:17.060 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:17.060 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:17.060 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:17.061 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:17.061 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:17.061 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:17.061 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:17.061 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:17.061 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:17.061 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:17.061 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:17.061 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:17.061 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T16:16:17.061 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T16:16:17.061 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T16:16:17.061 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:17.061 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 3, 2024-04-07T16:16:17.061 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:17.061 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:17.062 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:17.062 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:17.062 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T16:16:17.062 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T16:16:17.062 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T16:16:17.062 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:17.062 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 5, 2024-04-07T16:16:17.062 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:17.062 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:17.062 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T16:16:17.062 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:17.062 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:17.062 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:17.062 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.024445+0000", 2024-04-07T16:16:17.063 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.044830+0000", 2024-04-07T16:16:17.063 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.044830+0000", 2024-04-07T16:16:17.063 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.045131+0000", 2024-04-07T16:16:17.063 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.024445+0000", 2024-04-07T16:16:17.063 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:17.063 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:17.063 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:17.063 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 60, 2024-04-07T16:16:17.063 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.024445+0000", 2024-04-07T16:16:17.063 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.024445+0000", 2024-04-07T16:16:17.063 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.024445+0000", 2024-04-07T16:16:17.063 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:17.063 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:17.063 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:17.063 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.024445+0000", 2024-04-07T16:16:17.063 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.024445+0000", 2024-04-07T16:16:17.064 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:17.064 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:17.064 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:17.064 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:17.064 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 59, 2024-04-07T16:16:17.064 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:17.064 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:17.064 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:17.064 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:17.064 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:17.064 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:17.064 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:17.064 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:17.064 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.3", 2024-04-07T16:16:17.064 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:17.064 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:17.065 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:17.065 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 20, 2024-04-07T16:16:17.065 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:17.065 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T02:48:50.652491+0000", 2024-04-07T16:16:17.065 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:17.065 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:17.065 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:17.065 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:17.065 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:17.065 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:17.065 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:17.065 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:17.065 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:17.065 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:17.065 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:17.065 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:17.066 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:17.066 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:17.066 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:17.066 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:17.066 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:17.066 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:17.066 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:17.066 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:17.066 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:17.066 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:17.066 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:17.066 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:17.066 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:17.066 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:17.067 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:17.067 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:17.067 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:17.067 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:17.067 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:17.067 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:17.067 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:17.067 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:17.067 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:17.067 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:17.067 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:17.067 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:17.067 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:17.067 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:17.067 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:17.067 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:17.068 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:17.068 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:17.068 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:17.068 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:17.068 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:17.068 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T16:16:17.068 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T16:16:17.068 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T16:16:17.068 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:17.068 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 5, 2024-04-07T16:16:17.068 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:17.068 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:17.068 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:17.068 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:17.068 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T16:16:17.068 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T16:16:17.069 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T16:16:17.069 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:17.069 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 7, 2024-04-07T16:16:17.069 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:17.069 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:17.069 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 19, 2024-04-07T16:16:17.069 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:17.069 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:17.069 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:17.069 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:16:12.239722+0000", 2024-04-07T16:16:17.069 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:11:33.518060+0000", 2024-04-07T16:16:17.069 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:11:33.518060+0000", 2024-04-07T16:16:17.069 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:11:33.518868+0000", 2024-04-07T16:16:17.069 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:16:12.239722+0000", 2024-04-07T16:16:17.069 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:07:26.866003+0000", 2024-04-07T16:16:17.069 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:17.070 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:07:26.866003+0000", 2024-04-07T16:16:17.070 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 58, 2024-04-07T16:16:17.070 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:16:12.239722+0000", 2024-04-07T16:16:17.070 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:16:12.239722+0000", 2024-04-07T16:16:17.070 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:16:12.239722+0000", 2024-04-07T16:16:17.070 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:17.070 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:17.070 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:07:26.866003+0000", 2024-04-07T16:16:17.070 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:16:12.239722+0000", 2024-04-07T16:16:17.070 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:16:12.239722+0000", 2024-04-07T16:16:17.070 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:17.070 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 156, 2024-04-07T16:16:17.070 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:17.070 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:17.070 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 57, 2024-04-07T16:16:17.070 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:17.071 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:17.071 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:17.071 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:17.071 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 156, 2024-04-07T16:16:17.071 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:17.071 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:17.071 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:17.071 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "1.0", 2024-04-07T16:16:17.071 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:17.071 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:17.071 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:17.071 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 367, 2024-04-07T16:16:17.071 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:17.071 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T22:22:51.043941+0000", 2024-04-07T16:16:17.071 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:17.071 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:17.072 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:17.072 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 590368, 2024-04-07T16:16:17.072 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:17.072 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 1771104, 2024-04-07T16:16:17.072 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:17.072 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:17.072 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:17.072 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:17.072 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:17.072 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:17.072 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:17.072 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:17.072 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:17.072 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:17.072 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:17.072 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:17.073 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:17.073 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 6, 2024-04-07T16:16:17.073 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 2, 2024-04-07T16:16:17.073 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:17.073 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 2, 2024-04-07T16:16:17.073 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:17.073 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:17.073 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:17.073 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:17.073 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:17.073 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:17.073 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:17.073 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 6, 2024-04-07T16:16:17.073 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:17.073 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:17.073 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:17.074 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:17.074 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:17.074 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 466, 2024-04-07T16:16:17.074 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 644, 2024-04-07T16:16:17.074 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:17.074 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:17.074 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:17.074 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 235, 2024-04-07T16:16:17.074 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 2416 2024-04-07T16:16:17.074 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:17.074 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:17.074 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:17.074 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:17.074 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T16:16:17.074 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T16:16:17.075 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T16:16:17.075 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:17.075 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 7, 2024-04-07T16:16:17.075 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "69'156" 2024-04-07T16:16:17.075 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:17.075 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:17.075 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:17.075 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T16:16:17.075 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T16:16:17.075 INFO:tasks.workunit.client.0.smithi100.stdout: 2 2024-04-07T16:16:17.075 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:17.075 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 0, 2024-04-07T16:16:17.075 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:17.075 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:17.075 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 66, 2024-04-07T16:16:17.076 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:17.076 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:17.076 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:17.076 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.030611+0000", 2024-04-07T16:16:17.076 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:13:30.040114+0000", 2024-04-07T16:16:17.076 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:13:30.040114+0000", 2024-04-07T16:16:17.076 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:13:30.041000+0000", 2024-04-07T16:16:17.076 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.030611+0000", 2024-04-07T16:16:17.076 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:13:28.963534+0000", 2024-04-07T16:16:17.076 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:17.076 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:13:28.963534+0000", 2024-04-07T16:16:17.076 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 67, 2024-04-07T16:16:17.076 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.030611+0000", 2024-04-07T16:16:17.076 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.030611+0000", 2024-04-07T16:16:17.076 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.030611+0000", 2024-04-07T16:16:17.077 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:17.077 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:17.077 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:13:28.963534+0000", 2024-04-07T16:16:17.077 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.030611+0000", 2024-04-07T16:16:17.077 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.030611+0000", 2024-04-07T16:16:17.077 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:17.077 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:17.077 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:17.077 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:17.077 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 66, 2024-04-07T16:16:17.077 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:17.077 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:17.077 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:17.077 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:17.077 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:17.078 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:17.078 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:17.078 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:17.078 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "4.6", 2024-04-07T16:16:17.078 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:17.078 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:17.078 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:17.078 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 13, 2024-04-07T16:16:17.078 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:17.078 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T03:03:13.375776+0000", 2024-04-07T16:16:17.078 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:17.078 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:17.078 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:17.078 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:17.079 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:17.079 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:17.079 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:17.079 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:17.079 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:17.079 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:17.079 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:17.079 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:17.079 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:17.079 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:17.079 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:17.079 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:17.079 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:17.079 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:17.079 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:17.080 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:17.080 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:17.080 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:17.080 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:17.080 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:17.080 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:17.080 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:17.080 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:17.080 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:17.080 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:17.080 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:17.080 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:17.080 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:17.080 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:17.080 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:17.081 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:17.081 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:17.081 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:17.081 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:17.081 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:17.081 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:17.081 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:17.081 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:17.081 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:17.081 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:17.081 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:17.081 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:17.081 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:17.081 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T16:16:17.081 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T16:16:17.081 INFO:tasks.workunit.client.0.smithi100.stdout: 2 2024-04-07T16:16:17.082 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:17.082 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 0, 2024-04-07T16:16:17.082 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:17.082 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:17.082 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:17.082 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:17.082 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T16:16:17.082 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T16:16:17.082 INFO:tasks.workunit.client.0.smithi100.stdout: 3 2024-04-07T16:16:17.082 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:17.082 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 2, 2024-04-07T16:16:17.082 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:17.082 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:17.082 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T16:16:17.082 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:17.083 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:17.083 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:17.083 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.763809+0000", 2024-04-07T16:16:17.083 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:26.225518+0000", 2024-04-07T16:16:17.083 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:26.225518+0000", 2024-04-07T16:16:17.083 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:26.226945+0000", 2024-04-07T16:16:17.083 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.763809+0000", 2024-04-07T16:16:17.083 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:17.083 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:17.083 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:17.083 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 64, 2024-04-07T16:16:17.083 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.763809+0000", 2024-04-07T16:16:17.083 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.763809+0000", 2024-04-07T16:16:17.083 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.763809+0000", 2024-04-07T16:16:17.083 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:17.084 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:17.084 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:17.084 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.763809+0000", 2024-04-07T16:16:17.084 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.763809+0000", 2024-04-07T16:16:17.084 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:17.084 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:17.084 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:17.084 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:17.084 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 63, 2024-04-07T16:16:17.084 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:17.084 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:17.084 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:17.084 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:17.084 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:17.084 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:17.085 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:17.085 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:17.085 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.1", 2024-04-07T16:16:17.085 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:17.085 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:17.085 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:17.085 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 17, 2024-04-07T16:16:17.085 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:17.085 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T22:14:39.830063+0000", 2024-04-07T16:16:17.085 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:17.085 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:17.085 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:17.085 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:17.085 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:17.085 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:17.086 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:17.086 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:17.086 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:17.086 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:17.086 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:17.086 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:17.086 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:17.086 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:17.086 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:17.086 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:17.086 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:17.086 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:17.086 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:17.086 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:17.087 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:17.087 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:17.087 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:17.087 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:17.087 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:17.087 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:17.087 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:17.087 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:17.087 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:17.087 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:17.087 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:17.087 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:17.087 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:17.087 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:17.088 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:17.088 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:17.088 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:17.088 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:17.088 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:17.088 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:17.088 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:17.088 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:17.088 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:17.088 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:17.088 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:17.088 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:17.088 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:17.088 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T16:16:17.088 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T16:16:17.089 INFO:tasks.workunit.client.0.smithi100.stdout: 3 2024-04-07T16:16:17.089 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:17.089 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 2, 2024-04-07T16:16:17.089 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:17.089 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:17.089 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:17.089 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:17.089 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T16:16:17.089 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T16:16:17.089 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T16:16:17.089 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:17.089 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 7, 2024-04-07T16:16:17.089 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:17.089 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:17.089 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T16:16:17.090 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:17.090 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:17.090 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:17.090 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.636018+0000", 2024-04-07T16:16:17.090 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.034825+0000", 2024-04-07T16:16:17.090 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.034825+0000", 2024-04-07T16:16:17.090 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.035536+0000", 2024-04-07T16:16:17.090 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.636018+0000", 2024-04-07T16:16:17.090 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:17.090 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:17.090 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:17.090 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 60, 2024-04-07T16:16:17.090 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.636018+0000", 2024-04-07T16:16:17.090 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.636018+0000", 2024-04-07T16:16:17.091 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.636018+0000", 2024-04-07T16:16:17.091 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:17.091 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:17.091 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:17.091 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.636018+0000", 2024-04-07T16:16:17.091 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.636018+0000", 2024-04-07T16:16:17.091 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:17.091 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:17.091 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:17.091 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:17.091 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 59, 2024-04-07T16:16:17.091 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:17.091 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:17.091 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:17.092 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:17.092 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:17.092 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:17.092 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:17.092 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:17.092 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.0", 2024-04-07T16:16:17.092 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:17.092 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:17.092 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:17.092 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 20, 2024-04-07T16:16:17.092 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:17.092 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T20:11:57.033306+0000", 2024-04-07T16:16:17.092 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:17.092 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:17.092 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:17.093 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:17.093 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:17.093 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:17.093 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:17.093 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:17.093 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:17.093 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:17.093 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:17.093 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:17.093 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:17.093 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:17.093 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:17.093 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:17.093 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:17.093 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:17.093 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:17.094 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:17.094 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:17.094 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:17.094 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:17.094 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:17.094 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:17.094 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:17.094 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:17.094 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:17.094 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:17.094 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:17.094 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:17.094 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:17.094 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:17.094 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:17.095 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:17.095 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:17.095 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:17.095 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:17.095 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:17.095 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:17.095 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:17.095 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:17.095 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:17.095 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:17.095 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:17.095 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:17.095 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:17.095 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T16:16:17.095 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T16:16:17.095 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T16:16:17.096 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:17.096 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 7, 2024-04-07T16:16:17.096 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:17.096 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:17.096 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:17.096 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:17.096 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T16:16:17.096 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T16:16:17.096 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T16:16:17.096 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:17.096 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 0, 2024-04-07T16:16:17.096 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:17.096 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:17.096 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 66, 2024-04-07T16:16:17.097 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:17.097 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:17.097 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:17.097 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.029002+0000", 2024-04-07T16:16:17.097 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:13:30.040143+0000", 2024-04-07T16:16:17.097 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:13:30.040143+0000", 2024-04-07T16:16:17.097 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:13:30.041281+0000", 2024-04-07T16:16:17.097 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.029002+0000", 2024-04-07T16:16:17.097 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:13:28.963534+0000", 2024-04-07T16:16:17.097 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:17.097 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:13:28.963534+0000", 2024-04-07T16:16:17.097 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 67, 2024-04-07T16:16:17.097 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.029002+0000", 2024-04-07T16:16:17.097 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.029002+0000", 2024-04-07T16:16:17.098 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.029002+0000", 2024-04-07T16:16:17.098 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:17.098 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:17.098 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:13:28.963534+0000", 2024-04-07T16:16:17.098 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.029002+0000", 2024-04-07T16:16:17.098 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.029002+0000", 2024-04-07T16:16:17.098 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:17.098 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:17.098 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:17.098 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:17.098 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 66, 2024-04-07T16:16:17.098 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:17.098 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:17.098 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:17.098 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:17.099 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:17.099 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:17.099 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:17.099 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:17.099 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "4.3", 2024-04-07T16:16:17.099 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:17.099 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:17.099 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:17.099 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 13, 2024-04-07T16:16:17.099 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:17.099 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T01:04:02.813753+0000", 2024-04-07T16:16:17.099 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:17.099 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:17.099 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:17.099 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:17.100 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:17.100 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:17.100 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:17.100 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:17.100 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:17.100 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:17.100 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:17.100 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:17.100 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:17.100 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:17.100 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:17.100 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:17.100 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:17.100 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:17.100 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:17.101 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:17.101 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:17.101 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:17.101 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:17.101 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:17.101 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:17.101 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:17.101 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:17.101 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:17.101 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:17.101 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:17.101 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:17.101 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:17.101 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:17.101 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:17.101 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:17.102 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:17.102 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:17.102 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:17.102 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:17.102 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:17.102 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:17.102 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:17.102 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:17.102 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:17.102 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:17.102 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:17.102 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:17.102 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T16:16:17.102 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T16:16:17.102 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T16:16:17.102 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:17.103 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 0, 2024-04-07T16:16:17.103 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:17.103 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:17.103 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:17.103 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:17.103 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T16:16:17.103 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T16:16:17.103 INFO:tasks.workunit.client.0.smithi100.stdout: 4 2024-04-07T16:16:17.103 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:17.103 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 7, 2024-04-07T16:16:17.103 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:17.103 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:17.103 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T16:16:17.103 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:17.103 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:17.103 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:17.104 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.637358+0000", 2024-04-07T16:16:17.104 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.035201+0000", 2024-04-07T16:16:17.104 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.035201+0000", 2024-04-07T16:16:17.104 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.035747+0000", 2024-04-07T16:16:17.104 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.637358+0000", 2024-04-07T16:16:17.104 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:17.104 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:17.104 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:17.104 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 60, 2024-04-07T16:16:17.104 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.637358+0000", 2024-04-07T16:16:17.104 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.637358+0000", 2024-04-07T16:16:17.104 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.637358+0000", 2024-04-07T16:16:17.104 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:17.104 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:17.104 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:17.104 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.637358+0000", 2024-04-07T16:16:17.105 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.637358+0000", 2024-04-07T16:16:17.105 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:17.105 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:17.105 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:17.105 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:17.105 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 59, 2024-04-07T16:16:17.105 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:17.105 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:17.105 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:17.105 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:17.105 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:17.105 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:17.105 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:17.105 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:17.105 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.5", 2024-04-07T16:16:17.106 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:17.106 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:17.106 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:17.106 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 20, 2024-04-07T16:16:17.106 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:17.106 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T21:03:25.449728+0000", 2024-04-07T16:16:17.106 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:17.106 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:17.106 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:17.106 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:17.106 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:17.106 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:17.106 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:17.106 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:17.106 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:17.107 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:17.107 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:17.107 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:17.107 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:17.107 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:17.107 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:17.107 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:17.107 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:17.107 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:17.107 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:17.107 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:17.107 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:17.107 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:17.107 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:17.107 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:17.107 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:17.108 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:17.108 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:17.108 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:17.108 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:17.108 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:17.108 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:17.108 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:17.108 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:17.108 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:17.108 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:17.108 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:17.108 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:17.108 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:17.108 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:17.108 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:17.108 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:17.108 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:17.109 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:17.109 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:17.109 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:17.109 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:17.109 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:17.109 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T16:16:17.109 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T16:16:17.109 INFO:tasks.workunit.client.0.smithi100.stdout: 4 2024-04-07T16:16:17.109 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:17.109 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 7, 2024-04-07T16:16:17.109 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:17.109 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:17.109 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:17.109 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:17.109 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T16:16:17.109 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T16:16:17.110 INFO:tasks.workunit.client.0.smithi100.stdout: 5 2024-04-07T16:16:17.110 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:17.110 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 1, 2024-04-07T16:16:17.110 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:17.110 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:17.110 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T16:16:17.110 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:17.110 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:17.110 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:17.110 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.237019+0000", 2024-04-07T16:16:17.110 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.398210+0000", 2024-04-07T16:16:17.110 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.398210+0000", 2024-04-07T16:16:17.110 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.398785+0000", 2024-04-07T16:16:17.110 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.237019+0000", 2024-04-07T16:16:17.110 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:17.110 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:17.111 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:17.111 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 61, 2024-04-07T16:16:17.111 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.237019+0000", 2024-04-07T16:16:17.111 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.237019+0000", 2024-04-07T16:16:17.111 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.237019+0000", 2024-04-07T16:16:17.111 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:17.111 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:17.111 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:17.111 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.237019+0000", 2024-04-07T16:16:17.111 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.237019+0000", 2024-04-07T16:16:17.111 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:17.111 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:17.111 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:17.111 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:17.111 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 60, 2024-04-07T16:16:17.112 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:17.112 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:17.112 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:17.112 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:17.112 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:17.112 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:17.112 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:17.112 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:17.112 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.4", 2024-04-07T16:16:17.112 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:17.112 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:17.112 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:17.112 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 19, 2024-04-07T16:16:17.112 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:17.112 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T02:11:50.752075+0000", 2024-04-07T16:16:17.112 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:17.112 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:17.113 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:17.113 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:17.113 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:17.113 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:17.113 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:17.113 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:17.113 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:17.113 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:17.113 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:17.113 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:17.113 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:17.113 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:17.113 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:17.113 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:17.113 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:17.113 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:17.114 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:17.114 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:17.114 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:17.114 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:17.114 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:17.114 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:17.114 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:17.114 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:17.114 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:17.114 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:17.114 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:17.114 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:17.114 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:17.114 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:17.114 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:17.115 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:17.115 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:17.115 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:17.115 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:17.115 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:17.115 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:17.115 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:17.115 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:17.115 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:17.115 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:17.115 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:17.115 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:17.115 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:17.115 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:17.115 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T16:16:17.115 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T16:16:17.116 INFO:tasks.workunit.client.0.smithi100.stdout: 5 2024-04-07T16:16:17.116 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:17.116 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 1, 2024-04-07T16:16:17.116 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:17.116 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:17.116 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:17.116 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:17.116 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T16:16:17.116 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T16:16:17.116 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T16:16:17.116 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:17.116 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 3, 2024-04-07T16:16:17.116 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:17.116 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:17.116 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 66, 2024-04-07T16:16:17.116 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:17.117 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:17.117 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:17.117 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.186582+0000", 2024-04-07T16:16:17.117 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:13:30.040026+0000", 2024-04-07T16:16:17.117 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:13:30.040026+0000", 2024-04-07T16:16:17.117 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:13:30.040851+0000", 2024-04-07T16:16:17.117 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.186582+0000", 2024-04-07T16:16:17.117 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:13:28.963534+0000", 2024-04-07T16:16:17.117 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:17.117 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:13:28.963534+0000", 2024-04-07T16:16:17.117 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 67, 2024-04-07T16:16:17.117 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.186582+0000", 2024-04-07T16:16:17.117 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.186582+0000", 2024-04-07T16:16:17.117 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.186582+0000", 2024-04-07T16:16:17.117 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:17.118 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:17.118 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:13:28.963534+0000", 2024-04-07T16:16:17.118 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.186582+0000", 2024-04-07T16:16:17.118 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.186582+0000", 2024-04-07T16:16:17.118 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:17.118 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:17.118 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:17.118 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:17.118 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 66, 2024-04-07T16:16:17.118 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:17.118 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:17.118 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:17.118 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:17.118 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:17.118 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:17.119 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:17.119 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:17.119 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "4.0", 2024-04-07T16:16:17.119 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:17.119 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:17.119 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:17.119 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 13, 2024-04-07T16:16:17.119 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:17.119 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T18:29:22.422209+0000", 2024-04-07T16:16:17.119 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:17.119 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:17.119 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:17.119 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:17.119 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:17.119 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:17.120 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:17.120 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:17.120 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:17.120 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:17.120 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:17.120 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:17.120 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:17.120 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:17.120 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:17.120 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:17.120 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:17.120 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:17.120 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:17.120 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:17.120 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:17.121 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:17.121 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:17.121 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:17.121 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:17.121 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:17.121 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:17.121 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:17.121 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:17.121 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:17.121 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:17.121 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:17.121 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:17.121 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:17.121 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:17.121 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:17.121 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:17.122 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:17.122 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:17.122 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:17.122 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:17.122 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:17.122 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:17.122 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:17.122 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:17.122 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:17.122 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:17.122 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T16:16:17.122 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T16:16:17.122 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T16:16:17.122 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:17.123 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 3, 2024-04-07T16:16:17.123 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:17.123 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:17.123 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:17.123 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:17.123 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T16:16:17.123 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T16:16:17.123 INFO:tasks.workunit.client.0.smithi100.stdout: 4 2024-04-07T16:16:17.123 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:17.123 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 1, 2024-04-07T16:16:17.123 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:17.123 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:17.123 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T16:16:17.123 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:17.123 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:17.123 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:17.124 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.237606+0000", 2024-04-07T16:16:17.124 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.039676+0000", 2024-04-07T16:16:17.124 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.039676+0000", 2024-04-07T16:16:17.124 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.039980+0000", 2024-04-07T16:16:17.124 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.237606+0000", 2024-04-07T16:16:17.124 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:17.124 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:17.124 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:17.124 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 60, 2024-04-07T16:16:17.124 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.237606+0000", 2024-04-07T16:16:17.124 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.237606+0000", 2024-04-07T16:16:17.124 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.237606+0000", 2024-04-07T16:16:17.124 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:17.124 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:17.124 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:17.125 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.237606+0000", 2024-04-07T16:16:17.125 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.237606+0000", 2024-04-07T16:16:17.125 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:17.125 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 1, 2024-04-07T16:16:17.125 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:17.125 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:17.125 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 59, 2024-04-07T16:16:17.125 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:17.125 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:17.125 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:17.125 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:17.125 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 1, 2024-04-07T16:16:17.125 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:17.125 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:17.125 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:17.126 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.6", 2024-04-07T16:16:17.126 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:17.126 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:17.126 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:17.126 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 21, 2024-04-07T16:16:17.126 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:17.126 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T02:37:05.297496+0000", 2024-04-07T16:16:17.126 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:17.126 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:17.126 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:17.126 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:17.126 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:17.126 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:17.126 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:17.127 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:17.127 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:17.127 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:17.127 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:17.127 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:17.127 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:17.127 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:17.127 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:17.127 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:17.127 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:17.127 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:17.127 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:17.127 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 3, 2024-04-07T16:16:17.127 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 1, 2024-04-07T16:16:17.127 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:17.128 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 1, 2024-04-07T16:16:17.128 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:17.128 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:17.128 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:17.128 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:17.128 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:17.128 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 1, 2024-04-07T16:16:17.128 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:17.128 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:17.128 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:17.128 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:17.128 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:17.128 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:17.128 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:17.128 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:17.128 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:17.129 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:17.129 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:17.129 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:17.129 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 1, 2024-04-07T16:16:17.129 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:17.129 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:17.129 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:17.129 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:17.129 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:17.129 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T16:16:17.129 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T16:16:17.129 INFO:tasks.workunit.client.0.smithi100.stdout: 4 2024-04-07T16:16:17.129 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:17.129 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 1, 2024-04-07T16:16:17.129 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "62'1" 2024-04-07T16:16:17.130 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:17.130 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:17.130 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:17.130 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T16:16:17.130 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T16:16:17.130 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T16:16:17.130 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:17.130 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 3, 2024-04-07T16:16:17.130 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:17.130 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:17.130 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T16:16:17.130 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:17.130 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:17.130 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:17.131 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.187224+0000", 2024-04-07T16:16:17.131 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.399184+0000", 2024-04-07T16:16:17.131 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.399184+0000", 2024-04-07T16:16:17.131 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.399486+0000", 2024-04-07T16:16:17.131 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.187224+0000", 2024-04-07T16:16:17.131 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:17.131 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:17.131 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:17.131 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 61, 2024-04-07T16:16:17.131 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.187224+0000", 2024-04-07T16:16:17.131 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.187224+0000", 2024-04-07T16:16:17.131 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.187224+0000", 2024-04-07T16:16:17.131 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:17.131 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:17.131 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:17.132 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.187224+0000", 2024-04-07T16:16:17.132 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.187224+0000", 2024-04-07T16:16:17.132 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:17.132 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:17.132 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:17.132 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:17.132 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 60, 2024-04-07T16:16:17.132 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:17.132 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:17.132 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:17.132 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:17.132 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:17.132 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:17.132 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:17.132 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:17.133 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.7", 2024-04-07T16:16:17.133 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:17.133 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:17.133 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:17.133 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 19, 2024-04-07T16:16:17.133 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:17.133 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T21:07:25.104309+0000", 2024-04-07T16:16:17.133 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:17.133 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:17.133 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:17.133 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:17.133 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:17.133 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:17.133 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:17.133 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:17.134 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:17.134 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:17.134 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:17.134 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:17.134 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:17.134 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:17.134 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:17.134 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:17.134 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:17.134 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:17.134 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:17.134 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:17.134 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:17.134 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:17.134 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:17.135 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:17.135 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:17.135 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:17.135 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:17.135 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:17.135 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:17.135 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:17.135 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:17.135 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:17.135 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:17.135 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:17.135 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:17.135 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:17.135 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:17.135 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:17.135 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:17.136 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:17.136 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:17.136 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:17.136 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:17.136 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:17.136 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:17.136 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:17.136 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:17.136 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T16:16:17.136 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T16:16:17.136 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T16:16:17.136 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:17.136 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 3, 2024-04-07T16:16:17.136 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:17.136 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:17.137 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:17.137 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:17.137 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T16:16:17.137 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T16:16:17.137 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T16:16:17.137 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:17.137 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 4, 2024-04-07T16:16:17.137 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:17.137 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:17.137 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 66, 2024-04-07T16:16:17.137 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:17.137 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:17.137 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:17.137 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:39.823554+0000", 2024-04-07T16:16:17.138 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:13:30.002945+0000", 2024-04-07T16:16:17.138 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:13:30.002945+0000", 2024-04-07T16:16:17.138 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:13:30.003468+0000", 2024-04-07T16:16:17.138 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:39.823554+0000", 2024-04-07T16:16:17.138 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:13:28.963534+0000", 2024-04-07T16:16:17.138 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:17.138 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:13:28.963534+0000", 2024-04-07T16:16:17.138 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 67, 2024-04-07T16:16:17.138 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:39.823554+0000", 2024-04-07T16:16:17.138 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:39.823554+0000", 2024-04-07T16:16:17.138 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:39.823554+0000", 2024-04-07T16:16:17.138 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:17.138 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:17.138 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:13:28.963534+0000", 2024-04-07T16:16:17.139 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:39.823554+0000", 2024-04-07T16:16:17.139 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:39.823554+0000", 2024-04-07T16:16:17.139 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:17.139 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:17.139 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:17.139 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:17.139 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 66, 2024-04-07T16:16:17.139 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:17.139 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:17.139 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:17.139 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:17.139 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:17.139 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:17.139 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:17.139 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:17.139 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "4.1", 2024-04-07T16:16:17.140 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:17.140 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:17.140 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:17.140 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 13, 2024-04-07T16:16:17.140 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:17.140 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T03:30:42.844279+0000", 2024-04-07T16:16:17.140 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:17.140 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:17.140 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:17.140 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:17.140 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:17.140 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:17.140 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:17.140 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:17.140 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:17.141 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:17.141 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:17.141 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:17.141 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:17.141 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:17.141 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:17.141 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:17.141 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:17.141 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:17.141 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:17.141 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:17.141 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:17.141 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:17.141 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:17.141 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:17.141 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:17.142 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:17.142 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:17.142 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:17.142 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:17.142 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:17.142 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:17.142 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:17.142 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:17.142 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:17.142 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:17.142 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:17.142 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:17.142 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:17.142 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:17.142 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:17.142 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:17.143 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:17.143 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:17.143 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:17.143 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:17.143 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:17.143 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:17.143 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T16:16:17.143 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T16:16:17.143 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T16:16:17.143 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:17.143 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 4, 2024-04-07T16:16:17.143 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:17.143 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:17.143 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:17.143 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:17.144 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T16:16:17.144 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T16:16:17.144 INFO:tasks.workunit.client.0.smithi100.stdout: 2 2024-04-07T16:16:17.144 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:17.144 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 6, 2024-04-07T16:16:17.144 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:17.144 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:17.144 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T16:16:17.144 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:17.144 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:17.144 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:17.144 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.552993+0000", 2024-04-07T16:16:17.144 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:26.084985+0000", 2024-04-07T16:16:17.144 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:26.084985+0000", 2024-04-07T16:16:17.144 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:26.091787+0000", 2024-04-07T16:16:17.144 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.552993+0000", 2024-04-07T16:16:17.144 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:17.145 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:17.145 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:17.145 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 64, 2024-04-07T16:16:17.145 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.552993+0000", 2024-04-07T16:16:17.145 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.552993+0000", 2024-04-07T16:16:17.145 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.552993+0000", 2024-04-07T16:16:17.145 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:17.145 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:17.145 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:17.145 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.552993+0000", 2024-04-07T16:16:17.145 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.552993+0000", 2024-04-07T16:16:17.145 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:17.145 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 2, 2024-04-07T16:16:17.145 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:17.145 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:17.145 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 63, 2024-04-07T16:16:17.146 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:17.146 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:17.146 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:17.146 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:17.146 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 2, 2024-04-07T16:16:17.146 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:17.146 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:17.146 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:17.146 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.7", 2024-04-07T16:16:17.146 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:17.146 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:17.146 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:17.146 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 35, 2024-04-07T16:16:17.146 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:17.146 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T02:39:40.077566+0000", 2024-04-07T16:16:17.147 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:17.147 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:17.147 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:17.147 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:17.147 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:17.147 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:17.147 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:17.147 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:17.147 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:17.147 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:17.147 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:17.147 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:17.147 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:17.147 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:17.147 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:17.148 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:17.148 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:17.148 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:17.148 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:17.148 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 3, 2024-04-07T16:16:17.148 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 1, 2024-04-07T16:16:17.148 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:17.148 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 1, 2024-04-07T16:16:17.148 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:17.148 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:17.148 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:17.148 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:17.148 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:17.148 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 1, 2024-04-07T16:16:17.148 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:17.148 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 1, 2024-04-07T16:16:17.149 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:17.149 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:17.149 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:17.149 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:17.149 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:17.149 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:17.149 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:17.149 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:17.149 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:17.149 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:17.149 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 3, 2024-04-07T16:16:17.149 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:17.149 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:17.149 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:17.149 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:17.149 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:17.150 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T16:16:17.150 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T16:16:17.150 INFO:tasks.workunit.client.0.smithi100.stdout: 2 2024-04-07T16:16:17.150 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:17.150 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 6, 2024-04-07T16:16:17.150 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "61'2" 2024-04-07T16:16:17.150 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:17.150 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:17.150 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:17.150 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T16:16:17.150 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T16:16:17.150 INFO:tasks.workunit.client.0.smithi100.stdout: 4 2024-04-07T16:16:17.150 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:17.150 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 0, 2024-04-07T16:16:17.150 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:17.151 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:17.151 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T16:16:17.151 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:17.151 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:17.151 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:17.151 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.031201+0000", 2024-04-07T16:16:17.151 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.406018+0000", 2024-04-07T16:16:17.151 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.406018+0000", 2024-04-07T16:16:17.151 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.406541+0000", 2024-04-07T16:16:17.151 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.031201+0000", 2024-04-07T16:16:17.151 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:17.151 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:17.151 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:17.151 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 61, 2024-04-07T16:16:17.151 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.031201+0000", 2024-04-07T16:16:17.152 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.031201+0000", 2024-04-07T16:16:17.152 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.031201+0000", 2024-04-07T16:16:17.152 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:17.152 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:17.152 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:17.152 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.031201+0000", 2024-04-07T16:16:17.152 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.031201+0000", 2024-04-07T16:16:17.152 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:17.152 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:17.152 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:17.152 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:17.152 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 60, 2024-04-07T16:16:17.152 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:17.152 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:17.152 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:17.152 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:17.153 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:17.153 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:17.153 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:17.153 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:17.153 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.6", 2024-04-07T16:16:17.153 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:17.153 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:17.153 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:17.153 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 19, 2024-04-07T16:16:17.153 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:17.153 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T00:24:42.112088+0000", 2024-04-07T16:16:17.153 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:17.153 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:17.153 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:17.153 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:17.154 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:17.154 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:17.154 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:17.154 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:17.154 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:17.154 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:17.154 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:17.154 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:17.154 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:17.154 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:17.154 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:17.154 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:17.154 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:17.154 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:17.154 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:17.155 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:17.155 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:17.155 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:17.155 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:17.155 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:17.155 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:17.155 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:17.155 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:17.155 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:17.155 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:17.155 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:17.155 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:17.155 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:17.155 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:17.155 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:17.155 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:17.156 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:17.156 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:17.156 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:17.156 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:17.156 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:17.156 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:17.156 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:17.156 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:17.156 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:17.156 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:17.156 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:17.156 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:17.156 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T16:16:17.156 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T16:16:17.156 INFO:tasks.workunit.client.0.smithi100.stdout: 4 2024-04-07T16:16:17.157 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:17.157 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 0, 2024-04-07T16:16:17.157 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:17.157 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:17.157 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:17.157 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:17.157 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T16:16:17.157 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T16:16:17.157 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T16:16:17.157 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:17.157 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 7, 2024-04-07T16:16:17.157 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:17.157 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:17.157 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T16:16:17.158 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:17.158 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:17.158 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:17.158 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.636513+0000", 2024-04-07T16:16:17.158 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.032456+0000", 2024-04-07T16:16:17.158 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.032456+0000", 2024-04-07T16:16:17.158 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.033067+0000", 2024-04-07T16:16:17.158 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.636513+0000", 2024-04-07T16:16:17.158 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:17.158 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:17.158 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:17.158 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 60, 2024-04-07T16:16:17.158 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.636513+0000", 2024-04-07T16:16:17.158 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.636513+0000", 2024-04-07T16:16:17.159 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.636513+0000", 2024-04-07T16:16:17.159 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:17.159 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:17.159 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:17.159 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.636513+0000", 2024-04-07T16:16:17.159 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.636513+0000", 2024-04-07T16:16:17.159 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:17.159 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:17.159 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:17.159 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:17.159 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 59, 2024-04-07T16:16:17.159 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:17.159 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:17.159 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:17.159 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:17.159 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:17.160 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:17.160 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:17.160 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:17.160 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.8", 2024-04-07T16:16:17.160 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:17.160 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:17.160 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:17.160 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 20, 2024-04-07T16:16:17.160 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:17.160 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T23:02:09.025617+0000", 2024-04-07T16:16:17.160 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:17.160 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:17.160 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:17.160 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:17.160 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:17.161 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:17.161 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:17.161 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:17.161 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:17.161 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:17.161 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:17.161 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:17.161 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:17.161 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:17.161 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:17.161 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:17.161 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:17.161 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:17.161 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:17.161 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:17.162 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:17.162 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:17.162 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:17.162 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:17.162 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:17.162 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:17.162 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:17.162 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:17.162 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:17.162 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:17.162 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:17.162 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:17.162 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:17.162 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:17.163 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:17.163 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:17.163 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:17.163 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:17.163 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:17.163 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:17.163 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:17.163 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:17.163 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:17.163 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:17.163 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:17.163 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:17.163 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:17.163 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T16:16:17.163 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T16:16:17.164 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T16:16:17.164 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:17.164 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 7, 2024-04-07T16:16:17.164 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:17.164 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:17.164 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:17.164 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:17.164 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T16:16:17.164 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T16:16:17.164 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T16:16:17.164 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:17.164 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 4, 2024-04-07T16:16:17.164 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:17.164 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:17.164 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T16:16:17.164 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:17.165 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:17.165 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:17.165 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:39.824230+0000", 2024-04-07T16:16:17.165 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.404321+0000", 2024-04-07T16:16:17.165 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.404321+0000", 2024-04-07T16:16:17.165 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.404632+0000", 2024-04-07T16:16:17.165 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:39.824230+0000", 2024-04-07T16:16:17.165 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:17.165 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:17.165 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:17.165 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 61, 2024-04-07T16:16:17.165 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:39.824230+0000", 2024-04-07T16:16:17.165 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:39.824230+0000", 2024-04-07T16:16:17.165 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:39.824230+0000", 2024-04-07T16:16:17.165 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:17.166 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:17.166 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:17.166 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:39.824230+0000", 2024-04-07T16:16:17.166 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:39.824230+0000", 2024-04-07T16:16:17.166 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:17.166 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:17.166 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:17.166 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:17.166 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 60, 2024-04-07T16:16:17.166 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:17.166 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:17.166 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:17.166 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:17.166 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:17.167 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:17.167 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:17.167 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:17.167 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.9", 2024-04-07T16:16:17.167 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:17.167 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:17.167 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:17.167 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 19, 2024-04-07T16:16:17.167 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:17.167 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T19:53:50.339257+0000", 2024-04-07T16:16:17.167 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:17.167 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:17.167 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:17.167 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:17.167 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:17.168 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:17.168 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:17.168 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:17.168 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:17.168 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:17.168 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:17.168 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:17.168 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:17.168 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:17.168 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:17.168 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:17.168 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:17.168 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:17.168 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:17.168 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:17.168 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:17.169 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:17.169 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:17.169 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:17.169 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:17.169 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:17.169 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:17.169 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:17.169 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:17.169 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:17.169 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:17.169 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:17.169 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:17.169 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:17.169 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:17.169 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:17.170 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:17.170 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:17.170 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:17.170 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:17.170 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:17.170 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:17.170 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:17.170 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:17.170 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:17.170 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:17.170 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:17.170 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T16:16:17.170 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T16:16:17.170 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T16:16:17.170 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:17.170 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 4, 2024-04-07T16:16:17.171 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:17.171 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:17.171 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:17.171 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:17.171 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T16:16:17.171 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T16:16:17.171 INFO:tasks.workunit.client.0.smithi100.stdout: 3 2024-04-07T16:16:17.171 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:17.171 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 1, 2024-04-07T16:16:17.171 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:17.171 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:17.171 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T16:16:17.171 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:17.171 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:17.171 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:17.171 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.236054+0000", 2024-04-07T16:16:17.172 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.037783+0000", 2024-04-07T16:16:17.172 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.037783+0000", 2024-04-07T16:16:17.172 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.038113+0000", 2024-04-07T16:16:17.172 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.236054+0000", 2024-04-07T16:16:17.172 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:17.172 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:17.172 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:17.172 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 60, 2024-04-07T16:16:17.172 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.236054+0000", 2024-04-07T16:16:17.172 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.236054+0000", 2024-04-07T16:16:17.172 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.236054+0000", 2024-04-07T16:16:17.172 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:17.172 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:17.172 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:17.172 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.236054+0000", 2024-04-07T16:16:17.173 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.236054+0000", 2024-04-07T16:16:17.173 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:17.173 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:17.173 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:17.173 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:17.173 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 59, 2024-04-07T16:16:17.173 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:17.173 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:17.173 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:17.173 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:17.173 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:17.173 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:17.173 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:17.173 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:17.173 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.9", 2024-04-07T16:16:17.174 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:17.174 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:17.174 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:17.174 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 20, 2024-04-07T16:16:17.174 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:17.174 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T00:58:32.085451+0000", 2024-04-07T16:16:17.174 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:17.174 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:17.174 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:17.174 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:17.174 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:17.174 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:17.174 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:17.174 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:17.174 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:17.175 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:17.175 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:17.175 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:17.175 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:17.175 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:17.175 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:17.175 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:17.175 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:17.175 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:17.175 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:17.175 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:17.175 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:17.175 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:17.175 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:17.175 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:17.175 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:17.176 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:17.176 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:17.176 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:17.176 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:17.176 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:17.176 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:17.176 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:17.176 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:17.176 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:17.176 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:17.176 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:17.176 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:17.176 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:17.176 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:17.177 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:17.177 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:17.177 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:17.177 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:17.177 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:17.177 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:17.177 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:17.177 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:17.177 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T16:16:17.177 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T16:16:17.177 INFO:tasks.workunit.client.0.smithi100.stdout: 3 2024-04-07T16:16:17.177 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:17.177 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 1, 2024-04-07T16:16:17.177 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:17.178 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:17.178 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:17.178 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:17.178 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T16:16:17.178 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T16:16:17.178 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T16:16:17.178 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:17.178 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 3, 2024-04-07T16:16:17.178 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:17.178 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:17.178 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T16:16:17.178 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:17.178 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:17.178 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:17.178 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.187183+0000", 2024-04-07T16:16:17.179 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.392016+0000", 2024-04-07T16:16:17.179 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.392016+0000", 2024-04-07T16:16:17.179 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.392396+0000", 2024-04-07T16:16:17.179 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.187183+0000", 2024-04-07T16:16:17.179 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:17.179 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:17.179 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:17.179 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 61, 2024-04-07T16:16:17.179 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.187183+0000", 2024-04-07T16:16:17.179 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.187183+0000", 2024-04-07T16:16:17.179 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.187183+0000", 2024-04-07T16:16:17.179 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:17.179 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:17.179 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:17.179 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.187183+0000", 2024-04-07T16:16:17.180 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.187183+0000", 2024-04-07T16:16:17.180 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:17.180 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:17.180 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:17.180 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:17.180 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 60, 2024-04-07T16:16:17.180 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:17.180 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:17.180 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:17.180 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:17.180 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:17.180 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:17.180 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:17.180 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:17.180 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.8", 2024-04-07T16:16:17.181 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:17.181 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:17.181 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:17.181 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 19, 2024-04-07T16:16:17.181 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:17.181 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T02:35:50.790051+0000", 2024-04-07T16:16:17.181 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:17.181 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:17.181 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:17.181 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:17.181 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:17.181 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:17.181 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:17.181 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:17.181 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:17.181 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:17.182 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:17.182 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:17.182 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:17.182 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:17.182 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:17.182 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:17.182 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:17.182 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:17.182 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:17.182 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:17.182 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:17.182 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:17.182 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:17.182 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:17.183 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:17.183 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:17.183 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:17.183 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:17.183 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:17.183 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:17.183 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:17.183 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:17.183 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:17.183 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:17.183 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:17.183 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:17.183 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:17.183 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:17.183 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:17.184 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:17.184 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:17.184 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:17.184 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:17.184 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:17.184 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:17.184 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:17.184 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:17.184 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T16:16:17.184 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T16:16:17.184 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T16:16:17.184 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:17.184 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 3, 2024-04-07T16:16:17.184 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:17.184 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:17.185 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:17.185 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:17.185 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T16:16:17.185 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T16:16:17.185 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T16:16:17.185 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:17.185 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 1, 2024-04-07T16:16:17.185 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:17.185 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:17.185 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T16:16:17.185 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:17.185 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:17.185 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:17.185 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.237136+0000", 2024-04-07T16:16:17.185 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.038110+0000", 2024-04-07T16:16:17.185 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.038110+0000", 2024-04-07T16:16:17.186 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.038401+0000", 2024-04-07T16:16:17.186 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.237136+0000", 2024-04-07T16:16:17.186 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:17.186 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:17.186 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:17.186 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 60, 2024-04-07T16:16:17.186 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.237136+0000", 2024-04-07T16:16:17.186 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.237136+0000", 2024-04-07T16:16:17.186 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.237136+0000", 2024-04-07T16:16:17.186 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:17.186 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:17.186 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:17.186 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.237136+0000", 2024-04-07T16:16:17.186 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.237136+0000", 2024-04-07T16:16:17.187 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:17.187 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:17.187 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:17.187 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:17.187 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 59, 2024-04-07T16:16:17.187 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:17.187 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:17.187 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:17.187 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:17.187 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:17.187 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:17.187 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:17.187 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:17.188 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.a", 2024-04-07T16:16:17.188 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:17.188 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:17.188 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:17.188 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 20, 2024-04-07T16:16:17.188 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:17.188 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T01:16:50.295588+0000", 2024-04-07T16:16:17.188 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:17.188 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:17.188 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:17.188 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:17.188 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:17.188 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:17.188 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:17.188 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:17.189 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:17.189 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:17.189 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:17.189 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:17.189 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:17.189 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:17.189 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:17.189 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:17.189 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:17.189 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:17.189 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:17.189 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:17.189 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:17.189 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:17.189 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:17.189 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:17.190 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:17.190 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:17.190 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:17.190 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:17.190 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:17.190 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:17.190 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:17.190 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:17.190 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:17.190 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:17.190 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:17.190 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:17.190 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:17.190 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:17.190 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:17.191 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:17.191 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:17.191 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:17.191 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:17.191 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:17.191 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:17.191 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:17.191 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:17.191 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T16:16:17.191 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T16:16:17.191 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T16:16:17.191 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:17.191 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 1, 2024-04-07T16:16:17.191 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:17.191 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:17.192 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:17.192 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:17.192 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T16:16:17.192 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T16:16:17.192 INFO:tasks.workunit.client.0.smithi100.stdout: 4 2024-04-07T16:16:17.192 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:17.192 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 3, 2024-04-07T16:16:17.192 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:17.192 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:17.192 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T16:16:17.192 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:17.192 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:17.192 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:17.192 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.186955+0000", 2024-04-07T16:16:17.192 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.397593+0000", 2024-04-07T16:16:17.193 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.397593+0000", 2024-04-07T16:16:17.193 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.397819+0000", 2024-04-07T16:16:17.193 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.186955+0000", 2024-04-07T16:16:17.193 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:17.193 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:17.193 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:17.193 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 61, 2024-04-07T16:16:17.193 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.186955+0000", 2024-04-07T16:16:17.193 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.186955+0000", 2024-04-07T16:16:17.193 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.186955+0000", 2024-04-07T16:16:17.193 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:17.193 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:17.193 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:17.193 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.186955+0000", 2024-04-07T16:16:17.193 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.186955+0000", 2024-04-07T16:16:17.193 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:17.194 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:17.194 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:17.194 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:17.194 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 60, 2024-04-07T16:16:17.194 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:17.194 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:17.194 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:17.194 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:17.194 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:17.194 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:17.194 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:17.194 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:17.194 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.b", 2024-04-07T16:16:17.194 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:17.195 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:17.195 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:17.195 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 19, 2024-04-07T16:16:17.195 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:17.195 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T02:30:01.472030+0000", 2024-04-07T16:16:17.195 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:17.195 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:17.195 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:17.195 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:17.195 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:17.195 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:17.195 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:17.195 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:17.195 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:17.195 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:17.196 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:17.196 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:17.196 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:17.196 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:17.196 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:17.196 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:17.196 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:17.196 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:17.196 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:17.196 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:17.196 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:17.196 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:17.196 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:17.196 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:17.196 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:17.196 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:17.197 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:17.197 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:17.197 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:17.197 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:17.197 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:17.197 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:17.197 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:17.197 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:17.197 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:17.197 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:17.197 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:17.197 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:17.197 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:17.197 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:17.197 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:17.198 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:17.198 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:17.198 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:17.198 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:17.198 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:17.198 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:17.198 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T16:16:17.198 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T16:16:17.198 INFO:tasks.workunit.client.0.smithi100.stdout: 4 2024-04-07T16:16:17.198 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:17.198 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 3, 2024-04-07T16:16:17.198 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:17.198 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:17.198 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:17.198 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:17.199 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T16:16:17.199 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T16:16:17.199 INFO:tasks.workunit.client.0.smithi100.stdout: 5 2024-04-07T16:16:17.199 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:17.199 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 7, 2024-04-07T16:16:17.199 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:17.199 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:17.199 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T16:16:17.199 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:17.199 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:17.199 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:17.199 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.636990+0000", 2024-04-07T16:16:17.199 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.041873+0000", 2024-04-07T16:16:17.199 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.041873+0000", 2024-04-07T16:16:17.200 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.043982+0000", 2024-04-07T16:16:17.200 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.636990+0000", 2024-04-07T16:16:17.200 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:17.200 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:17.200 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:17.200 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 60, 2024-04-07T16:16:17.200 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.636990+0000", 2024-04-07T16:16:17.200 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.636990+0000", 2024-04-07T16:16:17.200 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.636990+0000", 2024-04-07T16:16:17.200 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:17.200 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:17.200 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:17.200 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.636990+0000", 2024-04-07T16:16:17.200 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.636990+0000", 2024-04-07T16:16:17.201 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:17.201 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 2, 2024-04-07T16:16:17.201 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:17.201 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:17.201 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 59, 2024-04-07T16:16:17.201 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:17.201 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:17.201 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:17.201 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:17.201 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 2, 2024-04-07T16:16:17.201 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:17.201 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:17.201 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:17.201 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.b", 2024-04-07T16:16:17.201 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:17.202 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:17.202 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:17.202 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 22, 2024-04-07T16:16:17.202 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:17.202 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T20:03:35.575133+0000", 2024-04-07T16:16:17.202 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:17.202 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:17.202 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:17.202 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:17.202 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:17.202 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:17.202 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:17.202 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:17.202 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:17.202 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:17.203 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:17.203 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:17.203 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:17.203 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:17.203 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:17.203 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:17.203 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:17.203 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:17.203 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:17.203 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 3, 2024-04-07T16:16:17.203 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 1, 2024-04-07T16:16:17.203 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:17.203 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 1, 2024-04-07T16:16:17.203 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:17.203 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:17.204 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:17.204 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:17.204 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:17.204 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 1, 2024-04-07T16:16:17.204 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:17.204 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:17.204 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:17.204 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:17.204 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:17.204 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:17.204 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:17.204 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:17.204 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:17.204 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:17.205 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:17.205 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:17.205 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 1, 2024-04-07T16:16:17.205 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:17.205 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:17.205 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:17.205 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:17.205 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:17.205 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T16:16:17.205 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T16:16:17.205 INFO:tasks.workunit.client.0.smithi100.stdout: 5 2024-04-07T16:16:17.205 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:17.205 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 7, 2024-04-07T16:16:17.205 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "61'2" 2024-04-07T16:16:17.205 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:17.206 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:17.206 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:17.206 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T16:16:17.206 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T16:16:17.206 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T16:16:17.206 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:17.206 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 6, 2024-04-07T16:16:17.206 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:17.206 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:17.206 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T16:16:17.206 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:17.206 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:17.206 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:17.207 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.526346+0000", 2024-04-07T16:16:17.207 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.381470+0000", 2024-04-07T16:16:17.207 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.381470+0000", 2024-04-07T16:16:17.207 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.381810+0000", 2024-04-07T16:16:17.207 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.526346+0000", 2024-04-07T16:16:17.207 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:17.207 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:17.207 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:17.207 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 61, 2024-04-07T16:16:17.207 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.526346+0000", 2024-04-07T16:16:17.207 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.526346+0000", 2024-04-07T16:16:17.207 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.526346+0000", 2024-04-07T16:16:17.207 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:17.207 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:17.208 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:17.208 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.526346+0000", 2024-04-07T16:16:17.208 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.526346+0000", 2024-04-07T16:16:17.208 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:17.208 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:17.208 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:17.208 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:17.208 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 60, 2024-04-07T16:16:17.208 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:17.208 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:17.208 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:17.208 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:17.208 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:17.208 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:17.208 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:17.209 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:17.209 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.a", 2024-04-07T16:16:17.209 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:17.209 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:17.209 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:17.209 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 19, 2024-04-07T16:16:17.209 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:17.209 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T17:26:21.450939+0000", 2024-04-07T16:16:17.209 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:17.209 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:17.209 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:17.209 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:17.209 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:17.209 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:17.209 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:17.210 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:17.210 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:17.210 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:17.210 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:17.210 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:17.210 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:17.210 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:17.210 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:17.210 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:17.210 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:17.210 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:17.210 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:17.210 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:17.210 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:17.210 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:17.211 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:17.211 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:17.211 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:17.211 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:17.211 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:17.211 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:17.211 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:17.211 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:17.211 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:17.211 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:17.211 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:17.211 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:17.211 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:17.211 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:17.211 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:17.212 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:17.212 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:17.212 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:17.212 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:17.212 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:17.212 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:17.212 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:17.212 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:17.212 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:17.212 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:17.212 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T16:16:17.212 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T16:16:17.212 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T16:16:17.212 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:17.212 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 6, 2024-04-07T16:16:17.212 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:17.213 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:17.213 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:17.213 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:17.213 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T16:16:17.213 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T16:16:17.213 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T16:16:17.213 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:17.213 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 2, 2024-04-07T16:16:17.213 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:17.213 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:17.213 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T16:16:17.213 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:17.213 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:17.213 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:17.213 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.764853+0000", 2024-04-07T16:16:17.214 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.037652+0000", 2024-04-07T16:16:17.214 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.037652+0000", 2024-04-07T16:16:17.214 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.038191+0000", 2024-04-07T16:16:17.214 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.764853+0000", 2024-04-07T16:16:17.214 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:17.214 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:17.214 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:17.214 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 60, 2024-04-07T16:16:17.214 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.764853+0000", 2024-04-07T16:16:17.214 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.764853+0000", 2024-04-07T16:16:17.214 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.764853+0000", 2024-04-07T16:16:17.214 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:17.214 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:17.214 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:17.215 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.764853+0000", 2024-04-07T16:16:17.215 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.764853+0000", 2024-04-07T16:16:17.215 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:17.215 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:17.215 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:17.215 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:17.215 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 59, 2024-04-07T16:16:17.215 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:17.215 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:17.215 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:17.215 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:17.215 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:17.215 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:17.215 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:17.215 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:17.216 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.c", 2024-04-07T16:16:17.216 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:17.216 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:17.216 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:17.216 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 20, 2024-04-07T16:16:17.216 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:17.216 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T17:04:59.525025+0000", 2024-04-07T16:16:17.216 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:17.216 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:17.216 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:17.216 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:17.216 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:17.216 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:17.216 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:17.216 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:17.217 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:17.217 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:17.217 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:17.217 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:17.217 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:17.217 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:17.217 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:17.217 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:17.217 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:17.217 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:17.217 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:17.217 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:17.217 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:17.217 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:17.217 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:17.218 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:17.218 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:17.218 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:17.218 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:17.218 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:17.218 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:17.218 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:17.218 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:17.218 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:17.218 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:17.218 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:17.218 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:17.218 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:17.218 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:17.219 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:17.219 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:17.219 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:17.219 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:17.219 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:17.219 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:17.219 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:17.219 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:17.219 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:17.219 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:17.219 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T16:16:17.219 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T16:16:17.219 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T16:16:17.219 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:17.219 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 2, 2024-04-07T16:16:17.220 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:17.220 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:17.220 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:17.220 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:17.220 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T16:16:17.220 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T16:16:17.220 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T16:16:17.220 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:17.220 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 7, 2024-04-07T16:16:17.220 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:17.220 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:17.220 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T16:16:17.220 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:17.220 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:17.220 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:17.221 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.637269+0000", 2024-04-07T16:16:17.221 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.400353+0000", 2024-04-07T16:16:17.221 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.400353+0000", 2024-04-07T16:16:17.221 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.400660+0000", 2024-04-07T16:16:17.221 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.637269+0000", 2024-04-07T16:16:17.221 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:17.221 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:17.221 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:17.221 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 61, 2024-04-07T16:16:17.221 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.637269+0000", 2024-04-07T16:16:17.221 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.637269+0000", 2024-04-07T16:16:17.221 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.637269+0000", 2024-04-07T16:16:17.221 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:17.221 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:17.221 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:17.222 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.637269+0000", 2024-04-07T16:16:17.222 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.637269+0000", 2024-04-07T16:16:17.222 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:17.222 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:17.222 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:17.222 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:17.222 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 60, 2024-04-07T16:16:17.222 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:17.222 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:17.222 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:17.222 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:17.222 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:17.222 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:17.222 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:17.222 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:17.223 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.d", 2024-04-07T16:16:17.223 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:17.223 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:17.223 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:17.223 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 19, 2024-04-07T16:16:17.223 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:17.223 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T00:47:57.573097+0000", 2024-04-07T16:16:17.223 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:17.223 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:17.223 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:17.223 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:17.223 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:17.223 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:17.223 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:17.223 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:17.224 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:17.224 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:17.224 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:17.224 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:17.224 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:17.224 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:17.224 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:17.224 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:17.224 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:17.224 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:17.224 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:17.224 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:17.224 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:17.224 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:17.224 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:17.224 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:17.225 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:17.225 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:17.225 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:17.225 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:17.225 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:17.225 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:17.225 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:17.225 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:17.225 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:17.225 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:17.225 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:17.225 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:17.225 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:17.225 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:17.225 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:17.226 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:17.226 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:17.226 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:17.226 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:17.226 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:17.226 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:17.226 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:17.226 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:17.226 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T16:16:17.226 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T16:16:17.226 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T16:16:17.226 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:17.226 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 7, 2024-04-07T16:16:17.226 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:17.227 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:17.227 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:17.227 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:17.227 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T16:16:17.227 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T16:16:17.227 INFO:tasks.workunit.client.0.smithi100.stdout: 3 2024-04-07T16:16:17.227 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:17.227 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 1, 2024-04-07T16:16:17.227 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:17.227 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:17.227 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T16:16:17.227 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:17.227 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:17.227 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:17.227 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.237152+0000", 2024-04-07T16:16:17.228 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.034455+0000", 2024-04-07T16:16:17.228 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.034455+0000", 2024-04-07T16:16:17.228 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.034772+0000", 2024-04-07T16:16:17.228 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.237152+0000", 2024-04-07T16:16:17.228 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:17.228 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:17.228 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:17.228 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 60, 2024-04-07T16:16:17.228 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.237152+0000", 2024-04-07T16:16:17.228 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.237152+0000", 2024-04-07T16:16:17.228 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.237152+0000", 2024-04-07T16:16:17.228 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:17.228 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:17.228 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:17.229 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.237152+0000", 2024-04-07T16:16:17.229 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.237152+0000", 2024-04-07T16:16:17.229 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:17.229 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 1, 2024-04-07T16:16:17.229 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:17.229 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:17.229 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 59, 2024-04-07T16:16:17.229 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:17.229 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:17.229 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:17.229 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:17.229 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 1, 2024-04-07T16:16:17.229 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:17.229 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:17.229 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:17.229 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.d", 2024-04-07T16:16:17.230 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:17.230 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:17.230 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:17.230 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 21, 2024-04-07T16:16:17.230 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:17.230 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T23:10:30.557880+0000", 2024-04-07T16:16:17.230 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:17.230 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:17.230 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:17.230 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 70, 2024-04-07T16:16:17.230 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:17.230 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:17.230 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:17.230 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:17.231 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:17.231 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:17.231 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:17.231 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:17.231 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:17.231 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:17.231 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:17.231 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:17.231 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:17.231 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:17.231 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:17.231 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 3, 2024-04-07T16:16:17.231 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 1, 2024-04-07T16:16:17.231 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:17.232 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 1, 2024-04-07T16:16:17.232 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:17.232 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:17.232 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:17.232 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:17.232 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:17.232 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:17.232 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:17.232 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:17.232 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:17.232 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:17.232 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:17.232 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:17.232 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:17.232 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:17.233 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:17.233 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:17.233 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:17.233 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:17.233 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 1, 2024-04-07T16:16:17.233 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 1 2024-04-07T16:16:17.233 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:17.233 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:17.233 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:17.233 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:17.233 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T16:16:17.233 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T16:16:17.233 INFO:tasks.workunit.client.0.smithi100.stdout: 3 2024-04-07T16:16:17.233 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:17.233 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 1, 2024-04-07T16:16:17.234 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "61'1" 2024-04-07T16:16:17.234 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:17.234 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:17.234 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:17.234 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T16:16:17.234 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T16:16:17.234 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T16:16:17.234 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:17.234 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 5, 2024-04-07T16:16:17.234 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:17.234 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:17.234 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T16:16:17.234 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:17.234 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:17.235 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:17.235 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.025357+0000", 2024-04-07T16:16:17.235 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.398120+0000", 2024-04-07T16:16:17.235 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.398120+0000", 2024-04-07T16:16:17.235 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.400762+0000", 2024-04-07T16:16:17.235 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.025357+0000", 2024-04-07T16:16:17.235 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:17.235 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:17.235 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:17.235 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 61, 2024-04-07T16:16:17.235 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.025357+0000", 2024-04-07T16:16:17.235 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.025357+0000", 2024-04-07T16:16:17.235 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.025357+0000", 2024-04-07T16:16:17.235 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:17.235 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:17.236 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:17.236 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.025357+0000", 2024-04-07T16:16:17.236 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.025357+0000", 2024-04-07T16:16:17.236 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:17.236 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:17.236 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:17.236 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:17.236 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 60, 2024-04-07T16:16:17.236 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:17.236 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:17.236 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:17.236 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:17.236 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:17.236 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:17.236 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:17.237 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:17.237 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.c", 2024-04-07T16:16:17.237 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:17.237 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:17.237 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:17.237 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 19, 2024-04-07T16:16:17.237 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:17.237 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T22:11:21.789998+0000", 2024-04-07T16:16:17.237 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:17.237 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:17.237 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:17.237 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:17.237 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:17.237 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:17.237 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:17.238 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:17.238 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:17.238 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:17.238 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:17.238 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:17.238 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:17.238 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:17.238 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:17.238 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:17.238 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:17.238 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:17.238 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:17.238 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:17.238 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:17.238 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:17.239 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:17.239 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:17.239 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:17.239 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:17.239 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:17.239 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:17.239 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:17.239 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:17.239 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:17.239 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:17.239 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:17.239 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:17.239 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:17.239 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:17.240 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:17.240 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:17.240 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:17.240 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:17.240 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:17.240 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:17.240 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:17.240 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:17.240 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:17.240 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:17.240 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:17.240 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T16:16:17.240 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T16:16:17.240 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T16:16:17.240 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:17.241 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 5, 2024-04-07T16:16:17.241 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:17.241 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:17.241 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:17.241 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:17.241 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T16:16:17.241 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T16:16:17.241 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T16:16:17.241 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:17.241 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 2, 2024-04-07T16:16:17.241 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:17.241 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:17.241 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T16:16:17.241 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:17.241 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:17.242 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:17.242 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.764862+0000", 2024-04-07T16:16:17.242 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.039797+0000", 2024-04-07T16:16:17.242 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.039797+0000", 2024-04-07T16:16:17.242 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.040101+0000", 2024-04-07T16:16:17.242 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.764862+0000", 2024-04-07T16:16:17.242 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:17.242 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:17.242 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:17.242 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 60, 2024-04-07T16:16:17.242 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.764862+0000", 2024-04-07T16:16:17.242 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.764862+0000", 2024-04-07T16:16:17.242 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.764862+0000", 2024-04-07T16:16:17.242 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:17.242 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:17.243 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T16:16:17.243 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.764862+0000", 2024-04-07T16:16:17.243 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.764862+0000", 2024-04-07T16:16:17.243 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:17.243 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 2, 2024-04-07T16:16:17.243 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:17.243 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:17.243 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 59, 2024-04-07T16:16:17.243 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:17.243 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:17.243 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:17.243 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:17.243 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 2, 2024-04-07T16:16:17.243 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:17.244 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:17.244 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:17.244 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.e", 2024-04-07T16:16:17.244 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:17.244 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:17.244 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:17.244 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 22, 2024-04-07T16:16:17.244 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:17.244 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T03:27:06.016325+0000", 2024-04-07T16:16:17.244 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:17.244 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:17.244 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:17.244 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:17.244 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:17.244 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:17.244 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:17.245 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:17.245 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:17.245 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:17.245 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:17.245 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:17.245 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:17.245 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:17.245 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:17.245 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:17.245 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:17.245 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:17.245 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:17.245 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 3, 2024-04-07T16:16:17.245 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 1, 2024-04-07T16:16:17.245 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:17.246 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 1, 2024-04-07T16:16:17.246 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:17.246 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:17.246 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:17.246 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:17.246 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:17.246 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 1, 2024-04-07T16:16:17.246 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:17.246 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:17.246 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:17.246 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:17.246 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:17.246 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:17.246 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:17.247 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:17.247 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:17.247 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:17.247 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:17.247 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:17.247 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 3, 2024-04-07T16:16:17.247 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:17.247 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:17.247 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:17.247 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:17.247 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:17.247 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T16:16:17.247 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T16:16:17.247 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T16:16:17.247 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:17.247 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 2, 2024-04-07T16:16:17.248 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "61'2" 2024-04-07T16:16:17.248 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:17.248 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:17.248 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:17.248 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T16:16:17.248 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T16:16:17.248 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T16:16:17.248 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:17.248 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 7, 2024-04-07T16:16:17.248 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:17.248 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:17.248 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T16:16:17.248 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:17.248 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:17.248 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:17.249 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.637921+0000", 2024-04-07T16:16:17.249 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.394090+0000", 2024-04-07T16:16:17.249 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.394090+0000", 2024-04-07T16:16:17.249 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.394720+0000", 2024-04-07T16:16:17.249 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.637921+0000", 2024-04-07T16:16:17.249 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:17.249 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:17.249 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:17.249 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 61, 2024-04-07T16:16:17.249 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.637921+0000", 2024-04-07T16:16:17.249 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.637921+0000", 2024-04-07T16:16:17.249 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.637921+0000", 2024-04-07T16:16:17.249 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:17.249 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:17.249 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:17.250 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.637921+0000", 2024-04-07T16:16:17.250 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.637921+0000", 2024-04-07T16:16:17.250 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:17.250 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:17.250 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:17.250 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:17.250 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 60, 2024-04-07T16:16:17.250 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:17.250 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:17.250 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:17.250 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:17.250 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:17.250 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:17.250 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:17.250 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:17.251 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.f", 2024-04-07T16:16:17.251 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:17.251 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:17.251 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:17.251 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 19, 2024-04-07T16:16:17.251 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:17.251 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T00:47:54.579168+0000", 2024-04-07T16:16:17.251 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:17.251 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:17.251 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:17.251 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:17.251 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:17.251 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:17.251 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:17.252 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:17.252 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:17.252 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:17.252 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:17.252 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:17.252 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:17.252 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:17.252 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:17.252 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:17.252 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:17.252 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:17.252 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:17.252 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:17.252 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:17.252 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:17.253 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:17.253 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:17.253 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:17.253 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:17.253 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:17.253 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:17.253 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:17.253 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:17.253 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:17.253 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:17.253 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:17.253 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:17.253 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:17.253 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:17.253 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:17.254 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:17.254 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:17.254 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:17.254 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:17.254 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:17.254 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:17.254 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:17.254 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:17.254 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:17.254 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:17.254 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T16:16:17.254 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T16:16:17.254 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T16:16:17.254 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:17.254 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 7, 2024-04-07T16:16:17.255 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:17.255 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:17.255 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:17.255 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:17.255 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T16:16:17.255 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T16:16:17.255 INFO:tasks.workunit.client.0.smithi100.stdout: 5 2024-04-07T16:16:17.255 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:17.255 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 6, 2024-04-07T16:16:17.255 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:17.255 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:17.255 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T16:16:17.255 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:17.255 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:17.256 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:17.256 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.525708+0000", 2024-04-07T16:16:17.256 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.395978+0000", 2024-04-07T16:16:17.256 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.395978+0000", 2024-04-07T16:16:17.256 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.396725+0000", 2024-04-07T16:16:17.256 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.525708+0000", 2024-04-07T16:16:17.256 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:17.256 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:17.256 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:17.256 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 61, 2024-04-07T16:16:17.256 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.525708+0000", 2024-04-07T16:16:17.256 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.525708+0000", 2024-04-07T16:16:17.256 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.525708+0000", 2024-04-07T16:16:17.256 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:17.256 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:17.256 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:17.257 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.525708+0000", 2024-04-07T16:16:17.257 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.525708+0000", 2024-04-07T16:16:17.257 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:17.257 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:17.257 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:17.257 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:17.257 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 60, 2024-04-07T16:16:17.257 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:17.257 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:17.257 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:17.257 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:17.257 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:17.257 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:17.257 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:17.257 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:17.258 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.20", 2024-04-07T16:16:17.258 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:17.258 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:17.258 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:17.258 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 19, 2024-04-07T16:16:17.258 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:17.258 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T21:30:43.276569+0000", 2024-04-07T16:16:17.258 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:17.258 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:17.258 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:17.258 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:17.258 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:17.258 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:17.258 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:17.259 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:17.259 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:17.259 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:17.259 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:17.259 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:17.259 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:17.259 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:17.259 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:17.259 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:17.259 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:17.259 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:17.259 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:17.259 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:17.259 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:17.259 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:17.260 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:17.260 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:17.260 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:17.260 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:17.260 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:17.260 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:17.260 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:17.260 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:17.260 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:17.260 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:17.260 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:17.260 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:17.260 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:17.260 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:17.260 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:17.260 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:17.261 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:17.261 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:17.261 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:17.261 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:17.261 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:17.261 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:17.261 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:17.261 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:17.261 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:17.261 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T16:16:17.261 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T16:16:17.261 INFO:tasks.workunit.client.0.smithi100.stdout: 5 2024-04-07T16:16:17.261 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:17.261 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 6, 2024-04-07T16:16:17.261 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:17.261 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:17.262 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:17.262 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:17.262 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T16:16:17.262 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T16:16:17.262 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T16:16:17.262 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:17.262 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 2, 2024-04-07T16:16:17.262 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:17.262 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:17.262 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T16:16:17.262 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:17.262 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:17.262 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:17.262 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.764525+0000", 2024-04-07T16:16:17.263 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.410814+0000", 2024-04-07T16:16:17.263 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.410814+0000", 2024-04-07T16:16:17.263 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.412091+0000", 2024-04-07T16:16:17.263 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.764525+0000", 2024-04-07T16:16:17.263 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:17.263 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:17.263 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:17.263 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 61, 2024-04-07T16:16:17.263 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.764525+0000", 2024-04-07T16:16:17.263 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.764525+0000", 2024-04-07T16:16:17.263 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.764525+0000", 2024-04-07T16:16:17.263 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:17.263 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:17.263 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:17.263 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.764525+0000", 2024-04-07T16:16:17.263 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.764525+0000", 2024-04-07T16:16:17.264 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:17.264 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:17.264 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:17.264 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:17.264 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 60, 2024-04-07T16:16:17.264 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:17.264 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:17.264 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:17.264 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:17.264 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:17.264 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:17.264 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:17.264 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:17.264 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.21", 2024-04-07T16:16:17.265 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:17.265 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:17.265 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:17.265 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 19, 2024-04-07T16:16:17.265 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:17.265 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T01:48:29.034291+0000", 2024-04-07T16:16:17.265 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:17.265 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:17.265 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:17.265 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:17.265 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:17.265 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:17.265 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:17.265 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:17.265 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:17.266 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:17.266 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:17.266 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:17.266 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:17.266 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:17.266 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:17.266 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:17.266 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:17.266 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:17.266 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:17.266 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:17.266 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:17.266 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:17.266 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:17.266 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:17.267 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:17.267 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:17.267 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:17.267 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:17.267 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:17.267 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:17.267 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:17.267 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:17.267 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:17.267 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:17.267 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:17.267 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:17.267 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:17.267 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:17.267 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:17.268 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:17.268 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:17.268 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:17.268 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:17.268 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:17.268 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:17.268 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:17.268 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:17.268 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T16:16:17.268 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T16:16:17.268 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T16:16:17.268 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:17.268 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 2, 2024-04-07T16:16:17.268 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:17.268 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:17.269 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:17.269 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:17.269 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T16:16:17.269 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T16:16:17.269 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T16:16:17.269 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:17.269 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 5, 2024-04-07T16:16:17.269 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:17.269 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:17.269 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T16:16:17.269 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:17.269 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:17.269 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:17.269 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.024540+0000", 2024-04-07T16:16:17.269 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.399206+0000", 2024-04-07T16:16:17.270 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.399206+0000", 2024-04-07T16:16:17.270 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.400651+0000", 2024-04-07T16:16:17.270 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.024540+0000", 2024-04-07T16:16:17.270 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:17.270 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:17.270 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:17.270 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 61, 2024-04-07T16:16:17.270 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.024540+0000", 2024-04-07T16:16:17.270 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.024540+0000", 2024-04-07T16:16:17.270 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.024540+0000", 2024-04-07T16:16:17.270 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:17.270 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:17.270 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:17.270 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.024540+0000", 2024-04-07T16:16:17.270 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.024540+0000", 2024-04-07T16:16:17.271 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:17.271 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:17.271 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:17.271 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:17.271 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 60, 2024-04-07T16:16:17.271 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:17.271 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:17.271 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:17.271 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:17.271 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:17.271 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:17.271 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:17.271 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:17.271 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.22", 2024-04-07T16:16:17.271 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:17.272 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:17.272 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:17.272 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 19, 2024-04-07T16:16:17.272 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:17.272 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T22:16:50.256754+0000", 2024-04-07T16:16:17.272 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:17.272 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:17.272 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:17.272 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:17.272 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:17.272 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:17.272 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:17.272 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:17.272 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:17.272 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:17.273 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:17.273 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:17.273 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:17.273 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:17.273 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:17.273 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:17.273 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:17.273 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:17.273 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:17.273 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:17.273 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:17.273 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:17.273 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:17.273 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:17.273 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:17.274 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:17.274 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:17.274 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:17.274 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:17.274 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:17.274 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:17.274 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:17.274 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:17.274 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:17.274 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:17.274 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:17.274 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:17.274 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:17.274 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:17.274 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:17.275 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:17.275 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:17.275 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:17.275 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:17.275 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:17.275 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:17.275 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:17.275 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T16:16:17.275 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T16:16:17.275 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T16:16:17.275 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:17.275 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 5, 2024-04-07T16:16:17.275 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:17.275 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:17.275 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:17.276 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:17.276 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T16:16:17.276 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T16:16:17.276 INFO:tasks.workunit.client.0.smithi100.stdout: 4 2024-04-07T16:16:17.276 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:17.276 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 0, 2024-04-07T16:16:17.276 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:17.276 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:17.276 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T16:16:17.276 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:17.276 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:17.276 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:17.276 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.029689+0000", 2024-04-07T16:16:17.276 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.405320+0000", 2024-04-07T16:16:17.276 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.405320+0000", 2024-04-07T16:16:17.277 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.405888+0000", 2024-04-07T16:16:17.277 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.029689+0000", 2024-04-07T16:16:17.277 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:17.277 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:17.277 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:17.277 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 61, 2024-04-07T16:16:17.277 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.029689+0000", 2024-04-07T16:16:17.277 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.029689+0000", 2024-04-07T16:16:17.277 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.029689+0000", 2024-04-07T16:16:17.277 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:17.277 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:17.277 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:17.277 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.029689+0000", 2024-04-07T16:16:17.277 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.029689+0000", 2024-04-07T16:16:17.278 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:17.278 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:17.278 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:17.278 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:17.278 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 60, 2024-04-07T16:16:17.278 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:17.278 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:17.278 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:17.278 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:17.278 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:17.278 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:17.278 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:17.278 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:17.278 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.23", 2024-04-07T16:16:17.279 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:17.279 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:17.279 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:17.279 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 19, 2024-04-07T16:16:17.279 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:17.279 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T22:07:50.421903+0000", 2024-04-07T16:16:17.279 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:17.279 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:17.279 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:17.279 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:17.279 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:17.279 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:17.279 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:17.279 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:17.279 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:17.280 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:17.280 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:17.280 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:17.280 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:17.280 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:17.280 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:17.280 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:17.280 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:17.280 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:17.280 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:17.280 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:17.280 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:17.280 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:17.280 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:17.280 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:17.281 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:17.281 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:17.281 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:17.281 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:17.281 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:17.281 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:17.281 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:17.281 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:17.281 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:17.281 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:17.281 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:17.281 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:17.281 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:17.281 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:17.281 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:17.282 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:17.282 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:17.282 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:17.282 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:17.282 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:17.282 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:17.282 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:17.282 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:17.282 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T16:16:17.282 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T16:16:17.282 INFO:tasks.workunit.client.0.smithi100.stdout: 4 2024-04-07T16:16:17.282 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:17.282 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 0, 2024-04-07T16:16:17.282 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:17.283 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:17.283 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:17.283 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:17.283 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T16:16:17.283 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T16:16:17.283 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T16:16:17.283 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:17.283 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 3, 2024-04-07T16:16:17.283 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:17.283 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:17.283 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T16:16:17.283 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:17.283 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:17.283 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:17.283 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.186832+0000", 2024-04-07T16:16:17.283 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.398259+0000", 2024-04-07T16:16:17.284 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.398259+0000", 2024-04-07T16:16:17.284 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.398730+0000", 2024-04-07T16:16:17.284 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.186832+0000", 2024-04-07T16:16:17.284 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:17.284 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:17.284 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:17.284 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 61, 2024-04-07T16:16:17.284 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.186832+0000", 2024-04-07T16:16:17.284 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.186832+0000", 2024-04-07T16:16:17.284 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.186832+0000", 2024-04-07T16:16:17.284 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:17.284 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:17.284 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:17.284 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.186832+0000", 2024-04-07T16:16:17.285 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.186832+0000", 2024-04-07T16:16:17.285 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:17.285 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:17.285 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:17.285 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:17.285 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 60, 2024-04-07T16:16:17.285 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:17.285 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:17.285 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:17.285 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:17.285 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:17.285 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:17.285 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:17.285 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:17.285 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.24", 2024-04-07T16:16:17.285 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:17.286 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:17.286 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:17.286 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 19, 2024-04-07T16:16:17.286 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:17.286 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T02:14:28.915631+0000", 2024-04-07T16:16:17.286 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:17.286 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:17.286 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:17.286 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:17.286 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:17.286 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:17.286 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:17.286 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:17.286 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:17.287 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:17.287 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:17.287 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:17.287 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:17.287 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:17.287 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:17.287 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:17.287 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:17.287 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:17.287 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:17.287 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:17.287 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:17.287 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:17.287 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:17.287 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:17.288 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:17.288 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:17.288 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:17.288 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:17.288 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:17.288 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:17.288 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:17.288 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:17.288 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:17.288 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:17.288 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:17.288 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:17.288 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:17.288 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:17.288 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:17.289 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:17.289 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:17.289 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:17.289 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:17.289 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:17.289 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:17.289 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:17.289 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:17.289 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T16:16:17.289 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T16:16:17.289 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T16:16:17.289 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:17.289 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 3, 2024-04-07T16:16:17.289 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:17.289 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:17.289 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:17.290 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:17.290 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T16:16:17.290 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T16:16:17.290 INFO:tasks.workunit.client.0.smithi100.stdout: 2 2024-04-07T16:16:17.290 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:17.290 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 1, 2024-04-07T16:16:17.290 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:17.290 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:17.290 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T16:16:17.290 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:17.290 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:17.290 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:17.290 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.236896+0000", 2024-04-07T16:16:17.290 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.390858+0000", 2024-04-07T16:16:17.291 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.390858+0000", 2024-04-07T16:16:17.291 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.391140+0000", 2024-04-07T16:16:17.291 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.236896+0000", 2024-04-07T16:16:17.291 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:17.291 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:17.291 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:17.291 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 61, 2024-04-07T16:16:17.291 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.236896+0000", 2024-04-07T16:16:17.291 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.236896+0000", 2024-04-07T16:16:17.291 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.236896+0000", 2024-04-07T16:16:17.291 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:17.291 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:17.291 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:17.291 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.236896+0000", 2024-04-07T16:16:17.291 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.236896+0000", 2024-04-07T16:16:17.292 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:17.292 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:17.292 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:17.292 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:17.292 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 60, 2024-04-07T16:16:17.292 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:17.292 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:17.292 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:17.292 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:17.292 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:17.292 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:17.292 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:17.292 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:17.292 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.25", 2024-04-07T16:16:17.292 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:17.293 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:17.293 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:17.293 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 19, 2024-04-07T16:16:17.293 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:17.293 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T22:44:20.569791+0000", 2024-04-07T16:16:17.293 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:17.293 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:17.293 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:17.293 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:17.293 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:17.293 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:17.293 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:17.293 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:17.293 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:17.293 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:17.293 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:17.294 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:17.294 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:17.294 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:17.294 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:17.294 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:17.294 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:17.294 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:17.294 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:17.294 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:17.294 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:17.294 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:17.294 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:17.294 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:17.294 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:17.295 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:17.295 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:17.295 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:17.295 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:17.295 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:17.295 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:17.295 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:17.295 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:17.295 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:17.295 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:17.295 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:17.295 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:17.295 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:17.295 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:17.295 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:17.296 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:17.296 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:17.296 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:17.296 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:17.296 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:17.296 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:17.296 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:17.296 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T16:16:17.296 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T16:16:17.296 INFO:tasks.workunit.client.0.smithi100.stdout: 2 2024-04-07T16:16:17.296 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:17.296 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 1, 2024-04-07T16:16:17.296 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:17.296 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:17.296 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:17.297 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:17.297 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T16:16:17.297 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T16:16:17.297 INFO:tasks.workunit.client.0.smithi100.stdout: 4 2024-04-07T16:16:17.297 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:17.297 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 6, 2024-04-07T16:16:17.297 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:17.297 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:17.297 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T16:16:17.297 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:17.297 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:17.297 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:17.297 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.553144+0000", 2024-04-07T16:16:17.297 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.382267+0000", 2024-04-07T16:16:17.297 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.382267+0000", 2024-04-07T16:16:17.298 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.382605+0000", 2024-04-07T16:16:17.298 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.553144+0000", 2024-04-07T16:16:17.298 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:17.298 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:17.298 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:17.298 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 61, 2024-04-07T16:16:17.298 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.553144+0000", 2024-04-07T16:16:17.298 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.553144+0000", 2024-04-07T16:16:17.298 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.553144+0000", 2024-04-07T16:16:17.298 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:17.298 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:17.298 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:17.298 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.553144+0000", 2024-04-07T16:16:17.298 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.553144+0000", 2024-04-07T16:16:17.298 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:17.299 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:17.299 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:17.299 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:17.299 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 60, 2024-04-07T16:16:17.299 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:17.299 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:17.299 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:17.299 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:17.299 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:17.299 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:17.299 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:17.299 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:17.299 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.26", 2024-04-07T16:16:17.299 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:17.299 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:17.300 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:17.300 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 19, 2024-04-07T16:16:17.300 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:17.300 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T20:14:03.942753+0000", 2024-04-07T16:16:17.300 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:17.300 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:17.300 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:17.300 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:17.300 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:17.300 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:17.300 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:17.300 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:17.300 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:17.300 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:17.300 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:17.300 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:17.301 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:17.301 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:17.301 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:17.301 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:17.301 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:17.301 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:17.301 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:17.301 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:17.301 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:17.301 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:17.301 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:17.301 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:17.301 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:17.301 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:17.301 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:17.301 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:17.302 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:17.302 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:17.302 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:17.302 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:17.302 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:17.302 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:17.302 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:17.302 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:17.302 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:17.302 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:17.302 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:17.302 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:17.303 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:17.303 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:17.303 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:17.303 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:17.303 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:17.303 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:17.303 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:17.303 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T16:16:17.303 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T16:16:17.303 INFO:tasks.workunit.client.0.smithi100.stdout: 4 2024-04-07T16:16:17.303 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:17.303 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 6, 2024-04-07T16:16:17.303 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:17.303 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:17.303 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:17.304 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:17.304 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T16:16:17.304 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T16:16:17.304 INFO:tasks.workunit.client.0.smithi100.stdout: 2 2024-04-07T16:16:17.304 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:17.304 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 3, 2024-04-07T16:16:17.304 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:17.304 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:17.304 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T16:16:17.304 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:17.304 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:17.304 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:17.304 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.187449+0000", 2024-04-07T16:16:17.304 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.404734+0000", 2024-04-07T16:16:17.304 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.404734+0000", 2024-04-07T16:16:17.305 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.405175+0000", 2024-04-07T16:16:17.305 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.187449+0000", 2024-04-07T16:16:17.305 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:17.305 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:17.305 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:17.305 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 61, 2024-04-07T16:16:17.305 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.187449+0000", 2024-04-07T16:16:17.305 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.187449+0000", 2024-04-07T16:16:17.305 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.187449+0000", 2024-04-07T16:16:17.305 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:17.305 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:17.305 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:17.305 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.187449+0000", 2024-04-07T16:16:17.305 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.187449+0000", 2024-04-07T16:16:17.305 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:17.306 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:17.306 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:17.306 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:17.306 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 60, 2024-04-07T16:16:17.306 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:17.306 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:17.306 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:17.306 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:17.306 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:17.306 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:17.306 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:17.306 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:17.306 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.27", 2024-04-07T16:16:17.306 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:17.307 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:17.307 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:17.307 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 19, 2024-04-07T16:16:17.307 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:17.307 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T17:55:23.469795+0000", 2024-04-07T16:16:17.307 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:17.307 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:17.307 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:17.307 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:17.307 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:17.307 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:17.307 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:17.307 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:17.307 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:17.307 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:17.308 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:17.308 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:17.308 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:17.308 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:17.308 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:17.308 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:17.308 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:17.308 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:17.308 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:17.308 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:17.308 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:17.308 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:17.308 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:17.308 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:17.308 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:17.309 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:17.309 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:17.309 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:17.309 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:17.309 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:17.309 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:17.309 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:17.309 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:17.309 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:17.309 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:17.309 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:17.309 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:17.309 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:17.309 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:17.309 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:17.310 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:17.310 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:17.310 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:17.310 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:17.310 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:17.310 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:17.310 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:17.310 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T16:16:17.310 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T16:16:17.310 INFO:tasks.workunit.client.0.smithi100.stdout: 2 2024-04-07T16:16:17.310 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:17.310 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 3, 2024-04-07T16:16:17.310 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:17.310 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:17.311 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:17.311 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:17.311 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T16:16:17.311 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T16:16:17.311 INFO:tasks.workunit.client.0.smithi100.stdout: 2 2024-04-07T16:16:17.311 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:17.311 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 5, 2024-04-07T16:16:17.311 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:17.311 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:17.311 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T16:16:17.311 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:17.311 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:17.311 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:17.311 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.024146+0000", 2024-04-07T16:16:17.311 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.395430+0000", 2024-04-07T16:16:17.312 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.395430+0000", 2024-04-07T16:16:17.312 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.397863+0000", 2024-04-07T16:16:17.312 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.024146+0000", 2024-04-07T16:16:17.312 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:17.312 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:17.312 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:17.312 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 61, 2024-04-07T16:16:17.312 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.024146+0000", 2024-04-07T16:16:17.312 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.024146+0000", 2024-04-07T16:16:17.312 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.024146+0000", 2024-04-07T16:16:17.312 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:17.312 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:17.312 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:17.312 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.024146+0000", 2024-04-07T16:16:17.312 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.024146+0000", 2024-04-07T16:16:17.312 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:17.313 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:17.313 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:17.313 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:17.313 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 60, 2024-04-07T16:16:17.313 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:17.313 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:17.313 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:17.313 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:17.313 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:17.313 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:17.313 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:17.313 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:17.313 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.28", 2024-04-07T16:16:17.313 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:17.314 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:17.314 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:17.314 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 19, 2024-04-07T16:16:17.314 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:17.314 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T18:20:22.678341+0000", 2024-04-07T16:16:17.314 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:17.314 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:17.314 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:17.314 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:17.314 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:17.314 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:17.314 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:17.314 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:17.314 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:17.314 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:17.315 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:17.315 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:17.315 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:17.315 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:17.315 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:17.315 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:17.315 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:17.315 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:17.315 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:17.315 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:17.315 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:17.315 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:17.315 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:17.315 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:17.315 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:17.316 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:17.316 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:17.316 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:17.316 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:17.316 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:17.316 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:17.316 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:17.316 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:17.316 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:17.316 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:17.316 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:17.316 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:17.316 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:17.316 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:17.316 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:17.317 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:17.317 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:17.317 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:17.317 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:17.317 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:17.317 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:17.317 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:17.317 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T16:16:17.317 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T16:16:17.317 INFO:tasks.workunit.client.0.smithi100.stdout: 2 2024-04-07T16:16:17.317 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:17.317 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 5, 2024-04-07T16:16:17.317 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:17.317 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:17.317 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:17.318 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:17.318 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T16:16:17.318 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T16:16:17.318 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T16:16:17.318 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:17.318 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 3, 2024-04-07T16:16:17.318 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:17.318 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:17.318 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T16:16:17.318 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:17.318 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:17.318 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:17.318 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.186791+0000", 2024-04-07T16:16:17.318 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.396766+0000", 2024-04-07T16:16:17.318 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.396766+0000", 2024-04-07T16:16:17.319 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.397638+0000", 2024-04-07T16:16:17.319 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.186791+0000", 2024-04-07T16:16:17.319 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:17.319 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:17.319 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:17.319 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 61, 2024-04-07T16:16:17.319 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.186791+0000", 2024-04-07T16:16:17.319 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.186791+0000", 2024-04-07T16:16:17.319 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.186791+0000", 2024-04-07T16:16:17.319 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:17.319 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:17.319 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:17.319 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.186791+0000", 2024-04-07T16:16:17.319 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.186791+0000", 2024-04-07T16:16:17.319 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:17.320 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:17.320 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:17.320 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:17.320 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 60, 2024-04-07T16:16:17.320 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:17.320 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:17.320 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:17.320 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:17.320 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:17.320 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:17.320 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:17.320 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:17.320 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.29", 2024-04-07T16:16:17.320 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:17.321 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:17.321 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:17.321 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 19, 2024-04-07T16:16:17.321 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:17.321 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T22:51:06.841718+0000", 2024-04-07T16:16:17.321 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:17.321 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:17.321 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:17.321 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:17.321 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:17.321 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:17.321 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:17.321 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:17.321 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:17.321 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:17.322 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:17.322 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:17.322 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:17.322 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:17.322 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:17.322 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:17.322 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:17.322 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:17.322 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:17.322 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:17.322 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:17.322 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:17.322 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:17.322 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:17.322 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:17.323 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:17.323 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:17.323 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:17.323 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:17.323 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:17.323 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:17.323 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:17.323 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:17.323 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:17.323 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:17.323 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:17.323 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:17.323 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:17.323 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:17.323 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:17.323 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:17.324 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:17.324 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:17.324 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:17.324 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:17.324 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:17.324 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:17.324 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T16:16:17.324 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T16:16:17.324 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T16:16:17.324 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:17.324 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 3, 2024-04-07T16:16:17.324 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:17.324 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:17.324 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:17.324 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:17.325 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T16:16:17.325 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T16:16:17.325 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T16:16:17.325 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:17.325 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 5, 2024-04-07T16:16:17.325 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:17.325 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:17.325 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T16:16:17.325 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:17.325 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:17.325 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:17.325 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.024630+0000", 2024-04-07T16:16:17.325 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.400042+0000", 2024-04-07T16:16:17.325 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.400042+0000", 2024-04-07T16:16:17.325 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.403603+0000", 2024-04-07T16:16:17.326 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.024630+0000", 2024-04-07T16:16:17.326 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:17.326 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:17.326 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:17.326 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 61, 2024-04-07T16:16:17.326 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.024630+0000", 2024-04-07T16:16:17.326 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.024630+0000", 2024-04-07T16:16:17.326 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.024630+0000", 2024-04-07T16:16:17.326 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:17.326 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:17.326 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:17.326 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.024630+0000", 2024-04-07T16:16:17.326 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.024630+0000", 2024-04-07T16:16:17.326 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:17.326 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:17.327 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:17.327 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:17.327 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 60, 2024-04-07T16:16:17.327 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:17.327 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:17.327 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:17.327 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:17.327 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:17.327 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:17.327 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:17.327 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:17.327 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.2a", 2024-04-07T16:16:17.327 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:17.327 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:17.327 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:17.328 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 19, 2024-04-07T16:16:17.328 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:17.328 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T20:49:17.570678+0000", 2024-04-07T16:16:17.328 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:17.328 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:17.328 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:17.328 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:17.328 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:17.328 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:17.328 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:17.328 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:17.328 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:17.328 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:17.328 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:17.329 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:17.329 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:17.329 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:17.329 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:17.329 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:17.329 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:17.329 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:17.329 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:17.329 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:17.329 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:17.329 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:17.329 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:17.329 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:17.329 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:17.329 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:17.329 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:17.330 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:17.330 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:17.330 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:17.330 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:17.330 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:17.330 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:17.330 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:17.330 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:17.330 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:17.330 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:17.330 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:17.330 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:17.330 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:17.330 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:17.330 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:17.331 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:17.331 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:17.331 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:17.331 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:17.331 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:17.331 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T16:16:17.331 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T16:16:17.331 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T16:16:17.331 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:17.331 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 5, 2024-04-07T16:16:17.331 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:17.331 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:17.331 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:17.331 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:17.331 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T16:16:17.332 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T16:16:17.332 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T16:16:17.332 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:17.332 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 0, 2024-04-07T16:16:17.332 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:17.332 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:17.332 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T16:16:17.332 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:17.332 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:17.332 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:17.332 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.029710+0000", 2024-04-07T16:16:17.332 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.386422+0000", 2024-04-07T16:16:17.332 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.386422+0000", 2024-04-07T16:16:17.332 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.386718+0000", 2024-04-07T16:16:17.332 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.029710+0000", 2024-04-07T16:16:17.332 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:17.333 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:17.333 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:17.333 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 61, 2024-04-07T16:16:17.333 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.029710+0000", 2024-04-07T16:16:17.333 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.029710+0000", 2024-04-07T16:16:17.333 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.029710+0000", 2024-04-07T16:16:17.333 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:17.333 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:17.333 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:17.333 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.029710+0000", 2024-04-07T16:16:17.333 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.029710+0000", 2024-04-07T16:16:17.333 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:17.333 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:17.333 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:17.333 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:17.334 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 60, 2024-04-07T16:16:17.334 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:17.334 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:17.334 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:17.334 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:17.334 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:17.334 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:17.334 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:17.334 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:17.334 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.2b", 2024-04-07T16:16:17.334 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:17.334 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:17.334 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:17.334 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 19, 2024-04-07T16:16:17.334 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:17.335 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T21:28:09.174468+0000", 2024-04-07T16:16:17.335 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:17.335 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:17.335 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:17.335 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:17.335 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:17.335 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:17.335 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:17.335 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:17.335 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:17.335 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:17.335 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:17.335 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:17.335 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:17.335 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:17.336 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:17.336 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:17.336 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:17.336 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:17.336 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:17.336 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:17.336 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:17.336 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:17.336 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:17.336 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:17.336 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:17.336 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:17.336 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:17.336 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:17.336 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:17.337 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:17.337 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:17.337 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:17.337 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:17.337 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:17.337 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:17.337 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:17.337 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:17.337 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:17.337 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:17.337 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:17.337 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:17.337 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:17.337 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:17.337 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:17.338 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:17.338 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:17.338 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:17.338 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T16:16:17.338 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T16:16:17.338 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T16:16:17.338 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:17.338 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 0, 2024-04-07T16:16:17.338 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:17.338 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:17.338 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:17.338 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T16:16:17.338 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T16:16:17.338 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T16:16:17.339 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T16:16:17.339 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:17.339 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 6, 2024-04-07T16:16:17.339 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T16:16:17.339 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T16:16:17.339 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T16:16:17.339 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T16:16:17.339 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T16:16:17.339 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T16:16:17.339 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T16:13:38.526600+0000", 2024-04-07T16:16:17.339 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:12:13.379758+0000", 2024-04-07T16:16:17.339 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:12:13.379758+0000", 2024-04-07T16:16:17.339 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:12:13.380077+0000", 2024-04-07T16:16:17.339 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T16:13:38.526600+0000", 2024-04-07T16:16:17.339 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:17.340 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T16:16:17.340 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:17.340 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 61, 2024-04-07T16:16:17.340 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T16:13:38.526600+0000", 2024-04-07T16:16:17.340 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T16:13:38.526600+0000", 2024-04-07T16:16:17.340 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T16:13:38.526600+0000", 2024-04-07T16:16:17.340 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T16:16:17.340 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T16:16:17.340 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T16:16:17.340 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T16:13:38.526600+0000", 2024-04-07T16:16:17.340 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T16:13:38.526600+0000", 2024-04-07T16:16:17.340 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T16:16:17.340 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:17.340 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T16:16:17.340 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T16:16:17.340 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 60, 2024-04-07T16:16:17.341 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T16:16:17.341 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T16:16:17.341 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T16:16:17.341 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T16:16:17.341 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:17.341 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T16:16:17.341 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T16:16:17.341 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T16:16:17.341 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.2c", 2024-04-07T16:16:17.341 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T16:16:17.341 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T16:16:17.341 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 69, 2024-04-07T16:16:17.341 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 19, 2024-04-07T16:16:17.341 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T16:16:17.342 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T22:47:34.836189+0000", 2024-04-07T16:16:17.342 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T16:16:17.342 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T16:16:17.342 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:17.342 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:17.342 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:17.342 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:17.342 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:17.342 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:17.342 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:17.342 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:17.342 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:17.342 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:17.342 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:17.342 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:17.342 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:17.343 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:17.343 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:17.343 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:17.343 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:17.343 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:17.343 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:17.343 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:17.343 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:17.343 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:17.343 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:17.343 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:17.343 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:17.343 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:17.343 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:17.343 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:17.344 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:17.344 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:17.344 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:17.344 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:17.344 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:17.344 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:17.344 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:17.344 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:17.344 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:17.344 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:17.344 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:17.344 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:17.344 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:17.344 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:17.344 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T16:16:17.345 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T16:16:17.345 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T16:16:17.345 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T16:16:17.345 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T16:16:17.345 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T16:16:17.345 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:17.345 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 6, 2024-04-07T16:16:17.345 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T16:16:17.345 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:17.345 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:17.345 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_per_pool": [ 2024-04-07T16:16:17.345 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:17.345 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": 24, 2024-04-07T16:16:17.345 INFO:tasks.workunit.client.0.smithi100.stdout: "application": [ 2024-04-07T16:16:17.346 INFO:tasks.workunit.client.0.smithi100.stdout: "rbd" 2024-04-07T16:16:17.346 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:17.346 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:17.346 INFO:tasks.workunit.client.0.smithi100.stdout: "num_pg": 8, 2024-04-07T16:16:17.346 INFO:tasks.workunit.client.0.smithi100.stdout: "num_store_stats": 8, 2024-04-07T16:16:17.346 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:17.346 INFO:tasks.workunit.client.0.smithi100.stdout: "poolid": 4, 2024-04-07T16:16:17.346 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:17.346 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:17.346 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:17.346 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:17.346 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:17.346 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:17.346 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:17.346 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:17.347 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:17.347 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:17.347 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:17.347 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:17.347 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:17.347 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:17.347 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:17.347 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:17.347 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:17.347 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:17.347 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:17.347 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:17.347 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:17.347 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:17.347 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:17.348 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:17.348 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:17.348 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:17.348 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:17.348 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:17.348 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:17.348 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:17.348 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:17.348 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:17.348 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:17.348 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:17.348 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:17.348 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:17.348 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:17.348 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:17.349 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:17.349 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:17.349 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:17.349 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:17.349 INFO:tasks.workunit.client.0.smithi100.stdout: "store_stats": { 2024-04-07T16:16:17.349 INFO:tasks.workunit.client.0.smithi100.stdout: "allocated": 0, 2024-04-07T16:16:17.349 INFO:tasks.workunit.client.0.smithi100.stdout: "available": 0, 2024-04-07T16:16:17.349 INFO:tasks.workunit.client.0.smithi100.stdout: "data_compressed": 0, 2024-04-07T16:16:17.349 INFO:tasks.workunit.client.0.smithi100.stdout: "data_compressed_allocated": 0, 2024-04-07T16:16:17.349 INFO:tasks.workunit.client.0.smithi100.stdout: "data_compressed_original": 0, 2024-04-07T16:16:17.349 INFO:tasks.workunit.client.0.smithi100.stdout: "data_stored": 0, 2024-04-07T16:16:17.349 INFO:tasks.workunit.client.0.smithi100.stdout: "internal_metadata": 0, 2024-04-07T16:16:17.349 INFO:tasks.workunit.client.0.smithi100.stdout: "internally_reserved": 0, 2024-04-07T16:16:17.349 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_allocated": 0, 2024-04-07T16:16:17.349 INFO:tasks.workunit.client.0.smithi100.stdout: "total": 0 2024-04-07T16:16:17.350 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:17.350 INFO:tasks.workunit.client.0.smithi100.stdout: "up": 24 2024-04-07T16:16:17.350 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:17.350 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:17.350 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": 192, 2024-04-07T16:16:17.350 INFO:tasks.workunit.client.0.smithi100.stdout: "application": [ 2024-04-07T16:16:17.350 INFO:tasks.workunit.client.0.smithi100.stdout: "cephfs" 2024-04-07T16:16:17.350 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:17.350 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T16:16:17.350 INFO:tasks.workunit.client.0.smithi100.stdout: "num_pg": 64, 2024-04-07T16:16:17.350 INFO:tasks.workunit.client.0.smithi100.stdout: "num_store_stats": 8, 2024-04-07T16:16:17.350 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T16:16:17.350 INFO:tasks.workunit.client.0.smithi100.stdout: "poolid": 3, 2024-04-07T16:16:17.350 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:17.350 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T16:16:17.351 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:17.351 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:17.351 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:17.351 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:17.351 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:17.351 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:17.351 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:17.351 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:17.351 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:17.351 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:17.351 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:17.351 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:17.351 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:17.351 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:17.351 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:17.352 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T16:16:17.352 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T16:16:17.352 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:17.352 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T16:16:17.352 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:17.352 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:17.352 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:17.352 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:17.352 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:17.352 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:17.352 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:17.352 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T16:16:17.352 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:17.352 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:17.352 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:17.353 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:17.353 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:17.353 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:17.353 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:17.353 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:17.353 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:17.353 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:17.353 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T16:16:17.353 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T16:16:17.353 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:17.353 INFO:tasks.workunit.client.0.smithi100.stdout: "store_stats": { 2024-04-07T16:16:17.353 INFO:tasks.workunit.client.0.smithi100.stdout: "allocated": 0, 2024-04-07T16:16:17.353 INFO:tasks.workunit.client.0.smithi100.stdout: "available": 0, 2024-04-07T16:16:17.353 INFO:tasks.workunit.client.0.smithi100.stdout: "data_compressed": 0, 2024-04-07T16:16:17.353 INFO:tasks.workunit.client.0.smithi100.stdout: "data_compressed_allocated": 0, 2024-04-07T16:16:17.354 INFO:tasks.workunit.client.0.smithi100.stdout: "data_compressed_original": 0, 2024-04-07T16:16:17.354 INFO:tasks.workunit.client.0.smithi100.stdout: "data_stored": 0, 2024-04-07T16:16:17.354 INFO:tasks.workunit.client.0.smithi100.stdout: "internal_metadata": 0, 2024-04-07T16:16:17.354 INFO:tasks.workunit.client.0.smithi100.stdout: "internally_reserved": 0, 2024-04-07T16:16:17.354 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_allocated": 0, 2024-04-07T16:16:17.354 INFO:tasks.workunit.client.0.smithi100.stdout: "total": 0 2024-04-07T16:16:17.354 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:17.354 INFO:tasks.workunit.client.0.smithi100.stdout: "up": 192 2024-04-07T16:16:17.354 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:17.354 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:17.354 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": 3, 2024-04-07T16:16:17.354 INFO:tasks.workunit.client.0.smithi100.stdout: "application": [ 2024-04-07T16:16:17.354 INFO:tasks.workunit.client.0.smithi100.stdout: "mgr" 2024-04-07T16:16:17.354 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:17.354 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 156, 2024-04-07T16:16:17.355 INFO:tasks.workunit.client.0.smithi100.stdout: "num_pg": 1, 2024-04-07T16:16:17.355 INFO:tasks.workunit.client.0.smithi100.stdout: "num_store_stats": 3, 2024-04-07T16:16:17.355 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 156, 2024-04-07T16:16:17.355 INFO:tasks.workunit.client.0.smithi100.stdout: "poolid": 1, 2024-04-07T16:16:17.355 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:17.355 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 590368, 2024-04-07T16:16:17.355 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:17.355 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 1771104, 2024-04-07T16:16:17.355 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:17.355 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:17.355 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:17.355 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:17.355 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:17.355 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:17.355 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:17.356 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:17.356 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:17.356 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:17.356 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:17.356 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:17.356 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:17.356 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 6, 2024-04-07T16:16:17.356 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 2, 2024-04-07T16:16:17.356 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:17.356 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 2, 2024-04-07T16:16:17.356 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:17.356 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:17.356 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:17.356 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:17.356 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:17.357 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T16:16:17.357 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:17.357 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 6, 2024-04-07T16:16:17.357 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:17.357 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:17.357 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:17.357 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:17.357 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:17.357 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 466, 2024-04-07T16:16:17.357 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 644, 2024-04-07T16:16:17.357 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:17.357 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:17.357 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:17.357 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 235, 2024-04-07T16:16:17.357 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 2416 2024-04-07T16:16:17.358 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:17.358 INFO:tasks.workunit.client.0.smithi100.stdout: "store_stats": { 2024-04-07T16:16:17.358 INFO:tasks.workunit.client.0.smithi100.stdout: "allocated": 1781760, 2024-04-07T16:16:17.358 INFO:tasks.workunit.client.0.smithi100.stdout: "available": 0, 2024-04-07T16:16:17.358 INFO:tasks.workunit.client.0.smithi100.stdout: "data_compressed": 0, 2024-04-07T16:16:17.358 INFO:tasks.workunit.client.0.smithi100.stdout: "data_compressed_allocated": 0, 2024-04-07T16:16:17.358 INFO:tasks.workunit.client.0.smithi100.stdout: "data_compressed_original": 0, 2024-04-07T16:16:17.358 INFO:tasks.workunit.client.0.smithi100.stdout: "data_stored": 1771104, 2024-04-07T16:16:17.358 INFO:tasks.workunit.client.0.smithi100.stdout: "internal_metadata": 0, 2024-04-07T16:16:17.358 INFO:tasks.workunit.client.0.smithi100.stdout: "internally_reserved": 0, 2024-04-07T16:16:17.358 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_allocated": 0, 2024-04-07T16:16:17.358 INFO:tasks.workunit.client.0.smithi100.stdout: "total": 0 2024-04-07T16:16:17.358 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:17.358 INFO:tasks.workunit.client.0.smithi100.stdout: "up": 3 2024-04-07T16:16:17.359 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:17.359 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T16:16:17.359 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": 96, 2024-04-07T16:16:17.359 INFO:tasks.workunit.client.0.smithi100.stdout: "application": [ 2024-04-07T16:16:17.359 INFO:tasks.workunit.client.0.smithi100.stdout: "cephfs" 2024-04-07T16:16:17.359 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:17.359 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 45, 2024-04-07T16:16:17.359 INFO:tasks.workunit.client.0.smithi100.stdout: "num_pg": 32, 2024-04-07T16:16:17.359 INFO:tasks.workunit.client.0.smithi100.stdout: "num_store_stats": 8, 2024-04-07T16:16:17.359 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 45, 2024-04-07T16:16:17.359 INFO:tasks.workunit.client.0.smithi100.stdout: "poolid": 2, 2024-04-07T16:16:17.359 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T16:16:17.359 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 3430, 2024-04-07T16:16:17.359 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T16:16:17.359 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T16:16:17.360 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T16:16:17.360 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T16:16:17.360 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T16:16:17.360 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T16:16:17.360 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T16:16:17.360 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T16:16:17.360 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T16:16:17.360 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T16:16:17.360 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T16:16:17.360 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T16:16:17.360 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T16:16:17.360 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T16:16:17.360 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T16:16:17.360 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 66, 2024-04-07T16:16:17.360 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 22, 2024-04-07T16:16:17.361 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T16:16:17.361 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 22, 2024-04-07T16:16:17.361 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T16:16:17.361 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T16:16:17.361 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T16:16:17.361 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T16:16:17.361 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T16:16:17.361 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 14, 2024-04-07T16:16:17.361 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T16:16:17.361 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 1, 2024-04-07T16:16:17.361 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T16:16:17.361 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T16:16:17.361 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T16:16:17.361 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T16:16:17.361 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T16:16:17.361 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T16:16:17.362 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T16:16:17.362 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T16:16:17.362 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T16:16:17.362 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T16:16:17.362 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 51, 2024-04-07T16:16:17.362 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 20 2024-04-07T16:16:17.362 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:17.362 INFO:tasks.workunit.client.0.smithi100.stdout: "store_stats": { 2024-04-07T16:16:17.362 INFO:tasks.workunit.client.0.smithi100.stdout: "allocated": 98304, 2024-04-07T16:16:17.362 INFO:tasks.workunit.client.0.smithi100.stdout: "available": 0, 2024-04-07T16:16:17.362 INFO:tasks.workunit.client.0.smithi100.stdout: "data_compressed": 0, 2024-04-07T16:16:17.362 INFO:tasks.workunit.client.0.smithi100.stdout: "data_compressed_allocated": 0, 2024-04-07T16:16:17.362 INFO:tasks.workunit.client.0.smithi100.stdout: "data_compressed_original": 0, 2024-04-07T16:16:17.362 INFO:tasks.workunit.client.0.smithi100.stdout: "data_stored": 10290, 2024-04-07T16:16:17.362 INFO:tasks.workunit.client.0.smithi100.stdout: "internal_metadata": 0, 2024-04-07T16:16:17.362 INFO:tasks.workunit.client.0.smithi100.stdout: "internally_reserved": 0, 2024-04-07T16:16:17.363 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_allocated": 0, 2024-04-07T16:16:17.363 INFO:tasks.workunit.client.0.smithi100.stdout: "total": 0 2024-04-07T16:16:17.363 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:17.363 INFO:tasks.workunit.client.0.smithi100.stdout: "up": 96 2024-04-07T16:16:17.363 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:17.363 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T16:16:17.363 INFO:tasks.workunit.client.0.smithi100.stdout: "usage": { 2024-04-07T16:16:17.363 INFO:tasks.workunit.client.0.smithi100.stdout: "pg_num": 105, 2024-04-07T16:16:17.363 INFO:tasks.workunit.client.0.smithi100.stdout: "pools": 4, 2024-04-07T16:16:17.363 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_by_class": { 2024-04-07T16:16:17.363 INFO:tasks.workunit.client.0.smithi100.stdout: "ssd": { 2024-04-07T16:16:17.363 INFO:tasks.workunit.client.0.smithi100.stdout: "total_avail_bytes": 767718830080, 2024-04-07T16:16:17.363 INFO:tasks.workunit.client.0.smithi100.stdout: "total_bytes": 767960285184, 2024-04-07T16:16:17.363 INFO:tasks.workunit.client.0.smithi100.stdout: "total_used_bytes": 241455104, 2024-04-07T16:16:17.363 INFO:tasks.workunit.client.0.smithi100.stdout: "total_used_raw_bytes": 241455104, 2024-04-07T16:16:17.363 INFO:tasks.workunit.client.0.smithi100.stdout: "total_used_raw_ratio": 0.00031441092141903937 2024-04-07T16:16:17.364 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:17.364 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T16:16:17.364 INFO:tasks.workunit.client.0.smithi100.stdout: "total_avail_bytes": 767718830080, 2024-04-07T16:16:17.364 INFO:tasks.workunit.client.0.smithi100.stdout: "total_bytes": 767960285184, 2024-04-07T16:16:17.364 INFO:tasks.workunit.client.0.smithi100.stdout: "total_used_bytes": 241455104 2024-04-07T16:16:17.364 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:17.364 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T16:16:17.364 INFO:tasks.workunit.client.0.smithi100.stdout:} 2024-04-07T16:16:17.365 INFO:tasks.workunit.client.0.smithi100.stderr:+ echo OK 2024-04-07T16:16:17.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:16 smithi144 bash[23522]: cluster 2024-04-07T16:16:14.677364+0000 mgr.y (mgr.14553) 154 : cluster 0 pgmap v108: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 28 KiB/s rd, 0 B/s wr, 25 op/s 2024-04-07T16:16:17.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:16 smithi144 bash[23522]: audit 2024-04-07T16:16:14.856487+0000 mgr.y (mgr.14553) 155 : audit 0 from='client.24875 -' entity='client.admin' cmd=[{"prefix": "telemetry show-all", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:16:17.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:16 smithi144 bash[23522]: audit 2024-04-07T16:16:15.444593+0000 mon.b (mon.1) 212 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:16:17.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:16 smithi144 bash[23522]: audit 2024-04-07T16:16:15.446869+0000 mgr.y (mgr.14553) 156 : audit 0 from='client.24875 -' entity='client.admin' cmd=[{"prefix": "telemetry show-all", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:16:17.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:16 smithi144 bash[23522]: audit 2024-04-07T16:16:15.451522+0000 mon.b (mon.1) 213 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config-key dump", "format": "json"}]: dispatch 2024-04-07T16:16:17.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:16 smithi144 bash[23522]: audit 2024-04-07T16:16:15.661958+0000 mon.b (mon.1) 214 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:16:17.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:16 smithi144 bash[23522]: audit 2024-04-07T16:16:15.662399+0000 mon.b (mon.1) 215 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:16:17.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:16 smithi144 bash[23522]: audit 2024-04-07T16:16:15.662846+0000 mon.b (mon.1) 216 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:16:17.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:16 smithi144 bash[23522]: audit 2024-04-07T16:16:15.666283+0000 mon.b (mon.1) 217 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:16:17.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:16 smithi144 bash[23522]: audit 2024-04-07T16:16:15.666697+0000 mon.b (mon.1) 218 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:16:17.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:16 smithi144 bash[23522]: audit 2024-04-07T16:16:15.667186+0000 mon.b (mon.1) 219 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:16:17.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:16 smithi144 bash[23522]: audit 2024-04-07T16:16:15.670426+0000 mon.b (mon.1) 220 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:16:17.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:16 smithi144 bash[23522]: audit 2024-04-07T16:16:15.670810+0000 mon.b (mon.1) 221 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:16:17.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:16 smithi144 bash[23522]: audit 2024-04-07T16:16:15.671186+0000 mon.b (mon.1) 222 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:16:17.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:16 smithi144 bash[23522]: audit 2024-04-07T16:16:15.690853+0000 mon.b (mon.1) 223 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:16:17.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:16 smithi144 bash[23522]: audit 2024-04-07T16:16:15.691243+0000 mon.b (mon.1) 224 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:16:17.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:16 smithi144 bash[23522]: audit 2024-04-07T16:16:15.691658+0000 mon.b (mon.1) 225 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:16:17.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:16 smithi144 bash[23522]: audit 2024-04-07T16:16:15.694435+0000 mon.b (mon.1) 226 : audit 1 from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T16:16:17.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:16 smithi144 bash[23522]: audit 2024-04-07T16:16:15.694546+0000 mon.b (mon.1) 227 : audit 1 from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T16:16:17.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:16 smithi144 bash[23522]: audit 2024-04-07T16:16:15.694987+0000 mon.b (mon.1) 228 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:16:17.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:16 smithi144 bash[23522]: audit 2024-04-07T16:16:15.695235+0000 mon.a (mon.0) 872 : audit 1 from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T16:16:17.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:16 smithi144 bash[23522]: audit 2024-04-07T16:16:15.695371+0000 mon.a (mon.0) 873 : audit 1 from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T16:16:17.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:16 smithi144 bash[23522]: audit 2024-04-07T16:16:15.695381+0000 mon.b (mon.1) 229 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:16:17.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:16 smithi144 bash[23522]: audit 2024-04-07T16:16:15.695768+0000 mon.b (mon.1) 230 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:16:17.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:16 smithi144 bash[23522]: audit 2024-04-07T16:16:15.698838+0000 mon.b (mon.1) 231 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:16:17.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:16 smithi144 bash[23522]: audit 2024-04-07T16:16:15.699221+0000 mon.b (mon.1) 232 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:16:17.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:16 smithi144 bash[23522]: audit 2024-04-07T16:16:15.699595+0000 mon.b (mon.1) 233 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:16:17.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:16 smithi144 bash[23522]: audit 2024-04-07T16:16:15.703240+0000 mon.c (mon.2) 49 : audit 1 from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T16:16:17.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:16 smithi144 bash[23522]: audit 2024-04-07T16:16:15.703346+0000 mon.c (mon.2) 50 : audit 1 from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T16:16:17.368 INFO:tasks.workunit.client.0.smithi100.stdout:OK 2024-04-07T16:16:17.368 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-07T16:16:17.368 DEBUG:teuthology.orchestra.run.smithi100:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-07T16:16:17.369 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:16 smithi100 bash[21346]: cluster 2024-04-07T16:16:14.677364+0000 mgr.y (mgr.14553) 154 : cluster 0 pgmap v108: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 28 KiB/s rd, 0 B/s wr, 25 op/s 2024-04-07T16:16:17.369 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:16 smithi100 bash[21346]: audit 2024-04-07T16:16:14.856487+0000 mgr.y (mgr.14553) 155 : audit 0 from='client.24875 -' entity='client.admin' cmd=[{"prefix": "telemetry show-all", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:16:17.369 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:16 smithi100 bash[21346]: audit 2024-04-07T16:16:15.444593+0000 mon.b (mon.1) 212 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:16:17.369 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:16 smithi100 bash[21346]: audit 2024-04-07T16:16:15.446869+0000 mgr.y (mgr.14553) 156 : audit 0 from='client.24875 -' entity='client.admin' cmd=[{"prefix": "telemetry show-all", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:16:17.369 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:16 smithi100 bash[21346]: audit 2024-04-07T16:16:15.451522+0000 mon.b (mon.1) 213 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config-key dump", "format": "json"}]: dispatch 2024-04-07T16:16:17.369 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:16 smithi100 bash[21346]: audit 2024-04-07T16:16:15.661958+0000 mon.b (mon.1) 214 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:16:17.369 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:16 smithi100 bash[21346]: audit 2024-04-07T16:16:15.662399+0000 mon.b (mon.1) 215 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:16:17.369 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:16 smithi100 bash[21346]: audit 2024-04-07T16:16:15.662846+0000 mon.b (mon.1) 216 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:16:17.369 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:16 smithi100 bash[21346]: audit 2024-04-07T16:16:15.666283+0000 mon.b (mon.1) 217 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:16:17.369 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:16 smithi100 bash[21346]: audit 2024-04-07T16:16:15.666697+0000 mon.b (mon.1) 218 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:16:17.369 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:16 smithi100 bash[21346]: audit 2024-04-07T16:16:15.667186+0000 mon.b (mon.1) 219 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:16:17.369 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:16 smithi100 bash[21346]: audit 2024-04-07T16:16:15.670426+0000 mon.b (mon.1) 220 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:16:17.369 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:16 smithi100 bash[21346]: audit 2024-04-07T16:16:15.670810+0000 mon.b (mon.1) 221 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:16:17.370 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:16 smithi100 bash[21346]: audit 2024-04-07T16:16:15.671186+0000 mon.b (mon.1) 222 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:16:17.370 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:16 smithi100 bash[21346]: audit 2024-04-07T16:16:15.690853+0000 mon.b (mon.1) 223 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:16:17.370 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:16 smithi100 bash[21346]: audit 2024-04-07T16:16:15.691243+0000 mon.b (mon.1) 224 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:16:17.370 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:16 smithi100 bash[21346]: audit 2024-04-07T16:16:15.691658+0000 mon.b (mon.1) 225 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:16:17.370 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:16 smithi100 bash[21346]: audit 2024-04-07T16:16:15.694435+0000 mon.b (mon.1) 226 : audit 1 from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T16:16:17.370 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:16 smithi100 bash[31660]: cluster 2024-04-07T16:16:14.677364+0000 mgr.y (mgr.14553) 154 : cluster 0 pgmap v108: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 28 KiB/s rd, 0 B/s wr, 25 op/s 2024-04-07T16:16:17.370 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:16 smithi100 bash[31660]: audit 2024-04-07T16:16:14.856487+0000 mgr.y (mgr.14553) 155 : audit 0 from='client.24875 -' entity='client.admin' cmd=[{"prefix": "telemetry show-all", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:16:17.370 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:16 smithi100 bash[31660]: audit 2024-04-07T16:16:15.444593+0000 mon.b (mon.1) 212 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:16:17.370 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:16 smithi100 bash[31660]: audit 2024-04-07T16:16:15.446869+0000 mgr.y (mgr.14553) 156 : audit 0 from='client.24875 -' entity='client.admin' cmd=[{"prefix": "telemetry show-all", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:16:17.370 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:16 smithi100 bash[31660]: audit 2024-04-07T16:16:15.451522+0000 mon.b (mon.1) 213 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config-key dump", "format": "json"}]: dispatch 2024-04-07T16:16:17.370 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:16 smithi100 bash[31660]: audit 2024-04-07T16:16:15.661958+0000 mon.b (mon.1) 214 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:16:17.371 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:16 smithi100 bash[31660]: audit 2024-04-07T16:16:15.662399+0000 mon.b (mon.1) 215 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:16:17.371 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:16 smithi100 bash[31660]: audit 2024-04-07T16:16:15.662846+0000 mon.b (mon.1) 216 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:16:17.371 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:16 smithi100 bash[31660]: audit 2024-04-07T16:16:15.666283+0000 mon.b (mon.1) 217 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:16:17.371 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:16 smithi100 bash[31660]: audit 2024-04-07T16:16:15.666697+0000 mon.b (mon.1) 218 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:16:17.371 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:16 smithi100 bash[31660]: audit 2024-04-07T16:16:15.667186+0000 mon.b (mon.1) 219 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:16:17.371 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:16 smithi100 bash[31660]: audit 2024-04-07T16:16:15.670426+0000 mon.b (mon.1) 220 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:16:17.371 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:16 smithi100 bash[31660]: audit 2024-04-07T16:16:15.670810+0000 mon.b (mon.1) 221 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:16:17.371 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:16 smithi100 bash[31660]: audit 2024-04-07T16:16:15.671186+0000 mon.b (mon.1) 222 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:16:17.371 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:16 smithi100 bash[31660]: audit 2024-04-07T16:16:15.690853+0000 mon.b (mon.1) 223 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:16:17.371 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:16 smithi100 bash[31660]: audit 2024-04-07T16:16:15.691243+0000 mon.b (mon.1) 224 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:16:17.371 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:16 smithi100 bash[31660]: audit 2024-04-07T16:16:15.691658+0000 mon.b (mon.1) 225 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:16:17.371 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:16 smithi100 bash[31660]: audit 2024-04-07T16:16:15.694435+0000 mon.b (mon.1) 226 : audit 1 from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T16:16:17.372 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:16 smithi100 bash[21346]: audit 2024-04-07T16:16:15.694546+0000 mon.b (mon.1) 227 : audit 1 from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T16:16:17.372 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:16 smithi100 bash[21346]: audit 2024-04-07T16:16:15.694987+0000 mon.b (mon.1) 228 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:16:17.372 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:16 smithi100 bash[21346]: audit 2024-04-07T16:16:15.695235+0000 mon.a (mon.0) 872 : audit 1 from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T16:16:17.372 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:16 smithi100 bash[21346]: audit 2024-04-07T16:16:15.695371+0000 mon.a (mon.0) 873 : audit 1 from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T16:16:17.372 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:16 smithi100 bash[21346]: audit 2024-04-07T16:16:15.695381+0000 mon.b (mon.1) 229 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:16:17.372 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:16 smithi100 bash[21346]: audit 2024-04-07T16:16:15.695768+0000 mon.b (mon.1) 230 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:16:17.372 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:16 smithi100 bash[21346]: audit 2024-04-07T16:16:15.698838+0000 mon.b (mon.1) 231 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:16:17.372 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:16 smithi100 bash[21346]: audit 2024-04-07T16:16:15.699221+0000 mon.b (mon.1) 232 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:16:17.372 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:16 smithi100 bash[21346]: audit 2024-04-07T16:16:15.699595+0000 mon.b (mon.1) 233 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:16:17.372 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:16 smithi100 bash[21346]: audit 2024-04-07T16:16:15.703240+0000 mon.c (mon.2) 49 : audit 1 from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T16:16:17.372 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:16 smithi100 bash[21346]: audit 2024-04-07T16:16:15.703346+0000 mon.c (mon.2) 50 : audit 1 from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T16:16:17.373 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:16 smithi100 bash[31660]: audit 2024-04-07T16:16:15.694546+0000 mon.b (mon.1) 227 : audit 1 from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T16:16:17.373 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:16 smithi100 bash[31660]: audit 2024-04-07T16:16:15.694987+0000 mon.b (mon.1) 228 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:16:17.373 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:16 smithi100 bash[31660]: audit 2024-04-07T16:16:15.695235+0000 mon.a (mon.0) 872 : audit 1 from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T16:16:17.373 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:16 smithi100 bash[31660]: audit 2024-04-07T16:16:15.695371+0000 mon.a (mon.0) 873 : audit 1 from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T16:16:17.373 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:16 smithi100 bash[31660]: audit 2024-04-07T16:16:15.695381+0000 mon.b (mon.1) 229 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:16:17.373 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:16 smithi100 bash[31660]: audit 2024-04-07T16:16:15.695768+0000 mon.b (mon.1) 230 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:16:17.373 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:16 smithi100 bash[31660]: audit 2024-04-07T16:16:15.698838+0000 mon.b (mon.1) 231 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:16:17.373 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:16 smithi100 bash[31660]: audit 2024-04-07T16:16:15.699221+0000 mon.b (mon.1) 232 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:16:17.373 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:16 smithi100 bash[31660]: audit 2024-04-07T16:16:15.699595+0000 mon.b (mon.1) 233 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:16:17.373 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:16 smithi100 bash[31660]: audit 2024-04-07T16:16:15.703240+0000 mon.c (mon.2) 49 : audit 1 from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T16:16:17.373 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:16 smithi100 bash[31660]: audit 2024-04-07T16:16:15.703346+0000 mon.c (mon.2) 50 : audit 1 from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T16:16:17.386 INFO:tasks.workunit:Stopping ['test_telemetry_reef.sh'] on client.0... 2024-04-07T16:16:17.387 DEBUG:teuthology.orchestra.run.smithi100:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-04-07T16:16:17.696 DEBUG:teuthology.parallel:result is None 2024-04-07T16:16:17.697 DEBUG:teuthology.orchestra.run.smithi100:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-04-07T16:16:17.708 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-04-07T16:16:17.709 DEBUG:teuthology.orchestra.run.smithi100:> rmdir -- /home/ubuntu/cephtest/mnt.0 2024-04-07T16:16:17.762 INFO:tasks.workunit:Deleted artificial mount point /home/ubuntu/cephtest/mnt.0/client.0 2024-04-07T16:16:17.762 INFO:teuthology.run_tasks:Running task print... 2024-04-07T16:16:17.773 INFO:teuthology.task.print:**** done end telemetry reef... 2024-04-07T16:16:17.773 INFO:teuthology.run_tasks:Running task print... 2024-04-07T16:16:17.781 INFO:teuthology.task.print:**** done start parallel 2024-04-07T16:16:17.781 INFO:teuthology.run_tasks:Running task parallel... 2024-04-07T16:16:17.816 INFO:teuthology.task.parallel:starting parallel... 2024-04-07T16:16:17.816 INFO:teuthology.task.parallel:In parallel, running task full_sequential... 2024-04-07T16:16:17.817 INFO:teuthology.task.full_sequential:In full_sequential, running task print... 2024-04-07T16:16:17.817 INFO:teuthology.task.print:**** done start ec-rados-default.yaml 2024-04-07T16:16:17.817 INFO:teuthology.task.full_sequential:In full_sequential, running task rados... 2024-04-07T16:16:17.819 INFO:tasks.rados:Beginning rados... 2024-04-07T16:16:17.819 INFO:tasks.rados:config is {'clients': ['client.0'], 'ec_pool': True, 'objects': 50, 'op_weights': {'append': 100, 'copy_from': 50, 'delete': 50, 'read': 100, 'rmattr': 25, 'rollback': 50, 'setattr': 25, 'snap_create': 50, 'snap_remove': 50, 'write': 0}, 'ops': 4000, 'write_append_excl': False} 2024-04-07T16:16:17.819 INFO:tasks.rados:overrides is {'admin_socket': {'branch': 'wip-yuri3-testing-2024-04-05-0825'}, 'ceph': {'conf': {'global': {'mon election default strategy': 1}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'debug ms': 1, 'debug osd': 20, 'osd shutdown pgref assert': True}}, 'create_rbd_pool': True, 'flavor': 'default', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'sha1': 'adce92d056533f22dc1ee37e3141d098b046ec1b'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'extra_system_packages': ['python3-pytest'], 'flavor': 'default', 'sha1': 'adce92d056533f22dc1ee37e3141d098b046ec1b'}}, 'workunit': {'branch': 'wip-yuri3-testing-2024-04-05-0825', 'sha1': 'adce92d056533f22dc1ee37e3141d098b046ec1b'}} 2024-04-07T16:16:17.819 INFO:tasks.rados:config is {'clients': ['client.0'], 'ec_pool': True, 'objects': 50, 'op_weights': {'append': 100, 'copy_from': 50, 'delete': 50, 'read': 100, 'rmattr': 25, 'rollback': 50, 'setattr': 25, 'snap_create': 50, 'snap_remove': 50, 'write': 0}, 'ops': 4000, 'write_append_excl': False} 2024-04-07T16:16:17.819 INFO:tasks.rados:joining rados 2024-04-07T16:16:17.819 INFO:teuthology.task.parallel:In parallel, running task sequential... 2024-04-07T16:16:17.846 INFO:teuthology.task.sequential:In sequential, running task print... 2024-04-07T16:16:17.846 INFO:teuthology.task.print:**** done start upgrade, wait 2024-04-07T16:16:17.846 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-04-07T16:16:17.846 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi100.front.sepia.ceph.com 2024-04-07T16:16:17.847 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0702e3a8-f4f8-11ee-b649-cb9ed24678a4 -e sha1=adce92d056533f22dc1ee37e3141d098b046ec1b -- bash -c 'ceph config set global log_to_journald false --force' 2024-04-07T16:16:17.847 INFO:tasks.rados:clients are ['client.0', 'client.1'] 2024-04-07T16:16:17.848 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 0702e3a8-f4f8-11ee-b649-cb9ed24678a4 -- ceph osd erasure-code-profile set teuthologyprofile k=2 m=1 crush-failure-domain=osd 2024-04-07T16:16:18.687 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:18 smithi100 bash[21346]: cluster 2024-04-07T16:16:16.678025+0000 mgr.y (mgr.14553) 157 : cluster 0 pgmap v109: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 28 KiB/s rd, 0 B/s wr, 25 op/s 2024-04-07T16:16:18.687 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:18 smithi100 bash[31660]: cluster 2024-04-07T16:16:16.678025+0000 mgr.y (mgr.14553) 157 : cluster 0 pgmap v109: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 28 KiB/s rd, 0 B/s wr, 25 op/s 2024-04-07T16:16:18.856 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:18 smithi144 bash[23522]: cluster 2024-04-07T16:16:16.678025+0000 mgr.y (mgr.14553) 157 : cluster 0 pgmap v109: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 28 KiB/s rd, 0 B/s wr, 25 op/s 2024-04-07T16:16:20.687 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:20 smithi100 bash[21346]: cluster 2024-04-07T16:16:18.678695+0000 mgr.y (mgr.14553) 158 : cluster 0 pgmap v110: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 28 KiB/s rd, 0 B/s wr, 25 op/s 2024-04-07T16:16:20.687 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:20 smithi100 bash[31660]: cluster 2024-04-07T16:16:18.678695+0000 mgr.y (mgr.14553) 158 : cluster 0 pgmap v110: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 28 KiB/s rd, 0 B/s wr, 25 op/s 2024-04-07T16:16:20.856 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:20 smithi144 bash[23522]: cluster 2024-04-07T16:16:18.678695+0000 mgr.y (mgr.14553) 158 : cluster 0 pgmap v110: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 28 KiB/s rd, 0 B/s wr, 25 op/s 2024-04-07T16:16:22.589 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/mon.a/config 2024-04-07T16:16:22.589 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/mon.a/config 2024-04-07T16:16:22.687 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:22 smithi100 bash[21346]: cluster 2024-04-07T16:16:20.679980+0000 mgr.y (mgr.14553) 159 : cluster 0 pgmap v111: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 38 KiB/s rd, 0 B/s wr, 40 op/s 2024-04-07T16:16:22.687 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:22 smithi100 bash[31660]: cluster 2024-04-07T16:16:20.679980+0000 mgr.y (mgr.14553) 159 : cluster 0 pgmap v111: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 38 KiB/s rd, 0 B/s wr, 40 op/s 2024-04-07T16:16:22.856 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:22 smithi144 bash[23522]: cluster 2024-04-07T16:16:20.679980+0000 mgr.y (mgr.14553) 159 : cluster 0 pgmap v111: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 38 KiB/s rd, 0 B/s wr, 40 op/s 2024-04-07T16:16:23.687 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:23 smithi100 bash[21346]: audit 2024-04-07T16:16:22.700951+0000 mon.b (mon.1) 234 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:16:23.687 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:23 smithi100 bash[31660]: audit 2024-04-07T16:16:22.700951+0000 mon.b (mon.1) 234 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:16:23.856 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:23 smithi144 bash[23522]: audit 2024-04-07T16:16:22.700951+0000 mon.b (mon.1) 234 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:16:24.701 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:24 smithi100 bash[31660]: cluster 2024-04-07T16:16:22.680486+0000 mgr.y (mgr.14553) 160 : cluster 0 pgmap v112: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 38 KiB/s rd, 0 B/s wr, 40 op/s 2024-04-07T16:16:24.702 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:24 smithi100 bash[21346]: cluster 2024-04-07T16:16:22.680486+0000 mgr.y (mgr.14553) 160 : cluster 0 pgmap v112: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 38 KiB/s rd, 0 B/s wr, 40 op/s 2024-04-07T16:16:24.855 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:24 smithi144 bash[23522]: cluster 2024-04-07T16:16:22.680486+0000 mgr.y (mgr.14553) 160 : cluster 0 pgmap v112: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 38 KiB/s rd, 0 B/s wr, 40 op/s 2024-04-07T16:16:25.445 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:16:25 smithi100 bash[21601]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:16:25] "GET /metrics HTTP/1.1" 200 33739 "" "Prometheus/2.43.0" 2024-04-07T16:16:26.687 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:26 smithi100 bash[21346]: cluster 2024-04-07T16:16:24.681703+0000 mgr.y (mgr.14553) 161 : cluster 0 pgmap v113: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 38 KiB/s rd, 0 B/s wr, 40 op/s 2024-04-07T16:16:26.687 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:26 smithi100 bash[31660]: cluster 2024-04-07T16:16:24.681703+0000 mgr.y (mgr.14553) 161 : cluster 0 pgmap v113: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 38 KiB/s rd, 0 B/s wr, 40 op/s 2024-04-07T16:16:26.856 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:26 smithi144 bash[23522]: cluster 2024-04-07T16:16:24.681703+0000 mgr.y (mgr.14553) 161 : cluster 0 pgmap v113: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 38 KiB/s rd, 0 B/s wr, 40 op/s 2024-04-07T16:16:27.481 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0702e3a8-f4f8-11ee-b649-cb9ed24678a4 -e sha1=adce92d056533f22dc1ee37e3141d098b046ec1b -- bash -c 'ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1' 2024-04-07T16:16:27.856 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:27 smithi144 bash[23522]: audit 2024-04-07T16:16:26.879694+0000 mon.a (mon.0) 874 : audit 1 from='client.? 172.21.15.100:0/475038168' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "teuthologyprofile", "profile": ["k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-07T16:16:27.937 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:27 smithi100 bash[21346]: audit 2024-04-07T16:16:26.879694+0000 mon.a (mon.0) 874 : audit 1 from='client.? 172.21.15.100:0/475038168' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "teuthologyprofile", "profile": ["k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-07T16:16:27.938 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:27 smithi100 bash[31660]: audit 2024-04-07T16:16:26.879694+0000 mon.a (mon.0) 874 : audit 1 from='client.? 172.21.15.100:0/475038168' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "teuthologyprofile", "profile": ["k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-07T16:16:28.107 INFO:tasks.rados:starting run 0 out of 1 2024-04-07T16:16:28.107 INFO:tasks.cephadm.ceph_manager.ceph:creating pool_name unique_pool_0 2024-04-07T16:16:28.108 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 0702e3a8-f4f8-11ee-b649-cb9ed24678a4 -- ceph osd pool create unique_pool_0 16 16 erasure teuthologyprofile 2024-04-07T16:16:28.856 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:28 smithi144 bash[23522]: cluster 2024-04-07T16:16:26.682113+0000 mgr.y (mgr.14553) 162 : cluster 0 pgmap v114: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 10 KiB/s rd, 0 B/s wr, 14 op/s 2024-04-07T16:16:28.856 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:28 smithi144 bash[23522]: audit 2024-04-07T16:16:27.470196+0000 mon.a (mon.0) 875 : audit 1 from='client.? 172.21.15.100:0/475038168' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "teuthologyprofile", "profile": ["k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-07T16:16:28.856 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:28 smithi144 bash[23522]: cluster 2024-04-07T16:16:27.470435+0000 mon.a (mon.0) 876 : cluster 0 osdmap e70: 8 total, 8 up, 8 in 2024-04-07T16:16:28.937 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:28 smithi100 bash[31660]: cluster 2024-04-07T16:16:26.682113+0000 mgr.y (mgr.14553) 162 : cluster 0 pgmap v114: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 10 KiB/s rd, 0 B/s wr, 14 op/s 2024-04-07T16:16:28.937 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:28 smithi100 bash[31660]: audit 2024-04-07T16:16:27.470196+0000 mon.a (mon.0) 875 : audit 1 from='client.? 172.21.15.100:0/475038168' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "teuthologyprofile", "profile": ["k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-07T16:16:28.937 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:28 smithi100 bash[31660]: cluster 2024-04-07T16:16:27.470435+0000 mon.a (mon.0) 876 : cluster 0 osdmap e70: 8 total, 8 up, 8 in 2024-04-07T16:16:28.938 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:28 smithi100 bash[21346]: cluster 2024-04-07T16:16:26.682113+0000 mgr.y (mgr.14553) 162 : cluster 0 pgmap v114: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 10 KiB/s rd, 0 B/s wr, 14 op/s 2024-04-07T16:16:28.938 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:28 smithi100 bash[21346]: audit 2024-04-07T16:16:27.470196+0000 mon.a (mon.0) 875 : audit 1 from='client.? 172.21.15.100:0/475038168' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "teuthologyprofile", "profile": ["k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-07T16:16:28.938 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:28 smithi100 bash[21346]: cluster 2024-04-07T16:16:27.470435+0000 mon.a (mon.0) 876 : cluster 0 osdmap e70: 8 total, 8 up, 8 in 2024-04-07T16:16:30.856 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:30 smithi144 bash[23522]: cluster 2024-04-07T16:16:28.682733+0000 mgr.y (mgr.14553) 163 : cluster 0 pgmap v116: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 12 KiB/s rd, 0 B/s wr, 17 op/s 2024-04-07T16:16:30.937 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:30 smithi100 bash[21346]: cluster 2024-04-07T16:16:28.682733+0000 mgr.y (mgr.14553) 163 : cluster 0 pgmap v116: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 12 KiB/s rd, 0 B/s wr, 17 op/s 2024-04-07T16:16:30.937 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:30 smithi100 bash[31660]: cluster 2024-04-07T16:16:28.682733+0000 mgr.y (mgr.14553) 163 : cluster 0 pgmap v116: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 12 KiB/s rd, 0 B/s wr, 17 op/s 2024-04-07T16:16:32.201 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/mon.a/config 2024-04-07T16:16:32.201 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/mon.a/config 2024-04-07T16:16:32.687 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:32 smithi100 bash[21346]: cluster 2024-04-07T16:16:30.684391+0000 mgr.y (mgr.14553) 164 : cluster 0 pgmap v117: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:16:32.687 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:32 smithi100 bash[31660]: cluster 2024-04-07T16:16:30.684391+0000 mgr.y (mgr.14553) 164 : cluster 0 pgmap v117: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:16:32.856 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:32 smithi144 bash[23522]: cluster 2024-04-07T16:16:30.684391+0000 mgr.y (mgr.14553) 164 : cluster 0 pgmap v117: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:16:34.669 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:34 smithi100 bash[21346]: cluster 2024-04-07T16:16:32.685075+0000 mgr.y (mgr.14553) 165 : cluster 0 pgmap v118: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:16:34.669 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:34 smithi100 bash[31660]: cluster 2024-04-07T16:16:32.685075+0000 mgr.y (mgr.14553) 165 : cluster 0 pgmap v118: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:16:34.856 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:34 smithi144 bash[23522]: cluster 2024-04-07T16:16:32.685075+0000 mgr.y (mgr.14553) 165 : cluster 0 pgmap v118: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:16:35.437 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:16:35 smithi100 bash[21601]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:16:35] "GET /metrics HTTP/1.1" 200 33739 "" "Prometheus/2.43.0" 2024-04-07T16:16:36.075 INFO:teuthology.orchestra.run.smithi100.stdout:Initiating upgrade to quay.ceph.io/ceph-ci/ceph:adce92d056533f22dc1ee37e3141d098b046ec1b 2024-04-07T16:16:36.762 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:36 smithi100 bash[21346]: cluster 2024-04-07T16:16:34.685832+0000 mgr.y (mgr.14553) 166 : cluster 0 pgmap v119: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:16:36.762 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:36 smithi100 bash[21346]: audit 2024-04-07T16:16:36.090012+0000 mon.a (mon.0) 877 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:16:36.762 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:36 smithi100 bash[21346]: audit 2024-04-07T16:16:36.094136+0000 mon.b (mon.1) 235 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:16:36.762 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:36 smithi100 bash[21346]: audit 2024-04-07T16:16:36.096383+0000 mon.b (mon.1) 236 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:16:36.762 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:36 smithi100 bash[21346]: audit 2024-04-07T16:16:36.097418+0000 mon.b (mon.1) 237 : audit 1 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:16:36.762 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:36 smithi100 bash[21346]: audit 2024-04-07T16:16:36.108206+0000 mon.a (mon.0) 878 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:16:36.762 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:36 smithi100 bash[21346]: audit 2024-04-07T16:16:36.160386+0000 mon.a (mon.0) 879 : audit 1 from='client.? 172.21.15.100:0/2739888233' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_0", "pg_num": 16, "pgp_num": 16, "pool_type": "erasure", "erasure_code_profile": "teuthologyprofile"}]: dispatch 2024-04-07T16:16:36.763 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:36 smithi100 bash[31660]: cluster 2024-04-07T16:16:34.685832+0000 mgr.y (mgr.14553) 166 : cluster 0 pgmap v119: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:16:36.763 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:36 smithi100 bash[31660]: audit 2024-04-07T16:16:36.090012+0000 mon.a (mon.0) 877 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:16:36.763 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:36 smithi100 bash[31660]: audit 2024-04-07T16:16:36.094136+0000 mon.b (mon.1) 235 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:16:36.763 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:36 smithi100 bash[31660]: audit 2024-04-07T16:16:36.096383+0000 mon.b (mon.1) 236 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:16:36.763 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:36 smithi100 bash[31660]: audit 2024-04-07T16:16:36.097418+0000 mon.b (mon.1) 237 : audit 1 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:16:36.763 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:36 smithi100 bash[31660]: audit 2024-04-07T16:16:36.108206+0000 mon.a (mon.0) 878 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:16:36.763 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:36 smithi100 bash[31660]: audit 2024-04-07T16:16:36.160386+0000 mon.a (mon.0) 879 : audit 1 from='client.? 172.21.15.100:0/2739888233' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_0", "pg_num": 16, "pgp_num": 16, "pool_type": "erasure", "erasure_code_profile": "teuthologyprofile"}]: dispatch 2024-04-07T16:16:36.764 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0702e3a8-f4f8-11ee-b649-cb9ed24678a4 -e sha1=adce92d056533f22dc1ee37e3141d098b046ec1b -- bash -c 'while ceph orch upgrade status | jq '"'"'.in_progress'"'"' | grep true ; do ceph orch ps ; ceph versions ; sleep 30 ; done' 2024-04-07T16:16:36.856 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:36 smithi144 bash[23522]: cluster 2024-04-07T16:16:34.685832+0000 mgr.y (mgr.14553) 166 : cluster 0 pgmap v119: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:16:36.856 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:36 smithi144 bash[23522]: audit 2024-04-07T16:16:36.090012+0000 mon.a (mon.0) 877 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:16:36.856 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:36 smithi144 bash[23522]: audit 2024-04-07T16:16:36.094136+0000 mon.b (mon.1) 235 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:16:36.856 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:36 smithi144 bash[23522]: audit 2024-04-07T16:16:36.096383+0000 mon.b (mon.1) 236 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:16:36.856 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:36 smithi144 bash[23522]: audit 2024-04-07T16:16:36.097418+0000 mon.b (mon.1) 237 : audit 1 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:16:36.856 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:36 smithi144 bash[23522]: audit 2024-04-07T16:16:36.108206+0000 mon.a (mon.0) 878 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:16:36.856 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:36 smithi144 bash[23522]: audit 2024-04-07T16:16:36.160386+0000 mon.a (mon.0) 879 : audit 1 from='client.? 172.21.15.100:0/2739888233' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_0", "pg_num": 16, "pgp_num": 16, "pool_type": "erasure", "erasure_code_profile": "teuthologyprofile"}]: dispatch 2024-04-07T16:16:37.856 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:37 smithi144 bash[23522]: audit 2024-04-07T16:16:36.080119+0000 mgr.y (mgr.14553) 167 : audit 0 from='client.14997 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:adce92d056533f22dc1ee37e3141d098b046ec1b", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:16:37.856 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:37 smithi144 bash[23522]: cephadm 2024-04-07T16:16:36.081680+0000 mgr.y (mgr.14553) 168 : cephadm 1 Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:adce92d056533f22dc1ee37e3141d098b046ec1b 2024-04-07T16:16:37.856 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:37 smithi144 bash[23522]: cephadm 2024-04-07T16:16:36.166671+0000 mgr.y (mgr.14553) 169 : cephadm 1 Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:adce92d056533f22dc1ee37e3141d098b046ec1b 2024-04-07T16:16:37.937 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:37 smithi100 bash[31660]: audit 2024-04-07T16:16:36.080119+0000 mgr.y (mgr.14553) 167 : audit 0 from='client.14997 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:adce92d056533f22dc1ee37e3141d098b046ec1b", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:16:37.937 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:37 smithi100 bash[31660]: cephadm 2024-04-07T16:16:36.081680+0000 mgr.y (mgr.14553) 168 : cephadm 1 Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:adce92d056533f22dc1ee37e3141d098b046ec1b 2024-04-07T16:16:37.937 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:37 smithi100 bash[31660]: cephadm 2024-04-07T16:16:36.166671+0000 mgr.y (mgr.14553) 169 : cephadm 1 Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:adce92d056533f22dc1ee37e3141d098b046ec1b 2024-04-07T16:16:37.938 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:37 smithi100 bash[21346]: audit 2024-04-07T16:16:36.080119+0000 mgr.y (mgr.14553) 167 : audit 0 from='client.14997 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:adce92d056533f22dc1ee37e3141d098b046ec1b", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:16:37.938 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:37 smithi100 bash[21346]: cephadm 2024-04-07T16:16:36.081680+0000 mgr.y (mgr.14553) 168 : cephadm 1 Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:adce92d056533f22dc1ee37e3141d098b046ec1b 2024-04-07T16:16:37.938 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:37 smithi100 bash[21346]: cephadm 2024-04-07T16:16:36.166671+0000 mgr.y (mgr.14553) 169 : cephadm 1 Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:adce92d056533f22dc1ee37e3141d098b046ec1b 2024-04-07T16:16:38.117 INFO:teuthology.orchestra.run.smithi100.stderr:pool 'unique_pool_0' created 2024-04-07T16:16:38.856 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:38 smithi144 bash[23522]: cluster 2024-04-07T16:16:36.686535+0000 mgr.y (mgr.14553) 170 : cluster 0 pgmap v120: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:16:38.856 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:38 smithi144 bash[23522]: cluster 2024-04-07T16:16:37.584760+0000 mon.a (mon.0) 880 : cluster 0 osdmap e71: 8 total, 8 up, 8 in 2024-04-07T16:16:38.856 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:38 smithi144 bash[23522]: audit 2024-04-07T16:16:37.702108+0000 mon.b (mon.1) 238 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:16:38.856 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:38 smithi144 bash[23522]: audit 2024-04-07T16:16:38.118296+0000 mon.a (mon.0) 881 : audit 1 from='client.? 172.21.15.100:0/2739888233' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_0", "pg_num": 16, "pgp_num": 16, "pool_type": "erasure", "erasure_code_profile": "teuthologyprofile"}]': finished 2024-04-07T16:16:38.856 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:38 smithi144 bash[23522]: cluster 2024-04-07T16:16:38.118718+0000 mon.a (mon.0) 882 : cluster 0 osdmap e72: 8 total, 8 up, 8 in 2024-04-07T16:16:38.937 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:38 smithi100 bash[21346]: cluster 2024-04-07T16:16:36.686535+0000 mgr.y (mgr.14553) 170 : cluster 0 pgmap v120: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:16:38.937 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:38 smithi100 bash[21346]: cluster 2024-04-07T16:16:37.584760+0000 mon.a (mon.0) 880 : cluster 0 osdmap e71: 8 total, 8 up, 8 in 2024-04-07T16:16:38.937 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:38 smithi100 bash[21346]: audit 2024-04-07T16:16:37.702108+0000 mon.b (mon.1) 238 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:16:38.937 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:38 smithi100 bash[21346]: audit 2024-04-07T16:16:38.118296+0000 mon.a (mon.0) 881 : audit 1 from='client.? 172.21.15.100:0/2739888233' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_0", "pg_num": 16, "pgp_num": 16, "pool_type": "erasure", "erasure_code_profile": "teuthologyprofile"}]': finished 2024-04-07T16:16:38.937 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:38 smithi100 bash[21346]: cluster 2024-04-07T16:16:38.118718+0000 mon.a (mon.0) 882 : cluster 0 osdmap e72: 8 total, 8 up, 8 in 2024-04-07T16:16:38.938 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:38 smithi100 bash[31660]: cluster 2024-04-07T16:16:36.686535+0000 mgr.y (mgr.14553) 170 : cluster 0 pgmap v120: 105 pgs: 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:16:38.938 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:38 smithi100 bash[31660]: cluster 2024-04-07T16:16:37.584760+0000 mon.a (mon.0) 880 : cluster 0 osdmap e71: 8 total, 8 up, 8 in 2024-04-07T16:16:38.938 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:38 smithi100 bash[31660]: audit 2024-04-07T16:16:37.702108+0000 mon.b (mon.1) 238 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:16:38.938 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:38 smithi100 bash[31660]: audit 2024-04-07T16:16:38.118296+0000 mon.a (mon.0) 881 : audit 1 from='client.? 172.21.15.100:0/2739888233' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_0", "pg_num": 16, "pgp_num": 16, "pool_type": "erasure", "erasure_code_profile": "teuthologyprofile"}]': finished 2024-04-07T16:16:38.938 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:38 smithi100 bash[31660]: cluster 2024-04-07T16:16:38.118718+0000 mon.a (mon.0) 882 : cluster 0 osdmap e72: 8 total, 8 up, 8 in 2024-04-07T16:16:39.138 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 0702e3a8-f4f8-11ee-b649-cb9ed24678a4 -- ceph osd pool application enable unique_pool_0 rados --yes-i-really-mean-it || true 2024-04-07T16:16:40.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:40 smithi100 bash[21346]: cluster 2024-04-07T16:16:38.687507+0000 mgr.y (mgr.14553) 171 : cluster 0 pgmap v123: 121 pgs: 16 unknown, 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:16:40.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:40 smithi100 bash[21346]: cluster 2024-04-07T16:16:39.127336+0000 mon.a (mon.0) 883 : cluster 0 osdmap e73: 8 total, 8 up, 8 in 2024-04-07T16:16:40.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:40 smithi100 bash[21346]: cluster 2024-04-07T16:16:39.596104+0000 mon.a (mon.0) 884 : cluster 3 Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T16:16:40.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:40 smithi100 bash[31660]: cluster 2024-04-07T16:16:38.687507+0000 mgr.y (mgr.14553) 171 : cluster 0 pgmap v123: 121 pgs: 16 unknown, 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:16:40.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:40 smithi100 bash[31660]: cluster 2024-04-07T16:16:39.127336+0000 mon.a (mon.0) 883 : cluster 0 osdmap e73: 8 total, 8 up, 8 in 2024-04-07T16:16:40.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:40 smithi100 bash[31660]: cluster 2024-04-07T16:16:39.596104+0000 mon.a (mon.0) 884 : cluster 3 Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T16:16:40.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:40 smithi144 bash[23522]: cluster 2024-04-07T16:16:38.687507+0000 mgr.y (mgr.14553) 171 : cluster 0 pgmap v123: 121 pgs: 16 unknown, 105 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:16:40.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:40 smithi144 bash[23522]: cluster 2024-04-07T16:16:39.127336+0000 mon.a (mon.0) 883 : cluster 0 osdmap e73: 8 total, 8 up, 8 in 2024-04-07T16:16:40.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:40 smithi144 bash[23522]: cluster 2024-04-07T16:16:39.596104+0000 mon.a (mon.0) 884 : cluster 3 Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T16:16:41.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:41 smithi100 bash[21346]: cluster 2024-04-07T16:16:40.124726+0000 mon.a (mon.0) 885 : cluster 0 osdmap e74: 8 total, 8 up, 8 in 2024-04-07T16:16:41.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:41 smithi100 bash[31660]: cluster 2024-04-07T16:16:40.124726+0000 mon.a (mon.0) 885 : cluster 0 osdmap e74: 8 total, 8 up, 8 in 2024-04-07T16:16:41.481 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/mon.a/config 2024-04-07T16:16:41.605 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:41 smithi144 bash[23522]: cluster 2024-04-07T16:16:40.124726+0000 mon.a (mon.0) 885 : cluster 0 osdmap e74: 8 total, 8 up, 8 in 2024-04-07T16:16:42.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:42 smithi100 bash[21346]: cluster 2024-04-07T16:16:40.688323+0000 mgr.y (mgr.14553) 172 : cluster 0 pgmap v126: 121 pgs: 16 creating+peering, 105 active+clean; 580 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:16:42.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:42 smithi100 bash[31660]: cluster 2024-04-07T16:16:40.688323+0000 mgr.y (mgr.14553) 172 : cluster 0 pgmap v126: 121 pgs: 16 creating+peering, 105 active+clean; 580 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:16:42.605 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:42 smithi144 bash[23522]: cluster 2024-04-07T16:16:40.688323+0000 mgr.y (mgr.14553) 172 : cluster 0 pgmap v126: 121 pgs: 16 creating+peering, 105 active+clean; 580 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:16:43.521 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/mon.a/config 2024-04-07T16:16:44.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:44 smithi100 bash[21346]: cluster 2024-04-07T16:16:42.688800+0000 mgr.y (mgr.14553) 173 : cluster 0 pgmap v127: 121 pgs: 16 creating+peering, 105 active+clean; 580 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:16:44.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:44 smithi100 bash[31660]: cluster 2024-04-07T16:16:42.688800+0000 mgr.y (mgr.14553) 173 : cluster 0 pgmap v127: 121 pgs: 16 creating+peering, 105 active+clean; 580 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:16:44.605 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:44 smithi144 bash[23522]: cluster 2024-04-07T16:16:42.688800+0000 mgr.y (mgr.14553) 173 : cluster 0 pgmap v127: 121 pgs: 16 creating+peering, 105 active+clean; 580 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:16:45.333 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:16:45 smithi100 bash[21601]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:16:45] "GET /metrics HTTP/1.1" 200 33741 "" "Prometheus/2.43.0" 2024-04-07T16:16:45.886 INFO:teuthology.orchestra.run.smithi100.stdout:true 2024-04-07T16:16:46.259 INFO:teuthology.orchestra.run.smithi100.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-07T16:16:46.259 INFO:teuthology.orchestra.run.smithi100.stdout:alertmanager.a smithi100 *:9093,9094 running (2m) 2m ago 3m 13.7M - 0.25.0 c8568f914cd2 5143075e7d26 2024-04-07T16:16:46.259 INFO:teuthology.orchestra.run.smithi100.stdout:grafana.a smithi144 *:3000 running (2m) 2m ago 2m 29.2M - 9.4.7 954c08fa6188 7180c0708acd 2024-04-07T16:16:46.259 INFO:teuthology.orchestra.run.smithi100.stdout:mds.a smithi100 running (4m) 2m ago 4m 12.4M - 18.2.2-1171-g267c8ff5 8a4a01e7186c 92593b0c28ae 2024-04-07T16:16:46.259 INFO:teuthology.orchestra.run.smithi100.stdout:mds.b smithi144 running (4m) 2m ago 4m 14.2M - 18.2.2-1171-g267c8ff5 8a4a01e7186c 3df2ee3e17a4 2024-04-07T16:16:46.259 INFO:teuthology.orchestra.run.smithi100.stdout:mgr.x smithi144 *:8443,8765 running (11m) 2m ago 11m 435M - 18.2.2-1171-g267c8ff5 8a4a01e7186c b7feffac74d1 2024-04-07T16:16:46.259 INFO:teuthology.orchestra.run.smithi100.stdout:mgr.y smithi100 *:9283,8765 running (14m) 2m ago 14m 484M - 18.2.2-1171-g267c8ff5 8a4a01e7186c da1d683b9737 2024-04-07T16:16:46.259 INFO:teuthology.orchestra.run.smithi100.stdout:mon.a smithi100 running (14m) 2m ago 14m 46.3M 2048M 18.2.2-1171-g267c8ff5 8a4a01e7186c ec8833f59ea0 2024-04-07T16:16:46.259 INFO:teuthology.orchestra.run.smithi100.stdout:mon.b smithi144 running (12m) 2m ago 12m 39.7M 2048M 18.2.2-1171-g267c8ff5 8a4a01e7186c 8dd687691b2e 2024-04-07T16:16:46.259 INFO:teuthology.orchestra.run.smithi100.stdout:mon.c smithi100 running (12m) 2m ago 12m 37.7M 2048M 18.2.2-1171-g267c8ff5 8a4a01e7186c a6de0151a3ba 2024-04-07T16:16:46.259 INFO:teuthology.orchestra.run.smithi100.stdout:node-exporter.a smithi100 *:9100 running (3m) 2m ago 3m 3116k - 1.5.0 0da6a335fe13 4bce264c1b26 2024-04-07T16:16:46.260 INFO:teuthology.orchestra.run.smithi100.stdout:node-exporter.b smithi144 *:9100 running (3m) 2m ago 3m 3151k - 1.5.0 0da6a335fe13 b6963413c560 2024-04-07T16:16:46.260 INFO:teuthology.orchestra.run.smithi100.stdout:osd.0 smithi100 running (11m) 2m ago 11m 45.7M 3036M 18.2.2-1171-g267c8ff5 8a4a01e7186c 188deb247979 2024-04-07T16:16:46.260 INFO:teuthology.orchestra.run.smithi100.stdout:osd.1 smithi100 running (10m) 2m ago 10m 44.6M 3036M 18.2.2-1171-g267c8ff5 8a4a01e7186c 7f57ca90894c 2024-04-07T16:16:46.260 INFO:teuthology.orchestra.run.smithi100.stdout:osd.2 smithi100 running (9m) 2m ago 9m 52.2M 3036M 18.2.2-1171-g267c8ff5 8a4a01e7186c f3912ddcc8cc 2024-04-07T16:16:46.260 INFO:teuthology.orchestra.run.smithi100.stdout:osd.3 smithi100 running (8m) 2m ago 8m 41.7M 3036M 18.2.2-1171-g267c8ff5 8a4a01e7186c b41004090c63 2024-04-07T16:16:46.260 INFO:teuthology.orchestra.run.smithi100.stdout:osd.4 smithi144 running (7m) 2m ago 7m 50.6M 3036M 18.2.2-1171-g267c8ff5 8a4a01e7186c 8c73b6b4dc13 2024-04-07T16:16:46.260 INFO:teuthology.orchestra.run.smithi100.stdout:osd.5 smithi144 running (7m) 2m ago 7m 42.6M 3036M 18.2.2-1171-g267c8ff5 8a4a01e7186c 3bfa0cf6f6cb 2024-04-07T16:16:46.260 INFO:teuthology.orchestra.run.smithi100.stdout:osd.6 smithi144 running (6m) 2m ago 6m 42.5M 3036M 18.2.2-1171-g267c8ff5 8a4a01e7186c 32b7f95d77f0 2024-04-07T16:16:46.260 INFO:teuthology.orchestra.run.smithi100.stdout:osd.7 smithi144 running (5m) 2m ago 5m 49.8M 3036M 18.2.2-1171-g267c8ff5 8a4a01e7186c 45f7709e576a 2024-04-07T16:16:46.260 INFO:teuthology.orchestra.run.smithi100.stdout:prometheus.a smithi144 *:9095 running (2m) 2m ago 4m 23.0M - 2.43.0 a07b618ecd1d e456da14a41a 2024-04-07T16:16:46.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:46 smithi100 bash[21346]: cluster 2024-04-07T16:16:44.690237+0000 mgr.y (mgr.14553) 174 : cluster 0 pgmap v128: 121 pgs: 121 active+clean; 580 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:16:46.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:46 smithi100 bash[31660]: cluster 2024-04-07T16:16:44.690237+0000 mgr.y (mgr.14553) 174 : cluster 0 pgmap v128: 121 pgs: 121 active+clean; 580 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:16:46.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:46 smithi144 bash[23522]: cluster 2024-04-07T16:16:44.690237+0000 mgr.y (mgr.14553) 174 : cluster 0 pgmap v128: 121 pgs: 121 active+clean; 580 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:16:46.715 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-04-07T16:16:46.716 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": { 2024-04-07T16:16:46.716 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph version 18.2.2-1171-g267c8ff5 (267c8ff5b1ff5815581b3b2381d19360e081b37b) reef (stable)": 3 2024-04-07T16:16:46.716 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-07T16:16:46.716 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": { 2024-04-07T16:16:46.716 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph version 18.2.2-1171-g267c8ff5 (267c8ff5b1ff5815581b3b2381d19360e081b37b) reef (stable)": 2 2024-04-07T16:16:46.716 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-07T16:16:46.716 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": { 2024-04-07T16:16:46.716 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph version 18.2.2-1171-g267c8ff5 (267c8ff5b1ff5815581b3b2381d19360e081b37b) reef (stable)": 8 2024-04-07T16:16:46.716 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-07T16:16:46.716 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": { 2024-04-07T16:16:46.716 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph version 18.2.2-1171-g267c8ff5 (267c8ff5b1ff5815581b3b2381d19360e081b37b) reef (stable)": 2 2024-04-07T16:16:46.716 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-07T16:16:46.716 INFO:teuthology.orchestra.run.smithi100.stdout: "overall": { 2024-04-07T16:16:46.716 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph version 18.2.2-1171-g267c8ff5 (267c8ff5b1ff5815581b3b2381d19360e081b37b) reef (stable)": 15 2024-04-07T16:16:46.717 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-04-07T16:16:46.717 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-04-07T16:16:47.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:47 smithi100 bash[21346]: audit 2024-04-07T16:16:45.888717+0000 mgr.y (mgr.14553) 175 : audit 0 from='client.15009 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:16:47.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:47 smithi100 bash[21346]: audit 2024-04-07T16:16:46.260377+0000 mgr.y (mgr.14553) 176 : audit 0 from='client.15015 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:16:47.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:47 smithi100 bash[21346]: audit 2024-04-07T16:16:46.730643+0000 mon.c (mon.2) 51 : audit 0 from='client.? 172.21.15.100:0/4205745129' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:16:47.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:47 smithi100 bash[31660]: audit 2024-04-07T16:16:45.888717+0000 mgr.y (mgr.14553) 175 : audit 0 from='client.15009 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:16:47.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:47 smithi100 bash[31660]: audit 2024-04-07T16:16:46.260377+0000 mgr.y (mgr.14553) 176 : audit 0 from='client.15015 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:16:47.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:47 smithi100 bash[31660]: audit 2024-04-07T16:16:46.730643+0000 mon.c (mon.2) 51 : audit 0 from='client.? 172.21.15.100:0/4205745129' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:16:47.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:47 smithi144 bash[23522]: audit 2024-04-07T16:16:45.888717+0000 mgr.y (mgr.14553) 175 : audit 0 from='client.15009 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:16:47.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:47 smithi144 bash[23522]: audit 2024-04-07T16:16:46.260377+0000 mgr.y (mgr.14553) 176 : audit 0 from='client.15015 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:16:47.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:47 smithi144 bash[23522]: audit 2024-04-07T16:16:46.730643+0000 mon.c (mon.2) 51 : audit 0 from='client.? 172.21.15.100:0/4205745129' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:16:48.171 INFO:teuthology.orchestra.run.smithi100.stderr:enabled application 'rados' on pool 'unique_pool_0' 2024-04-07T16:16:48.408 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:48 smithi100 bash[21346]: cluster 2024-04-07T16:16:46.690762+0000 mgr.y (mgr.14553) 177 : cluster 0 pgmap v129: 121 pgs: 121 active+clean; 580 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:16:48.408 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:48 smithi100 bash[21346]: audit 2024-04-07T16:16:47.596234+0000 mon.a (mon.0) 886 : audit 1 from='client.? 172.21.15.100:0/2274262938' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_0", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:16:48.408 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:48 smithi100 bash[31660]: cluster 2024-04-07T16:16:46.690762+0000 mgr.y (mgr.14553) 177 : cluster 0 pgmap v129: 121 pgs: 121 active+clean; 580 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:16:48.408 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:48 smithi100 bash[31660]: audit 2024-04-07T16:16:47.596234+0000 mon.a (mon.0) 886 : audit 1 from='client.? 172.21.15.100:0/2274262938' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_0", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:16:48.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:48 smithi144 bash[23522]: cluster 2024-04-07T16:16:46.690762+0000 mgr.y (mgr.14553) 177 : cluster 0 pgmap v129: 121 pgs: 121 active+clean; 580 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:16:48.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:48 smithi144 bash[23522]: audit 2024-04-07T16:16:47.596234+0000 mon.a (mon.0) 886 : audit 1 from='client.? 172.21.15.100:0/2274262938' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_0", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:16:49.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:49 smithi100 bash[21346]: audit 2024-04-07T16:16:48.179178+0000 mon.a (mon.0) 887 : audit 1 from='client.? 172.21.15.100:0/2274262938' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_0", "app": "rados", "yes_i_really_mean_it": true}]': finished 2024-04-07T16:16:49.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:49 smithi100 bash[21346]: cluster 2024-04-07T16:16:48.179351+0000 mon.a (mon.0) 888 : cluster 0 osdmap e75: 8 total, 8 up, 8 in 2024-04-07T16:16:49.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:49 smithi100 bash[31660]: audit 2024-04-07T16:16:48.179178+0000 mon.a (mon.0) 887 : audit 1 from='client.? 172.21.15.100:0/2274262938' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_0", "app": "rados", "yes_i_really_mean_it": true}]': finished 2024-04-07T16:16:49.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:49 smithi100 bash[31660]: cluster 2024-04-07T16:16:48.179351+0000 mon.a (mon.0) 888 : cluster 0 osdmap e75: 8 total, 8 up, 8 in 2024-04-07T16:16:49.605 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:49 smithi144 bash[23522]: audit 2024-04-07T16:16:48.179178+0000 mon.a (mon.0) 887 : audit 1 from='client.? 172.21.15.100:0/2274262938' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_0", "app": "rados", "yes_i_really_mean_it": true}]': finished 2024-04-07T16:16:49.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:49 smithi144 bash[23522]: cluster 2024-04-07T16:16:48.179351+0000 mon.a (mon.0) 888 : cluster 0 osdmap e75: 8 total, 8 up, 8 in 2024-04-07T16:16:50.158 DEBUG:teuthology.orchestra.run.smithi100:> CEPH_CLIENT_ID=0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_rados --no-omap --ec-pool --max-ops 4000 --objects 50 --max-in-flight 16 --size 4000000 --min-stride-size 400000 --max-stride-size 800000 --max-seconds 0 --op read 100 --op write 0 --op delete 50 --op snap_create 50 --op snap_remove 50 --op rollback 50 --op setattr 25 --op rmattr 25 --op copy_from 50 --op append 100 --pool unique_pool_0 2024-04-07T16:16:50.181 INFO:tasks.rados.rados.0.smithi100.stdout:adding op weight read -> 100 2024-04-07T16:16:50.181 INFO:tasks.rados.rados.0.smithi100.stdout:adding op weight delete -> 50 2024-04-07T16:16:50.181 INFO:tasks.rados.rados.0.smithi100.stdout:adding op weight snap_create -> 50 2024-04-07T16:16:50.181 INFO:tasks.rados.rados.0.smithi100.stdout:adding op weight snap_remove -> 50 2024-04-07T16:16:50.181 INFO:tasks.rados.rados.0.smithi100.stdout:adding op weight rollback -> 50 2024-04-07T16:16:50.181 INFO:tasks.rados.rados.0.smithi100.stdout:adding op weight setattr -> 25 2024-04-07T16:16:50.181 INFO:tasks.rados.rados.0.smithi100.stdout:adding op weight rmattr -> 25 2024-04-07T16:16:50.181 INFO:tasks.rados.rados.0.smithi100.stdout:adding op weight copy_from -> 50 2024-04-07T16:16:50.181 INFO:tasks.rados.rados.0.smithi100.stdout:adding op weight append -> 100 2024-04-07T16:16:50.181 INFO:tasks.rados.rados.0.smithi100.stdout:ceph version 18.2.2-1171-g267c8ff5 (267c8ff5b1ff5815581b3b2381d19360e081b37b) reef (stable) 2024-04-07T16:16:50.181 INFO:tasks.rados.rados.0.smithi100.stdout:Configuration: 2024-04-07T16:16:50.181 INFO:tasks.rados.rados.0.smithi100.stdout: Number of operations: 4000 2024-04-07T16:16:50.182 INFO:tasks.rados.rados.0.smithi100.stdout: Number of objects: 50 2024-04-07T16:16:50.182 INFO:tasks.rados.rados.0.smithi100.stdout: Max in flight operations: 16 2024-04-07T16:16:50.182 INFO:tasks.rados.rados.0.smithi100.stdout: Object size (in bytes): 4000000 2024-04-07T16:16:50.182 INFO:tasks.rados.rados.0.smithi100.stdout: Write stride min: 400000 2024-04-07T16:16:50.182 INFO:tasks.rados.rados.0.smithi100.stdout: Write stride max: 800000 2024-04-07T16:16:50.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:50 smithi100 bash[21346]: cluster 2024-04-07T16:16:48.691509+0000 mgr.y (mgr.14553) 178 : cluster 0 pgmap v131: 121 pgs: 121 active+clean; 580 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:16:50.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:50 smithi100 bash[21346]: cluster 2024-04-07T16:16:49.172178+0000 mon.a (mon.0) 889 : cluster 1 Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T16:16:50.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:50 smithi100 bash[21346]: cluster 2024-04-07T16:16:49.172219+0000 mon.a (mon.0) 890 : cluster 1 Cluster is now healthy 2024-04-07T16:16:50.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:50 smithi100 bash[31660]: cluster 2024-04-07T16:16:48.691509+0000 mgr.y (mgr.14553) 178 : cluster 0 pgmap v131: 121 pgs: 121 active+clean; 580 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:16:50.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:50 smithi100 bash[31660]: cluster 2024-04-07T16:16:49.172178+0000 mon.a (mon.0) 889 : cluster 1 Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T16:16:50.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:50 smithi100 bash[31660]: cluster 2024-04-07T16:16:49.172219+0000 mon.a (mon.0) 890 : cluster 1 Cluster is now healthy 2024-04-07T16:16:50.605 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:50 smithi144 bash[23522]: cluster 2024-04-07T16:16:48.691509+0000 mgr.y (mgr.14553) 178 : cluster 0 pgmap v131: 121 pgs: 121 active+clean; 580 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:16:50.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:50 smithi144 bash[23522]: cluster 2024-04-07T16:16:49.172178+0000 mon.a (mon.0) 889 : cluster 1 Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T16:16:50.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:50 smithi144 bash[23522]: cluster 2024-04-07T16:16:49.172219+0000 mon.a (mon.0) 890 : cluster 1 Cluster is now healthy 2024-04-07T16:16:51.189 INFO:tasks.rados.rados.0.smithi100.stdout:1: write initial oid 1 2024-04-07T16:16:51.189 INFO:tasks.rados.rados.0.smithi100.stdout:1: seq_num 0 ranges {0=24576} 2024-04-07T16:16:51.190 INFO:tasks.rados.rados.0.smithi100.stdout:1: writing smithi10066867-1 from 0 to 24576 tid 1 2024-04-07T16:16:51.190 INFO:tasks.rados.rados.0.smithi100.stdout:2: write initial oid 2 2024-04-07T16:16:51.190 INFO:tasks.rados.rados.0.smithi100.stdout:2: seq_num 1 ranges {0=24576} 2024-04-07T16:16:51.191 INFO:tasks.rados.rados.0.smithi100.stdout:2: writing smithi10066867-2 from 0 to 24576 tid 1 2024-04-07T16:16:51.191 INFO:tasks.rados.rados.0.smithi100.stdout:3: write initial oid 3 2024-04-07T16:16:51.191 INFO:tasks.rados.rados.0.smithi100.stdout:3: seq_num 2 ranges {0=40960} 2024-04-07T16:16:51.191 INFO:tasks.rados.rados.0.smithi100.stdout:3: writing smithi10066867-3 from 0 to 40960 tid 1 2024-04-07T16:16:51.192 INFO:tasks.rados.rados.0.smithi100.stdout:4: write initial oid 4 2024-04-07T16:16:51.192 INFO:tasks.rados.rados.0.smithi100.stdout:4: seq_num 3 ranges {0=57344} 2024-04-07T16:16:51.192 INFO:tasks.rados.rados.0.smithi100.stdout:4: writing smithi10066867-4 from 0 to 57344 tid 1 2024-04-07T16:16:51.192 INFO:tasks.rados.rados.0.smithi100.stdout:5: write initial oid 5 2024-04-07T16:16:51.192 INFO:tasks.rados.rados.0.smithi100.stdout:5: seq_num 4 ranges {0=73728} 2024-04-07T16:16:51.193 INFO:tasks.rados.rados.0.smithi100.stdout:5: writing smithi10066867-5 from 0 to 73728 tid 1 2024-04-07T16:16:51.193 INFO:tasks.rados.rados.0.smithi100.stdout:6: write initial oid 6 2024-04-07T16:16:51.193 INFO:tasks.rados.rados.0.smithi100.stdout:6: seq_num 5 ranges {0=90112} 2024-04-07T16:16:51.194 INFO:tasks.rados.rados.0.smithi100.stdout:6: writing smithi10066867-6 from 0 to 90112 tid 1 2024-04-07T16:16:51.194 INFO:tasks.rados.rados.0.smithi100.stdout:7: write initial oid 7 2024-04-07T16:16:51.194 INFO:tasks.rados.rados.0.smithi100.stdout:7: seq_num 6 ranges {0=106496} 2024-04-07T16:16:51.196 INFO:tasks.rados.rados.0.smithi100.stdout:7: writing smithi10066867-7 from 0 to 106496 tid 1 2024-04-07T16:16:51.196 INFO:tasks.rados.rados.0.smithi100.stdout:8: write initial oid 8 2024-04-07T16:16:51.196 INFO:tasks.rados.rados.0.smithi100.stdout:8: seq_num 7 ranges {0=122880} 2024-04-07T16:16:51.198 INFO:tasks.rados.rados.0.smithi100.stdout:8: writing smithi10066867-8 from 0 to 122880 tid 1 2024-04-07T16:16:51.198 INFO:tasks.rados.rados.0.smithi100.stdout:9: write initial oid 9 2024-04-07T16:16:51.198 INFO:tasks.rados.rados.0.smithi100.stdout:1: finishing write tid 1 to smithi10066867-1 2024-04-07T16:16:51.198 INFO:tasks.rados.rados.0.smithi100.stdout:9: seq_num 8 ranges {0=139264} 2024-04-07T16:16:51.199 INFO:tasks.rados.rados.0.smithi100.stdout:9: writing smithi10066867-9 from 0 to 139264 tid 1 2024-04-07T16:16:51.199 INFO:tasks.rados.rados.0.smithi100.stdout:10: write initial oid 10 2024-04-07T16:16:51.199 INFO:tasks.rados.rados.0.smithi100.stdout:10: seq_num 9 ranges {0=155648} 2024-04-07T16:16:51.201 INFO:tasks.rados.rados.0.smithi100.stdout:10: writing smithi10066867-10 from 0 to 155648 tid 1 2024-04-07T16:16:51.201 INFO:tasks.rados.rados.0.smithi100.stdout:11: write initial oid 11 2024-04-07T16:16:51.201 INFO:tasks.rados.rados.0.smithi100.stdout:11: seq_num 10 ranges {0=172032} 2024-04-07T16:16:51.202 INFO:tasks.rados.rados.0.smithi100.stdout:11: writing smithi10066867-11 from 0 to 172032 tid 1 2024-04-07T16:16:51.202 INFO:tasks.rados.rados.0.smithi100.stdout:12: write initial oid 12 2024-04-07T16:16:51.203 INFO:tasks.rados.rados.0.smithi100.stdout:1: finishing write tid 2 to smithi10066867-1 2024-04-07T16:16:51.203 INFO:tasks.rados.rados.0.smithi100.stdout:12: seq_num 11 ranges {0=188416} 2024-04-07T16:16:51.204 INFO:tasks.rados.rados.0.smithi100.stdout:12: writing smithi10066867-12 from 0 to 188416 tid 1 2024-04-07T16:16:51.204 INFO:tasks.rados.rados.0.smithi100.stdout:13: write initial oid 13 2024-04-07T16:16:51.204 INFO:tasks.rados.rados.0.smithi100.stdout:13: seq_num 12 ranges {0=204800} 2024-04-07T16:16:51.206 INFO:tasks.rados.rados.0.smithi100.stdout:13: writing smithi10066867-13 from 0 to 204800 tid 1 2024-04-07T16:16:51.206 INFO:tasks.rados.rados.0.smithi100.stdout:5: finishing write tid 1 to smithi10066867-5 2024-04-07T16:16:51.206 INFO:tasks.rados.rados.0.smithi100.stdout:5: finishing write tid 2 to smithi10066867-5 2024-04-07T16:16:51.206 INFO:tasks.rados.rados.0.smithi100.stdout:14: write initial oid 14 2024-04-07T16:16:51.206 INFO:tasks.rados.rados.0.smithi100.stdout:14: seq_num 13 ranges {0=221184} 2024-04-07T16:16:51.208 INFO:tasks.rados.rados.0.smithi100.stdout:14: writing smithi10066867-14 from 0 to 221184 tid 1 2024-04-07T16:16:51.208 INFO:tasks.rados.rados.0.smithi100.stdout:15: write initial oid 15 2024-04-07T16:16:51.208 INFO:tasks.rados.rados.0.smithi100.stdout:15: seq_num 14 ranges {0=237568} 2024-04-07T16:16:51.211 INFO:tasks.rados.rados.0.smithi100.stdout:15: writing smithi10066867-15 from 0 to 237568 tid 1 2024-04-07T16:16:51.211 INFO:tasks.rados.rados.0.smithi100.stdout:16: write initial oid 16 2024-04-07T16:16:51.211 INFO:tasks.rados.rados.0.smithi100.stdout:16: seq_num 15 ranges {0=253952} 2024-04-07T16:16:51.214 INFO:tasks.rados.rados.0.smithi100.stdout:16: writing smithi10066867-16 from 0 to 253952 tid 1 2024-04-07T16:16:51.214 INFO:tasks.rados.rados.0.smithi100.stdout: waiting on 16 2024-04-07T16:16:51.214 INFO:tasks.rados.rados.0.smithi100.stdout:3: finishing write tid 1 to smithi10066867-3 2024-04-07T16:16:51.214 INFO:tasks.rados.rados.0.smithi100.stdout:3: finishing write tid 2 to smithi10066867-3 2024-04-07T16:16:51.214 INFO:tasks.rados.rados.0.smithi100.stdout:5: finishing write tid 3 to smithi10066867-5 2024-04-07T16:16:51.214 INFO:tasks.rados.rados.0.smithi100.stdout:5: oid 5 updating version 0 to 1 2024-04-07T16:16:51.214 INFO:tasks.rados.rados.0.smithi100.stdout:5: oid 5 updating version 1 to 2 2024-04-07T16:16:51.214 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 5 v 2 (ObjNum 4 snap 0 seq_num 4) dirty exists 2024-04-07T16:16:51.214 INFO:tasks.rados.rados.0.smithi100.stdout:5: left oid 5 (ObjNum 4 snap 0 seq_num 4) 2024-04-07T16:16:51.214 INFO:tasks.rados.rados.0.smithi100.stdout:9: finishing write tid 1 to smithi10066867-9 2024-04-07T16:16:51.215 INFO:tasks.rados.rados.0.smithi100.stdout:3: finishing write tid 3 to smithi10066867-3 2024-04-07T16:16:51.215 INFO:tasks.rados.rados.0.smithi100.stdout:3: oid 3 updating version 0 to 1 2024-04-07T16:16:51.215 INFO:tasks.rados.rados.0.smithi100.stdout:3: oid 3 updating version 1 to 2 2024-04-07T16:16:51.215 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 3 v 2 (ObjNum 2 snap 0 seq_num 2) dirty exists 2024-04-07T16:16:51.215 INFO:tasks.rados.rados.0.smithi100.stdout:3: left oid 3 (ObjNum 2 snap 0 seq_num 2) 2024-04-07T16:16:51.215 INFO:tasks.rados.rados.0.smithi100.stdout:2: finishing write tid 1 to smithi10066867-2 2024-04-07T16:16:51.215 INFO:tasks.rados.rados.0.smithi100.stdout:2: finishing write tid 2 to smithi10066867-2 2024-04-07T16:16:51.215 INFO:tasks.rados.rados.0.smithi100.stdout:9: finishing write tid 2 to smithi10066867-9 2024-04-07T16:16:51.215 INFO:tasks.rados.rados.0.smithi100.stdout:1: finishing write tid 3 to smithi10066867-1 2024-04-07T16:16:51.215 INFO:tasks.rados.rados.0.smithi100.stdout:1: oid 1 updating version 0 to 1 2024-04-07T16:16:51.215 INFO:tasks.rados.rados.0.smithi100.stdout:1: oid 1 updating version 1 to 2 2024-04-07T16:16:51.215 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 1 v 2 (ObjNum 0 snap 0 seq_num 0) dirty exists 2024-04-07T16:16:51.215 INFO:tasks.rados.rados.0.smithi100.stdout:1: left oid 1 (ObjNum 0 snap 0 seq_num 0) 2024-04-07T16:16:51.215 INFO:tasks.rados.rados.0.smithi100.stdout:10: finishing write tid 1 to smithi10066867-10 2024-04-07T16:16:51.215 INFO:tasks.rados.rados.0.smithi100.stdout:2: finishing write tid 3 to smithi10066867-2 2024-04-07T16:16:51.215 INFO:tasks.rados.rados.0.smithi100.stdout:2: oid 2 updating version 0 to 2 2024-04-07T16:16:51.216 INFO:tasks.rados.rados.0.smithi100.stdout:2: oid 2 version 2 is already newer than 1 2024-04-07T16:16:51.216 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 2 v 2 (ObjNum 1 snap 0 seq_num 1) dirty exists 2024-04-07T16:16:51.216 INFO:tasks.rados.rados.0.smithi100.stdout:2: left oid 2 (ObjNum 1 snap 0 seq_num 1) 2024-04-07T16:16:51.216 INFO:tasks.rados.rados.0.smithi100.stdout:6: finishing write tid 1 to smithi10066867-6 2024-04-07T16:16:51.216 INFO:tasks.rados.rados.0.smithi100.stdout:6: finishing write tid 2 to smithi10066867-6 2024-04-07T16:16:51.216 INFO:tasks.rados.rados.0.smithi100.stdout:4: finishing write tid 1 to smithi10066867-4 2024-04-07T16:16:51.216 INFO:tasks.rados.rados.0.smithi100.stdout:4: finishing write tid 2 to smithi10066867-4 2024-04-07T16:16:51.216 INFO:tasks.rados.rados.0.smithi100.stdout:1: done (15 left) 2024-04-07T16:16:51.216 INFO:tasks.rados.rados.0.smithi100.stdout:2: done (14 left) 2024-04-07T16:16:51.216 INFO:tasks.rados.rados.0.smithi100.stdout:3: done (13 left) 2024-04-07T16:16:51.216 INFO:tasks.rados.rados.0.smithi100.stdout:5: done (12 left) 2024-04-07T16:16:51.216 INFO:tasks.rados.rados.0.smithi100.stdout:17: write initial oid 17 2024-04-07T16:16:51.216 INFO:tasks.rados.rados.0.smithi100.stdout:17: seq_num 16 ranges {0=270336} 2024-04-07T16:16:51.217 INFO:tasks.rados.rados.0.smithi100.stdout:17: writing smithi10066867-17 from 0 to 270336 tid 1 2024-04-07T16:16:51.217 INFO:tasks.rados.rados.0.smithi100.stdout:18: write initial oid 18 2024-04-07T16:16:51.217 INFO:tasks.rados.rados.0.smithi100.stdout:18: seq_num 17 ranges {0=286720} 2024-04-07T16:16:51.219 INFO:tasks.rados.rados.0.smithi100.stdout:18: writing smithi10066867-18 from 0 to 286720 tid 1 2024-04-07T16:16:51.219 INFO:tasks.rados.rados.0.smithi100.stdout:19: write initial oid 19 2024-04-07T16:16:51.219 INFO:tasks.rados.rados.0.smithi100.stdout:19: seq_num 18 ranges {0=303104} 2024-04-07T16:16:51.222 INFO:tasks.rados.rados.0.smithi100.stdout:19: writing smithi10066867-19 from 0 to 303104 tid 1 2024-04-07T16:16:51.222 INFO:tasks.rados.rados.0.smithi100.stdout:20: write initial oid 20 2024-04-07T16:16:51.222 INFO:tasks.rados.rados.0.smithi100.stdout:20: seq_num 19 ranges {0=319488} 2024-04-07T16:16:51.227 INFO:tasks.rados.rados.0.smithi100.stdout:20: writing smithi10066867-20 from 0 to 319488 tid 1 2024-04-07T16:16:51.227 INFO:tasks.rados.rados.0.smithi100.stdout: waiting on 16 2024-04-07T16:16:51.227 INFO:tasks.rados.rados.0.smithi100.stdout:6: finishing write tid 3 to smithi10066867-6 2024-04-07T16:16:51.227 INFO:tasks.rados.rados.0.smithi100.stdout:6: oid 6 updating version 0 to 3 2024-04-07T16:16:51.227 INFO:tasks.rados.rados.0.smithi100.stdout:6: oid 6 updating version 3 to 4 2024-04-07T16:16:51.227 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 6 v 4 (ObjNum 5 snap 0 seq_num 5) dirty exists 2024-04-07T16:16:51.227 INFO:tasks.rados.rados.0.smithi100.stdout:6: left oid 6 (ObjNum 5 snap 0 seq_num 5) 2024-04-07T16:16:51.227 INFO:tasks.rados.rados.0.smithi100.stdout:10: finishing write tid 2 to smithi10066867-10 2024-04-07T16:16:51.227 INFO:tasks.rados.rados.0.smithi100.stdout:8: finishing write tid 1 to smithi10066867-8 2024-04-07T16:16:51.227 INFO:tasks.rados.rados.0.smithi100.stdout:9: finishing write tid 3 to smithi10066867-9 2024-04-07T16:16:51.228 INFO:tasks.rados.rados.0.smithi100.stdout:9: oid 9 updating version 0 to 1 2024-04-07T16:16:51.228 INFO:tasks.rados.rados.0.smithi100.stdout:9: oid 9 updating version 1 to 2 2024-04-07T16:16:51.228 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 9 v 2 (ObjNum 8 snap 0 seq_num 8) dirty exists 2024-04-07T16:16:51.228 INFO:tasks.rados.rados.0.smithi100.stdout:9: left oid 9 (ObjNum 8 snap 0 seq_num 8) 2024-04-07T16:16:51.228 INFO:tasks.rados.rados.0.smithi100.stdout:8: finishing write tid 2 to smithi10066867-8 2024-04-07T16:16:51.228 INFO:tasks.rados.rados.0.smithi100.stdout:10: finishing write tid 3 to smithi10066867-10 2024-04-07T16:16:51.228 INFO:tasks.rados.rados.0.smithi100.stdout:10: oid 10 updating version 0 to 3 2024-04-07T16:16:51.228 INFO:tasks.rados.rados.0.smithi100.stdout:10: oid 10 updating version 3 to 4 2024-04-07T16:16:51.228 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 10 v 4 (ObjNum 9 snap 0 seq_num 9) dirty exists 2024-04-07T16:16:51.228 INFO:tasks.rados.rados.0.smithi100.stdout:10: left oid 10 (ObjNum 9 snap 0 seq_num 9) 2024-04-07T16:16:51.228 INFO:tasks.rados.rados.0.smithi100.stdout:12: finishing write tid 1 to smithi10066867-12 2024-04-07T16:16:51.228 INFO:tasks.rados.rados.0.smithi100.stdout:12: finishing write tid 2 to smithi10066867-12 2024-04-07T16:16:51.228 INFO:tasks.rados.rados.0.smithi100.stdout:11: finishing write tid 1 to smithi10066867-11 2024-04-07T16:16:51.228 INFO:tasks.rados.rados.0.smithi100.stdout:7: finishing write tid 1 to smithi10066867-7 2024-04-07T16:16:51.228 INFO:tasks.rados.rados.0.smithi100.stdout:16: finishing write tid 1 to smithi10066867-16 2024-04-07T16:16:51.229 INFO:tasks.rados.rados.0.smithi100.stdout:7: finishing write tid 2 to smithi10066867-7 2024-04-07T16:16:51.229 INFO:tasks.rados.rados.0.smithi100.stdout:16: finishing write tid 2 to smithi10066867-16 2024-04-07T16:16:51.229 INFO:tasks.rados.rados.0.smithi100.stdout:6: done (15 left) 2024-04-07T16:16:51.229 INFO:tasks.rados.rados.0.smithi100.stdout:9: done (14 left) 2024-04-07T16:16:51.229 INFO:tasks.rados.rados.0.smithi100.stdout:10: done (13 left) 2024-04-07T16:16:51.229 INFO:tasks.rados.rados.0.smithi100.stdout:21: write initial oid 21 2024-04-07T16:16:51.229 INFO:tasks.rados.rados.0.smithi100.stdout:21: seq_num 20 ranges {0=344064} 2024-04-07T16:16:51.230 INFO:tasks.rados.rados.0.smithi100.stdout:21: writing smithi10066867-21 from 0 to 344064 tid 1 2024-04-07T16:16:51.230 INFO:tasks.rados.rados.0.smithi100.stdout:22: write initial oid 22 2024-04-07T16:16:51.230 INFO:tasks.rados.rados.0.smithi100.stdout:22: seq_num 21 ranges {0=360448} 2024-04-07T16:16:51.233 INFO:tasks.rados.rados.0.smithi100.stdout:22: writing smithi10066867-22 from 0 to 360448 tid 1 2024-04-07T16:16:51.233 INFO:tasks.rados.rados.0.smithi100.stdout:23: write initial oid 23 2024-04-07T16:16:51.233 INFO:tasks.rados.rados.0.smithi100.stdout:23: seq_num 22 ranges {0=376832} 2024-04-07T16:16:51.238 INFO:tasks.rados.rados.0.smithi100.stdout:23: writing smithi10066867-23 from 0 to 376832 tid 1 2024-04-07T16:16:51.238 INFO:tasks.rados.rados.0.smithi100.stdout: waiting on 16 2024-04-07T16:16:51.238 INFO:tasks.rados.rados.0.smithi100.stdout:4: finishing write tid 3 to smithi10066867-4 2024-04-07T16:16:51.238 INFO:tasks.rados.rados.0.smithi100.stdout:4: oid 4 updating version 0 to 2 2024-04-07T16:16:51.238 INFO:tasks.rados.rados.0.smithi100.stdout:4: oid 4 version 2 is already newer than 1 2024-04-07T16:16:51.238 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 4 v 2 (ObjNum 3 snap 0 seq_num 3) dirty exists 2024-04-07T16:16:51.238 INFO:tasks.rados.rados.0.smithi100.stdout:4: left oid 4 (ObjNum 3 snap 0 seq_num 3) 2024-04-07T16:16:51.238 INFO:tasks.rados.rados.0.smithi100.stdout:14: finishing write tid 1 to smithi10066867-14 2024-04-07T16:16:51.238 INFO:tasks.rados.rados.0.smithi100.stdout:7: finishing write tid 3 to smithi10066867-7 2024-04-07T16:16:51.238 INFO:tasks.rados.rados.0.smithi100.stdout:7: oid 7 updating version 0 to 1 2024-04-07T16:16:51.238 INFO:tasks.rados.rados.0.smithi100.stdout:7: oid 7 updating version 1 to 2 2024-04-07T16:16:51.238 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 7 v 2 (ObjNum 6 snap 0 seq_num 6) dirty exists 2024-04-07T16:16:51.239 INFO:tasks.rados.rados.0.smithi100.stdout:7: left oid 7 (ObjNum 6 snap 0 seq_num 6) 2024-04-07T16:16:51.239 INFO:tasks.rados.rados.0.smithi100.stdout:14: finishing write tid 2 to smithi10066867-14 2024-04-07T16:16:51.239 INFO:tasks.rados.rados.0.smithi100.stdout:12: finishing write tid 3 to smithi10066867-12 2024-04-07T16:16:51.239 INFO:tasks.rados.rados.0.smithi100.stdout:12: oid 12 updating version 0 to 3 2024-04-07T16:16:51.239 INFO:tasks.rados.rados.0.smithi100.stdout:12: oid 12 updating version 3 to 4 2024-04-07T16:16:51.239 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 12 v 4 (ObjNum 11 snap 0 seq_num 11) dirty exists 2024-04-07T16:16:51.239 INFO:tasks.rados.rados.0.smithi100.stdout:12: left oid 12 (ObjNum 11 snap 0 seq_num 11) 2024-04-07T16:16:51.239 INFO:tasks.rados.rados.0.smithi100.stdout:16: finishing write tid 3 to smithi10066867-16 2024-04-07T16:16:51.239 INFO:tasks.rados.rados.0.smithi100.stdout:16: oid 16 updating version 0 to 5 2024-04-07T16:16:51.239 INFO:tasks.rados.rados.0.smithi100.stdout:16: oid 16 updating version 5 to 6 2024-04-07T16:16:51.239 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 16 v 6 (ObjNum 15 snap 0 seq_num 15) dirty exists 2024-04-07T16:16:51.239 INFO:tasks.rados.rados.0.smithi100.stdout:16: left oid 16 (ObjNum 15 snap 0 seq_num 15) 2024-04-07T16:16:51.240 INFO:tasks.rados.rados.0.smithi100.stdout:17: finishing write tid 1 to smithi10066867-17 2024-04-07T16:16:51.240 INFO:tasks.rados.rados.0.smithi100.stdout:17: finishing write tid 2 to smithi10066867-17 2024-04-07T16:16:51.240 INFO:tasks.rados.rados.0.smithi100.stdout:19: finishing write tid 1 to smithi10066867-19 2024-04-07T16:16:51.240 INFO:tasks.rados.rados.0.smithi100.stdout:11: finishing write tid 2 to smithi10066867-11 2024-04-07T16:16:51.240 INFO:tasks.rados.rados.0.smithi100.stdout:17: finishing write tid 3 to smithi10066867-17 2024-04-07T16:16:51.240 INFO:tasks.rados.rados.0.smithi100.stdout:17: oid 17 updating version 0 to 2 2024-04-07T16:16:51.240 INFO:tasks.rados.rados.0.smithi100.stdout:17: oid 17 version 2 is already newer than 1 2024-04-07T16:16:51.240 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 17 v 2 (ObjNum 16 snap 0 seq_num 16) dirty exists 2024-04-07T16:16:51.240 INFO:tasks.rados.rados.0.smithi100.stdout:17: left oid 17 (ObjNum 16 snap 0 seq_num 16) 2024-04-07T16:16:51.240 INFO:tasks.rados.rados.0.smithi100.stdout:19: finishing write tid 2 to smithi10066867-19 2024-04-07T16:16:51.240 INFO:tasks.rados.rados.0.smithi100.stdout:20: finishing write tid 1 to smithi10066867-20 2024-04-07T16:16:51.240 INFO:tasks.rados.rados.0.smithi100.stdout:20: finishing write tid 2 to smithi10066867-20 2024-04-07T16:16:51.240 INFO:tasks.rados.rados.0.smithi100.stdout:13: finishing write tid 1 to smithi10066867-13 2024-04-07T16:16:51.240 INFO:tasks.rados.rados.0.smithi100.stdout:15: finishing write tid 1 to smithi10066867-15 2024-04-07T16:16:51.240 INFO:tasks.rados.rados.0.smithi100.stdout:14: finishing write tid 3 to smithi10066867-14 2024-04-07T16:16:51.241 INFO:tasks.rados.rados.0.smithi100.stdout:14: oid 14 updating version 0 to 2 2024-04-07T16:16:51.241 INFO:tasks.rados.rados.0.smithi100.stdout:14: oid 14 version 2 is already newer than 1 2024-04-07T16:16:51.241 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 14 v 2 (ObjNum 13 snap 0 seq_num 13) dirty exists 2024-04-07T16:16:51.241 INFO:tasks.rados.rados.0.smithi100.stdout:14: left oid 14 (ObjNum 13 snap 0 seq_num 13) 2024-04-07T16:16:51.241 INFO:tasks.rados.rados.0.smithi100.stdout:19: finishing write tid 3 to smithi10066867-19 2024-04-07T16:16:51.241 INFO:tasks.rados.rados.0.smithi100.stdout:19: oid 19 updating version 0 to 4 2024-04-07T16:16:51.241 INFO:tasks.rados.rados.0.smithi100.stdout:19: oid 19 version 4 is already newer than 3 2024-04-07T16:16:51.241 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 19 v 4 (ObjNum 18 snap 0 seq_num 18) dirty exists 2024-04-07T16:16:51.241 INFO:tasks.rados.rados.0.smithi100.stdout:19: left oid 19 (ObjNum 18 snap 0 seq_num 18) 2024-04-07T16:16:51.241 INFO:tasks.rados.rados.0.smithi100.stdout:15: finishing write tid 2 to smithi10066867-15 2024-04-07T16:16:51.241 INFO:tasks.rados.rados.0.smithi100.stdout:13: finishing write tid 2 to smithi10066867-13 2024-04-07T16:16:51.241 INFO:tasks.rados.rados.0.smithi100.stdout:20: finishing write tid 3 to smithi10066867-20 2024-04-07T16:16:51.241 INFO:tasks.rados.rados.0.smithi100.stdout:20: oid 20 updating version 0 to 2 2024-04-07T16:16:51.241 INFO:tasks.rados.rados.0.smithi100.stdout:20: oid 20 version 2 is already newer than 1 2024-04-07T16:16:51.241 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 20 v 2 (ObjNum 19 snap 0 seq_num 19) dirty exists 2024-04-07T16:16:51.242 INFO:tasks.rados.rados.0.smithi100.stdout:20: left oid 20 (ObjNum 19 snap 0 seq_num 19) 2024-04-07T16:16:51.242 INFO:tasks.rados.rados.0.smithi100.stdout:18: finishing write tid 1 to smithi10066867-18 2024-04-07T16:16:51.242 INFO:tasks.rados.rados.0.smithi100.stdout:18: finishing write tid 2 to smithi10066867-18 2024-04-07T16:16:51.242 INFO:tasks.rados.rados.0.smithi100.stdout:11: finishing write tid 3 to smithi10066867-11 2024-04-07T16:16:51.242 INFO:tasks.rados.rados.0.smithi100.stdout:11: oid 11 updating version 0 to 3 2024-04-07T16:16:51.242 INFO:tasks.rados.rados.0.smithi100.stdout:11: oid 11 updating version 3 to 4 2024-04-07T16:16:51.242 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 11 v 4 (ObjNum 10 snap 0 seq_num 10) dirty exists 2024-04-07T16:16:51.242 INFO:tasks.rados.rados.0.smithi100.stdout:11: left oid 11 (ObjNum 10 snap 0 seq_num 10) 2024-04-07T16:16:51.242 INFO:tasks.rados.rados.0.smithi100.stdout:8: finishing write tid 3 to smithi10066867-8 2024-04-07T16:16:51.242 INFO:tasks.rados.rados.0.smithi100.stdout:8: oid 8 updating version 0 to 2 2024-04-07T16:16:51.242 INFO:tasks.rados.rados.0.smithi100.stdout:8: oid 8 version 2 is already newer than 1 2024-04-07T16:16:51.242 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 8 v 2 (ObjNum 7 snap 0 seq_num 7) dirty exists 2024-04-07T16:16:51.242 INFO:tasks.rados.rados.0.smithi100.stdout:8: left oid 8 (ObjNum 7 snap 0 seq_num 7) 2024-04-07T16:16:51.242 INFO:tasks.rados.rados.0.smithi100.stdout:15: finishing write tid 3 to smithi10066867-15 2024-04-07T16:16:51.242 INFO:tasks.rados.rados.0.smithi100.stdout:15: oid 15 updating version 0 to 6 2024-04-07T16:16:51.243 INFO:tasks.rados.rados.0.smithi100.stdout:15: oid 15 version 6 is already newer than 5 2024-04-07T16:16:51.243 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 15 v 6 (ObjNum 14 snap 0 seq_num 14) dirty exists 2024-04-07T16:16:51.243 INFO:tasks.rados.rados.0.smithi100.stdout:15: left oid 15 (ObjNum 14 snap 0 seq_num 14) 2024-04-07T16:16:51.243 INFO:tasks.rados.rados.0.smithi100.stdout:18: finishing write tid 3 to smithi10066867-18 2024-04-07T16:16:51.246 INFO:tasks.rados.rados.0.smithi100.stdout:18: oid 18 updating version 0 to 7 2024-04-07T16:16:51.246 INFO:tasks.rados.rados.0.smithi100.stdout:18: oid 18 updating version 7 to 8 2024-04-07T16:16:51.246 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 18 v 8 (ObjNum 17 snap 0 seq_num 17) dirty exists 2024-04-07T16:16:51.247 INFO:tasks.rados.rados.0.smithi100.stdout:18: left oid 18 (ObjNum 17 snap 0 seq_num 17) 2024-04-07T16:16:51.247 INFO:tasks.rados.rados.0.smithi100.stdout:21: finishing write tid 1 to smithi10066867-21 2024-04-07T16:16:51.247 INFO:tasks.rados.rados.0.smithi100.stdout:4: done (15 left) 2024-04-07T16:16:51.247 INFO:tasks.rados.rados.0.smithi100.stdout:7: done (14 left) 2024-04-07T16:16:51.247 INFO:tasks.rados.rados.0.smithi100.stdout:8: done (13 left) 2024-04-07T16:16:51.247 INFO:tasks.rados.rados.0.smithi100.stdout:11: done (12 left) 2024-04-07T16:16:51.247 INFO:tasks.rados.rados.0.smithi100.stdout:12: done (11 left) 2024-04-07T16:16:51.247 INFO:tasks.rados.rados.0.smithi100.stdout:14: done (10 left) 2024-04-07T16:16:51.247 INFO:tasks.rados.rados.0.smithi100.stdout:15: done (9 left) 2024-04-07T16:16:51.247 INFO:tasks.rados.rados.0.smithi100.stdout:16: done (8 left) 2024-04-07T16:16:51.247 INFO:tasks.rados.rados.0.smithi100.stdout:17: done (7 left) 2024-04-07T16:16:51.247 INFO:tasks.rados.rados.0.smithi100.stdout:18: done (6 left) 2024-04-07T16:16:51.247 INFO:tasks.rados.rados.0.smithi100.stdout:19: done (5 left) 2024-04-07T16:16:51.247 INFO:tasks.rados.rados.0.smithi100.stdout:20: done (4 left) 2024-04-07T16:16:51.247 INFO:tasks.rados.rados.0.smithi100.stdout:24: write initial oid 24 2024-04-07T16:16:51.248 INFO:tasks.rados.rados.0.smithi100.stdout:24: seq_num 23 ranges {0=393216} 2024-04-07T16:16:51.249 INFO:tasks.rados.rados.0.smithi100.stdout:24: writing smithi10066867-24 from 0 to 393216 tid 1 2024-04-07T16:16:51.249 INFO:tasks.rados.rados.0.smithi100.stdout:25: write initial oid 25 2024-04-07T16:16:51.249 INFO:tasks.rados.rados.0.smithi100.stdout:25: seq_num 24 ranges {0=409600} 2024-04-07T16:16:51.249 INFO:tasks.rados.rados.0.smithi100.stdout:25: writing smithi10066867-25 from 0 to 409600 tid 1 2024-04-07T16:16:51.249 INFO:tasks.rados.rados.0.smithi100.stdout:26: write initial oid 26 2024-04-07T16:16:51.249 INFO:tasks.rados.rados.0.smithi100.stdout:26: seq_num 25 ranges {0=425984} 2024-04-07T16:16:51.249 INFO:tasks.rados.rados.0.smithi100.stdout:26: writing smithi10066867-26 from 0 to 425984 tid 1 2024-04-07T16:16:51.249 INFO:tasks.rados.rados.0.smithi100.stdout:27: write initial oid 27 2024-04-07T16:16:51.249 INFO:tasks.rados.rados.0.smithi100.stdout:27: seq_num 26 ranges {0=442368} 2024-04-07T16:16:51.249 INFO:tasks.rados.rados.0.smithi100.stdout:27: writing smithi10066867-27 from 0 to 442368 tid 1 2024-04-07T16:16:51.249 INFO:tasks.rados.rados.0.smithi100.stdout:28: write initial oid 28 2024-04-07T16:16:51.250 INFO:tasks.rados.rados.0.smithi100.stdout:28: seq_num 27 ranges {0=458752} 2024-04-07T16:16:51.252 INFO:tasks.rados.rados.0.smithi100.stdout:28: writing smithi10066867-28 from 0 to 458752 tid 1 2024-04-07T16:16:51.252 INFO:tasks.rados.rados.0.smithi100.stdout:29: write initial oid 29 2024-04-07T16:16:51.252 INFO:tasks.rados.rados.0.smithi100.stdout:29: seq_num 28 ranges {0=475136} 2024-04-07T16:16:51.255 INFO:tasks.rados.rados.0.smithi100.stdout:29: writing smithi10066867-29 from 0 to 475136 tid 1 2024-04-07T16:16:51.255 INFO:tasks.rados.rados.0.smithi100.stdout:30: write initial oid 30 2024-04-07T16:16:51.255 INFO:tasks.rados.rados.0.smithi100.stdout:30: seq_num 29 ranges {0=491520} 2024-04-07T16:16:51.258 INFO:tasks.rados.rados.0.smithi100.stdout:30: writing smithi10066867-30 from 0 to 491520 tid 1 2024-04-07T16:16:51.258 INFO:tasks.rados.rados.0.smithi100.stdout:31: write initial oid 31 2024-04-07T16:16:51.258 INFO:tasks.rados.rados.0.smithi100.stdout:31: seq_num 30 ranges {0=507904} 2024-04-07T16:16:51.261 INFO:tasks.rados.rados.0.smithi100.stdout:31: writing smithi10066867-31 from 0 to 507904 tid 1 2024-04-07T16:16:51.262 INFO:tasks.rados.rados.0.smithi100.stdout:32: write initial oid 32 2024-04-07T16:16:51.262 INFO:tasks.rados.rados.0.smithi100.stdout:32: seq_num 31 ranges {0=524288} 2024-04-07T16:16:51.265 INFO:tasks.rados.rados.0.smithi100.stdout:32: writing smithi10066867-32 from 0 to 524288 tid 1 2024-04-07T16:16:51.265 INFO:tasks.rados.rados.0.smithi100.stdout:33: write initial oid 33 2024-04-07T16:16:51.265 INFO:tasks.rados.rados.0.smithi100.stdout:33: seq_num 32 ranges {0=540672} 2024-04-07T16:16:51.268 INFO:tasks.rados.rados.0.smithi100.stdout:33: writing smithi10066867-33 from 0 to 540672 tid 1 2024-04-07T16:16:51.268 INFO:tasks.rados.rados.0.smithi100.stdout:34: write initial oid 34 2024-04-07T16:16:51.268 INFO:tasks.rados.rados.0.smithi100.stdout:34: seq_num 33 ranges {0=557056} 2024-04-07T16:16:51.272 INFO:tasks.rados.rados.0.smithi100.stdout:34: writing smithi10066867-34 from 0 to 557056 tid 1 2024-04-07T16:16:51.272 INFO:tasks.rados.rados.0.smithi100.stdout:35: write initial oid 35 2024-04-07T16:16:51.272 INFO:tasks.rados.rados.0.smithi100.stdout:35: seq_num 34 ranges {0=573440} 2024-04-07T16:16:51.276 INFO:tasks.rados.rados.0.smithi100.stdout:35: writing smithi10066867-35 from 0 to 573440 tid 1 2024-04-07T16:16:51.276 INFO:tasks.rados.rados.0.smithi100.stdout: waiting on 16 2024-04-07T16:16:51.276 INFO:tasks.rados.rados.0.smithi100.stdout:21: finishing write tid 2 to smithi10066867-21 2024-04-07T16:16:51.276 INFO:tasks.rados.rados.0.smithi100.stdout:22: finishing write tid 1 to smithi10066867-22 2024-04-07T16:16:51.277 INFO:tasks.rados.rados.0.smithi100.stdout:22: finishing write tid 2 to smithi10066867-22 2024-04-07T16:16:51.277 INFO:tasks.rados.rados.0.smithi100.stdout:21: finishing write tid 3 to smithi10066867-21 2024-04-07T16:16:51.277 INFO:tasks.rados.rados.0.smithi100.stdout:21: oid 21 updating version 0 to 2 2024-04-07T16:16:51.277 INFO:tasks.rados.rados.0.smithi100.stdout:21: oid 21 version 2 is already newer than 1 2024-04-07T16:16:51.277 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 21 v 2 (ObjNum 20 snap 0 seq_num 20) dirty exists 2024-04-07T16:16:51.277 INFO:tasks.rados.rados.0.smithi100.stdout:21: left oid 21 (ObjNum 20 snap 0 seq_num 20) 2024-04-07T16:16:51.277 INFO:tasks.rados.rados.0.smithi100.stdout:22: finishing write tid 3 to smithi10066867-22 2024-04-07T16:16:51.277 INFO:tasks.rados.rados.0.smithi100.stdout:22: oid 22 updating version 0 to 4 2024-04-07T16:16:51.277 INFO:tasks.rados.rados.0.smithi100.stdout:22: oid 22 version 4 is already newer than 3 2024-04-07T16:16:51.277 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 22 v 4 (ObjNum 21 snap 0 seq_num 21) dirty exists 2024-04-07T16:16:51.277 INFO:tasks.rados.rados.0.smithi100.stdout:22: left oid 22 (ObjNum 21 snap 0 seq_num 21) 2024-04-07T16:16:51.277 INFO:tasks.rados.rados.0.smithi100.stdout:13: finishing write tid 3 to smithi10066867-13 2024-04-07T16:16:51.277 INFO:tasks.rados.rados.0.smithi100.stdout:13: oid 13 updating version 0 to 3 2024-04-07T16:16:51.278 INFO:tasks.rados.rados.0.smithi100.stdout:13: oid 13 updating version 3 to 4 2024-04-07T16:16:51.278 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 13 v 4 (ObjNum 12 snap 0 seq_num 12) dirty exists 2024-04-07T16:16:51.278 INFO:tasks.rados.rados.0.smithi100.stdout:13: left oid 13 (ObjNum 12 snap 0 seq_num 12) 2024-04-07T16:16:51.278 INFO:tasks.rados.rados.0.smithi100.stdout:23: finishing write tid 1 to smithi10066867-23 2024-04-07T16:16:51.278 INFO:tasks.rados.rados.0.smithi100.stdout:23: finishing write tid 2 to smithi10066867-23 2024-04-07T16:16:51.278 INFO:tasks.rados.rados.0.smithi100.stdout:23: finishing write tid 3 to smithi10066867-23 2024-04-07T16:16:51.278 INFO:tasks.rados.rados.0.smithi100.stdout:23: oid 23 updating version 0 to 9 2024-04-07T16:16:51.278 INFO:tasks.rados.rados.0.smithi100.stdout:23: oid 23 updating version 9 to 10 2024-04-07T16:16:51.278 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 23 v 10 (ObjNum 22 snap 0 seq_num 22) dirty exists 2024-04-07T16:16:51.278 INFO:tasks.rados.rados.0.smithi100.stdout:23: left oid 23 (ObjNum 22 snap 0 seq_num 22) 2024-04-07T16:16:51.278 INFO:tasks.rados.rados.0.smithi100.stdout:24: finishing write tid 1 to smithi10066867-24 2024-04-07T16:16:51.278 INFO:tasks.rados.rados.0.smithi100.stdout:24: finishing write tid 2 to smithi10066867-24 2024-04-07T16:16:51.278 INFO:tasks.rados.rados.0.smithi100.stdout:25: finishing write tid 1 to smithi10066867-25 2024-04-07T16:16:51.278 INFO:tasks.rados.rados.0.smithi100.stdout:25: finishing write tid 2 to smithi10066867-25 2024-04-07T16:16:51.278 INFO:tasks.rados.rados.0.smithi100.stdout:24: finishing write tid 3 to smithi10066867-24 2024-04-07T16:16:51.279 INFO:tasks.rados.rados.0.smithi100.stdout:24: oid 24 updating version 0 to 5 2024-04-07T16:16:51.279 INFO:tasks.rados.rados.0.smithi100.stdout:24: oid 24 updating version 5 to 6 2024-04-07T16:16:51.279 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 24 v 6 (ObjNum 23 snap 0 seq_num 23) dirty exists 2024-04-07T16:16:51.279 INFO:tasks.rados.rados.0.smithi100.stdout:24: left oid 24 (ObjNum 23 snap 0 seq_num 23) 2024-04-07T16:16:51.279 INFO:tasks.rados.rados.0.smithi100.stdout:25: finishing write tid 3 to smithi10066867-25 2024-04-07T16:16:51.279 INFO:tasks.rados.rados.0.smithi100.stdout:25: oid 25 updating version 0 to 6 2024-04-07T16:16:51.279 INFO:tasks.rados.rados.0.smithi100.stdout:25: oid 25 version 6 is already newer than 5 2024-04-07T16:16:51.279 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 25 v 6 (ObjNum 24 snap 0 seq_num 24) dirty exists 2024-04-07T16:16:51.279 INFO:tasks.rados.rados.0.smithi100.stdout:25: left oid 25 (ObjNum 24 snap 0 seq_num 24) 2024-04-07T16:16:51.279 INFO:tasks.rados.rados.0.smithi100.stdout:26: finishing write tid 1 to smithi10066867-26 2024-04-07T16:16:51.279 INFO:tasks.rados.rados.0.smithi100.stdout:26: finishing write tid 2 to smithi10066867-26 2024-04-07T16:16:51.279 INFO:tasks.rados.rados.0.smithi100.stdout:26: finishing write tid 3 to smithi10066867-26 2024-04-07T16:16:51.279 INFO:tasks.rados.rados.0.smithi100.stdout:26: oid 26 updating version 0 to 8 2024-04-07T16:16:51.279 INFO:tasks.rados.rados.0.smithi100.stdout:26: oid 26 version 8 is already newer than 7 2024-04-07T16:16:51.279 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 26 v 8 (ObjNum 25 snap 0 seq_num 25) dirty exists 2024-04-07T16:16:51.280 INFO:tasks.rados.rados.0.smithi100.stdout:26: left oid 26 (ObjNum 25 snap 0 seq_num 25) 2024-04-07T16:16:51.280 INFO:tasks.rados.rados.0.smithi100.stdout:27: finishing write tid 1 to smithi10066867-27 2024-04-07T16:16:51.280 INFO:tasks.rados.rados.0.smithi100.stdout:27: finishing write tid 2 to smithi10066867-27 2024-04-07T16:16:51.280 INFO:tasks.rados.rados.0.smithi100.stdout:27: finishing write tid 3 to smithi10066867-27 2024-04-07T16:16:51.280 INFO:tasks.rados.rados.0.smithi100.stdout:27: oid 27 updating version 0 to 4 2024-04-07T16:16:51.280 INFO:tasks.rados.rados.0.smithi100.stdout:27: oid 27 version 4 is already newer than 3 2024-04-07T16:16:51.280 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 27 v 4 (ObjNum 26 snap 0 seq_num 26) dirty exists 2024-04-07T16:16:51.280 INFO:tasks.rados.rados.0.smithi100.stdout:27: left oid 27 (ObjNum 26 snap 0 seq_num 26) 2024-04-07T16:16:51.280 INFO:tasks.rados.rados.0.smithi100.stdout:28: finishing write tid 1 to smithi10066867-28 2024-04-07T16:16:51.280 INFO:tasks.rados.rados.0.smithi100.stdout:28: finishing write tid 2 to smithi10066867-28 2024-04-07T16:16:51.281 INFO:tasks.rados.rados.0.smithi100.stdout:28: finishing write tid 3 to smithi10066867-28 2024-04-07T16:16:51.281 INFO:tasks.rados.rados.0.smithi100.stdout:28: oid 28 updating version 0 to 2 2024-04-07T16:16:51.281 INFO:tasks.rados.rados.0.smithi100.stdout:28: oid 28 version 2 is already newer than 1 2024-04-07T16:16:51.281 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 28 v 2 (ObjNum 27 snap 0 seq_num 27) dirty exists 2024-04-07T16:16:51.281 INFO:tasks.rados.rados.0.smithi100.stdout:28: left oid 28 (ObjNum 27 snap 0 seq_num 27) 2024-04-07T16:16:51.281 INFO:tasks.rados.rados.0.smithi100.stdout:29: finishing write tid 1 to smithi10066867-29 2024-04-07T16:16:51.281 INFO:tasks.rados.rados.0.smithi100.stdout:29: finishing write tid 2 to smithi10066867-29 2024-04-07T16:16:51.281 INFO:tasks.rados.rados.0.smithi100.stdout:30: finishing write tid 1 to smithi10066867-30 2024-04-07T16:16:51.281 INFO:tasks.rados.rados.0.smithi100.stdout:30: finishing write tid 2 to smithi10066867-30 2024-04-07T16:16:51.281 INFO:tasks.rados.rados.0.smithi100.stdout:29: finishing write tid 3 to smithi10066867-29 2024-04-07T16:16:51.281 INFO:tasks.rados.rados.0.smithi100.stdout:29: oid 29 updating version 0 to 2 2024-04-07T16:16:51.281 INFO:tasks.rados.rados.0.smithi100.stdout:29: oid 29 version 2 is already newer than 1 2024-04-07T16:16:51.281 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 29 v 2 (ObjNum 28 snap 0 seq_num 28) dirty exists 2024-04-07T16:16:51.281 INFO:tasks.rados.rados.0.smithi100.stdout:29: left oid 29 (ObjNum 28 snap 0 seq_num 28) 2024-04-07T16:16:51.281 INFO:tasks.rados.rados.0.smithi100.stdout:30: finishing write tid 3 to smithi10066867-30 2024-04-07T16:16:51.282 INFO:tasks.rados.rados.0.smithi100.stdout:30: oid 30 updating version 0 to 4 2024-04-07T16:16:51.282 INFO:tasks.rados.rados.0.smithi100.stdout:30: oid 30 version 4 is already newer than 3 2024-04-07T16:16:51.282 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 30 v 4 (ObjNum 29 snap 0 seq_num 29) dirty exists 2024-04-07T16:16:51.282 INFO:tasks.rados.rados.0.smithi100.stdout:30: left oid 30 (ObjNum 29 snap 0 seq_num 29) 2024-04-07T16:16:51.282 INFO:tasks.rados.rados.0.smithi100.stdout:31: finishing write tid 1 to smithi10066867-31 2024-04-07T16:16:51.282 INFO:tasks.rados.rados.0.smithi100.stdout:31: finishing write tid 2 to smithi10066867-31 2024-04-07T16:16:51.282 INFO:tasks.rados.rados.0.smithi100.stdout:31: finishing write tid 3 to smithi10066867-31 2024-04-07T16:16:51.282 INFO:tasks.rados.rados.0.smithi100.stdout:31: oid 31 updating version 0 to 2 2024-04-07T16:16:51.282 INFO:tasks.rados.rados.0.smithi100.stdout:31: oid 31 version 2 is already newer than 1 2024-04-07T16:16:51.282 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 31 v 2 (ObjNum 30 snap 0 seq_num 30) dirty exists 2024-04-07T16:16:51.282 INFO:tasks.rados.rados.0.smithi100.stdout:31: left oid 31 (ObjNum 30 snap 0 seq_num 30) 2024-04-07T16:16:51.282 INFO:tasks.rados.rados.0.smithi100.stdout:32: finishing write tid 1 to smithi10066867-32 2024-04-07T16:16:51.282 INFO:tasks.rados.rados.0.smithi100.stdout:32: finishing write tid 2 to smithi10066867-32 2024-04-07T16:16:51.283 INFO:tasks.rados.rados.0.smithi100.stdout:33: finishing write tid 1 to smithi10066867-33 2024-04-07T16:16:51.283 INFO:tasks.rados.rados.0.smithi100.stdout:13: done (15 left) 2024-04-07T16:16:51.283 INFO:tasks.rados.rados.0.smithi100.stdout:21: done (14 left) 2024-04-07T16:16:51.283 INFO:tasks.rados.rados.0.smithi100.stdout:22: done (13 left) 2024-04-07T16:16:51.283 INFO:tasks.rados.rados.0.smithi100.stdout:23: done (12 left) 2024-04-07T16:16:51.283 INFO:tasks.rados.rados.0.smithi100.stdout:24: done (11 left) 2024-04-07T16:16:51.283 INFO:tasks.rados.rados.0.smithi100.stdout:25: done (10 left) 2024-04-07T16:16:51.283 INFO:tasks.rados.rados.0.smithi100.stdout:26: done (9 left) 2024-04-07T16:16:51.283 INFO:tasks.rados.rados.0.smithi100.stdout:27: done (8 left) 2024-04-07T16:16:51.283 INFO:tasks.rados.rados.0.smithi100.stdout:28: done (7 left) 2024-04-07T16:16:51.283 INFO:tasks.rados.rados.0.smithi100.stdout:29: done (6 left) 2024-04-07T16:16:51.283 INFO:tasks.rados.rados.0.smithi100.stdout:30: done (5 left) 2024-04-07T16:16:51.283 INFO:tasks.rados.rados.0.smithi100.stdout:31: done (4 left) 2024-04-07T16:16:51.283 INFO:tasks.rados.rados.0.smithi100.stdout:36: write initial oid 36 2024-04-07T16:16:51.283 INFO:tasks.rados.rados.0.smithi100.stdout:36: seq_num 35 ranges {0=589824} 2024-04-07T16:16:51.284 INFO:tasks.rados.rados.0.smithi100.stdout:36: writing smithi10066867-36 from 0 to 589824 tid 1 2024-04-07T16:16:51.284 INFO:tasks.rados.rados.0.smithi100.stdout:37: write initial oid 37 2024-04-07T16:16:51.284 INFO:tasks.rados.rados.0.smithi100.stdout:37: seq_num 36 ranges {0=606208} 2024-04-07T16:16:51.284 INFO:tasks.rados.rados.0.smithi100.stdout:37: writing smithi10066867-37 from 0 to 606208 tid 1 2024-04-07T16:16:51.285 INFO:tasks.rados.rados.0.smithi100.stdout:38: write initial oid 38 2024-04-07T16:16:51.285 INFO:tasks.rados.rados.0.smithi100.stdout:38: seq_num 37 ranges {0=622592} 2024-04-07T16:16:51.288 INFO:tasks.rados.rados.0.smithi100.stdout:38: writing smithi10066867-38 from 0 to 622592 tid 1 2024-04-07T16:16:51.288 INFO:tasks.rados.rados.0.smithi100.stdout:39: write initial oid 39 2024-04-07T16:16:51.288 INFO:tasks.rados.rados.0.smithi100.stdout:39: seq_num 38 ranges {0=638976} 2024-04-07T16:16:51.292 INFO:tasks.rados.rados.0.smithi100.stdout:39: writing smithi10066867-39 from 0 to 638976 tid 1 2024-04-07T16:16:51.292 INFO:tasks.rados.rados.0.smithi100.stdout:40: write initial oid 40 2024-04-07T16:16:51.292 INFO:tasks.rados.rados.0.smithi100.stdout:40: seq_num 39 ranges {0=663552} 2024-04-07T16:16:51.296 INFO:tasks.rados.rados.0.smithi100.stdout:40: writing smithi10066867-40 from 0 to 663552 tid 1 2024-04-07T16:16:51.296 INFO:tasks.rados.rados.0.smithi100.stdout:41: write initial oid 41 2024-04-07T16:16:51.296 INFO:tasks.rados.rados.0.smithi100.stdout:41: seq_num 40 ranges {0=679936} 2024-04-07T16:16:51.301 INFO:tasks.rados.rados.0.smithi100.stdout:41: writing smithi10066867-41 from 0 to 679936 tid 1 2024-04-07T16:16:51.301 INFO:tasks.rados.rados.0.smithi100.stdout:42: write initial oid 42 2024-04-07T16:16:51.301 INFO:tasks.rados.rados.0.smithi100.stdout:42: seq_num 41 ranges {0=696320} 2024-04-07T16:16:51.305 INFO:tasks.rados.rados.0.smithi100.stdout:42: writing smithi10066867-42 from 0 to 696320 tid 1 2024-04-07T16:16:51.305 INFO:tasks.rados.rados.0.smithi100.stdout:43: write initial oid 43 2024-04-07T16:16:51.305 INFO:tasks.rados.rados.0.smithi100.stdout:43: seq_num 42 ranges {0=712704} 2024-04-07T16:16:51.310 INFO:tasks.rados.rados.0.smithi100.stdout:43: writing smithi10066867-43 from 0 to 712704 tid 1 2024-04-07T16:16:51.310 INFO:tasks.rados.rados.0.smithi100.stdout:44: write initial oid 44 2024-04-07T16:16:51.310 INFO:tasks.rados.rados.0.smithi100.stdout:44: seq_num 43 ranges {0=729088} 2024-04-07T16:16:51.314 INFO:tasks.rados.rados.0.smithi100.stdout:44: writing smithi10066867-44 from 0 to 729088 tid 1 2024-04-07T16:16:51.315 INFO:tasks.rados.rados.0.smithi100.stdout:45: write initial oid 45 2024-04-07T16:16:51.315 INFO:tasks.rados.rados.0.smithi100.stdout:45: seq_num 44 ranges {0=745472} 2024-04-07T16:16:51.319 INFO:tasks.rados.rados.0.smithi100.stdout:45: writing smithi10066867-45 from 0 to 745472 tid 1 2024-04-07T16:16:51.319 INFO:tasks.rados.rados.0.smithi100.stdout:46: write initial oid 46 2024-04-07T16:16:51.319 INFO:tasks.rados.rados.0.smithi100.stdout:46: seq_num 45 ranges {0=761856} 2024-04-07T16:16:51.323 INFO:tasks.rados.rados.0.smithi100.stdout:46: writing smithi10066867-46 from 0 to 761856 tid 1 2024-04-07T16:16:51.323 INFO:tasks.rados.rados.0.smithi100.stdout:47: write initial oid 47 2024-04-07T16:16:51.324 INFO:tasks.rados.rados.0.smithi100.stdout:47: seq_num 46 ranges {0=778240} 2024-04-07T16:16:51.328 INFO:tasks.rados.rados.0.smithi100.stdout:47: writing smithi10066867-47 from 0 to 778240 tid 1 2024-04-07T16:16:51.329 INFO:tasks.rados.rados.0.smithi100.stdout: waiting on 16 2024-04-07T16:16:51.329 INFO:tasks.rados.rados.0.smithi100.stdout:33: finishing write tid 2 to smithi10066867-33 2024-04-07T16:16:51.329 INFO:tasks.rados.rados.0.smithi100.stdout:32: finishing write tid 3 to smithi10066867-32 2024-04-07T16:16:51.329 INFO:tasks.rados.rados.0.smithi100.stdout:32: oid 32 updating version 0 to 3 2024-04-07T16:16:51.329 INFO:tasks.rados.rados.0.smithi100.stdout:32: oid 32 updating version 3 to 4 2024-04-07T16:16:51.329 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 32 v 4 (ObjNum 31 snap 0 seq_num 31) dirty exists 2024-04-07T16:16:51.329 INFO:tasks.rados.rados.0.smithi100.stdout:32: left oid 32 (ObjNum 31 snap 0 seq_num 31) 2024-04-07T16:16:51.329 INFO:tasks.rados.rados.0.smithi100.stdout:33: finishing write tid 3 to smithi10066867-33 2024-04-07T16:16:51.329 INFO:tasks.rados.rados.0.smithi100.stdout:33: oid 33 updating version 0 to 12 2024-04-07T16:16:51.329 INFO:tasks.rados.rados.0.smithi100.stdout:33: oid 33 version 12 is already newer than 11 2024-04-07T16:16:51.329 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 33 v 12 (ObjNum 32 snap 0 seq_num 32) dirty exists 2024-04-07T16:16:51.329 INFO:tasks.rados.rados.0.smithi100.stdout:33: left oid 33 (ObjNum 32 snap 0 seq_num 32) 2024-04-07T16:16:51.329 INFO:tasks.rados.rados.0.smithi100.stdout:34: finishing write tid 1 to smithi10066867-34 2024-04-07T16:16:51.329 INFO:tasks.rados.rados.0.smithi100.stdout:34: finishing write tid 2 to smithi10066867-34 2024-04-07T16:16:51.329 INFO:tasks.rados.rados.0.smithi100.stdout:32: done (15 left) 2024-04-07T16:16:51.330 INFO:tasks.rados.rados.0.smithi100.stdout:33: done (14 left) 2024-04-07T16:16:51.330 INFO:tasks.rados.rados.0.smithi100.stdout:48: write initial oid 48 2024-04-07T16:16:51.330 INFO:tasks.rados.rados.0.smithi100.stdout:48: seq_num 47 ranges {0=794624} 2024-04-07T16:16:51.333 INFO:tasks.rados.rados.0.smithi100.stdout:48: writing smithi10066867-48 from 0 to 794624 tid 1 2024-04-07T16:16:51.333 INFO:tasks.rados.rados.0.smithi100.stdout:49: write initial oid 49 2024-04-07T16:16:51.333 INFO:tasks.rados.rados.0.smithi100.stdout:49: seq_num 48 ranges {0=409600,409600=401408} 2024-04-07T16:16:51.336 INFO:tasks.rados.rados.0.smithi100.stdout:49: writing smithi10066867-49 from 0 to 409600 tid 1 2024-04-07T16:16:51.339 INFO:tasks.rados.rados.0.smithi100.stdout:49: writing smithi10066867-49 from 409600 to 811008 tid 2 2024-04-07T16:16:51.339 INFO:tasks.rados.rados.0.smithi100.stdout: waiting on 16 2024-04-07T16:16:51.339 INFO:tasks.rados.rados.0.smithi100.stdout:35: finishing write tid 1 to smithi10066867-35 2024-04-07T16:16:51.339 INFO:tasks.rados.rados.0.smithi100.stdout:35: finishing write tid 2 to smithi10066867-35 2024-04-07T16:16:51.339 INFO:tasks.rados.rados.0.smithi100.stdout:34: finishing write tid 3 to smithi10066867-34 2024-04-07T16:16:51.339 INFO:tasks.rados.rados.0.smithi100.stdout:34: oid 34 updating version 0 to 6 2024-04-07T16:16:51.339 INFO:tasks.rados.rados.0.smithi100.stdout:34: oid 34 version 6 is already newer than 5 2024-04-07T16:16:51.340 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 34 v 6 (ObjNum 33 snap 0 seq_num 33) dirty exists 2024-04-07T16:16:51.340 INFO:tasks.rados.rados.0.smithi100.stdout:34: left oid 34 (ObjNum 33 snap 0 seq_num 33) 2024-04-07T16:16:51.340 INFO:tasks.rados.rados.0.smithi100.stdout:35: finishing write tid 3 to smithi10066867-35 2024-04-07T16:16:51.340 INFO:tasks.rados.rados.0.smithi100.stdout:35: oid 35 updating version 0 to 4 2024-04-07T16:16:51.340 INFO:tasks.rados.rados.0.smithi100.stdout:35: oid 35 version 4 is already newer than 3 2024-04-07T16:16:51.340 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 35 v 4 (ObjNum 34 snap 0 seq_num 34) dirty exists 2024-04-07T16:16:51.340 INFO:tasks.rados.rados.0.smithi100.stdout:35: left oid 35 (ObjNum 34 snap 0 seq_num 34) 2024-04-07T16:16:51.340 INFO:tasks.rados.rados.0.smithi100.stdout:36: finishing write tid 1 to smithi10066867-36 2024-04-07T16:16:51.340 INFO:tasks.rados.rados.0.smithi100.stdout:34: done (15 left) 2024-04-07T16:16:51.340 INFO:tasks.rados.rados.0.smithi100.stdout:35: done (14 left) 2024-04-07T16:16:51.340 INFO:tasks.rados.rados.0.smithi100.stdout:50: write initial oid 50 2024-04-07T16:16:51.340 INFO:tasks.rados.rados.0.smithi100.stdout:50: seq_num 49 ranges {0=425984,425984=401408} 2024-04-07T16:16:51.341 INFO:tasks.rados.rados.0.smithi100.stdout:50: writing smithi10066867-50 from 0 to 425984 tid 1 2024-04-07T16:16:51.345 INFO:tasks.rados.rados.0.smithi100.stdout:50: writing smithi10066867-50 from 425984 to 827392 tid 2 2024-04-07T16:16:51.345 INFO:tasks.rados.rados.0.smithi100.stdout:51: copy_from oid 2 from oid 11 current snap is 0 2024-04-07T16:16:51.346 INFO:tasks.rados.rados.0.smithi100.stdout:36: finishing write tid 2 to smithi10066867-36 2024-04-07T16:16:51.346 INFO:tasks.rados.rados.0.smithi100.stdout: waiting on 16 2024-04-07T16:16:51.346 INFO:tasks.rados.rados.0.smithi100.stdout:36: finishing write tid 3 to smithi10066867-36 2024-04-07T16:16:51.346 INFO:tasks.rados.rados.0.smithi100.stdout:36: oid 36 updating version 0 to 8 2024-04-07T16:16:51.346 INFO:tasks.rados.rados.0.smithi100.stdout:36: oid 36 version 8 is already newer than 7 2024-04-07T16:16:51.346 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 36 v 8 (ObjNum 35 snap 0 seq_num 35) dirty exists 2024-04-07T16:16:51.346 INFO:tasks.rados.rados.0.smithi100.stdout:36: left oid 36 (ObjNum 35 snap 0 seq_num 35) 2024-04-07T16:16:51.346 INFO:tasks.rados.rados.0.smithi100.stdout:37: finishing write tid 1 to smithi10066867-37 2024-04-07T16:16:51.346 INFO:tasks.rados.rados.0.smithi100.stdout:37: finishing write tid 2 to smithi10066867-37 2024-04-07T16:16:51.346 INFO:tasks.rados.rados.0.smithi100.stdout:37: finishing write tid 3 to smithi10066867-37 2024-04-07T16:16:51.346 INFO:tasks.rados.rados.0.smithi100.stdout:37: oid 37 updating version 0 to 6 2024-04-07T16:16:51.346 INFO:tasks.rados.rados.0.smithi100.stdout:37: oid 37 version 6 is already newer than 5 2024-04-07T16:16:51.346 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 37 v 6 (ObjNum 36 snap 0 seq_num 36) dirty exists 2024-04-07T16:16:51.347 INFO:tasks.rados.rados.0.smithi100.stdout:37: left oid 37 (ObjNum 36 snap 0 seq_num 36) 2024-04-07T16:16:51.347 INFO:tasks.rados.rados.0.smithi100.stdout:38: finishing write tid 1 to smithi10066867-38 2024-04-07T16:16:51.347 INFO:tasks.rados.rados.0.smithi100.stdout:38: finishing write tid 2 to smithi10066867-38 2024-04-07T16:16:51.347 INFO:tasks.rados.rados.0.smithi100.stdout:39: finishing write tid 1 to smithi10066867-39 2024-04-07T16:16:51.347 INFO:tasks.rados.rados.0.smithi100.stdout:39: finishing write tid 2 to smithi10066867-39 2024-04-07T16:16:51.347 INFO:tasks.rados.rados.0.smithi100.stdout:38: finishing write tid 3 to smithi10066867-38 2024-04-07T16:16:51.347 INFO:tasks.rados.rados.0.smithi100.stdout:38: oid 38 updating version 0 to 4 2024-04-07T16:16:51.347 INFO:tasks.rados.rados.0.smithi100.stdout:38: oid 38 version 4 is already newer than 3 2024-04-07T16:16:51.347 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 38 v 4 (ObjNum 37 snap 0 seq_num 37) dirty exists 2024-04-07T16:16:51.347 INFO:tasks.rados.rados.0.smithi100.stdout:38: left oid 38 (ObjNum 37 snap 0 seq_num 37) 2024-04-07T16:16:51.347 INFO:tasks.rados.rados.0.smithi100.stdout:39: finishing write tid 3 to smithi10066867-39 2024-04-07T16:16:51.347 INFO:tasks.rados.rados.0.smithi100.stdout:39: oid 39 updating version 0 to 8 2024-04-07T16:16:51.347 INFO:tasks.rados.rados.0.smithi100.stdout:39: oid 39 version 8 is already newer than 7 2024-04-07T16:16:51.347 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 39 v 8 (ObjNum 38 snap 0 seq_num 38) dirty exists 2024-04-07T16:16:51.348 INFO:tasks.rados.rados.0.smithi100.stdout:39: left oid 39 (ObjNum 38 snap 0 seq_num 38) 2024-04-07T16:16:51.348 INFO:tasks.rados.rados.0.smithi100.stdout:40: finishing write tid 1 to smithi10066867-40 2024-04-07T16:16:51.348 INFO:tasks.rados.rados.0.smithi100.stdout:40: finishing write tid 2 to smithi10066867-40 2024-04-07T16:16:51.348 INFO:tasks.rados.rados.0.smithi100.stdout:40: finishing write tid 3 to smithi10066867-40 2024-04-07T16:16:51.348 INFO:tasks.rados.rados.0.smithi100.stdout:40: oid 40 updating version 0 to 6 2024-04-07T16:16:51.348 INFO:tasks.rados.rados.0.smithi100.stdout:40: oid 40 version 6 is already newer than 5 2024-04-07T16:16:51.348 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 40 v 6 (ObjNum 39 snap 0 seq_num 39) dirty exists 2024-04-07T16:16:51.348 INFO:tasks.rados.rados.0.smithi100.stdout:40: left oid 40 (ObjNum 39 snap 0 seq_num 39) 2024-04-07T16:16:51.348 INFO:tasks.rados.rados.0.smithi100.stdout:41: finishing write tid 1 to smithi10066867-41 2024-04-07T16:16:51.348 INFO:tasks.rados.rados.0.smithi100.stdout:41: finishing write tid 2 to smithi10066867-41 2024-04-07T16:16:51.348 INFO:tasks.rados.rados.0.smithi100.stdout:41: finishing write tid 3 to smithi10066867-41 2024-04-07T16:16:51.348 INFO:tasks.rados.rados.0.smithi100.stdout:41: oid 41 updating version 0 to 8 2024-04-07T16:16:51.348 INFO:tasks.rados.rados.0.smithi100.stdout:41: oid 41 version 8 is already newer than 7 2024-04-07T16:16:51.348 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 41 v 8 (ObjNum 40 snap 0 seq_num 40) dirty exists 2024-04-07T16:16:51.348 INFO:tasks.rados.rados.0.smithi100.stdout:41: left oid 41 (ObjNum 40 snap 0 seq_num 40) 2024-04-07T16:16:51.348 INFO:tasks.rados.rados.0.smithi100.stdout:42: finishing write tid 1 to smithi10066867-42 2024-04-07T16:16:51.349 INFO:tasks.rados.rados.0.smithi100.stdout:42: finishing write tid 2 to smithi10066867-42 2024-04-07T16:16:51.349 INFO:tasks.rados.rados.0.smithi100.stdout:42: finishing write tid 3 to smithi10066867-42 2024-04-07T16:16:51.349 INFO:tasks.rados.rados.0.smithi100.stdout:42: oid 42 updating version 0 to 9 2024-04-07T16:16:51.349 INFO:tasks.rados.rados.0.smithi100.stdout:42: oid 42 updating version 9 to 10 2024-04-07T16:16:51.349 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 42 v 10 (ObjNum 41 snap 0 seq_num 41) dirty exists 2024-04-07T16:16:51.349 INFO:tasks.rados.rados.0.smithi100.stdout:42: left oid 42 (ObjNum 41 snap 0 seq_num 41) 2024-04-07T16:16:51.349 INFO:tasks.rados.rados.0.smithi100.stdout:43: finishing write tid 1 to smithi10066867-43 2024-04-07T16:16:51.349 INFO:tasks.rados.rados.0.smithi100.stdout:43: finishing write tid 2 to smithi10066867-43 2024-04-07T16:16:51.349 INFO:tasks.rados.rados.0.smithi100.stdout:43: finishing write tid 3 to smithi10066867-43 2024-04-07T16:16:51.349 INFO:tasks.rados.rados.0.smithi100.stdout:43: oid 43 updating version 0 to 6 2024-04-07T16:16:51.349 INFO:tasks.rados.rados.0.smithi100.stdout:43: oid 43 version 6 is already newer than 5 2024-04-07T16:16:51.349 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 43 v 6 (ObjNum 42 snap 0 seq_num 42) dirty exists 2024-04-07T16:16:51.349 INFO:tasks.rados.rados.0.smithi100.stdout:43: left oid 43 (ObjNum 42 snap 0 seq_num 42) 2024-04-07T16:16:51.350 INFO:tasks.rados.rados.0.smithi100.stdout:44: finishing write tid 1 to smithi10066867-44 2024-04-07T16:16:51.350 INFO:tasks.rados.rados.0.smithi100.stdout:44: finishing write tid 2 to smithi10066867-44 2024-04-07T16:16:51.350 INFO:tasks.rados.rados.0.smithi100.stdout:44: finishing write tid 3 to smithi10066867-44 2024-04-07T16:16:51.350 INFO:tasks.rados.rados.0.smithi100.stdout:44: oid 44 updating version 0 to 8 2024-04-07T16:16:51.350 INFO:tasks.rados.rados.0.smithi100.stdout:44: oid 44 version 8 is already newer than 7 2024-04-07T16:16:51.350 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 44 v 8 (ObjNum 43 snap 0 seq_num 43) dirty exists 2024-04-07T16:16:51.350 INFO:tasks.rados.rados.0.smithi100.stdout:44: left oid 44 (ObjNum 43 snap 0 seq_num 43) 2024-04-07T16:16:51.350 INFO:tasks.rados.rados.0.smithi100.stdout:45: finishing write tid 1 to smithi10066867-45 2024-04-07T16:16:51.350 INFO:tasks.rados.rados.0.smithi100.stdout:45: finishing write tid 2 to smithi10066867-45 2024-04-07T16:16:51.350 INFO:tasks.rados.rados.0.smithi100.stdout:36: done (15 left) 2024-04-07T16:16:51.350 INFO:tasks.rados.rados.0.smithi100.stdout:37: done (14 left) 2024-04-07T16:16:51.350 INFO:tasks.rados.rados.0.smithi100.stdout:38: done (13 left) 2024-04-07T16:16:51.350 INFO:tasks.rados.rados.0.smithi100.stdout:39: done (12 left) 2024-04-07T16:16:51.350 INFO:tasks.rados.rados.0.smithi100.stdout:40: done (11 left) 2024-04-07T16:16:51.350 INFO:tasks.rados.rados.0.smithi100.stdout:41: done (10 left) 2024-04-07T16:16:51.351 INFO:tasks.rados.rados.0.smithi100.stdout:42: done (9 left) 2024-04-07T16:16:51.351 INFO:tasks.rados.rados.0.smithi100.stdout:43: done (8 left) 2024-04-07T16:16:51.351 INFO:tasks.rados.rados.0.smithi100.stdout:44: done (7 left) 2024-04-07T16:16:51.351 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 38 current snap is 0 2024-04-07T16:16:51.351 INFO:tasks.rados.rados.0.smithi100.stdout:52: seq_num 50 ranges {622592=442368,1064960=401408} 2024-04-07T16:16:51.351 INFO:tasks.rados.rados.0.smithi100.stdout:52: writing smithi10066867-38 from 622592 to 1064960 tid 1 2024-04-07T16:16:51.354 INFO:tasks.rados.rados.0.smithi100.stdout:52: writing smithi10066867-38 from 1064960 to 1466368 tid 2 2024-04-07T16:16:51.354 INFO:tasks.rados.rados.0.smithi100.stdout:53: rmattr oid 16 current snap is 0 2024-04-07T16:16:51.354 INFO:tasks.rados.rados.0.smithi100.stdout:53: done (8 left) 2024-04-07T16:16:51.354 INFO:tasks.rados.rados.0.smithi100.stdout:54: delete oid 27 current snap is 0 2024-04-07T16:16:51.354 INFO:tasks.rados.rados.0.smithi100.stdout:45: finishing write tid 3 to smithi10066867-45 2024-04-07T16:16:51.354 INFO:tasks.rados.rados.0.smithi100.stdout:45: oid 45 updating version 0 to 3 2024-04-07T16:16:51.354 INFO:tasks.rados.rados.0.smithi100.stdout:45: oid 45 updating version 3 to 4 2024-04-07T16:16:51.354 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 45 v 4 (ObjNum 44 snap 0 seq_num 44) dirty exists 2024-04-07T16:16:51.354 INFO:tasks.rados.rados.0.smithi100.stdout:45: left oid 45 (ObjNum 44 snap 0 seq_num 44) 2024-04-07T16:16:51.354 INFO:tasks.rados.rados.0.smithi100.stdout:46: finishing write tid 1 to smithi10066867-46 2024-04-07T16:16:51.354 INFO:tasks.rados.rados.0.smithi100.stdout:46: finishing write tid 2 to smithi10066867-46 2024-04-07T16:16:51.355 INFO:tasks.rados.rados.0.smithi100.stdout:46: finishing write tid 3 to smithi10066867-46 2024-04-07T16:16:51.355 INFO:tasks.rados.rados.0.smithi100.stdout:46: oid 46 updating version 0 to 8 2024-04-07T16:16:51.355 INFO:tasks.rados.rados.0.smithi100.stdout:46: oid 46 version 8 is already newer than 7 2024-04-07T16:16:51.355 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 46 v 8 (ObjNum 45 snap 0 seq_num 45) dirty exists 2024-04-07T16:16:51.355 INFO:tasks.rados.rados.0.smithi100.stdout:46: left oid 46 (ObjNum 45 snap 0 seq_num 45) 2024-04-07T16:16:51.355 INFO:tasks.rados.rados.0.smithi100.stdout:47: finishing write tid 1 to smithi10066867-47 2024-04-07T16:16:51.355 INFO:tasks.rados.rados.0.smithi100.stdout:47: finishing write tid 2 to smithi10066867-47 2024-04-07T16:16:51.355 INFO:tasks.rados.rados.0.smithi100.stdout:47: finishing write tid 3 to smithi10066867-47 2024-04-07T16:16:51.355 INFO:tasks.rados.rados.0.smithi100.stdout:47: oid 47 updating version 0 to 4 2024-04-07T16:16:51.355 INFO:tasks.rados.rados.0.smithi100.stdout:47: oid 47 version 4 is already newer than 3 2024-04-07T16:16:51.355 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 47 v 4 (ObjNum 46 snap 0 seq_num 46) dirty exists 2024-04-07T16:16:51.355 INFO:tasks.rados.rados.0.smithi100.stdout:47: left oid 47 (ObjNum 46 snap 0 seq_num 46) 2024-04-07T16:16:51.355 INFO:tasks.rados.rados.0.smithi100.stdout:48: finishing write tid 1 to smithi10066867-48 2024-04-07T16:16:51.355 INFO:tasks.rados.rados.0.smithi100.stdout:48: finishing write tid 2 to smithi10066867-48 2024-04-07T16:16:51.355 INFO:tasks.rados.rados.0.smithi100.stdout:48: finishing write tid 3 to smithi10066867-48 2024-04-07T16:16:51.356 INFO:tasks.rados.rados.0.smithi100.stdout:48: oid 48 updating version 0 to 5 2024-04-07T16:16:51.356 INFO:tasks.rados.rados.0.smithi100.stdout:48: oid 48 updating version 5 to 6 2024-04-07T16:16:51.356 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 48 v 6 (ObjNum 47 snap 0 seq_num 47) dirty exists 2024-04-07T16:16:51.356 INFO:tasks.rados.rados.0.smithi100.stdout:48: left oid 48 (ObjNum 47 snap 0 seq_num 47) 2024-04-07T16:16:51.356 INFO:tasks.rados.rados.0.smithi100.stdout:49: finishing write tid 1 to smithi10066867-49 2024-04-07T16:16:51.356 INFO:tasks.rados.rados.0.smithi100.stdout:49: finishing write tid 2 to smithi10066867-49 2024-04-07T16:16:51.356 INFO:tasks.rados.rados.0.smithi100.stdout:49: finishing write tid 3 to smithi10066867-49 2024-04-07T16:16:51.356 INFO:tasks.rados.rados.0.smithi100.stdout:49: finishing write tid 4 to smithi10066867-49 2024-04-07T16:16:51.356 INFO:tasks.rados.rados.0.smithi100.stdout:49: oid 49 updating version 0 to 11 2024-04-07T16:16:51.356 INFO:tasks.rados.rados.0.smithi100.stdout:49: oid 49 version 11 is already newer than 10 2024-04-07T16:16:51.356 INFO:tasks.rados.rados.0.smithi100.stdout:49: oid 49 version 11 is already newer than 9 2024-04-07T16:16:51.356 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 49 v 11 (ObjNum 48 snap 0 seq_num 48) dirty exists 2024-04-07T16:16:51.356 INFO:tasks.rados.rados.0.smithi100.stdout:49: left oid 49 (ObjNum 48 snap 0 seq_num 48) 2024-04-07T16:16:51.356 INFO:tasks.rados.rados.0.smithi100.stdout:50: finishing write tid 1 to smithi10066867-50 2024-04-07T16:16:51.357 INFO:tasks.rados.rados.0.smithi100.stdout:51: finishing copy_from racing read to smithi10066867-2 2024-04-07T16:16:51.357 INFO:tasks.rados.rados.0.smithi100.stdout:50: finishing write tid 2 to smithi10066867-50 2024-04-07T16:16:51.357 INFO:tasks.rados.rados.0.smithi100.stdout:50: finishing write tid 3 to smithi10066867-50 2024-04-07T16:16:51.357 INFO:tasks.rados.rados.0.smithi100.stdout:51: finishing copy_from to smithi10066867-2 2024-04-07T16:16:51.357 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 2 v 4 (ObjNum 10 snap 0 seq_num 10) dirty exists 2024-04-07T16:16:51.357 INFO:tasks.rados.rados.0.smithi100.stdout:50: finishing write tid 4 to smithi10066867-50 2024-04-07T16:16:51.357 INFO:tasks.rados.rados.0.smithi100.stdout:50: oid 50 updating version 0 to 5 2024-04-07T16:16:51.357 INFO:tasks.rados.rados.0.smithi100.stdout:50: oid 50 updating version 5 to 6 2024-04-07T16:16:51.357 INFO:tasks.rados.rados.0.smithi100.stdout:50: oid 50 updating version 6 to 7 2024-04-07T16:16:51.357 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 50 v 7 (ObjNum 49 snap 0 seq_num 49) dirty exists 2024-04-07T16:16:51.357 INFO:tasks.rados.rados.0.smithi100.stdout:50: left oid 50 (ObjNum 49 snap 0 seq_num 49) 2024-04-07T16:16:51.358 INFO:tasks.rados.rados.0.smithi100.stdout:45: done (8 left) 2024-04-07T16:16:51.358 INFO:tasks.rados.rados.0.smithi100.stdout:46: done (7 left) 2024-04-07T16:16:51.358 INFO:tasks.rados.rados.0.smithi100.stdout:47: done (6 left) 2024-04-07T16:16:51.358 INFO:tasks.rados.rados.0.smithi100.stdout:48: done (5 left) 2024-04-07T16:16:51.358 INFO:tasks.rados.rados.0.smithi100.stdout:49: done (4 left) 2024-04-07T16:16:51.358 INFO:tasks.rados.rados.0.smithi100.stdout:50: done (3 left) 2024-04-07T16:16:51.358 INFO:tasks.rados.rados.0.smithi100.stdout:51: done (2 left) 2024-04-07T16:16:51.358 INFO:tasks.rados.rados.0.smithi100.stdout:54: done (1 left) 2024-04-07T16:16:51.359 INFO:tasks.rados.rados.0.smithi100.stdout:55: read oid 15 snap -1 2024-04-07T16:16:51.359 INFO:tasks.rados.rados.0.smithi100.stdout:55: expect (ObjNum 14 snap 0 seq_num 14) 2024-04-07T16:16:51.359 INFO:tasks.rados.rados.0.smithi100.stdout:56: rollback oid 30 current snap is 0 2024-04-07T16:16:51.359 INFO:tasks.rados.rados.0.smithi100.stdout:56: done (2 left) 2024-04-07T16:16:51.359 INFO:tasks.rados.rados.0.smithi100.stdout:57: read oid 27 snap -1 2024-04-07T16:16:51.359 INFO:tasks.rados.rados.0.smithi100.stdout:57: expect deleted 2024-04-07T16:16:51.359 INFO:tasks.rados.rados.0.smithi100.stdout:58: copy_from oid 28 from oid 47 current snap is 0 2024-04-07T16:16:51.359 INFO:tasks.rados.rados.0.smithi100.stdout:59: rollback oid 45 current snap is 0 2024-04-07T16:16:51.359 INFO:tasks.rados.rados.0.smithi100.stdout:59: done (4 left) 2024-04-07T16:16:51.359 INFO:tasks.rados.rados.0.smithi100.stdout:60: copy_from oid 22 from oid 48 current snap is 0 2024-04-07T16:16:51.359 INFO:tasks.rados.rados.0.smithi100.stdout:61: delete oid 1 current snap is 0 2024-04-07T16:16:51.360 INFO:tasks.rados.rados.0.smithi100.stdout:55: expect (ObjNum 14 snap 0 seq_num 14) 2024-04-07T16:16:51.366 INFO:tasks.rados.rados.0.smithi100.stdout:52: finishing write tid 1 to smithi10066867-38 2024-04-07T16:16:51.366 INFO:tasks.rados.rados.0.smithi100.stdout:52: finishing write tid 2 to smithi10066867-38 2024-04-07T16:16:51.366 INFO:tasks.rados.rados.0.smithi100.stdout:52: finishing write tid 3 to smithi10066867-38 2024-04-07T16:16:51.366 INFO:tasks.rados.rados.0.smithi100.stdout:60: finishing copy_from racing read to smithi10066867-22 2024-04-07T16:16:51.366 INFO:tasks.rados.rados.0.smithi100.stdout:52: finishing write tid 4 to smithi10066867-38 2024-04-07T16:16:51.366 INFO:tasks.rados.rados.0.smithi100.stdout:52: oid 38 updating version 0 to 7 2024-04-07T16:16:51.366 INFO:tasks.rados.rados.0.smithi100.stdout:52: oid 38 version 7 is already newer than 6 2024-04-07T16:16:51.366 INFO:tasks.rados.rados.0.smithi100.stdout:52: oid 38 version 7 is already newer than 5 2024-04-07T16:16:51.366 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 38 v 7 (ObjNum 50 snap 0 seq_num 50) dirty exists 2024-04-07T16:16:51.366 INFO:tasks.rados.rados.0.smithi100.stdout:52: left oid 38 (ObjNum 50 snap 0 seq_num 50) 2024-04-07T16:16:51.367 INFO:tasks.rados.rados.0.smithi100.stdout:60: finishing copy_from to smithi10066867-22 2024-04-07T16:16:51.367 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 22 v 6 (ObjNum 47 snap 0 seq_num 47) dirty exists 2024-04-07T16:16:51.367 INFO:tasks.rados.rados.0.smithi100.stdout:52: done (5 left) 2024-04-07T16:16:51.367 INFO:tasks.rados.rados.0.smithi100.stdout:55: done (4 left) 2024-04-07T16:16:51.367 INFO:tasks.rados.rados.0.smithi100.stdout:57: done (3 left) 2024-04-07T16:16:51.367 INFO:tasks.rados.rados.0.smithi100.stdout:60: done (2 left) 2024-04-07T16:16:51.367 INFO:tasks.rados.rados.0.smithi100.stdout:61: done (1 left) 2024-04-07T16:16:51.367 INFO:tasks.rados.rados.0.smithi100.stdout:62: read oid 26 snap -1 2024-04-07T16:16:51.367 INFO:tasks.rados.rados.0.smithi100.stdout:62: expect (ObjNum 25 snap 0 seq_num 25) 2024-04-07T16:16:51.367 INFO:tasks.rados.rados.0.smithi100.stdout:63: snap_create 2024-04-07T16:16:51.368 INFO:tasks.rados.rados.0.smithi100.stdout:58: finishing copy_from racing read to smithi10066867-28 2024-04-07T16:16:51.368 INFO:tasks.rados.rados.0.smithi100.stdout:62: expect (ObjNum 25 snap 0 seq_num 25) 2024-04-07T16:16:51.379 INFO:tasks.rados.rados.0.smithi100.stdout:58: finishing copy_from to smithi10066867-28 2024-04-07T16:16:51.379 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 28 v 4 (ObjNum 46 snap 0 seq_num 46) dirty exists 2024-04-07T16:16:51.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:51 smithi100 bash[21346]: audit 2024-04-07T16:16:50.211503+0000 mon.c (mon.2) 52 : audit 1 from='client.? 172.21.15.100:0/330310245' entity='client.0' cmd=[{"prefix": "osd pool set", "pool": "unique_pool_0", "var": "write_fadvise_dontneed", "val": "false"}]: dispatch 2024-04-07T16:16:51.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:51 smithi100 bash[21346]: audit 2024-04-07T16:16:50.211915+0000 mon.a (mon.0) 891 : audit 1 from='client.? ' entity='client.0' cmd=[{"prefix": "osd pool set", "pool": "unique_pool_0", "var": "write_fadvise_dontneed", "val": "false"}]: dispatch 2024-04-07T16:16:51.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:51 smithi100 bash[31660]: audit 2024-04-07T16:16:50.211503+0000 mon.c (mon.2) 52 : audit 1 from='client.? 172.21.15.100:0/330310245' entity='client.0' cmd=[{"prefix": "osd pool set", "pool": "unique_pool_0", "var": "write_fadvise_dontneed", "val": "false"}]: dispatch 2024-04-07T16:16:51.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:51 smithi100 bash[31660]: audit 2024-04-07T16:16:50.211915+0000 mon.a (mon.0) 891 : audit 1 from='client.? ' entity='client.0' cmd=[{"prefix": "osd pool set", "pool": "unique_pool_0", "var": "write_fadvise_dontneed", "val": "false"}]: dispatch 2024-04-07T16:16:51.605 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:51 smithi144 bash[23522]: audit 2024-04-07T16:16:50.211503+0000 mon.c (mon.2) 52 : audit 1 from='client.? 172.21.15.100:0/330310245' entity='client.0' cmd=[{"prefix": "osd pool set", "pool": "unique_pool_0", "var": "write_fadvise_dontneed", "val": "false"}]: dispatch 2024-04-07T16:16:51.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:51 smithi144 bash[23522]: audit 2024-04-07T16:16:50.211915+0000 mon.a (mon.0) 891 : audit 1 from='client.? ' entity='client.0' cmd=[{"prefix": "osd pool set", "pool": "unique_pool_0", "var": "write_fadvise_dontneed", "val": "false"}]: dispatch 2024-04-07T16:16:52.193 INFO:tasks.rados.rados.0.smithi100.stdout:58: done (2 left) 2024-04-07T16:16:52.194 INFO:tasks.rados.rados.0.smithi100.stdout:62: done (1 left) 2024-04-07T16:16:52.194 INFO:tasks.rados.rados.0.smithi100.stdout:63: done (0 left) 2024-04-07T16:16:52.194 INFO:tasks.rados.rados.0.smithi100.stdout:64: copy_from oid 15 from oid 2 current snap is 1 2024-04-07T16:16:52.194 INFO:tasks.rados.rados.0.smithi100.stdout:65: read oid 30 snap -1 2024-04-07T16:16:52.194 INFO:tasks.rados.rados.0.smithi100.stdout:65: expect (ObjNum 29 snap 0 seq_num 29) 2024-04-07T16:16:52.194 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 23 current snap is 1 2024-04-07T16:16:52.194 INFO:tasks.rados.rados.0.smithi100.stdout:66: seq_num 53 ranges {376832=491520,868352=401408} 2024-04-07T16:16:52.199 INFO:tasks.rados.rados.0.smithi100.stdout:66: writing smithi10066867-23 from 376832 to 868352 tid 1 2024-04-07T16:16:52.202 INFO:tasks.rados.rados.0.smithi100.stdout:66: writing smithi10066867-23 from 868352 to 1269760 tid 2 2024-04-07T16:16:52.202 INFO:tasks.rados.rados.0.smithi100.stdout:67: rollback oid 10 current snap is 1 2024-04-07T16:16:52.202 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 10 to 0 2024-04-07T16:16:52.202 INFO:tasks.rados.rados.0.smithi100.stdout:68: setattr oid 32 current snap is 1 2024-04-07T16:16:52.203 INFO:tasks.rados.rados.0.smithi100.stdout:69: copy_from oid 7 from oid 17 current snap is 1 2024-04-07T16:16:52.203 INFO:tasks.rados.rados.0.smithi100.stdout:70: read oid 20 snap 0 2024-04-07T16:16:52.204 INFO:tasks.rados.rados.0.smithi100.stdout:70: expect (ObjNum 19 snap 0 seq_num 19) 2024-04-07T16:16:52.204 INFO:tasks.rados.rados.0.smithi100.stdout:71: copy_from oid 24 from oid 29 current snap is 1 2024-04-07T16:16:52.204 INFO:tasks.rados.rados.0.smithi100.stdout:72: rollback oid 6 current snap is 1 2024-04-07T16:16:52.204 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 6 to 0 2024-04-07T16:16:52.204 INFO:tasks.rados.rados.0.smithi100.stdout:73: read oid 13 snap 0 2024-04-07T16:16:52.204 INFO:tasks.rados.rados.0.smithi100.stdout:73: expect (ObjNum 12 snap 0 seq_num 12) 2024-04-07T16:16:52.204 INFO:tasks.rados.rados.0.smithi100.stdout:74: rmattr oid 4 current snap is 1 2024-04-07T16:16:52.204 INFO:tasks.rados.rados.0.smithi100.stdout:74: done (10 left) 2024-04-07T16:16:52.204 INFO:tasks.rados.rados.0.smithi100.stdout:75: copy_from oid 48 from oid 1 current snap is 1 2024-04-07T16:16:52.204 INFO:tasks.rados.rados.0.smithi100.stdout:76: read oid 27 snap -1 2024-04-07T16:16:52.204 INFO:tasks.rados.rados.0.smithi100.stdout:76: expect deleted 2024-04-07T16:16:52.204 INFO:tasks.rados.rados.0.smithi100.stdout:77: copy_from oid 28 from oid 34 current snap is 1 2024-04-07T16:16:52.204 INFO:tasks.rados.rados.0.smithi100.stdout:78: read oid 43 snap -1 2024-04-07T16:16:52.204 INFO:tasks.rados.rados.0.smithi100.stdout:78: expect (ObjNum 42 snap 0 seq_num 42) 2024-04-07T16:16:52.204 INFO:tasks.rados.rados.0.smithi100.stdout:79: setattr oid 14 current snap is 1 2024-04-07T16:16:52.205 INFO:tasks.rados.rados.0.smithi100.stdout:80: snap_create 2024-04-07T16:16:52.210 INFO:tasks.rados.rados.0.smithi100.stdout:67: finishing rollback tid 0 to smithi10066867-10 2024-04-07T16:16:52.212 INFO:tasks.rados.rados.0.smithi100.stdout:67: finishing rollback tid 1 to smithi10066867-10 2024-04-07T16:16:52.219 INFO:tasks.rados.rados.0.smithi100.stdout:73: expect (ObjNum 12 snap 0 seq_num 12) 2024-04-07T16:16:52.237 INFO:tasks.rados.rados.0.smithi100.stdout:67: finishing rollback tid 2 to smithi10066867-10 2024-04-07T16:16:52.237 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 10 v 9 (ObjNum 9 snap 0 seq_num 9) dirty exists 2024-04-07T16:16:52.237 INFO:tasks.rados.rados.0.smithi100.stdout:78: expect (ObjNum 42 snap 0 seq_num 42) 2024-04-07T16:16:52.276 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 32 v 7 (ObjNum 31 snap 0 seq_num 31) dirty exists 2024-04-07T16:16:52.276 INFO:tasks.rados.rados.0.smithi100.stdout:75: finishing copy_from to smithi10066867-48 2024-04-07T16:16:52.276 INFO:tasks.rados.rados.0.smithi100.stdout:75: got expected ENOENT (src dne) 2024-04-07T16:16:52.276 INFO:tasks.rados.rados.0.smithi100.stdout:75: finishing copy_from racing read to smithi10066867-48 2024-04-07T16:16:52.276 INFO:tasks.rados.rados.0.smithi100.stdout:69: finishing copy_from racing read to smithi10066867-7 2024-04-07T16:16:52.276 INFO:tasks.rados.rados.0.smithi100.stdout:65: expect (ObjNum 29 snap 0 seq_num 29) 2024-04-07T16:16:52.294 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 14 v 9 (ObjNum 13 snap 0 seq_num 13) dirty exists 2024-04-07T16:16:52.294 INFO:tasks.rados.rados.0.smithi100.stdout:66: finishing write tid 1 to smithi10066867-23 2024-04-07T16:16:52.294 INFO:tasks.rados.rados.0.smithi100.stdout:66: finishing write tid 2 to smithi10066867-23 2024-04-07T16:16:52.294 INFO:tasks.rados.rados.0.smithi100.stdout:70: expect (ObjNum 19 snap 0 seq_num 19) 2024-04-07T16:16:52.308 INFO:tasks.rados.rados.0.smithi100.stdout:66: finishing write tid 3 to smithi10066867-23 2024-04-07T16:16:52.308 INFO:tasks.rados.rados.0.smithi100.stdout:69: finishing copy_from to smithi10066867-7 2024-04-07T16:16:52.308 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 7 v 2 (ObjNum 16 snap 0 seq_num 16) dirty exists 2024-04-07T16:16:52.308 INFO:tasks.rados.rados.0.smithi100.stdout:72: finishing rollback tid 0 to smithi10066867-6 2024-04-07T16:16:52.308 INFO:tasks.rados.rados.0.smithi100.stdout:72: finishing rollback tid 1 to smithi10066867-6 2024-04-07T16:16:52.308 INFO:tasks.rados.rados.0.smithi100.stdout:72: finishing rollback tid 2 to smithi10066867-6 2024-04-07T16:16:52.308 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 6 v 13 (ObjNum 5 snap 0 seq_num 5) dirty exists 2024-04-07T16:16:52.309 INFO:tasks.rados.rados.0.smithi100.stdout:66: finishing write tid 4 to smithi10066867-23 2024-04-07T16:16:52.309 INFO:tasks.rados.rados.0.smithi100.stdout:66: oid 23 updating version 0 to 16 2024-04-07T16:16:52.309 INFO:tasks.rados.rados.0.smithi100.stdout:66: oid 23 version 16 is already newer than 14 2024-04-07T16:16:52.309 INFO:tasks.rados.rados.0.smithi100.stdout:66: oid 23 version 16 is already newer than 15 2024-04-07T16:16:52.309 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 23 v 16 (ObjNum 53 snap 1 seq_num 53) dirty exists 2024-04-07T16:16:52.309 INFO:tasks.rados.rados.0.smithi100.stdout:66: left oid 23 (ObjNum 53 snap 1 seq_num 53) 2024-04-07T16:16:52.309 INFO:tasks.rados.rados.0.smithi100.stdout:77: finishing copy_from racing read to smithi10066867-28 2024-04-07T16:16:52.309 INFO:tasks.rados.rados.0.smithi100.stdout:77: finishing copy_from to smithi10066867-28 2024-04-07T16:16:52.309 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 28 v 6 (ObjNum 33 snap 0 seq_num 33) dirty exists 2024-04-07T16:16:52.309 INFO:tasks.rados.rados.0.smithi100.stdout:71: finishing copy_from racing read to smithi10066867-24 2024-04-07T16:16:52.309 INFO:tasks.rados.rados.0.smithi100.stdout:64: finishing copy_from racing read to smithi10066867-15 2024-04-07T16:16:52.309 INFO:tasks.rados.rados.0.smithi100.stdout:71: finishing copy_from to smithi10066867-24 2024-04-07T16:16:52.309 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 24 v 2 (ObjNum 28 snap 0 seq_num 28) dirty exists 2024-04-07T16:16:52.309 INFO:tasks.rados.rados.0.smithi100.stdout:64: finishing copy_from to smithi10066867-15 2024-04-07T16:16:52.309 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 15 v 4 (ObjNum 10 snap 0 seq_num 10) dirty exists 2024-04-07T16:16:52.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:52 smithi144 bash[23522]: cluster 2024-04-07T16:16:50.692986+0000 mgr.y (mgr.14553) 179 : cluster 0 pgmap v132: 121 pgs: 121 active+clean; 580 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:16:52.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:52 smithi144 bash[23522]: audit 2024-04-07T16:16:51.202839+0000 mon.a (mon.0) 892 : audit 1 from='client.? ' entity='client.0' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_0", "var": "write_fadvise_dontneed", "val": "false"}]': finished 2024-04-07T16:16:52.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:52 smithi144 bash[23522]: cluster 2024-04-07T16:16:51.203014+0000 mon.a (mon.0) 893 : cluster 0 osdmap e76: 8 total, 8 up, 8 in 2024-04-07T16:16:52.687 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:52 smithi100 bash[21346]: cluster 2024-04-07T16:16:50.692986+0000 mgr.y (mgr.14553) 179 : cluster 0 pgmap v132: 121 pgs: 121 active+clean; 580 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:16:52.687 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:52 smithi100 bash[21346]: audit 2024-04-07T16:16:51.202839+0000 mon.a (mon.0) 892 : audit 1 from='client.? ' entity='client.0' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_0", "var": "write_fadvise_dontneed", "val": "false"}]': finished 2024-04-07T16:16:52.687 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:52 smithi100 bash[21346]: cluster 2024-04-07T16:16:51.203014+0000 mon.a (mon.0) 893 : cluster 0 osdmap e76: 8 total, 8 up, 8 in 2024-04-07T16:16:52.688 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:52 smithi100 bash[31660]: cluster 2024-04-07T16:16:50.692986+0000 mgr.y (mgr.14553) 179 : cluster 0 pgmap v132: 121 pgs: 121 active+clean; 580 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:16:52.688 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:52 smithi100 bash[31660]: audit 2024-04-07T16:16:51.202839+0000 mon.a (mon.0) 892 : audit 1 from='client.? ' entity='client.0' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_0", "var": "write_fadvise_dontneed", "val": "false"}]': finished 2024-04-07T16:16:52.688 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:52 smithi100 bash[31660]: cluster 2024-04-07T16:16:51.203014+0000 mon.a (mon.0) 893 : cluster 0 osdmap e76: 8 total, 8 up, 8 in 2024-04-07T16:16:53.223 INFO:tasks.rados.rados.0.smithi100.stdout:64: done (15 left) 2024-04-07T16:16:53.223 INFO:tasks.rados.rados.0.smithi100.stdout:65: done (14 left) 2024-04-07T16:16:53.223 INFO:tasks.rados.rados.0.smithi100.stdout:66: done (13 left) 2024-04-07T16:16:53.223 INFO:tasks.rados.rados.0.smithi100.stdout:67: done (12 left) 2024-04-07T16:16:53.223 INFO:tasks.rados.rados.0.smithi100.stdout:68: done (11 left) 2024-04-07T16:16:53.224 INFO:tasks.rados.rados.0.smithi100.stdout:69: done (10 left) 2024-04-07T16:16:53.224 INFO:tasks.rados.rados.0.smithi100.stdout:70: done (9 left) 2024-04-07T16:16:53.224 INFO:tasks.rados.rados.0.smithi100.stdout:71: done (8 left) 2024-04-07T16:16:53.224 INFO:tasks.rados.rados.0.smithi100.stdout:72: done (7 left) 2024-04-07T16:16:53.224 INFO:tasks.rados.rados.0.smithi100.stdout:73: done (6 left) 2024-04-07T16:16:53.224 INFO:tasks.rados.rados.0.smithi100.stdout:75: done (5 left) 2024-04-07T16:16:53.224 INFO:tasks.rados.rados.0.smithi100.stdout:76: done (4 left) 2024-04-07T16:16:53.224 INFO:tasks.rados.rados.0.smithi100.stdout:77: done (3 left) 2024-04-07T16:16:53.224 INFO:tasks.rados.rados.0.smithi100.stdout:78: done (2 left) 2024-04-07T16:16:53.224 INFO:tasks.rados.rados.0.smithi100.stdout:79: done (1 left) 2024-04-07T16:16:53.224 INFO:tasks.rados.rados.0.smithi100.stdout:80: done (0 left) 2024-04-07T16:16:53.224 INFO:tasks.rados.rados.0.smithi100.stdout:81: rollback oid 40 current snap is 2 2024-04-07T16:16:53.224 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 40 to 1 2024-04-07T16:16:53.225 INFO:tasks.rados.rados.0.smithi100.stdout:82: snap_create 2024-04-07T16:16:53.232 INFO:tasks.rados.rados.0.smithi100.stdout:81: finishing rollback tid 0 to smithi10066867-40 2024-04-07T16:16:53.235 INFO:tasks.rados.rados.0.smithi100.stdout:81: finishing rollback tid 1 to smithi10066867-40 2024-04-07T16:16:53.236 INFO:tasks.rados.rados.0.smithi100.stdout:81: finishing rollback tid 2 to smithi10066867-40 2024-04-07T16:16:53.236 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 40 v 17 (ObjNum 39 snap 0 seq_num 39) dirty exists 2024-04-07T16:16:53.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:53 smithi144 bash[23522]: cluster 2024-04-07T16:16:52.206052+0000 mon.a (mon.0) 894 : cluster 0 osdmap e77: 8 total, 8 up, 8 in 2024-04-07T16:16:53.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:53 smithi144 bash[23522]: audit 2024-04-07T16:16:52.702635+0000 mon.b (mon.1) 239 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:16:53.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:53 smithi144 bash[23522]: audit 2024-04-07T16:16:53.119175+0000 mon.a (mon.0) 895 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:16:53.687 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:53 smithi100 bash[21346]: cluster 2024-04-07T16:16:52.206052+0000 mon.a (mon.0) 894 : cluster 0 osdmap e77: 8 total, 8 up, 8 in 2024-04-07T16:16:53.687 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:53 smithi100 bash[21346]: audit 2024-04-07T16:16:52.702635+0000 mon.b (mon.1) 239 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:16:53.687 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:53 smithi100 bash[21346]: audit 2024-04-07T16:16:53.119175+0000 mon.a (mon.0) 895 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:16:53.687 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:53 smithi100 bash[31660]: cluster 2024-04-07T16:16:52.206052+0000 mon.a (mon.0) 894 : cluster 0 osdmap e77: 8 total, 8 up, 8 in 2024-04-07T16:16:53.687 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:53 smithi100 bash[31660]: audit 2024-04-07T16:16:52.702635+0000 mon.b (mon.1) 239 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:16:53.688 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:53 smithi100 bash[31660]: audit 2024-04-07T16:16:53.119175+0000 mon.a (mon.0) 895 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:16:54.220 INFO:tasks.rados.rados.0.smithi100.stdout:81: done (1 left) 2024-04-07T16:16:54.241 INFO:tasks.rados.rados.0.smithi100.stdout:82: done (0 left) 2024-04-07T16:16:54.242 INFO:tasks.rados.rados.0.smithi100.stdout:83: rollback oid 9 current snap is 3 2024-04-07T16:16:54.242 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 9 to 0 2024-04-07T16:16:54.242 INFO:tasks.rados.rados.0.smithi100.stdout:84: read oid 20 snap 1 2024-04-07T16:16:54.242 INFO:tasks.rados.rados.0.smithi100.stdout:84: expect (ObjNum 19 snap 0 seq_num 19) 2024-04-07T16:16:54.242 INFO:tasks.rados.rados.0.smithi100.stdout:85: read oid 41 snap -1 2024-04-07T16:16:54.242 INFO:tasks.rados.rados.0.smithi100.stdout:85: expect (ObjNum 40 snap 0 seq_num 40) 2024-04-07T16:16:54.242 INFO:tasks.rados.rados.0.smithi100.stdout:86: read oid 5 snap -1 2024-04-07T16:16:54.242 INFO:tasks.rados.rados.0.smithi100.stdout:86: expect (ObjNum 4 snap 0 seq_num 4) 2024-04-07T16:16:54.242 INFO:tasks.rados.rados.0.smithi100.stdout:87: read oid 4 snap -1 2024-04-07T16:16:54.243 INFO:tasks.rados.rados.0.smithi100.stdout:87: expect (ObjNum 3 snap 0 seq_num 3) 2024-04-07T16:16:54.243 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 3 current snap is 3 2024-04-07T16:16:54.243 INFO:tasks.rados.rados.0.smithi100.stdout:88: seq_num 54 ranges {40960=507904,548864=401408} 2024-04-07T16:16:54.244 INFO:tasks.rados.rados.0.smithi100.stdout:88: writing smithi10066867-3 from 40960 to 548864 tid 1 2024-04-07T16:16:54.244 INFO:tasks.rados.rados.0.smithi100.stdout:88: writing smithi10066867-3 from 548864 to 950272 tid 2 2024-04-07T16:16:54.244 INFO:tasks.rados.rados.0.smithi100.stdout:89: read oid 27 snap -1 2024-04-07T16:16:54.244 INFO:tasks.rados.rados.0.smithi100.stdout:89: expect deleted 2024-04-07T16:16:54.244 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 17 current snap is 3 2024-04-07T16:16:54.244 INFO:tasks.rados.rados.0.smithi100.stdout:90: seq_num 55 ranges {270336=524288,794624=401408} 2024-04-07T16:16:54.244 INFO:tasks.rados.rados.0.smithi100.stdout:90: writing smithi10066867-17 from 270336 to 794624 tid 1 2024-04-07T16:16:54.245 INFO:tasks.rados.rados.0.smithi100.stdout:90: writing smithi10066867-17 from 794624 to 1196032 tid 2 2024-04-07T16:16:54.245 INFO:tasks.rados.rados.0.smithi100.stdout:84: expect (ObjNum 19 snap 0 seq_num 19) 2024-04-07T16:16:54.247 INFO:tasks.rados.rados.0.smithi100.stdout:84: done (7 left) 2024-04-07T16:16:54.247 INFO:tasks.rados.rados.0.smithi100.stdout:91: rollback oid 37 current snap is 3 2024-04-07T16:16:54.247 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 37 to 2 2024-04-07T16:16:54.247 INFO:tasks.rados.rados.0.smithi100.stdout:92: rollback oid 22 current snap is 3 2024-04-07T16:16:54.247 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 22 to 0 2024-04-07T16:16:54.247 INFO:tasks.rados.rados.0.smithi100.stdout:93: snap_create 2024-04-07T16:16:54.247 INFO:tasks.rados.rados.0.smithi100.stdout:87: expect (ObjNum 3 snap 0 seq_num 3) 2024-04-07T16:16:54.248 INFO:tasks.rados.rados.0.smithi100.stdout:88: finishing write tid 1 to smithi10066867-3 2024-04-07T16:16:54.248 INFO:tasks.rados.rados.0.smithi100.stdout:83: finishing rollback tid 0 to smithi10066867-9 2024-04-07T16:16:54.248 INFO:tasks.rados.rados.0.smithi100.stdout:88: finishing write tid 2 to smithi10066867-3 2024-04-07T16:16:54.248 INFO:tasks.rados.rados.0.smithi100.stdout:83: finishing rollback tid 1 to smithi10066867-9 2024-04-07T16:16:54.249 INFO:tasks.rados.rados.0.smithi100.stdout:83: finishing rollback tid 2 to smithi10066867-9 2024-04-07T16:16:54.249 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 9 v 6 (ObjNum 8 snap 0 seq_num 8) dirty exists 2024-04-07T16:16:54.249 INFO:tasks.rados.rados.0.smithi100.stdout:88: finishing write tid 3 to smithi10066867-3 2024-04-07T16:16:54.249 INFO:tasks.rados.rados.0.smithi100.stdout:88: finishing write tid 4 to smithi10066867-3 2024-04-07T16:16:54.249 INFO:tasks.rados.rados.0.smithi100.stdout:88: oid 3 updating version 0 to 11 2024-04-07T16:16:54.249 INFO:tasks.rados.rados.0.smithi100.stdout:88: oid 3 version 11 is already newer than 10 2024-04-07T16:16:54.249 INFO:tasks.rados.rados.0.smithi100.stdout:88: oid 3 version 11 is already newer than 9 2024-04-07T16:16:54.249 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 3 v 11 (ObjNum 54 snap 3 seq_num 54) dirty exists 2024-04-07T16:16:54.249 INFO:tasks.rados.rados.0.smithi100.stdout:88: left oid 3 (ObjNum 54 snap 3 seq_num 54) 2024-04-07T16:16:54.249 INFO:tasks.rados.rados.0.smithi100.stdout:85: expect (ObjNum 40 snap 0 seq_num 40) 2024-04-07T16:16:54.269 INFO:tasks.rados.rados.0.smithi100.stdout:90: finishing write tid 1 to smithi10066867-17 2024-04-07T16:16:54.269 INFO:tasks.rados.rados.0.smithi100.stdout:90: finishing write tid 2 to smithi10066867-17 2024-04-07T16:16:54.269 INFO:tasks.rados.rados.0.smithi100.stdout:90: finishing write tid 3 to smithi10066867-17 2024-04-07T16:16:54.269 INFO:tasks.rados.rados.0.smithi100.stdout:90: finishing write tid 4 to smithi10066867-17 2024-04-07T16:16:54.269 INFO:tasks.rados.rados.0.smithi100.stdout:90: oid 17 updating version 0 to 12 2024-04-07T16:16:54.269 INFO:tasks.rados.rados.0.smithi100.stdout:90: oid 17 updating version 12 to 14 2024-04-07T16:16:54.269 INFO:tasks.rados.rados.0.smithi100.stdout:90: oid 17 version 14 is already newer than 13 2024-04-07T16:16:54.270 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 17 v 14 (ObjNum 55 snap 3 seq_num 55) dirty exists 2024-04-07T16:16:54.270 INFO:tasks.rados.rados.0.smithi100.stdout:90: left oid 17 (ObjNum 55 snap 3 seq_num 55) 2024-04-07T16:16:54.270 INFO:tasks.rados.rados.0.smithi100.stdout:86: expect (ObjNum 4 snap 0 seq_num 4) 2024-04-07T16:16:54.271 INFO:tasks.rados.rados.0.smithi100.stdout:92: finishing rollback tid 0 to smithi10066867-22 2024-04-07T16:16:54.271 INFO:tasks.rados.rados.0.smithi100.stdout:92: finishing rollback tid 1 to smithi10066867-22 2024-04-07T16:16:54.271 INFO:tasks.rados.rados.0.smithi100.stdout:91: finishing rollback tid 0 to smithi10066867-37 2024-04-07T16:16:54.271 INFO:tasks.rados.rados.0.smithi100.stdout:91: finishing rollback tid 1 to smithi10066867-37 2024-04-07T16:16:54.271 INFO:tasks.rados.rados.0.smithi100.stdout:91: finishing rollback tid 2 to smithi10066867-37 2024-04-07T16:16:54.271 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 37 v 14 (ObjNum 36 snap 0 seq_num 36) dirty exists 2024-04-07T16:16:54.271 INFO:tasks.rados.rados.0.smithi100.stdout:92: finishing rollback tid 2 to smithi10066867-22 2024-04-07T16:16:54.271 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 22 v 13 (ObjNum 47 snap 0 seq_num 47) dirty exists 2024-04-07T16:16:54.611 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:54 smithi144 bash[23522]: cluster 2024-04-07T16:16:52.693618+0000 mgr.y (mgr.14553) 180 : cluster 0 pgmap v135: 121 pgs: 121 active+clean; 580 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:16:54.611 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:54 smithi144 bash[23522]: cluster 2024-04-07T16:16:53.236763+0000 mon.a (mon.0) 896 : cluster 0 osdmap e78: 8 total, 8 up, 8 in 2024-04-07T16:16:54.687 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:54 smithi100 bash[21346]: cluster 2024-04-07T16:16:52.693618+0000 mgr.y (mgr.14553) 180 : cluster 0 pgmap v135: 121 pgs: 121 active+clean; 580 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:16:54.687 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:54 smithi100 bash[21346]: cluster 2024-04-07T16:16:53.236763+0000 mon.a (mon.0) 896 : cluster 0 osdmap e78: 8 total, 8 up, 8 in 2024-04-07T16:16:54.688 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:54 smithi100 bash[31660]: cluster 2024-04-07T16:16:52.693618+0000 mgr.y (mgr.14553) 180 : cluster 0 pgmap v135: 121 pgs: 121 active+clean; 580 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:16:54.688 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:54 smithi100 bash[31660]: cluster 2024-04-07T16:16:53.236763+0000 mon.a (mon.0) 896 : cluster 0 osdmap e78: 8 total, 8 up, 8 in 2024-04-07T16:16:55.236 INFO:tasks.rados.rados.0.smithi100.stdout:83: done (9 left) 2024-04-07T16:16:55.236 INFO:tasks.rados.rados.0.smithi100.stdout:85: done (8 left) 2024-04-07T16:16:55.236 INFO:tasks.rados.rados.0.smithi100.stdout:86: done (7 left) 2024-04-07T16:16:55.236 INFO:tasks.rados.rados.0.smithi100.stdout:87: done (6 left) 2024-04-07T16:16:55.236 INFO:tasks.rados.rados.0.smithi100.stdout:88: done (5 left) 2024-04-07T16:16:55.236 INFO:tasks.rados.rados.0.smithi100.stdout:89: done (4 left) 2024-04-07T16:16:55.236 INFO:tasks.rados.rados.0.smithi100.stdout:90: done (3 left) 2024-04-07T16:16:55.237 INFO:tasks.rados.rados.0.smithi100.stdout:91: done (2 left) 2024-04-07T16:16:55.237 INFO:tasks.rados.rados.0.smithi100.stdout:92: done (1 left) 2024-04-07T16:16:55.237 INFO:tasks.rados.rados.0.smithi100.stdout:93: done (0 left) 2024-04-07T16:16:55.237 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 27 current snap is 4 2024-04-07T16:16:55.237 INFO:tasks.rados.rados.0.smithi100.stdout:94: seq_num 56 ranges {0=540672,540672=401408} 2024-04-07T16:16:55.241 INFO:tasks.rados.rados.0.smithi100.stdout:94: writing smithi10066867-27 from 0 to 540672 tid 1 2024-04-07T16:16:55.245 INFO:tasks.rados.rados.0.smithi100.stdout:94: writing smithi10066867-27 from 540672 to 942080 tid 2 2024-04-07T16:16:55.245 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 39 current snap is 4 2024-04-07T16:16:55.245 INFO:tasks.rados.rados.0.smithi100.stdout:95: seq_num 57 ranges {638976=557056,1196032=401408} 2024-04-07T16:16:55.256 INFO:tasks.rados.rados.0.smithi100.stdout:95: writing smithi10066867-39 from 638976 to 1196032 tid 1 2024-04-07T16:16:55.260 INFO:tasks.rados.rados.0.smithi100.stdout:95: writing smithi10066867-39 from 1196032 to 1597440 tid 2 2024-04-07T16:16:55.260 INFO:tasks.rados.rados.0.smithi100.stdout:96: read oid 45 snap -1 2024-04-07T16:16:55.260 INFO:tasks.rados.rados.0.smithi100.stdout:96: expect (ObjNum 44 snap 0 seq_num 44) 2024-04-07T16:16:55.260 INFO:tasks.rados.rados.0.smithi100.stdout:97: rollback oid 12 current snap is 4 2024-04-07T16:16:55.261 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 12 to 3 2024-04-07T16:16:55.261 INFO:tasks.rados.rados.0.smithi100.stdout:94: finishing write tid 1 to smithi10066867-27 2024-04-07T16:16:55.261 INFO:tasks.rados.rados.0.smithi100.stdout:94: finishing write tid 2 to smithi10066867-27 2024-04-07T16:16:55.261 INFO:tasks.rados.rados.0.smithi100.stdout:94: finishing write tid 3 to smithi10066867-27 2024-04-07T16:16:55.261 INFO:tasks.rados.rados.0.smithi100.stdout:98: snap_remove snap 0 2024-04-07T16:16:55.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:55 smithi100 bash[21346]: cluster 2024-04-07T16:16:54.232756+0000 mon.a (mon.0) 897 : cluster 0 osdmap e79: 8 total, 8 up, 8 in 2024-04-07T16:16:55.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:55 smithi100 bash[31660]: cluster 2024-04-07T16:16:54.232756+0000 mon.a (mon.0) 897 : cluster 0 osdmap e79: 8 total, 8 up, 8 in 2024-04-07T16:16:55.438 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:16:55 smithi100 bash[21601]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:16:55] "GET /metrics HTTP/1.1" 200 36020 "" "Prometheus/2.43.0" 2024-04-07T16:16:55.605 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:55 smithi144 bash[23522]: cluster 2024-04-07T16:16:54.232756+0000 mon.a (mon.0) 897 : cluster 0 osdmap e79: 8 total, 8 up, 8 in 2024-04-07T16:16:56.251 INFO:tasks.rados.rados.0.smithi100.stdout:94: finishing write tid 4 to smithi10066867-27 2024-04-07T16:16:56.251 INFO:tasks.rados.rados.0.smithi100.stdout:94: oid 27 updating version 0 to 12 2024-04-07T16:16:56.251 INFO:tasks.rados.rados.0.smithi100.stdout:94: oid 27 version 12 is already newer than 10 2024-04-07T16:16:56.251 INFO:tasks.rados.rados.0.smithi100.stdout:94: oid 27 version 12 is already newer than 11 2024-04-07T16:16:56.252 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 27 v 12 (ObjNum 56 snap 4 seq_num 56) dirty exists 2024-04-07T16:16:56.252 INFO:tasks.rados.rados.0.smithi100.stdout:94: left oid 27 (ObjNum 56 snap 4 seq_num 56) 2024-04-07T16:16:56.252 INFO:tasks.rados.rados.0.smithi100.stdout:95: finishing write tid 1 to smithi10066867-39 2024-04-07T16:16:56.252 INFO:tasks.rados.rados.0.smithi100.stdout:97: finishing rollback tid 0 to smithi10066867-12 2024-04-07T16:16:56.252 INFO:tasks.rados.rados.0.smithi100.stdout:97: finishing rollback tid 1 to smithi10066867-12 2024-04-07T16:16:56.252 INFO:tasks.rados.rados.0.smithi100.stdout:97: finishing rollback tid 2 to smithi10066867-12 2024-04-07T16:16:56.252 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 12 v 15 (ObjNum 11 snap 0 seq_num 11) dirty exists 2024-04-07T16:16:56.252 INFO:tasks.rados.rados.0.smithi100.stdout:95: finishing write tid 2 to smithi10066867-39 2024-04-07T16:16:56.252 INFO:tasks.rados.rados.0.smithi100.stdout:95: finishing write tid 3 to smithi10066867-39 2024-04-07T16:16:56.252 INFO:tasks.rados.rados.0.smithi100.stdout:95: finishing write tid 4 to smithi10066867-39 2024-04-07T16:16:56.252 INFO:tasks.rados.rados.0.smithi100.stdout:95: oid 39 updating version 0 to 18 2024-04-07T16:16:56.252 INFO:tasks.rados.rados.0.smithi100.stdout:95: oid 39 version 18 is already newer than 16 2024-04-07T16:16:56.252 INFO:tasks.rados.rados.0.smithi100.stdout:95: oid 39 version 18 is already newer than 17 2024-04-07T16:16:56.252 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 39 v 18 (ObjNum 57 snap 4 seq_num 57) dirty exists 2024-04-07T16:16:56.253 INFO:tasks.rados.rados.0.smithi100.stdout:95: left oid 39 (ObjNum 57 snap 4 seq_num 57) 2024-04-07T16:16:56.253 INFO:tasks.rados.rados.0.smithi100.stdout:96: expect (ObjNum 44 snap 0 seq_num 44) 2024-04-07T16:16:56.273 INFO:tasks.rados.rados.0.smithi100.stdout:94: done (4 left) 2024-04-07T16:16:56.273 INFO:tasks.rados.rados.0.smithi100.stdout:95: done (3 left) 2024-04-07T16:16:56.273 INFO:tasks.rados.rados.0.smithi100.stdout:96: done (2 left) 2024-04-07T16:16:56.273 INFO:tasks.rados.rados.0.smithi100.stdout:97: done (1 left) 2024-04-07T16:16:56.273 INFO:tasks.rados.rados.0.smithi100.stdout:98: done (0 left) 2024-04-07T16:16:56.273 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 12 current snap is 4 2024-04-07T16:16:56.273 INFO:tasks.rados.rados.0.smithi100.stdout:99: seq_num 58 ranges {188416=573440,761856=401408} 2024-04-07T16:16:56.277 INFO:tasks.rados.rados.0.smithi100.stdout:99: writing smithi10066867-12 from 188416 to 761856 tid 1 2024-04-07T16:16:56.280 INFO:tasks.rados.rados.0.smithi100.stdout:99: writing smithi10066867-12 from 761856 to 1163264 tid 2 2024-04-07T16:16:56.280 INFO:tasks.rados.rados.0.smithi100.stdout:100: read oid 1 snap -1 2024-04-07T16:16:56.281 INFO:tasks.rados.rados.0.smithi100.stdout:100: expect deleted 2024-04-07T16:16:56.281 INFO:tasks.rados.rados.0.smithi100.stdout:101: snap_create 2024-04-07T16:16:56.282 INFO:tasks.rados.rados.0.smithi100.stdout:99: finishing write tid 1 to smithi10066867-12 2024-04-07T16:16:56.283 INFO:tasks.rados.rados.0.smithi100.stdout:99: finishing write tid 2 to smithi10066867-12 2024-04-07T16:16:56.283 INFO:tasks.rados.rados.0.smithi100.stdout:99: finishing write tid 3 to smithi10066867-12 2024-04-07T16:16:56.284 INFO:tasks.rados.rados.0.smithi100.stdout:99: finishing write tid 4 to smithi10066867-12 2024-04-07T16:16:56.284 INFO:tasks.rados.rados.0.smithi100.stdout:99: oid 12 updating version 0 to 20 2024-04-07T16:16:56.284 INFO:tasks.rados.rados.0.smithi100.stdout:99: oid 12 version 20 is already newer than 19 2024-04-07T16:16:56.284 INFO:tasks.rados.rados.0.smithi100.stdout:99: oid 12 version 20 is already newer than 18 2024-04-07T16:16:56.284 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 12 v 20 (ObjNum 58 snap 4 seq_num 58) dirty exists 2024-04-07T16:16:56.284 INFO:tasks.rados.rados.0.smithi100.stdout:99: left oid 12 (ObjNum 58 snap 4 seq_num 58) 2024-04-07T16:16:56.605 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:56 smithi144 bash[23522]: cluster 2024-04-07T16:16:54.694353+0000 mgr.y (mgr.14553) 181 : cluster 0 pgmap v138: 121 pgs: 121 active+clean; 24 MiB data, 266 MiB used, 715 GiB / 715 GiB avail; 1.7 MiB/s rd, 6.1 MiB/s wr, 74 op/s 2024-04-07T16:16:56.605 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:56 smithi144 bash[23522]: cluster 2024-04-07T16:16:55.249613+0000 mon.a (mon.0) 898 : cluster 0 osdmap e80: 8 total, 8 up, 8 in 2024-04-07T16:16:56.687 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:56 smithi100 bash[31660]: cluster 2024-04-07T16:16:54.694353+0000 mgr.y (mgr.14553) 181 : cluster 0 pgmap v138: 121 pgs: 121 active+clean; 24 MiB data, 266 MiB used, 715 GiB / 715 GiB avail; 1.7 MiB/s rd, 6.1 MiB/s wr, 74 op/s 2024-04-07T16:16:56.687 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:56 smithi100 bash[31660]: cluster 2024-04-07T16:16:55.249613+0000 mon.a (mon.0) 898 : cluster 0 osdmap e80: 8 total, 8 up, 8 in 2024-04-07T16:16:56.687 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:56 smithi100 bash[21346]: cluster 2024-04-07T16:16:54.694353+0000 mgr.y (mgr.14553) 181 : cluster 0 pgmap v138: 121 pgs: 121 active+clean; 24 MiB data, 266 MiB used, 715 GiB / 715 GiB avail; 1.7 MiB/s rd, 6.1 MiB/s wr, 74 op/s 2024-04-07T16:16:56.687 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:56 smithi100 bash[21346]: cluster 2024-04-07T16:16:55.249613+0000 mon.a (mon.0) 898 : cluster 0 osdmap e80: 8 total, 8 up, 8 in 2024-04-07T16:16:57.266 INFO:tasks.rados.rados.0.smithi100.stdout:99: done (2 left) 2024-04-07T16:16:57.266 INFO:tasks.rados.rados.0.smithi100.stdout:100: done (1 left) 2024-04-07T16:16:57.266 INFO:tasks.rados.rados.0.smithi100.stdout:101: done (0 left) 2024-04-07T16:16:57.266 INFO:tasks.rados.rados.0.smithi100.stdout:102: snap_remove snap 1 2024-04-07T16:16:57.605 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:57 smithi144 bash[23522]: cluster 2024-04-07T16:16:56.259503+0000 mon.a (mon.0) 899 : cluster 0 osdmap e81: 8 total, 8 up, 8 in 2024-04-07T16:16:57.687 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:57 smithi100 bash[31660]: cluster 2024-04-07T16:16:56.259503+0000 mon.a (mon.0) 899 : cluster 0 osdmap e81: 8 total, 8 up, 8 in 2024-04-07T16:16:57.687 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:57 smithi100 bash[21346]: cluster 2024-04-07T16:16:56.259503+0000 mon.a (mon.0) 899 : cluster 0 osdmap e81: 8 total, 8 up, 8 in 2024-04-07T16:16:58.276 INFO:tasks.rados.rados.0.smithi100.stdout:102: done (0 left) 2024-04-07T16:16:58.276 INFO:tasks.rados.rados.0.smithi100.stdout:103: read oid 29 snap 4 2024-04-07T16:16:58.276 INFO:tasks.rados.rados.0.smithi100.stdout:103: expect (ObjNum 28 snap 0 seq_num 28) 2024-04-07T16:16:58.276 INFO:tasks.rados.rados.0.smithi100.stdout:104: read oid 46 snap -1 2024-04-07T16:16:58.276 INFO:tasks.rados.rados.0.smithi100.stdout:104: expect (ObjNum 45 snap 0 seq_num 45) 2024-04-07T16:16:58.276 INFO:tasks.rados.rados.0.smithi100.stdout:105: copy_from oid 36 from oid 2 current snap is 5 2024-04-07T16:16:58.276 INFO:tasks.rados.rados.0.smithi100.stdout:106: rmattr oid 49 current snap is 5 2024-04-07T16:16:58.277 INFO:tasks.rados.rados.0.smithi100.stdout:106: done (3 left) 2024-04-07T16:16:58.277 INFO:tasks.rados.rados.0.smithi100.stdout:107: snap_remove snap 3 2024-04-07T16:16:58.605 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:58 smithi144 bash[23522]: cluster 2024-04-07T16:16:56.694778+0000 mgr.y (mgr.14553) 182 : cluster 0 pgmap v141: 121 pgs: 121 active+clean; 24 MiB data, 266 MiB used, 715 GiB / 715 GiB avail; 1.7 MiB/s rd, 6.1 MiB/s wr, 74 op/s 2024-04-07T16:16:58.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:58 smithi144 bash[23522]: cluster 2024-04-07T16:16:57.281622+0000 mon.a (mon.0) 900 : cluster 0 osdmap e82: 8 total, 8 up, 8 in 2024-04-07T16:16:58.687 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:58 smithi100 bash[21346]: cluster 2024-04-07T16:16:56.694778+0000 mgr.y (mgr.14553) 182 : cluster 0 pgmap v141: 121 pgs: 121 active+clean; 24 MiB data, 266 MiB used, 715 GiB / 715 GiB avail; 1.7 MiB/s rd, 6.1 MiB/s wr, 74 op/s 2024-04-07T16:16:58.687 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:58 smithi100 bash[21346]: cluster 2024-04-07T16:16:57.281622+0000 mon.a (mon.0) 900 : cluster 0 osdmap e82: 8 total, 8 up, 8 in 2024-04-07T16:16:58.687 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:58 smithi100 bash[31660]: cluster 2024-04-07T16:16:56.694778+0000 mgr.y (mgr.14553) 182 : cluster 0 pgmap v141: 121 pgs: 121 active+clean; 24 MiB data, 266 MiB used, 715 GiB / 715 GiB avail; 1.7 MiB/s rd, 6.1 MiB/s wr, 74 op/s 2024-04-07T16:16:58.687 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:58 smithi100 bash[31660]: cluster 2024-04-07T16:16:57.281622+0000 mon.a (mon.0) 900 : cluster 0 osdmap e82: 8 total, 8 up, 8 in 2024-04-07T16:16:59.287 INFO:tasks.rados.rados.0.smithi100.stdout:107: done (3 left) 2024-04-07T16:16:59.287 INFO:tasks.rados.rados.0.smithi100.stdout:108: rmattr oid 30 current snap is 5 2024-04-07T16:16:59.287 INFO:tasks.rados.rados.0.smithi100.stdout:108: done (3 left) 2024-04-07T16:16:59.287 INFO:tasks.rados.rados.0.smithi100.stdout:109: read oid 42 snap 2 2024-04-07T16:16:59.287 INFO:tasks.rados.rados.0.smithi100.stdout:109: expect (ObjNum 41 snap 0 seq_num 41) 2024-04-07T16:16:59.287 INFO:tasks.rados.rados.0.smithi100.stdout:110: rmattr oid 14 current snap is 5 2024-04-07T16:16:59.287 INFO:tasks.rados.rados.0.smithi100.stdout:111: read oid 32 snap 2 2024-04-07T16:16:59.287 INFO:tasks.rados.rados.0.smithi100.stdout:111: expect (ObjNum 31 snap 0 seq_num 31) 2024-04-07T16:16:59.288 INFO:tasks.rados.rados.0.smithi100.stdout:112: delete oid 28 current snap is 5 2024-04-07T16:16:59.288 INFO:tasks.rados.rados.0.smithi100.stdout:103: expect (ObjNum 28 snap 0 seq_num 28) 2024-04-07T16:16:59.302 INFO:tasks.rados.rados.0.smithi100.stdout:105: finishing copy_from racing read to smithi10066867-36 2024-04-07T16:16:59.302 INFO:tasks.rados.rados.0.smithi100.stdout:105: finishing copy_from to smithi10066867-36 2024-04-07T16:16:59.302 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 36 v 4 (ObjNum 10 snap 0 seq_num 10) dirty exists 2024-04-07T16:16:59.302 INFO:tasks.rados.rados.0.smithi100.stdout:104: expect (ObjNum 45 snap 0 seq_num 45) 2024-04-07T16:16:59.324 INFO:tasks.rados.rados.0.smithi100.stdout:109: expect (ObjNum 41 snap 0 seq_num 41) 2024-04-07T16:16:59.347 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 14 v 13 (ObjNum 13 snap 0 seq_num 13) dirty exists 2024-04-07T16:16:59.347 INFO:tasks.rados.rados.0.smithi100.stdout:103: done (6 left) 2024-04-07T16:16:59.347 INFO:tasks.rados.rados.0.smithi100.stdout:104: done (5 left) 2024-04-07T16:16:59.347 INFO:tasks.rados.rados.0.smithi100.stdout:105: done (4 left) 2024-04-07T16:16:59.348 INFO:tasks.rados.rados.0.smithi100.stdout:109: done (3 left) 2024-04-07T16:16:59.348 INFO:tasks.rados.rados.0.smithi100.stdout:110: done (2 left) 2024-04-07T16:16:59.348 INFO:tasks.rados.rados.0.smithi100.stdout:112: done (1 left) 2024-04-07T16:16:59.348 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 33 current snap is 5 2024-04-07T16:16:59.348 INFO:tasks.rados.rados.0.smithi100.stdout:111: expect (ObjNum 31 snap 0 seq_num 31) 2024-04-07T16:16:59.364 INFO:tasks.rados.rados.0.smithi100.stdout:113: seq_num 60 ranges {540672=614400,1155072=401408} 2024-04-07T16:16:59.371 INFO:tasks.rados.rados.0.smithi100.stdout:113: writing smithi10066867-33 from 540672 to 1155072 tid 1 2024-04-07T16:16:59.374 INFO:tasks.rados.rados.0.smithi100.stdout:113: writing smithi10066867-33 from 1155072 to 1556480 tid 2 2024-04-07T16:16:59.374 INFO:tasks.rados.rados.0.smithi100.stdout:111: done (1 left) 2024-04-07T16:16:59.374 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 29 current snap is 5 2024-04-07T16:16:59.374 INFO:tasks.rados.rados.0.smithi100.stdout:114: seq_num 61 ranges {475136=630784,1105920=401408} 2024-04-07T16:16:59.380 INFO:tasks.rados.rados.0.smithi100.stdout:114: writing smithi10066867-29 from 475136 to 1105920 tid 1 2024-04-07T16:16:59.383 INFO:tasks.rados.rados.0.smithi100.stdout:114: writing smithi10066867-29 from 1105920 to 1507328 tid 2 2024-04-07T16:16:59.383 INFO:tasks.rados.rados.0.smithi100.stdout:115: snap_remove snap 2 2024-04-07T16:16:59.605 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:16:59 smithi144 bash[23522]: cluster 2024-04-07T16:16:58.290854+0000 mon.a (mon.0) 901 : cluster 0 osdmap e83: 8 total, 8 up, 8 in 2024-04-07T16:16:59.687 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:16:59 smithi100 bash[21346]: cluster 2024-04-07T16:16:58.290854+0000 mon.a (mon.0) 901 : cluster 0 osdmap e83: 8 total, 8 up, 8 in 2024-04-07T16:16:59.687 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:16:59 smithi100 bash[31660]: cluster 2024-04-07T16:16:58.290854+0000 mon.a (mon.0) 901 : cluster 0 osdmap e83: 8 total, 8 up, 8 in 2024-04-07T16:17:00.295 INFO:tasks.rados.rados.0.smithi100.stdout:115: done (2 left) 2024-04-07T16:17:00.295 INFO:tasks.rados.rados.0.smithi100.stdout:116: snap_create 2024-04-07T16:17:00.296 INFO:tasks.rados.rados.0.smithi100.stdout:113: finishing write tid 1 to smithi10066867-33 2024-04-07T16:17:00.296 INFO:tasks.rados.rados.0.smithi100.stdout:113: finishing write tid 2 to smithi10066867-33 2024-04-07T16:17:00.296 INFO:tasks.rados.rados.0.smithi100.stdout:113: finishing write tid 3 to smithi10066867-33 2024-04-07T16:17:00.296 INFO:tasks.rados.rados.0.smithi100.stdout:113: finishing write tid 4 to smithi10066867-33 2024-04-07T16:17:00.296 INFO:tasks.rados.rados.0.smithi100.stdout:113: oid 33 updating version 0 to 26 2024-04-07T16:17:00.296 INFO:tasks.rados.rados.0.smithi100.stdout:113: oid 33 version 26 is already newer than 24 2024-04-07T16:17:00.296 INFO:tasks.rados.rados.0.smithi100.stdout:113: oid 33 version 26 is already newer than 25 2024-04-07T16:17:00.296 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 33 v 26 (ObjNum 60 snap 5 seq_num 60) dirty exists 2024-04-07T16:17:00.296 INFO:tasks.rados.rados.0.smithi100.stdout:113: left oid 33 (ObjNum 60 snap 5 seq_num 60) 2024-04-07T16:17:00.296 INFO:tasks.rados.rados.0.smithi100.stdout:114: finishing write tid 1 to smithi10066867-29 2024-04-07T16:17:00.296 INFO:tasks.rados.rados.0.smithi100.stdout:114: finishing write tid 2 to smithi10066867-29 2024-04-07T16:17:00.296 INFO:tasks.rados.rados.0.smithi100.stdout:114: finishing write tid 3 to smithi10066867-29 2024-04-07T16:17:00.296 INFO:tasks.rados.rados.0.smithi100.stdout:114: finishing write tid 4 to smithi10066867-29 2024-04-07T16:17:00.296 INFO:tasks.rados.rados.0.smithi100.stdout:114: oid 29 updating version 0 to 31 2024-04-07T16:17:00.296 INFO:tasks.rados.rados.0.smithi100.stdout:114: oid 29 version 31 is already newer than 30 2024-04-07T16:17:00.297 INFO:tasks.rados.rados.0.smithi100.stdout:114: oid 29 updating version 31 to 32 2024-04-07T16:17:00.297 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 29 v 32 (ObjNum 61 snap 5 seq_num 61) dirty exists 2024-04-07T16:17:00.297 INFO:tasks.rados.rados.0.smithi100.stdout:114: left oid 29 (ObjNum 61 snap 5 seq_num 61) 2024-04-07T16:17:00.605 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:17:00 smithi144 bash[23522]: cluster 2024-04-07T16:16:58.695123+0000 mgr.y (mgr.14553) 183 : cluster 0 pgmap v144: 121 pgs: 121 active+clean; 24 MiB data, 266 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:17:00.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:17:00 smithi144 bash[23522]: cluster 2024-04-07T16:16:59.298319+0000 mon.a (mon.0) 902 : cluster 0 osdmap e84: 8 total, 8 up, 8 in 2024-04-07T16:17:00.687 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:17:00 smithi100 bash[21346]: cluster 2024-04-07T16:16:58.695123+0000 mgr.y (mgr.14553) 183 : cluster 0 pgmap v144: 121 pgs: 121 active+clean; 24 MiB data, 266 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:17:00.687 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:17:00 smithi100 bash[21346]: cluster 2024-04-07T16:16:59.298319+0000 mon.a (mon.0) 902 : cluster 0 osdmap e84: 8 total, 8 up, 8 in 2024-04-07T16:17:00.687 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:17:00 smithi100 bash[31660]: cluster 2024-04-07T16:16:58.695123+0000 mgr.y (mgr.14553) 183 : cluster 0 pgmap v144: 121 pgs: 121 active+clean; 24 MiB data, 266 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:17:00.687 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:17:00 smithi100 bash[31660]: cluster 2024-04-07T16:16:59.298319+0000 mon.a (mon.0) 902 : cluster 0 osdmap e84: 8 total, 8 up, 8 in 2024-04-07T16:17:01.298 INFO:tasks.rados.rados.0.smithi100.stdout:113: done (2 left) 2024-04-07T16:17:01.299 INFO:tasks.rados.rados.0.smithi100.stdout:114: done (1 left) 2024-04-07T16:17:01.299 INFO:tasks.rados.rados.0.smithi100.stdout:116: done (0 left) 2024-04-07T16:17:01.299 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 9 current snap is 6 2024-04-07T16:17:01.299 INFO:tasks.rados.rados.0.smithi100.stdout:117: seq_num 62 ranges {139264=647168,786432=401408} 2024-04-07T16:17:01.303 INFO:tasks.rados.rados.0.smithi100.stdout:117: writing smithi10066867-9 from 139264 to 786432 tid 1 2024-04-07T16:17:01.306 INFO:tasks.rados.rados.0.smithi100.stdout:117: writing smithi10066867-9 from 786432 to 1187840 tid 2 2024-04-07T16:17:01.306 INFO:tasks.rados.rados.0.smithi100.stdout:118: rollback oid 3 current snap is 6 2024-04-07T16:17:01.306 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 3 to 4 2024-04-07T16:17:01.306 INFO:tasks.rados.rados.0.smithi100.stdout:119: read oid 25 snap -1 2024-04-07T16:17:01.306 INFO:tasks.rados.rados.0.smithi100.stdout:119: expect (ObjNum 24 snap 0 seq_num 24) 2024-04-07T16:17:01.306 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 2 current snap is 6 2024-04-07T16:17:01.306 INFO:tasks.rados.rados.0.smithi100.stdout:120: seq_num 63 ranges {172032=663552,835584=401408} 2024-04-07T16:17:01.311 INFO:tasks.rados.rados.0.smithi100.stdout:120: writing smithi10066867-2 from 172032 to 835584 tid 1 2024-04-07T16:17:01.314 INFO:tasks.rados.rados.0.smithi100.stdout:120: writing smithi10066867-2 from 835584 to 1236992 tid 2 2024-04-07T16:17:01.314 INFO:tasks.rados.rados.0.smithi100.stdout:121: snap_create 2024-04-07T16:17:01.314 INFO:tasks.rados.rados.0.smithi100.stdout:119: expect (ObjNum 24 snap 0 seq_num 24) 2024-04-07T16:17:01.329 INFO:tasks.rados.rados.0.smithi100.stdout:117: finishing write tid 1 to smithi10066867-9 2024-04-07T16:17:01.329 INFO:tasks.rados.rados.0.smithi100.stdout:120: finishing write tid 1 to smithi10066867-2 2024-04-07T16:17:01.329 INFO:tasks.rados.rados.0.smithi100.stdout:117: finishing write tid 2 to smithi10066867-9 2024-04-07T16:17:01.329 INFO:tasks.rados.rados.0.smithi100.stdout:118: finishing rollback tid 0 to smithi10066867-3 2024-04-07T16:17:01.329 INFO:tasks.rados.rados.0.smithi100.stdout:118: finishing rollback tid 1 to smithi10066867-3 2024-04-07T16:17:01.329 INFO:tasks.rados.rados.0.smithi100.stdout:118: finishing rollback tid 2 to smithi10066867-3 2024-04-07T16:17:01.329 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 3 v 21 (ObjNum 54 snap 3 seq_num 54) dirty exists 2024-04-07T16:17:01.329 INFO:tasks.rados.rados.0.smithi100.stdout:120: finishing write tid 2 to smithi10066867-2 2024-04-07T16:17:01.329 INFO:tasks.rados.rados.0.smithi100.stdout:117: finishing write tid 3 to smithi10066867-9 2024-04-07T16:17:01.329 INFO:tasks.rados.rados.0.smithi100.stdout:120: finishing write tid 3 to smithi10066867-2 2024-04-07T16:17:01.329 INFO:tasks.rados.rados.0.smithi100.stdout:120: finishing write tid 4 to smithi10066867-2 2024-04-07T16:17:01.329 INFO:tasks.rados.rados.0.smithi100.stdout:120: oid 2 updating version 0 to 25 2024-04-07T16:17:01.329 INFO:tasks.rados.rados.0.smithi100.stdout:120: oid 2 updating version 25 to 27 2024-04-07T16:17:01.330 INFO:tasks.rados.rados.0.smithi100.stdout:120: oid 2 version 27 is already newer than 26 2024-04-07T16:17:01.330 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 2 v 27 (ObjNum 63 snap 6 seq_num 63) dirty exists 2024-04-07T16:17:01.330 INFO:tasks.rados.rados.0.smithi100.stdout:120: left oid 2 (ObjNum 63 snap 6 seq_num 63) 2024-04-07T16:17:01.330 INFO:tasks.rados.rados.0.smithi100.stdout:117: finishing write tid 4 to smithi10066867-9 2024-04-07T16:17:01.330 INFO:tasks.rados.rados.0.smithi100.stdout:117: oid 9 updating version 0 to 16 2024-04-07T16:17:01.330 INFO:tasks.rados.rados.0.smithi100.stdout:117: oid 9 version 16 is already newer than 15 2024-04-07T16:17:01.330 INFO:tasks.rados.rados.0.smithi100.stdout:117: oid 9 version 16 is already newer than 14 2024-04-07T16:17:01.330 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 9 v 16 (ObjNum 62 snap 6 seq_num 62) dirty exists 2024-04-07T16:17:01.331 INFO:tasks.rados.rados.0.smithi100.stdout:117: left oid 9 (ObjNum 62 snap 6 seq_num 62) 2024-04-07T16:17:01.605 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:17:01 smithi144 bash[23522]: cluster 2024-04-07T16:17:00.310605+0000 mon.a (mon.0) 903 : cluster 0 osdmap e85: 8 total, 8 up, 8 in 2024-04-07T16:17:01.687 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:17:01 smithi100 bash[21346]: cluster 2024-04-07T16:17:00.310605+0000 mon.a (mon.0) 903 : cluster 0 osdmap e85: 8 total, 8 up, 8 in 2024-04-07T16:17:01.687 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:17:01 smithi100 bash[31660]: cluster 2024-04-07T16:17:00.310605+0000 mon.a (mon.0) 903 : cluster 0 osdmap e85: 8 total, 8 up, 8 in 2024-04-07T16:17:02.302 INFO:tasks.rados.rados.0.smithi100.stdout:117: done (4 left) 2024-04-07T16:17:02.302 INFO:tasks.rados.rados.0.smithi100.stdout:118: done (3 left) 2024-04-07T16:17:02.302 INFO:tasks.rados.rados.0.smithi100.stdout:119: done (2 left) 2024-04-07T16:17:02.302 INFO:tasks.rados.rados.0.smithi100.stdout:120: done (1 left) 2024-04-07T16:17:02.302 INFO:tasks.rados.rados.0.smithi100.stdout:121: done (0 left) 2024-04-07T16:17:02.302 INFO:tasks.rados.rados.0.smithi100.stdout:122: read oid 48 snap 5 2024-04-07T16:17:02.302 INFO:tasks.rados.rados.0.smithi100.stdout:122: expect (ObjNum 47 snap 0 seq_num 47) 2024-04-07T16:17:02.302 INFO:tasks.rados.rados.0.smithi100.stdout:123: read oid 49 snap 6 2024-04-07T16:17:02.302 INFO:tasks.rados.rados.0.smithi100.stdout:123: expect (ObjNum 48 snap 0 seq_num 48) 2024-04-07T16:17:02.302 INFO:tasks.rados.rados.0.smithi100.stdout:124: delete oid 44 current snap is 7 2024-04-07T16:17:02.310 INFO:tasks.rados.rados.0.smithi100.stdout:123: expect (ObjNum 48 snap 0 seq_num 48) 2024-04-07T16:17:02.335 INFO:tasks.rados.rados.0.smithi100.stdout:122: expect (ObjNum 47 snap 0 seq_num 47) 2024-04-07T16:17:02.361 INFO:tasks.rados.rados.0.smithi100.stdout:122: done (2 left) 2024-04-07T16:17:02.361 INFO:tasks.rados.rados.0.smithi100.stdout:123: done (1 left) 2024-04-07T16:17:02.361 INFO:tasks.rados.rados.0.smithi100.stdout:124: done (0 left) 2024-04-07T16:17:02.361 INFO:tasks.rados.rados.0.smithi100.stdout:125: rollback oid 49 current snap is 7 2024-04-07T16:17:02.361 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 49 to 5 2024-04-07T16:17:02.361 INFO:tasks.rados.rados.0.smithi100.stdout:126: read oid 27 snap -1 2024-04-07T16:17:02.361 INFO:tasks.rados.rados.0.smithi100.stdout:126: expect (ObjNum 56 snap 4 seq_num 56) 2024-04-07T16:17:02.361 INFO:tasks.rados.rados.0.smithi100.stdout:127: setattr oid 41 current snap is 7 2024-04-07T16:17:02.362 INFO:tasks.rados.rados.0.smithi100.stdout:128: delete oid 28 current snap is 7 2024-04-07T16:17:02.364 INFO:tasks.rados.rados.0.smithi100.stdout:125: finishing rollback tid 0 to smithi10066867-49 2024-04-07T16:17:02.364 INFO:tasks.rados.rados.0.smithi100.stdout:125: finishing rollback tid 1 to smithi10066867-49 2024-04-07T16:17:02.364 INFO:tasks.rados.rados.0.smithi100.stdout:125: finishing rollback tid 2 to smithi10066867-49 2024-04-07T16:17:02.364 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 49 v 30 (ObjNum 48 snap 0 seq_num 48) dirty exists 2024-04-07T16:17:02.365 INFO:tasks.rados.rados.0.smithi100.stdout:125: done (3 left) 2024-04-07T16:17:02.365 INFO:tasks.rados.rados.0.smithi100.stdout:128: done (2 left) 2024-04-07T16:17:02.365 INFO:tasks.rados.rados.0.smithi100.stdout:129: snap_remove snap 5 2024-04-07T16:17:02.605 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:17:02 smithi144 bash[23522]: cluster 2024-04-07T16:17:00.695477+0000 mgr.y (mgr.14553) 184 : cluster 0 pgmap v147: 121 pgs: 121 active+clean; 27 MiB data, 271 MiB used, 715 GiB / 715 GiB avail; 2.2 MiB/s rd, 1.2 MiB/s wr, 14 op/s 2024-04-07T16:17:02.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:17:02 smithi144 bash[23522]: cluster 2024-04-07T16:17:01.313190+0000 mon.a (mon.0) 904 : cluster 0 osdmap e86: 8 total, 8 up, 8 in 2024-04-07T16:17:02.687 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:17:02 smithi100 bash[21346]: cluster 2024-04-07T16:17:00.695477+0000 mgr.y (mgr.14553) 184 : cluster 0 pgmap v147: 121 pgs: 121 active+clean; 27 MiB data, 271 MiB used, 715 GiB / 715 GiB avail; 2.2 MiB/s rd, 1.2 MiB/s wr, 14 op/s 2024-04-07T16:17:02.687 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:17:02 smithi100 bash[21346]: cluster 2024-04-07T16:17:01.313190+0000 mon.a (mon.0) 904 : cluster 0 osdmap e86: 8 total, 8 up, 8 in 2024-04-07T16:17:02.687 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:17:02 smithi100 bash[31660]: cluster 2024-04-07T16:17:00.695477+0000 mgr.y (mgr.14553) 184 : cluster 0 pgmap v147: 121 pgs: 121 active+clean; 27 MiB data, 271 MiB used, 715 GiB / 715 GiB avail; 2.2 MiB/s rd, 1.2 MiB/s wr, 14 op/s 2024-04-07T16:17:02.687 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:17:02 smithi100 bash[31660]: cluster 2024-04-07T16:17:01.313190+0000 mon.a (mon.0) 904 : cluster 0 osdmap e86: 8 total, 8 up, 8 in 2024-04-07T16:17:02.811 INFO:tasks.rados.rados.0.smithi100.stdout:129: done (2 left) 2024-04-07T16:17:02.811 INFO:tasks.rados.rados.0.smithi100.stdout:130: setattr oid 32 current snap is 7 2024-04-07T16:17:02.811 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 41 v 32 (ObjNum 40 snap 0 seq_num 40) dirty exists 2024-04-07T16:17:02.812 INFO:tasks.rados.rados.0.smithi100.stdout:126: expect (ObjNum 56 snap 4 seq_num 56) 2024-04-07T16:17:02.842 INFO:tasks.rados.rados.0.smithi100.stdout:126: done (2 left) 2024-04-07T16:17:02.842 INFO:tasks.rados.rados.0.smithi100.stdout:127: done (1 left) 2024-04-07T16:17:02.842 INFO:tasks.rados.rados.0.smithi100.stdout:131: snap_remove snap 4 2024-04-07T16:17:03.605 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:17:03 smithi144 bash[23522]: cluster 2024-04-07T16:17:02.313618+0000 mon.a (mon.0) 905 : cluster 0 osdmap e87: 8 total, 8 up, 8 in 2024-04-07T16:17:03.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:17:03 smithi144 bash[23522]: cluster 2024-04-07T16:17:02.826570+0000 mon.a (mon.0) 906 : cluster 0 osdmap e88: 8 total, 8 up, 8 in 2024-04-07T16:17:03.687 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:17:03 smithi100 bash[21346]: cluster 2024-04-07T16:17:02.313618+0000 mon.a (mon.0) 905 : cluster 0 osdmap e87: 8 total, 8 up, 8 in 2024-04-07T16:17:03.687 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:17:03 smithi100 bash[21346]: cluster 2024-04-07T16:17:02.826570+0000 mon.a (mon.0) 906 : cluster 0 osdmap e88: 8 total, 8 up, 8 in 2024-04-07T16:17:03.687 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:17:03 smithi100 bash[31660]: cluster 2024-04-07T16:17:02.313618+0000 mon.a (mon.0) 905 : cluster 0 osdmap e87: 8 total, 8 up, 8 in 2024-04-07T16:17:03.687 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:17:03 smithi100 bash[31660]: cluster 2024-04-07T16:17:02.826570+0000 mon.a (mon.0) 906 : cluster 0 osdmap e88: 8 total, 8 up, 8 in 2024-04-07T16:17:03.808 INFO:tasks.rados.rados.0.smithi100.stdout:131: done (1 left) 2024-04-07T16:17:03.808 INFO:tasks.rados.rados.0.smithi100.stdout:132: delete oid 6 current snap is 7 2024-04-07T16:17:03.808 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 32 v 20 (ObjNum 31 snap 0 seq_num 31) dirty exists 2024-04-07T16:17:03.820 INFO:tasks.rados.rados.0.smithi100.stdout:130: done (1 left) 2024-04-07T16:17:03.820 INFO:tasks.rados.rados.0.smithi100.stdout:132: done (0 left) 2024-04-07T16:17:03.820 INFO:tasks.rados.rados.0.smithi100.stdout:133: snap_remove snap 6 2024-04-07T16:17:04.605 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:17:04 smithi144 bash[23522]: cluster 2024-04-07T16:17:02.695866+0000 mgr.y (mgr.14553) 185 : cluster 0 pgmap v150: 121 pgs: 121 active+clean; 27 MiB data, 271 MiB used, 715 GiB / 715 GiB avail; 2.2 MiB/s rd, 1.2 MiB/s wr, 14 op/s 2024-04-07T16:17:04.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:17:04 smithi144 bash[23522]: cluster 2024-04-07T16:17:03.822988+0000 mon.a (mon.0) 907 : cluster 0 osdmap e89: 8 total, 8 up, 8 in 2024-04-07T16:17:04.687 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:17:04 smithi100 bash[21346]: cluster 2024-04-07T16:17:02.695866+0000 mgr.y (mgr.14553) 185 : cluster 0 pgmap v150: 121 pgs: 121 active+clean; 27 MiB data, 271 MiB used, 715 GiB / 715 GiB avail; 2.2 MiB/s rd, 1.2 MiB/s wr, 14 op/s 2024-04-07T16:17:04.687 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:17:04 smithi100 bash[21346]: cluster 2024-04-07T16:17:03.822988+0000 mon.a (mon.0) 907 : cluster 0 osdmap e89: 8 total, 8 up, 8 in 2024-04-07T16:17:04.687 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:17:04 smithi100 bash[31660]: cluster 2024-04-07T16:17:02.695866+0000 mgr.y (mgr.14553) 185 : cluster 0 pgmap v150: 121 pgs: 121 active+clean; 27 MiB data, 271 MiB used, 715 GiB / 715 GiB avail; 2.2 MiB/s rd, 1.2 MiB/s wr, 14 op/s 2024-04-07T16:17:04.687 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:17:04 smithi100 bash[31660]: cluster 2024-04-07T16:17:03.822988+0000 mon.a (mon.0) 907 : cluster 0 osdmap e89: 8 total, 8 up, 8 in 2024-04-07T16:17:04.815 INFO:tasks.rados.rados.0.smithi100.stdout:133: done (0 left) 2024-04-07T16:17:04.815 INFO:tasks.rados.rados.0.smithi100.stdout:134: rollback oid 17 current snap is 7 2024-04-07T16:17:04.815 INFO:tasks.rados.rados.0.smithi100.stdout:134: done (0 left) 2024-04-07T16:17:04.815 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 43 current snap is 7 2024-04-07T16:17:04.815 INFO:tasks.rados.rados.0.smithi100.stdout:135: seq_num 67 ranges {712704=729088,1441792=401408} 2024-04-07T16:17:04.823 INFO:tasks.rados.rados.0.smithi100.stdout:135: writing smithi10066867-43 from 712704 to 1441792 tid 1 2024-04-07T16:17:04.826 INFO:tasks.rados.rados.0.smithi100.stdout:135: writing smithi10066867-43 from 1441792 to 1843200 tid 2 2024-04-07T16:17:04.826 INFO:tasks.rados.rados.0.smithi100.stdout:136: read oid 11 snap -1 2024-04-07T16:17:04.826 INFO:tasks.rados.rados.0.smithi100.stdout:136: expect (ObjNum 10 snap 0 seq_num 10) 2024-04-07T16:17:04.826 INFO:tasks.rados.rados.0.smithi100.stdout:137: read oid 6 snap -1 2024-04-07T16:17:04.826 INFO:tasks.rados.rados.0.smithi100.stdout:137: expect deleted 2024-04-07T16:17:04.826 INFO:tasks.rados.rados.0.smithi100.stdout:138: copy_from oid 18 from oid 46 current snap is 7 2024-04-07T16:17:04.826 INFO:tasks.rados.rados.0.smithi100.stdout:139: read oid 41 snap -1 2024-04-07T16:17:04.827 INFO:tasks.rados.rados.0.smithi100.stdout:139: expect (ObjNum 40 snap 0 seq_num 40) 2024-04-07T16:17:04.827 INFO:tasks.rados.rados.0.smithi100.stdout:140: rmattr oid 37 current snap is 7 2024-04-07T16:17:04.827 INFO:tasks.rados.rados.0.smithi100.stdout:140: done (5 left) 2024-04-07T16:17:04.827 INFO:tasks.rados.rados.0.smithi100.stdout:141: read oid 1 snap -1 2024-04-07T16:17:04.827 INFO:tasks.rados.rados.0.smithi100.stdout:141: expect deleted 2024-04-07T16:17:04.827 INFO:tasks.rados.rados.0.smithi100.stdout:142: rmattr oid 30 current snap is 7 2024-04-07T16:17:04.827 INFO:tasks.rados.rados.0.smithi100.stdout:142: done (6 left) 2024-04-07T16:17:04.827 INFO:tasks.rados.rados.0.smithi100.stdout:143: setattr oid 47 current snap is 7 2024-04-07T16:17:04.828 INFO:tasks.rados.rados.0.smithi100.stdout:144: copy_from oid 35 from oid 5 current snap is 7 2024-04-07T16:17:04.828 INFO:tasks.rados.rados.0.smithi100.stdout:145: setattr oid 9 current snap is 7 2024-04-07T16:17:04.830 INFO:tasks.rados.rados.0.smithi100.stdout:137: done (8 left) 2024-04-07T16:17:04.830 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 3 current snap is 7 2024-04-07T16:17:04.830 INFO:tasks.rados.rados.0.smithi100.stdout:146: seq_num 68 ranges {950272=745472,1695744=401408} 2024-04-07T16:17:04.841 INFO:tasks.rados.rados.0.smithi100.stdout:146: writing smithi10066867-3 from 950272 to 1695744 tid 1 2024-04-07T16:17:04.844 INFO:tasks.rados.rados.0.smithi100.stdout:146: writing smithi10066867-3 from 1695744 to 2097152 tid 2 2024-04-07T16:17:04.844 INFO:tasks.rados.rados.0.smithi100.stdout:147: setattr oid 8 current snap is 7 2024-04-07T16:17:04.844 INFO:tasks.rados.rados.0.smithi100.stdout:135: finishing write tid 1 to smithi10066867-43 2024-04-07T16:17:04.844 INFO:tasks.rados.rados.0.smithi100.stdout:135: finishing write tid 2 to smithi10066867-43 2024-04-07T16:17:04.844 INFO:tasks.rados.rados.0.smithi100.stdout:135: finishing write tid 3 to smithi10066867-43 2024-04-07T16:17:04.844 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 47 v 5 (ObjNum 46 snap 0 seq_num 46) dirty exists 2024-04-07T16:17:04.844 INFO:tasks.rados.rados.0.smithi100.stdout:135: finishing write tid 4 to smithi10066867-43 2024-04-07T16:17:04.844 INFO:tasks.rados.rados.0.smithi100.stdout:135: oid 43 updating version 0 to 16 2024-04-07T16:17:04.844 INFO:tasks.rados.rados.0.smithi100.stdout:135: oid 43 updating version 16 to 17 2024-04-07T16:17:04.844 INFO:tasks.rados.rados.0.smithi100.stdout:135: oid 43 version 17 is already newer than 15 2024-04-07T16:17:04.845 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 43 v 17 (ObjNum 67 snap 7 seq_num 67) dirty exists 2024-04-07T16:17:04.845 INFO:tasks.rados.rados.0.smithi100.stdout:135: left oid 43 (ObjNum 67 snap 7 seq_num 67) 2024-04-07T16:17:04.845 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 9 v 21 (ObjNum 62 snap 6 seq_num 62) dirty exists 2024-04-07T16:17:04.845 INFO:tasks.rados.rados.0.smithi100.stdout:136: expect (ObjNum 10 snap 0 seq_num 10) 2024-04-07T16:17:04.859 INFO:tasks.rados.rados.0.smithi100.stdout:144: finishing copy_from racing read to smithi10066867-35 2024-04-07T16:17:04.859 INFO:tasks.rados.rados.0.smithi100.stdout:139: expect (ObjNum 40 snap 0 seq_num 40) 2024-04-07T16:17:04.902 INFO:tasks.rados.rados.0.smithi100.stdout:138: finishing copy_from racing read to smithi10066867-18 2024-04-07T16:17:04.902 INFO:tasks.rados.rados.0.smithi100.stdout:144: finishing copy_from to smithi10066867-35 2024-04-07T16:17:04.902 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 35 v 2 (ObjNum 4 snap 0 seq_num 4) dirty exists 2024-04-07T16:17:04.902 INFO:tasks.rados.rados.0.smithi100.stdout:146: finishing write tid 1 to smithi10066867-3 2024-04-07T16:17:04.902 INFO:tasks.rados.rados.0.smithi100.stdout:146: finishing write tid 2 to smithi10066867-3 2024-04-07T16:17:04.902 INFO:tasks.rados.rados.0.smithi100.stdout:146: finishing write tid 3 to smithi10066867-3 2024-04-07T16:17:04.902 INFO:tasks.rados.rados.0.smithi100.stdout:138: finishing copy_from to smithi10066867-18 2024-04-07T16:17:04.902 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 18 v 8 (ObjNum 45 snap 0 seq_num 45) dirty exists 2024-04-07T16:17:04.903 INFO:tasks.rados.rados.0.smithi100.stdout:146: finishing write tid 4 to smithi10066867-3 2024-04-07T16:17:04.903 INFO:tasks.rados.rados.0.smithi100.stdout:146: oid 3 updating version 0 to 28 2024-04-07T16:17:04.903 INFO:tasks.rados.rados.0.smithi100.stdout:146: oid 3 version 28 is already newer than 27 2024-04-07T16:17:04.903 INFO:tasks.rados.rados.0.smithi100.stdout:146: oid 3 version 28 is already newer than 26 2024-04-07T16:17:04.903 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 3 v 28 (ObjNum 68 snap 7 seq_num 68) dirty exists 2024-04-07T16:17:04.903 INFO:tasks.rados.rados.0.smithi100.stdout:146: left oid 3 (ObjNum 68 snap 7 seq_num 68) 2024-04-07T16:17:04.903 INFO:tasks.rados.rados.0.smithi100.stdout:135: done (9 left) 2024-04-07T16:17:04.903 INFO:tasks.rados.rados.0.smithi100.stdout:136: done (8 left) 2024-04-07T16:17:04.903 INFO:tasks.rados.rados.0.smithi100.stdout:138: done (7 left) 2024-04-07T16:17:04.903 INFO:tasks.rados.rados.0.smithi100.stdout:139: done (6 left) 2024-04-07T16:17:04.903 INFO:tasks.rados.rados.0.smithi100.stdout:141: done (5 left) 2024-04-07T16:17:04.903 INFO:tasks.rados.rados.0.smithi100.stdout:143: done (4 left) 2024-04-07T16:17:04.903 INFO:tasks.rados.rados.0.smithi100.stdout:144: done (3 left) 2024-04-07T16:17:04.903 INFO:tasks.rados.rados.0.smithi100.stdout:145: done (2 left) 2024-04-07T16:17:04.903 INFO:tasks.rados.rados.0.smithi100.stdout:146: done (1 left) 2024-04-07T16:17:04.904 INFO:tasks.rados.rados.0.smithi100.stdout:148: snap_create 2024-04-07T16:17:04.906 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 8 v 9 (ObjNum 7 snap 0 seq_num 7) dirty exists 2024-04-07T16:17:05.437 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:17:05 smithi100 bash[21601]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:17:05] "GET /metrics HTTP/1.1" 200 36020 "" "Prometheus/2.43.0" 2024-04-07T16:17:05.831 INFO:tasks.rados.rados.0.smithi100.stdout:147: done (1 left) 2024-04-07T16:17:05.831 INFO:tasks.rados.rados.0.smithi100.stdout:148: done (0 left) 2024-04-07T16:17:05.831 INFO:tasks.rados.rados.0.smithi100.stdout:149: snap_remove snap 7 2024-04-07T16:17:06.105 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:17:05 smithi144 bash[23522]: cluster 2024-04-07T16:17:04.696326+0000 mgr.y (mgr.14553) 186 : cluster 0 pgmap v153: 121 pgs: 121 active+clean; 31 MiB data, 278 MiB used, 715 GiB / 715 GiB avail; 3.0 MiB/s rd, 1016 KiB/s wr, 24 op/s 2024-04-07T16:17:06.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:17:05 smithi144 bash[23522]: cluster 2024-04-07T16:17:04.828223+0000 mon.a (mon.0) 908 : cluster 0 osdmap e90: 8 total, 8 up, 8 in 2024-04-07T16:17:06.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:17:05 smithi100 bash[21346]: cluster 2024-04-07T16:17:04.696326+0000 mgr.y (mgr.14553) 186 : cluster 0 pgmap v153: 121 pgs: 121 active+clean; 31 MiB data, 278 MiB used, 715 GiB / 715 GiB avail; 3.0 MiB/s rd, 1016 KiB/s wr, 24 op/s 2024-04-07T16:17:06.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:17:05 smithi100 bash[21346]: cluster 2024-04-07T16:17:04.828223+0000 mon.a (mon.0) 908 : cluster 0 osdmap e90: 8 total, 8 up, 8 in 2024-04-07T16:17:06.188 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:17:05 smithi100 bash[31660]: cluster 2024-04-07T16:17:04.696326+0000 mgr.y (mgr.14553) 186 : cluster 0 pgmap v153: 121 pgs: 121 active+clean; 31 MiB data, 278 MiB used, 715 GiB / 715 GiB avail; 3.0 MiB/s rd, 1016 KiB/s wr, 24 op/s 2024-04-07T16:17:06.188 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:17:05 smithi100 bash[31660]: cluster 2024-04-07T16:17:04.828223+0000 mon.a (mon.0) 908 : cluster 0 osdmap e90: 8 total, 8 up, 8 in 2024-04-07T16:17:06.841 INFO:tasks.rados.rados.0.smithi100.stdout:149: done (0 left) 2024-04-07T16:17:06.842 INFO:tasks.rados.rados.0.smithi100.stdout:150: delete oid 1 current snap is 8 2024-04-07T16:17:06.848 INFO:tasks.rados.rados.0.smithi100.stdout:150: done (0 left) 2024-04-07T16:17:06.848 INFO:tasks.rados.rados.0.smithi100.stdout:151: snap_create 2024-04-07T16:17:07.105 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:17:06 smithi144 bash[23522]: cluster 2024-04-07T16:17:05.845696+0000 mon.a (mon.0) 909 : cluster 0 osdmap e91: 8 total, 8 up, 8 in 2024-04-07T16:17:07.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:17:06 smithi100 bash[21346]: cluster 2024-04-07T16:17:05.845696+0000 mon.a (mon.0) 909 : cluster 0 osdmap e91: 8 total, 8 up, 8 in 2024-04-07T16:17:07.188 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:17:06 smithi100 bash[31660]: cluster 2024-04-07T16:17:05.845696+0000 mon.a (mon.0) 909 : cluster 0 osdmap e91: 8 total, 8 up, 8 in 2024-04-07T16:17:07.813 INFO:tasks.rados.rados.0.smithi100.stdout:151: done (0 left) 2024-04-07T16:17:07.813 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 31 current snap is 9 2024-04-07T16:17:07.814 INFO:tasks.rados.rados.0.smithi100.stdout:152: seq_num 70 ranges {507904=778240,1286144=401408} 2024-04-07T16:17:07.835 INFO:tasks.rados.rados.0.smithi100.stdout:152: writing smithi10066867-31 from 507904 to 1286144 tid 1 2024-04-07T16:17:07.844 INFO:tasks.rados.rados.0.smithi100.stdout:152: writing smithi10066867-31 from 1286144 to 1687552 tid 2 2024-04-07T16:17:07.844 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 16 current snap is 9 2024-04-07T16:17:07.844 INFO:tasks.rados.rados.0.smithi100.stdout:153: seq_num 71 ranges {253952=794624,1048576=401408} 2024-04-07T16:17:07.864 INFO:tasks.rados.rados.0.smithi100.stdout:153: writing smithi10066867-16 from 253952 to 1048576 tid 1 2024-04-07T16:17:07.872 INFO:tasks.rados.rados.0.smithi100.stdout:153: writing smithi10066867-16 from 1048576 to 1449984 tid 2 2024-04-07T16:17:07.872 INFO:tasks.rados.rados.0.smithi100.stdout:154: copy_from oid 10 from oid 39 current snap is 9 2024-04-07T16:17:07.872 INFO:tasks.rados.rados.0.smithi100.stdout:152: finishing write tid 1 to smithi10066867-31 2024-04-07T16:17:07.872 INFO:tasks.rados.rados.0.smithi100.stdout:152: finishing write tid 2 to smithi10066867-31 2024-04-07T16:17:07.872 INFO:tasks.rados.rados.0.smithi100.stdout:152: finishing write tid 3 to smithi10066867-31 2024-04-07T16:17:07.872 INFO:tasks.rados.rados.0.smithi100.stdout:152: finishing write tid 4 to smithi10066867-31 2024-04-07T16:17:07.872 INFO:tasks.rados.rados.0.smithi100.stdout:152: oid 31 updating version 0 to 11 2024-04-07T16:17:07.872 INFO:tasks.rados.rados.0.smithi100.stdout:152: oid 31 version 11 is already newer than 9 2024-04-07T16:17:07.872 INFO:tasks.rados.rados.0.smithi100.stdout:152: oid 31 version 11 is already newer than 10 2024-04-07T16:17:07.873 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 31 v 11 (ObjNum 70 snap 9 seq_num 70) dirty exists 2024-04-07T16:17:07.873 INFO:tasks.rados.rados.0.smithi100.stdout:152: left oid 31 (ObjNum 70 snap 9 seq_num 70) 2024-04-07T16:17:07.873 INFO:tasks.rados.rados.0.smithi100.stdout:153: finishing write tid 1 to smithi10066867-16 2024-04-07T16:17:07.873 INFO:tasks.rados.rados.0.smithi100.stdout:152: done (2 left) 2024-04-07T16:17:07.873 INFO:tasks.rados.rados.0.smithi100.stdout:155: read oid 25 snap 8 2024-04-07T16:17:07.873 INFO:tasks.rados.rados.0.smithi100.stdout:155: expect (ObjNum 24 snap 0 seq_num 24) 2024-04-07T16:17:07.873 INFO:tasks.rados.rados.0.smithi100.stdout:156: copy_from oid 21 from oid 35 current snap is 9 2024-04-07T16:17:07.873 INFO:tasks.rados.rados.0.smithi100.stdout:157: rollback oid 19 current snap is 9 2024-04-07T16:17:07.873 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 19 to 8 2024-04-07T16:17:07.873 INFO:tasks.rados.rados.0.smithi100.stdout:158: copy_from oid 38 from oid 24 current snap is 9 2024-04-07T16:17:07.873 INFO:tasks.rados.rados.0.smithi100.stdout:159: snap_create 2024-04-07T16:17:07.880 INFO:tasks.rados.rados.0.smithi100.stdout:153: finishing write tid 2 to smithi10066867-16 2024-04-07T16:17:07.883 INFO:tasks.rados.rados.0.smithi100.stdout:153: finishing write tid 3 to smithi10066867-16 2024-04-07T16:17:07.884 INFO:tasks.rados.rados.0.smithi100.stdout:155: expect (ObjNum 24 snap 0 seq_num 24) 2024-04-07T16:17:07.909 INFO:tasks.rados.rados.0.smithi100.stdout:157: finishing rollback tid 0 to smithi10066867-19 2024-04-07T16:17:07.909 INFO:tasks.rados.rados.0.smithi100.stdout:153: finishing write tid 4 to smithi10066867-16 2024-04-07T16:17:07.909 INFO:tasks.rados.rados.0.smithi100.stdout:153: oid 16 updating version 0 to 46 2024-04-07T16:17:07.909 INFO:tasks.rados.rados.0.smithi100.stdout:153: oid 16 updating version 46 to 47 2024-04-07T16:17:07.909 INFO:tasks.rados.rados.0.smithi100.stdout:153: oid 16 updating version 47 to 48 2024-04-07T16:17:07.909 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 16 v 48 (ObjNum 71 snap 9 seq_num 71) dirty exists 2024-04-07T16:17:07.909 INFO:tasks.rados.rados.0.smithi100.stdout:153: left oid 16 (ObjNum 71 snap 9 seq_num 71) 2024-04-07T16:17:07.909 INFO:tasks.rados.rados.0.smithi100.stdout:156: finishing copy_from racing read to smithi10066867-21 2024-04-07T16:17:07.910 INFO:tasks.rados.rados.0.smithi100.stdout:157: finishing rollback tid 1 to smithi10066867-19 2024-04-07T16:17:07.910 INFO:tasks.rados.rados.0.smithi100.stdout:156: finishing copy_from to smithi10066867-21 2024-04-07T16:17:07.910 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 21 v 2 (ObjNum 4 snap 0 seq_num 4) dirty exists 2024-04-07T16:17:07.910 INFO:tasks.rados.rados.0.smithi100.stdout:157: finishing rollback tid 2 to smithi10066867-19 2024-04-07T16:17:07.910 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 19 v 34 (ObjNum 18 snap 0 seq_num 18) dirty exists 2024-04-07T16:17:07.910 INFO:tasks.rados.rados.0.smithi100.stdout:158: finishing copy_from racing read to smithi10066867-38 2024-04-07T16:17:07.910 INFO:tasks.rados.rados.0.smithi100.stdout:158: finishing copy_from to smithi10066867-38 2024-04-07T16:17:07.910 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 38 v 2 (ObjNum 28 snap 0 seq_num 28) dirty exists 2024-04-07T16:17:07.910 INFO:tasks.rados.rados.0.smithi100.stdout:154: finishing copy_from racing read to smithi10066867-10 2024-04-07T16:17:07.910 INFO:tasks.rados.rados.0.smithi100.stdout:154: finishing copy_from to smithi10066867-10 2024-04-07T16:17:07.910 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 10 v 18 (ObjNum 57 snap 4 seq_num 57) dirty exists 2024-04-07T16:17:08.105 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:17:07 smithi144 bash[23522]: cluster 2024-04-07T16:17:06.696959+0000 mgr.y (mgr.14553) 187 : cluster 0 pgmap v156: 121 pgs: 121 active+clean; 31 MiB data, 278 MiB used, 715 GiB / 715 GiB avail; 3.0 MiB/s rd, 1016 KiB/s wr, 24 op/s 2024-04-07T16:17:08.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:17:07 smithi144 bash[23522]: cluster 2024-04-07T16:17:06.852869+0000 mon.a (mon.0) 910 : cluster 0 osdmap e92: 8 total, 8 up, 8 in 2024-04-07T16:17:08.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:17:07 smithi144 bash[23522]: audit 2024-04-07T16:17:07.703175+0000 mon.b (mon.1) 240 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:17:08.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:17:07 smithi144 bash[23522]: cluster 2024-04-07T16:17:07.827177+0000 mon.a (mon.0) 911 : cluster 0 osdmap e93: 8 total, 8 up, 8 in 2024-04-07T16:17:08.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:17:07 smithi100 bash[21346]: cluster 2024-04-07T16:17:06.696959+0000 mgr.y (mgr.14553) 187 : cluster 0 pgmap v156: 121 pgs: 121 active+clean; 31 MiB data, 278 MiB used, 715 GiB / 715 GiB avail; 3.0 MiB/s rd, 1016 KiB/s wr, 24 op/s 2024-04-07T16:17:08.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:17:07 smithi100 bash[21346]: cluster 2024-04-07T16:17:06.852869+0000 mon.a (mon.0) 910 : cluster 0 osdmap e92: 8 total, 8 up, 8 in 2024-04-07T16:17:08.188 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:17:07 smithi100 bash[21346]: audit 2024-04-07T16:17:07.703175+0000 mon.b (mon.1) 240 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:17:08.188 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:17:07 smithi100 bash[21346]: cluster 2024-04-07T16:17:07.827177+0000 mon.a (mon.0) 911 : cluster 0 osdmap e93: 8 total, 8 up, 8 in 2024-04-07T16:17:08.188 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:17:07 smithi100 bash[31660]: cluster 2024-04-07T16:17:06.696959+0000 mgr.y (mgr.14553) 187 : cluster 0 pgmap v156: 121 pgs: 121 active+clean; 31 MiB data, 278 MiB used, 715 GiB / 715 GiB avail; 3.0 MiB/s rd, 1016 KiB/s wr, 24 op/s 2024-04-07T16:17:08.188 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:17:07 smithi100 bash[31660]: cluster 2024-04-07T16:17:06.852869+0000 mon.a (mon.0) 910 : cluster 0 osdmap e92: 8 total, 8 up, 8 in 2024-04-07T16:17:08.188 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:17:07 smithi100 bash[31660]: audit 2024-04-07T16:17:07.703175+0000 mon.b (mon.1) 240 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:17:08.188 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:17:07 smithi100 bash[31660]: cluster 2024-04-07T16:17:07.827177+0000 mon.a (mon.0) 911 : cluster 0 osdmap e93: 8 total, 8 up, 8 in 2024-04-07T16:17:08.847 INFO:tasks.rados.rados.0.smithi100.stdout:153: done (6 left) 2024-04-07T16:17:08.847 INFO:tasks.rados.rados.0.smithi100.stdout:154: done (5 left) 2024-04-07T16:17:08.847 INFO:tasks.rados.rados.0.smithi100.stdout:155: done (4 left) 2024-04-07T16:17:08.847 INFO:tasks.rados.rados.0.smithi100.stdout:156: done (3 left) 2024-04-07T16:17:08.847 INFO:tasks.rados.rados.0.smithi100.stdout:157: done (2 left) 2024-04-07T16:17:08.848 INFO:tasks.rados.rados.0.smithi100.stdout:158: done (1 left) 2024-04-07T16:17:08.848 INFO:tasks.rados.rados.0.smithi100.stdout:159: done (0 left) 2024-04-07T16:17:08.848 INFO:tasks.rados.rados.0.smithi100.stdout:160: read oid 46 snap -1 2024-04-07T16:17:08.848 INFO:tasks.rados.rados.0.smithi100.stdout:160: expect (ObjNum 45 snap 0 seq_num 45) 2024-04-07T16:17:08.848 INFO:tasks.rados.rados.0.smithi100.stdout:161: read oid 7 snap 8 2024-04-07T16:17:08.848 INFO:tasks.rados.rados.0.smithi100.stdout:161: expect (ObjNum 16 snap 0 seq_num 16) 2024-04-07T16:17:08.848 INFO:tasks.rados.rados.0.smithi100.stdout:162: snap_create 2024-04-07T16:17:08.858 INFO:tasks.rados.rados.0.smithi100.stdout:161: expect (ObjNum 16 snap 0 seq_num 16) 2024-04-07T16:17:08.879 INFO:tasks.rados.rados.0.smithi100.stdout:160: expect (ObjNum 45 snap 0 seq_num 45) 2024-04-07T16:17:09.850 INFO:tasks.rados.rados.0.smithi100.stdout:160: done (2 left) 2024-04-07T16:17:09.850 INFO:tasks.rados.rados.0.smithi100.stdout:161: done (1 left) 2024-04-07T16:17:09.850 INFO:tasks.rados.rados.0.smithi100.stdout:162: done (0 left) 2024-04-07T16:17:09.850 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 13 current snap is 11 2024-04-07T16:17:09.850 INFO:tasks.rados.rados.0.smithi100.stdout:163: seq_num 72 ranges {204800=409600,614400=524288,1138688=278528} 2024-04-07T16:17:09.860 INFO:tasks.rados.rados.0.smithi100.stdout:163: writing smithi10066867-13 from 204800 to 614400 tid 1 2024-04-07T16:17:09.869 INFO:tasks.rados.rados.0.smithi100.stdout:163: writing smithi10066867-13 from 614400 to 1138688 tid 2 2024-04-07T16:17:09.875 INFO:tasks.rados.rados.0.smithi100.stdout:163: writing smithi10066867-13 from 1138688 to 1417216 tid 3 2024-04-07T16:17:09.876 INFO:tasks.rados.rados.0.smithi100.stdout:164: copy_from oid 15 from oid 5 current snap is 11 2024-04-07T16:17:09.876 INFO:tasks.rados.rados.0.smithi100.stdout:165: rollback oid 19 current snap is 11 2024-04-07T16:17:09.876 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 19 to 9 2024-04-07T16:17:09.876 INFO:tasks.rados.rados.0.smithi100.stdout:166: copy_from oid 21 from oid 24 current snap is 11 2024-04-07T16:17:09.877 INFO:tasks.rados.rados.0.smithi100.stdout:163: finishing write tid 1 to smithi10066867-13 2024-04-07T16:17:09.877 INFO:tasks.rados.rados.0.smithi100.stdout:167: copy_from oid 3 from oid 7 current snap is 11 2024-04-07T16:17:09.877 INFO:tasks.rados.rados.0.smithi100.stdout:168: read oid 33 snap -1 2024-04-07T16:17:09.877 INFO:tasks.rados.rados.0.smithi100.stdout:168: expect (ObjNum 60 snap 5 seq_num 60) 2024-04-07T16:17:09.877 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 16 current snap is 11 2024-04-07T16:17:09.878 INFO:tasks.rados.rados.0.smithi100.stdout:169: seq_num 73 ranges {1449984=425984,1875968=409600,2285568=393216} 2024-04-07T16:17:09.909 INFO:tasks.rados.rados.0.smithi100.stdout:169: writing smithi10066867-16 from 1449984 to 1875968 tid 1 2024-04-07T16:17:09.916 INFO:tasks.rados.rados.0.smithi100.stdout:169: writing smithi10066867-16 from 1875968 to 2285568 tid 2 2024-04-07T16:17:09.921 INFO:tasks.rados.rados.0.smithi100.stdout:169: writing smithi10066867-16 from 2285568 to 2678784 tid 3 2024-04-07T16:17:09.921 INFO:tasks.rados.rados.0.smithi100.stdout:170: rmattr oid 34 current snap is 11 2024-04-07T16:17:09.921 INFO:tasks.rados.rados.0.smithi100.stdout:170: done (7 left) 2024-04-07T16:17:09.921 INFO:tasks.rados.rados.0.smithi100.stdout:171: snap_remove snap 10 2024-04-07T16:17:10.105 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:17:09 smithi144 bash[23522]: cluster 2024-04-07T16:17:08.697612+0000 mgr.y (mgr.14553) 188 : cluster 0 pgmap v159: 121 pgs: 121 active+clean; 31 MiB data, 278 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:17:10.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:17:09 smithi144 bash[23522]: cluster 2024-04-07T16:17:08.861421+0000 mon.a (mon.0) 912 : cluster 0 osdmap e94: 8 total, 8 up, 8 in 2024-04-07T16:17:10.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:17:09 smithi100 bash[21346]: cluster 2024-04-07T16:17:08.697612+0000 mgr.y (mgr.14553) 188 : cluster 0 pgmap v159: 121 pgs: 121 active+clean; 31 MiB data, 278 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:17:10.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:17:09 smithi100 bash[21346]: cluster 2024-04-07T16:17:08.861421+0000 mon.a (mon.0) 912 : cluster 0 osdmap e94: 8 total, 8 up, 8 in 2024-04-07T16:17:10.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:17:09 smithi100 bash[31660]: cluster 2024-04-07T16:17:08.697612+0000 mgr.y (mgr.14553) 188 : cluster 0 pgmap v159: 121 pgs: 121 active+clean; 31 MiB data, 278 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:17:10.188 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:17:09 smithi100 bash[31660]: cluster 2024-04-07T16:17:08.861421+0000 mon.a (mon.0) 912 : cluster 0 osdmap e94: 8 total, 8 up, 8 in 2024-04-07T16:17:10.868 INFO:tasks.rados.rados.0.smithi100.stdout:171: done (7 left) 2024-04-07T16:17:10.868 INFO:tasks.rados.rados.0.smithi100.stdout:172: rollback oid 36 current snap is 11 2024-04-07T16:17:10.869 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 36 to 8 2024-04-07T16:17:10.869 INFO:tasks.rados.rados.0.smithi100.stdout:163: finishing write tid 2 to smithi10066867-13 2024-04-07T16:17:10.869 INFO:tasks.rados.rados.0.smithi100.stdout:163: finishing write tid 3 to smithi10066867-13 2024-04-07T16:17:10.869 INFO:tasks.rados.rados.0.smithi100.stdout:165: finishing rollback tid 0 to smithi10066867-19 2024-04-07T16:17:10.869 INFO:tasks.rados.rados.0.smithi100.stdout:163: finishing write tid 4 to smithi10066867-13 2024-04-07T16:17:10.869 INFO:tasks.rados.rados.0.smithi100.stdout:165: finishing rollback tid 1 to smithi10066867-19 2024-04-07T16:17:10.869 INFO:tasks.rados.rados.0.smithi100.stdout:163: finishing write tid 5 to smithi10066867-13 2024-04-07T16:17:10.869 INFO:tasks.rados.rados.0.smithi100.stdout:163: oid 13 updating version 0 to 13 2024-04-07T16:17:10.869 INFO:tasks.rados.rados.0.smithi100.stdout:163: oid 13 updating version 13 to 14 2024-04-07T16:17:10.869 INFO:tasks.rados.rados.0.smithi100.stdout:163: oid 13 version 14 is already newer than 12 2024-04-07T16:17:10.869 INFO:tasks.rados.rados.0.smithi100.stdout:163: oid 13 version 14 is already newer than 11 2024-04-07T16:17:10.869 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 13 v 14 (ObjNum 72 snap 11 seq_num 72) dirty exists 2024-04-07T16:17:10.869 INFO:tasks.rados.rados.0.smithi100.stdout:163: left oid 13 (ObjNum 72 snap 11 seq_num 72) 2024-04-07T16:17:10.869 INFO:tasks.rados.rados.0.smithi100.stdout:165: finishing rollback tid 2 to smithi10066867-19 2024-04-07T16:17:10.869 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 19 v 38 (ObjNum 18 snap 0 seq_num 18) dirty exists 2024-04-07T16:17:10.870 INFO:tasks.rados.rados.0.smithi100.stdout:164: finishing copy_from racing read to smithi10066867-15 2024-04-07T16:17:10.870 INFO:tasks.rados.rados.0.smithi100.stdout:167: finishing copy_from racing read to smithi10066867-3 2024-04-07T16:17:10.870 INFO:tasks.rados.rados.0.smithi100.stdout:163: done (7 left) 2024-04-07T16:17:10.870 INFO:tasks.rados.rados.0.smithi100.stdout:165: done (6 left) 2024-04-07T16:17:10.870 INFO:tasks.rados.rados.0.smithi100.stdout:173: copy_from oid 9 from oid 13 current snap is 11 2024-04-07T16:17:10.870 INFO:tasks.rados.rados.0.smithi100.stdout:168: expect (ObjNum 60 snap 5 seq_num 60) 2024-04-07T16:17:10.976 INFO:tasks.rados.rados.0.smithi100.stdout:164: finishing copy_from to smithi10066867-15 2024-04-07T16:17:10.976 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 15 v 2 (ObjNum 4 snap 0 seq_num 4) dirty exists 2024-04-07T16:17:10.976 INFO:tasks.rados.rados.0.smithi100.stdout:166: finishing copy_from racing read to smithi10066867-21 2024-04-07T16:17:10.976 INFO:tasks.rados.rados.0.smithi100.stdout:166: finishing copy_from to smithi10066867-21 2024-04-07T16:17:10.976 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 21 v 2 (ObjNum 28 snap 0 seq_num 28) dirty exists 2024-04-07T16:17:10.976 INFO:tasks.rados.rados.0.smithi100.stdout:167: finishing copy_from to smithi10066867-3 2024-04-07T16:17:10.977 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 3 v 2 (ObjNum 16 snap 0 seq_num 16) dirty exists 2024-04-07T16:17:10.977 INFO:tasks.rados.rados.0.smithi100.stdout:169: finishing write tid 1 to smithi10066867-16 2024-04-07T16:17:10.977 INFO:tasks.rados.rados.0.smithi100.stdout:169: finishing write tid 2 to smithi10066867-16 2024-04-07T16:17:10.977 INFO:tasks.rados.rados.0.smithi100.stdout:169: finishing write tid 3 to smithi10066867-16 2024-04-07T16:17:10.977 INFO:tasks.rados.rados.0.smithi100.stdout:164: done (6 left) 2024-04-07T16:17:10.977 INFO:tasks.rados.rados.0.smithi100.stdout:166: done (5 left) 2024-04-07T16:17:10.977 INFO:tasks.rados.rados.0.smithi100.stdout:167: done (4 left) 2024-04-07T16:17:10.977 INFO:tasks.rados.rados.0.smithi100.stdout:168: done (3 left) 2024-04-07T16:17:10.977 INFO:tasks.rados.rados.0.smithi100.stdout:174: copy_from oid 31 from oid 50 current snap is 11 2024-04-07T16:17:10.977 INFO:tasks.rados.rados.0.smithi100.stdout:169: finishing write tid 4 to smithi10066867-16 2024-04-07T16:17:10.977 INFO:tasks.rados.rados.0.smithi100.stdout:169: finishing write tid 5 to smithi10066867-16 2024-04-07T16:17:10.977 INFO:tasks.rados.rados.0.smithi100.stdout:169: oid 16 updating version 0 to 53 2024-04-07T16:17:10.977 INFO:tasks.rados.rados.0.smithi100.stdout:169: oid 16 version 53 is already newer than 50 2024-04-07T16:17:10.977 INFO:tasks.rados.rados.0.smithi100.stdout:169: oid 16 version 53 is already newer than 52 2024-04-07T16:17:10.977 INFO:tasks.rados.rados.0.smithi100.stdout:169: oid 16 version 53 is already newer than 51 2024-04-07T16:17:10.977 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 16 v 53 (ObjNum 73 snap 11 seq_num 73) dirty exists 2024-04-07T16:17:10.978 INFO:tasks.rados.rados.0.smithi100.stdout:169: left oid 16 (ObjNum 73 snap 11 seq_num 73) 2024-04-07T16:17:10.978 INFO:tasks.rados.rados.0.smithi100.stdout:172: finishing rollback tid 0 to smithi10066867-36 2024-04-07T16:17:10.978 INFO:tasks.rados.rados.0.smithi100.stdout:172: finishing rollback tid 1 to smithi10066867-36 2024-04-07T16:17:10.978 INFO:tasks.rados.rados.0.smithi100.stdout:172: finishing rollback tid 2 to smithi10066867-36 2024-04-07T16:17:10.978 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 36 v 42 (ObjNum 10 snap 0 seq_num 10) dirty exists 2024-04-07T16:17:10.978 INFO:tasks.rados.rados.0.smithi100.stdout:173: finishing copy_from racing read to smithi10066867-9 2024-04-07T16:17:10.978 INFO:tasks.rados.rados.0.smithi100.stdout:173: finishing copy_from to smithi10066867-9 2024-04-07T16:17:10.978 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 9 v 14 (ObjNum 72 snap 11 seq_num 72) dirty exists 2024-04-07T16:17:10.978 INFO:tasks.rados.rados.0.smithi100.stdout:169: done (3 left) 2024-04-07T16:17:10.978 INFO:tasks.rados.rados.0.smithi100.stdout:172: done (2 left) 2024-04-07T16:17:10.978 INFO:tasks.rados.rados.0.smithi100.stdout:173: done (1 left) 2024-04-07T16:17:10.978 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 43 current snap is 11 2024-04-07T16:17:10.978 INFO:tasks.rados.rados.0.smithi100.stdout:175: seq_num 74 ranges {1843200=442368,2285568=696320,2981888=106496} 2024-04-07T16:17:10.994 INFO:tasks.rados.rados.0.smithi100.stdout:175: writing smithi10066867-43 from 1843200 to 2285568 tid 1 2024-04-07T16:17:11.003 INFO:tasks.rados.rados.0.smithi100.stdout:175: writing smithi10066867-43 from 2285568 to 2981888 tid 2 2024-04-07T16:17:11.004 INFO:tasks.rados.rados.0.smithi100.stdout:175: writing smithi10066867-43 from 2981888 to 3088384 tid 3 2024-04-07T16:17:11.004 INFO:tasks.rados.rados.0.smithi100.stdout:176: setattr oid 49 current snap is 11 2024-04-07T16:17:11.004 INFO:tasks.rados.rados.0.smithi100.stdout:174: finishing copy_from racing read to smithi10066867-31 2024-04-07T16:17:11.004 INFO:tasks.rados.rados.0.smithi100.stdout:174: finishing copy_from to smithi10066867-31 2024-04-07T16:17:11.005 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 31 v 7 (ObjNum 49 snap 0 seq_num 49) dirty exists 2024-04-07T16:17:11.005 INFO:tasks.rados.rados.0.smithi100.stdout:175: finishing write tid 1 to smithi10066867-43 2024-04-07T16:17:11.009 INFO:tasks.rados.rados.0.smithi100.stdout:174: done (2 left) 2024-04-07T16:17:11.009 INFO:tasks.rados.rados.0.smithi100.stdout:177: snap_remove snap 9 2024-04-07T16:17:11.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:17:10 smithi100 bash[21346]: cluster 2024-04-07T16:17:09.863380+0000 mon.a (mon.0) 913 : cluster 0 osdmap e95: 8 total, 8 up, 8 in 2024-04-07T16:17:11.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:17:10 smithi100 bash[31660]: cluster 2024-04-07T16:17:09.863380+0000 mon.a (mon.0) 913 : cluster 0 osdmap e95: 8 total, 8 up, 8 in 2024-04-07T16:17:11.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:17:10 smithi144 bash[23522]: cluster 2024-04-07T16:17:09.863380+0000 mon.a (mon.0) 913 : cluster 0 osdmap e95: 8 total, 8 up, 8 in 2024-04-07T16:17:11.881 INFO:tasks.rados.rados.0.smithi100.stdout:177: done (2 left) 2024-04-07T16:17:11.881 INFO:tasks.rados.rados.0.smithi100.stdout:178: rmattr oid 44 current snap is 11 2024-04-07T16:17:11.881 INFO:tasks.rados.rados.0.smithi100.stdout:178: done (2 left) 2024-04-07T16:17:11.881 INFO:tasks.rados.rados.0.smithi100.stdout:179: copy_from oid 5 from oid 38 current snap is 11 2024-04-07T16:17:11.881 INFO:tasks.rados.rados.0.smithi100.stdout:175: finishing write tid 2 to smithi10066867-43 2024-04-07T16:17:11.881 INFO:tasks.rados.rados.0.smithi100.stdout:175: finishing write tid 3 to smithi10066867-43 2024-04-07T16:17:11.881 INFO:tasks.rados.rados.0.smithi100.stdout:175: finishing write tid 4 to smithi10066867-43 2024-04-07T16:17:11.881 INFO:tasks.rados.rados.0.smithi100.stdout:175: finishing write tid 5 to smithi10066867-43 2024-04-07T16:17:11.882 INFO:tasks.rados.rados.0.smithi100.stdout:175: oid 43 updating version 0 to 22 2024-04-07T16:17:11.882 INFO:tasks.rados.rados.0.smithi100.stdout:175: oid 43 version 22 is already newer than 20 2024-04-07T16:17:11.882 INFO:tasks.rados.rados.0.smithi100.stdout:175: oid 43 version 22 is already newer than 19 2024-04-07T16:17:11.882 INFO:tasks.rados.rados.0.smithi100.stdout:175: oid 43 version 22 is already newer than 21 2024-04-07T16:17:11.882 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 43 v 22 (ObjNum 74 snap 11 seq_num 74) dirty exists 2024-04-07T16:17:11.882 INFO:tasks.rados.rados.0.smithi100.stdout:175: left oid 43 (ObjNum 74 snap 11 seq_num 74) 2024-04-07T16:17:11.882 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 49 v 57 (ObjNum 48 snap 0 seq_num 48) dirty exists 2024-04-07T16:17:11.882 INFO:tasks.rados.rados.0.smithi100.stdout:175: done (2 left) 2024-04-07T16:17:11.882 INFO:tasks.rados.rados.0.smithi100.stdout:176: done (1 left) 2024-04-07T16:17:11.882 INFO:tasks.rados.rados.0.smithi100.stdout:180: snap_remove snap 8 2024-04-07T16:17:12.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:17:11 smithi100 bash[21346]: cluster 2024-04-07T16:17:10.698292+0000 mgr.y (mgr.14553) 189 : cluster 0 pgmap v162: 121 pgs: 121 active+clean; 36 MiB data, 283 MiB used, 715 GiB / 715 GiB avail; 941 KiB/s rd, 1.8 MiB/s wr, 25 op/s 2024-04-07T16:17:12.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:17:11 smithi100 bash[21346]: cluster 2024-04-07T16:17:10.881494+0000 mon.a (mon.0) 914 : cluster 0 osdmap e96: 8 total, 8 up, 8 in 2024-04-07T16:17:12.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:17:11 smithi100 bash[31660]: cluster 2024-04-07T16:17:10.698292+0000 mgr.y (mgr.14553) 189 : cluster 0 pgmap v162: 121 pgs: 121 active+clean; 36 MiB data, 283 MiB used, 715 GiB / 715 GiB avail; 941 KiB/s rd, 1.8 MiB/s wr, 25 op/s 2024-04-07T16:17:12.188 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:17:11 smithi100 bash[31660]: cluster 2024-04-07T16:17:10.881494+0000 mon.a (mon.0) 914 : cluster 0 osdmap e96: 8 total, 8 up, 8 in 2024-04-07T16:17:12.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:17:11 smithi144 bash[23522]: cluster 2024-04-07T16:17:10.698292+0000 mgr.y (mgr.14553) 189 : cluster 0 pgmap v162: 121 pgs: 121 active+clean; 36 MiB data, 283 MiB used, 715 GiB / 715 GiB avail; 941 KiB/s rd, 1.8 MiB/s wr, 25 op/s 2024-04-07T16:17:12.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:17:11 smithi144 bash[23522]: cluster 2024-04-07T16:17:10.881494+0000 mon.a (mon.0) 914 : cluster 0 osdmap e96: 8 total, 8 up, 8 in 2024-04-07T16:17:12.819 INFO:tasks.rados.rados.0.smithi100.stdout:180: done (1 left) 2024-04-07T16:17:12.819 INFO:tasks.rados.rados.0.smithi100.stdout:181: read oid 28 snap -1 2024-04-07T16:17:12.819 INFO:tasks.rados.rados.0.smithi100.stdout:181: expect deleted 2024-04-07T16:17:12.819 INFO:tasks.rados.rados.0.smithi100.stdout:182: rollback oid 34 current snap is 11 2024-04-07T16:17:12.819 INFO:tasks.rados.rados.0.smithi100.stdout:182: done (2 left) 2024-04-07T16:17:12.819 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 19 current snap is 11 2024-04-07T16:17:12.819 INFO:tasks.rados.rados.0.smithi100.stdout:183: seq_num 75 ranges {303104=458752,761856=581632,1343488=221184} 2024-04-07T16:17:12.827 INFO:tasks.rados.rados.0.smithi100.stdout:183: writing smithi10066867-19 from 303104 to 761856 tid 1 2024-04-07T16:17:12.833 INFO:tasks.rados.rados.0.smithi100.stdout:183: writing smithi10066867-19 from 761856 to 1343488 tid 2 2024-04-07T16:17:12.837 INFO:tasks.rados.rados.0.smithi100.stdout:183: writing smithi10066867-19 from 1343488 to 1564672 tid 3 2024-04-07T16:17:12.837 INFO:tasks.rados.rados.0.smithi100.stdout:184: read oid 42 snap -1 2024-04-07T16:17:12.837 INFO:tasks.rados.rados.0.smithi100.stdout:184: expect (ObjNum 41 snap 0 seq_num 41) 2024-04-07T16:17:12.837 INFO:tasks.rados.rados.0.smithi100.stdout:185: rollback oid 31 current snap is 11 2024-04-07T16:17:12.837 INFO:tasks.rados.rados.0.smithi100.stdout:185: done (4 left) 2024-04-07T16:17:12.837 INFO:tasks.rados.rados.0.smithi100.stdout:186: snap_create 2024-04-07T16:17:12.837 INFO:tasks.rados.rados.0.smithi100.stdout:179: finishing copy_from racing read to smithi10066867-5 2024-04-07T16:17:12.837 INFO:tasks.rados.rados.0.smithi100.stdout:179: finishing copy_from to smithi10066867-5 2024-04-07T16:17:12.837 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 5 v 2 (ObjNum 28 snap 0 seq_num 28) dirty exists 2024-04-07T16:17:12.851 INFO:tasks.rados.rados.0.smithi100.stdout:183: finishing write tid 1 to smithi10066867-19 2024-04-07T16:17:12.854 INFO:tasks.rados.rados.0.smithi100.stdout:183: finishing write tid 2 to smithi10066867-19 2024-04-07T16:17:12.854 INFO:tasks.rados.rados.0.smithi100.stdout:183: finishing write tid 3 to smithi10066867-19 2024-04-07T16:17:12.854 INFO:tasks.rados.rados.0.smithi100.stdout:183: finishing write tid 4 to smithi10066867-19 2024-04-07T16:17:12.856 INFO:tasks.rados.rados.0.smithi100.stdout:184: expect (ObjNum 41 snap 0 seq_num 41) 2024-04-07T16:17:12.900 INFO:tasks.rados.rados.0.smithi100.stdout:183: finishing write tid 5 to smithi10066867-19 2024-04-07T16:17:12.900 INFO:tasks.rados.rados.0.smithi100.stdout:183: oid 19 updating version 0 to 50 2024-04-07T16:17:12.900 INFO:tasks.rados.rados.0.smithi100.stdout:183: oid 19 updating version 50 to 51 2024-04-07T16:17:12.900 INFO:tasks.rados.rados.0.smithi100.stdout:183: oid 19 updating version 51 to 52 2024-04-07T16:17:12.900 INFO:tasks.rados.rados.0.smithi100.stdout:183: oid 19 version 52 is already newer than 49 2024-04-07T16:17:12.900 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 19 v 52 (ObjNum 75 snap 11 seq_num 75) dirty exists 2024-04-07T16:17:12.900 INFO:tasks.rados.rados.0.smithi100.stdout:183: left oid 19 (ObjNum 75 snap 11 seq_num 75) 2024-04-07T16:17:13.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:17:12 smithi100 bash[21346]: cluster 2024-04-07T16:17:11.893654+0000 mon.a (mon.0) 915 : cluster 0 osdmap e97: 8 total, 8 up, 8 in 2024-04-07T16:17:13.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:17:12 smithi100 bash[21346]: cluster 2024-04-07T16:17:12.831145+0000 mon.a (mon.0) 916 : cluster 0 osdmap e98: 8 total, 8 up, 8 in 2024-04-07T16:17:13.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:17:12 smithi100 bash[31660]: cluster 2024-04-07T16:17:11.893654+0000 mon.a (mon.0) 915 : cluster 0 osdmap e97: 8 total, 8 up, 8 in 2024-04-07T16:17:13.188 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:17:12 smithi100 bash[31660]: cluster 2024-04-07T16:17:12.831145+0000 mon.a (mon.0) 916 : cluster 0 osdmap e98: 8 total, 8 up, 8 in 2024-04-07T16:17:13.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:17:12 smithi144 bash[23522]: cluster 2024-04-07T16:17:11.893654+0000 mon.a (mon.0) 915 : cluster 0 osdmap e97: 8 total, 8 up, 8 in 2024-04-07T16:17:13.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:17:12 smithi144 bash[23522]: cluster 2024-04-07T16:17:12.831145+0000 mon.a (mon.0) 916 : cluster 0 osdmap e98: 8 total, 8 up, 8 in 2024-04-07T16:17:13.819 INFO:tasks.rados.rados.0.smithi100.stdout:179: done (4 left) 2024-04-07T16:17:13.819 INFO:tasks.rados.rados.0.smithi100.stdout:181: done (3 left) 2024-04-07T16:17:13.819 INFO:tasks.rados.rados.0.smithi100.stdout:183: done (2 left) 2024-04-07T16:17:13.819 INFO:tasks.rados.rados.0.smithi100.stdout:184: done (1 left) 2024-04-07T16:17:13.819 INFO:tasks.rados.rados.0.smithi100.stdout:186: done (0 left) 2024-04-07T16:17:13.819 INFO:tasks.rados.rados.0.smithi100.stdout:187: delete oid 11 current snap is 12 2024-04-07T16:17:13.826 INFO:tasks.rados.rados.0.smithi100.stdout:187: done (0 left) 2024-04-07T16:17:13.826 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 44 current snap is 12 2024-04-07T16:17:13.826 INFO:tasks.rados.rados.0.smithi100.stdout:188: seq_num 77 ranges {0=491520,491520=794624,1286144=8192} 2024-04-07T16:17:13.833 INFO:tasks.rados.rados.0.smithi100.stdout:188: writing smithi10066867-44 from 0 to 491520 tid 1 2024-04-07T16:17:13.846 INFO:tasks.rados.rados.0.smithi100.stdout:188: writing smithi10066867-44 from 491520 to 1286144 tid 2 2024-04-07T16:17:13.846 INFO:tasks.rados.rados.0.smithi100.stdout:188: writing smithi10066867-44 from 1286144 to 1294336 tid 3 2024-04-07T16:17:13.846 INFO:tasks.rados.rados.0.smithi100.stdout:189: snap_create 2024-04-07T16:17:13.847 INFO:tasks.rados.rados.0.smithi100.stdout:188: finishing write tid 1 to smithi10066867-44 2024-04-07T16:17:13.852 INFO:tasks.rados.rados.0.smithi100.stdout:188: finishing write tid 2 to smithi10066867-44 2024-04-07T16:17:13.853 INFO:tasks.rados.rados.0.smithi100.stdout:188: finishing write tid 3 to smithi10066867-44 2024-04-07T16:17:13.853 INFO:tasks.rados.rados.0.smithi100.stdout:188: finishing write tid 4 to smithi10066867-44 2024-04-07T16:17:13.855 INFO:tasks.rados.rados.0.smithi100.stdout:188: finishing write tid 5 to smithi10066867-44 2024-04-07T16:17:13.855 INFO:tasks.rados.rados.0.smithi100.stdout:188: oid 44 updating version 0 to 47 2024-04-07T16:17:13.855 INFO:tasks.rados.rados.0.smithi100.stdout:188: oid 44 version 47 is already newer than 44 2024-04-07T16:17:13.855 INFO:tasks.rados.rados.0.smithi100.stdout:188: oid 44 version 47 is already newer than 45 2024-04-07T16:17:13.855 INFO:tasks.rados.rados.0.smithi100.stdout:188: oid 44 version 47 is already newer than 46 2024-04-07T16:17:13.856 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 44 v 47 (ObjNum 77 snap 12 seq_num 77) dirty exists 2024-04-07T16:17:13.856 INFO:tasks.rados.rados.0.smithi100.stdout:188: left oid 44 (ObjNum 77 snap 12 seq_num 77) 2024-04-07T16:17:14.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:17:13 smithi100 bash[21346]: cluster 2024-04-07T16:17:12.698970+0000 mgr.y (mgr.14553) 190 : cluster 0 pgmap v165: 121 pgs: 121 active+clean; 36 MiB data, 283 MiB used, 715 GiB / 715 GiB avail; 941 KiB/s rd, 1.8 MiB/s wr, 25 op/s 2024-04-07T16:17:14.188 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:17:13 smithi100 bash[21346]: cluster 2024-04-07T16:17:13.832422+0000 mon.a (mon.0) 917 : cluster 0 osdmap e99: 8 total, 8 up, 8 in 2024-04-07T16:17:14.188 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:17:13 smithi100 bash[31660]: cluster 2024-04-07T16:17:12.698970+0000 mgr.y (mgr.14553) 190 : cluster 0 pgmap v165: 121 pgs: 121 active+clean; 36 MiB data, 283 MiB used, 715 GiB / 715 GiB avail; 941 KiB/s rd, 1.8 MiB/s wr, 25 op/s 2024-04-07T16:17:14.188 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:17:13 smithi100 bash[31660]: cluster 2024-04-07T16:17:13.832422+0000 mon.a (mon.0) 917 : cluster 0 osdmap e99: 8 total, 8 up, 8 in 2024-04-07T16:17:14.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:17:13 smithi144 bash[23522]: cluster 2024-04-07T16:17:12.698970+0000 mgr.y (mgr.14553) 190 : cluster 0 pgmap v165: 121 pgs: 121 active+clean; 36 MiB data, 283 MiB used, 715 GiB / 715 GiB avail; 941 KiB/s rd, 1.8 MiB/s wr, 25 op/s 2024-04-07T16:17:14.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:17:13 smithi144 bash[23522]: cluster 2024-04-07T16:17:13.832422+0000 mon.a (mon.0) 917 : cluster 0 osdmap e99: 8 total, 8 up, 8 in 2024-04-07T16:17:14.820 INFO:tasks.rados.rados.0.smithi100.stdout:188: done (1 left) 2024-04-07T16:17:14.821 INFO:tasks.rados.rados.0.smithi100.stdout:189: done (0 left) 2024-04-07T16:17:14.821 INFO:tasks.rados.rados.0.smithi100.stdout:190: snap_remove snap 11 2024-04-07T16:17:15.437 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:17:15 smithi100 bash[21601]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:17:15] "GET /metrics HTTP/1.1" 200 36077 "" "Prometheus/2.43.0" 2024-04-07T16:17:15.827 INFO:tasks.rados.rados.0.smithi100.stdout:190: done (0 left) 2024-04-07T16:17:15.827 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 22 current snap is 13 2024-04-07T16:17:15.827 INFO:tasks.rados.rados.0.smithi100.stdout:191: seq_num 78 ranges {794624=516096,1310720=679936,1990656=122880} 2024-04-07T16:17:15.848 INFO:tasks.rados.rados.0.smithi100.stdout:191: writing smithi10066867-22 from 794624 to 1310720 tid 1 2024-04-07T16:17:15.862 INFO:tasks.rados.rados.0.smithi100.stdout:191: writing smithi10066867-22 from 1310720 to 1990656 tid 2 2024-04-07T16:17:15.865 INFO:tasks.rados.rados.0.smithi100.stdout:191: writing smithi10066867-22 from 1990656 to 2113536 tid 3 2024-04-07T16:17:15.865 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 10 current snap is 13 2024-04-07T16:17:15.865 INFO:tasks.rados.rados.0.smithi100.stdout:192: seq_num 79 ranges {1597440=532480,2129920=565248,2695168=237568} 2024-04-07T16:17:15.894 INFO:tasks.rados.rados.0.smithi100.stdout:192: writing smithi10066867-10 from 1597440 to 2129920 tid 1 2024-04-07T16:17:15.900 INFO:tasks.rados.rados.0.smithi100.stdout:192: writing smithi10066867-10 from 2129920 to 2695168 tid 2 2024-04-07T16:17:15.903 INFO:tasks.rados.rados.0.smithi100.stdout:192: writing smithi10066867-10 from 2695168 to 2932736 tid 3 2024-04-07T16:17:15.904 INFO:tasks.rados.rados.0.smithi100.stdout:193: setattr oid 6 current snap is 13 2024-04-07T16:17:15.904 INFO:tasks.rados.rados.0.smithi100.stdout:191: finishing write tid 1 to smithi10066867-22 2024-04-07T16:17:15.904 INFO:tasks.rados.rados.0.smithi100.stdout:191: finishing write tid 2 to smithi10066867-22 2024-04-07T16:17:15.904 INFO:tasks.rados.rados.0.smithi100.stdout:191: finishing write tid 3 to smithi10066867-22 2024-04-07T16:17:15.904 INFO:tasks.rados.rados.0.smithi100.stdout:191: finishing write tid 4 to smithi10066867-22 2024-04-07T16:17:15.904 INFO:tasks.rados.rados.0.smithi100.stdout:191: finishing write tid 5 to smithi10066867-22 2024-04-07T16:17:15.904 INFO:tasks.rados.rados.0.smithi100.stdout:191: oid 22 updating version 0 to 21 2024-04-07T16:17:15.904 INFO:tasks.rados.rados.0.smithi100.stdout:191: oid 22 updating version 21 to 22 2024-04-07T16:17:15.904 INFO:tasks.rados.rados.0.smithi100.stdout:191: oid 22 updating version 22 to 23 2024-04-07T16:17:15.904 INFO:tasks.rados.rados.0.smithi100.stdout:191: oid 22 updating version 23 to 24 2024-04-07T16:17:15.904 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 22 v 24 (ObjNum 78 snap 13 seq_num 78) dirty exists 2024-04-07T16:17:15.905 INFO:tasks.rados.rados.0.smithi100.stdout:191: left oid 22 (ObjNum 78 snap 13 seq_num 78) 2024-04-07T16:17:15.905 INFO:tasks.rados.rados.0.smithi100.stdout:192: finishing write tid 1 to smithi10066867-10 2024-04-07T16:17:15.906 INFO:tasks.rados.rados.0.smithi100.stdout:191: done (2 left) 2024-04-07T16:17:15.906 INFO:tasks.rados.rados.0.smithi100.stdout:194: read oid 12 snap 12 2024-04-07T16:17:15.906 INFO:tasks.rados.rados.0.smithi100.stdout:194: expect (ObjNum 58 snap 4 seq_num 58) 2024-04-07T16:17:15.907 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 3 current snap is 13 2024-04-07T16:17:15.907 INFO:tasks.rados.rados.0.smithi100.stdout:195: seq_num 80 ranges {270336=548864,819200=450560,1269760=352256} 2024-04-07T16:17:15.914 INFO:tasks.rados.rados.0.smithi100.stdout:195: writing smithi10066867-3 from 270336 to 819200 tid 1 2024-04-07T16:17:15.918 INFO:tasks.rados.rados.0.smithi100.stdout:195: writing smithi10066867-3 from 819200 to 1269760 tid 2 2024-04-07T16:17:15.921 INFO:tasks.rados.rados.0.smithi100.stdout:195: writing smithi10066867-3 from 1269760 to 1622016 tid 3 2024-04-07T16:17:15.921 INFO:tasks.rados.rados.0.smithi100.stdout:196: snap_create 2024-04-07T16:17:15.922 INFO:tasks.rados.rados.0.smithi100.stdout:192: finishing write tid 2 to smithi10066867-10 2024-04-07T16:17:15.922 INFO:tasks.rados.rados.0.smithi100.stdout:192: finishing write tid 3 to smithi10066867-10 2024-04-07T16:17:15.922 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 6 v 48 (ObjNum 289 snap 0 seq_num 0) dirty exists 2024-04-07T16:17:15.922 INFO:tasks.rados.rados.0.smithi100.stdout:192: finishing write tid 4 to smithi10066867-10 2024-04-07T16:17:15.922 INFO:tasks.rados.rados.0.smithi100.stdout:194: expect (ObjNum 58 snap 4 seq_num 58) 2024-04-07T16:17:15.985 INFO:tasks.rados.rados.0.smithi100.stdout:192: finishing write tid 5 to smithi10066867-10 2024-04-07T16:17:15.985 INFO:tasks.rados.rados.0.smithi100.stdout:192: oid 10 updating version 0 to 19 2024-04-07T16:17:15.985 INFO:tasks.rados.rados.0.smithi100.stdout:192: oid 10 updating version 19 to 20 2024-04-07T16:17:15.985 INFO:tasks.rados.rados.0.smithi100.stdout:192: oid 10 updating version 20 to 21 2024-04-07T16:17:15.986 INFO:tasks.rados.rados.0.smithi100.stdout:192: oid 10 updating version 21 to 22 2024-04-07T16:17:15.986 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 10 v 22 (ObjNum 79 snap 13 seq_num 79) dirty exists 2024-04-07T16:17:15.986 INFO:tasks.rados.rados.0.smithi100.stdout:192: left oid 10 (ObjNum 79 snap 13 seq_num 79) 2024-04-07T16:17:15.986 INFO:tasks.rados.rados.0.smithi100.stdout:195: finishing write tid 1 to smithi10066867-3 2024-04-07T16:17:15.986 INFO:tasks.rados.rados.0.smithi100.stdout:195: finishing write tid 2 to smithi10066867-3 2024-04-07T16:17:15.986 INFO:tasks.rados.rados.0.smithi100.stdout:195: finishing write tid 3 to smithi10066867-3 2024-04-07T16:17:15.986 INFO:tasks.rados.rados.0.smithi100.stdout:195: finishing write tid 4 to smithi10066867-3 2024-04-07T16:17:15.986 INFO:tasks.rados.rados.0.smithi100.stdout:195: finishing write tid 5 to smithi10066867-3 2024-04-07T16:17:15.986 INFO:tasks.rados.rados.0.smithi100.stdout:195: oid 3 updating version 0 to 45 2024-04-07T16:17:15.986 INFO:tasks.rados.rados.0.smithi100.stdout:195: oid 3 version 45 is already newer than 44 2024-04-07T16:17:15.986 INFO:tasks.rados.rados.0.smithi100.stdout:195: oid 3 version 45 is already newer than 42 2024-04-07T16:17:15.986 INFO:tasks.rados.rados.0.smithi100.stdout:195: oid 3 version 45 is already newer than 43 2024-04-07T16:17:15.986 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 3 v 45 (ObjNum 80 snap 13 seq_num 80) dirty exists 2024-04-07T16:17:15.986 INFO:tasks.rados.rados.0.smithi100.stdout:195: left oid 3 (ObjNum 80 snap 13 seq_num 80) 2024-04-07T16:17:16.105 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:17:15 smithi144 bash[23522]: cluster 2024-04-07T16:17:14.699677+0000 mgr.y (mgr.14553) 191 : cluster 0 pgmap v168: 121 pgs: 121 active+clean; 37 MiB data, 293 MiB used, 715 GiB / 715 GiB avail; 2.4 MiB/s rd, 2.0 MiB/s wr, 17 op/s 2024-04-07T16:17:16.105 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:17:15 smithi144 bash[23522]: cluster 2024-04-07T16:17:14.834826+0000 mon.a (mon.0) 918 : cluster 0 osdmap e100: 8 total, 8 up, 8 in 2024-04-07T16:17:16.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:17:15 smithi100 bash[31660]: cluster 2024-04-07T16:17:14.699677+0000 mgr.y (mgr.14553) 191 : cluster 0 pgmap v168: 121 pgs: 121 active+clean; 37 MiB data, 293 MiB used, 715 GiB / 715 GiB avail; 2.4 MiB/s rd, 2.0 MiB/s wr, 17 op/s 2024-04-07T16:17:16.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:17:15 smithi100 bash[31660]: cluster 2024-04-07T16:17:14.834826+0000 mon.a (mon.0) 918 : cluster 0 osdmap e100: 8 total, 8 up, 8 in 2024-04-07T16:17:16.188 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:17:15 smithi100 bash[21346]: cluster 2024-04-07T16:17:14.699677+0000 mgr.y (mgr.14553) 191 : cluster 0 pgmap v168: 121 pgs: 121 active+clean; 37 MiB data, 293 MiB used, 715 GiB / 715 GiB avail; 2.4 MiB/s rd, 2.0 MiB/s wr, 17 op/s 2024-04-07T16:17:16.188 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:17:15 smithi100 bash[21346]: cluster 2024-04-07T16:17:14.834826+0000 mon.a (mon.0) 918 : cluster 0 osdmap e100: 8 total, 8 up, 8 in 2024-04-07T16:17:16.843 INFO:tasks.rados.rados.0.smithi100.stdout:192: done (4 left) 2024-04-07T16:17:16.844 INFO:tasks.rados.rados.0.smithi100.stdout:193: done (3 left) 2024-04-07T16:17:16.844 INFO:tasks.rados.rados.0.smithi100.stdout:194: done (2 left) 2024-04-07T16:17:16.844 INFO:tasks.rados.rados.0.smithi100.stdout:195: done (1 left) 2024-04-07T16:17:16.844 INFO:tasks.rados.rados.0.smithi100.stdout:196: done (0 left) 2024-04-07T16:17:16.844 INFO:tasks.rados.rados.0.smithi100.stdout:197: snap_create 2024-04-07T16:17:17.105 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:17:16 smithi144 bash[23522]: cluster 2024-04-07T16:17:15.840672+0000 mon.a (mon.0) 919 : cluster 0 osdmap e101: 8 total, 8 up, 8 in 2024-04-07T16:17:17.107 INFO:teuthology.orchestra.run.smithi100.stdout:true 2024-04-07T16:17:17.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:17:16 smithi100 bash[21346]: cluster 2024-04-07T16:17:15.840672+0000 mon.a (mon.0) 919 : cluster 0 osdmap e101: 8 total, 8 up, 8 in 2024-04-07T16:17:17.189 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:17:16 smithi100 bash[31660]: cluster 2024-04-07T16:17:15.840672+0000 mon.a (mon.0) 919 : cluster 0 osdmap e101: 8 total, 8 up, 8 in 2024-04-07T16:17:17.466 INFO:teuthology.orchestra.run.smithi100.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-07T16:17:17.466 INFO:teuthology.orchestra.run.smithi100.stdout:alertmanager.a smithi100 *:9093,9094 running (3m) 3m ago 4m 13.7M - 0.25.0 c8568f914cd2 5143075e7d26 2024-04-07T16:17:17.466 INFO:teuthology.orchestra.run.smithi100.stdout:grafana.a smithi144 *:3000 running (3m) 3m ago 3m 29.2M - 9.4.7 954c08fa6188 7180c0708acd 2024-04-07T16:17:17.466 INFO:teuthology.orchestra.run.smithi100.stdout:mds.a smithi100 running (5m) 3m ago 5m 12.4M - 18.2.2-1171-g267c8ff5 8a4a01e7186c 92593b0c28ae 2024-04-07T16:17:17.466 INFO:teuthology.orchestra.run.smithi100.stdout:mds.b smithi144 running (5m) 3m ago 5m 14.2M - 18.2.2-1171-g267c8ff5 8a4a01e7186c 3df2ee3e17a4 2024-04-07T16:17:17.467 INFO:teuthology.orchestra.run.smithi100.stdout:mgr.x smithi144 *:8443,8765 running (12m) 3m ago 12m 435M - 18.2.2-1171-g267c8ff5 8a4a01e7186c b7feffac74d1 2024-04-07T16:17:17.467 INFO:teuthology.orchestra.run.smithi100.stdout:mgr.y smithi100 *:9283,8765 running (14m) 3m ago 14m 484M - 18.2.2-1171-g267c8ff5 8a4a01e7186c da1d683b9737 2024-04-07T16:17:17.467 INFO:teuthology.orchestra.run.smithi100.stdout:mon.a smithi100 running (14m) 3m ago 14m 46.3M 2048M 18.2.2-1171-g267c8ff5 8a4a01e7186c ec8833f59ea0 2024-04-07T16:17:17.467 INFO:teuthology.orchestra.run.smithi100.stdout:mon.b smithi144 running (12m) 3m ago 12m 39.7M 2048M 18.2.2-1171-g267c8ff5 8a4a01e7186c 8dd687691b2e 2024-04-07T16:17:17.467 INFO:teuthology.orchestra.run.smithi100.stdout:mon.c smithi100 running (12m) 3m ago 12m 37.7M 2048M 18.2.2-1171-g267c8ff5 8a4a01e7186c a6de0151a3ba 2024-04-07T16:17:17.467 INFO:teuthology.orchestra.run.smithi100.stdout:node-exporter.a smithi100 *:9100 running (4m) 3m ago 4m 3116k - 1.5.0 0da6a335fe13 4bce264c1b26 2024-04-07T16:17:17.467 INFO:teuthology.orchestra.run.smithi100.stdout:node-exporter.b smithi144 *:9100 running (4m) 3m ago 4m 3151k - 1.5.0 0da6a335fe13 b6963413c560 2024-04-07T16:17:17.467 INFO:teuthology.orchestra.run.smithi100.stdout:osd.0 smithi100 running (11m) 3m ago 11m 45.7M 3036M 18.2.2-1171-g267c8ff5 8a4a01e7186c 188deb247979 2024-04-07T16:17:17.467 INFO:teuthology.orchestra.run.smithi100.stdout:osd.1 smithi100 running (10m) 3m ago 10m 44.6M 3036M 18.2.2-1171-g267c8ff5 8a4a01e7186c 7f57ca90894c 2024-04-07T16:17:17.467 INFO:teuthology.orchestra.run.smithi100.stdout:osd.2 smithi100 running (9m) 3m ago 10m 52.2M 3036M 18.2.2-1171-g267c8ff5 8a4a01e7186c f3912ddcc8cc 2024-04-07T16:17:17.467 INFO:teuthology.orchestra.run.smithi100.stdout:osd.3 smithi100 running (9m) 3m ago 9m 41.7M 3036M 18.2.2-1171-g267c8ff5 8a4a01e7186c b41004090c63 2024-04-07T16:17:17.467 INFO:teuthology.orchestra.run.smithi100.stdout:osd.4 smithi144 running (8m) 3m ago 8m 50.6M 3036M 18.2.2-1171-g267c8ff5 8a4a01e7186c 8c73b6b4dc13 2024-04-07T16:17:17.467 INFO:teuthology.orchestra.run.smithi100.stdout:osd.5 smithi144 running (7m) 3m ago 7m 42.6M 3036M 18.2.2-1171-g267c8ff5 8a4a01e7186c 3bfa0cf6f6cb 2024-04-07T16:17:17.467 INFO:teuthology.orchestra.run.smithi100.stdout:osd.6 smithi144 running (6m) 3m ago 6m 42.5M 3036M 18.2.2-1171-g267c8ff5 8a4a01e7186c 32b7f95d77f0 2024-04-07T16:17:17.467 INFO:teuthology.orchestra.run.smithi100.stdout:osd.7 smithi144 running (5m) 3m ago 5m 49.8M 3036M 18.2.2-1171-g267c8ff5 8a4a01e7186c 45f7709e576a 2024-04-07T16:17:17.468 INFO:teuthology.orchestra.run.smithi100.stdout:prometheus.a smithi144 *:9095 running (3m) 3m ago 4m 23.0M - 2.43.0 a07b618ecd1d e456da14a41a 2024-04-07T16:17:17.817 INFO:tasks.rados.rados.0.smithi100.stdout:197: done (0 left) 2024-04-07T16:17:17.817 INFO:tasks.rados.rados.0.smithi100.stdout:198: read oid 45 snap -1 2024-04-07T16:17:17.817 INFO:tasks.rados.rados.0.smithi100.stdout:198: expect (ObjNum 44 snap 0 seq_num 44) 2024-04-07T16:17:17.817 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 31 current snap is 15 2024-04-07T16:17:17.817 INFO:tasks.rados.rados.0.smithi100.stdout:199: seq_num 81 ranges {827392=565248,1392640=737280,2129920=65536} 2024-04-07T16:17:17.831 INFO:tasks.rados.rados.0.smithi100.stdout:199: writing smithi10066867-31 from 827392 to 1392640 tid 1 2024-04-07T16:17:17.839 INFO:tasks.rados.rados.0.smithi100.stdout:199: writing smithi10066867-31 from 1392640 to 2129920 tid 2 2024-04-07T16:17:17.841 INFO:tasks.rados.rados.0.smithi100.stdout:199: writing smithi10066867-31 from 2129920 to 2195456 tid 3 2024-04-07T16:17:17.841 INFO:tasks.rados.rados.0.smithi100.stdout:200: read oid 49 snap -1 2024-04-07T16:17:17.841 INFO:tasks.rados.rados.0.smithi100.stdout:200: expect (ObjNum 48 snap 0 seq_num 48) 2024-04-07T16:17:17.841 INFO:tasks.rados.rados.0.smithi100.stdout:201: delete oid 16 current snap is 15 2024-04-07T16:17:17.841 INFO:tasks.rados.rados.0.smithi100.stdout:198: expect (ObjNum 44 snap 0 seq_num 44) 2024-04-07T16:17:17.876 INFO:tasks.rados.rados.0.smithi100.stdout:199: finishing write tid 1 to smithi10066867-31 2024-04-07T16:17:17.876 INFO:tasks.rados.rados.0.smithi100.stdout:199: finishing write tid 2 to smithi10066867-31 2024-04-07T16:17:17.876 INFO:tasks.rados.rados.0.smithi100.stdout:199: finishing write tid 3 to smithi10066867-31 2024-04-07T16:17:17.876 INFO:tasks.rados.rados.0.smithi100.stdout:200: expect (ObjNum 48 snap 0 seq_num 48) 2024-04-07T16:17:17.900 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-04-07T16:17:17.901 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": { 2024-04-07T16:17:17.901 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph version 18.2.2-1171-g267c8ff5 (267c8ff5b1ff5815581b3b2381d19360e081b37b) reef (stable)": 3 2024-04-07T16:17:17.901 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-07T16:17:17.901 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": { 2024-04-07T16:17:17.901 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph version 18.2.2-1171-g267c8ff5 (267c8ff5b1ff5815581b3b2381d19360e081b37b) reef (stable)": 2 2024-04-07T16:17:17.901 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-07T16:17:17.901 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": { 2024-04-07T16:17:17.901 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph version 18.2.2-1171-g267c8ff5 (267c8ff5b1ff5815581b3b2381d19360e081b37b) reef (stable)": 8 2024-04-07T16:17:17.901 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-07T16:17:17.901 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": { 2024-04-07T16:17:17.901 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph version 18.2.2-1171-g267c8ff5 (267c8ff5b1ff5815581b3b2381d19360e081b37b) reef (stable)": 2 2024-04-07T16:17:17.901 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-07T16:17:17.901 INFO:teuthology.orchestra.run.smithi100.stdout: "overall": { 2024-04-07T16:17:17.902 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph version 18.2.2-1171-g267c8ff5 (267c8ff5b1ff5815581b3b2381d19360e081b37b) reef (stable)": 15 2024-04-07T16:17:17.902 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-04-07T16:17:17.902 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-04-07T16:17:17.903 INFO:tasks.rados.rados.0.smithi100.stdout:199: finishing write tid 4 to smithi10066867-31 2024-04-07T16:17:17.903 INFO:tasks.rados.rados.0.smithi100.stdout:199: finishing write tid 5 to smithi10066867-31 2024-04-07T16:17:17.903 INFO:tasks.rados.rados.0.smithi100.stdout:199: oid 31 updating version 0 to 22 2024-04-07T16:17:17.903 INFO:tasks.rados.rados.0.smithi100.stdout:199: oid 31 version 22 is already newer than 21 2024-04-07T16:17:17.903 INFO:tasks.rados.rados.0.smithi100.stdout:199: oid 31 version 22 is already newer than 19 2024-04-07T16:17:17.903 INFO:tasks.rados.rados.0.smithi100.stdout:199: oid 31 version 22 is already newer than 20 2024-04-07T16:17:17.903 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 31 v 22 (ObjNum 81 snap 15 seq_num 81) dirty exists 2024-04-07T16:17:17.903 INFO:tasks.rados.rados.0.smithi100.stdout:199: left oid 31 (ObjNum 81 snap 15 seq_num 81) 2024-04-07T16:17:17.903 INFO:tasks.rados.rados.0.smithi100.stdout:198: done (3 left) 2024-04-07T16:17:17.903 INFO:tasks.rados.rados.0.smithi100.stdout:199: done (2 left) 2024-04-07T16:17:17.903 INFO:tasks.rados.rados.0.smithi100.stdout:200: done (1 left) 2024-04-07T16:17:17.904 INFO:tasks.rados.rados.0.smithi100.stdout:201: done (0 left) 2024-04-07T16:17:17.904 INFO:tasks.rados.rados.0.smithi100.stdout:202: read oid 18 snap -1 2024-04-07T16:17:17.904 INFO:tasks.rados.rados.0.smithi100.stdout:202: expect (ObjNum 45 snap 0 seq_num 45) 2024-04-07T16:17:17.904 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 2 current snap is 15 2024-04-07T16:17:17.904 INFO:tasks.rados.rados.0.smithi100.stdout:203: seq_num 83 ranges {1236992=598016,1835008=499712,2334720=303104} 2024-04-07T16:17:17.915 INFO:tasks.rados.rados.0.smithi100.stdout:203: writing smithi10066867-2 from 1236992 to 1835008 tid 1 2024-04-07T16:17:17.918 INFO:tasks.rados.rados.0.smithi100.stdout:203: writing smithi10066867-2 from 1835008 to 2334720 tid 2 2024-04-07T16:17:17.920 INFO:tasks.rados.rados.0.smithi100.stdout:203: writing smithi10066867-2 from 2334720 to 2637824 tid 3 2024-04-07T16:17:17.920 INFO:tasks.rados.rados.0.smithi100.stdout:204: copy_from oid 14 from oid 5 current snap is 15 2024-04-07T16:17:17.921 INFO:tasks.rados.rados.0.smithi100.stdout:202: expect (ObjNum 45 snap 0 seq_num 45) 2024-04-07T16:17:17.946 INFO:tasks.rados.rados.0.smithi100.stdout:203: finishing write tid 1 to smithi10066867-2 2024-04-07T16:17:17.946 INFO:tasks.rados.rados.0.smithi100.stdout:203: finishing write tid 2 to smithi10066867-2 2024-04-07T16:17:17.946 INFO:tasks.rados.rados.0.smithi100.stdout:203: finishing write tid 3 to smithi10066867-2 2024-04-07T16:17:17.946 INFO:tasks.rados.rados.0.smithi100.stdout:203: finishing write tid 4 to smithi10066867-2 2024-04-07T16:17:17.946 INFO:tasks.rados.rados.0.smithi100.stdout:203: finishing write tid 5 to smithi10066867-2 2024-04-07T16:17:17.946 INFO:tasks.rados.rados.0.smithi100.stdout:203: oid 2 updating version 0 to 53 2024-04-07T16:17:17.946 INFO:tasks.rados.rados.0.smithi100.stdout:203: oid 2 version 53 is already newer than 51 2024-04-07T16:17:17.946 INFO:tasks.rados.rados.0.smithi100.stdout:203: oid 2 version 53 is already newer than 52 2024-04-07T16:17:17.946 INFO:tasks.rados.rados.0.smithi100.stdout:203: oid 2 version 53 is already newer than 50 2024-04-07T16:17:17.946 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 2 v 53 (ObjNum 83 snap 15 seq_num 83) dirty exists 2024-04-07T16:17:17.946 INFO:tasks.rados.rados.0.smithi100.stdout:203: left oid 2 (ObjNum 83 snap 15 seq_num 83) 2024-04-07T16:17:17.947 INFO:tasks.rados.rados.0.smithi100.stdout:204: finishing copy_from racing read to smithi10066867-14 2024-04-07T16:17:17.947 INFO:tasks.rados.rados.0.smithi100.stdout:204: finishing copy_from to smithi10066867-14 2024-04-07T16:17:17.947 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 14 v 2 (ObjNum 28 snap 0 seq_num 28) dirty exists 2024-04-07T16:17:17.947 INFO:tasks.rados.rados.0.smithi100.stdout:202: done (2 left) 2024-04-07T16:17:17.947 INFO:tasks.rados.rados.0.smithi100.stdout:203: done (1 left) 2024-04-07T16:17:17.947 INFO:tasks.rados.rados.0.smithi100.stdout:204: done (0 left) 2024-04-07T16:17:17.947 INFO:tasks.rados.rados.0.smithi100.stdout:205: snap_create 2024-04-07T16:17:18.105 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:17:17 smithi144 bash[23522]: cluster 2024-04-07T16:17:16.700294+0000 mgr.y (mgr.14553) 192 : cluster 0 pgmap v171: 121 pgs: 121 active+clean; 37 MiB data, 293 MiB used, 715 GiB / 715 GiB avail; 2.4 MiB/s rd, 2.0 MiB/s wr, 17 op/s 2024-04-07T16:17:18.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:17:17 smithi144 bash[23522]: cluster 2024-04-07T16:17:16.859004+0000 mon.a (mon.0) 920 : cluster 0 osdmap e102: 8 total, 8 up, 8 in 2024-04-07T16:17:18.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:17:17 smithi144 bash[23522]: audit 2024-04-07T16:17:17.108272+0000 mgr.y (mgr.14553) 193 : audit 0 from='client.24947 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:17:18.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:17:17 smithi144 bash[23522]: audit 2024-04-07T16:17:17.467607+0000 mgr.y (mgr.14553) 194 : audit 0 from='client.24953 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:17:18.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:17:17 smithi144 bash[23522]: cluster 2024-04-07T16:17:17.831503+0000 mon.a (mon.0) 921 : cluster 0 osdmap e103: 8 total, 8 up, 8 in 2024-04-07T16:17:18.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:17:17 smithi100 bash[21346]: cluster 2024-04-07T16:17:16.700294+0000 mgr.y (mgr.14553) 192 : cluster 0 pgmap v171: 121 pgs: 121 active+clean; 37 MiB data, 293 MiB used, 715 GiB / 715 GiB avail; 2.4 MiB/s rd, 2.0 MiB/s wr, 17 op/s 2024-04-07T16:17:18.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:17:17 smithi100 bash[21346]: cluster 2024-04-07T16:17:16.859004+0000 mon.a (mon.0) 920 : cluster 0 osdmap e102: 8 total, 8 up, 8 in 2024-04-07T16:17:18.188 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:17:17 smithi100 bash[21346]: audit 2024-04-07T16:17:17.108272+0000 mgr.y (mgr.14553) 193 : audit 0 from='client.24947 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:17:18.188 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:17:17 smithi100 bash[21346]: audit 2024-04-07T16:17:17.467607+0000 mgr.y (mgr.14553) 194 : audit 0 from='client.24953 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:17:18.188 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:17:17 smithi100 bash[21346]: cluster 2024-04-07T16:17:17.831503+0000 mon.a (mon.0) 921 : cluster 0 osdmap e103: 8 total, 8 up, 8 in 2024-04-07T16:17:18.188 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:17:17 smithi100 bash[31660]: cluster 2024-04-07T16:17:16.700294+0000 mgr.y (mgr.14553) 192 : cluster 0 pgmap v171: 121 pgs: 121 active+clean; 37 MiB data, 293 MiB used, 715 GiB / 715 GiB avail; 2.4 MiB/s rd, 2.0 MiB/s wr, 17 op/s 2024-04-07T16:17:18.188 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:17:17 smithi100 bash[31660]: cluster 2024-04-07T16:17:16.859004+0000 mon.a (mon.0) 920 : cluster 0 osdmap e102: 8 total, 8 up, 8 in 2024-04-07T16:17:18.188 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:17:17 smithi100 bash[31660]: audit 2024-04-07T16:17:17.108272+0000 mgr.y (mgr.14553) 193 : audit 0 from='client.24947 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:17:18.188 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:17:17 smithi100 bash[31660]: audit 2024-04-07T16:17:17.467607+0000 mgr.y (mgr.14553) 194 : audit 0 from='client.24953 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:17:18.188 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:17:17 smithi100 bash[31660]: cluster 2024-04-07T16:17:17.831503+0000 mon.a (mon.0) 921 : cluster 0 osdmap e103: 8 total, 8 up, 8 in 2024-04-07T16:17:18.854 INFO:tasks.rados.rados.0.smithi100.stdout:205: done (0 left) 2024-04-07T16:17:18.854 INFO:tasks.rados.rados.0.smithi100.stdout:206: delete oid 31 current snap is 16 2024-04-07T16:17:18.861 INFO:tasks.rados.rados.0.smithi100.stdout:206: done (0 left) 2024-04-07T16:17:18.862 INFO:tasks.rados.rados.0.smithi100.stdout:207: delete oid 26 current snap is 16 2024-04-07T16:17:18.868 INFO:tasks.rados.rados.0.smithi100.stdout:207: done (0 left) 2024-04-07T16:17:18.868 INFO:tasks.rados.rados.0.smithi100.stdout:208: copy_from oid 20 from oid 42 current snap is 16 2024-04-07T16:17:18.868 INFO:tasks.rados.rados.0.smithi100.stdout:209: delete oid 5 current snap is 16 2024-04-07T16:17:18.873 INFO:tasks.rados.rados.0.smithi100.stdout:209: done (1 left) 2024-04-07T16:17:18.873 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 13 current snap is 16 2024-04-07T16:17:18.873 INFO:tasks.rados.rados.0.smithi100.stdout:210: seq_num 87 ranges {1417216=663552,2080768=491520,2572288=311296} 2024-04-07T16:17:18.885 INFO:tasks.rados.rados.0.smithi100.stdout:210: writing smithi10066867-13 from 1417216 to 2080768 tid 1 2024-04-07T16:17:18.888 INFO:tasks.rados.rados.0.smithi100.stdout:210: writing smithi10066867-13 from 2080768 to 2572288 tid 2 2024-04-07T16:17:18.892 INFO:tasks.rados.rados.0.smithi100.stdout:210: writing smithi10066867-13 from 2572288 to 2883584 tid 3 2024-04-07T16:17:18.892 INFO:tasks.rados.rados.0.smithi100.stdout:211: read oid 19 snap -1 2024-04-07T16:17:18.892 INFO:tasks.rados.rados.0.smithi100.stdout:211: expect (ObjNum 75 snap 11 seq_num 75) 2024-04-07T16:17:18.892 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 21 current snap is 16 2024-04-07T16:17:18.892 INFO:tasks.rados.rados.0.smithi100.stdout:212: seq_num 88 ranges {475136=679936,1155072=778240,1933312=24576} 2024-04-07T16:17:18.904 INFO:tasks.rados.rados.0.smithi100.stdout:212: writing smithi10066867-21 from 475136 to 1155072 tid 1 2024-04-07T16:17:18.910 INFO:tasks.rados.rados.0.smithi100.stdout:212: writing smithi10066867-21 from 1155072 to 1933312 tid 2 2024-04-07T16:17:18.911 INFO:tasks.rados.rados.0.smithi100.stdout:212: writing smithi10066867-21 from 1933312 to 1957888 tid 3 2024-04-07T16:17:18.911 INFO:tasks.rados.rados.0.smithi100.stdout:213: delete oid 22 current snap is 16 2024-04-07T16:17:18.911 INFO:tasks.rados.rados.0.smithi100.stdout:208: finishing copy_from racing read to smithi10066867-20 2024-04-07T16:17:18.911 INFO:tasks.rados.rados.0.smithi100.stdout:208: finishing copy_from to smithi10066867-20 2024-04-07T16:17:18.911 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 20 v 10 (ObjNum 41 snap 0 seq_num 41) dirty exists 2024-04-07T16:17:18.911 INFO:tasks.rados.rados.0.smithi100.stdout:210: finishing write tid 1 to smithi10066867-13 2024-04-07T16:17:18.911 INFO:tasks.rados.rados.0.smithi100.stdout:210: finishing write tid 2 to smithi10066867-13 2024-04-07T16:17:18.911 INFO:tasks.rados.rados.0.smithi100.stdout:210: finishing write tid 3 to smithi10066867-13 2024-04-07T16:17:18.911 INFO:tasks.rados.rados.0.smithi100.stdout:210: finishing write tid 4 to smithi10066867-13 2024-04-07T16:17:18.912 INFO:tasks.rados.rados.0.smithi100.stdout:210: finishing write tid 5 to smithi10066867-13 2024-04-07T16:17:18.912 INFO:tasks.rados.rados.0.smithi100.stdout:210: oid 13 updating version 0 to 25 2024-04-07T16:17:18.912 INFO:tasks.rados.rados.0.smithi100.stdout:210: oid 13 version 25 is already newer than 23 2024-04-07T16:17:18.912 INFO:tasks.rados.rados.0.smithi100.stdout:210: oid 13 version 25 is already newer than 24 2024-04-07T16:17:18.912 INFO:tasks.rados.rados.0.smithi100.stdout:210: oid 13 version 25 is already newer than 22 2024-04-07T16:17:18.912 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 13 v 25 (ObjNum 87 snap 16 seq_num 87) dirty exists 2024-04-07T16:17:18.912 INFO:tasks.rados.rados.0.smithi100.stdout:210: left oid 13 (ObjNum 87 snap 16 seq_num 87) 2024-04-07T16:17:18.912 INFO:tasks.rados.rados.0.smithi100.stdout:211: expect (ObjNum 75 snap 11 seq_num 75) 2024-04-07T16:17:19.001 INFO:tasks.rados.rados.0.smithi100.stdout:212: finishing write tid 1 to smithi10066867-21 2024-04-07T16:17:19.001 INFO:tasks.rados.rados.0.smithi100.stdout:212: finishing write tid 2 to smithi10066867-21 2024-04-07T16:17:19.001 INFO:tasks.rados.rados.0.smithi100.stdout:212: finishing write tid 3 to smithi10066867-21 2024-04-07T16:17:19.001 INFO:tasks.rados.rados.0.smithi100.stdout:212: finishing write tid 4 to smithi10066867-21 2024-04-07T16:17:19.001 INFO:tasks.rados.rados.0.smithi100.stdout:212: finishing write tid 5 to smithi10066867-21 2024-04-07T16:17:19.001 INFO:tasks.rados.rados.0.smithi100.stdout:212: oid 21 updating version 0 to 17 2024-04-07T16:17:19.001 INFO:tasks.rados.rados.0.smithi100.stdout:212: oid 21 updating version 17 to 18 2024-04-07T16:17:19.001 INFO:tasks.rados.rados.0.smithi100.stdout:212: oid 21 updating version 18 to 20 2024-04-07T16:17:19.001 INFO:tasks.rados.rados.0.smithi100.stdout:212: oid 21 version 20 is already newer than 19 2024-04-07T16:17:19.001 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 21 v 20 (ObjNum 88 snap 16 seq_num 88) dirty exists 2024-04-07T16:17:19.002 INFO:tasks.rados.rados.0.smithi100.stdout:212: left oid 21 (ObjNum 88 snap 16 seq_num 88) 2024-04-07T16:17:19.002 INFO:tasks.rados.rados.0.smithi100.stdout:208: done (4 left) 2024-04-07T16:17:19.002 INFO:tasks.rados.rados.0.smithi100.stdout:210: done (3 left) 2024-04-07T16:17:19.002 INFO:tasks.rados.rados.0.smithi100.stdout:211: done (2 left) 2024-04-07T16:17:19.002 INFO:tasks.rados.rados.0.smithi100.stdout:212: done (1 left) 2024-04-07T16:17:19.002 INFO:tasks.rados.rados.0.smithi100.stdout:213: done (0 left) 2024-04-07T16:17:19.002 INFO:tasks.rados.rados.0.smithi100.stdout:214: read oid 38 snap -1 2024-04-07T16:17:19.002 INFO:tasks.rados.rados.0.smithi100.stdout:214: expect (ObjNum 28 snap 0 seq_num 28) 2024-04-07T16:17:19.002 INFO:tasks.rados.rados.0.smithi100.stdout:215: copy_from oid 41 from oid 6 current snap is 16 2024-04-07T16:17:19.002 INFO:tasks.rados.rados.0.smithi100.stdout:216: copy_from oid 30 from oid 19 current snap is 16 2024-04-07T16:17:19.002 INFO:tasks.rados.rados.0.smithi100.stdout:217: read oid 14 snap 15 2024-04-07T16:17:19.002 INFO:tasks.rados.rados.0.smithi100.stdout:217: expect (ObjNum 28 snap 0 seq_num 28) 2024-04-07T16:17:19.002 INFO:tasks.rados.rados.0.smithi100.stdout:218: delete oid 4 current snap is 16 2024-04-07T16:17:19.004 INFO:tasks.rados.rados.0.smithi100.stdout:218: done (4 left) 2024-04-07T16:17:19.004 INFO:tasks.rados.rados.0.smithi100.stdout:219: rollback oid 9 current snap is 16 2024-04-07T16:17:19.005 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 9 to 13 2024-04-07T16:17:19.005 INFO:tasks.rados.rados.0.smithi100.stdout:220: delete oid 20 current snap is 16 2024-04-07T16:17:19.006 INFO:tasks.rados.rados.0.smithi100.stdout:215: finishing copy_from racing read to smithi10066867-41 2024-04-07T16:17:19.008 INFO:tasks.rados.rados.0.smithi100.stdout:219: finishing rollback tid 0 to smithi10066867-9 2024-04-07T16:17:19.008 INFO:tasks.rados.rados.0.smithi100.stdout:214: expect (ObjNum 28 snap 0 seq_num 28) 2024-04-07T16:17:19.024 INFO:tasks.rados.rados.0.smithi100.stdout:219: finishing rollback tid 1 to smithi10066867-9 2024-04-07T16:17:19.024 INFO:tasks.rados.rados.0.smithi100.stdout:219: finishing rollback tid 2 to smithi10066867-9 2024-04-07T16:17:19.024 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 9 v 31 (ObjNum 72 snap 11 seq_num 72) dirty exists 2024-04-07T16:17:19.024 INFO:tasks.rados.rados.0.smithi100.stdout:217: expect (ObjNum 28 snap 0 seq_num 28) 2024-04-07T16:17:19.038 INFO:tasks.rados.rados.0.smithi100.stdout:215: finishing copy_from to smithi10066867-41 2024-04-07T16:17:19.038 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 41 v 48 (ObjNum 129 snap 0 seq_num 2699842440) dirty exists 2024-04-07T16:17:19.038 INFO:tasks.rados.rados.0.smithi100.stdout:216: finishing copy_from racing read to smithi10066867-30 2024-04-07T16:17:19.038 INFO:tasks.rados.rados.0.smithi100.stdout:216: finishing copy_from to smithi10066867-30 2024-04-07T16:17:19.039 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 30 v 52 (ObjNum 75 snap 11 seq_num 75) dirty exists 2024-04-07T16:17:19.039 INFO:tasks.rados.rados.0.smithi100.stdout:214: done (5 left) 2024-04-07T16:17:19.039 INFO:tasks.rados.rados.0.smithi100.stdout:215: done (4 left) 2024-04-07T16:17:19.039 INFO:tasks.rados.rados.0.smithi100.stdout:216: done (3 left) 2024-04-07T16:17:19.039 INFO:tasks.rados.rados.0.smithi100.stdout:217: done (2 left) 2024-04-07T16:17:19.039 INFO:tasks.rados.rados.0.smithi100.stdout:219: done (1 left) 2024-04-07T16:17:19.039 INFO:tasks.rados.rados.0.smithi100.stdout:220: done (0 left) 2024-04-07T16:17:19.039 INFO:tasks.rados.rados.0.smithi100.stdout:221: copy_from oid 44 from oid 24 current snap is 16 2024-04-07T16:17:19.039 INFO:tasks.rados.rados.0.smithi100.stdout:222: rollback oid 13 current snap is 16 2024-04-07T16:17:19.039 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 13 to 12 2024-04-07T16:17:19.039 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 17 current snap is 16 2024-04-07T16:17:19.039 INFO:tasks.rados.rados.0.smithi100.stdout:223: seq_num 92 ranges {1196032=745472,1941504=761856,2703360=40960} 2024-04-07T16:17:19.051 INFO:tasks.rados.rados.0.smithi100.stdout:223: writing smithi10066867-17 from 1196032 to 1941504 tid 1 2024-04-07T16:17:19.056 INFO:tasks.rados.rados.0.smithi100.stdout:223: writing smithi10066867-17 from 1941504 to 2703360 tid 2 2024-04-07T16:17:19.056 INFO:tasks.rados.rados.0.smithi100.stdout:223: writing smithi10066867-17 from 2703360 to 2744320 tid 3 2024-04-07T16:17:19.056 INFO:tasks.rados.rados.0.smithi100.stdout:224: rollback oid 35 current snap is 16 2024-04-07T16:17:19.056 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 35 to 13 2024-04-07T16:17:19.057 INFO:tasks.rados.rados.0.smithi100.stdout:222: finishing rollback tid 0 to smithi10066867-13 2024-04-07T16:17:19.057 INFO:tasks.rados.rados.0.smithi100.stdout:222: finishing rollback tid 1 to smithi10066867-13 2024-04-07T16:17:19.057 INFO:tasks.rados.rados.0.smithi100.stdout:222: finishing rollback tid 2 to smithi10066867-13 2024-04-07T16:17:19.057 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 13 v 28 (ObjNum 72 snap 11 seq_num 72) dirty exists 2024-04-07T16:17:19.057 INFO:tasks.rados.rados.0.smithi100.stdout:221: finishing copy_from racing read to smithi10066867-44 2024-04-07T16:17:19.057 INFO:tasks.rados.rados.0.smithi100.stdout:221: finishing copy_from to smithi10066867-44 2024-04-07T16:17:19.057 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 44 v 2 (ObjNum 28 snap 0 seq_num 28) dirty exists 2024-04-07T16:17:19.057 INFO:tasks.rados.rados.0.smithi100.stdout:223: finishing write tid 1 to smithi10066867-17 2024-04-07T16:17:19.057 INFO:tasks.rados.rados.0.smithi100.stdout:221: done (3 left) 2024-04-07T16:17:19.057 INFO:tasks.rados.rados.0.smithi100.stdout:222: done (2 left) 2024-04-07T16:17:19.057 INFO:tasks.rados.rados.0.smithi100.stdout:225: snap_remove snap 15 2024-04-07T16:17:19.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:17:18 smithi100 bash[21346]: audit 2024-04-07T16:17:17.916008+0000 mon.a (mon.0) 922 : audit 0 from='client.? 172.21.15.100:0/2545538431' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:17:19.188 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:17:18 smithi100 bash[21346]: audit 2024-04-07T16:17:18.298697+0000 mon.b (mon.1) 241 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:17:19.188 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:17:18 smithi100 bash[21346]: audit 2024-04-07T16:17:18.299544+0000 mon.a (mon.0) 923 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:17:19.188 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:17:18 smithi100 bash[21346]: cephadm 2024-04-07T16:17:18.300733+0000 mgr.y (mgr.14553) 195 : cephadm 1 Upgrade: Target is version 19.0.0-2785-gadce92d0 (unknown) 2024-04-07T16:17:19.188 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:17:18 smithi100 bash[21346]: cephadm 2024-04-07T16:17:18.300840+0000 mgr.y (mgr.14553) 196 : cephadm 1 Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:64f4a2ca2a6cc5bd0c05ad4055626f356495ea7dcf7cb712c8ecd6fbc1381562, digests ['quay.ceph.io/ceph-ci/ceph@sha256:64f4a2ca2a6cc5bd0c05ad4055626f356495ea7dcf7cb712c8ecd6fbc1381562'] 2024-04-07T16:17:19.188 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:17:18 smithi100 bash[21346]: audit 2024-04-07T16:17:18.315983+0000 mon.a (mon.0) 924 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:17:19.188 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:17:18 smithi100 bash[21346]: cephadm 2024-04-07T16:17:18.318167+0000 mgr.y (mgr.14553) 197 : cephadm 1 Upgrade: Need to upgrade myself (mgr.y) 2024-04-07T16:17:19.188 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:17:18 smithi100 bash[31660]: audit 2024-04-07T16:17:17.916008+0000 mon.a (mon.0) 922 : audit 0 from='client.? 172.21.15.100:0/2545538431' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:17:19.188 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:17:18 smithi100 bash[31660]: audit 2024-04-07T16:17:18.298697+0000 mon.b (mon.1) 241 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:17:19.188 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:17:18 smithi100 bash[31660]: audit 2024-04-07T16:17:18.299544+0000 mon.a (mon.0) 923 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:17:19.188 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:17:18 smithi100 bash[31660]: cephadm 2024-04-07T16:17:18.300733+0000 mgr.y (mgr.14553) 195 : cephadm 1 Upgrade: Target is version 19.0.0-2785-gadce92d0 (unknown) 2024-04-07T16:17:19.189 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:17:18 smithi100 bash[31660]: cephadm 2024-04-07T16:17:18.300840+0000 mgr.y (mgr.14553) 196 : cephadm 1 Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:64f4a2ca2a6cc5bd0c05ad4055626f356495ea7dcf7cb712c8ecd6fbc1381562, digests ['quay.ceph.io/ceph-ci/ceph@sha256:64f4a2ca2a6cc5bd0c05ad4055626f356495ea7dcf7cb712c8ecd6fbc1381562'] 2024-04-07T16:17:19.189 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:17:18 smithi100 bash[31660]: audit 2024-04-07T16:17:18.315983+0000 mon.a (mon.0) 924 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:17:19.189 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:17:18 smithi100 bash[31660]: cephadm 2024-04-07T16:17:18.318167+0000 mgr.y (mgr.14553) 197 : cephadm 1 Upgrade: Need to upgrade myself (mgr.y) 2024-04-07T16:17:19.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:17:18 smithi144 bash[23522]: audit 2024-04-07T16:17:17.916008+0000 mon.a (mon.0) 922 : audit 0 from='client.? 172.21.15.100:0/2545538431' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:17:19.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:17:18 smithi144 bash[23522]: audit 2024-04-07T16:17:18.298697+0000 mon.b (mon.1) 241 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:17:19.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:17:18 smithi144 bash[23522]: audit 2024-04-07T16:17:18.299544+0000 mon.a (mon.0) 923 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:17:19.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:17:18 smithi144 bash[23522]: cephadm 2024-04-07T16:17:18.300733+0000 mgr.y (mgr.14553) 195 : cephadm 1 Upgrade: Target is version 19.0.0-2785-gadce92d0 (unknown) 2024-04-07T16:17:19.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:17:18 smithi144 bash[23522]: cephadm 2024-04-07T16:17:18.300840+0000 mgr.y (mgr.14553) 196 : cephadm 1 Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:64f4a2ca2a6cc5bd0c05ad4055626f356495ea7dcf7cb712c8ecd6fbc1381562, digests ['quay.ceph.io/ceph-ci/ceph@sha256:64f4a2ca2a6cc5bd0c05ad4055626f356495ea7dcf7cb712c8ecd6fbc1381562'] 2024-04-07T16:17:19.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:17:18 smithi144 bash[23522]: audit 2024-04-07T16:17:18.315983+0000 mon.a (mon.0) 924 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:17:19.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:17:18 smithi144 bash[23522]: cephadm 2024-04-07T16:17:18.318167+0000 mgr.y (mgr.14553) 197 : cephadm 1 Upgrade: Need to upgrade myself (mgr.y) 2024-04-07T16:17:19.875 INFO:tasks.rados.rados.0.smithi100.stdout:225: done (2 left) 2024-04-07T16:17:19.875 INFO:tasks.rados.rados.0.smithi100.stdout:226: snap_remove snap 12 2024-04-07T16:17:20.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:17:19 smithi100 bash[21346]: cephadm 2024-04-07T16:17:18.684827+0000 mgr.y (mgr.14553) 198 : cephadm 1 Upgrade: Pulling quay.ceph.io/ceph-ci/ceph@sha256:64f4a2ca2a6cc5bd0c05ad4055626f356495ea7dcf7cb712c8ecd6fbc1381562 on smithi144 2024-04-07T16:17:20.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:17:19 smithi100 bash[21346]: cluster 2024-04-07T16:17:18.700970+0000 mgr.y (mgr.14553) 199 : cluster 0 pgmap v174: 121 pgs: 121 active+clean; 37 MiB data, 293 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:17:20.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:17:19 smithi100 bash[21346]: cluster 2024-04-07T16:17:18.868073+0000 mon.a (mon.0) 925 : cluster 0 osdmap e104: 8 total, 8 up, 8 in 2024-04-07T16:17:20.188 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:17:19 smithi100 bash[31660]: cephadm 2024-04-07T16:17:18.684827+0000 mgr.y (mgr.14553) 198 : cephadm 1 Upgrade: Pulling quay.ceph.io/ceph-ci/ceph@sha256:64f4a2ca2a6cc5bd0c05ad4055626f356495ea7dcf7cb712c8ecd6fbc1381562 on smithi144 2024-04-07T16:17:20.188 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:17:19 smithi100 bash[31660]: cluster 2024-04-07T16:17:18.700970+0000 mgr.y (mgr.14553) 199 : cluster 0 pgmap v174: 121 pgs: 121 active+clean; 37 MiB data, 293 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:17:20.188 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:17:19 smithi100 bash[31660]: cluster 2024-04-07T16:17:18.868073+0000 mon.a (mon.0) 925 : cluster 0 osdmap e104: 8 total, 8 up, 8 in 2024-04-07T16:17:20.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:17:19 smithi144 bash[23522]: cephadm 2024-04-07T16:17:18.684827+0000 mgr.y (mgr.14553) 198 : cephadm 1 Upgrade: Pulling quay.ceph.io/ceph-ci/ceph@sha256:64f4a2ca2a6cc5bd0c05ad4055626f356495ea7dcf7cb712c8ecd6fbc1381562 on smithi144 2024-04-07T16:17:20.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:17:19 smithi144 bash[23522]: cluster 2024-04-07T16:17:18.700970+0000 mgr.y (mgr.14553) 199 : cluster 0 pgmap v174: 121 pgs: 121 active+clean; 37 MiB data, 293 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:17:20.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:17:19 smithi144 bash[23522]: cluster 2024-04-07T16:17:18.868073+0000 mon.a (mon.0) 925 : cluster 0 osdmap e104: 8 total, 8 up, 8 in 2024-04-07T16:17:20.886 INFO:tasks.rados.rados.0.smithi100.stdout:226: done (2 left) 2024-04-07T16:17:20.886 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 24 current snap is 16 2024-04-07T16:17:20.886 INFO:tasks.rados.rados.0.smithi100.stdout:227: seq_num 93 ranges {475136=761856,1236992=647168,1884160=155648} 2024-04-07T16:17:20.904 INFO:tasks.rados.rados.0.smithi100.stdout:227: writing smithi10066867-24 from 475136 to 1236992 tid 1 2024-04-07T16:17:20.915 INFO:tasks.rados.rados.0.smithi100.stdout:227: writing smithi10066867-24 from 1236992 to 1884160 tid 2 2024-04-07T16:17:20.917 INFO:tasks.rados.rados.0.smithi100.stdout:227: writing smithi10066867-24 from 1884160 to 2039808 tid 3 2024-04-07T16:17:20.917 INFO:tasks.rados.rados.0.smithi100.stdout:228: copy_from oid 18 from oid 39 current snap is 16 2024-04-07T16:17:20.918 INFO:tasks.rados.rados.0.smithi100.stdout:229: delete oid 2 current snap is 16 2024-04-07T16:17:20.918 INFO:tasks.rados.rados.0.smithi100.stdout:223: finishing write tid 2 to smithi10066867-17 2024-04-07T16:17:20.918 INFO:tasks.rados.rados.0.smithi100.stdout:224: finishing rollback tid 0 to smithi10066867-35 2024-04-07T16:17:20.918 INFO:tasks.rados.rados.0.smithi100.stdout:223: finishing write tid 3 to smithi10066867-17 2024-04-07T16:17:20.918 INFO:tasks.rados.rados.0.smithi100.stdout:224: finishing rollback tid 1 to smithi10066867-35 2024-04-07T16:17:20.918 INFO:tasks.rados.rados.0.smithi100.stdout:223: finishing write tid 4 to smithi10066867-17 2024-04-07T16:17:20.918 INFO:tasks.rados.rados.0.smithi100.stdout:224: finishing rollback tid 2 to smithi10066867-35 2024-04-07T16:17:20.918 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 35 v 28 (ObjNum 4 snap 0 seq_num 4) dirty exists 2024-04-07T16:17:20.918 INFO:tasks.rados.rados.0.smithi100.stdout:223: finishing write tid 5 to smithi10066867-17 2024-04-07T16:17:20.918 INFO:tasks.rados.rados.0.smithi100.stdout:223: oid 17 updating version 0 to 58 2024-04-07T16:17:20.919 INFO:tasks.rados.rados.0.smithi100.stdout:223: oid 17 updating version 58 to 59 2024-04-07T16:17:20.919 INFO:tasks.rados.rados.0.smithi100.stdout:223: oid 17 updating version 59 to 60 2024-04-07T16:17:20.919 INFO:tasks.rados.rados.0.smithi100.stdout:223: oid 17 updating version 60 to 61 2024-04-07T16:17:20.919 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 17 v 61 (ObjNum 92 snap 16 seq_num 92) dirty exists 2024-04-07T16:17:20.919 INFO:tasks.rados.rados.0.smithi100.stdout:223: left oid 17 (ObjNum 92 snap 16 seq_num 92) 2024-04-07T16:17:20.926 INFO:tasks.rados.rados.0.smithi100.stdout:223: done (4 left) 2024-04-07T16:17:20.926 INFO:tasks.rados.rados.0.smithi100.stdout:224: done (3 left) 2024-04-07T16:17:20.926 INFO:tasks.rados.rados.0.smithi100.stdout:229: done (2 left) 2024-04-07T16:17:20.927 INFO:tasks.rados.rados.0.smithi100.stdout:230: delete oid 41 current snap is 16 2024-04-07T16:17:20.930 INFO:tasks.rados.rados.0.smithi100.stdout:227: finishing write tid 1 to smithi10066867-24 2024-04-07T16:17:20.931 INFO:tasks.rados.rados.0.smithi100.stdout:230: done (2 left) 2024-04-07T16:17:20.931 INFO:tasks.rados.rados.0.smithi100.stdout:231: setattr oid 8 current snap is 16 2024-04-07T16:17:20.931 INFO:tasks.rados.rados.0.smithi100.stdout:227: finishing write tid 2 to smithi10066867-24 2024-04-07T16:17:20.931 INFO:tasks.rados.rados.0.smithi100.stdout:227: finishing write tid 3 to smithi10066867-24 2024-04-07T16:17:20.932 INFO:tasks.rados.rados.0.smithi100.stdout:227: finishing write tid 4 to smithi10066867-24 2024-04-07T16:17:20.934 INFO:tasks.rados.rados.0.smithi100.stdout:227: finishing write tid 5 to smithi10066867-24 2024-04-07T16:17:20.934 INFO:tasks.rados.rados.0.smithi100.stdout:227: oid 24 updating version 0 to 66 2024-04-07T16:17:20.934 INFO:tasks.rados.rados.0.smithi100.stdout:227: oid 24 updating version 66 to 68 2024-04-07T16:17:20.934 INFO:tasks.rados.rados.0.smithi100.stdout:227: oid 24 version 68 is already newer than 65 2024-04-07T16:17:20.934 INFO:tasks.rados.rados.0.smithi100.stdout:227: oid 24 version 68 is already newer than 67 2024-04-07T16:17:20.934 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 24 v 68 (ObjNum 93 snap 16 seq_num 93) dirty exists 2024-04-07T16:17:20.934 INFO:tasks.rados.rados.0.smithi100.stdout:227: left oid 24 (ObjNum 93 snap 16 seq_num 93) 2024-04-07T16:17:20.936 INFO:tasks.rados.rados.0.smithi100.stdout:227: done (2 left) 2024-04-07T16:17:20.936 INFO:tasks.rados.rados.0.smithi100.stdout:232: snap_remove snap 13 2024-04-07T16:17:21.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:17:20 smithi100 bash[21346]: cluster 2024-04-07T16:17:19.888218+0000 mon.a (mon.0) 926 : cluster 0 osdmap e105: 8 total, 8 up, 8 in 2024-04-07T16:17:21.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:17:20 smithi100 bash[31660]: cluster 2024-04-07T16:17:19.888218+0000 mon.a (mon.0) 926 : cluster 0 osdmap e105: 8 total, 8 up, 8 in 2024-04-07T16:17:21.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:17:20 smithi144 bash[23522]: cluster 2024-04-07T16:17:19.888218+0000 mon.a (mon.0) 926 : cluster 0 osdmap e105: 8 total, 8 up, 8 in 2024-04-07T16:17:21.894 INFO:tasks.rados.rados.0.smithi100.stdout:232: done (2 left) 2024-04-07T16:17:21.894 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 48 current snap is 16 2024-04-07T16:17:21.894 INFO:tasks.rados.rados.0.smithi100.stdout:228: finishing copy_from racing read to smithi10066867-18 2024-04-07T16:17:21.894 INFO:tasks.rados.rados.0.smithi100.stdout:228: finishing copy_from to smithi10066867-18 2024-04-07T16:17:21.894 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 18 v 18 (ObjNum 57 snap 4 seq_num 57) dirty exists 2024-04-07T16:17:21.894 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 8 v 34 (ObjNum 7 snap 0 seq_num 7) dirty exists 2024-04-07T16:17:21.895 INFO:tasks.rados.rados.0.smithi100.stdout:233: seq_num 96 ranges {794624=409600,1204224=696320,1900544=507904} 2024-04-07T16:17:21.913 INFO:tasks.rados.rados.0.smithi100.stdout:233: writing smithi10066867-48 from 794624 to 1204224 tid 1 2024-04-07T16:17:21.935 INFO:tasks.rados.rados.0.smithi100.stdout:233: writing smithi10066867-48 from 1204224 to 1900544 tid 2 2024-04-07T16:17:21.938 INFO:tasks.rados.rados.0.smithi100.stdout:233: writing smithi10066867-48 from 1900544 to 2408448 tid 3 2024-04-07T16:17:21.938 INFO:tasks.rados.rados.0.smithi100.stdout:228: done (2 left) 2024-04-07T16:17:21.938 INFO:tasks.rados.rados.0.smithi100.stdout:231: done (1 left) 2024-04-07T16:17:21.938 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 2 current snap is 16 2024-04-07T16:17:21.938 INFO:tasks.rados.rados.0.smithi100.stdout:234: seq_num 97 ranges {0=434176,434176=581632,1015808=540672,1556480=81920} 2024-04-07T16:17:21.944 INFO:tasks.rados.rados.0.smithi100.stdout:234: writing smithi10066867-2 from 0 to 434176 tid 1 2024-04-07T16:17:21.952 INFO:tasks.rados.rados.0.smithi100.stdout:234: writing smithi10066867-2 from 434176 to 1015808 tid 2 2024-04-07T16:17:21.959 INFO:tasks.rados.rados.0.smithi100.stdout:234: writing smithi10066867-2 from 1015808 to 1556480 tid 3 2024-04-07T16:17:21.961 INFO:tasks.rados.rados.0.smithi100.stdout:234: writing smithi10066867-2 from 1556480 to 1638400 tid 4 2024-04-07T16:17:21.961 INFO:tasks.rados.rados.0.smithi100.stdout:235: copy_from oid 27 from oid 26 current snap is 16 2024-04-07T16:17:21.961 INFO:tasks.rados.rados.0.smithi100.stdout:233: finishing write tid 1 to smithi10066867-48 2024-04-07T16:17:21.961 INFO:tasks.rados.rados.0.smithi100.stdout:234: finishing write tid 1 to smithi10066867-2 2024-04-07T16:17:21.961 INFO:tasks.rados.rados.0.smithi100.stdout:236: snap_create 2024-04-07T16:17:21.961 INFO:tasks.rados.rados.0.smithi100.stdout:233: finishing write tid 2 to smithi10066867-48 2024-04-07T16:17:21.961 INFO:tasks.rados.rados.0.smithi100.stdout:233: finishing write tid 3 to smithi10066867-48 2024-04-07T16:17:21.961 INFO:tasks.rados.rados.0.smithi100.stdout:233: finishing write tid 4 to smithi10066867-48 2024-04-07T16:17:21.961 INFO:tasks.rados.rados.0.smithi100.stdout:233: finishing write tid 5 to smithi10066867-48 2024-04-07T16:17:21.961 INFO:tasks.rados.rados.0.smithi100.stdout:233: oid 48 updating version 0 to 44 2024-04-07T16:17:21.961 INFO:tasks.rados.rados.0.smithi100.stdout:233: oid 48 version 44 is already newer than 38 2024-04-07T16:17:21.961 INFO:tasks.rados.rados.0.smithi100.stdout:233: oid 48 updating version 44 to 45 2024-04-07T16:17:21.962 INFO:tasks.rados.rados.0.smithi100.stdout:233: oid 48 version 45 is already newer than 43 2024-04-07T16:17:21.962 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 48 v 45 (ObjNum 96 snap 16 seq_num 96) dirty exists 2024-04-07T16:17:21.962 INFO:tasks.rados.rados.0.smithi100.stdout:233: left oid 48 (ObjNum 96 snap 16 seq_num 96) 2024-04-07T16:17:21.964 INFO:tasks.rados.rados.0.smithi100.stdout:235: finishing copy_from to smithi10066867-27 2024-04-07T16:17:21.964 INFO:tasks.rados.rados.0.smithi100.stdout:235: got expected ENOENT (src dne) 2024-04-07T16:17:21.965 INFO:tasks.rados.rados.0.smithi100.stdout:235: finishing copy_from racing read to smithi10066867-27 2024-04-07T16:17:21.967 INFO:tasks.rados.rados.0.smithi100.stdout:234: finishing write tid 2 to smithi10066867-2 2024-04-07T16:17:21.968 INFO:tasks.rados.rados.0.smithi100.stdout:234: finishing write tid 3 to smithi10066867-2 2024-04-07T16:17:21.968 INFO:tasks.rados.rados.0.smithi100.stdout:234: finishing write tid 4 to smithi10066867-2 2024-04-07T16:17:21.968 INFO:tasks.rados.rados.0.smithi100.stdout:234: finishing write tid 5 to smithi10066867-2 2024-04-07T16:17:21.971 INFO:tasks.rados.rados.0.smithi100.stdout:234: finishing write tid 6 to smithi10066867-2 2024-04-07T16:17:21.971 INFO:tasks.rados.rados.0.smithi100.stdout:234: oid 2 updating version 0 to 68 2024-04-07T16:17:21.971 INFO:tasks.rados.rados.0.smithi100.stdout:234: oid 2 updating version 68 to 70 2024-04-07T16:17:21.971 INFO:tasks.rados.rados.0.smithi100.stdout:234: oid 2 version 70 is already newer than 67 2024-04-07T16:17:21.971 INFO:tasks.rados.rados.0.smithi100.stdout:234: oid 2 version 70 is already newer than 69 2024-04-07T16:17:21.971 INFO:tasks.rados.rados.0.smithi100.stdout:234: oid 2 updating version 70 to 71 2024-04-07T16:17:21.971 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 2 v 71 (ObjNum 97 snap 16 seq_num 97) dirty exists 2024-04-07T16:17:21.971 INFO:tasks.rados.rados.0.smithi100.stdout:234: left oid 2 (ObjNum 97 snap 16 seq_num 97) 2024-04-07T16:17:22.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:17:21 smithi100 bash[21346]: cluster 2024-04-07T16:17:20.701699+0000 mgr.y (mgr.14553) 200 : cluster 0 pgmap v177: 121 pgs: 121 active+clean; 45 MiB data, 303 MiB used, 715 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.0 MiB/s wr, 17 op/s 2024-04-07T16:17:22.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:17:21 smithi100 bash[21346]: cluster 2024-04-07T16:17:20.899127+0000 mon.a (mon.0) 927 : cluster 0 osdmap e106: 8 total, 8 up, 8 in 2024-04-07T16:17:22.188 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:17:21 smithi100 bash[31660]: cluster 2024-04-07T16:17:20.701699+0000 mgr.y (mgr.14553) 200 : cluster 0 pgmap v177: 121 pgs: 121 active+clean; 45 MiB data, 303 MiB used, 715 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.0 MiB/s wr, 17 op/s 2024-04-07T16:17:22.188 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:17:21 smithi100 bash[31660]: cluster 2024-04-07T16:17:20.899127+0000 mon.a (mon.0) 927 : cluster 0 osdmap e106: 8 total, 8 up, 8 in 2024-04-07T16:17:22.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:17:21 smithi144 bash[23522]: cluster 2024-04-07T16:17:20.701699+0000 mgr.y (mgr.14553) 200 : cluster 0 pgmap v177: 121 pgs: 121 active+clean; 45 MiB data, 303 MiB used, 715 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.0 MiB/s wr, 17 op/s 2024-04-07T16:17:22.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:17:21 smithi144 bash[23522]: cluster 2024-04-07T16:17:20.899127+0000 mon.a (mon.0) 927 : cluster 0 osdmap e106: 8 total, 8 up, 8 in 2024-04-07T16:17:22.820 INFO:tasks.rados.rados.0.smithi100.stdout:233: done (3 left) 2024-04-07T16:17:22.820 INFO:tasks.rados.rados.0.smithi100.stdout:234: done (2 left) 2024-04-07T16:17:22.820 INFO:tasks.rados.rados.0.smithi100.stdout:235: done (1 left) 2024-04-07T16:17:22.820 INFO:tasks.rados.rados.0.smithi100.stdout:236: done (0 left) 2024-04-07T16:17:22.820 INFO:tasks.rados.rados.0.smithi100.stdout:237: snap_create 2024-04-07T16:17:23.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:17:22 smithi100 bash[21346]: cluster 2024-04-07T16:17:21.907829+0000 mon.a (mon.0) 928 : cluster 0 osdmap e107: 8 total, 8 up, 8 in 2024-04-07T16:17:23.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:17:22 smithi100 bash[21346]: audit 2024-04-07T16:17:22.703676+0000 mon.b (mon.1) 242 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:17:23.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:17:22 smithi100 bash[21346]: cluster 2024-04-07T16:17:22.833576+0000 mon.a (mon.0) 929 : cluster 0 osdmap e108: 8 total, 8 up, 8 in 2024-04-07T16:17:23.188 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:17:22 smithi100 bash[31660]: cluster 2024-04-07T16:17:21.907829+0000 mon.a (mon.0) 928 : cluster 0 osdmap e107: 8 total, 8 up, 8 in 2024-04-07T16:17:23.188 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:17:22 smithi100 bash[31660]: audit 2024-04-07T16:17:22.703676+0000 mon.b (mon.1) 242 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:17:23.188 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:17:22 smithi100 bash[31660]: cluster 2024-04-07T16:17:22.833576+0000 mon.a (mon.0) 929 : cluster 0 osdmap e108: 8 total, 8 up, 8 in 2024-04-07T16:17:23.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:17:22 smithi144 bash[23522]: cluster 2024-04-07T16:17:21.907829+0000 mon.a (mon.0) 928 : cluster 0 osdmap e107: 8 total, 8 up, 8 in 2024-04-07T16:17:23.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:17:22 smithi144 bash[23522]: audit 2024-04-07T16:17:22.703676+0000 mon.b (mon.1) 242 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:17:23.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:17:22 smithi144 bash[23522]: cluster 2024-04-07T16:17:22.833576+0000 mon.a (mon.0) 929 : cluster 0 osdmap e108: 8 total, 8 up, 8 in 2024-04-07T16:17:23.823 INFO:tasks.rados.rados.0.smithi100.stdout:237: done (0 left) 2024-04-07T16:17:23.823 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 17 current snap is 18 2024-04-07T16:17:23.824 INFO:tasks.rados.rados.0.smithi100.stdout:238: seq_num 98 ranges {2744320=450560,3194880=466944,3661824=737280} 2024-04-07T16:17:23.866 INFO:tasks.rados.rados.0.smithi100.stdout:238: writing smithi10066867-17 from 2744320 to 3194880 tid 1 2024-04-07T16:17:23.872 INFO:tasks.rados.rados.0.smithi100.stdout:238: writing smithi10066867-17 from 3194880 to 3661824 tid 2 2024-04-07T16:17:23.882 INFO:tasks.rados.rados.0.smithi100.stdout:238: writing smithi10066867-17 from 3661824 to 4399104 tid 3 2024-04-07T16:17:23.882 INFO:tasks.rados.rados.0.smithi100.stdout:239: rollback oid 8 current snap is 18 2024-04-07T16:17:23.882 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 8 to 16 2024-04-07T16:17:23.883 INFO:tasks.rados.rados.0.smithi100.stdout:238: finishing write tid 1 to smithi10066867-17 2024-04-07T16:17:23.883 INFO:tasks.rados.rados.0.smithi100.stdout:238: finishing write tid 2 to smithi10066867-17 2024-04-07T16:17:23.883 INFO:tasks.rados.rados.0.smithi100.stdout:240: rmattr oid 26 current snap is 18 2024-04-07T16:17:23.883 INFO:tasks.rados.rados.0.smithi100.stdout:240: done (2 left) 2024-04-07T16:17:23.883 INFO:tasks.rados.rados.0.smithi100.stdout:241: delete oid 2 current snap is 18 2024-04-07T16:17:23.892 INFO:tasks.rados.rados.0.smithi100.stdout:241: done (2 left) 2024-04-07T16:17:23.892 INFO:tasks.rados.rados.0.smithi100.stdout:242: rmattr oid 24 current snap is 18 2024-04-07T16:17:23.892 INFO:tasks.rados.rados.0.smithi100.stdout:242: done (2 left) 2024-04-07T16:17:23.892 INFO:tasks.rados.rados.0.smithi100.stdout:243: snap_create 2024-04-07T16:17:23.892 INFO:tasks.rados.rados.0.smithi100.stdout:239: finishing rollback tid 0 to smithi10066867-8 2024-04-07T16:17:23.893 INFO:tasks.rados.rados.0.smithi100.stdout:238: finishing write tid 3 to smithi10066867-17 2024-04-07T16:17:23.893 INFO:tasks.rados.rados.0.smithi100.stdout:238: finishing write tid 4 to smithi10066867-17 2024-04-07T16:17:23.894 INFO:tasks.rados.rados.0.smithi100.stdout:239: finishing rollback tid 1 to smithi10066867-8 2024-04-07T16:17:23.894 INFO:tasks.rados.rados.0.smithi100.stdout:239: finishing rollback tid 2 to smithi10066867-8 2024-04-07T16:17:23.895 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 8 v 42 (ObjNum 7 snap 0 seq_num 7) dirty exists 2024-04-07T16:17:23.896 INFO:tasks.rados.rados.0.smithi100.stdout:238: finishing write tid 5 to smithi10066867-17 2024-04-07T16:17:23.896 INFO:tasks.rados.rados.0.smithi100.stdout:238: oid 17 updating version 0 to 78 2024-04-07T16:17:23.897 INFO:tasks.rados.rados.0.smithi100.stdout:238: oid 17 updating version 78 to 79 2024-04-07T16:17:23.897 INFO:tasks.rados.rados.0.smithi100.stdout:238: oid 17 version 79 is already newer than 77 2024-04-07T16:17:23.897 INFO:tasks.rados.rados.0.smithi100.stdout:238: oid 17 version 79 is already newer than 76 2024-04-07T16:17:23.897 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 17 v 79 (ObjNum 98 snap 18 seq_num 98) dirty exists 2024-04-07T16:17:23.897 INFO:tasks.rados.rados.0.smithi100.stdout:238: left oid 17 (ObjNum 98 snap 18 seq_num 98) 2024-04-07T16:17:24.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:17:23 smithi100 bash[21346]: cluster 2024-04-07T16:17:22.702383+0000 mgr.y (mgr.14553) 201 : cluster 0 pgmap v180: 121 pgs: 121 active+clean; 45 MiB data, 303 MiB used, 715 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.0 MiB/s wr, 17 op/s 2024-04-07T16:17:24.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:17:23 smithi100 bash[21346]: cluster 2024-04-07T16:17:23.835096+0000 mon.a (mon.0) 930 : cluster 0 osdmap e109: 8 total, 8 up, 8 in 2024-04-07T16:17:24.188 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:17:23 smithi100 bash[31660]: cluster 2024-04-07T16:17:22.702383+0000 mgr.y (mgr.14553) 201 : cluster 0 pgmap v180: 121 pgs: 121 active+clean; 45 MiB data, 303 MiB used, 715 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.0 MiB/s wr, 17 op/s 2024-04-07T16:17:24.188 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:17:23 smithi100 bash[31660]: cluster 2024-04-07T16:17:23.835096+0000 mon.a (mon.0) 930 : cluster 0 osdmap e109: 8 total, 8 up, 8 in 2024-04-07T16:17:24.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:17:23 smithi144 bash[23522]: cluster 2024-04-07T16:17:22.702383+0000 mgr.y (mgr.14553) 201 : cluster 0 pgmap v180: 121 pgs: 121 active+clean; 45 MiB data, 303 MiB used, 715 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.0 MiB/s wr, 17 op/s 2024-04-07T16:17:24.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:17:23 smithi144 bash[23522]: cluster 2024-04-07T16:17:23.835096+0000 mon.a (mon.0) 930 : cluster 0 osdmap e109: 8 total, 8 up, 8 in 2024-04-07T16:17:24.824 INFO:tasks.rados.rados.0.smithi100.stdout:238: done (2 left) 2024-04-07T16:17:24.824 INFO:tasks.rados.rados.0.smithi100.stdout:239: done (1 left) 2024-04-07T16:17:24.824 INFO:tasks.rados.rados.0.smithi100.stdout:243: done (0 left) 2024-04-07T16:17:24.824 INFO:tasks.rados.rados.0.smithi100.stdout:244: copy_from oid 48 from oid 47 current snap is 19 2024-04-07T16:17:24.824 INFO:tasks.rados.rados.0.smithi100.stdout:245: read oid 34 snap -1 2024-04-07T16:17:24.824 INFO:tasks.rados.rados.0.smithi100.stdout:245: expect (ObjNum 33 snap 0 seq_num 33) 2024-04-07T16:17:24.824 INFO:tasks.rados.rados.0.smithi100.stdout:246: snap_create 2024-04-07T16:17:24.842 INFO:tasks.rados.rados.0.smithi100.stdout:244: finishing copy_from racing read to smithi10066867-48 2024-04-07T16:17:24.842 INFO:tasks.rados.rados.0.smithi100.stdout:245: expect (ObjNum 33 snap 0 seq_num 33) 2024-04-07T16:17:24.884 INFO:tasks.rados.rados.0.smithi100.stdout:244: finishing copy_from to smithi10066867-48 2024-04-07T16:17:24.884 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 48 v 5 (ObjNum 46 snap 0 seq_num 46) dirty exists 2024-04-07T16:17:25.437 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:17:25 smithi100 bash[21601]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:17:25] "GET /metrics HTTP/1.1" 200 36080 "" "Prometheus/2.43.0" 2024-04-07T16:17:25.828 INFO:tasks.rados.rados.0.smithi100.stdout:244: done (2 left) 2024-04-07T16:17:25.828 INFO:tasks.rados.rados.0.smithi100.stdout:245: done (1 left) 2024-04-07T16:17:25.828 INFO:tasks.rados.rados.0.smithi100.stdout:246: done (0 left) 2024-04-07T16:17:25.828 INFO:tasks.rados.rados.0.smithi100.stdout:247: read oid 47 snap -1 2024-04-07T16:17:25.828 INFO:tasks.rados.rados.0.smithi100.stdout:247: expect (ObjNum 46 snap 0 seq_num 46) 2024-04-07T16:17:25.828 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 36 current snap is 20 2024-04-07T16:17:25.828 INFO:tasks.rados.rados.0.smithi100.stdout:248: seq_num 100 ranges {172032=483328,655360=688128,1343488=516096} 2024-04-07T16:17:25.838 INFO:tasks.rados.rados.0.smithi100.stdout:248: writing smithi10066867-36 from 172032 to 655360 tid 1 2024-04-07T16:17:25.851 INFO:tasks.rados.rados.0.smithi100.stdout:248: writing smithi10066867-36 from 655360 to 1343488 tid 2 2024-04-07T16:17:25.860 INFO:tasks.rados.rados.0.smithi100.stdout:248: writing smithi10066867-36 from 1343488 to 1859584 tid 3 2024-04-07T16:17:25.860 INFO:tasks.rados.rados.0.smithi100.stdout:249: rollback oid 18 current snap is 20 2024-04-07T16:17:25.860 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 18 to 19 2024-04-07T16:17:25.861 INFO:tasks.rados.rados.0.smithi100.stdout:247: expect (ObjNum 46 snap 0 seq_num 46) 2024-04-07T16:17:25.913 INFO:tasks.rados.rados.0.smithi100.stdout:248: finishing write tid 1 to smithi10066867-36 2024-04-07T16:17:25.913 INFO:tasks.rados.rados.0.smithi100.stdout:248: finishing write tid 2 to smithi10066867-36 2024-04-07T16:17:25.913 INFO:tasks.rados.rados.0.smithi100.stdout:248: finishing write tid 3 to smithi10066867-36 2024-04-07T16:17:25.914 INFO:tasks.rados.rados.0.smithi100.stdout:248: finishing write tid 4 to smithi10066867-36 2024-04-07T16:17:25.914 INFO:tasks.rados.rados.0.smithi100.stdout:249: finishing rollback tid 0 to smithi10066867-18 2024-04-07T16:17:25.914 INFO:tasks.rados.rados.0.smithi100.stdout:249: finishing rollback tid 1 to smithi10066867-18 2024-04-07T16:17:25.914 INFO:tasks.rados.rados.0.smithi100.stdout:249: finishing rollback tid 2 to smithi10066867-18 2024-04-07T16:17:25.914 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 18 v 59 (ObjNum 57 snap 4 seq_num 57) dirty exists 2024-04-07T16:17:25.914 INFO:tasks.rados.rados.0.smithi100.stdout:248: finishing write tid 5 to smithi10066867-36 2024-04-07T16:17:25.914 INFO:tasks.rados.rados.0.smithi100.stdout:248: oid 36 updating version 0 to 82 2024-04-07T16:17:25.914 INFO:tasks.rados.rados.0.smithi100.stdout:248: oid 36 updating version 82 to 84 2024-04-07T16:17:25.914 INFO:tasks.rados.rados.0.smithi100.stdout:248: oid 36 version 84 is already newer than 83 2024-04-07T16:17:25.914 INFO:tasks.rados.rados.0.smithi100.stdout:248: oid 36 version 84 is already newer than 81 2024-04-07T16:17:25.914 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 36 v 84 (ObjNum 100 snap 20 seq_num 100) dirty exists 2024-04-07T16:17:25.914 INFO:tasks.rados.rados.0.smithi100.stdout:248: left oid 36 (ObjNum 100 snap 20 seq_num 100) 2024-04-07T16:17:25.914 INFO:tasks.rados.rados.0.smithi100.stdout:247: done (2 left) 2024-04-07T16:17:25.914 INFO:tasks.rados.rados.0.smithi100.stdout:248: done (1 left) 2024-04-07T16:17:25.915 INFO:tasks.rados.rados.0.smithi100.stdout:249: done (0 left) 2024-04-07T16:17:25.915 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 2 current snap is 20 2024-04-07T16:17:25.915 INFO:tasks.rados.rados.0.smithi100.stdout:250: seq_num 101 ranges {0=499712,499712=573440,1073152=524288,1597440=106496} 2024-04-07T16:17:25.921 INFO:tasks.rados.rados.0.smithi100.stdout:250: writing smithi10066867-2 from 0 to 499712 tid 1 2024-04-07T16:17:25.932 INFO:tasks.rados.rados.0.smithi100.stdout:250: writing smithi10066867-2 from 499712 to 1073152 tid 2 2024-04-07T16:17:25.942 INFO:tasks.rados.rados.0.smithi100.stdout:250: writing smithi10066867-2 from 1073152 to 1597440 tid 3 2024-04-07T16:17:25.944 INFO:tasks.rados.rados.0.smithi100.stdout:250: writing smithi10066867-2 from 1597440 to 1703936 tid 4 2024-04-07T16:17:25.944 INFO:tasks.rados.rados.0.smithi100.stdout:251: delete oid 40 current snap is 20 2024-04-07T16:17:25.944 INFO:tasks.rados.rados.0.smithi100.stdout:250: finishing write tid 1 to smithi10066867-2 2024-04-07T16:17:25.944 INFO:tasks.rados.rados.0.smithi100.stdout:250: finishing write tid 2 to smithi10066867-2 2024-04-07T16:17:25.949 INFO:tasks.rados.rados.0.smithi100.stdout:250: finishing write tid 3 to smithi10066867-2 2024-04-07T16:17:25.949 INFO:tasks.rados.rados.0.smithi100.stdout:250: finishing write tid 4 to smithi10066867-2 2024-04-07T16:17:25.949 INFO:tasks.rados.rados.0.smithi100.stdout:250: finishing write tid 5 to smithi10066867-2 2024-04-07T16:17:25.956 INFO:tasks.rados.rados.0.smithi100.stdout:250: finishing write tid 6 to smithi10066867-2 2024-04-07T16:17:25.956 INFO:tasks.rados.rados.0.smithi100.stdout:250: oid 2 updating version 0 to 75 2024-04-07T16:17:25.956 INFO:tasks.rados.rados.0.smithi100.stdout:250: oid 2 updating version 75 to 76 2024-04-07T16:17:25.957 INFO:tasks.rados.rados.0.smithi100.stdout:250: oid 2 updating version 76 to 78 2024-04-07T16:17:25.957 INFO:tasks.rados.rados.0.smithi100.stdout:250: oid 2 version 78 is already newer than 77 2024-04-07T16:17:25.957 INFO:tasks.rados.rados.0.smithi100.stdout:250: oid 2 version 78 is already newer than 74 2024-04-07T16:17:25.957 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 2 v 78 (ObjNum 101 snap 20 seq_num 101) dirty exists 2024-04-07T16:17:25.957 INFO:tasks.rados.rados.0.smithi100.stdout:250: left oid 2 (ObjNum 101 snap 20 seq_num 101) 2024-04-07T16:17:25.957 INFO:tasks.rados.rados.0.smithi100.stdout:250: done (1 left) 2024-04-07T16:17:25.957 INFO:tasks.rados.rados.0.smithi100.stdout:251: done (0 left) 2024-04-07T16:17:25.957 INFO:tasks.rados.rados.0.smithi100.stdout:252: read oid 26 snap -1 2024-04-07T16:17:25.957 INFO:tasks.rados.rados.0.smithi100.stdout:252: expect deleted 2024-04-07T16:17:25.957 INFO:tasks.rados.rados.0.smithi100.stdout:253: read oid 23 snap 16 2024-04-07T16:17:25.957 INFO:tasks.rados.rados.0.smithi100.stdout:253: expect (ObjNum 53 snap 1 seq_num 53) 2024-04-07T16:17:25.957 INFO:tasks.rados.rados.0.smithi100.stdout:254: setattr oid 22 current snap is 20 2024-04-07T16:17:25.960 INFO:tasks.rados.rados.0.smithi100.stdout:252: done (2 left) 2024-04-07T16:17:25.960 INFO:tasks.rados.rados.0.smithi100.stdout:255: rollback oid 25 current snap is 20 2024-04-07T16:17:25.960 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 25 to 14 2024-04-07T16:17:25.960 INFO:tasks.rados.rados.0.smithi100.stdout:256: delete oid 33 current snap is 20 2024-04-07T16:17:25.966 INFO:tasks.rados.rados.0.smithi100.stdout:253: expect (ObjNum 53 snap 1 seq_num 53) 2024-04-07T16:17:26.046 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 22 v 50 (ObjNum 49 snap 0 seq_num 1716807775) dirty exists 2024-04-07T16:17:26.046 INFO:tasks.rados.rados.0.smithi100.stdout:255: finishing rollback tid 0 to smithi10066867-25 2024-04-07T16:17:26.046 INFO:tasks.rados.rados.0.smithi100.stdout:255: finishing rollback tid 1 to smithi10066867-25 2024-04-07T16:17:26.046 INFO:tasks.rados.rados.0.smithi100.stdout:255: finishing rollback tid 2 to smithi10066867-25 2024-04-07T16:17:26.046 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 25 v 54 (ObjNum 24 snap 0 seq_num 24) dirty exists 2024-04-07T16:17:26.046 INFO:tasks.rados.rados.0.smithi100.stdout:253: done (3 left) 2024-04-07T16:17:26.046 INFO:tasks.rados.rados.0.smithi100.stdout:254: done (2 left) 2024-04-07T16:17:26.046 INFO:tasks.rados.rados.0.smithi100.stdout:255: done (1 left) 2024-04-07T16:17:26.046 INFO:tasks.rados.rados.0.smithi100.stdout:256: done (0 left) 2024-04-07T16:17:26.046 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 26 current snap is 20 2024-04-07T16:17:26.046 INFO:tasks.rados.rados.0.smithi100.stdout:257: seq_num 104 ranges {0=548864,548864=622592,1171456=581632} 2024-04-07T16:17:26.054 INFO:tasks.rados.rados.0.smithi100.stdout:257: writing smithi10066867-26 from 0 to 548864 tid 1 2024-04-07T16:17:26.065 INFO:tasks.rados.rados.0.smithi100.stdout:257: writing smithi10066867-26 from 548864 to 1171456 tid 2 2024-04-07T16:17:26.076 INFO:tasks.rados.rados.0.smithi100.stdout:257: writing smithi10066867-26 from 1171456 to 1753088 tid 3 2024-04-07T16:17:26.076 INFO:tasks.rados.rados.0.smithi100.stdout:258: setattr oid 47 current snap is 20 2024-04-07T16:17:26.076 INFO:tasks.rados.rados.0.smithi100.stdout:257: finishing write tid 1 to smithi10066867-26 2024-04-07T16:17:26.076 INFO:tasks.rados.rados.0.smithi100.stdout:257: finishing write tid 2 to smithi10066867-26 2024-04-07T16:17:26.081 INFO:tasks.rados.rados.0.smithi100.stdout:257: finishing write tid 3 to smithi10066867-26 2024-04-07T16:17:26.081 INFO:tasks.rados.rados.0.smithi100.stdout:257: finishing write tid 4 to smithi10066867-26 2024-04-07T16:17:26.082 INFO:tasks.rados.rados.0.smithi100.stdout:259: setattr oid 30 current snap is 20 2024-04-07T16:17:26.086 INFO:tasks.rados.rados.0.smithi100.stdout:257: finishing write tid 5 to smithi10066867-26 2024-04-07T16:17:26.086 INFO:tasks.rados.rados.0.smithi100.stdout:257: oid 26 updating version 0 to 57 2024-04-07T16:17:26.086 INFO:tasks.rados.rados.0.smithi100.stdout:257: oid 26 updating version 57 to 58 2024-04-07T16:17:26.086 INFO:tasks.rados.rados.0.smithi100.stdout:257: oid 26 version 58 is already newer than 56 2024-04-07T16:17:26.086 INFO:tasks.rados.rados.0.smithi100.stdout:257: oid 26 version 58 is already newer than 55 2024-04-07T16:17:26.086 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 26 v 58 (ObjNum 104 snap 20 seq_num 104) dirty exists 2024-04-07T16:17:26.087 INFO:tasks.rados.rados.0.smithi100.stdout:257: left oid 26 (ObjNum 104 snap 20 seq_num 104) 2024-04-07T16:17:26.088 INFO:tasks.rados.rados.0.smithi100.stdout:257: done (2 left) 2024-04-07T16:17:26.088 INFO:tasks.rados.rados.0.smithi100.stdout:260: snap_remove snap 14 2024-04-07T16:17:26.105 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:17:25 smithi144 bash[23522]: cluster 2024-04-07T16:17:24.703088+0000 mgr.y (mgr.14553) 202 : cluster 0 pgmap v183: 121 pgs: 121 active+clean; 53 MiB data, 321 MiB used, 715 GiB / 715 GiB avail; 1.8 MiB/s rd, 3.3 MiB/s wr, 23 op/s 2024-04-07T16:17:26.105 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:17:25 smithi144 bash[23522]: cluster 2024-04-07T16:17:24.838582+0000 mon.a (mon.0) 931 : cluster 0 osdmap e110: 8 total, 8 up, 8 in 2024-04-07T16:17:26.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:17:25 smithi100 bash[31660]: cluster 2024-04-07T16:17:24.703088+0000 mgr.y (mgr.14553) 202 : cluster 0 pgmap v183: 121 pgs: 121 active+clean; 53 MiB data, 321 MiB used, 715 GiB / 715 GiB avail; 1.8 MiB/s rd, 3.3 MiB/s wr, 23 op/s 2024-04-07T16:17:26.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:17:25 smithi100 bash[31660]: cluster 2024-04-07T16:17:24.838582+0000 mon.a (mon.0) 931 : cluster 0 osdmap e110: 8 total, 8 up, 8 in 2024-04-07T16:17:26.188 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:17:25 smithi100 bash[21346]: cluster 2024-04-07T16:17:24.703088+0000 mgr.y (mgr.14553) 202 : cluster 0 pgmap v183: 121 pgs: 121 active+clean; 53 MiB data, 321 MiB used, 715 GiB / 715 GiB avail; 1.8 MiB/s rd, 3.3 MiB/s wr, 23 op/s 2024-04-07T16:17:26.188 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:17:25 smithi100 bash[21346]: cluster 2024-04-07T16:17:24.838582+0000 mon.a (mon.0) 931 : cluster 0 osdmap e110: 8 total, 8 up, 8 in 2024-04-07T16:17:26.851 INFO:tasks.rados.rados.0.smithi100.stdout:260: done (2 left) 2024-04-07T16:17:26.851 INFO:tasks.rados.rados.0.smithi100.stdout:261: read oid 45 snap -1 2024-04-07T16:17:26.851 INFO:tasks.rados.rados.0.smithi100.stdout:261: expect (ObjNum 44 snap 0 seq_num 44) 2024-04-07T16:17:26.851 INFO:tasks.rados.rados.0.smithi100.stdout:262: read oid 17 snap -1 2024-04-07T16:17:26.851 INFO:tasks.rados.rados.0.smithi100.stdout:262: expect (ObjNum 98 snap 18 seq_num 98) 2024-04-07T16:17:26.851 INFO:tasks.rados.rados.0.smithi100.stdout:263: read oid 18 snap -1 2024-04-07T16:17:26.851 INFO:tasks.rados.rados.0.smithi100.stdout:263: expect (ObjNum 57 snap 4 seq_num 57) 2024-04-07T16:17:26.851 INFO:tasks.rados.rados.0.smithi100.stdout:264: snap_remove snap 19 2024-04-07T16:17:27.105 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:17:26 smithi144 bash[23522]: cluster 2024-04-07T16:17:25.841352+0000 mon.a (mon.0) 932 : cluster 0 osdmap e111: 8 total, 8 up, 8 in 2024-04-07T16:17:27.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:17:26 smithi100 bash[21346]: cluster 2024-04-07T16:17:25.841352+0000 mon.a (mon.0) 932 : cluster 0 osdmap e111: 8 total, 8 up, 8 in 2024-04-07T16:17:27.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:17:26 smithi100 bash[31660]: cluster 2024-04-07T16:17:25.841352+0000 mon.a (mon.0) 932 : cluster 0 osdmap e111: 8 total, 8 up, 8 in 2024-04-07T16:17:27.823 INFO:tasks.rados.rados.0.smithi100.stdout:264: done (5 left) 2024-04-07T16:17:27.823 INFO:tasks.rados.rados.0.smithi100.stdout:265: copy_from oid 37 from oid 44 current snap is 20 2024-04-07T16:17:27.823 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 47 v 28 (ObjNum 46 snap 0 seq_num 46) dirty exists 2024-04-07T16:17:27.823 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 30 v 53 (ObjNum 75 snap 11 seq_num 75) dirty exists 2024-04-07T16:17:27.823 INFO:tasks.rados.rados.0.smithi100.stdout:261: expect (ObjNum 44 snap 0 seq_num 44) 2024-04-07T16:17:27.867 INFO:tasks.rados.rados.0.smithi100.stdout:263: expect (ObjNum 57 snap 4 seq_num 57) 2024-04-07T16:17:27.928 INFO:tasks.rados.rados.0.smithi100.stdout:262: expect (ObjNum 98 snap 18 seq_num 98) 2024-04-07T16:17:28.105 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:17:27 smithi144 bash[23522]: cluster 2024-04-07T16:17:26.703769+0000 mgr.y (mgr.14553) 203 : cluster 0 pgmap v186: 121 pgs: 121 active+clean; 53 MiB data, 321 MiB used, 715 GiB / 715 GiB avail; 1.8 MiB/s rd, 3.3 MiB/s wr, 23 op/s 2024-04-07T16:17:28.105 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:17:27 smithi144 bash[23522]: cluster 2024-04-07T16:17:26.865125+0000 mon.a (mon.0) 933 : cluster 0 osdmap e112: 8 total, 8 up, 8 in 2024-04-07T16:17:28.105 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:17:27 smithi144 bash[23522]: cluster 2024-04-07T16:17:27.836685+0000 mon.a (mon.0) 934 : cluster 0 osdmap e113: 8 total, 8 up, 8 in 2024-04-07T16:17:28.109 INFO:tasks.rados.rados.0.smithi100.stdout:265: finishing copy_from racing read to smithi10066867-37 2024-04-07T16:17:28.109 INFO:tasks.rados.rados.0.smithi100.stdout:265: finishing copy_from to smithi10066867-37 2024-04-07T16:17:28.109 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 37 v 2 (ObjNum 28 snap 0 seq_num 28) dirty exists 2024-04-07T16:17:28.110 INFO:tasks.rados.rados.0.smithi100.stdout:258: done (5 left) 2024-04-07T16:17:28.110 INFO:tasks.rados.rados.0.smithi100.stdout:259: done (4 left) 2024-04-07T16:17:28.110 INFO:tasks.rados.rados.0.smithi100.stdout:261: done (3 left) 2024-04-07T16:17:28.110 INFO:tasks.rados.rados.0.smithi100.stdout:262: done (2 left) 2024-04-07T16:17:28.110 INFO:tasks.rados.rados.0.smithi100.stdout:263: done (1 left) 2024-04-07T16:17:28.110 INFO:tasks.rados.rados.0.smithi100.stdout:265: done (0 left) 2024-04-07T16:17:28.110 INFO:tasks.rados.rados.0.smithi100.stdout:266: read oid 27 snap -1 2024-04-07T16:17:28.110 INFO:tasks.rados.rados.0.smithi100.stdout:266: expect (ObjNum 56 snap 4 seq_num 56) 2024-04-07T16:17:28.110 INFO:tasks.rados.rados.0.smithi100.stdout:267: snap_remove snap 16 2024-04-07T16:17:28.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:17:27 smithi100 bash[21346]: cluster 2024-04-07T16:17:26.703769+0000 mgr.y (mgr.14553) 203 : cluster 0 pgmap v186: 121 pgs: 121 active+clean; 53 MiB data, 321 MiB used, 715 GiB / 715 GiB avail; 1.8 MiB/s rd, 3.3 MiB/s wr, 23 op/s 2024-04-07T16:17:28.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:17:27 smithi100 bash[21346]: cluster 2024-04-07T16:17:26.865125+0000 mon.a (mon.0) 933 : cluster 0 osdmap e112: 8 total, 8 up, 8 in 2024-04-07T16:17:28.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:17:27 smithi100 bash[21346]: cluster 2024-04-07T16:17:27.836685+0000 mon.a (mon.0) 934 : cluster 0 osdmap e113: 8 total, 8 up, 8 in 2024-04-07T16:17:28.188 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:17:27 smithi100 bash[31660]: cluster 2024-04-07T16:17:26.703769+0000 mgr.y (mgr.14553) 203 : cluster 0 pgmap v186: 121 pgs: 121 active+clean; 53 MiB data, 321 MiB used, 715 GiB / 715 GiB avail; 1.8 MiB/s rd, 3.3 MiB/s wr, 23 op/s 2024-04-07T16:17:28.188 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:17:27 smithi100 bash[31660]: cluster 2024-04-07T16:17:26.865125+0000 mon.a (mon.0) 933 : cluster 0 osdmap e112: 8 total, 8 up, 8 in 2024-04-07T16:17:28.188 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:17:27 smithi100 bash[31660]: cluster 2024-04-07T16:17:27.836685+0000 mon.a (mon.0) 934 : cluster 0 osdmap e113: 8 total, 8 up, 8 in 2024-04-07T16:17:28.865 INFO:tasks.rados.rados.0.smithi100.stdout:267: done (1 left) 2024-04-07T16:17:28.865 INFO:tasks.rados.rados.0.smithi100.stdout:268: read oid 33 snap -1 2024-04-07T16:17:28.865 INFO:tasks.rados.rados.0.smithi100.stdout:268: expect deleted 2024-04-07T16:17:28.865 INFO:tasks.rados.rados.0.smithi100.stdout:269: delete oid 36 current snap is 20 2024-04-07T16:17:28.871 INFO:tasks.rados.rados.0.smithi100.stdout:266: expect (ObjNum 56 snap 4 seq_num 56) 2024-04-07T16:17:28.898 INFO:tasks.rados.rados.0.smithi100.stdout:266: done (2 left) 2024-04-07T16:17:28.898 INFO:tasks.rados.rados.0.smithi100.stdout:268: done (1 left) 2024-04-07T16:17:28.898 INFO:tasks.rados.rados.0.smithi100.stdout:269: done (0 left) 2024-04-07T16:17:28.898 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 30 current snap is 20 2024-04-07T16:17:28.898 INFO:tasks.rados.rados.0.smithi100.stdout:270: seq_num 106 ranges {1564672=581632,2146304=794624,2940928=409600} 2024-04-07T16:17:28.931 INFO:tasks.rados.rados.0.smithi100.stdout:270: writing smithi10066867-30 from 1564672 to 2146304 tid 1 2024-04-07T16:17:28.946 INFO:tasks.rados.rados.0.smithi100.stdout:270: writing smithi10066867-30 from 2146304 to 2940928 tid 2 2024-04-07T16:17:28.950 INFO:tasks.rados.rados.0.smithi100.stdout:270: writing smithi10066867-30 from 2940928 to 3350528 tid 3 2024-04-07T16:17:28.950 INFO:tasks.rados.rados.0.smithi100.stdout:271: rmattr oid 17 current snap is 20 2024-04-07T16:17:28.950 INFO:tasks.rados.rados.0.smithi100.stdout:271: done (1 left) 2024-04-07T16:17:28.950 INFO:tasks.rados.rados.0.smithi100.stdout:272: snap_remove snap 17 2024-04-07T16:17:29.868 INFO:tasks.rados.rados.0.smithi100.stdout:272: done (1 left) 2024-04-07T16:17:29.868 INFO:tasks.rados.rados.0.smithi100.stdout:273: snap_remove snap 18 2024-04-07T16:17:30.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:17:29 smithi100 bash[21346]: cluster 2024-04-07T16:17:28.704412+0000 mgr.y (mgr.14553) 204 : cluster 0 pgmap v189: 121 pgs: 121 active+clean; 53 MiB data, 321 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:17:30.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:17:29 smithi100 bash[21346]: cluster 2024-04-07T16:17:28.875606+0000 mon.a (mon.0) 935 : cluster 0 osdmap e114: 8 total, 8 up, 8 in 2024-04-07T16:17:30.188 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:17:29 smithi100 bash[31660]: cluster 2024-04-07T16:17:28.704412+0000 mgr.y (mgr.14553) 204 : cluster 0 pgmap v189: 121 pgs: 121 active+clean; 53 MiB data, 321 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:17:30.188 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:17:29 smithi100 bash[31660]: cluster 2024-04-07T16:17:28.875606+0000 mon.a (mon.0) 935 : cluster 0 osdmap e114: 8 total, 8 up, 8 in 2024-04-07T16:17:30.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:17:29 smithi144 bash[23522]: cluster 2024-04-07T16:17:28.704412+0000 mgr.y (mgr.14553) 204 : cluster 0 pgmap v189: 121 pgs: 121 active+clean; 53 MiB data, 321 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:17:30.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:17:29 smithi144 bash[23522]: cluster 2024-04-07T16:17:28.875606+0000 mon.a (mon.0) 935 : cluster 0 osdmap e114: 8 total, 8 up, 8 in 2024-04-07T16:17:30.887 INFO:tasks.rados.rados.0.smithi100.stdout:273: done (1 left) 2024-04-07T16:17:30.888 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 6 current snap is 20 2024-04-07T16:17:30.888 INFO:tasks.rados.rados.0.smithi100.stdout:270: finishing write tid 1 to smithi10066867-30 2024-04-07T16:17:30.888 INFO:tasks.rados.rados.0.smithi100.stdout:270: finishing write tid 2 to smithi10066867-30 2024-04-07T16:17:30.888 INFO:tasks.rados.rados.0.smithi100.stdout:270: finishing write tid 3 to smithi10066867-30 2024-04-07T16:17:30.888 INFO:tasks.rados.rados.0.smithi100.stdout:270: finishing write tid 4 to smithi10066867-30 2024-04-07T16:17:30.888 INFO:tasks.rados.rados.0.smithi100.stdout:270: finishing write tid 5 to smithi10066867-30 2024-04-07T16:17:30.888 INFO:tasks.rados.rados.0.smithi100.stdout:270: oid 30 updating version 0 to 64 2024-04-07T16:17:30.888 INFO:tasks.rados.rados.0.smithi100.stdout:270: oid 30 version 64 is already newer than 63 2024-04-07T16:17:30.888 INFO:tasks.rados.rados.0.smithi100.stdout:270: oid 30 version 64 is already newer than 61 2024-04-07T16:17:30.888 INFO:tasks.rados.rados.0.smithi100.stdout:270: oid 30 version 64 is already newer than 62 2024-04-07T16:17:30.888 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 30 v 64 (ObjNum 106 snap 20 seq_num 106) dirty exists 2024-04-07T16:17:30.888 INFO:tasks.rados.rados.0.smithi100.stdout:270: left oid 30 (ObjNum 106 snap 20 seq_num 106) 2024-04-07T16:17:30.889 INFO:tasks.rados.rados.0.smithi100.stdout:274: seq_num 107 ranges {0=598016,598016=729088,1327104=466944,1794048=8192} 2024-04-07T16:17:30.895 INFO:tasks.rados.rados.0.smithi100.stdout:274: writing smithi10066867-6 from 0 to 598016 tid 1 2024-04-07T16:17:30.906 INFO:tasks.rados.rados.0.smithi100.stdout:274: writing smithi10066867-6 from 598016 to 1327104 tid 2 2024-04-07T16:17:30.912 INFO:tasks.rados.rados.0.smithi100.stdout:274: writing smithi10066867-6 from 1327104 to 1794048 tid 3 2024-04-07T16:17:30.913 INFO:tasks.rados.rados.0.smithi100.stdout:274: writing smithi10066867-6 from 1794048 to 1802240 tid 4 2024-04-07T16:17:30.913 INFO:tasks.rados.rados.0.smithi100.stdout:270: done (1 left) 2024-04-07T16:17:30.913 INFO:tasks.rados.rados.0.smithi100.stdout:275: read oid 29 snap -1 2024-04-07T16:17:30.913 INFO:tasks.rados.rados.0.smithi100.stdout:275: expect (ObjNum 61 snap 5 seq_num 61) 2024-04-07T16:17:30.913 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 2 current snap is 20 2024-04-07T16:17:30.914 INFO:tasks.rados.rados.0.smithi100.stdout:276: seq_num 108 ranges {1703936=614400,2318336=606208,2924544=598016} 2024-04-07T16:17:30.940 INFO:tasks.rados.rados.0.smithi100.stdout:276: writing smithi10066867-2 from 1703936 to 2318336 tid 1 2024-04-07T16:17:30.945 INFO:tasks.rados.rados.0.smithi100.stdout:276: writing smithi10066867-2 from 2318336 to 2924544 tid 2 2024-04-07T16:17:30.951 INFO:tasks.rados.rados.0.smithi100.stdout:276: writing smithi10066867-2 from 2924544 to 3522560 tid 3 2024-04-07T16:17:30.951 INFO:tasks.rados.rados.0.smithi100.stdout:277: snap_create 2024-04-07T16:17:30.951 INFO:tasks.rados.rados.0.smithi100.stdout:274: finishing write tid 1 to smithi10066867-6 2024-04-07T16:17:30.951 INFO:tasks.rados.rados.0.smithi100.stdout:274: finishing write tid 2 to smithi10066867-6 2024-04-07T16:17:30.952 INFO:tasks.rados.rados.0.smithi100.stdout:274: finishing write tid 3 to smithi10066867-6 2024-04-07T16:17:30.952 INFO:tasks.rados.rados.0.smithi100.stdout:274: finishing write tid 4 to smithi10066867-6 2024-04-07T16:17:30.952 INFO:tasks.rados.rados.0.smithi100.stdout:274: finishing write tid 5 to smithi10066867-6 2024-04-07T16:17:30.952 INFO:tasks.rados.rados.0.smithi100.stdout:275: expect (ObjNum 61 snap 5 seq_num 61) 2024-04-07T16:17:31.029 INFO:tasks.rados.rados.0.smithi100.stdout:274: finishing write tid 6 to smithi10066867-6 2024-04-07T16:17:31.029 INFO:tasks.rados.rados.0.smithi100.stdout:274: oid 6 updating version 0 to 97 2024-04-07T16:17:31.029 INFO:tasks.rados.rados.0.smithi100.stdout:274: oid 6 updating version 97 to 98 2024-04-07T16:17:31.029 INFO:tasks.rados.rados.0.smithi100.stdout:274: oid 6 updating version 98 to 100 2024-04-07T16:17:31.029 INFO:tasks.rados.rados.0.smithi100.stdout:274: oid 6 version 100 is already newer than 99 2024-04-07T16:17:31.030 INFO:tasks.rados.rados.0.smithi100.stdout:274: oid 6 updating version 100 to 101 2024-04-07T16:17:31.030 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 6 v 101 (ObjNum 107 snap 20 seq_num 107) dirty exists 2024-04-07T16:17:31.030 INFO:tasks.rados.rados.0.smithi100.stdout:274: left oid 6 (ObjNum 107 snap 20 seq_num 107) 2024-04-07T16:17:31.030 INFO:tasks.rados.rados.0.smithi100.stdout:276: finishing write tid 1 to smithi10066867-2 2024-04-07T16:17:31.030 INFO:tasks.rados.rados.0.smithi100.stdout:276: finishing write tid 2 to smithi10066867-2 2024-04-07T16:17:31.030 INFO:tasks.rados.rados.0.smithi100.stdout:276: finishing write tid 3 to smithi10066867-2 2024-04-07T16:17:31.030 INFO:tasks.rados.rados.0.smithi100.stdout:276: finishing write tid 4 to smithi10066867-2 2024-04-07T16:17:31.030 INFO:tasks.rados.rados.0.smithi100.stdout:276: finishing write tid 5 to smithi10066867-2 2024-04-07T16:17:31.030 INFO:tasks.rados.rados.0.smithi100.stdout:276: oid 2 updating version 0 to 107 2024-04-07T16:17:31.030 INFO:tasks.rados.rados.0.smithi100.stdout:276: oid 2 version 107 is already newer than 104 2024-04-07T16:17:31.030 INFO:tasks.rados.rados.0.smithi100.stdout:276: oid 2 version 107 is already newer than 105 2024-04-07T16:17:31.030 INFO:tasks.rados.rados.0.smithi100.stdout:276: oid 2 version 107 is already newer than 106 2024-04-07T16:17:31.030 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 2 v 107 (ObjNum 108 snap 20 seq_num 108) dirty exists 2024-04-07T16:17:31.030 INFO:tasks.rados.rados.0.smithi100.stdout:276: left oid 2 (ObjNum 108 snap 20 seq_num 108) 2024-04-07T16:17:31.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:17:30 smithi100 bash[21346]: cluster 2024-04-07T16:17:29.881893+0000 mon.a (mon.0) 936 : cluster 0 osdmap e115: 8 total, 8 up, 8 in 2024-04-07T16:17:31.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:17:30 smithi100 bash[31660]: cluster 2024-04-07T16:17:29.881893+0000 mon.a (mon.0) 936 : cluster 0 osdmap e115: 8 total, 8 up, 8 in 2024-04-07T16:17:31.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:17:30 smithi144 bash[23522]: cluster 2024-04-07T16:17:29.881893+0000 mon.a (mon.0) 936 : cluster 0 osdmap e115: 8 total, 8 up, 8 in 2024-04-07T16:17:31.906 INFO:tasks.rados.rados.0.smithi100.stdout:274: done (3 left) 2024-04-07T16:17:31.906 INFO:tasks.rados.rados.0.smithi100.stdout:275: done (2 left) 2024-04-07T16:17:31.907 INFO:tasks.rados.rados.0.smithi100.stdout:276: done (1 left) 2024-04-07T16:17:31.907 INFO:tasks.rados.rados.0.smithi100.stdout:277: done (0 left) 2024-04-07T16:17:31.907 INFO:tasks.rados.rados.0.smithi100.stdout:278: delete oid 41 current snap is 21 2024-04-07T16:17:31.908 INFO:tasks.rados.rados.0.smithi100.stdout:278: done (0 left) 2024-04-07T16:17:31.908 INFO:tasks.rados.rados.0.smithi100.stdout:279: read oid 13 snap -1 2024-04-07T16:17:31.908 INFO:tasks.rados.rados.0.smithi100.stdout:279: expect (ObjNum 72 snap 11 seq_num 72) 2024-04-07T16:17:31.908 INFO:tasks.rados.rados.0.smithi100.stdout:280: setattr oid 26 current snap is 21 2024-04-07T16:17:31.912 INFO:tasks.rados.rados.0.smithi100.stdout:281: copy_from oid 32 from oid 7 current snap is 21 2024-04-07T16:17:31.912 INFO:tasks.rados.rados.0.smithi100.stdout:282: read oid 22 snap -1 2024-04-07T16:17:31.912 INFO:tasks.rados.rados.0.smithi100.stdout:282: expect (ObjNum 0 snap 0 seq_num 0) 2024-04-07T16:17:31.912 INFO:tasks.rados.rados.0.smithi100.stdout:283: read oid 9 snap -1 2024-04-07T16:17:31.912 INFO:tasks.rados.rados.0.smithi100.stdout:283: expect (ObjNum 72 snap 11 seq_num 72) 2024-04-07T16:17:31.912 INFO:tasks.rados.rados.0.smithi100.stdout:284: read oid 25 snap 20 2024-04-07T16:17:31.912 INFO:tasks.rados.rados.0.smithi100.stdout:284: expect (ObjNum 24 snap 0 seq_num 24) 2024-04-07T16:17:31.912 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 11 current snap is 21 2024-04-07T16:17:31.913 INFO:tasks.rados.rados.0.smithi100.stdout:285: seq_num 110 ranges {0=647168,647168=778240,1425408=425984} 2024-04-07T16:17:31.920 INFO:tasks.rados.rados.0.smithi100.stdout:285: writing smithi10066867-11 from 0 to 647168 tid 1 2024-04-07T16:17:31.929 INFO:tasks.rados.rados.0.smithi100.stdout:285: writing smithi10066867-11 from 647168 to 1425408 tid 2 2024-04-07T16:17:31.935 INFO:tasks.rados.rados.0.smithi100.stdout:285: writing smithi10066867-11 from 1425408 to 1851392 tid 3 2024-04-07T16:17:31.935 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 45 current snap is 21 2024-04-07T16:17:31.935 INFO:tasks.rados.rados.0.smithi100.stdout:286: seq_num 111 ranges {745472=663552,1409024=663552,2072576=450560,2523136=90112} 2024-04-07T16:17:31.949 INFO:tasks.rados.rados.0.smithi100.stdout:286: writing smithi10066867-45 from 745472 to 1409024 tid 1 2024-04-07T16:17:31.956 INFO:tasks.rados.rados.0.smithi100.stdout:286: writing smithi10066867-45 from 1409024 to 2072576 tid 2 2024-04-07T16:17:31.960 INFO:tasks.rados.rados.0.smithi100.stdout:286: writing smithi10066867-45 from 2072576 to 2523136 tid 3 2024-04-07T16:17:31.961 INFO:tasks.rados.rados.0.smithi100.stdout:286: writing smithi10066867-45 from 2523136 to 2613248 tid 4 2024-04-07T16:17:31.961 INFO:tasks.rados.rados.0.smithi100.stdout:287: copy_from oid 12 from oid 2 current snap is 21 2024-04-07T16:17:31.961 INFO:tasks.rados.rados.0.smithi100.stdout:282: expect (ObjNum 0 snap 0 seq_num 0) 2024-04-07T16:17:31.969 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 26 v 76 (ObjNum 104 snap 20 seq_num 104) dirty exists 2024-04-07T16:17:31.969 INFO:tasks.rados.rados.0.smithi100.stdout:284: expect (ObjNum 24 snap 0 seq_num 24) 2024-04-07T16:17:31.992 INFO:tasks.rados.rados.0.smithi100.stdout:279: expect (ObjNum 72 snap 11 seq_num 72) 2024-04-07T16:17:32.050 INFO:tasks.rados.rados.0.smithi100.stdout:285: finishing write tid 1 to smithi10066867-11 2024-04-07T16:17:32.050 INFO:tasks.rados.rados.0.smithi100.stdout:283: expect (ObjNum 72 snap 11 seq_num 72) 2024-04-07T16:17:32.091 INFO:tasks.rados.rados.0.smithi100.stdout:281: finishing copy_from racing read to smithi10066867-32 2024-04-07T16:17:32.091 INFO:tasks.rados.rados.0.smithi100.stdout:281: finishing copy_from to smithi10066867-32 2024-04-07T16:17:32.091 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 32 v 2 (ObjNum 16 snap 0 seq_num 16) dirty exists 2024-04-07T16:17:32.091 INFO:tasks.rados.rados.0.smithi100.stdout:285: finishing write tid 2 to smithi10066867-11 2024-04-07T16:17:32.091 INFO:tasks.rados.rados.0.smithi100.stdout:285: finishing write tid 3 to smithi10066867-11 2024-04-07T16:17:32.092 INFO:tasks.rados.rados.0.smithi100.stdout:285: finishing write tid 4 to smithi10066867-11 2024-04-07T16:17:32.092 INFO:tasks.rados.rados.0.smithi100.stdout:285: finishing write tid 5 to smithi10066867-11 2024-04-07T16:17:32.092 INFO:tasks.rados.rados.0.smithi100.stdout:285: oid 11 updating version 0 to 87 2024-04-07T16:17:32.092 INFO:tasks.rados.rados.0.smithi100.stdout:285: oid 11 version 87 is already newer than 85 2024-04-07T16:17:32.092 INFO:tasks.rados.rados.0.smithi100.stdout:285: oid 11 version 87 is already newer than 86 2024-04-07T16:17:32.092 INFO:tasks.rados.rados.0.smithi100.stdout:285: oid 11 version 87 is already newer than 84 2024-04-07T16:17:32.092 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 11 v 87 (ObjNum 110 snap 21 seq_num 110) dirty exists 2024-04-07T16:17:32.092 INFO:tasks.rados.rados.0.smithi100.stdout:285: left oid 11 (ObjNum 110 snap 21 seq_num 110) 2024-04-07T16:17:32.092 INFO:tasks.rados.rados.0.smithi100.stdout:286: finishing write tid 1 to smithi10066867-45 2024-04-07T16:17:32.092 INFO:tasks.rados.rados.0.smithi100.stdout:286: finishing write tid 2 to smithi10066867-45 2024-04-07T16:17:32.092 INFO:tasks.rados.rados.0.smithi100.stdout:286: finishing write tid 3 to smithi10066867-45 2024-04-07T16:17:32.092 INFO:tasks.rados.rados.0.smithi100.stdout:286: finishing write tid 4 to smithi10066867-45 2024-04-07T16:17:32.092 INFO:tasks.rados.rados.0.smithi100.stdout:286: finishing write tid 5 to smithi10066867-45 2024-04-07T16:17:32.092 INFO:tasks.rados.rados.0.smithi100.stdout:286: finishing write tid 6 to smithi10066867-45 2024-04-07T16:17:32.093 INFO:tasks.rados.rados.0.smithi100.stdout:286: oid 45 updating version 0 to 29 2024-04-07T16:17:32.093 INFO:tasks.rados.rados.0.smithi100.stdout:286: oid 45 version 29 is already newer than 28 2024-04-07T16:17:32.093 INFO:tasks.rados.rados.0.smithi100.stdout:286: oid 45 updating version 29 to 30 2024-04-07T16:17:32.093 INFO:tasks.rados.rados.0.smithi100.stdout:286: oid 45 updating version 30 to 31 2024-04-07T16:17:32.093 INFO:tasks.rados.rados.0.smithi100.stdout:286: oid 45 version 31 is already newer than 27 2024-04-07T16:17:32.093 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 45 v 31 (ObjNum 111 snap 21 seq_num 111) dirty exists 2024-04-07T16:17:32.093 INFO:tasks.rados.rados.0.smithi100.stdout:286: left oid 45 (ObjNum 111 snap 21 seq_num 111) 2024-04-07T16:17:32.093 INFO:tasks.rados.rados.0.smithi100.stdout:279: done (8 left) 2024-04-07T16:17:32.093 INFO:tasks.rados.rados.0.smithi100.stdout:280: done (7 left) 2024-04-07T16:17:32.093 INFO:tasks.rados.rados.0.smithi100.stdout:281: done (6 left) 2024-04-07T16:17:32.093 INFO:tasks.rados.rados.0.smithi100.stdout:282: done (5 left) 2024-04-07T16:17:32.093 INFO:tasks.rados.rados.0.smithi100.stdout:283: done (4 left) 2024-04-07T16:17:32.093 INFO:tasks.rados.rados.0.smithi100.stdout:284: done (3 left) 2024-04-07T16:17:32.093 INFO:tasks.rados.rados.0.smithi100.stdout:285: done (2 left) 2024-04-07T16:17:32.093 INFO:tasks.rados.rados.0.smithi100.stdout:286: done (1 left) 2024-04-07T16:17:32.094 INFO:tasks.rados.rados.0.smithi100.stdout:288: rollback oid 17 current snap is 21 2024-04-07T16:17:32.094 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 17 to 20 2024-04-07T16:17:32.094 INFO:tasks.rados.rados.0.smithi100.stdout:287: finishing copy_from racing read to smithi10066867-12 2024-04-07T16:17:32.094 INFO:tasks.rados.rados.0.smithi100.stdout:287: finishing copy_from to smithi10066867-12 2024-04-07T16:17:32.094 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 12 v 107 (ObjNum 108 snap 20 seq_num 108) dirty exists 2024-04-07T16:17:32.094 INFO:tasks.rados.rados.0.smithi100.stdout:287: done (1 left) 2024-04-07T16:17:32.094 INFO:tasks.rados.rados.0.smithi100.stdout:289: rollback oid 50 current snap is 21 2024-04-07T16:17:32.094 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 50 to 20 2024-04-07T16:17:32.094 INFO:tasks.rados.rados.0.smithi100.stdout:290: rmattr oid 5 current snap is 21 2024-04-07T16:17:32.094 INFO:tasks.rados.rados.0.smithi100.stdout:290: done (2 left) 2024-04-07T16:17:32.094 INFO:tasks.rados.rados.0.smithi100.stdout:291: rollback oid 27 current snap is 21 2024-04-07T16:17:32.094 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 27 to 20 2024-04-07T16:17:32.094 INFO:tasks.rados.rados.0.smithi100.stdout:292: rmattr oid 38 current snap is 21 2024-04-07T16:17:32.094 INFO:tasks.rados.rados.0.smithi100.stdout:292: done (3 left) 2024-04-07T16:17:32.095 INFO:tasks.rados.rados.0.smithi100.stdout:293: copy_from oid 39 from oid 13 current snap is 21 2024-04-07T16:17:32.095 INFO:tasks.rados.rados.0.smithi100.stdout:294: rollback oid 2 current snap is 21 2024-04-07T16:17:32.095 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 2 to 20 2024-04-07T16:17:32.095 INFO:tasks.rados.rados.0.smithi100.stdout:295: snap_create 2024-04-07T16:17:32.095 INFO:tasks.rados.rados.0.smithi100.stdout:291: finishing rollback tid 0 to smithi10066867-27 2024-04-07T16:17:32.096 INFO:tasks.rados.rados.0.smithi100.stdout:291: finishing rollback tid 1 to smithi10066867-27 2024-04-07T16:17:32.097 INFO:tasks.rados.rados.0.smithi100.stdout:294: finishing rollback tid 0 to smithi10066867-2 2024-04-07T16:17:32.097 INFO:tasks.rados.rados.0.smithi100.stdout:294: finishing rollback tid 1 to smithi10066867-2 2024-04-07T16:17:32.098 INFO:tasks.rados.rados.0.smithi100.stdout:288: finishing rollback tid 0 to smithi10066867-17 2024-04-07T16:17:32.098 INFO:tasks.rados.rados.0.smithi100.stdout:294: finishing rollback tid 2 to smithi10066867-2 2024-04-07T16:17:32.098 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 2 v 111 (ObjNum 108 snap 20 seq_num 108) dirty exists 2024-04-07T16:17:32.098 INFO:tasks.rados.rados.0.smithi100.stdout:288: finishing rollback tid 1 to smithi10066867-17 2024-04-07T16:17:32.099 INFO:tasks.rados.rados.0.smithi100.stdout:288: finishing rollback tid 2 to smithi10066867-17 2024-04-07T16:17:32.099 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 17 v 107 (ObjNum 98 snap 18 seq_num 98) dirty exists 2024-04-07T16:17:32.100 INFO:tasks.rados.rados.0.smithi100.stdout:291: finishing rollback tid 2 to smithi10066867-27 2024-04-07T16:17:32.100 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 27 v 30 (ObjNum 56 snap 4 seq_num 56) dirty exists 2024-04-07T16:17:32.100 INFO:tasks.rados.rados.0.smithi100.stdout:289: finishing rollback tid 0 to smithi10066867-50 2024-04-07T16:17:32.100 INFO:tasks.rados.rados.0.smithi100.stdout:289: finishing rollback tid 1 to smithi10066867-50 2024-04-07T16:17:32.101 INFO:tasks.rados.rados.0.smithi100.stdout:289: finishing rollback tid 2 to smithi10066867-50 2024-04-07T16:17:32.101 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 50 v 35 (ObjNum 49 snap 0 seq_num 49) dirty exists 2024-04-07T16:17:32.106 INFO:tasks.rados.rados.0.smithi100.stdout:293: finishing copy_from racing read to smithi10066867-39 2024-04-07T16:17:32.110 INFO:tasks.rados.rados.0.smithi100.stdout:293: finishing copy_from to smithi10066867-39 2024-04-07T16:17:32.110 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 39 v 28 (ObjNum 72 snap 11 seq_num 72) dirty exists 2024-04-07T16:17:32.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:17:31 smithi100 bash[21346]: cluster 2024-04-07T16:17:30.705214+0000 mgr.y (mgr.14553) 205 : cluster 0 pgmap v192: 121 pgs: 121 active+clean; 50 MiB data, 316 MiB used, 715 GiB / 715 GiB avail; 7.4 MiB/s rd, 1.9 MiB/s wr, 30 op/s 2024-04-07T16:17:32.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:17:31 smithi100 bash[21346]: cluster 2024-04-07T16:17:30.901218+0000 mon.a (mon.0) 937 : cluster 0 osdmap e116: 8 total, 8 up, 8 in 2024-04-07T16:17:32.188 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:17:31 smithi100 bash[31660]: cluster 2024-04-07T16:17:30.705214+0000 mgr.y (mgr.14553) 205 : cluster 0 pgmap v192: 121 pgs: 121 active+clean; 50 MiB data, 316 MiB used, 715 GiB / 715 GiB avail; 7.4 MiB/s rd, 1.9 MiB/s wr, 30 op/s 2024-04-07T16:17:32.188 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:17:31 smithi100 bash[31660]: cluster 2024-04-07T16:17:30.901218+0000 mon.a (mon.0) 937 : cluster 0 osdmap e116: 8 total, 8 up, 8 in 2024-04-07T16:17:32.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:17:31 smithi144 bash[23522]: cluster 2024-04-07T16:17:30.705214+0000 mgr.y (mgr.14553) 205 : cluster 0 pgmap v192: 121 pgs: 121 active+clean; 50 MiB data, 316 MiB used, 715 GiB / 715 GiB avail; 7.4 MiB/s rd, 1.9 MiB/s wr, 30 op/s 2024-04-07T16:17:32.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:17:31 smithi144 bash[23522]: cluster 2024-04-07T16:17:30.901218+0000 mon.a (mon.0) 937 : cluster 0 osdmap e116: 8 total, 8 up, 8 in 2024-04-07T16:17:32.827 INFO:tasks.rados.rados.0.smithi100.stdout:288: done (5 left) 2024-04-07T16:17:32.827 INFO:tasks.rados.rados.0.smithi100.stdout:289: done (4 left) 2024-04-07T16:17:32.827 INFO:tasks.rados.rados.0.smithi100.stdout:291: done (3 left) 2024-04-07T16:17:32.827 INFO:tasks.rados.rados.0.smithi100.stdout:293: done (2 left) 2024-04-07T16:17:32.827 INFO:tasks.rados.rados.0.smithi100.stdout:294: done (1 left) 2024-04-07T16:17:32.828 INFO:tasks.rados.rados.0.smithi100.stdout:295: done (0 left) 2024-04-07T16:17:32.828 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 16 current snap is 22 2024-04-07T16:17:32.828 INFO:tasks.rados.rados.0.smithi100.stdout:296: seq_num 112 ranges {0=679936,679936=548864,1228800=655360} 2024-04-07T16:17:32.838 INFO:tasks.rados.rados.0.smithi100.stdout:296: writing smithi10066867-16 from 0 to 679936 tid 1 2024-04-07T16:17:32.847 INFO:tasks.rados.rados.0.smithi100.stdout:296: writing smithi10066867-16 from 679936 to 1228800 tid 2 2024-04-07T16:17:32.852 INFO:tasks.rados.rados.0.smithi100.stdout:296: writing smithi10066867-16 from 1228800 to 1884160 tid 3 2024-04-07T16:17:32.853 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 18 current snap is 22 2024-04-07T16:17:32.853 INFO:tasks.rados.rados.0.smithi100.stdout:296: finishing write tid 1 to smithi10066867-16 2024-04-07T16:17:32.853 INFO:tasks.rados.rados.0.smithi100.stdout:297: seq_num 113 ranges {1597440=696320,2293760=434176,2727936=466944,3194880=303104} 2024-04-07T16:17:32.872 INFO:tasks.rados.rados.0.smithi100.stdout:297: writing smithi10066867-18 from 1597440 to 2293760 tid 1 2024-04-07T16:17:32.876 INFO:tasks.rados.rados.0.smithi100.stdout:297: writing smithi10066867-18 from 2293760 to 2727936 tid 2 2024-04-07T16:17:32.879 INFO:tasks.rados.rados.0.smithi100.stdout:297: writing smithi10066867-18 from 2727936 to 3194880 tid 3 2024-04-07T16:17:32.883 INFO:tasks.rados.rados.0.smithi100.stdout:297: writing smithi10066867-18 from 3194880 to 3497984 tid 4 2024-04-07T16:17:32.883 INFO:tasks.rados.rados.0.smithi100.stdout:298: setattr oid 45 current snap is 22 2024-04-07T16:17:32.883 INFO:tasks.rados.rados.0.smithi100.stdout:296: finishing write tid 2 to smithi10066867-16 2024-04-07T16:17:32.883 INFO:tasks.rados.rados.0.smithi100.stdout:296: finishing write tid 3 to smithi10066867-16 2024-04-07T16:17:32.883 INFO:tasks.rados.rados.0.smithi100.stdout:296: finishing write tid 4 to smithi10066867-16 2024-04-07T16:17:32.883 INFO:tasks.rados.rados.0.smithi100.stdout:296: finishing write tid 5 to smithi10066867-16 2024-04-07T16:17:32.883 INFO:tasks.rados.rados.0.smithi100.stdout:296: oid 16 updating version 0 to 111 2024-04-07T16:17:32.883 INFO:tasks.rados.rados.0.smithi100.stdout:296: oid 16 version 111 is already newer than 110 2024-04-07T16:17:32.883 INFO:tasks.rados.rados.0.smithi100.stdout:296: oid 16 version 111 is already newer than 109 2024-04-07T16:17:32.884 INFO:tasks.rados.rados.0.smithi100.stdout:296: oid 16 version 111 is already newer than 108 2024-04-07T16:17:32.884 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 16 v 111 (ObjNum 112 snap 22 seq_num 112) dirty exists 2024-04-07T16:17:32.884 INFO:tasks.rados.rados.0.smithi100.stdout:296: left oid 16 (ObjNum 112 snap 22 seq_num 112) 2024-04-07T16:17:32.884 INFO:tasks.rados.rados.0.smithi100.stdout:297: finishing write tid 1 to smithi10066867-18 2024-04-07T16:17:32.884 INFO:tasks.rados.rados.0.smithi100.stdout:297: finishing write tid 2 to smithi10066867-18 2024-04-07T16:17:32.885 INFO:tasks.rados.rados.0.smithi100.stdout:296: done (2 left) 2024-04-07T16:17:32.885 INFO:tasks.rados.rados.0.smithi100.stdout:299: snap_remove snap 20 2024-04-07T16:17:33.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:17:32 smithi100 bash[21346]: cluster 2024-04-07T16:17:31.914798+0000 mon.a (mon.0) 938 : cluster 0 osdmap e117: 8 total, 8 up, 8 in 2024-04-07T16:17:33.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:17:32 smithi100 bash[21346]: cluster 2024-04-07T16:17:32.837266+0000 mon.a (mon.0) 939 : cluster 0 osdmap e118: 8 total, 8 up, 8 in 2024-04-07T16:17:33.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:17:32 smithi100 bash[31660]: cluster 2024-04-07T16:17:31.914798+0000 mon.a (mon.0) 938 : cluster 0 osdmap e117: 8 total, 8 up, 8 in 2024-04-07T16:17:33.188 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:17:32 smithi100 bash[31660]: cluster 2024-04-07T16:17:32.837266+0000 mon.a (mon.0) 939 : cluster 0 osdmap e118: 8 total, 8 up, 8 in 2024-04-07T16:17:33.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:17:32 smithi144 bash[23522]: cluster 2024-04-07T16:17:31.914798+0000 mon.a (mon.0) 938 : cluster 0 osdmap e117: 8 total, 8 up, 8 in 2024-04-07T16:17:33.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:17:32 smithi144 bash[23522]: cluster 2024-04-07T16:17:32.837266+0000 mon.a (mon.0) 939 : cluster 0 osdmap e118: 8 total, 8 up, 8 in 2024-04-07T16:17:33.831 INFO:tasks.rados.rados.0.smithi100.stdout:299: done (2 left) 2024-04-07T16:17:33.832 INFO:tasks.rados.rados.0.smithi100.stdout:300: setattr oid 15 current snap is 22 2024-04-07T16:17:33.832 INFO:tasks.rados.rados.0.smithi100.stdout:297: finishing write tid 3 to smithi10066867-18 2024-04-07T16:17:33.832 INFO:tasks.rados.rados.0.smithi100.stdout:297: finishing write tid 4 to smithi10066867-18 2024-04-07T16:17:33.832 INFO:tasks.rados.rados.0.smithi100.stdout:297: finishing write tid 5 to smithi10066867-18 2024-04-07T16:17:33.832 INFO:tasks.rados.rados.0.smithi100.stdout:297: finishing write tid 6 to smithi10066867-18 2024-04-07T16:17:33.832 INFO:tasks.rados.rados.0.smithi100.stdout:297: oid 18 updating version 0 to 91 2024-04-07T16:17:33.832 INFO:tasks.rados.rados.0.smithi100.stdout:297: oid 18 version 91 is already newer than 90 2024-04-07T16:17:33.832 INFO:tasks.rados.rados.0.smithi100.stdout:297: oid 18 version 91 is already newer than 89 2024-04-07T16:17:33.832 INFO:tasks.rados.rados.0.smithi100.stdout:297: oid 18 updating version 91 to 93 2024-04-07T16:17:33.832 INFO:tasks.rados.rados.0.smithi100.stdout:297: oid 18 version 93 is already newer than 92 2024-04-07T16:17:33.832 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 18 v 93 (ObjNum 113 snap 22 seq_num 113) dirty exists 2024-04-07T16:17:33.832 INFO:tasks.rados.rados.0.smithi100.stdout:297: left oid 18 (ObjNum 113 snap 22 seq_num 113) 2024-04-07T16:17:33.832 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 45 v 37 (ObjNum 111 snap 21 seq_num 111) dirty exists 2024-04-07T16:17:33.834 INFO:tasks.rados.rados.0.smithi100.stdout:297: done (2 left) 2024-04-07T16:17:33.835 INFO:tasks.rados.rados.0.smithi100.stdout:298: done (1 left) 2024-04-07T16:17:33.835 INFO:tasks.rados.rados.0.smithi100.stdout:301: snap_create 2024-04-07T16:17:33.847 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 15 v 95 (ObjNum 4 snap 0 seq_num 4) dirty exists 2024-04-07T16:17:34.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:17:33 smithi100 bash[21346]: cluster 2024-04-07T16:17:32.705934+0000 mgr.y (mgr.14553) 206 : cluster 0 pgmap v195: 121 pgs: 121 active+clean; 50 MiB data, 316 MiB used, 715 GiB / 715 GiB avail; 7.4 MiB/s rd, 1.9 MiB/s wr, 30 op/s 2024-04-07T16:17:34.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:17:33 smithi100 bash[21346]: cluster 2024-04-07T16:17:33.842536+0000 mon.a (mon.0) 940 : cluster 0 osdmap e119: 8 total, 8 up, 8 in 2024-04-07T16:17:34.188 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:17:33 smithi100 bash[31660]: cluster 2024-04-07T16:17:32.705934+0000 mgr.y (mgr.14553) 206 : cluster 0 pgmap v195: 121 pgs: 121 active+clean; 50 MiB data, 316 MiB used, 715 GiB / 715 GiB avail; 7.4 MiB/s rd, 1.9 MiB/s wr, 30 op/s 2024-04-07T16:17:34.188 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:17:33 smithi100 bash[31660]: cluster 2024-04-07T16:17:33.842536+0000 mon.a (mon.0) 940 : cluster 0 osdmap e119: 8 total, 8 up, 8 in 2024-04-07T16:17:34.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:17:33 smithi144 bash[23522]: cluster 2024-04-07T16:17:32.705934+0000 mgr.y (mgr.14553) 206 : cluster 0 pgmap v195: 121 pgs: 121 active+clean; 50 MiB data, 316 MiB used, 715 GiB / 715 GiB avail; 7.4 MiB/s rd, 1.9 MiB/s wr, 30 op/s 2024-04-07T16:17:34.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:17:33 smithi144 bash[23522]: cluster 2024-04-07T16:17:33.842536+0000 mon.a (mon.0) 940 : cluster 0 osdmap e119: 8 total, 8 up, 8 in 2024-04-07T16:17:34.831 INFO:tasks.rados.rados.0.smithi100.stdout:300: done (1 left) 2024-04-07T16:17:34.831 INFO:tasks.rados.rados.0.smithi100.stdout:301: done (0 left) 2024-04-07T16:17:34.831 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 33 current snap is 23 2024-04-07T16:17:34.831 INFO:tasks.rados.rados.0.smithi100.stdout:302: seq_num 114 ranges {0=712704,712704=712704,1425408=491520} 2024-04-07T16:17:34.839 INFO:tasks.rados.rados.0.smithi100.stdout:302: writing smithi10066867-33 from 0 to 712704 tid 1 2024-04-07T16:17:34.850 INFO:tasks.rados.rados.0.smithi100.stdout:302: writing smithi10066867-33 from 712704 to 1425408 tid 2 2024-04-07T16:17:34.856 INFO:tasks.rados.rados.0.smithi100.stdout:302: writing smithi10066867-33 from 1425408 to 1916928 tid 3 2024-04-07T16:17:34.856 INFO:tasks.rados.rados.0.smithi100.stdout:303: copy_from oid 18 from oid 13 current snap is 23 2024-04-07T16:17:34.856 INFO:tasks.rados.rados.0.smithi100.stdout:302: finishing write tid 1 to smithi10066867-33 2024-04-07T16:17:34.857 INFO:tasks.rados.rados.0.smithi100.stdout:302: finishing write tid 2 to smithi10066867-33 2024-04-07T16:17:34.857 INFO:tasks.rados.rados.0.smithi100.stdout:304: snap_create 2024-04-07T16:17:34.861 INFO:tasks.rados.rados.0.smithi100.stdout:302: finishing write tid 3 to smithi10066867-33 2024-04-07T16:17:34.861 INFO:tasks.rados.rados.0.smithi100.stdout:302: finishing write tid 4 to smithi10066867-33 2024-04-07T16:17:34.864 INFO:tasks.rados.rados.0.smithi100.stdout:302: finishing write tid 5 to smithi10066867-33 2024-04-07T16:17:34.864 INFO:tasks.rados.rados.0.smithi100.stdout:302: oid 33 updating version 0 to 99 2024-04-07T16:17:34.864 INFO:tasks.rados.rados.0.smithi100.stdout:302: oid 33 updating version 99 to 101 2024-04-07T16:17:34.864 INFO:tasks.rados.rados.0.smithi100.stdout:302: oid 33 version 101 is already newer than 98 2024-04-07T16:17:34.864 INFO:tasks.rados.rados.0.smithi100.stdout:302: oid 33 version 101 is already newer than 100 2024-04-07T16:17:34.864 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 33 v 101 (ObjNum 114 snap 23 seq_num 114) dirty exists 2024-04-07T16:17:34.864 INFO:tasks.rados.rados.0.smithi100.stdout:302: left oid 33 (ObjNum 114 snap 23 seq_num 114) 2024-04-07T16:17:34.880 INFO:tasks.rados.rados.0.smithi100.stdout:303: finishing copy_from racing read to smithi10066867-18 2024-04-07T16:17:34.883 INFO:tasks.rados.rados.0.smithi100.stdout:303: finishing copy_from to smithi10066867-18 2024-04-07T16:17:34.883 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 18 v 28 (ObjNum 72 snap 11 seq_num 72) dirty exists 2024-04-07T16:17:35.437 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:17:35 smithi100 bash[21601]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:17:35] "GET /metrics HTTP/1.1" 200 36080 "" "Prometheus/2.43.0" 2024-04-07T16:17:35.835 INFO:tasks.rados.rados.0.smithi100.stdout:302: done (2 left) 2024-04-07T16:17:35.835 INFO:tasks.rados.rados.0.smithi100.stdout:303: done (1 left) 2024-04-07T16:17:35.835 INFO:tasks.rados.rados.0.smithi100.stdout:304: done (0 left) 2024-04-07T16:17:35.835 INFO:tasks.rados.rados.0.smithi100.stdout:305: snap_create 2024-04-07T16:17:36.105 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:17:35 smithi144 bash[23522]: cluster 2024-04-07T16:17:34.706617+0000 mgr.y (mgr.14553) 207 : cluster 0 pgmap v198: 121 pgs: 121 active+clean; 59 MiB data, 337 MiB used, 715 GiB / 715 GiB avail; 3.4 MiB/s rd, 4.3 MiB/s wr, 28 op/s 2024-04-07T16:17:36.105 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:17:35 smithi144 bash[23522]: cluster 2024-04-07T16:17:34.843955+0000 mon.a (mon.0) 941 : cluster 0 osdmap e120: 8 total, 8 up, 8 in 2024-04-07T16:17:36.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:17:35 smithi100 bash[21346]: cluster 2024-04-07T16:17:34.706617+0000 mgr.y (mgr.14553) 207 : cluster 0 pgmap v198: 121 pgs: 121 active+clean; 59 MiB data, 337 MiB used, 715 GiB / 715 GiB avail; 3.4 MiB/s rd, 4.3 MiB/s wr, 28 op/s 2024-04-07T16:17:36.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:17:35 smithi100 bash[21346]: cluster 2024-04-07T16:17:34.843955+0000 mon.a (mon.0) 941 : cluster 0 osdmap e120: 8 total, 8 up, 8 in 2024-04-07T16:17:36.188 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:17:35 smithi100 bash[31660]: cluster 2024-04-07T16:17:34.706617+0000 mgr.y (mgr.14553) 207 : cluster 0 pgmap v198: 121 pgs: 121 active+clean; 59 MiB data, 337 MiB used, 715 GiB / 715 GiB avail; 3.4 MiB/s rd, 4.3 MiB/s wr, 28 op/s 2024-04-07T16:17:36.188 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:17:35 smithi100 bash[31660]: cluster 2024-04-07T16:17:34.843955+0000 mon.a (mon.0) 941 : cluster 0 osdmap e120: 8 total, 8 up, 8 in 2024-04-07T16:17:36.860 INFO:tasks.rados.rados.0.smithi100.stdout:305: done (0 left) 2024-04-07T16:17:36.860 INFO:tasks.rados.rados.0.smithi100.stdout:306: read oid 15 snap -1 2024-04-07T16:17:36.860 INFO:tasks.rados.rados.0.smithi100.stdout:306: expect (ObjNum 4 snap 0 seq_num 4) 2024-04-07T16:17:36.860 INFO:tasks.rados.rados.0.smithi100.stdout:307: snap_create 2024-04-07T16:17:36.871 INFO:tasks.rados.rados.0.smithi100.stdout:306: expect (ObjNum 4 snap 0 seq_num 4) 2024-04-07T16:17:37.105 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:17:36 smithi144 bash[23522]: cluster 2024-04-07T16:17:35.847570+0000 mon.a (mon.0) 942 : cluster 0 osdmap e121: 8 total, 8 up, 8 in 2024-04-07T16:17:37.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:17:36 smithi100 bash[21346]: cluster 2024-04-07T16:17:35.847570+0000 mon.a (mon.0) 942 : cluster 0 osdmap e121: 8 total, 8 up, 8 in 2024-04-07T16:17:37.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:17:36 smithi100 bash[31660]: cluster 2024-04-07T16:17:35.847570+0000 mon.a (mon.0) 942 : cluster 0 osdmap e121: 8 total, 8 up, 8 in 2024-04-07T16:17:37.830 INFO:tasks.rados.rados.0.smithi100.stdout:306: done (1 left) 2024-04-07T16:17:37.830 INFO:tasks.rados.rados.0.smithi100.stdout:307: done (0 left) 2024-04-07T16:17:37.830 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 23 current snap is 26 2024-04-07T16:17:37.830 INFO:tasks.rados.rados.0.smithi100.stdout:308: seq_num 115 ranges {1269760=729088,1998848=647168,2646016=434176,3080192=122880} 2024-04-07T16:17:37.862 INFO:tasks.rados.rados.0.smithi100.stdout:308: writing smithi10066867-23 from 1269760 to 1998848 tid 1 2024-04-07T16:17:37.875 INFO:tasks.rados.rados.0.smithi100.stdout:308: writing smithi10066867-23 from 1998848 to 2646016 tid 2 2024-04-07T16:17:37.882 INFO:tasks.rados.rados.0.smithi100.stdout:308: writing smithi10066867-23 from 2646016 to 3080192 tid 3 2024-04-07T16:17:37.884 INFO:tasks.rados.rados.0.smithi100.stdout:308: writing smithi10066867-23 from 3080192 to 3203072 tid 4 2024-04-07T16:17:37.885 INFO:tasks.rados.rados.0.smithi100.stdout:309: rollback oid 9 current snap is 26 2024-04-07T16:17:37.885 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 9 to 25 2024-04-07T16:17:37.885 INFO:tasks.rados.rados.0.smithi100.stdout:308: finishing write tid 1 to smithi10066867-23 2024-04-07T16:17:37.885 INFO:tasks.rados.rados.0.smithi100.stdout:308: finishing write tid 2 to smithi10066867-23 2024-04-07T16:17:37.885 INFO:tasks.rados.rados.0.smithi100.stdout:310: snap_remove snap 23 2024-04-07T16:17:38.105 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:17:37 smithi144 bash[23522]: cluster 2024-04-07T16:17:36.707302+0000 mgr.y (mgr.14553) 208 : cluster 0 pgmap v201: 121 pgs: 121 active+clean; 59 MiB data, 337 MiB used, 715 GiB / 715 GiB avail; 3.4 MiB/s rd, 4.3 MiB/s wr, 28 op/s 2024-04-07T16:17:38.105 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:17:37 smithi144 bash[23522]: cluster 2024-04-07T16:17:36.874017+0000 mon.a (mon.0) 943 : cluster 0 osdmap e122: 8 total, 8 up, 8 in 2024-04-07T16:17:38.105 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:17:37 smithi144 bash[23522]: audit 2024-04-07T16:17:37.704191+0000 mon.b (mon.1) 243 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:17:38.105 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:17:37 smithi144 bash[23522]: cluster 2024-04-07T16:17:37.844404+0000 mon.a (mon.0) 944 : cluster 0 osdmap e123: 8 total, 8 up, 8 in 2024-04-07T16:17:38.192 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:17:37 smithi100 bash[21346]: cluster 2024-04-07T16:17:36.707302+0000 mgr.y (mgr.14553) 208 : cluster 0 pgmap v201: 121 pgs: 121 active+clean; 59 MiB data, 337 MiB used, 715 GiB / 715 GiB avail; 3.4 MiB/s rd, 4.3 MiB/s wr, 28 op/s 2024-04-07T16:17:38.192 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:17:37 smithi100 bash[21346]: cluster 2024-04-07T16:17:36.874017+0000 mon.a (mon.0) 943 : cluster 0 osdmap e122: 8 total, 8 up, 8 in 2024-04-07T16:17:38.192 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:17:37 smithi100 bash[21346]: audit 2024-04-07T16:17:37.704191+0000 mon.b (mon.1) 243 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:17:38.192 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:17:37 smithi100 bash[21346]: cluster 2024-04-07T16:17:37.844404+0000 mon.a (mon.0) 944 : cluster 0 osdmap e123: 8 total, 8 up, 8 in 2024-04-07T16:17:38.192 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:17:37 smithi100 bash[31660]: cluster 2024-04-07T16:17:36.707302+0000 mgr.y (mgr.14553) 208 : cluster 0 pgmap v201: 121 pgs: 121 active+clean; 59 MiB data, 337 MiB used, 715 GiB / 715 GiB avail; 3.4 MiB/s rd, 4.3 MiB/s wr, 28 op/s 2024-04-07T16:17:38.192 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:17:37 smithi100 bash[31660]: cluster 2024-04-07T16:17:36.874017+0000 mon.a (mon.0) 943 : cluster 0 osdmap e122: 8 total, 8 up, 8 in 2024-04-07T16:17:38.192 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:17:37 smithi100 bash[31660]: audit 2024-04-07T16:17:37.704191+0000 mon.b (mon.1) 243 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:17:38.193 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:17:37 smithi100 bash[31660]: cluster 2024-04-07T16:17:37.844404+0000 mon.a (mon.0) 944 : cluster 0 osdmap e123: 8 total, 8 up, 8 in 2024-04-07T16:17:38.866 INFO:tasks.rados.rados.0.smithi100.stdout:310: done (2 left) 2024-04-07T16:17:38.867 INFO:tasks.rados.rados.0.smithi100.stdout:308: finishing write tid 3 to smithi10066867-23 2024-04-07T16:17:38.867 INFO:tasks.rados.rados.0.smithi100.stdout:308: finishing write tid 4 to smithi10066867-23 2024-04-07T16:17:38.867 INFO:tasks.rados.rados.0.smithi100.stdout:309: finishing rollback tid 0 to smithi10066867-9 2024-04-07T16:17:38.867 INFO:tasks.rados.rados.0.smithi100.stdout:309: finishing rollback tid 1 to smithi10066867-9 2024-04-07T16:17:38.867 INFO:tasks.rados.rados.0.smithi100.stdout:308: finishing write tid 5 to smithi10066867-23 2024-04-07T16:17:38.867 INFO:tasks.rados.rados.0.smithi100.stdout:309: finishing rollback tid 2 to smithi10066867-9 2024-04-07T16:17:38.867 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 9 v 43 (ObjNum 72 snap 11 seq_num 72) dirty exists 2024-04-07T16:17:38.867 INFO:tasks.rados.rados.0.smithi100.stdout:308: finishing write tid 6 to smithi10066867-23 2024-04-07T16:17:38.867 INFO:tasks.rados.rados.0.smithi100.stdout:308: oid 23 updating version 0 to 108 2024-04-07T16:17:38.867 INFO:tasks.rados.rados.0.smithi100.stdout:308: oid 23 version 108 is already newer than 107 2024-04-07T16:17:38.867 INFO:tasks.rados.rados.0.smithi100.stdout:308: oid 23 version 108 is already newer than 105 2024-04-07T16:17:38.867 INFO:tasks.rados.rados.0.smithi100.stdout:308: oid 23 updating version 108 to 109 2024-04-07T16:17:38.867 INFO:tasks.rados.rados.0.smithi100.stdout:308: oid 23 version 109 is already newer than 106 2024-04-07T16:17:38.867 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 23 v 109 (ObjNum 115 snap 26 seq_num 115) dirty exists 2024-04-07T16:17:38.868 INFO:tasks.rados.rados.0.smithi100.stdout:308: left oid 23 (ObjNum 115 snap 26 seq_num 115) 2024-04-07T16:17:38.868 INFO:tasks.rados.rados.0.smithi100.stdout:311: read oid 18 snap 24 2024-04-07T16:17:38.868 INFO:tasks.rados.rados.0.smithi100.stdout:311: expect (ObjNum 72 snap 11 seq_num 72) 2024-04-07T16:17:38.868 INFO:tasks.rados.rados.0.smithi100.stdout:308: done (2 left) 2024-04-07T16:17:38.868 INFO:tasks.rados.rados.0.smithi100.stdout:309: done (1 left) 2024-04-07T16:17:38.868 INFO:tasks.rados.rados.0.smithi100.stdout:312: snap_create 2024-04-07T16:17:38.882 INFO:tasks.rados.rados.0.smithi100.stdout:311: expect (ObjNum 72 snap 11 seq_num 72) 2024-04-07T16:17:39.880 INFO:tasks.rados.rados.0.smithi100.stdout:311: done (1 left) 2024-04-07T16:17:39.880 INFO:tasks.rados.rados.0.smithi100.stdout:312: done (0 left) 2024-04-07T16:17:39.880 INFO:tasks.rados.rados.0.smithi100.stdout:313: read oid 23 snap -1 2024-04-07T16:17:39.880 INFO:tasks.rados.rados.0.smithi100.stdout:313: expect (ObjNum 115 snap 26 seq_num 115) 2024-04-07T16:17:39.880 INFO:tasks.rados.rados.0.smithi100.stdout:314: snap_remove snap 21 2024-04-07T16:17:40.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:17:39 smithi100 bash[21346]: cluster 2024-04-07T16:17:38.707936+0000 mgr.y (mgr.14553) 209 : cluster 0 pgmap v204: 121 pgs: 121 active+clean; 59 MiB data, 337 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:17:40.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:17:39 smithi100 bash[21346]: cluster 2024-04-07T16:17:38.878242+0000 mon.a (mon.0) 945 : cluster 0 osdmap e124: 8 total, 8 up, 8 in 2024-04-07T16:17:40.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:17:39 smithi100 bash[31660]: cluster 2024-04-07T16:17:38.707936+0000 mgr.y (mgr.14553) 209 : cluster 0 pgmap v204: 121 pgs: 121 active+clean; 59 MiB data, 337 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:17:40.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:17:39 smithi100 bash[31660]: cluster 2024-04-07T16:17:38.878242+0000 mon.a (mon.0) 945 : cluster 0 osdmap e124: 8 total, 8 up, 8 in 2024-04-07T16:17:40.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:17:39 smithi144 bash[23522]: cluster 2024-04-07T16:17:38.707936+0000 mgr.y (mgr.14553) 209 : cluster 0 pgmap v204: 121 pgs: 121 active+clean; 59 MiB data, 337 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:17:40.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:17:39 smithi144 bash[23522]: cluster 2024-04-07T16:17:38.878242+0000 mon.a (mon.0) 945 : cluster 0 osdmap e124: 8 total, 8 up, 8 in 2024-04-07T16:17:40.893 INFO:tasks.rados.rados.0.smithi100.stdout:314: done (1 left) 2024-04-07T16:17:40.894 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 45 current snap is 27 2024-04-07T16:17:40.894 INFO:tasks.rados.rados.0.smithi100.stdout:315: seq_num 116 ranges {2613248=745472,3358720=532480,3891200=638976,4530176=32768} 2024-04-07T16:17:40.946 INFO:tasks.rados.rados.0.smithi100.stdout:315: writing smithi10066867-45 from 2613248 to 3358720 tid 1 2024-04-07T16:17:40.951 INFO:tasks.rados.rados.0.smithi100.stdout:315: writing smithi10066867-45 from 3358720 to 3891200 tid 2 2024-04-07T16:17:40.958 INFO:tasks.rados.rados.0.smithi100.stdout:315: writing smithi10066867-45 from 3891200 to 4530176 tid 3 2024-04-07T16:17:40.958 INFO:tasks.rados.rados.0.smithi100.stdout:315: writing smithi10066867-45 from 4530176 to 4562944 tid 4 2024-04-07T16:17:40.958 INFO:tasks.rados.rados.0.smithi100.stdout:316: rollback oid 35 current snap is 27 2024-04-07T16:17:40.958 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 35 to 22 2024-04-07T16:17:40.959 INFO:tasks.rados.rados.0.smithi100.stdout:313: expect (ObjNum 115 snap 26 seq_num 115) 2024-04-07T16:17:41.075 INFO:tasks.rados.rados.0.smithi100.stdout:315: finishing write tid 1 to smithi10066867-45 2024-04-07T16:17:41.075 INFO:tasks.rados.rados.0.smithi100.stdout:315: finishing write tid 2 to smithi10066867-45 2024-04-07T16:17:41.075 INFO:tasks.rados.rados.0.smithi100.stdout:315: finishing write tid 3 to smithi10066867-45 2024-04-07T16:17:41.075 INFO:tasks.rados.rados.0.smithi100.stdout:316: finishing rollback tid 0 to smithi10066867-35 2024-04-07T16:17:41.075 INFO:tasks.rados.rados.0.smithi100.stdout:316: finishing rollback tid 1 to smithi10066867-35 2024-04-07T16:17:41.075 INFO:tasks.rados.rados.0.smithi100.stdout:315: finishing write tid 4 to smithi10066867-45 2024-04-07T16:17:41.075 INFO:tasks.rados.rados.0.smithi100.stdout:315: finishing write tid 5 to smithi10066867-45 2024-04-07T16:17:41.075 INFO:tasks.rados.rados.0.smithi100.stdout:316: finishing rollback tid 2 to smithi10066867-35 2024-04-07T16:17:41.075 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 35 v 63 (ObjNum 4 snap 0 seq_num 4) dirty exists 2024-04-07T16:17:41.075 INFO:tasks.rados.rados.0.smithi100.stdout:315: finishing write tid 6 to smithi10066867-45 2024-04-07T16:17:41.075 INFO:tasks.rados.rados.0.smithi100.stdout:315: oid 45 updating version 0 to 45 2024-04-07T16:17:41.076 INFO:tasks.rados.rados.0.smithi100.stdout:315: oid 45 updating version 45 to 48 2024-04-07T16:17:41.076 INFO:tasks.rados.rados.0.smithi100.stdout:315: oid 45 updating version 48 to 49 2024-04-07T16:17:41.076 INFO:tasks.rados.rados.0.smithi100.stdout:315: oid 45 version 49 is already newer than 47 2024-04-07T16:17:41.076 INFO:tasks.rados.rados.0.smithi100.stdout:315: oid 45 version 49 is already newer than 46 2024-04-07T16:17:41.076 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 45 v 49 (ObjNum 116 snap 27 seq_num 116) dirty exists 2024-04-07T16:17:41.076 INFO:tasks.rados.rados.0.smithi100.stdout:315: left oid 45 (ObjNum 116 snap 27 seq_num 116) 2024-04-07T16:17:41.076 INFO:tasks.rados.rados.0.smithi100.stdout:313: done (2 left) 2024-04-07T16:17:41.076 INFO:tasks.rados.rados.0.smithi100.stdout:315: done (1 left) 2024-04-07T16:17:41.076 INFO:tasks.rados.rados.0.smithi100.stdout:316: done (0 left) 2024-04-07T16:17:41.076 INFO:tasks.rados.rados.0.smithi100.stdout:317: snap_create 2024-04-07T16:17:41.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:17:40 smithi100 bash[21346]: cluster 2024-04-07T16:17:39.890702+0000 mon.a (mon.0) 946 : cluster 0 osdmap e125: 8 total, 8 up, 8 in 2024-04-07T16:17:41.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:17:40 smithi100 bash[31660]: cluster 2024-04-07T16:17:39.890702+0000 mon.a (mon.0) 946 : cluster 0 osdmap e125: 8 total, 8 up, 8 in 2024-04-07T16:17:41.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:17:40 smithi144 bash[23522]: cluster 2024-04-07T16:17:39.890702+0000 mon.a (mon.0) 946 : cluster 0 osdmap e125: 8 total, 8 up, 8 in 2024-04-07T16:17:41.891 INFO:tasks.rados.rados.0.smithi100.stdout:317: done (0 left) 2024-04-07T16:17:41.891 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 18 current snap is 28 2024-04-07T16:17:41.891 INFO:tasks.rados.rados.0.smithi100.stdout:318: seq_num 117 ranges {1417216=770048,2187264=417792,2605056=442368,3047424=344064} 2024-04-07T16:17:41.919 INFO:tasks.rados.rados.0.smithi100.stdout:318: writing smithi10066867-18 from 1417216 to 2187264 tid 1 2024-04-07T16:17:41.925 INFO:tasks.rados.rados.0.smithi100.stdout:318: writing smithi10066867-18 from 2187264 to 2605056 tid 2 2024-04-07T16:17:41.932 INFO:tasks.rados.rados.0.smithi100.stdout:318: writing smithi10066867-18 from 2605056 to 3047424 tid 3 2024-04-07T16:17:41.938 INFO:tasks.rados.rados.0.smithi100.stdout:318: writing smithi10066867-18 from 3047424 to 3391488 tid 4 2024-04-07T16:17:41.938 INFO:tasks.rados.rados.0.smithi100.stdout:319: read oid 31 snap -1 2024-04-07T16:17:41.938 INFO:tasks.rados.rados.0.smithi100.stdout:319: expect deleted 2024-04-07T16:17:41.938 INFO:tasks.rados.rados.0.smithi100.stdout:320: read oid 5 snap -1 2024-04-07T16:17:41.938 INFO:tasks.rados.rados.0.smithi100.stdout:320: expect deleted 2024-04-07T16:17:41.938 INFO:tasks.rados.rados.0.smithi100.stdout:321: copy_from oid 21 from oid 32 current snap is 28 2024-04-07T16:17:41.938 INFO:tasks.rados.rados.0.smithi100.stdout:318: finishing write tid 1 to smithi10066867-18 2024-04-07T16:17:41.938 INFO:tasks.rados.rados.0.smithi100.stdout:318: finishing write tid 2 to smithi10066867-18 2024-04-07T16:17:41.938 INFO:tasks.rados.rados.0.smithi100.stdout:318: finishing write tid 3 to smithi10066867-18 2024-04-07T16:17:41.938 INFO:tasks.rados.rados.0.smithi100.stdout:322: snap_create 2024-04-07T16:17:41.941 INFO:tasks.rados.rados.0.smithi100.stdout:318: finishing write tid 4 to smithi10066867-18 2024-04-07T16:17:41.942 INFO:tasks.rados.rados.0.smithi100.stdout:318: finishing write tid 5 to smithi10066867-18 2024-04-07T16:17:41.944 INFO:tasks.rados.rados.0.smithi100.stdout:318: finishing write tid 6 to smithi10066867-18 2024-04-07T16:17:41.944 INFO:tasks.rados.rados.0.smithi100.stdout:318: oid 18 updating version 0 to 123 2024-04-07T16:17:41.944 INFO:tasks.rados.rados.0.smithi100.stdout:318: oid 18 version 123 is already newer than 120 2024-04-07T16:17:41.945 INFO:tasks.rados.rados.0.smithi100.stdout:318: oid 18 version 123 is already newer than 119 2024-04-07T16:17:41.945 INFO:tasks.rados.rados.0.smithi100.stdout:318: oid 18 version 123 is already newer than 121 2024-04-07T16:17:41.945 INFO:tasks.rados.rados.0.smithi100.stdout:318: oid 18 version 123 is already newer than 122 2024-04-07T16:17:41.945 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 18 v 123 (ObjNum 117 snap 28 seq_num 117) dirty exists 2024-04-07T16:17:41.945 INFO:tasks.rados.rados.0.smithi100.stdout:318: left oid 18 (ObjNum 117 snap 28 seq_num 117) 2024-04-07T16:17:41.951 INFO:tasks.rados.rados.0.smithi100.stdout:321: finishing copy_from racing read to smithi10066867-21 2024-04-07T16:17:41.952 INFO:tasks.rados.rados.0.smithi100.stdout:321: finishing copy_from to smithi10066867-21 2024-04-07T16:17:41.952 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 21 v 2 (ObjNum 16 snap 0 seq_num 16) dirty exists 2024-04-07T16:17:42.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:17:41 smithi100 bash[21346]: cluster 2024-04-07T16:17:40.708651+0000 mgr.y (mgr.14553) 210 : cluster 0 pgmap v207: 121 pgs: 121 active+clean; 61 MiB data, 341 MiB used, 715 GiB / 715 GiB avail; 70 KiB/s rd, 1.3 MiB/s wr, 9 op/s 2024-04-07T16:17:42.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:17:41 smithi100 bash[21346]: cluster 2024-04-07T16:17:40.907885+0000 mon.a (mon.0) 947 : cluster 0 osdmap e126: 8 total, 8 up, 8 in 2024-04-07T16:17:42.188 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:17:41 smithi100 bash[31660]: cluster 2024-04-07T16:17:40.708651+0000 mgr.y (mgr.14553) 210 : cluster 0 pgmap v207: 121 pgs: 121 active+clean; 61 MiB data, 341 MiB used, 715 GiB / 715 GiB avail; 70 KiB/s rd, 1.3 MiB/s wr, 9 op/s 2024-04-07T16:17:42.188 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:17:41 smithi100 bash[31660]: cluster 2024-04-07T16:17:40.907885+0000 mon.a (mon.0) 947 : cluster 0 osdmap e126: 8 total, 8 up, 8 in 2024-04-07T16:17:42.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:17:41 smithi144 bash[23522]: cluster 2024-04-07T16:17:40.708651+0000 mgr.y (mgr.14553) 210 : cluster 0 pgmap v207: 121 pgs: 121 active+clean; 61 MiB data, 341 MiB used, 715 GiB / 715 GiB avail; 70 KiB/s rd, 1.3 MiB/s wr, 9 op/s 2024-04-07T16:17:42.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:17:41 smithi144 bash[23522]: cluster 2024-04-07T16:17:40.907885+0000 mon.a (mon.0) 947 : cluster 0 osdmap e126: 8 total, 8 up, 8 in 2024-04-07T16:17:42.830 INFO:tasks.rados.rados.0.smithi100.stdout:318: done (4 left) 2024-04-07T16:17:42.830 INFO:tasks.rados.rados.0.smithi100.stdout:319: done (3 left) 2024-04-07T16:17:42.830 INFO:tasks.rados.rados.0.smithi100.stdout:320: done (2 left) 2024-04-07T16:17:42.830 INFO:tasks.rados.rados.0.smithi100.stdout:321: done (1 left) 2024-04-07T16:17:42.830 INFO:tasks.rados.rados.0.smithi100.stdout:322: done (0 left) 2024-04-07T16:17:42.831 INFO:tasks.rados.rados.0.smithi100.stdout:323: read oid 41 snap -1 2024-04-07T16:17:42.831 INFO:tasks.rados.rados.0.smithi100.stdout:323: expect deleted 2024-04-07T16:17:42.831 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 16 current snap is 29 2024-04-07T16:17:42.831 INFO:tasks.rados.rados.0.smithi100.stdout:324: seq_num 118 ranges {1884160=786432,2670592=704512,3375104=499712} 2024-04-07T16:17:42.868 INFO:tasks.rados.rados.0.smithi100.stdout:324: writing smithi10066867-16 from 1884160 to 2670592 tid 1 2024-04-07T16:17:42.878 INFO:tasks.rados.rados.0.smithi100.stdout:324: writing smithi10066867-16 from 2670592 to 3375104 tid 2 2024-04-07T16:17:42.885 INFO:tasks.rados.rados.0.smithi100.stdout:324: writing smithi10066867-16 from 3375104 to 3874816 tid 3 2024-04-07T16:17:42.885 INFO:tasks.rados.rados.0.smithi100.stdout:325: setattr oid 13 current snap is 29 2024-04-07T16:17:42.885 INFO:tasks.rados.rados.0.smithi100.stdout:324: finishing write tid 1 to smithi10066867-16 2024-04-07T16:17:42.885 INFO:tasks.rados.rados.0.smithi100.stdout:324: finishing write tid 2 to smithi10066867-16 2024-04-07T16:17:42.887 INFO:tasks.rados.rados.0.smithi100.stdout:323: done (2 left) 2024-04-07T16:17:42.887 INFO:tasks.rados.rados.0.smithi100.stdout:326: delete oid 37 current snap is 29 2024-04-07T16:17:42.892 INFO:tasks.rados.rados.0.smithi100.stdout:324: finishing write tid 3 to smithi10066867-16 2024-04-07T16:17:42.892 INFO:tasks.rados.rados.0.smithi100.stdout:324: finishing write tid 4 to smithi10066867-16 2024-04-07T16:17:42.893 INFO:tasks.rados.rados.0.smithi100.stdout:326: done (2 left) 2024-04-07T16:17:42.893 INFO:tasks.rados.rados.0.smithi100.stdout:327: read oid 33 snap -1 2024-04-07T16:17:42.893 INFO:tasks.rados.rados.0.smithi100.stdout:327: expect (ObjNum 114 snap 23 seq_num 114) 2024-04-07T16:17:42.893 INFO:tasks.rados.rados.0.smithi100.stdout:328: read oid 15 snap 22 2024-04-07T16:17:42.894 INFO:tasks.rados.rados.0.smithi100.stdout:328: expect (ObjNum 4 snap 0 seq_num 4) 2024-04-07T16:17:42.894 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 34 current snap is 29 2024-04-07T16:17:42.894 INFO:tasks.rados.rados.0.smithi100.stdout:329: seq_num 120 ranges {557056=417792,974848=475136,1449984=622592,2072576=507904} 2024-04-07T16:17:42.905 INFO:tasks.rados.rados.0.smithi100.stdout:329: writing smithi10066867-34 from 557056 to 974848 tid 1 2024-04-07T16:17:42.911 INFO:tasks.rados.rados.0.smithi100.stdout:329: writing smithi10066867-34 from 974848 to 1449984 tid 2 2024-04-07T16:17:42.919 INFO:tasks.rados.rados.0.smithi100.stdout:329: writing smithi10066867-34 from 1449984 to 2072576 tid 3 2024-04-07T16:17:42.925 INFO:tasks.rados.rados.0.smithi100.stdout:329: writing smithi10066867-34 from 2072576 to 2580480 tid 4 2024-04-07T16:17:42.926 INFO:tasks.rados.rados.0.smithi100.stdout:330: rollback oid 9 current snap is 29 2024-04-07T16:17:42.926 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 9 to 26 2024-04-07T16:17:42.926 INFO:tasks.rados.rados.0.smithi100.stdout:331: read oid 1 snap -1 2024-04-07T16:17:42.926 INFO:tasks.rados.rados.0.smithi100.stdout:331: expect deleted 2024-04-07T16:17:42.926 INFO:tasks.rados.rados.0.smithi100.stdout:332: delete oid 37 current snap is 29 2024-04-07T16:17:42.926 INFO:tasks.rados.rados.0.smithi100.stdout:324: finishing write tid 5 to smithi10066867-16 2024-04-07T16:17:42.926 INFO:tasks.rados.rados.0.smithi100.stdout:324: oid 16 updating version 0 to 115 2024-04-07T16:17:42.926 INFO:tasks.rados.rados.0.smithi100.stdout:324: oid 16 version 115 is already newer than 114 2024-04-07T16:17:42.926 INFO:tasks.rados.rados.0.smithi100.stdout:324: oid 16 version 115 is already newer than 112 2024-04-07T16:17:42.926 INFO:tasks.rados.rados.0.smithi100.stdout:324: oid 16 version 115 is already newer than 113 2024-04-07T16:17:42.926 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 16 v 115 (ObjNum 118 snap 29 seq_num 118) dirty exists 2024-04-07T16:17:42.926 INFO:tasks.rados.rados.0.smithi100.stdout:324: left oid 16 (ObjNum 118 snap 29 seq_num 118) 2024-04-07T16:17:42.926 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 13 v 52 (ObjNum 72 snap 11 seq_num 72) dirty exists 2024-04-07T16:17:42.927 INFO:tasks.rados.rados.0.smithi100.stdout:329: finishing write tid 1 to smithi10066867-34 2024-04-07T16:17:42.927 INFO:tasks.rados.rados.0.smithi100.stdout:329: finishing write tid 2 to smithi10066867-34 2024-04-07T16:17:42.927 INFO:tasks.rados.rados.0.smithi100.stdout:329: finishing write tid 3 to smithi10066867-34 2024-04-07T16:17:42.930 INFO:tasks.rados.rados.0.smithi100.stdout:324: done (7 left) 2024-04-07T16:17:42.930 INFO:tasks.rados.rados.0.smithi100.stdout:325: done (6 left) 2024-04-07T16:17:42.930 INFO:tasks.rados.rados.0.smithi100.stdout:331: done (5 left) 2024-04-07T16:17:42.930 INFO:tasks.rados.rados.0.smithi100.stdout:332: done (4 left) 2024-04-07T16:17:42.930 INFO:tasks.rados.rados.0.smithi100.stdout:333: rollback oid 38 current snap is 29 2024-04-07T16:17:42.930 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 38 to 26 2024-04-07T16:17:42.930 INFO:tasks.rados.rados.0.smithi100.stdout:334: rollback oid 29 current snap is 29 2024-04-07T16:17:42.930 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 29 to 27 2024-04-07T16:17:42.930 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 23 current snap is 29 2024-04-07T16:17:42.930 INFO:tasks.rados.rados.0.smithi100.stdout:335: seq_num 122 ranges {3203072=450560,3653632=688128,4341760=630784,4972544=286720} 2024-04-07T16:17:42.957 INFO:tasks.rados.rados.0.smithi100.stdout:335: writing smithi10066867-23 from 3203072 to 3653632 tid 1 2024-04-07T16:17:42.962 INFO:tasks.rados.rados.0.smithi100.stdout:335: writing smithi10066867-23 from 3653632 to 4341760 tid 2 2024-04-07T16:17:42.966 INFO:tasks.rados.rados.0.smithi100.stdout:335: writing smithi10066867-23 from 4341760 to 4972544 tid 3 2024-04-07T16:17:42.970 INFO:tasks.rados.rados.0.smithi100.stdout:335: writing smithi10066867-23 from 4972544 to 5259264 tid 4 2024-04-07T16:17:42.970 INFO:tasks.rados.rados.0.smithi100.stdout:336: read oid 48 snap -1 2024-04-07T16:17:42.970 INFO:tasks.rados.rados.0.smithi100.stdout:336: expect (ObjNum 46 snap 0 seq_num 46) 2024-04-07T16:17:42.970 INFO:tasks.rados.rados.0.smithi100.stdout:337: copy_from oid 49 from oid 24 current snap is 29 2024-04-07T16:17:42.970 INFO:tasks.rados.rados.0.smithi100.stdout:330: finishing rollback tid 0 to smithi10066867-9 2024-04-07T16:17:42.970 INFO:tasks.rados.rados.0.smithi100.stdout:338: rollback oid 20 current snap is 29 2024-04-07T16:17:42.970 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 20 to 25 2024-04-07T16:17:42.971 INFO:tasks.rados.rados.0.smithi100.stdout:327: expect (ObjNum 114 snap 23 seq_num 114) 2024-04-07T16:17:43.063 INFO:tasks.rados.rados.0.smithi100.stdout:329: finishing write tid 4 to smithi10066867-34 2024-04-07T16:17:43.063 INFO:tasks.rados.rados.0.smithi100.stdout:329: finishing write tid 5 to smithi10066867-34 2024-04-07T16:17:43.064 INFO:tasks.rados.rados.0.smithi100.stdout:333: finishing rollback tid 0 to smithi10066867-38 2024-04-07T16:17:43.064 INFO:tasks.rados.rados.0.smithi100.stdout:330: finishing rollback tid 1 to smithi10066867-9 2024-04-07T16:17:43.064 INFO:tasks.rados.rados.0.smithi100.stdout:330: finishing rollback tid 2 to smithi10066867-9 2024-04-07T16:17:43.064 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 9 v 51 (ObjNum 72 snap 11 seq_num 72) dirty exists 2024-04-07T16:17:43.064 INFO:tasks.rados.rados.0.smithi100.stdout:333: finishing rollback tid 1 to smithi10066867-38 2024-04-07T16:17:43.064 INFO:tasks.rados.rados.0.smithi100.stdout:329: finishing write tid 6 to smithi10066867-34 2024-04-07T16:17:43.064 INFO:tasks.rados.rados.0.smithi100.stdout:329: oid 34 updating version 0 to 54 2024-04-07T16:17:43.064 INFO:tasks.rados.rados.0.smithi100.stdout:329: oid 34 updating version 54 to 55 2024-04-07T16:17:43.064 INFO:tasks.rados.rados.0.smithi100.stdout:329: oid 34 updating version 55 to 57 2024-04-07T16:17:43.064 INFO:tasks.rados.rados.0.smithi100.stdout:329: oid 34 updating version 57 to 58 2024-04-07T16:17:43.064 INFO:tasks.rados.rados.0.smithi100.stdout:329: oid 34 version 58 is already newer than 56 2024-04-07T16:17:43.064 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 34 v 58 (ObjNum 120 snap 29 seq_num 120) dirty exists 2024-04-07T16:17:43.064 INFO:tasks.rados.rados.0.smithi100.stdout:329: left oid 34 (ObjNum 120 snap 29 seq_num 120) 2024-04-07T16:17:43.064 INFO:tasks.rados.rados.0.smithi100.stdout:333: finishing rollback tid 2 to smithi10066867-38 2024-04-07T16:17:43.064 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 38 v 51 (ObjNum 28 snap 0 seq_num 28) dirty exists 2024-04-07T16:17:43.065 INFO:tasks.rados.rados.0.smithi100.stdout:328: expect (ObjNum 4 snap 0 seq_num 4) 2024-04-07T16:17:43.067 INFO:tasks.rados.rados.0.smithi100.stdout:334: finishing rollback tid 0 to smithi10066867-29 2024-04-07T16:17:43.068 INFO:tasks.rados.rados.0.smithi100.stdout:334: finishing rollback tid 1 to smithi10066867-29 2024-04-07T16:17:43.068 INFO:tasks.rados.rados.0.smithi100.stdout:334: finishing rollback tid 2 to smithi10066867-29 2024-04-07T16:17:43.068 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 29 v 83 (ObjNum 61 snap 5 seq_num 61) dirty exists 2024-04-07T16:17:43.068 INFO:tasks.rados.rados.0.smithi100.stdout:335: finishing write tid 1 to smithi10066867-23 2024-04-07T16:17:43.068 INFO:tasks.rados.rados.0.smithi100.stdout:338: finishing rollback tid 1 to smithi10066867-20 2024-04-07T16:17:43.068 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 20 v 0 (ObjNum 33 snap 0 seq_num 2699882032) dirty dne 2024-04-07T16:17:43.068 INFO:tasks.rados.rados.0.smithi100.stdout:335: finishing write tid 2 to smithi10066867-23 2024-04-07T16:17:43.068 INFO:tasks.rados.rados.0.smithi100.stdout:336: expect (ObjNum 46 snap 0 seq_num 46) 2024-04-07T16:17:43.090 INFO:tasks.rados.rados.0.smithi100.stdout:335: finishing write tid 3 to smithi10066867-23 2024-04-07T16:17:43.091 INFO:tasks.rados.rados.0.smithi100.stdout:335: finishing write tid 4 to smithi10066867-23 2024-04-07T16:17:43.091 INFO:tasks.rados.rados.0.smithi100.stdout:335: finishing write tid 5 to smithi10066867-23 2024-04-07T16:17:43.091 INFO:tasks.rados.rados.0.smithi100.stdout:335: finishing write tid 6 to smithi10066867-23 2024-04-07T16:17:43.091 INFO:tasks.rados.rados.0.smithi100.stdout:335: oid 23 updating version 0 to 128 2024-04-07T16:17:43.091 INFO:tasks.rados.rados.0.smithi100.stdout:335: oid 23 version 128 is already newer than 126 2024-04-07T16:17:43.091 INFO:tasks.rados.rados.0.smithi100.stdout:335: oid 23 version 128 is already newer than 127 2024-04-07T16:17:43.091 INFO:tasks.rados.rados.0.smithi100.stdout:335: oid 23 version 128 is already newer than 125 2024-04-07T16:17:43.091 INFO:tasks.rados.rados.0.smithi100.stdout:335: oid 23 updating version 128 to 129 2024-04-07T16:17:43.091 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 23 v 129 (ObjNum 122 snap 29 seq_num 122) dirty exists 2024-04-07T16:17:43.091 INFO:tasks.rados.rados.0.smithi100.stdout:335: left oid 23 (ObjNum 122 snap 29 seq_num 122) 2024-04-07T16:17:43.091 INFO:tasks.rados.rados.0.smithi100.stdout:337: finishing copy_from racing read to smithi10066867-49 2024-04-07T16:17:43.091 INFO:tasks.rados.rados.0.smithi100.stdout:337: finishing copy_from to smithi10066867-49 2024-04-07T16:17:43.091 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 49 v 68 (ObjNum 93 snap 16 seq_num 93) dirty exists 2024-04-07T16:17:43.092 INFO:tasks.rados.rados.0.smithi100.stdout:327: done (9 left) 2024-04-07T16:17:43.092 INFO:tasks.rados.rados.0.smithi100.stdout:328: done (8 left) 2024-04-07T16:17:43.092 INFO:tasks.rados.rados.0.smithi100.stdout:329: done (7 left) 2024-04-07T16:17:43.092 INFO:tasks.rados.rados.0.smithi100.stdout:330: done (6 left) 2024-04-07T16:17:43.092 INFO:tasks.rados.rados.0.smithi100.stdout:333: done (5 left) 2024-04-07T16:17:43.092 INFO:tasks.rados.rados.0.smithi100.stdout:334: done (4 left) 2024-04-07T16:17:43.092 INFO:tasks.rados.rados.0.smithi100.stdout:335: done (3 left) 2024-04-07T16:17:43.092 INFO:tasks.rados.rados.0.smithi100.stdout:336: done (2 left) 2024-04-07T16:17:43.092 INFO:tasks.rados.rados.0.smithi100.stdout:337: done (1 left) 2024-04-07T16:17:43.092 INFO:tasks.rados.rados.0.smithi100.stdout:338: done (0 left) 2024-04-07T16:17:43.092 INFO:tasks.rados.rados.0.smithi100.stdout:339: read oid 43 snap -1 2024-04-07T16:17:43.093 INFO:tasks.rados.rados.0.smithi100.stdout:339: expect (ObjNum 74 snap 11 seq_num 74) 2024-04-07T16:17:43.093 INFO:tasks.rados.rados.0.smithi100.stdout:340: rmattr oid 17 current snap is 29 2024-04-07T16:17:43.093 INFO:tasks.rados.rados.0.smithi100.stdout:340: done (1 left) 2024-04-07T16:17:43.093 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 10 current snap is 29 2024-04-07T16:17:43.093 INFO:tasks.rados.rados.0.smithi100.stdout:341: seq_num 123 ranges {2932736=466944,3399680=573440,3973120=794624,4767744=237568} 2024-04-07T16:17:43.128 INFO:tasks.rados.rados.0.smithi100.stdout:341: writing smithi10066867-10 from 2932736 to 3399680 tid 1 2024-04-07T16:17:43.135 INFO:tasks.rados.rados.0.smithi100.stdout:341: writing smithi10066867-10 from 3399680 to 3973120 tid 2 2024-04-07T16:17:43.143 INFO:tasks.rados.rados.0.smithi100.stdout:341: writing smithi10066867-10 from 3973120 to 4767744 tid 3 2024-04-07T16:17:43.145 INFO:tasks.rados.rados.0.smithi100.stdout:341: writing smithi10066867-10 from 4767744 to 5005312 tid 4 2024-04-07T16:17:43.146 INFO:tasks.rados.rados.0.smithi100.stdout:342: read oid 15 snap -1 2024-04-07T16:17:43.146 INFO:tasks.rados.rados.0.smithi100.stdout:342: expect (ObjNum 4 snap 0 seq_num 4) 2024-04-07T16:17:43.146 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 6 current snap is 29 2024-04-07T16:17:43.146 INFO:tasks.rados.rados.0.smithi100.stdout:343: seq_num 124 ranges {1802240=483328,2285568=458752,2744320=598016,3342336=548864} 2024-04-07T16:17:43.163 INFO:tasks.rados.rados.0.smithi100.stdout:343: writing smithi10066867-6 from 1802240 to 2285568 tid 1 2024-04-07T16:17:43.166 INFO:tasks.rados.rados.0.smithi100.stdout:343: writing smithi10066867-6 from 2285568 to 2744320 tid 2 2024-04-07T16:17:43.170 INFO:tasks.rados.rados.0.smithi100.stdout:343: writing smithi10066867-6 from 2744320 to 3342336 tid 3 2024-04-07T16:17:43.174 INFO:tasks.rados.rados.0.smithi100.stdout:343: writing smithi10066867-6 from 3342336 to 3891200 tid 4 2024-04-07T16:17:43.175 INFO:tasks.rados.rados.0.smithi100.stdout:344: rmattr oid 14 current snap is 29 2024-04-07T16:17:43.175 INFO:tasks.rados.rados.0.smithi100.stdout:344: done (4 left) 2024-04-07T16:17:43.175 INFO:tasks.rados.rados.0.smithi100.stdout:345: read oid 2 snap -1 2024-04-07T16:17:43.175 INFO:tasks.rados.rados.0.smithi100.stdout:345: expect (ObjNum 108 snap 20 seq_num 108) 2024-04-07T16:17:43.175 INFO:tasks.rados.rados.0.smithi100.stdout:346: snap_create 2024-04-07T16:17:43.175 INFO:tasks.rados.rados.0.smithi100.stdout:339: expect (ObjNum 74 snap 11 seq_num 74) 2024-04-07T16:17:43.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:17:42 smithi100 bash[31660]: cluster 2024-04-07T16:17:41.904790+0000 mon.a (mon.0) 948 : cluster 0 osdmap e127: 8 total, 8 up, 8 in 2024-04-07T16:17:43.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:17:42 smithi100 bash[31660]: cluster 2024-04-07T16:17:42.843277+0000 mon.a (mon.0) 949 : cluster 0 osdmap e128: 8 total, 8 up, 8 in 2024-04-07T16:17:43.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:17:42 smithi100 bash[21346]: cluster 2024-04-07T16:17:41.904790+0000 mon.a (mon.0) 948 : cluster 0 osdmap e127: 8 total, 8 up, 8 in 2024-04-07T16:17:43.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:17:42 smithi100 bash[21346]: cluster 2024-04-07T16:17:42.843277+0000 mon.a (mon.0) 949 : cluster 0 osdmap e128: 8 total, 8 up, 8 in 2024-04-07T16:17:43.344 INFO:tasks.rados.rados.0.smithi100.stdout:341: finishing write tid 1 to smithi10066867-10 2024-04-07T16:17:43.344 INFO:tasks.rados.rados.0.smithi100.stdout:341: finishing write tid 2 to smithi10066867-10 2024-04-07T16:17:43.344 INFO:tasks.rados.rados.0.smithi100.stdout:342: expect (ObjNum 4 snap 0 seq_num 4) 2024-04-07T16:17:43.348 INFO:tasks.rados.rados.0.smithi100.stdout:341: finishing write tid 3 to smithi10066867-10 2024-04-07T16:17:43.348 INFO:tasks.rados.rados.0.smithi100.stdout:341: finishing write tid 4 to smithi10066867-10 2024-04-07T16:17:43.348 INFO:tasks.rados.rados.0.smithi100.stdout:341: finishing write tid 5 to smithi10066867-10 2024-04-07T16:17:43.348 INFO:tasks.rados.rados.0.smithi100.stdout:341: finishing write tid 6 to smithi10066867-10 2024-04-07T16:17:43.348 INFO:tasks.rados.rados.0.smithi100.stdout:341: oid 10 updating version 0 to 28 2024-04-07T16:17:43.348 INFO:tasks.rados.rados.0.smithi100.stdout:341: oid 10 version 28 is already newer than 27 2024-04-07T16:17:43.348 INFO:tasks.rados.rados.0.smithi100.stdout:341: oid 10 version 28 is already newer than 26 2024-04-07T16:17:43.348 INFO:tasks.rados.rados.0.smithi100.stdout:341: oid 10 version 28 is already newer than 25 2024-04-07T16:17:43.348 INFO:tasks.rados.rados.0.smithi100.stdout:341: oid 10 updating version 28 to 29 2024-04-07T16:17:43.348 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 10 v 29 (ObjNum 123 snap 29 seq_num 123) dirty exists 2024-04-07T16:17:43.348 INFO:tasks.rados.rados.0.smithi100.stdout:341: left oid 10 (ObjNum 123 snap 29 seq_num 123) 2024-04-07T16:17:43.348 INFO:tasks.rados.rados.0.smithi100.stdout:343: finishing write tid 1 to smithi10066867-6 2024-04-07T16:17:43.349 INFO:tasks.rados.rados.0.smithi100.stdout:343: finishing write tid 2 to smithi10066867-6 2024-04-07T16:17:43.349 INFO:tasks.rados.rados.0.smithi100.stdout:343: finishing write tid 3 to smithi10066867-6 2024-04-07T16:17:43.349 INFO:tasks.rados.rados.0.smithi100.stdout:343: finishing write tid 4 to smithi10066867-6 2024-04-07T16:17:43.349 INFO:tasks.rados.rados.0.smithi100.stdout:343: finishing write tid 5 to smithi10066867-6 2024-04-07T16:17:43.349 INFO:tasks.rados.rados.0.smithi100.stdout:343: finishing write tid 6 to smithi10066867-6 2024-04-07T16:17:43.349 INFO:tasks.rados.rados.0.smithi100.stdout:343: oid 6 updating version 0 to 116 2024-04-07T16:17:43.349 INFO:tasks.rados.rados.0.smithi100.stdout:343: oid 6 updating version 116 to 117 2024-04-07T16:17:43.349 INFO:tasks.rados.rados.0.smithi100.stdout:343: oid 6 updating version 117 to 119 2024-04-07T16:17:43.349 INFO:tasks.rados.rados.0.smithi100.stdout:343: oid 6 version 119 is already newer than 118 2024-04-07T16:17:43.349 INFO:tasks.rados.rados.0.smithi100.stdout:343: oid 6 version 119 is already newer than 115 2024-04-07T16:17:43.349 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 6 v 119 (ObjNum 124 snap 29 seq_num 124) dirty exists 2024-04-07T16:17:43.349 INFO:tasks.rados.rados.0.smithi100.stdout:343: left oid 6 (ObjNum 124 snap 29 seq_num 124) 2024-04-07T16:17:43.349 INFO:tasks.rados.rados.0.smithi100.stdout:345: expect (ObjNum 108 snap 20 seq_num 108) 2024-04-07T16:17:43.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:17:42 smithi144 bash[23522]: cluster 2024-04-07T16:17:41.904790+0000 mon.a (mon.0) 948 : cluster 0 osdmap e127: 8 total, 8 up, 8 in 2024-04-07T16:17:43.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:17:42 smithi144 bash[23522]: cluster 2024-04-07T16:17:42.843277+0000 mon.a (mon.0) 949 : cluster 0 osdmap e128: 8 total, 8 up, 8 in 2024-04-07T16:17:43.924 INFO:tasks.rados.rados.0.smithi100.stdout:339: done (5 left) 2024-04-07T16:17:43.924 INFO:tasks.rados.rados.0.smithi100.stdout:341: done (4 left) 2024-04-07T16:17:43.924 INFO:tasks.rados.rados.0.smithi100.stdout:342: done (3 left) 2024-04-07T16:17:43.924 INFO:tasks.rados.rados.0.smithi100.stdout:343: done (2 left) 2024-04-07T16:17:43.924 INFO:tasks.rados.rados.0.smithi100.stdout:345: done (1 left) 2024-04-07T16:17:43.924 INFO:tasks.rados.rados.0.smithi100.stdout:346: done (0 left) 2024-04-07T16:17:43.924 INFO:tasks.rados.rados.0.smithi100.stdout:347: delete oid 49 current snap is 30 2024-04-07T16:17:43.930 INFO:tasks.rados.rados.0.smithi100.stdout:347: done (0 left) 2024-04-07T16:17:43.930 INFO:tasks.rados.rados.0.smithi100.stdout:348: rollback oid 18 current snap is 30 2024-04-07T16:17:43.930 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 18 to 29 2024-04-07T16:17:43.930 INFO:tasks.rados.rados.0.smithi100.stdout:349: snap_create 2024-04-07T16:17:43.941 INFO:tasks.rados.rados.0.smithi100.stdout:348: finishing rollback tid 0 to smithi10066867-18 2024-04-07T16:17:43.941 INFO:tasks.rados.rados.0.smithi100.stdout:348: finishing rollback tid 1 to smithi10066867-18 2024-04-07T16:17:43.942 INFO:tasks.rados.rados.0.smithi100.stdout:348: finishing rollback tid 2 to smithi10066867-18 2024-04-07T16:17:43.942 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 18 v 133 (ObjNum 117 snap 28 seq_num 117) dirty exists 2024-04-07T16:17:44.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:17:43 smithi100 bash[21346]: cluster 2024-04-07T16:17:42.709326+0000 mgr.y (mgr.14553) 211 : cluster 0 pgmap v210: 121 pgs: 121 active+clean; 61 MiB data, 341 MiB used, 715 GiB / 715 GiB avail; 70 KiB/s rd, 1.3 MiB/s wr, 9 op/s 2024-04-07T16:17:44.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:17:43 smithi100 bash[31660]: cluster 2024-04-07T16:17:42.709326+0000 mgr.y (mgr.14553) 211 : cluster 0 pgmap v210: 121 pgs: 121 active+clean; 61 MiB data, 341 MiB used, 715 GiB / 715 GiB avail; 70 KiB/s rd, 1.3 MiB/s wr, 9 op/s 2024-04-07T16:17:44.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:17:43 smithi144 bash[23522]: cluster 2024-04-07T16:17:42.709326+0000 mgr.y (mgr.14553) 211 : cluster 0 pgmap v210: 121 pgs: 121 active+clean; 61 MiB data, 341 MiB used, 715 GiB / 715 GiB avail; 70 KiB/s rd, 1.3 MiB/s wr, 9 op/s 2024-04-07T16:17:44.931 INFO:tasks.rados.rados.0.smithi100.stdout:348: done (1 left) 2024-04-07T16:17:44.931 INFO:tasks.rados.rados.0.smithi100.stdout:349: done (0 left) 2024-04-07T16:17:44.931 INFO:tasks.rados.rados.0.smithi100.stdout:350: rollback oid 41 current snap is 31 2024-04-07T16:17:44.931 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 41 to 27 2024-04-07T16:17:44.931 INFO:tasks.rados.rados.0.smithi100.stdout:351: rollback oid 18 current snap is 31 2024-04-07T16:17:44.931 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 18 to 24 2024-04-07T16:17:44.932 INFO:tasks.rados.rados.0.smithi100.stdout:352: rmattr oid 23 current snap is 31 2024-04-07T16:17:44.932 INFO:tasks.rados.rados.0.smithi100.stdout:352: done (2 left) 2024-04-07T16:17:44.932 INFO:tasks.rados.rados.0.smithi100.stdout:353: copy_from oid 43 from oid 10 current snap is 31 2024-04-07T16:17:44.932 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 19 current snap is 31 2024-04-07T16:17:44.932 INFO:tasks.rados.rados.0.smithi100.stdout:354: seq_num 126 ranges {1564672=516096,2080768=630784,2711552=614400,3325952=360448} 2024-04-07T16:17:44.948 INFO:tasks.rados.rados.0.smithi100.stdout:354: writing smithi10066867-19 from 1564672 to 2080768 tid 1 2024-04-07T16:17:44.954 INFO:tasks.rados.rados.0.smithi100.stdout:354: writing smithi10066867-19 from 2080768 to 2711552 tid 2 2024-04-07T16:17:44.959 INFO:tasks.rados.rados.0.smithi100.stdout:354: writing smithi10066867-19 from 2711552 to 3325952 tid 3 2024-04-07T16:17:44.962 INFO:tasks.rados.rados.0.smithi100.stdout:354: writing smithi10066867-19 from 3325952 to 3686400 tid 4 2024-04-07T16:17:44.963 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 13 current snap is 31 2024-04-07T16:17:44.963 INFO:tasks.rados.rados.0.smithi100.stdout:351: finishing rollback tid 0 to smithi10066867-18 2024-04-07T16:17:44.963 INFO:tasks.rados.rados.0.smithi100.stdout:351: finishing rollback tid 1 to smithi10066867-18 2024-04-07T16:17:44.963 INFO:tasks.rados.rados.0.smithi100.stdout:350: finishing rollback tid 1 to smithi10066867-41 2024-04-07T16:17:44.963 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 41 v 0 (ObjNum 65 snap 0 seq_num 1953066355) dirty dne 2024-04-07T16:17:44.963 INFO:tasks.rados.rados.0.smithi100.stdout:351: finishing rollback tid 2 to smithi10066867-18 2024-04-07T16:17:44.963 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 18 v 137 (ObjNum 72 snap 11 seq_num 72) dirty exists 2024-04-07T16:17:44.963 INFO:tasks.rados.rados.0.smithi100.stdout:354: finishing write tid 1 to smithi10066867-19 2024-04-07T16:17:44.963 INFO:tasks.rados.rados.0.smithi100.stdout:355: seq_num 127 ranges {1417216=532480,1949696=507904,2457600=778240,3235840=319488} 2024-04-07T16:17:44.977 INFO:tasks.rados.rados.0.smithi100.stdout:355: writing smithi10066867-13 from 1417216 to 1949696 tid 1 2024-04-07T16:17:44.981 INFO:tasks.rados.rados.0.smithi100.stdout:355: writing smithi10066867-13 from 1949696 to 2457600 tid 2 2024-04-07T16:17:44.988 INFO:tasks.rados.rados.0.smithi100.stdout:355: writing smithi10066867-13 from 2457600 to 3235840 tid 3 2024-04-07T16:17:44.994 INFO:tasks.rados.rados.0.smithi100.stdout:355: writing smithi10066867-13 from 3235840 to 3555328 tid 4 2024-04-07T16:17:44.994 INFO:tasks.rados.rados.0.smithi100.stdout:350: done (4 left) 2024-04-07T16:17:44.994 INFO:tasks.rados.rados.0.smithi100.stdout:351: done (3 left) 2024-04-07T16:17:44.994 INFO:tasks.rados.rados.0.smithi100.stdout:356: setattr oid 17 current snap is 31 2024-04-07T16:17:44.994 INFO:tasks.rados.rados.0.smithi100.stdout:354: finishing write tid 2 to smithi10066867-19 2024-04-07T16:17:44.995 INFO:tasks.rados.rados.0.smithi100.stdout:354: finishing write tid 3 to smithi10066867-19 2024-04-07T16:17:44.995 INFO:tasks.rados.rados.0.smithi100.stdout:354: finishing write tid 4 to smithi10066867-19 2024-04-07T16:17:44.995 INFO:tasks.rados.rados.0.smithi100.stdout:354: finishing write tid 5 to smithi10066867-19 2024-04-07T16:17:44.995 INFO:tasks.rados.rados.0.smithi100.stdout:354: finishing write tid 6 to smithi10066867-19 2024-04-07T16:17:44.995 INFO:tasks.rados.rados.0.smithi100.stdout:354: oid 19 updating version 0 to 114 2024-04-07T16:17:44.995 INFO:tasks.rados.rados.0.smithi100.stdout:354: oid 19 version 114 is already newer than 111 2024-04-07T16:17:44.995 INFO:tasks.rados.rados.0.smithi100.stdout:354: oid 19 version 114 is already newer than 112 2024-04-07T16:17:44.995 INFO:tasks.rados.rados.0.smithi100.stdout:354: oid 19 version 114 is already newer than 113 2024-04-07T16:17:44.995 INFO:tasks.rados.rados.0.smithi100.stdout:354: oid 19 updating version 114 to 115 2024-04-07T16:17:44.995 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 19 v 115 (ObjNum 126 snap 31 seq_num 126) dirty exists 2024-04-07T16:17:44.995 INFO:tasks.rados.rados.0.smithi100.stdout:354: left oid 19 (ObjNum 126 snap 31 seq_num 126) 2024-04-07T16:17:44.995 INFO:tasks.rados.rados.0.smithi100.stdout:355: finishing write tid 1 to smithi10066867-13 2024-04-07T16:17:44.995 INFO:tasks.rados.rados.0.smithi100.stdout:353: finishing copy_from racing read to smithi10066867-43 2024-04-07T16:17:44.995 INFO:tasks.rados.rados.0.smithi100.stdout:355: finishing write tid 2 to smithi10066867-13 2024-04-07T16:17:44.995 INFO:tasks.rados.rados.0.smithi100.stdout:353: finishing copy_from to smithi10066867-43 2024-04-07T16:17:44.996 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 43 v 29 (ObjNum 123 snap 29 seq_num 123) dirty exists 2024-04-07T16:17:44.996 INFO:tasks.rados.rados.0.smithi100.stdout:355: finishing write tid 3 to smithi10066867-13 2024-04-07T16:17:44.996 INFO:tasks.rados.rados.0.smithi100.stdout:355: finishing write tid 4 to smithi10066867-13 2024-04-07T16:17:44.996 INFO:tasks.rados.rados.0.smithi100.stdout:355: finishing write tid 5 to smithi10066867-13 2024-04-07T16:17:44.996 INFO:tasks.rados.rados.0.smithi100.stdout:353: done (3 left) 2024-04-07T16:17:44.996 INFO:tasks.rados.rados.0.smithi100.stdout:354: done (2 left) 2024-04-07T16:17:44.996 INFO:tasks.rados.rados.0.smithi100.stdout:357: snap_create 2024-04-07T16:17:44.996 INFO:tasks.rados.rados.0.smithi100.stdout:355: finishing write tid 6 to smithi10066867-13 2024-04-07T16:17:44.996 INFO:tasks.rados.rados.0.smithi100.stdout:355: oid 13 updating version 0 to 63 2024-04-07T16:17:44.997 INFO:tasks.rados.rados.0.smithi100.stdout:355: oid 13 version 63 is already newer than 60 2024-04-07T16:17:44.997 INFO:tasks.rados.rados.0.smithi100.stdout:355: oid 13 updating version 63 to 64 2024-04-07T16:17:44.997 INFO:tasks.rados.rados.0.smithi100.stdout:355: oid 13 version 64 is already newer than 62 2024-04-07T16:17:44.997 INFO:tasks.rados.rados.0.smithi100.stdout:355: oid 13 version 64 is already newer than 61 2024-04-07T16:17:44.997 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 13 v 64 (ObjNum 127 snap 31 seq_num 127) dirty exists 2024-04-07T16:17:44.997 INFO:tasks.rados.rados.0.smithi100.stdout:355: left oid 13 (ObjNum 127 snap 31 seq_num 127) 2024-04-07T16:17:44.998 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 17 v 117 (ObjNum 98 snap 18 seq_num 98) dirty exists 2024-04-07T16:17:45.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:17:44 smithi100 bash[21346]: cluster 2024-04-07T16:17:43.938381+0000 mon.a (mon.0) 950 : cluster 0 osdmap e129: 8 total, 8 up, 8 in 2024-04-07T16:17:45.187 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:17:45 smithi100 bash[21601]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:17:45] "GET /metrics HTTP/1.1" 200 36085 "" "Prometheus/2.43.0" 2024-04-07T16:17:45.188 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:17:44 smithi100 bash[31660]: cluster 2024-04-07T16:17:43.938381+0000 mon.a (mon.0) 950 : cluster 0 osdmap e129: 8 total, 8 up, 8 in 2024-04-07T16:17:45.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:17:44 smithi144 bash[23522]: cluster 2024-04-07T16:17:43.938381+0000 mon.a (mon.0) 950 : cluster 0 osdmap e129: 8 total, 8 up, 8 in 2024-04-07T16:17:45.933 INFO:tasks.rados.rados.0.smithi100.stdout:355: done (2 left) 2024-04-07T16:17:45.933 INFO:tasks.rados.rados.0.smithi100.stdout:356: done (1 left) 2024-04-07T16:17:45.933 INFO:tasks.rados.rados.0.smithi100.stdout:357: done (0 left) 2024-04-07T16:17:45.933 INFO:tasks.rados.rados.0.smithi100.stdout:358: copy_from oid 34 from oid 46 current snap is 32 2024-04-07T16:17:45.933 INFO:tasks.rados.rados.0.smithi100.stdout:359: snap_remove snap 29 2024-04-07T16:17:46.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:17:45 smithi100 bash[21346]: cluster 2024-04-07T16:17:44.710078+0000 mgr.y (mgr.14553) 212 : cluster 0 pgmap v213: 121 pgs: 121 active+clean; 76 MiB data, 359 MiB used, 715 GiB / 715 GiB avail; 10 MiB/s rd, 3.9 MiB/s wr, 26 op/s 2024-04-07T16:17:46.188 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:17:45 smithi100 bash[21346]: cluster 2024-04-07T16:17:44.945732+0000 mon.a (mon.0) 951 : cluster 0 osdmap e130: 8 total, 8 up, 8 in 2024-04-07T16:17:46.188 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:17:45 smithi100 bash[31660]: cluster 2024-04-07T16:17:44.710078+0000 mgr.y (mgr.14553) 212 : cluster 0 pgmap v213: 121 pgs: 121 active+clean; 76 MiB data, 359 MiB used, 715 GiB / 715 GiB avail; 10 MiB/s rd, 3.9 MiB/s wr, 26 op/s 2024-04-07T16:17:46.188 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:17:45 smithi100 bash[31660]: cluster 2024-04-07T16:17:44.945732+0000 mon.a (mon.0) 951 : cluster 0 osdmap e130: 8 total, 8 up, 8 in 2024-04-07T16:17:46.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:17:45 smithi144 bash[23522]: cluster 2024-04-07T16:17:44.710078+0000 mgr.y (mgr.14553) 212 : cluster 0 pgmap v213: 121 pgs: 121 active+clean; 76 MiB data, 359 MiB used, 715 GiB / 715 GiB avail; 10 MiB/s rd, 3.9 MiB/s wr, 26 op/s 2024-04-07T16:17:46.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:17:45 smithi144 bash[23522]: cluster 2024-04-07T16:17:44.945732+0000 mon.a (mon.0) 951 : cluster 0 osdmap e130: 8 total, 8 up, 8 in 2024-04-07T16:17:46.963 INFO:tasks.rados.rados.0.smithi100.stdout:359: done (1 left) 2024-04-07T16:17:46.963 INFO:tasks.rados.rados.0.smithi100.stdout:360: rollback oid 45 current snap is 32 2024-04-07T16:17:46.964 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 45 to 26 2024-04-07T16:17:46.964 INFO:tasks.rados.rados.0.smithi100.stdout:358: finishing copy_from racing read to smithi10066867-34 2024-04-07T16:17:46.964 INFO:tasks.rados.rados.0.smithi100.stdout:358: finishing copy_from to smithi10066867-34 2024-04-07T16:17:46.964 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 34 v 8 (ObjNum 45 snap 0 seq_num 45) dirty exists 2024-04-07T16:17:46.964 INFO:tasks.rados.rados.0.smithi100.stdout:358: done (1 left) 2024-04-07T16:17:46.964 INFO:tasks.rados.rados.0.smithi100.stdout:361: read oid 28 snap -1 2024-04-07T16:17:46.964 INFO:tasks.rados.rados.0.smithi100.stdout:361: expect deleted 2024-04-07T16:17:46.964 INFO:tasks.rados.rados.0.smithi100.stdout:362: read oid 36 snap -1 2024-04-07T16:17:46.964 INFO:tasks.rados.rados.0.smithi100.stdout:362: expect deleted 2024-04-07T16:17:46.964 INFO:tasks.rados.rados.0.smithi100.stdout:363: delete oid 12 current snap is 32 2024-04-07T16:17:46.971 INFO:tasks.rados.rados.0.smithi100.stdout:360: finishing rollback tid 0 to smithi10066867-45 2024-04-07T16:17:46.972 INFO:tasks.rados.rados.0.smithi100.stdout:360: finishing rollback tid 1 to smithi10066867-45 2024-04-07T16:17:46.974 INFO:tasks.rados.rados.0.smithi100.stdout:360: finishing rollback tid 2 to smithi10066867-45 2024-04-07T16:17:46.974 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 45 v 53 (ObjNum 111 snap 21 seq_num 111) dirty exists 2024-04-07T16:17:46.981 INFO:tasks.rados.rados.0.smithi100.stdout:360: done (3 left) 2024-04-07T16:17:46.981 INFO:tasks.rados.rados.0.smithi100.stdout:361: done (2 left) 2024-04-07T16:17:46.981 INFO:tasks.rados.rados.0.smithi100.stdout:362: done (1 left) 2024-04-07T16:17:46.981 INFO:tasks.rados.rados.0.smithi100.stdout:363: done (0 left) 2024-04-07T16:17:46.981 INFO:tasks.rados.rados.0.smithi100.stdout:364: rollback oid 5 current snap is 32 2024-04-07T16:17:46.981 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 5 to 28 2024-04-07T16:17:46.981 INFO:tasks.rados.rados.0.smithi100.stdout:365: snap_remove snap 31 2024-04-07T16:17:47.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:17:46 smithi144 bash[23522]: cluster 2024-04-07T16:17:45.946558+0000 mon.a (mon.0) 952 : cluster 0 osdmap e131: 8 total, 8 up, 8 in 2024-04-07T16:17:47.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:17:46 smithi100 bash[21346]: cluster 2024-04-07T16:17:45.946558+0000 mon.a (mon.0) 952 : cluster 0 osdmap e131: 8 total, 8 up, 8 in 2024-04-07T16:17:47.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:17:46 smithi100 bash[31660]: cluster 2024-04-07T16:17:45.946558+0000 mon.a (mon.0) 952 : cluster 0 osdmap e131: 8 total, 8 up, 8 in 2024-04-07T16:17:47.834 INFO:tasks.rados.rados.0.smithi100.stdout:365: done (1 left) 2024-04-07T16:17:47.834 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 33 current snap is 32 2024-04-07T16:17:47.834 INFO:tasks.rados.rados.0.smithi100.stdout:364: finishing rollback tid 1 to smithi10066867-5 2024-04-07T16:17:47.834 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 5 v 0 (ObjNum 30993 snap 0 seq_num 3067286560) dirty dne 2024-04-07T16:17:47.834 INFO:tasks.rados.rados.0.smithi100.stdout:366: seq_num 129 ranges {1916928=565248,2482176=679936,3162112=786432,3948544=139264} 2024-04-07T16:17:47.857 INFO:tasks.rados.rados.0.smithi100.stdout:366: writing smithi10066867-33 from 1916928 to 2482176 tid 1 2024-04-07T16:17:47.864 INFO:tasks.rados.rados.0.smithi100.stdout:366: writing smithi10066867-33 from 2482176 to 3162112 tid 2 2024-04-07T16:17:47.873 INFO:tasks.rados.rados.0.smithi100.stdout:366: writing smithi10066867-33 from 3162112 to 3948544 tid 3 2024-04-07T16:17:47.875 INFO:tasks.rados.rados.0.smithi100.stdout:366: writing smithi10066867-33 from 3948544 to 4087808 tid 4 2024-04-07T16:17:47.875 INFO:tasks.rados.rados.0.smithi100.stdout:364: done (1 left) 2024-04-07T16:17:47.875 INFO:tasks.rados.rados.0.smithi100.stdout:367: read oid 45 snap -1 2024-04-07T16:17:47.875 INFO:tasks.rados.rados.0.smithi100.stdout:367: expect (ObjNum 111 snap 21 seq_num 111) 2024-04-07T16:17:47.875 INFO:tasks.rados.rados.0.smithi100.stdout:368: copy_from oid 21 from oid 20 current snap is 32 2024-04-07T16:17:47.875 INFO:tasks.rados.rados.0.smithi100.stdout:369: setattr oid 34 current snap is 32 2024-04-07T16:17:47.876 INFO:tasks.rados.rados.0.smithi100.stdout:368: finishing copy_from to smithi10066867-21 2024-04-07T16:17:47.876 INFO:tasks.rados.rados.0.smithi100.stdout:368: got expected ENOENT (src dne) 2024-04-07T16:17:47.876 INFO:tasks.rados.rados.0.smithi100.stdout:368: finishing copy_from racing read to smithi10066867-21 2024-04-07T16:17:47.877 INFO:tasks.rados.rados.0.smithi100.stdout:368: done (3 left) 2024-04-07T16:17:47.877 INFO:tasks.rados.rados.0.smithi100.stdout:370: read oid 36 snap -1 2024-04-07T16:17:47.877 INFO:tasks.rados.rados.0.smithi100.stdout:370: expect deleted 2024-04-07T16:17:47.877 INFO:tasks.rados.rados.0.smithi100.stdout:371: snap_create 2024-04-07T16:17:47.879 INFO:tasks.rados.rados.0.smithi100.stdout:366: finishing write tid 1 to smithi10066867-33 2024-04-07T16:17:47.880 INFO:tasks.rados.rados.0.smithi100.stdout:366: finishing write tid 2 to smithi10066867-33 2024-04-07T16:17:47.882 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 34 v 73 (ObjNum 45 snap 0 seq_num 45) dirty exists 2024-04-07T16:17:47.885 INFO:tasks.rados.rados.0.smithi100.stdout:366: finishing write tid 3 to smithi10066867-33 2024-04-07T16:17:47.885 INFO:tasks.rados.rados.0.smithi100.stdout:366: finishing write tid 4 to smithi10066867-33 2024-04-07T16:17:47.886 INFO:tasks.rados.rados.0.smithi100.stdout:366: finishing write tid 5 to smithi10066867-33 2024-04-07T16:17:47.888 INFO:tasks.rados.rados.0.smithi100.stdout:367: expect (ObjNum 111 snap 21 seq_num 111) 2024-04-07T16:17:48.002 INFO:tasks.rados.rados.0.smithi100.stdout:366: finishing write tid 6 to smithi10066867-33 2024-04-07T16:17:48.002 INFO:tasks.rados.rados.0.smithi100.stdout:366: oid 33 updating version 0 to 144 2024-04-07T16:17:48.002 INFO:tasks.rados.rados.0.smithi100.stdout:366: oid 33 version 144 is already newer than 142 2024-04-07T16:17:48.002 INFO:tasks.rados.rados.0.smithi100.stdout:366: oid 33 version 144 is already newer than 141 2024-04-07T16:17:48.002 INFO:tasks.rados.rados.0.smithi100.stdout:366: oid 33 updating version 144 to 145 2024-04-07T16:17:48.002 INFO:tasks.rados.rados.0.smithi100.stdout:366: oid 33 version 145 is already newer than 143 2024-04-07T16:17:48.002 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 33 v 145 (ObjNum 129 snap 32 seq_num 129) dirty exists 2024-04-07T16:17:48.002 INFO:tasks.rados.rados.0.smithi100.stdout:366: left oid 33 (ObjNum 129 snap 32 seq_num 129) 2024-04-07T16:17:48.294 INFO:teuthology.orchestra.run.smithi100.stdout:true 2024-04-07T16:17:48.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:17:47 smithi144 bash[23522]: cluster 2024-04-07T16:17:46.710779+0000 mgr.y (mgr.14553) 213 : cluster 0 pgmap v216: 121 pgs: 121 active+clean; 76 MiB data, 359 MiB used, 715 GiB / 715 GiB avail; 10 MiB/s rd, 3.9 MiB/s wr, 26 op/s 2024-04-07T16:17:48.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:17:47 smithi144 bash[23522]: cluster 2024-04-07T16:17:46.974033+0000 mon.a (mon.0) 953 : cluster 0 osdmap e132: 8 total, 8 up, 8 in 2024-04-07T16:17:48.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:17:47 smithi144 bash[23522]: cluster 2024-04-07T16:17:47.844839+0000 mon.a (mon.0) 954 : cluster 0 osdmap e133: 8 total, 8 up, 8 in 2024-04-07T16:17:48.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:17:47 smithi100 bash[21346]: cluster 2024-04-07T16:17:46.710779+0000 mgr.y (mgr.14553) 213 : cluster 0 pgmap v216: 121 pgs: 121 active+clean; 76 MiB data, 359 MiB used, 715 GiB / 715 GiB avail; 10 MiB/s rd, 3.9 MiB/s wr, 26 op/s 2024-04-07T16:17:48.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:17:47 smithi100 bash[21346]: cluster 2024-04-07T16:17:46.974033+0000 mon.a (mon.0) 953 : cluster 0 osdmap e132: 8 total, 8 up, 8 in 2024-04-07T16:17:48.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:17:47 smithi100 bash[21346]: cluster 2024-04-07T16:17:47.844839+0000 mon.a (mon.0) 954 : cluster 0 osdmap e133: 8 total, 8 up, 8 in 2024-04-07T16:17:48.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:17:47 smithi100 bash[31660]: cluster 2024-04-07T16:17:46.710779+0000 mgr.y (mgr.14553) 213 : cluster 0 pgmap v216: 121 pgs: 121 active+clean; 76 MiB data, 359 MiB used, 715 GiB / 715 GiB avail; 10 MiB/s rd, 3.9 MiB/s wr, 26 op/s 2024-04-07T16:17:48.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:17:47 smithi100 bash[31660]: cluster 2024-04-07T16:17:46.974033+0000 mon.a (mon.0) 953 : cluster 0 osdmap e132: 8 total, 8 up, 8 in 2024-04-07T16:17:48.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:17:47 smithi100 bash[31660]: cluster 2024-04-07T16:17:47.844839+0000 mon.a (mon.0) 954 : cluster 0 osdmap e133: 8 total, 8 up, 8 in 2024-04-07T16:17:48.657 INFO:teuthology.orchestra.run.smithi100.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-07T16:17:48.657 INFO:teuthology.orchestra.run.smithi100.stdout:alertmanager.a smithi100 *:9093,9094 running (3m) 3m ago 4m 13.7M - 0.25.0 c8568f914cd2 5143075e7d26 2024-04-07T16:17:48.657 INFO:teuthology.orchestra.run.smithi100.stdout:grafana.a smithi144 *:3000 running (4m) 3m ago 4m 29.2M - 9.4.7 954c08fa6188 7180c0708acd 2024-04-07T16:17:48.657 INFO:teuthology.orchestra.run.smithi100.stdout:mds.a smithi100 running (5m) 3m ago 5m 12.4M - 18.2.2-1171-g267c8ff5 8a4a01e7186c 92593b0c28ae 2024-04-07T16:17:48.657 INFO:teuthology.orchestra.run.smithi100.stdout:mds.b smithi144 running (5m) 3m ago 5m 14.2M - 18.2.2-1171-g267c8ff5 8a4a01e7186c 3df2ee3e17a4 2024-04-07T16:17:48.657 INFO:teuthology.orchestra.run.smithi100.stdout:mgr.x smithi144 *:8443,8765 running (12m) 3m ago 12m 435M - 18.2.2-1171-g267c8ff5 8a4a01e7186c b7feffac74d1 2024-04-07T16:17:48.657 INFO:teuthology.orchestra.run.smithi100.stdout:mgr.y smithi100 *:9283,8765 running (15m) 3m ago 15m 484M - 18.2.2-1171-g267c8ff5 8a4a01e7186c da1d683b9737 2024-04-07T16:17:48.657 INFO:teuthology.orchestra.run.smithi100.stdout:mon.a smithi100 running (15m) 3m ago 15m 46.3M 2048M 18.2.2-1171-g267c8ff5 8a4a01e7186c ec8833f59ea0 2024-04-07T16:17:48.658 INFO:teuthology.orchestra.run.smithi100.stdout:mon.b smithi144 running (13m) 3m ago 13m 39.7M 2048M 18.2.2-1171-g267c8ff5 8a4a01e7186c 8dd687691b2e 2024-04-07T16:17:48.658 INFO:teuthology.orchestra.run.smithi100.stdout:mon.c smithi100 running (13m) 3m ago 13m 37.7M 2048M 18.2.2-1171-g267c8ff5 8a4a01e7186c a6de0151a3ba 2024-04-07T16:17:48.658 INFO:teuthology.orchestra.run.smithi100.stdout:node-exporter.a smithi100 *:9100 running (4m) 3m ago 4m 3116k - 1.5.0 0da6a335fe13 4bce264c1b26 2024-04-07T16:17:48.658 INFO:teuthology.orchestra.run.smithi100.stdout:node-exporter.b smithi144 *:9100 running (4m) 3m ago 4m 3151k - 1.5.0 0da6a335fe13 b6963413c560 2024-04-07T16:17:48.658 INFO:teuthology.orchestra.run.smithi100.stdout:osd.0 smithi100 running (12m) 3m ago 12m 45.7M 3036M 18.2.2-1171-g267c8ff5 8a4a01e7186c 188deb247979 2024-04-07T16:17:48.658 INFO:teuthology.orchestra.run.smithi100.stdout:osd.1 smithi100 running (11m) 3m ago 11m 44.6M 3036M 18.2.2-1171-g267c8ff5 8a4a01e7186c 7f57ca90894c 2024-04-07T16:17:48.658 INFO:teuthology.orchestra.run.smithi100.stdout:osd.2 smithi100 running (10m) 3m ago 10m 52.2M 3036M 18.2.2-1171-g267c8ff5 8a4a01e7186c f3912ddcc8cc 2024-04-07T16:17:48.658 INFO:teuthology.orchestra.run.smithi100.stdout:osd.3 smithi100 running (9m) 3m ago 9m 41.7M 3036M 18.2.2-1171-g267c8ff5 8a4a01e7186c b41004090c63 2024-04-07T16:17:48.658 INFO:teuthology.orchestra.run.smithi100.stdout:osd.4 smithi144 running (8m) 3m ago 8m 50.6M 3036M 18.2.2-1171-g267c8ff5 8a4a01e7186c 8c73b6b4dc13 2024-04-07T16:17:48.658 INFO:teuthology.orchestra.run.smithi100.stdout:osd.5 smithi144 running (8m) 3m ago 8m 42.6M 3036M 18.2.2-1171-g267c8ff5 8a4a01e7186c 3bfa0cf6f6cb 2024-04-07T16:17:48.658 INFO:teuthology.orchestra.run.smithi100.stdout:osd.6 smithi144 running (7m) 3m ago 7m 42.5M 3036M 18.2.2-1171-g267c8ff5 8a4a01e7186c 32b7f95d77f0 2024-04-07T16:17:48.658 INFO:teuthology.orchestra.run.smithi100.stdout:osd.7 smithi144 running (6m) 3m ago 6m 49.8M 3036M 18.2.2-1171-g267c8ff5 8a4a01e7186c 45f7709e576a 2024-04-07T16:17:48.658 INFO:teuthology.orchestra.run.smithi100.stdout:prometheus.a smithi144 *:9095 running (3m) 3m ago 5m 23.0M - 2.43.0 a07b618ecd1d e456da14a41a 2024-04-07T16:17:48.833 INFO:tasks.rados.rados.0.smithi100.stdout:366: done (4 left) 2024-04-07T16:17:48.833 INFO:tasks.rados.rados.0.smithi100.stdout:367: done (3 left) 2024-04-07T16:17:48.833 INFO:tasks.rados.rados.0.smithi100.stdout:369: done (2 left) 2024-04-07T16:17:48.833 INFO:tasks.rados.rados.0.smithi100.stdout:370: done (1 left) 2024-04-07T16:17:48.833 INFO:tasks.rados.rados.0.smithi100.stdout:371: done (0 left) 2024-04-07T16:17:48.833 INFO:tasks.rados.rados.0.smithi100.stdout:372: read oid 31 snap -1 2024-04-07T16:17:48.833 INFO:tasks.rados.rados.0.smithi100.stdout:372: expect deleted 2024-04-07T16:17:48.834 INFO:tasks.rados.rados.0.smithi100.stdout:373: rollback oid 22 current snap is 33 2024-04-07T16:17:48.834 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 22 to 30 2024-04-07T16:17:48.834 INFO:tasks.rados.rados.0.smithi100.stdout:374: read oid 48 snap -1 2024-04-07T16:17:48.834 INFO:tasks.rados.rados.0.smithi100.stdout:374: expect (ObjNum 46 snap 0 seq_num 46) 2024-04-07T16:17:48.834 INFO:tasks.rados.rados.0.smithi100.stdout:375: rollback oid 3 current snap is 33 2024-04-07T16:17:48.834 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 3 to 24 2024-04-07T16:17:48.834 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 18 current snap is 33 2024-04-07T16:17:48.834 INFO:tasks.rados.rados.0.smithi100.stdout:376: seq_num 130 ranges {1417216=581632,1998848=614400,2613248=598016,3211264=393216} 2024-04-07T16:17:48.852 INFO:tasks.rados.rados.0.smithi100.stdout:376: writing smithi10066867-18 from 1417216 to 1998848 tid 1 2024-04-07T16:17:48.858 INFO:tasks.rados.rados.0.smithi100.stdout:376: writing smithi10066867-18 from 1998848 to 2613248 tid 2 2024-04-07T16:17:48.863 INFO:tasks.rados.rados.0.smithi100.stdout:376: writing smithi10066867-18 from 2613248 to 3211264 tid 3 2024-04-07T16:17:48.867 INFO:tasks.rados.rados.0.smithi100.stdout:376: writing smithi10066867-18 from 3211264 to 3604480 tid 4 2024-04-07T16:17:48.867 INFO:tasks.rados.rados.0.smithi100.stdout:377: rmattr oid 24 current snap is 33 2024-04-07T16:17:48.867 INFO:tasks.rados.rados.0.smithi100.stdout:377: done (5 left) 2024-04-07T16:17:48.867 INFO:tasks.rados.rados.0.smithi100.stdout:378: rmattr oid 41 current snap is 33 2024-04-07T16:17:48.867 INFO:tasks.rados.rados.0.smithi100.stdout:375: finishing rollback tid 0 to smithi10066867-3 2024-04-07T16:17:48.867 INFO:tasks.rados.rados.0.smithi100.stdout:375: finishing rollback tid 1 to smithi10066867-3 2024-04-07T16:17:48.867 INFO:tasks.rados.rados.0.smithi100.stdout:375: finishing rollback tid 2 to smithi10066867-3 2024-04-07T16:17:48.867 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 3 v 55 (ObjNum 80 snap 13 seq_num 80) dirty exists 2024-04-07T16:17:48.867 INFO:tasks.rados.rados.0.smithi100.stdout:373: finishing rollback tid 0 to smithi10066867-22 2024-04-07T16:17:48.867 INFO:tasks.rados.rados.0.smithi100.stdout:372: done (5 left) 2024-04-07T16:17:48.867 INFO:tasks.rados.rados.0.smithi100.stdout:375: done (4 left) 2024-04-07T16:17:48.867 INFO:tasks.rados.rados.0.smithi100.stdout:378: done (3 left) 2024-04-07T16:17:48.867 INFO:tasks.rados.rados.0.smithi100.stdout:379: rmattr oid 44 current snap is 33 2024-04-07T16:17:48.867 INFO:tasks.rados.rados.0.smithi100.stdout:379: done (3 left) 2024-04-07T16:17:48.867 INFO:tasks.rados.rados.0.smithi100.stdout:380: copy_from oid 13 from oid 3 current snap is 33 2024-04-07T16:17:48.868 INFO:tasks.rados.rados.0.smithi100.stdout:373: finishing rollback tid 1 to smithi10066867-22 2024-04-07T16:17:48.868 INFO:tasks.rados.rados.0.smithi100.stdout:373: finishing rollback tid 2 to smithi10066867-22 2024-04-07T16:17:48.868 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 22 v 82 (ObjNum 161 snap 0 seq_num 1) dirty exists 2024-04-07T16:17:48.868 INFO:tasks.rados.rados.0.smithi100.stdout:374: expect (ObjNum 46 snap 0 seq_num 46) 2024-04-07T16:17:48.895 INFO:tasks.rados.rados.0.smithi100.stdout:376: finishing write tid 1 to smithi10066867-18 2024-04-07T16:17:48.895 INFO:tasks.rados.rados.0.smithi100.stdout:376: finishing write tid 2 to smithi10066867-18 2024-04-07T16:17:48.895 INFO:tasks.rados.rados.0.smithi100.stdout:376: finishing write tid 3 to smithi10066867-18 2024-04-07T16:17:48.896 INFO:tasks.rados.rados.0.smithi100.stdout:376: finishing write tid 4 to smithi10066867-18 2024-04-07T16:17:48.896 INFO:tasks.rados.rados.0.smithi100.stdout:376: finishing write tid 5 to smithi10066867-18 2024-04-07T16:17:48.896 INFO:tasks.rados.rados.0.smithi100.stdout:376: finishing write tid 6 to smithi10066867-18 2024-04-07T16:17:48.896 INFO:tasks.rados.rados.0.smithi100.stdout:376: oid 18 updating version 0 to 150 2024-04-07T16:17:48.896 INFO:tasks.rados.rados.0.smithi100.stdout:376: oid 18 version 150 is already newer than 148 2024-04-07T16:17:48.896 INFO:tasks.rados.rados.0.smithi100.stdout:376: oid 18 updating version 150 to 151 2024-04-07T16:17:48.896 INFO:tasks.rados.rados.0.smithi100.stdout:376: oid 18 version 151 is already newer than 149 2024-04-07T16:17:48.896 INFO:tasks.rados.rados.0.smithi100.stdout:376: oid 18 version 151 is already newer than 147 2024-04-07T16:17:48.896 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 18 v 151 (ObjNum 130 snap 33 seq_num 130) dirty exists 2024-04-07T16:17:48.896 INFO:tasks.rados.rados.0.smithi100.stdout:376: left oid 18 (ObjNum 130 snap 33 seq_num 130) 2024-04-07T16:17:48.896 INFO:tasks.rados.rados.0.smithi100.stdout:380: finishing copy_from racing read to smithi10066867-13 2024-04-07T16:17:48.896 INFO:tasks.rados.rados.0.smithi100.stdout:380: finishing copy_from to smithi10066867-13 2024-04-07T16:17:48.896 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 13 v 55 (ObjNum 80 snap 13 seq_num 80) dirty exists 2024-04-07T16:17:48.896 INFO:tasks.rados.rados.0.smithi100.stdout:373: done (3 left) 2024-04-07T16:17:48.897 INFO:tasks.rados.rados.0.smithi100.stdout:374: done (2 left) 2024-04-07T16:17:48.897 INFO:tasks.rados.rados.0.smithi100.stdout:376: done (1 left) 2024-04-07T16:17:48.897 INFO:tasks.rados.rados.0.smithi100.stdout:380: done (0 left) 2024-04-07T16:17:48.897 INFO:tasks.rados.rados.0.smithi100.stdout:381: delete oid 43 current snap is 33 2024-04-07T16:17:48.897 INFO:tasks.rados.rados.0.smithi100.stdout:381: done (0 left) 2024-04-07T16:17:48.897 INFO:tasks.rados.rados.0.smithi100.stdout:382: rollback oid 30 current snap is 33 2024-04-07T16:17:48.898 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 30 to 26 2024-04-07T16:17:48.898 INFO:tasks.rados.rados.0.smithi100.stdout:383: read oid 19 snap -1 2024-04-07T16:17:48.898 INFO:tasks.rados.rados.0.smithi100.stdout:383: expect (ObjNum 126 snap 31 seq_num 126) 2024-04-07T16:17:48.898 INFO:tasks.rados.rados.0.smithi100.stdout:384: snap_remove snap 27 2024-04-07T16:17:49.105 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-04-07T16:17:49.106 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": { 2024-04-07T16:17:49.106 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph version 18.2.2-1171-g267c8ff5 (267c8ff5b1ff5815581b3b2381d19360e081b37b) reef (stable)": 3 2024-04-07T16:17:49.106 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-07T16:17:49.106 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": { 2024-04-07T16:17:49.106 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph version 18.2.2-1171-g267c8ff5 (267c8ff5b1ff5815581b3b2381d19360e081b37b) reef (stable)": 2 2024-04-07T16:17:49.106 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-07T16:17:49.106 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": { 2024-04-07T16:17:49.106 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph version 18.2.2-1171-g267c8ff5 (267c8ff5b1ff5815581b3b2381d19360e081b37b) reef (stable)": 8 2024-04-07T16:17:49.106 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-07T16:17:49.106 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": { 2024-04-07T16:17:49.106 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph version 18.2.2-1171-g267c8ff5 (267c8ff5b1ff5815581b3b2381d19360e081b37b) reef (stable)": 2 2024-04-07T16:17:49.106 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-07T16:17:49.107 INFO:teuthology.orchestra.run.smithi100.stdout: "overall": { 2024-04-07T16:17:49.107 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph version 18.2.2-1171-g267c8ff5 (267c8ff5b1ff5815581b3b2381d19360e081b37b) reef (stable)": 15 2024-04-07T16:17:49.107 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-04-07T16:17:49.107 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-04-07T16:17:49.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:17:48 smithi144 bash[23522]: audit 2024-04-07T16:17:48.294646+0000 mgr.y (mgr.14553) 214 : audit 0 from='client.15057 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:17:49.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:17:48 smithi144 bash[23522]: cluster 2024-04-07T16:17:48.845177+0000 mon.a (mon.0) 955 : cluster 0 osdmap e134: 8 total, 8 up, 8 in 2024-04-07T16:17:49.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:17:48 smithi100 bash[21346]: audit 2024-04-07T16:17:48.294646+0000 mgr.y (mgr.14553) 214 : audit 0 from='client.15057 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:17:49.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:17:48 smithi100 bash[21346]: cluster 2024-04-07T16:17:48.845177+0000 mon.a (mon.0) 955 : cluster 0 osdmap e134: 8 total, 8 up, 8 in 2024-04-07T16:17:49.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:17:48 smithi100 bash[31660]: audit 2024-04-07T16:17:48.294646+0000 mgr.y (mgr.14553) 214 : audit 0 from='client.15057 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:17:49.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:17:48 smithi100 bash[31660]: cluster 2024-04-07T16:17:48.845177+0000 mon.a (mon.0) 955 : cluster 0 osdmap e134: 8 total, 8 up, 8 in 2024-04-07T16:17:49.843 INFO:tasks.rados.rados.0.smithi100.stdout:382: finishing rollback tid 0 to smithi10066867-30 2024-04-07T16:17:49.843 INFO:tasks.rados.rados.0.smithi100.stdout:382: finishing rollback tid 1 to smithi10066867-30 2024-04-07T16:17:49.843 INFO:tasks.rados.rados.0.smithi100.stdout:382: finishing rollback tid 2 to smithi10066867-30 2024-04-07T16:17:49.844 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 30 v 87 (ObjNum 106 snap 20 seq_num 106) dirty exists 2024-04-07T16:17:49.844 INFO:tasks.rados.rados.0.smithi100.stdout:383: expect (ObjNum 126 snap 31 seq_num 126) 2024-04-07T16:17:49.987 INFO:tasks.rados.rados.0.smithi100.stdout:382: done (2 left) 2024-04-07T16:17:49.987 INFO:tasks.rados.rados.0.smithi100.stdout:383: done (1 left) 2024-04-07T16:17:49.988 INFO:tasks.rados.rados.0.smithi100.stdout:384: done (0 left) 2024-04-07T16:17:49.988 INFO:tasks.rados.rados.0.smithi100.stdout:385: delete oid 18 current snap is 33 2024-04-07T16:17:49.996 INFO:tasks.rados.rados.0.smithi100.stdout:385: done (0 left) 2024-04-07T16:17:49.996 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 12 current snap is 33 2024-04-07T16:17:49.996 INFO:tasks.rados.rados.0.smithi100.stdout:386: seq_num 133 ranges {0=630784,630784=663552,1294336=770048,2064384=172032} 2024-04-07T16:17:50.007 INFO:tasks.rados.rados.0.smithi100.stdout:386: writing smithi10066867-12 from 0 to 630784 tid 1 2024-04-07T16:17:50.018 INFO:tasks.rados.rados.0.smithi100.stdout:386: writing smithi10066867-12 from 630784 to 1294336 tid 2 2024-04-07T16:17:50.032 INFO:tasks.rados.rados.0.smithi100.stdout:386: writing smithi10066867-12 from 1294336 to 2064384 tid 3 2024-04-07T16:17:50.035 INFO:tasks.rados.rados.0.smithi100.stdout:386: writing smithi10066867-12 from 2064384 to 2236416 tid 4 2024-04-07T16:17:50.036 INFO:tasks.rados.rados.0.smithi100.stdout:387: snap_remove snap 24 2024-04-07T16:17:50.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:17:49 smithi144 bash[23522]: audit 2024-04-07T16:17:48.649982+0000 mgr.y (mgr.14553) 215 : audit 0 from='client.24971 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:17:50.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:17:49 smithi144 bash[23522]: cluster 2024-04-07T16:17:48.711465+0000 mgr.y (mgr.14553) 216 : cluster 0 pgmap v219: 121 pgs: 121 active+clean; 76 MiB data, 359 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:17:50.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:17:49 smithi144 bash[23522]: audit 2024-04-07T16:17:49.120878+0000 mon.a (mon.0) 956 : audit 0 from='client.? 172.21.15.100:0/1607938394' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:17:50.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:17:49 smithi144 bash[23522]: cluster 2024-04-07T16:17:49.850977+0000 mon.a (mon.0) 957 : cluster 0 osdmap e135: 8 total, 8 up, 8 in 2024-04-07T16:17:50.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:17:49 smithi100 bash[21346]: audit 2024-04-07T16:17:48.649982+0000 mgr.y (mgr.14553) 215 : audit 0 from='client.24971 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:17:50.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:17:49 smithi100 bash[21346]: cluster 2024-04-07T16:17:48.711465+0000 mgr.y (mgr.14553) 216 : cluster 0 pgmap v219: 121 pgs: 121 active+clean; 76 MiB data, 359 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:17:50.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:17:49 smithi100 bash[21346]: audit 2024-04-07T16:17:49.120878+0000 mon.a (mon.0) 956 : audit 0 from='client.? 172.21.15.100:0/1607938394' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:17:50.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:17:49 smithi100 bash[21346]: cluster 2024-04-07T16:17:49.850977+0000 mon.a (mon.0) 957 : cluster 0 osdmap e135: 8 total, 8 up, 8 in 2024-04-07T16:17:50.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:17:49 smithi100 bash[31660]: audit 2024-04-07T16:17:48.649982+0000 mgr.y (mgr.14553) 215 : audit 0 from='client.24971 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:17:50.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:17:49 smithi100 bash[31660]: cluster 2024-04-07T16:17:48.711465+0000 mgr.y (mgr.14553) 216 : cluster 0 pgmap v219: 121 pgs: 121 active+clean; 76 MiB data, 359 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:17:50.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:17:49 smithi100 bash[31660]: audit 2024-04-07T16:17:49.120878+0000 mon.a (mon.0) 956 : audit 0 from='client.? 172.21.15.100:0/1607938394' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:17:50.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:17:49 smithi100 bash[31660]: cluster 2024-04-07T16:17:49.850977+0000 mon.a (mon.0) 957 : cluster 0 osdmap e135: 8 total, 8 up, 8 in 2024-04-07T16:17:50.977 INFO:tasks.rados.rados.0.smithi100.stdout:387: done (1 left) 2024-04-07T16:17:50.977 INFO:tasks.rados.rados.0.smithi100.stdout:388: snap_remove snap 30 2024-04-07T16:17:50.977 INFO:tasks.rados.rados.0.smithi100.stdout:386: finishing write tid 1 to smithi10066867-12 2024-04-07T16:17:50.977 INFO:tasks.rados.rados.0.smithi100.stdout:386: finishing write tid 2 to smithi10066867-12 2024-04-07T16:17:50.977 INFO:tasks.rados.rados.0.smithi100.stdout:386: finishing write tid 3 to smithi10066867-12 2024-04-07T16:17:50.978 INFO:tasks.rados.rados.0.smithi100.stdout:386: finishing write tid 4 to smithi10066867-12 2024-04-07T16:17:50.978 INFO:tasks.rados.rados.0.smithi100.stdout:386: finishing write tid 5 to smithi10066867-12 2024-04-07T16:17:50.978 INFO:tasks.rados.rados.0.smithi100.stdout:386: finishing write tid 6 to smithi10066867-12 2024-04-07T16:17:50.978 INFO:tasks.rados.rados.0.smithi100.stdout:386: oid 12 updating version 0 to 132 2024-04-07T16:17:50.978 INFO:tasks.rados.rados.0.smithi100.stdout:386: oid 12 version 132 is already newer than 130 2024-04-07T16:17:50.978 INFO:tasks.rados.rados.0.smithi100.stdout:386: oid 12 version 132 is already newer than 129 2024-04-07T16:17:50.978 INFO:tasks.rados.rados.0.smithi100.stdout:386: oid 12 version 132 is already newer than 131 2024-04-07T16:17:50.978 INFO:tasks.rados.rados.0.smithi100.stdout:386: oid 12 updating version 132 to 133 2024-04-07T16:17:50.978 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 12 v 133 (ObjNum 133 snap 33 seq_num 133) dirty exists 2024-04-07T16:17:50.978 INFO:tasks.rados.rados.0.smithi100.stdout:386: left oid 12 (ObjNum 133 snap 33 seq_num 133) 2024-04-07T16:17:51.978 INFO:tasks.rados.rados.0.smithi100.stdout:386: done (1 left) 2024-04-07T16:17:51.978 INFO:tasks.rados.rados.0.smithi100.stdout:388: done (0 left) 2024-04-07T16:17:51.978 INFO:tasks.rados.rados.0.smithi100.stdout:389: snap_remove snap 32 2024-04-07T16:17:52.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:17:51 smithi144 bash[23522]: cluster 2024-04-07T16:17:50.712133+0000 mgr.y (mgr.14553) 217 : cluster 0 pgmap v222: 121 pgs: 121 active+clean; 86 MiB data, 380 MiB used, 715 GiB / 715 GiB avail; 1.9 MiB/s rd, 2.9 MiB/s wr, 18 op/s 2024-04-07T16:17:52.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:17:51 smithi144 bash[23522]: cluster 2024-04-07T16:17:50.986205+0000 mon.a (mon.0) 958 : cluster 0 osdmap e136: 8 total, 8 up, 8 in 2024-04-07T16:17:52.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:17:51 smithi100 bash[21346]: cluster 2024-04-07T16:17:50.712133+0000 mgr.y (mgr.14553) 217 : cluster 0 pgmap v222: 121 pgs: 121 active+clean; 86 MiB data, 380 MiB used, 715 GiB / 715 GiB avail; 1.9 MiB/s rd, 2.9 MiB/s wr, 18 op/s 2024-04-07T16:17:52.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:17:51 smithi100 bash[21346]: cluster 2024-04-07T16:17:50.986205+0000 mon.a (mon.0) 958 : cluster 0 osdmap e136: 8 total, 8 up, 8 in 2024-04-07T16:17:52.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:17:51 smithi100 bash[31660]: cluster 2024-04-07T16:17:50.712133+0000 mgr.y (mgr.14553) 217 : cluster 0 pgmap v222: 121 pgs: 121 active+clean; 86 MiB data, 380 MiB used, 715 GiB / 715 GiB avail; 1.9 MiB/s rd, 2.9 MiB/s wr, 18 op/s 2024-04-07T16:17:52.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:17:51 smithi100 bash[31660]: cluster 2024-04-07T16:17:50.986205+0000 mon.a (mon.0) 958 : cluster 0 osdmap e136: 8 total, 8 up, 8 in 2024-04-07T16:17:52.837 INFO:tasks.rados.rados.0.smithi100.stdout:389: done (0 left) 2024-04-07T16:17:52.837 INFO:tasks.rados.rados.0.smithi100.stdout:390: snap_remove snap 28 2024-04-07T16:17:53.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:17:53 smithi144 bash[23522]: cluster 2024-04-07T16:17:51.992020+0000 mon.a (mon.0) 959 : cluster 0 osdmap e137: 8 total, 8 up, 8 in 2024-04-07T16:17:53.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:17:53 smithi144 bash[23522]: audit 2024-04-07T16:17:52.704992+0000 mon.b (mon.1) 244 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:17:53.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:17:53 smithi144 bash[23522]: cluster 2024-04-07T16:17:52.847480+0000 mon.a (mon.0) 960 : cluster 0 osdmap e138: 8 total, 8 up, 8 in 2024-04-07T16:17:53.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:17:53 smithi100 bash[21346]: cluster 2024-04-07T16:17:51.992020+0000 mon.a (mon.0) 959 : cluster 0 osdmap e137: 8 total, 8 up, 8 in 2024-04-07T16:17:53.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:17:53 smithi100 bash[21346]: audit 2024-04-07T16:17:52.704992+0000 mon.b (mon.1) 244 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:17:53.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:17:53 smithi100 bash[21346]: cluster 2024-04-07T16:17:52.847480+0000 mon.a (mon.0) 960 : cluster 0 osdmap e138: 8 total, 8 up, 8 in 2024-04-07T16:17:53.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:17:53 smithi100 bash[31660]: cluster 2024-04-07T16:17:51.992020+0000 mon.a (mon.0) 959 : cluster 0 osdmap e137: 8 total, 8 up, 8 in 2024-04-07T16:17:53.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:17:53 smithi100 bash[31660]: audit 2024-04-07T16:17:52.704992+0000 mon.b (mon.1) 244 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:17:53.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:17:53 smithi100 bash[31660]: cluster 2024-04-07T16:17:52.847480+0000 mon.a (mon.0) 960 : cluster 0 osdmap e138: 8 total, 8 up, 8 in 2024-04-07T16:17:53.836 INFO:tasks.rados.rados.0.smithi100.stdout:390: done (0 left) 2024-04-07T16:17:53.837 INFO:tasks.rados.rados.0.smithi100.stdout:391: setattr oid 41 current snap is 33 2024-04-07T16:17:53.838 INFO:tasks.rados.rados.0.smithi100.stdout:392: read oid 26 snap -1 2024-04-07T16:17:53.838 INFO:tasks.rados.rados.0.smithi100.stdout:392: expect (ObjNum 104 snap 20 seq_num 104) 2024-04-07T16:17:53.838 INFO:tasks.rados.rados.0.smithi100.stdout:393: read oid 5 snap -1 2024-04-07T16:17:53.838 INFO:tasks.rados.rados.0.smithi100.stdout:393: expect deleted 2024-04-07T16:17:53.839 INFO:tasks.rados.rados.0.smithi100.stdout:394: read oid 27 snap 25 2024-04-07T16:17:53.839 INFO:tasks.rados.rados.0.smithi100.stdout:394: expect (ObjNum 56 snap 4 seq_num 56) 2024-04-07T16:17:53.839 INFO:tasks.rados.rados.0.smithi100.stdout:395: setattr oid 6 current snap is 33 2024-04-07T16:17:53.840 INFO:tasks.rados.rados.0.smithi100.stdout:396: snap_create 2024-04-07T16:17:53.858 INFO:tasks.rados.rados.0.smithi100.stdout:394: expect (ObjNum 56 snap 4 seq_num 56) 2024-04-07T16:17:53.915 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 41 v 142 (ObjNum 161 snap 0 seq_num 1) dirty exists 2024-04-07T16:17:53.915 INFO:tasks.rados.rados.0.smithi100.stdout:392: expect (ObjNum 104 snap 20 seq_num 104) 2024-04-07T16:17:53.970 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 6 v 124 (ObjNum 124 snap 29 seq_num 124) dirty exists 2024-04-07T16:17:54.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:17:54 smithi144 bash[23522]: cluster 2024-04-07T16:17:52.712775+0000 mgr.y (mgr.14553) 218 : cluster 0 pgmap v225: 121 pgs: 121 active+clean; 86 MiB data, 380 MiB used, 715 GiB / 715 GiB avail; 1.9 MiB/s rd, 2.9 MiB/s wr, 18 op/s 2024-04-07T16:17:54.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:17:54 smithi144 bash[23522]: cluster 2024-04-07T16:17:53.850619+0000 mon.a (mon.0) 961 : cluster 0 osdmap e139: 8 total, 8 up, 8 in 2024-04-07T16:17:54.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:17:54 smithi100 bash[21346]: cluster 2024-04-07T16:17:52.712775+0000 mgr.y (mgr.14553) 218 : cluster 0 pgmap v225: 121 pgs: 121 active+clean; 86 MiB data, 380 MiB used, 715 GiB / 715 GiB avail; 1.9 MiB/s rd, 2.9 MiB/s wr, 18 op/s 2024-04-07T16:17:54.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:17:54 smithi100 bash[21346]: cluster 2024-04-07T16:17:53.850619+0000 mon.a (mon.0) 961 : cluster 0 osdmap e139: 8 total, 8 up, 8 in 2024-04-07T16:17:54.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:17:54 smithi100 bash[31660]: cluster 2024-04-07T16:17:52.712775+0000 mgr.y (mgr.14553) 218 : cluster 0 pgmap v225: 121 pgs: 121 active+clean; 86 MiB data, 380 MiB used, 715 GiB / 715 GiB avail; 1.9 MiB/s rd, 2.9 MiB/s wr, 18 op/s 2024-04-07T16:17:54.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:17:54 smithi100 bash[31660]: cluster 2024-04-07T16:17:53.850619+0000 mon.a (mon.0) 961 : cluster 0 osdmap e139: 8 total, 8 up, 8 in 2024-04-07T16:17:54.840 INFO:tasks.rados.rados.0.smithi100.stdout:391: done (5 left) 2024-04-07T16:17:54.840 INFO:tasks.rados.rados.0.smithi100.stdout:392: done (4 left) 2024-04-07T16:17:54.840 INFO:tasks.rados.rados.0.smithi100.stdout:393: done (3 left) 2024-04-07T16:17:54.840 INFO:tasks.rados.rados.0.smithi100.stdout:394: done (2 left) 2024-04-07T16:17:54.840 INFO:tasks.rados.rados.0.smithi100.stdout:395: done (1 left) 2024-04-07T16:17:54.840 INFO:tasks.rados.rados.0.smithi100.stdout:396: done (0 left) 2024-04-07T16:17:54.840 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 13 current snap is 34 2024-04-07T16:17:54.840 INFO:tasks.rados.rados.0.smithi100.stdout:397: seq_num 134 ranges {1622016=647168,2269184=548864,2818048=573440,3391488=483328} 2024-04-07T16:17:54.876 INFO:tasks.rados.rados.0.smithi100.stdout:397: writing smithi10066867-13 from 1622016 to 2269184 tid 1 2024-04-07T16:17:54.884 INFO:tasks.rados.rados.0.smithi100.stdout:397: writing smithi10066867-13 from 2269184 to 2818048 tid 2 2024-04-07T16:17:54.893 INFO:tasks.rados.rados.0.smithi100.stdout:397: writing smithi10066867-13 from 2818048 to 3391488 tid 3 2024-04-07T16:17:54.899 INFO:tasks.rados.rados.0.smithi100.stdout:397: writing smithi10066867-13 from 3391488 to 3874816 tid 4 2024-04-07T16:17:54.899 INFO:tasks.rados.rados.0.smithi100.stdout:398: delete oid 37 current snap is 34 2024-04-07T16:17:54.899 INFO:tasks.rados.rados.0.smithi100.stdout:397: finishing write tid 1 to smithi10066867-13 2024-04-07T16:17:54.899 INFO:tasks.rados.rados.0.smithi100.stdout:397: finishing write tid 2 to smithi10066867-13 2024-04-07T16:17:54.899 INFO:tasks.rados.rados.0.smithi100.stdout:397: finishing write tid 3 to smithi10066867-13 2024-04-07T16:17:54.901 INFO:tasks.rados.rados.0.smithi100.stdout:398: done (1 left) 2024-04-07T16:17:54.902 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 38 current snap is 34 2024-04-07T16:17:54.902 INFO:tasks.rados.rados.0.smithi100.stdout:399: seq_num 136 ranges {475136=688128,1163264=720896,1884160=540672,2424832=344064} 2024-04-07T16:17:54.914 INFO:tasks.rados.rados.0.smithi100.stdout:399: writing smithi10066867-38 from 475136 to 1163264 tid 1 2024-04-07T16:17:54.921 INFO:tasks.rados.rados.0.smithi100.stdout:399: writing smithi10066867-38 from 1163264 to 1884160 tid 2 2024-04-07T16:17:54.927 INFO:tasks.rados.rados.0.smithi100.stdout:399: writing smithi10066867-38 from 1884160 to 2424832 tid 3 2024-04-07T16:17:54.932 INFO:tasks.rados.rados.0.smithi100.stdout:399: writing smithi10066867-38 from 2424832 to 2768896 tid 4 2024-04-07T16:17:54.932 INFO:tasks.rados.rados.0.smithi100.stdout:400: setattr oid 20 current snap is 34 2024-04-07T16:17:54.932 INFO:tasks.rados.rados.0.smithi100.stdout:397: finishing write tid 4 to smithi10066867-13 2024-04-07T16:17:54.932 INFO:tasks.rados.rados.0.smithi100.stdout:397: finishing write tid 5 to smithi10066867-13 2024-04-07T16:17:54.932 INFO:tasks.rados.rados.0.smithi100.stdout:397: finishing write tid 6 to smithi10066867-13 2024-04-07T16:17:54.932 INFO:tasks.rados.rados.0.smithi100.stdout:397: oid 13 updating version 0 to 96 2024-04-07T16:17:54.932 INFO:tasks.rados.rados.0.smithi100.stdout:397: oid 13 updating version 96 to 97 2024-04-07T16:17:54.932 INFO:tasks.rados.rados.0.smithi100.stdout:397: oid 13 updating version 97 to 99 2024-04-07T16:17:54.932 INFO:tasks.rados.rados.0.smithi100.stdout:397: oid 13 version 99 is already newer than 95 2024-04-07T16:17:54.932 INFO:tasks.rados.rados.0.smithi100.stdout:397: oid 13 version 99 is already newer than 98 2024-04-07T16:17:54.932 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 13 v 99 (ObjNum 134 snap 34 seq_num 134) dirty exists 2024-04-07T16:17:54.933 INFO:tasks.rados.rados.0.smithi100.stdout:397: left oid 13 (ObjNum 134 snap 34 seq_num 134) 2024-04-07T16:17:54.933 INFO:tasks.rados.rados.0.smithi100.stdout:399: finishing write tid 1 to smithi10066867-38 2024-04-07T16:17:54.933 INFO:tasks.rados.rados.0.smithi100.stdout:399: finishing write tid 2 to smithi10066867-38 2024-04-07T16:17:54.933 INFO:tasks.rados.rados.0.smithi100.stdout:399: finishing write tid 3 to smithi10066867-38 2024-04-07T16:17:54.935 INFO:tasks.rados.rados.0.smithi100.stdout:397: done (2 left) 2024-04-07T16:17:54.935 INFO:tasks.rados.rados.0.smithi100.stdout:401: read oid 14 snap -1 2024-04-07T16:17:54.936 INFO:tasks.rados.rados.0.smithi100.stdout:401: expect (ObjNum 28 snap 0 seq_num 28) 2024-04-07T16:17:54.936 INFO:tasks.rados.rados.0.smithi100.stdout:402: snap_create 2024-04-07T16:17:54.937 INFO:tasks.rados.rados.0.smithi100.stdout:399: finishing write tid 4 to smithi10066867-38 2024-04-07T16:17:54.938 INFO:tasks.rados.rados.0.smithi100.stdout:399: finishing write tid 5 to smithi10066867-38 2024-04-07T16:17:54.941 INFO:tasks.rados.rados.0.smithi100.stdout:399: finishing write tid 6 to smithi10066867-38 2024-04-07T16:17:54.941 INFO:tasks.rados.rados.0.smithi100.stdout:399: oid 38 updating version 0 to 74 2024-04-07T16:17:54.941 INFO:tasks.rados.rados.0.smithi100.stdout:399: oid 38 version 74 is already newer than 73 2024-04-07T16:17:54.941 INFO:tasks.rados.rados.0.smithi100.stdout:399: oid 38 updating version 74 to 75 2024-04-07T16:17:54.941 INFO:tasks.rados.rados.0.smithi100.stdout:399: oid 38 updating version 75 to 77 2024-04-07T16:17:54.941 INFO:tasks.rados.rados.0.smithi100.stdout:399: oid 38 version 77 is already newer than 76 2024-04-07T16:17:54.941 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 38 v 77 (ObjNum 136 snap 34 seq_num 136) dirty exists 2024-04-07T16:17:54.941 INFO:tasks.rados.rados.0.smithi100.stdout:399: left oid 38 (ObjNum 136 snap 34 seq_num 136) 2024-04-07T16:17:54.943 INFO:tasks.rados.rados.0.smithi100.stdout:401: expect (ObjNum 28 snap 0 seq_num 28) 2024-04-07T16:17:54.958 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 20 v 93 (ObjNum 4113 snap 0 seq_num 1049730242) dirty exists 2024-04-07T16:17:55.437 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:17:55 smithi100 bash[21601]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:17:55] "GET /metrics HTTP/1.1" 200 36080 "" "Prometheus/2.43.0" 2024-04-07T16:17:55.843 INFO:tasks.rados.rados.0.smithi100.stdout:399: done (3 left) 2024-04-07T16:17:55.843 INFO:tasks.rados.rados.0.smithi100.stdout:400: done (2 left) 2024-04-07T16:17:55.843 INFO:tasks.rados.rados.0.smithi100.stdout:401: done (1 left) 2024-04-07T16:17:55.843 INFO:tasks.rados.rados.0.smithi100.stdout:402: done (0 left) 2024-04-07T16:17:55.843 INFO:tasks.rados.rados.0.smithi100.stdout:403: delete oid 27 current snap is 35 2024-04-07T16:17:55.863 INFO:tasks.rados.rados.0.smithi100.stdout:403: done (0 left) 2024-04-07T16:17:55.863 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 3 current snap is 35 2024-04-07T16:17:55.863 INFO:tasks.rados.rados.0.smithi100.stdout:404: seq_num 138 ranges {1622016=720896,2342912=540672,2883584=557056,3440640=507904} 2024-04-07T16:17:55.886 INFO:tasks.rados.rados.0.smithi100.stdout:404: writing smithi10066867-3 from 1622016 to 2342912 tid 1 2024-04-07T16:17:55.892 INFO:tasks.rados.rados.0.smithi100.stdout:404: writing smithi10066867-3 from 2342912 to 2883584 tid 2 2024-04-07T16:17:55.899 INFO:tasks.rados.rados.0.smithi100.stdout:404: writing smithi10066867-3 from 2883584 to 3440640 tid 3 2024-04-07T16:17:55.905 INFO:tasks.rados.rados.0.smithi100.stdout:404: writing smithi10066867-3 from 3440640 to 3948544 tid 4 2024-04-07T16:17:55.905 INFO:tasks.rados.rados.0.smithi100.stdout:405: snap_create 2024-04-07T16:17:55.905 INFO:tasks.rados.rados.0.smithi100.stdout:404: finishing write tid 1 to smithi10066867-3 2024-04-07T16:17:55.905 INFO:tasks.rados.rados.0.smithi100.stdout:404: finishing write tid 2 to smithi10066867-3 2024-04-07T16:17:55.905 INFO:tasks.rados.rados.0.smithi100.stdout:404: finishing write tid 3 to smithi10066867-3 2024-04-07T16:17:55.910 INFO:tasks.rados.rados.0.smithi100.stdout:404: finishing write tid 4 to smithi10066867-3 2024-04-07T16:17:55.911 INFO:tasks.rados.rados.0.smithi100.stdout:404: finishing write tid 5 to smithi10066867-3 2024-04-07T16:17:55.913 INFO:tasks.rados.rados.0.smithi100.stdout:404: finishing write tid 6 to smithi10066867-3 2024-04-07T16:17:55.913 INFO:tasks.rados.rados.0.smithi100.stdout:404: oid 3 updating version 0 to 79 2024-04-07T16:17:55.914 INFO:tasks.rados.rados.0.smithi100.stdout:404: oid 3 updating version 79 to 82 2024-04-07T16:17:55.914 INFO:tasks.rados.rados.0.smithi100.stdout:404: oid 3 version 82 is already newer than 81 2024-04-07T16:17:55.914 INFO:tasks.rados.rados.0.smithi100.stdout:404: oid 3 updating version 82 to 83 2024-04-07T16:17:55.914 INFO:tasks.rados.rados.0.smithi100.stdout:404: oid 3 version 83 is already newer than 80 2024-04-07T16:17:55.914 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 3 v 83 (ObjNum 138 snap 35 seq_num 138) dirty exists 2024-04-07T16:17:55.914 INFO:tasks.rados.rados.0.smithi100.stdout:404: left oid 3 (ObjNum 138 snap 35 seq_num 138) 2024-04-07T16:17:56.105 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:17:55 smithi144 bash[23522]: cluster 2024-04-07T16:17:54.713490+0000 mgr.y (mgr.14553) 219 : cluster 0 pgmap v228: 121 pgs: 121 active+clean; 81 MiB data, 379 MiB used, 715 GiB / 715 GiB avail; 3.2 MiB/s rd, 1.4 MiB/s wr, 7 op/s 2024-04-07T16:17:56.105 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:17:55 smithi144 bash[23522]: cluster 2024-04-07T16:17:54.854347+0000 mon.a (mon.0) 962 : cluster 0 osdmap e140: 8 total, 8 up, 8 in 2024-04-07T16:17:56.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:17:55 smithi100 bash[21346]: cluster 2024-04-07T16:17:54.713490+0000 mgr.y (mgr.14553) 219 : cluster 0 pgmap v228: 121 pgs: 121 active+clean; 81 MiB data, 379 MiB used, 715 GiB / 715 GiB avail; 3.2 MiB/s rd, 1.4 MiB/s wr, 7 op/s 2024-04-07T16:17:56.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:17:55 smithi100 bash[21346]: cluster 2024-04-07T16:17:54.854347+0000 mon.a (mon.0) 962 : cluster 0 osdmap e140: 8 total, 8 up, 8 in 2024-04-07T16:17:56.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:17:55 smithi100 bash[31660]: cluster 2024-04-07T16:17:54.713490+0000 mgr.y (mgr.14553) 219 : cluster 0 pgmap v228: 121 pgs: 121 active+clean; 81 MiB data, 379 MiB used, 715 GiB / 715 GiB avail; 3.2 MiB/s rd, 1.4 MiB/s wr, 7 op/s 2024-04-07T16:17:56.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:17:55 smithi100 bash[31660]: cluster 2024-04-07T16:17:54.854347+0000 mon.a (mon.0) 962 : cluster 0 osdmap e140: 8 total, 8 up, 8 in 2024-04-07T16:17:56.863 INFO:tasks.rados.rados.0.smithi100.stdout:404: done (1 left) 2024-04-07T16:17:56.863 INFO:tasks.rados.rados.0.smithi100.stdout:405: done (0 left) 2024-04-07T16:17:56.863 INFO:tasks.rados.rados.0.smithi100.stdout:406: rollback oid 27 current snap is 36 2024-04-07T16:17:56.863 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 27 to 35 2024-04-07T16:17:56.863 INFO:tasks.rados.rados.0.smithi100.stdout:407: setattr oid 20 current snap is 36 2024-04-07T16:17:56.866 INFO:tasks.rados.rados.0.smithi100.stdout:408: snap_create 2024-04-07T16:17:56.866 INFO:tasks.rados.rados.0.smithi100.stdout:406: finishing rollback tid 1 to smithi10066867-27 2024-04-07T16:17:56.866 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 27 v 50 (ObjNum 129 snap 0 seq_num 2702367512) dirty dne 2024-04-07T16:17:56.878 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 20 v 95 (ObjNum 273 snap 0 seq_num 0) dirty exists 2024-04-07T16:17:57.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:17:56 smithi100 bash[21346]: cluster 2024-04-07T16:17:55.854856+0000 mon.a (mon.0) 963 : cluster 0 osdmap e141: 8 total, 8 up, 8 in 2024-04-07T16:17:57.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:17:56 smithi100 bash[31660]: cluster 2024-04-07T16:17:55.854856+0000 mon.a (mon.0) 963 : cluster 0 osdmap e141: 8 total, 8 up, 8 in 2024-04-07T16:17:57.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:17:56 smithi144 bash[23522]: cluster 2024-04-07T16:17:55.854856+0000 mon.a (mon.0) 963 : cluster 0 osdmap e141: 8 total, 8 up, 8 in 2024-04-07T16:17:57.837 INFO:tasks.rados.rados.0.smithi100.stdout:406: done (2 left) 2024-04-07T16:17:57.837 INFO:tasks.rados.rados.0.smithi100.stdout:407: done (1 left) 2024-04-07T16:17:57.837 INFO:tasks.rados.rados.0.smithi100.stdout:408: done (0 left) 2024-04-07T16:17:57.837 INFO:tasks.rados.rados.0.smithi100.stdout:409: read oid 46 snap -1 2024-04-07T16:17:57.837 INFO:tasks.rados.rados.0.smithi100.stdout:409: expect (ObjNum 45 snap 0 seq_num 45) 2024-04-07T16:17:57.837 INFO:tasks.rados.rados.0.smithi100.stdout:410: snap_remove snap 36 2024-04-07T16:17:58.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:17:57 smithi100 bash[21346]: cluster 2024-04-07T16:17:56.714189+0000 mgr.y (mgr.14553) 220 : cluster 0 pgmap v231: 121 pgs: 121 active+clean; 81 MiB data, 379 MiB used, 715 GiB / 715 GiB avail; 3.2 MiB/s rd, 1.4 MiB/s wr, 7 op/s 2024-04-07T16:17:58.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:17:57 smithi100 bash[21346]: cluster 2024-04-07T16:17:56.877694+0000 mon.a (mon.0) 964 : cluster 0 osdmap e142: 8 total, 8 up, 8 in 2024-04-07T16:17:58.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:17:57 smithi100 bash[21346]: cluster 2024-04-07T16:17:57.851895+0000 mon.a (mon.0) 965 : cluster 0 osdmap e143: 8 total, 8 up, 8 in 2024-04-07T16:17:58.188 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:17:57 smithi100 bash[31660]: cluster 2024-04-07T16:17:56.714189+0000 mgr.y (mgr.14553) 220 : cluster 0 pgmap v231: 121 pgs: 121 active+clean; 81 MiB data, 379 MiB used, 715 GiB / 715 GiB avail; 3.2 MiB/s rd, 1.4 MiB/s wr, 7 op/s 2024-04-07T16:17:58.188 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:17:57 smithi100 bash[31660]: cluster 2024-04-07T16:17:56.877694+0000 mon.a (mon.0) 964 : cluster 0 osdmap e142: 8 total, 8 up, 8 in 2024-04-07T16:17:58.188 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:17:57 smithi100 bash[31660]: cluster 2024-04-07T16:17:57.851895+0000 mon.a (mon.0) 965 : cluster 0 osdmap e143: 8 total, 8 up, 8 in 2024-04-07T16:17:58.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:17:57 smithi144 bash[23522]: cluster 2024-04-07T16:17:56.714189+0000 mgr.y (mgr.14553) 220 : cluster 0 pgmap v231: 121 pgs: 121 active+clean; 81 MiB data, 379 MiB used, 715 GiB / 715 GiB avail; 3.2 MiB/s rd, 1.4 MiB/s wr, 7 op/s 2024-04-07T16:17:58.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:17:57 smithi144 bash[23522]: cluster 2024-04-07T16:17:56.877694+0000 mon.a (mon.0) 964 : cluster 0 osdmap e142: 8 total, 8 up, 8 in 2024-04-07T16:17:58.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:17:57 smithi144 bash[23522]: cluster 2024-04-07T16:17:57.851895+0000 mon.a (mon.0) 965 : cluster 0 osdmap e143: 8 total, 8 up, 8 in 2024-04-07T16:17:58.841 INFO:tasks.rados.rados.0.smithi100.stdout:410: done (1 left) 2024-04-07T16:17:58.841 INFO:tasks.rados.rados.0.smithi100.stdout:411: read oid 12 snap -1 2024-04-07T16:17:58.841 INFO:tasks.rados.rados.0.smithi100.stdout:411: expect (ObjNum 133 snap 33 seq_num 133) 2024-04-07T16:17:58.841 INFO:tasks.rados.rados.0.smithi100.stdout:412: read oid 38 snap 22 2024-04-07T16:17:58.841 INFO:tasks.rados.rados.0.smithi100.stdout:412: expect (ObjNum 28 snap 0 seq_num 28) 2024-04-07T16:17:58.841 INFO:tasks.rados.rados.0.smithi100.stdout:413: snap_create 2024-04-07T16:17:58.841 INFO:tasks.rados.rados.0.smithi100.stdout:409: expect (ObjNum 45 snap 0 seq_num 45) 2024-04-07T16:17:58.901 INFO:tasks.rados.rados.0.smithi100.stdout:412: expect (ObjNum 28 snap 0 seq_num 28) 2024-04-07T16:17:58.922 INFO:tasks.rados.rados.0.smithi100.stdout:411: expect (ObjNum 133 snap 33 seq_num 133) 2024-04-07T16:17:59.843 INFO:tasks.rados.rados.0.smithi100.stdout:409: done (3 left) 2024-04-07T16:17:59.843 INFO:tasks.rados.rados.0.smithi100.stdout:411: done (2 left) 2024-04-07T16:17:59.843 INFO:tasks.rados.rados.0.smithi100.stdout:412: done (1 left) 2024-04-07T16:17:59.843 INFO:tasks.rados.rados.0.smithi100.stdout:413: done (0 left) 2024-04-07T16:17:59.843 INFO:tasks.rados.rados.0.smithi100.stdout:414: delete oid 4 current snap is 38 2024-04-07T16:17:59.854 INFO:tasks.rados.rados.0.smithi100.stdout:414: done (0 left) 2024-04-07T16:17:59.855 INFO:tasks.rados.rados.0.smithi100.stdout:415: snap_create 2024-04-07T16:18:00.105 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:17:59 smithi144 bash[23522]: cluster 2024-04-07T16:17:58.714926+0000 mgr.y (mgr.14553) 221 : cluster 0 pgmap v234: 121 pgs: 121 active+clean; 81 MiB data, 379 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:18:00.105 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:17:59 smithi144 bash[23522]: cluster 2024-04-07T16:17:58.854651+0000 mon.a (mon.0) 966 : cluster 0 osdmap e144: 8 total, 8 up, 8 in 2024-04-07T16:18:00.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:17:59 smithi100 bash[21346]: cluster 2024-04-07T16:17:58.714926+0000 mgr.y (mgr.14553) 221 : cluster 0 pgmap v234: 121 pgs: 121 active+clean; 81 MiB data, 379 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:18:00.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:17:59 smithi100 bash[21346]: cluster 2024-04-07T16:17:58.854651+0000 mon.a (mon.0) 966 : cluster 0 osdmap e144: 8 total, 8 up, 8 in 2024-04-07T16:18:00.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:17:59 smithi100 bash[31660]: cluster 2024-04-07T16:17:58.714926+0000 mgr.y (mgr.14553) 221 : cluster 0 pgmap v234: 121 pgs: 121 active+clean; 81 MiB data, 379 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:18:00.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:17:59 smithi100 bash[31660]: cluster 2024-04-07T16:17:58.854651+0000 mon.a (mon.0) 966 : cluster 0 osdmap e144: 8 total, 8 up, 8 in 2024-04-07T16:18:00.864 INFO:tasks.rados.rados.0.smithi100.stdout:415: done (0 left) 2024-04-07T16:18:00.864 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 28 current snap is 39 2024-04-07T16:18:00.865 INFO:tasks.rados.rados.0.smithi100.stdout:416: seq_num 140 ranges {0=753664,753664=704512,1458176=524288,1982464=376832} 2024-04-07T16:18:00.877 INFO:tasks.rados.rados.0.smithi100.stdout:416: writing smithi10066867-28 from 0 to 753664 tid 1 2024-04-07T16:18:00.889 INFO:tasks.rados.rados.0.smithi100.stdout:416: writing smithi10066867-28 from 753664 to 1458176 tid 2 2024-04-07T16:18:00.898 INFO:tasks.rados.rados.0.smithi100.stdout:416: writing smithi10066867-28 from 1458176 to 1982464 tid 3 2024-04-07T16:18:00.904 INFO:tasks.rados.rados.0.smithi100.stdout:416: writing smithi10066867-28 from 1982464 to 2359296 tid 4 2024-04-07T16:18:00.904 INFO:tasks.rados.rados.0.smithi100.stdout:417: rollback oid 34 current snap is 39 2024-04-07T16:18:00.905 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 34 to 33 2024-04-07T16:18:00.905 INFO:tasks.rados.rados.0.smithi100.stdout:416: finishing write tid 1 to smithi10066867-28 2024-04-07T16:18:00.905 INFO:tasks.rados.rados.0.smithi100.stdout:416: finishing write tid 2 to smithi10066867-28 2024-04-07T16:18:00.905 INFO:tasks.rados.rados.0.smithi100.stdout:416: finishing write tid 3 to smithi10066867-28 2024-04-07T16:18:00.905 INFO:tasks.rados.rados.0.smithi100.stdout:418: delete oid 8 current snap is 39 2024-04-07T16:18:00.912 INFO:tasks.rados.rados.0.smithi100.stdout:416: finishing write tid 4 to smithi10066867-28 2024-04-07T16:18:00.912 INFO:tasks.rados.rados.0.smithi100.stdout:416: finishing write tid 5 to smithi10066867-28 2024-04-07T16:18:00.914 INFO:tasks.rados.rados.0.smithi100.stdout:417: finishing rollback tid 0 to smithi10066867-34 2024-04-07T16:18:00.914 INFO:tasks.rados.rados.0.smithi100.stdout:417: finishing rollback tid 1 to smithi10066867-34 2024-04-07T16:18:00.915 INFO:tasks.rados.rados.0.smithi100.stdout:417: finishing rollback tid 2 to smithi10066867-34 2024-04-07T16:18:00.915 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 34 v 103 (ObjNum 45 snap 0 seq_num 45) dirty exists 2024-04-07T16:18:00.915 INFO:tasks.rados.rados.0.smithi100.stdout:417: done (2 left) 2024-04-07T16:18:00.915 INFO:tasks.rados.rados.0.smithi100.stdout:418: done (1 left) 2024-04-07T16:18:00.916 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 41 current snap is 39 2024-04-07T16:18:00.916 INFO:tasks.rados.rados.0.smithi100.stdout:419: seq_num 142 ranges {0=786432,786432=475136,1261568=540672,1802240=589824} 2024-04-07T16:18:00.921 INFO:tasks.rados.rados.0.smithi100.stdout:419: writing smithi10066867-41 from 0 to 786432 tid 1 2024-04-07T16:18:00.924 INFO:tasks.rados.rados.0.smithi100.stdout:419: writing smithi10066867-41 from 786432 to 1261568 tid 2 2024-04-07T16:18:00.928 INFO:tasks.rados.rados.0.smithi100.stdout:419: writing smithi10066867-41 from 1261568 to 1802240 tid 3 2024-04-07T16:18:00.933 INFO:tasks.rados.rados.0.smithi100.stdout:419: writing smithi10066867-41 from 1802240 to 2392064 tid 4 2024-04-07T16:18:00.934 INFO:tasks.rados.rados.0.smithi100.stdout:420: snap_create 2024-04-07T16:18:00.934 INFO:tasks.rados.rados.0.smithi100.stdout:416: finishing write tid 6 to smithi10066867-28 2024-04-07T16:18:00.934 INFO:tasks.rados.rados.0.smithi100.stdout:416: oid 28 updating version 0 to 34 2024-04-07T16:18:00.934 INFO:tasks.rados.rados.0.smithi100.stdout:416: oid 28 version 34 is already newer than 31 2024-04-07T16:18:00.934 INFO:tasks.rados.rados.0.smithi100.stdout:416: oid 28 updating version 34 to 35 2024-04-07T16:18:00.934 INFO:tasks.rados.rados.0.smithi100.stdout:416: oid 28 version 35 is already newer than 33 2024-04-07T16:18:00.934 INFO:tasks.rados.rados.0.smithi100.stdout:416: oid 28 version 35 is already newer than 32 2024-04-07T16:18:00.934 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 28 v 35 (ObjNum 140 snap 39 seq_num 140) dirty exists 2024-04-07T16:18:00.934 INFO:tasks.rados.rados.0.smithi100.stdout:416: left oid 28 (ObjNum 140 snap 39 seq_num 140) 2024-04-07T16:18:00.934 INFO:tasks.rados.rados.0.smithi100.stdout:419: finishing write tid 1 to smithi10066867-41 2024-04-07T16:18:00.934 INFO:tasks.rados.rados.0.smithi100.stdout:419: finishing write tid 2 to smithi10066867-41 2024-04-07T16:18:00.934 INFO:tasks.rados.rados.0.smithi100.stdout:419: finishing write tid 3 to smithi10066867-41 2024-04-07T16:18:00.936 INFO:tasks.rados.rados.0.smithi100.stdout:419: finishing write tid 4 to smithi10066867-41 2024-04-07T16:18:00.936 INFO:tasks.rados.rados.0.smithi100.stdout:419: finishing write tid 5 to smithi10066867-41 2024-04-07T16:18:00.938 INFO:tasks.rados.rados.0.smithi100.stdout:419: finishing write tid 6 to smithi10066867-41 2024-04-07T16:18:00.938 INFO:tasks.rados.rados.0.smithi100.stdout:419: oid 41 updating version 0 to 154 2024-04-07T16:18:00.938 INFO:tasks.rados.rados.0.smithi100.stdout:419: oid 41 version 154 is already newer than 152 2024-04-07T16:18:00.938 INFO:tasks.rados.rados.0.smithi100.stdout:419: oid 41 version 154 is already newer than 150 2024-04-07T16:18:00.939 INFO:tasks.rados.rados.0.smithi100.stdout:419: oid 41 version 154 is already newer than 153 2024-04-07T16:18:00.939 INFO:tasks.rados.rados.0.smithi100.stdout:419: oid 41 version 154 is already newer than 151 2024-04-07T16:18:00.939 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 41 v 154 (ObjNum 142 snap 39 seq_num 142) dirty exists 2024-04-07T16:18:00.939 INFO:tasks.rados.rados.0.smithi100.stdout:419: left oid 41 (ObjNum 142 snap 39 seq_num 142) 2024-04-07T16:18:01.105 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:00 smithi144 bash[23522]: cluster 2024-04-07T16:17:59.854783+0000 mon.a (mon.0) 967 : cluster 0 osdmap e145: 8 total, 8 up, 8 in 2024-04-07T16:18:01.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:00 smithi100 bash[21346]: cluster 2024-04-07T16:17:59.854783+0000 mon.a (mon.0) 967 : cluster 0 osdmap e145: 8 total, 8 up, 8 in 2024-04-07T16:18:01.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:00 smithi100 bash[31660]: cluster 2024-04-07T16:17:59.854783+0000 mon.a (mon.0) 967 : cluster 0 osdmap e145: 8 total, 8 up, 8 in 2024-04-07T16:18:01.874 INFO:tasks.rados.rados.0.smithi100.stdout:416: done (2 left) 2024-04-07T16:18:01.874 INFO:tasks.rados.rados.0.smithi100.stdout:419: done (1 left) 2024-04-07T16:18:01.874 INFO:tasks.rados.rados.0.smithi100.stdout:420: done (0 left) 2024-04-07T16:18:01.874 INFO:tasks.rados.rados.0.smithi100.stdout:421: read oid 45 snap -1 2024-04-07T16:18:01.874 INFO:tasks.rados.rados.0.smithi100.stdout:421: expect (ObjNum 111 snap 21 seq_num 111) 2024-04-07T16:18:01.874 INFO:tasks.rados.rados.0.smithi100.stdout:422: rmattr oid 18 current snap is 40 2024-04-07T16:18:01.875 INFO:tasks.rados.rados.0.smithi100.stdout:422: done (1 left) 2024-04-07T16:18:01.875 INFO:tasks.rados.rados.0.smithi100.stdout:423: read oid 28 snap -1 2024-04-07T16:18:01.875 INFO:tasks.rados.rados.0.smithi100.stdout:423: expect (ObjNum 140 snap 39 seq_num 140) 2024-04-07T16:18:01.875 INFO:tasks.rados.rados.0.smithi100.stdout:424: copy_from oid 19 from oid 20 current snap is 40 2024-04-07T16:18:01.875 INFO:tasks.rados.rados.0.smithi100.stdout:425: rmattr oid 26 current snap is 40 2024-04-07T16:18:01.875 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 23 current snap is 40 2024-04-07T16:18:01.875 INFO:tasks.rados.rados.0.smithi100.stdout:426: seq_num 143 ranges {5259264=802816,6062080=761856,6823936=745472,7569408=98304} 2024-04-07T16:18:01.940 INFO:tasks.rados.rados.0.smithi100.stdout:426: writing smithi10066867-23 from 5259264 to 6062080 tid 1 2024-04-07T16:18:01.946 INFO:tasks.rados.rados.0.smithi100.stdout:426: writing smithi10066867-23 from 6062080 to 6823936 tid 2 2024-04-07T16:18:01.953 INFO:tasks.rados.rados.0.smithi100.stdout:426: writing smithi10066867-23 from 6823936 to 7569408 tid 3 2024-04-07T16:18:01.954 INFO:tasks.rados.rados.0.smithi100.stdout:426: writing smithi10066867-23 from 7569408 to 7667712 tid 4 2024-04-07T16:18:01.954 INFO:tasks.rados.rados.0.smithi100.stdout:427: delete oid 42 current snap is 40 2024-04-07T16:18:01.954 INFO:tasks.rados.rados.0.smithi100.stdout:424: finishing copy_from racing read to smithi10066867-19 2024-04-07T16:18:01.954 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 26 v 97 (ObjNum 104 snap 20 seq_num 104) dirty exists 2024-04-07T16:18:01.954 INFO:tasks.rados.rados.0.smithi100.stdout:424: finishing copy_from to smithi10066867-19 2024-04-07T16:18:01.954 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 19 v 95 (ObjNum 161 snap 0 seq_num 1) dirty exists 2024-04-07T16:18:01.954 INFO:tasks.rados.rados.0.smithi100.stdout:421: expect (ObjNum 111 snap 21 seq_num 111) 2024-04-07T16:18:02.087 INFO:tasks.rados.rados.0.smithi100.stdout:423: expect (ObjNum 140 snap 39 seq_num 140) 2024-04-07T16:18:02.134 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:01 smithi144 bash[23522]: cluster 2024-04-07T16:18:00.715639+0000 mgr.y (mgr.14553) 222 : cluster 0 pgmap v237: 121 pgs: 121 active+clean; 84 MiB data, 378 MiB used, 715 GiB / 715 GiB avail; 2.8 MiB/s rd, 1.6 MiB/s wr, 28 op/s 2024-04-07T16:18:02.134 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:01 smithi144 bash[23522]: cluster 2024-04-07T16:18:00.875578+0000 mon.a (mon.0) 968 : cluster 0 osdmap e146: 8 total, 8 up, 8 in 2024-04-07T16:18:02.134 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:01 smithi144 bash[23522]: cephadm 2024-04-07T16:18:01.075918+0000 mgr.y (mgr.14553) 223 : cephadm 1 Upgrade: Updating mgr.x 2024-04-07T16:18:02.134 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:01 smithi144 bash[23522]: audit 2024-04-07T16:18:01.086060+0000 mon.a (mon.0) 969 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:18:02.134 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:01 smithi144 bash[23522]: audit 2024-04-07T16:18:01.086728+0000 mon.b (mon.1) 245 : audit 1 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-07T16:18:02.134 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:01 smithi144 bash[23522]: audit 2024-04-07T16:18:01.090187+0000 mon.b (mon.1) 246 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-07T16:18:02.134 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:01 smithi144 bash[23522]: audit 2024-04-07T16:18:01.091695+0000 mon.a (mon.0) 970 : audit 1 from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-07T16:18:02.134 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:01 smithi144 bash[23522]: audit 2024-04-07T16:18:01.092477+0000 mon.b (mon.1) 247 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:18:02.134 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:01 smithi144 bash[23522]: cephadm 2024-04-07T16:18:01.098883+0000 mgr.y (mgr.14553) 224 : cephadm 1 Deploying daemon mgr.x on smithi144 2024-04-07T16:18:02.146 INFO:tasks.rados.rados.0.smithi100.stdout:426: finishing write tid 1 to smithi10066867-23 2024-04-07T16:18:02.147 INFO:tasks.rados.rados.0.smithi100.stdout:426: finishing write tid 2 to smithi10066867-23 2024-04-07T16:18:02.147 INFO:tasks.rados.rados.0.smithi100.stdout:426: finishing write tid 3 to smithi10066867-23 2024-04-07T16:18:02.147 INFO:tasks.rados.rados.0.smithi100.stdout:426: finishing write tid 4 to smithi10066867-23 2024-04-07T16:18:02.147 INFO:tasks.rados.rados.0.smithi100.stdout:426: finishing write tid 5 to smithi10066867-23 2024-04-07T16:18:02.147 INFO:tasks.rados.rados.0.smithi100.stdout:426: finishing write tid 6 to smithi10066867-23 2024-04-07T16:18:02.147 INFO:tasks.rados.rados.0.smithi100.stdout:426: oid 23 updating version 0 to 181 2024-04-07T16:18:02.147 INFO:tasks.rados.rados.0.smithi100.stdout:426: oid 23 version 181 is already newer than 179 2024-04-07T16:18:02.147 INFO:tasks.rados.rados.0.smithi100.stdout:426: oid 23 version 181 is already newer than 180 2024-04-07T16:18:02.147 INFO:tasks.rados.rados.0.smithi100.stdout:426: oid 23 updating version 181 to 182 2024-04-07T16:18:02.147 INFO:tasks.rados.rados.0.smithi100.stdout:426: oid 23 updating version 182 to 183 2024-04-07T16:18:02.147 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 23 v 183 (ObjNum 143 snap 40 seq_num 143) dirty exists 2024-04-07T16:18:02.147 INFO:tasks.rados.rados.0.smithi100.stdout:426: left oid 23 (ObjNum 143 snap 40 seq_num 143) 2024-04-07T16:18:02.148 INFO:tasks.rados.rados.0.smithi100.stdout:421: done (5 left) 2024-04-07T16:18:02.148 INFO:tasks.rados.rados.0.smithi100.stdout:423: done (4 left) 2024-04-07T16:18:02.148 INFO:tasks.rados.rados.0.smithi100.stdout:424: done (3 left) 2024-04-07T16:18:02.148 INFO:tasks.rados.rados.0.smithi100.stdout:425: done (2 left) 2024-04-07T16:18:02.148 INFO:tasks.rados.rados.0.smithi100.stdout:426: done (1 left) 2024-04-07T16:18:02.148 INFO:tasks.rados.rados.0.smithi100.stdout:427: done (0 left) 2024-04-07T16:18:02.148 INFO:tasks.rados.rados.0.smithi100.stdout:428: delete oid 47 current snap is 40 2024-04-07T16:18:02.151 INFO:tasks.rados.rados.0.smithi100.stdout:428: done (0 left) 2024-04-07T16:18:02.151 INFO:tasks.rados.rados.0.smithi100.stdout:429: read oid 33 snap 39 2024-04-07T16:18:02.151 INFO:tasks.rados.rados.0.smithi100.stdout:429: expect (ObjNum 129 snap 32 seq_num 129) 2024-04-07T16:18:02.151 INFO:tasks.rados.rados.0.smithi100.stdout:430: setattr oid 15 current snap is 40 2024-04-07T16:18:02.153 INFO:tasks.rados.rados.0.smithi100.stdout:431: setattr oid 29 current snap is 40 2024-04-07T16:18:02.155 INFO:tasks.rados.rados.0.smithi100.stdout:432: delete oid 34 current snap is 40 2024-04-07T16:18:02.162 INFO:tasks.rados.rados.0.smithi100.stdout:432: done (3 left) 2024-04-07T16:18:02.162 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 4 current snap is 40 2024-04-07T16:18:02.162 INFO:tasks.rados.rados.0.smithi100.stdout:433: seq_num 147 ranges {0=65536} 2024-04-07T16:18:02.162 INFO:tasks.rados.rados.0.smithi100.stdout:433: writing smithi10066867-4 from 0 to 65536 tid 1 2024-04-07T16:18:02.162 INFO:tasks.rados.rados.0.smithi100.stdout:434: snap_remove snap 26 2024-04-07T16:18:02.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:01 smithi100 bash[31660]: cluster 2024-04-07T16:18:00.715639+0000 mgr.y (mgr.14553) 222 : cluster 0 pgmap v237: 121 pgs: 121 active+clean; 84 MiB data, 378 MiB used, 715 GiB / 715 GiB avail; 2.8 MiB/s rd, 1.6 MiB/s wr, 28 op/s 2024-04-07T16:18:02.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:01 smithi100 bash[31660]: cluster 2024-04-07T16:18:00.875578+0000 mon.a (mon.0) 968 : cluster 0 osdmap e146: 8 total, 8 up, 8 in 2024-04-07T16:18:02.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:01 smithi100 bash[31660]: cephadm 2024-04-07T16:18:01.075918+0000 mgr.y (mgr.14553) 223 : cephadm 1 Upgrade: Updating mgr.x 2024-04-07T16:18:02.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:01 smithi100 bash[31660]: audit 2024-04-07T16:18:01.086060+0000 mon.a (mon.0) 969 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:18:02.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:01 smithi100 bash[31660]: audit 2024-04-07T16:18:01.086728+0000 mon.b (mon.1) 245 : audit 1 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-07T16:18:02.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:01 smithi100 bash[31660]: audit 2024-04-07T16:18:01.090187+0000 mon.b (mon.1) 246 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-07T16:18:02.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:01 smithi100 bash[31660]: audit 2024-04-07T16:18:01.091695+0000 mon.a (mon.0) 970 : audit 1 from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-07T16:18:02.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:01 smithi100 bash[31660]: audit 2024-04-07T16:18:01.092477+0000 mon.b (mon.1) 247 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:18:02.188 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:01 smithi100 bash[31660]: cephadm 2024-04-07T16:18:01.098883+0000 mgr.y (mgr.14553) 224 : cephadm 1 Deploying daemon mgr.x on smithi144 2024-04-07T16:18:02.188 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:01 smithi100 bash[21346]: cluster 2024-04-07T16:18:00.715639+0000 mgr.y (mgr.14553) 222 : cluster 0 pgmap v237: 121 pgs: 121 active+clean; 84 MiB data, 378 MiB used, 715 GiB / 715 GiB avail; 2.8 MiB/s rd, 1.6 MiB/s wr, 28 op/s 2024-04-07T16:18:02.188 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:01 smithi100 bash[21346]: cluster 2024-04-07T16:18:00.875578+0000 mon.a (mon.0) 968 : cluster 0 osdmap e146: 8 total, 8 up, 8 in 2024-04-07T16:18:02.188 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:01 smithi100 bash[21346]: cephadm 2024-04-07T16:18:01.075918+0000 mgr.y (mgr.14553) 223 : cephadm 1 Upgrade: Updating mgr.x 2024-04-07T16:18:02.188 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:01 smithi100 bash[21346]: audit 2024-04-07T16:18:01.086060+0000 mon.a (mon.0) 969 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:18:02.188 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:01 smithi100 bash[21346]: audit 2024-04-07T16:18:01.086728+0000 mon.b (mon.1) 245 : audit 1 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-07T16:18:02.188 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:01 smithi100 bash[21346]: audit 2024-04-07T16:18:01.090187+0000 mon.b (mon.1) 246 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-07T16:18:02.188 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:01 smithi100 bash[21346]: audit 2024-04-07T16:18:01.091695+0000 mon.a (mon.0) 970 : audit 1 from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-07T16:18:02.188 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:01 smithi100 bash[21346]: audit 2024-04-07T16:18:01.092477+0000 mon.b (mon.1) 247 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:18:02.188 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:01 smithi100 bash[21346]: cephadm 2024-04-07T16:18:01.098883+0000 mgr.y (mgr.14553) 224 : cephadm 1 Deploying daemon mgr.x on smithi144 2024-04-07T16:18:02.845 INFO:tasks.rados.rados.0.smithi100.stdout:434: done (4 left) 2024-04-07T16:18:02.845 INFO:tasks.rados.rados.0.smithi100.stdout:435: copy_from oid 40 from oid 6 current snap is 40 2024-04-07T16:18:02.845 INFO:tasks.rados.rados.0.smithi100.stdout:436: read oid 17 snap -1 2024-04-07T16:18:02.845 INFO:tasks.rados.rados.0.smithi100.stdout:436: expect (ObjNum 98 snap 18 seq_num 98) 2024-04-07T16:18:02.845 INFO:tasks.rados.rados.0.smithi100.stdout:437: rollback oid 42 current snap is 40 2024-04-07T16:18:02.845 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 42 to 33 2024-04-07T16:18:02.845 INFO:tasks.rados.rados.0.smithi100.stdout:438: snap_create 2024-04-07T16:18:02.846 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 29 v 114 (ObjNum 61 snap 5 seq_num 61) dirty exists 2024-04-07T16:18:02.846 INFO:tasks.rados.rados.0.smithi100.stdout:429: expect (ObjNum 129 snap 32 seq_num 129) 2024-04-07T16:18:03.000 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 15 v 185 (ObjNum 4 snap 0 seq_num 4) dirty exists 2024-04-07T16:18:03.000 INFO:tasks.rados.rados.0.smithi100.stdout:433: finishing write tid 1 to smithi10066867-4 2024-04-07T16:18:03.000 INFO:tasks.rados.rados.0.smithi100.stdout:433: finishing write tid 2 to smithi10066867-4 2024-04-07T16:18:03.000 INFO:tasks.rados.rados.0.smithi100.stdout:433: finishing write tid 3 to smithi10066867-4 2024-04-07T16:18:03.001 INFO:tasks.rados.rados.0.smithi100.stdout:433: oid 4 updating version 0 to 187 2024-04-07T16:18:03.001 INFO:tasks.rados.rados.0.smithi100.stdout:433: oid 4 version 187 is already newer than 186 2024-04-07T16:18:03.001 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 4 v 187 (ObjNum 147 snap 40 seq_num 147) dirty exists 2024-04-07T16:18:03.001 INFO:tasks.rados.rados.0.smithi100.stdout:433: left oid 4 (ObjNum 147 snap 40 seq_num 147) 2024-04-07T16:18:03.001 INFO:tasks.rados.rados.0.smithi100.stdout:437: finishing rollback tid 1 to smithi10066867-42 2024-04-07T16:18:03.001 INFO:tasks.rados.rados.0.smithi100.stdout:437: finishing rollback tid 2 to smithi10066867-42 2024-04-07T16:18:03.001 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 42 v 120 (ObjNum 41 snap 0 seq_num 41) dirty exists 2024-04-07T16:18:03.001 INFO:tasks.rados.rados.0.smithi100.stdout:436: expect (ObjNum 98 snap 18 seq_num 98) 2024-04-07T16:18:03.184 INFO:tasks.rados.rados.0.smithi100.stdout:435: finishing copy_from racing read to smithi10066867-40 2024-04-07T16:18:03.184 INFO:tasks.rados.rados.0.smithi100.stdout:435: finishing copy_from to smithi10066867-40 2024-04-07T16:18:03.184 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 40 v 124 (ObjNum 124 snap 29 seq_num 124) dirty exists 2024-04-07T16:18:03.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:02 smithi100 bash[31660]: cluster 2024-04-07T16:18:01.887821+0000 mon.a (mon.0) 971 : cluster 0 osdmap e147: 8 total, 8 up, 8 in 2024-04-07T16:18:03.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:02 smithi100 bash[31660]: cluster 2024-04-07T16:18:02.854734+0000 mon.a (mon.0) 972 : cluster 0 osdmap e148: 8 total, 8 up, 8 in 2024-04-07T16:18:03.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:02 smithi100 bash[21346]: cluster 2024-04-07T16:18:01.887821+0000 mon.a (mon.0) 971 : cluster 0 osdmap e147: 8 total, 8 up, 8 in 2024-04-07T16:18:03.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:02 smithi100 bash[21346]: cluster 2024-04-07T16:18:02.854734+0000 mon.a (mon.0) 972 : cluster 0 osdmap e148: 8 total, 8 up, 8 in 2024-04-07T16:18:03.226 INFO:journalctl@ceph.osd.7.smithi144.stdout:Apr 07 16:18:02 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:18:03.226 INFO:journalctl@ceph.mds.b.smithi144.stdout:Apr 07 16:18:03 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:18:03.226 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:18:02 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:18:03.227 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:18:03 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:18:03.227 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:02 smithi144 bash[23522]: cluster 2024-04-07T16:18:01.887821+0000 mon.a (mon.0) 971 : cluster 0 osdmap e147: 8 total, 8 up, 8 in 2024-04-07T16:18:03.227 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:02 smithi144 bash[23522]: cluster 2024-04-07T16:18:02.854734+0000 mon.a (mon.0) 972 : cluster 0 osdmap e148: 8 total, 8 up, 8 in 2024-04-07T16:18:03.227 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:03 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:18:03.228 INFO:journalctl@ceph.osd.5.smithi144.stdout:Apr 07 16:18:02 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:18:03.228 INFO:journalctl@ceph.osd.6.smithi144.stdout:Apr 07 16:18:02 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:18:03.228 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 16:18:03 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:18:03.228 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:18:03 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:18:03.229 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:18:03 smithi144 systemd[1]: Stopping Ceph mgr.x for 0702e3a8-f4f8-11ee-b649-cb9ed24678a4... 2024-04-07T16:18:03.229 INFO:journalctl@ceph.osd.4.smithi144.stdout:Apr 07 16:18:03 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:18:03.845 INFO:tasks.rados.rados.0.smithi100.stdout:429: done (7 left) 2024-04-07T16:18:03.846 INFO:tasks.rados.rados.0.smithi100.stdout:430: done (6 left) 2024-04-07T16:18:03.846 INFO:tasks.rados.rados.0.smithi100.stdout:431: done (5 left) 2024-04-07T16:18:03.846 INFO:tasks.rados.rados.0.smithi100.stdout:433: done (4 left) 2024-04-07T16:18:03.846 INFO:tasks.rados.rados.0.smithi100.stdout:435: done (3 left) 2024-04-07T16:18:03.846 INFO:tasks.rados.rados.0.smithi100.stdout:436: done (2 left) 2024-04-07T16:18:03.846 INFO:tasks.rados.rados.0.smithi100.stdout:437: done (1 left) 2024-04-07T16:18:03.846 INFO:tasks.rados.rados.0.smithi100.stdout:438: done (0 left) 2024-04-07T16:18:03.846 INFO:tasks.rados.rados.0.smithi100.stdout:439: read oid 14 snap -1 2024-04-07T16:18:03.846 INFO:tasks.rados.rados.0.smithi100.stdout:439: expect (ObjNum 28 snap 0 seq_num 28) 2024-04-07T16:18:03.846 INFO:tasks.rados.rados.0.smithi100.stdout:440: delete oid 12 current snap is 41 2024-04-07T16:18:03.847 INFO:tasks.rados.rados.0.smithi100.stdout:440: done (1 left) 2024-04-07T16:18:03.847 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 47 current snap is 41 2024-04-07T16:18:03.847 INFO:tasks.rados.rados.0.smithi100.stdout:441: seq_num 149 ranges {0=98304} 2024-04-07T16:18:03.850 INFO:tasks.rados.rados.0.smithi100.stdout:441: writing smithi10066867-47 from 0 to 98304 tid 1 2024-04-07T16:18:03.850 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 44 current snap is 41 2024-04-07T16:18:03.850 INFO:tasks.rados.rados.0.smithi100.stdout:442: seq_num 150 ranges {475136=114688} 2024-04-07T16:18:03.860 INFO:tasks.rados.rados.0.smithi100.stdout:442: writing smithi10066867-44 from 475136 to 589824 tid 1 2024-04-07T16:18:03.860 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 9 current snap is 41 2024-04-07T16:18:03.860 INFO:tasks.rados.rados.0.smithi100.stdout:443: seq_num 151 ranges {1417216=131072} 2024-04-07T16:18:03.868 INFO:tasks.rados.rados.0.smithi100.stdout:443: writing smithi10066867-9 from 1417216 to 1548288 tid 1 2024-04-07T16:18:03.869 INFO:tasks.rados.rados.0.smithi100.stdout:444: snap_remove snap 35 2024-04-07T16:18:04.037 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:03 smithi144 bash[23522]: cluster 2024-04-07T16:18:02.716285+0000 mgr.y (mgr.14553) 225 : cluster 0 pgmap v240: 121 pgs: 121 active+clean; 84 MiB data, 378 MiB used, 715 GiB / 715 GiB avail; 2.8 MiB/s rd, 1.6 MiB/s wr, 28 op/s 2024-04-07T16:18:04.037 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:03 smithi144 bash[23522]: cluster 2024-04-07T16:18:03.854304+0000 mon.a (mon.0) 973 : cluster 0 osdmap e149: 8 total, 8 up, 8 in 2024-04-07T16:18:04.038 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:18:03 smithi144 bash[51835]: ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4-mgr-x 2024-04-07T16:18:04.038 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:18:03 smithi144 systemd[1]: ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@mgr.x.service: Main process exited, code=exited, status=143/n/a 2024-04-07T16:18:04.038 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:18:03 smithi144 bash[51894]: Error response from daemon: No such container: ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4-mgr-x 2024-04-07T16:18:04.038 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:18:03 smithi144 systemd[1]: ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@mgr.x.service: Failed with result 'exit-code'. 2024-04-07T16:18:04.038 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:18:03 smithi144 systemd[1]: Stopped Ceph mgr.x for 0702e3a8-f4f8-11ee-b649-cb9ed24678a4. 2024-04-07T16:18:04.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:03 smithi100 bash[21346]: cluster 2024-04-07T16:18:02.716285+0000 mgr.y (mgr.14553) 225 : cluster 0 pgmap v240: 121 pgs: 121 active+clean; 84 MiB data, 378 MiB used, 715 GiB / 715 GiB avail; 2.8 MiB/s rd, 1.6 MiB/s wr, 28 op/s 2024-04-07T16:18:04.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:03 smithi100 bash[21346]: cluster 2024-04-07T16:18:03.854304+0000 mon.a (mon.0) 973 : cluster 0 osdmap e149: 8 total, 8 up, 8 in 2024-04-07T16:18:04.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:03 smithi100 bash[31660]: cluster 2024-04-07T16:18:02.716285+0000 mgr.y (mgr.14553) 225 : cluster 0 pgmap v240: 121 pgs: 121 active+clean; 84 MiB data, 378 MiB used, 715 GiB / 715 GiB avail; 2.8 MiB/s rd, 1.6 MiB/s wr, 28 op/s 2024-04-07T16:18:04.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:03 smithi100 bash[31660]: cluster 2024-04-07T16:18:03.854304+0000 mon.a (mon.0) 973 : cluster 0 osdmap e149: 8 total, 8 up, 8 in 2024-04-07T16:18:04.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:04 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:18:04.355 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:18:04 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:18:04.355 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:18:04 smithi144 systemd[1]: Started Ceph mgr.x for 0702e3a8-f4f8-11ee-b649-cb9ed24678a4. 2024-04-07T16:18:04.356 INFO:journalctl@ceph.osd.5.smithi144.stdout:Apr 07 16:18:04 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:18:04.356 INFO:journalctl@ceph.osd.6.smithi144.stdout:Apr 07 16:18:04 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:18:04.357 INFO:journalctl@ceph.osd.7.smithi144.stdout:Apr 07 16:18:04 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:18:04.357 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:18:04 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:18:04.357 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:18:04 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:18:04.358 INFO:journalctl@ceph.osd.4.smithi144.stdout:Apr 07 16:18:04 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:18:04.358 INFO:journalctl@ceph.mds.b.smithi144.stdout:Apr 07 16:18:04 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:18:04.358 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 16:18:04 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:24: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:18:04.844 INFO:tasks.rados.rados.0.smithi100.stdout:444: done (4 left) 2024-04-07T16:18:04.845 INFO:tasks.rados.rados.0.smithi100.stdout:445: rollback oid 29 current snap is 41 2024-04-07T16:18:04.845 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 29 to 39 2024-04-07T16:18:04.845 INFO:tasks.rados.rados.0.smithi100.stdout:441: finishing write tid 1 to smithi10066867-47 2024-04-07T16:18:04.845 INFO:tasks.rados.rados.0.smithi100.stdout:441: finishing write tid 2 to smithi10066867-47 2024-04-07T16:18:04.845 INFO:tasks.rados.rados.0.smithi100.stdout:439: expect (ObjNum 28 snap 0 seq_num 28) 2024-04-07T16:18:04.876 INFO:tasks.rados.rados.0.smithi100.stdout:441: finishing write tid 3 to smithi10066867-47 2024-04-07T16:18:04.876 INFO:tasks.rados.rados.0.smithi100.stdout:441: oid 47 updating version 0 to 53 2024-04-07T16:18:04.876 INFO:tasks.rados.rados.0.smithi100.stdout:441: oid 47 updating version 53 to 54 2024-04-07T16:18:04.876 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 47 v 54 (ObjNum 149 snap 41 seq_num 149) dirty exists 2024-04-07T16:18:04.876 INFO:tasks.rados.rados.0.smithi100.stdout:441: left oid 47 (ObjNum 149 snap 41 seq_num 149) 2024-04-07T16:18:04.876 INFO:tasks.rados.rados.0.smithi100.stdout:442: finishing write tid 1 to smithi10066867-44 2024-04-07T16:18:04.876 INFO:tasks.rados.rados.0.smithi100.stdout:442: finishing write tid 2 to smithi10066867-44 2024-04-07T16:18:04.876 INFO:tasks.rados.rados.0.smithi100.stdout:442: finishing write tid 3 to smithi10066867-44 2024-04-07T16:18:04.876 INFO:tasks.rados.rados.0.smithi100.stdout:442: oid 44 updating version 0 to 132 2024-04-07T16:18:04.876 INFO:tasks.rados.rados.0.smithi100.stdout:442: oid 44 version 132 is already newer than 131 2024-04-07T16:18:04.876 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 44 v 132 (ObjNum 150 snap 41 seq_num 150) dirty exists 2024-04-07T16:18:04.876 INFO:tasks.rados.rados.0.smithi100.stdout:442: left oid 44 (ObjNum 150 snap 41 seq_num 150) 2024-04-07T16:18:04.877 INFO:tasks.rados.rados.0.smithi100.stdout:443: finishing write tid 1 to smithi10066867-9 2024-04-07T16:18:04.877 INFO:tasks.rados.rados.0.smithi100.stdout:443: finishing write tid 2 to smithi10066867-9 2024-04-07T16:18:04.877 INFO:tasks.rados.rados.0.smithi100.stdout:443: finishing write tid 3 to smithi10066867-9 2024-04-07T16:18:04.877 INFO:tasks.rados.rados.0.smithi100.stdout:443: oid 9 updating version 0 to 62 2024-04-07T16:18:04.877 INFO:tasks.rados.rados.0.smithi100.stdout:443: oid 9 version 62 is already newer than 61 2024-04-07T16:18:04.877 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 9 v 62 (ObjNum 151 snap 41 seq_num 151) dirty exists 2024-04-07T16:18:04.877 INFO:tasks.rados.rados.0.smithi100.stdout:443: left oid 9 (ObjNum 151 snap 41 seq_num 151) 2024-04-07T16:18:04.877 INFO:tasks.rados.rados.0.smithi100.stdout:445: finishing rollback tid 0 to smithi10066867-29 2024-04-07T16:18:04.877 INFO:tasks.rados.rados.0.smithi100.stdout:445: finishing rollback tid 1 to smithi10066867-29 2024-04-07T16:18:04.877 INFO:tasks.rados.rados.0.smithi100.stdout:445: finishing rollback tid 2 to smithi10066867-29 2024-04-07T16:18:04.877 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 29 v 128 (ObjNum 61 snap 5 seq_num 61) dirty exists 2024-04-07T16:18:04.877 INFO:tasks.rados.rados.0.smithi100.stdout:439: done (4 left) 2024-04-07T16:18:04.877 INFO:tasks.rados.rados.0.smithi100.stdout:441: done (3 left) 2024-04-07T16:18:04.877 INFO:tasks.rados.rados.0.smithi100.stdout:442: done (2 left) 2024-04-07T16:18:04.878 INFO:tasks.rados.rados.0.smithi100.stdout:443: done (1 left) 2024-04-07T16:18:04.878 INFO:tasks.rados.rados.0.smithi100.stdout:445: done (0 left) 2024-04-07T16:18:04.878 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 18 current snap is 41 2024-04-07T16:18:04.878 INFO:tasks.rados.rados.0.smithi100.stdout:446: seq_num 152 ranges {0=147456} 2024-04-07T16:18:04.878 INFO:tasks.rados.rados.0.smithi100.stdout:446: writing smithi10066867-18 from 0 to 147456 tid 1 2024-04-07T16:18:04.878 INFO:tasks.rados.rados.0.smithi100.stdout:447: copy_from oid 47 from oid 8 current snap is 41 2024-04-07T16:18:04.878 INFO:tasks.rados.rados.0.smithi100.stdout:448: snap_create 2024-04-07T16:18:04.880 INFO:tasks.rados.rados.0.smithi100.stdout:447: finishing copy_from to smithi10066867-47 2024-04-07T16:18:04.880 INFO:tasks.rados.rados.0.smithi100.stdout:447: got expected ENOENT (src dne) 2024-04-07T16:18:04.881 INFO:tasks.rados.rados.0.smithi100.stdout:447: finishing copy_from racing read to smithi10066867-47 2024-04-07T16:18:04.892 INFO:tasks.rados.rados.0.smithi100.stdout:446: finishing write tid 1 to smithi10066867-18 2024-04-07T16:18:04.892 INFO:tasks.rados.rados.0.smithi100.stdout:446: finishing write tid 2 to smithi10066867-18 2024-04-07T16:18:04.894 INFO:tasks.rados.rados.0.smithi100.stdout:446: finishing write tid 3 to smithi10066867-18 2024-04-07T16:18:04.894 INFO:tasks.rados.rados.0.smithi100.stdout:446: oid 18 updating version 0 to 196 2024-04-07T16:18:04.894 INFO:tasks.rados.rados.0.smithi100.stdout:446: oid 18 updating version 196 to 197 2024-04-07T16:18:04.894 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 18 v 197 (ObjNum 152 snap 41 seq_num 152) dirty exists 2024-04-07T16:18:04.895 INFO:tasks.rados.rados.0.smithi100.stdout:446: left oid 18 (ObjNum 152 snap 41 seq_num 152) 2024-04-07T16:18:05.202 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:05 smithi144 bash[23522]: audit 2024-04-07T16:18:04.212590+0000 mon.a (mon.0) 974 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:18:05.203 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:05 smithi144 bash[23522]: audit 2024-04-07T16:18:04.226105+0000 mon.a (mon.0) 975 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:18:05.203 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:05 smithi144 bash[23522]: audit 2024-04-07T16:18:04.268749+0000 mon.b (mon.1) 248 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:18:05.203 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:05 smithi144 bash[23522]: cluster 2024-04-07T16:18:04.858373+0000 mon.a (mon.0) 976 : cluster 0 osdmap e150: 8 total, 8 up, 8 in 2024-04-07T16:18:05.203 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:18:05 smithi144 bash[51952]: debug 2024-04-07T16:18:05.070+0000 7f6047eba700 1 -- 172.21.15.144:0/531282259 <== mon.1 v2:172.21.15.144:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55be14fda5a0 con 0x55be15d5a800 2024-04-07T16:18:05.437 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:18:05 smithi100 bash[21601]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:18:05] "GET /metrics HTTP/1.1" 200 36080 "" "Prometheus/2.43.0" 2024-04-07T16:18:05.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:05 smithi100 bash[21346]: audit 2024-04-07T16:18:04.212590+0000 mon.a (mon.0) 974 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:18:05.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:05 smithi100 bash[21346]: audit 2024-04-07T16:18:04.226105+0000 mon.a (mon.0) 975 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:18:05.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:05 smithi100 bash[21346]: audit 2024-04-07T16:18:04.268749+0000 mon.b (mon.1) 248 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:18:05.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:05 smithi100 bash[21346]: cluster 2024-04-07T16:18:04.858373+0000 mon.a (mon.0) 976 : cluster 0 osdmap e150: 8 total, 8 up, 8 in 2024-04-07T16:18:05.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:05 smithi100 bash[31660]: audit 2024-04-07T16:18:04.212590+0000 mon.a (mon.0) 974 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:18:05.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:05 smithi100 bash[31660]: audit 2024-04-07T16:18:04.226105+0000 mon.a (mon.0) 975 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:18:05.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:05 smithi100 bash[31660]: audit 2024-04-07T16:18:04.268749+0000 mon.b (mon.1) 248 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:18:05.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:05 smithi100 bash[31660]: cluster 2024-04-07T16:18:04.858373+0000 mon.a (mon.0) 976 : cluster 0 osdmap e150: 8 total, 8 up, 8 in 2024-04-07T16:18:05.605 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:18:05 smithi144 bash[51952]: debug 2024-04-07T16:18:05.210+0000 7f60520ca200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-07T16:18:05.846 INFO:tasks.rados.rados.0.smithi100.stdout:446: done (2 left) 2024-04-07T16:18:05.847 INFO:tasks.rados.rados.0.smithi100.stdout:447: done (1 left) 2024-04-07T16:18:05.847 INFO:tasks.rados.rados.0.smithi100.stdout:448: done (0 left) 2024-04-07T16:18:05.847 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 12 current snap is 42 2024-04-07T16:18:05.847 INFO:tasks.rados.rados.0.smithi100.stdout:449: seq_num 153 ranges {0=163840} 2024-04-07T16:18:05.849 INFO:tasks.rados.rados.0.smithi100.stdout:449: writing smithi10066867-12 from 0 to 163840 tid 1 2024-04-07T16:18:05.849 INFO:tasks.rados.rados.0.smithi100.stdout:450: read oid 9 snap -1 2024-04-07T16:18:05.849 INFO:tasks.rados.rados.0.smithi100.stdout:450: expect (ObjNum 151 snap 41 seq_num 151) 2024-04-07T16:18:05.850 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 26 current snap is 42 2024-04-07T16:18:05.850 INFO:tasks.rados.rados.0.smithi100.stdout:451: seq_num 154 ranges {1753088=180224} 2024-04-07T16:18:05.882 INFO:tasks.rados.rados.0.smithi100.stdout:451: writing smithi10066867-26 from 1753088 to 1933312 tid 1 2024-04-07T16:18:05.883 INFO:tasks.rados.rados.0.smithi100.stdout:452: snap_create 2024-04-07T16:18:05.883 INFO:tasks.rados.rados.0.smithi100.stdout:449: finishing write tid 1 to smithi10066867-12 2024-04-07T16:18:05.883 INFO:tasks.rados.rados.0.smithi100.stdout:449: finishing write tid 2 to smithi10066867-12 2024-04-07T16:18:05.883 INFO:tasks.rados.rados.0.smithi100.stdout:450: expect (ObjNum 151 snap 41 seq_num 151) 2024-04-07T16:18:05.997 INFO:tasks.rados.rados.0.smithi100.stdout:449: finishing write tid 3 to smithi10066867-12 2024-04-07T16:18:05.997 INFO:tasks.rados.rados.0.smithi100.stdout:449: oid 12 updating version 0 to 167 2024-04-07T16:18:05.997 INFO:tasks.rados.rados.0.smithi100.stdout:449: oid 12 updating version 167 to 168 2024-04-07T16:18:05.998 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 12 v 168 (ObjNum 153 snap 42 seq_num 153) dirty exists 2024-04-07T16:18:05.998 INFO:tasks.rados.rados.0.smithi100.stdout:449: left oid 12 (ObjNum 153 snap 42 seq_num 153) 2024-04-07T16:18:05.998 INFO:tasks.rados.rados.0.smithi100.stdout:451: finishing write tid 1 to smithi10066867-26 2024-04-07T16:18:05.998 INFO:tasks.rados.rados.0.smithi100.stdout:451: finishing write tid 2 to smithi10066867-26 2024-04-07T16:18:05.998 INFO:tasks.rados.rados.0.smithi100.stdout:451: finishing write tid 3 to smithi10066867-26 2024-04-07T16:18:05.998 INFO:tasks.rados.rados.0.smithi100.stdout:451: oid 26 updating version 0 to 107 2024-04-07T16:18:05.998 INFO:tasks.rados.rados.0.smithi100.stdout:451: oid 26 updating version 107 to 108 2024-04-07T16:18:05.998 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 26 v 108 (ObjNum 154 snap 42 seq_num 154) dirty exists 2024-04-07T16:18:05.998 INFO:tasks.rados.rados.0.smithi100.stdout:451: left oid 26 (ObjNum 154 snap 42 seq_num 154) 2024-04-07T16:18:06.605 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:06 smithi144 bash[23522]: cluster 2024-04-07T16:18:04.716918+0000 mgr.y (mgr.14553) 226 : cluster 0 pgmap v243: 121 pgs: 121 active+clean; 95 MiB data, 402 MiB used, 715 GiB / 715 GiB avail; 12 MiB/s rd, 2.6 MiB/s wr, 25 op/s 2024-04-07T16:18:06.605 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:06 smithi144 bash[23522]: cluster 2024-04-07T16:18:05.860142+0000 mon.a (mon.0) 977 : cluster 0 osdmap e151: 8 total, 8 up, 8 in 2024-04-07T16:18:06.687 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:06 smithi100 bash[31660]: cluster 2024-04-07T16:18:04.716918+0000 mgr.y (mgr.14553) 226 : cluster 0 pgmap v243: 121 pgs: 121 active+clean; 95 MiB data, 402 MiB used, 715 GiB / 715 GiB avail; 12 MiB/s rd, 2.6 MiB/s wr, 25 op/s 2024-04-07T16:18:06.687 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:06 smithi100 bash[31660]: cluster 2024-04-07T16:18:05.860142+0000 mon.a (mon.0) 977 : cluster 0 osdmap e151: 8 total, 8 up, 8 in 2024-04-07T16:18:06.687 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:06 smithi100 bash[21346]: cluster 2024-04-07T16:18:04.716918+0000 mgr.y (mgr.14553) 226 : cluster 0 pgmap v243: 121 pgs: 121 active+clean; 95 MiB data, 402 MiB used, 715 GiB / 715 GiB avail; 12 MiB/s rd, 2.6 MiB/s wr, 25 op/s 2024-04-07T16:18:06.687 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:06 smithi100 bash[21346]: cluster 2024-04-07T16:18:05.860142+0000 mon.a (mon.0) 977 : cluster 0 osdmap e151: 8 total, 8 up, 8 in 2024-04-07T16:18:06.851 INFO:tasks.rados.rados.0.smithi100.stdout:449: done (3 left) 2024-04-07T16:18:06.851 INFO:tasks.rados.rados.0.smithi100.stdout:450: done (2 left) 2024-04-07T16:18:06.851 INFO:tasks.rados.rados.0.smithi100.stdout:451: done (1 left) 2024-04-07T16:18:06.851 INFO:tasks.rados.rados.0.smithi100.stdout:452: done (0 left) 2024-04-07T16:18:06.851 INFO:tasks.rados.rados.0.smithi100.stdout:453: copy_from oid 15 from oid 40 current snap is 43 2024-04-07T16:18:06.851 INFO:tasks.rados.rados.0.smithi100.stdout:454: read oid 7 snap 41 2024-04-07T16:18:06.851 INFO:tasks.rados.rados.0.smithi100.stdout:454: expect (ObjNum 16 snap 0 seq_num 16) 2024-04-07T16:18:06.852 INFO:tasks.rados.rados.0.smithi100.stdout:455: rollback oid 31 current snap is 43 2024-04-07T16:18:06.852 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 31 to 34 2024-04-07T16:18:06.852 INFO:tasks.rados.rados.0.smithi100.stdout:456: delete oid 35 current snap is 43 2024-04-07T16:18:06.855 INFO:tasks.rados.rados.0.smithi100.stdout:455: finishing rollback tid 1 to smithi10066867-31 2024-04-07T16:18:06.855 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 31 v 0 (ObjNum 33 snap 0 seq_num 2670634328) dirty dne 2024-04-07T16:18:06.861 INFO:tasks.rados.rados.0.smithi100.stdout:455: done (3 left) 2024-04-07T16:18:06.861 INFO:tasks.rados.rados.0.smithi100.stdout:456: done (2 left) 2024-04-07T16:18:06.861 INFO:tasks.rados.rados.0.smithi100.stdout:457: read oid 8 snap -1 2024-04-07T16:18:06.861 INFO:tasks.rados.rados.0.smithi100.stdout:457: expect deleted 2024-04-07T16:18:06.861 INFO:tasks.rados.rados.0.smithi100.stdout:458: delete oid 17 current snap is 43 2024-04-07T16:18:06.862 INFO:tasks.rados.rados.0.smithi100.stdout:454: expect (ObjNum 16 snap 0 seq_num 16) 2024-04-07T16:18:06.889 INFO:tasks.rados.rados.0.smithi100.stdout:454: done (3 left) 2024-04-07T16:18:06.890 INFO:tasks.rados.rados.0.smithi100.stdout:457: done (2 left) 2024-04-07T16:18:06.890 INFO:tasks.rados.rados.0.smithi100.stdout:458: done (1 left) 2024-04-07T16:18:06.890 INFO:tasks.rados.rados.0.smithi100.stdout:459: rmattr oid 14 current snap is 43 2024-04-07T16:18:06.890 INFO:tasks.rados.rados.0.smithi100.stdout:459: done (1 left) 2024-04-07T16:18:06.890 INFO:tasks.rados.rados.0.smithi100.stdout:460: copy_from oid 5 from oid 36 current snap is 43 2024-04-07T16:18:06.890 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 24 current snap is 43 2024-04-07T16:18:06.890 INFO:tasks.rados.rados.0.smithi100.stdout:461: seq_num 157 ranges {2039808=237568} 2024-04-07T16:18:06.927 INFO:tasks.rados.rados.0.smithi100.stdout:461: writing smithi10066867-24 from 2039808 to 2277376 tid 1 2024-04-07T16:18:06.927 INFO:tasks.rados.rados.0.smithi100.stdout:462: rollback oid 14 current snap is 43 2024-04-07T16:18:06.927 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 14 to 39 2024-04-07T16:18:06.927 INFO:tasks.rados.rados.0.smithi100.stdout:460: finishing copy_from to smithi10066867-5 2024-04-07T16:18:06.927 INFO:tasks.rados.rados.0.smithi100.stdout:460: got expected ENOENT (src dne) 2024-04-07T16:18:06.927 INFO:tasks.rados.rados.0.smithi100.stdout:460: finishing copy_from racing read to smithi10066867-5 2024-04-07T16:18:06.927 INFO:tasks.rados.rados.0.smithi100.stdout:453: finishing copy_from racing read to smithi10066867-15 2024-04-07T16:18:06.927 INFO:tasks.rados.rados.0.smithi100.stdout:453: finishing copy_from to smithi10066867-15 2024-04-07T16:18:06.927 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 15 v 124 (ObjNum 124 snap 29 seq_num 124) dirty exists 2024-04-07T16:18:06.927 INFO:tasks.rados.rados.0.smithi100.stdout:453: done (3 left) 2024-04-07T16:18:06.928 INFO:tasks.rados.rados.0.smithi100.stdout:460: done (2 left) 2024-04-07T16:18:06.928 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 42 current snap is 43 2024-04-07T16:18:06.928 INFO:tasks.rados.rados.0.smithi100.stdout:463: seq_num 158 ranges {696320=253952} 2024-04-07T16:18:06.941 INFO:tasks.rados.rados.0.smithi100.stdout:463: writing smithi10066867-42 from 696320 to 950272 tid 1 2024-04-07T16:18:06.941 INFO:tasks.rados.rados.0.smithi100.stdout:464: copy_from oid 13 from oid 40 current snap is 43 2024-04-07T16:18:06.941 INFO:tasks.rados.rados.0.smithi100.stdout:462: finishing rollback tid 0 to smithi10066867-14 2024-04-07T16:18:06.941 INFO:tasks.rados.rados.0.smithi100.stdout:461: finishing write tid 1 to smithi10066867-24 2024-04-07T16:18:06.941 INFO:tasks.rados.rados.0.smithi100.stdout:462: finishing rollback tid 1 to smithi10066867-14 2024-04-07T16:18:06.941 INFO:tasks.rados.rados.0.smithi100.stdout:461: finishing write tid 2 to smithi10066867-24 2024-04-07T16:18:06.941 INFO:tasks.rados.rados.0.smithi100.stdout:462: finishing rollback tid 2 to smithi10066867-14 2024-04-07T16:18:06.942 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 14 v 69 (ObjNum 28 snap 0 seq_num 28) dirty exists 2024-04-07T16:18:06.942 INFO:tasks.rados.rados.0.smithi100.stdout:461: finishing write tid 3 to smithi10066867-24 2024-04-07T16:18:06.942 INFO:tasks.rados.rados.0.smithi100.stdout:461: oid 24 updating version 0 to 149 2024-04-07T16:18:06.942 INFO:tasks.rados.rados.0.smithi100.stdout:461: oid 24 updating version 149 to 150 2024-04-07T16:18:06.942 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 24 v 150 (ObjNum 157 snap 43 seq_num 157) dirty exists 2024-04-07T16:18:06.942 INFO:tasks.rados.rados.0.smithi100.stdout:461: left oid 24 (ObjNum 157 snap 43 seq_num 157) 2024-04-07T16:18:06.942 INFO:tasks.rados.rados.0.smithi100.stdout:461: done (3 left) 2024-04-07T16:18:06.942 INFO:tasks.rados.rados.0.smithi100.stdout:462: done (2 left) 2024-04-07T16:18:06.942 INFO:tasks.rados.rados.0.smithi100.stdout:465: read oid 28 snap -1 2024-04-07T16:18:06.942 INFO:tasks.rados.rados.0.smithi100.stdout:465: expect (ObjNum 140 snap 39 seq_num 140) 2024-04-07T16:18:06.942 INFO:tasks.rados.rados.0.smithi100.stdout:466: copy_from oid 20 from oid 10 current snap is 43 2024-04-07T16:18:06.942 INFO:tasks.rados.rados.0.smithi100.stdout:467: rmattr oid 39 current snap is 43 2024-04-07T16:18:06.943 INFO:tasks.rados.rados.0.smithi100.stdout:467: done (4 left) 2024-04-07T16:18:06.943 INFO:tasks.rados.rados.0.smithi100.stdout:468: rmattr oid 16 current snap is 43 2024-04-07T16:18:06.943 INFO:tasks.rados.rados.0.smithi100.stdout:468: done (4 left) 2024-04-07T16:18:06.943 INFO:tasks.rados.rados.0.smithi100.stdout:469: setattr oid 12 current snap is 43 2024-04-07T16:18:06.945 INFO:tasks.rados.rados.0.smithi100.stdout:470: rollback oid 23 current snap is 43 2024-04-07T16:18:06.945 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 23 to 34 2024-04-07T16:18:06.945 INFO:tasks.rados.rados.0.smithi100.stdout:471: delete oid 35 current snap is 43 2024-04-07T16:18:06.950 INFO:tasks.rados.rados.0.smithi100.stdout:471: done (6 left) 2024-04-07T16:18:06.950 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 43 current snap is 43 2024-04-07T16:18:06.951 INFO:tasks.rados.rados.0.smithi100.stdout:472: seq_num 160 ranges {0=286720} 2024-04-07T16:18:06.954 INFO:tasks.rados.rados.0.smithi100.stdout:472: writing smithi10066867-43 from 0 to 286720 tid 1 2024-04-07T16:18:06.954 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 33 current snap is 43 2024-04-07T16:18:06.954 INFO:tasks.rados.rados.0.smithi100.stdout:473: seq_num 161 ranges {4087808=303104} 2024-04-07T16:18:06.990 INFO:tasks.rados.rados.0.smithi100.stdout:473: writing smithi10066867-33 from 4087808 to 4390912 tid 1 2024-04-07T16:18:06.990 INFO:tasks.rados.rados.0.smithi100.stdout:474: read oid 1 snap -1 2024-04-07T16:18:06.990 INFO:tasks.rados.rados.0.smithi100.stdout:474: expect deleted 2024-04-07T16:18:06.990 INFO:tasks.rados.rados.0.smithi100.stdout:475: read oid 2 snap -1 2024-04-07T16:18:06.990 INFO:tasks.rados.rados.0.smithi100.stdout:475: expect (ObjNum 108 snap 20 seq_num 108) 2024-04-07T16:18:06.990 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 6 current snap is 43 2024-04-07T16:18:06.990 INFO:tasks.rados.rados.0.smithi100.stdout:476: seq_num 162 ranges {3891200=319488} 2024-04-07T16:18:07.016 INFO:tasks.rados.rados.0.smithi100.stdout:476: writing smithi10066867-6 from 3891200 to 4210688 tid 1 2024-04-07T16:18:07.016 INFO:tasks.rados.rados.0.smithi100.stdout:477: read oid 36 snap -1 2024-04-07T16:18:07.016 INFO:tasks.rados.rados.0.smithi100.stdout:477: expect deleted 2024-04-07T16:18:07.017 INFO:tasks.rados.rados.0.smithi100.stdout:478: read oid 21 snap -1 2024-04-07T16:18:07.017 INFO:tasks.rados.rados.0.smithi100.stdout:478: expect (ObjNum 16 snap 0 seq_num 16) 2024-04-07T16:18:07.017 INFO:tasks.rados.rados.0.smithi100.stdout:479: rmattr oid 29 current snap is 43 2024-04-07T16:18:07.017 INFO:tasks.rados.rados.0.smithi100.stdout:479: done (13 left) 2024-04-07T16:18:07.017 INFO:tasks.rados.rados.0.smithi100.stdout:480: setattr oid 18 current snap is 43 2024-04-07T16:18:07.017 INFO:tasks.rados.rados.0.smithi100.stdout:463: finishing write tid 1 to smithi10066867-42 2024-04-07T16:18:07.017 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 12 v 170 (ObjNum 153 snap 42 seq_num 153) dirty exists 2024-04-07T16:18:07.017 INFO:tasks.rados.rados.0.smithi100.stdout:470: finishing rollback tid 0 to smithi10066867-23 2024-04-07T16:18:07.017 INFO:tasks.rados.rados.0.smithi100.stdout:463: finishing write tid 2 to smithi10066867-42 2024-04-07T16:18:07.017 INFO:tasks.rados.rados.0.smithi100.stdout:463: finishing write tid 3 to smithi10066867-42 2024-04-07T16:18:07.018 INFO:tasks.rados.rados.0.smithi100.stdout:463: oid 42 updating version 0 to 134 2024-04-07T16:18:07.018 INFO:tasks.rados.rados.0.smithi100.stdout:463: oid 42 updating version 134 to 135 2024-04-07T16:18:07.018 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 42 v 135 (ObjNum 158 snap 43 seq_num 158) dirty exists 2024-04-07T16:18:07.018 INFO:tasks.rados.rados.0.smithi100.stdout:463: left oid 42 (ObjNum 158 snap 43 seq_num 158) 2024-04-07T16:18:07.018 INFO:tasks.rados.rados.0.smithi100.stdout:470: finishing rollback tid 1 to smithi10066867-23 2024-04-07T16:18:07.018 INFO:tasks.rados.rados.0.smithi100.stdout:470: finishing rollback tid 2 to smithi10066867-23 2024-04-07T16:18:07.018 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 23 v 207 (ObjNum 122 snap 29 seq_num 122) dirty exists 2024-04-07T16:18:07.018 INFO:tasks.rados.rados.0.smithi100.stdout:472: finishing write tid 1 to smithi10066867-43 2024-04-07T16:18:07.018 INFO:tasks.rados.rados.0.smithi100.stdout:472: finishing write tid 2 to smithi10066867-43 2024-04-07T16:18:07.018 INFO:tasks.rados.rados.0.smithi100.stdout:465: expect (ObjNum 140 snap 39 seq_num 140) 2024-04-07T16:18:07.105 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:18:06 smithi144 bash[51952]: debug 2024-04-07T16:18:06.826+0000 7f60520ca200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-07T16:18:07.105 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:18:06 smithi144 bash[51952]: debug 2024-04-07T16:18:06.930+0000 7f60520ca200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-07T16:18:07.117 INFO:tasks.rados.rados.0.smithi100.stdout:472: finishing write tid 3 to smithi10066867-43 2024-04-07T16:18:07.117 INFO:tasks.rados.rados.0.smithi100.stdout:472: oid 43 updating version 0 to 57 2024-04-07T16:18:07.117 INFO:tasks.rados.rados.0.smithi100.stdout:472: oid 43 version 57 is already newer than 56 2024-04-07T16:18:07.117 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 43 v 57 (ObjNum 160 snap 43 seq_num 160) dirty exists 2024-04-07T16:18:07.117 INFO:tasks.rados.rados.0.smithi100.stdout:472: left oid 43 (ObjNum 160 snap 43 seq_num 160) 2024-04-07T16:18:07.117 INFO:tasks.rados.rados.0.smithi100.stdout:466: finishing copy_from racing read to smithi10066867-20 2024-04-07T16:18:07.117 INFO:tasks.rados.rados.0.smithi100.stdout:466: finishing copy_from to smithi10066867-20 2024-04-07T16:18:07.117 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 20 v 29 (ObjNum 123 snap 29 seq_num 123) dirty exists 2024-04-07T16:18:07.117 INFO:tasks.rados.rados.0.smithi100.stdout:464: finishing copy_from racing read to smithi10066867-13 2024-04-07T16:18:07.117 INFO:tasks.rados.rados.0.smithi100.stdout:473: finishing write tid 1 to smithi10066867-33 2024-04-07T16:18:07.117 INFO:tasks.rados.rados.0.smithi100.stdout:473: finishing write tid 2 to smithi10066867-33 2024-04-07T16:18:07.117 INFO:tasks.rados.rados.0.smithi100.stdout:464: finishing copy_from to smithi10066867-13 2024-04-07T16:18:07.118 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 13 v 124 (ObjNum 124 snap 29 seq_num 124) dirty exists 2024-04-07T16:18:07.118 INFO:tasks.rados.rados.0.smithi100.stdout:473: finishing write tid 3 to smithi10066867-33 2024-04-07T16:18:07.118 INFO:tasks.rados.rados.0.smithi100.stdout:473: oid 33 updating version 0 to 209 2024-04-07T16:18:07.118 INFO:tasks.rados.rados.0.smithi100.stdout:473: oid 33 updating version 209 to 210 2024-04-07T16:18:07.118 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 33 v 210 (ObjNum 161 snap 43 seq_num 161) dirty exists 2024-04-07T16:18:07.118 INFO:tasks.rados.rados.0.smithi100.stdout:473: left oid 33 (ObjNum 161 snap 43 seq_num 161) 2024-04-07T16:18:07.118 INFO:tasks.rados.rados.0.smithi100.stdout:475: expect (ObjNum 108 snap 20 seq_num 108) 2024-04-07T16:18:07.234 INFO:tasks.rados.rados.0.smithi100.stdout:478: expect (ObjNum 16 snap 0 seq_num 16) 2024-04-07T16:18:07.242 INFO:tasks.rados.rados.0.smithi100.stdout:476: finishing write tid 1 to smithi10066867-6 2024-04-07T16:18:07.242 INFO:tasks.rados.rados.0.smithi100.stdout:476: finishing write tid 2 to smithi10066867-6 2024-04-07T16:18:07.242 INFO:tasks.rados.rados.0.smithi100.stdout:476: finishing write tid 3 to smithi10066867-6 2024-04-07T16:18:07.242 INFO:tasks.rados.rados.0.smithi100.stdout:476: oid 6 updating version 0 to 137 2024-04-07T16:18:07.242 INFO:tasks.rados.rados.0.smithi100.stdout:476: oid 6 version 137 is already newer than 136 2024-04-07T16:18:07.242 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 6 v 137 (ObjNum 162 snap 43 seq_num 162) dirty exists 2024-04-07T16:18:07.242 INFO:tasks.rados.rados.0.smithi100.stdout:476: left oid 6 (ObjNum 162 snap 43 seq_num 162) 2024-04-07T16:18:07.242 INFO:tasks.rados.rados.0.smithi100.stdout:463: done (13 left) 2024-04-07T16:18:07.242 INFO:tasks.rados.rados.0.smithi100.stdout:464: done (12 left) 2024-04-07T16:18:07.242 INFO:tasks.rados.rados.0.smithi100.stdout:465: done (11 left) 2024-04-07T16:18:07.242 INFO:tasks.rados.rados.0.smithi100.stdout:466: done (10 left) 2024-04-07T16:18:07.242 INFO:tasks.rados.rados.0.smithi100.stdout:469: done (9 left) 2024-04-07T16:18:07.242 INFO:tasks.rados.rados.0.smithi100.stdout:470: done (8 left) 2024-04-07T16:18:07.243 INFO:tasks.rados.rados.0.smithi100.stdout:472: done (7 left) 2024-04-07T16:18:07.243 INFO:tasks.rados.rados.0.smithi100.stdout:473: done (6 left) 2024-04-07T16:18:07.243 INFO:tasks.rados.rados.0.smithi100.stdout:474: done (5 left) 2024-04-07T16:18:07.243 INFO:tasks.rados.rados.0.smithi100.stdout:475: done (4 left) 2024-04-07T16:18:07.243 INFO:tasks.rados.rados.0.smithi100.stdout:476: done (3 left) 2024-04-07T16:18:07.243 INFO:tasks.rados.rados.0.smithi100.stdout:477: done (2 left) 2024-04-07T16:18:07.243 INFO:tasks.rados.rados.0.smithi100.stdout:478: done (1 left) 2024-04-07T16:18:07.243 INFO:tasks.rados.rados.0.smithi100.stdout:481: rmattr oid 6 current snap is 43 2024-04-07T16:18:07.243 INFO:tasks.rados.rados.0.smithi100.stdout:482: setattr oid 4 current snap is 43 2024-04-07T16:18:07.245 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 6 v 138 (ObjNum 162 snap 43 seq_num 162) dirty exists 2024-04-07T16:18:07.245 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 18 v 212 (ObjNum 152 snap 41 seq_num 152) dirty exists 2024-04-07T16:18:07.246 INFO:tasks.rados.rados.0.smithi100.stdout:480: done (2 left) 2024-04-07T16:18:07.246 INFO:tasks.rados.rados.0.smithi100.stdout:481: done (1 left) 2024-04-07T16:18:07.246 INFO:tasks.rados.rados.0.smithi100.stdout:483: snap_create 2024-04-07T16:18:07.250 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 4 v 214 (ObjNum 147 snap 40 seq_num 147) dirty exists 2024-04-07T16:18:07.605 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:18:07 smithi144 bash[51952]: debug 2024-04-07T16:18:07.198+0000 7f60520ca200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-07T16:18:07.841 INFO:tasks.rados.rados.0.smithi100.stdout:482: done (1 left) 2024-04-07T16:18:07.841 INFO:tasks.rados.rados.0.smithi100.stdout:483: done (0 left) 2024-04-07T16:18:07.841 INFO:tasks.rados.rados.0.smithi100.stdout:484: setattr oid 39 current snap is 44 2024-04-07T16:18:07.847 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 22 current snap is 44 2024-04-07T16:18:07.847 INFO:tasks.rados.rados.0.smithi100.stdout:485: seq_num 163 ranges {0=335872} 2024-04-07T16:18:07.852 INFO:tasks.rados.rados.0.smithi100.stdout:485: writing smithi10066867-22 from 0 to 335872 tid 1 2024-04-07T16:18:07.852 INFO:tasks.rados.rados.0.smithi100.stdout:486: snap_remove snap 25 2024-04-07T16:18:08.105 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:18:08 smithi144 bash[51952]: debug 2024-04-07T16:18:08.010+0000 7f60520ca200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-07T16:18:08.105 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:07 smithi144 bash[23522]: cluster 2024-04-07T16:18:06.717617+0000 mgr.y (mgr.14553) 227 : cluster 0 pgmap v246: 121 pgs: 121 active+clean; 95 MiB data, 402 MiB used, 715 GiB / 715 GiB avail; 12 MiB/s rd, 2.6 MiB/s wr, 25 op/s 2024-04-07T16:18:08.105 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:07 smithi144 bash[23522]: cluster 2024-04-07T16:18:06.862452+0000 mon.a (mon.0) 978 : cluster 0 osdmap e152: 8 total, 8 up, 8 in 2024-04-07T16:18:08.105 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:07 smithi144 bash[23522]: audit 2024-04-07T16:18:07.705491+0000 mon.b (mon.1) 249 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:18:08.105 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:07 smithi144 bash[23522]: cluster 2024-04-07T16:18:07.855301+0000 mon.a (mon.0) 979 : cluster 0 osdmap e153: 8 total, 8 up, 8 in 2024-04-07T16:18:08.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:07 smithi100 bash[21346]: cluster 2024-04-07T16:18:06.717617+0000 mgr.y (mgr.14553) 227 : cluster 0 pgmap v246: 121 pgs: 121 active+clean; 95 MiB data, 402 MiB used, 715 GiB / 715 GiB avail; 12 MiB/s rd, 2.6 MiB/s wr, 25 op/s 2024-04-07T16:18:08.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:07 smithi100 bash[21346]: cluster 2024-04-07T16:18:06.862452+0000 mon.a (mon.0) 978 : cluster 0 osdmap e152: 8 total, 8 up, 8 in 2024-04-07T16:18:08.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:07 smithi100 bash[21346]: audit 2024-04-07T16:18:07.705491+0000 mon.b (mon.1) 249 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:18:08.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:07 smithi100 bash[21346]: cluster 2024-04-07T16:18:07.855301+0000 mon.a (mon.0) 979 : cluster 0 osdmap e153: 8 total, 8 up, 8 in 2024-04-07T16:18:08.188 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:07 smithi100 bash[31660]: cluster 2024-04-07T16:18:06.717617+0000 mgr.y (mgr.14553) 227 : cluster 0 pgmap v246: 121 pgs: 121 active+clean; 95 MiB data, 402 MiB used, 715 GiB / 715 GiB avail; 12 MiB/s rd, 2.6 MiB/s wr, 25 op/s 2024-04-07T16:18:08.188 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:07 smithi100 bash[31660]: cluster 2024-04-07T16:18:06.862452+0000 mon.a (mon.0) 978 : cluster 0 osdmap e152: 8 total, 8 up, 8 in 2024-04-07T16:18:08.188 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:07 smithi100 bash[31660]: audit 2024-04-07T16:18:07.705491+0000 mon.b (mon.1) 249 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:18:08.188 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:07 smithi100 bash[31660]: cluster 2024-04-07T16:18:07.855301+0000 mon.a (mon.0) 979 : cluster 0 osdmap e153: 8 total, 8 up, 8 in 2024-04-07T16:18:08.574 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:18:08 smithi144 bash[51952]: debug 2024-04-07T16:18:08.122+0000 7f60520ca200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-07T16:18:08.575 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:18:08 smithi144 bash[51952]: debug 2024-04-07T16:18:08.226+0000 7f60520ca200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-07T16:18:08.856 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:18:08 smithi144 bash[51952]: debug 2024-04-07T16:18:08.582+0000 7f60520ca200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-07T16:18:08.856 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:18:08 smithi144 bash[51952]: debug 2024-04-07T16:18:08.698+0000 7f60520ca200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-07T16:18:08.856 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:18:08 smithi144 bash[51952]: debug 2024-04-07T16:18:08.806+0000 7f60520ca200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-07T16:18:08.860 INFO:tasks.rados.rados.0.smithi100.stdout:485: finishing write tid 1 to smithi10066867-22 2024-04-07T16:18:08.860 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 39 v 137 (ObjNum 72 snap 11 seq_num 72) dirty exists 2024-04-07T16:18:08.860 INFO:tasks.rados.rados.0.smithi100.stdout:485: finishing write tid 2 to smithi10066867-22 2024-04-07T16:18:08.860 INFO:tasks.rados.rados.0.smithi100.stdout:485: finishing write tid 3 to smithi10066867-22 2024-04-07T16:18:08.860 INFO:tasks.rados.rados.0.smithi100.stdout:485: oid 22 updating version 0 to 112 2024-04-07T16:18:08.860 INFO:tasks.rados.rados.0.smithi100.stdout:485: oid 22 updating version 112 to 113 2024-04-07T16:18:08.860 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 22 v 113 (ObjNum 163 snap 44 seq_num 163) dirty exists 2024-04-07T16:18:08.860 INFO:tasks.rados.rados.0.smithi100.stdout:485: left oid 22 (ObjNum 163 snap 44 seq_num 163) 2024-04-07T16:18:08.860 INFO:tasks.rados.rados.0.smithi100.stdout:484: done (2 left) 2024-04-07T16:18:08.861 INFO:tasks.rados.rados.0.smithi100.stdout:485: done (1 left) 2024-04-07T16:18:08.861 INFO:tasks.rados.rados.0.smithi100.stdout:486: done (0 left) 2024-04-07T16:18:08.861 INFO:tasks.rados.rados.0.smithi100.stdout:487: rmattr oid 43 current snap is 44 2024-04-07T16:18:08.861 INFO:tasks.rados.rados.0.smithi100.stdout:487: done (0 left) 2024-04-07T16:18:08.861 INFO:tasks.rados.rados.0.smithi100.stdout:488: copy_from oid 11 from oid 38 current snap is 44 2024-04-07T16:18:08.861 INFO:tasks.rados.rados.0.smithi100.stdout:489: copy_from oid 48 from oid 31 current snap is 44 2024-04-07T16:18:08.861 INFO:tasks.rados.rados.0.smithi100.stdout:490: delete oid 17 current snap is 44 2024-04-07T16:18:08.867 INFO:tasks.rados.rados.0.smithi100.stdout:489: finishing copy_from to smithi10066867-48 2024-04-07T16:18:08.867 INFO:tasks.rados.rados.0.smithi100.stdout:489: got expected ENOENT (src dne) 2024-04-07T16:18:08.868 INFO:tasks.rados.rados.0.smithi100.stdout:489: finishing copy_from racing read to smithi10066867-48 2024-04-07T16:18:08.879 INFO:tasks.rados.rados.0.smithi100.stdout:489: done (2 left) 2024-04-07T16:18:08.879 INFO:tasks.rados.rados.0.smithi100.stdout:490: done (1 left) 2024-04-07T16:18:08.879 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 44 current snap is 44 2024-04-07T16:18:08.879 INFO:tasks.rados.rados.0.smithi100.stdout:491: seq_num 165 ranges {589824=368640} 2024-04-07T16:18:08.885 INFO:tasks.rados.rados.0.smithi100.stdout:491: writing smithi10066867-44 from 589824 to 958464 tid 1 2024-04-07T16:18:08.885 INFO:tasks.rados.rados.0.smithi100.stdout:492: delete oid 19 current snap is 44 2024-04-07T16:18:08.891 INFO:tasks.rados.rados.0.smithi100.stdout:492: done (2 left) 2024-04-07T16:18:08.891 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 43 current snap is 44 2024-04-07T16:18:08.891 INFO:tasks.rados.rados.0.smithi100.stdout:493: seq_num 167 ranges {286720=401408} 2024-04-07T16:18:08.903 INFO:tasks.rados.rados.0.smithi100.stdout:493: writing smithi10066867-43 from 286720 to 688128 tid 1 2024-04-07T16:18:08.903 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 37 current snap is 44 2024-04-07T16:18:08.903 INFO:tasks.rados.rados.0.smithi100.stdout:494: seq_num 168 ranges {0=417792} 2024-04-07T16:18:08.903 INFO:tasks.rados.rados.0.smithi100.stdout:494: writing smithi10066867-37 from 0 to 417792 tid 1 2024-04-07T16:18:08.903 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 16 current snap is 44 2024-04-07T16:18:08.903 INFO:tasks.rados.rados.0.smithi100.stdout:495: seq_num 169 ranges {3874816=434176} 2024-04-07T16:18:08.923 INFO:tasks.rados.rados.0.smithi100.stdout:495: writing smithi10066867-16 from 3874816 to 4308992 tid 1 2024-04-07T16:18:08.923 INFO:tasks.rados.rados.0.smithi100.stdout:496: read oid 25 snap -1 2024-04-07T16:18:08.923 INFO:tasks.rados.rados.0.smithi100.stdout:496: expect (ObjNum 24 snap 0 seq_num 24) 2024-04-07T16:18:08.923 INFO:tasks.rados.rados.0.smithi100.stdout:497: snap_remove snap 34 2024-04-07T16:18:09.431 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:18:09 smithi144 bash[51952]: debug 2024-04-07T16:18:09.182+0000 7f60520ca200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-07T16:18:09.718 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:18:09 smithi144 bash[51952]: debug 2024-04-07T16:18:09.438+0000 7f60520ca200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-07T16:18:09.867 INFO:tasks.rados.rados.0.smithi100.stdout:491: finishing write tid 1 to smithi10066867-44 2024-04-07T16:18:09.867 INFO:tasks.rados.rados.0.smithi100.stdout:491: finishing write tid 2 to smithi10066867-44 2024-04-07T16:18:09.867 INFO:tasks.rados.rados.0.smithi100.stdout:493: finishing write tid 1 to smithi10066867-43 2024-04-07T16:18:09.867 INFO:tasks.rados.rados.0.smithi100.stdout:493: finishing write tid 2 to smithi10066867-43 2024-04-07T16:18:09.867 INFO:tasks.rados.rados.0.smithi100.stdout:494: finishing write tid 1 to smithi10066867-37 2024-04-07T16:18:09.867 INFO:tasks.rados.rados.0.smithi100.stdout:494: finishing write tid 2 to smithi10066867-37 2024-04-07T16:18:09.868 INFO:tasks.rados.rados.0.smithi100.stdout:493: finishing write tid 3 to smithi10066867-43 2024-04-07T16:18:09.868 INFO:tasks.rados.rados.0.smithi100.stdout:493: oid 43 updating version 0 to 63 2024-04-07T16:18:09.868 INFO:tasks.rados.rados.0.smithi100.stdout:493: oid 43 updating version 63 to 64 2024-04-07T16:18:09.868 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 43 v 64 (ObjNum 167 snap 44 seq_num 167) dirty exists 2024-04-07T16:18:09.868 INFO:tasks.rados.rados.0.smithi100.stdout:493: left oid 43 (ObjNum 167 snap 44 seq_num 167) 2024-04-07T16:18:09.868 INFO:tasks.rados.rados.0.smithi100.stdout:494: finishing write tid 3 to smithi10066867-37 2024-04-07T16:18:09.868 INFO:tasks.rados.rados.0.smithi100.stdout:494: oid 37 updating version 0 to 148 2024-04-07T16:18:09.868 INFO:tasks.rados.rados.0.smithi100.stdout:494: oid 37 updating version 148 to 149 2024-04-07T16:18:09.868 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 37 v 149 (ObjNum 168 snap 44 seq_num 168) dirty exists 2024-04-07T16:18:09.868 INFO:tasks.rados.rados.0.smithi100.stdout:494: left oid 37 (ObjNum 168 snap 44 seq_num 168) 2024-04-07T16:18:09.868 INFO:tasks.rados.rados.0.smithi100.stdout:491: finishing write tid 3 to smithi10066867-44 2024-04-07T16:18:09.868 INFO:tasks.rados.rados.0.smithi100.stdout:491: oid 44 updating version 0 to 145 2024-04-07T16:18:09.868 INFO:tasks.rados.rados.0.smithi100.stdout:491: oid 44 version 145 is already newer than 144 2024-04-07T16:18:09.868 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 44 v 145 (ObjNum 165 snap 44 seq_num 165) dirty exists 2024-04-07T16:18:09.869 INFO:tasks.rados.rados.0.smithi100.stdout:491: left oid 44 (ObjNum 165 snap 44 seq_num 165) 2024-04-07T16:18:09.869 INFO:tasks.rados.rados.0.smithi100.stdout:488: finishing copy_from racing read to smithi10066867-11 2024-04-07T16:18:09.869 INFO:tasks.rados.rados.0.smithi100.stdout:488: finishing copy_from to smithi10066867-11 2024-04-07T16:18:09.869 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 11 v 77 (ObjNum 136 snap 34 seq_num 136) dirty exists 2024-04-07T16:18:09.869 INFO:tasks.rados.rados.0.smithi100.stdout:496: expect (ObjNum 24 snap 0 seq_num 24) 2024-04-07T16:18:09.883 INFO:tasks.rados.rados.0.smithi100.stdout:495: finishing write tid 1 to smithi10066867-16 2024-04-07T16:18:09.883 INFO:tasks.rados.rados.0.smithi100.stdout:495: finishing write tid 2 to smithi10066867-16 2024-04-07T16:18:09.883 INFO:tasks.rados.rados.0.smithi100.stdout:495: finishing write tid 3 to smithi10066867-16 2024-04-07T16:18:09.883 INFO:tasks.rados.rados.0.smithi100.stdout:495: oid 16 updating version 0 to 179 2024-04-07T16:18:09.884 INFO:tasks.rados.rados.0.smithi100.stdout:495: oid 16 version 179 is already newer than 178 2024-04-07T16:18:09.884 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 16 v 179 (ObjNum 169 snap 44 seq_num 169) dirty exists 2024-04-07T16:18:09.884 INFO:tasks.rados.rados.0.smithi100.stdout:495: left oid 16 (ObjNum 169 snap 44 seq_num 169) 2024-04-07T16:18:09.884 INFO:tasks.rados.rados.0.smithi100.stdout:488: done (6 left) 2024-04-07T16:18:09.884 INFO:tasks.rados.rados.0.smithi100.stdout:491: done (5 left) 2024-04-07T16:18:09.884 INFO:tasks.rados.rados.0.smithi100.stdout:493: done (4 left) 2024-04-07T16:18:09.884 INFO:tasks.rados.rados.0.smithi100.stdout:494: done (3 left) 2024-04-07T16:18:09.884 INFO:tasks.rados.rados.0.smithi100.stdout:495: done (2 left) 2024-04-07T16:18:09.884 INFO:tasks.rados.rados.0.smithi100.stdout:496: done (1 left) 2024-04-07T16:18:09.884 INFO:tasks.rados.rados.0.smithi100.stdout:497: done (0 left) 2024-04-07T16:18:09.884 INFO:tasks.rados.rados.0.smithi100.stdout:498: rollback oid 1 current snap is 44 2024-04-07T16:18:09.884 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 1 to 33 2024-04-07T16:18:09.884 INFO:tasks.rados.rados.0.smithi100.stdout:499: delete oid 32 current snap is 44 2024-04-07T16:18:09.891 INFO:tasks.rados.rados.0.smithi100.stdout:499: done (1 left) 2024-04-07T16:18:09.891 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 4 current snap is 44 2024-04-07T16:18:09.891 INFO:tasks.rados.rados.0.smithi100.stdout:500: seq_num 171 ranges {65536=466944} 2024-04-07T16:18:09.895 INFO:tasks.rados.rados.0.smithi100.stdout:500: writing smithi10066867-4 from 65536 to 532480 tid 1 2024-04-07T16:18:09.895 INFO:tasks.rados.rados.0.smithi100.stdout:501: snap_remove snap 41 2024-04-07T16:18:10.105 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:09 smithi144 bash[23522]: cluster 2024-04-07T16:18:08.718590+0000 mgr.y (mgr.14553) 228 : cluster 0 pgmap v249: 121 pgs: 121 active+clean; 95 MiB data, 405 MiB used, 715 GiB / 715 GiB avail; 547 KiB/s rd, 24 KiB/s wr, 3 op/s 2024-04-07T16:18:10.105 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:09 smithi144 bash[23522]: cluster 2024-04-07T16:18:08.873723+0000 mon.a (mon.0) 980 : cluster 0 osdmap e154: 8 total, 8 up, 8 in 2024-04-07T16:18:10.105 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:18:09 smithi144 bash[51952]: debug 2024-04-07T16:18:09.726+0000 7f60520ca200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-07T16:18:10.105 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:18:09 smithi144 bash[51952]: debug 2024-04-07T16:18:09.866+0000 7f60520ca200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-07T16:18:10.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:09 smithi100 bash[21346]: cluster 2024-04-07T16:18:08.718590+0000 mgr.y (mgr.14553) 228 : cluster 0 pgmap v249: 121 pgs: 121 active+clean; 95 MiB data, 405 MiB used, 715 GiB / 715 GiB avail; 547 KiB/s rd, 24 KiB/s wr, 3 op/s 2024-04-07T16:18:10.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:09 smithi100 bash[21346]: cluster 2024-04-07T16:18:08.873723+0000 mon.a (mon.0) 980 : cluster 0 osdmap e154: 8 total, 8 up, 8 in 2024-04-07T16:18:10.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:09 smithi100 bash[31660]: cluster 2024-04-07T16:18:08.718590+0000 mgr.y (mgr.14553) 228 : cluster 0 pgmap v249: 121 pgs: 121 active+clean; 95 MiB data, 405 MiB used, 715 GiB / 715 GiB avail; 547 KiB/s rd, 24 KiB/s wr, 3 op/s 2024-04-07T16:18:10.188 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:09 smithi100 bash[31660]: cluster 2024-04-07T16:18:08.873723+0000 mon.a (mon.0) 980 : cluster 0 osdmap e154: 8 total, 8 up, 8 in 2024-04-07T16:18:10.603 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:18:10 smithi144 bash[51952]: debug 2024-04-07T16:18:10.178+0000 7f60520ca200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-07T16:18:10.855 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:18:10 smithi144 bash[51952]: debug 2024-04-07T16:18:10.610+0000 7f60520ca200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-07T16:18:10.855 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:18:10 smithi144 bash[51952]: debug 2024-04-07T16:18:10.854+0000 7f60520ca200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-07T16:18:10.894 INFO:tasks.rados.rados.0.smithi100.stdout:501: done (2 left) 2024-04-07T16:18:10.894 INFO:tasks.rados.rados.0.smithi100.stdout:502: copy_from oid 14 from oid 43 current snap is 44 2024-04-07T16:18:10.894 INFO:tasks.rados.rados.0.smithi100.stdout:498: finishing rollback tid 1 to smithi10066867-1 2024-04-07T16:18:10.894 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 1 v 0 (ObjNum 161 snap 0 seq_num 1) dirty dne 2024-04-07T16:18:10.894 INFO:tasks.rados.rados.0.smithi100.stdout:500: finishing write tid 1 to smithi10066867-4 2024-04-07T16:18:10.894 INFO:tasks.rados.rados.0.smithi100.stdout:500: finishing write tid 2 to smithi10066867-4 2024-04-07T16:18:10.894 INFO:tasks.rados.rados.0.smithi100.stdout:500: finishing write tid 3 to smithi10066867-4 2024-04-07T16:18:10.894 INFO:tasks.rados.rados.0.smithi100.stdout:500: oid 4 updating version 0 to 226 2024-04-07T16:18:10.894 INFO:tasks.rados.rados.0.smithi100.stdout:500: oid 4 updating version 226 to 227 2024-04-07T16:18:10.894 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 4 v 227 (ObjNum 171 snap 44 seq_num 171) dirty exists 2024-04-07T16:18:10.894 INFO:tasks.rados.rados.0.smithi100.stdout:500: left oid 4 (ObjNum 171 snap 44 seq_num 171) 2024-04-07T16:18:10.895 INFO:tasks.rados.rados.0.smithi100.stdout:498: done (2 left) 2024-04-07T16:18:10.895 INFO:tasks.rados.rados.0.smithi100.stdout:500: done (1 left) 2024-04-07T16:18:10.895 INFO:tasks.rados.rados.0.smithi100.stdout:503: setattr oid 27 current snap is 44 2024-04-07T16:18:10.900 INFO:tasks.rados.rados.0.smithi100.stdout:504: rollback oid 3 current snap is 44 2024-04-07T16:18:10.900 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 3 to 33 2024-04-07T16:18:10.900 INFO:tasks.rados.rados.0.smithi100.stdout:505: setattr oid 40 current snap is 44 2024-04-07T16:18:10.902 INFO:tasks.rados.rados.0.smithi100.stdout:506: snap_create 2024-04-07T16:18:10.908 INFO:tasks.rados.rados.0.smithi100.stdout:504: finishing rollback tid 0 to smithi10066867-3 2024-04-07T16:18:10.908 INFO:tasks.rados.rados.0.smithi100.stdout:504: finishing rollback tid 1 to smithi10066867-3 2024-04-07T16:18:10.908 INFO:tasks.rados.rados.0.smithi100.stdout:504: finishing rollback tid 2 to smithi10066867-3 2024-04-07T16:18:10.908 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 3 v 97 (ObjNum 80 snap 13 seq_num 80) dirty exists 2024-04-07T16:18:10.916 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 40 v 151 (ObjNum 124 snap 29 seq_num 124) dirty exists 2024-04-07T16:18:10.921 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 27 v 67 (ObjNum 28785 snap 0 seq_num 2701827280) dirty exists 2024-04-07T16:18:10.925 INFO:tasks.rados.rados.0.smithi100.stdout:502: finishing copy_from racing read to smithi10066867-14 2024-04-07T16:18:10.926 INFO:tasks.rados.rados.0.smithi100.stdout:502: finishing copy_from to smithi10066867-14 2024-04-07T16:18:10.926 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 14 v 64 (ObjNum 167 snap 44 seq_num 167) dirty exists 2024-04-07T16:18:11.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:10 smithi100 bash[21346]: cluster 2024-04-07T16:18:09.875795+0000 mon.a (mon.0) 981 : cluster 0 osdmap e155: 8 total, 8 up, 8 in 2024-04-07T16:18:11.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:10 smithi100 bash[21346]: audit 2024-04-07T16:18:10.235189+0000 mon.a (mon.0) 982 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:18:11.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:10 smithi100 bash[21346]: audit 2024-04-07T16:18:10.253423+0000 mon.a (mon.0) 983 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:18:11.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:10 smithi100 bash[31660]: cluster 2024-04-07T16:18:09.875795+0000 mon.a (mon.0) 981 : cluster 0 osdmap e155: 8 total, 8 up, 8 in 2024-04-07T16:18:11.188 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:10 smithi100 bash[31660]: audit 2024-04-07T16:18:10.235189+0000 mon.a (mon.0) 982 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:18:11.188 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:10 smithi100 bash[31660]: audit 2024-04-07T16:18:10.253423+0000 mon.a (mon.0) 983 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:18:11.249 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:18:10 smithi144 bash[51952]: debug 2024-04-07T16:18:10.994+0000 7f60520ca200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-07T16:18:11.250 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:10 smithi144 bash[23522]: cluster 2024-04-07T16:18:09.875795+0000 mon.a (mon.0) 981 : cluster 0 osdmap e155: 8 total, 8 up, 8 in 2024-04-07T16:18:11.250 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:10 smithi144 bash[23522]: audit 2024-04-07T16:18:10.235189+0000 mon.a (mon.0) 982 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:18:11.250 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:10 smithi144 bash[23522]: audit 2024-04-07T16:18:10.253423+0000 mon.a (mon.0) 983 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:18:11.605 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:18:11 smithi144 bash[51952]: debug 2024-04-07T16:18:11.258+0000 7f60520ca200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-07T16:18:11.905 INFO:tasks.rados.rados.0.smithi100.stdout:502: done (4 left) 2024-04-07T16:18:11.905 INFO:tasks.rados.rados.0.smithi100.stdout:503: done (3 left) 2024-04-07T16:18:11.905 INFO:tasks.rados.rados.0.smithi100.stdout:504: done (2 left) 2024-04-07T16:18:11.905 INFO:tasks.rados.rados.0.smithi100.stdout:505: done (1 left) 2024-04-07T16:18:11.905 INFO:tasks.rados.rados.0.smithi100.stdout:506: done (0 left) 2024-04-07T16:18:11.905 INFO:tasks.rados.rados.0.smithi100.stdout:507: snap_create 2024-04-07T16:18:12.188 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:11 smithi100 bash[21346]: cluster 2024-04-07T16:18:10.719370+0000 mgr.y (mgr.14553) 229 : cluster 0 pgmap v252: 121 pgs: 121 active+clean; 110 MiB data, 446 MiB used, 715 GiB / 715 GiB avail; 6.0 MiB/s rd, 3.7 MiB/s wr, 42 op/s 2024-04-07T16:18:12.188 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:11 smithi100 bash[21346]: cluster 2024-04-07T16:18:10.908695+0000 mon.a (mon.0) 984 : cluster 0 osdmap e156: 8 total, 8 up, 8 in 2024-04-07T16:18:12.188 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:11 smithi100 bash[21346]: audit 2024-04-07T16:18:11.112655+0000 mon.a (mon.0) 985 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:18:12.188 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:11 smithi100 bash[21346]: audit 2024-04-07T16:18:11.129161+0000 mon.a (mon.0) 986 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:18:12.188 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:11 smithi100 bash[31660]: cluster 2024-04-07T16:18:10.719370+0000 mgr.y (mgr.14553) 229 : cluster 0 pgmap v252: 121 pgs: 121 active+clean; 110 MiB data, 446 MiB used, 715 GiB / 715 GiB avail; 6.0 MiB/s rd, 3.7 MiB/s wr, 42 op/s 2024-04-07T16:18:12.188 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:11 smithi100 bash[31660]: cluster 2024-04-07T16:18:10.908695+0000 mon.a (mon.0) 984 : cluster 0 osdmap e156: 8 total, 8 up, 8 in 2024-04-07T16:18:12.188 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:11 smithi100 bash[31660]: audit 2024-04-07T16:18:11.112655+0000 mon.a (mon.0) 985 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:18:12.188 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:11 smithi100 bash[31660]: audit 2024-04-07T16:18:11.129161+0000 mon.a (mon.0) 986 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:18:12.282 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:11 smithi144 bash[23522]: cluster 2024-04-07T16:18:10.719370+0000 mgr.y (mgr.14553) 229 : cluster 0 pgmap v252: 121 pgs: 121 active+clean; 110 MiB data, 446 MiB used, 715 GiB / 715 GiB avail; 6.0 MiB/s rd, 3.7 MiB/s wr, 42 op/s 2024-04-07T16:18:12.283 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:11 smithi144 bash[23522]: cluster 2024-04-07T16:18:10.908695+0000 mon.a (mon.0) 984 : cluster 0 osdmap e156: 8 total, 8 up, 8 in 2024-04-07T16:18:12.283 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:11 smithi144 bash[23522]: audit 2024-04-07T16:18:11.112655+0000 mon.a (mon.0) 985 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:18:12.283 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:11 smithi144 bash[23522]: audit 2024-04-07T16:18:11.129161+0000 mon.a (mon.0) 986 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:18:12.605 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:18:12 smithi144 bash[51952]: debug 2024-04-07T16:18:12.290+0000 7f60520ca200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-07T16:18:12.605 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:18:12 smithi144 bash[51952]: debug 2024-04-07T16:18:12.410+0000 7f60520ca200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-07T16:18:12.846 INFO:tasks.rados.rados.0.smithi100.stdout:507: done (0 left) 2024-04-07T16:18:12.846 INFO:tasks.rados.rados.0.smithi100.stdout:508: setattr oid 4 current snap is 46 2024-04-07T16:18:12.852 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 14 current snap is 46 2024-04-07T16:18:12.852 INFO:tasks.rados.rados.0.smithi100.stdout:509: seq_num 172 ranges {688128=483328} 2024-04-07T16:18:12.871 INFO:tasks.rados.rados.0.smithi100.stdout:509: writing smithi10066867-14 from 688128 to 1171456 tid 1 2024-04-07T16:18:12.871 INFO:tasks.rados.rados.0.smithi100.stdout:510: read oid 16 snap 45 2024-04-07T16:18:12.871 INFO:tasks.rados.rados.0.smithi100.stdout:510: expect (ObjNum 169 snap 44 seq_num 169) 2024-04-07T16:18:12.871 INFO:tasks.rados.rados.0.smithi100.stdout:511: rmattr oid 10 current snap is 46 2024-04-07T16:18:12.871 INFO:tasks.rados.rados.0.smithi100.stdout:511: done (3 left) 2024-04-07T16:18:12.871 INFO:tasks.rados.rados.0.smithi100.stdout:512: copy_from oid 41 from oid 17 current snap is 46 2024-04-07T16:18:12.871 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 4 v 249 (ObjNum 171 snap 44 seq_num 171) dirty exists 2024-04-07T16:18:12.871 INFO:tasks.rados.rados.0.smithi100.stdout:508: done (3 left) 2024-04-07T16:18:12.871 INFO:tasks.rados.rados.0.smithi100.stdout:513: delete oid 22 current snap is 46 2024-04-07T16:18:12.879 INFO:tasks.rados.rados.0.smithi100.stdout:509: finishing write tid 1 to smithi10066867-14 2024-04-07T16:18:12.879 INFO:tasks.rados.rados.0.smithi100.stdout:513: done (3 left) 2024-04-07T16:18:12.879 INFO:tasks.rados.rados.0.smithi100.stdout:514: rmattr oid 28 current snap is 46 2024-04-07T16:18:12.879 INFO:tasks.rados.rados.0.smithi100.stdout:514: done (3 left) 2024-04-07T16:18:12.880 INFO:tasks.rados.rados.0.smithi100.stdout:515: rollback oid 13 current snap is 46 2024-04-07T16:18:12.880 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 13 to 39 2024-04-07T16:18:12.880 INFO:tasks.rados.rados.0.smithi100.stdout:509: finishing write tid 2 to smithi10066867-14 2024-04-07T16:18:12.880 INFO:tasks.rados.rados.0.smithi100.stdout:516: copy_from oid 23 from oid 3 current snap is 46 2024-04-07T16:18:12.880 INFO:tasks.rados.rados.0.smithi100.stdout:517: read oid 6 snap -1 2024-04-07T16:18:12.880 INFO:tasks.rados.rados.0.smithi100.stdout:517: expect (ObjNum 162 snap 43 seq_num 162) 2024-04-07T16:18:12.880 INFO:tasks.rados.rados.0.smithi100.stdout:518: read oid 44 snap 38 2024-04-07T16:18:12.880 INFO:tasks.rados.rados.0.smithi100.stdout:518: expect (ObjNum 28 snap 0 seq_num 28) 2024-04-07T16:18:12.880 INFO:tasks.rados.rados.0.smithi100.stdout:519: delete oid 8 current snap is 46 2024-04-07T16:18:12.883 INFO:tasks.rados.rados.0.smithi100.stdout:509: finishing write tid 3 to smithi10066867-14 2024-04-07T16:18:12.883 INFO:tasks.rados.rados.0.smithi100.stdout:509: oid 14 updating version 0 to 81 2024-04-07T16:18:12.884 INFO:tasks.rados.rados.0.smithi100.stdout:509: oid 14 updating version 81 to 82 2024-04-07T16:18:12.884 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 14 v 82 (ObjNum 172 snap 46 seq_num 172) dirty exists 2024-04-07T16:18:12.884 INFO:tasks.rados.rados.0.smithi100.stdout:509: left oid 14 (ObjNum 172 snap 46 seq_num 172) 2024-04-07T16:18:12.895 INFO:tasks.rados.rados.0.smithi100.stdout:515: finishing rollback tid 0 to smithi10066867-13 2024-04-07T16:18:12.899 INFO:tasks.rados.rados.0.smithi100.stdout:515: finishing rollback tid 1 to smithi10066867-13 2024-04-07T16:18:12.899 INFO:tasks.rados.rados.0.smithi100.stdout:515: finishing rollback tid 2 to smithi10066867-13 2024-04-07T16:18:12.900 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 13 v 137 (ObjNum 134 snap 34 seq_num 134) dirty exists 2024-04-07T16:18:12.901 INFO:tasks.rados.rados.0.smithi100.stdout:509: done (7 left) 2024-04-07T16:18:12.901 INFO:tasks.rados.rados.0.smithi100.stdout:515: done (6 left) 2024-04-07T16:18:12.901 INFO:tasks.rados.rados.0.smithi100.stdout:519: done (5 left) 2024-04-07T16:18:12.901 INFO:tasks.rados.rados.0.smithi100.stdout:520: copy_from oid 11 from oid 30 current snap is 46 2024-04-07T16:18:12.901 INFO:tasks.rados.rados.0.smithi100.stdout:521: read oid 20 snap 37 2024-04-07T16:18:12.901 INFO:tasks.rados.rados.0.smithi100.stdout:521: expect (ObjNum -1592421568 snap 21978 seq_num 2701953936) 2024-04-07T16:18:12.901 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 31 current snap is 46 2024-04-07T16:18:12.901 INFO:tasks.rados.rados.0.smithi100.stdout:522: seq_num 175 ranges {0=540672} 2024-04-07T16:18:12.909 INFO:tasks.rados.rados.0.smithi100.stdout:522: writing smithi10066867-31 from 0 to 540672 tid 1 2024-04-07T16:18:12.909 INFO:tasks.rados.rados.0.smithi100.stdout:523: copy_from oid 34 from oid 28 current snap is 46 2024-04-07T16:18:12.909 INFO:tasks.rados.rados.0.smithi100.stdout:510: expect (ObjNum 169 snap 44 seq_num 169) 2024-04-07T16:18:13.105 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:12 smithi144 bash[23522]: cluster 2024-04-07T16:18:11.916387+0000 mon.a (mon.0) 987 : cluster 0 osdmap e157: 8 total, 8 up, 8 in 2024-04-07T16:18:13.105 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:12 smithi144 bash[23522]: cluster 2024-04-07T16:18:12.860235+0000 mon.a (mon.0) 988 : cluster 0 osdmap e158: 8 total, 8 up, 8 in 2024-04-07T16:18:13.105 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:18:12 smithi144 bash[51952]: debug 2024-04-07T16:18:12.714+0000 7f60520ca200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-07T16:18:13.105 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:18:12 smithi144 bash[51952]: debug 2024-04-07T16:18:12.826+0000 7f60520ca200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-07T16:18:13.121 INFO:tasks.rados.rados.0.smithi100.stdout:512: finishing copy_from to smithi10066867-41 2024-04-07T16:18:13.121 INFO:tasks.rados.rados.0.smithi100.stdout:512: got expected ENOENT (src dne) 2024-04-07T16:18:13.121 INFO:tasks.rados.rados.0.smithi100.stdout:512: finishing copy_from racing read to smithi10066867-41 2024-04-07T16:18:13.121 INFO:tasks.rados.rados.0.smithi100.stdout:521: expect (ObjNum -1592421568 snap 21978 seq_num 2701953936) 2024-04-07T16:18:13.126 INFO:tasks.rados.rados.0.smithi100.stdout:516: finishing copy_from racing read to smithi10066867-23 2024-04-07T16:18:13.126 INFO:tasks.rados.rados.0.smithi100.stdout:517: expect (ObjNum 162 snap 43 seq_num 162) 2024-04-07T16:18:13.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:12 smithi100 bash[21346]: cluster 2024-04-07T16:18:11.916387+0000 mon.a (mon.0) 987 : cluster 0 osdmap e157: 8 total, 8 up, 8 in 2024-04-07T16:18:13.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:12 smithi100 bash[21346]: cluster 2024-04-07T16:18:12.860235+0000 mon.a (mon.0) 988 : cluster 0 osdmap e158: 8 total, 8 up, 8 in 2024-04-07T16:18:13.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:12 smithi100 bash[31660]: cluster 2024-04-07T16:18:11.916387+0000 mon.a (mon.0) 987 : cluster 0 osdmap e157: 8 total, 8 up, 8 in 2024-04-07T16:18:13.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:12 smithi100 bash[31660]: cluster 2024-04-07T16:18:12.860235+0000 mon.a (mon.0) 988 : cluster 0 osdmap e158: 8 total, 8 up, 8 in 2024-04-07T16:18:13.330 INFO:tasks.rados.rados.0.smithi100.stdout:516: finishing copy_from to smithi10066867-23 2024-04-07T16:18:13.330 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 23 v 97 (ObjNum 80 snap 13 seq_num 80) dirty exists 2024-04-07T16:18:13.330 INFO:tasks.rados.rados.0.smithi100.stdout:518: expect (ObjNum 28 snap 0 seq_num 28) 2024-04-07T16:18:13.342 INFO:tasks.rados.rados.0.smithi100.stdout:522: finishing write tid 1 to smithi10066867-31 2024-04-07T16:18:13.342 INFO:tasks.rados.rados.0.smithi100.stdout:522: finishing write tid 2 to smithi10066867-31 2024-04-07T16:18:13.343 INFO:tasks.rados.rados.0.smithi100.stdout:522: finishing write tid 3 to smithi10066867-31 2024-04-07T16:18:13.343 INFO:tasks.rados.rados.0.smithi100.stdout:522: oid 31 updating version 0 to 78 2024-04-07T16:18:13.343 INFO:tasks.rados.rados.0.smithi100.stdout:522: oid 31 version 78 is already newer than 77 2024-04-07T16:18:13.343 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 31 v 78 (ObjNum 175 snap 46 seq_num 175) dirty exists 2024-04-07T16:18:13.343 INFO:tasks.rados.rados.0.smithi100.stdout:522: left oid 31 (ObjNum 175 snap 46 seq_num 175) 2024-04-07T16:18:13.343 INFO:tasks.rados.rados.0.smithi100.stdout:520: finishing copy_from racing read to smithi10066867-11 2024-04-07T16:18:13.343 INFO:tasks.rados.rados.0.smithi100.stdout:523: finishing copy_from racing read to smithi10066867-34 2024-04-07T16:18:13.343 INFO:tasks.rados.rados.0.smithi100.stdout:523: finishing copy_from to smithi10066867-34 2024-04-07T16:18:13.343 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 34 v 35 (ObjNum 140 snap 39 seq_num 140) dirty exists 2024-04-07T16:18:13.343 INFO:tasks.rados.rados.0.smithi100.stdout:520: finishing copy_from to smithi10066867-11 2024-04-07T16:18:13.343 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 11 v 87 (ObjNum 106 snap 20 seq_num 106) dirty exists 2024-04-07T16:18:13.343 INFO:tasks.rados.rados.0.smithi100.stdout:510: done (8 left) 2024-04-07T16:18:13.343 INFO:tasks.rados.rados.0.smithi100.stdout:512: done (7 left) 2024-04-07T16:18:13.343 INFO:tasks.rados.rados.0.smithi100.stdout:516: done (6 left) 2024-04-07T16:18:13.343 INFO:tasks.rados.rados.0.smithi100.stdout:517: done (5 left) 2024-04-07T16:18:13.343 INFO:tasks.rados.rados.0.smithi100.stdout:518: done (4 left) 2024-04-07T16:18:13.344 INFO:tasks.rados.rados.0.smithi100.stdout:520: done (3 left) 2024-04-07T16:18:13.344 INFO:tasks.rados.rados.0.smithi100.stdout:521: done (2 left) 2024-04-07T16:18:13.344 INFO:tasks.rados.rados.0.smithi100.stdout:522: done (1 left) 2024-04-07T16:18:13.344 INFO:tasks.rados.rados.0.smithi100.stdout:523: done (0 left) 2024-04-07T16:18:13.344 INFO:tasks.rados.rados.0.smithi100.stdout:524: delete oid 18 current snap is 46 2024-04-07T16:18:13.346 INFO:tasks.rados.rados.0.smithi100.stdout:524: done (0 left) 2024-04-07T16:18:13.346 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 39 current snap is 46 2024-04-07T16:18:13.346 INFO:tasks.rados.rados.0.smithi100.stdout:525: seq_num 177 ranges {1417216=573440} 2024-04-07T16:18:13.379 INFO:tasks.rados.rados.0.smithi100.stdout:525: writing smithi10066867-39 from 1417216 to 1990656 tid 1 2024-04-07T16:18:13.379 INFO:tasks.rados.rados.0.smithi100.stdout:526: rollback oid 8 current snap is 46 2024-04-07T16:18:13.379 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 8 to 38 2024-04-07T16:18:13.379 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 43 current snap is 46 2024-04-07T16:18:13.379 INFO:tasks.rados.rados.0.smithi100.stdout:527: seq_num 178 ranges {688128=589824} 2024-04-07T16:18:13.401 INFO:tasks.rados.rados.0.smithi100.stdout:527: writing smithi10066867-43 from 688128 to 1277952 tid 1 2024-04-07T16:18:13.401 INFO:tasks.rados.rados.0.smithi100.stdout:528: delete oid 49 current snap is 46 2024-04-07T16:18:13.401 INFO:tasks.rados.rados.0.smithi100.stdout:526: finishing rollback tid 1 to smithi10066867-8 2024-04-07T16:18:13.401 INFO:tasks.rados.rados.0.smithi100.stdout:526: finishing rollback tid 2 to smithi10066867-8 2024-04-07T16:18:13.401 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 8 v 141 (ObjNum 7 snap 0 seq_num 7) dirty exists 2024-04-07T16:18:13.401 INFO:tasks.rados.rados.0.smithi100.stdout:525: finishing write tid 1 to smithi10066867-39 2024-04-07T16:18:13.401 INFO:tasks.rados.rados.0.smithi100.stdout:525: finishing write tid 2 to smithi10066867-39 2024-04-07T16:18:13.401 INFO:tasks.rados.rados.0.smithi100.stdout:525: finishing write tid 3 to smithi10066867-39 2024-04-07T16:18:13.401 INFO:tasks.rados.rados.0.smithi100.stdout:525: oid 39 updating version 0 to 162 2024-04-07T16:18:13.401 INFO:tasks.rados.rados.0.smithi100.stdout:525: oid 39 version 162 is already newer than 161 2024-04-07T16:18:13.401 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 39 v 162 (ObjNum 177 snap 46 seq_num 177) dirty exists 2024-04-07T16:18:13.402 INFO:tasks.rados.rados.0.smithi100.stdout:525: left oid 39 (ObjNum 177 snap 46 seq_num 177) 2024-04-07T16:18:13.404 INFO:tasks.rados.rados.0.smithi100.stdout:525: done (3 left) 2024-04-07T16:18:13.404 INFO:tasks.rados.rados.0.smithi100.stdout:526: done (2 left) 2024-04-07T16:18:13.404 INFO:tasks.rados.rados.0.smithi100.stdout:528: done (1 left) 2024-04-07T16:18:13.404 INFO:tasks.rados.rados.0.smithi100.stdout:529: delete oid 10 current snap is 46 2024-04-07T16:18:13.405 INFO:tasks.rados.rados.0.smithi100.stdout:527: finishing write tid 1 to smithi10066867-43 2024-04-07T16:18:13.407 INFO:tasks.rados.rados.0.smithi100.stdout:527: finishing write tid 2 to smithi10066867-43 2024-04-07T16:18:13.408 INFO:tasks.rados.rados.0.smithi100.stdout:529: done (1 left) 2024-04-07T16:18:13.408 INFO:tasks.rados.rados.0.smithi100.stdout:530: rmattr oid 29 current snap is 46 2024-04-07T16:18:13.408 INFO:tasks.rados.rados.0.smithi100.stdout:530: done (1 left) 2024-04-07T16:18:13.408 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 37 current snap is 46 2024-04-07T16:18:13.408 INFO:tasks.rados.rados.0.smithi100.stdout:531: seq_num 181 ranges {417792=638976} 2024-04-07T16:18:13.416 INFO:tasks.rados.rados.0.smithi100.stdout:531: writing smithi10066867-37 from 417792 to 1056768 tid 1 2024-04-07T16:18:13.416 INFO:tasks.rados.rados.0.smithi100.stdout:532: read oid 29 snap -1 2024-04-07T16:18:13.416 INFO:tasks.rados.rados.0.smithi100.stdout:532: expect (ObjNum 61 snap 5 seq_num 61) 2024-04-07T16:18:13.416 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 12 current snap is 46 2024-04-07T16:18:13.416 INFO:tasks.rados.rados.0.smithi100.stdout:527: finishing write tid 3 to smithi10066867-43 2024-04-07T16:18:13.416 INFO:tasks.rados.rados.0.smithi100.stdout:527: oid 43 updating version 0 to 70 2024-04-07T16:18:13.416 INFO:tasks.rados.rados.0.smithi100.stdout:527: oid 43 version 70 is already newer than 69 2024-04-07T16:18:13.416 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 43 v 70 (ObjNum 178 snap 46 seq_num 178) dirty exists 2024-04-07T16:18:13.416 INFO:tasks.rados.rados.0.smithi100.stdout:527: left oid 43 (ObjNum 178 snap 46 seq_num 178) 2024-04-07T16:18:13.416 INFO:tasks.rados.rados.0.smithi100.stdout:533: seq_num 182 ranges {163840=655360} 2024-04-07T16:18:13.422 INFO:tasks.rados.rados.0.smithi100.stdout:533: writing smithi10066867-12 from 163840 to 819200 tid 1 2024-04-07T16:18:13.422 INFO:tasks.rados.rados.0.smithi100.stdout:527: done (3 left) 2024-04-07T16:18:13.422 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 36 current snap is 46 2024-04-07T16:18:13.422 INFO:tasks.rados.rados.0.smithi100.stdout:531: finishing write tid 1 to smithi10066867-37 2024-04-07T16:18:13.422 INFO:tasks.rados.rados.0.smithi100.stdout:531: finishing write tid 2 to smithi10066867-37 2024-04-07T16:18:13.422 INFO:tasks.rados.rados.0.smithi100.stdout:534: seq_num 183 ranges {0=671744} 2024-04-07T16:18:13.427 INFO:tasks.rados.rados.0.smithi100.stdout:534: writing smithi10066867-36 from 0 to 671744 tid 1 2024-04-07T16:18:13.427 INFO:tasks.rados.rados.0.smithi100.stdout:535: snap_remove snap 44 2024-04-07T16:18:13.575 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:18:13 smithi144 bash[51952]: debug 2024-04-07T16:18:13.322+0000 7f60520ca200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-07T16:18:13.575 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:18:13 smithi144 bash[51952]: debug 2024-04-07T16:18:13.454+0000 7f60520ca200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-07T16:18:13.575 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:18:13 smithi144 bash[51952]: [07/Apr/2024:16:18:13] ENGINE Bus STARTING 2024-04-07T16:18:13.855 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:18:13 smithi144 bash[51952]: CherryPy Checker: 2024-04-07T16:18:13.855 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:18:13 smithi144 bash[51952]: The Application mounted at '' has an empty config. 2024-04-07T16:18:13.855 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:18:13 smithi144 bash[51952]: [07/Apr/2024:16:18:13] ENGINE Serving on http://:::9283 2024-04-07T16:18:13.855 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:18:13 smithi144 bash[51952]: [07/Apr/2024:16:18:13] ENGINE Bus STARTED 2024-04-07T16:18:13.953 INFO:tasks.rados.rados.0.smithi100.stdout:535: done (4 left) 2024-04-07T16:18:13.953 INFO:tasks.rados.rados.0.smithi100.stdout:536: snap_create 2024-04-07T16:18:13.954 INFO:tasks.rados.rados.0.smithi100.stdout:531: finishing write tid 3 to smithi10066867-37 2024-04-07T16:18:13.954 INFO:tasks.rados.rados.0.smithi100.stdout:531: oid 37 updating version 0 to 165 2024-04-07T16:18:13.954 INFO:tasks.rados.rados.0.smithi100.stdout:531: oid 37 version 165 is already newer than 164 2024-04-07T16:18:13.954 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 37 v 165 (ObjNum 181 snap 46 seq_num 181) dirty exists 2024-04-07T16:18:13.954 INFO:tasks.rados.rados.0.smithi100.stdout:531: left oid 37 (ObjNum 181 snap 46 seq_num 181) 2024-04-07T16:18:13.954 INFO:tasks.rados.rados.0.smithi100.stdout:532: expect (ObjNum 61 snap 5 seq_num 61) 2024-04-07T16:18:14.018 INFO:tasks.rados.rados.0.smithi100.stdout:533: finishing write tid 1 to smithi10066867-12 2024-04-07T16:18:14.019 INFO:tasks.rados.rados.0.smithi100.stdout:533: finishing write tid 2 to smithi10066867-12 2024-04-07T16:18:14.019 INFO:tasks.rados.rados.0.smithi100.stdout:533: finishing write tid 3 to smithi10066867-12 2024-04-07T16:18:14.019 INFO:tasks.rados.rados.0.smithi100.stdout:533: oid 12 updating version 0 to 191 2024-04-07T16:18:14.019 INFO:tasks.rados.rados.0.smithi100.stdout:533: oid 12 version 191 is already newer than 190 2024-04-07T16:18:14.019 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 12 v 191 (ObjNum 182 snap 46 seq_num 182) dirty exists 2024-04-07T16:18:14.019 INFO:tasks.rados.rados.0.smithi100.stdout:533: left oid 12 (ObjNum 182 snap 46 seq_num 182) 2024-04-07T16:18:14.019 INFO:tasks.rados.rados.0.smithi100.stdout:534: finishing write tid 1 to smithi10066867-36 2024-04-07T16:18:14.019 INFO:tasks.rados.rados.0.smithi100.stdout:534: finishing write tid 2 to smithi10066867-36 2024-04-07T16:18:14.019 INFO:tasks.rados.rados.0.smithi100.stdout:534: finishing write tid 3 to smithi10066867-36 2024-04-07T16:18:14.019 INFO:tasks.rados.rados.0.smithi100.stdout:534: oid 36 updating version 0 to 172 2024-04-07T16:18:14.019 INFO:tasks.rados.rados.0.smithi100.stdout:534: oid 36 updating version 172 to 173 2024-04-07T16:18:14.019 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 36 v 173 (ObjNum 183 snap 46 seq_num 183) dirty exists 2024-04-07T16:18:14.019 INFO:tasks.rados.rados.0.smithi100.stdout:534: left oid 36 (ObjNum 183 snap 46 seq_num 183) 2024-04-07T16:18:14.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:13 smithi100 bash[21346]: cluster 2024-04-07T16:18:12.720117+0000 mgr.y (mgr.14553) 230 : cluster 0 pgmap v255: 121 pgs: 121 active+clean; 110 MiB data, 446 MiB used, 715 GiB / 715 GiB avail; 5.5 MiB/s rd, 3.6 MiB/s wr, 38 op/s 2024-04-07T16:18:14.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:13 smithi100 bash[21346]: audit 2024-04-07T16:18:13.481047+0000 mon.b (mon.1) 250 : audit 0 from='mgr.? 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-04-07T16:18:14.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:13 smithi100 bash[21346]: audit 2024-04-07T16:18:13.482010+0000 mon.b (mon.1) 251 : audit 0 from='mgr.? 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-07T16:18:14.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:13 smithi100 bash[21346]: cluster 2024-04-07T16:18:13.484186+0000 mon.a (mon.0) 989 : cluster 0 Standby manager daemon x restarted 2024-04-07T16:18:14.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:13 smithi100 bash[21346]: cluster 2024-04-07T16:18:13.484498+0000 mon.a (mon.0) 990 : cluster 0 Standby manager daemon x started 2024-04-07T16:18:14.188 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:13 smithi100 bash[21346]: audit 2024-04-07T16:18:13.485131+0000 mon.b (mon.1) 252 : audit 0 from='mgr.? 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-04-07T16:18:14.188 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:13 smithi100 bash[21346]: audit 2024-04-07T16:18:13.485655+0000 mon.b (mon.1) 253 : audit 0 from='mgr.? 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-07T16:18:14.188 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:13 smithi100 bash[31660]: cluster 2024-04-07T16:18:12.720117+0000 mgr.y (mgr.14553) 230 : cluster 0 pgmap v255: 121 pgs: 121 active+clean; 110 MiB data, 446 MiB used, 715 GiB / 715 GiB avail; 5.5 MiB/s rd, 3.6 MiB/s wr, 38 op/s 2024-04-07T16:18:14.188 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:13 smithi100 bash[31660]: audit 2024-04-07T16:18:13.481047+0000 mon.b (mon.1) 250 : audit 0 from='mgr.? 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-04-07T16:18:14.188 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:13 smithi100 bash[31660]: audit 2024-04-07T16:18:13.482010+0000 mon.b (mon.1) 251 : audit 0 from='mgr.? 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-07T16:18:14.188 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:13 smithi100 bash[31660]: cluster 2024-04-07T16:18:13.484186+0000 mon.a (mon.0) 989 : cluster 0 Standby manager daemon x restarted 2024-04-07T16:18:14.188 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:13 smithi100 bash[31660]: cluster 2024-04-07T16:18:13.484498+0000 mon.a (mon.0) 990 : cluster 0 Standby manager daemon x started 2024-04-07T16:18:14.188 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:13 smithi100 bash[31660]: audit 2024-04-07T16:18:13.485131+0000 mon.b (mon.1) 252 : audit 0 from='mgr.? 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-04-07T16:18:14.188 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:13 smithi100 bash[31660]: audit 2024-04-07T16:18:13.485655+0000 mon.b (mon.1) 253 : audit 0 from='mgr.? 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-07T16:18:14.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:13 smithi144 bash[23522]: cluster 2024-04-07T16:18:12.720117+0000 mgr.y (mgr.14553) 230 : cluster 0 pgmap v255: 121 pgs: 121 active+clean; 110 MiB data, 446 MiB used, 715 GiB / 715 GiB avail; 5.5 MiB/s rd, 3.6 MiB/s wr, 38 op/s 2024-04-07T16:18:14.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:13 smithi144 bash[23522]: audit 2024-04-07T16:18:13.481047+0000 mon.b (mon.1) 250 : audit 0 from='mgr.? 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-04-07T16:18:14.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:13 smithi144 bash[23522]: audit 2024-04-07T16:18:13.482010+0000 mon.b (mon.1) 251 : audit 0 from='mgr.? 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-07T16:18:14.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:13 smithi144 bash[23522]: cluster 2024-04-07T16:18:13.484186+0000 mon.a (mon.0) 989 : cluster 0 Standby manager daemon x restarted 2024-04-07T16:18:14.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:13 smithi144 bash[23522]: cluster 2024-04-07T16:18:13.484498+0000 mon.a (mon.0) 990 : cluster 0 Standby manager daemon x started 2024-04-07T16:18:14.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:13 smithi144 bash[23522]: audit 2024-04-07T16:18:13.485131+0000 mon.b (mon.1) 252 : audit 0 from='mgr.? 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-04-07T16:18:14.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:13 smithi144 bash[23522]: audit 2024-04-07T16:18:13.485655+0000 mon.b (mon.1) 253 : audit 0 from='mgr.? 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-07T16:18:14.951 INFO:tasks.rados.rados.0.smithi100.stdout:531: done (4 left) 2024-04-07T16:18:14.951 INFO:tasks.rados.rados.0.smithi100.stdout:532: done (3 left) 2024-04-07T16:18:14.951 INFO:tasks.rados.rados.0.smithi100.stdout:533: done (2 left) 2024-04-07T16:18:14.951 INFO:tasks.rados.rados.0.smithi100.stdout:534: done (1 left) 2024-04-07T16:18:14.951 INFO:tasks.rados.rados.0.smithi100.stdout:536: done (0 left) 2024-04-07T16:18:14.951 INFO:tasks.rados.rados.0.smithi100.stdout:537: snap_remove snap 22 2024-04-07T16:18:15.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:14 smithi144 bash[23522]: cluster 2024-04-07T16:18:13.960143+0000 mon.a (mon.0) 991 : cluster 0 osdmap e159: 8 total, 8 up, 8 in 2024-04-07T16:18:15.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:14 smithi144 bash[23522]: cluster 2024-04-07T16:18:13.961580+0000 mon.a (mon.0) 992 : cluster 0 mgrmap e23: y(active, since 5m), standbys: x 2024-04-07T16:18:15.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:14 smithi100 bash[21346]: cluster 2024-04-07T16:18:13.960143+0000 mon.a (mon.0) 991 : cluster 0 osdmap e159: 8 total, 8 up, 8 in 2024-04-07T16:18:15.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:14 smithi100 bash[21346]: cluster 2024-04-07T16:18:13.961580+0000 mon.a (mon.0) 992 : cluster 0 mgrmap e23: y(active, since 5m), standbys: x 2024-04-07T16:18:15.437 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:18:15 smithi100 bash[21601]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:18:15] "GET /metrics HTTP/1.1" 200 36084 "" "Prometheus/2.43.0" 2024-04-07T16:18:15.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:14 smithi100 bash[31660]: cluster 2024-04-07T16:18:13.960143+0000 mon.a (mon.0) 991 : cluster 0 osdmap e159: 8 total, 8 up, 8 in 2024-04-07T16:18:15.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:14 smithi100 bash[31660]: cluster 2024-04-07T16:18:13.961580+0000 mon.a (mon.0) 992 : cluster 0 mgrmap e23: y(active, since 5m), standbys: x 2024-04-07T16:18:15.954 INFO:tasks.rados.rados.0.smithi100.stdout:537: done (0 left) 2024-04-07T16:18:15.955 INFO:tasks.rados.rados.0.smithi100.stdout:538: read oid 16 snap 42 2024-04-07T16:18:15.955 INFO:tasks.rados.rados.0.smithi100.stdout:538: expect (ObjNum 118 snap 29 seq_num 118) 2024-04-07T16:18:15.955 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 2 current snap is 47 2024-04-07T16:18:15.955 INFO:tasks.rados.rados.0.smithi100.stdout:539: seq_num 184 ranges {3522560=688128} 2024-04-07T16:18:15.988 INFO:tasks.rados.rados.0.smithi100.stdout:539: writing smithi10066867-2 from 3522560 to 4210688 tid 1 2024-04-07T16:18:15.988 INFO:tasks.rados.rados.0.smithi100.stdout:540: setattr oid 49 current snap is 47 2024-04-07T16:18:15.989 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 45 current snap is 47 2024-04-07T16:18:15.989 INFO:tasks.rados.rados.0.smithi100.stdout:541: seq_num 185 ranges {2613248=704512} 2024-04-07T16:18:16.017 INFO:tasks.rados.rados.0.smithi100.stdout:541: writing smithi10066867-45 from 2613248 to 3317760 tid 1 2024-04-07T16:18:16.017 INFO:tasks.rados.rados.0.smithi100.stdout:542: read oid 35 snap -1 2024-04-07T16:18:16.017 INFO:tasks.rados.rados.0.smithi100.stdout:542: expect deleted 2024-04-07T16:18:16.017 INFO:tasks.rados.rados.0.smithi100.stdout:543: read oid 14 snap -1 2024-04-07T16:18:16.018 INFO:tasks.rados.rados.0.smithi100.stdout:543: expect (ObjNum 172 snap 46 seq_num 172) 2024-04-07T16:18:16.018 INFO:tasks.rados.rados.0.smithi100.stdout:544: copy_from oid 19 from oid 32 current snap is 47 2024-04-07T16:18:16.018 INFO:tasks.rados.rados.0.smithi100.stdout:539: finishing write tid 1 to smithi10066867-2 2024-04-07T16:18:16.018 INFO:tasks.rados.rados.0.smithi100.stdout:539: finishing write tid 2 to smithi10066867-2 2024-04-07T16:18:16.018 INFO:tasks.rados.rados.0.smithi100.stdout:539: finishing write tid 3 to smithi10066867-2 2024-04-07T16:18:16.018 INFO:tasks.rados.rados.0.smithi100.stdout:539: oid 2 updating version 0 to 158 2024-04-07T16:18:16.018 INFO:tasks.rados.rados.0.smithi100.stdout:539: oid 2 version 158 is already newer than 157 2024-04-07T16:18:16.018 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 2 v 158 (ObjNum 184 snap 47 seq_num 184) dirty exists 2024-04-07T16:18:16.018 INFO:tasks.rados.rados.0.smithi100.stdout:539: left oid 2 (ObjNum 184 snap 47 seq_num 184) 2024-04-07T16:18:16.018 INFO:tasks.rados.rados.0.smithi100.stdout:539: done (6 left) 2024-04-07T16:18:16.018 INFO:tasks.rados.rados.0.smithi100.stdout:545: rollback oid 3 current snap is 47 2024-04-07T16:18:16.018 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 3 to 39 2024-04-07T16:18:16.018 INFO:tasks.rados.rados.0.smithi100.stdout:546: snap_remove snap 38 2024-04-07T16:18:16.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:15 smithi144 bash[23522]: cluster 2024-04-07T16:18:14.720826+0000 mgr.y (mgr.14553) 231 : cluster 0 pgmap v258: 121 pgs: 121 active+clean; 128 MiB data, 499 MiB used, 715 GiB / 715 GiB avail; 7.9 MiB/s rd, 4.1 MiB/s wr, 35 op/s 2024-04-07T16:18:16.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:15 smithi144 bash[23522]: cluster 2024-04-07T16:18:14.965718+0000 mon.a (mon.0) 993 : cluster 0 osdmap e160: 8 total, 8 up, 8 in 2024-04-07T16:18:16.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:15 smithi100 bash[21346]: cluster 2024-04-07T16:18:14.720826+0000 mgr.y (mgr.14553) 231 : cluster 0 pgmap v258: 121 pgs: 121 active+clean; 128 MiB data, 499 MiB used, 715 GiB / 715 GiB avail; 7.9 MiB/s rd, 4.1 MiB/s wr, 35 op/s 2024-04-07T16:18:16.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:15 smithi100 bash[21346]: cluster 2024-04-07T16:18:14.965718+0000 mon.a (mon.0) 993 : cluster 0 osdmap e160: 8 total, 8 up, 8 in 2024-04-07T16:18:16.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:15 smithi100 bash[31660]: cluster 2024-04-07T16:18:14.720826+0000 mgr.y (mgr.14553) 231 : cluster 0 pgmap v258: 121 pgs: 121 active+clean; 128 MiB data, 499 MiB used, 715 GiB / 715 GiB avail; 7.9 MiB/s rd, 4.1 MiB/s wr, 35 op/s 2024-04-07T16:18:16.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:15 smithi100 bash[31660]: cluster 2024-04-07T16:18:14.965718+0000 mon.a (mon.0) 993 : cluster 0 osdmap e160: 8 total, 8 up, 8 in 2024-04-07T16:18:16.977 INFO:tasks.rados.rados.0.smithi100.stdout:546: done (7 left) 2024-04-07T16:18:16.977 INFO:tasks.rados.rados.0.smithi100.stdout:547: read oid 18 snap -1 2024-04-07T16:18:16.977 INFO:tasks.rados.rados.0.smithi100.stdout:547: expect deleted 2024-04-07T16:18:16.977 INFO:tasks.rados.rados.0.smithi100.stdout:548: rollback oid 7 current snap is 47 2024-04-07T16:18:16.977 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 7 to 45 2024-04-07T16:18:16.977 INFO:tasks.rados.rados.0.smithi100.stdout:549: delete oid 48 current snap is 47 2024-04-07T16:18:16.977 INFO:tasks.rados.rados.0.smithi100.stdout:538: expect (ObjNum 118 snap 29 seq_num 118) 2024-04-07T16:18:17.118 INFO:tasks.rados.rados.0.smithi100.stdout:544: finishing copy_from to smithi10066867-19 2024-04-07T16:18:17.118 INFO:tasks.rados.rados.0.smithi100.stdout:544: got expected ENOENT (src dne) 2024-04-07T16:18:17.118 INFO:tasks.rados.rados.0.smithi100.stdout:544: finishing copy_from racing read to smithi10066867-19 2024-04-07T16:18:17.118 INFO:tasks.rados.rados.0.smithi100.stdout:545: finishing rollback tid 0 to smithi10066867-3 2024-04-07T16:18:17.118 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 49 v 198 (ObjNum 177 snap 0 seq_num 0) dirty exists 2024-04-07T16:18:17.119 INFO:tasks.rados.rados.0.smithi100.stdout:545: finishing rollback tid 1 to smithi10066867-3 2024-04-07T16:18:17.119 INFO:tasks.rados.rados.0.smithi100.stdout:545: finishing rollback tid 2 to smithi10066867-3 2024-04-07T16:18:17.119 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 3 v 105 (ObjNum 138 snap 35 seq_num 138) dirty exists 2024-04-07T16:18:17.119 INFO:tasks.rados.rados.0.smithi100.stdout:541: finishing write tid 1 to smithi10066867-45 2024-04-07T16:18:17.119 INFO:tasks.rados.rados.0.smithi100.stdout:541: finishing write tid 2 to smithi10066867-45 2024-04-07T16:18:17.119 INFO:tasks.rados.rados.0.smithi100.stdout:541: finishing write tid 3 to smithi10066867-45 2024-04-07T16:18:17.119 INFO:tasks.rados.rados.0.smithi100.stdout:541: oid 45 updating version 0 to 90 2024-04-07T16:18:17.119 INFO:tasks.rados.rados.0.smithi100.stdout:541: oid 45 updating version 90 to 91 2024-04-07T16:18:17.119 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 45 v 91 (ObjNum 185 snap 47 seq_num 185) dirty exists 2024-04-07T16:18:17.119 INFO:tasks.rados.rados.0.smithi100.stdout:541: left oid 45 (ObjNum 185 snap 47 seq_num 185) 2024-04-07T16:18:17.119 INFO:tasks.rados.rados.0.smithi100.stdout:543: expect (ObjNum 172 snap 46 seq_num 172) 2024-04-07T16:18:17.164 INFO:tasks.rados.rados.0.smithi100.stdout:548: finishing rollback tid 0 to smithi10066867-7 2024-04-07T16:18:17.164 INFO:tasks.rados.rados.0.smithi100.stdout:548: finishing rollback tid 1 to smithi10066867-7 2024-04-07T16:18:17.164 INFO:tasks.rados.rados.0.smithi100.stdout:548: finishing rollback tid 2 to smithi10066867-7 2024-04-07T16:18:17.164 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 7 v 80 (ObjNum 16 snap 0 seq_num 16) dirty exists 2024-04-07T16:18:17.164 INFO:tasks.rados.rados.0.smithi100.stdout:538: done (9 left) 2024-04-07T16:18:17.164 INFO:tasks.rados.rados.0.smithi100.stdout:540: done (8 left) 2024-04-07T16:18:17.164 INFO:tasks.rados.rados.0.smithi100.stdout:541: done (7 left) 2024-04-07T16:18:17.165 INFO:tasks.rados.rados.0.smithi100.stdout:542: done (6 left) 2024-04-07T16:18:17.165 INFO:tasks.rados.rados.0.smithi100.stdout:543: done (5 left) 2024-04-07T16:18:17.165 INFO:tasks.rados.rados.0.smithi100.stdout:544: done (4 left) 2024-04-07T16:18:17.165 INFO:tasks.rados.rados.0.smithi100.stdout:545: done (3 left) 2024-04-07T16:18:17.165 INFO:tasks.rados.rados.0.smithi100.stdout:547: done (2 left) 2024-04-07T16:18:17.165 INFO:tasks.rados.rados.0.smithi100.stdout:548: done (1 left) 2024-04-07T16:18:17.165 INFO:tasks.rados.rados.0.smithi100.stdout:549: done (0 left) 2024-04-07T16:18:17.165 INFO:tasks.rados.rados.0.smithi100.stdout:550: rmattr oid 4 current snap is 47 2024-04-07T16:18:17.165 INFO:tasks.rados.rados.0.smithi100.stdout:551: snap_create 2024-04-07T16:18:17.171 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 4 v 281 (ObjNum 171 snap 44 seq_num 171) dirty exists 2024-04-07T16:18:17.222 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:16 smithi144 bash[23522]: cluster 2024-04-07T16:18:15.966311+0000 mon.a (mon.0) 994 : cluster 0 osdmap e161: 8 total, 8 up, 8 in 2024-04-07T16:18:17.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:16 smithi100 bash[21346]: cluster 2024-04-07T16:18:15.966311+0000 mon.a (mon.0) 994 : cluster 0 osdmap e161: 8 total, 8 up, 8 in 2024-04-07T16:18:17.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:16 smithi100 bash[31660]: cluster 2024-04-07T16:18:15.966311+0000 mon.a (mon.0) 994 : cluster 0 osdmap e161: 8 total, 8 up, 8 in 2024-04-07T16:18:17.846 INFO:tasks.rados.rados.0.smithi100.stdout:550: done (1 left) 2024-04-07T16:18:17.846 INFO:tasks.rados.rados.0.smithi100.stdout:551: done (0 left) 2024-04-07T16:18:17.846 INFO:tasks.rados.rados.0.smithi100.stdout:552: delete oid 37 current snap is 48 2024-04-07T16:18:17.858 INFO:tasks.rados.rados.0.smithi100.stdout:552: done (0 left) 2024-04-07T16:18:17.858 INFO:tasks.rados.rados.0.smithi100.stdout:553: read oid 28 snap 47 2024-04-07T16:18:17.858 INFO:tasks.rados.rados.0.smithi100.stdout:553: expect (ObjNum 140 snap 39 seq_num 140) 2024-04-07T16:18:17.858 INFO:tasks.rados.rados.0.smithi100.stdout:554: read oid 2 snap 43 2024-04-07T16:18:17.858 INFO:tasks.rados.rados.0.smithi100.stdout:554: expect (ObjNum 108 snap 20 seq_num 108) 2024-04-07T16:18:17.858 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 42 current snap is 48 2024-04-07T16:18:17.858 INFO:tasks.rados.rados.0.smithi100.stdout:555: seq_num 188 ranges {950272=753664} 2024-04-07T16:18:17.889 INFO:tasks.rados.rados.0.smithi100.stdout:555: writing smithi10066867-42 from 950272 to 1703936 tid 1 2024-04-07T16:18:17.889 INFO:tasks.rados.rados.0.smithi100.stdout:556: snap_create 2024-04-07T16:18:17.889 INFO:tasks.rados.rados.0.smithi100.stdout:554: expect (ObjNum 108 snap 20 seq_num 108) 2024-04-07T16:18:18.055 INFO:tasks.rados.rados.0.smithi100.stdout:553: expect (ObjNum 140 snap 39 seq_num 140) 2024-04-07T16:18:18.114 INFO:tasks.rados.rados.0.smithi100.stdout:555: finishing write tid 1 to smithi10066867-42 2024-04-07T16:18:18.114 INFO:tasks.rados.rados.0.smithi100.stdout:555: finishing write tid 2 to smithi10066867-42 2024-04-07T16:18:18.114 INFO:tasks.rados.rados.0.smithi100.stdout:555: finishing write tid 3 to smithi10066867-42 2024-04-07T16:18:18.114 INFO:tasks.rados.rados.0.smithi100.stdout:555: oid 42 updating version 0 to 190 2024-04-07T16:18:18.114 INFO:tasks.rados.rados.0.smithi100.stdout:555: oid 42 version 190 is already newer than 189 2024-04-07T16:18:18.114 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 42 v 190 (ObjNum 188 snap 48 seq_num 188) dirty exists 2024-04-07T16:18:18.114 INFO:tasks.rados.rados.0.smithi100.stdout:555: left oid 42 (ObjNum 188 snap 48 seq_num 188) 2024-04-07T16:18:18.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:17 smithi144 bash[23522]: cluster 2024-04-07T16:18:16.721411+0000 mgr.y (mgr.14553) 232 : cluster 0 pgmap v261: 121 pgs: 121 active+clean; 128 MiB data, 499 MiB used, 715 GiB / 715 GiB avail; 7.9 MiB/s rd, 4.1 MiB/s wr, 35 op/s 2024-04-07T16:18:18.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:17 smithi144 bash[23522]: cluster 2024-04-07T16:18:16.991339+0000 mon.a (mon.0) 995 : cluster 0 osdmap e162: 8 total, 8 up, 8 in 2024-04-07T16:18:18.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:17 smithi144 bash[23522]: cluster 2024-04-07T16:18:17.860559+0000 mon.a (mon.0) 996 : cluster 0 osdmap e163: 8 total, 8 up, 8 in 2024-04-07T16:18:18.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:17 smithi100 bash[21346]: cluster 2024-04-07T16:18:16.721411+0000 mgr.y (mgr.14553) 232 : cluster 0 pgmap v261: 121 pgs: 121 active+clean; 128 MiB data, 499 MiB used, 715 GiB / 715 GiB avail; 7.9 MiB/s rd, 4.1 MiB/s wr, 35 op/s 2024-04-07T16:18:18.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:17 smithi100 bash[21346]: cluster 2024-04-07T16:18:16.991339+0000 mon.a (mon.0) 995 : cluster 0 osdmap e162: 8 total, 8 up, 8 in 2024-04-07T16:18:18.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:17 smithi100 bash[21346]: cluster 2024-04-07T16:18:17.860559+0000 mon.a (mon.0) 996 : cluster 0 osdmap e163: 8 total, 8 up, 8 in 2024-04-07T16:18:18.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:17 smithi100 bash[31660]: cluster 2024-04-07T16:18:16.721411+0000 mgr.y (mgr.14553) 232 : cluster 0 pgmap v261: 121 pgs: 121 active+clean; 128 MiB data, 499 MiB used, 715 GiB / 715 GiB avail; 7.9 MiB/s rd, 4.1 MiB/s wr, 35 op/s 2024-04-07T16:18:18.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:17 smithi100 bash[31660]: cluster 2024-04-07T16:18:16.991339+0000 mon.a (mon.0) 995 : cluster 0 osdmap e162: 8 total, 8 up, 8 in 2024-04-07T16:18:18.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:17 smithi100 bash[31660]: cluster 2024-04-07T16:18:17.860559+0000 mon.a (mon.0) 996 : cluster 0 osdmap e163: 8 total, 8 up, 8 in 2024-04-07T16:18:18.849 INFO:tasks.rados.rados.0.smithi100.stdout:553: done (3 left) 2024-04-07T16:18:18.850 INFO:tasks.rados.rados.0.smithi100.stdout:554: done (2 left) 2024-04-07T16:18:18.850 INFO:tasks.rados.rados.0.smithi100.stdout:555: done (1 left) 2024-04-07T16:18:18.850 INFO:tasks.rados.rados.0.smithi100.stdout:556: done (0 left) 2024-04-07T16:18:18.850 INFO:tasks.rados.rados.0.smithi100.stdout:557: setattr oid 5 current snap is 49 2024-04-07T16:18:18.853 INFO:tasks.rados.rados.0.smithi100.stdout:558: rollback oid 50 current snap is 49 2024-04-07T16:18:18.853 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 50 to 42 2024-04-07T16:18:18.853 INFO:tasks.rados.rados.0.smithi100.stdout:559: setattr oid 47 current snap is 49 2024-04-07T16:18:18.856 INFO:tasks.rados.rados.0.smithi100.stdout:558: finishing rollback tid 0 to smithi10066867-50 2024-04-07T16:18:18.858 INFO:tasks.rados.rados.0.smithi100.stdout:558: finishing rollback tid 1 to smithi10066867-50 2024-04-07T16:18:18.858 INFO:tasks.rados.rados.0.smithi100.stdout:558: finishing rollback tid 2 to smithi10066867-50 2024-04-07T16:18:18.858 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 50 v 99 (ObjNum 49 snap 0 seq_num 49) dirty exists 2024-04-07T16:18:18.859 INFO:tasks.rados.rados.0.smithi100.stdout:558: done (2 left) 2024-04-07T16:18:18.859 INFO:tasks.rados.rados.0.smithi100.stdout:560: rollback oid 19 current snap is 49 2024-04-07T16:18:18.859 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 19 to 46 2024-04-07T16:18:18.859 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 40 current snap is 49 2024-04-07T16:18:18.859 INFO:tasks.rados.rados.0.smithi100.stdout:561: seq_num 189 ranges {3891200=770048} 2024-04-07T16:18:18.919 INFO:tasks.rados.rados.0.smithi100.stdout:561: writing smithi10066867-40 from 3891200 to 4661248 tid 1 2024-04-07T16:18:18.919 INFO:tasks.rados.rados.0.smithi100.stdout:562: snap_remove snap 39 2024-04-07T16:18:19.493 INFO:teuthology.orchestra.run.smithi100.stdout:true 2024-04-07T16:18:19.859 INFO:tasks.rados.rados.0.smithi100.stdout:562: done (4 left) 2024-04-07T16:18:19.860 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 42 current snap is 49 2024-04-07T16:18:19.860 INFO:tasks.rados.rados.0.smithi100.stdout:563: seq_num 190 ranges {1703936=786432} 2024-04-07T16:18:19.875 INFO:tasks.rados.rados.0.smithi100.stdout:563: writing smithi10066867-42 from 1703936 to 2490368 tid 1 2024-04-07T16:18:19.875 INFO:tasks.rados.rados.0.smithi100.stdout:564: rollback oid 3 current snap is 49 2024-04-07T16:18:19.875 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 3 to 46 2024-04-07T16:18:19.875 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 47 v 70 (ObjNum 149 snap 41 seq_num 149) dirty exists 2024-04-07T16:18:19.875 INFO:tasks.rados.rados.0.smithi100.stdout:560: finishing rollback tid 1 to smithi10066867-19 2024-04-07T16:18:19.875 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 19 v 153 (ObjNum 65 snap 0 seq_num 1650020444) dirty dne 2024-04-07T16:18:19.875 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 5 v 207 (ObjNum 129 snap 0 seq_num 2702547176) dirty exists 2024-04-07T16:18:19.876 INFO:tasks.rados.rados.0.smithi100.stdout:561: finishing write tid 1 to smithi10066867-40 2024-04-07T16:18:19.876 INFO:tasks.rados.rados.0.smithi100.stdout:561: finishing write tid 2 to smithi10066867-40 2024-04-07T16:18:19.876 INFO:tasks.rados.rados.0.smithi100.stdout:561: finishing write tid 3 to smithi10066867-40 2024-04-07T16:18:19.876 INFO:tasks.rados.rados.0.smithi100.stdout:561: oid 40 updating version 0 to 170 2024-04-07T16:18:19.876 INFO:tasks.rados.rados.0.smithi100.stdout:561: oid 40 updating version 170 to 171 2024-04-07T16:18:19.876 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 40 v 171 (ObjNum 189 snap 49 seq_num 189) dirty exists 2024-04-07T16:18:19.876 INFO:tasks.rados.rados.0.smithi100.stdout:561: left oid 40 (ObjNum 189 snap 49 seq_num 189) 2024-04-07T16:18:19.876 INFO:tasks.rados.rados.0.smithi100.stdout:557: done (5 left) 2024-04-07T16:18:19.876 INFO:tasks.rados.rados.0.smithi100.stdout:559: done (4 left) 2024-04-07T16:18:19.876 INFO:tasks.rados.rados.0.smithi100.stdout:560: done (3 left) 2024-04-07T16:18:19.876 INFO:tasks.rados.rados.0.smithi100.stdout:561: done (2 left) 2024-04-07T16:18:19.876 INFO:tasks.rados.rados.0.smithi100.stdout:565: rmattr oid 40 current snap is 49 2024-04-07T16:18:19.876 INFO:tasks.rados.rados.0.smithi100.stdout:566: read oid 18 snap -1 2024-04-07T16:18:19.876 INFO:tasks.rados.rados.0.smithi100.stdout:566: expect deleted 2024-04-07T16:18:19.876 INFO:tasks.rados.rados.0.smithi100.stdout:567: snap_create 2024-04-07T16:18:19.877 INFO:teuthology.orchestra.run.smithi100.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-07T16:18:19.877 INFO:teuthology.orchestra.run.smithi100.stdout:alertmanager.a smithi100 *:9093,9094 running (4m) 4m ago 5m 13.7M - 0.25.0 c8568f914cd2 5143075e7d26 2024-04-07T16:18:19.877 INFO:teuthology.orchestra.run.smithi100.stdout:grafana.a smithi144 *:3000 running (4m) 9s ago 4m 78.7M - 9.4.7 954c08fa6188 7180c0708acd 2024-04-07T16:18:19.877 INFO:teuthology.orchestra.run.smithi100.stdout:mds.a smithi100 running (6m) 4m ago 6m 12.4M - 18.2.2-1171-g267c8ff5 8a4a01e7186c 92593b0c28ae 2024-04-07T16:18:19.877 INFO:teuthology.orchestra.run.smithi100.stdout:mds.b smithi144 running (6m) 9s ago 6m 16.0M - 18.2.2-1171-g267c8ff5 8a4a01e7186c 3df2ee3e17a4 2024-04-07T16:18:19.877 INFO:teuthology.orchestra.run.smithi100.stdout:mgr.x smithi144 *:8443,9283,8765 running (15s) 9s ago 13m 135M - 19.0.0-2785-gadce92d0 c0ecd4226ac9 45451b348d34 2024-04-07T16:18:19.877 INFO:teuthology.orchestra.run.smithi100.stdout:mgr.y smithi100 *:9283,8765 running (15m) 4m ago 15m 484M - 18.2.2-1171-g267c8ff5 8a4a01e7186c da1d683b9737 2024-04-07T16:18:19.877 INFO:teuthology.orchestra.run.smithi100.stdout:mon.a smithi100 running (15m) 4m ago 15m 46.3M 2048M 18.2.2-1171-g267c8ff5 8a4a01e7186c ec8833f59ea0 2024-04-07T16:18:19.877 INFO:teuthology.orchestra.run.smithi100.stdout:mon.b smithi144 running (13m) 9s ago 13m 43.1M 2048M 18.2.2-1171-g267c8ff5 8a4a01e7186c 8dd687691b2e 2024-04-07T16:18:19.877 INFO:teuthology.orchestra.run.smithi100.stdout:mon.c smithi100 running (13m) 4m ago 13m 37.7M 2048M 18.2.2-1171-g267c8ff5 8a4a01e7186c a6de0151a3ba 2024-04-07T16:18:19.877 INFO:teuthology.orchestra.run.smithi100.stdout:node-exporter.a smithi100 *:9100 running (5m) 4m ago 5m 3116k - 1.5.0 0da6a335fe13 4bce264c1b26 2024-04-07T16:18:19.878 INFO:teuthology.orchestra.run.smithi100.stdout:node-exporter.b smithi144 *:9100 running (5m) 9s ago 5m 8152k - 1.5.0 0da6a335fe13 b6963413c560 2024-04-07T16:18:19.878 INFO:teuthology.orchestra.run.smithi100.stdout:osd.0 smithi100 running (12m) 4m ago 12m 45.7M 3036M 18.2.2-1171-g267c8ff5 8a4a01e7186c 188deb247979 2024-04-07T16:18:19.878 INFO:teuthology.orchestra.run.smithi100.stdout:osd.1 smithi100 running (11m) 4m ago 11m 44.6M 3036M 18.2.2-1171-g267c8ff5 8a4a01e7186c 7f57ca90894c 2024-04-07T16:18:19.878 INFO:teuthology.orchestra.run.smithi100.stdout:osd.2 smithi100 running (11m) 4m ago 11m 52.2M 3036M 18.2.2-1171-g267c8ff5 8a4a01e7186c f3912ddcc8cc 2024-04-07T16:18:19.878 INFO:teuthology.orchestra.run.smithi100.stdout:osd.3 smithi100 running (10m) 4m ago 10m 41.7M 3036M 18.2.2-1171-g267c8ff5 8a4a01e7186c b41004090c63 2024-04-07T16:18:19.878 INFO:teuthology.orchestra.run.smithi100.stdout:osd.4 smithi144 running (9m) 9s ago 9m 84.8M 3036M 18.2.2-1171-g267c8ff5 8a4a01e7186c 8c73b6b4dc13 2024-04-07T16:18:19.878 INFO:teuthology.orchestra.run.smithi100.stdout:osd.5 smithi144 running (8m) 9s ago 8m 87.3M 3036M 18.2.2-1171-g267c8ff5 8a4a01e7186c 3bfa0cf6f6cb 2024-04-07T16:18:19.878 INFO:teuthology.orchestra.run.smithi100.stdout:osd.6 smithi144 running (7m) 9s ago 7m 58.8M 3036M 18.2.2-1171-g267c8ff5 8a4a01e7186c 32b7f95d77f0 2024-04-07T16:18:19.878 INFO:teuthology.orchestra.run.smithi100.stdout:osd.7 smithi144 running (6m) 9s ago 6m 76.7M 3036M 18.2.2-1171-g267c8ff5 8a4a01e7186c 45f7709e576a 2024-04-07T16:18:19.878 INFO:teuthology.orchestra.run.smithi100.stdout:prometheus.a smithi144 *:9095 running (4m) 9s ago 5m 42.6M - 2.43.0 a07b618ecd1d e456da14a41a 2024-04-07T16:18:19.881 INFO:tasks.rados.rados.0.smithi100.stdout:564: finishing rollback tid 0 to smithi10066867-3 2024-04-07T16:18:19.884 INFO:tasks.rados.rados.0.smithi100.stdout:564: finishing rollback tid 1 to smithi10066867-3 2024-04-07T16:18:19.886 INFO:tasks.rados.rados.0.smithi100.stdout:564: finishing rollback tid 2 to smithi10066867-3 2024-04-07T16:18:19.886 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 3 v 113 (ObjNum 80 snap 13 seq_num 80) dirty exists 2024-04-07T16:18:19.887 INFO:tasks.rados.rados.0.smithi100.stdout:563: finishing write tid 1 to smithi10066867-42 2024-04-07T16:18:19.887 INFO:tasks.rados.rados.0.smithi100.stdout:563: finishing write tid 2 to smithi10066867-42 2024-04-07T16:18:19.888 INFO:tasks.rados.rados.0.smithi100.stdout:563: finishing write tid 3 to smithi10066867-42 2024-04-07T16:18:19.888 INFO:tasks.rados.rados.0.smithi100.stdout:563: oid 42 updating version 0 to 197 2024-04-07T16:18:19.888 INFO:tasks.rados.rados.0.smithi100.stdout:563: oid 42 version 197 is already newer than 196 2024-04-07T16:18:19.888 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 42 v 197 (ObjNum 190 snap 49 seq_num 190) dirty exists 2024-04-07T16:18:19.888 INFO:tasks.rados.rados.0.smithi100.stdout:563: left oid 42 (ObjNum 190 snap 49 seq_num 190) 2024-04-07T16:18:19.889 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 40 v 172 (ObjNum 189 snap 49 seq_num 189) dirty exists 2024-04-07T16:18:20.105 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:19 smithi144 bash[23522]: cluster 2024-04-07T16:18:18.722194+0000 mgr.y (mgr.14553) 233 : cluster 0 pgmap v264: 121 pgs: 121 active+clean; 127 MiB data, 511 MiB used, 715 GiB / 715 GiB avail; 858 KiB/s rd, 172 KiB/s wr, 1 op/s 2024-04-07T16:18:20.105 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:19 smithi144 bash[23522]: cluster 2024-04-07T16:18:18.864554+0000 mon.a (mon.0) 997 : cluster 0 osdmap e164: 8 total, 8 up, 8 in 2024-04-07T16:18:20.105 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:19 smithi144 bash[23522]: audit 2024-04-07T16:18:19.493790+0000 mgr.y (mgr.14553) 234 : audit 0 from='client.15081 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:18:20.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:19 smithi100 bash[21346]: cluster 2024-04-07T16:18:18.722194+0000 mgr.y (mgr.14553) 233 : cluster 0 pgmap v264: 121 pgs: 121 active+clean; 127 MiB data, 511 MiB used, 715 GiB / 715 GiB avail; 858 KiB/s rd, 172 KiB/s wr, 1 op/s 2024-04-07T16:18:20.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:19 smithi100 bash[21346]: cluster 2024-04-07T16:18:18.864554+0000 mon.a (mon.0) 997 : cluster 0 osdmap e164: 8 total, 8 up, 8 in 2024-04-07T16:18:20.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:19 smithi100 bash[21346]: audit 2024-04-07T16:18:19.493790+0000 mgr.y (mgr.14553) 234 : audit 0 from='client.15081 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:18:20.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:19 smithi100 bash[31660]: cluster 2024-04-07T16:18:18.722194+0000 mgr.y (mgr.14553) 233 : cluster 0 pgmap v264: 121 pgs: 121 active+clean; 127 MiB data, 511 MiB used, 715 GiB / 715 GiB avail; 858 KiB/s rd, 172 KiB/s wr, 1 op/s 2024-04-07T16:18:20.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:19 smithi100 bash[31660]: cluster 2024-04-07T16:18:18.864554+0000 mon.a (mon.0) 997 : cluster 0 osdmap e164: 8 total, 8 up, 8 in 2024-04-07T16:18:20.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:19 smithi100 bash[31660]: audit 2024-04-07T16:18:19.493790+0000 mgr.y (mgr.14553) 234 : audit 0 from='client.15081 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:18:20.305 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-04-07T16:18:20.305 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": { 2024-04-07T16:18:20.305 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph version 18.2.2-1171-g267c8ff5 (267c8ff5b1ff5815581b3b2381d19360e081b37b) reef (stable)": 3 2024-04-07T16:18:20.305 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-07T16:18:20.305 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": { 2024-04-07T16:18:20.305 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph version 18.2.2-1171-g267c8ff5 (267c8ff5b1ff5815581b3b2381d19360e081b37b) reef (stable)": 1, 2024-04-07T16:18:20.305 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph version 19.0.0-2785-gadce92d0 (adce92d056533f22dc1ee37e3141d098b046ec1b) squid (dev)": 1 2024-04-07T16:18:20.306 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-07T16:18:20.306 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": { 2024-04-07T16:18:20.306 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph version 18.2.2-1171-g267c8ff5 (267c8ff5b1ff5815581b3b2381d19360e081b37b) reef (stable)": 8 2024-04-07T16:18:20.306 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-07T16:18:20.306 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": { 2024-04-07T16:18:20.306 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph version 18.2.2-1171-g267c8ff5 (267c8ff5b1ff5815581b3b2381d19360e081b37b) reef (stable)": 2 2024-04-07T16:18:20.306 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-07T16:18:20.306 INFO:teuthology.orchestra.run.smithi100.stdout: "overall": { 2024-04-07T16:18:20.306 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph version 18.2.2-1171-g267c8ff5 (267c8ff5b1ff5815581b3b2381d19360e081b37b) reef (stable)": 14, 2024-04-07T16:18:20.306 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph version 19.0.0-2785-gadce92d0 (adce92d056533f22dc1ee37e3141d098b046ec1b) squid (dev)": 1 2024-04-07T16:18:20.306 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-04-07T16:18:20.306 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-04-07T16:18:20.621 INFO:tasks.rados.rados.0.smithi100.stdout:563: done (4 left) 2024-04-07T16:18:20.621 INFO:tasks.rados.rados.0.smithi100.stdout:564: done (3 left) 2024-04-07T16:18:20.621 INFO:tasks.rados.rados.0.smithi100.stdout:565: done (2 left) 2024-04-07T16:18:20.621 INFO:tasks.rados.rados.0.smithi100.stdout:566: done (1 left) 2024-04-07T16:18:20.621 INFO:tasks.rados.rados.0.smithi100.stdout:567: done (0 left) 2024-04-07T16:18:20.621 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 3 current snap is 50 2024-04-07T16:18:20.621 INFO:tasks.rados.rados.0.smithi100.stdout:568: seq_num 191 ranges {1622016=802816} 2024-04-07T16:18:20.634 INFO:tasks.rados.rados.0.smithi100.stdout:568: writing smithi10066867-3 from 1622016 to 2424832 tid 1 2024-04-07T16:18:20.635 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 49 current snap is 50 2024-04-07T16:18:20.635 INFO:tasks.rados.rados.0.smithi100.stdout:569: seq_num 192 ranges {0=417792,417792=401408} 2024-04-07T16:18:20.636 INFO:tasks.rados.rados.0.smithi100.stdout:569: writing smithi10066867-49 from 0 to 417792 tid 1 2024-04-07T16:18:20.639 INFO:tasks.rados.rados.0.smithi100.stdout:569: writing smithi10066867-49 from 417792 to 819200 tid 2 2024-04-07T16:18:20.640 INFO:tasks.rados.rados.0.smithi100.stdout:570: read oid 32 snap 47 2024-04-07T16:18:20.640 INFO:tasks.rados.rados.0.smithi100.stdout:570: expect deleted 2024-04-07T16:18:20.640 INFO:tasks.rados.rados.0.smithi100.stdout:571: rollback oid 50 current snap is 50 2024-04-07T16:18:20.640 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 50 to 49 2024-04-07T16:18:20.640 INFO:tasks.rados.rados.0.smithi100.stdout:572: snap_create 2024-04-07T16:18:20.645 INFO:tasks.rados.rados.0.smithi100.stdout:569: finishing write tid 1 to smithi10066867-49 2024-04-07T16:18:20.646 INFO:tasks.rados.rados.0.smithi100.stdout:569: finishing write tid 2 to smithi10066867-49 2024-04-07T16:18:20.647 INFO:tasks.rados.rados.0.smithi100.stdout:569: finishing write tid 3 to smithi10066867-49 2024-04-07T16:18:20.647 INFO:tasks.rados.rados.0.smithi100.stdout:568: finishing write tid 1 to smithi10066867-3 2024-04-07T16:18:20.647 INFO:tasks.rados.rados.0.smithi100.stdout:568: finishing write tid 2 to smithi10066867-3 2024-04-07T16:18:20.647 INFO:tasks.rados.rados.0.smithi100.stdout:571: finishing rollback tid 0 to smithi10066867-50 2024-04-07T16:18:20.648 INFO:tasks.rados.rados.0.smithi100.stdout:571: finishing rollback tid 1 to smithi10066867-50 2024-04-07T16:18:20.648 INFO:tasks.rados.rados.0.smithi100.stdout:569: finishing write tid 4 to smithi10066867-49 2024-04-07T16:18:20.648 INFO:tasks.rados.rados.0.smithi100.stdout:569: oid 49 updating version 0 to 215 2024-04-07T16:18:20.648 INFO:tasks.rados.rados.0.smithi100.stdout:569: oid 49 version 215 is already newer than 214 2024-04-07T16:18:20.648 INFO:tasks.rados.rados.0.smithi100.stdout:569: oid 49 version 215 is already newer than 213 2024-04-07T16:18:20.648 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 49 v 215 (ObjNum 192 snap 50 seq_num 192) dirty exists 2024-04-07T16:18:20.648 INFO:tasks.rados.rados.0.smithi100.stdout:569: left oid 49 (ObjNum 192 snap 50 seq_num 192) 2024-04-07T16:18:20.649 INFO:tasks.rados.rados.0.smithi100.stdout:571: finishing rollback tid 2 to smithi10066867-50 2024-04-07T16:18:20.649 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 50 v 109 (ObjNum 49 snap 0 seq_num 49) dirty exists 2024-04-07T16:18:20.651 INFO:tasks.rados.rados.0.smithi100.stdout:568: finishing write tid 3 to smithi10066867-3 2024-04-07T16:18:20.651 INFO:tasks.rados.rados.0.smithi100.stdout:568: oid 3 updating version 0 to 116 2024-04-07T16:18:20.651 INFO:tasks.rados.rados.0.smithi100.stdout:568: oid 3 version 116 is already newer than 115 2024-04-07T16:18:20.651 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 3 v 116 (ObjNum 191 snap 50 seq_num 191) dirty exists 2024-04-07T16:18:20.651 INFO:tasks.rados.rados.0.smithi100.stdout:568: left oid 3 (ObjNum 191 snap 50 seq_num 191) 2024-04-07T16:18:20.937 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:18:20 smithi100 bash[21601]: ignoring --setuser ceph since I am not root 2024-04-07T16:18:20.937 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:18:20 smithi100 bash[21601]: ignoring --setgroup ceph since I am not root 2024-04-07T16:18:20.937 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:18:20 smithi100 bash[21601]: debug 2024-04-07T16:18:20.840+0000 7fcfdfc22200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-07T16:18:20.937 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:20 smithi100 bash[21346]: cluster 2024-04-07T16:18:19.872482+0000 mon.a (mon.0) 998 : cluster 0 osdmap e165: 8 total, 8 up, 8 in 2024-04-07T16:18:20.938 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:20 smithi100 bash[21346]: audit 2024-04-07T16:18:20.320736+0000 mon.a (mon.0) 999 : audit 0 from='client.? 172.21.15.100:0/2055622250' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:18:20.938 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:20 smithi100 bash[21346]: audit 2024-04-07T16:18:20.488089+0000 mon.a (mon.0) 1000 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:18:20.938 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:20 smithi100 bash[21346]: audit 2024-04-07T16:18:20.504013+0000 mon.b (mon.1) 254 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:18:20.938 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:20 smithi100 bash[21346]: audit 2024-04-07T16:18:20.504071+0000 mon.a (mon.0) 1001 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:18:20.938 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:20 smithi100 bash[21346]: audit 2024-04-07T16:18:20.506022+0000 mon.b (mon.1) 255 : audit 1 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:18:20.938 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:20 smithi100 bash[21346]: audit 2024-04-07T16:18:20.521308+0000 mon.a (mon.0) 1002 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:18:20.938 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:20 smithi100 bash[21346]: audit 2024-04-07T16:18:20.601079+0000 mon.b (mon.1) 256 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:18:20.938 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:20 smithi100 bash[21346]: audit 2024-04-07T16:18:20.612281+0000 mon.b (mon.1) 257 : audit 1 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mgr fail", "who": "y"}]: dispatch 2024-04-07T16:18:20.938 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:20 smithi100 bash[21346]: audit 2024-04-07T16:18:20.617607+0000 mon.a (mon.0) 1003 : audit 1 from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "mgr fail", "who": "y"}]: dispatch 2024-04-07T16:18:20.938 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:20 smithi100 bash[21346]: cluster 2024-04-07T16:18:20.618684+0000 mon.a (mon.0) 1004 : cluster 1 Activating manager daemon x 2024-04-07T16:18:20.938 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:20 smithi100 bash[21346]: audit 2024-04-07T16:18:20.633684+0000 mon.b (mon.1) 258 : audit 0 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:18:20.938 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:20 smithi100 bash[21346]: audit 2024-04-07T16:18:20.634221+0000 mon.b (mon.1) 259 : audit 0 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:18:20.938 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:20 smithi100 bash[21346]: audit 2024-04-07T16:18:20.634734+0000 mon.b (mon.1) 260 : audit 0 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:18:20.939 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:20 smithi100 bash[21346]: audit 2024-04-07T16:18:20.635246+0000 mon.b (mon.1) 261 : audit 0 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-07T16:18:20.939 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:20 smithi100 bash[21346]: audit 2024-04-07T16:18:20.635958+0000 mon.b (mon.1) 262 : audit 0 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-07T16:18:20.939 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:20 smithi100 bash[21346]: cluster 2024-04-07T16:18:20.636358+0000 mon.a (mon.0) 1005 : cluster 0 osdmap e166: 8 total, 8 up, 8 in 2024-04-07T16:18:20.939 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:20 smithi100 bash[21346]: audit 2024-04-07T16:18:20.636706+0000 mon.b (mon.1) 263 : audit 0 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-07T16:18:20.939 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:20 smithi100 bash[21346]: audit 2024-04-07T16:18:20.636798+0000 mon.a (mon.0) 1006 : audit 1 from='mgr.14553 ' entity='mgr.y' cmd='[{"prefix": "mgr fail", "who": "y"}]': finished 2024-04-07T16:18:20.939 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:20 smithi100 bash[21346]: cluster 2024-04-07T16:18:20.636861+0000 mon.a (mon.0) 1007 : cluster 0 mgrmap e24: x(active, starting, since 0.0181805s) 2024-04-07T16:18:20.939 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:20 smithi100 bash[21346]: audit 2024-04-07T16:18:20.637341+0000 mon.b (mon.1) 264 : audit 0 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-07T16:18:20.939 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:20 smithi100 bash[21346]: audit 2024-04-07T16:18:20.638079+0000 mon.b (mon.1) 265 : audit 0 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-07T16:18:20.939 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:20 smithi100 bash[21346]: audit 2024-04-07T16:18:20.638778+0000 mon.b (mon.1) 266 : audit 0 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-07T16:18:20.939 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:20 smithi100 bash[21346]: audit 2024-04-07T16:18:20.639548+0000 mon.b (mon.1) 267 : audit 0 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-07T16:18:20.939 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:20 smithi100 bash[21346]: audit 2024-04-07T16:18:20.640287+0000 mon.b (mon.1) 268 : audit 0 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-07T16:18:20.939 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:20 smithi100 bash[21346]: audit 2024-04-07T16:18:20.641024+0000 mon.b (mon.1) 269 : audit 0 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-07T16:18:20.939 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:20 smithi100 bash[21346]: audit 2024-04-07T16:18:20.641723+0000 mon.b (mon.1) 270 : audit 0 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-07T16:18:20.939 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:20 smithi100 bash[21346]: audit 2024-04-07T16:18:20.642452+0000 mon.b (mon.1) 271 : audit 0 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-07T16:18:20.939 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:20 smithi100 bash[21346]: audit 2024-04-07T16:18:20.643223+0000 mon.b (mon.1) 272 : audit 0 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-07T16:18:20.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:20 smithi100 bash[21346]: audit 2024-04-07T16:18:20.644042+0000 mon.b (mon.1) 273 : audit 0 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-07T16:18:20.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:20 smithi100 bash[21346]: audit 2024-04-07T16:18:20.645426+0000 mon.b (mon.1) 274 : audit 0 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-07T16:18:20.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:20 smithi100 bash[21346]: cluster 2024-04-07T16:18:20.692311+0000 mon.a (mon.0) 1008 : cluster 1 Manager daemon x is now available 2024-04-07T16:18:20.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:20 smithi100 bash[21346]: audit 2024-04-07T16:18:20.734021+0000 mon.b (mon.1) 275 : audit 0 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:18:20.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:20 smithi100 bash[21346]: audit 2024-04-07T16:18:20.766090+0000 mon.b (mon.1) 276 : audit 0 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:18:20.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:20 smithi100 bash[21346]: audit 2024-04-07T16:18:20.769222+0000 mon.b (mon.1) 277 : audit 1 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-07T16:18:20.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:20 smithi100 bash[21346]: audit 2024-04-07T16:18:20.774108+0000 mon.a (mon.0) 1009 : audit 1 from='mgr.24553 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-07T16:18:20.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:20 smithi100 bash[21346]: audit 2024-04-07T16:18:20.806308+0000 mon.b (mon.1) 278 : audit 1 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-07T16:18:20.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:20 smithi100 bash[21346]: audit 2024-04-07T16:18:20.811331+0000 mon.a (mon.0) 1010 : audit 1 from='mgr.24553 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-07T16:18:20.940 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:20 smithi100 bash[31660]: cluster 2024-04-07T16:18:19.872482+0000 mon.a (mon.0) 998 : cluster 0 osdmap e165: 8 total, 8 up, 8 in 2024-04-07T16:18:20.940 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:20 smithi100 bash[31660]: audit 2024-04-07T16:18:20.320736+0000 mon.a (mon.0) 999 : audit 0 from='client.? 172.21.15.100:0/2055622250' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:18:20.941 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:20 smithi100 bash[31660]: audit 2024-04-07T16:18:20.488089+0000 mon.a (mon.0) 1000 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:18:20.941 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:20 smithi100 bash[31660]: audit 2024-04-07T16:18:20.504013+0000 mon.b (mon.1) 254 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:18:20.941 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:20 smithi100 bash[31660]: audit 2024-04-07T16:18:20.504071+0000 mon.a (mon.0) 1001 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:18:20.941 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:20 smithi100 bash[31660]: audit 2024-04-07T16:18:20.506022+0000 mon.b (mon.1) 255 : audit 1 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:18:20.941 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:20 smithi100 bash[31660]: audit 2024-04-07T16:18:20.521308+0000 mon.a (mon.0) 1002 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:18:20.941 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:20 smithi100 bash[31660]: audit 2024-04-07T16:18:20.601079+0000 mon.b (mon.1) 256 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:18:20.941 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:20 smithi100 bash[31660]: audit 2024-04-07T16:18:20.612281+0000 mon.b (mon.1) 257 : audit 1 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mgr fail", "who": "y"}]: dispatch 2024-04-07T16:18:20.941 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:20 smithi100 bash[31660]: audit 2024-04-07T16:18:20.617607+0000 mon.a (mon.0) 1003 : audit 1 from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "mgr fail", "who": "y"}]: dispatch 2024-04-07T16:18:20.941 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:20 smithi100 bash[31660]: cluster 2024-04-07T16:18:20.618684+0000 mon.a (mon.0) 1004 : cluster 1 Activating manager daemon x 2024-04-07T16:18:20.941 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:20 smithi100 bash[31660]: audit 2024-04-07T16:18:20.633684+0000 mon.b (mon.1) 258 : audit 0 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:18:20.941 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:20 smithi100 bash[31660]: audit 2024-04-07T16:18:20.634221+0000 mon.b (mon.1) 259 : audit 0 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:18:20.941 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:20 smithi100 bash[31660]: audit 2024-04-07T16:18:20.634734+0000 mon.b (mon.1) 260 : audit 0 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:18:20.941 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:20 smithi100 bash[31660]: audit 2024-04-07T16:18:20.635246+0000 mon.b (mon.1) 261 : audit 0 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-07T16:18:20.941 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:20 smithi100 bash[31660]: audit 2024-04-07T16:18:20.635958+0000 mon.b (mon.1) 262 : audit 0 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-07T16:18:20.942 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:20 smithi100 bash[31660]: cluster 2024-04-07T16:18:20.636358+0000 mon.a (mon.0) 1005 : cluster 0 osdmap e166: 8 total, 8 up, 8 in 2024-04-07T16:18:20.942 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:20 smithi100 bash[31660]: audit 2024-04-07T16:18:20.636706+0000 mon.b (mon.1) 263 : audit 0 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-07T16:18:20.942 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:20 smithi100 bash[31660]: audit 2024-04-07T16:18:20.636798+0000 mon.a (mon.0) 1006 : audit 1 from='mgr.14553 ' entity='mgr.y' cmd='[{"prefix": "mgr fail", "who": "y"}]': finished 2024-04-07T16:18:20.942 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:20 smithi100 bash[31660]: cluster 2024-04-07T16:18:20.636861+0000 mon.a (mon.0) 1007 : cluster 0 mgrmap e24: x(active, starting, since 0.0181805s) 2024-04-07T16:18:20.942 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:20 smithi100 bash[31660]: audit 2024-04-07T16:18:20.637341+0000 mon.b (mon.1) 264 : audit 0 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-07T16:18:20.942 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:20 smithi100 bash[31660]: audit 2024-04-07T16:18:20.638079+0000 mon.b (mon.1) 265 : audit 0 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-07T16:18:20.942 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:20 smithi100 bash[31660]: audit 2024-04-07T16:18:20.638778+0000 mon.b (mon.1) 266 : audit 0 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-07T16:18:20.942 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:20 smithi100 bash[31660]: audit 2024-04-07T16:18:20.639548+0000 mon.b (mon.1) 267 : audit 0 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-07T16:18:20.942 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:20 smithi100 bash[31660]: audit 2024-04-07T16:18:20.640287+0000 mon.b (mon.1) 268 : audit 0 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-07T16:18:20.942 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:20 smithi100 bash[31660]: audit 2024-04-07T16:18:20.641024+0000 mon.b (mon.1) 269 : audit 0 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-07T16:18:20.942 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:20 smithi100 bash[31660]: audit 2024-04-07T16:18:20.641723+0000 mon.b (mon.1) 270 : audit 0 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-07T16:18:20.942 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:20 smithi100 bash[31660]: audit 2024-04-07T16:18:20.642452+0000 mon.b (mon.1) 271 : audit 0 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-07T16:18:20.942 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:20 smithi100 bash[31660]: audit 2024-04-07T16:18:20.643223+0000 mon.b (mon.1) 272 : audit 0 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-07T16:18:20.942 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:20 smithi100 bash[31660]: audit 2024-04-07T16:18:20.644042+0000 mon.b (mon.1) 273 : audit 0 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-07T16:18:20.942 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:20 smithi100 bash[31660]: audit 2024-04-07T16:18:20.645426+0000 mon.b (mon.1) 274 : audit 0 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-07T16:18:20.943 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:20 smithi100 bash[31660]: cluster 2024-04-07T16:18:20.692311+0000 mon.a (mon.0) 1008 : cluster 1 Manager daemon x is now available 2024-04-07T16:18:20.943 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:20 smithi100 bash[31660]: audit 2024-04-07T16:18:20.734021+0000 mon.b (mon.1) 275 : audit 0 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:18:20.943 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:20 smithi100 bash[31660]: audit 2024-04-07T16:18:20.766090+0000 mon.b (mon.1) 276 : audit 0 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:18:20.943 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:20 smithi100 bash[31660]: audit 2024-04-07T16:18:20.769222+0000 mon.b (mon.1) 277 : audit 1 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-07T16:18:20.943 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:20 smithi100 bash[31660]: audit 2024-04-07T16:18:20.774108+0000 mon.a (mon.0) 1009 : audit 1 from='mgr.24553 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-07T16:18:20.943 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:20 smithi100 bash[31660]: audit 2024-04-07T16:18:20.806308+0000 mon.b (mon.1) 278 : audit 1 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-07T16:18:20.943 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:20 smithi100 bash[31660]: audit 2024-04-07T16:18:20.811331+0000 mon.a (mon.0) 1010 : audit 1 from='mgr.24553 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-07T16:18:21.105 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:18:20 smithi144 bash[51952]: [07/Apr/2024:16:18:20] ENGINE Bus STOPPING 2024-04-07T16:18:21.105 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:18:20 smithi144 bash[51952]: [07/Apr/2024:16:18:20] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-04-07T16:18:21.105 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:18:20 smithi144 bash[51952]: [07/Apr/2024:16:18:20] ENGINE Bus STOPPED 2024-04-07T16:18:21.105 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:18:20 smithi144 bash[51952]: [07/Apr/2024:16:18:20] ENGINE Bus STARTING 2024-04-07T16:18:21.105 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:18:20 smithi144 bash[51952]: debug 2024-04-07T16:18:20.838+0000 7f5fae4b3700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-07T16:18:21.105 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:18:20 smithi144 bash[51952]: debug 2024-04-07T16:18:20.838+0000 7f5fae4b3700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-07T16:18:21.105 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:18:20 smithi144 bash[51952]: debug 2024-04-07T16:18:20.838+0000 7f5fae4b3700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-07T16:18:21.106 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:18:20 smithi144 bash[51952]: debug 2024-04-07T16:18:20.838+0000 7f5fae4b3700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-07T16:18:21.106 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:18:20 smithi144 bash[51952]: debug 2024-04-07T16:18:20.838+0000 7f5fae4b3700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-07T16:18:21.106 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:18:20 smithi144 bash[51952]: debug 2024-04-07T16:18:20.846+0000 7f5fb2cbc700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-07T16:18:21.106 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:18:20 smithi144 bash[51952]: debug 2024-04-07T16:18:20.846+0000 7f5fb2cbc700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-07T16:18:21.106 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:18:20 smithi144 bash[51952]: debug 2024-04-07T16:18:20.846+0000 7f5fb2cbc700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-07T16:18:21.106 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:18:20 smithi144 bash[51952]: debug 2024-04-07T16:18:20.846+0000 7f5fb2cbc700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-07T16:18:21.106 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:18:20 smithi144 bash[51952]: debug 2024-04-07T16:18:20.846+0000 7f5fb2cbc700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-07T16:18:21.106 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:18:20 smithi144 bash[51952]: [07/Apr/2024:16:18:20] ENGINE Serving on http://:::9283 2024-04-07T16:18:21.106 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:18:20 smithi144 bash[51952]: [07/Apr/2024:16:18:20] ENGINE Bus STARTED 2024-04-07T16:18:21.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:20 smithi144 bash[23522]: cluster 2024-04-07T16:18:19.872482+0000 mon.a (mon.0) 998 : cluster 0 osdmap e165: 8 total, 8 up, 8 in 2024-04-07T16:18:21.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:20 smithi144 bash[23522]: audit 2024-04-07T16:18:20.320736+0000 mon.a (mon.0) 999 : audit 0 from='client.? 172.21.15.100:0/2055622250' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:18:21.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:20 smithi144 bash[23522]: audit 2024-04-07T16:18:20.488089+0000 mon.a (mon.0) 1000 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:18:21.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:20 smithi144 bash[23522]: audit 2024-04-07T16:18:20.504013+0000 mon.b (mon.1) 254 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:18:21.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:20 smithi144 bash[23522]: audit 2024-04-07T16:18:20.504071+0000 mon.a (mon.0) 1001 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:18:21.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:20 smithi144 bash[23522]: audit 2024-04-07T16:18:20.506022+0000 mon.b (mon.1) 255 : audit 1 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:18:21.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:20 smithi144 bash[23522]: audit 2024-04-07T16:18:20.521308+0000 mon.a (mon.0) 1002 : audit 1 from='mgr.14553 ' entity='mgr.y' 2024-04-07T16:18:21.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:20 smithi144 bash[23522]: audit 2024-04-07T16:18:20.601079+0000 mon.b (mon.1) 256 : audit 0 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:18:21.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:20 smithi144 bash[23522]: audit 2024-04-07T16:18:20.612281+0000 mon.b (mon.1) 257 : audit 1 from='mgr.14553 172.21.15.100:0/2762529013' entity='mgr.y' cmd=[{"prefix": "mgr fail", "who": "y"}]: dispatch 2024-04-07T16:18:21.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:20 smithi144 bash[23522]: audit 2024-04-07T16:18:20.617607+0000 mon.a (mon.0) 1003 : audit 1 from='mgr.14553 ' entity='mgr.y' cmd=[{"prefix": "mgr fail", "who": "y"}]: dispatch 2024-04-07T16:18:21.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:20 smithi144 bash[23522]: cluster 2024-04-07T16:18:20.618684+0000 mon.a (mon.0) 1004 : cluster 1 Activating manager daemon x 2024-04-07T16:18:21.108 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:20 smithi144 bash[23522]: audit 2024-04-07T16:18:20.633684+0000 mon.b (mon.1) 258 : audit 0 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:18:21.108 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:20 smithi144 bash[23522]: audit 2024-04-07T16:18:20.634221+0000 mon.b (mon.1) 259 : audit 0 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:18:21.108 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:20 smithi144 bash[23522]: audit 2024-04-07T16:18:20.634734+0000 mon.b (mon.1) 260 : audit 0 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:18:21.108 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:20 smithi144 bash[23522]: audit 2024-04-07T16:18:20.635246+0000 mon.b (mon.1) 261 : audit 0 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-07T16:18:21.108 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:20 smithi144 bash[23522]: audit 2024-04-07T16:18:20.635958+0000 mon.b (mon.1) 262 : audit 0 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-07T16:18:21.108 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:20 smithi144 bash[23522]: cluster 2024-04-07T16:18:20.636358+0000 mon.a (mon.0) 1005 : cluster 0 osdmap e166: 8 total, 8 up, 8 in 2024-04-07T16:18:21.108 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:20 smithi144 bash[23522]: audit 2024-04-07T16:18:20.636706+0000 mon.b (mon.1) 263 : audit 0 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-07T16:18:21.108 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:20 smithi144 bash[23522]: audit 2024-04-07T16:18:20.636798+0000 mon.a (mon.0) 1006 : audit 1 from='mgr.14553 ' entity='mgr.y' cmd='[{"prefix": "mgr fail", "who": "y"}]': finished 2024-04-07T16:18:21.108 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:20 smithi144 bash[23522]: cluster 2024-04-07T16:18:20.636861+0000 mon.a (mon.0) 1007 : cluster 0 mgrmap e24: x(active, starting, since 0.0181805s) 2024-04-07T16:18:21.108 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:20 smithi144 bash[23522]: audit 2024-04-07T16:18:20.637341+0000 mon.b (mon.1) 264 : audit 0 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-07T16:18:21.108 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:20 smithi144 bash[23522]: audit 2024-04-07T16:18:20.638079+0000 mon.b (mon.1) 265 : audit 0 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-07T16:18:21.108 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:20 smithi144 bash[23522]: audit 2024-04-07T16:18:20.638778+0000 mon.b (mon.1) 266 : audit 0 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-07T16:18:21.108 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:20 smithi144 bash[23522]: audit 2024-04-07T16:18:20.639548+0000 mon.b (mon.1) 267 : audit 0 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-07T16:18:21.108 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:20 smithi144 bash[23522]: audit 2024-04-07T16:18:20.640287+0000 mon.b (mon.1) 268 : audit 0 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-07T16:18:21.108 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:20 smithi144 bash[23522]: audit 2024-04-07T16:18:20.641024+0000 mon.b (mon.1) 269 : audit 0 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-07T16:18:21.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:20 smithi144 bash[23522]: audit 2024-04-07T16:18:20.641723+0000 mon.b (mon.1) 270 : audit 0 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-07T16:18:21.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:20 smithi144 bash[23522]: audit 2024-04-07T16:18:20.642452+0000 mon.b (mon.1) 271 : audit 0 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-07T16:18:21.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:20 smithi144 bash[23522]: audit 2024-04-07T16:18:20.643223+0000 mon.b (mon.1) 272 : audit 0 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-07T16:18:21.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:20 smithi144 bash[23522]: audit 2024-04-07T16:18:20.644042+0000 mon.b (mon.1) 273 : audit 0 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-07T16:18:21.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:20 smithi144 bash[23522]: audit 2024-04-07T16:18:20.645426+0000 mon.b (mon.1) 274 : audit 0 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-07T16:18:21.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:20 smithi144 bash[23522]: cluster 2024-04-07T16:18:20.692311+0000 mon.a (mon.0) 1008 : cluster 1 Manager daemon x is now available 2024-04-07T16:18:21.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:20 smithi144 bash[23522]: audit 2024-04-07T16:18:20.734021+0000 mon.b (mon.1) 275 : audit 0 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:18:21.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:20 smithi144 bash[23522]: audit 2024-04-07T16:18:20.766090+0000 mon.b (mon.1) 276 : audit 0 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:18:21.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:20 smithi144 bash[23522]: audit 2024-04-07T16:18:20.769222+0000 mon.b (mon.1) 277 : audit 1 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-07T16:18:21.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:20 smithi144 bash[23522]: audit 2024-04-07T16:18:20.774108+0000 mon.a (mon.0) 1009 : audit 1 from='mgr.24553 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-07T16:18:21.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:20 smithi144 bash[23522]: audit 2024-04-07T16:18:20.806308+0000 mon.b (mon.1) 278 : audit 1 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-07T16:18:21.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:20 smithi144 bash[23522]: audit 2024-04-07T16:18:20.811331+0000 mon.a (mon.0) 1010 : audit 1 from='mgr.24553 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-07T16:18:21.646 INFO:tasks.rados.rados.0.smithi100.stdout:568: done (4 left) 2024-04-07T16:18:21.646 INFO:tasks.rados.rados.0.smithi100.stdout:569: done (3 left) 2024-04-07T16:18:21.646 INFO:tasks.rados.rados.0.smithi100.stdout:570: done (2 left) 2024-04-07T16:18:21.646 INFO:tasks.rados.rados.0.smithi100.stdout:571: done (1 left) 2024-04-07T16:18:21.646 INFO:tasks.rados.rados.0.smithi100.stdout:572: done (0 left) 2024-04-07T16:18:21.647 INFO:tasks.rados.rados.0.smithi100.stdout:573: read oid 22 snap -1 2024-04-07T16:18:21.647 INFO:tasks.rados.rados.0.smithi100.stdout:573: expect deleted 2024-04-07T16:18:21.647 INFO:tasks.rados.rados.0.smithi100.stdout:574: rollback oid 47 current snap is 51 2024-04-07T16:18:21.647 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 47 to 47 2024-04-07T16:18:21.647 INFO:tasks.rados.rados.0.smithi100.stdout:575: snap_remove snap 49 2024-04-07T16:18:22.649 INFO:tasks.rados.rados.0.smithi100.stdout:575: done (2 left) 2024-04-07T16:18:22.649 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 6 current snap is 51 2024-04-07T16:18:22.649 INFO:tasks.rados.rados.0.smithi100.stdout:576: seq_num 193 ranges {4210688=434176,4644864=401408} 2024-04-07T16:18:22.687 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:18:22 smithi100 bash[21601]: debug 2024-04-07T16:18:22.444+0000 7fcfdfc22200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-07T16:18:22.687 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:18:22 smithi100 bash[21601]: debug 2024-04-07T16:18:22.540+0000 7fcfdfc22200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-07T16:18:22.687 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:22 smithi100 bash[31660]: cephadm 2024-04-07T16:18:21.219538+0000 mgr.x (mgr.24553) 1 : cephadm 1 Deploying cephadm binary to smithi144 2024-04-07T16:18:22.687 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:22 smithi100 bash[31660]: cephadm 2024-04-07T16:18:21.220476+0000 mgr.x (mgr.24553) 2 : cephadm 1 Deploying cephadm binary to smithi100 2024-04-07T16:18:22.687 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:22 smithi100 bash[31660]: cephadm 2024-04-07T16:18:21.365735+0000 mgr.x (mgr.24553) 3 : cephadm 1 [07/Apr/2024:16:18:21] ENGINE Bus STARTING 2024-04-07T16:18:22.687 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:22 smithi100 bash[31660]: cephadm 2024-04-07T16:18:21.507635+0000 mgr.x (mgr.24553) 4 : cephadm 1 [07/Apr/2024:16:18:21] ENGINE Serving on http://172.21.15.144:8765 2024-04-07T16:18:22.688 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:22 smithi100 bash[31660]: cluster 2024-04-07T16:18:21.634646+0000 mon.a (mon.0) 1011 : cluster 0 mgrmap e25: x(active, since 1.01596s) 2024-04-07T16:18:22.688 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:22 smithi100 bash[31660]: cluster 2024-04-07T16:18:21.654672+0000 mon.a (mon.0) 1012 : cluster 0 osdmap e167: 8 total, 8 up, 8 in 2024-04-07T16:18:22.688 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:22 smithi100 bash[31660]: cluster 2024-04-07T16:18:21.658432+0000 mgr.x (mgr.24553) 5 : cluster 0 pgmap v4: 121 pgs: 121 active+clean; 121 MiB data, 513 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:18:22.688 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:22 smithi100 bash[31660]: cephadm 2024-04-07T16:18:21.713312+0000 mgr.x (mgr.24553) 6 : cephadm 1 [07/Apr/2024:16:18:21] ENGINE Serving on https://172.21.15.144:7150 2024-04-07T16:18:22.688 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:22 smithi100 bash[31660]: cephadm 2024-04-07T16:18:21.713487+0000 mgr.x (mgr.24553) 7 : cephadm 1 [07/Apr/2024:16:18:21] ENGINE Bus STARTED 2024-04-07T16:18:22.688 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:22 smithi100 bash[21346]: cephadm 2024-04-07T16:18:21.219538+0000 mgr.x (mgr.24553) 1 : cephadm 1 Deploying cephadm binary to smithi144 2024-04-07T16:18:22.688 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:22 smithi100 bash[21346]: cephadm 2024-04-07T16:18:21.220476+0000 mgr.x (mgr.24553) 2 : cephadm 1 Deploying cephadm binary to smithi100 2024-04-07T16:18:22.688 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:22 smithi100 bash[21346]: cephadm 2024-04-07T16:18:21.365735+0000 mgr.x (mgr.24553) 3 : cephadm 1 [07/Apr/2024:16:18:21] ENGINE Bus STARTING 2024-04-07T16:18:22.688 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:22 smithi100 bash[21346]: cephadm 2024-04-07T16:18:21.507635+0000 mgr.x (mgr.24553) 4 : cephadm 1 [07/Apr/2024:16:18:21] ENGINE Serving on http://172.21.15.144:8765 2024-04-07T16:18:22.688 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:22 smithi100 bash[21346]: cluster 2024-04-07T16:18:21.634646+0000 mon.a (mon.0) 1011 : cluster 0 mgrmap e25: x(active, since 1.01596s) 2024-04-07T16:18:22.688 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:22 smithi100 bash[21346]: cluster 2024-04-07T16:18:21.654672+0000 mon.a (mon.0) 1012 : cluster 0 osdmap e167: 8 total, 8 up, 8 in 2024-04-07T16:18:22.688 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:22 smithi100 bash[21346]: cluster 2024-04-07T16:18:21.658432+0000 mgr.x (mgr.24553) 5 : cluster 0 pgmap v4: 121 pgs: 121 active+clean; 121 MiB data, 513 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:18:22.688 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:22 smithi100 bash[21346]: cephadm 2024-04-07T16:18:21.713312+0000 mgr.x (mgr.24553) 6 : cephadm 1 [07/Apr/2024:16:18:21] ENGINE Serving on https://172.21.15.144:7150 2024-04-07T16:18:22.689 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:22 smithi100 bash[21346]: cephadm 2024-04-07T16:18:21.713487+0000 mgr.x (mgr.24553) 7 : cephadm 1 [07/Apr/2024:16:18:21] ENGINE Bus STARTED 2024-04-07T16:18:22.710 INFO:tasks.rados.rados.0.smithi100.stdout:576: writing smithi10066867-6 from 4210688 to 4644864 tid 1 2024-04-07T16:18:22.714 INFO:tasks.rados.rados.0.smithi100.stdout:576: writing smithi10066867-6 from 4644864 to 5046272 tid 2 2024-04-07T16:18:22.715 INFO:tasks.rados.rados.0.smithi100.stdout:577: copy_from oid 41 from oid 46 current snap is 51 2024-04-07T16:18:22.715 INFO:tasks.rados.rados.0.smithi100.stdout:574: finishing rollback tid 0 to smithi10066867-47 2024-04-07T16:18:22.715 INFO:tasks.rados.rados.0.smithi100.stdout:574: finishing rollback tid 1 to smithi10066867-47 2024-04-07T16:18:22.715 INFO:tasks.rados.rados.0.smithi100.stdout:574: finishing rollback tid 2 to smithi10066867-47 2024-04-07T16:18:22.715 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 47 v 76 (ObjNum 149 snap 41 seq_num 149) dirty exists 2024-04-07T16:18:22.715 INFO:tasks.rados.rados.0.smithi100.stdout:573: done (3 left) 2024-04-07T16:18:22.716 INFO:tasks.rados.rados.0.smithi100.stdout:574: done (2 left) 2024-04-07T16:18:22.716 INFO:tasks.rados.rados.0.smithi100.stdout:578: rollback oid 27 current snap is 51 2024-04-07T16:18:22.716 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 27 to 46 2024-04-07T16:18:22.716 INFO:tasks.rados.rados.0.smithi100.stdout:579: snap_remove snap 40 2024-04-07T16:18:22.852 INFO:tasks.rados.rados.0.smithi100.stdout:579: done (3 left) 2024-04-07T16:18:22.852 INFO:tasks.rados.rados.0.smithi100.stdout:580: snap_remove snap 37 2024-04-07T16:18:22.852 INFO:tasks.rados.rados.0.smithi100.stdout:578: finishing rollback tid 0 to smithi10066867-27 2024-04-07T16:18:22.852 INFO:tasks.rados.rados.0.smithi100.stdout:576: finishing write tid 1 to smithi10066867-6 2024-04-07T16:18:22.852 INFO:tasks.rados.rados.0.smithi100.stdout:577: finishing copy_from racing read to smithi10066867-41 2024-04-07T16:18:22.852 INFO:tasks.rados.rados.0.smithi100.stdout:578: finishing rollback tid 1 to smithi10066867-27 2024-04-07T16:18:22.852 INFO:tasks.rados.rados.0.smithi100.stdout:578: finishing rollback tid 2 to smithi10066867-27 2024-04-07T16:18:22.852 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 27 v 86 (ObjNum 161 snap 0 seq_num 1) dirty exists 2024-04-07T16:18:22.852 INFO:tasks.rados.rados.0.smithi100.stdout:576: finishing write tid 2 to smithi10066867-6 2024-04-07T16:18:22.852 INFO:tasks.rados.rados.0.smithi100.stdout:576: finishing write tid 3 to smithi10066867-6 2024-04-07T16:18:22.852 INFO:tasks.rados.rados.0.smithi100.stdout:577: finishing copy_from to smithi10066867-41 2024-04-07T16:18:22.853 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 41 v 8 (ObjNum 45 snap 0 seq_num 45) dirty exists 2024-04-07T16:18:22.853 INFO:tasks.rados.rados.0.smithi100.stdout:576: finishing write tid 4 to smithi10066867-6 2024-04-07T16:18:22.853 INFO:tasks.rados.rados.0.smithi100.stdout:576: oid 6 updating version 0 to 181 2024-04-07T16:18:22.853 INFO:tasks.rados.rados.0.smithi100.stdout:576: oid 6 updating version 181 to 182 2024-04-07T16:18:22.853 INFO:tasks.rados.rados.0.smithi100.stdout:576: oid 6 version 182 is already newer than 180 2024-04-07T16:18:22.853 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 6 v 182 (ObjNum 193 snap 51 seq_num 193) dirty exists 2024-04-07T16:18:22.853 INFO:tasks.rados.rados.0.smithi100.stdout:576: left oid 6 (ObjNum 193 snap 51 seq_num 193) 2024-04-07T16:18:23.105 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:22 smithi144 bash[23522]: cephadm 2024-04-07T16:18:21.219538+0000 mgr.x (mgr.24553) 1 : cephadm 1 Deploying cephadm binary to smithi144 2024-04-07T16:18:23.105 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:22 smithi144 bash[23522]: cephadm 2024-04-07T16:18:21.220476+0000 mgr.x (mgr.24553) 2 : cephadm 1 Deploying cephadm binary to smithi100 2024-04-07T16:18:23.105 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:22 smithi144 bash[23522]: cephadm 2024-04-07T16:18:21.365735+0000 mgr.x (mgr.24553) 3 : cephadm 1 [07/Apr/2024:16:18:21] ENGINE Bus STARTING 2024-04-07T16:18:23.105 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:22 smithi144 bash[23522]: cephadm 2024-04-07T16:18:21.507635+0000 mgr.x (mgr.24553) 4 : cephadm 1 [07/Apr/2024:16:18:21] ENGINE Serving on http://172.21.15.144:8765 2024-04-07T16:18:23.105 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:22 smithi144 bash[23522]: cluster 2024-04-07T16:18:21.634646+0000 mon.a (mon.0) 1011 : cluster 0 mgrmap e25: x(active, since 1.01596s) 2024-04-07T16:18:23.105 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:22 smithi144 bash[23522]: cluster 2024-04-07T16:18:21.654672+0000 mon.a (mon.0) 1012 : cluster 0 osdmap e167: 8 total, 8 up, 8 in 2024-04-07T16:18:23.105 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:22 smithi144 bash[23522]: cluster 2024-04-07T16:18:21.658432+0000 mgr.x (mgr.24553) 5 : cluster 0 pgmap v4: 121 pgs: 121 active+clean; 121 MiB data, 513 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:18:23.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:22 smithi144 bash[23522]: cephadm 2024-04-07T16:18:21.713312+0000 mgr.x (mgr.24553) 6 : cephadm 1 [07/Apr/2024:16:18:21] ENGINE Serving on https://172.21.15.144:7150 2024-04-07T16:18:23.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:22 smithi144 bash[23522]: cephadm 2024-04-07T16:18:21.713487+0000 mgr.x (mgr.24553) 7 : cephadm 1 [07/Apr/2024:16:18:21] ENGINE Bus STARTED 2024-04-07T16:18:23.186 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:18:22 smithi100 bash[21601]: debug 2024-04-07T16:18:22.812+0000 7fcfdfc22200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-07T16:18:23.867 INFO:tasks.rados.rados.0.smithi100.stdout:576: done (3 left) 2024-04-07T16:18:23.867 INFO:tasks.rados.rados.0.smithi100.stdout:577: done (2 left) 2024-04-07T16:18:23.867 INFO:tasks.rados.rados.0.smithi100.stdout:578: done (1 left) 2024-04-07T16:18:23.867 INFO:tasks.rados.rados.0.smithi100.stdout:580: done (0 left) 2024-04-07T16:18:23.867 INFO:tasks.rados.rados.0.smithi100.stdout:581: read oid 47 snap -1 2024-04-07T16:18:23.867 INFO:tasks.rados.rados.0.smithi100.stdout:581: expect (ObjNum 149 snap 41 seq_num 149) 2024-04-07T16:18:23.867 INFO:tasks.rados.rados.0.smithi100.stdout:582: delete oid 33 current snap is 51 2024-04-07T16:18:23.886 INFO:tasks.rados.rados.0.smithi100.stdout:582: done (1 left) 2024-04-07T16:18:23.886 INFO:tasks.rados.rados.0.smithi100.stdout:583: rollback oid 8 current snap is 51 2024-04-07T16:18:23.886 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 8 to 48 2024-04-07T16:18:23.887 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 12 current snap is 51 2024-04-07T16:18:23.887 INFO:tasks.rados.rados.0.smithi100.stdout:584: seq_num 195 ranges {819200=475136,1294336=401408} 2024-04-07T16:18:23.901 INFO:tasks.rados.rados.0.smithi100.stdout:584: writing smithi10066867-12 from 819200 to 1294336 tid 1 2024-04-07T16:18:23.906 INFO:tasks.rados.rados.0.smithi100.stdout:584: writing smithi10066867-12 from 1294336 to 1695744 tid 2 2024-04-07T16:18:23.907 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 31 current snap is 51 2024-04-07T16:18:23.907 INFO:tasks.rados.rados.0.smithi100.stdout:581: expect (ObjNum 149 snap 41 seq_num 149) 2024-04-07T16:18:23.912 INFO:tasks.rados.rados.0.smithi100.stdout:583: finishing rollback tid 0 to smithi10066867-8 2024-04-07T16:18:23.912 INFO:tasks.rados.rados.0.smithi100.stdout:583: finishing rollback tid 1 to smithi10066867-8 2024-04-07T16:18:23.913 INFO:tasks.rados.rados.0.smithi100.stdout:583: finishing rollback tid 2 to smithi10066867-8 2024-04-07T16:18:23.913 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 8 v 171 (ObjNum 7 snap 0 seq_num 7) dirty exists 2024-04-07T16:18:23.913 INFO:tasks.rados.rados.0.smithi100.stdout:584: finishing write tid 1 to smithi10066867-12 2024-04-07T16:18:23.913 INFO:tasks.rados.rados.0.smithi100.stdout:584: finishing write tid 2 to smithi10066867-12 2024-04-07T16:18:23.913 INFO:tasks.rados.rados.0.smithi100.stdout:585: seq_num 196 ranges {540672=491520,1032192=401408} 2024-04-07T16:18:23.919 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:23 smithi100 bash[31660]: cluster 2024-04-07T16:18:22.640769+0000 mgr.x (mgr.24553) 8 : cluster 0 pgmap v5: 121 pgs: 121 active+clean; 121 MiB data, 513 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:18:23.919 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:23 smithi100 bash[31660]: cluster 2024-04-07T16:18:22.662449+0000 mon.a (mon.0) 1013 : cluster 0 osdmap e168: 8 total, 8 up, 8 in 2024-04-07T16:18:23.919 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:23 smithi100 bash[31660]: cluster 2024-04-07T16:18:22.863814+0000 mon.a (mon.0) 1014 : cluster 0 osdmap e169: 8 total, 8 up, 8 in 2024-04-07T16:18:23.919 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:23 smithi100 bash[21346]: cluster 2024-04-07T16:18:22.640769+0000 mgr.x (mgr.24553) 8 : cluster 0 pgmap v5: 121 pgs: 121 active+clean; 121 MiB data, 513 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:18:23.919 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:23 smithi100 bash[21346]: cluster 2024-04-07T16:18:22.662449+0000 mon.a (mon.0) 1013 : cluster 0 osdmap e168: 8 total, 8 up, 8 in 2024-04-07T16:18:23.919 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:23 smithi100 bash[21346]: cluster 2024-04-07T16:18:22.863814+0000 mon.a (mon.0) 1014 : cluster 0 osdmap e169: 8 total, 8 up, 8 in 2024-04-07T16:18:23.920 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:18:23 smithi100 bash[21601]: debug 2024-04-07T16:18:23.692+0000 7fcfdfc22200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-07T16:18:23.920 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:18:23 smithi100 bash[21601]: debug 2024-04-07T16:18:23.804+0000 7fcfdfc22200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-07T16:18:23.920 INFO:tasks.rados.rados.0.smithi100.stdout:585: writing smithi10066867-31 from 540672 to 1032192 tid 1 2024-04-07T16:18:23.923 INFO:tasks.rados.rados.0.smithi100.stdout:585: writing smithi10066867-31 from 1032192 to 1433600 tid 2 2024-04-07T16:18:23.923 INFO:tasks.rados.rados.0.smithi100.stdout:581: done (3 left) 2024-04-07T16:18:23.923 INFO:tasks.rados.rados.0.smithi100.stdout:583: done (2 left) 2024-04-07T16:18:23.923 INFO:tasks.rados.rados.0.smithi100.stdout:586: rollback oid 32 current snap is 51 2024-04-07T16:18:23.923 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 32 to 48 2024-04-07T16:18:23.923 INFO:tasks.rados.rados.0.smithi100.stdout:584: finishing write tid 3 to smithi10066867-12 2024-04-07T16:18:23.923 INFO:tasks.rados.rados.0.smithi100.stdout:584: finishing write tid 4 to smithi10066867-12 2024-04-07T16:18:23.923 INFO:tasks.rados.rados.0.smithi100.stdout:584: oid 12 updating version 0 to 235 2024-04-07T16:18:23.923 INFO:tasks.rados.rados.0.smithi100.stdout:584: oid 12 version 235 is already newer than 234 2024-04-07T16:18:23.923 INFO:tasks.rados.rados.0.smithi100.stdout:584: oid 12 version 235 is already newer than 233 2024-04-07T16:18:23.923 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 12 v 235 (ObjNum 195 snap 51 seq_num 195) dirty exists 2024-04-07T16:18:23.924 INFO:tasks.rados.rados.0.smithi100.stdout:584: left oid 12 (ObjNum 195 snap 51 seq_num 195) 2024-04-07T16:18:23.924 INFO:tasks.rados.rados.0.smithi100.stdout:585: finishing write tid 1 to smithi10066867-31 2024-04-07T16:18:23.924 INFO:tasks.rados.rados.0.smithi100.stdout:584: done (2 left) 2024-04-07T16:18:23.924 INFO:tasks.rados.rados.0.smithi100.stdout:587: read oid 39 snap -1 2024-04-07T16:18:23.924 INFO:tasks.rados.rados.0.smithi100.stdout:587: expect (ObjNum 177 snap 46 seq_num 177) 2024-04-07T16:18:23.924 INFO:tasks.rados.rados.0.smithi100.stdout:588: read oid 36 snap -1 2024-04-07T16:18:23.924 INFO:tasks.rados.rados.0.smithi100.stdout:588: expect (ObjNum 183 snap 46 seq_num 183) 2024-04-07T16:18:23.924 INFO:tasks.rados.rados.0.smithi100.stdout:589: snap_create 2024-04-07T16:18:23.924 INFO:tasks.rados.rados.0.smithi100.stdout:586: finishing rollback tid 1 to smithi10066867-32 2024-04-07T16:18:23.924 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 32 v 37 (ObjNum 129 snap 0 seq_num 2701975128) dirty dne 2024-04-07T16:18:23.925 INFO:tasks.rados.rados.0.smithi100.stdout:585: finishing write tid 2 to smithi10066867-31 2024-04-07T16:18:23.927 INFO:tasks.rados.rados.0.smithi100.stdout:585: finishing write tid 3 to smithi10066867-31 2024-04-07T16:18:23.928 INFO:tasks.rados.rados.0.smithi100.stdout:588: expect (ObjNum 183 snap 46 seq_num 183) 2024-04-07T16:18:23.964 INFO:tasks.rados.rados.0.smithi100.stdout:585: finishing write tid 4 to smithi10066867-31 2024-04-07T16:18:23.964 INFO:tasks.rados.rados.0.smithi100.stdout:585: oid 31 updating version 0 to 100 2024-04-07T16:18:23.964 INFO:tasks.rados.rados.0.smithi100.stdout:585: oid 31 version 100 is already newer than 98 2024-04-07T16:18:23.964 INFO:tasks.rados.rados.0.smithi100.stdout:585: oid 31 version 100 is already newer than 99 2024-04-07T16:18:23.964 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 31 v 100 (ObjNum 196 snap 51 seq_num 196) dirty exists 2024-04-07T16:18:23.964 INFO:tasks.rados.rados.0.smithi100.stdout:585: left oid 31 (ObjNum 196 snap 51 seq_num 196) 2024-04-07T16:18:23.964 INFO:tasks.rados.rados.0.smithi100.stdout:587: expect (ObjNum 177 snap 46 seq_num 177) 2024-04-07T16:18:24.105 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:23 smithi144 bash[23522]: cluster 2024-04-07T16:18:22.640769+0000 mgr.x (mgr.24553) 8 : cluster 0 pgmap v5: 121 pgs: 121 active+clean; 121 MiB data, 513 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:18:24.105 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:23 smithi144 bash[23522]: cluster 2024-04-07T16:18:22.662449+0000 mon.a (mon.0) 1013 : cluster 0 osdmap e168: 8 total, 8 up, 8 in 2024-04-07T16:18:24.105 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:23 smithi144 bash[23522]: cluster 2024-04-07T16:18:22.863814+0000 mon.a (mon.0) 1014 : cluster 0 osdmap e169: 8 total, 8 up, 8 in 2024-04-07T16:18:24.686 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:18:23 smithi100 bash[21601]: debug 2024-04-07T16:18:23.928+0000 7fcfdfc22200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-07T16:18:24.687 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:18:24 smithi100 bash[21601]: debug 2024-04-07T16:18:24.300+0000 7fcfdfc22200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-07T16:18:24.687 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:18:24 smithi100 bash[21601]: debug 2024-04-07T16:18:24.412+0000 7fcfdfc22200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-07T16:18:24.687 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:18:24 smithi100 bash[21601]: debug 2024-04-07T16:18:24.520+0000 7fcfdfc22200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-07T16:18:24.884 INFO:tasks.rados.rados.0.smithi100.stdout:585: done (4 left) 2024-04-07T16:18:24.884 INFO:tasks.rados.rados.0.smithi100.stdout:586: done (3 left) 2024-04-07T16:18:24.884 INFO:tasks.rados.rados.0.smithi100.stdout:587: done (2 left) 2024-04-07T16:18:24.884 INFO:tasks.rados.rados.0.smithi100.stdout:588: done (1 left) 2024-04-07T16:18:24.884 INFO:tasks.rados.rados.0.smithi100.stdout:589: done (0 left) 2024-04-07T16:18:24.884 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 38 current snap is 52 2024-04-07T16:18:24.884 INFO:tasks.rados.rados.0.smithi100.stdout:590: seq_num 197 ranges {2768896=507904,3276800=401408} 2024-04-07T16:18:24.920 INFO:tasks.rados.rados.0.smithi100.stdout:590: writing smithi10066867-38 from 2768896 to 3276800 tid 1 2024-04-07T16:18:24.925 INFO:tasks.rados.rados.0.smithi100.stdout:590: writing smithi10066867-38 from 3276800 to 3678208 tid 2 2024-04-07T16:18:24.925 INFO:tasks.rados.rados.0.smithi100.stdout:591: setattr oid 26 current snap is 52 2024-04-07T16:18:24.929 INFO:tasks.rados.rados.0.smithi100.stdout:592: delete oid 43 current snap is 52 2024-04-07T16:18:24.933 INFO:tasks.rados.rados.0.smithi100.stdout:590: finishing write tid 1 to smithi10066867-38 2024-04-07T16:18:24.933 INFO:tasks.rados.rados.0.smithi100.stdout:590: finishing write tid 2 to smithi10066867-38 2024-04-07T16:18:24.933 INFO:tasks.rados.rados.0.smithi100.stdout:590: finishing write tid 3 to smithi10066867-38 2024-04-07T16:18:24.936 INFO:tasks.rados.rados.0.smithi100.stdout:592: done (2 left) 2024-04-07T16:18:24.936 INFO:tasks.rados.rados.0.smithi100.stdout:593: snap_remove snap 50 2024-04-07T16:18:25.121 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:18:24 smithi100 bash[21601]: debug 2024-04-07T16:18:24.884+0000 7fcfdfc22200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-07T16:18:25.122 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:24 smithi100 bash[21346]: cluster 2024-04-07T16:18:23.880426+0000 mon.a (mon.0) 1015 : cluster 0 osdmap e170: 8 total, 8 up, 8 in 2024-04-07T16:18:25.122 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:24 smithi100 bash[21346]: cluster 2024-04-07T16:18:23.890572+0000 mon.a (mon.0) 1016 : cluster 0 mgrmap e26: x(active, since 3s) 2024-04-07T16:18:25.122 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:24 smithi100 bash[31660]: cluster 2024-04-07T16:18:23.880426+0000 mon.a (mon.0) 1015 : cluster 0 osdmap e170: 8 total, 8 up, 8 in 2024-04-07T16:18:25.122 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:24 smithi100 bash[31660]: cluster 2024-04-07T16:18:23.890572+0000 mon.a (mon.0) 1016 : cluster 0 mgrmap e26: x(active, since 3s) 2024-04-07T16:18:25.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:24 smithi144 bash[23522]: cluster 2024-04-07T16:18:23.880426+0000 mon.a (mon.0) 1015 : cluster 0 osdmap e170: 8 total, 8 up, 8 in 2024-04-07T16:18:25.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:24 smithi144 bash[23522]: cluster 2024-04-07T16:18:23.890572+0000 mon.a (mon.0) 1016 : cluster 0 mgrmap e26: x(active, since 3s) 2024-04-07T16:18:25.389 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:18:25 smithi100 bash[21601]: debug 2024-04-07T16:18:25.132+0000 7fcfdfc22200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-07T16:18:25.687 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:18:25 smithi100 bash[21601]: debug 2024-04-07T16:18:25.400+0000 7fcfdfc22200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-07T16:18:25.687 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:18:25 smithi100 bash[21601]: debug 2024-04-07T16:18:25.516+0000 7fcfdfc22200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-07T16:18:25.891 INFO:tasks.rados.rados.0.smithi100.stdout:590: finishing write tid 4 to smithi10066867-38 2024-04-07T16:18:25.891 INFO:tasks.rados.rados.0.smithi100.stdout:590: oid 38 updating version 0 to 125 2024-04-07T16:18:25.891 INFO:tasks.rados.rados.0.smithi100.stdout:590: oid 38 updating version 125 to 126 2024-04-07T16:18:25.891 INFO:tasks.rados.rados.0.smithi100.stdout:590: oid 38 version 126 is already newer than 124 2024-04-07T16:18:25.891 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 38 v 126 (ObjNum 197 snap 52 seq_num 197) dirty exists 2024-04-07T16:18:25.891 INFO:tasks.rados.rados.0.smithi100.stdout:590: left oid 38 (ObjNum 197 snap 52 seq_num 197) 2024-04-07T16:18:25.891 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 26 v 163 (ObjNum 154 snap 42 seq_num 154) dirty exists 2024-04-07T16:18:25.891 INFO:tasks.rados.rados.0.smithi100.stdout:590: done (2 left) 2024-04-07T16:18:25.891 INFO:tasks.rados.rados.0.smithi100.stdout:591: done (1 left) 2024-04-07T16:18:25.892 INFO:tasks.rados.rados.0.smithi100.stdout:593: done (0 left) 2024-04-07T16:18:25.892 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 2 current snap is 52 2024-04-07T16:18:25.892 INFO:tasks.rados.rados.0.smithi100.stdout:594: seq_num 199 ranges {4210688=540672,4751360=401408} 2024-04-07T16:18:25.927 INFO:tasks.rados.rados.0.smithi100.stdout:594: writing smithi10066867-2 from 4210688 to 4751360 tid 1 2024-04-07T16:18:25.931 INFO:tasks.rados.rados.0.smithi100.stdout:594: writing smithi10066867-2 from 4751360 to 5152768 tid 2 2024-04-07T16:18:25.931 INFO:tasks.rados.rados.0.smithi100.stdout:595: snap_create 2024-04-07T16:18:25.933 INFO:tasks.rados.rados.0.smithi100.stdout:594: finishing write tid 1 to smithi10066867-2 2024-04-07T16:18:25.934 INFO:tasks.rados.rados.0.smithi100.stdout:594: finishing write tid 2 to smithi10066867-2 2024-04-07T16:18:25.934 INFO:tasks.rados.rados.0.smithi100.stdout:594: finishing write tid 3 to smithi10066867-2 2024-04-07T16:18:25.936 INFO:tasks.rados.rados.0.smithi100.stdout:594: finishing write tid 4 to smithi10066867-2 2024-04-07T16:18:25.936 INFO:tasks.rados.rados.0.smithi100.stdout:594: oid 2 updating version 0 to 200 2024-04-07T16:18:25.936 INFO:tasks.rados.rados.0.smithi100.stdout:594: oid 2 updating version 200 to 201 2024-04-07T16:18:25.936 INFO:tasks.rados.rados.0.smithi100.stdout:594: oid 2 updating version 201 to 202 2024-04-07T16:18:25.936 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 2 v 202 (ObjNum 199 snap 52 seq_num 199) dirty exists 2024-04-07T16:18:25.936 INFO:tasks.rados.rados.0.smithi100.stdout:594: left oid 2 (ObjNum 199 snap 52 seq_num 199) 2024-04-07T16:18:26.052 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:25 smithi100 bash[21346]: cluster 2024-04-07T16:18:24.641363+0000 mgr.x (mgr.24553) 9 : cluster 0 pgmap v9: 121 pgs: 121 active+clean; 121 MiB data, 513 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:18:26.052 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:25 smithi100 bash[21346]: cluster 2024-04-07T16:18:24.899168+0000 mon.a (mon.0) 1017 : cluster 0 osdmap e171: 8 total, 8 up, 8 in 2024-04-07T16:18:26.052 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:18:25 smithi100 bash[21601]: debug 2024-04-07T16:18:25.796+0000 7fcfdfc22200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-07T16:18:26.052 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:25 smithi100 bash[31660]: cluster 2024-04-07T16:18:24.641363+0000 mgr.x (mgr.24553) 9 : cluster 0 pgmap v9: 121 pgs: 121 active+clean; 121 MiB data, 513 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:18:26.052 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:25 smithi100 bash[31660]: cluster 2024-04-07T16:18:24.899168+0000 mon.a (mon.0) 1017 : cluster 0 osdmap e171: 8 total, 8 up, 8 in 2024-04-07T16:18:26.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:25 smithi144 bash[23522]: cluster 2024-04-07T16:18:24.641363+0000 mgr.x (mgr.24553) 9 : cluster 0 pgmap v9: 121 pgs: 121 active+clean; 121 MiB data, 513 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:18:26.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:25 smithi144 bash[23522]: cluster 2024-04-07T16:18:24.899168+0000 mon.a (mon.0) 1017 : cluster 0 osdmap e171: 8 total, 8 up, 8 in 2024-04-07T16:18:26.437 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:18:26 smithi100 bash[21601]: debug 2024-04-07T16:18:26.064+0000 7fcfdfc22200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-07T16:18:26.842 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:18:26 smithi100 bash[21601]: debug 2024-04-07T16:18:26.476+0000 7fcfdfc22200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-07T16:18:26.842 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:18:26 smithi100 bash[21601]: debug 2024-04-07T16:18:26.600+0000 7fcfdfc22200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-07T16:18:26.842 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:18:26 smithi100 bash[21601]: debug 2024-04-07T16:18:26.852+0000 7fcfdfc22200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-07T16:18:26.905 INFO:tasks.rados.rados.0.smithi100.stdout:594: done (1 left) 2024-04-07T16:18:26.905 INFO:tasks.rados.rados.0.smithi100.stdout:595: done (0 left) 2024-04-07T16:18:26.905 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 27 current snap is 53 2024-04-07T16:18:26.905 INFO:tasks.rados.rados.0.smithi100.stdout:596: seq_num 200 ranges {0=557056,557056=401408} 2024-04-07T16:18:26.909 INFO:tasks.rados.rados.0.smithi100.stdout:596: writing smithi10066867-27 from 0 to 557056 tid 1 2024-04-07T16:18:26.914 INFO:tasks.rados.rados.0.smithi100.stdout:596: writing smithi10066867-27 from 557056 to 958464 tid 2 2024-04-07T16:18:26.914 INFO:tasks.rados.rados.0.smithi100.stdout:597: rollback oid 16 current snap is 53 2024-04-07T16:18:26.914 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 16 to 33 2024-04-07T16:18:26.914 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 32 current snap is 53 2024-04-07T16:18:26.914 INFO:tasks.rados.rados.0.smithi100.stdout:598: seq_num 201 ranges {0=573440,573440=401408} 2024-04-07T16:18:26.920 INFO:tasks.rados.rados.0.smithi100.stdout:598: writing smithi10066867-32 from 0 to 573440 tid 1 2024-04-07T16:18:26.925 INFO:tasks.rados.rados.0.smithi100.stdout:598: writing smithi10066867-32 from 573440 to 974848 tid 2 2024-04-07T16:18:26.925 INFO:tasks.rados.rados.0.smithi100.stdout:599: snap_create 2024-04-07T16:18:26.925 INFO:tasks.rados.rados.0.smithi100.stdout:597: finishing rollback tid 0 to smithi10066867-16 2024-04-07T16:18:26.926 INFO:tasks.rados.rados.0.smithi100.stdout:596: finishing write tid 1 to smithi10066867-27 2024-04-07T16:18:26.926 INFO:tasks.rados.rados.0.smithi100.stdout:597: finishing rollback tid 1 to smithi10066867-16 2024-04-07T16:18:26.926 INFO:tasks.rados.rados.0.smithi100.stdout:596: finishing write tid 2 to smithi10066867-27 2024-04-07T16:18:26.926 INFO:tasks.rados.rados.0.smithi100.stdout:597: finishing rollback tid 2 to smithi10066867-16 2024-04-07T16:18:26.926 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 16 v 243 (ObjNum 118 snap 29 seq_num 118) dirty exists 2024-04-07T16:18:26.926 INFO:tasks.rados.rados.0.smithi100.stdout:596: finishing write tid 3 to smithi10066867-27 2024-04-07T16:18:26.927 INFO:tasks.rados.rados.0.smithi100.stdout:596: finishing write tid 4 to smithi10066867-27 2024-04-07T16:18:26.927 INFO:tasks.rados.rados.0.smithi100.stdout:596: oid 27 updating version 0 to 99 2024-04-07T16:18:26.927 INFO:tasks.rados.rados.0.smithi100.stdout:596: oid 27 version 99 is already newer than 98 2024-04-07T16:18:26.927 INFO:tasks.rados.rados.0.smithi100.stdout:596: oid 27 updating version 99 to 100 2024-04-07T16:18:26.928 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 27 v 100 (ObjNum 200 snap 53 seq_num 200) dirty exists 2024-04-07T16:18:26.928 INFO:tasks.rados.rados.0.smithi100.stdout:596: left oid 27 (ObjNum 200 snap 53 seq_num 200) 2024-04-07T16:18:26.937 INFO:tasks.rados.rados.0.smithi100.stdout:598: finishing write tid 1 to smithi10066867-32 2024-04-07T16:18:26.937 INFO:tasks.rados.rados.0.smithi100.stdout:598: finishing write tid 2 to smithi10066867-32 2024-04-07T16:18:26.938 INFO:tasks.rados.rados.0.smithi100.stdout:598: finishing write tid 3 to smithi10066867-32 2024-04-07T16:18:26.941 INFO:tasks.rados.rados.0.smithi100.stdout:598: finishing write tid 4 to smithi10066867-32 2024-04-07T16:18:26.941 INFO:tasks.rados.rados.0.smithi100.stdout:598: oid 32 updating version 0 to 53 2024-04-07T16:18:26.941 INFO:tasks.rados.rados.0.smithi100.stdout:598: oid 32 version 53 is already newer than 52 2024-04-07T16:18:26.941 INFO:tasks.rados.rados.0.smithi100.stdout:598: oid 32 version 53 is already newer than 51 2024-04-07T16:18:26.941 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 32 v 53 (ObjNum 201 snap 53 seq_num 201) dirty exists 2024-04-07T16:18:26.941 INFO:tasks.rados.rados.0.smithi100.stdout:598: left oid 32 (ObjNum 201 snap 53 seq_num 201) 2024-04-07T16:18:27.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:26 smithi100 bash[21346]: cluster 2024-04-07T16:18:25.905901+0000 mon.a (mon.0) 1018 : cluster 0 osdmap e172: 8 total, 8 up, 8 in 2024-04-07T16:18:27.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:26 smithi100 bash[31660]: cluster 2024-04-07T16:18:25.905901+0000 mon.a (mon.0) 1018 : cluster 0 osdmap e172: 8 total, 8 up, 8 in 2024-04-07T16:18:27.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:26 smithi144 bash[23522]: cluster 2024-04-07T16:18:25.905901+0000 mon.a (mon.0) 1018 : cluster 0 osdmap e172: 8 total, 8 up, 8 in 2024-04-07T16:18:27.906 INFO:tasks.rados.rados.0.smithi100.stdout:596: done (3 left) 2024-04-07T16:18:27.906 INFO:tasks.rados.rados.0.smithi100.stdout:597: done (2 left) 2024-04-07T16:18:27.906 INFO:tasks.rados.rados.0.smithi100.stdout:598: done (1 left) 2024-04-07T16:18:27.906 INFO:tasks.rados.rados.0.smithi100.stdout:599: done (0 left) 2024-04-07T16:18:27.906 INFO:tasks.rados.rados.0.smithi100.stdout:600: setattr oid 7 current snap is 54 2024-04-07T16:18:27.909 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 19 current snap is 54 2024-04-07T16:18:27.909 INFO:tasks.rados.rados.0.smithi100.stdout:601: seq_num 202 ranges {0=589824,589824=401408} 2024-04-07T16:18:27.916 INFO:tasks.rados.rados.0.smithi100.stdout:601: writing smithi10066867-19 from 0 to 589824 tid 1 2024-04-07T16:18:27.922 INFO:tasks.rados.rados.0.smithi100.stdout:601: writing smithi10066867-19 from 589824 to 991232 tid 2 2024-04-07T16:18:27.922 INFO:tasks.rados.rados.0.smithi100.stdout:602: copy_from oid 13 from oid 8 current snap is 54 2024-04-07T16:18:27.922 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 46 current snap is 54 2024-04-07T16:18:27.922 INFO:tasks.rados.rados.0.smithi100.stdout:603: seq_num 203 ranges {761856=606208,1368064=401408} 2024-04-07T16:18:27.936 INFO:tasks.rados.rados.0.smithi100.stdout:603: writing smithi10066867-46 from 761856 to 1368064 tid 1 2024-04-07T16:18:27.941 INFO:tasks.rados.rados.0.smithi100.stdout:603: writing smithi10066867-46 from 1368064 to 1769472 tid 2 2024-04-07T16:18:27.941 INFO:tasks.rados.rados.0.smithi100.stdout:604: read oid 28 snap 43 2024-04-07T16:18:27.941 INFO:tasks.rados.rados.0.smithi100.stdout:604: expect (ObjNum 140 snap 39 seq_num 140) 2024-04-07T16:18:27.941 INFO:tasks.rados.rados.0.smithi100.stdout:605: read oid 15 snap -1 2024-04-07T16:18:27.942 INFO:tasks.rados.rados.0.smithi100.stdout:605: expect (ObjNum 124 snap 29 seq_num 124) 2024-04-07T16:18:27.942 INFO:tasks.rados.rados.0.smithi100.stdout:606: read oid 23 snap -1 2024-04-07T16:18:27.942 INFO:tasks.rados.rados.0.smithi100.stdout:606: expect (ObjNum 80 snap 13 seq_num 80) 2024-04-07T16:18:27.942 INFO:tasks.rados.rados.0.smithi100.stdout:607: snap_create 2024-04-07T16:18:27.942 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 7 v 102 (ObjNum 16 snap 0 seq_num 16) dirty exists 2024-04-07T16:18:27.942 INFO:tasks.rados.rados.0.smithi100.stdout:601: finishing write tid 1 to smithi10066867-19 2024-04-07T16:18:27.942 INFO:tasks.rados.rados.0.smithi100.stdout:601: finishing write tid 2 to smithi10066867-19 2024-04-07T16:18:27.942 INFO:tasks.rados.rados.0.smithi100.stdout:601: finishing write tid 3 to smithi10066867-19 2024-04-07T16:18:27.943 INFO:tasks.rados.rados.0.smithi100.stdout:601: finishing write tid 4 to smithi10066867-19 2024-04-07T16:18:27.943 INFO:tasks.rados.rados.0.smithi100.stdout:601: oid 19 updating version 0 to 206 2024-04-07T16:18:27.943 INFO:tasks.rados.rados.0.smithi100.stdout:601: oid 19 version 206 is already newer than 205 2024-04-07T16:18:27.943 INFO:tasks.rados.rados.0.smithi100.stdout:601: oid 19 updating version 206 to 207 2024-04-07T16:18:27.943 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 19 v 207 (ObjNum 202 snap 54 seq_num 202) dirty exists 2024-04-07T16:18:27.943 INFO:tasks.rados.rados.0.smithi100.stdout:601: left oid 19 (ObjNum 202 snap 54 seq_num 202) 2024-04-07T16:18:27.943 INFO:tasks.rados.rados.0.smithi100.stdout:602: finishing copy_from racing read to smithi10066867-13 2024-04-07T16:18:27.944 INFO:tasks.rados.rados.0.smithi100.stdout:602: finishing copy_from to smithi10066867-13 2024-04-07T16:18:27.944 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 13 v 171 (ObjNum 7 snap 0 seq_num 7) dirty exists 2024-04-07T16:18:27.945 INFO:tasks.rados.rados.0.smithi100.stdout:603: finishing write tid 1 to smithi10066867-46 2024-04-07T16:18:27.946 INFO:tasks.rados.rados.0.smithi100.stdout:603: finishing write tid 2 to smithi10066867-46 2024-04-07T16:18:27.948 INFO:tasks.rados.rados.0.smithi100.stdout:603: finishing write tid 3 to smithi10066867-46 2024-04-07T16:18:27.951 INFO:tasks.rados.rados.0.smithi100.stdout:603: finishing write tid 4 to smithi10066867-46 2024-04-07T16:18:27.951 INFO:tasks.rados.rados.0.smithi100.stdout:603: oid 46 updating version 0 to 177 2024-04-07T16:18:27.951 INFO:tasks.rados.rados.0.smithi100.stdout:603: oid 46 updating version 177 to 178 2024-04-07T16:18:27.951 INFO:tasks.rados.rados.0.smithi100.stdout:603: oid 46 updating version 178 to 179 2024-04-07T16:18:27.951 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 46 v 179 (ObjNum 203 snap 54 seq_num 203) dirty exists 2024-04-07T16:18:27.951 INFO:tasks.rados.rados.0.smithi100.stdout:603: left oid 46 (ObjNum 203 snap 54 seq_num 203) 2024-04-07T16:18:27.967 INFO:tasks.rados.rados.0.smithi100.stdout:604: expect (ObjNum 140 snap 39 seq_num 140) 2024-04-07T16:18:28.066 INFO:tasks.rados.rados.0.smithi100.stdout:605: expect (ObjNum 124 snap 29 seq_num 124) 2024-04-07T16:18:28.187 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:18:27 smithi100 bash[21601]: debug 2024-04-07T16:18:27.884+0000 7fcfdfc22200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-07T16:18:28.187 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:18:28 smithi100 bash[21601]: debug 2024-04-07T16:18:28.020+0000 7fcfdfc22200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-07T16:18:28.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:27 smithi100 bash[31660]: cluster 2024-04-07T16:18:26.642052+0000 mgr.x (mgr.24553) 10 : cluster 0 pgmap v12: 121 pgs: 121 active+clean; 121 MiB data, 513 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:18:28.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:27 smithi100 bash[31660]: cluster 2024-04-07T16:18:26.916959+0000 mon.a (mon.0) 1019 : cluster 0 osdmap e173: 8 total, 8 up, 8 in 2024-04-07T16:18:28.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:27 smithi100 bash[21346]: cluster 2024-04-07T16:18:26.642052+0000 mgr.x (mgr.24553) 10 : cluster 0 pgmap v12: 121 pgs: 121 active+clean; 121 MiB data, 513 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:18:28.188 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:27 smithi100 bash[21346]: cluster 2024-04-07T16:18:26.916959+0000 mon.a (mon.0) 1019 : cluster 0 osdmap e173: 8 total, 8 up, 8 in 2024-04-07T16:18:28.198 INFO:tasks.rados.rados.0.smithi100.stdout:606: expect (ObjNum 80 snap 13 seq_num 80) 2024-04-07T16:18:28.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:27 smithi144 bash[23522]: cluster 2024-04-07T16:18:26.642052+0000 mgr.x (mgr.24553) 10 : cluster 0 pgmap v12: 121 pgs: 121 active+clean; 121 MiB data, 513 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:18:28.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:27 smithi144 bash[23522]: cluster 2024-04-07T16:18:26.916959+0000 mon.a (mon.0) 1019 : cluster 0 osdmap e173: 8 total, 8 up, 8 in 2024-04-07T16:18:28.687 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:18:28 smithi100 bash[21601]: debug 2024-04-07T16:18:28.324+0000 7fcfdfc22200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-07T16:18:28.687 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:18:28 smithi100 bash[21601]: debug 2024-04-07T16:18:28.440+0000 7fcfdfc22200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-07T16:18:28.920 INFO:tasks.rados.rados.0.smithi100.stdout:600: done (7 left) 2024-04-07T16:18:28.920 INFO:tasks.rados.rados.0.smithi100.stdout:601: done (6 left) 2024-04-07T16:18:28.920 INFO:tasks.rados.rados.0.smithi100.stdout:602: done (5 left) 2024-04-07T16:18:28.920 INFO:tasks.rados.rados.0.smithi100.stdout:603: done (4 left) 2024-04-07T16:18:28.921 INFO:tasks.rados.rados.0.smithi100.stdout:604: done (3 left) 2024-04-07T16:18:28.921 INFO:tasks.rados.rados.0.smithi100.stdout:605: done (2 left) 2024-04-07T16:18:28.921 INFO:tasks.rados.rados.0.smithi100.stdout:606: done (1 left) 2024-04-07T16:18:28.921 INFO:tasks.rados.rados.0.smithi100.stdout:607: done (0 left) 2024-04-07T16:18:28.921 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 39 current snap is 55 2024-04-07T16:18:28.921 INFO:tasks.rados.rados.0.smithi100.stdout:608: seq_num 204 ranges {1990656=622592,2613248=401408} 2024-04-07T16:18:28.939 INFO:tasks.rados.rados.0.smithi100.stdout:608: writing smithi10066867-39 from 1990656 to 2613248 tid 1 2024-04-07T16:18:28.943 INFO:tasks.rados.rados.0.smithi100.stdout:608: writing smithi10066867-39 from 2613248 to 3014656 tid 2 2024-04-07T16:18:28.943 INFO:tasks.rados.rados.0.smithi100.stdout:609: read oid 48 snap -1 2024-04-07T16:18:28.943 INFO:tasks.rados.rados.0.smithi100.stdout:609: expect deleted 2024-04-07T16:18:28.943 INFO:tasks.rados.rados.0.smithi100.stdout:610: rollback oid 18 current snap is 55 2024-04-07T16:18:28.943 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 18 to 48 2024-04-07T16:18:28.943 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 44 current snap is 55 2024-04-07T16:18:28.943 INFO:tasks.rados.rados.0.smithi100.stdout:611: seq_num 205 ranges {958464=638976,1597440=401408} 2024-04-07T16:18:28.953 INFO:tasks.rados.rados.0.smithi100.stdout:611: writing smithi10066867-44 from 958464 to 1597440 tid 1 2024-04-07T16:18:28.957 INFO:tasks.rados.rados.0.smithi100.stdout:611: writing smithi10066867-44 from 1597440 to 1998848 tid 2 2024-04-07T16:18:28.957 INFO:tasks.rados.rados.0.smithi100.stdout:612: snap_create 2024-04-07T16:18:28.957 INFO:tasks.rados.rados.0.smithi100.stdout:610: finishing rollback tid 1 to smithi10066867-18 2024-04-07T16:18:28.957 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 18 v 255 (ObjNum 161 snap 0 seq_num 2702298992) dirty dne 2024-04-07T16:18:28.958 INFO:tasks.rados.rados.0.smithi100.stdout:608: finishing write tid 1 to smithi10066867-39 2024-04-07T16:18:28.959 INFO:tasks.rados.rados.0.smithi100.stdout:608: finishing write tid 2 to smithi10066867-39 2024-04-07T16:18:28.959 INFO:tasks.rados.rados.0.smithi100.stdout:608: finishing write tid 3 to smithi10066867-39 2024-04-07T16:18:28.960 INFO:tasks.rados.rados.0.smithi100.stdout:611: finishing write tid 1 to smithi10066867-44 2024-04-07T16:18:28.961 INFO:tasks.rados.rados.0.smithi100.stdout:611: finishing write tid 2 to smithi10066867-44 2024-04-07T16:18:28.961 INFO:tasks.rados.rados.0.smithi100.stdout:608: finishing write tid 4 to smithi10066867-39 2024-04-07T16:18:28.961 INFO:tasks.rados.rados.0.smithi100.stdout:608: oid 39 updating version 0 to 215 2024-04-07T16:18:28.961 INFO:tasks.rados.rados.0.smithi100.stdout:608: oid 39 version 215 is already newer than 214 2024-04-07T16:18:28.962 INFO:tasks.rados.rados.0.smithi100.stdout:608: oid 39 version 215 is already newer than 213 2024-04-07T16:18:28.962 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 39 v 215 (ObjNum 204 snap 55 seq_num 204) dirty exists 2024-04-07T16:18:28.962 INFO:tasks.rados.rados.0.smithi100.stdout:608: left oid 39 (ObjNum 204 snap 55 seq_num 204) 2024-04-07T16:18:28.962 INFO:tasks.rados.rados.0.smithi100.stdout:611: finishing write tid 3 to smithi10066867-44 2024-04-07T16:18:28.962 INFO:tasks.rados.rados.0.smithi100.stdout:611: finishing write tid 4 to smithi10066867-44 2024-04-07T16:18:28.963 INFO:tasks.rados.rados.0.smithi100.stdout:611: oid 44 updating version 0 to 205 2024-04-07T16:18:28.963 INFO:tasks.rados.rados.0.smithi100.stdout:611: oid 44 version 205 is already newer than 204 2024-04-07T16:18:28.963 INFO:tasks.rados.rados.0.smithi100.stdout:611: oid 44 updating version 205 to 206 2024-04-07T16:18:28.963 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 44 v 206 (ObjNum 205 snap 55 seq_num 205) dirty exists 2024-04-07T16:18:28.963 INFO:tasks.rados.rados.0.smithi100.stdout:611: left oid 44 (ObjNum 205 snap 55 seq_num 205) 2024-04-07T16:18:29.156 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:28 smithi100 bash[21346]: cluster 2024-04-07T16:18:27.919499+0000 mon.a (mon.0) 1020 : cluster 0 osdmap e174: 8 total, 8 up, 8 in 2024-04-07T16:18:29.156 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:28 smithi100 bash[31660]: cluster 2024-04-07T16:18:27.919499+0000 mon.a (mon.0) 1020 : cluster 0 osdmap e174: 8 total, 8 up, 8 in 2024-04-07T16:18:29.156 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:18:28 smithi100 bash[21601]: debug 2024-04-07T16:18:28.908+0000 7fcfdfc22200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-07T16:18:29.157 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:18:29 smithi100 bash[21601]: debug 2024-04-07T16:18:29.060+0000 7fcfdfc22200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-07T16:18:29.157 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:18:29 smithi100 bash[21601]: [07/Apr/2024:16:18:29] ENGINE Bus STARTING 2024-04-07T16:18:29.157 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:18:29 smithi100 bash[21601]: CherryPy Checker: 2024-04-07T16:18:29.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:28 smithi144 bash[23522]: cluster 2024-04-07T16:18:27.919499+0000 mon.a (mon.0) 1020 : cluster 0 osdmap e174: 8 total, 8 up, 8 in 2024-04-07T16:18:29.437 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:18:29 smithi100 bash[21601]: The Application mounted at '' has an empty config. 2024-04-07T16:18:29.437 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:18:29 smithi100 bash[21601]: [07/Apr/2024:16:18:29] ENGINE Serving on http://:::9283 2024-04-07T16:18:29.437 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:18:29 smithi100 bash[21601]: [07/Apr/2024:16:18:29] ENGINE Bus STARTED 2024-04-07T16:18:29.954 INFO:tasks.rados.rados.0.smithi100.stdout:608: done (4 left) 2024-04-07T16:18:29.955 INFO:tasks.rados.rados.0.smithi100.stdout:609: done (3 left) 2024-04-07T16:18:29.955 INFO:tasks.rados.rados.0.smithi100.stdout:610: done (2 left) 2024-04-07T16:18:29.955 INFO:tasks.rados.rados.0.smithi100.stdout:611: done (1 left) 2024-04-07T16:18:29.955 INFO:tasks.rados.rados.0.smithi100.stdout:612: done (0 left) 2024-04-07T16:18:29.955 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 18 current snap is 56 2024-04-07T16:18:29.955 INFO:tasks.rados.rados.0.smithi100.stdout:613: seq_num 206 ranges {0=655360,655360=401408} 2024-04-07T16:18:29.960 INFO:tasks.rados.rados.0.smithi100.stdout:613: writing smithi10066867-18 from 0 to 655360 tid 1 2024-04-07T16:18:29.963 INFO:tasks.rados.rados.0.smithi100.stdout:613: writing smithi10066867-18 from 655360 to 1056768 tid 2 2024-04-07T16:18:29.963 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 39 current snap is 56 2024-04-07T16:18:29.964 INFO:tasks.rados.rados.0.smithi100.stdout:614: seq_num 207 ranges {3014656=671744,3686400=401408} 2024-04-07T16:18:29.994 INFO:tasks.rados.rados.0.smithi100.stdout:614: writing smithi10066867-39 from 3014656 to 3686400 tid 1 2024-04-07T16:18:29.998 INFO:tasks.rados.rados.0.smithi100.stdout:614: writing smithi10066867-39 from 3686400 to 4087808 tid 2 2024-04-07T16:18:29.998 INFO:tasks.rados.rados.0.smithi100.stdout:615: setattr oid 49 current snap is 56 2024-04-07T16:18:29.998 INFO:tasks.rados.rados.0.smithi100.stdout:613: finishing write tid 1 to smithi10066867-18 2024-04-07T16:18:29.998 INFO:tasks.rados.rados.0.smithi100.stdout:613: finishing write tid 2 to smithi10066867-18 2024-04-07T16:18:29.998 INFO:tasks.rados.rados.0.smithi100.stdout:613: finishing write tid 3 to smithi10066867-18 2024-04-07T16:18:29.998 INFO:tasks.rados.rados.0.smithi100.stdout:613: finishing write tid 4 to smithi10066867-18 2024-04-07T16:18:29.998 INFO:tasks.rados.rados.0.smithi100.stdout:613: oid 18 updating version 0 to 313 2024-04-07T16:18:29.998 INFO:tasks.rados.rados.0.smithi100.stdout:613: oid 18 updating version 313 to 315 2024-04-07T16:18:29.999 INFO:tasks.rados.rados.0.smithi100.stdout:613: oid 18 version 315 is already newer than 314 2024-04-07T16:18:29.999 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 18 v 315 (ObjNum 206 snap 56 seq_num 206) dirty exists 2024-04-07T16:18:29.999 INFO:tasks.rados.rados.0.smithi100.stdout:613: left oid 18 (ObjNum 206 snap 56 seq_num 206) 2024-04-07T16:18:30.002 INFO:tasks.rados.rados.0.smithi100.stdout:613: done (2 left) 2024-04-07T16:18:30.002 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 8 current snap is 56 2024-04-07T16:18:30.002 INFO:tasks.rados.rados.0.smithi100.stdout:616: seq_num 208 ranges {122880=688128,811008=401408} 2024-04-07T16:18:30.009 INFO:tasks.rados.rados.0.smithi100.stdout:616: writing smithi10066867-8 from 122880 to 811008 tid 1 2024-04-07T16:18:30.012 INFO:tasks.rados.rados.0.smithi100.stdout:616: writing smithi10066867-8 from 811008 to 1212416 tid 2 2024-04-07T16:18:30.012 INFO:tasks.rados.rados.0.smithi100.stdout:617: rollback oid 23 current snap is 56 2024-04-07T16:18:30.012 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 23 to 51 2024-04-07T16:18:30.012 INFO:tasks.rados.rados.0.smithi100.stdout:614: finishing write tid 1 to smithi10066867-39 2024-04-07T16:18:30.012 INFO:tasks.rados.rados.0.smithi100.stdout:618: copy_from oid 9 from oid 18 current snap is 56 2024-04-07T16:18:30.013 INFO:tasks.rados.rados.0.smithi100.stdout:614: finishing write tid 2 to smithi10066867-39 2024-04-07T16:18:30.013 INFO:tasks.rados.rados.0.smithi100.stdout:614: finishing write tid 3 to smithi10066867-39 2024-04-07T16:18:30.013 INFO:tasks.rados.rados.0.smithi100.stdout:614: finishing write tid 4 to smithi10066867-39 2024-04-07T16:18:30.013 INFO:tasks.rados.rados.0.smithi100.stdout:614: oid 39 updating version 0 to 218 2024-04-07T16:18:30.013 INFO:tasks.rados.rados.0.smithi100.stdout:614: oid 39 version 218 is already newer than 217 2024-04-07T16:18:30.013 INFO:tasks.rados.rados.0.smithi100.stdout:614: oid 39 updating version 218 to 219 2024-04-07T16:18:30.013 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 39 v 219 (ObjNum 207 snap 56 seq_num 207) dirty exists 2024-04-07T16:18:30.013 INFO:tasks.rados.rados.0.smithi100.stdout:614: left oid 39 (ObjNum 207 snap 56 seq_num 207) 2024-04-07T16:18:30.013 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 49 v 245 (ObjNum 192 snap 50 seq_num 192) dirty exists 2024-04-07T16:18:30.013 INFO:tasks.rados.rados.0.smithi100.stdout:614: done (4 left) 2024-04-07T16:18:30.013 INFO:tasks.rados.rados.0.smithi100.stdout:615: done (3 left) 2024-04-07T16:18:30.013 INFO:tasks.rados.rados.0.smithi100.stdout:619: snap_remove snap 55 2024-04-07T16:18:30.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:29 smithi144 bash[23522]: cluster 2024-04-07T16:18:28.642739+0000 mgr.x (mgr.24553) 11 : cluster 0 pgmap v15: 121 pgs: 121 active+clean; 126 MiB data, 556 MiB used, 715 GiB / 715 GiB avail; 2.0 MiB/s rd, 1.2 MiB/s wr, 44 op/s 2024-04-07T16:18:30.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:29 smithi144 bash[23522]: cluster 2024-04-07T16:18:28.933317+0000 mon.a (mon.0) 1021 : cluster 0 osdmap e175: 8 total, 8 up, 8 in 2024-04-07T16:18:30.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:29 smithi144 bash[23522]: cluster 2024-04-07T16:18:29.066327+0000 mon.a (mon.0) 1022 : cluster 0 Standby manager daemon y started 2024-04-07T16:18:30.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:29 smithi144 bash[23522]: audit 2024-04-07T16:18:29.067610+0000 mon.c (mon.2) 53 : audit 0 from='mgr.? 172.21.15.100:0/3739310115' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-07T16:18:30.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:29 smithi144 bash[23522]: audit 2024-04-07T16:18:29.068297+0000 mon.c (mon.2) 54 : audit 0 from='mgr.? 172.21.15.100:0/3739310115' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-07T16:18:30.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:29 smithi144 bash[23522]: audit 2024-04-07T16:18:29.071326+0000 mon.c (mon.2) 55 : audit 0 from='mgr.? 172.21.15.100:0/3739310115' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-07T16:18:30.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:29 smithi144 bash[23522]: audit 2024-04-07T16:18:29.071882+0000 mon.c (mon.2) 56 : audit 0 from='mgr.? 172.21.15.100:0/3739310115' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-07T16:18:30.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:29 smithi100 bash[21346]: cluster 2024-04-07T16:18:28.642739+0000 mgr.x (mgr.24553) 11 : cluster 0 pgmap v15: 121 pgs: 121 active+clean; 126 MiB data, 556 MiB used, 715 GiB / 715 GiB avail; 2.0 MiB/s rd, 1.2 MiB/s wr, 44 op/s 2024-04-07T16:18:30.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:29 smithi100 bash[21346]: cluster 2024-04-07T16:18:28.933317+0000 mon.a (mon.0) 1021 : cluster 0 osdmap e175: 8 total, 8 up, 8 in 2024-04-07T16:18:30.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:29 smithi100 bash[21346]: cluster 2024-04-07T16:18:29.066327+0000 mon.a (mon.0) 1022 : cluster 0 Standby manager daemon y started 2024-04-07T16:18:30.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:29 smithi100 bash[21346]: audit 2024-04-07T16:18:29.067610+0000 mon.c (mon.2) 53 : audit 0 from='mgr.? 172.21.15.100:0/3739310115' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-07T16:18:30.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:29 smithi100 bash[21346]: audit 2024-04-07T16:18:29.068297+0000 mon.c (mon.2) 54 : audit 0 from='mgr.? 172.21.15.100:0/3739310115' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-07T16:18:30.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:29 smithi100 bash[21346]: audit 2024-04-07T16:18:29.071326+0000 mon.c (mon.2) 55 : audit 0 from='mgr.? 172.21.15.100:0/3739310115' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-07T16:18:30.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:29 smithi100 bash[21346]: audit 2024-04-07T16:18:29.071882+0000 mon.c (mon.2) 56 : audit 0 from='mgr.? 172.21.15.100:0/3739310115' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-07T16:18:30.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:29 smithi100 bash[31660]: cluster 2024-04-07T16:18:28.642739+0000 mgr.x (mgr.24553) 11 : cluster 0 pgmap v15: 121 pgs: 121 active+clean; 126 MiB data, 556 MiB used, 715 GiB / 715 GiB avail; 2.0 MiB/s rd, 1.2 MiB/s wr, 44 op/s 2024-04-07T16:18:30.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:29 smithi100 bash[31660]: cluster 2024-04-07T16:18:28.933317+0000 mon.a (mon.0) 1021 : cluster 0 osdmap e175: 8 total, 8 up, 8 in 2024-04-07T16:18:30.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:29 smithi100 bash[31660]: cluster 2024-04-07T16:18:29.066327+0000 mon.a (mon.0) 1022 : cluster 0 Standby manager daemon y started 2024-04-07T16:18:30.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:29 smithi100 bash[31660]: audit 2024-04-07T16:18:29.067610+0000 mon.c (mon.2) 53 : audit 0 from='mgr.? 172.21.15.100:0/3739310115' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-07T16:18:30.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:29 smithi100 bash[31660]: audit 2024-04-07T16:18:29.068297+0000 mon.c (mon.2) 54 : audit 0 from='mgr.? 172.21.15.100:0/3739310115' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-07T16:18:30.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:29 smithi100 bash[31660]: audit 2024-04-07T16:18:29.071326+0000 mon.c (mon.2) 55 : audit 0 from='mgr.? 172.21.15.100:0/3739310115' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-07T16:18:30.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:29 smithi100 bash[31660]: audit 2024-04-07T16:18:29.071882+0000 mon.c (mon.2) 56 : audit 0 from='mgr.? 172.21.15.100:0/3739310115' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-07T16:18:30.967 INFO:tasks.rados.rados.0.smithi100.stdout:617: finishing rollback tid 0 to smithi10066867-23 2024-04-07T16:18:30.967 INFO:tasks.rados.rados.0.smithi100.stdout:617: finishing rollback tid 1 to smithi10066867-23 2024-04-07T16:18:30.967 INFO:tasks.rados.rados.0.smithi100.stdout:617: finishing rollback tid 2 to smithi10066867-23 2024-04-07T16:18:30.967 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 23 v 319 (ObjNum 80 snap 13 seq_num 80) dirty exists 2024-04-07T16:18:30.967 INFO:tasks.rados.rados.0.smithi100.stdout:616: finishing write tid 1 to smithi10066867-8 2024-04-07T16:18:30.967 INFO:tasks.rados.rados.0.smithi100.stdout:616: finishing write tid 2 to smithi10066867-8 2024-04-07T16:18:30.967 INFO:tasks.rados.rados.0.smithi100.stdout:616: finishing write tid 3 to smithi10066867-8 2024-04-07T16:18:30.967 INFO:tasks.rados.rados.0.smithi100.stdout:616: finishing write tid 4 to smithi10066867-8 2024-04-07T16:18:30.967 INFO:tasks.rados.rados.0.smithi100.stdout:616: oid 8 updating version 0 to 183 2024-04-07T16:18:30.968 INFO:tasks.rados.rados.0.smithi100.stdout:616: oid 8 version 183 is already newer than 181 2024-04-07T16:18:30.968 INFO:tasks.rados.rados.0.smithi100.stdout:616: oid 8 version 183 is already newer than 182 2024-04-07T16:18:30.968 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 8 v 183 (ObjNum 208 snap 56 seq_num 208) dirty exists 2024-04-07T16:18:30.968 INFO:tasks.rados.rados.0.smithi100.stdout:616: left oid 8 (ObjNum 208 snap 56 seq_num 208) 2024-04-07T16:18:30.968 INFO:tasks.rados.rados.0.smithi100.stdout:618: finishing copy_from racing read to smithi10066867-9 2024-04-07T16:18:30.968 INFO:tasks.rados.rados.0.smithi100.stdout:618: finishing copy_from to smithi10066867-9 2024-04-07T16:18:30.968 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 9 v 315 (ObjNum 206 snap 56 seq_num 206) dirty exists 2024-04-07T16:18:30.968 INFO:tasks.rados.rados.0.smithi100.stdout:616: done (3 left) 2024-04-07T16:18:30.968 INFO:tasks.rados.rados.0.smithi100.stdout:617: done (2 left) 2024-04-07T16:18:30.968 INFO:tasks.rados.rados.0.smithi100.stdout:618: done (1 left) 2024-04-07T16:18:30.968 INFO:tasks.rados.rados.0.smithi100.stdout:619: done (0 left) 2024-04-07T16:18:30.968 INFO:tasks.rados.rados.0.smithi100.stdout:620: copy_from oid 21 from oid 20 current snap is 56 2024-04-07T16:18:30.968 INFO:tasks.rados.rados.0.smithi100.stdout:621: read oid 44 snap 52 2024-04-07T16:18:30.968 INFO:tasks.rados.rados.0.smithi100.stdout:621: expect (ObjNum 165 snap 44 seq_num 165) 2024-04-07T16:18:30.968 INFO:tasks.rados.rados.0.smithi100.stdout:622: copy_from oid 46 from oid 10 current snap is 56 2024-04-07T16:18:30.969 INFO:tasks.rados.rados.0.smithi100.stdout:623: delete oid 19 current snap is 56 2024-04-07T16:18:30.975 INFO:tasks.rados.rados.0.smithi100.stdout:622: finishing copy_from to smithi10066867-46 2024-04-07T16:18:30.976 INFO:tasks.rados.rados.0.smithi100.stdout:622: got expected ENOENT (src dne) 2024-04-07T16:18:30.976 INFO:tasks.rados.rados.0.smithi100.stdout:622: finishing copy_from racing read to smithi10066867-46 2024-04-07T16:18:30.984 INFO:tasks.rados.rados.0.smithi100.stdout:622: done (3 left) 2024-04-07T16:18:30.984 INFO:tasks.rados.rados.0.smithi100.stdout:623: done (2 left) 2024-04-07T16:18:30.984 INFO:tasks.rados.rados.0.smithi100.stdout:624: snap_create 2024-04-07T16:18:30.991 INFO:tasks.rados.rados.0.smithi100.stdout:621: expect (ObjNum 165 snap 44 seq_num 165) 2024-04-07T16:18:31.066 INFO:tasks.rados.rados.0.smithi100.stdout:620: finishing copy_from racing read to smithi10066867-21 2024-04-07T16:18:31.066 INFO:tasks.rados.rados.0.smithi100.stdout:620: finishing copy_from to smithi10066867-21 2024-04-07T16:18:31.066 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 21 v 29 (ObjNum 123 snap 29 seq_num 123) dirty exists 2024-04-07T16:18:31.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:30 smithi144 bash[23522]: cluster 2024-04-07T16:18:29.952660+0000 mon.a (mon.0) 1023 : cluster 0 mgrmap e27: x(active, since 9s), standbys: y 2024-04-07T16:18:31.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:30 smithi144 bash[23522]: audit 2024-04-07T16:18:29.955495+0000 mon.b (mon.1) 279 : audit 0 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-07T16:18:31.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:30 smithi144 bash[23522]: cluster 2024-04-07T16:18:29.968458+0000 mon.a (mon.0) 1024 : cluster 0 osdmap e176: 8 total, 8 up, 8 in 2024-04-07T16:18:31.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:30 smithi144 bash[23522]: audit 2024-04-07T16:18:30.827375+0000 mon.b (mon.1) 280 : audit 0 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:18:31.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:30 smithi144 bash[23522]: audit 2024-04-07T16:18:30.846069+0000 mon.b (mon.1) 281 : audit 0 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "config-key dump", "format": "json"}]: dispatch 2024-04-07T16:18:31.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:30 smithi100 bash[21346]: cluster 2024-04-07T16:18:29.952660+0000 mon.a (mon.0) 1023 : cluster 0 mgrmap e27: x(active, since 9s), standbys: y 2024-04-07T16:18:31.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:30 smithi100 bash[21346]: audit 2024-04-07T16:18:29.955495+0000 mon.b (mon.1) 279 : audit 0 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-07T16:18:31.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:30 smithi100 bash[21346]: cluster 2024-04-07T16:18:29.968458+0000 mon.a (mon.0) 1024 : cluster 0 osdmap e176: 8 total, 8 up, 8 in 2024-04-07T16:18:31.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:30 smithi100 bash[21346]: audit 2024-04-07T16:18:30.827375+0000 mon.b (mon.1) 280 : audit 0 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:18:31.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:30 smithi100 bash[21346]: audit 2024-04-07T16:18:30.846069+0000 mon.b (mon.1) 281 : audit 0 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "config-key dump", "format": "json"}]: dispatch 2024-04-07T16:18:31.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:30 smithi100 bash[31660]: cluster 2024-04-07T16:18:29.952660+0000 mon.a (mon.0) 1023 : cluster 0 mgrmap e27: x(active, since 9s), standbys: y 2024-04-07T16:18:31.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:30 smithi100 bash[31660]: audit 2024-04-07T16:18:29.955495+0000 mon.b (mon.1) 279 : audit 0 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-07T16:18:31.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:30 smithi100 bash[31660]: cluster 2024-04-07T16:18:29.968458+0000 mon.a (mon.0) 1024 : cluster 0 osdmap e176: 8 total, 8 up, 8 in 2024-04-07T16:18:31.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:30 smithi100 bash[31660]: audit 2024-04-07T16:18:30.827375+0000 mon.b (mon.1) 280 : audit 0 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:18:31.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:30 smithi100 bash[31660]: audit 2024-04-07T16:18:30.846069+0000 mon.b (mon.1) 281 : audit 0 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "config-key dump", "format": "json"}]: dispatch 2024-04-07T16:18:31.986 INFO:tasks.rados.rados.0.smithi100.stdout:620: done (2 left) 2024-04-07T16:18:31.987 INFO:tasks.rados.rados.0.smithi100.stdout:621: done (1 left) 2024-04-07T16:18:31.987 INFO:tasks.rados.rados.0.smithi100.stdout:624: done (0 left) 2024-04-07T16:18:31.987 INFO:tasks.rados.rados.0.smithi100.stdout:625: read oid 45 snap -1 2024-04-07T16:18:31.987 INFO:tasks.rados.rados.0.smithi100.stdout:625: expect (ObjNum 185 snap 47 seq_num 185) 2024-04-07T16:18:31.987 INFO:tasks.rados.rados.0.smithi100.stdout:626: setattr oid 1 current snap is 57 2024-04-07T16:18:31.990 INFO:tasks.rados.rados.0.smithi100.stdout:627: rmattr oid 34 current snap is 57 2024-04-07T16:18:31.990 INFO:tasks.rados.rados.0.smithi100.stdout:627: done (2 left) 2024-04-07T16:18:31.990 INFO:tasks.rados.rados.0.smithi100.stdout:628: read oid 10 snap -1 2024-04-07T16:18:31.991 INFO:tasks.rados.rados.0.smithi100.stdout:628: expect deleted 2024-04-07T16:18:31.991 INFO:tasks.rados.rados.0.smithi100.stdout:629: rollback oid 46 current snap is 57 2024-04-07T16:18:31.991 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 46 to 43 2024-04-07T16:18:31.991 INFO:tasks.rados.rados.0.smithi100.stdout:630: copy_from oid 25 from oid 44 current snap is 57 2024-04-07T16:18:31.991 INFO:tasks.rados.rados.0.smithi100.stdout:631: rollback oid 41 current snap is 57 2024-04-07T16:18:31.991 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 41 to 43 2024-04-07T16:18:31.991 INFO:tasks.rados.rados.0.smithi100.stdout:632: rollback oid 50 current snap is 57 2024-04-07T16:18:31.991 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 50 to 45 2024-04-07T16:18:31.991 INFO:tasks.rados.rados.0.smithi100.stdout:633: read oid 5 snap -1 2024-04-07T16:18:31.991 INFO:tasks.rados.rados.0.smithi100.stdout:633: expect (ObjNum 0 snap 0 seq_num 0) 2024-04-07T16:18:31.991 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 36 current snap is 57 2024-04-07T16:18:31.991 INFO:tasks.rados.rados.0.smithi100.stdout:634: seq_num 210 ranges {671744=720896,1392640=401408} 2024-04-07T16:18:32.009 INFO:tasks.rados.rados.0.smithi100.stdout:634: writing smithi10066867-36 from 671744 to 1392640 tid 1 2024-04-07T16:18:32.014 INFO:tasks.rados.rados.0.smithi100.stdout:634: writing smithi10066867-36 from 1392640 to 1794048 tid 2 2024-04-07T16:18:32.014 INFO:tasks.rados.rados.0.smithi100.stdout:635: rollback oid 49 current snap is 57 2024-04-07T16:18:32.014 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 49 to 54 2024-04-07T16:18:32.014 INFO:tasks.rados.rados.0.smithi100.stdout:629: finishing rollback tid 0 to smithi10066867-46 2024-04-07T16:18:32.014 INFO:tasks.rados.rados.0.smithi100.stdout:629: finishing rollback tid 1 to smithi10066867-46 2024-04-07T16:18:32.015 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 1 v 55 (ObjNum 129 snap 0 seq_num 2703014016) dirty exists 2024-04-07T16:18:32.015 INFO:tasks.rados.rados.0.smithi100.stdout:629: finishing rollback tid 2 to smithi10066867-46 2024-04-07T16:18:32.015 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 46 v 189 (ObjNum 45 snap 0 seq_num 45) dirty exists 2024-04-07T16:18:32.015 INFO:tasks.rados.rados.0.smithi100.stdout:633: expect (ObjNum 0 snap 0 seq_num 0) 2024-04-07T16:18:32.019 INFO:tasks.rados.rados.0.smithi100.stdout:631: finishing rollback tid 0 to smithi10066867-41 2024-04-07T16:18:32.019 INFO:tasks.rados.rados.0.smithi100.stdout:631: finishing rollback tid 1 to smithi10066867-41 2024-04-07T16:18:32.019 INFO:tasks.rados.rados.0.smithi100.stdout:631: finishing rollback tid 2 to smithi10066867-41 2024-04-07T16:18:32.019 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 41 v 251 (ObjNum 142 snap 39 seq_num 142) dirty exists 2024-04-07T16:18:32.019 INFO:tasks.rados.rados.0.smithi100.stdout:625: expect (ObjNum 185 snap 47 seq_num 185) 2024-04-07T16:18:32.163 INFO:tasks.rados.rados.0.smithi100.stdout:632: finishing rollback tid 0 to smithi10066867-50 2024-04-07T16:18:32.164 INFO:tasks.rados.rados.0.smithi100.stdout:632: finishing rollback tid 1 to smithi10066867-50 2024-04-07T16:18:32.164 INFO:tasks.rados.rados.0.smithi100.stdout:632: finishing rollback tid 2 to smithi10066867-50 2024-04-07T16:18:32.164 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 50 v 127 (ObjNum 49 snap 0 seq_num 49) dirty exists 2024-04-07T16:18:32.164 INFO:tasks.rados.rados.0.smithi100.stdout:630: finishing copy_from racing read to smithi10066867-25 2024-04-07T16:18:32.164 INFO:tasks.rados.rados.0.smithi100.stdout:630: finishing copy_from to smithi10066867-25 2024-04-07T16:18:32.164 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 25 v 206 (ObjNum 205 snap 55 seq_num 205) dirty exists 2024-04-07T16:18:32.164 INFO:tasks.rados.rados.0.smithi100.stdout:635: finishing rollback tid 0 to smithi10066867-49 2024-04-07T16:18:32.164 INFO:tasks.rados.rados.0.smithi100.stdout:635: finishing rollback tid 1 to smithi10066867-49 2024-04-07T16:18:32.164 INFO:tasks.rados.rados.0.smithi100.stdout:635: finishing rollback tid 2 to smithi10066867-49 2024-04-07T16:18:32.164 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 49 v 255 (ObjNum 192 snap 50 seq_num 192) dirty exists 2024-04-07T16:18:32.164 INFO:tasks.rados.rados.0.smithi100.stdout:634: finishing write tid 1 to smithi10066867-36 2024-04-07T16:18:32.164 INFO:tasks.rados.rados.0.smithi100.stdout:634: finishing write tid 2 to smithi10066867-36 2024-04-07T16:18:32.164 INFO:tasks.rados.rados.0.smithi100.stdout:634: finishing write tid 3 to smithi10066867-36 2024-04-07T16:18:32.165 INFO:tasks.rados.rados.0.smithi100.stdout:634: finishing write tid 4 to smithi10066867-36 2024-04-07T16:18:32.165 INFO:tasks.rados.rados.0.smithi100.stdout:634: oid 36 updating version 0 to 211 2024-04-07T16:18:32.165 INFO:tasks.rados.rados.0.smithi100.stdout:634: oid 36 updating version 211 to 213 2024-04-07T16:18:32.165 INFO:tasks.rados.rados.0.smithi100.stdout:634: oid 36 version 213 is already newer than 212 2024-04-07T16:18:32.165 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 36 v 213 (ObjNum 210 snap 57 seq_num 210) dirty exists 2024-04-07T16:18:32.165 INFO:tasks.rados.rados.0.smithi100.stdout:634: left oid 36 (ObjNum 210 snap 57 seq_num 210) 2024-04-07T16:18:32.165 INFO:tasks.rados.rados.0.smithi100.stdout:625: done (9 left) 2024-04-07T16:18:32.165 INFO:tasks.rados.rados.0.smithi100.stdout:626: done (8 left) 2024-04-07T16:18:32.165 INFO:tasks.rados.rados.0.smithi100.stdout:628: done (7 left) 2024-04-07T16:18:32.165 INFO:tasks.rados.rados.0.smithi100.stdout:629: done (6 left) 2024-04-07T16:18:32.165 INFO:tasks.rados.rados.0.smithi100.stdout:630: done (5 left) 2024-04-07T16:18:32.165 INFO:tasks.rados.rados.0.smithi100.stdout:631: done (4 left) 2024-04-07T16:18:32.165 INFO:tasks.rados.rados.0.smithi100.stdout:632: done (3 left) 2024-04-07T16:18:32.165 INFO:tasks.rados.rados.0.smithi100.stdout:633: done (2 left) 2024-04-07T16:18:32.165 INFO:tasks.rados.rados.0.smithi100.stdout:634: done (1 left) 2024-04-07T16:18:32.166 INFO:tasks.rados.rados.0.smithi100.stdout:635: done (0 left) 2024-04-07T16:18:32.166 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 15 current snap is 57 2024-04-07T16:18:32.166 INFO:tasks.rados.rados.0.smithi100.stdout:636: seq_num 211 ranges {3891200=737280,4628480=401408} 2024-04-07T16:18:32.227 INFO:tasks.rados.rados.0.smithi100.stdout:636: writing smithi10066867-15 from 3891200 to 4628480 tid 1 2024-04-07T16:18:32.231 INFO:tasks.rados.rados.0.smithi100.stdout:636: writing smithi10066867-15 from 4628480 to 5029888 tid 2 2024-04-07T16:18:32.231 INFO:tasks.rados.rados.0.smithi100.stdout:637: rollback oid 10 current snap is 57 2024-04-07T16:18:32.232 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 10 to 42 2024-04-07T16:18:32.232 INFO:tasks.rados.rados.0.smithi100.stdout:638: rollback oid 4 current snap is 57 2024-04-07T16:18:32.232 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 4 to 56 2024-04-07T16:18:32.232 INFO:tasks.rados.rados.0.smithi100.stdout:639: read oid 2 snap 45 2024-04-07T16:18:32.232 INFO:tasks.rados.rados.0.smithi100.stdout:639: expect (ObjNum 108 snap 20 seq_num 108) 2024-04-07T16:18:32.232 INFO:tasks.rados.rados.0.smithi100.stdout:640: copy_from oid 18 from oid 44 current snap is 57 2024-04-07T16:18:32.232 INFO:tasks.rados.rados.0.smithi100.stdout:641: snap_remove snap 43 2024-04-07T16:18:32.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:31 smithi144 bash[23522]: cluster 2024-04-07T16:18:30.643440+0000 mgr.x (mgr.24553) 12 : cluster 0 pgmap v18: 121 pgs: 121 active+clean; 126 MiB data, 556 MiB used, 715 GiB / 715 GiB avail; 2.0 MiB/s rd, 1.2 MiB/s wr, 44 op/s 2024-04-07T16:18:32.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:31 smithi144 bash[23522]: cluster 2024-04-07T16:18:30.977268+0000 mon.a (mon.0) 1025 : cluster 0 osdmap e177: 8 total, 8 up, 8 in 2024-04-07T16:18:32.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:31 smithi144 bash[23522]: audit 2024-04-07T16:18:31.281185+0000 mon.b (mon.1) 282 : audit 0 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:18:32.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:31 smithi144 bash[23522]: audit 2024-04-07T16:18:31.281767+0000 mon.b (mon.1) 283 : audit 0 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:18:32.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:31 smithi144 bash[23522]: audit 2024-04-07T16:18:31.282126+0000 mon.b (mon.1) 284 : audit 0 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:18:32.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:31 smithi144 bash[23522]: audit 2024-04-07T16:18:31.285287+0000 mon.b (mon.1) 285 : audit 0 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:18:32.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:31 smithi144 bash[23522]: audit 2024-04-07T16:18:31.285934+0000 mon.b (mon.1) 286 : audit 0 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:18:32.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:31 smithi144 bash[23522]: audit 2024-04-07T16:18:31.286398+0000 mon.b (mon.1) 287 : audit 0 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:18:32.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:31 smithi144 bash[23522]: audit 2024-04-07T16:18:31.289633+0000 mon.b (mon.1) 288 : audit 0 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:18:32.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:31 smithi144 bash[23522]: audit 2024-04-07T16:18:31.290206+0000 mon.b (mon.1) 289 : audit 0 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:18:32.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:31 smithi144 bash[23522]: audit 2024-04-07T16:18:31.290591+0000 mon.b (mon.1) 290 : audit 0 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:18:32.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:31 smithi144 bash[23522]: audit 2024-04-07T16:18:31.311972+0000 mon.b (mon.1) 291 : audit 0 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:18:32.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:31 smithi144 bash[23522]: audit 2024-04-07T16:18:31.312527+0000 mon.b (mon.1) 292 : audit 0 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:18:32.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:31 smithi144 bash[23522]: audit 2024-04-07T16:18:31.312865+0000 mon.b (mon.1) 293 : audit 0 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:18:32.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:31 smithi144 bash[23522]: audit 2024-04-07T16:18:31.315318+0000 mon.b (mon.1) 294 : audit 1 from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T16:18:32.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:31 smithi144 bash[23522]: audit 2024-04-07T16:18:31.315421+0000 mon.b (mon.1) 295 : audit 1 from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T16:18:32.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:31 smithi144 bash[23522]: audit 2024-04-07T16:18:31.315667+0000 mon.a (mon.0) 1026 : audit 1 from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T16:18:32.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:31 smithi144 bash[23522]: audit 2024-04-07T16:18:31.315789+0000 mon.a (mon.0) 1027 : audit 1 from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T16:18:32.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:31 smithi144 bash[23522]: audit 2024-04-07T16:18:31.316015+0000 mon.b (mon.1) 296 : audit 0 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:18:32.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:31 smithi144 bash[23522]: audit 2024-04-07T16:18:31.316489+0000 mon.b (mon.1) 297 : audit 0 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:18:32.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:31 smithi144 bash[23522]: audit 2024-04-07T16:18:31.316872+0000 mon.b (mon.1) 298 : audit 0 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:18:32.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:31 smithi144 bash[23522]: audit 2024-04-07T16:18:31.319990+0000 mon.b (mon.1) 299 : audit 0 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:18:32.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:31 smithi144 bash[23522]: audit 2024-04-07T16:18:31.320555+0000 mon.b (mon.1) 300 : audit 0 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:18:32.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:31 smithi144 bash[23522]: audit 2024-04-07T16:18:31.320988+0000 mon.b (mon.1) 301 : audit 0 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:18:32.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:31 smithi144 bash[23522]: audit 2024-04-07T16:18:31.323924+0000 mon.c (mon.2) 57 : audit 1 from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T16:18:32.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:31 smithi144 bash[23522]: audit 2024-04-07T16:18:31.324030+0000 mon.c (mon.2) 58 : audit 1 from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T16:18:32.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:31 smithi100 bash[21346]: cluster 2024-04-07T16:18:30.643440+0000 mgr.x (mgr.24553) 12 : cluster 0 pgmap v18: 121 pgs: 121 active+clean; 126 MiB data, 556 MiB used, 715 GiB / 715 GiB avail; 2.0 MiB/s rd, 1.2 MiB/s wr, 44 op/s 2024-04-07T16:18:32.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:31 smithi100 bash[21346]: cluster 2024-04-07T16:18:30.977268+0000 mon.a (mon.0) 1025 : cluster 0 osdmap e177: 8 total, 8 up, 8 in 2024-04-07T16:18:32.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:31 smithi100 bash[21346]: audit 2024-04-07T16:18:31.281185+0000 mon.b (mon.1) 282 : audit 0 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:18:32.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:31 smithi100 bash[21346]: audit 2024-04-07T16:18:31.281767+0000 mon.b (mon.1) 283 : audit 0 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:18:32.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:31 smithi100 bash[21346]: audit 2024-04-07T16:18:31.282126+0000 mon.b (mon.1) 284 : audit 0 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:18:32.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:31 smithi100 bash[21346]: audit 2024-04-07T16:18:31.285287+0000 mon.b (mon.1) 285 : audit 0 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:18:32.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:31 smithi100 bash[21346]: audit 2024-04-07T16:18:31.285934+0000 mon.b (mon.1) 286 : audit 0 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:18:32.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:31 smithi100 bash[21346]: audit 2024-04-07T16:18:31.286398+0000 mon.b (mon.1) 287 : audit 0 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:18:32.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:31 smithi100 bash[21346]: audit 2024-04-07T16:18:31.289633+0000 mon.b (mon.1) 288 : audit 0 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:18:32.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:31 smithi100 bash[21346]: audit 2024-04-07T16:18:31.290206+0000 mon.b (mon.1) 289 : audit 0 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:18:32.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:31 smithi100 bash[21346]: audit 2024-04-07T16:18:31.290591+0000 mon.b (mon.1) 290 : audit 0 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:18:32.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:31 smithi100 bash[21346]: audit 2024-04-07T16:18:31.311972+0000 mon.b (mon.1) 291 : audit 0 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:18:32.439 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:31 smithi100 bash[21346]: audit 2024-04-07T16:18:31.312527+0000 mon.b (mon.1) 292 : audit 0 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:18:32.439 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:31 smithi100 bash[21346]: audit 2024-04-07T16:18:31.312865+0000 mon.b (mon.1) 293 : audit 0 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:18:32.439 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:31 smithi100 bash[21346]: audit 2024-04-07T16:18:31.315318+0000 mon.b (mon.1) 294 : audit 1 from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T16:18:32.439 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:31 smithi100 bash[21346]: audit 2024-04-07T16:18:31.315421+0000 mon.b (mon.1) 295 : audit 1 from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T16:18:32.439 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:31 smithi100 bash[21346]: audit 2024-04-07T16:18:31.315667+0000 mon.a (mon.0) 1026 : audit 1 from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T16:18:32.439 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:31 smithi100 bash[21346]: audit 2024-04-07T16:18:31.315789+0000 mon.a (mon.0) 1027 : audit 1 from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T16:18:32.439 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:31 smithi100 bash[21346]: audit 2024-04-07T16:18:31.316015+0000 mon.b (mon.1) 296 : audit 0 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:18:32.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:31 smithi100 bash[31660]: cluster 2024-04-07T16:18:30.643440+0000 mgr.x (mgr.24553) 12 : cluster 0 pgmap v18: 121 pgs: 121 active+clean; 126 MiB data, 556 MiB used, 715 GiB / 715 GiB avail; 2.0 MiB/s rd, 1.2 MiB/s wr, 44 op/s 2024-04-07T16:18:32.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:31 smithi100 bash[31660]: cluster 2024-04-07T16:18:30.977268+0000 mon.a (mon.0) 1025 : cluster 0 osdmap e177: 8 total, 8 up, 8 in 2024-04-07T16:18:32.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:31 smithi100 bash[31660]: audit 2024-04-07T16:18:31.281185+0000 mon.b (mon.1) 282 : audit 0 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:18:32.440 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:31 smithi100 bash[31660]: audit 2024-04-07T16:18:31.281767+0000 mon.b (mon.1) 283 : audit 0 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:18:32.440 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:31 smithi100 bash[31660]: audit 2024-04-07T16:18:31.282126+0000 mon.b (mon.1) 284 : audit 0 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:18:32.440 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:31 smithi100 bash[31660]: audit 2024-04-07T16:18:31.285287+0000 mon.b (mon.1) 285 : audit 0 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:18:32.440 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:31 smithi100 bash[31660]: audit 2024-04-07T16:18:31.285934+0000 mon.b (mon.1) 286 : audit 0 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:18:32.440 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:31 smithi100 bash[31660]: audit 2024-04-07T16:18:31.286398+0000 mon.b (mon.1) 287 : audit 0 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:18:32.440 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:31 smithi100 bash[31660]: audit 2024-04-07T16:18:31.289633+0000 mon.b (mon.1) 288 : audit 0 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:18:32.440 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:31 smithi100 bash[31660]: audit 2024-04-07T16:18:31.290206+0000 mon.b (mon.1) 289 : audit 0 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:18:32.440 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:31 smithi100 bash[31660]: audit 2024-04-07T16:18:31.290591+0000 mon.b (mon.1) 290 : audit 0 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:18:32.440 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:31 smithi100 bash[31660]: audit 2024-04-07T16:18:31.311972+0000 mon.b (mon.1) 291 : audit 0 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:18:32.440 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:31 smithi100 bash[31660]: audit 2024-04-07T16:18:31.312527+0000 mon.b (mon.1) 292 : audit 0 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:18:32.440 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:31 smithi100 bash[31660]: audit 2024-04-07T16:18:31.312865+0000 mon.b (mon.1) 293 : audit 0 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:18:32.440 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:31 smithi100 bash[31660]: audit 2024-04-07T16:18:31.315318+0000 mon.b (mon.1) 294 : audit 1 from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T16:18:32.440 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:31 smithi100 bash[31660]: audit 2024-04-07T16:18:31.315421+0000 mon.b (mon.1) 295 : audit 1 from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T16:18:32.441 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:31 smithi100 bash[31660]: audit 2024-04-07T16:18:31.315667+0000 mon.a (mon.0) 1026 : audit 1 from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T16:18:32.441 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:31 smithi100 bash[31660]: audit 2024-04-07T16:18:31.315789+0000 mon.a (mon.0) 1027 : audit 1 from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T16:18:32.441 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:31 smithi100 bash[31660]: audit 2024-04-07T16:18:31.316015+0000 mon.b (mon.1) 296 : audit 0 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:18:32.441 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:31 smithi100 bash[21346]: audit 2024-04-07T16:18:31.316489+0000 mon.b (mon.1) 297 : audit 0 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:18:32.441 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:31 smithi100 bash[21346]: audit 2024-04-07T16:18:31.316872+0000 mon.b (mon.1) 298 : audit 0 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:18:32.441 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:31 smithi100 bash[21346]: audit 2024-04-07T16:18:31.319990+0000 mon.b (mon.1) 299 : audit 0 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:18:32.441 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:31 smithi100 bash[21346]: audit 2024-04-07T16:18:31.320555+0000 mon.b (mon.1) 300 : audit 0 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:18:32.441 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:31 smithi100 bash[21346]: audit 2024-04-07T16:18:31.320988+0000 mon.b (mon.1) 301 : audit 0 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:18:32.441 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:31 smithi100 bash[21346]: audit 2024-04-07T16:18:31.323924+0000 mon.c (mon.2) 57 : audit 1 from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T16:18:32.441 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:31 smithi100 bash[21346]: audit 2024-04-07T16:18:31.324030+0000 mon.c (mon.2) 58 : audit 1 from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T16:18:32.442 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:31 smithi100 bash[31660]: audit 2024-04-07T16:18:31.316489+0000 mon.b (mon.1) 297 : audit 0 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:18:32.442 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:31 smithi100 bash[31660]: audit 2024-04-07T16:18:31.316872+0000 mon.b (mon.1) 298 : audit 0 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:18:32.442 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:31 smithi100 bash[31660]: audit 2024-04-07T16:18:31.319990+0000 mon.b (mon.1) 299 : audit 0 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:18:32.442 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:31 smithi100 bash[31660]: audit 2024-04-07T16:18:31.320555+0000 mon.b (mon.1) 300 : audit 0 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:18:32.442 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:31 smithi100 bash[31660]: audit 2024-04-07T16:18:31.320988+0000 mon.b (mon.1) 301 : audit 0 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:18:32.442 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:31 smithi100 bash[31660]: audit 2024-04-07T16:18:31.323924+0000 mon.c (mon.2) 57 : audit 1 from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T16:18:32.442 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:31 smithi100 bash[31660]: audit 2024-04-07T16:18:31.324030+0000 mon.c (mon.2) 58 : audit 1 from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T16:18:32.853 INFO:tasks.rados.rados.0.smithi100.stdout:641: done (5 left) 2024-04-07T16:18:32.853 INFO:tasks.rados.rados.0.smithi100.stdout:642: snap_remove snap 51 2024-04-07T16:18:33.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:32 smithi144 bash[23522]: cluster 2024-04-07T16:18:31.981339+0000 mon.a (mon.0) 1028 : cluster 0 mgrmap e28: x(active, since 11s), standbys: y 2024-04-07T16:18:33.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:32 smithi144 bash[23522]: cluster 2024-04-07T16:18:32.001650+0000 mon.a (mon.0) 1029 : cluster 0 osdmap e178: 8 total, 8 up, 8 in 2024-04-07T16:18:33.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:32 smithi144 bash[23522]: cluster 2024-04-07T16:18:32.866968+0000 mon.a (mon.0) 1030 : cluster 0 osdmap e179: 8 total, 8 up, 8 in 2024-04-07T16:18:33.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:32 smithi100 bash[21346]: cluster 2024-04-07T16:18:31.981339+0000 mon.a (mon.0) 1028 : cluster 0 mgrmap e28: x(active, since 11s), standbys: y 2024-04-07T16:18:33.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:32 smithi100 bash[21346]: cluster 2024-04-07T16:18:32.001650+0000 mon.a (mon.0) 1029 : cluster 0 osdmap e178: 8 total, 8 up, 8 in 2024-04-07T16:18:33.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:32 smithi100 bash[21346]: cluster 2024-04-07T16:18:32.866968+0000 mon.a (mon.0) 1030 : cluster 0 osdmap e179: 8 total, 8 up, 8 in 2024-04-07T16:18:33.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:33 smithi100 bash[31660]: cluster 2024-04-07T16:18:31.981339+0000 mon.a (mon.0) 1028 : cluster 0 mgrmap e28: x(active, since 11s), standbys: y 2024-04-07T16:18:33.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:33 smithi100 bash[31660]: cluster 2024-04-07T16:18:32.001650+0000 mon.a (mon.0) 1029 : cluster 0 osdmap e178: 8 total, 8 up, 8 in 2024-04-07T16:18:33.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:33 smithi100 bash[31660]: cluster 2024-04-07T16:18:32.866968+0000 mon.a (mon.0) 1030 : cluster 0 osdmap e179: 8 total, 8 up, 8 in 2024-04-07T16:18:33.856 INFO:tasks.rados.rados.0.smithi100.stdout:642: done (5 left) 2024-04-07T16:18:33.857 INFO:tasks.rados.rados.0.smithi100.stdout:643: copy_from oid 17 from oid 36 current snap is 57 2024-04-07T16:18:33.857 INFO:tasks.rados.rados.0.smithi100.stdout:637: finishing rollback tid 1 to smithi10066867-10 2024-04-07T16:18:33.857 INFO:tasks.rados.rados.0.smithi100.stdout:637: finishing rollback tid 2 to smithi10066867-10 2024-04-07T16:18:33.857 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 10 v 57 (ObjNum 123 snap 29 seq_num 123) dirty exists 2024-04-07T16:18:33.857 INFO:tasks.rados.rados.0.smithi100.stdout:636: finishing write tid 1 to smithi10066867-15 2024-04-07T16:18:33.857 INFO:tasks.rados.rados.0.smithi100.stdout:636: finishing write tid 2 to smithi10066867-15 2024-04-07T16:18:33.857 INFO:tasks.rados.rados.0.smithi100.stdout:636: finishing write tid 3 to smithi10066867-15 2024-04-07T16:18:33.857 INFO:tasks.rados.rados.0.smithi100.stdout:636: finishing write tid 4 to smithi10066867-15 2024-04-07T16:18:33.857 INFO:tasks.rados.rados.0.smithi100.stdout:636: oid 15 updating version 0 to 324 2024-04-07T16:18:33.857 INFO:tasks.rados.rados.0.smithi100.stdout:636: oid 15 version 324 is already newer than 323 2024-04-07T16:18:33.857 INFO:tasks.rados.rados.0.smithi100.stdout:636: oid 15 updating version 324 to 325 2024-04-07T16:18:33.857 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 15 v 325 (ObjNum 211 snap 57 seq_num 211) dirty exists 2024-04-07T16:18:33.857 INFO:tasks.rados.rados.0.smithi100.stdout:636: left oid 15 (ObjNum 211 snap 57 seq_num 211) 2024-04-07T16:18:33.857 INFO:tasks.rados.rados.0.smithi100.stdout:638: finishing rollback tid 0 to smithi10066867-4 2024-04-07T16:18:33.858 INFO:tasks.rados.rados.0.smithi100.stdout:638: finishing rollback tid 1 to smithi10066867-4 2024-04-07T16:18:33.858 INFO:tasks.rados.rados.0.smithi100.stdout:639: expect (ObjNum 108 snap 20 seq_num 108) 2024-04-07T16:18:33.988 INFO:tasks.rados.rados.0.smithi100.stdout:638: finishing rollback tid 2 to smithi10066867-4 2024-04-07T16:18:33.988 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 4 v 329 (ObjNum 171 snap 44 seq_num 171) dirty exists 2024-04-07T16:18:33.988 INFO:tasks.rados.rados.0.smithi100.stdout:640: finishing copy_from racing read to smithi10066867-18 2024-04-07T16:18:33.988 INFO:tasks.rados.rados.0.smithi100.stdout:640: finishing copy_from to smithi10066867-18 2024-04-07T16:18:33.988 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 18 v 206 (ObjNum 205 snap 55 seq_num 205) dirty exists 2024-04-07T16:18:33.988 INFO:tasks.rados.rados.0.smithi100.stdout:636: done (5 left) 2024-04-07T16:18:33.988 INFO:tasks.rados.rados.0.smithi100.stdout:637: done (4 left) 2024-04-07T16:18:33.988 INFO:tasks.rados.rados.0.smithi100.stdout:638: done (3 left) 2024-04-07T16:18:33.989 INFO:tasks.rados.rados.0.smithi100.stdout:639: done (2 left) 2024-04-07T16:18:33.989 INFO:tasks.rados.rados.0.smithi100.stdout:640: done (1 left) 2024-04-07T16:18:33.989 INFO:tasks.rados.rados.0.smithi100.stdout:644: copy_from oid 6 from oid 29 current snap is 57 2024-04-07T16:18:33.989 INFO:tasks.rados.rados.0.smithi100.stdout:643: finishing copy_from racing read to smithi10066867-17 2024-04-07T16:18:33.989 INFO:tasks.rados.rados.0.smithi100.stdout:643: finishing copy_from to smithi10066867-17 2024-04-07T16:18:33.989 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 17 v 213 (ObjNum 210 snap 57 seq_num 210) dirty exists 2024-04-07T16:18:33.989 INFO:tasks.rados.rados.0.smithi100.stdout:643: done (1 left) 2024-04-07T16:18:33.989 INFO:tasks.rados.rados.0.smithi100.stdout:645: rollback oid 11 current snap is 57 2024-04-07T16:18:33.989 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 11 to 48 2024-04-07T16:18:33.989 INFO:tasks.rados.rados.0.smithi100.stdout:646: read oid 7 snap -1 2024-04-07T16:18:33.989 INFO:tasks.rados.rados.0.smithi100.stdout:646: expect (ObjNum 16 snap 0 seq_num 16) 2024-04-07T16:18:33.989 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 39 current snap is 57 2024-04-07T16:18:33.989 INFO:tasks.rados.rados.0.smithi100.stdout:647: seq_num 212 ranges {4087808=753664,4841472=401408} 2024-04-07T16:18:34.016 INFO:tasks.rados.rados.0.smithi100.stdout:647: writing smithi10066867-39 from 4087808 to 4841472 tid 1 2024-04-07T16:18:34.019 INFO:tasks.rados.rados.0.smithi100.stdout:647: writing smithi10066867-39 from 4841472 to 5242880 tid 2 2024-04-07T16:18:34.019 INFO:tasks.rados.rados.0.smithi100.stdout:648: read oid 34 snap -1 2024-04-07T16:18:34.019 INFO:tasks.rados.rados.0.smithi100.stdout:648: expect (ObjNum 140 snap 39 seq_num 140) 2024-04-07T16:18:34.019 INFO:tasks.rados.rados.0.smithi100.stdout:646: expect (ObjNum 16 snap 0 seq_num 16) 2024-04-07T16:18:34.031 INFO:tasks.rados.rados.0.smithi100.stdout:645: finishing rollback tid 0 to smithi10066867-11 2024-04-07T16:18:34.031 INFO:tasks.rados.rados.0.smithi100.stdout:645: finishing rollback tid 1 to smithi10066867-11 2024-04-07T16:18:34.031 INFO:tasks.rados.rados.0.smithi100.stdout:645: finishing rollback tid 2 to smithi10066867-11 2024-04-07T16:18:34.031 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 11 v 353 (ObjNum 106 snap 20 seq_num 106) dirty exists 2024-04-07T16:18:34.031 INFO:tasks.rados.rados.0.smithi100.stdout:644: finishing copy_from racing read to smithi10066867-6 2024-04-07T16:18:34.031 INFO:tasks.rados.rados.0.smithi100.stdout:644: finishing copy_from to smithi10066867-6 2024-04-07T16:18:34.031 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 6 v 128 (ObjNum 61 snap 5 seq_num 61) dirty exists 2024-04-07T16:18:34.031 INFO:tasks.rados.rados.0.smithi100.stdout:644: done (4 left) 2024-04-07T16:18:34.031 INFO:tasks.rados.rados.0.smithi100.stdout:645: done (3 left) 2024-04-07T16:18:34.031 INFO:tasks.rados.rados.0.smithi100.stdout:646: done (2 left) 2024-04-07T16:18:34.031 INFO:tasks.rados.rados.0.smithi100.stdout:649: snap_create 2024-04-07T16:18:34.031 INFO:tasks.rados.rados.0.smithi100.stdout:647: finishing write tid 1 to smithi10066867-39 2024-04-07T16:18:34.031 INFO:tasks.rados.rados.0.smithi100.stdout:647: finishing write tid 2 to smithi10066867-39 2024-04-07T16:18:34.032 INFO:tasks.rados.rados.0.smithi100.stdout:647: finishing write tid 3 to smithi10066867-39 2024-04-07T16:18:34.032 INFO:tasks.rados.rados.0.smithi100.stdout:647: finishing write tid 4 to smithi10066867-39 2024-04-07T16:18:34.032 INFO:tasks.rados.rados.0.smithi100.stdout:647: oid 39 updating version 0 to 230 2024-04-07T16:18:34.032 INFO:tasks.rados.rados.0.smithi100.stdout:647: oid 39 version 230 is already newer than 229 2024-04-07T16:18:34.032 INFO:tasks.rados.rados.0.smithi100.stdout:647: oid 39 updating version 230 to 231 2024-04-07T16:18:34.032 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 39 v 231 (ObjNum 212 snap 57 seq_num 212) dirty exists 2024-04-07T16:18:34.032 INFO:tasks.rados.rados.0.smithi100.stdout:647: left oid 39 (ObjNum 212 snap 57 seq_num 212) 2024-04-07T16:18:34.032 INFO:tasks.rados.rados.0.smithi100.stdout:648: expect (ObjNum 140 snap 39 seq_num 140) 2024-04-07T16:18:34.354 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:33 smithi144 bash[23522]: cluster 2024-04-07T16:18:32.644341+0000 mgr.x (mgr.24553) 13 : cluster 0 pgmap v21: 121 pgs: 121 active+clean; 141 MiB data, 576 MiB used, 715 GiB / 715 GiB avail; 6.4 MiB/s rd, 3.7 MiB/s wr, 37 op/s 2024-04-07T16:18:34.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:33 smithi144 bash[23522]: cluster 2024-04-07T16:18:33.870176+0000 mon.a (mon.0) 1031 : cluster 0 osdmap e180: 8 total, 8 up, 8 in 2024-04-07T16:18:34.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:34 smithi100 bash[31660]: cluster 2024-04-07T16:18:32.644341+0000 mgr.x (mgr.24553) 13 : cluster 0 pgmap v21: 121 pgs: 121 active+clean; 141 MiB data, 576 MiB used, 715 GiB / 715 GiB avail; 6.4 MiB/s rd, 3.7 MiB/s wr, 37 op/s 2024-04-07T16:18:34.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:34 smithi100 bash[31660]: cluster 2024-04-07T16:18:33.870176+0000 mon.a (mon.0) 1031 : cluster 0 osdmap e180: 8 total, 8 up, 8 in 2024-04-07T16:18:34.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:34 smithi100 bash[21346]: cluster 2024-04-07T16:18:32.644341+0000 mgr.x (mgr.24553) 13 : cluster 0 pgmap v21: 121 pgs: 121 active+clean; 141 MiB data, 576 MiB used, 715 GiB / 715 GiB avail; 6.4 MiB/s rd, 3.7 MiB/s wr, 37 op/s 2024-04-07T16:18:34.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:34 smithi100 bash[21346]: cluster 2024-04-07T16:18:33.870176+0000 mon.a (mon.0) 1031 : cluster 0 osdmap e180: 8 total, 8 up, 8 in 2024-04-07T16:18:34.994 INFO:tasks.rados.rados.0.smithi100.stdout:647: done (2 left) 2024-04-07T16:18:34.994 INFO:tasks.rados.rados.0.smithi100.stdout:648: done (1 left) 2024-04-07T16:18:34.994 INFO:tasks.rados.rados.0.smithi100.stdout:649: done (0 left) 2024-04-07T16:18:34.994 INFO:tasks.rados.rados.0.smithi100.stdout:650: delete oid 38 current snap is 58 2024-04-07T16:18:35.000 INFO:tasks.rados.rados.0.smithi100.stdout:650: done (0 left) 2024-04-07T16:18:35.000 INFO:tasks.rados.rados.0.smithi100.stdout:651: rollback oid 44 current snap is 58 2024-04-07T16:18:35.000 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 44 to 54 2024-04-07T16:18:35.001 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 40 current snap is 58 2024-04-07T16:18:35.001 INFO:tasks.rados.rados.0.smithi100.stdout:652: seq_num 214 ranges {4661248=794624,5455872=401408} 2024-04-07T16:18:35.047 INFO:tasks.rados.rados.0.smithi100.stdout:652: writing smithi10066867-40 from 4661248 to 5455872 tid 1 2024-04-07T16:18:35.051 INFO:tasks.rados.rados.0.smithi100.stdout:652: writing smithi10066867-40 from 5455872 to 5857280 tid 2 2024-04-07T16:18:35.051 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 22 current snap is 58 2024-04-07T16:18:35.051 INFO:tasks.rados.rados.0.smithi100.stdout:651: finishing rollback tid 0 to smithi10066867-44 2024-04-07T16:18:35.052 INFO:tasks.rados.rados.0.smithi100.stdout:653: seq_num 215 ranges {0=409600,409600=532480,942080=270336} 2024-04-07T16:18:35.056 INFO:tasks.rados.rados.0.smithi100.stdout:653: writing smithi10066867-22 from 0 to 409600 tid 1 2024-04-07T16:18:35.061 INFO:tasks.rados.rados.0.smithi100.stdout:653: writing smithi10066867-22 from 409600 to 942080 tid 2 2024-04-07T16:18:35.064 INFO:tasks.rados.rados.0.smithi100.stdout:653: writing smithi10066867-22 from 942080 to 1212416 tid 3 2024-04-07T16:18:35.064 INFO:tasks.rados.rados.0.smithi100.stdout:654: rollback oid 6 current snap is 58 2024-04-07T16:18:35.064 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 6 to 57 2024-04-07T16:18:35.064 INFO:tasks.rados.rados.0.smithi100.stdout:655: read oid 2 snap -1 2024-04-07T16:18:35.064 INFO:tasks.rados.rados.0.smithi100.stdout:655: expect (ObjNum 199 snap 52 seq_num 199) 2024-04-07T16:18:35.064 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 27 current snap is 58 2024-04-07T16:18:35.065 INFO:tasks.rados.rados.0.smithi100.stdout:651: finishing rollback tid 1 to smithi10066867-44 2024-04-07T16:18:35.065 INFO:tasks.rados.rados.0.smithi100.stdout:651: finishing rollback tid 2 to smithi10066867-44 2024-04-07T16:18:35.065 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 44 v 224 (ObjNum 165 snap 44 seq_num 165) dirty exists 2024-04-07T16:18:35.065 INFO:tasks.rados.rados.0.smithi100.stdout:652: finishing write tid 1 to smithi10066867-40 2024-04-07T16:18:35.065 INFO:tasks.rados.rados.0.smithi100.stdout:652: finishing write tid 2 to smithi10066867-40 2024-04-07T16:18:35.065 INFO:tasks.rados.rados.0.smithi100.stdout:652: finishing write tid 3 to smithi10066867-40 2024-04-07T16:18:35.065 INFO:tasks.rados.rados.0.smithi100.stdout:652: finishing write tid 4 to smithi10066867-40 2024-04-07T16:18:35.065 INFO:tasks.rados.rados.0.smithi100.stdout:652: oid 40 updating version 0 to 227 2024-04-07T16:18:35.065 INFO:tasks.rados.rados.0.smithi100.stdout:652: oid 40 version 227 is already newer than 226 2024-04-07T16:18:35.065 INFO:tasks.rados.rados.0.smithi100.stdout:652: oid 40 updating version 227 to 228 2024-04-07T16:18:35.065 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 40 v 228 (ObjNum 214 snap 58 seq_num 214) dirty exists 2024-04-07T16:18:35.065 INFO:tasks.rados.rados.0.smithi100.stdout:652: left oid 40 (ObjNum 214 snap 58 seq_num 214) 2024-04-07T16:18:35.065 INFO:tasks.rados.rados.0.smithi100.stdout:653: finishing write tid 1 to smithi10066867-22 2024-04-07T16:18:35.065 INFO:tasks.rados.rados.0.smithi100.stdout:653: finishing write tid 2 to smithi10066867-22 2024-04-07T16:18:35.066 INFO:tasks.rados.rados.0.smithi100.stdout:656: seq_num 216 ranges {958464=425984,1384448=409600,1794048=393216} 2024-04-07T16:18:35.072 INFO:tasks.rados.rados.0.smithi100.stdout:656: writing smithi10066867-27 from 958464 to 1384448 tid 1 2024-04-07T16:18:35.075 INFO:tasks.rados.rados.0.smithi100.stdout:656: writing smithi10066867-27 from 1384448 to 1794048 tid 2 2024-04-07T16:18:35.078 INFO:tasks.rados.rados.0.smithi100.stdout:656: writing smithi10066867-27 from 1794048 to 2187264 tid 3 2024-04-07T16:18:35.078 INFO:tasks.rados.rados.0.smithi100.stdout:651: done (5 left) 2024-04-07T16:18:35.078 INFO:tasks.rados.rados.0.smithi100.stdout:652: done (4 left) 2024-04-07T16:18:35.078 INFO:tasks.rados.rados.0.smithi100.stdout:657: snap_create 2024-04-07T16:18:35.078 INFO:tasks.rados.rados.0.smithi100.stdout:653: finishing write tid 3 to smithi10066867-22 2024-04-07T16:18:35.078 INFO:tasks.rados.rados.0.smithi100.stdout:653: finishing write tid 4 to smithi10066867-22 2024-04-07T16:18:35.078 INFO:tasks.rados.rados.0.smithi100.stdout:654: finishing rollback tid 0 to smithi10066867-6 2024-04-07T16:18:35.078 INFO:tasks.rados.rados.0.smithi100.stdout:654: finishing rollback tid 1 to smithi10066867-6 2024-04-07T16:18:35.078 INFO:tasks.rados.rados.0.smithi100.stdout:653: finishing write tid 5 to smithi10066867-22 2024-04-07T16:18:35.078 INFO:tasks.rados.rados.0.smithi100.stdout:653: oid 22 updating version 0 to 207 2024-04-07T16:18:35.079 INFO:tasks.rados.rados.0.smithi100.stdout:653: oid 22 updating version 207 to 208 2024-04-07T16:18:35.079 INFO:tasks.rados.rados.0.smithi100.stdout:653: oid 22 updating version 208 to 210 2024-04-07T16:18:35.079 INFO:tasks.rados.rados.0.smithi100.stdout:653: oid 22 version 210 is already newer than 209 2024-04-07T16:18:35.079 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 22 v 210 (ObjNum 215 snap 58 seq_num 215) dirty exists 2024-04-07T16:18:35.079 INFO:tasks.rados.rados.0.smithi100.stdout:653: left oid 22 (ObjNum 215 snap 58 seq_num 215) 2024-04-07T16:18:35.079 INFO:tasks.rados.rados.0.smithi100.stdout:656: finishing write tid 1 to smithi10066867-27 2024-04-07T16:18:35.082 INFO:tasks.rados.rados.0.smithi100.stdout:656: finishing write tid 2 to smithi10066867-27 2024-04-07T16:18:35.083 INFO:tasks.rados.rados.0.smithi100.stdout:654: finishing rollback tid 2 to smithi10066867-6 2024-04-07T16:18:35.083 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 6 v 232 (ObjNum 61 snap 5 seq_num 61) dirty exists 2024-04-07T16:18:35.084 INFO:tasks.rados.rados.0.smithi100.stdout:656: finishing write tid 3 to smithi10066867-27 2024-04-07T16:18:35.084 INFO:tasks.rados.rados.0.smithi100.stdout:656: finishing write tid 4 to smithi10066867-27 2024-04-07T16:18:35.086 INFO:tasks.rados.rados.0.smithi100.stdout:656: finishing write tid 5 to smithi10066867-27 2024-04-07T16:18:35.086 INFO:tasks.rados.rados.0.smithi100.stdout:656: oid 27 updating version 0 to 117 2024-04-07T16:18:35.086 INFO:tasks.rados.rados.0.smithi100.stdout:656: oid 27 version 117 is already newer than 115 2024-04-07T16:18:35.086 INFO:tasks.rados.rados.0.smithi100.stdout:656: oid 27 version 117 is already newer than 116 2024-04-07T16:18:35.086 INFO:tasks.rados.rados.0.smithi100.stdout:656: oid 27 version 117 is already newer than 114 2024-04-07T16:18:35.087 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 27 v 117 (ObjNum 216 snap 58 seq_num 216) dirty exists 2024-04-07T16:18:35.087 INFO:tasks.rados.rados.0.smithi100.stdout:656: left oid 27 (ObjNum 216 snap 58 seq_num 216) 2024-04-07T16:18:35.091 INFO:tasks.rados.rados.0.smithi100.stdout:655: expect (ObjNum 199 snap 52 seq_num 199) 2024-04-07T16:18:35.437 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:18:35 smithi100 bash[21601]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:18:35] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-04-07T16:18:35.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:35 smithi100 bash[21346]: audit 2024-04-07T16:18:34.310981+0000 mon.a (mon.0) 1032 : audit 1 from='mgr.24553 ' entity='mgr.x' 2024-04-07T16:18:35.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:35 smithi100 bash[21346]: audit 2024-04-07T16:18:34.320228+0000 mon.a (mon.0) 1033 : audit 1 from='mgr.24553 ' entity='mgr.x' 2024-04-07T16:18:35.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:35 smithi100 bash[21346]: audit 2024-04-07T16:18:34.601695+0000 mon.a (mon.0) 1034 : audit 1 from='mgr.24553 ' entity='mgr.x' 2024-04-07T16:18:35.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:35 smithi100 bash[21346]: audit 2024-04-07T16:18:34.615413+0000 mon.a (mon.0) 1035 : audit 1 from='mgr.24553 ' entity='mgr.x' 2024-04-07T16:18:35.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:35 smithi100 bash[21346]: cluster 2024-04-07T16:18:34.644836+0000 mgr.x (mgr.24553) 14 : cluster 0 pgmap v24: 121 pgs: 121 active+clean; 141 MiB data, 576 MiB used, 715 GiB / 715 GiB avail; 6.4 MiB/s rd, 3.7 MiB/s wr, 37 op/s 2024-04-07T16:18:35.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:35 smithi100 bash[21346]: cluster 2024-04-07T16:18:35.009107+0000 mon.a (mon.0) 1036 : cluster 0 osdmap e181: 8 total, 8 up, 8 in 2024-04-07T16:18:35.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:35 smithi100 bash[31660]: audit 2024-04-07T16:18:34.310981+0000 mon.a (mon.0) 1032 : audit 1 from='mgr.24553 ' entity='mgr.x' 2024-04-07T16:18:35.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:35 smithi100 bash[31660]: audit 2024-04-07T16:18:34.320228+0000 mon.a (mon.0) 1033 : audit 1 from='mgr.24553 ' entity='mgr.x' 2024-04-07T16:18:35.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:35 smithi100 bash[31660]: audit 2024-04-07T16:18:34.601695+0000 mon.a (mon.0) 1034 : audit 1 from='mgr.24553 ' entity='mgr.x' 2024-04-07T16:18:35.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:35 smithi100 bash[31660]: audit 2024-04-07T16:18:34.615413+0000 mon.a (mon.0) 1035 : audit 1 from='mgr.24553 ' entity='mgr.x' 2024-04-07T16:18:35.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:35 smithi100 bash[31660]: cluster 2024-04-07T16:18:34.644836+0000 mgr.x (mgr.24553) 14 : cluster 0 pgmap v24: 121 pgs: 121 active+clean; 141 MiB data, 576 MiB used, 715 GiB / 715 GiB avail; 6.4 MiB/s rd, 3.7 MiB/s wr, 37 op/s 2024-04-07T16:18:35.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:35 smithi100 bash[31660]: cluster 2024-04-07T16:18:35.009107+0000 mon.a (mon.0) 1036 : cluster 0 osdmap e181: 8 total, 8 up, 8 in 2024-04-07T16:18:35.604 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:35 smithi144 bash[23522]: audit 2024-04-07T16:18:34.310981+0000 mon.a (mon.0) 1032 : audit 1 from='mgr.24553 ' entity='mgr.x' 2024-04-07T16:18:35.605 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:35 smithi144 bash[23522]: audit 2024-04-07T16:18:34.320228+0000 mon.a (mon.0) 1033 : audit 1 from='mgr.24553 ' entity='mgr.x' 2024-04-07T16:18:35.605 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:35 smithi144 bash[23522]: audit 2024-04-07T16:18:34.601695+0000 mon.a (mon.0) 1034 : audit 1 from='mgr.24553 ' entity='mgr.x' 2024-04-07T16:18:35.605 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:35 smithi144 bash[23522]: audit 2024-04-07T16:18:34.615413+0000 mon.a (mon.0) 1035 : audit 1 from='mgr.24553 ' entity='mgr.x' 2024-04-07T16:18:35.605 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:35 smithi144 bash[23522]: cluster 2024-04-07T16:18:34.644836+0000 mgr.x (mgr.24553) 14 : cluster 0 pgmap v24: 121 pgs: 121 active+clean; 141 MiB data, 576 MiB used, 715 GiB / 715 GiB avail; 6.4 MiB/s rd, 3.7 MiB/s wr, 37 op/s 2024-04-07T16:18:35.605 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:35 smithi144 bash[23522]: cluster 2024-04-07T16:18:35.009107+0000 mon.a (mon.0) 1036 : cluster 0 osdmap e181: 8 total, 8 up, 8 in 2024-04-07T16:18:35.996 INFO:tasks.rados.rados.0.smithi100.stdout:653: done (4 left) 2024-04-07T16:18:35.996 INFO:tasks.rados.rados.0.smithi100.stdout:654: done (3 left) 2024-04-07T16:18:35.996 INFO:tasks.rados.rados.0.smithi100.stdout:655: done (2 left) 2024-04-07T16:18:35.996 INFO:tasks.rados.rados.0.smithi100.stdout:656: done (1 left) 2024-04-07T16:18:35.996 INFO:tasks.rados.rados.0.smithi100.stdout:657: done (0 left) 2024-04-07T16:18:35.997 INFO:tasks.rados.rados.0.smithi100.stdout:658: delete oid 2 current snap is 59 2024-04-07T16:18:36.006 INFO:tasks.rados.rados.0.smithi100.stdout:658: done (0 left) 2024-04-07T16:18:36.006 INFO:tasks.rados.rados.0.smithi100.stdout:659: delete oid 5 current snap is 59 2024-04-07T16:18:36.017 INFO:tasks.rados.rados.0.smithi100.stdout:659: done (0 left) 2024-04-07T16:18:36.017 INFO:tasks.rados.rados.0.smithi100.stdout:660: read oid 43 snap -1 2024-04-07T16:18:36.018 INFO:tasks.rados.rados.0.smithi100.stdout:660: expect deleted 2024-04-07T16:18:36.018 INFO:tasks.rados.rados.0.smithi100.stdout:661: read oid 28 snap 45 2024-04-07T16:18:36.018 INFO:tasks.rados.rados.0.smithi100.stdout:661: expect (ObjNum 140 snap 39 seq_num 140) 2024-04-07T16:18:36.018 INFO:tasks.rados.rados.0.smithi100.stdout:662: delete oid 17 current snap is 59 2024-04-07T16:18:36.023 INFO:tasks.rados.rados.0.smithi100.stdout:660: done (2 left) 2024-04-07T16:18:36.023 INFO:tasks.rados.rados.0.smithi100.stdout:662: done (1 left) 2024-04-07T16:18:36.023 INFO:tasks.rados.rados.0.smithi100.stdout:663: copy_from oid 45 from oid 13 current snap is 59 2024-04-07T16:18:36.023 INFO:tasks.rados.rados.0.smithi100.stdout:664: rmattr oid 36 current snap is 59 2024-04-07T16:18:36.023 INFO:tasks.rados.rados.0.smithi100.stdout:664: done (2 left) 2024-04-07T16:18:36.023 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 11 current snap is 59 2024-04-07T16:18:36.023 INFO:tasks.rados.rados.0.smithi100.stdout:665: seq_num 220 ranges {3350528=491520,3842048=753664,4595712=49152} 2024-04-07T16:18:36.052 INFO:tasks.rados.rados.0.smithi100.stdout:665: writing smithi10066867-11 from 3350528 to 3842048 tid 1 2024-04-07T16:18:36.062 INFO:tasks.rados.rados.0.smithi100.stdout:665: writing smithi10066867-11 from 3842048 to 4595712 tid 2 2024-04-07T16:18:36.063 INFO:tasks.rados.rados.0.smithi100.stdout:665: writing smithi10066867-11 from 4595712 to 4644864 tid 3 2024-04-07T16:18:36.063 INFO:tasks.rados.rados.0.smithi100.stdout:666: read oid 19 snap -1 2024-04-07T16:18:36.063 INFO:tasks.rados.rados.0.smithi100.stdout:666: expect deleted 2024-04-07T16:18:36.063 INFO:tasks.rados.rados.0.smithi100.stdout:667: read oid 49 snap 52 2024-04-07T16:18:36.063 INFO:tasks.rados.rados.0.smithi100.stdout:667: expect (ObjNum 192 snap 50 seq_num 192) 2024-04-07T16:18:36.063 INFO:tasks.rados.rados.0.smithi100.stdout:668: rollback oid 44 current snap is 59 2024-04-07T16:18:36.063 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 44 to 53 2024-04-07T16:18:36.063 INFO:tasks.rados.rados.0.smithi100.stdout:663: finishing copy_from racing read to smithi10066867-45 2024-04-07T16:18:36.063 INFO:tasks.rados.rados.0.smithi100.stdout:663: finishing copy_from to smithi10066867-45 2024-04-07T16:18:36.063 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 45 v 171 (ObjNum 7 snap 0 seq_num 7) dirty exists 2024-04-07T16:18:36.063 INFO:tasks.rados.rados.0.smithi100.stdout:661: expect (ObjNum 140 snap 39 seq_num 140) 2024-04-07T16:18:36.125 INFO:tasks.rados.rados.0.smithi100.stdout:665: finishing write tid 1 to smithi10066867-11 2024-04-07T16:18:36.125 INFO:tasks.rados.rados.0.smithi100.stdout:668: finishing rollback tid 0 to smithi10066867-44 2024-04-07T16:18:36.125 INFO:tasks.rados.rados.0.smithi100.stdout:668: finishing rollback tid 1 to smithi10066867-44 2024-04-07T16:18:36.125 INFO:tasks.rados.rados.0.smithi100.stdout:667: expect (ObjNum 192 snap 50 seq_num 192) 2024-04-07T16:18:36.147 INFO:tasks.rados.rados.0.smithi100.stdout:668: finishing rollback tid 2 to smithi10066867-44 2024-04-07T16:18:36.147 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 44 v 238 (ObjNum 165 snap 44 seq_num 165) dirty exists 2024-04-07T16:18:36.147 INFO:tasks.rados.rados.0.smithi100.stdout:665: finishing write tid 2 to smithi10066867-11 2024-04-07T16:18:36.147 INFO:tasks.rados.rados.0.smithi100.stdout:665: finishing write tid 3 to smithi10066867-11 2024-04-07T16:18:36.147 INFO:tasks.rados.rados.0.smithi100.stdout:665: finishing write tid 4 to smithi10066867-11 2024-04-07T16:18:36.147 INFO:tasks.rados.rados.0.smithi100.stdout:665: finishing write tid 5 to smithi10066867-11 2024-04-07T16:18:36.147 INFO:tasks.rados.rados.0.smithi100.stdout:665: oid 11 updating version 0 to 356 2024-04-07T16:18:36.147 INFO:tasks.rados.rados.0.smithi100.stdout:665: oid 11 version 356 is already newer than 355 2024-04-07T16:18:36.147 INFO:tasks.rados.rados.0.smithi100.stdout:665: oid 11 updating version 356 to 357 2024-04-07T16:18:36.147 INFO:tasks.rados.rados.0.smithi100.stdout:665: oid 11 updating version 357 to 358 2024-04-07T16:18:36.147 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 11 v 358 (ObjNum 220 snap 59 seq_num 220) dirty exists 2024-04-07T16:18:36.147 INFO:tasks.rados.rados.0.smithi100.stdout:665: left oid 11 (ObjNum 220 snap 59 seq_num 220) 2024-04-07T16:18:36.147 INFO:tasks.rados.rados.0.smithi100.stdout:661: done (5 left) 2024-04-07T16:18:36.147 INFO:tasks.rados.rados.0.smithi100.stdout:663: done (4 left) 2024-04-07T16:18:36.148 INFO:tasks.rados.rados.0.smithi100.stdout:665: done (3 left) 2024-04-07T16:18:36.148 INFO:tasks.rados.rados.0.smithi100.stdout:666: done (2 left) 2024-04-07T16:18:36.148 INFO:tasks.rados.rados.0.smithi100.stdout:667: done (1 left) 2024-04-07T16:18:36.148 INFO:tasks.rados.rados.0.smithi100.stdout:668: done (0 left) 2024-04-07T16:18:36.148 INFO:tasks.rados.rados.0.smithi100.stdout:669: setattr oid 28 current snap is 59 2024-04-07T16:18:36.148 INFO:tasks.rados.rados.0.smithi100.stdout:670: snap_remove snap 42 2024-04-07T16:18:36.855 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:36 smithi144 bash[23522]: audit 2024-04-07T16:18:35.477254+0000 mon.a (mon.0) 1037 : audit 1 from='mgr.24553 ' entity='mgr.x' 2024-04-07T16:18:36.855 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:36 smithi144 bash[23522]: audit 2024-04-07T16:18:35.494262+0000 mon.a (mon.0) 1038 : audit 1 from='mgr.24553 ' entity='mgr.x' 2024-04-07T16:18:36.855 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:36 smithi144 bash[23522]: audit 2024-04-07T16:18:35.494380+0000 mon.b (mon.1) 302 : audit 1 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-07T16:18:36.855 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:36 smithi144 bash[23522]: audit 2024-04-07T16:18:35.497584+0000 mon.b (mon.1) 303 : audit 1 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-07T16:18:36.856 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:36 smithi144 bash[23522]: audit 2024-04-07T16:18:35.499752+0000 mon.a (mon.0) 1039 : audit 1 from='mgr.24553 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-07T16:18:36.856 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:36 smithi144 bash[23522]: audit 2024-04-07T16:18:35.500281+0000 mon.b (mon.1) 304 : audit 1 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-07T16:18:36.856 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:36 smithi144 bash[23522]: audit 2024-04-07T16:18:35.502483+0000 mon.a (mon.0) 1040 : audit 1 from='mgr.24553 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-07T16:18:36.856 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:36 smithi144 bash[23522]: audit 2024-04-07T16:18:35.502967+0000 mon.b (mon.1) 305 : audit 1 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-07T16:18:36.856 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:36 smithi144 bash[23522]: audit 2024-04-07T16:18:35.505223+0000 mon.a (mon.0) 1041 : audit 1 from='mgr.24553 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-07T16:18:36.856 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:36 smithi144 bash[23522]: cephadm 2024-04-07T16:18:35.505276+0000 mgr.x (mgr.24553) 15 : cephadm 1 Adjusting osd_memory_target on smithi100 to 2524M 2024-04-07T16:18:36.856 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:36 smithi144 bash[23522]: audit 2024-04-07T16:18:35.507846+0000 mon.a (mon.0) 1042 : audit 1 from='mgr.24553 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-07T16:18:36.856 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:36 smithi144 bash[23522]: audit 2024-04-07T16:18:35.521143+0000 mon.a (mon.0) 1043 : audit 1 from='mgr.24553 ' entity='mgr.x' 2024-04-07T16:18:36.856 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:36 smithi144 bash[23522]: audit 2024-04-07T16:18:35.622732+0000 mon.a (mon.0) 1044 : audit 1 from='mgr.24553 ' entity='mgr.x' 2024-04-07T16:18:36.856 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:36 smithi144 bash[23522]: audit 2024-04-07T16:18:35.637100+0000 mon.a (mon.0) 1045 : audit 1 from='mgr.24553 ' entity='mgr.x' 2024-04-07T16:18:36.856 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:36 smithi144 bash[23522]: audit 2024-04-07T16:18:35.765603+0000 mon.b (mon.1) 306 : audit 0 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:18:36.856 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:36 smithi144 bash[23522]: cluster 2024-04-07T16:18:36.011460+0000 mon.a (mon.0) 1046 : cluster 0 osdmap e182: 8 total, 8 up, 8 in 2024-04-07T16:18:36.937 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:36 smithi100 bash[21346]: audit 2024-04-07T16:18:35.477254+0000 mon.a (mon.0) 1037 : audit 1 from='mgr.24553 ' entity='mgr.x' 2024-04-07T16:18:36.937 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:36 smithi100 bash[21346]: audit 2024-04-07T16:18:35.494262+0000 mon.a (mon.0) 1038 : audit 1 from='mgr.24553 ' entity='mgr.x' 2024-04-07T16:18:36.938 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:36 smithi100 bash[21346]: audit 2024-04-07T16:18:35.494380+0000 mon.b (mon.1) 302 : audit 1 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-07T16:18:36.938 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:36 smithi100 bash[21346]: audit 2024-04-07T16:18:35.497584+0000 mon.b (mon.1) 303 : audit 1 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-07T16:18:36.938 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:36 smithi100 bash[21346]: audit 2024-04-07T16:18:35.499752+0000 mon.a (mon.0) 1039 : audit 1 from='mgr.24553 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-07T16:18:36.938 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:36 smithi100 bash[21346]: audit 2024-04-07T16:18:35.500281+0000 mon.b (mon.1) 304 : audit 1 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-07T16:18:36.938 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:36 smithi100 bash[21346]: audit 2024-04-07T16:18:35.502483+0000 mon.a (mon.0) 1040 : audit 1 from='mgr.24553 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-07T16:18:36.938 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:36 smithi100 bash[21346]: audit 2024-04-07T16:18:35.502967+0000 mon.b (mon.1) 305 : audit 1 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-07T16:18:36.938 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:36 smithi100 bash[21346]: audit 2024-04-07T16:18:35.505223+0000 mon.a (mon.0) 1041 : audit 1 from='mgr.24553 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-07T16:18:36.938 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:36 smithi100 bash[21346]: cephadm 2024-04-07T16:18:35.505276+0000 mgr.x (mgr.24553) 15 : cephadm 1 Adjusting osd_memory_target on smithi100 to 2524M 2024-04-07T16:18:36.938 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:36 smithi100 bash[21346]: audit 2024-04-07T16:18:35.507846+0000 mon.a (mon.0) 1042 : audit 1 from='mgr.24553 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-07T16:18:36.938 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:36 smithi100 bash[21346]: audit 2024-04-07T16:18:35.521143+0000 mon.a (mon.0) 1043 : audit 1 from='mgr.24553 ' entity='mgr.x' 2024-04-07T16:18:36.938 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:36 smithi100 bash[21346]: audit 2024-04-07T16:18:35.622732+0000 mon.a (mon.0) 1044 : audit 1 from='mgr.24553 ' entity='mgr.x' 2024-04-07T16:18:36.938 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:36 smithi100 bash[21346]: audit 2024-04-07T16:18:35.637100+0000 mon.a (mon.0) 1045 : audit 1 from='mgr.24553 ' entity='mgr.x' 2024-04-07T16:18:36.938 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:36 smithi100 bash[21346]: audit 2024-04-07T16:18:35.765603+0000 mon.b (mon.1) 306 : audit 0 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:18:36.939 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:36 smithi100 bash[21346]: cluster 2024-04-07T16:18:36.011460+0000 mon.a (mon.0) 1046 : cluster 0 osdmap e182: 8 total, 8 up, 8 in 2024-04-07T16:18:36.939 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:36 smithi100 bash[31660]: audit 2024-04-07T16:18:35.477254+0000 mon.a (mon.0) 1037 : audit 1 from='mgr.24553 ' entity='mgr.x' 2024-04-07T16:18:36.939 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:36 smithi100 bash[31660]: audit 2024-04-07T16:18:35.494262+0000 mon.a (mon.0) 1038 : audit 1 from='mgr.24553 ' entity='mgr.x' 2024-04-07T16:18:36.939 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:36 smithi100 bash[31660]: audit 2024-04-07T16:18:35.494380+0000 mon.b (mon.1) 302 : audit 1 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-07T16:18:36.939 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:36 smithi100 bash[31660]: audit 2024-04-07T16:18:35.497584+0000 mon.b (mon.1) 303 : audit 1 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-07T16:18:36.939 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:36 smithi100 bash[31660]: audit 2024-04-07T16:18:35.499752+0000 mon.a (mon.0) 1039 : audit 1 from='mgr.24553 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-07T16:18:36.939 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:36 smithi100 bash[31660]: audit 2024-04-07T16:18:35.500281+0000 mon.b (mon.1) 304 : audit 1 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-07T16:18:36.939 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:36 smithi100 bash[31660]: audit 2024-04-07T16:18:35.502483+0000 mon.a (mon.0) 1040 : audit 1 from='mgr.24553 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-07T16:18:36.939 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:36 smithi100 bash[31660]: audit 2024-04-07T16:18:35.502967+0000 mon.b (mon.1) 305 : audit 1 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-07T16:18:36.939 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:36 smithi100 bash[31660]: audit 2024-04-07T16:18:35.505223+0000 mon.a (mon.0) 1041 : audit 1 from='mgr.24553 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-07T16:18:36.939 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:36 smithi100 bash[31660]: cephadm 2024-04-07T16:18:35.505276+0000 mgr.x (mgr.24553) 15 : cephadm 1 Adjusting osd_memory_target on smithi100 to 2524M 2024-04-07T16:18:36.939 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:36 smithi100 bash[31660]: audit 2024-04-07T16:18:35.507846+0000 mon.a (mon.0) 1042 : audit 1 from='mgr.24553 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-07T16:18:36.939 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:36 smithi100 bash[31660]: audit 2024-04-07T16:18:35.521143+0000 mon.a (mon.0) 1043 : audit 1 from='mgr.24553 ' entity='mgr.x' 2024-04-07T16:18:36.940 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:36 smithi100 bash[31660]: audit 2024-04-07T16:18:35.622732+0000 mon.a (mon.0) 1044 : audit 1 from='mgr.24553 ' entity='mgr.x' 2024-04-07T16:18:36.940 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:36 smithi100 bash[31660]: audit 2024-04-07T16:18:35.637100+0000 mon.a (mon.0) 1045 : audit 1 from='mgr.24553 ' entity='mgr.x' 2024-04-07T16:18:36.940 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:36 smithi100 bash[31660]: audit 2024-04-07T16:18:35.765603+0000 mon.b (mon.1) 306 : audit 0 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:18:36.940 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:36 smithi100 bash[31660]: cluster 2024-04-07T16:18:36.011460+0000 mon.a (mon.0) 1046 : cluster 0 osdmap e182: 8 total, 8 up, 8 in 2024-04-07T16:18:37.001 INFO:tasks.rados.rados.0.smithi100.stdout:670: done (1 left) 2024-04-07T16:18:37.001 INFO:tasks.rados.rados.0.smithi100.stdout:671: delete oid 14 current snap is 59 2024-04-07T16:18:37.001 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 28 v 57 (ObjNum 140 snap 39 seq_num 140) dirty exists 2024-04-07T16:18:37.017 INFO:tasks.rados.rados.0.smithi100.stdout:669: done (1 left) 2024-04-07T16:18:37.017 INFO:tasks.rados.rados.0.smithi100.stdout:671: done (0 left) 2024-04-07T16:18:37.017 INFO:tasks.rados.rados.0.smithi100.stdout:672: read oid 42 snap -1 2024-04-07T16:18:37.017 INFO:tasks.rados.rados.0.smithi100.stdout:672: expect (ObjNum 190 snap 49 seq_num 190) 2024-04-07T16:18:37.017 INFO:tasks.rados.rados.0.smithi100.stdout:673: read oid 24 snap -1 2024-04-07T16:18:37.017 INFO:tasks.rados.rados.0.smithi100.stdout:673: expect (ObjNum 157 snap 43 seq_num 157) 2024-04-07T16:18:37.018 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 21 current snap is 59 2024-04-07T16:18:37.018 INFO:tasks.rados.rados.0.smithi100.stdout:674: seq_num 222 ranges {5005312=524288,5529600=565248,6094848=237568} 2024-04-07T16:18:37.055 INFO:tasks.rados.rados.0.smithi100.stdout:674: writing smithi10066867-21 from 5005312 to 5529600 tid 1 2024-04-07T16:18:37.058 INFO:tasks.rados.rados.0.smithi100.stdout:674: writing smithi10066867-21 from 5529600 to 6094848 tid 2 2024-04-07T16:18:37.060 INFO:tasks.rados.rados.0.smithi100.stdout:674: writing smithi10066867-21 from 6094848 to 6332416 tid 3 2024-04-07T16:18:37.060 INFO:tasks.rados.rados.0.smithi100.stdout:675: snap_remove snap 57 2024-04-07T16:18:38.004 INFO:tasks.rados.rados.0.smithi100.stdout:675: done (3 left) 2024-04-07T16:18:38.004 INFO:tasks.rados.rados.0.smithi100.stdout:676: snap_remove snap 47 2024-04-07T16:18:38.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:38 smithi144 bash[23522]: cluster 2024-04-07T16:18:36.645498+0000 mgr.x (mgr.24553) 16 : cluster 0 pgmap v27: 121 pgs: 121 active+clean; 141 MiB data, 576 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:18:38.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:38 smithi144 bash[23522]: cluster 2024-04-07T16:18:37.015311+0000 mon.a (mon.0) 1047 : cluster 0 osdmap e183: 8 total, 8 up, 8 in 2024-04-07T16:18:38.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:38 smithi100 bash[21346]: cluster 2024-04-07T16:18:36.645498+0000 mgr.x (mgr.24553) 16 : cluster 0 pgmap v27: 121 pgs: 121 active+clean; 141 MiB data, 576 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:18:38.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:38 smithi100 bash[21346]: cluster 2024-04-07T16:18:37.015311+0000 mon.a (mon.0) 1047 : cluster 0 osdmap e183: 8 total, 8 up, 8 in 2024-04-07T16:18:38.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:38 smithi100 bash[31660]: cluster 2024-04-07T16:18:36.645498+0000 mgr.x (mgr.24553) 16 : cluster 0 pgmap v27: 121 pgs: 121 active+clean; 141 MiB data, 576 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:18:38.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:38 smithi100 bash[31660]: cluster 2024-04-07T16:18:37.015311+0000 mon.a (mon.0) 1047 : cluster 0 osdmap e183: 8 total, 8 up, 8 in 2024-04-07T16:18:39.028 INFO:tasks.rados.rados.0.smithi100.stdout:676: done (3 left) 2024-04-07T16:18:39.028 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 31 current snap is 59 2024-04-07T16:18:39.028 INFO:tasks.rados.rados.0.smithi100.stdout:673: expect (ObjNum 157 snap 43 seq_num 157) 2024-04-07T16:18:39.157 INFO:tasks.rados.rados.0.smithi100.stdout:672: expect (ObjNum 190 snap 49 seq_num 190) 2024-04-07T16:18:39.275 INFO:tasks.rados.rados.0.smithi100.stdout:674: finishing write tid 1 to smithi10066867-21 2024-04-07T16:18:39.275 INFO:tasks.rados.rados.0.smithi100.stdout:674: finishing write tid 2 to smithi10066867-21 2024-04-07T16:18:39.275 INFO:tasks.rados.rados.0.smithi100.stdout:674: finishing write tid 3 to smithi10066867-21 2024-04-07T16:18:39.275 INFO:tasks.rados.rados.0.smithi100.stdout:674: finishing write tid 4 to smithi10066867-21 2024-04-07T16:18:39.275 INFO:tasks.rados.rados.0.smithi100.stdout:674: finishing write tid 5 to smithi10066867-21 2024-04-07T16:18:39.276 INFO:tasks.rados.rados.0.smithi100.stdout:674: oid 21 updating version 0 to 97 2024-04-07T16:18:39.276 INFO:tasks.rados.rados.0.smithi100.stdout:674: oid 21 updating version 97 to 99 2024-04-07T16:18:39.276 INFO:tasks.rados.rados.0.smithi100.stdout:674: oid 21 version 99 is already newer than 98 2024-04-07T16:18:39.276 INFO:tasks.rados.rados.0.smithi100.stdout:674: oid 21 version 99 is already newer than 96 2024-04-07T16:18:39.276 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 21 v 99 (ObjNum 222 snap 59 seq_num 222) dirty exists 2024-04-07T16:18:39.276 INFO:tasks.rados.rados.0.smithi100.stdout:674: left oid 21 (ObjNum 222 snap 59 seq_num 222) 2024-04-07T16:18:39.276 INFO:tasks.rados.rados.0.smithi100.stdout:677: seq_num 223 ranges {1433600=540672,1974272=450560,2424832=352256} 2024-04-07T16:18:39.294 INFO:tasks.rados.rados.0.smithi100.stdout:677: writing smithi10066867-31 from 1433600 to 1974272 tid 1 2024-04-07T16:18:39.299 INFO:tasks.rados.rados.0.smithi100.stdout:677: writing smithi10066867-31 from 1974272 to 2424832 tid 2 2024-04-07T16:18:39.304 INFO:tasks.rados.rados.0.smithi100.stdout:677: writing smithi10066867-31 from 2424832 to 2777088 tid 3 2024-04-07T16:18:39.304 INFO:tasks.rados.rados.0.smithi100.stdout:672: done (3 left) 2024-04-07T16:18:39.304 INFO:tasks.rados.rados.0.smithi100.stdout:673: done (2 left) 2024-04-07T16:18:39.304 INFO:tasks.rados.rados.0.smithi100.stdout:674: done (1 left) 2024-04-07T16:18:39.304 INFO:tasks.rados.rados.0.smithi100.stdout:678: read oid 9 snap -1 2024-04-07T16:18:39.304 INFO:tasks.rados.rados.0.smithi100.stdout:678: expect (ObjNum 206 snap 56 seq_num 206) 2024-04-07T16:18:39.305 INFO:tasks.rados.rados.0.smithi100.stdout:679: rollback oid 16 current snap is 59 2024-04-07T16:18:39.305 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 16 to 33 2024-04-07T16:18:39.305 INFO:tasks.rados.rados.0.smithi100.stdout:677: finishing write tid 1 to smithi10066867-31 2024-04-07T16:18:39.305 INFO:tasks.rados.rados.0.smithi100.stdout:677: finishing write tid 2 to smithi10066867-31 2024-04-07T16:18:39.305 INFO:tasks.rados.rados.0.smithi100.stdout:680: delete oid 11 current snap is 59 2024-04-07T16:18:39.310 INFO:tasks.rados.rados.0.smithi100.stdout:677: finishing write tid 3 to smithi10066867-31 2024-04-07T16:18:39.311 INFO:tasks.rados.rados.0.smithi100.stdout:680: done (3 left) 2024-04-07T16:18:39.311 INFO:tasks.rados.rados.0.smithi100.stdout:681: read oid 34 snap -1 2024-04-07T16:18:39.311 INFO:tasks.rados.rados.0.smithi100.stdout:681: expect (ObjNum 140 snap 39 seq_num 140) 2024-04-07T16:18:39.311 INFO:tasks.rados.rados.0.smithi100.stdout:682: read oid 44 snap -1 2024-04-07T16:18:39.311 INFO:tasks.rados.rados.0.smithi100.stdout:682: expect (ObjNum 165 snap 44 seq_num 165) 2024-04-07T16:18:39.311 INFO:tasks.rados.rados.0.smithi100.stdout:683: snap_remove snap 45 2024-04-07T16:18:39.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:39 smithi144 bash[23522]: cluster 2024-04-07T16:18:38.017805+0000 mon.a (mon.0) 1048 : cluster 0 osdmap e184: 8 total, 8 up, 8 in 2024-04-07T16:18:39.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:39 smithi100 bash[21346]: cluster 2024-04-07T16:18:38.017805+0000 mon.a (mon.0) 1048 : cluster 0 osdmap e184: 8 total, 8 up, 8 in 2024-04-07T16:18:39.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:39 smithi100 bash[31660]: cluster 2024-04-07T16:18:38.017805+0000 mon.a (mon.0) 1048 : cluster 0 osdmap e184: 8 total, 8 up, 8 in 2024-04-07T16:18:40.030 INFO:tasks.rados.rados.0.smithi100.stdout:683: done (5 left) 2024-04-07T16:18:40.030 INFO:tasks.rados.rados.0.smithi100.stdout:684: snap_remove snap 52 2024-04-07T16:18:40.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:40 smithi144 bash[23522]: cluster 2024-04-07T16:18:38.645967+0000 mgr.x (mgr.24553) 17 : cluster 0 pgmap v30: 121 pgs: 121 active+clean; 163 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 13 MiB/s rd, 3.8 MiB/s wr, 34 op/s 2024-04-07T16:18:40.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:40 smithi144 bash[23522]: cluster 2024-04-07T16:18:39.040309+0000 mon.a (mon.0) 1049 : cluster 0 osdmap e185: 8 total, 8 up, 8 in 2024-04-07T16:18:40.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:40 smithi100 bash[21346]: cluster 2024-04-07T16:18:38.645967+0000 mgr.x (mgr.24553) 17 : cluster 0 pgmap v30: 121 pgs: 121 active+clean; 163 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 13 MiB/s rd, 3.8 MiB/s wr, 34 op/s 2024-04-07T16:18:40.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:40 smithi100 bash[21346]: cluster 2024-04-07T16:18:39.040309+0000 mon.a (mon.0) 1049 : cluster 0 osdmap e185: 8 total, 8 up, 8 in 2024-04-07T16:18:40.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:40 smithi100 bash[31660]: cluster 2024-04-07T16:18:38.645967+0000 mgr.x (mgr.24553) 17 : cluster 0 pgmap v30: 121 pgs: 121 active+clean; 163 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 13 MiB/s rd, 3.8 MiB/s wr, 34 op/s 2024-04-07T16:18:40.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:40 smithi100 bash[31660]: cluster 2024-04-07T16:18:39.040309+0000 mon.a (mon.0) 1049 : cluster 0 osdmap e185: 8 total, 8 up, 8 in 2024-04-07T16:18:41.063 INFO:tasks.rados.rados.0.smithi100.stdout:684: done (5 left) 2024-04-07T16:18:41.063 INFO:tasks.rados.rados.0.smithi100.stdout:685: snap_remove snap 54 2024-04-07T16:18:41.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:41 smithi144 bash[23522]: cluster 2024-04-07T16:18:40.043743+0000 mon.a (mon.0) 1050 : cluster 0 osdmap e186: 8 total, 8 up, 8 in 2024-04-07T16:18:41.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:41 smithi100 bash[21346]: cluster 2024-04-07T16:18:40.043743+0000 mon.a (mon.0) 1050 : cluster 0 osdmap e186: 8 total, 8 up, 8 in 2024-04-07T16:18:41.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:41 smithi100 bash[31660]: cluster 2024-04-07T16:18:40.043743+0000 mon.a (mon.0) 1050 : cluster 0 osdmap e186: 8 total, 8 up, 8 in 2024-04-07T16:18:42.063 INFO:tasks.rados.rados.0.smithi100.stdout:685: done (5 left) 2024-04-07T16:18:42.063 INFO:tasks.rados.rados.0.smithi100.stdout:686: copy_from oid 17 from oid 5 current snap is 59 2024-04-07T16:18:42.063 INFO:tasks.rados.rados.0.smithi100.stdout:679: finishing rollback tid 0 to smithi10066867-16 2024-04-07T16:18:42.063 INFO:tasks.rados.rados.0.smithi100.stdout:679: finishing rollback tid 1 to smithi10066867-16 2024-04-07T16:18:42.063 INFO:tasks.rados.rados.0.smithi100.stdout:677: finishing write tid 4 to smithi10066867-31 2024-04-07T16:18:42.063 INFO:tasks.rados.rados.0.smithi100.stdout:677: finishing write tid 5 to smithi10066867-31 2024-04-07T16:18:42.063 INFO:tasks.rados.rados.0.smithi100.stdout:677: oid 31 updating version 0 to 113 2024-04-07T16:18:42.063 INFO:tasks.rados.rados.0.smithi100.stdout:677: oid 31 updating version 113 to 114 2024-04-07T16:18:42.063 INFO:tasks.rados.rados.0.smithi100.stdout:677: oid 31 version 114 is already newer than 112 2024-04-07T16:18:42.064 INFO:tasks.rados.rados.0.smithi100.stdout:677: oid 31 updating version 114 to 115 2024-04-07T16:18:42.064 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 31 v 115 (ObjNum 223 snap 59 seq_num 223) dirty exists 2024-04-07T16:18:42.064 INFO:tasks.rados.rados.0.smithi100.stdout:677: left oid 31 (ObjNum 223 snap 59 seq_num 223) 2024-04-07T16:18:42.064 INFO:tasks.rados.rados.0.smithi100.stdout:679: finishing rollback tid 2 to smithi10066867-16 2024-04-07T16:18:42.064 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 16 v 289 (ObjNum 118 snap 29 seq_num 118) dirty exists 2024-04-07T16:18:42.064 INFO:tasks.rados.rados.0.smithi100.stdout:678: expect (ObjNum 206 snap 56 seq_num 206) 2024-04-07T16:18:42.144 INFO:tasks.rados.rados.0.smithi100.stdout:682: expect (ObjNum 165 snap 44 seq_num 165) 2024-04-07T16:18:42.192 INFO:tasks.rados.rados.0.smithi100.stdout:681: expect (ObjNum 140 snap 39 seq_num 140) 2024-04-07T16:18:42.252 INFO:tasks.rados.rados.0.smithi100.stdout:686: finishing copy_from to smithi10066867-17 2024-04-07T16:18:42.252 INFO:tasks.rados.rados.0.smithi100.stdout:686: got expected ENOENT (src dne) 2024-04-07T16:18:42.252 INFO:tasks.rados.rados.0.smithi100.stdout:686: finishing copy_from racing read to smithi10066867-17 2024-04-07T16:18:42.252 INFO:tasks.rados.rados.0.smithi100.stdout:677: done (5 left) 2024-04-07T16:18:42.252 INFO:tasks.rados.rados.0.smithi100.stdout:678: done (4 left) 2024-04-07T16:18:42.252 INFO:tasks.rados.rados.0.smithi100.stdout:679: done (3 left) 2024-04-07T16:18:42.252 INFO:tasks.rados.rados.0.smithi100.stdout:681: done (2 left) 2024-04-07T16:18:42.252 INFO:tasks.rados.rados.0.smithi100.stdout:682: done (1 left) 2024-04-07T16:18:42.252 INFO:tasks.rados.rados.0.smithi100.stdout:686: done (0 left) 2024-04-07T16:18:42.252 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 17 current snap is 59 2024-04-07T16:18:42.252 INFO:tasks.rados.rados.0.smithi100.stdout:687: seq_num 225 ranges {0=573440,573440=622592,1196032=180224} 2024-04-07T16:18:42.261 INFO:tasks.rados.rados.0.smithi100.stdout:687: writing smithi10066867-17 from 0 to 573440 tid 1 2024-04-07T16:18:42.273 INFO:tasks.rados.rados.0.smithi100.stdout:687: writing smithi10066867-17 from 573440 to 1196032 tid 2 2024-04-07T16:18:42.276 INFO:tasks.rados.rados.0.smithi100.stdout:687: writing smithi10066867-17 from 1196032 to 1376256 tid 3 2024-04-07T16:18:42.276 INFO:tasks.rados.rados.0.smithi100.stdout:688: read oid 27 snap -1 2024-04-07T16:18:42.276 INFO:tasks.rados.rados.0.smithi100.stdout:688: expect (ObjNum 216 snap 58 seq_num 216) 2024-04-07T16:18:42.276 INFO:tasks.rados.rados.0.smithi100.stdout:689: copy_from oid 41 from oid 28 current snap is 59 2024-04-07T16:18:42.276 INFO:tasks.rados.rados.0.smithi100.stdout:687: finishing write tid 1 to smithi10066867-17 2024-04-07T16:18:42.276 INFO:tasks.rados.rados.0.smithi100.stdout:690: snap_remove snap 46 2024-04-07T16:18:42.350 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:42 smithi144 bash[23522]: cluster 2024-04-07T16:18:40.646694+0000 mgr.x (mgr.24553) 18 : cluster 0 pgmap v33: 121 pgs: 121 active+clean; 163 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 13 MiB/s rd, 3.8 MiB/s wr, 34 op/s 2024-04-07T16:18:42.350 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:42 smithi144 bash[23522]: cluster 2024-04-07T16:18:41.067188+0000 mon.a (mon.0) 1051 : cluster 0 osdmap e187: 8 total, 8 up, 8 in 2024-04-07T16:18:42.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:42 smithi100 bash[21346]: cluster 2024-04-07T16:18:40.646694+0000 mgr.x (mgr.24553) 18 : cluster 0 pgmap v33: 121 pgs: 121 active+clean; 163 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 13 MiB/s rd, 3.8 MiB/s wr, 34 op/s 2024-04-07T16:18:42.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:42 smithi100 bash[21346]: cluster 2024-04-07T16:18:41.067188+0000 mon.a (mon.0) 1051 : cluster 0 osdmap e187: 8 total, 8 up, 8 in 2024-04-07T16:18:42.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:42 smithi100 bash[31660]: cluster 2024-04-07T16:18:40.646694+0000 mgr.x (mgr.24553) 18 : cluster 0 pgmap v33: 121 pgs: 121 active+clean; 163 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 13 MiB/s rd, 3.8 MiB/s wr, 34 op/s 2024-04-07T16:18:42.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:42 smithi100 bash[31660]: cluster 2024-04-07T16:18:41.067188+0000 mon.a (mon.0) 1051 : cluster 0 osdmap e187: 8 total, 8 up, 8 in 2024-04-07T16:18:42.856 INFO:tasks.rados.rados.0.smithi100.stdout:690: done (3 left) 2024-04-07T16:18:42.856 INFO:tasks.rados.rados.0.smithi100.stdout:691: delete oid 3 current snap is 59 2024-04-07T16:18:42.857 INFO:tasks.rados.rados.0.smithi100.stdout:687: finishing write tid 2 to smithi10066867-17 2024-04-07T16:18:42.857 INFO:tasks.rados.rados.0.smithi100.stdout:687: finishing write tid 3 to smithi10066867-17 2024-04-07T16:18:42.857 INFO:tasks.rados.rados.0.smithi100.stdout:687: finishing write tid 4 to smithi10066867-17 2024-04-07T16:18:42.857 INFO:tasks.rados.rados.0.smithi100.stdout:687: finishing write tid 5 to smithi10066867-17 2024-04-07T16:18:42.857 INFO:tasks.rados.rados.0.smithi100.stdout:687: oid 17 updating version 0 to 239 2024-04-07T16:18:42.857 INFO:tasks.rados.rados.0.smithi100.stdout:687: oid 17 version 239 is already newer than 237 2024-04-07T16:18:42.857 INFO:tasks.rados.rados.0.smithi100.stdout:687: oid 17 version 239 is already newer than 238 2024-04-07T16:18:42.857 INFO:tasks.rados.rados.0.smithi100.stdout:687: oid 17 updating version 239 to 240 2024-04-07T16:18:42.857 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 17 v 240 (ObjNum 225 snap 59 seq_num 225) dirty exists 2024-04-07T16:18:42.857 INFO:tasks.rados.rados.0.smithi100.stdout:687: left oid 17 (ObjNum 225 snap 59 seq_num 225) 2024-04-07T16:18:42.857 INFO:tasks.rados.rados.0.smithi100.stdout:688: expect (ObjNum 216 snap 58 seq_num 216) 2024-04-07T16:18:42.948 INFO:tasks.rados.rados.0.smithi100.stdout:689: finishing copy_from racing read to smithi10066867-41 2024-04-07T16:18:42.948 INFO:tasks.rados.rados.0.smithi100.stdout:689: finishing copy_from to smithi10066867-41 2024-04-07T16:18:42.948 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 41 v 57 (ObjNum 140 snap 39 seq_num 140) dirty exists 2024-04-07T16:18:42.948 INFO:tasks.rados.rados.0.smithi100.stdout:687: done (3 left) 2024-04-07T16:18:42.948 INFO:tasks.rados.rados.0.smithi100.stdout:688: done (2 left) 2024-04-07T16:18:42.948 INFO:tasks.rados.rados.0.smithi100.stdout:689: done (1 left) 2024-04-07T16:18:42.948 INFO:tasks.rados.rados.0.smithi100.stdout:691: done (0 left) 2024-04-07T16:18:42.948 INFO:tasks.rados.rados.0.smithi100.stdout:692: read oid 24 snap -1 2024-04-07T16:18:42.948 INFO:tasks.rados.rados.0.smithi100.stdout:692: expect (ObjNum 157 snap 43 seq_num 157) 2024-04-07T16:18:42.948 INFO:tasks.rados.rados.0.smithi100.stdout:693: read oid 13 snap -1 2024-04-07T16:18:42.948 INFO:tasks.rados.rados.0.smithi100.stdout:693: expect (ObjNum 7 snap 0 seq_num 7) 2024-04-07T16:18:42.948 INFO:tasks.rados.rados.0.smithi100.stdout:694: rollback oid 1 current snap is 59 2024-04-07T16:18:42.949 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 1 to 58 2024-04-07T16:18:42.949 INFO:tasks.rados.rados.0.smithi100.stdout:695: rollback oid 7 current snap is 59 2024-04-07T16:18:42.949 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 7 to 53 2024-04-07T16:18:42.949 INFO:tasks.rados.rados.0.smithi100.stdout:696: setattr oid 45 current snap is 59 2024-04-07T16:18:42.952 INFO:tasks.rados.rados.0.smithi100.stdout:697: delete oid 18 current snap is 59 2024-04-07T16:18:42.955 INFO:tasks.rados.rados.0.smithi100.stdout:693: expect (ObjNum 7 snap 0 seq_num 7) 2024-04-07T16:18:42.967 INFO:tasks.rados.rados.0.smithi100.stdout:695: finishing rollback tid 0 to smithi10066867-7 2024-04-07T16:18:42.967 INFO:tasks.rados.rados.0.smithi100.stdout:695: finishing rollback tid 1 to smithi10066867-7 2024-04-07T16:18:42.967 INFO:tasks.rados.rados.0.smithi100.stdout:694: finishing rollback tid 0 to smithi10066867-1 2024-04-07T16:18:42.967 INFO:tasks.rados.rados.0.smithi100.stdout:695: finishing rollback tid 2 to smithi10066867-7 2024-04-07T16:18:42.967 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 7 v 149 (ObjNum 16 snap 0 seq_num 16) dirty exists 2024-04-07T16:18:42.967 INFO:tasks.rados.rados.0.smithi100.stdout:694: finishing rollback tid 1 to smithi10066867-1 2024-04-07T16:18:42.967 INFO:tasks.rados.rados.0.smithi100.stdout:694: finishing rollback tid 2 to smithi10066867-1 2024-04-07T16:18:42.967 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 1 v 67 (ObjNum 161 snap 0 seq_num 1) dirty exists 2024-04-07T16:18:42.967 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 45 v 180 (ObjNum 7 snap 0 seq_num 7) dirty exists 2024-04-07T16:18:42.967 INFO:tasks.rados.rados.0.smithi100.stdout:692: expect (ObjNum 157 snap 43 seq_num 157) 2024-04-07T16:18:43.066 INFO:tasks.rados.rados.0.smithi100.stdout:692: done (5 left) 2024-04-07T16:18:43.066 INFO:tasks.rados.rados.0.smithi100.stdout:693: done (4 left) 2024-04-07T16:18:43.066 INFO:tasks.rados.rados.0.smithi100.stdout:694: done (3 left) 2024-04-07T16:18:43.067 INFO:tasks.rados.rados.0.smithi100.stdout:695: done (2 left) 2024-04-07T16:18:43.067 INFO:tasks.rados.rados.0.smithi100.stdout:696: done (1 left) 2024-04-07T16:18:43.067 INFO:tasks.rados.rados.0.smithi100.stdout:697: done (0 left) 2024-04-07T16:18:43.067 INFO:tasks.rados.rados.0.smithi100.stdout:698: delete oid 49 current snap is 59 2024-04-07T16:18:43.070 INFO:tasks.rados.rados.0.smithi100.stdout:698: done (0 left) 2024-04-07T16:18:43.070 INFO:tasks.rados.rados.0.smithi100.stdout:699: rmattr oid 13 current snap is 59 2024-04-07T16:18:43.070 INFO:tasks.rados.rados.0.smithi100.stdout:700: read oid 28 snap -1 2024-04-07T16:18:43.070 INFO:tasks.rados.rados.0.smithi100.stdout:700: expect (ObjNum 140 snap 39 seq_num 140) 2024-04-07T16:18:43.070 INFO:tasks.rados.rados.0.smithi100.stdout:701: delete oid 29 current snap is 59 2024-04-07T16:18:43.074 INFO:tasks.rados.rados.0.smithi100.stdout:701: done (2 left) 2024-04-07T16:18:43.074 INFO:tasks.rados.rados.0.smithi100.stdout:702: rollback oid 24 current snap is 59 2024-04-07T16:18:43.074 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 24 to 58 2024-04-07T16:18:43.074 INFO:tasks.rados.rados.0.smithi100.stdout:703: read oid 4 snap -1 2024-04-07T16:18:43.074 INFO:tasks.rados.rados.0.smithi100.stdout:703: expect (ObjNum 171 snap 44 seq_num 171) 2024-04-07T16:18:43.074 INFO:tasks.rados.rados.0.smithi100.stdout:704: snap_remove snap 56 2024-04-07T16:18:43.354 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:43 smithi144 bash[23522]: cluster 2024-04-07T16:18:42.077049+0000 mon.a (mon.0) 1052 : cluster 0 osdmap e188: 8 total, 8 up, 8 in 2024-04-07T16:18:43.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:43 smithi144 bash[23522]: cluster 2024-04-07T16:18:42.870128+0000 mon.a (mon.0) 1053 : cluster 0 osdmap e189: 8 total, 8 up, 8 in 2024-04-07T16:18:43.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:43 smithi100 bash[21346]: cluster 2024-04-07T16:18:42.077049+0000 mon.a (mon.0) 1052 : cluster 0 osdmap e188: 8 total, 8 up, 8 in 2024-04-07T16:18:43.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:43 smithi100 bash[21346]: cluster 2024-04-07T16:18:42.870128+0000 mon.a (mon.0) 1053 : cluster 0 osdmap e189: 8 total, 8 up, 8 in 2024-04-07T16:18:43.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:43 smithi100 bash[31660]: cluster 2024-04-07T16:18:42.077049+0000 mon.a (mon.0) 1052 : cluster 0 osdmap e188: 8 total, 8 up, 8 in 2024-04-07T16:18:43.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:43 smithi100 bash[31660]: cluster 2024-04-07T16:18:42.870128+0000 mon.a (mon.0) 1053 : cluster 0 osdmap e189: 8 total, 8 up, 8 in 2024-04-07T16:18:44.080 INFO:tasks.rados.rados.0.smithi100.stdout:704: done (4 left) 2024-04-07T16:18:44.080 INFO:tasks.rados.rados.0.smithi100.stdout:705: read oid 42 snap -1 2024-04-07T16:18:44.080 INFO:tasks.rados.rados.0.smithi100.stdout:705: expect (ObjNum 190 snap 49 seq_num 190) 2024-04-07T16:18:44.080 INFO:tasks.rados.rados.0.smithi100.stdout:706: snap_remove snap 33 2024-04-07T16:18:44.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:44 smithi144 bash[23522]: cluster 2024-04-07T16:18:42.647451+0000 mgr.x (mgr.24553) 19 : cluster 0 pgmap v36: 121 pgs: 121 active+clean; 151 MiB data, 712 MiB used, 715 GiB / 715 GiB avail; 6.5 MiB/s rd, 652 KiB/s wr, 8 op/s 2024-04-07T16:18:44.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:44 smithi100 bash[31660]: cluster 2024-04-07T16:18:42.647451+0000 mgr.x (mgr.24553) 19 : cluster 0 pgmap v36: 121 pgs: 121 active+clean; 151 MiB data, 712 MiB used, 715 GiB / 715 GiB avail; 6.5 MiB/s rd, 652 KiB/s wr, 8 op/s 2024-04-07T16:18:44.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:44 smithi100 bash[21346]: cluster 2024-04-07T16:18:42.647451+0000 mgr.x (mgr.24553) 19 : cluster 0 pgmap v36: 121 pgs: 121 active+clean; 151 MiB data, 712 MiB used, 715 GiB / 715 GiB avail; 6.5 MiB/s rd, 652 KiB/s wr, 8 op/s 2024-04-07T16:18:45.100 INFO:tasks.rados.rados.0.smithi100.stdout:706: done (5 left) 2024-04-07T16:18:45.107 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 13 v 231 (ObjNum 7 snap 0 seq_num 7) dirty exists 2024-04-07T16:18:45.107 INFO:tasks.rados.rados.0.smithi100.stdout:702: finishing rollback tid 0 to smithi10066867-24 2024-04-07T16:18:45.107 INFO:tasks.rados.rados.0.smithi100.stdout:702: finishing rollback tid 1 to smithi10066867-24 2024-04-07T16:18:45.107 INFO:tasks.rados.rados.0.smithi100.stdout:702: finishing rollback tid 2 to smithi10066867-24 2024-04-07T16:18:45.107 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 24 v 246 (ObjNum 157 snap 43 seq_num 157) dirty exists 2024-04-07T16:18:45.107 INFO:tasks.rados.rados.0.smithi100.stdout:703: expect (ObjNum 171 snap 44 seq_num 171) 2024-04-07T16:18:45.130 INFO:tasks.rados.rados.0.smithi100.stdout:700: expect (ObjNum 140 snap 39 seq_num 140) 2024-04-07T16:18:45.193 INFO:tasks.rados.rados.0.smithi100.stdout:705: expect (ObjNum 190 snap 49 seq_num 190) 2024-04-07T16:18:45.313 INFO:tasks.rados.rados.0.smithi100.stdout:707: read oid 1 snap -1 2024-04-07T16:18:45.313 INFO:tasks.rados.rados.0.smithi100.stdout:707: expect (ObjNum 0 snap 0 seq_num 0) 2024-04-07T16:18:45.313 INFO:tasks.rados.rados.0.smithi100.stdout:699: done (5 left) 2024-04-07T16:18:45.313 INFO:tasks.rados.rados.0.smithi100.stdout:700: done (4 left) 2024-04-07T16:18:45.313 INFO:tasks.rados.rados.0.smithi100.stdout:702: done (3 left) 2024-04-07T16:18:45.313 INFO:tasks.rados.rados.0.smithi100.stdout:703: done (2 left) 2024-04-07T16:18:45.313 INFO:tasks.rados.rados.0.smithi100.stdout:705: done (1 left) 2024-04-07T16:18:45.313 INFO:tasks.rados.rados.0.smithi100.stdout:708: read oid 2 snap 53 2024-04-07T16:18:45.313 INFO:tasks.rados.rados.0.smithi100.stdout:708: expect (ObjNum 199 snap 52 seq_num 199) 2024-04-07T16:18:45.313 INFO:tasks.rados.rados.0.smithi100.stdout:709: rollback oid 42 current snap is 59 2024-04-07T16:18:45.314 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 42 to 58 2024-04-07T16:18:45.314 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 46 current snap is 59 2024-04-07T16:18:45.314 INFO:tasks.rados.rados.0.smithi100.stdout:710: seq_num 230 ranges {761856=655360,1417216=491520,1908736=311296} 2024-04-07T16:18:45.325 INFO:tasks.rados.rados.0.smithi100.stdout:710: writing smithi10066867-46 from 761856 to 1417216 tid 1 2024-04-07T16:18:45.329 INFO:tasks.rados.rados.0.smithi100.stdout:710: writing smithi10066867-46 from 1417216 to 1908736 tid 2 2024-04-07T16:18:45.332 INFO:tasks.rados.rados.0.smithi100.stdout:710: writing smithi10066867-46 from 1908736 to 2220032 tid 3 2024-04-07T16:18:45.332 INFO:tasks.rados.rados.0.smithi100.stdout:711: rollback oid 33 current snap is 59 2024-04-07T16:18:45.332 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 33 to 48 2024-04-07T16:18:45.332 INFO:tasks.rados.rados.0.smithi100.stdout:707: expect (ObjNum 0 snap 0 seq_num 0) 2024-04-07T16:18:45.337 INFO:tasks.rados.rados.0.smithi100.stdout:709: finishing rollback tid 0 to smithi10066867-42 2024-04-07T16:18:45.337 INFO:tasks.rados.rados.0.smithi100.stdout:709: finishing rollback tid 1 to smithi10066867-42 2024-04-07T16:18:45.337 INFO:tasks.rados.rados.0.smithi100.stdout:709: finishing rollback tid 2 to smithi10066867-42 2024-04-07T16:18:45.337 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 42 v 273 (ObjNum 190 snap 49 seq_num 190) dirty exists 2024-04-07T16:18:45.337 INFO:tasks.rados.rados.0.smithi100.stdout:710: finishing write tid 1 to smithi10066867-46 2024-04-07T16:18:45.337 INFO:tasks.rados.rados.0.smithi100.stdout:707: done (4 left) 2024-04-07T16:18:45.337 INFO:tasks.rados.rados.0.smithi100.stdout:709: done (3 left) 2024-04-07T16:18:45.337 INFO:tasks.rados.rados.0.smithi100.stdout:712: read oid 45 snap -1 2024-04-07T16:18:45.337 INFO:tasks.rados.rados.0.smithi100.stdout:712: expect (ObjNum 7 snap 0 seq_num 7) 2024-04-07T16:18:45.337 INFO:tasks.rados.rados.0.smithi100.stdout:713: snap_create 2024-04-07T16:18:45.337 INFO:tasks.rados.rados.0.smithi100.stdout:710: finishing write tid 2 to smithi10066867-46 2024-04-07T16:18:45.338 INFO:tasks.rados.rados.0.smithi100.stdout:708: expect (ObjNum 199 snap 52 seq_num 199) 2024-04-07T16:18:45.354 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:45 smithi144 bash[23522]: cluster 2024-04-07T16:18:44.093835+0000 mon.a (mon.0) 1054 : cluster 0 osdmap e190: 8 total, 8 up, 8 in 2024-04-07T16:18:45.354 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:45 smithi144 bash[23522]: cluster 2024-04-07T16:18:44.648242+0000 mgr.x (mgr.24553) 20 : cluster 0 pgmap v39: 121 pgs: 121 active+clean; 151 MiB data, 712 MiB used, 715 GiB / 715 GiB avail; 6.5 MiB/s rd, 652 KiB/s wr, 8 op/s 2024-04-07T16:18:45.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:45 smithi144 bash[23522]: audit 2024-04-07T16:18:44.990463+0000 mon.a (mon.0) 1055 : audit 1 from='mgr.24553 ' entity='mgr.x' 2024-04-07T16:18:45.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:45 smithi144 bash[23522]: audit 2024-04-07T16:18:45.008551+0000 mon.a (mon.0) 1056 : audit 1 from='mgr.24553 ' entity='mgr.x' 2024-04-07T16:18:45.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:45 smithi144 bash[23522]: audit 2024-04-07T16:18:45.008982+0000 mon.b (mon.1) 307 : audit 1 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-07T16:18:45.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:45 smithi144 bash[23522]: audit 2024-04-07T16:18:45.012396+0000 mon.b (mon.1) 308 : audit 1 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-07T16:18:45.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:45 smithi144 bash[23522]: audit 2024-04-07T16:18:45.014148+0000 mon.a (mon.0) 1057 : audit 1 from='mgr.24553 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-07T16:18:45.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:45 smithi144 bash[23522]: audit 2024-04-07T16:18:45.015342+0000 mon.b (mon.1) 309 : audit 1 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-07T16:18:45.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:45 smithi144 bash[23522]: audit 2024-04-07T16:18:45.017494+0000 mon.a (mon.0) 1058 : audit 1 from='mgr.24553 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-07T16:18:45.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:45 smithi144 bash[23522]: audit 2024-04-07T16:18:45.018220+0000 mon.b (mon.1) 310 : audit 1 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-07T16:18:45.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:45 smithi144 bash[23522]: audit 2024-04-07T16:18:45.020301+0000 mon.a (mon.0) 1059 : audit 1 from='mgr.24553 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-07T16:18:45.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:45 smithi144 bash[23522]: cephadm 2024-04-07T16:18:45.020768+0000 mgr.x (mgr.24553) 21 : cephadm 1 Adjusting osd_memory_target on smithi144 to 2524M 2024-04-07T16:18:45.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:45 smithi144 bash[23522]: audit 2024-04-07T16:18:45.023196+0000 mon.a (mon.0) 1060 : audit 1 from='mgr.24553 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-07T16:18:45.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:45 smithi144 bash[23522]: audit 2024-04-07T16:18:45.035829+0000 mon.b (mon.1) 311 : audit 0 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:18:45.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:45 smithi144 bash[23522]: audit 2024-04-07T16:18:45.036001+0000 mon.a (mon.0) 1061 : audit 1 from='mgr.24553 ' entity='mgr.x' 2024-04-07T16:18:45.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:45 smithi144 bash[23522]: audit 2024-04-07T16:18:45.037628+0000 mon.b (mon.1) 312 : audit 1 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:18:45.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:45 smithi144 bash[23522]: cephadm 2024-04-07T16:18:45.039986+0000 mgr.x (mgr.24553) 22 : cephadm 1 Updating smithi100:/etc/ceph/ceph.conf 2024-04-07T16:18:45.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:45 smithi144 bash[23522]: cephadm 2024-04-07T16:18:45.040525+0000 mgr.x (mgr.24553) 23 : cephadm 1 Updating smithi144:/etc/ceph/ceph.conf 2024-04-07T16:18:45.436 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:18:45 smithi100 bash[21601]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:18:45] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-04-07T16:18:45.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:45 smithi100 bash[21346]: cluster 2024-04-07T16:18:44.093835+0000 mon.a (mon.0) 1054 : cluster 0 osdmap e190: 8 total, 8 up, 8 in 2024-04-07T16:18:45.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:45 smithi100 bash[21346]: cluster 2024-04-07T16:18:44.648242+0000 mgr.x (mgr.24553) 20 : cluster 0 pgmap v39: 121 pgs: 121 active+clean; 151 MiB data, 712 MiB used, 715 GiB / 715 GiB avail; 6.5 MiB/s rd, 652 KiB/s wr, 8 op/s 2024-04-07T16:18:45.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:45 smithi100 bash[21346]: audit 2024-04-07T16:18:44.990463+0000 mon.a (mon.0) 1055 : audit 1 from='mgr.24553 ' entity='mgr.x' 2024-04-07T16:18:45.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:45 smithi100 bash[21346]: audit 2024-04-07T16:18:45.008551+0000 mon.a (mon.0) 1056 : audit 1 from='mgr.24553 ' entity='mgr.x' 2024-04-07T16:18:45.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:45 smithi100 bash[21346]: audit 2024-04-07T16:18:45.008982+0000 mon.b (mon.1) 307 : audit 1 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-07T16:18:45.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:45 smithi100 bash[21346]: audit 2024-04-07T16:18:45.012396+0000 mon.b (mon.1) 308 : audit 1 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-07T16:18:45.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:45 smithi100 bash[21346]: audit 2024-04-07T16:18:45.014148+0000 mon.a (mon.0) 1057 : audit 1 from='mgr.24553 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-07T16:18:45.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:45 smithi100 bash[21346]: audit 2024-04-07T16:18:45.015342+0000 mon.b (mon.1) 309 : audit 1 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-07T16:18:45.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:45 smithi100 bash[21346]: audit 2024-04-07T16:18:45.017494+0000 mon.a (mon.0) 1058 : audit 1 from='mgr.24553 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-07T16:18:45.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:45 smithi100 bash[21346]: audit 2024-04-07T16:18:45.018220+0000 mon.b (mon.1) 310 : audit 1 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-07T16:18:45.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:45 smithi100 bash[21346]: audit 2024-04-07T16:18:45.020301+0000 mon.a (mon.0) 1059 : audit 1 from='mgr.24553 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-07T16:18:45.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:45 smithi100 bash[21346]: cephadm 2024-04-07T16:18:45.020768+0000 mgr.x (mgr.24553) 21 : cephadm 1 Adjusting osd_memory_target on smithi144 to 2524M 2024-04-07T16:18:45.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:45 smithi100 bash[21346]: audit 2024-04-07T16:18:45.023196+0000 mon.a (mon.0) 1060 : audit 1 from='mgr.24553 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-07T16:18:45.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:45 smithi100 bash[21346]: audit 2024-04-07T16:18:45.035829+0000 mon.b (mon.1) 311 : audit 0 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:18:45.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:45 smithi100 bash[21346]: audit 2024-04-07T16:18:45.036001+0000 mon.a (mon.0) 1061 : audit 1 from='mgr.24553 ' entity='mgr.x' 2024-04-07T16:18:45.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:45 smithi100 bash[21346]: audit 2024-04-07T16:18:45.037628+0000 mon.b (mon.1) 312 : audit 1 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:18:45.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:45 smithi100 bash[21346]: cephadm 2024-04-07T16:18:45.039986+0000 mgr.x (mgr.24553) 22 : cephadm 1 Updating smithi100:/etc/ceph/ceph.conf 2024-04-07T16:18:45.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:45 smithi100 bash[21346]: cephadm 2024-04-07T16:18:45.040525+0000 mgr.x (mgr.24553) 23 : cephadm 1 Updating smithi144:/etc/ceph/ceph.conf 2024-04-07T16:18:45.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:45 smithi100 bash[31660]: cluster 2024-04-07T16:18:44.093835+0000 mon.a (mon.0) 1054 : cluster 0 osdmap e190: 8 total, 8 up, 8 in 2024-04-07T16:18:45.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:45 smithi100 bash[31660]: cluster 2024-04-07T16:18:44.648242+0000 mgr.x (mgr.24553) 20 : cluster 0 pgmap v39: 121 pgs: 121 active+clean; 151 MiB data, 712 MiB used, 715 GiB / 715 GiB avail; 6.5 MiB/s rd, 652 KiB/s wr, 8 op/s 2024-04-07T16:18:45.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:45 smithi100 bash[31660]: audit 2024-04-07T16:18:44.990463+0000 mon.a (mon.0) 1055 : audit 1 from='mgr.24553 ' entity='mgr.x' 2024-04-07T16:18:45.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:45 smithi100 bash[31660]: audit 2024-04-07T16:18:45.008551+0000 mon.a (mon.0) 1056 : audit 1 from='mgr.24553 ' entity='mgr.x' 2024-04-07T16:18:45.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:45 smithi100 bash[31660]: audit 2024-04-07T16:18:45.008982+0000 mon.b (mon.1) 307 : audit 1 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-07T16:18:45.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:45 smithi100 bash[31660]: audit 2024-04-07T16:18:45.012396+0000 mon.b (mon.1) 308 : audit 1 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-07T16:18:45.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:45 smithi100 bash[31660]: audit 2024-04-07T16:18:45.014148+0000 mon.a (mon.0) 1057 : audit 1 from='mgr.24553 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-07T16:18:45.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:45 smithi100 bash[31660]: audit 2024-04-07T16:18:45.015342+0000 mon.b (mon.1) 309 : audit 1 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-07T16:18:45.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:45 smithi100 bash[31660]: audit 2024-04-07T16:18:45.017494+0000 mon.a (mon.0) 1058 : audit 1 from='mgr.24553 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-07T16:18:45.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:45 smithi100 bash[31660]: audit 2024-04-07T16:18:45.018220+0000 mon.b (mon.1) 310 : audit 1 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-07T16:18:45.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:45 smithi100 bash[31660]: audit 2024-04-07T16:18:45.020301+0000 mon.a (mon.0) 1059 : audit 1 from='mgr.24553 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-07T16:18:45.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:45 smithi100 bash[31660]: cephadm 2024-04-07T16:18:45.020768+0000 mgr.x (mgr.24553) 21 : cephadm 1 Adjusting osd_memory_target on smithi144 to 2524M 2024-04-07T16:18:45.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:45 smithi100 bash[31660]: audit 2024-04-07T16:18:45.023196+0000 mon.a (mon.0) 1060 : audit 1 from='mgr.24553 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-07T16:18:45.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:45 smithi100 bash[31660]: audit 2024-04-07T16:18:45.035829+0000 mon.b (mon.1) 311 : audit 0 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:18:45.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:45 smithi100 bash[31660]: audit 2024-04-07T16:18:45.036001+0000 mon.a (mon.0) 1061 : audit 1 from='mgr.24553 ' entity='mgr.x' 2024-04-07T16:18:45.440 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:45 smithi100 bash[31660]: audit 2024-04-07T16:18:45.037628+0000 mon.b (mon.1) 312 : audit 1 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:18:45.440 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:45 smithi100 bash[31660]: cephadm 2024-04-07T16:18:45.039986+0000 mgr.x (mgr.24553) 22 : cephadm 1 Updating smithi100:/etc/ceph/ceph.conf 2024-04-07T16:18:45.440 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:45 smithi100 bash[31660]: cephadm 2024-04-07T16:18:45.040525+0000 mgr.x (mgr.24553) 23 : cephadm 1 Updating smithi144:/etc/ceph/ceph.conf 2024-04-07T16:18:45.624 INFO:tasks.rados.rados.0.smithi100.stdout:711: finishing rollback tid 1 to smithi10066867-33 2024-04-07T16:18:45.624 INFO:tasks.rados.rados.0.smithi100.stdout:710: finishing write tid 3 to smithi10066867-46 2024-04-07T16:18:45.624 INFO:tasks.rados.rados.0.smithi100.stdout:710: finishing write tid 4 to smithi10066867-46 2024-04-07T16:18:45.624 INFO:tasks.rados.rados.0.smithi100.stdout:711: finishing rollback tid 2 to smithi10066867-33 2024-04-07T16:18:45.624 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 33 v 441 (ObjNum 161 snap 43 seq_num 161) dirty exists 2024-04-07T16:18:45.624 INFO:tasks.rados.rados.0.smithi100.stdout:710: finishing write tid 5 to smithi10066867-46 2024-04-07T16:18:45.624 INFO:tasks.rados.rados.0.smithi100.stdout:710: oid 46 updating version 0 to 245 2024-04-07T16:18:45.624 INFO:tasks.rados.rados.0.smithi100.stdout:710: oid 46 updating version 245 to 248 2024-04-07T16:18:45.625 INFO:tasks.rados.rados.0.smithi100.stdout:710: oid 46 version 248 is already newer than 247 2024-04-07T16:18:45.625 INFO:tasks.rados.rados.0.smithi100.stdout:710: oid 46 version 248 is already newer than 246 2024-04-07T16:18:45.625 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 46 v 248 (ObjNum 230 snap 59 seq_num 230) dirty exists 2024-04-07T16:18:45.625 INFO:tasks.rados.rados.0.smithi100.stdout:710: left oid 46 (ObjNum 230 snap 59 seq_num 230) 2024-04-07T16:18:45.625 INFO:tasks.rados.rados.0.smithi100.stdout:712: expect (ObjNum 7 snap 0 seq_num 7) 2024-04-07T16:18:46.104 INFO:tasks.rados.rados.0.smithi100.stdout:708: done (4 left) 2024-04-07T16:18:46.104 INFO:tasks.rados.rados.0.smithi100.stdout:710: done (3 left) 2024-04-07T16:18:46.104 INFO:tasks.rados.rados.0.smithi100.stdout:711: done (2 left) 2024-04-07T16:18:46.104 INFO:tasks.rados.rados.0.smithi100.stdout:712: done (1 left) 2024-04-07T16:18:46.105 INFO:tasks.rados.rados.0.smithi100.stdout:713: done (0 left) 2024-04-07T16:18:46.105 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 1 current snap is 60 2024-04-07T16:18:46.105 INFO:tasks.rados.rados.0.smithi100.stdout:714: seq_num 231 ranges {0=671744,671744=778240,1449984=24576} 2024-04-07T16:18:46.115 INFO:tasks.rados.rados.0.smithi100.stdout:714: writing smithi10066867-1 from 0 to 671744 tid 1 2024-04-07T16:18:46.130 INFO:tasks.rados.rados.0.smithi100.stdout:714: writing smithi10066867-1 from 671744 to 1449984 tid 2 2024-04-07T16:18:46.130 INFO:tasks.rados.rados.0.smithi100.stdout:714: writing smithi10066867-1 from 1449984 to 1474560 tid 3 2024-04-07T16:18:46.130 INFO:tasks.rados.rados.0.smithi100.stdout:715: delete oid 25 current snap is 60 2024-04-07T16:18:46.130 INFO:tasks.rados.rados.0.smithi100.stdout:714: finishing write tid 1 to smithi10066867-1 2024-04-07T16:18:46.137 INFO:tasks.rados.rados.0.smithi100.stdout:715: done (1 left) 2024-04-07T16:18:46.137 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 34 current snap is 60 2024-04-07T16:18:46.137 INFO:tasks.rados.rados.0.smithi100.stdout:716: seq_num 233 ranges {2359296=712704,3072000=548864,3620864=253952} 2024-04-07T16:18:46.163 INFO:tasks.rados.rados.0.smithi100.stdout:716: writing smithi10066867-34 from 2359296 to 3072000 tid 1 2024-04-07T16:18:46.167 INFO:tasks.rados.rados.0.smithi100.stdout:716: writing smithi10066867-34 from 3072000 to 3620864 tid 2 2024-04-07T16:18:46.170 INFO:tasks.rados.rados.0.smithi100.stdout:716: writing smithi10066867-34 from 3620864 to 3874816 tid 3 2024-04-07T16:18:46.171 INFO:tasks.rados.rados.0.smithi100.stdout:717: delete oid 3 current snap is 60 2024-04-07T16:18:46.171 INFO:tasks.rados.rados.0.smithi100.stdout:714: finishing write tid 2 to smithi10066867-1 2024-04-07T16:18:46.171 INFO:tasks.rados.rados.0.smithi100.stdout:714: finishing write tid 3 to smithi10066867-1 2024-04-07T16:18:46.171 INFO:tasks.rados.rados.0.smithi100.stdout:714: finishing write tid 4 to smithi10066867-1 2024-04-07T16:18:46.171 INFO:tasks.rados.rados.0.smithi100.stdout:714: finishing write tid 5 to smithi10066867-1 2024-04-07T16:18:46.171 INFO:tasks.rados.rados.0.smithi100.stdout:714: oid 1 updating version 0 to 73 2024-04-07T16:18:46.171 INFO:tasks.rados.rados.0.smithi100.stdout:714: oid 1 updating version 73 to 74 2024-04-07T16:18:46.171 INFO:tasks.rados.rados.0.smithi100.stdout:714: oid 1 version 74 is already newer than 71 2024-04-07T16:18:46.171 INFO:tasks.rados.rados.0.smithi100.stdout:714: oid 1 version 74 is already newer than 72 2024-04-07T16:18:46.171 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 1 v 74 (ObjNum 231 snap 60 seq_num 231) dirty exists 2024-04-07T16:18:46.171 INFO:tasks.rados.rados.0.smithi100.stdout:714: left oid 1 (ObjNum 231 snap 60 seq_num 231) 2024-04-07T16:18:46.171 INFO:tasks.rados.rados.0.smithi100.stdout:716: finishing write tid 1 to smithi10066867-34 2024-04-07T16:18:46.174 INFO:tasks.rados.rados.0.smithi100.stdout:716: finishing write tid 2 to smithi10066867-34 2024-04-07T16:18:46.174 INFO:tasks.rados.rados.0.smithi100.stdout:716: finishing write tid 3 to smithi10066867-34 2024-04-07T16:18:46.176 INFO:tasks.rados.rados.0.smithi100.stdout:716: finishing write tid 4 to smithi10066867-34 2024-04-07T16:18:46.178 INFO:tasks.rados.rados.0.smithi100.stdout:716: finishing write tid 5 to smithi10066867-34 2024-04-07T16:18:46.178 INFO:tasks.rados.rados.0.smithi100.stdout:716: oid 34 updating version 0 to 252 2024-04-07T16:18:46.178 INFO:tasks.rados.rados.0.smithi100.stdout:716: oid 34 updating version 252 to 253 2024-04-07T16:18:46.178 INFO:tasks.rados.rados.0.smithi100.stdout:716: oid 34 version 253 is already newer than 251 2024-04-07T16:18:46.178 INFO:tasks.rados.rados.0.smithi100.stdout:716: oid 34 version 253 is already newer than 250 2024-04-07T16:18:46.179 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 34 v 253 (ObjNum 233 snap 60 seq_num 233) dirty exists 2024-04-07T16:18:46.179 INFO:tasks.rados.rados.0.smithi100.stdout:716: left oid 34 (ObjNum 233 snap 60 seq_num 233) 2024-04-07T16:18:46.179 INFO:tasks.rados.rados.0.smithi100.stdout:714: done (2 left) 2024-04-07T16:18:46.179 INFO:tasks.rados.rados.0.smithi100.stdout:716: done (1 left) 2024-04-07T16:18:46.179 INFO:tasks.rados.rados.0.smithi100.stdout:717: done (0 left) 2024-04-07T16:18:46.179 INFO:tasks.rados.rados.0.smithi100.stdout:718: snap_create 2024-04-07T16:18:46.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:46 smithi144 bash[23522]: cluster 2024-04-07T16:18:45.110929+0000 mon.a (mon.0) 1062 : cluster 0 osdmap e191: 8 total, 8 up, 8 in 2024-04-07T16:18:46.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:46 smithi144 bash[23522]: cephadm 2024-04-07T16:18:45.256975+0000 mgr.x (mgr.24553) 24 : cephadm 1 Updating smithi144:/var/lib/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/config/ceph.conf 2024-04-07T16:18:46.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:46 smithi144 bash[23522]: cephadm 2024-04-07T16:18:45.258284+0000 mgr.x (mgr.24553) 25 : cephadm 1 Updating smithi100:/var/lib/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/config/ceph.conf 2024-04-07T16:18:46.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:46 smithi144 bash[23522]: cephadm 2024-04-07T16:18:45.421523+0000 mgr.x (mgr.24553) 26 : cephadm 1 Updating smithi144:/etc/ceph/ceph.client.admin.keyring 2024-04-07T16:18:46.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:46 smithi144 bash[23522]: cephadm 2024-04-07T16:18:45.423567+0000 mgr.x (mgr.24553) 27 : cephadm 1 Updating smithi100:/etc/ceph/ceph.client.admin.keyring 2024-04-07T16:18:46.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:46 smithi144 bash[23522]: cephadm 2024-04-07T16:18:45.585816+0000 mgr.x (mgr.24553) 28 : cephadm 1 Updating smithi144:/var/lib/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/config/ceph.client.admin.keyring 2024-04-07T16:18:46.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:46 smithi144 bash[23522]: cephadm 2024-04-07T16:18:45.588648+0000 mgr.x (mgr.24553) 29 : cephadm 1 Updating smithi100:/var/lib/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/config/ceph.client.admin.keyring 2024-04-07T16:18:46.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:46 smithi144 bash[23522]: audit 2024-04-07T16:18:45.762650+0000 mon.a (mon.0) 1063 : audit 1 from='mgr.24553 ' entity='mgr.x' 2024-04-07T16:18:46.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:46 smithi144 bash[23522]: audit 2024-04-07T16:18:45.769396+0000 mon.a (mon.0) 1064 : audit 1 from='mgr.24553 ' entity='mgr.x' 2024-04-07T16:18:46.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:46 smithi144 bash[23522]: audit 2024-04-07T16:18:45.781779+0000 mon.a (mon.0) 1065 : audit 1 from='mgr.24553 ' entity='mgr.x' 2024-04-07T16:18:46.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:46 smithi144 bash[23522]: audit 2024-04-07T16:18:45.790620+0000 mon.a (mon.0) 1066 : audit 1 from='mgr.24553 ' entity='mgr.x' 2024-04-07T16:18:46.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:46 smithi144 bash[23522]: audit 2024-04-07T16:18:45.803834+0000 mon.a (mon.0) 1067 : audit 1 from='mgr.24553 ' entity='mgr.x' 2024-04-07T16:18:46.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:46 smithi144 bash[23522]: cephadm 2024-04-07T16:18:45.876983+0000 mgr.x (mgr.24553) 30 : cephadm 1 Reconfiguring prometheus.a (dependencies changed)... 2024-04-07T16:18:46.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:46 smithi100 bash[21346]: cluster 2024-04-07T16:18:45.110929+0000 mon.a (mon.0) 1062 : cluster 0 osdmap e191: 8 total, 8 up, 8 in 2024-04-07T16:18:46.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:46 smithi100 bash[21346]: cephadm 2024-04-07T16:18:45.256975+0000 mgr.x (mgr.24553) 24 : cephadm 1 Updating smithi144:/var/lib/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/config/ceph.conf 2024-04-07T16:18:46.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:46 smithi100 bash[21346]: cephadm 2024-04-07T16:18:45.258284+0000 mgr.x (mgr.24553) 25 : cephadm 1 Updating smithi100:/var/lib/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/config/ceph.conf 2024-04-07T16:18:46.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:46 smithi100 bash[21346]: cephadm 2024-04-07T16:18:45.421523+0000 mgr.x (mgr.24553) 26 : cephadm 1 Updating smithi144:/etc/ceph/ceph.client.admin.keyring 2024-04-07T16:18:46.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:46 smithi100 bash[21346]: cephadm 2024-04-07T16:18:45.423567+0000 mgr.x (mgr.24553) 27 : cephadm 1 Updating smithi100:/etc/ceph/ceph.client.admin.keyring 2024-04-07T16:18:46.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:46 smithi100 bash[21346]: cephadm 2024-04-07T16:18:45.585816+0000 mgr.x (mgr.24553) 28 : cephadm 1 Updating smithi144:/var/lib/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/config/ceph.client.admin.keyring 2024-04-07T16:18:46.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:46 smithi100 bash[21346]: cephadm 2024-04-07T16:18:45.588648+0000 mgr.x (mgr.24553) 29 : cephadm 1 Updating smithi100:/var/lib/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/config/ceph.client.admin.keyring 2024-04-07T16:18:46.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:46 smithi100 bash[21346]: audit 2024-04-07T16:18:45.762650+0000 mon.a (mon.0) 1063 : audit 1 from='mgr.24553 ' entity='mgr.x' 2024-04-07T16:18:46.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:46 smithi100 bash[21346]: audit 2024-04-07T16:18:45.769396+0000 mon.a (mon.0) 1064 : audit 1 from='mgr.24553 ' entity='mgr.x' 2024-04-07T16:18:46.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:46 smithi100 bash[21346]: audit 2024-04-07T16:18:45.781779+0000 mon.a (mon.0) 1065 : audit 1 from='mgr.24553 ' entity='mgr.x' 2024-04-07T16:18:46.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:46 smithi100 bash[21346]: audit 2024-04-07T16:18:45.790620+0000 mon.a (mon.0) 1066 : audit 1 from='mgr.24553 ' entity='mgr.x' 2024-04-07T16:18:46.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:46 smithi100 bash[21346]: audit 2024-04-07T16:18:45.803834+0000 mon.a (mon.0) 1067 : audit 1 from='mgr.24553 ' entity='mgr.x' 2024-04-07T16:18:46.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:46 smithi100 bash[21346]: cephadm 2024-04-07T16:18:45.876983+0000 mgr.x (mgr.24553) 30 : cephadm 1 Reconfiguring prometheus.a (dependencies changed)... 2024-04-07T16:18:46.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:46 smithi100 bash[31660]: cluster 2024-04-07T16:18:45.110929+0000 mon.a (mon.0) 1062 : cluster 0 osdmap e191: 8 total, 8 up, 8 in 2024-04-07T16:18:46.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:46 smithi100 bash[31660]: cephadm 2024-04-07T16:18:45.256975+0000 mgr.x (mgr.24553) 24 : cephadm 1 Updating smithi144:/var/lib/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/config/ceph.conf 2024-04-07T16:18:46.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:46 smithi100 bash[31660]: cephadm 2024-04-07T16:18:45.258284+0000 mgr.x (mgr.24553) 25 : cephadm 1 Updating smithi100:/var/lib/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/config/ceph.conf 2024-04-07T16:18:46.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:46 smithi100 bash[31660]: cephadm 2024-04-07T16:18:45.421523+0000 mgr.x (mgr.24553) 26 : cephadm 1 Updating smithi144:/etc/ceph/ceph.client.admin.keyring 2024-04-07T16:18:46.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:46 smithi100 bash[31660]: cephadm 2024-04-07T16:18:45.423567+0000 mgr.x (mgr.24553) 27 : cephadm 1 Updating smithi100:/etc/ceph/ceph.client.admin.keyring 2024-04-07T16:18:46.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:46 smithi100 bash[31660]: cephadm 2024-04-07T16:18:45.585816+0000 mgr.x (mgr.24553) 28 : cephadm 1 Updating smithi144:/var/lib/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/config/ceph.client.admin.keyring 2024-04-07T16:18:46.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:46 smithi100 bash[31660]: cephadm 2024-04-07T16:18:45.588648+0000 mgr.x (mgr.24553) 29 : cephadm 1 Updating smithi100:/var/lib/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/config/ceph.client.admin.keyring 2024-04-07T16:18:46.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:46 smithi100 bash[31660]: audit 2024-04-07T16:18:45.762650+0000 mon.a (mon.0) 1063 : audit 1 from='mgr.24553 ' entity='mgr.x' 2024-04-07T16:18:46.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:46 smithi100 bash[31660]: audit 2024-04-07T16:18:45.769396+0000 mon.a (mon.0) 1064 : audit 1 from='mgr.24553 ' entity='mgr.x' 2024-04-07T16:18:46.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:46 smithi100 bash[31660]: audit 2024-04-07T16:18:45.781779+0000 mon.a (mon.0) 1065 : audit 1 from='mgr.24553 ' entity='mgr.x' 2024-04-07T16:18:46.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:46 smithi100 bash[31660]: audit 2024-04-07T16:18:45.790620+0000 mon.a (mon.0) 1066 : audit 1 from='mgr.24553 ' entity='mgr.x' 2024-04-07T16:18:46.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:46 smithi100 bash[31660]: audit 2024-04-07T16:18:45.803834+0000 mon.a (mon.0) 1067 : audit 1 from='mgr.24553 ' entity='mgr.x' 2024-04-07T16:18:46.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:46 smithi100 bash[31660]: cephadm 2024-04-07T16:18:45.876983+0000 mgr.x (mgr.24553) 30 : cephadm 1 Reconfiguring prometheus.a (dependencies changed)... 2024-04-07T16:18:47.114 INFO:tasks.rados.rados.0.smithi100.stdout:718: done (0 left) 2024-04-07T16:18:47.114 INFO:tasks.rados.rados.0.smithi100.stdout:719: rmattr oid 13 current snap is 61 2024-04-07T16:18:47.114 INFO:tasks.rados.rados.0.smithi100.stdout:720: snap_create 2024-04-07T16:18:47.128 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 13 v 255 (ObjNum 7 snap 0 seq_num 7) dirty exists 2024-04-07T16:18:47.354 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:47 smithi144 bash[23522]: cluster 2024-04-07T16:18:46.118867+0000 mon.a (mon.0) 1068 : cluster 0 osdmap e192: 8 total, 8 up, 8 in 2024-04-07T16:18:47.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:47 smithi144 bash[23522]: cephadm 2024-04-07T16:18:46.136951+0000 mgr.x (mgr.24553) 31 : cephadm 1 Reconfiguring daemon prometheus.a on smithi144 2024-04-07T16:18:47.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:47 smithi144 bash[23522]: cluster 2024-04-07T16:18:46.648857+0000 mgr.x (mgr.24553) 32 : cluster 0 pgmap v42: 121 pgs: 121 active+clean; 151 MiB data, 712 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:18:47.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:47 smithi100 bash[21346]: cluster 2024-04-07T16:18:46.118867+0000 mon.a (mon.0) 1068 : cluster 0 osdmap e192: 8 total, 8 up, 8 in 2024-04-07T16:18:47.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:47 smithi100 bash[21346]: cephadm 2024-04-07T16:18:46.136951+0000 mgr.x (mgr.24553) 31 : cephadm 1 Reconfiguring daemon prometheus.a on smithi144 2024-04-07T16:18:47.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:47 smithi100 bash[21346]: cluster 2024-04-07T16:18:46.648857+0000 mgr.x (mgr.24553) 32 : cluster 0 pgmap v42: 121 pgs: 121 active+clean; 151 MiB data, 712 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:18:47.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:47 smithi100 bash[31660]: cluster 2024-04-07T16:18:46.118867+0000 mon.a (mon.0) 1068 : cluster 0 osdmap e192: 8 total, 8 up, 8 in 2024-04-07T16:18:47.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:47 smithi100 bash[31660]: cephadm 2024-04-07T16:18:46.136951+0000 mgr.x (mgr.24553) 31 : cephadm 1 Reconfiguring daemon prometheus.a on smithi144 2024-04-07T16:18:47.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:47 smithi100 bash[31660]: cluster 2024-04-07T16:18:46.648857+0000 mgr.x (mgr.24553) 32 : cluster 0 pgmap v42: 121 pgs: 121 active+clean; 151 MiB data, 712 MiB used, 715 GiB / 715 GiB avail 2024-04-07T16:18:47.858 INFO:tasks.rados.rados.0.smithi100.stdout:719: done (1 left) 2024-04-07T16:18:47.858 INFO:tasks.rados.rados.0.smithi100.stdout:720: done (0 left) 2024-04-07T16:18:47.859 INFO:tasks.rados.rados.0.smithi100.stdout:721: copy_from oid 43 from oid 39 current snap is 62 2024-04-07T16:18:47.859 INFO:tasks.rados.rados.0.smithi100.stdout:722: delete oid 49 current snap is 62 2024-04-07T16:18:47.866 INFO:tasks.rados.rados.0.smithi100.stdout:722: done (1 left) 2024-04-07T16:18:47.866 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 23 current snap is 62 2024-04-07T16:18:47.867 INFO:tasks.rados.rados.0.smithi100.stdout:723: seq_num 236 ranges {1622016=761856,2383872=647168,3031040=155648} 2024-04-07T16:18:47.901 INFO:tasks.rados.rados.0.smithi100.stdout:723: writing smithi10066867-23 from 1622016 to 2383872 tid 1 2024-04-07T16:18:47.914 INFO:tasks.rados.rados.0.smithi100.stdout:723: writing smithi10066867-23 from 2383872 to 3031040 tid 2 2024-04-07T16:18:47.914 INFO:tasks.rados.rados.0.smithi100.stdout:723: writing smithi10066867-23 from 3031040 to 3186688 tid 3 2024-04-07T16:18:47.914 INFO:tasks.rados.rados.0.smithi100.stdout:724: rollback oid 3 current snap is 62 2024-04-07T16:18:47.915 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 3 to 53 2024-04-07T16:18:47.915 INFO:tasks.rados.rados.0.smithi100.stdout:721: finishing copy_from racing read to smithi10066867-43 2024-04-07T16:18:47.915 INFO:tasks.rados.rados.0.smithi100.stdout:721: finishing copy_from to smithi10066867-43 2024-04-07T16:18:47.915 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 43 v 231 (ObjNum 212 snap 57 seq_num 212) dirty exists 2024-04-07T16:18:47.915 INFO:tasks.rados.rados.0.smithi100.stdout:721: done (2 left) 2024-04-07T16:18:47.915 INFO:tasks.rados.rados.0.smithi100.stdout:725: snap_create 2024-04-07T16:18:47.920 INFO:tasks.rados.rados.0.smithi100.stdout:723: finishing write tid 1 to smithi10066867-23 2024-04-07T16:18:47.922 INFO:tasks.rados.rados.0.smithi100.stdout:723: finishing write tid 2 to smithi10066867-23 2024-04-07T16:18:47.923 INFO:tasks.rados.rados.0.smithi100.stdout:723: finishing write tid 3 to smithi10066867-23 2024-04-07T16:18:47.923 INFO:tasks.rados.rados.0.smithi100.stdout:723: finishing write tid 4 to smithi10066867-23 2024-04-07T16:18:47.925 INFO:tasks.rados.rados.0.smithi100.stdout:723: finishing write tid 5 to smithi10066867-23 2024-04-07T16:18:47.925 INFO:tasks.rados.rados.0.smithi100.stdout:723: oid 23 updating version 0 to 444 2024-04-07T16:18:47.925 INFO:tasks.rados.rados.0.smithi100.stdout:723: oid 23 version 444 is already newer than 443 2024-04-07T16:18:47.925 INFO:tasks.rados.rados.0.smithi100.stdout:723: oid 23 updating version 444 to 445 2024-04-07T16:18:47.925 INFO:tasks.rados.rados.0.smithi100.stdout:723: oid 23 updating version 445 to 446 2024-04-07T16:18:47.925 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 23 v 446 (ObjNum 236 snap 62 seq_num 236) dirty exists 2024-04-07T16:18:47.925 INFO:tasks.rados.rados.0.smithi100.stdout:723: left oid 23 (ObjNum 236 snap 62 seq_num 236) 2024-04-07T16:18:47.926 INFO:tasks.rados.rados.0.smithi100.stdout:724: finishing rollback tid 1 to smithi10066867-3 2024-04-07T16:18:47.926 INFO:tasks.rados.rados.0.smithi100.stdout:724: finishing rollback tid 2 to smithi10066867-3 2024-04-07T16:18:47.926 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 3 v 171 (ObjNum 191 snap 50 seq_num 191) dirty exists 2024-04-07T16:18:48.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:48 smithi100 bash[21346]: cluster 2024-04-07T16:18:47.129119+0000 mon.a (mon.0) 1069 : cluster 0 osdmap e193: 8 total, 8 up, 8 in 2024-04-07T16:18:48.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:48 smithi100 bash[21346]: cluster 2024-04-07T16:18:47.872323+0000 mon.a (mon.0) 1070 : cluster 0 osdmap e194: 8 total, 8 up, 8 in 2024-04-07T16:18:48.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:48 smithi100 bash[31660]: cluster 2024-04-07T16:18:47.129119+0000 mon.a (mon.0) 1069 : cluster 0 osdmap e193: 8 total, 8 up, 8 in 2024-04-07T16:18:48.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:48 smithi100 bash[31660]: cluster 2024-04-07T16:18:47.872323+0000 mon.a (mon.0) 1070 : cluster 0 osdmap e194: 8 total, 8 up, 8 in 2024-04-07T16:18:48.542 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:48 smithi144 bash[23522]: cluster 2024-04-07T16:18:47.129119+0000 mon.a (mon.0) 1069 : cluster 0 osdmap e193: 8 total, 8 up, 8 in 2024-04-07T16:18:48.543 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:48 smithi144 bash[23522]: cluster 2024-04-07T16:18:47.872323+0000 mon.a (mon.0) 1070 : cluster 0 osdmap e194: 8 total, 8 up, 8 in 2024-04-07T16:18:48.860 INFO:tasks.rados.rados.0.smithi100.stdout:723: done (2 left) 2024-04-07T16:18:48.860 INFO:tasks.rados.rados.0.smithi100.stdout:724: done (1 left) 2024-04-07T16:18:48.860 INFO:tasks.rados.rados.0.smithi100.stdout:725: done (0 left) 2024-04-07T16:18:48.860 INFO:tasks.rados.rados.0.smithi100.stdout:726: read oid 42 snap -1 2024-04-07T16:18:48.860 INFO:tasks.rados.rados.0.smithi100.stdout:726: expect (ObjNum 190 snap 49 seq_num 190) 2024-04-07T16:18:48.860 INFO:tasks.rados.rados.0.smithi100.stdout:727: read oid 24 snap -1 2024-04-07T16:18:48.860 INFO:tasks.rados.rados.0.smithi100.stdout:727: expect (ObjNum 157 snap 43 seq_num 157) 2024-04-07T16:18:48.861 INFO:tasks.rados.rados.0.smithi100.stdout:728: snap_create 2024-04-07T16:18:48.880 INFO:tasks.rados.rados.0.smithi100.stdout:727: expect (ObjNum 157 snap 43 seq_num 157) 2024-04-07T16:18:49.027 INFO:tasks.rados.rados.0.smithi100.stdout:726: expect (ObjNum 190 snap 49 seq_num 190) 2024-04-07T16:18:49.864 INFO:tasks.rados.rados.0.smithi100.stdout:726: done (2 left) 2024-04-07T16:18:49.864 INFO:tasks.rados.rados.0.smithi100.stdout:727: done (1 left) 2024-04-07T16:18:49.864 INFO:tasks.rados.rados.0.smithi100.stdout:728: done (0 left) 2024-04-07T16:18:49.864 INFO:tasks.rados.rados.0.smithi100.stdout:729: delete oid 10 current snap is 64 2024-04-07T16:18:49.869 INFO:tasks.rados.rados.0.smithi100.stdout:729: done (0 left) 2024-04-07T16:18:49.869 INFO:tasks.rados.rados.0.smithi100.stdout:730: snap_create 2024-04-07T16:18:49.958 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:49 smithi144 bash[23522]: cluster 2024-04-07T16:18:48.649553+0000 mgr.x (mgr.24553) 33 : cluster 0 pgmap v45: 121 pgs: 121 active+clean; 130 MiB data, 717 MiB used, 715 GiB / 715 GiB avail; 11 MiB/s rd, 2.0 MiB/s wr, 26 op/s 2024-04-07T16:18:49.958 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:49 smithi144 bash[23522]: cluster 2024-04-07T16:18:48.874808+0000 mon.a (mon.0) 1071 : cluster 0 osdmap e195: 8 total, 8 up, 8 in 2024-04-07T16:18:49.958 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:18:49 smithi144 systemd[1]: Stopping Ceph prometheus.a for 0702e3a8-f4f8-11ee-b649-cb9ed24678a4... 2024-04-07T16:18:49.959 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:18:49 smithi144 bash[51076]: ts=2024-04-07T16:18:49.779Z caller=main.go:840 level=warn msg="Received SIGTERM, exiting gracefully..." 2024-04-07T16:18:49.959 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:18:49 smithi144 bash[51076]: ts=2024-04-07T16:18:49.779Z caller=main.go:864 level=info msg="Stopping scrape discovery manager..." 2024-04-07T16:18:49.959 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:18:49 smithi144 bash[51076]: ts=2024-04-07T16:18:49.779Z caller=main.go:878 level=info msg="Stopping notify discovery manager..." 2024-04-07T16:18:49.959 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:18:49 smithi144 bash[51076]: ts=2024-04-07T16:18:49.779Z caller=manager.go:988 level=info component="rule manager" msg="Stopping rule manager..." 2024-04-07T16:18:49.959 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:18:49 smithi144 bash[51076]: ts=2024-04-07T16:18:49.779Z caller=main.go:860 level=info msg="Scrape discovery manager stopped" 2024-04-07T16:18:49.959 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:18:49 smithi144 bash[51076]: ts=2024-04-07T16:18:49.779Z caller=main.go:874 level=info msg="Notify discovery manager stopped" 2024-04-07T16:18:49.959 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:18:49 smithi144 bash[51076]: ts=2024-04-07T16:18:49.779Z caller=manager.go:998 level=info component="rule manager" msg="Rule manager stopped" 2024-04-07T16:18:49.959 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:18:49 smithi144 bash[51076]: ts=2024-04-07T16:18:49.779Z caller=main.go:915 level=info msg="Stopping scrape manager..." 2024-04-07T16:18:49.959 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:18:49 smithi144 bash[51076]: ts=2024-04-07T16:18:49.779Z caller=main.go:907 level=info msg="Scrape manager stopped" 2024-04-07T16:18:49.959 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:18:49 smithi144 bash[51076]: ts=2024-04-07T16:18:49.780Z caller=notifier.go:606 level=info component=notifier msg="Stopping notification manager..." 2024-04-07T16:18:49.959 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:18:49 smithi144 bash[51076]: ts=2024-04-07T16:18:49.780Z caller=main.go:1135 level=info msg="Notifier manager stopped" 2024-04-07T16:18:49.959 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:18:49 smithi144 bash[51076]: ts=2024-04-07T16:18:49.780Z caller=main.go:1147 level=info msg="See you next time!" 2024-04-07T16:18:50.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:49 smithi100 bash[21346]: cluster 2024-04-07T16:18:48.649553+0000 mgr.x (mgr.24553) 33 : cluster 0 pgmap v45: 121 pgs: 121 active+clean; 130 MiB data, 717 MiB used, 715 GiB / 715 GiB avail; 11 MiB/s rd, 2.0 MiB/s wr, 26 op/s 2024-04-07T16:18:50.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:49 smithi100 bash[21346]: cluster 2024-04-07T16:18:48.874808+0000 mon.a (mon.0) 1071 : cluster 0 osdmap e195: 8 total, 8 up, 8 in 2024-04-07T16:18:50.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:49 smithi100 bash[31660]: cluster 2024-04-07T16:18:48.649553+0000 mgr.x (mgr.24553) 33 : cluster 0 pgmap v45: 121 pgs: 121 active+clean; 130 MiB data, 717 MiB used, 715 GiB / 715 GiB avail; 11 MiB/s rd, 2.0 MiB/s wr, 26 op/s 2024-04-07T16:18:50.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:49 smithi100 bash[31660]: cluster 2024-04-07T16:18:48.874808+0000 mon.a (mon.0) 1071 : cluster 0 osdmap e195: 8 total, 8 up, 8 in 2024-04-07T16:18:50.351 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:18:50 smithi144 bash[58376]: ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4-prometheus-a 2024-04-07T16:18:50.693 INFO:teuthology.orchestra.run.smithi100.stdout:true 2024-04-07T16:18:50.854 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:18:50 smithi144 bash[58438]: Error response from daemon: No such container: ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4-prometheus-a 2024-04-07T16:18:50.855 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:18:50 smithi144 systemd[1]: ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@prometheus.a.service: Deactivated successfully. 2024-04-07T16:18:50.855 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:18:50 smithi144 systemd[1]: Stopped Ceph prometheus.a for 0702e3a8-f4f8-11ee-b649-cb9ed24678a4. 2024-04-07T16:18:50.855 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:18:50 smithi144 systemd[1]: Started Ceph prometheus.a for 0702e3a8-f4f8-11ee-b649-cb9ed24678a4. 2024-04-07T16:18:50.883 INFO:tasks.rados.rados.0.smithi100.stdout:730: done (0 left) 2024-04-07T16:18:50.883 INFO:tasks.rados.rados.0.smithi100.stdout:731: read oid 17 snap -1 2024-04-07T16:18:50.883 INFO:tasks.rados.rados.0.smithi100.stdout:731: expect (ObjNum 225 snap 59 seq_num 225) 2024-04-07T16:18:50.883 INFO:tasks.rados.rados.0.smithi100.stdout:732: rollback oid 6 current snap is 65 2024-04-07T16:18:50.883 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 6 to 48 2024-04-07T16:18:50.883 INFO:tasks.rados.rados.0.smithi100.stdout:733: read oid 27 snap 48 2024-04-07T16:18:50.883 INFO:tasks.rados.rados.0.smithi100.stdout:733: expect (ObjNum -1592247472 snap 21978 seq_num 2702447216) 2024-04-07T16:18:50.883 INFO:tasks.rados.rados.0.smithi100.stdout:734: copy_from oid 11 from oid 8 current snap is 65 2024-04-07T16:18:50.883 INFO:tasks.rados.rados.0.smithi100.stdout:735: copy_from oid 13 from oid 12 current snap is 65 2024-04-07T16:18:50.883 INFO:tasks.rados.rados.0.smithi100.stdout:736: setattr oid 45 current snap is 65 2024-04-07T16:18:50.885 INFO:tasks.rados.rados.0.smithi100.stdout:737: rollback oid 29 current snap is 65 2024-04-07T16:18:50.885 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 29 to 60 2024-04-07T16:18:50.885 INFO:tasks.rados.rados.0.smithi100.stdout:738: copy_from oid 24 from oid 21 current snap is 65 2024-04-07T16:18:50.885 INFO:tasks.rados.rados.0.smithi100.stdout:739: read oid 32 snap -1 2024-04-07T16:18:50.885 INFO:tasks.rados.rados.0.smithi100.stdout:739: expect (ObjNum 201 snap 53 seq_num 201) 2024-04-07T16:18:50.885 INFO:tasks.rados.rados.0.smithi100.stdout:740: copy_from oid 1 from oid 36 current snap is 65 2024-04-07T16:18:50.885 INFO:tasks.rados.rados.0.smithi100.stdout:741: delete oid 5 current snap is 65 2024-04-07T16:18:50.885 INFO:tasks.rados.rados.0.smithi100.stdout:733: expect (ObjNum -1592247472 snap 21978 seq_num 2702447216) 2024-04-07T16:18:50.896 INFO:tasks.rados.rados.0.smithi100.stdout:731: expect (ObjNum 225 snap 59 seq_num 225) 2024-04-07T16:18:50.936 INFO:tasks.rados.rados.0.smithi100.stdout:732: finishing rollback tid 0 to smithi10066867-6 2024-04-07T16:18:50.936 INFO:tasks.rados.rados.0.smithi100.stdout:732: finishing rollback tid 1 to smithi10066867-6 2024-04-07T16:18:50.936 INFO:tasks.rados.rados.0.smithi100.stdout:732: finishing rollback tid 2 to smithi10066867-6 2024-04-07T16:18:50.936 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 6 v 312 (ObjNum 162 snap 43 seq_num 162) dirty exists 2024-04-07T16:18:50.936 INFO:tasks.rados.rados.0.smithi100.stdout:739: expect (ObjNum 201 snap 53 seq_num 201) 2024-04-07T16:18:50.963 INFO:tasks.rados.rados.0.smithi100.stdout:740: finishing copy_from racing read to smithi10066867-1 2024-04-07T16:18:50.963 INFO:tasks.rados.rados.0.smithi100.stdout:734: finishing copy_from racing read to smithi10066867-11 2024-04-07T16:18:50.963 INFO:tasks.rados.rados.0.smithi100.stdout:731: done (10 left) 2024-04-07T16:18:50.963 INFO:tasks.rados.rados.0.smithi100.stdout:732: done (9 left) 2024-04-07T16:18:50.963 INFO:tasks.rados.rados.0.smithi100.stdout:733: done (8 left) 2024-04-07T16:18:50.963 INFO:tasks.rados.rados.0.smithi100.stdout:739: done (7 left) 2024-04-07T16:18:50.963 INFO:tasks.rados.rados.0.smithi100.stdout:741: done (6 left) 2024-04-07T16:18:50.963 INFO:tasks.rados.rados.0.smithi100.stdout:742: read oid 9 snap -1 2024-04-07T16:18:50.964 INFO:tasks.rados.rados.0.smithi100.stdout:742: expect (ObjNum 206 snap 56 seq_num 206) 2024-04-07T16:18:50.964 INFO:tasks.rados.rados.0.smithi100.stdout:743: read oid 42 snap 58 2024-04-07T16:18:50.964 INFO:tasks.rados.rados.0.smithi100.stdout:743: expect (ObjNum 190 snap 49 seq_num 190) 2024-04-07T16:18:50.964 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 7 current snap is 65 2024-04-07T16:18:50.964 INFO:tasks.rados.rados.0.smithi100.stdout:740: finishing copy_from to smithi10066867-1 2024-04-07T16:18:50.964 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 1 v 213 (ObjNum 210 snap 57 seq_num 210) dirty exists 2024-04-07T16:18:50.964 INFO:tasks.rados.rados.0.smithi100.stdout:734: finishing copy_from to smithi10066867-11 2024-04-07T16:18:50.964 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 11 v 183 (ObjNum 208 snap 56 seq_num 208) dirty exists 2024-04-07T16:18:50.964 INFO:tasks.rados.rados.0.smithi100.stdout:738: finishing copy_from racing read to smithi10066867-24 2024-04-07T16:18:50.964 INFO:tasks.rados.rados.0.smithi100.stdout:738: finishing copy_from to smithi10066867-24 2024-04-07T16:18:50.964 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 24 v 99 (ObjNum 222 snap 59 seq_num 222) dirty exists 2024-04-07T16:18:50.964 INFO:tasks.rados.rados.0.smithi100.stdout:737: finishing rollback tid 1 to smithi10066867-29 2024-04-07T16:18:50.965 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 29 v 259 (ObjNum 289 snap 0 seq_num 0) dirty dne 2024-04-07T16:18:50.965 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 45 v 194 (ObjNum 7 snap 0 seq_num 7) dirty exists 2024-04-07T16:18:50.965 INFO:tasks.rados.rados.0.smithi100.stdout:735: finishing copy_from racing read to smithi10066867-13 2024-04-07T16:18:50.965 INFO:tasks.rados.rados.0.smithi100.stdout:735: finishing copy_from to smithi10066867-13 2024-04-07T16:18:50.965 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 13 v 235 (ObjNum 195 snap 51 seq_num 195) dirty exists 2024-04-07T16:18:50.965 INFO:tasks.rados.rados.0.smithi100.stdout:744: seq_num 239 ranges {270336=409600,679936=704512,1384448=499712} 2024-04-07T16:18:50.968 INFO:tasks.rados.rados.0.smithi100.stdout:744: writing smithi10066867-7 from 270336 to 679936 tid 1 2024-04-07T16:18:50.974 INFO:tasks.rados.rados.0.smithi100.stdout:744: writing smithi10066867-7 from 679936 to 1384448 tid 2 2024-04-07T16:18:50.979 INFO:tasks.rados.rados.0.smithi100.stdout:744: writing smithi10066867-7 from 1384448 to 1884160 tid 3 2024-04-07T16:18:50.979 INFO:tasks.rados.rados.0.smithi100.stdout:734: done (8 left) 2024-04-07T16:18:50.980 INFO:tasks.rados.rados.0.smithi100.stdout:735: done (7 left) 2024-04-07T16:18:50.980 INFO:tasks.rados.rados.0.smithi100.stdout:736: done (6 left) 2024-04-07T16:18:50.980 INFO:tasks.rados.rados.0.smithi100.stdout:737: done (5 left) 2024-04-07T16:18:50.980 INFO:tasks.rados.rados.0.smithi100.stdout:738: done (4 left) 2024-04-07T16:18:50.980 INFO:tasks.rados.rados.0.smithi100.stdout:740: done (3 left) 2024-04-07T16:18:50.980 INFO:tasks.rados.rados.0.smithi100.stdout:745: rmattr oid 29 current snap is 65 2024-04-07T16:18:50.980 INFO:tasks.rados.rados.0.smithi100.stdout:745: done (3 left) 2024-04-07T16:18:50.980 INFO:tasks.rados.rados.0.smithi100.stdout:746: snap_remove snap 61 2024-04-07T16:18:51.089 INFO:teuthology.orchestra.run.smithi100.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-07T16:18:51.089 INFO:teuthology.orchestra.run.smithi100.stdout:alertmanager.a smithi100 *:9093,9094 running (4m) 16s ago 5m 14.7M - 0.25.0 c8568f914cd2 5143075e7d26 2024-04-07T16:18:51.089 INFO:teuthology.orchestra.run.smithi100.stdout:grafana.a smithi144 *:3000 running (5m) 16s ago 5m 78.8M - 9.4.7 954c08fa6188 7180c0708acd 2024-04-07T16:18:51.089 INFO:teuthology.orchestra.run.smithi100.stdout:mds.a smithi100 running (6m) 16s ago 6m 13.9M - 18.2.2-1171-g267c8ff5 8a4a01e7186c 92593b0c28ae 2024-04-07T16:18:51.089 INFO:teuthology.orchestra.run.smithi100.stdout:mds.b smithi144 running (6m) 16s ago 6m 16.6M - 18.2.2-1171-g267c8ff5 8a4a01e7186c 3df2ee3e17a4 2024-04-07T16:18:51.089 INFO:teuthology.orchestra.run.smithi100.stdout:mgr.x smithi144 *:8443,9283,8765 running (46s) 16s ago 13m 483M - 19.0.0-2785-gadce92d0 c0ecd4226ac9 45451b348d34 2024-04-07T16:18:51.089 INFO:teuthology.orchestra.run.smithi100.stdout:mgr.y smithi100 *:9283,8765 running (16m) 16s ago 16m 433M - 18.2.2-1171-g267c8ff5 8a4a01e7186c da1d683b9737 2024-04-07T16:18:51.089 INFO:teuthology.orchestra.run.smithi100.stdout:mon.a smithi100 running (16m) 16s ago 16m 55.8M 2048M 18.2.2-1171-g267c8ff5 8a4a01e7186c ec8833f59ea0 2024-04-07T16:18:51.090 INFO:teuthology.orchestra.run.smithi100.stdout:mon.b smithi144 running (14m) 16s ago 14m 50.3M 2048M 18.2.2-1171-g267c8ff5 8a4a01e7186c 8dd687691b2e 2024-04-07T16:18:51.090 INFO:teuthology.orchestra.run.smithi100.stdout:mon.c smithi100 running (14m) 16s ago 14m 46.5M 2048M 18.2.2-1171-g267c8ff5 8a4a01e7186c a6de0151a3ba 2024-04-07T16:18:51.090 INFO:teuthology.orchestra.run.smithi100.stdout:node-exporter.a smithi100 *:9100 running (5m) 16s ago 5m 8576k - 1.5.0 0da6a335fe13 4bce264c1b26 2024-04-07T16:18:51.090 INFO:teuthology.orchestra.run.smithi100.stdout:node-exporter.b smithi144 *:9100 running (5m) 16s ago 5m 8424k - 1.5.0 0da6a335fe13 b6963413c560 2024-04-07T16:18:51.090 INFO:teuthology.orchestra.run.smithi100.stdout:osd.0 smithi100 running (13m) 16s ago 13m 122M 2524M 18.2.2-1171-g267c8ff5 8a4a01e7186c 188deb247979 2024-04-07T16:18:51.090 INFO:teuthology.orchestra.run.smithi100.stdout:osd.1 smithi100 running (12m) 16s ago 12m 104M 2524M 18.2.2-1171-g267c8ff5 8a4a01e7186c 7f57ca90894c 2024-04-07T16:18:51.090 INFO:teuthology.orchestra.run.smithi100.stdout:osd.2 smithi100 running (11m) 16s ago 11m 138M 2524M 18.2.2-1171-g267c8ff5 8a4a01e7186c f3912ddcc8cc 2024-04-07T16:18:51.090 INFO:teuthology.orchestra.run.smithi100.stdout:osd.3 smithi100 running (10m) 16s ago 10m 88.2M 2524M 18.2.2-1171-g267c8ff5 8a4a01e7186c b41004090c63 2024-04-07T16:18:51.090 INFO:teuthology.orchestra.run.smithi100.stdout:osd.4 smithi144 running (9m) 16s ago 9m 114M 2524M 18.2.2-1171-g267c8ff5 8a4a01e7186c 8c73b6b4dc13 2024-04-07T16:18:51.090 INFO:teuthology.orchestra.run.smithi100.stdout:osd.5 smithi144 running (9m) 16s ago 9m 112M 2524M 18.2.2-1171-g267c8ff5 8a4a01e7186c 3bfa0cf6f6cb 2024-04-07T16:18:51.090 INFO:teuthology.orchestra.run.smithi100.stdout:osd.6 smithi144 running (8m) 16s ago 8m 66.8M 2524M 18.2.2-1171-g267c8ff5 8a4a01e7186c 32b7f95d77f0 2024-04-07T16:18:51.090 INFO:teuthology.orchestra.run.smithi100.stdout:osd.7 smithi144 running (7m) 16s ago 7m 87.4M 2524M 18.2.2-1171-g267c8ff5 8a4a01e7186c 45f7709e576a 2024-04-07T16:18:51.090 INFO:teuthology.orchestra.run.smithi100.stdout:prometheus.a smithi144 *:9095 starting - - - - 2024-04-07T16:18:51.167 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:50 smithi144 bash[23522]: cluster 2024-04-07T16:18:49.876977+0000 mon.a (mon.0) 1072 : cluster 0 osdmap e196: 8 total, 8 up, 8 in 2024-04-07T16:18:51.167 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:50 smithi144 bash[23522]: audit 2024-04-07T16:18:50.429867+0000 mon.a (mon.0) 1073 : audit 1 from='mgr.24553 ' entity='mgr.x' 2024-04-07T16:18:51.168 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:50 smithi144 bash[23522]: audit 2024-04-07T16:18:50.442019+0000 mon.a (mon.0) 1074 : audit 1 from='mgr.24553 ' entity='mgr.x' 2024-04-07T16:18:51.168 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:50 smithi144 bash[23522]: audit 2024-04-07T16:18:50.466482+0000 mon.b (mon.1) 313 : audit 0 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-07T16:18:51.168 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:50 smithi144 bash[23522]: audit 2024-04-07T16:18:50.473814+0000 mon.b (mon.1) 314 : audit 0 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:18:51.168 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:50 smithi144 bash[23522]: audit 2024-04-07T16:18:50.766174+0000 mon.b (mon.1) 315 : audit 0 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:18:51.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:50 smithi100 bash[31660]: cluster 2024-04-07T16:18:49.876977+0000 mon.a (mon.0) 1072 : cluster 0 osdmap e196: 8 total, 8 up, 8 in 2024-04-07T16:18:51.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:50 smithi100 bash[31660]: audit 2024-04-07T16:18:50.429867+0000 mon.a (mon.0) 1073 : audit 1 from='mgr.24553 ' entity='mgr.x' 2024-04-07T16:18:51.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:50 smithi100 bash[31660]: audit 2024-04-07T16:18:50.442019+0000 mon.a (mon.0) 1074 : audit 1 from='mgr.24553 ' entity='mgr.x' 2024-04-07T16:18:51.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:50 smithi100 bash[31660]: audit 2024-04-07T16:18:50.466482+0000 mon.b (mon.1) 313 : audit 0 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-07T16:18:51.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:50 smithi100 bash[31660]: audit 2024-04-07T16:18:50.473814+0000 mon.b (mon.1) 314 : audit 0 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:18:51.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:50 smithi100 bash[31660]: audit 2024-04-07T16:18:50.766174+0000 mon.b (mon.1) 315 : audit 0 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:18:51.188 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:50 smithi100 bash[21346]: cluster 2024-04-07T16:18:49.876977+0000 mon.a (mon.0) 1072 : cluster 0 osdmap e196: 8 total, 8 up, 8 in 2024-04-07T16:18:51.188 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:50 smithi100 bash[21346]: audit 2024-04-07T16:18:50.429867+0000 mon.a (mon.0) 1073 : audit 1 from='mgr.24553 ' entity='mgr.x' 2024-04-07T16:18:51.188 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:50 smithi100 bash[21346]: audit 2024-04-07T16:18:50.442019+0000 mon.a (mon.0) 1074 : audit 1 from='mgr.24553 ' entity='mgr.x' 2024-04-07T16:18:51.188 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:50 smithi100 bash[21346]: audit 2024-04-07T16:18:50.466482+0000 mon.b (mon.1) 313 : audit 0 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-07T16:18:51.188 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:50 smithi100 bash[21346]: audit 2024-04-07T16:18:50.473814+0000 mon.b (mon.1) 314 : audit 0 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:18:51.188 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:50 smithi100 bash[21346]: audit 2024-04-07T16:18:50.766174+0000 mon.b (mon.1) 315 : audit 0 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:18:51.519 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-04-07T16:18:51.519 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": { 2024-04-07T16:18:51.519 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph version 18.2.2-1171-g267c8ff5 (267c8ff5b1ff5815581b3b2381d19360e081b37b) reef (stable)": 3 2024-04-07T16:18:51.519 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-07T16:18:51.519 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": { 2024-04-07T16:18:51.519 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph version 18.2.2-1171-g267c8ff5 (267c8ff5b1ff5815581b3b2381d19360e081b37b) reef (stable)": 1, 2024-04-07T16:18:51.519 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph version 19.0.0-2785-gadce92d0 (adce92d056533f22dc1ee37e3141d098b046ec1b) squid (dev)": 1 2024-04-07T16:18:51.519 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-07T16:18:51.519 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": { 2024-04-07T16:18:51.520 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph version 18.2.2-1171-g267c8ff5 (267c8ff5b1ff5815581b3b2381d19360e081b37b) reef (stable)": 8 2024-04-07T16:18:51.520 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-07T16:18:51.520 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": { 2024-04-07T16:18:51.520 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph version 18.2.2-1171-g267c8ff5 (267c8ff5b1ff5815581b3b2381d19360e081b37b) reef (stable)": 2 2024-04-07T16:18:51.520 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-07T16:18:51.520 INFO:teuthology.orchestra.run.smithi100.stdout: "overall": { 2024-04-07T16:18:51.520 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph version 18.2.2-1171-g267c8ff5 (267c8ff5b1ff5815581b3b2381d19360e081b37b) reef (stable)": 14, 2024-04-07T16:18:51.520 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph version 19.0.0-2785-gadce92d0 (adce92d056533f22dc1ee37e3141d098b046ec1b) squid (dev)": 1 2024-04-07T16:18:51.520 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-04-07T16:18:51.520 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-04-07T16:18:51.605 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:18:51 smithi144 bash[58467]: ts=2024-04-07T16:18:51.179Z caller=main.go:564 level=info msg="Starting Prometheus Server" mode=server version="(version=2.43.0, branch=HEAD, revision=edfc3bcd025dd6fe296c167a14a216cab1e552ee)" 2024-04-07T16:18:51.605 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:18:51 smithi144 bash[58467]: ts=2024-04-07T16:18:51.179Z caller=main.go:569 level=info build_context="(go=go1.19.7, platform=linux/amd64, user=root@8a0ee342e522, date=20230321-12:56:07, tags=netgo,builtinassets)" 2024-04-07T16:18:51.605 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:18:51 smithi144 bash[58467]: ts=2024-04-07T16:18:51.179Z caller=main.go:570 level=info host_details="(Linux 5.15.0-101-generic #111-Ubuntu SMP Tue Mar 5 20:16:58 UTC 2024 x86_64 smithi144 (none))" 2024-04-07T16:18:51.605 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:18:51 smithi144 bash[58467]: ts=2024-04-07T16:18:51.179Z caller=main.go:571 level=info fd_limits="(soft=1048576, hard=1048576)" 2024-04-07T16:18:51.606 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:18:51 smithi144 bash[58467]: ts=2024-04-07T16:18:51.179Z caller=main.go:572 level=info vm_limits="(soft=unlimited, hard=unlimited)" 2024-04-07T16:18:51.606 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:18:51 smithi144 bash[58467]: ts=2024-04-07T16:18:51.181Z caller=web.go:561 level=info component=web msg="Start listening for connections" address=:9095 2024-04-07T16:18:51.606 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:18:51 smithi144 bash[58467]: ts=2024-04-07T16:18:51.181Z caller=main.go:1005 level=info msg="Starting TSDB ..." 2024-04-07T16:18:51.606 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:18:51 smithi144 bash[58467]: ts=2024-04-07T16:18:51.183Z caller=tls_config.go:232 level=info component=web msg="Listening on" address=[::]:9095 2024-04-07T16:18:51.606 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:18:51 smithi144 bash[58467]: ts=2024-04-07T16:18:51.183Z caller=tls_config.go:235 level=info component=web msg="TLS is disabled." http2=false address=[::]:9095 2024-04-07T16:18:51.606 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:18:51 smithi144 bash[58467]: ts=2024-04-07T16:18:51.185Z caller=head.go:587 level=info component=tsdb msg="Replaying on-disk memory mappable chunks if any" 2024-04-07T16:18:51.606 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:18:51 smithi144 bash[58467]: ts=2024-04-07T16:18:51.185Z caller=head.go:658 level=info component=tsdb msg="On-disk memory mappable chunks replay completed" duration=1.93µs 2024-04-07T16:18:51.606 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:18:51 smithi144 bash[58467]: ts=2024-04-07T16:18:51.185Z caller=head.go:664 level=info component=tsdb msg="Replaying WAL, this may take a while" 2024-04-07T16:18:51.606 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:18:51 smithi144 bash[58467]: ts=2024-04-07T16:18:51.187Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=0 maxSegment=2 2024-04-07T16:18:51.606 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:18:51 smithi144 bash[58467]: ts=2024-04-07T16:18:51.197Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=1 maxSegment=2 2024-04-07T16:18:51.606 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:18:51 smithi144 bash[58467]: ts=2024-04-07T16:18:51.197Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=2 maxSegment=2 2024-04-07T16:18:51.606 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:18:51 smithi144 bash[58467]: ts=2024-04-07T16:18:51.197Z caller=head.go:772 level=info component=tsdb msg="WAL replay completed" checkpoint_replay_duration=24.117µs wal_replay_duration=12.229831ms wbl_replay_duration=136ns total_replay_duration=12.276362ms 2024-04-07T16:18:51.606 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:18:51 smithi144 bash[58467]: ts=2024-04-07T16:18:51.199Z caller=main.go:1026 level=info fs_type=XFS_SUPER_MAGIC 2024-04-07T16:18:51.606 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:18:51 smithi144 bash[58467]: ts=2024-04-07T16:18:51.199Z caller=main.go:1029 level=info msg="TSDB started" 2024-04-07T16:18:51.607 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:18:51 smithi144 bash[58467]: ts=2024-04-07T16:18:51.199Z caller=main.go:1209 level=info msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml 2024-04-07T16:18:51.607 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:18:51 smithi144 bash[58467]: ts=2024-04-07T16:18:51.225Z caller=main.go:1246 level=info msg="Completed loading of configuration file" filename=/etc/prometheus/prometheus.yml totalDuration=26.131977ms db_storage=1.184µs remote_storage=1.214µs web_handler=392ns query_engine=715ns scrape=287.384µs scrape_sd=100.054µs notify=23.089µs notify_sd=28.682µs rules=25.310851ms tracing=7.097µs 2024-04-07T16:18:51.607 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:18:51 smithi144 bash[58467]: ts=2024-04-07T16:18:51.225Z caller=main.go:990 level=info msg="Server is ready to receive web requests." 2024-04-07T16:18:51.607 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:18:51 smithi144 bash[58467]: ts=2024-04-07T16:18:51.225Z caller=manager.go:974 level=info component="rule manager" msg="Starting rule manager..." 2024-04-07T16:18:51.891 INFO:tasks.rados.rados.0.smithi100.stdout:743: expect (ObjNum 190 snap 49 seq_num 190) 2024-04-07T16:18:52.019 INFO:tasks.rados.rados.0.smithi100.stdout:744: finishing write tid 1 to smithi10066867-7 2024-04-07T16:18:52.019 INFO:tasks.rados.rados.0.smithi100.stdout:742: expect (ObjNum 206 snap 56 seq_num 206) 2024-04-07T16:18:52.046 INFO:tasks.rados.rados.0.smithi100.stdout:744: finishing write tid 2 to smithi10066867-7 2024-04-07T16:18:52.046 INFO:tasks.rados.rados.0.smithi100.stdout:744: finishing write tid 3 to smithi10066867-7 2024-04-07T16:18:52.047 INFO:tasks.rados.rados.0.smithi100.stdout:744: finishing write tid 4 to smithi10066867-7 2024-04-07T16:18:52.047 INFO:tasks.rados.rados.0.smithi100.stdout:744: finishing write tid 5 to smithi10066867-7 2024-04-07T16:18:52.047 INFO:tasks.rados.rados.0.smithi100.stdout:744: oid 7 updating version 0 to 233 2024-04-07T16:18:52.047 INFO:tasks.rados.rados.0.smithi100.stdout:744: oid 7 version 233 is already newer than 232 2024-04-07T16:18:52.047 INFO:tasks.rados.rados.0.smithi100.stdout:744: oid 7 updating version 233 to 235 2024-04-07T16:18:52.047 INFO:tasks.rados.rados.0.smithi100.stdout:744: oid 7 version 235 is already newer than 234 2024-04-07T16:18:52.047 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 7 v 235 (ObjNum 239 snap 65 seq_num 239) dirty exists 2024-04-07T16:18:52.047 INFO:tasks.rados.rados.0.smithi100.stdout:744: left oid 7 (ObjNum 239 snap 65 seq_num 239) 2024-04-07T16:18:52.047 INFO:tasks.rados.rados.0.smithi100.stdout:742: done (3 left) 2024-04-07T16:18:52.047 INFO:tasks.rados.rados.0.smithi100.stdout:743: done (2 left) 2024-04-07T16:18:52.047 INFO:tasks.rados.rados.0.smithi100.stdout:744: done (1 left) 2024-04-07T16:18:52.047 INFO:tasks.rados.rados.0.smithi100.stdout:746: done (0 left) 2024-04-07T16:18:52.047 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 23 current snap is 65 2024-04-07T16:18:52.048 INFO:tasks.rados.rados.0.smithi100.stdout:747: seq_num 240 ranges {3186688=425984,3612672=589824,4202496=483328,4685824=131072} 2024-04-07T16:18:52.068 INFO:tasks.rados.rados.0.smithi100.stdout:747: writing smithi10066867-23 from 3186688 to 3612672 tid 1 2024-04-07T16:18:52.072 INFO:tasks.rados.rados.0.smithi100.stdout:747: writing smithi10066867-23 from 3612672 to 4202496 tid 2 2024-04-07T16:18:52.076 INFO:tasks.rados.rados.0.smithi100.stdout:747: writing smithi10066867-23 from 4202496 to 4685824 tid 3 2024-04-07T16:18:52.077 INFO:tasks.rados.rados.0.smithi100.stdout:747: writing smithi10066867-23 from 4685824 to 4816896 tid 4 2024-04-07T16:18:52.077 INFO:tasks.rados.rados.0.smithi100.stdout:748: rollback oid 44 current snap is 65 2024-04-07T16:18:52.077 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 44 to 64 2024-04-07T16:18:52.077 INFO:tasks.rados.rados.0.smithi100.stdout:749: snap_create 2024-04-07T16:18:52.083 INFO:tasks.rados.rados.0.smithi100.stdout:747: finishing write tid 1 to smithi10066867-23 2024-04-07T16:18:52.084 INFO:tasks.rados.rados.0.smithi100.stdout:747: finishing write tid 2 to smithi10066867-23 2024-04-07T16:18:52.084 INFO:tasks.rados.rados.0.smithi100.stdout:747: finishing write tid 3 to smithi10066867-23 2024-04-07T16:18:52.084 INFO:tasks.rados.rados.0.smithi100.stdout:747: finishing write tid 4 to smithi10066867-23 2024-04-07T16:18:52.084 INFO:tasks.rados.rados.0.smithi100.stdout:747: finishing write tid 5 to smithi10066867-23 2024-04-07T16:18:52.085 INFO:tasks.rados.rados.0.smithi100.stdout:748: finishing rollback tid 0 to smithi10066867-44 2024-04-07T16:18:52.085 INFO:tasks.rados.rados.0.smithi100.stdout:748: finishing rollback tid 1 to smithi10066867-44 2024-04-07T16:18:52.085 INFO:tasks.rados.rados.0.smithi100.stdout:748: finishing rollback tid 2 to smithi10066867-44 2024-04-07T16:18:52.085 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 44 v 318 (ObjNum 165 snap 44 seq_num 165) dirty exists 2024-04-07T16:18:52.085 INFO:tasks.rados.rados.0.smithi100.stdout:747: finishing write tid 6 to smithi10066867-23 2024-04-07T16:18:52.086 INFO:tasks.rados.rados.0.smithi100.stdout:747: oid 23 updating version 0 to 450 2024-04-07T16:18:52.086 INFO:tasks.rados.rados.0.smithi100.stdout:747: oid 23 updating version 450 to 453 2024-04-07T16:18:52.086 INFO:tasks.rados.rados.0.smithi100.stdout:747: oid 23 version 453 is already newer than 452 2024-04-07T16:18:52.086 INFO:tasks.rados.rados.0.smithi100.stdout:747: oid 23 version 453 is already newer than 449 2024-04-07T16:18:52.086 INFO:tasks.rados.rados.0.smithi100.stdout:747: oid 23 version 453 is already newer than 451 2024-04-07T16:18:52.086 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 23 v 453 (ObjNum 240 snap 65 seq_num 240) dirty exists 2024-04-07T16:18:52.086 INFO:tasks.rados.rados.0.smithi100.stdout:747: left oid 23 (ObjNum 240 snap 65 seq_num 240) 2024-04-07T16:18:52.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:51 smithi100 bash[21346]: audit 2024-04-07T16:18:50.467074+0000 mgr.x (mgr.24553) 34 : audit 0 from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-07T16:18:52.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:51 smithi100 bash[21346]: cephadm 2024-04-07T16:18:50.475331+0000 mgr.x (mgr.24553) 35 : cephadm 1 Upgrade: Need to upgrade myself (mgr.x) 2024-04-07T16:18:52.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:51 smithi100 bash[21346]: cephadm 2024-04-07T16:18:50.475953+0000 mgr.x (mgr.24553) 36 : cephadm 1 Upgrade: Need to upgrade myself (mgr.x) 2024-04-07T16:18:52.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:51 smithi100 bash[21346]: cluster 2024-04-07T16:18:50.650295+0000 mgr.x (mgr.24553) 37 : cluster 0 pgmap v48: 121 pgs: 121 active+clean; 130 MiB data, 717 MiB used, 715 GiB / 715 GiB avail; 11 MiB/s rd, 2.0 MiB/s wr, 26 op/s 2024-04-07T16:18:52.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:51 smithi100 bash[21346]: audit 2024-04-07T16:18:50.690670+0000 mgr.x (mgr.24553) 38 : audit 0 from='client.25055 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:18:52.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:51 smithi100 bash[21346]: cluster 2024-04-07T16:18:50.896598+0000 mon.a (mon.0) 1075 : cluster 0 osdmap e197: 8 total, 8 up, 8 in 2024-04-07T16:18:52.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:51 smithi100 bash[21346]: audit 2024-04-07T16:18:51.079193+0000 mgr.x (mgr.24553) 39 : audit 0 from='client.25061 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:18:52.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:51 smithi100 bash[21346]: audit 2024-04-07T16:18:51.534607+0000 mon.a (mon.0) 1076 : audit 0 from='client.? 172.21.15.100:0/148223919' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:18:52.188 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:51 smithi100 bash[31660]: audit 2024-04-07T16:18:50.467074+0000 mgr.x (mgr.24553) 34 : audit 0 from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-07T16:18:52.188 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:51 smithi100 bash[31660]: cephadm 2024-04-07T16:18:50.475331+0000 mgr.x (mgr.24553) 35 : cephadm 1 Upgrade: Need to upgrade myself (mgr.x) 2024-04-07T16:18:52.188 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:51 smithi100 bash[31660]: cephadm 2024-04-07T16:18:50.475953+0000 mgr.x (mgr.24553) 36 : cephadm 1 Upgrade: Need to upgrade myself (mgr.x) 2024-04-07T16:18:52.188 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:51 smithi100 bash[31660]: cluster 2024-04-07T16:18:50.650295+0000 mgr.x (mgr.24553) 37 : cluster 0 pgmap v48: 121 pgs: 121 active+clean; 130 MiB data, 717 MiB used, 715 GiB / 715 GiB avail; 11 MiB/s rd, 2.0 MiB/s wr, 26 op/s 2024-04-07T16:18:52.188 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:51 smithi100 bash[31660]: audit 2024-04-07T16:18:50.690670+0000 mgr.x (mgr.24553) 38 : audit 0 from='client.25055 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:18:52.188 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:51 smithi100 bash[31660]: cluster 2024-04-07T16:18:50.896598+0000 mon.a (mon.0) 1075 : cluster 0 osdmap e197: 8 total, 8 up, 8 in 2024-04-07T16:18:52.188 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:51 smithi100 bash[31660]: audit 2024-04-07T16:18:51.079193+0000 mgr.x (mgr.24553) 39 : audit 0 from='client.25061 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:18:52.188 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:51 smithi100 bash[31660]: audit 2024-04-07T16:18:51.534607+0000 mon.a (mon.0) 1076 : audit 0 from='client.? 172.21.15.100:0/148223919' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:18:52.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:51 smithi144 bash[23522]: audit 2024-04-07T16:18:50.467074+0000 mgr.x (mgr.24553) 34 : audit 0 from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-07T16:18:52.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:51 smithi144 bash[23522]: cephadm 2024-04-07T16:18:50.475331+0000 mgr.x (mgr.24553) 35 : cephadm 1 Upgrade: Need to upgrade myself (mgr.x) 2024-04-07T16:18:52.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:51 smithi144 bash[23522]: cephadm 2024-04-07T16:18:50.475953+0000 mgr.x (mgr.24553) 36 : cephadm 1 Upgrade: Need to upgrade myself (mgr.x) 2024-04-07T16:18:52.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:51 smithi144 bash[23522]: cluster 2024-04-07T16:18:50.650295+0000 mgr.x (mgr.24553) 37 : cluster 0 pgmap v48: 121 pgs: 121 active+clean; 130 MiB data, 717 MiB used, 715 GiB / 715 GiB avail; 11 MiB/s rd, 2.0 MiB/s wr, 26 op/s 2024-04-07T16:18:52.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:51 smithi144 bash[23522]: audit 2024-04-07T16:18:50.690670+0000 mgr.x (mgr.24553) 38 : audit 0 from='client.25055 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:18:52.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:51 smithi144 bash[23522]: cluster 2024-04-07T16:18:50.896598+0000 mon.a (mon.0) 1075 : cluster 0 osdmap e197: 8 total, 8 up, 8 in 2024-04-07T16:18:52.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:51 smithi144 bash[23522]: audit 2024-04-07T16:18:51.079193+0000 mgr.x (mgr.24553) 39 : audit 0 from='client.25061 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:18:52.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:51 smithi144 bash[23522]: audit 2024-04-07T16:18:51.534607+0000 mon.a (mon.0) 1076 : audit 0 from='client.? 172.21.15.100:0/148223919' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:18:52.861 INFO:tasks.rados.rados.0.smithi100.stdout:747: done (2 left) 2024-04-07T16:18:52.861 INFO:tasks.rados.rados.0.smithi100.stdout:748: done (1 left) 2024-04-07T16:18:52.861 INFO:tasks.rados.rados.0.smithi100.stdout:749: done (0 left) 2024-04-07T16:18:52.861 INFO:tasks.rados.rados.0.smithi100.stdout:750: copy_from oid 48 from oid 7 current snap is 66 2024-04-07T16:18:52.861 INFO:tasks.rados.rados.0.smithi100.stdout:751: read oid 2 snap -1 2024-04-07T16:18:52.862 INFO:tasks.rados.rados.0.smithi100.stdout:751: expect deleted 2024-04-07T16:18:52.862 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 13 current snap is 66 2024-04-07T16:18:52.862 INFO:tasks.rados.rados.0.smithi100.stdout:752: seq_num 241 ranges {1695744=442368,2138112=466944,2605056=688128,3293184=49152} 2024-04-07T16:18:52.881 INFO:tasks.rados.rados.0.smithi100.stdout:752: writing smithi10066867-13 from 1695744 to 2138112 tid 1 2024-04-07T16:18:52.885 INFO:tasks.rados.rados.0.smithi100.stdout:752: writing smithi10066867-13 from 2138112 to 2605056 tid 2 2024-04-07T16:18:52.892 INFO:tasks.rados.rados.0.smithi100.stdout:752: writing smithi10066867-13 from 2605056 to 3293184 tid 3 2024-04-07T16:18:52.893 INFO:tasks.rados.rados.0.smithi100.stdout:752: writing smithi10066867-13 from 3293184 to 3342336 tid 4 2024-04-07T16:18:52.893 INFO:tasks.rados.rados.0.smithi100.stdout:753: read oid 47 snap -1 2024-04-07T16:18:52.893 INFO:tasks.rados.rados.0.smithi100.stdout:753: expect (ObjNum 149 snap 41 seq_num 149) 2024-04-07T16:18:52.893 INFO:tasks.rados.rados.0.smithi100.stdout:754: delete oid 26 current snap is 66 2024-04-07T16:18:52.893 INFO:tasks.rados.rados.0.smithi100.stdout:750: finishing copy_from racing read to smithi10066867-48 2024-04-07T16:18:52.897 INFO:tasks.rados.rados.0.smithi100.stdout:750: finishing copy_from to smithi10066867-48 2024-04-07T16:18:52.897 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 48 v 235 (ObjNum 239 snap 65 seq_num 239) dirty exists 2024-04-07T16:18:52.900 INFO:tasks.rados.rados.0.smithi100.stdout:753: expect (ObjNum 149 snap 41 seq_num 149) 2024-04-07T16:18:52.905 INFO:tasks.rados.rados.0.smithi100.stdout:752: finishing write tid 1 to smithi10066867-13 2024-04-07T16:18:52.906 INFO:tasks.rados.rados.0.smithi100.stdout:752: finishing write tid 2 to smithi10066867-13 2024-04-07T16:18:52.906 INFO:tasks.rados.rados.0.smithi100.stdout:750: done (4 left) 2024-04-07T16:18:52.906 INFO:tasks.rados.rados.0.smithi100.stdout:751: done (3 left) 2024-04-07T16:18:52.906 INFO:tasks.rados.rados.0.smithi100.stdout:753: done (2 left) 2024-04-07T16:18:52.906 INFO:tasks.rados.rados.0.smithi100.stdout:754: done (1 left) 2024-04-07T16:18:52.906 INFO:tasks.rados.rados.0.smithi100.stdout:755: read oid 21 snap -1 2024-04-07T16:18:52.906 INFO:tasks.rados.rados.0.smithi100.stdout:755: expect (ObjNum 222 snap 59 seq_num 222) 2024-04-07T16:18:52.906 INFO:tasks.rados.rados.0.smithi100.stdout:756: delete oid 35 current snap is 66 2024-04-07T16:18:52.906 INFO:tasks.rados.rados.0.smithi100.stdout:752: finishing write tid 3 to smithi10066867-13 2024-04-07T16:18:52.907 INFO:tasks.rados.rados.0.smithi100.stdout:752: finishing write tid 4 to smithi10066867-13 2024-04-07T16:18:52.907 INFO:tasks.rados.rados.0.smithi100.stdout:752: finishing write tid 5 to smithi10066867-13 2024-04-07T16:18:52.909 INFO:tasks.rados.rados.0.smithi100.stdout:756: done (2 left) 2024-04-07T16:18:52.909 INFO:tasks.rados.rados.0.smithi100.stdout:757: rmattr oid 20 current snap is 66 2024-04-07T16:18:52.909 INFO:tasks.rados.rados.0.smithi100.stdout:757: done (2 left) 2024-04-07T16:18:52.909 INFO:tasks.rados.rados.0.smithi100.stdout:758: copy_from oid 35 from oid 8 current snap is 66 2024-04-07T16:18:52.909 INFO:tasks.rados.rados.0.smithi100.stdout:759: read oid 41 snap -1 2024-04-07T16:18:52.909 INFO:tasks.rados.rados.0.smithi100.stdout:759: expect (ObjNum 140 snap 39 seq_num 140) 2024-04-07T16:18:52.909 INFO:tasks.rados.rados.0.smithi100.stdout:760: snap_remove snap 65 2024-04-07T16:18:53.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:52 smithi100 bash[21346]: cluster 2024-04-07T16:18:51.902234+0000 mon.a (mon.0) 1077 : cluster 0 osdmap e198: 8 total, 8 up, 8 in 2024-04-07T16:18:53.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:52 smithi100 bash[21346]: cluster 2024-04-07T16:18:52.874689+0000 mon.a (mon.0) 1078 : cluster 0 osdmap e199: 8 total, 8 up, 8 in 2024-04-07T16:18:53.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:52 smithi100 bash[31660]: cluster 2024-04-07T16:18:51.902234+0000 mon.a (mon.0) 1077 : cluster 0 osdmap e198: 8 total, 8 up, 8 in 2024-04-07T16:18:53.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:52 smithi100 bash[31660]: cluster 2024-04-07T16:18:52.874689+0000 mon.a (mon.0) 1078 : cluster 0 osdmap e199: 8 total, 8 up, 8 in 2024-04-07T16:18:53.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:52 smithi144 bash[23522]: cluster 2024-04-07T16:18:51.902234+0000 mon.a (mon.0) 1077 : cluster 0 osdmap e198: 8 total, 8 up, 8 in 2024-04-07T16:18:53.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:52 smithi144 bash[23522]: cluster 2024-04-07T16:18:52.874689+0000 mon.a (mon.0) 1078 : cluster 0 osdmap e199: 8 total, 8 up, 8 in 2024-04-07T16:18:53.916 INFO:tasks.rados.rados.0.smithi100.stdout:760: done (4 left) 2024-04-07T16:18:53.916 INFO:tasks.rados.rados.0.smithi100.stdout:761: rmattr oid 40 current snap is 66 2024-04-07T16:18:53.917 INFO:tasks.rados.rados.0.smithi100.stdout:752: finishing write tid 6 to smithi10066867-13 2024-04-07T16:18:53.917 INFO:tasks.rados.rados.0.smithi100.stdout:752: oid 13 updating version 0 to 262 2024-04-07T16:18:53.917 INFO:tasks.rados.rados.0.smithi100.stdout:752: oid 13 updating version 262 to 265 2024-04-07T16:18:53.917 INFO:tasks.rados.rados.0.smithi100.stdout:752: oid 13 version 265 is already newer than 263 2024-04-07T16:18:53.917 INFO:tasks.rados.rados.0.smithi100.stdout:752: oid 13 version 265 is already newer than 264 2024-04-07T16:18:53.917 INFO:tasks.rados.rados.0.smithi100.stdout:752: oid 13 version 265 is already newer than 261 2024-04-07T16:18:53.917 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 13 v 265 (ObjNum 241 snap 66 seq_num 241) dirty exists 2024-04-07T16:18:53.918 INFO:tasks.rados.rados.0.smithi100.stdout:752: left oid 13 (ObjNum 241 snap 66 seq_num 241) 2024-04-07T16:18:53.918 INFO:tasks.rados.rados.0.smithi100.stdout:759: expect (ObjNum 140 snap 39 seq_num 140) 2024-04-07T16:18:54.029 INFO:tasks.rados.rados.0.smithi100.stdout:758: finishing copy_from racing read to smithi10066867-35 2024-04-07T16:18:54.029 INFO:tasks.rados.rados.0.smithi100.stdout:758: finishing copy_from to smithi10066867-35 2024-04-07T16:18:54.029 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 35 v 183 (ObjNum 208 snap 56 seq_num 208) dirty exists 2024-04-07T16:18:54.030 INFO:tasks.rados.rados.0.smithi100.stdout:755: expect (ObjNum 222 snap 59 seq_num 222) 2024-04-07T16:18:54.295 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:54 smithi100 bash[21346]: cluster 2024-04-07T16:18:52.651076+0000 mgr.x (mgr.24553) 40 : cluster 0 pgmap v51: 121 pgs: 121 active+clean; 155 MiB data, 761 MiB used, 714 GiB / 715 GiB avail; 7.7 MiB/s rd, 4.6 MiB/s wr, 20 op/s 2024-04-07T16:18:54.295 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:54 smithi100 bash[21346]: cephadm 2024-04-07T16:18:53.001334+0000 mgr.x (mgr.24553) 41 : cephadm 1 Upgrade: Updating mgr.y 2024-04-07T16:18:54.295 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:54 smithi100 bash[21346]: audit 2024-04-07T16:18:53.016352+0000 mon.b (mon.1) 316 : audit 1 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-07T16:18:54.295 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:54 smithi100 bash[21346]: audit 2024-04-07T16:18:53.018348+0000 mon.a (mon.0) 1079 : audit 1 from='mgr.24553 ' entity='mgr.x' 2024-04-07T16:18:54.295 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:54 smithi100 bash[21346]: audit 2024-04-07T16:18:53.019481+0000 mon.b (mon.1) 317 : audit 0 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-07T16:18:54.295 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:54 smithi100 bash[21346]: audit 2024-04-07T16:18:53.021340+0000 mon.b (mon.1) 318 : audit 0 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:18:54.295 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:54 smithi100 bash[21346]: audit 2024-04-07T16:18:53.021548+0000 mon.a (mon.0) 1080 : audit 1 from='mgr.24553 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-07T16:18:54.296 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:54 smithi100 bash[21346]: cephadm 2024-04-07T16:18:53.023347+0000 mgr.x (mgr.24553) 42 : cephadm 1 Deploying daemon mgr.y on smithi100 2024-04-07T16:18:54.296 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:54 smithi100 bash[21346]: cluster 2024-04-07T16:18:53.930439+0000 mon.a (mon.0) 1081 : cluster 0 osdmap e200: 8 total, 8 up, 8 in 2024-04-07T16:18:54.296 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:54 smithi100 bash[31660]: cluster 2024-04-07T16:18:52.651076+0000 mgr.x (mgr.24553) 40 : cluster 0 pgmap v51: 121 pgs: 121 active+clean; 155 MiB data, 761 MiB used, 714 GiB / 715 GiB avail; 7.7 MiB/s rd, 4.6 MiB/s wr, 20 op/s 2024-04-07T16:18:54.296 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:54 smithi100 bash[31660]: cephadm 2024-04-07T16:18:53.001334+0000 mgr.x (mgr.24553) 41 : cephadm 1 Upgrade: Updating mgr.y 2024-04-07T16:18:54.296 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:54 smithi100 bash[31660]: audit 2024-04-07T16:18:53.016352+0000 mon.b (mon.1) 316 : audit 1 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-07T16:18:54.296 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:54 smithi100 bash[31660]: audit 2024-04-07T16:18:53.018348+0000 mon.a (mon.0) 1079 : audit 1 from='mgr.24553 ' entity='mgr.x' 2024-04-07T16:18:54.296 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:54 smithi100 bash[31660]: audit 2024-04-07T16:18:53.019481+0000 mon.b (mon.1) 317 : audit 0 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-07T16:18:54.296 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:54 smithi100 bash[31660]: audit 2024-04-07T16:18:53.021340+0000 mon.b (mon.1) 318 : audit 0 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:18:54.296 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:54 smithi100 bash[31660]: audit 2024-04-07T16:18:53.021548+0000 mon.a (mon.0) 1080 : audit 1 from='mgr.24553 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-07T16:18:54.296 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:54 smithi100 bash[31660]: cephadm 2024-04-07T16:18:53.023347+0000 mgr.x (mgr.24553) 42 : cephadm 1 Deploying daemon mgr.y on smithi100 2024-04-07T16:18:54.296 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:54 smithi100 bash[31660]: cluster 2024-04-07T16:18:53.930439+0000 mon.a (mon.0) 1081 : cluster 0 osdmap e200: 8 total, 8 up, 8 in 2024-04-07T16:18:54.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:54 smithi144 bash[23522]: cluster 2024-04-07T16:18:52.651076+0000 mgr.x (mgr.24553) 40 : cluster 0 pgmap v51: 121 pgs: 121 active+clean; 155 MiB data, 761 MiB used, 714 GiB / 715 GiB avail; 7.7 MiB/s rd, 4.6 MiB/s wr, 20 op/s 2024-04-07T16:18:54.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:54 smithi144 bash[23522]: cephadm 2024-04-07T16:18:53.001334+0000 mgr.x (mgr.24553) 41 : cephadm 1 Upgrade: Updating mgr.y 2024-04-07T16:18:54.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:54 smithi144 bash[23522]: audit 2024-04-07T16:18:53.016352+0000 mon.b (mon.1) 316 : audit 1 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-07T16:18:54.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:54 smithi144 bash[23522]: audit 2024-04-07T16:18:53.018348+0000 mon.a (mon.0) 1079 : audit 1 from='mgr.24553 ' entity='mgr.x' 2024-04-07T16:18:54.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:54 smithi144 bash[23522]: audit 2024-04-07T16:18:53.019481+0000 mon.b (mon.1) 317 : audit 0 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-07T16:18:54.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:54 smithi144 bash[23522]: audit 2024-04-07T16:18:53.021340+0000 mon.b (mon.1) 318 : audit 0 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:18:54.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:54 smithi144 bash[23522]: audit 2024-04-07T16:18:53.021548+0000 mon.a (mon.0) 1080 : audit 1 from='mgr.24553 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-07T16:18:54.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:54 smithi144 bash[23522]: cephadm 2024-04-07T16:18:53.023347+0000 mgr.x (mgr.24553) 42 : cephadm 1 Deploying daemon mgr.y on smithi100 2024-04-07T16:18:54.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:54 smithi144 bash[23522]: cluster 2024-04-07T16:18:53.930439+0000 mon.a (mon.0) 1081 : cluster 0 osdmap e200: 8 total, 8 up, 8 in 2024-04-07T16:18:54.360 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 40 v 320 (ObjNum 214 snap 58 seq_num 214) dirty exists 2024-04-07T16:18:54.360 INFO:tasks.rados.rados.0.smithi100.stdout:752: done (4 left) 2024-04-07T16:18:54.360 INFO:tasks.rados.rados.0.smithi100.stdout:755: done (3 left) 2024-04-07T16:18:54.360 INFO:tasks.rados.rados.0.smithi100.stdout:758: done (2 left) 2024-04-07T16:18:54.361 INFO:tasks.rados.rados.0.smithi100.stdout:759: done (1 left) 2024-04-07T16:18:54.361 INFO:tasks.rados.rados.0.smithi100.stdout:761: done (0 left) 2024-04-07T16:18:54.361 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 44 current snap is 66 2024-04-07T16:18:54.361 INFO:tasks.rados.rados.0.smithi100.stdout:762: seq_num 244 ranges {958464=491520,1449984=573440,2023424=458752,2482176=172032} 2024-04-07T16:18:54.368 INFO:tasks.rados.rados.0.smithi100.stdout:762: writing smithi10066867-44 from 958464 to 1449984 tid 1 2024-04-07T16:18:54.371 INFO:tasks.rados.rados.0.smithi100.stdout:762: writing smithi10066867-44 from 1449984 to 2023424 tid 2 2024-04-07T16:18:54.374 INFO:tasks.rados.rados.0.smithi100.stdout:762: writing smithi10066867-44 from 2023424 to 2482176 tid 3 2024-04-07T16:18:54.375 INFO:tasks.rados.rados.0.smithi100.stdout:762: writing smithi10066867-44 from 2482176 to 2654208 tid 4 2024-04-07T16:18:54.376 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 11 current snap is 66 2024-04-07T16:18:54.376 INFO:tasks.rados.rados.0.smithi100.stdout:762: finishing write tid 1 to smithi10066867-44 2024-04-07T16:18:54.376 INFO:tasks.rados.rados.0.smithi100.stdout:763: seq_num 245 ranges {1212416=507904,1720320=458752,2179072=671744,2850816=73728} 2024-04-07T16:18:54.385 INFO:tasks.rados.rados.0.smithi100.stdout:763: writing smithi10066867-11 from 1212416 to 1720320 tid 1 2024-04-07T16:18:54.388 INFO:tasks.rados.rados.0.smithi100.stdout:763: writing smithi10066867-11 from 1720320 to 2179072 tid 2 2024-04-07T16:18:54.391 INFO:tasks.rados.rados.0.smithi100.stdout:763: writing smithi10066867-11 from 2179072 to 2850816 tid 3 2024-04-07T16:18:54.392 INFO:tasks.rados.rados.0.smithi100.stdout:763: writing smithi10066867-11 from 2850816 to 2924544 tid 4 2024-04-07T16:18:54.392 INFO:tasks.rados.rados.0.smithi100.stdout:764: setattr oid 1 current snap is 66 2024-04-07T16:18:54.393 INFO:tasks.rados.rados.0.smithi100.stdout:762: finishing write tid 2 to smithi10066867-44 2024-04-07T16:18:54.393 INFO:tasks.rados.rados.0.smithi100.stdout:762: finishing write tid 3 to smithi10066867-44 2024-04-07T16:18:54.393 INFO:tasks.rados.rados.0.smithi100.stdout:762: finishing write tid 4 to smithi10066867-44 2024-04-07T16:18:54.393 INFO:tasks.rados.rados.0.smithi100.stdout:762: finishing write tid 5 to smithi10066867-44 2024-04-07T16:18:54.393 INFO:tasks.rados.rados.0.smithi100.stdout:762: finishing write tid 6 to smithi10066867-44 2024-04-07T16:18:54.393 INFO:tasks.rados.rados.0.smithi100.stdout:762: oid 44 updating version 0 to 325 2024-04-07T16:18:54.393 INFO:tasks.rados.rados.0.smithi100.stdout:762: oid 44 version 325 is already newer than 322 2024-04-07T16:18:54.393 INFO:tasks.rados.rados.0.smithi100.stdout:762: oid 44 version 325 is already newer than 324 2024-04-07T16:18:54.393 INFO:tasks.rados.rados.0.smithi100.stdout:762: oid 44 version 325 is already newer than 323 2024-04-07T16:18:54.393 INFO:tasks.rados.rados.0.smithi100.stdout:762: oid 44 version 325 is already newer than 321 2024-04-07T16:18:54.393 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 44 v 325 (ObjNum 244 snap 66 seq_num 244) dirty exists 2024-04-07T16:18:54.393 INFO:tasks.rados.rados.0.smithi100.stdout:762: left oid 44 (ObjNum 244 snap 66 seq_num 244) 2024-04-07T16:18:54.393 INFO:tasks.rados.rados.0.smithi100.stdout:763: finishing write tid 1 to smithi10066867-11 2024-04-07T16:18:54.393 INFO:tasks.rados.rados.0.smithi100.stdout:763: finishing write tid 2 to smithi10066867-11 2024-04-07T16:18:54.395 INFO:tasks.rados.rados.0.smithi100.stdout:762: done (2 left) 2024-04-07T16:18:54.395 INFO:tasks.rados.rados.0.smithi100.stdout:765: rollback oid 31 current snap is 66 2024-04-07T16:18:54.395 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 31 to 63 2024-04-07T16:18:54.395 INFO:tasks.rados.rados.0.smithi100.stdout:766: setattr oid 16 current snap is 66 2024-04-07T16:18:54.398 INFO:tasks.rados.rados.0.smithi100.stdout:763: finishing write tid 3 to smithi10066867-11 2024-04-07T16:18:54.398 INFO:tasks.rados.rados.0.smithi100.stdout:767: read oid 40 snap 58 2024-04-07T16:18:54.398 INFO:tasks.rados.rados.0.smithi100.stdout:767: expect (ObjNum 214 snap 58 seq_num 214) 2024-04-07T16:18:54.398 INFO:tasks.rados.rados.0.smithi100.stdout:768: read oid 47 snap -1 2024-04-07T16:18:54.398 INFO:tasks.rados.rados.0.smithi100.stdout:768: expect (ObjNum 149 snap 41 seq_num 149) 2024-04-07T16:18:54.398 INFO:tasks.rados.rados.0.smithi100.stdout:769: rollback oid 41 current snap is 66 2024-04-07T16:18:54.398 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 41 to 53 2024-04-07T16:18:54.398 INFO:tasks.rados.rados.0.smithi100.stdout:770: rollback oid 14 current snap is 66 2024-04-07T16:18:54.398 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 14 to 64 2024-04-07T16:18:54.399 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 28 current snap is 66 2024-04-07T16:18:54.399 INFO:tasks.rados.rados.0.smithi100.stdout:771: seq_num 246 ranges {2359296=524288,2883584=745472,3629056=425984,4055040=32768} 2024-04-07T16:18:54.416 INFO:tasks.rados.rados.0.smithi100.stdout:771: writing smithi10066867-28 from 2359296 to 2883584 tid 1 2024-04-07T16:18:54.420 INFO:tasks.rados.rados.0.smithi100.stdout:771: writing smithi10066867-28 from 2883584 to 3629056 tid 2 2024-04-07T16:18:54.423 INFO:tasks.rados.rados.0.smithi100.stdout:771: writing smithi10066867-28 from 3629056 to 4055040 tid 3 2024-04-07T16:18:54.424 INFO:tasks.rados.rados.0.smithi100.stdout:771: writing smithi10066867-28 from 4055040 to 4087808 tid 4 2024-04-07T16:18:54.424 INFO:tasks.rados.rados.0.smithi100.stdout:772: copy_from oid 8 from oid 34 current snap is 66 2024-04-07T16:18:54.424 INFO:tasks.rados.rados.0.smithi100.stdout:765: finishing rollback tid 0 to smithi10066867-31 2024-04-07T16:18:54.424 INFO:tasks.rados.rados.0.smithi100.stdout:763: finishing write tid 4 to smithi10066867-11 2024-04-07T16:18:54.424 INFO:tasks.rados.rados.0.smithi100.stdout:763: finishing write tid 5 to smithi10066867-11 2024-04-07T16:18:54.424 INFO:tasks.rados.rados.0.smithi100.stdout:765: finishing rollback tid 1 to smithi10066867-31 2024-04-07T16:18:54.424 INFO:tasks.rados.rados.0.smithi100.stdout:765: finishing rollback tid 2 to smithi10066867-31 2024-04-07T16:18:54.424 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 31 v 238 (ObjNum 223 snap 59 seq_num 223) dirty exists 2024-04-07T16:18:54.425 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 1 v 214 (ObjNum 210 snap 57 seq_num 210) dirty exists 2024-04-07T16:18:54.425 INFO:tasks.rados.rados.0.smithi100.stdout:764: done (9 left) 2024-04-07T16:18:54.425 INFO:tasks.rados.rados.0.smithi100.stdout:765: done (8 left) 2024-04-07T16:18:54.425 INFO:tasks.rados.rados.0.smithi100.stdout:773: snap_remove snap 59 2024-04-07T16:18:55.015 INFO:tasks.rados.rados.0.smithi100.stdout:773: done (8 left) 2024-04-07T16:18:55.015 INFO:tasks.rados.rados.0.smithi100.stdout:774: copy_from oid 21 from oid 18 current snap is 66 2024-04-07T16:18:55.015 INFO:tasks.rados.rados.0.smithi100.stdout:770: finishing rollback tid 1 to smithi10066867-14 2024-04-07T16:18:55.016 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 14 v 172 (ObjNum 273 snap 0 seq_num 0) dirty dne 2024-04-07T16:18:55.016 INFO:tasks.rados.rados.0.smithi100.stdout:763: finishing write tid 6 to smithi10066867-11 2024-04-07T16:18:55.016 INFO:tasks.rados.rados.0.smithi100.stdout:763: oid 11 updating version 0 to 458 2024-04-07T16:18:55.016 INFO:tasks.rados.rados.0.smithi100.stdout:763: oid 11 updating version 458 to 459 2024-04-07T16:18:55.016 INFO:tasks.rados.rados.0.smithi100.stdout:763: oid 11 updating version 459 to 460 2024-04-07T16:18:55.016 INFO:tasks.rados.rados.0.smithi100.stdout:763: oid 11 version 460 is already newer than 457 2024-04-07T16:18:55.016 INFO:tasks.rados.rados.0.smithi100.stdout:763: oid 11 version 460 is already newer than 456 2024-04-07T16:18:55.016 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 11 v 460 (ObjNum 245 snap 66 seq_num 245) dirty exists 2024-04-07T16:18:55.016 INFO:tasks.rados.rados.0.smithi100.stdout:763: left oid 11 (ObjNum 245 snap 66 seq_num 245) 2024-04-07T16:18:55.016 INFO:tasks.rados.rados.0.smithi100.stdout:763: done (8 left) 2024-04-07T16:18:55.016 INFO:tasks.rados.rados.0.smithi100.stdout:770: done (7 left) 2024-04-07T16:18:55.016 INFO:tasks.rados.rados.0.smithi100.stdout:775: copy_from oid 29 from oid 33 current snap is 66 2024-04-07T16:18:55.016 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 16 v 339 (ObjNum 118 snap 29 seq_num 118) dirty exists 2024-04-07T16:18:55.016 INFO:tasks.rados.rados.0.smithi100.stdout:768: expect (ObjNum 149 snap 41 seq_num 149) 2024-04-07T16:18:55.020 INFO:tasks.rados.rados.0.smithi100.stdout:769: finishing rollback tid 0 to smithi10066867-41 2024-04-07T16:18:55.020 INFO:tasks.rados.rados.0.smithi100.stdout:769: finishing rollback tid 1 to smithi10066867-41 2024-04-07T16:18:55.020 INFO:tasks.rados.rados.0.smithi100.stdout:769: finishing rollback tid 2 to smithi10066867-41 2024-04-07T16:18:55.020 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 41 v 343 (ObjNum 45 snap 0 seq_num 45) dirty exists 2024-04-07T16:18:55.020 INFO:tasks.rados.rados.0.smithi100.stdout:767: expect (ObjNum 214 snap 58 seq_num 214) 2024-04-07T16:18:55.352 INFO:tasks.rados.rados.0.smithi100.stdout:771: finishing write tid 1 to smithi10066867-28 2024-04-07T16:18:55.352 INFO:tasks.rados.rados.0.smithi100.stdout:771: finishing write tid 2 to smithi10066867-28 2024-04-07T16:18:55.353 INFO:tasks.rados.rados.0.smithi100.stdout:771: finishing write tid 3 to smithi10066867-28 2024-04-07T16:18:55.353 INFO:tasks.rados.rados.0.smithi100.stdout:771: finishing write tid 4 to smithi10066867-28 2024-04-07T16:18:55.353 INFO:tasks.rados.rados.0.smithi100.stdout:771: finishing write tid 5 to smithi10066867-28 2024-04-07T16:18:55.353 INFO:tasks.rados.rados.0.smithi100.stdout:771: finishing write tid 6 to smithi10066867-28 2024-04-07T16:18:55.353 INFO:tasks.rados.rados.0.smithi100.stdout:771: oid 28 updating version 0 to 79 2024-04-07T16:18:55.353 INFO:tasks.rados.rados.0.smithi100.stdout:771: oid 28 updating version 79 to 81 2024-04-07T16:18:55.353 INFO:tasks.rados.rados.0.smithi100.stdout:771: oid 28 updating version 81 to 83 2024-04-07T16:18:55.353 INFO:tasks.rados.rados.0.smithi100.stdout:771: oid 28 version 83 is already newer than 82 2024-04-07T16:18:55.353 INFO:tasks.rados.rados.0.smithi100.stdout:771: oid 28 version 83 is already newer than 80 2024-04-07T16:18:55.353 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 28 v 83 (ObjNum 246 snap 66 seq_num 246) dirty exists 2024-04-07T16:18:55.353 INFO:tasks.rados.rados.0.smithi100.stdout:771: left oid 28 (ObjNum 246 snap 66 seq_num 246) 2024-04-07T16:18:55.353 INFO:tasks.rados.rados.0.smithi100.stdout:772: finishing copy_from racing read to smithi10066867-8 2024-04-07T16:18:55.353 INFO:tasks.rados.rados.0.smithi100.stdout:772: finishing copy_from to smithi10066867-8 2024-04-07T16:18:55.353 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 8 v 253 (ObjNum 233 snap 60 seq_num 233) dirty exists 2024-04-07T16:18:55.353 INFO:tasks.rados.rados.0.smithi100.stdout:774: finishing copy_from to smithi10066867-21 2024-04-07T16:18:55.353 INFO:tasks.rados.rados.0.smithi100.stdout:774: got expected ENOENT (src dne) 2024-04-07T16:18:55.354 INFO:tasks.rados.rados.0.smithi100.stdout:774: finishing copy_from racing read to smithi10066867-21 2024-04-07T16:18:55.354 INFO:tasks.rados.rados.0.smithi100.stdout:766: done (7 left) 2024-04-07T16:18:55.354 INFO:tasks.rados.rados.0.smithi100.stdout:767: done (6 left) 2024-04-07T16:18:55.354 INFO:tasks.rados.rados.0.smithi100.stdout:768: done (5 left) 2024-04-07T16:18:55.354 INFO:tasks.rados.rados.0.smithi100.stdout:769: done (4 left) 2024-04-07T16:18:55.354 INFO:tasks.rados.rados.0.smithi100.stdout:771: done (3 left) 2024-04-07T16:18:55.354 INFO:tasks.rados.rados.0.smithi100.stdout:772: done (2 left) 2024-04-07T16:18:55.354 INFO:tasks.rados.rados.0.smithi100.stdout:774: done (1 left) 2024-04-07T16:18:55.354 INFO:tasks.rados.rados.0.smithi100.stdout:776: setattr oid 21 current snap is 66 2024-04-07T16:18:55.355 INFO:tasks.rados.rados.0.smithi100.stdout:777: setattr oid 45 current snap is 66 2024-04-07T16:18:55.355 INFO:tasks.rados.rados.0.smithi100.stdout:778: snap_create 2024-04-07T16:18:55.359 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 21 v 125 (ObjNum 222 snap 59 seq_num 222) dirty exists 2024-04-07T16:18:55.361 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 45 v 200 (ObjNum 7 snap 0 seq_num 7) dirty exists 2024-04-07T16:18:55.368 INFO:tasks.rados.rados.0.smithi100.stdout:775: finishing copy_from racing read to smithi10066867-29 2024-04-07T16:18:55.376 INFO:tasks.rados.rados.0.smithi100.stdout:775: finishing copy_from to smithi10066867-29 2024-04-07T16:18:55.376 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 29 v 441 (ObjNum 161 snap 43 seq_num 161) dirty exists 2024-04-07T16:18:55.510 INFO:journalctl@ceph.osd.2.smithi100.stdout:Apr 07 16:18:55 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:18:55.510 INFO:journalctl@ceph.osd.1.smithi100.stdout:Apr 07 16:18:55 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:18:55.510 INFO:journalctl@ceph.osd.3.smithi100.stdout:Apr 07 16:18:55 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:18:55.511 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 16:18:55 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:18:55.511 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:55 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:18:55.511 INFO:journalctl@ceph.osd.0.smithi100.stdout:Apr 07 16:18:55 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:18:55.512 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:18:55 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:18:55.512 INFO:journalctl@ceph.mds.a.smithi100.stdout:Apr 07 16:18:55 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:18:55.512 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:55 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:18:55.512 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Apr 07 16:18:55 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:18:55.888 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:18:55 smithi100 systemd[1]: Stopping Ceph mgr.y for 0702e3a8-f4f8-11ee-b649-cb9ed24678a4... 2024-04-07T16:18:56.018 INFO:tasks.rados.rados.0.smithi100.stdout:775: done (3 left) 2024-04-07T16:18:56.018 INFO:tasks.rados.rados.0.smithi100.stdout:776: done (2 left) 2024-04-07T16:18:56.018 INFO:tasks.rados.rados.0.smithi100.stdout:777: done (1 left) 2024-04-07T16:18:56.018 INFO:tasks.rados.rados.0.smithi100.stdout:778: done (0 left) 2024-04-07T16:18:56.018 INFO:tasks.rados.rados.0.smithi100.stdout:779: rollback oid 27 current snap is 67 2024-04-07T16:18:56.018 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 27 to 53 2024-04-07T16:18:56.018 INFO:tasks.rados.rados.0.smithi100.stdout:780: snap_remove snap 66 2024-04-07T16:18:56.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:56 smithi100 bash[31660]: cluster 2024-04-07T16:18:54.651845+0000 mgr.x (mgr.24553) 43 : cluster 0 pgmap v54: 121 pgs: 121 active+clean; 155 MiB data, 761 MiB used, 714 GiB / 715 GiB avail; 7.7 MiB/s rd, 4.6 MiB/s wr, 20 op/s 2024-04-07T16:18:56.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:56 smithi100 bash[31660]: cluster 2024-04-07T16:18:55.028037+0000 mon.a (mon.0) 1082 : cluster 0 osdmap e201: 8 total, 8 up, 8 in 2024-04-07T16:18:56.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:56 smithi100 bash[21346]: cluster 2024-04-07T16:18:54.651845+0000 mgr.x (mgr.24553) 43 : cluster 0 pgmap v54: 121 pgs: 121 active+clean; 155 MiB data, 761 MiB used, 714 GiB / 715 GiB avail; 7.7 MiB/s rd, 4.6 MiB/s wr, 20 op/s 2024-04-07T16:18:56.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:56 smithi100 bash[21346]: cluster 2024-04-07T16:18:55.028037+0000 mon.a (mon.0) 1082 : cluster 0 osdmap e201: 8 total, 8 up, 8 in 2024-04-07T16:18:56.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:56 smithi144 bash[23522]: cluster 2024-04-07T16:18:54.651845+0000 mgr.x (mgr.24553) 43 : cluster 0 pgmap v54: 121 pgs: 121 active+clean; 155 MiB data, 761 MiB used, 714 GiB / 715 GiB avail; 7.7 MiB/s rd, 4.6 MiB/s wr, 20 op/s 2024-04-07T16:18:56.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:56 smithi144 bash[23522]: cluster 2024-04-07T16:18:55.028037+0000 mon.a (mon.0) 1082 : cluster 0 osdmap e201: 8 total, 8 up, 8 in 2024-04-07T16:18:56.741 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:18:56 smithi100 bash[68020]: ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4-mgr-y 2024-04-07T16:18:56.741 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:18:56 smithi100 systemd[1]: ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@mgr.y.service: Main process exited, code=exited, status=143/n/a 2024-04-07T16:18:56.741 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:18:56 smithi100 bash[68080]: Error response from daemon: No such container: ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4-mgr-y 2024-04-07T16:18:56.741 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:18:56 smithi100 systemd[1]: ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@mgr.y.service: Failed with result 'exit-code'. 2024-04-07T16:18:56.741 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:18:56 smithi100 systemd[1]: Stopped Ceph mgr.y for 0702e3a8-f4f8-11ee-b649-cb9ed24678a4. 2024-04-07T16:18:57.028 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:56 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:18:57.028 INFO:journalctl@ceph.osd.2.smithi100.stdout:Apr 07 16:18:56 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:18:57.029 INFO:journalctl@ceph.osd.3.smithi100.stdout:Apr 07 16:18:56 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:18:57.029 INFO:journalctl@ceph.osd.1.smithi100.stdout:Apr 07 16:18:56 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:18:57.029 INFO:journalctl@ceph.osd.0.smithi100.stdout:Apr 07 16:18:56 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:18:57.029 INFO:journalctl@ceph.mds.a.smithi100.stdout:Apr 07 16:18:56 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:18:57.030 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Apr 07 16:18:56 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:18:57.030 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:18:56 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:18:57.030 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:18:56 smithi100 systemd[1]: Started Ceph mgr.y for 0702e3a8-f4f8-11ee-b649-cb9ed24678a4. 2024-04-07T16:18:57.030 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 16:18:56 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:18:57.031 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:56 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:18:57.032 INFO:tasks.rados.rados.0.smithi100.stdout:780: done (1 left) 2024-04-07T16:18:57.032 INFO:tasks.rados.rados.0.smithi100.stdout:781: copy_from oid 11 from oid 41 current snap is 67 2024-04-07T16:18:57.033 INFO:tasks.rados.rados.0.smithi100.stdout:779: finishing rollback tid 0 to smithi10066867-27 2024-04-07T16:18:57.033 INFO:tasks.rados.rados.0.smithi100.stdout:779: finishing rollback tid 1 to smithi10066867-27 2024-04-07T16:18:57.033 INFO:tasks.rados.rados.0.smithi100.stdout:779: finishing rollback tid 2 to smithi10066867-27 2024-04-07T16:18:57.033 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 27 v 238 (ObjNum 200 snap 53 seq_num 200) dirty exists 2024-04-07T16:18:57.033 INFO:tasks.rados.rados.0.smithi100.stdout:779: done (1 left) 2024-04-07T16:18:57.033 INFO:tasks.rados.rados.0.smithi100.stdout:782: copy_from oid 46 from oid 17 current snap is 67 2024-04-07T16:18:57.033 INFO:tasks.rados.rados.0.smithi100.stdout:783: delete oid 29 current snap is 67 2024-04-07T16:18:57.040 INFO:tasks.rados.rados.0.smithi100.stdout:783: done (2 left) 2024-04-07T16:18:57.040 INFO:tasks.rados.rados.0.smithi100.stdout:784: snap_remove snap 62 2024-04-07T16:18:57.354 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:57 smithi144 bash[23522]: cluster 2024-04-07T16:18:56.030036+0000 mon.a (mon.0) 1083 : cluster 0 osdmap e202: 8 total, 8 up, 8 in 2024-04-07T16:18:57.436 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:57 smithi100 bash[21346]: cluster 2024-04-07T16:18:56.030036+0000 mon.a (mon.0) 1083 : cluster 0 osdmap e202: 8 total, 8 up, 8 in 2024-04-07T16:18:57.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:57 smithi100 bash[31660]: cluster 2024-04-07T16:18:56.030036+0000 mon.a (mon.0) 1083 : cluster 0 osdmap e202: 8 total, 8 up, 8 in 2024-04-07T16:18:58.049 INFO:tasks.rados.rados.0.smithi100.stdout:784: done (2 left) 2024-04-07T16:18:58.049 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 12 current snap is 67 2024-04-07T16:18:58.049 INFO:tasks.rados.rados.0.smithi100.stdout:781: finishing copy_from racing read to smithi10066867-11 2024-04-07T16:18:58.049 INFO:tasks.rados.rados.0.smithi100.stdout:781: finishing copy_from to smithi10066867-11 2024-04-07T16:18:58.049 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 11 v 343 (ObjNum 45 snap 0 seq_num 45) dirty exists 2024-04-07T16:18:58.049 INFO:tasks.rados.rados.0.smithi100.stdout:782: finishing copy_from racing read to smithi10066867-46 2024-04-07T16:18:58.049 INFO:tasks.rados.rados.0.smithi100.stdout:782: finishing copy_from to smithi10066867-46 2024-04-07T16:18:58.049 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 46 v 240 (ObjNum 225 snap 59 seq_num 225) dirty exists 2024-04-07T16:18:58.050 INFO:tasks.rados.rados.0.smithi100.stdout:785: seq_num 248 ranges {1695744=557056,2252800=507904,2760704=442368,3203072=253952} 2024-04-07T16:18:58.079 INFO:tasks.rados.rados.0.smithi100.stdout:785: writing smithi10066867-12 from 1695744 to 2252800 tid 1 2024-04-07T16:18:58.085 INFO:tasks.rados.rados.0.smithi100.stdout:785: writing smithi10066867-12 from 2252800 to 2760704 tid 2 2024-04-07T16:18:58.090 INFO:tasks.rados.rados.0.smithi100.stdout:785: writing smithi10066867-12 from 2760704 to 3203072 tid 3 2024-04-07T16:18:58.093 INFO:tasks.rados.rados.0.smithi100.stdout:785: writing smithi10066867-12 from 3203072 to 3457024 tid 4 2024-04-07T16:18:58.093 INFO:tasks.rados.rados.0.smithi100.stdout:781: done (2 left) 2024-04-07T16:18:58.093 INFO:tasks.rados.rados.0.smithi100.stdout:782: done (1 left) 2024-04-07T16:18:58.093 INFO:tasks.rados.rados.0.smithi100.stdout:786: delete oid 3 current snap is 67 2024-04-07T16:18:58.093 INFO:tasks.rados.rados.0.smithi100.stdout:785: finishing write tid 1 to smithi10066867-12 2024-04-07T16:18:58.096 INFO:tasks.rados.rados.0.smithi100.stdout:785: finishing write tid 2 to smithi10066867-12 2024-04-07T16:18:58.096 INFO:tasks.rados.rados.0.smithi100.stdout:785: finishing write tid 3 to smithi10066867-12 2024-04-07T16:18:58.100 INFO:tasks.rados.rados.0.smithi100.stdout:785: finishing write tid 4 to smithi10066867-12 2024-04-07T16:18:58.100 INFO:tasks.rados.rados.0.smithi100.stdout:785: finishing write tid 5 to smithi10066867-12 2024-04-07T16:18:58.100 INFO:tasks.rados.rados.0.smithi100.stdout:786: done (1 left) 2024-04-07T16:18:58.100 INFO:tasks.rados.rados.0.smithi100.stdout:787: snap_remove snap 58 2024-04-07T16:18:58.196 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:58 smithi100 bash[21346]: cluster 2024-04-07T16:18:56.652620+0000 mgr.x (mgr.24553) 44 : cluster 0 pgmap v57: 121 pgs: 121 active+clean; 155 MiB data, 761 MiB used, 714 GiB / 715 GiB avail 2024-04-07T16:18:58.196 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:58 smithi100 bash[21346]: cluster 2024-04-07T16:18:57.048388+0000 mon.a (mon.0) 1084 : cluster 0 osdmap e203: 8 total, 8 up, 8 in 2024-04-07T16:18:58.196 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:58 smithi100 bash[21346]: audit 2024-04-07T16:18:57.064014+0000 mon.a (mon.0) 1085 : audit 1 from='mgr.24553 ' entity='mgr.x' 2024-04-07T16:18:58.196 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:58 smithi100 bash[21346]: audit 2024-04-07T16:18:57.080135+0000 mon.a (mon.0) 1086 : audit 1 from='mgr.24553 ' entity='mgr.x' 2024-04-07T16:18:58.196 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:58 smithi100 bash[21346]: audit 2024-04-07T16:18:57.101174+0000 mon.b (mon.1) 319 : audit 0 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:18:58.196 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:58 smithi100 bash[31660]: cluster 2024-04-07T16:18:56.652620+0000 mgr.x (mgr.24553) 44 : cluster 0 pgmap v57: 121 pgs: 121 active+clean; 155 MiB data, 761 MiB used, 714 GiB / 715 GiB avail 2024-04-07T16:18:58.197 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:58 smithi100 bash[31660]: cluster 2024-04-07T16:18:57.048388+0000 mon.a (mon.0) 1084 : cluster 0 osdmap e203: 8 total, 8 up, 8 in 2024-04-07T16:18:58.197 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:58 smithi100 bash[31660]: audit 2024-04-07T16:18:57.064014+0000 mon.a (mon.0) 1085 : audit 1 from='mgr.24553 ' entity='mgr.x' 2024-04-07T16:18:58.197 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:58 smithi100 bash[31660]: audit 2024-04-07T16:18:57.080135+0000 mon.a (mon.0) 1086 : audit 1 from='mgr.24553 ' entity='mgr.x' 2024-04-07T16:18:58.197 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:58 smithi100 bash[31660]: audit 2024-04-07T16:18:57.101174+0000 mon.b (mon.1) 319 : audit 0 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:18:58.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:58 smithi144 bash[23522]: cluster 2024-04-07T16:18:56.652620+0000 mgr.x (mgr.24553) 44 : cluster 0 pgmap v57: 121 pgs: 121 active+clean; 155 MiB data, 761 MiB used, 714 GiB / 715 GiB avail 2024-04-07T16:18:58.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:58 smithi144 bash[23522]: cluster 2024-04-07T16:18:57.048388+0000 mon.a (mon.0) 1084 : cluster 0 osdmap e203: 8 total, 8 up, 8 in 2024-04-07T16:18:58.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:58 smithi144 bash[23522]: audit 2024-04-07T16:18:57.064014+0000 mon.a (mon.0) 1085 : audit 1 from='mgr.24553 ' entity='mgr.x' 2024-04-07T16:18:58.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:58 smithi144 bash[23522]: audit 2024-04-07T16:18:57.080135+0000 mon.a (mon.0) 1086 : audit 1 from='mgr.24553 ' entity='mgr.x' 2024-04-07T16:18:58.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:58 smithi144 bash[23522]: audit 2024-04-07T16:18:57.101174+0000 mon.b (mon.1) 319 : audit 0 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:18:58.686 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:18:58 smithi100 bash[68137]: debug 2024-04-07T16:18:58.283+0000 7f0982797700 1 -- 172.21.15.100:0/3926879415 <== mon.2 v2:172.21.15.100:3301/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564f16ba05a0 con 0x564f17920800 2024-04-07T16:18:58.687 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:18:58 smithi100 bash[68137]: debug 2024-04-07T16:18:58.431+0000 7f098c9a7200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-07T16:18:59.062 INFO:tasks.rados.rados.0.smithi100.stdout:787: done (1 left) 2024-04-07T16:18:59.062 INFO:tasks.rados.rados.0.smithi100.stdout:788: read oid 9 snap -1 2024-04-07T16:18:59.062 INFO:tasks.rados.rados.0.smithi100.stdout:788: expect (ObjNum 206 snap 56 seq_num 206) 2024-04-07T16:18:59.062 INFO:tasks.rados.rados.0.smithi100.stdout:789: snap_remove snap 60 2024-04-07T16:18:59.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:18:59 smithi144 bash[23522]: cluster 2024-04-07T16:18:58.060451+0000 mon.a (mon.0) 1087 : cluster 0 osdmap e204: 8 total, 8 up, 8 in 2024-04-07T16:18:59.436 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:18:59 smithi100 bash[21346]: cluster 2024-04-07T16:18:58.060451+0000 mon.a (mon.0) 1087 : cluster 0 osdmap e204: 8 total, 8 up, 8 in 2024-04-07T16:18:59.436 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:18:59 smithi100 bash[31660]: cluster 2024-04-07T16:18:58.060451+0000 mon.a (mon.0) 1087 : cluster 0 osdmap e204: 8 total, 8 up, 8 in 2024-04-07T16:19:00.057 INFO:tasks.rados.rados.0.smithi100.stdout:789: done (2 left) 2024-04-07T16:19:00.057 INFO:tasks.rados.rados.0.smithi100.stdout:790: read oid 47 snap -1 2024-04-07T16:19:00.057 INFO:tasks.rados.rados.0.smithi100.stdout:790: expect (ObjNum 149 snap 41 seq_num 149) 2024-04-07T16:19:00.057 INFO:tasks.rados.rados.0.smithi100.stdout:791: rollback oid 33 current snap is 67 2024-04-07T16:19:00.057 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 33 to 63 2024-04-07T16:19:00.058 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 35 current snap is 67 2024-04-07T16:19:00.058 INFO:tasks.rados.rados.0.smithi100.stdout:792: seq_num 250 ranges {1212416=589824,1802240=679936,2482176=409600,2891776=114688} 2024-04-07T16:19:00.072 INFO:tasks.rados.rados.0.smithi100.stdout:792: writing smithi10066867-35 from 1212416 to 1802240 tid 1 2024-04-07T16:19:00.082 INFO:tasks.rados.rados.0.smithi100.stdout:792: writing smithi10066867-35 from 1802240 to 2482176 tid 2 2024-04-07T16:19:00.088 INFO:tasks.rados.rados.0.smithi100.stdout:792: writing smithi10066867-35 from 2482176 to 2891776 tid 3 2024-04-07T16:19:00.090 INFO:tasks.rados.rados.0.smithi100.stdout:792: writing smithi10066867-35 from 2891776 to 3006464 tid 4 2024-04-07T16:19:00.090 INFO:tasks.rados.rados.0.smithi100.stdout:793: read oid 25 snap -1 2024-04-07T16:19:00.090 INFO:tasks.rados.rados.0.smithi100.stdout:793: expect deleted 2024-04-07T16:19:00.090 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 32 current snap is 67 2024-04-07T16:19:00.090 INFO:tasks.rados.rados.0.smithi100.stdout:785: finishing write tid 6 to smithi10066867-12 2024-04-07T16:19:00.090 INFO:tasks.rados.rados.0.smithi100.stdout:785: oid 12 updating version 0 to 353 2024-04-07T16:19:00.090 INFO:tasks.rados.rados.0.smithi100.stdout:785: oid 12 updating version 353 to 355 2024-04-07T16:19:00.090 INFO:tasks.rados.rados.0.smithi100.stdout:785: oid 12 updating version 355 to 357 2024-04-07T16:19:00.091 INFO:tasks.rados.rados.0.smithi100.stdout:785: oid 12 version 357 is already newer than 354 2024-04-07T16:19:00.091 INFO:tasks.rados.rados.0.smithi100.stdout:785: oid 12 version 357 is already newer than 356 2024-04-07T16:19:00.091 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 12 v 357 (ObjNum 248 snap 67 seq_num 248) dirty exists 2024-04-07T16:19:00.091 INFO:tasks.rados.rados.0.smithi100.stdout:785: left oid 12 (ObjNum 248 snap 67 seq_num 248) 2024-04-07T16:19:00.091 INFO:tasks.rados.rados.0.smithi100.stdout:788: expect (ObjNum 206 snap 56 seq_num 206) 2024-04-07T16:19:00.134 INFO:tasks.rados.rados.0.smithi100.stdout:791: finishing rollback tid 0 to smithi10066867-33 2024-04-07T16:19:00.134 INFO:tasks.rados.rados.0.smithi100.stdout:794: seq_num 251 ranges {974848=606208,1581056=614400,2195456=589824} 2024-04-07T16:19:00.142 INFO:tasks.rados.rados.0.smithi100.stdout:794: writing smithi10066867-32 from 974848 to 1581056 tid 1 2024-04-07T16:19:00.146 INFO:tasks.rados.rados.0.smithi100.stdout:794: writing smithi10066867-32 from 1581056 to 2195456 tid 2 2024-04-07T16:19:00.149 INFO:tasks.rados.rados.0.smithi100.stdout:794: writing smithi10066867-32 from 2195456 to 2785280 tid 3 2024-04-07T16:19:00.149 INFO:tasks.rados.rados.0.smithi100.stdout:785: done (6 left) 2024-04-07T16:19:00.149 INFO:tasks.rados.rados.0.smithi100.stdout:788: done (5 left) 2024-04-07T16:19:00.149 INFO:tasks.rados.rados.0.smithi100.stdout:795: rollback oid 42 current snap is 67 2024-04-07T16:19:00.150 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 42 to 48 2024-04-07T16:19:00.150 INFO:tasks.rados.rados.0.smithi100.stdout:790: expect (ObjNum 149 snap 41 seq_num 149) 2024-04-07T16:19:00.153 INFO:tasks.rados.rados.0.smithi100.stdout:791: finishing rollback tid 1 to smithi10066867-33 2024-04-07T16:19:00.153 INFO:tasks.rados.rados.0.smithi100.stdout:791: finishing rollback tid 2 to smithi10066867-33 2024-04-07T16:19:00.153 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 33 v 475 (ObjNum 161 snap 43 seq_num 161) dirty exists 2024-04-07T16:19:00.153 INFO:tasks.rados.rados.0.smithi100.stdout:792: finishing write tid 1 to smithi10066867-35 2024-04-07T16:19:00.153 INFO:tasks.rados.rados.0.smithi100.stdout:790: done (5 left) 2024-04-07T16:19:00.153 INFO:tasks.rados.rados.0.smithi100.stdout:791: done (4 left) 2024-04-07T16:19:00.153 INFO:tasks.rados.rados.0.smithi100.stdout:793: done (3 left) 2024-04-07T16:19:00.153 INFO:tasks.rados.rados.0.smithi100.stdout:796: read oid 28 snap -1 2024-04-07T16:19:00.153 INFO:tasks.rados.rados.0.smithi100.stdout:796: expect (ObjNum 246 snap 66 seq_num 246) 2024-04-07T16:19:00.153 INFO:tasks.rados.rados.0.smithi100.stdout:797: delete oid 36 current snap is 67 2024-04-07T16:19:00.153 INFO:tasks.rados.rados.0.smithi100.stdout:792: finishing write tid 2 to smithi10066867-35 2024-04-07T16:19:00.154 INFO:tasks.rados.rados.0.smithi100.stdout:792: finishing write tid 3 to smithi10066867-35 2024-04-07T16:19:00.154 INFO:tasks.rados.rados.0.smithi100.stdout:792: finishing write tid 4 to smithi10066867-35 2024-04-07T16:19:00.154 INFO:tasks.rados.rados.0.smithi100.stdout:792: finishing write tid 5 to smithi10066867-35 2024-04-07T16:19:00.154 INFO:tasks.rados.rados.0.smithi100.stdout:792: finishing write tid 6 to smithi10066867-35 2024-04-07T16:19:00.154 INFO:tasks.rados.rados.0.smithi100.stdout:792: oid 35 updating version 0 to 240 2024-04-07T16:19:00.154 INFO:tasks.rados.rados.0.smithi100.stdout:792: oid 35 updating version 240 to 241 2024-04-07T16:19:00.154 INFO:tasks.rados.rados.0.smithi100.stdout:792: oid 35 updating version 241 to 242 2024-04-07T16:19:00.154 INFO:tasks.rados.rados.0.smithi100.stdout:792: oid 35 updating version 242 to 243 2024-04-07T16:19:00.154 INFO:tasks.rados.rados.0.smithi100.stdout:792: oid 35 version 243 is already newer than 239 2024-04-07T16:19:00.154 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 35 v 243 (ObjNum 250 snap 67 seq_num 250) dirty exists 2024-04-07T16:19:00.154 INFO:tasks.rados.rados.0.smithi100.stdout:792: left oid 35 (ObjNum 250 snap 67 seq_num 250) 2024-04-07T16:19:00.154 INFO:tasks.rados.rados.0.smithi100.stdout:794: finishing write tid 1 to smithi10066867-32 2024-04-07T16:19:00.154 INFO:tasks.rados.rados.0.smithi100.stdout:794: finishing write tid 2 to smithi10066867-32 2024-04-07T16:19:00.154 INFO:tasks.rados.rados.0.smithi100.stdout:794: finishing write tid 3 to smithi10066867-32 2024-04-07T16:19:00.155 INFO:tasks.rados.rados.0.smithi100.stdout:794: finishing write tid 4 to smithi10066867-32 2024-04-07T16:19:00.155 INFO:tasks.rados.rados.0.smithi100.stdout:795: finishing rollback tid 0 to smithi10066867-42 2024-04-07T16:19:00.156 INFO:tasks.rados.rados.0.smithi100.stdout:795: finishing rollback tid 1 to smithi10066867-42 2024-04-07T16:19:00.156 INFO:tasks.rados.rados.0.smithi100.stdout:795: finishing rollback tid 2 to smithi10066867-42 2024-04-07T16:19:00.156 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 42 v 445 (ObjNum 188 snap 48 seq_num 188) dirty exists 2024-04-07T16:19:00.156 INFO:tasks.rados.rados.0.smithi100.stdout:792: done (4 left) 2024-04-07T16:19:00.156 INFO:tasks.rados.rados.0.smithi100.stdout:795: done (3 left) 2024-04-07T16:19:00.156 INFO:tasks.rados.rados.0.smithi100.stdout:797: done (2 left) 2024-04-07T16:19:00.156 INFO:tasks.rados.rados.0.smithi100.stdout:798: copy_from oid 19 from oid 47 current snap is 67 2024-04-07T16:19:00.156 INFO:tasks.rados.rados.0.smithi100.stdout:799: read oid 46 snap -1 2024-04-07T16:19:00.156 INFO:tasks.rados.rados.0.smithi100.stdout:799: expect (ObjNum 225 snap 59 seq_num 225) 2024-04-07T16:19:00.156 INFO:tasks.rados.rados.0.smithi100.stdout:800: setattr oid 7 current snap is 67 2024-04-07T16:19:00.157 INFO:tasks.rados.rados.0.smithi100.stdout:801: rollback oid 23 current snap is 67 2024-04-07T16:19:00.157 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 23 to 48 2024-04-07T16:19:00.157 INFO:tasks.rados.rados.0.smithi100.stdout:802: copy_from oid 33 from oid 21 current snap is 67 2024-04-07T16:19:00.157 INFO:tasks.rados.rados.0.smithi100.stdout:803: read oid 15 snap -1 2024-04-07T16:19:00.158 INFO:tasks.rados.rados.0.smithi100.stdout:803: expect (ObjNum 211 snap 57 seq_num 211) 2024-04-07T16:19:00.158 INFO:tasks.rados.rados.0.smithi100.stdout:804: rollback oid 41 current snap is 67 2024-04-07T16:19:00.158 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 41 to 48 2024-04-07T16:19:00.158 INFO:tasks.rados.rados.0.smithi100.stdout:805: snap_create 2024-04-07T16:19:00.158 INFO:tasks.rados.rados.0.smithi100.stdout:798: finishing copy_from racing read to smithi10066867-19 2024-04-07T16:19:00.160 INFO:tasks.rados.rados.0.smithi100.stdout:794: finishing write tid 5 to smithi10066867-32 2024-04-07T16:19:00.160 INFO:tasks.rados.rados.0.smithi100.stdout:794: oid 32 updating version 0 to 94 2024-04-07T16:19:00.160 INFO:tasks.rados.rados.0.smithi100.stdout:794: oid 32 updating version 94 to 95 2024-04-07T16:19:00.160 INFO:tasks.rados.rados.0.smithi100.stdout:794: oid 32 updating version 95 to 96 2024-04-07T16:19:00.160 INFO:tasks.rados.rados.0.smithi100.stdout:794: oid 32 version 96 is already newer than 93 2024-04-07T16:19:00.161 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 32 v 96 (ObjNum 251 snap 67 seq_num 251) dirty exists 2024-04-07T16:19:00.161 INFO:tasks.rados.rados.0.smithi100.stdout:794: left oid 32 (ObjNum 251 snap 67 seq_num 251) 2024-04-07T16:19:00.161 INFO:tasks.rados.rados.0.smithi100.stdout:801: finishing rollback tid 0 to smithi10066867-23 2024-04-07T16:19:00.162 INFO:tasks.rados.rados.0.smithi100.stdout:804: finishing rollback tid 0 to smithi10066867-41 2024-04-07T16:19:00.163 INFO:tasks.rados.rados.0.smithi100.stdout:804: finishing rollback tid 1 to smithi10066867-41 2024-04-07T16:19:00.163 INFO:tasks.rados.rados.0.smithi100.stdout:804: finishing rollback tid 2 to smithi10066867-41 2024-04-07T16:19:00.163 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 41 v 376 (ObjNum 142 snap 39 seq_num 142) dirty exists 2024-04-07T16:19:00.163 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 7 v 239 (ObjNum 239 snap 65 seq_num 239) dirty exists 2024-04-07T16:19:00.163 INFO:tasks.rados.rados.0.smithi100.stdout:798: finishing copy_from to smithi10066867-19 2024-04-07T16:19:00.163 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 19 v 76 (ObjNum 149 snap 41 seq_num 149) dirty exists 2024-04-07T16:19:00.163 INFO:tasks.rados.rados.0.smithi100.stdout:801: finishing rollback tid 1 to smithi10066867-23 2024-04-07T16:19:00.167 INFO:tasks.rados.rados.0.smithi100.stdout:799: expect (ObjNum 225 snap 59 seq_num 225) 2024-04-07T16:19:00.205 INFO:tasks.rados.rados.0.smithi100.stdout:801: finishing rollback tid 2 to smithi10066867-23 2024-04-07T16:19:00.205 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 23 v 480 (ObjNum 80 snap 13 seq_num 80) dirty exists 2024-04-07T16:19:00.205 INFO:tasks.rados.rados.0.smithi100.stdout:796: expect (ObjNum 246 snap 66 seq_num 246) 2024-04-07T16:19:00.349 INFO:tasks.rados.rados.0.smithi100.stdout:803: expect (ObjNum 211 snap 57 seq_num 211) 2024-04-07T16:19:00.354 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:00 smithi144 bash[23522]: cluster 2024-04-07T16:18:58.653353+0000 mgr.x (mgr.24553) 45 : cluster 0 pgmap v60: 121 pgs: 121 active+clean; 175 MiB data, 813 MiB used, 714 GiB / 715 GiB avail; 11 MiB/s rd, 4.7 MiB/s wr, 39 op/s 2024-04-07T16:19:00.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:00 smithi144 bash[23522]: cluster 2024-04-07T16:18:59.076883+0000 mon.a (mon.0) 1088 : cluster 0 osdmap e205: 8 total, 8 up, 8 in 2024-04-07T16:19:00.436 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:00 smithi100 bash[21346]: cluster 2024-04-07T16:18:58.653353+0000 mgr.x (mgr.24553) 45 : cluster 0 pgmap v60: 121 pgs: 121 active+clean; 175 MiB data, 813 MiB used, 714 GiB / 715 GiB avail; 11 MiB/s rd, 4.7 MiB/s wr, 39 op/s 2024-04-07T16:19:00.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:00 smithi100 bash[21346]: cluster 2024-04-07T16:18:59.076883+0000 mon.a (mon.0) 1088 : cluster 0 osdmap e205: 8 total, 8 up, 8 in 2024-04-07T16:19:00.437 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:19:00 smithi100 bash[68137]: debug 2024-04-07T16:19:00.051+0000 7f098c9a7200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-07T16:19:00.437 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:19:00 smithi100 bash[68137]: debug 2024-04-07T16:19:00.187+0000 7f098c9a7200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-07T16:19:00.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:00 smithi100 bash[31660]: cluster 2024-04-07T16:18:58.653353+0000 mgr.x (mgr.24553) 45 : cluster 0 pgmap v60: 121 pgs: 121 active+clean; 175 MiB data, 813 MiB used, 714 GiB / 715 GiB avail; 11 MiB/s rd, 4.7 MiB/s wr, 39 op/s 2024-04-07T16:19:00.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:00 smithi100 bash[31660]: cluster 2024-04-07T16:18:59.076883+0000 mon.a (mon.0) 1088 : cluster 0 osdmap e205: 8 total, 8 up, 8 in 2024-04-07T16:19:00.574 INFO:tasks.rados.rados.0.smithi100.stdout:802: finishing copy_from racing read to smithi10066867-33 2024-04-07T16:19:00.574 INFO:tasks.rados.rados.0.smithi100.stdout:802: finishing copy_from to smithi10066867-33 2024-04-07T16:19:00.575 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 33 v 125 (ObjNum 222 snap 59 seq_num 222) dirty exists 2024-04-07T16:19:00.937 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:19:00 smithi100 bash[68137]: debug 2024-04-07T16:19:00.467+0000 7f098c9a7200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-07T16:19:01.095 INFO:tasks.rados.rados.0.smithi100.stdout:794: done (9 left) 2024-04-07T16:19:01.096 INFO:tasks.rados.rados.0.smithi100.stdout:796: done (8 left) 2024-04-07T16:19:01.096 INFO:tasks.rados.rados.0.smithi100.stdout:798: done (7 left) 2024-04-07T16:19:01.096 INFO:tasks.rados.rados.0.smithi100.stdout:799: done (6 left) 2024-04-07T16:19:01.096 INFO:tasks.rados.rados.0.smithi100.stdout:800: done (5 left) 2024-04-07T16:19:01.096 INFO:tasks.rados.rados.0.smithi100.stdout:801: done (4 left) 2024-04-07T16:19:01.096 INFO:tasks.rados.rados.0.smithi100.stdout:802: done (3 left) 2024-04-07T16:19:01.096 INFO:tasks.rados.rados.0.smithi100.stdout:803: done (2 left) 2024-04-07T16:19:01.096 INFO:tasks.rados.rados.0.smithi100.stdout:804: done (1 left) 2024-04-07T16:19:01.096 INFO:tasks.rados.rados.0.smithi100.stdout:805: done (0 left) 2024-04-07T16:19:01.096 INFO:tasks.rados.rados.0.smithi100.stdout:806: read oid 27 snap -1 2024-04-07T16:19:01.096 INFO:tasks.rados.rados.0.smithi100.stdout:806: expect (ObjNum 200 snap 53 seq_num 200) 2024-04-07T16:19:01.096 INFO:tasks.rados.rados.0.smithi100.stdout:807: snap_create 2024-04-07T16:19:01.104 INFO:tasks.rados.rados.0.smithi100.stdout:806: expect (ObjNum 200 snap 53 seq_num 200) 2024-04-07T16:19:01.354 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:01 smithi144 bash[23522]: cluster 2024-04-07T16:19:00.072473+0000 mon.a (mon.0) 1089 : cluster 0 osdmap e206: 8 total, 8 up, 8 in 2024-04-07T16:19:01.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:01 smithi144 bash[23522]: cluster 2024-04-07T16:19:00.654109+0000 mgr.x (mgr.24553) 46 : cluster 0 pgmap v63: 121 pgs: 121 active+clean; 175 MiB data, 813 MiB used, 714 GiB / 715 GiB avail; 11 MiB/s rd, 4.7 MiB/s wr, 39 op/s 2024-04-07T16:19:01.397 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:19:01 smithi100 bash[68137]: debug 2024-04-07T16:19:01.295+0000 7f098c9a7200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-07T16:19:01.397 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:01 smithi100 bash[31660]: cluster 2024-04-07T16:19:00.072473+0000 mon.a (mon.0) 1089 : cluster 0 osdmap e206: 8 total, 8 up, 8 in 2024-04-07T16:19:01.397 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:01 smithi100 bash[31660]: cluster 2024-04-07T16:19:00.654109+0000 mgr.x (mgr.24553) 46 : cluster 0 pgmap v63: 121 pgs: 121 active+clean; 175 MiB data, 813 MiB used, 714 GiB / 715 GiB avail; 11 MiB/s rd, 4.7 MiB/s wr, 39 op/s 2024-04-07T16:19:01.397 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:01 smithi100 bash[21346]: cluster 2024-04-07T16:19:00.072473+0000 mon.a (mon.0) 1089 : cluster 0 osdmap e206: 8 total, 8 up, 8 in 2024-04-07T16:19:01.397 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:01 smithi100 bash[21346]: cluster 2024-04-07T16:19:00.654109+0000 mgr.x (mgr.24553) 46 : cluster 0 pgmap v63: 121 pgs: 121 active+clean; 175 MiB data, 813 MiB used, 714 GiB / 715 GiB avail; 11 MiB/s rd, 4.7 MiB/s wr, 39 op/s 2024-04-07T16:19:01.686 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:19:01 smithi100 bash[68137]: debug 2024-04-07T16:19:01.407+0000 7f098c9a7200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-07T16:19:01.687 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:19:01 smithi100 bash[68137]: debug 2024-04-07T16:19:01.515+0000 7f098c9a7200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-07T16:19:02.097 INFO:tasks.rados.rados.0.smithi100.stdout:806: done (1 left) 2024-04-07T16:19:02.097 INFO:tasks.rados.rados.0.smithi100.stdout:807: done (0 left) 2024-04-07T16:19:02.097 INFO:tasks.rados.rados.0.smithi100.stdout:808: read oid 9 snap -1 2024-04-07T16:19:02.097 INFO:tasks.rados.rados.0.smithi100.stdout:808: expect (ObjNum 206 snap 56 seq_num 206) 2024-04-07T16:19:02.097 INFO:tasks.rados.rados.0.smithi100.stdout:809: snap_remove snap 48 2024-04-07T16:19:02.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:02 smithi100 bash[21346]: cluster 2024-04-07T16:19:01.109092+0000 mon.a (mon.0) 1090 : cluster 0 osdmap e207: 8 total, 8 up, 8 in 2024-04-07T16:19:02.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:02 smithi100 bash[31660]: cluster 2024-04-07T16:19:01.109092+0000 mon.a (mon.0) 1090 : cluster 0 osdmap e207: 8 total, 8 up, 8 in 2024-04-07T16:19:02.187 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:19:01 smithi100 bash[68137]: debug 2024-04-07T16:19:01.875+0000 7f098c9a7200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-07T16:19:02.187 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:19:01 smithi100 bash[68137]: debug 2024-04-07T16:19:01.991+0000 7f098c9a7200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-07T16:19:02.187 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:19:02 smithi100 bash[68137]: debug 2024-04-07T16:19:02.095+0000 7f098c9a7200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-07T16:19:02.354 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:02 smithi144 bash[23522]: cluster 2024-04-07T16:19:01.109092+0000 mon.a (mon.0) 1090 : cluster 0 osdmap e207: 8 total, 8 up, 8 in 2024-04-07T16:19:02.744 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:19:02 smithi100 bash[68137]: debug 2024-04-07T16:19:02.483+0000 7f098c9a7200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-07T16:19:02.745 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:19:02 smithi100 bash[68137]: debug 2024-04-07T16:19:02.755+0000 7f098c9a7200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-07T16:19:02.858 INFO:tasks.rados.rados.0.smithi100.stdout:809: done (1 left) 2024-04-07T16:19:02.858 INFO:tasks.rados.rados.0.smithi100.stdout:810: copy_from oid 21 from oid 38 current snap is 69 2024-04-07T16:19:02.858 INFO:tasks.rados.rados.0.smithi100.stdout:808: expect (ObjNum 206 snap 56 seq_num 206) 2024-04-07T16:19:02.899 INFO:tasks.rados.rados.0.smithi100.stdout:810: finishing copy_from to smithi10066867-21 2024-04-07T16:19:02.899 INFO:tasks.rados.rados.0.smithi100.stdout:810: got expected ENOENT (src dne) 2024-04-07T16:19:02.900 INFO:tasks.rados.rados.0.smithi100.stdout:810: finishing copy_from racing read to smithi10066867-21 2024-04-07T16:19:02.900 INFO:tasks.rados.rados.0.smithi100.stdout:808: done (1 left) 2024-04-07T16:19:02.900 INFO:tasks.rados.rados.0.smithi100.stdout:810: done (0 left) 2024-04-07T16:19:02.900 INFO:tasks.rados.rados.0.smithi100.stdout:811: rmattr oid 2 current snap is 69 2024-04-07T16:19:02.900 INFO:tasks.rados.rados.0.smithi100.stdout:811: done (0 left) 2024-04-07T16:19:02.900 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 26 current snap is 69 2024-04-07T16:19:02.900 INFO:tasks.rados.rados.0.smithi100.stdout:812: seq_num 253 ranges {0=647168,647168=786432,1433600=417792} 2024-04-07T16:19:02.903 INFO:tasks.rados.rados.0.smithi100.stdout:812: writing smithi10066867-26 from 0 to 647168 tid 1 2024-04-07T16:19:02.908 INFO:tasks.rados.rados.0.smithi100.stdout:812: writing smithi10066867-26 from 647168 to 1433600 tid 2 2024-04-07T16:19:02.911 INFO:tasks.rados.rados.0.smithi100.stdout:812: writing smithi10066867-26 from 1433600 to 1851392 tid 3 2024-04-07T16:19:02.911 INFO:tasks.rados.rados.0.smithi100.stdout:813: read oid 20 snap 64 2024-04-07T16:19:02.911 INFO:tasks.rados.rados.0.smithi100.stdout:813: expect (ObjNum 123 snap 29 seq_num 123) 2024-04-07T16:19:02.911 INFO:tasks.rados.rados.0.smithi100.stdout:814: read oid 25 snap -1 2024-04-07T16:19:02.911 INFO:tasks.rados.rados.0.smithi100.stdout:814: expect deleted 2024-04-07T16:19:02.911 INFO:tasks.rados.rados.0.smithi100.stdout:815: rmattr oid 49 current snap is 69 2024-04-07T16:19:02.912 INFO:tasks.rados.rados.0.smithi100.stdout:815: done (3 left) 2024-04-07T16:19:02.912 INFO:tasks.rados.rados.0.smithi100.stdout:816: rmattr oid 30 current snap is 69 2024-04-07T16:19:02.912 INFO:tasks.rados.rados.0.smithi100.stdout:812: finishing write tid 1 to smithi10066867-26 2024-04-07T16:19:02.912 INFO:tasks.rados.rados.0.smithi100.stdout:812: finishing write tid 2 to smithi10066867-26 2024-04-07T16:19:02.912 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 27 current snap is 69 2024-04-07T16:19:02.912 INFO:tasks.rados.rados.0.smithi100.stdout:817: seq_num 254 ranges {958464=663552,1622016=663552,2285568=540672} 2024-04-07T16:19:02.921 INFO:tasks.rados.rados.0.smithi100.stdout:817: writing smithi10066867-27 from 958464 to 1622016 tid 1 2024-04-07T16:19:02.925 INFO:tasks.rados.rados.0.smithi100.stdout:817: writing smithi10066867-27 from 1622016 to 2285568 tid 2 2024-04-07T16:19:02.929 INFO:tasks.rados.rados.0.smithi100.stdout:817: writing smithi10066867-27 from 2285568 to 2826240 tid 3 2024-04-07T16:19:02.929 INFO:tasks.rados.rados.0.smithi100.stdout:818: rollback oid 34 current snap is 69 2024-04-07T16:19:02.930 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 34 to 53 2024-04-07T16:19:02.930 INFO:tasks.rados.rados.0.smithi100.stdout:812: finishing write tid 3 to smithi10066867-26 2024-04-07T16:19:02.930 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 30 v 446 (ObjNum 106 snap 20 seq_num 106) dirty exists 2024-04-07T16:19:02.930 INFO:tasks.rados.rados.0.smithi100.stdout:812: finishing write tid 4 to smithi10066867-26 2024-04-07T16:19:02.930 INFO:tasks.rados.rados.0.smithi100.stdout:814: done (5 left) 2024-04-07T16:19:02.930 INFO:tasks.rados.rados.0.smithi100.stdout:816: done (4 left) 2024-04-07T16:19:02.930 INFO:tasks.rados.rados.0.smithi100.stdout:819: delete oid 37 current snap is 69 2024-04-07T16:19:02.930 INFO:tasks.rados.rados.0.smithi100.stdout:817: finishing write tid 1 to smithi10066867-27 2024-04-07T16:19:02.930 INFO:tasks.rados.rados.0.smithi100.stdout:817: finishing write tid 2 to smithi10066867-27 2024-04-07T16:19:02.932 INFO:tasks.rados.rados.0.smithi100.stdout:819: done (4 left) 2024-04-07T16:19:02.932 INFO:tasks.rados.rados.0.smithi100.stdout:820: rmattr oid 6 current snap is 69 2024-04-07T16:19:02.932 INFO:tasks.rados.rados.0.smithi100.stdout:818: finishing rollback tid 0 to smithi10066867-34 2024-04-07T16:19:02.932 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 22 current snap is 69 2024-04-07T16:19:02.932 INFO:tasks.rados.rados.0.smithi100.stdout:821: seq_num 256 ranges {1212416=696320,1908736=434176,2342912=770048} 2024-04-07T16:19:02.943 INFO:tasks.rados.rados.0.smithi100.stdout:821: writing smithi10066867-22 from 1212416 to 1908736 tid 1 2024-04-07T16:19:02.945 INFO:tasks.rados.rados.0.smithi100.stdout:821: writing smithi10066867-22 from 1908736 to 2342912 tid 2 2024-04-07T16:19:02.950 INFO:tasks.rados.rados.0.smithi100.stdout:821: writing smithi10066867-22 from 2342912 to 3112960 tid 3 2024-04-07T16:19:02.950 INFO:tasks.rados.rados.0.smithi100.stdout:822: rollback oid 16 current snap is 69 2024-04-07T16:19:02.950 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 16 to 67 2024-04-07T16:19:02.950 INFO:tasks.rados.rados.0.smithi100.stdout:813: expect (ObjNum 123 snap 29 seq_num 123) 2024-04-07T16:19:03.189 INFO:tasks.rados.rados.0.smithi100.stdout:812: finishing write tid 5 to smithi10066867-26 2024-04-07T16:19:03.189 INFO:tasks.rados.rados.0.smithi100.stdout:812: oid 26 updating version 0 to 241 2024-04-07T16:19:03.189 INFO:tasks.rados.rados.0.smithi100.stdout:812: oid 26 version 241 is already newer than 240 2024-04-07T16:19:03.189 INFO:tasks.rados.rados.0.smithi100.stdout:812: oid 26 version 241 is already newer than 239 2024-04-07T16:19:03.190 INFO:tasks.rados.rados.0.smithi100.stdout:812: oid 26 version 241 is already newer than 238 2024-04-07T16:19:03.190 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 26 v 241 (ObjNum 253 snap 69 seq_num 253) dirty exists 2024-04-07T16:19:03.190 INFO:tasks.rados.rados.0.smithi100.stdout:812: left oid 26 (ObjNum 253 snap 69 seq_num 253) 2024-04-07T16:19:03.190 INFO:tasks.rados.rados.0.smithi100.stdout:818: finishing rollback tid 1 to smithi10066867-34 2024-04-07T16:19:03.190 INFO:tasks.rados.rados.0.smithi100.stdout:818: finishing rollback tid 2 to smithi10066867-34 2024-04-07T16:19:03.190 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 34 v 309 (ObjNum 140 snap 39 seq_num 140) dirty exists 2024-04-07T16:19:03.190 INFO:tasks.rados.rados.0.smithi100.stdout:817: finishing write tid 3 to smithi10066867-27 2024-04-07T16:19:03.190 INFO:tasks.rados.rados.0.smithi100.stdout:817: finishing write tid 4 to smithi10066867-27 2024-04-07T16:19:03.190 INFO:tasks.rados.rados.0.smithi100.stdout:817: finishing write tid 5 to smithi10066867-27 2024-04-07T16:19:03.190 INFO:tasks.rados.rados.0.smithi100.stdout:817: oid 27 updating version 0 to 243 2024-04-07T16:19:03.190 INFO:tasks.rados.rados.0.smithi100.stdout:817: oid 27 version 243 is already newer than 242 2024-04-07T16:19:03.190 INFO:tasks.rados.rados.0.smithi100.stdout:817: oid 27 version 243 is already newer than 241 2024-04-07T16:19:03.191 INFO:tasks.rados.rados.0.smithi100.stdout:817: oid 27 version 243 is already newer than 240 2024-04-07T16:19:03.191 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 27 v 243 (ObjNum 254 snap 69 seq_num 254) dirty exists 2024-04-07T16:19:03.191 INFO:tasks.rados.rados.0.smithi100.stdout:817: left oid 27 (ObjNum 254 snap 69 seq_num 254) 2024-04-07T16:19:03.191 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 6 v 361 (ObjNum 162 snap 43 seq_num 162) dirty exists 2024-04-07T16:19:03.191 INFO:tasks.rados.rados.0.smithi100.stdout:821: finishing write tid 1 to smithi10066867-22 2024-04-07T16:19:03.191 INFO:tasks.rados.rados.0.smithi100.stdout:821: finishing write tid 2 to smithi10066867-22 2024-04-07T16:19:03.191 INFO:tasks.rados.rados.0.smithi100.stdout:822: finishing rollback tid 0 to smithi10066867-16 2024-04-07T16:19:03.191 INFO:tasks.rados.rados.0.smithi100.stdout:822: finishing rollback tid 1 to smithi10066867-16 2024-04-07T16:19:03.191 INFO:tasks.rados.rados.0.smithi100.stdout:812: done (6 left) 2024-04-07T16:19:03.191 INFO:tasks.rados.rados.0.smithi100.stdout:813: done (5 left) 2024-04-07T16:19:03.191 INFO:tasks.rados.rados.0.smithi100.stdout:817: done (4 left) 2024-04-07T16:19:03.191 INFO:tasks.rados.rados.0.smithi100.stdout:818: done (3 left) 2024-04-07T16:19:03.191 INFO:tasks.rados.rados.0.smithi100.stdout:820: done (2 left) 2024-04-07T16:19:03.191 INFO:tasks.rados.rados.0.smithi100.stdout:823: read oid 38 snap -1 2024-04-07T16:19:03.191 INFO:tasks.rados.rados.0.smithi100.stdout:823: expect deleted 2024-04-07T16:19:03.192 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 13 current snap is 69 2024-04-07T16:19:03.192 INFO:tasks.rados.rados.0.smithi100.stdout:822: finishing rollback tid 2 to smithi10066867-16 2024-04-07T16:19:03.192 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 16 v 388 (ObjNum 118 snap 29 seq_num 118) dirty exists 2024-04-07T16:19:03.192 INFO:tasks.rados.rados.0.smithi100.stdout:821: finishing write tid 3 to smithi10066867-22 2024-04-07T16:19:03.192 INFO:tasks.rados.rados.0.smithi100.stdout:824: seq_num 257 ranges {3342336=712704,4055040=720896,4775936=483328} 2024-04-07T16:19:03.216 INFO:tasks.rados.rados.0.smithi100.stdout:824: writing smithi10066867-13 from 3342336 to 4055040 tid 1 2024-04-07T16:19:03.221 INFO:tasks.rados.rados.0.smithi100.stdout:824: writing smithi10066867-13 from 4055040 to 4775936 tid 2 2024-04-07T16:19:03.226 INFO:tasks.rados.rados.0.smithi100.stdout:824: writing smithi10066867-13 from 4775936 to 5259264 tid 3 2024-04-07T16:19:03.227 INFO:tasks.rados.rados.0.smithi100.stdout:822: done (3 left) 2024-04-07T16:19:03.227 INFO:tasks.rados.rados.0.smithi100.stdout:825: setattr oid 19 current snap is 69 2024-04-07T16:19:03.227 INFO:tasks.rados.rados.0.smithi100.stdout:821: finishing write tid 4 to smithi10066867-22 2024-04-07T16:19:03.227 INFO:tasks.rados.rados.0.smithi100.stdout:821: finishing write tid 5 to smithi10066867-22 2024-04-07T16:19:03.227 INFO:tasks.rados.rados.0.smithi100.stdout:821: oid 22 updating version 0 to 246 2024-04-07T16:19:03.227 INFO:tasks.rados.rados.0.smithi100.stdout:821: oid 22 version 246 is already newer than 245 2024-04-07T16:19:03.227 INFO:tasks.rados.rados.0.smithi100.stdout:821: oid 22 version 246 is already newer than 244 2024-04-07T16:19:03.227 INFO:tasks.rados.rados.0.smithi100.stdout:821: oid 22 version 246 is already newer than 243 2024-04-07T16:19:03.227 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 22 v 246 (ObjNum 256 snap 69 seq_num 256) dirty exists 2024-04-07T16:19:03.227 INFO:tasks.rados.rados.0.smithi100.stdout:821: left oid 22 (ObjNum 256 snap 69 seq_num 256) 2024-04-07T16:19:03.227 INFO:tasks.rados.rados.0.smithi100.stdout:824: finishing write tid 1 to smithi10066867-13 2024-04-07T16:19:03.228 INFO:tasks.rados.rados.0.smithi100.stdout:824: finishing write tid 2 to smithi10066867-13 2024-04-07T16:19:03.228 INFO:tasks.rados.rados.0.smithi100.stdout:821: done (3 left) 2024-04-07T16:19:03.228 INFO:tasks.rados.rados.0.smithi100.stdout:823: done (2 left) 2024-04-07T16:19:03.229 INFO:tasks.rados.rados.0.smithi100.stdout:826: copy_from oid 41 from oid 48 current snap is 69 2024-04-07T16:19:03.229 INFO:tasks.rados.rados.0.smithi100.stdout:827: read oid 11 snap 68 2024-04-07T16:19:03.229 INFO:tasks.rados.rados.0.smithi100.stdout:827: expect (ObjNum 45 snap 0 seq_num 45) 2024-04-07T16:19:03.229 INFO:tasks.rados.rados.0.smithi100.stdout:828: rmattr oid 24 current snap is 69 2024-04-07T16:19:03.229 INFO:tasks.rados.rados.0.smithi100.stdout:828: done (4 left) 2024-04-07T16:19:03.229 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 45 current snap is 69 2024-04-07T16:19:03.229 INFO:tasks.rados.rados.0.smithi100.stdout:829: seq_num 258 ranges {122880=729088,851968=606208,1458176=598016} 2024-04-07T16:19:03.233 INFO:tasks.rados.rados.0.smithi100.stdout:829: writing smithi10066867-45 from 122880 to 851968 tid 1 2024-04-07T16:19:03.237 INFO:tasks.rados.rados.0.smithi100.stdout:829: writing smithi10066867-45 from 851968 to 1458176 tid 2 2024-04-07T16:19:03.241 INFO:tasks.rados.rados.0.smithi100.stdout:829: writing smithi10066867-45 from 1458176 to 2056192 tid 3 2024-04-07T16:19:03.241 INFO:tasks.rados.rados.0.smithi100.stdout:830: rmattr oid 42 current snap is 69 2024-04-07T16:19:03.241 INFO:tasks.rados.rados.0.smithi100.stdout:830: done (5 left) 2024-04-07T16:19:03.241 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 17 current snap is 69 2024-04-07T16:19:03.241 INFO:tasks.rados.rados.0.smithi100.stdout:824: finishing write tid 3 to smithi10066867-13 2024-04-07T16:19:03.241 INFO:tasks.rados.rados.0.smithi100.stdout:824: finishing write tid 4 to smithi10066867-13 2024-04-07T16:19:03.241 INFO:tasks.rados.rados.0.smithi100.stdout:824: finishing write tid 5 to smithi10066867-13 2024-04-07T16:19:03.242 INFO:tasks.rados.rados.0.smithi100.stdout:824: oid 13 updating version 0 to 314 2024-04-07T16:19:03.242 INFO:tasks.rados.rados.0.smithi100.stdout:824: oid 13 version 314 is already newer than 311 2024-04-07T16:19:03.242 INFO:tasks.rados.rados.0.smithi100.stdout:824: oid 13 version 314 is already newer than 312 2024-04-07T16:19:03.242 INFO:tasks.rados.rados.0.smithi100.stdout:824: oid 13 version 314 is already newer than 313 2024-04-07T16:19:03.242 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 13 v 314 (ObjNum 257 snap 69 seq_num 257) dirty exists 2024-04-07T16:19:03.242 INFO:tasks.rados.rados.0.smithi100.stdout:824: left oid 13 (ObjNum 257 snap 69 seq_num 257) 2024-04-07T16:19:03.242 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 19 v 279 (ObjNum 149 snap 41 seq_num 149) dirty exists 2024-04-07T16:19:03.242 INFO:tasks.rados.rados.0.smithi100.stdout:831: seq_num 259 ranges {1376256=745472,2121728=540672,2662400=581632,3244032=81920} 2024-04-07T16:19:03.252 INFO:tasks.rados.rados.0.smithi100.stdout:831: writing smithi10066867-17 from 1376256 to 2121728 tid 1 2024-04-07T16:19:03.256 INFO:tasks.rados.rados.0.smithi100.stdout:831: writing smithi10066867-17 from 2121728 to 2662400 tid 2 2024-04-07T16:19:03.260 INFO:tasks.rados.rados.0.smithi100.stdout:831: writing smithi10066867-17 from 2662400 to 3244032 tid 3 2024-04-07T16:19:03.260 INFO:tasks.rados.rados.0.smithi100.stdout:831: writing smithi10066867-17 from 3244032 to 3325952 tid 4 2024-04-07T16:19:03.260 INFO:tasks.rados.rados.0.smithi100.stdout:824: done (5 left) 2024-04-07T16:19:03.260 INFO:tasks.rados.rados.0.smithi100.stdout:825: done (4 left) 2024-04-07T16:19:03.260 INFO:tasks.rados.rados.0.smithi100.stdout:832: read oid 1 snap 67 2024-04-07T16:19:03.261 INFO:tasks.rados.rados.0.smithi100.stdout:832: expect (ObjNum 210 snap 57 seq_num 210) 2024-04-07T16:19:03.261 INFO:tasks.rados.rados.0.smithi100.stdout:833: rmattr oid 18 current snap is 69 2024-04-07T16:19:03.261 INFO:tasks.rados.rados.0.smithi100.stdout:833: done (5 left) 2024-04-07T16:19:03.261 INFO:tasks.rados.rados.0.smithi100.stdout:834: rmattr oid 10 current snap is 69 2024-04-07T16:19:03.261 INFO:tasks.rados.rados.0.smithi100.stdout:834: done (5 left) 2024-04-07T16:19:03.261 INFO:tasks.rados.rados.0.smithi100.stdout:835: snap_remove snap 64 2024-04-07T16:19:03.437 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:19:03 smithi100 bash[68137]: debug 2024-04-07T16:19:03.079+0000 7f098c9a7200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-07T16:19:03.437 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:19:03 smithi100 bash[68137]: debug 2024-04-07T16:19:03.203+0000 7f098c9a7200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-07T16:19:03.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:03 smithi100 bash[21346]: cluster 2024-04-07T16:19:02.111661+0000 mon.a (mon.0) 1091 : cluster 0 osdmap e208: 8 total, 8 up, 8 in 2024-04-07T16:19:03.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:03 smithi100 bash[21346]: cluster 2024-04-07T16:19:02.654571+0000 mgr.x (mgr.24553) 47 : cluster 0 pgmap v66: 121 pgs: 121 active+clean; 173 MiB data, 834 MiB used, 714 GiB / 715 GiB avail; 9.1 MiB/s rd, 3.3 MiB/s wr, 20 op/s 2024-04-07T16:19:03.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:03 smithi100 bash[21346]: cluster 2024-04-07T16:19:02.872614+0000 mon.a (mon.0) 1092 : cluster 0 osdmap e209: 8 total, 8 up, 8 in 2024-04-07T16:19:03.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:03 smithi100 bash[21346]: audit 2024-04-07T16:19:02.898260+0000 mon.a (mon.0) 1093 : audit 1 from='mgr.24553 ' entity='mgr.x' 2024-04-07T16:19:03.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:03 smithi100 bash[21346]: audit 2024-04-07T16:19:02.907529+0000 mon.a (mon.0) 1094 : audit 1 from='mgr.24553 ' entity='mgr.x' 2024-04-07T16:19:03.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:03 smithi100 bash[31660]: cluster 2024-04-07T16:19:02.111661+0000 mon.a (mon.0) 1091 : cluster 0 osdmap e208: 8 total, 8 up, 8 in 2024-04-07T16:19:03.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:03 smithi100 bash[31660]: cluster 2024-04-07T16:19:02.654571+0000 mgr.x (mgr.24553) 47 : cluster 0 pgmap v66: 121 pgs: 121 active+clean; 173 MiB data, 834 MiB used, 714 GiB / 715 GiB avail; 9.1 MiB/s rd, 3.3 MiB/s wr, 20 op/s 2024-04-07T16:19:03.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:03 smithi100 bash[31660]: cluster 2024-04-07T16:19:02.872614+0000 mon.a (mon.0) 1092 : cluster 0 osdmap e209: 8 total, 8 up, 8 in 2024-04-07T16:19:03.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:03 smithi100 bash[31660]: audit 2024-04-07T16:19:02.898260+0000 mon.a (mon.0) 1093 : audit 1 from='mgr.24553 ' entity='mgr.x' 2024-04-07T16:19:03.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:03 smithi100 bash[31660]: audit 2024-04-07T16:19:02.907529+0000 mon.a (mon.0) 1094 : audit 1 from='mgr.24553 ' entity='mgr.x' 2024-04-07T16:19:03.499 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:03 smithi144 bash[23522]: cluster 2024-04-07T16:19:02.111661+0000 mon.a (mon.0) 1091 : cluster 0 osdmap e208: 8 total, 8 up, 8 in 2024-04-07T16:19:03.499 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:03 smithi144 bash[23522]: cluster 2024-04-07T16:19:02.654571+0000 mgr.x (mgr.24553) 47 : cluster 0 pgmap v66: 121 pgs: 121 active+clean; 173 MiB data, 834 MiB used, 714 GiB / 715 GiB avail; 9.1 MiB/s rd, 3.3 MiB/s wr, 20 op/s 2024-04-07T16:19:03.499 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:03 smithi144 bash[23522]: cluster 2024-04-07T16:19:02.872614+0000 mon.a (mon.0) 1092 : cluster 0 osdmap e209: 8 total, 8 up, 8 in 2024-04-07T16:19:03.499 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:03 smithi144 bash[23522]: audit 2024-04-07T16:19:02.898260+0000 mon.a (mon.0) 1093 : audit 1 from='mgr.24553 ' entity='mgr.x' 2024-04-07T16:19:03.499 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:03 smithi144 bash[23522]: audit 2024-04-07T16:19:02.907529+0000 mon.a (mon.0) 1094 : audit 1 from='mgr.24553 ' entity='mgr.x' 2024-04-07T16:19:03.855 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:19:03 smithi144 bash[58467]: ts=2024-04-07T16:19:03.510Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=nodes name=CephNodeDiskspaceWarning index=4 msg="Evaluating rule failed" rule="alert: CephNodeDiskspaceWarning\nexpr: predict_linear(node_filesystem_free_bytes{device=~\"/.*\"}[2d], 3600 * 24 * 5)\n * on (instance) group_left (nodename) node_uname_info < 0\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.8.4\n severity: warning\n type: ceph_default\nannotations:\n description: Mountpoint {{ $labels.mountpoint }} on {{ $labels.nodename }} will\n be full in less than 5 days based on the 48 hour trailing fill rate.\n summary: Host filesystem free space is getting low\n" err="found duplicate series for the match group {instance=\"smithi100\"} on the right hand-side of the operation: [{__name__=\"node_uname_info\", domainname=\"(none)\", instance=\"smithi100\", job=\"node\", machine=\"x86_64\", nodename=\"smithi100\", release=\"5.15.0-101-generic\", sysname=\"Linux\", version=\"#111-Ubuntu SMP Tue Mar 5 20:16:58 UTC 2024\"}, {__name__=\"node_uname_info\", cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", domainname=\"(none)\", instance=\"smithi100\", job=\"node\", machine=\"x86_64\", nodename=\"smithi100\", release=\"5.15.0-101-generic\", sysname=\"Linux\", version=\"#111-Ubuntu SMP Tue Mar 5 20:16:58 UTC 2024\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:19:03.929 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:19:03 smithi100 bash[68137]: debug 2024-04-07T16:19:03.507+0000 7f098c9a7200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-07T16:19:03.929 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:19:03 smithi100 bash[68137]: debug 2024-04-07T16:19:03.939+0000 7f098c9a7200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-07T16:19:04.224 INFO:tasks.rados.rados.0.smithi100.stdout:835: done (5 left) 2024-04-07T16:19:04.224 INFO:tasks.rados.rados.0.smithi100.stdout:836: snap_create 2024-04-07T16:19:04.224 INFO:tasks.rados.rados.0.smithi100.stdout:829: finishing write tid 1 to smithi10066867-45 2024-04-07T16:19:04.224 INFO:tasks.rados.rados.0.smithi100.stdout:826: finishing copy_from racing read to smithi10066867-41 2024-04-07T16:19:04.224 INFO:tasks.rados.rados.0.smithi100.stdout:827: expect (ObjNum 45 snap 0 seq_num 45) 2024-04-07T16:19:04.256 INFO:tasks.rados.rados.0.smithi100.stdout:829: finishing write tid 2 to smithi10066867-45 2024-04-07T16:19:04.256 INFO:tasks.rados.rados.0.smithi100.stdout:829: finishing write tid 3 to smithi10066867-45 2024-04-07T16:19:04.256 INFO:tasks.rados.rados.0.smithi100.stdout:826: finishing copy_from to smithi10066867-41 2024-04-07T16:19:04.256 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 41 v 235 (ObjNum 239 snap 65 seq_num 239) dirty exists 2024-04-07T16:19:04.256 INFO:tasks.rados.rados.0.smithi100.stdout:829: finishing write tid 4 to smithi10066867-45 2024-04-07T16:19:04.257 INFO:tasks.rados.rados.0.smithi100.stdout:829: finishing write tid 5 to smithi10066867-45 2024-04-07T16:19:04.257 INFO:tasks.rados.rados.0.smithi100.stdout:829: oid 45 updating version 0 to 219 2024-04-07T16:19:04.257 INFO:tasks.rados.rados.0.smithi100.stdout:829: oid 45 version 219 is already newer than 218 2024-04-07T16:19:04.257 INFO:tasks.rados.rados.0.smithi100.stdout:829: oid 45 version 219 is already newer than 216 2024-04-07T16:19:04.257 INFO:tasks.rados.rados.0.smithi100.stdout:829: oid 45 version 219 is already newer than 217 2024-04-07T16:19:04.257 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 45 v 219 (ObjNum 258 snap 69 seq_num 258) dirty exists 2024-04-07T16:19:04.257 INFO:tasks.rados.rados.0.smithi100.stdout:829: left oid 45 (ObjNum 258 snap 69 seq_num 258) 2024-04-07T16:19:04.257 INFO:tasks.rados.rados.0.smithi100.stdout:831: finishing write tid 1 to smithi10066867-17 2024-04-07T16:19:04.257 INFO:tasks.rados.rados.0.smithi100.stdout:831: finishing write tid 2 to smithi10066867-17 2024-04-07T16:19:04.257 INFO:tasks.rados.rados.0.smithi100.stdout:831: finishing write tid 3 to smithi10066867-17 2024-04-07T16:19:04.257 INFO:tasks.rados.rados.0.smithi100.stdout:831: finishing write tid 4 to smithi10066867-17 2024-04-07T16:19:04.257 INFO:tasks.rados.rados.0.smithi100.stdout:831: finishing write tid 5 to smithi10066867-17 2024-04-07T16:19:04.257 INFO:tasks.rados.rados.0.smithi100.stdout:831: finishing write tid 6 to smithi10066867-17 2024-04-07T16:19:04.257 INFO:tasks.rados.rados.0.smithi100.stdout:831: oid 17 updating version 0 to 282 2024-04-07T16:19:04.258 INFO:tasks.rados.rados.0.smithi100.stdout:831: oid 17 updating version 282 to 285 2024-04-07T16:19:04.258 INFO:tasks.rados.rados.0.smithi100.stdout:831: oid 17 version 285 is already newer than 284 2024-04-07T16:19:04.258 INFO:tasks.rados.rados.0.smithi100.stdout:831: oid 17 version 285 is already newer than 283 2024-04-07T16:19:04.258 INFO:tasks.rados.rados.0.smithi100.stdout:831: oid 17 version 285 is already newer than 281 2024-04-07T16:19:04.258 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 17 v 285 (ObjNum 259 snap 69 seq_num 259) dirty exists 2024-04-07T16:19:04.258 INFO:tasks.rados.rados.0.smithi100.stdout:831: left oid 17 (ObjNum 259 snap 69 seq_num 259) 2024-04-07T16:19:04.258 INFO:tasks.rados.rados.0.smithi100.stdout:832: expect (ObjNum 210 snap 57 seq_num 210) 2024-04-07T16:19:04.436 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:19:04 smithi100 bash[68137]: debug 2024-04-07T16:19:04.191+0000 7f098c9a7200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-07T16:19:04.437 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:19:04 smithi100 bash[68137]: debug 2024-04-07T16:19:04.335+0000 7f098c9a7200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-07T16:19:04.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:04 smithi100 bash[21346]: audit 2024-04-07T16:19:03.208783+0000 mon.a (mon.0) 1095 : audit 1 from='mgr.24553 ' entity='mgr.x' 2024-04-07T16:19:04.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:04 smithi100 bash[21346]: audit 2024-04-07T16:19:03.223018+0000 mon.a (mon.0) 1096 : audit 1 from='mgr.24553 ' entity='mgr.x' 2024-04-07T16:19:04.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:04 smithi100 bash[21346]: audit 2024-04-07T16:19:04.059849+0000 mon.a (mon.0) 1097 : audit 1 from='mgr.24553 ' entity='mgr.x' 2024-04-07T16:19:04.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:04 smithi100 bash[21346]: audit 2024-04-07T16:19:04.075534+0000 mon.a (mon.0) 1098 : audit 1 from='mgr.24553 ' entity='mgr.x' 2024-04-07T16:19:04.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:04 smithi100 bash[31660]: audit 2024-04-07T16:19:03.208783+0000 mon.a (mon.0) 1095 : audit 1 from='mgr.24553 ' entity='mgr.x' 2024-04-07T16:19:04.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:04 smithi100 bash[31660]: audit 2024-04-07T16:19:03.223018+0000 mon.a (mon.0) 1096 : audit 1 from='mgr.24553 ' entity='mgr.x' 2024-04-07T16:19:04.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:04 smithi100 bash[31660]: audit 2024-04-07T16:19:04.059849+0000 mon.a (mon.0) 1097 : audit 1 from='mgr.24553 ' entity='mgr.x' 2024-04-07T16:19:04.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:04 smithi100 bash[31660]: audit 2024-04-07T16:19:04.075534+0000 mon.a (mon.0) 1098 : audit 1 from='mgr.24553 ' entity='mgr.x' 2024-04-07T16:19:04.605 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:04 smithi144 bash[23522]: audit 2024-04-07T16:19:03.208783+0000 mon.a (mon.0) 1095 : audit 1 from='mgr.24553 ' entity='mgr.x' 2024-04-07T16:19:04.605 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:04 smithi144 bash[23522]: audit 2024-04-07T16:19:03.223018+0000 mon.a (mon.0) 1096 : audit 1 from='mgr.24553 ' entity='mgr.x' 2024-04-07T16:19:04.605 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:04 smithi144 bash[23522]: audit 2024-04-07T16:19:04.059849+0000 mon.a (mon.0) 1097 : audit 1 from='mgr.24553 ' entity='mgr.x' 2024-04-07T16:19:04.605 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:04 smithi144 bash[23522]: audit 2024-04-07T16:19:04.075534+0000 mon.a (mon.0) 1098 : audit 1 from='mgr.24553 ' entity='mgr.x' 2024-04-07T16:19:04.936 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:19:04 smithi100 bash[68137]: debug 2024-04-07T16:19:04.611+0000 7f098c9a7200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-07T16:19:05.223 INFO:tasks.rados.rados.0.smithi100.stdout:826: done (5 left) 2024-04-07T16:19:05.223 INFO:tasks.rados.rados.0.smithi100.stdout:827: done (4 left) 2024-04-07T16:19:05.223 INFO:tasks.rados.rados.0.smithi100.stdout:829: done (3 left) 2024-04-07T16:19:05.223 INFO:tasks.rados.rados.0.smithi100.stdout:831: done (2 left) 2024-04-07T16:19:05.224 INFO:tasks.rados.rados.0.smithi100.stdout:832: done (1 left) 2024-04-07T16:19:05.224 INFO:tasks.rados.rados.0.smithi100.stdout:836: done (0 left) 2024-04-07T16:19:05.224 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 18 current snap is 70 2024-04-07T16:19:05.224 INFO:tasks.rados.rados.0.smithi100.stdout:837: seq_num 260 ranges {0=761856,761856=417792,1179648=786432} 2024-04-07T16:19:05.228 INFO:tasks.rados.rados.0.smithi100.stdout:837: writing smithi10066867-18 from 0 to 761856 tid 1 2024-04-07T16:19:05.231 INFO:tasks.rados.rados.0.smithi100.stdout:837: writing smithi10066867-18 from 761856 to 1179648 tid 2 2024-04-07T16:19:05.237 INFO:tasks.rados.rados.0.smithi100.stdout:837: writing smithi10066867-18 from 1179648 to 1966080 tid 3 2024-04-07T16:19:05.237 INFO:tasks.rados.rados.0.smithi100.stdout:838: copy_from oid 5 from oid 33 current snap is 70 2024-04-07T16:19:05.237 INFO:tasks.rados.rados.0.smithi100.stdout:839: snap_remove snap 53 2024-04-07T16:19:05.605 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:05 smithi144 bash[23522]: cluster 2024-04-07T16:19:04.231522+0000 mon.a (mon.0) 1099 : cluster 0 osdmap e210: 8 total, 8 up, 8 in 2024-04-07T16:19:05.605 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:05 smithi144 bash[23522]: cluster 2024-04-07T16:19:04.655326+0000 mgr.x (mgr.24553) 48 : cluster 0 pgmap v69: 121 pgs: 121 active+clean; 173 MiB data, 834 MiB used, 714 GiB / 715 GiB avail; 9.1 MiB/s rd, 3.3 MiB/s wr, 20 op/s 2024-04-07T16:19:05.677 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:05 smithi100 bash[21346]: cluster 2024-04-07T16:19:04.231522+0000 mon.a (mon.0) 1099 : cluster 0 osdmap e210: 8 total, 8 up, 8 in 2024-04-07T16:19:05.677 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:05 smithi100 bash[21346]: cluster 2024-04-07T16:19:04.655326+0000 mgr.x (mgr.24553) 48 : cluster 0 pgmap v69: 121 pgs: 121 active+clean; 173 MiB data, 834 MiB used, 714 GiB / 715 GiB avail; 9.1 MiB/s rd, 3.3 MiB/s wr, 20 op/s 2024-04-07T16:19:05.677 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:05 smithi100 bash[31660]: cluster 2024-04-07T16:19:04.231522+0000 mon.a (mon.0) 1099 : cluster 0 osdmap e210: 8 total, 8 up, 8 in 2024-04-07T16:19:05.677 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:05 smithi100 bash[31660]: cluster 2024-04-07T16:19:04.655326+0000 mgr.x (mgr.24553) 48 : cluster 0 pgmap v69: 121 pgs: 121 active+clean; 173 MiB data, 834 MiB used, 714 GiB / 715 GiB avail; 9.1 MiB/s rd, 3.3 MiB/s wr, 20 op/s 2024-04-07T16:19:05.936 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:19:05 smithi100 bash[68137]: debug 2024-04-07T16:19:05.687+0000 7f098c9a7200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-07T16:19:05.936 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:19:05 smithi100 bash[68137]: debug 2024-04-07T16:19:05.807+0000 7f098c9a7200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-07T16:19:06.236 INFO:tasks.rados.rados.0.smithi100.stdout:837: finishing write tid 1 to smithi10066867-18 2024-04-07T16:19:06.237 INFO:tasks.rados.rados.0.smithi100.stdout:837: finishing write tid 2 to smithi10066867-18 2024-04-07T16:19:06.237 INFO:tasks.rados.rados.0.smithi100.stdout:837: finishing write tid 3 to smithi10066867-18 2024-04-07T16:19:06.237 INFO:tasks.rados.rados.0.smithi100.stdout:837: finishing write tid 4 to smithi10066867-18 2024-04-07T16:19:06.237 INFO:tasks.rados.rados.0.smithi100.stdout:837: finishing write tid 5 to smithi10066867-18 2024-04-07T16:19:06.237 INFO:tasks.rados.rados.0.smithi100.stdout:837: oid 18 updating version 0 to 498 2024-04-07T16:19:06.237 INFO:tasks.rados.rados.0.smithi100.stdout:837: oid 18 updating version 498 to 499 2024-04-07T16:19:06.237 INFO:tasks.rados.rados.0.smithi100.stdout:837: oid 18 version 499 is already newer than 497 2024-04-07T16:19:06.237 INFO:tasks.rados.rados.0.smithi100.stdout:837: oid 18 version 499 is already newer than 496 2024-04-07T16:19:06.237 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 18 v 499 (ObjNum 260 snap 70 seq_num 260) dirty exists 2024-04-07T16:19:06.237 INFO:tasks.rados.rados.0.smithi100.stdout:837: left oid 18 (ObjNum 260 snap 70 seq_num 260) 2024-04-07T16:19:06.237 INFO:tasks.rados.rados.0.smithi100.stdout:838: finishing copy_from racing read to smithi10066867-5 2024-04-07T16:19:06.237 INFO:tasks.rados.rados.0.smithi100.stdout:838: finishing copy_from to smithi10066867-5 2024-04-07T16:19:06.237 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 5 v 125 (ObjNum 222 snap 59 seq_num 222) dirty exists 2024-04-07T16:19:06.237 INFO:tasks.rados.rados.0.smithi100.stdout:837: done (2 left) 2024-04-07T16:19:06.238 INFO:tasks.rados.rados.0.smithi100.stdout:838: done (1 left) 2024-04-07T16:19:06.238 INFO:tasks.rados.rados.0.smithi100.stdout:839: done (0 left) 2024-04-07T16:19:06.238 INFO:tasks.rados.rados.0.smithi100.stdout:840: rollback oid 28 current snap is 70 2024-04-07T16:19:06.238 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 28 to 63 2024-04-07T16:19:06.238 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 42 current snap is 70 2024-04-07T16:19:06.238 INFO:tasks.rados.rados.0.smithi100.stdout:841: seq_num 261 ranges {1703936=778240,2482176=704512,3186688=499712} 2024-04-07T16:19:06.257 INFO:tasks.rados.rados.0.smithi100.stdout:841: writing smithi10066867-42 from 1703936 to 2482176 tid 1 2024-04-07T16:19:06.267 INFO:tasks.rados.rados.0.smithi100.stdout:841: writing smithi10066867-42 from 2482176 to 3186688 tid 2 2024-04-07T16:19:06.274 INFO:tasks.rados.rados.0.smithi100.stdout:841: writing smithi10066867-42 from 3186688 to 3686400 tid 3 2024-04-07T16:19:06.274 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 23 current snap is 70 2024-04-07T16:19:06.275 INFO:tasks.rados.rados.0.smithi100.stdout:840: finishing rollback tid 0 to smithi10066867-28 2024-04-07T16:19:06.275 INFO:tasks.rados.rados.0.smithi100.stdout:840: finishing rollback tid 1 to smithi10066867-28 2024-04-07T16:19:06.275 INFO:tasks.rados.rados.0.smithi100.stdout:840: finishing rollback tid 2 to smithi10066867-28 2024-04-07T16:19:06.275 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 28 v 110 (ObjNum 140 snap 39 seq_num 140) dirty exists 2024-04-07T16:19:06.275 INFO:tasks.rados.rados.0.smithi100.stdout:842: seq_num 262 ranges {1622016=794624,2416640=589824,3006464=614400} 2024-04-07T16:19:06.295 INFO:tasks.rados.rados.0.smithi100.stdout:842: writing smithi10066867-23 from 1622016 to 2416640 tid 1 2024-04-07T16:19:06.303 INFO:tasks.rados.rados.0.smithi100.stdout:842: writing smithi10066867-23 from 2416640 to 3006464 tid 2 2024-04-07T16:19:06.312 INFO:tasks.rados.rados.0.smithi100.stdout:842: writing smithi10066867-23 from 3006464 to 3620864 tid 3 2024-04-07T16:19:06.312 INFO:tasks.rados.rados.0.smithi100.stdout:840: done (2 left) 2024-04-07T16:19:06.312 INFO:tasks.rados.rados.0.smithi100.stdout:843: snap_create 2024-04-07T16:19:06.312 INFO:tasks.rados.rados.0.smithi100.stdout:841: finishing write tid 1 to smithi10066867-42 2024-04-07T16:19:06.312 INFO:tasks.rados.rados.0.smithi100.stdout:841: finishing write tid 2 to smithi10066867-42 2024-04-07T16:19:06.312 INFO:tasks.rados.rados.0.smithi100.stdout:841: finishing write tid 3 to smithi10066867-42 2024-04-07T16:19:06.312 INFO:tasks.rados.rados.0.smithi100.stdout:841: finishing write tid 4 to smithi10066867-42 2024-04-07T16:19:06.313 INFO:tasks.rados.rados.0.smithi100.stdout:841: finishing write tid 5 to smithi10066867-42 2024-04-07T16:19:06.313 INFO:tasks.rados.rados.0.smithi100.stdout:841: oid 42 updating version 0 to 449 2024-04-07T16:19:06.313 INFO:tasks.rados.rados.0.smithi100.stdout:841: oid 42 version 449 is already newer than 448 2024-04-07T16:19:06.313 INFO:tasks.rados.rados.0.smithi100.stdout:841: oid 42 version 449 is already newer than 447 2024-04-07T16:19:06.313 INFO:tasks.rados.rados.0.smithi100.stdout:841: oid 42 updating version 449 to 450 2024-04-07T16:19:06.313 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 42 v 450 (ObjNum 261 snap 70 seq_num 261) dirty exists 2024-04-07T16:19:06.313 INFO:tasks.rados.rados.0.smithi100.stdout:841: left oid 42 (ObjNum 261 snap 70 seq_num 261) 2024-04-07T16:19:06.313 INFO:tasks.rados.rados.0.smithi100.stdout:842: finishing write tid 1 to smithi10066867-23 2024-04-07T16:19:06.313 INFO:tasks.rados.rados.0.smithi100.stdout:842: finishing write tid 2 to smithi10066867-23 2024-04-07T16:19:06.316 INFO:tasks.rados.rados.0.smithi100.stdout:842: finishing write tid 3 to smithi10066867-23 2024-04-07T16:19:06.316 INFO:tasks.rados.rados.0.smithi100.stdout:842: finishing write tid 4 to smithi10066867-23 2024-04-07T16:19:06.317 INFO:tasks.rados.rados.0.smithi100.stdout:842: finishing write tid 5 to smithi10066867-23 2024-04-07T16:19:06.317 INFO:tasks.rados.rados.0.smithi100.stdout:842: oid 23 updating version 0 to 510 2024-04-07T16:19:06.317 INFO:tasks.rados.rados.0.smithi100.stdout:842: oid 23 updating version 510 to 511 2024-04-07T16:19:06.318 INFO:tasks.rados.rados.0.smithi100.stdout:842: oid 23 version 511 is already newer than 509 2024-04-07T16:19:06.318 INFO:tasks.rados.rados.0.smithi100.stdout:842: oid 23 updating version 511 to 512 2024-04-07T16:19:06.318 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 23 v 512 (ObjNum 262 snap 70 seq_num 262) dirty exists 2024-04-07T16:19:06.318 INFO:tasks.rados.rados.0.smithi100.stdout:842: left oid 23 (ObjNum 262 snap 70 seq_num 262) 2024-04-07T16:19:06.354 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:06 smithi144 bash[23522]: cluster 2024-04-07T16:19:05.236242+0000 mon.a (mon.0) 1100 : cluster 0 osdmap e211: 8 total, 8 up, 8 in 2024-04-07T16:19:06.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:06 smithi144 bash[23522]: audit 2024-04-07T16:19:05.766745+0000 mon.b (mon.1) 320 : audit 0 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:19:06.355 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:19:06 smithi144 bash[51952]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:19:06] "GET /metrics HTTP/1.1" 200 36085 "" "Prometheus/2.43.0" 2024-04-07T16:19:06.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:06 smithi100 bash[31660]: cluster 2024-04-07T16:19:05.236242+0000 mon.a (mon.0) 1100 : cluster 0 osdmap e211: 8 total, 8 up, 8 in 2024-04-07T16:19:06.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:06 smithi100 bash[31660]: audit 2024-04-07T16:19:05.766745+0000 mon.b (mon.1) 320 : audit 0 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:19:06.437 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:19:06 smithi100 bash[68137]: debug 2024-04-07T16:19:06.107+0000 7f098c9a7200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-07T16:19:06.437 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:19:06 smithi100 bash[68137]: debug 2024-04-07T16:19:06.223+0000 7f098c9a7200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-07T16:19:06.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:06 smithi100 bash[21346]: cluster 2024-04-07T16:19:05.236242+0000 mon.a (mon.0) 1100 : cluster 0 osdmap e211: 8 total, 8 up, 8 in 2024-04-07T16:19:06.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:06 smithi100 bash[21346]: audit 2024-04-07T16:19:05.766745+0000 mon.b (mon.1) 320 : audit 0 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:19:06.956 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:19:06 smithi100 bash[68137]: debug 2024-04-07T16:19:06.707+0000 7f098c9a7200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-07T16:19:06.956 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:19:06 smithi100 bash[68137]: debug 2024-04-07T16:19:06.831+0000 7f098c9a7200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-07T16:19:06.956 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:19:06 smithi100 bash[68137]: [07/Apr/2024:16:19:06] ENGINE Bus STARTING 2024-04-07T16:19:06.956 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:19:06 smithi100 bash[68137]: CherryPy Checker: 2024-04-07T16:19:06.956 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:19:06 smithi100 bash[68137]: The Application mounted at '' has an empty config. 2024-04-07T16:19:06.957 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:19:06 smithi100 bash[68137]: [07/Apr/2024:16:19:06] ENGINE Serving on http://:::9283 2024-04-07T16:19:06.957 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:19:06 smithi100 bash[68137]: [07/Apr/2024:16:19:06] ENGINE Bus STARTED 2024-04-07T16:19:07.278 INFO:tasks.rados.rados.0.smithi100.stdout:841: done (2 left) 2024-04-07T16:19:07.278 INFO:tasks.rados.rados.0.smithi100.stdout:842: done (1 left) 2024-04-07T16:19:07.278 INFO:tasks.rados.rados.0.smithi100.stdout:843: done (0 left) 2024-04-07T16:19:07.279 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 45 current snap is 71 2024-04-07T16:19:07.279 INFO:tasks.rados.rados.0.smithi100.stdout:844: seq_num 263 ranges {2056192=409600,2465792=475136,2940928=557056,3497984=573440} 2024-04-07T16:19:07.292 INFO:tasks.rados.rados.0.smithi100.stdout:844: writing smithi10066867-45 from 2056192 to 2465792 tid 1 2024-04-07T16:19:07.294 INFO:tasks.rados.rados.0.smithi100.stdout:844: writing smithi10066867-45 from 2465792 to 2940928 tid 2 2024-04-07T16:19:07.299 INFO:tasks.rados.rados.0.smithi100.stdout:844: writing smithi10066867-45 from 2940928 to 3497984 tid 3 2024-04-07T16:19:07.302 INFO:tasks.rados.rados.0.smithi100.stdout:844: writing smithi10066867-45 from 3497984 to 4071424 tid 4 2024-04-07T16:19:07.302 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 10 current snap is 71 2024-04-07T16:19:07.302 INFO:tasks.rados.rados.0.smithi100.stdout:845: seq_num 264 ranges {0=425984,425984=761856,1187840=770048,1957888=73728} 2024-04-07T16:19:07.305 INFO:tasks.rados.rados.0.smithi100.stdout:845: writing smithi10066867-10 from 0 to 425984 tid 1 2024-04-07T16:19:07.309 INFO:tasks.rados.rados.0.smithi100.stdout:845: writing smithi10066867-10 from 425984 to 1187840 tid 2 2024-04-07T16:19:07.314 INFO:tasks.rados.rados.0.smithi100.stdout:845: writing smithi10066867-10 from 1187840 to 1957888 tid 3 2024-04-07T16:19:07.316 INFO:tasks.rados.rados.0.smithi100.stdout:845: writing smithi10066867-10 from 1957888 to 2031616 tid 4 2024-04-07T16:19:07.316 INFO:tasks.rados.rados.0.smithi100.stdout:846: delete oid 7 current snap is 71 2024-04-07T16:19:07.316 INFO:tasks.rados.rados.0.smithi100.stdout:844: finishing write tid 1 to smithi10066867-45 2024-04-07T16:19:07.316 INFO:tasks.rados.rados.0.smithi100.stdout:844: finishing write tid 2 to smithi10066867-45 2024-04-07T16:19:07.316 INFO:tasks.rados.rados.0.smithi100.stdout:844: finishing write tid 3 to smithi10066867-45 2024-04-07T16:19:07.316 INFO:tasks.rados.rados.0.smithi100.stdout:844: finishing write tid 4 to smithi10066867-45 2024-04-07T16:19:07.316 INFO:tasks.rados.rados.0.smithi100.stdout:844: finishing write tid 5 to smithi10066867-45 2024-04-07T16:19:07.316 INFO:tasks.rados.rados.0.smithi100.stdout:844: finishing write tid 6 to smithi10066867-45 2024-04-07T16:19:07.316 INFO:tasks.rados.rados.0.smithi100.stdout:844: oid 45 updating version 0 to 229 2024-04-07T16:19:07.316 INFO:tasks.rados.rados.0.smithi100.stdout:844: oid 45 updating version 229 to 230 2024-04-07T16:19:07.316 INFO:tasks.rados.rados.0.smithi100.stdout:844: oid 45 updating version 230 to 232 2024-04-07T16:19:07.317 INFO:tasks.rados.rados.0.smithi100.stdout:844: oid 45 version 232 is already newer than 231 2024-04-07T16:19:07.317 INFO:tasks.rados.rados.0.smithi100.stdout:844: oid 45 updating version 232 to 233 2024-04-07T16:19:07.317 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 45 v 233 (ObjNum 263 snap 71 seq_num 263) dirty exists 2024-04-07T16:19:07.317 INFO:tasks.rados.rados.0.smithi100.stdout:844: left oid 45 (ObjNum 263 snap 71 seq_num 263) 2024-04-07T16:19:07.317 INFO:tasks.rados.rados.0.smithi100.stdout:845: finishing write tid 1 to smithi10066867-10 2024-04-07T16:19:07.319 INFO:tasks.rados.rados.0.smithi100.stdout:845: finishing write tid 2 to smithi10066867-10 2024-04-07T16:19:07.320 INFO:tasks.rados.rados.0.smithi100.stdout:844: done (2 left) 2024-04-07T16:19:07.320 INFO:tasks.rados.rados.0.smithi100.stdout:846: done (1 left) 2024-04-07T16:19:07.320 INFO:tasks.rados.rados.0.smithi100.stdout:847: rmattr oid 20 current snap is 71 2024-04-07T16:19:07.320 INFO:tasks.rados.rados.0.smithi100.stdout:847: done (1 left) 2024-04-07T16:19:07.320 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 24 current snap is 71 2024-04-07T16:19:07.320 INFO:tasks.rados.rados.0.smithi100.stdout:848: seq_num 266 ranges {6332416=458752,6791168=524288,7315456=729088,8044544=352256} 2024-04-07T16:19:07.357 INFO:tasks.rados.rados.0.smithi100.stdout:848: writing smithi10066867-24 from 6332416 to 6791168 tid 1 2024-04-07T16:19:07.361 INFO:tasks.rados.rados.0.smithi100.stdout:848: writing smithi10066867-24 from 6791168 to 7315456 tid 2 2024-04-07T16:19:07.365 INFO:tasks.rados.rados.0.smithi100.stdout:848: writing smithi10066867-24 from 7315456 to 8044544 tid 3 2024-04-07T16:19:07.368 INFO:tasks.rados.rados.0.smithi100.stdout:848: writing smithi10066867-24 from 8044544 to 8396800 tid 4 2024-04-07T16:19:07.368 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 37 current snap is 71 2024-04-07T16:19:07.368 INFO:tasks.rados.rados.0.smithi100.stdout:845: finishing write tid 3 to smithi10066867-10 2024-04-07T16:19:07.369 INFO:tasks.rados.rados.0.smithi100.stdout:845: finishing write tid 4 to smithi10066867-10 2024-04-07T16:19:07.369 INFO:tasks.rados.rados.0.smithi100.stdout:845: finishing write tid 5 to smithi10066867-10 2024-04-07T16:19:07.369 INFO:tasks.rados.rados.0.smithi100.stdout:845: finishing write tid 6 to smithi10066867-10 2024-04-07T16:19:07.369 INFO:tasks.rados.rados.0.smithi100.stdout:845: oid 10 updating version 0 to 219 2024-04-07T16:19:07.369 INFO:tasks.rados.rados.0.smithi100.stdout:845: oid 10 version 219 is already newer than 218 2024-04-07T16:19:07.369 INFO:tasks.rados.rados.0.smithi100.stdout:845: oid 10 version 219 is already newer than 217 2024-04-07T16:19:07.369 INFO:tasks.rados.rados.0.smithi100.stdout:845: oid 10 version 219 is already newer than 216 2024-04-07T16:19:07.369 INFO:tasks.rados.rados.0.smithi100.stdout:845: oid 10 version 219 is already newer than 215 2024-04-07T16:19:07.369 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 10 v 219 (ObjNum 264 snap 71 seq_num 264) dirty exists 2024-04-07T16:19:07.369 INFO:tasks.rados.rados.0.smithi100.stdout:845: left oid 10 (ObjNum 264 snap 71 seq_num 264) 2024-04-07T16:19:07.369 INFO:tasks.rados.rados.0.smithi100.stdout:848: finishing write tid 1 to smithi10066867-24 2024-04-07T16:19:07.369 INFO:tasks.rados.rados.0.smithi100.stdout:848: finishing write tid 2 to smithi10066867-24 2024-04-07T16:19:07.369 INFO:tasks.rados.rados.0.smithi100.stdout:849: seq_num 267 ranges {0=475136,475136=458752,933888=540672,1474560=606208} 2024-04-07T16:19:07.370 INFO:tasks.rados.rados.0.smithi100.stdout:849: writing smithi10066867-37 from 0 to 475136 tid 1 2024-04-07T16:19:07.374 INFO:tasks.rados.rados.0.smithi100.stdout:849: writing smithi10066867-37 from 475136 to 933888 tid 2 2024-04-07T16:19:07.378 INFO:tasks.rados.rados.0.smithi100.stdout:849: writing smithi10066867-37 from 933888 to 1474560 tid 3 2024-04-07T16:19:07.382 INFO:tasks.rados.rados.0.smithi100.stdout:849: writing smithi10066867-37 from 1474560 to 2080768 tid 4 2024-04-07T16:19:07.382 INFO:tasks.rados.rados.0.smithi100.stdout:845: done (2 left) 2024-04-07T16:19:07.382 INFO:tasks.rados.rados.0.smithi100.stdout:850: delete oid 20 current snap is 71 2024-04-07T16:19:07.382 INFO:tasks.rados.rados.0.smithi100.stdout:848: finishing write tid 3 to smithi10066867-24 2024-04-07T16:19:07.382 INFO:tasks.rados.rados.0.smithi100.stdout:848: finishing write tid 4 to smithi10066867-24 2024-04-07T16:19:07.382 INFO:tasks.rados.rados.0.smithi100.stdout:848: finishing write tid 5 to smithi10066867-24 2024-04-07T16:19:07.382 INFO:tasks.rados.rados.0.smithi100.stdout:849: finishing write tid 1 to smithi10066867-37 2024-04-07T16:19:07.382 INFO:tasks.rados.rados.0.smithi100.stdout:849: finishing write tid 2 to smithi10066867-37 2024-04-07T16:19:07.383 INFO:tasks.rados.rados.0.smithi100.stdout:848: finishing write tid 6 to smithi10066867-24 2024-04-07T16:19:07.383 INFO:tasks.rados.rados.0.smithi100.stdout:848: oid 24 updating version 0 to 299 2024-04-07T16:19:07.383 INFO:tasks.rados.rados.0.smithi100.stdout:848: oid 24 updating version 299 to 301 2024-04-07T16:19:07.383 INFO:tasks.rados.rados.0.smithi100.stdout:848: oid 24 version 301 is already newer than 300 2024-04-07T16:19:07.383 INFO:tasks.rados.rados.0.smithi100.stdout:848: oid 24 version 301 is already newer than 297 2024-04-07T16:19:07.383 INFO:tasks.rados.rados.0.smithi100.stdout:848: oid 24 version 301 is already newer than 298 2024-04-07T16:19:07.383 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 24 v 301 (ObjNum 266 snap 71 seq_num 266) dirty exists 2024-04-07T16:19:07.383 INFO:tasks.rados.rados.0.smithi100.stdout:848: left oid 24 (ObjNum 266 snap 71 seq_num 266) 2024-04-07T16:19:07.386 INFO:tasks.rados.rados.0.smithi100.stdout:849: finishing write tid 3 to smithi10066867-37 2024-04-07T16:19:07.386 INFO:tasks.rados.rados.0.smithi100.stdout:849: finishing write tid 4 to smithi10066867-37 2024-04-07T16:19:07.386 INFO:tasks.rados.rados.0.smithi100.stdout:849: finishing write tid 5 to smithi10066867-37 2024-04-07T16:19:07.389 INFO:tasks.rados.rados.0.smithi100.stdout:848: done (2 left) 2024-04-07T16:19:07.389 INFO:tasks.rados.rados.0.smithi100.stdout:850: done (1 left) 2024-04-07T16:19:07.389 INFO:tasks.rados.rados.0.smithi100.stdout:851: snap_create 2024-04-07T16:19:07.389 INFO:tasks.rados.rados.0.smithi100.stdout:849: finishing write tid 6 to smithi10066867-37 2024-04-07T16:19:07.389 INFO:tasks.rados.rados.0.smithi100.stdout:849: oid 37 updating version 0 to 451 2024-04-07T16:19:07.390 INFO:tasks.rados.rados.0.smithi100.stdout:849: oid 37 updating version 451 to 452 2024-04-07T16:19:07.390 INFO:tasks.rados.rados.0.smithi100.stdout:849: oid 37 updating version 452 to 453 2024-04-07T16:19:07.390 INFO:tasks.rados.rados.0.smithi100.stdout:849: oid 37 updating version 453 to 454 2024-04-07T16:19:07.390 INFO:tasks.rados.rados.0.smithi100.stdout:849: oid 37 updating version 454 to 455 2024-04-07T16:19:07.390 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 37 v 455 (ObjNum 267 snap 71 seq_num 267) dirty exists 2024-04-07T16:19:07.390 INFO:tasks.rados.rados.0.smithi100.stdout:849: left oid 37 (ObjNum 267 snap 71 seq_num 267) 2024-04-07T16:19:07.605 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:07 smithi144 bash[23522]: cluster 2024-04-07T16:19:06.249933+0000 mon.a (mon.0) 1101 : cluster 0 osdmap e212: 8 total, 8 up, 8 in 2024-04-07T16:19:07.605 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:07 smithi144 bash[23522]: cluster 2024-04-07T16:19:06.656064+0000 mgr.x (mgr.24553) 49 : cluster 0 pgmap v72: 121 pgs: 121 active+clean; 173 MiB data, 834 MiB used, 714 GiB / 715 GiB avail 2024-04-07T16:19:07.605 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:07 smithi144 bash[23522]: cluster 2024-04-07T16:19:06.863393+0000 mon.a (mon.0) 1102 : cluster 0 Standby manager daemon y restarted 2024-04-07T16:19:07.605 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:07 smithi144 bash[23522]: cluster 2024-04-07T16:19:06.863674+0000 mon.a (mon.0) 1103 : cluster 0 Standby manager daemon y started 2024-04-07T16:19:07.605 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:07 smithi144 bash[23522]: audit 2024-04-07T16:19:06.867474+0000 mon.c (mon.2) 59 : audit 0 from='mgr.? 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-07T16:19:07.605 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:07 smithi144 bash[23522]: audit 2024-04-07T16:19:06.868410+0000 mon.c (mon.2) 60 : audit 0 from='mgr.? 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-07T16:19:07.605 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:07 smithi144 bash[23522]: audit 2024-04-07T16:19:06.871675+0000 mon.c (mon.2) 61 : audit 0 from='mgr.? 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-07T16:19:07.605 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:07 smithi144 bash[23522]: audit 2024-04-07T16:19:06.872294+0000 mon.c (mon.2) 62 : audit 0 from='mgr.? 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-07T16:19:07.687 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:07 smithi100 bash[21346]: cluster 2024-04-07T16:19:06.249933+0000 mon.a (mon.0) 1101 : cluster 0 osdmap e212: 8 total, 8 up, 8 in 2024-04-07T16:19:07.687 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:07 smithi100 bash[21346]: cluster 2024-04-07T16:19:06.656064+0000 mgr.x (mgr.24553) 49 : cluster 0 pgmap v72: 121 pgs: 121 active+clean; 173 MiB data, 834 MiB used, 714 GiB / 715 GiB avail 2024-04-07T16:19:07.687 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:07 smithi100 bash[21346]: cluster 2024-04-07T16:19:06.863393+0000 mon.a (mon.0) 1102 : cluster 0 Standby manager daemon y restarted 2024-04-07T16:19:07.687 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:07 smithi100 bash[21346]: cluster 2024-04-07T16:19:06.863674+0000 mon.a (mon.0) 1103 : cluster 0 Standby manager daemon y started 2024-04-07T16:19:07.687 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:07 smithi100 bash[21346]: audit 2024-04-07T16:19:06.867474+0000 mon.c (mon.2) 59 : audit 0 from='mgr.? 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-07T16:19:07.687 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:07 smithi100 bash[21346]: audit 2024-04-07T16:19:06.868410+0000 mon.c (mon.2) 60 : audit 0 from='mgr.? 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-07T16:19:07.687 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:07 smithi100 bash[21346]: audit 2024-04-07T16:19:06.871675+0000 mon.c (mon.2) 61 : audit 0 from='mgr.? 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-07T16:19:07.687 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:07 smithi100 bash[21346]: audit 2024-04-07T16:19:06.872294+0000 mon.c (mon.2) 62 : audit 0 from='mgr.? 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-07T16:19:07.688 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:07 smithi100 bash[31660]: cluster 2024-04-07T16:19:06.249933+0000 mon.a (mon.0) 1101 : cluster 0 osdmap e212: 8 total, 8 up, 8 in 2024-04-07T16:19:07.688 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:07 smithi100 bash[31660]: cluster 2024-04-07T16:19:06.656064+0000 mgr.x (mgr.24553) 49 : cluster 0 pgmap v72: 121 pgs: 121 active+clean; 173 MiB data, 834 MiB used, 714 GiB / 715 GiB avail 2024-04-07T16:19:07.688 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:07 smithi100 bash[31660]: cluster 2024-04-07T16:19:06.863393+0000 mon.a (mon.0) 1102 : cluster 0 Standby manager daemon y restarted 2024-04-07T16:19:07.688 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:07 smithi100 bash[31660]: cluster 2024-04-07T16:19:06.863674+0000 mon.a (mon.0) 1103 : cluster 0 Standby manager daemon y started 2024-04-07T16:19:07.688 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:07 smithi100 bash[31660]: audit 2024-04-07T16:19:06.867474+0000 mon.c (mon.2) 59 : audit 0 from='mgr.? 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-07T16:19:07.688 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:07 smithi100 bash[31660]: audit 2024-04-07T16:19:06.868410+0000 mon.c (mon.2) 60 : audit 0 from='mgr.? 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-07T16:19:07.688 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:07 smithi100 bash[31660]: audit 2024-04-07T16:19:06.871675+0000 mon.c (mon.2) 61 : audit 0 from='mgr.? 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-07T16:19:07.688 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:07 smithi100 bash[31660]: audit 2024-04-07T16:19:06.872294+0000 mon.c (mon.2) 62 : audit 0 from='mgr.? 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-07T16:19:08.277 INFO:tasks.rados.rados.0.smithi100.stdout:849: done (1 left) 2024-04-07T16:19:08.277 INFO:tasks.rados.rados.0.smithi100.stdout:851: done (0 left) 2024-04-07T16:19:08.277 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 13 current snap is 72 2024-04-07T16:19:08.277 INFO:tasks.rados.rados.0.smithi100.stdout:852: seq_num 269 ranges {5259264=507904,5767168=630784,6397952=557056,6955008=417792} 2024-04-07T16:19:08.311 INFO:tasks.rados.rados.0.smithi100.stdout:852: writing smithi10066867-13 from 5259264 to 5767168 tid 1 2024-04-07T16:19:08.314 INFO:tasks.rados.rados.0.smithi100.stdout:852: writing smithi10066867-13 from 5767168 to 6397952 tid 2 2024-04-07T16:19:08.318 INFO:tasks.rados.rados.0.smithi100.stdout:852: writing smithi10066867-13 from 6397952 to 6955008 tid 3 2024-04-07T16:19:08.320 INFO:tasks.rados.rados.0.smithi100.stdout:852: writing smithi10066867-13 from 6955008 to 7372800 tid 4 2024-04-07T16:19:08.321 INFO:tasks.rados.rados.0.smithi100.stdout:853: delete oid 50 current snap is 72 2024-04-07T16:19:08.322 INFO:tasks.rados.rados.0.smithi100.stdout:852: finishing write tid 1 to smithi10066867-13 2024-04-07T16:19:08.327 INFO:tasks.rados.rados.0.smithi100.stdout:853: done (1 left) 2024-04-07T16:19:08.327 INFO:tasks.rados.rados.0.smithi100.stdout:854: rollback oid 21 current snap is 72 2024-04-07T16:19:08.327 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 21 to 63 2024-04-07T16:19:08.327 INFO:tasks.rados.rados.0.smithi100.stdout:855: snap_remove snap 67 2024-04-07T16:19:08.605 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:08 smithi144 bash[23522]: cluster 2024-04-07T16:19:07.288502+0000 mon.a (mon.0) 1104 : cluster 0 osdmap e213: 8 total, 8 up, 8 in 2024-04-07T16:19:08.605 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:08 smithi144 bash[23522]: cluster 2024-04-07T16:19:07.290200+0000 mon.a (mon.0) 1105 : cluster 0 mgrmap e29: x(active, since 46s), standbys: y 2024-04-07T16:19:08.686 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:08 smithi100 bash[21346]: cluster 2024-04-07T16:19:07.288502+0000 mon.a (mon.0) 1104 : cluster 0 osdmap e213: 8 total, 8 up, 8 in 2024-04-07T16:19:08.686 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:08 smithi100 bash[21346]: cluster 2024-04-07T16:19:07.290200+0000 mon.a (mon.0) 1105 : cluster 0 mgrmap e29: x(active, since 46s), standbys: y 2024-04-07T16:19:08.687 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:08 smithi100 bash[31660]: cluster 2024-04-07T16:19:07.288502+0000 mon.a (mon.0) 1104 : cluster 0 osdmap e213: 8 total, 8 up, 8 in 2024-04-07T16:19:08.687 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:08 smithi100 bash[31660]: cluster 2024-04-07T16:19:07.290200+0000 mon.a (mon.0) 1105 : cluster 0 mgrmap e29: x(active, since 46s), standbys: y 2024-04-07T16:19:09.359 INFO:tasks.rados.rados.0.smithi100.stdout:855: done (2 left) 2024-04-07T16:19:09.359 INFO:tasks.rados.rados.0.smithi100.stdout:856: snap_create 2024-04-07T16:19:09.359 INFO:tasks.rados.rados.0.smithi100.stdout:852: finishing write tid 2 to smithi10066867-13 2024-04-07T16:19:09.359 INFO:tasks.rados.rados.0.smithi100.stdout:852: finishing write tid 3 to smithi10066867-13 2024-04-07T16:19:09.359 INFO:tasks.rados.rados.0.smithi100.stdout:854: finishing rollback tid 0 to smithi10066867-21 2024-04-07T16:19:09.359 INFO:tasks.rados.rados.0.smithi100.stdout:854: finishing rollback tid 1 to smithi10066867-21 2024-04-07T16:19:09.360 INFO:tasks.rados.rados.0.smithi100.stdout:854: finishing rollback tid 2 to smithi10066867-21 2024-04-07T16:19:09.360 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 21 v 143 (ObjNum 222 snap 59 seq_num 222) dirty exists 2024-04-07T16:19:09.360 INFO:tasks.rados.rados.0.smithi100.stdout:852: finishing write tid 4 to smithi10066867-13 2024-04-07T16:19:09.360 INFO:tasks.rados.rados.0.smithi100.stdout:852: finishing write tid 5 to smithi10066867-13 2024-04-07T16:19:09.360 INFO:tasks.rados.rados.0.smithi100.stdout:852: finishing write tid 6 to smithi10066867-13 2024-04-07T16:19:09.360 INFO:tasks.rados.rados.0.smithi100.stdout:852: oid 13 updating version 0 to 336 2024-04-07T16:19:09.360 INFO:tasks.rados.rados.0.smithi100.stdout:852: oid 13 version 336 is already newer than 332 2024-04-07T16:19:09.360 INFO:tasks.rados.rados.0.smithi100.stdout:852: oid 13 version 336 is already newer than 335 2024-04-07T16:19:09.360 INFO:tasks.rados.rados.0.smithi100.stdout:852: oid 13 version 336 is already newer than 333 2024-04-07T16:19:09.360 INFO:tasks.rados.rados.0.smithi100.stdout:852: oid 13 version 336 is already newer than 334 2024-04-07T16:19:09.360 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 13 v 336 (ObjNum 269 snap 72 seq_num 269) dirty exists 2024-04-07T16:19:09.360 INFO:tasks.rados.rados.0.smithi100.stdout:852: left oid 13 (ObjNum 269 snap 72 seq_num 269) 2024-04-07T16:19:09.605 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:09 smithi144 bash[23522]: cluster 2024-04-07T16:19:08.291693+0000 mon.a (mon.0) 1106 : cluster 0 osdmap e214: 8 total, 8 up, 8 in 2024-04-07T16:19:09.605 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:09 smithi144 bash[23522]: cluster 2024-04-07T16:19:08.656849+0000 mgr.x (mgr.24553) 50 : cluster 0 pgmap v75: 121 pgs: 121 active+clean; 155 MiB data, 916 MiB used, 714 GiB / 715 GiB avail; 6.2 MiB/s rd, 6.1 MiB/s wr, 26 op/s 2024-04-07T16:19:09.686 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:09 smithi100 bash[21346]: cluster 2024-04-07T16:19:08.291693+0000 mon.a (mon.0) 1106 : cluster 0 osdmap e214: 8 total, 8 up, 8 in 2024-04-07T16:19:09.687 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:09 smithi100 bash[21346]: cluster 2024-04-07T16:19:08.656849+0000 mgr.x (mgr.24553) 50 : cluster 0 pgmap v75: 121 pgs: 121 active+clean; 155 MiB data, 916 MiB used, 714 GiB / 715 GiB avail; 6.2 MiB/s rd, 6.1 MiB/s wr, 26 op/s 2024-04-07T16:19:09.687 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:09 smithi100 bash[31660]: cluster 2024-04-07T16:19:08.291693+0000 mon.a (mon.0) 1106 : cluster 0 osdmap e214: 8 total, 8 up, 8 in 2024-04-07T16:19:09.687 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:09 smithi100 bash[31660]: cluster 2024-04-07T16:19:08.656849+0000 mgr.x (mgr.24553) 50 : cluster 0 pgmap v75: 121 pgs: 121 active+clean; 155 MiB data, 916 MiB used, 714 GiB / 715 GiB avail; 6.2 MiB/s rd, 6.1 MiB/s wr, 26 op/s 2024-04-07T16:19:10.309 INFO:tasks.rados.rados.0.smithi100.stdout:852: done (2 left) 2024-04-07T16:19:10.310 INFO:tasks.rados.rados.0.smithi100.stdout:854: done (1 left) 2024-04-07T16:19:10.310 INFO:tasks.rados.rados.0.smithi100.stdout:856: done (0 left) 2024-04-07T16:19:10.310 INFO:tasks.rados.rados.0.smithi100.stdout:857: read oid 46 snap -1 2024-04-07T16:19:10.310 INFO:tasks.rados.rados.0.smithi100.stdout:857: expect (ObjNum 225 snap 59 seq_num 225) 2024-04-07T16:19:10.310 INFO:tasks.rados.rados.0.smithi100.stdout:858: read oid 12 snap -1 2024-04-07T16:19:10.310 INFO:tasks.rados.rados.0.smithi100.stdout:858: expect (ObjNum 248 snap 67 seq_num 248) 2024-04-07T16:19:10.310 INFO:tasks.rados.rados.0.smithi100.stdout:859: copy_from oid 19 from oid 41 current snap is 73 2024-04-07T16:19:10.310 INFO:tasks.rados.rados.0.smithi100.stdout:860: copy_from oid 49 from oid 18 current snap is 73 2024-04-07T16:19:10.310 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 35 current snap is 73 2024-04-07T16:19:10.310 INFO:tasks.rados.rados.0.smithi100.stdout:861: seq_num 271 ranges {3006464=540672,3547136=802816,4349952=516096,4866048=286720} 2024-04-07T16:19:10.330 INFO:tasks.rados.rados.0.smithi100.stdout:861: writing smithi10066867-35 from 3006464 to 3547136 tid 1 2024-04-07T16:19:10.335 INFO:tasks.rados.rados.0.smithi100.stdout:861: writing smithi10066867-35 from 3547136 to 4349952 tid 2 2024-04-07T16:19:10.339 INFO:tasks.rados.rados.0.smithi100.stdout:861: writing smithi10066867-35 from 4349952 to 4866048 tid 3 2024-04-07T16:19:10.341 INFO:tasks.rados.rados.0.smithi100.stdout:861: writing smithi10066867-35 from 4866048 to 5152768 tid 4 2024-04-07T16:19:10.341 INFO:tasks.rados.rados.0.smithi100.stdout:862: rollback oid 15 current snap is 73 2024-04-07T16:19:10.341 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 15 to 71 2024-04-07T16:19:10.344 INFO:tasks.rados.rados.0.smithi100.stdout:859: finishing copy_from racing read to smithi10066867-19 2024-04-07T16:19:10.344 INFO:tasks.rados.rados.0.smithi100.stdout:857: expect (ObjNum 225 snap 59 seq_num 225) 2024-04-07T16:19:10.380 INFO:tasks.rados.rados.0.smithi100.stdout:858: expect (ObjNum 248 snap 67 seq_num 248) 2024-04-07T16:19:10.515 INFO:tasks.rados.rados.0.smithi100.stdout:859: finishing copy_from to smithi10066867-19 2024-04-07T16:19:10.515 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 19 v 235 (ObjNum 239 snap 65 seq_num 239) dirty exists 2024-04-07T16:19:10.515 INFO:tasks.rados.rados.0.smithi100.stdout:861: finishing write tid 1 to smithi10066867-35 2024-04-07T16:19:10.515 INFO:tasks.rados.rados.0.smithi100.stdout:860: finishing copy_from racing read to smithi10066867-49 2024-04-07T16:19:10.515 INFO:tasks.rados.rados.0.smithi100.stdout:860: finishing copy_from to smithi10066867-49 2024-04-07T16:19:10.515 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 49 v 499 (ObjNum 260 snap 70 seq_num 260) dirty exists 2024-04-07T16:19:10.516 INFO:tasks.rados.rados.0.smithi100.stdout:861: finishing write tid 2 to smithi10066867-35 2024-04-07T16:19:10.516 INFO:tasks.rados.rados.0.smithi100.stdout:861: finishing write tid 3 to smithi10066867-35 2024-04-07T16:19:10.516 INFO:tasks.rados.rados.0.smithi100.stdout:861: finishing write tid 4 to smithi10066867-35 2024-04-07T16:19:10.516 INFO:tasks.rados.rados.0.smithi100.stdout:857: done (5 left) 2024-04-07T16:19:10.516 INFO:tasks.rados.rados.0.smithi100.stdout:858: done (4 left) 2024-04-07T16:19:10.516 INFO:tasks.rados.rados.0.smithi100.stdout:859: done (3 left) 2024-04-07T16:19:10.516 INFO:tasks.rados.rados.0.smithi100.stdout:860: done (2 left) 2024-04-07T16:19:10.516 INFO:tasks.rados.rados.0.smithi100.stdout:863: rollback oid 44 current snap is 73 2024-04-07T16:19:10.516 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 44 to 63 2024-04-07T16:19:10.516 INFO:tasks.rados.rados.0.smithi100.stdout:861: finishing write tid 5 to smithi10066867-35 2024-04-07T16:19:10.516 INFO:tasks.rados.rados.0.smithi100.stdout:862: finishing rollback tid 0 to smithi10066867-15 2024-04-07T16:19:10.516 INFO:tasks.rados.rados.0.smithi100.stdout:861: finishing write tid 6 to smithi10066867-35 2024-04-07T16:19:10.516 INFO:tasks.rados.rados.0.smithi100.stdout:861: oid 35 updating version 0 to 244 2024-04-07T16:19:10.517 INFO:tasks.rados.rados.0.smithi100.stdout:861: oid 35 updating version 244 to 247 2024-04-07T16:19:10.517 INFO:tasks.rados.rados.0.smithi100.stdout:861: oid 35 version 247 is already newer than 246 2024-04-07T16:19:10.517 INFO:tasks.rados.rados.0.smithi100.stdout:861: oid 35 version 247 is already newer than 245 2024-04-07T16:19:10.517 INFO:tasks.rados.rados.0.smithi100.stdout:861: oid 35 updating version 247 to 248 2024-04-07T16:19:10.517 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 35 v 248 (ObjNum 271 snap 73 seq_num 271) dirty exists 2024-04-07T16:19:10.517 INFO:tasks.rados.rados.0.smithi100.stdout:861: left oid 35 (ObjNum 271 snap 73 seq_num 271) 2024-04-07T16:19:10.517 INFO:tasks.rados.rados.0.smithi100.stdout:862: finishing rollback tid 1 to smithi10066867-15 2024-04-07T16:19:10.517 INFO:tasks.rados.rados.0.smithi100.stdout:861: done (2 left) 2024-04-07T16:19:10.517 INFO:tasks.rados.rados.0.smithi100.stdout:864: rollback oid 14 current snap is 73 2024-04-07T16:19:10.517 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 14 to 63 2024-04-07T16:19:10.517 INFO:tasks.rados.rados.0.smithi100.stdout:862: finishing rollback tid 2 to smithi10066867-15 2024-04-07T16:19:10.517 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 15 v 518 (ObjNum 211 snap 57 seq_num 211) dirty exists 2024-04-07T16:19:10.517 INFO:tasks.rados.rados.0.smithi100.stdout:862: done (2 left) 2024-04-07T16:19:10.517 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 46 current snap is 73 2024-04-07T16:19:10.517 INFO:tasks.rados.rados.0.smithi100.stdout:865: seq_num 272 ranges {1376256=565248,1941504=688128,2629632=729088,3358720=188416} 2024-04-07T16:19:10.525 INFO:tasks.rados.rados.0.smithi100.stdout:865: writing smithi10066867-46 from 1376256 to 1941504 tid 1 2024-04-07T16:19:10.530 INFO:tasks.rados.rados.0.smithi100.stdout:865: writing smithi10066867-46 from 1941504 to 2629632 tid 2 2024-04-07T16:19:10.534 INFO:tasks.rados.rados.0.smithi100.stdout:865: writing smithi10066867-46 from 2629632 to 3358720 tid 3 2024-04-07T16:19:10.536 INFO:tasks.rados.rados.0.smithi100.stdout:865: writing smithi10066867-46 from 3358720 to 3547136 tid 4 2024-04-07T16:19:10.536 INFO:tasks.rados.rados.0.smithi100.stdout:866: rollback oid 28 current snap is 73 2024-04-07T16:19:10.536 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 28 to 63 2024-04-07T16:19:10.536 INFO:tasks.rados.rados.0.smithi100.stdout:864: finishing rollback tid 1 to smithi10066867-14 2024-04-07T16:19:10.536 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 14 v 0 (ObjNum 353 snap 0 seq_num 3067285040) dirty dne 2024-04-07T16:19:10.536 INFO:tasks.rados.rados.0.smithi100.stdout:863: finishing rollback tid 0 to smithi10066867-44 2024-04-07T16:19:10.536 INFO:tasks.rados.rados.0.smithi100.stdout:863: finishing rollback tid 1 to smithi10066867-44 2024-04-07T16:19:10.536 INFO:tasks.rados.rados.0.smithi100.stdout:863: finishing rollback tid 2 to smithi10066867-44 2024-04-07T16:19:10.536 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 44 v 381 (ObjNum 165 snap 44 seq_num 165) dirty exists 2024-04-07T16:19:10.536 INFO:tasks.rados.rados.0.smithi100.stdout:865: finishing write tid 1 to smithi10066867-46 2024-04-07T16:19:10.536 INFO:tasks.rados.rados.0.smithi100.stdout:863: done (3 left) 2024-04-07T16:19:10.537 INFO:tasks.rados.rados.0.smithi100.stdout:864: done (2 left) 2024-04-07T16:19:10.537 INFO:tasks.rados.rados.0.smithi100.stdout:867: read oid 11 snap -1 2024-04-07T16:19:10.537 INFO:tasks.rados.rados.0.smithi100.stdout:867: expect (ObjNum 45 snap 0 seq_num 45) 2024-04-07T16:19:10.537 INFO:tasks.rados.rados.0.smithi100.stdout:868: snap_create 2024-04-07T16:19:10.537 INFO:tasks.rados.rados.0.smithi100.stdout:865: finishing write tid 2 to smithi10066867-46 2024-04-07T16:19:10.541 INFO:tasks.rados.rados.0.smithi100.stdout:867: expect (ObjNum 45 snap 0 seq_num 45) 2024-04-07T16:19:10.596 INFO:tasks.rados.rados.0.smithi100.stdout:865: finishing write tid 3 to smithi10066867-46 2024-04-07T16:19:10.596 INFO:tasks.rados.rados.0.smithi100.stdout:865: finishing write tid 4 to smithi10066867-46 2024-04-07T16:19:10.596 INFO:tasks.rados.rados.0.smithi100.stdout:866: finishing rollback tid 0 to smithi10066867-28 2024-04-07T16:19:10.596 INFO:tasks.rados.rados.0.smithi100.stdout:866: finishing rollback tid 1 to smithi10066867-28 2024-04-07T16:19:10.596 INFO:tasks.rados.rados.0.smithi100.stdout:866: finishing rollback tid 2 to smithi10066867-28 2024-04-07T16:19:10.596 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 28 v 116 (ObjNum 140 snap 39 seq_num 140) dirty exists 2024-04-07T16:19:10.597 INFO:tasks.rados.rados.0.smithi100.stdout:865: finishing write tid 5 to smithi10066867-46 2024-04-07T16:19:10.597 INFO:tasks.rados.rados.0.smithi100.stdout:865: finishing write tid 6 to smithi10066867-46 2024-04-07T16:19:10.597 INFO:tasks.rados.rados.0.smithi100.stdout:865: oid 46 updating version 0 to 346 2024-04-07T16:19:10.597 INFO:tasks.rados.rados.0.smithi100.stdout:865: oid 46 version 346 is already newer than 344 2024-04-07T16:19:10.597 INFO:tasks.rados.rados.0.smithi100.stdout:865: oid 46 version 346 is already newer than 342 2024-04-07T16:19:10.597 INFO:tasks.rados.rados.0.smithi100.stdout:865: oid 46 version 346 is already newer than 345 2024-04-07T16:19:10.597 INFO:tasks.rados.rados.0.smithi100.stdout:865: oid 46 version 346 is already newer than 343 2024-04-07T16:19:10.597 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 46 v 346 (ObjNum 272 snap 73 seq_num 272) dirty exists 2024-04-07T16:19:10.597 INFO:tasks.rados.rados.0.smithi100.stdout:865: left oid 46 (ObjNum 272 snap 73 seq_num 272) 2024-04-07T16:19:10.605 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:10 smithi144 bash[23522]: cluster 2024-04-07T16:19:09.371657+0000 mon.a (mon.0) 1107 : cluster 0 osdmap e215: 8 total, 8 up, 8 in 2024-04-07T16:19:10.609 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:10 smithi100 bash[21346]: cluster 2024-04-07T16:19:09.371657+0000 mon.a (mon.0) 1107 : cluster 0 osdmap e215: 8 total, 8 up, 8 in 2024-04-07T16:19:10.609 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:10 smithi100 bash[31660]: cluster 2024-04-07T16:19:09.371657+0000 mon.a (mon.0) 1107 : cluster 0 osdmap e215: 8 total, 8 up, 8 in 2024-04-07T16:19:11.319 INFO:tasks.rados.rados.0.smithi100.stdout:865: done (3 left) 2024-04-07T16:19:11.319 INFO:tasks.rados.rados.0.smithi100.stdout:866: done (2 left) 2024-04-07T16:19:11.319 INFO:tasks.rados.rados.0.smithi100.stdout:867: done (1 left) 2024-04-07T16:19:11.319 INFO:tasks.rados.rados.0.smithi100.stdout:868: done (0 left) 2024-04-07T16:19:11.319 INFO:tasks.rados.rados.0.smithi100.stdout:869: snap_remove snap 71 2024-04-07T16:19:11.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:11 smithi100 bash[21346]: cluster 2024-04-07T16:19:10.324781+0000 mon.a (mon.0) 1108 : cluster 0 osdmap e216: 8 total, 8 up, 8 in 2024-04-07T16:19:11.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:11 smithi100 bash[21346]: cluster 2024-04-07T16:19:10.657617+0000 mgr.x (mgr.24553) 51 : cluster 0 pgmap v78: 121 pgs: 121 active+clean; 155 MiB data, 916 MiB used, 714 GiB / 715 GiB avail; 6.2 MiB/s rd, 6.1 MiB/s wr, 26 op/s 2024-04-07T16:19:11.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:11 smithi100 bash[31660]: cluster 2024-04-07T16:19:10.324781+0000 mon.a (mon.0) 1108 : cluster 0 osdmap e216: 8 total, 8 up, 8 in 2024-04-07T16:19:11.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:11 smithi100 bash[31660]: cluster 2024-04-07T16:19:10.657617+0000 mgr.x (mgr.24553) 51 : cluster 0 pgmap v78: 121 pgs: 121 active+clean; 155 MiB data, 916 MiB used, 714 GiB / 715 GiB avail; 6.2 MiB/s rd, 6.1 MiB/s wr, 26 op/s 2024-04-07T16:19:11.605 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:11 smithi144 bash[23522]: cluster 2024-04-07T16:19:10.324781+0000 mon.a (mon.0) 1108 : cluster 0 osdmap e216: 8 total, 8 up, 8 in 2024-04-07T16:19:11.605 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:11 smithi144 bash[23522]: cluster 2024-04-07T16:19:10.657617+0000 mgr.x (mgr.24553) 51 : cluster 0 pgmap v78: 121 pgs: 121 active+clean; 155 MiB data, 916 MiB used, 714 GiB / 715 GiB avail; 6.2 MiB/s rd, 6.1 MiB/s wr, 26 op/s 2024-04-07T16:19:12.326 INFO:tasks.rados.rados.0.smithi100.stdout:869: done (0 left) 2024-04-07T16:19:12.327 INFO:tasks.rados.rados.0.smithi100.stdout:870: rmattr oid 41 current snap is 74 2024-04-07T16:19:12.327 INFO:tasks.rados.rados.0.smithi100.stdout:870: done (0 left) 2024-04-07T16:19:12.327 INFO:tasks.rados.rados.0.smithi100.stdout:871: snap_remove snap 73 2024-04-07T16:19:12.605 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:12 smithi144 bash[23522]: cluster 2024-04-07T16:19:11.333140+0000 mon.a (mon.0) 1109 : cluster 0 osdmap e217: 8 total, 8 up, 8 in 2024-04-07T16:19:12.686 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:12 smithi100 bash[21346]: cluster 2024-04-07T16:19:11.333140+0000 mon.a (mon.0) 1109 : cluster 0 osdmap e217: 8 total, 8 up, 8 in 2024-04-07T16:19:12.687 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:12 smithi100 bash[31660]: cluster 2024-04-07T16:19:11.333140+0000 mon.a (mon.0) 1109 : cluster 0 osdmap e217: 8 total, 8 up, 8 in 2024-04-07T16:19:12.864 INFO:tasks.rados.rados.0.smithi100.stdout:871: done (0 left) 2024-04-07T16:19:12.864 INFO:tasks.rados.rados.0.smithi100.stdout:872: setattr oid 11 current snap is 74 2024-04-07T16:19:12.866 INFO:tasks.rados.rados.0.smithi100.stdout:873: rmattr oid 31 current snap is 74 2024-04-07T16:19:12.866 INFO:tasks.rados.rados.0.smithi100.stdout:873: done (1 left) 2024-04-07T16:19:12.867 INFO:tasks.rados.rados.0.smithi100.stdout:874: rollback oid 42 current snap is 74 2024-04-07T16:19:12.867 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 42 to 63 2024-04-07T16:19:12.867 INFO:tasks.rados.rados.0.smithi100.stdout:875: rollback oid 38 current snap is 74 2024-04-07T16:19:12.867 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 38 to 68 2024-04-07T16:19:12.867 INFO:tasks.rados.rados.0.smithi100.stdout:876: snap_create 2024-04-07T16:19:12.872 INFO:tasks.rados.rados.0.smithi100.stdout:875: finishing rollback tid 1 to smithi10066867-38 2024-04-07T16:19:12.872 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 38 v 0 (ObjNum 273 snap 0 seq_num 0) dirty dne 2024-04-07T16:19:12.877 INFO:tasks.rados.rados.0.smithi100.stdout:874: finishing rollback tid 0 to smithi10066867-42 2024-04-07T16:19:12.879 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 11 v 522 (ObjNum 45 snap 0 seq_num 45) dirty exists 2024-04-07T16:19:12.880 INFO:tasks.rados.rados.0.smithi100.stdout:874: finishing rollback tid 1 to smithi10066867-42 2024-04-07T16:19:12.880 INFO:tasks.rados.rados.0.smithi100.stdout:874: finishing rollback tid 2 to smithi10066867-42 2024-04-07T16:19:12.880 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 42 v 458 (ObjNum 190 snap 49 seq_num 190) dirty exists 2024-04-07T16:19:13.605 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:13 smithi144 bash[23522]: cluster 2024-04-07T16:19:12.339373+0000 mon.a (mon.0) 1110 : cluster 0 osdmap e218: 8 total, 8 up, 8 in 2024-04-07T16:19:13.605 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:13 smithi144 bash[23522]: cluster 2024-04-07T16:19:12.658359+0000 mgr.x (mgr.24553) 52 : cluster 0 pgmap v81: 121 pgs: 121 active+clean; 173 MiB data, 1017 MiB used, 714 GiB / 715 GiB avail; 4.0 MiB/s rd, 4.4 MiB/s wr, 17 op/s 2024-04-07T16:19:13.605 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:13 smithi144 bash[23522]: cluster 2024-04-07T16:19:12.879783+0000 mon.a (mon.0) 1111 : cluster 0 osdmap e219: 8 total, 8 up, 8 in 2024-04-07T16:19:13.605 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:19:13 smithi144 bash[58467]: ts=2024-04-07T16:19:13.510Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=nodes name=CephNodeDiskspaceWarning index=4 msg="Evaluating rule failed" rule="alert: CephNodeDiskspaceWarning\nexpr: predict_linear(node_filesystem_free_bytes{device=~\"/.*\"}[2d], 3600 * 24 * 5)\n * on (instance) group_left (nodename) node_uname_info < 0\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.8.4\n severity: warning\n type: ceph_default\nannotations:\n description: Mountpoint {{ $labels.mountpoint }} on {{ $labels.nodename }} will\n be full in less than 5 days based on the 48 hour trailing fill rate.\n summary: Host filesystem free space is getting low\n" err="found duplicate series for the match group {instance=\"smithi100\"} on the right hand-side of the operation: [{__name__=\"node_uname_info\", domainname=\"(none)\", instance=\"smithi100\", job=\"node\", machine=\"x86_64\", nodename=\"smithi100\", release=\"5.15.0-101-generic\", sysname=\"Linux\", version=\"#111-Ubuntu SMP Tue Mar 5 20:16:58 UTC 2024\"}, {__name__=\"node_uname_info\", cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", domainname=\"(none)\", instance=\"smithi100\", job=\"node\", machine=\"x86_64\", nodename=\"smithi100\", release=\"5.15.0-101-generic\", sysname=\"Linux\", version=\"#111-Ubuntu SMP Tue Mar 5 20:16:58 UTC 2024\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:19:13.686 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:13 smithi100 bash[21346]: cluster 2024-04-07T16:19:12.339373+0000 mon.a (mon.0) 1110 : cluster 0 osdmap e218: 8 total, 8 up, 8 in 2024-04-07T16:19:13.687 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:13 smithi100 bash[21346]: cluster 2024-04-07T16:19:12.658359+0000 mgr.x (mgr.24553) 52 : cluster 0 pgmap v81: 121 pgs: 121 active+clean; 173 MiB data, 1017 MiB used, 714 GiB / 715 GiB avail; 4.0 MiB/s rd, 4.4 MiB/s wr, 17 op/s 2024-04-07T16:19:13.687 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:13 smithi100 bash[21346]: cluster 2024-04-07T16:19:12.879783+0000 mon.a (mon.0) 1111 : cluster 0 osdmap e219: 8 total, 8 up, 8 in 2024-04-07T16:19:13.687 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:13 smithi100 bash[31660]: cluster 2024-04-07T16:19:12.339373+0000 mon.a (mon.0) 1110 : cluster 0 osdmap e218: 8 total, 8 up, 8 in 2024-04-07T16:19:13.687 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:13 smithi100 bash[31660]: cluster 2024-04-07T16:19:12.658359+0000 mgr.x (mgr.24553) 52 : cluster 0 pgmap v81: 121 pgs: 121 active+clean; 173 MiB data, 1017 MiB used, 714 GiB / 715 GiB avail; 4.0 MiB/s rd, 4.4 MiB/s wr, 17 op/s 2024-04-07T16:19:13.687 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:13 smithi100 bash[31660]: cluster 2024-04-07T16:19:12.879783+0000 mon.a (mon.0) 1111 : cluster 0 osdmap e219: 8 total, 8 up, 8 in 2024-04-07T16:19:13.870 INFO:tasks.rados.rados.0.smithi100.stdout:872: done (3 left) 2024-04-07T16:19:13.871 INFO:tasks.rados.rados.0.smithi100.stdout:874: done (2 left) 2024-04-07T16:19:13.871 INFO:tasks.rados.rados.0.smithi100.stdout:875: done (1 left) 2024-04-07T16:19:13.871 INFO:tasks.rados.rados.0.smithi100.stdout:876: done (0 left) 2024-04-07T16:19:13.871 INFO:tasks.rados.rados.0.smithi100.stdout:877: rollback oid 43 current snap is 75 2024-04-07T16:19:13.871 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 43 to 72 2024-04-07T16:19:13.871 INFO:tasks.rados.rados.0.smithi100.stdout:878: snap_create 2024-04-07T16:19:13.880 INFO:tasks.rados.rados.0.smithi100.stdout:877: finishing rollback tid 0 to smithi10066867-43 2024-04-07T16:19:13.883 INFO:tasks.rados.rados.0.smithi100.stdout:877: finishing rollback tid 1 to smithi10066867-43 2024-04-07T16:19:13.883 INFO:tasks.rados.rados.0.smithi100.stdout:877: finishing rollback tid 2 to smithi10066867-43 2024-04-07T16:19:13.884 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 43 v 247 (ObjNum 212 snap 57 seq_num 212) dirty exists 2024-04-07T16:19:13.987 INFO:tasks.rados.rados.0.smithi100.stdout:877: done (1 left) 2024-04-07T16:19:13.987 INFO:tasks.rados.rados.0.smithi100.stdout:878: done (0 left) 2024-04-07T16:19:13.987 INFO:tasks.rados.rados.0.smithi100.stdout:879: rmattr oid 29 current snap is 76 2024-04-07T16:19:13.987 INFO:tasks.rados.rados.0.smithi100.stdout:879: done (0 left) 2024-04-07T16:19:13.987 INFO:tasks.rados.rados.0.smithi100.stdout:880: snap_create 2024-04-07T16:19:14.237 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:19:13 smithi144 bash[51952]: debug 2024-04-07T16:19:13.993+0000 7f60456b5700 -1 mgr handle_mgr_map I was active but no longer am 2024-04-07T16:19:14.237 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:19:14 smithi144 bash[51952]: ignoring --setuser ceph since I am not root 2024-04-07T16:19:14.237 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:19:14 smithi144 bash[51952]: ignoring --setgroup ceph since I am not root 2024-04-07T16:19:14.237 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:19:14 smithi144 bash[51952]: debug 2024-04-07T16:19:14.057+0000 7f86d4c78700 1 -- 172.21.15.144:0/84442053 <== mon.1 v2:172.21.15.144:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5629bf5d25a0 con 0x5629c0350800 2024-04-07T16:19:14.238 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:19:14 smithi144 bash[51952]: debug 2024-04-07T16:19:14.205+0000 7f86dea6a200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-07T16:19:14.437 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:19:14 smithi100 bash[68137]: [07/Apr/2024:16:19:14] ENGINE Bus STOPPING 2024-04-07T16:19:14.437 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:19:14 smithi100 bash[68137]: [07/Apr/2024:16:19:14] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-04-07T16:19:14.437 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:19:14 smithi100 bash[68137]: [07/Apr/2024:16:19:14] ENGINE Bus STOPPED 2024-04-07T16:19:14.437 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:19:14 smithi100 bash[68137]: [07/Apr/2024:16:19:14] ENGINE Bus STARTING 2024-04-07T16:19:14.437 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:19:14 smithi100 bash[68137]: debug 2024-04-07T16:19:14.195+0000 7f08ed599700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-07T16:19:14.437 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:19:14 smithi100 bash[68137]: debug 2024-04-07T16:19:14.195+0000 7f08ed599700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-07T16:19:14.437 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:19:14 smithi100 bash[68137]: debug 2024-04-07T16:19:14.195+0000 7f08ed599700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-07T16:19:14.437 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:19:14 smithi100 bash[68137]: debug 2024-04-07T16:19:14.195+0000 7f08ed599700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-07T16:19:14.437 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:19:14 smithi100 bash[68137]: debug 2024-04-07T16:19:14.195+0000 7f08ed599700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-07T16:19:14.437 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:19:14 smithi100 bash[68137]: debug 2024-04-07T16:19:14.199+0000 7f08ea593700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-07T16:19:14.437 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:19:14 smithi100 bash[68137]: debug 2024-04-07T16:19:14.199+0000 7f08ea593700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-07T16:19:14.437 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:19:14 smithi100 bash[68137]: debug 2024-04-07T16:19:14.199+0000 7f08ea593700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-07T16:19:14.438 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:19:14 smithi100 bash[68137]: debug 2024-04-07T16:19:14.199+0000 7f08ea593700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-07T16:19:14.438 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:19:14 smithi100 bash[68137]: debug 2024-04-07T16:19:14.199+0000 7f08ea593700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-07T16:19:14.438 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:19:14 smithi100 bash[68137]: [07/Apr/2024:16:19:14] ENGINE Serving on http://:::9283 2024-04-07T16:19:14.438 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:19:14 smithi100 bash[68137]: [07/Apr/2024:16:19:14] ENGINE Bus STARTED 2024-04-07T16:19:14.982 INFO:tasks.rados.rados.0.smithi100.stdout:880: done (0 left) 2024-04-07T16:19:14.982 INFO:tasks.rados.rados.0.smithi100.stdout:881: delete oid 33 current snap is 77 2024-04-07T16:19:15.001 INFO:tasks.rados.rados.0.smithi100.stdout:881: done (0 left) 2024-04-07T16:19:15.002 INFO:tasks.rados.rados.0.smithi100.stdout:882: setattr oid 48 current snap is 77 2024-04-07T16:19:15.004 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 2 current snap is 77 2024-04-07T16:19:15.004 INFO:tasks.rados.rados.0.smithi100.stdout:883: seq_num 274 ranges {0=598016,598016=499712,1097728=745472,1843200=360448} 2024-04-07T16:19:15.013 INFO:tasks.rados.rados.0.smithi100.stdout:883: writing smithi10066867-2 from 0 to 598016 tid 1 2024-04-07T16:19:15.020 INFO:tasks.rados.rados.0.smithi100.stdout:883: writing smithi10066867-2 from 598016 to 1097728 tid 2 2024-04-07T16:19:15.031 INFO:tasks.rados.rados.0.smithi100.stdout:883: writing smithi10066867-2 from 1097728 to 1843200 tid 3 2024-04-07T16:19:15.036 INFO:tasks.rados.rados.0.smithi100.stdout:883: writing smithi10066867-2 from 1843200 to 2203648 tid 4 2024-04-07T16:19:15.036 INFO:tasks.rados.rados.0.smithi100.stdout:884: read oid 37 snap -1 2024-04-07T16:19:15.036 INFO:tasks.rados.rados.0.smithi100.stdout:884: expect (ObjNum 267 snap 71 seq_num 267) 2024-04-07T16:19:15.036 INFO:tasks.rados.rados.0.smithi100.stdout:885: copy_from oid 40 from oid 33 current snap is 77 2024-04-07T16:19:15.037 INFO:tasks.rados.rados.0.smithi100.stdout:886: snap_remove snap 72 2024-04-07T16:19:15.105 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:14 smithi144 bash[23522]: audit 2024-04-07T16:19:13.833968+0000 mon.a (mon.0) 1112 : audit 1 from='mgr.24553 ' entity='mgr.x' 2024-04-07T16:19:15.105 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:14 smithi144 bash[23522]: audit 2024-04-07T16:19:13.852150+0000 mon.b (mon.1) 321 : audit 0 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:19:15.105 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:14 smithi144 bash[23522]: audit 2024-04-07T16:19:13.852443+0000 mon.a (mon.0) 1113 : audit 1 from='mgr.24553 ' entity='mgr.x' 2024-04-07T16:19:15.105 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:14 smithi144 bash[23522]: audit 2024-04-07T16:19:13.854121+0000 mon.b (mon.1) 322 : audit 1 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:19:15.105 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:14 smithi144 bash[23522]: audit 2024-04-07T16:19:13.870097+0000 mon.a (mon.0) 1114 : audit 1 from='mgr.24553 ' entity='mgr.x' 2024-04-07T16:19:15.105 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:14 smithi144 bash[23522]: cluster 2024-04-07T16:19:13.881118+0000 mon.a (mon.0) 1115 : cluster 0 osdmap e220: 8 total, 8 up, 8 in 2024-04-07T16:19:15.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:14 smithi144 bash[23522]: audit 2024-04-07T16:19:13.964898+0000 mon.b (mon.1) 323 : audit 0 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:19:15.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:14 smithi144 bash[23522]: audit 2024-04-07T16:19:13.976738+0000 mon.b (mon.1) 324 : audit 1 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "mgr fail", "who": "x"}]: dispatch 2024-04-07T16:19:15.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:14 smithi144 bash[23522]: audit 2024-04-07T16:19:13.981876+0000 mon.a (mon.0) 1116 : audit 1 from='mgr.24553 ' entity='mgr.x' cmd=[{"prefix": "mgr fail", "who": "x"}]: dispatch 2024-04-07T16:19:15.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:14 smithi144 bash[23522]: cluster 2024-04-07T16:19:13.982511+0000 mon.a (mon.0) 1117 : cluster 1 Activating manager daemon y 2024-04-07T16:19:15.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:14 smithi144 bash[23522]: cluster 2024-04-07T16:19:13.994815+0000 mon.a (mon.0) 1118 : cluster 0 osdmap e221: 8 total, 8 up, 8 in 2024-04-07T16:19:15.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:14 smithi144 bash[23522]: audit 2024-04-07T16:19:13.995528+0000 mon.a (mon.0) 1119 : audit 1 from='mgr.24553 ' entity='mgr.x' cmd='[{"prefix": "mgr fail", "who": "x"}]': finished 2024-04-07T16:19:15.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:14 smithi144 bash[23522]: cluster 2024-04-07T16:19:13.995755+0000 mon.a (mon.0) 1120 : cluster 0 mgrmap e30: y(active, starting, since 0.0132279s) 2024-04-07T16:19:15.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:14 smithi144 bash[23522]: audit 2024-04-07T16:19:14.001695+0000 mon.c (mon.2) 63 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:19:15.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:14 smithi144 bash[23522]: audit 2024-04-07T16:19:14.002135+0000 mon.c (mon.2) 64 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:19:15.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:14 smithi144 bash[23522]: audit 2024-04-07T16:19:14.002484+0000 mon.c (mon.2) 65 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:19:15.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:14 smithi144 bash[23522]: audit 2024-04-07T16:19:14.003305+0000 mon.c (mon.2) 66 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-07T16:19:15.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:14 smithi144 bash[23522]: audit 2024-04-07T16:19:14.003741+0000 mon.c (mon.2) 67 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-07T16:19:15.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:14 smithi144 bash[23522]: audit 2024-04-07T16:19:14.004451+0000 mon.c (mon.2) 68 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-07T16:19:15.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:14 smithi144 bash[23522]: audit 2024-04-07T16:19:14.005343+0000 mon.c (mon.2) 69 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-07T16:19:15.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:14 smithi144 bash[23522]: audit 2024-04-07T16:19:14.005795+0000 mon.c (mon.2) 70 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-07T16:19:15.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:14 smithi144 bash[23522]: audit 2024-04-07T16:19:14.006138+0000 mon.c (mon.2) 71 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-07T16:19:15.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:14 smithi144 bash[23522]: audit 2024-04-07T16:19:14.006922+0000 mon.c (mon.2) 72 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-07T16:19:15.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:14 smithi144 bash[23522]: audit 2024-04-07T16:19:14.007305+0000 mon.c (mon.2) 73 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-07T16:19:15.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:14 smithi144 bash[23522]: audit 2024-04-07T16:19:14.007774+0000 mon.c (mon.2) 74 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-07T16:19:15.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:14 smithi144 bash[23522]: audit 2024-04-07T16:19:14.008190+0000 mon.c (mon.2) 75 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-07T16:19:15.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:14 smithi144 bash[23522]: audit 2024-04-07T16:19:14.008560+0000 mon.c (mon.2) 76 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-07T16:19:15.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:14 smithi144 bash[23522]: audit 2024-04-07T16:19:14.008950+0000 mon.c (mon.2) 77 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-07T16:19:15.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:14 smithi144 bash[23522]: audit 2024-04-07T16:19:14.009364+0000 mon.c (mon.2) 78 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-07T16:19:15.111 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:14 smithi144 bash[23522]: audit 2024-04-07T16:19:14.010088+0000 mon.c (mon.2) 79 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-07T16:19:15.111 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:14 smithi144 bash[23522]: cluster 2024-04-07T16:19:14.058885+0000 mon.a (mon.0) 1121 : cluster 1 Manager daemon y is now available 2024-04-07T16:19:15.111 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:14 smithi144 bash[23522]: audit 2024-04-07T16:19:14.104817+0000 mon.c (mon.2) 80 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:19:15.111 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:14 smithi144 bash[23522]: audit 2024-04-07T16:19:14.133053+0000 mon.c (mon.2) 81 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:19:15.111 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:14 smithi144 bash[23522]: audit 2024-04-07T16:19:14.137750+0000 mon.c (mon.2) 82 : audit 1 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-04-07T16:19:15.111 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:14 smithi144 bash[23522]: audit 2024-04-07T16:19:14.138158+0000 mon.a (mon.0) 1122 : audit 1 from='mgr.25073 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-04-07T16:19:15.111 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:14 smithi144 bash[23522]: audit 2024-04-07T16:19:14.171501+0000 mon.c (mon.2) 83 : audit 1 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-04-07T16:19:15.111 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:14 smithi144 bash[23522]: audit 2024-04-07T16:19:14.172211+0000 mon.a (mon.0) 1123 : audit 1 from='mgr.25073 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-04-07T16:19:15.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:14 smithi100 bash[21346]: audit 2024-04-07T16:19:13.833968+0000 mon.a (mon.0) 1112 : audit 1 from='mgr.24553 ' entity='mgr.x' 2024-04-07T16:19:15.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:14 smithi100 bash[21346]: audit 2024-04-07T16:19:13.852150+0000 mon.b (mon.1) 321 : audit 0 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:19:15.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:14 smithi100 bash[21346]: audit 2024-04-07T16:19:13.852443+0000 mon.a (mon.0) 1113 : audit 1 from='mgr.24553 ' entity='mgr.x' 2024-04-07T16:19:15.188 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:14 smithi100 bash[21346]: audit 2024-04-07T16:19:13.854121+0000 mon.b (mon.1) 322 : audit 1 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:19:15.188 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:14 smithi100 bash[21346]: audit 2024-04-07T16:19:13.870097+0000 mon.a (mon.0) 1114 : audit 1 from='mgr.24553 ' entity='mgr.x' 2024-04-07T16:19:15.188 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:14 smithi100 bash[21346]: cluster 2024-04-07T16:19:13.881118+0000 mon.a (mon.0) 1115 : cluster 0 osdmap e220: 8 total, 8 up, 8 in 2024-04-07T16:19:15.188 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:14 smithi100 bash[21346]: audit 2024-04-07T16:19:13.964898+0000 mon.b (mon.1) 323 : audit 0 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:19:15.188 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:14 smithi100 bash[21346]: audit 2024-04-07T16:19:13.976738+0000 mon.b (mon.1) 324 : audit 1 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "mgr fail", "who": "x"}]: dispatch 2024-04-07T16:19:15.188 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:14 smithi100 bash[21346]: audit 2024-04-07T16:19:13.981876+0000 mon.a (mon.0) 1116 : audit 1 from='mgr.24553 ' entity='mgr.x' cmd=[{"prefix": "mgr fail", "who": "x"}]: dispatch 2024-04-07T16:19:15.188 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:14 smithi100 bash[21346]: cluster 2024-04-07T16:19:13.982511+0000 mon.a (mon.0) 1117 : cluster 1 Activating manager daemon y 2024-04-07T16:19:15.188 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:14 smithi100 bash[21346]: cluster 2024-04-07T16:19:13.994815+0000 mon.a (mon.0) 1118 : cluster 0 osdmap e221: 8 total, 8 up, 8 in 2024-04-07T16:19:15.188 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:14 smithi100 bash[21346]: audit 2024-04-07T16:19:13.995528+0000 mon.a (mon.0) 1119 : audit 1 from='mgr.24553 ' entity='mgr.x' cmd='[{"prefix": "mgr fail", "who": "x"}]': finished 2024-04-07T16:19:15.188 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:14 smithi100 bash[21346]: cluster 2024-04-07T16:19:13.995755+0000 mon.a (mon.0) 1120 : cluster 0 mgrmap e30: y(active, starting, since 0.0132279s) 2024-04-07T16:19:15.188 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:14 smithi100 bash[21346]: audit 2024-04-07T16:19:14.001695+0000 mon.c (mon.2) 63 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:19:15.188 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:14 smithi100 bash[21346]: audit 2024-04-07T16:19:14.002135+0000 mon.c (mon.2) 64 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:19:15.188 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:14 smithi100 bash[21346]: audit 2024-04-07T16:19:14.002484+0000 mon.c (mon.2) 65 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:19:15.189 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:14 smithi100 bash[21346]: audit 2024-04-07T16:19:14.003305+0000 mon.c (mon.2) 66 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-07T16:19:15.189 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:14 smithi100 bash[21346]: audit 2024-04-07T16:19:14.003741+0000 mon.c (mon.2) 67 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-07T16:19:15.189 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:14 smithi100 bash[21346]: audit 2024-04-07T16:19:14.004451+0000 mon.c (mon.2) 68 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-07T16:19:15.189 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:14 smithi100 bash[21346]: audit 2024-04-07T16:19:14.005343+0000 mon.c (mon.2) 69 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-07T16:19:15.189 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:14 smithi100 bash[21346]: audit 2024-04-07T16:19:14.005795+0000 mon.c (mon.2) 70 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-07T16:19:15.189 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:14 smithi100 bash[31660]: audit 2024-04-07T16:19:13.833968+0000 mon.a (mon.0) 1112 : audit 1 from='mgr.24553 ' entity='mgr.x' 2024-04-07T16:19:15.189 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:14 smithi100 bash[31660]: audit 2024-04-07T16:19:13.852150+0000 mon.b (mon.1) 321 : audit 0 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:19:15.189 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:14 smithi100 bash[31660]: audit 2024-04-07T16:19:13.852443+0000 mon.a (mon.0) 1113 : audit 1 from='mgr.24553 ' entity='mgr.x' 2024-04-07T16:19:15.189 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:14 smithi100 bash[31660]: audit 2024-04-07T16:19:13.854121+0000 mon.b (mon.1) 322 : audit 1 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:19:15.189 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:14 smithi100 bash[31660]: audit 2024-04-07T16:19:13.870097+0000 mon.a (mon.0) 1114 : audit 1 from='mgr.24553 ' entity='mgr.x' 2024-04-07T16:19:15.189 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:14 smithi100 bash[31660]: cluster 2024-04-07T16:19:13.881118+0000 mon.a (mon.0) 1115 : cluster 0 osdmap e220: 8 total, 8 up, 8 in 2024-04-07T16:19:15.189 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:14 smithi100 bash[31660]: audit 2024-04-07T16:19:13.964898+0000 mon.b (mon.1) 323 : audit 0 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:19:15.190 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:14 smithi100 bash[31660]: audit 2024-04-07T16:19:13.976738+0000 mon.b (mon.1) 324 : audit 1 from='mgr.24553 172.21.15.144:0/1428787240' entity='mgr.x' cmd=[{"prefix": "mgr fail", "who": "x"}]: dispatch 2024-04-07T16:19:15.190 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:14 smithi100 bash[31660]: audit 2024-04-07T16:19:13.981876+0000 mon.a (mon.0) 1116 : audit 1 from='mgr.24553 ' entity='mgr.x' cmd=[{"prefix": "mgr fail", "who": "x"}]: dispatch 2024-04-07T16:19:15.190 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:14 smithi100 bash[31660]: cluster 2024-04-07T16:19:13.982511+0000 mon.a (mon.0) 1117 : cluster 1 Activating manager daemon y 2024-04-07T16:19:15.190 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:14 smithi100 bash[31660]: cluster 2024-04-07T16:19:13.994815+0000 mon.a (mon.0) 1118 : cluster 0 osdmap e221: 8 total, 8 up, 8 in 2024-04-07T16:19:15.190 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:14 smithi100 bash[31660]: audit 2024-04-07T16:19:13.995528+0000 mon.a (mon.0) 1119 : audit 1 from='mgr.24553 ' entity='mgr.x' cmd='[{"prefix": "mgr fail", "who": "x"}]': finished 2024-04-07T16:19:15.190 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:14 smithi100 bash[31660]: cluster 2024-04-07T16:19:13.995755+0000 mon.a (mon.0) 1120 : cluster 0 mgrmap e30: y(active, starting, since 0.0132279s) 2024-04-07T16:19:15.190 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:14 smithi100 bash[31660]: audit 2024-04-07T16:19:14.001695+0000 mon.c (mon.2) 63 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:19:15.190 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:14 smithi100 bash[31660]: audit 2024-04-07T16:19:14.002135+0000 mon.c (mon.2) 64 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:19:15.190 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:14 smithi100 bash[31660]: audit 2024-04-07T16:19:14.002484+0000 mon.c (mon.2) 65 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:19:15.190 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:14 smithi100 bash[31660]: audit 2024-04-07T16:19:14.003305+0000 mon.c (mon.2) 66 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-07T16:19:15.190 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:14 smithi100 bash[31660]: audit 2024-04-07T16:19:14.003741+0000 mon.c (mon.2) 67 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-07T16:19:15.190 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:14 smithi100 bash[31660]: audit 2024-04-07T16:19:14.004451+0000 mon.c (mon.2) 68 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-07T16:19:15.190 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:14 smithi100 bash[31660]: audit 2024-04-07T16:19:14.005343+0000 mon.c (mon.2) 69 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-07T16:19:15.190 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:14 smithi100 bash[31660]: audit 2024-04-07T16:19:14.005795+0000 mon.c (mon.2) 70 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-07T16:19:15.191 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:14 smithi100 bash[21346]: audit 2024-04-07T16:19:14.006138+0000 mon.c (mon.2) 71 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-07T16:19:15.191 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:14 smithi100 bash[21346]: audit 2024-04-07T16:19:14.006922+0000 mon.c (mon.2) 72 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-07T16:19:15.191 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:14 smithi100 bash[21346]: audit 2024-04-07T16:19:14.007305+0000 mon.c (mon.2) 73 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-07T16:19:15.191 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:14 smithi100 bash[21346]: audit 2024-04-07T16:19:14.007774+0000 mon.c (mon.2) 74 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-07T16:19:15.191 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:14 smithi100 bash[21346]: audit 2024-04-07T16:19:14.008190+0000 mon.c (mon.2) 75 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-07T16:19:15.191 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:14 smithi100 bash[21346]: audit 2024-04-07T16:19:14.008560+0000 mon.c (mon.2) 76 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-07T16:19:15.191 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:14 smithi100 bash[21346]: audit 2024-04-07T16:19:14.008950+0000 mon.c (mon.2) 77 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-07T16:19:15.191 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:14 smithi100 bash[21346]: audit 2024-04-07T16:19:14.009364+0000 mon.c (mon.2) 78 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-07T16:19:15.191 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:14 smithi100 bash[21346]: audit 2024-04-07T16:19:14.010088+0000 mon.c (mon.2) 79 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-07T16:19:15.191 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:14 smithi100 bash[21346]: cluster 2024-04-07T16:19:14.058885+0000 mon.a (mon.0) 1121 : cluster 1 Manager daemon y is now available 2024-04-07T16:19:15.191 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:14 smithi100 bash[21346]: audit 2024-04-07T16:19:14.104817+0000 mon.c (mon.2) 80 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:19:15.191 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:14 smithi100 bash[21346]: audit 2024-04-07T16:19:14.133053+0000 mon.c (mon.2) 81 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:19:15.191 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:14 smithi100 bash[21346]: audit 2024-04-07T16:19:14.137750+0000 mon.c (mon.2) 82 : audit 1 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-04-07T16:19:15.192 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:14 smithi100 bash[21346]: audit 2024-04-07T16:19:14.138158+0000 mon.a (mon.0) 1122 : audit 1 from='mgr.25073 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-04-07T16:19:15.192 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:14 smithi100 bash[21346]: audit 2024-04-07T16:19:14.171501+0000 mon.c (mon.2) 83 : audit 1 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-04-07T16:19:15.192 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:14 smithi100 bash[21346]: audit 2024-04-07T16:19:14.172211+0000 mon.a (mon.0) 1123 : audit 1 from='mgr.25073 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-04-07T16:19:15.192 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:14 smithi100 bash[31660]: audit 2024-04-07T16:19:14.006138+0000 mon.c (mon.2) 71 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-07T16:19:15.192 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:14 smithi100 bash[31660]: audit 2024-04-07T16:19:14.006922+0000 mon.c (mon.2) 72 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-07T16:19:15.192 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:14 smithi100 bash[31660]: audit 2024-04-07T16:19:14.007305+0000 mon.c (mon.2) 73 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-07T16:19:15.192 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:14 smithi100 bash[31660]: audit 2024-04-07T16:19:14.007774+0000 mon.c (mon.2) 74 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-07T16:19:15.192 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:14 smithi100 bash[31660]: audit 2024-04-07T16:19:14.008190+0000 mon.c (mon.2) 75 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-07T16:19:15.192 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:14 smithi100 bash[31660]: audit 2024-04-07T16:19:14.008560+0000 mon.c (mon.2) 76 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-07T16:19:15.192 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:14 smithi100 bash[31660]: audit 2024-04-07T16:19:14.008950+0000 mon.c (mon.2) 77 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-07T16:19:15.192 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:14 smithi100 bash[31660]: audit 2024-04-07T16:19:14.009364+0000 mon.c (mon.2) 78 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-07T16:19:15.192 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:14 smithi100 bash[31660]: audit 2024-04-07T16:19:14.010088+0000 mon.c (mon.2) 79 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-07T16:19:15.192 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:14 smithi100 bash[31660]: cluster 2024-04-07T16:19:14.058885+0000 mon.a (mon.0) 1121 : cluster 1 Manager daemon y is now available 2024-04-07T16:19:15.193 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:14 smithi100 bash[31660]: audit 2024-04-07T16:19:14.104817+0000 mon.c (mon.2) 80 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:19:15.193 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:14 smithi100 bash[31660]: audit 2024-04-07T16:19:14.133053+0000 mon.c (mon.2) 81 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:19:15.193 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:14 smithi100 bash[31660]: audit 2024-04-07T16:19:14.137750+0000 mon.c (mon.2) 82 : audit 1 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-04-07T16:19:15.193 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:14 smithi100 bash[31660]: audit 2024-04-07T16:19:14.138158+0000 mon.a (mon.0) 1122 : audit 1 from='mgr.25073 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-04-07T16:19:15.193 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:14 smithi100 bash[31660]: audit 2024-04-07T16:19:14.171501+0000 mon.c (mon.2) 83 : audit 1 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-04-07T16:19:15.193 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:14 smithi100 bash[31660]: audit 2024-04-07T16:19:14.172211+0000 mon.a (mon.0) 1123 : audit 1 from='mgr.25073 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-04-07T16:19:16.004 INFO:tasks.rados.rados.0.smithi100.stdout:886: done (4 left) 2024-04-07T16:19:16.004 INFO:tasks.rados.rados.0.smithi100.stdout:887: setattr oid 18 current snap is 77 2024-04-07T16:19:16.004 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 48 v 249 (ObjNum 239 snap 65 seq_num 239) dirty exists 2024-04-07T16:19:16.004 INFO:tasks.rados.rados.0.smithi100.stdout:883: finishing write tid 1 to smithi10066867-2 2024-04-07T16:19:16.004 INFO:tasks.rados.rados.0.smithi100.stdout:883: finishing write tid 2 to smithi10066867-2 2024-04-07T16:19:16.005 INFO:tasks.rados.rados.0.smithi100.stdout:883: finishing write tid 3 to smithi10066867-2 2024-04-07T16:19:16.005 INFO:tasks.rados.rados.0.smithi100.stdout:883: finishing write tid 4 to smithi10066867-2 2024-04-07T16:19:16.005 INFO:tasks.rados.rados.0.smithi100.stdout:883: finishing write tid 5 to smithi10066867-2 2024-04-07T16:19:16.005 INFO:tasks.rados.rados.0.smithi100.stdout:885: finishing copy_from to smithi10066867-40 2024-04-07T16:19:16.005 INFO:tasks.rados.rados.0.smithi100.stdout:885: got expected ENOENT (src dne) 2024-04-07T16:19:16.005 INFO:tasks.rados.rados.0.smithi100.stdout:885: finishing copy_from racing read to smithi10066867-40 2024-04-07T16:19:16.005 INFO:tasks.rados.rados.0.smithi100.stdout:883: finishing write tid 6 to smithi10066867-2 2024-04-07T16:19:16.005 INFO:tasks.rados.rados.0.smithi100.stdout:883: oid 2 updating version 0 to 388 2024-04-07T16:19:16.005 INFO:tasks.rados.rados.0.smithi100.stdout:883: oid 2 version 388 is already newer than 384 2024-04-07T16:19:16.005 INFO:tasks.rados.rados.0.smithi100.stdout:883: oid 2 version 388 is already newer than 387 2024-04-07T16:19:16.005 INFO:tasks.rados.rados.0.smithi100.stdout:883: oid 2 version 388 is already newer than 385 2024-04-07T16:19:16.005 INFO:tasks.rados.rados.0.smithi100.stdout:883: oid 2 version 388 is already newer than 386 2024-04-07T16:19:16.005 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 2 v 388 (ObjNum 274 snap 77 seq_num 274) dirty exists 2024-04-07T16:19:16.005 INFO:tasks.rados.rados.0.smithi100.stdout:883: left oid 2 (ObjNum 274 snap 77 seq_num 274) 2024-04-07T16:19:16.006 INFO:tasks.rados.rados.0.smithi100.stdout:884: expect (ObjNum 267 snap 71 seq_num 267) 2024-04-07T16:19:16.074 INFO:tasks.rados.rados.0.smithi100.stdout:882: done (4 left) 2024-04-07T16:19:16.074 INFO:tasks.rados.rados.0.smithi100.stdout:883: done (3 left) 2024-04-07T16:19:16.074 INFO:tasks.rados.rados.0.smithi100.stdout:884: done (2 left) 2024-04-07T16:19:16.074 INFO:tasks.rados.rados.0.smithi100.stdout:885: done (1 left) 2024-04-07T16:19:16.074 INFO:tasks.rados.rados.0.smithi100.stdout:888: snap_create 2024-04-07T16:19:16.078 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 18 v 532 (ObjNum 260 snap 70 seq_num 260) dirty exists 2024-04-07T16:19:16.105 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:15 smithi144 bash[23522]: cluster 2024-04-07T16:19:14.997852+0000 mon.a (mon.0) 1124 : cluster 0 osdmap e222: 8 total, 8 up, 8 in 2024-04-07T16:19:16.105 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:15 smithi144 bash[23522]: cluster 2024-04-07T16:19:15.016516+0000 mon.a (mon.0) 1125 : cluster 0 mgrmap e31: y(active, since 1.03398s) 2024-04-07T16:19:16.105 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:15 smithi144 bash[23522]: cluster 2024-04-07T16:19:15.042745+0000 mgr.y (mgr.25073) 1 : cluster 0 pgmap v4: 121 pgs: 121 active+clean; 174 MiB data, 1017 MiB used, 714 GiB / 715 GiB avail 2024-04-07T16:19:16.105 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:19:15 smithi144 bash[51952]: debug 2024-04-07T16:19:15.801+0000 7f86dea6a200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-07T16:19:16.105 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:19:15 smithi144 bash[51952]: debug 2024-04-07T16:19:15.897+0000 7f86dea6a200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-07T16:19:16.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:15 smithi100 bash[21346]: cluster 2024-04-07T16:19:14.997852+0000 mon.a (mon.0) 1124 : cluster 0 osdmap e222: 8 total, 8 up, 8 in 2024-04-07T16:19:16.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:15 smithi100 bash[21346]: cluster 2024-04-07T16:19:15.016516+0000 mon.a (mon.0) 1125 : cluster 0 mgrmap e31: y(active, since 1.03398s) 2024-04-07T16:19:16.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:15 smithi100 bash[21346]: cluster 2024-04-07T16:19:15.042745+0000 mgr.y (mgr.25073) 1 : cluster 0 pgmap v4: 121 pgs: 121 active+clean; 174 MiB data, 1017 MiB used, 714 GiB / 715 GiB avail 2024-04-07T16:19:16.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:15 smithi100 bash[31660]: cluster 2024-04-07T16:19:14.997852+0000 mon.a (mon.0) 1124 : cluster 0 osdmap e222: 8 total, 8 up, 8 in 2024-04-07T16:19:16.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:15 smithi100 bash[31660]: cluster 2024-04-07T16:19:15.016516+0000 mon.a (mon.0) 1125 : cluster 0 mgrmap e31: y(active, since 1.03398s) 2024-04-07T16:19:16.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:15 smithi100 bash[31660]: cluster 2024-04-07T16:19:15.042745+0000 mgr.y (mgr.25073) 1 : cluster 0 pgmap v4: 121 pgs: 121 active+clean; 174 MiB data, 1017 MiB used, 714 GiB / 715 GiB avail 2024-04-07T16:19:16.605 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:19:16 smithi144 bash[51952]: debug 2024-04-07T16:19:16.165+0000 7f86dea6a200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-07T16:19:17.008 INFO:tasks.rados.rados.0.smithi100.stdout:887: done (1 left) 2024-04-07T16:19:17.008 INFO:tasks.rados.rados.0.smithi100.stdout:888: done (0 left) 2024-04-07T16:19:17.008 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 37 current snap is 78 2024-04-07T16:19:17.008 INFO:tasks.rados.rados.0.smithi100.stdout:889: seq_num 275 ranges {2080768=614400,2695168=786432,3481600=499712,3981312=319488} 2024-04-07T16:19:17.044 INFO:tasks.rados.rados.0.smithi100.stdout:889: writing smithi10066867-37 from 2080768 to 2695168 tid 1 2024-04-07T16:19:17.056 INFO:tasks.rados.rados.0.smithi100.stdout:889: writing smithi10066867-37 from 2695168 to 3481600 tid 2 2024-04-07T16:19:17.062 INFO:tasks.rados.rados.0.smithi100.stdout:889: writing smithi10066867-37 from 3481600 to 3981312 tid 3 2024-04-07T16:19:17.066 INFO:tasks.rados.rados.0.smithi100.stdout:889: writing smithi10066867-37 from 3981312 to 4300800 tid 4 2024-04-07T16:19:17.066 INFO:tasks.rados.rados.0.smithi100.stdout:890: snap_create 2024-04-07T16:19:17.066 INFO:tasks.rados.rados.0.smithi100.stdout:889: finishing write tid 1 to smithi10066867-37 2024-04-07T16:19:17.066 INFO:tasks.rados.rados.0.smithi100.stdout:889: finishing write tid 2 to smithi10066867-37 2024-04-07T16:19:17.066 INFO:tasks.rados.rados.0.smithi100.stdout:889: finishing write tid 3 to smithi10066867-37 2024-04-07T16:19:17.072 INFO:tasks.rados.rados.0.smithi100.stdout:889: finishing write tid 4 to smithi10066867-37 2024-04-07T16:19:17.072 INFO:tasks.rados.rados.0.smithi100.stdout:889: finishing write tid 5 to smithi10066867-37 2024-04-07T16:19:17.075 INFO:tasks.rados.rados.0.smithi100.stdout:889: finishing write tid 6 to smithi10066867-37 2024-04-07T16:19:17.075 INFO:tasks.rados.rados.0.smithi100.stdout:889: oid 37 updating version 0 to 462 2024-04-07T16:19:17.075 INFO:tasks.rados.rados.0.smithi100.stdout:889: oid 37 updating version 462 to 463 2024-04-07T16:19:17.075 INFO:tasks.rados.rados.0.smithi100.stdout:889: oid 37 version 463 is already newer than 460 2024-04-07T16:19:17.075 INFO:tasks.rados.rados.0.smithi100.stdout:889: oid 37 version 463 is already newer than 461 2024-04-07T16:19:17.075 INFO:tasks.rados.rados.0.smithi100.stdout:889: oid 37 version 463 is already newer than 459 2024-04-07T16:19:17.075 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 37 v 463 (ObjNum 275 snap 78 seq_num 275) dirty exists 2024-04-07T16:19:17.076 INFO:tasks.rados.rados.0.smithi100.stdout:889: left oid 37 (ObjNum 275 snap 78 seq_num 275) 2024-04-07T16:19:17.355 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:19:16 smithi144 bash[51952]: debug 2024-04-07T16:19:16.969+0000 7f86dea6a200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-07T16:19:17.355 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:19:17 smithi144 bash[51952]: debug 2024-04-07T16:19:17.089+0000 7f86dea6a200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-07T16:19:17.355 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:19:17 smithi144 bash[51952]: debug 2024-04-07T16:19:17.193+0000 7f86dea6a200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-07T16:19:17.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:17 smithi144 bash[23522]: cephadm 2024-04-07T16:19:15.393319+0000 mgr.y (mgr.25073) 2 : cephadm 1 [07/Apr/2024:16:19:15] ENGINE Bus STARTING 2024-04-07T16:19:17.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:17 smithi144 bash[23522]: cephadm 2024-04-07T16:19:15.495291+0000 mgr.y (mgr.25073) 3 : cephadm 1 [07/Apr/2024:16:19:15] ENGINE Serving on http://172.21.15.100:8765 2024-04-07T16:19:17.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:17 smithi144 bash[23522]: cephadm 2024-04-07T16:19:15.613279+0000 mgr.y (mgr.25073) 4 : cephadm 1 [07/Apr/2024:16:19:15] ENGINE Serving on https://172.21.15.100:7150 2024-04-07T16:19:17.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:17 smithi144 bash[23522]: cephadm 2024-04-07T16:19:15.613480+0000 mgr.y (mgr.25073) 5 : cephadm 1 [07/Apr/2024:16:19:15] ENGINE Bus STARTED 2024-04-07T16:19:17.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:17 smithi144 bash[23522]: cluster 2024-04-07T16:19:16.008044+0000 mgr.y (mgr.25073) 6 : cluster 0 pgmap v5: 121 pgs: 121 active+clean; 174 MiB data, 1017 MiB used, 714 GiB / 715 GiB avail 2024-04-07T16:19:17.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:17 smithi144 bash[23522]: cluster 2024-04-07T16:19:16.015798+0000 mon.a (mon.0) 1126 : cluster 0 osdmap e223: 8 total, 8 up, 8 in 2024-04-07T16:19:17.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:17 smithi100 bash[21346]: cephadm 2024-04-07T16:19:15.393319+0000 mgr.y (mgr.25073) 2 : cephadm 1 [07/Apr/2024:16:19:15] ENGINE Bus STARTING 2024-04-07T16:19:17.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:17 smithi100 bash[21346]: cephadm 2024-04-07T16:19:15.495291+0000 mgr.y (mgr.25073) 3 : cephadm 1 [07/Apr/2024:16:19:15] ENGINE Serving on http://172.21.15.100:8765 2024-04-07T16:19:17.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:17 smithi100 bash[21346]: cephadm 2024-04-07T16:19:15.613279+0000 mgr.y (mgr.25073) 4 : cephadm 1 [07/Apr/2024:16:19:15] ENGINE Serving on https://172.21.15.100:7150 2024-04-07T16:19:17.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:17 smithi100 bash[21346]: cephadm 2024-04-07T16:19:15.613480+0000 mgr.y (mgr.25073) 5 : cephadm 1 [07/Apr/2024:16:19:15] ENGINE Bus STARTED 2024-04-07T16:19:17.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:17 smithi100 bash[21346]: cluster 2024-04-07T16:19:16.008044+0000 mgr.y (mgr.25073) 6 : cluster 0 pgmap v5: 121 pgs: 121 active+clean; 174 MiB data, 1017 MiB used, 714 GiB / 715 GiB avail 2024-04-07T16:19:17.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:17 smithi100 bash[21346]: cluster 2024-04-07T16:19:16.015798+0000 mon.a (mon.0) 1126 : cluster 0 osdmap e223: 8 total, 8 up, 8 in 2024-04-07T16:19:17.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:17 smithi100 bash[31660]: cephadm 2024-04-07T16:19:15.393319+0000 mgr.y (mgr.25073) 2 : cephadm 1 [07/Apr/2024:16:19:15] ENGINE Bus STARTING 2024-04-07T16:19:17.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:17 smithi100 bash[31660]: cephadm 2024-04-07T16:19:15.495291+0000 mgr.y (mgr.25073) 3 : cephadm 1 [07/Apr/2024:16:19:15] ENGINE Serving on http://172.21.15.100:8765 2024-04-07T16:19:17.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:17 smithi100 bash[31660]: cephadm 2024-04-07T16:19:15.613279+0000 mgr.y (mgr.25073) 4 : cephadm 1 [07/Apr/2024:16:19:15] ENGINE Serving on https://172.21.15.100:7150 2024-04-07T16:19:17.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:17 smithi100 bash[31660]: cephadm 2024-04-07T16:19:15.613480+0000 mgr.y (mgr.25073) 5 : cephadm 1 [07/Apr/2024:16:19:15] ENGINE Bus STARTED 2024-04-07T16:19:17.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:17 smithi100 bash[31660]: cluster 2024-04-07T16:19:16.008044+0000 mgr.y (mgr.25073) 6 : cluster 0 pgmap v5: 121 pgs: 121 active+clean; 174 MiB data, 1017 MiB used, 714 GiB / 715 GiB avail 2024-04-07T16:19:17.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:17 smithi100 bash[31660]: cluster 2024-04-07T16:19:16.015798+0000 mon.a (mon.0) 1126 : cluster 0 osdmap e223: 8 total, 8 up, 8 in 2024-04-07T16:19:17.855 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:19:17 smithi144 bash[51952]: debug 2024-04-07T16:19:17.561+0000 7f86dea6a200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-07T16:19:17.855 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:19:17 smithi144 bash[51952]: debug 2024-04-07T16:19:17.677+0000 7f86dea6a200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-07T16:19:17.855 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:19:17 smithi144 bash[51952]: debug 2024-04-07T16:19:17.781+0000 7f86dea6a200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-07T16:19:17.869 INFO:tasks.rados.rados.0.smithi100.stdout:889: done (1 left) 2024-04-07T16:19:17.869 INFO:tasks.rados.rados.0.smithi100.stdout:890: done (0 left) 2024-04-07T16:19:17.869 INFO:tasks.rados.rados.0.smithi100.stdout:891: read oid 11 snap 63 2024-04-07T16:19:17.869 INFO:tasks.rados.rados.0.smithi100.stdout:891: expect deleted 2024-04-07T16:19:17.869 INFO:tasks.rados.rados.0.smithi100.stdout:892: snap_create 2024-04-07T16:19:18.355 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:19:18 smithi144 bash[51952]: debug 2024-04-07T16:19:18.149+0000 7f86dea6a200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-07T16:19:18.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:18 smithi144 bash[23522]: cluster 2024-04-07T16:19:17.023126+0000 mon.a (mon.0) 1127 : cluster 0 osdmap e224: 8 total, 8 up, 8 in 2024-04-07T16:19:18.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:18 smithi144 bash[23522]: cluster 2024-04-07T16:19:17.039265+0000 mon.a (mon.0) 1128 : cluster 0 mgrmap e32: y(active, since 3s) 2024-04-07T16:19:18.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:18 smithi144 bash[23522]: cluster 2024-04-07T16:19:17.882733+0000 mon.a (mon.0) 1129 : cluster 0 osdmap e225: 8 total, 8 up, 8 in 2024-04-07T16:19:18.436 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:18 smithi100 bash[21346]: cluster 2024-04-07T16:19:17.023126+0000 mon.a (mon.0) 1127 : cluster 0 osdmap e224: 8 total, 8 up, 8 in 2024-04-07T16:19:18.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:18 smithi100 bash[21346]: cluster 2024-04-07T16:19:17.039265+0000 mon.a (mon.0) 1128 : cluster 0 mgrmap e32: y(active, since 3s) 2024-04-07T16:19:18.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:18 smithi100 bash[21346]: cluster 2024-04-07T16:19:17.882733+0000 mon.a (mon.0) 1129 : cluster 0 osdmap e225: 8 total, 8 up, 8 in 2024-04-07T16:19:18.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:18 smithi100 bash[31660]: cluster 2024-04-07T16:19:17.023126+0000 mon.a (mon.0) 1127 : cluster 0 osdmap e224: 8 total, 8 up, 8 in 2024-04-07T16:19:18.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:18 smithi100 bash[31660]: cluster 2024-04-07T16:19:17.039265+0000 mon.a (mon.0) 1128 : cluster 0 mgrmap e32: y(active, since 3s) 2024-04-07T16:19:18.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:18 smithi100 bash[31660]: cluster 2024-04-07T16:19:17.882733+0000 mon.a (mon.0) 1129 : cluster 0 osdmap e225: 8 total, 8 up, 8 in 2024-04-07T16:19:18.655 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:19:18 smithi144 bash[51952]: debug 2024-04-07T16:19:18.397+0000 7f86dea6a200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-07T16:19:18.871 INFO:tasks.rados.rados.0.smithi100.stdout:891: done (1 left) 2024-04-07T16:19:18.871 INFO:tasks.rados.rados.0.smithi100.stdout:892: done (0 left) 2024-04-07T16:19:18.871 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 10 current snap is 80 2024-04-07T16:19:18.871 INFO:tasks.rados.rados.0.smithi100.stdout:893: seq_num 276 ranges {2031616=630784,2662400=671744,3334144=704512,4038656=229376} 2024-04-07T16:19:18.897 INFO:tasks.rados.rados.0.smithi100.stdout:893: writing smithi10066867-10 from 2031616 to 2662400 tid 1 2024-04-07T16:19:18.905 INFO:tasks.rados.rados.0.smithi100.stdout:893: writing smithi10066867-10 from 2662400 to 3334144 tid 2 2024-04-07T16:19:18.912 INFO:tasks.rados.rados.0.smithi100.stdout:893: writing smithi10066867-10 from 3334144 to 4038656 tid 3 2024-04-07T16:19:18.915 INFO:tasks.rados.rados.0.smithi100.stdout:893: writing smithi10066867-10 from 4038656 to 4268032 tid 4 2024-04-07T16:19:18.915 INFO:tasks.rados.rados.0.smithi100.stdout:894: copy_from oid 5 from oid 16 current snap is 80 2024-04-07T16:19:18.915 INFO:tasks.rados.rados.0.smithi100.stdout:893: finishing write tid 1 to smithi10066867-10 2024-04-07T16:19:18.916 INFO:tasks.rados.rados.0.smithi100.stdout:893: finishing write tid 2 to smithi10066867-10 2024-04-07T16:19:18.916 INFO:tasks.rados.rados.0.smithi100.stdout:895: copy_from oid 36 from oid 3 current snap is 80 2024-04-07T16:19:18.916 INFO:tasks.rados.rados.0.smithi100.stdout:896: read oid 6 snap -1 2024-04-07T16:19:18.916 INFO:tasks.rados.rados.0.smithi100.stdout:896: expect (ObjNum 162 snap 43 seq_num 162) 2024-04-07T16:19:18.916 INFO:tasks.rados.rados.0.smithi100.stdout:897: read oid 39 snap -1 2024-04-07T16:19:18.916 INFO:tasks.rados.rados.0.smithi100.stdout:897: expect (ObjNum 212 snap 57 seq_num 212) 2024-04-07T16:19:18.916 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 11 current snap is 80 2024-04-07T16:19:18.916 INFO:tasks.rados.rados.0.smithi100.stdout:898: seq_num 277 ranges {761856=647168,1409024=557056,1966080=516096,2482176=532480} 2024-04-07T16:19:18.930 INFO:tasks.rados.rados.0.smithi100.stdout:898: writing smithi10066867-11 from 761856 to 1409024 tid 1 2024-04-07T16:19:18.936 INFO:tasks.rados.rados.0.smithi100.stdout:898: writing smithi10066867-11 from 1409024 to 1966080 tid 2 2024-04-07T16:19:18.940 INFO:tasks.rados.rados.0.smithi100.stdout:898: writing smithi10066867-11 from 1966080 to 2482176 tid 3 2024-04-07T16:19:18.945 INFO:tasks.rados.rados.0.smithi100.stdout:898: writing smithi10066867-11 from 2482176 to 3014656 tid 4 2024-04-07T16:19:18.945 INFO:tasks.rados.rados.0.smithi100.stdout:899: snap_remove snap 74 2024-04-07T16:19:19.015 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:19:18 smithi144 bash[51952]: debug 2024-04-07T16:19:18.661+0000 7f86dea6a200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-07T16:19:19.015 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:19:18 smithi144 bash[51952]: debug 2024-04-07T16:19:18.777+0000 7f86dea6a200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-07T16:19:19.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:19 smithi144 bash[23522]: cluster 2024-04-07T16:19:18.008900+0000 mgr.y (mgr.25073) 7 : cluster 0 pgmap v9: 121 pgs: 121 active+clean; 174 MiB data, 1017 MiB used, 714 GiB / 715 GiB avail 2024-04-07T16:19:19.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:19 smithi144 bash[23522]: cluster 2024-04-07T16:19:18.885079+0000 mon.a (mon.0) 1130 : cluster 0 osdmap e226: 8 total, 8 up, 8 in 2024-04-07T16:19:19.355 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:19:19 smithi144 bash[51952]: debug 2024-04-07T16:19:19.069+0000 7f86dea6a200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-07T16:19:19.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:19 smithi100 bash[21346]: cluster 2024-04-07T16:19:18.008900+0000 mgr.y (mgr.25073) 7 : cluster 0 pgmap v9: 121 pgs: 121 active+clean; 174 MiB data, 1017 MiB used, 714 GiB / 715 GiB avail 2024-04-07T16:19:19.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:19 smithi100 bash[21346]: cluster 2024-04-07T16:19:18.885079+0000 mon.a (mon.0) 1130 : cluster 0 osdmap e226: 8 total, 8 up, 8 in 2024-04-07T16:19:19.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:19 smithi100 bash[31660]: cluster 2024-04-07T16:19:18.008900+0000 mgr.y (mgr.25073) 7 : cluster 0 pgmap v9: 121 pgs: 121 active+clean; 174 MiB data, 1017 MiB used, 714 GiB / 715 GiB avail 2024-04-07T16:19:19.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:19 smithi100 bash[31660]: cluster 2024-04-07T16:19:18.885079+0000 mon.a (mon.0) 1130 : cluster 0 osdmap e226: 8 total, 8 up, 8 in 2024-04-07T16:19:19.745 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:19:19 smithi144 bash[51952]: debug 2024-04-07T16:19:19.493+0000 7f86dea6a200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-07T16:19:19.890 INFO:tasks.rados.rados.0.smithi100.stdout:899: done (6 left) 2024-04-07T16:19:19.890 INFO:tasks.rados.rados.0.smithi100.stdout:900: setattr oid 4 current snap is 80 2024-04-07T16:19:19.890 INFO:tasks.rados.rados.0.smithi100.stdout:893: finishing write tid 3 to smithi10066867-10 2024-04-07T16:19:19.890 INFO:tasks.rados.rados.0.smithi100.stdout:895: finishing copy_from to smithi10066867-36 2024-04-07T16:19:19.890 INFO:tasks.rados.rados.0.smithi100.stdout:895: got expected ENOENT (src dne) 2024-04-07T16:19:19.890 INFO:tasks.rados.rados.0.smithi100.stdout:895: finishing copy_from racing read to smithi10066867-36 2024-04-07T16:19:19.890 INFO:tasks.rados.rados.0.smithi100.stdout:893: finishing write tid 4 to smithi10066867-10 2024-04-07T16:19:19.890 INFO:tasks.rados.rados.0.smithi100.stdout:893: finishing write tid 5 to smithi10066867-10 2024-04-07T16:19:19.890 INFO:tasks.rados.rados.0.smithi100.stdout:893: finishing write tid 6 to smithi10066867-10 2024-04-07T16:19:19.890 INFO:tasks.rados.rados.0.smithi100.stdout:893: oid 10 updating version 0 to 223 2024-04-07T16:19:19.890 INFO:tasks.rados.rados.0.smithi100.stdout:893: oid 10 version 223 is already newer than 221 2024-04-07T16:19:19.890 INFO:tasks.rados.rados.0.smithi100.stdout:893: oid 10 version 223 is already newer than 220 2024-04-07T16:19:19.891 INFO:tasks.rados.rados.0.smithi100.stdout:893: oid 10 version 223 is already newer than 222 2024-04-07T16:19:19.891 INFO:tasks.rados.rados.0.smithi100.stdout:893: oid 10 updating version 223 to 224 2024-04-07T16:19:19.891 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 10 v 224 (ObjNum 276 snap 80 seq_num 276) dirty exists 2024-04-07T16:19:19.891 INFO:tasks.rados.rados.0.smithi100.stdout:893: left oid 10 (ObjNum 276 snap 80 seq_num 276) 2024-04-07T16:19:19.891 INFO:tasks.rados.rados.0.smithi100.stdout:898: finishing write tid 1 to smithi10066867-11 2024-04-07T16:19:19.891 INFO:tasks.rados.rados.0.smithi100.stdout:898: finishing write tid 2 to smithi10066867-11 2024-04-07T16:19:19.891 INFO:tasks.rados.rados.0.smithi100.stdout:896: expect (ObjNum 162 snap 43 seq_num 162) 2024-04-07T16:19:20.099 INFO:tasks.rados.rados.0.smithi100.stdout:898: finishing write tid 3 to smithi10066867-11 2024-04-07T16:19:20.099 INFO:tasks.rados.rados.0.smithi100.stdout:898: finishing write tid 4 to smithi10066867-11 2024-04-07T16:19:20.099 INFO:tasks.rados.rados.0.smithi100.stdout:898: finishing write tid 5 to smithi10066867-11 2024-04-07T16:19:20.099 INFO:tasks.rados.rados.0.smithi100.stdout:898: finishing write tid 6 to smithi10066867-11 2024-04-07T16:19:20.100 INFO:tasks.rados.rados.0.smithi100.stdout:898: oid 11 updating version 0 to 535 2024-04-07T16:19:20.100 INFO:tasks.rados.rados.0.smithi100.stdout:898: oid 11 version 535 is already newer than 534 2024-04-07T16:19:20.100 INFO:tasks.rados.rados.0.smithi100.stdout:898: oid 11 updating version 535 to 538 2024-04-07T16:19:20.100 INFO:tasks.rados.rados.0.smithi100.stdout:898: oid 11 version 538 is already newer than 537 2024-04-07T16:19:20.100 INFO:tasks.rados.rados.0.smithi100.stdout:898: oid 11 version 538 is already newer than 536 2024-04-07T16:19:20.100 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 11 v 538 (ObjNum 277 snap 80 seq_num 277) dirty exists 2024-04-07T16:19:20.100 INFO:tasks.rados.rados.0.smithi100.stdout:898: left oid 11 (ObjNum 277 snap 80 seq_num 277) 2024-04-07T16:19:20.100 INFO:tasks.rados.rados.0.smithi100.stdout:897: expect (ObjNum 212 snap 57 seq_num 212) 2024-04-07T16:19:20.104 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:19:19 smithi144 bash[51952]: debug 2024-04-07T16:19:19.753+0000 7f86dea6a200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-07T16:19:20.104 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:19:19 smithi144 bash[51952]: debug 2024-04-07T16:19:19.881+0000 7f86dea6a200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-07T16:19:20.420 INFO:tasks.rados.rados.0.smithi100.stdout:894: finishing copy_from racing read to smithi10066867-5 2024-04-07T16:19:20.420 INFO:tasks.rados.rados.0.smithi100.stdout:894: finishing copy_from to smithi10066867-5 2024-04-07T16:19:20.420 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 5 v 388 (ObjNum 118 snap 29 seq_num 118) dirty exists 2024-04-07T16:19:20.421 INFO:tasks.rados.rados.0.smithi100.stdout:893: done (6 left) 2024-04-07T16:19:20.421 INFO:tasks.rados.rados.0.smithi100.stdout:894: done (5 left) 2024-04-07T16:19:20.421 INFO:tasks.rados.rados.0.smithi100.stdout:895: done (4 left) 2024-04-07T16:19:20.421 INFO:tasks.rados.rados.0.smithi100.stdout:896: done (3 left) 2024-04-07T16:19:20.421 INFO:tasks.rados.rados.0.smithi100.stdout:897: done (2 left) 2024-04-07T16:19:20.421 INFO:tasks.rados.rados.0.smithi100.stdout:898: done (1 left) 2024-04-07T16:19:20.421 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 32 current snap is 80 2024-04-07T16:19:20.421 INFO:tasks.rados.rados.0.smithi100.stdout:901: seq_num 278 ranges {2785280=663552,3448832=442368,3891200=720896,4612096=442368} 2024-04-07T16:19:20.462 INFO:tasks.rados.rados.0.smithi100.stdout:901: writing smithi10066867-32 from 2785280 to 3448832 tid 1 2024-04-07T16:19:20.468 INFO:tasks.rados.rados.0.smithi100.stdout:901: writing smithi10066867-32 from 3448832 to 3891200 tid 2 2024-04-07T16:19:20.476 INFO:tasks.rados.rados.0.smithi100.stdout:901: writing smithi10066867-32 from 3891200 to 4612096 tid 3 2024-04-07T16:19:20.481 INFO:tasks.rados.rados.0.smithi100.stdout:901: writing smithi10066867-32 from 4612096 to 5054464 tid 4 2024-04-07T16:19:20.481 INFO:tasks.rados.rados.0.smithi100.stdout:902: delete oid 43 current snap is 80 2024-04-07T16:19:20.481 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 4 v 546 (ObjNum 171 snap 44 seq_num 171) dirty exists 2024-04-07T16:19:20.481 INFO:tasks.rados.rados.0.smithi100.stdout:901: finishing write tid 1 to smithi10066867-32 2024-04-07T16:19:20.481 INFO:tasks.rados.rados.0.smithi100.stdout:901: finishing write tid 2 to smithi10066867-32 2024-04-07T16:19:20.481 INFO:tasks.rados.rados.0.smithi100.stdout:901: finishing write tid 3 to smithi10066867-32 2024-04-07T16:19:20.484 INFO:tasks.rados.rados.0.smithi100.stdout:901: finishing write tid 4 to smithi10066867-32 2024-04-07T16:19:20.484 INFO:tasks.rados.rados.0.smithi100.stdout:901: finishing write tid 5 to smithi10066867-32 2024-04-07T16:19:20.485 INFO:tasks.rados.rados.0.smithi100.stdout:900: done (2 left) 2024-04-07T16:19:20.485 INFO:tasks.rados.rados.0.smithi100.stdout:902: done (1 left) 2024-04-07T16:19:20.485 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 48 current snap is 80 2024-04-07T16:19:20.485 INFO:tasks.rados.rados.0.smithi100.stdout:903: seq_num 280 ranges {1884160=696320,2580480=606208,3186688=688128,3874816=311296} 2024-04-07T16:19:20.512 INFO:tasks.rados.rados.0.smithi100.stdout:903: writing smithi10066867-48 from 1884160 to 2580480 tid 1 2024-04-07T16:19:20.518 INFO:tasks.rados.rados.0.smithi100.stdout:903: writing smithi10066867-48 from 2580480 to 3186688 tid 2 2024-04-07T16:19:20.525 INFO:tasks.rados.rados.0.smithi100.stdout:903: writing smithi10066867-48 from 3186688 to 3874816 tid 3 2024-04-07T16:19:20.528 INFO:tasks.rados.rados.0.smithi100.stdout:903: writing smithi10066867-48 from 3874816 to 4186112 tid 4 2024-04-07T16:19:20.528 INFO:tasks.rados.rados.0.smithi100.stdout:904: snap_create 2024-04-07T16:19:20.528 INFO:tasks.rados.rados.0.smithi100.stdout:901: finishing write tid 6 to smithi10066867-32 2024-04-07T16:19:20.528 INFO:tasks.rados.rados.0.smithi100.stdout:901: oid 32 updating version 0 to 122 2024-04-07T16:19:20.528 INFO:tasks.rados.rados.0.smithi100.stdout:901: oid 32 updating version 122 to 123 2024-04-07T16:19:20.528 INFO:tasks.rados.rados.0.smithi100.stdout:901: oid 32 updating version 123 to 126 2024-04-07T16:19:20.528 INFO:tasks.rados.rados.0.smithi100.stdout:901: oid 32 version 126 is already newer than 125 2024-04-07T16:19:20.528 INFO:tasks.rados.rados.0.smithi100.stdout:901: oid 32 version 126 is already newer than 124 2024-04-07T16:19:20.529 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 32 v 126 (ObjNum 278 snap 80 seq_num 278) dirty exists 2024-04-07T16:19:20.529 INFO:tasks.rados.rados.0.smithi100.stdout:901: left oid 32 (ObjNum 278 snap 80 seq_num 278) 2024-04-07T16:19:20.529 INFO:tasks.rados.rados.0.smithi100.stdout:903: finishing write tid 1 to smithi10066867-48 2024-04-07T16:19:20.529 INFO:tasks.rados.rados.0.smithi100.stdout:903: finishing write tid 2 to smithi10066867-48 2024-04-07T16:19:20.535 INFO:tasks.rados.rados.0.smithi100.stdout:903: finishing write tid 3 to smithi10066867-48 2024-04-07T16:19:20.535 INFO:tasks.rados.rados.0.smithi100.stdout:903: finishing write tid 4 to smithi10066867-48 2024-04-07T16:19:20.535 INFO:tasks.rados.rados.0.smithi100.stdout:903: finishing write tid 5 to smithi10066867-48 2024-04-07T16:19:20.537 INFO:tasks.rados.rados.0.smithi100.stdout:903: finishing write tid 6 to smithi10066867-48 2024-04-07T16:19:20.537 INFO:tasks.rados.rados.0.smithi100.stdout:903: oid 48 updating version 0 to 250 2024-04-07T16:19:20.537 INFO:tasks.rados.rados.0.smithi100.stdout:903: oid 48 updating version 250 to 253 2024-04-07T16:19:20.537 INFO:tasks.rados.rados.0.smithi100.stdout:903: oid 48 version 253 is already newer than 251 2024-04-07T16:19:20.537 INFO:tasks.rados.rados.0.smithi100.stdout:903: oid 48 version 253 is already newer than 252 2024-04-07T16:19:20.537 INFO:tasks.rados.rados.0.smithi100.stdout:903: oid 48 updating version 253 to 254 2024-04-07T16:19:20.537 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 48 v 254 (ObjNum 280 snap 80 seq_num 280) dirty exists 2024-04-07T16:19:20.537 INFO:tasks.rados.rados.0.smithi100.stdout:903: left oid 48 (ObjNum 280 snap 80 seq_num 280) 2024-04-07T16:19:20.605 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:19:20 smithi144 bash[51952]: debug 2024-04-07T16:19:20.169+0000 7f86dea6a200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-07T16:19:21.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:20 smithi100 bash[31660]: cluster 2024-04-07T16:19:19.894131+0000 mon.a (mon.0) 1131 : cluster 0 osdmap e227: 8 total, 8 up, 8 in 2024-04-07T16:19:21.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:20 smithi100 bash[31660]: cluster 2024-04-07T16:19:20.009344+0000 mgr.y (mgr.25073) 8 : cluster 0 pgmap v12: 121 pgs: 121 active+clean; 174 MiB data, 1017 MiB used, 714 GiB / 715 GiB avail 2024-04-07T16:19:21.188 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:20 smithi100 bash[31660]: audit 2024-04-07T16:19:20.273283+0000 mon.a (mon.0) 1132 : audit 1 from='mgr.25073 ' entity='mgr.y' 2024-04-07T16:19:21.188 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:20 smithi100 bash[31660]: audit 2024-04-07T16:19:20.279391+0000 mon.a (mon.0) 1133 : audit 1 from='mgr.25073 ' entity='mgr.y' 2024-04-07T16:19:21.188 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:20 smithi100 bash[31660]: audit 2024-04-07T16:19:20.550492+0000 mon.a (mon.0) 1134 : audit 1 from='mgr.25073 ' entity='mgr.y' 2024-04-07T16:19:21.188 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:20 smithi100 bash[31660]: audit 2024-04-07T16:19:20.560279+0000 mon.a (mon.0) 1135 : audit 1 from='mgr.25073 ' entity='mgr.y' 2024-04-07T16:19:21.188 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:20 smithi100 bash[21346]: cluster 2024-04-07T16:19:19.894131+0000 mon.a (mon.0) 1131 : cluster 0 osdmap e227: 8 total, 8 up, 8 in 2024-04-07T16:19:21.188 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:20 smithi100 bash[21346]: cluster 2024-04-07T16:19:20.009344+0000 mgr.y (mgr.25073) 8 : cluster 0 pgmap v12: 121 pgs: 121 active+clean; 174 MiB data, 1017 MiB used, 714 GiB / 715 GiB avail 2024-04-07T16:19:21.188 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:20 smithi100 bash[21346]: audit 2024-04-07T16:19:20.273283+0000 mon.a (mon.0) 1132 : audit 1 from='mgr.25073 ' entity='mgr.y' 2024-04-07T16:19:21.188 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:20 smithi100 bash[21346]: audit 2024-04-07T16:19:20.279391+0000 mon.a (mon.0) 1133 : audit 1 from='mgr.25073 ' entity='mgr.y' 2024-04-07T16:19:21.188 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:20 smithi100 bash[21346]: audit 2024-04-07T16:19:20.550492+0000 mon.a (mon.0) 1134 : audit 1 from='mgr.25073 ' entity='mgr.y' 2024-04-07T16:19:21.188 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:20 smithi100 bash[21346]: audit 2024-04-07T16:19:20.560279+0000 mon.a (mon.0) 1135 : audit 1 from='mgr.25073 ' entity='mgr.y' 2024-04-07T16:19:21.237 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:20 smithi144 bash[23522]: cluster 2024-04-07T16:19:19.894131+0000 mon.a (mon.0) 1131 : cluster 0 osdmap e227: 8 total, 8 up, 8 in 2024-04-07T16:19:21.237 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:20 smithi144 bash[23522]: cluster 2024-04-07T16:19:20.009344+0000 mgr.y (mgr.25073) 8 : cluster 0 pgmap v12: 121 pgs: 121 active+clean; 174 MiB data, 1017 MiB used, 714 GiB / 715 GiB avail 2024-04-07T16:19:21.237 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:20 smithi144 bash[23522]: audit 2024-04-07T16:19:20.273283+0000 mon.a (mon.0) 1132 : audit 1 from='mgr.25073 ' entity='mgr.y' 2024-04-07T16:19:21.237 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:20 smithi144 bash[23522]: audit 2024-04-07T16:19:20.279391+0000 mon.a (mon.0) 1133 : audit 1 from='mgr.25073 ' entity='mgr.y' 2024-04-07T16:19:21.237 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:20 smithi144 bash[23522]: audit 2024-04-07T16:19:20.550492+0000 mon.a (mon.0) 1134 : audit 1 from='mgr.25073 ' entity='mgr.y' 2024-04-07T16:19:21.237 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:20 smithi144 bash[23522]: audit 2024-04-07T16:19:20.560279+0000 mon.a (mon.0) 1135 : audit 1 from='mgr.25073 ' entity='mgr.y' 2024-04-07T16:19:21.270 INFO:tasks.rados.rados.0.smithi100.stdout:901: done (2 left) 2024-04-07T16:19:21.270 INFO:tasks.rados.rados.0.smithi100.stdout:903: done (1 left) 2024-04-07T16:19:21.270 INFO:tasks.rados.rados.0.smithi100.stdout:904: done (0 left) 2024-04-07T16:19:21.271 INFO:tasks.rados.rados.0.smithi100.stdout:905: snap_remove snap 78 2024-04-07T16:19:21.605 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:19:21 smithi144 bash[51952]: debug 2024-04-07T16:19:21.245+0000 7f86dea6a200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-07T16:19:21.605 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:19:21 smithi144 bash[51952]: debug 2024-04-07T16:19:21.369+0000 7f86dea6a200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-07T16:19:21.924 INFO:teuthology.orchestra.run.smithi100.stdout:true 2024-04-07T16:19:22.105 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:19:21 smithi144 bash[51952]: debug 2024-04-07T16:19:21.685+0000 7f86dea6a200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-07T16:19:22.105 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:19:21 smithi144 bash[51952]: debug 2024-04-07T16:19:21.801+0000 7f86dea6a200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-07T16:19:22.273 INFO:tasks.rados.rados.0.smithi100.stdout:905: done (0 left) 2024-04-07T16:19:22.274 INFO:tasks.rados.rados.0.smithi100.stdout:906: rmattr oid 46 current snap is 81 2024-04-07T16:19:22.274 INFO:tasks.rados.rados.0.smithi100.stdout:906: done (0 left) 2024-04-07T16:19:22.274 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 21 current snap is 81 2024-04-07T16:19:22.274 INFO:tasks.rados.rados.0.smithi100.stdout:907: seq_num 281 ranges {6332416=712704,7045120=491520,7536640=491520,8028160=622592} 2024-04-07T16:19:22.284 INFO:teuthology.orchestra.run.smithi100.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-07T16:19:22.284 INFO:teuthology.orchestra.run.smithi100.stdout:alertmanager.a smithi100 *:9093,9094 running (5m) 2s ago 6m 14.7M - 0.25.0 c8568f914cd2 5143075e7d26 2024-04-07T16:19:22.284 INFO:teuthology.orchestra.run.smithi100.stdout:grafana.a smithi144 *:3000 running (5m) 1s ago 5m 79.8M - 9.4.7 954c08fa6188 7180c0708acd 2024-04-07T16:19:22.284 INFO:teuthology.orchestra.run.smithi100.stdout:mds.a smithi100 running (7m) 2s ago 7m 14.2M - 18.2.2-1171-g267c8ff5 8a4a01e7186c 92593b0c28ae 2024-04-07T16:19:22.284 INFO:teuthology.orchestra.run.smithi100.stdout:mds.b smithi144 running (7m) 1s ago 7m 17.1M - 18.2.2-1171-g267c8ff5 8a4a01e7186c 3df2ee3e17a4 2024-04-07T16:19:22.284 INFO:teuthology.orchestra.run.smithi100.stdout:mgr.x smithi144 *:8443,9283,8765 running (78s) 1s ago 14m 199M - 19.0.0-2785-gadce92d0 c0ecd4226ac9 45451b348d34 2024-04-07T16:19:22.284 INFO:teuthology.orchestra.run.smithi100.stdout:mgr.y smithi100 *:8443,9283,8765 running (25s) 2s ago 16m 471M - 19.0.0-2785-gadce92d0 c0ecd4226ac9 254dc65cdaa7 2024-04-07T16:19:22.284 INFO:teuthology.orchestra.run.smithi100.stdout:mon.a smithi100 running (16m) 2s ago 17m 59.4M 2048M 18.2.2-1171-g267c8ff5 8a4a01e7186c ec8833f59ea0 2024-04-07T16:19:22.284 INFO:teuthology.orchestra.run.smithi100.stdout:mon.b smithi144 running (14m) 1s ago 14m 55.1M 2048M 18.2.2-1171-g267c8ff5 8a4a01e7186c 8dd687691b2e 2024-04-07T16:19:22.284 INFO:teuthology.orchestra.run.smithi100.stdout:mon.c smithi100 running (14m) 2s ago 14m 52.1M 2048M 18.2.2-1171-g267c8ff5 8a4a01e7186c a6de0151a3ba 2024-04-07T16:19:22.285 INFO:teuthology.orchestra.run.smithi100.stdout:node-exporter.a smithi100 *:9100 running (6m) 2s ago 6m 9295k - 1.5.0 0da6a335fe13 4bce264c1b26 2024-04-07T16:19:22.285 INFO:teuthology.orchestra.run.smithi100.stdout:node-exporter.b smithi144 *:9100 running (6m) 1s ago 6m 8647k - 1.5.0 0da6a335fe13 b6963413c560 2024-04-07T16:19:22.285 INFO:teuthology.orchestra.run.smithi100.stdout:osd.0 smithi100 running (13m) 2s ago 13m 182M 2524M 18.2.2-1171-g267c8ff5 8a4a01e7186c 188deb247979 2024-04-07T16:19:22.285 INFO:teuthology.orchestra.run.smithi100.stdout:osd.1 smithi100 running (12m) 2s ago 12m 148M 2524M 18.2.2-1171-g267c8ff5 8a4a01e7186c 7f57ca90894c 2024-04-07T16:19:22.285 INFO:teuthology.orchestra.run.smithi100.stdout:osd.2 smithi100 running (12m) 2s ago 12m 174M 2524M 18.2.2-1171-g267c8ff5 8a4a01e7186c f3912ddcc8cc 2024-04-07T16:19:22.285 INFO:teuthology.orchestra.run.smithi100.stdout:osd.3 smithi100 running (11m) 2s ago 11m 114M 2524M 18.2.2-1171-g267c8ff5 8a4a01e7186c b41004090c63 2024-04-07T16:19:22.285 INFO:teuthology.orchestra.run.smithi100.stdout:osd.4 smithi144 running (10m) 1s ago 10m 159M 2524M 18.2.2-1171-g267c8ff5 8a4a01e7186c 8c73b6b4dc13 2024-04-07T16:19:22.285 INFO:teuthology.orchestra.run.smithi100.stdout:osd.5 smithi144 running (9m) 1s ago 9m 151M 2524M 18.2.2-1171-g267c8ff5 8a4a01e7186c 3bfa0cf6f6cb 2024-04-07T16:19:22.285 INFO:teuthology.orchestra.run.smithi100.stdout:osd.6 smithi144 running (8m) 1s ago 8m 72.6M 2524M 18.2.2-1171-g267c8ff5 8a4a01e7186c 32b7f95d77f0 2024-04-07T16:19:22.285 INFO:teuthology.orchestra.run.smithi100.stdout:osd.7 smithi144 running (7m) 1s ago 8m 102M 2524M 18.2.2-1171-g267c8ff5 8a4a01e7186c 45f7709e576a 2024-04-07T16:19:22.285 INFO:teuthology.orchestra.run.smithi100.stdout:prometheus.a smithi144 *:9095 running (31s) 1s ago 6m 35.4M - 2.43.0 a07b618ecd1d 67df1391f57e 2024-04-07T16:19:22.313 INFO:tasks.rados.rados.0.smithi100.stdout:907: writing smithi10066867-21 from 6332416 to 7045120 tid 1 2024-04-07T16:19:22.316 INFO:tasks.rados.rados.0.smithi100.stdout:907: writing smithi10066867-21 from 7045120 to 7536640 tid 2 2024-04-07T16:19:22.319 INFO:tasks.rados.rados.0.smithi100.stdout:907: writing smithi10066867-21 from 7536640 to 8028160 tid 3 2024-04-07T16:19:22.322 INFO:tasks.rados.rados.0.smithi100.stdout:907: writing smithi10066867-21 from 8028160 to 8650752 tid 4 2024-04-07T16:19:22.322 INFO:tasks.rados.rados.0.smithi100.stdout:908: delete oid 39 current snap is 81 2024-04-07T16:19:22.323 INFO:tasks.rados.rados.0.smithi100.stdout:907: finishing write tid 1 to smithi10066867-21 2024-04-07T16:19:22.324 INFO:tasks.rados.rados.0.smithi100.stdout:907: finishing write tid 2 to smithi10066867-21 2024-04-07T16:19:22.326 INFO:tasks.rados.rados.0.smithi100.stdout:908: done (1 left) 2024-04-07T16:19:22.326 INFO:tasks.rados.rados.0.smithi100.stdout:909: read oid 49 snap -1 2024-04-07T16:19:22.327 INFO:tasks.rados.rados.0.smithi100.stdout:909: expect (ObjNum 260 snap 70 seq_num 260) 2024-04-07T16:19:22.327 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 28 current snap is 81 2024-04-07T16:19:22.327 INFO:tasks.rados.rados.0.smithi100.stdout:910: seq_num 283 ranges {2359296=745472,3104768=712704,3817472=458752,4276224=434176} 2024-04-07T16:19:22.344 INFO:tasks.rados.rados.0.smithi100.stdout:910: writing smithi10066867-28 from 2359296 to 3104768 tid 1 2024-04-07T16:19:22.348 INFO:tasks.rados.rados.0.smithi100.stdout:910: writing smithi10066867-28 from 3104768 to 3817472 tid 2 2024-04-07T16:19:22.350 INFO:tasks.rados.rados.0.smithi100.stdout:910: writing smithi10066867-28 from 3817472 to 4276224 tid 3 2024-04-07T16:19:22.354 INFO:tasks.rados.rados.0.smithi100.stdout:910: writing smithi10066867-28 from 4276224 to 4710400 tid 4 2024-04-07T16:19:22.354 INFO:tasks.rados.rados.0.smithi100.stdout:911: read oid 39 snap -1 2024-04-07T16:19:22.354 INFO:tasks.rados.rados.0.smithi100.stdout:911: expect deleted 2024-04-07T16:19:22.354 INFO:tasks.rados.rados.0.smithi100.stdout:912: rmattr oid 12 current snap is 81 2024-04-07T16:19:22.354 INFO:tasks.rados.rados.0.smithi100.stdout:907: finishing write tid 3 to smithi10066867-21 2024-04-07T16:19:22.354 INFO:tasks.rados.rados.0.smithi100.stdout:907: finishing write tid 4 to smithi10066867-21 2024-04-07T16:19:22.354 INFO:tasks.rados.rados.0.smithi100.stdout:907: finishing write tid 5 to smithi10066867-21 2024-04-07T16:19:22.354 INFO:tasks.rados.rados.0.smithi100.stdout:907: finishing write tid 6 to smithi10066867-21 2024-04-07T16:19:22.354 INFO:tasks.rados.rados.0.smithi100.stdout:907: oid 21 updating version 0 to 151 2024-04-07T16:19:22.355 INFO:tasks.rados.rados.0.smithi100.stdout:907: oid 21 version 151 is already newer than 149 2024-04-07T16:19:22.355 INFO:tasks.rados.rados.0.smithi100.stdout:907: oid 21 updating version 151 to 152 2024-04-07T16:19:22.355 INFO:tasks.rados.rados.0.smithi100.stdout:907: oid 21 updating version 152 to 153 2024-04-07T16:19:22.355 INFO:tasks.rados.rados.0.smithi100.stdout:907: oid 21 version 153 is already newer than 150 2024-04-07T16:19:22.355 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 21 v 153 (ObjNum 281 snap 81 seq_num 281) dirty exists 2024-04-07T16:19:22.355 INFO:tasks.rados.rados.0.smithi100.stdout:907: left oid 21 (ObjNum 281 snap 81 seq_num 281) 2024-04-07T16:19:22.355 INFO:tasks.rados.rados.0.smithi100.stdout:909: expect (ObjNum 260 snap 70 seq_num 260) 2024-04-07T16:19:22.438 INFO:tasks.rados.rados.0.smithi100.stdout:910: finishing write tid 1 to smithi10066867-28 2024-04-07T16:19:22.438 INFO:tasks.rados.rados.0.smithi100.stdout:910: finishing write tid 2 to smithi10066867-28 2024-04-07T16:19:22.438 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 12 v 500 (ObjNum 248 snap 67 seq_num 248) dirty exists 2024-04-07T16:19:22.438 INFO:tasks.rados.rados.0.smithi100.stdout:910: finishing write tid 3 to smithi10066867-28 2024-04-07T16:19:22.438 INFO:tasks.rados.rados.0.smithi100.stdout:910: finishing write tid 4 to smithi10066867-28 2024-04-07T16:19:22.438 INFO:tasks.rados.rados.0.smithi100.stdout:910: finishing write tid 5 to smithi10066867-28 2024-04-07T16:19:22.438 INFO:tasks.rados.rados.0.smithi100.stdout:910: finishing write tid 6 to smithi10066867-28 2024-04-07T16:19:22.438 INFO:tasks.rados.rados.0.smithi100.stdout:910: oid 28 updating version 0 to 130 2024-04-07T16:19:22.438 INFO:tasks.rados.rados.0.smithi100.stdout:910: oid 28 updating version 130 to 132 2024-04-07T16:19:22.438 INFO:tasks.rados.rados.0.smithi100.stdout:910: oid 28 version 132 is already newer than 131 2024-04-07T16:19:22.438 INFO:tasks.rados.rados.0.smithi100.stdout:910: oid 28 updating version 132 to 133 2024-04-07T16:19:22.438 INFO:tasks.rados.rados.0.smithi100.stdout:910: oid 28 updating version 133 to 134 2024-04-07T16:19:22.438 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 28 v 134 (ObjNum 283 snap 81 seq_num 283) dirty exists 2024-04-07T16:19:22.439 INFO:tasks.rados.rados.0.smithi100.stdout:910: left oid 28 (ObjNum 283 snap 81 seq_num 283) 2024-04-07T16:19:22.439 INFO:tasks.rados.rados.0.smithi100.stdout:907: done (4 left) 2024-04-07T16:19:22.439 INFO:tasks.rados.rados.0.smithi100.stdout:909: done (3 left) 2024-04-07T16:19:22.439 INFO:tasks.rados.rados.0.smithi100.stdout:910: done (2 left) 2024-04-07T16:19:22.439 INFO:tasks.rados.rados.0.smithi100.stdout:911: done (1 left) 2024-04-07T16:19:22.439 INFO:tasks.rados.rados.0.smithi100.stdout:912: done (0 left) 2024-04-07T16:19:22.439 INFO:tasks.rados.rados.0.smithi100.stdout:913: delete oid 26 current snap is 81 2024-04-07T16:19:22.439 INFO:tasks.rados.rados.0.smithi100.stdout:913: done (0 left) 2024-04-07T16:19:22.439 INFO:tasks.rados.rados.0.smithi100.stdout:914: copy_from oid 50 from oid 30 current snap is 81 2024-04-07T16:19:22.440 INFO:tasks.rados.rados.0.smithi100.stdout:915: snap_remove snap 76 2024-04-07T16:19:22.521 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:19:22 smithi144 bash[51952]: debug 2024-04-07T16:19:22.256+0000 7f86dea6a200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-07T16:19:22.521 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:19:22 smithi144 bash[51952]: debug 2024-04-07T16:19:22.400+0000 7f86dea6a200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-07T16:19:22.521 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:19:22 smithi144 bash[51952]: [07/Apr/2024:16:19:22] ENGINE Bus STARTING 2024-04-07T16:19:22.522 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:22 smithi144 bash[23522]: cluster 2024-04-07T16:19:21.282866+0000 mon.a (mon.0) 1136 : cluster 0 osdmap e228: 8 total, 8 up, 8 in 2024-04-07T16:19:22.522 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:22 smithi144 bash[23522]: audit 2024-04-07T16:19:21.392247+0000 mon.a (mon.0) 1137 : audit 1 from='mgr.25073 ' entity='mgr.y' 2024-04-07T16:19:22.522 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:22 smithi144 bash[23522]: audit 2024-04-07T16:19:21.402410+0000 mon.a (mon.0) 1138 : audit 1 from='mgr.25073 ' entity='mgr.y' 2024-04-07T16:19:22.522 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:22 smithi144 bash[23522]: audit 2024-04-07T16:19:21.675411+0000 mon.a (mon.0) 1139 : audit 1 from='mgr.25073 ' entity='mgr.y' 2024-04-07T16:19:22.522 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:22 smithi144 bash[23522]: audit 2024-04-07T16:19:21.682330+0000 mon.a (mon.0) 1140 : audit 1 from='mgr.25073 ' entity='mgr.y' 2024-04-07T16:19:22.687 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:22 smithi100 bash[21346]: cluster 2024-04-07T16:19:21.282866+0000 mon.a (mon.0) 1136 : cluster 0 osdmap e228: 8 total, 8 up, 8 in 2024-04-07T16:19:22.687 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:22 smithi100 bash[21346]: audit 2024-04-07T16:19:21.392247+0000 mon.a (mon.0) 1137 : audit 1 from='mgr.25073 ' entity='mgr.y' 2024-04-07T16:19:22.687 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:22 smithi100 bash[21346]: audit 2024-04-07T16:19:21.402410+0000 mon.a (mon.0) 1138 : audit 1 from='mgr.25073 ' entity='mgr.y' 2024-04-07T16:19:22.687 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:22 smithi100 bash[21346]: audit 2024-04-07T16:19:21.675411+0000 mon.a (mon.0) 1139 : audit 1 from='mgr.25073 ' entity='mgr.y' 2024-04-07T16:19:22.687 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:22 smithi100 bash[21346]: audit 2024-04-07T16:19:21.682330+0000 mon.a (mon.0) 1140 : audit 1 from='mgr.25073 ' entity='mgr.y' 2024-04-07T16:19:22.687 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:22 smithi100 bash[31660]: cluster 2024-04-07T16:19:21.282866+0000 mon.a (mon.0) 1136 : cluster 0 osdmap e228: 8 total, 8 up, 8 in 2024-04-07T16:19:22.687 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:22 smithi100 bash[31660]: audit 2024-04-07T16:19:21.392247+0000 mon.a (mon.0) 1137 : audit 1 from='mgr.25073 ' entity='mgr.y' 2024-04-07T16:19:22.687 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:22 smithi100 bash[31660]: audit 2024-04-07T16:19:21.402410+0000 mon.a (mon.0) 1138 : audit 1 from='mgr.25073 ' entity='mgr.y' 2024-04-07T16:19:22.687 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:22 smithi100 bash[31660]: audit 2024-04-07T16:19:21.675411+0000 mon.a (mon.0) 1139 : audit 1 from='mgr.25073 ' entity='mgr.y' 2024-04-07T16:19:22.687 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:22 smithi100 bash[31660]: audit 2024-04-07T16:19:21.682330+0000 mon.a (mon.0) 1140 : audit 1 from='mgr.25073 ' entity='mgr.y' 2024-04-07T16:19:22.725 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-04-07T16:19:22.725 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": { 2024-04-07T16:19:22.726 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph version 18.2.2-1171-g267c8ff5 (267c8ff5b1ff5815581b3b2381d19360e081b37b) reef (stable)": 3 2024-04-07T16:19:22.726 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-07T16:19:22.726 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": { 2024-04-07T16:19:22.726 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph version 19.0.0-2785-gadce92d0 (adce92d056533f22dc1ee37e3141d098b046ec1b) squid (dev)": 1 2024-04-07T16:19:22.726 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-07T16:19:22.726 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": { 2024-04-07T16:19:22.726 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph version 18.2.2-1171-g267c8ff5 (267c8ff5b1ff5815581b3b2381d19360e081b37b) reef (stable)": 8 2024-04-07T16:19:22.726 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-07T16:19:22.726 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": { 2024-04-07T16:19:22.726 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph version 18.2.2-1171-g267c8ff5 (267c8ff5b1ff5815581b3b2381d19360e081b37b) reef (stable)": 2 2024-04-07T16:19:22.726 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-07T16:19:22.726 INFO:teuthology.orchestra.run.smithi100.stdout: "overall": { 2024-04-07T16:19:22.726 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph version 18.2.2-1171-g267c8ff5 (267c8ff5b1ff5815581b3b2381d19360e081b37b) reef (stable)": 13, 2024-04-07T16:19:22.726 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph version 19.0.0-2785-gadce92d0 (adce92d056533f22dc1ee37e3141d098b046ec1b) squid (dev)": 1 2024-04-07T16:19:22.727 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-04-07T16:19:22.727 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-04-07T16:19:22.855 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:19:22 smithi144 bash[51952]: CherryPy Checker: 2024-04-07T16:19:22.855 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:19:22 smithi144 bash[51952]: The Application mounted at '' has an empty config. 2024-04-07T16:19:22.855 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:19:22 smithi144 bash[51952]: [07/Apr/2024:16:19:22] ENGINE Serving on http://:::9283 2024-04-07T16:19:22.855 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:19:22 smithi144 bash[51952]: [07/Apr/2024:16:19:22] ENGINE Bus STARTED 2024-04-07T16:19:22.869 INFO:tasks.rados.rados.0.smithi100.stdout:915: done (1 left) 2024-04-07T16:19:22.869 INFO:tasks.rados.rados.0.smithi100.stdout:916: snap_create 2024-04-07T16:19:22.869 INFO:tasks.rados.rados.0.smithi100.stdout:914: finishing copy_from racing read to smithi10066867-50 2024-04-07T16:19:22.869 INFO:tasks.rados.rados.0.smithi100.stdout:914: finishing copy_from to smithi10066867-50 2024-04-07T16:19:22.869 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 50 v 446 (ObjNum 106 snap 20 seq_num 106) dirty exists 2024-04-07T16:19:23.605 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:19:23 smithi144 bash[58467]: ts=2024-04-07T16:19:23.510Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=nodes name=CephNodeDiskspaceWarning index=4 msg="Evaluating rule failed" rule="alert: CephNodeDiskspaceWarning\nexpr: predict_linear(node_filesystem_free_bytes{device=~\"/.*\"}[2d], 3600 * 24 * 5)\n * on (instance) group_left (nodename) node_uname_info < 0\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.8.4\n severity: warning\n type: ceph_default\nannotations:\n description: Mountpoint {{ $labels.mountpoint }} on {{ $labels.nodename }} will\n be full in less than 5 days based on the 48 hour trailing fill rate.\n summary: Host filesystem free space is getting low\n" err="found duplicate series for the match group {instance=\"smithi100\"} on the right hand-side of the operation: [{__name__=\"node_uname_info\", domainname=\"(none)\", instance=\"smithi100\", job=\"node\", machine=\"x86_64\", nodename=\"smithi100\", release=\"5.15.0-101-generic\", sysname=\"Linux\", version=\"#111-Ubuntu SMP Tue Mar 5 20:16:58 UTC 2024\"}, {__name__=\"node_uname_info\", cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", domainname=\"(none)\", instance=\"smithi100\", job=\"node\", machine=\"x86_64\", nodename=\"smithi100\", release=\"5.15.0-101-generic\", sysname=\"Linux\", version=\"#111-Ubuntu SMP Tue Mar 5 20:16:58 UTC 2024\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:19:23.605 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:23 smithi144 bash[23522]: audit 2024-04-07T16:19:21.926666+0000 mgr.y (mgr.25073) 9 : audit 0 from='client.15213 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:19:23.605 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:23 smithi144 bash[23522]: cluster 2024-04-07T16:19:22.009853+0000 mgr.y (mgr.25073) 10 : cluster 0 pgmap v14: 121 pgs: 121 active+clean; 186 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 8.1 MiB/s rd, 2.8 MiB/s wr, 42 op/s 2024-04-07T16:19:23.605 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:23 smithi144 bash[23522]: audit 2024-04-07T16:19:22.277265+0000 mgr.y (mgr.25073) 11 : audit 0 from='client.25124 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:19:23.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:23 smithi144 bash[23522]: cluster 2024-04-07T16:19:22.284881+0000 mon.a (mon.0) 1141 : cluster 0 osdmap e229: 8 total, 8 up, 8 in 2024-04-07T16:19:23.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:23 smithi144 bash[23522]: audit 2024-04-07T16:19:22.429871+0000 mon.b (mon.1) 325 : audit 0 from='mgr.? 172.21.15.144:0/1950494446' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-04-07T16:19:23.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:23 smithi144 bash[23522]: cluster 2024-04-07T16:19:22.431549+0000 mon.a (mon.0) 1142 : cluster 0 Standby manager daemon x started 2024-04-07T16:19:23.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:23 smithi144 bash[23522]: audit 2024-04-07T16:19:22.431949+0000 mon.b (mon.1) 326 : audit 0 from='mgr.? 172.21.15.144:0/1950494446' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-07T16:19:23.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:23 smithi144 bash[23522]: audit 2024-04-07T16:19:22.433695+0000 mon.b (mon.1) 327 : audit 0 from='mgr.? 172.21.15.144:0/1950494446' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-04-07T16:19:23.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:23 smithi144 bash[23522]: audit 2024-04-07T16:19:22.433986+0000 mon.b (mon.1) 328 : audit 0 from='mgr.? 172.21.15.144:0/1950494446' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-07T16:19:23.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:23 smithi144 bash[23522]: audit 2024-04-07T16:19:22.741584+0000 mon.a (mon.0) 1143 : audit 0 from='client.? 172.21.15.100:0/838008498' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:19:23.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:23 smithi144 bash[23522]: cluster 2024-04-07T16:19:22.882914+0000 mon.a (mon.0) 1144 : cluster 0 osdmap e230: 8 total, 8 up, 8 in 2024-04-07T16:19:23.687 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:23 smithi100 bash[21346]: audit 2024-04-07T16:19:21.926666+0000 mgr.y (mgr.25073) 9 : audit 0 from='client.15213 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:19:23.687 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:23 smithi100 bash[21346]: cluster 2024-04-07T16:19:22.009853+0000 mgr.y (mgr.25073) 10 : cluster 0 pgmap v14: 121 pgs: 121 active+clean; 186 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 8.1 MiB/s rd, 2.8 MiB/s wr, 42 op/s 2024-04-07T16:19:23.687 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:23 smithi100 bash[21346]: audit 2024-04-07T16:19:22.277265+0000 mgr.y (mgr.25073) 11 : audit 0 from='client.25124 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:19:23.687 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:23 smithi100 bash[21346]: cluster 2024-04-07T16:19:22.284881+0000 mon.a (mon.0) 1141 : cluster 0 osdmap e229: 8 total, 8 up, 8 in 2024-04-07T16:19:23.687 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:23 smithi100 bash[21346]: audit 2024-04-07T16:19:22.429871+0000 mon.b (mon.1) 325 : audit 0 from='mgr.? 172.21.15.144:0/1950494446' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-04-07T16:19:23.687 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:23 smithi100 bash[21346]: cluster 2024-04-07T16:19:22.431549+0000 mon.a (mon.0) 1142 : cluster 0 Standby manager daemon x started 2024-04-07T16:19:23.687 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:23 smithi100 bash[21346]: audit 2024-04-07T16:19:22.431949+0000 mon.b (mon.1) 326 : audit 0 from='mgr.? 172.21.15.144:0/1950494446' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-07T16:19:23.688 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:23 smithi100 bash[21346]: audit 2024-04-07T16:19:22.433695+0000 mon.b (mon.1) 327 : audit 0 from='mgr.? 172.21.15.144:0/1950494446' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-04-07T16:19:23.688 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:23 smithi100 bash[21346]: audit 2024-04-07T16:19:22.433986+0000 mon.b (mon.1) 328 : audit 0 from='mgr.? 172.21.15.144:0/1950494446' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-07T16:19:23.688 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:23 smithi100 bash[21346]: audit 2024-04-07T16:19:22.741584+0000 mon.a (mon.0) 1143 : audit 0 from='client.? 172.21.15.100:0/838008498' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:19:23.688 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:23 smithi100 bash[21346]: cluster 2024-04-07T16:19:22.882914+0000 mon.a (mon.0) 1144 : cluster 0 osdmap e230: 8 total, 8 up, 8 in 2024-04-07T16:19:23.688 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:23 smithi100 bash[31660]: audit 2024-04-07T16:19:21.926666+0000 mgr.y (mgr.25073) 9 : audit 0 from='client.15213 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:19:23.688 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:23 smithi100 bash[31660]: cluster 2024-04-07T16:19:22.009853+0000 mgr.y (mgr.25073) 10 : cluster 0 pgmap v14: 121 pgs: 121 active+clean; 186 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 8.1 MiB/s rd, 2.8 MiB/s wr, 42 op/s 2024-04-07T16:19:23.688 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:23 smithi100 bash[31660]: audit 2024-04-07T16:19:22.277265+0000 mgr.y (mgr.25073) 11 : audit 0 from='client.25124 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:19:23.688 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:23 smithi100 bash[31660]: cluster 2024-04-07T16:19:22.284881+0000 mon.a (mon.0) 1141 : cluster 0 osdmap e229: 8 total, 8 up, 8 in 2024-04-07T16:19:23.688 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:23 smithi100 bash[31660]: audit 2024-04-07T16:19:22.429871+0000 mon.b (mon.1) 325 : audit 0 from='mgr.? 172.21.15.144:0/1950494446' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-04-07T16:19:23.688 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:23 smithi100 bash[31660]: cluster 2024-04-07T16:19:22.431549+0000 mon.a (mon.0) 1142 : cluster 0 Standby manager daemon x started 2024-04-07T16:19:23.688 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:23 smithi100 bash[31660]: audit 2024-04-07T16:19:22.431949+0000 mon.b (mon.1) 326 : audit 0 from='mgr.? 172.21.15.144:0/1950494446' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-07T16:19:23.688 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:23 smithi100 bash[31660]: audit 2024-04-07T16:19:22.433695+0000 mon.b (mon.1) 327 : audit 0 from='mgr.? 172.21.15.144:0/1950494446' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-04-07T16:19:23.689 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:23 smithi100 bash[31660]: audit 2024-04-07T16:19:22.433986+0000 mon.b (mon.1) 328 : audit 0 from='mgr.? 172.21.15.144:0/1950494446' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-07T16:19:23.689 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:23 smithi100 bash[31660]: audit 2024-04-07T16:19:22.741584+0000 mon.a (mon.0) 1143 : audit 0 from='client.? 172.21.15.100:0/838008498' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:19:23.689 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:23 smithi100 bash[31660]: cluster 2024-04-07T16:19:22.882914+0000 mon.a (mon.0) 1144 : cluster 0 osdmap e230: 8 total, 8 up, 8 in 2024-04-07T16:19:23.873 INFO:tasks.rados.rados.0.smithi100.stdout:914: done (1 left) 2024-04-07T16:19:23.873 INFO:tasks.rados.rados.0.smithi100.stdout:916: done (0 left) 2024-04-07T16:19:23.873 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 21 current snap is 82 2024-04-07T16:19:23.873 INFO:tasks.rados.rados.0.smithi100.stdout:917: seq_num 285 ranges {8650752=778240,9428992=475136,9904128=475136,10379264=655360} 2024-04-07T16:19:23.938 INFO:tasks.rados.rados.0.smithi100.stdout:917: writing smithi10066867-21 from 8650752 to 9428992 tid 1 2024-04-07T16:19:23.941 INFO:tasks.rados.rados.0.smithi100.stdout:917: writing smithi10066867-21 from 9428992 to 9904128 tid 2 2024-04-07T16:19:23.944 INFO:tasks.rados.rados.0.smithi100.stdout:917: writing smithi10066867-21 from 9904128 to 10379264 tid 3 2024-04-07T16:19:23.948 INFO:tasks.rados.rados.0.smithi100.stdout:917: writing smithi10066867-21 from 10379264 to 11034624 tid 4 2024-04-07T16:19:23.948 INFO:tasks.rados.rados.0.smithi100.stdout:918: rollback oid 42 current snap is 82 2024-04-07T16:19:23.948 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 42 to 79 2024-04-07T16:19:23.948 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 47 current snap is 82 2024-04-07T16:19:23.949 INFO:tasks.rados.rados.0.smithi100.stdout:919: seq_num 286 ranges {98304=794624,892928=761856,1654784=679936,2334720=163840} 2024-04-07T16:19:23.954 INFO:tasks.rados.rados.0.smithi100.stdout:919: writing smithi10066867-47 from 98304 to 892928 tid 1 2024-04-07T16:19:23.959 INFO:tasks.rados.rados.0.smithi100.stdout:919: writing smithi10066867-47 from 892928 to 1654784 tid 2 2024-04-07T16:19:23.963 INFO:tasks.rados.rados.0.smithi100.stdout:919: writing smithi10066867-47 from 1654784 to 2334720 tid 3 2024-04-07T16:19:23.964 INFO:tasks.rados.rados.0.smithi100.stdout:919: writing smithi10066867-47 from 2334720 to 2498560 tid 4 2024-04-07T16:19:23.965 INFO:tasks.rados.rados.0.smithi100.stdout:920: copy_from oid 13 from oid 7 current snap is 82 2024-04-07T16:19:23.965 INFO:tasks.rados.rados.0.smithi100.stdout:918: finishing rollback tid 0 to smithi10066867-42 2024-04-07T16:19:23.965 INFO:tasks.rados.rados.0.smithi100.stdout:918: finishing rollback tid 1 to smithi10066867-42 2024-04-07T16:19:23.965 INFO:tasks.rados.rados.0.smithi100.stdout:918: finishing rollback tid 2 to smithi10066867-42 2024-04-07T16:19:23.965 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 42 v 467 (ObjNum 190 snap 49 seq_num 190) dirty exists 2024-04-07T16:19:23.965 INFO:tasks.rados.rados.0.smithi100.stdout:917: finishing write tid 1 to smithi10066867-21 2024-04-07T16:19:23.965 INFO:tasks.rados.rados.0.smithi100.stdout:917: finishing write tid 2 to smithi10066867-21 2024-04-07T16:19:23.965 INFO:tasks.rados.rados.0.smithi100.stdout:917: finishing write tid 3 to smithi10066867-21 2024-04-07T16:19:23.965 INFO:tasks.rados.rados.0.smithi100.stdout:917: finishing write tid 4 to smithi10066867-21 2024-04-07T16:19:23.965 INFO:tasks.rados.rados.0.smithi100.stdout:917: finishing write tid 5 to smithi10066867-21 2024-04-07T16:19:23.965 INFO:tasks.rados.rados.0.smithi100.stdout:918: done (3 left) 2024-04-07T16:19:23.965 INFO:tasks.rados.rados.0.smithi100.stdout:921: snap_create 2024-04-07T16:19:23.971 INFO:tasks.rados.rados.0.smithi100.stdout:920: finishing copy_from to smithi10066867-13 2024-04-07T16:19:23.971 INFO:tasks.rados.rados.0.smithi100.stdout:920: got expected ENOENT (src dne) 2024-04-07T16:19:23.972 INFO:tasks.rados.rados.0.smithi100.stdout:920: finishing copy_from racing read to smithi10066867-13 2024-04-07T16:19:23.972 INFO:tasks.rados.rados.0.smithi100.stdout:919: finishing write tid 1 to smithi10066867-47 2024-04-07T16:19:23.973 INFO:tasks.rados.rados.0.smithi100.stdout:917: finishing write tid 6 to smithi10066867-21 2024-04-07T16:19:23.973 INFO:tasks.rados.rados.0.smithi100.stdout:917: oid 21 updating version 0 to 161 2024-04-07T16:19:23.973 INFO:tasks.rados.rados.0.smithi100.stdout:917: oid 21 version 161 is already newer than 157 2024-04-07T16:19:23.973 INFO:tasks.rados.rados.0.smithi100.stdout:917: oid 21 version 161 is already newer than 160 2024-04-07T16:19:23.973 INFO:tasks.rados.rados.0.smithi100.stdout:917: oid 21 version 161 is already newer than 158 2024-04-07T16:19:23.973 INFO:tasks.rados.rados.0.smithi100.stdout:917: oid 21 version 161 is already newer than 159 2024-04-07T16:19:23.973 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 21 v 161 (ObjNum 285 snap 82 seq_num 285) dirty exists 2024-04-07T16:19:23.973 INFO:tasks.rados.rados.0.smithi100.stdout:917: left oid 21 (ObjNum 285 snap 82 seq_num 285) 2024-04-07T16:19:23.973 INFO:tasks.rados.rados.0.smithi100.stdout:919: finishing write tid 2 to smithi10066867-47 2024-04-07T16:19:23.978 INFO:tasks.rados.rados.0.smithi100.stdout:919: finishing write tid 3 to smithi10066867-47 2024-04-07T16:19:23.979 INFO:tasks.rados.rados.0.smithi100.stdout:919: finishing write tid 4 to smithi10066867-47 2024-04-07T16:19:23.982 INFO:tasks.rados.rados.0.smithi100.stdout:919: finishing write tid 5 to smithi10066867-47 2024-04-07T16:19:23.985 INFO:tasks.rados.rados.0.smithi100.stdout:919: finishing write tid 6 to smithi10066867-47 2024-04-07T16:19:23.985 INFO:tasks.rados.rados.0.smithi100.stdout:919: oid 47 updating version 0 to 164 2024-04-07T16:19:23.985 INFO:tasks.rados.rados.0.smithi100.stdout:919: oid 47 version 164 is already newer than 163 2024-04-07T16:19:23.985 INFO:tasks.rados.rados.0.smithi100.stdout:919: oid 47 updating version 164 to 165 2024-04-07T16:19:23.985 INFO:tasks.rados.rados.0.smithi100.stdout:919: oid 47 updating version 165 to 166 2024-04-07T16:19:23.985 INFO:tasks.rados.rados.0.smithi100.stdout:919: oid 47 updating version 166 to 167 2024-04-07T16:19:23.985 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 47 v 167 (ObjNum 286 snap 82 seq_num 286) dirty exists 2024-04-07T16:19:23.985 INFO:tasks.rados.rados.0.smithi100.stdout:919: left oid 47 (ObjNum 286 snap 82 seq_num 286) 2024-04-07T16:19:24.605 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:24 smithi144 bash[23522]: cluster 2024-04-07T16:19:23.312615+0000 mon.a (mon.0) 1145 : cluster 0 mgrmap e33: y(active, since 9s), standbys: x 2024-04-07T16:19:24.605 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:24 smithi144 bash[23522]: audit 2024-04-07T16:19:23.317629+0000 mon.c (mon.2) 84 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-07T16:19:24.605 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:24 smithi144 bash[23522]: cluster 2024-04-07T16:19:23.887035+0000 mon.a (mon.0) 1146 : cluster 0 osdmap e231: 8 total, 8 up, 8 in 2024-04-07T16:19:24.605 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:24 smithi144 bash[23522]: cluster 2024-04-07T16:19:24.010549+0000 mgr.y (mgr.25073) 12 : cluster 0 pgmap v18: 121 pgs: 121 active+clean; 186 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 8.3 MiB/s rd, 2.9 MiB/s wr, 43 op/s 2024-04-07T16:19:24.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:24 smithi144 bash[23522]: audit 2024-04-07T16:19:24.189802+0000 mon.c (mon.2) 85 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:19:24.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:24 smithi144 bash[23522]: audit 2024-04-07T16:19:24.208351+0000 mon.c (mon.2) 86 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "config-key dump", "format": "json"}]: dispatch 2024-04-07T16:19:24.686 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:24 smithi100 bash[31660]: cluster 2024-04-07T16:19:23.312615+0000 mon.a (mon.0) 1145 : cluster 0 mgrmap e33: y(active, since 9s), standbys: x 2024-04-07T16:19:24.686 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:24 smithi100 bash[31660]: audit 2024-04-07T16:19:23.317629+0000 mon.c (mon.2) 84 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-07T16:19:24.687 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:24 smithi100 bash[31660]: cluster 2024-04-07T16:19:23.887035+0000 mon.a (mon.0) 1146 : cluster 0 osdmap e231: 8 total, 8 up, 8 in 2024-04-07T16:19:24.687 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:24 smithi100 bash[31660]: cluster 2024-04-07T16:19:24.010549+0000 mgr.y (mgr.25073) 12 : cluster 0 pgmap v18: 121 pgs: 121 active+clean; 186 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 8.3 MiB/s rd, 2.9 MiB/s wr, 43 op/s 2024-04-07T16:19:24.687 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:24 smithi100 bash[31660]: audit 2024-04-07T16:19:24.189802+0000 mon.c (mon.2) 85 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:19:24.687 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:24 smithi100 bash[31660]: audit 2024-04-07T16:19:24.208351+0000 mon.c (mon.2) 86 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "config-key dump", "format": "json"}]: dispatch 2024-04-07T16:19:24.687 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:24 smithi100 bash[21346]: cluster 2024-04-07T16:19:23.312615+0000 mon.a (mon.0) 1145 : cluster 0 mgrmap e33: y(active, since 9s), standbys: x 2024-04-07T16:19:24.687 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:24 smithi100 bash[21346]: audit 2024-04-07T16:19:23.317629+0000 mon.c (mon.2) 84 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-07T16:19:24.687 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:24 smithi100 bash[21346]: cluster 2024-04-07T16:19:23.887035+0000 mon.a (mon.0) 1146 : cluster 0 osdmap e231: 8 total, 8 up, 8 in 2024-04-07T16:19:24.687 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:24 smithi100 bash[21346]: cluster 2024-04-07T16:19:24.010549+0000 mgr.y (mgr.25073) 12 : cluster 0 pgmap v18: 121 pgs: 121 active+clean; 186 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 8.3 MiB/s rd, 2.9 MiB/s wr, 43 op/s 2024-04-07T16:19:24.687 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:24 smithi100 bash[21346]: audit 2024-04-07T16:19:24.189802+0000 mon.c (mon.2) 85 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:19:24.687 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:24 smithi100 bash[21346]: audit 2024-04-07T16:19:24.208351+0000 mon.c (mon.2) 86 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "config-key dump", "format": "json"}]: dispatch 2024-04-07T16:19:24.889 INFO:tasks.rados.rados.0.smithi100.stdout:917: done (3 left) 2024-04-07T16:19:24.889 INFO:tasks.rados.rados.0.smithi100.stdout:919: done (2 left) 2024-04-07T16:19:24.889 INFO:tasks.rados.rados.0.smithi100.stdout:920: done (1 left) 2024-04-07T16:19:24.889 INFO:tasks.rados.rados.0.smithi100.stdout:921: done (0 left) 2024-04-07T16:19:24.889 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 15 current snap is 83 2024-04-07T16:19:24.889 INFO:tasks.rados.rados.0.smithi100.stdout:922: seq_num 287 ranges {5029888=8192} 2024-04-07T16:19:24.919 INFO:tasks.rados.rados.0.smithi100.stdout:922: writing smithi10066867-15 from 5029888 to 5038080 tid 1 2024-04-07T16:19:24.919 INFO:tasks.rados.rados.0.smithi100.stdout:923: delete oid 31 current snap is 83 2024-04-07T16:19:24.932 INFO:tasks.rados.rados.0.smithi100.stdout:922: finishing write tid 1 to smithi10066867-15 2024-04-07T16:19:24.932 INFO:tasks.rados.rados.0.smithi100.stdout:923: done (1 left) 2024-04-07T16:19:24.932 INFO:tasks.rados.rados.0.smithi100.stdout:924: snap_create 2024-04-07T16:19:24.932 INFO:tasks.rados.rados.0.smithi100.stdout:922: finishing write tid 2 to smithi10066867-15 2024-04-07T16:19:24.935 INFO:tasks.rados.rados.0.smithi100.stdout:922: finishing write tid 3 to smithi10066867-15 2024-04-07T16:19:24.935 INFO:tasks.rados.rados.0.smithi100.stdout:922: oid 15 updating version 0 to 560 2024-04-07T16:19:24.935 INFO:tasks.rados.rados.0.smithi100.stdout:922: oid 15 updating version 560 to 561 2024-04-07T16:19:24.936 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 15 v 561 (ObjNum 287 snap 83 seq_num 287) dirty exists 2024-04-07T16:19:24.936 INFO:tasks.rados.rados.0.smithi100.stdout:922: left oid 15 (ObjNum 287 snap 83 seq_num 287) 2024-04-07T16:19:25.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:25 smithi144 bash[23522]: audit 2024-04-07T16:19:24.626578+0000 mon.c (mon.2) 87 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:19:25.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:25 smithi144 bash[23522]: audit 2024-04-07T16:19:24.627290+0000 mon.c (mon.2) 88 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:19:25.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:25 smithi144 bash[23522]: audit 2024-04-07T16:19:24.627795+0000 mon.c (mon.2) 89 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:19:25.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:25 smithi144 bash[23522]: audit 2024-04-07T16:19:24.630381+0000 mon.c (mon.2) 90 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:19:25.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:25 smithi144 bash[23522]: audit 2024-04-07T16:19:24.630859+0000 mon.c (mon.2) 91 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:19:25.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:25 smithi144 bash[23522]: audit 2024-04-07T16:19:24.631278+0000 mon.c (mon.2) 92 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:19:25.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:25 smithi144 bash[23522]: audit 2024-04-07T16:19:24.633638+0000 mon.c (mon.2) 93 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:19:25.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:25 smithi144 bash[23522]: audit 2024-04-07T16:19:24.633797+0000 mon.c (mon.2) 94 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:19:25.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:25 smithi144 bash[23522]: audit 2024-04-07T16:19:24.634094+0000 mon.c (mon.2) 95 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:19:25.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:25 smithi144 bash[23522]: audit 2024-04-07T16:19:24.646413+0000 mon.b (mon.1) 329 : audit 1 from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T16:19:25.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:25 smithi144 bash[23522]: audit 2024-04-07T16:19:24.646519+0000 mon.b (mon.1) 330 : audit 1 from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T16:19:25.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:25 smithi144 bash[23522]: audit 2024-04-07T16:19:24.648030+0000 mon.a (mon.0) 1147 : audit 1 from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T16:19:25.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:25 smithi144 bash[23522]: audit 2024-04-07T16:19:24.648096+0000 mon.a (mon.0) 1148 : audit 1 from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T16:19:25.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:25 smithi144 bash[23522]: audit 2024-04-07T16:19:24.648355+0000 mon.c (mon.2) 96 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:19:25.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:25 smithi144 bash[23522]: audit 2024-04-07T16:19:24.648582+0000 mon.c (mon.2) 97 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:19:25.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:25 smithi144 bash[23522]: audit 2024-04-07T16:19:24.648818+0000 mon.c (mon.2) 98 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:19:25.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:25 smithi144 bash[23522]: audit 2024-04-07T16:19:24.651234+0000 mon.c (mon.2) 99 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:19:25.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:25 smithi144 bash[23522]: audit 2024-04-07T16:19:24.651462+0000 mon.c (mon.2) 100 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:19:25.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:25 smithi144 bash[23522]: audit 2024-04-07T16:19:24.651837+0000 mon.c (mon.2) 101 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:19:25.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:25 smithi144 bash[23522]: audit 2024-04-07T16:19:24.653621+0000 mon.c (mon.2) 102 : audit 1 from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T16:19:25.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:25 smithi144 bash[23522]: audit 2024-04-07T16:19:24.653751+0000 mon.c (mon.2) 103 : audit 1 from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T16:19:25.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:25 smithi144 bash[23522]: audit 2024-04-07T16:19:24.653899+0000 mon.c (mon.2) 104 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:19:25.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:25 smithi144 bash[23522]: audit 2024-04-07T16:19:24.654770+0000 mon.c (mon.2) 105 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:19:25.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:25 smithi144 bash[23522]: audit 2024-04-07T16:19:24.655299+0000 mon.c (mon.2) 106 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:19:25.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:25 smithi144 bash[23522]: cluster 2024-04-07T16:19:24.898634+0000 mon.a (mon.0) 1149 : cluster 0 osdmap e232: 8 total, 8 up, 8 in 2024-04-07T16:19:25.687 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:25 smithi100 bash[21346]: audit 2024-04-07T16:19:24.626578+0000 mon.c (mon.2) 87 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:19:25.687 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:25 smithi100 bash[21346]: audit 2024-04-07T16:19:24.627290+0000 mon.c (mon.2) 88 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:19:25.687 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:25 smithi100 bash[21346]: audit 2024-04-07T16:19:24.627795+0000 mon.c (mon.2) 89 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:19:25.688 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:25 smithi100 bash[21346]: audit 2024-04-07T16:19:24.630381+0000 mon.c (mon.2) 90 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:19:25.688 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:25 smithi100 bash[21346]: audit 2024-04-07T16:19:24.630859+0000 mon.c (mon.2) 91 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:19:25.688 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:25 smithi100 bash[21346]: audit 2024-04-07T16:19:24.631278+0000 mon.c (mon.2) 92 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:19:25.688 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:25 smithi100 bash[21346]: audit 2024-04-07T16:19:24.633638+0000 mon.c (mon.2) 93 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:19:25.688 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:25 smithi100 bash[21346]: audit 2024-04-07T16:19:24.633797+0000 mon.c (mon.2) 94 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:19:25.688 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:25 smithi100 bash[21346]: audit 2024-04-07T16:19:24.634094+0000 mon.c (mon.2) 95 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:19:25.688 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:25 smithi100 bash[21346]: audit 2024-04-07T16:19:24.646413+0000 mon.b (mon.1) 329 : audit 1 from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T16:19:25.688 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:25 smithi100 bash[21346]: audit 2024-04-07T16:19:24.646519+0000 mon.b (mon.1) 330 : audit 1 from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T16:19:25.688 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:25 smithi100 bash[21346]: audit 2024-04-07T16:19:24.648030+0000 mon.a (mon.0) 1147 : audit 1 from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T16:19:25.688 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:25 smithi100 bash[21346]: audit 2024-04-07T16:19:24.648096+0000 mon.a (mon.0) 1148 : audit 1 from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T16:19:25.688 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:25 smithi100 bash[21346]: audit 2024-04-07T16:19:24.648355+0000 mon.c (mon.2) 96 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:19:25.688 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:25 smithi100 bash[21346]: audit 2024-04-07T16:19:24.648582+0000 mon.c (mon.2) 97 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:19:25.688 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:25 smithi100 bash[21346]: audit 2024-04-07T16:19:24.648818+0000 mon.c (mon.2) 98 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:19:25.688 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:25 smithi100 bash[21346]: audit 2024-04-07T16:19:24.651234+0000 mon.c (mon.2) 99 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:19:25.688 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:25 smithi100 bash[21346]: audit 2024-04-07T16:19:24.651462+0000 mon.c (mon.2) 100 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:19:25.688 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:25 smithi100 bash[21346]: audit 2024-04-07T16:19:24.651837+0000 mon.c (mon.2) 101 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:19:25.689 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:25 smithi100 bash[31660]: audit 2024-04-07T16:19:24.626578+0000 mon.c (mon.2) 87 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:19:25.689 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:25 smithi100 bash[31660]: audit 2024-04-07T16:19:24.627290+0000 mon.c (mon.2) 88 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:19:25.689 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:25 smithi100 bash[31660]: audit 2024-04-07T16:19:24.627795+0000 mon.c (mon.2) 89 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:19:25.689 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:25 smithi100 bash[31660]: audit 2024-04-07T16:19:24.630381+0000 mon.c (mon.2) 90 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:19:25.689 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:25 smithi100 bash[31660]: audit 2024-04-07T16:19:24.630859+0000 mon.c (mon.2) 91 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:19:25.689 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:25 smithi100 bash[31660]: audit 2024-04-07T16:19:24.631278+0000 mon.c (mon.2) 92 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:19:25.689 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:25 smithi100 bash[31660]: audit 2024-04-07T16:19:24.633638+0000 mon.c (mon.2) 93 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:19:25.689 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:25 smithi100 bash[31660]: audit 2024-04-07T16:19:24.633797+0000 mon.c (mon.2) 94 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:19:25.689 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:25 smithi100 bash[31660]: audit 2024-04-07T16:19:24.634094+0000 mon.c (mon.2) 95 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:19:25.689 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:25 smithi100 bash[31660]: audit 2024-04-07T16:19:24.646413+0000 mon.b (mon.1) 329 : audit 1 from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T16:19:25.689 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:25 smithi100 bash[31660]: audit 2024-04-07T16:19:24.646519+0000 mon.b (mon.1) 330 : audit 1 from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T16:19:25.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:25 smithi100 bash[31660]: audit 2024-04-07T16:19:24.648030+0000 mon.a (mon.0) 1147 : audit 1 from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T16:19:25.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:25 smithi100 bash[31660]: audit 2024-04-07T16:19:24.648096+0000 mon.a (mon.0) 1148 : audit 1 from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T16:19:25.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:25 smithi100 bash[31660]: audit 2024-04-07T16:19:24.648355+0000 mon.c (mon.2) 96 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:19:25.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:25 smithi100 bash[31660]: audit 2024-04-07T16:19:24.648582+0000 mon.c (mon.2) 97 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:19:25.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:25 smithi100 bash[31660]: audit 2024-04-07T16:19:24.648818+0000 mon.c (mon.2) 98 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:19:25.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:25 smithi100 bash[31660]: audit 2024-04-07T16:19:24.651234+0000 mon.c (mon.2) 99 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:19:25.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:25 smithi100 bash[31660]: audit 2024-04-07T16:19:24.651462+0000 mon.c (mon.2) 100 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:19:25.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:25 smithi100 bash[31660]: audit 2024-04-07T16:19:24.651837+0000 mon.c (mon.2) 101 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:19:25.690 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:25 smithi100 bash[21346]: audit 2024-04-07T16:19:24.653621+0000 mon.c (mon.2) 102 : audit 1 from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T16:19:25.690 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:25 smithi100 bash[21346]: audit 2024-04-07T16:19:24.653751+0000 mon.c (mon.2) 103 : audit 1 from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T16:19:25.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:25 smithi100 bash[21346]: audit 2024-04-07T16:19:24.653899+0000 mon.c (mon.2) 104 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:19:25.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:25 smithi100 bash[21346]: audit 2024-04-07T16:19:24.654770+0000 mon.c (mon.2) 105 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:19:25.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:25 smithi100 bash[21346]: audit 2024-04-07T16:19:24.655299+0000 mon.c (mon.2) 106 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:19:25.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:25 smithi100 bash[21346]: cluster 2024-04-07T16:19:24.898634+0000 mon.a (mon.0) 1149 : cluster 0 osdmap e232: 8 total, 8 up, 8 in 2024-04-07T16:19:25.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:25 smithi100 bash[31660]: audit 2024-04-07T16:19:24.653621+0000 mon.c (mon.2) 102 : audit 1 from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T16:19:25.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:25 smithi100 bash[31660]: audit 2024-04-07T16:19:24.653751+0000 mon.c (mon.2) 103 : audit 1 from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T16:19:25.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:25 smithi100 bash[31660]: audit 2024-04-07T16:19:24.653899+0000 mon.c (mon.2) 104 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:19:25.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:25 smithi100 bash[31660]: audit 2024-04-07T16:19:24.654770+0000 mon.c (mon.2) 105 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:19:25.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:25 smithi100 bash[31660]: audit 2024-04-07T16:19:24.655299+0000 mon.c (mon.2) 106 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:19:25.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:25 smithi100 bash[31660]: cluster 2024-04-07T16:19:24.898634+0000 mon.a (mon.0) 1149 : cluster 0 osdmap e232: 8 total, 8 up, 8 in 2024-04-07T16:19:25.890 INFO:tasks.rados.rados.0.smithi100.stdout:922: done (1 left) 2024-04-07T16:19:25.890 INFO:tasks.rados.rados.0.smithi100.stdout:924: done (0 left) 2024-04-07T16:19:25.891 INFO:tasks.rados.rados.0.smithi100.stdout:925: copy_from oid 33 from oid 44 current snap is 84 2024-04-07T16:19:25.891 INFO:tasks.rados.rados.0.smithi100.stdout:926: delete oid 3 current snap is 84 2024-04-07T16:19:25.899 INFO:tasks.rados.rados.0.smithi100.stdout:926: done (1 left) 2024-04-07T16:19:25.899 INFO:tasks.rados.rados.0.smithi100.stdout:927: snap_remove snap 79 2024-04-07T16:19:26.355 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:19:26 smithi144 bash[51952]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:19:26] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-04-07T16:19:26.908 INFO:tasks.rados.rados.0.smithi100.stdout:927: done (1 left) 2024-04-07T16:19:26.908 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 21 current snap is 84 2024-04-07T16:19:26.909 INFO:tasks.rados.rados.0.smithi100.stdout:925: finishing copy_from racing read to smithi10066867-33 2024-04-07T16:19:26.909 INFO:tasks.rados.rados.0.smithi100.stdout:928: seq_num 290 ranges {11034624=57344} 2024-04-07T16:19:26.978 INFO:tasks.rados.rados.0.smithi100.stdout:928: writing smithi10066867-21 from 11034624 to 11091968 tid 1 2024-04-07T16:19:26.978 INFO:tasks.rados.rados.0.smithi100.stdout:929: read oid 3 snap -1 2024-04-07T16:19:26.978 INFO:tasks.rados.rados.0.smithi100.stdout:929: expect deleted 2024-04-07T16:19:26.978 INFO:tasks.rados.rados.0.smithi100.stdout:930: read oid 23 snap -1 2024-04-07T16:19:26.978 INFO:tasks.rados.rados.0.smithi100.stdout:930: expect (ObjNum 262 snap 70 seq_num 262) 2024-04-07T16:19:26.978 INFO:tasks.rados.rados.0.smithi100.stdout:931: delete oid 2 current snap is 84 2024-04-07T16:19:26.978 INFO:tasks.rados.rados.0.smithi100.stdout:925: finishing copy_from to smithi10066867-33 2024-04-07T16:19:26.978 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 33 v 381 (ObjNum 165 snap 44 seq_num 165) dirty exists 2024-04-07T16:19:26.981 INFO:tasks.rados.rados.0.smithi100.stdout:925: done (4 left) 2024-04-07T16:19:26.981 INFO:tasks.rados.rados.0.smithi100.stdout:929: done (3 left) 2024-04-07T16:19:26.981 INFO:tasks.rados.rados.0.smithi100.stdout:931: done (2 left) 2024-04-07T16:19:26.981 INFO:tasks.rados.rados.0.smithi100.stdout:932: read oid 15 snap -1 2024-04-07T16:19:26.981 INFO:tasks.rados.rados.0.smithi100.stdout:932: expect (ObjNum 287 snap 83 seq_num 287) 2024-04-07T16:19:26.982 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 5 current snap is 84 2024-04-07T16:19:26.982 INFO:tasks.rados.rados.0.smithi100.stdout:933: seq_num 292 ranges {3874816=98304} 2024-04-07T16:19:27.005 INFO:tasks.rados.rados.0.smithi100.stdout:933: writing smithi10066867-5 from 3874816 to 3973120 tid 1 2024-04-07T16:19:27.005 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 2 current snap is 84 2024-04-07T16:19:27.005 INFO:tasks.rados.rados.0.smithi100.stdout:928: finishing write tid 1 to smithi10066867-21 2024-04-07T16:19:27.005 INFO:tasks.rados.rados.0.smithi100.stdout:934: seq_num 293 ranges {0=114688} 2024-04-07T16:19:27.006 INFO:tasks.rados.rados.0.smithi100.stdout:934: writing smithi10066867-2 from 0 to 114688 tid 1 2024-04-07T16:19:27.006 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 41 current snap is 84 2024-04-07T16:19:27.006 INFO:tasks.rados.rados.0.smithi100.stdout:928: finishing write tid 2 to smithi10066867-21 2024-04-07T16:19:27.006 INFO:tasks.rados.rados.0.smithi100.stdout:928: finishing write tid 3 to smithi10066867-21 2024-04-07T16:19:27.006 INFO:tasks.rados.rados.0.smithi100.stdout:928: oid 21 updating version 0 to 173 2024-04-07T16:19:27.006 INFO:tasks.rados.rados.0.smithi100.stdout:928: oid 21 updating version 173 to 174 2024-04-07T16:19:27.006 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 21 v 174 (ObjNum 290 snap 84 seq_num 290) dirty exists 2024-04-07T16:19:27.006 INFO:tasks.rados.rados.0.smithi100.stdout:928: left oid 21 (ObjNum 290 snap 84 seq_num 290) 2024-04-07T16:19:27.006 INFO:tasks.rados.rados.0.smithi100.stdout:930: expect (ObjNum 262 snap 70 seq_num 262) 2024-04-07T16:19:27.142 INFO:tasks.rados.rados.0.smithi100.stdout:933: finishing write tid 1 to smithi10066867-5 2024-04-07T16:19:27.142 INFO:tasks.rados.rados.0.smithi100.stdout:933: finishing write tid 2 to smithi10066867-5 2024-04-07T16:19:27.142 INFO:tasks.rados.rados.0.smithi100.stdout:933: finishing write tid 3 to smithi10066867-5 2024-04-07T16:19:27.142 INFO:tasks.rados.rados.0.smithi100.stdout:933: oid 5 updating version 0 to 502 2024-04-07T16:19:27.142 INFO:tasks.rados.rados.0.smithi100.stdout:933: oid 5 version 502 is already newer than 501 2024-04-07T16:19:27.142 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 5 v 502 (ObjNum 292 snap 84 seq_num 292) dirty exists 2024-04-07T16:19:27.143 INFO:tasks.rados.rados.0.smithi100.stdout:933: left oid 5 (ObjNum 292 snap 84 seq_num 292) 2024-04-07T16:19:27.143 INFO:tasks.rados.rados.0.smithi100.stdout:932: expect (ObjNum 287 snap 83 seq_num 287) 2024-04-07T16:19:27.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:26 smithi100 bash[21346]: cluster 2024-04-07T16:19:25.905692+0000 mon.a (mon.0) 1150 : cluster 0 osdmap e233: 8 total, 8 up, 8 in 2024-04-07T16:19:27.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:26 smithi100 bash[21346]: cluster 2024-04-07T16:19:25.922194+0000 mon.a (mon.0) 1151 : cluster 0 mgrmap e34: y(active, since 11s), standbys: x 2024-04-07T16:19:27.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:26 smithi100 bash[21346]: cluster 2024-04-07T16:19:26.011290+0000 mgr.y (mgr.25073) 13 : cluster 0 pgmap v21: 121 pgs: 121 active+clean; 202 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 1.6 MiB/s rd, 4.4 MiB/s wr, 35 op/s 2024-04-07T16:19:27.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:26 smithi100 bash[31660]: cluster 2024-04-07T16:19:25.905692+0000 mon.a (mon.0) 1150 : cluster 0 osdmap e233: 8 total, 8 up, 8 in 2024-04-07T16:19:27.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:26 smithi100 bash[31660]: cluster 2024-04-07T16:19:25.922194+0000 mon.a (mon.0) 1151 : cluster 0 mgrmap e34: y(active, since 11s), standbys: x 2024-04-07T16:19:27.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:26 smithi100 bash[31660]: cluster 2024-04-07T16:19:26.011290+0000 mgr.y (mgr.25073) 13 : cluster 0 pgmap v21: 121 pgs: 121 active+clean; 202 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 1.6 MiB/s rd, 4.4 MiB/s wr, 35 op/s 2024-04-07T16:19:27.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:26 smithi144 bash[23522]: cluster 2024-04-07T16:19:25.905692+0000 mon.a (mon.0) 1150 : cluster 0 osdmap e233: 8 total, 8 up, 8 in 2024-04-07T16:19:27.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:26 smithi144 bash[23522]: cluster 2024-04-07T16:19:25.922194+0000 mon.a (mon.0) 1151 : cluster 0 mgrmap e34: y(active, since 11s), standbys: x 2024-04-07T16:19:27.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:26 smithi144 bash[23522]: cluster 2024-04-07T16:19:26.011290+0000 mgr.y (mgr.25073) 13 : cluster 0 pgmap v21: 121 pgs: 121 active+clean; 202 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 1.6 MiB/s rd, 4.4 MiB/s wr, 35 op/s 2024-04-07T16:19:27.452 INFO:tasks.rados.rados.0.smithi100.stdout:934: finishing write tid 1 to smithi10066867-2 2024-04-07T16:19:27.452 INFO:tasks.rados.rados.0.smithi100.stdout:934: finishing write tid 2 to smithi10066867-2 2024-04-07T16:19:27.453 INFO:tasks.rados.rados.0.smithi100.stdout:934: finishing write tid 3 to smithi10066867-2 2024-04-07T16:19:27.453 INFO:tasks.rados.rados.0.smithi100.stdout:934: oid 2 updating version 0 to 393 2024-04-07T16:19:27.453 INFO:tasks.rados.rados.0.smithi100.stdout:934: oid 2 updating version 393 to 394 2024-04-07T16:19:27.453 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 2 v 394 (ObjNum 293 snap 84 seq_num 293) dirty exists 2024-04-07T16:19:27.453 INFO:tasks.rados.rados.0.smithi100.stdout:934: left oid 2 (ObjNum 293 snap 84 seq_num 293) 2024-04-07T16:19:27.453 INFO:tasks.rados.rados.0.smithi100.stdout:935: seq_num 294 ranges {1884160=131072} 2024-04-07T16:19:27.467 INFO:tasks.rados.rados.0.smithi100.stdout:935: writing smithi10066867-41 from 1884160 to 2015232 tid 1 2024-04-07T16:19:27.467 INFO:tasks.rados.rados.0.smithi100.stdout:928: done (5 left) 2024-04-07T16:19:27.467 INFO:tasks.rados.rados.0.smithi100.stdout:930: done (4 left) 2024-04-07T16:19:27.467 INFO:tasks.rados.rados.0.smithi100.stdout:932: done (3 left) 2024-04-07T16:19:27.467 INFO:tasks.rados.rados.0.smithi100.stdout:933: done (2 left) 2024-04-07T16:19:27.467 INFO:tasks.rados.rados.0.smithi100.stdout:934: done (1 left) 2024-04-07T16:19:27.467 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 45 current snap is 84 2024-04-07T16:19:27.468 INFO:tasks.rados.rados.0.smithi100.stdout:936: seq_num 295 ranges {4071424=147456} 2024-04-07T16:19:27.496 INFO:tasks.rados.rados.0.smithi100.stdout:936: writing smithi10066867-45 from 4071424 to 4218880 tid 1 2024-04-07T16:19:27.496 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 10 current snap is 84 2024-04-07T16:19:27.496 INFO:tasks.rados.rados.0.smithi100.stdout:937: seq_num 296 ranges {4268032=163840} 2024-04-07T16:19:27.526 INFO:tasks.rados.rados.0.smithi100.stdout:937: writing smithi10066867-10 from 4268032 to 4431872 tid 1 2024-04-07T16:19:27.527 INFO:tasks.rados.rados.0.smithi100.stdout:938: rmattr oid 35 current snap is 84 2024-04-07T16:19:27.527 INFO:tasks.rados.rados.0.smithi100.stdout:935: finishing write tid 1 to smithi10066867-41 2024-04-07T16:19:27.527 INFO:tasks.rados.rados.0.smithi100.stdout:939: delete oid 7 current snap is 84 2024-04-07T16:19:27.527 INFO:tasks.rados.rados.0.smithi100.stdout:935: finishing write tid 2 to smithi10066867-41 2024-04-07T16:19:27.527 INFO:tasks.rados.rados.0.smithi100.stdout:935: finishing write tid 3 to smithi10066867-41 2024-04-07T16:19:27.527 INFO:tasks.rados.rados.0.smithi100.stdout:935: oid 41 updating version 0 to 504 2024-04-07T16:19:27.527 INFO:tasks.rados.rados.0.smithi100.stdout:935: oid 41 version 504 is already newer than 503 2024-04-07T16:19:27.527 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 41 v 504 (ObjNum 294 snap 84 seq_num 294) dirty exists 2024-04-07T16:19:27.527 INFO:tasks.rados.rados.0.smithi100.stdout:935: left oid 41 (ObjNum 294 snap 84 seq_num 294) 2024-04-07T16:19:27.527 INFO:tasks.rados.rados.0.smithi100.stdout:936: finishing write tid 1 to smithi10066867-45 2024-04-07T16:19:27.527 INFO:tasks.rados.rados.0.smithi100.stdout:936: finishing write tid 2 to smithi10066867-45 2024-04-07T16:19:27.527 INFO:tasks.rados.rados.0.smithi100.stdout:936: finishing write tid 3 to smithi10066867-45 2024-04-07T16:19:27.527 INFO:tasks.rados.rados.0.smithi100.stdout:936: oid 45 updating version 0 to 448 2024-04-07T16:19:27.528 INFO:tasks.rados.rados.0.smithi100.stdout:936: oid 45 version 448 is already newer than 447 2024-04-07T16:19:27.528 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 45 v 448 (ObjNum 295 snap 84 seq_num 295) dirty exists 2024-04-07T16:19:27.528 INFO:tasks.rados.rados.0.smithi100.stdout:936: left oid 45 (ObjNum 295 snap 84 seq_num 295) 2024-04-07T16:19:27.530 INFO:tasks.rados.rados.0.smithi100.stdout:935: done (4 left) 2024-04-07T16:19:27.530 INFO:tasks.rados.rados.0.smithi100.stdout:936: done (3 left) 2024-04-07T16:19:27.530 INFO:tasks.rados.rados.0.smithi100.stdout:939: done (2 left) 2024-04-07T16:19:27.530 INFO:tasks.rados.rados.0.smithi100.stdout:940: copy_from oid 22 from oid 41 current snap is 84 2024-04-07T16:19:27.530 INFO:tasks.rados.rados.0.smithi100.stdout:941: snap_create 2024-04-07T16:19:27.532 INFO:tasks.rados.rados.0.smithi100.stdout:937: finishing write tid 1 to smithi10066867-10 2024-04-07T16:19:27.533 INFO:tasks.rados.rados.0.smithi100.stdout:937: finishing write tid 2 to smithi10066867-10 2024-04-07T16:19:27.534 INFO:tasks.rados.rados.0.smithi100.stdout:937: finishing write tid 3 to smithi10066867-10 2024-04-07T16:19:27.534 INFO:tasks.rados.rados.0.smithi100.stdout:937: oid 10 updating version 0 to 226 2024-04-07T16:19:27.535 INFO:tasks.rados.rados.0.smithi100.stdout:937: oid 10 version 226 is already newer than 225 2024-04-07T16:19:27.535 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 10 v 226 (ObjNum 296 snap 84 seq_num 296) dirty exists 2024-04-07T16:19:27.535 INFO:tasks.rados.rados.0.smithi100.stdout:937: left oid 10 (ObjNum 296 snap 84 seq_num 296) 2024-04-07T16:19:27.537 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 35 v 256 (ObjNum 271 snap 73 seq_num 271) dirty exists 2024-04-07T16:19:27.545 INFO:tasks.rados.rados.0.smithi100.stdout:940: finishing copy_from racing read to smithi10066867-22 2024-04-07T16:19:27.546 INFO:tasks.rados.rados.0.smithi100.stdout:940: finishing copy_from to smithi10066867-22 2024-04-07T16:19:27.546 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 22 v 504 (ObjNum 294 snap 84 seq_num 294) dirty exists 2024-04-07T16:19:27.872 INFO:tasks.rados.rados.0.smithi100.stdout:937: done (3 left) 2024-04-07T16:19:27.872 INFO:tasks.rados.rados.0.smithi100.stdout:938: done (2 left) 2024-04-07T16:19:27.873 INFO:tasks.rados.rados.0.smithi100.stdout:940: done (1 left) 2024-04-07T16:19:27.873 INFO:tasks.rados.rados.0.smithi100.stdout:941: done (0 left) 2024-04-07T16:19:27.873 INFO:tasks.rados.rados.0.smithi100.stdout:942: rollback oid 44 current snap is 85 2024-04-07T16:19:27.873 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 44 to 68 2024-04-07T16:19:27.873 INFO:tasks.rados.rados.0.smithi100.stdout:943: copy_from oid 18 from oid 38 current snap is 85 2024-04-07T16:19:27.873 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 3 current snap is 85 2024-04-07T16:19:27.873 INFO:tasks.rados.rados.0.smithi100.stdout:944: seq_num 298 ranges {0=196608} 2024-04-07T16:19:27.874 INFO:tasks.rados.rados.0.smithi100.stdout:944: writing smithi10066867-3 from 0 to 196608 tid 1 2024-04-07T16:19:27.874 INFO:tasks.rados.rados.0.smithi100.stdout:945: snap_remove snap 84 2024-04-07T16:19:28.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:27 smithi100 bash[31660]: cluster 2024-04-07T16:19:26.918689+0000 mon.a (mon.0) 1152 : cluster 0 osdmap e234: 8 total, 8 up, 8 in 2024-04-07T16:19:28.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:27 smithi100 bash[31660]: cluster 2024-04-07T16:19:27.883965+0000 mon.a (mon.0) 1153 : cluster 0 osdmap e235: 8 total, 8 up, 8 in 2024-04-07T16:19:28.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:27 smithi100 bash[21346]: cluster 2024-04-07T16:19:26.918689+0000 mon.a (mon.0) 1152 : cluster 0 osdmap e234: 8 total, 8 up, 8 in 2024-04-07T16:19:28.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:27 smithi100 bash[21346]: cluster 2024-04-07T16:19:27.883965+0000 mon.a (mon.0) 1153 : cluster 0 osdmap e235: 8 total, 8 up, 8 in 2024-04-07T16:19:28.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:27 smithi144 bash[23522]: cluster 2024-04-07T16:19:26.918689+0000 mon.a (mon.0) 1152 : cluster 0 osdmap e234: 8 total, 8 up, 8 in 2024-04-07T16:19:28.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:27 smithi144 bash[23522]: cluster 2024-04-07T16:19:27.883965+0000 mon.a (mon.0) 1153 : cluster 0 osdmap e235: 8 total, 8 up, 8 in 2024-04-07T16:19:28.876 INFO:tasks.rados.rados.0.smithi100.stdout:945: done (3 left) 2024-04-07T16:19:28.876 INFO:tasks.rados.rados.0.smithi100.stdout:946: snap_create 2024-04-07T16:19:28.876 INFO:tasks.rados.rados.0.smithi100.stdout:943: finishing copy_from to smithi10066867-18 2024-04-07T16:19:28.876 INFO:tasks.rados.rados.0.smithi100.stdout:943: got expected ENOENT (src dne) 2024-04-07T16:19:28.876 INFO:tasks.rados.rados.0.smithi100.stdout:943: finishing copy_from racing read to smithi10066867-18 2024-04-07T16:19:28.876 INFO:tasks.rados.rados.0.smithi100.stdout:942: finishing rollback tid 0 to smithi10066867-44 2024-04-07T16:19:28.876 INFO:tasks.rados.rados.0.smithi100.stdout:942: finishing rollback tid 1 to smithi10066867-44 2024-04-07T16:19:28.876 INFO:tasks.rados.rados.0.smithi100.stdout:944: finishing write tid 1 to smithi10066867-3 2024-04-07T16:19:28.876 INFO:tasks.rados.rados.0.smithi100.stdout:942: finishing rollback tid 2 to smithi10066867-44 2024-04-07T16:19:28.876 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 44 v 398 (ObjNum 244 snap 66 seq_num 244) dirty exists 2024-04-07T16:19:28.876 INFO:tasks.rados.rados.0.smithi100.stdout:944: finishing write tid 2 to smithi10066867-3 2024-04-07T16:19:28.877 INFO:tasks.rados.rados.0.smithi100.stdout:944: finishing write tid 3 to smithi10066867-3 2024-04-07T16:19:28.877 INFO:tasks.rados.rados.0.smithi100.stdout:944: oid 3 updating version 0 to 187 2024-04-07T16:19:28.877 INFO:tasks.rados.rados.0.smithi100.stdout:944: oid 3 updating version 187 to 188 2024-04-07T16:19:28.877 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 3 v 188 (ObjNum 298 snap 85 seq_num 298) dirty exists 2024-04-07T16:19:28.877 INFO:tasks.rados.rados.0.smithi100.stdout:944: left oid 3 (ObjNum 298 snap 85 seq_num 298) 2024-04-07T16:19:29.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:28 smithi100 bash[21346]: cluster 2024-04-07T16:19:28.011981+0000 mgr.y (mgr.25073) 14 : cluster 0 pgmap v24: 121 pgs: 121 active+clean; 202 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 1.4 MiB/s rd, 4.1 MiB/s wr, 32 op/s 2024-04-07T16:19:29.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:28 smithi100 bash[21346]: cluster 2024-04-07T16:19:28.890287+0000 mon.a (mon.0) 1154 : cluster 0 osdmap e236: 8 total, 8 up, 8 in 2024-04-07T16:19:29.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:28 smithi100 bash[31660]: cluster 2024-04-07T16:19:28.011981+0000 mgr.y (mgr.25073) 14 : cluster 0 pgmap v24: 121 pgs: 121 active+clean; 202 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 1.4 MiB/s rd, 4.1 MiB/s wr, 32 op/s 2024-04-07T16:19:29.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:28 smithi100 bash[31660]: cluster 2024-04-07T16:19:28.890287+0000 mon.a (mon.0) 1154 : cluster 0 osdmap e236: 8 total, 8 up, 8 in 2024-04-07T16:19:29.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:28 smithi144 bash[23522]: cluster 2024-04-07T16:19:28.011981+0000 mgr.y (mgr.25073) 14 : cluster 0 pgmap v24: 121 pgs: 121 active+clean; 202 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 1.4 MiB/s rd, 4.1 MiB/s wr, 32 op/s 2024-04-07T16:19:29.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:28 smithi144 bash[23522]: cluster 2024-04-07T16:19:28.890287+0000 mon.a (mon.0) 1154 : cluster 0 osdmap e236: 8 total, 8 up, 8 in 2024-04-07T16:19:29.874 INFO:tasks.rados.rados.0.smithi100.stdout:942: done (3 left) 2024-04-07T16:19:29.874 INFO:tasks.rados.rados.0.smithi100.stdout:943: done (2 left) 2024-04-07T16:19:29.874 INFO:tasks.rados.rados.0.smithi100.stdout:944: done (1 left) 2024-04-07T16:19:29.874 INFO:tasks.rados.rados.0.smithi100.stdout:946: done (0 left) 2024-04-07T16:19:29.874 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 1 current snap is 86 2024-04-07T16:19:29.874 INFO:tasks.rados.rados.0.smithi100.stdout:947: seq_num 299 ranges {1794048=212992} 2024-04-07T16:19:29.889 INFO:tasks.rados.rados.0.smithi100.stdout:947: writing smithi10066867-1 from 1794048 to 2007040 tid 1 2024-04-07T16:19:29.889 INFO:tasks.rados.rados.0.smithi100.stdout:948: read oid 35 snap -1 2024-04-07T16:19:29.889 INFO:tasks.rados.rados.0.smithi100.stdout:948: expect (ObjNum 271 snap 73 seq_num 271) 2024-04-07T16:19:29.889 INFO:tasks.rados.rados.0.smithi100.stdout:949: snap_create 2024-04-07T16:19:29.898 INFO:tasks.rados.rados.0.smithi100.stdout:947: finishing write tid 1 to smithi10066867-1 2024-04-07T16:19:29.899 INFO:tasks.rados.rados.0.smithi100.stdout:947: finishing write tid 2 to smithi10066867-1 2024-04-07T16:19:29.900 INFO:tasks.rados.rados.0.smithi100.stdout:947: finishing write tid 3 to smithi10066867-1 2024-04-07T16:19:29.901 INFO:tasks.rados.rados.0.smithi100.stdout:947: oid 1 updating version 0 to 228 2024-04-07T16:19:29.901 INFO:tasks.rados.rados.0.smithi100.stdout:947: oid 1 version 228 is already newer than 227 2024-04-07T16:19:29.901 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 1 v 228 (ObjNum 299 snap 86 seq_num 299) dirty exists 2024-04-07T16:19:29.901 INFO:tasks.rados.rados.0.smithi100.stdout:947: left oid 1 (ObjNum 299 snap 86 seq_num 299) 2024-04-07T16:19:29.915 INFO:tasks.rados.rados.0.smithi100.stdout:948: expect (ObjNum 271 snap 73 seq_num 271) 2024-04-07T16:19:30.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:29 smithi100 bash[21346]: audit 2024-04-07T16:19:29.133413+0000 mon.c (mon.2) 107 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:19:30.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:29 smithi100 bash[21346]: cluster 2024-04-07T16:19:29.889814+0000 mon.a (mon.0) 1155 : cluster 0 osdmap e237: 8 total, 8 up, 8 in 2024-04-07T16:19:30.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:29 smithi100 bash[31660]: audit 2024-04-07T16:19:29.133413+0000 mon.c (mon.2) 107 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:19:30.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:29 smithi100 bash[31660]: cluster 2024-04-07T16:19:29.889814+0000 mon.a (mon.0) 1155 : cluster 0 osdmap e237: 8 total, 8 up, 8 in 2024-04-07T16:19:30.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:29 smithi144 bash[23522]: audit 2024-04-07T16:19:29.133413+0000 mon.c (mon.2) 107 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:19:30.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:29 smithi144 bash[23522]: cluster 2024-04-07T16:19:29.889814+0000 mon.a (mon.0) 1155 : cluster 0 osdmap e237: 8 total, 8 up, 8 in 2024-04-07T16:19:30.884 INFO:tasks.rados.rados.0.smithi100.stdout:947: done (2 left) 2024-04-07T16:19:30.884 INFO:tasks.rados.rados.0.smithi100.stdout:948: done (1 left) 2024-04-07T16:19:30.884 INFO:tasks.rados.rados.0.smithi100.stdout:949: done (0 left) 2024-04-07T16:19:30.884 INFO:tasks.rados.rados.0.smithi100.stdout:950: rollback oid 37 current snap is 87 2024-04-07T16:19:30.885 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 37 to 83 2024-04-07T16:19:30.885 INFO:tasks.rados.rados.0.smithi100.stdout:951: delete oid 44 current snap is 87 2024-04-07T16:19:30.890 INFO:tasks.rados.rados.0.smithi100.stdout:951: done (1 left) 2024-04-07T16:19:30.890 INFO:tasks.rados.rados.0.smithi100.stdout:952: copy_from oid 39 from oid 8 current snap is 87 2024-04-07T16:19:30.890 INFO:tasks.rados.rados.0.smithi100.stdout:953: read oid 46 snap 82 2024-04-07T16:19:30.890 INFO:tasks.rados.rados.0.smithi100.stdout:953: expect (ObjNum 272 snap 73 seq_num 272) 2024-04-07T16:19:30.890 INFO:tasks.rados.rados.0.smithi100.stdout:954: rollback oid 29 current snap is 87 2024-04-07T16:19:30.890 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 29 to 77 2024-04-07T16:19:30.890 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 50 current snap is 87 2024-04-07T16:19:30.890 INFO:tasks.rados.rados.0.smithi100.stdout:955: seq_num 301 ranges {3350528=245760} 2024-04-07T16:19:30.919 INFO:tasks.rados.rados.0.smithi100.stdout:955: writing smithi10066867-50 from 3350528 to 3596288 tid 1 2024-04-07T16:19:30.919 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 24 current snap is 87 2024-04-07T16:19:30.919 INFO:tasks.rados.rados.0.smithi100.stdout:950: finishing rollback tid 0 to smithi10066867-37 2024-04-07T16:19:30.919 INFO:tasks.rados.rados.0.smithi100.stdout:950: finishing rollback tid 1 to smithi10066867-37 2024-04-07T16:19:30.919 INFO:tasks.rados.rados.0.smithi100.stdout:950: finishing rollback tid 2 to smithi10066867-37 2024-04-07T16:19:30.919 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 37 v 470 (ObjNum 275 snap 78 seq_num 275) dirty exists 2024-04-07T16:19:30.919 INFO:tasks.rados.rados.0.smithi100.stdout:954: finishing rollback tid 1 to smithi10066867-29 2024-04-07T16:19:30.919 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 29 v 0 (ObjNum 161 snap 0 seq_num 4239544151) dirty dne 2024-04-07T16:19:30.919 INFO:tasks.rados.rados.0.smithi100.stdout:956: seq_num 302 ranges {8396800=262144} 2024-04-07T16:19:30.970 INFO:tasks.rados.rados.0.smithi100.stdout:956: writing smithi10066867-24 from 8396800 to 8658944 tid 1 2024-04-07T16:19:30.970 INFO:tasks.rados.rados.0.smithi100.stdout:950: done (5 left) 2024-04-07T16:19:30.970 INFO:tasks.rados.rados.0.smithi100.stdout:954: done (4 left) 2024-04-07T16:19:30.971 INFO:tasks.rados.rados.0.smithi100.stdout:957: snap_remove snap 70 2024-04-07T16:19:31.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:30 smithi100 bash[31660]: cluster 2024-04-07T16:19:30.012453+0000 mgr.y (mgr.25073) 15 : cluster 0 pgmap v27: 121 pgs: 121 active+clean; 202 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail 2024-04-07T16:19:31.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:30 smithi100 bash[31660]: cluster 2024-04-07T16:19:30.898854+0000 mon.a (mon.0) 1156 : cluster 0 osdmap e238: 8 total, 8 up, 8 in 2024-04-07T16:19:31.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:30 smithi100 bash[21346]: cluster 2024-04-07T16:19:30.012453+0000 mgr.y (mgr.25073) 15 : cluster 0 pgmap v27: 121 pgs: 121 active+clean; 202 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail 2024-04-07T16:19:31.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:30 smithi100 bash[21346]: cluster 2024-04-07T16:19:30.898854+0000 mon.a (mon.0) 1156 : cluster 0 osdmap e238: 8 total, 8 up, 8 in 2024-04-07T16:19:31.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:30 smithi144 bash[23522]: cluster 2024-04-07T16:19:30.012453+0000 mgr.y (mgr.25073) 15 : cluster 0 pgmap v27: 121 pgs: 121 active+clean; 202 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail 2024-04-07T16:19:31.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:30 smithi144 bash[23522]: cluster 2024-04-07T16:19:30.898854+0000 mon.a (mon.0) 1156 : cluster 0 osdmap e238: 8 total, 8 up, 8 in 2024-04-07T16:19:31.935 INFO:tasks.rados.rados.0.smithi100.stdout:957: done (4 left) 2024-04-07T16:19:31.935 INFO:tasks.rados.rados.0.smithi100.stdout:958: rmattr oid 20 current snap is 87 2024-04-07T16:19:31.935 INFO:tasks.rados.rados.0.smithi100.stdout:953: expect (ObjNum 272 snap 73 seq_num 272) 2024-04-07T16:19:32.063 INFO:tasks.rados.rados.0.smithi100.stdout:955: finishing write tid 1 to smithi10066867-50 2024-04-07T16:19:32.064 INFO:tasks.rados.rados.0.smithi100.stdout:955: finishing write tid 2 to smithi10066867-50 2024-04-07T16:19:32.064 INFO:tasks.rados.rados.0.smithi100.stdout:955: finishing write tid 3 to smithi10066867-50 2024-04-07T16:19:32.064 INFO:tasks.rados.rados.0.smithi100.stdout:955: oid 50 updating version 0 to 450 2024-04-07T16:19:32.064 INFO:tasks.rados.rados.0.smithi100.stdout:955: oid 50 version 450 is already newer than 449 2024-04-07T16:19:32.064 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 50 v 450 (ObjNum 301 snap 87 seq_num 301) dirty exists 2024-04-07T16:19:32.064 INFO:tasks.rados.rados.0.smithi100.stdout:955: left oid 50 (ObjNum 301 snap 87 seq_num 301) 2024-04-07T16:19:32.064 INFO:tasks.rados.rados.0.smithi100.stdout:952: finishing copy_from racing read to smithi10066867-39 2024-04-07T16:19:32.064 INFO:tasks.rados.rados.0.smithi100.stdout:952: finishing copy_from to smithi10066867-39 2024-04-07T16:19:32.064 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 39 v 253 (ObjNum 233 snap 60 seq_num 233) dirty exists 2024-04-07T16:19:32.064 INFO:tasks.rados.rados.0.smithi100.stdout:956: finishing write tid 1 to smithi10066867-24 2024-04-07T16:19:32.064 INFO:tasks.rados.rados.0.smithi100.stdout:956: finishing write tid 2 to smithi10066867-24 2024-04-07T16:19:32.064 INFO:tasks.rados.rados.0.smithi100.stdout:956: finishing write tid 3 to smithi10066867-24 2024-04-07T16:19:32.064 INFO:tasks.rados.rados.0.smithi100.stdout:956: oid 24 updating version 0 to 315 2024-04-07T16:19:32.065 INFO:tasks.rados.rados.0.smithi100.stdout:956: oid 24 updating version 315 to 316 2024-04-07T16:19:32.065 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 24 v 316 (ObjNum 302 snap 87 seq_num 302) dirty exists 2024-04-07T16:19:32.065 INFO:tasks.rados.rados.0.smithi100.stdout:956: left oid 24 (ObjNum 302 snap 87 seq_num 302) 2024-04-07T16:19:32.065 INFO:tasks.rados.rados.0.smithi100.stdout:952: done (4 left) 2024-04-07T16:19:32.065 INFO:tasks.rados.rados.0.smithi100.stdout:953: done (3 left) 2024-04-07T16:19:32.065 INFO:tasks.rados.rados.0.smithi100.stdout:955: done (2 left) 2024-04-07T16:19:32.065 INFO:tasks.rados.rados.0.smithi100.stdout:956: done (1 left) 2024-04-07T16:19:32.065 INFO:tasks.rados.rados.0.smithi100.stdout:958: done (0 left) 2024-04-07T16:19:32.065 INFO:tasks.rados.rados.0.smithi100.stdout:959: setattr oid 27 current snap is 87 2024-04-07T16:19:32.065 INFO:tasks.rados.rados.0.smithi100.stdout:960: snap_remove snap 81 2024-04-07T16:19:32.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:32 smithi144 bash[23522]: audit 2024-04-07T16:19:31.056094+0000 mon.a (mon.0) 1157 : audit 1 from='mgr.25073 ' entity='mgr.y' 2024-04-07T16:19:32.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:32 smithi144 bash[23522]: audit 2024-04-07T16:19:31.074769+0000 mon.a (mon.0) 1158 : audit 1 from='mgr.25073 ' entity='mgr.y' 2024-04-07T16:19:32.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:32 smithi144 bash[23522]: audit 2024-04-07T16:19:31.079382+0000 mon.c (mon.2) 108 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:19:32.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:32 smithi144 bash[23522]: audit 2024-04-07T16:19:31.080839+0000 mon.c (mon.2) 109 : audit 1 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:19:32.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:32 smithi144 bash[23522]: cephadm 2024-04-07T16:19:31.082717+0000 mgr.y (mgr.25073) 16 : cephadm 1 Updating smithi100:/etc/ceph/ceph.conf 2024-04-07T16:19:32.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:32 smithi144 bash[23522]: cephadm 2024-04-07T16:19:31.083171+0000 mgr.y (mgr.25073) 17 : cephadm 1 Updating smithi144:/etc/ceph/ceph.conf 2024-04-07T16:19:32.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:32 smithi144 bash[23522]: audit 2024-04-07T16:19:31.852580+0000 mon.a (mon.0) 1159 : audit 1 from='mgr.25073 ' entity='mgr.y' 2024-04-07T16:19:32.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:32 smithi144 bash[23522]: audit 2024-04-07T16:19:31.859058+0000 mon.a (mon.0) 1160 : audit 1 from='mgr.25073 ' entity='mgr.y' 2024-04-07T16:19:32.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:32 smithi144 bash[23522]: audit 2024-04-07T16:19:31.871212+0000 mon.a (mon.0) 1161 : audit 1 from='mgr.25073 ' entity='mgr.y' 2024-04-07T16:19:32.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:32 smithi144 bash[23522]: audit 2024-04-07T16:19:31.878250+0000 mon.a (mon.0) 1162 : audit 1 from='mgr.25073 ' entity='mgr.y' 2024-04-07T16:19:32.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:32 smithi144 bash[23522]: audit 2024-04-07T16:19:31.889895+0000 mon.a (mon.0) 1163 : audit 1 from='mgr.25073 ' entity='mgr.y' 2024-04-07T16:19:32.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:32 smithi144 bash[23522]: cluster 2024-04-07T16:19:31.948899+0000 mon.a (mon.0) 1164 : cluster 0 osdmap e239: 8 total, 8 up, 8 in 2024-04-07T16:19:32.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:32 smithi100 bash[21346]: audit 2024-04-07T16:19:31.056094+0000 mon.a (mon.0) 1157 : audit 1 from='mgr.25073 ' entity='mgr.y' 2024-04-07T16:19:32.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:32 smithi100 bash[21346]: audit 2024-04-07T16:19:31.074769+0000 mon.a (mon.0) 1158 : audit 1 from='mgr.25073 ' entity='mgr.y' 2024-04-07T16:19:32.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:32 smithi100 bash[21346]: audit 2024-04-07T16:19:31.079382+0000 mon.c (mon.2) 108 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:19:32.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:32 smithi100 bash[21346]: audit 2024-04-07T16:19:31.080839+0000 mon.c (mon.2) 109 : audit 1 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:19:32.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:32 smithi100 bash[21346]: cephadm 2024-04-07T16:19:31.082717+0000 mgr.y (mgr.25073) 16 : cephadm 1 Updating smithi100:/etc/ceph/ceph.conf 2024-04-07T16:19:32.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:32 smithi100 bash[21346]: cephadm 2024-04-07T16:19:31.083171+0000 mgr.y (mgr.25073) 17 : cephadm 1 Updating smithi144:/etc/ceph/ceph.conf 2024-04-07T16:19:32.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:32 smithi100 bash[21346]: audit 2024-04-07T16:19:31.852580+0000 mon.a (mon.0) 1159 : audit 1 from='mgr.25073 ' entity='mgr.y' 2024-04-07T16:19:32.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:32 smithi100 bash[21346]: audit 2024-04-07T16:19:31.859058+0000 mon.a (mon.0) 1160 : audit 1 from='mgr.25073 ' entity='mgr.y' 2024-04-07T16:19:32.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:32 smithi100 bash[21346]: audit 2024-04-07T16:19:31.871212+0000 mon.a (mon.0) 1161 : audit 1 from='mgr.25073 ' entity='mgr.y' 2024-04-07T16:19:32.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:32 smithi100 bash[21346]: audit 2024-04-07T16:19:31.878250+0000 mon.a (mon.0) 1162 : audit 1 from='mgr.25073 ' entity='mgr.y' 2024-04-07T16:19:32.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:32 smithi100 bash[21346]: audit 2024-04-07T16:19:31.889895+0000 mon.a (mon.0) 1163 : audit 1 from='mgr.25073 ' entity='mgr.y' 2024-04-07T16:19:32.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:32 smithi100 bash[21346]: cluster 2024-04-07T16:19:31.948899+0000 mon.a (mon.0) 1164 : cluster 0 osdmap e239: 8 total, 8 up, 8 in 2024-04-07T16:19:32.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:32 smithi100 bash[31660]: audit 2024-04-07T16:19:31.056094+0000 mon.a (mon.0) 1157 : audit 1 from='mgr.25073 ' entity='mgr.y' 2024-04-07T16:19:32.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:32 smithi100 bash[31660]: audit 2024-04-07T16:19:31.074769+0000 mon.a (mon.0) 1158 : audit 1 from='mgr.25073 ' entity='mgr.y' 2024-04-07T16:19:32.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:32 smithi100 bash[31660]: audit 2024-04-07T16:19:31.079382+0000 mon.c (mon.2) 108 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:19:32.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:32 smithi100 bash[31660]: audit 2024-04-07T16:19:31.080839+0000 mon.c (mon.2) 109 : audit 1 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:19:32.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:32 smithi100 bash[31660]: cephadm 2024-04-07T16:19:31.082717+0000 mgr.y (mgr.25073) 16 : cephadm 1 Updating smithi100:/etc/ceph/ceph.conf 2024-04-07T16:19:32.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:32 smithi100 bash[31660]: cephadm 2024-04-07T16:19:31.083171+0000 mgr.y (mgr.25073) 17 : cephadm 1 Updating smithi144:/etc/ceph/ceph.conf 2024-04-07T16:19:32.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:32 smithi100 bash[31660]: audit 2024-04-07T16:19:31.852580+0000 mon.a (mon.0) 1159 : audit 1 from='mgr.25073 ' entity='mgr.y' 2024-04-07T16:19:32.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:32 smithi100 bash[31660]: audit 2024-04-07T16:19:31.859058+0000 mon.a (mon.0) 1160 : audit 1 from='mgr.25073 ' entity='mgr.y' 2024-04-07T16:19:32.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:32 smithi100 bash[31660]: audit 2024-04-07T16:19:31.871212+0000 mon.a (mon.0) 1161 : audit 1 from='mgr.25073 ' entity='mgr.y' 2024-04-07T16:19:32.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:32 smithi100 bash[31660]: audit 2024-04-07T16:19:31.878250+0000 mon.a (mon.0) 1162 : audit 1 from='mgr.25073 ' entity='mgr.y' 2024-04-07T16:19:32.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:32 smithi100 bash[31660]: audit 2024-04-07T16:19:31.889895+0000 mon.a (mon.0) 1163 : audit 1 from='mgr.25073 ' entity='mgr.y' 2024-04-07T16:19:32.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:32 smithi100 bash[31660]: cluster 2024-04-07T16:19:31.948899+0000 mon.a (mon.0) 1164 : cluster 0 osdmap e239: 8 total, 8 up, 8 in 2024-04-07T16:19:33.050 INFO:tasks.rados.rados.0.smithi100.stdout:960: done (1 left) 2024-04-07T16:19:33.051 INFO:tasks.rados.rados.0.smithi100.stdout:961: rollback oid 16 current snap is 87 2024-04-07T16:19:33.051 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 16 to 77 2024-04-07T16:19:33.051 INFO:tasks.rados.rados.0.smithi100.stdout:962: snap_remove snap 83 2024-04-07T16:19:33.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:33 smithi144 bash[23522]: cephadm 2024-04-07T16:19:31.348052+0000 mgr.y (mgr.25073) 18 : cephadm 1 Updating smithi100:/var/lib/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/config/ceph.conf 2024-04-07T16:19:33.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:33 smithi144 bash[23522]: cephadm 2024-04-07T16:19:31.348323+0000 mgr.y (mgr.25073) 19 : cephadm 1 Updating smithi144:/var/lib/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/config/ceph.conf 2024-04-07T16:19:33.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:33 smithi144 bash[23522]: cephadm 2024-04-07T16:19:31.513395+0000 mgr.y (mgr.25073) 20 : cephadm 1 Updating smithi100:/etc/ceph/ceph.client.admin.keyring 2024-04-07T16:19:33.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:33 smithi144 bash[23522]: cephadm 2024-04-07T16:19:31.513783+0000 mgr.y (mgr.25073) 21 : cephadm 1 Updating smithi144:/etc/ceph/ceph.client.admin.keyring 2024-04-07T16:19:33.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:33 smithi144 bash[23522]: cephadm 2024-04-07T16:19:31.678436+0000 mgr.y (mgr.25073) 22 : cephadm 1 Updating smithi100:/var/lib/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/config/ceph.client.admin.keyring 2024-04-07T16:19:33.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:33 smithi144 bash[23522]: cephadm 2024-04-07T16:19:31.679776+0000 mgr.y (mgr.25073) 23 : cephadm 1 Updating smithi144:/var/lib/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/config/ceph.client.admin.keyring 2024-04-07T16:19:33.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:33 smithi144 bash[23522]: cephadm 2024-04-07T16:19:31.962505+0000 mgr.y (mgr.25073) 24 : cephadm 1 Reconfiguring prometheus.a (dependencies changed)... 2024-04-07T16:19:33.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:33 smithi144 bash[23522]: cluster 2024-04-07T16:19:32.013029+0000 mgr.y (mgr.25073) 25 : cluster 0 pgmap v30: 121 pgs: 121 active+clean; 208 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 9.9 MiB/s rd, 1000 KiB/s wr, 20 op/s 2024-04-07T16:19:33.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:33 smithi144 bash[23522]: cephadm 2024-04-07T16:19:32.211775+0000 mgr.y (mgr.25073) 26 : cephadm 1 Reconfiguring daemon prometheus.a on smithi144 2024-04-07T16:19:33.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:33 smithi100 bash[21346]: cephadm 2024-04-07T16:19:31.348052+0000 mgr.y (mgr.25073) 18 : cephadm 1 Updating smithi100:/var/lib/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/config/ceph.conf 2024-04-07T16:19:33.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:33 smithi100 bash[21346]: cephadm 2024-04-07T16:19:31.348323+0000 mgr.y (mgr.25073) 19 : cephadm 1 Updating smithi144:/var/lib/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/config/ceph.conf 2024-04-07T16:19:33.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:33 smithi100 bash[21346]: cephadm 2024-04-07T16:19:31.513395+0000 mgr.y (mgr.25073) 20 : cephadm 1 Updating smithi100:/etc/ceph/ceph.client.admin.keyring 2024-04-07T16:19:33.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:33 smithi100 bash[21346]: cephadm 2024-04-07T16:19:31.513783+0000 mgr.y (mgr.25073) 21 : cephadm 1 Updating smithi144:/etc/ceph/ceph.client.admin.keyring 2024-04-07T16:19:33.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:33 smithi100 bash[21346]: cephadm 2024-04-07T16:19:31.678436+0000 mgr.y (mgr.25073) 22 : cephadm 1 Updating smithi100:/var/lib/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/config/ceph.client.admin.keyring 2024-04-07T16:19:33.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:33 smithi100 bash[21346]: cephadm 2024-04-07T16:19:31.679776+0000 mgr.y (mgr.25073) 23 : cephadm 1 Updating smithi144:/var/lib/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/config/ceph.client.admin.keyring 2024-04-07T16:19:33.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:33 smithi100 bash[21346]: cephadm 2024-04-07T16:19:31.962505+0000 mgr.y (mgr.25073) 24 : cephadm 1 Reconfiguring prometheus.a (dependencies changed)... 2024-04-07T16:19:33.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:33 smithi100 bash[21346]: cluster 2024-04-07T16:19:32.013029+0000 mgr.y (mgr.25073) 25 : cluster 0 pgmap v30: 121 pgs: 121 active+clean; 208 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 9.9 MiB/s rd, 1000 KiB/s wr, 20 op/s 2024-04-07T16:19:33.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:33 smithi100 bash[21346]: cephadm 2024-04-07T16:19:32.211775+0000 mgr.y (mgr.25073) 26 : cephadm 1 Reconfiguring daemon prometheus.a on smithi144 2024-04-07T16:19:33.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:33 smithi100 bash[31660]: cephadm 2024-04-07T16:19:31.348052+0000 mgr.y (mgr.25073) 18 : cephadm 1 Updating smithi100:/var/lib/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/config/ceph.conf 2024-04-07T16:19:33.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:33 smithi100 bash[31660]: cephadm 2024-04-07T16:19:31.348323+0000 mgr.y (mgr.25073) 19 : cephadm 1 Updating smithi144:/var/lib/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/config/ceph.conf 2024-04-07T16:19:33.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:33 smithi100 bash[31660]: cephadm 2024-04-07T16:19:31.513395+0000 mgr.y (mgr.25073) 20 : cephadm 1 Updating smithi100:/etc/ceph/ceph.client.admin.keyring 2024-04-07T16:19:33.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:33 smithi100 bash[31660]: cephadm 2024-04-07T16:19:31.513783+0000 mgr.y (mgr.25073) 21 : cephadm 1 Updating smithi144:/etc/ceph/ceph.client.admin.keyring 2024-04-07T16:19:33.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:33 smithi100 bash[31660]: cephadm 2024-04-07T16:19:31.678436+0000 mgr.y (mgr.25073) 22 : cephadm 1 Updating smithi100:/var/lib/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/config/ceph.client.admin.keyring 2024-04-07T16:19:33.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:33 smithi100 bash[31660]: cephadm 2024-04-07T16:19:31.679776+0000 mgr.y (mgr.25073) 23 : cephadm 1 Updating smithi144:/var/lib/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/config/ceph.client.admin.keyring 2024-04-07T16:19:33.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:33 smithi100 bash[31660]: cephadm 2024-04-07T16:19:31.962505+0000 mgr.y (mgr.25073) 24 : cephadm 1 Reconfiguring prometheus.a (dependencies changed)... 2024-04-07T16:19:33.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:33 smithi100 bash[31660]: cluster 2024-04-07T16:19:32.013029+0000 mgr.y (mgr.25073) 25 : cluster 0 pgmap v30: 121 pgs: 121 active+clean; 208 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 9.9 MiB/s rd, 1000 KiB/s wr, 20 op/s 2024-04-07T16:19:33.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:33 smithi100 bash[31660]: cephadm 2024-04-07T16:19:32.211775+0000 mgr.y (mgr.25073) 26 : cephadm 1 Reconfiguring daemon prometheus.a on smithi144 2024-04-07T16:19:33.795 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:19:33 smithi144 bash[58467]: ts=2024-04-07T16:19:33.513Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=nodes name=CephNodeDiskspaceWarning index=4 msg="Evaluating rule failed" rule="alert: CephNodeDiskspaceWarning\nexpr: predict_linear(node_filesystem_free_bytes{device=~\"/.*\"}[2d], 3600 * 24 * 5)\n * on (instance) group_left (nodename) node_uname_info < 0\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.8.4\n severity: warning\n type: ceph_default\nannotations:\n description: Mountpoint {{ $labels.mountpoint }} on {{ $labels.nodename }} will\n be full in less than 5 days based on the 48 hour trailing fill rate.\n summary: Host filesystem free space is getting low\n" err="found duplicate series for the match group {instance=\"smithi100\"} on the right hand-side of the operation: [{__name__=\"node_uname_info\", domainname=\"(none)\", instance=\"smithi100\", job=\"node\", machine=\"x86_64\", nodename=\"smithi100\", release=\"5.15.0-101-generic\", sysname=\"Linux\", version=\"#111-Ubuntu SMP Tue Mar 5 20:16:58 UTC 2024\"}, {__name__=\"node_uname_info\", cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", domainname=\"(none)\", instance=\"smithi100\", job=\"node\", machine=\"x86_64\", nodename=\"smithi100\", release=\"5.15.0-101-generic\", sysname=\"Linux\", version=\"#111-Ubuntu SMP Tue Mar 5 20:16:58 UTC 2024\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:19:34.071 INFO:tasks.rados.rados.0.smithi100.stdout:962: done (2 left) 2024-04-07T16:19:34.072 INFO:tasks.rados.rados.0.smithi100.stdout:963: read oid 28 snap -1 2024-04-07T16:19:34.072 INFO:tasks.rados.rados.0.smithi100.stdout:963: expect (ObjNum 283 snap 81 seq_num 283) 2024-04-07T16:19:34.072 INFO:tasks.rados.rados.0.smithi100.stdout:964: copy_from oid 13 from oid 7 current snap is 87 2024-04-07T16:19:34.072 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 40 current snap is 87 2024-04-07T16:19:34.072 INFO:tasks.rados.rados.0.smithi100.stdout:965: seq_num 303 ranges {5857280=278528} 2024-04-07T16:19:34.108 INFO:tasks.rados.rados.0.smithi100.stdout:965: writing smithi10066867-40 from 5857280 to 6135808 tid 1 2024-04-07T16:19:34.108 INFO:tasks.rados.rados.0.smithi100.stdout:966: read oid 48 snap 69 2024-04-07T16:19:34.108 INFO:tasks.rados.rados.0.smithi100.stdout:966: expect (ObjNum 239 snap 65 seq_num 239) 2024-04-07T16:19:34.108 INFO:tasks.rados.rados.0.smithi100.stdout:967: read oid 17 snap -1 2024-04-07T16:19:34.108 INFO:tasks.rados.rados.0.smithi100.stdout:967: expect (ObjNum 259 snap 69 seq_num 259) 2024-04-07T16:19:34.108 INFO:tasks.rados.rados.0.smithi100.stdout:968: read oid 34 snap 69 2024-04-07T16:19:34.108 INFO:tasks.rados.rados.0.smithi100.stdout:968: expect (ObjNum 140 snap 39 seq_num 140) 2024-04-07T16:19:34.109 INFO:tasks.rados.rados.0.smithi100.stdout:969: snap_create 2024-04-07T16:19:34.109 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 27 v 249 (ObjNum 254 snap 69 seq_num 254) dirty exists 2024-04-07T16:19:34.109 INFO:tasks.rados.rados.0.smithi100.stdout:961: finishing rollback tid 0 to smithi10066867-16 2024-04-07T16:19:34.109 INFO:tasks.rados.rados.0.smithi100.stdout:961: finishing rollback tid 1 to smithi10066867-16 2024-04-07T16:19:34.109 INFO:tasks.rados.rados.0.smithi100.stdout:961: finishing rollback tid 2 to smithi10066867-16 2024-04-07T16:19:34.109 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 16 v 507 (ObjNum 118 snap 29 seq_num 118) dirty exists 2024-04-07T16:19:34.109 INFO:tasks.rados.rados.0.smithi100.stdout:964: finishing copy_from to smithi10066867-13 2024-04-07T16:19:34.109 INFO:tasks.rados.rados.0.smithi100.stdout:964: got expected ENOENT (src dne) 2024-04-07T16:19:34.109 INFO:tasks.rados.rados.0.smithi100.stdout:964: finishing copy_from racing read to smithi10066867-13 2024-04-07T16:19:34.111 INFO:tasks.rados.rados.0.smithi100.stdout:963: expect (ObjNum 283 snap 81 seq_num 283) 2024-04-07T16:19:34.275 INFO:tasks.rados.rados.0.smithi100.stdout:966: expect (ObjNum 239 snap 65 seq_num 239) 2024-04-07T16:19:34.327 INFO:tasks.rados.rados.0.smithi100.stdout:965: finishing write tid 1 to smithi10066867-40 2024-04-07T16:19:34.327 INFO:tasks.rados.rados.0.smithi100.stdout:965: finishing write tid 2 to smithi10066867-40 2024-04-07T16:19:34.327 INFO:tasks.rados.rados.0.smithi100.stdout:965: finishing write tid 3 to smithi10066867-40 2024-04-07T16:19:34.327 INFO:tasks.rados.rados.0.smithi100.stdout:965: oid 40 updating version 0 to 415 2024-04-07T16:19:34.327 INFO:tasks.rados.rados.0.smithi100.stdout:965: oid 40 version 415 is already newer than 414 2024-04-07T16:19:34.327 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 40 v 415 (ObjNum 303 snap 87 seq_num 303) dirty exists 2024-04-07T16:19:34.327 INFO:tasks.rados.rados.0.smithi100.stdout:965: left oid 40 (ObjNum 303 snap 87 seq_num 303) 2024-04-07T16:19:34.327 INFO:tasks.rados.rados.0.smithi100.stdout:968: expect (ObjNum 140 snap 39 seq_num 140) 2024-04-07T16:19:34.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:34 smithi144 bash[23522]: cluster 2024-04-07T16:19:33.064080+0000 mon.a (mon.0) 1165 : cluster 0 osdmap e240: 8 total, 8 up, 8 in 2024-04-07T16:19:34.386 INFO:tasks.rados.rados.0.smithi100.stdout:967: expect (ObjNum 259 snap 69 seq_num 259) 2024-04-07T16:19:34.436 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:34 smithi100 bash[21346]: cluster 2024-04-07T16:19:33.064080+0000 mon.a (mon.0) 1165 : cluster 0 osdmap e240: 8 total, 8 up, 8 in 2024-04-07T16:19:34.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:34 smithi100 bash[31660]: cluster 2024-04-07T16:19:33.064080+0000 mon.a (mon.0) 1165 : cluster 0 osdmap e240: 8 total, 8 up, 8 in 2024-04-07T16:19:35.079 INFO:tasks.rados.rados.0.smithi100.stdout:959: done (8 left) 2024-04-07T16:19:35.079 INFO:tasks.rados.rados.0.smithi100.stdout:961: done (7 left) 2024-04-07T16:19:35.079 INFO:tasks.rados.rados.0.smithi100.stdout:963: done (6 left) 2024-04-07T16:19:35.079 INFO:tasks.rados.rados.0.smithi100.stdout:964: done (5 left) 2024-04-07T16:19:35.079 INFO:tasks.rados.rados.0.smithi100.stdout:965: done (4 left) 2024-04-07T16:19:35.079 INFO:tasks.rados.rados.0.smithi100.stdout:966: done (3 left) 2024-04-07T16:19:35.079 INFO:tasks.rados.rados.0.smithi100.stdout:967: done (2 left) 2024-04-07T16:19:35.079 INFO:tasks.rados.rados.0.smithi100.stdout:968: done (1 left) 2024-04-07T16:19:35.079 INFO:tasks.rados.rados.0.smithi100.stdout:969: done (0 left) 2024-04-07T16:19:35.079 INFO:tasks.rados.rados.0.smithi100.stdout:970: read oid 3 snap -1 2024-04-07T16:19:35.079 INFO:tasks.rados.rados.0.smithi100.stdout:970: expect (ObjNum 298 snap 85 seq_num 298) 2024-04-07T16:19:35.079 INFO:tasks.rados.rados.0.smithi100.stdout:971: copy_from oid 50 from oid 8 current snap is 88 2024-04-07T16:19:35.079 INFO:tasks.rados.rados.0.smithi100.stdout:972: snap_remove snap 82 2024-04-07T16:19:35.327 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:35 smithi144 bash[23522]: cluster 2024-04-07T16:19:34.013715+0000 mgr.y (mgr.25073) 27 : cluster 0 pgmap v32: 121 pgs: 121 active+clean; 208 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 9.6 MiB/s rd, 969 KiB/s wr, 20 op/s 2024-04-07T16:19:35.327 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:35 smithi144 bash[23522]: cluster 2024-04-07T16:19:34.083666+0000 mon.a (mon.0) 1166 : cluster 0 osdmap e241: 8 total, 8 up, 8 in 2024-04-07T16:19:35.436 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:35 smithi100 bash[21346]: cluster 2024-04-07T16:19:34.013715+0000 mgr.y (mgr.25073) 27 : cluster 0 pgmap v32: 121 pgs: 121 active+clean; 208 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 9.6 MiB/s rd, 969 KiB/s wr, 20 op/s 2024-04-07T16:19:35.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:35 smithi100 bash[21346]: cluster 2024-04-07T16:19:34.083666+0000 mon.a (mon.0) 1166 : cluster 0 osdmap e241: 8 total, 8 up, 8 in 2024-04-07T16:19:35.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:35 smithi100 bash[31660]: cluster 2024-04-07T16:19:34.013715+0000 mgr.y (mgr.25073) 27 : cluster 0 pgmap v32: 121 pgs: 121 active+clean; 208 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 9.6 MiB/s rd, 969 KiB/s wr, 20 op/s 2024-04-07T16:19:35.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:35 smithi100 bash[31660]: cluster 2024-04-07T16:19:34.083666+0000 mon.a (mon.0) 1166 : cluster 0 osdmap e241: 8 total, 8 up, 8 in 2024-04-07T16:19:36.088 INFO:tasks.rados.rados.0.smithi100.stdout:972: done (2 left) 2024-04-07T16:19:36.088 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 38 current snap is 88 2024-04-07T16:19:36.088 INFO:tasks.rados.rados.0.smithi100.stdout:973: seq_num 304 ranges {0=294912} 2024-04-07T16:19:36.090 INFO:tasks.rados.rados.0.smithi100.stdout:973: writing smithi10066867-38 from 0 to 294912 tid 1 2024-04-07T16:19:36.090 INFO:tasks.rados.rados.0.smithi100.stdout:974: read oid 20 snap -1 2024-04-07T16:19:36.091 INFO:tasks.rados.rados.0.smithi100.stdout:974: expect deleted 2024-04-07T16:19:36.091 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 41 current snap is 88 2024-04-07T16:19:36.091 INFO:tasks.rados.rados.0.smithi100.stdout:970: expect (ObjNum 298 snap 85 seq_num 298) 2024-04-07T16:19:36.098 INFO:tasks.rados.rados.0.smithi100.stdout:975: seq_num 305 ranges {2015232=311296} 2024-04-07T16:19:36.112 INFO:tasks.rados.rados.0.smithi100.stdout:975: writing smithi10066867-41 from 2015232 to 2326528 tid 1 2024-04-07T16:19:36.113 INFO:tasks.rados.rados.0.smithi100.stdout:970: done (4 left) 2024-04-07T16:19:36.113 INFO:tasks.rados.rados.0.smithi100.stdout:976: delete oid 42 current snap is 88 2024-04-07T16:19:36.113 INFO:tasks.rados.rados.0.smithi100.stdout:971: finishing copy_from racing read to smithi10066867-50 2024-04-07T16:19:36.113 INFO:tasks.rados.rados.0.smithi100.stdout:971: finishing copy_from to smithi10066867-50 2024-04-07T16:19:36.113 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 50 v 253 (ObjNum 233 snap 60 seq_num 233) dirty exists 2024-04-07T16:19:36.113 INFO:tasks.rados.rados.0.smithi100.stdout:973: finishing write tid 1 to smithi10066867-38 2024-04-07T16:19:36.113 INFO:tasks.rados.rados.0.smithi100.stdout:973: finishing write tid 2 to smithi10066867-38 2024-04-07T16:19:36.113 INFO:tasks.rados.rados.0.smithi100.stdout:973: finishing write tid 3 to smithi10066867-38 2024-04-07T16:19:36.113 INFO:tasks.rados.rados.0.smithi100.stdout:973: oid 38 updating version 0 to 190 2024-04-07T16:19:36.113 INFO:tasks.rados.rados.0.smithi100.stdout:973: oid 38 version 190 is already newer than 189 2024-04-07T16:19:36.113 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 38 v 190 (ObjNum 304 snap 88 seq_num 304) dirty exists 2024-04-07T16:19:36.113 INFO:tasks.rados.rados.0.smithi100.stdout:973: left oid 38 (ObjNum 304 snap 88 seq_num 304) 2024-04-07T16:19:36.120 INFO:tasks.rados.rados.0.smithi100.stdout:975: finishing write tid 1 to smithi10066867-41 2024-04-07T16:19:36.120 INFO:tasks.rados.rados.0.smithi100.stdout:975: finishing write tid 2 to smithi10066867-41 2024-04-07T16:19:36.121 INFO:tasks.rados.rados.0.smithi100.stdout:971: done (4 left) 2024-04-07T16:19:36.121 INFO:tasks.rados.rados.0.smithi100.stdout:973: done (3 left) 2024-04-07T16:19:36.121 INFO:tasks.rados.rados.0.smithi100.stdout:974: done (2 left) 2024-04-07T16:19:36.121 INFO:tasks.rados.rados.0.smithi100.stdout:976: done (1 left) 2024-04-07T16:19:36.121 INFO:tasks.rados.rados.0.smithi100.stdout:977: snap_remove snap 80 2024-04-07T16:19:36.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:36 smithi144 bash[23522]: cluster 2024-04-07T16:19:35.093235+0000 mon.a (mon.0) 1167 : cluster 0 osdmap e242: 8 total, 8 up, 8 in 2024-04-07T16:19:36.355 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:19:36 smithi144 bash[51952]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:19:36] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-04-07T16:19:36.356 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:19:36 smithi144 systemd[1]: Stopping Ceph prometheus.a for 0702e3a8-f4f8-11ee-b649-cb9ed24678a4... 2024-04-07T16:19:36.356 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:19:36 smithi144 bash[58467]: ts=2024-04-07T16:19:36.189Z caller=main.go:840 level=warn msg="Received SIGTERM, exiting gracefully..." 2024-04-07T16:19:36.356 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:19:36 smithi144 bash[58467]: ts=2024-04-07T16:19:36.189Z caller=main.go:864 level=info msg="Stopping scrape discovery manager..." 2024-04-07T16:19:36.356 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:19:36 smithi144 bash[58467]: ts=2024-04-07T16:19:36.189Z caller=main.go:878 level=info msg="Stopping notify discovery manager..." 2024-04-07T16:19:36.356 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:19:36 smithi144 bash[58467]: ts=2024-04-07T16:19:36.189Z caller=manager.go:988 level=info component="rule manager" msg="Stopping rule manager..." 2024-04-07T16:19:36.356 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:19:36 smithi144 bash[58467]: ts=2024-04-07T16:19:36.189Z caller=main.go:860 level=info msg="Scrape discovery manager stopped" 2024-04-07T16:19:36.356 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:19:36 smithi144 bash[58467]: ts=2024-04-07T16:19:36.189Z caller=main.go:874 level=info msg="Notify discovery manager stopped" 2024-04-07T16:19:36.356 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:19:36 smithi144 bash[58467]: ts=2024-04-07T16:19:36.189Z caller=manager.go:998 level=info component="rule manager" msg="Rule manager stopped" 2024-04-07T16:19:36.356 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:19:36 smithi144 bash[58467]: ts=2024-04-07T16:19:36.189Z caller=main.go:915 level=info msg="Stopping scrape manager..." 2024-04-07T16:19:36.356 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:19:36 smithi144 bash[58467]: ts=2024-04-07T16:19:36.189Z caller=main.go:907 level=info msg="Scrape manager stopped" 2024-04-07T16:19:36.356 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:19:36 smithi144 bash[58467]: ts=2024-04-07T16:19:36.189Z caller=notifier.go:606 level=info component=notifier msg="Stopping notification manager..." 2024-04-07T16:19:36.356 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:19:36 smithi144 bash[58467]: ts=2024-04-07T16:19:36.189Z caller=main.go:1135 level=info msg="Notifier manager stopped" 2024-04-07T16:19:36.356 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:19:36 smithi144 bash[58467]: ts=2024-04-07T16:19:36.190Z caller=main.go:1147 level=info msg="See you next time!" 2024-04-07T16:19:36.436 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:36 smithi100 bash[21346]: cluster 2024-04-07T16:19:35.093235+0000 mon.a (mon.0) 1167 : cluster 0 osdmap e242: 8 total, 8 up, 8 in 2024-04-07T16:19:36.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:36 smithi100 bash[31660]: cluster 2024-04-07T16:19:35.093235+0000 mon.a (mon.0) 1167 : cluster 0 osdmap e242: 8 total, 8 up, 8 in 2024-04-07T16:19:36.901 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:19:36 smithi144 bash[59705]: ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4-prometheus-a 2024-04-07T16:19:36.902 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:19:36 smithi144 bash[59766]: Error response from daemon: No such container: ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4-prometheus-a 2024-04-07T16:19:36.902 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:19:36 smithi144 systemd[1]: ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@prometheus.a.service: Deactivated successfully. 2024-04-07T16:19:36.902 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:19:36 smithi144 systemd[1]: Stopped Ceph prometheus.a for 0702e3a8-f4f8-11ee-b649-cb9ed24678a4. 2024-04-07T16:19:36.902 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:19:36 smithi144 systemd[1]: Started Ceph prometheus.a for 0702e3a8-f4f8-11ee-b649-cb9ed24678a4. 2024-04-07T16:19:37.107 INFO:tasks.rados.rados.0.smithi100.stdout:977: done (1 left) 2024-04-07T16:19:37.107 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 49 current snap is 88 2024-04-07T16:19:37.107 INFO:tasks.rados.rados.0.smithi100.stdout:978: seq_num 307 ranges {1966080=344064} 2024-04-07T16:19:37.140 INFO:tasks.rados.rados.0.smithi100.stdout:978: writing smithi10066867-49 from 1966080 to 2310144 tid 1 2024-04-07T16:19:37.140 INFO:tasks.rados.rados.0.smithi100.stdout:979: copy_from oid 50 from oid 23 current snap is 88 2024-04-07T16:19:37.140 INFO:tasks.rados.rados.0.smithi100.stdout:980: snap_remove snap 77 2024-04-07T16:19:37.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:37 smithi144 bash[23522]: cluster 2024-04-07T16:19:36.014488+0000 mgr.y (mgr.25073) 28 : cluster 0 pgmap v35: 121 pgs: 121 active+clean; 211 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 11 MiB/s rd, 1.1 MiB/s wr, 14 op/s 2024-04-07T16:19:37.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:37 smithi144 bash[23522]: cluster 2024-04-07T16:19:36.102824+0000 mon.a (mon.0) 1168 : cluster 0 osdmap e243: 8 total, 8 up, 8 in 2024-04-07T16:19:37.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:37 smithi144 bash[23522]: audit 2024-04-07T16:19:36.923545+0000 mon.a (mon.0) 1169 : audit 1 from='mgr.25073 ' entity='mgr.y' 2024-04-07T16:19:37.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:37 smithi144 bash[23522]: audit 2024-04-07T16:19:36.949028+0000 mon.a (mon.0) 1170 : audit 1 from='mgr.25073 ' entity='mgr.y' 2024-04-07T16:19:37.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:37 smithi144 bash[23522]: audit 2024-04-07T16:19:37.008124+0000 mon.c (mon.2) 110 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-07T16:19:37.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:37 smithi144 bash[23522]: audit 2024-04-07T16:19:37.017641+0000 mon.c (mon.2) 111 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:19:37.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:37 smithi100 bash[21346]: cluster 2024-04-07T16:19:36.014488+0000 mgr.y (mgr.25073) 28 : cluster 0 pgmap v35: 121 pgs: 121 active+clean; 211 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 11 MiB/s rd, 1.1 MiB/s wr, 14 op/s 2024-04-07T16:19:37.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:37 smithi100 bash[21346]: cluster 2024-04-07T16:19:36.102824+0000 mon.a (mon.0) 1168 : cluster 0 osdmap e243: 8 total, 8 up, 8 in 2024-04-07T16:19:37.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:37 smithi100 bash[21346]: audit 2024-04-07T16:19:36.923545+0000 mon.a (mon.0) 1169 : audit 1 from='mgr.25073 ' entity='mgr.y' 2024-04-07T16:19:37.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:37 smithi100 bash[21346]: audit 2024-04-07T16:19:36.949028+0000 mon.a (mon.0) 1170 : audit 1 from='mgr.25073 ' entity='mgr.y' 2024-04-07T16:19:37.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:37 smithi100 bash[21346]: audit 2024-04-07T16:19:37.008124+0000 mon.c (mon.2) 110 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-07T16:19:37.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:37 smithi100 bash[21346]: audit 2024-04-07T16:19:37.017641+0000 mon.c (mon.2) 111 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:19:37.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:37 smithi100 bash[31660]: cluster 2024-04-07T16:19:36.014488+0000 mgr.y (mgr.25073) 28 : cluster 0 pgmap v35: 121 pgs: 121 active+clean; 211 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 11 MiB/s rd, 1.1 MiB/s wr, 14 op/s 2024-04-07T16:19:37.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:37 smithi100 bash[31660]: cluster 2024-04-07T16:19:36.102824+0000 mon.a (mon.0) 1168 : cluster 0 osdmap e243: 8 total, 8 up, 8 in 2024-04-07T16:19:37.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:37 smithi100 bash[31660]: audit 2024-04-07T16:19:36.923545+0000 mon.a (mon.0) 1169 : audit 1 from='mgr.25073 ' entity='mgr.y' 2024-04-07T16:19:37.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:37 smithi100 bash[31660]: audit 2024-04-07T16:19:36.949028+0000 mon.a (mon.0) 1170 : audit 1 from='mgr.25073 ' entity='mgr.y' 2024-04-07T16:19:37.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:37 smithi100 bash[31660]: audit 2024-04-07T16:19:37.008124+0000 mon.c (mon.2) 110 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-07T16:19:37.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:37 smithi100 bash[31660]: audit 2024-04-07T16:19:37.017641+0000 mon.c (mon.2) 111 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:19:37.886 INFO:tasks.rados.rados.0.smithi100.stdout:975: finishing write tid 3 to smithi10066867-41 2024-04-07T16:19:37.886 INFO:tasks.rados.rados.0.smithi100.stdout:975: oid 41 updating version 0 to 509 2024-04-07T16:19:37.886 INFO:tasks.rados.rados.0.smithi100.stdout:975: oid 41 version 509 is already newer than 508 2024-04-07T16:19:37.886 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 41 v 509 (ObjNum 305 snap 88 seq_num 305) dirty exists 2024-04-07T16:19:37.886 INFO:tasks.rados.rados.0.smithi100.stdout:975: left oid 41 (ObjNum 305 snap 88 seq_num 305) 2024-04-07T16:19:37.886 INFO:tasks.rados.rados.0.smithi100.stdout:978: finishing write tid 1 to smithi10066867-49 2024-04-07T16:19:37.886 INFO:tasks.rados.rados.0.smithi100.stdout:978: finishing write tid 2 to smithi10066867-49 2024-04-07T16:19:37.887 INFO:tasks.rados.rados.0.smithi100.stdout:978: finishing write tid 3 to smithi10066867-49 2024-04-07T16:19:37.887 INFO:tasks.rados.rados.0.smithi100.stdout:978: oid 49 updating version 0 to 511 2024-04-07T16:19:37.887 INFO:tasks.rados.rados.0.smithi100.stdout:978: oid 49 version 511 is already newer than 510 2024-04-07T16:19:37.887 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 49 v 511 (ObjNum 307 snap 88 seq_num 307) dirty exists 2024-04-07T16:19:37.887 INFO:tasks.rados.rados.0.smithi100.stdout:978: left oid 49 (ObjNum 307 snap 88 seq_num 307) 2024-04-07T16:19:37.887 INFO:tasks.rados.rados.0.smithi100.stdout:979: finishing copy_from racing read to smithi10066867-50 2024-04-07T16:19:37.887 INFO:tasks.rados.rados.0.smithi100.stdout:979: finishing copy_from to smithi10066867-50 2024-04-07T16:19:37.887 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 50 v 512 (ObjNum 262 snap 70 seq_num 262) dirty exists 2024-04-07T16:19:37.887 INFO:tasks.rados.rados.0.smithi100.stdout:975: done (3 left) 2024-04-07T16:19:37.887 INFO:tasks.rados.rados.0.smithi100.stdout:978: done (2 left) 2024-04-07T16:19:37.887 INFO:tasks.rados.rados.0.smithi100.stdout:979: done (1 left) 2024-04-07T16:19:37.887 INFO:tasks.rados.rados.0.smithi100.stdout:980: done (0 left) 2024-04-07T16:19:37.887 INFO:tasks.rados.rados.0.smithi100.stdout:981: snap_remove snap 63 2024-04-07T16:19:38.349 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:38 smithi144 bash[23522]: audit 2024-04-07T16:19:37.009692+0000 mgr.y (mgr.25073) 29 : audit 0 from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-07T16:19:38.350 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:38 smithi144 bash[23522]: cluster 2024-04-07T16:19:37.121056+0000 mon.a (mon.0) 1171 : cluster 0 osdmap e244: 8 total, 8 up, 8 in 2024-04-07T16:19:38.350 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:38 smithi144 bash[23522]: cluster 2024-04-07T16:19:37.893219+0000 mon.a (mon.0) 1172 : cluster 0 osdmap e245: 8 total, 8 up, 8 in 2024-04-07T16:19:38.350 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:19:38 smithi144 bash[59794]: ts=2024-04-07T16:19:38.170Z caller=main.go:564 level=info msg="Starting Prometheus Server" mode=server version="(version=2.43.0, branch=HEAD, revision=edfc3bcd025dd6fe296c167a14a216cab1e552ee)" 2024-04-07T16:19:38.350 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:19:38 smithi144 bash[59794]: ts=2024-04-07T16:19:38.170Z caller=main.go:569 level=info build_context="(go=go1.19.7, platform=linux/amd64, user=root@8a0ee342e522, date=20230321-12:56:07, tags=netgo,builtinassets)" 2024-04-07T16:19:38.350 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:19:38 smithi144 bash[59794]: ts=2024-04-07T16:19:38.171Z caller=main.go:570 level=info host_details="(Linux 5.15.0-101-generic #111-Ubuntu SMP Tue Mar 5 20:16:58 UTC 2024 x86_64 smithi144 (none))" 2024-04-07T16:19:38.350 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:19:38 smithi144 bash[59794]: ts=2024-04-07T16:19:38.171Z caller=main.go:571 level=info fd_limits="(soft=1048576, hard=1048576)" 2024-04-07T16:19:38.350 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:19:38 smithi144 bash[59794]: ts=2024-04-07T16:19:38.171Z caller=main.go:572 level=info vm_limits="(soft=unlimited, hard=unlimited)" 2024-04-07T16:19:38.350 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:19:38 smithi144 bash[59794]: ts=2024-04-07T16:19:38.173Z caller=web.go:561 level=info component=web msg="Start listening for connections" address=:9095 2024-04-07T16:19:38.351 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:19:38 smithi144 bash[59794]: ts=2024-04-07T16:19:38.173Z caller=main.go:1005 level=info msg="Starting TSDB ..." 2024-04-07T16:19:38.351 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:19:38 smithi144 bash[59794]: ts=2024-04-07T16:19:38.174Z caller=tls_config.go:232 level=info component=web msg="Listening on" address=[::]:9095 2024-04-07T16:19:38.351 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:19:38 smithi144 bash[59794]: ts=2024-04-07T16:19:38.174Z caller=tls_config.go:235 level=info component=web msg="TLS is disabled." http2=false address=[::]:9095 2024-04-07T16:19:38.351 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:19:38 smithi144 bash[59794]: ts=2024-04-07T16:19:38.177Z caller=head.go:587 level=info component=tsdb msg="Replaying on-disk memory mappable chunks if any" 2024-04-07T16:19:38.351 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:19:38 smithi144 bash[59794]: ts=2024-04-07T16:19:38.177Z caller=head.go:658 level=info component=tsdb msg="On-disk memory mappable chunks replay completed" duration=4.589µs 2024-04-07T16:19:38.351 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:19:38 smithi144 bash[59794]: ts=2024-04-07T16:19:38.177Z caller=head.go:664 level=info component=tsdb msg="Replaying WAL, this may take a while" 2024-04-07T16:19:38.351 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:19:38 smithi144 bash[59794]: ts=2024-04-07T16:19:38.179Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=0 maxSegment=3 2024-04-07T16:19:38.351 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:19:38 smithi144 bash[59794]: ts=2024-04-07T16:19:38.189Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=1 maxSegment=3 2024-04-07T16:19:38.351 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:19:38 smithi144 bash[59794]: ts=2024-04-07T16:19:38.196Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=2 maxSegment=3 2024-04-07T16:19:38.351 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:19:38 smithi144 bash[59794]: ts=2024-04-07T16:19:38.196Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=3 maxSegment=3 2024-04-07T16:19:38.351 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:19:38 smithi144 bash[59794]: ts=2024-04-07T16:19:38.196Z caller=head.go:772 level=info component=tsdb msg="WAL replay completed" checkpoint_replay_duration=37.948µs wal_replay_duration=19.072788ms wbl_replay_duration=141ns total_replay_duration=19.146525ms 2024-04-07T16:19:38.351 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:19:38 smithi144 bash[59794]: ts=2024-04-07T16:19:38.199Z caller=main.go:1026 level=info fs_type=XFS_SUPER_MAGIC 2024-04-07T16:19:38.351 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:19:38 smithi144 bash[59794]: ts=2024-04-07T16:19:38.199Z caller=main.go:1029 level=info msg="TSDB started" 2024-04-07T16:19:38.351 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:19:38 smithi144 bash[59794]: ts=2024-04-07T16:19:38.199Z caller=main.go:1209 level=info msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml 2024-04-07T16:19:38.351 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:19:38 smithi144 bash[59794]: ts=2024-04-07T16:19:38.224Z caller=main.go:1246 level=info msg="Completed loading of configuration file" filename=/etc/prometheus/prometheus.yml totalDuration=24.949785ms db_storage=1.15µs remote_storage=1.319µs web_handler=411ns query_engine=1.112µs scrape=329.683µs scrape_sd=104.945µs notify=13.248µs notify_sd=17.567µs rules=24.059152ms tracing=5.604µs 2024-04-07T16:19:38.352 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:19:38 smithi144 bash[59794]: ts=2024-04-07T16:19:38.224Z caller=main.go:990 level=info msg="Server is ready to receive web requests." 2024-04-07T16:19:38.352 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:19:38 smithi144 bash[59794]: ts=2024-04-07T16:19:38.224Z caller=manager.go:974 level=info component="rule manager" msg="Starting rule manager..." 2024-04-07T16:19:38.436 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:38 smithi100 bash[21346]: audit 2024-04-07T16:19:37.009692+0000 mgr.y (mgr.25073) 29 : audit 0 from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-07T16:19:38.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:38 smithi100 bash[21346]: cluster 2024-04-07T16:19:37.121056+0000 mon.a (mon.0) 1171 : cluster 0 osdmap e244: 8 total, 8 up, 8 in 2024-04-07T16:19:38.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:38 smithi100 bash[21346]: cluster 2024-04-07T16:19:37.893219+0000 mon.a (mon.0) 1172 : cluster 0 osdmap e245: 8 total, 8 up, 8 in 2024-04-07T16:19:38.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:38 smithi100 bash[31660]: audit 2024-04-07T16:19:37.009692+0000 mgr.y (mgr.25073) 29 : audit 0 from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-07T16:19:38.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:38 smithi100 bash[31660]: cluster 2024-04-07T16:19:37.121056+0000 mon.a (mon.0) 1171 : cluster 0 osdmap e244: 8 total, 8 up, 8 in 2024-04-07T16:19:38.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:38 smithi100 bash[31660]: cluster 2024-04-07T16:19:37.893219+0000 mon.a (mon.0) 1172 : cluster 0 osdmap e245: 8 total, 8 up, 8 in 2024-04-07T16:19:38.892 INFO:tasks.rados.rados.0.smithi100.stdout:981: done (0 left) 2024-04-07T16:19:38.892 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 16 current snap is 88 2024-04-07T16:19:38.892 INFO:tasks.rados.rados.0.smithi100.stdout:982: seq_num 308 ranges {3874816=360448} 2024-04-07T16:19:38.945 INFO:tasks.rados.rados.0.smithi100.stdout:982: writing smithi10066867-16 from 3874816 to 4235264 tid 1 2024-04-07T16:19:38.945 INFO:tasks.rados.rados.0.smithi100.stdout:983: read oid 27 snap -1 2024-04-07T16:19:38.945 INFO:tasks.rados.rados.0.smithi100.stdout:983: expect (ObjNum 254 snap 69 seq_num 254) 2024-04-07T16:19:38.945 INFO:tasks.rados.rados.0.smithi100.stdout:984: snap_create 2024-04-07T16:19:38.952 INFO:tasks.rados.rados.0.smithi100.stdout:982: finishing write tid 1 to smithi10066867-16 2024-04-07T16:19:38.953 INFO:tasks.rados.rados.0.smithi100.stdout:982: finishing write tid 2 to smithi10066867-16 2024-04-07T16:19:38.955 INFO:tasks.rados.rados.0.smithi100.stdout:982: finishing write tid 3 to smithi10066867-16 2024-04-07T16:19:38.955 INFO:tasks.rados.rados.0.smithi100.stdout:982: oid 16 updating version 0 to 512 2024-04-07T16:19:38.955 INFO:tasks.rados.rados.0.smithi100.stdout:982: oid 16 updating version 512 to 513 2024-04-07T16:19:38.955 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 16 v 513 (ObjNum 308 snap 88 seq_num 308) dirty exists 2024-04-07T16:19:38.955 INFO:tasks.rados.rados.0.smithi100.stdout:982: left oid 16 (ObjNum 308 snap 88 seq_num 308) 2024-04-07T16:19:38.962 INFO:tasks.rados.rados.0.smithi100.stdout:983: expect (ObjNum 254 snap 69 seq_num 254) 2024-04-07T16:19:39.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:39 smithi144 bash[23522]: cluster 2024-04-07T16:19:38.015236+0000 mgr.y (mgr.25073) 30 : cluster 0 pgmap v39: 121 pgs: 121 active+clean; 211 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 12 MiB/s rd, 1.1 MiB/s wr, 14 op/s 2024-04-07T16:19:39.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:39 smithi144 bash[23522]: cluster 2024-04-07T16:19:38.903874+0000 mon.a (mon.0) 1173 : cluster 0 osdmap e246: 8 total, 8 up, 8 in 2024-04-07T16:19:39.436 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:39 smithi100 bash[21346]: cluster 2024-04-07T16:19:38.015236+0000 mgr.y (mgr.25073) 30 : cluster 0 pgmap v39: 121 pgs: 121 active+clean; 211 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 12 MiB/s rd, 1.1 MiB/s wr, 14 op/s 2024-04-07T16:19:39.436 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:39 smithi100 bash[21346]: cluster 2024-04-07T16:19:38.903874+0000 mon.a (mon.0) 1173 : cluster 0 osdmap e246: 8 total, 8 up, 8 in 2024-04-07T16:19:39.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:39 smithi100 bash[31660]: cluster 2024-04-07T16:19:38.015236+0000 mgr.y (mgr.25073) 30 : cluster 0 pgmap v39: 121 pgs: 121 active+clean; 211 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 12 MiB/s rd, 1.1 MiB/s wr, 14 op/s 2024-04-07T16:19:39.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:39 smithi100 bash[31660]: cluster 2024-04-07T16:19:38.903874+0000 mon.a (mon.0) 1173 : cluster 0 osdmap e246: 8 total, 8 up, 8 in 2024-04-07T16:19:39.891 INFO:tasks.rados.rados.0.smithi100.stdout:982: done (2 left) 2024-04-07T16:19:39.891 INFO:tasks.rados.rados.0.smithi100.stdout:983: done (1 left) 2024-04-07T16:19:39.891 INFO:tasks.rados.rados.0.smithi100.stdout:984: done (0 left) 2024-04-07T16:19:39.891 INFO:tasks.rados.rados.0.smithi100.stdout:985: copy_from oid 2 from oid 18 current snap is 89 2024-04-07T16:19:39.891 INFO:tasks.rados.rados.0.smithi100.stdout:986: snap_create 2024-04-07T16:19:39.908 INFO:tasks.rados.rados.0.smithi100.stdout:985: finishing copy_from racing read to smithi10066867-2 2024-04-07T16:19:39.913 INFO:tasks.rados.rados.0.smithi100.stdout:985: finishing copy_from to smithi10066867-2 2024-04-07T16:19:39.913 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 2 v 532 (ObjNum 260 snap 70 seq_num 260) dirty exists 2024-04-07T16:19:40.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:40 smithi144 bash[23522]: cephadm 2024-04-07T16:19:39.190969+0000 mgr.y (mgr.25073) 31 : cephadm 1 Upgrade: Updating mgr.x 2024-04-07T16:19:40.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:40 smithi144 bash[23522]: audit 2024-04-07T16:19:39.191761+0000 mon.c (mon.2) 112 : audit 1 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-07T16:19:40.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:40 smithi144 bash[23522]: audit 2024-04-07T16:19:39.192636+0000 mon.a (mon.0) 1174 : audit 1 from='mgr.25073 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-07T16:19:40.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:40 smithi144 bash[23522]: audit 2024-04-07T16:19:39.194989+0000 mon.c (mon.2) 113 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-07T16:19:40.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:40 smithi144 bash[23522]: audit 2024-04-07T16:19:39.197062+0000 mon.c (mon.2) 114 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:19:40.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:40 smithi144 bash[23522]: cephadm 2024-04-07T16:19:39.199199+0000 mgr.y (mgr.25073) 32 : cephadm 1 Deploying daemon mgr.x on smithi144 2024-04-07T16:19:40.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:40 smithi144 bash[23522]: cluster 2024-04-07T16:19:39.905217+0000 mon.a (mon.0) 1175 : cluster 0 osdmap e247: 8 total, 8 up, 8 in 2024-04-07T16:19:40.436 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:40 smithi100 bash[21346]: cephadm 2024-04-07T16:19:39.190969+0000 mgr.y (mgr.25073) 31 : cephadm 1 Upgrade: Updating mgr.x 2024-04-07T16:19:40.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:40 smithi100 bash[21346]: audit 2024-04-07T16:19:39.191761+0000 mon.c (mon.2) 112 : audit 1 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-07T16:19:40.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:40 smithi100 bash[21346]: audit 2024-04-07T16:19:39.192636+0000 mon.a (mon.0) 1174 : audit 1 from='mgr.25073 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-07T16:19:40.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:40 smithi100 bash[21346]: audit 2024-04-07T16:19:39.194989+0000 mon.c (mon.2) 113 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-07T16:19:40.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:40 smithi100 bash[21346]: audit 2024-04-07T16:19:39.197062+0000 mon.c (mon.2) 114 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:19:40.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:40 smithi100 bash[21346]: cephadm 2024-04-07T16:19:39.199199+0000 mgr.y (mgr.25073) 32 : cephadm 1 Deploying daemon mgr.x on smithi144 2024-04-07T16:19:40.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:40 smithi100 bash[21346]: cluster 2024-04-07T16:19:39.905217+0000 mon.a (mon.0) 1175 : cluster 0 osdmap e247: 8 total, 8 up, 8 in 2024-04-07T16:19:40.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:40 smithi100 bash[31660]: cephadm 2024-04-07T16:19:39.190969+0000 mgr.y (mgr.25073) 31 : cephadm 1 Upgrade: Updating mgr.x 2024-04-07T16:19:40.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:40 smithi100 bash[31660]: audit 2024-04-07T16:19:39.191761+0000 mon.c (mon.2) 112 : audit 1 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-07T16:19:40.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:40 smithi100 bash[31660]: audit 2024-04-07T16:19:39.192636+0000 mon.a (mon.0) 1174 : audit 1 from='mgr.25073 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-07T16:19:40.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:40 smithi100 bash[31660]: audit 2024-04-07T16:19:39.194989+0000 mon.c (mon.2) 113 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-07T16:19:40.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:40 smithi100 bash[31660]: audit 2024-04-07T16:19:39.197062+0000 mon.c (mon.2) 114 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:19:40.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:40 smithi100 bash[31660]: cephadm 2024-04-07T16:19:39.199199+0000 mgr.y (mgr.25073) 32 : cephadm 1 Deploying daemon mgr.x on smithi144 2024-04-07T16:19:40.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:40 smithi100 bash[31660]: cluster 2024-04-07T16:19:39.905217+0000 mon.a (mon.0) 1175 : cluster 0 osdmap e247: 8 total, 8 up, 8 in 2024-04-07T16:19:40.901 INFO:tasks.rados.rados.0.smithi100.stdout:985: done (1 left) 2024-04-07T16:19:40.902 INFO:tasks.rados.rados.0.smithi100.stdout:986: done (0 left) 2024-04-07T16:19:40.902 INFO:tasks.rados.rados.0.smithi100.stdout:987: read oid 19 snap -1 2024-04-07T16:19:40.902 INFO:tasks.rados.rados.0.smithi100.stdout:987: expect (ObjNum 239 snap 65 seq_num 239) 2024-04-07T16:19:40.902 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 44 current snap is 90 2024-04-07T16:19:40.902 INFO:tasks.rados.rados.0.smithi100.stdout:988: seq_num 309 ranges {0=376832} 2024-04-07T16:19:40.907 INFO:tasks.rados.rados.0.smithi100.stdout:988: writing smithi10066867-44 from 0 to 376832 tid 1 2024-04-07T16:19:40.908 INFO:tasks.rados.rados.0.smithi100.stdout:989: snap_create 2024-04-07T16:19:40.916 INFO:tasks.rados.rados.0.smithi100.stdout:988: finishing write tid 1 to smithi10066867-44 2024-04-07T16:19:40.916 INFO:tasks.rados.rados.0.smithi100.stdout:988: finishing write tid 2 to smithi10066867-44 2024-04-07T16:19:40.918 INFO:tasks.rados.rados.0.smithi100.stdout:988: finishing write tid 3 to smithi10066867-44 2024-04-07T16:19:40.919 INFO:tasks.rados.rados.0.smithi100.stdout:988: oid 44 updating version 0 to 534 2024-04-07T16:19:40.919 INFO:tasks.rados.rados.0.smithi100.stdout:988: oid 44 version 534 is already newer than 533 2024-04-07T16:19:40.919 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 44 v 534 (ObjNum 309 snap 90 seq_num 309) dirty exists 2024-04-07T16:19:40.919 INFO:tasks.rados.rados.0.smithi100.stdout:988: left oid 44 (ObjNum 309 snap 90 seq_num 309) 2024-04-07T16:19:40.919 INFO:tasks.rados.rados.0.smithi100.stdout:987: expect (ObjNum 239 snap 65 seq_num 239) 2024-04-07T16:19:41.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:41 smithi144 bash[23522]: cluster 2024-04-07T16:19:40.015956+0000 mgr.y (mgr.25073) 33 : cluster 0 pgmap v42: 121 pgs: 121 active+clean; 211 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail 2024-04-07T16:19:41.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:41 smithi144 bash[23522]: cluster 2024-04-07T16:19:40.916080+0000 mon.a (mon.0) 1176 : cluster 0 osdmap e248: 8 total, 8 up, 8 in 2024-04-07T16:19:41.436 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:41 smithi100 bash[21346]: cluster 2024-04-07T16:19:40.015956+0000 mgr.y (mgr.25073) 33 : cluster 0 pgmap v42: 121 pgs: 121 active+clean; 211 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail 2024-04-07T16:19:41.436 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:41 smithi100 bash[21346]: cluster 2024-04-07T16:19:40.916080+0000 mon.a (mon.0) 1176 : cluster 0 osdmap e248: 8 total, 8 up, 8 in 2024-04-07T16:19:41.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:41 smithi100 bash[31660]: cluster 2024-04-07T16:19:40.015956+0000 mgr.y (mgr.25073) 33 : cluster 0 pgmap v42: 121 pgs: 121 active+clean; 211 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail 2024-04-07T16:19:41.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:41 smithi100 bash[31660]: cluster 2024-04-07T16:19:40.916080+0000 mon.a (mon.0) 1176 : cluster 0 osdmap e248: 8 total, 8 up, 8 in 2024-04-07T16:19:41.648 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:41 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:19:41.649 INFO:journalctl@ceph.osd.6.smithi144.stdout:Apr 07 16:19:41 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:19:41.649 INFO:journalctl@ceph.osd.7.smithi144.stdout:Apr 07 16:19:41 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:19:41.649 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:19:41 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:19:41.650 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:19:41 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:19:41.650 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:19:41 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:19:41.650 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:19:41 smithi144 systemd[1]: Stopping Ceph mgr.x for 0702e3a8-f4f8-11ee-b649-cb9ed24678a4... 2024-04-07T16:19:41.650 INFO:journalctl@ceph.osd.5.smithi144.stdout:Apr 07 16:19:41 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:19:41.651 INFO:journalctl@ceph.mds.b.smithi144.stdout:Apr 07 16:19:41 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:19:41.651 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 16:19:41 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:19:41.651 INFO:journalctl@ceph.osd.4.smithi144.stdout:Apr 07 16:19:41 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:19:41.904 INFO:tasks.rados.rados.0.smithi100.stdout:987: done (2 left) 2024-04-07T16:19:41.904 INFO:tasks.rados.rados.0.smithi100.stdout:988: done (1 left) 2024-04-07T16:19:41.905 INFO:tasks.rados.rados.0.smithi100.stdout:989: done (0 left) 2024-04-07T16:19:41.905 INFO:tasks.rados.rados.0.smithi100.stdout:990: copy_from oid 44 from oid 47 current snap is 91 2024-04-07T16:19:41.905 INFO:tasks.rados.rados.0.smithi100.stdout:991: copy_from oid 18 from oid 21 current snap is 91 2024-04-07T16:19:41.905 INFO:tasks.rados.rados.0.smithi100.stdout:992: snap_create 2024-04-07T16:19:41.975 INFO:tasks.rados.rados.0.smithi100.stdout:990: finishing copy_from racing read to smithi10066867-44 2024-04-07T16:19:41.985 INFO:tasks.rados.rados.0.smithi100.stdout:990: finishing copy_from to smithi10066867-44 2024-04-07T16:19:41.985 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 44 v 167 (ObjNum 286 snap 82 seq_num 286) dirty exists 2024-04-07T16:19:41.994 INFO:tasks.rados.rados.0.smithi100.stdout:991: finishing copy_from racing read to smithi10066867-18 2024-04-07T16:19:41.995 INFO:tasks.rados.rados.0.smithi100.stdout:991: finishing copy_from to smithi10066867-18 2024-04-07T16:19:41.995 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 18 v 174 (ObjNum 290 snap 84 seq_num 290) dirty exists 2024-04-07T16:19:42.474 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:19:42 smithi144 bash[60076]: ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4-mgr-x 2024-04-07T16:19:42.474 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:19:42 smithi144 systemd[1]: ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@mgr.x.service: Main process exited, code=exited, status=143/n/a 2024-04-07T16:19:42.474 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:19:42 smithi144 bash[60135]: Error response from daemon: No such container: ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4-mgr-x 2024-04-07T16:19:42.474 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:19:42 smithi144 systemd[1]: ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@mgr.x.service: Failed with result 'exit-code'. 2024-04-07T16:19:42.474 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:19:42 smithi144 systemd[1]: Stopped Ceph mgr.x for 0702e3a8-f4f8-11ee-b649-cb9ed24678a4. 2024-04-07T16:19:42.855 INFO:journalctl@ceph.mds.b.smithi144.stdout:Apr 07 16:19:42 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:19:42.855 INFO:journalctl@ceph.osd.4.smithi144.stdout:Apr 07 16:19:42 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:19:42.855 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:19:42 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:19:42.856 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:42 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:19:42.856 INFO:journalctl@ceph.osd.7.smithi144.stdout:Apr 07 16:19:42 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:19:42.856 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:19:42 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:19:42.857 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:19:42 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:19:42.857 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:19:42 smithi144 systemd[1]: Started Ceph mgr.x for 0702e3a8-f4f8-11ee-b649-cb9ed24678a4. 2024-04-07T16:19:42.857 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 16:19:42 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:19:42.857 INFO:journalctl@ceph.osd.6.smithi144.stdout:Apr 07 16:19:42 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:19:42.857 INFO:journalctl@ceph.osd.5.smithi144.stdout:Apr 07 16:19:42 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:19:42.906 INFO:tasks.rados.rados.0.smithi100.stdout:990: done (2 left) 2024-04-07T16:19:42.906 INFO:tasks.rados.rados.0.smithi100.stdout:991: done (1 left) 2024-04-07T16:19:42.906 INFO:tasks.rados.rados.0.smithi100.stdout:992: done (0 left) 2024-04-07T16:19:42.906 INFO:tasks.rados.rados.0.smithi100.stdout:993: read oid 16 snap 87 2024-04-07T16:19:42.906 INFO:tasks.rados.rados.0.smithi100.stdout:993: expect (ObjNum 118 snap 29 seq_num 118) 2024-04-07T16:19:42.906 INFO:tasks.rados.rados.0.smithi100.stdout:994: delete oid 5 current snap is 92 2024-04-07T16:19:42.941 INFO:tasks.rados.rados.0.smithi100.stdout:993: expect (ObjNum 118 snap 29 seq_num 118) 2024-04-07T16:19:43.107 INFO:tasks.rados.rados.0.smithi100.stdout:993: done (1 left) 2024-04-07T16:19:43.107 INFO:tasks.rados.rados.0.smithi100.stdout:994: done (0 left) 2024-04-07T16:19:43.107 INFO:tasks.rados.rados.0.smithi100.stdout:995: delete oid 16 current snap is 92 2024-04-07T16:19:43.112 INFO:tasks.rados.rados.0.smithi100.stdout:995: done (0 left) 2024-04-07T16:19:43.112 INFO:tasks.rados.rados.0.smithi100.stdout:996: snap_remove snap 88 2024-04-07T16:19:43.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:42 smithi100 bash[21346]: cluster 2024-04-07T16:19:41.917831+0000 mon.a (mon.0) 1177 : cluster 0 osdmap e249: 8 total, 8 up, 8 in 2024-04-07T16:19:43.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:42 smithi100 bash[21346]: cluster 2024-04-07T16:19:42.016632+0000 mgr.y (mgr.25073) 34 : cluster 0 pgmap v45: 121 pgs: 121 active+clean; 186 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 2.2 MiB/s rd, 2.6 MiB/s wr, 6 op/s 2024-04-07T16:19:43.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:42 smithi100 bash[21346]: audit 2024-04-07T16:19:42.662118+0000 mon.a (mon.0) 1178 : audit 1 from='mgr.25073 ' entity='mgr.y' 2024-04-07T16:19:43.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:42 smithi100 bash[21346]: audit 2024-04-07T16:19:42.679100+0000 mon.a (mon.0) 1179 : audit 1 from='mgr.25073 ' entity='mgr.y' 2024-04-07T16:19:43.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:42 smithi100 bash[21346]: audit 2024-04-07T16:19:42.722460+0000 mon.c (mon.2) 115 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:19:43.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:42 smithi100 bash[31660]: cluster 2024-04-07T16:19:41.917831+0000 mon.a (mon.0) 1177 : cluster 0 osdmap e249: 8 total, 8 up, 8 in 2024-04-07T16:19:43.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:42 smithi100 bash[31660]: cluster 2024-04-07T16:19:42.016632+0000 mgr.y (mgr.25073) 34 : cluster 0 pgmap v45: 121 pgs: 121 active+clean; 186 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 2.2 MiB/s rd, 2.6 MiB/s wr, 6 op/s 2024-04-07T16:19:43.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:42 smithi100 bash[31660]: audit 2024-04-07T16:19:42.662118+0000 mon.a (mon.0) 1178 : audit 1 from='mgr.25073 ' entity='mgr.y' 2024-04-07T16:19:43.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:42 smithi100 bash[31660]: audit 2024-04-07T16:19:42.679100+0000 mon.a (mon.0) 1179 : audit 1 from='mgr.25073 ' entity='mgr.y' 2024-04-07T16:19:43.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:42 smithi100 bash[31660]: audit 2024-04-07T16:19:42.722460+0000 mon.c (mon.2) 115 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:19:43.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:42 smithi144 bash[23522]: cluster 2024-04-07T16:19:41.917831+0000 mon.a (mon.0) 1177 : cluster 0 osdmap e249: 8 total, 8 up, 8 in 2024-04-07T16:19:43.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:42 smithi144 bash[23522]: cluster 2024-04-07T16:19:42.016632+0000 mgr.y (mgr.25073) 34 : cluster 0 pgmap v45: 121 pgs: 121 active+clean; 186 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 2.2 MiB/s rd, 2.6 MiB/s wr, 6 op/s 2024-04-07T16:19:43.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:42 smithi144 bash[23522]: audit 2024-04-07T16:19:42.662118+0000 mon.a (mon.0) 1178 : audit 1 from='mgr.25073 ' entity='mgr.y' 2024-04-07T16:19:43.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:42 smithi144 bash[23522]: audit 2024-04-07T16:19:42.679100+0000 mon.a (mon.0) 1179 : audit 1 from='mgr.25073 ' entity='mgr.y' 2024-04-07T16:19:43.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:42 smithi144 bash[23522]: audit 2024-04-07T16:19:42.722460+0000 mon.c (mon.2) 115 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:19:43.728 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:19:43 smithi144 bash[60191]: debug 2024-04-07T16:19:43.584+0000 7f1c2a82a700 1 -- 172.21.15.144:0/2565941730 <== mon.1 v2:172.21.15.144:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5647524b25a0 con 0x564753232400 2024-04-07T16:19:43.728 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:19:43 smithi144 bash[59794]: ts=2024-04-07T16:19:43.505Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=nodes name=CephNodeDiskspaceWarning index=4 msg="Evaluating rule failed" rule="alert: CephNodeDiskspaceWarning\nexpr: predict_linear(node_filesystem_free_bytes{device=~\"/.*\"}[2d], 3600 * 24 * 5)\n * on (instance) group_left (nodename) node_uname_info < 0\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.8.4\n severity: warning\n type: ceph_default\nannotations:\n description: Mountpoint {{ $labels.mountpoint }} on {{ $labels.nodename }} will\n be full in less than 5 days based on the 48 hour trailing fill rate.\n summary: Host filesystem free space is getting low\n" err="found duplicate series for the match group {instance=\"smithi100\"} on the right hand-side of the operation: [{__name__=\"node_uname_info\", domainname=\"(none)\", instance=\"smithi100\", job=\"node\", machine=\"x86_64\", nodename=\"smithi100\", release=\"5.15.0-101-generic\", sysname=\"Linux\", version=\"#111-Ubuntu SMP Tue Mar 5 20:16:58 UTC 2024\"}, {__name__=\"node_uname_info\", cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", domainname=\"(none)\", instance=\"smithi100\", job=\"node\", machine=\"x86_64\", nodename=\"smithi100\", release=\"5.15.0-101-generic\", sysname=\"Linux\", version=\"#111-Ubuntu SMP Tue Mar 5 20:16:58 UTC 2024\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:19:43.924 INFO:tasks.rados.rados.0.smithi100.stdout:996: done (0 left) 2024-04-07T16:19:43.925 INFO:tasks.rados.rados.0.smithi100.stdout:997: snap_remove snap 90 2024-04-07T16:19:44.105 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:19:43 smithi144 bash[60191]: debug 2024-04-07T16:19:43.732+0000 7f1c34a3a200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-07T16:19:44.105 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:43 smithi144 bash[23522]: cluster 2024-04-07T16:19:42.921192+0000 mon.a (mon.0) 1180 : cluster 0 osdmap e250: 8 total, 8 up, 8 in 2024-04-07T16:19:44.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:43 smithi100 bash[21346]: cluster 2024-04-07T16:19:42.921192+0000 mon.a (mon.0) 1180 : cluster 0 osdmap e250: 8 total, 8 up, 8 in 2024-04-07T16:19:44.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:43 smithi100 bash[31660]: cluster 2024-04-07T16:19:42.921192+0000 mon.a (mon.0) 1180 : cluster 0 osdmap e250: 8 total, 8 up, 8 in 2024-04-07T16:19:44.935 INFO:tasks.rados.rados.0.smithi100.stdout:997: done (0 left) 2024-04-07T16:19:44.935 INFO:tasks.rados.rados.0.smithi100.stdout:998: snap_create 2024-04-07T16:19:45.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:44 smithi100 bash[21346]: cluster 2024-04-07T16:19:43.938997+0000 mon.a (mon.0) 1181 : cluster 0 osdmap e251: 8 total, 8 up, 8 in 2024-04-07T16:19:45.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:44 smithi100 bash[21346]: cluster 2024-04-07T16:19:44.017387+0000 mgr.y (mgr.25073) 35 : cluster 0 pgmap v48: 121 pgs: 121 active+clean; 186 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 2.2 MiB/s rd, 2.6 MiB/s wr, 6 op/s 2024-04-07T16:19:45.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:44 smithi100 bash[21346]: audit 2024-04-07T16:19:44.134398+0000 mon.c (mon.2) 116 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:19:45.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:44 smithi100 bash[31660]: cluster 2024-04-07T16:19:43.938997+0000 mon.a (mon.0) 1181 : cluster 0 osdmap e251: 8 total, 8 up, 8 in 2024-04-07T16:19:45.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:44 smithi100 bash[31660]: cluster 2024-04-07T16:19:44.017387+0000 mgr.y (mgr.25073) 35 : cluster 0 pgmap v48: 121 pgs: 121 active+clean; 186 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 2.2 MiB/s rd, 2.6 MiB/s wr, 6 op/s 2024-04-07T16:19:45.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:44 smithi100 bash[31660]: audit 2024-04-07T16:19:44.134398+0000 mon.c (mon.2) 116 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:19:45.339 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:44 smithi144 bash[23522]: cluster 2024-04-07T16:19:43.938997+0000 mon.a (mon.0) 1181 : cluster 0 osdmap e251: 8 total, 8 up, 8 in 2024-04-07T16:19:45.340 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:44 smithi144 bash[23522]: cluster 2024-04-07T16:19:44.017387+0000 mgr.y (mgr.25073) 35 : cluster 0 pgmap v48: 121 pgs: 121 active+clean; 186 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 2.2 MiB/s rd, 2.6 MiB/s wr, 6 op/s 2024-04-07T16:19:45.340 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:44 smithi144 bash[23522]: audit 2024-04-07T16:19:44.134398+0000 mon.c (mon.2) 116 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:19:45.605 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:19:45 smithi144 bash[60191]: debug 2024-04-07T16:19:45.344+0000 7f1c34a3a200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-07T16:19:45.605 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:19:45 smithi144 bash[60191]: debug 2024-04-07T16:19:45.440+0000 7f1c34a3a200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-07T16:19:45.936 INFO:tasks.rados.rados.0.smithi100.stdout:998: done (0 left) 2024-04-07T16:19:45.936 INFO:tasks.rados.rados.0.smithi100.stdout:999: rollback oid 48 current snap is 93 2024-04-07T16:19:45.936 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 48 to 85 2024-04-07T16:19:45.936 INFO:tasks.rados.rados.0.smithi100.stdout:1000: copy_from oid 4 from oid 24 current snap is 93 2024-04-07T16:19:45.937 INFO:tasks.rados.rados.0.smithi100.stdout:1001: snap_create 2024-04-07T16:19:45.957 INFO:tasks.rados.rados.0.smithi100.stdout:999: finishing rollback tid 0 to smithi10066867-48 2024-04-07T16:19:45.962 INFO:tasks.rados.rados.0.smithi100.stdout:999: finishing rollback tid 1 to smithi10066867-48 2024-04-07T16:19:45.962 INFO:tasks.rados.rados.0.smithi100.stdout:999: finishing rollback tid 2 to smithi10066867-48 2024-04-07T16:19:45.962 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 48 v 259 (ObjNum 280 snap 80 seq_num 280) dirty exists 2024-04-07T16:19:45.993 INFO:tasks.rados.rados.0.smithi100.stdout:1000: finishing copy_from racing read to smithi10066867-4 2024-04-07T16:19:46.004 INFO:tasks.rados.rados.0.smithi100.stdout:1000: finishing copy_from to smithi10066867-4 2024-04-07T16:19:46.004 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 4 v 316 (ObjNum 302 snap 87 seq_num 302) dirty exists 2024-04-07T16:19:46.105 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:45 smithi144 bash[23522]: cluster 2024-04-07T16:19:44.948378+0000 mon.a (mon.0) 1182 : cluster 0 osdmap e252: 8 total, 8 up, 8 in 2024-04-07T16:19:46.105 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:19:45 smithi144 bash[60191]: debug 2024-04-07T16:19:45.692+0000 7f1c34a3a200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-07T16:19:46.436 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:45 smithi100 bash[21346]: cluster 2024-04-07T16:19:44.948378+0000 mon.a (mon.0) 1182 : cluster 0 osdmap e252: 8 total, 8 up, 8 in 2024-04-07T16:19:46.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:45 smithi100 bash[31660]: cluster 2024-04-07T16:19:44.948378+0000 mon.a (mon.0) 1182 : cluster 0 osdmap e252: 8 total, 8 up, 8 in 2024-04-07T16:19:46.855 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:19:46 smithi144 bash[60191]: debug 2024-04-07T16:19:46.508+0000 7f1c34a3a200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-07T16:19:46.855 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:19:46 smithi144 bash[60191]: debug 2024-04-07T16:19:46.620+0000 7f1c34a3a200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-07T16:19:46.855 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:19:46 smithi144 bash[60191]: debug 2024-04-07T16:19:46.724+0000 7f1c34a3a200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-07T16:19:46.957 INFO:tasks.rados.rados.0.smithi100.stdout:999: done (2 left) 2024-04-07T16:19:46.957 INFO:tasks.rados.rados.0.smithi100.stdout:1000: done (1 left) 2024-04-07T16:19:46.957 INFO:tasks.rados.rados.0.smithi100.stdout:1001: done (0 left) 2024-04-07T16:19:46.957 INFO:tasks.rados.rados.0.smithi100.stdout:1002: read oid 34 snap 68 2024-04-07T16:19:46.957 INFO:tasks.rados.rados.0.smithi100.stdout:1002: expect (ObjNum 233 snap 60 seq_num 233) 2024-04-07T16:19:46.957 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 45 current snap is 94 2024-04-07T16:19:46.957 INFO:tasks.rados.rados.0.smithi100.stdout:1003: seq_num 312 ranges {4218880=434176} 2024-04-07T16:19:47.024 INFO:tasks.rados.rados.0.smithi100.stdout:1003: writing smithi10066867-45 from 4218880 to 4653056 tid 1 2024-04-07T16:19:47.024 INFO:tasks.rados.rados.0.smithi100.stdout:1004: delete oid 48 current snap is 94 2024-04-07T16:19:47.025 INFO:tasks.rados.rados.0.smithi100.stdout:1002: expect (ObjNum 233 snap 60 seq_num 233) 2024-04-07T16:19:47.190 INFO:tasks.rados.rados.0.smithi100.stdout:1003: finishing write tid 1 to smithi10066867-45 2024-04-07T16:19:47.190 INFO:tasks.rados.rados.0.smithi100.stdout:1003: finishing write tid 2 to smithi10066867-45 2024-04-07T16:19:47.191 INFO:tasks.rados.rados.0.smithi100.stdout:1002: done (2 left) 2024-04-07T16:19:47.191 INFO:tasks.rados.rados.0.smithi100.stdout:1004: done (1 left) 2024-04-07T16:19:47.191 INFO:tasks.rados.rados.0.smithi100.stdout:1005: setattr oid 50 current snap is 94 2024-04-07T16:19:47.191 INFO:tasks.rados.rados.0.smithi100.stdout:1003: finishing write tid 3 to smithi10066867-45 2024-04-07T16:19:47.191 INFO:tasks.rados.rados.0.smithi100.stdout:1003: oid 45 updating version 0 to 513 2024-04-07T16:19:47.191 INFO:tasks.rados.rados.0.smithi100.stdout:1003: oid 45 updating version 513 to 514 2024-04-07T16:19:47.191 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 45 v 514 (ObjNum 312 snap 94 seq_num 312) dirty exists 2024-04-07T16:19:47.191 INFO:tasks.rados.rados.0.smithi100.stdout:1003: left oid 45 (ObjNum 312 snap 94 seq_num 312) 2024-04-07T16:19:47.192 INFO:tasks.rados.rados.0.smithi100.stdout:1003: done (1 left) 2024-04-07T16:19:47.192 INFO:tasks.rados.rados.0.smithi100.stdout:1006: rollback oid 29 current snap is 94 2024-04-07T16:19:47.192 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 29 to 92 2024-04-07T16:19:47.192 INFO:tasks.rados.rados.0.smithi100.stdout:1007: snap_create 2024-04-07T16:19:47.195 INFO:tasks.rados.rados.0.smithi100.stdout:1006: finishing rollback tid 1 to smithi10066867-29 2024-04-07T16:19:47.195 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 29 v 0 (ObjNum 33 snap 0 seq_num 2702436304) dirty dne 2024-04-07T16:19:47.198 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 50 v 515 (ObjNum 262 snap 70 seq_num 262) dirty exists 2024-04-07T16:19:47.205 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:46 smithi144 bash[23522]: cluster 2024-04-07T16:19:45.950074+0000 mon.a (mon.0) 1183 : cluster 0 osdmap e253: 8 total, 8 up, 8 in 2024-04-07T16:19:47.205 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:46 smithi144 bash[23522]: cluster 2024-04-07T16:19:46.018239+0000 mgr.y (mgr.25073) 36 : cluster 0 pgmap v51: 121 pgs: 121 active+clean; 199 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 4.1 MiB/s rd, 3.3 MiB/s wr, 4 op/s 2024-04-07T16:19:47.206 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:19:47 smithi144 bash[60191]: debug 2024-04-07T16:19:47.092+0000 7f1c34a3a200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-07T16:19:47.436 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:46 smithi100 bash[21346]: cluster 2024-04-07T16:19:45.950074+0000 mon.a (mon.0) 1183 : cluster 0 osdmap e253: 8 total, 8 up, 8 in 2024-04-07T16:19:47.436 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:46 smithi100 bash[21346]: cluster 2024-04-07T16:19:46.018239+0000 mgr.y (mgr.25073) 36 : cluster 0 pgmap v51: 121 pgs: 121 active+clean; 199 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 4.1 MiB/s rd, 3.3 MiB/s wr, 4 op/s 2024-04-07T16:19:47.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:46 smithi100 bash[31660]: cluster 2024-04-07T16:19:45.950074+0000 mon.a (mon.0) 1183 : cluster 0 osdmap e253: 8 total, 8 up, 8 in 2024-04-07T16:19:47.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:46 smithi100 bash[31660]: cluster 2024-04-07T16:19:46.018239+0000 mgr.y (mgr.25073) 36 : cluster 0 pgmap v51: 121 pgs: 121 active+clean; 199 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 4.1 MiB/s rd, 3.3 MiB/s wr, 4 op/s 2024-04-07T16:19:47.605 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:19:47 smithi144 bash[60191]: debug 2024-04-07T16:19:47.212+0000 7f1c34a3a200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-07T16:19:47.605 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:19:47 smithi144 bash[60191]: debug 2024-04-07T16:19:47.316+0000 7f1c34a3a200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-07T16:19:47.883 INFO:tasks.rados.rados.0.smithi100.stdout:1005: done (2 left) 2024-04-07T16:19:47.883 INFO:tasks.rados.rados.0.smithi100.stdout:1006: done (1 left) 2024-04-07T16:19:47.883 INFO:tasks.rados.rados.0.smithi100.stdout:1007: done (0 left) 2024-04-07T16:19:47.883 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 36 current snap is 95 2024-04-07T16:19:47.883 INFO:tasks.rados.rados.0.smithi100.stdout:1008: seq_num 314 ranges {0=466944} 2024-04-07T16:19:47.890 INFO:tasks.rados.rados.0.smithi100.stdout:1008: writing smithi10066867-36 from 0 to 466944 tid 1 2024-04-07T16:19:47.890 INFO:tasks.rados.rados.0.smithi100.stdout:1009: read oid 24 snap -1 2024-04-07T16:19:47.891 INFO:tasks.rados.rados.0.smithi100.stdout:1009: expect (ObjNum 302 snap 87 seq_num 302) 2024-04-07T16:19:47.891 INFO:tasks.rados.rados.0.smithi100.stdout:1010: snap_remove snap 91 2024-04-07T16:19:47.944 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:19:47 smithi144 bash[60191]: debug 2024-04-07T16:19:47.684+0000 7f1c34a3a200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-07T16:19:48.242 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:19:47 smithi144 bash[60191]: debug 2024-04-07T16:19:47.948+0000 7f1c34a3a200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-07T16:19:48.243 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:47 smithi144 bash[23522]: cluster 2024-04-07T16:19:46.972194+0000 mon.a (mon.0) 1184 : cluster 0 osdmap e254: 8 total, 8 up, 8 in 2024-04-07T16:19:48.243 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:47 smithi144 bash[23522]: cluster 2024-04-07T16:19:47.894372+0000 mon.a (mon.0) 1185 : cluster 0 osdmap e255: 8 total, 8 up, 8 in 2024-04-07T16:19:48.417 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:47 smithi100 bash[21346]: cluster 2024-04-07T16:19:46.972194+0000 mon.a (mon.0) 1184 : cluster 0 osdmap e254: 8 total, 8 up, 8 in 2024-04-07T16:19:48.417 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:47 smithi100 bash[21346]: cluster 2024-04-07T16:19:47.894372+0000 mon.a (mon.0) 1185 : cluster 0 osdmap e255: 8 total, 8 up, 8 in 2024-04-07T16:19:48.418 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:47 smithi100 bash[31660]: cluster 2024-04-07T16:19:46.972194+0000 mon.a (mon.0) 1184 : cluster 0 osdmap e254: 8 total, 8 up, 8 in 2024-04-07T16:19:48.418 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:47 smithi100 bash[31660]: cluster 2024-04-07T16:19:47.894372+0000 mon.a (mon.0) 1185 : cluster 0 osdmap e255: 8 total, 8 up, 8 in 2024-04-07T16:19:48.605 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:19:48 smithi144 bash[60191]: debug 2024-04-07T16:19:48.248+0000 7f1c34a3a200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-07T16:19:48.605 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:19:48 smithi144 bash[60191]: debug 2024-04-07T16:19:48.384+0000 7f1c34a3a200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-07T16:19:48.686 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:19:48 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:19:48] "GET /metrics HTTP/1.1" 200 36084 "" "Prometheus/2.43.0" 2024-04-07T16:19:48.887 INFO:tasks.rados.rados.0.smithi100.stdout:1010: done (2 left) 2024-04-07T16:19:48.887 INFO:tasks.rados.rados.0.smithi100.stdout:1011: read oid 22 snap -1 2024-04-07T16:19:48.887 INFO:tasks.rados.rados.0.smithi100.stdout:1011: expect (ObjNum 294 snap 84 seq_num 294) 2024-04-07T16:19:48.887 INFO:tasks.rados.rados.0.smithi100.stdout:1012: read oid 46 snap -1 2024-04-07T16:19:48.887 INFO:tasks.rados.rados.0.smithi100.stdout:1012: expect (ObjNum 272 snap 73 seq_num 272) 2024-04-07T16:19:48.887 INFO:tasks.rados.rados.0.smithi100.stdout:1013: snap_remove snap 94 2024-04-07T16:19:48.962 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:19:48 smithi144 bash[60191]: debug 2024-04-07T16:19:48.688+0000 7f1c34a3a200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-07T16:19:49.355 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:19:49 smithi144 bash[60191]: debug 2024-04-07T16:19:49.136+0000 7f1c34a3a200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-07T16:19:49.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:48 smithi144 bash[23522]: cluster 2024-04-07T16:19:48.019009+0000 mgr.y (mgr.25073) 37 : cluster 0 pgmap v54: 121 pgs: 121 active+clean; 199 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 4.1 MiB/s rd, 3.3 MiB/s wr, 4 op/s 2024-04-07T16:19:49.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:48 smithi144 bash[23522]: audit 2024-04-07T16:19:48.774240+0000 mon.a (mon.0) 1186 : audit 1 from='mgr.25073 ' entity='mgr.y' 2024-04-07T16:19:49.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:48 smithi144 bash[23522]: audit 2024-04-07T16:19:48.791752+0000 mon.a (mon.0) 1187 : audit 1 from='mgr.25073 ' entity='mgr.y' 2024-04-07T16:19:49.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:48 smithi144 bash[23522]: cluster 2024-04-07T16:19:48.899230+0000 mon.a (mon.0) 1188 : cluster 0 osdmap e256: 8 total, 8 up, 8 in 2024-04-07T16:19:49.436 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:48 smithi100 bash[21346]: cluster 2024-04-07T16:19:48.019009+0000 mgr.y (mgr.25073) 37 : cluster 0 pgmap v54: 121 pgs: 121 active+clean; 199 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 4.1 MiB/s rd, 3.3 MiB/s wr, 4 op/s 2024-04-07T16:19:49.436 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:48 smithi100 bash[21346]: audit 2024-04-07T16:19:48.774240+0000 mon.a (mon.0) 1186 : audit 1 from='mgr.25073 ' entity='mgr.y' 2024-04-07T16:19:49.436 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:48 smithi100 bash[21346]: audit 2024-04-07T16:19:48.791752+0000 mon.a (mon.0) 1187 : audit 1 from='mgr.25073 ' entity='mgr.y' 2024-04-07T16:19:49.436 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:48 smithi100 bash[21346]: cluster 2024-04-07T16:19:48.899230+0000 mon.a (mon.0) 1188 : cluster 0 osdmap e256: 8 total, 8 up, 8 in 2024-04-07T16:19:49.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:48 smithi100 bash[31660]: cluster 2024-04-07T16:19:48.019009+0000 mgr.y (mgr.25073) 37 : cluster 0 pgmap v54: 121 pgs: 121 active+clean; 199 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 4.1 MiB/s rd, 3.3 MiB/s wr, 4 op/s 2024-04-07T16:19:49.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:48 smithi100 bash[31660]: audit 2024-04-07T16:19:48.774240+0000 mon.a (mon.0) 1186 : audit 1 from='mgr.25073 ' entity='mgr.y' 2024-04-07T16:19:49.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:48 smithi100 bash[31660]: audit 2024-04-07T16:19:48.791752+0000 mon.a (mon.0) 1187 : audit 1 from='mgr.25073 ' entity='mgr.y' 2024-04-07T16:19:49.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:48 smithi100 bash[31660]: cluster 2024-04-07T16:19:48.899230+0000 mon.a (mon.0) 1188 : cluster 0 osdmap e256: 8 total, 8 up, 8 in 2024-04-07T16:19:49.769 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:19:49 smithi144 bash[60191]: debug 2024-04-07T16:19:49.384+0000 7f1c34a3a200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-07T16:19:49.769 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:19:49 smithi144 bash[60191]: debug 2024-04-07T16:19:49.512+0000 7f1c34a3a200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-07T16:19:49.888 INFO:tasks.rados.rados.0.smithi100.stdout:1013: done (4 left) 2024-04-07T16:19:49.888 INFO:tasks.rados.rados.0.smithi100.stdout:1014: read oid 40 snap 75 2024-04-07T16:19:49.888 INFO:tasks.rados.rados.0.smithi100.stdout:1014: expect (ObjNum 214 snap 58 seq_num 214) 2024-04-07T16:19:49.888 INFO:tasks.rados.rados.0.smithi100.stdout:1015: read oid 35 snap -1 2024-04-07T16:19:49.888 INFO:tasks.rados.rados.0.smithi100.stdout:1015: expect (ObjNum 271 snap 73 seq_num 271) 2024-04-07T16:19:49.888 INFO:tasks.rados.rados.0.smithi100.stdout:1016: setattr oid 11 current snap is 95 2024-04-07T16:19:49.888 INFO:tasks.rados.rados.0.smithi100.stdout:1008: finishing write tid 1 to smithi10066867-36 2024-04-07T16:19:49.888 INFO:tasks.rados.rados.0.smithi100.stdout:1008: finishing write tid 2 to smithi10066867-36 2024-04-07T16:19:49.889 INFO:tasks.rados.rados.0.smithi100.stdout:1008: finishing write tid 3 to smithi10066867-36 2024-04-07T16:19:49.889 INFO:tasks.rados.rados.0.smithi100.stdout:1008: oid 36 updating version 0 to 338 2024-04-07T16:19:49.889 INFO:tasks.rados.rados.0.smithi100.stdout:1008: oid 36 version 338 is already newer than 337 2024-04-07T16:19:49.889 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 36 v 338 (ObjNum 314 snap 95 seq_num 314) dirty exists 2024-04-07T16:19:49.889 INFO:tasks.rados.rados.0.smithi100.stdout:1008: left oid 36 (ObjNum 314 snap 95 seq_num 314) 2024-04-07T16:19:49.889 INFO:tasks.rados.rados.0.smithi100.stdout:1009: expect (ObjNum 302 snap 87 seq_num 302) 2024-04-07T16:19:50.105 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:19:49 smithi144 bash[60191]: debug 2024-04-07T16:19:49.776+0000 7f1c34a3a200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-07T16:19:50.505 INFO:tasks.rados.rados.0.smithi100.stdout:1012: expect (ObjNum 272 snap 73 seq_num 272) 2024-04-07T16:19:50.609 INFO:tasks.rados.rados.0.smithi100.stdout:1011: expect (ObjNum 294 snap 84 seq_num 294) 2024-04-07T16:19:50.689 INFO:tasks.rados.rados.0.smithi100.stdout:1015: expect (ObjNum 271 snap 73 seq_num 271) 2024-04-07T16:19:50.876 INFO:tasks.rados.rados.0.smithi100.stdout:1014: expect (ObjNum 214 snap 58 seq_num 214) 2024-04-07T16:19:51.105 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:19:50 smithi144 bash[60191]: debug 2024-04-07T16:19:50.828+0000 7f1c34a3a200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-07T16:19:51.105 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:19:50 smithi144 bash[60191]: debug 2024-04-07T16:19:50.944+0000 7f1c34a3a200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-07T16:19:51.105 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:50 smithi144 bash[23522]: cluster 2024-04-07T16:19:49.903208+0000 mon.a (mon.0) 1189 : cluster 0 osdmap e257: 8 total, 8 up, 8 in 2024-04-07T16:19:51.105 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:50 smithi144 bash[23522]: audit 2024-04-07T16:19:49.949479+0000 mon.a (mon.0) 1190 : audit 1 from='mgr.25073 ' entity='mgr.y' 2024-04-07T16:19:51.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:50 smithi144 bash[23522]: audit 2024-04-07T16:19:49.961786+0000 mon.a (mon.0) 1191 : audit 1 from='mgr.25073 ' entity='mgr.y' 2024-04-07T16:19:51.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:50 smithi144 bash[23522]: cluster 2024-04-07T16:19:50.019480+0000 mgr.y (mgr.25073) 38 : cluster 0 pgmap v57: 121 pgs: 121 active+clean; 199 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail 2024-04-07T16:19:51.183 INFO:tasks.rados.rados.0.smithi100.stdout:1008: done (6 left) 2024-04-07T16:19:51.183 INFO:tasks.rados.rados.0.smithi100.stdout:1009: done (5 left) 2024-04-07T16:19:51.183 INFO:tasks.rados.rados.0.smithi100.stdout:1011: done (4 left) 2024-04-07T16:19:51.184 INFO:tasks.rados.rados.0.smithi100.stdout:1012: done (3 left) 2024-04-07T16:19:51.184 INFO:tasks.rados.rados.0.smithi100.stdout:1014: done (2 left) 2024-04-07T16:19:51.184 INFO:tasks.rados.rados.0.smithi100.stdout:1015: done (1 left) 2024-04-07T16:19:51.184 INFO:tasks.rados.rados.0.smithi100.stdout:1017: rmattr oid 40 current snap is 95 2024-04-07T16:19:51.184 INFO:tasks.rados.rados.0.smithi100.stdout:1018: read oid 9 snap -1 2024-04-07T16:19:51.184 INFO:tasks.rados.rados.0.smithi100.stdout:1018: expect (ObjNum 206 snap 56 seq_num 206) 2024-04-07T16:19:51.184 INFO:tasks.rados.rados.0.smithi100.stdout:1019: rmattr oid 50 current snap is 95 2024-04-07T16:19:51.184 INFO:tasks.rados.rados.0.smithi100.stdout:1020: setattr oid 23 current snap is 95 2024-04-07T16:19:51.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:50 smithi100 bash[21346]: cluster 2024-04-07T16:19:49.903208+0000 mon.a (mon.0) 1189 : cluster 0 osdmap e257: 8 total, 8 up, 8 in 2024-04-07T16:19:51.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:50 smithi100 bash[21346]: audit 2024-04-07T16:19:49.949479+0000 mon.a (mon.0) 1190 : audit 1 from='mgr.25073 ' entity='mgr.y' 2024-04-07T16:19:51.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:50 smithi100 bash[21346]: audit 2024-04-07T16:19:49.961786+0000 mon.a (mon.0) 1191 : audit 1 from='mgr.25073 ' entity='mgr.y' 2024-04-07T16:19:51.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:50 smithi100 bash[21346]: cluster 2024-04-07T16:19:50.019480+0000 mgr.y (mgr.25073) 38 : cluster 0 pgmap v57: 121 pgs: 121 active+clean; 199 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail 2024-04-07T16:19:51.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:50 smithi100 bash[31660]: cluster 2024-04-07T16:19:49.903208+0000 mon.a (mon.0) 1189 : cluster 0 osdmap e257: 8 total, 8 up, 8 in 2024-04-07T16:19:51.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:50 smithi100 bash[31660]: audit 2024-04-07T16:19:49.949479+0000 mon.a (mon.0) 1190 : audit 1 from='mgr.25073 ' entity='mgr.y' 2024-04-07T16:19:51.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:50 smithi100 bash[31660]: audit 2024-04-07T16:19:49.961786+0000 mon.a (mon.0) 1191 : audit 1 from='mgr.25073 ' entity='mgr.y' 2024-04-07T16:19:51.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:50 smithi100 bash[31660]: cluster 2024-04-07T16:19:50.019480+0000 mgr.y (mgr.25073) 38 : cluster 0 pgmap v57: 121 pgs: 121 active+clean; 199 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail 2024-04-07T16:19:51.188 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 50 v 516 (ObjNum 262 snap 70 seq_num 262) dirty exists 2024-04-07T16:19:51.190 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 40 v 535 (ObjNum 303 snap 87 seq_num 303) dirty exists 2024-04-07T16:19:51.190 INFO:tasks.rados.rados.0.smithi100.stdout:1017: done (4 left) 2024-04-07T16:19:51.190 INFO:tasks.rados.rados.0.smithi100.stdout:1019: done (3 left) 2024-04-07T16:19:51.190 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 41 current snap is 95 2024-04-07T16:19:51.190 INFO:tasks.rados.rados.0.smithi100.stdout:1021: seq_num 315 ranges {2326528=483328} 2024-04-07T16:19:51.235 INFO:tasks.rados.rados.0.smithi100.stdout:1021: writing smithi10066867-41 from 2326528 to 2809856 tid 1 2024-04-07T16:19:51.235 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 27 current snap is 95 2024-04-07T16:19:51.235 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 11 v 610 (ObjNum 277 snap 80 seq_num 277) dirty exists 2024-04-07T16:19:51.235 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 23 v 612 (ObjNum 262 snap 70 seq_num 262) dirty exists 2024-04-07T16:19:51.235 INFO:tasks.rados.rados.0.smithi100.stdout:1018: expect (ObjNum 206 snap 56 seq_num 206) 2024-04-07T16:19:51.295 INFO:tasks.rados.rados.0.smithi100.stdout:1021: finishing write tid 1 to smithi10066867-41 2024-04-07T16:19:51.295 INFO:tasks.rados.rados.0.smithi100.stdout:1021: finishing write tid 2 to smithi10066867-41 2024-04-07T16:19:51.295 INFO:tasks.rados.rados.0.smithi100.stdout:1021: finishing write tid 3 to smithi10066867-41 2024-04-07T16:19:51.295 INFO:tasks.rados.rados.0.smithi100.stdout:1021: oid 41 updating version 0 to 516 2024-04-07T16:19:51.295 INFO:tasks.rados.rados.0.smithi100.stdout:1021: oid 41 updating version 516 to 517 2024-04-07T16:19:51.295 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 41 v 517 (ObjNum 315 snap 95 seq_num 315) dirty exists 2024-04-07T16:19:51.295 INFO:tasks.rados.rados.0.smithi100.stdout:1021: left oid 41 (ObjNum 315 snap 95 seq_num 315) 2024-04-07T16:19:51.295 INFO:tasks.rados.rados.0.smithi100.stdout:1022: seq_num 316 ranges {2826240=499712} 2024-04-07T16:19:51.343 INFO:tasks.rados.rados.0.smithi100.stdout:1022: writing smithi10066867-27 from 2826240 to 3325952 tid 1 2024-04-07T16:19:51.343 INFO:tasks.rados.rados.0.smithi100.stdout:1016: done (4 left) 2024-04-07T16:19:51.343 INFO:tasks.rados.rados.0.smithi100.stdout:1018: done (3 left) 2024-04-07T16:19:51.343 INFO:tasks.rados.rados.0.smithi100.stdout:1020: done (2 left) 2024-04-07T16:19:51.343 INFO:tasks.rados.rados.0.smithi100.stdout:1021: done (1 left) 2024-04-07T16:19:51.343 INFO:tasks.rados.rados.0.smithi100.stdout:1023: setattr oid 30 current snap is 95 2024-04-07T16:19:51.346 INFO:tasks.rados.rados.0.smithi100.stdout:1024: read oid 9 snap -1 2024-04-07T16:19:51.346 INFO:tasks.rados.rados.0.smithi100.stdout:1024: expect (ObjNum 206 snap 56 seq_num 206) 2024-04-07T16:19:51.346 INFO:tasks.rados.rados.0.smithi100.stdout:1025: snap_remove snap 86 2024-04-07T16:19:51.605 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:19:51 smithi144 bash[60191]: debug 2024-04-07T16:19:51.256+0000 7f1c34a3a200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-07T16:19:51.605 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:19:51 smithi144 bash[60191]: debug 2024-04-07T16:19:51.380+0000 7f1c34a3a200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-07T16:19:51.954 INFO:tasks.rados.rados.0.smithi100.stdout:1025: done (3 left) 2024-04-07T16:19:51.954 INFO:tasks.rados.rados.0.smithi100.stdout:1026: read oid 24 snap -1 2024-04-07T16:19:51.954 INFO:tasks.rados.rados.0.smithi100.stdout:1026: expect (ObjNum 302 snap 87 seq_num 302) 2024-04-07T16:19:51.954 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 20 current snap is 95 2024-04-07T16:19:51.955 INFO:tasks.rados.rados.0.smithi100.stdout:1027: seq_num 317 ranges {0=516096} 2024-04-07T16:19:51.958 INFO:tasks.rados.rados.0.smithi100.stdout:1027: writing smithi10066867-20 from 0 to 516096 tid 1 2024-04-07T16:19:51.958 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 5 current snap is 95 2024-04-07T16:19:51.958 INFO:tasks.rados.rados.0.smithi100.stdout:1022: finishing write tid 1 to smithi10066867-27 2024-04-07T16:19:51.958 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 30 v 472 (ObjNum 106 snap 20 seq_num 106) dirty exists 2024-04-07T16:19:51.958 INFO:tasks.rados.rados.0.smithi100.stdout:1022: finishing write tid 2 to smithi10066867-27 2024-04-07T16:19:51.958 INFO:tasks.rados.rados.0.smithi100.stdout:1028: seq_num 318 ranges {0=532480} 2024-04-07T16:19:51.964 INFO:tasks.rados.rados.0.smithi100.stdout:1028: writing smithi10066867-5 from 0 to 532480 tid 1 2024-04-07T16:19:51.964 INFO:tasks.rados.rados.0.smithi100.stdout:1023: done (5 left) 2024-04-07T16:19:51.964 INFO:tasks.rados.rados.0.smithi100.stdout:1029: copy_from oid 33 from oid 10 current snap is 95 2024-04-07T16:19:51.964 INFO:tasks.rados.rados.0.smithi100.stdout:1030: rollback oid 34 current snap is 95 2024-04-07T16:19:51.965 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 34 to 85 2024-04-07T16:19:51.965 INFO:tasks.rados.rados.0.smithi100.stdout:1031: copy_from oid 39 from oid 38 current snap is 95 2024-04-07T16:19:51.965 INFO:tasks.rados.rados.0.smithi100.stdout:1032: read oid 7 snap 87 2024-04-07T16:19:51.965 INFO:tasks.rados.rados.0.smithi100.stdout:1032: expect deleted 2024-04-07T16:19:51.965 INFO:tasks.rados.rados.0.smithi100.stdout:1033: copy_from oid 13 from oid 4 current snap is 95 2024-04-07T16:19:51.965 INFO:tasks.rados.rados.0.smithi100.stdout:1034: snap_create 2024-04-07T16:19:51.965 INFO:tasks.rados.rados.0.smithi100.stdout:1024: expect (ObjNum 206 snap 56 seq_num 206) 2024-04-07T16:19:52.008 INFO:tasks.rados.rados.0.smithi100.stdout:1022: finishing write tid 3 to smithi10066867-27 2024-04-07T16:19:52.008 INFO:tasks.rados.rados.0.smithi100.stdout:1022: oid 27 updating version 0 to 254 2024-04-07T16:19:52.008 INFO:tasks.rados.rados.0.smithi100.stdout:1022: oid 27 updating version 254 to 255 2024-04-07T16:19:52.008 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 27 v 255 (ObjNum 316 snap 95 seq_num 316) dirty exists 2024-04-07T16:19:52.008 INFO:tasks.rados.rados.0.smithi100.stdout:1022: left oid 27 (ObjNum 316 snap 95 seq_num 316) 2024-04-07T16:19:52.008 INFO:tasks.rados.rados.0.smithi100.stdout:1030: finishing rollback tid 0 to smithi10066867-34 2024-04-07T16:19:52.008 INFO:tasks.rados.rados.0.smithi100.stdout:1030: finishing rollback tid 1 to smithi10066867-34 2024-04-07T16:19:52.008 INFO:tasks.rados.rados.0.smithi100.stdout:1030: finishing rollback tid 2 to smithi10066867-34 2024-04-07T16:19:52.008 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 34 v 368 (ObjNum 140 snap 39 seq_num 140) dirty exists 2024-04-07T16:19:52.008 INFO:tasks.rados.rados.0.smithi100.stdout:1028: finishing write tid 1 to smithi10066867-5 2024-04-07T16:19:52.009 INFO:tasks.rados.rados.0.smithi100.stdout:1028: finishing write tid 2 to smithi10066867-5 2024-04-07T16:19:52.009 INFO:tasks.rados.rados.0.smithi100.stdout:1031: finishing copy_from racing read to smithi10066867-39 2024-04-07T16:19:52.009 INFO:tasks.rados.rados.0.smithi100.stdout:1031: finishing copy_from to smithi10066867-39 2024-04-07T16:19:52.009 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 39 v 190 (ObjNum 304 snap 88 seq_num 304) dirty exists 2024-04-07T16:19:52.009 INFO:tasks.rados.rados.0.smithi100.stdout:1028: finishing write tid 3 to smithi10066867-5 2024-04-07T16:19:52.009 INFO:tasks.rados.rados.0.smithi100.stdout:1028: oid 5 updating version 0 to 519 2024-04-07T16:19:52.009 INFO:tasks.rados.rados.0.smithi100.stdout:1028: oid 5 version 519 is already newer than 518 2024-04-07T16:19:52.009 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 5 v 519 (ObjNum 318 snap 95 seq_num 318) dirty exists 2024-04-07T16:19:52.009 INFO:tasks.rados.rados.0.smithi100.stdout:1028: left oid 5 (ObjNum 318 snap 95 seq_num 318) 2024-04-07T16:19:52.009 INFO:tasks.rados.rados.0.smithi100.stdout:1027: finishing write tid 1 to smithi10066867-20 2024-04-07T16:19:52.010 INFO:tasks.rados.rados.0.smithi100.stdout:1027: finishing write tid 2 to smithi10066867-20 2024-04-07T16:19:52.021 INFO:tasks.rados.rados.0.smithi100.stdout:1026: expect (ObjNum 302 snap 87 seq_num 302) 2024-04-07T16:19:52.080 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:19:51 smithi144 bash[60191]: debug 2024-04-07T16:19:51.836+0000 7f1c34a3a200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-07T16:19:52.080 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:19:51 smithi144 bash[60191]: debug 2024-04-07T16:19:51.956+0000 7f1c34a3a200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-07T16:19:52.080 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:19:51 smithi144 bash[60191]: [07/Apr/2024:16:19:51] ENGINE Bus STARTING 2024-04-07T16:19:52.355 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:19:52 smithi144 bash[60191]: CherryPy Checker: 2024-04-07T16:19:52.355 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:19:52 smithi144 bash[60191]: The Application mounted at '' has an empty config. 2024-04-07T16:19:52.355 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:19:52 smithi144 bash[60191]: [07/Apr/2024:16:19:52] ENGINE Serving on http://:::9283 2024-04-07T16:19:52.355 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:19:52 smithi144 bash[60191]: [07/Apr/2024:16:19:52] ENGINE Bus STARTED 2024-04-07T16:19:52.643 INFO:tasks.rados.rados.0.smithi100.stdout:1027: finishing write tid 3 to smithi10066867-20 2024-04-07T16:19:52.643 INFO:tasks.rados.rados.0.smithi100.stdout:1027: oid 20 updating version 0 to 506 2024-04-07T16:19:52.643 INFO:tasks.rados.rados.0.smithi100.stdout:1027: oid 20 version 506 is already newer than 505 2024-04-07T16:19:52.643 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 20 v 506 (ObjNum 317 snap 95 seq_num 317) dirty exists 2024-04-07T16:19:52.643 INFO:tasks.rados.rados.0.smithi100.stdout:1027: left oid 20 (ObjNum 317 snap 95 seq_num 317) 2024-04-07T16:19:52.643 INFO:tasks.rados.rados.0.smithi100.stdout:1029: finishing copy_from racing read to smithi10066867-33 2024-04-07T16:19:52.643 INFO:tasks.rados.rados.0.smithi100.stdout:1029: finishing copy_from to smithi10066867-33 2024-04-07T16:19:52.643 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 33 v 226 (ObjNum 296 snap 84 seq_num 296) dirty exists 2024-04-07T16:19:52.643 INFO:tasks.rados.rados.0.smithi100.stdout:1033: finishing copy_from racing read to smithi10066867-13 2024-04-07T16:19:52.643 INFO:tasks.rados.rados.0.smithi100.stdout:1033: finishing copy_from to smithi10066867-13 2024-04-07T16:19:52.644 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 13 v 316 (ObjNum 302 snap 87 seq_num 302) dirty exists 2024-04-07T16:19:52.976 INFO:tasks.rados.rados.0.smithi100.stdout:1022: done (10 left) 2024-04-07T16:19:52.977 INFO:tasks.rados.rados.0.smithi100.stdout:1024: done (9 left) 2024-04-07T16:19:52.977 INFO:tasks.rados.rados.0.smithi100.stdout:1026: done (8 left) 2024-04-07T16:19:52.977 INFO:tasks.rados.rados.0.smithi100.stdout:1027: done (7 left) 2024-04-07T16:19:52.977 INFO:tasks.rados.rados.0.smithi100.stdout:1028: done (6 left) 2024-04-07T16:19:52.977 INFO:tasks.rados.rados.0.smithi100.stdout:1029: done (5 left) 2024-04-07T16:19:52.977 INFO:tasks.rados.rados.0.smithi100.stdout:1030: done (4 left) 2024-04-07T16:19:52.977 INFO:tasks.rados.rados.0.smithi100.stdout:1031: done (3 left) 2024-04-07T16:19:52.977 INFO:tasks.rados.rados.0.smithi100.stdout:1032: done (2 left) 2024-04-07T16:19:52.977 INFO:tasks.rados.rados.0.smithi100.stdout:1033: done (1 left) 2024-04-07T16:19:52.977 INFO:tasks.rados.rados.0.smithi100.stdout:1034: done (0 left) 2024-04-07T16:19:52.977 INFO:tasks.rados.rados.0.smithi100.stdout:1035: read oid 48 snap -1 2024-04-07T16:19:52.977 INFO:tasks.rados.rados.0.smithi100.stdout:1035: expect deleted 2024-04-07T16:19:52.977 INFO:tasks.rados.rados.0.smithi100.stdout:1036: read oid 35 snap 87 2024-04-07T16:19:52.977 INFO:tasks.rados.rados.0.smithi100.stdout:1036: expect (ObjNum 271 snap 73 seq_num 271) 2024-04-07T16:19:52.977 INFO:tasks.rados.rados.0.smithi100.stdout:1037: rollback oid 21 current snap is 96 2024-04-07T16:19:52.978 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 21 to 69 2024-04-07T16:19:52.978 INFO:tasks.rados.rados.0.smithi100.stdout:1038: snap_remove snap 85 2024-04-07T16:19:53.131 INFO:teuthology.orchestra.run.smithi100.stdout:true 2024-04-07T16:19:53.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:52 smithi144 bash[23522]: cluster 2024-04-07T16:19:51.968612+0000 mon.a (mon.0) 1192 : cluster 0 osdmap e258: 8 total, 8 up, 8 in 2024-04-07T16:19:53.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:52 smithi144 bash[23522]: audit 2024-04-07T16:19:51.988779+0000 mon.b (mon.1) 331 : audit 0 from='mgr.? 172.21.15.144:0/684343795' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-04-07T16:19:53.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:52 smithi144 bash[23522]: cluster 2024-04-07T16:19:51.989628+0000 mon.a (mon.0) 1193 : cluster 0 Standby manager daemon x restarted 2024-04-07T16:19:53.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:52 smithi144 bash[23522]: cluster 2024-04-07T16:19:51.989937+0000 mon.a (mon.0) 1194 : cluster 0 Standby manager daemon x started 2024-04-07T16:19:53.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:52 smithi144 bash[23522]: audit 2024-04-07T16:19:51.990369+0000 mon.b (mon.1) 332 : audit 0 from='mgr.? 172.21.15.144:0/684343795' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-07T16:19:53.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:52 smithi144 bash[23522]: audit 2024-04-07T16:19:51.992956+0000 mon.b (mon.1) 333 : audit 0 from='mgr.? 172.21.15.144:0/684343795' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-04-07T16:19:53.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:52 smithi144 bash[23522]: audit 2024-04-07T16:19:51.993309+0000 mon.b (mon.1) 334 : audit 0 from='mgr.? 172.21.15.144:0/684343795' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-07T16:19:53.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:52 smithi144 bash[23522]: cluster 2024-04-07T16:19:52.019976+0000 mgr.y (mgr.25073) 39 : cluster 0 pgmap v59: 121 pgs: 121 active+clean; 208 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 20 MiB/s rd, 2.2 MiB/s wr, 13 op/s 2024-04-07T16:19:53.436 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:52 smithi100 bash[21346]: cluster 2024-04-07T16:19:51.968612+0000 mon.a (mon.0) 1192 : cluster 0 osdmap e258: 8 total, 8 up, 8 in 2024-04-07T16:19:53.436 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:52 smithi100 bash[21346]: audit 2024-04-07T16:19:51.988779+0000 mon.b (mon.1) 331 : audit 0 from='mgr.? 172.21.15.144:0/684343795' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-04-07T16:19:53.436 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:52 smithi100 bash[21346]: cluster 2024-04-07T16:19:51.989628+0000 mon.a (mon.0) 1193 : cluster 0 Standby manager daemon x restarted 2024-04-07T16:19:53.436 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:52 smithi100 bash[21346]: cluster 2024-04-07T16:19:51.989937+0000 mon.a (mon.0) 1194 : cluster 0 Standby manager daemon x started 2024-04-07T16:19:53.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:52 smithi100 bash[21346]: audit 2024-04-07T16:19:51.990369+0000 mon.b (mon.1) 332 : audit 0 from='mgr.? 172.21.15.144:0/684343795' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-07T16:19:53.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:52 smithi100 bash[21346]: audit 2024-04-07T16:19:51.992956+0000 mon.b (mon.1) 333 : audit 0 from='mgr.? 172.21.15.144:0/684343795' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-04-07T16:19:53.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:52 smithi100 bash[21346]: audit 2024-04-07T16:19:51.993309+0000 mon.b (mon.1) 334 : audit 0 from='mgr.? 172.21.15.144:0/684343795' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-07T16:19:53.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:52 smithi100 bash[21346]: cluster 2024-04-07T16:19:52.019976+0000 mgr.y (mgr.25073) 39 : cluster 0 pgmap v59: 121 pgs: 121 active+clean; 208 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 20 MiB/s rd, 2.2 MiB/s wr, 13 op/s 2024-04-07T16:19:53.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:52 smithi100 bash[31660]: cluster 2024-04-07T16:19:51.968612+0000 mon.a (mon.0) 1192 : cluster 0 osdmap e258: 8 total, 8 up, 8 in 2024-04-07T16:19:53.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:52 smithi100 bash[31660]: audit 2024-04-07T16:19:51.988779+0000 mon.b (mon.1) 331 : audit 0 from='mgr.? 172.21.15.144:0/684343795' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-04-07T16:19:53.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:52 smithi100 bash[31660]: cluster 2024-04-07T16:19:51.989628+0000 mon.a (mon.0) 1193 : cluster 0 Standby manager daemon x restarted 2024-04-07T16:19:53.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:52 smithi100 bash[31660]: cluster 2024-04-07T16:19:51.989937+0000 mon.a (mon.0) 1194 : cluster 0 Standby manager daemon x started 2024-04-07T16:19:53.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:52 smithi100 bash[31660]: audit 2024-04-07T16:19:51.990369+0000 mon.b (mon.1) 332 : audit 0 from='mgr.? 172.21.15.144:0/684343795' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-07T16:19:53.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:52 smithi100 bash[31660]: audit 2024-04-07T16:19:51.992956+0000 mon.b (mon.1) 333 : audit 0 from='mgr.? 172.21.15.144:0/684343795' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-04-07T16:19:53.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:52 smithi100 bash[31660]: audit 2024-04-07T16:19:51.993309+0000 mon.b (mon.1) 334 : audit 0 from='mgr.? 172.21.15.144:0/684343795' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-07T16:19:53.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:52 smithi100 bash[31660]: cluster 2024-04-07T16:19:52.019976+0000 mgr.y (mgr.25073) 39 : cluster 0 pgmap v59: 121 pgs: 121 active+clean; 208 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 20 MiB/s rd, 2.2 MiB/s wr, 13 op/s 2024-04-07T16:19:53.480 INFO:teuthology.orchestra.run.smithi100.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-07T16:19:53.480 INFO:teuthology.orchestra.run.smithi100.stdout:alertmanager.a smithi100 *:9093,9094 running (5m) 33s ago 6m 14.7M - 0.25.0 c8568f914cd2 5143075e7d26 2024-04-07T16:19:53.480 INFO:teuthology.orchestra.run.smithi100.stdout:grafana.a smithi144 *:3000 running (6m) 4s ago 6m 79.8M - 9.4.7 954c08fa6188 7180c0708acd 2024-04-07T16:19:53.480 INFO:teuthology.orchestra.run.smithi100.stdout:mds.a smithi100 running (7m) 33s ago 7m 14.2M - 18.2.2-1171-g267c8ff5 8a4a01e7186c 92593b0c28ae 2024-04-07T16:19:53.480 INFO:teuthology.orchestra.run.smithi100.stdout:mds.b smithi144 running (7m) 4s ago 7m 17.4M - 18.2.2-1171-g267c8ff5 8a4a01e7186c 3df2ee3e17a4 2024-04-07T16:19:53.480 INFO:teuthology.orchestra.run.smithi100.stdout:mgr.x smithi144 *:8443,9283,8765 running (10s) 4s ago 14m 137M - 19.0.0-2785-gadce92d0 c0ecd4226ac9 233745c292a6 2024-04-07T16:19:53.480 INFO:teuthology.orchestra.run.smithi100.stdout:mgr.y smithi100 *:8443,9283,8765 running (56s) 33s ago 17m 471M - 19.0.0-2785-gadce92d0 c0ecd4226ac9 254dc65cdaa7 2024-04-07T16:19:53.481 INFO:teuthology.orchestra.run.smithi100.stdout:mon.a smithi100 running (17m) 33s ago 17m 59.4M 2048M 18.2.2-1171-g267c8ff5 8a4a01e7186c ec8833f59ea0 2024-04-07T16:19:53.481 INFO:teuthology.orchestra.run.smithi100.stdout:mon.b smithi144 running (15m) 4s ago 15m 45.9M 2048M 18.2.2-1171-g267c8ff5 8a4a01e7186c 8dd687691b2e 2024-04-07T16:19:53.481 INFO:teuthology.orchestra.run.smithi100.stdout:mon.c smithi100 running (15m) 33s ago 15m 52.1M 2048M 18.2.2-1171-g267c8ff5 8a4a01e7186c a6de0151a3ba 2024-04-07T16:19:53.481 INFO:teuthology.orchestra.run.smithi100.stdout:node-exporter.a smithi100 *:9100 running (6m) 33s ago 6m 9295k - 1.5.0 0da6a335fe13 4bce264c1b26 2024-04-07T16:19:53.481 INFO:teuthology.orchestra.run.smithi100.stdout:node-exporter.b smithi144 *:9100 running (6m) 4s ago 6m 8511k - 1.5.0 0da6a335fe13 b6963413c560 2024-04-07T16:19:53.481 INFO:teuthology.orchestra.run.smithi100.stdout:osd.0 smithi100 running (14m) 33s ago 14m 182M 2524M 18.2.2-1171-g267c8ff5 8a4a01e7186c 188deb247979 2024-04-07T16:19:53.481 INFO:teuthology.orchestra.run.smithi100.stdout:osd.1 smithi100 running (13m) 33s ago 13m 148M 2524M 18.2.2-1171-g267c8ff5 8a4a01e7186c 7f57ca90894c 2024-04-07T16:19:53.481 INFO:teuthology.orchestra.run.smithi100.stdout:osd.2 smithi100 running (12m) 33s ago 12m 174M 2524M 18.2.2-1171-g267c8ff5 8a4a01e7186c f3912ddcc8cc 2024-04-07T16:19:53.481 INFO:teuthology.orchestra.run.smithi100.stdout:osd.3 smithi100 running (11m) 33s ago 11m 114M 2524M 18.2.2-1171-g267c8ff5 8a4a01e7186c b41004090c63 2024-04-07T16:19:53.481 INFO:teuthology.orchestra.run.smithi100.stdout:osd.4 smithi144 running (10m) 4s ago 11m 189M 2524M 18.2.2-1171-g267c8ff5 8a4a01e7186c 8c73b6b4dc13 2024-04-07T16:19:53.481 INFO:teuthology.orchestra.run.smithi100.stdout:osd.5 smithi144 running (10m) 4s ago 10m 166M 2524M 18.2.2-1171-g267c8ff5 8a4a01e7186c 3bfa0cf6f6cb 2024-04-07T16:19:53.481 INFO:teuthology.orchestra.run.smithi100.stdout:osd.6 smithi144 running (9m) 4s ago 9m 75.1M 2524M 18.2.2-1171-g267c8ff5 8a4a01e7186c 32b7f95d77f0 2024-04-07T16:19:53.481 INFO:teuthology.orchestra.run.smithi100.stdout:osd.7 smithi144 running (8m) 4s ago 8m 110M 2524M 18.2.2-1171-g267c8ff5 8a4a01e7186c 45f7709e576a 2024-04-07T16:19:53.481 INFO:teuthology.orchestra.run.smithi100.stdout:prometheus.a smithi144 *:9095 running (16s) 4s ago 7m 34.8M - 2.43.0 a07b618ecd1d 2d0a8131f303 2024-04-07T16:19:53.855 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:19:53 smithi144 bash[59794]: ts=2024-04-07T16:19:53.515Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=nodes name=CephNodeDiskspaceWarning index=4 msg="Evaluating rule failed" rule="alert: CephNodeDiskspaceWarning\nexpr: predict_linear(node_filesystem_free_bytes{device=~\"/.*\"}[2d], 3600 * 24 * 5)\n * on (instance) group_left (nodename) node_uname_info < 0\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.8.4\n severity: warning\n type: ceph_default\nannotations:\n description: Mountpoint {{ $labels.mountpoint }} on {{ $labels.nodename }} will\n be full in less than 5 days based on the 48 hour trailing fill rate.\n summary: Host filesystem free space is getting low\n" err="found duplicate series for the match group {instance=\"smithi100\"} on the right hand-side of the operation: [{__name__=\"node_uname_info\", domainname=\"(none)\", instance=\"smithi100\", job=\"node\", machine=\"x86_64\", nodename=\"smithi100\", release=\"5.15.0-101-generic\", sysname=\"Linux\", version=\"#111-Ubuntu SMP Tue Mar 5 20:16:58 UTC 2024\"}, {__name__=\"node_uname_info\", cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", domainname=\"(none)\", instance=\"smithi100\", job=\"node\", machine=\"x86_64\", nodename=\"smithi100\", release=\"5.15.0-101-generic\", sysname=\"Linux\", version=\"#111-Ubuntu SMP Tue Mar 5 20:16:58 UTC 2024\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:19:53.897 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-04-07T16:19:53.897 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": { 2024-04-07T16:19:53.897 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph version 18.2.2-1171-g267c8ff5 (267c8ff5b1ff5815581b3b2381d19360e081b37b) reef (stable)": 3 2024-04-07T16:19:53.897 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-07T16:19:53.897 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": { 2024-04-07T16:19:53.897 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph version 19.0.0-2785-gadce92d0 (adce92d056533f22dc1ee37e3141d098b046ec1b) squid (dev)": 2 2024-04-07T16:19:53.897 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-07T16:19:53.897 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": { 2024-04-07T16:19:53.897 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph version 18.2.2-1171-g267c8ff5 (267c8ff5b1ff5815581b3b2381d19360e081b37b) reef (stable)": 8 2024-04-07T16:19:53.898 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-07T16:19:53.898 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": { 2024-04-07T16:19:53.898 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph version 18.2.2-1171-g267c8ff5 (267c8ff5b1ff5815581b3b2381d19360e081b37b) reef (stable)": 2 2024-04-07T16:19:53.898 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-07T16:19:53.898 INFO:teuthology.orchestra.run.smithi100.stdout: "overall": { 2024-04-07T16:19:53.898 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph version 18.2.2-1171-g267c8ff5 (267c8ff5b1ff5815581b3b2381d19360e081b37b) reef (stable)": 13, 2024-04-07T16:19:53.898 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph version 19.0.0-2785-gadce92d0 (adce92d056533f22dc1ee37e3141d098b046ec1b) squid (dev)": 2 2024-04-07T16:19:53.898 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-04-07T16:19:53.898 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-04-07T16:19:53.974 INFO:tasks.rados.rados.0.smithi100.stdout:1038: done (3 left) 2024-04-07T16:19:53.974 INFO:tasks.rados.rados.0.smithi100.stdout:1039: read oid 32 snap -1 2024-04-07T16:19:53.974 INFO:tasks.rados.rados.0.smithi100.stdout:1039: expect (ObjNum 278 snap 80 seq_num 278) 2024-04-07T16:19:53.975 INFO:tasks.rados.rados.0.smithi100.stdout:1040: read oid 25 snap -1 2024-04-07T16:19:53.975 INFO:tasks.rados.rados.0.smithi100.stdout:1040: expect deleted 2024-04-07T16:19:53.975 INFO:tasks.rados.rados.0.smithi100.stdout:1041: read oid 23 snap -1 2024-04-07T16:19:53.975 INFO:tasks.rados.rados.0.smithi100.stdout:1041: expect (ObjNum 262 snap 70 seq_num 262) 2024-04-07T16:19:53.975 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 26 current snap is 96 2024-04-07T16:19:53.975 INFO:tasks.rados.rados.0.smithi100.stdout:1042: seq_num 319 ranges {0=548864} 2024-04-07T16:19:53.975 INFO:tasks.rados.rados.0.smithi100.stdout:1042: writing smithi10066867-26 from 0 to 548864 tid 1 2024-04-07T16:19:53.975 INFO:tasks.rados.rados.0.smithi100.stdout:1043: copy_from oid 34 from oid 45 current snap is 96 2024-04-07T16:19:53.975 INFO:tasks.rados.rados.0.smithi100.stdout:1044: read oid 49 snap -1 2024-04-07T16:19:53.975 INFO:tasks.rados.rados.0.smithi100.stdout:1044: expect (ObjNum 307 snap 88 seq_num 307) 2024-04-07T16:19:53.975 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 5 current snap is 96 2024-04-07T16:19:53.975 INFO:tasks.rados.rados.0.smithi100.stdout:1045: seq_num 320 ranges {532480=565248} 2024-04-07T16:19:53.980 INFO:tasks.rados.rados.0.smithi100.stdout:1045: writing smithi10066867-5 from 532480 to 1097728 tid 1 2024-04-07T16:19:53.980 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 12 current snap is 96 2024-04-07T16:19:53.980 INFO:tasks.rados.rados.0.smithi100.stdout:1046: seq_num 321 ranges {3457024=581632} 2024-04-07T16:19:54.005 INFO:tasks.rados.rados.0.smithi100.stdout:1046: writing smithi10066867-12 from 3457024 to 4038656 tid 1 2024-04-07T16:19:54.005 INFO:tasks.rados.rados.0.smithi100.stdout:1047: rollback oid 8 current snap is 96 2024-04-07T16:19:54.005 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 8 to 75 2024-04-07T16:19:54.011 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 39 current snap is 96 2024-04-07T16:19:54.011 INFO:tasks.rados.rados.0.smithi100.stdout:1048: seq_num 322 ranges {294912=598016} 2024-04-07T16:19:54.011 INFO:tasks.rados.rados.0.smithi100.stdout:1048: writing smithi10066867-39 from 294912 to 892928 tid 1 2024-04-07T16:19:54.012 INFO:tasks.rados.rados.0.smithi100.stdout:1049: copy_from oid 9 from oid 40 current snap is 96 2024-04-07T16:19:54.012 INFO:tasks.rados.rados.0.smithi100.stdout:1037: finishing rollback tid 0 to smithi10066867-21 2024-04-07T16:19:54.012 INFO:tasks.rados.rados.0.smithi100.stdout:1035: done (13 left) 2024-04-07T16:19:54.012 INFO:tasks.rados.rados.0.smithi100.stdout:1050: copy_from oid 38 from oid 19 current snap is 96 2024-04-07T16:19:54.012 INFO:tasks.rados.rados.0.smithi100.stdout:1037: finishing rollback tid 1 to smithi10066867-21 2024-04-07T16:19:54.012 INFO:tasks.rados.rados.0.smithi100.stdout:1037: finishing rollback tid 2 to smithi10066867-21 2024-04-07T16:19:54.012 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 21 v 202 (ObjNum 222 snap 59 seq_num 222) dirty exists 2024-04-07T16:19:54.012 INFO:tasks.rados.rados.0.smithi100.stdout:1037: done (13 left) 2024-04-07T16:19:54.012 INFO:tasks.rados.rados.0.smithi100.stdout:1051: read oid 33 snap -1 2024-04-07T16:19:54.012 INFO:tasks.rados.rados.0.smithi100.stdout:1051: expect (ObjNum 296 snap 84 seq_num 296) 2024-04-07T16:19:54.012 INFO:tasks.rados.rados.0.smithi100.stdout:1052: setattr oid 16 current snap is 96 2024-04-07T16:19:54.012 INFO:tasks.rados.rados.0.smithi100.stdout:1036: expect (ObjNum 271 snap 73 seq_num 271) 2024-04-07T16:19:54.230 INFO:tasks.rados.rados.0.smithi100.stdout:1044: expect (ObjNum 307 snap 88 seq_num 307) 2024-04-07T16:19:54.316 INFO:tasks.rados.rados.0.smithi100.stdout:1045: finishing write tid 1 to smithi10066867-5 2024-04-07T16:19:54.317 INFO:tasks.rados.rados.0.smithi100.stdout:1045: finishing write tid 2 to smithi10066867-5 2024-04-07T16:19:54.317 INFO:tasks.rados.rados.0.smithi100.stdout:1045: finishing write tid 3 to smithi10066867-5 2024-04-07T16:19:54.317 INFO:tasks.rados.rados.0.smithi100.stdout:1045: oid 5 updating version 0 to 524 2024-04-07T16:19:54.317 INFO:tasks.rados.rados.0.smithi100.stdout:1045: oid 5 updating version 524 to 525 2024-04-07T16:19:54.317 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 5 v 525 (ObjNum 320 snap 96 seq_num 320) dirty exists 2024-04-07T16:19:54.317 INFO:tasks.rados.rados.0.smithi100.stdout:1045: left oid 5 (ObjNum 320 snap 96 seq_num 320) 2024-04-07T16:19:54.317 INFO:tasks.rados.rados.0.smithi100.stdout:1042: finishing write tid 1 to smithi10066867-26 2024-04-07T16:19:54.317 INFO:tasks.rados.rados.0.smithi100.stdout:1042: finishing write tid 2 to smithi10066867-26 2024-04-07T16:19:54.317 INFO:tasks.rados.rados.0.smithi100.stdout:1042: finishing write tid 3 to smithi10066867-26 2024-04-07T16:19:54.317 INFO:tasks.rados.rados.0.smithi100.stdout:1042: oid 26 updating version 0 to 507 2024-04-07T16:19:54.317 INFO:tasks.rados.rados.0.smithi100.stdout:1042: oid 26 updating version 507 to 508 2024-04-07T16:19:54.317 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 26 v 508 (ObjNum 319 snap 96 seq_num 319) dirty exists 2024-04-07T16:19:54.317 INFO:tasks.rados.rados.0.smithi100.stdout:1042: left oid 26 (ObjNum 319 snap 96 seq_num 319) 2024-04-07T16:19:54.318 INFO:tasks.rados.rados.0.smithi100.stdout:1046: finishing write tid 1 to smithi10066867-12 2024-04-07T16:19:54.318 INFO:tasks.rados.rados.0.smithi100.stdout:1046: finishing write tid 2 to smithi10066867-12 2024-04-07T16:19:54.318 INFO:tasks.rados.rados.0.smithi100.stdout:1048: finishing write tid 1 to smithi10066867-39 2024-04-07T16:19:54.318 INFO:tasks.rados.rados.0.smithi100.stdout:1048: finishing write tid 2 to smithi10066867-39 2024-04-07T16:19:54.318 INFO:tasks.rados.rados.0.smithi100.stdout:1046: finishing write tid 3 to smithi10066867-12 2024-04-07T16:19:54.318 INFO:tasks.rados.rados.0.smithi100.stdout:1046: oid 12 updating version 0 to 532 2024-04-07T16:19:54.318 INFO:tasks.rados.rados.0.smithi100.stdout:1046: oid 12 version 532 is already newer than 531 2024-04-07T16:19:54.318 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 12 v 532 (ObjNum 321 snap 96 seq_num 321) dirty exists 2024-04-07T16:19:54.318 INFO:tasks.rados.rados.0.smithi100.stdout:1046: left oid 12 (ObjNum 321 snap 96 seq_num 321) 2024-04-07T16:19:54.318 INFO:tasks.rados.rados.0.smithi100.stdout:1043: finishing copy_from racing read to smithi10066867-34 2024-04-07T16:19:54.318 INFO:tasks.rados.rados.0.smithi100.stdout:1047: finishing rollback tid 0 to smithi10066867-8 2024-04-07T16:19:54.318 INFO:tasks.rados.rados.0.smithi100.stdout:1047: finishing rollback tid 1 to smithi10066867-8 2024-04-07T16:19:54.318 INFO:tasks.rados.rados.0.smithi100.stdout:1041: expect (ObjNum 262 snap 70 seq_num 262) 2024-04-07T16:19:54.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:53 smithi144 bash[23522]: cluster 2024-04-07T16:19:52.989361+0000 mon.a (mon.0) 1195 : cluster 0 osdmap e259: 8 total, 8 up, 8 in 2024-04-07T16:19:54.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:53 smithi144 bash[23522]: cluster 2024-04-07T16:19:52.991650+0000 mon.a (mon.0) 1196 : cluster 0 mgrmap e35: y(active, since 39s), standbys: x 2024-04-07T16:19:54.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:53 smithi144 bash[23522]: audit 2024-04-07T16:19:53.133116+0000 mgr.y (mgr.25073) 40 : audit 0 from='client.25151 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:19:54.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:53 smithi144 bash[23522]: audit 2024-04-07T16:19:53.913317+0000 mon.a (mon.0) 1197 : audit 0 from='client.? 172.21.15.100:0/2728307779' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:19:54.432 INFO:tasks.rados.rados.0.smithi100.stdout:1048: finishing write tid 3 to smithi10066867-39 2024-04-07T16:19:54.433 INFO:tasks.rados.rados.0.smithi100.stdout:1048: oid 39 updating version 0 to 473 2024-04-07T16:19:54.433 INFO:tasks.rados.rados.0.smithi100.stdout:1048: oid 39 updating version 473 to 474 2024-04-07T16:19:54.433 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 39 v 474 (ObjNum 322 snap 96 seq_num 322) dirty exists 2024-04-07T16:19:54.433 INFO:tasks.rados.rados.0.smithi100.stdout:1048: left oid 39 (ObjNum 322 snap 96 seq_num 322) 2024-04-07T16:19:54.433 INFO:tasks.rados.rados.0.smithi100.stdout:1043: finishing copy_from to smithi10066867-34 2024-04-07T16:19:54.433 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 34 v 514 (ObjNum 312 snap 94 seq_num 312) dirty exists 2024-04-07T16:19:54.433 INFO:tasks.rados.rados.0.smithi100.stdout:1047: finishing rollback tid 2 to smithi10066867-8 2024-04-07T16:19:54.433 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 8 v 515 (ObjNum 233 snap 60 seq_num 233) dirty exists 2024-04-07T16:19:54.433 INFO:tasks.rados.rados.0.smithi100.stdout:1039: expect (ObjNum 278 snap 80 seq_num 278) 2024-04-07T16:19:54.436 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:54 smithi100 bash[21346]: cluster 2024-04-07T16:19:52.989361+0000 mon.a (mon.0) 1195 : cluster 0 osdmap e259: 8 total, 8 up, 8 in 2024-04-07T16:19:54.436 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:54 smithi100 bash[21346]: cluster 2024-04-07T16:19:52.991650+0000 mon.a (mon.0) 1196 : cluster 0 mgrmap e35: y(active, since 39s), standbys: x 2024-04-07T16:19:54.436 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:54 smithi100 bash[21346]: audit 2024-04-07T16:19:53.133116+0000 mgr.y (mgr.25073) 40 : audit 0 from='client.25151 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:19:54.436 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:54 smithi100 bash[21346]: audit 2024-04-07T16:19:53.913317+0000 mon.a (mon.0) 1197 : audit 0 from='client.? 172.21.15.100:0/2728307779' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:19:54.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:54 smithi100 bash[31660]: cluster 2024-04-07T16:19:52.989361+0000 mon.a (mon.0) 1195 : cluster 0 osdmap e259: 8 total, 8 up, 8 in 2024-04-07T16:19:54.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:54 smithi100 bash[31660]: cluster 2024-04-07T16:19:52.991650+0000 mon.a (mon.0) 1196 : cluster 0 mgrmap e35: y(active, since 39s), standbys: x 2024-04-07T16:19:54.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:54 smithi100 bash[31660]: audit 2024-04-07T16:19:53.133116+0000 mgr.y (mgr.25073) 40 : audit 0 from='client.25151 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:19:54.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:54 smithi100 bash[31660]: audit 2024-04-07T16:19:53.913317+0000 mon.a (mon.0) 1197 : audit 0 from='client.? 172.21.15.100:0/2728307779' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:19:54.609 INFO:tasks.rados.rados.0.smithi100.stdout:1050: finishing copy_from racing read to smithi10066867-38 2024-04-07T16:19:54.609 INFO:tasks.rados.rados.0.smithi100.stdout:1050: finishing copy_from to smithi10066867-38 2024-04-07T16:19:54.609 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 38 v 235 (ObjNum 239 snap 65 seq_num 239) dirty exists 2024-04-07T16:19:54.609 INFO:tasks.rados.rados.0.smithi100.stdout:1051: expect (ObjNum 296 snap 84 seq_num 296) 2024-04-07T16:19:54.811 INFO:tasks.rados.rados.0.smithi100.stdout:1049: finishing copy_from racing read to smithi10066867-9 2024-04-07T16:19:54.811 INFO:tasks.rados.rados.0.smithi100.stdout:1049: finishing copy_from to smithi10066867-9 2024-04-07T16:19:54.812 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 9 v 535 (ObjNum 303 snap 87 seq_num 303) dirty exists 2024-04-07T16:19:54.812 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 16 v 537 (ObjNum 161 snap 0 seq_num 1) dirty exists 2024-04-07T16:19:54.812 INFO:tasks.rados.rados.0.smithi100.stdout:1036: done (14 left) 2024-04-07T16:19:54.812 INFO:tasks.rados.rados.0.smithi100.stdout:1039: done (13 left) 2024-04-07T16:19:54.812 INFO:tasks.rados.rados.0.smithi100.stdout:1040: done (12 left) 2024-04-07T16:19:54.812 INFO:tasks.rados.rados.0.smithi100.stdout:1041: done (11 left) 2024-04-07T16:19:54.812 INFO:tasks.rados.rados.0.smithi100.stdout:1042: done (10 left) 2024-04-07T16:19:54.812 INFO:tasks.rados.rados.0.smithi100.stdout:1043: done (9 left) 2024-04-07T16:19:54.812 INFO:tasks.rados.rados.0.smithi100.stdout:1044: done (8 left) 2024-04-07T16:19:54.812 INFO:tasks.rados.rados.0.smithi100.stdout:1045: done (7 left) 2024-04-07T16:19:54.812 INFO:tasks.rados.rados.0.smithi100.stdout:1046: done (6 left) 2024-04-07T16:19:54.812 INFO:tasks.rados.rados.0.smithi100.stdout:1047: done (5 left) 2024-04-07T16:19:54.812 INFO:tasks.rados.rados.0.smithi100.stdout:1048: done (4 left) 2024-04-07T16:19:54.812 INFO:tasks.rados.rados.0.smithi100.stdout:1049: done (3 left) 2024-04-07T16:19:54.812 INFO:tasks.rados.rados.0.smithi100.stdout:1050: done (2 left) 2024-04-07T16:19:54.813 INFO:tasks.rados.rados.0.smithi100.stdout:1051: done (1 left) 2024-04-07T16:19:54.813 INFO:tasks.rados.rados.0.smithi100.stdout:1052: done (0 left) 2024-04-07T16:19:54.813 INFO:tasks.rados.rados.0.smithi100.stdout:1053: rollback oid 44 current snap is 96 2024-04-07T16:19:54.813 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 44 to 75 2024-04-07T16:19:54.813 INFO:tasks.rados.rados.0.smithi100.stdout:1054: rollback oid 4 current snap is 96 2024-04-07T16:19:54.813 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 4 to 92 2024-04-07T16:19:54.813 INFO:tasks.rados.rados.0.smithi100.stdout:1055: read oid 46 snap -1 2024-04-07T16:19:54.813 INFO:tasks.rados.rados.0.smithi100.stdout:1055: expect (ObjNum 272 snap 73 seq_num 272) 2024-04-07T16:19:54.813 INFO:tasks.rados.rados.0.smithi100.stdout:1056: rollback oid 3 current snap is 96 2024-04-07T16:19:54.813 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 3 to 92 2024-04-07T16:19:54.813 INFO:tasks.rados.rados.0.smithi100.stdout:1057: delete oid 17 current snap is 96 2024-04-07T16:19:54.818 INFO:tasks.rados.rados.0.smithi100.stdout:1053: finishing rollback tid 0 to smithi10066867-44 2024-04-07T16:19:54.819 INFO:tasks.rados.rados.0.smithi100.stdout:1053: finishing rollback tid 1 to smithi10066867-44 2024-04-07T16:19:54.820 INFO:tasks.rados.rados.0.smithi100.stdout:1054: finishing rollback tid 0 to smithi10066867-4 2024-04-07T16:19:54.820 INFO:tasks.rados.rados.0.smithi100.stdout:1057: done (4 left) 2024-04-07T16:19:54.820 INFO:tasks.rados.rados.0.smithi100.stdout:1058: copy_from oid 17 from oid 40 current snap is 96 2024-04-07T16:19:54.820 INFO:tasks.rados.rados.0.smithi100.stdout:1059: setattr oid 10 current snap is 96 2024-04-07T16:19:54.820 INFO:tasks.rados.rados.0.smithi100.stdout:1053: finishing rollback tid 2 to smithi10066867-44 2024-04-07T16:19:54.820 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 44 v 538 (ObjNum 165 snap 44 seq_num 165) dirty exists 2024-04-07T16:19:54.820 INFO:tasks.rados.rados.0.smithi100.stdout:1056: finishing rollback tid 0 to smithi10066867-3 2024-04-07T16:19:54.820 INFO:tasks.rados.rados.0.smithi100.stdout:1056: finishing rollback tid 1 to smithi10066867-3 2024-04-07T16:19:54.820 INFO:tasks.rados.rados.0.smithi100.stdout:1056: finishing rollback tid 2 to smithi10066867-3 2024-04-07T16:19:54.820 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 3 v 238 (ObjNum 298 snap 85 seq_num 298) dirty exists 2024-04-07T16:19:54.821 INFO:tasks.rados.rados.0.smithi100.stdout:1054: finishing rollback tid 1 to smithi10066867-4 2024-04-07T16:19:54.824 INFO:tasks.rados.rados.0.smithi100.stdout:1053: done (5 left) 2024-04-07T16:19:54.824 INFO:tasks.rados.rados.0.smithi100.stdout:1056: done (4 left) 2024-04-07T16:19:54.824 INFO:tasks.rados.rados.0.smithi100.stdout:1060: snap_create 2024-04-07T16:19:54.824 INFO:tasks.rados.rados.0.smithi100.stdout:1054: finishing rollback tid 2 to smithi10066867-4 2024-04-07T16:19:54.824 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 4 v 636 (ObjNum 171 snap 44 seq_num 171) dirty exists 2024-04-07T16:19:54.828 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 10 v 229 (ObjNum 296 snap 84 seq_num 296) dirty exists 2024-04-07T16:19:54.841 INFO:tasks.rados.rados.0.smithi100.stdout:1055: expect (ObjNum 272 snap 73 seq_num 272) 2024-04-07T16:19:54.949 INFO:tasks.rados.rados.0.smithi100.stdout:1058: finishing copy_from racing read to smithi10066867-17 2024-04-07T16:19:54.949 INFO:tasks.rados.rados.0.smithi100.stdout:1058: finishing copy_from to smithi10066867-17 2024-04-07T16:19:54.949 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 17 v 535 (ObjNum 303 snap 87 seq_num 303) dirty exists 2024-04-07T16:19:55.000 INFO:tasks.rados.rados.0.smithi100.stdout:1054: done (4 left) 2024-04-07T16:19:55.000 INFO:tasks.rados.rados.0.smithi100.stdout:1055: done (3 left) 2024-04-07T16:19:55.000 INFO:tasks.rados.rados.0.smithi100.stdout:1058: done (2 left) 2024-04-07T16:19:55.000 INFO:tasks.rados.rados.0.smithi100.stdout:1059: done (1 left) 2024-04-07T16:19:55.000 INFO:tasks.rados.rados.0.smithi100.stdout:1060: done (0 left) 2024-04-07T16:19:55.000 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 6 current snap is 97 2024-04-07T16:19:55.000 INFO:tasks.rados.rados.0.smithi100.stdout:1061: seq_num 324 ranges {4210688=630784} 2024-04-07T16:19:55.070 INFO:tasks.rados.rados.0.smithi100.stdout:1061: writing smithi10066867-6 from 4210688 to 4841472 tid 1 2024-04-07T16:19:55.070 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 44 current snap is 97 2024-04-07T16:19:55.070 INFO:tasks.rados.rados.0.smithi100.stdout:1062: seq_num 325 ranges {958464=647168} 2024-04-07T16:19:55.084 INFO:tasks.rados.rados.0.smithi100.stdout:1062: writing smithi10066867-44 from 958464 to 1605632 tid 1 2024-04-07T16:19:55.084 INFO:tasks.rados.rados.0.smithi100.stdout:1063: read oid 50 snap -1 2024-04-07T16:19:55.084 INFO:tasks.rados.rados.0.smithi100.stdout:1063: expect (ObjNum 262 snap 70 seq_num 262) 2024-04-07T16:19:55.084 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 9 current snap is 97 2024-04-07T16:19:55.084 INFO:tasks.rados.rados.0.smithi100.stdout:1064: seq_num 326 ranges {6135808=663552} 2024-04-07T16:19:55.128 INFO:tasks.rados.rados.0.smithi100.stdout:1064: writing smithi10066867-9 from 6135808 to 6799360 tid 1 2024-04-07T16:19:55.128 INFO:tasks.rados.rados.0.smithi100.stdout:1065: read oid 22 snap -1 2024-04-07T16:19:55.128 INFO:tasks.rados.rados.0.smithi100.stdout:1065: expect (ObjNum 294 snap 84 seq_num 294) 2024-04-07T16:19:55.128 INFO:tasks.rados.rados.0.smithi100.stdout:1066: setattr oid 3 current snap is 97 2024-04-07T16:19:55.128 INFO:tasks.rados.rados.0.smithi100.stdout:1061: finishing write tid 1 to smithi10066867-6 2024-04-07T16:19:55.128 INFO:tasks.rados.rados.0.smithi100.stdout:1061: finishing write tid 2 to smithi10066867-6 2024-04-07T16:19:55.128 INFO:tasks.rados.rados.0.smithi100.stdout:1062: finishing write tid 1 to smithi10066867-44 2024-04-07T16:19:55.128 INFO:tasks.rados.rados.0.smithi100.stdout:1062: finishing write tid 2 to smithi10066867-44 2024-04-07T16:19:55.128 INFO:tasks.rados.rados.0.smithi100.stdout:1061: finishing write tid 3 to smithi10066867-6 2024-04-07T16:19:55.128 INFO:tasks.rados.rados.0.smithi100.stdout:1061: oid 6 updating version 0 to 539 2024-04-07T16:19:55.128 INFO:tasks.rados.rados.0.smithi100.stdout:1061: oid 6 updating version 539 to 540 2024-04-07T16:19:55.129 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 6 v 540 (ObjNum 324 snap 97 seq_num 324) dirty exists 2024-04-07T16:19:55.129 INFO:tasks.rados.rados.0.smithi100.stdout:1061: left oid 6 (ObjNum 324 snap 97 seq_num 324) 2024-04-07T16:19:55.129 INFO:tasks.rados.rados.0.smithi100.stdout:1062: finishing write tid 3 to smithi10066867-44 2024-04-07T16:19:55.129 INFO:tasks.rados.rados.0.smithi100.stdout:1062: oid 44 updating version 0 to 541 2024-04-07T16:19:55.129 INFO:tasks.rados.rados.0.smithi100.stdout:1062: oid 44 updating version 541 to 542 2024-04-07T16:19:55.129 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 44 v 542 (ObjNum 325 snap 97 seq_num 325) dirty exists 2024-04-07T16:19:55.129 INFO:tasks.rados.rados.0.smithi100.stdout:1062: left oid 44 (ObjNum 325 snap 97 seq_num 325) 2024-04-07T16:19:55.129 INFO:tasks.rados.rados.0.smithi100.stdout:1061: done (5 left) 2024-04-07T16:19:55.130 INFO:tasks.rados.rados.0.smithi100.stdout:1062: done (4 left) 2024-04-07T16:19:55.130 INFO:tasks.rados.rados.0.smithi100.stdout:1067: read oid 35 snap 89 2024-04-07T16:19:55.130 INFO:tasks.rados.rados.0.smithi100.stdout:1067: expect (ObjNum 271 snap 73 seq_num 271) 2024-04-07T16:19:55.130 INFO:tasks.rados.rados.0.smithi100.stdout:1068: snap_create 2024-04-07T16:19:55.138 INFO:tasks.rados.rados.0.smithi100.stdout:1063: expect (ObjNum 262 snap 70 seq_num 262) 2024-04-07T16:19:55.262 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 3 v 239 (ObjNum 298 snap 85 seq_num 298) dirty exists 2024-04-07T16:19:55.262 INFO:tasks.rados.rados.0.smithi100.stdout:1064: finishing write tid 1 to smithi10066867-9 2024-04-07T16:19:55.262 INFO:tasks.rados.rados.0.smithi100.stdout:1064: finishing write tid 2 to smithi10066867-9 2024-04-07T16:19:55.262 INFO:tasks.rados.rados.0.smithi100.stdout:1064: finishing write tid 3 to smithi10066867-9 2024-04-07T16:19:55.262 INFO:tasks.rados.rados.0.smithi100.stdout:1064: oid 9 updating version 0 to 537 2024-04-07T16:19:55.262 INFO:tasks.rados.rados.0.smithi100.stdout:1064: oid 9 version 537 is already newer than 536 2024-04-07T16:19:55.262 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 9 v 537 (ObjNum 326 snap 97 seq_num 326) dirty exists 2024-04-07T16:19:55.262 INFO:tasks.rados.rados.0.smithi100.stdout:1064: left oid 9 (ObjNum 326 snap 97 seq_num 326) 2024-04-07T16:19:55.262 INFO:tasks.rados.rados.0.smithi100.stdout:1065: expect (ObjNum 294 snap 84 seq_num 294) 2024-04-07T16:19:55.342 INFO:tasks.rados.rados.0.smithi100.stdout:1067: expect (ObjNum 271 snap 73 seq_num 271) 2024-04-07T16:19:55.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:55 smithi144 bash[23522]: audit 2024-04-07T16:19:53.482556+0000 mgr.y (mgr.25073) 41 : audit 0 from='client.25157 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:19:55.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:55 smithi144 bash[23522]: cluster 2024-04-07T16:19:53.980913+0000 mon.a (mon.0) 1198 : cluster 0 osdmap e260: 8 total, 8 up, 8 in 2024-04-07T16:19:55.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:55 smithi144 bash[23522]: cluster 2024-04-07T16:19:54.020481+0000 mgr.y (mgr.25073) 42 : cluster 0 pgmap v62: 121 pgs: 121 active+clean; 208 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 20 MiB/s rd, 2.2 MiB/s wr, 13 op/s 2024-04-07T16:19:55.436 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:55 smithi100 bash[21346]: audit 2024-04-07T16:19:53.482556+0000 mgr.y (mgr.25073) 41 : audit 0 from='client.25157 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:19:55.436 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:55 smithi100 bash[21346]: cluster 2024-04-07T16:19:53.980913+0000 mon.a (mon.0) 1198 : cluster 0 osdmap e260: 8 total, 8 up, 8 in 2024-04-07T16:19:55.436 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:55 smithi100 bash[21346]: cluster 2024-04-07T16:19:54.020481+0000 mgr.y (mgr.25073) 42 : cluster 0 pgmap v62: 121 pgs: 121 active+clean; 208 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 20 MiB/s rd, 2.2 MiB/s wr, 13 op/s 2024-04-07T16:19:55.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:55 smithi100 bash[31660]: audit 2024-04-07T16:19:53.482556+0000 mgr.y (mgr.25073) 41 : audit 0 from='client.25157 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:19:55.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:55 smithi100 bash[31660]: cluster 2024-04-07T16:19:53.980913+0000 mon.a (mon.0) 1198 : cluster 0 osdmap e260: 8 total, 8 up, 8 in 2024-04-07T16:19:55.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:55 smithi100 bash[31660]: cluster 2024-04-07T16:19:54.020481+0000 mgr.y (mgr.25073) 42 : cluster 0 pgmap v62: 121 pgs: 121 active+clean; 208 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 20 MiB/s rd, 2.2 MiB/s wr, 13 op/s 2024-04-07T16:19:56.011 INFO:tasks.rados.rados.0.smithi100.stdout:1063: done (5 left) 2024-04-07T16:19:56.011 INFO:tasks.rados.rados.0.smithi100.stdout:1064: done (4 left) 2024-04-07T16:19:56.011 INFO:tasks.rados.rados.0.smithi100.stdout:1065: done (3 left) 2024-04-07T16:19:56.011 INFO:tasks.rados.rados.0.smithi100.stdout:1066: done (2 left) 2024-04-07T16:19:56.011 INFO:tasks.rados.rados.0.smithi100.stdout:1067: done (1 left) 2024-04-07T16:19:56.011 INFO:tasks.rados.rados.0.smithi100.stdout:1068: done (0 left) 2024-04-07T16:19:56.011 INFO:tasks.rados.rados.0.smithi100.stdout:1069: read oid 50 snap -1 2024-04-07T16:19:56.011 INFO:tasks.rados.rados.0.smithi100.stdout:1069: expect (ObjNum 262 snap 70 seq_num 262) 2024-04-07T16:19:56.011 INFO:tasks.rados.rados.0.smithi100.stdout:1070: read oid 41 snap -1 2024-04-07T16:19:56.011 INFO:tasks.rados.rados.0.smithi100.stdout:1070: expect (ObjNum 315 snap 95 seq_num 315) 2024-04-07T16:19:56.012 INFO:tasks.rados.rados.0.smithi100.stdout:1071: read oid 25 snap -1 2024-04-07T16:19:56.012 INFO:tasks.rados.rados.0.smithi100.stdout:1071: expect deleted 2024-04-07T16:19:56.012 INFO:tasks.rados.rados.0.smithi100.stdout:1072: snap_create 2024-04-07T16:19:56.028 INFO:tasks.rados.rados.0.smithi100.stdout:1069: expect (ObjNum 262 snap 70 seq_num 262) 2024-04-07T16:19:56.152 INFO:tasks.rados.rados.0.smithi100.stdout:1070: expect (ObjNum 315 snap 95 seq_num 315) 2024-04-07T16:19:56.188 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:56 smithi144 bash[23522]: cluster 2024-04-07T16:19:55.012398+0000 mon.a (mon.0) 1199 : cluster 0 osdmap e261: 8 total, 8 up, 8 in 2024-04-07T16:19:56.436 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:56 smithi100 bash[21346]: cluster 2024-04-07T16:19:55.012398+0000 mon.a (mon.0) 1199 : cluster 0 osdmap e261: 8 total, 8 up, 8 in 2024-04-07T16:19:56.436 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:56 smithi100 bash[31660]: cluster 2024-04-07T16:19:55.012398+0000 mon.a (mon.0) 1199 : cluster 0 osdmap e261: 8 total, 8 up, 8 in 2024-04-07T16:19:57.027 INFO:tasks.rados.rados.0.smithi100.stdout:1069: done (3 left) 2024-04-07T16:19:57.028 INFO:tasks.rados.rados.0.smithi100.stdout:1070: done (2 left) 2024-04-07T16:19:57.028 INFO:tasks.rados.rados.0.smithi100.stdout:1071: done (1 left) 2024-04-07T16:19:57.028 INFO:tasks.rados.rados.0.smithi100.stdout:1072: done (0 left) 2024-04-07T16:19:57.028 INFO:tasks.rados.rados.0.smithi100.stdout:1073: copy_from oid 31 from oid 33 current snap is 99 2024-04-07T16:19:57.028 INFO:tasks.rados.rados.0.smithi100.stdout:1074: read oid 10 snap -1 2024-04-07T16:19:57.028 INFO:tasks.rados.rados.0.smithi100.stdout:1074: expect (ObjNum 296 snap 84 seq_num 296) 2024-04-07T16:19:57.028 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 11 current snap is 99 2024-04-07T16:19:57.028 INFO:tasks.rados.rados.0.smithi100.stdout:1075: seq_num 327 ranges {3014656=679936} 2024-04-07T16:19:57.049 INFO:tasks.rados.rados.0.smithi100.stdout:1075: writing smithi10066867-11 from 3014656 to 3694592 tid 1 2024-04-07T16:19:57.050 INFO:tasks.rados.rados.0.smithi100.stdout:1076: setattr oid 29 current snap is 99 2024-04-07T16:19:57.054 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 20 current snap is 99 2024-04-07T16:19:57.054 INFO:tasks.rados.rados.0.smithi100.stdout:1077: seq_num 328 ranges {516096=696320} 2024-04-07T16:19:57.061 INFO:tasks.rados.rados.0.smithi100.stdout:1077: writing smithi10066867-20 from 516096 to 1212416 tid 1 2024-04-07T16:19:57.061 INFO:tasks.rados.rados.0.smithi100.stdout:1078: rmattr oid 27 current snap is 99 2024-04-07T16:19:57.061 INFO:tasks.rados.rados.0.smithi100.stdout:1079: rmattr oid 6 current snap is 99 2024-04-07T16:19:57.061 INFO:tasks.rados.rados.0.smithi100.stdout:1080: snap_remove snap 87 2024-04-07T16:19:57.278 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:57 smithi144 bash[23522]: cluster 2024-04-07T16:19:56.022079+0000 mgr.y (mgr.25073) 43 : cluster 0 pgmap v64: 121 pgs: 121 active+clean; 243 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 25 MiB/s rd, 8.6 MiB/s wr, 49 op/s 2024-04-07T16:19:57.278 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:57 smithi144 bash[23522]: cluster 2024-04-07T16:19:56.025751+0000 mon.a (mon.0) 1200 : cluster 0 osdmap e262: 8 total, 8 up, 8 in 2024-04-07T16:19:57.436 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:57 smithi100 bash[21346]: cluster 2024-04-07T16:19:56.022079+0000 mgr.y (mgr.25073) 43 : cluster 0 pgmap v64: 121 pgs: 121 active+clean; 243 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 25 MiB/s rd, 8.6 MiB/s wr, 49 op/s 2024-04-07T16:19:57.436 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:57 smithi100 bash[21346]: cluster 2024-04-07T16:19:56.025751+0000 mon.a (mon.0) 1200 : cluster 0 osdmap e262: 8 total, 8 up, 8 in 2024-04-07T16:19:57.436 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:57 smithi100 bash[31660]: cluster 2024-04-07T16:19:56.022079+0000 mgr.y (mgr.25073) 43 : cluster 0 pgmap v64: 121 pgs: 121 active+clean; 243 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 25 MiB/s rd, 8.6 MiB/s wr, 49 op/s 2024-04-07T16:19:57.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:57 smithi100 bash[31660]: cluster 2024-04-07T16:19:56.025751+0000 mon.a (mon.0) 1200 : cluster 0 osdmap e262: 8 total, 8 up, 8 in 2024-04-07T16:19:57.891 INFO:tasks.rados.rados.0.smithi100.stdout:1080: done (7 left) 2024-04-07T16:19:57.891 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 26 current snap is 99 2024-04-07T16:19:57.891 INFO:tasks.rados.rados.0.smithi100.stdout:1081: seq_num 329 ranges {548864=712704} 2024-04-07T16:19:57.902 INFO:tasks.rados.rados.0.smithi100.stdout:1081: writing smithi10066867-26 from 548864 to 1261568 tid 1 2024-04-07T16:19:57.902 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 24 current snap is 99 2024-04-07T16:19:57.902 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 29 v 475 (ObjNum 161 snap 0 seq_num 1) dirty exists 2024-04-07T16:19:57.902 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 6 v 543 (ObjNum 324 snap 97 seq_num 324) dirty exists 2024-04-07T16:19:57.902 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 27 v 261 (ObjNum 316 snap 95 seq_num 316) dirty exists 2024-04-07T16:19:57.902 INFO:tasks.rados.rados.0.smithi100.stdout:1077: finishing write tid 1 to smithi10066867-20 2024-04-07T16:19:57.903 INFO:tasks.rados.rados.0.smithi100.stdout:1077: finishing write tid 2 to smithi10066867-20 2024-04-07T16:19:57.903 INFO:tasks.rados.rados.0.smithi100.stdout:1074: expect (ObjNum 296 snap 84 seq_num 296) 2024-04-07T16:19:58.144 INFO:tasks.rados.rados.0.smithi100.stdout:1077: finishing write tid 3 to smithi10066867-20 2024-04-07T16:19:58.144 INFO:tasks.rados.rados.0.smithi100.stdout:1077: oid 20 updating version 0 to 509 2024-04-07T16:19:58.144 INFO:tasks.rados.rados.0.smithi100.stdout:1077: oid 20 updating version 509 to 510 2024-04-07T16:19:58.144 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 20 v 510 (ObjNum 328 snap 99 seq_num 328) dirty exists 2024-04-07T16:19:58.144 INFO:tasks.rados.rados.0.smithi100.stdout:1077: left oid 20 (ObjNum 328 snap 99 seq_num 328) 2024-04-07T16:19:58.144 INFO:tasks.rados.rados.0.smithi100.stdout:1075: finishing write tid 1 to smithi10066867-11 2024-04-07T16:19:58.144 INFO:tasks.rados.rados.0.smithi100.stdout:1073: finishing copy_from racing read to smithi10066867-31 2024-04-07T16:19:58.145 INFO:tasks.rados.rados.0.smithi100.stdout:1075: finishing write tid 2 to smithi10066867-11 2024-04-07T16:19:58.145 INFO:tasks.rados.rados.0.smithi100.stdout:1073: finishing copy_from to smithi10066867-31 2024-04-07T16:19:58.145 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 31 v 226 (ObjNum 296 snap 84 seq_num 296) dirty exists 2024-04-07T16:19:58.145 INFO:tasks.rados.rados.0.smithi100.stdout:1075: finishing write tid 3 to smithi10066867-11 2024-04-07T16:19:58.145 INFO:tasks.rados.rados.0.smithi100.stdout:1075: oid 11 updating version 0 to 639 2024-04-07T16:19:58.145 INFO:tasks.rados.rados.0.smithi100.stdout:1075: oid 11 version 639 is already newer than 638 2024-04-07T16:19:58.145 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 11 v 639 (ObjNum 327 snap 99 seq_num 327) dirty exists 2024-04-07T16:19:58.145 INFO:tasks.rados.rados.0.smithi100.stdout:1075: left oid 11 (ObjNum 327 snap 99 seq_num 327) 2024-04-07T16:19:58.145 INFO:tasks.rados.rados.0.smithi100.stdout:1081: finishing write tid 1 to smithi10066867-26 2024-04-07T16:19:58.145 INFO:tasks.rados.rados.0.smithi100.stdout:1081: finishing write tid 2 to smithi10066867-26 2024-04-07T16:19:58.145 INFO:tasks.rados.rados.0.smithi100.stdout:1081: finishing write tid 3 to smithi10066867-26 2024-04-07T16:19:58.145 INFO:tasks.rados.rados.0.smithi100.stdout:1081: oid 26 updating version 0 to 511 2024-04-07T16:19:58.145 INFO:tasks.rados.rados.0.smithi100.stdout:1081: oid 26 updating version 511 to 512 2024-04-07T16:19:58.145 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 26 v 512 (ObjNum 329 snap 99 seq_num 329) dirty exists 2024-04-07T16:19:58.145 INFO:tasks.rados.rados.0.smithi100.stdout:1081: left oid 26 (ObjNum 329 snap 99 seq_num 329) 2024-04-07T16:19:58.146 INFO:tasks.rados.rados.0.smithi100.stdout:1082: seq_num 330 ranges {8658944=737280} 2024-04-07T16:19:58.223 INFO:tasks.rados.rados.0.smithi100.stdout:1082: writing smithi10066867-24 from 8658944 to 9396224 tid 1 2024-04-07T16:19:58.223 INFO:tasks.rados.rados.0.smithi100.stdout:1073: done (8 left) 2024-04-07T16:19:58.223 INFO:tasks.rados.rados.0.smithi100.stdout:1074: done (7 left) 2024-04-07T16:19:58.223 INFO:tasks.rados.rados.0.smithi100.stdout:1075: done (6 left) 2024-04-07T16:19:58.223 INFO:tasks.rados.rados.0.smithi100.stdout:1076: done (5 left) 2024-04-07T16:19:58.223 INFO:tasks.rados.rados.0.smithi100.stdout:1077: done (4 left) 2024-04-07T16:19:58.223 INFO:tasks.rados.rados.0.smithi100.stdout:1078: done (3 left) 2024-04-07T16:19:58.224 INFO:tasks.rados.rados.0.smithi100.stdout:1079: done (2 left) 2024-04-07T16:19:58.224 INFO:tasks.rados.rados.0.smithi100.stdout:1081: done (1 left) 2024-04-07T16:19:58.224 INFO:tasks.rados.rados.0.smithi100.stdout:1083: snap_create 2024-04-07T16:19:58.230 INFO:tasks.rados.rados.0.smithi100.stdout:1082: finishing write tid 1 to smithi10066867-24 2024-04-07T16:19:58.230 INFO:tasks.rados.rados.0.smithi100.stdout:1082: finishing write tid 2 to smithi10066867-24 2024-04-07T16:19:58.232 INFO:tasks.rados.rados.0.smithi100.stdout:1082: finishing write tid 3 to smithi10066867-24 2024-04-07T16:19:58.232 INFO:tasks.rados.rados.0.smithi100.stdout:1082: oid 24 updating version 0 to 537 2024-04-07T16:19:58.232 INFO:tasks.rados.rados.0.smithi100.stdout:1082: oid 24 version 537 is already newer than 536 2024-04-07T16:19:58.232 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 24 v 537 (ObjNum 330 snap 99 seq_num 330) dirty exists 2024-04-07T16:19:58.233 INFO:tasks.rados.rados.0.smithi100.stdout:1082: left oid 24 (ObjNum 330 snap 99 seq_num 330) 2024-04-07T16:19:58.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:58 smithi144 bash[23522]: cluster 2024-04-07T16:19:57.042722+0000 mon.a (mon.0) 1201 : cluster 0 osdmap e263: 8 total, 8 up, 8 in 2024-04-07T16:19:58.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:58 smithi144 bash[23522]: cluster 2024-04-07T16:19:57.904340+0000 mon.a (mon.0) 1202 : cluster 0 osdmap e264: 8 total, 8 up, 8 in 2024-04-07T16:19:58.416 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:58 smithi100 bash[21346]: cluster 2024-04-07T16:19:57.042722+0000 mon.a (mon.0) 1201 : cluster 0 osdmap e263: 8 total, 8 up, 8 in 2024-04-07T16:19:58.417 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:58 smithi100 bash[21346]: cluster 2024-04-07T16:19:57.904340+0000 mon.a (mon.0) 1202 : cluster 0 osdmap e264: 8 total, 8 up, 8 in 2024-04-07T16:19:58.417 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:58 smithi100 bash[31660]: cluster 2024-04-07T16:19:57.042722+0000 mon.a (mon.0) 1201 : cluster 0 osdmap e263: 8 total, 8 up, 8 in 2024-04-07T16:19:58.417 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:58 smithi100 bash[31660]: cluster 2024-04-07T16:19:57.904340+0000 mon.a (mon.0) 1202 : cluster 0 osdmap e264: 8 total, 8 up, 8 in 2024-04-07T16:19:58.686 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:19:58 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:19:58] "GET /metrics HTTP/1.1" 200 36084 "" "Prometheus/2.43.0" 2024-04-07T16:19:59.028 INFO:tasks.rados.rados.0.smithi100.stdout:1082: done (1 left) 2024-04-07T16:19:59.029 INFO:tasks.rados.rados.0.smithi100.stdout:1083: done (0 left) 2024-04-07T16:19:59.029 INFO:tasks.rados.rados.0.smithi100.stdout:1084: read oid 5 snap 96 2024-04-07T16:19:59.029 INFO:tasks.rados.rados.0.smithi100.stdout:1084: expect (ObjNum 320 snap 96 seq_num 320) 2024-04-07T16:19:59.029 INFO:tasks.rados.rados.0.smithi100.stdout:1085: read oid 47 snap 93 2024-04-07T16:19:59.029 INFO:tasks.rados.rados.0.smithi100.stdout:1085: expect (ObjNum 286 snap 82 seq_num 286) 2024-04-07T16:19:59.029 INFO:tasks.rados.rados.0.smithi100.stdout:1086: copy_from oid 23 from oid 15 current snap is 100 2024-04-07T16:19:59.029 INFO:tasks.rados.rados.0.smithi100.stdout:1087: delete oid 9 current snap is 100 2024-04-07T16:19:59.050 INFO:tasks.rados.rados.0.smithi100.stdout:1087: done (3 left) 2024-04-07T16:19:59.050 INFO:tasks.rados.rados.0.smithi100.stdout:1088: setattr oid 21 current snap is 100 2024-04-07T16:19:59.051 INFO:tasks.rados.rados.0.smithi100.stdout:1089: rollback oid 31 current snap is 100 2024-04-07T16:19:59.051 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 31 to 89 2024-04-07T16:19:59.051 INFO:tasks.rados.rados.0.smithi100.stdout:1090: delete oid 29 current snap is 100 2024-04-07T16:19:59.051 INFO:tasks.rados.rados.0.smithi100.stdout:1084: expect (ObjNum 320 snap 96 seq_num 320) 2024-04-07T16:19:59.093 INFO:tasks.rados.rados.0.smithi100.stdout:1085: expect (ObjNum 286 snap 82 seq_num 286) 2024-04-07T16:19:59.183 INFO:tasks.rados.rados.0.smithi100.stdout:1089: finishing rollback tid 0 to smithi10066867-31 2024-04-07T16:19:59.183 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 21 v 208 (ObjNum 222 snap 59 seq_num 222) dirty exists 2024-04-07T16:19:59.184 INFO:tasks.rados.rados.0.smithi100.stdout:1089: finishing rollback tid 1 to smithi10066867-31 2024-04-07T16:19:59.184 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 31 v 262 (ObjNum 161 snap 0 seq_num 1) dirty dne 2024-04-07T16:19:59.184 INFO:tasks.rados.rados.0.smithi100.stdout:1086: finishing copy_from racing read to smithi10066867-23 2024-04-07T16:19:59.184 INFO:tasks.rados.rados.0.smithi100.stdout:1086: finishing copy_from to smithi10066867-23 2024-04-07T16:19:59.184 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 23 v 561 (ObjNum 287 snap 83 seq_num 287) dirty exists 2024-04-07T16:19:59.184 INFO:tasks.rados.rados.0.smithi100.stdout:1084: done (5 left) 2024-04-07T16:19:59.184 INFO:tasks.rados.rados.0.smithi100.stdout:1085: done (4 left) 2024-04-07T16:19:59.184 INFO:tasks.rados.rados.0.smithi100.stdout:1086: done (3 left) 2024-04-07T16:19:59.184 INFO:tasks.rados.rados.0.smithi100.stdout:1088: done (2 left) 2024-04-07T16:19:59.184 INFO:tasks.rados.rados.0.smithi100.stdout:1089: done (1 left) 2024-04-07T16:19:59.184 INFO:tasks.rados.rados.0.smithi100.stdout:1090: done (0 left) 2024-04-07T16:19:59.184 INFO:tasks.rados.rados.0.smithi100.stdout:1091: copy_from oid 34 from oid 6 current snap is 100 2024-04-07T16:19:59.185 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 1 current snap is 100 2024-04-07T16:19:59.185 INFO:tasks.rados.rados.0.smithi100.stdout:1092: seq_num 333 ranges {2007040=786432} 2024-04-07T16:19:59.235 INFO:tasks.rados.rados.0.smithi100.stdout:1092: writing smithi10066867-1 from 2007040 to 2793472 tid 1 2024-04-07T16:19:59.235 INFO:tasks.rados.rados.0.smithi100.stdout:1093: delete oid 36 current snap is 100 2024-04-07T16:19:59.235 INFO:tasks.rados.rados.0.smithi100.stdout:1091: finishing copy_from racing read to smithi10066867-34 2024-04-07T16:19:59.235 INFO:tasks.rados.rados.0.smithi100.stdout:1091: finishing copy_from to smithi10066867-34 2024-04-07T16:19:59.235 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 34 v 543 (ObjNum 324 snap 97 seq_num 324) dirty exists 2024-04-07T16:19:59.238 INFO:tasks.rados.rados.0.smithi100.stdout:1091: done (2 left) 2024-04-07T16:19:59.238 INFO:tasks.rados.rados.0.smithi100.stdout:1093: done (1 left) 2024-04-07T16:19:59.238 INFO:tasks.rados.rados.0.smithi100.stdout:1094: snap_create 2024-04-07T16:19:59.240 INFO:tasks.rados.rados.0.smithi100.stdout:1092: finishing write tid 1 to smithi10066867-1 2024-04-07T16:19:59.240 INFO:tasks.rados.rados.0.smithi100.stdout:1092: finishing write tid 2 to smithi10066867-1 2024-04-07T16:19:59.241 INFO:tasks.rados.rados.0.smithi100.stdout:1092: finishing write tid 3 to smithi10066867-1 2024-04-07T16:19:59.241 INFO:tasks.rados.rados.0.smithi100.stdout:1092: oid 1 updating version 0 to 230 2024-04-07T16:19:59.241 INFO:tasks.rados.rados.0.smithi100.stdout:1092: oid 1 updating version 230 to 231 2024-04-07T16:19:59.241 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 1 v 231 (ObjNum 333 snap 100 seq_num 333) dirty exists 2024-04-07T16:19:59.241 INFO:tasks.rados.rados.0.smithi100.stdout:1092: left oid 1 (ObjNum 333 snap 100 seq_num 333) 2024-04-07T16:19:59.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:19:59 smithi144 bash[23522]: cluster 2024-04-07T16:19:58.022916+0000 mgr.y (mgr.25073) 44 : cluster 0 pgmap v68: 121 pgs: 121 active+clean; 243 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 25 MiB/s rd, 8.7 MiB/s wr, 50 op/s 2024-04-07T16:19:59.436 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:19:59 smithi100 bash[21346]: cluster 2024-04-07T16:19:58.022916+0000 mgr.y (mgr.25073) 44 : cluster 0 pgmap v68: 121 pgs: 121 active+clean; 243 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 25 MiB/s rd, 8.7 MiB/s wr, 50 op/s 2024-04-07T16:19:59.436 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:19:59 smithi100 bash[31660]: cluster 2024-04-07T16:19:58.022916+0000 mgr.y (mgr.25073) 44 : cluster 0 pgmap v68: 121 pgs: 121 active+clean; 243 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 25 MiB/s rd, 8.7 MiB/s wr, 50 op/s 2024-04-07T16:20:00.159 INFO:tasks.rados.rados.0.smithi100.stdout:1092: done (1 left) 2024-04-07T16:20:00.164 INFO:tasks.rados.rados.0.smithi100.stdout:1094: done (0 left) 2024-04-07T16:20:00.164 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 47 current snap is 101 2024-04-07T16:20:00.164 INFO:tasks.rados.rados.0.smithi100.stdout:1095: seq_num 335 ranges {2498560=417792,2916352=401408} 2024-04-07T16:20:00.192 INFO:tasks.rados.rados.0.smithi100.stdout:1095: writing smithi10066867-47 from 2498560 to 2916352 tid 1 2024-04-07T16:20:00.196 INFO:tasks.rados.rados.0.smithi100.stdout:1095: writing smithi10066867-47 from 2916352 to 3317760 tid 2 2024-04-07T16:20:00.197 INFO:tasks.rados.rados.0.smithi100.stdout:1096: snap_remove snap 93 2024-04-07T16:20:00.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:00 smithi144 bash[23522]: cluster 2024-04-07T16:19:59.042322+0000 mon.a (mon.0) 1203 : cluster 0 osdmap e265: 8 total, 8 up, 8 in 2024-04-07T16:20:00.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:00 smithi144 bash[23522]: audit 2024-04-07T16:19:59.120885+0000 mon.a (mon.0) 1204 : audit 1 from='mgr.25073 ' entity='mgr.y' 2024-04-07T16:20:00.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:00 smithi144 bash[23522]: audit 2024-04-07T16:19:59.133672+0000 mon.a (mon.0) 1205 : audit 1 from='mgr.25073 ' entity='mgr.y' 2024-04-07T16:20:00.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:00 smithi144 bash[23522]: audit 2024-04-07T16:19:59.136528+0000 mon.c (mon.2) 117 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:20:00.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:00 smithi144 bash[23522]: audit 2024-04-07T16:19:59.137756+0000 mon.c (mon.2) 118 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:20:00.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:00 smithi144 bash[23522]: audit 2024-04-07T16:19:59.140502+0000 mon.c (mon.2) 119 : audit 1 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:20:00.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:00 smithi144 bash[23522]: audit 2024-04-07T16:19:59.164877+0000 mon.a (mon.0) 1206 : audit 1 from='mgr.25073 ' entity='mgr.y' 2024-04-07T16:20:00.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:00 smithi144 bash[23522]: audit 2024-04-07T16:19:59.254310+0000 mon.c (mon.2) 120 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:20:00.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:00 smithi144 bash[23522]: audit 2024-04-07T16:19:59.258390+0000 mon.c (mon.2) 121 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:20:00.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:00 smithi144 bash[23522]: cephadm 2024-04-07T16:19:59.259440+0000 mgr.y (mgr.25073) 45 : cephadm 1 Upgrade: Setting container_image for all mgr 2024-04-07T16:20:00.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:00 smithi144 bash[23522]: audit 2024-04-07T16:19:59.265568+0000 mon.a (mon.0) 1207 : audit 1 from='mgr.25073 ' entity='mgr.y' 2024-04-07T16:20:00.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:00 smithi144 bash[23522]: audit 2024-04-07T16:19:59.267770+0000 mon.c (mon.2) 122 : audit 1 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.x"}]: dispatch 2024-04-07T16:20:00.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:00 smithi144 bash[23522]: audit 2024-04-07T16:19:59.268095+0000 mon.a (mon.0) 1208 : audit 1 from='mgr.25073 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.x"}]: dispatch 2024-04-07T16:20:00.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:00 smithi144 bash[23522]: audit 2024-04-07T16:19:59.274333+0000 mon.a (mon.0) 1209 : audit 1 from='mgr.25073 ' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.x"}]': finished 2024-04-07T16:20:00.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:00 smithi144 bash[23522]: audit 2024-04-07T16:19:59.277881+0000 mon.c (mon.2) 123 : audit 1 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.y"}]: dispatch 2024-04-07T16:20:00.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:00 smithi144 bash[23522]: audit 2024-04-07T16:19:59.278184+0000 mon.a (mon.0) 1210 : audit 1 from='mgr.25073 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.y"}]: dispatch 2024-04-07T16:20:00.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:00 smithi144 bash[23522]: audit 2024-04-07T16:19:59.284251+0000 mon.a (mon.0) 1211 : audit 1 from='mgr.25073 ' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.y"}]': finished 2024-04-07T16:20:00.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:00 smithi144 bash[23522]: audit 2024-04-07T16:19:59.288276+0000 mon.c (mon.2) 124 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-04-07T16:20:00.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:00 smithi144 bash[23522]: audit 2024-04-07T16:19:59.289142+0000 mon.c (mon.2) 125 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "mon ok-to-stop", "ids": ["a"]}]: dispatch 2024-04-07T16:20:00.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:00 smithi144 bash[23522]: cephadm 2024-04-07T16:19:59.289742+0000 mgr.y (mgr.25073) 46 : cephadm 1 Upgrade: It appears safe to stop mon.a 2024-04-07T16:20:00.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:00 smithi144 bash[23522]: cluster 2024-04-07T16:20:00.000117+0000 mon.a (mon.0) 1212 : cluster 1 overall HEALTH_OK 2024-04-07T16:20:00.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:00 smithi100 bash[21346]: cluster 2024-04-07T16:19:59.042322+0000 mon.a (mon.0) 1203 : cluster 0 osdmap e265: 8 total, 8 up, 8 in 2024-04-07T16:20:00.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:00 smithi100 bash[21346]: audit 2024-04-07T16:19:59.120885+0000 mon.a (mon.0) 1204 : audit 1 from='mgr.25073 ' entity='mgr.y' 2024-04-07T16:20:00.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:00 smithi100 bash[21346]: audit 2024-04-07T16:19:59.133672+0000 mon.a (mon.0) 1205 : audit 1 from='mgr.25073 ' entity='mgr.y' 2024-04-07T16:20:00.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:00 smithi100 bash[21346]: audit 2024-04-07T16:19:59.136528+0000 mon.c (mon.2) 117 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:20:00.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:00 smithi100 bash[21346]: audit 2024-04-07T16:19:59.137756+0000 mon.c (mon.2) 118 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:20:00.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:00 smithi100 bash[21346]: audit 2024-04-07T16:19:59.140502+0000 mon.c (mon.2) 119 : audit 1 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:20:00.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:00 smithi100 bash[21346]: audit 2024-04-07T16:19:59.164877+0000 mon.a (mon.0) 1206 : audit 1 from='mgr.25073 ' entity='mgr.y' 2024-04-07T16:20:00.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:00 smithi100 bash[21346]: audit 2024-04-07T16:19:59.254310+0000 mon.c (mon.2) 120 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:20:00.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:00 smithi100 bash[21346]: audit 2024-04-07T16:19:59.258390+0000 mon.c (mon.2) 121 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:20:00.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:00 smithi100 bash[21346]: cephadm 2024-04-07T16:19:59.259440+0000 mgr.y (mgr.25073) 45 : cephadm 1 Upgrade: Setting container_image for all mgr 2024-04-07T16:20:00.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:00 smithi100 bash[21346]: audit 2024-04-07T16:19:59.265568+0000 mon.a (mon.0) 1207 : audit 1 from='mgr.25073 ' entity='mgr.y' 2024-04-07T16:20:00.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:00 smithi100 bash[21346]: audit 2024-04-07T16:19:59.267770+0000 mon.c (mon.2) 122 : audit 1 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.x"}]: dispatch 2024-04-07T16:20:00.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:00 smithi100 bash[21346]: audit 2024-04-07T16:19:59.268095+0000 mon.a (mon.0) 1208 : audit 1 from='mgr.25073 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.x"}]: dispatch 2024-04-07T16:20:00.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:00 smithi100 bash[21346]: audit 2024-04-07T16:19:59.274333+0000 mon.a (mon.0) 1209 : audit 1 from='mgr.25073 ' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.x"}]': finished 2024-04-07T16:20:00.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:00 smithi100 bash[21346]: audit 2024-04-07T16:19:59.277881+0000 mon.c (mon.2) 123 : audit 1 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.y"}]: dispatch 2024-04-07T16:20:00.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:00 smithi100 bash[21346]: audit 2024-04-07T16:19:59.278184+0000 mon.a (mon.0) 1210 : audit 1 from='mgr.25073 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.y"}]: dispatch 2024-04-07T16:20:00.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:00 smithi100 bash[21346]: audit 2024-04-07T16:19:59.284251+0000 mon.a (mon.0) 1211 : audit 1 from='mgr.25073 ' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.y"}]': finished 2024-04-07T16:20:00.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:00 smithi100 bash[21346]: audit 2024-04-07T16:19:59.288276+0000 mon.c (mon.2) 124 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-04-07T16:20:00.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:00 smithi100 bash[21346]: audit 2024-04-07T16:19:59.289142+0000 mon.c (mon.2) 125 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "mon ok-to-stop", "ids": ["a"]}]: dispatch 2024-04-07T16:20:00.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:00 smithi100 bash[21346]: cephadm 2024-04-07T16:19:59.289742+0000 mgr.y (mgr.25073) 46 : cephadm 1 Upgrade: It appears safe to stop mon.a 2024-04-07T16:20:00.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:00 smithi100 bash[21346]: cluster 2024-04-07T16:20:00.000117+0000 mon.a (mon.0) 1212 : cluster 1 overall HEALTH_OK 2024-04-07T16:20:00.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:00 smithi100 bash[31660]: cluster 2024-04-07T16:19:59.042322+0000 mon.a (mon.0) 1203 : cluster 0 osdmap e265: 8 total, 8 up, 8 in 2024-04-07T16:20:00.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:00 smithi100 bash[31660]: audit 2024-04-07T16:19:59.120885+0000 mon.a (mon.0) 1204 : audit 1 from='mgr.25073 ' entity='mgr.y' 2024-04-07T16:20:00.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:00 smithi100 bash[31660]: audit 2024-04-07T16:19:59.133672+0000 mon.a (mon.0) 1205 : audit 1 from='mgr.25073 ' entity='mgr.y' 2024-04-07T16:20:00.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:00 smithi100 bash[31660]: audit 2024-04-07T16:19:59.136528+0000 mon.c (mon.2) 117 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:20:00.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:00 smithi100 bash[31660]: audit 2024-04-07T16:19:59.137756+0000 mon.c (mon.2) 118 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:20:00.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:00 smithi100 bash[31660]: audit 2024-04-07T16:19:59.140502+0000 mon.c (mon.2) 119 : audit 1 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:20:00.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:00 smithi100 bash[31660]: audit 2024-04-07T16:19:59.164877+0000 mon.a (mon.0) 1206 : audit 1 from='mgr.25073 ' entity='mgr.y' 2024-04-07T16:20:00.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:00 smithi100 bash[31660]: audit 2024-04-07T16:19:59.254310+0000 mon.c (mon.2) 120 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:20:00.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:00 smithi100 bash[31660]: audit 2024-04-07T16:19:59.258390+0000 mon.c (mon.2) 121 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:20:00.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:00 smithi100 bash[31660]: cephadm 2024-04-07T16:19:59.259440+0000 mgr.y (mgr.25073) 45 : cephadm 1 Upgrade: Setting container_image for all mgr 2024-04-07T16:20:00.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:00 smithi100 bash[31660]: audit 2024-04-07T16:19:59.265568+0000 mon.a (mon.0) 1207 : audit 1 from='mgr.25073 ' entity='mgr.y' 2024-04-07T16:20:00.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:00 smithi100 bash[31660]: audit 2024-04-07T16:19:59.267770+0000 mon.c (mon.2) 122 : audit 1 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.x"}]: dispatch 2024-04-07T16:20:00.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:00 smithi100 bash[31660]: audit 2024-04-07T16:19:59.268095+0000 mon.a (mon.0) 1208 : audit 1 from='mgr.25073 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.x"}]: dispatch 2024-04-07T16:20:00.440 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:00 smithi100 bash[31660]: audit 2024-04-07T16:19:59.274333+0000 mon.a (mon.0) 1209 : audit 1 from='mgr.25073 ' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.x"}]': finished 2024-04-07T16:20:00.440 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:00 smithi100 bash[31660]: audit 2024-04-07T16:19:59.277881+0000 mon.c (mon.2) 123 : audit 1 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.y"}]: dispatch 2024-04-07T16:20:00.440 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:00 smithi100 bash[31660]: audit 2024-04-07T16:19:59.278184+0000 mon.a (mon.0) 1210 : audit 1 from='mgr.25073 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.y"}]: dispatch 2024-04-07T16:20:00.440 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:00 smithi100 bash[31660]: audit 2024-04-07T16:19:59.284251+0000 mon.a (mon.0) 1211 : audit 1 from='mgr.25073 ' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.y"}]': finished 2024-04-07T16:20:00.440 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:00 smithi100 bash[31660]: audit 2024-04-07T16:19:59.288276+0000 mon.c (mon.2) 124 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-04-07T16:20:00.440 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:00 smithi100 bash[31660]: audit 2024-04-07T16:19:59.289142+0000 mon.c (mon.2) 125 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "mon ok-to-stop", "ids": ["a"]}]: dispatch 2024-04-07T16:20:00.440 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:00 smithi100 bash[31660]: cephadm 2024-04-07T16:19:59.289742+0000 mgr.y (mgr.25073) 46 : cephadm 1 Upgrade: It appears safe to stop mon.a 2024-04-07T16:20:00.440 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:00 smithi100 bash[31660]: cluster 2024-04-07T16:20:00.000117+0000 mon.a (mon.0) 1212 : cluster 1 overall HEALTH_OK 2024-04-07T16:20:01.169 INFO:tasks.rados.rados.0.smithi100.stdout:1096: done (1 left) 2024-04-07T16:20:01.169 INFO:tasks.rados.rados.0.smithi100.stdout:1097: delete oid 9 current snap is 101 2024-04-07T16:20:01.169 INFO:tasks.rados.rados.0.smithi100.stdout:1095: finishing write tid 1 to smithi10066867-47 2024-04-07T16:20:01.169 INFO:tasks.rados.rados.0.smithi100.stdout:1095: finishing write tid 2 to smithi10066867-47 2024-04-07T16:20:01.169 INFO:tasks.rados.rados.0.smithi100.stdout:1095: finishing write tid 3 to smithi10066867-47 2024-04-07T16:20:01.169 INFO:tasks.rados.rados.0.smithi100.stdout:1095: finishing write tid 4 to smithi10066867-47 2024-04-07T16:20:01.169 INFO:tasks.rados.rados.0.smithi100.stdout:1095: oid 47 updating version 0 to 211 2024-04-07T16:20:01.169 INFO:tasks.rados.rados.0.smithi100.stdout:1095: oid 47 version 211 is already newer than 210 2024-04-07T16:20:01.169 INFO:tasks.rados.rados.0.smithi100.stdout:1095: oid 47 updating version 211 to 212 2024-04-07T16:20:01.169 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 47 v 212 (ObjNum 335 snap 101 seq_num 335) dirty exists 2024-04-07T16:20:01.169 INFO:tasks.rados.rados.0.smithi100.stdout:1095: left oid 47 (ObjNum 335 snap 101 seq_num 335) 2024-04-07T16:20:01.187 INFO:tasks.rados.rados.0.smithi100.stdout:1095: done (1 left) 2024-04-07T16:20:01.187 INFO:tasks.rados.rados.0.smithi100.stdout:1097: done (0 left) 2024-04-07T16:20:01.187 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 46 current snap is 101 2024-04-07T16:20:01.187 INFO:tasks.rados.rados.0.smithi100.stdout:1098: seq_num 337 ranges {3547136=450560,3997696=401408} 2024-04-07T16:20:01.210 INFO:tasks.rados.rados.0.smithi100.stdout:1098: writing smithi10066867-46 from 3547136 to 3997696 tid 1 2024-04-07T16:20:01.213 INFO:tasks.rados.rados.0.smithi100.stdout:1098: writing smithi10066867-46 from 3997696 to 4399104 tid 2 2024-04-07T16:20:01.213 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 15 current snap is 101 2024-04-07T16:20:01.213 INFO:tasks.rados.rados.0.smithi100.stdout:1099: seq_num 338 ranges {5038080=466944,5505024=401408} 2024-04-07T16:20:01.244 INFO:tasks.rados.rados.0.smithi100.stdout:1099: writing smithi10066867-15 from 5038080 to 5505024 tid 1 2024-04-07T16:20:01.248 INFO:tasks.rados.rados.0.smithi100.stdout:1099: writing smithi10066867-15 from 5505024 to 5906432 tid 2 2024-04-07T16:20:01.248 INFO:tasks.rados.rados.0.smithi100.stdout:1100: read oid 31 snap -1 2024-04-07T16:20:01.248 INFO:tasks.rados.rados.0.smithi100.stdout:1100: expect deleted 2024-04-07T16:20:01.248 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 23 current snap is 101 2024-04-07T16:20:01.248 INFO:tasks.rados.rados.0.smithi100.stdout:1098: finishing write tid 1 to smithi10066867-46 2024-04-07T16:20:01.248 INFO:tasks.rados.rados.0.smithi100.stdout:1101: seq_num 339 ranges {5038080=483328,5521408=401408} 2024-04-07T16:20:01.268 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:01 smithi100 bash[21346]: cluster 2024-04-07T16:20:00.023831+0000 mgr.y (mgr.25073) 47 : cluster 0 pgmap v70: 121 pgs: 121 active+clean; 243 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 0 B/s wr, 4 op/s 2024-04-07T16:20:01.268 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:01 smithi100 bash[21346]: cluster 2024-04-07T16:20:00.171403+0000 mon.a (mon.0) 1213 : cluster 0 osdmap e266: 8 total, 8 up, 8 in 2024-04-07T16:20:01.268 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:01 smithi100 bash[31660]: cluster 2024-04-07T16:20:00.023831+0000 mgr.y (mgr.25073) 47 : cluster 0 pgmap v70: 121 pgs: 121 active+clean; 243 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 0 B/s wr, 4 op/s 2024-04-07T16:20:01.268 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:01 smithi100 bash[31660]: cluster 2024-04-07T16:20:00.171403+0000 mon.a (mon.0) 1213 : cluster 0 osdmap e266: 8 total, 8 up, 8 in 2024-04-07T16:20:01.278 INFO:tasks.rados.rados.0.smithi100.stdout:1101: writing smithi10066867-23 from 5038080 to 5521408 tid 1 2024-04-07T16:20:01.281 INFO:tasks.rados.rados.0.smithi100.stdout:1101: writing smithi10066867-23 from 5521408 to 5922816 tid 2 2024-04-07T16:20:01.281 INFO:tasks.rados.rados.0.smithi100.stdout:1102: snap_remove snap 68 2024-04-07T16:20:01.605 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:01 smithi144 bash[23522]: cluster 2024-04-07T16:20:00.023831+0000 mgr.y (mgr.25073) 47 : cluster 0 pgmap v70: 121 pgs: 121 active+clean; 243 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 0 B/s wr, 4 op/s 2024-04-07T16:20:01.605 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:01 smithi144 bash[23522]: cluster 2024-04-07T16:20:00.171403+0000 mon.a (mon.0) 1213 : cluster 0 osdmap e266: 8 total, 8 up, 8 in 2024-04-07T16:20:02.179 INFO:tasks.rados.rados.0.smithi100.stdout:1098: finishing write tid 2 to smithi10066867-46 2024-04-07T16:20:02.179 INFO:tasks.rados.rados.0.smithi100.stdout:1098: finishing write tid 3 to smithi10066867-46 2024-04-07T16:20:02.180 INFO:tasks.rados.rados.0.smithi100.stdout:1098: finishing write tid 4 to smithi10066867-46 2024-04-07T16:20:02.180 INFO:tasks.rados.rados.0.smithi100.stdout:1098: oid 46 updating version 0 to 546 2024-04-07T16:20:02.180 INFO:tasks.rados.rados.0.smithi100.stdout:1098: oid 46 version 546 is already newer than 545 2024-04-07T16:20:02.180 INFO:tasks.rados.rados.0.smithi100.stdout:1098: oid 46 version 546 is already newer than 544 2024-04-07T16:20:02.180 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 46 v 546 (ObjNum 337 snap 101 seq_num 337) dirty exists 2024-04-07T16:20:02.180 INFO:tasks.rados.rados.0.smithi100.stdout:1098: left oid 46 (ObjNum 337 snap 101 seq_num 337) 2024-04-07T16:20:02.180 INFO:tasks.rados.rados.0.smithi100.stdout:1099: finishing write tid 1 to smithi10066867-15 2024-04-07T16:20:02.180 INFO:tasks.rados.rados.0.smithi100.stdout:1099: finishing write tid 2 to smithi10066867-15 2024-04-07T16:20:02.180 INFO:tasks.rados.rados.0.smithi100.stdout:1099: finishing write tid 3 to smithi10066867-15 2024-04-07T16:20:02.180 INFO:tasks.rados.rados.0.smithi100.stdout:1099: finishing write tid 4 to smithi10066867-15 2024-04-07T16:20:02.180 INFO:tasks.rados.rados.0.smithi100.stdout:1099: oid 15 updating version 0 to 662 2024-04-07T16:20:02.180 INFO:tasks.rados.rados.0.smithi100.stdout:1099: oid 15 version 662 is already newer than 661 2024-04-07T16:20:02.180 INFO:tasks.rados.rados.0.smithi100.stdout:1099: oid 15 updating version 662 to 663 2024-04-07T16:20:02.180 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 15 v 663 (ObjNum 338 snap 101 seq_num 338) dirty exists 2024-04-07T16:20:02.180 INFO:tasks.rados.rados.0.smithi100.stdout:1099: left oid 15 (ObjNum 338 snap 101 seq_num 338) 2024-04-07T16:20:02.180 INFO:tasks.rados.rados.0.smithi100.stdout:1101: finishing write tid 1 to smithi10066867-23 2024-04-07T16:20:02.180 INFO:tasks.rados.rados.0.smithi100.stdout:1101: finishing write tid 2 to smithi10066867-23 2024-04-07T16:20:02.181 INFO:tasks.rados.rados.0.smithi100.stdout:1101: finishing write tid 3 to smithi10066867-23 2024-04-07T16:20:02.181 INFO:tasks.rados.rados.0.smithi100.stdout:1101: finishing write tid 4 to smithi10066867-23 2024-04-07T16:20:02.181 INFO:tasks.rados.rados.0.smithi100.stdout:1101: oid 23 updating version 0 to 666 2024-04-07T16:20:02.181 INFO:tasks.rados.rados.0.smithi100.stdout:1101: oid 23 version 666 is already newer than 665 2024-04-07T16:20:02.181 INFO:tasks.rados.rados.0.smithi100.stdout:1101: oid 23 updating version 666 to 667 2024-04-07T16:20:02.181 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 23 v 667 (ObjNum 339 snap 101 seq_num 339) dirty exists 2024-04-07T16:20:02.181 INFO:tasks.rados.rados.0.smithi100.stdout:1101: left oid 23 (ObjNum 339 snap 101 seq_num 339) 2024-04-07T16:20:02.181 INFO:tasks.rados.rados.0.smithi100.stdout:1098: done (4 left) 2024-04-07T16:20:02.181 INFO:tasks.rados.rados.0.smithi100.stdout:1099: done (3 left) 2024-04-07T16:20:02.181 INFO:tasks.rados.rados.0.smithi100.stdout:1100: done (2 left) 2024-04-07T16:20:02.181 INFO:tasks.rados.rados.0.smithi100.stdout:1101: done (1 left) 2024-04-07T16:20:02.181 INFO:tasks.rados.rados.0.smithi100.stdout:1102: done (0 left) 2024-04-07T16:20:02.181 INFO:tasks.rados.rados.0.smithi100.stdout:1103: rmattr oid 30 current snap is 101 2024-04-07T16:20:02.181 INFO:tasks.rados.rados.0.smithi100.stdout:1104: snap_remove snap 100 2024-04-07T16:20:02.436 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:02 smithi100 bash[21346]: cluster 2024-04-07T16:20:01.176860+0000 mon.a (mon.0) 1214 : cluster 0 osdmap e267: 8 total, 8 up, 8 in 2024-04-07T16:20:02.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:02 smithi100 bash[21346]: audit 2024-04-07T16:20:02.022961+0000 mon.a (mon.0) 1215 : audit 1 from='mgr.25073 ' entity='mgr.y' 2024-04-07T16:20:02.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:02 smithi100 bash[21346]: audit 2024-04-07T16:20:02.029291+0000 mon.c (mon.2) 126 : audit 1 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-07T16:20:02.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:02 smithi100 bash[21346]: audit 2024-04-07T16:20:02.030844+0000 mon.c (mon.2) 127 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-07T16:20:02.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:02 smithi100 bash[21346]: audit 2024-04-07T16:20:02.032360+0000 mon.c (mon.2) 128 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:20:02.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:02 smithi100 bash[31660]: cluster 2024-04-07T16:20:01.176860+0000 mon.a (mon.0) 1214 : cluster 0 osdmap e267: 8 total, 8 up, 8 in 2024-04-07T16:20:02.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:02 smithi100 bash[31660]: audit 2024-04-07T16:20:02.022961+0000 mon.a (mon.0) 1215 : audit 1 from='mgr.25073 ' entity='mgr.y' 2024-04-07T16:20:02.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:02 smithi100 bash[31660]: audit 2024-04-07T16:20:02.029291+0000 mon.c (mon.2) 126 : audit 1 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-07T16:20:02.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:02 smithi100 bash[31660]: audit 2024-04-07T16:20:02.030844+0000 mon.c (mon.2) 127 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-07T16:20:02.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:02 smithi100 bash[31660]: audit 2024-04-07T16:20:02.032360+0000 mon.c (mon.2) 128 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:20:02.605 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:20:02 smithi144 bash[59794]: ts=2024-04-07T16:20:02.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:20:02.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:02 smithi144 bash[23522]: cluster 2024-04-07T16:20:01.176860+0000 mon.a (mon.0) 1214 : cluster 0 osdmap e267: 8 total, 8 up, 8 in 2024-04-07T16:20:02.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:02 smithi144 bash[23522]: audit 2024-04-07T16:20:02.022961+0000 mon.a (mon.0) 1215 : audit 1 from='mgr.25073 ' entity='mgr.y' 2024-04-07T16:20:02.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:02 smithi144 bash[23522]: audit 2024-04-07T16:20:02.029291+0000 mon.c (mon.2) 126 : audit 1 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-07T16:20:02.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:02 smithi144 bash[23522]: audit 2024-04-07T16:20:02.030844+0000 mon.c (mon.2) 127 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-07T16:20:02.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:02 smithi144 bash[23522]: audit 2024-04-07T16:20:02.032360+0000 mon.c (mon.2) 128 : audit 0 from='mgr.25073 172.21.15.100:0/602283374' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:20:03.193 INFO:tasks.rados.rados.0.smithi100.stdout:1104: done (1 left) 2024-04-07T16:20:03.193 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 38 current snap is 101 2024-04-07T16:20:03.193 INFO:tasks.rados.rados.0.smithi100.stdout:1105: seq_num 340 ranges {1884160=499712,2383872=401408} 2024-04-07T16:20:03.206 INFO:tasks.rados.rados.0.smithi100.stdout:1105: writing smithi10066867-38 from 1884160 to 2383872 tid 1 2024-04-07T16:20:03.209 INFO:tasks.rados.rados.0.smithi100.stdout:1105: writing smithi10066867-38 from 2383872 to 2785280 tid 2 2024-04-07T16:20:03.209 INFO:tasks.rados.rados.0.smithi100.stdout:1106: read oid 40 snap -1 2024-04-07T16:20:03.209 INFO:tasks.rados.rados.0.smithi100.stdout:1106: expect (ObjNum 303 snap 87 seq_num 303) 2024-04-07T16:20:03.209 INFO:tasks.rados.rados.0.smithi100.stdout:1107: snap_remove snap 95 2024-04-07T16:20:03.436 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:03 smithi100 bash[21346]: cephadm 2024-04-07T16:20:02.013712+0000 mgr.y (mgr.25073) 48 : cephadm 1 Upgrade: Updating mon.a 2024-04-07T16:20:03.436 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:03 smithi100 bash[21346]: cluster 2024-04-07T16:20:02.024512+0000 mgr.y (mgr.25073) 49 : cluster 0 pgmap v73: 121 pgs: 121 active+clean; 258 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 18 MiB/s rd, 4.6 MiB/s wr, 38 op/s 2024-04-07T16:20:03.436 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:03 smithi100 bash[21346]: cephadm 2024-04-07T16:20:02.034145+0000 mgr.y (mgr.25073) 50 : cephadm 1 Deploying daemon mon.a on smithi100 2024-04-07T16:20:03.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:03 smithi100 bash[21346]: cluster 2024-04-07T16:20:02.191184+0000 mon.a (mon.0) 1216 : cluster 0 osdmap e268: 8 total, 8 up, 8 in 2024-04-07T16:20:03.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:03 smithi100 bash[31660]: cephadm 2024-04-07T16:20:02.013712+0000 mgr.y (mgr.25073) 48 : cephadm 1 Upgrade: Updating mon.a 2024-04-07T16:20:03.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:03 smithi100 bash[31660]: cluster 2024-04-07T16:20:02.024512+0000 mgr.y (mgr.25073) 49 : cluster 0 pgmap v73: 121 pgs: 121 active+clean; 258 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 18 MiB/s rd, 4.6 MiB/s wr, 38 op/s 2024-04-07T16:20:03.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:03 smithi100 bash[31660]: cephadm 2024-04-07T16:20:02.034145+0000 mgr.y (mgr.25073) 50 : cephadm 1 Deploying daemon mon.a on smithi100 2024-04-07T16:20:03.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:03 smithi100 bash[31660]: cluster 2024-04-07T16:20:02.191184+0000 mon.a (mon.0) 1216 : cluster 0 osdmap e268: 8 total, 8 up, 8 in 2024-04-07T16:20:03.498 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:03 smithi144 bash[23522]: cephadm 2024-04-07T16:20:02.013712+0000 mgr.y (mgr.25073) 48 : cephadm 1 Upgrade: Updating mon.a 2024-04-07T16:20:03.498 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:03 smithi144 bash[23522]: cluster 2024-04-07T16:20:02.024512+0000 mgr.y (mgr.25073) 49 : cluster 0 pgmap v73: 121 pgs: 121 active+clean; 258 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 18 MiB/s rd, 4.6 MiB/s wr, 38 op/s 2024-04-07T16:20:03.498 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:03 smithi144 bash[23522]: cephadm 2024-04-07T16:20:02.034145+0000 mgr.y (mgr.25073) 50 : cephadm 1 Deploying daemon mon.a on smithi100 2024-04-07T16:20:03.498 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:03 smithi144 bash[23522]: cluster 2024-04-07T16:20:02.191184+0000 mon.a (mon.0) 1216 : cluster 0 osdmap e268: 8 total, 8 up, 8 in 2024-04-07T16:20:03.855 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:20:03 smithi144 bash[59794]: ts=2024-04-07T16:20:03.508Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=nodes name=CephNodeDiskspaceWarning index=4 msg="Evaluating rule failed" rule="alert: CephNodeDiskspaceWarning\nexpr: predict_linear(node_filesystem_free_bytes{device=~\"/.*\"}[2d], 3600 * 24 * 5)\n * on (instance) group_left (nodename) node_uname_info < 0\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.8.4\n severity: warning\n type: ceph_default\nannotations:\n description: Mountpoint {{ $labels.mountpoint }} on {{ $labels.nodename }} will\n be full in less than 5 days based on the 48 hour trailing fill rate.\n summary: Host filesystem free space is getting low\n" err="found duplicate series for the match group {instance=\"smithi100\"} on the right hand-side of the operation: [{__name__=\"node_uname_info\", domainname=\"(none)\", instance=\"smithi100\", job=\"node\", machine=\"x86_64\", nodename=\"smithi100\", release=\"5.15.0-101-generic\", sysname=\"Linux\", version=\"#111-Ubuntu SMP Tue Mar 5 20:16:58 UTC 2024\"}, {__name__=\"node_uname_info\", cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", domainname=\"(none)\", instance=\"smithi100\", job=\"node\", machine=\"x86_64\", nodename=\"smithi100\", release=\"5.15.0-101-generic\", sysname=\"Linux\", version=\"#111-Ubuntu SMP Tue Mar 5 20:16:58 UTC 2024\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:20:04.210 INFO:tasks.rados.rados.0.smithi100.stdout:1107: done (3 left) 2024-04-07T16:20:04.211 INFO:tasks.rados.rados.0.smithi100.stdout:1108: delete oid 50 current snap is 101 2024-04-07T16:20:04.211 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 30 v 477 (ObjNum 106 snap 20 seq_num 106) dirty exists 2024-04-07T16:20:04.211 INFO:tasks.rados.rados.0.smithi100.stdout:1105: finishing write tid 1 to smithi10066867-38 2024-04-07T16:20:04.211 INFO:tasks.rados.rados.0.smithi100.stdout:1105: finishing write tid 2 to smithi10066867-38 2024-04-07T16:20:04.211 INFO:tasks.rados.rados.0.smithi100.stdout:1105: finishing write tid 3 to smithi10066867-38 2024-04-07T16:20:04.211 INFO:tasks.rados.rados.0.smithi100.stdout:1105: finishing write tid 4 to smithi10066867-38 2024-04-07T16:20:04.211 INFO:tasks.rados.rados.0.smithi100.stdout:1105: oid 38 updating version 0 to 240 2024-04-07T16:20:04.211 INFO:tasks.rados.rados.0.smithi100.stdout:1105: oid 38 updating version 240 to 242 2024-04-07T16:20:04.211 INFO:tasks.rados.rados.0.smithi100.stdout:1105: oid 38 version 242 is already newer than 241 2024-04-07T16:20:04.212 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 38 v 242 (ObjNum 340 snap 101 seq_num 340) dirty exists 2024-04-07T16:20:04.212 INFO:tasks.rados.rados.0.smithi100.stdout:1105: left oid 38 (ObjNum 340 snap 101 seq_num 340) 2024-04-07T16:20:04.212 INFO:tasks.rados.rados.0.smithi100.stdout:1106: expect (ObjNum 303 snap 87 seq_num 303) 2024-04-07T16:20:04.553 INFO:journalctl@ceph.osd.1.smithi100.stdout:Apr 07 16:20:04 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:20:04.553 INFO:journalctl@ceph.osd.2.smithi100.stdout:Apr 07 16:20:04 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:20:04.554 INFO:journalctl@ceph.osd.3.smithi100.stdout:Apr 07 16:20:04 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:20:04.554 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 16:20:04 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:20:04.555 INFO:journalctl@ceph.mds.a.smithi100.stdout:Apr 07 16:20:04 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:20:04.555 INFO:journalctl@ceph.osd.0.smithi100.stdout:Apr 07 16:20:04 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:20:04.556 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:04 smithi100 bash[31660]: cluster 2024-04-07T16:20:03.202624+0000 mon.a (mon.0) 1217 : cluster 0 osdmap e269: 8 total, 8 up, 8 in 2024-04-07T16:20:04.556 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:04 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:20:04.557 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Apr 07 16:20:04 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:20:04.557 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:04 smithi100 bash[21346]: cluster 2024-04-07T16:20:03.202624+0000 mon.a (mon.0) 1217 : cluster 0 osdmap e269: 8 total, 8 up, 8 in 2024-04-07T16:20:04.557 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:04 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:20:04.557 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:04 smithi100 systemd[1]: Stopping Ceph mon.a for 0702e3a8-f4f8-11ee-b649-cb9ed24678a4... 2024-04-07T16:20:04.558 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:04 smithi100 bash[21346]: debug 2024-04-07T16:20:04.566+0000 7fea09519700 -1 received signal: Terminated from /sbin/docker-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-stderr=true (PID: 1) UID: 0 2024-04-07T16:20:04.558 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:20:04 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:20:04.605 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:04 smithi144 bash[23522]: cluster 2024-04-07T16:20:03.202624+0000 mon.a (mon.0) 1217 : cluster 0 osdmap e269: 8 total, 8 up, 8 in 2024-04-07T16:20:04.637 INFO:tasks.rados.rados.0.smithi100.stdout:1103: done (3 left) 2024-04-07T16:20:04.637 INFO:tasks.rados.rados.0.smithi100.stdout:1105: done (2 left) 2024-04-07T16:20:04.637 INFO:tasks.rados.rados.0.smithi100.stdout:1106: done (1 left) 2024-04-07T16:20:04.637 INFO:tasks.rados.rados.0.smithi100.stdout:1108: done (0 left) 2024-04-07T16:20:04.637 INFO:tasks.rados.rados.0.smithi100.stdout:1109: read oid 23 snap 99 2024-04-07T16:20:04.638 INFO:tasks.rados.rados.0.smithi100.stdout:1109: expect (ObjNum 262 snap 70 seq_num 262) 2024-04-07T16:20:04.638 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 14 current snap is 101 2024-04-07T16:20:04.638 INFO:tasks.rados.rados.0.smithi100.stdout:1110: seq_num 342 ranges {0=532480,532480=401408} 2024-04-07T16:20:04.639 INFO:tasks.rados.rados.0.smithi100.stdout:1110: writing smithi10066867-14 from 0 to 532480 tid 1 2024-04-07T16:20:04.643 INFO:tasks.rados.rados.0.smithi100.stdout:1110: writing smithi10066867-14 from 532480 to 933888 tid 2 2024-04-07T16:20:04.643 INFO:tasks.rados.rados.0.smithi100.stdout:1111: delete oid 37 current snap is 101 2024-04-07T16:20:04.648 INFO:tasks.rados.rados.0.smithi100.stdout:1111: done (2 left) 2024-04-07T16:20:04.648 INFO:tasks.rados.rados.0.smithi100.stdout:1112: read oid 4 snap -1 2024-04-07T16:20:04.648 INFO:tasks.rados.rados.0.smithi100.stdout:1112: expect (ObjNum 171 snap 44 seq_num 171) 2024-04-07T16:20:04.648 INFO:tasks.rados.rados.0.smithi100.stdout:1113: snap_remove snap 98 2024-04-07T16:20:04.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:04 smithi100 bash[21346]: debug 2024-04-07T16:20:04.566+0000 7fea09519700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-04-07T16:20:05.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:05 smithi100 bash[74825]: ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4-mon-a 2024-04-07T16:20:05.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:05 smithi100 bash[74886]: Error response from daemon: No such container: ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4-mon-a 2024-04-07T16:20:05.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:05 smithi100 systemd[1]: ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@mon.a.service: Deactivated successfully. 2024-04-07T16:20:05.564 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:05 smithi100 systemd[1]: Stopped Ceph mon.a for 0702e3a8-f4f8-11ee-b649-cb9ed24678a4. 2024-04-07T16:20:05.936 INFO:journalctl@ceph.osd.0.smithi100.stdout:Apr 07 16:20:05 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:20:05.937 INFO:journalctl@ceph.osd.1.smithi100.stdout:Apr 07 16:20:05 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:20:05.937 INFO:journalctl@ceph.osd.3.smithi100.stdout:Apr 07 16:20:05 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:20:05.937 INFO:journalctl@ceph.mds.a.smithi100.stdout:Apr 07 16:20:05 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:20:05.937 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 16:20:05 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:20:05.938 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Apr 07 16:20:05 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:20:05.938 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:05 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:20:05.938 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:05 smithi100 systemd[1]: Started Ceph mon.a for 0702e3a8-f4f8-11ee-b649-cb9ed24678a4. 2024-04-07T16:20:05.938 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:20:05 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:20:05.939 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:05 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:20:05.939 INFO:journalctl@ceph.osd.2.smithi100.stdout:Apr 07 16:20:05 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:20:06.937 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.702+0000 7f53c05a7c80 0 set uid:gid to 167:167 (ceph:ceph) 2024-04-07T16:20:06.937 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.702+0000 7f53c05a7c80 0 ceph version 19.0.0-2785-gadce92d0 (adce92d056533f22dc1ee37e3141d098b046ec1b) squid (dev), process ceph-mon, pid 7 2024-04-07T16:20:06.937 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.702+0000 7f53c05a7c80 0 pidfile_write: ignore empty --pid-file 2024-04-07T16:20:06.937 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 0 load: jerasure load: lrc 2024-04-07T16:20:06.937 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: RocksDB version: 7.9.2 2024-04-07T16:20:06.937 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Git sha 0 2024-04-07T16:20:06.937 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Compile date 2024-04-05 22:33:20 2024-04-07T16:20:06.938 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: DB SUMMARY 2024-04-07T16:20:06.938 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: DB Session ID: BP3OKQ97D18MY4CGTGPW 2024-04-07T16:20:06.938 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: CURRENT file: CURRENT 2024-04-07T16:20:06.938 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: IDENTITY file: IDENTITY 2024-04-07T16:20:06.938 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: MANIFEST file: MANIFEST-000015 size: 1732 Bytes 2024-04-07T16:20:06.938 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000035.sst 2024-04-07T16:20:06.938 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000033.log size: 1220215 ; 2024-04-07T16:20:06.938 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.error_if_exists: 0 2024-04-07T16:20:06.938 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.create_if_missing: 0 2024-04-07T16:20:06.938 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.paranoid_checks: 1 2024-04-07T16:20:06.938 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.flush_verify_memtable_count: 1 2024-04-07T16:20:06.938 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-07T16:20:06.939 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-07T16:20:06.939 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.env: 0x55aab4ca9c60 2024-04-07T16:20:06.939 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.fs: PosixFileSystem 2024-04-07T16:20:06.939 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.info_log: 0x55aab5e5dc00 2024-04-07T16:20:06.939 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.max_file_opening_threads: 16 2024-04-07T16:20:06.939 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.statistics: (nil) 2024-04-07T16:20:06.939 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.use_fsync: 0 2024-04-07T16:20:06.939 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.max_log_file_size: 0 2024-04-07T16:20:06.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-07T16:20:06.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.log_file_time_to_roll: 0 2024-04-07T16:20:06.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.keep_log_file_num: 1000 2024-04-07T16:20:06.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.recycle_log_file_num: 0 2024-04-07T16:20:06.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.allow_fallocate: 1 2024-04-07T16:20:06.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.allow_mmap_reads: 0 2024-04-07T16:20:06.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.allow_mmap_writes: 0 2024-04-07T16:20:06.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.use_direct_reads: 0 2024-04-07T16:20:06.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-07T16:20:06.941 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.create_missing_column_families: 0 2024-04-07T16:20:06.941 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.db_log_dir: 2024-04-07T16:20:06.941 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.wal_dir: 2024-04-07T16:20:06.941 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.table_cache_numshardbits: 6 2024-04-07T16:20:06.941 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.WAL_ttl_seconds: 0 2024-04-07T16:20:06.941 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.WAL_size_limit_MB: 0 2024-04-07T16:20:06.941 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-07T16:20:06.941 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-07T16:20:06.941 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.is_fd_close_on_exec: 1 2024-04-07T16:20:06.941 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.advise_random_on_open: 1 2024-04-07T16:20:06.941 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.db_write_buffer_size: 0 2024-04-07T16:20:06.942 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.write_buffer_manager: 0x55aab5e2e820 2024-04-07T16:20:06.942 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-07T16:20:06.942 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-07T16:20:06.942 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.use_adaptive_mutex: 0 2024-04-07T16:20:06.942 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.rate_limiter: (nil) 2024-04-07T16:20:06.942 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-07T16:20:06.942 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.wal_recovery_mode: 2 2024-04-07T16:20:06.942 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.enable_thread_tracking: 0 2024-04-07T16:20:06.942 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.enable_pipelined_write: 0 2024-04-07T16:20:06.942 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.unordered_write: 0 2024-04-07T16:20:06.942 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-07T16:20:06.942 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-07T16:20:06.943 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-07T16:20:06.943 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-07T16:20:06.943 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.row_cache: None 2024-04-07T16:20:06.943 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.wal_filter: None 2024-04-07T16:20:06.943 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-07T16:20:06.943 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.allow_ingest_behind: 0 2024-04-07T16:20:06.943 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.two_write_queues: 0 2024-04-07T16:20:06.943 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.manual_wal_flush: 0 2024-04-07T16:20:06.943 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.wal_compression: 0 2024-04-07T16:20:06.943 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.atomic_flush: 0 2024-04-07T16:20:06.943 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-07T16:20:06.943 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.persist_stats_to_disk: 0 2024-04-07T16:20:06.943 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.write_dbid_to_manifest: 0 2024-04-07T16:20:06.944 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.log_readahead_size: 0 2024-04-07T16:20:06.944 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-07T16:20:06.944 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.best_efforts_recovery: 0 2024-04-07T16:20:06.944 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-07T16:20:06.944 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-07T16:20:06.944 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.allow_data_in_errors: 0 2024-04-07T16:20:06.944 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.db_host_id: __hostname__ 2024-04-07T16:20:06.944 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.enforce_single_del_contracts: true 2024-04-07T16:20:06.944 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.max_background_jobs: 2 2024-04-07T16:20:06.944 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.max_background_compactions: -1 2024-04-07T16:20:06.944 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.max_subcompactions: 1 2024-04-07T16:20:06.944 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-07T16:20:06.944 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-07T16:20:06.944 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.delayed_write_rate : 16777216 2024-04-07T16:20:06.945 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.max_total_wal_size: 0 2024-04-07T16:20:06.945 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-07T16:20:06.945 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.stats_dump_period_sec: 600 2024-04-07T16:20:06.945 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.stats_persist_period_sec: 600 2024-04-07T16:20:06.945 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-07T16:20:06.945 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.max_open_files: -1 2024-04-07T16:20:06.945 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.bytes_per_sync: 0 2024-04-07T16:20:06.945 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.wal_bytes_per_sync: 0 2024-04-07T16:20:06.945 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.strict_bytes_per_sync: 0 2024-04-07T16:20:06.945 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.compaction_readahead_size: 0 2024-04-07T16:20:06.945 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.max_background_flushes: -1 2024-04-07T16:20:06.945 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Compression algorithms supported: 2024-04-07T16:20:06.945 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-07T16:20:06.945 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: kZSTD supported: 0 2024-04-07T16:20:06.946 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: kXpressCompression supported: 0 2024-04-07T16:20:06.946 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: kLZ4HCCompression supported: 1 2024-04-07T16:20:06.946 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: kZlibCompression supported: 1 2024-04-07T16:20:06.946 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: kSnappyCompression supported: 1 2024-04-07T16:20:06.946 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: kLZ4Compression supported: 1 2024-04-07T16:20:06.946 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: kBZip2Compression supported: 0 2024-04-07T16:20:06.946 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Fast CRC32 supported: Supported on x86 2024-04-07T16:20:06.946 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: DMutex implementation: pthread_mutex_t 2024-04-07T16:20:06.946 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000015 2024-04-07T16:20:06.947 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-07T16:20:06.947 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-07T16:20:06.947 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.merge_operator: 2024-04-07T16:20:06.948 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.compaction_filter: None 2024-04-07T16:20:06.948 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.compaction_filter_factory: None 2024-04-07T16:20:06.948 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.sst_partitioner_factory: None 2024-04-07T16:20:06.948 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.memtable_factory: SkipListFactory 2024-04-07T16:20:06.948 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.table_factory: BlockBasedTable 2024-04-07T16:20:06.948 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55aab5e5dde0) 2024-04-07T16:20:06.948 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: cache_index_and_filter_blocks: 1 2024-04-07T16:20:06.948 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-07T16:20:06.948 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-07T16:20:06.948 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: pin_top_level_index_and_filter: 1 2024-04-07T16:20:06.948 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: index_type: 0 2024-04-07T16:20:06.948 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: data_block_index_type: 0 2024-04-07T16:20:06.948 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: index_shortening: 1 2024-04-07T16:20:06.948 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: data_block_hash_table_util_ratio: 0.750000 2024-04-07T16:20:06.949 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: checksum: 4 2024-04-07T16:20:06.949 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: no_block_cache: 0 2024-04-07T16:20:06.949 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: block_cache: 0x55aab5e5f090 2024-04-07T16:20:06.949 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: block_cache_name: BinnedLRUCache 2024-04-07T16:20:06.949 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: block_cache_options: 2024-04-07T16:20:06.949 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: capacity : 536870912 2024-04-07T16:20:06.949 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: num_shard_bits : 4 2024-04-07T16:20:06.949 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: strict_capacity_limit : 0 2024-04-07T16:20:06.949 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: high_pri_pool_ratio: 0.000 2024-04-07T16:20:06.949 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: block_cache_compressed: (nil) 2024-04-07T16:20:06.949 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: persistent_cache: (nil) 2024-04-07T16:20:06.949 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: block_size: 4096 2024-04-07T16:20:06.949 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: block_size_deviation: 10 2024-04-07T16:20:06.949 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: block_restart_interval: 16 2024-04-07T16:20:06.949 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: index_block_restart_interval: 1 2024-04-07T16:20:06.950 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: metadata_block_size: 4096 2024-04-07T16:20:06.950 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: partition_filters: 0 2024-04-07T16:20:06.950 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: use_delta_encoding: 1 2024-04-07T16:20:06.950 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: filter_policy: bloomfilter 2024-04-07T16:20:06.950 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: whole_key_filtering: 1 2024-04-07T16:20:06.950 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: verify_compression: 0 2024-04-07T16:20:06.950 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: read_amp_bytes_per_bit: 0 2024-04-07T16:20:06.950 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: format_version: 5 2024-04-07T16:20:06.950 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: enable_index_compression: 1 2024-04-07T16:20:06.950 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: block_align: 0 2024-04-07T16:20:06.950 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: max_auto_readahead_size: 262144 2024-04-07T16:20:06.950 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: prepopulate_block_cache: 0 2024-04-07T16:20:06.950 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: initial_auto_readahead_size: 8192 2024-04-07T16:20:06.950 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: num_file_reads_for_auto_readahead: 2 2024-04-07T16:20:06.951 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.write_buffer_size: 33554432 2024-04-07T16:20:06.951 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.max_write_buffer_number: 2 2024-04-07T16:20:06.951 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.compression: NoCompression 2024-04-07T16:20:06.951 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.bottommost_compression: Disabled 2024-04-07T16:20:06.951 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.prefix_extractor: nullptr 2024-04-07T16:20:06.951 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-07T16:20:06.951 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.num_levels: 7 2024-04-07T16:20:06.951 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-07T16:20:06.951 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-07T16:20:06.951 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-07T16:20:06.951 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-07T16:20:06.951 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-07T16:20:06.951 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-07T16:20:06.951 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-07T16:20:06.952 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-07T16:20:06.952 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-07T16:20:06.952 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-07T16:20:06.952 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-07T16:20:06.952 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-07T16:20:06.952 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.compression_opts.window_bits: -14 2024-04-07T16:20:06.952 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.compression_opts.level: 32767 2024-04-07T16:20:06.952 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.compression_opts.strategy: 0 2024-04-07T16:20:06.952 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-07T16:20:06.952 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-07T16:20:06.952 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-07T16:20:06.952 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-07T16:20:06.952 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.compression_opts.enabled: false 2024-04-07T16:20:06.952 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-07T16:20:06.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-07T16:20:06.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-07T16:20:06.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-07T16:20:06.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.target_file_size_base: 67108864 2024-04-07T16:20:06.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.target_file_size_multiplier: 1 2024-04-07T16:20:06.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-07T16:20:06.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-07T16:20:06.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-07T16:20:06.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-07T16:20:06.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-07T16:20:06.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-07T16:20:06.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-07T16:20:06.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-07T16:20:06.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-07T16:20:06.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-07T16:20:06.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-07T16:20:06.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-07T16:20:06.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-07T16:20:06.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.arena_block_size: 1048576 2024-04-07T16:20:06.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-07T16:20:06.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-07T16:20:06.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.disable_auto_compactions: 0 2024-04-07T16:20:06.955 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-07T16:20:06.955 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-07T16:20:06.955 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-07T16:20:06.955 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-07T16:20:06.955 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-07T16:20:06.955 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-07T16:20:06.955 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-07T16:20:06.955 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-07T16:20:06.955 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-07T16:20:06.955 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-07T16:20:06.955 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-07T16:20:06.955 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.inplace_update_support: 0 2024-04-07T16:20:06.955 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.inplace_update_num_locks: 10000 2024-04-07T16:20:06.955 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-07T16:20:06.955 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-07T16:20:06.955 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.memtable_huge_page_size: 0 2024-04-07T16:20:06.956 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.bloom_locality: 0 2024-04-07T16:20:06.956 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.max_successive_merges: 0 2024-04-07T16:20:06.956 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.optimize_filters_for_hits: 0 2024-04-07T16:20:06.956 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.paranoid_file_checks: 0 2024-04-07T16:20:06.956 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.force_consistency_checks: 1 2024-04-07T16:20:06.956 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.report_bg_io_stats: 0 2024-04-07T16:20:06.956 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.ttl: 2592000 2024-04-07T16:20:06.956 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.periodic_compaction_seconds: 0 2024-04-07T16:20:06.956 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-07T16:20:06.956 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-07T16:20:06.956 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.enable_blob_files: false 2024-04-07T16:20:06.956 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.min_blob_size: 0 2024-04-07T16:20:06.956 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.blob_file_size: 268435456 2024-04-07T16:20:06.956 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.blob_compression_type: NoCompression 2024-04-07T16:20:06.956 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.enable_blob_garbage_collection: false 2024-04-07T16:20:06.957 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-07T16:20:06.957 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-07T16:20:06.957 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-07T16:20:06.957 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.blob_file_starting_level: 0 2024-04-07T16:20:06.957 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.706+0000 7f53c05a7c80 4 rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-07T16:20:06.957 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.710+0000 7f53c05a7c80 4 rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-04-07T16:20:06.957 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.710+0000 7f53c05a7c80 4 rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000015 succeeded,manifest_file_number is 15, next_file_number is 37, last_sequence is 17498, log_number is 33,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 33 2024-04-07T16:20:06.957 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.710+0000 7f53c05a7c80 4 rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 33 2024-04-07T16:20:06.957 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.710+0000 7f53c05a7c80 4 rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 8e6083b8-6eb2-43a6-936e-e3f8952735eb 2024-04-07T16:20:06.957 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.710+0000 7f53c05a7c80 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1712506806712138, "job": 1, "event": "recovery_started", "wal_files": [33]} 2024-04-07T16:20:06.957 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.710+0000 7f53c05a7c80 4 rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #33 mode 2 2024-04-07T16:20:06.957 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.714+0000 7f53c05a7c80 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1712506806717596, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 38, "file_size": 1162265, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 17499, "largest_seqno": 18255, "table_properties": {"data_size": 1157719, "index_size": 2610, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 901, "raw_key_size": 8736, "raw_average_key_size": 25, "raw_value_size": 1149820, "raw_average_value_size": 3352, "num_data_blocks": 115, "num_entries": 343, "num_filter_entries": 343, "num_deletions": 21, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1712506806, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "8e6083b8-6eb2-43a6-936e-e3f8952735eb", "db_session_id": "BP3OKQ97D18MY4CGTGPW", "orig_file_number": 38, "seqno_to_time_mapping": "N/A"}} 2024-04-07T16:20:06.957 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.714+0000 7f53c05a7c80 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1712506806717725, "job": 1, "event": "recovery_finished"} 2024-04-07T16:20:06.957 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.714+0000 7f53c05a7c80 4 rocksdb: [db/version_set.cc:5047] Creating manifest 40 2024-04-07T16:20:06.958 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.714+0000 7f53c05a7c80 4 rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-04-07T16:20:06.958 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.714+0000 7f53c05a7c80 4 rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000033.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-07T16:20:06.958 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.714+0000 7f53c05a7c80 4 rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55aab5f52000 2024-04-07T16:20:06.958 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.714+0000 7f53c05a7c80 4 rocksdb: DB pointer 0x55aab5f3c000 2024-04-07T16:20:06.958 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.718+0000 7f53aed01700 4 rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-07T16:20:06.958 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.718+0000 7f53aed01700 4 rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-07T16:20:06.958 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: ** DB Stats ** 2024-04-07T16:20:06.958 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: Uptime(secs): 0.0 total, 0.0 interval 2024-04-07T16:20:06.958 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-07T16:20:06.958 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-07T16:20:06.958 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-07T16:20:06.958 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-07T16:20:06.958 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-07T16:20:06.958 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-07T16:20:06.958 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: ** Compaction Stats [default] ** 2024-04-07T16:20:06.959 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-07T16:20:06.959 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-07T16:20:06.959 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: L0 1/0 1.11 MB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 369.0 0.00 0.00 1 0.003 0 0 0.0 0.0 2024-04-07T16:20:06.959 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: L6 1/0 10.86 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 2024-04-07T16:20:06.959 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: Sum 2/0 11.97 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 369.0 0.00 0.00 1 0.003 0 0 0.0 0.0 2024-04-07T16:20:06.959 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 369.0 0.00 0.00 1 0.003 0 0 0.0 0.0 2024-04-07T16:20:06.959 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: ** Compaction Stats [default] ** 2024-04-07T16:20:06.959 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-07T16:20:06.959 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-07T16:20:06.959 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 369.0 0.00 0.00 1 0.003 0 0 0.0 0.0 2024-04-07T16:20:06.959 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-07T16:20:06.959 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: Uptime(secs): 0.0 total, 0.0 interval 2024-04-07T16:20:06.959 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: Flush(GB): cumulative 0.001, interval 0.001 2024-04-07T16:20:06.959 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-07T16:20:06.959 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: AddFile(Total Files): cumulative 0, interval 0 2024-04-07T16:20:06.960 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: AddFile(L0 Files): cumulative 0, interval 0 2024-04-07T16:20:06.960 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: AddFile(Keys): cumulative 0, interval 0 2024-04-07T16:20:06.960 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: Cumulative compaction: 0.00 GB write, 131.28 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-07T16:20:06.960 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: Interval compaction: 0.00 GB write, 131.28 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-07T16:20:06.960 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-07T16:20:06.960 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: Block cache BinnedLRUCache@0x55aab5e5f090#7 capacity: 512.00 MB usage: 3.86 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.2e-05 secs_since: 0 2024-04-07T16:20:06.960 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: Block cache entry stats(count,size,portion): FilterBlock(1,1.03 KB,0.000196695%) IndexBlock(1,2.83 KB,0.000539422%) Misc(1,0.00 KB,0%) 2024-04-07T16:20:06.960 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: ** File Read Latency Histogram By Level [default] ** 2024-04-07T16:20:06.960 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.718+0000 7f53c05a7c80 0 starting mon.a rank 0 at public addrs [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] at bind addrs [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 0702e3a8-f4f8-11ee-b649-cb9ed24678a4 2024-04-07T16:20:06.960 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.718+0000 7f53c05a7c80 1 mon.a@-1(???) e3 preinit fsid 0702e3a8-f4f8-11ee-b649-cb9ed24678a4 2024-04-07T16:20:06.960 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.718+0000 7f53c05a7c80 0 mon.a@-1(???).mds e6 new map 2024-04-07T16:20:06.960 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.718+0000 7f53c05a7c80 0 mon.a@-1(???).mds e6 print_map 2024-04-07T16:20:06.960 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: e6 2024-04-07T16:20:06.960 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: enable_multiple, ever_enabled_multiple: 1,1 2024-04-07T16:20:06.960 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-07T16:20:06.960 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: legacy client fscid: 1 2024-04-07T16:20:06.961 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: 2024-04-07T16:20:06.961 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: Filesystem 'cephfs' (1) 2024-04-07T16:20:06.961 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: fs_name cephfs 2024-04-07T16:20:06.961 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: epoch 6 2024-04-07T16:20:06.961 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-07T16:20:06.961 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: created 2024-04-07T16:12:13.365189+0000 2024-04-07T16:20:06.961 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: modified 2024-04-07T16:12:15.375075+0000 2024-04-07T16:20:06.961 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: tableserver 0 2024-04-07T16:20:06.961 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: root 0 2024-04-07T16:20:06.961 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: session_timeout 60 2024-04-07T16:20:06.961 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: session_autoclose 300 2024-04-07T16:20:06.961 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: max_file_size 1099511627776 2024-04-07T16:20:06.961 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: max_xattr_size 65536 2024-04-07T16:20:06.961 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: required_client_features {} 2024-04-07T16:20:06.961 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: last_failure 0 2024-04-07T16:20:06.962 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: last_failure_osd_epoch 0 2024-04-07T16:20:06.962 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-07T16:20:06.962 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: max_mds 1 2024-04-07T16:20:06.962 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: in 0 2024-04-07T16:20:06.962 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: up {0=24322} 2024-04-07T16:20:06.962 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: failed 2024-04-07T16:20:06.962 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: damaged 2024-04-07T16:20:06.962 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: stopped 2024-04-07T16:20:06.962 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: data_pools [3] 2024-04-07T16:20:06.962 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: metadata_pool 2 2024-04-07T16:20:06.962 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: inline_data disabled 2024-04-07T16:20:06.962 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: balancer 2024-04-07T16:20:06.962 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: bal_rank_mask -1 2024-04-07T16:20:06.962 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: standby_count_wanted 0 2024-04-07T16:20:06.963 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: qdb_cluster leader: 0 members: 2024-04-07T16:20:06.963 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: [mds.b{0:24322} state up:active seq 3 addr [v2:172.21.15.144:6832/226244023,v1:172.21.15.144:6833/226244023] compat {c=[1],r=[1],i=[7ff]}] 2024-04-07T16:20:06.963 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: 2024-04-07T16:20:06.963 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: 2024-04-07T16:20:06.963 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: Standby daemons: 2024-04-07T16:20:06.963 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: 2024-04-07T16:20:06.963 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: [mds.a{-1:14424} state up:standby seq 1 addr [v2:172.21.15.100:6834/3487006602,v1:172.21.15.100:6835/3487006602] compat {c=[1],r=[1],i=[7ff]}] 2024-04-07T16:20:06.963 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.718+0000 7f53c05a7c80 0 mon.a@-1(???).osd e270 crush map has features 3314933069573799936, adjusting msgr requires 2024-04-07T16:20:06.963 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.718+0000 7f53c05a7c80 0 mon.a@-1(???).osd e270 crush map has features 432629308056666112, adjusting msgr requires 2024-04-07T16:20:06.963 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.718+0000 7f53c05a7c80 0 mon.a@-1(???).osd e270 crush map has features 432629308056666112, adjusting msgr requires 2024-04-07T16:20:06.963 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.718+0000 7f53c05a7c80 0 mon.a@-1(???).osd e270 crush map has features 432629308056666112, adjusting msgr requires 2024-04-07T16:20:06.963 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:06 smithi100 bash[74942]: debug 2024-04-07T16:20:06.718+0000 7f53c05a7c80 1 mon.a@-1(???).paxosservice(auth 1..20) refresh upgraded, format 0 -> 3 2024-04-07T16:20:08.248 INFO:tasks.rados.rados.0.smithi100.stdout:1113: done (3 left) 2024-04-07T16:20:08.248 INFO:tasks.rados.rados.0.smithi100.stdout:1114: snap_create 2024-04-07T16:20:08.258 INFO:tasks.rados.rados.0.smithi100.stdout:1110: finishing write tid 1 to smithi10066867-14 2024-04-07T16:20:08.258 INFO:tasks.rados.rados.0.smithi100.stdout:1110: finishing write tid 2 to smithi10066867-14 2024-04-07T16:20:08.258 INFO:tasks.rados.rados.0.smithi100.stdout:1110: finishing write tid 3 to smithi10066867-14 2024-04-07T16:20:08.258 INFO:tasks.rados.rados.0.smithi100.stdout:1110: finishing write tid 4 to smithi10066867-14 2024-04-07T16:20:08.258 INFO:tasks.rados.rados.0.smithi100.stdout:1110: oid 14 updating version 0 to 519 2024-04-07T16:20:08.258 INFO:tasks.rados.rados.0.smithi100.stdout:1110: oid 14 version 519 is already newer than 518 2024-04-07T16:20:08.259 INFO:tasks.rados.rados.0.smithi100.stdout:1110: oid 14 updating version 519 to 520 2024-04-07T16:20:08.259 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 14 v 520 (ObjNum 342 snap 101 seq_num 342) dirty exists 2024-04-07T16:20:08.259 INFO:tasks.rados.rados.0.smithi100.stdout:1110: left oid 14 (ObjNum 342 snap 101 seq_num 342) 2024-04-07T16:20:08.259 INFO:tasks.rados.rados.0.smithi100.stdout:1109: expect (ObjNum 262 snap 70 seq_num 262) 2024-04-07T16:20:08.378 INFO:tasks.rados.rados.0.smithi100.stdout:1112: expect (ObjNum 171 snap 44 seq_num 171) 2024-04-07T16:20:08.605 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:20:08 smithi144 bash[60191]: ignoring --setuser ceph since I am not root 2024-04-07T16:20:08.605 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:20:08 smithi144 bash[60191]: ignoring --setgroup ceph since I am not root 2024-04-07T16:20:08.605 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:20:08 smithi144 bash[60191]: debug 2024-04-07T16:20:08.279+0000 7fe2ad2b9700 1 -- 172.21.15.144:0/121742493 <== mon.2 v2:172.21.15.100:3301/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5634fc33c5a0 con 0x5634fd0bac00 2024-04-07T16:20:08.606 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:20:08 smithi144 bash[60191]: debug 2024-04-07T16:20:08.427+0000 7fe2b70ab200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-07T16:20:08.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:08 smithi144 bash[23522]: cluster 2024-04-07T16:20:06.026116+0000 mgr.y (mgr.25073) 52 : cluster 0 pgmap v78: 121 pgs: 121 active+clean; 251 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 6.1 MiB/s rd, 1.0 MiB/s wr, 9 op/s 2024-04-07T16:20:08.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:08 smithi144 bash[23522]: cluster 2024-04-07T16:20:08.141478+0000 mon.a (mon.0) 1 : cluster 1 mon.a calling monitor election 2024-04-07T16:20:08.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:08 smithi144 bash[23522]: cluster 2024-04-07T16:20:08.165657+0000 mon.a (mon.0) 2 : cluster 1 mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-04-07T16:20:08.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:08 smithi144 bash[23522]: cluster 2024-04-07T16:20:08.177267+0000 mon.a (mon.0) 3 : cluster 0 monmap epoch 3 2024-04-07T16:20:08.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:08 smithi144 bash[23522]: cluster 2024-04-07T16:20:08.177341+0000 mon.a (mon.0) 4 : cluster 0 fsid 0702e3a8-f4f8-11ee-b649-cb9ed24678a4 2024-04-07T16:20:08.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:08 smithi144 bash[23522]: cluster 2024-04-07T16:20:08.177377+0000 mon.a (mon.0) 5 : cluster 0 last_changed 2024-04-07T16:04:34.801459+0000 2024-04-07T16:20:08.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:08 smithi144 bash[23522]: cluster 2024-04-07T16:20:08.177422+0000 mon.a (mon.0) 6 : cluster 0 created 2024-04-07T16:02:17.280630+0000 2024-04-07T16:20:08.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:08 smithi144 bash[23522]: cluster 2024-04-07T16:20:08.177463+0000 mon.a (mon.0) 7 : cluster 0 min_mon_release 18 (reef) 2024-04-07T16:20:08.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:08 smithi144 bash[23522]: cluster 2024-04-07T16:20:08.177508+0000 mon.a (mon.0) 8 : cluster 0 election_strategy: 1 2024-04-07T16:20:08.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:08 smithi144 bash[23522]: cluster 2024-04-07T16:20:08.177544+0000 mon.a (mon.0) 9 : cluster 0 0: [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] mon.a 2024-04-07T16:20:08.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:08 smithi144 bash[23522]: cluster 2024-04-07T16:20:08.177582+0000 mon.a (mon.0) 10 : cluster 0 1: [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] mon.b 2024-04-07T16:20:08.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:08 smithi144 bash[23522]: cluster 2024-04-07T16:20:08.177629+0000 mon.a (mon.0) 11 : cluster 0 2: [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] mon.c 2024-04-07T16:20:08.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:08 smithi144 bash[23522]: cluster 2024-04-07T16:20:08.178888+0000 mon.a (mon.0) 12 : cluster 0 fsmap cephfs:1 {0=b=up:active} 1 up:standby 2024-04-07T16:20:08.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:08 smithi144 bash[23522]: cluster 2024-04-07T16:20:08.178984+0000 mon.a (mon.0) 13 : cluster 0 osdmap e270: 8 total, 8 up, 8 in 2024-04-07T16:20:08.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:08 smithi144 bash[23522]: cluster 2024-04-07T16:20:08.181291+0000 mon.a (mon.0) 14 : cluster 0 mgrmap e35: y(active, since 54s), standbys: x 2024-04-07T16:20:08.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:08 smithi144 bash[23522]: cluster 2024-04-07T16:20:08.182420+0000 mon.a (mon.0) 15 : cluster 1 overall HEALTH_OK 2024-04-07T16:20:08.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:08 smithi144 bash[23522]: audit 2024-04-07T16:20:08.214896+0000 mon.a (mon.0) 16 : audit 1 from='mgr.25073 ' entity='' 2024-04-07T16:20:08.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:08 smithi144 bash[23522]: cluster 2024-04-07T16:20:08.216815+0000 mon.a (mon.0) 17 : cluster 0 mgrmap e36: y(active, since 54s), standbys: x 2024-04-07T16:20:08.686 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:20:08 smithi100 bash[68137]: ignoring --setuser ceph since I am not root 2024-04-07T16:20:08.686 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:20:08 smithi100 bash[68137]: ignoring --setgroup ceph since I am not root 2024-04-07T16:20:08.686 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:20:08 smithi100 bash[68137]: debug 2024-04-07T16:20:08.294+0000 7f8540625700 1 -- 172.21.15.100:0/1250808586 <== mon.2 v2:172.21.15.100:3301/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557b700645a0 con 0x557b70de2800 2024-04-07T16:20:08.686 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:20:08 smithi100 bash[68137]: debug 2024-04-07T16:20:08.438+0000 7f854a417200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-07T16:20:08.687 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:08 smithi100 bash[74942]: cluster 2024-04-07T16:20:06.026116+0000 mgr.y (mgr.25073) 52 : cluster [DBG] pgmap v78: 121 pgs: 121 active+clean; 251 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 6.1 MiB/s rd, 1.0 MiB/s wr, 9 op/s 2024-04-07T16:20:08.687 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:08 smithi100 bash[74942]: cluster 2024-04-07T16:20:08.141478+0000 mon.a (mon.0) 1 : cluster [INF] mon.a calling monitor election 2024-04-07T16:20:08.687 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:08 smithi100 bash[74942]: cluster 2024-04-07T16:20:08.165657+0000 mon.a (mon.0) 2 : cluster [INF] mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-04-07T16:20:08.687 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:08 smithi100 bash[74942]: cluster 2024-04-07T16:20:08.177267+0000 mon.a (mon.0) 3 : cluster [DBG] monmap epoch 3 2024-04-07T16:20:08.687 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:08 smithi100 bash[74942]: cluster 2024-04-07T16:20:08.177341+0000 mon.a (mon.0) 4 : cluster [DBG] fsid 0702e3a8-f4f8-11ee-b649-cb9ed24678a4 2024-04-07T16:20:08.687 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:08 smithi100 bash[74942]: cluster 2024-04-07T16:20:08.177377+0000 mon.a (mon.0) 5 : cluster [DBG] last_changed 2024-04-07T16:04:34.801459+0000 2024-04-07T16:20:08.687 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:08 smithi100 bash[74942]: cluster 2024-04-07T16:20:08.177422+0000 mon.a (mon.0) 6 : cluster [DBG] created 2024-04-07T16:02:17.280630+0000 2024-04-07T16:20:08.687 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:08 smithi100 bash[74942]: cluster 2024-04-07T16:20:08.177463+0000 mon.a (mon.0) 7 : cluster [DBG] min_mon_release 18 (reef) 2024-04-07T16:20:08.687 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:08 smithi100 bash[74942]: cluster 2024-04-07T16:20:08.177508+0000 mon.a (mon.0) 8 : cluster [DBG] election_strategy: 1 2024-04-07T16:20:08.687 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:08 smithi100 bash[74942]: cluster 2024-04-07T16:20:08.177544+0000 mon.a (mon.0) 9 : cluster [DBG] 0: [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] mon.a 2024-04-07T16:20:08.687 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:08 smithi100 bash[74942]: cluster 2024-04-07T16:20:08.177582+0000 mon.a (mon.0) 10 : cluster [DBG] 1: [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] mon.b 2024-04-07T16:20:08.688 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:08 smithi100 bash[74942]: cluster 2024-04-07T16:20:08.177629+0000 mon.a (mon.0) 11 : cluster [DBG] 2: [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] mon.c 2024-04-07T16:20:08.688 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:08 smithi100 bash[74942]: cluster 2024-04-07T16:20:08.178888+0000 mon.a (mon.0) 12 : cluster [DBG] fsmap cephfs:1 {0=b=up:active} 1 up:standby 2024-04-07T16:20:08.688 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:08 smithi100 bash[74942]: cluster 2024-04-07T16:20:08.178984+0000 mon.a (mon.0) 13 : cluster [DBG] osdmap e270: 8 total, 8 up, 8 in 2024-04-07T16:20:08.688 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:08 smithi100 bash[74942]: cluster 2024-04-07T16:20:08.181291+0000 mon.a (mon.0) 14 : cluster [DBG] mgrmap e35: y(active, since 54s), standbys: x 2024-04-07T16:20:08.688 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:08 smithi100 bash[74942]: cluster 2024-04-07T16:20:08.182420+0000 mon.a (mon.0) 15 : cluster [INF] overall HEALTH_OK 2024-04-07T16:20:08.688 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:08 smithi100 bash[74942]: audit 2024-04-07T16:20:08.214896+0000 mon.a (mon.0) 16 : audit [INF] from='mgr.25073 ' entity='' 2024-04-07T16:20:08.688 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:08 smithi100 bash[74942]: cluster 2024-04-07T16:20:08.216815+0000 mon.a (mon.0) 17 : cluster [DBG] mgrmap e36: y(active, since 54s), standbys: x 2024-04-07T16:20:08.688 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:08 smithi100 bash[31660]: cluster 2024-04-07T16:20:06.026116+0000 mgr.y (mgr.25073) 52 : cluster 0 pgmap v78: 121 pgs: 121 active+clean; 251 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 6.1 MiB/s rd, 1.0 MiB/s wr, 9 op/s 2024-04-07T16:20:08.688 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:08 smithi100 bash[31660]: cluster 2024-04-07T16:20:08.141478+0000 mon.a (mon.0) 1 : cluster 1 mon.a calling monitor election 2024-04-07T16:20:08.688 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:08 smithi100 bash[31660]: cluster 2024-04-07T16:20:08.165657+0000 mon.a (mon.0) 2 : cluster 1 mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-04-07T16:20:08.688 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:08 smithi100 bash[31660]: cluster 2024-04-07T16:20:08.177267+0000 mon.a (mon.0) 3 : cluster 0 monmap epoch 3 2024-04-07T16:20:08.688 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:08 smithi100 bash[31660]: cluster 2024-04-07T16:20:08.177341+0000 mon.a (mon.0) 4 : cluster 0 fsid 0702e3a8-f4f8-11ee-b649-cb9ed24678a4 2024-04-07T16:20:08.689 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:08 smithi100 bash[31660]: cluster 2024-04-07T16:20:08.177377+0000 mon.a (mon.0) 5 : cluster 0 last_changed 2024-04-07T16:04:34.801459+0000 2024-04-07T16:20:08.689 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:08 smithi100 bash[31660]: cluster 2024-04-07T16:20:08.177422+0000 mon.a (mon.0) 6 : cluster 0 created 2024-04-07T16:02:17.280630+0000 2024-04-07T16:20:08.689 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:08 smithi100 bash[31660]: cluster 2024-04-07T16:20:08.177463+0000 mon.a (mon.0) 7 : cluster 0 min_mon_release 18 (reef) 2024-04-07T16:20:08.689 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:08 smithi100 bash[31660]: cluster 2024-04-07T16:20:08.177508+0000 mon.a (mon.0) 8 : cluster 0 election_strategy: 1 2024-04-07T16:20:08.689 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:08 smithi100 bash[31660]: cluster 2024-04-07T16:20:08.177544+0000 mon.a (mon.0) 9 : cluster 0 0: [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] mon.a 2024-04-07T16:20:08.689 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:08 smithi100 bash[31660]: cluster 2024-04-07T16:20:08.177582+0000 mon.a (mon.0) 10 : cluster 0 1: [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] mon.b 2024-04-07T16:20:08.689 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:08 smithi100 bash[31660]: cluster 2024-04-07T16:20:08.177629+0000 mon.a (mon.0) 11 : cluster 0 2: [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] mon.c 2024-04-07T16:20:08.689 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:08 smithi100 bash[31660]: cluster 2024-04-07T16:20:08.178888+0000 mon.a (mon.0) 12 : cluster 0 fsmap cephfs:1 {0=b=up:active} 1 up:standby 2024-04-07T16:20:08.689 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:08 smithi100 bash[31660]: cluster 2024-04-07T16:20:08.178984+0000 mon.a (mon.0) 13 : cluster 0 osdmap e270: 8 total, 8 up, 8 in 2024-04-07T16:20:08.689 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:08 smithi100 bash[31660]: cluster 2024-04-07T16:20:08.181291+0000 mon.a (mon.0) 14 : cluster 0 mgrmap e35: y(active, since 54s), standbys: x 2024-04-07T16:20:08.689 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:08 smithi100 bash[31660]: cluster 2024-04-07T16:20:08.182420+0000 mon.a (mon.0) 15 : cluster 1 overall HEALTH_OK 2024-04-07T16:20:08.689 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:08 smithi100 bash[31660]: audit 2024-04-07T16:20:08.214896+0000 mon.a (mon.0) 16 : audit 1 from='mgr.25073 ' entity='' 2024-04-07T16:20:08.689 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:08 smithi100 bash[31660]: cluster 2024-04-07T16:20:08.216815+0000 mon.a (mon.0) 17 : cluster 0 mgrmap e36: y(active, since 54s), standbys: x 2024-04-07T16:20:09.241 INFO:tasks.rados.rados.0.smithi100.stdout:1109: done (3 left) 2024-04-07T16:20:09.241 INFO:tasks.rados.rados.0.smithi100.stdout:1110: done (2 left) 2024-04-07T16:20:09.241 INFO:tasks.rados.rados.0.smithi100.stdout:1112: done (1 left) 2024-04-07T16:20:09.242 INFO:tasks.rados.rados.0.smithi100.stdout:1114: done (0 left) 2024-04-07T16:20:09.242 INFO:tasks.rados.rados.0.smithi100.stdout:1115: copy_from oid 23 from oid 38 current snap is 102 2024-04-07T16:20:09.242 INFO:tasks.rados.rados.0.smithi100.stdout:1116: snap_remove snap 96 2024-04-07T16:20:09.605 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:09 smithi144 bash[23522]: cluster 2024-04-07T16:20:08.248564+0000 mon.a (mon.0) 18 : cluster 0 osdmap e271: 8 total, 8 up, 8 in 2024-04-07T16:20:09.686 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:09 smithi100 bash[74942]: cluster 2024-04-07T16:20:08.248564+0000 mon.a (mon.0) 18 : cluster [DBG] osdmap e271: 8 total, 8 up, 8 in 2024-04-07T16:20:09.686 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:09 smithi100 bash[31660]: cluster 2024-04-07T16:20:08.248564+0000 mon.a (mon.0) 18 : cluster 0 osdmap e271: 8 total, 8 up, 8 in 2024-04-07T16:20:10.248 INFO:tasks.rados.rados.0.smithi100.stdout:1116: done (1 left) 2024-04-07T16:20:10.249 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 12 current snap is 102 2024-04-07T16:20:10.249 INFO:tasks.rados.rados.0.smithi100.stdout:1117: seq_num 344 ranges {4038656=565248,4603904=401408} 2024-04-07T16:20:10.252 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:20:09 smithi144 bash[60191]: debug 2024-04-07T16:20:09.995+0000 7fe2b70ab200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-07T16:20:10.253 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:20:10 smithi144 bash[60191]: debug 2024-04-07T16:20:10.095+0000 7fe2b70ab200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-07T16:20:10.278 INFO:tasks.rados.rados.0.smithi100.stdout:1117: writing smithi10066867-12 from 4038656 to 4603904 tid 1 2024-04-07T16:20:10.281 INFO:tasks.rados.rados.0.smithi100.stdout:1117: writing smithi10066867-12 from 4603904 to 5005312 tid 2 2024-04-07T16:20:10.281 INFO:tasks.rados.rados.0.smithi100.stdout:1118: copy_from oid 44 from oid 4 current snap is 102 2024-04-07T16:20:10.281 INFO:tasks.rados.rados.0.smithi100.stdout:1115: finishing copy_from racing read to smithi10066867-23 2024-04-07T16:20:10.281 INFO:tasks.rados.rados.0.smithi100.stdout:1115: finishing copy_from to smithi10066867-23 2024-04-07T16:20:10.281 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 23 v 242 (ObjNum 340 snap 101 seq_num 340) dirty exists 2024-04-07T16:20:10.281 INFO:tasks.rados.rados.0.smithi100.stdout:1115: done (2 left) 2024-04-07T16:20:10.281 INFO:tasks.rados.rados.0.smithi100.stdout:1119: rmattr oid 15 current snap is 102 2024-04-07T16:20:10.281 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 5 current snap is 102 2024-04-07T16:20:10.281 INFO:tasks.rados.rados.0.smithi100.stdout:1120: seq_num 345 ranges {1097728=581632,1679360=401408} 2024-04-07T16:20:10.291 INFO:tasks.rados.rados.0.smithi100.stdout:1120: writing smithi10066867-5 from 1097728 to 1679360 tid 1 2024-04-07T16:20:10.294 INFO:tasks.rados.rados.0.smithi100.stdout:1120: writing smithi10066867-5 from 1679360 to 2080768 tid 2 2024-04-07T16:20:10.294 INFO:tasks.rados.rados.0.smithi100.stdout:1121: snap_create 2024-04-07T16:20:10.294 INFO:tasks.rados.rados.0.smithi100.stdout:1117: finishing write tid 1 to smithi10066867-12 2024-04-07T16:20:10.294 INFO:tasks.rados.rados.0.smithi100.stdout:1117: finishing write tid 2 to smithi10066867-12 2024-04-07T16:20:10.294 INFO:tasks.rados.rados.0.smithi100.stdout:1117: finishing write tid 3 to smithi10066867-12 2024-04-07T16:20:10.294 INFO:tasks.rados.rados.0.smithi100.stdout:1117: finishing write tid 4 to smithi10066867-12 2024-04-07T16:20:10.294 INFO:tasks.rados.rados.0.smithi100.stdout:1117: oid 12 updating version 0 to 565 2024-04-07T16:20:10.295 INFO:tasks.rados.rados.0.smithi100.stdout:1117: oid 12 version 565 is already newer than 563 2024-04-07T16:20:10.295 INFO:tasks.rados.rados.0.smithi100.stdout:1117: oid 12 version 565 is already newer than 564 2024-04-07T16:20:10.295 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 12 v 565 (ObjNum 344 snap 102 seq_num 344) dirty exists 2024-04-07T16:20:10.295 INFO:tasks.rados.rados.0.smithi100.stdout:1117: left oid 12 (ObjNum 344 snap 102 seq_num 344) 2024-04-07T16:20:10.295 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 15 v 699 (ObjNum 338 snap 101 seq_num 338) dirty exists 2024-04-07T16:20:10.298 INFO:tasks.rados.rados.0.smithi100.stdout:1120: finishing write tid 1 to smithi10066867-5 2024-04-07T16:20:10.298 INFO:tasks.rados.rados.0.smithi100.stdout:1120: finishing write tid 2 to smithi10066867-5 2024-04-07T16:20:10.298 INFO:tasks.rados.rados.0.smithi100.stdout:1118: finishing copy_from racing read to smithi10066867-44 2024-04-07T16:20:10.299 INFO:tasks.rados.rados.0.smithi100.stdout:1120: finishing write tid 3 to smithi10066867-5 2024-04-07T16:20:10.301 INFO:tasks.rados.rados.0.smithi100.stdout:1120: finishing write tid 4 to smithi10066867-5 2024-04-07T16:20:10.301 INFO:tasks.rados.rados.0.smithi100.stdout:1120: oid 5 updating version 0 to 568 2024-04-07T16:20:10.301 INFO:tasks.rados.rados.0.smithi100.stdout:1120: oid 5 updating version 568 to 569 2024-04-07T16:20:10.301 INFO:tasks.rados.rados.0.smithi100.stdout:1120: oid 5 version 569 is already newer than 567 2024-04-07T16:20:10.301 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 5 v 569 (ObjNum 345 snap 102 seq_num 345) dirty exists 2024-04-07T16:20:10.301 INFO:tasks.rados.rados.0.smithi100.stdout:1120: left oid 5 (ObjNum 345 snap 102 seq_num 345) 2024-04-07T16:20:10.301 INFO:tasks.rados.rados.0.smithi100.stdout:1118: finishing copy_from to smithi10066867-44 2024-04-07T16:20:10.301 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 44 v 636 (ObjNum 171 snap 44 seq_num 171) dirty exists 2024-04-07T16:20:10.373 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:10 smithi100 bash[31660]: cluster 2024-04-07T16:20:09.255715+0000 mon.a (mon.0) 19 : cluster 0 osdmap e272: 8 total, 8 up, 8 in 2024-04-07T16:20:10.373 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:10 smithi100 bash[74942]: cluster 2024-04-07T16:20:09.255715+0000 mon.a (mon.0) 19 : cluster [DBG] osdmap e272: 8 total, 8 up, 8 in 2024-04-07T16:20:10.373 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:20:10 smithi100 bash[68137]: debug 2024-04-07T16:20:10.018+0000 7f854a417200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-07T16:20:10.374 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:20:10 smithi100 bash[68137]: debug 2024-04-07T16:20:10.114+0000 7f854a417200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-07T16:20:10.605 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:20:10 smithi144 bash[60191]: debug 2024-04-07T16:20:10.363+0000 7fe2b70ab200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-07T16:20:10.605 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:10 smithi144 bash[23522]: cluster 2024-04-07T16:20:09.255715+0000 mon.a (mon.0) 19 : cluster 0 osdmap e272: 8 total, 8 up, 8 in 2024-04-07T16:20:10.686 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:20:10 smithi100 bash[68137]: debug 2024-04-07T16:20:10.386+0000 7f854a417200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-07T16:20:11.264 INFO:tasks.rados.rados.0.smithi100.stdout:1117: done (4 left) 2024-04-07T16:20:11.265 INFO:tasks.rados.rados.0.smithi100.stdout:1118: done (3 left) 2024-04-07T16:20:11.265 INFO:tasks.rados.rados.0.smithi100.stdout:1119: done (2 left) 2024-04-07T16:20:11.265 INFO:tasks.rados.rados.0.smithi100.stdout:1120: done (1 left) 2024-04-07T16:20:11.265 INFO:tasks.rados.rados.0.smithi100.stdout:1121: done (0 left) 2024-04-07T16:20:11.265 INFO:tasks.rados.rados.0.smithi100.stdout:1122: rollback oid 26 current snap is 103 2024-04-07T16:20:11.265 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 26 to 92 2024-04-07T16:20:11.265 INFO:tasks.rados.rados.0.smithi100.stdout:1123: copy_from oid 48 from oid 14 current snap is 103 2024-04-07T16:20:11.265 INFO:tasks.rados.rados.0.smithi100.stdout:1124: snap_create 2024-04-07T16:20:11.274 INFO:tasks.rados.rados.0.smithi100.stdout:1122: finishing rollback tid 0 to smithi10066867-26 2024-04-07T16:20:11.275 INFO:tasks.rados.rados.0.smithi100.stdout:1122: finishing rollback tid 1 to smithi10066867-26 2024-04-07T16:20:11.275 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 26 v 514 (ObjNum 161 snap 0 seq_num 1) dirty dne 2024-04-07T16:20:11.279 INFO:tasks.rados.rados.0.smithi100.stdout:1123: finishing copy_from racing read to smithi10066867-48 2024-04-07T16:20:11.280 INFO:tasks.rados.rados.0.smithi100.stdout:1123: finishing copy_from to smithi10066867-48 2024-04-07T16:20:11.281 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 48 v 520 (ObjNum 342 snap 101 seq_num 342) dirty exists 2024-04-07T16:20:11.436 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:20:11 smithi100 bash[68137]: debug 2024-04-07T16:20:11.182+0000 7f854a417200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-07T16:20:11.436 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:20:11 smithi100 bash[68137]: debug 2024-04-07T16:20:11.302+0000 7f854a417200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-07T16:20:11.436 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:20:11 smithi100 bash[68137]: debug 2024-04-07T16:20:11.410+0000 7f854a417200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-07T16:20:11.436 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:11 smithi100 bash[74942]: cluster 2024-04-07T16:20:10.259384+0000 mon.a (mon.0) 20 : cluster [DBG] osdmap e273: 8 total, 8 up, 8 in 2024-04-07T16:20:11.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:11 smithi100 bash[31660]: cluster 2024-04-07T16:20:10.259384+0000 mon.a (mon.0) 20 : cluster 0 osdmap e273: 8 total, 8 up, 8 in 2024-04-07T16:20:11.605 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:11 smithi144 bash[23522]: cluster 2024-04-07T16:20:10.259384+0000 mon.a (mon.0) 20 : cluster 0 osdmap e273: 8 total, 8 up, 8 in 2024-04-07T16:20:11.605 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:20:11 smithi144 bash[60191]: debug 2024-04-07T16:20:11.159+0000 7fe2b70ab200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-07T16:20:11.606 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:20:11 smithi144 bash[60191]: debug 2024-04-07T16:20:11.275+0000 7fe2b70ab200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-07T16:20:11.606 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:20:11 smithi144 bash[60191]: debug 2024-04-07T16:20:11.387+0000 7fe2b70ab200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-07T16:20:11.721 INFO:tasks.rados.rados.0.smithi100.stdout:1122: done (2 left) 2024-04-07T16:20:11.721 INFO:tasks.rados.rados.0.smithi100.stdout:1123: done (1 left) 2024-04-07T16:20:11.721 INFO:tasks.rados.rados.0.smithi100.stdout:1124: done (0 left) 2024-04-07T16:20:11.721 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 23 current snap is 104 2024-04-07T16:20:11.721 INFO:tasks.rados.rados.0.smithi100.stdout:1125: seq_num 346 ranges {2785280=598016,3383296=401408} 2024-04-07T16:20:11.745 INFO:tasks.rados.rados.0.smithi100.stdout:1125: writing smithi10066867-23 from 2785280 to 3383296 tid 1 2024-04-07T16:20:11.748 INFO:tasks.rados.rados.0.smithi100.stdout:1125: writing smithi10066867-23 from 3383296 to 3784704 tid 2 2024-04-07T16:20:11.748 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 36 current snap is 104 2024-04-07T16:20:11.748 INFO:tasks.rados.rados.0.smithi100.stdout:1126: seq_num 347 ranges {0=614400,614400=401408} 2024-04-07T16:20:11.753 INFO:tasks.rados.rados.0.smithi100.stdout:1126: writing smithi10066867-36 from 0 to 614400 tid 1 2024-04-07T16:20:11.757 INFO:tasks.rados.rados.0.smithi100.stdout:1126: writing smithi10066867-36 from 614400 to 1015808 tid 2 2024-04-07T16:20:11.757 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 12 current snap is 104 2024-04-07T16:20:11.757 INFO:tasks.rados.rados.0.smithi100.stdout:1125: finishing write tid 1 to smithi10066867-23 2024-04-07T16:20:11.757 INFO:tasks.rados.rados.0.smithi100.stdout:1125: finishing write tid 2 to smithi10066867-23 2024-04-07T16:20:11.757 INFO:tasks.rados.rados.0.smithi100.stdout:1127: seq_num 348 ranges {5005312=630784,5636096=401408} 2024-04-07T16:20:11.797 INFO:tasks.rados.rados.0.smithi100.stdout:1127: writing smithi10066867-12 from 5005312 to 5636096 tid 1 2024-04-07T16:20:11.800 INFO:tasks.rados.rados.0.smithi100.stdout:1127: writing smithi10066867-12 from 5636096 to 6037504 tid 2 2024-04-07T16:20:11.800 INFO:tasks.rados.rados.0.smithi100.stdout:1128: snap_create 2024-04-07T16:20:11.800 INFO:tasks.rados.rados.0.smithi100.stdout:1125: finishing write tid 3 to smithi10066867-23 2024-04-07T16:20:11.800 INFO:tasks.rados.rados.0.smithi100.stdout:1126: finishing write tid 1 to smithi10066867-36 2024-04-07T16:20:11.800 INFO:tasks.rados.rados.0.smithi100.stdout:1125: finishing write tid 4 to smithi10066867-23 2024-04-07T16:20:11.800 INFO:tasks.rados.rados.0.smithi100.stdout:1125: oid 23 updating version 0 to 709 2024-04-07T16:20:11.800 INFO:tasks.rados.rados.0.smithi100.stdout:1125: oid 23 version 709 is already newer than 707 2024-04-07T16:20:11.800 INFO:tasks.rados.rados.0.smithi100.stdout:1125: oid 23 version 709 is already newer than 708 2024-04-07T16:20:11.800 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 23 v 709 (ObjNum 346 snap 104 seq_num 346) dirty exists 2024-04-07T16:20:11.800 INFO:tasks.rados.rados.0.smithi100.stdout:1125: left oid 23 (ObjNum 346 snap 104 seq_num 346) 2024-04-07T16:20:11.800 INFO:tasks.rados.rados.0.smithi100.stdout:1126: finishing write tid 2 to smithi10066867-36 2024-04-07T16:20:11.800 INFO:tasks.rados.rados.0.smithi100.stdout:1126: finishing write tid 3 to smithi10066867-36 2024-04-07T16:20:11.800 INFO:tasks.rados.rados.0.smithi100.stdout:1126: finishing write tid 4 to smithi10066867-36 2024-04-07T16:20:11.800 INFO:tasks.rados.rados.0.smithi100.stdout:1126: oid 36 updating version 0 to 541 2024-04-07T16:20:11.801 INFO:tasks.rados.rados.0.smithi100.stdout:1126: oid 36 version 541 is already newer than 539 2024-04-07T16:20:11.801 INFO:tasks.rados.rados.0.smithi100.stdout:1126: oid 36 version 541 is already newer than 540 2024-04-07T16:20:11.801 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 36 v 541 (ObjNum 347 snap 104 seq_num 347) dirty exists 2024-04-07T16:20:11.801 INFO:tasks.rados.rados.0.smithi100.stdout:1126: left oid 36 (ObjNum 347 snap 104 seq_num 347) 2024-04-07T16:20:11.808 INFO:tasks.rados.rados.0.smithi100.stdout:1127: finishing write tid 1 to smithi10066867-12 2024-04-07T16:20:11.808 INFO:tasks.rados.rados.0.smithi100.stdout:1127: finishing write tid 2 to smithi10066867-12 2024-04-07T16:20:11.808 INFO:tasks.rados.rados.0.smithi100.stdout:1127: finishing write tid 3 to smithi10066867-12 2024-04-07T16:20:11.811 INFO:tasks.rados.rados.0.smithi100.stdout:1127: finishing write tid 4 to smithi10066867-12 2024-04-07T16:20:11.811 INFO:tasks.rados.rados.0.smithi100.stdout:1127: oid 12 updating version 0 to 571 2024-04-07T16:20:11.811 INFO:tasks.rados.rados.0.smithi100.stdout:1127: oid 12 updating version 571 to 573 2024-04-07T16:20:11.811 INFO:tasks.rados.rados.0.smithi100.stdout:1127: oid 12 version 573 is already newer than 572 2024-04-07T16:20:11.811 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 12 v 573 (ObjNum 348 snap 104 seq_num 348) dirty exists 2024-04-07T16:20:11.811 INFO:tasks.rados.rados.0.smithi100.stdout:1127: left oid 12 (ObjNum 348 snap 104 seq_num 348) 2024-04-07T16:20:12.105 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:20:11 smithi144 bash[60191]: debug 2024-04-07T16:20:11.751+0000 7fe2b70ab200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-07T16:20:12.105 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:20:11 smithi144 bash[60191]: debug 2024-04-07T16:20:11.871+0000 7fe2b70ab200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-07T16:20:12.105 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:20:11 smithi144 bash[60191]: debug 2024-04-07T16:20:11.975+0000 7fe2b70ab200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-07T16:20:12.186 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:20:11 smithi100 bash[68137]: debug 2024-04-07T16:20:11.778+0000 7f854a417200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-07T16:20:12.186 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:20:11 smithi100 bash[68137]: debug 2024-04-07T16:20:11.898+0000 7f854a417200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-07T16:20:12.186 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:20:12 smithi100 bash[68137]: debug 2024-04-07T16:20:12.006+0000 7f854a417200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-07T16:20:12.575 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:20:12 smithi144 bash[60191]: debug 2024-04-07T16:20:12.339+0000 7fe2b70ab200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-07T16:20:12.575 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:12 smithi144 bash[23522]: cluster 2024-04-07T16:20:11.276682+0000 mon.a (mon.0) 21 : cluster 0 osdmap e274: 8 total, 8 up, 8 in 2024-04-07T16:20:12.575 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:12 smithi144 bash[23522]: cluster 2024-04-07T16:20:11.733178+0000 mon.a (mon.0) 22 : cluster 0 osdmap e275: 8 total, 8 up, 8 in 2024-04-07T16:20:12.607 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:12 smithi100 bash[74942]: cluster 2024-04-07T16:20:11.276682+0000 mon.a (mon.0) 21 : cluster [DBG] osdmap e274: 8 total, 8 up, 8 in 2024-04-07T16:20:12.607 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:12 smithi100 bash[74942]: cluster 2024-04-07T16:20:11.733178+0000 mon.a (mon.0) 22 : cluster [DBG] osdmap e275: 8 total, 8 up, 8 in 2024-04-07T16:20:12.607 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:20:12 smithi100 bash[68137]: debug 2024-04-07T16:20:12.378+0000 7f854a417200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-07T16:20:12.608 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:12 smithi100 bash[31660]: cluster 2024-04-07T16:20:11.276682+0000 mon.a (mon.0) 21 : cluster 0 osdmap e274: 8 total, 8 up, 8 in 2024-04-07T16:20:12.608 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:12 smithi100 bash[31660]: cluster 2024-04-07T16:20:11.733178+0000 mon.a (mon.0) 22 : cluster 0 osdmap e275: 8 total, 8 up, 8 in 2024-04-07T16:20:12.724 INFO:tasks.rados.rados.0.smithi100.stdout:1125: done (3 left) 2024-04-07T16:20:12.724 INFO:tasks.rados.rados.0.smithi100.stdout:1126: done (2 left) 2024-04-07T16:20:12.724 INFO:tasks.rados.rados.0.smithi100.stdout:1127: done (1 left) 2024-04-07T16:20:12.724 INFO:tasks.rados.rados.0.smithi100.stdout:1128: done (0 left) 2024-04-07T16:20:12.724 INFO:tasks.rados.rados.0.smithi100.stdout:1129: read oid 16 snap -1 2024-04-07T16:20:12.724 INFO:tasks.rados.rados.0.smithi100.stdout:1129: expect (ObjNum 0 snap 0 seq_num 0) 2024-04-07T16:20:12.724 INFO:tasks.rados.rados.0.smithi100.stdout:1130: snap_create 2024-04-07T16:20:12.725 INFO:tasks.rados.rados.0.smithi100.stdout:1129: expect (ObjNum 0 snap 0 seq_num 0) 2024-04-07T16:20:12.842 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:20:12 smithi144 bash[60191]: debug 2024-04-07T16:20:12.583+0000 7fe2b70ab200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-07T16:20:12.877 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:20:12 smithi100 bash[68137]: debug 2024-04-07T16:20:12.622+0000 7f854a417200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-07T16:20:13.105 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:20:12 smithi144 bash[60191]: debug 2024-04-07T16:20:12.851+0000 7fe2b70ab200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-07T16:20:13.105 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:20:12 smithi144 bash[60191]: debug 2024-04-07T16:20:12.967+0000 7fe2b70ab200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-07T16:20:13.186 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:20:12 smithi100 bash[68137]: debug 2024-04-07T16:20:12.890+0000 7f854a417200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-07T16:20:13.186 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:20:13 smithi100 bash[68137]: debug 2024-04-07T16:20:13.006+0000 7f854a417200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-07T16:20:13.495 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:20:13 smithi144 bash[60191]: debug 2024-04-07T16:20:13.247+0000 7fe2b70ab200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-07T16:20:13.686 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:20:13 smithi100 bash[68137]: debug 2024-04-07T16:20:13.286+0000 7f854a417200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-07T16:20:13.729 INFO:tasks.rados.rados.0.smithi100.stdout:1129: done (1 left) 2024-04-07T16:20:13.729 INFO:tasks.rados.rados.0.smithi100.stdout:1130: done (0 left) 2024-04-07T16:20:13.729 INFO:tasks.rados.rados.0.smithi100.stdout:1131: rmattr oid 40 current snap is 106 2024-04-07T16:20:13.730 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 32 current snap is 106 2024-04-07T16:20:13.730 INFO:tasks.rados.rados.0.smithi100.stdout:1132: seq_num 349 ranges {5054464=655360,5709824=401408} 2024-04-07T16:20:13.775 INFO:tasks.rados.rados.0.smithi100.stdout:1132: writing smithi10066867-32 from 5054464 to 5709824 tid 1 2024-04-07T16:20:13.778 INFO:tasks.rados.rados.0.smithi100.stdout:1132: writing smithi10066867-32 from 5709824 to 6111232 tid 2 2024-04-07T16:20:13.778 INFO:tasks.rados.rados.0.smithi100.stdout:1133: snap_create 2024-04-07T16:20:13.778 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 40 v 637 (ObjNum 303 snap 87 seq_num 303) dirty exists 2024-04-07T16:20:13.782 INFO:tasks.rados.rados.0.smithi100.stdout:1132: finishing write tid 1 to smithi10066867-32 2024-04-07T16:20:13.782 INFO:tasks.rados.rados.0.smithi100.stdout:1132: finishing write tid 2 to smithi10066867-32 2024-04-07T16:20:13.783 INFO:tasks.rados.rados.0.smithi100.stdout:1132: finishing write tid 3 to smithi10066867-32 2024-04-07T16:20:13.784 INFO:tasks.rados.rados.0.smithi100.stdout:1132: finishing write tid 4 to smithi10066867-32 2024-04-07T16:20:13.784 INFO:tasks.rados.rados.0.smithi100.stdout:1132: oid 32 updating version 0 to 164 2024-04-07T16:20:13.784 INFO:tasks.rados.rados.0.smithi100.stdout:1132: oid 32 version 164 is already newer than 162 2024-04-07T16:20:13.784 INFO:tasks.rados.rados.0.smithi100.stdout:1132: oid 32 version 164 is already newer than 163 2024-04-07T16:20:13.784 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 32 v 164 (ObjNum 349 snap 106 seq_num 349) dirty exists 2024-04-07T16:20:13.785 INFO:tasks.rados.rados.0.smithi100.stdout:1132: left oid 32 (ObjNum 349 snap 106 seq_num 349) 2024-04-07T16:20:13.855 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:20:13 smithi144 bash[59794]: ts=2024-04-07T16:20:13.506Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=nodes name=CephNodeDiskspaceWarning index=4 msg="Evaluating rule failed" rule="alert: CephNodeDiskspaceWarning\nexpr: predict_linear(node_filesystem_free_bytes{device=~\"/.*\"}[2d], 3600 * 24 * 5)\n * on (instance) group_left (nodename) node_uname_info < 0\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.8.4\n severity: warning\n type: ceph_default\nannotations:\n description: Mountpoint {{ $labels.mountpoint }} on {{ $labels.nodename }} will\n be full in less than 5 days based on the 48 hour trailing fill rate.\n summary: Host filesystem free space is getting low\n" err="found duplicate series for the match group {instance=\"smithi100\"} on the right hand-side of the operation: [{__name__=\"node_uname_info\", domainname=\"(none)\", instance=\"smithi100\", job=\"node\", machine=\"x86_64\", nodename=\"smithi100\", release=\"5.15.0-101-generic\", sysname=\"Linux\", version=\"#111-Ubuntu SMP Tue Mar 5 20:16:58 UTC 2024\"}, {__name__=\"node_uname_info\", cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", domainname=\"(none)\", instance=\"smithi100\", job=\"node\", machine=\"x86_64\", nodename=\"smithi100\", release=\"5.15.0-101-generic\", sysname=\"Linux\", version=\"#111-Ubuntu SMP Tue Mar 5 20:16:58 UTC 2024\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:20:13.855 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:13 smithi144 bash[23522]: cluster 2024-04-07T16:20:12.740106+0000 mon.a (mon.0) 23 : cluster 0 osdmap e276: 8 total, 8 up, 8 in 2024-04-07T16:20:13.856 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:20:13 smithi144 bash[60191]: debug 2024-04-07T16:20:13.671+0000 7fe2b70ab200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-07T16:20:13.950 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:13 smithi100 bash[74942]: cluster 2024-04-07T16:20:12.740106+0000 mon.a (mon.0) 23 : cluster [DBG] osdmap e276: 8 total, 8 up, 8 in 2024-04-07T16:20:13.951 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:20:13 smithi100 bash[68137]: debug 2024-04-07T16:20:13.710+0000 7f854a417200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-07T16:20:13.951 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:13 smithi100 bash[31660]: cluster 2024-04-07T16:20:12.740106+0000 mon.a (mon.0) 23 : cluster 0 osdmap e276: 8 total, 8 up, 8 in 2024-04-07T16:20:14.289 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:20:13 smithi144 bash[60191]: debug 2024-04-07T16:20:13.923+0000 7fe2b70ab200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-07T16:20:14.289 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:20:14 smithi144 bash[60191]: debug 2024-04-07T16:20:14.043+0000 7fe2b70ab200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-07T16:20:14.328 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:20:13 smithi100 bash[68137]: debug 2024-04-07T16:20:13.962+0000 7f854a417200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-07T16:20:14.328 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:20:14 smithi100 bash[68137]: debug 2024-04-07T16:20:14.086+0000 7f854a417200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-07T16:20:14.605 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:20:14 smithi144 bash[60191]: debug 2024-04-07T16:20:14.295+0000 7fe2b70ab200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-07T16:20:14.686 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:20:14 smithi100 bash[68137]: debug 2024-04-07T16:20:14.342+0000 7f854a417200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-07T16:20:14.747 INFO:tasks.rados.rados.0.smithi100.stdout:1131: done (2 left) 2024-04-07T16:20:14.747 INFO:tasks.rados.rados.0.smithi100.stdout:1132: done (1 left) 2024-04-07T16:20:14.747 INFO:tasks.rados.rados.0.smithi100.stdout:1133: done (0 left) 2024-04-07T16:20:14.747 INFO:tasks.rados.rados.0.smithi100.stdout:1134: snap_remove snap 69 2024-04-07T16:20:15.105 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:14 smithi144 bash[23522]: cluster 2024-04-07T16:20:13.740277+0000 mon.a (mon.0) 24 : cluster 0 osdmap e277: 8 total, 8 up, 8 in 2024-04-07T16:20:15.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:14 smithi100 bash[74942]: cluster 2024-04-07T16:20:13.740277+0000 mon.a (mon.0) 24 : cluster [DBG] osdmap e277: 8 total, 8 up, 8 in 2024-04-07T16:20:15.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:14 smithi100 bash[31660]: cluster 2024-04-07T16:20:13.740277+0000 mon.a (mon.0) 24 : cluster 0 osdmap e277: 8 total, 8 up, 8 in 2024-04-07T16:20:15.605 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:20:15 smithi144 bash[60191]: debug 2024-04-07T16:20:15.331+0000 7fe2b70ab200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-07T16:20:15.605 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:20:15 smithi144 bash[60191]: debug 2024-04-07T16:20:15.447+0000 7fe2b70ab200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-07T16:20:15.686 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:20:15 smithi100 bash[68137]: debug 2024-04-07T16:20:15.374+0000 7f854a417200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-07T16:20:15.686 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:20:15 smithi100 bash[68137]: debug 2024-04-07T16:20:15.494+0000 7f854a417200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-07T16:20:15.751 INFO:tasks.rados.rados.0.smithi100.stdout:1134: done (0 left) 2024-04-07T16:20:15.751 INFO:tasks.rados.rados.0.smithi100.stdout:1135: copy_from oid 33 from oid 1 current snap is 107 2024-04-07T16:20:15.751 INFO:tasks.rados.rados.0.smithi100.stdout:1136: read oid 46 snap -1 2024-04-07T16:20:15.751 INFO:tasks.rados.rados.0.smithi100.stdout:1136: expect (ObjNum 337 snap 101 seq_num 337) 2024-04-07T16:20:15.752 INFO:tasks.rados.rados.0.smithi100.stdout:1137: snap_remove snap 97 2024-04-07T16:20:16.105 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:15 smithi144 bash[23522]: cluster 2024-04-07T16:20:14.763295+0000 mon.a (mon.0) 25 : cluster 0 osdmap e278: 8 total, 8 up, 8 in 2024-04-07T16:20:16.106 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:20:15 smithi144 bash[60191]: debug 2024-04-07T16:20:15.751+0000 7fe2b70ab200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-07T16:20:16.106 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:20:15 smithi144 bash[60191]: debug 2024-04-07T16:20:15.891+0000 7fe2b70ab200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-07T16:20:16.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:15 smithi100 bash[74942]: cluster 2024-04-07T16:20:14.763295+0000 mon.a (mon.0) 25 : cluster [DBG] osdmap e278: 8 total, 8 up, 8 in 2024-04-07T16:20:16.186 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:20:15 smithi100 bash[68137]: debug 2024-04-07T16:20:15.826+0000 7f854a417200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-07T16:20:16.187 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:20:15 smithi100 bash[68137]: debug 2024-04-07T16:20:15.950+0000 7f854a417200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-07T16:20:16.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:15 smithi100 bash[31660]: cluster 2024-04-07T16:20:14.763295+0000 mon.a (mon.0) 25 : cluster 0 osdmap e278: 8 total, 8 up, 8 in 2024-04-07T16:20:16.564 INFO:tasks.rados.rados.0.smithi100.stdout:1136: expect (ObjNum 337 snap 101 seq_num 337) 2024-04-07T16:20:16.598 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:20:16 smithi144 bash[60191]: debug 2024-04-07T16:20:16.347+0000 7fe2b70ab200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-07T16:20:16.598 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:20:16 smithi144 bash[60191]: debug 2024-04-07T16:20:16.475+0000 7fe2b70ab200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-07T16:20:16.598 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:20:16 smithi144 bash[60191]: [07/Apr/2024:16:20:16] ENGINE Bus STARTING 2024-04-07T16:20:16.686 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:20:16 smithi100 bash[68137]: debug 2024-04-07T16:20:16.410+0000 7f854a417200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-07T16:20:16.686 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:20:16 smithi100 bash[68137]: debug 2024-04-07T16:20:16.534+0000 7f854a417200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-07T16:20:16.784 INFO:tasks.rados.rados.0.smithi100.stdout:1135: finishing copy_from racing read to smithi10066867-33 2024-04-07T16:20:16.784 INFO:tasks.rados.rados.0.smithi100.stdout:1135: finishing copy_from to smithi10066867-33 2024-04-07T16:20:16.784 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 33 v 231 (ObjNum 333 snap 100 seq_num 333) dirty exists 2024-04-07T16:20:16.784 INFO:tasks.rados.rados.0.smithi100.stdout:1135: done (2 left) 2024-04-07T16:20:16.784 INFO:tasks.rados.rados.0.smithi100.stdout:1136: done (1 left) 2024-04-07T16:20:16.784 INFO:tasks.rados.rados.0.smithi100.stdout:1137: done (0 left) 2024-04-07T16:20:16.784 INFO:tasks.rados.rados.0.smithi100.stdout:1138: snap_remove snap 104 2024-04-07T16:20:16.855 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:20:16 smithi144 bash[60191]: CherryPy Checker: 2024-04-07T16:20:16.855 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:20:16 smithi144 bash[60191]: The Application mounted at '' has an empty config. 2024-04-07T16:20:16.855 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:20:16 smithi144 bash[60191]: [07/Apr/2024:16:20:16] ENGINE Serving on http://:::9283 2024-04-07T16:20:16.856 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:20:16 smithi144 bash[60191]: [07/Apr/2024:16:20:16] ENGINE Bus STARTED 2024-04-07T16:20:16.856 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:16 smithi144 bash[23522]: cluster 2024-04-07T16:20:15.765332+0000 mon.a (mon.0) 26 : cluster 0 osdmap e279: 8 total, 8 up, 8 in 2024-04-07T16:20:16.856 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:16 smithi144 bash[23522]: cluster 2024-04-07T16:20:16.505412+0000 mon.a (mon.0) 27 : cluster 0 Standby manager daemon x restarted 2024-04-07T16:20:16.856 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:16 smithi144 bash[23522]: cluster 2024-04-07T16:20:16.505530+0000 mon.a (mon.0) 28 : cluster 0 Standby manager daemon x started 2024-04-07T16:20:16.856 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:16 smithi144 bash[23522]: audit 2024-04-07T16:20:16.509385+0000 mon.c (mon.2) 129 : audit 0 from='mgr.? 172.21.15.144:0/1916599647' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-04-07T16:20:16.856 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:16 smithi144 bash[23522]: audit 2024-04-07T16:20:16.510603+0000 mon.c (mon.2) 130 : audit 0 from='mgr.? 172.21.15.144:0/1916599647' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-07T16:20:16.856 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:16 smithi144 bash[23522]: audit 2024-04-07T16:20:16.514099+0000 mon.c (mon.2) 131 : audit 0 from='mgr.? 172.21.15.144:0/1916599647' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-04-07T16:20:16.856 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:16 smithi144 bash[23522]: audit 2024-04-07T16:20:16.515178+0000 mon.c (mon.2) 132 : audit 0 from='mgr.? 172.21.15.144:0/1916599647' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-07T16:20:16.856 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:16 smithi144 bash[23522]: cluster 2024-04-07T16:20:16.558619+0000 mon.a (mon.0) 29 : cluster 1 Active manager daemon y restarted 2024-04-07T16:20:16.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:16 smithi144 bash[23522]: cluster 2024-04-07T16:20:16.559244+0000 mon.a (mon.0) 30 : cluster 1 Activating manager daemon y 2024-04-07T16:20:16.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:16 smithi144 bash[23522]: cluster 2024-04-07T16:20:16.572204+0000 mon.a (mon.0) 31 : cluster 0 osdmap e280: 8 total, 8 up, 8 in 2024-04-07T16:20:16.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:16 smithi144 bash[23522]: cluster 2024-04-07T16:20:16.575456+0000 mon.a (mon.0) 32 : cluster 0 mgrmap e37: y(active, starting, since 0.016365s), standbys: x 2024-04-07T16:20:16.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:16 smithi144 bash[23522]: audit 2024-04-07T16:20:16.585377+0000 mon.c (mon.2) 133 : audit 0 from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:20:16.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:16 smithi144 bash[23522]: audit 2024-04-07T16:20:16.585637+0000 mon.c (mon.2) 134 : audit 0 from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:20:16.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:16 smithi144 bash[23522]: audit 2024-04-07T16:20:16.585849+0000 mon.c (mon.2) 135 : audit 0 from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:20:16.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:16 smithi144 bash[23522]: audit 2024-04-07T16:20:16.586018+0000 mon.c (mon.2) 136 : audit 0 from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-07T16:20:16.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:16 smithi144 bash[23522]: audit 2024-04-07T16:20:16.586294+0000 mon.c (mon.2) 137 : audit 0 from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-07T16:20:16.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:16 smithi144 bash[23522]: audit 2024-04-07T16:20:16.588605+0000 mon.c (mon.2) 138 : audit 0 from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-07T16:20:16.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:16 smithi144 bash[23522]: audit 2024-04-07T16:20:16.588953+0000 mon.c (mon.2) 139 : audit 0 from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-07T16:20:16.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:16 smithi144 bash[23522]: audit 2024-04-07T16:20:16.589433+0000 mon.c (mon.2) 140 : audit 0 from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-07T16:20:16.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:16 smithi144 bash[23522]: audit 2024-04-07T16:20:16.589762+0000 mon.c (mon.2) 141 : audit 0 from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-07T16:20:16.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:16 smithi144 bash[23522]: audit 2024-04-07T16:20:16.590219+0000 mon.c (mon.2) 142 : audit 0 from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-07T16:20:16.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:16 smithi144 bash[23522]: audit 2024-04-07T16:20:16.591083+0000 mon.c (mon.2) 143 : audit 0 from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-07T16:20:16.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:16 smithi144 bash[23522]: audit 2024-04-07T16:20:16.591844+0000 mon.c (mon.2) 144 : audit 0 from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-07T16:20:16.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:16 smithi144 bash[23522]: audit 2024-04-07T16:20:16.592124+0000 mon.c (mon.2) 145 : audit 0 from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-07T16:20:16.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:16 smithi144 bash[23522]: audit 2024-04-07T16:20:16.592405+0000 mon.c (mon.2) 146 : audit 0 from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-07T16:20:16.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:16 smithi144 bash[23522]: audit 2024-04-07T16:20:16.592695+0000 mon.c (mon.2) 147 : audit 0 from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-07T16:20:16.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:16 smithi144 bash[23522]: audit 2024-04-07T16:20:16.597398+0000 mon.c (mon.2) 148 : audit 0 from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-07T16:20:16.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:16 smithi144 bash[23522]: audit 2024-04-07T16:20:16.597905+0000 mon.c (mon.2) 149 : audit 0 from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-07T16:20:16.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:16 smithi144 bash[23522]: audit 2024-04-07T16:20:16.598729+0000 mon.c (mon.2) 150 : audit 0 from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-07T16:20:16.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:16 smithi144 bash[23522]: cluster 2024-04-07T16:20:16.630267+0000 mon.a (mon.0) 33 : cluster 1 Manager daemon y is now available 2024-04-07T16:20:16.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:16 smithi144 bash[23522]: audit 2024-04-07T16:20:16.672042+0000 mon.c (mon.2) 151 : audit 0 from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:20:16.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:16 smithi144 bash[23522]: audit 2024-04-07T16:20:16.699548+0000 mon.c (mon.2) 152 : audit 0 from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:20:16.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:16 smithi144 bash[23522]: audit 2024-04-07T16:20:16.705790+0000 mon.c (mon.2) 153 : audit 1 from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-04-07T16:20:16.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:16 smithi144 bash[23522]: audit 2024-04-07T16:20:16.706261+0000 mon.a (mon.0) 34 : audit 1 from='mgr.25175 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-04-07T16:20:16.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:16 smithi144 bash[23522]: audit 2024-04-07T16:20:16.741308+0000 mon.c (mon.2) 154 : audit 1 from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-04-07T16:20:16.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:16 smithi144 bash[23522]: audit 2024-04-07T16:20:16.741809+0000 mon.a (mon.0) 35 : audit 1 from='mgr.25175 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-04-07T16:20:17.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:16 smithi100 bash[74942]: cluster 2024-04-07T16:20:15.765332+0000 mon.a (mon.0) 26 : cluster [DBG] osdmap e279: 8 total, 8 up, 8 in 2024-04-07T16:20:17.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:16 smithi100 bash[74942]: cluster 2024-04-07T16:20:16.505412+0000 mon.a (mon.0) 27 : cluster [DBG] Standby manager daemon x restarted 2024-04-07T16:20:17.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:16 smithi100 bash[74942]: cluster 2024-04-07T16:20:16.505530+0000 mon.a (mon.0) 28 : cluster [DBG] Standby manager daemon x started 2024-04-07T16:20:17.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:16 smithi100 bash[74942]: audit 2024-04-07T16:20:16.509385+0000 mon.c (mon.2) 129 : audit [DBG] from='mgr.? 172.21.15.144:0/1916599647' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-04-07T16:20:17.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:16 smithi100 bash[74942]: audit 2024-04-07T16:20:16.510603+0000 mon.c (mon.2) 130 : audit [DBG] from='mgr.? 172.21.15.144:0/1916599647' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-07T16:20:17.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:16 smithi100 bash[74942]: audit 2024-04-07T16:20:16.514099+0000 mon.c (mon.2) 131 : audit [DBG] from='mgr.? 172.21.15.144:0/1916599647' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-04-07T16:20:17.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:16 smithi100 bash[74942]: audit 2024-04-07T16:20:16.515178+0000 mon.c (mon.2) 132 : audit [DBG] from='mgr.? 172.21.15.144:0/1916599647' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-07T16:20:17.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:16 smithi100 bash[74942]: cluster 2024-04-07T16:20:16.558619+0000 mon.a (mon.0) 29 : cluster [INF] Active manager daemon y restarted 2024-04-07T16:20:17.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:16 smithi100 bash[74942]: cluster 2024-04-07T16:20:16.559244+0000 mon.a (mon.0) 30 : cluster [INF] Activating manager daemon y 2024-04-07T16:20:17.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:16 smithi100 bash[74942]: cluster 2024-04-07T16:20:16.572204+0000 mon.a (mon.0) 31 : cluster [DBG] osdmap e280: 8 total, 8 up, 8 in 2024-04-07T16:20:17.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:16 smithi100 bash[74942]: cluster 2024-04-07T16:20:16.575456+0000 mon.a (mon.0) 32 : cluster [DBG] mgrmap e37: y(active, starting, since 0.016365s), standbys: x 2024-04-07T16:20:17.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:16 smithi100 bash[74942]: audit 2024-04-07T16:20:16.585377+0000 mon.c (mon.2) 133 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:20:17.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:16 smithi100 bash[74942]: audit 2024-04-07T16:20:16.585637+0000 mon.c (mon.2) 134 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:20:17.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:16 smithi100 bash[74942]: audit 2024-04-07T16:20:16.585849+0000 mon.c (mon.2) 135 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:20:17.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:16 smithi100 bash[74942]: audit 2024-04-07T16:20:16.586018+0000 mon.c (mon.2) 136 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-07T16:20:17.188 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:16 smithi100 bash[74942]: audit 2024-04-07T16:20:16.586294+0000 mon.c (mon.2) 137 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-07T16:20:17.188 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:16 smithi100 bash[74942]: audit 2024-04-07T16:20:16.588605+0000 mon.c (mon.2) 138 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-07T16:20:17.188 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:16 smithi100 bash[74942]: audit 2024-04-07T16:20:16.588953+0000 mon.c (mon.2) 139 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-07T16:20:17.188 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:16 smithi100 bash[74942]: audit 2024-04-07T16:20:16.589433+0000 mon.c (mon.2) 140 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-07T16:20:17.188 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:16 smithi100 bash[74942]: audit 2024-04-07T16:20:16.589762+0000 mon.c (mon.2) 141 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-07T16:20:17.188 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:20:16 smithi100 bash[68137]: [07/Apr/2024:16:20:16] ENGINE Bus STARTING 2024-04-07T16:20:17.188 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:20:16 smithi100 bash[68137]: debug 2024-04-07T16:20:16.758+0000 7f84adf4f700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-07T16:20:17.188 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:20:16 smithi100 bash[68137]: debug 2024-04-07T16:20:16.758+0000 7f84adf4f700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-07T16:20:17.188 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:20:16 smithi100 bash[68137]: debug 2024-04-07T16:20:16.758+0000 7f84adf4f700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-07T16:20:17.188 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:20:16 smithi100 bash[68137]: debug 2024-04-07T16:20:16.758+0000 7f84adf4f700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-07T16:20:17.188 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:20:16 smithi100 bash[68137]: debug 2024-04-07T16:20:16.758+0000 7f84adf4f700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-07T16:20:17.188 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:20:16 smithi100 bash[68137]: debug 2024-04-07T16:20:16.758+0000 7f84b0754700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-07T16:20:17.189 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:20:16 smithi100 bash[68137]: debug 2024-04-07T16:20:16.758+0000 7f84b0754700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-07T16:20:17.189 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:20:16 smithi100 bash[68137]: debug 2024-04-07T16:20:16.758+0000 7f84b0754700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-07T16:20:17.189 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:20:16 smithi100 bash[68137]: debug 2024-04-07T16:20:16.758+0000 7f84b0754700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-07T16:20:17.189 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:20:16 smithi100 bash[68137]: debug 2024-04-07T16:20:16.758+0000 7f84b0754700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-07T16:20:17.189 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:20:16 smithi100 bash[68137]: CherryPy Checker: 2024-04-07T16:20:17.189 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:20:16 smithi100 bash[68137]: The Application mounted at '' has an empty config. 2024-04-07T16:20:17.189 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:20:16 smithi100 bash[68137]: [07/Apr/2024:16:20:16] ENGINE Serving on http://:::9283 2024-04-07T16:20:17.189 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:20:16 smithi100 bash[68137]: [07/Apr/2024:16:20:16] ENGINE Bus STARTED 2024-04-07T16:20:17.189 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:16 smithi100 bash[31660]: cluster 2024-04-07T16:20:15.765332+0000 mon.a (mon.0) 26 : cluster 0 osdmap e279: 8 total, 8 up, 8 in 2024-04-07T16:20:17.189 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:16 smithi100 bash[31660]: cluster 2024-04-07T16:20:16.505412+0000 mon.a (mon.0) 27 : cluster 0 Standby manager daemon x restarted 2024-04-07T16:20:17.189 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:16 smithi100 bash[31660]: cluster 2024-04-07T16:20:16.505530+0000 mon.a (mon.0) 28 : cluster 0 Standby manager daemon x started 2024-04-07T16:20:17.189 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:16 smithi100 bash[31660]: audit 2024-04-07T16:20:16.509385+0000 mon.c (mon.2) 129 : audit 0 from='mgr.? 172.21.15.144:0/1916599647' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-04-07T16:20:17.190 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:16 smithi100 bash[31660]: audit 2024-04-07T16:20:16.510603+0000 mon.c (mon.2) 130 : audit 0 from='mgr.? 172.21.15.144:0/1916599647' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-07T16:20:17.190 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:16 smithi100 bash[31660]: audit 2024-04-07T16:20:16.514099+0000 mon.c (mon.2) 131 : audit 0 from='mgr.? 172.21.15.144:0/1916599647' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-04-07T16:20:17.190 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:16 smithi100 bash[31660]: audit 2024-04-07T16:20:16.515178+0000 mon.c (mon.2) 132 : audit 0 from='mgr.? 172.21.15.144:0/1916599647' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-07T16:20:17.190 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:16 smithi100 bash[31660]: cluster 2024-04-07T16:20:16.558619+0000 mon.a (mon.0) 29 : cluster 1 Active manager daemon y restarted 2024-04-07T16:20:17.190 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:16 smithi100 bash[31660]: cluster 2024-04-07T16:20:16.559244+0000 mon.a (mon.0) 30 : cluster 1 Activating manager daemon y 2024-04-07T16:20:17.190 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:16 smithi100 bash[31660]: cluster 2024-04-07T16:20:16.572204+0000 mon.a (mon.0) 31 : cluster 0 osdmap e280: 8 total, 8 up, 8 in 2024-04-07T16:20:17.190 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:16 smithi100 bash[31660]: cluster 2024-04-07T16:20:16.575456+0000 mon.a (mon.0) 32 : cluster 0 mgrmap e37: y(active, starting, since 0.016365s), standbys: x 2024-04-07T16:20:17.190 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:16 smithi100 bash[31660]: audit 2024-04-07T16:20:16.585377+0000 mon.c (mon.2) 133 : audit 0 from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:20:17.190 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:16 smithi100 bash[31660]: audit 2024-04-07T16:20:16.585637+0000 mon.c (mon.2) 134 : audit 0 from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:20:17.190 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:16 smithi100 bash[31660]: audit 2024-04-07T16:20:16.585849+0000 mon.c (mon.2) 135 : audit 0 from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:20:17.190 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:16 smithi100 bash[31660]: audit 2024-04-07T16:20:16.586018+0000 mon.c (mon.2) 136 : audit 0 from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-07T16:20:17.190 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:16 smithi100 bash[31660]: audit 2024-04-07T16:20:16.586294+0000 mon.c (mon.2) 137 : audit 0 from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-07T16:20:17.190 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:16 smithi100 bash[31660]: audit 2024-04-07T16:20:16.588605+0000 mon.c (mon.2) 138 : audit 0 from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-07T16:20:17.190 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:16 smithi100 bash[31660]: audit 2024-04-07T16:20:16.588953+0000 mon.c (mon.2) 139 : audit 0 from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-07T16:20:17.191 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:16 smithi100 bash[31660]: audit 2024-04-07T16:20:16.589433+0000 mon.c (mon.2) 140 : audit 0 from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-07T16:20:17.191 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:16 smithi100 bash[31660]: audit 2024-04-07T16:20:16.589762+0000 mon.c (mon.2) 141 : audit 0 from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-07T16:20:17.191 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:16 smithi100 bash[74942]: audit 2024-04-07T16:20:16.590219+0000 mon.c (mon.2) 142 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-07T16:20:17.191 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:16 smithi100 bash[74942]: audit 2024-04-07T16:20:16.591083+0000 mon.c (mon.2) 143 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-07T16:20:17.191 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:16 smithi100 bash[74942]: audit 2024-04-07T16:20:16.591844+0000 mon.c (mon.2) 144 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-07T16:20:17.191 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:16 smithi100 bash[74942]: audit 2024-04-07T16:20:16.592124+0000 mon.c (mon.2) 145 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-07T16:20:17.191 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:16 smithi100 bash[74942]: audit 2024-04-07T16:20:16.592405+0000 mon.c (mon.2) 146 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-07T16:20:17.191 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:16 smithi100 bash[74942]: audit 2024-04-07T16:20:16.592695+0000 mon.c (mon.2) 147 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-07T16:20:17.191 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:16 smithi100 bash[74942]: audit 2024-04-07T16:20:16.597398+0000 mon.c (mon.2) 148 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-07T16:20:17.191 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:16 smithi100 bash[74942]: audit 2024-04-07T16:20:16.597905+0000 mon.c (mon.2) 149 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-07T16:20:17.191 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:16 smithi100 bash[74942]: audit 2024-04-07T16:20:16.598729+0000 mon.c (mon.2) 150 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-07T16:20:17.191 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:16 smithi100 bash[74942]: cluster 2024-04-07T16:20:16.630267+0000 mon.a (mon.0) 33 : cluster [INF] Manager daemon y is now available 2024-04-07T16:20:17.192 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:16 smithi100 bash[74942]: audit 2024-04-07T16:20:16.672042+0000 mon.c (mon.2) 151 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:20:17.192 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:16 smithi100 bash[74942]: audit 2024-04-07T16:20:16.699548+0000 mon.c (mon.2) 152 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:20:17.192 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:16 smithi100 bash[74942]: audit 2024-04-07T16:20:16.705790+0000 mon.c (mon.2) 153 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-04-07T16:20:17.192 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:16 smithi100 bash[74942]: audit 2024-04-07T16:20:16.706261+0000 mon.a (mon.0) 34 : audit [INF] from='mgr.25175 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-04-07T16:20:17.192 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:16 smithi100 bash[74942]: audit 2024-04-07T16:20:16.741308+0000 mon.c (mon.2) 154 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-04-07T16:20:17.192 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:16 smithi100 bash[74942]: audit 2024-04-07T16:20:16.741809+0000 mon.a (mon.0) 35 : audit [INF] from='mgr.25175 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-04-07T16:20:17.192 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:16 smithi100 bash[31660]: audit 2024-04-07T16:20:16.590219+0000 mon.c (mon.2) 142 : audit 0 from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-07T16:20:17.192 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:16 smithi100 bash[31660]: audit 2024-04-07T16:20:16.591083+0000 mon.c (mon.2) 143 : audit 0 from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-07T16:20:17.192 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:16 smithi100 bash[31660]: audit 2024-04-07T16:20:16.591844+0000 mon.c (mon.2) 144 : audit 0 from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-07T16:20:17.192 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:16 smithi100 bash[31660]: audit 2024-04-07T16:20:16.592124+0000 mon.c (mon.2) 145 : audit 0 from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-07T16:20:17.192 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:16 smithi100 bash[31660]: audit 2024-04-07T16:20:16.592405+0000 mon.c (mon.2) 146 : audit 0 from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-07T16:20:17.193 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:16 smithi100 bash[31660]: audit 2024-04-07T16:20:16.592695+0000 mon.c (mon.2) 147 : audit 0 from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-07T16:20:17.193 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:16 smithi100 bash[31660]: audit 2024-04-07T16:20:16.597398+0000 mon.c (mon.2) 148 : audit 0 from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-07T16:20:17.193 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:16 smithi100 bash[31660]: audit 2024-04-07T16:20:16.597905+0000 mon.c (mon.2) 149 : audit 0 from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-07T16:20:17.193 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:16 smithi100 bash[31660]: audit 2024-04-07T16:20:16.598729+0000 mon.c (mon.2) 150 : audit 0 from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-07T16:20:17.193 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:16 smithi100 bash[31660]: cluster 2024-04-07T16:20:16.630267+0000 mon.a (mon.0) 33 : cluster 1 Manager daemon y is now available 2024-04-07T16:20:17.193 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:16 smithi100 bash[31660]: audit 2024-04-07T16:20:16.672042+0000 mon.c (mon.2) 151 : audit 0 from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:20:17.193 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:16 smithi100 bash[31660]: audit 2024-04-07T16:20:16.699548+0000 mon.c (mon.2) 152 : audit 0 from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:20:17.193 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:16 smithi100 bash[31660]: audit 2024-04-07T16:20:16.705790+0000 mon.c (mon.2) 153 : audit 1 from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-04-07T16:20:17.193 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:16 smithi100 bash[31660]: audit 2024-04-07T16:20:16.706261+0000 mon.a (mon.0) 34 : audit 1 from='mgr.25175 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-04-07T16:20:17.193 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:16 smithi100 bash[31660]: audit 2024-04-07T16:20:16.741308+0000 mon.c (mon.2) 154 : audit 1 from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-04-07T16:20:17.193 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:16 smithi100 bash[31660]: audit 2024-04-07T16:20:16.741809+0000 mon.a (mon.0) 35 : audit 1 from='mgr.25175 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-04-07T16:20:17.767 INFO:tasks.rados.rados.0.smithi100.stdout:1138: done (0 left) 2024-04-07T16:20:17.767 INFO:tasks.rados.rados.0.smithi100.stdout:1139: snap_create 2024-04-07T16:20:17.936 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:20:17 smithi100 bash[68137]: debug 2024-04-07T16:20:17.570+0000 7f8508b76700 -1 mgr.server handle_report got status from non-daemon mon.a 2024-04-07T16:20:18.686 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:20:18 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:20:18] "GET /metrics HTTP/1.1" 200 33362 "" "Prometheus/2.43.0" 2024-04-07T16:20:18.686 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:18 smithi100 bash[74942]: cephadm 2024-04-07T16:20:17.553587+0000 mgr.y (mgr.25175) 1 : cephadm [INF] [07/Apr/2024:16:20:17] ENGINE Bus STARTING 2024-04-07T16:20:18.687 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:18 smithi100 bash[74942]: cluster 2024-04-07T16:20:17.571661+0000 mon.a (mon.0) 36 : cluster [DBG] mgrmap e38: y(active, since 1.01261s), standbys: x 2024-04-07T16:20:18.687 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:18 smithi100 bash[74942]: cluster 2024-04-07T16:20:17.580026+0000 mgr.y (mgr.25175) 2 : cluster [DBG] pgmap v3: 121 pgs: 121 active+clean; 253 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail 2024-04-07T16:20:18.687 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:18 smithi100 bash[74942]: cephadm 2024-04-07T16:20:17.655471+0000 mgr.y (mgr.25175) 3 : cephadm [INF] [07/Apr/2024:16:20:17] ENGINE Serving on http://172.21.15.100:8765 2024-04-07T16:20:18.687 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:18 smithi100 bash[74942]: cephadm 2024-04-07T16:20:17.773516+0000 mgr.y (mgr.25175) 4 : cephadm [INF] [07/Apr/2024:16:20:17] ENGINE Serving on https://172.21.15.100:7150 2024-04-07T16:20:18.687 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:18 smithi100 bash[74942]: cephadm 2024-04-07T16:20:17.773839+0000 mgr.y (mgr.25175) 5 : cephadm [INF] [07/Apr/2024:16:20:17] ENGINE Bus STARTED 2024-04-07T16:20:18.687 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:18 smithi100 bash[74942]: cluster 2024-04-07T16:20:17.783138+0000 mon.a (mon.0) 37 : cluster [DBG] osdmap e281: 8 total, 8 up, 8 in 2024-04-07T16:20:18.687 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:18 smithi100 bash[31660]: cephadm 2024-04-07T16:20:17.553587+0000 mgr.y (mgr.25175) 1 : cephadm 1 [07/Apr/2024:16:20:17] ENGINE Bus STARTING 2024-04-07T16:20:18.687 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:18 smithi100 bash[31660]: cluster 2024-04-07T16:20:17.571661+0000 mon.a (mon.0) 36 : cluster 0 mgrmap e38: y(active, since 1.01261s), standbys: x 2024-04-07T16:20:18.687 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:18 smithi100 bash[31660]: cluster 2024-04-07T16:20:17.580026+0000 mgr.y (mgr.25175) 2 : cluster 0 pgmap v3: 121 pgs: 121 active+clean; 253 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail 2024-04-07T16:20:18.687 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:18 smithi100 bash[31660]: cephadm 2024-04-07T16:20:17.655471+0000 mgr.y (mgr.25175) 3 : cephadm 1 [07/Apr/2024:16:20:17] ENGINE Serving on http://172.21.15.100:8765 2024-04-07T16:20:18.687 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:18 smithi100 bash[31660]: cephadm 2024-04-07T16:20:17.773516+0000 mgr.y (mgr.25175) 4 : cephadm 1 [07/Apr/2024:16:20:17] ENGINE Serving on https://172.21.15.100:7150 2024-04-07T16:20:18.688 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:18 smithi100 bash[31660]: cephadm 2024-04-07T16:20:17.773839+0000 mgr.y (mgr.25175) 5 : cephadm 1 [07/Apr/2024:16:20:17] ENGINE Bus STARTED 2024-04-07T16:20:18.688 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:18 smithi100 bash[31660]: cluster 2024-04-07T16:20:17.783138+0000 mon.a (mon.0) 37 : cluster 0 osdmap e281: 8 total, 8 up, 8 in 2024-04-07T16:20:18.764 INFO:tasks.rados.rados.0.smithi100.stdout:1139: done (0 left) 2024-04-07T16:20:18.764 INFO:tasks.rados.rados.0.smithi100.stdout:1140: rmattr oid 26 current snap is 108 2024-04-07T16:20:18.764 INFO:tasks.rados.rados.0.smithi100.stdout:1140: done (0 left) 2024-04-07T16:20:18.765 INFO:tasks.rados.rados.0.smithi100.stdout:1141: rmattr oid 47 current snap is 108 2024-04-07T16:20:18.765 INFO:tasks.rados.rados.0.smithi100.stdout:1141: done (0 left) 2024-04-07T16:20:18.765 INFO:tasks.rados.rados.0.smithi100.stdout:1142: rmattr oid 22 current snap is 108 2024-04-07T16:20:18.765 INFO:tasks.rados.rados.0.smithi100.stdout:1142: done (0 left) 2024-04-07T16:20:18.765 INFO:tasks.rados.rados.0.smithi100.stdout:1143: snap_remove snap 75 2024-04-07T16:20:18.855 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:18 smithi144 bash[23522]: cephadm 2024-04-07T16:20:17.553587+0000 mgr.y (mgr.25175) 1 : cephadm 1 [07/Apr/2024:16:20:17] ENGINE Bus STARTING 2024-04-07T16:20:18.856 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:18 smithi144 bash[23522]: cluster 2024-04-07T16:20:17.571661+0000 mon.a (mon.0) 36 : cluster 0 mgrmap e38: y(active, since 1.01261s), standbys: x 2024-04-07T16:20:18.856 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:18 smithi144 bash[23522]: cluster 2024-04-07T16:20:17.580026+0000 mgr.y (mgr.25175) 2 : cluster 0 pgmap v3: 121 pgs: 121 active+clean; 253 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail 2024-04-07T16:20:18.856 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:18 smithi144 bash[23522]: cephadm 2024-04-07T16:20:17.655471+0000 mgr.y (mgr.25175) 3 : cephadm 1 [07/Apr/2024:16:20:17] ENGINE Serving on http://172.21.15.100:8765 2024-04-07T16:20:18.856 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:18 smithi144 bash[23522]: cephadm 2024-04-07T16:20:17.773516+0000 mgr.y (mgr.25175) 4 : cephadm 1 [07/Apr/2024:16:20:17] ENGINE Serving on https://172.21.15.100:7150 2024-04-07T16:20:18.856 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:18 smithi144 bash[23522]: cephadm 2024-04-07T16:20:17.773839+0000 mgr.y (mgr.25175) 5 : cephadm 1 [07/Apr/2024:16:20:17] ENGINE Bus STARTED 2024-04-07T16:20:18.856 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:18 smithi144 bash[23522]: cluster 2024-04-07T16:20:17.783138+0000 mon.a (mon.0) 37 : cluster 0 osdmap e281: 8 total, 8 up, 8 in 2024-04-07T16:20:19.780 INFO:tasks.rados.rados.0.smithi100.stdout:1143: done (0 left) 2024-04-07T16:20:19.780 INFO:tasks.rados.rados.0.smithi100.stdout:1144: delete oid 38 current snap is 108 2024-04-07T16:20:19.808 INFO:tasks.rados.rados.0.smithi100.stdout:1144: done (0 left) 2024-04-07T16:20:19.808 INFO:tasks.rados.rados.0.smithi100.stdout:1145: rollback oid 3 current snap is 108 2024-04-07T16:20:19.808 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 3 to 101 2024-04-07T16:20:19.808 INFO:tasks.rados.rados.0.smithi100.stdout:1146: setattr oid 37 current snap is 108 2024-04-07T16:20:19.816 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 18 current snap is 108 2024-04-07T16:20:19.816 INFO:tasks.rados.rados.0.smithi100.stdout:1147: seq_num 351 ranges {11091968=688128,11780096=401408} 2024-04-07T16:20:19.919 INFO:tasks.rados.rados.0.smithi100.stdout:1147: writing smithi10066867-18 from 11091968 to 11780096 tid 1 2024-04-07T16:20:19.921 INFO:tasks.rados.rados.0.smithi100.stdout:1147: writing smithi10066867-18 from 11780096 to 12181504 tid 2 2024-04-07T16:20:19.921 INFO:tasks.rados.rados.0.smithi100.stdout:1148: snap_create 2024-04-07T16:20:19.922 INFO:tasks.rados.rados.0.smithi100.stdout:1145: finishing rollback tid 0 to smithi10066867-3 2024-04-07T16:20:19.922 INFO:tasks.rados.rados.0.smithi100.stdout:1145: finishing rollback tid 1 to smithi10066867-3 2024-04-07T16:20:19.922 INFO:tasks.rados.rados.0.smithi100.stdout:1145: finishing rollback tid 2 to smithi10066867-3 2024-04-07T16:20:19.922 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 3 v 246 (ObjNum 298 snap 85 seq_num 298) dirty exists 2024-04-07T16:20:19.922 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 37 v 479 (ObjNum 161 snap 0 seq_num 2702390864) dirty exists 2024-04-07T16:20:19.930 INFO:tasks.rados.rados.0.smithi100.stdout:1147: finishing write tid 1 to smithi10066867-18 2024-04-07T16:20:19.930 INFO:tasks.rados.rados.0.smithi100.stdout:1147: finishing write tid 2 to smithi10066867-18 2024-04-07T16:20:19.931 INFO:tasks.rados.rados.0.smithi100.stdout:1147: finishing write tid 3 to smithi10066867-18 2024-04-07T16:20:19.933 INFO:tasks.rados.rados.0.smithi100.stdout:1147: finishing write tid 4 to smithi10066867-18 2024-04-07T16:20:19.933 INFO:tasks.rados.rados.0.smithi100.stdout:1147: oid 18 updating version 0 to 747 2024-04-07T16:20:19.933 INFO:tasks.rados.rados.0.smithi100.stdout:1147: oid 18 version 747 is already newer than 745 2024-04-07T16:20:19.933 INFO:tasks.rados.rados.0.smithi100.stdout:1147: oid 18 version 747 is already newer than 746 2024-04-07T16:20:19.934 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 18 v 747 (ObjNum 351 snap 108 seq_num 351) dirty exists 2024-04-07T16:20:19.934 INFO:tasks.rados.rados.0.smithi100.stdout:1147: left oid 18 (ObjNum 351 snap 108 seq_num 351) 2024-04-07T16:20:20.105 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:19 smithi144 bash[23522]: cluster 2024-04-07T16:20:18.597295+0000 mgr.y (mgr.25175) 6 : cluster 0 pgmap v5: 121 pgs: 121 active+clean; 253 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail 2024-04-07T16:20:20.105 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:19 smithi144 bash[23522]: cluster 2024-04-07T16:20:18.779611+0000 mon.a (mon.0) 38 : cluster 0 osdmap e282: 8 total, 8 up, 8 in 2024-04-07T16:20:20.105 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:19 smithi144 bash[23522]: cluster 2024-04-07T16:20:19.594462+0000 mon.a (mon.0) 39 : cluster 0 mgrmap e39: y(active, since 3s), standbys: x 2024-04-07T16:20:20.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:19 smithi100 bash[74942]: cluster 2024-04-07T16:20:18.597295+0000 mgr.y (mgr.25175) 6 : cluster [DBG] pgmap v5: 121 pgs: 121 active+clean; 253 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail 2024-04-07T16:20:20.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:19 smithi100 bash[74942]: cluster 2024-04-07T16:20:18.779611+0000 mon.a (mon.0) 38 : cluster [DBG] osdmap e282: 8 total, 8 up, 8 in 2024-04-07T16:20:20.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:19 smithi100 bash[74942]: cluster 2024-04-07T16:20:19.594462+0000 mon.a (mon.0) 39 : cluster [DBG] mgrmap e39: y(active, since 3s), standbys: x 2024-04-07T16:20:20.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:19 smithi100 bash[31660]: cluster 2024-04-07T16:20:18.597295+0000 mgr.y (mgr.25175) 6 : cluster 0 pgmap v5: 121 pgs: 121 active+clean; 253 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail 2024-04-07T16:20:20.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:19 smithi100 bash[31660]: cluster 2024-04-07T16:20:18.779611+0000 mon.a (mon.0) 38 : cluster 0 osdmap e282: 8 total, 8 up, 8 in 2024-04-07T16:20:20.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:19 smithi100 bash[31660]: cluster 2024-04-07T16:20:19.594462+0000 mon.a (mon.0) 39 : cluster 0 mgrmap e39: y(active, since 3s), standbys: x 2024-04-07T16:20:20.804 INFO:tasks.rados.rados.0.smithi100.stdout:1145: done (3 left) 2024-04-07T16:20:20.804 INFO:tasks.rados.rados.0.smithi100.stdout:1146: done (2 left) 2024-04-07T16:20:20.804 INFO:tasks.rados.rados.0.smithi100.stdout:1147: done (1 left) 2024-04-07T16:20:20.804 INFO:tasks.rados.rados.0.smithi100.stdout:1148: done (0 left) 2024-04-07T16:20:20.804 INFO:tasks.rados.rados.0.smithi100.stdout:1149: read oid 7 snap 103 2024-04-07T16:20:20.805 INFO:tasks.rados.rados.0.smithi100.stdout:1149: expect deleted 2024-04-07T16:20:20.805 INFO:tasks.rados.rados.0.smithi100.stdout:1150: rmattr oid 2 current snap is 109 2024-04-07T16:20:20.805 INFO:tasks.rados.rados.0.smithi100.stdout:1151: copy_from oid 46 from oid 47 current snap is 109 2024-04-07T16:20:20.805 INFO:tasks.rados.rados.0.smithi100.stdout:1152: copy_from oid 38 from oid 20 current snap is 109 2024-04-07T16:20:20.805 INFO:tasks.rados.rados.0.smithi100.stdout:1153: copy_from oid 44 from oid 39 current snap is 109 2024-04-07T16:20:20.805 INFO:tasks.rados.rados.0.smithi100.stdout:1154: delete oid 25 current snap is 109 2024-04-07T16:20:20.815 INFO:tasks.rados.rados.0.smithi100.stdout:1149: done (5 left) 2024-04-07T16:20:20.815 INFO:tasks.rados.rados.0.smithi100.stdout:1154: done (4 left) 2024-04-07T16:20:20.815 INFO:tasks.rados.rados.0.smithi100.stdout:1155: rollback oid 5 current snap is 109 2024-04-07T16:20:20.815 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 5 to 106 2024-04-07T16:20:20.815 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 16 current snap is 109 2024-04-07T16:20:20.815 INFO:tasks.rados.rados.0.smithi100.stdout:1156: seq_num 353 ranges {0=720896,720896=401408} 2024-04-07T16:20:20.820 INFO:tasks.rados.rados.0.smithi100.stdout:1156: writing smithi10066867-16 from 0 to 720896 tid 1 2024-04-07T16:20:20.823 INFO:tasks.rados.rados.0.smithi100.stdout:1156: writing smithi10066867-16 from 720896 to 1122304 tid 2 2024-04-07T16:20:20.823 INFO:tasks.rados.rados.0.smithi100.stdout:1157: snap_remove snap 108 2024-04-07T16:20:21.105 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:20 smithi144 bash[23522]: cluster 2024-04-07T16:20:19.794900+0000 mon.a (mon.0) 40 : cluster 0 osdmap e283: 8 total, 8 up, 8 in 2024-04-07T16:20:21.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:20 smithi100 bash[74942]: cluster 2024-04-07T16:20:19.794900+0000 mon.a (mon.0) 40 : cluster [DBG] osdmap e283: 8 total, 8 up, 8 in 2024-04-07T16:20:21.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:20 smithi100 bash[31660]: cluster 2024-04-07T16:20:19.794900+0000 mon.a (mon.0) 40 : cluster 0 osdmap e283: 8 total, 8 up, 8 in 2024-04-07T16:20:21.724 INFO:tasks.rados.rados.0.smithi100.stdout:1157: done (6 left) 2024-04-07T16:20:21.724 INFO:tasks.rados.rados.0.smithi100.stdout:1158: snap_remove snap 92 2024-04-07T16:20:22.105 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:21 smithi144 bash[23522]: cluster 2024-04-07T16:20:20.597978+0000 mgr.y (mgr.25175) 7 : cluster 0 pgmap v8: 121 pgs: 121 active+clean; 253 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail 2024-04-07T16:20:22.105 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:21 smithi144 bash[23522]: cluster 2024-04-07T16:20:20.813744+0000 mon.a (mon.0) 41 : cluster 0 mgrmap e40: y(active, since 4s), standbys: x 2024-04-07T16:20:22.105 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:21 smithi144 bash[23522]: cluster 2024-04-07T16:20:20.813942+0000 mon.a (mon.0) 42 : cluster 0 osdmap e284: 8 total, 8 up, 8 in 2024-04-07T16:20:22.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:21 smithi144 bash[23522]: cluster 2024-04-07T16:20:21.738898+0000 mon.a (mon.0) 43 : cluster 0 osdmap e285: 8 total, 8 up, 8 in 2024-04-07T16:20:22.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:21 smithi100 bash[74942]: cluster 2024-04-07T16:20:20.597978+0000 mgr.y (mgr.25175) 7 : cluster [DBG] pgmap v8: 121 pgs: 121 active+clean; 253 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail 2024-04-07T16:20:22.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:21 smithi100 bash[74942]: cluster 2024-04-07T16:20:20.813744+0000 mon.a (mon.0) 41 : cluster [DBG] mgrmap e40: y(active, since 4s), standbys: x 2024-04-07T16:20:22.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:21 smithi100 bash[74942]: cluster 2024-04-07T16:20:20.813942+0000 mon.a (mon.0) 42 : cluster [DBG] osdmap e284: 8 total, 8 up, 8 in 2024-04-07T16:20:22.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:21 smithi100 bash[74942]: cluster 2024-04-07T16:20:21.738898+0000 mon.a (mon.0) 43 : cluster [DBG] osdmap e285: 8 total, 8 up, 8 in 2024-04-07T16:20:22.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:21 smithi100 bash[31660]: cluster 2024-04-07T16:20:20.597978+0000 mgr.y (mgr.25175) 7 : cluster 0 pgmap v8: 121 pgs: 121 active+clean; 253 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail 2024-04-07T16:20:22.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:21 smithi100 bash[31660]: cluster 2024-04-07T16:20:20.813744+0000 mon.a (mon.0) 41 : cluster 0 mgrmap e40: y(active, since 4s), standbys: x 2024-04-07T16:20:22.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:21 smithi100 bash[31660]: cluster 2024-04-07T16:20:20.813942+0000 mon.a (mon.0) 42 : cluster 0 osdmap e284: 8 total, 8 up, 8 in 2024-04-07T16:20:22.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:21 smithi100 bash[31660]: cluster 2024-04-07T16:20:21.738898+0000 mon.a (mon.0) 43 : cluster 0 osdmap e285: 8 total, 8 up, 8 in 2024-04-07T16:20:22.605 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:20:22 smithi144 bash[59794]: ts=2024-04-07T16:20:22.313Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:20:22.727 INFO:tasks.rados.rados.0.smithi100.stdout:1158: done (6 left) 2024-04-07T16:20:22.728 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 30 current snap is 109 2024-04-07T16:20:22.728 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 2 v 638 (ObjNum 260 snap 70 seq_num 260) dirty exists 2024-04-07T16:20:22.728 INFO:tasks.rados.rados.0.smithi100.stdout:1155: finishing rollback tid 0 to smithi10066867-5 2024-04-07T16:20:22.728 INFO:tasks.rados.rados.0.smithi100.stdout:1155: finishing rollback tid 1 to smithi10066867-5 2024-04-07T16:20:22.728 INFO:tasks.rados.rados.0.smithi100.stdout:1155: finishing rollback tid 2 to smithi10066867-5 2024-04-07T16:20:22.728 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 5 v 597 (ObjNum 345 snap 102 seq_num 345) dirty exists 2024-04-07T16:20:22.728 INFO:tasks.rados.rados.0.smithi100.stdout:1153: finishing copy_from racing read to smithi10066867-44 2024-04-07T16:20:22.728 INFO:tasks.rados.rados.0.smithi100.stdout:1156: finishing write tid 1 to smithi10066867-16 2024-04-07T16:20:22.728 INFO:tasks.rados.rados.0.smithi100.stdout:1152: finishing copy_from racing read to smithi10066867-38 2024-04-07T16:20:22.728 INFO:tasks.rados.rados.0.smithi100.stdout:1156: finishing write tid 2 to smithi10066867-16 2024-04-07T16:20:22.728 INFO:tasks.rados.rados.0.smithi100.stdout:1159: seq_num 354 ranges {3350528=737280,4087808=401408} 2024-04-07T16:20:22.752 INFO:tasks.rados.rados.0.smithi100.stdout:1159: writing smithi10066867-30 from 3350528 to 4087808 tid 1 2024-04-07T16:20:22.755 INFO:tasks.rados.rados.0.smithi100.stdout:1159: writing smithi10066867-30 from 4087808 to 4489216 tid 2 2024-04-07T16:20:22.756 INFO:tasks.rados.rados.0.smithi100.stdout:1150: done (6 left) 2024-04-07T16:20:22.756 INFO:tasks.rados.rados.0.smithi100.stdout:1155: done (5 left) 2024-04-07T16:20:22.756 INFO:tasks.rados.rados.0.smithi100.stdout:1160: read oid 13 snap -1 2024-04-07T16:20:22.756 INFO:tasks.rados.rados.0.smithi100.stdout:1160: expect (ObjNum 302 snap 87 seq_num 302) 2024-04-07T16:20:22.756 INFO:tasks.rados.rados.0.smithi100.stdout:1161: delete oid 12 current snap is 109 2024-04-07T16:20:22.756 INFO:tasks.rados.rados.0.smithi100.stdout:1156: finishing write tid 3 to smithi10066867-16 2024-04-07T16:20:22.756 INFO:tasks.rados.rados.0.smithi100.stdout:1153: finishing copy_from to smithi10066867-44 2024-04-07T16:20:22.756 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 44 v 474 (ObjNum 322 snap 96 seq_num 322) dirty exists 2024-04-07T16:20:22.756 INFO:tasks.rados.rados.0.smithi100.stdout:1152: finishing copy_from to smithi10066867-38 2024-04-07T16:20:22.756 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 38 v 510 (ObjNum 328 snap 99 seq_num 328) dirty exists 2024-04-07T16:20:22.756 INFO:tasks.rados.rados.0.smithi100.stdout:1156: finishing write tid 4 to smithi10066867-16 2024-04-07T16:20:22.757 INFO:tasks.rados.rados.0.smithi100.stdout:1156: oid 16 updating version 0 to 601 2024-04-07T16:20:22.757 INFO:tasks.rados.rados.0.smithi100.stdout:1156: oid 16 version 601 is already newer than 600 2024-04-07T16:20:22.757 INFO:tasks.rados.rados.0.smithi100.stdout:1156: oid 16 version 601 is already newer than 599 2024-04-07T16:20:22.757 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 16 v 601 (ObjNum 353 snap 109 seq_num 353) dirty exists 2024-04-07T16:20:22.757 INFO:tasks.rados.rados.0.smithi100.stdout:1156: left oid 16 (ObjNum 353 snap 109 seq_num 353) 2024-04-07T16:20:22.757 INFO:tasks.rados.rados.0.smithi100.stdout:1151: finishing copy_from racing read to smithi10066867-46 2024-04-07T16:20:22.757 INFO:tasks.rados.rados.0.smithi100.stdout:1151: finishing copy_from to smithi10066867-46 2024-04-07T16:20:22.757 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 46 v 212 (ObjNum 335 snap 101 seq_num 335) dirty exists 2024-04-07T16:20:22.758 INFO:tasks.rados.rados.0.smithi100.stdout:1159: finishing write tid 1 to smithi10066867-30 2024-04-07T16:20:22.760 INFO:tasks.rados.rados.0.smithi100.stdout:1151: done (6 left) 2024-04-07T16:20:22.760 INFO:tasks.rados.rados.0.smithi100.stdout:1152: done (5 left) 2024-04-07T16:20:22.761 INFO:tasks.rados.rados.0.smithi100.stdout:1153: done (4 left) 2024-04-07T16:20:22.761 INFO:tasks.rados.rados.0.smithi100.stdout:1156: done (3 left) 2024-04-07T16:20:22.761 INFO:tasks.rados.rados.0.smithi100.stdout:1161: done (2 left) 2024-04-07T16:20:22.761 INFO:tasks.rados.rados.0.smithi100.stdout:1162: rollback oid 10 current snap is 109 2024-04-07T16:20:22.761 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 10 to 103 2024-04-07T16:20:22.761 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 8 current snap is 109 2024-04-07T16:20:22.761 INFO:tasks.rados.rados.0.smithi100.stdout:1163: seq_num 356 ranges {3874816=770048,4644864=401408} 2024-04-07T16:20:22.786 INFO:tasks.rados.rados.0.smithi100.stdout:1163: writing smithi10066867-8 from 3874816 to 4644864 tid 1 2024-04-07T16:20:22.790 INFO:tasks.rados.rados.0.smithi100.stdout:1163: writing smithi10066867-8 from 4644864 to 5046272 tid 2 2024-04-07T16:20:22.790 INFO:tasks.rados.rados.0.smithi100.stdout:1164: rollback oid 25 current snap is 109 2024-04-07T16:20:22.790 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 25 to 106 2024-04-07T16:20:22.790 INFO:tasks.rados.rados.0.smithi100.stdout:1159: finishing write tid 2 to smithi10066867-30 2024-04-07T16:20:22.790 INFO:tasks.rados.rados.0.smithi100.stdout:1159: finishing write tid 3 to smithi10066867-30 2024-04-07T16:20:22.790 INFO:tasks.rados.rados.0.smithi100.stdout:1159: finishing write tid 4 to smithi10066867-30 2024-04-07T16:20:22.790 INFO:tasks.rados.rados.0.smithi100.stdout:1159: oid 30 updating version 0 to 481 2024-04-07T16:20:22.790 INFO:tasks.rados.rados.0.smithi100.stdout:1159: oid 30 version 481 is already newer than 480 2024-04-07T16:20:22.791 INFO:tasks.rados.rados.0.smithi100.stdout:1159: oid 30 updating version 481 to 482 2024-04-07T16:20:22.791 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 30 v 482 (ObjNum 354 snap 109 seq_num 354) dirty exists 2024-04-07T16:20:22.791 INFO:tasks.rados.rados.0.smithi100.stdout:1159: left oid 30 (ObjNum 354 snap 109 seq_num 354) 2024-04-07T16:20:22.791 INFO:tasks.rados.rados.0.smithi100.stdout:1162: finishing rollback tid 0 to smithi10066867-10 2024-04-07T16:20:22.791 INFO:tasks.rados.rados.0.smithi100.stdout:1162: finishing rollback tid 1 to smithi10066867-10 2024-04-07T16:20:22.791 INFO:tasks.rados.rados.0.smithi100.stdout:1162: finishing rollback tid 2 to smithi10066867-10 2024-04-07T16:20:22.791 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 10 v 234 (ObjNum 296 snap 84 seq_num 296) dirty exists 2024-04-07T16:20:22.791 INFO:tasks.rados.rados.0.smithi100.stdout:1159: done (4 left) 2024-04-07T16:20:22.791 INFO:tasks.rados.rados.0.smithi100.stdout:1162: done (3 left) 2024-04-07T16:20:22.791 INFO:tasks.rados.rados.0.smithi100.stdout:1165: delete oid 4 current snap is 109 2024-04-07T16:20:22.791 INFO:tasks.rados.rados.0.smithi100.stdout:1164: finishing rollback tid 1 to smithi10066867-25 2024-04-07T16:20:22.791 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 25 v 0 (ObjNum 65 snap 0 seq_num 1701409901) dirty dne 2024-04-07T16:20:22.793 INFO:tasks.rados.rados.0.smithi100.stdout:1164: done (3 left) 2024-04-07T16:20:22.793 INFO:tasks.rados.rados.0.smithi100.stdout:1165: done (2 left) 2024-04-07T16:20:22.793 INFO:tasks.rados.rados.0.smithi100.stdout:1166: delete oid 41 current snap is 109 2024-04-07T16:20:22.795 INFO:tasks.rados.rados.0.smithi100.stdout:1166: done (2 left) 2024-04-07T16:20:22.795 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 3 current snap is 109 2024-04-07T16:20:22.795 INFO:tasks.rados.rados.0.smithi100.stdout:1167: seq_num 359 ranges {196608=417792,614400=417792,1032192=385024} 2024-04-07T16:20:22.799 INFO:tasks.rados.rados.0.smithi100.stdout:1167: writing smithi10066867-3 from 196608 to 614400 tid 1 2024-04-07T16:20:22.801 INFO:tasks.rados.rados.0.smithi100.stdout:1167: writing smithi10066867-3 from 614400 to 1032192 tid 2 2024-04-07T16:20:22.804 INFO:tasks.rados.rados.0.smithi100.stdout:1167: writing smithi10066867-3 from 1032192 to 1417216 tid 3 2024-04-07T16:20:22.804 INFO:tasks.rados.rados.0.smithi100.stdout:1168: rmattr oid 40 current snap is 109 2024-04-07T16:20:22.804 INFO:tasks.rados.rados.0.smithi100.stdout:1160: expect (ObjNum 302 snap 87 seq_num 302) 2024-04-07T16:20:23.419 INFO:tasks.rados.rados.0.smithi100.stdout:1163: finishing write tid 1 to smithi10066867-8 2024-04-07T16:20:23.419 INFO:tasks.rados.rados.0.smithi100.stdout:1163: finishing write tid 2 to smithi10066867-8 2024-04-07T16:20:23.419 INFO:tasks.rados.rados.0.smithi100.stdout:1163: finishing write tid 3 to smithi10066867-8 2024-04-07T16:20:23.419 INFO:tasks.rados.rados.0.smithi100.stdout:1163: finishing write tid 4 to smithi10066867-8 2024-04-07T16:20:23.419 INFO:tasks.rados.rados.0.smithi100.stdout:1163: oid 8 updating version 0 to 548 2024-04-07T16:20:23.419 INFO:tasks.rados.rados.0.smithi100.stdout:1163: oid 8 updating version 548 to 549 2024-04-07T16:20:23.419 INFO:tasks.rados.rados.0.smithi100.stdout:1163: oid 8 version 549 is already newer than 547 2024-04-07T16:20:23.419 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 8 v 549 (ObjNum 356 snap 109 seq_num 356) dirty exists 2024-04-07T16:20:23.419 INFO:tasks.rados.rados.0.smithi100.stdout:1163: left oid 8 (ObjNum 356 snap 109 seq_num 356) 2024-04-07T16:20:23.419 INFO:tasks.rados.rados.0.smithi100.stdout:1167: finishing write tid 1 to smithi10066867-3 2024-04-07T16:20:23.420 INFO:tasks.rados.rados.0.smithi100.stdout:1167: finishing write tid 2 to smithi10066867-3 2024-04-07T16:20:23.420 INFO:tasks.rados.rados.0.smithi100.stdout:1167: finishing write tid 3 to smithi10066867-3 2024-04-07T16:20:23.420 INFO:tasks.rados.rados.0.smithi100.stdout:1167: finishing write tid 4 to smithi10066867-3 2024-04-07T16:20:23.420 INFO:tasks.rados.rados.0.smithi100.stdout:1167: finishing write tid 5 to smithi10066867-3 2024-04-07T16:20:23.420 INFO:tasks.rados.rados.0.smithi100.stdout:1167: oid 3 updating version 0 to 513 2024-04-07T16:20:23.420 INFO:tasks.rados.rados.0.smithi100.stdout:1167: oid 3 updating version 513 to 514 2024-04-07T16:20:23.420 INFO:tasks.rados.rados.0.smithi100.stdout:1167: oid 3 version 514 is already newer than 511 2024-04-07T16:20:23.420 INFO:tasks.rados.rados.0.smithi100.stdout:1167: oid 3 version 514 is already newer than 512 2024-04-07T16:20:23.420 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 3 v 514 (ObjNum 359 snap 109 seq_num 359) dirty exists 2024-04-07T16:20:23.420 INFO:tasks.rados.rados.0.smithi100.stdout:1167: left oid 3 (ObjNum 359 snap 109 seq_num 359) 2024-04-07T16:20:23.420 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 40 v 639 (ObjNum 303 snap 87 seq_num 303) dirty exists 2024-04-07T16:20:23.420 INFO:tasks.rados.rados.0.smithi100.stdout:1160: done (3 left) 2024-04-07T16:20:23.420 INFO:tasks.rados.rados.0.smithi100.stdout:1163: done (2 left) 2024-04-07T16:20:23.420 INFO:tasks.rados.rados.0.smithi100.stdout:1167: done (1 left) 2024-04-07T16:20:23.420 INFO:tasks.rados.rados.0.smithi100.stdout:1168: done (0 left) 2024-04-07T16:20:23.420 INFO:tasks.rados.rados.0.smithi100.stdout:1169: read oid 25 snap 89 2024-04-07T16:20:23.421 INFO:tasks.rados.rados.0.smithi100.stdout:1169: expect deleted 2024-04-07T16:20:23.421 INFO:tasks.rados.rados.0.smithi100.stdout:1170: snap_remove snap 101 2024-04-07T16:20:23.856 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:23 smithi144 bash[23522]: cluster 2024-04-07T16:20:22.598641+0000 mgr.y (mgr.25175) 8 : cluster 0 pgmap v11: 121 pgs: 121 active+clean; 228 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 52 KiB/s rd, 1.8 MiB/s wr, 40 op/s 2024-04-07T16:20:23.856 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:23 smithi144 bash[23522]: cluster 2024-04-07T16:20:22.737508+0000 mon.a (mon.0) 44 : cluster 0 osdmap e286: 8 total, 8 up, 8 in 2024-04-07T16:20:23.856 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:23 smithi144 bash[23522]: audit 2024-04-07T16:20:22.854687+0000 mon.a (mon.0) 45 : audit 1 from='mgr.25175 ' entity='mgr.y' 2024-04-07T16:20:23.856 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:23 smithi144 bash[23522]: audit 2024-04-07T16:20:22.862420+0000 mon.a (mon.0) 46 : audit 1 from='mgr.25175 ' entity='mgr.y' 2024-04-07T16:20:23.856 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:23 smithi144 bash[23522]: audit 2024-04-07T16:20:23.164258+0000 mon.a (mon.0) 47 : audit 1 from='mgr.25175 ' entity='mgr.y' 2024-04-07T16:20:23.856 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:23 smithi144 bash[23522]: audit 2024-04-07T16:20:23.177950+0000 mon.a (mon.0) 48 : audit 1 from='mgr.25175 ' entity='mgr.y' 2024-04-07T16:20:23.856 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:20:23 smithi144 bash[59794]: ts=2024-04-07T16:20:23.507Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=nodes name=CephNodeDiskspaceWarning index=4 msg="Evaluating rule failed" rule="alert: CephNodeDiskspaceWarning\nexpr: predict_linear(node_filesystem_free_bytes{device=~\"/.*\"}[2d], 3600 * 24 * 5)\n * on (instance) group_left (nodename) node_uname_info < 0\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.8.4\n severity: warning\n type: ceph_default\nannotations:\n description: Mountpoint {{ $labels.mountpoint }} on {{ $labels.nodename }} will\n be full in less than 5 days based on the 48 hour trailing fill rate.\n summary: Host filesystem free space is getting low\n" err="found duplicate series for the match group {instance=\"smithi100\"} on the right hand-side of the operation: [{__name__=\"node_uname_info\", domainname=\"(none)\", instance=\"smithi100\", job=\"node\", machine=\"x86_64\", nodename=\"smithi100\", release=\"5.15.0-101-generic\", sysname=\"Linux\", version=\"#111-Ubuntu SMP Tue Mar 5 20:16:58 UTC 2024\"}, {__name__=\"node_uname_info\", cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", domainname=\"(none)\", instance=\"smithi100\", job=\"node\", machine=\"x86_64\", nodename=\"smithi100\", release=\"5.15.0-101-generic\", sysname=\"Linux\", version=\"#111-Ubuntu SMP Tue Mar 5 20:16:58 UTC 2024\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:20:24.162 INFO:tasks.rados.rados.0.smithi100.stdout:1170: done (1 left) 2024-04-07T16:20:24.162 INFO:tasks.rados.rados.0.smithi100.stdout:1171: read oid 2 snap -1 2024-04-07T16:20:24.162 INFO:tasks.rados.rados.0.smithi100.stdout:1171: expect (ObjNum 260 snap 70 seq_num 260) 2024-04-07T16:20:24.162 INFO:tasks.rados.rados.0.smithi100.stdout:1172: copy_from oid 3 from oid 24 current snap is 109 2024-04-07T16:20:24.162 INFO:tasks.rados.rados.0.smithi100.stdout:1173: rollback oid 50 current snap is 109 2024-04-07T16:20:24.162 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 50 to 105 2024-04-07T16:20:24.162 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 1 current snap is 109 2024-04-07T16:20:24.162 INFO:tasks.rados.rados.0.smithi100.stdout:1174: seq_num 360 ranges {2793472=434176,3227648=704512,3932160=98304} 2024-04-07T16:20:24.181 INFO:tasks.rados.rados.0.smithi100.stdout:1174: writing smithi10066867-1 from 2793472 to 3227648 tid 1 2024-04-07T16:20:24.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:23 smithi100 bash[74942]: cluster 2024-04-07T16:20:22.598641+0000 mgr.y (mgr.25175) 8 : cluster [DBG] pgmap v11: 121 pgs: 121 active+clean; 228 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 52 KiB/s rd, 1.8 MiB/s wr, 40 op/s 2024-04-07T16:20:24.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:23 smithi100 bash[74942]: cluster 2024-04-07T16:20:22.737508+0000 mon.a (mon.0) 44 : cluster [DBG] osdmap e286: 8 total, 8 up, 8 in 2024-04-07T16:20:24.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:23 smithi100 bash[74942]: audit 2024-04-07T16:20:22.854687+0000 mon.a (mon.0) 45 : audit [INF] from='mgr.25175 ' entity='mgr.y' 2024-04-07T16:20:24.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:23 smithi100 bash[74942]: audit 2024-04-07T16:20:22.862420+0000 mon.a (mon.0) 46 : audit [INF] from='mgr.25175 ' entity='mgr.y' 2024-04-07T16:20:24.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:23 smithi100 bash[74942]: audit 2024-04-07T16:20:23.164258+0000 mon.a (mon.0) 47 : audit [INF] from='mgr.25175 ' entity='mgr.y' 2024-04-07T16:20:24.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:23 smithi100 bash[74942]: audit 2024-04-07T16:20:23.177950+0000 mon.a (mon.0) 48 : audit [INF] from='mgr.25175 ' entity='mgr.y' 2024-04-07T16:20:24.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:23 smithi100 bash[31660]: cluster 2024-04-07T16:20:22.598641+0000 mgr.y (mgr.25175) 8 : cluster 0 pgmap v11: 121 pgs: 121 active+clean; 228 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 52 KiB/s rd, 1.8 MiB/s wr, 40 op/s 2024-04-07T16:20:24.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:23 smithi100 bash[31660]: cluster 2024-04-07T16:20:22.737508+0000 mon.a (mon.0) 44 : cluster 0 osdmap e286: 8 total, 8 up, 8 in 2024-04-07T16:20:24.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:23 smithi100 bash[31660]: audit 2024-04-07T16:20:22.854687+0000 mon.a (mon.0) 45 : audit 1 from='mgr.25175 ' entity='mgr.y' 2024-04-07T16:20:24.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:23 smithi100 bash[31660]: audit 2024-04-07T16:20:22.862420+0000 mon.a (mon.0) 46 : audit 1 from='mgr.25175 ' entity='mgr.y' 2024-04-07T16:20:24.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:23 smithi100 bash[31660]: audit 2024-04-07T16:20:23.164258+0000 mon.a (mon.0) 47 : audit 1 from='mgr.25175 ' entity='mgr.y' 2024-04-07T16:20:24.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:23 smithi100 bash[31660]: audit 2024-04-07T16:20:23.177950+0000 mon.a (mon.0) 48 : audit 1 from='mgr.25175 ' entity='mgr.y' 2024-04-07T16:20:24.188 INFO:tasks.rados.rados.0.smithi100.stdout:1174: writing smithi10066867-1 from 3227648 to 3932160 tid 2 2024-04-07T16:20:24.188 INFO:tasks.rados.rados.0.smithi100.stdout:1174: writing smithi10066867-1 from 3932160 to 4030464 tid 3 2024-04-07T16:20:24.188 INFO:tasks.rados.rados.0.smithi100.stdout:1173: finishing rollback tid 1 to smithi10066867-50 2024-04-07T16:20:24.188 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 50 v 517 (ObjNum 289 snap 0 seq_num 0) dirty dne 2024-04-07T16:20:24.188 INFO:tasks.rados.rados.0.smithi100.stdout:1169: done (4 left) 2024-04-07T16:20:24.188 INFO:tasks.rados.rados.0.smithi100.stdout:1173: done (3 left) 2024-04-07T16:20:24.188 INFO:tasks.rados.rados.0.smithi100.stdout:1175: delete oid 48 current snap is 109 2024-04-07T16:20:24.189 INFO:tasks.rados.rados.0.smithi100.stdout:1174: finishing write tid 1 to smithi10066867-1 2024-04-07T16:20:24.193 INFO:tasks.rados.rados.0.smithi100.stdout:1174: finishing write tid 2 to smithi10066867-1 2024-04-07T16:20:24.193 INFO:tasks.rados.rados.0.smithi100.stdout:1171: expect (ObjNum 260 snap 70 seq_num 260) 2024-04-07T16:20:24.250 INFO:tasks.rados.rados.0.smithi100.stdout:1174: finishing write tid 3 to smithi10066867-1 2024-04-07T16:20:24.250 INFO:tasks.rados.rados.0.smithi100.stdout:1174: finishing write tid 4 to smithi10066867-1 2024-04-07T16:20:24.250 INFO:tasks.rados.rados.0.smithi100.stdout:1174: finishing write tid 5 to smithi10066867-1 2024-04-07T16:20:24.250 INFO:tasks.rados.rados.0.smithi100.stdout:1174: oid 1 updating version 0 to 238 2024-04-07T16:20:24.250 INFO:tasks.rados.rados.0.smithi100.stdout:1174: oid 1 version 238 is already newer than 236 2024-04-07T16:20:24.250 INFO:tasks.rados.rados.0.smithi100.stdout:1174: oid 1 version 238 is already newer than 235 2024-04-07T16:20:24.250 INFO:tasks.rados.rados.0.smithi100.stdout:1174: oid 1 version 238 is already newer than 237 2024-04-07T16:20:24.250 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 1 v 238 (ObjNum 360 snap 109 seq_num 360) dirty exists 2024-04-07T16:20:24.250 INFO:tasks.rados.rados.0.smithi100.stdout:1174: left oid 1 (ObjNum 360 snap 109 seq_num 360) 2024-04-07T16:20:24.250 INFO:tasks.rados.rados.0.smithi100.stdout:1172: finishing copy_from racing read to smithi10066867-3 2024-04-07T16:20:24.251 INFO:tasks.rados.rados.0.smithi100.stdout:1172: finishing copy_from to smithi10066867-3 2024-04-07T16:20:24.251 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 3 v 537 (ObjNum 330 snap 99 seq_num 330) dirty exists 2024-04-07T16:20:24.251 INFO:tasks.rados.rados.0.smithi100.stdout:1171: done (3 left) 2024-04-07T16:20:24.251 INFO:tasks.rados.rados.0.smithi100.stdout:1172: done (2 left) 2024-04-07T16:20:24.251 INFO:tasks.rados.rados.0.smithi100.stdout:1174: done (1 left) 2024-04-07T16:20:24.251 INFO:tasks.rados.rados.0.smithi100.stdout:1175: done (0 left) 2024-04-07T16:20:24.251 INFO:tasks.rados.rados.0.smithi100.stdout:1176: rollback oid 25 current snap is 109 2024-04-07T16:20:24.251 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 25 to 99 2024-04-07T16:20:24.251 INFO:tasks.rados.rados.0.smithi100.stdout:1177: snap_remove snap 105 2024-04-07T16:20:24.302 INFO:teuthology.orchestra.run.smithi100.stdout:true 2024-04-07T16:20:24.640 INFO:teuthology.orchestra.run.smithi100.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-07T16:20:24.640 INFO:teuthology.orchestra.run.smithi100.stdout:alertmanager.a smithi100 *:9093,9094 running (6m) 1s ago 7m 14.0M - 0.25.0 c8568f914cd2 5143075e7d26 2024-04-07T16:20:24.640 INFO:teuthology.orchestra.run.smithi100.stdout:grafana.a smithi144 *:3000 running (6m) 1s ago 6m 79.4M - 9.4.7 954c08fa6188 7180c0708acd 2024-04-07T16:20:24.641 INFO:teuthology.orchestra.run.smithi100.stdout:mds.a smithi100 running (8m) 1s ago 8m 14.9M - 18.2.2-1171-g267c8ff5 8a4a01e7186c 92593b0c28ae 2024-04-07T16:20:24.641 INFO:teuthology.orchestra.run.smithi100.stdout:mds.b smithi144 running (8m) 1s ago 8m 17.8M - 18.2.2-1171-g267c8ff5 8a4a01e7186c 3df2ee3e17a4 2024-04-07T16:20:24.641 INFO:teuthology.orchestra.run.smithi100.stdout:mgr.x smithi144 *:8443,9283,8765 running (42s) 1s ago 15m 436M - 19.0.0-2785-gadce92d0 c0ecd4226ac9 233745c292a6 2024-04-07T16:20:24.641 INFO:teuthology.orchestra.run.smithi100.stdout:mgr.y smithi100 *:8443,9283,8765 running (87s) 1s ago 17m 474M - 19.0.0-2785-gadce92d0 c0ecd4226ac9 254dc65cdaa7 2024-04-07T16:20:24.641 INFO:teuthology.orchestra.run.smithi100.stdout:mon.a smithi100 running (18s) 1s ago 18m 32.5M 2048M 19.0.0-2785-gadce92d0 c0ecd4226ac9 718e12655c04 2024-04-07T16:20:24.641 INFO:teuthology.orchestra.run.smithi100.stdout:mon.b smithi144 running (15m) 1s ago 15m 52.2M 2048M 18.2.2-1171-g267c8ff5 8a4a01e7186c 8dd687691b2e 2024-04-07T16:20:24.641 INFO:teuthology.orchestra.run.smithi100.stdout:mon.c smithi100 running (15m) 1s ago 15m 48.0M 2048M 18.2.2-1171-g267c8ff5 8a4a01e7186c a6de0151a3ba 2024-04-07T16:20:24.641 INFO:teuthology.orchestra.run.smithi100.stdout:node-exporter.a smithi100 *:9100 running (7m) 1s ago 7m 8796k - 1.5.0 0da6a335fe13 4bce264c1b26 2024-04-07T16:20:24.641 INFO:teuthology.orchestra.run.smithi100.stdout:node-exporter.b smithi144 *:9100 running (7m) 1s ago 7m 8328k - 1.5.0 0da6a335fe13 b6963413c560 2024-04-07T16:20:24.642 INFO:teuthology.orchestra.run.smithi100.stdout:osd.0 smithi100 running (14m) 1s ago 14m 230M 2524M 18.2.2-1171-g267c8ff5 8a4a01e7186c 188deb247979 2024-04-07T16:20:24.642 INFO:teuthology.orchestra.run.smithi100.stdout:osd.1 smithi100 running (13m) 1s ago 13m 178M 2524M 18.2.2-1171-g267c8ff5 8a4a01e7186c 7f57ca90894c 2024-04-07T16:20:24.642 INFO:teuthology.orchestra.run.smithi100.stdout:osd.2 smithi100 running (13m) 1s ago 13m 220M 2524M 18.2.2-1171-g267c8ff5 8a4a01e7186c f3912ddcc8cc 2024-04-07T16:20:24.642 INFO:teuthology.orchestra.run.smithi100.stdout:osd.3 smithi100 running (12m) 1s ago 12m 153M 2524M 18.2.2-1171-g267c8ff5 8a4a01e7186c b41004090c63 2024-04-07T16:20:24.642 INFO:teuthology.orchestra.run.smithi100.stdout:osd.4 smithi144 running (11m) 1s ago 11m 227M 2524M 18.2.2-1171-g267c8ff5 8a4a01e7186c 8c73b6b4dc13 2024-04-07T16:20:24.642 INFO:teuthology.orchestra.run.smithi100.stdout:osd.5 smithi144 running (10m) 1s ago 10m 191M 2524M 18.2.2-1171-g267c8ff5 8a4a01e7186c 3bfa0cf6f6cb 2024-04-07T16:20:24.642 INFO:teuthology.orchestra.run.smithi100.stdout:osd.6 smithi144 running (9m) 1s ago 9m 88.9M 2524M 18.2.2-1171-g267c8ff5 8a4a01e7186c 32b7f95d77f0 2024-04-07T16:20:24.643 INFO:teuthology.orchestra.run.smithi100.stdout:osd.7 smithi144 running (9m) 1s ago 9m 123M 2524M 18.2.2-1171-g267c8ff5 8a4a01e7186c 45f7709e576a 2024-04-07T16:20:24.643 INFO:teuthology.orchestra.run.smithi100.stdout:prometheus.a smithi144 *:9095 running (47s) 1s ago 7m 42.6M - 2.43.0 a07b618ecd1d 2d0a8131f303 2024-04-07T16:20:25.062 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-04-07T16:20:25.062 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": { 2024-04-07T16:20:25.063 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph version 18.2.2-1171-g267c8ff5 (267c8ff5b1ff5815581b3b2381d19360e081b37b) reef (stable)": 2, 2024-04-07T16:20:25.063 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph version 19.0.0-2785-gadce92d0 (adce92d056533f22dc1ee37e3141d098b046ec1b) squid (dev)": 1 2024-04-07T16:20:25.063 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-07T16:20:25.063 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": { 2024-04-07T16:20:25.063 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph version 19.0.0-2785-gadce92d0 (adce92d056533f22dc1ee37e3141d098b046ec1b) squid (dev)": 2 2024-04-07T16:20:25.063 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-07T16:20:25.063 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": { 2024-04-07T16:20:25.063 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph version 18.2.2-1171-g267c8ff5 (267c8ff5b1ff5815581b3b2381d19360e081b37b) reef (stable)": 8 2024-04-07T16:20:25.063 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-07T16:20:25.063 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": { 2024-04-07T16:20:25.063 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph version 18.2.2-1171-g267c8ff5 (267c8ff5b1ff5815581b3b2381d19360e081b37b) reef (stable)": 2 2024-04-07T16:20:25.063 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-07T16:20:25.063 INFO:teuthology.orchestra.run.smithi100.stdout: "overall": { 2024-04-07T16:20:25.063 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph version 18.2.2-1171-g267c8ff5 (267c8ff5b1ff5815581b3b2381d19360e081b37b) reef (stable)": 12, 2024-04-07T16:20:25.064 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph version 19.0.0-2785-gadce92d0 (adce92d056533f22dc1ee37e3141d098b046ec1b) squid (dev)": 3 2024-04-07T16:20:25.064 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-04-07T16:20:25.064 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-04-07T16:20:25.250 INFO:tasks.rados.rados.0.smithi100.stdout:1176: finishing rollback tid 1 to smithi10066867-25 2024-04-07T16:20:25.251 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 25 v 0 (ObjNum 65 snap 0 seq_num 1599102315) dirty dne 2024-04-07T16:20:25.251 INFO:tasks.rados.rados.0.smithi100.stdout:1176: done (1 left) 2024-04-07T16:20:25.251 INFO:tasks.rados.rados.0.smithi100.stdout:1177: done (0 left) 2024-04-07T16:20:25.251 INFO:tasks.rados.rados.0.smithi100.stdout:1178: rmattr oid 1 current snap is 109 2024-04-07T16:20:25.251 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 16 current snap is 109 2024-04-07T16:20:25.251 INFO:tasks.rados.rados.0.smithi100.stdout:1179: seq_num 362 ranges {1122304=466944,1589248=466944,2056192=335872} 2024-04-07T16:20:25.261 INFO:tasks.rados.rados.0.smithi100.stdout:1179: writing smithi10066867-16 from 1122304 to 1589248 tid 1 2024-04-07T16:20:25.265 INFO:tasks.rados.rados.0.smithi100.stdout:1179: writing smithi10066867-16 from 1589248 to 2056192 tid 2 2024-04-07T16:20:25.269 INFO:tasks.rados.rados.0.smithi100.stdout:1179: writing smithi10066867-16 from 2056192 to 2392064 tid 3 2024-04-07T16:20:25.269 INFO:tasks.rados.rados.0.smithi100.stdout:1180: rollback oid 46 current snap is 109 2024-04-07T16:20:25.269 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 46 to 103 2024-04-07T16:20:25.269 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 1 v 239 (ObjNum 360 snap 109 seq_num 360) dirty exists 2024-04-07T16:20:25.269 INFO:tasks.rados.rados.0.smithi100.stdout:1178: done (2 left) 2024-04-07T16:20:25.269 INFO:tasks.rados.rados.0.smithi100.stdout:1181: rmattr oid 43 current snap is 109 2024-04-07T16:20:25.269 INFO:tasks.rados.rados.0.smithi100.stdout:1181: done (2 left) 2024-04-07T16:20:25.269 INFO:tasks.rados.rados.0.smithi100.stdout:1182: delete oid 8 current snap is 109 2024-04-07T16:20:25.283 INFO:tasks.rados.rados.0.smithi100.stdout:1179: finishing write tid 1 to smithi10066867-16 2024-04-07T16:20:25.285 INFO:tasks.rados.rados.0.smithi100.stdout:1180: finishing rollback tid 0 to smithi10066867-46 2024-04-07T16:20:25.286 INFO:tasks.rados.rados.0.smithi100.stdout:1180: finishing rollback tid 1 to smithi10066867-46 2024-04-07T16:20:25.287 INFO:tasks.rados.rados.0.smithi100.stdout:1179: finishing write tid 2 to smithi10066867-16 2024-04-07T16:20:25.287 INFO:tasks.rados.rados.0.smithi100.stdout:1179: finishing write tid 3 to smithi10066867-16 2024-04-07T16:20:25.289 INFO:tasks.rados.rados.0.smithi100.stdout:1180: finishing rollback tid 2 to smithi10066867-46 2024-04-07T16:20:25.289 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 46 v 552 (ObjNum 337 snap 101 seq_num 337) dirty exists 2024-04-07T16:20:25.290 INFO:tasks.rados.rados.0.smithi100.stdout:1180: done (2 left) 2024-04-07T16:20:25.290 INFO:tasks.rados.rados.0.smithi100.stdout:1182: done (1 left) 2024-04-07T16:20:25.290 INFO:tasks.rados.rados.0.smithi100.stdout:1183: read oid 17 snap -1 2024-04-07T16:20:25.291 INFO:tasks.rados.rados.0.smithi100.stdout:1183: expect (ObjNum 303 snap 87 seq_num 303) 2024-04-07T16:20:25.291 INFO:tasks.rados.rados.0.smithi100.stdout:1184: read oid 4 snap -1 2024-04-07T16:20:25.291 INFO:tasks.rados.rados.0.smithi100.stdout:1184: expect deleted 2024-04-07T16:20:25.291 INFO:tasks.rados.rados.0.smithi100.stdout:1185: setattr oid 19 current snap is 109 2024-04-07T16:20:25.293 INFO:tasks.rados.rados.0.smithi100.stdout:1179: finishing write tid 4 to smithi10066867-16 2024-04-07T16:20:25.295 INFO:tasks.rados.rados.0.smithi100.stdout:1179: finishing write tid 5 to smithi10066867-16 2024-04-07T16:20:25.295 INFO:tasks.rados.rados.0.smithi100.stdout:1179: oid 16 updating version 0 to 628 2024-04-07T16:20:25.295 INFO:tasks.rados.rados.0.smithi100.stdout:1179: oid 16 updating version 628 to 629 2024-04-07T16:20:25.295 INFO:tasks.rados.rados.0.smithi100.stdout:1179: oid 16 version 629 is already newer than 627 2024-04-07T16:20:25.295 INFO:tasks.rados.rados.0.smithi100.stdout:1179: oid 16 version 629 is already newer than 622 2024-04-07T16:20:25.295 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 16 v 629 (ObjNum 362 snap 109 seq_num 362) dirty exists 2024-04-07T16:20:25.295 INFO:tasks.rados.rados.0.smithi100.stdout:1179: left oid 16 (ObjNum 362 snap 109 seq_num 362) 2024-04-07T16:20:25.297 INFO:tasks.rados.rados.0.smithi100.stdout:1179: done (3 left) 2024-04-07T16:20:25.298 INFO:tasks.rados.rados.0.smithi100.stdout:1184: done (2 left) 2024-04-07T16:20:25.298 INFO:tasks.rados.rados.0.smithi100.stdout:1186: snap_create 2024-04-07T16:20:25.323 INFO:tasks.rados.rados.0.smithi100.stdout:1183: expect (ObjNum 303 snap 87 seq_num 303) 2024-04-07T16:20:25.436 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:25 smithi100 bash[31660]: cluster 2024-04-07T16:20:24.177504+0000 mon.a (mon.0) 49 : cluster 0 osdmap e287: 8 total, 8 up, 8 in 2024-04-07T16:20:25.436 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:25 smithi100 bash[31660]: audit 2024-04-07T16:20:24.237918+0000 mon.a (mon.0) 50 : audit 1 from='mgr.25175 ' entity='mgr.y' 2024-04-07T16:20:25.436 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:25 smithi100 bash[31660]: audit 2024-04-07T16:20:24.252171+0000 mon.a (mon.0) 51 : audit 1 from='mgr.25175 ' entity='mgr.y' 2024-04-07T16:20:25.436 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:25 smithi100 bash[31660]: audit 2024-04-07T16:20:24.290592+0000 mon.a (mon.0) 52 : audit 1 from='mgr.25175 ' entity='mgr.y' 2024-04-07T16:20:25.436 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:25 smithi100 bash[31660]: audit 2024-04-07T16:20:24.302303+0000 mon.a (mon.0) 53 : audit 1 from='mgr.25175 ' entity='mgr.y' 2024-04-07T16:20:25.436 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:25 smithi100 bash[31660]: audit 2024-04-07T16:20:25.078817+0000 mon.a (mon.0) 54 : audit 0 from='client.? 172.21.15.100:0/3968431244' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:20:25.436 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:25 smithi100 bash[74942]: cluster 2024-04-07T16:20:24.177504+0000 mon.a (mon.0) 49 : cluster [DBG] osdmap e287: 8 total, 8 up, 8 in 2024-04-07T16:20:25.436 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:25 smithi100 bash[74942]: audit 2024-04-07T16:20:24.237918+0000 mon.a (mon.0) 50 : audit [INF] from='mgr.25175 ' entity='mgr.y' 2024-04-07T16:20:25.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:25 smithi100 bash[74942]: audit 2024-04-07T16:20:24.252171+0000 mon.a (mon.0) 51 : audit [INF] from='mgr.25175 ' entity='mgr.y' 2024-04-07T16:20:25.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:25 smithi100 bash[74942]: audit 2024-04-07T16:20:24.290592+0000 mon.a (mon.0) 52 : audit [INF] from='mgr.25175 ' entity='mgr.y' 2024-04-07T16:20:25.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:25 smithi100 bash[74942]: audit 2024-04-07T16:20:24.302303+0000 mon.a (mon.0) 53 : audit [INF] from='mgr.25175 ' entity='mgr.y' 2024-04-07T16:20:25.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:25 smithi100 bash[74942]: audit 2024-04-07T16:20:25.078817+0000 mon.a (mon.0) 54 : audit [DBG] from='client.? 172.21.15.100:0/3968431244' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:20:25.605 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:25 smithi144 bash[23522]: cluster 2024-04-07T16:20:24.177504+0000 mon.a (mon.0) 49 : cluster 0 osdmap e287: 8 total, 8 up, 8 in 2024-04-07T16:20:25.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:25 smithi144 bash[23522]: audit 2024-04-07T16:20:24.237918+0000 mon.a (mon.0) 50 : audit 1 from='mgr.25175 ' entity='mgr.y' 2024-04-07T16:20:25.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:25 smithi144 bash[23522]: audit 2024-04-07T16:20:24.252171+0000 mon.a (mon.0) 51 : audit 1 from='mgr.25175 ' entity='mgr.y' 2024-04-07T16:20:25.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:25 smithi144 bash[23522]: audit 2024-04-07T16:20:24.290592+0000 mon.a (mon.0) 52 : audit 1 from='mgr.25175 ' entity='mgr.y' 2024-04-07T16:20:25.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:25 smithi144 bash[23522]: audit 2024-04-07T16:20:24.302303+0000 mon.a (mon.0) 53 : audit 1 from='mgr.25175 ' entity='mgr.y' 2024-04-07T16:20:25.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:25 smithi144 bash[23522]: audit 2024-04-07T16:20:25.078817+0000 mon.a (mon.0) 54 : audit 0 from='client.? 172.21.15.100:0/3968431244' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:20:25.780 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 19 v 542 (ObjNum 239 snap 65 seq_num 239) dirty exists 2024-04-07T16:20:26.300 INFO:tasks.rados.rados.0.smithi100.stdout:1183: done (2 left) 2024-04-07T16:20:26.300 INFO:tasks.rados.rados.0.smithi100.stdout:1185: done (1 left) 2024-04-07T16:20:26.300 INFO:tasks.rados.rados.0.smithi100.stdout:1186: done (0 left) 2024-04-07T16:20:26.300 INFO:tasks.rados.rados.0.smithi100.stdout:1187: setattr oid 40 current snap is 110 2024-04-07T16:20:26.307 INFO:tasks.rados.rados.0.smithi100.stdout:1188: read oid 8 snap -1 2024-04-07T16:20:26.307 INFO:tasks.rados.rados.0.smithi100.stdout:1188: expect deleted 2024-04-07T16:20:26.307 INFO:tasks.rados.rados.0.smithi100.stdout:1189: snap_remove snap 102 2024-04-07T16:20:26.605 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:26 smithi144 bash[23522]: audit 2024-04-07T16:20:24.305250+0000 mgr.y (mgr.25175) 9 : audit 0 from='client.34149 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:20:26.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:26 smithi144 bash[23522]: cluster 2024-04-07T16:20:24.599047+0000 mgr.y (mgr.25175) 10 : cluster 0 pgmap v14: 121 pgs: 121 active+clean; 228 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 52 KiB/s rd, 1.8 MiB/s wr, 40 op/s 2024-04-07T16:20:26.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:26 smithi144 bash[23522]: audit 2024-04-07T16:20:24.637597+0000 mgr.y (mgr.25175) 11 : audit 0 from='client.25217 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:20:26.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:26 smithi144 bash[23522]: cluster 2024-04-07T16:20:25.262744+0000 mon.a (mon.0) 55 : cluster 0 osdmap e288: 8 total, 8 up, 8 in 2024-04-07T16:20:26.686 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:26 smithi100 bash[74942]: audit 2024-04-07T16:20:24.305250+0000 mgr.y (mgr.25175) 9 : audit [DBG] from='client.34149 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:20:26.686 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:26 smithi100 bash[74942]: cluster 2024-04-07T16:20:24.599047+0000 mgr.y (mgr.25175) 10 : cluster [DBG] pgmap v14: 121 pgs: 121 active+clean; 228 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 52 KiB/s rd, 1.8 MiB/s wr, 40 op/s 2024-04-07T16:20:26.686 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:26 smithi100 bash[74942]: audit 2024-04-07T16:20:24.637597+0000 mgr.y (mgr.25175) 11 : audit [DBG] from='client.25217 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:20:26.686 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:26 smithi100 bash[74942]: cluster 2024-04-07T16:20:25.262744+0000 mon.a (mon.0) 55 : cluster [DBG] osdmap e288: 8 total, 8 up, 8 in 2024-04-07T16:20:26.686 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:26 smithi100 bash[31660]: audit 2024-04-07T16:20:24.305250+0000 mgr.y (mgr.25175) 9 : audit 0 from='client.34149 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:20:26.687 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:26 smithi100 bash[31660]: cluster 2024-04-07T16:20:24.599047+0000 mgr.y (mgr.25175) 10 : cluster 0 pgmap v14: 121 pgs: 121 active+clean; 228 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 52 KiB/s rd, 1.8 MiB/s wr, 40 op/s 2024-04-07T16:20:26.687 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:26 smithi100 bash[31660]: audit 2024-04-07T16:20:24.637597+0000 mgr.y (mgr.25175) 11 : audit 0 from='client.25217 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:20:26.687 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:26 smithi100 bash[31660]: cluster 2024-04-07T16:20:25.262744+0000 mon.a (mon.0) 55 : cluster 0 osdmap e288: 8 total, 8 up, 8 in 2024-04-07T16:20:26.733 INFO:tasks.rados.rados.0.smithi100.stdout:1189: done (2 left) 2024-04-07T16:20:26.733 INFO:tasks.rados.rados.0.smithi100.stdout:1190: read oid 6 snap -1 2024-04-07T16:20:26.733 INFO:tasks.rados.rados.0.smithi100.stdout:1190: expect (ObjNum 324 snap 97 seq_num 324) 2024-04-07T16:20:26.733 INFO:tasks.rados.rados.0.smithi100.stdout:1191: snap_remove snap 99 2024-04-07T16:20:27.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:27 smithi144 bash[23522]: cluster 2024-04-07T16:20:26.312199+0000 mon.a (mon.0) 56 : cluster 0 osdmap e289: 8 total, 8 up, 8 in 2024-04-07T16:20:27.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:27 smithi144 bash[23522]: cluster 2024-04-07T16:20:26.599736+0000 mgr.y (mgr.25175) 12 : cluster 0 pgmap v17: 121 pgs: 121 active+clean; 228 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail 2024-04-07T16:20:27.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:27 smithi144 bash[23522]: cluster 2024-04-07T16:20:26.740580+0000 mon.a (mon.0) 57 : cluster 0 osdmap e290: 8 total, 8 up, 8 in 2024-04-07T16:20:27.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:27 smithi144 bash[23522]: audit 2024-04-07T16:20:26.747621+0000 mon.c (mon.2) 155 : audit 0 from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:20:27.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:27 smithi144 bash[23522]: audit 2024-04-07T16:20:26.783361+0000 mon.c (mon.2) 156 : audit 0 from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config-key dump", "format": "json"}]: dispatch 2024-04-07T16:20:27.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:27 smithi144 bash[23522]: audit 2024-04-07T16:20:26.899288+0000 mon.c (mon.2) 157 : audit 0 from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:20:27.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:27 smithi144 bash[23522]: audit 2024-04-07T16:20:26.899949+0000 mon.c (mon.2) 158 : audit 0 from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:20:27.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:27 smithi144 bash[23522]: audit 2024-04-07T16:20:26.900402+0000 mon.c (mon.2) 159 : audit 0 from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:20:27.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:27 smithi144 bash[23522]: audit 2024-04-07T16:20:26.903490+0000 mon.c (mon.2) 160 : audit 0 from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:20:27.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:27 smithi144 bash[23522]: audit 2024-04-07T16:20:26.903709+0000 mon.c (mon.2) 161 : audit 0 from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:20:27.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:27 smithi144 bash[23522]: audit 2024-04-07T16:20:26.904020+0000 mon.c (mon.2) 162 : audit 0 from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:20:27.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:27 smithi144 bash[23522]: audit 2024-04-07T16:20:26.906217+0000 mon.c (mon.2) 163 : audit 0 from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:20:27.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:27 smithi144 bash[23522]: audit 2024-04-07T16:20:26.906412+0000 mon.c (mon.2) 164 : audit 0 from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:20:27.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:27 smithi144 bash[23522]: audit 2024-04-07T16:20:26.906722+0000 mon.c (mon.2) 165 : audit 0 from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:20:27.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:27 smithi144 bash[23522]: audit 2024-04-07T16:20:26.919690+0000 mon.b (mon.1) 335 : audit 1 from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T16:20:27.608 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:27 smithi144 bash[23522]: audit 2024-04-07T16:20:26.919797+0000 mon.b (mon.1) 336 : audit 1 from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T16:20:27.608 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:27 smithi144 bash[23522]: audit 2024-04-07T16:20:26.921172+0000 mon.a (mon.0) 58 : audit 1 from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T16:20:27.608 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:27 smithi144 bash[23522]: audit 2024-04-07T16:20:26.921284+0000 mon.a (mon.0) 59 : audit 1 from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T16:20:27.608 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:27 smithi144 bash[23522]: audit 2024-04-07T16:20:26.921562+0000 mon.c (mon.2) 166 : audit 0 from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:20:27.608 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:27 smithi144 bash[23522]: audit 2024-04-07T16:20:26.922161+0000 mon.c (mon.2) 167 : audit 0 from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:20:27.608 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:27 smithi144 bash[23522]: audit 2024-04-07T16:20:26.922699+0000 mon.c (mon.2) 168 : audit 0 from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:20:27.608 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:27 smithi144 bash[23522]: audit 2024-04-07T16:20:26.925820+0000 mon.c (mon.2) 169 : audit 0 from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:20:27.608 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:27 smithi144 bash[23522]: audit 2024-04-07T16:20:26.926485+0000 mon.c (mon.2) 170 : audit 0 from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:20:27.608 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:27 smithi144 bash[23522]: audit 2024-04-07T16:20:26.927032+0000 mon.c (mon.2) 171 : audit 0 from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:20:27.608 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:27 smithi144 bash[23522]: audit 2024-04-07T16:20:26.929732+0000 mon.c (mon.2) 172 : audit 1 from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T16:20:27.608 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:27 smithi144 bash[23522]: audit 2024-04-07T16:20:26.929880+0000 mon.c (mon.2) 173 : audit 1 from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T16:20:27.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:27 smithi144 bash[23522]: audit 2024-04-07T16:20:26.930051+0000 mon.c (mon.2) 174 : audit 0 from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:20:27.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:27 smithi144 bash[23522]: audit 2024-04-07T16:20:26.931086+0000 mon.c (mon.2) 175 : audit 0 from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:20:27.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:27 smithi144 bash[23522]: audit 2024-04-07T16:20:26.931627+0000 mon.c (mon.2) 176 : audit 0 from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:20:27.687 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:27 smithi100 bash[74942]: cluster 2024-04-07T16:20:26.312199+0000 mon.a (mon.0) 56 : cluster [DBG] osdmap e289: 8 total, 8 up, 8 in 2024-04-07T16:20:27.687 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:27 smithi100 bash[74942]: cluster 2024-04-07T16:20:26.599736+0000 mgr.y (mgr.25175) 12 : cluster [DBG] pgmap v17: 121 pgs: 121 active+clean; 228 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail 2024-04-07T16:20:27.687 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:27 smithi100 bash[74942]: cluster 2024-04-07T16:20:26.740580+0000 mon.a (mon.0) 57 : cluster [DBG] osdmap e290: 8 total, 8 up, 8 in 2024-04-07T16:20:27.687 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:27 smithi100 bash[74942]: audit 2024-04-07T16:20:26.747621+0000 mon.c (mon.2) 155 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:20:27.687 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:27 smithi100 bash[74942]: audit 2024-04-07T16:20:26.783361+0000 mon.c (mon.2) 156 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config-key dump", "format": "json"}]: dispatch 2024-04-07T16:20:27.687 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:27 smithi100 bash[74942]: audit 2024-04-07T16:20:26.899288+0000 mon.c (mon.2) 157 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:20:27.687 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:27 smithi100 bash[74942]: audit 2024-04-07T16:20:26.899949+0000 mon.c (mon.2) 158 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:20:27.687 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:27 smithi100 bash[74942]: audit 2024-04-07T16:20:26.900402+0000 mon.c (mon.2) 159 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:20:27.688 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:27 smithi100 bash[74942]: audit 2024-04-07T16:20:26.903490+0000 mon.c (mon.2) 160 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:20:27.688 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:27 smithi100 bash[74942]: audit 2024-04-07T16:20:26.903709+0000 mon.c (mon.2) 161 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:20:27.688 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:27 smithi100 bash[74942]: audit 2024-04-07T16:20:26.904020+0000 mon.c (mon.2) 162 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:20:27.688 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:27 smithi100 bash[74942]: audit 2024-04-07T16:20:26.906217+0000 mon.c (mon.2) 163 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:20:27.688 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:27 smithi100 bash[74942]: audit 2024-04-07T16:20:26.906412+0000 mon.c (mon.2) 164 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:20:27.688 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:27 smithi100 bash[74942]: audit 2024-04-07T16:20:26.906722+0000 mon.c (mon.2) 165 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:20:27.688 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:27 smithi100 bash[74942]: audit 2024-04-07T16:20:26.919690+0000 mon.b (mon.1) 335 : audit [INF] from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T16:20:27.688 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:27 smithi100 bash[74942]: audit 2024-04-07T16:20:26.919797+0000 mon.b (mon.1) 336 : audit [INF] from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T16:20:27.688 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:27 smithi100 bash[74942]: audit 2024-04-07T16:20:26.921172+0000 mon.a (mon.0) 58 : audit [INF] from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T16:20:27.688 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:27 smithi100 bash[74942]: audit 2024-04-07T16:20:26.921284+0000 mon.a (mon.0) 59 : audit [INF] from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T16:20:27.688 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:27 smithi100 bash[74942]: audit 2024-04-07T16:20:26.921562+0000 mon.c (mon.2) 166 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:20:27.689 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:27 smithi100 bash[31660]: cluster 2024-04-07T16:20:26.312199+0000 mon.a (mon.0) 56 : cluster 0 osdmap e289: 8 total, 8 up, 8 in 2024-04-07T16:20:27.689 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:27 smithi100 bash[31660]: cluster 2024-04-07T16:20:26.599736+0000 mgr.y (mgr.25175) 12 : cluster 0 pgmap v17: 121 pgs: 121 active+clean; 228 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail 2024-04-07T16:20:27.689 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:27 smithi100 bash[31660]: cluster 2024-04-07T16:20:26.740580+0000 mon.a (mon.0) 57 : cluster 0 osdmap e290: 8 total, 8 up, 8 in 2024-04-07T16:20:27.689 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:27 smithi100 bash[31660]: audit 2024-04-07T16:20:26.747621+0000 mon.c (mon.2) 155 : audit 0 from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:20:27.689 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:27 smithi100 bash[31660]: audit 2024-04-07T16:20:26.783361+0000 mon.c (mon.2) 156 : audit 0 from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config-key dump", "format": "json"}]: dispatch 2024-04-07T16:20:27.689 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:27 smithi100 bash[31660]: audit 2024-04-07T16:20:26.899288+0000 mon.c (mon.2) 157 : audit 0 from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:20:27.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:27 smithi100 bash[31660]: audit 2024-04-07T16:20:26.899949+0000 mon.c (mon.2) 158 : audit 0 from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:20:27.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:27 smithi100 bash[31660]: audit 2024-04-07T16:20:26.900402+0000 mon.c (mon.2) 159 : audit 0 from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:20:27.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:27 smithi100 bash[31660]: audit 2024-04-07T16:20:26.903490+0000 mon.c (mon.2) 160 : audit 0 from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:20:27.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:27 smithi100 bash[31660]: audit 2024-04-07T16:20:26.903709+0000 mon.c (mon.2) 161 : audit 0 from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:20:27.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:27 smithi100 bash[31660]: audit 2024-04-07T16:20:26.904020+0000 mon.c (mon.2) 162 : audit 0 from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:20:27.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:27 smithi100 bash[31660]: audit 2024-04-07T16:20:26.906217+0000 mon.c (mon.2) 163 : audit 0 from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:20:27.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:27 smithi100 bash[31660]: audit 2024-04-07T16:20:26.906412+0000 mon.c (mon.2) 164 : audit 0 from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:20:27.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:27 smithi100 bash[31660]: audit 2024-04-07T16:20:26.906722+0000 mon.c (mon.2) 165 : audit 0 from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:20:27.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:27 smithi100 bash[31660]: audit 2024-04-07T16:20:26.919690+0000 mon.b (mon.1) 335 : audit 1 from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T16:20:27.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:27 smithi100 bash[31660]: audit 2024-04-07T16:20:26.919797+0000 mon.b (mon.1) 336 : audit 1 from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T16:20:27.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:27 smithi100 bash[31660]: audit 2024-04-07T16:20:26.921172+0000 mon.a (mon.0) 58 : audit 1 from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T16:20:27.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:27 smithi100 bash[31660]: audit 2024-04-07T16:20:26.921284+0000 mon.a (mon.0) 59 : audit 1 from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T16:20:27.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:27 smithi100 bash[31660]: audit 2024-04-07T16:20:26.921562+0000 mon.c (mon.2) 166 : audit 0 from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:20:27.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:27 smithi100 bash[31660]: audit 2024-04-07T16:20:26.922161+0000 mon.c (mon.2) 167 : audit 0 from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:20:27.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:27 smithi100 bash[31660]: audit 2024-04-07T16:20:26.922699+0000 mon.c (mon.2) 168 : audit 0 from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:20:27.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:27 smithi100 bash[31660]: audit 2024-04-07T16:20:26.925820+0000 mon.c (mon.2) 169 : audit 0 from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:20:27.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:27 smithi100 bash[31660]: audit 2024-04-07T16:20:26.926485+0000 mon.c (mon.2) 170 : audit 0 from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:20:27.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:27 smithi100 bash[31660]: audit 2024-04-07T16:20:26.927032+0000 mon.c (mon.2) 171 : audit 0 from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:20:27.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:27 smithi100 bash[31660]: audit 2024-04-07T16:20:26.929732+0000 mon.c (mon.2) 172 : audit 1 from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T16:20:27.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:27 smithi100 bash[31660]: audit 2024-04-07T16:20:26.929880+0000 mon.c (mon.2) 173 : audit 1 from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T16:20:27.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:27 smithi100 bash[31660]: audit 2024-04-07T16:20:26.930051+0000 mon.c (mon.2) 174 : audit 0 from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:20:27.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:27 smithi100 bash[31660]: audit 2024-04-07T16:20:26.931086+0000 mon.c (mon.2) 175 : audit 0 from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:20:27.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:27 smithi100 bash[31660]: audit 2024-04-07T16:20:26.931627+0000 mon.c (mon.2) 176 : audit 0 from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:20:27.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:27 smithi100 bash[74942]: audit 2024-04-07T16:20:26.922161+0000 mon.c (mon.2) 167 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:20:27.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:27 smithi100 bash[74942]: audit 2024-04-07T16:20:26.922699+0000 mon.c (mon.2) 168 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:20:27.692 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:27 smithi100 bash[74942]: audit 2024-04-07T16:20:26.925820+0000 mon.c (mon.2) 169 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:20:27.692 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:27 smithi100 bash[74942]: audit 2024-04-07T16:20:26.926485+0000 mon.c (mon.2) 170 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:20:27.692 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:27 smithi100 bash[74942]: audit 2024-04-07T16:20:26.927032+0000 mon.c (mon.2) 171 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:20:27.692 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:27 smithi100 bash[74942]: audit 2024-04-07T16:20:26.929732+0000 mon.c (mon.2) 172 : audit [INF] from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T16:20:27.692 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:27 smithi100 bash[74942]: audit 2024-04-07T16:20:26.929880+0000 mon.c (mon.2) 173 : audit [INF] from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T16:20:27.692 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:27 smithi100 bash[74942]: audit 2024-04-07T16:20:26.930051+0000 mon.c (mon.2) 174 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:20:27.692 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:27 smithi100 bash[74942]: audit 2024-04-07T16:20:26.931086+0000 mon.c (mon.2) 175 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:20:27.692 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:27 smithi100 bash[74942]: audit 2024-04-07T16:20:26.931627+0000 mon.c (mon.2) 176 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:20:27.736 INFO:tasks.rados.rados.0.smithi100.stdout:1191: done (3 left) 2024-04-07T16:20:27.736 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 19 current snap is 110 2024-04-07T16:20:27.736 INFO:tasks.rados.rados.0.smithi100.stdout:1192: seq_num 364 ranges {1884160=499712,2383872=638976,3022848=163840} 2024-04-07T16:20:27.751 INFO:tasks.rados.rados.0.smithi100.stdout:1192: writing smithi10066867-19 from 1884160 to 2383872 tid 1 2024-04-07T16:20:27.755 INFO:tasks.rados.rados.0.smithi100.stdout:1192: writing smithi10066867-19 from 2383872 to 3022848 tid 2 2024-04-07T16:20:27.756 INFO:tasks.rados.rados.0.smithi100.stdout:1192: writing smithi10066867-19 from 3022848 to 3186688 tid 3 2024-04-07T16:20:27.763 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 40 v 640 (ObjNum 303 snap 87 seq_num 303) dirty exists 2024-04-07T16:20:27.763 INFO:tasks.rados.rados.0.smithi100.stdout:1190: expect (ObjNum 324 snap 97 seq_num 324) 2024-04-07T16:20:28.053 INFO:tasks.rados.rados.0.smithi100.stdout:1192: finishing write tid 1 to smithi10066867-19 2024-04-07T16:20:28.053 INFO:tasks.rados.rados.0.smithi100.stdout:1192: finishing write tid 2 to smithi10066867-19 2024-04-07T16:20:28.053 INFO:tasks.rados.rados.0.smithi100.stdout:1192: finishing write tid 3 to smithi10066867-19 2024-04-07T16:20:28.053 INFO:tasks.rados.rados.0.smithi100.stdout:1192: finishing write tid 4 to smithi10066867-19 2024-04-07T16:20:28.053 INFO:tasks.rados.rados.0.smithi100.stdout:1192: finishing write tid 5 to smithi10066867-19 2024-04-07T16:20:28.053 INFO:tasks.rados.rados.0.smithi100.stdout:1192: oid 19 updating version 0 to 545 2024-04-07T16:20:28.053 INFO:tasks.rados.rados.0.smithi100.stdout:1192: oid 19 updating version 545 to 546 2024-04-07T16:20:28.053 INFO:tasks.rados.rados.0.smithi100.stdout:1192: oid 19 version 546 is already newer than 544 2024-04-07T16:20:28.054 INFO:tasks.rados.rados.0.smithi100.stdout:1192: oid 19 version 546 is already newer than 543 2024-04-07T16:20:28.054 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 19 v 546 (ObjNum 364 snap 110 seq_num 364) dirty exists 2024-04-07T16:20:28.054 INFO:tasks.rados.rados.0.smithi100.stdout:1192: left oid 19 (ObjNum 364 snap 110 seq_num 364) 2024-04-07T16:20:28.054 INFO:tasks.rados.rados.0.smithi100.stdout:1187: done (3 left) 2024-04-07T16:20:28.054 INFO:tasks.rados.rados.0.smithi100.stdout:1188: done (2 left) 2024-04-07T16:20:28.054 INFO:tasks.rados.rados.0.smithi100.stdout:1190: done (1 left) 2024-04-07T16:20:28.054 INFO:tasks.rados.rados.0.smithi100.stdout:1192: done (0 left) 2024-04-07T16:20:28.054 INFO:tasks.rados.rados.0.smithi100.stdout:1193: setattr oid 17 current snap is 110 2024-04-07T16:20:28.056 INFO:tasks.rados.rados.0.smithi100.stdout:1194: rollback oid 9 current snap is 110 2024-04-07T16:20:28.056 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 9 to 109 2024-04-07T16:20:28.057 INFO:tasks.rados.rados.0.smithi100.stdout:1195: snap_remove snap 107 2024-04-07T16:20:28.686 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:20:28 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:20:28] "GET /metrics HTTP/1.1" 200 33362 "" "Prometheus/2.43.0" 2024-04-07T16:20:28.735 INFO:tasks.rados.rados.0.smithi100.stdout:1194: finishing rollback tid 1 to smithi10066867-9 2024-04-07T16:20:28.735 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 9 v 538 (ObjNum 1681 snap 0 seq_num 2701828320) dirty dne 2024-04-07T16:20:28.735 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 17 v 547 (ObjNum 303 snap 87 seq_num 303) dirty exists 2024-04-07T16:20:28.735 INFO:tasks.rados.rados.0.smithi100.stdout:1193: done (2 left) 2024-04-07T16:20:28.735 INFO:tasks.rados.rados.0.smithi100.stdout:1194: done (1 left) 2024-04-07T16:20:28.735 INFO:tasks.rados.rados.0.smithi100.stdout:1195: done (0 left) 2024-04-07T16:20:28.735 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 5 current snap is 110 2024-04-07T16:20:28.735 INFO:tasks.rados.rados.0.smithi100.stdout:1196: seq_num 365 ranges {2080768=516096,2596864=573440,3170304=229376} 2024-04-07T16:20:28.750 INFO:tasks.rados.rados.0.smithi100.stdout:1196: writing smithi10066867-5 from 2080768 to 2596864 tid 1 2024-04-07T16:20:28.755 INFO:tasks.rados.rados.0.smithi100.stdout:1196: writing smithi10066867-5 from 2596864 to 3170304 tid 2 2024-04-07T16:20:28.756 INFO:tasks.rados.rados.0.smithi100.stdout:1196: writing smithi10066867-5 from 3170304 to 3399680 tid 3 2024-04-07T16:20:28.756 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 6 current snap is 110 2024-04-07T16:20:28.757 INFO:tasks.rados.rados.0.smithi100.stdout:1197: seq_num 366 ranges {4841472=532480,5373952=458752,5832704=344064} 2024-04-07T16:20:28.787 INFO:tasks.rados.rados.0.smithi100.stdout:1197: writing smithi10066867-6 from 4841472 to 5373952 tid 1 2024-04-07T16:20:28.790 INFO:tasks.rados.rados.0.smithi100.stdout:1197: writing smithi10066867-6 from 5373952 to 5832704 tid 2 2024-04-07T16:20:28.793 INFO:tasks.rados.rados.0.smithi100.stdout:1197: writing smithi10066867-6 from 5832704 to 6176768 tid 3 2024-04-07T16:20:28.793 INFO:tasks.rados.rados.0.smithi100.stdout:1198: read oid 28 snap -1 2024-04-07T16:20:28.793 INFO:tasks.rados.rados.0.smithi100.stdout:1198: expect (ObjNum 283 snap 81 seq_num 283) 2024-04-07T16:20:28.793 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 49 current snap is 110 2024-04-07T16:20:28.793 INFO:tasks.rados.rados.0.smithi100.stdout:1196: finishing write tid 1 to smithi10066867-5 2024-04-07T16:20:28.793 INFO:tasks.rados.rados.0.smithi100.stdout:1196: finishing write tid 2 to smithi10066867-5 2024-04-07T16:20:28.793 INFO:tasks.rados.rados.0.smithi100.stdout:1196: finishing write tid 3 to smithi10066867-5 2024-04-07T16:20:28.793 INFO:tasks.rados.rados.0.smithi100.stdout:1196: finishing write tid 4 to smithi10066867-5 2024-04-07T16:20:28.793 INFO:tasks.rados.rados.0.smithi100.stdout:1196: finishing write tid 5 to smithi10066867-5 2024-04-07T16:20:28.793 INFO:tasks.rados.rados.0.smithi100.stdout:1196: oid 5 updating version 0 to 653 2024-04-07T16:20:28.793 INFO:tasks.rados.rados.0.smithi100.stdout:1196: oid 5 version 653 is already newer than 652 2024-04-07T16:20:28.794 INFO:tasks.rados.rados.0.smithi100.stdout:1196: oid 5 version 653 is already newer than 651 2024-04-07T16:20:28.794 INFO:tasks.rados.rados.0.smithi100.stdout:1196: oid 5 updating version 653 to 654 2024-04-07T16:20:28.794 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 5 v 654 (ObjNum 365 snap 110 seq_num 365) dirty exists 2024-04-07T16:20:28.794 INFO:tasks.rados.rados.0.smithi100.stdout:1196: left oid 5 (ObjNum 365 snap 110 seq_num 365) 2024-04-07T16:20:28.794 INFO:tasks.rados.rados.0.smithi100.stdout:1199: seq_num 367 ranges {2310144=548864,2859008=737280,3596288=65536} 2024-04-07T16:20:28.811 INFO:tasks.rados.rados.0.smithi100.stdout:1199: writing smithi10066867-49 from 2310144 to 2859008 tid 1 2024-04-07T16:20:28.816 INFO:tasks.rados.rados.0.smithi100.stdout:1199: writing smithi10066867-49 from 2859008 to 3596288 tid 2 2024-04-07T16:20:28.817 INFO:tasks.rados.rados.0.smithi100.stdout:1199: writing smithi10066867-49 from 3596288 to 3661824 tid 3 2024-04-07T16:20:28.817 INFO:tasks.rados.rados.0.smithi100.stdout:1196: done (3 left) 2024-04-07T16:20:28.817 INFO:tasks.rados.rados.0.smithi100.stdout:1200: copy_from oid 48 from oid 43 current snap is 110 2024-04-07T16:20:28.817 INFO:tasks.rados.rados.0.smithi100.stdout:1197: finishing write tid 1 to smithi10066867-6 2024-04-07T16:20:28.817 INFO:tasks.rados.rados.0.smithi100.stdout:1197: finishing write tid 2 to smithi10066867-6 2024-04-07T16:20:28.818 INFO:tasks.rados.rados.0.smithi100.stdout:1197: finishing write tid 3 to smithi10066867-6 2024-04-07T16:20:28.818 INFO:tasks.rados.rados.0.smithi100.stdout:1197: finishing write tid 4 to smithi10066867-6 2024-04-07T16:20:28.818 INFO:tasks.rados.rados.0.smithi100.stdout:1199: finishing write tid 1 to smithi10066867-49 2024-04-07T16:20:28.818 INFO:tasks.rados.rados.0.smithi100.stdout:1201: snap_create 2024-04-07T16:20:28.818 INFO:tasks.rados.rados.0.smithi100.stdout:1200: finishing copy_from to smithi10066867-48 2024-04-07T16:20:28.818 INFO:tasks.rados.rados.0.smithi100.stdout:1200: got expected ENOENT (src dne) 2024-04-07T16:20:28.819 INFO:tasks.rados.rados.0.smithi100.stdout:1200: finishing copy_from racing read to smithi10066867-48 2024-04-07T16:20:28.819 INFO:tasks.rados.rados.0.smithi100.stdout:1197: finishing write tid 5 to smithi10066867-6 2024-04-07T16:20:28.819 INFO:tasks.rados.rados.0.smithi100.stdout:1197: oid 6 updating version 0 to 643 2024-04-07T16:20:28.820 INFO:tasks.rados.rados.0.smithi100.stdout:1197: oid 6 version 643 is already newer than 641 2024-04-07T16:20:28.820 INFO:tasks.rados.rados.0.smithi100.stdout:1197: oid 6 updating version 643 to 644 2024-04-07T16:20:28.820 INFO:tasks.rados.rados.0.smithi100.stdout:1197: oid 6 version 644 is already newer than 642 2024-04-07T16:20:28.820 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 6 v 644 (ObjNum 366 snap 110 seq_num 366) dirty exists 2024-04-07T16:20:28.820 INFO:tasks.rados.rados.0.smithi100.stdout:1197: left oid 6 (ObjNum 366 snap 110 seq_num 366) 2024-04-07T16:20:28.823 INFO:tasks.rados.rados.0.smithi100.stdout:1199: finishing write tid 2 to smithi10066867-49 2024-04-07T16:20:28.823 INFO:tasks.rados.rados.0.smithi100.stdout:1199: finishing write tid 3 to smithi10066867-49 2024-04-07T16:20:28.823 INFO:tasks.rados.rados.0.smithi100.stdout:1199: finishing write tid 4 to smithi10066867-49 2024-04-07T16:20:28.824 INFO:tasks.rados.rados.0.smithi100.stdout:1198: expect (ObjNum 283 snap 81 seq_num 283) 2024-04-07T16:20:28.997 INFO:tasks.rados.rados.0.smithi100.stdout:1199: finishing write tid 5 to smithi10066867-49 2024-04-07T16:20:28.998 INFO:tasks.rados.rados.0.smithi100.stdout:1199: oid 49 updating version 0 to 664 2024-04-07T16:20:28.998 INFO:tasks.rados.rados.0.smithi100.stdout:1199: oid 49 updating version 664 to 666 2024-04-07T16:20:28.998 INFO:tasks.rados.rados.0.smithi100.stdout:1199: oid 49 version 666 is already newer than 665 2024-04-07T16:20:28.998 INFO:tasks.rados.rados.0.smithi100.stdout:1199: oid 49 updating version 666 to 667 2024-04-07T16:20:28.998 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 49 v 667 (ObjNum 367 snap 110 seq_num 367) dirty exists 2024-04-07T16:20:28.998 INFO:tasks.rados.rados.0.smithi100.stdout:1199: left oid 49 (ObjNum 367 snap 110 seq_num 367) 2024-04-07T16:20:29.105 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:28 smithi144 bash[23522]: cluster 2024-04-07T16:20:27.744785+0000 mon.a (mon.0) 60 : cluster 0 osdmap e291: 8 total, 8 up, 8 in 2024-04-07T16:20:29.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:28 smithi100 bash[74942]: cluster 2024-04-07T16:20:27.744785+0000 mon.a (mon.0) 60 : cluster [DBG] osdmap e291: 8 total, 8 up, 8 in 2024-04-07T16:20:29.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:28 smithi100 bash[31660]: cluster 2024-04-07T16:20:27.744785+0000 mon.a (mon.0) 60 : cluster 0 osdmap e291: 8 total, 8 up, 8 in 2024-04-07T16:20:29.757 INFO:tasks.rados.rados.0.smithi100.stdout:1197: done (4 left) 2024-04-07T16:20:29.757 INFO:tasks.rados.rados.0.smithi100.stdout:1198: done (3 left) 2024-04-07T16:20:29.757 INFO:tasks.rados.rados.0.smithi100.stdout:1199: done (2 left) 2024-04-07T16:20:29.757 INFO:tasks.rados.rados.0.smithi100.stdout:1200: done (1 left) 2024-04-07T16:20:29.758 INFO:tasks.rados.rados.0.smithi100.stdout:1201: done (0 left) 2024-04-07T16:20:29.758 INFO:tasks.rados.rados.0.smithi100.stdout:1202: snap_create 2024-04-07T16:20:30.105 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:29 smithi144 bash[23522]: cluster 2024-04-07T16:20:28.600476+0000 mgr.y (mgr.25175) 13 : cluster 0 pgmap v20: 121 pgs: 121 active+clean; 233 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 16 MiB/s rd, 3.7 MiB/s wr, 37 op/s 2024-04-07T16:20:30.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:29 smithi144 bash[23522]: cluster 2024-04-07T16:20:28.746946+0000 mon.a (mon.0) 61 : cluster 0 osdmap e292: 8 total, 8 up, 8 in 2024-04-07T16:20:30.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:29 smithi144 bash[23522]: cluster 2024-04-07T16:20:28.761708+0000 mon.a (mon.0) 62 : cluster 0 mgrmap e41: y(active, since 12s), standbys: x 2024-04-07T16:20:30.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:29 smithi100 bash[74942]: cluster 2024-04-07T16:20:28.600476+0000 mgr.y (mgr.25175) 13 : cluster [DBG] pgmap v20: 121 pgs: 121 active+clean; 233 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 16 MiB/s rd, 3.7 MiB/s wr, 37 op/s 2024-04-07T16:20:30.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:29 smithi100 bash[74942]: cluster 2024-04-07T16:20:28.746946+0000 mon.a (mon.0) 61 : cluster [DBG] osdmap e292: 8 total, 8 up, 8 in 2024-04-07T16:20:30.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:29 smithi100 bash[74942]: cluster 2024-04-07T16:20:28.761708+0000 mon.a (mon.0) 62 : cluster [DBG] mgrmap e41: y(active, since 12s), standbys: x 2024-04-07T16:20:30.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:29 smithi100 bash[31660]: cluster 2024-04-07T16:20:28.600476+0000 mgr.y (mgr.25175) 13 : cluster 0 pgmap v20: 121 pgs: 121 active+clean; 233 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 16 MiB/s rd, 3.7 MiB/s wr, 37 op/s 2024-04-07T16:20:30.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:29 smithi100 bash[31660]: cluster 2024-04-07T16:20:28.746946+0000 mon.a (mon.0) 61 : cluster 0 osdmap e292: 8 total, 8 up, 8 in 2024-04-07T16:20:30.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:29 smithi100 bash[31660]: cluster 2024-04-07T16:20:28.761708+0000 mon.a (mon.0) 62 : cluster 0 mgrmap e41: y(active, since 12s), standbys: x 2024-04-07T16:20:30.769 INFO:tasks.rados.rados.0.smithi100.stdout:1202: done (0 left) 2024-04-07T16:20:30.769 INFO:tasks.rados.rados.0.smithi100.stdout:1203: snap_remove snap 110 2024-04-07T16:20:31.105 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:30 smithi144 bash[23522]: cluster 2024-04-07T16:20:29.773361+0000 mon.a (mon.0) 63 : cluster 0 osdmap e293: 8 total, 8 up, 8 in 2024-04-07T16:20:31.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:30 smithi100 bash[74942]: cluster 2024-04-07T16:20:29.773361+0000 mon.a (mon.0) 63 : cluster [DBG] osdmap e293: 8 total, 8 up, 8 in 2024-04-07T16:20:31.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:30 smithi100 bash[31660]: cluster 2024-04-07T16:20:29.773361+0000 mon.a (mon.0) 63 : cluster 0 osdmap e293: 8 total, 8 up, 8 in 2024-04-07T16:20:31.730 INFO:tasks.rados.rados.0.smithi100.stdout:1203: done (0 left) 2024-04-07T16:20:31.730 INFO:tasks.rados.rados.0.smithi100.stdout:1204: rollback oid 5 current snap is 112 2024-04-07T16:20:31.731 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 5 to 106 2024-04-07T16:20:31.731 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 29 current snap is 112 2024-04-07T16:20:31.731 INFO:tasks.rados.rados.0.smithi100.stdout:1205: seq_num 368 ranges {0=573440,573440=622592,1196032=180224} 2024-04-07T16:20:31.737 INFO:tasks.rados.rados.0.smithi100.stdout:1205: writing smithi10066867-29 from 0 to 573440 tid 1 2024-04-07T16:20:31.743 INFO:tasks.rados.rados.0.smithi100.stdout:1205: writing smithi10066867-29 from 573440 to 1196032 tid 2 2024-04-07T16:20:31.746 INFO:tasks.rados.rados.0.smithi100.stdout:1205: writing smithi10066867-29 from 1196032 to 1376256 tid 3 2024-04-07T16:20:31.746 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 10 current snap is 112 2024-04-07T16:20:31.746 INFO:tasks.rados.rados.0.smithi100.stdout:1206: seq_num 369 ranges {4431872=589824,5021696=507904,5529600=294912} 2024-04-07T16:20:31.788 INFO:tasks.rados.rados.0.smithi100.stdout:1206: writing smithi10066867-10 from 4431872 to 5021696 tid 1 2024-04-07T16:20:31.793 INFO:tasks.rados.rados.0.smithi100.stdout:1206: writing smithi10066867-10 from 5021696 to 5529600 tid 2 2024-04-07T16:20:31.795 INFO:tasks.rados.rados.0.smithi100.stdout:1206: writing smithi10066867-10 from 5529600 to 5824512 tid 3 2024-04-07T16:20:31.796 INFO:tasks.rados.rados.0.smithi100.stdout:1207: delete oid 26 current snap is 112 2024-04-07T16:20:31.796 INFO:tasks.rados.rados.0.smithi100.stdout:1204: finishing rollback tid 0 to smithi10066867-5 2024-04-07T16:20:31.796 INFO:tasks.rados.rados.0.smithi100.stdout:1204: finishing rollback tid 1 to smithi10066867-5 2024-04-07T16:20:31.796 INFO:tasks.rados.rados.0.smithi100.stdout:1204: finishing rollback tid 2 to smithi10066867-5 2024-04-07T16:20:31.796 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 5 v 671 (ObjNum 345 snap 102 seq_num 345) dirty exists 2024-04-07T16:20:31.796 INFO:tasks.rados.rados.0.smithi100.stdout:1205: finishing write tid 1 to smithi10066867-29 2024-04-07T16:20:31.796 INFO:tasks.rados.rados.0.smithi100.stdout:1205: finishing write tid 2 to smithi10066867-29 2024-04-07T16:20:31.796 INFO:tasks.rados.rados.0.smithi100.stdout:1205: finishing write tid 3 to smithi10066867-29 2024-04-07T16:20:31.796 INFO:tasks.rados.rados.0.smithi100.stdout:1205: finishing write tid 4 to smithi10066867-29 2024-04-07T16:20:31.796 INFO:tasks.rados.rados.0.smithi100.stdout:1205: finishing write tid 5 to smithi10066867-29 2024-04-07T16:20:31.796 INFO:tasks.rados.rados.0.smithi100.stdout:1205: oid 29 updating version 0 to 485 2024-04-07T16:20:31.796 INFO:tasks.rados.rados.0.smithi100.stdout:1205: oid 29 updating version 485 to 486 2024-04-07T16:20:31.796 INFO:tasks.rados.rados.0.smithi100.stdout:1205: oid 29 updating version 486 to 487 2024-04-07T16:20:31.796 INFO:tasks.rados.rados.0.smithi100.stdout:1205: oid 29 updating version 487 to 488 2024-04-07T16:20:31.797 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 29 v 488 (ObjNum 368 snap 112 seq_num 368) dirty exists 2024-04-07T16:20:31.797 INFO:tasks.rados.rados.0.smithi100.stdout:1205: left oid 29 (ObjNum 368 snap 112 seq_num 368) 2024-04-07T16:20:31.797 INFO:tasks.rados.rados.0.smithi100.stdout:1206: finishing write tid 1 to smithi10066867-10 2024-04-07T16:20:31.801 INFO:tasks.rados.rados.0.smithi100.stdout:1206: finishing write tid 2 to smithi10066867-10 2024-04-07T16:20:31.801 INFO:tasks.rados.rados.0.smithi100.stdout:1206: finishing write tid 3 to smithi10066867-10 2024-04-07T16:20:31.802 INFO:tasks.rados.rados.0.smithi100.stdout:1206: finishing write tid 4 to smithi10066867-10 2024-04-07T16:20:31.804 INFO:tasks.rados.rados.0.smithi100.stdout:1206: finishing write tid 5 to smithi10066867-10 2024-04-07T16:20:31.804 INFO:tasks.rados.rados.0.smithi100.stdout:1206: oid 10 updating version 0 to 240 2024-04-07T16:20:31.804 INFO:tasks.rados.rados.0.smithi100.stdout:1206: oid 10 updating version 240 to 242 2024-04-07T16:20:31.804 INFO:tasks.rados.rados.0.smithi100.stdout:1206: oid 10 version 242 is already newer than 241 2024-04-07T16:20:31.804 INFO:tasks.rados.rados.0.smithi100.stdout:1206: oid 10 updating version 242 to 243 2024-04-07T16:20:31.804 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 10 v 243 (ObjNum 369 snap 112 seq_num 369) dirty exists 2024-04-07T16:20:31.804 INFO:tasks.rados.rados.0.smithi100.stdout:1206: left oid 10 (ObjNum 369 snap 112 seq_num 369) 2024-04-07T16:20:31.804 INFO:tasks.rados.rados.0.smithi100.stdout:1204: done (3 left) 2024-04-07T16:20:31.804 INFO:tasks.rados.rados.0.smithi100.stdout:1205: done (2 left) 2024-04-07T16:20:31.804 INFO:tasks.rados.rados.0.smithi100.stdout:1206: done (1 left) 2024-04-07T16:20:31.804 INFO:tasks.rados.rados.0.smithi100.stdout:1207: done (0 left) 2024-04-07T16:20:31.804 INFO:tasks.rados.rados.0.smithi100.stdout:1208: read oid 28 snap -1 2024-04-07T16:20:31.804 INFO:tasks.rados.rados.0.smithi100.stdout:1208: expect (ObjNum 283 snap 81 seq_num 283) 2024-04-07T16:20:31.805 INFO:tasks.rados.rados.0.smithi100.stdout:1209: delete oid 35 current snap is 112 2024-04-07T16:20:31.805 INFO:tasks.rados.rados.0.smithi100.stdout:1209: done (1 left) 2024-04-07T16:20:31.805 INFO:tasks.rados.rados.0.smithi100.stdout:1210: snap_remove snap 111 2024-04-07T16:20:32.105 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:31 smithi144 bash[23522]: cluster 2024-04-07T16:20:30.601218+0000 mgr.y (mgr.25175) 14 : cluster 0 pgmap v23: 121 pgs: 121 active+clean; 233 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 16 MiB/s rd, 3.7 MiB/s wr, 37 op/s 2024-04-07T16:20:32.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:31 smithi144 bash[23522]: cluster 2024-04-07T16:20:30.782729+0000 mon.a (mon.0) 64 : cluster 0 osdmap e294: 8 total, 8 up, 8 in 2024-04-07T16:20:32.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:31 smithi144 bash[23522]: audit 2024-04-07T16:20:31.700837+0000 mon.c (mon.2) 177 : audit 0 from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:20:32.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:31 smithi144 bash[23522]: cluster 2024-04-07T16:20:31.744377+0000 mon.a (mon.0) 65 : cluster 0 osdmap e295: 8 total, 8 up, 8 in 2024-04-07T16:20:32.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:31 smithi100 bash[74942]: cluster 2024-04-07T16:20:30.601218+0000 mgr.y (mgr.25175) 14 : cluster [DBG] pgmap v23: 121 pgs: 121 active+clean; 233 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 16 MiB/s rd, 3.7 MiB/s wr, 37 op/s 2024-04-07T16:20:32.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:31 smithi100 bash[74942]: cluster 2024-04-07T16:20:30.782729+0000 mon.a (mon.0) 64 : cluster [DBG] osdmap e294: 8 total, 8 up, 8 in 2024-04-07T16:20:32.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:31 smithi100 bash[74942]: audit 2024-04-07T16:20:31.700837+0000 mon.c (mon.2) 177 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:20:32.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:31 smithi100 bash[74942]: cluster 2024-04-07T16:20:31.744377+0000 mon.a (mon.0) 65 : cluster [DBG] osdmap e295: 8 total, 8 up, 8 in 2024-04-07T16:20:32.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:31 smithi100 bash[31660]: cluster 2024-04-07T16:20:30.601218+0000 mgr.y (mgr.25175) 14 : cluster 0 pgmap v23: 121 pgs: 121 active+clean; 233 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 16 MiB/s rd, 3.7 MiB/s wr, 37 op/s 2024-04-07T16:20:32.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:31 smithi100 bash[31660]: cluster 2024-04-07T16:20:30.782729+0000 mon.a (mon.0) 64 : cluster 0 osdmap e294: 8 total, 8 up, 8 in 2024-04-07T16:20:32.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:31 smithi100 bash[31660]: audit 2024-04-07T16:20:31.700837+0000 mon.c (mon.2) 177 : audit 0 from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:20:32.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:31 smithi100 bash[31660]: cluster 2024-04-07T16:20:31.744377+0000 mon.a (mon.0) 65 : cluster 0 osdmap e295: 8 total, 8 up, 8 in 2024-04-07T16:20:32.605 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:20:32 smithi144 bash[59794]: ts=2024-04-07T16:20:32.313Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:20:32.777 INFO:tasks.rados.rados.0.smithi100.stdout:1210: done (1 left) 2024-04-07T16:20:32.778 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 48 current snap is 112 2024-04-07T16:20:32.778 INFO:tasks.rados.rados.0.smithi100.stdout:1211: seq_num 372 ranges {0=638976,638976=565248,1204224=237568} 2024-04-07T16:20:32.783 INFO:tasks.rados.rados.0.smithi100.stdout:1211: writing smithi10066867-48 from 0 to 638976 tid 1 2024-04-07T16:20:32.788 INFO:tasks.rados.rados.0.smithi100.stdout:1211: writing smithi10066867-48 from 638976 to 1204224 tid 2 2024-04-07T16:20:32.791 INFO:tasks.rados.rados.0.smithi100.stdout:1211: writing smithi10066867-48 from 1204224 to 1441792 tid 3 2024-04-07T16:20:32.791 INFO:tasks.rados.rados.0.smithi100.stdout:1212: read oid 29 snap -1 2024-04-07T16:20:32.791 INFO:tasks.rados.rados.0.smithi100.stdout:1212: expect (ObjNum 368 snap 112 seq_num 368) 2024-04-07T16:20:32.791 INFO:tasks.rados.rados.0.smithi100.stdout:1213: rollback oid 39 current snap is 112 2024-04-07T16:20:32.791 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 39 to 109 2024-04-07T16:20:32.791 INFO:tasks.rados.rados.0.smithi100.stdout:1208: expect (ObjNum 283 snap 81 seq_num 283) 2024-04-07T16:20:32.969 INFO:tasks.rados.rados.0.smithi100.stdout:1211: finishing write tid 1 to smithi10066867-48 2024-04-07T16:20:32.969 INFO:tasks.rados.rados.0.smithi100.stdout:1211: finishing write tid 2 to smithi10066867-48 2024-04-07T16:20:32.969 INFO:tasks.rados.rados.0.smithi100.stdout:1211: finishing write tid 3 to smithi10066867-48 2024-04-07T16:20:32.969 INFO:tasks.rados.rados.0.smithi100.stdout:1211: finishing write tid 4 to smithi10066867-48 2024-04-07T16:20:32.969 INFO:tasks.rados.rados.0.smithi100.stdout:1212: expect (ObjNum 368 snap 112 seq_num 368) 2024-04-07T16:20:33.004 INFO:tasks.rados.rados.0.smithi100.stdout:1211: finishing write tid 5 to smithi10066867-48 2024-04-07T16:20:33.004 INFO:tasks.rados.rados.0.smithi100.stdout:1211: oid 48 updating version 0 to 526 2024-04-07T16:20:33.004 INFO:tasks.rados.rados.0.smithi100.stdout:1211: oid 48 version 526 is already newer than 524 2024-04-07T16:20:33.004 INFO:tasks.rados.rados.0.smithi100.stdout:1211: oid 48 version 526 is already newer than 523 2024-04-07T16:20:33.004 INFO:tasks.rados.rados.0.smithi100.stdout:1211: oid 48 version 526 is already newer than 525 2024-04-07T16:20:33.004 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 48 v 526 (ObjNum 372 snap 112 seq_num 372) dirty exists 2024-04-07T16:20:33.004 INFO:tasks.rados.rados.0.smithi100.stdout:1211: left oid 48 (ObjNum 372 snap 112 seq_num 372) 2024-04-07T16:20:33.004 INFO:tasks.rados.rados.0.smithi100.stdout:1213: finishing rollback tid 0 to smithi10066867-39 2024-04-07T16:20:33.004 INFO:tasks.rados.rados.0.smithi100.stdout:1213: finishing rollback tid 1 to smithi10066867-39 2024-04-07T16:20:33.004 INFO:tasks.rados.rados.0.smithi100.stdout:1213: finishing rollback tid 2 to smithi10066867-39 2024-04-07T16:20:33.004 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 39 v 492 (ObjNum 322 snap 96 seq_num 322) dirty exists 2024-04-07T16:20:33.005 INFO:tasks.rados.rados.0.smithi100.stdout:1208: done (3 left) 2024-04-07T16:20:33.005 INFO:tasks.rados.rados.0.smithi100.stdout:1211: done (2 left) 2024-04-07T16:20:33.005 INFO:tasks.rados.rados.0.smithi100.stdout:1212: done (1 left) 2024-04-07T16:20:33.005 INFO:tasks.rados.rados.0.smithi100.stdout:1213: done (0 left) 2024-04-07T16:20:33.005 INFO:tasks.rados.rados.0.smithi100.stdout:1214: read oid 2 snap 106 2024-04-07T16:20:33.005 INFO:tasks.rados.rados.0.smithi100.stdout:1214: expect (ObjNum 260 snap 70 seq_num 260) 2024-04-07T16:20:33.005 INFO:tasks.rados.rados.0.smithi100.stdout:1215: setattr oid 22 current snap is 112 2024-04-07T16:20:33.007 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 38 current snap is 112 2024-04-07T16:20:33.007 INFO:tasks.rados.rados.0.smithi100.stdout:1216: seq_num 373 ranges {1212416=655360,1867776=499712,2367488=303104} 2024-04-07T16:20:33.030 INFO:tasks.rados.rados.0.smithi100.stdout:1216: writing smithi10066867-38 from 1212416 to 1867776 tid 1 2024-04-07T16:20:33.037 INFO:tasks.rados.rados.0.smithi100.stdout:1216: writing smithi10066867-38 from 1867776 to 2367488 tid 2 2024-04-07T16:20:33.042 INFO:tasks.rados.rados.0.smithi100.stdout:1216: writing smithi10066867-38 from 2367488 to 2670592 tid 3 2024-04-07T16:20:33.042 INFO:tasks.rados.rados.0.smithi100.stdout:1217: snap_create 2024-04-07T16:20:33.042 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 22 v 515 (ObjNum 294 snap 84 seq_num 294) dirty exists 2024-04-07T16:20:33.042 INFO:tasks.rados.rados.0.smithi100.stdout:1214: expect (ObjNum 260 snap 70 seq_num 260) 2024-04-07T16:20:33.135 INFO:tasks.rados.rados.0.smithi100.stdout:1216: finishing write tid 1 to smithi10066867-38 2024-04-07T16:20:33.135 INFO:tasks.rados.rados.0.smithi100.stdout:1216: finishing write tid 2 to smithi10066867-38 2024-04-07T16:20:33.135 INFO:tasks.rados.rados.0.smithi100.stdout:1216: finishing write tid 3 to smithi10066867-38 2024-04-07T16:20:33.135 INFO:tasks.rados.rados.0.smithi100.stdout:1216: finishing write tid 4 to smithi10066867-38 2024-04-07T16:20:33.135 INFO:tasks.rados.rados.0.smithi100.stdout:1216: finishing write tid 5 to smithi10066867-38 2024-04-07T16:20:33.135 INFO:tasks.rados.rados.0.smithi100.stdout:1216: oid 38 updating version 0 to 538 2024-04-07T16:20:33.136 INFO:tasks.rados.rados.0.smithi100.stdout:1216: oid 38 updating version 538 to 541 2024-04-07T16:20:33.136 INFO:tasks.rados.rados.0.smithi100.stdout:1216: oid 38 version 541 is already newer than 539 2024-04-07T16:20:33.136 INFO:tasks.rados.rados.0.smithi100.stdout:1216: oid 38 version 541 is already newer than 540 2024-04-07T16:20:33.136 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 38 v 541 (ObjNum 373 snap 112 seq_num 373) dirty exists 2024-04-07T16:20:33.136 INFO:tasks.rados.rados.0.smithi100.stdout:1216: left oid 38 (ObjNum 373 snap 112 seq_num 373) 2024-04-07T16:20:33.775 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:20:33 smithi144 bash[59794]: ts=2024-04-07T16:20:33.507Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=nodes name=CephNodeDiskspaceWarning index=4 msg="Evaluating rule failed" rule="alert: CephNodeDiskspaceWarning\nexpr: predict_linear(node_filesystem_free_bytes{device=~\"/.*\"}[2d], 3600 * 24 * 5)\n * on (instance) group_left (nodename) node_uname_info < 0\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.8.4\n severity: warning\n type: ceph_default\nannotations:\n description: Mountpoint {{ $labels.mountpoint }} on {{ $labels.nodename }} will\n be full in less than 5 days based on the 48 hour trailing fill rate.\n summary: Host filesystem free space is getting low\n" err="found duplicate series for the match group {instance=\"smithi100\"} on the right hand-side of the operation: [{__name__=\"node_uname_info\", domainname=\"(none)\", instance=\"smithi100\", job=\"node\", machine=\"x86_64\", nodename=\"smithi100\", release=\"5.15.0-101-generic\", sysname=\"Linux\", version=\"#111-Ubuntu SMP Tue Mar 5 20:16:58 UTC 2024\"}, {__name__=\"node_uname_info\", cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", domainname=\"(none)\", instance=\"smithi100\", job=\"node\", machine=\"x86_64\", nodename=\"smithi100\", release=\"5.15.0-101-generic\", sysname=\"Linux\", version=\"#111-Ubuntu SMP Tue Mar 5 20:16:58 UTC 2024\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:20:33.792 INFO:tasks.rados.rados.0.smithi100.stdout:1214: done (3 left) 2024-04-07T16:20:33.792 INFO:tasks.rados.rados.0.smithi100.stdout:1215: done (2 left) 2024-04-07T16:20:33.792 INFO:tasks.rados.rados.0.smithi100.stdout:1216: done (1 left) 2024-04-07T16:20:33.792 INFO:tasks.rados.rados.0.smithi100.stdout:1217: done (0 left) 2024-04-07T16:20:33.792 INFO:tasks.rados.rados.0.smithi100.stdout:1218: copy_from oid 6 from oid 49 current snap is 113 2024-04-07T16:20:33.792 INFO:tasks.rados.rados.0.smithi100.stdout:1219: rollback oid 33 current snap is 113 2024-04-07T16:20:33.792 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 33 to 109 2024-04-07T16:20:33.792 INFO:tasks.rados.rados.0.smithi100.stdout:1220: copy_from oid 38 from oid 1 current snap is 113 2024-04-07T16:20:33.792 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 16 current snap is 113 2024-04-07T16:20:33.793 INFO:tasks.rados.rados.0.smithi100.stdout:1221: seq_num 374 ranges {2392064=671744,3063808=778240,3842048=24576} 2024-04-07T16:20:33.833 INFO:tasks.rados.rados.0.smithi100.stdout:1221: writing smithi10066867-16 from 2392064 to 3063808 tid 1 2024-04-07T16:20:33.843 INFO:tasks.rados.rados.0.smithi100.stdout:1221: writing smithi10066867-16 from 3063808 to 3842048 tid 2 2024-04-07T16:20:33.844 INFO:tasks.rados.rados.0.smithi100.stdout:1221: writing smithi10066867-16 from 3842048 to 3866624 tid 3 2024-04-07T16:20:33.844 INFO:tasks.rados.rados.0.smithi100.stdout:1222: snap_create 2024-04-07T16:20:33.844 INFO:tasks.rados.rados.0.smithi100.stdout:1219: finishing rollback tid 0 to smithi10066867-33 2024-04-07T16:20:33.844 INFO:tasks.rados.rados.0.smithi100.stdout:1219: finishing rollback tid 1 to smithi10066867-33 2024-04-07T16:20:33.844 INFO:tasks.rados.rados.0.smithi100.stdout:1219: finishing rollback tid 2 to smithi10066867-33 2024-04-07T16:20:33.844 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 33 v 803 (ObjNum 333 snap 100 seq_num 333) dirty exists 2024-04-07T16:20:33.844 INFO:tasks.rados.rados.0.smithi100.stdout:1218: finishing copy_from racing read to smithi10066867-6 2024-04-07T16:20:33.844 INFO:tasks.rados.rados.0.smithi100.stdout:1220: finishing copy_from racing read to smithi10066867-38 2024-04-07T16:20:33.844 INFO:tasks.rados.rados.0.smithi100.stdout:1218: finishing copy_from to smithi10066867-6 2024-04-07T16:20:33.844 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 6 v 667 (ObjNum 367 snap 110 seq_num 367) dirty exists 2024-04-07T16:20:33.844 INFO:tasks.rados.rados.0.smithi100.stdout:1221: finishing write tid 1 to smithi10066867-16 2024-04-07T16:20:33.844 INFO:tasks.rados.rados.0.smithi100.stdout:1220: finishing copy_from to smithi10066867-38 2024-04-07T16:20:33.844 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 38 v 239 (ObjNum 360 snap 109 seq_num 360) dirty exists 2024-04-07T16:20:33.848 INFO:tasks.rados.rados.0.smithi100.stdout:1221: finishing write tid 2 to smithi10066867-16 2024-04-07T16:20:33.848 INFO:tasks.rados.rados.0.smithi100.stdout:1221: finishing write tid 3 to smithi10066867-16 2024-04-07T16:20:33.848 INFO:tasks.rados.rados.0.smithi100.stdout:1221: finishing write tid 4 to smithi10066867-16 2024-04-07T16:20:33.850 INFO:tasks.rados.rados.0.smithi100.stdout:1221: finishing write tid 5 to smithi10066867-16 2024-04-07T16:20:33.850 INFO:tasks.rados.rados.0.smithi100.stdout:1221: oid 16 updating version 0 to 676 2024-04-07T16:20:33.850 INFO:tasks.rados.rados.0.smithi100.stdout:1221: oid 16 updating version 676 to 677 2024-04-07T16:20:33.850 INFO:tasks.rados.rados.0.smithi100.stdout:1221: oid 16 updating version 677 to 678 2024-04-07T16:20:33.850 INFO:tasks.rados.rados.0.smithi100.stdout:1221: oid 16 version 678 is already newer than 675 2024-04-07T16:20:33.850 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 16 v 678 (ObjNum 374 snap 113 seq_num 374) dirty exists 2024-04-07T16:20:33.850 INFO:tasks.rados.rados.0.smithi100.stdout:1221: left oid 16 (ObjNum 374 snap 113 seq_num 374) 2024-04-07T16:20:34.105 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:33 smithi144 bash[23522]: cluster 2024-04-07T16:20:32.602679+0000 mgr.y (mgr.25175) 15 : cluster 0 pgmap v26: 121 pgs: 121 active+clean; 222 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 6.8 MiB/s rd, 1.9 MiB/s wr, 15 op/s 2024-04-07T16:20:34.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:33 smithi144 bash[23522]: cluster 2024-04-07T16:20:32.792917+0000 mon.a (mon.0) 66 : cluster 0 osdmap e296: 8 total, 8 up, 8 in 2024-04-07T16:20:34.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:33 smithi144 bash[23522]: audit 2024-04-07T16:20:33.768280+0000 mon.a (mon.0) 67 : audit 1 from='mgr.25175 ' entity='mgr.y' 2024-04-07T16:20:34.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:33 smithi144 bash[23522]: audit 2024-04-07T16:20:33.784500+0000 mon.a (mon.0) 68 : audit 1 from='mgr.25175 ' entity='mgr.y' 2024-04-07T16:20:34.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:33 smithi100 bash[74942]: cluster 2024-04-07T16:20:32.602679+0000 mgr.y (mgr.25175) 15 : cluster [DBG] pgmap v26: 121 pgs: 121 active+clean; 222 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 6.8 MiB/s rd, 1.9 MiB/s wr, 15 op/s 2024-04-07T16:20:34.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:33 smithi100 bash[74942]: cluster 2024-04-07T16:20:32.792917+0000 mon.a (mon.0) 66 : cluster [DBG] osdmap e296: 8 total, 8 up, 8 in 2024-04-07T16:20:34.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:33 smithi100 bash[74942]: audit 2024-04-07T16:20:33.768280+0000 mon.a (mon.0) 67 : audit [INF] from='mgr.25175 ' entity='mgr.y' 2024-04-07T16:20:34.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:33 smithi100 bash[74942]: audit 2024-04-07T16:20:33.784500+0000 mon.a (mon.0) 68 : audit [INF] from='mgr.25175 ' entity='mgr.y' 2024-04-07T16:20:34.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:33 smithi100 bash[31660]: cluster 2024-04-07T16:20:32.602679+0000 mgr.y (mgr.25175) 15 : cluster 0 pgmap v26: 121 pgs: 121 active+clean; 222 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 6.8 MiB/s rd, 1.9 MiB/s wr, 15 op/s 2024-04-07T16:20:34.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:33 smithi100 bash[31660]: cluster 2024-04-07T16:20:32.792917+0000 mon.a (mon.0) 66 : cluster 0 osdmap e296: 8 total, 8 up, 8 in 2024-04-07T16:20:34.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:33 smithi100 bash[31660]: audit 2024-04-07T16:20:33.768280+0000 mon.a (mon.0) 67 : audit 1 from='mgr.25175 ' entity='mgr.y' 2024-04-07T16:20:34.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:33 smithi100 bash[31660]: audit 2024-04-07T16:20:33.784500+0000 mon.a (mon.0) 68 : audit 1 from='mgr.25175 ' entity='mgr.y' 2024-04-07T16:20:34.793 INFO:tasks.rados.rados.0.smithi100.stdout:1218: done (4 left) 2024-04-07T16:20:34.794 INFO:tasks.rados.rados.0.smithi100.stdout:1219: done (3 left) 2024-04-07T16:20:34.794 INFO:tasks.rados.rados.0.smithi100.stdout:1220: done (2 left) 2024-04-07T16:20:34.794 INFO:tasks.rados.rados.0.smithi100.stdout:1221: done (1 left) 2024-04-07T16:20:34.794 INFO:tasks.rados.rados.0.smithi100.stdout:1222: done (0 left) 2024-04-07T16:20:34.794 INFO:tasks.rados.rados.0.smithi100.stdout:1223: copy_from oid 31 from oid 47 current snap is 114 2024-04-07T16:20:34.794 INFO:tasks.rados.rados.0.smithi100.stdout:1224: read oid 12 snap 109 2024-04-07T16:20:34.794 INFO:tasks.rados.rados.0.smithi100.stdout:1224: expect deleted 2024-04-07T16:20:34.794 INFO:tasks.rados.rados.0.smithi100.stdout:1225: delete oid 32 current snap is 114 2024-04-07T16:20:34.807 INFO:tasks.rados.rados.0.smithi100.stdout:1224: done (2 left) 2024-04-07T16:20:34.807 INFO:tasks.rados.rados.0.smithi100.stdout:1225: done (1 left) 2024-04-07T16:20:34.807 INFO:tasks.rados.rados.0.smithi100.stdout:1226: snap_create 2024-04-07T16:20:34.822 INFO:tasks.rados.rados.0.smithi100.stdout:1223: finishing copy_from racing read to smithi10066867-31 2024-04-07T16:20:34.825 INFO:tasks.rados.rados.0.smithi100.stdout:1223: finishing copy_from to smithi10066867-31 2024-04-07T16:20:34.825 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 31 v 212 (ObjNum 335 snap 101 seq_num 335) dirty exists 2024-04-07T16:20:35.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:34 smithi144 bash[23522]: audit 2024-04-07T16:20:33.790014+0000 mon.c (mon.2) 178 : audit 0 from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:20:35.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:34 smithi144 bash[23522]: cluster 2024-04-07T16:20:33.807412+0000 mon.a (mon.0) 69 : cluster 0 osdmap e297: 8 total, 8 up, 8 in 2024-04-07T16:20:35.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:34 smithi144 bash[23522]: audit 2024-04-07T16:20:33.809783+0000 mon.c (mon.2) 179 : audit 1 from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:20:35.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:34 smithi144 bash[23522]: cephadm 2024-04-07T16:20:33.812839+0000 mgr.y (mgr.25175) 16 : cephadm 1 Updating smithi100:/etc/ceph/ceph.conf 2024-04-07T16:20:35.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:34 smithi144 bash[23522]: cephadm 2024-04-07T16:20:33.813224+0000 mgr.y (mgr.25175) 17 : cephadm 1 Updating smithi144:/etc/ceph/ceph.conf 2024-04-07T16:20:35.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:34 smithi144 bash[23522]: cephadm 2024-04-07T16:20:34.029060+0000 mgr.y (mgr.25175) 18 : cephadm 1 Updating smithi100:/var/lib/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/config/ceph.conf 2024-04-07T16:20:35.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:34 smithi144 bash[23522]: cephadm 2024-04-07T16:20:34.029636+0000 mgr.y (mgr.25175) 19 : cephadm 1 Updating smithi144:/var/lib/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/config/ceph.conf 2024-04-07T16:20:35.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:34 smithi144 bash[23522]: cephadm 2024-04-07T16:20:34.193722+0000 mgr.y (mgr.25175) 20 : cephadm 1 Updating smithi100:/etc/ceph/ceph.client.admin.keyring 2024-04-07T16:20:35.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:34 smithi144 bash[23522]: cephadm 2024-04-07T16:20:34.194291+0000 mgr.y (mgr.25175) 21 : cephadm 1 Updating smithi144:/etc/ceph/ceph.client.admin.keyring 2024-04-07T16:20:35.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:34 smithi144 bash[23522]: audit 2024-04-07T16:20:34.532395+0000 mon.a (mon.0) 70 : audit 1 from='mgr.25175 ' entity='mgr.y' 2024-04-07T16:20:35.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:34 smithi144 bash[23522]: audit 2024-04-07T16:20:34.538480+0000 mon.a (mon.0) 71 : audit 1 from='mgr.25175 ' entity='mgr.y' 2024-04-07T16:20:35.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:34 smithi144 bash[23522]: audit 2024-04-07T16:20:34.552030+0000 mon.a (mon.0) 72 : audit 1 from='mgr.25175 ' entity='mgr.y' 2024-04-07T16:20:35.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:34 smithi144 bash[23522]: audit 2024-04-07T16:20:34.558831+0000 mon.a (mon.0) 73 : audit 1 from='mgr.25175 ' entity='mgr.y' 2024-04-07T16:20:35.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:34 smithi144 bash[23522]: audit 2024-04-07T16:20:34.573859+0000 mon.a (mon.0) 74 : audit 1 from='mgr.25175 ' entity='mgr.y' 2024-04-07T16:20:35.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:34 smithi144 bash[23522]: audit 2024-04-07T16:20:34.674140+0000 mon.c (mon.2) 180 : audit 0 from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:20:35.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:34 smithi144 bash[23522]: audit 2024-04-07T16:20:34.679040+0000 mon.c (mon.2) 181 : audit 0 from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:20:35.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:34 smithi144 bash[23522]: audit 2024-04-07T16:20:34.682105+0000 mon.c (mon.2) 182 : audit 0 from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-04-07T16:20:35.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:34 smithi144 bash[23522]: audit 2024-04-07T16:20:34.683739+0000 mon.c (mon.2) 183 : audit 0 from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon ok-to-stop", "ids": ["c"]}]: dispatch 2024-04-07T16:20:35.128 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:34 smithi100 bash[74942]: audit 2024-04-07T16:20:33.790014+0000 mon.c (mon.2) 178 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:20:35.128 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:34 smithi100 bash[74942]: cluster 2024-04-07T16:20:33.807412+0000 mon.a (mon.0) 69 : cluster [DBG] osdmap e297: 8 total, 8 up, 8 in 2024-04-07T16:20:35.128 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:34 smithi100 bash[74942]: audit 2024-04-07T16:20:33.809783+0000 mon.c (mon.2) 179 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:20:35.128 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:34 smithi100 bash[74942]: cephadm 2024-04-07T16:20:33.812839+0000 mgr.y (mgr.25175) 16 : cephadm [INF] Updating smithi100:/etc/ceph/ceph.conf 2024-04-07T16:20:35.128 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:34 smithi100 bash[74942]: cephadm 2024-04-07T16:20:33.813224+0000 mgr.y (mgr.25175) 17 : cephadm [INF] Updating smithi144:/etc/ceph/ceph.conf 2024-04-07T16:20:35.128 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:34 smithi100 bash[74942]: cephadm 2024-04-07T16:20:34.029060+0000 mgr.y (mgr.25175) 18 : cephadm [INF] Updating smithi100:/var/lib/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/config/ceph.conf 2024-04-07T16:20:35.128 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:34 smithi100 bash[74942]: cephadm 2024-04-07T16:20:34.029636+0000 mgr.y (mgr.25175) 19 : cephadm [INF] Updating smithi144:/var/lib/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/config/ceph.conf 2024-04-07T16:20:35.128 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:34 smithi100 bash[74942]: cephadm 2024-04-07T16:20:34.193722+0000 mgr.y (mgr.25175) 20 : cephadm [INF] Updating smithi100:/etc/ceph/ceph.client.admin.keyring 2024-04-07T16:20:35.129 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:34 smithi100 bash[74942]: cephadm 2024-04-07T16:20:34.194291+0000 mgr.y (mgr.25175) 21 : cephadm [INF] Updating smithi144:/etc/ceph/ceph.client.admin.keyring 2024-04-07T16:20:35.129 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:34 smithi100 bash[74942]: audit 2024-04-07T16:20:34.532395+0000 mon.a (mon.0) 70 : audit [INF] from='mgr.25175 ' entity='mgr.y' 2024-04-07T16:20:35.129 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:34 smithi100 bash[74942]: audit 2024-04-07T16:20:34.538480+0000 mon.a (mon.0) 71 : audit [INF] from='mgr.25175 ' entity='mgr.y' 2024-04-07T16:20:35.129 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:34 smithi100 bash[74942]: audit 2024-04-07T16:20:34.552030+0000 mon.a (mon.0) 72 : audit [INF] from='mgr.25175 ' entity='mgr.y' 2024-04-07T16:20:35.129 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:34 smithi100 bash[74942]: audit 2024-04-07T16:20:34.558831+0000 mon.a (mon.0) 73 : audit [INF] from='mgr.25175 ' entity='mgr.y' 2024-04-07T16:20:35.129 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:34 smithi100 bash[74942]: audit 2024-04-07T16:20:34.573859+0000 mon.a (mon.0) 74 : audit [INF] from='mgr.25175 ' entity='mgr.y' 2024-04-07T16:20:35.129 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:34 smithi100 bash[74942]: audit 2024-04-07T16:20:34.674140+0000 mon.c (mon.2) 180 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:20:35.129 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:34 smithi100 bash[74942]: audit 2024-04-07T16:20:34.679040+0000 mon.c (mon.2) 181 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:20:35.131 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:34 smithi100 bash[74942]: audit 2024-04-07T16:20:34.682105+0000 mon.c (mon.2) 182 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-04-07T16:20:35.131 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:34 smithi100 bash[74942]: audit 2024-04-07T16:20:34.683739+0000 mon.c (mon.2) 183 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon ok-to-stop", "ids": ["c"]}]: dispatch 2024-04-07T16:20:35.131 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:34 smithi100 bash[31660]: audit 2024-04-07T16:20:33.790014+0000 mon.c (mon.2) 178 : audit 0 from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:20:35.131 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:34 smithi100 bash[31660]: cluster 2024-04-07T16:20:33.807412+0000 mon.a (mon.0) 69 : cluster 0 osdmap e297: 8 total, 8 up, 8 in 2024-04-07T16:20:35.131 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:34 smithi100 bash[31660]: audit 2024-04-07T16:20:33.809783+0000 mon.c (mon.2) 179 : audit 1 from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:20:35.131 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:34 smithi100 bash[31660]: cephadm 2024-04-07T16:20:33.812839+0000 mgr.y (mgr.25175) 16 : cephadm 1 Updating smithi100:/etc/ceph/ceph.conf 2024-04-07T16:20:35.131 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:34 smithi100 bash[31660]: cephadm 2024-04-07T16:20:33.813224+0000 mgr.y (mgr.25175) 17 : cephadm 1 Updating smithi144:/etc/ceph/ceph.conf 2024-04-07T16:20:35.131 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:34 smithi100 bash[31660]: cephadm 2024-04-07T16:20:34.029060+0000 mgr.y (mgr.25175) 18 : cephadm 1 Updating smithi100:/var/lib/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/config/ceph.conf 2024-04-07T16:20:35.131 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:34 smithi100 bash[31660]: cephadm 2024-04-07T16:20:34.029636+0000 mgr.y (mgr.25175) 19 : cephadm 1 Updating smithi144:/var/lib/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/config/ceph.conf 2024-04-07T16:20:35.131 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:34 smithi100 bash[31660]: cephadm 2024-04-07T16:20:34.193722+0000 mgr.y (mgr.25175) 20 : cephadm 1 Updating smithi100:/etc/ceph/ceph.client.admin.keyring 2024-04-07T16:20:35.132 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:34 smithi100 bash[31660]: cephadm 2024-04-07T16:20:34.194291+0000 mgr.y (mgr.25175) 21 : cephadm 1 Updating smithi144:/etc/ceph/ceph.client.admin.keyring 2024-04-07T16:20:35.132 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:34 smithi100 bash[31660]: audit 2024-04-07T16:20:34.532395+0000 mon.a (mon.0) 70 : audit 1 from='mgr.25175 ' entity='mgr.y' 2024-04-07T16:20:35.132 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:34 smithi100 bash[31660]: audit 2024-04-07T16:20:34.538480+0000 mon.a (mon.0) 71 : audit 1 from='mgr.25175 ' entity='mgr.y' 2024-04-07T16:20:35.132 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:34 smithi100 bash[31660]: audit 2024-04-07T16:20:34.552030+0000 mon.a (mon.0) 72 : audit 1 from='mgr.25175 ' entity='mgr.y' 2024-04-07T16:20:35.132 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:34 smithi100 bash[31660]: audit 2024-04-07T16:20:34.558831+0000 mon.a (mon.0) 73 : audit 1 from='mgr.25175 ' entity='mgr.y' 2024-04-07T16:20:35.132 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:34 smithi100 bash[31660]: audit 2024-04-07T16:20:34.573859+0000 mon.a (mon.0) 74 : audit 1 from='mgr.25175 ' entity='mgr.y' 2024-04-07T16:20:35.132 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:34 smithi100 bash[31660]: audit 2024-04-07T16:20:34.674140+0000 mon.c (mon.2) 180 : audit 0 from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:20:35.132 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:34 smithi100 bash[31660]: audit 2024-04-07T16:20:34.679040+0000 mon.c (mon.2) 181 : audit 0 from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:20:35.132 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:34 smithi100 bash[31660]: audit 2024-04-07T16:20:34.682105+0000 mon.c (mon.2) 182 : audit 0 from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-04-07T16:20:35.132 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:34 smithi100 bash[31660]: audit 2024-04-07T16:20:34.683739+0000 mon.c (mon.2) 183 : audit 0 from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon ok-to-stop", "ids": ["c"]}]: dispatch 2024-04-07T16:20:35.805 INFO:tasks.rados.rados.0.smithi100.stdout:1223: done (1 left) 2024-04-07T16:20:35.805 INFO:tasks.rados.rados.0.smithi100.stdout:1226: done (0 left) 2024-04-07T16:20:35.805 INFO:tasks.rados.rados.0.smithi100.stdout:1227: copy_from oid 38 from oid 8 current snap is 115 2024-04-07T16:20:35.805 INFO:tasks.rados.rados.0.smithi100.stdout:1228: delete oid 42 current snap is 115 2024-04-07T16:20:35.812 INFO:tasks.rados.rados.0.smithi100.stdout:1228: done (1 left) 2024-04-07T16:20:35.813 INFO:tasks.rados.rados.0.smithi100.stdout:1229: delete oid 29 current snap is 115 2024-04-07T16:20:35.817 INFO:tasks.rados.rados.0.smithi100.stdout:1227: finishing copy_from to smithi10066867-38 2024-04-07T16:20:35.817 INFO:tasks.rados.rados.0.smithi100.stdout:1227: got expected ENOENT (src dne) 2024-04-07T16:20:35.817 INFO:tasks.rados.rados.0.smithi100.stdout:1227: finishing copy_from racing read to smithi10066867-38 2024-04-07T16:20:35.818 INFO:tasks.rados.rados.0.smithi100.stdout:1227: done (1 left) 2024-04-07T16:20:35.818 INFO:tasks.rados.rados.0.smithi100.stdout:1229: done (0 left) 2024-04-07T16:20:35.818 INFO:tasks.rados.rados.0.smithi100.stdout:1230: rollback oid 34 current snap is 115 2024-04-07T16:20:35.819 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 34 to 89 2024-04-07T16:20:35.819 INFO:tasks.rados.rados.0.smithi100.stdout:1231: snap_remove snap 114 2024-04-07T16:20:36.105 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:35 smithi144 bash[23522]: cephadm 2024-04-07T16:20:34.358006+0000 mgr.y (mgr.25175) 22 : cephadm 1 Updating smithi100:/var/lib/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/config/ceph.client.admin.keyring 2024-04-07T16:20:36.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:35 smithi144 bash[23522]: cephadm 2024-04-07T16:20:34.358890+0000 mgr.y (mgr.25175) 23 : cephadm 1 Updating smithi144:/var/lib/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/config/ceph.client.admin.keyring 2024-04-07T16:20:36.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:35 smithi144 bash[23522]: cluster 2024-04-07T16:20:34.603294+0000 mgr.y (mgr.25175) 24 : cluster 0 pgmap v29: 121 pgs: 121 active+clean; 222 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 6.8 MiB/s rd, 1.9 MiB/s wr, 15 op/s 2024-04-07T16:20:36.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:35 smithi144 bash[23522]: cephadm 2024-04-07T16:20:34.684960+0000 mgr.y (mgr.25175) 25 : cephadm 1 Upgrade: It appears safe to stop mon.c 2024-04-07T16:20:36.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:35 smithi144 bash[23522]: cluster 2024-04-07T16:20:34.808657+0000 mon.a (mon.0) 75 : cluster 0 osdmap e298: 8 total, 8 up, 8 in 2024-04-07T16:20:36.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:35 smithi100 bash[31660]: cephadm 2024-04-07T16:20:34.358006+0000 mgr.y (mgr.25175) 22 : cephadm 1 Updating smithi100:/var/lib/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/config/ceph.client.admin.keyring 2024-04-07T16:20:36.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:35 smithi100 bash[31660]: cephadm 2024-04-07T16:20:34.358890+0000 mgr.y (mgr.25175) 23 : cephadm 1 Updating smithi144:/var/lib/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/config/ceph.client.admin.keyring 2024-04-07T16:20:36.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:35 smithi100 bash[31660]: cluster 2024-04-07T16:20:34.603294+0000 mgr.y (mgr.25175) 24 : cluster 0 pgmap v29: 121 pgs: 121 active+clean; 222 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 6.8 MiB/s rd, 1.9 MiB/s wr, 15 op/s 2024-04-07T16:20:36.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:35 smithi100 bash[31660]: cephadm 2024-04-07T16:20:34.684960+0000 mgr.y (mgr.25175) 25 : cephadm 1 Upgrade: It appears safe to stop mon.c 2024-04-07T16:20:36.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:35 smithi100 bash[31660]: cluster 2024-04-07T16:20:34.808657+0000 mon.a (mon.0) 75 : cluster 0 osdmap e298: 8 total, 8 up, 8 in 2024-04-07T16:20:36.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:35 smithi100 bash[74942]: cephadm 2024-04-07T16:20:34.358006+0000 mgr.y (mgr.25175) 22 : cephadm [INF] Updating smithi100:/var/lib/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/config/ceph.client.admin.keyring 2024-04-07T16:20:36.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:35 smithi100 bash[74942]: cephadm 2024-04-07T16:20:34.358890+0000 mgr.y (mgr.25175) 23 : cephadm [INF] Updating smithi144:/var/lib/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/config/ceph.client.admin.keyring 2024-04-07T16:20:36.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:35 smithi100 bash[74942]: cluster 2024-04-07T16:20:34.603294+0000 mgr.y (mgr.25175) 24 : cluster [DBG] pgmap v29: 121 pgs: 121 active+clean; 222 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 6.8 MiB/s rd, 1.9 MiB/s wr, 15 op/s 2024-04-07T16:20:36.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:35 smithi100 bash[74942]: cephadm 2024-04-07T16:20:34.684960+0000 mgr.y (mgr.25175) 25 : cephadm [INF] Upgrade: It appears safe to stop mon.c 2024-04-07T16:20:36.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:35 smithi100 bash[74942]: cluster 2024-04-07T16:20:34.808657+0000 mon.a (mon.0) 75 : cluster [DBG] osdmap e298: 8 total, 8 up, 8 in 2024-04-07T16:20:36.734 INFO:tasks.rados.rados.0.smithi100.stdout:1231: done (1 left) 2024-04-07T16:20:36.734 INFO:tasks.rados.rados.0.smithi100.stdout:1232: snap_remove snap 112 2024-04-07T16:20:36.936 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:36 smithi100 bash[74942]: cluster 2024-04-07T16:20:35.820455+0000 mon.a (mon.0) 76 : cluster [DBG] osdmap e299: 8 total, 8 up, 8 in 2024-04-07T16:20:36.936 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:36 smithi100 bash[74942]: cluster 2024-04-07T16:20:36.745884+0000 mon.a (mon.0) 77 : cluster [DBG] osdmap e300: 8 total, 8 up, 8 in 2024-04-07T16:20:36.936 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:36 smithi100 bash[31660]: cluster 2024-04-07T16:20:35.820455+0000 mon.a (mon.0) 76 : cluster 0 osdmap e299: 8 total, 8 up, 8 in 2024-04-07T16:20:36.936 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:36 smithi100 bash[31660]: cluster 2024-04-07T16:20:36.745884+0000 mon.a (mon.0) 77 : cluster 0 osdmap e300: 8 total, 8 up, 8 in 2024-04-07T16:20:37.105 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:36 smithi144 bash[23522]: cluster 2024-04-07T16:20:35.820455+0000 mon.a (mon.0) 76 : cluster 0 osdmap e299: 8 total, 8 up, 8 in 2024-04-07T16:20:37.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:36 smithi144 bash[23522]: cluster 2024-04-07T16:20:36.745884+0000 mon.a (mon.0) 77 : cluster 0 osdmap e300: 8 total, 8 up, 8 in 2024-04-07T16:20:37.738 INFO:tasks.rados.rados.0.smithi100.stdout:1232: done (1 left) 2024-04-07T16:20:37.739 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 4 current snap is 115 2024-04-07T16:20:37.739 INFO:tasks.rados.rados.0.smithi100.stdout:1233: seq_num 378 ranges {0=737280,737280=720896,1458176=81920} 2024-04-07T16:20:37.753 INFO:tasks.rados.rados.0.smithi100.stdout:1233: writing smithi10066867-4 from 0 to 737280 tid 1 2024-04-07T16:20:37.768 INFO:tasks.rados.rados.0.smithi100.stdout:1233: writing smithi10066867-4 from 737280 to 1458176 tid 2 2024-04-07T16:20:37.770 INFO:tasks.rados.rados.0.smithi100.stdout:1233: writing smithi10066867-4 from 1458176 to 1540096 tid 3 2024-04-07T16:20:37.770 INFO:tasks.rados.rados.0.smithi100.stdout:1234: rollback oid 44 current snap is 115 2024-04-07T16:20:37.770 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 44 to 103 2024-04-07T16:20:37.770 INFO:tasks.rados.rados.0.smithi100.stdout:1230: finishing rollback tid 0 to smithi10066867-34 2024-04-07T16:20:37.770 INFO:tasks.rados.rados.0.smithi100.stdout:1230: finishing rollback tid 1 to smithi10066867-34 2024-04-07T16:20:37.771 INFO:tasks.rados.rados.0.smithi100.stdout:1230: finishing rollback tid 2 to smithi10066867-34 2024-04-07T16:20:37.771 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 34 v 556 (ObjNum 140 snap 39 seq_num 140) dirty exists 2024-04-07T16:20:37.771 INFO:tasks.rados.rados.0.smithi100.stdout:1233: finishing write tid 1 to smithi10066867-4 2024-04-07T16:20:37.771 INFO:tasks.rados.rados.0.smithi100.stdout:1230: done (2 left) 2024-04-07T16:20:37.771 INFO:tasks.rados.rados.0.smithi100.stdout:1235: snap_remove snap 89 2024-04-07T16:20:38.347 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:38 smithi100 bash[74942]: cluster 2024-04-07T16:20:36.604272+0000 mgr.y (mgr.25175) 26 : cluster [DBG] pgmap v32: 121 pgs: 121 active+clean; 222 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail 2024-04-07T16:20:38.347 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:38 smithi100 bash[74942]: cephadm 2024-04-07T16:20:37.105902+0000 mgr.y (mgr.25175) 27 : cephadm [INF] Upgrade: Updating mon.c 2024-04-07T16:20:38.347 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:38 smithi100 bash[74942]: audit 2024-04-07T16:20:37.115710+0000 mon.a (mon.0) 78 : audit [INF] from='mgr.25175 ' entity='mgr.y' 2024-04-07T16:20:38.347 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:38 smithi100 bash[74942]: audit 2024-04-07T16:20:37.121759+0000 mon.c (mon.2) 184 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-07T16:20:38.348 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:38 smithi100 bash[74942]: audit 2024-04-07T16:20:37.123235+0000 mon.c (mon.2) 185 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-07T16:20:38.348 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:38 smithi100 bash[74942]: audit 2024-04-07T16:20:37.124921+0000 mon.c (mon.2) 186 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:20:38.348 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:38 smithi100 bash[74942]: cephadm 2024-04-07T16:20:37.126741+0000 mgr.y (mgr.25175) 28 : cephadm [INF] Deploying daemon mon.c on smithi100 2024-04-07T16:20:38.348 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:38 smithi100 bash[74942]: cluster 2024-04-07T16:20:37.753858+0000 mon.a (mon.0) 79 : cluster [DBG] osdmap e301: 8 total, 8 up, 8 in 2024-04-07T16:20:38.348 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:38 smithi100 bash[31660]: cluster 2024-04-07T16:20:36.604272+0000 mgr.y (mgr.25175) 26 : cluster 0 pgmap v32: 121 pgs: 121 active+clean; 222 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail 2024-04-07T16:20:38.348 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:38 smithi100 bash[31660]: cephadm 2024-04-07T16:20:37.105902+0000 mgr.y (mgr.25175) 27 : cephadm 1 Upgrade: Updating mon.c 2024-04-07T16:20:38.348 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:38 smithi100 bash[31660]: audit 2024-04-07T16:20:37.115710+0000 mon.a (mon.0) 78 : audit 1 from='mgr.25175 ' entity='mgr.y' 2024-04-07T16:20:38.348 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:38 smithi100 bash[31660]: audit 2024-04-07T16:20:37.121759+0000 mon.c (mon.2) 184 : audit 1 from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-07T16:20:38.348 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:38 smithi100 bash[31660]: audit 2024-04-07T16:20:37.123235+0000 mon.c (mon.2) 185 : audit 0 from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-07T16:20:38.348 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:38 smithi100 bash[31660]: audit 2024-04-07T16:20:37.124921+0000 mon.c (mon.2) 186 : audit 0 from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:20:38.348 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:38 smithi100 bash[31660]: cephadm 2024-04-07T16:20:37.126741+0000 mgr.y (mgr.25175) 28 : cephadm 1 Deploying daemon mon.c on smithi100 2024-04-07T16:20:38.348 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:38 smithi100 bash[31660]: cluster 2024-04-07T16:20:37.753858+0000 mon.a (mon.0) 79 : cluster 0 osdmap e301: 8 total, 8 up, 8 in 2024-04-07T16:20:38.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:38 smithi144 bash[23522]: cluster 2024-04-07T16:20:36.604272+0000 mgr.y (mgr.25175) 26 : cluster 0 pgmap v32: 121 pgs: 121 active+clean; 222 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail 2024-04-07T16:20:38.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:38 smithi144 bash[23522]: cephadm 2024-04-07T16:20:37.105902+0000 mgr.y (mgr.25175) 27 : cephadm 1 Upgrade: Updating mon.c 2024-04-07T16:20:38.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:38 smithi144 bash[23522]: audit 2024-04-07T16:20:37.115710+0000 mon.a (mon.0) 78 : audit 1 from='mgr.25175 ' entity='mgr.y' 2024-04-07T16:20:38.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:38 smithi144 bash[23522]: audit 2024-04-07T16:20:37.121759+0000 mon.c (mon.2) 184 : audit 1 from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-07T16:20:38.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:38 smithi144 bash[23522]: audit 2024-04-07T16:20:37.123235+0000 mon.c (mon.2) 185 : audit 0 from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-07T16:20:38.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:38 smithi144 bash[23522]: audit 2024-04-07T16:20:37.124921+0000 mon.c (mon.2) 186 : audit 0 from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:20:38.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:38 smithi144 bash[23522]: cephadm 2024-04-07T16:20:37.126741+0000 mgr.y (mgr.25175) 28 : cephadm 1 Deploying daemon mon.c on smithi100 2024-04-07T16:20:38.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:38 smithi144 bash[23522]: cluster 2024-04-07T16:20:37.753858+0000 mon.a (mon.0) 79 : cluster 0 osdmap e301: 8 total, 8 up, 8 in 2024-04-07T16:20:38.628 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:20:38 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:20:38] "GET /metrics HTTP/1.1" 200 36093 "" "Prometheus/2.43.0" 2024-04-07T16:20:38.738 INFO:tasks.rados.rados.0.smithi100.stdout:1234: finishing rollback tid 0 to smithi10066867-44 2024-04-07T16:20:38.739 INFO:tasks.rados.rados.0.smithi100.stdout:1234: finishing rollback tid 1 to smithi10066867-44 2024-04-07T16:20:38.739 INFO:tasks.rados.rados.0.smithi100.stdout:1233: finishing write tid 2 to smithi10066867-4 2024-04-07T16:20:38.739 INFO:tasks.rados.rados.0.smithi100.stdout:1233: finishing write tid 3 to smithi10066867-4 2024-04-07T16:20:38.739 INFO:tasks.rados.rados.0.smithi100.stdout:1234: finishing rollback tid 2 to smithi10066867-44 2024-04-07T16:20:38.739 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 44 v 670 (ObjNum 171 snap 44 seq_num 171) dirty exists 2024-04-07T16:20:38.739 INFO:tasks.rados.rados.0.smithi100.stdout:1233: finishing write tid 4 to smithi10066867-4 2024-04-07T16:20:38.739 INFO:tasks.rados.rados.0.smithi100.stdout:1233: finishing write tid 5 to smithi10066867-4 2024-04-07T16:20:38.739 INFO:tasks.rados.rados.0.smithi100.stdout:1233: oid 4 updating version 0 to 809 2024-04-07T16:20:38.739 INFO:tasks.rados.rados.0.smithi100.stdout:1233: oid 4 version 809 is already newer than 808 2024-04-07T16:20:38.739 INFO:tasks.rados.rados.0.smithi100.stdout:1233: oid 4 version 809 is already newer than 807 2024-04-07T16:20:38.739 INFO:tasks.rados.rados.0.smithi100.stdout:1233: oid 4 version 809 is already newer than 806 2024-04-07T16:20:38.739 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 4 v 809 (ObjNum 378 snap 115 seq_num 378) dirty exists 2024-04-07T16:20:38.739 INFO:tasks.rados.rados.0.smithi100.stdout:1233: left oid 4 (ObjNum 378 snap 115 seq_num 378) 2024-04-07T16:20:38.740 INFO:tasks.rados.rados.0.smithi100.stdout:1233: done (2 left) 2024-04-07T16:20:38.740 INFO:tasks.rados.rados.0.smithi100.stdout:1234: done (1 left) 2024-04-07T16:20:38.740 INFO:tasks.rados.rados.0.smithi100.stdout:1235: done (0 left) 2024-04-07T16:20:38.740 INFO:tasks.rados.rados.0.smithi100.stdout:1236: copy_from oid 33 from oid 48 current snap is 115 2024-04-07T16:20:38.740 INFO:tasks.rados.rados.0.smithi100.stdout:1237: snap_create 2024-04-07T16:20:38.775 INFO:tasks.rados.rados.0.smithi100.stdout:1236: finishing copy_from racing read to smithi10066867-33 2024-04-07T16:20:38.784 INFO:tasks.rados.rados.0.smithi100.stdout:1236: finishing copy_from to smithi10066867-33 2024-04-07T16:20:38.784 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 33 v 526 (ObjNum 372 snap 112 seq_num 372) dirty exists 2024-04-07T16:20:39.435 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:39 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:20:39.436 INFO:journalctl@ceph.osd.3.smithi100.stdout:Apr 07 16:20:39 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:20:39.436 INFO:journalctl@ceph.osd.1.smithi100.stdout:Apr 07 16:20:39 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:20:39.436 INFO:journalctl@ceph.osd.2.smithi100.stdout:Apr 07 16:20:39 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:20:39.437 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:20:39 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:20:39.437 INFO:journalctl@ceph.osd.0.smithi100.stdout:Apr 07 16:20:39 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:20:39.437 INFO:journalctl@ceph.mds.a.smithi100.stdout:Apr 07 16:20:39 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:20:39.438 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 16:20:39 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:20:39.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:39 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:20:39.438 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Apr 07 16:20:39 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:20:39.700 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:20:39 smithi100 bash[68137]: [07/Apr/2024:16:20:39] ENGINE Bus STOPPING 2024-04-07T16:20:39.700 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:20:39 smithi100 bash[68137]: [07/Apr/2024:16:20:39] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-04-07T16:20:39.700 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:20:39 smithi100 bash[68137]: [07/Apr/2024:16:20:39] ENGINE Bus STOPPED 2024-04-07T16:20:39.700 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:20:39 smithi100 bash[68137]: [07/Apr/2024:16:20:39] ENGINE Bus STARTING 2024-04-07T16:20:39.701 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:39 smithi100 systemd[1]: Stopping Ceph mon.c for 0702e3a8-f4f8-11ee-b649-cb9ed24678a4... 2024-04-07T16:20:39.701 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:39 smithi100 bash[31660]: debug 2024-04-07T16:20:39.581+0000 7f8edf276700 -1 received signal: Terminated from /sbin/docker-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-stderr=true (PID: 1) UID: 0 2024-04-07T16:20:39.701 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:39 smithi100 bash[31660]: debug 2024-04-07T16:20:39.581+0000 7f8edf276700 -1 mon.c@2(peon) e3 *** Got Signal Terminated *** 2024-04-07T16:20:40.186 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:20:39 smithi100 bash[68137]: [07/Apr/2024:16:20:39] ENGINE Serving on http://:::9283 2024-04-07T16:20:40.186 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:20:39 smithi100 bash[68137]: [07/Apr/2024:16:20:39] ENGINE Bus STARTED 2024-04-07T16:20:40.637 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:40 smithi100 bash[76049]: ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4-mon-c 2024-04-07T16:20:40.638 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:40 smithi100 bash[76120]: Error response from daemon: No such container: ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4-mon-c 2024-04-07T16:20:40.638 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:40 smithi100 systemd[1]: ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@mon.c.service: Deactivated successfully. 2024-04-07T16:20:40.638 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:40 smithi100 systemd[1]: Stopped Ceph mon.c for 0702e3a8-f4f8-11ee-b649-cb9ed24678a4. 2024-04-07T16:20:40.936 INFO:journalctl@ceph.mds.a.smithi100.stdout:Apr 07 16:20:40 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:20:40.936 INFO:journalctl@ceph.osd.1.smithi100.stdout:Apr 07 16:20:40 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:20:40.936 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:40 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:20:40.937 INFO:journalctl@ceph.osd.0.smithi100.stdout:Apr 07 16:20:40 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:20:40.937 INFO:journalctl@ceph.osd.2.smithi100.stdout:Apr 07 16:20:40 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:20:40.937 INFO:journalctl@ceph.osd.3.smithi100.stdout:Apr 07 16:20:40 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:20:40.937 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 16:20:40 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:20:40.938 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:20:40 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:20:40.938 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Apr 07 16:20:40 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:20:40.938 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:40 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:20:40.938 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:40 smithi100 systemd[1]: Started Ceph mon.c for 0702e3a8-f4f8-11ee-b649-cb9ed24678a4. 2024-04-07T16:20:41.757 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.741+0000 7fd7c508ec80 0 set uid:gid to 167:167 (ceph:ceph) 2024-04-07T16:20:41.758 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.741+0000 7fd7c508ec80 0 ceph version 19.0.0-2785-gadce92d0 (adce92d056533f22dc1ee37e3141d098b046ec1b) squid (dev), process ceph-mon, pid 7 2024-04-07T16:20:41.758 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.741+0000 7fd7c508ec80 0 pidfile_write: ignore empty --pid-file 2024-04-07T16:20:41.758 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.745+0000 7fd7c508ec80 0 load: jerasure load: lrc 2024-04-07T16:20:41.758 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: RocksDB version: 7.9.2 2024-04-07T16:20:41.758 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Git sha 0 2024-04-07T16:20:41.758 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Compile date 2024-04-05 22:33:20 2024-04-07T16:20:41.758 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: DB SUMMARY 2024-04-07T16:20:41.758 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: DB Session ID: 9CDXMMWY1QBV7TCFGNJW 2024-04-07T16:20:41.758 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: CURRENT file: CURRENT 2024-04-07T16:20:41.759 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: IDENTITY file: IDENTITY 2024-04-07T16:20:41.759 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: MANIFEST file: MANIFEST-000010 size: 1626 Bytes 2024-04-07T16:20:41.759 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 1, files: 000030.sst 2024-04-07T16:20:41.759 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000028.log size: 5773644 ; 2024-04-07T16:20:41.759 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.error_if_exists: 0 2024-04-07T16:20:41.759 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.create_if_missing: 0 2024-04-07T16:20:41.759 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.paranoid_checks: 1 2024-04-07T16:20:41.759 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.flush_verify_memtable_count: 1 2024-04-07T16:20:41.759 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-07T16:20:41.759 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-07T16:20:41.759 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.env: 0x556cf6693c60 2024-04-07T16:20:41.759 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.fs: PosixFileSystem 2024-04-07T16:20:41.760 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.info_log: 0x556cf6fe9c00 2024-04-07T16:20:41.760 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.max_file_opening_threads: 16 2024-04-07T16:20:41.760 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.statistics: (nil) 2024-04-07T16:20:41.760 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.use_fsync: 0 2024-04-07T16:20:41.760 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.max_log_file_size: 0 2024-04-07T16:20:41.760 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-07T16:20:41.760 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.log_file_time_to_roll: 0 2024-04-07T16:20:41.760 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.keep_log_file_num: 1000 2024-04-07T16:20:41.760 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.recycle_log_file_num: 0 2024-04-07T16:20:41.760 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.allow_fallocate: 1 2024-04-07T16:20:41.760 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.allow_mmap_reads: 0 2024-04-07T16:20:41.761 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.allow_mmap_writes: 0 2024-04-07T16:20:41.761 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.use_direct_reads: 0 2024-04-07T16:20:41.761 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-07T16:20:41.761 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.create_missing_column_families: 0 2024-04-07T16:20:41.761 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.db_log_dir: 2024-04-07T16:20:41.761 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.wal_dir: 2024-04-07T16:20:41.761 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.table_cache_numshardbits: 6 2024-04-07T16:20:41.761 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.WAL_ttl_seconds: 0 2024-04-07T16:20:41.761 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.WAL_size_limit_MB: 0 2024-04-07T16:20:41.761 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-07T16:20:41.761 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-07T16:20:41.761 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.is_fd_close_on_exec: 1 2024-04-07T16:20:41.761 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.advise_random_on_open: 1 2024-04-07T16:20:41.761 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.db_write_buffer_size: 0 2024-04-07T16:20:41.762 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.write_buffer_manager: 0x556cf6fba820 2024-04-07T16:20:41.762 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-07T16:20:41.762 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-07T16:20:41.762 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.use_adaptive_mutex: 0 2024-04-07T16:20:41.762 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.rate_limiter: (nil) 2024-04-07T16:20:41.762 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-07T16:20:41.762 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.wal_recovery_mode: 2 2024-04-07T16:20:41.762 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.enable_thread_tracking: 0 2024-04-07T16:20:41.762 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.enable_pipelined_write: 0 2024-04-07T16:20:41.762 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.unordered_write: 0 2024-04-07T16:20:41.762 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-07T16:20:41.762 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-07T16:20:41.762 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-07T16:20:41.763 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-07T16:20:41.763 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.row_cache: None 2024-04-07T16:20:41.763 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.wal_filter: None 2024-04-07T16:20:41.763 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-07T16:20:41.763 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.allow_ingest_behind: 0 2024-04-07T16:20:41.763 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.two_write_queues: 0 2024-04-07T16:20:41.763 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.manual_wal_flush: 0 2024-04-07T16:20:41.763 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.wal_compression: 0 2024-04-07T16:20:41.763 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.atomic_flush: 0 2024-04-07T16:20:41.763 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-07T16:20:41.763 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.persist_stats_to_disk: 0 2024-04-07T16:20:41.763 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.write_dbid_to_manifest: 0 2024-04-07T16:20:41.763 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.log_readahead_size: 0 2024-04-07T16:20:41.764 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-07T16:20:41.764 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.best_efforts_recovery: 0 2024-04-07T16:20:41.764 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-07T16:20:41.764 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-07T16:20:41.764 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.allow_data_in_errors: 0 2024-04-07T16:20:41.764 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.db_host_id: __hostname__ 2024-04-07T16:20:41.764 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.enforce_single_del_contracts: true 2024-04-07T16:20:41.764 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.max_background_jobs: 2 2024-04-07T16:20:41.764 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.max_background_compactions: -1 2024-04-07T16:20:41.765 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.max_subcompactions: 1 2024-04-07T16:20:41.766 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-07T16:20:41.766 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-07T16:20:41.766 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.delayed_write_rate : 16777216 2024-04-07T16:20:41.766 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.max_total_wal_size: 0 2024-04-07T16:20:41.766 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-07T16:20:41.766 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.stats_dump_period_sec: 600 2024-04-07T16:20:41.766 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.stats_persist_period_sec: 600 2024-04-07T16:20:41.766 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-07T16:20:41.766 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.max_open_files: -1 2024-04-07T16:20:41.766 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.bytes_per_sync: 0 2024-04-07T16:20:41.766 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.wal_bytes_per_sync: 0 2024-04-07T16:20:41.767 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.strict_bytes_per_sync: 0 2024-04-07T16:20:41.767 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.compaction_readahead_size: 0 2024-04-07T16:20:41.767 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.max_background_flushes: -1 2024-04-07T16:20:41.767 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Compression algorithms supported: 2024-04-07T16:20:41.767 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-07T16:20:41.767 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: kZSTD supported: 0 2024-04-07T16:20:41.767 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: kXpressCompression supported: 0 2024-04-07T16:20:41.767 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: kLZ4HCCompression supported: 1 2024-04-07T16:20:41.767 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: kZlibCompression supported: 1 2024-04-07T16:20:41.767 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: kSnappyCompression supported: 1 2024-04-07T16:20:41.767 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: kLZ4Compression supported: 1 2024-04-07T16:20:41.768 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: kBZip2Compression supported: 0 2024-04-07T16:20:41.768 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Fast CRC32 supported: Supported on x86 2024-04-07T16:20:41.768 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: DMutex implementation: pthread_mutex_t 2024-04-07T16:20:41.768 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000010 2024-04-07T16:20:41.768 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-07T16:20:41.768 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-07T16:20:41.768 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.merge_operator: 2024-04-07T16:20:41.768 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.compaction_filter: None 2024-04-07T16:20:41.768 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.compaction_filter_factory: None 2024-04-07T16:20:41.768 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.sst_partitioner_factory: None 2024-04-07T16:20:41.768 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.memtable_factory: SkipListFactory 2024-04-07T16:20:41.768 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.table_factory: BlockBasedTable 2024-04-07T16:20:41.768 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x556cf6fe9de0) 2024-04-07T16:20:41.768 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: cache_index_and_filter_blocks: 1 2024-04-07T16:20:41.769 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-07T16:20:41.769 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-07T16:20:41.769 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: pin_top_level_index_and_filter: 1 2024-04-07T16:20:41.769 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: index_type: 0 2024-04-07T16:20:41.769 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: data_block_index_type: 0 2024-04-07T16:20:41.769 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: index_shortening: 1 2024-04-07T16:20:41.769 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: data_block_hash_table_util_ratio: 0.750000 2024-04-07T16:20:41.769 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: checksum: 4 2024-04-07T16:20:41.769 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: no_block_cache: 0 2024-04-07T16:20:41.769 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: block_cache: 0x556cf6feb090 2024-04-07T16:20:41.769 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: block_cache_name: BinnedLRUCache 2024-04-07T16:20:41.769 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: block_cache_options: 2024-04-07T16:20:41.769 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: capacity : 536870912 2024-04-07T16:20:41.769 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: num_shard_bits : 4 2024-04-07T16:20:41.770 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: strict_capacity_limit : 0 2024-04-07T16:20:41.770 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: high_pri_pool_ratio: 0.000 2024-04-07T16:20:41.770 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: block_cache_compressed: (nil) 2024-04-07T16:20:41.770 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: persistent_cache: (nil) 2024-04-07T16:20:41.770 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: block_size: 4096 2024-04-07T16:20:41.770 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: block_size_deviation: 10 2024-04-07T16:20:41.770 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: block_restart_interval: 16 2024-04-07T16:20:41.770 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: index_block_restart_interval: 1 2024-04-07T16:20:41.770 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: metadata_block_size: 4096 2024-04-07T16:20:41.770 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: partition_filters: 0 2024-04-07T16:20:41.770 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: use_delta_encoding: 1 2024-04-07T16:20:41.770 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: filter_policy: bloomfilter 2024-04-07T16:20:41.770 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: whole_key_filtering: 1 2024-04-07T16:20:41.770 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: verify_compression: 0 2024-04-07T16:20:41.771 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: read_amp_bytes_per_bit: 0 2024-04-07T16:20:41.771 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: format_version: 5 2024-04-07T16:20:41.771 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: enable_index_compression: 1 2024-04-07T16:20:41.771 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: block_align: 0 2024-04-07T16:20:41.771 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: max_auto_readahead_size: 262144 2024-04-07T16:20:41.771 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: prepopulate_block_cache: 0 2024-04-07T16:20:41.771 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: initial_auto_readahead_size: 8192 2024-04-07T16:20:41.771 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: num_file_reads_for_auto_readahead: 2 2024-04-07T16:20:41.771 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.write_buffer_size: 33554432 2024-04-07T16:20:41.771 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.max_write_buffer_number: 2 2024-04-07T16:20:41.771 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.compression: NoCompression 2024-04-07T16:20:41.771 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.bottommost_compression: Disabled 2024-04-07T16:20:41.771 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.prefix_extractor: nullptr 2024-04-07T16:20:41.771 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-07T16:20:41.772 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.num_levels: 7 2024-04-07T16:20:41.772 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-07T16:20:41.772 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-07T16:20:41.772 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-07T16:20:41.772 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-07T16:20:41.772 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-07T16:20:41.772 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-07T16:20:41.772 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-07T16:20:41.772 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-07T16:20:41.772 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-07T16:20:41.772 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-07T16:20:41.772 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-07T16:20:41.772 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-07T16:20:41.773 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.compression_opts.window_bits: -14 2024-04-07T16:20:41.773 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.compression_opts.level: 32767 2024-04-07T16:20:41.773 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.compression_opts.strategy: 0 2024-04-07T16:20:41.773 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-07T16:20:41.773 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-07T16:20:41.773 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-07T16:20:41.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-07T16:20:41.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.compression_opts.enabled: false 2024-04-07T16:20:41.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-07T16:20:41.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-07T16:20:41.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-07T16:20:41.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-07T16:20:41.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.target_file_size_base: 67108864 2024-04-07T16:20:41.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.target_file_size_multiplier: 1 2024-04-07T16:20:41.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-07T16:20:41.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-07T16:20:41.774 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-07T16:20:41.775 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-07T16:20:41.775 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-07T16:20:41.775 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-07T16:20:41.775 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-07T16:20:41.775 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-07T16:20:41.775 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-07T16:20:41.775 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-07T16:20:41.775 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-07T16:20:41.775 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-07T16:20:41.775 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-07T16:20:41.775 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.arena_block_size: 1048576 2024-04-07T16:20:41.775 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-07T16:20:41.775 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-07T16:20:41.775 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.disable_auto_compactions: 0 2024-04-07T16:20:41.775 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-07T16:20:41.776 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-07T16:20:41.776 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-07T16:20:41.776 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-07T16:20:41.776 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-07T16:20:41.776 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-07T16:20:41.776 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-07T16:20:41.776 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-07T16:20:41.776 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-07T16:20:41.776 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-07T16:20:41.776 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-07T16:20:41.776 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.inplace_update_support: 0 2024-04-07T16:20:41.776 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.inplace_update_num_locks: 10000 2024-04-07T16:20:41.776 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-07T16:20:41.777 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-07T16:20:41.777 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.memtable_huge_page_size: 0 2024-04-07T16:20:41.777 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.bloom_locality: 0 2024-04-07T16:20:41.777 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.max_successive_merges: 0 2024-04-07T16:20:41.777 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.optimize_filters_for_hits: 0 2024-04-07T16:20:41.777 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.paranoid_file_checks: 0 2024-04-07T16:20:41.777 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.force_consistency_checks: 1 2024-04-07T16:20:41.777 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.report_bg_io_stats: 0 2024-04-07T16:20:41.777 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.ttl: 2592000 2024-04-07T16:20:41.777 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.periodic_compaction_seconds: 0 2024-04-07T16:20:41.777 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-07T16:20:41.777 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-07T16:20:41.777 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.enable_blob_files: false 2024-04-07T16:20:41.777 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.min_blob_size: 0 2024-04-07T16:20:41.778 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.blob_file_size: 268435456 2024-04-07T16:20:41.778 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.blob_compression_type: NoCompression 2024-04-07T16:20:41.778 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.enable_blob_garbage_collection: false 2024-04-07T16:20:41.778 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-07T16:20:41.778 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-07T16:20:41.778 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-07T16:20:41.778 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.blob_file_starting_level: 0 2024-04-07T16:20:41.778 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-07T16:20:41.778 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-04-07T16:20:41.778 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 32, last_sequence is 18459, log_number is 28,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 28 2024-04-07T16:20:41.778 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 28 2024-04-07T16:20:41.778 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 353249be-0d7d-41e6-a59e-7eb2cd2ed96a 2024-04-07T16:20:41.778 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1712506841753439, "job": 1, "event": "recovery_started", "wal_files": [28]} 2024-04-07T16:20:41.778 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.749+0000 7fd7c508ec80 4 rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #28 mode 2 2024-04-07T16:20:41.779 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.765+0000 7fd7c508ec80 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1712506841769995, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 33, "file_size": 3699090, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 18464, "largest_seqno": 20298, "table_properties": {"data_size": 3690131, "index_size": 5993, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 1925, "raw_key_size": 18405, "raw_average_key_size": 24, "raw_value_size": 3673148, "raw_average_value_size": 4884, "num_data_blocks": 262, "num_entries": 752, "num_filter_entries": 752, "num_deletions": 31, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1712506841, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "353249be-0d7d-41e6-a59e-7eb2cd2ed96a", "db_session_id": "9CDXMMWY1QBV7TCFGNJW", "orig_file_number": 33, "seqno_to_time_mapping": "N/A"}} 2024-04-07T16:20:41.779 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.765+0000 7fd7c508ec80 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1712506841770108, "job": 1, "event": "recovery_finished"} 2024-04-07T16:20:41.779 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.765+0000 7fd7c508ec80 4 rocksdb: [db/version_set.cc:5047] Creating manifest 35 2024-04-07T16:20:41.779 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.765+0000 7fd7c508ec80 4 rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-04-07T16:20:41.779 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.769+0000 7fd7c508ec80 4 rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000028.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-07T16:20:41.779 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.769+0000 7fd7c508ec80 4 rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x556cf70de000 2024-04-07T16:20:41.779 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.769+0000 7fd7c508ec80 4 rocksdb: DB pointer 0x556cf70c8000 2024-04-07T16:20:41.779 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.769+0000 7fd7b37e8700 4 rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-07T16:20:41.779 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.769+0000 7fd7b37e8700 4 rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-07T16:20:41.779 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: ** DB Stats ** 2024-04-07T16:20:41.779 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: Uptime(secs): 0.0 total, 0.0 interval 2024-04-07T16:20:41.779 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-07T16:20:41.779 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-07T16:20:41.779 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-07T16:20:41.779 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-07T16:20:41.780 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-07T16:20:41.780 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-07T16:20:41.780 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: ** Compaction Stats [default] ** 2024-04-07T16:20:41.780 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-07T16:20:41.780 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-07T16:20:41.780 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: L0 1/0 3.53 MB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 474.2 0.01 0.00 1 0.007 0 0 0.0 0.0 2024-04-07T16:20:41.780 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: L6 1/0 10.86 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 2024-04-07T16:20:41.780 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: Sum 2/0 14.39 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 474.2 0.01 0.00 1 0.007 0 0 0.0 0.0 2024-04-07T16:20:41.780 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 474.2 0.01 0.00 1 0.007 0 0 0.0 0.0 2024-04-07T16:20:41.780 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: ** Compaction Stats [default] ** 2024-04-07T16:20:41.780 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-07T16:20:41.780 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-07T16:20:41.780 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 474.2 0.01 0.00 1 0.007 0 0 0.0 0.0 2024-04-07T16:20:41.780 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-07T16:20:41.780 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: Uptime(secs): 0.0 total, 0.0 interval 2024-04-07T16:20:41.781 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: Flush(GB): cumulative 0.003, interval 0.003 2024-04-07T16:20:41.781 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-07T16:20:41.781 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: AddFile(Total Files): cumulative 0, interval 0 2024-04-07T16:20:41.781 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: AddFile(L0 Files): cumulative 0, interval 0 2024-04-07T16:20:41.781 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: AddFile(Keys): cumulative 0, interval 0 2024-04-07T16:20:41.781 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: Cumulative compaction: 0.00 GB write, 167.37 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-07T16:20:41.781 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: Interval compaction: 0.00 GB write, 167.37 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-07T16:20:41.781 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-07T16:20:41.781 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: Block cache BinnedLRUCache@0x556cf6feb090#7 capacity: 512.00 MB usage: 8.11 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.6e-05 secs_since: 0 2024-04-07T16:20:41.781 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: Block cache entry stats(count,size,portion): FilterBlock(1,2.03 KB,0.00038743%) IndexBlock(1,6.08 KB,0.00115931%) Misc(1,0.00 KB,0%) 2024-04-07T16:20:41.781 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: ** File Read Latency Histogram By Level [default] ** 2024-04-07T16:20:41.781 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.769+0000 7fd7c508ec80 0 starting mon.c rank 2 at public addrs [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] at bind addrs [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] mon_data /var/lib/ceph/mon/ceph-c fsid 0702e3a8-f4f8-11ee-b649-cb9ed24678a4 2024-04-07T16:20:41.781 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.769+0000 7fd7c508ec80 1 mon.c@-1(???) e3 preinit fsid 0702e3a8-f4f8-11ee-b649-cb9ed24678a4 2024-04-07T16:20:41.781 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.769+0000 7fd7c508ec80 0 mon.c@-1(???).mds e6 new map 2024-04-07T16:20:41.781 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.769+0000 7fd7c508ec80 0 mon.c@-1(???).mds e6 print_map 2024-04-07T16:20:41.782 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: e6 2024-04-07T16:20:41.782 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: enable_multiple, ever_enabled_multiple: 1,1 2024-04-07T16:20:41.782 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-07T16:20:41.782 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: legacy client fscid: 1 2024-04-07T16:20:41.782 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: 2024-04-07T16:20:41.782 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: Filesystem 'cephfs' (1) 2024-04-07T16:20:41.782 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: fs_name cephfs 2024-04-07T16:20:41.782 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: epoch 6 2024-04-07T16:20:41.782 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-07T16:20:41.782 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: created 2024-04-07T16:12:13.365189+0000 2024-04-07T16:20:41.782 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: modified 2024-04-07T16:12:15.375075+0000 2024-04-07T16:20:41.782 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: tableserver 0 2024-04-07T16:20:41.782 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: root 0 2024-04-07T16:20:41.782 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: session_timeout 60 2024-04-07T16:20:41.783 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: session_autoclose 300 2024-04-07T16:20:41.783 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: max_file_size 1099511627776 2024-04-07T16:20:41.783 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: max_xattr_size 65536 2024-04-07T16:20:41.783 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: required_client_features {} 2024-04-07T16:20:41.783 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: last_failure 0 2024-04-07T16:20:41.783 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: last_failure_osd_epoch 0 2024-04-07T16:20:41.783 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-07T16:20:41.783 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: max_mds 1 2024-04-07T16:20:41.783 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: in 0 2024-04-07T16:20:41.783 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: up {0=24322} 2024-04-07T16:20:41.783 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: failed 2024-04-07T16:20:41.783 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: damaged 2024-04-07T16:20:41.783 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: stopped 2024-04-07T16:20:41.783 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: data_pools [3] 2024-04-07T16:20:41.784 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: metadata_pool 2 2024-04-07T16:20:41.784 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: inline_data disabled 2024-04-07T16:20:41.784 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: balancer 2024-04-07T16:20:41.784 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: bal_rank_mask -1 2024-04-07T16:20:41.784 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: standby_count_wanted 0 2024-04-07T16:20:41.784 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: qdb_cluster leader: 0 members: 2024-04-07T16:20:41.784 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: [mds.b{0:24322} state up:active seq 3 addr [v2:172.21.15.144:6832/226244023,v1:172.21.15.144:6833/226244023] compat {c=[1],r=[1],i=[7ff]}] 2024-04-07T16:20:41.784 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: 2024-04-07T16:20:41.784 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: 2024-04-07T16:20:41.784 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: Standby daemons: 2024-04-07T16:20:41.784 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: 2024-04-07T16:20:41.784 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: [mds.a{-1:14424} state up:standby seq 1 addr [v2:172.21.15.100:6834/3487006602,v1:172.21.15.100:6835/3487006602] compat {c=[1],r=[1],i=[7ff]}] 2024-04-07T16:20:41.784 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.773+0000 7fd7c508ec80 0 mon.c@-1(???).osd e302 crush map has features 3314933069573799936, adjusting msgr requires 2024-04-07T16:20:41.784 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.773+0000 7fd7c508ec80 0 mon.c@-1(???).osd e302 crush map has features 432629308056666112, adjusting msgr requires 2024-04-07T16:20:41.784 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.773+0000 7fd7c508ec80 0 mon.c@-1(???).osd e302 crush map has features 432629308056666112, adjusting msgr requires 2024-04-07T16:20:41.784 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.773+0000 7fd7c508ec80 0 mon.c@-1(???).osd e302 crush map has features 432629308056666112, adjusting msgr requires 2024-04-07T16:20:41.785 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:41 smithi100 bash[76202]: debug 2024-04-07T16:20:41.773+0000 7fd7c508ec80 1 mon.c@-1(???).paxosservice(auth 1..21) refresh upgraded, format 0 -> 3 2024-04-07T16:20:42.605 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:20:42 smithi144 bash[59794]: ts=2024-04-07T16:20:42.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:20:43.856 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:20:43 smithi144 bash[59794]: ts=2024-04-07T16:20:43.508Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=nodes name=CephNodeDiskspaceWarning index=4 msg="Evaluating rule failed" rule="alert: CephNodeDiskspaceWarning\nexpr: predict_linear(node_filesystem_free_bytes{device=~\"/.*\"}[2d], 3600 * 24 * 5)\n * on (instance) group_left (nodename) node_uname_info < 0\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.8.4\n severity: warning\n type: ceph_default\nannotations:\n description: Mountpoint {{ $labels.mountpoint }} on {{ $labels.nodename }} will\n be full in less than 5 days based on the 48 hour trailing fill rate.\n summary: Host filesystem free space is getting low\n" err="found duplicate series for the match group {instance=\"smithi100\"} on the right hand-side of the operation: [{__name__=\"node_uname_info\", domainname=\"(none)\", instance=\"smithi100\", job=\"node\", machine=\"x86_64\", nodename=\"smithi100\", release=\"5.15.0-101-generic\", sysname=\"Linux\", version=\"#111-Ubuntu SMP Tue Mar 5 20:16:58 UTC 2024\"}, {__name__=\"node_uname_info\", cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", domainname=\"(none)\", instance=\"smithi100\", job=\"node\", machine=\"x86_64\", nodename=\"smithi100\", release=\"5.15.0-101-generic\", sysname=\"Linux\", version=\"#111-Ubuntu SMP Tue Mar 5 20:16:58 UTC 2024\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:20:47.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:46 smithi144 bash[23522]: cluster 2024-04-07T16:20:38.604820+0000 mgr.y (mgr.25175) 29 : cluster 0 pgmap v35: 121 pgs: 121 active+clean; 238 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 2.4 MiB/s rd, 3.7 MiB/s wr, 14 op/s 2024-04-07T16:20:47.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:46 smithi144 bash[23522]: cluster 2024-04-07T16:20:38.752007+0000 mon.a (mon.0) 80 : cluster 0 osdmap e302: 8 total, 8 up, 8 in 2024-04-07T16:20:47.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:46 smithi144 bash[23522]: audit 2024-04-07T16:20:39.595174+0000 mon.a (mon.0) 81 : audit 0 from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:20:47.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:46 smithi144 bash[23522]: audit 2024-04-07T16:20:39.595449+0000 mon.a (mon.0) 82 : audit 0 from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:20:47.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:46 smithi144 bash[23522]: audit 2024-04-07T16:20:39.595688+0000 mon.a (mon.0) 83 : audit 0 from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:20:47.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:46 smithi144 bash[23522]: audit 2024-04-07T16:20:39.637989+0000 mon.a (mon.0) 84 : audit 0 from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:20:47.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:46 smithi144 bash[23522]: audit 2024-04-07T16:20:39.642369+0000 mon.a (mon.0) 85 : audit 0 from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config-key dump", "format": "json"}]: dispatch 2024-04-07T16:20:47.436 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:46 smithi100 bash[74942]: cluster 2024-04-07T16:20:38.604820+0000 mgr.y (mgr.25175) 29 : cluster [DBG] pgmap v35: 121 pgs: 121 active+clean; 238 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 2.4 MiB/s rd, 3.7 MiB/s wr, 14 op/s 2024-04-07T16:20:47.436 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:46 smithi100 bash[74942]: cluster 2024-04-07T16:20:38.752007+0000 mon.a (mon.0) 80 : cluster [DBG] osdmap e302: 8 total, 8 up, 8 in 2024-04-07T16:20:47.436 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:46 smithi100 bash[74942]: audit 2024-04-07T16:20:39.595174+0000 mon.a (mon.0) 81 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:20:47.436 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:46 smithi100 bash[74942]: audit 2024-04-07T16:20:39.595449+0000 mon.a (mon.0) 82 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:20:47.436 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:46 smithi100 bash[74942]: audit 2024-04-07T16:20:39.595688+0000 mon.a (mon.0) 83 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:20:47.436 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:46 smithi100 bash[74942]: audit 2024-04-07T16:20:39.637989+0000 mon.a (mon.0) 84 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:20:47.436 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:46 smithi100 bash[74942]: audit 2024-04-07T16:20:39.642369+0000 mon.a (mon.0) 85 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config-key dump", "format": "json"}]: dispatch 2024-04-07T16:20:48.001 INFO:tasks.rados.rados.0.smithi100.stdout:1236: done (1 left) 2024-04-07T16:20:48.002 INFO:tasks.rados.rados.0.smithi100.stdout:1237: done (0 left) 2024-04-07T16:20:48.002 INFO:tasks.rados.rados.0.smithi100.stdout:1238: copy_from oid 35 from oid 22 current snap is 116 2024-04-07T16:20:48.002 INFO:tasks.rados.rados.0.smithi100.stdout:1239: rollback oid 20 current snap is 116 2024-04-07T16:20:48.002 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 20 to 115 2024-04-07T16:20:48.002 INFO:tasks.rados.rados.0.smithi100.stdout:1240: delete oid 27 current snap is 116 2024-04-07T16:20:48.013 INFO:tasks.rados.rados.0.smithi100.stdout:1240: done (2 left) 2024-04-07T16:20:48.013 INFO:tasks.rados.rados.0.smithi100.stdout:1241: rollback oid 36 current snap is 116 2024-04-07T16:20:48.013 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 36 to 106 2024-04-07T16:20:48.013 INFO:tasks.rados.rados.0.smithi100.stdout:1242: delete oid 7 current snap is 116 2024-04-07T16:20:48.021 INFO:tasks.rados.rados.0.smithi100.stdout:1242: done (3 left) 2024-04-07T16:20:48.022 INFO:tasks.rados.rados.0.smithi100.stdout:1243: copy_from oid 1 from oid 4 current snap is 116 2024-04-07T16:20:48.022 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 21 current snap is 116 2024-04-07T16:20:48.022 INFO:tasks.rados.rados.0.smithi100.stdout:1244: seq_num 381 ranges {6332416=786432,7118848=417792,7536640=385024} 2024-04-07T16:20:48.108 INFO:tasks.rados.rados.0.smithi100.stdout:1244: writing smithi10066867-21 from 6332416 to 7118848 tid 1 2024-04-07T16:20:48.111 INFO:tasks.rados.rados.0.smithi100.stdout:1244: writing smithi10066867-21 from 7118848 to 7536640 tid 2 2024-04-07T16:20:48.114 INFO:tasks.rados.rados.0.smithi100.stdout:1244: writing smithi10066867-21 from 7536640 to 7921664 tid 3 2024-04-07T16:20:48.114 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 30 current snap is 116 2024-04-07T16:20:48.114 INFO:tasks.rados.rados.0.smithi100.stdout:1245: seq_num 382 ranges {4489216=802816,5292032=704512,5996544=98304} 2024-04-07T16:20:48.145 INFO:tasks.rados.rados.0.smithi100.stdout:1245: writing smithi10066867-30 from 4489216 to 5292032 tid 1 2024-04-07T16:20:48.150 INFO:tasks.rados.rados.0.smithi100.stdout:1245: writing smithi10066867-30 from 5292032 to 5996544 tid 2 2024-04-07T16:20:48.151 INFO:tasks.rados.rados.0.smithi100.stdout:1245: writing smithi10066867-30 from 5996544 to 6094848 tid 3 2024-04-07T16:20:48.151 INFO:tasks.rados.rados.0.smithi100.stdout:1246: rollback oid 10 current snap is 116 2024-04-07T16:20:48.151 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 10 to 109 2024-04-07T16:20:48.151 INFO:tasks.rados.rados.0.smithi100.stdout:1239: finishing rollback tid 0 to smithi10066867-20 2024-04-07T16:20:48.151 INFO:tasks.rados.rados.0.smithi100.stdout:1247: read oid 32 snap -1 2024-04-07T16:20:48.151 INFO:tasks.rados.rados.0.smithi100.stdout:1247: expect deleted 2024-04-07T16:20:48.151 INFO:tasks.rados.rados.0.smithi100.stdout:1248: snap_remove snap 113 2024-04-07T16:20:48.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:48 smithi144 bash[23522]: audit 2024-04-07T16:20:39.779810+0000 mon.a (mon.0) 86 : audit 0 from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:20:48.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:48 smithi144 bash[23522]: audit 2024-04-07T16:20:39.780423+0000 mon.a (mon.0) 87 : audit 0 from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:20:48.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:48 smithi144 bash[23522]: audit 2024-04-07T16:20:39.780881+0000 mon.a (mon.0) 88 : audit 0 from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:20:48.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:48 smithi144 bash[23522]: audit 2024-04-07T16:20:39.785524+0000 mon.a (mon.0) 89 : audit 0 from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:20:48.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:48 smithi144 bash[23522]: audit 2024-04-07T16:20:39.786009+0000 mon.a (mon.0) 90 : audit 0 from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:20:48.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:48 smithi144 bash[23522]: audit 2024-04-07T16:20:39.786474+0000 mon.a (mon.0) 91 : audit 0 from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:20:48.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:48 smithi144 bash[23522]: cluster 2024-04-07T16:20:40.605490+0000 mgr.y (mgr.25175) 30 : cluster 0 pgmap v37: 121 pgs: 121 active+clean; 238 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 2.0 MiB/s rd, 3.1 MiB/s wr, 12 op/s 2024-04-07T16:20:48.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:48 smithi144 bash[23522]: cluster 2024-04-07T16:20:41.980606+0000 mon.b (mon.1) 337 : cluster 1 mon.b calling monitor election 2024-04-07T16:20:48.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:48 smithi144 bash[23522]: cluster 2024-04-07T16:20:41.990025+0000 mon.a (mon.0) 92 : cluster 1 mon.a calling monitor election 2024-04-07T16:20:48.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:48 smithi144 bash[23522]: cluster 2024-04-07T16:20:42.606930+0000 mgr.y (mgr.25175) 31 : cluster 0 pgmap v38: 121 pgs: 121 active+clean; 191 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 1.6 MiB/s rd, 2.9 MiB/s wr, 11 op/s 2024-04-07T16:20:48.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:48 smithi144 bash[23522]: cluster 2024-04-07T16:20:44.607718+0000 mgr.y (mgr.25175) 32 : cluster 0 pgmap v39: 121 pgs: 121 active+clean; 191 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 1.2 MiB/s rd, 2.2 MiB/s wr, 8 op/s 2024-04-07T16:20:48.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:48 smithi144 bash[23522]: cluster 2024-04-07T16:20:46.608551+0000 mgr.y (mgr.25175) 33 : cluster 0 pgmap v40: 121 pgs: 121 active+clean; 191 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 115 B/s rd, 329 KiB/s wr, 1 op/s 2024-04-07T16:20:48.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:48 smithi144 bash[23522]: audit 2024-04-07T16:20:46.701412+0000 mon.a (mon.0) 93 : audit 0 from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:20:48.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:48 smithi144 bash[23522]: cluster 2024-04-07T16:20:46.992359+0000 mon.a (mon.0) 94 : cluster 1 mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-04-07T16:20:48.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:48 smithi144 bash[23522]: cluster 2024-04-07T16:20:47.002252+0000 mon.a (mon.0) 95 : cluster 0 monmap epoch 3 2024-04-07T16:20:48.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:48 smithi144 bash[23522]: cluster 2024-04-07T16:20:47.002290+0000 mon.a (mon.0) 96 : cluster 0 fsid 0702e3a8-f4f8-11ee-b649-cb9ed24678a4 2024-04-07T16:20:48.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:48 smithi144 bash[23522]: cluster 2024-04-07T16:20:47.002313+0000 mon.a (mon.0) 97 : cluster 0 last_changed 2024-04-07T16:04:34.801459+0000 2024-04-07T16:20:48.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:48 smithi144 bash[23522]: cluster 2024-04-07T16:20:47.002331+0000 mon.a (mon.0) 98 : cluster 0 created 2024-04-07T16:02:17.280630+0000 2024-04-07T16:20:48.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:48 smithi144 bash[23522]: cluster 2024-04-07T16:20:47.002348+0000 mon.a (mon.0) 99 : cluster 0 min_mon_release 18 (reef) 2024-04-07T16:20:48.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:48 smithi144 bash[23522]: cluster 2024-04-07T16:20:47.002368+0000 mon.a (mon.0) 100 : cluster 0 election_strategy: 1 2024-04-07T16:20:48.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:48 smithi144 bash[23522]: cluster 2024-04-07T16:20:47.002393+0000 mon.a (mon.0) 101 : cluster 0 0: [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] mon.a 2024-04-07T16:20:48.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:48 smithi144 bash[23522]: cluster 2024-04-07T16:20:47.002422+0000 mon.a (mon.0) 102 : cluster 0 1: [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] mon.b 2024-04-07T16:20:48.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:48 smithi144 bash[23522]: cluster 2024-04-07T16:20:47.002445+0000 mon.a (mon.0) 103 : cluster 0 2: [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] mon.c 2024-04-07T16:20:48.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:48 smithi144 bash[23522]: cluster 2024-04-07T16:20:47.020415+0000 mon.a (mon.0) 104 : cluster 0 fsmap cephfs:1 {0=b=up:active} 1 up:standby 2024-04-07T16:20:48.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:48 smithi144 bash[23522]: cluster 2024-04-07T16:20:47.020474+0000 mon.a (mon.0) 105 : cluster 0 osdmap e302: 8 total, 8 up, 8 in 2024-04-07T16:20:48.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:48 smithi144 bash[23522]: cluster 2024-04-07T16:20:47.021016+0000 mon.a (mon.0) 106 : cluster 0 mgrmap e41: y(active, since 30s), standbys: x 2024-04-07T16:20:48.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:48 smithi144 bash[23522]: cluster 2024-04-07T16:20:47.021294+0000 mon.a (mon.0) 107 : cluster 1 overall HEALTH_OK 2024-04-07T16:20:48.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:48 smithi144 bash[23522]: audit 2024-04-07T16:20:47.034735+0000 mon.a (mon.0) 108 : audit 1 from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:20:48.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:48 smithi144 bash[23522]: audit 2024-04-07T16:20:47.047726+0000 mon.a (mon.0) 109 : audit 1 from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:20:48.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:48 smithi144 bash[23522]: audit 2024-04-07T16:20:47.070160+0000 mon.a (mon.0) 110 : audit 0 from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:20:48.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:48 smithi144 bash[23522]: audit 2024-04-07T16:20:47.120424+0000 mon.b (mon.1) 338 : audit 1 from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T16:20:48.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:48 smithi144 bash[23522]: audit 2024-04-07T16:20:47.120542+0000 mon.b (mon.1) 339 : audit 1 from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T16:20:48.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:48 smithi144 bash[23522]: audit 2024-04-07T16:20:47.120799+0000 mon.a (mon.0) 111 : audit 1 from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T16:20:48.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:48 smithi144 bash[23522]: audit 2024-04-07T16:20:47.120938+0000 mon.a (mon.0) 112 : audit 1 from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T16:20:48.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:48 smithi144 bash[23522]: audit 2024-04-07T16:20:47.121406+0000 mon.a (mon.0) 113 : audit 0 from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:20:48.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:48 smithi144 bash[23522]: audit 2024-04-07T16:20:47.121982+0000 mon.a (mon.0) 114 : audit 0 from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:20:48.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:48 smithi144 bash[23522]: audit 2024-04-07T16:20:47.122571+0000 mon.a (mon.0) 115 : audit 0 from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:20:48.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:48 smithi144 bash[23522]: audit 2024-04-07T16:20:47.127214+0000 mon.a (mon.0) 116 : audit 0 from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:20:48.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:48 smithi144 bash[23522]: audit 2024-04-07T16:20:47.127764+0000 mon.a (mon.0) 117 : audit 0 from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:20:48.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:48 smithi144 bash[23522]: audit 2024-04-07T16:20:47.128392+0000 mon.a (mon.0) 118 : audit 0 from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:20:48.419 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:48 smithi100 bash[76202]: cluster 2024-04-07T16:20:38.604820+0000 mgr.y (mgr.25175) 29 : cluster [DBG] pgmap v35: 121 pgs: 121 active+clean; 238 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 2.4 MiB/s rd, 3.7 MiB/s wr, 14 op/s 2024-04-07T16:20:48.419 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:48 smithi100 bash[76202]: cluster 2024-04-07T16:20:38.752007+0000 mon.a (mon.0) 80 : cluster [DBG] osdmap e302: 8 total, 8 up, 8 in 2024-04-07T16:20:48.419 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:48 smithi100 bash[76202]: audit 2024-04-07T16:20:39.595174+0000 mon.a (mon.0) 81 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:20:48.419 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:48 smithi100 bash[76202]: audit 2024-04-07T16:20:39.595449+0000 mon.a (mon.0) 82 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:20:48.419 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:48 smithi100 bash[76202]: audit 2024-04-07T16:20:39.595688+0000 mon.a (mon.0) 83 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:20:48.419 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:48 smithi100 bash[76202]: audit 2024-04-07T16:20:39.637989+0000 mon.a (mon.0) 84 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:20:48.419 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:48 smithi100 bash[76202]: audit 2024-04-07T16:20:39.642369+0000 mon.a (mon.0) 85 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config-key dump", "format": "json"}]: dispatch 2024-04-07T16:20:48.419 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:48 smithi100 bash[76202]: audit 2024-04-07T16:20:39.779810+0000 mon.a (mon.0) 86 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:20:48.420 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:48 smithi100 bash[76202]: audit 2024-04-07T16:20:39.780423+0000 mon.a (mon.0) 87 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:20:48.420 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:48 smithi100 bash[76202]: audit 2024-04-07T16:20:39.780881+0000 mon.a (mon.0) 88 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:20:48.420 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:48 smithi100 bash[76202]: audit 2024-04-07T16:20:39.785524+0000 mon.a (mon.0) 89 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:20:48.420 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:48 smithi100 bash[76202]: audit 2024-04-07T16:20:39.786009+0000 mon.a (mon.0) 90 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:20:48.420 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:48 smithi100 bash[76202]: audit 2024-04-07T16:20:39.786474+0000 mon.a (mon.0) 91 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:20:48.420 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:48 smithi100 bash[76202]: cluster 2024-04-07T16:20:40.605490+0000 mgr.y (mgr.25175) 30 : cluster [DBG] pgmap v37: 121 pgs: 121 active+clean; 238 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 2.0 MiB/s rd, 3.1 MiB/s wr, 12 op/s 2024-04-07T16:20:48.420 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:48 smithi100 bash[76202]: cluster 2024-04-07T16:20:41.980606+0000 mon.b (mon.1) 337 : cluster [INF] mon.b calling monitor election 2024-04-07T16:20:48.420 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:48 smithi100 bash[76202]: cluster 2024-04-07T16:20:41.990025+0000 mon.a (mon.0) 92 : cluster [INF] mon.a calling monitor election 2024-04-07T16:20:48.420 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:48 smithi100 bash[76202]: cluster 2024-04-07T16:20:42.606930+0000 mgr.y (mgr.25175) 31 : cluster [DBG] pgmap v38: 121 pgs: 121 active+clean; 191 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 1.6 MiB/s rd, 2.9 MiB/s wr, 11 op/s 2024-04-07T16:20:48.420 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:48 smithi100 bash[76202]: cluster 2024-04-07T16:20:44.607718+0000 mgr.y (mgr.25175) 32 : cluster [DBG] pgmap v39: 121 pgs: 121 active+clean; 191 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 1.2 MiB/s rd, 2.2 MiB/s wr, 8 op/s 2024-04-07T16:20:48.420 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:48 smithi100 bash[76202]: cluster 2024-04-07T16:20:46.608551+0000 mgr.y (mgr.25175) 33 : cluster [DBG] pgmap v40: 121 pgs: 121 active+clean; 191 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 115 B/s rd, 329 KiB/s wr, 1 op/s 2024-04-07T16:20:48.421 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:48 smithi100 bash[74942]: audit 2024-04-07T16:20:39.779810+0000 mon.a (mon.0) 86 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:20:48.421 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:48 smithi100 bash[74942]: audit 2024-04-07T16:20:39.780423+0000 mon.a (mon.0) 87 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:20:48.421 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:48 smithi100 bash[74942]: audit 2024-04-07T16:20:39.780881+0000 mon.a (mon.0) 88 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:20:48.421 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:48 smithi100 bash[74942]: audit 2024-04-07T16:20:39.785524+0000 mon.a (mon.0) 89 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:20:48.421 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:48 smithi100 bash[74942]: audit 2024-04-07T16:20:39.786009+0000 mon.a (mon.0) 90 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:20:48.421 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:48 smithi100 bash[74942]: audit 2024-04-07T16:20:39.786474+0000 mon.a (mon.0) 91 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:20:48.421 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:48 smithi100 bash[74942]: cluster 2024-04-07T16:20:40.605490+0000 mgr.y (mgr.25175) 30 : cluster [DBG] pgmap v37: 121 pgs: 121 active+clean; 238 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 2.0 MiB/s rd, 3.1 MiB/s wr, 12 op/s 2024-04-07T16:20:48.421 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:48 smithi100 bash[74942]: cluster 2024-04-07T16:20:41.980606+0000 mon.b (mon.1) 337 : cluster [INF] mon.b calling monitor election 2024-04-07T16:20:48.421 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:48 smithi100 bash[74942]: cluster 2024-04-07T16:20:41.990025+0000 mon.a (mon.0) 92 : cluster [INF] mon.a calling monitor election 2024-04-07T16:20:48.421 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:48 smithi100 bash[74942]: cluster 2024-04-07T16:20:42.606930+0000 mgr.y (mgr.25175) 31 : cluster [DBG] pgmap v38: 121 pgs: 121 active+clean; 191 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 1.6 MiB/s rd, 2.9 MiB/s wr, 11 op/s 2024-04-07T16:20:48.421 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:48 smithi100 bash[74942]: cluster 2024-04-07T16:20:44.607718+0000 mgr.y (mgr.25175) 32 : cluster [DBG] pgmap v39: 121 pgs: 121 active+clean; 191 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 1.2 MiB/s rd, 2.2 MiB/s wr, 8 op/s 2024-04-07T16:20:48.421 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:48 smithi100 bash[74942]: cluster 2024-04-07T16:20:46.608551+0000 mgr.y (mgr.25175) 33 : cluster [DBG] pgmap v40: 121 pgs: 121 active+clean; 191 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 115 B/s rd, 329 KiB/s wr, 1 op/s 2024-04-07T16:20:48.422 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:48 smithi100 bash[74942]: audit 2024-04-07T16:20:46.701412+0000 mon.a (mon.0) 93 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:20:48.422 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:48 smithi100 bash[74942]: cluster 2024-04-07T16:20:46.992359+0000 mon.a (mon.0) 94 : cluster [INF] mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-04-07T16:20:48.422 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:48 smithi100 bash[74942]: cluster 2024-04-07T16:20:47.002252+0000 mon.a (mon.0) 95 : cluster [DBG] monmap epoch 3 2024-04-07T16:20:48.422 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:48 smithi100 bash[74942]: cluster 2024-04-07T16:20:47.002290+0000 mon.a (mon.0) 96 : cluster [DBG] fsid 0702e3a8-f4f8-11ee-b649-cb9ed24678a4 2024-04-07T16:20:48.422 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:48 smithi100 bash[74942]: cluster 2024-04-07T16:20:47.002313+0000 mon.a (mon.0) 97 : cluster [DBG] last_changed 2024-04-07T16:04:34.801459+0000 2024-04-07T16:20:48.422 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:48 smithi100 bash[74942]: cluster 2024-04-07T16:20:47.002331+0000 mon.a (mon.0) 98 : cluster [DBG] created 2024-04-07T16:02:17.280630+0000 2024-04-07T16:20:48.422 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:48 smithi100 bash[74942]: cluster 2024-04-07T16:20:47.002348+0000 mon.a (mon.0) 99 : cluster [DBG] min_mon_release 18 (reef) 2024-04-07T16:20:48.422 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:48 smithi100 bash[74942]: cluster 2024-04-07T16:20:47.002368+0000 mon.a (mon.0) 100 : cluster [DBG] election_strategy: 1 2024-04-07T16:20:48.422 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:48 smithi100 bash[74942]: cluster 2024-04-07T16:20:47.002393+0000 mon.a (mon.0) 101 : cluster [DBG] 0: [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] mon.a 2024-04-07T16:20:48.422 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:48 smithi100 bash[74942]: cluster 2024-04-07T16:20:47.002422+0000 mon.a (mon.0) 102 : cluster [DBG] 1: [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] mon.b 2024-04-07T16:20:48.422 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:48 smithi100 bash[74942]: cluster 2024-04-07T16:20:47.002445+0000 mon.a (mon.0) 103 : cluster [DBG] 2: [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] mon.c 2024-04-07T16:20:48.422 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:48 smithi100 bash[74942]: cluster 2024-04-07T16:20:47.020415+0000 mon.a (mon.0) 104 : cluster [DBG] fsmap cephfs:1 {0=b=up:active} 1 up:standby 2024-04-07T16:20:48.422 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:48 smithi100 bash[74942]: cluster 2024-04-07T16:20:47.020474+0000 mon.a (mon.0) 105 : cluster [DBG] osdmap e302: 8 total, 8 up, 8 in 2024-04-07T16:20:48.422 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:48 smithi100 bash[74942]: cluster 2024-04-07T16:20:47.021016+0000 mon.a (mon.0) 106 : cluster [DBG] mgrmap e41: y(active, since 30s), standbys: x 2024-04-07T16:20:48.423 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:48 smithi100 bash[74942]: cluster 2024-04-07T16:20:47.021294+0000 mon.a (mon.0) 107 : cluster [INF] overall HEALTH_OK 2024-04-07T16:20:48.423 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:48 smithi100 bash[74942]: audit 2024-04-07T16:20:47.034735+0000 mon.a (mon.0) 108 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:20:48.423 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:48 smithi100 bash[74942]: audit 2024-04-07T16:20:47.047726+0000 mon.a (mon.0) 109 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:20:48.423 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:48 smithi100 bash[74942]: audit 2024-04-07T16:20:47.070160+0000 mon.a (mon.0) 110 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:20:48.423 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:48 smithi100 bash[74942]: audit 2024-04-07T16:20:47.120424+0000 mon.b (mon.1) 338 : audit [INF] from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T16:20:48.423 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:48 smithi100 bash[74942]: audit 2024-04-07T16:20:47.120542+0000 mon.b (mon.1) 339 : audit [INF] from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T16:20:48.423 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:48 smithi100 bash[74942]: audit 2024-04-07T16:20:47.120799+0000 mon.a (mon.0) 111 : audit [INF] from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T16:20:48.423 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:48 smithi100 bash[74942]: audit 2024-04-07T16:20:47.120938+0000 mon.a (mon.0) 112 : audit [INF] from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T16:20:48.423 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:48 smithi100 bash[74942]: audit 2024-04-07T16:20:47.121406+0000 mon.a (mon.0) 113 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:20:48.423 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:48 smithi100 bash[74942]: audit 2024-04-07T16:20:47.121982+0000 mon.a (mon.0) 114 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:20:48.423 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:48 smithi100 bash[74942]: audit 2024-04-07T16:20:47.122571+0000 mon.a (mon.0) 115 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:20:48.423 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:48 smithi100 bash[74942]: audit 2024-04-07T16:20:47.127214+0000 mon.a (mon.0) 116 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:20:48.423 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:48 smithi100 bash[74942]: audit 2024-04-07T16:20:47.127764+0000 mon.a (mon.0) 117 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:20:48.423 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:48 smithi100 bash[74942]: audit 2024-04-07T16:20:47.128392+0000 mon.a (mon.0) 118 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:20:48.424 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:48 smithi100 bash[76202]: audit 2024-04-07T16:20:46.701412+0000 mon.a (mon.0) 93 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:20:48.424 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:48 smithi100 bash[76202]: cluster 2024-04-07T16:20:46.992359+0000 mon.a (mon.0) 94 : cluster [INF] mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-04-07T16:20:48.424 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:48 smithi100 bash[76202]: cluster 2024-04-07T16:20:47.002252+0000 mon.a (mon.0) 95 : cluster [DBG] monmap epoch 3 2024-04-07T16:20:48.424 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:48 smithi100 bash[76202]: cluster 2024-04-07T16:20:47.002290+0000 mon.a (mon.0) 96 : cluster [DBG] fsid 0702e3a8-f4f8-11ee-b649-cb9ed24678a4 2024-04-07T16:20:48.424 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:48 smithi100 bash[76202]: cluster 2024-04-07T16:20:47.002313+0000 mon.a (mon.0) 97 : cluster [DBG] last_changed 2024-04-07T16:04:34.801459+0000 2024-04-07T16:20:48.424 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:48 smithi100 bash[76202]: cluster 2024-04-07T16:20:47.002331+0000 mon.a (mon.0) 98 : cluster [DBG] created 2024-04-07T16:02:17.280630+0000 2024-04-07T16:20:48.424 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:48 smithi100 bash[76202]: cluster 2024-04-07T16:20:47.002348+0000 mon.a (mon.0) 99 : cluster [DBG] min_mon_release 18 (reef) 2024-04-07T16:20:48.424 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:48 smithi100 bash[76202]: cluster 2024-04-07T16:20:47.002368+0000 mon.a (mon.0) 100 : cluster [DBG] election_strategy: 1 2024-04-07T16:20:48.424 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:48 smithi100 bash[76202]: cluster 2024-04-07T16:20:47.002393+0000 mon.a (mon.0) 101 : cluster [DBG] 0: [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] mon.a 2024-04-07T16:20:48.424 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:48 smithi100 bash[76202]: cluster 2024-04-07T16:20:47.002422+0000 mon.a (mon.0) 102 : cluster [DBG] 1: [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] mon.b 2024-04-07T16:20:48.424 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:48 smithi100 bash[76202]: cluster 2024-04-07T16:20:47.002445+0000 mon.a (mon.0) 103 : cluster [DBG] 2: [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] mon.c 2024-04-07T16:20:48.425 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:48 smithi100 bash[76202]: cluster 2024-04-07T16:20:47.020415+0000 mon.a (mon.0) 104 : cluster [DBG] fsmap cephfs:1 {0=b=up:active} 1 up:standby 2024-04-07T16:20:48.425 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:48 smithi100 bash[76202]: cluster 2024-04-07T16:20:47.020474+0000 mon.a (mon.0) 105 : cluster [DBG] osdmap e302: 8 total, 8 up, 8 in 2024-04-07T16:20:48.425 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:48 smithi100 bash[76202]: cluster 2024-04-07T16:20:47.021016+0000 mon.a (mon.0) 106 : cluster [DBG] mgrmap e41: y(active, since 30s), standbys: x 2024-04-07T16:20:48.425 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:48 smithi100 bash[76202]: cluster 2024-04-07T16:20:47.021294+0000 mon.a (mon.0) 107 : cluster [INF] overall HEALTH_OK 2024-04-07T16:20:48.425 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:48 smithi100 bash[76202]: audit 2024-04-07T16:20:47.034735+0000 mon.a (mon.0) 108 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:20:48.425 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:48 smithi100 bash[76202]: audit 2024-04-07T16:20:47.047726+0000 mon.a (mon.0) 109 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:20:48.425 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:48 smithi100 bash[76202]: audit 2024-04-07T16:20:47.070160+0000 mon.a (mon.0) 110 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:20:48.425 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:48 smithi100 bash[76202]: audit 2024-04-07T16:20:47.120424+0000 mon.b (mon.1) 338 : audit [INF] from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T16:20:48.425 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:48 smithi100 bash[76202]: audit 2024-04-07T16:20:47.120542+0000 mon.b (mon.1) 339 : audit [INF] from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T16:20:48.425 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:48 smithi100 bash[76202]: audit 2024-04-07T16:20:47.120799+0000 mon.a (mon.0) 111 : audit [INF] from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T16:20:48.425 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:48 smithi100 bash[76202]: audit 2024-04-07T16:20:47.120938+0000 mon.a (mon.0) 112 : audit [INF] from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T16:20:48.425 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:48 smithi100 bash[76202]: audit 2024-04-07T16:20:47.121406+0000 mon.a (mon.0) 113 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:20:48.425 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:48 smithi100 bash[76202]: audit 2024-04-07T16:20:47.121982+0000 mon.a (mon.0) 114 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:20:48.426 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:48 smithi100 bash[76202]: audit 2024-04-07T16:20:47.122571+0000 mon.a (mon.0) 115 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:20:48.426 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:48 smithi100 bash[76202]: audit 2024-04-07T16:20:47.127214+0000 mon.a (mon.0) 116 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:20:48.426 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:48 smithi100 bash[76202]: audit 2024-04-07T16:20:47.127764+0000 mon.a (mon.0) 117 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:20:48.426 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:48 smithi100 bash[76202]: audit 2024-04-07T16:20:47.128392+0000 mon.a (mon.0) 118 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:20:48.686 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:20:48 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:20:48] "GET /metrics HTTP/1.1" 200 36082 "" "Prometheus/2.43.0" 2024-04-07T16:20:49.085 INFO:tasks.rados.rados.0.smithi100.stdout:1248: done (8 left) 2024-04-07T16:20:49.085 INFO:tasks.rados.rados.0.smithi100.stdout:1249: delete oid 34 current snap is 116 2024-04-07T16:20:49.085 INFO:tasks.rados.rados.0.smithi100.stdout:1239: finishing rollback tid 1 to smithi10066867-20 2024-04-07T16:20:49.086 INFO:tasks.rados.rados.0.smithi100.stdout:1239: finishing rollback tid 2 to smithi10066867-20 2024-04-07T16:20:49.086 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 20 v 518 (ObjNum 328 snap 99 seq_num 328) dirty exists 2024-04-07T16:20:49.086 INFO:tasks.rados.rados.0.smithi100.stdout:1241: finishing rollback tid 0 to smithi10066867-36 2024-04-07T16:20:49.086 INFO:tasks.rados.rados.0.smithi100.stdout:1241: finishing rollback tid 1 to smithi10066867-36 2024-04-07T16:20:49.086 INFO:tasks.rados.rados.0.smithi100.stdout:1241: finishing rollback tid 2 to smithi10066867-36 2024-04-07T16:20:49.086 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 36 v 550 (ObjNum 347 snap 104 seq_num 347) dirty exists 2024-04-07T16:20:49.086 INFO:tasks.rados.rados.0.smithi100.stdout:1238: finishing copy_from racing read to smithi10066867-35 2024-04-07T16:20:49.086 INFO:tasks.rados.rados.0.smithi100.stdout:1238: finishing copy_from to smithi10066867-35 2024-04-07T16:20:49.086 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 35 v 515 (ObjNum 294 snap 84 seq_num 294) dirty exists 2024-04-07T16:20:49.086 INFO:tasks.rados.rados.0.smithi100.stdout:1243: finishing copy_from racing read to smithi10066867-1 2024-04-07T16:20:49.086 INFO:tasks.rados.rados.0.smithi100.stdout:1243: finishing copy_from to smithi10066867-1 2024-04-07T16:20:49.086 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 1 v 809 (ObjNum 378 snap 115 seq_num 378) dirty exists 2024-04-07T16:20:49.086 INFO:tasks.rados.rados.0.smithi100.stdout:1244: finishing write tid 1 to smithi10066867-21 2024-04-07T16:20:49.086 INFO:tasks.rados.rados.0.smithi100.stdout:1244: finishing write tid 2 to smithi10066867-21 2024-04-07T16:20:49.086 INFO:tasks.rados.rados.0.smithi100.stdout:1244: finishing write tid 3 to smithi10066867-21 2024-04-07T16:20:49.087 INFO:tasks.rados.rados.0.smithi100.stdout:1244: finishing write tid 4 to smithi10066867-21 2024-04-07T16:20:49.087 INFO:tasks.rados.rados.0.smithi100.stdout:1244: finishing write tid 5 to smithi10066867-21 2024-04-07T16:20:49.087 INFO:tasks.rados.rados.0.smithi100.stdout:1244: oid 21 updating version 0 to 263 2024-04-07T16:20:49.087 INFO:tasks.rados.rados.0.smithi100.stdout:1244: oid 21 version 263 is already newer than 261 2024-04-07T16:20:49.087 INFO:tasks.rados.rados.0.smithi100.stdout:1244: oid 21 version 263 is already newer than 262 2024-04-07T16:20:49.087 INFO:tasks.rados.rados.0.smithi100.stdout:1244: oid 21 version 263 is already newer than 260 2024-04-07T16:20:49.087 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 21 v 263 (ObjNum 381 snap 116 seq_num 381) dirty exists 2024-04-07T16:20:49.087 INFO:tasks.rados.rados.0.smithi100.stdout:1244: left oid 21 (ObjNum 381 snap 116 seq_num 381) 2024-04-07T16:20:49.087 INFO:tasks.rados.rados.0.smithi100.stdout:1245: finishing write tid 1 to smithi10066867-30 2024-04-07T16:20:49.087 INFO:tasks.rados.rados.0.smithi100.stdout:1245: finishing write tid 2 to smithi10066867-30 2024-04-07T16:20:49.087 INFO:tasks.rados.rados.0.smithi100.stdout:1245: finishing write tid 3 to smithi10066867-30 2024-04-07T16:20:49.087 INFO:tasks.rados.rados.0.smithi100.stdout:1246: finishing rollback tid 0 to smithi10066867-10 2024-04-07T16:20:49.087 INFO:tasks.rados.rados.0.smithi100.stdout:1246: finishing rollback tid 1 to smithi10066867-10 2024-04-07T16:20:49.087 INFO:tasks.rados.rados.0.smithi100.stdout:1245: finishing write tid 4 to smithi10066867-30 2024-04-07T16:20:49.087 INFO:tasks.rados.rados.0.smithi100.stdout:1246: finishing rollback tid 2 to smithi10066867-10 2024-04-07T16:20:49.087 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 10 v 812 (ObjNum 296 snap 84 seq_num 296) dirty exists 2024-04-07T16:20:49.088 INFO:tasks.rados.rados.0.smithi100.stdout:1245: finishing write tid 5 to smithi10066867-30 2024-04-07T16:20:49.088 INFO:tasks.rados.rados.0.smithi100.stdout:1245: oid 30 updating version 0 to 507 2024-04-07T16:20:49.088 INFO:tasks.rados.rados.0.smithi100.stdout:1245: oid 30 updating version 507 to 510 2024-04-07T16:20:49.088 INFO:tasks.rados.rados.0.smithi100.stdout:1245: oid 30 version 510 is already newer than 509 2024-04-07T16:20:49.088 INFO:tasks.rados.rados.0.smithi100.stdout:1245: oid 30 version 510 is already newer than 508 2024-04-07T16:20:49.088 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 30 v 510 (ObjNum 382 snap 116 seq_num 382) dirty exists 2024-04-07T16:20:49.088 INFO:tasks.rados.rados.0.smithi100.stdout:1245: left oid 30 (ObjNum 382 snap 116 seq_num 382) 2024-04-07T16:20:49.099 INFO:tasks.rados.rados.0.smithi100.stdout:1238: done (8 left) 2024-04-07T16:20:49.099 INFO:tasks.rados.rados.0.smithi100.stdout:1239: done (7 left) 2024-04-07T16:20:49.099 INFO:tasks.rados.rados.0.smithi100.stdout:1241: done (6 left) 2024-04-07T16:20:49.099 INFO:tasks.rados.rados.0.smithi100.stdout:1243: done (5 left) 2024-04-07T16:20:49.099 INFO:tasks.rados.rados.0.smithi100.stdout:1244: done (4 left) 2024-04-07T16:20:49.099 INFO:tasks.rados.rados.0.smithi100.stdout:1245: done (3 left) 2024-04-07T16:20:49.099 INFO:tasks.rados.rados.0.smithi100.stdout:1246: done (2 left) 2024-04-07T16:20:49.099 INFO:tasks.rados.rados.0.smithi100.stdout:1247: done (1 left) 2024-04-07T16:20:49.099 INFO:tasks.rados.rados.0.smithi100.stdout:1249: done (0 left) 2024-04-07T16:20:49.099 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 19 current snap is 116 2024-04-07T16:20:49.099 INFO:tasks.rados.rados.0.smithi100.stdout:1250: seq_num 384 ranges {3186688=434176,3620864=475136,4096000=630784,4726784=98304} 2024-04-07T16:20:49.119 INFO:tasks.rados.rados.0.smithi100.stdout:1250: writing smithi10066867-19 from 3186688 to 3620864 tid 1 2024-04-07T16:20:49.123 INFO:tasks.rados.rados.0.smithi100.stdout:1250: writing smithi10066867-19 from 3620864 to 4096000 tid 2 2024-04-07T16:20:49.127 INFO:tasks.rados.rados.0.smithi100.stdout:1250: writing smithi10066867-19 from 4096000 to 4726784 tid 3 2024-04-07T16:20:49.127 INFO:tasks.rados.rados.0.smithi100.stdout:1250: writing smithi10066867-19 from 4726784 to 4825088 tid 4 2024-04-07T16:20:49.127 INFO:tasks.rados.rados.0.smithi100.stdout:1251: snap_create 2024-04-07T16:20:49.127 INFO:tasks.rados.rados.0.smithi100.stdout:1250: finishing write tid 1 to smithi10066867-19 2024-04-07T16:20:49.128 INFO:tasks.rados.rados.0.smithi100.stdout:1250: finishing write tid 2 to smithi10066867-19 2024-04-07T16:20:49.133 INFO:tasks.rados.rados.0.smithi100.stdout:1250: finishing write tid 3 to smithi10066867-19 2024-04-07T16:20:49.135 INFO:tasks.rados.rados.0.smithi100.stdout:1250: finishing write tid 4 to smithi10066867-19 2024-04-07T16:20:49.136 INFO:tasks.rados.rados.0.smithi100.stdout:1250: finishing write tid 5 to smithi10066867-19 2024-04-07T16:20:49.137 INFO:tasks.rados.rados.0.smithi100.stdout:1250: finishing write tid 6 to smithi10066867-19 2024-04-07T16:20:49.138 INFO:tasks.rados.rados.0.smithi100.stdout:1250: oid 19 updating version 0 to 555 2024-04-07T16:20:49.138 INFO:tasks.rados.rados.0.smithi100.stdout:1250: oid 19 version 555 is already newer than 553 2024-04-07T16:20:49.138 INFO:tasks.rados.rados.0.smithi100.stdout:1250: oid 19 version 555 is already newer than 554 2024-04-07T16:20:49.138 INFO:tasks.rados.rados.0.smithi100.stdout:1250: oid 19 version 555 is already newer than 552 2024-04-07T16:20:49.138 INFO:tasks.rados.rados.0.smithi100.stdout:1250: oid 19 version 555 is already newer than 551 2024-04-07T16:20:49.138 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 19 v 555 (ObjNum 384 snap 116 seq_num 384) dirty exists 2024-04-07T16:20:49.138 INFO:tasks.rados.rados.0.smithi100.stdout:1250: left oid 19 (ObjNum 384 snap 116 seq_num 384) 2024-04-07T16:20:49.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:49 smithi144 bash[23522]: cluster 2024-04-07T16:20:41.985431+0000 mon.c (mon.2) 1 : cluster 1 mon.c calling monitor election 2024-04-07T16:20:49.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:49 smithi144 bash[23522]: cluster 2024-04-07T16:20:48.023724+0000 mon.b (mon.1) 340 : cluster 1 mon.b calling monitor election 2024-04-07T16:20:49.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:49 smithi144 bash[23522]: cluster 2024-04-07T16:20:48.029971+0000 mon.a (mon.0) 120 : cluster 1 mon.a calling monitor election 2024-04-07T16:20:49.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:49 smithi144 bash[23522]: cluster 2024-04-07T16:20:48.057266+0000 mon.a (mon.0) 121 : cluster 1 mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-04-07T16:20:49.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:49 smithi144 bash[23522]: cluster 2024-04-07T16:20:48.067849+0000 mon.a (mon.0) 122 : cluster 0 monmap epoch 3 2024-04-07T16:20:49.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:49 smithi144 bash[23522]: cluster 2024-04-07T16:20:48.067944+0000 mon.a (mon.0) 123 : cluster 0 fsid 0702e3a8-f4f8-11ee-b649-cb9ed24678a4 2024-04-07T16:20:49.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:49 smithi144 bash[23522]: cluster 2024-04-07T16:20:48.068007+0000 mon.a (mon.0) 124 : cluster 0 last_changed 2024-04-07T16:04:34.801459+0000 2024-04-07T16:20:49.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:49 smithi144 bash[23522]: cluster 2024-04-07T16:20:48.068028+0000 mon.a (mon.0) 125 : cluster 0 created 2024-04-07T16:02:17.280630+0000 2024-04-07T16:20:49.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:49 smithi144 bash[23522]: cluster 2024-04-07T16:20:48.068044+0000 mon.a (mon.0) 126 : cluster 0 min_mon_release 18 (reef) 2024-04-07T16:20:49.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:49 smithi144 bash[23522]: cluster 2024-04-07T16:20:48.068060+0000 mon.a (mon.0) 127 : cluster 0 election_strategy: 1 2024-04-07T16:20:49.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:49 smithi144 bash[23522]: cluster 2024-04-07T16:20:48.068076+0000 mon.a (mon.0) 128 : cluster 0 0: [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] mon.a 2024-04-07T16:20:49.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:49 smithi144 bash[23522]: cluster 2024-04-07T16:20:48.068091+0000 mon.a (mon.0) 129 : cluster 0 1: [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] mon.b 2024-04-07T16:20:49.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:49 smithi144 bash[23522]: cluster 2024-04-07T16:20:48.068108+0000 mon.a (mon.0) 130 : cluster 0 2: [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] mon.c 2024-04-07T16:20:49.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:49 smithi144 bash[23522]: cluster 2024-04-07T16:20:48.069429+0000 mon.a (mon.0) 131 : cluster 0 fsmap cephfs:1 {0=b=up:active} 1 up:standby 2024-04-07T16:20:49.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:49 smithi144 bash[23522]: cluster 2024-04-07T16:20:48.069516+0000 mon.a (mon.0) 132 : cluster 0 osdmap e303: 8 total, 8 up, 8 in 2024-04-07T16:20:49.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:49 smithi144 bash[23522]: cluster 2024-04-07T16:20:48.070547+0000 mon.a (mon.0) 133 : cluster 0 mgrmap e41: y(active, since 31s), standbys: x 2024-04-07T16:20:49.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:49 smithi144 bash[23522]: cluster 2024-04-07T16:20:48.070931+0000 mon.a (mon.0) 134 : cluster 1 overall HEALTH_OK 2024-04-07T16:20:49.436 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:49 smithi100 bash[74942]: cluster 2024-04-07T16:20:41.985431+0000 mon.c (mon.2) 1 : cluster [INF] mon.c calling monitor election 2024-04-07T16:20:49.436 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:49 smithi100 bash[74942]: cluster 2024-04-07T16:20:48.023724+0000 mon.b (mon.1) 340 : cluster [INF] mon.b calling monitor election 2024-04-07T16:20:49.436 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:49 smithi100 bash[74942]: cluster 2024-04-07T16:20:48.029971+0000 mon.a (mon.0) 120 : cluster [INF] mon.a calling monitor election 2024-04-07T16:20:49.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:49 smithi100 bash[74942]: cluster 2024-04-07T16:20:48.057266+0000 mon.a (mon.0) 121 : cluster [INF] mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-04-07T16:20:49.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:49 smithi100 bash[74942]: cluster 2024-04-07T16:20:48.067849+0000 mon.a (mon.0) 122 : cluster [DBG] monmap epoch 3 2024-04-07T16:20:49.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:49 smithi100 bash[74942]: cluster 2024-04-07T16:20:48.067944+0000 mon.a (mon.0) 123 : cluster [DBG] fsid 0702e3a8-f4f8-11ee-b649-cb9ed24678a4 2024-04-07T16:20:49.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:49 smithi100 bash[74942]: cluster 2024-04-07T16:20:48.068007+0000 mon.a (mon.0) 124 : cluster [DBG] last_changed 2024-04-07T16:04:34.801459+0000 2024-04-07T16:20:49.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:49 smithi100 bash[74942]: cluster 2024-04-07T16:20:48.068028+0000 mon.a (mon.0) 125 : cluster [DBG] created 2024-04-07T16:02:17.280630+0000 2024-04-07T16:20:49.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:49 smithi100 bash[74942]: cluster 2024-04-07T16:20:48.068044+0000 mon.a (mon.0) 126 : cluster [DBG] min_mon_release 18 (reef) 2024-04-07T16:20:49.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:49 smithi100 bash[74942]: cluster 2024-04-07T16:20:48.068060+0000 mon.a (mon.0) 127 : cluster [DBG] election_strategy: 1 2024-04-07T16:20:49.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:49 smithi100 bash[74942]: cluster 2024-04-07T16:20:48.068076+0000 mon.a (mon.0) 128 : cluster [DBG] 0: [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] mon.a 2024-04-07T16:20:49.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:49 smithi100 bash[74942]: cluster 2024-04-07T16:20:48.068091+0000 mon.a (mon.0) 129 : cluster [DBG] 1: [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] mon.b 2024-04-07T16:20:49.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:49 smithi100 bash[74942]: cluster 2024-04-07T16:20:48.068108+0000 mon.a (mon.0) 130 : cluster [DBG] 2: [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] mon.c 2024-04-07T16:20:49.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:49 smithi100 bash[74942]: cluster 2024-04-07T16:20:48.069429+0000 mon.a (mon.0) 131 : cluster [DBG] fsmap cephfs:1 {0=b=up:active} 1 up:standby 2024-04-07T16:20:49.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:49 smithi100 bash[74942]: cluster 2024-04-07T16:20:48.069516+0000 mon.a (mon.0) 132 : cluster [DBG] osdmap e303: 8 total, 8 up, 8 in 2024-04-07T16:20:49.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:49 smithi100 bash[74942]: cluster 2024-04-07T16:20:48.070547+0000 mon.a (mon.0) 133 : cluster [DBG] mgrmap e41: y(active, since 31s), standbys: x 2024-04-07T16:20:49.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:49 smithi100 bash[74942]: cluster 2024-04-07T16:20:48.070931+0000 mon.a (mon.0) 134 : cluster [INF] overall HEALTH_OK 2024-04-07T16:20:49.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:49 smithi100 bash[76202]: cluster 2024-04-07T16:20:41.985431+0000 mon.c (mon.2) 1 : cluster [INF] mon.c calling monitor election 2024-04-07T16:20:49.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:49 smithi100 bash[76202]: cluster 2024-04-07T16:20:48.023724+0000 mon.b (mon.1) 340 : cluster [INF] mon.b calling monitor election 2024-04-07T16:20:49.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:49 smithi100 bash[76202]: cluster 2024-04-07T16:20:48.029971+0000 mon.a (mon.0) 120 : cluster [INF] mon.a calling monitor election 2024-04-07T16:20:49.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:49 smithi100 bash[76202]: cluster 2024-04-07T16:20:48.057266+0000 mon.a (mon.0) 121 : cluster [INF] mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-04-07T16:20:49.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:49 smithi100 bash[76202]: cluster 2024-04-07T16:20:48.067849+0000 mon.a (mon.0) 122 : cluster [DBG] monmap epoch 3 2024-04-07T16:20:49.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:49 smithi100 bash[76202]: cluster 2024-04-07T16:20:48.067944+0000 mon.a (mon.0) 123 : cluster [DBG] fsid 0702e3a8-f4f8-11ee-b649-cb9ed24678a4 2024-04-07T16:20:49.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:49 smithi100 bash[76202]: cluster 2024-04-07T16:20:48.068007+0000 mon.a (mon.0) 124 : cluster [DBG] last_changed 2024-04-07T16:04:34.801459+0000 2024-04-07T16:20:49.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:49 smithi100 bash[76202]: cluster 2024-04-07T16:20:48.068028+0000 mon.a (mon.0) 125 : cluster [DBG] created 2024-04-07T16:02:17.280630+0000 2024-04-07T16:20:49.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:49 smithi100 bash[76202]: cluster 2024-04-07T16:20:48.068044+0000 mon.a (mon.0) 126 : cluster [DBG] min_mon_release 18 (reef) 2024-04-07T16:20:49.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:49 smithi100 bash[76202]: cluster 2024-04-07T16:20:48.068060+0000 mon.a (mon.0) 127 : cluster [DBG] election_strategy: 1 2024-04-07T16:20:49.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:49 smithi100 bash[76202]: cluster 2024-04-07T16:20:48.068076+0000 mon.a (mon.0) 128 : cluster [DBG] 0: [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] mon.a 2024-04-07T16:20:49.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:49 smithi100 bash[76202]: cluster 2024-04-07T16:20:48.068091+0000 mon.a (mon.0) 129 : cluster [DBG] 1: [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] mon.b 2024-04-07T16:20:49.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:49 smithi100 bash[76202]: cluster 2024-04-07T16:20:48.068108+0000 mon.a (mon.0) 130 : cluster [DBG] 2: [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] mon.c 2024-04-07T16:20:49.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:49 smithi100 bash[76202]: cluster 2024-04-07T16:20:48.069429+0000 mon.a (mon.0) 131 : cluster [DBG] fsmap cephfs:1 {0=b=up:active} 1 up:standby 2024-04-07T16:20:49.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:49 smithi100 bash[76202]: cluster 2024-04-07T16:20:48.069516+0000 mon.a (mon.0) 132 : cluster [DBG] osdmap e303: 8 total, 8 up, 8 in 2024-04-07T16:20:49.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:49 smithi100 bash[76202]: cluster 2024-04-07T16:20:48.070547+0000 mon.a (mon.0) 133 : cluster [DBG] mgrmap e41: y(active, since 31s), standbys: x 2024-04-07T16:20:49.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:49 smithi100 bash[76202]: cluster 2024-04-07T16:20:48.070931+0000 mon.a (mon.0) 134 : cluster [INF] overall HEALTH_OK 2024-04-07T16:20:50.096 INFO:tasks.rados.rados.0.smithi100.stdout:1250: done (1 left) 2024-04-07T16:20:50.096 INFO:tasks.rados.rados.0.smithi100.stdout:1251: done (0 left) 2024-04-07T16:20:50.096 INFO:tasks.rados.rados.0.smithi100.stdout:1252: snap_create 2024-04-07T16:20:50.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:50 smithi144 bash[23522]: cluster 2024-04-07T16:20:48.609316+0000 mgr.y (mgr.25175) 34 : cluster 0 pgmap v42: 121 pgs: 121 active+clean; 191 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 103 B/s rd, 295 KiB/s wr, 0 op/s 2024-04-07T16:20:50.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:50 smithi144 bash[23522]: cluster 2024-04-07T16:20:49.100634+0000 mon.a (mon.0) 135 : cluster 0 osdmap e304: 8 total, 8 up, 8 in 2024-04-07T16:20:50.436 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:50 smithi100 bash[74942]: cluster 2024-04-07T16:20:48.609316+0000 mgr.y (mgr.25175) 34 : cluster [DBG] pgmap v42: 121 pgs: 121 active+clean; 191 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 103 B/s rd, 295 KiB/s wr, 0 op/s 2024-04-07T16:20:50.436 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:50 smithi100 bash[74942]: cluster 2024-04-07T16:20:49.100634+0000 mon.a (mon.0) 135 : cluster [DBG] osdmap e304: 8 total, 8 up, 8 in 2024-04-07T16:20:50.436 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:50 smithi100 bash[76202]: cluster 2024-04-07T16:20:48.609316+0000 mgr.y (mgr.25175) 34 : cluster [DBG] pgmap v42: 121 pgs: 121 active+clean; 191 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 103 B/s rd, 295 KiB/s wr, 0 op/s 2024-04-07T16:20:50.436 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:50 smithi100 bash[76202]: cluster 2024-04-07T16:20:49.100634+0000 mon.a (mon.0) 135 : cluster [DBG] osdmap e304: 8 total, 8 up, 8 in 2024-04-07T16:20:51.109 INFO:tasks.rados.rados.0.smithi100.stdout:1252: done (0 left) 2024-04-07T16:20:51.109 INFO:tasks.rados.rados.0.smithi100.stdout:1253: rollback oid 44 current snap is 118 2024-04-07T16:20:51.109 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 44 to 115 2024-04-07T16:20:51.109 INFO:tasks.rados.rados.0.smithi100.stdout:1254: read oid 36 snap -1 2024-04-07T16:20:51.109 INFO:tasks.rados.rados.0.smithi100.stdout:1254: expect (ObjNum 347 snap 104 seq_num 347) 2024-04-07T16:20:51.109 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 1 current snap is 118 2024-04-07T16:20:51.109 INFO:tasks.rados.rados.0.smithi100.stdout:1255: seq_num 385 ranges {1540096=450560,1990656=761856,2752512=442368} 2024-04-07T16:20:51.129 INFO:tasks.rados.rados.0.smithi100.stdout:1255: writing smithi10066867-1 from 1540096 to 1990656 tid 1 2024-04-07T16:20:51.136 INFO:tasks.rados.rados.0.smithi100.stdout:1255: writing smithi10066867-1 from 1990656 to 2752512 tid 2 2024-04-07T16:20:51.141 INFO:tasks.rados.rados.0.smithi100.stdout:1255: writing smithi10066867-1 from 2752512 to 3194880 tid 3 2024-04-07T16:20:51.141 INFO:tasks.rados.rados.0.smithi100.stdout:1256: delete oid 50 current snap is 118 2024-04-07T16:20:51.141 INFO:tasks.rados.rados.0.smithi100.stdout:1253: finishing rollback tid 0 to smithi10066867-44 2024-04-07T16:20:51.141 INFO:tasks.rados.rados.0.smithi100.stdout:1253: finishing rollback tid 1 to smithi10066867-44 2024-04-07T16:20:51.141 INFO:tasks.rados.rados.0.smithi100.stdout:1253: finishing rollback tid 2 to smithi10066867-44 2024-04-07T16:20:51.141 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 44 v 673 (ObjNum 171 snap 44 seq_num 171) dirty exists 2024-04-07T16:20:51.141 INFO:tasks.rados.rados.0.smithi100.stdout:1254: expect (ObjNum 347 snap 104 seq_num 347) 2024-04-07T16:20:51.211 INFO:tasks.rados.rados.0.smithi100.stdout:1255: finishing write tid 1 to smithi10066867-1 2024-04-07T16:20:51.211 INFO:tasks.rados.rados.0.smithi100.stdout:1255: finishing write tid 2 to smithi10066867-1 2024-04-07T16:20:51.211 INFO:tasks.rados.rados.0.smithi100.stdout:1255: finishing write tid 3 to smithi10066867-1 2024-04-07T16:20:51.211 INFO:tasks.rados.rados.0.smithi100.stdout:1255: finishing write tid 4 to smithi10066867-1 2024-04-07T16:20:51.211 INFO:tasks.rados.rados.0.smithi100.stdout:1255: finishing write tid 5 to smithi10066867-1 2024-04-07T16:20:51.211 INFO:tasks.rados.rados.0.smithi100.stdout:1255: oid 1 updating version 0 to 815 2024-04-07T16:20:51.211 INFO:tasks.rados.rados.0.smithi100.stdout:1255: oid 1 version 815 is already newer than 813 2024-04-07T16:20:51.211 INFO:tasks.rados.rados.0.smithi100.stdout:1255: oid 1 version 815 is already newer than 814 2024-04-07T16:20:51.211 INFO:tasks.rados.rados.0.smithi100.stdout:1255: oid 1 updating version 815 to 816 2024-04-07T16:20:51.211 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 1 v 816 (ObjNum 385 snap 118 seq_num 385) dirty exists 2024-04-07T16:20:51.211 INFO:tasks.rados.rados.0.smithi100.stdout:1255: left oid 1 (ObjNum 385 snap 118 seq_num 385) 2024-04-07T16:20:51.211 INFO:tasks.rados.rados.0.smithi100.stdout:1253: done (3 left) 2024-04-07T16:20:51.212 INFO:tasks.rados.rados.0.smithi100.stdout:1254: done (2 left) 2024-04-07T16:20:51.212 INFO:tasks.rados.rados.0.smithi100.stdout:1255: done (1 left) 2024-04-07T16:20:51.212 INFO:tasks.rados.rados.0.smithi100.stdout:1256: done (0 left) 2024-04-07T16:20:51.212 INFO:tasks.rados.rados.0.smithi100.stdout:1257: read oid 15 snap -1 2024-04-07T16:20:51.212 INFO:tasks.rados.rados.0.smithi100.stdout:1257: expect (ObjNum 338 snap 101 seq_num 338) 2024-04-07T16:20:51.212 INFO:tasks.rados.rados.0.smithi100.stdout:1258: snap_create 2024-04-07T16:20:51.245 INFO:tasks.rados.rados.0.smithi100.stdout:1257: expect (ObjNum 338 snap 101 seq_num 338) 2024-04-07T16:20:51.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:51 smithi144 bash[23522]: cluster 2024-04-07T16:20:50.112842+0000 mon.a (mon.0) 136 : cluster 0 osdmap e305: 8 total, 8 up, 8 in 2024-04-07T16:20:51.436 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:51 smithi100 bash[74942]: cluster 2024-04-07T16:20:50.112842+0000 mon.a (mon.0) 136 : cluster [DBG] osdmap e305: 8 total, 8 up, 8 in 2024-04-07T16:20:51.436 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:51 smithi100 bash[76202]: cluster 2024-04-07T16:20:50.112842+0000 mon.a (mon.0) 136 : cluster [DBG] osdmap e305: 8 total, 8 up, 8 in 2024-04-07T16:20:51.735 INFO:tasks.rados.rados.0.smithi100.stdout:1257: done (1 left) 2024-04-07T16:20:51.735 INFO:tasks.rados.rados.0.smithi100.stdout:1258: done (0 left) 2024-04-07T16:20:51.735 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 42 current snap is 119 2024-04-07T16:20:51.736 INFO:tasks.rados.rados.0.smithi100.stdout:1259: seq_num 387 ranges {0=483328,483328=524288,1007616=679936} 2024-04-07T16:20:51.743 INFO:tasks.rados.rados.0.smithi100.stdout:1259: writing smithi10066867-42 from 0 to 483328 tid 1 2024-04-07T16:20:51.752 INFO:tasks.rados.rados.0.smithi100.stdout:1259: writing smithi10066867-42 from 483328 to 1007616 tid 2 2024-04-07T16:20:51.762 INFO:tasks.rados.rados.0.smithi100.stdout:1259: writing smithi10066867-42 from 1007616 to 1687552 tid 3 2024-04-07T16:20:51.762 INFO:tasks.rados.rados.0.smithi100.stdout:1260: rmattr oid 36 current snap is 119 2024-04-07T16:20:51.762 INFO:tasks.rados.rados.0.smithi100.stdout:1260: done (1 left) 2024-04-07T16:20:51.762 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 32 current snap is 119 2024-04-07T16:20:51.762 INFO:tasks.rados.rados.0.smithi100.stdout:1259: finishing write tid 1 to smithi10066867-42 2024-04-07T16:20:51.762 INFO:tasks.rados.rados.0.smithi100.stdout:1259: finishing write tid 2 to smithi10066867-42 2024-04-07T16:20:51.762 INFO:tasks.rados.rados.0.smithi100.stdout:1261: seq_num 388 ranges {0=507904,507904=458752,966656=606208,1572864=139264} 2024-04-07T16:20:51.770 INFO:tasks.rados.rados.0.smithi100.stdout:1261: writing smithi10066867-32 from 0 to 507904 tid 1 2024-04-07T16:20:51.776 INFO:tasks.rados.rados.0.smithi100.stdout:1261: writing smithi10066867-32 from 507904 to 966656 tid 2 2024-04-07T16:20:51.785 INFO:tasks.rados.rados.0.smithi100.stdout:1261: writing smithi10066867-32 from 966656 to 1572864 tid 3 2024-04-07T16:20:51.787 INFO:tasks.rados.rados.0.smithi100.stdout:1261: writing smithi10066867-32 from 1572864 to 1712128 tid 4 2024-04-07T16:20:51.787 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 5 current snap is 119 2024-04-07T16:20:51.788 INFO:tasks.rados.rados.0.smithi100.stdout:1262: seq_num 389 ranges {2080768=524288,2605056=745472,3350528=458752} 2024-04-07T16:20:51.814 INFO:tasks.rados.rados.0.smithi100.stdout:1262: writing smithi10066867-5 from 2080768 to 2605056 tid 1 2024-04-07T16:20:51.821 INFO:tasks.rados.rados.0.smithi100.stdout:1262: writing smithi10066867-5 from 2605056 to 3350528 tid 2 2024-04-07T16:20:51.825 INFO:tasks.rados.rados.0.smithi100.stdout:1262: writing smithi10066867-5 from 3350528 to 3809280 tid 3 2024-04-07T16:20:51.825 INFO:tasks.rados.rados.0.smithi100.stdout:1263: snap_remove snap 115 2024-04-07T16:20:52.101 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:20:51 smithi100 bash[68137]: debug 2024-04-07T16:20:51.777+0000 7f8508b76700 -1 mgr.server handle_report got status from non-daemon mon.c 2024-04-07T16:20:52.356 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:20:52 smithi144 bash[59794]: ts=2024-04-07T16:20:52.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:20:52.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:52 smithi144 bash[23522]: cluster 2024-04-07T16:20:50.610252+0000 mgr.y (mgr.25175) 35 : cluster 0 pgmap v45: 121 pgs: 121 active+clean; 191 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail 2024-04-07T16:20:52.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:52 smithi144 bash[23522]: cluster 2024-04-07T16:20:51.122320+0000 mon.a (mon.0) 137 : cluster 0 osdmap e306: 8 total, 8 up, 8 in 2024-04-07T16:20:52.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:52 smithi144 bash[23522]: cluster 2024-04-07T16:20:51.751458+0000 mon.a (mon.0) 138 : cluster 0 osdmap e307: 8 total, 8 up, 8 in 2024-04-07T16:20:52.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:52 smithi100 bash[74942]: cluster 2024-04-07T16:20:50.610252+0000 mgr.y (mgr.25175) 35 : cluster [DBG] pgmap v45: 121 pgs: 121 active+clean; 191 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail 2024-04-07T16:20:52.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:52 smithi100 bash[74942]: cluster 2024-04-07T16:20:51.122320+0000 mon.a (mon.0) 137 : cluster [DBG] osdmap e306: 8 total, 8 up, 8 in 2024-04-07T16:20:52.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:52 smithi100 bash[74942]: cluster 2024-04-07T16:20:51.751458+0000 mon.a (mon.0) 138 : cluster [DBG] osdmap e307: 8 total, 8 up, 8 in 2024-04-07T16:20:52.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:52 smithi100 bash[76202]: cluster 2024-04-07T16:20:50.610252+0000 mgr.y (mgr.25175) 35 : cluster [DBG] pgmap v45: 121 pgs: 121 active+clean; 191 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail 2024-04-07T16:20:52.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:52 smithi100 bash[76202]: cluster 2024-04-07T16:20:51.122320+0000 mon.a (mon.0) 137 : cluster [DBG] osdmap e306: 8 total, 8 up, 8 in 2024-04-07T16:20:52.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:52 smithi100 bash[76202]: cluster 2024-04-07T16:20:51.751458+0000 mon.a (mon.0) 138 : cluster [DBG] osdmap e307: 8 total, 8 up, 8 in 2024-04-07T16:20:52.780 INFO:tasks.rados.rados.0.smithi100.stdout:1263: done (3 left) 2024-04-07T16:20:52.780 INFO:tasks.rados.rados.0.smithi100.stdout:1264: snap_remove snap 116 2024-04-07T16:20:53.773 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:20:53 smithi144 bash[59794]: ts=2024-04-07T16:20:53.507Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=nodes name=CephNodeDiskspaceWarning index=4 msg="Evaluating rule failed" rule="alert: CephNodeDiskspaceWarning\nexpr: predict_linear(node_filesystem_free_bytes{device=~\"/.*\"}[2d], 3600 * 24 * 5)\n * on (instance) group_left (nodename) node_uname_info < 0\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.8.4\n severity: warning\n type: ceph_default\nannotations:\n description: Mountpoint {{ $labels.mountpoint }} on {{ $labels.nodename }} will\n be full in less than 5 days based on the 48 hour trailing fill rate.\n summary: Host filesystem free space is getting low\n" err="found duplicate series for the match group {instance=\"smithi100\"} on the right hand-side of the operation: [{__name__=\"node_uname_info\", domainname=\"(none)\", instance=\"smithi100\", job=\"node\", machine=\"x86_64\", nodename=\"smithi100\", release=\"5.15.0-101-generic\", sysname=\"Linux\", version=\"#111-Ubuntu SMP Tue Mar 5 20:16:58 UTC 2024\"}, {__name__=\"node_uname_info\", cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", domainname=\"(none)\", instance=\"smithi100\", job=\"node\", machine=\"x86_64\", nodename=\"smithi100\", release=\"5.15.0-101-generic\", sysname=\"Linux\", version=\"#111-Ubuntu SMP Tue Mar 5 20:16:58 UTC 2024\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:20:53.787 INFO:tasks.rados.rados.0.smithi100.stdout:1264: done (3 left) 2024-04-07T16:20:53.787 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 6 current snap is 119 2024-04-07T16:20:53.787 INFO:tasks.rados.rados.0.smithi100.stdout:1259: finishing write tid 3 to smithi10066867-42 2024-04-07T16:20:53.787 INFO:tasks.rados.rados.0.smithi100.stdout:1265: seq_num 390 ranges {3661824=540672,4202496=630784,4833280=573440} 2024-04-07T16:20:53.813 INFO:tasks.rados.rados.0.smithi100.stdout:1265: writing smithi10066867-6 from 3661824 to 4202496 tid 1 2024-04-07T16:20:53.818 INFO:tasks.rados.rados.0.smithi100.stdout:1265: writing smithi10066867-6 from 4202496 to 4833280 tid 2 2024-04-07T16:20:53.823 INFO:tasks.rados.rados.0.smithi100.stdout:1265: writing smithi10066867-6 from 4833280 to 5406720 tid 3 2024-04-07T16:20:53.823 INFO:tasks.rados.rados.0.smithi100.stdout:1266: read oid 22 snap 109 2024-04-07T16:20:53.823 INFO:tasks.rados.rados.0.smithi100.stdout:1266: expect (ObjNum 294 snap 84 seq_num 294) 2024-04-07T16:20:53.823 INFO:tasks.rados.rados.0.smithi100.stdout:1267: rmattr oid 47 current snap is 119 2024-04-07T16:20:53.824 INFO:tasks.rados.rados.0.smithi100.stdout:1267: done (5 left) 2024-04-07T16:20:53.824 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 29 current snap is 119 2024-04-07T16:20:53.824 INFO:tasks.rados.rados.0.smithi100.stdout:1259: finishing write tid 4 to smithi10066867-42 2024-04-07T16:20:53.824 INFO:tasks.rados.rados.0.smithi100.stdout:1259: finishing write tid 5 to smithi10066867-42 2024-04-07T16:20:53.824 INFO:tasks.rados.rados.0.smithi100.stdout:1259: oid 42 updating version 0 to 516 2024-04-07T16:20:53.824 INFO:tasks.rados.rados.0.smithi100.stdout:1259: oid 42 updating version 516 to 518 2024-04-07T16:20:53.824 INFO:tasks.rados.rados.0.smithi100.stdout:1259: oid 42 version 518 is already newer than 517 2024-04-07T16:20:53.824 INFO:tasks.rados.rados.0.smithi100.stdout:1259: oid 42 version 518 is already newer than 515 2024-04-07T16:20:53.824 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 42 v 518 (ObjNum 387 snap 119 seq_num 387) dirty exists 2024-04-07T16:20:53.824 INFO:tasks.rados.rados.0.smithi100.stdout:1259: left oid 42 (ObjNum 387 snap 119 seq_num 387) 2024-04-07T16:20:53.824 INFO:tasks.rados.rados.0.smithi100.stdout:1261: finishing write tid 1 to smithi10066867-32 2024-04-07T16:20:53.824 INFO:tasks.rados.rados.0.smithi100.stdout:1261: finishing write tid 2 to smithi10066867-32 2024-04-07T16:20:53.824 INFO:tasks.rados.rados.0.smithi100.stdout:1261: finishing write tid 3 to smithi10066867-32 2024-04-07T16:20:53.824 INFO:tasks.rados.rados.0.smithi100.stdout:1261: finishing write tid 4 to smithi10066867-32 2024-04-07T16:20:53.825 INFO:tasks.rados.rados.0.smithi100.stdout:1261: finishing write tid 5 to smithi10066867-32 2024-04-07T16:20:53.825 INFO:tasks.rados.rados.0.smithi100.stdout:1261: finishing write tid 6 to smithi10066867-32 2024-04-07T16:20:53.825 INFO:tasks.rados.rados.0.smithi100.stdout:1261: oid 32 updating version 0 to 199 2024-04-07T16:20:53.825 INFO:tasks.rados.rados.0.smithi100.stdout:1261: oid 32 version 199 is already newer than 198 2024-04-07T16:20:53.825 INFO:tasks.rados.rados.0.smithi100.stdout:1261: oid 32 version 199 is already newer than 196 2024-04-07T16:20:53.825 INFO:tasks.rados.rados.0.smithi100.stdout:1261: oid 32 version 199 is already newer than 197 2024-04-07T16:20:53.825 INFO:tasks.rados.rados.0.smithi100.stdout:1261: oid 32 version 199 is already newer than 195 2024-04-07T16:20:53.825 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 32 v 199 (ObjNum 388 snap 119 seq_num 388) dirty exists 2024-04-07T16:20:53.825 INFO:tasks.rados.rados.0.smithi100.stdout:1261: left oid 32 (ObjNum 388 snap 119 seq_num 388) 2024-04-07T16:20:53.825 INFO:tasks.rados.rados.0.smithi100.stdout:1262: finishing write tid 1 to smithi10066867-5 2024-04-07T16:20:53.825 INFO:tasks.rados.rados.0.smithi100.stdout:1262: finishing write tid 2 to smithi10066867-5 2024-04-07T16:20:53.825 INFO:tasks.rados.rados.0.smithi100.stdout:1262: finishing write tid 3 to smithi10066867-5 2024-04-07T16:20:53.825 INFO:tasks.rados.rados.0.smithi100.stdout:1262: finishing write tid 4 to smithi10066867-5 2024-04-07T16:20:53.825 INFO:tasks.rados.rados.0.smithi100.stdout:1262: finishing write tid 5 to smithi10066867-5 2024-04-07T16:20:53.825 INFO:tasks.rados.rados.0.smithi100.stdout:1262: oid 5 updating version 0 to 693 2024-04-07T16:20:53.825 INFO:tasks.rados.rados.0.smithi100.stdout:1262: oid 5 updating version 693 to 694 2024-04-07T16:20:53.826 INFO:tasks.rados.rados.0.smithi100.stdout:1262: oid 5 updating version 694 to 695 2024-04-07T16:20:53.826 INFO:tasks.rados.rados.0.smithi100.stdout:1262: oid 5 version 695 is already newer than 692 2024-04-07T16:20:53.826 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 5 v 695 (ObjNum 389 snap 119 seq_num 389) dirty exists 2024-04-07T16:20:53.826 INFO:tasks.rados.rados.0.smithi100.stdout:1262: left oid 5 (ObjNum 389 snap 119 seq_num 389) 2024-04-07T16:20:53.826 INFO:tasks.rados.rados.0.smithi100.stdout:1268: seq_num 391 ranges {0=557056,557056=516096,1073152=688128} 2024-04-07T16:20:53.827 INFO:tasks.rados.rados.0.smithi100.stdout:1268: writing smithi10066867-29 from 0 to 557056 tid 1 2024-04-07T16:20:53.831 INFO:tasks.rados.rados.0.smithi100.stdout:1268: writing smithi10066867-29 from 557056 to 1073152 tid 2 2024-04-07T16:20:53.837 INFO:tasks.rados.rados.0.smithi100.stdout:1268: writing smithi10066867-29 from 1073152 to 1761280 tid 3 2024-04-07T16:20:53.837 INFO:tasks.rados.rados.0.smithi100.stdout:1259: done (5 left) 2024-04-07T16:20:53.837 INFO:tasks.rados.rados.0.smithi100.stdout:1261: done (4 left) 2024-04-07T16:20:53.837 INFO:tasks.rados.rados.0.smithi100.stdout:1262: done (3 left) 2024-04-07T16:20:53.837 INFO:tasks.rados.rados.0.smithi100.stdout:1269: read oid 25 snap -1 2024-04-07T16:20:53.837 INFO:tasks.rados.rados.0.smithi100.stdout:1269: expect deleted 2024-04-07T16:20:53.838 INFO:tasks.rados.rados.0.smithi100.stdout:1270: delete oid 27 current snap is 119 2024-04-07T16:20:53.838 INFO:tasks.rados.rados.0.smithi100.stdout:1265: finishing write tid 1 to smithi10066867-6 2024-04-07T16:20:53.838 INFO:tasks.rados.rados.0.smithi100.stdout:1265: finishing write tid 2 to smithi10066867-6 2024-04-07T16:20:53.842 INFO:tasks.rados.rados.0.smithi100.stdout:1270: done (4 left) 2024-04-07T16:20:53.843 INFO:tasks.rados.rados.0.smithi100.stdout:1271: rollback oid 40 current snap is 119 2024-04-07T16:20:53.843 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 40 to 103 2024-04-07T16:20:53.843 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 13 current snap is 119 2024-04-07T16:20:53.843 INFO:tasks.rados.rados.0.smithi100.stdout:1272: seq_num 393 ranges {8658944=589824,9248768=679936,9928704=524288} 2024-04-07T16:20:53.896 INFO:tasks.rados.rados.0.smithi100.stdout:1272: writing smithi10066867-13 from 8658944 to 9248768 tid 1 2024-04-07T16:20:53.900 INFO:tasks.rados.rados.0.smithi100.stdout:1272: writing smithi10066867-13 from 9248768 to 9928704 tid 2 2024-04-07T16:20:53.903 INFO:tasks.rados.rados.0.smithi100.stdout:1272: writing smithi10066867-13 from 9928704 to 10452992 tid 3 2024-04-07T16:20:53.903 INFO:tasks.rados.rados.0.smithi100.stdout:1273: read oid 49 snap -1 2024-04-07T16:20:53.904 INFO:tasks.rados.rados.0.smithi100.stdout:1273: expect (ObjNum 367 snap 110 seq_num 367) 2024-04-07T16:20:53.904 INFO:tasks.rados.rados.0.smithi100.stdout:1274: read oid 44 snap 103 2024-04-07T16:20:53.904 INFO:tasks.rados.rados.0.smithi100.stdout:1274: expect (ObjNum 171 snap 44 seq_num 171) 2024-04-07T16:20:53.904 INFO:tasks.rados.rados.0.smithi100.stdout:1275: delete oid 3 current snap is 119 2024-04-07T16:20:53.904 INFO:tasks.rados.rados.0.smithi100.stdout:1268: finishing write tid 1 to smithi10066867-29 2024-04-07T16:20:53.904 INFO:tasks.rados.rados.0.smithi100.stdout:1265: finishing write tid 3 to smithi10066867-6 2024-04-07T16:20:53.904 INFO:tasks.rados.rados.0.smithi100.stdout:1265: finishing write tid 4 to smithi10066867-6 2024-04-07T16:20:53.904 INFO:tasks.rados.rados.0.smithi100.stdout:1268: finishing write tid 2 to smithi10066867-29 2024-04-07T16:20:53.904 INFO:tasks.rados.rados.0.smithi100.stdout:1268: finishing write tid 3 to smithi10066867-29 2024-04-07T16:20:53.904 INFO:tasks.rados.rados.0.smithi100.stdout:1268: finishing write tid 4 to smithi10066867-29 2024-04-07T16:20:53.904 INFO:tasks.rados.rados.0.smithi100.stdout:1266: expect (ObjNum 294 snap 84 seq_num 294) 2024-04-07T16:20:53.995 INFO:tasks.rados.rados.0.smithi100.stdout:1268: finishing write tid 5 to smithi10066867-29 2024-04-07T16:20:53.995 INFO:tasks.rados.rados.0.smithi100.stdout:1268: oid 29 updating version 0 to 523 2024-04-07T16:20:53.995 INFO:tasks.rados.rados.0.smithi100.stdout:1268: oid 29 version 523 is already newer than 521 2024-04-07T16:20:53.995 INFO:tasks.rados.rados.0.smithi100.stdout:1268: oid 29 updating version 523 to 524 2024-04-07T16:20:53.995 INFO:tasks.rados.rados.0.smithi100.stdout:1268: oid 29 version 524 is already newer than 522 2024-04-07T16:20:53.995 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 29 v 524 (ObjNum 391 snap 119 seq_num 391) dirty exists 2024-04-07T16:20:53.995 INFO:tasks.rados.rados.0.smithi100.stdout:1268: left oid 29 (ObjNum 391 snap 119 seq_num 391) 2024-04-07T16:20:53.995 INFO:tasks.rados.rados.0.smithi100.stdout:1265: finishing write tid 5 to smithi10066867-6 2024-04-07T16:20:53.995 INFO:tasks.rados.rados.0.smithi100.stdout:1265: oid 6 updating version 0 to 676 2024-04-07T16:20:53.995 INFO:tasks.rados.rados.0.smithi100.stdout:1265: oid 6 version 676 is already newer than 674 2024-04-07T16:20:53.996 INFO:tasks.rados.rados.0.smithi100.stdout:1265: oid 6 version 676 is already newer than 675 2024-04-07T16:20:53.996 INFO:tasks.rados.rados.0.smithi100.stdout:1265: oid 6 updating version 676 to 677 2024-04-07T16:20:53.996 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 6 v 677 (ObjNum 390 snap 119 seq_num 390) dirty exists 2024-04-07T16:20:53.996 INFO:tasks.rados.rados.0.smithi100.stdout:1265: left oid 6 (ObjNum 390 snap 119 seq_num 390) 2024-04-07T16:20:53.996 INFO:tasks.rados.rados.0.smithi100.stdout:1271: finishing rollback tid 0 to smithi10066867-40 2024-04-07T16:20:53.996 INFO:tasks.rados.rados.0.smithi100.stdout:1271: finishing rollback tid 1 to smithi10066867-40 2024-04-07T16:20:53.996 INFO:tasks.rados.rados.0.smithi100.stdout:1271: finishing rollback tid 2 to smithi10066867-40 2024-04-07T16:20:53.996 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 40 v 680 (ObjNum 303 snap 87 seq_num 303) dirty exists 2024-04-07T16:20:53.996 INFO:tasks.rados.rados.0.smithi100.stdout:1265: done (8 left) 2024-04-07T16:20:53.996 INFO:tasks.rados.rados.0.smithi100.stdout:1266: done (7 left) 2024-04-07T16:20:53.996 INFO:tasks.rados.rados.0.smithi100.stdout:1268: done (6 left) 2024-04-07T16:20:53.996 INFO:tasks.rados.rados.0.smithi100.stdout:1269: done (5 left) 2024-04-07T16:20:53.996 INFO:tasks.rados.rados.0.smithi100.stdout:1271: done (4 left) 2024-04-07T16:20:53.996 INFO:tasks.rados.rados.0.smithi100.stdout:1275: done (3 left) 2024-04-07T16:20:53.996 INFO:tasks.rados.rados.0.smithi100.stdout:1276: read oid 50 snap -1 2024-04-07T16:20:53.997 INFO:tasks.rados.rados.0.smithi100.stdout:1276: expect deleted 2024-04-07T16:20:53.997 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 17 current snap is 119 2024-04-07T16:20:53.997 INFO:tasks.rados.rados.0.smithi100.stdout:1272: finishing write tid 1 to smithi10066867-13 2024-04-07T16:20:53.997 INFO:tasks.rados.rados.0.smithi100.stdout:1277: seq_num 395 ranges {6135808=622592,6758400=450560,7208960=753664} 2024-04-07T16:20:54.032 INFO:tasks.rados.rados.0.smithi100.stdout:1277: writing smithi10066867-17 from 6135808 to 6758400 tid 1 2024-04-07T16:20:54.035 INFO:tasks.rados.rados.0.smithi100.stdout:1277: writing smithi10066867-17 from 6758400 to 7208960 tid 2 2024-04-07T16:20:54.040 INFO:tasks.rados.rados.0.smithi100.stdout:1277: writing smithi10066867-17 from 7208960 to 7962624 tid 3 2024-04-07T16:20:54.040 INFO:tasks.rados.rados.0.smithi100.stdout:1278: rollback oid 12 current snap is 119 2024-04-07T16:20:54.040 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 12 to 109 2024-04-07T16:20:54.040 INFO:tasks.rados.rados.0.smithi100.stdout:1272: finishing write tid 2 to smithi10066867-13 2024-04-07T16:20:54.040 INFO:tasks.rados.rados.0.smithi100.stdout:1274: expect (ObjNum 171 snap 44 seq_num 171) 2024-04-07T16:20:54.062 INFO:tasks.rados.rados.0.smithi100.stdout:1272: finishing write tid 3 to smithi10066867-13 2024-04-07T16:20:54.062 INFO:tasks.rados.rados.0.smithi100.stdout:1272: finishing write tid 4 to smithi10066867-13 2024-04-07T16:20:54.063 INFO:tasks.rados.rados.0.smithi100.stdout:1272: finishing write tid 5 to smithi10066867-13 2024-04-07T16:20:54.063 INFO:tasks.rados.rados.0.smithi100.stdout:1272: oid 13 updating version 0 to 558 2024-04-07T16:20:54.063 INFO:tasks.rados.rados.0.smithi100.stdout:1272: oid 13 updating version 558 to 559 2024-04-07T16:20:54.063 INFO:tasks.rados.rados.0.smithi100.stdout:1272: oid 13 updating version 559 to 560 2024-04-07T16:20:54.063 INFO:tasks.rados.rados.0.smithi100.stdout:1272: oid 13 updating version 560 to 561 2024-04-07T16:20:54.063 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 13 v 561 (ObjNum 393 snap 119 seq_num 393) dirty exists 2024-04-07T16:20:54.063 INFO:tasks.rados.rados.0.smithi100.stdout:1272: left oid 13 (ObjNum 393 snap 119 seq_num 393) 2024-04-07T16:20:54.063 INFO:tasks.rados.rados.0.smithi100.stdout:1273: expect (ObjNum 367 snap 110 seq_num 367) 2024-04-07T16:20:54.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:53 smithi144 bash[23522]: cluster 2024-04-07T16:20:52.611221+0000 mgr.y (mgr.25175) 36 : cluster 0 pgmap v48: 121 pgs: 121 active+clean; 204 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 5.0 MiB/s rd, 3.6 MiB/s wr, 15 op/s 2024-04-07T16:20:54.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:53 smithi144 bash[23522]: cluster 2024-04-07T16:20:52.791831+0000 mon.a (mon.0) 139 : cluster 0 osdmap e308: 8 total, 8 up, 8 in 2024-04-07T16:20:54.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:53 smithi144 bash[23522]: audit 2024-04-07T16:20:52.894928+0000 mon.a (mon.0) 140 : audit 1 from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:20:54.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:53 smithi144 bash[23522]: audit 2024-04-07T16:20:52.901479+0000 mon.a (mon.0) 141 : audit 1 from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:20:54.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:53 smithi100 bash[74942]: cluster 2024-04-07T16:20:52.611221+0000 mgr.y (mgr.25175) 36 : cluster [DBG] pgmap v48: 121 pgs: 121 active+clean; 204 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 5.0 MiB/s rd, 3.6 MiB/s wr, 15 op/s 2024-04-07T16:20:54.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:53 smithi100 bash[74942]: cluster 2024-04-07T16:20:52.791831+0000 mon.a (mon.0) 139 : cluster [DBG] osdmap e308: 8 total, 8 up, 8 in 2024-04-07T16:20:54.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:53 smithi100 bash[74942]: audit 2024-04-07T16:20:52.894928+0000 mon.a (mon.0) 140 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:20:54.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:53 smithi100 bash[74942]: audit 2024-04-07T16:20:52.901479+0000 mon.a (mon.0) 141 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:20:54.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:53 smithi100 bash[76202]: cluster 2024-04-07T16:20:52.611221+0000 mgr.y (mgr.25175) 36 : cluster [DBG] pgmap v48: 121 pgs: 121 active+clean; 204 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 5.0 MiB/s rd, 3.6 MiB/s wr, 15 op/s 2024-04-07T16:20:54.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:53 smithi100 bash[76202]: cluster 2024-04-07T16:20:52.791831+0000 mon.a (mon.0) 139 : cluster [DBG] osdmap e308: 8 total, 8 up, 8 in 2024-04-07T16:20:54.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:53 smithi100 bash[76202]: audit 2024-04-07T16:20:52.894928+0000 mon.a (mon.0) 140 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:20:54.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:53 smithi100 bash[76202]: audit 2024-04-07T16:20:52.901479+0000 mon.a (mon.0) 141 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:20:54.228 INFO:tasks.rados.rados.0.smithi100.stdout:1277: finishing write tid 1 to smithi10066867-17 2024-04-07T16:20:54.228 INFO:tasks.rados.rados.0.smithi100.stdout:1277: finishing write tid 2 to smithi10066867-17 2024-04-07T16:20:54.228 INFO:tasks.rados.rados.0.smithi100.stdout:1278: finishing rollback tid 1 to smithi10066867-12 2024-04-07T16:20:54.228 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 12 v 611 (ObjNum 609 snap 0 seq_num 0) dirty dne 2024-04-07T16:20:54.229 INFO:tasks.rados.rados.0.smithi100.stdout:1277: finishing write tid 3 to smithi10066867-17 2024-04-07T16:20:54.229 INFO:tasks.rados.rados.0.smithi100.stdout:1277: finishing write tid 4 to smithi10066867-17 2024-04-07T16:20:54.229 INFO:tasks.rados.rados.0.smithi100.stdout:1277: finishing write tid 5 to smithi10066867-17 2024-04-07T16:20:54.229 INFO:tasks.rados.rados.0.smithi100.stdout:1277: oid 17 updating version 0 to 559 2024-04-07T16:20:54.229 INFO:tasks.rados.rados.0.smithi100.stdout:1277: oid 17 version 559 is already newer than 558 2024-04-07T16:20:54.229 INFO:tasks.rados.rados.0.smithi100.stdout:1277: oid 17 version 559 is already newer than 557 2024-04-07T16:20:54.229 INFO:tasks.rados.rados.0.smithi100.stdout:1277: oid 17 version 559 is already newer than 556 2024-04-07T16:20:54.229 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 17 v 559 (ObjNum 395 snap 119 seq_num 395) dirty exists 2024-04-07T16:20:54.229 INFO:tasks.rados.rados.0.smithi100.stdout:1277: left oid 17 (ObjNum 395 snap 119 seq_num 395) 2024-04-07T16:20:54.229 INFO:tasks.rados.rados.0.smithi100.stdout:1272: done (5 left) 2024-04-07T16:20:54.229 INFO:tasks.rados.rados.0.smithi100.stdout:1273: done (4 left) 2024-04-07T16:20:54.229 INFO:tasks.rados.rados.0.smithi100.stdout:1274: done (3 left) 2024-04-07T16:20:54.229 INFO:tasks.rados.rados.0.smithi100.stdout:1276: done (2 left) 2024-04-07T16:20:54.229 INFO:tasks.rados.rados.0.smithi100.stdout:1277: done (1 left) 2024-04-07T16:20:54.230 INFO:tasks.rados.rados.0.smithi100.stdout:1278: done (0 left) 2024-04-07T16:20:54.230 INFO:tasks.rados.rados.0.smithi100.stdout:1279: copy_from oid 19 from oid 16 current snap is 119 2024-04-07T16:20:54.230 INFO:tasks.rados.rados.0.smithi100.stdout:1280: snap_remove snap 117 2024-04-07T16:20:55.082 INFO:tasks.rados.rados.0.smithi100.stdout:1280: done (1 left) 2024-04-07T16:20:55.082 INFO:tasks.rados.rados.0.smithi100.stdout:1281: copy_from oid 5 from oid 11 current snap is 119 2024-04-07T16:20:55.082 INFO:tasks.rados.rados.0.smithi100.stdout:1279: finishing copy_from racing read to smithi10066867-19 2024-04-07T16:20:55.082 INFO:tasks.rados.rados.0.smithi100.stdout:1279: finishing copy_from to smithi10066867-19 2024-04-07T16:20:55.082 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 19 v 678 (ObjNum 374 snap 113 seq_num 374) dirty exists 2024-04-07T16:20:55.082 INFO:tasks.rados.rados.0.smithi100.stdout:1279: done (1 left) 2024-04-07T16:20:55.082 INFO:tasks.rados.rados.0.smithi100.stdout:1282: snap_create 2024-04-07T16:20:55.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:54 smithi144 bash[23522]: cluster 2024-04-07T16:20:53.801948+0000 mon.a (mon.0) 142 : cluster 0 osdmap e309: 8 total, 8 up, 8 in 2024-04-07T16:20:55.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:54 smithi144 bash[23522]: audit 2024-04-07T16:20:54.078687+0000 mon.a (mon.0) 143 : audit 1 from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:20:55.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:54 smithi144 bash[23522]: audit 2024-04-07T16:20:54.085307+0000 mon.a (mon.0) 144 : audit 1 from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:20:55.124 INFO:tasks.rados.rados.0.smithi100.stdout:1281: finishing copy_from racing read to smithi10066867-5 2024-04-07T16:20:55.126 INFO:tasks.rados.rados.0.smithi100.stdout:1281: finishing copy_from to smithi10066867-5 2024-04-07T16:20:55.126 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 5 v 639 (ObjNum 327 snap 99 seq_num 327) dirty exists 2024-04-07T16:20:55.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:54 smithi100 bash[76202]: cluster 2024-04-07T16:20:53.801948+0000 mon.a (mon.0) 142 : cluster [DBG] osdmap e309: 8 total, 8 up, 8 in 2024-04-07T16:20:55.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:54 smithi100 bash[76202]: audit 2024-04-07T16:20:54.078687+0000 mon.a (mon.0) 143 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:20:55.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:54 smithi100 bash[76202]: audit 2024-04-07T16:20:54.085307+0000 mon.a (mon.0) 144 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:20:55.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:54 smithi100 bash[74942]: cluster 2024-04-07T16:20:53.801948+0000 mon.a (mon.0) 142 : cluster [DBG] osdmap e309: 8 total, 8 up, 8 in 2024-04-07T16:20:55.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:54 smithi100 bash[74942]: audit 2024-04-07T16:20:54.078687+0000 mon.a (mon.0) 143 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:20:55.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:54 smithi100 bash[74942]: audit 2024-04-07T16:20:54.085307+0000 mon.a (mon.0) 144 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:20:55.466 INFO:teuthology.orchestra.run.smithi100.stdout:true 2024-04-07T16:20:55.858 INFO:teuthology.orchestra.run.smithi100.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-07T16:20:55.859 INFO:teuthology.orchestra.run.smithi100.stdout:alertmanager.a smithi100 *:9093,9094 running (6m) 2s ago 7m 14.0M - 0.25.0 c8568f914cd2 5143075e7d26 2024-04-07T16:20:55.859 INFO:teuthology.orchestra.run.smithi100.stdout:grafana.a smithi144 *:3000 running (7m) 32s ago 7m 79.4M - 9.4.7 954c08fa6188 7180c0708acd 2024-04-07T16:20:55.859 INFO:teuthology.orchestra.run.smithi100.stdout:mds.a smithi100 running (8m) 2s ago 8m 15.0M - 18.2.2-1171-g267c8ff5 8a4a01e7186c 92593b0c28ae 2024-04-07T16:20:55.859 INFO:teuthology.orchestra.run.smithi100.stdout:mds.b smithi144 running (8m) 32s ago 8m 17.8M - 18.2.2-1171-g267c8ff5 8a4a01e7186c 3df2ee3e17a4 2024-04-07T16:20:55.859 INFO:teuthology.orchestra.run.smithi100.stdout:mgr.x smithi144 *:8443,9283,8765 running (73s) 32s ago 15m 436M - 19.0.0-2785-gadce92d0 c0ecd4226ac9 233745c292a6 2024-04-07T16:20:55.859 INFO:teuthology.orchestra.run.smithi100.stdout:mgr.y smithi100 *:8443,9283,8765 running (118s) 2s ago 18m 495M - 19.0.0-2785-gadce92d0 c0ecd4226ac9 254dc65cdaa7 2024-04-07T16:20:55.859 INFO:teuthology.orchestra.run.smithi100.stdout:mon.a smithi100 running (50s) 2s ago 18m 38.4M 2048M 19.0.0-2785-gadce92d0 c0ecd4226ac9 718e12655c04 2024-04-07T16:20:55.859 INFO:teuthology.orchestra.run.smithi100.stdout:mon.b smithi144 running (16m) 32s ago 16m 52.2M 2048M 18.2.2-1171-g267c8ff5 8a4a01e7186c 8dd687691b2e 2024-04-07T16:20:55.859 INFO:teuthology.orchestra.run.smithi100.stdout:mon.c smithi100 running (15s) 2s ago 16m 20.1M 2048M 19.0.0-2785-gadce92d0 c0ecd4226ac9 63e889040f6e 2024-04-07T16:20:55.860 INFO:teuthology.orchestra.run.smithi100.stdout:node-exporter.a smithi100 *:9100 running (7m) 2s ago 7m 8248k - 1.5.0 0da6a335fe13 4bce264c1b26 2024-04-07T16:20:55.860 INFO:teuthology.orchestra.run.smithi100.stdout:node-exporter.b smithi144 *:9100 running (7m) 32s ago 7m 8328k - 1.5.0 0da6a335fe13 b6963413c560 2024-04-07T16:20:55.860 INFO:teuthology.orchestra.run.smithi100.stdout:osd.0 smithi100 running (15m) 2s ago 15m 240M 2524M 18.2.2-1171-g267c8ff5 8a4a01e7186c 188deb247979 2024-04-07T16:20:55.860 INFO:teuthology.orchestra.run.smithi100.stdout:osd.1 smithi100 running (14m) 2s ago 14m 196M 2524M 18.2.2-1171-g267c8ff5 8a4a01e7186c 7f57ca90894c 2024-04-07T16:20:55.860 INFO:teuthology.orchestra.run.smithi100.stdout:osd.2 smithi100 running (13m) 2s ago 13m 232M 2524M 18.2.2-1171-g267c8ff5 8a4a01e7186c f3912ddcc8cc 2024-04-07T16:20:55.860 INFO:teuthology.orchestra.run.smithi100.stdout:osd.3 smithi100 running (12m) 2s ago 12m 169M 2524M 18.2.2-1171-g267c8ff5 8a4a01e7186c b41004090c63 2024-04-07T16:20:55.860 INFO:teuthology.orchestra.run.smithi100.stdout:osd.4 smithi144 running (12m) 32s ago 12m 227M 2524M 18.2.2-1171-g267c8ff5 8a4a01e7186c 8c73b6b4dc13 2024-04-07T16:20:55.860 INFO:teuthology.orchestra.run.smithi100.stdout:osd.5 smithi144 running (11m) 32s ago 11m 191M 2524M 18.2.2-1171-g267c8ff5 8a4a01e7186c 3bfa0cf6f6cb 2024-04-07T16:20:55.860 INFO:teuthology.orchestra.run.smithi100.stdout:osd.6 smithi144 running (10m) 32s ago 10m 88.9M 2524M 18.2.2-1171-g267c8ff5 8a4a01e7186c 32b7f95d77f0 2024-04-07T16:20:55.860 INFO:teuthology.orchestra.run.smithi100.stdout:osd.7 smithi144 running (9m) 32s ago 9m 123M 2524M 18.2.2-1171-g267c8ff5 8a4a01e7186c 45f7709e576a 2024-04-07T16:20:55.860 INFO:teuthology.orchestra.run.smithi100.stdout:prometheus.a smithi144 *:9095 running (78s) 32s ago 8m 42.6M - 2.43.0 a07b618ecd1d 2d0a8131f303 2024-04-07T16:20:56.085 INFO:tasks.rados.rados.0.smithi100.stdout:1281: done (1 left) 2024-04-07T16:20:56.086 INFO:tasks.rados.rados.0.smithi100.stdout:1282: done (0 left) 2024-04-07T16:20:56.086 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 11 current snap is 120 2024-04-07T16:20:56.086 INFO:tasks.rados.rados.0.smithi100.stdout:1283: seq_num 396 ranges {3694592=638976,4333568=786432,5120000=417792} 2024-04-07T16:20:56.146 INFO:tasks.rados.rados.0.smithi100.stdout:1283: writing smithi10066867-11 from 3694592 to 4333568 tid 1 2024-04-07T16:20:56.154 INFO:tasks.rados.rados.0.smithi100.stdout:1283: writing smithi10066867-11 from 4333568 to 5120000 tid 2 2024-04-07T16:20:56.158 INFO:tasks.rados.rados.0.smithi100.stdout:1283: writing smithi10066867-11 from 5120000 to 5537792 tid 3 2024-04-07T16:20:56.158 INFO:tasks.rados.rados.0.smithi100.stdout:1284: rmattr oid 30 current snap is 120 2024-04-07T16:20:56.158 INFO:tasks.rados.rados.0.smithi100.stdout:1283: finishing write tid 1 to smithi10066867-11 2024-04-07T16:20:56.158 INFO:tasks.rados.rados.0.smithi100.stdout:1285: delete oid 23 current snap is 120 2024-04-07T16:20:56.163 INFO:tasks.rados.rados.0.smithi100.stdout:1283: finishing write tid 2 to smithi10066867-11 2024-04-07T16:20:56.164 INFO:tasks.rados.rados.0.smithi100.stdout:1283: finishing write tid 3 to smithi10066867-11 2024-04-07T16:20:56.166 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 30 v 526 (ObjNum 382 snap 116 seq_num 382) dirty exists 2024-04-07T16:20:56.166 INFO:tasks.rados.rados.0.smithi100.stdout:1283: finishing write tid 4 to smithi10066867-11 2024-04-07T16:20:56.168 INFO:tasks.rados.rados.0.smithi100.stdout:1284: done (2 left) 2024-04-07T16:20:56.168 INFO:tasks.rados.rados.0.smithi100.stdout:1285: done (1 left) 2024-04-07T16:20:56.168 INFO:tasks.rados.rados.0.smithi100.stdout:1286: rollback oid 30 current snap is 120 2024-04-07T16:20:56.168 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 30 to 118 2024-04-07T16:20:56.168 INFO:tasks.rados.rados.0.smithi100.stdout:1287: copy_from oid 23 from oid 17 current snap is 120 2024-04-07T16:20:56.168 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 24 current snap is 120 2024-04-07T16:20:56.168 INFO:tasks.rados.rados.0.smithi100.stdout:1288: seq_num 398 ranges {9396224=671744,10067968=557056,10625024=532480,11157504=114688} 2024-04-07T16:20:56.238 INFO:tasks.rados.rados.0.smithi100.stdout:1288: writing smithi10066867-24 from 9396224 to 10067968 tid 1 2024-04-07T16:20:56.241 INFO:tasks.rados.rados.0.smithi100.stdout:1288: writing smithi10066867-24 from 10067968 to 10625024 tid 2 2024-04-07T16:20:56.245 INFO:tasks.rados.rados.0.smithi100.stdout:1288: writing smithi10066867-24 from 10625024 to 11157504 tid 3 2024-04-07T16:20:56.246 INFO:tasks.rados.rados.0.smithi100.stdout:1288: writing smithi10066867-24 from 11157504 to 11272192 tid 4 2024-04-07T16:20:56.246 INFO:tasks.rados.rados.0.smithi100.stdout:1289: read oid 34 snap 119 2024-04-07T16:20:56.246 INFO:tasks.rados.rados.0.smithi100.stdout:1289: expect deleted 2024-04-07T16:20:56.246 INFO:tasks.rados.rados.0.smithi100.stdout:1290: snap_create 2024-04-07T16:20:56.246 INFO:tasks.rados.rados.0.smithi100.stdout:1283: finishing write tid 5 to smithi10066867-11 2024-04-07T16:20:56.246 INFO:tasks.rados.rados.0.smithi100.stdout:1283: oid 11 updating version 0 to 827 2024-04-07T16:20:56.247 INFO:tasks.rados.rados.0.smithi100.stdout:1283: oid 11 updating version 827 to 829 2024-04-07T16:20:56.247 INFO:tasks.rados.rados.0.smithi100.stdout:1283: oid 11 updating version 829 to 830 2024-04-07T16:20:56.247 INFO:tasks.rados.rados.0.smithi100.stdout:1283: oid 11 version 830 is already newer than 828 2024-04-07T16:20:56.247 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 11 v 830 (ObjNum 396 snap 120 seq_num 396) dirty exists 2024-04-07T16:20:56.247 INFO:tasks.rados.rados.0.smithi100.stdout:1283: left oid 11 (ObjNum 396 snap 120 seq_num 396) 2024-04-07T16:20:56.247 INFO:tasks.rados.rados.0.smithi100.stdout:1286: finishing rollback tid 0 to smithi10066867-30 2024-04-07T16:20:56.247 INFO:tasks.rados.rados.0.smithi100.stdout:1286: finishing rollback tid 1 to smithi10066867-30 2024-04-07T16:20:56.247 INFO:tasks.rados.rados.0.smithi100.stdout:1286: finishing rollback tid 2 to smithi10066867-30 2024-04-07T16:20:56.247 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 30 v 529 (ObjNum 382 snap 116 seq_num 382) dirty exists 2024-04-07T16:20:56.247 INFO:tasks.rados.rados.0.smithi100.stdout:1287: finishing copy_from racing read to smithi10066867-23 2024-04-07T16:20:56.247 INFO:tasks.rados.rados.0.smithi100.stdout:1287: finishing copy_from to smithi10066867-23 2024-04-07T16:20:56.247 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 23 v 559 (ObjNum 395 snap 119 seq_num 395) dirty exists 2024-04-07T16:20:56.247 INFO:tasks.rados.rados.0.smithi100.stdout:1288: finishing write tid 1 to smithi10066867-24 2024-04-07T16:20:56.248 INFO:tasks.rados.rados.0.smithi100.stdout:1288: finishing write tid 2 to smithi10066867-24 2024-04-07T16:20:56.249 INFO:tasks.rados.rados.0.smithi100.stdout:1288: finishing write tid 3 to smithi10066867-24 2024-04-07T16:20:56.249 INFO:tasks.rados.rados.0.smithi100.stdout:1288: finishing write tid 4 to smithi10066867-24 2024-04-07T16:20:56.250 INFO:tasks.rados.rados.0.smithi100.stdout:1288: finishing write tid 5 to smithi10066867-24 2024-04-07T16:20:56.252 INFO:tasks.rados.rados.0.smithi100.stdout:1288: finishing write tid 6 to smithi10066867-24 2024-04-07T16:20:56.252 INFO:tasks.rados.rados.0.smithi100.stdout:1288: oid 24 updating version 0 to 681 2024-04-07T16:20:56.252 INFO:tasks.rados.rados.0.smithi100.stdout:1288: oid 24 updating version 681 to 683 2024-04-07T16:20:56.252 INFO:tasks.rados.rados.0.smithi100.stdout:1288: oid 24 version 683 is already newer than 679 2024-04-07T16:20:56.252 INFO:tasks.rados.rados.0.smithi100.stdout:1288: oid 24 version 683 is already newer than 682 2024-04-07T16:20:56.252 INFO:tasks.rados.rados.0.smithi100.stdout:1288: oid 24 version 683 is already newer than 680 2024-04-07T16:20:56.252 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 24 v 683 (ObjNum 398 snap 120 seq_num 398) dirty exists 2024-04-07T16:20:56.252 INFO:tasks.rados.rados.0.smithi100.stdout:1288: left oid 24 (ObjNum 398 snap 120 seq_num 398) 2024-04-07T16:20:56.317 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-04-07T16:20:56.317 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": { 2024-04-07T16:20:56.317 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph version 18.2.2-1171-g267c8ff5 (267c8ff5b1ff5815581b3b2381d19360e081b37b) reef (stable)": 1, 2024-04-07T16:20:56.317 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph version 19.0.0-2785-gadce92d0 (adce92d056533f22dc1ee37e3141d098b046ec1b) squid (dev)": 2 2024-04-07T16:20:56.317 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-07T16:20:56.317 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": { 2024-04-07T16:20:56.317 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph version 19.0.0-2785-gadce92d0 (adce92d056533f22dc1ee37e3141d098b046ec1b) squid (dev)": 2 2024-04-07T16:20:56.317 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-07T16:20:56.317 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": { 2024-04-07T16:20:56.317 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph version 18.2.2-1171-g267c8ff5 (267c8ff5b1ff5815581b3b2381d19360e081b37b) reef (stable)": 8 2024-04-07T16:20:56.317 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-07T16:20:56.317 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": { 2024-04-07T16:20:56.318 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph version 18.2.2-1171-g267c8ff5 (267c8ff5b1ff5815581b3b2381d19360e081b37b) reef (stable)": 2 2024-04-07T16:20:56.318 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-07T16:20:56.318 INFO:teuthology.orchestra.run.smithi100.stdout: "overall": { 2024-04-07T16:20:56.318 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph version 18.2.2-1171-g267c8ff5 (267c8ff5b1ff5815581b3b2381d19360e081b37b) reef (stable)": 11, 2024-04-07T16:20:56.318 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph version 19.0.0-2785-gadce92d0 (adce92d056533f22dc1ee37e3141d098b046ec1b) squid (dev)": 4 2024-04-07T16:20:56.318 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-04-07T16:20:56.318 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-04-07T16:20:56.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:56 smithi144 bash[23522]: cluster 2024-04-07T16:20:54.611938+0000 mgr.y (mgr.25175) 37 : cluster 0 pgmap v51: 121 pgs: 121 active+clean; 204 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 5.0 MiB/s rd, 3.6 MiB/s wr, 15 op/s 2024-04-07T16:20:56.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:56 smithi144 bash[23522]: cluster 2024-04-07T16:20:55.097399+0000 mon.a (mon.0) 145 : cluster 0 osdmap e310: 8 total, 8 up, 8 in 2024-04-07T16:20:56.436 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:56 smithi100 bash[74942]: cluster 2024-04-07T16:20:54.611938+0000 mgr.y (mgr.25175) 37 : cluster [DBG] pgmap v51: 121 pgs: 121 active+clean; 204 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 5.0 MiB/s rd, 3.6 MiB/s wr, 15 op/s 2024-04-07T16:20:56.436 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:56 smithi100 bash[74942]: cluster 2024-04-07T16:20:55.097399+0000 mon.a (mon.0) 145 : cluster [DBG] osdmap e310: 8 total, 8 up, 8 in 2024-04-07T16:20:56.436 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:56 smithi100 bash[76202]: cluster 2024-04-07T16:20:54.611938+0000 mgr.y (mgr.25175) 37 : cluster [DBG] pgmap v51: 121 pgs: 121 active+clean; 204 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 5.0 MiB/s rd, 3.6 MiB/s wr, 15 op/s 2024-04-07T16:20:56.436 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:56 smithi100 bash[76202]: cluster 2024-04-07T16:20:55.097399+0000 mon.a (mon.0) 145 : cluster [DBG] osdmap e310: 8 total, 8 up, 8 in 2024-04-07T16:20:56.745 INFO:tasks.rados.rados.0.smithi100.stdout:1283: done (5 left) 2024-04-07T16:20:56.745 INFO:tasks.rados.rados.0.smithi100.stdout:1286: done (4 left) 2024-04-07T16:20:56.745 INFO:tasks.rados.rados.0.smithi100.stdout:1287: done (3 left) 2024-04-07T16:20:56.745 INFO:tasks.rados.rados.0.smithi100.stdout:1288: done (2 left) 2024-04-07T16:20:56.746 INFO:tasks.rados.rados.0.smithi100.stdout:1289: done (1 left) 2024-04-07T16:20:56.746 INFO:tasks.rados.rados.0.smithi100.stdout:1290: done (0 left) 2024-04-07T16:20:56.746 INFO:tasks.rados.rados.0.smithi100.stdout:1291: delete oid 26 current snap is 121 2024-04-07T16:20:56.762 INFO:tasks.rados.rados.0.smithi100.stdout:1291: done (0 left) 2024-04-07T16:20:56.762 INFO:tasks.rados.rados.0.smithi100.stdout:1292: delete oid 5 current snap is 121 2024-04-07T16:20:56.775 INFO:tasks.rados.rados.0.smithi100.stdout:1292: done (0 left) 2024-04-07T16:20:56.775 INFO:tasks.rados.rados.0.smithi100.stdout:1293: copy_from oid 21 from oid 29 current snap is 121 2024-04-07T16:20:56.775 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 3 current snap is 121 2024-04-07T16:20:56.775 INFO:tasks.rados.rados.0.smithi100.stdout:1294: seq_num 401 ranges {0=720896,720896=606208,1327104=598016} 2024-04-07T16:20:56.782 INFO:tasks.rados.rados.0.smithi100.stdout:1294: writing smithi10066867-3 from 0 to 720896 tid 1 2024-04-07T16:20:56.787 INFO:tasks.rados.rados.0.smithi100.stdout:1294: writing smithi10066867-3 from 720896 to 1327104 tid 2 2024-04-07T16:20:56.793 INFO:tasks.rados.rados.0.smithi100.stdout:1294: writing smithi10066867-3 from 1327104 to 1925120 tid 3 2024-04-07T16:20:56.793 INFO:tasks.rados.rados.0.smithi100.stdout:1295: copy_from oid 35 from oid 34 current snap is 121 2024-04-07T16:20:56.793 INFO:tasks.rados.rados.0.smithi100.stdout:1296: read oid 19 snap -1 2024-04-07T16:20:56.794 INFO:tasks.rados.rados.0.smithi100.stdout:1296: expect (ObjNum 374 snap 113 seq_num 374) 2024-04-07T16:20:56.794 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 41 current snap is 121 2024-04-07T16:20:56.794 INFO:tasks.rados.rados.0.smithi100.stdout:1297: seq_num 402 ranges {0=737280,737280=491520,1228800=516096,1744896=196608} 2024-04-07T16:20:56.800 INFO:tasks.rados.rados.0.smithi100.stdout:1297: writing smithi10066867-41 from 0 to 737280 tid 1 2024-04-07T16:20:56.804 INFO:tasks.rados.rados.0.smithi100.stdout:1297: writing smithi10066867-41 from 737280 to 1228800 tid 2 2024-04-07T16:20:56.807 INFO:tasks.rados.rados.0.smithi100.stdout:1297: writing smithi10066867-41 from 1228800 to 1744896 tid 3 2024-04-07T16:20:56.809 INFO:tasks.rados.rados.0.smithi100.stdout:1297: writing smithi10066867-41 from 1744896 to 1941504 tid 4 2024-04-07T16:20:56.809 INFO:tasks.rados.rados.0.smithi100.stdout:1298: copy_from oid 31 from oid 36 current snap is 121 2024-04-07T16:20:56.809 INFO:tasks.rados.rados.0.smithi100.stdout:1295: finishing copy_from to smithi10066867-35 2024-04-07T16:20:56.809 INFO:tasks.rados.rados.0.smithi100.stdout:1295: got expected ENOENT (src dne) 2024-04-07T16:20:56.809 INFO:tasks.rados.rados.0.smithi100.stdout:1299: rmattr oid 44 current snap is 121 2024-04-07T16:20:56.809 INFO:tasks.rados.rados.0.smithi100.stdout:1295: finishing copy_from racing read to smithi10066867-35 2024-04-07T16:20:56.809 INFO:tasks.rados.rados.0.smithi100.stdout:1293: finishing copy_from racing read to smithi10066867-21 2024-04-07T16:20:56.809 INFO:tasks.rados.rados.0.smithi100.stdout:1295: done (6 left) 2024-04-07T16:20:56.809 INFO:tasks.rados.rados.0.smithi100.stdout:1300: snap_remove snap 120 2024-04-07T16:20:57.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:57 smithi144 bash[23522]: audit 2024-04-07T16:20:55.469050+0000 mgr.y (mgr.25175) 38 : audit 0 from='client.34185 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:20:57.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:57 smithi144 bash[23522]: audit 2024-04-07T16:20:55.835818+0000 mgr.y (mgr.25175) 39 : audit 0 from='client.34191 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:20:57.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:57 smithi144 bash[23522]: cluster 2024-04-07T16:20:56.101647+0000 mon.a (mon.0) 146 : cluster 0 osdmap e311: 8 total, 8 up, 8 in 2024-04-07T16:20:57.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:57 smithi144 bash[23522]: audit 2024-04-07T16:20:56.333657+0000 mon.a (mon.0) 147 : audit 0 from='client.? 172.21.15.100:0/3549603971' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:20:57.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:57 smithi144 bash[23522]: cluster 2024-04-07T16:20:56.759112+0000 mon.a (mon.0) 148 : cluster 0 osdmap e312: 8 total, 8 up, 8 in 2024-04-07T16:20:57.436 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:57 smithi100 bash[74942]: audit 2024-04-07T16:20:55.469050+0000 mgr.y (mgr.25175) 38 : audit [DBG] from='client.34185 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:20:57.436 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:57 smithi100 bash[74942]: audit 2024-04-07T16:20:55.835818+0000 mgr.y (mgr.25175) 39 : audit [DBG] from='client.34191 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:20:57.436 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:57 smithi100 bash[74942]: cluster 2024-04-07T16:20:56.101647+0000 mon.a (mon.0) 146 : cluster [DBG] osdmap e311: 8 total, 8 up, 8 in 2024-04-07T16:20:57.436 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:57 smithi100 bash[74942]: audit 2024-04-07T16:20:56.333657+0000 mon.a (mon.0) 147 : audit [DBG] from='client.? 172.21.15.100:0/3549603971' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:20:57.436 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:57 smithi100 bash[74942]: cluster 2024-04-07T16:20:56.759112+0000 mon.a (mon.0) 148 : cluster [DBG] osdmap e312: 8 total, 8 up, 8 in 2024-04-07T16:20:57.436 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:57 smithi100 bash[76202]: audit 2024-04-07T16:20:55.469050+0000 mgr.y (mgr.25175) 38 : audit [DBG] from='client.34185 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:20:57.436 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:57 smithi100 bash[76202]: audit 2024-04-07T16:20:55.835818+0000 mgr.y (mgr.25175) 39 : audit [DBG] from='client.34191 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:20:57.436 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:57 smithi100 bash[76202]: cluster 2024-04-07T16:20:56.101647+0000 mon.a (mon.0) 146 : cluster [DBG] osdmap e311: 8 total, 8 up, 8 in 2024-04-07T16:20:57.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:57 smithi100 bash[76202]: audit 2024-04-07T16:20:56.333657+0000 mon.a (mon.0) 147 : audit [DBG] from='client.? 172.21.15.100:0/3549603971' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:20:57.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:57 smithi100 bash[76202]: cluster 2024-04-07T16:20:56.759112+0000 mon.a (mon.0) 148 : cluster [DBG] osdmap e312: 8 total, 8 up, 8 in 2024-04-07T16:20:57.760 INFO:tasks.rados.rados.0.smithi100.stdout:1300: done (6 left) 2024-04-07T16:20:57.760 INFO:tasks.rados.rados.0.smithi100.stdout:1301: copy_from oid 4 from oid 25 current snap is 121 2024-04-07T16:20:57.760 INFO:tasks.rados.rados.0.smithi100.stdout:1294: finishing write tid 1 to smithi10066867-3 2024-04-07T16:20:57.760 INFO:tasks.rados.rados.0.smithi100.stdout:1293: finishing copy_from to smithi10066867-21 2024-04-07T16:20:57.761 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 21 v 524 (ObjNum 391 snap 119 seq_num 391) dirty exists 2024-04-07T16:20:57.761 INFO:tasks.rados.rados.0.smithi100.stdout:1296: expect (ObjNum 374 snap 113 seq_num 374) 2024-04-07T16:20:57.975 INFO:tasks.rados.rados.0.smithi100.stdout:1294: finishing write tid 2 to smithi10066867-3 2024-04-07T16:20:57.975 INFO:tasks.rados.rados.0.smithi100.stdout:1297: finishing write tid 1 to smithi10066867-41 2024-04-07T16:20:57.975 INFO:tasks.rados.rados.0.smithi100.stdout:1294: finishing write tid 3 to smithi10066867-3 2024-04-07T16:20:57.975 INFO:tasks.rados.rados.0.smithi100.stdout:1297: finishing write tid 2 to smithi10066867-41 2024-04-07T16:20:57.975 INFO:tasks.rados.rados.0.smithi100.stdout:1294: finishing write tid 4 to smithi10066867-3 2024-04-07T16:20:57.975 INFO:tasks.rados.rados.0.smithi100.stdout:1297: finishing write tid 3 to smithi10066867-41 2024-04-07T16:20:57.975 INFO:tasks.rados.rados.0.smithi100.stdout:1297: finishing write tid 4 to smithi10066867-41 2024-04-07T16:20:57.975 INFO:tasks.rados.rados.0.smithi100.stdout:1297: finishing write tid 5 to smithi10066867-41 2024-04-07T16:20:57.975 INFO:tasks.rados.rados.0.smithi100.stdout:1297: finishing write tid 6 to smithi10066867-41 2024-04-07T16:20:57.976 INFO:tasks.rados.rados.0.smithi100.stdout:1297: oid 41 updating version 0 to 706 2024-04-07T16:20:57.976 INFO:tasks.rados.rados.0.smithi100.stdout:1297: oid 41 updating version 706 to 707 2024-04-07T16:20:57.976 INFO:tasks.rados.rados.0.smithi100.stdout:1297: oid 41 updating version 707 to 709 2024-04-07T16:20:57.976 INFO:tasks.rados.rados.0.smithi100.stdout:1297: oid 41 updating version 709 to 710 2024-04-07T16:20:57.976 INFO:tasks.rados.rados.0.smithi100.stdout:1297: oid 41 version 710 is already newer than 708 2024-04-07T16:20:57.976 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 41 v 710 (ObjNum 402 snap 121 seq_num 402) dirty exists 2024-04-07T16:20:57.976 INFO:tasks.rados.rados.0.smithi100.stdout:1297: left oid 41 (ObjNum 402 snap 121 seq_num 402) 2024-04-07T16:20:57.976 INFO:tasks.rados.rados.0.smithi100.stdout:1294: finishing write tid 5 to smithi10066867-3 2024-04-07T16:20:57.976 INFO:tasks.rados.rados.0.smithi100.stdout:1294: oid 3 updating version 0 to 545 2024-04-07T16:20:57.976 INFO:tasks.rados.rados.0.smithi100.stdout:1294: oid 3 updating version 545 to 546 2024-04-07T16:20:57.976 INFO:tasks.rados.rados.0.smithi100.stdout:1294: oid 3 version 546 is already newer than 544 2024-04-07T16:20:57.976 INFO:tasks.rados.rados.0.smithi100.stdout:1294: oid 3 version 546 is already newer than 543 2024-04-07T16:20:57.976 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 3 v 546 (ObjNum 401 snap 121 seq_num 401) dirty exists 2024-04-07T16:20:57.976 INFO:tasks.rados.rados.0.smithi100.stdout:1294: left oid 3 (ObjNum 401 snap 121 seq_num 401) 2024-04-07T16:20:57.976 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 44 v 681 (ObjNum 171 snap 44 seq_num 171) dirty exists 2024-04-07T16:20:57.977 INFO:tasks.rados.rados.0.smithi100.stdout:1298: finishing copy_from racing read to smithi10066867-31 2024-04-07T16:20:57.977 INFO:tasks.rados.rados.0.smithi100.stdout:1298: finishing copy_from to smithi10066867-31 2024-04-07T16:20:57.977 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 31 v 550 (ObjNum 347 snap 104 seq_num 347) dirty exists 2024-04-07T16:20:57.977 INFO:tasks.rados.rados.0.smithi100.stdout:1301: finishing copy_from to smithi10066867-4 2024-04-07T16:20:57.977 INFO:tasks.rados.rados.0.smithi100.stdout:1301: got expected ENOENT (src dne) 2024-04-07T16:20:57.977 INFO:tasks.rados.rados.0.smithi100.stdout:1301: finishing copy_from racing read to smithi10066867-4 2024-04-07T16:20:57.977 INFO:tasks.rados.rados.0.smithi100.stdout:1293: done (6 left) 2024-04-07T16:20:57.977 INFO:tasks.rados.rados.0.smithi100.stdout:1294: done (5 left) 2024-04-07T16:20:57.977 INFO:tasks.rados.rados.0.smithi100.stdout:1296: done (4 left) 2024-04-07T16:20:57.977 INFO:tasks.rados.rados.0.smithi100.stdout:1297: done (3 left) 2024-04-07T16:20:57.977 INFO:tasks.rados.rados.0.smithi100.stdout:1298: done (2 left) 2024-04-07T16:20:57.977 INFO:tasks.rados.rados.0.smithi100.stdout:1299: done (1 left) 2024-04-07T16:20:57.977 INFO:tasks.rados.rados.0.smithi100.stdout:1301: done (0 left) 2024-04-07T16:20:57.977 INFO:tasks.rados.rados.0.smithi100.stdout:1302: snap_create 2024-04-07T16:20:58.416 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:58 smithi100 bash[74942]: cluster 2024-04-07T16:20:56.612648+0000 mgr.y (mgr.25175) 40 : cluster [DBG] pgmap v54: 121 pgs: 121 active+clean; 204 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail 2024-04-07T16:20:58.416 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:58 smithi100 bash[74942]: cluster 2024-04-07T16:20:57.773362+0000 mon.a (mon.0) 149 : cluster [DBG] osdmap e313: 8 total, 8 up, 8 in 2024-04-07T16:20:58.417 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:58 smithi100 bash[76202]: cluster 2024-04-07T16:20:56.612648+0000 mgr.y (mgr.25175) 40 : cluster [DBG] pgmap v54: 121 pgs: 121 active+clean; 204 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail 2024-04-07T16:20:58.417 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:58 smithi100 bash[76202]: cluster 2024-04-07T16:20:57.773362+0000 mon.a (mon.0) 149 : cluster [DBG] osdmap e313: 8 total, 8 up, 8 in 2024-04-07T16:20:58.605 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:58 smithi144 bash[23522]: cluster 2024-04-07T16:20:56.612648+0000 mgr.y (mgr.25175) 40 : cluster 0 pgmap v54: 121 pgs: 121 active+clean; 204 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail 2024-04-07T16:20:58.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:58 smithi144 bash[23522]: cluster 2024-04-07T16:20:57.773362+0000 mon.a (mon.0) 149 : cluster 0 osdmap e313: 8 total, 8 up, 8 in 2024-04-07T16:20:58.686 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:20:58 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:20:58] "GET /metrics HTTP/1.1" 200 36082 "" "Prometheus/2.43.0" 2024-04-07T16:20:58.760 INFO:tasks.rados.rados.0.smithi100.stdout:1302: done (0 left) 2024-04-07T16:20:58.760 INFO:tasks.rados.rados.0.smithi100.stdout:1303: read oid 31 snap 121 2024-04-07T16:20:58.760 INFO:tasks.rados.rados.0.smithi100.stdout:1303: expect (ObjNum 347 snap 104 seq_num 347) 2024-04-07T16:20:58.760 INFO:tasks.rados.rados.0.smithi100.stdout:1304: delete oid 4 current snap is 122 2024-04-07T16:20:58.771 INFO:tasks.rados.rados.0.smithi100.stdout:1304: done (1 left) 2024-04-07T16:20:58.771 INFO:tasks.rados.rados.0.smithi100.stdout:1305: snap_remove snap 103 2024-04-07T16:20:59.771 INFO:tasks.rados.rados.0.smithi100.stdout:1305: done (1 left) 2024-04-07T16:20:59.771 INFO:tasks.rados.rados.0.smithi100.stdout:1306: read oid 9 snap -1 2024-04-07T16:20:59.771 INFO:tasks.rados.rados.0.smithi100.stdout:1306: expect deleted 2024-04-07T16:20:59.771 INFO:tasks.rados.rados.0.smithi100.stdout:1307: delete oid 47 current snap is 122 2024-04-07T16:20:59.773 INFO:tasks.rados.rados.0.smithi100.stdout:1303: expect (ObjNum 347 snap 104 seq_num 347) 2024-04-07T16:20:59.816 INFO:tasks.rados.rados.0.smithi100.stdout:1303: done (2 left) 2024-04-07T16:20:59.816 INFO:tasks.rados.rados.0.smithi100.stdout:1307: done (1 left) 2024-04-07T16:20:59.816 INFO:tasks.rados.rados.0.smithi100.stdout:1308: snap_remove snap 121 2024-04-07T16:21:00.007 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:59 smithi100 bash[74942]: cluster 2024-04-07T16:20:58.613304+0000 mgr.y (mgr.25175) 41 : cluster [DBG] pgmap v57: 121 pgs: 121 active+clean; 230 MiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 7.3 MiB/s rd, 7.9 MiB/s wr, 36 op/s 2024-04-07T16:21:00.008 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:20:59 smithi100 bash[74942]: cluster 2024-04-07T16:20:58.775644+0000 mon.a (mon.0) 150 : cluster [DBG] osdmap e314: 8 total, 8 up, 8 in 2024-04-07T16:21:00.008 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:59 smithi100 bash[76202]: cluster 2024-04-07T16:20:58.613304+0000 mgr.y (mgr.25175) 41 : cluster [DBG] pgmap v57: 121 pgs: 121 active+clean; 230 MiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 7.3 MiB/s rd, 7.9 MiB/s wr, 36 op/s 2024-04-07T16:21:00.008 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:20:59 smithi100 bash[76202]: cluster 2024-04-07T16:20:58.775644+0000 mon.a (mon.0) 150 : cluster [DBG] osdmap e314: 8 total, 8 up, 8 in 2024-04-07T16:21:00.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:59 smithi144 bash[23522]: cluster 2024-04-07T16:20:58.613304+0000 mgr.y (mgr.25175) 41 : cluster 0 pgmap v57: 121 pgs: 121 active+clean; 230 MiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 7.3 MiB/s rd, 7.9 MiB/s wr, 36 op/s 2024-04-07T16:21:00.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:20:59 smithi144 bash[23522]: cluster 2024-04-07T16:20:58.775644+0000 mon.a (mon.0) 150 : cluster 0 osdmap e314: 8 total, 8 up, 8 in 2024-04-07T16:21:00.784 INFO:tasks.rados.rados.0.smithi100.stdout:1308: done (1 left) 2024-04-07T16:21:00.784 INFO:tasks.rados.rados.0.smithi100.stdout:1309: read oid 39 snap 118 2024-04-07T16:21:00.784 INFO:tasks.rados.rados.0.smithi100.stdout:1309: expect (ObjNum 322 snap 96 seq_num 322) 2024-04-07T16:21:00.784 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 34 current snap is 122 2024-04-07T16:21:00.785 INFO:tasks.rados.rados.0.smithi100.stdout:1310: seq_num 405 ranges {0=786432,786432=589824,1376256=614400} 2024-04-07T16:21:00.792 INFO:tasks.rados.rados.0.smithi100.stdout:1310: writing smithi10066867-34 from 0 to 786432 tid 1 2024-04-07T16:21:00.799 INFO:tasks.rados.rados.0.smithi100.stdout:1310: writing smithi10066867-34 from 786432 to 1376256 tid 2 2024-04-07T16:21:00.805 INFO:tasks.rados.rados.0.smithi100.stdout:1310: writing smithi10066867-34 from 1376256 to 1990656 tid 3 2024-04-07T16:21:00.805 INFO:tasks.rados.rados.0.smithi100.stdout:1311: snap_create 2024-04-07T16:21:00.805 INFO:tasks.rados.rados.0.smithi100.stdout:1309: expect (ObjNum 322 snap 96 seq_num 322) 2024-04-07T16:21:00.846 INFO:tasks.rados.rados.0.smithi100.stdout:1310: finishing write tid 1 to smithi10066867-34 2024-04-07T16:21:00.846 INFO:tasks.rados.rados.0.smithi100.stdout:1310: finishing write tid 2 to smithi10066867-34 2024-04-07T16:21:00.846 INFO:tasks.rados.rados.0.smithi100.stdout:1310: finishing write tid 3 to smithi10066867-34 2024-04-07T16:21:00.847 INFO:tasks.rados.rados.0.smithi100.stdout:1310: finishing write tid 4 to smithi10066867-34 2024-04-07T16:21:00.847 INFO:tasks.rados.rados.0.smithi100.stdout:1310: finishing write tid 5 to smithi10066867-34 2024-04-07T16:21:00.847 INFO:tasks.rados.rados.0.smithi100.stdout:1310: oid 34 updating version 0 to 563 2024-04-07T16:21:00.847 INFO:tasks.rados.rados.0.smithi100.stdout:1310: oid 34 updating version 563 to 565 2024-04-07T16:21:00.847 INFO:tasks.rados.rados.0.smithi100.stdout:1310: oid 34 version 565 is already newer than 562 2024-04-07T16:21:00.847 INFO:tasks.rados.rados.0.smithi100.stdout:1310: oid 34 version 565 is already newer than 564 2024-04-07T16:21:00.847 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 34 v 565 (ObjNum 405 snap 122 seq_num 405) dirty exists 2024-04-07T16:21:00.848 INFO:tasks.rados.rados.0.smithi100.stdout:1310: left oid 34 (ObjNum 405 snap 122 seq_num 405) 2024-04-07T16:21:01.105 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:00 smithi144 bash[23522]: cluster 2024-04-07T16:20:59.785582+0000 mon.a (mon.0) 151 : cluster 0 osdmap e315: 8 total, 8 up, 8 in 2024-04-07T16:21:01.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:00 smithi100 bash[74942]: cluster 2024-04-07T16:20:59.785582+0000 mon.a (mon.0) 151 : cluster [DBG] osdmap e315: 8 total, 8 up, 8 in 2024-04-07T16:21:01.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:00 smithi100 bash[76202]: cluster 2024-04-07T16:20:59.785582+0000 mon.a (mon.0) 151 : cluster [DBG] osdmap e315: 8 total, 8 up, 8 in 2024-04-07T16:21:01.744 INFO:tasks.rados.rados.0.smithi100.stdout:1306: done (3 left) 2024-04-07T16:21:01.745 INFO:tasks.rados.rados.0.smithi100.stdout:1309: done (2 left) 2024-04-07T16:21:01.745 INFO:tasks.rados.rados.0.smithi100.stdout:1310: done (1 left) 2024-04-07T16:21:01.745 INFO:tasks.rados.rados.0.smithi100.stdout:1311: done (0 left) 2024-04-07T16:21:01.745 INFO:tasks.rados.rados.0.smithi100.stdout:1312: read oid 39 snap -1 2024-04-07T16:21:01.745 INFO:tasks.rados.rados.0.smithi100.stdout:1312: expect (ObjNum 322 snap 96 seq_num 322) 2024-04-07T16:21:01.745 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 31 current snap is 123 2024-04-07T16:21:01.745 INFO:tasks.rados.rados.0.smithi100.stdout:1313: seq_num 406 ranges {1015808=802816,1818624=475136,2293760=499712,2793472=229376} 2024-04-07T16:21:01.755 INFO:tasks.rados.rados.0.smithi100.stdout:1313: writing smithi10066867-31 from 1015808 to 1818624 tid 1 2024-04-07T16:21:01.759 INFO:tasks.rados.rados.0.smithi100.stdout:1313: writing smithi10066867-31 from 1818624 to 2293760 tid 2 2024-04-07T16:21:01.762 INFO:tasks.rados.rados.0.smithi100.stdout:1313: writing smithi10066867-31 from 2293760 to 2793472 tid 3 2024-04-07T16:21:01.764 INFO:tasks.rados.rados.0.smithi100.stdout:1313: writing smithi10066867-31 from 2793472 to 3022848 tid 4 2024-04-07T16:21:01.764 INFO:tasks.rados.rados.0.smithi100.stdout:1314: read oid 20 snap -1 2024-04-07T16:21:01.764 INFO:tasks.rados.rados.0.smithi100.stdout:1314: expect (ObjNum 328 snap 99 seq_num 328) 2024-04-07T16:21:01.765 INFO:tasks.rados.rados.0.smithi100.stdout:1315: setattr oid 1 current snap is 123 2024-04-07T16:21:01.773 INFO:tasks.rados.rados.0.smithi100.stdout:1312: expect (ObjNum 322 snap 96 seq_num 322) 2024-04-07T16:21:01.798 INFO:tasks.rados.rados.0.smithi100.stdout:1313: finishing write tid 1 to smithi10066867-31 2024-04-07T16:21:01.798 INFO:tasks.rados.rados.0.smithi100.stdout:1313: finishing write tid 2 to smithi10066867-31 2024-04-07T16:21:01.798 INFO:tasks.rados.rados.0.smithi100.stdout:1313: finishing write tid 3 to smithi10066867-31 2024-04-07T16:21:01.798 INFO:tasks.rados.rados.0.smithi100.stdout:1313: finishing write tid 4 to smithi10066867-31 2024-04-07T16:21:01.799 INFO:tasks.rados.rados.0.smithi100.stdout:1313: finishing write tid 5 to smithi10066867-31 2024-04-07T16:21:01.799 INFO:tasks.rados.rados.0.smithi100.stdout:1313: finishing write tid 6 to smithi10066867-31 2024-04-07T16:21:01.799 INFO:tasks.rados.rados.0.smithi100.stdout:1313: oid 31 updating version 0 to 555 2024-04-07T16:21:01.799 INFO:tasks.rados.rados.0.smithi100.stdout:1313: oid 31 version 555 is already newer than 552 2024-04-07T16:21:01.799 INFO:tasks.rados.rados.0.smithi100.stdout:1313: oid 31 version 555 is already newer than 554 2024-04-07T16:21:01.799 INFO:tasks.rados.rados.0.smithi100.stdout:1313: oid 31 version 555 is already newer than 553 2024-04-07T16:21:01.799 INFO:tasks.rados.rados.0.smithi100.stdout:1313: oid 31 version 555 is already newer than 551 2024-04-07T16:21:01.799 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 31 v 555 (ObjNum 406 snap 123 seq_num 406) dirty exists 2024-04-07T16:21:01.799 INFO:tasks.rados.rados.0.smithi100.stdout:1313: left oid 31 (ObjNum 406 snap 123 seq_num 406) 2024-04-07T16:21:01.799 INFO:tasks.rados.rados.0.smithi100.stdout:1314: expect (ObjNum 328 snap 99 seq_num 328) 2024-04-07T16:21:01.838 INFO:tasks.rados.rados.0.smithi100.stdout:1312: done (3 left) 2024-04-07T16:21:01.838 INFO:tasks.rados.rados.0.smithi100.stdout:1313: done (2 left) 2024-04-07T16:21:01.838 INFO:tasks.rados.rados.0.smithi100.stdout:1314: done (1 left) 2024-04-07T16:21:01.839 INFO:tasks.rados.rados.0.smithi100.stdout:1316: setattr oid 15 current snap is 123 2024-04-07T16:21:01.841 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 6 current snap is 123 2024-04-07T16:21:01.841 INFO:tasks.rados.rados.0.smithi100.stdout:1317: seq_num 407 ranges {5406720=425984,5832704=761856,6594560=704512,7299072=139264} 2024-04-07T16:21:01.887 INFO:tasks.rados.rados.0.smithi100.stdout:1317: writing smithi10066867-6 from 5406720 to 5832704 tid 1 2024-04-07T16:21:01.893 INFO:tasks.rados.rados.0.smithi100.stdout:1317: writing smithi10066867-6 from 5832704 to 6594560 tid 2 2024-04-07T16:21:01.898 INFO:tasks.rados.rados.0.smithi100.stdout:1317: writing smithi10066867-6 from 6594560 to 7299072 tid 3 2024-04-07T16:21:01.899 INFO:tasks.rados.rados.0.smithi100.stdout:1317: writing smithi10066867-6 from 7299072 to 7438336 tid 4 2024-04-07T16:21:01.899 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 26 current snap is 123 2024-04-07T16:21:01.899 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 1 v 817 (ObjNum 385 snap 118 seq_num 385) dirty exists 2024-04-07T16:21:01.899 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 15 v 849 (ObjNum 338 snap 101 seq_num 338) dirty exists 2024-04-07T16:21:01.899 INFO:tasks.rados.rados.0.smithi100.stdout:1317: finishing write tid 1 to smithi10066867-6 2024-04-07T16:21:01.899 INFO:tasks.rados.rados.0.smithi100.stdout:1317: finishing write tid 2 to smithi10066867-6 2024-04-07T16:21:01.899 INFO:tasks.rados.rados.0.smithi100.stdout:1318: seq_num 408 ranges {0=442368,442368=647168,1089536=507904,1597440=450560} 2024-04-07T16:21:01.902 INFO:tasks.rados.rados.0.smithi100.stdout:1318: writing smithi10066867-26 from 0 to 442368 tid 1 2024-04-07T16:21:01.907 INFO:tasks.rados.rados.0.smithi100.stdout:1318: writing smithi10066867-26 from 442368 to 1089536 tid 2 2024-04-07T16:21:01.911 INFO:tasks.rados.rados.0.smithi100.stdout:1318: writing smithi10066867-26 from 1089536 to 1597440 tid 3 2024-04-07T16:21:01.915 INFO:tasks.rados.rados.0.smithi100.stdout:1318: writing smithi10066867-26 from 1597440 to 2048000 tid 4 2024-04-07T16:21:01.915 INFO:tasks.rados.rados.0.smithi100.stdout:1315: done (3 left) 2024-04-07T16:21:01.915 INFO:tasks.rados.rados.0.smithi100.stdout:1316: done (2 left) 2024-04-07T16:21:01.915 INFO:tasks.rados.rados.0.smithi100.stdout:1319: rollback oid 30 current snap is 123 2024-04-07T16:21:01.915 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 30 to 118 2024-04-07T16:21:01.915 INFO:tasks.rados.rados.0.smithi100.stdout:1317: finishing write tid 3 to smithi10066867-6 2024-04-07T16:21:01.915 INFO:tasks.rados.rados.0.smithi100.stdout:1317: finishing write tid 4 to smithi10066867-6 2024-04-07T16:21:01.915 INFO:tasks.rados.rados.0.smithi100.stdout:1317: finishing write tid 5 to smithi10066867-6 2024-04-07T16:21:01.915 INFO:tasks.rados.rados.0.smithi100.stdout:1318: finishing write tid 1 to smithi10066867-26 2024-04-07T16:21:01.916 INFO:tasks.rados.rados.0.smithi100.stdout:1317: finishing write tid 6 to smithi10066867-6 2024-04-07T16:21:01.916 INFO:tasks.rados.rados.0.smithi100.stdout:1317: oid 6 updating version 0 to 685 2024-04-07T16:21:01.916 INFO:tasks.rados.rados.0.smithi100.stdout:1317: oid 6 version 685 is already newer than 684 2024-04-07T16:21:01.916 INFO:tasks.rados.rados.0.smithi100.stdout:1317: oid 6 updating version 685 to 686 2024-04-07T16:21:01.916 INFO:tasks.rados.rados.0.smithi100.stdout:1317: oid 6 version 686 is already newer than 682 2024-04-07T16:21:01.916 INFO:tasks.rados.rados.0.smithi100.stdout:1317: oid 6 version 686 is already newer than 683 2024-04-07T16:21:01.916 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 6 v 686 (ObjNum 407 snap 123 seq_num 407) dirty exists 2024-04-07T16:21:01.916 INFO:tasks.rados.rados.0.smithi100.stdout:1317: left oid 6 (ObjNum 407 snap 123 seq_num 407) 2024-04-07T16:21:01.916 INFO:tasks.rados.rados.0.smithi100.stdout:1318: finishing write tid 2 to smithi10066867-26 2024-04-07T16:21:01.916 INFO:tasks.rados.rados.0.smithi100.stdout:1318: finishing write tid 3 to smithi10066867-26 2024-04-07T16:21:01.916 INFO:tasks.rados.rados.0.smithi100.stdout:1317: done (2 left) 2024-04-07T16:21:01.916 INFO:tasks.rados.rados.0.smithi100.stdout:1320: snap_remove snap 109 2024-04-07T16:21:01.935 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:01 smithi100 bash[74942]: cluster 2024-04-07T16:21:00.614050+0000 mgr.y (mgr.25175) 42 : cluster [DBG] pgmap v60: 121 pgs: 121 active+clean; 230 MiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 7.3 MiB/s rd, 7.9 MiB/s wr, 36 op/s 2024-04-07T16:21:01.935 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:01 smithi100 bash[74942]: cluster 2024-04-07T16:21:00.799679+0000 mon.a (mon.0) 152 : cluster [DBG] osdmap e316: 8 total, 8 up, 8 in 2024-04-07T16:21:01.935 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:01 smithi100 bash[74942]: audit 2024-04-07T16:21:01.700934+0000 mon.a (mon.0) 153 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:21:01.935 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:01 smithi100 bash[74942]: cluster 2024-04-07T16:21:01.756708+0000 mon.a (mon.0) 154 : cluster [DBG] osdmap e317: 8 total, 8 up, 8 in 2024-04-07T16:21:01.936 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:01 smithi100 bash[76202]: cluster 2024-04-07T16:21:00.614050+0000 mgr.y (mgr.25175) 42 : cluster [DBG] pgmap v60: 121 pgs: 121 active+clean; 230 MiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 7.3 MiB/s rd, 7.9 MiB/s wr, 36 op/s 2024-04-07T16:21:01.936 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:01 smithi100 bash[76202]: cluster 2024-04-07T16:21:00.799679+0000 mon.a (mon.0) 152 : cluster [DBG] osdmap e316: 8 total, 8 up, 8 in 2024-04-07T16:21:01.936 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:01 smithi100 bash[76202]: audit 2024-04-07T16:21:01.700934+0000 mon.a (mon.0) 153 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:21:01.936 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:01 smithi100 bash[76202]: cluster 2024-04-07T16:21:01.756708+0000 mon.a (mon.0) 154 : cluster [DBG] osdmap e317: 8 total, 8 up, 8 in 2024-04-07T16:21:02.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:01 smithi144 bash[23522]: cluster 2024-04-07T16:21:00.614050+0000 mgr.y (mgr.25175) 42 : cluster 0 pgmap v60: 121 pgs: 121 active+clean; 230 MiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 7.3 MiB/s rd, 7.9 MiB/s wr, 36 op/s 2024-04-07T16:21:02.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:01 smithi144 bash[23522]: cluster 2024-04-07T16:21:00.799679+0000 mon.a (mon.0) 152 : cluster 0 osdmap e316: 8 total, 8 up, 8 in 2024-04-07T16:21:02.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:01 smithi144 bash[23522]: audit 2024-04-07T16:21:01.700934+0000 mon.a (mon.0) 153 : audit 0 from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:21:02.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:01 smithi144 bash[23522]: cluster 2024-04-07T16:21:01.756708+0000 mon.a (mon.0) 154 : cluster 0 osdmap e317: 8 total, 8 up, 8 in 2024-04-07T16:21:02.606 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:21:02 smithi144 bash[59794]: ts=2024-04-07T16:21:02.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:21:02.798 INFO:tasks.rados.rados.0.smithi100.stdout:1320: done (2 left) 2024-04-07T16:21:02.798 INFO:tasks.rados.rados.0.smithi100.stdout:1321: delete oid 2 current snap is 123 2024-04-07T16:21:02.798 INFO:tasks.rados.rados.0.smithi100.stdout:1318: finishing write tid 4 to smithi10066867-26 2024-04-07T16:21:02.799 INFO:tasks.rados.rados.0.smithi100.stdout:1318: finishing write tid 5 to smithi10066867-26 2024-04-07T16:21:02.799 INFO:tasks.rados.rados.0.smithi100.stdout:1319: finishing rollback tid 0 to smithi10066867-30 2024-04-07T16:21:02.799 INFO:tasks.rados.rados.0.smithi100.stdout:1318: finishing write tid 6 to smithi10066867-26 2024-04-07T16:21:02.799 INFO:tasks.rados.rados.0.smithi100.stdout:1318: oid 26 updating version 0 to 522 2024-04-07T16:21:02.799 INFO:tasks.rados.rados.0.smithi100.stdout:1318: oid 26 version 522 is already newer than 519 2024-04-07T16:21:02.799 INFO:tasks.rados.rados.0.smithi100.stdout:1318: oid 26 updating version 522 to 523 2024-04-07T16:21:02.799 INFO:tasks.rados.rados.0.smithi100.stdout:1318: oid 26 version 523 is already newer than 521 2024-04-07T16:21:02.799 INFO:tasks.rados.rados.0.smithi100.stdout:1318: oid 26 version 523 is already newer than 520 2024-04-07T16:21:02.799 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 26 v 523 (ObjNum 408 snap 123 seq_num 408) dirty exists 2024-04-07T16:21:02.799 INFO:tasks.rados.rados.0.smithi100.stdout:1318: left oid 26 (ObjNum 408 snap 123 seq_num 408) 2024-04-07T16:21:02.799 INFO:tasks.rados.rados.0.smithi100.stdout:1319: finishing rollback tid 1 to smithi10066867-30 2024-04-07T16:21:02.799 INFO:tasks.rados.rados.0.smithi100.stdout:1319: finishing rollback tid 2 to smithi10066867-30 2024-04-07T16:21:02.799 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 30 v 537 (ObjNum 382 snap 116 seq_num 382) dirty exists 2024-04-07T16:21:02.823 INFO:tasks.rados.rados.0.smithi100.stdout:1318: done (2 left) 2024-04-07T16:21:02.823 INFO:tasks.rados.rados.0.smithi100.stdout:1319: done (1 left) 2024-04-07T16:21:02.823 INFO:tasks.rados.rados.0.smithi100.stdout:1321: done (0 left) 2024-04-07T16:21:02.823 INFO:tasks.rados.rados.0.smithi100.stdout:1322: snap_create 2024-04-07T16:21:03.796 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:21:03 smithi144 bash[59794]: ts=2024-04-07T16:21:03.507Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=nodes name=CephNodeDiskspaceWarning index=4 msg="Evaluating rule failed" rule="alert: CephNodeDiskspaceWarning\nexpr: predict_linear(node_filesystem_free_bytes{device=~\"/.*\"}[2d], 3600 * 24 * 5)\n * on (instance) group_left (nodename) node_uname_info < 0\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.8.4\n severity: warning\n type: ceph_default\nannotations:\n description: Mountpoint {{ $labels.mountpoint }} on {{ $labels.nodename }} will\n be full in less than 5 days based on the 48 hour trailing fill rate.\n summary: Host filesystem free space is getting low\n" err="found duplicate series for the match group {instance=\"smithi100\"} on the right hand-side of the operation: [{__name__=\"node_uname_info\", domainname=\"(none)\", instance=\"smithi100\", job=\"node\", machine=\"x86_64\", nodename=\"smithi100\", release=\"5.15.0-101-generic\", sysname=\"Linux\", version=\"#111-Ubuntu SMP Tue Mar 5 20:16:58 UTC 2024\"}, {__name__=\"node_uname_info\", cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", domainname=\"(none)\", instance=\"smithi100\", job=\"node\", machine=\"x86_64\", nodename=\"smithi100\", release=\"5.15.0-101-generic\", sysname=\"Linux\", version=\"#111-Ubuntu SMP Tue Mar 5 20:16:58 UTC 2024\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:21:03.812 INFO:tasks.rados.rados.0.smithi100.stdout:1322: done (0 left) 2024-04-07T16:21:03.813 INFO:tasks.rados.rados.0.smithi100.stdout:1323: snap_create 2024-04-07T16:21:04.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:03 smithi144 bash[23522]: cluster 2024-04-07T16:21:02.615195+0000 mgr.y (mgr.25175) 43 : cluster 0 pgmap v63: 121 pgs: 121 active+clean; 238 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 2.9 MiB/s rd, 1.9 MiB/s wr, 21 op/s 2024-04-07T16:21:04.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:03 smithi144 bash[23522]: cluster 2024-04-07T16:21:02.812160+0000 mon.a (mon.0) 155 : cluster 0 osdmap e318: 8 total, 8 up, 8 in 2024-04-07T16:21:04.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:03 smithi144 bash[23522]: audit 2024-04-07T16:21:03.635639+0000 mon.a (mon.0) 156 : audit 1 from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:21:04.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:03 smithi144 bash[23522]: audit 2024-04-07T16:21:03.647185+0000 mon.a (mon.0) 157 : audit 1 from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:21:04.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:03 smithi144 bash[23522]: audit 2024-04-07T16:21:03.649722+0000 mon.a (mon.0) 158 : audit 0 from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:21:04.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:03 smithi144 bash[23522]: audit 2024-04-07T16:21:03.651487+0000 mon.a (mon.0) 159 : audit 1 from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:21:04.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:03 smithi144 bash[23522]: audit 2024-04-07T16:21:03.661799+0000 mon.a (mon.0) 160 : audit 1 from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:21:04.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:03 smithi144 bash[23522]: audit 2024-04-07T16:21:03.759009+0000 mon.a (mon.0) 161 : audit 0 from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:21:04.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:03 smithi144 bash[23522]: audit 2024-04-07T16:21:03.764342+0000 mon.a (mon.0) 162 : audit 0 from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:21:04.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:03 smithi144 bash[23522]: audit 2024-04-07T16:21:03.767799+0000 mon.a (mon.0) 163 : audit 0 from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-04-07T16:21:04.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:03 smithi144 bash[23522]: audit 2024-04-07T16:21:03.769727+0000 mon.a (mon.0) 164 : audit 0 from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon ok-to-stop", "ids": ["b"]}]: dispatch 2024-04-07T16:21:04.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:03 smithi100 bash[74942]: cluster 2024-04-07T16:21:02.615195+0000 mgr.y (mgr.25175) 43 : cluster [DBG] pgmap v63: 121 pgs: 121 active+clean; 238 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 2.9 MiB/s rd, 1.9 MiB/s wr, 21 op/s 2024-04-07T16:21:04.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:03 smithi100 bash[74942]: cluster 2024-04-07T16:21:02.812160+0000 mon.a (mon.0) 155 : cluster [DBG] osdmap e318: 8 total, 8 up, 8 in 2024-04-07T16:21:04.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:03 smithi100 bash[74942]: audit 2024-04-07T16:21:03.635639+0000 mon.a (mon.0) 156 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:21:04.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:03 smithi100 bash[74942]: audit 2024-04-07T16:21:03.647185+0000 mon.a (mon.0) 157 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:21:04.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:03 smithi100 bash[74942]: audit 2024-04-07T16:21:03.649722+0000 mon.a (mon.0) 158 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:21:04.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:03 smithi100 bash[74942]: audit 2024-04-07T16:21:03.651487+0000 mon.a (mon.0) 159 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:21:04.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:03 smithi100 bash[74942]: audit 2024-04-07T16:21:03.661799+0000 mon.a (mon.0) 160 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:21:04.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:03 smithi100 bash[74942]: audit 2024-04-07T16:21:03.759009+0000 mon.a (mon.0) 161 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:21:04.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:03 smithi100 bash[74942]: audit 2024-04-07T16:21:03.764342+0000 mon.a (mon.0) 162 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:21:04.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:03 smithi100 bash[74942]: audit 2024-04-07T16:21:03.767799+0000 mon.a (mon.0) 163 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-04-07T16:21:04.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:03 smithi100 bash[74942]: audit 2024-04-07T16:21:03.769727+0000 mon.a (mon.0) 164 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon ok-to-stop", "ids": ["b"]}]: dispatch 2024-04-07T16:21:04.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:03 smithi100 bash[76202]: cluster 2024-04-07T16:21:02.615195+0000 mgr.y (mgr.25175) 43 : cluster [DBG] pgmap v63: 121 pgs: 121 active+clean; 238 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 2.9 MiB/s rd, 1.9 MiB/s wr, 21 op/s 2024-04-07T16:21:04.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:03 smithi100 bash[76202]: cluster 2024-04-07T16:21:02.812160+0000 mon.a (mon.0) 155 : cluster [DBG] osdmap e318: 8 total, 8 up, 8 in 2024-04-07T16:21:04.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:03 smithi100 bash[76202]: audit 2024-04-07T16:21:03.635639+0000 mon.a (mon.0) 156 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:21:04.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:03 smithi100 bash[76202]: audit 2024-04-07T16:21:03.647185+0000 mon.a (mon.0) 157 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:21:04.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:03 smithi100 bash[76202]: audit 2024-04-07T16:21:03.649722+0000 mon.a (mon.0) 158 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:21:04.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:03 smithi100 bash[76202]: audit 2024-04-07T16:21:03.651487+0000 mon.a (mon.0) 159 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:21:04.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:03 smithi100 bash[76202]: audit 2024-04-07T16:21:03.661799+0000 mon.a (mon.0) 160 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:21:04.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:03 smithi100 bash[76202]: audit 2024-04-07T16:21:03.759009+0000 mon.a (mon.0) 161 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:21:04.188 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:03 smithi100 bash[76202]: audit 2024-04-07T16:21:03.764342+0000 mon.a (mon.0) 162 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:21:04.188 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:03 smithi100 bash[76202]: audit 2024-04-07T16:21:03.767799+0000 mon.a (mon.0) 163 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-04-07T16:21:04.188 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:03 smithi100 bash[76202]: audit 2024-04-07T16:21:03.769727+0000 mon.a (mon.0) 164 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon ok-to-stop", "ids": ["b"]}]: dispatch 2024-04-07T16:21:04.825 INFO:tasks.rados.rados.0.smithi100.stdout:1323: done (0 left) 2024-04-07T16:21:04.825 INFO:tasks.rados.rados.0.smithi100.stdout:1324: rollback oid 2 current snap is 125 2024-04-07T16:21:04.825 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 2 to 106 2024-04-07T16:21:04.825 INFO:tasks.rados.rados.0.smithi100.stdout:1325: read oid 21 snap -1 2024-04-07T16:21:04.825 INFO:tasks.rados.rados.0.smithi100.stdout:1325: expect (ObjNum 391 snap 119 seq_num 391) 2024-04-07T16:21:04.826 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 44 current snap is 125 2024-04-07T16:21:04.826 INFO:tasks.rados.rados.0.smithi100.stdout:1326: seq_num 410 ranges {532480=475136,1007616=417792,1425408=475136,1900544=425984,2326528=286720} 2024-04-07T16:21:04.843 INFO:tasks.rados.rados.0.smithi100.stdout:1326: writing smithi10066867-44 from 532480 to 1007616 tid 1 2024-04-07T16:21:04.850 INFO:tasks.rados.rados.0.smithi100.stdout:1326: writing smithi10066867-44 from 1007616 to 1425408 tid 2 2024-04-07T16:21:04.857 INFO:tasks.rados.rados.0.smithi100.stdout:1326: writing smithi10066867-44 from 1425408 to 1900544 tid 3 2024-04-07T16:21:04.866 INFO:tasks.rados.rados.0.smithi100.stdout:1326: writing smithi10066867-44 from 1900544 to 2326528 tid 4 2024-04-07T16:21:04.870 INFO:tasks.rados.rados.0.smithi100.stdout:1326: writing smithi10066867-44 from 2326528 to 2613248 tid 5 2024-04-07T16:21:04.870 INFO:tasks.rados.rados.0.smithi100.stdout:1327: delete oid 1 current snap is 125 2024-04-07T16:21:04.870 INFO:tasks.rados.rados.0.smithi100.stdout:1325: expect (ObjNum 391 snap 119 seq_num 391) 2024-04-07T16:21:04.922 INFO:tasks.rados.rados.0.smithi100.stdout:1324: finishing rollback tid 1 to smithi10066867-2 2024-04-07T16:21:04.923 INFO:tasks.rados.rados.0.smithi100.stdout:1324: finishing rollback tid 2 to smithi10066867-2 2024-04-07T16:21:04.923 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 2 v 689 (ObjNum 260 snap 70 seq_num 260) dirty exists 2024-04-07T16:21:04.923 INFO:tasks.rados.rados.0.smithi100.stdout:1326: finishing write tid 1 to smithi10066867-44 2024-04-07T16:21:04.923 INFO:tasks.rados.rados.0.smithi100.stdout:1326: finishing write tid 2 to smithi10066867-44 2024-04-07T16:21:04.923 INFO:tasks.rados.rados.0.smithi100.stdout:1326: finishing write tid 3 to smithi10066867-44 2024-04-07T16:21:04.923 INFO:tasks.rados.rados.0.smithi100.stdout:1324: done (3 left) 2024-04-07T16:21:04.923 INFO:tasks.rados.rados.0.smithi100.stdout:1325: done (2 left) 2024-04-07T16:21:04.923 INFO:tasks.rados.rados.0.smithi100.stdout:1327: done (1 left) 2024-04-07T16:21:04.923 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 29 current snap is 125 2024-04-07T16:21:04.923 INFO:tasks.rados.rados.0.smithi100.stdout:1326: finishing write tid 4 to smithi10066867-44 2024-04-07T16:21:04.923 INFO:tasks.rados.rados.0.smithi100.stdout:1328: seq_num 412 ranges {1761280=507904,2269184=630784,2899968=491520,3391488=483328} 2024-04-07T16:21:04.940 INFO:tasks.rados.rados.0.smithi100.stdout:1328: writing smithi10066867-29 from 1761280 to 2269184 tid 1 2024-04-07T16:21:04.945 INFO:tasks.rados.rados.0.smithi100.stdout:1328: writing smithi10066867-29 from 2269184 to 2899968 tid 2 2024-04-07T16:21:04.950 INFO:tasks.rados.rados.0.smithi100.stdout:1328: writing smithi10066867-29 from 2899968 to 3391488 tid 3 2024-04-07T16:21:04.956 INFO:tasks.rados.rados.0.smithi100.stdout:1328: writing smithi10066867-29 from 3391488 to 3874816 tid 4 2024-04-07T16:21:04.956 INFO:tasks.rados.rados.0.smithi100.stdout:1329: snap_remove snap 118 2024-04-07T16:21:05.105 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:04 smithi144 bash[23522]: cephadm 2024-04-07T16:21:03.771315+0000 mgr.y (mgr.25175) 44 : cephadm 1 Upgrade: It appears safe to stop mon.b 2024-04-07T16:21:05.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:04 smithi144 bash[23522]: cluster 2024-04-07T16:21:03.827109+0000 mon.a (mon.0) 165 : cluster 0 osdmap e319: 8 total, 8 up, 8 in 2024-04-07T16:21:05.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:04 smithi100 bash[74942]: cephadm 2024-04-07T16:21:03.771315+0000 mgr.y (mgr.25175) 44 : cephadm [INF] Upgrade: It appears safe to stop mon.b 2024-04-07T16:21:05.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:04 smithi100 bash[74942]: cluster 2024-04-07T16:21:03.827109+0000 mon.a (mon.0) 165 : cluster [DBG] osdmap e319: 8 total, 8 up, 8 in 2024-04-07T16:21:05.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:04 smithi100 bash[76202]: cephadm 2024-04-07T16:21:03.771315+0000 mgr.y (mgr.25175) 44 : cephadm [INF] Upgrade: It appears safe to stop mon.b 2024-04-07T16:21:05.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:04 smithi100 bash[76202]: cluster 2024-04-07T16:21:03.827109+0000 mon.a (mon.0) 165 : cluster [DBG] osdmap e319: 8 total, 8 up, 8 in 2024-04-07T16:21:05.832 INFO:tasks.rados.rados.0.smithi100.stdout:1329: done (2 left) 2024-04-07T16:21:05.833 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 39 current snap is 125 2024-04-07T16:21:05.833 INFO:tasks.rados.rados.0.smithi100.stdout:1326: finishing write tid 5 to smithi10066867-44 2024-04-07T16:21:05.833 INFO:tasks.rados.rados.0.smithi100.stdout:1326: finishing write tid 6 to smithi10066867-44 2024-04-07T16:21:05.833 INFO:tasks.rados.rados.0.smithi100.stdout:1330: seq_num 413 ranges {892928=524288,1417216=516096,1933312=696320,2629632=393216} 2024-04-07T16:21:05.847 INFO:tasks.rados.rados.0.smithi100.stdout:1330: writing smithi10066867-39 from 892928 to 1417216 tid 1 2024-04-07T16:21:05.853 INFO:tasks.rados.rados.0.smithi100.stdout:1330: writing smithi10066867-39 from 1417216 to 1933312 tid 2 2024-04-07T16:21:05.862 INFO:tasks.rados.rados.0.smithi100.stdout:1330: writing smithi10066867-39 from 1933312 to 2629632 tid 3 2024-04-07T16:21:05.868 INFO:tasks.rados.rados.0.smithi100.stdout:1330: writing smithi10066867-39 from 2629632 to 3022848 tid 4 2024-04-07T16:21:05.869 INFO:tasks.rados.rados.0.smithi100.stdout:1331: setattr oid 47 current snap is 125 2024-04-07T16:21:05.869 INFO:tasks.rados.rados.0.smithi100.stdout:1326: finishing write tid 7 to smithi10066867-44 2024-04-07T16:21:05.869 INFO:tasks.rados.rados.0.smithi100.stdout:1326: oid 44 updating version 0 to 694 2024-04-07T16:21:05.869 INFO:tasks.rados.rados.0.smithi100.stdout:1326: oid 44 version 694 is already newer than 691 2024-04-07T16:21:05.869 INFO:tasks.rados.rados.0.smithi100.stdout:1326: oid 44 version 694 is already newer than 692 2024-04-07T16:21:05.869 INFO:tasks.rados.rados.0.smithi100.stdout:1326: oid 44 version 694 is already newer than 690 2024-04-07T16:21:05.869 INFO:tasks.rados.rados.0.smithi100.stdout:1326: oid 44 version 694 is already newer than 693 2024-04-07T16:21:05.869 INFO:tasks.rados.rados.0.smithi100.stdout:1326: oid 44 updating version 694 to 695 2024-04-07T16:21:05.869 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 44 v 695 (ObjNum 410 snap 125 seq_num 410) dirty exists 2024-04-07T16:21:05.869 INFO:tasks.rados.rados.0.smithi100.stdout:1326: left oid 44 (ObjNum 410 snap 125 seq_num 410) 2024-04-07T16:21:05.869 INFO:tasks.rados.rados.0.smithi100.stdout:1328: finishing write tid 1 to smithi10066867-29 2024-04-07T16:21:05.869 INFO:tasks.rados.rados.0.smithi100.stdout:1328: finishing write tid 2 to smithi10066867-29 2024-04-07T16:21:05.869 INFO:tasks.rados.rados.0.smithi100.stdout:1328: finishing write tid 3 to smithi10066867-29 2024-04-07T16:21:05.869 INFO:tasks.rados.rados.0.smithi100.stdout:1328: finishing write tid 4 to smithi10066867-29 2024-04-07T16:21:05.870 INFO:tasks.rados.rados.0.smithi100.stdout:1328: finishing write tid 5 to smithi10066867-29 2024-04-07T16:21:05.870 INFO:tasks.rados.rados.0.smithi100.stdout:1328: finishing write tid 6 to smithi10066867-29 2024-04-07T16:21:05.870 INFO:tasks.rados.rados.0.smithi100.stdout:1328: oid 29 updating version 0 to 545 2024-04-07T16:21:05.870 INFO:tasks.rados.rados.0.smithi100.stdout:1328: oid 29 version 545 is already newer than 544 2024-04-07T16:21:05.870 INFO:tasks.rados.rados.0.smithi100.stdout:1328: oid 29 updating version 545 to 546 2024-04-07T16:21:05.870 INFO:tasks.rados.rados.0.smithi100.stdout:1328: oid 29 version 546 is already newer than 543 2024-04-07T16:21:05.870 INFO:tasks.rados.rados.0.smithi100.stdout:1328: oid 29 updating version 546 to 547 2024-04-07T16:21:05.870 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 29 v 547 (ObjNum 412 snap 125 seq_num 412) dirty exists 2024-04-07T16:21:05.870 INFO:tasks.rados.rados.0.smithi100.stdout:1328: left oid 29 (ObjNum 412 snap 125 seq_num 412) 2024-04-07T16:21:05.870 INFO:tasks.rados.rados.0.smithi100.stdout:1330: finishing write tid 1 to smithi10066867-39 2024-04-07T16:21:05.873 INFO:tasks.rados.rados.0.smithi100.stdout:1326: done (3 left) 2024-04-07T16:21:05.873 INFO:tasks.rados.rados.0.smithi100.stdout:1328: done (2 left) 2024-04-07T16:21:05.873 INFO:tasks.rados.rados.0.smithi100.stdout:1332: copy_from oid 46 from oid 9 current snap is 125 2024-04-07T16:21:05.874 INFO:tasks.rados.rados.0.smithi100.stdout:1333: snap_create 2024-04-07T16:21:05.874 INFO:tasks.rados.rados.0.smithi100.stdout:1330: finishing write tid 2 to smithi10066867-39 2024-04-07T16:21:05.876 INFO:tasks.rados.rados.0.smithi100.stdout:1330: finishing write tid 3 to smithi10066867-39 2024-04-07T16:21:05.876 INFO:tasks.rados.rados.0.smithi100.stdout:1330: finishing write tid 4 to smithi10066867-39 2024-04-07T16:21:05.877 INFO:tasks.rados.rados.0.smithi100.stdout:1330: finishing write tid 5 to smithi10066867-39 2024-04-07T16:21:05.879 INFO:tasks.rados.rados.0.smithi100.stdout:1330: finishing write tid 6 to smithi10066867-39 2024-04-07T16:21:05.879 INFO:tasks.rados.rados.0.smithi100.stdout:1330: oid 39 updating version 0 to 555 2024-04-07T16:21:05.879 INFO:tasks.rados.rados.0.smithi100.stdout:1330: oid 39 version 555 is already newer than 554 2024-04-07T16:21:05.879 INFO:tasks.rados.rados.0.smithi100.stdout:1330: oid 39 version 555 is already newer than 549 2024-04-07T16:21:05.879 INFO:tasks.rados.rados.0.smithi100.stdout:1330: oid 39 version 555 is already newer than 553 2024-04-07T16:21:05.879 INFO:tasks.rados.rados.0.smithi100.stdout:1330: oid 39 version 555 is already newer than 552 2024-04-07T16:21:05.879 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 39 v 555 (ObjNum 413 snap 125 seq_num 413) dirty exists 2024-04-07T16:21:05.879 INFO:tasks.rados.rados.0.smithi100.stdout:1330: left oid 39 (ObjNum 413 snap 125 seq_num 413) 2024-04-07T16:21:05.880 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 47 v 526 (ObjNum 129 snap 0 seq_num 2702497792) dirty exists 2024-04-07T16:21:05.882 INFO:tasks.rados.rados.0.smithi100.stdout:1332: finishing copy_from to smithi10066867-46 2024-04-07T16:21:05.882 INFO:tasks.rados.rados.0.smithi100.stdout:1332: got expected ENOENT (src dne) 2024-04-07T16:21:05.883 INFO:tasks.rados.rados.0.smithi100.stdout:1332: finishing copy_from racing read to smithi10066867-46 2024-04-07T16:21:06.105 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:05 smithi144 bash[23522]: cluster 2024-04-07T16:21:04.615946+0000 mgr.y (mgr.25175) 45 : cluster 0 pgmap v66: 121 pgs: 121 active+clean; 238 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 2.9 MiB/s rd, 1.9 MiB/s wr, 21 op/s 2024-04-07T16:21:06.105 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:05 smithi144 bash[23522]: cluster 2024-04-07T16:21:04.837857+0000 mon.a (mon.0) 166 : cluster 0 osdmap e320: 8 total, 8 up, 8 in 2024-04-07T16:21:06.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:05 smithi100 bash[74942]: cluster 2024-04-07T16:21:04.615946+0000 mgr.y (mgr.25175) 45 : cluster [DBG] pgmap v66: 121 pgs: 121 active+clean; 238 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 2.9 MiB/s rd, 1.9 MiB/s wr, 21 op/s 2024-04-07T16:21:06.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:05 smithi100 bash[74942]: cluster 2024-04-07T16:21:04.837857+0000 mon.a (mon.0) 166 : cluster [DBG] osdmap e320: 8 total, 8 up, 8 in 2024-04-07T16:21:06.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:05 smithi100 bash[76202]: cluster 2024-04-07T16:21:04.615946+0000 mgr.y (mgr.25175) 45 : cluster [DBG] pgmap v66: 121 pgs: 121 active+clean; 238 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 2.9 MiB/s rd, 1.9 MiB/s wr, 21 op/s 2024-04-07T16:21:06.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:05 smithi100 bash[76202]: cluster 2024-04-07T16:21:04.837857+0000 mon.a (mon.0) 166 : cluster [DBG] osdmap e320: 8 total, 8 up, 8 in 2024-04-07T16:21:06.748 INFO:tasks.rados.rados.0.smithi100.stdout:1330: done (3 left) 2024-04-07T16:21:06.748 INFO:tasks.rados.rados.0.smithi100.stdout:1331: done (2 left) 2024-04-07T16:21:06.748 INFO:tasks.rados.rados.0.smithi100.stdout:1332: done (1 left) 2024-04-07T16:21:06.748 INFO:tasks.rados.rados.0.smithi100.stdout:1333: done (0 left) 2024-04-07T16:21:06.748 INFO:tasks.rados.rados.0.smithi100.stdout:1334: snap_remove snap 122 2024-04-07T16:21:07.085 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:06 smithi144 bash[23522]: cluster 2024-04-07T16:21:05.848595+0000 mon.a (mon.0) 167 : cluster 0 osdmap e321: 8 total, 8 up, 8 in 2024-04-07T16:21:07.085 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:06 smithi144 bash[23522]: cephadm 2024-04-07T16:21:05.944462+0000 mgr.y (mgr.25175) 46 : cephadm 1 Upgrade: Updating mon.b 2024-04-07T16:21:07.085 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:06 smithi144 bash[23522]: audit 2024-04-07T16:21:05.952078+0000 mon.a (mon.0) 168 : audit 1 from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:21:07.085 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:06 smithi144 bash[23522]: audit 2024-04-07T16:21:05.956626+0000 mon.a (mon.0) 169 : audit 1 from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-07T16:21:07.085 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:06 smithi144 bash[23522]: audit 2024-04-07T16:21:05.958205+0000 mon.a (mon.0) 170 : audit 0 from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-07T16:21:07.085 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:06 smithi144 bash[23522]: audit 2024-04-07T16:21:05.959750+0000 mon.a (mon.0) 171 : audit 0 from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:21:07.085 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:06 smithi144 bash[23522]: cephadm 2024-04-07T16:21:05.961847+0000 mgr.y (mgr.25175) 47 : cephadm 1 Deploying daemon mon.b on smithi144 2024-04-07T16:21:07.085 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:06 smithi144 bash[23522]: cluster 2024-04-07T16:21:06.764129+0000 mon.a (mon.0) 172 : cluster 0 osdmap e322: 8 total, 8 up, 8 in 2024-04-07T16:21:07.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:06 smithi100 bash[74942]: cluster 2024-04-07T16:21:05.848595+0000 mon.a (mon.0) 167 : cluster [DBG] osdmap e321: 8 total, 8 up, 8 in 2024-04-07T16:21:07.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:06 smithi100 bash[74942]: cephadm 2024-04-07T16:21:05.944462+0000 mgr.y (mgr.25175) 46 : cephadm [INF] Upgrade: Updating mon.b 2024-04-07T16:21:07.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:06 smithi100 bash[74942]: audit 2024-04-07T16:21:05.952078+0000 mon.a (mon.0) 168 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:21:07.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:06 smithi100 bash[74942]: audit 2024-04-07T16:21:05.956626+0000 mon.a (mon.0) 169 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-07T16:21:07.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:06 smithi100 bash[74942]: audit 2024-04-07T16:21:05.958205+0000 mon.a (mon.0) 170 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-07T16:21:07.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:06 smithi100 bash[74942]: audit 2024-04-07T16:21:05.959750+0000 mon.a (mon.0) 171 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:21:07.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:06 smithi100 bash[74942]: cephadm 2024-04-07T16:21:05.961847+0000 mgr.y (mgr.25175) 47 : cephadm [INF] Deploying daemon mon.b on smithi144 2024-04-07T16:21:07.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:06 smithi100 bash[74942]: cluster 2024-04-07T16:21:06.764129+0000 mon.a (mon.0) 172 : cluster [DBG] osdmap e322: 8 total, 8 up, 8 in 2024-04-07T16:21:07.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:06 smithi100 bash[76202]: cluster 2024-04-07T16:21:05.848595+0000 mon.a (mon.0) 167 : cluster [DBG] osdmap e321: 8 total, 8 up, 8 in 2024-04-07T16:21:07.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:06 smithi100 bash[76202]: cephadm 2024-04-07T16:21:05.944462+0000 mgr.y (mgr.25175) 46 : cephadm [INF] Upgrade: Updating mon.b 2024-04-07T16:21:07.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:06 smithi100 bash[76202]: audit 2024-04-07T16:21:05.952078+0000 mon.a (mon.0) 168 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:21:07.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:06 smithi100 bash[76202]: audit 2024-04-07T16:21:05.956626+0000 mon.a (mon.0) 169 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-07T16:21:07.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:06 smithi100 bash[76202]: audit 2024-04-07T16:21:05.958205+0000 mon.a (mon.0) 170 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-07T16:21:07.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:06 smithi100 bash[76202]: audit 2024-04-07T16:21:05.959750+0000 mon.a (mon.0) 171 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:21:07.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:06 smithi100 bash[76202]: cephadm 2024-04-07T16:21:05.961847+0000 mgr.y (mgr.25175) 47 : cephadm [INF] Deploying daemon mon.b on smithi144 2024-04-07T16:21:07.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:06 smithi100 bash[76202]: cluster 2024-04-07T16:21:06.764129+0000 mon.a (mon.0) 172 : cluster [DBG] osdmap e322: 8 total, 8 up, 8 in 2024-04-07T16:21:07.752 INFO:tasks.rados.rados.0.smithi100.stdout:1334: done (0 left) 2024-04-07T16:21:07.752 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 8 current snap is 126 2024-04-07T16:21:07.752 INFO:tasks.rados.rados.0.smithi100.stdout:1335: seq_num 414 ranges {0=540672,540672=802816,1343488=458752,1802240=344064} 2024-04-07T16:21:07.760 INFO:tasks.rados.rados.0.smithi100.stdout:1335: writing smithi10066867-8 from 0 to 540672 tid 1 2024-04-07T16:21:07.773 INFO:tasks.rados.rados.0.smithi100.stdout:1335: writing smithi10066867-8 from 540672 to 1343488 tid 2 2024-04-07T16:21:07.779 INFO:tasks.rados.rados.0.smithi100.stdout:1335: writing smithi10066867-8 from 1343488 to 1802240 tid 3 2024-04-07T16:21:07.785 INFO:tasks.rados.rados.0.smithi100.stdout:1335: writing smithi10066867-8 from 1802240 to 2146304 tid 4 2024-04-07T16:21:07.785 INFO:tasks.rados.rados.0.smithi100.stdout:1336: rollback oid 31 current snap is 126 2024-04-07T16:21:07.786 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 31 to 123 2024-04-07T16:21:07.786 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 20 current snap is 126 2024-04-07T16:21:07.786 INFO:tasks.rados.rados.0.smithi100.stdout:1337: seq_num 415 ranges {1212416=557056,1769472=688128,2457600=663552,3121152=253952} 2024-04-07T16:21:07.806 INFO:tasks.rados.rados.0.smithi100.stdout:1337: writing smithi10066867-20 from 1212416 to 1769472 tid 1 2024-04-07T16:21:07.813 INFO:tasks.rados.rados.0.smithi100.stdout:1337: writing smithi10066867-20 from 1769472 to 2457600 tid 2 2024-04-07T16:21:07.820 INFO:tasks.rados.rados.0.smithi100.stdout:1337: writing smithi10066867-20 from 2457600 to 3121152 tid 3 2024-04-07T16:21:07.823 INFO:tasks.rados.rados.0.smithi100.stdout:1337: writing smithi10066867-20 from 3121152 to 3375104 tid 4 2024-04-07T16:21:07.823 INFO:tasks.rados.rados.0.smithi100.stdout:1338: read oid 41 snap -1 2024-04-07T16:21:07.823 INFO:tasks.rados.rados.0.smithi100.stdout:1338: expect (ObjNum 402 snap 121 seq_num 402) 2024-04-07T16:21:07.823 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 50 current snap is 126 2024-04-07T16:21:07.823 INFO:tasks.rados.rados.0.smithi100.stdout:1335: finishing write tid 1 to smithi10066867-8 2024-04-07T16:21:07.823 INFO:tasks.rados.rados.0.smithi100.stdout:1335: finishing write tid 2 to smithi10066867-8 2024-04-07T16:21:07.823 INFO:tasks.rados.rados.0.smithi100.stdout:1335: finishing write tid 3 to smithi10066867-8 2024-04-07T16:21:07.823 INFO:tasks.rados.rados.0.smithi100.stdout:1335: finishing write tid 4 to smithi10066867-8 2024-04-07T16:21:07.823 INFO:tasks.rados.rados.0.smithi100.stdout:1335: finishing write tid 5 to smithi10066867-8 2024-04-07T16:21:07.824 INFO:tasks.rados.rados.0.smithi100.stdout:1336: finishing rollback tid 0 to smithi10066867-31 2024-04-07T16:21:07.824 INFO:tasks.rados.rados.0.smithi100.stdout:1336: finishing rollback tid 1 to smithi10066867-31 2024-04-07T16:21:07.824 INFO:tasks.rados.rados.0.smithi100.stdout:1335: finishing write tid 6 to smithi10066867-8 2024-04-07T16:21:07.824 INFO:tasks.rados.rados.0.smithi100.stdout:1335: oid 8 updating version 0 to 569 2024-04-07T16:21:07.824 INFO:tasks.rados.rados.0.smithi100.stdout:1335: oid 8 version 569 is already newer than 567 2024-04-07T16:21:07.824 INFO:tasks.rados.rados.0.smithi100.stdout:1335: oid 8 version 569 is already newer than 566 2024-04-07T16:21:07.824 INFO:tasks.rados.rados.0.smithi100.stdout:1335: oid 8 updating version 569 to 570 2024-04-07T16:21:07.824 INFO:tasks.rados.rados.0.smithi100.stdout:1335: oid 8 version 570 is already newer than 568 2024-04-07T16:21:07.824 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 8 v 570 (ObjNum 414 snap 126 seq_num 414) dirty exists 2024-04-07T16:21:07.824 INFO:tasks.rados.rados.0.smithi100.stdout:1335: left oid 8 (ObjNum 414 snap 126 seq_num 414) 2024-04-07T16:21:07.824 INFO:tasks.rados.rados.0.smithi100.stdout:1336: finishing rollback tid 2 to smithi10066867-31 2024-04-07T16:21:07.824 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 31 v 558 (ObjNum 406 snap 123 seq_num 406) dirty exists 2024-04-07T16:21:07.824 INFO:tasks.rados.rados.0.smithi100.stdout:1337: finishing write tid 1 to smithi10066867-20 2024-04-07T16:21:07.824 INFO:tasks.rados.rados.0.smithi100.stdout:1337: finishing write tid 2 to smithi10066867-20 2024-04-07T16:21:07.824 INFO:tasks.rados.rados.0.smithi100.stdout:1339: seq_num 416 ranges {0=573440,573440=573440,1146880=475136,1622016=557056} 2024-04-07T16:21:07.828 INFO:tasks.rados.rados.0.smithi100.stdout:1339: writing smithi10066867-50 from 0 to 573440 tid 1 2024-04-07T16:21:07.833 INFO:tasks.rados.rados.0.smithi100.stdout:1339: writing smithi10066867-50 from 573440 to 1146880 tid 2 2024-04-07T16:21:07.838 INFO:tasks.rados.rados.0.smithi100.stdout:1339: writing smithi10066867-50 from 1146880 to 1622016 tid 3 2024-04-07T16:21:07.842 INFO:tasks.rados.rados.0.smithi100.stdout:1339: writing smithi10066867-50 from 1622016 to 2179072 tid 4 2024-04-07T16:21:07.843 INFO:tasks.rados.rados.0.smithi100.stdout:1335: done (4 left) 2024-04-07T16:21:07.843 INFO:tasks.rados.rados.0.smithi100.stdout:1336: done (3 left) 2024-04-07T16:21:07.843 INFO:tasks.rados.rados.0.smithi100.stdout:1340: rollback oid 47 current snap is 126 2024-04-07T16:21:07.843 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 47 to 124 2024-04-07T16:21:07.843 INFO:tasks.rados.rados.0.smithi100.stdout:1337: finishing write tid 3 to smithi10066867-20 2024-04-07T16:21:07.843 INFO:tasks.rados.rados.0.smithi100.stdout:1337: finishing write tid 4 to smithi10066867-20 2024-04-07T16:21:07.843 INFO:tasks.rados.rados.0.smithi100.stdout:1337: finishing write tid 5 to smithi10066867-20 2024-04-07T16:21:07.843 INFO:tasks.rados.rados.0.smithi100.stdout:1339: finishing write tid 1 to smithi10066867-50 2024-04-07T16:21:07.843 INFO:tasks.rados.rados.0.smithi100.stdout:1337: finishing write tid 6 to smithi10066867-20 2024-04-07T16:21:07.843 INFO:tasks.rados.rados.0.smithi100.stdout:1337: oid 20 updating version 0 to 524 2024-04-07T16:21:07.843 INFO:tasks.rados.rados.0.smithi100.stdout:1337: oid 20 updating version 524 to 527 2024-04-07T16:21:07.843 INFO:tasks.rados.rados.0.smithi100.stdout:1337: oid 20 version 527 is already newer than 525 2024-04-07T16:21:07.843 INFO:tasks.rados.rados.0.smithi100.stdout:1337: oid 20 version 527 is already newer than 526 2024-04-07T16:21:07.844 INFO:tasks.rados.rados.0.smithi100.stdout:1337: oid 20 updating version 527 to 528 2024-04-07T16:21:07.844 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 20 v 528 (ObjNum 415 snap 126 seq_num 415) dirty exists 2024-04-07T16:21:07.844 INFO:tasks.rados.rados.0.smithi100.stdout:1337: left oid 20 (ObjNum 415 snap 126 seq_num 415) 2024-04-07T16:21:07.844 INFO:tasks.rados.rados.0.smithi100.stdout:1338: expect (ObjNum 402 snap 121 seq_num 402) 2024-04-07T16:21:07.917 INFO:tasks.rados.rados.0.smithi100.stdout:1339: finishing write tid 2 to smithi10066867-50 2024-04-07T16:21:07.917 INFO:tasks.rados.rados.0.smithi100.stdout:1339: finishing write tid 3 to smithi10066867-50 2024-04-07T16:21:07.918 INFO:tasks.rados.rados.0.smithi100.stdout:1337: done (3 left) 2024-04-07T16:21:07.918 INFO:tasks.rados.rados.0.smithi100.stdout:1338: done (2 left) 2024-04-07T16:21:07.918 INFO:tasks.rados.rados.0.smithi100.stdout:1341: read oid 14 snap -1 2024-04-07T16:21:07.918 INFO:tasks.rados.rados.0.smithi100.stdout:1341: expect (ObjNum 342 snap 101 seq_num 342) 2024-04-07T16:21:07.918 INFO:tasks.rados.rados.0.smithi100.stdout:1342: rollback oid 15 current snap is 126 2024-04-07T16:21:07.918 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 15 to 123 2024-04-07T16:21:07.918 INFO:tasks.rados.rados.0.smithi100.stdout:1340: finishing rollback tid 0 to smithi10066867-47 2024-04-07T16:21:07.918 INFO:tasks.rados.rados.0.smithi100.stdout:1340: finishing rollback tid 1 to smithi10066867-47 2024-04-07T16:21:07.918 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 47 v 528 (ObjNum 33 snap 0 seq_num 2702798560) dirty dne 2024-04-07T16:21:07.918 INFO:tasks.rados.rados.0.smithi100.stdout:1339: finishing write tid 4 to smithi10066867-50 2024-04-07T16:21:07.918 INFO:tasks.rados.rados.0.smithi100.stdout:1339: finishing write tid 5 to smithi10066867-50 2024-04-07T16:21:07.918 INFO:tasks.rados.rados.0.smithi100.stdout:1339: finishing write tid 6 to smithi10066867-50 2024-04-07T16:21:07.918 INFO:tasks.rados.rados.0.smithi100.stdout:1339: oid 50 updating version 0 to 523 2024-04-07T16:21:07.918 INFO:tasks.rados.rados.0.smithi100.stdout:1339: oid 50 updating version 523 to 524 2024-04-07T16:21:07.919 INFO:tasks.rados.rados.0.smithi100.stdout:1339: oid 50 version 524 is already newer than 521 2024-04-07T16:21:07.919 INFO:tasks.rados.rados.0.smithi100.stdout:1339: oid 50 updating version 524 to 525 2024-04-07T16:21:07.919 INFO:tasks.rados.rados.0.smithi100.stdout:1339: oid 50 version 525 is already newer than 522 2024-04-07T16:21:07.919 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 50 v 525 (ObjNum 416 snap 126 seq_num 416) dirty exists 2024-04-07T16:21:07.919 INFO:tasks.rados.rados.0.smithi100.stdout:1339: left oid 50 (ObjNum 416 snap 126 seq_num 416) 2024-04-07T16:21:07.919 INFO:tasks.rados.rados.0.smithi100.stdout:1339: done (3 left) 2024-04-07T16:21:07.919 INFO:tasks.rados.rados.0.smithi100.stdout:1340: done (2 left) 2024-04-07T16:21:07.919 INFO:tasks.rados.rados.0.smithi100.stdout:1343: read oid 17 snap -1 2024-04-07T16:21:07.919 INFO:tasks.rados.rados.0.smithi100.stdout:1343: expect (ObjNum 395 snap 119 seq_num 395) 2024-04-07T16:21:07.919 INFO:tasks.rados.rados.0.smithi100.stdout:1344: rollback oid 28 current snap is 126 2024-04-07T16:21:07.919 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 28 to 119 2024-04-07T16:21:07.919 INFO:tasks.rados.rados.0.smithi100.stdout:1345: rollback oid 16 current snap is 126 2024-04-07T16:21:07.919 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 16 to 125 2024-04-07T16:21:07.919 INFO:tasks.rados.rados.0.smithi100.stdout:1346: rmattr oid 23 current snap is 126 2024-04-07T16:21:07.919 INFO:tasks.rados.rados.0.smithi100.stdout:1347: snap_remove snap 124 2024-04-07T16:21:08.162 INFO:journalctl@ceph.mds.b.smithi144.stdout:Apr 07 16:21:08 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:21:08.163 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:21:08 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:21:08.163 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 16:21:08 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:21:08.163 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:21:08 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:21:08.164 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:21:08 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:21:08.164 INFO:journalctl@ceph.osd.6.smithi144.stdout:Apr 07 16:21:08 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:21:08.164 INFO:journalctl@ceph.osd.4.smithi144.stdout:Apr 07 16:21:08 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:21:08.165 INFO:journalctl@ceph.osd.5.smithi144.stdout:Apr 07 16:21:08 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:21:08.165 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:07 smithi144 bash[23522]: cluster 2024-04-07T16:21:06.616615+0000 mgr.y (mgr.25175) 48 : cluster 0 pgmap v69: 121 pgs: 121 active+clean; 238 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail 2024-04-07T16:21:08.165 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:07 smithi144 bash[23522]: cluster 2024-04-07T16:21:07.767187+0000 mon.a (mon.0) 173 : cluster 0 osdmap e323: 8 total, 8 up, 8 in 2024-04-07T16:21:08.165 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:08 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:21:08.165 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:08 smithi144 systemd[1]: Stopping Ceph mon.b for 0702e3a8-f4f8-11ee-b649-cb9ed24678a4... 2024-04-07T16:21:08.165 INFO:journalctl@ceph.osd.7.smithi144.stdout:Apr 07 16:21:08 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:21:08.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:07 smithi100 bash[74942]: cluster 2024-04-07T16:21:06.616615+0000 mgr.y (mgr.25175) 48 : cluster [DBG] pgmap v69: 121 pgs: 121 active+clean; 238 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail 2024-04-07T16:21:08.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:07 smithi100 bash[74942]: cluster 2024-04-07T16:21:07.767187+0000 mon.a (mon.0) 173 : cluster [DBG] osdmap e323: 8 total, 8 up, 8 in 2024-04-07T16:21:08.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:07 smithi100 bash[76202]: cluster 2024-04-07T16:21:06.616615+0000 mgr.y (mgr.25175) 48 : cluster [DBG] pgmap v69: 121 pgs: 121 active+clean; 238 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail 2024-04-07T16:21:08.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:07 smithi100 bash[76202]: cluster 2024-04-07T16:21:07.767187+0000 mon.a (mon.0) 173 : cluster [DBG] osdmap e323: 8 total, 8 up, 8 in 2024-04-07T16:21:08.508 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:08 smithi144 bash[23522]: debug 2024-04-07T16:21:08.294+0000 7f27af88b700 -1 received signal: Terminated from /sbin/docker-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-stderr=true (PID: 1) UID: 0 2024-04-07T16:21:08.508 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:08 smithi144 bash[23522]: debug 2024-04-07T16:21:08.294+0000 7f27af88b700 -1 mon.b@1(peon) e3 *** Got Signal Terminated *** 2024-04-07T16:21:08.685 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:21:08 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:21:08] "GET /metrics HTTP/1.1" 200 36082 "" "Prometheus/2.43.0" 2024-04-07T16:21:09.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:08 smithi144 bash[66534]: ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4-mon-b 2024-04-07T16:21:09.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:09 smithi144 bash[66595]: Error response from daemon: No such container: ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4-mon-b 2024-04-07T16:21:09.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:09 smithi144 systemd[1]: ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@mon.b.service: Deactivated successfully. 2024-04-07T16:21:09.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:09 smithi144 systemd[1]: Stopped Ceph mon.b for 0702e3a8-f4f8-11ee-b649-cb9ed24678a4. 2024-04-07T16:21:09.605 INFO:journalctl@ceph.mds.b.smithi144.stdout:Apr 07 16:21:09 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:21:09.606 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:21:09 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:21:09.606 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:21:09 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:21:09.607 INFO:journalctl@ceph.osd.6.smithi144.stdout:Apr 07 16:21:09 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:21:09.607 INFO:journalctl@ceph.osd.7.smithi144.stdout:Apr 07 16:21:09 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:21:09.608 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:21:09 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:21:09.608 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 16:21:09 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:21:09.608 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:09 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:21:09.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:09 smithi144 systemd[1]: Started Ceph mon.b for 0702e3a8-f4f8-11ee-b649-cb9ed24678a4. 2024-04-07T16:21:09.609 INFO:journalctl@ceph.osd.4.smithi144.stdout:Apr 07 16:21:09 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:21:09.609 INFO:journalctl@ceph.osd.5.smithi144.stdout:Apr 07 16:21:09 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:21:10.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.214+0000 7fdc8746bc80 0 set uid:gid to 167:167 (ceph:ceph) 2024-04-07T16:21:10.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.214+0000 7fdc8746bc80 0 ceph version 19.0.0-2785-gadce92d0 (adce92d056533f22dc1ee37e3141d098b046ec1b) squid (dev), process ceph-mon, pid 7 2024-04-07T16:21:10.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.214+0000 7fdc8746bc80 0 pidfile_write: ignore empty --pid-file 2024-04-07T16:21:10.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 0 load: jerasure load: lrc 2024-04-07T16:21:10.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: RocksDB version: 7.9.2 2024-04-07T16:21:10.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Git sha 0 2024-04-07T16:21:10.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Compile date 2024-04-05 22:33:20 2024-04-07T16:21:10.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: DB SUMMARY 2024-04-07T16:21:10.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: DB Session ID: 2Z9PR1UQWRAIHLOV4UZO 2024-04-07T16:21:10.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: CURRENT file: CURRENT 2024-04-07T16:21:10.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: IDENTITY file: IDENTITY 2024-04-07T16:21:10.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: MANIFEST file: MANIFEST-000010 size: 2120 Bytes 2024-04-07T16:21:10.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 1, files: 000036.sst 2024-04-07T16:21:10.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000034.log size: 355211 ; 2024-04-07T16:21:10.608 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.error_if_exists: 0 2024-04-07T16:21:10.608 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.create_if_missing: 0 2024-04-07T16:21:10.608 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.paranoid_checks: 1 2024-04-07T16:21:10.608 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.flush_verify_memtable_count: 1 2024-04-07T16:21:10.608 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-07T16:21:10.608 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-07T16:21:10.608 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.env: 0x557e0cb91c60 2024-04-07T16:21:10.608 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.fs: PosixFileSystem 2024-04-07T16:21:10.608 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.info_log: 0x557e0e119c00 2024-04-07T16:21:10.608 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.max_file_opening_threads: 16 2024-04-07T16:21:10.608 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.statistics: (nil) 2024-04-07T16:21:10.608 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.use_fsync: 0 2024-04-07T16:21:10.608 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.max_log_file_size: 0 2024-04-07T16:21:10.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-07T16:21:10.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.log_file_time_to_roll: 0 2024-04-07T16:21:10.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.keep_log_file_num: 1000 2024-04-07T16:21:10.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.recycle_log_file_num: 0 2024-04-07T16:21:10.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.allow_fallocate: 1 2024-04-07T16:21:10.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.allow_mmap_reads: 0 2024-04-07T16:21:10.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.allow_mmap_writes: 0 2024-04-07T16:21:10.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.use_direct_reads: 0 2024-04-07T16:21:10.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-07T16:21:10.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.create_missing_column_families: 0 2024-04-07T16:21:10.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.db_log_dir: 2024-04-07T16:21:10.611 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.wal_dir: 2024-04-07T16:21:10.611 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.table_cache_numshardbits: 6 2024-04-07T16:21:10.611 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.WAL_ttl_seconds: 0 2024-04-07T16:21:10.611 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.WAL_size_limit_MB: 0 2024-04-07T16:21:10.611 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-07T16:21:10.611 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-07T16:21:10.611 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.is_fd_close_on_exec: 1 2024-04-07T16:21:10.611 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.advise_random_on_open: 1 2024-04-07T16:21:10.611 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.db_write_buffer_size: 0 2024-04-07T16:21:10.611 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.write_buffer_manager: 0x557e0e0ea820 2024-04-07T16:21:10.611 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-07T16:21:10.611 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-07T16:21:10.611 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.use_adaptive_mutex: 0 2024-04-07T16:21:10.612 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.rate_limiter: (nil) 2024-04-07T16:21:10.612 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-07T16:21:10.612 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.wal_recovery_mode: 2 2024-04-07T16:21:10.612 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.enable_thread_tracking: 0 2024-04-07T16:21:10.612 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.enable_pipelined_write: 0 2024-04-07T16:21:10.612 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.unordered_write: 0 2024-04-07T16:21:10.612 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-07T16:21:10.612 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-07T16:21:10.612 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-07T16:21:10.612 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-07T16:21:10.612 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.row_cache: None 2024-04-07T16:21:10.612 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.wal_filter: None 2024-04-07T16:21:10.612 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-07T16:21:10.612 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.allow_ingest_behind: 0 2024-04-07T16:21:10.612 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.two_write_queues: 0 2024-04-07T16:21:10.613 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.manual_wal_flush: 0 2024-04-07T16:21:10.613 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.wal_compression: 0 2024-04-07T16:21:10.613 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.atomic_flush: 0 2024-04-07T16:21:10.613 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-07T16:21:10.613 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.persist_stats_to_disk: 0 2024-04-07T16:21:10.613 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.write_dbid_to_manifest: 0 2024-04-07T16:21:10.613 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.log_readahead_size: 0 2024-04-07T16:21:10.613 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-07T16:21:10.613 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.best_efforts_recovery: 0 2024-04-07T16:21:10.613 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-07T16:21:10.613 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-07T16:21:10.613 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.allow_data_in_errors: 0 2024-04-07T16:21:10.613 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.db_host_id: __hostname__ 2024-04-07T16:21:10.613 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.enforce_single_del_contracts: true 2024-04-07T16:21:10.613 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.max_background_jobs: 2 2024-04-07T16:21:10.614 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.max_background_compactions: -1 2024-04-07T16:21:10.614 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.max_subcompactions: 1 2024-04-07T16:21:10.614 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-07T16:21:10.614 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-07T16:21:10.614 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.delayed_write_rate : 16777216 2024-04-07T16:21:10.614 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.max_total_wal_size: 0 2024-04-07T16:21:10.614 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-07T16:21:10.614 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.stats_dump_period_sec: 600 2024-04-07T16:21:10.614 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.stats_persist_period_sec: 600 2024-04-07T16:21:10.614 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-07T16:21:10.614 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.max_open_files: -1 2024-04-07T16:21:10.614 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.bytes_per_sync: 0 2024-04-07T16:21:10.614 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.wal_bytes_per_sync: 0 2024-04-07T16:21:10.614 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.strict_bytes_per_sync: 0 2024-04-07T16:21:10.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.compaction_readahead_size: 0 2024-04-07T16:21:10.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.max_background_flushes: -1 2024-04-07T16:21:10.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Compression algorithms supported: 2024-04-07T16:21:10.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-07T16:21:10.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: kZSTD supported: 0 2024-04-07T16:21:10.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: kXpressCompression supported: 0 2024-04-07T16:21:10.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: kLZ4HCCompression supported: 1 2024-04-07T16:21:10.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: kZlibCompression supported: 1 2024-04-07T16:21:10.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: kSnappyCompression supported: 1 2024-04-07T16:21:10.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: kLZ4Compression supported: 1 2024-04-07T16:21:10.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: kBZip2Compression supported: 0 2024-04-07T16:21:10.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Fast CRC32 supported: Supported on x86 2024-04-07T16:21:10.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: DMutex implementation: pthread_mutex_t 2024-04-07T16:21:10.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000010 2024-04-07T16:21:10.617 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-07T16:21:10.617 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-07T16:21:10.617 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.merge_operator: 2024-04-07T16:21:10.617 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.compaction_filter: None 2024-04-07T16:21:10.617 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.compaction_filter_factory: None 2024-04-07T16:21:10.617 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.sst_partitioner_factory: None 2024-04-07T16:21:10.617 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.memtable_factory: SkipListFactory 2024-04-07T16:21:10.617 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.table_factory: BlockBasedTable 2024-04-07T16:21:10.617 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x557e0e119de0) 2024-04-07T16:21:10.617 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: cache_index_and_filter_blocks: 1 2024-04-07T16:21:10.617 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-07T16:21:10.618 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-07T16:21:10.618 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: pin_top_level_index_and_filter: 1 2024-04-07T16:21:10.618 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: index_type: 0 2024-04-07T16:21:10.618 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: data_block_index_type: 0 2024-04-07T16:21:10.618 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: index_shortening: 1 2024-04-07T16:21:10.618 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: data_block_hash_table_util_ratio: 0.750000 2024-04-07T16:21:10.618 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: checksum: 4 2024-04-07T16:21:10.618 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: no_block_cache: 0 2024-04-07T16:21:10.618 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: block_cache: 0x557e0e11b090 2024-04-07T16:21:10.619 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: block_cache_name: BinnedLRUCache 2024-04-07T16:21:10.619 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: block_cache_options: 2024-04-07T16:21:10.619 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: capacity : 536870912 2024-04-07T16:21:10.619 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: num_shard_bits : 4 2024-04-07T16:21:10.619 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: strict_capacity_limit : 0 2024-04-07T16:21:10.619 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: high_pri_pool_ratio: 0.000 2024-04-07T16:21:10.620 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: block_cache_compressed: (nil) 2024-04-07T16:21:10.620 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: persistent_cache: (nil) 2024-04-07T16:21:10.620 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: block_size: 4096 2024-04-07T16:21:10.620 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: block_size_deviation: 10 2024-04-07T16:21:10.620 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: block_restart_interval: 16 2024-04-07T16:21:10.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: index_block_restart_interval: 1 2024-04-07T16:21:10.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: metadata_block_size: 4096 2024-04-07T16:21:10.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: partition_filters: 0 2024-04-07T16:21:10.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: use_delta_encoding: 1 2024-04-07T16:21:10.621 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: filter_policy: bloomfilter 2024-04-07T16:21:10.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: whole_key_filtering: 1 2024-04-07T16:21:10.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: verify_compression: 0 2024-04-07T16:21:10.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: read_amp_bytes_per_bit: 0 2024-04-07T16:21:10.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: format_version: 5 2024-04-07T16:21:10.622 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: enable_index_compression: 1 2024-04-07T16:21:10.623 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: block_align: 0 2024-04-07T16:21:10.623 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: max_auto_readahead_size: 262144 2024-04-07T16:21:10.623 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: prepopulate_block_cache: 0 2024-04-07T16:21:10.623 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: initial_auto_readahead_size: 8192 2024-04-07T16:21:10.623 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: num_file_reads_for_auto_readahead: 2 2024-04-07T16:21:10.623 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.write_buffer_size: 33554432 2024-04-07T16:21:10.623 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.max_write_buffer_number: 2 2024-04-07T16:21:10.624 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.compression: NoCompression 2024-04-07T16:21:10.624 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.bottommost_compression: Disabled 2024-04-07T16:21:10.624 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.prefix_extractor: nullptr 2024-04-07T16:21:10.624 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-07T16:21:10.624 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.num_levels: 7 2024-04-07T16:21:10.624 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-07T16:21:10.624 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-07T16:21:10.624 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-07T16:21:10.625 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-07T16:21:10.625 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-07T16:21:10.625 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-07T16:21:10.625 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-07T16:21:10.625 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-07T16:21:10.625 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-07T16:21:10.625 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-07T16:21:10.625 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-07T16:21:10.626 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-07T16:21:10.626 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.compression_opts.window_bits: -14 2024-04-07T16:21:10.626 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.compression_opts.level: 32767 2024-04-07T16:21:10.626 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.compression_opts.strategy: 0 2024-04-07T16:21:10.626 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-07T16:21:10.626 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-07T16:21:10.626 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-07T16:21:10.627 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-07T16:21:10.627 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.compression_opts.enabled: false 2024-04-07T16:21:10.627 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-07T16:21:10.627 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-07T16:21:10.627 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-07T16:21:10.627 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-07T16:21:10.627 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.target_file_size_base: 67108864 2024-04-07T16:21:10.628 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.target_file_size_multiplier: 1 2024-04-07T16:21:10.628 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-07T16:21:10.628 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-07T16:21:10.628 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-07T16:21:10.628 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-07T16:21:10.628 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-07T16:21:10.628 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-07T16:21:10.628 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-07T16:21:10.629 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-07T16:21:10.629 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-07T16:21:10.629 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-07T16:21:10.629 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-07T16:21:10.629 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-07T16:21:10.629 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-07T16:21:10.629 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.arena_block_size: 1048576 2024-04-07T16:21:10.630 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-07T16:21:10.630 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-07T16:21:10.630 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.disable_auto_compactions: 0 2024-04-07T16:21:10.630 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-07T16:21:10.630 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-07T16:21:10.630 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-07T16:21:10.630 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-07T16:21:10.630 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-07T16:21:10.631 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-07T16:21:10.631 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-07T16:21:10.631 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-07T16:21:10.631 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-07T16:21:10.631 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-07T16:21:10.631 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-07T16:21:10.631 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.inplace_update_support: 0 2024-04-07T16:21:10.632 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.inplace_update_num_locks: 10000 2024-04-07T16:21:10.632 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-07T16:21:10.632 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-07T16:21:10.632 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.memtable_huge_page_size: 0 2024-04-07T16:21:10.632 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.bloom_locality: 0 2024-04-07T16:21:10.632 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.max_successive_merges: 0 2024-04-07T16:21:10.632 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.optimize_filters_for_hits: 0 2024-04-07T16:21:10.632 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.paranoid_file_checks: 0 2024-04-07T16:21:10.633 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.force_consistency_checks: 1 2024-04-07T16:21:10.633 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.report_bg_io_stats: 0 2024-04-07T16:21:10.633 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.ttl: 2592000 2024-04-07T16:21:10.633 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.periodic_compaction_seconds: 0 2024-04-07T16:21:10.633 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-07T16:21:10.633 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-07T16:21:10.633 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.enable_blob_files: false 2024-04-07T16:21:10.633 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.min_blob_size: 0 2024-04-07T16:21:10.634 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.blob_file_size: 268435456 2024-04-07T16:21:10.634 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.blob_compression_type: NoCompression 2024-04-07T16:21:10.634 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.enable_blob_garbage_collection: false 2024-04-07T16:21:10.634 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-07T16:21:10.634 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-07T16:21:10.634 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-07T16:21:10.635 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.blob_file_starting_level: 0 2024-04-07T16:21:10.635 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-07T16:21:10.635 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-04-07T16:21:10.635 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 38, last_sequence is 21393, log_number is 34,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 34 2024-04-07T16:21:10.635 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 34 2024-04-07T16:21:10.635 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.218+0000 7fdc8746bc80 4 rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: aaaa2253-d796-4d76-9a21-61c166965630 2024-04-07T16:21:10.635 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.222+0000 7fdc8746bc80 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1712506870224276, "job": 1, "event": "recovery_started", "wal_files": [34]} 2024-04-07T16:21:10.635 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.222+0000 7fdc8746bc80 4 rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #34 mode 2 2024-04-07T16:21:10.635 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.222+0000 7fdc8746bc80 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1712506870226230, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 39, "file_size": 235438, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 21398, "largest_seqno": 21590, "table_properties": {"data_size": 233475, "index_size": 669, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 261, "raw_key_size": 2436, "raw_average_key_size": 25, "raw_value_size": 231130, "raw_average_value_size": 2432, "num_data_blocks": 28, "num_entries": 95, "num_filter_entries": 95, "num_deletions": 1, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1712506870, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "aaaa2253-d796-4d76-9a21-61c166965630", "db_session_id": "2Z9PR1UQWRAIHLOV4UZO", "orig_file_number": 39, "seqno_to_time_mapping": "N/A"}} 2024-04-07T16:21:10.635 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.222+0000 7fdc8746bc80 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1712506870226348, "job": 1, "event": "recovery_finished"} 2024-04-07T16:21:10.635 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.222+0000 7fdc8746bc80 4 rocksdb: [db/version_set.cc:5047] Creating manifest 41 2024-04-07T16:21:10.636 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.222+0000 7fdc8746bc80 4 rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-04-07T16:21:10.636 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.222+0000 7fdc8746bc80 4 rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000034.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-07T16:21:10.636 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.222+0000 7fdc8746bc80 4 rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x557e0e20e000 2024-04-07T16:21:10.636 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.222+0000 7fdc8746bc80 4 rocksdb: DB pointer 0x557e0e1f8000 2024-04-07T16:21:10.636 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.226+0000 7fdc75bc5700 4 rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-07T16:21:10.636 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.226+0000 7fdc75bc5700 4 rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-07T16:21:10.636 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: ** DB Stats ** 2024-04-07T16:21:10.636 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: Uptime(secs): 0.0 total, 0.0 interval 2024-04-07T16:21:10.637 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-07T16:21:10.637 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-07T16:21:10.637 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-07T16:21:10.637 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-07T16:21:10.637 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-07T16:21:10.637 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-07T16:21:10.637 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: ** Compaction Stats [default] ** 2024-04-07T16:21:10.637 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-07T16:21:10.638 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-07T16:21:10.638 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: L0 1/0 229.92 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 180.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-07T16:21:10.638 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: L6 1/0 13.25 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 2024-04-07T16:21:10.638 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: Sum 2/0 13.47 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 180.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-07T16:21:10.638 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 180.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-07T16:21:10.638 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: ** Compaction Stats [default] ** 2024-04-07T16:21:10.638 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-07T16:21:10.638 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-07T16:21:10.638 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 180.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-07T16:21:10.638 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-07T16:21:10.639 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: Uptime(secs): 0.0 total, 0.0 interval 2024-04-07T16:21:10.639 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: Flush(GB): cumulative 0.000, interval 0.000 2024-04-07T16:21:10.639 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-07T16:21:10.639 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: AddFile(Total Files): cumulative 0, interval 0 2024-04-07T16:21:10.639 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: AddFile(L0 Files): cumulative 0, interval 0 2024-04-07T16:21:10.639 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: AddFile(Keys): cumulative 0, interval 0 2024-04-07T16:21:10.639 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: Cumulative compaction: 0.00 GB write, 45.50 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-07T16:21:10.639 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: Interval compaction: 0.00 GB write, 45.50 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-07T16:21:10.639 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-07T16:21:10.639 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: Block cache BinnedLRUCache@0x557e0e11b090#7 capacity: 512.00 MB usage: 1.08 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.9e-05 secs_since: 0 2024-04-07T16:21:10.639 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: Block cache entry stats(count,size,portion): FilterBlock(1,0.31 KB,5.96046e-05%) IndexBlock(1,0.77 KB,0.000146031%) Misc(1,0.00 KB,0%) 2024-04-07T16:21:10.639 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: ** File Read Latency Histogram By Level [default] ** 2024-04-07T16:21:10.639 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.226+0000 7fdc8746bc80 0 starting mon.b rank 1 at public addrs [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] at bind addrs [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid 0702e3a8-f4f8-11ee-b649-cb9ed24678a4 2024-04-07T16:21:10.639 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.226+0000 7fdc8746bc80 1 mon.b@-1(???) e3 preinit fsid 0702e3a8-f4f8-11ee-b649-cb9ed24678a4 2024-04-07T16:21:10.640 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.226+0000 7fdc8746bc80 0 mon.b@-1(???).mds e6 new map 2024-04-07T16:21:10.640 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.226+0000 7fdc8746bc80 0 mon.b@-1(???).mds e6 print_map 2024-04-07T16:21:10.640 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: e6 2024-04-07T16:21:10.640 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: enable_multiple, ever_enabled_multiple: 1,1 2024-04-07T16:21:10.640 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-07T16:21:10.640 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: legacy client fscid: 1 2024-04-07T16:21:10.640 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: 2024-04-07T16:21:10.640 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: Filesystem 'cephfs' (1) 2024-04-07T16:21:10.640 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: fs_name cephfs 2024-04-07T16:21:10.640 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: epoch 6 2024-04-07T16:21:10.640 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-07T16:21:10.640 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: created 2024-04-07T16:12:13.365189+0000 2024-04-07T16:21:10.640 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: modified 2024-04-07T16:12:15.375075+0000 2024-04-07T16:21:10.640 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: tableserver 0 2024-04-07T16:21:10.641 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: root 0 2024-04-07T16:21:10.641 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: session_timeout 60 2024-04-07T16:21:10.641 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: session_autoclose 300 2024-04-07T16:21:10.641 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: max_file_size 1099511627776 2024-04-07T16:21:10.641 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: max_xattr_size 65536 2024-04-07T16:21:10.641 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: required_client_features {} 2024-04-07T16:21:10.641 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: last_failure 0 2024-04-07T16:21:10.641 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: last_failure_osd_epoch 0 2024-04-07T16:21:10.641 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-07T16:21:10.641 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: max_mds 1 2024-04-07T16:21:10.641 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: in 0 2024-04-07T16:21:10.641 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: up {0=24322} 2024-04-07T16:21:10.641 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: failed 2024-04-07T16:21:10.641 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: damaged 2024-04-07T16:21:10.641 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: stopped 2024-04-07T16:21:10.642 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: data_pools [3] 2024-04-07T16:21:10.642 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: metadata_pool 2 2024-04-07T16:21:10.642 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: inline_data disabled 2024-04-07T16:21:10.642 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: balancer 2024-04-07T16:21:10.642 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: bal_rank_mask -1 2024-04-07T16:21:10.642 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: standby_count_wanted 0 2024-04-07T16:21:10.642 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: qdb_cluster leader: 0 members: 2024-04-07T16:21:10.642 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: [mds.b{0:24322} state up:active seq 3 addr [v2:172.21.15.144:6832/226244023,v1:172.21.15.144:6833/226244023] compat {c=[1],r=[1],i=[7ff]}] 2024-04-07T16:21:10.642 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: 2024-04-07T16:21:10.642 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: 2024-04-07T16:21:10.642 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: Standby daemons: 2024-04-07T16:21:10.642 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: 2024-04-07T16:21:10.642 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: [mds.a{-1:14424} state up:standby seq 1 addr [v2:172.21.15.100:6834/3487006602,v1:172.21.15.100:6835/3487006602] compat {c=[1],r=[1],i=[7ff]}] 2024-04-07T16:21:10.642 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.226+0000 7fdc8746bc80 0 mon.b@-1(???).osd e323 crush map has features 3314933069573799936, adjusting msgr requires 2024-04-07T16:21:10.643 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.226+0000 7fdc8746bc80 0 mon.b@-1(???).osd e323 crush map has features 432629308056666112, adjusting msgr requires 2024-04-07T16:21:10.643 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.226+0000 7fdc8746bc80 0 mon.b@-1(???).osd e323 crush map has features 432629308056666112, adjusting msgr requires 2024-04-07T16:21:10.643 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.226+0000 7fdc8746bc80 0 mon.b@-1(???).osd e323 crush map has features 432629308056666112, adjusting msgr requires 2024-04-07T16:21:10.643 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:10 smithi144 bash[66654]: debug 2024-04-07T16:21:10.226+0000 7fdc8746bc80 1 mon.b@-1(???).paxosservice(auth 1..22) refresh upgraded, format 0 -> 3 2024-04-07T16:21:11.834 INFO:tasks.rados.rados.0.smithi100.stdout:1347: done (6 left) 2024-04-07T16:21:11.834 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 1 current snap is 126 2024-04-07T16:21:11.834 INFO:tasks.rados.rados.0.smithi100.stdout:1348: seq_num 417 ranges {0=589824,589824=458752,1048576=679936,1728512=466944} 2024-04-07T16:21:11.844 INFO:tasks.rados.rados.0.smithi100.stdout:1348: writing smithi10066867-1 from 0 to 589824 tid 1 2024-04-07T16:21:11.852 INFO:tasks.rados.rados.0.smithi100.stdout:1348: writing smithi10066867-1 from 589824 to 1048576 tid 2 2024-04-07T16:21:11.865 INFO:tasks.rados.rados.0.smithi100.stdout:1348: writing smithi10066867-1 from 1048576 to 1728512 tid 3 2024-04-07T16:21:11.874 INFO:tasks.rados.rados.0.smithi100.stdout:1348: writing smithi10066867-1 from 1728512 to 2195456 tid 4 2024-04-07T16:21:11.874 INFO:tasks.rados.rados.0.smithi100.stdout:1349: read oid 13 snap 119 2024-04-07T16:21:11.874 INFO:tasks.rados.rados.0.smithi100.stdout:1349: expect (ObjNum 393 snap 119 seq_num 393) 2024-04-07T16:21:11.874 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 2 current snap is 126 2024-04-07T16:21:11.874 INFO:tasks.rados.rados.0.smithi100.stdout:1345: finishing rollback tid 0 to smithi10066867-16 2024-04-07T16:21:11.874 INFO:tasks.rados.rados.0.smithi100.stdout:1345: finishing rollback tid 1 to smithi10066867-16 2024-04-07T16:21:11.874 INFO:tasks.rados.rados.0.smithi100.stdout:1344: finishing rollback tid 0 to smithi10066867-28 2024-04-07T16:21:11.874 INFO:tasks.rados.rados.0.smithi100.stdout:1345: finishing rollback tid 2 to smithi10066867-16 2024-04-07T16:21:11.875 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 16 v 734 (ObjNum 374 snap 113 seq_num 374) dirty exists 2024-04-07T16:21:11.875 INFO:tasks.rados.rados.0.smithi100.stdout:1341: expect (ObjNum 342 snap 101 seq_num 342) 2024-04-07T16:21:11.932 INFO:tasks.rados.rados.0.smithi100.stdout:1342: finishing rollback tid 0 to smithi10066867-15 2024-04-07T16:21:11.932 INFO:tasks.rados.rados.0.smithi100.stdout:1342: finishing rollback tid 1 to smithi10066867-15 2024-04-07T16:21:11.932 INFO:tasks.rados.rados.0.smithi100.stdout:1342: finishing rollback tid 2 to smithi10066867-15 2024-04-07T16:21:11.932 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 15 v 871 (ObjNum 338 snap 101 seq_num 338) dirty exists 2024-04-07T16:21:11.933 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 23 v 873 (ObjNum 395 snap 119 seq_num 395) dirty exists 2024-04-07T16:21:11.933 INFO:tasks.rados.rados.0.smithi100.stdout:1344: finishing rollback tid 1 to smithi10066867-28 2024-04-07T16:21:11.933 INFO:tasks.rados.rados.0.smithi100.stdout:1350: seq_num 418 ranges {1966080=606208,2572288=737280,3309568=434176,3743744=434176} 2024-04-07T16:21:11.961 INFO:tasks.rados.rados.0.smithi100.stdout:1350: writing smithi10066867-2 from 1966080 to 2572288 tid 1 2024-04-07T16:21:11.969 INFO:tasks.rados.rados.0.smithi100.stdout:1350: writing smithi10066867-2 from 2572288 to 3309568 tid 2 2024-04-07T16:21:11.973 INFO:tasks.rados.rados.0.smithi100.stdout:1350: writing smithi10066867-2 from 3309568 to 3743744 tid 3 2024-04-07T16:21:11.978 INFO:tasks.rados.rados.0.smithi100.stdout:1350: writing smithi10066867-2 from 3743744 to 4177920 tid 4 2024-04-07T16:21:11.978 INFO:tasks.rados.rados.0.smithi100.stdout:1341: done (8 left) 2024-04-07T16:21:11.978 INFO:tasks.rados.rados.0.smithi100.stdout:1342: done (7 left) 2024-04-07T16:21:11.979 INFO:tasks.rados.rados.0.smithi100.stdout:1345: done (6 left) 2024-04-07T16:21:11.979 INFO:tasks.rados.rados.0.smithi100.stdout:1346: done (5 left) 2024-04-07T16:21:11.979 INFO:tasks.rados.rados.0.smithi100.stdout:1351: copy_from oid 45 from oid 41 current snap is 126 2024-04-07T16:21:11.979 INFO:tasks.rados.rados.0.smithi100.stdout:1352: snap_remove snap 123 2024-04-07T16:21:12.606 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:21:12 smithi144 bash[59794]: ts=2024-04-07T16:21:12.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:21:12.876 INFO:tasks.rados.rados.0.smithi100.stdout:1352: done (6 left) 2024-04-07T16:21:12.876 INFO:tasks.rados.rados.0.smithi100.stdout:1344: finishing rollback tid 2 to smithi10066867-28 2024-04-07T16:21:12.876 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 28 v 207 (ObjNum 283 snap 81 seq_num 283) dirty exists 2024-04-07T16:21:12.876 INFO:tasks.rados.rados.0.smithi100.stdout:1343: expect (ObjNum 395 snap 119 seq_num 395) 2024-04-07T16:21:13.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:12 smithi144 bash[66654]: cluster 2024-04-07T16:21:11.814412+0000 mon.a (mon.0) 182 : cluster [DBG] 0: [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] mon.a 2024-04-07T16:21:13.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:12 smithi144 bash[66654]: cluster 2024-04-07T16:21:11.814446+0000 mon.a (mon.0) 183 : cluster [DBG] 1: [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] mon.b 2024-04-07T16:21:13.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:12 smithi144 bash[66654]: cluster 2024-04-07T16:21:11.814475+0000 mon.a (mon.0) 184 : cluster [DBG] 2: [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] mon.c 2024-04-07T16:21:13.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:12 smithi144 bash[66654]: cluster 2024-04-07T16:21:11.836507+0000 mon.a (mon.0) 185 : cluster [DBG] fsmap cephfs:1 {0=b=up:active} 1 up:standby 2024-04-07T16:21:13.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:12 smithi144 bash[66654]: cluster 2024-04-07T16:21:11.836588+0000 mon.a (mon.0) 186 : cluster [DBG] osdmap e324: 8 total, 8 up, 8 in 2024-04-07T16:21:13.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:12 smithi144 bash[66654]: cluster 2024-04-07T16:21:11.837221+0000 mon.a (mon.0) 187 : cluster [DBG] mgrmap e41: y(active, since 55s), standbys: x 2024-04-07T16:21:13.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:12 smithi144 bash[66654]: cluster 2024-04-07T16:21:11.837508+0000 mon.a (mon.0) 188 : cluster [INF] overall HEALTH_OK 2024-04-07T16:21:13.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:12 smithi144 bash[66654]: cluster 2024-04-07T16:21:11.842224+0000 mon.b (mon.1) 2 : cluster [INF] mon.b calling monitor election 2024-04-07T16:21:13.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:12 smithi144 bash[66654]: cluster 2024-04-07T16:21:11.848773+0000 mon.c (mon.2) 2 : cluster [INF] mon.c calling monitor election 2024-04-07T16:21:13.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:12 smithi144 bash[66654]: cluster 2024-04-07T16:21:11.848845+0000 mon.a (mon.0) 189 : cluster [INF] mon.a calling monitor election 2024-04-07T16:21:13.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:12 smithi144 bash[66654]: audit 2024-04-07T16:21:11.851562+0000 mon.a (mon.0) 190 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:21:13.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:12 smithi144 bash[66654]: audit 2024-04-07T16:21:11.851912+0000 mon.a (mon.0) 191 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:21:13.108 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:12 smithi144 bash[66654]: audit 2024-04-07T16:21:11.852312+0000 mon.a (mon.0) 192 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:21:13.108 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:12 smithi144 bash[66654]: cluster 2024-04-07T16:21:11.855638+0000 mon.a (mon.0) 193 : cluster [INF] mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-04-07T16:21:13.108 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:12 smithi144 bash[66654]: cluster 2024-04-07T16:21:11.862730+0000 mon.a (mon.0) 194 : cluster [DBG] monmap epoch 4 2024-04-07T16:21:13.108 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:12 smithi144 bash[66654]: cluster 2024-04-07T16:21:11.862769+0000 mon.a (mon.0) 195 : cluster [DBG] fsid 0702e3a8-f4f8-11ee-b649-cb9ed24678a4 2024-04-07T16:21:13.108 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:12 smithi144 bash[66654]: cluster 2024-04-07T16:21:11.862792+0000 mon.a (mon.0) 196 : cluster [DBG] last_changed 2024-04-07T16:21:11.814093+0000 2024-04-07T16:21:13.108 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:12 smithi144 bash[66654]: cluster 2024-04-07T16:21:11.862847+0000 mon.a (mon.0) 197 : cluster [DBG] created 2024-04-07T16:02:17.280630+0000 2024-04-07T16:21:13.108 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:12 smithi144 bash[66654]: cluster 2024-04-07T16:21:11.862893+0000 mon.a (mon.0) 198 : cluster [DBG] min_mon_release 19 (squid) 2024-04-07T16:21:13.108 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:12 smithi144 bash[66654]: cluster 2024-04-07T16:21:11.862936+0000 mon.a (mon.0) 199 : cluster [DBG] election_strategy: 1 2024-04-07T16:21:13.108 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:12 smithi144 bash[66654]: cluster 2024-04-07T16:21:11.862979+0000 mon.a (mon.0) 200 : cluster [DBG] 0: [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] mon.a 2024-04-07T16:21:13.108 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:12 smithi144 bash[66654]: cluster 2024-04-07T16:21:11.863027+0000 mon.a (mon.0) 201 : cluster [DBG] 1: [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] mon.b 2024-04-07T16:21:13.108 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:12 smithi144 bash[66654]: cluster 2024-04-07T16:21:11.863071+0000 mon.a (mon.0) 202 : cluster [DBG] 2: [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] mon.c 2024-04-07T16:21:13.108 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:12 smithi144 bash[66654]: cluster 2024-04-07T16:21:11.864608+0000 mon.a (mon.0) 203 : cluster [DBG] fsmap cephfs:1 {0=b=up:active} 1 up:standby 2024-04-07T16:21:13.108 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:12 smithi144 bash[66654]: cluster 2024-04-07T16:21:11.864710+0000 mon.a (mon.0) 204 : cluster [DBG] osdmap e324: 8 total, 8 up, 8 in 2024-04-07T16:21:13.108 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:12 smithi144 bash[66654]: cluster 2024-04-07T16:21:11.865622+0000 mon.a (mon.0) 205 : cluster [DBG] mgrmap e41: y(active, since 55s), standbys: x 2024-04-07T16:21:13.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:12 smithi144 bash[66654]: cluster 2024-04-07T16:21:11.865949+0000 mon.a (mon.0) 206 : cluster [INF] overall HEALTH_OK 2024-04-07T16:21:13.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:12 smithi144 bash[66654]: audit 2024-04-07T16:21:11.875455+0000 mon.a (mon.0) 207 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:21:13.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:12 smithi144 bash[66654]: audit 2024-04-07T16:21:11.888454+0000 mon.a (mon.0) 208 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:21:13.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:12 smithi144 bash[66654]: audit 2024-04-07T16:21:11.895096+0000 mon.a (mon.0) 209 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:21:13.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:12 smithi100 bash[74942]: cluster 2024-04-07T16:21:11.814412+0000 mon.a (mon.0) 182 : cluster [DBG] 0: [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] mon.a 2024-04-07T16:21:13.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:12 smithi100 bash[74942]: cluster 2024-04-07T16:21:11.814446+0000 mon.a (mon.0) 183 : cluster [DBG] 1: [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] mon.b 2024-04-07T16:21:13.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:12 smithi100 bash[74942]: cluster 2024-04-07T16:21:11.814475+0000 mon.a (mon.0) 184 : cluster [DBG] 2: [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] mon.c 2024-04-07T16:21:13.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:12 smithi100 bash[74942]: cluster 2024-04-07T16:21:11.836507+0000 mon.a (mon.0) 185 : cluster [DBG] fsmap cephfs:1 {0=b=up:active} 1 up:standby 2024-04-07T16:21:13.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:12 smithi100 bash[74942]: cluster 2024-04-07T16:21:11.836588+0000 mon.a (mon.0) 186 : cluster [DBG] osdmap e324: 8 total, 8 up, 8 in 2024-04-07T16:21:13.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:12 smithi100 bash[74942]: cluster 2024-04-07T16:21:11.837221+0000 mon.a (mon.0) 187 : cluster [DBG] mgrmap e41: y(active, since 55s), standbys: x 2024-04-07T16:21:13.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:12 smithi100 bash[74942]: cluster 2024-04-07T16:21:11.837508+0000 mon.a (mon.0) 188 : cluster [INF] overall HEALTH_OK 2024-04-07T16:21:13.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:12 smithi100 bash[74942]: cluster 2024-04-07T16:21:11.842224+0000 mon.b (mon.1) 2 : cluster [INF] mon.b calling monitor election 2024-04-07T16:21:13.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:12 smithi100 bash[74942]: cluster 2024-04-07T16:21:11.848773+0000 mon.c (mon.2) 2 : cluster [INF] mon.c calling monitor election 2024-04-07T16:21:13.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:12 smithi100 bash[74942]: cluster 2024-04-07T16:21:11.848845+0000 mon.a (mon.0) 189 : cluster [INF] mon.a calling monitor election 2024-04-07T16:21:13.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:12 smithi100 bash[74942]: audit 2024-04-07T16:21:11.851562+0000 mon.a (mon.0) 190 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:21:13.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:12 smithi100 bash[74942]: audit 2024-04-07T16:21:11.851912+0000 mon.a (mon.0) 191 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:21:13.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:12 smithi100 bash[74942]: audit 2024-04-07T16:21:11.852312+0000 mon.a (mon.0) 192 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:21:13.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:12 smithi100 bash[74942]: cluster 2024-04-07T16:21:11.855638+0000 mon.a (mon.0) 193 : cluster [INF] mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-04-07T16:21:13.188 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:12 smithi100 bash[74942]: cluster 2024-04-07T16:21:11.862730+0000 mon.a (mon.0) 194 : cluster [DBG] monmap epoch 4 2024-04-07T16:21:13.188 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:12 smithi100 bash[74942]: cluster 2024-04-07T16:21:11.862769+0000 mon.a (mon.0) 195 : cluster [DBG] fsid 0702e3a8-f4f8-11ee-b649-cb9ed24678a4 2024-04-07T16:21:13.188 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:12 smithi100 bash[74942]: cluster 2024-04-07T16:21:11.862792+0000 mon.a (mon.0) 196 : cluster [DBG] last_changed 2024-04-07T16:21:11.814093+0000 2024-04-07T16:21:13.188 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:12 smithi100 bash[74942]: cluster 2024-04-07T16:21:11.862847+0000 mon.a (mon.0) 197 : cluster [DBG] created 2024-04-07T16:02:17.280630+0000 2024-04-07T16:21:13.188 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:12 smithi100 bash[74942]: cluster 2024-04-07T16:21:11.862893+0000 mon.a (mon.0) 198 : cluster [DBG] min_mon_release 19 (squid) 2024-04-07T16:21:13.188 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:12 smithi100 bash[74942]: cluster 2024-04-07T16:21:11.862936+0000 mon.a (mon.0) 199 : cluster [DBG] election_strategy: 1 2024-04-07T16:21:13.188 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:12 smithi100 bash[74942]: cluster 2024-04-07T16:21:11.862979+0000 mon.a (mon.0) 200 : cluster [DBG] 0: [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] mon.a 2024-04-07T16:21:13.188 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:12 smithi100 bash[74942]: cluster 2024-04-07T16:21:11.863027+0000 mon.a (mon.0) 201 : cluster [DBG] 1: [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] mon.b 2024-04-07T16:21:13.188 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:12 smithi100 bash[74942]: cluster 2024-04-07T16:21:11.863071+0000 mon.a (mon.0) 202 : cluster [DBG] 2: [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] mon.c 2024-04-07T16:21:13.188 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:12 smithi100 bash[74942]: cluster 2024-04-07T16:21:11.864608+0000 mon.a (mon.0) 203 : cluster [DBG] fsmap cephfs:1 {0=b=up:active} 1 up:standby 2024-04-07T16:21:13.188 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:12 smithi100 bash[74942]: cluster 2024-04-07T16:21:11.864710+0000 mon.a (mon.0) 204 : cluster [DBG] osdmap e324: 8 total, 8 up, 8 in 2024-04-07T16:21:13.188 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:12 smithi100 bash[74942]: cluster 2024-04-07T16:21:11.865622+0000 mon.a (mon.0) 205 : cluster [DBG] mgrmap e41: y(active, since 55s), standbys: x 2024-04-07T16:21:13.188 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:12 smithi100 bash[74942]: cluster 2024-04-07T16:21:11.865949+0000 mon.a (mon.0) 206 : cluster [INF] overall HEALTH_OK 2024-04-07T16:21:13.188 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:12 smithi100 bash[74942]: audit 2024-04-07T16:21:11.875455+0000 mon.a (mon.0) 207 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:21:13.188 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:12 smithi100 bash[74942]: audit 2024-04-07T16:21:11.888454+0000 mon.a (mon.0) 208 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:21:13.189 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:12 smithi100 bash[74942]: audit 2024-04-07T16:21:11.895096+0000 mon.a (mon.0) 209 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:21:13.189 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:12 smithi100 bash[76202]: cluster 2024-04-07T16:21:11.814412+0000 mon.a (mon.0) 182 : cluster [DBG] 0: [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] mon.a 2024-04-07T16:21:13.189 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:12 smithi100 bash[76202]: cluster 2024-04-07T16:21:11.814446+0000 mon.a (mon.0) 183 : cluster [DBG] 1: [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] mon.b 2024-04-07T16:21:13.189 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:12 smithi100 bash[76202]: cluster 2024-04-07T16:21:11.814475+0000 mon.a (mon.0) 184 : cluster [DBG] 2: [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] mon.c 2024-04-07T16:21:13.189 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:12 smithi100 bash[76202]: cluster 2024-04-07T16:21:11.836507+0000 mon.a (mon.0) 185 : cluster [DBG] fsmap cephfs:1 {0=b=up:active} 1 up:standby 2024-04-07T16:21:13.189 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:12 smithi100 bash[76202]: cluster 2024-04-07T16:21:11.836588+0000 mon.a (mon.0) 186 : cluster [DBG] osdmap e324: 8 total, 8 up, 8 in 2024-04-07T16:21:13.189 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:12 smithi100 bash[76202]: cluster 2024-04-07T16:21:11.837221+0000 mon.a (mon.0) 187 : cluster [DBG] mgrmap e41: y(active, since 55s), standbys: x 2024-04-07T16:21:13.189 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:12 smithi100 bash[76202]: cluster 2024-04-07T16:21:11.837508+0000 mon.a (mon.0) 188 : cluster [INF] overall HEALTH_OK 2024-04-07T16:21:13.189 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:12 smithi100 bash[76202]: cluster 2024-04-07T16:21:11.842224+0000 mon.b (mon.1) 2 : cluster [INF] mon.b calling monitor election 2024-04-07T16:21:13.189 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:12 smithi100 bash[76202]: cluster 2024-04-07T16:21:11.848773+0000 mon.c (mon.2) 2 : cluster [INF] mon.c calling monitor election 2024-04-07T16:21:13.189 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:12 smithi100 bash[76202]: cluster 2024-04-07T16:21:11.848845+0000 mon.a (mon.0) 189 : cluster [INF] mon.a calling monitor election 2024-04-07T16:21:13.190 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:12 smithi100 bash[76202]: audit 2024-04-07T16:21:11.851562+0000 mon.a (mon.0) 190 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T16:21:13.190 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:12 smithi100 bash[76202]: audit 2024-04-07T16:21:11.851912+0000 mon.a (mon.0) 191 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T16:21:13.190 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:12 smithi100 bash[76202]: audit 2024-04-07T16:21:11.852312+0000 mon.a (mon.0) 192 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T16:21:13.190 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:12 smithi100 bash[76202]: cluster 2024-04-07T16:21:11.855638+0000 mon.a (mon.0) 193 : cluster [INF] mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-04-07T16:21:13.190 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:12 smithi100 bash[76202]: cluster 2024-04-07T16:21:11.862730+0000 mon.a (mon.0) 194 : cluster [DBG] monmap epoch 4 2024-04-07T16:21:13.190 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:12 smithi100 bash[76202]: cluster 2024-04-07T16:21:11.862769+0000 mon.a (mon.0) 195 : cluster [DBG] fsid 0702e3a8-f4f8-11ee-b649-cb9ed24678a4 2024-04-07T16:21:13.190 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:12 smithi100 bash[76202]: cluster 2024-04-07T16:21:11.862792+0000 mon.a (mon.0) 196 : cluster [DBG] last_changed 2024-04-07T16:21:11.814093+0000 2024-04-07T16:21:13.190 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:12 smithi100 bash[76202]: cluster 2024-04-07T16:21:11.862847+0000 mon.a (mon.0) 197 : cluster [DBG] created 2024-04-07T16:02:17.280630+0000 2024-04-07T16:21:13.190 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:12 smithi100 bash[76202]: cluster 2024-04-07T16:21:11.862893+0000 mon.a (mon.0) 198 : cluster [DBG] min_mon_release 19 (squid) 2024-04-07T16:21:13.190 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:12 smithi100 bash[76202]: cluster 2024-04-07T16:21:11.862936+0000 mon.a (mon.0) 199 : cluster [DBG] election_strategy: 1 2024-04-07T16:21:13.190 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:12 smithi100 bash[76202]: cluster 2024-04-07T16:21:11.862979+0000 mon.a (mon.0) 200 : cluster [DBG] 0: [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] mon.a 2024-04-07T16:21:13.190 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:12 smithi100 bash[76202]: cluster 2024-04-07T16:21:11.863027+0000 mon.a (mon.0) 201 : cluster [DBG] 1: [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] mon.b 2024-04-07T16:21:13.190 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:12 smithi100 bash[76202]: cluster 2024-04-07T16:21:11.863071+0000 mon.a (mon.0) 202 : cluster [DBG] 2: [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] mon.c 2024-04-07T16:21:13.190 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:12 smithi100 bash[76202]: cluster 2024-04-07T16:21:11.864608+0000 mon.a (mon.0) 203 : cluster [DBG] fsmap cephfs:1 {0=b=up:active} 1 up:standby 2024-04-07T16:21:13.191 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:12 smithi100 bash[76202]: cluster 2024-04-07T16:21:11.864710+0000 mon.a (mon.0) 204 : cluster [DBG] osdmap e324: 8 total, 8 up, 8 in 2024-04-07T16:21:13.191 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:12 smithi100 bash[76202]: cluster 2024-04-07T16:21:11.865622+0000 mon.a (mon.0) 205 : cluster [DBG] mgrmap e41: y(active, since 55s), standbys: x 2024-04-07T16:21:13.191 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:12 smithi100 bash[76202]: cluster 2024-04-07T16:21:11.865949+0000 mon.a (mon.0) 206 : cluster [INF] overall HEALTH_OK 2024-04-07T16:21:13.191 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:12 smithi100 bash[76202]: audit 2024-04-07T16:21:11.875455+0000 mon.a (mon.0) 207 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:21:13.191 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:12 smithi100 bash[76202]: audit 2024-04-07T16:21:11.888454+0000 mon.a (mon.0) 208 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:21:13.191 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:12 smithi100 bash[76202]: audit 2024-04-07T16:21:11.895096+0000 mon.a (mon.0) 209 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:21:13.498 INFO:tasks.rados.rados.0.smithi100.stdout:1348: finishing write tid 1 to smithi10066867-1 2024-04-07T16:21:13.498 INFO:tasks.rados.rados.0.smithi100.stdout:1348: finishing write tid 2 to smithi10066867-1 2024-04-07T16:21:13.498 INFO:tasks.rados.rados.0.smithi100.stdout:1348: finishing write tid 3 to smithi10066867-1 2024-04-07T16:21:13.499 INFO:tasks.rados.rados.0.smithi100.stdout:1348: finishing write tid 4 to smithi10066867-1 2024-04-07T16:21:13.499 INFO:tasks.rados.rados.0.smithi100.stdout:1348: finishing write tid 5 to smithi10066867-1 2024-04-07T16:21:13.499 INFO:tasks.rados.rados.0.smithi100.stdout:1348: finishing write tid 6 to smithi10066867-1 2024-04-07T16:21:13.499 INFO:tasks.rados.rados.0.smithi100.stdout:1348: oid 1 updating version 0 to 823 2024-04-07T16:21:13.499 INFO:tasks.rados.rados.0.smithi100.stdout:1348: oid 1 version 823 is already newer than 821 2024-04-07T16:21:13.499 INFO:tasks.rados.rados.0.smithi100.stdout:1348: oid 1 version 823 is already newer than 822 2024-04-07T16:21:13.499 INFO:tasks.rados.rados.0.smithi100.stdout:1348: oid 1 version 823 is already newer than 819 2024-04-07T16:21:13.499 INFO:tasks.rados.rados.0.smithi100.stdout:1348: oid 1 version 823 is already newer than 820 2024-04-07T16:21:13.499 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 1 v 823 (ObjNum 417 snap 126 seq_num 417) dirty exists 2024-04-07T16:21:13.499 INFO:tasks.rados.rados.0.smithi100.stdout:1348: left oid 1 (ObjNum 417 snap 126 seq_num 417) 2024-04-07T16:21:13.499 INFO:tasks.rados.rados.0.smithi100.stdout:1349: expect (ObjNum 393 snap 119 seq_num 393) 2024-04-07T16:21:13.856 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:21:13 smithi144 bash[59794]: ts=2024-04-07T16:21:13.508Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=nodes name=CephNodeDiskspaceWarning index=4 msg="Evaluating rule failed" rule="alert: CephNodeDiskspaceWarning\nexpr: predict_linear(node_filesystem_free_bytes{device=~\"/.*\"}[2d], 3600 * 24 * 5)\n * on (instance) group_left (nodename) node_uname_info < 0\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.8.4\n severity: warning\n type: ceph_default\nannotations:\n description: Mountpoint {{ $labels.mountpoint }} on {{ $labels.nodename }} will\n be full in less than 5 days based on the 48 hour trailing fill rate.\n summary: Host filesystem free space is getting low\n" err="found duplicate series for the match group {instance=\"smithi100\"} on the right hand-side of the operation: [{__name__=\"node_uname_info\", domainname=\"(none)\", instance=\"smithi100\", job=\"node\", machine=\"x86_64\", nodename=\"smithi100\", release=\"5.15.0-101-generic\", sysname=\"Linux\", version=\"#111-Ubuntu SMP Tue Mar 5 20:16:58 UTC 2024\"}, {__name__=\"node_uname_info\", cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", domainname=\"(none)\", instance=\"smithi100\", job=\"node\", machine=\"x86_64\", nodename=\"smithi100\", release=\"5.15.0-101-generic\", sysname=\"Linux\", version=\"#111-Ubuntu SMP Tue Mar 5 20:16:58 UTC 2024\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:21:14.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:13 smithi100 bash[74942]: cluster 2024-04-07T16:21:12.619730+0000 mgr.y (mgr.25175) 51 : cluster [DBG] pgmap v75: 121 pgs: 121 active+clean; 240 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 11 MiB/s rd, 3.0 MiB/s wr, 20 op/s 2024-04-07T16:21:14.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:13 smithi100 bash[74942]: cluster 2024-04-07T16:21:12.892214+0000 mon.a (mon.0) 210 : cluster [DBG] osdmap e325: 8 total, 8 up, 8 in 2024-04-07T16:21:14.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:13 smithi100 bash[76202]: cluster 2024-04-07T16:21:12.619730+0000 mgr.y (mgr.25175) 51 : cluster [DBG] pgmap v75: 121 pgs: 121 active+clean; 240 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 11 MiB/s rd, 3.0 MiB/s wr, 20 op/s 2024-04-07T16:21:14.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:13 smithi100 bash[76202]: cluster 2024-04-07T16:21:12.892214+0000 mon.a (mon.0) 210 : cluster [DBG] osdmap e325: 8 total, 8 up, 8 in 2024-04-07T16:21:14.276 INFO:tasks.rados.rados.0.smithi100.stdout:1350: finishing write tid 1 to smithi10066867-2 2024-04-07T16:21:14.276 INFO:tasks.rados.rados.0.smithi100.stdout:1350: finishing write tid 2 to smithi10066867-2 2024-04-07T16:21:14.276 INFO:tasks.rados.rados.0.smithi100.stdout:1350: finishing write tid 3 to smithi10066867-2 2024-04-07T16:21:14.276 INFO:tasks.rados.rados.0.smithi100.stdout:1350: finishing write tid 4 to smithi10066867-2 2024-04-07T16:21:14.276 INFO:tasks.rados.rados.0.smithi100.stdout:1350: finishing write tid 5 to smithi10066867-2 2024-04-07T16:21:14.277 INFO:tasks.rados.rados.0.smithi100.stdout:1350: finishing write tid 6 to smithi10066867-2 2024-04-07T16:21:14.277 INFO:tasks.rados.rados.0.smithi100.stdout:1350: oid 2 updating version 0 to 699 2024-04-07T16:21:14.277 INFO:tasks.rados.rados.0.smithi100.stdout:1350: oid 2 version 699 is already newer than 696 2024-04-07T16:21:14.277 INFO:tasks.rados.rados.0.smithi100.stdout:1350: oid 2 updating version 699 to 700 2024-04-07T16:21:14.277 INFO:tasks.rados.rados.0.smithi100.stdout:1350: oid 2 version 700 is already newer than 698 2024-04-07T16:21:14.277 INFO:tasks.rados.rados.0.smithi100.stdout:1350: oid 2 version 700 is already newer than 697 2024-04-07T16:21:14.277 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 2 v 700 (ObjNum 418 snap 126 seq_num 418) dirty exists 2024-04-07T16:21:14.277 INFO:tasks.rados.rados.0.smithi100.stdout:1350: left oid 2 (ObjNum 418 snap 126 seq_num 418) 2024-04-07T16:21:14.277 INFO:tasks.rados.rados.0.smithi100.stdout:1351: finishing copy_from racing read to smithi10066867-45 2024-04-07T16:21:14.277 INFO:tasks.rados.rados.0.smithi100.stdout:1351: finishing copy_from to smithi10066867-45 2024-04-07T16:21:14.277 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 45 v 710 (ObjNum 402 snap 121 seq_num 402) dirty exists 2024-04-07T16:21:14.277 INFO:tasks.rados.rados.0.smithi100.stdout:1353: read oid 40 snap -1 2024-04-07T16:21:14.277 INFO:tasks.rados.rados.0.smithi100.stdout:1353: expect (ObjNum 303 snap 87 seq_num 303) 2024-04-07T16:21:14.277 INFO:tasks.rados.rados.0.smithi100.stdout:1343: done (6 left) 2024-04-07T16:21:14.278 INFO:tasks.rados.rados.0.smithi100.stdout:1344: done (5 left) 2024-04-07T16:21:14.278 INFO:tasks.rados.rados.0.smithi100.stdout:1348: done (4 left) 2024-04-07T16:21:14.278 INFO:tasks.rados.rados.0.smithi100.stdout:1349: done (3 left) 2024-04-07T16:21:14.278 INFO:tasks.rados.rados.0.smithi100.stdout:1350: done (2 left) 2024-04-07T16:21:14.278 INFO:tasks.rados.rados.0.smithi100.stdout:1351: done (1 left) 2024-04-07T16:21:14.278 INFO:tasks.rados.rados.0.smithi100.stdout:1354: delete oid 28 current snap is 126 2024-04-07T16:21:14.280 INFO:tasks.rados.rados.0.smithi100.stdout:1354: done (1 left) 2024-04-07T16:21:14.280 INFO:tasks.rados.rados.0.smithi100.stdout:1355: rollback oid 42 current snap is 126 2024-04-07T16:21:14.280 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 42 to 106 2024-04-07T16:21:14.280 INFO:tasks.rados.rados.0.smithi100.stdout:1356: read oid 38 snap -1 2024-04-07T16:21:14.281 INFO:tasks.rados.rados.0.smithi100.stdout:1356: expect (ObjNum 360 snap 109 seq_num 360) 2024-04-07T16:21:14.281 INFO:tasks.rados.rados.0.smithi100.stdout:1357: snap_create 2024-04-07T16:21:14.286 INFO:tasks.rados.rados.0.smithi100.stdout:1355: finishing rollback tid 0 to smithi10066867-42 2024-04-07T16:21:14.286 INFO:tasks.rados.rados.0.smithi100.stdout:1355: finishing rollback tid 1 to smithi10066867-42 2024-04-07T16:21:14.286 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 42 v 572 (ObjNum 129 snap 0 seq_num 2701816624) dirty dne 2024-04-07T16:21:14.309 INFO:tasks.rados.rados.0.smithi100.stdout:1353: expect (ObjNum 303 snap 87 seq_num 303) 2024-04-07T16:21:14.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:13 smithi144 bash[66654]: cluster 2024-04-07T16:21:12.619730+0000 mgr.y (mgr.25175) 51 : cluster [DBG] pgmap v75: 121 pgs: 121 active+clean; 240 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 11 MiB/s rd, 3.0 MiB/s wr, 20 op/s 2024-04-07T16:21:14.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:13 smithi144 bash[66654]: cluster 2024-04-07T16:21:12.892214+0000 mon.a (mon.0) 210 : cluster [DBG] osdmap e325: 8 total, 8 up, 8 in 2024-04-07T16:21:14.780 INFO:tasks.rados.rados.0.smithi100.stdout:1356: expect (ObjNum 360 snap 109 seq_num 360) 2024-04-07T16:21:15.007 INFO:tasks.rados.rados.0.smithi100.stdout:1353: done (3 left) 2024-04-07T16:21:15.007 INFO:tasks.rados.rados.0.smithi100.stdout:1355: done (2 left) 2024-04-07T16:21:15.007 INFO:tasks.rados.rados.0.smithi100.stdout:1356: done (1 left) 2024-04-07T16:21:15.008 INFO:tasks.rados.rados.0.smithi100.stdout:1357: done (0 left) 2024-04-07T16:21:15.008 INFO:tasks.rados.rados.0.smithi100.stdout:1358: copy_from oid 34 from oid 38 current snap is 127 2024-04-07T16:21:15.008 INFO:tasks.rados.rados.0.smithi100.stdout:1359: rollback oid 46 current snap is 127 2024-04-07T16:21:15.008 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 46 to 125 2024-04-07T16:21:15.008 INFO:tasks.rados.rados.0.smithi100.stdout:1360: rollback oid 23 current snap is 127 2024-04-07T16:21:15.008 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 23 to 126 2024-04-07T16:21:15.008 INFO:tasks.rados.rados.0.smithi100.stdout:1361: delete oid 22 current snap is 127 2024-04-07T16:21:15.013 INFO:tasks.rados.rados.0.smithi100.stdout:1361: done (3 left) 2024-04-07T16:21:15.013 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 12 current snap is 127 2024-04-07T16:21:15.013 INFO:tasks.rados.rados.0.smithi100.stdout:1362: seq_num 421 ranges {0=655360,655360=442368,1097728=663552,1761280=499712} 2024-04-07T16:21:15.017 INFO:tasks.rados.rados.0.smithi100.stdout:1362: writing smithi10066867-12 from 0 to 655360 tid 1 2024-04-07T16:21:15.020 INFO:tasks.rados.rados.0.smithi100.stdout:1362: writing smithi10066867-12 from 655360 to 1097728 tid 2 2024-04-07T16:21:15.024 INFO:tasks.rados.rados.0.smithi100.stdout:1362: writing smithi10066867-12 from 1097728 to 1761280 tid 3 2024-04-07T16:21:15.027 INFO:tasks.rados.rados.0.smithi100.stdout:1362: writing smithi10066867-12 from 1761280 to 2260992 tid 4 2024-04-07T16:21:15.027 INFO:tasks.rados.rados.0.smithi100.stdout:1363: copy_from oid 28 from oid 42 current snap is 127 2024-04-07T16:21:15.027 INFO:tasks.rados.rados.0.smithi100.stdout:1360: finishing rollback tid 0 to smithi10066867-23 2024-04-07T16:21:15.027 INFO:tasks.rados.rados.0.smithi100.stdout:1359: finishing rollback tid 0 to smithi10066867-46 2024-04-07T16:21:15.027 INFO:tasks.rados.rados.0.smithi100.stdout:1359: finishing rollback tid 1 to smithi10066867-46 2024-04-07T16:21:15.027 INFO:tasks.rados.rados.0.smithi100.stdout:1359: finishing rollback tid 2 to smithi10066867-46 2024-04-07T16:21:15.028 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 46 v 573 (ObjNum 337 snap 101 seq_num 337) dirty exists 2024-04-07T16:21:15.028 INFO:tasks.rados.rados.0.smithi100.stdout:1360: finishing rollback tid 1 to smithi10066867-23 2024-04-07T16:21:15.028 INFO:tasks.rados.rados.0.smithi100.stdout:1360: finishing rollback tid 2 to smithi10066867-23 2024-04-07T16:21:15.028 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 23 v 885 (ObjNum 395 snap 119 seq_num 395) dirty exists 2024-04-07T16:21:15.028 INFO:tasks.rados.rados.0.smithi100.stdout:1362: finishing write tid 1 to smithi10066867-12 2024-04-07T16:21:15.028 INFO:tasks.rados.rados.0.smithi100.stdout:1359: done (4 left) 2024-04-07T16:21:15.028 INFO:tasks.rados.rados.0.smithi100.stdout:1360: done (3 left) 2024-04-07T16:21:15.028 INFO:tasks.rados.rados.0.smithi100.stdout:1364: snap_remove snap 125 2024-04-07T16:21:15.685 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:21:15 smithi100 bash[68137]: debug 2024-04-07T16:21:15.241+0000 7f8508b76700 -1 mgr.server handle_report got status from non-daemon mon.b 2024-04-07T16:21:15.898 INFO:tasks.rados.rados.0.smithi100.stdout:1364: done (3 left) 2024-04-07T16:21:15.898 INFO:tasks.rados.rados.0.smithi100.stdout:1365: copy_from oid 35 from oid 46 current snap is 127 2024-04-07T16:21:15.898 INFO:tasks.rados.rados.0.smithi100.stdout:1362: finishing write tid 2 to smithi10066867-12 2024-04-07T16:21:15.898 INFO:tasks.rados.rados.0.smithi100.stdout:1362: finishing write tid 3 to smithi10066867-12 2024-04-07T16:21:15.899 INFO:tasks.rados.rados.0.smithi100.stdout:1363: finishing copy_from to smithi10066867-28 2024-04-07T16:21:15.899 INFO:tasks.rados.rados.0.smithi100.stdout:1363: got expected ENOENT (src dne) 2024-04-07T16:21:15.899 INFO:tasks.rados.rados.0.smithi100.stdout:1358: finishing copy_from racing read to smithi10066867-34 2024-04-07T16:21:15.899 INFO:tasks.rados.rados.0.smithi100.stdout:1363: finishing copy_from racing read to smithi10066867-28 2024-04-07T16:21:15.899 INFO:tasks.rados.rados.0.smithi100.stdout:1362: finishing write tid 4 to smithi10066867-12 2024-04-07T16:21:15.899 INFO:tasks.rados.rados.0.smithi100.stdout:1362: finishing write tid 5 to smithi10066867-12 2024-04-07T16:21:15.899 INFO:tasks.rados.rados.0.smithi100.stdout:1358: finishing copy_from to smithi10066867-34 2024-04-07T16:21:15.899 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 34 v 239 (ObjNum 360 snap 109 seq_num 360) dirty exists 2024-04-07T16:21:15.899 INFO:tasks.rados.rados.0.smithi100.stdout:1362: finishing write tid 6 to smithi10066867-12 2024-04-07T16:21:15.899 INFO:tasks.rados.rados.0.smithi100.stdout:1362: oid 12 updating version 0 to 739 2024-04-07T16:21:15.899 INFO:tasks.rados.rados.0.smithi100.stdout:1362: oid 12 updating version 739 to 742 2024-04-07T16:21:15.899 INFO:tasks.rados.rados.0.smithi100.stdout:1362: oid 12 version 742 is already newer than 740 2024-04-07T16:21:15.900 INFO:tasks.rados.rados.0.smithi100.stdout:1362: oid 12 updating version 742 to 743 2024-04-07T16:21:15.900 INFO:tasks.rados.rados.0.smithi100.stdout:1362: oid 12 version 743 is already newer than 741 2024-04-07T16:21:15.900 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 12 v 743 (ObjNum 421 snap 127 seq_num 421) dirty exists 2024-04-07T16:21:15.900 INFO:tasks.rados.rados.0.smithi100.stdout:1362: left oid 12 (ObjNum 421 snap 127 seq_num 421) 2024-04-07T16:21:15.900 INFO:tasks.rados.rados.0.smithi100.stdout:1358: done (3 left) 2024-04-07T16:21:15.900 INFO:tasks.rados.rados.0.smithi100.stdout:1362: done (2 left) 2024-04-07T16:21:15.900 INFO:tasks.rados.rados.0.smithi100.stdout:1363: done (1 left) 2024-04-07T16:21:15.900 INFO:tasks.rados.rados.0.smithi100.stdout:1366: copy_from oid 7 from oid 48 current snap is 127 2024-04-07T16:21:15.900 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 27 current snap is 127 2024-04-07T16:21:15.900 INFO:tasks.rados.rados.0.smithi100.stdout:1367: seq_num 422 ranges {0=671744,671744=729088,1400832=417792,1818624=458752} 2024-04-07T16:21:15.912 INFO:tasks.rados.rados.0.smithi100.stdout:1367: writing smithi10066867-27 from 0 to 671744 tid 1 2024-04-07T16:21:15.925 INFO:tasks.rados.rados.0.smithi100.stdout:1367: writing smithi10066867-27 from 671744 to 1400832 tid 2 2024-04-07T16:21:15.932 INFO:tasks.rados.rados.0.smithi100.stdout:1367: writing smithi10066867-27 from 1400832 to 1818624 tid 3 2024-04-07T16:21:15.941 INFO:tasks.rados.rados.0.smithi100.stdout:1367: writing smithi10066867-27 from 1818624 to 2277376 tid 4 2024-04-07T16:21:15.941 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 31 current snap is 127 2024-04-07T16:21:15.941 INFO:tasks.rados.rados.0.smithi100.stdout:1367: finishing write tid 1 to smithi10066867-27 2024-04-07T16:21:15.941 INFO:tasks.rados.rados.0.smithi100.stdout:1366: finishing copy_from racing read to smithi10066867-7 2024-04-07T16:21:15.941 INFO:tasks.rados.rados.0.smithi100.stdout:1368: seq_num 423 ranges {3022848=688128,3710976=614400,4325376=622592,4947968=368640} 2024-04-07T16:21:15.962 INFO:tasks.rados.rados.0.smithi100.stdout:1368: writing smithi10066867-31 from 3022848 to 3710976 tid 1 2024-04-07T16:21:15.966 INFO:tasks.rados.rados.0.smithi100.stdout:1368: writing smithi10066867-31 from 3710976 to 4325376 tid 2 2024-04-07T16:21:15.970 INFO:tasks.rados.rados.0.smithi100.stdout:1368: writing smithi10066867-31 from 4325376 to 4947968 tid 3 2024-04-07T16:21:15.972 INFO:tasks.rados.rados.0.smithi100.stdout:1368: writing smithi10066867-31 from 4947968 to 5316608 tid 4 2024-04-07T16:21:15.973 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 30 current snap is 127 2024-04-07T16:21:15.973 INFO:tasks.rados.rados.0.smithi100.stdout:1367: finishing write tid 2 to smithi10066867-27 2024-04-07T16:21:15.973 INFO:tasks.rados.rados.0.smithi100.stdout:1367: finishing write tid 3 to smithi10066867-27 2024-04-07T16:21:15.973 INFO:tasks.rados.rados.0.smithi100.stdout:1366: finishing copy_from to smithi10066867-7 2024-04-07T16:21:15.973 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 7 v 526 (ObjNum 372 snap 112 seq_num 372) dirty exists 2024-04-07T16:21:15.973 INFO:tasks.rados.rados.0.smithi100.stdout:1367: finishing write tid 4 to smithi10066867-27 2024-04-07T16:21:15.973 INFO:tasks.rados.rados.0.smithi100.stdout:1369: seq_num 424 ranges {6094848=704512,6799360=491520,7290880=434176,7725056=679936} 2024-04-07T16:21:16.011 INFO:tasks.rados.rados.0.smithi100.stdout:1369: writing smithi10066867-30 from 6094848 to 6799360 tid 1 2024-04-07T16:21:16.014 INFO:tasks.rados.rados.0.smithi100.stdout:1369: writing smithi10066867-30 from 6799360 to 7290880 tid 2 2024-04-07T16:21:16.017 INFO:tasks.rados.rados.0.smithi100.stdout:1369: writing smithi10066867-30 from 7290880 to 7725056 tid 3 2024-04-07T16:21:16.022 INFO:tasks.rados.rados.0.smithi100.stdout:1369: writing smithi10066867-30 from 7725056 to 8404992 tid 4 2024-04-07T16:21:16.022 INFO:tasks.rados.rados.0.smithi100.stdout:1366: done (4 left) 2024-04-07T16:21:16.022 INFO:tasks.rados.rados.0.smithi100.stdout:1370: rmattr oid 5 current snap is 127 2024-04-07T16:21:16.022 INFO:tasks.rados.rados.0.smithi100.stdout:1370: done (4 left) 2024-04-07T16:21:16.022 INFO:tasks.rados.rados.0.smithi100.stdout:1371: delete oid 15 current snap is 127 2024-04-07T16:21:16.022 INFO:tasks.rados.rados.0.smithi100.stdout:1367: finishing write tid 5 to smithi10066867-27 2024-04-07T16:21:16.022 INFO:tasks.rados.rados.0.smithi100.stdout:1365: finishing copy_from racing read to smithi10066867-35 2024-04-07T16:21:16.022 INFO:tasks.rados.rados.0.smithi100.stdout:1365: finishing copy_from to smithi10066867-35 2024-04-07T16:21:16.022 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 35 v 573 (ObjNum 337 snap 101 seq_num 337) dirty exists 2024-04-07T16:21:16.022 INFO:tasks.rados.rados.0.smithi100.stdout:1367: finishing write tid 6 to smithi10066867-27 2024-04-07T16:21:16.022 INFO:tasks.rados.rados.0.smithi100.stdout:1367: oid 27 updating version 0 to 527 2024-04-07T16:21:16.023 INFO:tasks.rados.rados.0.smithi100.stdout:1367: oid 27 version 527 is already newer than 307 2024-04-07T16:21:16.023 INFO:tasks.rados.rados.0.smithi100.stdout:1367: oid 27 version 527 is already newer than 308 2024-04-07T16:21:16.023 INFO:tasks.rados.rados.0.smithi100.stdout:1367: oid 27 version 527 is already newer than 306 2024-04-07T16:21:16.023 INFO:tasks.rados.rados.0.smithi100.stdout:1367: oid 27 updating version 527 to 528 2024-04-07T16:21:16.023 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 27 v 528 (ObjNum 422 snap 127 seq_num 422) dirty exists 2024-04-07T16:21:16.023 INFO:tasks.rados.rados.0.smithi100.stdout:1367: left oid 27 (ObjNum 422 snap 127 seq_num 422) 2024-04-07T16:21:16.023 INFO:tasks.rados.rados.0.smithi100.stdout:1368: finishing write tid 1 to smithi10066867-31 2024-04-07T16:21:16.023 INFO:tasks.rados.rados.0.smithi100.stdout:1368: finishing write tid 2 to smithi10066867-31 2024-04-07T16:21:16.023 INFO:tasks.rados.rados.0.smithi100.stdout:1368: finishing write tid 3 to smithi10066867-31 2024-04-07T16:21:16.023 INFO:tasks.rados.rados.0.smithi100.stdout:1368: finishing write tid 4 to smithi10066867-31 2024-04-07T16:21:16.023 INFO:tasks.rados.rados.0.smithi100.stdout:1368: finishing write tid 5 to smithi10066867-31 2024-04-07T16:21:16.023 INFO:tasks.rados.rados.0.smithi100.stdout:1368: finishing write tid 6 to smithi10066867-31 2024-04-07T16:21:16.023 INFO:tasks.rados.rados.0.smithi100.stdout:1368: oid 31 updating version 0 to 575 2024-04-07T16:21:16.023 INFO:tasks.rados.rados.0.smithi100.stdout:1368: oid 31 updating version 575 to 578 2024-04-07T16:21:16.024 INFO:tasks.rados.rados.0.smithi100.stdout:1368: oid 31 version 578 is already newer than 577 2024-04-07T16:21:16.024 INFO:tasks.rados.rados.0.smithi100.stdout:1368: oid 31 version 578 is already newer than 576 2024-04-07T16:21:16.024 INFO:tasks.rados.rados.0.smithi100.stdout:1368: oid 31 version 578 is already newer than 574 2024-04-07T16:21:16.024 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 31 v 578 (ObjNum 423 snap 127 seq_num 423) dirty exists 2024-04-07T16:21:16.024 INFO:tasks.rados.rados.0.smithi100.stdout:1368: left oid 31 (ObjNum 423 snap 127 seq_num 423) 2024-04-07T16:21:16.024 INFO:tasks.rados.rados.0.smithi100.stdout:1369: finishing write tid 1 to smithi10066867-30 2024-04-07T16:21:16.024 INFO:tasks.rados.rados.0.smithi100.stdout:1369: finishing write tid 2 to smithi10066867-30 2024-04-07T16:21:16.024 INFO:tasks.rados.rados.0.smithi100.stdout:1369: finishing write tid 3 to smithi10066867-30 2024-04-07T16:21:16.027 INFO:tasks.rados.rados.0.smithi100.stdout:1369: finishing write tid 4 to smithi10066867-30 2024-04-07T16:21:16.027 INFO:tasks.rados.rados.0.smithi100.stdout:1365: done (4 left) 2024-04-07T16:21:16.027 INFO:tasks.rados.rados.0.smithi100.stdout:1367: done (3 left) 2024-04-07T16:21:16.027 INFO:tasks.rados.rados.0.smithi100.stdout:1368: done (2 left) 2024-04-07T16:21:16.027 INFO:tasks.rados.rados.0.smithi100.stdout:1371: done (1 left) 2024-04-07T16:21:16.028 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 5 current snap is 127 2024-04-07T16:21:16.028 INFO:tasks.rados.rados.0.smithi100.stdout:1372: seq_num 426 ranges {0=737280,737280=712704,1449984=802816,2252800=90112} 2024-04-07T16:21:16.031 INFO:tasks.rados.rados.0.smithi100.stdout:1372: writing smithi10066867-5 from 0 to 737280 tid 1 2024-04-07T16:21:16.036 INFO:tasks.rados.rados.0.smithi100.stdout:1372: writing smithi10066867-5 from 737280 to 1449984 tid 2 2024-04-07T16:21:16.041 INFO:tasks.rados.rados.0.smithi100.stdout:1372: writing smithi10066867-5 from 1449984 to 2252800 tid 3 2024-04-07T16:21:16.042 INFO:tasks.rados.rados.0.smithi100.stdout:1372: writing smithi10066867-5 from 2252800 to 2342912 tid 4 2024-04-07T16:21:16.042 INFO:tasks.rados.rados.0.smithi100.stdout:1373: delete oid 38 current snap is 127 2024-04-07T16:21:16.042 INFO:tasks.rados.rados.0.smithi100.stdout:1369: finishing write tid 5 to smithi10066867-30 2024-04-07T16:21:16.042 INFO:tasks.rados.rados.0.smithi100.stdout:1369: finishing write tid 6 to smithi10066867-30 2024-04-07T16:21:16.042 INFO:tasks.rados.rados.0.smithi100.stdout:1369: oid 30 updating version 0 to 579 2024-04-07T16:21:16.042 INFO:tasks.rados.rados.0.smithi100.stdout:1369: oid 30 version 579 is already newer than 578 2024-04-07T16:21:16.042 INFO:tasks.rados.rados.0.smithi100.stdout:1369: oid 30 updating version 579 to 580 2024-04-07T16:21:16.042 INFO:tasks.rados.rados.0.smithi100.stdout:1369: oid 30 version 580 is already newer than 577 2024-04-07T16:21:16.042 INFO:tasks.rados.rados.0.smithi100.stdout:1369: oid 30 version 580 is already newer than 576 2024-04-07T16:21:16.042 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 30 v 580 (ObjNum 424 snap 127 seq_num 424) dirty exists 2024-04-07T16:21:16.043 INFO:tasks.rados.rados.0.smithi100.stdout:1369: left oid 30 (ObjNum 424 snap 127 seq_num 424) 2024-04-07T16:21:16.043 INFO:tasks.rados.rados.0.smithi100.stdout:1372: finishing write tid 1 to smithi10066867-5 2024-04-07T16:21:16.043 INFO:tasks.rados.rados.0.smithi100.stdout:1372: finishing write tid 2 to smithi10066867-5 2024-04-07T16:21:16.046 INFO:tasks.rados.rados.0.smithi100.stdout:1372: finishing write tid 3 to smithi10066867-5 2024-04-07T16:21:16.047 INFO:tasks.rados.rados.0.smithi100.stdout:1372: finishing write tid 4 to smithi10066867-5 2024-04-07T16:21:16.047 INFO:tasks.rados.rados.0.smithi100.stdout:1369: done (2 left) 2024-04-07T16:21:16.047 INFO:tasks.rados.rados.0.smithi100.stdout:1373: done (1 left) 2024-04-07T16:21:16.047 INFO:tasks.rados.rados.0.smithi100.stdout:1374: setattr oid 17 current snap is 127 2024-04-07T16:21:16.047 INFO:tasks.rados.rados.0.smithi100.stdout:1372: finishing write tid 5 to smithi10066867-5 2024-04-07T16:21:16.048 INFO:tasks.rados.rados.0.smithi100.stdout:1375: read oid 21 snap -1 2024-04-07T16:21:16.049 INFO:tasks.rados.rados.0.smithi100.stdout:1375: expect (ObjNum 391 snap 119 seq_num 391) 2024-04-07T16:21:16.049 INFO:tasks.rados.rados.0.smithi100.stdout:1376: snap_remove snap 119 2024-04-07T16:21:16.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:15 smithi100 bash[74942]: cluster 2024-04-07T16:21:14.620281+0000 mgr.y (mgr.25175) 52 : cluster [DBG] pgmap v77: 121 pgs: 121 active+clean; 240 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 8.9 MiB/s rd, 1.8 MiB/s wr, 10 op/s 2024-04-07T16:21:16.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:15 smithi100 bash[74942]: cluster 2024-04-07T16:21:14.905373+0000 mon.a (mon.0) 211 : cluster [DBG] osdmap e326: 8 total, 8 up, 8 in 2024-04-07T16:21:16.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:15 smithi100 bash[76202]: cluster 2024-04-07T16:21:14.620281+0000 mgr.y (mgr.25175) 52 : cluster [DBG] pgmap v77: 121 pgs: 121 active+clean; 240 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 8.9 MiB/s rd, 1.8 MiB/s wr, 10 op/s 2024-04-07T16:21:16.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:15 smithi100 bash[76202]: cluster 2024-04-07T16:21:14.905373+0000 mon.a (mon.0) 211 : cluster [DBG] osdmap e326: 8 total, 8 up, 8 in 2024-04-07T16:21:16.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:15 smithi144 bash[66654]: cluster 2024-04-07T16:21:14.620281+0000 mgr.y (mgr.25175) 52 : cluster [DBG] pgmap v77: 121 pgs: 121 active+clean; 240 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 8.9 MiB/s rd, 1.8 MiB/s wr, 10 op/s 2024-04-07T16:21:16.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:15 smithi144 bash[66654]: cluster 2024-04-07T16:21:14.905373+0000 mon.a (mon.0) 211 : cluster [DBG] osdmap e326: 8 total, 8 up, 8 in 2024-04-07T16:21:16.759 INFO:tasks.rados.rados.0.smithi100.stdout:1376: done (3 left) 2024-04-07T16:21:16.759 INFO:tasks.rados.rados.0.smithi100.stdout:1377: snap_remove snap 106 2024-04-07T16:21:17.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:16 smithi100 bash[74942]: cluster 2024-04-07T16:21:15.914536+0000 mon.a (mon.0) 212 : cluster [DBG] osdmap e327: 8 total, 8 up, 8 in 2024-04-07T16:21:17.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:16 smithi100 bash[74942]: audit 2024-04-07T16:21:16.703087+0000 mon.a (mon.0) 213 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:21:17.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:16 smithi100 bash[74942]: cluster 2024-04-07T16:21:16.774736+0000 mon.a (mon.0) 214 : cluster [DBG] osdmap e328: 8 total, 8 up, 8 in 2024-04-07T16:21:17.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:16 smithi100 bash[76202]: cluster 2024-04-07T16:21:15.914536+0000 mon.a (mon.0) 212 : cluster [DBG] osdmap e327: 8 total, 8 up, 8 in 2024-04-07T16:21:17.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:16 smithi100 bash[76202]: audit 2024-04-07T16:21:16.703087+0000 mon.a (mon.0) 213 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:21:17.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:16 smithi100 bash[76202]: cluster 2024-04-07T16:21:16.774736+0000 mon.a (mon.0) 214 : cluster [DBG] osdmap e328: 8 total, 8 up, 8 in 2024-04-07T16:21:17.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:16 smithi144 bash[66654]: cluster 2024-04-07T16:21:15.914536+0000 mon.a (mon.0) 212 : cluster [DBG] osdmap e327: 8 total, 8 up, 8 in 2024-04-07T16:21:17.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:16 smithi144 bash[66654]: audit 2024-04-07T16:21:16.703087+0000 mon.a (mon.0) 213 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:21:17.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:16 smithi144 bash[66654]: cluster 2024-04-07T16:21:16.774736+0000 mon.a (mon.0) 214 : cluster [DBG] osdmap e328: 8 total, 8 up, 8 in 2024-04-07T16:21:17.752 INFO:tasks.rados.rados.0.smithi100.stdout:1377: done (3 left) 2024-04-07T16:21:17.752 INFO:tasks.rados.rados.0.smithi100.stdout:1378: snap_create 2024-04-07T16:21:17.752 INFO:tasks.rados.rados.0.smithi100.stdout:1372: finishing write tid 6 to smithi10066867-5 2024-04-07T16:21:17.752 INFO:tasks.rados.rados.0.smithi100.stdout:1372: oid 5 updating version 0 to 747 2024-04-07T16:21:17.753 INFO:tasks.rados.rados.0.smithi100.stdout:1372: oid 5 updating version 747 to 749 2024-04-07T16:21:17.753 INFO:tasks.rados.rados.0.smithi100.stdout:1372: oid 5 version 749 is already newer than 746 2024-04-07T16:21:17.753 INFO:tasks.rados.rados.0.smithi100.stdout:1372: oid 5 version 749 is already newer than 748 2024-04-07T16:21:17.753 INFO:tasks.rados.rados.0.smithi100.stdout:1372: oid 5 updating version 749 to 750 2024-04-07T16:21:17.753 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 5 v 750 (ObjNum 426 snap 127 seq_num 426) dirty exists 2024-04-07T16:21:17.753 INFO:tasks.rados.rados.0.smithi100.stdout:1372: left oid 5 (ObjNum 426 snap 127 seq_num 426) 2024-04-07T16:21:17.753 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 17 v 684 (ObjNum 395 snap 119 seq_num 395) dirty exists 2024-04-07T16:21:17.753 INFO:tasks.rados.rados.0.smithi100.stdout:1375: expect (ObjNum 391 snap 119 seq_num 391) 2024-04-07T16:21:18.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:17 smithi100 bash[74942]: cluster 2024-04-07T16:21:16.620983+0000 mgr.y (mgr.25175) 53 : cluster [DBG] pgmap v80: 121 pgs: 121 active+clean; 240 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 13 MiB/s rd, 2.5 MiB/s wr, 15 op/s 2024-04-07T16:21:18.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:17 smithi100 bash[74942]: cluster 2024-04-07T16:21:17.768243+0000 mon.a (mon.0) 215 : cluster [DBG] osdmap e329: 8 total, 8 up, 8 in 2024-04-07T16:21:18.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:17 smithi100 bash[76202]: cluster 2024-04-07T16:21:16.620983+0000 mgr.y (mgr.25175) 53 : cluster [DBG] pgmap v80: 121 pgs: 121 active+clean; 240 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 13 MiB/s rd, 2.5 MiB/s wr, 15 op/s 2024-04-07T16:21:18.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:17 smithi100 bash[76202]: cluster 2024-04-07T16:21:17.768243+0000 mon.a (mon.0) 215 : cluster [DBG] osdmap e329: 8 total, 8 up, 8 in 2024-04-07T16:21:18.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:17 smithi144 bash[66654]: cluster 2024-04-07T16:21:16.620983+0000 mgr.y (mgr.25175) 53 : cluster [DBG] pgmap v80: 121 pgs: 121 active+clean; 240 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 13 MiB/s rd, 2.5 MiB/s wr, 15 op/s 2024-04-07T16:21:18.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:17 smithi144 bash[66654]: cluster 2024-04-07T16:21:17.768243+0000 mon.a (mon.0) 215 : cluster [DBG] osdmap e329: 8 total, 8 up, 8 in 2024-04-07T16:21:18.685 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:21:18 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:21:18] "GET /metrics HTTP/1.1" 200 36077 "" "Prometheus/2.43.0" 2024-04-07T16:21:18.759 INFO:tasks.rados.rados.0.smithi100.stdout:1372: done (3 left) 2024-04-07T16:21:18.759 INFO:tasks.rados.rados.0.smithi100.stdout:1374: done (2 left) 2024-04-07T16:21:18.759 INFO:tasks.rados.rados.0.smithi100.stdout:1375: done (1 left) 2024-04-07T16:21:18.759 INFO:tasks.rados.rados.0.smithi100.stdout:1378: done (0 left) 2024-04-07T16:21:18.759 INFO:tasks.rados.rados.0.smithi100.stdout:1379: rollback oid 14 current snap is 128 2024-04-07T16:21:18.759 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 14 to 126 2024-04-07T16:21:18.759 INFO:tasks.rados.rados.0.smithi100.stdout:1380: snap_remove snap 127 2024-04-07T16:21:19.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:18 smithi144 bash[66654]: audit 2024-04-07T16:21:17.964808+0000 mon.a (mon.0) 216 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:21:19.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:18 smithi144 bash[66654]: audit 2024-04-07T16:21:17.976624+0000 mon.a (mon.0) 217 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:21:19.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:18 smithi144 bash[66654]: cluster 2024-04-07T16:21:18.775432+0000 mon.a (mon.0) 218 : cluster [DBG] osdmap e330: 8 total, 8 up, 8 in 2024-04-07T16:21:19.436 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:18 smithi100 bash[76202]: audit 2024-04-07T16:21:17.964808+0000 mon.a (mon.0) 216 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:21:19.436 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:18 smithi100 bash[76202]: audit 2024-04-07T16:21:17.976624+0000 mon.a (mon.0) 217 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:21:19.436 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:18 smithi100 bash[76202]: cluster 2024-04-07T16:21:18.775432+0000 mon.a (mon.0) 218 : cluster [DBG] osdmap e330: 8 total, 8 up, 8 in 2024-04-07T16:21:19.436 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:18 smithi100 bash[74942]: audit 2024-04-07T16:21:17.964808+0000 mon.a (mon.0) 216 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:21:19.436 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:18 smithi100 bash[74942]: audit 2024-04-07T16:21:17.976624+0000 mon.a (mon.0) 217 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:21:19.436 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:18 smithi100 bash[74942]: cluster 2024-04-07T16:21:18.775432+0000 mon.a (mon.0) 218 : cluster [DBG] osdmap e330: 8 total, 8 up, 8 in 2024-04-07T16:21:19.760 INFO:tasks.rados.rados.0.smithi100.stdout:1380: done (1 left) 2024-04-07T16:21:19.760 INFO:tasks.rados.rados.0.smithi100.stdout:1381: snap_create 2024-04-07T16:21:19.761 INFO:tasks.rados.rados.0.smithi100.stdout:1379: finishing rollback tid 0 to smithi10066867-14 2024-04-07T16:21:19.761 INFO:tasks.rados.rados.0.smithi100.stdout:1379: finishing rollback tid 1 to smithi10066867-14 2024-04-07T16:21:19.761 INFO:tasks.rados.rados.0.smithi100.stdout:1379: finishing rollback tid 2 to smithi10066867-14 2024-04-07T16:21:19.761 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 14 v 713 (ObjNum 342 snap 101 seq_num 342) dirty exists 2024-04-07T16:21:20.435 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:20 smithi100 bash[74942]: cluster 2024-04-07T16:21:18.621738+0000 mgr.y (mgr.25175) 54 : cluster [DBG] pgmap v83: 121 pgs: 121 active+clean; 246 MiB data, 1.7 GiB used, 714 GiB / 715 GiB avail; 8.6 MiB/s rd, 5.1 MiB/s wr, 19 op/s 2024-04-07T16:21:20.436 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:20 smithi100 bash[74942]: audit 2024-04-07T16:21:19.139896+0000 mon.a (mon.0) 219 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:21:20.436 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:20 smithi100 bash[74942]: audit 2024-04-07T16:21:19.153959+0000 mon.a (mon.0) 220 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:21:20.436 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:20 smithi100 bash[74942]: cluster 2024-04-07T16:21:19.776063+0000 mon.a (mon.0) 221 : cluster [DBG] osdmap e331: 8 total, 8 up, 8 in 2024-04-07T16:21:20.436 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:20 smithi100 bash[76202]: cluster 2024-04-07T16:21:18.621738+0000 mgr.y (mgr.25175) 54 : cluster [DBG] pgmap v83: 121 pgs: 121 active+clean; 246 MiB data, 1.7 GiB used, 714 GiB / 715 GiB avail; 8.6 MiB/s rd, 5.1 MiB/s wr, 19 op/s 2024-04-07T16:21:20.436 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:20 smithi100 bash[76202]: audit 2024-04-07T16:21:19.139896+0000 mon.a (mon.0) 219 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:21:20.436 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:20 smithi100 bash[76202]: audit 2024-04-07T16:21:19.153959+0000 mon.a (mon.0) 220 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:21:20.436 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:20 smithi100 bash[76202]: cluster 2024-04-07T16:21:19.776063+0000 mon.a (mon.0) 221 : cluster [DBG] osdmap e331: 8 total, 8 up, 8 in 2024-04-07T16:21:20.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:20 smithi144 bash[66654]: cluster 2024-04-07T16:21:18.621738+0000 mgr.y (mgr.25175) 54 : cluster [DBG] pgmap v83: 121 pgs: 121 active+clean; 246 MiB data, 1.7 GiB used, 714 GiB / 715 GiB avail; 8.6 MiB/s rd, 5.1 MiB/s wr, 19 op/s 2024-04-07T16:21:20.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:20 smithi144 bash[66654]: audit 2024-04-07T16:21:19.139896+0000 mon.a (mon.0) 219 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:21:20.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:20 smithi144 bash[66654]: audit 2024-04-07T16:21:19.153959+0000 mon.a (mon.0) 220 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:21:20.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:20 smithi144 bash[66654]: cluster 2024-04-07T16:21:19.776063+0000 mon.a (mon.0) 221 : cluster [DBG] osdmap e331: 8 total, 8 up, 8 in 2024-04-07T16:21:20.761 INFO:tasks.rados.rados.0.smithi100.stdout:1379: done (1 left) 2024-04-07T16:21:20.761 INFO:tasks.rados.rados.0.smithi100.stdout:1381: done (0 left) 2024-04-07T16:21:20.761 INFO:tasks.rados.rados.0.smithi100.stdout:1382: rollback oid 16 current snap is 129 2024-04-07T16:21:20.761 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 16 to 128 2024-04-07T16:21:20.762 INFO:tasks.rados.rados.0.smithi100.stdout:1383: read oid 31 snap -1 2024-04-07T16:21:20.762 INFO:tasks.rados.rados.0.smithi100.stdout:1383: expect (ObjNum 423 snap 127 seq_num 423) 2024-04-07T16:21:20.762 INFO:tasks.rados.rados.0.smithi100.stdout:1384: read oid 6 snap -1 2024-04-07T16:21:20.762 INFO:tasks.rados.rados.0.smithi100.stdout:1384: expect (ObjNum 407 snap 123 seq_num 407) 2024-04-07T16:21:20.762 INFO:tasks.rados.rados.0.smithi100.stdout:1385: rollback oid 23 current snap is 129 2024-04-07T16:21:20.762 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 23 to 126 2024-04-07T16:21:20.762 INFO:tasks.rados.rados.0.smithi100.stdout:1386: read oid 50 snap -1 2024-04-07T16:21:20.762 INFO:tasks.rados.rados.0.smithi100.stdout:1386: expect (ObjNum 416 snap 126 seq_num 416) 2024-04-07T16:21:20.762 INFO:tasks.rados.rados.0.smithi100.stdout:1387: rmattr oid 20 current snap is 129 2024-04-07T16:21:20.762 INFO:tasks.rados.rados.0.smithi100.stdout:1387: done (5 left) 2024-04-07T16:21:20.762 INFO:tasks.rados.rados.0.smithi100.stdout:1388: copy_from oid 1 from oid 2 current snap is 129 2024-04-07T16:21:20.762 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 4 current snap is 129 2024-04-07T16:21:20.762 INFO:tasks.rados.rados.0.smithi100.stdout:1389: seq_num 428 ranges {0=778240,778240=483328,1261568=409600,1671168=417792,2088960=294912} 2024-04-07T16:21:20.773 INFO:tasks.rados.rados.0.smithi100.stdout:1389: writing smithi10066867-4 from 0 to 778240 tid 1 2024-04-07T16:21:20.780 INFO:tasks.rados.rados.0.smithi100.stdout:1389: writing smithi10066867-4 from 778240 to 1261568 tid 2 2024-04-07T16:21:20.786 INFO:tasks.rados.rados.0.smithi100.stdout:1389: writing smithi10066867-4 from 1261568 to 1671168 tid 3 2024-04-07T16:21:20.792 INFO:tasks.rados.rados.0.smithi100.stdout:1389: writing smithi10066867-4 from 1671168 to 2088960 tid 4 2024-04-07T16:21:20.797 INFO:tasks.rados.rados.0.smithi100.stdout:1389: writing smithi10066867-4 from 2088960 to 2383872 tid 5 2024-04-07T16:21:20.797 INFO:tasks.rados.rados.0.smithi100.stdout:1390: copy_from oid 40 from oid 35 current snap is 129 2024-04-07T16:21:20.797 INFO:tasks.rados.rados.0.smithi100.stdout:1391: read oid 22 snap -1 2024-04-07T16:21:20.797 INFO:tasks.rados.rados.0.smithi100.stdout:1391: expect deleted 2024-04-07T16:21:20.797 INFO:tasks.rados.rados.0.smithi100.stdout:1392: rollback oid 7 current snap is 129 2024-04-07T16:21:20.797 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 7 to 128 2024-04-07T16:21:20.797 INFO:tasks.rados.rados.0.smithi100.stdout:1393: rollback oid 17 current snap is 129 2024-04-07T16:21:20.797 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 17 to 126 2024-04-07T16:21:20.801 INFO:tasks.rados.rados.0.smithi100.stdout:1386: expect (ObjNum 416 snap 126 seq_num 416) 2024-04-07T16:21:20.872 INFO:tasks.rados.rados.0.smithi100.stdout:1382: finishing rollback tid 0 to smithi10066867-16 2024-04-07T16:21:20.872 INFO:tasks.rados.rados.0.smithi100.stdout:1382: finishing rollback tid 1 to smithi10066867-16 2024-04-07T16:21:20.872 INFO:tasks.rados.rados.0.smithi100.stdout:1385: finishing rollback tid 0 to smithi10066867-23 2024-04-07T16:21:20.872 INFO:tasks.rados.rados.0.smithi100.stdout:1385: finishing rollback tid 1 to smithi10066867-23 2024-04-07T16:21:20.872 INFO:tasks.rados.rados.0.smithi100.stdout:1382: finishing rollback tid 2 to smithi10066867-16 2024-04-07T16:21:20.872 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 16 v 768 (ObjNum 374 snap 113 seq_num 374) dirty exists 2024-04-07T16:21:20.873 INFO:tasks.rados.rados.0.smithi100.stdout:1392: finishing rollback tid 0 to smithi10066867-7 2024-04-07T16:21:20.873 INFO:tasks.rados.rados.0.smithi100.stdout:1392: finishing rollback tid 1 to smithi10066867-7 2024-04-07T16:21:20.873 INFO:tasks.rados.rados.0.smithi100.stdout:1385: finishing rollback tid 2 to smithi10066867-23 2024-04-07T16:21:20.873 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 23 v 915 (ObjNum 395 snap 119 seq_num 395) dirty exists 2024-04-07T16:21:20.873 INFO:tasks.rados.rados.0.smithi100.stdout:1392: finishing rollback tid 2 to smithi10066867-7 2024-04-07T16:21:20.873 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 7 v 531 (ObjNum 372 snap 112 seq_num 372) dirty exists 2024-04-07T16:21:20.873 INFO:tasks.rados.rados.0.smithi100.stdout:1382: done (10 left) 2024-04-07T16:21:20.873 INFO:tasks.rados.rados.0.smithi100.stdout:1385: done (9 left) 2024-04-07T16:21:20.873 INFO:tasks.rados.rados.0.smithi100.stdout:1386: done (8 left) 2024-04-07T16:21:20.873 INFO:tasks.rados.rados.0.smithi100.stdout:1391: done (7 left) 2024-04-07T16:21:20.873 INFO:tasks.rados.rados.0.smithi100.stdout:1392: done (6 left) 2024-04-07T16:21:20.873 INFO:tasks.rados.rados.0.smithi100.stdout:1394: snap_create 2024-04-07T16:21:20.873 INFO:tasks.rados.rados.0.smithi100.stdout:1389: finishing write tid 1 to smithi10066867-4 2024-04-07T16:21:20.873 INFO:tasks.rados.rados.0.smithi100.stdout:1389: finishing write tid 2 to smithi10066867-4 2024-04-07T16:21:20.873 INFO:tasks.rados.rados.0.smithi100.stdout:1389: finishing write tid 3 to smithi10066867-4 2024-04-07T16:21:20.873 INFO:tasks.rados.rados.0.smithi100.stdout:1389: finishing write tid 4 to smithi10066867-4 2024-04-07T16:21:20.873 INFO:tasks.rados.rados.0.smithi100.stdout:1393: finishing rollback tid 0 to smithi10066867-17 2024-04-07T16:21:20.874 INFO:tasks.rados.rados.0.smithi100.stdout:1393: finishing rollback tid 1 to smithi10066867-17 2024-04-07T16:21:20.874 INFO:tasks.rados.rados.0.smithi100.stdout:1393: finishing rollback tid 2 to smithi10066867-17 2024-04-07T16:21:20.874 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 17 v 687 (ObjNum 395 snap 119 seq_num 395) dirty exists 2024-04-07T16:21:20.874 INFO:tasks.rados.rados.0.smithi100.stdout:1389: finishing write tid 5 to smithi10066867-4 2024-04-07T16:21:20.874 INFO:tasks.rados.rados.0.smithi100.stdout:1389: finishing write tid 6 to smithi10066867-4 2024-04-07T16:21:20.874 INFO:tasks.rados.rados.0.smithi100.stdout:1389: finishing write tid 7 to smithi10066867-4 2024-04-07T16:21:20.874 INFO:tasks.rados.rados.0.smithi100.stdout:1389: oid 4 updating version 0 to 921 2024-04-07T16:21:20.874 INFO:tasks.rados.rados.0.smithi100.stdout:1389: oid 4 version 921 is already newer than 920 2024-04-07T16:21:20.874 INFO:tasks.rados.rados.0.smithi100.stdout:1389: oid 4 version 921 is already newer than 917 2024-04-07T16:21:20.874 INFO:tasks.rados.rados.0.smithi100.stdout:1389: oid 4 version 921 is already newer than 916 2024-04-07T16:21:20.874 INFO:tasks.rados.rados.0.smithi100.stdout:1389: oid 4 version 921 is already newer than 919 2024-04-07T16:21:20.874 INFO:tasks.rados.rados.0.smithi100.stdout:1389: oid 4 version 921 is already newer than 918 2024-04-07T16:21:20.874 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 4 v 921 (ObjNum 428 snap 129 seq_num 428) dirty exists 2024-04-07T16:21:20.874 INFO:tasks.rados.rados.0.smithi100.stdout:1389: left oid 4 (ObjNum 428 snap 129 seq_num 428) 2024-04-07T16:21:20.874 INFO:tasks.rados.rados.0.smithi100.stdout:1383: expect (ObjNum 423 snap 127 seq_num 423) 2024-04-07T16:21:21.073 INFO:tasks.rados.rados.0.smithi100.stdout:1384: expect (ObjNum 407 snap 123 seq_num 407) 2024-04-07T16:21:21.466 INFO:tasks.rados.rados.0.smithi100.stdout:1388: finishing copy_from racing read to smithi10066867-1 2024-04-07T16:21:21.466 INFO:tasks.rados.rados.0.smithi100.stdout:1388: finishing copy_from to smithi10066867-1 2024-04-07T16:21:21.466 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 1 v 700 (ObjNum 418 snap 126 seq_num 418) dirty exists 2024-04-07T16:21:21.466 INFO:tasks.rados.rados.0.smithi100.stdout:1390: finishing copy_from racing read to smithi10066867-40 2024-04-07T16:21:21.466 INFO:tasks.rados.rados.0.smithi100.stdout:1390: finishing copy_from to smithi10066867-40 2024-04-07T16:21:21.466 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 40 v 573 (ObjNum 337 snap 101 seq_num 337) dirty exists 2024-04-07T16:21:21.759 INFO:tasks.rados.rados.0.smithi100.stdout:1383: done (6 left) 2024-04-07T16:21:21.759 INFO:tasks.rados.rados.0.smithi100.stdout:1384: done (5 left) 2024-04-07T16:21:21.760 INFO:tasks.rados.rados.0.smithi100.stdout:1388: done (4 left) 2024-04-07T16:21:21.760 INFO:tasks.rados.rados.0.smithi100.stdout:1389: done (3 left) 2024-04-07T16:21:21.760 INFO:tasks.rados.rados.0.smithi100.stdout:1390: done (2 left) 2024-04-07T16:21:21.760 INFO:tasks.rados.rados.0.smithi100.stdout:1393: done (1 left) 2024-04-07T16:21:21.760 INFO:tasks.rados.rados.0.smithi100.stdout:1394: done (0 left) 2024-04-07T16:21:21.760 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 5 current snap is 130 2024-04-07T16:21:21.760 INFO:tasks.rados.rados.0.smithi100.stdout:1395: seq_num 429 ranges {2342912=794624,3137536=770048,3907584=622592,4530176=212992} 2024-04-07T16:21:21.806 INFO:tasks.rados.rados.0.smithi100.stdout:1395: writing smithi10066867-5 from 2342912 to 3137536 tid 1 2024-04-07T16:21:21.816 INFO:tasks.rados.rados.0.smithi100.stdout:1395: writing smithi10066867-5 from 3137536 to 3907584 tid 2 2024-04-07T16:21:21.823 INFO:tasks.rados.rados.0.smithi100.stdout:1395: writing smithi10066867-5 from 3907584 to 4530176 tid 3 2024-04-07T16:21:21.825 INFO:tasks.rados.rados.0.smithi100.stdout:1395: writing smithi10066867-5 from 4530176 to 4743168 tid 4 2024-04-07T16:21:21.825 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 10 current snap is 130 2024-04-07T16:21:21.825 INFO:tasks.rados.rados.0.smithi100.stdout:1396: seq_num 430 ranges {4431872=8192} 2024-04-07T16:21:21.853 INFO:tasks.rados.rados.0.smithi100.stdout:1396: writing smithi10066867-10 from 4431872 to 4440064 tid 1 2024-04-07T16:21:21.853 INFO:tasks.rados.rados.0.smithi100.stdout:1397: rmattr oid 30 current snap is 130 2024-04-07T16:21:21.853 INFO:tasks.rados.rados.0.smithi100.stdout:1395: finishing write tid 1 to smithi10066867-5 2024-04-07T16:21:21.853 INFO:tasks.rados.rados.0.smithi100.stdout:1395: finishing write tid 2 to smithi10066867-5 2024-04-07T16:21:21.853 INFO:tasks.rados.rados.0.smithi100.stdout:1395: finishing write tid 3 to smithi10066867-5 2024-04-07T16:21:21.853 INFO:tasks.rados.rados.0.smithi100.stdout:1395: finishing write tid 4 to smithi10066867-5 2024-04-07T16:21:21.853 INFO:tasks.rados.rados.0.smithi100.stdout:1395: finishing write tid 5 to smithi10066867-5 2024-04-07T16:21:21.853 INFO:tasks.rados.rados.0.smithi100.stdout:1395: finishing write tid 6 to smithi10066867-5 2024-04-07T16:21:21.853 INFO:tasks.rados.rados.0.smithi100.stdout:1395: oid 5 updating version 0 to 771 2024-04-07T16:21:21.854 INFO:tasks.rados.rados.0.smithi100.stdout:1395: oid 5 updating version 771 to 772 2024-04-07T16:21:21.854 INFO:tasks.rados.rados.0.smithi100.stdout:1395: oid 5 updating version 772 to 773 2024-04-07T16:21:21.854 INFO:tasks.rados.rados.0.smithi100.stdout:1395: oid 5 updating version 773 to 774 2024-04-07T16:21:21.854 INFO:tasks.rados.rados.0.smithi100.stdout:1395: oid 5 version 774 is already newer than 770 2024-04-07T16:21:21.854 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 5 v 774 (ObjNum 429 snap 130 seq_num 429) dirty exists 2024-04-07T16:21:21.854 INFO:tasks.rados.rados.0.smithi100.stdout:1395: left oid 5 (ObjNum 429 snap 130 seq_num 429) 2024-04-07T16:21:21.854 INFO:tasks.rados.rados.0.smithi100.stdout:1395: done (2 left) 2024-04-07T16:21:21.854 INFO:tasks.rados.rados.0.smithi100.stdout:1398: copy_from oid 34 from oid 50 current snap is 130 2024-04-07T16:21:21.854 INFO:tasks.rados.rados.0.smithi100.stdout:1399: read oid 28 snap -1 2024-04-07T16:21:21.854 INFO:tasks.rados.rados.0.smithi100.stdout:1399: expect deleted 2024-04-07T16:21:21.854 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 22 current snap is 130 2024-04-07T16:21:21.854 INFO:tasks.rados.rados.0.smithi100.stdout:1400: seq_num 431 ranges {0=24576} 2024-04-07T16:21:21.854 INFO:tasks.rados.rados.0.smithi100.stdout:1400: writing smithi10066867-22 from 0 to 24576 tid 1 2024-04-07T16:21:21.854 INFO:tasks.rados.rados.0.smithi100.stdout:1401: copy_from oid 42 from oid 25 current snap is 130 2024-04-07T16:21:21.855 INFO:tasks.rados.rados.0.smithi100.stdout:1402: snap_remove snap 129 2024-04-07T16:21:22.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:21 smithi144 bash[66654]: cluster 2024-04-07T16:21:20.622390+0000 mgr.y (mgr.25175) 55 : cluster [DBG] pgmap v86: 121 pgs: 121 active+clean; 246 MiB data, 1.7 GiB used, 714 GiB / 715 GiB avail; 8.6 MiB/s rd, 5.1 MiB/s wr, 19 op/s 2024-04-07T16:21:22.121 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:21 smithi144 bash[66654]: cluster 2024-04-07T16:21:20.776496+0000 mon.a (mon.0) 222 : cluster [DBG] osdmap e332: 8 total, 8 up, 8 in 2024-04-07T16:21:22.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:21 smithi100 bash[74942]: cluster 2024-04-07T16:21:20.622390+0000 mgr.y (mgr.25175) 55 : cluster [DBG] pgmap v86: 121 pgs: 121 active+clean; 246 MiB data, 1.7 GiB used, 714 GiB / 715 GiB avail; 8.6 MiB/s rd, 5.1 MiB/s wr, 19 op/s 2024-04-07T16:21:22.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:21 smithi100 bash[74942]: cluster 2024-04-07T16:21:20.776496+0000 mon.a (mon.0) 222 : cluster [DBG] osdmap e332: 8 total, 8 up, 8 in 2024-04-07T16:21:22.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:21 smithi100 bash[76202]: cluster 2024-04-07T16:21:20.622390+0000 mgr.y (mgr.25175) 55 : cluster [DBG] pgmap v86: 121 pgs: 121 active+clean; 246 MiB data, 1.7 GiB used, 714 GiB / 715 GiB avail; 8.6 MiB/s rd, 5.1 MiB/s wr, 19 op/s 2024-04-07T16:21:22.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:21 smithi100 bash[76202]: cluster 2024-04-07T16:21:20.776496+0000 mon.a (mon.0) 222 : cluster [DBG] osdmap e332: 8 total, 8 up, 8 in 2024-04-07T16:21:22.605 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:21:22 smithi144 bash[59794]: ts=2024-04-07T16:21:22.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:21:22.787 INFO:tasks.rados.rados.0.smithi100.stdout:1402: done (6 left) 2024-04-07T16:21:22.788 INFO:tasks.rados.rados.0.smithi100.stdout:1403: snap_remove snap 128 2024-04-07T16:21:22.788 INFO:tasks.rados.rados.0.smithi100.stdout:1396: finishing write tid 1 to smithi10066867-10 2024-04-07T16:21:22.788 INFO:tasks.rados.rados.0.smithi100.stdout:1396: finishing write tid 2 to smithi10066867-10 2024-04-07T16:21:22.788 INFO:tasks.rados.rados.0.smithi100.stdout:1401: finishing copy_from to smithi10066867-42 2024-04-07T16:21:22.788 INFO:tasks.rados.rados.0.smithi100.stdout:1401: got expected ENOENT (src dne) 2024-04-07T16:21:22.788 INFO:tasks.rados.rados.0.smithi100.stdout:1401: finishing copy_from racing read to smithi10066867-42 2024-04-07T16:21:22.788 INFO:tasks.rados.rados.0.smithi100.stdout:1400: finishing write tid 1 to smithi10066867-22 2024-04-07T16:21:22.788 INFO:tasks.rados.rados.0.smithi100.stdout:1396: finishing write tid 3 to smithi10066867-10 2024-04-07T16:21:22.788 INFO:tasks.rados.rados.0.smithi100.stdout:1396: oid 10 updating version 0 to 824 2024-04-07T16:21:22.788 INFO:tasks.rados.rados.0.smithi100.stdout:1396: oid 10 updating version 824 to 825 2024-04-07T16:21:22.788 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 10 v 825 (ObjNum 430 snap 130 seq_num 430) dirty exists 2024-04-07T16:21:22.788 INFO:tasks.rados.rados.0.smithi100.stdout:1396: left oid 10 (ObjNum 430 snap 130 seq_num 430) 2024-04-07T16:21:22.788 INFO:tasks.rados.rados.0.smithi100.stdout:1400: finishing write tid 2 to smithi10066867-22 2024-04-07T16:21:22.788 INFO:tasks.rados.rados.0.smithi100.stdout:1400: finishing write tid 3 to smithi10066867-22 2024-04-07T16:21:22.789 INFO:tasks.rados.rados.0.smithi100.stdout:1400: oid 22 updating version 0 to 530 2024-04-07T16:21:22.789 INFO:tasks.rados.rados.0.smithi100.stdout:1400: oid 22 updating version 530 to 531 2024-04-07T16:21:22.789 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 22 v 531 (ObjNum 431 snap 130 seq_num 431) dirty exists 2024-04-07T16:21:22.789 INFO:tasks.rados.rados.0.smithi100.stdout:1400: left oid 22 (ObjNum 431 snap 130 seq_num 431) 2024-04-07T16:21:22.789 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 30 v 594 (ObjNum 424 snap 127 seq_num 424) dirty exists 2024-04-07T16:21:22.789 INFO:tasks.rados.rados.0.smithi100.stdout:1398: finishing copy_from racing read to smithi10066867-34 2024-04-07T16:21:22.789 INFO:tasks.rados.rados.0.smithi100.stdout:1398: finishing copy_from to smithi10066867-34 2024-04-07T16:21:22.789 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 34 v 525 (ObjNum 416 snap 126 seq_num 416) dirty exists 2024-04-07T16:21:23.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:22 smithi144 bash[66654]: cluster 2024-04-07T16:21:21.770674+0000 mon.a (mon.0) 223 : cluster [DBG] osdmap e333: 8 total, 8 up, 8 in 2024-04-07T16:21:23.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:22 smithi100 bash[74942]: cluster 2024-04-07T16:21:21.770674+0000 mon.a (mon.0) 223 : cluster [DBG] osdmap e333: 8 total, 8 up, 8 in 2024-04-07T16:21:23.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:22 smithi100 bash[76202]: cluster 2024-04-07T16:21:21.770674+0000 mon.a (mon.0) 223 : cluster [DBG] osdmap e333: 8 total, 8 up, 8 in 2024-04-07T16:21:23.782 INFO:tasks.rados.rados.0.smithi100.stdout:1396: done (6 left) 2024-04-07T16:21:23.782 INFO:tasks.rados.rados.0.smithi100.stdout:1397: done (5 left) 2024-04-07T16:21:23.782 INFO:tasks.rados.rados.0.smithi100.stdout:1398: done (4 left) 2024-04-07T16:21:23.782 INFO:tasks.rados.rados.0.smithi100.stdout:1399: done (3 left) 2024-04-07T16:21:23.782 INFO:tasks.rados.rados.0.smithi100.stdout:1400: done (2 left) 2024-04-07T16:21:23.782 INFO:tasks.rados.rados.0.smithi100.stdout:1401: done (1 left) 2024-04-07T16:21:23.782 INFO:tasks.rados.rados.0.smithi100.stdout:1403: done (0 left) 2024-04-07T16:21:23.782 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 12 current snap is 130 2024-04-07T16:21:23.782 INFO:tasks.rados.rados.0.smithi100.stdout:1404: seq_num 432 ranges {2260992=40960} 2024-04-07T16:21:23.788 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:21:23 smithi144 bash[59794]: ts=2024-04-07T16:21:23.508Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=nodes name=CephNodeDiskspaceWarning index=4 msg="Evaluating rule failed" rule="alert: CephNodeDiskspaceWarning\nexpr: predict_linear(node_filesystem_free_bytes{device=~\"/.*\"}[2d], 3600 * 24 * 5)\n * on (instance) group_left (nodename) node_uname_info < 0\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.8.4\n severity: warning\n type: ceph_default\nannotations:\n description: Mountpoint {{ $labels.mountpoint }} on {{ $labels.nodename }} will\n be full in less than 5 days based on the 48 hour trailing fill rate.\n summary: Host filesystem free space is getting low\n" err="found duplicate series for the match group {instance=\"smithi100\"} on the right hand-side of the operation: [{__name__=\"node_uname_info\", domainname=\"(none)\", instance=\"smithi100\", job=\"node\", machine=\"x86_64\", nodename=\"smithi100\", release=\"5.15.0-101-generic\", sysname=\"Linux\", version=\"#111-Ubuntu SMP Tue Mar 5 20:16:58 UTC 2024\"}, {__name__=\"node_uname_info\", cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", domainname=\"(none)\", instance=\"smithi100\", job=\"node\", machine=\"x86_64\", nodename=\"smithi100\", release=\"5.15.0-101-generic\", sysname=\"Linux\", version=\"#111-Ubuntu SMP Tue Mar 5 20:16:58 UTC 2024\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:21:23.799 INFO:tasks.rados.rados.0.smithi100.stdout:1404: writing smithi10066867-12 from 2260992 to 2301952 tid 1 2024-04-07T16:21:23.799 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 30 current snap is 130 2024-04-07T16:21:23.799 INFO:tasks.rados.rados.0.smithi100.stdout:1405: seq_num 433 ranges {8404992=57344} 2024-04-07T16:21:23.859 INFO:tasks.rados.rados.0.smithi100.stdout:1405: writing smithi10066867-30 from 8404992 to 8462336 tid 1 2024-04-07T16:21:23.859 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 48 current snap is 130 2024-04-07T16:21:23.859 INFO:tasks.rados.rados.0.smithi100.stdout:1404: finishing write tid 1 to smithi10066867-12 2024-04-07T16:21:23.859 INFO:tasks.rados.rados.0.smithi100.stdout:1404: finishing write tid 2 to smithi10066867-12 2024-04-07T16:21:23.859 INFO:tasks.rados.rados.0.smithi100.stdout:1404: finishing write tid 3 to smithi10066867-12 2024-04-07T16:21:23.860 INFO:tasks.rados.rados.0.smithi100.stdout:1404: oid 12 updating version 0 to 777 2024-04-07T16:21:23.860 INFO:tasks.rados.rados.0.smithi100.stdout:1404: oid 12 updating version 777 to 778 2024-04-07T16:21:23.860 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 12 v 778 (ObjNum 432 snap 130 seq_num 432) dirty exists 2024-04-07T16:21:23.860 INFO:tasks.rados.rados.0.smithi100.stdout:1404: left oid 12 (ObjNum 432 snap 130 seq_num 432) 2024-04-07T16:21:23.860 INFO:tasks.rados.rados.0.smithi100.stdout:1406: seq_num 434 ranges {1441792=73728} 2024-04-07T16:21:23.868 INFO:tasks.rados.rados.0.smithi100.stdout:1406: writing smithi10066867-48 from 1441792 to 1515520 tid 1 2024-04-07T16:21:23.868 INFO:tasks.rados.rados.0.smithi100.stdout:1404: done (2 left) 2024-04-07T16:21:23.869 INFO:tasks.rados.rados.0.smithi100.stdout:1407: copy_from oid 6 from oid 42 current snap is 130 2024-04-07T16:21:23.869 INFO:tasks.rados.rados.0.smithi100.stdout:1405: finishing write tid 1 to smithi10066867-30 2024-04-07T16:21:23.869 INFO:tasks.rados.rados.0.smithi100.stdout:1408: copy_from oid 2 from oid 14 current snap is 130 2024-04-07T16:21:23.869 INFO:tasks.rados.rados.0.smithi100.stdout:1405: finishing write tid 2 to smithi10066867-30 2024-04-07T16:21:23.869 INFO:tasks.rados.rados.0.smithi100.stdout:1405: finishing write tid 3 to smithi10066867-30 2024-04-07T16:21:23.869 INFO:tasks.rados.rados.0.smithi100.stdout:1405: oid 30 updating version 0 to 599 2024-04-07T16:21:23.869 INFO:tasks.rados.rados.0.smithi100.stdout:1405: oid 30 updating version 599 to 600 2024-04-07T16:21:23.869 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 30 v 600 (ObjNum 433 snap 130 seq_num 433) dirty exists 2024-04-07T16:21:23.869 INFO:tasks.rados.rados.0.smithi100.stdout:1405: left oid 30 (ObjNum 433 snap 130 seq_num 433) 2024-04-07T16:21:23.869 INFO:tasks.rados.rados.0.smithi100.stdout:1405: done (3 left) 2024-04-07T16:21:23.869 INFO:tasks.rados.rados.0.smithi100.stdout:1409: rollback oid 36 current snap is 130 2024-04-07T16:21:23.869 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 36 to 126 2024-04-07T16:21:23.869 INFO:tasks.rados.rados.0.smithi100.stdout:1410: read oid 40 snap -1 2024-04-07T16:21:23.869 INFO:tasks.rados.rados.0.smithi100.stdout:1410: expect (ObjNum 337 snap 101 seq_num 337) 2024-04-07T16:21:23.869 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 5 current snap is 130 2024-04-07T16:21:23.870 INFO:tasks.rados.rados.0.smithi100.stdout:1411: seq_num 435 ranges {4743168=90112} 2024-04-07T16:21:23.899 INFO:tasks.rados.rados.0.smithi100.stdout:1411: writing smithi10066867-5 from 4743168 to 4833280 tid 1 2024-04-07T16:21:23.899 INFO:tasks.rados.rados.0.smithi100.stdout:1412: rollback oid 34 current snap is 130 2024-04-07T16:21:23.899 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 34 to 126 2024-04-07T16:21:23.899 INFO:tasks.rados.rados.0.smithi100.stdout:1407: finishing copy_from to smithi10066867-6 2024-04-07T16:21:23.899 INFO:tasks.rados.rados.0.smithi100.stdout:1407: got expected ENOENT (src dne) 2024-04-07T16:21:23.899 INFO:tasks.rados.rados.0.smithi100.stdout:1407: finishing copy_from racing read to smithi10066867-6 2024-04-07T16:21:23.899 INFO:tasks.rados.rados.0.smithi100.stdout:1406: finishing write tid 1 to smithi10066867-48 2024-04-07T16:21:23.899 INFO:tasks.rados.rados.0.smithi100.stdout:1406: finishing write tid 2 to smithi10066867-48 2024-04-07T16:21:23.899 INFO:tasks.rados.rados.0.smithi100.stdout:1406: finishing write tid 3 to smithi10066867-48 2024-04-07T16:21:23.899 INFO:tasks.rados.rados.0.smithi100.stdout:1406: oid 48 updating version 0 to 579 2024-04-07T16:21:23.900 INFO:tasks.rados.rados.0.smithi100.stdout:1406: oid 48 updating version 579 to 580 2024-04-07T16:21:23.900 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 48 v 580 (ObjNum 434 snap 130 seq_num 434) dirty exists 2024-04-07T16:21:23.900 INFO:tasks.rados.rados.0.smithi100.stdout:1406: left oid 48 (ObjNum 434 snap 130 seq_num 434) 2024-04-07T16:21:23.900 INFO:tasks.rados.rados.0.smithi100.stdout:1406: done (6 left) 2024-04-07T16:21:23.900 INFO:tasks.rados.rados.0.smithi100.stdout:1407: done (5 left) 2024-04-07T16:21:23.900 INFO:tasks.rados.rados.0.smithi100.stdout:1413: snap_create 2024-04-07T16:21:23.900 INFO:tasks.rados.rados.0.smithi100.stdout:1409: finishing rollback tid 0 to smithi10066867-36 2024-04-07T16:21:23.900 INFO:tasks.rados.rados.0.smithi100.stdout:1409: finishing rollback tid 1 to smithi10066867-36 2024-04-07T16:21:23.900 INFO:tasks.rados.rados.0.smithi100.stdout:1409: finishing rollback tid 2 to smithi10066867-36 2024-04-07T16:21:23.900 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 36 v 690 (ObjNum 347 snap 104 seq_num 347) dirty exists 2024-04-07T16:21:23.900 INFO:tasks.rados.rados.0.smithi100.stdout:1408: finishing copy_from racing read to smithi10066867-2 2024-04-07T16:21:23.905 INFO:tasks.rados.rados.0.smithi100.stdout:1412: finishing rollback tid 0 to smithi10066867-34 2024-04-07T16:21:23.905 INFO:tasks.rados.rados.0.smithi100.stdout:1411: finishing write tid 1 to smithi10066867-5 2024-04-07T16:21:23.905 INFO:tasks.rados.rados.0.smithi100.stdout:1411: finishing write tid 2 to smithi10066867-5 2024-04-07T16:21:23.905 INFO:tasks.rados.rados.0.smithi100.stdout:1412: finishing rollback tid 1 to smithi10066867-34 2024-04-07T16:21:23.905 INFO:tasks.rados.rados.0.smithi100.stdout:1412: finishing rollback tid 2 to smithi10066867-34 2024-04-07T16:21:23.905 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 34 v 576 (ObjNum 405 snap 122 seq_num 405) dirty exists 2024-04-07T16:21:23.907 INFO:tasks.rados.rados.0.smithi100.stdout:1411: finishing write tid 3 to smithi10066867-5 2024-04-07T16:21:23.907 INFO:tasks.rados.rados.0.smithi100.stdout:1411: oid 5 updating version 0 to 783 2024-04-07T16:21:23.907 INFO:tasks.rados.rados.0.smithi100.stdout:1411: oid 5 updating version 783 to 784 2024-04-07T16:21:23.907 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 5 v 784 (ObjNum 435 snap 130 seq_num 435) dirty exists 2024-04-07T16:21:23.908 INFO:tasks.rados.rados.0.smithi100.stdout:1411: left oid 5 (ObjNum 435 snap 130 seq_num 435) 2024-04-07T16:21:23.914 INFO:tasks.rados.rados.0.smithi100.stdout:1410: expect (ObjNum 337 snap 101 seq_num 337) 2024-04-07T16:21:24.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:23 smithi144 bash[66654]: cluster 2024-04-07T16:21:22.623752+0000 mgr.y (mgr.25175) 56 : cluster [DBG] pgmap v89: 121 pgs: 121 active+clean; 183 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 11 MiB/s rd, 3.7 MiB/s wr, 13 op/s 2024-04-07T16:21:24.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:23 smithi144 bash[66654]: cluster 2024-04-07T16:21:22.802445+0000 mon.a (mon.0) 224 : cluster [DBG] osdmap e334: 8 total, 8 up, 8 in 2024-04-07T16:21:24.142 INFO:tasks.rados.rados.0.smithi100.stdout:1408: finishing copy_from to smithi10066867-2 2024-04-07T16:21:24.143 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 2 v 713 (ObjNum 342 snap 101 seq_num 342) dirty exists 2024-04-07T16:21:24.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:23 smithi100 bash[74942]: cluster 2024-04-07T16:21:22.623752+0000 mgr.y (mgr.25175) 56 : cluster [DBG] pgmap v89: 121 pgs: 121 active+clean; 183 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 11 MiB/s rd, 3.7 MiB/s wr, 13 op/s 2024-04-07T16:21:24.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:23 smithi100 bash[74942]: cluster 2024-04-07T16:21:22.802445+0000 mon.a (mon.0) 224 : cluster [DBG] osdmap e334: 8 total, 8 up, 8 in 2024-04-07T16:21:24.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:23 smithi100 bash[76202]: cluster 2024-04-07T16:21:22.623752+0000 mgr.y (mgr.25175) 56 : cluster [DBG] pgmap v89: 121 pgs: 121 active+clean; 183 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 11 MiB/s rd, 3.7 MiB/s wr, 13 op/s 2024-04-07T16:21:24.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:23 smithi100 bash[76202]: cluster 2024-04-07T16:21:22.802445+0000 mon.a (mon.0) 224 : cluster [DBG] osdmap e334: 8 total, 8 up, 8 in 2024-04-07T16:21:24.809 INFO:tasks.rados.rados.0.smithi100.stdout:1408: done (5 left) 2024-04-07T16:21:24.809 INFO:tasks.rados.rados.0.smithi100.stdout:1409: done (4 left) 2024-04-07T16:21:24.810 INFO:tasks.rados.rados.0.smithi100.stdout:1410: done (3 left) 2024-04-07T16:21:24.810 INFO:tasks.rados.rados.0.smithi100.stdout:1411: done (2 left) 2024-04-07T16:21:24.810 INFO:tasks.rados.rados.0.smithi100.stdout:1412: done (1 left) 2024-04-07T16:21:24.810 INFO:tasks.rados.rados.0.smithi100.stdout:1413: done (0 left) 2024-04-07T16:21:24.810 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 11 current snap is 131 2024-04-07T16:21:24.810 INFO:tasks.rados.rados.0.smithi100.stdout:1414: seq_num 436 ranges {5537792=106496} 2024-04-07T16:21:24.841 INFO:tasks.rados.rados.0.smithi100.stdout:1414: writing smithi10066867-11 from 5537792 to 5644288 tid 1 2024-04-07T16:21:24.841 INFO:tasks.rados.rados.0.smithi100.stdout:1415: rollback oid 15 current snap is 131 2024-04-07T16:21:24.842 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 15 to 126 2024-04-07T16:21:24.842 INFO:tasks.rados.rados.0.smithi100.stdout:1416: read oid 23 snap -1 2024-04-07T16:21:24.842 INFO:tasks.rados.rados.0.smithi100.stdout:1416: expect (ObjNum 395 snap 119 seq_num 395) 2024-04-07T16:21:24.842 INFO:tasks.rados.rados.0.smithi100.stdout:1417: rmattr oid 18 current snap is 131 2024-04-07T16:21:24.842 INFO:tasks.rados.rados.0.smithi100.stdout:1417: done (3 left) 2024-04-07T16:21:24.842 INFO:tasks.rados.rados.0.smithi100.stdout:1418: read oid 9 snap -1 2024-04-07T16:21:24.843 INFO:tasks.rados.rados.0.smithi100.stdout:1418: expect deleted 2024-04-07T16:21:24.843 INFO:tasks.rados.rados.0.smithi100.stdout:1419: snap_create 2024-04-07T16:21:24.850 INFO:tasks.rados.rados.0.smithi100.stdout:1414: finishing write tid 1 to smithi10066867-11 2024-04-07T16:21:24.853 INFO:tasks.rados.rados.0.smithi100.stdout:1414: finishing write tid 2 to smithi10066867-11 2024-04-07T16:21:24.855 INFO:tasks.rados.rados.0.smithi100.stdout:1415: finishing rollback tid 1 to smithi10066867-15 2024-04-07T16:21:24.855 INFO:tasks.rados.rados.0.smithi100.stdout:1414: finishing write tid 3 to smithi10066867-11 2024-04-07T16:21:24.855 INFO:tasks.rados.rados.0.smithi100.stdout:1414: oid 11 updating version 0 to 925 2024-04-07T16:21:24.856 INFO:tasks.rados.rados.0.smithi100.stdout:1414: oid 11 updating version 925 to 926 2024-04-07T16:21:24.856 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 11 v 926 (ObjNum 436 snap 131 seq_num 436) dirty exists 2024-04-07T16:21:24.856 INFO:tasks.rados.rados.0.smithi100.stdout:1414: left oid 11 (ObjNum 436 snap 131 seq_num 436) 2024-04-07T16:21:24.860 INFO:tasks.rados.rados.0.smithi100.stdout:1415: finishing rollback tid 2 to smithi10066867-15 2024-04-07T16:21:24.860 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 15 v 928 (ObjNum 338 snap 101 seq_num 338) dirty exists 2024-04-07T16:21:24.892 INFO:tasks.rados.rados.0.smithi100.stdout:1416: expect (ObjNum 395 snap 119 seq_num 395) 2024-04-07T16:21:25.090 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:24 smithi144 bash[66654]: cluster 2024-04-07T16:21:23.798376+0000 mon.a (mon.0) 225 : cluster [DBG] osdmap e335: 8 total, 8 up, 8 in 2024-04-07T16:21:25.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:24 smithi100 bash[74942]: cluster 2024-04-07T16:21:23.798376+0000 mon.a (mon.0) 225 : cluster [DBG] osdmap e335: 8 total, 8 up, 8 in 2024-04-07T16:21:25.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:24 smithi100 bash[76202]: cluster 2024-04-07T16:21:23.798376+0000 mon.a (mon.0) 225 : cluster [DBG] osdmap e335: 8 total, 8 up, 8 in 2024-04-07T16:21:25.808 INFO:tasks.rados.rados.0.smithi100.stdout:1414: done (4 left) 2024-04-07T16:21:25.808 INFO:tasks.rados.rados.0.smithi100.stdout:1415: done (3 left) 2024-04-07T16:21:25.809 INFO:tasks.rados.rados.0.smithi100.stdout:1416: done (2 left) 2024-04-07T16:21:25.809 INFO:tasks.rados.rados.0.smithi100.stdout:1418: done (1 left) 2024-04-07T16:21:25.809 INFO:tasks.rados.rados.0.smithi100.stdout:1419: done (0 left) 2024-04-07T16:21:25.809 INFO:tasks.rados.rados.0.smithi100.stdout:1420: read oid 49 snap -1 2024-04-07T16:21:25.809 INFO:tasks.rados.rados.0.smithi100.stdout:1420: expect (ObjNum 367 snap 110 seq_num 367) 2024-04-07T16:21:25.809 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 26 current snap is 132 2024-04-07T16:21:25.809 INFO:tasks.rados.rados.0.smithi100.stdout:1421: seq_num 437 ranges {2048000=122880} 2024-04-07T16:21:25.823 INFO:tasks.rados.rados.0.smithi100.stdout:1421: writing smithi10066867-26 from 2048000 to 2170880 tid 1 2024-04-07T16:21:25.823 INFO:tasks.rados.rados.0.smithi100.stdout:1422: snap_create 2024-04-07T16:21:25.836 INFO:tasks.rados.rados.0.smithi100.stdout:1421: finishing write tid 1 to smithi10066867-26 2024-04-07T16:21:25.836 INFO:tasks.rados.rados.0.smithi100.stdout:1421: finishing write tid 2 to smithi10066867-26 2024-04-07T16:21:25.838 INFO:tasks.rados.rados.0.smithi100.stdout:1420: expect (ObjNum 367 snap 110 seq_num 367) 2024-04-07T16:21:26.001 INFO:tasks.rados.rados.0.smithi100.stdout:1421: finishing write tid 3 to smithi10066867-26 2024-04-07T16:21:26.001 INFO:tasks.rados.rados.0.smithi100.stdout:1421: oid 26 updating version 0 to 532 2024-04-07T16:21:26.001 INFO:tasks.rados.rados.0.smithi100.stdout:1421: oid 26 updating version 532 to 533 2024-04-07T16:21:26.001 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 26 v 533 (ObjNum 437 snap 132 seq_num 437) dirty exists 2024-04-07T16:21:26.001 INFO:tasks.rados.rados.0.smithi100.stdout:1421: left oid 26 (ObjNum 437 snap 132 seq_num 437) 2024-04-07T16:21:26.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:25 smithi144 bash[66654]: cluster 2024-04-07T16:21:24.624476+0000 mgr.y (mgr.25175) 57 : cluster [DBG] pgmap v92: 121 pgs: 121 active+clean; 183 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 11 MiB/s rd, 3.7 MiB/s wr, 13 op/s 2024-04-07T16:21:26.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:25 smithi144 bash[66654]: cluster 2024-04-07T16:21:24.825366+0000 mon.a (mon.0) 226 : cluster [DBG] osdmap e336: 8 total, 8 up, 8 in 2024-04-07T16:21:26.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:25 smithi100 bash[76202]: cluster 2024-04-07T16:21:24.624476+0000 mgr.y (mgr.25175) 57 : cluster [DBG] pgmap v92: 121 pgs: 121 active+clean; 183 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 11 MiB/s rd, 3.7 MiB/s wr, 13 op/s 2024-04-07T16:21:26.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:25 smithi100 bash[76202]: cluster 2024-04-07T16:21:24.825366+0000 mon.a (mon.0) 226 : cluster [DBG] osdmap e336: 8 total, 8 up, 8 in 2024-04-07T16:21:26.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:25 smithi100 bash[74942]: cluster 2024-04-07T16:21:24.624476+0000 mgr.y (mgr.25175) 57 : cluster [DBG] pgmap v92: 121 pgs: 121 active+clean; 183 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 11 MiB/s rd, 3.7 MiB/s wr, 13 op/s 2024-04-07T16:21:26.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:25 smithi100 bash[74942]: cluster 2024-04-07T16:21:24.825366+0000 mon.a (mon.0) 226 : cluster [DBG] osdmap e336: 8 total, 8 up, 8 in 2024-04-07T16:21:26.694 INFO:teuthology.orchestra.run.smithi100.stdout:true 2024-04-07T16:21:26.753 INFO:tasks.rados.rados.0.smithi100.stdout:1420: done (2 left) 2024-04-07T16:21:26.754 INFO:tasks.rados.rados.0.smithi100.stdout:1421: done (1 left) 2024-04-07T16:21:26.754 INFO:tasks.rados.rados.0.smithi100.stdout:1422: done (0 left) 2024-04-07T16:21:26.754 INFO:tasks.rados.rados.0.smithi100.stdout:1423: snap_create 2024-04-07T16:21:27.063 INFO:teuthology.orchestra.run.smithi100.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-07T16:21:27.063 INFO:teuthology.orchestra.run.smithi100.stdout:alertmanager.a smithi100 *:9093,9094 running (7m) 34s ago 8m 14.0M - 0.25.0 c8568f914cd2 5143075e7d26 2024-04-07T16:21:27.063 INFO:teuthology.orchestra.run.smithi100.stdout:grafana.a smithi144 *:3000 running (7m) 9s ago 7m 79.3M - 9.4.7 954c08fa6188 7180c0708acd 2024-04-07T16:21:27.063 INFO:teuthology.orchestra.run.smithi100.stdout:mds.a smithi100 running (9m) 34s ago 9m 15.0M - 18.2.2-1171-g267c8ff5 8a4a01e7186c 92593b0c28ae 2024-04-07T16:21:27.063 INFO:teuthology.orchestra.run.smithi100.stdout:mds.b smithi144 running (9m) 9s ago 9m 18.3M - 18.2.2-1171-g267c8ff5 8a4a01e7186c 3df2ee3e17a4 2024-04-07T16:21:27.064 INFO:teuthology.orchestra.run.smithi100.stdout:mgr.x smithi144 *:8443,9283,8765 running (104s) 9s ago 16m 438M - 19.0.0-2785-gadce92d0 c0ecd4226ac9 233745c292a6 2024-04-07T16:21:27.064 INFO:teuthology.orchestra.run.smithi100.stdout:mgr.y smithi100 *:8443,9283,8765 running (2m) 34s ago 18m 495M - 19.0.0-2785-gadce92d0 c0ecd4226ac9 254dc65cdaa7 2024-04-07T16:21:27.064 INFO:teuthology.orchestra.run.smithi100.stdout:mon.a smithi100 running (81s) 34s ago 19m 38.4M 2048M 19.0.0-2785-gadce92d0 c0ecd4226ac9 718e12655c04 2024-04-07T16:21:27.064 INFO:teuthology.orchestra.run.smithi100.stdout:mon.b smithi144 running (17s) 9s ago 16m 18.2M 2048M 19.0.0-2785-gadce92d0 c0ecd4226ac9 aef817664bff 2024-04-07T16:21:27.064 INFO:teuthology.orchestra.run.smithi100.stdout:mon.c smithi100 running (46s) 34s ago 16m 20.1M 2048M 19.0.0-2785-gadce92d0 c0ecd4226ac9 63e889040f6e 2024-04-07T16:21:27.064 INFO:teuthology.orchestra.run.smithi100.stdout:node-exporter.a smithi100 *:9100 running (8m) 34s ago 8m 8248k - 1.5.0 0da6a335fe13 4bce264c1b26 2024-04-07T16:21:27.064 INFO:teuthology.orchestra.run.smithi100.stdout:node-exporter.b smithi144 *:9100 running (8m) 9s ago 8m 8443k - 1.5.0 0da6a335fe13 b6963413c560 2024-04-07T16:21:27.064 INFO:teuthology.orchestra.run.smithi100.stdout:osd.0 smithi100 running (15m) 34s ago 15m 240M 2524M 18.2.2-1171-g267c8ff5 8a4a01e7186c 188deb247979 2024-04-07T16:21:27.064 INFO:teuthology.orchestra.run.smithi100.stdout:osd.1 smithi100 running (14m) 34s ago 15m 196M 2524M 18.2.2-1171-g267c8ff5 8a4a01e7186c 7f57ca90894c 2024-04-07T16:21:27.064 INFO:teuthology.orchestra.run.smithi100.stdout:osd.2 smithi100 running (14m) 34s ago 14m 232M 2524M 18.2.2-1171-g267c8ff5 8a4a01e7186c f3912ddcc8cc 2024-04-07T16:21:27.064 INFO:teuthology.orchestra.run.smithi100.stdout:osd.3 smithi100 running (13m) 34s ago 13m 169M 2524M 18.2.2-1171-g267c8ff5 8a4a01e7186c b41004090c63 2024-04-07T16:21:27.064 INFO:teuthology.orchestra.run.smithi100.stdout:osd.4 smithi144 running (12m) 9s ago 12m 257M 2524M 18.2.2-1171-g267c8ff5 8a4a01e7186c 8c73b6b4dc13 2024-04-07T16:21:27.064 INFO:teuthology.orchestra.run.smithi100.stdout:osd.5 smithi144 running (11m) 9s ago 11m 216M 2524M 18.2.2-1171-g267c8ff5 8a4a01e7186c 3bfa0cf6f6cb 2024-04-07T16:21:27.064 INFO:teuthology.orchestra.run.smithi100.stdout:osd.6 smithi144 running (10m) 9s ago 10m 98.7M 2524M 18.2.2-1171-g267c8ff5 8a4a01e7186c 32b7f95d77f0 2024-04-07T16:21:27.064 INFO:teuthology.orchestra.run.smithi100.stdout:osd.7 smithi144 running (10m) 9s ago 10m 156M 2524M 18.2.2-1171-g267c8ff5 8a4a01e7186c 45f7709e576a 2024-04-07T16:21:27.064 INFO:teuthology.orchestra.run.smithi100.stdout:prometheus.a smithi144 *:9095 running (110s) 9s ago 8m 46.5M - 2.43.0 a07b618ecd1d 2d0a8131f303 2024-04-07T16:21:27.105 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:26 smithi144 bash[66654]: cluster 2024-04-07T16:21:25.824545+0000 mon.a (mon.0) 227 : cluster [DBG] osdmap e337: 8 total, 8 up, 8 in 2024-04-07T16:21:27.105 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:26 smithi144 bash[66654]: cluster 2024-04-07T16:21:26.770918+0000 mon.a (mon.0) 228 : cluster [DBG] osdmap e338: 8 total, 8 up, 8 in 2024-04-07T16:21:27.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:26 smithi100 bash[74942]: cluster 2024-04-07T16:21:25.824545+0000 mon.a (mon.0) 227 : cluster [DBG] osdmap e337: 8 total, 8 up, 8 in 2024-04-07T16:21:27.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:26 smithi100 bash[74942]: cluster 2024-04-07T16:21:26.770918+0000 mon.a (mon.0) 228 : cluster [DBG] osdmap e338: 8 total, 8 up, 8 in 2024-04-07T16:21:27.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:26 smithi100 bash[76202]: cluster 2024-04-07T16:21:25.824545+0000 mon.a (mon.0) 227 : cluster [DBG] osdmap e337: 8 total, 8 up, 8 in 2024-04-07T16:21:27.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:26 smithi100 bash[76202]: cluster 2024-04-07T16:21:26.770918+0000 mon.a (mon.0) 228 : cluster [DBG] osdmap e338: 8 total, 8 up, 8 in 2024-04-07T16:21:27.495 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-04-07T16:21:27.496 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": { 2024-04-07T16:21:27.496 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph version 19.0.0-2785-gadce92d0 (adce92d056533f22dc1ee37e3141d098b046ec1b) squid (dev)": 3 2024-04-07T16:21:27.496 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-07T16:21:27.496 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": { 2024-04-07T16:21:27.496 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph version 19.0.0-2785-gadce92d0 (adce92d056533f22dc1ee37e3141d098b046ec1b) squid (dev)": 2 2024-04-07T16:21:27.496 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-07T16:21:27.496 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": { 2024-04-07T16:21:27.496 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph version 18.2.2-1171-g267c8ff5 (267c8ff5b1ff5815581b3b2381d19360e081b37b) reef (stable)": 8 2024-04-07T16:21:27.496 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-07T16:21:27.496 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": { 2024-04-07T16:21:27.496 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph version 18.2.2-1171-g267c8ff5 (267c8ff5b1ff5815581b3b2381d19360e081b37b) reef (stable)": 2 2024-04-07T16:21:27.496 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-07T16:21:27.496 INFO:teuthology.orchestra.run.smithi100.stdout: "overall": { 2024-04-07T16:21:27.497 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph version 18.2.2-1171-g267c8ff5 (267c8ff5b1ff5815581b3b2381d19360e081b37b) reef (stable)": 10, 2024-04-07T16:21:27.497 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph version 19.0.0-2785-gadce92d0 (adce92d056533f22dc1ee37e3141d098b046ec1b) squid (dev)": 5 2024-04-07T16:21:27.497 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-04-07T16:21:27.497 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-04-07T16:21:27.760 INFO:tasks.rados.rados.0.smithi100.stdout:1423: done (0 left) 2024-04-07T16:21:27.761 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 44 current snap is 134 2024-04-07T16:21:27.761 INFO:tasks.rados.rados.0.smithi100.stdout:1424: seq_num 438 ranges {2613248=139264} 2024-04-07T16:21:27.781 INFO:tasks.rados.rados.0.smithi100.stdout:1424: writing smithi10066867-44 from 2613248 to 2752512 tid 1 2024-04-07T16:21:27.781 INFO:tasks.rados.rados.0.smithi100.stdout:1425: read oid 10 snap -1 2024-04-07T16:21:27.781 INFO:tasks.rados.rados.0.smithi100.stdout:1425: expect (ObjNum 430 snap 130 seq_num 430) 2024-04-07T16:21:27.781 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 28 current snap is 134 2024-04-07T16:21:27.781 INFO:tasks.rados.rados.0.smithi100.stdout:1426: seq_num 439 ranges {0=155648} 2024-04-07T16:21:27.782 INFO:tasks.rados.rados.0.smithi100.stdout:1426: writing smithi10066867-28 from 0 to 155648 tid 1 2024-04-07T16:21:27.782 INFO:tasks.rados.rados.0.smithi100.stdout:1427: rollback oid 26 current snap is 134 2024-04-07T16:21:27.783 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 26 to 133 2024-04-07T16:21:27.783 INFO:tasks.rados.rados.0.smithi100.stdout:1428: rollback oid 46 current snap is 134 2024-04-07T16:21:27.783 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 46 to 130 2024-04-07T16:21:27.783 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 19 current snap is 134 2024-04-07T16:21:27.783 INFO:tasks.rados.rados.0.smithi100.stdout:1429: seq_num 440 ranges {3866624=172032} 2024-04-07T16:21:27.812 INFO:tasks.rados.rados.0.smithi100.stdout:1429: writing smithi10066867-19 from 3866624 to 4038656 tid 1 2024-04-07T16:21:27.812 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 1 current snap is 134 2024-04-07T16:21:27.812 INFO:tasks.rados.rados.0.smithi100.stdout:1428: finishing rollback tid 0 to smithi10066867-46 2024-04-07T16:21:27.813 INFO:tasks.rados.rados.0.smithi100.stdout:1428: finishing rollback tid 1 to smithi10066867-46 2024-04-07T16:21:27.813 INFO:tasks.rados.rados.0.smithi100.stdout:1428: finishing rollback tid 2 to smithi10066867-46 2024-04-07T16:21:27.813 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 46 v 579 (ObjNum 337 snap 101 seq_num 337) dirty exists 2024-04-07T16:21:27.813 INFO:tasks.rados.rados.0.smithi100.stdout:1426: finishing write tid 1 to smithi10066867-28 2024-04-07T16:21:27.813 INFO:tasks.rados.rados.0.smithi100.stdout:1424: finishing write tid 1 to smithi10066867-44 2024-04-07T16:21:27.813 INFO:tasks.rados.rados.0.smithi100.stdout:1424: finishing write tid 2 to smithi10066867-44 2024-04-07T16:21:27.813 INFO:tasks.rados.rados.0.smithi100.stdout:1430: seq_num 441 ranges {4177920=188416} 2024-04-07T16:21:27.843 INFO:tasks.rados.rados.0.smithi100.stdout:1430: writing smithi10066867-1 from 4177920 to 4366336 tid 1 2024-04-07T16:21:27.843 INFO:tasks.rados.rados.0.smithi100.stdout:1428: done (6 left) 2024-04-07T16:21:27.843 INFO:tasks.rados.rados.0.smithi100.stdout:1431: copy_from oid 45 from oid 20 current snap is 134 2024-04-07T16:21:27.844 INFO:tasks.rados.rados.0.smithi100.stdout:1427: finishing rollback tid 0 to smithi10066867-26 2024-04-07T16:21:27.844 INFO:tasks.rados.rados.0.smithi100.stdout:1427: finishing rollback tid 1 to smithi10066867-26 2024-04-07T16:21:27.844 INFO:tasks.rados.rados.0.smithi100.stdout:1426: finishing write tid 2 to smithi10066867-28 2024-04-07T16:21:27.844 INFO:tasks.rados.rados.0.smithi100.stdout:1427: finishing rollback tid 2 to smithi10066867-26 2024-04-07T16:21:27.844 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 26 v 536 (ObjNum 437 snap 132 seq_num 437) dirty exists 2024-04-07T16:21:27.844 INFO:tasks.rados.rados.0.smithi100.stdout:1424: finishing write tid 3 to smithi10066867-44 2024-04-07T16:21:27.844 INFO:tasks.rados.rados.0.smithi100.stdout:1424: oid 44 updating version 0 to 714 2024-04-07T16:21:27.844 INFO:tasks.rados.rados.0.smithi100.stdout:1424: oid 44 updating version 714 to 715 2024-04-07T16:21:27.844 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 44 v 715 (ObjNum 438 snap 134 seq_num 438) dirty exists 2024-04-07T16:21:27.844 INFO:tasks.rados.rados.0.smithi100.stdout:1424: left oid 44 (ObjNum 438 snap 134 seq_num 438) 2024-04-07T16:21:27.844 INFO:tasks.rados.rados.0.smithi100.stdout:1424: done (6 left) 2024-04-07T16:21:27.844 INFO:tasks.rados.rados.0.smithi100.stdout:1427: done (5 left) 2024-04-07T16:21:27.844 INFO:tasks.rados.rados.0.smithi100.stdout:1432: read oid 41 snap 133 2024-04-07T16:21:27.844 INFO:tasks.rados.rados.0.smithi100.stdout:1432: expect (ObjNum 402 snap 121 seq_num 402) 2024-04-07T16:21:27.844 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 2 current snap is 134 2024-04-07T16:21:27.844 INFO:tasks.rados.rados.0.smithi100.stdout:1426: finishing write tid 3 to smithi10066867-28 2024-04-07T16:21:27.845 INFO:tasks.rados.rados.0.smithi100.stdout:1426: oid 28 updating version 0 to 222 2024-04-07T16:21:27.845 INFO:tasks.rados.rados.0.smithi100.stdout:1426: oid 28 version 222 is already newer than 221 2024-04-07T16:21:27.845 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 28 v 222 (ObjNum 439 snap 134 seq_num 439) dirty exists 2024-04-07T16:21:27.845 INFO:tasks.rados.rados.0.smithi100.stdout:1426: left oid 28 (ObjNum 439 snap 134 seq_num 439) 2024-04-07T16:21:27.845 INFO:tasks.rados.rados.0.smithi100.stdout:1433: seq_num 442 ranges {933888=204800} 2024-04-07T16:21:27.850 INFO:tasks.rados.rados.0.smithi100.stdout:1433: writing smithi10066867-2 from 933888 to 1138688 tid 1 2024-04-07T16:21:27.850 INFO:tasks.rados.rados.0.smithi100.stdout:1426: done (6 left) 2024-04-07T16:21:27.851 INFO:tasks.rados.rados.0.smithi100.stdout:1434: setattr oid 12 current snap is 134 2024-04-07T16:21:27.851 INFO:tasks.rados.rados.0.smithi100.stdout:1425: expect (ObjNum 430 snap 130 seq_num 430) 2024-04-07T16:21:28.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:27 smithi144 bash[66654]: cluster 2024-04-07T16:21:26.625125+0000 mgr.y (mgr.25175) 58 : cluster [DBG] pgmap v95: 121 pgs: 121 active+clean; 183 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail 2024-04-07T16:21:28.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:27 smithi144 bash[66654]: audit 2024-04-07T16:21:26.697907+0000 mgr.y (mgr.25175) 59 : audit [DBG] from='client.34203 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:21:28.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:27 smithi144 bash[66654]: audit 2024-04-07T16:21:27.060073+0000 mgr.y (mgr.25175) 60 : audit [DBG] from='client.34209 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:21:28.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:27 smithi144 bash[66654]: audit 2024-04-07T16:21:27.512508+0000 mon.a (mon.0) 229 : audit [DBG] from='client.? 172.21.15.100:0/2312440054' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:21:28.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:27 smithi144 bash[66654]: cluster 2024-04-07T16:21:27.775079+0000 mon.a (mon.0) 230 : cluster [DBG] osdmap e339: 8 total, 8 up, 8 in 2024-04-07T16:21:28.183 INFO:tasks.rados.rados.0.smithi100.stdout:1429: finishing write tid 1 to smithi10066867-19 2024-04-07T16:21:28.183 INFO:tasks.rados.rados.0.smithi100.stdout:1429: finishing write tid 2 to smithi10066867-19 2024-04-07T16:21:28.183 INFO:tasks.rados.rados.0.smithi100.stdout:1429: finishing write tid 3 to smithi10066867-19 2024-04-07T16:21:28.183 INFO:tasks.rados.rados.0.smithi100.stdout:1429: oid 19 updating version 0 to 692 2024-04-07T16:21:28.183 INFO:tasks.rados.rados.0.smithi100.stdout:1429: oid 19 version 692 is already newer than 691 2024-04-07T16:21:28.184 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 19 v 692 (ObjNum 440 snap 134 seq_num 440) dirty exists 2024-04-07T16:21:28.184 INFO:tasks.rados.rados.0.smithi100.stdout:1429: left oid 19 (ObjNum 440 snap 134 seq_num 440) 2024-04-07T16:21:28.184 INFO:tasks.rados.rados.0.smithi100.stdout:1430: finishing write tid 1 to smithi10066867-1 2024-04-07T16:21:28.184 INFO:tasks.rados.rados.0.smithi100.stdout:1430: finishing write tid 2 to smithi10066867-1 2024-04-07T16:21:28.184 INFO:tasks.rados.rados.0.smithi100.stdout:1432: expect (ObjNum 402 snap 121 seq_num 402) 2024-04-07T16:21:28.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:27 smithi100 bash[76202]: cluster 2024-04-07T16:21:26.625125+0000 mgr.y (mgr.25175) 58 : cluster [DBG] pgmap v95: 121 pgs: 121 active+clean; 183 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail 2024-04-07T16:21:28.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:27 smithi100 bash[76202]: audit 2024-04-07T16:21:26.697907+0000 mgr.y (mgr.25175) 59 : audit [DBG] from='client.34203 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:21:28.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:27 smithi100 bash[76202]: audit 2024-04-07T16:21:27.060073+0000 mgr.y (mgr.25175) 60 : audit [DBG] from='client.34209 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:21:28.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:27 smithi100 bash[76202]: audit 2024-04-07T16:21:27.512508+0000 mon.a (mon.0) 229 : audit [DBG] from='client.? 172.21.15.100:0/2312440054' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:21:28.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:27 smithi100 bash[76202]: cluster 2024-04-07T16:21:27.775079+0000 mon.a (mon.0) 230 : cluster [DBG] osdmap e339: 8 total, 8 up, 8 in 2024-04-07T16:21:28.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:27 smithi100 bash[74942]: cluster 2024-04-07T16:21:26.625125+0000 mgr.y (mgr.25175) 58 : cluster [DBG] pgmap v95: 121 pgs: 121 active+clean; 183 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail 2024-04-07T16:21:28.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:27 smithi100 bash[74942]: audit 2024-04-07T16:21:26.697907+0000 mgr.y (mgr.25175) 59 : audit [DBG] from='client.34203 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:21:28.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:27 smithi100 bash[74942]: audit 2024-04-07T16:21:27.060073+0000 mgr.y (mgr.25175) 60 : audit [DBG] from='client.34209 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:21:28.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:27 smithi100 bash[74942]: audit 2024-04-07T16:21:27.512508+0000 mon.a (mon.0) 229 : audit [DBG] from='client.? 172.21.15.100:0/2312440054' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:21:28.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:27 smithi100 bash[74942]: cluster 2024-04-07T16:21:27.775079+0000 mon.a (mon.0) 230 : cluster [DBG] osdmap e339: 8 total, 8 up, 8 in 2024-04-07T16:21:28.232 INFO:tasks.rados.rados.0.smithi100.stdout:1430: finishing write tid 3 to smithi10066867-1 2024-04-07T16:21:28.232 INFO:tasks.rados.rados.0.smithi100.stdout:1430: oid 1 updating version 0 to 827 2024-04-07T16:21:28.233 INFO:tasks.rados.rados.0.smithi100.stdout:1430: oid 1 version 827 is already newer than 826 2024-04-07T16:21:28.233 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 1 v 827 (ObjNum 441 snap 134 seq_num 441) dirty exists 2024-04-07T16:21:28.233 INFO:tasks.rados.rados.0.smithi100.stdout:1430: left oid 1 (ObjNum 441 snap 134 seq_num 441) 2024-04-07T16:21:28.233 INFO:tasks.rados.rados.0.smithi100.stdout:1433: finishing write tid 1 to smithi10066867-2 2024-04-07T16:21:28.233 INFO:tasks.rados.rados.0.smithi100.stdout:1433: finishing write tid 2 to smithi10066867-2 2024-04-07T16:21:28.233 INFO:tasks.rados.rados.0.smithi100.stdout:1433: finishing write tid 3 to smithi10066867-2 2024-04-07T16:21:28.233 INFO:tasks.rados.rados.0.smithi100.stdout:1433: oid 2 updating version 0 to 716 2024-04-07T16:21:28.233 INFO:tasks.rados.rados.0.smithi100.stdout:1433: oid 2 updating version 716 to 717 2024-04-07T16:21:28.233 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 2 v 717 (ObjNum 442 snap 134 seq_num 442) dirty exists 2024-04-07T16:21:28.233 INFO:tasks.rados.rados.0.smithi100.stdout:1433: left oid 2 (ObjNum 442 snap 134 seq_num 442) 2024-04-07T16:21:28.233 INFO:tasks.rados.rados.0.smithi100.stdout:1431: finishing copy_from racing read to smithi10066867-45 2024-04-07T16:21:28.233 INFO:tasks.rados.rados.0.smithi100.stdout:1431: finishing copy_from to smithi10066867-45 2024-04-07T16:21:28.233 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 45 v 528 (ObjNum 415 snap 126 seq_num 415) dirty exists 2024-04-07T16:21:28.233 INFO:tasks.rados.rados.0.smithi100.stdout:1425: done (6 left) 2024-04-07T16:21:28.234 INFO:tasks.rados.rados.0.smithi100.stdout:1429: done (5 left) 2024-04-07T16:21:28.234 INFO:tasks.rados.rados.0.smithi100.stdout:1430: done (4 left) 2024-04-07T16:21:28.234 INFO:tasks.rados.rados.0.smithi100.stdout:1431: done (3 left) 2024-04-07T16:21:28.234 INFO:tasks.rados.rados.0.smithi100.stdout:1432: done (2 left) 2024-04-07T16:21:28.234 INFO:tasks.rados.rados.0.smithi100.stdout:1433: done (1 left) 2024-04-07T16:21:28.234 INFO:tasks.rados.rados.0.smithi100.stdout:1435: snap_create 2024-04-07T16:21:28.237 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 12 v 786 (ObjNum 432 snap 130 seq_num 432) dirty exists 2024-04-07T16:21:28.685 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:21:28 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:21:28] "GET /metrics HTTP/1.1" 200 36077 "" "Prometheus/2.43.0" 2024-04-07T16:21:28.844 INFO:tasks.rados.rados.0.smithi100.stdout:1434: done (1 left) 2024-04-07T16:21:28.844 INFO:tasks.rados.rados.0.smithi100.stdout:1435: done (0 left) 2024-04-07T16:21:28.844 INFO:tasks.rados.rados.0.smithi100.stdout:1436: rmattr oid 41 current snap is 135 2024-04-07T16:21:28.844 INFO:tasks.rados.rados.0.smithi100.stdout:1436: done (0 left) 2024-04-07T16:21:28.844 INFO:tasks.rados.rados.0.smithi100.stdout:1437: snap_remove snap 130 2024-04-07T16:21:29.849 INFO:tasks.rados.rados.0.smithi100.stdout:1437: done (0 left) 2024-04-07T16:21:29.849 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 45 current snap is 135 2024-04-07T16:21:29.850 INFO:tasks.rados.rados.0.smithi100.stdout:1438: seq_num 443 ranges {3375104=221184} 2024-04-07T16:21:29.866 INFO:tasks.rados.rados.0.smithi100.stdout:1438: writing smithi10066867-45 from 3375104 to 3596288 tid 1 2024-04-07T16:21:29.866 INFO:tasks.rados.rados.0.smithi100.stdout:1439: delete oid 26 current snap is 135 2024-04-07T16:21:29.875 INFO:tasks.rados.rados.0.smithi100.stdout:1438: finishing write tid 1 to smithi10066867-45 2024-04-07T16:21:29.876 INFO:tasks.rados.rados.0.smithi100.stdout:1439: done (1 left) 2024-04-07T16:21:29.877 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 18 current snap is 135 2024-04-07T16:21:29.877 INFO:tasks.rados.rados.0.smithi100.stdout:1440: seq_num 445 ranges {12181504=253952} 2024-04-07T16:21:29.935 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:29 smithi100 bash[76202]: cluster 2024-04-07T16:21:28.625788+0000 mgr.y (mgr.25175) 61 : cluster [DBG] pgmap v98: 121 pgs: 121 active+clean; 186 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 11 MiB/s rd, 348 KiB/s wr, 9 op/s 2024-04-07T16:21:29.936 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:29 smithi100 bash[76202]: audit 2024-04-07T16:21:28.656147+0000 mon.a (mon.0) 231 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:21:29.936 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:29 smithi100 bash[76202]: audit 2024-04-07T16:21:28.669765+0000 mon.a (mon.0) 232 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:21:29.936 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:29 smithi100 bash[76202]: audit 2024-04-07T16:21:28.672312+0000 mon.a (mon.0) 233 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:21:29.936 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:29 smithi100 bash[76202]: audit 2024-04-07T16:21:28.674268+0000 mon.a (mon.0) 234 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:21:29.936 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:29 smithi100 bash[76202]: audit 2024-04-07T16:21:28.684634+0000 mon.a (mon.0) 235 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:21:29.936 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:29 smithi100 bash[76202]: cephadm 2024-04-07T16:21:28.761421+0000 mgr.y (mgr.25175) 62 : cephadm [INF] Reconfiguring mon.a (monmap changed)... 2024-04-07T16:21:29.936 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:29 smithi100 bash[76202]: audit 2024-04-07T16:21:28.762147+0000 mon.a (mon.0) 236 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-07T16:21:29.936 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:29 smithi100 bash[76202]: audit 2024-04-07T16:21:28.763629+0000 mon.a (mon.0) 237 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-07T16:21:29.936 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:29 smithi100 bash[76202]: audit 2024-04-07T16:21:28.764902+0000 mon.a (mon.0) 238 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:21:29.936 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:29 smithi100 bash[76202]: cephadm 2024-04-07T16:21:28.766773+0000 mgr.y (mgr.25175) 63 : cephadm [INF] Reconfiguring daemon mon.a on smithi100 2024-04-07T16:21:29.936 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:29 smithi100 bash[76202]: cluster 2024-04-07T16:21:28.861143+0000 mon.a (mon.0) 239 : cluster [DBG] osdmap e340: 8 total, 8 up, 8 in 2024-04-07T16:21:29.937 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:29 smithi100 bash[74942]: cluster 2024-04-07T16:21:28.625788+0000 mgr.y (mgr.25175) 61 : cluster [DBG] pgmap v98: 121 pgs: 121 active+clean; 186 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 11 MiB/s rd, 348 KiB/s wr, 9 op/s 2024-04-07T16:21:29.937 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:29 smithi100 bash[74942]: audit 2024-04-07T16:21:28.656147+0000 mon.a (mon.0) 231 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:21:29.937 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:29 smithi100 bash[74942]: audit 2024-04-07T16:21:28.669765+0000 mon.a (mon.0) 232 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:21:29.937 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:29 smithi100 bash[74942]: audit 2024-04-07T16:21:28.672312+0000 mon.a (mon.0) 233 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:21:29.937 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:29 smithi100 bash[74942]: audit 2024-04-07T16:21:28.674268+0000 mon.a (mon.0) 234 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:21:29.937 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:29 smithi100 bash[74942]: audit 2024-04-07T16:21:28.684634+0000 mon.a (mon.0) 235 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:21:29.937 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:29 smithi100 bash[74942]: cephadm 2024-04-07T16:21:28.761421+0000 mgr.y (mgr.25175) 62 : cephadm [INF] Reconfiguring mon.a (monmap changed)... 2024-04-07T16:21:29.937 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:29 smithi100 bash[74942]: audit 2024-04-07T16:21:28.762147+0000 mon.a (mon.0) 236 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-07T16:21:29.937 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:29 smithi100 bash[74942]: audit 2024-04-07T16:21:28.763629+0000 mon.a (mon.0) 237 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-07T16:21:29.937 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:29 smithi100 bash[74942]: audit 2024-04-07T16:21:28.764902+0000 mon.a (mon.0) 238 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:21:29.937 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:29 smithi100 bash[74942]: cephadm 2024-04-07T16:21:28.766773+0000 mgr.y (mgr.25175) 63 : cephadm [INF] Reconfiguring daemon mon.a on smithi100 2024-04-07T16:21:29.937 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:29 smithi100 bash[74942]: cluster 2024-04-07T16:21:28.861143+0000 mon.a (mon.0) 239 : cluster [DBG] osdmap e340: 8 total, 8 up, 8 in 2024-04-07T16:21:29.946 INFO:tasks.rados.rados.0.smithi100.stdout:1440: writing smithi10066867-18 from 12181504 to 12435456 tid 1 2024-04-07T16:21:29.946 INFO:tasks.rados.rados.0.smithi100.stdout:1441: delete oid 1 current snap is 135 2024-04-07T16:21:29.947 INFO:tasks.rados.rados.0.smithi100.stdout:1438: finishing write tid 2 to smithi10066867-45 2024-04-07T16:21:29.947 INFO:tasks.rados.rados.0.smithi100.stdout:1438: finishing write tid 3 to smithi10066867-45 2024-04-07T16:21:29.947 INFO:tasks.rados.rados.0.smithi100.stdout:1438: oid 45 updating version 0 to 714 2024-04-07T16:21:29.947 INFO:tasks.rados.rados.0.smithi100.stdout:1438: oid 45 updating version 714 to 715 2024-04-07T16:21:29.947 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 45 v 715 (ObjNum 443 snap 135 seq_num 443) dirty exists 2024-04-07T16:21:29.947 INFO:tasks.rados.rados.0.smithi100.stdout:1438: left oid 45 (ObjNum 443 snap 135 seq_num 443) 2024-04-07T16:21:29.951 INFO:tasks.rados.rados.0.smithi100.stdout:1438: done (2 left) 2024-04-07T16:21:29.951 INFO:tasks.rados.rados.0.smithi100.stdout:1441: done (1 left) 2024-04-07T16:21:29.951 INFO:tasks.rados.rados.0.smithi100.stdout:1442: copy_from oid 4 from oid 1 current snap is 135 2024-04-07T16:21:29.951 INFO:tasks.rados.rados.0.smithi100.stdout:1443: snap_create 2024-04-07T16:21:29.951 INFO:tasks.rados.rados.0.smithi100.stdout:1440: finishing write tid 1 to smithi10066867-18 2024-04-07T16:21:29.952 INFO:tasks.rados.rados.0.smithi100.stdout:1440: finishing write tid 2 to smithi10066867-18 2024-04-07T16:21:29.953 INFO:tasks.rados.rados.0.smithi100.stdout:1442: finishing copy_from to smithi10066867-4 2024-04-07T16:21:29.953 INFO:tasks.rados.rados.0.smithi100.stdout:1442: got expected ENOENT (src dne) 2024-04-07T16:21:29.954 INFO:tasks.rados.rados.0.smithi100.stdout:1442: finishing copy_from racing read to smithi10066867-4 2024-04-07T16:21:29.954 INFO:tasks.rados.rados.0.smithi100.stdout:1440: finishing write tid 3 to smithi10066867-18 2024-04-07T16:21:29.955 INFO:tasks.rados.rados.0.smithi100.stdout:1440: oid 18 updating version 0 to 933 2024-04-07T16:21:29.955 INFO:tasks.rados.rados.0.smithi100.stdout:1440: oid 18 version 933 is already newer than 932 2024-04-07T16:21:29.955 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 18 v 933 (ObjNum 445 snap 135 seq_num 445) dirty exists 2024-04-07T16:21:29.955 INFO:tasks.rados.rados.0.smithi100.stdout:1440: left oid 18 (ObjNum 445 snap 135 seq_num 445) 2024-04-07T16:21:30.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:29 smithi144 bash[66654]: cluster 2024-04-07T16:21:28.625788+0000 mgr.y (mgr.25175) 61 : cluster [DBG] pgmap v98: 121 pgs: 121 active+clean; 186 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 11 MiB/s rd, 348 KiB/s wr, 9 op/s 2024-04-07T16:21:30.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:29 smithi144 bash[66654]: audit 2024-04-07T16:21:28.656147+0000 mon.a (mon.0) 231 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:21:30.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:29 smithi144 bash[66654]: audit 2024-04-07T16:21:28.669765+0000 mon.a (mon.0) 232 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:21:30.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:29 smithi144 bash[66654]: audit 2024-04-07T16:21:28.672312+0000 mon.a (mon.0) 233 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:21:30.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:29 smithi144 bash[66654]: audit 2024-04-07T16:21:28.674268+0000 mon.a (mon.0) 234 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:21:30.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:29 smithi144 bash[66654]: audit 2024-04-07T16:21:28.684634+0000 mon.a (mon.0) 235 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:21:30.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:29 smithi144 bash[66654]: cephadm 2024-04-07T16:21:28.761421+0000 mgr.y (mgr.25175) 62 : cephadm [INF] Reconfiguring mon.a (monmap changed)... 2024-04-07T16:21:30.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:29 smithi144 bash[66654]: audit 2024-04-07T16:21:28.762147+0000 mon.a (mon.0) 236 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-07T16:21:30.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:29 smithi144 bash[66654]: audit 2024-04-07T16:21:28.763629+0000 mon.a (mon.0) 237 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-07T16:21:30.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:29 smithi144 bash[66654]: audit 2024-04-07T16:21:28.764902+0000 mon.a (mon.0) 238 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:21:30.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:29 smithi144 bash[66654]: cephadm 2024-04-07T16:21:28.766773+0000 mgr.y (mgr.25175) 63 : cephadm [INF] Reconfiguring daemon mon.a on smithi100 2024-04-07T16:21:30.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:29 smithi144 bash[66654]: cluster 2024-04-07T16:21:28.861143+0000 mon.a (mon.0) 239 : cluster [DBG] osdmap e340: 8 total, 8 up, 8 in 2024-04-07T16:21:30.855 INFO:tasks.rados.rados.0.smithi100.stdout:1440: done (2 left) 2024-04-07T16:21:30.856 INFO:tasks.rados.rados.0.smithi100.stdout:1442: done (1 left) 2024-04-07T16:21:30.856 INFO:tasks.rados.rados.0.smithi100.stdout:1443: done (0 left) 2024-04-07T16:21:30.856 INFO:tasks.rados.rados.0.smithi100.stdout:1444: setattr oid 25 current snap is 136 2024-04-07T16:21:30.857 INFO:tasks.rados.rados.0.smithi100.stdout:1445: setattr oid 47 current snap is 136 2024-04-07T16:21:30.859 INFO:tasks.rados.rados.0.smithi100.stdout:1446: snap_create 2024-04-07T16:21:30.861 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 25 v 538 (ObjNum 161 snap 0 seq_num 1) dirty exists 2024-04-07T16:21:30.869 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 47 v 529 (ObjNum 273 snap 0 seq_num 0) dirty exists 2024-04-07T16:21:31.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:30 smithi144 bash[66654]: cluster 2024-04-07T16:21:29.860743+0000 mon.a (mon.0) 240 : cluster [DBG] osdmap e341: 8 total, 8 up, 8 in 2024-04-07T16:21:31.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:30 smithi100 bash[74942]: cluster 2024-04-07T16:21:29.860743+0000 mon.a (mon.0) 240 : cluster [DBG] osdmap e341: 8 total, 8 up, 8 in 2024-04-07T16:21:31.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:30 smithi100 bash[76202]: cluster 2024-04-07T16:21:29.860743+0000 mon.a (mon.0) 240 : cluster [DBG] osdmap e341: 8 total, 8 up, 8 in 2024-04-07T16:21:31.757 INFO:tasks.rados.rados.0.smithi100.stdout:1444: done (2 left) 2024-04-07T16:21:31.757 INFO:tasks.rados.rados.0.smithi100.stdout:1445: done (1 left) 2024-04-07T16:21:31.757 INFO:tasks.rados.rados.0.smithi100.stdout:1446: done (0 left) 2024-04-07T16:21:31.758 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 7 current snap is 137 2024-04-07T16:21:31.758 INFO:tasks.rados.rados.0.smithi100.stdout:1447: seq_num 447 ranges {1441792=294912} 2024-04-07T16:21:31.768 INFO:tasks.rados.rados.0.smithi100.stdout:1447: writing smithi10066867-7 from 1441792 to 1736704 tid 1 2024-04-07T16:21:31.768 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 32 current snap is 137 2024-04-07T16:21:31.769 INFO:tasks.rados.rados.0.smithi100.stdout:1448: seq_num 448 ranges {1712128=311296} 2024-04-07T16:21:31.782 INFO:tasks.rados.rados.0.smithi100.stdout:1448: writing smithi10066867-32 from 1712128 to 2023424 tid 1 2024-04-07T16:21:31.783 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 34 current snap is 137 2024-04-07T16:21:31.783 INFO:tasks.rados.rados.0.smithi100.stdout:1447: finishing write tid 1 to smithi10066867-7 2024-04-07T16:21:31.783 INFO:tasks.rados.rados.0.smithi100.stdout:1449: seq_num 449 ranges {1990656=327680} 2024-04-07T16:21:31.798 INFO:tasks.rados.rados.0.smithi100.stdout:1449: writing smithi10066867-34 from 1990656 to 2318336 tid 1 2024-04-07T16:21:31.798 INFO:tasks.rados.rados.0.smithi100.stdout:1450: snap_create 2024-04-07T16:21:31.799 INFO:tasks.rados.rados.0.smithi100.stdout:1447: finishing write tid 2 to smithi10066867-7 2024-04-07T16:21:31.799 INFO:tasks.rados.rados.0.smithi100.stdout:1447: finishing write tid 3 to smithi10066867-7 2024-04-07T16:21:31.799 INFO:tasks.rados.rados.0.smithi100.stdout:1447: oid 7 updating version 0 to 533 2024-04-07T16:21:31.799 INFO:tasks.rados.rados.0.smithi100.stdout:1447: oid 7 version 533 is already newer than 532 2024-04-07T16:21:31.799 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 7 v 533 (ObjNum 447 snap 137 seq_num 447) dirty exists 2024-04-07T16:21:31.799 INFO:tasks.rados.rados.0.smithi100.stdout:1447: left oid 7 (ObjNum 447 snap 137 seq_num 447) 2024-04-07T16:21:31.799 INFO:tasks.rados.rados.0.smithi100.stdout:1448: finishing write tid 1 to smithi10066867-32 2024-04-07T16:21:31.799 INFO:tasks.rados.rados.0.smithi100.stdout:1448: finishing write tid 2 to smithi10066867-32 2024-04-07T16:21:31.799 INFO:tasks.rados.rados.0.smithi100.stdout:1448: finishing write tid 3 to smithi10066867-32 2024-04-07T16:21:31.799 INFO:tasks.rados.rados.0.smithi100.stdout:1448: oid 32 updating version 0 to 224 2024-04-07T16:21:31.799 INFO:tasks.rados.rados.0.smithi100.stdout:1448: oid 32 updating version 224 to 225 2024-04-07T16:21:31.799 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 32 v 225 (ObjNum 448 snap 137 seq_num 448) dirty exists 2024-04-07T16:21:31.799 INFO:tasks.rados.rados.0.smithi100.stdout:1448: left oid 32 (ObjNum 448 snap 137 seq_num 448) 2024-04-07T16:21:31.804 INFO:tasks.rados.rados.0.smithi100.stdout:1449: finishing write tid 1 to smithi10066867-34 2024-04-07T16:21:31.804 INFO:tasks.rados.rados.0.smithi100.stdout:1449: finishing write tid 2 to smithi10066867-34 2024-04-07T16:21:31.807 INFO:tasks.rados.rados.0.smithi100.stdout:1449: finishing write tid 3 to smithi10066867-34 2024-04-07T16:21:31.807 INFO:tasks.rados.rados.0.smithi100.stdout:1449: oid 34 updating version 0 to 580 2024-04-07T16:21:31.807 INFO:tasks.rados.rados.0.smithi100.stdout:1449: oid 34 updating version 580 to 581 2024-04-07T16:21:31.808 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 34 v 581 (ObjNum 449 snap 137 seq_num 449) dirty exists 2024-04-07T16:21:31.808 INFO:tasks.rados.rados.0.smithi100.stdout:1449: left oid 34 (ObjNum 449 snap 137 seq_num 449) 2024-04-07T16:21:32.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:31 smithi144 bash[66654]: cluster 2024-04-07T16:21:30.626494+0000 mgr.y (mgr.25175) 64 : cluster [DBG] pgmap v101: 121 pgs: 121 active+clean; 186 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 11 MiB/s rd, 348 KiB/s wr, 9 op/s 2024-04-07T16:21:32.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:31 smithi144 bash[66654]: cluster 2024-04-07T16:21:30.872314+0000 mon.a (mon.0) 241 : cluster [DBG] osdmap e342: 8 total, 8 up, 8 in 2024-04-07T16:21:32.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:31 smithi144 bash[66654]: audit 2024-04-07T16:21:30.912740+0000 mon.a (mon.0) 242 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:21:32.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:31 smithi144 bash[66654]: audit 2024-04-07T16:21:30.921348+0000 mon.a (mon.0) 243 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:21:32.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:31 smithi144 bash[66654]: cephadm 2024-04-07T16:21:30.935094+0000 mgr.y (mgr.25175) 65 : cephadm [INF] Reconfiguring mgr.y (monmap changed)... 2024-04-07T16:21:32.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:31 smithi144 bash[66654]: audit 2024-04-07T16:21:30.935547+0000 mon.a (mon.0) 244 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-07T16:21:32.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:31 smithi144 bash[66654]: audit 2024-04-07T16:21:30.936592+0000 mon.a (mon.0) 245 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-07T16:21:32.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:31 smithi144 bash[66654]: audit 2024-04-07T16:21:30.937524+0000 mon.a (mon.0) 246 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:21:32.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:31 smithi144 bash[66654]: cephadm 2024-04-07T16:21:30.938754+0000 mgr.y (mgr.25175) 66 : cephadm [INF] Reconfiguring daemon mgr.y on smithi100 2024-04-07T16:21:32.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:31 smithi144 bash[66654]: audit 2024-04-07T16:21:31.703642+0000 mon.a (mon.0) 247 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:21:32.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:31 smithi144 bash[66654]: cluster 2024-04-07T16:21:31.773690+0000 mon.a (mon.0) 248 : cluster [DBG] osdmap e343: 8 total, 8 up, 8 in 2024-04-07T16:21:32.119 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:31 smithi100 bash[74942]: cluster 2024-04-07T16:21:30.626494+0000 mgr.y (mgr.25175) 64 : cluster [DBG] pgmap v101: 121 pgs: 121 active+clean; 186 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 11 MiB/s rd, 348 KiB/s wr, 9 op/s 2024-04-07T16:21:32.119 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:31 smithi100 bash[74942]: cluster 2024-04-07T16:21:30.872314+0000 mon.a (mon.0) 241 : cluster [DBG] osdmap e342: 8 total, 8 up, 8 in 2024-04-07T16:21:32.119 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:31 smithi100 bash[74942]: audit 2024-04-07T16:21:30.912740+0000 mon.a (mon.0) 242 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:21:32.119 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:31 smithi100 bash[74942]: audit 2024-04-07T16:21:30.921348+0000 mon.a (mon.0) 243 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:21:32.119 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:31 smithi100 bash[74942]: cephadm 2024-04-07T16:21:30.935094+0000 mgr.y (mgr.25175) 65 : cephadm [INF] Reconfiguring mgr.y (monmap changed)... 2024-04-07T16:21:32.119 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:31 smithi100 bash[74942]: audit 2024-04-07T16:21:30.935547+0000 mon.a (mon.0) 244 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-07T16:21:32.119 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:31 smithi100 bash[74942]: audit 2024-04-07T16:21:30.936592+0000 mon.a (mon.0) 245 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-07T16:21:32.120 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:31 smithi100 bash[74942]: audit 2024-04-07T16:21:30.937524+0000 mon.a (mon.0) 246 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:21:32.120 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:31 smithi100 bash[74942]: cephadm 2024-04-07T16:21:30.938754+0000 mgr.y (mgr.25175) 66 : cephadm [INF] Reconfiguring daemon mgr.y on smithi100 2024-04-07T16:21:32.120 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:31 smithi100 bash[74942]: audit 2024-04-07T16:21:31.703642+0000 mon.a (mon.0) 247 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:21:32.120 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:31 smithi100 bash[74942]: cluster 2024-04-07T16:21:31.773690+0000 mon.a (mon.0) 248 : cluster [DBG] osdmap e343: 8 total, 8 up, 8 in 2024-04-07T16:21:32.120 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:31 smithi100 bash[76202]: cluster 2024-04-07T16:21:30.626494+0000 mgr.y (mgr.25175) 64 : cluster [DBG] pgmap v101: 121 pgs: 121 active+clean; 186 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 11 MiB/s rd, 348 KiB/s wr, 9 op/s 2024-04-07T16:21:32.120 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:31 smithi100 bash[76202]: cluster 2024-04-07T16:21:30.872314+0000 mon.a (mon.0) 241 : cluster [DBG] osdmap e342: 8 total, 8 up, 8 in 2024-04-07T16:21:32.120 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:31 smithi100 bash[76202]: audit 2024-04-07T16:21:30.912740+0000 mon.a (mon.0) 242 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:21:32.120 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:31 smithi100 bash[76202]: audit 2024-04-07T16:21:30.921348+0000 mon.a (mon.0) 243 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:21:32.120 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:31 smithi100 bash[76202]: cephadm 2024-04-07T16:21:30.935094+0000 mgr.y (mgr.25175) 65 : cephadm [INF] Reconfiguring mgr.y (monmap changed)... 2024-04-07T16:21:32.120 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:31 smithi100 bash[76202]: audit 2024-04-07T16:21:30.935547+0000 mon.a (mon.0) 244 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-07T16:21:32.121 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:31 smithi100 bash[76202]: audit 2024-04-07T16:21:30.936592+0000 mon.a (mon.0) 245 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-07T16:21:32.121 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:31 smithi100 bash[76202]: audit 2024-04-07T16:21:30.937524+0000 mon.a (mon.0) 246 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:21:32.121 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:31 smithi100 bash[76202]: cephadm 2024-04-07T16:21:30.938754+0000 mgr.y (mgr.25175) 66 : cephadm [INF] Reconfiguring daemon mgr.y on smithi100 2024-04-07T16:21:32.121 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:31 smithi100 bash[76202]: audit 2024-04-07T16:21:31.703642+0000 mon.a (mon.0) 247 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:21:32.121 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:31 smithi100 bash[76202]: cluster 2024-04-07T16:21:31.773690+0000 mon.a (mon.0) 248 : cluster [DBG] osdmap e343: 8 total, 8 up, 8 in 2024-04-07T16:21:32.606 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:21:32 smithi144 bash[59794]: ts=2024-04-07T16:21:32.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:21:32.759 INFO:tasks.rados.rados.0.smithi100.stdout:1447: done (3 left) 2024-04-07T16:21:32.759 INFO:tasks.rados.rados.0.smithi100.stdout:1448: done (2 left) 2024-04-07T16:21:32.760 INFO:tasks.rados.rados.0.smithi100.stdout:1449: done (1 left) 2024-04-07T16:21:32.760 INFO:tasks.rados.rados.0.smithi100.stdout:1450: done (0 left) 2024-04-07T16:21:32.760 INFO:tasks.rados.rados.0.smithi100.stdout:1451: delete oid 40 current snap is 138 2024-04-07T16:21:32.770 INFO:tasks.rados.rados.0.smithi100.stdout:1451: done (0 left) 2024-04-07T16:21:32.770 INFO:tasks.rados.rados.0.smithi100.stdout:1452: snap_remove snap 133 2024-04-07T16:21:33.756 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:21:33 smithi144 bash[59794]: ts=2024-04-07T16:21:33.507Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=nodes name=CephNodeDiskspaceWarning index=4 msg="Evaluating rule failed" rule="alert: CephNodeDiskspaceWarning\nexpr: predict_linear(node_filesystem_free_bytes{device=~\"/.*\"}[2d], 3600 * 24 * 5)\n * on (instance) group_left (nodename) node_uname_info < 0\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.8.4\n severity: warning\n type: ceph_default\nannotations:\n description: Mountpoint {{ $labels.mountpoint }} on {{ $labels.nodename }} will\n be full in less than 5 days based on the 48 hour trailing fill rate.\n summary: Host filesystem free space is getting low\n" err="found duplicate series for the match group {instance=\"smithi100\"} on the right hand-side of the operation: [{__name__=\"node_uname_info\", domainname=\"(none)\", instance=\"smithi100\", job=\"node\", machine=\"x86_64\", nodename=\"smithi100\", release=\"5.15.0-101-generic\", sysname=\"Linux\", version=\"#111-Ubuntu SMP Tue Mar 5 20:16:58 UTC 2024\"}, {__name__=\"node_uname_info\", cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", domainname=\"(none)\", instance=\"smithi100\", job=\"node\", machine=\"x86_64\", nodename=\"smithi100\", release=\"5.15.0-101-generic\", sysname=\"Linux\", version=\"#111-Ubuntu SMP Tue Mar 5 20:16:58 UTC 2024\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:21:33.772 INFO:tasks.rados.rados.0.smithi100.stdout:1452: done (0 left) 2024-04-07T16:21:33.772 INFO:tasks.rados.rados.0.smithi100.stdout:1453: snap_remove snap 131 2024-04-07T16:21:34.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:33 smithi144 bash[66654]: cluster 2024-04-07T16:21:32.627972+0000 mgr.y (mgr.25175) 67 : cluster [DBG] pgmap v104: 121 pgs: 121 active+clean; 191 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 4.6 MiB/s rd, 1.3 MiB/s wr, 26 op/s 2024-04-07T16:21:34.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:33 smithi144 bash[66654]: cluster 2024-04-07T16:21:32.776002+0000 mon.a (mon.0) 249 : cluster [DBG] osdmap e344: 8 total, 8 up, 8 in 2024-04-07T16:21:34.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:33 smithi144 bash[66654]: audit 2024-04-07T16:21:32.906290+0000 mon.a (mon.0) 250 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:21:34.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:33 smithi144 bash[66654]: audit 2024-04-07T16:21:32.916555+0000 mon.a (mon.0) 251 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:21:34.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:33 smithi144 bash[66654]: cephadm 2024-04-07T16:21:32.961896+0000 mgr.y (mgr.25175) 68 : cephadm [INF] Reconfiguring mon.c (monmap changed)... 2024-04-07T16:21:34.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:33 smithi144 bash[66654]: audit 2024-04-07T16:21:32.962556+0000 mon.a (mon.0) 252 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-07T16:21:34.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:33 smithi144 bash[66654]: audit 2024-04-07T16:21:32.964223+0000 mon.a (mon.0) 253 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-07T16:21:34.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:33 smithi144 bash[66654]: audit 2024-04-07T16:21:32.965761+0000 mon.a (mon.0) 254 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:21:34.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:33 smithi144 bash[66654]: cephadm 2024-04-07T16:21:32.967677+0000 mgr.y (mgr.25175) 69 : cephadm [INF] Reconfiguring daemon mon.c on smithi100 2024-04-07T16:21:34.111 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:33 smithi100 bash[74942]: cluster 2024-04-07T16:21:32.627972+0000 mgr.y (mgr.25175) 67 : cluster [DBG] pgmap v104: 121 pgs: 121 active+clean; 191 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 4.6 MiB/s rd, 1.3 MiB/s wr, 26 op/s 2024-04-07T16:21:34.111 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:33 smithi100 bash[74942]: cluster 2024-04-07T16:21:32.776002+0000 mon.a (mon.0) 249 : cluster [DBG] osdmap e344: 8 total, 8 up, 8 in 2024-04-07T16:21:34.111 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:33 smithi100 bash[74942]: audit 2024-04-07T16:21:32.906290+0000 mon.a (mon.0) 250 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:21:34.111 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:33 smithi100 bash[74942]: audit 2024-04-07T16:21:32.916555+0000 mon.a (mon.0) 251 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:21:34.111 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:33 smithi100 bash[74942]: cephadm 2024-04-07T16:21:32.961896+0000 mgr.y (mgr.25175) 68 : cephadm [INF] Reconfiguring mon.c (monmap changed)... 2024-04-07T16:21:34.111 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:33 smithi100 bash[74942]: audit 2024-04-07T16:21:32.962556+0000 mon.a (mon.0) 252 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-07T16:21:34.111 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:33 smithi100 bash[74942]: audit 2024-04-07T16:21:32.964223+0000 mon.a (mon.0) 253 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-07T16:21:34.111 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:33 smithi100 bash[74942]: audit 2024-04-07T16:21:32.965761+0000 mon.a (mon.0) 254 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:21:34.111 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:33 smithi100 bash[74942]: cephadm 2024-04-07T16:21:32.967677+0000 mgr.y (mgr.25175) 69 : cephadm [INF] Reconfiguring daemon mon.c on smithi100 2024-04-07T16:21:34.111 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:33 smithi100 bash[76202]: cluster 2024-04-07T16:21:32.627972+0000 mgr.y (mgr.25175) 67 : cluster [DBG] pgmap v104: 121 pgs: 121 active+clean; 191 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 4.6 MiB/s rd, 1.3 MiB/s wr, 26 op/s 2024-04-07T16:21:34.111 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:33 smithi100 bash[76202]: cluster 2024-04-07T16:21:32.776002+0000 mon.a (mon.0) 249 : cluster [DBG] osdmap e344: 8 total, 8 up, 8 in 2024-04-07T16:21:34.111 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:33 smithi100 bash[76202]: audit 2024-04-07T16:21:32.906290+0000 mon.a (mon.0) 250 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:21:34.112 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:33 smithi100 bash[76202]: audit 2024-04-07T16:21:32.916555+0000 mon.a (mon.0) 251 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:21:34.112 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:33 smithi100 bash[76202]: cephadm 2024-04-07T16:21:32.961896+0000 mgr.y (mgr.25175) 68 : cephadm [INF] Reconfiguring mon.c (monmap changed)... 2024-04-07T16:21:34.112 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:33 smithi100 bash[76202]: audit 2024-04-07T16:21:32.962556+0000 mon.a (mon.0) 252 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-07T16:21:34.112 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:33 smithi100 bash[76202]: audit 2024-04-07T16:21:32.964223+0000 mon.a (mon.0) 253 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-07T16:21:34.112 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:33 smithi100 bash[76202]: audit 2024-04-07T16:21:32.965761+0000 mon.a (mon.0) 254 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:21:34.112 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:33 smithi100 bash[76202]: cephadm 2024-04-07T16:21:32.967677+0000 mgr.y (mgr.25175) 69 : cephadm [INF] Reconfiguring daemon mon.c on smithi100 2024-04-07T16:21:34.780 INFO:tasks.rados.rados.0.smithi100.stdout:1453: done (0 left) 2024-04-07T16:21:34.780 INFO:tasks.rados.rados.0.smithi100.stdout:1454: snap_remove snap 126 2024-04-07T16:21:35.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:34 smithi144 bash[66654]: cluster 2024-04-07T16:21:33.787611+0000 mon.a (mon.0) 255 : cluster [DBG] osdmap e345: 8 total, 8 up, 8 in 2024-04-07T16:21:35.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:34 smithi100 bash[74942]: cluster 2024-04-07T16:21:33.787611+0000 mon.a (mon.0) 255 : cluster [DBG] osdmap e345: 8 total, 8 up, 8 in 2024-04-07T16:21:35.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:34 smithi100 bash[76202]: cluster 2024-04-07T16:21:33.787611+0000 mon.a (mon.0) 255 : cluster [DBG] osdmap e345: 8 total, 8 up, 8 in 2024-04-07T16:21:35.786 INFO:tasks.rados.rados.0.smithi100.stdout:1454: done (0 left) 2024-04-07T16:21:35.786 INFO:tasks.rados.rados.0.smithi100.stdout:1455: delete oid 34 current snap is 138 2024-04-07T16:21:35.805 INFO:tasks.rados.rados.0.smithi100.stdout:1455: done (0 left) 2024-04-07T16:21:35.805 INFO:tasks.rados.rados.0.smithi100.stdout:1456: read oid 33 snap -1 2024-04-07T16:21:35.805 INFO:tasks.rados.rados.0.smithi100.stdout:1456: expect (ObjNum 372 snap 112 seq_num 372) 2024-04-07T16:21:35.805 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 48 current snap is 138 2024-04-07T16:21:35.805 INFO:tasks.rados.rados.0.smithi100.stdout:1457: seq_num 452 ranges {1515520=376832} 2024-04-07T16:21:35.816 INFO:tasks.rados.rados.0.smithi100.stdout:1457: writing smithi10066867-48 from 1515520 to 1892352 tid 1 2024-04-07T16:21:35.816 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 5 current snap is 138 2024-04-07T16:21:35.816 INFO:tasks.rados.rados.0.smithi100.stdout:1458: seq_num 453 ranges {4833280=393216} 2024-04-07T16:21:35.848 INFO:tasks.rados.rados.0.smithi100.stdout:1458: writing smithi10066867-5 from 4833280 to 5226496 tid 1 2024-04-07T16:21:35.848 INFO:tasks.rados.rados.0.smithi100.stdout:1459: read oid 42 snap -1 2024-04-07T16:21:35.848 INFO:tasks.rados.rados.0.smithi100.stdout:1459: expect deleted 2024-04-07T16:21:35.848 INFO:tasks.rados.rados.0.smithi100.stdout:1460: delete oid 15 current snap is 138 2024-04-07T16:21:35.848 INFO:tasks.rados.rados.0.smithi100.stdout:1457: finishing write tid 1 to smithi10066867-48 2024-04-07T16:21:35.848 INFO:tasks.rados.rados.0.smithi100.stdout:1457: finishing write tid 2 to smithi10066867-48 2024-04-07T16:21:35.848 INFO:tasks.rados.rados.0.smithi100.stdout:1457: finishing write tid 3 to smithi10066867-48 2024-04-07T16:21:35.848 INFO:tasks.rados.rados.0.smithi100.stdout:1457: oid 48 updating version 0 to 581 2024-04-07T16:21:35.848 INFO:tasks.rados.rados.0.smithi100.stdout:1457: oid 48 updating version 581 to 582 2024-04-07T16:21:35.848 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 48 v 582 (ObjNum 452 snap 138 seq_num 452) dirty exists 2024-04-07T16:21:35.849 INFO:tasks.rados.rados.0.smithi100.stdout:1457: left oid 48 (ObjNum 452 snap 138 seq_num 452) 2024-04-07T16:21:35.849 INFO:tasks.rados.rados.0.smithi100.stdout:1456: expect (ObjNum 372 snap 112 seq_num 372) 2024-04-07T16:21:35.906 INFO:tasks.rados.rados.0.smithi100.stdout:1458: finishing write tid 1 to smithi10066867-5 2024-04-07T16:21:35.906 INFO:tasks.rados.rados.0.smithi100.stdout:1458: finishing write tid 2 to smithi10066867-5 2024-04-07T16:21:35.906 INFO:tasks.rados.rados.0.smithi100.stdout:1458: finishing write tid 3 to smithi10066867-5 2024-04-07T16:21:35.906 INFO:tasks.rados.rados.0.smithi100.stdout:1458: oid 5 updating version 0 to 797 2024-04-07T16:21:35.906 INFO:tasks.rados.rados.0.smithi100.stdout:1458: oid 5 version 797 is already newer than 796 2024-04-07T16:21:35.906 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 5 v 797 (ObjNum 453 snap 138 seq_num 453) dirty exists 2024-04-07T16:21:35.906 INFO:tasks.rados.rados.0.smithi100.stdout:1458: left oid 5 (ObjNum 453 snap 138 seq_num 453) 2024-04-07T16:21:35.906 INFO:tasks.rados.rados.0.smithi100.stdout:1456: done (4 left) 2024-04-07T16:21:35.906 INFO:tasks.rados.rados.0.smithi100.stdout:1457: done (3 left) 2024-04-07T16:21:35.906 INFO:tasks.rados.rados.0.smithi100.stdout:1458: done (2 left) 2024-04-07T16:21:35.906 INFO:tasks.rados.rados.0.smithi100.stdout:1459: done (1 left) 2024-04-07T16:21:35.906 INFO:tasks.rados.rados.0.smithi100.stdout:1460: done (0 left) 2024-04-07T16:21:35.907 INFO:tasks.rados.rados.0.smithi100.stdout:1461: rollback oid 19 current snap is 138 2024-04-07T16:21:35.907 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 19 to 135 2024-04-07T16:21:35.907 INFO:tasks.rados.rados.0.smithi100.stdout:1462: read oid 31 snap 134 2024-04-07T16:21:35.907 INFO:tasks.rados.rados.0.smithi100.stdout:1462: expect (ObjNum 423 snap 127 seq_num 423) 2024-04-07T16:21:35.907 INFO:tasks.rados.rados.0.smithi100.stdout:1463: rmattr oid 25 current snap is 138 2024-04-07T16:21:35.907 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 8 current snap is 138 2024-04-07T16:21:35.907 INFO:tasks.rados.rados.0.smithi100.stdout:1464: seq_num 455 ranges {2146304=425984} 2024-04-07T16:21:35.925 INFO:tasks.rados.rados.0.smithi100.stdout:1464: writing smithi10066867-8 from 2146304 to 2572288 tid 1 2024-04-07T16:21:35.925 INFO:tasks.rados.rados.0.smithi100.stdout:1465: snap_remove snap 137 2024-04-07T16:21:36.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:35 smithi144 bash[66654]: cluster 2024-04-07T16:21:34.628668+0000 mgr.y (mgr.25175) 70 : cluster [DBG] pgmap v107: 121 pgs: 121 active+clean; 191 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 4.6 MiB/s rd, 1.3 MiB/s wr, 26 op/s 2024-04-07T16:21:36.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:35 smithi144 bash[66654]: cluster 2024-04-07T16:21:34.797008+0000 mon.a (mon.0) 256 : cluster [DBG] osdmap e346: 8 total, 8 up, 8 in 2024-04-07T16:21:36.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:35 smithi144 bash[66654]: audit 2024-04-07T16:21:35.014045+0000 mon.a (mon.0) 257 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:21:36.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:35 smithi144 bash[66654]: audit 2024-04-07T16:21:35.024255+0000 mon.a (mon.0) 258 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:21:36.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:35 smithi144 bash[66654]: cephadm 2024-04-07T16:21:35.036287+0000 mgr.y (mgr.25175) 71 : cephadm [INF] Reconfiguring osd.0 (monmap changed)... 2024-04-07T16:21:36.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:35 smithi144 bash[66654]: audit 2024-04-07T16:21:35.036860+0000 mon.a (mon.0) 259 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-07T16:21:36.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:35 smithi144 bash[66654]: audit 2024-04-07T16:21:35.038595+0000 mon.a (mon.0) 260 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:21:36.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:35 smithi144 bash[66654]: cephadm 2024-04-07T16:21:35.043553+0000 mgr.y (mgr.25175) 72 : cephadm [INF] Reconfiguring daemon osd.0 on smithi100 2024-04-07T16:21:36.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:35 smithi100 bash[74942]: cluster 2024-04-07T16:21:34.628668+0000 mgr.y (mgr.25175) 70 : cluster [DBG] pgmap v107: 121 pgs: 121 active+clean; 191 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 4.6 MiB/s rd, 1.3 MiB/s wr, 26 op/s 2024-04-07T16:21:36.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:35 smithi100 bash[74942]: cluster 2024-04-07T16:21:34.797008+0000 mon.a (mon.0) 256 : cluster [DBG] osdmap e346: 8 total, 8 up, 8 in 2024-04-07T16:21:36.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:35 smithi100 bash[74942]: audit 2024-04-07T16:21:35.014045+0000 mon.a (mon.0) 257 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:21:36.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:35 smithi100 bash[74942]: audit 2024-04-07T16:21:35.024255+0000 mon.a (mon.0) 258 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:21:36.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:35 smithi100 bash[74942]: cephadm 2024-04-07T16:21:35.036287+0000 mgr.y (mgr.25175) 71 : cephadm [INF] Reconfiguring osd.0 (monmap changed)... 2024-04-07T16:21:36.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:35 smithi100 bash[74942]: audit 2024-04-07T16:21:35.036860+0000 mon.a (mon.0) 259 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-07T16:21:36.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:35 smithi100 bash[74942]: audit 2024-04-07T16:21:35.038595+0000 mon.a (mon.0) 260 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:21:36.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:35 smithi100 bash[74942]: cephadm 2024-04-07T16:21:35.043553+0000 mgr.y (mgr.25175) 72 : cephadm [INF] Reconfiguring daemon osd.0 on smithi100 2024-04-07T16:21:36.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:35 smithi100 bash[76202]: cluster 2024-04-07T16:21:34.628668+0000 mgr.y (mgr.25175) 70 : cluster [DBG] pgmap v107: 121 pgs: 121 active+clean; 191 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 4.6 MiB/s rd, 1.3 MiB/s wr, 26 op/s 2024-04-07T16:21:36.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:35 smithi100 bash[76202]: cluster 2024-04-07T16:21:34.797008+0000 mon.a (mon.0) 256 : cluster [DBG] osdmap e346: 8 total, 8 up, 8 in 2024-04-07T16:21:36.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:35 smithi100 bash[76202]: audit 2024-04-07T16:21:35.014045+0000 mon.a (mon.0) 257 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:21:36.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:35 smithi100 bash[76202]: audit 2024-04-07T16:21:35.024255+0000 mon.a (mon.0) 258 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:21:36.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:35 smithi100 bash[76202]: cephadm 2024-04-07T16:21:35.036287+0000 mgr.y (mgr.25175) 71 : cephadm [INF] Reconfiguring osd.0 (monmap changed)... 2024-04-07T16:21:36.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:35 smithi100 bash[76202]: audit 2024-04-07T16:21:35.036860+0000 mon.a (mon.0) 259 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-07T16:21:36.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:35 smithi100 bash[76202]: audit 2024-04-07T16:21:35.038595+0000 mon.a (mon.0) 260 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:21:36.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:35 smithi100 bash[76202]: cephadm 2024-04-07T16:21:35.043553+0000 mgr.y (mgr.25175) 72 : cephadm [INF] Reconfiguring daemon osd.0 on smithi100 2024-04-07T16:21:36.764 INFO:tasks.rados.rados.0.smithi100.stdout:1465: done (4 left) 2024-04-07T16:21:36.764 INFO:tasks.rados.rados.0.smithi100.stdout:1466: snap_remove snap 136 2024-04-07T16:21:36.935 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:36 smithi100 bash[74942]: cluster 2024-04-07T16:21:35.798786+0000 mon.a (mon.0) 261 : cluster [DBG] osdmap e347: 8 total, 8 up, 8 in 2024-04-07T16:21:36.935 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:36 smithi100 bash[74942]: cluster 2024-04-07T16:21:36.778131+0000 mon.a (mon.0) 262 : cluster [DBG] osdmap e348: 8 total, 8 up, 8 in 2024-04-07T16:21:36.936 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:36 smithi100 bash[76202]: cluster 2024-04-07T16:21:35.798786+0000 mon.a (mon.0) 261 : cluster [DBG] osdmap e347: 8 total, 8 up, 8 in 2024-04-07T16:21:36.936 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:36 smithi100 bash[76202]: cluster 2024-04-07T16:21:36.778131+0000 mon.a (mon.0) 262 : cluster [DBG] osdmap e348: 8 total, 8 up, 8 in 2024-04-07T16:21:37.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:36 smithi144 bash[66654]: cluster 2024-04-07T16:21:35.798786+0000 mon.a (mon.0) 261 : cluster [DBG] osdmap e347: 8 total, 8 up, 8 in 2024-04-07T16:21:37.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:36 smithi144 bash[66654]: cluster 2024-04-07T16:21:36.778131+0000 mon.a (mon.0) 262 : cluster [DBG] osdmap e348: 8 total, 8 up, 8 in 2024-04-07T16:21:37.766 INFO:tasks.rados.rados.0.smithi100.stdout:1466: done (4 left) 2024-04-07T16:21:37.766 INFO:tasks.rados.rados.0.smithi100.stdout:1467: rollback oid 1 current snap is 138 2024-04-07T16:21:37.766 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 1 to 132 2024-04-07T16:21:37.766 INFO:tasks.rados.rados.0.smithi100.stdout:1461: finishing rollback tid 0 to smithi10066867-19 2024-04-07T16:21:37.767 INFO:tasks.rados.rados.0.smithi100.stdout:1461: finishing rollback tid 1 to smithi10066867-19 2024-04-07T16:21:37.767 INFO:tasks.rados.rados.0.smithi100.stdout:1461: finishing rollback tid 2 to smithi10066867-19 2024-04-07T16:21:37.767 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 19 v 695 (ObjNum 440 snap 134 seq_num 440) dirty exists 2024-04-07T16:21:37.767 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 25 v 539 (ObjNum 465 snap 0 seq_num 4238694702) dirty exists 2024-04-07T16:21:37.767 INFO:tasks.rados.rados.0.smithi100.stdout:1462: expect (ObjNum 423 snap 127 seq_num 423) 2024-04-07T16:21:38.007 INFO:tasks.rados.rados.0.smithi100.stdout:1464: finishing write tid 1 to smithi10066867-8 2024-04-07T16:21:38.007 INFO:tasks.rados.rados.0.smithi100.stdout:1464: finishing write tid 2 to smithi10066867-8 2024-04-07T16:21:38.007 INFO:tasks.rados.rados.0.smithi100.stdout:1464: finishing write tid 3 to smithi10066867-8 2024-04-07T16:21:38.007 INFO:tasks.rados.rados.0.smithi100.stdout:1464: oid 8 updating version 0 to 590 2024-04-07T16:21:38.007 INFO:tasks.rados.rados.0.smithi100.stdout:1464: oid 8 updating version 590 to 591 2024-04-07T16:21:38.007 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 8 v 591 (ObjNum 455 snap 138 seq_num 455) dirty exists 2024-04-07T16:21:38.007 INFO:tasks.rados.rados.0.smithi100.stdout:1464: left oid 8 (ObjNum 455 snap 138 seq_num 455) 2024-04-07T16:21:38.007 INFO:tasks.rados.rados.0.smithi100.stdout:1467: finishing rollback tid 1 to smithi10066867-1 2024-04-07T16:21:38.008 INFO:tasks.rados.rados.0.smithi100.stdout:1467: finishing rollback tid 2 to smithi10066867-1 2024-04-07T16:21:38.008 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 1 v 830 (ObjNum 418 snap 126 seq_num 418) dirty exists 2024-04-07T16:21:38.008 INFO:tasks.rados.rados.0.smithi100.stdout:1461: done (4 left) 2024-04-07T16:21:38.008 INFO:tasks.rados.rados.0.smithi100.stdout:1462: done (3 left) 2024-04-07T16:21:38.008 INFO:tasks.rados.rados.0.smithi100.stdout:1463: done (2 left) 2024-04-07T16:21:38.008 INFO:tasks.rados.rados.0.smithi100.stdout:1464: done (1 left) 2024-04-07T16:21:38.008 INFO:tasks.rados.rados.0.smithi100.stdout:1467: done (0 left) 2024-04-07T16:21:38.008 INFO:tasks.rados.rados.0.smithi100.stdout:1468: read oid 16 snap -1 2024-04-07T16:21:38.008 INFO:tasks.rados.rados.0.smithi100.stdout:1468: expect (ObjNum 374 snap 113 seq_num 374) 2024-04-07T16:21:38.008 INFO:tasks.rados.rados.0.smithi100.stdout:1469: copy_from oid 1 from oid 37 current snap is 138 2024-04-07T16:21:38.008 INFO:tasks.rados.rados.0.smithi100.stdout:1470: read oid 40 snap -1 2024-04-07T16:21:38.008 INFO:tasks.rados.rados.0.smithi100.stdout:1470: expect deleted 2024-04-07T16:21:38.008 INFO:tasks.rados.rados.0.smithi100.stdout:1471: read oid 13 snap -1 2024-04-07T16:21:38.008 INFO:tasks.rados.rados.0.smithi100.stdout:1471: expect (ObjNum 393 snap 119 seq_num 393) 2024-04-07T16:21:38.009 INFO:tasks.rados.rados.0.smithi100.stdout:1472: rmattr oid 45 current snap is 138 2024-04-07T16:21:38.009 INFO:tasks.rados.rados.0.smithi100.stdout:1472: done (4 left) 2024-04-07T16:21:38.009 INFO:tasks.rados.rados.0.smithi100.stdout:1473: snap_create 2024-04-07T16:21:38.010 INFO:tasks.rados.rados.0.smithi100.stdout:1469: finishing copy_from racing read to smithi10066867-1 2024-04-07T16:21:38.011 INFO:tasks.rados.rados.0.smithi100.stdout:1469: finishing copy_from to smithi10066867-1 2024-04-07T16:21:38.012 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 1 v 479 (ObjNum 273 snap 0 seq_num 0) dirty exists 2024-04-07T16:21:38.026 INFO:tasks.rados.rados.0.smithi100.stdout:1468: expect (ObjNum 374 snap 113 seq_num 374) 2024-04-07T16:21:38.185 INFO:tasks.rados.rados.0.smithi100.stdout:1471: expect (ObjNum 393 snap 119 seq_num 393) 2024-04-07T16:21:38.494 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:21:38 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:21:38] "GET /metrics HTTP/1.1" 200 36074 "" "Prometheus/2.43.0" 2024-04-07T16:21:38.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:38 smithi100 bash[74942]: cluster 2024-04-07T16:21:36.629357+0000 mgr.y (mgr.25175) 73 : cluster [DBG] pgmap v110: 121 pgs: 121 active+clean; 191 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail 2024-04-07T16:21:38.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:38 smithi100 bash[74942]: audit 2024-04-07T16:21:37.240319+0000 mon.a (mon.0) 263 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:21:38.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:38 smithi100 bash[74942]: audit 2024-04-07T16:21:37.248867+0000 mon.a (mon.0) 264 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:21:38.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:38 smithi100 bash[74942]: cephadm 2024-04-07T16:21:37.262685+0000 mgr.y (mgr.25175) 74 : cephadm [INF] Reconfiguring osd.1 (monmap changed)... 2024-04-07T16:21:38.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:38 smithi100 bash[74942]: audit 2024-04-07T16:21:37.263290+0000 mon.a (mon.0) 265 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-07T16:21:38.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:38 smithi100 bash[74942]: audit 2024-04-07T16:21:37.264500+0000 mon.a (mon.0) 266 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:21:38.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:38 smithi100 bash[74942]: cephadm 2024-04-07T16:21:37.267404+0000 mgr.y (mgr.25175) 75 : cephadm [INF] Reconfiguring daemon osd.1 on smithi100 2024-04-07T16:21:38.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:38 smithi100 bash[74942]: cluster 2024-04-07T16:21:37.782976+0000 mon.a (mon.0) 267 : cluster [DBG] osdmap e349: 8 total, 8 up, 8 in 2024-04-07T16:21:38.495 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:38 smithi100 bash[76202]: cluster 2024-04-07T16:21:36.629357+0000 mgr.y (mgr.25175) 73 : cluster [DBG] pgmap v110: 121 pgs: 121 active+clean; 191 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail 2024-04-07T16:21:38.496 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:38 smithi100 bash[76202]: audit 2024-04-07T16:21:37.240319+0000 mon.a (mon.0) 263 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:21:38.496 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:38 smithi100 bash[76202]: audit 2024-04-07T16:21:37.248867+0000 mon.a (mon.0) 264 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:21:38.496 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:38 smithi100 bash[76202]: cephadm 2024-04-07T16:21:37.262685+0000 mgr.y (mgr.25175) 74 : cephadm [INF] Reconfiguring osd.1 (monmap changed)... 2024-04-07T16:21:38.496 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:38 smithi100 bash[76202]: audit 2024-04-07T16:21:37.263290+0000 mon.a (mon.0) 265 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-07T16:21:38.496 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:38 smithi100 bash[76202]: audit 2024-04-07T16:21:37.264500+0000 mon.a (mon.0) 266 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:21:38.496 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:38 smithi100 bash[76202]: cephadm 2024-04-07T16:21:37.267404+0000 mgr.y (mgr.25175) 75 : cephadm [INF] Reconfiguring daemon osd.1 on smithi100 2024-04-07T16:21:38.496 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:38 smithi100 bash[76202]: cluster 2024-04-07T16:21:37.782976+0000 mon.a (mon.0) 267 : cluster [DBG] osdmap e349: 8 total, 8 up, 8 in 2024-04-07T16:21:38.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:38 smithi144 bash[66654]: cluster 2024-04-07T16:21:36.629357+0000 mgr.y (mgr.25175) 73 : cluster [DBG] pgmap v110: 121 pgs: 121 active+clean; 191 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail 2024-04-07T16:21:38.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:38 smithi144 bash[66654]: audit 2024-04-07T16:21:37.240319+0000 mon.a (mon.0) 263 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:21:38.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:38 smithi144 bash[66654]: audit 2024-04-07T16:21:37.248867+0000 mon.a (mon.0) 264 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:21:38.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:38 smithi144 bash[66654]: cephadm 2024-04-07T16:21:37.262685+0000 mgr.y (mgr.25175) 74 : cephadm [INF] Reconfiguring osd.1 (monmap changed)... 2024-04-07T16:21:38.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:38 smithi144 bash[66654]: audit 2024-04-07T16:21:37.263290+0000 mon.a (mon.0) 265 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-07T16:21:38.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:38 smithi144 bash[66654]: audit 2024-04-07T16:21:37.264500+0000 mon.a (mon.0) 266 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:21:38.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:38 smithi144 bash[66654]: cephadm 2024-04-07T16:21:37.267404+0000 mgr.y (mgr.25175) 75 : cephadm [INF] Reconfiguring daemon osd.1 on smithi100 2024-04-07T16:21:38.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:38 smithi144 bash[66654]: cluster 2024-04-07T16:21:37.782976+0000 mon.a (mon.0) 267 : cluster [DBG] osdmap e349: 8 total, 8 up, 8 in 2024-04-07T16:21:38.990 INFO:tasks.rados.rados.0.smithi100.stdout:1468: done (4 left) 2024-04-07T16:21:38.990 INFO:tasks.rados.rados.0.smithi100.stdout:1469: done (3 left) 2024-04-07T16:21:38.990 INFO:tasks.rados.rados.0.smithi100.stdout:1470: done (2 left) 2024-04-07T16:21:38.991 INFO:tasks.rados.rados.0.smithi100.stdout:1471: done (1 left) 2024-04-07T16:21:38.991 INFO:tasks.rados.rados.0.smithi100.stdout:1473: done (0 left) 2024-04-07T16:21:38.991 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 35 current snap is 139 2024-04-07T16:21:38.991 INFO:tasks.rados.rados.0.smithi100.stdout:1474: seq_num 456 ranges {4399104=442368} 2024-04-07T16:21:39.061 INFO:tasks.rados.rados.0.smithi100.stdout:1474: writing smithi10066867-35 from 4399104 to 4841472 tid 1 2024-04-07T16:21:39.061 INFO:tasks.rados.rados.0.smithi100.stdout:1475: delete oid 36 current snap is 139 2024-04-07T16:21:39.073 INFO:tasks.rados.rados.0.smithi100.stdout:1474: finishing write tid 1 to smithi10066867-35 2024-04-07T16:21:39.074 INFO:tasks.rados.rados.0.smithi100.stdout:1475: done (1 left) 2024-04-07T16:21:39.074 INFO:tasks.rados.rados.0.smithi100.stdout:1476: read oid 3 snap -1 2024-04-07T16:21:39.074 INFO:tasks.rados.rados.0.smithi100.stdout:1476: expect (ObjNum 401 snap 121 seq_num 401) 2024-04-07T16:21:39.074 INFO:tasks.rados.rados.0.smithi100.stdout:1477: delete oid 13 current snap is 139 2024-04-07T16:21:39.074 INFO:tasks.rados.rados.0.smithi100.stdout:1474: finishing write tid 2 to smithi10066867-35 2024-04-07T16:21:39.077 INFO:tasks.rados.rados.0.smithi100.stdout:1474: finishing write tid 3 to smithi10066867-35 2024-04-07T16:21:39.077 INFO:tasks.rados.rados.0.smithi100.stdout:1474: oid 35 updating version 0 to 584 2024-04-07T16:21:39.077 INFO:tasks.rados.rados.0.smithi100.stdout:1474: oid 35 version 584 is already newer than 583 2024-04-07T16:21:39.077 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 35 v 584 (ObjNum 456 snap 139 seq_num 456) dirty exists 2024-04-07T16:21:39.077 INFO:tasks.rados.rados.0.smithi100.stdout:1474: left oid 35 (ObjNum 456 snap 139 seq_num 456) 2024-04-07T16:21:39.079 INFO:tasks.rados.rados.0.smithi100.stdout:1474: done (2 left) 2024-04-07T16:21:39.079 INFO:tasks.rados.rados.0.smithi100.stdout:1477: done (1 left) 2024-04-07T16:21:39.079 INFO:tasks.rados.rados.0.smithi100.stdout:1478: snap_remove snap 135 2024-04-07T16:21:39.776 INFO:tasks.rados.rados.0.smithi100.stdout:1478: done (1 left) 2024-04-07T16:21:39.776 INFO:tasks.rados.rados.0.smithi100.stdout:1479: read oid 34 snap -1 2024-04-07T16:21:39.777 INFO:tasks.rados.rados.0.smithi100.stdout:1479: expect deleted 2024-04-07T16:21:39.777 INFO:tasks.rados.rados.0.smithi100.stdout:1480: snap_create 2024-04-07T16:21:39.777 INFO:tasks.rados.rados.0.smithi100.stdout:1476: expect (ObjNum 401 snap 121 seq_num 401) 2024-04-07T16:21:40.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:39 smithi144 bash[66654]: cluster 2024-04-07T16:21:38.629771+0000 mgr.y (mgr.25175) 76 : cluster [DBG] pgmap v113: 121 pgs: 121 active+clean; 165 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 4.8 MiB/s rd, 292 KiB/s wr, 7 op/s 2024-04-07T16:21:40.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:39 smithi144 bash[66654]: cluster 2024-04-07T16:21:38.780901+0000 mon.a (mon.0) 268 : cluster [DBG] osdmap e350: 8 total, 8 up, 8 in 2024-04-07T16:21:40.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:39 smithi144 bash[66654]: audit 2024-04-07T16:21:39.375252+0000 mon.a (mon.0) 269 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:21:40.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:39 smithi144 bash[66654]: audit 2024-04-07T16:21:39.381231+0000 mon.a (mon.0) 270 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:21:40.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:39 smithi144 bash[66654]: audit 2024-04-07T16:21:39.390282+0000 mon.a (mon.0) 271 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-07T16:21:40.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:39 smithi144 bash[66654]: audit 2024-04-07T16:21:39.391091+0000 mon.a (mon.0) 272 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:21:40.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:39 smithi100 bash[74942]: cluster 2024-04-07T16:21:38.629771+0000 mgr.y (mgr.25175) 76 : cluster [DBG] pgmap v113: 121 pgs: 121 active+clean; 165 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 4.8 MiB/s rd, 292 KiB/s wr, 7 op/s 2024-04-07T16:21:40.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:39 smithi100 bash[74942]: cluster 2024-04-07T16:21:38.780901+0000 mon.a (mon.0) 268 : cluster [DBG] osdmap e350: 8 total, 8 up, 8 in 2024-04-07T16:21:40.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:39 smithi100 bash[74942]: audit 2024-04-07T16:21:39.375252+0000 mon.a (mon.0) 269 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:21:40.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:39 smithi100 bash[74942]: audit 2024-04-07T16:21:39.381231+0000 mon.a (mon.0) 270 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:21:40.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:39 smithi100 bash[74942]: audit 2024-04-07T16:21:39.390282+0000 mon.a (mon.0) 271 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-07T16:21:40.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:39 smithi100 bash[74942]: audit 2024-04-07T16:21:39.391091+0000 mon.a (mon.0) 272 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:21:40.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:39 smithi100 bash[76202]: cluster 2024-04-07T16:21:38.629771+0000 mgr.y (mgr.25175) 76 : cluster [DBG] pgmap v113: 121 pgs: 121 active+clean; 165 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 4.8 MiB/s rd, 292 KiB/s wr, 7 op/s 2024-04-07T16:21:40.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:39 smithi100 bash[76202]: cluster 2024-04-07T16:21:38.780901+0000 mon.a (mon.0) 268 : cluster [DBG] osdmap e350: 8 total, 8 up, 8 in 2024-04-07T16:21:40.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:39 smithi100 bash[76202]: audit 2024-04-07T16:21:39.375252+0000 mon.a (mon.0) 269 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:21:40.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:39 smithi100 bash[76202]: audit 2024-04-07T16:21:39.381231+0000 mon.a (mon.0) 270 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:21:40.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:39 smithi100 bash[76202]: audit 2024-04-07T16:21:39.390282+0000 mon.a (mon.0) 271 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-07T16:21:40.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:39 smithi100 bash[76202]: audit 2024-04-07T16:21:39.391091+0000 mon.a (mon.0) 272 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:21:40.781 INFO:tasks.rados.rados.0.smithi100.stdout:1476: done (2 left) 2024-04-07T16:21:40.782 INFO:tasks.rados.rados.0.smithi100.stdout:1479: done (1 left) 2024-04-07T16:21:40.782 INFO:tasks.rados.rados.0.smithi100.stdout:1480: done (0 left) 2024-04-07T16:21:40.782 INFO:tasks.rados.rados.0.smithi100.stdout:1481: copy_from oid 45 from oid 50 current snap is 140 2024-04-07T16:21:40.782 INFO:tasks.rados.rados.0.smithi100.stdout:1482: copy_from oid 17 from oid 29 current snap is 140 2024-04-07T16:21:40.782 INFO:tasks.rados.rados.0.smithi100.stdout:1483: read oid 30 snap -1 2024-04-07T16:21:40.782 INFO:tasks.rados.rados.0.smithi100.stdout:1483: expect (ObjNum 433 snap 130 seq_num 433) 2024-04-07T16:21:40.782 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 1 current snap is 140 2024-04-07T16:21:40.782 INFO:tasks.rados.rados.0.smithi100.stdout:1484: seq_num 459 ranges {0=491520} 2024-04-07T16:21:40.785 INFO:tasks.rados.rados.0.smithi100.stdout:1484: writing smithi10066867-1 from 0 to 491520 tid 1 2024-04-07T16:21:40.785 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 43 current snap is 140 2024-04-07T16:21:40.785 INFO:tasks.rados.rados.0.smithi100.stdout:1485: seq_num 460 ranges {0=507904} 2024-04-07T16:21:40.789 INFO:tasks.rados.rados.0.smithi100.stdout:1485: writing smithi10066867-43 from 0 to 507904 tid 1 2024-04-07T16:21:40.789 INFO:tasks.rados.rados.0.smithi100.stdout:1486: read oid 48 snap 132 2024-04-07T16:21:40.789 INFO:tasks.rados.rados.0.smithi100.stdout:1486: expect (ObjNum 434 snap 130 seq_num 434) 2024-04-07T16:21:40.789 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 2 current snap is 140 2024-04-07T16:21:40.789 INFO:tasks.rados.rados.0.smithi100.stdout:1487: seq_num 461 ranges {1138688=524288} 2024-04-07T16:21:40.799 INFO:tasks.rados.rados.0.smithi100.stdout:1487: writing smithi10066867-2 from 1138688 to 1662976 tid 1 2024-04-07T16:21:40.799 INFO:tasks.rados.rados.0.smithi100.stdout:1488: setattr oid 18 current snap is 140 2024-04-07T16:21:40.799 INFO:tasks.rados.rados.0.smithi100.stdout:1484: finishing write tid 1 to smithi10066867-1 2024-04-07T16:21:40.799 INFO:tasks.rados.rados.0.smithi100.stdout:1484: finishing write tid 2 to smithi10066867-1 2024-04-07T16:21:40.800 INFO:tasks.rados.rados.0.smithi100.stdout:1485: finishing write tid 1 to smithi10066867-43 2024-04-07T16:21:40.800 INFO:tasks.rados.rados.0.smithi100.stdout:1485: finishing write tid 2 to smithi10066867-43 2024-04-07T16:21:40.801 INFO:tasks.rados.rados.0.smithi100.stdout:1489: read oid 39 snap -1 2024-04-07T16:21:40.801 INFO:tasks.rados.rados.0.smithi100.stdout:1489: expect (ObjNum 413 snap 125 seq_num 413) 2024-04-07T16:21:40.801 INFO:tasks.rados.rados.0.smithi100.stdout:1490: read oid 40 snap -1 2024-04-07T16:21:40.801 INFO:tasks.rados.rados.0.smithi100.stdout:1490: expect deleted 2024-04-07T16:21:40.801 INFO:tasks.rados.rados.0.smithi100.stdout:1491: snap_remove snap 134 2024-04-07T16:21:40.884 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:40 smithi100 bash[74942]: cephadm 2024-04-07T16:21:39.389914+0000 mgr.y (mgr.25175) 77 : cephadm [INF] Reconfiguring osd.2 (monmap changed)... 2024-04-07T16:21:40.884 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:40 smithi100 bash[74942]: cephadm 2024-04-07T16:21:39.393351+0000 mgr.y (mgr.25175) 78 : cephadm [INF] Reconfiguring daemon osd.2 on smithi100 2024-04-07T16:21:40.884 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:40 smithi100 bash[74942]: cluster 2024-04-07T16:21:39.792845+0000 mon.a (mon.0) 273 : cluster [DBG] osdmap e351: 8 total, 8 up, 8 in 2024-04-07T16:21:40.884 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:40 smithi100 bash[76202]: cephadm 2024-04-07T16:21:39.389914+0000 mgr.y (mgr.25175) 77 : cephadm [INF] Reconfiguring osd.2 (monmap changed)... 2024-04-07T16:21:40.884 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:40 smithi100 bash[76202]: cephadm 2024-04-07T16:21:39.393351+0000 mgr.y (mgr.25175) 78 : cephadm [INF] Reconfiguring daemon osd.2 on smithi100 2024-04-07T16:21:40.885 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:40 smithi100 bash[76202]: cluster 2024-04-07T16:21:39.792845+0000 mon.a (mon.0) 273 : cluster [DBG] osdmap e351: 8 total, 8 up, 8 in 2024-04-07T16:21:41.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:40 smithi144 bash[66654]: cephadm 2024-04-07T16:21:39.389914+0000 mgr.y (mgr.25175) 77 : cephadm [INF] Reconfiguring osd.2 (monmap changed)... 2024-04-07T16:21:41.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:40 smithi144 bash[66654]: cephadm 2024-04-07T16:21:39.393351+0000 mgr.y (mgr.25175) 78 : cephadm [INF] Reconfiguring daemon osd.2 on smithi100 2024-04-07T16:21:41.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:40 smithi144 bash[66654]: cluster 2024-04-07T16:21:39.792845+0000 mon.a (mon.0) 273 : cluster [DBG] osdmap e351: 8 total, 8 up, 8 in 2024-04-07T16:21:41.761 INFO:tasks.rados.rados.0.smithi100.stdout:1491: done (10 left) 2024-04-07T16:21:41.762 INFO:tasks.rados.rados.0.smithi100.stdout:1492: rollback oid 47 current snap is 140 2024-04-07T16:21:41.762 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 47 to 138 2024-04-07T16:21:41.762 INFO:tasks.rados.rados.0.smithi100.stdout:1485: finishing write tid 3 to smithi10066867-43 2024-04-07T16:21:41.762 INFO:tasks.rados.rados.0.smithi100.stdout:1485: oid 43 updating version 0 to 535 2024-04-07T16:21:41.762 INFO:tasks.rados.rados.0.smithi100.stdout:1485: oid 43 version 535 is already newer than 534 2024-04-07T16:21:41.762 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 43 v 535 (ObjNum 460 snap 140 seq_num 460) dirty exists 2024-04-07T16:21:41.762 INFO:tasks.rados.rados.0.smithi100.stdout:1485: left oid 43 (ObjNum 460 snap 140 seq_num 460) 2024-04-07T16:21:41.762 INFO:tasks.rados.rados.0.smithi100.stdout:1485: done (10 left) 2024-04-07T16:21:41.762 INFO:tasks.rados.rados.0.smithi100.stdout:1493: copy_from oid 11 from oid 37 current snap is 140 2024-04-07T16:21:41.762 INFO:tasks.rados.rados.0.smithi100.stdout:1494: read oid 28 snap 139 2024-04-07T16:21:41.762 INFO:tasks.rados.rados.0.smithi100.stdout:1494: expect (ObjNum 439 snap 134 seq_num 439) 2024-04-07T16:21:41.762 INFO:tasks.rados.rados.0.smithi100.stdout:1495: snap_create 2024-04-07T16:21:41.762 INFO:tasks.rados.rados.0.smithi100.stdout:1484: finishing write tid 3 to smithi10066867-1 2024-04-07T16:21:41.763 INFO:tasks.rados.rados.0.smithi100.stdout:1484: oid 1 updating version 0 to 831 2024-04-07T16:21:41.763 INFO:tasks.rados.rados.0.smithi100.stdout:1484: oid 1 updating version 831 to 832 2024-04-07T16:21:41.763 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 1 v 832 (ObjNum 459 snap 140 seq_num 459) dirty exists 2024-04-07T16:21:41.763 INFO:tasks.rados.rados.0.smithi100.stdout:1484: left oid 1 (ObjNum 459 snap 140 seq_num 459) 2024-04-07T16:21:41.763 INFO:tasks.rados.rados.0.smithi100.stdout:1481: finishing copy_from racing read to smithi10066867-45 2024-04-07T16:21:41.763 INFO:tasks.rados.rados.0.smithi100.stdout:1486: expect (ObjNum 434 snap 130 seq_num 434) 2024-04-07T16:21:41.890 INFO:tasks.rados.rados.0.smithi100.stdout:1482: finishing copy_from racing read to smithi10066867-17 2024-04-07T16:21:41.890 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 18 v 955 (ObjNum 445 snap 135 seq_num 445) dirty exists 2024-04-07T16:21:41.890 INFO:tasks.rados.rados.0.smithi100.stdout:1481: finishing copy_from to smithi10066867-45 2024-04-07T16:21:41.891 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 45 v 525 (ObjNum 416 snap 126 seq_num 416) dirty exists 2024-04-07T16:21:41.891 INFO:tasks.rados.rados.0.smithi100.stdout:1487: finishing write tid 1 to smithi10066867-2 2024-04-07T16:21:41.891 INFO:tasks.rados.rados.0.smithi100.stdout:1487: finishing write tid 2 to smithi10066867-2 2024-04-07T16:21:41.891 INFO:tasks.rados.rados.0.smithi100.stdout:1482: finishing copy_from to smithi10066867-17 2024-04-07T16:21:41.891 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 17 v 547 (ObjNum 412 snap 125 seq_num 412) dirty exists 2024-04-07T16:21:41.891 INFO:tasks.rados.rados.0.smithi100.stdout:1487: finishing write tid 3 to smithi10066867-2 2024-04-07T16:21:41.891 INFO:tasks.rados.rados.0.smithi100.stdout:1487: oid 2 updating version 0 to 740 2024-04-07T16:21:41.891 INFO:tasks.rados.rados.0.smithi100.stdout:1487: oid 2 updating version 740 to 741 2024-04-07T16:21:41.891 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 2 v 741 (ObjNum 461 snap 140 seq_num 461) dirty exists 2024-04-07T16:21:41.891 INFO:tasks.rados.rados.0.smithi100.stdout:1487: left oid 2 (ObjNum 461 snap 140 seq_num 461) 2024-04-07T16:21:41.891 INFO:tasks.rados.rados.0.smithi100.stdout:1483: expect (ObjNum 433 snap 130 seq_num 433) 2024-04-07T16:21:42.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:41 smithi144 bash[66654]: cluster 2024-04-07T16:21:40.630184+0000 mgr.y (mgr.25175) 79 : cluster [DBG] pgmap v116: 121 pgs: 121 active+clean; 165 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 4.8 MiB/s rd, 292 KiB/s wr, 7 op/s 2024-04-07T16:21:42.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:41 smithi144 bash[66654]: cluster 2024-04-07T16:21:40.798706+0000 mon.a (mon.0) 274 : cluster [DBG] osdmap e352: 8 total, 8 up, 8 in 2024-04-07T16:21:42.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:41 smithi144 bash[66654]: audit 2024-04-07T16:21:41.476511+0000 mon.a (mon.0) 275 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:21:42.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:41 smithi144 bash[66654]: audit 2024-04-07T16:21:41.486786+0000 mon.a (mon.0) 276 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:21:42.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:41 smithi144 bash[66654]: audit 2024-04-07T16:21:41.514303+0000 mon.a (mon.0) 277 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-07T16:21:42.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:41 smithi144 bash[66654]: audit 2024-04-07T16:21:41.515897+0000 mon.a (mon.0) 278 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:21:42.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:41 smithi144 bash[66654]: cluster 2024-04-07T16:21:41.777913+0000 mon.a (mon.0) 279 : cluster [DBG] osdmap e353: 8 total, 8 up, 8 in 2024-04-07T16:21:42.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:41 smithi100 bash[76202]: cluster 2024-04-07T16:21:40.630184+0000 mgr.y (mgr.25175) 79 : cluster [DBG] pgmap v116: 121 pgs: 121 active+clean; 165 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 4.8 MiB/s rd, 292 KiB/s wr, 7 op/s 2024-04-07T16:21:42.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:41 smithi100 bash[76202]: cluster 2024-04-07T16:21:40.798706+0000 mon.a (mon.0) 274 : cluster [DBG] osdmap e352: 8 total, 8 up, 8 in 2024-04-07T16:21:42.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:41 smithi100 bash[76202]: audit 2024-04-07T16:21:41.476511+0000 mon.a (mon.0) 275 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:21:42.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:41 smithi100 bash[76202]: audit 2024-04-07T16:21:41.486786+0000 mon.a (mon.0) 276 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:21:42.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:41 smithi100 bash[76202]: audit 2024-04-07T16:21:41.514303+0000 mon.a (mon.0) 277 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-07T16:21:42.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:41 smithi100 bash[76202]: audit 2024-04-07T16:21:41.515897+0000 mon.a (mon.0) 278 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:21:42.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:41 smithi100 bash[76202]: cluster 2024-04-07T16:21:41.777913+0000 mon.a (mon.0) 279 : cluster [DBG] osdmap e353: 8 total, 8 up, 8 in 2024-04-07T16:21:42.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:41 smithi100 bash[74942]: cluster 2024-04-07T16:21:40.630184+0000 mgr.y (mgr.25175) 79 : cluster [DBG] pgmap v116: 121 pgs: 121 active+clean; 165 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 4.8 MiB/s rd, 292 KiB/s wr, 7 op/s 2024-04-07T16:21:42.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:41 smithi100 bash[74942]: cluster 2024-04-07T16:21:40.798706+0000 mon.a (mon.0) 274 : cluster [DBG] osdmap e352: 8 total, 8 up, 8 in 2024-04-07T16:21:42.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:41 smithi100 bash[74942]: audit 2024-04-07T16:21:41.476511+0000 mon.a (mon.0) 275 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:21:42.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:41 smithi100 bash[74942]: audit 2024-04-07T16:21:41.486786+0000 mon.a (mon.0) 276 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:21:42.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:41 smithi100 bash[74942]: audit 2024-04-07T16:21:41.514303+0000 mon.a (mon.0) 277 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-07T16:21:42.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:41 smithi100 bash[74942]: audit 2024-04-07T16:21:41.515897+0000 mon.a (mon.0) 278 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:21:42.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:41 smithi100 bash[74942]: cluster 2024-04-07T16:21:41.777913+0000 mon.a (mon.0) 279 : cluster [DBG] osdmap e353: 8 total, 8 up, 8 in 2024-04-07T16:21:42.510 INFO:tasks.rados.rados.0.smithi100.stdout:1489: expect (ObjNum 413 snap 125 seq_num 413) 2024-04-07T16:21:42.602 INFO:tasks.rados.rados.0.smithi100.stdout:1492: finishing rollback tid 0 to smithi10066867-47 2024-04-07T16:21:42.658 INFO:tasks.rados.rados.0.smithi100.stdout:1494: expect (ObjNum 439 snap 134 seq_num 439) 2024-04-07T16:21:42.658 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:21:42 smithi144 bash[59794]: ts=2024-04-07T16:21:42.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:21:42.658 INFO:tasks.rados.rados.0.smithi100.stdout:1493: finishing copy_from racing read to smithi10066867-11 2024-04-07T16:21:42.659 INFO:tasks.rados.rados.0.smithi100.stdout:1492: finishing rollback tid 1 to smithi10066867-47 2024-04-07T16:21:42.659 INFO:tasks.rados.rados.0.smithi100.stdout:1492: finishing rollback tid 2 to smithi10066867-47 2024-04-07T16:21:42.659 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 47 v 532 (ObjNum 177 snap 0 seq_num 0) dirty exists 2024-04-07T16:21:42.659 INFO:tasks.rados.rados.0.smithi100.stdout:1493: finishing copy_from to smithi10066867-11 2024-04-07T16:21:42.659 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 11 v 479 (ObjNum 129 snap 0 seq_num 2702296544) dirty exists 2024-04-07T16:21:42.760 INFO:tasks.rados.rados.0.smithi100.stdout:1481: done (12 left) 2024-04-07T16:21:42.761 INFO:tasks.rados.rados.0.smithi100.stdout:1482: done (11 left) 2024-04-07T16:21:42.761 INFO:tasks.rados.rados.0.smithi100.stdout:1483: done (10 left) 2024-04-07T16:21:42.761 INFO:tasks.rados.rados.0.smithi100.stdout:1484: done (9 left) 2024-04-07T16:21:42.761 INFO:tasks.rados.rados.0.smithi100.stdout:1486: done (8 left) 2024-04-07T16:21:42.761 INFO:tasks.rados.rados.0.smithi100.stdout:1487: done (7 left) 2024-04-07T16:21:42.761 INFO:tasks.rados.rados.0.smithi100.stdout:1488: done (6 left) 2024-04-07T16:21:42.761 INFO:tasks.rados.rados.0.smithi100.stdout:1489: done (5 left) 2024-04-07T16:21:42.761 INFO:tasks.rados.rados.0.smithi100.stdout:1490: done (4 left) 2024-04-07T16:21:42.761 INFO:tasks.rados.rados.0.smithi100.stdout:1492: done (3 left) 2024-04-07T16:21:42.761 INFO:tasks.rados.rados.0.smithi100.stdout:1493: done (2 left) 2024-04-07T16:21:42.761 INFO:tasks.rados.rados.0.smithi100.stdout:1494: done (1 left) 2024-04-07T16:21:42.761 INFO:tasks.rados.rados.0.smithi100.stdout:1495: done (0 left) 2024-04-07T16:21:42.761 INFO:tasks.rados.rados.0.smithi100.stdout:1496: rmattr oid 11 current snap is 141 2024-04-07T16:21:42.761 INFO:tasks.rados.rados.0.smithi100.stdout:1497: snap_remove snap 139 2024-04-07T16:21:43.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:42 smithi144 bash[66654]: cephadm 2024-04-07T16:21:41.513701+0000 mgr.y (mgr.25175) 80 : cephadm [INF] Reconfiguring osd.3 (monmap changed)... 2024-04-07T16:21:43.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:42 smithi144 bash[66654]: cephadm 2024-04-07T16:21:41.519112+0000 mgr.y (mgr.25175) 81 : cephadm [INF] Reconfiguring daemon osd.3 on smithi100 2024-04-07T16:21:43.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:42 smithi144 bash[66654]: cluster 2024-04-07T16:21:42.777937+0000 mon.a (mon.0) 280 : cluster [DBG] osdmap e354: 8 total, 8 up, 8 in 2024-04-07T16:21:43.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:42 smithi100 bash[74942]: cephadm 2024-04-07T16:21:41.513701+0000 mgr.y (mgr.25175) 80 : cephadm [INF] Reconfiguring osd.3 (monmap changed)... 2024-04-07T16:21:43.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:42 smithi100 bash[74942]: cephadm 2024-04-07T16:21:41.519112+0000 mgr.y (mgr.25175) 81 : cephadm [INF] Reconfiguring daemon osd.3 on smithi100 2024-04-07T16:21:43.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:42 smithi100 bash[74942]: cluster 2024-04-07T16:21:42.777937+0000 mon.a (mon.0) 280 : cluster [DBG] osdmap e354: 8 total, 8 up, 8 in 2024-04-07T16:21:43.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:42 smithi100 bash[76202]: cephadm 2024-04-07T16:21:41.513701+0000 mgr.y (mgr.25175) 80 : cephadm [INF] Reconfiguring osd.3 (monmap changed)... 2024-04-07T16:21:43.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:42 smithi100 bash[76202]: cephadm 2024-04-07T16:21:41.519112+0000 mgr.y (mgr.25175) 81 : cephadm [INF] Reconfiguring daemon osd.3 on smithi100 2024-04-07T16:21:43.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:42 smithi100 bash[76202]: cluster 2024-04-07T16:21:42.777937+0000 mon.a (mon.0) 280 : cluster [DBG] osdmap e354: 8 total, 8 up, 8 in 2024-04-07T16:21:43.776 INFO:tasks.rados.rados.0.smithi100.stdout:1497: done (1 left) 2024-04-07T16:21:43.776 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 14 current snap is 141 2024-04-07T16:21:43.776 INFO:tasks.rados.rados.0.smithi100.stdout:1498: seq_num 462 ranges {933888=540672} 2024-04-07T16:21:43.783 INFO:tasks.rados.rados.0.smithi100.stdout:1498: writing smithi10066867-14 from 933888 to 1474560 tid 1 2024-04-07T16:21:43.783 INFO:tasks.rados.rados.0.smithi100.stdout:1499: read oid 42 snap -1 2024-04-07T16:21:43.783 INFO:tasks.rados.rados.0.smithi100.stdout:1499: expect deleted 2024-04-07T16:21:43.783 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 48 current snap is 141 2024-04-07T16:21:43.783 INFO:tasks.rados.rados.0.smithi100.stdout:1500: seq_num 463 ranges {1892352=557056} 2024-04-07T16:21:43.795 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:21:43 smithi144 bash[59794]: ts=2024-04-07T16:21:43.508Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=nodes name=CephNodeDiskspaceWarning index=4 msg="Evaluating rule failed" rule="alert: CephNodeDiskspaceWarning\nexpr: predict_linear(node_filesystem_free_bytes{device=~\"/.*\"}[2d], 3600 * 24 * 5)\n * on (instance) group_left (nodename) node_uname_info < 0\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.8.4\n severity: warning\n type: ceph_default\nannotations:\n description: Mountpoint {{ $labels.mountpoint }} on {{ $labels.nodename }} will\n be full in less than 5 days based on the 48 hour trailing fill rate.\n summary: Host filesystem free space is getting low\n" err="found duplicate series for the match group {instance=\"smithi100\"} on the right hand-side of the operation: [{__name__=\"node_uname_info\", domainname=\"(none)\", instance=\"smithi100\", job=\"node\", machine=\"x86_64\", nodename=\"smithi100\", release=\"5.15.0-101-generic\", sysname=\"Linux\", version=\"#111-Ubuntu SMP Tue Mar 5 20:16:58 UTC 2024\"}, {__name__=\"node_uname_info\", cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", domainname=\"(none)\", instance=\"smithi100\", job=\"node\", machine=\"x86_64\", nodename=\"smithi100\", release=\"5.15.0-101-generic\", sysname=\"Linux\", version=\"#111-Ubuntu SMP Tue Mar 5 20:16:58 UTC 2024\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:21:43.798 INFO:tasks.rados.rados.0.smithi100.stdout:1500: writing smithi10066867-48 from 1892352 to 2449408 tid 1 2024-04-07T16:21:43.798 INFO:tasks.rados.rados.0.smithi100.stdout:1501: read oid 29 snap -1 2024-04-07T16:21:43.798 INFO:tasks.rados.rados.0.smithi100.stdout:1501: expect (ObjNum 412 snap 125 seq_num 412) 2024-04-07T16:21:43.798 INFO:tasks.rados.rados.0.smithi100.stdout:1502: snap_create 2024-04-07T16:21:43.798 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 11 v 963 (ObjNum 65 snap 0 seq_num 1600412521) dirty exists 2024-04-07T16:21:43.798 INFO:tasks.rados.rados.0.smithi100.stdout:1498: finishing write tid 1 to smithi10066867-14 2024-04-07T16:21:43.798 INFO:tasks.rados.rados.0.smithi100.stdout:1498: finishing write tid 2 to smithi10066867-14 2024-04-07T16:21:43.798 INFO:tasks.rados.rados.0.smithi100.stdout:1498: finishing write tid 3 to smithi10066867-14 2024-04-07T16:21:43.799 INFO:tasks.rados.rados.0.smithi100.stdout:1498: oid 14 updating version 0 to 716 2024-04-07T16:21:43.799 INFO:tasks.rados.rados.0.smithi100.stdout:1498: oid 14 updating version 716 to 717 2024-04-07T16:21:43.799 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 14 v 717 (ObjNum 462 snap 141 seq_num 462) dirty exists 2024-04-07T16:21:43.799 INFO:tasks.rados.rados.0.smithi100.stdout:1498: left oid 14 (ObjNum 462 snap 141 seq_num 462) 2024-04-07T16:21:43.804 INFO:tasks.rados.rados.0.smithi100.stdout:1500: finishing write tid 1 to smithi10066867-48 2024-04-07T16:21:43.804 INFO:tasks.rados.rados.0.smithi100.stdout:1500: finishing write tid 2 to smithi10066867-48 2024-04-07T16:21:43.805 INFO:tasks.rados.rados.0.smithi100.stdout:1500: finishing write tid 3 to smithi10066867-48 2024-04-07T16:21:43.805 INFO:tasks.rados.rados.0.smithi100.stdout:1500: oid 48 updating version 0 to 585 2024-04-07T16:21:43.805 INFO:tasks.rados.rados.0.smithi100.stdout:1500: oid 48 updating version 585 to 586 2024-04-07T16:21:43.805 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 48 v 586 (ObjNum 463 snap 141 seq_num 463) dirty exists 2024-04-07T16:21:43.806 INFO:tasks.rados.rados.0.smithi100.stdout:1500: left oid 48 (ObjNum 463 snap 141 seq_num 463) 2024-04-07T16:21:43.813 INFO:tasks.rados.rados.0.smithi100.stdout:1501: expect (ObjNum 412 snap 125 seq_num 412) 2024-04-07T16:21:44.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:43 smithi144 bash[66654]: cluster 2024-04-07T16:21:42.631491+0000 mgr.y (mgr.25175) 82 : cluster [DBG] pgmap v119: 121 pgs: 121 active+clean; 172 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 21 MiB/s rd, 1.9 MiB/s wr, 18 op/s 2024-04-07T16:21:44.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:43 smithi144 bash[66654]: audit 2024-04-07T16:21:43.478460+0000 mon.a (mon.0) 281 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:21:44.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:43 smithi144 bash[66654]: audit 2024-04-07T16:21:43.484520+0000 mon.a (mon.0) 282 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:21:44.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:43 smithi144 bash[66654]: audit 2024-04-07T16:21:43.486965+0000 mon.a (mon.0) 283 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-07T16:21:44.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:43 smithi144 bash[66654]: audit 2024-04-07T16:21:43.487978+0000 mon.a (mon.0) 284 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:21:44.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:43 smithi144 bash[66654]: cluster 2024-04-07T16:21:43.784400+0000 mon.a (mon.0) 285 : cluster [DBG] osdmap e355: 8 total, 8 up, 8 in 2024-04-07T16:21:44.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:43 smithi100 bash[74942]: cluster 2024-04-07T16:21:42.631491+0000 mgr.y (mgr.25175) 82 : cluster [DBG] pgmap v119: 121 pgs: 121 active+clean; 172 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 21 MiB/s rd, 1.9 MiB/s wr, 18 op/s 2024-04-07T16:21:44.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:43 smithi100 bash[74942]: audit 2024-04-07T16:21:43.478460+0000 mon.a (mon.0) 281 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:21:44.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:43 smithi100 bash[74942]: audit 2024-04-07T16:21:43.484520+0000 mon.a (mon.0) 282 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:21:44.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:43 smithi100 bash[74942]: audit 2024-04-07T16:21:43.486965+0000 mon.a (mon.0) 283 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-07T16:21:44.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:43 smithi100 bash[74942]: audit 2024-04-07T16:21:43.487978+0000 mon.a (mon.0) 284 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:21:44.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:43 smithi100 bash[74942]: cluster 2024-04-07T16:21:43.784400+0000 mon.a (mon.0) 285 : cluster [DBG] osdmap e355: 8 total, 8 up, 8 in 2024-04-07T16:21:44.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:43 smithi100 bash[76202]: cluster 2024-04-07T16:21:42.631491+0000 mgr.y (mgr.25175) 82 : cluster [DBG] pgmap v119: 121 pgs: 121 active+clean; 172 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 21 MiB/s rd, 1.9 MiB/s wr, 18 op/s 2024-04-07T16:21:44.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:43 smithi100 bash[76202]: audit 2024-04-07T16:21:43.478460+0000 mon.a (mon.0) 281 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:21:44.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:43 smithi100 bash[76202]: audit 2024-04-07T16:21:43.484520+0000 mon.a (mon.0) 282 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:21:44.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:43 smithi100 bash[76202]: audit 2024-04-07T16:21:43.486965+0000 mon.a (mon.0) 283 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-07T16:21:44.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:43 smithi100 bash[76202]: audit 2024-04-07T16:21:43.487978+0000 mon.a (mon.0) 284 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:21:44.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:43 smithi100 bash[76202]: cluster 2024-04-07T16:21:43.784400+0000 mon.a (mon.0) 285 : cluster [DBG] osdmap e355: 8 total, 8 up, 8 in 2024-04-07T16:21:44.808 INFO:tasks.rados.rados.0.smithi100.stdout:1496: done (5 left) 2024-04-07T16:21:44.808 INFO:tasks.rados.rados.0.smithi100.stdout:1498: done (4 left) 2024-04-07T16:21:44.808 INFO:tasks.rados.rados.0.smithi100.stdout:1499: done (3 left) 2024-04-07T16:21:44.808 INFO:tasks.rados.rados.0.smithi100.stdout:1500: done (2 left) 2024-04-07T16:21:44.808 INFO:tasks.rados.rados.0.smithi100.stdout:1501: done (1 left) 2024-04-07T16:21:44.808 INFO:tasks.rados.rados.0.smithi100.stdout:1502: done (0 left) 2024-04-07T16:21:44.808 INFO:tasks.rados.rados.0.smithi100.stdout:1503: snap_remove snap 132 2024-04-07T16:21:44.935 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:44 smithi100 bash[76202]: cephadm 2024-04-07T16:21:43.486563+0000 mgr.y (mgr.25175) 83 : cephadm [INF] Reconfiguring mds.a (monmap changed)... 2024-04-07T16:21:44.935 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:44 smithi100 bash[76202]: cephadm 2024-04-07T16:21:43.488945+0000 mgr.y (mgr.25175) 84 : cephadm [INF] Reconfiguring daemon mds.a on smithi100 2024-04-07T16:21:44.935 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:44 smithi100 bash[74942]: cephadm 2024-04-07T16:21:43.486563+0000 mgr.y (mgr.25175) 83 : cephadm [INF] Reconfiguring mds.a (monmap changed)... 2024-04-07T16:21:44.936 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:44 smithi100 bash[74942]: cephadm 2024-04-07T16:21:43.488945+0000 mgr.y (mgr.25175) 84 : cephadm [INF] Reconfiguring daemon mds.a on smithi100 2024-04-07T16:21:45.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:44 smithi144 bash[66654]: cephadm 2024-04-07T16:21:43.486563+0000 mgr.y (mgr.25175) 83 : cephadm [INF] Reconfiguring mds.a (monmap changed)... 2024-04-07T16:21:45.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:44 smithi144 bash[66654]: cephadm 2024-04-07T16:21:43.488945+0000 mgr.y (mgr.25175) 84 : cephadm [INF] Reconfiguring daemon mds.a on smithi100 2024-04-07T16:21:45.814 INFO:tasks.rados.rados.0.smithi100.stdout:1503: done (0 left) 2024-04-07T16:21:45.814 INFO:tasks.rados.rados.0.smithi100.stdout:1504: rollback oid 10 current snap is 142 2024-04-07T16:21:45.814 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 10 to 140 2024-04-07T16:21:45.814 INFO:tasks.rados.rados.0.smithi100.stdout:1505: snap_remove snap 138 2024-04-07T16:21:46.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:45 smithi144 bash[66654]: cluster 2024-04-07T16:21:44.632149+0000 mgr.y (mgr.25175) 85 : cluster [DBG] pgmap v122: 121 pgs: 121 active+clean; 172 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 21 MiB/s rd, 1.9 MiB/s wr, 18 op/s 2024-04-07T16:21:46.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:45 smithi144 bash[66654]: cluster 2024-04-07T16:21:44.825337+0000 mon.a (mon.0) 286 : cluster [DBG] osdmap e356: 8 total, 8 up, 8 in 2024-04-07T16:21:46.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:45 smithi144 bash[66654]: audit 2024-04-07T16:21:45.513800+0000 mon.a (mon.0) 287 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:21:46.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:45 smithi144 bash[66654]: audit 2024-04-07T16:21:45.523352+0000 mon.a (mon.0) 288 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:21:46.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:45 smithi144 bash[66654]: audit 2024-04-07T16:21:45.559207+0000 mon.a (mon.0) 289 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-07T16:21:46.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:45 smithi144 bash[66654]: audit 2024-04-07T16:21:45.560961+0000 mon.a (mon.0) 290 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-07T16:21:46.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:45 smithi144 bash[66654]: audit 2024-04-07T16:21:45.562749+0000 mon.a (mon.0) 291 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:21:46.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:45 smithi100 bash[74942]: cluster 2024-04-07T16:21:44.632149+0000 mgr.y (mgr.25175) 85 : cluster [DBG] pgmap v122: 121 pgs: 121 active+clean; 172 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 21 MiB/s rd, 1.9 MiB/s wr, 18 op/s 2024-04-07T16:21:46.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:45 smithi100 bash[74942]: cluster 2024-04-07T16:21:44.825337+0000 mon.a (mon.0) 286 : cluster [DBG] osdmap e356: 8 total, 8 up, 8 in 2024-04-07T16:21:46.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:45 smithi100 bash[74942]: audit 2024-04-07T16:21:45.513800+0000 mon.a (mon.0) 287 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:21:46.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:45 smithi100 bash[74942]: audit 2024-04-07T16:21:45.523352+0000 mon.a (mon.0) 288 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:21:46.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:45 smithi100 bash[74942]: audit 2024-04-07T16:21:45.559207+0000 mon.a (mon.0) 289 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-07T16:21:46.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:45 smithi100 bash[74942]: audit 2024-04-07T16:21:45.560961+0000 mon.a (mon.0) 290 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-07T16:21:46.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:45 smithi100 bash[74942]: audit 2024-04-07T16:21:45.562749+0000 mon.a (mon.0) 291 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:21:46.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:45 smithi100 bash[76202]: cluster 2024-04-07T16:21:44.632149+0000 mgr.y (mgr.25175) 85 : cluster [DBG] pgmap v122: 121 pgs: 121 active+clean; 172 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 21 MiB/s rd, 1.9 MiB/s wr, 18 op/s 2024-04-07T16:21:46.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:45 smithi100 bash[76202]: cluster 2024-04-07T16:21:44.825337+0000 mon.a (mon.0) 286 : cluster [DBG] osdmap e356: 8 total, 8 up, 8 in 2024-04-07T16:21:46.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:45 smithi100 bash[76202]: audit 2024-04-07T16:21:45.513800+0000 mon.a (mon.0) 287 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:21:46.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:45 smithi100 bash[76202]: audit 2024-04-07T16:21:45.523352+0000 mon.a (mon.0) 288 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:21:46.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:45 smithi100 bash[76202]: audit 2024-04-07T16:21:45.559207+0000 mon.a (mon.0) 289 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-07T16:21:46.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:45 smithi100 bash[76202]: audit 2024-04-07T16:21:45.560961+0000 mon.a (mon.0) 290 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-07T16:21:46.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:45 smithi100 bash[76202]: audit 2024-04-07T16:21:45.562749+0000 mon.a (mon.0) 291 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:21:46.770 INFO:tasks.rados.rados.0.smithi100.stdout:1505: done (1 left) 2024-04-07T16:21:46.770 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 5 current snap is 142 2024-04-07T16:21:46.771 INFO:tasks.rados.rados.0.smithi100.stdout:1504: finishing rollback tid 0 to smithi10066867-10 2024-04-07T16:21:46.771 INFO:tasks.rados.rados.0.smithi100.stdout:1504: finishing rollback tid 1 to smithi10066867-10 2024-04-07T16:21:46.771 INFO:tasks.rados.rados.0.smithi100.stdout:1504: finishing rollback tid 2 to smithi10066867-10 2024-04-07T16:21:46.771 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 10 v 835 (ObjNum 430 snap 130 seq_num 430) dirty exists 2024-04-07T16:21:46.771 INFO:tasks.rados.rados.0.smithi100.stdout:1506: seq_num 464 ranges {5226496=573440} 2024-04-07T16:21:46.844 INFO:tasks.rados.rados.0.smithi100.stdout:1506: writing smithi10066867-5 from 5226496 to 5799936 tid 1 2024-04-07T16:21:46.844 INFO:tasks.rados.rados.0.smithi100.stdout:1504: done (1 left) 2024-04-07T16:21:46.844 INFO:tasks.rados.rados.0.smithi100.stdout:1507: copy_from oid 10 from oid 49 current snap is 142 2024-04-07T16:21:46.844 INFO:tasks.rados.rados.0.smithi100.stdout:1508: rollback oid 2 current snap is 142 2024-04-07T16:21:46.844 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 2 to 140 2024-04-07T16:21:46.845 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 39 current snap is 142 2024-04-07T16:21:46.845 INFO:tasks.rados.rados.0.smithi100.stdout:1509: seq_num 465 ranges {3022848=598016} 2024-04-07T16:21:46.871 INFO:tasks.rados.rados.0.smithi100.stdout:1509: writing smithi10066867-39 from 3022848 to 3620864 tid 1 2024-04-07T16:21:46.871 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 47 current snap is 142 2024-04-07T16:21:46.871 INFO:tasks.rados.rados.0.smithi100.stdout:1510: seq_num 466 ranges {0=614400} 2024-04-07T16:21:46.875 INFO:tasks.rados.rados.0.smithi100.stdout:1510: writing smithi10066867-47 from 0 to 614400 tid 1 2024-04-07T16:21:46.875 INFO:tasks.rados.rados.0.smithi100.stdout:1511: setattr oid 8 current snap is 142 2024-04-07T16:21:46.875 INFO:tasks.rados.rados.0.smithi100.stdout:1506: finishing write tid 1 to smithi10066867-5 2024-04-07T16:21:46.876 INFO:tasks.rados.rados.0.smithi100.stdout:1506: finishing write tid 2 to smithi10066867-5 2024-04-07T16:21:46.876 INFO:tasks.rados.rados.0.smithi100.stdout:1508: finishing rollback tid 0 to smithi10066867-2 2024-04-07T16:21:46.876 INFO:tasks.rados.rados.0.smithi100.stdout:1506: finishing write tid 3 to smithi10066867-5 2024-04-07T16:21:46.876 INFO:tasks.rados.rados.0.smithi100.stdout:1506: oid 5 updating version 0 to 811 2024-04-07T16:21:46.876 INFO:tasks.rados.rados.0.smithi100.stdout:1506: oid 5 updating version 811 to 812 2024-04-07T16:21:46.876 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 5 v 812 (ObjNum 464 snap 142 seq_num 464) dirty exists 2024-04-07T16:21:46.876 INFO:tasks.rados.rados.0.smithi100.stdout:1506: left oid 5 (ObjNum 464 snap 142 seq_num 464) 2024-04-07T16:21:46.876 INFO:tasks.rados.rados.0.smithi100.stdout:1508: finishing rollback tid 1 to smithi10066867-2 2024-04-07T16:21:46.876 INFO:tasks.rados.rados.0.smithi100.stdout:1508: finishing rollback tid 2 to smithi10066867-2 2024-04-07T16:21:46.876 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 2 v 757 (ObjNum 461 snap 140 seq_num 461) dirty exists 2024-04-07T16:21:46.878 INFO:tasks.rados.rados.0.smithi100.stdout:1506: done (5 left) 2024-04-07T16:21:46.878 INFO:tasks.rados.rados.0.smithi100.stdout:1508: done (4 left) 2024-04-07T16:21:46.878 INFO:tasks.rados.rados.0.smithi100.stdout:1512: snap_remove snap 140 2024-04-07T16:21:47.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:46 smithi144 bash[66654]: cephadm 2024-04-07T16:21:45.558643+0000 mgr.y (mgr.25175) 86 : cephadm [INF] Reconfiguring mon.b (monmap changed)... 2024-04-07T16:21:47.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:46 smithi144 bash[66654]: cephadm 2024-04-07T16:21:45.565033+0000 mgr.y (mgr.25175) 87 : cephadm [INF] Reconfiguring daemon mon.b on smithi144 2024-04-07T16:21:47.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:46 smithi144 bash[66654]: cluster 2024-04-07T16:21:45.830314+0000 mon.a (mon.0) 292 : cluster [DBG] osdmap e357: 8 total, 8 up, 8 in 2024-04-07T16:21:47.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:46 smithi144 bash[66654]: audit 2024-04-07T16:21:46.704143+0000 mon.a (mon.0) 293 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:21:47.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:46 smithi144 bash[66654]: cluster 2024-04-07T16:21:46.783825+0000 mon.a (mon.0) 294 : cluster [DBG] osdmap e358: 8 total, 8 up, 8 in 2024-04-07T16:21:47.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:46 smithi100 bash[74942]: cephadm 2024-04-07T16:21:45.558643+0000 mgr.y (mgr.25175) 86 : cephadm [INF] Reconfiguring mon.b (monmap changed)... 2024-04-07T16:21:47.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:46 smithi100 bash[74942]: cephadm 2024-04-07T16:21:45.565033+0000 mgr.y (mgr.25175) 87 : cephadm [INF] Reconfiguring daemon mon.b on smithi144 2024-04-07T16:21:47.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:46 smithi100 bash[74942]: cluster 2024-04-07T16:21:45.830314+0000 mon.a (mon.0) 292 : cluster [DBG] osdmap e357: 8 total, 8 up, 8 in 2024-04-07T16:21:47.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:46 smithi100 bash[74942]: audit 2024-04-07T16:21:46.704143+0000 mon.a (mon.0) 293 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:21:47.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:46 smithi100 bash[74942]: cluster 2024-04-07T16:21:46.783825+0000 mon.a (mon.0) 294 : cluster [DBG] osdmap e358: 8 total, 8 up, 8 in 2024-04-07T16:21:47.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:46 smithi100 bash[76202]: cephadm 2024-04-07T16:21:45.558643+0000 mgr.y (mgr.25175) 86 : cephadm [INF] Reconfiguring mon.b (monmap changed)... 2024-04-07T16:21:47.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:46 smithi100 bash[76202]: cephadm 2024-04-07T16:21:45.565033+0000 mgr.y (mgr.25175) 87 : cephadm [INF] Reconfiguring daemon mon.b on smithi144 2024-04-07T16:21:47.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:46 smithi100 bash[76202]: cluster 2024-04-07T16:21:45.830314+0000 mon.a (mon.0) 292 : cluster [DBG] osdmap e357: 8 total, 8 up, 8 in 2024-04-07T16:21:47.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:46 smithi100 bash[76202]: audit 2024-04-07T16:21:46.704143+0000 mon.a (mon.0) 293 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:21:47.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:46 smithi100 bash[76202]: cluster 2024-04-07T16:21:46.783825+0000 mon.a (mon.0) 294 : cluster [DBG] osdmap e358: 8 total, 8 up, 8 in 2024-04-07T16:21:47.875 INFO:tasks.rados.rados.0.smithi100.stdout:1509: finishing write tid 1 to smithi10066867-39 2024-04-07T16:21:47.875 INFO:tasks.rados.rados.0.smithi100.stdout:1509: finishing write tid 2 to smithi10066867-39 2024-04-07T16:21:47.875 INFO:tasks.rados.rados.0.smithi100.stdout:1510: finishing write tid 1 to smithi10066867-47 2024-04-07T16:21:47.875 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 8 v 625 (ObjNum 455 snap 138 seq_num 455) dirty exists 2024-04-07T16:21:47.875 INFO:tasks.rados.rados.0.smithi100.stdout:1507: finishing copy_from racing read to smithi10066867-10 2024-04-07T16:21:47.875 INFO:tasks.rados.rados.0.smithi100.stdout:1510: finishing write tid 2 to smithi10066867-47 2024-04-07T16:21:47.875 INFO:tasks.rados.rados.0.smithi100.stdout:1509: finishing write tid 3 to smithi10066867-39 2024-04-07T16:21:47.875 INFO:tasks.rados.rados.0.smithi100.stdout:1509: oid 39 updating version 0 to 605 2024-04-07T16:21:47.875 INFO:tasks.rados.rados.0.smithi100.stdout:1509: oid 39 version 605 is already newer than 604 2024-04-07T16:21:47.875 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 39 v 605 (ObjNum 465 snap 142 seq_num 465) dirty exists 2024-04-07T16:21:47.876 INFO:tasks.rados.rados.0.smithi100.stdout:1509: left oid 39 (ObjNum 465 snap 142 seq_num 465) 2024-04-07T16:21:47.876 INFO:tasks.rados.rados.0.smithi100.stdout:1507: finishing copy_from to smithi10066867-10 2024-04-07T16:21:47.876 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 10 v 667 (ObjNum 367 snap 110 seq_num 367) dirty exists 2024-04-07T16:21:47.876 INFO:tasks.rados.rados.0.smithi100.stdout:1510: finishing write tid 3 to smithi10066867-47 2024-04-07T16:21:47.876 INFO:tasks.rados.rados.0.smithi100.stdout:1510: oid 47 updating version 0 to 534 2024-04-07T16:21:47.876 INFO:tasks.rados.rados.0.smithi100.stdout:1510: oid 47 version 534 is already newer than 533 2024-04-07T16:21:47.876 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 47 v 534 (ObjNum 466 snap 142 seq_num 466) dirty exists 2024-04-07T16:21:47.876 INFO:tasks.rados.rados.0.smithi100.stdout:1510: left oid 47 (ObjNum 466 snap 142 seq_num 466) 2024-04-07T16:21:47.876 INFO:tasks.rados.rados.0.smithi100.stdout:1507: done (4 left) 2024-04-07T16:21:47.876 INFO:tasks.rados.rados.0.smithi100.stdout:1509: done (3 left) 2024-04-07T16:21:47.876 INFO:tasks.rados.rados.0.smithi100.stdout:1510: done (2 left) 2024-04-07T16:21:47.876 INFO:tasks.rados.rados.0.smithi100.stdout:1511: done (1 left) 2024-04-07T16:21:47.876 INFO:tasks.rados.rados.0.smithi100.stdout:1512: done (0 left) 2024-04-07T16:21:47.876 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 8 current snap is 142 2024-04-07T16:21:47.876 INFO:tasks.rados.rados.0.smithi100.stdout:1513: seq_num 467 ranges {2572288=630784} 2024-04-07T16:21:47.895 INFO:tasks.rados.rados.0.smithi100.stdout:1513: writing smithi10066867-8 from 2572288 to 3203072 tid 1 2024-04-07T16:21:47.895 INFO:tasks.rados.rados.0.smithi100.stdout:1514: setattr oid 14 current snap is 142 2024-04-07T16:21:47.896 INFO:tasks.rados.rados.0.smithi100.stdout:1515: snap_remove snap 141 2024-04-07T16:21:48.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:47 smithi144 bash[66654]: cluster 2024-04-07T16:21:46.632861+0000 mgr.y (mgr.25175) 88 : cluster [DBG] pgmap v125: 121 pgs: 121 active+clean; 172 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail 2024-04-07T16:21:48.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:47 smithi144 bash[66654]: audit 2024-04-07T16:21:47.637600+0000 mon.a (mon.0) 295 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:21:48.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:47 smithi144 bash[66654]: audit 2024-04-07T16:21:47.650976+0000 mon.a (mon.0) 296 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:21:48.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:47 smithi144 bash[66654]: audit 2024-04-07T16:21:47.690417+0000 mon.a (mon.0) 297 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-07T16:21:48.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:47 smithi144 bash[66654]: audit 2024-04-07T16:21:47.692531+0000 mon.a (mon.0) 298 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-07T16:21:48.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:47 smithi144 bash[66654]: audit 2024-04-07T16:21:47.694377+0000 mon.a (mon.0) 299 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:21:48.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:47 smithi100 bash[74942]: cluster 2024-04-07T16:21:46.632861+0000 mgr.y (mgr.25175) 88 : cluster [DBG] pgmap v125: 121 pgs: 121 active+clean; 172 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail 2024-04-07T16:21:48.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:47 smithi100 bash[74942]: audit 2024-04-07T16:21:47.637600+0000 mon.a (mon.0) 295 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:21:48.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:47 smithi100 bash[74942]: audit 2024-04-07T16:21:47.650976+0000 mon.a (mon.0) 296 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:21:48.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:47 smithi100 bash[74942]: audit 2024-04-07T16:21:47.690417+0000 mon.a (mon.0) 297 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-07T16:21:48.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:47 smithi100 bash[74942]: audit 2024-04-07T16:21:47.692531+0000 mon.a (mon.0) 298 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-07T16:21:48.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:47 smithi100 bash[74942]: audit 2024-04-07T16:21:47.694377+0000 mon.a (mon.0) 299 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:21:48.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:47 smithi100 bash[76202]: cluster 2024-04-07T16:21:46.632861+0000 mgr.y (mgr.25175) 88 : cluster [DBG] pgmap v125: 121 pgs: 121 active+clean; 172 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail 2024-04-07T16:21:48.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:47 smithi100 bash[76202]: audit 2024-04-07T16:21:47.637600+0000 mon.a (mon.0) 295 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:21:48.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:47 smithi100 bash[76202]: audit 2024-04-07T16:21:47.650976+0000 mon.a (mon.0) 296 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:21:48.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:47 smithi100 bash[76202]: audit 2024-04-07T16:21:47.690417+0000 mon.a (mon.0) 297 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-07T16:21:48.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:47 smithi100 bash[76202]: audit 2024-04-07T16:21:47.692531+0000 mon.a (mon.0) 298 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-07T16:21:48.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:47 smithi100 bash[76202]: audit 2024-04-07T16:21:47.694377+0000 mon.a (mon.0) 299 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:21:48.685 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:21:48 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:21:48] "GET /metrics HTTP/1.1" 200 36081 "" "Prometheus/2.43.0" 2024-04-07T16:21:48.869 INFO:tasks.rados.rados.0.smithi100.stdout:1515: done (2 left) 2024-04-07T16:21:48.869 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 5 current snap is 142 2024-04-07T16:21:48.869 INFO:tasks.rados.rados.0.smithi100.stdout:1516: seq_num 468 ranges {5799936=647168} 2024-04-07T16:21:48.917 INFO:tasks.rados.rados.0.smithi100.stdout:1516: writing smithi10066867-5 from 5799936 to 6447104 tid 1 2024-04-07T16:21:48.917 INFO:tasks.rados.rados.0.smithi100.stdout:1517: copy_from oid 28 from oid 2 current snap is 142 2024-04-07T16:21:48.917 INFO:tasks.rados.rados.0.smithi100.stdout:1518: setattr oid 46 current snap is 142 2024-04-07T16:21:48.918 INFO:tasks.rados.rados.0.smithi100.stdout:1513: finishing write tid 1 to smithi10066867-8 2024-04-07T16:21:48.918 INFO:tasks.rados.rados.0.smithi100.stdout:1513: finishing write tid 2 to smithi10066867-8 2024-04-07T16:21:48.918 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 14 v 718 (ObjNum 462 snap 141 seq_num 462) dirty exists 2024-04-07T16:21:48.918 INFO:tasks.rados.rados.0.smithi100.stdout:1513: finishing write tid 3 to smithi10066867-8 2024-04-07T16:21:48.918 INFO:tasks.rados.rados.0.smithi100.stdout:1513: oid 8 updating version 0 to 629 2024-04-07T16:21:48.918 INFO:tasks.rados.rados.0.smithi100.stdout:1513: oid 8 version 629 is already newer than 628 2024-04-07T16:21:48.918 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 8 v 629 (ObjNum 467 snap 142 seq_num 467) dirty exists 2024-04-07T16:21:48.918 INFO:tasks.rados.rados.0.smithi100.stdout:1513: left oid 8 (ObjNum 467 snap 142 seq_num 467) 2024-04-07T16:21:48.918 INFO:tasks.rados.rados.0.smithi100.stdout:1513: done (4 left) 2024-04-07T16:21:48.918 INFO:tasks.rados.rados.0.smithi100.stdout:1514: done (3 left) 2024-04-07T16:21:48.919 INFO:tasks.rados.rados.0.smithi100.stdout:1519: rollback oid 44 current snap is 142 2024-04-07T16:21:48.919 INFO:tasks.rados.rados.0.smithi100.stdout:1519: done (3 left) 2024-04-07T16:21:48.919 INFO:tasks.rados.rados.0.smithi100.stdout:1520: rollback oid 41 current snap is 142 2024-04-07T16:21:48.919 INFO:tasks.rados.rados.0.smithi100.stdout:1520: done (3 left) 2024-04-07T16:21:48.919 INFO:tasks.rados.rados.0.smithi100.stdout:1521: snap_create 2024-04-07T16:21:48.922 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 46 v 632 (ObjNum 337 snap 101 seq_num 337) dirty exists 2024-04-07T16:21:48.924 INFO:tasks.rados.rados.0.smithi100.stdout:1516: finishing write tid 1 to smithi10066867-5 2024-04-07T16:21:48.924 INFO:tasks.rados.rados.0.smithi100.stdout:1516: finishing write tid 2 to smithi10066867-5 2024-04-07T16:21:48.925 INFO:tasks.rados.rados.0.smithi100.stdout:1516: finishing write tid 3 to smithi10066867-5 2024-04-07T16:21:48.926 INFO:tasks.rados.rados.0.smithi100.stdout:1516: oid 5 updating version 0 to 817 2024-04-07T16:21:48.926 INFO:tasks.rados.rados.0.smithi100.stdout:1516: oid 5 updating version 817 to 818 2024-04-07T16:21:48.926 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 5 v 818 (ObjNum 468 snap 142 seq_num 468) dirty exists 2024-04-07T16:21:48.926 INFO:tasks.rados.rados.0.smithi100.stdout:1516: left oid 5 (ObjNum 468 snap 142 seq_num 468) 2024-04-07T16:21:48.930 INFO:tasks.rados.rados.0.smithi100.stdout:1517: finishing copy_from racing read to smithi10066867-28 2024-04-07T16:21:48.930 INFO:tasks.rados.rados.0.smithi100.stdout:1517: finishing copy_from to smithi10066867-28 2024-04-07T16:21:48.930 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 28 v 757 (ObjNum 461 snap 140 seq_num 461) dirty exists 2024-04-07T16:21:49.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:48 smithi144 bash[66654]: cephadm 2024-04-07T16:21:47.689654+0000 mgr.y (mgr.25175) 89 : cephadm [INF] Reconfiguring mgr.x (monmap changed)... 2024-04-07T16:21:49.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:48 smithi144 bash[66654]: cephadm 2024-04-07T16:21:47.696456+0000 mgr.y (mgr.25175) 90 : cephadm [INF] Reconfiguring daemon mgr.x on smithi144 2024-04-07T16:21:49.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:48 smithi144 bash[66654]: cluster 2024-04-07T16:21:47.883941+0000 mon.a (mon.0) 300 : cluster [DBG] mgrmap e42: y(active, since 91s), standbys: x 2024-04-07T16:21:49.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:48 smithi144 bash[66654]: cluster 2024-04-07T16:21:47.884151+0000 mon.a (mon.0) 301 : cluster [DBG] osdmap e359: 8 total, 8 up, 8 in 2024-04-07T16:21:49.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:48 smithi100 bash[74942]: cephadm 2024-04-07T16:21:47.689654+0000 mgr.y (mgr.25175) 89 : cephadm [INF] Reconfiguring mgr.x (monmap changed)... 2024-04-07T16:21:49.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:48 smithi100 bash[74942]: cephadm 2024-04-07T16:21:47.696456+0000 mgr.y (mgr.25175) 90 : cephadm [INF] Reconfiguring daemon mgr.x on smithi144 2024-04-07T16:21:49.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:48 smithi100 bash[74942]: cluster 2024-04-07T16:21:47.883941+0000 mon.a (mon.0) 300 : cluster [DBG] mgrmap e42: y(active, since 91s), standbys: x 2024-04-07T16:21:49.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:48 smithi100 bash[74942]: cluster 2024-04-07T16:21:47.884151+0000 mon.a (mon.0) 301 : cluster [DBG] osdmap e359: 8 total, 8 up, 8 in 2024-04-07T16:21:49.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:48 smithi100 bash[76202]: cephadm 2024-04-07T16:21:47.689654+0000 mgr.y (mgr.25175) 89 : cephadm [INF] Reconfiguring mgr.x (monmap changed)... 2024-04-07T16:21:49.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:48 smithi100 bash[76202]: cephadm 2024-04-07T16:21:47.696456+0000 mgr.y (mgr.25175) 90 : cephadm [INF] Reconfiguring daemon mgr.x on smithi144 2024-04-07T16:21:49.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:48 smithi100 bash[76202]: cluster 2024-04-07T16:21:47.883941+0000 mon.a (mon.0) 300 : cluster [DBG] mgrmap e42: y(active, since 91s), standbys: x 2024-04-07T16:21:49.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:48 smithi100 bash[76202]: cluster 2024-04-07T16:21:47.884151+0000 mon.a (mon.0) 301 : cluster [DBG] osdmap e359: 8 total, 8 up, 8 in 2024-04-07T16:21:49.872 INFO:tasks.rados.rados.0.smithi100.stdout:1516: done (3 left) 2024-04-07T16:21:49.872 INFO:tasks.rados.rados.0.smithi100.stdout:1517: done (2 left) 2024-04-07T16:21:49.872 INFO:tasks.rados.rados.0.smithi100.stdout:1518: done (1 left) 2024-04-07T16:21:49.872 INFO:tasks.rados.rados.0.smithi100.stdout:1521: done (0 left) 2024-04-07T16:21:49.872 INFO:tasks.rados.rados.0.smithi100.stdout:1522: delete oid 14 current snap is 143 2024-04-07T16:21:49.878 INFO:tasks.rados.rados.0.smithi100.stdout:1522: done (0 left) 2024-04-07T16:21:49.878 INFO:tasks.rados.rados.0.smithi100.stdout:1523: read oid 24 snap -1 2024-04-07T16:21:49.878 INFO:tasks.rados.rados.0.smithi100.stdout:1523: expect (ObjNum 398 snap 120 seq_num 398) 2024-04-07T16:21:49.878 INFO:tasks.rados.rados.0.smithi100.stdout:1524: read oid 4 snap 142 2024-04-07T16:21:49.878 INFO:tasks.rados.rados.0.smithi100.stdout:1524: expect (ObjNum 428 snap 129 seq_num 428) 2024-04-07T16:21:49.879 INFO:tasks.rados.rados.0.smithi100.stdout:1525: read oid 26 snap 142 2024-04-07T16:21:49.879 INFO:tasks.rados.rados.0.smithi100.stdout:1525: expect deleted 2024-04-07T16:21:49.879 INFO:tasks.rados.rados.0.smithi100.stdout:1526: copy_from oid 2 from oid 27 current snap is 143 2024-04-07T16:21:49.879 INFO:tasks.rados.rados.0.smithi100.stdout:1527: copy_from oid 8 from oid 19 current snap is 143 2024-04-07T16:21:49.879 INFO:tasks.rados.rados.0.smithi100.stdout:1528: delete oid 49 current snap is 143 2024-04-07T16:21:49.884 INFO:tasks.rados.rados.0.smithi100.stdout:1528: done (5 left) 2024-04-07T16:21:49.884 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 22 current snap is 143 2024-04-07T16:21:49.884 INFO:tasks.rados.rados.0.smithi100.stdout:1529: seq_num 471 ranges {24576=696320} 2024-04-07T16:21:49.891 INFO:tasks.rados.rados.0.smithi100.stdout:1529: writing smithi10066867-22 from 24576 to 720896 tid 1 2024-04-07T16:21:49.891 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 41 current snap is 143 2024-04-07T16:21:49.891 INFO:tasks.rados.rados.0.smithi100.stdout:1530: seq_num 472 ranges {1941504=712704} 2024-04-07T16:21:49.917 INFO:tasks.rados.rados.0.smithi100.stdout:1530: writing smithi10066867-41 from 1941504 to 2654208 tid 1 2024-04-07T16:21:49.917 INFO:tasks.rados.rados.0.smithi100.stdout:1531: copy_from oid 3 from oid 36 current snap is 143 2024-04-07T16:21:49.917 INFO:tasks.rados.rados.0.smithi100.stdout:1524: expect (ObjNum 428 snap 129 seq_num 428) 2024-04-07T16:21:50.029 INFO:tasks.rados.rados.0.smithi100.stdout:1526: finishing copy_from racing read to smithi10066867-2 2024-04-07T16:21:50.030 INFO:tasks.rados.rados.0.smithi100.stdout:1530: finishing write tid 1 to smithi10066867-41 2024-04-07T16:21:50.030 INFO:tasks.rados.rados.0.smithi100.stdout:1530: finishing write tid 2 to smithi10066867-41 2024-04-07T16:21:50.030 INFO:tasks.rados.rados.0.smithi100.stdout:1530: finishing write tid 3 to smithi10066867-41 2024-04-07T16:21:50.030 INFO:tasks.rados.rados.0.smithi100.stdout:1530: oid 41 updating version 0 to 822 2024-04-07T16:21:50.030 INFO:tasks.rados.rados.0.smithi100.stdout:1530: oid 41 updating version 822 to 823 2024-04-07T16:21:50.030 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 41 v 823 (ObjNum 472 snap 143 seq_num 472) dirty exists 2024-04-07T16:21:50.030 INFO:tasks.rados.rados.0.smithi100.stdout:1530: left oid 41 (ObjNum 472 snap 143 seq_num 472) 2024-04-07T16:21:50.030 INFO:tasks.rados.rados.0.smithi100.stdout:1526: finishing copy_from to smithi10066867-2 2024-04-07T16:21:50.030 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 2 v 528 (ObjNum 422 snap 127 seq_num 422) dirty exists 2024-04-07T16:21:50.030 INFO:tasks.rados.rados.0.smithi100.stdout:1529: finishing write tid 1 to smithi10066867-22 2024-04-07T16:21:50.030 INFO:tasks.rados.rados.0.smithi100.stdout:1529: finishing write tid 2 to smithi10066867-22 2024-04-07T16:21:50.031 INFO:tasks.rados.rados.0.smithi100.stdout:1529: finishing write tid 3 to smithi10066867-22 2024-04-07T16:21:50.031 INFO:tasks.rados.rados.0.smithi100.stdout:1529: oid 22 updating version 0 to 540 2024-04-07T16:21:50.031 INFO:tasks.rados.rados.0.smithi100.stdout:1529: oid 22 updating version 540 to 541 2024-04-07T16:21:50.031 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 22 v 541 (ObjNum 471 snap 143 seq_num 471) dirty exists 2024-04-07T16:21:50.031 INFO:tasks.rados.rados.0.smithi100.stdout:1529: left oid 22 (ObjNum 471 snap 143 seq_num 471) 2024-04-07T16:21:50.031 INFO:tasks.rados.rados.0.smithi100.stdout:1531: finishing copy_from to smithi10066867-3 2024-04-07T16:21:50.031 INFO:tasks.rados.rados.0.smithi100.stdout:1531: got expected ENOENT (src dne) 2024-04-07T16:21:50.031 INFO:tasks.rados.rados.0.smithi100.stdout:1531: finishing copy_from racing read to smithi10066867-3 2024-04-07T16:21:50.031 INFO:tasks.rados.rados.0.smithi100.stdout:1523: expect (ObjNum 398 snap 120 seq_num 398) 2024-04-07T16:21:50.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:49 smithi100 bash[74942]: cluster 2024-04-07T16:21:48.633645+0000 mgr.y (mgr.25175) 91 : cluster [DBG] pgmap v128: 121 pgs: 121 active+clean; 131 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 2.8 MiB/s rd, 1.6 MiB/s wr, 13 op/s 2024-04-07T16:21:50.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:49 smithi100 bash[74942]: cluster 2024-04-07T16:21:48.885905+0000 mon.a (mon.0) 302 : cluster [DBG] osdmap e360: 8 total, 8 up, 8 in 2024-04-07T16:21:50.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:49 smithi100 bash[74942]: audit 2024-04-07T16:21:49.564843+0000 mon.a (mon.0) 303 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:21:50.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:49 smithi100 bash[74942]: audit 2024-04-07T16:21:49.576217+0000 mon.a (mon.0) 304 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:21:50.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:49 smithi100 bash[74942]: audit 2024-04-07T16:21:49.617645+0000 mon.a (mon.0) 305 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-07T16:21:50.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:49 smithi100 bash[74942]: audit 2024-04-07T16:21:49.619362+0000 mon.a (mon.0) 306 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:21:50.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:49 smithi100 bash[76202]: cluster 2024-04-07T16:21:48.633645+0000 mgr.y (mgr.25175) 91 : cluster [DBG] pgmap v128: 121 pgs: 121 active+clean; 131 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 2.8 MiB/s rd, 1.6 MiB/s wr, 13 op/s 2024-04-07T16:21:50.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:49 smithi100 bash[76202]: cluster 2024-04-07T16:21:48.885905+0000 mon.a (mon.0) 302 : cluster [DBG] osdmap e360: 8 total, 8 up, 8 in 2024-04-07T16:21:50.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:49 smithi100 bash[76202]: audit 2024-04-07T16:21:49.564843+0000 mon.a (mon.0) 303 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:21:50.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:49 smithi100 bash[76202]: audit 2024-04-07T16:21:49.576217+0000 mon.a (mon.0) 304 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:21:50.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:49 smithi100 bash[76202]: audit 2024-04-07T16:21:49.617645+0000 mon.a (mon.0) 305 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-07T16:21:50.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:49 smithi100 bash[76202]: audit 2024-04-07T16:21:49.619362+0000 mon.a (mon.0) 306 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:21:50.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:49 smithi144 bash[66654]: cluster 2024-04-07T16:21:48.633645+0000 mgr.y (mgr.25175) 91 : cluster [DBG] pgmap v128: 121 pgs: 121 active+clean; 131 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 2.8 MiB/s rd, 1.6 MiB/s wr, 13 op/s 2024-04-07T16:21:50.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:49 smithi144 bash[66654]: cluster 2024-04-07T16:21:48.885905+0000 mon.a (mon.0) 302 : cluster [DBG] osdmap e360: 8 total, 8 up, 8 in 2024-04-07T16:21:50.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:49 smithi144 bash[66654]: audit 2024-04-07T16:21:49.564843+0000 mon.a (mon.0) 303 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:21:50.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:49 smithi144 bash[66654]: audit 2024-04-07T16:21:49.576217+0000 mon.a (mon.0) 304 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:21:50.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:49 smithi144 bash[66654]: audit 2024-04-07T16:21:49.617645+0000 mon.a (mon.0) 305 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-07T16:21:50.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:49 smithi144 bash[66654]: audit 2024-04-07T16:21:49.619362+0000 mon.a (mon.0) 306 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:21:51.008 INFO:tasks.rados.rados.0.smithi100.stdout:1527: finishing copy_from racing read to smithi10066867-8 2024-04-07T16:21:51.008 INFO:tasks.rados.rados.0.smithi100.stdout:1527: finishing copy_from to smithi10066867-8 2024-04-07T16:21:51.008 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 8 v 695 (ObjNum 440 snap 134 seq_num 440) dirty exists 2024-04-07T16:21:51.008 INFO:tasks.rados.rados.0.smithi100.stdout:1523: done (7 left) 2024-04-07T16:21:51.010 INFO:tasks.rados.rados.0.smithi100.stdout:1524: done (6 left) 2024-04-07T16:21:51.010 INFO:tasks.rados.rados.0.smithi100.stdout:1525: done (5 left) 2024-04-07T16:21:51.010 INFO:tasks.rados.rados.0.smithi100.stdout:1526: done (4 left) 2024-04-07T16:21:51.010 INFO:tasks.rados.rados.0.smithi100.stdout:1527: done (3 left) 2024-04-07T16:21:51.010 INFO:tasks.rados.rados.0.smithi100.stdout:1529: done (2 left) 2024-04-07T16:21:51.010 INFO:tasks.rados.rados.0.smithi100.stdout:1530: done (1 left) 2024-04-07T16:21:51.010 INFO:tasks.rados.rados.0.smithi100.stdout:1531: done (0 left) 2024-04-07T16:21:51.010 INFO:tasks.rados.rados.0.smithi100.stdout:1532: read oid 20 snap -1 2024-04-07T16:21:51.011 INFO:tasks.rados.rados.0.smithi100.stdout:1532: expect (ObjNum 415 snap 126 seq_num 415) 2024-04-07T16:21:51.011 INFO:tasks.rados.rados.0.smithi100.stdout:1533: snap_remove snap 142 2024-04-07T16:21:51.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:50 smithi144 bash[66654]: cephadm 2024-04-07T16:21:49.616981+0000 mgr.y (mgr.25175) 92 : cephadm [INF] Reconfiguring osd.4 (monmap changed)... 2024-04-07T16:21:51.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:50 smithi144 bash[66654]: cephadm 2024-04-07T16:21:49.624508+0000 mgr.y (mgr.25175) 93 : cephadm [INF] Reconfiguring daemon osd.4 on smithi144 2024-04-07T16:21:51.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:50 smithi144 bash[66654]: cluster 2024-04-07T16:21:49.888285+0000 mon.a (mon.0) 307 : cluster [DBG] osdmap e361: 8 total, 8 up, 8 in 2024-04-07T16:21:51.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:50 smithi100 bash[74942]: cephadm 2024-04-07T16:21:49.616981+0000 mgr.y (mgr.25175) 92 : cephadm [INF] Reconfiguring osd.4 (monmap changed)... 2024-04-07T16:21:51.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:50 smithi100 bash[74942]: cephadm 2024-04-07T16:21:49.624508+0000 mgr.y (mgr.25175) 93 : cephadm [INF] Reconfiguring daemon osd.4 on smithi144 2024-04-07T16:21:51.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:50 smithi100 bash[74942]: cluster 2024-04-07T16:21:49.888285+0000 mon.a (mon.0) 307 : cluster [DBG] osdmap e361: 8 total, 8 up, 8 in 2024-04-07T16:21:51.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:50 smithi100 bash[76202]: cephadm 2024-04-07T16:21:49.616981+0000 mgr.y (mgr.25175) 92 : cephadm [INF] Reconfiguring osd.4 (monmap changed)... 2024-04-07T16:21:51.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:50 smithi100 bash[76202]: cephadm 2024-04-07T16:21:49.624508+0000 mgr.y (mgr.25175) 93 : cephadm [INF] Reconfiguring daemon osd.4 on smithi144 2024-04-07T16:21:51.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:50 smithi100 bash[76202]: cluster 2024-04-07T16:21:49.888285+0000 mon.a (mon.0) 307 : cluster [DBG] osdmap e361: 8 total, 8 up, 8 in 2024-04-07T16:21:51.770 INFO:tasks.rados.rados.0.smithi100.stdout:1533: done (1 left) 2024-04-07T16:21:51.770 INFO:tasks.rados.rados.0.smithi100.stdout:1532: expect (ObjNum 415 snap 126 seq_num 415) 2024-04-07T16:21:51.933 INFO:tasks.rados.rados.0.smithi100.stdout:1534: read oid 10 snap -1 2024-04-07T16:21:51.933 INFO:tasks.rados.rados.0.smithi100.stdout:1534: expect (ObjNum 367 snap 110 seq_num 367) 2024-04-07T16:21:51.933 INFO:tasks.rados.rados.0.smithi100.stdout:1532: done (1 left) 2024-04-07T16:21:51.933 INFO:tasks.rados.rados.0.smithi100.stdout:1535: read oid 21 snap -1 2024-04-07T16:21:51.933 INFO:tasks.rados.rados.0.smithi100.stdout:1535: expect (ObjNum 391 snap 119 seq_num 391) 2024-04-07T16:21:51.933 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 20 current snap is 143 2024-04-07T16:21:51.933 INFO:tasks.rados.rados.0.smithi100.stdout:1536: seq_num 473 ranges {3375104=729088} 2024-04-07T16:21:51.995 INFO:tasks.rados.rados.0.smithi100.stdout:1536: writing smithi10066867-20 from 3375104 to 4104192 tid 1 2024-04-07T16:21:51.995 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 49 current snap is 143 2024-04-07T16:21:51.995 INFO:tasks.rados.rados.0.smithi100.stdout:1537: seq_num 474 ranges {0=745472} 2024-04-07T16:21:52.004 INFO:tasks.rados.rados.0.smithi100.stdout:1537: writing smithi10066867-49 from 0 to 745472 tid 1 2024-04-07T16:21:52.004 INFO:tasks.rados.rados.0.smithi100.stdout:1538: rollback oid 12 current snap is 143 2024-04-07T16:21:52.004 INFO:tasks.rados.rados.0.smithi100.stdout:1538: done (4 left) 2024-04-07T16:21:52.004 INFO:tasks.rados.rados.0.smithi100.stdout:1539: read oid 29 snap -1 2024-04-07T16:21:52.004 INFO:tasks.rados.rados.0.smithi100.stdout:1539: expect (ObjNum 412 snap 125 seq_num 412) 2024-04-07T16:21:52.004 INFO:tasks.rados.rados.0.smithi100.stdout:1540: read oid 47 snap -1 2024-04-07T16:21:52.004 INFO:tasks.rados.rados.0.smithi100.stdout:1540: expect (ObjNum 466 snap 142 seq_num 466) 2024-04-07T16:21:52.004 INFO:tasks.rados.rados.0.smithi100.stdout:1541: read oid 45 snap -1 2024-04-07T16:21:52.004 INFO:tasks.rados.rados.0.smithi100.stdout:1541: expect (ObjNum 416 snap 126 seq_num 416) 2024-04-07T16:21:52.004 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 13 current snap is 143 2024-04-07T16:21:52.004 INFO:tasks.rados.rados.0.smithi100.stdout:1542: seq_num 475 ranges {0=761856} 2024-04-07T16:21:52.012 INFO:tasks.rados.rados.0.smithi100.stdout:1542: writing smithi10066867-13 from 0 to 761856 tid 1 2024-04-07T16:21:52.012 INFO:tasks.rados.rados.0.smithi100.stdout:1543: read oid 14 snap -1 2024-04-07T16:21:52.012 INFO:tasks.rados.rados.0.smithi100.stdout:1543: expect deleted 2024-04-07T16:21:52.013 INFO:tasks.rados.rados.0.smithi100.stdout:1544: read oid 4 snap -1 2024-04-07T16:21:52.013 INFO:tasks.rados.rados.0.smithi100.stdout:1544: expect (ObjNum 428 snap 129 seq_num 428) 2024-04-07T16:21:52.013 INFO:tasks.rados.rados.0.smithi100.stdout:1545: setattr oid 15 current snap is 143 2024-04-07T16:21:52.013 INFO:tasks.rados.rados.0.smithi100.stdout:1535: expect (ObjNum 391 snap 119 seq_num 391) 2024-04-07T16:21:52.062 INFO:tasks.rados.rados.0.smithi100.stdout:1534: expect (ObjNum 367 snap 110 seq_num 367) 2024-04-07T16:21:52.137 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:51 smithi144 bash[66654]: cluster 2024-04-07T16:21:50.634329+0000 mgr.y (mgr.25175) 94 : cluster [DBG] pgmap v131: 121 pgs: 121 active+clean; 131 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 2.8 MiB/s rd, 1.6 MiB/s wr, 13 op/s 2024-04-07T16:21:52.137 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:51 smithi144 bash[66654]: audit 2024-04-07T16:21:51.676146+0000 mon.a (mon.0) 308 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:21:52.137 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:51 smithi144 bash[66654]: audit 2024-04-07T16:21:51.688183+0000 mon.a (mon.0) 309 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:21:52.137 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:51 smithi144 bash[66654]: audit 2024-04-07T16:21:51.694810+0000 mon.a (mon.0) 310 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-07T16:21:52.137 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:51 smithi144 bash[66654]: audit 2024-04-07T16:21:51.696536+0000 mon.a (mon.0) 311 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:21:52.137 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:51 smithi144 bash[66654]: cluster 2024-04-07T16:21:51.784476+0000 mon.a (mon.0) 312 : cluster [DBG] osdmap e362: 8 total, 8 up, 8 in 2024-04-07T16:21:52.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:51 smithi100 bash[76202]: cluster 2024-04-07T16:21:50.634329+0000 mgr.y (mgr.25175) 94 : cluster [DBG] pgmap v131: 121 pgs: 121 active+clean; 131 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 2.8 MiB/s rd, 1.6 MiB/s wr, 13 op/s 2024-04-07T16:21:52.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:51 smithi100 bash[76202]: audit 2024-04-07T16:21:51.676146+0000 mon.a (mon.0) 308 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:21:52.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:51 smithi100 bash[76202]: audit 2024-04-07T16:21:51.688183+0000 mon.a (mon.0) 309 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:21:52.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:51 smithi100 bash[76202]: audit 2024-04-07T16:21:51.694810+0000 mon.a (mon.0) 310 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-07T16:21:52.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:51 smithi100 bash[76202]: audit 2024-04-07T16:21:51.696536+0000 mon.a (mon.0) 311 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:21:52.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:51 smithi100 bash[76202]: cluster 2024-04-07T16:21:51.784476+0000 mon.a (mon.0) 312 : cluster [DBG] osdmap e362: 8 total, 8 up, 8 in 2024-04-07T16:21:52.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:51 smithi100 bash[74942]: cluster 2024-04-07T16:21:50.634329+0000 mgr.y (mgr.25175) 94 : cluster [DBG] pgmap v131: 121 pgs: 121 active+clean; 131 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 2.8 MiB/s rd, 1.6 MiB/s wr, 13 op/s 2024-04-07T16:21:52.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:51 smithi100 bash[74942]: audit 2024-04-07T16:21:51.676146+0000 mon.a (mon.0) 308 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:21:52.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:51 smithi100 bash[74942]: audit 2024-04-07T16:21:51.688183+0000 mon.a (mon.0) 309 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:21:52.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:51 smithi100 bash[74942]: audit 2024-04-07T16:21:51.694810+0000 mon.a (mon.0) 310 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-07T16:21:52.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:51 smithi100 bash[74942]: audit 2024-04-07T16:21:51.696536+0000 mon.a (mon.0) 311 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:21:52.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:51 smithi100 bash[74942]: cluster 2024-04-07T16:21:51.784476+0000 mon.a (mon.0) 312 : cluster [DBG] osdmap e362: 8 total, 8 up, 8 in 2024-04-07T16:21:52.226 INFO:tasks.rados.rados.0.smithi100.stdout:1536: finishing write tid 1 to smithi10066867-20 2024-04-07T16:21:52.226 INFO:tasks.rados.rados.0.smithi100.stdout:1536: finishing write tid 2 to smithi10066867-20 2024-04-07T16:21:52.226 INFO:tasks.rados.rados.0.smithi100.stdout:1536: finishing write tid 3 to smithi10066867-20 2024-04-07T16:21:52.226 INFO:tasks.rados.rados.0.smithi100.stdout:1536: oid 20 updating version 0 to 542 2024-04-07T16:21:52.226 INFO:tasks.rados.rados.0.smithi100.stdout:1536: oid 20 updating version 542 to 543 2024-04-07T16:21:52.226 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 20 v 543 (ObjNum 473 snap 143 seq_num 473) dirty exists 2024-04-07T16:21:52.226 INFO:tasks.rados.rados.0.smithi100.stdout:1536: left oid 20 (ObjNum 473 snap 143 seq_num 473) 2024-04-07T16:21:52.226 INFO:tasks.rados.rados.0.smithi100.stdout:1537: finishing write tid 1 to smithi10066867-49 2024-04-07T16:21:52.226 INFO:tasks.rados.rados.0.smithi100.stdout:1537: finishing write tid 2 to smithi10066867-49 2024-04-07T16:21:52.227 INFO:tasks.rados.rados.0.smithi100.stdout:1540: expect (ObjNum 466 snap 142 seq_num 466) 2024-04-07T16:21:52.244 INFO:tasks.rados.rados.0.smithi100.stdout:1542: finishing write tid 1 to smithi10066867-13 2024-04-07T16:21:52.244 INFO:tasks.rados.rados.0.smithi100.stdout:1542: finishing write tid 2 to smithi10066867-13 2024-04-07T16:21:52.244 INFO:tasks.rados.rados.0.smithi100.stdout:1541: expect (ObjNum 416 snap 126 seq_num 416) 2024-04-07T16:21:52.298 INFO:tasks.rados.rados.0.smithi100.stdout:1537: finishing write tid 3 to smithi10066867-49 2024-04-07T16:21:52.298 INFO:tasks.rados.rados.0.smithi100.stdout:1537: oid 49 updating version 0 to 829 2024-04-07T16:21:52.299 INFO:tasks.rados.rados.0.smithi100.stdout:1537: oid 49 version 829 is already newer than 828 2024-04-07T16:21:52.299 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 49 v 829 (ObjNum 474 snap 143 seq_num 474) dirty exists 2024-04-07T16:21:52.299 INFO:tasks.rados.rados.0.smithi100.stdout:1537: left oid 49 (ObjNum 474 snap 143 seq_num 474) 2024-04-07T16:21:52.299 INFO:tasks.rados.rados.0.smithi100.stdout:1542: finishing write tid 3 to smithi10066867-13 2024-04-07T16:21:52.299 INFO:tasks.rados.rados.0.smithi100.stdout:1542: oid 13 updating version 0 to 697 2024-04-07T16:21:52.299 INFO:tasks.rados.rados.0.smithi100.stdout:1542: oid 13 version 697 is already newer than 696 2024-04-07T16:21:52.299 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 13 v 697 (ObjNum 475 snap 143 seq_num 475) dirty exists 2024-04-07T16:21:52.299 INFO:tasks.rados.rados.0.smithi100.stdout:1542: left oid 13 (ObjNum 475 snap 143 seq_num 475) 2024-04-07T16:21:52.299 INFO:tasks.rados.rados.0.smithi100.stdout:1539: expect (ObjNum 412 snap 125 seq_num 412) 2024-04-07T16:21:52.416 INFO:tasks.rados.rados.0.smithi100.stdout:1544: expect (ObjNum 428 snap 129 seq_num 428) 2024-04-07T16:21:52.472 INFO:tasks.rados.rados.0.smithi100.stdout:1534: done (10 left) 2024-04-07T16:21:52.472 INFO:tasks.rados.rados.0.smithi100.stdout:1535: done (9 left) 2024-04-07T16:21:52.472 INFO:tasks.rados.rados.0.smithi100.stdout:1536: done (8 left) 2024-04-07T16:21:52.472 INFO:tasks.rados.rados.0.smithi100.stdout:1537: done (7 left) 2024-04-07T16:21:52.472 INFO:tasks.rados.rados.0.smithi100.stdout:1539: done (6 left) 2024-04-07T16:21:52.472 INFO:tasks.rados.rados.0.smithi100.stdout:1540: done (5 left) 2024-04-07T16:21:52.472 INFO:tasks.rados.rados.0.smithi100.stdout:1541: done (4 left) 2024-04-07T16:21:52.473 INFO:tasks.rados.rados.0.smithi100.stdout:1542: done (3 left) 2024-04-07T16:21:52.473 INFO:tasks.rados.rados.0.smithi100.stdout:1543: done (2 left) 2024-04-07T16:21:52.473 INFO:tasks.rados.rados.0.smithi100.stdout:1544: done (1 left) 2024-04-07T16:21:52.473 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 6 current snap is 143 2024-04-07T16:21:52.473 INFO:tasks.rados.rados.0.smithi100.stdout:1546: seq_num 476 ranges {7438336=778240} 2024-04-07T16:21:52.571 INFO:tasks.rados.rados.0.smithi100.stdout:1546: writing smithi10066867-6 from 7438336 to 8216576 tid 1 2024-04-07T16:21:52.571 INFO:tasks.rados.rados.0.smithi100.stdout:1547: read oid 42 snap -1 2024-04-07T16:21:52.571 INFO:tasks.rados.rados.0.smithi100.stdout:1547: expect deleted 2024-04-07T16:21:52.571 INFO:tasks.rados.rados.0.smithi100.stdout:1548: rollback oid 40 current snap is 143 2024-04-07T16:21:52.571 INFO:tasks.rados.rados.0.smithi100.stdout:1548: done (3 left) 2024-04-07T16:21:52.571 INFO:tasks.rados.rados.0.smithi100.stdout:1549: copy_from oid 1 from oid 20 current snap is 143 2024-04-07T16:21:52.571 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 15 v 980 (ObjNum 33 snap 0 seq_num 4) dirty exists 2024-04-07T16:21:52.571 INFO:tasks.rados.rados.0.smithi100.stdout:1545: done (3 left) 2024-04-07T16:21:52.571 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 3 current snap is 143 2024-04-07T16:21:52.572 INFO:tasks.rados.rados.0.smithi100.stdout:1550: seq_num 477 ranges {1925120=794624} 2024-04-07T16:21:52.590 INFO:tasks.rados.rados.0.smithi100.stdout:1550: writing smithi10066867-3 from 1925120 to 2719744 tid 1 2024-04-07T16:21:52.590 INFO:tasks.rados.rados.0.smithi100.stdout:1551: read oid 30 snap -1 2024-04-07T16:21:52.591 INFO:tasks.rados.rados.0.smithi100.stdout:1551: expect (ObjNum 433 snap 130 seq_num 433) 2024-04-07T16:21:52.591 INFO:tasks.rados.rados.0.smithi100.stdout:1552: setattr oid 22 current snap is 143 2024-04-07T16:21:52.591 INFO:tasks.rados.rados.0.smithi100.stdout:1546: finishing write tid 1 to smithi10066867-6 2024-04-07T16:21:52.591 INFO:tasks.rados.rados.0.smithi100.stdout:1546: finishing write tid 2 to smithi10066867-6 2024-04-07T16:21:52.591 INFO:tasks.rados.rados.0.smithi100.stdout:1546: finishing write tid 3 to smithi10066867-6 2024-04-07T16:21:52.591 INFO:tasks.rados.rados.0.smithi100.stdout:1546: oid 6 updating version 0 to 768 2024-04-07T16:21:52.592 INFO:tasks.rados.rados.0.smithi100.stdout:1546: oid 6 updating version 768 to 769 2024-04-07T16:21:52.592 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 6 v 769 (ObjNum 476 snap 143 seq_num 476) dirty exists 2024-04-07T16:21:52.592 INFO:tasks.rados.rados.0.smithi100.stdout:1546: left oid 6 (ObjNum 476 snap 143 seq_num 476) 2024-04-07T16:21:52.592 INFO:tasks.rados.rados.0.smithi100.stdout:1549: finishing copy_from racing read to smithi10066867-1 2024-04-07T16:21:52.593 INFO:tasks.rados.rados.0.smithi100.stdout:1546: done (5 left) 2024-04-07T16:21:52.593 INFO:tasks.rados.rados.0.smithi100.stdout:1547: done (4 left) 2024-04-07T16:21:52.593 INFO:tasks.rados.rados.0.smithi100.stdout:1553: rollback oid 37 current snap is 143 2024-04-07T16:21:52.594 INFO:tasks.rados.rados.0.smithi100.stdout:1553: done (4 left) 2024-04-07T16:21:52.594 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 44 current snap is 143 2024-04-07T16:21:52.594 INFO:tasks.rados.rados.0.smithi100.stdout:1554: seq_num 478 ranges {2752512=409600,3162112=401408} 2024-04-07T16:21:52.606 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:21:52 smithi144 bash[59794]: ts=2024-04-07T16:21:52.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:21:52.613 INFO:tasks.rados.rados.0.smithi100.stdout:1554: writing smithi10066867-44 from 2752512 to 3162112 tid 1 2024-04-07T16:21:52.616 INFO:tasks.rados.rados.0.smithi100.stdout:1554: writing smithi10066867-44 from 3162112 to 3563520 tid 2 2024-04-07T16:21:52.617 INFO:tasks.rados.rados.0.smithi100.stdout:1555: rmattr oid 11 current snap is 143 2024-04-07T16:21:52.617 INFO:tasks.rados.rados.0.smithi100.stdout:1549: finishing copy_from to smithi10066867-1 2024-04-07T16:21:52.617 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 1 v 543 (ObjNum 473 snap 143 seq_num 473) dirty exists 2024-04-07T16:21:52.617 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 22 v 544 (ObjNum 471 snap 143 seq_num 471) dirty exists 2024-04-07T16:21:52.617 INFO:tasks.rados.rados.0.smithi100.stdout:1550: finishing write tid 1 to smithi10066867-3 2024-04-07T16:21:52.617 INFO:tasks.rados.rados.0.smithi100.stdout:1550: finishing write tid 2 to smithi10066867-3 2024-04-07T16:21:52.617 INFO:tasks.rados.rados.0.smithi100.stdout:1550: finishing write tid 3 to smithi10066867-3 2024-04-07T16:21:52.617 INFO:tasks.rados.rados.0.smithi100.stdout:1550: oid 3 updating version 0 to 548 2024-04-07T16:21:52.617 INFO:tasks.rados.rados.0.smithi100.stdout:1550: oid 3 updating version 548 to 549 2024-04-07T16:21:52.617 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 3 v 549 (ObjNum 477 snap 143 seq_num 477) dirty exists 2024-04-07T16:21:52.617 INFO:tasks.rados.rados.0.smithi100.stdout:1550: left oid 3 (ObjNum 477 snap 143 seq_num 477) 2024-04-07T16:21:52.617 INFO:tasks.rados.rados.0.smithi100.stdout:1549: done (5 left) 2024-04-07T16:21:52.617 INFO:tasks.rados.rados.0.smithi100.stdout:1550: done (4 left) 2024-04-07T16:21:52.617 INFO:tasks.rados.rados.0.smithi100.stdout:1552: done (3 left) 2024-04-07T16:21:52.618 INFO:tasks.rados.rados.0.smithi100.stdout:1556: setattr oid 9 current snap is 143 2024-04-07T16:21:52.619 INFO:tasks.rados.rados.0.smithi100.stdout:1557: read oid 28 snap -1 2024-04-07T16:21:52.619 INFO:tasks.rados.rados.0.smithi100.stdout:1557: expect (ObjNum 461 snap 140 seq_num 461) 2024-04-07T16:21:52.619 INFO:tasks.rados.rados.0.smithi100.stdout:1558: setattr oid 46 current snap is 143 2024-04-07T16:21:52.622 INFO:tasks.rados.rados.0.smithi100.stdout:1554: finishing write tid 1 to smithi10066867-44 2024-04-07T16:21:52.622 INFO:tasks.rados.rados.0.smithi100.stdout:1559: read oid 34 snap -1 2024-04-07T16:21:52.622 INFO:tasks.rados.rados.0.smithi100.stdout:1559: expect deleted 2024-04-07T16:21:52.622 INFO:tasks.rados.rados.0.smithi100.stdout:1560: read oid 29 snap -1 2024-04-07T16:21:52.622 INFO:tasks.rados.rados.0.smithi100.stdout:1560: expect (ObjNum 412 snap 125 seq_num 412) 2024-04-07T16:21:52.622 INFO:tasks.rados.rados.0.smithi100.stdout:1561: delete oid 17 current snap is 143 2024-04-07T16:21:52.622 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 11 v 981 (ObjNum 2161 snap 0 seq_num 2703025840) dirty exists 2024-04-07T16:21:52.626 INFO:tasks.rados.rados.0.smithi100.stdout:1554: finishing write tid 2 to smithi10066867-44 2024-04-07T16:21:52.627 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 9 v 539 (ObjNum 113 snap 0 seq_num 2670636976) dirty exists 2024-04-07T16:21:52.628 INFO:tasks.rados.rados.0.smithi100.stdout:1555: done (8 left) 2024-04-07T16:21:52.628 INFO:tasks.rados.rados.0.smithi100.stdout:1556: done (7 left) 2024-04-07T16:21:52.628 INFO:tasks.rados.rados.0.smithi100.stdout:1559: done (6 left) 2024-04-07T16:21:52.628 INFO:tasks.rados.rados.0.smithi100.stdout:1561: done (5 left) 2024-04-07T16:21:52.628 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 10 current snap is 143 2024-04-07T16:21:52.628 INFO:tasks.rados.rados.0.smithi100.stdout:1562: seq_num 480 ranges {3661824=442368,4104192=401408} 2024-04-07T16:21:52.651 INFO:tasks.rados.rados.0.smithi100.stdout:1562: writing smithi10066867-10 from 3661824 to 4104192 tid 1 2024-04-07T16:21:52.654 INFO:tasks.rados.rados.0.smithi100.stdout:1562: writing smithi10066867-10 from 4104192 to 4505600 tid 2 2024-04-07T16:21:52.654 INFO:tasks.rados.rados.0.smithi100.stdout:1563: snap_create 2024-04-07T16:21:52.654 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 46 v 698 (ObjNum 337 snap 101 seq_num 337) dirty exists 2024-04-07T16:21:52.654 INFO:tasks.rados.rados.0.smithi100.stdout:1554: finishing write tid 3 to smithi10066867-44 2024-04-07T16:21:52.655 INFO:tasks.rados.rados.0.smithi100.stdout:1554: finishing write tid 4 to smithi10066867-44 2024-04-07T16:21:52.655 INFO:tasks.rados.rados.0.smithi100.stdout:1554: oid 44 updating version 0 to 770 2024-04-07T16:21:52.655 INFO:tasks.rados.rados.0.smithi100.stdout:1554: oid 44 updating version 770 to 771 2024-04-07T16:21:52.655 INFO:tasks.rados.rados.0.smithi100.stdout:1554: oid 44 updating version 771 to 772 2024-04-07T16:21:52.655 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 44 v 772 (ObjNum 478 snap 143 seq_num 478) dirty exists 2024-04-07T16:21:52.655 INFO:tasks.rados.rados.0.smithi100.stdout:1554: left oid 44 (ObjNum 478 snap 143 seq_num 478) 2024-04-07T16:21:52.655 INFO:tasks.rados.rados.0.smithi100.stdout:1551: expect (ObjNum 433 snap 130 seq_num 433) 2024-04-07T16:21:53.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:52 smithi100 bash[76202]: cephadm 2024-04-07T16:21:51.694083+0000 mgr.y (mgr.25175) 95 : cephadm [INF] Reconfiguring osd.5 (monmap changed)... 2024-04-07T16:21:53.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:52 smithi100 bash[76202]: cephadm 2024-04-07T16:21:51.701703+0000 mgr.y (mgr.25175) 96 : cephadm [INF] Reconfiguring daemon osd.5 on smithi144 2024-04-07T16:21:53.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:52 smithi100 bash[74942]: cephadm 2024-04-07T16:21:51.694083+0000 mgr.y (mgr.25175) 95 : cephadm [INF] Reconfiguring osd.5 (monmap changed)... 2024-04-07T16:21:53.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:52 smithi100 bash[74942]: cephadm 2024-04-07T16:21:51.701703+0000 mgr.y (mgr.25175) 96 : cephadm [INF] Reconfiguring daemon osd.5 on smithi144 2024-04-07T16:21:53.250 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:52 smithi144 bash[66654]: cephadm 2024-04-07T16:21:51.694083+0000 mgr.y (mgr.25175) 95 : cephadm [INF] Reconfiguring osd.5 (monmap changed)... 2024-04-07T16:21:53.250 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:52 smithi144 bash[66654]: cephadm 2024-04-07T16:21:51.701703+0000 mgr.y (mgr.25175) 96 : cephadm [INF] Reconfiguring daemon osd.5 on smithi144 2024-04-07T16:21:53.297 INFO:tasks.rados.rados.0.smithi100.stdout:1557: expect (ObjNum 461 snap 140 seq_num 461) 2024-04-07T16:21:53.371 INFO:tasks.rados.rados.0.smithi100.stdout:1560: expect (ObjNum 412 snap 125 seq_num 412) 2024-04-07T16:21:53.489 INFO:tasks.rados.rados.0.smithi100.stdout:1562: finishing write tid 1 to smithi10066867-10 2024-04-07T16:21:53.490 INFO:tasks.rados.rados.0.smithi100.stdout:1562: finishing write tid 2 to smithi10066867-10 2024-04-07T16:21:53.490 INFO:tasks.rados.rados.0.smithi100.stdout:1562: finishing write tid 3 to smithi10066867-10 2024-04-07T16:21:53.490 INFO:tasks.rados.rados.0.smithi100.stdout:1562: finishing write tid 4 to smithi10066867-10 2024-04-07T16:21:53.490 INFO:tasks.rados.rados.0.smithi100.stdout:1562: oid 10 updating version 0 to 838 2024-04-07T16:21:53.490 INFO:tasks.rados.rados.0.smithi100.stdout:1562: oid 10 version 838 is already newer than 836 2024-04-07T16:21:53.490 INFO:tasks.rados.rados.0.smithi100.stdout:1562: oid 10 version 838 is already newer than 837 2024-04-07T16:21:53.490 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 10 v 838 (ObjNum 480 snap 143 seq_num 480) dirty exists 2024-04-07T16:21:53.490 INFO:tasks.rados.rados.0.smithi100.stdout:1562: left oid 10 (ObjNum 480 snap 143 seq_num 480) 2024-04-07T16:21:53.490 INFO:tasks.rados.rados.0.smithi100.stdout:1551: done (6 left) 2024-04-07T16:21:53.490 INFO:tasks.rados.rados.0.smithi100.stdout:1554: done (5 left) 2024-04-07T16:21:53.490 INFO:tasks.rados.rados.0.smithi100.stdout:1557: done (4 left) 2024-04-07T16:21:53.490 INFO:tasks.rados.rados.0.smithi100.stdout:1558: done (3 left) 2024-04-07T16:21:53.490 INFO:tasks.rados.rados.0.smithi100.stdout:1560: done (2 left) 2024-04-07T16:21:53.490 INFO:tasks.rados.rados.0.smithi100.stdout:1562: done (1 left) 2024-04-07T16:21:53.491 INFO:tasks.rados.rados.0.smithi100.stdout:1563: done (0 left) 2024-04-07T16:21:53.491 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 9 current snap is 144 2024-04-07T16:21:53.491 INFO:tasks.rados.rados.0.smithi100.stdout:1564: seq_num 481 ranges {0=458752,458752=401408} 2024-04-07T16:21:53.491 INFO:tasks.rados.rados.0.smithi100.stdout:1564: writing smithi10066867-9 from 0 to 458752 tid 1 2024-04-07T16:21:53.494 INFO:tasks.rados.rados.0.smithi100.stdout:1564: writing smithi10066867-9 from 458752 to 860160 tid 2 2024-04-07T16:21:53.494 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 10 current snap is 144 2024-04-07T16:21:53.494 INFO:tasks.rados.rados.0.smithi100.stdout:1565: seq_num 482 ranges {4505600=475136,4980736=401408} 2024-04-07T16:21:53.521 INFO:tasks.rados.rados.0.smithi100.stdout:1565: writing smithi10066867-10 from 4505600 to 4980736 tid 1 2024-04-07T16:21:53.525 INFO:tasks.rados.rados.0.smithi100.stdout:1565: writing smithi10066867-10 from 4980736 to 5382144 tid 2 2024-04-07T16:21:53.525 INFO:tasks.rados.rados.0.smithi100.stdout:1566: copy_from oid 6 from oid 7 current snap is 144 2024-04-07T16:21:53.525 INFO:tasks.rados.rados.0.smithi100.stdout:1564: finishing write tid 1 to smithi10066867-9 2024-04-07T16:21:53.525 INFO:tasks.rados.rados.0.smithi100.stdout:1564: finishing write tid 2 to smithi10066867-9 2024-04-07T16:21:53.525 INFO:tasks.rados.rados.0.smithi100.stdout:1564: finishing write tid 3 to smithi10066867-9 2024-04-07T16:21:53.525 INFO:tasks.rados.rados.0.smithi100.stdout:1564: finishing write tid 4 to smithi10066867-9 2024-04-07T16:21:53.525 INFO:tasks.rados.rados.0.smithi100.stdout:1564: oid 9 updating version 0 to 541 2024-04-07T16:21:53.525 INFO:tasks.rados.rados.0.smithi100.stdout:1564: oid 9 updating version 541 to 542 2024-04-07T16:21:53.525 INFO:tasks.rados.rados.0.smithi100.stdout:1564: oid 9 version 542 is already newer than 540 2024-04-07T16:21:53.525 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 9 v 542 (ObjNum 481 snap 144 seq_num 481) dirty exists 2024-04-07T16:21:53.526 INFO:tasks.rados.rados.0.smithi100.stdout:1564: left oid 9 (ObjNum 481 snap 144 seq_num 481) 2024-04-07T16:21:53.526 INFO:tasks.rados.rados.0.smithi100.stdout:1564: done (2 left) 2024-04-07T16:21:53.526 INFO:tasks.rados.rados.0.smithi100.stdout:1567: rmattr oid 50 current snap is 144 2024-04-07T16:21:53.526 INFO:tasks.rados.rados.0.smithi100.stdout:1567: done (2 left) 2024-04-07T16:21:53.526 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 38 current snap is 144 2024-04-07T16:21:53.526 INFO:tasks.rados.rados.0.smithi100.stdout:1568: seq_num 483 ranges {0=491520,491520=401408} 2024-04-07T16:21:53.529 INFO:tasks.rados.rados.0.smithi100.stdout:1568: writing smithi10066867-38 from 0 to 491520 tid 1 2024-04-07T16:21:53.531 INFO:tasks.rados.rados.0.smithi100.stdout:1568: writing smithi10066867-38 from 491520 to 892928 tid 2 2024-04-07T16:21:53.532 INFO:tasks.rados.rados.0.smithi100.stdout:1569: delete oid 18 current snap is 144 2024-04-07T16:21:53.532 INFO:tasks.rados.rados.0.smithi100.stdout:1565: finishing write tid 1 to smithi10066867-10 2024-04-07T16:21:53.536 INFO:tasks.rados.rados.0.smithi100.stdout:1565: finishing write tid 2 to smithi10066867-10 2024-04-07T16:21:53.536 INFO:tasks.rados.rados.0.smithi100.stdout:1565: finishing write tid 3 to smithi10066867-10 2024-04-07T16:21:53.538 INFO:tasks.rados.rados.0.smithi100.stdout:1569: done (3 left) 2024-04-07T16:21:53.538 INFO:tasks.rados.rados.0.smithi100.stdout:1570: read oid 3 snap -1 2024-04-07T16:21:53.538 INFO:tasks.rados.rados.0.smithi100.stdout:1570: expect (ObjNum 477 snap 143 seq_num 477) 2024-04-07T16:21:53.538 INFO:tasks.rados.rados.0.smithi100.stdout:1571: snap_create 2024-04-07T16:21:53.539 INFO:tasks.rados.rados.0.smithi100.stdout:1565: finishing write tid 4 to smithi10066867-10 2024-04-07T16:21:53.539 INFO:tasks.rados.rados.0.smithi100.stdout:1565: oid 10 updating version 0 to 841 2024-04-07T16:21:53.539 INFO:tasks.rados.rados.0.smithi100.stdout:1565: oid 10 version 841 is already newer than 839 2024-04-07T16:21:53.539 INFO:tasks.rados.rados.0.smithi100.stdout:1565: oid 10 version 841 is already newer than 840 2024-04-07T16:21:53.539 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 10 v 841 (ObjNum 482 snap 144 seq_num 482) dirty exists 2024-04-07T16:21:53.539 INFO:tasks.rados.rados.0.smithi100.stdout:1565: left oid 10 (ObjNum 482 snap 144 seq_num 482) 2024-04-07T16:21:53.543 INFO:tasks.rados.rados.0.smithi100.stdout:1568: finishing write tid 1 to smithi10066867-38 2024-04-07T16:21:53.543 INFO:tasks.rados.rados.0.smithi100.stdout:1568: finishing write tid 2 to smithi10066867-38 2024-04-07T16:21:53.544 INFO:tasks.rados.rados.0.smithi100.stdout:1568: finishing write tid 3 to smithi10066867-38 2024-04-07T16:21:53.546 INFO:tasks.rados.rados.0.smithi100.stdout:1566: finishing copy_from racing read to smithi10066867-6 2024-04-07T16:21:53.549 INFO:tasks.rados.rados.0.smithi100.stdout:1566: finishing copy_from to smithi10066867-6 2024-04-07T16:21:53.550 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 6 v 533 (ObjNum 447 snap 137 seq_num 447) dirty exists 2024-04-07T16:21:53.553 INFO:tasks.rados.rados.0.smithi100.stdout:1568: finishing write tid 4 to smithi10066867-38 2024-04-07T16:21:53.553 INFO:tasks.rados.rados.0.smithi100.stdout:1568: oid 38 updating version 0 to 551 2024-04-07T16:21:53.553 INFO:tasks.rados.rados.0.smithi100.stdout:1568: oid 38 version 551 is already newer than 550 2024-04-07T16:21:53.553 INFO:tasks.rados.rados.0.smithi100.stdout:1568: oid 38 updating version 551 to 552 2024-04-07T16:21:53.553 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 38 v 552 (ObjNum 483 snap 144 seq_num 483) dirty exists 2024-04-07T16:21:53.553 INFO:tasks.rados.rados.0.smithi100.stdout:1568: left oid 38 (ObjNum 483 snap 144 seq_num 483) 2024-04-07T16:21:53.566 INFO:tasks.rados.rados.0.smithi100.stdout:1570: expect (ObjNum 477 snap 143 seq_num 477) 2024-04-07T16:21:53.606 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:21:53 smithi144 bash[59794]: ts=2024-04-07T16:21:53.509Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=nodes name=CephNodeDiskspaceWarning index=4 msg="Evaluating rule failed" rule="alert: CephNodeDiskspaceWarning\nexpr: predict_linear(node_filesystem_free_bytes{device=~\"/.*\"}[2d], 3600 * 24 * 5)\n * on (instance) group_left (nodename) node_uname_info < 0\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.8.4\n severity: warning\n type: ceph_default\nannotations:\n description: Mountpoint {{ $labels.mountpoint }} on {{ $labels.nodename }} will\n be full in less than 5 days based on the 48 hour trailing fill rate.\n summary: Host filesystem free space is getting low\n" err="found duplicate series for the match group {instance=\"smithi100\"} on the right hand-side of the operation: [{__name__=\"node_uname_info\", domainname=\"(none)\", instance=\"smithi100\", job=\"node\", machine=\"x86_64\", nodename=\"smithi100\", release=\"5.15.0-101-generic\", sysname=\"Linux\", version=\"#111-Ubuntu SMP Tue Mar 5 20:16:58 UTC 2024\"}, {__name__=\"node_uname_info\", cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", domainname=\"(none)\", instance=\"smithi100\", job=\"node\", machine=\"x86_64\", nodename=\"smithi100\", release=\"5.15.0-101-generic\", sysname=\"Linux\", version=\"#111-Ubuntu SMP Tue Mar 5 20:16:58 UTC 2024\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:21:53.907 INFO:tasks.rados.rados.0.smithi100.stdout:1565: done (4 left) 2024-04-07T16:21:53.908 INFO:tasks.rados.rados.0.smithi100.stdout:1566: done (3 left) 2024-04-07T16:21:53.908 INFO:tasks.rados.rados.0.smithi100.stdout:1568: done (2 left) 2024-04-07T16:21:53.908 INFO:tasks.rados.rados.0.smithi100.stdout:1570: done (1 left) 2024-04-07T16:21:53.908 INFO:tasks.rados.rados.0.smithi100.stdout:1571: done (0 left) 2024-04-07T16:21:53.908 INFO:tasks.rados.rados.0.smithi100.stdout:1572: delete oid 24 current snap is 145 2024-04-07T16:21:53.920 INFO:tasks.rados.rados.0.smithi100.stdout:1572: done (0 left) 2024-04-07T16:21:53.920 INFO:tasks.rados.rados.0.smithi100.stdout:1573: snap_create 2024-04-07T16:21:54.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:53 smithi100 bash[74942]: cluster 2024-04-07T16:21:52.636355+0000 mgr.y (mgr.25175) 97 : cluster [DBG] pgmap v133: 121 pgs: 121 active+clean; 130 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 19 MiB/s rd, 2.6 MiB/s wr, 26 op/s 2024-04-07T16:21:54.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:53 smithi100 bash[74942]: cluster 2024-04-07T16:21:52.903500+0000 mon.a (mon.0) 313 : cluster [DBG] osdmap e363: 8 total, 8 up, 8 in 2024-04-07T16:21:54.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:53 smithi100 bash[74942]: audit 2024-04-07T16:21:53.748000+0000 mon.a (mon.0) 314 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:21:54.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:53 smithi100 bash[74942]: audit 2024-04-07T16:21:53.755344+0000 mon.a (mon.0) 315 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:21:54.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:53 smithi100 bash[74942]: audit 2024-04-07T16:21:53.777324+0000 mon.a (mon.0) 316 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-07T16:21:54.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:53 smithi100 bash[74942]: audit 2024-04-07T16:21:53.778857+0000 mon.a (mon.0) 317 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:21:54.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:53 smithi100 bash[76202]: cluster 2024-04-07T16:21:52.636355+0000 mgr.y (mgr.25175) 97 : cluster [DBG] pgmap v133: 121 pgs: 121 active+clean; 130 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 19 MiB/s rd, 2.6 MiB/s wr, 26 op/s 2024-04-07T16:21:54.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:53 smithi100 bash[76202]: cluster 2024-04-07T16:21:52.903500+0000 mon.a (mon.0) 313 : cluster [DBG] osdmap e363: 8 total, 8 up, 8 in 2024-04-07T16:21:54.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:53 smithi100 bash[76202]: audit 2024-04-07T16:21:53.748000+0000 mon.a (mon.0) 314 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:21:54.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:53 smithi100 bash[76202]: audit 2024-04-07T16:21:53.755344+0000 mon.a (mon.0) 315 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:21:54.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:53 smithi100 bash[76202]: audit 2024-04-07T16:21:53.777324+0000 mon.a (mon.0) 316 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-07T16:21:54.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:53 smithi100 bash[76202]: audit 2024-04-07T16:21:53.778857+0000 mon.a (mon.0) 317 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:21:54.210 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:53 smithi144 bash[66654]: cluster 2024-04-07T16:21:52.636355+0000 mgr.y (mgr.25175) 97 : cluster [DBG] pgmap v133: 121 pgs: 121 active+clean; 130 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 19 MiB/s rd, 2.6 MiB/s wr, 26 op/s 2024-04-07T16:21:54.210 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:53 smithi144 bash[66654]: cluster 2024-04-07T16:21:52.903500+0000 mon.a (mon.0) 313 : cluster [DBG] osdmap e363: 8 total, 8 up, 8 in 2024-04-07T16:21:54.210 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:53 smithi144 bash[66654]: audit 2024-04-07T16:21:53.748000+0000 mon.a (mon.0) 314 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:21:54.211 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:53 smithi144 bash[66654]: audit 2024-04-07T16:21:53.755344+0000 mon.a (mon.0) 315 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:21:54.211 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:53 smithi144 bash[66654]: audit 2024-04-07T16:21:53.777324+0000 mon.a (mon.0) 316 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-07T16:21:54.211 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:53 smithi144 bash[66654]: audit 2024-04-07T16:21:53.778857+0000 mon.a (mon.0) 317 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:21:54.917 INFO:tasks.rados.rados.0.smithi100.stdout:1573: done (0 left) 2024-04-07T16:21:54.918 INFO:tasks.rados.rados.0.smithi100.stdout:1574: snap_remove snap 143 2024-04-07T16:21:55.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:54 smithi100 bash[74942]: cephadm 2024-04-07T16:21:53.776727+0000 mgr.y (mgr.25175) 98 : cephadm [INF] Reconfiguring osd.6 (monmap changed)... 2024-04-07T16:21:55.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:54 smithi100 bash[74942]: cephadm 2024-04-07T16:21:53.783697+0000 mgr.y (mgr.25175) 99 : cephadm [INF] Reconfiguring daemon osd.6 on smithi144 2024-04-07T16:21:55.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:54 smithi100 bash[74942]: cluster 2024-04-07T16:21:53.924258+0000 mon.a (mon.0) 318 : cluster [DBG] osdmap e364: 8 total, 8 up, 8 in 2024-04-07T16:21:55.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:54 smithi100 bash[76202]: cephadm 2024-04-07T16:21:53.776727+0000 mgr.y (mgr.25175) 98 : cephadm [INF] Reconfiguring osd.6 (monmap changed)... 2024-04-07T16:21:55.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:54 smithi100 bash[76202]: cephadm 2024-04-07T16:21:53.783697+0000 mgr.y (mgr.25175) 99 : cephadm [INF] Reconfiguring daemon osd.6 on smithi144 2024-04-07T16:21:55.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:54 smithi100 bash[76202]: cluster 2024-04-07T16:21:53.924258+0000 mon.a (mon.0) 318 : cluster [DBG] osdmap e364: 8 total, 8 up, 8 in 2024-04-07T16:21:55.292 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:54 smithi144 bash[66654]: cephadm 2024-04-07T16:21:53.776727+0000 mgr.y (mgr.25175) 98 : cephadm [INF] Reconfiguring osd.6 (monmap changed)... 2024-04-07T16:21:55.292 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:54 smithi144 bash[66654]: cephadm 2024-04-07T16:21:53.783697+0000 mgr.y (mgr.25175) 99 : cephadm [INF] Reconfiguring daemon osd.6 on smithi144 2024-04-07T16:21:55.292 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:54 smithi144 bash[66654]: cluster 2024-04-07T16:21:53.924258+0000 mon.a (mon.0) 318 : cluster [DBG] osdmap e364: 8 total, 8 up, 8 in 2024-04-07T16:21:55.929 INFO:tasks.rados.rados.0.smithi100.stdout:1574: done (0 left) 2024-04-07T16:21:55.929 INFO:tasks.rados.rados.0.smithi100.stdout:1575: snap_create 2024-04-07T16:21:56.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:55 smithi100 bash[74942]: cluster 2024-04-07T16:21:54.637130+0000 mgr.y (mgr.25175) 100 : cluster [DBG] pgmap v136: 121 pgs: 121 active+clean; 130 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 19 MiB/s rd, 2.6 MiB/s wr, 26 op/s 2024-04-07T16:21:56.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:55 smithi100 bash[74942]: cluster 2024-04-07T16:21:54.934762+0000 mon.a (mon.0) 319 : cluster [DBG] osdmap e365: 8 total, 8 up, 8 in 2024-04-07T16:21:56.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:55 smithi100 bash[74942]: audit 2024-04-07T16:21:55.796697+0000 mon.a (mon.0) 320 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:21:56.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:55 smithi100 bash[74942]: audit 2024-04-07T16:21:55.810414+0000 mon.a (mon.0) 321 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:21:56.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:55 smithi100 bash[74942]: audit 2024-04-07T16:21:55.853052+0000 mon.a (mon.0) 322 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-07T16:21:56.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:55 smithi100 bash[74942]: audit 2024-04-07T16:21:55.854785+0000 mon.a (mon.0) 323 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:21:56.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:55 smithi100 bash[76202]: cluster 2024-04-07T16:21:54.637130+0000 mgr.y (mgr.25175) 100 : cluster [DBG] pgmap v136: 121 pgs: 121 active+clean; 130 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 19 MiB/s rd, 2.6 MiB/s wr, 26 op/s 2024-04-07T16:21:56.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:55 smithi100 bash[76202]: cluster 2024-04-07T16:21:54.934762+0000 mon.a (mon.0) 319 : cluster [DBG] osdmap e365: 8 total, 8 up, 8 in 2024-04-07T16:21:56.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:55 smithi100 bash[76202]: audit 2024-04-07T16:21:55.796697+0000 mon.a (mon.0) 320 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:21:56.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:55 smithi100 bash[76202]: audit 2024-04-07T16:21:55.810414+0000 mon.a (mon.0) 321 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:21:56.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:55 smithi100 bash[76202]: audit 2024-04-07T16:21:55.853052+0000 mon.a (mon.0) 322 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-07T16:21:56.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:55 smithi100 bash[76202]: audit 2024-04-07T16:21:55.854785+0000 mon.a (mon.0) 323 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:21:56.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:55 smithi144 bash[66654]: cluster 2024-04-07T16:21:54.637130+0000 mgr.y (mgr.25175) 100 : cluster [DBG] pgmap v136: 121 pgs: 121 active+clean; 130 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 19 MiB/s rd, 2.6 MiB/s wr, 26 op/s 2024-04-07T16:21:56.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:55 smithi144 bash[66654]: cluster 2024-04-07T16:21:54.934762+0000 mon.a (mon.0) 319 : cluster [DBG] osdmap e365: 8 total, 8 up, 8 in 2024-04-07T16:21:56.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:55 smithi144 bash[66654]: audit 2024-04-07T16:21:55.796697+0000 mon.a (mon.0) 320 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:21:56.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:55 smithi144 bash[66654]: audit 2024-04-07T16:21:55.810414+0000 mon.a (mon.0) 321 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:21:56.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:55 smithi144 bash[66654]: audit 2024-04-07T16:21:55.853052+0000 mon.a (mon.0) 322 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-07T16:21:56.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:55 smithi144 bash[66654]: audit 2024-04-07T16:21:55.854785+0000 mon.a (mon.0) 323 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:21:56.770 INFO:tasks.rados.rados.0.smithi100.stdout:1575: done (0 left) 2024-04-07T16:21:56.770 INFO:tasks.rados.rados.0.smithi100.stdout:1576: read oid 11 snap -1 2024-04-07T16:21:56.770 INFO:tasks.rados.rados.0.smithi100.stdout:1576: expect (ObjNum 0 snap 0 seq_num 0) 2024-04-07T16:21:56.770 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 24 current snap is 147 2024-04-07T16:21:56.770 INFO:tasks.rados.rados.0.smithi100.stdout:1577: seq_num 486 ranges {0=548864,548864=401408} 2024-04-07T16:21:56.776 INFO:tasks.rados.rados.0.smithi100.stdout:1577: writing smithi10066867-24 from 0 to 548864 tid 1 2024-04-07T16:21:56.781 INFO:tasks.rados.rados.0.smithi100.stdout:1577: writing smithi10066867-24 from 548864 to 950272 tid 2 2024-04-07T16:21:56.781 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 32 current snap is 147 2024-04-07T16:21:56.781 INFO:tasks.rados.rados.0.smithi100.stdout:1578: seq_num 487 ranges {2023424=565248,2588672=401408} 2024-04-07T16:21:56.806 INFO:tasks.rados.rados.0.smithi100.stdout:1578: writing smithi10066867-32 from 2023424 to 2588672 tid 1 2024-04-07T16:21:56.810 INFO:tasks.rados.rados.0.smithi100.stdout:1578: writing smithi10066867-32 from 2588672 to 2990080 tid 2 2024-04-07T16:21:56.810 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 36 current snap is 147 2024-04-07T16:21:56.810 INFO:tasks.rados.rados.0.smithi100.stdout:1579: seq_num 488 ranges {0=581632,581632=401408} 2024-04-07T16:21:56.816 INFO:tasks.rados.rados.0.smithi100.stdout:1579: writing smithi10066867-36 from 0 to 581632 tid 1 2024-04-07T16:21:56.820 INFO:tasks.rados.rados.0.smithi100.stdout:1579: writing smithi10066867-36 from 581632 to 983040 tid 2 2024-04-07T16:21:56.820 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 22 current snap is 147 2024-04-07T16:21:56.820 INFO:tasks.rados.rados.0.smithi100.stdout:1580: seq_num 489 ranges {720896=598016,1318912=401408} 2024-04-07T16:21:56.831 INFO:tasks.rados.rados.0.smithi100.stdout:1580: writing smithi10066867-22 from 720896 to 1318912 tid 1 2024-04-07T16:21:56.835 INFO:tasks.rados.rados.0.smithi100.stdout:1580: writing smithi10066867-22 from 1318912 to 1720320 tid 2 2024-04-07T16:21:56.835 INFO:tasks.rados.rados.0.smithi100.stdout:1581: snap_remove snap 145 2024-04-07T16:21:57.182 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:56 smithi144 bash[66654]: cephadm 2024-04-07T16:21:55.852309+0000 mgr.y (mgr.25175) 101 : cephadm [INF] Reconfiguring osd.7 (monmap changed)... 2024-04-07T16:21:57.183 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:56 smithi144 bash[66654]: cephadm 2024-04-07T16:21:55.860159+0000 mgr.y (mgr.25175) 102 : cephadm [INF] Reconfiguring daemon osd.7 on smithi144 2024-04-07T16:21:57.183 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:56 smithi144 bash[66654]: cluster 2024-04-07T16:21:55.946093+0000 mon.a (mon.0) 324 : cluster [DBG] osdmap e366: 8 total, 8 up, 8 in 2024-04-07T16:21:57.183 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:56 smithi144 bash[66654]: cluster 2024-04-07T16:21:56.786450+0000 mon.a (mon.0) 325 : cluster [DBG] osdmap e367: 8 total, 8 up, 8 in 2024-04-07T16:21:57.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:56 smithi100 bash[74942]: cephadm 2024-04-07T16:21:55.852309+0000 mgr.y (mgr.25175) 101 : cephadm [INF] Reconfiguring osd.7 (monmap changed)... 2024-04-07T16:21:57.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:56 smithi100 bash[74942]: cephadm 2024-04-07T16:21:55.860159+0000 mgr.y (mgr.25175) 102 : cephadm [INF] Reconfiguring daemon osd.7 on smithi144 2024-04-07T16:21:57.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:56 smithi100 bash[74942]: cluster 2024-04-07T16:21:55.946093+0000 mon.a (mon.0) 324 : cluster [DBG] osdmap e366: 8 total, 8 up, 8 in 2024-04-07T16:21:57.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:56 smithi100 bash[74942]: cluster 2024-04-07T16:21:56.786450+0000 mon.a (mon.0) 325 : cluster [DBG] osdmap e367: 8 total, 8 up, 8 in 2024-04-07T16:21:57.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:56 smithi100 bash[76202]: cephadm 2024-04-07T16:21:55.852309+0000 mgr.y (mgr.25175) 101 : cephadm [INF] Reconfiguring osd.7 (monmap changed)... 2024-04-07T16:21:57.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:56 smithi100 bash[76202]: cephadm 2024-04-07T16:21:55.860159+0000 mgr.y (mgr.25175) 102 : cephadm [INF] Reconfiguring daemon osd.7 on smithi144 2024-04-07T16:21:57.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:56 smithi100 bash[76202]: cluster 2024-04-07T16:21:55.946093+0000 mon.a (mon.0) 324 : cluster [DBG] osdmap e366: 8 total, 8 up, 8 in 2024-04-07T16:21:57.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:56 smithi100 bash[76202]: cluster 2024-04-07T16:21:56.786450+0000 mon.a (mon.0) 325 : cluster [DBG] osdmap e367: 8 total, 8 up, 8 in 2024-04-07T16:21:57.773 INFO:tasks.rados.rados.0.smithi100.stdout:1581: done (5 left) 2024-04-07T16:21:57.773 INFO:tasks.rados.rados.0.smithi100.stdout:1582: rollback oid 34 current snap is 147 2024-04-07T16:21:57.773 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 34 to 146 2024-04-07T16:21:57.773 INFO:tasks.rados.rados.0.smithi100.stdout:1576: expect (ObjNum 0 snap 0 seq_num 0) 2024-04-07T16:21:57.773 INFO:tasks.rados.rados.0.smithi100.stdout:1577: finishing write tid 1 to smithi10066867-24 2024-04-07T16:21:57.773 INFO:tasks.rados.rados.0.smithi100.stdout:1577: finishing write tid 2 to smithi10066867-24 2024-04-07T16:21:57.773 INFO:tasks.rados.rados.0.smithi100.stdout:1577: finishing write tid 3 to smithi10066867-24 2024-04-07T16:21:57.773 INFO:tasks.rados.rados.0.smithi100.stdout:1577: finishing write tid 4 to smithi10066867-24 2024-04-07T16:21:57.773 INFO:tasks.rados.rados.0.smithi100.stdout:1577: oid 24 updating version 0 to 700 2024-04-07T16:21:57.773 INFO:tasks.rados.rados.0.smithi100.stdout:1577: oid 24 version 700 is already newer than 699 2024-04-07T16:21:57.773 INFO:tasks.rados.rados.0.smithi100.stdout:1577: oid 24 updating version 700 to 701 2024-04-07T16:21:57.774 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 24 v 701 (ObjNum 486 snap 147 seq_num 486) dirty exists 2024-04-07T16:21:57.774 INFO:tasks.rados.rados.0.smithi100.stdout:1577: left oid 24 (ObjNum 486 snap 147 seq_num 486) 2024-04-07T16:21:57.774 INFO:tasks.rados.rados.0.smithi100.stdout:1578: finishing write tid 1 to smithi10066867-32 2024-04-07T16:21:57.774 INFO:tasks.rados.rados.0.smithi100.stdout:1578: finishing write tid 2 to smithi10066867-32 2024-04-07T16:21:57.774 INFO:tasks.rados.rados.0.smithi100.stdout:1576: done (5 left) 2024-04-07T16:21:57.774 INFO:tasks.rados.rados.0.smithi100.stdout:1577: done (4 left) 2024-04-07T16:21:57.774 INFO:tasks.rados.rados.0.smithi100.stdout:1583: snap_remove snap 144 2024-04-07T16:21:57.902 INFO:teuthology.orchestra.run.smithi100.stdout:true 2024-04-07T16:21:58.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:57 smithi100 bash[74942]: cluster 2024-04-07T16:21:56.637811+0000 mgr.y (mgr.25175) 103 : cluster [DBG] pgmap v139: 121 pgs: 121 active+clean; 130 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail 2024-04-07T16:21:58.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:57 smithi100 bash[74942]: cluster 2024-04-07T16:21:57.788698+0000 mon.a (mon.0) 326 : cluster [DBG] osdmap e368: 8 total, 8 up, 8 in 2024-04-07T16:21:58.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:57 smithi100 bash[74942]: audit 2024-04-07T16:21:57.832839+0000 mon.a (mon.0) 327 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:21:58.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:57 smithi100 bash[74942]: audit 2024-04-07T16:21:57.844723+0000 mon.a (mon.0) 328 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:21:58.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:57 smithi100 bash[74942]: audit 2024-04-07T16:21:57.879687+0000 mon.a (mon.0) 329 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-07T16:21:58.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:57 smithi100 bash[74942]: audit 2024-04-07T16:21:57.881159+0000 mon.a (mon.0) 330 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:21:58.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:57 smithi100 bash[76202]: cluster 2024-04-07T16:21:56.637811+0000 mgr.y (mgr.25175) 103 : cluster [DBG] pgmap v139: 121 pgs: 121 active+clean; 130 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail 2024-04-07T16:21:58.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:57 smithi100 bash[76202]: cluster 2024-04-07T16:21:57.788698+0000 mon.a (mon.0) 326 : cluster [DBG] osdmap e368: 8 total, 8 up, 8 in 2024-04-07T16:21:58.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:57 smithi100 bash[76202]: audit 2024-04-07T16:21:57.832839+0000 mon.a (mon.0) 327 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:21:58.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:57 smithi100 bash[76202]: audit 2024-04-07T16:21:57.844723+0000 mon.a (mon.0) 328 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:21:58.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:57 smithi100 bash[76202]: audit 2024-04-07T16:21:57.879687+0000 mon.a (mon.0) 329 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-07T16:21:58.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:57 smithi100 bash[76202]: audit 2024-04-07T16:21:57.881159+0000 mon.a (mon.0) 330 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:21:58.290 INFO:teuthology.orchestra.run.smithi100.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-07T16:21:58.290 INFO:teuthology.orchestra.run.smithi100.stdout:alertmanager.a smithi100 *:9093,9094 running (8m) 65s ago 8m 14.0M - 0.25.0 c8568f914cd2 5143075e7d26 2024-04-07T16:21:58.290 INFO:teuthology.orchestra.run.smithi100.stdout:grafana.a smithi144 *:3000 running (8m) 40s ago 8m 79.3M - 9.4.7 954c08fa6188 7180c0708acd 2024-04-07T16:21:58.290 INFO:teuthology.orchestra.run.smithi100.stdout:mds.a smithi100 running (9m) 65s ago 9m 15.0M - 18.2.2-1171-g267c8ff5 8a4a01e7186c 92593b0c28ae 2024-04-07T16:21:58.291 INFO:teuthology.orchestra.run.smithi100.stdout:mds.b smithi144 running (9m) 40s ago 9m 18.3M - 18.2.2-1171-g267c8ff5 8a4a01e7186c 3df2ee3e17a4 2024-04-07T16:21:58.291 INFO:teuthology.orchestra.run.smithi100.stdout:mgr.x smithi144 *:8443,9283,8765 running (2m) 40s ago 17m 438M - 19.0.0-2785-gadce92d0 c0ecd4226ac9 233745c292a6 2024-04-07T16:21:58.291 INFO:teuthology.orchestra.run.smithi100.stdout:mgr.y smithi100 *:8443,9283,8765 running (3m) 65s ago 19m 495M - 19.0.0-2785-gadce92d0 c0ecd4226ac9 254dc65cdaa7 2024-04-07T16:21:58.291 INFO:teuthology.orchestra.run.smithi100.stdout:mon.a smithi100 running (112s) 65s ago 19m 38.4M 2048M 19.0.0-2785-gadce92d0 c0ecd4226ac9 718e12655c04 2024-04-07T16:21:58.291 INFO:teuthology.orchestra.run.smithi100.stdout:mon.b smithi144 running (48s) 40s ago 17m 18.2M 2048M 19.0.0-2785-gadce92d0 c0ecd4226ac9 aef817664bff 2024-04-07T16:21:58.291 INFO:teuthology.orchestra.run.smithi100.stdout:mon.c smithi100 running (77s) 65s ago 17m 20.1M 2048M 19.0.0-2785-gadce92d0 c0ecd4226ac9 63e889040f6e 2024-04-07T16:21:58.291 INFO:teuthology.orchestra.run.smithi100.stdout:node-exporter.a smithi100 *:9100 running (8m) 65s ago 8m 8248k - 1.5.0 0da6a335fe13 4bce264c1b26 2024-04-07T16:21:58.291 INFO:teuthology.orchestra.run.smithi100.stdout:node-exporter.b smithi144 *:9100 running (8m) 40s ago 8m 8443k - 1.5.0 0da6a335fe13 b6963413c560 2024-04-07T16:21:58.291 INFO:teuthology.orchestra.run.smithi100.stdout:osd.0 smithi100 running (16m) 65s ago 16m 240M 2524M 18.2.2-1171-g267c8ff5 8a4a01e7186c 188deb247979 2024-04-07T16:21:58.291 INFO:teuthology.orchestra.run.smithi100.stdout:osd.1 smithi100 running (15m) 65s ago 15m 196M 2524M 18.2.2-1171-g267c8ff5 8a4a01e7186c 7f57ca90894c 2024-04-07T16:21:58.291 INFO:teuthology.orchestra.run.smithi100.stdout:osd.2 smithi100 running (14m) 65s ago 14m 232M 2524M 18.2.2-1171-g267c8ff5 8a4a01e7186c f3912ddcc8cc 2024-04-07T16:21:58.291 INFO:teuthology.orchestra.run.smithi100.stdout:osd.3 smithi100 running (13m) 65s ago 13m 169M 2524M 18.2.2-1171-g267c8ff5 8a4a01e7186c b41004090c63 2024-04-07T16:21:58.291 INFO:teuthology.orchestra.run.smithi100.stdout:osd.4 smithi144 running (13m) 40s ago 13m 257M 2524M 18.2.2-1171-g267c8ff5 8a4a01e7186c 8c73b6b4dc13 2024-04-07T16:21:58.291 INFO:teuthology.orchestra.run.smithi100.stdout:osd.5 smithi144 running (12m) 40s ago 12m 216M 2524M 18.2.2-1171-g267c8ff5 8a4a01e7186c 3bfa0cf6f6cb 2024-04-07T16:21:58.291 INFO:teuthology.orchestra.run.smithi100.stdout:osd.6 smithi144 running (11m) 40s ago 11m 98.7M 2524M 18.2.2-1171-g267c8ff5 8a4a01e7186c 32b7f95d77f0 2024-04-07T16:21:58.292 INFO:teuthology.orchestra.run.smithi100.stdout:osd.7 smithi144 running (10m) 40s ago 10m 156M 2524M 18.2.2-1171-g267c8ff5 8a4a01e7186c 45f7709e576a 2024-04-07T16:21:58.292 INFO:teuthology.orchestra.run.smithi100.stdout:prometheus.a smithi144 *:9095 running (2m) 40s ago 9m 46.5M - 2.43.0 a07b618ecd1d 2d0a8131f303 2024-04-07T16:21:58.299 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:57 smithi144 bash[66654]: cluster 2024-04-07T16:21:56.637811+0000 mgr.y (mgr.25175) 103 : cluster [DBG] pgmap v139: 121 pgs: 121 active+clean; 130 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail 2024-04-07T16:21:58.300 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:57 smithi144 bash[66654]: cluster 2024-04-07T16:21:57.788698+0000 mon.a (mon.0) 326 : cluster [DBG] osdmap e368: 8 total, 8 up, 8 in 2024-04-07T16:21:58.300 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:57 smithi144 bash[66654]: audit 2024-04-07T16:21:57.832839+0000 mon.a (mon.0) 327 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:21:58.300 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:57 smithi144 bash[66654]: audit 2024-04-07T16:21:57.844723+0000 mon.a (mon.0) 328 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:21:58.300 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:57 smithi144 bash[66654]: audit 2024-04-07T16:21:57.879687+0000 mon.a (mon.0) 329 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-07T16:21:58.300 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:57 smithi144 bash[66654]: audit 2024-04-07T16:21:57.881159+0000 mon.a (mon.0) 330 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:21:58.685 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:21:58 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:21:58] "GET /metrics HTTP/1.1" 200 36081 "" "Prometheus/2.43.0" 2024-04-07T16:21:58.733 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-04-07T16:21:58.733 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": { 2024-04-07T16:21:58.733 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph version 19.0.0-2785-gadce92d0 (adce92d056533f22dc1ee37e3141d098b046ec1b) squid (dev)": 3 2024-04-07T16:21:58.733 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-07T16:21:58.733 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": { 2024-04-07T16:21:58.733 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph version 19.0.0-2785-gadce92d0 (adce92d056533f22dc1ee37e3141d098b046ec1b) squid (dev)": 2 2024-04-07T16:21:58.733 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-07T16:21:58.733 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": { 2024-04-07T16:21:58.733 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph version 18.2.2-1171-g267c8ff5 (267c8ff5b1ff5815581b3b2381d19360e081b37b) reef (stable)": 8 2024-04-07T16:21:58.733 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-07T16:21:58.733 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": { 2024-04-07T16:21:58.733 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph version 18.2.2-1171-g267c8ff5 (267c8ff5b1ff5815581b3b2381d19360e081b37b) reef (stable)": 2 2024-04-07T16:21:58.733 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-07T16:21:58.734 INFO:teuthology.orchestra.run.smithi100.stdout: "overall": { 2024-04-07T16:21:58.734 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph version 18.2.2-1171-g267c8ff5 (267c8ff5b1ff5815581b3b2381d19360e081b37b) reef (stable)": 10, 2024-04-07T16:21:58.734 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph version 19.0.0-2785-gadce92d0 (adce92d056533f22dc1ee37e3141d098b046ec1b) squid (dev)": 5 2024-04-07T16:21:58.734 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-04-07T16:21:58.734 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-04-07T16:21:58.770 INFO:tasks.rados.rados.0.smithi100.stdout:1583: done (4 left) 2024-04-07T16:21:58.770 INFO:tasks.rados.rados.0.smithi100.stdout:1584: read oid 13 snap 146 2024-04-07T16:21:58.770 INFO:tasks.rados.rados.0.smithi100.stdout:1584: expect (ObjNum 475 snap 143 seq_num 475) 2024-04-07T16:21:58.770 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 6 current snap is 147 2024-04-07T16:21:58.771 INFO:tasks.rados.rados.0.smithi100.stdout:1578: finishing write tid 3 to smithi10066867-32 2024-04-07T16:21:58.771 INFO:tasks.rados.rados.0.smithi100.stdout:1578: finishing write tid 4 to smithi10066867-32 2024-04-07T16:21:58.771 INFO:tasks.rados.rados.0.smithi100.stdout:1578: oid 32 updating version 0 to 760 2024-04-07T16:21:58.771 INFO:tasks.rados.rados.0.smithi100.stdout:1578: oid 32 version 760 is already newer than 758 2024-04-07T16:21:58.771 INFO:tasks.rados.rados.0.smithi100.stdout:1578: oid 32 version 760 is already newer than 759 2024-04-07T16:21:58.771 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 32 v 760 (ObjNum 487 snap 147 seq_num 487) dirty exists 2024-04-07T16:21:58.771 INFO:tasks.rados.rados.0.smithi100.stdout:1578: left oid 32 (ObjNum 487 snap 147 seq_num 487) 2024-04-07T16:21:58.771 INFO:tasks.rados.rados.0.smithi100.stdout:1579: finishing write tid 1 to smithi10066867-36 2024-04-07T16:21:58.771 INFO:tasks.rados.rados.0.smithi100.stdout:1579: finishing write tid 2 to smithi10066867-36 2024-04-07T16:21:58.771 INFO:tasks.rados.rados.0.smithi100.stdout:1585: seq_num 490 ranges {1736704=614400,2351104=401408} 2024-04-07T16:21:58.789 INFO:tasks.rados.rados.0.smithi100.stdout:1585: writing smithi10066867-6 from 1736704 to 2351104 tid 1 2024-04-07T16:21:58.792 INFO:tasks.rados.rados.0.smithi100.stdout:1585: writing smithi10066867-6 from 2351104 to 2752512 tid 2 2024-04-07T16:21:58.792 INFO:tasks.rados.rados.0.smithi100.stdout:1578: done (5 left) 2024-04-07T16:21:58.792 INFO:tasks.rados.rados.0.smithi100.stdout:1586: copy_from oid 35 from oid 15 current snap is 147 2024-04-07T16:21:58.793 INFO:tasks.rados.rados.0.smithi100.stdout:1579: finishing write tid 3 to smithi10066867-36 2024-04-07T16:21:58.793 INFO:tasks.rados.rados.0.smithi100.stdout:1579: finishing write tid 4 to smithi10066867-36 2024-04-07T16:21:58.793 INFO:tasks.rados.rados.0.smithi100.stdout:1579: oid 36 updating version 0 to 702 2024-04-07T16:21:58.793 INFO:tasks.rados.rados.0.smithi100.stdout:1579: oid 36 updating version 702 to 704 2024-04-07T16:21:58.793 INFO:tasks.rados.rados.0.smithi100.stdout:1579: oid 36 version 704 is already newer than 703 2024-04-07T16:21:58.793 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 36 v 704 (ObjNum 488 snap 147 seq_num 488) dirty exists 2024-04-07T16:21:58.793 INFO:tasks.rados.rados.0.smithi100.stdout:1579: left oid 36 (ObjNum 488 snap 147 seq_num 488) 2024-04-07T16:21:58.793 INFO:tasks.rados.rados.0.smithi100.stdout:1580: finishing write tid 1 to smithi10066867-22 2024-04-07T16:21:58.793 INFO:tasks.rados.rados.0.smithi100.stdout:1580: finishing write tid 2 to smithi10066867-22 2024-04-07T16:21:58.793 INFO:tasks.rados.rados.0.smithi100.stdout:1580: finishing write tid 3 to smithi10066867-22 2024-04-07T16:21:58.793 INFO:tasks.rados.rados.0.smithi100.stdout:1580: finishing write tid 4 to smithi10066867-22 2024-04-07T16:21:58.793 INFO:tasks.rados.rados.0.smithi100.stdout:1580: oid 22 updating version 0 to 547 2024-04-07T16:21:58.793 INFO:tasks.rados.rados.0.smithi100.stdout:1580: oid 22 version 547 is already newer than 546 2024-04-07T16:21:58.793 INFO:tasks.rados.rados.0.smithi100.stdout:1580: oid 22 version 547 is already newer than 545 2024-04-07T16:21:58.793 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 22 v 547 (ObjNum 489 snap 147 seq_num 489) dirty exists 2024-04-07T16:21:58.794 INFO:tasks.rados.rados.0.smithi100.stdout:1580: left oid 22 (ObjNum 489 snap 147 seq_num 489) 2024-04-07T16:21:58.794 INFO:tasks.rados.rados.0.smithi100.stdout:1582: finishing rollback tid 1 to smithi10066867-34 2024-04-07T16:21:58.794 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 34 v 0 (ObjNum 305 snap 0 seq_num 3067284992) dirty dne 2024-04-07T16:21:58.794 INFO:tasks.rados.rados.0.smithi100.stdout:1584: expect (ObjNum 475 snap 143 seq_num 475) 2024-04-07T16:21:58.813 INFO:tasks.rados.rados.0.smithi100.stdout:1585: finishing write tid 1 to smithi10066867-6 2024-04-07T16:21:58.813 INFO:tasks.rados.rados.0.smithi100.stdout:1586: finishing copy_from racing read to smithi10066867-35 2024-04-07T16:21:58.813 INFO:tasks.rados.rados.0.smithi100.stdout:1579: done (5 left) 2024-04-07T16:21:58.813 INFO:tasks.rados.rados.0.smithi100.stdout:1580: done (4 left) 2024-04-07T16:21:58.813 INFO:tasks.rados.rados.0.smithi100.stdout:1582: done (3 left) 2024-04-07T16:21:58.813 INFO:tasks.rados.rados.0.smithi100.stdout:1584: done (2 left) 2024-04-07T16:21:58.813 INFO:tasks.rados.rados.0.smithi100.stdout:1587: snap_remove snap 146 2024-04-07T16:21:59.300 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:58 smithi144 bash[66654]: cephadm 2024-04-07T16:21:57.879206+0000 mgr.y (mgr.25175) 104 : cephadm [INF] Reconfiguring mds.b (monmap changed)... 2024-04-07T16:21:59.301 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:58 smithi144 bash[66654]: cephadm 2024-04-07T16:21:57.882680+0000 mgr.y (mgr.25175) 105 : cephadm [INF] Reconfiguring daemon mds.b on smithi144 2024-04-07T16:21:59.301 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:58 smithi144 bash[66654]: audit 2024-04-07T16:21:57.903594+0000 mgr.y (mgr.25175) 106 : audit [DBG] from='client.34221 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:21:59.301 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:58 smithi144 bash[66654]: audit 2024-04-07T16:21:58.266340+0000 mgr.y (mgr.25175) 107 : audit [DBG] from='client.44144 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:21:59.301 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:58 smithi144 bash[66654]: audit 2024-04-07T16:21:58.750189+0000 mon.c (mon.2) 3 : audit [DBG] from='client.? 172.21.15.100:0/1211380758' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:21:59.301 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:58 smithi144 bash[66654]: cluster 2024-04-07T16:21:58.787246+0000 mon.a (mon.0) 331 : cluster [DBG] osdmap e369: 8 total, 8 up, 8 in 2024-04-07T16:21:59.435 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:58 smithi100 bash[74942]: cephadm 2024-04-07T16:21:57.879206+0000 mgr.y (mgr.25175) 104 : cephadm [INF] Reconfiguring mds.b (monmap changed)... 2024-04-07T16:21:59.435 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:58 smithi100 bash[74942]: cephadm 2024-04-07T16:21:57.882680+0000 mgr.y (mgr.25175) 105 : cephadm [INF] Reconfiguring daemon mds.b on smithi144 2024-04-07T16:21:59.435 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:58 smithi100 bash[74942]: audit 2024-04-07T16:21:57.903594+0000 mgr.y (mgr.25175) 106 : audit [DBG] from='client.34221 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:21:59.435 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:58 smithi100 bash[74942]: audit 2024-04-07T16:21:58.266340+0000 mgr.y (mgr.25175) 107 : audit [DBG] from='client.44144 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:21:59.435 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:58 smithi100 bash[74942]: audit 2024-04-07T16:21:58.750189+0000 mon.c (mon.2) 3 : audit [DBG] from='client.? 172.21.15.100:0/1211380758' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:21:59.436 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:58 smithi100 bash[74942]: cluster 2024-04-07T16:21:58.787246+0000 mon.a (mon.0) 331 : cluster [DBG] osdmap e369: 8 total, 8 up, 8 in 2024-04-07T16:21:59.436 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:58 smithi100 bash[76202]: cephadm 2024-04-07T16:21:57.879206+0000 mgr.y (mgr.25175) 104 : cephadm [INF] Reconfiguring mds.b (monmap changed)... 2024-04-07T16:21:59.436 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:58 smithi100 bash[76202]: cephadm 2024-04-07T16:21:57.882680+0000 mgr.y (mgr.25175) 105 : cephadm [INF] Reconfiguring daemon mds.b on smithi144 2024-04-07T16:21:59.436 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:58 smithi100 bash[76202]: audit 2024-04-07T16:21:57.903594+0000 mgr.y (mgr.25175) 106 : audit [DBG] from='client.34221 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:21:59.436 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:58 smithi100 bash[76202]: audit 2024-04-07T16:21:58.266340+0000 mgr.y (mgr.25175) 107 : audit [DBG] from='client.44144 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:21:59.436 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:58 smithi100 bash[76202]: audit 2024-04-07T16:21:58.750189+0000 mon.c (mon.2) 3 : audit [DBG] from='client.? 172.21.15.100:0/1211380758' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:21:59.436 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:58 smithi100 bash[76202]: cluster 2024-04-07T16:21:58.787246+0000 mon.a (mon.0) 331 : cluster [DBG] osdmap e369: 8 total, 8 up, 8 in 2024-04-07T16:21:59.785 INFO:tasks.rados.rados.0.smithi100.stdout:1587: done (2 left) 2024-04-07T16:21:59.785 INFO:tasks.rados.rados.0.smithi100.stdout:1588: snap_create 2024-04-07T16:21:59.786 INFO:tasks.rados.rados.0.smithi100.stdout:1585: finishing write tid 2 to smithi10066867-6 2024-04-07T16:21:59.786 INFO:tasks.rados.rados.0.smithi100.stdout:1585: finishing write tid 3 to smithi10066867-6 2024-04-07T16:21:59.786 INFO:tasks.rados.rados.0.smithi100.stdout:1586: finishing copy_from to smithi10066867-35 2024-04-07T16:21:59.786 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 35 v 980 (ObjNum 65 snap 0 seq_num 1833851508) dirty exists 2024-04-07T16:21:59.786 INFO:tasks.rados.rados.0.smithi100.stdout:1585: finishing write tid 4 to smithi10066867-6 2024-04-07T16:21:59.786 INFO:tasks.rados.rados.0.smithi100.stdout:1585: oid 6 updating version 0 to 780 2024-04-07T16:21:59.786 INFO:tasks.rados.rados.0.smithi100.stdout:1585: oid 6 version 780 is already newer than 779 2024-04-07T16:21:59.786 INFO:tasks.rados.rados.0.smithi100.stdout:1585: oid 6 version 780 is already newer than 778 2024-04-07T16:21:59.786 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 6 v 780 (ObjNum 490 snap 147 seq_num 490) dirty exists 2024-04-07T16:21:59.786 INFO:tasks.rados.rados.0.smithi100.stdout:1585: left oid 6 (ObjNum 490 snap 147 seq_num 490) 2024-04-07T16:22:00.274 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:59 smithi100 bash[74942]: cluster 2024-04-07T16:21:58.638470+0000 mgr.y (mgr.25175) 108 : cluster [DBG] pgmap v142: 121 pgs: 121 active+clean; 121 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 12 MiB/s rd, 3.7 MiB/s wr, 36 op/s 2024-04-07T16:22:00.275 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:59 smithi100 bash[74942]: audit 2024-04-07T16:21:59.702302+0000 mon.a (mon.0) 332 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:22:00.275 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:59 smithi100 bash[74942]: audit 2024-04-07T16:21:59.713763+0000 mon.a (mon.0) 333 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:22:00.275 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:59 smithi100 bash[74942]: audit 2024-04-07T16:21:59.772059+0000 mon.a (mon.0) 334 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:22:00.275 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:59 smithi100 bash[74942]: audit 2024-04-07T16:21:59.777328+0000 mon.a (mon.0) 335 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:22:00.275 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:59 smithi100 bash[74942]: audit 2024-04-07T16:21:59.781419+0000 mon.a (mon.0) 336 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:22:00.275 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:59 smithi100 bash[74942]: audit 2024-04-07T16:21:59.789743+0000 mon.a (mon.0) 337 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:22:00.275 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:59 smithi100 bash[74942]: cluster 2024-04-07T16:21:59.800909+0000 mon.a (mon.0) 338 : cluster [DBG] osdmap e370: 8 total, 8 up, 8 in 2024-04-07T16:22:00.275 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:59 smithi100 bash[74942]: audit 2024-04-07T16:21:59.801350+0000 mon.a (mon.0) 339 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.a"}]: dispatch 2024-04-07T16:22:00.275 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:59 smithi100 bash[74942]: audit 2024-04-07T16:21:59.809954+0000 mon.a (mon.0) 340 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.a"}]': finished 2024-04-07T16:22:00.275 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:59 smithi100 bash[74942]: audit 2024-04-07T16:21:59.814690+0000 mon.a (mon.0) 341 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.b"}]: dispatch 2024-04-07T16:22:00.275 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:59 smithi100 bash[74942]: audit 2024-04-07T16:21:59.820203+0000 mon.a (mon.0) 342 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.b"}]': finished 2024-04-07T16:22:00.275 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:59 smithi100 bash[74942]: audit 2024-04-07T16:21:59.824207+0000 mon.a (mon.0) 343 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.c"}]: dispatch 2024-04-07T16:22:00.275 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:59 smithi100 bash[74942]: audit 2024-04-07T16:21:59.829290+0000 mon.a (mon.0) 344 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.c"}]': finished 2024-04-07T16:22:00.276 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:59 smithi100 bash[74942]: audit 2024-04-07T16:21:59.834417+0000 mon.a (mon.0) 345 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:22:00.276 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:59 smithi100 bash[74942]: audit 2024-04-07T16:21:59.843321+0000 mon.a (mon.0) 346 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:22:00.276 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:21:59 smithi100 bash[74942]: audit 2024-04-07T16:21:59.849971+0000 mon.a (mon.0) 347 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-04-07T16:22:00.276 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:59 smithi100 bash[76202]: cluster 2024-04-07T16:21:58.638470+0000 mgr.y (mgr.25175) 108 : cluster [DBG] pgmap v142: 121 pgs: 121 active+clean; 121 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 12 MiB/s rd, 3.7 MiB/s wr, 36 op/s 2024-04-07T16:22:00.276 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:59 smithi100 bash[76202]: audit 2024-04-07T16:21:59.702302+0000 mon.a (mon.0) 332 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:22:00.276 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:59 smithi100 bash[76202]: audit 2024-04-07T16:21:59.713763+0000 mon.a (mon.0) 333 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:22:00.276 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:59 smithi100 bash[76202]: audit 2024-04-07T16:21:59.772059+0000 mon.a (mon.0) 334 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:22:00.276 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:59 smithi100 bash[76202]: audit 2024-04-07T16:21:59.777328+0000 mon.a (mon.0) 335 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:22:00.276 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:59 smithi100 bash[76202]: audit 2024-04-07T16:21:59.781419+0000 mon.a (mon.0) 336 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:22:00.276 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:59 smithi100 bash[76202]: audit 2024-04-07T16:21:59.789743+0000 mon.a (mon.0) 337 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:22:00.276 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:59 smithi100 bash[76202]: cluster 2024-04-07T16:21:59.800909+0000 mon.a (mon.0) 338 : cluster [DBG] osdmap e370: 8 total, 8 up, 8 in 2024-04-07T16:22:00.277 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:59 smithi100 bash[76202]: audit 2024-04-07T16:21:59.801350+0000 mon.a (mon.0) 339 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.a"}]: dispatch 2024-04-07T16:22:00.277 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:59 smithi100 bash[76202]: audit 2024-04-07T16:21:59.809954+0000 mon.a (mon.0) 340 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.a"}]': finished 2024-04-07T16:22:00.277 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:59 smithi100 bash[76202]: audit 2024-04-07T16:21:59.814690+0000 mon.a (mon.0) 341 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.b"}]: dispatch 2024-04-07T16:22:00.277 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:59 smithi100 bash[76202]: audit 2024-04-07T16:21:59.820203+0000 mon.a (mon.0) 342 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.b"}]': finished 2024-04-07T16:22:00.277 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:59 smithi100 bash[76202]: audit 2024-04-07T16:21:59.824207+0000 mon.a (mon.0) 343 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.c"}]: dispatch 2024-04-07T16:22:00.277 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:59 smithi100 bash[76202]: audit 2024-04-07T16:21:59.829290+0000 mon.a (mon.0) 344 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.c"}]': finished 2024-04-07T16:22:00.277 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:59 smithi100 bash[76202]: audit 2024-04-07T16:21:59.834417+0000 mon.a (mon.0) 345 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:22:00.277 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:59 smithi100 bash[76202]: audit 2024-04-07T16:21:59.843321+0000 mon.a (mon.0) 346 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:22:00.277 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:21:59 smithi100 bash[76202]: audit 2024-04-07T16:21:59.849971+0000 mon.a (mon.0) 347 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-04-07T16:22:00.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:59 smithi144 bash[66654]: cluster 2024-04-07T16:21:58.638470+0000 mgr.y (mgr.25175) 108 : cluster [DBG] pgmap v142: 121 pgs: 121 active+clean; 121 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 12 MiB/s rd, 3.7 MiB/s wr, 36 op/s 2024-04-07T16:22:00.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:59 smithi144 bash[66654]: audit 2024-04-07T16:21:59.702302+0000 mon.a (mon.0) 332 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:22:00.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:59 smithi144 bash[66654]: audit 2024-04-07T16:21:59.713763+0000 mon.a (mon.0) 333 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:22:00.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:59 smithi144 bash[66654]: audit 2024-04-07T16:21:59.772059+0000 mon.a (mon.0) 334 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:22:00.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:59 smithi144 bash[66654]: audit 2024-04-07T16:21:59.777328+0000 mon.a (mon.0) 335 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:22:00.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:59 smithi144 bash[66654]: audit 2024-04-07T16:21:59.781419+0000 mon.a (mon.0) 336 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:22:00.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:59 smithi144 bash[66654]: audit 2024-04-07T16:21:59.789743+0000 mon.a (mon.0) 337 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:22:00.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:59 smithi144 bash[66654]: cluster 2024-04-07T16:21:59.800909+0000 mon.a (mon.0) 338 : cluster [DBG] osdmap e370: 8 total, 8 up, 8 in 2024-04-07T16:22:00.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:59 smithi144 bash[66654]: audit 2024-04-07T16:21:59.801350+0000 mon.a (mon.0) 339 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.a"}]: dispatch 2024-04-07T16:22:00.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:59 smithi144 bash[66654]: audit 2024-04-07T16:21:59.809954+0000 mon.a (mon.0) 340 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.a"}]': finished 2024-04-07T16:22:00.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:59 smithi144 bash[66654]: audit 2024-04-07T16:21:59.814690+0000 mon.a (mon.0) 341 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.b"}]: dispatch 2024-04-07T16:22:00.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:59 smithi144 bash[66654]: audit 2024-04-07T16:21:59.820203+0000 mon.a (mon.0) 342 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.b"}]': finished 2024-04-07T16:22:00.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:59 smithi144 bash[66654]: audit 2024-04-07T16:21:59.824207+0000 mon.a (mon.0) 343 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.c"}]: dispatch 2024-04-07T16:22:00.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:59 smithi144 bash[66654]: audit 2024-04-07T16:21:59.829290+0000 mon.a (mon.0) 344 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.c"}]': finished 2024-04-07T16:22:00.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:59 smithi144 bash[66654]: audit 2024-04-07T16:21:59.834417+0000 mon.a (mon.0) 345 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:22:00.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:59 smithi144 bash[66654]: audit 2024-04-07T16:21:59.843321+0000 mon.a (mon.0) 346 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:22:00.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:21:59 smithi144 bash[66654]: audit 2024-04-07T16:21:59.849971+0000 mon.a (mon.0) 347 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-04-07T16:22:00.793 INFO:tasks.rados.rados.0.smithi100.stdout:1585: done (2 left) 2024-04-07T16:22:00.793 INFO:tasks.rados.rados.0.smithi100.stdout:1586: done (1 left) 2024-04-07T16:22:00.793 INFO:tasks.rados.rados.0.smithi100.stdout:1588: done (0 left) 2024-04-07T16:22:00.793 INFO:tasks.rados.rados.0.smithi100.stdout:1589: read oid 28 snap -1 2024-04-07T16:22:00.793 INFO:tasks.rados.rados.0.smithi100.stdout:1589: expect (ObjNum 461 snap 140 seq_num 461) 2024-04-07T16:22:00.793 INFO:tasks.rados.rados.0.smithi100.stdout:1590: read oid 11 snap -1 2024-04-07T16:22:00.794 INFO:tasks.rados.rados.0.smithi100.stdout:1590: expect (ObjNum 0 snap 0 seq_num 0) 2024-04-07T16:22:00.794 INFO:tasks.rados.rados.0.smithi100.stdout:1591: read oid 14 snap -1 2024-04-07T16:22:00.794 INFO:tasks.rados.rados.0.smithi100.stdout:1591: expect deleted 2024-04-07T16:22:00.794 INFO:tasks.rados.rados.0.smithi100.stdout:1592: copy_from oid 50 from oid 12 current snap is 148 2024-04-07T16:22:00.794 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 45 current snap is 148 2024-04-07T16:22:00.794 INFO:tasks.rados.rados.0.smithi100.stdout:1593: seq_num 491 ranges {2179072=630784,2809856=401408} 2024-04-07T16:22:00.818 INFO:tasks.rados.rados.0.smithi100.stdout:1593: writing smithi10066867-45 from 2179072 to 2809856 tid 1 2024-04-07T16:22:00.823 INFO:tasks.rados.rados.0.smithi100.stdout:1593: writing smithi10066867-45 from 2809856 to 3211264 tid 2 2024-04-07T16:22:00.823 INFO:tasks.rados.rados.0.smithi100.stdout:1594: read oid 2 snap 147 2024-04-07T16:22:00.823 INFO:tasks.rados.rados.0.smithi100.stdout:1594: expect (ObjNum 422 snap 127 seq_num 422) 2024-04-07T16:22:00.823 INFO:tasks.rados.rados.0.smithi100.stdout:1595: rollback oid 37 current snap is 148 2024-04-07T16:22:00.823 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 37 to 147 2024-04-07T16:22:00.823 INFO:tasks.rados.rados.0.smithi100.stdout:1590: expect (ObjNum 0 snap 0 seq_num 0) 2024-04-07T16:22:00.823 INFO:tasks.rados.rados.0.smithi100.stdout:1589: expect (ObjNum 461 snap 140 seq_num 461) 2024-04-07T16:22:00.952 INFO:tasks.rados.rados.0.smithi100.stdout:1592: finishing copy_from racing read to smithi10066867-50 2024-04-07T16:22:00.952 INFO:tasks.rados.rados.0.smithi100.stdout:1595: finishing rollback tid 0 to smithi10066867-37 2024-04-07T16:22:00.953 INFO:tasks.rados.rados.0.smithi100.stdout:1592: finishing copy_from to smithi10066867-50 2024-04-07T16:22:00.953 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 50 v 786 (ObjNum 432 snap 130 seq_num 432) dirty exists 2024-04-07T16:22:00.953 INFO:tasks.rados.rados.0.smithi100.stdout:1595: finishing rollback tid 1 to smithi10066867-37 2024-04-07T16:22:00.953 INFO:tasks.rados.rados.0.smithi100.stdout:1595: finishing rollback tid 2 to smithi10066867-37 2024-04-07T16:22:00.953 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 37 v 613 (ObjNum 81 snap 0 seq_num 4240385169) dirty exists 2024-04-07T16:22:00.953 INFO:tasks.rados.rados.0.smithi100.stdout:1593: finishing write tid 1 to smithi10066867-45 2024-04-07T16:22:00.953 INFO:tasks.rados.rados.0.smithi100.stdout:1593: finishing write tid 2 to smithi10066867-45 2024-04-07T16:22:00.953 INFO:tasks.rados.rados.0.smithi100.stdout:1589: done (6 left) 2024-04-07T16:22:00.953 INFO:tasks.rados.rados.0.smithi100.stdout:1590: done (5 left) 2024-04-07T16:22:00.954 INFO:tasks.rados.rados.0.smithi100.stdout:1591: done (4 left) 2024-04-07T16:22:00.954 INFO:tasks.rados.rados.0.smithi100.stdout:1592: done (3 left) 2024-04-07T16:22:00.954 INFO:tasks.rados.rados.0.smithi100.stdout:1595: done (2 left) 2024-04-07T16:22:00.954 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 36 current snap is 148 2024-04-07T16:22:00.954 INFO:tasks.rados.rados.0.smithi100.stdout:1593: finishing write tid 3 to smithi10066867-45 2024-04-07T16:22:00.954 INFO:tasks.rados.rados.0.smithi100.stdout:1596: seq_num 492 ranges {983040=647168,1630208=401408} 2024-04-07T16:22:00.967 INFO:tasks.rados.rados.0.smithi100.stdout:1596: writing smithi10066867-36 from 983040 to 1630208 tid 1 2024-04-07T16:22:00.971 INFO:tasks.rados.rados.0.smithi100.stdout:1596: writing smithi10066867-36 from 1630208 to 2031616 tid 2 2024-04-07T16:22:00.971 INFO:tasks.rados.rados.0.smithi100.stdout:1597: read oid 37 snap 147 2024-04-07T16:22:00.971 INFO:tasks.rados.rados.0.smithi100.stdout:1597: expect (ObjNum -1592773760 snap 21978 seq_num 2702021040) 2024-04-07T16:22:00.971 INFO:tasks.rados.rados.0.smithi100.stdout:1598: copy_from oid 4 from oid 32 current snap is 148 2024-04-07T16:22:00.971 INFO:tasks.rados.rados.0.smithi100.stdout:1593: finishing write tid 4 to smithi10066867-45 2024-04-07T16:22:00.971 INFO:tasks.rados.rados.0.smithi100.stdout:1593: oid 45 updating version 0 to 788 2024-04-07T16:22:00.972 INFO:tasks.rados.rados.0.smithi100.stdout:1593: oid 45 version 788 is already newer than 787 2024-04-07T16:22:00.972 INFO:tasks.rados.rados.0.smithi100.stdout:1593: oid 45 updating version 788 to 789 2024-04-07T16:22:00.972 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 45 v 789 (ObjNum 491 snap 148 seq_num 491) dirty exists 2024-04-07T16:22:00.972 INFO:tasks.rados.rados.0.smithi100.stdout:1593: left oid 45 (ObjNum 491 snap 148 seq_num 491) 2024-04-07T16:22:00.972 INFO:tasks.rados.rados.0.smithi100.stdout:1594: expect (ObjNum 422 snap 127 seq_num 422) 2024-04-07T16:22:01.033 INFO:tasks.rados.rados.0.smithi100.stdout:1597: expect (ObjNum -1592773760 snap 21978 seq_num 2702021040) 2024-04-07T16:22:01.035 INFO:tasks.rados.rados.0.smithi100.stdout:1596: finishing write tid 1 to smithi10066867-36 2024-04-07T16:22:01.035 INFO:tasks.rados.rados.0.smithi100.stdout:1596: finishing write tid 2 to smithi10066867-36 2024-04-07T16:22:01.035 INFO:tasks.rados.rados.0.smithi100.stdout:1596: finishing write tid 3 to smithi10066867-36 2024-04-07T16:22:01.035 INFO:tasks.rados.rados.0.smithi100.stdout:1596: finishing write tid 4 to smithi10066867-36 2024-04-07T16:22:01.035 INFO:tasks.rados.rados.0.smithi100.stdout:1596: oid 36 updating version 0 to 706 2024-04-07T16:22:01.035 INFO:tasks.rados.rados.0.smithi100.stdout:1596: oid 36 version 706 is already newer than 705 2024-04-07T16:22:01.035 INFO:tasks.rados.rados.0.smithi100.stdout:1596: oid 36 updating version 706 to 707 2024-04-07T16:22:01.035 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 36 v 707 (ObjNum 492 snap 148 seq_num 492) dirty exists 2024-04-07T16:22:01.036 INFO:tasks.rados.rados.0.smithi100.stdout:1596: left oid 36 (ObjNum 492 snap 148 seq_num 492) 2024-04-07T16:22:01.036 INFO:tasks.rados.rados.0.smithi100.stdout:1598: finishing copy_from racing read to smithi10066867-4 2024-04-07T16:22:01.036 INFO:tasks.rados.rados.0.smithi100.stdout:1598: finishing copy_from to smithi10066867-4 2024-04-07T16:22:01.036 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 4 v 760 (ObjNum 487 snap 147 seq_num 487) dirty exists 2024-04-07T16:22:01.036 INFO:tasks.rados.rados.0.smithi100.stdout:1593: done (4 left) 2024-04-07T16:22:01.036 INFO:tasks.rados.rados.0.smithi100.stdout:1594: done (3 left) 2024-04-07T16:22:01.036 INFO:tasks.rados.rados.0.smithi100.stdout:1596: done (2 left) 2024-04-07T16:22:01.036 INFO:tasks.rados.rados.0.smithi100.stdout:1597: done (1 left) 2024-04-07T16:22:01.036 INFO:tasks.rados.rados.0.smithi100.stdout:1598: done (0 left) 2024-04-07T16:22:01.036 INFO:tasks.rados.rados.0.smithi100.stdout:1599: copy_from oid 23 from oid 32 current snap is 148 2024-04-07T16:22:01.036 INFO:tasks.rados.rados.0.smithi100.stdout:1600: delete oid 16 current snap is 148 2024-04-07T16:22:01.039 INFO:tasks.rados.rados.0.smithi100.stdout:1600: done (1 left) 2024-04-07T16:22:01.040 INFO:tasks.rados.rados.0.smithi100.stdout:1601: read oid 29 snap -1 2024-04-07T16:22:01.040 INFO:tasks.rados.rados.0.smithi100.stdout:1601: expect (ObjNum 412 snap 125 seq_num 412) 2024-04-07T16:22:01.040 INFO:tasks.rados.rados.0.smithi100.stdout:1602: setattr oid 31 current snap is 148 2024-04-07T16:22:01.042 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 44 current snap is 148 2024-04-07T16:22:01.042 INFO:tasks.rados.rados.0.smithi100.stdout:1603: seq_num 494 ranges {3563520=679936,4243456=401408} 2024-04-07T16:22:01.103 INFO:tasks.rados.rados.0.smithi100.stdout:1603: writing smithi10066867-44 from 3563520 to 4243456 tid 1 2024-04-07T16:22:01.106 INFO:tasks.rados.rados.0.smithi100.stdout:1603: writing smithi10066867-44 from 4243456 to 4644864 tid 2 2024-04-07T16:22:01.107 INFO:tasks.rados.rados.0.smithi100.stdout:1604: copy_from oid 34 from oid 38 current snap is 148 2024-04-07T16:22:01.107 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 31 v 981 (ObjNum 423 snap 127 seq_num 423) dirty exists 2024-04-07T16:22:01.107 INFO:tasks.rados.rados.0.smithi100.stdout:1599: finishing copy_from racing read to smithi10066867-23 2024-04-07T16:22:01.107 INFO:tasks.rados.rados.0.smithi100.stdout:1601: expect (ObjNum 412 snap 125 seq_num 412) 2024-04-07T16:22:01.205 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:00 smithi100 bash[76202]: cephadm 2024-04-07T16:21:59.783167+0000 mgr.y (mgr.25175) 109 : cephadm [INF] Upgrade: Setting container_image for all mon 2024-04-07T16:22:01.205 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:00 smithi100 bash[76202]: cephadm 2024-04-07T16:21:59.835991+0000 mgr.y (mgr.25175) 110 : cephadm [INF] Upgrade: Setting container_image for all crash 2024-04-07T16:22:01.206 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:00 smithi100 bash[76202]: audit 2024-04-07T16:21:59.850293+0000 mgr.y (mgr.25175) 111 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-04-07T16:22:01.206 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:00 smithi100 bash[76202]: cephadm 2024-04-07T16:21:59.852508+0000 mgr.y (mgr.25175) 112 : cephadm [INF] Upgrade: osd.0 is safe to restart 2024-04-07T16:22:01.206 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:00 smithi100 bash[76202]: cluster 2024-04-07T16:22:00.807200+0000 mon.a (mon.0) 348 : cluster [DBG] osdmap e371: 8 total, 8 up, 8 in 2024-04-07T16:22:01.206 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:00 smithi100 bash[74942]: cephadm 2024-04-07T16:21:59.783167+0000 mgr.y (mgr.25175) 109 : cephadm [INF] Upgrade: Setting container_image for all mon 2024-04-07T16:22:01.206 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:00 smithi100 bash[74942]: cephadm 2024-04-07T16:21:59.835991+0000 mgr.y (mgr.25175) 110 : cephadm [INF] Upgrade: Setting container_image for all crash 2024-04-07T16:22:01.206 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:00 smithi100 bash[74942]: audit 2024-04-07T16:21:59.850293+0000 mgr.y (mgr.25175) 111 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-04-07T16:22:01.206 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:00 smithi100 bash[74942]: cephadm 2024-04-07T16:21:59.852508+0000 mgr.y (mgr.25175) 112 : cephadm [INF] Upgrade: osd.0 is safe to restart 2024-04-07T16:22:01.206 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:00 smithi100 bash[74942]: cluster 2024-04-07T16:22:00.807200+0000 mon.a (mon.0) 348 : cluster [DBG] osdmap e371: 8 total, 8 up, 8 in 2024-04-07T16:22:01.244 INFO:tasks.rados.rados.0.smithi100.stdout:1599: finishing copy_from to smithi10066867-23 2024-04-07T16:22:01.244 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 23 v 760 (ObjNum 487 snap 147 seq_num 487) dirty exists 2024-04-07T16:22:01.244 INFO:tasks.rados.rados.0.smithi100.stdout:1603: finishing write tid 1 to smithi10066867-44 2024-04-07T16:22:01.244 INFO:tasks.rados.rados.0.smithi100.stdout:1603: finishing write tid 2 to smithi10066867-44 2024-04-07T16:22:01.244 INFO:tasks.rados.rados.0.smithi100.stdout:1599: done (4 left) 2024-04-07T16:22:01.244 INFO:tasks.rados.rados.0.smithi100.stdout:1601: done (3 left) 2024-04-07T16:22:01.244 INFO:tasks.rados.rados.0.smithi100.stdout:1602: done (2 left) 2024-04-07T16:22:01.244 INFO:tasks.rados.rados.0.smithi100.stdout:1605: read oid 25 snap 147 2024-04-07T16:22:01.244 INFO:tasks.rados.rados.0.smithi100.stdout:1605: expect (ObjNum -1594754864 snap 21978 seq_num 2701953264) 2024-04-07T16:22:01.244 INFO:tasks.rados.rados.0.smithi100.stdout:1603: finishing write tid 3 to smithi10066867-44 2024-04-07T16:22:01.244 INFO:tasks.rados.rados.0.smithi100.stdout:1603: finishing write tid 4 to smithi10066867-44 2024-04-07T16:22:01.244 INFO:tasks.rados.rados.0.smithi100.stdout:1603: oid 44 updating version 0 to 784 2024-04-07T16:22:01.244 INFO:tasks.rados.rados.0.smithi100.stdout:1603: oid 44 updating version 784 to 786 2024-04-07T16:22:01.245 INFO:tasks.rados.rados.0.smithi100.stdout:1603: oid 44 version 786 is already newer than 785 2024-04-07T16:22:01.245 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 44 v 786 (ObjNum 494 snap 148 seq_num 494) dirty exists 2024-04-07T16:22:01.245 INFO:tasks.rados.rados.0.smithi100.stdout:1603: left oid 44 (ObjNum 494 snap 148 seq_num 494) 2024-04-07T16:22:01.245 INFO:tasks.rados.rados.0.smithi100.stdout:1604: finishing copy_from racing read to smithi10066867-34 2024-04-07T16:22:01.245 INFO:tasks.rados.rados.0.smithi100.stdout:1604: finishing copy_from to smithi10066867-34 2024-04-07T16:22:01.245 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 34 v 552 (ObjNum 483 snap 144 seq_num 483) dirty exists 2024-04-07T16:22:01.245 INFO:tasks.rados.rados.0.smithi100.stdout:1606: read oid 29 snap -1 2024-04-07T16:22:01.245 INFO:tasks.rados.rados.0.smithi100.stdout:1606: expect (ObjNum 412 snap 125 seq_num 412) 2024-04-07T16:22:01.245 INFO:tasks.rados.rados.0.smithi100.stdout:1603: done (3 left) 2024-04-07T16:22:01.245 INFO:tasks.rados.rados.0.smithi100.stdout:1604: done (2 left) 2024-04-07T16:22:01.245 INFO:tasks.rados.rados.0.smithi100.stdout:1607: copy_from oid 27 from oid 12 current snap is 148 2024-04-07T16:22:01.245 INFO:tasks.rados.rados.0.smithi100.stdout:1608: read oid 11 snap -1 2024-04-07T16:22:01.245 INFO:tasks.rados.rados.0.smithi100.stdout:1608: expect (ObjNum 0 snap 0 seq_num 0) 2024-04-07T16:22:01.245 INFO:tasks.rados.rados.0.smithi100.stdout:1609: delete oid 14 current snap is 148 2024-04-07T16:22:01.246 INFO:tasks.rados.rados.0.smithi100.stdout:1605: expect (ObjNum -1594754864 snap 21978 seq_num 2701953264) 2024-04-07T16:22:01.246 INFO:tasks.rados.rados.0.smithi100.stdout:1605: done (4 left) 2024-04-07T16:22:01.246 INFO:tasks.rados.rados.0.smithi100.stdout:1609: done (3 left) 2024-04-07T16:22:01.246 INFO:tasks.rados.rados.0.smithi100.stdout:1610: snap_create 2024-04-07T16:22:01.246 INFO:tasks.rados.rados.0.smithi100.stdout:1608: expect (ObjNum 0 snap 0 seq_num 0) 2024-04-07T16:22:01.254 INFO:tasks.rados.rados.0.smithi100.stdout:1607: finishing copy_from racing read to smithi10066867-27 2024-04-07T16:22:01.256 INFO:tasks.rados.rados.0.smithi100.stdout:1607: finishing copy_from to smithi10066867-27 2024-04-07T16:22:01.256 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 27 v 786 (ObjNum 432 snap 130 seq_num 432) dirty exists 2024-04-07T16:22:01.257 INFO:tasks.rados.rados.0.smithi100.stdout:1606: expect (ObjNum 412 snap 125 seq_num 412) 2024-04-07T16:22:01.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:00 smithi144 bash[66654]: cephadm 2024-04-07T16:21:59.783167+0000 mgr.y (mgr.25175) 109 : cephadm [INF] Upgrade: Setting container_image for all mon 2024-04-07T16:22:01.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:00 smithi144 bash[66654]: cephadm 2024-04-07T16:21:59.835991+0000 mgr.y (mgr.25175) 110 : cephadm [INF] Upgrade: Setting container_image for all crash 2024-04-07T16:22:01.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:00 smithi144 bash[66654]: audit 2024-04-07T16:21:59.850293+0000 mgr.y (mgr.25175) 111 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-04-07T16:22:01.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:00 smithi144 bash[66654]: cephadm 2024-04-07T16:21:59.852508+0000 mgr.y (mgr.25175) 112 : cephadm [INF] Upgrade: osd.0 is safe to restart 2024-04-07T16:22:01.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:00 smithi144 bash[66654]: cluster 2024-04-07T16:22:00.807200+0000 mon.a (mon.0) 348 : cluster [DBG] osdmap e371: 8 total, 8 up, 8 in 2024-04-07T16:22:01.775 INFO:tasks.rados.rados.0.smithi100.stdout:1606: done (3 left) 2024-04-07T16:22:01.775 INFO:tasks.rados.rados.0.smithi100.stdout:1607: done (2 left) 2024-04-07T16:22:01.775 INFO:tasks.rados.rados.0.smithi100.stdout:1608: done (1 left) 2024-04-07T16:22:01.775 INFO:tasks.rados.rados.0.smithi100.stdout:1610: done (0 left) 2024-04-07T16:22:01.775 INFO:tasks.rados.rados.0.smithi100.stdout:1611: read oid 13 snap -1 2024-04-07T16:22:01.775 INFO:tasks.rados.rados.0.smithi100.stdout:1611: expect (ObjNum 475 snap 143 seq_num 475) 2024-04-07T16:22:01.775 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 32 current snap is 149 2024-04-07T16:22:01.775 INFO:tasks.rados.rados.0.smithi100.stdout:1612: seq_num 496 ranges {2990080=712704,3702784=401408} 2024-04-07T16:22:01.827 INFO:tasks.rados.rados.0.smithi100.stdout:1612: writing smithi10066867-32 from 2990080 to 3702784 tid 1 2024-04-07T16:22:01.831 INFO:tasks.rados.rados.0.smithi100.stdout:1612: writing smithi10066867-32 from 3702784 to 4104192 tid 2 2024-04-07T16:22:01.832 INFO:tasks.rados.rados.0.smithi100.stdout:1613: copy_from oid 3 from oid 14 current snap is 149 2024-04-07T16:22:01.832 INFO:tasks.rados.rados.0.smithi100.stdout:1611: expect (ObjNum 475 snap 143 seq_num 475) 2024-04-07T16:22:01.852 INFO:tasks.rados.rados.0.smithi100.stdout:1613: finishing copy_from to smithi10066867-3 2024-04-07T16:22:01.853 INFO:tasks.rados.rados.0.smithi100.stdout:1613: got expected ENOENT (src dne) 2024-04-07T16:22:01.853 INFO:tasks.rados.rados.0.smithi100.stdout:1613: finishing copy_from racing read to smithi10066867-3 2024-04-07T16:22:01.853 INFO:tasks.rados.rados.0.smithi100.stdout:1612: finishing write tid 1 to smithi10066867-32 2024-04-07T16:22:01.853 INFO:tasks.rados.rados.0.smithi100.stdout:1612: finishing write tid 2 to smithi10066867-32 2024-04-07T16:22:01.853 INFO:tasks.rados.rados.0.smithi100.stdout:1612: finishing write tid 3 to smithi10066867-32 2024-04-07T16:22:01.853 INFO:tasks.rados.rados.0.smithi100.stdout:1612: finishing write tid 4 to smithi10066867-32 2024-04-07T16:22:01.853 INFO:tasks.rados.rados.0.smithi100.stdout:1612: oid 32 updating version 0 to 761 2024-04-07T16:22:01.853 INFO:tasks.rados.rados.0.smithi100.stdout:1612: oid 32 updating version 761 to 763 2024-04-07T16:22:01.853 INFO:tasks.rados.rados.0.smithi100.stdout:1612: oid 32 version 763 is already newer than 762 2024-04-07T16:22:01.853 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 32 v 763 (ObjNum 496 snap 149 seq_num 496) dirty exists 2024-04-07T16:22:01.853 INFO:tasks.rados.rados.0.smithi100.stdout:1612: left oid 32 (ObjNum 496 snap 149 seq_num 496) 2024-04-07T16:22:01.853 INFO:tasks.rados.rados.0.smithi100.stdout:1611: done (2 left) 2024-04-07T16:22:01.853 INFO:tasks.rados.rados.0.smithi100.stdout:1612: done (1 left) 2024-04-07T16:22:01.853 INFO:tasks.rados.rados.0.smithi100.stdout:1613: done (0 left) 2024-04-07T16:22:01.853 INFO:tasks.rados.rados.0.smithi100.stdout:1614: snap_create 2024-04-07T16:22:02.304 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:01 smithi144 bash[66654]: cluster 2024-04-07T16:22:00.639160+0000 mgr.y (mgr.25175) 113 : cluster [DBG] pgmap v145: 121 pgs: 121 active+clean; 121 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 12 MiB/s rd, 3.7 MiB/s wr, 36 op/s 2024-04-07T16:22:02.304 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:01 smithi144 bash[66654]: audit 2024-04-07T16:22:01.704590+0000 mon.a (mon.0) 349 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:22:02.304 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:01 smithi144 bash[66654]: cluster 2024-04-07T16:22:01.791076+0000 mon.a (mon.0) 350 : cluster [DBG] osdmap e372: 8 total, 8 up, 8 in 2024-04-07T16:22:02.435 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:01 smithi100 bash[76202]: cluster 2024-04-07T16:22:00.639160+0000 mgr.y (mgr.25175) 113 : cluster [DBG] pgmap v145: 121 pgs: 121 active+clean; 121 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 12 MiB/s rd, 3.7 MiB/s wr, 36 op/s 2024-04-07T16:22:02.435 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:01 smithi100 bash[76202]: audit 2024-04-07T16:22:01.704590+0000 mon.a (mon.0) 349 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:22:02.435 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:01 smithi100 bash[76202]: cluster 2024-04-07T16:22:01.791076+0000 mon.a (mon.0) 350 : cluster [DBG] osdmap e372: 8 total, 8 up, 8 in 2024-04-07T16:22:02.435 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:01 smithi100 bash[74942]: cluster 2024-04-07T16:22:00.639160+0000 mgr.y (mgr.25175) 113 : cluster [DBG] pgmap v145: 121 pgs: 121 active+clean; 121 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 12 MiB/s rd, 3.7 MiB/s wr, 36 op/s 2024-04-07T16:22:02.435 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:01 smithi100 bash[74942]: audit 2024-04-07T16:22:01.704590+0000 mon.a (mon.0) 349 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:22:02.435 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:01 smithi100 bash[74942]: cluster 2024-04-07T16:22:01.791076+0000 mon.a (mon.0) 350 : cluster [DBG] osdmap e372: 8 total, 8 up, 8 in 2024-04-07T16:22:02.606 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:22:02 smithi144 bash[59794]: ts=2024-04-07T16:22:02.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:22:02.772 INFO:tasks.rados.rados.0.smithi100.stdout:1614: done (0 left) 2024-04-07T16:22:02.772 INFO:tasks.rados.rados.0.smithi100.stdout:1615: delete oid 33 current snap is 150 2024-04-07T16:22:02.779 INFO:tasks.rados.rados.0.smithi100.stdout:1615: done (0 left) 2024-04-07T16:22:02.779 INFO:tasks.rados.rados.0.smithi100.stdout:1616: read oid 12 snap -1 2024-04-07T16:22:02.779 INFO:tasks.rados.rados.0.smithi100.stdout:1616: expect (ObjNum 432 snap 130 seq_num 432) 2024-04-07T16:22:02.779 INFO:tasks.rados.rados.0.smithi100.stdout:1617: read oid 50 snap 147 2024-04-07T16:22:02.779 INFO:tasks.rados.rados.0.smithi100.stdout:1617: expect (ObjNum 416 snap 126 seq_num 416) 2024-04-07T16:22:02.779 INFO:tasks.rados.rados.0.smithi100.stdout:1618: snap_remove snap 148 2024-04-07T16:22:03.334 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:03 smithi100 bash[74942]: cephadm 2024-04-07T16:22:02.072657+0000 mgr.y (mgr.25175) 114 : cephadm [INF] Upgrade: Updating osd.0 2024-04-07T16:22:03.334 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:03 smithi100 bash[74942]: audit 2024-04-07T16:22:02.081174+0000 mon.a (mon.0) 351 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:22:03.334 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:03 smithi100 bash[74942]: audit 2024-04-07T16:22:02.085009+0000 mon.a (mon.0) 352 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-07T16:22:03.334 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:03 smithi100 bash[74942]: audit 2024-04-07T16:22:02.086409+0000 mon.a (mon.0) 353 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:22:03.334 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:03 smithi100 bash[74942]: cephadm 2024-04-07T16:22:02.091543+0000 mgr.y (mgr.25175) 115 : cephadm [INF] Deploying daemon osd.0 on smithi100 2024-04-07T16:22:03.335 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:03 smithi100 bash[74942]: cluster 2024-04-07T16:22:02.788253+0000 mon.a (mon.0) 354 : cluster [DBG] osdmap e373: 8 total, 8 up, 8 in 2024-04-07T16:22:03.335 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:03 smithi100 bash[76202]: cephadm 2024-04-07T16:22:02.072657+0000 mgr.y (mgr.25175) 114 : cephadm [INF] Upgrade: Updating osd.0 2024-04-07T16:22:03.335 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:03 smithi100 bash[76202]: audit 2024-04-07T16:22:02.081174+0000 mon.a (mon.0) 351 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:22:03.335 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:03 smithi100 bash[76202]: audit 2024-04-07T16:22:02.085009+0000 mon.a (mon.0) 352 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-07T16:22:03.335 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:03 smithi100 bash[76202]: audit 2024-04-07T16:22:02.086409+0000 mon.a (mon.0) 353 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:22:03.335 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:03 smithi100 bash[76202]: cephadm 2024-04-07T16:22:02.091543+0000 mgr.y (mgr.25175) 115 : cephadm [INF] Deploying daemon osd.0 on smithi100 2024-04-07T16:22:03.335 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:03 smithi100 bash[76202]: cluster 2024-04-07T16:22:02.788253+0000 mon.a (mon.0) 354 : cluster [DBG] osdmap e373: 8 total, 8 up, 8 in 2024-04-07T16:22:03.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:03 smithi144 bash[66654]: cephadm 2024-04-07T16:22:02.072657+0000 mgr.y (mgr.25175) 114 : cephadm [INF] Upgrade: Updating osd.0 2024-04-07T16:22:03.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:03 smithi144 bash[66654]: audit 2024-04-07T16:22:02.081174+0000 mon.a (mon.0) 351 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:22:03.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:03 smithi144 bash[66654]: audit 2024-04-07T16:22:02.085009+0000 mon.a (mon.0) 352 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-07T16:22:03.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:03 smithi144 bash[66654]: audit 2024-04-07T16:22:02.086409+0000 mon.a (mon.0) 353 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:22:03.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:03 smithi144 bash[66654]: cephadm 2024-04-07T16:22:02.091543+0000 mgr.y (mgr.25175) 115 : cephadm [INF] Deploying daemon osd.0 on smithi100 2024-04-07T16:22:03.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:03 smithi144 bash[66654]: cluster 2024-04-07T16:22:02.788253+0000 mon.a (mon.0) 354 : cluster [DBG] osdmap e373: 8 total, 8 up, 8 in 2024-04-07T16:22:03.781 INFO:tasks.rados.rados.0.smithi100.stdout:1618: done (2 left) 2024-04-07T16:22:03.782 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 22 current snap is 150 2024-04-07T16:22:03.782 INFO:tasks.rados.rados.0.smithi100.stdout:1619: seq_num 498 ranges {1720320=745472,2465792=401408} 2024-04-07T16:22:03.797 INFO:tasks.rados.rados.0.smithi100.stdout:1619: writing smithi10066867-22 from 1720320 to 2465792 tid 1 2024-04-07T16:22:03.800 INFO:tasks.rados.rados.0.smithi100.stdout:1619: writing smithi10066867-22 from 2465792 to 2867200 tid 2 2024-04-07T16:22:03.800 INFO:tasks.rados.rados.0.smithi100.stdout:1620: rollback oid 3 current snap is 150 2024-04-07T16:22:03.800 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 3 to 149 2024-04-07T16:22:03.800 INFO:tasks.rados.rados.0.smithi100.stdout:1616: expect (ObjNum 432 snap 130 seq_num 432) 2024-04-07T16:22:03.856 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:22:03 smithi144 bash[59794]: ts=2024-04-07T16:22:03.508Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=nodes name=CephNodeDiskspaceWarning index=4 msg="Evaluating rule failed" rule="alert: CephNodeDiskspaceWarning\nexpr: predict_linear(node_filesystem_free_bytes{device=~\"/.*\"}[2d], 3600 * 24 * 5)\n * on (instance) group_left (nodename) node_uname_info < 0\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.8.4\n severity: warning\n type: ceph_default\nannotations:\n description: Mountpoint {{ $labels.mountpoint }} on {{ $labels.nodename }} will\n be full in less than 5 days based on the 48 hour trailing fill rate.\n summary: Host filesystem free space is getting low\n" err="found duplicate series for the match group {instance=\"smithi100\"} on the right hand-side of the operation: [{__name__=\"node_uname_info\", domainname=\"(none)\", instance=\"smithi100\", job=\"node\", machine=\"x86_64\", nodename=\"smithi100\", release=\"5.15.0-101-generic\", sysname=\"Linux\", version=\"#111-Ubuntu SMP Tue Mar 5 20:16:58 UTC 2024\"}, {__name__=\"node_uname_info\", cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", domainname=\"(none)\", instance=\"smithi100\", job=\"node\", machine=\"x86_64\", nodename=\"smithi100\", release=\"5.15.0-101-generic\", sysname=\"Linux\", version=\"#111-Ubuntu SMP Tue Mar 5 20:16:58 UTC 2024\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:22:03.923 INFO:tasks.rados.rados.0.smithi100.stdout:1617: expect (ObjNum 416 snap 126 seq_num 416) 2024-04-07T16:22:03.978 INFO:tasks.rados.rados.0.smithi100.stdout:1620: finishing rollback tid 0 to smithi10066867-3 2024-04-07T16:22:03.978 INFO:tasks.rados.rados.0.smithi100.stdout:1620: finishing rollback tid 1 to smithi10066867-3 2024-04-07T16:22:03.978 INFO:tasks.rados.rados.0.smithi100.stdout:1620: finishing rollback tid 2 to smithi10066867-3 2024-04-07T16:22:03.978 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 3 v 555 (ObjNum 477 snap 143 seq_num 477) dirty exists 2024-04-07T16:22:03.978 INFO:tasks.rados.rados.0.smithi100.stdout:1619: finishing write tid 1 to smithi10066867-22 2024-04-07T16:22:03.978 INFO:tasks.rados.rados.0.smithi100.stdout:1619: finishing write tid 2 to smithi10066867-22 2024-04-07T16:22:03.978 INFO:tasks.rados.rados.0.smithi100.stdout:1619: finishing write tid 3 to smithi10066867-22 2024-04-07T16:22:03.978 INFO:tasks.rados.rados.0.smithi100.stdout:1619: finishing write tid 4 to smithi10066867-22 2024-04-07T16:22:03.978 INFO:tasks.rados.rados.0.smithi100.stdout:1619: oid 22 updating version 0 to 548 2024-04-07T16:22:03.978 INFO:tasks.rados.rados.0.smithi100.stdout:1619: oid 22 updating version 548 to 550 2024-04-07T16:22:03.979 INFO:tasks.rados.rados.0.smithi100.stdout:1619: oid 22 version 550 is already newer than 549 2024-04-07T16:22:03.979 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 22 v 550 (ObjNum 498 snap 150 seq_num 498) dirty exists 2024-04-07T16:22:03.979 INFO:tasks.rados.rados.0.smithi100.stdout:1619: left oid 22 (ObjNum 498 snap 150 seq_num 498) 2024-04-07T16:22:03.979 INFO:tasks.rados.rados.0.smithi100.stdout:1616: done (3 left) 2024-04-07T16:22:03.979 INFO:tasks.rados.rados.0.smithi100.stdout:1617: done (2 left) 2024-04-07T16:22:03.979 INFO:tasks.rados.rados.0.smithi100.stdout:1619: done (1 left) 2024-04-07T16:22:03.979 INFO:tasks.rados.rados.0.smithi100.stdout:1620: done (0 left) 2024-04-07T16:22:03.979 INFO:tasks.rados.rados.0.smithi100.stdout:1621: setattr oid 19 current snap is 150 2024-04-07T16:22:03.982 INFO:tasks.rados.rados.0.smithi100.stdout:1622: rollback oid 8 current snap is 150 2024-04-07T16:22:03.982 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 8 to 147 2024-04-07T16:22:03.982 INFO:tasks.rados.rados.0.smithi100.stdout:1623: snap_remove snap 149 2024-04-07T16:22:04.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:04 smithi144 bash[66654]: cluster 2024-04-07T16:22:02.640519+0000 mgr.y (mgr.25175) 116 : cluster [DBG] pgmap v148: 121 pgs: 121 active+clean; 121 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 9.5 MiB/s rd, 4.0 MiB/s wr, 24 op/s 2024-04-07T16:22:04.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:04 smithi144 bash[66654]: cluster 2024-04-07T16:22:03.796662+0000 mon.a (mon.0) 355 : cluster [DBG] osdmap e374: 8 total, 8 up, 8 in 2024-04-07T16:22:04.435 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:04 smithi100 bash[74942]: cluster 2024-04-07T16:22:02.640519+0000 mgr.y (mgr.25175) 116 : cluster [DBG] pgmap v148: 121 pgs: 121 active+clean; 121 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 9.5 MiB/s rd, 4.0 MiB/s wr, 24 op/s 2024-04-07T16:22:04.435 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:04 smithi100 bash[74942]: cluster 2024-04-07T16:22:03.796662+0000 mon.a (mon.0) 355 : cluster [DBG] osdmap e374: 8 total, 8 up, 8 in 2024-04-07T16:22:04.435 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:04 smithi100 bash[76202]: cluster 2024-04-07T16:22:02.640519+0000 mgr.y (mgr.25175) 116 : cluster [DBG] pgmap v148: 121 pgs: 121 active+clean; 121 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 9.5 MiB/s rd, 4.0 MiB/s wr, 24 op/s 2024-04-07T16:22:04.436 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:04 smithi100 bash[76202]: cluster 2024-04-07T16:22:03.796662+0000 mon.a (mon.0) 355 : cluster [DBG] osdmap e374: 8 total, 8 up, 8 in 2024-04-07T16:22:04.780 INFO:tasks.rados.rados.0.smithi100.stdout:1623: done (2 left) 2024-04-07T16:22:04.780 INFO:tasks.rados.rados.0.smithi100.stdout:1624: rollback oid 35 current snap is 150 2024-04-07T16:22:04.780 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 35 to 147 2024-04-07T16:22:04.780 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 19 v 708 (ObjNum 440 snap 134 seq_num 440) dirty exists 2024-04-07T16:22:04.780 INFO:tasks.rados.rados.0.smithi100.stdout:1622: finishing rollback tid 0 to smithi10066867-8 2024-04-07T16:22:04.780 INFO:tasks.rados.rados.0.smithi100.stdout:1622: finishing rollback tid 1 to smithi10066867-8 2024-04-07T16:22:04.780 INFO:tasks.rados.rados.0.smithi100.stdout:1622: finishing rollback tid 2 to smithi10066867-8 2024-04-07T16:22:04.780 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 8 v 701 (ObjNum 440 snap 134 seq_num 440) dirty exists 2024-04-07T16:22:04.780 INFO:tasks.rados.rados.0.smithi100.stdout:1621: done (2 left) 2024-04-07T16:22:04.780 INFO:tasks.rados.rados.0.smithi100.stdout:1622: done (1 left) 2024-04-07T16:22:04.780 INFO:tasks.rados.rados.0.smithi100.stdout:1625: delete oid 30 current snap is 150 2024-04-07T16:22:04.793 INFO:tasks.rados.rados.0.smithi100.stdout:1624: finishing rollback tid 0 to smithi10066867-35 2024-04-07T16:22:04.795 INFO:tasks.rados.rados.0.smithi100.stdout:1624: finishing rollback tid 1 to smithi10066867-35 2024-04-07T16:22:04.796 INFO:tasks.rados.rados.0.smithi100.stdout:1624: finishing rollback tid 2 to smithi10066867-35 2024-04-07T16:22:04.796 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 35 v 984 (ObjNum 65 snap 0 seq_num 1819237472) dirty exists 2024-04-07T16:22:04.798 INFO:tasks.rados.rados.0.smithi100.stdout:1624: done (1 left) 2024-04-07T16:22:04.798 INFO:tasks.rados.rados.0.smithi100.stdout:1625: done (0 left) 2024-04-07T16:22:04.798 INFO:tasks.rados.rados.0.smithi100.stdout:1626: snap_remove snap 147 2024-04-07T16:22:05.792 INFO:tasks.rados.rados.0.smithi100.stdout:1626: done (0 left) 2024-04-07T16:22:05.792 INFO:tasks.rados.rados.0.smithi100.stdout:1627: read oid 12 snap -1 2024-04-07T16:22:05.792 INFO:tasks.rados.rados.0.smithi100.stdout:1627: expect (ObjNum 432 snap 130 seq_num 432) 2024-04-07T16:22:05.794 INFO:tasks.rados.rados.0.smithi100.stdout:1628: read oid 46 snap -1 2024-04-07T16:22:05.794 INFO:tasks.rados.rados.0.smithi100.stdout:1628: expect (ObjNum 337 snap 101 seq_num 337) 2024-04-07T16:22:05.794 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 47 current snap is 150 2024-04-07T16:22:05.794 INFO:tasks.rados.rados.0.smithi100.stdout:1629: seq_num 500 ranges {614400=778240,1392640=401408} 2024-04-07T16:22:05.811 INFO:tasks.rados.rados.0.smithi100.stdout:1629: writing smithi10066867-47 from 614400 to 1392640 tid 1 2024-04-07T16:22:05.814 INFO:tasks.rados.rados.0.smithi100.stdout:1629: writing smithi10066867-47 from 1392640 to 1794048 tid 2 2024-04-07T16:22:05.815 INFO:tasks.rados.rados.0.smithi100.stdout:1630: rollback oid 43 current snap is 150 2024-04-07T16:22:05.815 INFO:tasks.rados.rados.0.smithi100.stdout:1630: done (3 left) 2024-04-07T16:22:05.815 INFO:tasks.rados.rados.0.smithi100.stdout:1631: copy_from oid 25 from oid 28 current snap is 150 2024-04-07T16:22:05.815 INFO:tasks.rados.rados.0.smithi100.stdout:1627: expect (ObjNum 432 snap 130 seq_num 432) 2024-04-07T16:22:05.927 INFO:tasks.rados.rados.0.smithi100.stdout:1629: finishing write tid 1 to smithi10066867-47 2024-04-07T16:22:05.928 INFO:tasks.rados.rados.0.smithi100.stdout:1629: finishing write tid 2 to smithi10066867-47 2024-04-07T16:22:05.928 INFO:tasks.rados.rados.0.smithi100.stdout:1629: finishing write tid 3 to smithi10066867-47 2024-04-07T16:22:05.928 INFO:tasks.rados.rados.0.smithi100.stdout:1629: finishing write tid 4 to smithi10066867-47 2024-04-07T16:22:05.928 INFO:tasks.rados.rados.0.smithi100.stdout:1629: oid 47 updating version 0 to 536 2024-04-07T16:22:05.928 INFO:tasks.rados.rados.0.smithi100.stdout:1629: oid 47 version 536 is already newer than 535 2024-04-07T16:22:05.928 INFO:tasks.rados.rados.0.smithi100.stdout:1629: oid 47 updating version 536 to 537 2024-04-07T16:22:05.928 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 47 v 537 (ObjNum 500 snap 150 seq_num 500) dirty exists 2024-04-07T16:22:05.928 INFO:tasks.rados.rados.0.smithi100.stdout:1629: left oid 47 (ObjNum 500 snap 150 seq_num 500) 2024-04-07T16:22:05.928 INFO:tasks.rados.rados.0.smithi100.stdout:1631: finishing copy_from racing read to smithi10066867-25 2024-04-07T16:22:05.928 INFO:tasks.rados.rados.0.smithi100.stdout:1631: finishing copy_from to smithi10066867-25 2024-04-07T16:22:05.928 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 25 v 757 (ObjNum 461 snap 140 seq_num 461) dirty exists 2024-04-07T16:22:05.928 INFO:tasks.rados.rados.0.smithi100.stdout:1628: expect (ObjNum 337 snap 101 seq_num 337) 2024-04-07T16:22:06.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:05 smithi144 bash[66654]: cluster 2024-04-07T16:22:04.641184+0000 mgr.y (mgr.25175) 117 : cluster [DBG] pgmap v151: 121 pgs: 121 active+clean; 121 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 9.5 MiB/s rd, 4.0 MiB/s wr, 24 op/s 2024-04-07T16:22:06.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:05 smithi144 bash[66654]: cluster 2024-04-07T16:22:04.796556+0000 mon.a (mon.0) 356 : cluster [DBG] osdmap e375: 8 total, 8 up, 8 in 2024-04-07T16:22:06.126 INFO:tasks.rados.rados.0.smithi100.stdout:1627: done (3 left) 2024-04-07T16:22:06.126 INFO:tasks.rados.rados.0.smithi100.stdout:1628: done (2 left) 2024-04-07T16:22:06.126 INFO:tasks.rados.rados.0.smithi100.stdout:1629: done (1 left) 2024-04-07T16:22:06.126 INFO:tasks.rados.rados.0.smithi100.stdout:1631: done (0 left) 2024-04-07T16:22:06.126 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 11 current snap is 150 2024-04-07T16:22:06.126 INFO:tasks.rados.rados.0.smithi100.stdout:1632: seq_num 501 ranges {0=794624,794624=401408} 2024-04-07T16:22:06.138 INFO:tasks.rados.rados.0.smithi100.stdout:1632: writing smithi10066867-11 from 0 to 794624 tid 1 2024-04-07T16:22:06.146 INFO:tasks.rados.rados.0.smithi100.stdout:1632: writing smithi10066867-11 from 794624 to 1196032 tid 2 2024-04-07T16:22:06.146 INFO:tasks.rados.rados.0.smithi100.stdout:1633: copy_from oid 10 from oid 20 current snap is 150 2024-04-07T16:22:06.146 INFO:tasks.rados.rados.0.smithi100.stdout:1632: finishing write tid 1 to smithi10066867-11 2024-04-07T16:22:06.146 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 47 current snap is 150 2024-04-07T16:22:06.146 INFO:tasks.rados.rados.0.smithi100.stdout:1634: seq_num 502 ranges {1794048=409600,2203648=417792,2621440=385024} 2024-04-07T16:22:06.177 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:05 smithi100 bash[76202]: cluster 2024-04-07T16:22:04.641184+0000 mgr.y (mgr.25175) 117 : cluster [DBG] pgmap v151: 121 pgs: 121 active+clean; 121 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 9.5 MiB/s rd, 4.0 MiB/s wr, 24 op/s 2024-04-07T16:22:06.177 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:05 smithi100 bash[76202]: cluster 2024-04-07T16:22:04.796556+0000 mon.a (mon.0) 356 : cluster [DBG] osdmap e375: 8 total, 8 up, 8 in 2024-04-07T16:22:06.177 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:05 smithi100 bash[74942]: cluster 2024-04-07T16:22:04.641184+0000 mgr.y (mgr.25175) 117 : cluster [DBG] pgmap v151: 121 pgs: 121 active+clean; 121 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 9.5 MiB/s rd, 4.0 MiB/s wr, 24 op/s 2024-04-07T16:22:06.177 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:05 smithi100 bash[74942]: cluster 2024-04-07T16:22:04.796556+0000 mon.a (mon.0) 356 : cluster [DBG] osdmap e375: 8 total, 8 up, 8 in 2024-04-07T16:22:06.181 INFO:tasks.rados.rados.0.smithi100.stdout:1634: writing smithi10066867-47 from 1794048 to 2203648 tid 1 2024-04-07T16:22:06.186 INFO:tasks.rados.rados.0.smithi100.stdout:1634: writing smithi10066867-47 from 2203648 to 2621440 tid 2 2024-04-07T16:22:06.191 INFO:tasks.rados.rados.0.smithi100.stdout:1634: writing smithi10066867-47 from 2621440 to 3006464 tid 3 2024-04-07T16:22:06.191 INFO:tasks.rados.rados.0.smithi100.stdout:1635: delete oid 43 current snap is 150 2024-04-07T16:22:06.191 INFO:tasks.rados.rados.0.smithi100.stdout:1632: finishing write tid 2 to smithi10066867-11 2024-04-07T16:22:06.191 INFO:tasks.rados.rados.0.smithi100.stdout:1632: finishing write tid 3 to smithi10066867-11 2024-04-07T16:22:06.191 INFO:tasks.rados.rados.0.smithi100.stdout:1632: finishing write tid 4 to smithi10066867-11 2024-04-07T16:22:06.191 INFO:tasks.rados.rados.0.smithi100.stdout:1632: oid 11 updating version 0 to 1002 2024-04-07T16:22:06.191 INFO:tasks.rados.rados.0.smithi100.stdout:1632: oid 11 updating version 1002 to 1003 2024-04-07T16:22:06.192 INFO:tasks.rados.rados.0.smithi100.stdout:1632: oid 11 updating version 1003 to 1004 2024-04-07T16:22:06.192 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 11 v 1004 (ObjNum 501 snap 150 seq_num 501) dirty exists 2024-04-07T16:22:06.192 INFO:tasks.rados.rados.0.smithi100.stdout:1632: left oid 11 (ObjNum 501 snap 150 seq_num 501) 2024-04-07T16:22:06.192 INFO:tasks.rados.rados.0.smithi100.stdout:1633: finishing copy_from racing read to smithi10066867-10 2024-04-07T16:22:06.192 INFO:tasks.rados.rados.0.smithi100.stdout:1633: finishing copy_from to smithi10066867-10 2024-04-07T16:22:06.192 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 10 v 543 (ObjNum 473 snap 143 seq_num 473) dirty exists 2024-04-07T16:22:06.192 INFO:tasks.rados.rados.0.smithi100.stdout:1634: finishing write tid 1 to smithi10066867-47 2024-04-07T16:22:06.192 INFO:tasks.rados.rados.0.smithi100.stdout:1634: finishing write tid 2 to smithi10066867-47 2024-04-07T16:22:06.194 INFO:tasks.rados.rados.0.smithi100.stdout:1632: done (3 left) 2024-04-07T16:22:06.194 INFO:tasks.rados.rados.0.smithi100.stdout:1633: done (2 left) 2024-04-07T16:22:06.194 INFO:tasks.rados.rados.0.smithi100.stdout:1635: done (1 left) 2024-04-07T16:22:06.194 INFO:tasks.rados.rados.0.smithi100.stdout:1636: delete oid 40 current snap is 150 2024-04-07T16:22:06.197 INFO:tasks.rados.rados.0.smithi100.stdout:1634: finishing write tid 3 to smithi10066867-47 2024-04-07T16:22:06.197 INFO:tasks.rados.rados.0.smithi100.stdout:1636: done (1 left) 2024-04-07T16:22:06.198 INFO:tasks.rados.rados.0.smithi100.stdout:1637: rollback oid 1 current snap is 150 2024-04-07T16:22:06.198 INFO:tasks.rados.rados.0.smithi100.stdout:1637: done (1 left) 2024-04-07T16:22:06.198 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 2 current snap is 150 2024-04-07T16:22:06.198 INFO:tasks.rados.rados.0.smithi100.stdout:1638: seq_num 505 ranges {2277376=466944,2744320=475136,3219456=327680} 2024-04-07T16:22:06.213 INFO:tasks.rados.rados.0.smithi100.stdout:1638: writing smithi10066867-2 from 2277376 to 2744320 tid 1 2024-04-07T16:22:06.217 INFO:tasks.rados.rados.0.smithi100.stdout:1638: writing smithi10066867-2 from 2744320 to 3219456 tid 2 2024-04-07T16:22:06.219 INFO:tasks.rados.rados.0.smithi100.stdout:1638: writing smithi10066867-2 from 3219456 to 3547136 tid 3 2024-04-07T16:22:06.220 INFO:tasks.rados.rados.0.smithi100.stdout:1639: snap_create 2024-04-07T16:22:06.220 INFO:tasks.rados.rados.0.smithi100.stdout:1634: finishing write tid 4 to smithi10066867-47 2024-04-07T16:22:06.220 INFO:tasks.rados.rados.0.smithi100.stdout:1634: finishing write tid 5 to smithi10066867-47 2024-04-07T16:22:06.220 INFO:tasks.rados.rados.0.smithi100.stdout:1634: oid 47 updating version 0 to 540 2024-04-07T16:22:06.220 INFO:tasks.rados.rados.0.smithi100.stdout:1634: oid 47 version 540 is already newer than 539 2024-04-07T16:22:06.220 INFO:tasks.rados.rados.0.smithi100.stdout:1634: oid 47 updating version 540 to 541 2024-04-07T16:22:06.220 INFO:tasks.rados.rados.0.smithi100.stdout:1634: oid 47 version 541 is already newer than 538 2024-04-07T16:22:06.220 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 47 v 541 (ObjNum 502 snap 150 seq_num 502) dirty exists 2024-04-07T16:22:06.220 INFO:tasks.rados.rados.0.smithi100.stdout:1634: left oid 47 (ObjNum 502 snap 150 seq_num 502) 2024-04-07T16:22:06.220 INFO:tasks.rados.rados.0.smithi100.stdout:1638: finishing write tid 1 to smithi10066867-2 2024-04-07T16:22:06.223 INFO:tasks.rados.rados.0.smithi100.stdout:1638: finishing write tid 2 to smithi10066867-2 2024-04-07T16:22:06.224 INFO:tasks.rados.rados.0.smithi100.stdout:1638: finishing write tid 3 to smithi10066867-2 2024-04-07T16:22:06.225 INFO:tasks.rados.rados.0.smithi100.stdout:1638: finishing write tid 4 to smithi10066867-2 2024-04-07T16:22:06.226 INFO:tasks.rados.rados.0.smithi100.stdout:1638: finishing write tid 5 to smithi10066867-2 2024-04-07T16:22:06.226 INFO:tasks.rados.rados.0.smithi100.stdout:1638: oid 2 updating version 0 to 792 2024-04-07T16:22:06.227 INFO:tasks.rados.rados.0.smithi100.stdout:1638: oid 2 version 792 is already newer than 791 2024-04-07T16:22:06.227 INFO:tasks.rados.rados.0.smithi100.stdout:1638: oid 2 updating version 792 to 793 2024-04-07T16:22:06.227 INFO:tasks.rados.rados.0.smithi100.stdout:1638: oid 2 version 793 is already newer than 790 2024-04-07T16:22:06.227 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 2 v 793 (ObjNum 505 snap 150 seq_num 505) dirty exists 2024-04-07T16:22:06.227 INFO:tasks.rados.rados.0.smithi100.stdout:1638: left oid 2 (ObjNum 505 snap 150 seq_num 505) 2024-04-07T16:22:06.435 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:06 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:22:06.435 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:22:06 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:22:06.436 INFO:journalctl@ceph.osd.1.smithi100.stdout:Apr 07 16:22:06 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:22:06.436 INFO:journalctl@ceph.osd.2.smithi100.stdout:Apr 07 16:22:06 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:22:06.436 INFO:journalctl@ceph.osd.3.smithi100.stdout:Apr 07 16:22:06 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:22:06.437 INFO:journalctl@ceph.mds.a.smithi100.stdout:Apr 07 16:22:06 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:22:06.437 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 16:22:06 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:22:06.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:06 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:22:06.438 INFO:journalctl@ceph.osd.0.smithi100.stdout:Apr 07 16:22:06 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:22:06.438 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Apr 07 16:22:06 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:22:06.778 INFO:tasks.rados.rados.0.smithi100.stdout:1634: done (2 left) 2024-04-07T16:22:06.778 INFO:tasks.rados.rados.0.smithi100.stdout:1638: done (1 left) 2024-04-07T16:22:06.778 INFO:tasks.rados.rados.0.smithi100.stdout:1639: done (0 left) 2024-04-07T16:22:06.778 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 19 current snap is 151 2024-04-07T16:22:06.778 INFO:tasks.rados.rados.0.smithi100.stdout:1640: seq_num 506 ranges {4038656=483328,4521984=761856,5283840=40960} 2024-04-07T16:22:06.806 INFO:journalctl@ceph.osd.0.smithi100.stdout:Apr 07 16:22:06 smithi100 systemd[1]: Stopping Ceph osd.0 for 0702e3a8-f4f8-11ee-b649-cb9ed24678a4... 2024-04-07T16:22:06.806 INFO:journalctl@ceph.osd.0.smithi100.stdout:Apr 07 16:22:06 smithi100 bash[34689]: debug 2024-04-07T16:22:06.600+0000 7fd2f6226700 -1 received signal: Terminated from /sbin/docker-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug (PID: 1) UID: 0 2024-04-07T16:22:06.806 INFO:journalctl@ceph.osd.0.smithi100.stdout:Apr 07 16:22:06 smithi100 bash[34689]: debug 2024-04-07T16:22:06.600+0000 7fd2f6226700 -1 osd.0 376 *** Got signal Terminated *** 2024-04-07T16:22:06.807 INFO:journalctl@ceph.osd.0.smithi100.stdout:Apr 07 16:22:06 smithi100 bash[34689]: debug 2024-04-07T16:22:06.600+0000 7fd2f6226700 -1 osd.0 376 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-07T16:22:06.815 INFO:tasks.rados.rados.0.smithi100.stdout:1640: writing smithi10066867-19 from 4038656 to 4521984 tid 1 2024-04-07T16:22:06.822 INFO:tasks.rados.rados.0.smithi100.stdout:1640: writing smithi10066867-19 from 4521984 to 5283840 tid 2 2024-04-07T16:22:06.822 INFO:tasks.rados.rados.0.smithi100.stdout:1640: writing smithi10066867-19 from 5283840 to 5324800 tid 3 2024-04-07T16:22:06.823 INFO:tasks.rados.rados.0.smithi100.stdout:1641: copy_from oid 10 from oid 3 current snap is 151 2024-04-07T16:22:06.823 INFO:tasks.rados.rados.0.smithi100.stdout:1640: finishing write tid 1 to smithi10066867-19 2024-04-07T16:22:06.823 INFO:tasks.rados.rados.0.smithi100.stdout:1642: snap_remove snap 150 2024-04-07T16:22:07.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:06 smithi144 bash[66654]: cluster 2024-04-07T16:22:05.806810+0000 mon.a (mon.0) 357 : cluster [DBG] osdmap e376: 8 total, 8 up, 8 in 2024-04-07T16:22:07.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:06 smithi144 bash[66654]: cluster 2024-04-07T16:22:06.606195+0000 mon.a (mon.0) 358 : cluster [INF] osd.0 marked itself down and dead 2024-04-07T16:22:07.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:06 smithi144 bash[66654]: cluster 2024-04-07T16:22:06.783529+0000 mon.a (mon.0) 359 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN) 2024-04-07T16:22:07.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:06 smithi144 bash[66654]: cluster 2024-04-07T16:22:06.794393+0000 mon.a (mon.0) 360 : cluster [DBG] osdmap e377: 8 total, 7 up, 8 in 2024-04-07T16:22:07.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:06 smithi100 bash[76202]: cluster 2024-04-07T16:22:05.806810+0000 mon.a (mon.0) 357 : cluster [DBG] osdmap e376: 8 total, 8 up, 8 in 2024-04-07T16:22:07.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:06 smithi100 bash[76202]: cluster 2024-04-07T16:22:06.606195+0000 mon.a (mon.0) 358 : cluster [INF] osd.0 marked itself down and dead 2024-04-07T16:22:07.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:06 smithi100 bash[76202]: cluster 2024-04-07T16:22:06.783529+0000 mon.a (mon.0) 359 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN) 2024-04-07T16:22:07.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:06 smithi100 bash[76202]: cluster 2024-04-07T16:22:06.794393+0000 mon.a (mon.0) 360 : cluster [DBG] osdmap e377: 8 total, 7 up, 8 in 2024-04-07T16:22:07.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:06 smithi100 bash[74942]: cluster 2024-04-07T16:22:05.806810+0000 mon.a (mon.0) 357 : cluster [DBG] osdmap e376: 8 total, 8 up, 8 in 2024-04-07T16:22:07.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:06 smithi100 bash[74942]: cluster 2024-04-07T16:22:06.606195+0000 mon.a (mon.0) 358 : cluster [INF] osd.0 marked itself down and dead 2024-04-07T16:22:07.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:06 smithi100 bash[74942]: cluster 2024-04-07T16:22:06.783529+0000 mon.a (mon.0) 359 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN) 2024-04-07T16:22:07.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:06 smithi100 bash[74942]: cluster 2024-04-07T16:22:06.794393+0000 mon.a (mon.0) 360 : cluster [DBG] osdmap e377: 8 total, 7 up, 8 in 2024-04-07T16:22:07.781 INFO:tasks.rados.rados.0.smithi100.stdout:1642: done (2 left) 2024-04-07T16:22:07.781 INFO:tasks.rados.rados.0.smithi100.stdout:1643: delete oid 29 current snap is 151 2024-04-07T16:22:07.781 INFO:tasks.rados.rados.0.smithi100.stdout:1640: finishing write tid 2 to smithi10066867-19 2024-04-07T16:22:07.782 INFO:tasks.rados.rados.0.smithi100.stdout:1640: finishing write tid 3 to smithi10066867-19 2024-04-07T16:22:07.782 INFO:tasks.rados.rados.0.smithi100.stdout:1640: finishing write tid 4 to smithi10066867-19 2024-04-07T16:22:07.782 INFO:tasks.rados.rados.0.smithi100.stdout:1640: finishing write tid 5 to smithi10066867-19 2024-04-07T16:22:07.782 INFO:tasks.rados.rados.0.smithi100.stdout:1640: oid 19 updating version 0 to 710 2024-04-07T16:22:07.782 INFO:tasks.rados.rados.0.smithi100.stdout:1640: oid 19 updating version 710 to 712 2024-04-07T16:22:07.782 INFO:tasks.rados.rados.0.smithi100.stdout:1640: oid 19 version 712 is already newer than 711 2024-04-07T16:22:07.782 INFO:tasks.rados.rados.0.smithi100.stdout:1640: oid 19 version 712 is already newer than 709 2024-04-07T16:22:07.782 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 19 v 712 (ObjNum 506 snap 151 seq_num 506) dirty exists 2024-04-07T16:22:07.782 INFO:tasks.rados.rados.0.smithi100.stdout:1640: left oid 19 (ObjNum 506 snap 151 seq_num 506) 2024-04-07T16:22:07.782 INFO:tasks.rados.rados.0.smithi100.stdout:1641: finishing copy_from racing read to smithi10066867-10 2024-04-07T16:22:07.782 INFO:tasks.rados.rados.0.smithi100.stdout:1641: finishing copy_from to smithi10066867-10 2024-04-07T16:22:07.782 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 10 v 555 (ObjNum 477 snap 143 seq_num 477) dirty exists 2024-04-07T16:22:07.801 INFO:tasks.rados.rados.0.smithi100.stdout:1640: done (2 left) 2024-04-07T16:22:07.801 INFO:tasks.rados.rados.0.smithi100.stdout:1641: done (1 left) 2024-04-07T16:22:07.801 INFO:tasks.rados.rados.0.smithi100.stdout:1643: done (0 left) 2024-04-07T16:22:07.801 INFO:tasks.rados.rados.0.smithi100.stdout:1644: read oid 24 snap -1 2024-04-07T16:22:07.802 INFO:tasks.rados.rados.0.smithi100.stdout:1644: expect (ObjNum 486 snap 147 seq_num 486) 2024-04-07T16:22:07.802 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 17 current snap is 151 2024-04-07T16:22:07.802 INFO:tasks.rados.rados.0.smithi100.stdout:1645: seq_num 508 ranges {0=516096,516096=524288,1040384=278528} 2024-04-07T16:22:07.806 INFO:tasks.rados.rados.0.smithi100.stdout:1645: writing smithi10066867-17 from 0 to 516096 tid 1 2024-04-07T16:22:07.812 INFO:tasks.rados.rados.0.smithi100.stdout:1645: writing smithi10066867-17 from 516096 to 1040384 tid 2 2024-04-07T16:22:07.815 INFO:tasks.rados.rados.0.smithi100.stdout:1645: writing smithi10066867-17 from 1040384 to 1318912 tid 3 2024-04-07T16:22:07.815 INFO:tasks.rados.rados.0.smithi100.stdout:1646: read oid 36 snap -1 2024-04-07T16:22:07.815 INFO:tasks.rados.rados.0.smithi100.stdout:1646: expect (ObjNum 492 snap 148 seq_num 492) 2024-04-07T16:22:07.815 INFO:tasks.rados.rados.0.smithi100.stdout:1647: snap_create 2024-04-07T16:22:07.824 INFO:tasks.rados.rados.0.smithi100.stdout:1644: expect (ObjNum 486 snap 147 seq_num 486) 2024-04-07T16:22:07.862 INFO:tasks.rados.rados.0.smithi100.stdout:1645: finishing write tid 1 to smithi10066867-17 2024-04-07T16:22:07.862 INFO:tasks.rados.rados.0.smithi100.stdout:1645: finishing write tid 2 to smithi10066867-17 2024-04-07T16:22:07.862 INFO:tasks.rados.rados.0.smithi100.stdout:1645: finishing write tid 3 to smithi10066867-17 2024-04-07T16:22:07.862 INFO:tasks.rados.rados.0.smithi100.stdout:1645: finishing write tid 4 to smithi10066867-17 2024-04-07T16:22:07.862 INFO:tasks.rados.rados.0.smithi100.stdout:1646: expect (ObjNum 492 snap 148 seq_num 492) 2024-04-07T16:22:07.937 INFO:tasks.rados.rados.0.smithi100.stdout:1645: finishing write tid 5 to smithi10066867-17 2024-04-07T16:22:07.937 INFO:tasks.rados.rados.0.smithi100.stdout:1645: oid 17 updating version 0 to 715 2024-04-07T16:22:07.937 INFO:tasks.rados.rados.0.smithi100.stdout:1645: oid 17 updating version 715 to 716 2024-04-07T16:22:07.937 INFO:tasks.rados.rados.0.smithi100.stdout:1645: oid 17 version 716 is already newer than 713 2024-04-07T16:22:07.937 INFO:tasks.rados.rados.0.smithi100.stdout:1645: oid 17 version 716 is already newer than 714 2024-04-07T16:22:07.937 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 17 v 716 (ObjNum 508 snap 151 seq_num 508) dirty exists 2024-04-07T16:22:07.937 INFO:tasks.rados.rados.0.smithi100.stdout:1645: left oid 17 (ObjNum 508 snap 151 seq_num 508) 2024-04-07T16:22:08.094 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:07 smithi100 bash[74942]: cluster 2024-04-07T16:22:06.641719+0000 mgr.y (mgr.25175) 118 : cluster [DBG] pgmap v154: 121 pgs: 121 active+clean; 121 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail 2024-04-07T16:22:08.094 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:07 smithi100 bash[74942]: cluster 2024-04-07T16:22:07.795229+0000 mon.a (mon.0) 361 : cluster [DBG] osdmap e378: 8 total, 7 up, 8 in 2024-04-07T16:22:08.095 INFO:journalctl@ceph.osd.0.smithi100.stdout:Apr 07 16:22:07 smithi100 bash[80489]: ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4-osd-0 2024-04-07T16:22:08.095 INFO:journalctl@ceph.osd.0.smithi100.stdout:Apr 07 16:22:08 smithi100 bash[80555]: Error response from daemon: No such container: ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4-osd-0 2024-04-07T16:22:08.095 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:07 smithi100 bash[76202]: cluster 2024-04-07T16:22:06.641719+0000 mgr.y (mgr.25175) 118 : cluster [DBG] pgmap v154: 121 pgs: 121 active+clean; 121 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail 2024-04-07T16:22:08.095 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:07 smithi100 bash[76202]: cluster 2024-04-07T16:22:07.795229+0000 mon.a (mon.0) 361 : cluster [DBG] osdmap e378: 8 total, 7 up, 8 in 2024-04-07T16:22:08.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:07 smithi144 bash[66654]: cluster 2024-04-07T16:22:06.641719+0000 mgr.y (mgr.25175) 118 : cluster [DBG] pgmap v154: 121 pgs: 121 active+clean; 121 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail 2024-04-07T16:22:08.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:07 smithi144 bash[66654]: cluster 2024-04-07T16:22:07.795229+0000 mon.a (mon.0) 361 : cluster [DBG] osdmap e378: 8 total, 7 up, 8 in 2024-04-07T16:22:08.685 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:22:08 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:22:08] "GET /metrics HTTP/1.1" 200 36076 "" "Prometheus/2.43.0" 2024-04-07T16:22:08.829 INFO:tasks.rados.rados.0.smithi100.stdout:1644: done (3 left) 2024-04-07T16:22:08.829 INFO:tasks.rados.rados.0.smithi100.stdout:1645: done (2 left) 2024-04-07T16:22:08.829 INFO:tasks.rados.rados.0.smithi100.stdout:1646: done (1 left) 2024-04-07T16:22:08.829 INFO:tasks.rados.rados.0.smithi100.stdout:1647: done (0 left) 2024-04-07T16:22:08.829 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 44 current snap is 152 2024-04-07T16:22:08.829 INFO:tasks.rados.rados.0.smithi100.stdout:1648: seq_num 509 ranges {4644864=532480,5177344=409600,5586944=393216} 2024-04-07T16:22:08.858 INFO:tasks.rados.rados.0.smithi100.stdout:1648: writing smithi10066867-44 from 4644864 to 5177344 tid 1 2024-04-07T16:22:08.860 INFO:tasks.rados.rados.0.smithi100.stdout:1648: writing smithi10066867-44 from 5177344 to 5586944 tid 2 2024-04-07T16:22:08.863 INFO:tasks.rados.rados.0.smithi100.stdout:1648: writing smithi10066867-44 from 5586944 to 5980160 tid 3 2024-04-07T16:22:08.863 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 48 current snap is 152 2024-04-07T16:22:08.863 INFO:tasks.rados.rados.0.smithi100.stdout:1649: seq_num 510 ranges {2449408=548864,2998272=745472,3743744=57344} 2024-04-07T16:22:08.881 INFO:tasks.rados.rados.0.smithi100.stdout:1649: writing smithi10066867-48 from 2449408 to 2998272 tid 1 2024-04-07T16:22:08.886 INFO:tasks.rados.rados.0.smithi100.stdout:1649: writing smithi10066867-48 from 2998272 to 3743744 tid 2 2024-04-07T16:22:08.887 INFO:tasks.rados.rados.0.smithi100.stdout:1649: writing smithi10066867-48 from 3743744 to 3801088 tid 3 2024-04-07T16:22:08.887 INFO:tasks.rados.rados.0.smithi100.stdout:1650: delete oid 25 current snap is 152 2024-04-07T16:22:08.887 INFO:tasks.rados.rados.0.smithi100.stdout:1648: finishing write tid 1 to smithi10066867-44 2024-04-07T16:22:08.887 INFO:tasks.rados.rados.0.smithi100.stdout:1648: finishing write tid 2 to smithi10066867-44 2024-04-07T16:22:08.887 INFO:tasks.rados.rados.0.smithi100.stdout:1648: finishing write tid 3 to smithi10066867-44 2024-04-07T16:22:08.888 INFO:tasks.rados.rados.0.smithi100.stdout:1648: finishing write tid 4 to smithi10066867-44 2024-04-07T16:22:08.888 INFO:tasks.rados.rados.0.smithi100.stdout:1648: finishing write tid 5 to smithi10066867-44 2024-04-07T16:22:08.888 INFO:tasks.rados.rados.0.smithi100.stdout:1648: oid 44 updating version 0 to 796 2024-04-07T16:22:08.888 INFO:tasks.rados.rados.0.smithi100.stdout:1648: oid 44 version 796 is already newer than 795 2024-04-07T16:22:08.888 INFO:tasks.rados.rados.0.smithi100.stdout:1648: oid 44 updating version 796 to 797 2024-04-07T16:22:08.888 INFO:tasks.rados.rados.0.smithi100.stdout:1648: oid 44 updating version 797 to 798 2024-04-07T16:22:08.888 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 44 v 798 (ObjNum 509 snap 152 seq_num 509) dirty exists 2024-04-07T16:22:08.888 INFO:tasks.rados.rados.0.smithi100.stdout:1648: left oid 44 (ObjNum 509 snap 152 seq_num 509) 2024-04-07T16:22:08.888 INFO:tasks.rados.rados.0.smithi100.stdout:1649: finishing write tid 1 to smithi10066867-48 2024-04-07T16:22:08.890 INFO:tasks.rados.rados.0.smithi100.stdout:1649: finishing write tid 2 to smithi10066867-48 2024-04-07T16:22:08.891 INFO:tasks.rados.rados.0.smithi100.stdout:1648: done (2 left) 2024-04-07T16:22:08.891 INFO:tasks.rados.rados.0.smithi100.stdout:1650: done (1 left) 2024-04-07T16:22:08.891 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 22 current snap is 152 2024-04-07T16:22:08.891 INFO:tasks.rados.rados.0.smithi100.stdout:1649: finishing write tid 3 to smithi10066867-48 2024-04-07T16:22:08.891 INFO:tasks.rados.rados.0.smithi100.stdout:1649: finishing write tid 4 to smithi10066867-48 2024-04-07T16:22:08.891 INFO:tasks.rados.rados.0.smithi100.stdout:1651: seq_num 512 ranges {2867200=581632,3448832=516096,3964928=286720} 2024-04-07T16:22:08.910 INFO:tasks.rados.rados.0.smithi100.stdout:1651: writing smithi10066867-22 from 2867200 to 3448832 tid 1 2024-04-07T16:22:08.913 INFO:tasks.rados.rados.0.smithi100.stdout:1651: writing smithi10066867-22 from 3448832 to 3964928 tid 2 2024-04-07T16:22:08.915 INFO:tasks.rados.rados.0.smithi100.stdout:1651: writing smithi10066867-22 from 3964928 to 4251648 tid 3 2024-04-07T16:22:08.915 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 49 current snap is 152 2024-04-07T16:22:08.915 INFO:tasks.rados.rados.0.smithi100.stdout:1652: seq_num 513 ranges {745472=598016,1343488=794624,2138112=8192} 2024-04-07T16:22:08.923 INFO:tasks.rados.rados.0.smithi100.stdout:1652: writing smithi10066867-49 from 745472 to 1343488 tid 1 2024-04-07T16:22:08.929 INFO:tasks.rados.rados.0.smithi100.stdout:1652: writing smithi10066867-49 from 1343488 to 2138112 tid 2 2024-04-07T16:22:08.929 INFO:tasks.rados.rados.0.smithi100.stdout:1652: writing smithi10066867-49 from 2138112 to 2146304 tid 3 2024-04-07T16:22:08.929 INFO:tasks.rados.rados.0.smithi100.stdout:1653: read oid 44 snap 151 2024-04-07T16:22:08.929 INFO:tasks.rados.rados.0.smithi100.stdout:1653: expect (ObjNum 494 snap 148 seq_num 494) 2024-04-07T16:22:08.929 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 41 current snap is 152 2024-04-07T16:22:08.929 INFO:tasks.rados.rados.0.smithi100.stdout:1649: finishing write tid 5 to smithi10066867-48 2024-04-07T16:22:08.929 INFO:tasks.rados.rados.0.smithi100.stdout:1649: oid 48 updating version 0 to 986 2024-04-07T16:22:08.929 INFO:tasks.rados.rados.0.smithi100.stdout:1649: oid 48 version 986 is already newer than 985 2024-04-07T16:22:08.929 INFO:tasks.rados.rados.0.smithi100.stdout:1649: oid 48 updating version 986 to 988 2024-04-07T16:22:08.929 INFO:tasks.rados.rados.0.smithi100.stdout:1649: oid 48 version 988 is already newer than 987 2024-04-07T16:22:08.929 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 48 v 988 (ObjNum 510 snap 152 seq_num 510) dirty exists 2024-04-07T16:22:08.929 INFO:tasks.rados.rados.0.smithi100.stdout:1649: left oid 48 (ObjNum 510 snap 152 seq_num 510) 2024-04-07T16:22:08.929 INFO:tasks.rados.rados.0.smithi100.stdout:1651: finishing write tid 1 to smithi10066867-22 2024-04-07T16:22:08.930 INFO:tasks.rados.rados.0.smithi100.stdout:1651: finishing write tid 2 to smithi10066867-22 2024-04-07T16:22:08.930 INFO:tasks.rados.rados.0.smithi100.stdout:1651: finishing write tid 3 to smithi10066867-22 2024-04-07T16:22:08.930 INFO:tasks.rados.rados.0.smithi100.stdout:1651: finishing write tid 4 to smithi10066867-22 2024-04-07T16:22:08.930 INFO:tasks.rados.rados.0.smithi100.stdout:1651: finishing write tid 5 to smithi10066867-22 2024-04-07T16:22:08.930 INFO:tasks.rados.rados.0.smithi100.stdout:1651: oid 22 updating version 0 to 761 2024-04-07T16:22:08.930 INFO:tasks.rados.rados.0.smithi100.stdout:1651: oid 22 updating version 761 to 762 2024-04-07T16:22:08.930 INFO:tasks.rados.rados.0.smithi100.stdout:1651: oid 22 version 762 is already newer than 759 2024-04-07T16:22:08.930 INFO:tasks.rados.rados.0.smithi100.stdout:1651: oid 22 version 762 is already newer than 760 2024-04-07T16:22:08.930 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 22 v 762 (ObjNum 512 snap 152 seq_num 512) dirty exists 2024-04-07T16:22:08.930 INFO:tasks.rados.rados.0.smithi100.stdout:1651: left oid 22 (ObjNum 512 snap 152 seq_num 512) 2024-04-07T16:22:08.930 INFO:tasks.rados.rados.0.smithi100.stdout:1654: seq_num 514 ranges {2654208=614400,3268608=679936,3948544=122880} 2024-04-07T16:22:08.950 INFO:tasks.rados.rados.0.smithi100.stdout:1654: writing smithi10066867-41 from 2654208 to 3268608 tid 1 2024-04-07T16:22:08.953 INFO:tasks.rados.rados.0.smithi100.stdout:1654: writing smithi10066867-41 from 3268608 to 3948544 tid 2 2024-04-07T16:22:08.953 INFO:tasks.rados.rados.0.smithi100.stdout:1654: writing smithi10066867-41 from 3948544 to 4071424 tid 3 2024-04-07T16:22:08.953 INFO:tasks.rados.rados.0.smithi100.stdout:1649: done (4 left) 2024-04-07T16:22:08.953 INFO:tasks.rados.rados.0.smithi100.stdout:1651: done (3 left) 2024-04-07T16:22:08.954 INFO:tasks.rados.rados.0.smithi100.stdout:1655: read oid 39 snap -1 2024-04-07T16:22:08.954 INFO:tasks.rados.rados.0.smithi100.stdout:1655: expect (ObjNum 465 snap 142 seq_num 465) 2024-04-07T16:22:08.954 INFO:tasks.rados.rados.0.smithi100.stdout:1656: read oid 23 snap -1 2024-04-07T16:22:08.954 INFO:tasks.rados.rados.0.smithi100.stdout:1656: expect (ObjNum 487 snap 147 seq_num 487) 2024-04-07T16:22:08.954 INFO:tasks.rados.rados.0.smithi100.stdout:1657: delete oid 1 current snap is 152 2024-04-07T16:22:08.954 INFO:tasks.rados.rados.0.smithi100.stdout:1652: finishing write tid 1 to smithi10066867-49 2024-04-07T16:22:08.954 INFO:tasks.rados.rados.0.smithi100.stdout:1652: finishing write tid 2 to smithi10066867-49 2024-04-07T16:22:08.954 INFO:tasks.rados.rados.0.smithi100.stdout:1652: finishing write tid 3 to smithi10066867-49 2024-04-07T16:22:08.954 INFO:tasks.rados.rados.0.smithi100.stdout:1652: finishing write tid 4 to smithi10066867-49 2024-04-07T16:22:08.954 INFO:tasks.rados.rados.0.smithi100.stdout:1652: finishing write tid 5 to smithi10066867-49 2024-04-07T16:22:08.954 INFO:tasks.rados.rados.0.smithi100.stdout:1652: oid 49 updating version 0 to 838 2024-04-07T16:22:08.954 INFO:tasks.rados.rados.0.smithi100.stdout:1652: oid 49 version 838 is already newer than 837 2024-04-07T16:22:08.954 INFO:tasks.rados.rados.0.smithi100.stdout:1652: oid 49 version 838 is already newer than 835 2024-04-07T16:22:08.955 INFO:tasks.rados.rados.0.smithi100.stdout:1652: oid 49 version 838 is already newer than 836 2024-04-07T16:22:08.955 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 49 v 838 (ObjNum 513 snap 152 seq_num 513) dirty exists 2024-04-07T16:22:08.955 INFO:tasks.rados.rados.0.smithi100.stdout:1652: left oid 49 (ObjNum 513 snap 152 seq_num 513) 2024-04-07T16:22:08.955 INFO:tasks.rados.rados.0.smithi100.stdout:1653: expect (ObjNum 494 snap 148 seq_num 494) 2024-04-07T16:22:09.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:08 smithi100 bash[76202]: cluster 2024-04-07T16:22:08.826485+0000 mon.a (mon.0) 362 : cluster [WRN] Health check failed: Reduced data availability: 9 pgs inactive, 11 pgs peering (PG_AVAILABILITY) 2024-04-07T16:22:09.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:08 smithi100 bash[76202]: cluster 2024-04-07T16:22:08.844411+0000 mon.a (mon.0) 363 : cluster [DBG] osdmap e379: 8 total, 7 up, 8 in 2024-04-07T16:22:09.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:08 smithi100 bash[74942]: cluster 2024-04-07T16:22:08.826485+0000 mon.a (mon.0) 362 : cluster [WRN] Health check failed: Reduced data availability: 9 pgs inactive, 11 pgs peering (PG_AVAILABILITY) 2024-04-07T16:22:09.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:08 smithi100 bash[74942]: cluster 2024-04-07T16:22:08.844411+0000 mon.a (mon.0) 363 : cluster [DBG] osdmap e379: 8 total, 7 up, 8 in 2024-04-07T16:22:09.237 INFO:tasks.rados.rados.0.smithi100.stdout:1654: finishing write tid 1 to smithi10066867-41 2024-04-07T16:22:09.237 INFO:tasks.rados.rados.0.smithi100.stdout:1654: finishing write tid 2 to smithi10066867-41 2024-04-07T16:22:09.237 INFO:tasks.rados.rados.0.smithi100.stdout:1654: finishing write tid 3 to smithi10066867-41 2024-04-07T16:22:09.237 INFO:tasks.rados.rados.0.smithi100.stdout:1654: finishing write tid 4 to smithi10066867-41 2024-04-07T16:22:09.237 INFO:tasks.rados.rados.0.smithi100.stdout:1654: finishing write tid 5 to smithi10066867-41 2024-04-07T16:22:09.237 INFO:tasks.rados.rados.0.smithi100.stdout:1654: oid 41 updating version 0 to 843 2024-04-07T16:22:09.237 INFO:tasks.rados.rados.0.smithi100.stdout:1654: oid 41 version 843 is already newer than 842 2024-04-07T16:22:09.237 INFO:tasks.rados.rados.0.smithi100.stdout:1654: oid 41 version 843 is already newer than 841 2024-04-07T16:22:09.237 INFO:tasks.rados.rados.0.smithi100.stdout:1654: oid 41 version 843 is already newer than 840 2024-04-07T16:22:09.238 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 41 v 843 (ObjNum 514 snap 152 seq_num 514) dirty exists 2024-04-07T16:22:09.238 INFO:tasks.rados.rados.0.smithi100.stdout:1654: left oid 41 (ObjNum 514 snap 152 seq_num 514) 2024-04-07T16:22:09.238 INFO:tasks.rados.rados.0.smithi100.stdout:1656: expect (ObjNum 487 snap 147 seq_num 487) 2024-04-07T16:22:09.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:08 smithi144 bash[66654]: cluster 2024-04-07T16:22:08.826485+0000 mon.a (mon.0) 362 : cluster [WRN] Health check failed: Reduced data availability: 9 pgs inactive, 11 pgs peering (PG_AVAILABILITY) 2024-04-07T16:22:09.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:08 smithi144 bash[66654]: cluster 2024-04-07T16:22:08.844411+0000 mon.a (mon.0) 363 : cluster [DBG] osdmap e379: 8 total, 7 up, 8 in 2024-04-07T16:22:09.371 INFO:tasks.rados.rados.0.smithi100.stdout:1655: expect (ObjNum 465 snap 142 seq_num 465) 2024-04-07T16:22:09.524 INFO:tasks.rados.rados.0.smithi100.stdout:1652: done (5 left) 2024-04-07T16:22:09.525 INFO:tasks.rados.rados.0.smithi100.stdout:1653: done (4 left) 2024-04-07T16:22:09.525 INFO:tasks.rados.rados.0.smithi100.stdout:1654: done (3 left) 2024-04-07T16:22:09.525 INFO:tasks.rados.rados.0.smithi100.stdout:1655: done (2 left) 2024-04-07T16:22:09.525 INFO:tasks.rados.rados.0.smithi100.stdout:1656: done (1 left) 2024-04-07T16:22:09.525 INFO:tasks.rados.rados.0.smithi100.stdout:1657: done (0 left) 2024-04-07T16:22:09.525 INFO:tasks.rados.rados.0.smithi100.stdout:1658: copy_from oid 3 from oid 11 current snap is 152 2024-04-07T16:22:09.525 INFO:tasks.rados.rados.0.smithi100.stdout:1659: snap_create 2024-04-07T16:22:09.539 INFO:tasks.rados.rados.0.smithi100.stdout:1658: finishing copy_from racing read to smithi10066867-3 2024-04-07T16:22:09.540 INFO:tasks.rados.rados.0.smithi100.stdout:1658: finishing copy_from to smithi10066867-3 2024-04-07T16:22:09.540 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 3 v 1004 (ObjNum 501 snap 150 seq_num 501) dirty exists 2024-04-07T16:22:09.882 INFO:tasks.rados.rados.0.smithi100.stdout:1658: done (1 left) 2024-04-07T16:22:09.882 INFO:tasks.rados.rados.0.smithi100.stdout:1659: done (0 left) 2024-04-07T16:22:09.882 INFO:tasks.rados.rados.0.smithi100.stdout:1660: read oid 44 snap 152 2024-04-07T16:22:09.882 INFO:tasks.rados.rados.0.smithi100.stdout:1660: expect (ObjNum 509 snap 152 seq_num 509) 2024-04-07T16:22:09.882 INFO:tasks.rados.rados.0.smithi100.stdout:1661: read oid 8 snap -1 2024-04-07T16:22:09.883 INFO:tasks.rados.rados.0.smithi100.stdout:1661: expect (ObjNum 440 snap 134 seq_num 440) 2024-04-07T16:22:09.883 INFO:tasks.rados.rados.0.smithi100.stdout:1662: copy_from oid 40 from oid 21 current snap is 153 2024-04-07T16:22:09.883 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 10 current snap is 153 2024-04-07T16:22:09.883 INFO:tasks.rados.rados.0.smithi100.stdout:1663: seq_num 516 ranges {2719744=647168,3366912=450560,3817472=352256} 2024-04-07T16:22:09.924 INFO:tasks.rados.rados.0.smithi100.stdout:1663: writing smithi10066867-10 from 2719744 to 3366912 tid 1 2024-04-07T16:22:09.929 INFO:tasks.rados.rados.0.smithi100.stdout:1663: writing smithi10066867-10 from 3366912 to 3817472 tid 2 2024-04-07T16:22:09.934 INFO:tasks.rados.rados.0.smithi100.stdout:1663: writing smithi10066867-10 from 3817472 to 4169728 tid 3 2024-04-07T16:22:09.934 INFO:tasks.rados.rados.0.smithi100.stdout:1664: rollback oid 43 current snap is 153 2024-04-07T16:22:09.934 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 43 to 151 2024-04-07T16:22:09.934 INFO:tasks.rados.rados.0.smithi100.stdout:1662: finishing copy_from racing read to smithi10066867-40 2024-04-07T16:22:09.934 INFO:tasks.rados.rados.0.smithi100.stdout:1661: expect (ObjNum 440 snap 134 seq_num 440) 2024-04-07T16:22:10.126 INFO:journalctl@ceph.osd.0.smithi100.stdout:Apr 07 16:22:10 smithi100 systemd[1]: ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@osd.0.service: Deactivated successfully. 2024-04-07T16:22:10.126 INFO:journalctl@ceph.osd.0.smithi100.stdout:Apr 07 16:22:10 smithi100 systemd[1]: Stopped Ceph osd.0 for 0702e3a8-f4f8-11ee-b649-cb9ed24678a4. 2024-04-07T16:22:10.126 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:09 smithi100 bash[74942]: cluster 2024-04-07T16:22:08.642565+0000 mgr.y (mgr.25175) 119 : cluster [DBG] pgmap v157: 121 pgs: 52 peering, 69 active+clean; 104 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 5.3 MiB/s rd, 3.8 MiB/s wr, 18 op/s 2024-04-07T16:22:10.126 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:09 smithi100 bash[76202]: cluster 2024-04-07T16:22:08.642565+0000 mgr.y (mgr.25175) 119 : cluster [DBG] pgmap v157: 121 pgs: 52 peering, 69 active+clean; 104 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 5.3 MiB/s rd, 3.8 MiB/s wr, 18 op/s 2024-04-07T16:22:10.163 INFO:tasks.rados.rados.0.smithi100.stdout:1663: finishing write tid 1 to smithi10066867-10 2024-04-07T16:22:10.163 INFO:tasks.rados.rados.0.smithi100.stdout:1660: expect (ObjNum 509 snap 152 seq_num 509) 2024-04-07T16:22:10.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:09 smithi144 bash[66654]: cluster 2024-04-07T16:22:08.642565+0000 mgr.y (mgr.25175) 119 : cluster [DBG] pgmap v157: 121 pgs: 52 peering, 69 active+clean; 104 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 5.3 MiB/s rd, 3.8 MiB/s wr, 18 op/s 2024-04-07T16:22:10.435 INFO:journalctl@ceph.osd.2.smithi100.stdout:Apr 07 16:22:10 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:22:10.435 INFO:journalctl@ceph.osd.3.smithi100.stdout:Apr 07 16:22:10 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:22:10.435 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:10 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:22:10.436 INFO:journalctl@ceph.osd.1.smithi100.stdout:Apr 07 16:22:10 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:22:10.436 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:10 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:22:10.436 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Apr 07 16:22:10 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:22:10.436 INFO:journalctl@ceph.osd.0.smithi100.stdout:Apr 07 16:22:10 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:22:10.437 INFO:journalctl@ceph.osd.0.smithi100.stdout:Apr 07 16:22:10 smithi100 systemd[1]: Started Ceph osd.0 for 0702e3a8-f4f8-11ee-b649-cb9ed24678a4. 2024-04-07T16:22:10.437 INFO:journalctl@ceph.mds.a.smithi100.stdout:Apr 07 16:22:10 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:22:10.437 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:22:10 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:22:10.437 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 16:22:10 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:22:10.564 INFO:tasks.rados.rados.0.smithi100.stdout:1662: finishing copy_from to smithi10066867-40 2024-04-07T16:22:10.564 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 40 v 524 (ObjNum 391 snap 119 seq_num 391) dirty exists 2024-04-07T16:22:10.564 INFO:tasks.rados.rados.0.smithi100.stdout:1663: finishing write tid 2 to smithi10066867-10 2024-04-07T16:22:10.564 INFO:tasks.rados.rados.0.smithi100.stdout:1660: done (4 left) 2024-04-07T16:22:10.564 INFO:tasks.rados.rados.0.smithi100.stdout:1661: done (3 left) 2024-04-07T16:22:10.564 INFO:tasks.rados.rados.0.smithi100.stdout:1662: done (2 left) 2024-04-07T16:22:10.564 INFO:tasks.rados.rados.0.smithi100.stdout:1665: snap_remove snap 152 2024-04-07T16:22:11.131 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:10 smithi100 bash[74942]: cluster 2024-04-07T16:22:09.898783+0000 mon.a (mon.0) 364 : cluster [DBG] osdmap e380: 8 total, 7 up, 8 in 2024-04-07T16:22:11.131 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:10 smithi100 bash[74942]: audit 2024-04-07T16:22:10.300037+0000 mon.a (mon.0) 365 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:22:11.132 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:10 smithi100 bash[74942]: audit 2024-04-07T16:22:10.305774+0000 mon.a (mon.0) 366 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:22:11.132 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:10 smithi100 bash[74942]: audit 2024-04-07T16:22:10.321712+0000 mon.a (mon.0) 367 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:22:11.132 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:10 smithi100 bash[76202]: cluster 2024-04-07T16:22:09.898783+0000 mon.a (mon.0) 364 : cluster [DBG] osdmap e380: 8 total, 7 up, 8 in 2024-04-07T16:22:11.132 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:10 smithi100 bash[76202]: audit 2024-04-07T16:22:10.300037+0000 mon.a (mon.0) 365 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:22:11.132 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:10 smithi100 bash[76202]: audit 2024-04-07T16:22:10.305774+0000 mon.a (mon.0) 366 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:22:11.132 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:10 smithi100 bash[76202]: audit 2024-04-07T16:22:10.321712+0000 mon.a (mon.0) 367 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:22:11.301 INFO:tasks.rados.rados.0.smithi100.stdout:1665: done (2 left) 2024-04-07T16:22:11.301 INFO:tasks.rados.rados.0.smithi100.stdout:1666: rollback oid 4 current snap is 153 2024-04-07T16:22:11.301 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 4 to 151 2024-04-07T16:22:11.302 INFO:tasks.rados.rados.0.smithi100.stdout:1664: finishing rollback tid 1 to smithi10066867-43 2024-04-07T16:22:11.302 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 43 v 0 (ObjNum 289 snap 0 seq_num 1) dirty dne 2024-04-07T16:22:11.302 INFO:tasks.rados.rados.0.smithi100.stdout:1663: finishing write tid 3 to smithi10066867-10 2024-04-07T16:22:11.302 INFO:tasks.rados.rados.0.smithi100.stdout:1663: finishing write tid 4 to smithi10066867-10 2024-04-07T16:22:11.302 INFO:tasks.rados.rados.0.smithi100.stdout:1663: finishing write tid 5 to smithi10066867-10 2024-04-07T16:22:11.302 INFO:tasks.rados.rados.0.smithi100.stdout:1663: oid 10 updating version 0 to 845 2024-04-07T16:22:11.302 INFO:tasks.rados.rados.0.smithi100.stdout:1663: oid 10 version 845 is already newer than 844 2024-04-07T16:22:11.302 INFO:tasks.rados.rados.0.smithi100.stdout:1663: oid 10 updating version 845 to 846 2024-04-07T16:22:11.302 INFO:tasks.rados.rados.0.smithi100.stdout:1663: oid 10 version 846 is already newer than 843 2024-04-07T16:22:11.302 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 10 v 846 (ObjNum 516 snap 153 seq_num 516) dirty exists 2024-04-07T16:22:11.302 INFO:tasks.rados.rados.0.smithi100.stdout:1663: left oid 10 (ObjNum 516 snap 153 seq_num 516) 2024-04-07T16:22:11.302 INFO:tasks.rados.rados.0.smithi100.stdout:1663: done (2 left) 2024-04-07T16:22:11.302 INFO:tasks.rados.rados.0.smithi100.stdout:1664: done (1 left) 2024-04-07T16:22:11.302 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 8 current snap is 153 2024-04-07T16:22:11.303 INFO:tasks.rados.rados.0.smithi100.stdout:1667: seq_num 517 ranges {4038656=663552,4702208=786432,5488640=16384} 2024-04-07T16:22:11.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:10 smithi144 bash[66654]: cluster 2024-04-07T16:22:09.898783+0000 mon.a (mon.0) 364 : cluster [DBG] osdmap e380: 8 total, 7 up, 8 in 2024-04-07T16:22:11.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:10 smithi144 bash[66654]: audit 2024-04-07T16:22:10.300037+0000 mon.a (mon.0) 365 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:22:11.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:10 smithi144 bash[66654]: audit 2024-04-07T16:22:10.305774+0000 mon.a (mon.0) 366 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:22:11.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:10 smithi144 bash[66654]: audit 2024-04-07T16:22:10.321712+0000 mon.a (mon.0) 367 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:22:11.364 INFO:tasks.rados.rados.0.smithi100.stdout:1667: writing smithi10066867-8 from 4038656 to 4702208 tid 1 2024-04-07T16:22:11.372 INFO:tasks.rados.rados.0.smithi100.stdout:1667: writing smithi10066867-8 from 4702208 to 5488640 tid 2 2024-04-07T16:22:11.372 INFO:tasks.rados.rados.0.smithi100.stdout:1667: writing smithi10066867-8 from 5488640 to 5505024 tid 3 2024-04-07T16:22:11.372 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 50 current snap is 153 2024-04-07T16:22:11.372 INFO:tasks.rados.rados.0.smithi100.stdout:1666: finishing rollback tid 0 to smithi10066867-4 2024-04-07T16:22:11.372 INFO:tasks.rados.rados.0.smithi100.stdout:1666: finishing rollback tid 1 to smithi10066867-4 2024-04-07T16:22:11.372 INFO:tasks.rados.rados.0.smithi100.stdout:1666: finishing rollback tid 2 to smithi10066867-4 2024-04-07T16:22:11.372 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 4 v 1008 (ObjNum 487 snap 147 seq_num 487) dirty exists 2024-04-07T16:22:11.372 INFO:tasks.rados.rados.0.smithi100.stdout:1668: seq_num 518 ranges {2301952=679936,2981888=671744,3653632=131072} 2024-04-07T16:22:11.395 INFO:tasks.rados.rados.0.smithi100.stdout:1668: writing smithi10066867-50 from 2301952 to 2981888 tid 1 2024-04-07T16:22:11.400 INFO:tasks.rados.rados.0.smithi100.stdout:1668: writing smithi10066867-50 from 2981888 to 3653632 tid 2 2024-04-07T16:22:11.402 INFO:tasks.rados.rados.0.smithi100.stdout:1668: writing smithi10066867-50 from 3653632 to 3784704 tid 3 2024-04-07T16:22:11.402 INFO:tasks.rados.rados.0.smithi100.stdout:1666: done (2 left) 2024-04-07T16:22:11.402 INFO:tasks.rados.rados.0.smithi100.stdout:1669: read oid 37 snap 151 2024-04-07T16:22:11.402 INFO:tasks.rados.rados.0.smithi100.stdout:1669: expect (ObjNum -1593008320 snap 21978 seq_num 2701835216) 2024-04-07T16:22:11.402 INFO:tasks.rados.rados.0.smithi100.stdout:1670: rollback oid 45 current snap is 153 2024-04-07T16:22:11.402 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 45 to 151 2024-04-07T16:22:11.402 INFO:tasks.rados.rados.0.smithi100.stdout:1667: finishing write tid 1 to smithi10066867-8 2024-04-07T16:22:11.402 INFO:tasks.rados.rados.0.smithi100.stdout:1667: finishing write tid 2 to smithi10066867-8 2024-04-07T16:22:11.403 INFO:tasks.rados.rados.0.smithi100.stdout:1667: finishing write tid 3 to smithi10066867-8 2024-04-07T16:22:11.403 INFO:tasks.rados.rados.0.smithi100.stdout:1667: finishing write tid 4 to smithi10066867-8 2024-04-07T16:22:11.403 INFO:tasks.rados.rados.0.smithi100.stdout:1667: finishing write tid 5 to smithi10066867-8 2024-04-07T16:22:11.403 INFO:tasks.rados.rados.0.smithi100.stdout:1667: oid 8 updating version 0 to 703 2024-04-07T16:22:11.403 INFO:tasks.rados.rados.0.smithi100.stdout:1667: oid 8 updating version 703 to 704 2024-04-07T16:22:11.403 INFO:tasks.rados.rados.0.smithi100.stdout:1667: oid 8 version 704 is already newer than 702 2024-04-07T16:22:11.403 INFO:tasks.rados.rados.0.smithi100.stdout:1667: oid 8 updating version 704 to 705 2024-04-07T16:22:11.403 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 8 v 705 (ObjNum 517 snap 153 seq_num 517) dirty exists 2024-04-07T16:22:11.403 INFO:tasks.rados.rados.0.smithi100.stdout:1667: left oid 8 (ObjNum 517 snap 153 seq_num 517) 2024-04-07T16:22:11.403 INFO:tasks.rados.rados.0.smithi100.stdout:1667: done (3 left) 2024-04-07T16:22:11.403 INFO:tasks.rados.rados.0.smithi100.stdout:1671: read oid 28 snap -1 2024-04-07T16:22:11.403 INFO:tasks.rados.rados.0.smithi100.stdout:1671: expect (ObjNum 461 snap 140 seq_num 461) 2024-04-07T16:22:11.403 INFO:tasks.rados.rados.0.smithi100.stdout:1672: snap_create 2024-04-07T16:22:11.404 INFO:tasks.rados.rados.0.smithi100.stdout:1668: finishing write tid 1 to smithi10066867-50 2024-04-07T16:22:11.404 INFO:tasks.rados.rados.0.smithi100.stdout:1669: expect (ObjNum -1593008320 snap 21978 seq_num 2701835216) 2024-04-07T16:22:11.417 INFO:tasks.rados.rados.0.smithi100.stdout:1668: finishing write tid 2 to smithi10066867-50 2024-04-07T16:22:11.420 INFO:tasks.rados.rados.0.smithi100.stdout:1668: finishing write tid 3 to smithi10066867-50 2024-04-07T16:22:11.420 INFO:tasks.rados.rados.0.smithi100.stdout:1668: finishing write tid 4 to smithi10066867-50 2024-04-07T16:22:11.422 INFO:tasks.rados.rados.0.smithi100.stdout:1670: finishing rollback tid 0 to smithi10066867-45 2024-04-07T16:22:11.422 INFO:tasks.rados.rados.0.smithi100.stdout:1671: expect (ObjNum 461 snap 140 seq_num 461) 2024-04-07T16:22:11.503 INFO:tasks.rados.rados.0.smithi100.stdout:1670: finishing rollback tid 1 to smithi10066867-45 2024-04-07T16:22:11.504 INFO:tasks.rados.rados.0.smithi100.stdout:1670: finishing rollback tid 2 to smithi10066867-45 2024-04-07T16:22:11.504 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 45 v 796 (ObjNum 491 snap 148 seq_num 491) dirty exists 2024-04-07T16:22:11.504 INFO:tasks.rados.rados.0.smithi100.stdout:1668: finishing write tid 5 to smithi10066867-50 2024-04-07T16:22:11.504 INFO:tasks.rados.rados.0.smithi100.stdout:1668: oid 50 updating version 0 to 790 2024-04-07T16:22:11.504 INFO:tasks.rados.rados.0.smithi100.stdout:1668: oid 50 updating version 790 to 792 2024-04-07T16:22:11.504 INFO:tasks.rados.rados.0.smithi100.stdout:1668: oid 50 version 792 is already newer than 791 2024-04-07T16:22:11.504 INFO:tasks.rados.rados.0.smithi100.stdout:1668: oid 50 updating version 792 to 793 2024-04-07T16:22:11.504 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 50 v 793 (ObjNum 518 snap 153 seq_num 518) dirty exists 2024-04-07T16:22:11.504 INFO:tasks.rados.rados.0.smithi100.stdout:1668: left oid 50 (ObjNum 518 snap 153 seq_num 518) 2024-04-07T16:22:11.775 INFO:tasks.rados.rados.0.smithi100.stdout:1668: done (4 left) 2024-04-07T16:22:11.775 INFO:tasks.rados.rados.0.smithi100.stdout:1669: done (3 left) 2024-04-07T16:22:11.775 INFO:tasks.rados.rados.0.smithi100.stdout:1670: done (2 left) 2024-04-07T16:22:11.775 INFO:tasks.rados.rados.0.smithi100.stdout:1671: done (1 left) 2024-04-07T16:22:11.776 INFO:tasks.rados.rados.0.smithi100.stdout:1672: done (0 left) 2024-04-07T16:22:11.776 INFO:tasks.rados.rados.0.smithi100.stdout:1673: snap_create 2024-04-07T16:22:12.185 INFO:journalctl@ceph.osd.0.smithi100.stdout:Apr 07 16:22:11 smithi100 bash[80704]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-07T16:22:12.185 INFO:journalctl@ceph.osd.0.smithi100.stdout:Apr 07 16:22:11 smithi100 bash[80704]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-0 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-04-07T16:22:12.185 INFO:journalctl@ceph.osd.0.smithi100.stdout:Apr 07 16:22:11 smithi100 bash[80704]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-04-07T16:22:12.185 INFO:journalctl@ceph.osd.0.smithi100.stdout:Apr 07 16:22:11 smithi100 bash[80704]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-04-07T16:22:12.185 INFO:journalctl@ceph.osd.0.smithi100.stdout:Apr 07 16:22:11 smithi100 bash[80704]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-0/block 2024-04-07T16:22:12.185 INFO:journalctl@ceph.osd.0.smithi100.stdout:Apr 07 16:22:11 smithi100 bash[80704]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-07T16:22:12.185 INFO:journalctl@ceph.osd.0.smithi100.stdout:Apr 07 16:22:11 smithi100 bash[80704]: --> ceph-volume raw activate successful for osd ID: 0 2024-04-07T16:22:12.606 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:22:12 smithi144 bash[59794]: ts=2024-04-07T16:22:12.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:22:12.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:12 smithi144 bash[66654]: cluster 2024-04-07T16:22:10.643390+0000 mgr.y (mgr.25175) 120 : cluster [DBG] pgmap v160: 121 pgs: 52 peering, 69 active+clean; 104 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 5.3 MiB/s rd, 3.8 MiB/s wr, 18 op/s 2024-04-07T16:22:12.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:12 smithi144 bash[66654]: cluster 2024-04-07T16:22:11.318283+0000 mon.a (mon.0) 368 : cluster [DBG] osdmap e381: 8 total, 7 up, 8 in 2024-04-07T16:22:12.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:12 smithi144 bash[66654]: cluster 2024-04-07T16:22:11.793099+0000 mon.a (mon.0) 369 : cluster [DBG] osdmap e382: 8 total, 7 up, 8 in 2024-04-07T16:22:12.685 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:12 smithi100 bash[74942]: cluster 2024-04-07T16:22:10.643390+0000 mgr.y (mgr.25175) 120 : cluster [DBG] pgmap v160: 121 pgs: 52 peering, 69 active+clean; 104 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 5.3 MiB/s rd, 3.8 MiB/s wr, 18 op/s 2024-04-07T16:22:12.685 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:12 smithi100 bash[74942]: cluster 2024-04-07T16:22:11.318283+0000 mon.a (mon.0) 368 : cluster [DBG] osdmap e381: 8 total, 7 up, 8 in 2024-04-07T16:22:12.685 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:12 smithi100 bash[74942]: cluster 2024-04-07T16:22:11.793099+0000 mon.a (mon.0) 369 : cluster [DBG] osdmap e382: 8 total, 7 up, 8 in 2024-04-07T16:22:12.685 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:12 smithi100 bash[76202]: cluster 2024-04-07T16:22:10.643390+0000 mgr.y (mgr.25175) 120 : cluster [DBG] pgmap v160: 121 pgs: 52 peering, 69 active+clean; 104 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 5.3 MiB/s rd, 3.8 MiB/s wr, 18 op/s 2024-04-07T16:22:12.685 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:12 smithi100 bash[76202]: cluster 2024-04-07T16:22:11.318283+0000 mon.a (mon.0) 368 : cluster [DBG] osdmap e381: 8 total, 7 up, 8 in 2024-04-07T16:22:12.685 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:12 smithi100 bash[76202]: cluster 2024-04-07T16:22:11.793099+0000 mon.a (mon.0) 369 : cluster [DBG] osdmap e382: 8 total, 7 up, 8 in 2024-04-07T16:22:12.786 INFO:tasks.rados.rados.0.smithi100.stdout:1673: done (0 left) 2024-04-07T16:22:12.786 INFO:tasks.rados.rados.0.smithi100.stdout:1674: copy_from oid 10 from oid 15 current snap is 155 2024-04-07T16:22:12.786 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 35 current snap is 155 2024-04-07T16:22:12.786 INFO:tasks.rados.rados.0.smithi100.stdout:1675: seq_num 519 ranges {0=696320,696320=548864,1245184=253952} 2024-04-07T16:22:12.794 INFO:tasks.rados.rados.0.smithi100.stdout:1675: writing smithi10066867-35 from 0 to 696320 tid 1 2024-04-07T16:22:12.799 INFO:tasks.rados.rados.0.smithi100.stdout:1675: writing smithi10066867-35 from 696320 to 1245184 tid 2 2024-04-07T16:22:12.801 INFO:tasks.rados.rados.0.smithi100.stdout:1675: writing smithi10066867-35 from 1245184 to 1499136 tid 3 2024-04-07T16:22:12.801 INFO:tasks.rados.rados.0.smithi100.stdout:1676: copy_from oid 6 from oid 14 current snap is 155 2024-04-07T16:22:12.801 INFO:tasks.rados.rados.0.smithi100.stdout:1674: finishing copy_from racing read to smithi10066867-10 2024-04-07T16:22:12.801 INFO:tasks.rados.rados.0.smithi100.stdout:1677: snap_create 2024-04-07T16:22:12.802 INFO:tasks.rados.rados.0.smithi100.stdout:1674: finishing copy_from to smithi10066867-10 2024-04-07T16:22:12.802 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 10 v 980 (ObjNum 257 snap 0 seq_num 3067284944) dirty exists 2024-04-07T16:22:12.805 INFO:tasks.rados.rados.0.smithi100.stdout:1676: finishing copy_from to smithi10066867-6 2024-04-07T16:22:12.805 INFO:tasks.rados.rados.0.smithi100.stdout:1676: got expected ENOENT (src dne) 2024-04-07T16:22:12.806 INFO:tasks.rados.rados.0.smithi100.stdout:1676: finishing copy_from racing read to smithi10066867-6 2024-04-07T16:22:12.809 INFO:tasks.rados.rados.0.smithi100.stdout:1675: finishing write tid 1 to smithi10066867-35 2024-04-07T16:22:12.810 INFO:tasks.rados.rados.0.smithi100.stdout:1675: finishing write tid 2 to smithi10066867-35 2024-04-07T16:22:12.810 INFO:tasks.rados.rados.0.smithi100.stdout:1675: finishing write tid 3 to smithi10066867-35 2024-04-07T16:22:12.811 INFO:tasks.rados.rados.0.smithi100.stdout:1675: finishing write tid 4 to smithi10066867-35 2024-04-07T16:22:12.813 INFO:tasks.rados.rados.0.smithi100.stdout:1675: finishing write tid 5 to smithi10066867-35 2024-04-07T16:22:12.813 INFO:tasks.rados.rados.0.smithi100.stdout:1675: oid 35 updating version 0 to 992 2024-04-07T16:22:12.813 INFO:tasks.rados.rados.0.smithi100.stdout:1675: oid 35 version 992 is already newer than 991 2024-04-07T16:22:12.814 INFO:tasks.rados.rados.0.smithi100.stdout:1675: oid 35 version 992 is already newer than 989 2024-04-07T16:22:12.814 INFO:tasks.rados.rados.0.smithi100.stdout:1675: oid 35 version 992 is already newer than 990 2024-04-07T16:22:12.814 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 35 v 992 (ObjNum 519 snap 155 seq_num 519) dirty exists 2024-04-07T16:22:12.814 INFO:tasks.rados.rados.0.smithi100.stdout:1675: left oid 35 (ObjNum 519 snap 155 seq_num 519) 2024-04-07T16:22:13.685 INFO:journalctl@ceph.osd.0.smithi100.stdout:Apr 07 16:22:13 smithi100 bash[80932]: debug 2024-04-07T16:22:13.312+0000 7f992eadc700 1 -- 172.21.15.100:0/582624338 <== mon.0 v2:172.21.15.100:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b4c1d8d680 con 0x55b4c1dcb000 2024-04-07T16:22:13.779 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:22:13 smithi144 bash[59794]: ts=2024-04-07T16:22:13.513Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=nodes name=CephNodeDiskspaceWarning index=4 msg="Evaluating rule failed" rule="alert: CephNodeDiskspaceWarning\nexpr: predict_linear(node_filesystem_free_bytes{device=~\"/.*\"}[2d], 3600 * 24 * 5)\n * on (instance) group_left (nodename) node_uname_info < 0\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.8.4\n severity: warning\n type: ceph_default\nannotations:\n description: Mountpoint {{ $labels.mountpoint }} on {{ $labels.nodename }} will\n be full in less than 5 days based on the 48 hour trailing fill rate.\n summary: Host filesystem free space is getting low\n" err="found duplicate series for the match group {instance=\"smithi100\"} on the right hand-side of the operation: [{__name__=\"node_uname_info\", domainname=\"(none)\", instance=\"smithi100\", job=\"node\", machine=\"x86_64\", nodename=\"smithi100\", release=\"5.15.0-101-generic\", sysname=\"Linux\", version=\"#111-Ubuntu SMP Tue Mar 5 20:16:58 UTC 2024\"}, {__name__=\"node_uname_info\", cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", domainname=\"(none)\", instance=\"smithi100\", job=\"node\", machine=\"x86_64\", nodename=\"smithi100\", release=\"5.15.0-101-generic\", sysname=\"Linux\", version=\"#111-Ubuntu SMP Tue Mar 5 20:16:58 UTC 2024\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:22:13.794 INFO:tasks.rados.rados.0.smithi100.stdout:1674: done (3 left) 2024-04-07T16:22:13.794 INFO:tasks.rados.rados.0.smithi100.stdout:1675: done (2 left) 2024-04-07T16:22:13.794 INFO:tasks.rados.rados.0.smithi100.stdout:1676: done (1 left) 2024-04-07T16:22:13.795 INFO:tasks.rados.rados.0.smithi100.stdout:1677: done (0 left) 2024-04-07T16:22:13.795 INFO:tasks.rados.rados.0.smithi100.stdout:1678: delete oid 36 current snap is 156 2024-04-07T16:22:13.807 INFO:tasks.rados.rados.0.smithi100.stdout:1678: done (0 left) 2024-04-07T16:22:13.807 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 4 current snap is 156 2024-04-07T16:22:13.807 INFO:tasks.rados.rados.0.smithi100.stdout:1679: seq_num 521 ranges {2990080=729088,3719168=720896,4440064=81920} 2024-04-07T16:22:13.854 INFO:tasks.rados.rados.0.smithi100.stdout:1679: writing smithi10066867-4 from 2990080 to 3719168 tid 1 2024-04-07T16:22:13.859 INFO:tasks.rados.rados.0.smithi100.stdout:1679: writing smithi10066867-4 from 3719168 to 4440064 tid 2 2024-04-07T16:22:13.860 INFO:tasks.rados.rados.0.smithi100.stdout:1679: writing smithi10066867-4 from 4440064 to 4521984 tid 3 2024-04-07T16:22:13.860 INFO:tasks.rados.rados.0.smithi100.stdout:1680: rollback oid 39 current snap is 156 2024-04-07T16:22:13.860 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 39 to 155 2024-04-07T16:22:13.860 INFO:tasks.rados.rados.0.smithi100.stdout:1681: read oid 48 snap 151 2024-04-07T16:22:13.860 INFO:tasks.rados.rados.0.smithi100.stdout:1681: expect (ObjNum 463 snap 141 seq_num 463) 2024-04-07T16:22:13.860 INFO:tasks.rados.rados.0.smithi100.stdout:1682: rollback oid 36 current snap is 156 2024-04-07T16:22:13.861 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 36 to 155 2024-04-07T16:22:13.861 INFO:tasks.rados.rados.0.smithi100.stdout:1683: rmattr oid 5 current snap is 156 2024-04-07T16:22:13.861 INFO:tasks.rados.rados.0.smithi100.stdout:1683: done (4 left) 2024-04-07T16:22:13.861 INFO:tasks.rados.rados.0.smithi100.stdout:1684: setattr oid 8 current snap is 156 2024-04-07T16:22:13.863 INFO:tasks.rados.rados.0.smithi100.stdout:1685: read oid 14 snap 151 2024-04-07T16:22:13.863 INFO:tasks.rados.rados.0.smithi100.stdout:1685: expect deleted 2024-04-07T16:22:13.863 INFO:tasks.rados.rados.0.smithi100.stdout:1686: setattr oid 50 current snap is 156 2024-04-07T16:22:13.865 INFO:tasks.rados.rados.0.smithi100.stdout:1679: finishing write tid 1 to smithi10066867-4 2024-04-07T16:22:13.866 INFO:tasks.rados.rados.0.smithi100.stdout:1682: finishing rollback tid 1 to smithi10066867-36 2024-04-07T16:22:13.867 INFO:tasks.rados.rados.0.smithi100.stdout:1685: done (6 left) 2024-04-07T16:22:13.867 INFO:tasks.rados.rados.0.smithi100.stdout:1687: snap_remove snap 154 2024-04-07T16:22:14.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:13 smithi144 bash[66654]: cluster 2024-04-07T16:22:12.645099+0000 mgr.y (mgr.25175) 121 : cluster [DBG] pgmap v163: 121 pgs: 15 active+undersized+degraded, 37 active+undersized, 69 active+clean; 111 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 19 MiB/s rd, 3.7 MiB/s wr, 23 op/s; 43/234 objects degraded (18.376%) 2024-04-07T16:22:14.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:13 smithi144 bash[66654]: cluster 2024-04-07T16:22:12.803192+0000 mon.a (mon.0) 370 : cluster [DBG] osdmap e383: 8 total, 7 up, 8 in 2024-04-07T16:22:14.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:13 smithi144 bash[66654]: cluster 2024-04-07T16:22:13.315317+0000 mon.a (mon.0) 371 : cluster [WRN] Health check failed: Degraded data redundancy: 43/234 objects degraded (18.376%), 15 pgs degraded (PG_DEGRADED) 2024-04-07T16:22:14.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:13 smithi144 bash[66654]: cluster 2024-04-07T16:22:13.315358+0000 mon.a (mon.0) 372 : cluster [INF] Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 9 pgs inactive, 11 pgs peering) 2024-04-07T16:22:14.185 INFO:journalctl@ceph.osd.0.smithi100.stdout:Apr 07 16:22:13 smithi100 bash[80932]: debug 2024-04-07T16:22:13.864+0000 7f9937c747c0 -1 Falling back to public interface 2024-04-07T16:22:14.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:13 smithi100 bash[74942]: cluster 2024-04-07T16:22:12.645099+0000 mgr.y (mgr.25175) 121 : cluster [DBG] pgmap v163: 121 pgs: 15 active+undersized+degraded, 37 active+undersized, 69 active+clean; 111 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 19 MiB/s rd, 3.7 MiB/s wr, 23 op/s; 43/234 objects degraded (18.376%) 2024-04-07T16:22:14.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:13 smithi100 bash[74942]: cluster 2024-04-07T16:22:12.803192+0000 mon.a (mon.0) 370 : cluster [DBG] osdmap e383: 8 total, 7 up, 8 in 2024-04-07T16:22:14.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:13 smithi100 bash[74942]: cluster 2024-04-07T16:22:13.315317+0000 mon.a (mon.0) 371 : cluster [WRN] Health check failed: Degraded data redundancy: 43/234 objects degraded (18.376%), 15 pgs degraded (PG_DEGRADED) 2024-04-07T16:22:14.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:13 smithi100 bash[74942]: cluster 2024-04-07T16:22:13.315358+0000 mon.a (mon.0) 372 : cluster [INF] Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 9 pgs inactive, 11 pgs peering) 2024-04-07T16:22:14.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:13 smithi100 bash[76202]: cluster 2024-04-07T16:22:12.645099+0000 mgr.y (mgr.25175) 121 : cluster [DBG] pgmap v163: 121 pgs: 15 active+undersized+degraded, 37 active+undersized, 69 active+clean; 111 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 19 MiB/s rd, 3.7 MiB/s wr, 23 op/s; 43/234 objects degraded (18.376%) 2024-04-07T16:22:14.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:13 smithi100 bash[76202]: cluster 2024-04-07T16:22:12.803192+0000 mon.a (mon.0) 370 : cluster [DBG] osdmap e383: 8 total, 7 up, 8 in 2024-04-07T16:22:14.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:13 smithi100 bash[76202]: cluster 2024-04-07T16:22:13.315317+0000 mon.a (mon.0) 371 : cluster [WRN] Health check failed: Degraded data redundancy: 43/234 objects degraded (18.376%), 15 pgs degraded (PG_DEGRADED) 2024-04-07T16:22:14.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:13 smithi100 bash[76202]: cluster 2024-04-07T16:22:13.315358+0000 mon.a (mon.0) 372 : cluster [INF] Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 9 pgs inactive, 11 pgs peering) 2024-04-07T16:22:14.800 INFO:tasks.rados.rados.0.smithi100.stdout:1687: done (6 left) 2024-04-07T16:22:14.800 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 45 current snap is 156 2024-04-07T16:22:14.800 INFO:tasks.rados.rados.0.smithi100.stdout:1688: seq_num 522 ranges {3211264=745472,3956736=606208,4562944=196608} 2024-04-07T16:22:14.832 INFO:tasks.rados.rados.0.smithi100.stdout:1688: writing smithi10066867-45 from 3211264 to 3956736 tid 1 2024-04-07T16:22:14.837 INFO:tasks.rados.rados.0.smithi100.stdout:1688: writing smithi10066867-45 from 3956736 to 4562944 tid 2 2024-04-07T16:22:14.839 INFO:tasks.rados.rados.0.smithi100.stdout:1688: writing smithi10066867-45 from 4562944 to 4759552 tid 3 2024-04-07T16:22:14.839 INFO:tasks.rados.rados.0.smithi100.stdout:1689: rollback oid 47 current snap is 156 2024-04-07T16:22:14.839 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 47 to 151 2024-04-07T16:22:14.839 INFO:tasks.rados.rados.0.smithi100.stdout:1680: finishing rollback tid 0 to smithi10066867-39 2024-04-07T16:22:14.840 INFO:tasks.rados.rados.0.smithi100.stdout:1680: finishing rollback tid 1 to smithi10066867-39 2024-04-07T16:22:14.840 INFO:tasks.rados.rados.0.smithi100.stdout:1680: finishing rollback tid 2 to smithi10066867-39 2024-04-07T16:22:14.840 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 39 v 624 (ObjNum 465 snap 142 seq_num 465) dirty exists 2024-04-07T16:22:14.840 INFO:tasks.rados.rados.0.smithi100.stdout:1682: finishing rollback tid 2 to smithi10066867-36 2024-04-07T16:22:14.840 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 36 v 719 (ObjNum 492 snap 148 seq_num 492) dirty exists 2024-04-07T16:22:14.840 INFO:tasks.rados.rados.0.smithi100.stdout:1680: done (7 left) 2024-04-07T16:22:14.840 INFO:tasks.rados.rados.0.smithi100.stdout:1682: done (6 left) 2024-04-07T16:22:14.840 INFO:tasks.rados.rados.0.smithi100.stdout:1690: rollback oid 12 current snap is 156 2024-04-07T16:22:14.840 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 12 to 155 2024-04-07T16:22:14.840 INFO:tasks.rados.rados.0.smithi100.stdout:1679: finishing write tid 2 to smithi10066867-4 2024-04-07T16:22:14.840 INFO:tasks.rados.rados.0.smithi100.stdout:1679: finishing write tid 3 to smithi10066867-4 2024-04-07T16:22:14.840 INFO:tasks.rados.rados.0.smithi100.stdout:1679: finishing write tid 4 to smithi10066867-4 2024-04-07T16:22:14.840 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 8 v 706 (ObjNum 517 snap 153 seq_num 517) dirty exists 2024-04-07T16:22:14.840 INFO:tasks.rados.rados.0.smithi100.stdout:1684: done (6 left) 2024-04-07T16:22:14.841 INFO:tasks.rados.rados.0.smithi100.stdout:1691: read oid 16 snap -1 2024-04-07T16:22:14.841 INFO:tasks.rados.rados.0.smithi100.stdout:1691: expect deleted 2024-04-07T16:22:14.841 INFO:tasks.rados.rados.0.smithi100.stdout:1692: read oid 35 snap -1 2024-04-07T16:22:14.841 INFO:tasks.rados.rados.0.smithi100.stdout:1692: expect (ObjNum 519 snap 155 seq_num 519) 2024-04-07T16:22:14.841 INFO:tasks.rados.rados.0.smithi100.stdout:1693: read oid 33 snap -1 2024-04-07T16:22:14.841 INFO:tasks.rados.rados.0.smithi100.stdout:1693: expect deleted 2024-04-07T16:22:14.841 INFO:tasks.rados.rados.0.smithi100.stdout:1694: delete oid 25 current snap is 156 2024-04-07T16:22:14.841 INFO:tasks.rados.rados.0.smithi100.stdout:1679: finishing write tid 5 to smithi10066867-4 2024-04-07T16:22:14.841 INFO:tasks.rados.rados.0.smithi100.stdout:1679: oid 4 updating version 0 to 1010 2024-04-07T16:22:14.841 INFO:tasks.rados.rados.0.smithi100.stdout:1679: oid 4 updating version 1010 to 1013 2024-04-07T16:22:14.841 INFO:tasks.rados.rados.0.smithi100.stdout:1679: oid 4 version 1013 is already newer than 1011 2024-04-07T16:22:14.841 INFO:tasks.rados.rados.0.smithi100.stdout:1679: oid 4 version 1013 is already newer than 1012 2024-04-07T16:22:14.841 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 4 v 1013 (ObjNum 521 snap 156 seq_num 521) dirty exists 2024-04-07T16:22:14.842 INFO:tasks.rados.rados.0.smithi100.stdout:1679: left oid 4 (ObjNum 521 snap 156 seq_num 521) 2024-04-07T16:22:14.842 INFO:tasks.rados.rados.0.smithi100.stdout:1681: expect (ObjNum 463 snap 141 seq_num 463) 2024-04-07T16:22:15.038 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 50 v 797 (ObjNum 518 snap 153 seq_num 518) dirty exists 2024-04-07T16:22:15.039 INFO:tasks.rados.rados.0.smithi100.stdout:1688: finishing write tid 1 to smithi10066867-45 2024-04-07T16:22:15.039 INFO:tasks.rados.rados.0.smithi100.stdout:1688: finishing write tid 2 to smithi10066867-45 2024-04-07T16:22:15.039 INFO:tasks.rados.rados.0.smithi100.stdout:1688: finishing write tid 3 to smithi10066867-45 2024-04-07T16:22:15.039 INFO:tasks.rados.rados.0.smithi100.stdout:1688: finishing write tid 4 to smithi10066867-45 2024-04-07T16:22:15.039 INFO:tasks.rados.rados.0.smithi100.stdout:1689: finishing rollback tid 0 to smithi10066867-47 2024-04-07T16:22:15.039 INFO:tasks.rados.rados.0.smithi100.stdout:1688: finishing write tid 5 to smithi10066867-45 2024-04-07T16:22:15.039 INFO:tasks.rados.rados.0.smithi100.stdout:1688: oid 45 updating version 0 to 798 2024-04-07T16:22:15.039 INFO:tasks.rados.rados.0.smithi100.stdout:1688: oid 45 updating version 798 to 800 2024-04-07T16:22:15.039 INFO:tasks.rados.rados.0.smithi100.stdout:1688: oid 45 updating version 800 to 801 2024-04-07T16:22:15.039 INFO:tasks.rados.rados.0.smithi100.stdout:1688: oid 45 version 801 is already newer than 799 2024-04-07T16:22:15.039 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 45 v 801 (ObjNum 522 snap 156 seq_num 522) dirty exists 2024-04-07T16:22:15.039 INFO:tasks.rados.rados.0.smithi100.stdout:1688: left oid 45 (ObjNum 522 snap 156 seq_num 522) 2024-04-07T16:22:15.039 INFO:tasks.rados.rados.0.smithi100.stdout:1689: finishing rollback tid 1 to smithi10066867-47 2024-04-07T16:22:15.039 INFO:tasks.rados.rados.0.smithi100.stdout:1689: finishing rollback tid 2 to smithi10066867-47 2024-04-07T16:22:15.040 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 47 v 544 (ObjNum 502 snap 150 seq_num 502) dirty exists 2024-04-07T16:22:15.040 INFO:tasks.rados.rados.0.smithi100.stdout:1690: finishing rollback tid 0 to smithi10066867-12 2024-04-07T16:22:15.040 INFO:tasks.rados.rados.0.smithi100.stdout:1690: finishing rollback tid 1 to smithi10066867-12 2024-04-07T16:22:15.040 INFO:tasks.rados.rados.0.smithi100.stdout:1690: finishing rollback tid 2 to smithi10066867-12 2024-04-07T16:22:15.040 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 12 v 847 (ObjNum 432 snap 130 seq_num 432) dirty exists 2024-04-07T16:22:15.040 INFO:tasks.rados.rados.0.smithi100.stdout:1692: expect (ObjNum 519 snap 155 seq_num 519) 2024-04-07T16:22:15.078 INFO:tasks.rados.rados.0.smithi100.stdout:1679: done (9 left) 2024-04-07T16:22:15.078 INFO:tasks.rados.rados.0.smithi100.stdout:1681: done (8 left) 2024-04-07T16:22:15.078 INFO:tasks.rados.rados.0.smithi100.stdout:1686: done (7 left) 2024-04-07T16:22:15.078 INFO:tasks.rados.rados.0.smithi100.stdout:1688: done (6 left) 2024-04-07T16:22:15.079 INFO:tasks.rados.rados.0.smithi100.stdout:1689: done (5 left) 2024-04-07T16:22:15.079 INFO:tasks.rados.rados.0.smithi100.stdout:1690: done (4 left) 2024-04-07T16:22:15.079 INFO:tasks.rados.rados.0.smithi100.stdout:1691: done (3 left) 2024-04-07T16:22:15.079 INFO:tasks.rados.rados.0.smithi100.stdout:1692: done (2 left) 2024-04-07T16:22:15.079 INFO:tasks.rados.rados.0.smithi100.stdout:1693: done (1 left) 2024-04-07T16:22:15.079 INFO:tasks.rados.rados.0.smithi100.stdout:1694: done (0 left) 2024-04-07T16:22:15.079 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 8 current snap is 156 2024-04-07T16:22:15.079 INFO:tasks.rados.rados.0.smithi100.stdout:1695: seq_num 524 ranges {5505024=786432,6291456=778240,7069696=24576} 2024-04-07T16:22:15.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:14 smithi144 bash[66654]: cluster 2024-04-07T16:22:13.811865+0000 mon.a (mon.0) 373 : cluster [DBG] osdmap e384: 8 total, 7 up, 8 in 2024-04-07T16:22:15.147 INFO:tasks.rados.rados.0.smithi100.stdout:1695: writing smithi10066867-8 from 5505024 to 6291456 tid 1 2024-04-07T16:22:15.153 INFO:tasks.rados.rados.0.smithi100.stdout:1695: writing smithi10066867-8 from 6291456 to 7069696 tid 2 2024-04-07T16:22:15.153 INFO:tasks.rados.rados.0.smithi100.stdout:1695: writing smithi10066867-8 from 7069696 to 7094272 tid 3 2024-04-07T16:22:15.153 INFO:tasks.rados.rados.0.smithi100.stdout:1696: read oid 2 snap -1 2024-04-07T16:22:15.153 INFO:tasks.rados.rados.0.smithi100.stdout:1696: expect (ObjNum 505 snap 150 seq_num 505) 2024-04-07T16:22:15.153 INFO:tasks.rados.rados.0.smithi100.stdout:1697: snap_create 2024-04-07T16:22:15.162 INFO:tasks.rados.rados.0.smithi100.stdout:1695: finishing write tid 1 to smithi10066867-8 2024-04-07T16:22:15.162 INFO:tasks.rados.rados.0.smithi100.stdout:1695: finishing write tid 2 to smithi10066867-8 2024-04-07T16:22:15.166 INFO:tasks.rados.rados.0.smithi100.stdout:1695: finishing write tid 3 to smithi10066867-8 2024-04-07T16:22:15.166 INFO:tasks.rados.rados.0.smithi100.stdout:1695: finishing write tid 4 to smithi10066867-8 2024-04-07T16:22:15.169 INFO:tasks.rados.rados.0.smithi100.stdout:1695: finishing write tid 5 to smithi10066867-8 2024-04-07T16:22:15.169 INFO:tasks.rados.rados.0.smithi100.stdout:1695: oid 8 updating version 0 to 708 2024-04-07T16:22:15.169 INFO:tasks.rados.rados.0.smithi100.stdout:1695: oid 8 updating version 708 to 709 2024-04-07T16:22:15.170 INFO:tasks.rados.rados.0.smithi100.stdout:1695: oid 8 updating version 709 to 710 2024-04-07T16:22:15.170 INFO:tasks.rados.rados.0.smithi100.stdout:1695: oid 8 version 710 is already newer than 707 2024-04-07T16:22:15.170 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 8 v 710 (ObjNum 524 snap 156 seq_num 524) dirty exists 2024-04-07T16:22:15.170 INFO:tasks.rados.rados.0.smithi100.stdout:1695: left oid 8 (ObjNum 524 snap 156 seq_num 524) 2024-04-07T16:22:15.173 INFO:tasks.rados.rados.0.smithi100.stdout:1696: expect (ObjNum 505 snap 150 seq_num 505) 2024-04-07T16:22:15.184 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:14 smithi100 bash[74942]: cluster 2024-04-07T16:22:13.811865+0000 mon.a (mon.0) 373 : cluster [DBG] osdmap e384: 8 total, 7 up, 8 in 2024-04-07T16:22:15.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:14 smithi100 bash[76202]: cluster 2024-04-07T16:22:13.811865+0000 mon.a (mon.0) 373 : cluster [DBG] osdmap e384: 8 total, 7 up, 8 in 2024-04-07T16:22:15.800 INFO:tasks.rados.rados.0.smithi100.stdout:1695: done (2 left) 2024-04-07T16:22:15.800 INFO:tasks.rados.rados.0.smithi100.stdout:1696: done (1 left) 2024-04-07T16:22:15.800 INFO:tasks.rados.rados.0.smithi100.stdout:1697: done (0 left) 2024-04-07T16:22:15.800 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 27 current snap is 157 2024-04-07T16:22:15.800 INFO:tasks.rados.rados.0.smithi100.stdout:1698: seq_num 525 ranges {2301952=802816,3104768=712704,3817472=90112} 2024-04-07T16:22:15.817 INFO:tasks.rados.rados.0.smithi100.stdout:1698: writing smithi10066867-27 from 2301952 to 3104768 tid 1 2024-04-07T16:22:15.823 INFO:tasks.rados.rados.0.smithi100.stdout:1698: writing smithi10066867-27 from 3104768 to 3817472 tid 2 2024-04-07T16:22:15.823 INFO:tasks.rados.rados.0.smithi100.stdout:1698: writing smithi10066867-27 from 3817472 to 3907584 tid 3 2024-04-07T16:22:15.823 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 15 current snap is 157 2024-04-07T16:22:15.823 INFO:tasks.rados.rados.0.smithi100.stdout:1699: seq_num 526 ranges {0=417792,417792=589824,1007616=614400} 2024-04-07T16:22:15.826 INFO:tasks.rados.rados.0.smithi100.stdout:1699: writing smithi10066867-15 from 0 to 417792 tid 1 2024-04-07T16:22:15.831 INFO:tasks.rados.rados.0.smithi100.stdout:1699: writing smithi10066867-15 from 417792 to 1007616 tid 2 2024-04-07T16:22:15.835 INFO:tasks.rados.rados.0.smithi100.stdout:1699: writing smithi10066867-15 from 1007616 to 1622016 tid 3 2024-04-07T16:22:15.835 INFO:tasks.rados.rados.0.smithi100.stdout:1700: read oid 29 snap -1 2024-04-07T16:22:15.835 INFO:tasks.rados.rados.0.smithi100.stdout:1700: expect deleted 2024-04-07T16:22:15.835 INFO:tasks.rados.rados.0.smithi100.stdout:1698: finishing write tid 1 to smithi10066867-27 2024-04-07T16:22:15.836 INFO:tasks.rados.rados.0.smithi100.stdout:1698: finishing write tid 2 to smithi10066867-27 2024-04-07T16:22:15.836 INFO:tasks.rados.rados.0.smithi100.stdout:1698: finishing write tid 3 to smithi10066867-27 2024-04-07T16:22:15.836 INFO:tasks.rados.rados.0.smithi100.stdout:1698: finishing write tid 4 to smithi10066867-27 2024-04-07T16:22:15.836 INFO:tasks.rados.rados.0.smithi100.stdout:1699: finishing write tid 1 to smithi10066867-15 2024-04-07T16:22:15.836 INFO:tasks.rados.rados.0.smithi100.stdout:1698: finishing write tid 5 to smithi10066867-27 2024-04-07T16:22:15.836 INFO:tasks.rados.rados.0.smithi100.stdout:1698: oid 27 updating version 0 to 790 2024-04-07T16:22:15.836 INFO:tasks.rados.rados.0.smithi100.stdout:1698: oid 27 updating version 790 to 791 2024-04-07T16:22:15.836 INFO:tasks.rados.rados.0.smithi100.stdout:1698: oid 27 version 791 is already newer than 789 2024-04-07T16:22:15.836 INFO:tasks.rados.rados.0.smithi100.stdout:1698: oid 27 version 791 is already newer than 788 2024-04-07T16:22:15.836 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 27 v 791 (ObjNum 525 snap 157 seq_num 525) dirty exists 2024-04-07T16:22:15.836 INFO:tasks.rados.rados.0.smithi100.stdout:1698: left oid 27 (ObjNum 525 snap 157 seq_num 525) 2024-04-07T16:22:15.836 INFO:tasks.rados.rados.0.smithi100.stdout:1698: done (2 left) 2024-04-07T16:22:15.836 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 6 current snap is 157 2024-04-07T16:22:15.836 INFO:tasks.rados.rados.0.smithi100.stdout:1701: seq_num 527 ranges {2752512=434176,3186688=475136,3661824=565248,4227072=163840} 2024-04-07T16:22:15.854 INFO:tasks.rados.rados.0.smithi100.stdout:1701: writing smithi10066867-6 from 2752512 to 3186688 tid 1 2024-04-07T16:22:15.856 INFO:tasks.rados.rados.0.smithi100.stdout:1701: writing smithi10066867-6 from 3186688 to 3661824 tid 2 2024-04-07T16:22:15.860 INFO:tasks.rados.rados.0.smithi100.stdout:1701: writing smithi10066867-6 from 3661824 to 4227072 tid 3 2024-04-07T16:22:15.861 INFO:tasks.rados.rados.0.smithi100.stdout:1701: writing smithi10066867-6 from 4227072 to 4390912 tid 4 2024-04-07T16:22:15.861 INFO:tasks.rados.rados.0.smithi100.stdout:1702: read oid 37 snap -1 2024-04-07T16:22:15.861 INFO:tasks.rados.rados.0.smithi100.stdout:1702: expect (ObjNum 0 snap 0 seq_num 0) 2024-04-07T16:22:15.861 INFO:tasks.rados.rados.0.smithi100.stdout:1703: read oid 3 snap -1 2024-04-07T16:22:15.862 INFO:tasks.rados.rados.0.smithi100.stdout:1703: expect (ObjNum 501 snap 150 seq_num 501) 2024-04-07T16:22:15.862 INFO:tasks.rados.rados.0.smithi100.stdout:1704: snap_remove snap 153 2024-04-07T16:22:16.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:15 smithi144 bash[66654]: cluster 2024-04-07T16:22:14.646028+0000 mgr.y (mgr.25175) 122 : cluster [DBG] pgmap v166: 121 pgs: 15 active+undersized+degraded, 37 active+undersized, 69 active+clean; 111 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 19 MiB/s rd, 3.7 MiB/s wr, 23 op/s; 43/234 objects degraded (18.376%) 2024-04-07T16:22:16.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:15 smithi144 bash[66654]: cluster 2024-04-07T16:22:14.817351+0000 mon.a (mon.0) 374 : cluster [DBG] osdmap e385: 8 total, 7 up, 8 in 2024-04-07T16:22:16.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:15 smithi100 bash[76202]: cluster 2024-04-07T16:22:14.646028+0000 mgr.y (mgr.25175) 122 : cluster [DBG] pgmap v166: 121 pgs: 15 active+undersized+degraded, 37 active+undersized, 69 active+clean; 111 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 19 MiB/s rd, 3.7 MiB/s wr, 23 op/s; 43/234 objects degraded (18.376%) 2024-04-07T16:22:16.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:15 smithi100 bash[76202]: cluster 2024-04-07T16:22:14.817351+0000 mon.a (mon.0) 374 : cluster [DBG] osdmap e385: 8 total, 7 up, 8 in 2024-04-07T16:22:16.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:15 smithi100 bash[74942]: cluster 2024-04-07T16:22:14.646028+0000 mgr.y (mgr.25175) 122 : cluster [DBG] pgmap v166: 121 pgs: 15 active+undersized+degraded, 37 active+undersized, 69 active+clean; 111 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 19 MiB/s rd, 3.7 MiB/s wr, 23 op/s; 43/234 objects degraded (18.376%) 2024-04-07T16:22:16.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:15 smithi100 bash[74942]: cluster 2024-04-07T16:22:14.817351+0000 mon.a (mon.0) 374 : cluster [DBG] osdmap e385: 8 total, 7 up, 8 in 2024-04-07T16:22:16.820 INFO:tasks.rados.rados.0.smithi100.stdout:1704: done (5 left) 2024-04-07T16:22:16.820 INFO:tasks.rados.rados.0.smithi100.stdout:1705: snap_create 2024-04-07T16:22:16.820 INFO:tasks.rados.rados.0.smithi100.stdout:1699: finishing write tid 2 to smithi10066867-15 2024-04-07T16:22:16.820 INFO:tasks.rados.rados.0.smithi100.stdout:1699: finishing write tid 3 to smithi10066867-15 2024-04-07T16:22:16.820 INFO:tasks.rados.rados.0.smithi100.stdout:1699: finishing write tid 4 to smithi10066867-15 2024-04-07T16:22:16.820 INFO:tasks.rados.rados.0.smithi100.stdout:1699: finishing write tid 5 to smithi10066867-15 2024-04-07T16:22:16.820 INFO:tasks.rados.rados.0.smithi100.stdout:1699: oid 15 updating version 0 to 1017 2024-04-07T16:22:16.820 INFO:tasks.rados.rados.0.smithi100.stdout:1699: oid 15 updating version 1017 to 1018 2024-04-07T16:22:16.820 INFO:tasks.rados.rados.0.smithi100.stdout:1699: oid 15 updating version 1018 to 1020 2024-04-07T16:22:16.820 INFO:tasks.rados.rados.0.smithi100.stdout:1699: oid 15 version 1020 is already newer than 1019 2024-04-07T16:22:16.820 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 15 v 1020 (ObjNum 526 snap 157 seq_num 526) dirty exists 2024-04-07T16:22:16.821 INFO:tasks.rados.rados.0.smithi100.stdout:1699: left oid 15 (ObjNum 526 snap 157 seq_num 526) 2024-04-07T16:22:16.821 INFO:tasks.rados.rados.0.smithi100.stdout:1702: expect (ObjNum 0 snap 0 seq_num 0) 2024-04-07T16:22:16.821 INFO:tasks.rados.rados.0.smithi100.stdout:1701: finishing write tid 1 to smithi10066867-6 2024-04-07T16:22:16.821 INFO:tasks.rados.rados.0.smithi100.stdout:1701: finishing write tid 2 to smithi10066867-6 2024-04-07T16:22:16.821 INFO:tasks.rados.rados.0.smithi100.stdout:1701: finishing write tid 3 to smithi10066867-6 2024-04-07T16:22:16.821 INFO:tasks.rados.rados.0.smithi100.stdout:1701: finishing write tid 4 to smithi10066867-6 2024-04-07T16:22:16.822 INFO:tasks.rados.rados.0.smithi100.stdout:1701: finishing write tid 5 to smithi10066867-6 2024-04-07T16:22:16.822 INFO:tasks.rados.rados.0.smithi100.stdout:1701: finishing write tid 6 to smithi10066867-6 2024-04-07T16:22:16.822 INFO:tasks.rados.rados.0.smithi100.stdout:1701: oid 6 updating version 0 to 805 2024-04-07T16:22:16.822 INFO:tasks.rados.rados.0.smithi100.stdout:1701: oid 6 version 805 is already newer than 804 2024-04-07T16:22:16.822 INFO:tasks.rados.rados.0.smithi100.stdout:1701: oid 6 version 805 is already newer than 803 2024-04-07T16:22:16.822 INFO:tasks.rados.rados.0.smithi100.stdout:1701: oid 6 version 805 is already newer than 802 2024-04-07T16:22:16.822 INFO:tasks.rados.rados.0.smithi100.stdout:1701: oid 6 version 805 is already newer than 801 2024-04-07T16:22:16.822 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 6 v 805 (ObjNum 527 snap 157 seq_num 527) dirty exists 2024-04-07T16:22:16.822 INFO:tasks.rados.rados.0.smithi100.stdout:1701: left oid 6 (ObjNum 527 snap 157 seq_num 527) 2024-04-07T16:22:16.822 INFO:tasks.rados.rados.0.smithi100.stdout:1703: expect (ObjNum 501 snap 150 seq_num 501) 2024-04-07T16:22:17.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:16 smithi144 bash[66654]: cluster 2024-04-07T16:22:15.817349+0000 mon.a (mon.0) 375 : cluster [DBG] osdmap e386: 8 total, 7 up, 8 in 2024-04-07T16:22:17.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:16 smithi144 bash[66654]: audit 2024-04-07T16:22:16.005135+0000 mon.a (mon.0) 376 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:22:17.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:16 smithi144 bash[66654]: audit 2024-04-07T16:22:16.011667+0000 mon.a (mon.0) 377 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:22:17.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:16 smithi144 bash[66654]: audit 2024-04-07T16:22:16.709948+0000 mon.a (mon.0) 378 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:22:17.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:16 smithi144 bash[66654]: audit 2024-04-07T16:22:16.711547+0000 mon.a (mon.0) 379 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:22:17.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:16 smithi100 bash[74942]: cluster 2024-04-07T16:22:15.817349+0000 mon.a (mon.0) 375 : cluster [DBG] osdmap e386: 8 total, 7 up, 8 in 2024-04-07T16:22:17.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:16 smithi100 bash[74942]: audit 2024-04-07T16:22:16.005135+0000 mon.a (mon.0) 376 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:22:17.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:16 smithi100 bash[74942]: audit 2024-04-07T16:22:16.011667+0000 mon.a (mon.0) 377 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:22:17.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:16 smithi100 bash[74942]: audit 2024-04-07T16:22:16.709948+0000 mon.a (mon.0) 378 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:22:17.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:16 smithi100 bash[74942]: audit 2024-04-07T16:22:16.711547+0000 mon.a (mon.0) 379 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:22:17.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:16 smithi100 bash[76202]: cluster 2024-04-07T16:22:15.817349+0000 mon.a (mon.0) 375 : cluster [DBG] osdmap e386: 8 total, 7 up, 8 in 2024-04-07T16:22:17.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:16 smithi100 bash[76202]: audit 2024-04-07T16:22:16.005135+0000 mon.a (mon.0) 376 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:22:17.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:16 smithi100 bash[76202]: audit 2024-04-07T16:22:16.011667+0000 mon.a (mon.0) 377 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:22:17.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:16 smithi100 bash[76202]: audit 2024-04-07T16:22:16.709948+0000 mon.a (mon.0) 378 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:22:17.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:16 smithi100 bash[76202]: audit 2024-04-07T16:22:16.711547+0000 mon.a (mon.0) 379 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:22:17.685 INFO:journalctl@ceph.osd.0.smithi100.stdout:Apr 07 16:22:17 smithi100 bash[80932]: debug 2024-04-07T16:22:17.432+0000 7f9937c747c0 -1 osd.0 376 log_to_monitors true 2024-04-07T16:22:17.825 INFO:tasks.rados.rados.0.smithi100.stdout:1699: done (5 left) 2024-04-07T16:22:17.826 INFO:tasks.rados.rados.0.smithi100.stdout:1700: done (4 left) 2024-04-07T16:22:17.826 INFO:tasks.rados.rados.0.smithi100.stdout:1701: done (3 left) 2024-04-07T16:22:17.826 INFO:tasks.rados.rados.0.smithi100.stdout:1702: done (2 left) 2024-04-07T16:22:17.826 INFO:tasks.rados.rados.0.smithi100.stdout:1703: done (1 left) 2024-04-07T16:22:17.826 INFO:tasks.rados.rados.0.smithi100.stdout:1705: done (0 left) 2024-04-07T16:22:17.826 INFO:tasks.rados.rados.0.smithi100.stdout:1706: snap_create 2024-04-07T16:22:18.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:17 smithi144 bash[66654]: cluster 2024-04-07T16:22:16.646816+0000 mgr.y (mgr.25175) 123 : cluster [DBG] pgmap v169: 121 pgs: 15 active+undersized+degraded, 37 active+undersized, 69 active+clean; 111 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 43/234 objects degraded (18.376%) 2024-04-07T16:22:18.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:17 smithi144 bash[66654]: cluster 2024-04-07T16:22:16.837479+0000 mon.a (mon.0) 380 : cluster [DBG] osdmap e387: 8 total, 7 up, 8 in 2024-04-07T16:22:18.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:17 smithi144 bash[66654]: audit 2024-04-07T16:22:17.068127+0000 mon.a (mon.0) 381 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:22:18.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:17 smithi144 bash[66654]: audit 2024-04-07T16:22:17.077968+0000 mon.a (mon.0) 382 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:22:18.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:17 smithi144 bash[66654]: audit 2024-04-07T16:22:17.441869+0000 mon.c (mon.2) 4 : audit [INF] from='osd.0 [v2:172.21.15.100:6802/165581137,v1:172.21.15.100:6803/165581137]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-07T16:22:18.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:17 smithi144 bash[66654]: audit 2024-04-07T16:22:17.442158+0000 mon.a (mon.0) 383 : audit [INF] from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-07T16:22:18.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:17 smithi100 bash[76202]: cluster 2024-04-07T16:22:16.646816+0000 mgr.y (mgr.25175) 123 : cluster [DBG] pgmap v169: 121 pgs: 15 active+undersized+degraded, 37 active+undersized, 69 active+clean; 111 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 43/234 objects degraded (18.376%) 2024-04-07T16:22:18.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:17 smithi100 bash[76202]: cluster 2024-04-07T16:22:16.837479+0000 mon.a (mon.0) 380 : cluster [DBG] osdmap e387: 8 total, 7 up, 8 in 2024-04-07T16:22:18.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:17 smithi100 bash[76202]: audit 2024-04-07T16:22:17.068127+0000 mon.a (mon.0) 381 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:22:18.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:17 smithi100 bash[76202]: audit 2024-04-07T16:22:17.077968+0000 mon.a (mon.0) 382 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:22:18.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:17 smithi100 bash[76202]: audit 2024-04-07T16:22:17.441869+0000 mon.c (mon.2) 4 : audit [INF] from='osd.0 [v2:172.21.15.100:6802/165581137,v1:172.21.15.100:6803/165581137]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-07T16:22:18.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:17 smithi100 bash[76202]: audit 2024-04-07T16:22:17.442158+0000 mon.a (mon.0) 383 : audit [INF] from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-07T16:22:18.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:17 smithi100 bash[74942]: cluster 2024-04-07T16:22:16.646816+0000 mgr.y (mgr.25175) 123 : cluster [DBG] pgmap v169: 121 pgs: 15 active+undersized+degraded, 37 active+undersized, 69 active+clean; 111 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 43/234 objects degraded (18.376%) 2024-04-07T16:22:18.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:17 smithi100 bash[74942]: cluster 2024-04-07T16:22:16.837479+0000 mon.a (mon.0) 380 : cluster [DBG] osdmap e387: 8 total, 7 up, 8 in 2024-04-07T16:22:18.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:17 smithi100 bash[74942]: audit 2024-04-07T16:22:17.068127+0000 mon.a (mon.0) 381 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:22:18.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:17 smithi100 bash[74942]: audit 2024-04-07T16:22:17.077968+0000 mon.a (mon.0) 382 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:22:18.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:17 smithi100 bash[74942]: audit 2024-04-07T16:22:17.441869+0000 mon.c (mon.2) 4 : audit [INF] from='osd.0 [v2:172.21.15.100:6802/165581137,v1:172.21.15.100:6803/165581137]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-07T16:22:18.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:17 smithi100 bash[74942]: audit 2024-04-07T16:22:17.442158+0000 mon.a (mon.0) 383 : audit [INF] from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-07T16:22:18.685 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:22:18 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:22:18] "GET /metrics HTTP/1.1" 200 36206 "" "Prometheus/2.43.0" 2024-04-07T16:22:18.685 INFO:journalctl@ceph.osd.0.smithi100.stdout:Apr 07 16:22:18 smithi100 bash[80932]: debug 2024-04-07T16:22:18.400+0000 7f99250c4700 -1 osd.0 376 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-07T16:22:18.829 INFO:tasks.rados.rados.0.smithi100.stdout:1706: done (0 left) 2024-04-07T16:22:18.829 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 40 current snap is 159 2024-04-07T16:22:18.829 INFO:tasks.rados.rados.0.smithi100.stdout:1707: seq_num 528 ranges {1761280=450560,2211840=761856,2973696=442368} 2024-04-07T16:22:18.843 INFO:tasks.rados.rados.0.smithi100.stdout:1707: writing smithi10066867-40 from 1761280 to 2211840 tid 1 2024-04-07T16:22:18.849 INFO:tasks.rados.rados.0.smithi100.stdout:1707: writing smithi10066867-40 from 2211840 to 2973696 tid 2 2024-04-07T16:22:18.853 INFO:tasks.rados.rados.0.smithi100.stdout:1707: writing smithi10066867-40 from 2973696 to 3416064 tid 3 2024-04-07T16:22:18.853 INFO:tasks.rados.rados.0.smithi100.stdout:1708: snap_create 2024-04-07T16:22:18.870 INFO:tasks.rados.rados.0.smithi100.stdout:1707: finishing write tid 1 to smithi10066867-40 2024-04-07T16:22:18.871 INFO:tasks.rados.rados.0.smithi100.stdout:1707: finishing write tid 2 to smithi10066867-40 2024-04-07T16:22:18.873 INFO:tasks.rados.rados.0.smithi100.stdout:1707: finishing write tid 3 to smithi10066867-40 2024-04-07T16:22:18.874 INFO:tasks.rados.rados.0.smithi100.stdout:1707: finishing write tid 4 to smithi10066867-40 2024-04-07T16:22:18.876 INFO:tasks.rados.rados.0.smithi100.stdout:1707: finishing write tid 5 to smithi10066867-40 2024-04-07T16:22:18.876 INFO:tasks.rados.rados.0.smithi100.stdout:1707: oid 40 updating version 0 to 809 2024-04-07T16:22:18.877 INFO:tasks.rados.rados.0.smithi100.stdout:1707: oid 40 updating version 809 to 810 2024-04-07T16:22:18.877 INFO:tasks.rados.rados.0.smithi100.stdout:1707: oid 40 updating version 810 to 812 2024-04-07T16:22:18.877 INFO:tasks.rados.rados.0.smithi100.stdout:1707: oid 40 version 812 is already newer than 811 2024-04-07T16:22:18.877 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 40 v 812 (ObjNum 528 snap 159 seq_num 528) dirty exists 2024-04-07T16:22:18.877 INFO:tasks.rados.rados.0.smithi100.stdout:1707: left oid 40 (ObjNum 528 snap 159 seq_num 528) 2024-04-07T16:22:19.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:18 smithi144 bash[66654]: audit 2024-04-07T16:22:17.834742+0000 mon.a (mon.0) 384 : audit [INF] from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-07T16:22:19.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:18 smithi144 bash[66654]: cluster 2024-04-07T16:22:17.841976+0000 mon.a (mon.0) 385 : cluster [DBG] osdmap e388: 8 total, 7 up, 8 in 2024-04-07T16:22:19.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:18 smithi144 bash[66654]: audit 2024-04-07T16:22:17.851644+0000 mon.c (mon.2) 5 : audit [INF] from='osd.0 [v2:172.21.15.100:6802/165581137,v1:172.21.15.100:6803/165581137]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-04-07T16:22:19.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:18 smithi144 bash[66654]: audit 2024-04-07T16:22:17.853291+0000 mon.a (mon.0) 386 : audit [INF] from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-04-07T16:22:19.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:18 smithi144 bash[66654]: audit 2024-04-07T16:22:18.381480+0000 mon.a (mon.0) 387 : audit [INF] from='osd.0 ' entity='osd.0' 2024-04-07T16:22:19.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:18 smithi144 bash[66654]: cluster 2024-04-07T16:22:18.835842+0000 mon.a (mon.0) 388 : cluster [INF] Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-07T16:22:19.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:18 smithi144 bash[66654]: cluster 2024-04-07T16:22:18.844169+0000 mon.a (mon.0) 389 : cluster [INF] osd.0 [v2:172.21.15.100:6802/165581137,v1:172.21.15.100:6803/165581137] boot 2024-04-07T16:22:19.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:18 smithi144 bash[66654]: cluster 2024-04-07T16:22:18.844323+0000 mon.a (mon.0) 390 : cluster [DBG] osdmap e389: 8 total, 8 up, 8 in 2024-04-07T16:22:19.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:18 smithi100 bash[74942]: audit 2024-04-07T16:22:17.834742+0000 mon.a (mon.0) 384 : audit [INF] from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-07T16:22:19.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:18 smithi100 bash[74942]: cluster 2024-04-07T16:22:17.841976+0000 mon.a (mon.0) 385 : cluster [DBG] osdmap e388: 8 total, 7 up, 8 in 2024-04-07T16:22:19.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:18 smithi100 bash[74942]: audit 2024-04-07T16:22:17.851644+0000 mon.c (mon.2) 5 : audit [INF] from='osd.0 [v2:172.21.15.100:6802/165581137,v1:172.21.15.100:6803/165581137]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-04-07T16:22:19.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:18 smithi100 bash[74942]: audit 2024-04-07T16:22:17.853291+0000 mon.a (mon.0) 386 : audit [INF] from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-04-07T16:22:19.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:18 smithi100 bash[74942]: audit 2024-04-07T16:22:18.381480+0000 mon.a (mon.0) 387 : audit [INF] from='osd.0 ' entity='osd.0' 2024-04-07T16:22:19.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:18 smithi100 bash[74942]: cluster 2024-04-07T16:22:18.835842+0000 mon.a (mon.0) 388 : cluster [INF] Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-07T16:22:19.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:18 smithi100 bash[74942]: cluster 2024-04-07T16:22:18.844169+0000 mon.a (mon.0) 389 : cluster [INF] osd.0 [v2:172.21.15.100:6802/165581137,v1:172.21.15.100:6803/165581137] boot 2024-04-07T16:22:19.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:18 smithi100 bash[74942]: cluster 2024-04-07T16:22:18.844323+0000 mon.a (mon.0) 390 : cluster [DBG] osdmap e389: 8 total, 8 up, 8 in 2024-04-07T16:22:19.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:18 smithi100 bash[76202]: audit 2024-04-07T16:22:17.834742+0000 mon.a (mon.0) 384 : audit [INF] from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-07T16:22:19.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:18 smithi100 bash[76202]: cluster 2024-04-07T16:22:17.841976+0000 mon.a (mon.0) 385 : cluster [DBG] osdmap e388: 8 total, 7 up, 8 in 2024-04-07T16:22:19.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:18 smithi100 bash[76202]: audit 2024-04-07T16:22:17.851644+0000 mon.c (mon.2) 5 : audit [INF] from='osd.0 [v2:172.21.15.100:6802/165581137,v1:172.21.15.100:6803/165581137]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-04-07T16:22:19.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:18 smithi100 bash[76202]: audit 2024-04-07T16:22:17.853291+0000 mon.a (mon.0) 386 : audit [INF] from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-04-07T16:22:19.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:18 smithi100 bash[76202]: audit 2024-04-07T16:22:18.381480+0000 mon.a (mon.0) 387 : audit [INF] from='osd.0 ' entity='osd.0' 2024-04-07T16:22:19.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:18 smithi100 bash[76202]: cluster 2024-04-07T16:22:18.835842+0000 mon.a (mon.0) 388 : cluster [INF] Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-07T16:22:19.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:18 smithi100 bash[76202]: cluster 2024-04-07T16:22:18.844169+0000 mon.a (mon.0) 389 : cluster [INF] osd.0 [v2:172.21.15.100:6802/165581137,v1:172.21.15.100:6803/165581137] boot 2024-04-07T16:22:19.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:18 smithi100 bash[76202]: cluster 2024-04-07T16:22:18.844323+0000 mon.a (mon.0) 390 : cluster [DBG] osdmap e389: 8 total, 8 up, 8 in 2024-04-07T16:22:19.850 INFO:tasks.rados.rados.0.smithi100.stdout:1707: done (1 left) 2024-04-07T16:22:19.850 INFO:tasks.rados.rados.0.smithi100.stdout:1708: done (0 left) 2024-04-07T16:22:19.850 INFO:tasks.rados.rados.0.smithi100.stdout:1709: copy_from oid 50 from oid 10 current snap is 160 2024-04-07T16:22:19.850 INFO:tasks.rados.rados.0.smithi100.stdout:1710: snap_remove snap 156 2024-04-07T16:22:20.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:19 smithi144 bash[66654]: cluster 2024-04-07T16:22:18.647834+0000 mgr.y (mgr.25175) 124 : cluster [DBG] pgmap v172: 121 pgs: 15 active+undersized+degraded, 37 active+undersized, 69 active+clean; 122 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 6.2 MiB/s rd, 2.6 MiB/s wr, 26 op/s; 50/270 objects degraded (18.519%) 2024-04-07T16:22:20.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:19 smithi144 bash[66654]: audit 2024-04-07T16:22:18.845101+0000 mon.a (mon.0) 391 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-07T16:22:20.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:19 smithi144 bash[66654]: cluster 2024-04-07T16:22:19.378695+0000 mon.a (mon.0) 392 : cluster [WRN] Health check update: Degraded data redundancy: 50/270 objects degraded (18.519%), 15 pgs degraded (PG_DEGRADED) 2024-04-07T16:22:20.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:19 smithi100 bash[74942]: cluster 2024-04-07T16:22:18.647834+0000 mgr.y (mgr.25175) 124 : cluster [DBG] pgmap v172: 121 pgs: 15 active+undersized+degraded, 37 active+undersized, 69 active+clean; 122 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 6.2 MiB/s rd, 2.6 MiB/s wr, 26 op/s; 50/270 objects degraded (18.519%) 2024-04-07T16:22:20.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:19 smithi100 bash[74942]: audit 2024-04-07T16:22:18.845101+0000 mon.a (mon.0) 391 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-07T16:22:20.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:19 smithi100 bash[74942]: cluster 2024-04-07T16:22:19.378695+0000 mon.a (mon.0) 392 : cluster [WRN] Health check update: Degraded data redundancy: 50/270 objects degraded (18.519%), 15 pgs degraded (PG_DEGRADED) 2024-04-07T16:22:20.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:19 smithi100 bash[76202]: cluster 2024-04-07T16:22:18.647834+0000 mgr.y (mgr.25175) 124 : cluster [DBG] pgmap v172: 121 pgs: 15 active+undersized+degraded, 37 active+undersized, 69 active+clean; 122 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 6.2 MiB/s rd, 2.6 MiB/s wr, 26 op/s; 50/270 objects degraded (18.519%) 2024-04-07T16:22:20.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:19 smithi100 bash[76202]: audit 2024-04-07T16:22:18.845101+0000 mon.a (mon.0) 391 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-07T16:22:20.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:19 smithi100 bash[76202]: cluster 2024-04-07T16:22:19.378695+0000 mon.a (mon.0) 392 : cluster [WRN] Health check update: Degraded data redundancy: 50/270 objects degraded (18.519%), 15 pgs degraded (PG_DEGRADED) 2024-04-07T16:22:20.860 INFO:tasks.rados.rados.0.smithi100.stdout:1709: finishing copy_from racing read to smithi10066867-50 2024-04-07T16:22:20.860 INFO:tasks.rados.rados.0.smithi100.stdout:1709: finishing copy_from to smithi10066867-50 2024-04-07T16:22:20.860 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 50 v 980 (ObjNum 49 snap 0 seq_num 1953066355) dirty exists 2024-04-07T16:22:20.860 INFO:tasks.rados.rados.0.smithi100.stdout:1709: done (1 left) 2024-04-07T16:22:20.860 INFO:tasks.rados.rados.0.smithi100.stdout:1710: done (0 left) 2024-04-07T16:22:20.860 INFO:tasks.rados.rados.0.smithi100.stdout:1711: copy_from oid 29 from oid 44 current snap is 160 2024-04-07T16:22:20.861 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 7 current snap is 160 2024-04-07T16:22:20.861 INFO:tasks.rados.rados.0.smithi100.stdout:1712: seq_num 529 ranges {1736704=466944,2203648=647168,2850816=532480,3383296=24576} 2024-04-07T16:22:20.873 INFO:tasks.rados.rados.0.smithi100.stdout:1712: writing smithi10066867-7 from 1736704 to 2203648 tid 1 2024-04-07T16:22:20.878 INFO:tasks.rados.rados.0.smithi100.stdout:1712: writing smithi10066867-7 from 2203648 to 2850816 tid 2 2024-04-07T16:22:20.882 INFO:tasks.rados.rados.0.smithi100.stdout:1712: writing smithi10066867-7 from 2850816 to 3383296 tid 3 2024-04-07T16:22:20.882 INFO:tasks.rados.rados.0.smithi100.stdout:1712: writing smithi10066867-7 from 3383296 to 3407872 tid 4 2024-04-07T16:22:20.882 INFO:tasks.rados.rados.0.smithi100.stdout:1713: snap_create 2024-04-07T16:22:20.889 INFO:tasks.rados.rados.0.smithi100.stdout:1712: finishing write tid 1 to smithi10066867-7 2024-04-07T16:22:20.889 INFO:tasks.rados.rados.0.smithi100.stdout:1712: finishing write tid 2 to smithi10066867-7 2024-04-07T16:22:20.893 INFO:tasks.rados.rados.0.smithi100.stdout:1712: finishing write tid 3 to smithi10066867-7 2024-04-07T16:22:20.893 INFO:tasks.rados.rados.0.smithi100.stdout:1712: finishing write tid 4 to smithi10066867-7 2024-04-07T16:22:20.895 INFO:tasks.rados.rados.0.smithi100.stdout:1712: finishing write tid 5 to smithi10066867-7 2024-04-07T16:22:20.897 INFO:tasks.rados.rados.0.smithi100.stdout:1712: finishing write tid 6 to smithi10066867-7 2024-04-07T16:22:20.897 INFO:tasks.rados.rados.0.smithi100.stdout:1712: oid 7 updating version 0 to 792 2024-04-07T16:22:20.897 INFO:tasks.rados.rados.0.smithi100.stdout:1712: oid 7 updating version 792 to 796 2024-04-07T16:22:20.897 INFO:tasks.rados.rados.0.smithi100.stdout:1712: oid 7 version 796 is already newer than 794 2024-04-07T16:22:20.897 INFO:tasks.rados.rados.0.smithi100.stdout:1712: oid 7 version 796 is already newer than 793 2024-04-07T16:22:20.897 INFO:tasks.rados.rados.0.smithi100.stdout:1712: oid 7 version 796 is already newer than 795 2024-04-07T16:22:20.897 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 7 v 796 (ObjNum 529 snap 160 seq_num 529) dirty exists 2024-04-07T16:22:20.898 INFO:tasks.rados.rados.0.smithi100.stdout:1712: left oid 7 (ObjNum 529 snap 160 seq_num 529) 2024-04-07T16:22:20.914 INFO:tasks.rados.rados.0.smithi100.stdout:1711: finishing copy_from racing read to smithi10066867-29 2024-04-07T16:22:20.919 INFO:tasks.rados.rados.0.smithi100.stdout:1711: finishing copy_from to smithi10066867-29 2024-04-07T16:22:20.919 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 29 v 798 (ObjNum 509 snap 152 seq_num 509) dirty exists 2024-04-07T16:22:21.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:20 smithi100 bash[76202]: cluster 2024-04-07T16:22:19.866948+0000 mon.a (mon.0) 393 : cluster [DBG] osdmap e390: 8 total, 8 up, 8 in 2024-04-07T16:22:21.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:20 smithi100 bash[74942]: cluster 2024-04-07T16:22:19.866948+0000 mon.a (mon.0) 393 : cluster [DBG] osdmap e390: 8 total, 8 up, 8 in 2024-04-07T16:22:21.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:20 smithi144 bash[66654]: cluster 2024-04-07T16:22:19.866948+0000 mon.a (mon.0) 393 : cluster [DBG] osdmap e390: 8 total, 8 up, 8 in 2024-04-07T16:22:21.886 INFO:tasks.rados.rados.0.smithi100.stdout:1711: done (2 left) 2024-04-07T16:22:21.886 INFO:tasks.rados.rados.0.smithi100.stdout:1712: done (1 left) 2024-04-07T16:22:21.886 INFO:tasks.rados.rados.0.smithi100.stdout:1713: done (0 left) 2024-04-07T16:22:21.887 INFO:tasks.rados.rados.0.smithi100.stdout:1714: read oid 9 snap -1 2024-04-07T16:22:21.887 INFO:tasks.rados.rados.0.smithi100.stdout:1714: expect (ObjNum 481 snap 144 seq_num 481) 2024-04-07T16:22:21.887 INFO:tasks.rados.rados.0.smithi100.stdout:1715: setattr oid 15 current snap is 161 2024-04-07T16:22:21.890 INFO:tasks.rados.rados.0.smithi100.stdout:1716: snap_remove snap 157 2024-04-07T16:22:22.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:21 smithi100 bash[76202]: cluster 2024-04-07T16:22:20.648770+0000 mgr.y (mgr.25175) 125 : cluster [DBG] pgmap v175: 121 pgs: 15 active+undersized+degraded, 37 active+undersized, 69 active+clean; 122 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 6.2 MiB/s rd, 2.6 MiB/s wr, 26 op/s; 50/270 objects degraded (18.519%) 2024-04-07T16:22:22.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:21 smithi100 bash[76202]: cluster 2024-04-07T16:22:20.874724+0000 mon.a (mon.0) 394 : cluster [DBG] osdmap e391: 8 total, 8 up, 8 in 2024-04-07T16:22:22.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:21 smithi100 bash[74942]: cluster 2024-04-07T16:22:20.648770+0000 mgr.y (mgr.25175) 125 : cluster [DBG] pgmap v175: 121 pgs: 15 active+undersized+degraded, 37 active+undersized, 69 active+clean; 122 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 6.2 MiB/s rd, 2.6 MiB/s wr, 26 op/s; 50/270 objects degraded (18.519%) 2024-04-07T16:22:22.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:21 smithi100 bash[74942]: cluster 2024-04-07T16:22:20.874724+0000 mon.a (mon.0) 394 : cluster [DBG] osdmap e391: 8 total, 8 up, 8 in 2024-04-07T16:22:22.305 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:21 smithi144 bash[66654]: cluster 2024-04-07T16:22:20.648770+0000 mgr.y (mgr.25175) 125 : cluster [DBG] pgmap v175: 121 pgs: 15 active+undersized+degraded, 37 active+undersized, 69 active+clean; 122 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 6.2 MiB/s rd, 2.6 MiB/s wr, 26 op/s; 50/270 objects degraded (18.519%) 2024-04-07T16:22:22.305 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:21 smithi144 bash[66654]: cluster 2024-04-07T16:22:20.874724+0000 mon.a (mon.0) 394 : cluster [DBG] osdmap e391: 8 total, 8 up, 8 in 2024-04-07T16:22:22.606 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:22:22 smithi144 bash[59794]: ts=2024-04-07T16:22:22.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:22:22.897 INFO:tasks.rados.rados.0.smithi100.stdout:1716: done (2 left) 2024-04-07T16:22:22.898 INFO:tasks.rados.rados.0.smithi100.stdout:1717: snap_remove snap 158 2024-04-07T16:22:23.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:22 smithi100 bash[74942]: cluster 2024-04-07T16:22:21.903675+0000 mon.a (mon.0) 395 : cluster [DBG] osdmap e392: 8 total, 8 up, 8 in 2024-04-07T16:22:23.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:22 smithi100 bash[76202]: cluster 2024-04-07T16:22:21.903675+0000 mon.a (mon.0) 395 : cluster [DBG] osdmap e392: 8 total, 8 up, 8 in 2024-04-07T16:22:23.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:22 smithi144 bash[66654]: cluster 2024-04-07T16:22:21.903675+0000 mon.a (mon.0) 395 : cluster [DBG] osdmap e392: 8 total, 8 up, 8 in 2024-04-07T16:22:23.856 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:22:23 smithi144 bash[59794]: ts=2024-04-07T16:22:23.509Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=nodes name=CephNodeDiskspaceWarning index=4 msg="Evaluating rule failed" rule="alert: CephNodeDiskspaceWarning\nexpr: predict_linear(node_filesystem_free_bytes{device=~\"/.*\"}[2d], 3600 * 24 * 5)\n * on (instance) group_left (nodename) node_uname_info < 0\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.8.4\n severity: warning\n type: ceph_default\nannotations:\n description: Mountpoint {{ $labels.mountpoint }} on {{ $labels.nodename }} will\n be full in less than 5 days based on the 48 hour trailing fill rate.\n summary: Host filesystem free space is getting low\n" err="found duplicate series for the match group {instance=\"smithi100\"} on the right hand-side of the operation: [{__name__=\"node_uname_info\", domainname=\"(none)\", instance=\"smithi100\", job=\"node\", machine=\"x86_64\", nodename=\"smithi100\", release=\"5.15.0-101-generic\", sysname=\"Linux\", version=\"#111-Ubuntu SMP Tue Mar 5 20:16:58 UTC 2024\"}, {__name__=\"node_uname_info\", cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", domainname=\"(none)\", instance=\"smithi100\", job=\"node\", machine=\"x86_64\", nodename=\"smithi100\", release=\"5.15.0-101-generic\", sysname=\"Linux\", version=\"#111-Ubuntu SMP Tue Mar 5 20:16:58 UTC 2024\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:22:23.905 INFO:tasks.rados.rados.0.smithi100.stdout:1717: done (2 left) 2024-04-07T16:22:23.905 INFO:tasks.rados.rados.0.smithi100.stdout:1718: copy_from oid 7 from oid 33 current snap is 161 2024-04-07T16:22:23.905 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 15 v 1028 (ObjNum 526 snap 157 seq_num 526) dirty exists 2024-04-07T16:22:23.906 INFO:tasks.rados.rados.0.smithi100.stdout:1714: expect (ObjNum 481 snap 144 seq_num 481) 2024-04-07T16:22:23.955 INFO:tasks.rados.rados.0.smithi100.stdout:1718: finishing copy_from to smithi10066867-7 2024-04-07T16:22:23.955 INFO:tasks.rados.rados.0.smithi100.stdout:1718: got expected ENOENT (src dne) 2024-04-07T16:22:23.955 INFO:tasks.rados.rados.0.smithi100.stdout:1718: finishing copy_from racing read to smithi10066867-7 2024-04-07T16:22:23.955 INFO:tasks.rados.rados.0.smithi100.stdout:1714: done (2 left) 2024-04-07T16:22:23.955 INFO:tasks.rados.rados.0.smithi100.stdout:1715: done (1 left) 2024-04-07T16:22:23.955 INFO:tasks.rados.rados.0.smithi100.stdout:1718: done (0 left) 2024-04-07T16:22:23.956 INFO:tasks.rados.rados.0.smithi100.stdout:1719: rollback oid 48 current snap is 161 2024-04-07T16:22:23.956 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 48 to 159 2024-04-07T16:22:23.956 INFO:tasks.rados.rados.0.smithi100.stdout:1720: read oid 17 snap -1 2024-04-07T16:22:23.956 INFO:tasks.rados.rados.0.smithi100.stdout:1720: expect (ObjNum 508 snap 151 seq_num 508) 2024-04-07T16:22:23.956 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 3 current snap is 161 2024-04-07T16:22:23.956 INFO:tasks.rados.rados.0.smithi100.stdout:1721: seq_num 530 ranges {1196032=483328,1679360=532480,2211840=671744} 2024-04-07T16:22:23.969 INFO:tasks.rados.rados.0.smithi100.stdout:1721: writing smithi10066867-3 from 1196032 to 1679360 tid 1 2024-04-07T16:22:23.974 INFO:tasks.rados.rados.0.smithi100.stdout:1721: writing smithi10066867-3 from 1679360 to 2211840 tid 2 2024-04-07T16:22:23.980 INFO:tasks.rados.rados.0.smithi100.stdout:1721: writing smithi10066867-3 from 2211840 to 2883584 tid 3 2024-04-07T16:22:23.980 INFO:tasks.rados.rados.0.smithi100.stdout:1722: copy_from oid 16 from oid 13 current snap is 161 2024-04-07T16:22:23.980 INFO:tasks.rados.rados.0.smithi100.stdout:1719: finishing rollback tid 0 to smithi10066867-48 2024-04-07T16:22:23.980 INFO:tasks.rados.rados.0.smithi100.stdout:1719: finishing rollback tid 1 to smithi10066867-48 2024-04-07T16:22:23.980 INFO:tasks.rados.rados.0.smithi100.stdout:1719: finishing rollback tid 2 to smithi10066867-48 2024-04-07T16:22:23.980 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 48 v 995 (ObjNum 510 snap 152 seq_num 510) dirty exists 2024-04-07T16:22:23.980 INFO:tasks.rados.rados.0.smithi100.stdout:1720: expect (ObjNum 508 snap 151 seq_num 508) 2024-04-07T16:22:24.058 INFO:tasks.rados.rados.0.smithi100.stdout:1721: finishing write tid 1 to smithi10066867-3 2024-04-07T16:22:24.059 INFO:tasks.rados.rados.0.smithi100.stdout:1721: finishing write tid 2 to smithi10066867-3 2024-04-07T16:22:24.059 INFO:tasks.rados.rados.0.smithi100.stdout:1721: finishing write tid 3 to smithi10066867-3 2024-04-07T16:22:24.059 INFO:tasks.rados.rados.0.smithi100.stdout:1721: finishing write tid 4 to smithi10066867-3 2024-04-07T16:22:24.059 INFO:tasks.rados.rados.0.smithi100.stdout:1721: finishing write tid 5 to smithi10066867-3 2024-04-07T16:22:24.059 INFO:tasks.rados.rados.0.smithi100.stdout:1721: oid 3 updating version 0 to 1006 2024-04-07T16:22:24.059 INFO:tasks.rados.rados.0.smithi100.stdout:1721: oid 3 version 1006 is already newer than 1005 2024-04-07T16:22:24.059 INFO:tasks.rados.rados.0.smithi100.stdout:1721: oid 3 updating version 1006 to 1007 2024-04-07T16:22:24.059 INFO:tasks.rados.rados.0.smithi100.stdout:1721: oid 3 updating version 1007 to 1008 2024-04-07T16:22:24.059 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 3 v 1008 (ObjNum 530 snap 161 seq_num 530) dirty exists 2024-04-07T16:22:24.059 INFO:tasks.rados.rados.0.smithi100.stdout:1721: left oid 3 (ObjNum 530 snap 161 seq_num 530) 2024-04-07T16:22:24.059 INFO:tasks.rados.rados.0.smithi100.stdout:1722: finishing copy_from racing read to smithi10066867-16 2024-04-07T16:22:24.059 INFO:tasks.rados.rados.0.smithi100.stdout:1722: finishing copy_from to smithi10066867-16 2024-04-07T16:22:24.059 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 16 v 697 (ObjNum 475 snap 143 seq_num 475) dirty exists 2024-04-07T16:22:24.059 INFO:tasks.rados.rados.0.smithi100.stdout:1719: done (3 left) 2024-04-07T16:22:24.060 INFO:tasks.rados.rados.0.smithi100.stdout:1720: done (2 left) 2024-04-07T16:22:24.060 INFO:tasks.rados.rados.0.smithi100.stdout:1721: done (1 left) 2024-04-07T16:22:24.060 INFO:tasks.rados.rados.0.smithi100.stdout:1722: done (0 left) 2024-04-07T16:22:24.060 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 46 current snap is 161 2024-04-07T16:22:24.060 INFO:tasks.rados.rados.0.smithi100.stdout:1723: seq_num 531 ranges {4399104=499712,4898816=417792,5316608=548864,5865472=237568} 2024-04-07T16:22:24.119 INFO:tasks.rados.rados.0.smithi100.stdout:1723: writing smithi10066867-46 from 4399104 to 4898816 tid 1 2024-04-07T16:22:24.123 INFO:tasks.rados.rados.0.smithi100.stdout:1723: writing smithi10066867-46 from 4898816 to 5316608 tid 2 2024-04-07T16:22:24.128 INFO:tasks.rados.rados.0.smithi100.stdout:1723: writing smithi10066867-46 from 5316608 to 5865472 tid 3 2024-04-07T16:22:24.130 INFO:tasks.rados.rados.0.smithi100.stdout:1723: writing smithi10066867-46 from 5865472 to 6103040 tid 4 2024-04-07T16:22:24.130 INFO:tasks.rados.rados.0.smithi100.stdout:1724: copy_from oid 18 from oid 20 current snap is 161 2024-04-07T16:22:24.130 INFO:tasks.rados.rados.0.smithi100.stdout:1723: finishing write tid 1 to smithi10066867-46 2024-04-07T16:22:24.131 INFO:tasks.rados.rados.0.smithi100.stdout:1723: finishing write tid 2 to smithi10066867-46 2024-04-07T16:22:24.131 INFO:tasks.rados.rados.0.smithi100.stdout:1725: delete oid 45 current snap is 161 2024-04-07T16:22:24.136 INFO:tasks.rados.rados.0.smithi100.stdout:1725: done (2 left) 2024-04-07T16:22:24.136 INFO:tasks.rados.rados.0.smithi100.stdout:1726: snap_remove snap 159 2024-04-07T16:22:24.146 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:23 smithi100 bash[74942]: cluster 2024-04-07T16:22:22.650205+0000 mgr.y (mgr.25175) 126 : cluster [DBG] pgmap v178: 121 pgs: 4 active+undersized+degraded, 9 active+undersized, 108 active+clean; 125 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 646 KiB/s rd, 812 KiB/s wr, 9 op/s; 14/282 objects degraded (4.965%); 14 MiB/s, 4 objects/s recovering 2024-04-07T16:22:24.146 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:23 smithi100 bash[74942]: cluster 2024-04-07T16:22:22.915162+0000 mon.a (mon.0) 396 : cluster [DBG] osdmap e393: 8 total, 8 up, 8 in 2024-04-07T16:22:24.148 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:23 smithi100 bash[76202]: cluster 2024-04-07T16:22:22.650205+0000 mgr.y (mgr.25175) 126 : cluster [DBG] pgmap v178: 121 pgs: 4 active+undersized+degraded, 9 active+undersized, 108 active+clean; 125 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 646 KiB/s rd, 812 KiB/s wr, 9 op/s; 14/282 objects degraded (4.965%); 14 MiB/s, 4 objects/s recovering 2024-04-07T16:22:24.148 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:23 smithi100 bash[76202]: cluster 2024-04-07T16:22:22.915162+0000 mon.a (mon.0) 396 : cluster [DBG] osdmap e393: 8 total, 8 up, 8 in 2024-04-07T16:22:24.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:23 smithi144 bash[66654]: cluster 2024-04-07T16:22:22.650205+0000 mgr.y (mgr.25175) 126 : cluster [DBG] pgmap v178: 121 pgs: 4 active+undersized+degraded, 9 active+undersized, 108 active+clean; 125 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 646 KiB/s rd, 812 KiB/s wr, 9 op/s; 14/282 objects degraded (4.965%); 14 MiB/s, 4 objects/s recovering 2024-04-07T16:22:24.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:23 smithi144 bash[66654]: cluster 2024-04-07T16:22:22.915162+0000 mon.a (mon.0) 396 : cluster [DBG] osdmap e393: 8 total, 8 up, 8 in 2024-04-07T16:22:24.922 INFO:tasks.rados.rados.0.smithi100.stdout:1726: done (2 left) 2024-04-07T16:22:24.922 INFO:tasks.rados.rados.0.smithi100.stdout:1727: read oid 1 snap 160 2024-04-07T16:22:24.922 INFO:tasks.rados.rados.0.smithi100.stdout:1727: expect deleted 2024-04-07T16:22:24.922 INFO:tasks.rados.rados.0.smithi100.stdout:1728: rmattr oid 38 current snap is 161 2024-04-07T16:22:24.922 INFO:tasks.rados.rados.0.smithi100.stdout:1728: done (3 left) 2024-04-07T16:22:24.922 INFO:tasks.rados.rados.0.smithi100.stdout:1729: read oid 33 snap -1 2024-04-07T16:22:24.922 INFO:tasks.rados.rados.0.smithi100.stdout:1729: expect deleted 2024-04-07T16:22:24.923 INFO:tasks.rados.rados.0.smithi100.stdout:1730: copy_from oid 26 from oid 8 current snap is 161 2024-04-07T16:22:24.923 INFO:tasks.rados.rados.0.smithi100.stdout:1723: finishing write tid 3 to smithi10066867-46 2024-04-07T16:22:24.923 INFO:tasks.rados.rados.0.smithi100.stdout:1723: finishing write tid 4 to smithi10066867-46 2024-04-07T16:22:24.923 INFO:tasks.rados.rados.0.smithi100.stdout:1723: finishing write tid 5 to smithi10066867-46 2024-04-07T16:22:24.923 INFO:tasks.rados.rados.0.smithi100.stdout:1723: finishing write tid 6 to smithi10066867-46 2024-04-07T16:22:24.923 INFO:tasks.rados.rados.0.smithi100.stdout:1723: oid 46 updating version 0 to 712 2024-04-07T16:22:24.923 INFO:tasks.rados.rados.0.smithi100.stdout:1723: oid 46 updating version 712 to 715 2024-04-07T16:22:24.923 INFO:tasks.rados.rados.0.smithi100.stdout:1723: oid 46 version 715 is already newer than 713 2024-04-07T16:22:24.923 INFO:tasks.rados.rados.0.smithi100.stdout:1723: oid 46 version 715 is already newer than 714 2024-04-07T16:22:24.923 INFO:tasks.rados.rados.0.smithi100.stdout:1723: oid 46 version 715 is already newer than 711 2024-04-07T16:22:24.923 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 46 v 715 (ObjNum 531 snap 161 seq_num 531) dirty exists 2024-04-07T16:22:24.923 INFO:tasks.rados.rados.0.smithi100.stdout:1723: left oid 46 (ObjNum 531 snap 161 seq_num 531) 2024-04-07T16:22:24.923 INFO:tasks.rados.rados.0.smithi100.stdout:1724: finishing copy_from racing read to smithi10066867-18 2024-04-07T16:22:24.923 INFO:tasks.rados.rados.0.smithi100.stdout:1724: finishing copy_from to smithi10066867-18 2024-04-07T16:22:24.923 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 18 v 543 (ObjNum 473 snap 143 seq_num 473) dirty exists 2024-04-07T16:22:24.924 INFO:tasks.rados.rados.0.smithi100.stdout:1723: done (4 left) 2024-04-07T16:22:24.924 INFO:tasks.rados.rados.0.smithi100.stdout:1724: done (3 left) 2024-04-07T16:22:24.924 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 42 current snap is 161 2024-04-07T16:22:24.924 INFO:tasks.rados.rados.0.smithi100.stdout:1731: seq_num 533 ranges {0=532480,532480=630784,1163264=516096,1679360=57344} 2024-04-07T16:22:24.926 INFO:tasks.rados.rados.0.smithi100.stdout:1731: writing smithi10066867-42 from 0 to 532480 tid 1 2024-04-07T16:22:24.933 INFO:tasks.rados.rados.0.smithi100.stdout:1731: writing smithi10066867-42 from 532480 to 1163264 tid 2 2024-04-07T16:22:24.938 INFO:tasks.rados.rados.0.smithi100.stdout:1731: writing smithi10066867-42 from 1163264 to 1679360 tid 3 2024-04-07T16:22:24.939 INFO:tasks.rados.rados.0.smithi100.stdout:1731: writing smithi10066867-42 from 1679360 to 1736704 tid 4 2024-04-07T16:22:24.939 INFO:tasks.rados.rados.0.smithi100.stdout:1732: setattr oid 9 current snap is 161 2024-04-07T16:22:24.943 INFO:tasks.rados.rados.0.smithi100.stdout:1727: done (4 left) 2024-04-07T16:22:24.943 INFO:tasks.rados.rados.0.smithi100.stdout:1729: done (3 left) 2024-04-07T16:22:24.943 INFO:tasks.rados.rados.0.smithi100.stdout:1733: snap_remove snap 160 2024-04-07T16:22:25.184 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:24 smithi100 bash[74942]: cluster 2024-04-07T16:22:23.922976+0000 mon.a (mon.0) 397 : cluster [DBG] osdmap e394: 8 total, 8 up, 8 in 2024-04-07T16:22:25.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:24 smithi100 bash[76202]: cluster 2024-04-07T16:22:23.922976+0000 mon.a (mon.0) 397 : cluster [DBG] osdmap e394: 8 total, 8 up, 8 in 2024-04-07T16:22:25.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:24 smithi144 bash[66654]: cluster 2024-04-07T16:22:23.922976+0000 mon.a (mon.0) 397 : cluster [DBG] osdmap e394: 8 total, 8 up, 8 in 2024-04-07T16:22:25.918 INFO:tasks.rados.rados.0.smithi100.stdout:1733: done (3 left) 2024-04-07T16:22:25.918 INFO:tasks.rados.rados.0.smithi100.stdout:1734: rmattr oid 1 current snap is 161 2024-04-07T16:22:25.919 INFO:tasks.rados.rados.0.smithi100.stdout:1734: done (3 left) 2024-04-07T16:22:25.919 INFO:tasks.rados.rados.0.smithi100.stdout:1735: read oid 24 snap 151 2024-04-07T16:22:25.919 INFO:tasks.rados.rados.0.smithi100.stdout:1735: expect (ObjNum 486 snap 147 seq_num 486) 2024-04-07T16:22:25.919 INFO:tasks.rados.rados.0.smithi100.stdout:1736: read oid 16 snap 151 2024-04-07T16:22:25.919 INFO:tasks.rados.rados.0.smithi100.stdout:1736: expect deleted 2024-04-07T16:22:25.919 INFO:tasks.rados.rados.0.smithi100.stdout:1731: finishing write tid 1 to smithi10066867-42 2024-04-07T16:22:25.919 INFO:tasks.rados.rados.0.smithi100.stdout:1737: delete oid 41 current snap is 161 2024-04-07T16:22:25.919 INFO:tasks.rados.rados.0.smithi100.stdout:1731: finishing write tid 2 to smithi10066867-42 2024-04-07T16:22:25.919 INFO:tasks.rados.rados.0.smithi100.stdout:1731: finishing write tid 3 to smithi10066867-42 2024-04-07T16:22:25.919 INFO:tasks.rados.rados.0.smithi100.stdout:1731: finishing write tid 4 to smithi10066867-42 2024-04-07T16:22:25.919 INFO:tasks.rados.rados.0.smithi100.stdout:1731: finishing write tid 5 to smithi10066867-42 2024-04-07T16:22:25.919 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 9 v 543 (ObjNum 481 snap 144 seq_num 481) dirty exists 2024-04-07T16:22:25.919 INFO:tasks.rados.rados.0.smithi100.stdout:1731: finishing write tid 6 to smithi10066867-42 2024-04-07T16:22:25.919 INFO:tasks.rados.rados.0.smithi100.stdout:1731: oid 42 updating version 0 to 799 2024-04-07T16:22:25.920 INFO:tasks.rados.rados.0.smithi100.stdout:1731: oid 42 updating version 799 to 802 2024-04-07T16:22:25.920 INFO:tasks.rados.rados.0.smithi100.stdout:1731: oid 42 updating version 802 to 803 2024-04-07T16:22:25.920 INFO:tasks.rados.rados.0.smithi100.stdout:1731: oid 42 version 803 is already newer than 800 2024-04-07T16:22:25.920 INFO:tasks.rados.rados.0.smithi100.stdout:1731: oid 42 version 803 is already newer than 801 2024-04-07T16:22:25.920 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 42 v 803 (ObjNum 533 snap 161 seq_num 533) dirty exists 2024-04-07T16:22:25.920 INFO:tasks.rados.rados.0.smithi100.stdout:1731: left oid 42 (ObjNum 533 snap 161 seq_num 533) 2024-04-07T16:22:25.920 INFO:tasks.rados.rados.0.smithi100.stdout:1730: finishing copy_from racing read to smithi10066867-26 2024-04-07T16:22:25.920 INFO:tasks.rados.rados.0.smithi100.stdout:1730: finishing copy_from to smithi10066867-26 2024-04-07T16:22:25.920 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 26 v 710 (ObjNum 524 snap 156 seq_num 524) dirty exists 2024-04-07T16:22:25.930 INFO:tasks.rados.rados.0.smithi100.stdout:1730: done (5 left) 2024-04-07T16:22:25.930 INFO:tasks.rados.rados.0.smithi100.stdout:1731: done (4 left) 2024-04-07T16:22:25.930 INFO:tasks.rados.rados.0.smithi100.stdout:1732: done (3 left) 2024-04-07T16:22:25.931 INFO:tasks.rados.rados.0.smithi100.stdout:1736: done (2 left) 2024-04-07T16:22:25.931 INFO:tasks.rados.rados.0.smithi100.stdout:1737: done (1 left) 2024-04-07T16:22:25.931 INFO:tasks.rados.rados.0.smithi100.stdout:1738: read oid 9 snap 151 2024-04-07T16:22:25.931 INFO:tasks.rados.rados.0.smithi100.stdout:1738: expect (ObjNum 481 snap 144 seq_num 481) 2024-04-07T16:22:25.931 INFO:tasks.rados.rados.0.smithi100.stdout:1739: read oid 37 snap -1 2024-04-07T16:22:25.931 INFO:tasks.rados.rados.0.smithi100.stdout:1739: expect (ObjNum 0 snap 0 seq_num 0) 2024-04-07T16:22:25.931 INFO:tasks.rados.rados.0.smithi100.stdout:1740: copy_from oid 20 from oid 26 current snap is 161 2024-04-07T16:22:25.931 INFO:tasks.rados.rados.0.smithi100.stdout:1741: delete oid 49 current snap is 161 2024-04-07T16:22:25.941 INFO:tasks.rados.rados.0.smithi100.stdout:1735: expect (ObjNum 486 snap 147 seq_num 486) 2024-04-07T16:22:25.960 INFO:tasks.rados.rados.0.smithi100.stdout:1739: expect (ObjNum 0 snap 0 seq_num 0) 2024-04-07T16:22:25.961 INFO:tasks.rados.rados.0.smithi100.stdout:1738: expect (ObjNum 481 snap 144 seq_num 481) 2024-04-07T16:22:25.986 INFO:tasks.rados.rados.0.smithi100.stdout:1740: finishing copy_from racing read to smithi10066867-20 2024-04-07T16:22:25.986 INFO:tasks.rados.rados.0.smithi100.stdout:1740: finishing copy_from to smithi10066867-20 2024-04-07T16:22:25.986 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 20 v 710 (ObjNum 524 snap 156 seq_num 524) dirty exists 2024-04-07T16:22:25.986 INFO:tasks.rados.rados.0.smithi100.stdout:1735: done (4 left) 2024-04-07T16:22:25.987 INFO:tasks.rados.rados.0.smithi100.stdout:1738: done (3 left) 2024-04-07T16:22:25.987 INFO:tasks.rados.rados.0.smithi100.stdout:1739: done (2 left) 2024-04-07T16:22:25.987 INFO:tasks.rados.rados.0.smithi100.stdout:1740: done (1 left) 2024-04-07T16:22:25.987 INFO:tasks.rados.rados.0.smithi100.stdout:1741: done (0 left) 2024-04-07T16:22:25.987 INFO:tasks.rados.rados.0.smithi100.stdout:1742: read oid 25 snap -1 2024-04-07T16:22:25.987 INFO:tasks.rados.rados.0.smithi100.stdout:1742: expect deleted 2024-04-07T16:22:25.987 INFO:tasks.rados.rados.0.smithi100.stdout:1743: snap_remove snap 151 2024-04-07T16:22:26.178 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:25 smithi100 bash[74942]: cluster 2024-04-07T16:22:24.650927+0000 mgr.y (mgr.25175) 127 : cluster [DBG] pgmap v181: 121 pgs: 121 active+clean; 131 MiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 646 KiB/s rd, 2.2 MiB/s wr, 9 op/s; 20 MiB/s, 6 objects/s recovering 2024-04-07T16:22:26.178 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:25 smithi100 bash[74942]: cluster 2024-04-07T16:22:24.915793+0000 mon.a (mon.0) 398 : cluster [INF] Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 14/282 objects degraded (4.965%), 4 pgs degraded) 2024-04-07T16:22:26.179 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:25 smithi100 bash[74942]: cluster 2024-04-07T16:22:24.915871+0000 mon.a (mon.0) 399 : cluster [INF] Cluster is now healthy 2024-04-07T16:22:26.179 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:25 smithi100 bash[74942]: cluster 2024-04-07T16:22:24.937813+0000 mon.a (mon.0) 400 : cluster [DBG] osdmap e395: 8 total, 8 up, 8 in 2024-04-07T16:22:26.179 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:25 smithi100 bash[76202]: cluster 2024-04-07T16:22:24.650927+0000 mgr.y (mgr.25175) 127 : cluster [DBG] pgmap v181: 121 pgs: 121 active+clean; 131 MiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 646 KiB/s rd, 2.2 MiB/s wr, 9 op/s; 20 MiB/s, 6 objects/s recovering 2024-04-07T16:22:26.179 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:25 smithi100 bash[76202]: cluster 2024-04-07T16:22:24.915793+0000 mon.a (mon.0) 398 : cluster [INF] Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 14/282 objects degraded (4.965%), 4 pgs degraded) 2024-04-07T16:22:26.179 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:25 smithi100 bash[76202]: cluster 2024-04-07T16:22:24.915871+0000 mon.a (mon.0) 399 : cluster [INF] Cluster is now healthy 2024-04-07T16:22:26.179 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:25 smithi100 bash[76202]: cluster 2024-04-07T16:22:24.937813+0000 mon.a (mon.0) 400 : cluster [DBG] osdmap e395: 8 total, 8 up, 8 in 2024-04-07T16:22:26.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:25 smithi144 bash[66654]: cluster 2024-04-07T16:22:24.650927+0000 mgr.y (mgr.25175) 127 : cluster [DBG] pgmap v181: 121 pgs: 121 active+clean; 131 MiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 646 KiB/s rd, 2.2 MiB/s wr, 9 op/s; 20 MiB/s, 6 objects/s recovering 2024-04-07T16:22:26.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:25 smithi144 bash[66654]: cluster 2024-04-07T16:22:24.915793+0000 mon.a (mon.0) 398 : cluster [INF] Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 14/282 objects degraded (4.965%), 4 pgs degraded) 2024-04-07T16:22:26.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:25 smithi144 bash[66654]: cluster 2024-04-07T16:22:24.915871+0000 mon.a (mon.0) 399 : cluster [INF] Cluster is now healthy 2024-04-07T16:22:26.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:25 smithi144 bash[66654]: cluster 2024-04-07T16:22:24.937813+0000 mon.a (mon.0) 400 : cluster [DBG] osdmap e395: 8 total, 8 up, 8 in 2024-04-07T16:22:26.792 INFO:tasks.rados.rados.0.smithi100.stdout:1742: done (1 left) 2024-04-07T16:22:26.792 INFO:tasks.rados.rados.0.smithi100.stdout:1743: done (0 left) 2024-04-07T16:22:26.793 INFO:tasks.rados.rados.0.smithi100.stdout:1744: rmattr oid 46 current snap is 161 2024-04-07T16:22:26.793 INFO:tasks.rados.rados.0.smithi100.stdout:1745: rollback oid 50 current snap is 161 2024-04-07T16:22:26.793 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 50 to 155 2024-04-07T16:22:26.794 INFO:tasks.rados.rados.0.smithi100.stdout:1746: read oid 25 snap -1 2024-04-07T16:22:26.794 INFO:tasks.rados.rados.0.smithi100.stdout:1746: expect deleted 2024-04-07T16:22:26.794 INFO:tasks.rados.rados.0.smithi100.stdout:1747: snap_create 2024-04-07T16:22:26.812 INFO:tasks.rados.rados.0.smithi100.stdout:1745: finishing rollback tid 0 to smithi10066867-50 2024-04-07T16:22:26.812 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 46 v 716 (ObjNum 531 snap 161 seq_num 531) dirty exists 2024-04-07T16:22:26.813 INFO:tasks.rados.rados.0.smithi100.stdout:1745: finishing rollback tid 1 to smithi10066867-50 2024-04-07T16:22:26.814 INFO:tasks.rados.rados.0.smithi100.stdout:1745: finishing rollback tid 2 to smithi10066867-50 2024-04-07T16:22:26.814 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 50 v 984 (ObjNum 518 snap 153 seq_num 518) dirty exists 2024-04-07T16:22:27.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:26 smithi100 bash[74942]: cluster 2024-04-07T16:22:25.934408+0000 mon.a (mon.0) 401 : cluster [DBG] osdmap e396: 8 total, 8 up, 8 in 2024-04-07T16:22:27.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:26 smithi100 bash[74942]: cluster 2024-04-07T16:22:26.805105+0000 mon.a (mon.0) 402 : cluster [DBG] osdmap e397: 8 total, 8 up, 8 in 2024-04-07T16:22:27.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:26 smithi100 bash[74942]: audit 2024-04-07T16:22:26.869861+0000 mon.a (mon.0) 403 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:22:27.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:26 smithi100 bash[74942]: audit 2024-04-07T16:22:26.876506+0000 mon.a (mon.0) 404 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:22:27.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:26 smithi100 bash[74942]: audit 2024-04-07T16:22:26.877898+0000 mon.a (mon.0) 405 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:22:27.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:26 smithi100 bash[74942]: audit 2024-04-07T16:22:26.878865+0000 mon.a (mon.0) 406 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:22:27.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:26 smithi100 bash[74942]: audit 2024-04-07T16:22:26.884380+0000 mon.a (mon.0) 407 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:22:27.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:26 smithi100 bash[76202]: cluster 2024-04-07T16:22:25.934408+0000 mon.a (mon.0) 401 : cluster [DBG] osdmap e396: 8 total, 8 up, 8 in 2024-04-07T16:22:27.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:26 smithi100 bash[76202]: cluster 2024-04-07T16:22:26.805105+0000 mon.a (mon.0) 402 : cluster [DBG] osdmap e397: 8 total, 8 up, 8 in 2024-04-07T16:22:27.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:26 smithi100 bash[76202]: audit 2024-04-07T16:22:26.869861+0000 mon.a (mon.0) 403 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:22:27.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:26 smithi100 bash[76202]: audit 2024-04-07T16:22:26.876506+0000 mon.a (mon.0) 404 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:22:27.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:26 smithi100 bash[76202]: audit 2024-04-07T16:22:26.877898+0000 mon.a (mon.0) 405 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:22:27.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:26 smithi100 bash[76202]: audit 2024-04-07T16:22:26.878865+0000 mon.a (mon.0) 406 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:22:27.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:26 smithi100 bash[76202]: audit 2024-04-07T16:22:26.884380+0000 mon.a (mon.0) 407 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:22:27.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:26 smithi144 bash[66654]: cluster 2024-04-07T16:22:25.934408+0000 mon.a (mon.0) 401 : cluster [DBG] osdmap e396: 8 total, 8 up, 8 in 2024-04-07T16:22:27.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:26 smithi144 bash[66654]: cluster 2024-04-07T16:22:26.805105+0000 mon.a (mon.0) 402 : cluster [DBG] osdmap e397: 8 total, 8 up, 8 in 2024-04-07T16:22:27.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:26 smithi144 bash[66654]: audit 2024-04-07T16:22:26.869861+0000 mon.a (mon.0) 403 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:22:27.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:26 smithi144 bash[66654]: audit 2024-04-07T16:22:26.876506+0000 mon.a (mon.0) 404 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:22:27.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:26 smithi144 bash[66654]: audit 2024-04-07T16:22:26.877898+0000 mon.a (mon.0) 405 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:22:27.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:26 smithi144 bash[66654]: audit 2024-04-07T16:22:26.878865+0000 mon.a (mon.0) 406 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:22:27.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:26 smithi144 bash[66654]: audit 2024-04-07T16:22:26.884380+0000 mon.a (mon.0) 407 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:22:27.790 INFO:tasks.rados.rados.0.smithi100.stdout:1744: done (3 left) 2024-04-07T16:22:27.790 INFO:tasks.rados.rados.0.smithi100.stdout:1745: done (2 left) 2024-04-07T16:22:27.790 INFO:tasks.rados.rados.0.smithi100.stdout:1746: done (1 left) 2024-04-07T16:22:27.790 INFO:tasks.rados.rados.0.smithi100.stdout:1747: done (0 left) 2024-04-07T16:22:27.790 INFO:tasks.rados.rados.0.smithi100.stdout:1748: read oid 1 snap -1 2024-04-07T16:22:27.790 INFO:tasks.rados.rados.0.smithi100.stdout:1748: expect deleted 2024-04-07T16:22:27.790 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 14 current snap is 162 2024-04-07T16:22:27.790 INFO:tasks.rados.rados.0.smithi100.stdout:1749: seq_num 536 ranges {0=581632,581632=688128,1269760=516096} 2024-04-07T16:22:27.799 INFO:tasks.rados.rados.0.smithi100.stdout:1749: writing smithi10066867-14 from 0 to 581632 tid 1 2024-04-07T16:22:27.812 INFO:tasks.rados.rados.0.smithi100.stdout:1749: writing smithi10066867-14 from 581632 to 1269760 tid 2 2024-04-07T16:22:27.822 INFO:tasks.rados.rados.0.smithi100.stdout:1749: writing smithi10066867-14 from 1269760 to 1785856 tid 3 2024-04-07T16:22:27.822 INFO:tasks.rados.rados.0.smithi100.stdout:1750: snap_remove snap 161 2024-04-07T16:22:28.182 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:27 smithi100 bash[74942]: cluster 2024-04-07T16:22:26.651706+0000 mgr.y (mgr.25175) 128 : cluster [DBG] pgmap v184: 121 pgs: 121 active+clean; 131 MiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 0 B/s rd, 1.4 MiB/s wr, 0 op/s; 5.1 MiB/s, 2 objects/s recovering 2024-04-07T16:22:28.182 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:27 smithi100 bash[74942]: audit 2024-04-07T16:22:26.946691+0000 mon.a (mon.0) 408 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:22:28.183 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:27 smithi100 bash[74942]: audit 2024-04-07T16:22:26.950463+0000 mon.a (mon.0) 409 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:22:28.183 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:27 smithi100 bash[74942]: audit 2024-04-07T16:22:26.952710+0000 mon.a (mon.0) 410 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:22:28.183 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:27 smithi100 bash[74942]: audit 2024-04-07T16:22:26.954447+0000 mon.a (mon.0) 411 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:22:28.183 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:27 smithi100 bash[74942]: audit 2024-04-07T16:22:26.956490+0000 mon.a (mon.0) 412 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-04-07T16:22:28.183 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:27 smithi100 bash[74942]: audit 2024-04-07T16:22:26.956683+0000 mgr.y (mgr.25175) 129 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-04-07T16:22:28.183 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:27 smithi100 bash[74942]: cephadm 2024-04-07T16:22:26.957988+0000 mgr.y (mgr.25175) 130 : cephadm [INF] Upgrade: osd.1 is safe to restart 2024-04-07T16:22:28.183 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:27 smithi100 bash[74942]: cluster 2024-04-07T16:22:27.807160+0000 mon.a (mon.0) 413 : cluster [DBG] osdmap e398: 8 total, 8 up, 8 in 2024-04-07T16:22:28.183 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:27 smithi100 bash[76202]: cluster 2024-04-07T16:22:26.651706+0000 mgr.y (mgr.25175) 128 : cluster [DBG] pgmap v184: 121 pgs: 121 active+clean; 131 MiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 0 B/s rd, 1.4 MiB/s wr, 0 op/s; 5.1 MiB/s, 2 objects/s recovering 2024-04-07T16:22:28.183 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:27 smithi100 bash[76202]: audit 2024-04-07T16:22:26.946691+0000 mon.a (mon.0) 408 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:22:28.183 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:27 smithi100 bash[76202]: audit 2024-04-07T16:22:26.950463+0000 mon.a (mon.0) 409 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:22:28.184 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:27 smithi100 bash[76202]: audit 2024-04-07T16:22:26.952710+0000 mon.a (mon.0) 410 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:22:28.184 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:27 smithi100 bash[76202]: audit 2024-04-07T16:22:26.954447+0000 mon.a (mon.0) 411 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:22:28.184 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:27 smithi100 bash[76202]: audit 2024-04-07T16:22:26.956490+0000 mon.a (mon.0) 412 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-04-07T16:22:28.184 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:27 smithi100 bash[76202]: audit 2024-04-07T16:22:26.956683+0000 mgr.y (mgr.25175) 129 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-04-07T16:22:28.184 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:27 smithi100 bash[76202]: cephadm 2024-04-07T16:22:26.957988+0000 mgr.y (mgr.25175) 130 : cephadm [INF] Upgrade: osd.1 is safe to restart 2024-04-07T16:22:28.184 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:27 smithi100 bash[76202]: cluster 2024-04-07T16:22:27.807160+0000 mon.a (mon.0) 413 : cluster [DBG] osdmap e398: 8 total, 8 up, 8 in 2024-04-07T16:22:28.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:27 smithi144 bash[66654]: cluster 2024-04-07T16:22:26.651706+0000 mgr.y (mgr.25175) 128 : cluster [DBG] pgmap v184: 121 pgs: 121 active+clean; 131 MiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 0 B/s rd, 1.4 MiB/s wr, 0 op/s; 5.1 MiB/s, 2 objects/s recovering 2024-04-07T16:22:28.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:27 smithi144 bash[66654]: audit 2024-04-07T16:22:26.946691+0000 mon.a (mon.0) 408 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:22:28.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:27 smithi144 bash[66654]: audit 2024-04-07T16:22:26.950463+0000 mon.a (mon.0) 409 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:22:28.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:27 smithi144 bash[66654]: audit 2024-04-07T16:22:26.952710+0000 mon.a (mon.0) 410 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:22:28.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:27 smithi144 bash[66654]: audit 2024-04-07T16:22:26.954447+0000 mon.a (mon.0) 411 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:22:28.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:27 smithi144 bash[66654]: audit 2024-04-07T16:22:26.956490+0000 mon.a (mon.0) 412 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-04-07T16:22:28.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:27 smithi144 bash[66654]: audit 2024-04-07T16:22:26.956683+0000 mgr.y (mgr.25175) 129 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-04-07T16:22:28.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:27 smithi144 bash[66654]: cephadm 2024-04-07T16:22:26.957988+0000 mgr.y (mgr.25175) 130 : cephadm [INF] Upgrade: osd.1 is safe to restart 2024-04-07T16:22:28.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:27 smithi144 bash[66654]: cluster 2024-04-07T16:22:27.807160+0000 mon.a (mon.0) 413 : cluster [DBG] osdmap e398: 8 total, 8 up, 8 in 2024-04-07T16:22:28.685 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:22:28 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:22:28] "GET /metrics HTTP/1.1" 200 36206 "" "Prometheus/2.43.0" 2024-04-07T16:22:28.789 INFO:tasks.rados.rados.0.smithi100.stdout:1750: done (2 left) 2024-04-07T16:22:28.789 INFO:tasks.rados.rados.0.smithi100.stdout:1751: rollback oid 30 current snap is 162 2024-04-07T16:22:28.789 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 30 to 155 2024-04-07T16:22:28.789 INFO:tasks.rados.rados.0.smithi100.stdout:1749: finishing write tid 1 to smithi10066867-14 2024-04-07T16:22:28.789 INFO:tasks.rados.rados.0.smithi100.stdout:1749: finishing write tid 2 to smithi10066867-14 2024-04-07T16:22:28.789 INFO:tasks.rados.rados.0.smithi100.stdout:1749: finishing write tid 3 to smithi10066867-14 2024-04-07T16:22:28.789 INFO:tasks.rados.rados.0.smithi100.stdout:1749: finishing write tid 4 to smithi10066867-14 2024-04-07T16:22:28.789 INFO:tasks.rados.rados.0.smithi100.stdout:1749: finishing write tid 5 to smithi10066867-14 2024-04-07T16:22:28.790 INFO:tasks.rados.rados.0.smithi100.stdout:1749: oid 14 updating version 0 to 986 2024-04-07T16:22:28.790 INFO:tasks.rados.rados.0.smithi100.stdout:1749: oid 14 version 986 is already newer than 985 2024-04-07T16:22:28.790 INFO:tasks.rados.rados.0.smithi100.stdout:1749: oid 14 updating version 986 to 988 2024-04-07T16:22:28.790 INFO:tasks.rados.rados.0.smithi100.stdout:1749: oid 14 version 988 is already newer than 987 2024-04-07T16:22:28.790 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 14 v 988 (ObjNum 536 snap 162 seq_num 536) dirty exists 2024-04-07T16:22:28.790 INFO:tasks.rados.rados.0.smithi100.stdout:1749: left oid 14 (ObjNum 536 snap 162 seq_num 536) 2024-04-07T16:22:28.790 INFO:tasks.rados.rados.0.smithi100.stdout:1748: done (2 left) 2024-04-07T16:22:28.790 INFO:tasks.rados.rados.0.smithi100.stdout:1749: done (1 left) 2024-04-07T16:22:28.790 INFO:tasks.rados.rados.0.smithi100.stdout:1752: copy_from oid 9 from oid 25 current snap is 162 2024-04-07T16:22:28.790 INFO:tasks.rados.rados.0.smithi100.stdout:1753: setattr oid 6 current snap is 162 2024-04-07T16:22:28.792 INFO:tasks.rados.rados.0.smithi100.stdout:1754: copy_from oid 14 from oid 49 current snap is 162 2024-04-07T16:22:28.792 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 13 current snap is 162 2024-04-07T16:22:28.792 INFO:tasks.rados.rados.0.smithi100.stdout:1755: seq_num 537 ranges {761856=598016,1359872=573440,1933312=491520,2424832=139264} 2024-04-07T16:22:28.803 INFO:tasks.rados.rados.0.smithi100.stdout:1755: writing smithi10066867-13 from 761856 to 1359872 tid 1 2024-04-07T16:22:28.807 INFO:tasks.rados.rados.0.smithi100.stdout:1755: writing smithi10066867-13 from 1359872 to 1933312 tid 2 2024-04-07T16:22:28.812 INFO:tasks.rados.rados.0.smithi100.stdout:1755: writing smithi10066867-13 from 1933312 to 2424832 tid 3 2024-04-07T16:22:28.814 INFO:tasks.rados.rados.0.smithi100.stdout:1755: writing smithi10066867-13 from 2424832 to 2564096 tid 4 2024-04-07T16:22:28.814 INFO:tasks.rados.rados.0.smithi100.stdout:1756: read oid 43 snap 155 2024-04-07T16:22:28.814 INFO:tasks.rados.rados.0.smithi100.stdout:1756: expect deleted 2024-04-07T16:22:28.814 INFO:tasks.rados.rados.0.smithi100.stdout:1757: snap_create 2024-04-07T16:22:28.815 INFO:tasks.rados.rados.0.smithi100.stdout:1752: finishing copy_from to smithi10066867-9 2024-04-07T16:22:28.815 INFO:tasks.rados.rados.0.smithi100.stdout:1752: got expected ENOENT (src dne) 2024-04-07T16:22:28.815 INFO:tasks.rados.rados.0.smithi100.stdout:1752: finishing copy_from racing read to smithi10066867-9 2024-04-07T16:22:28.815 INFO:tasks.rados.rados.0.smithi100.stdout:1754: finishing copy_from to smithi10066867-14 2024-04-07T16:22:28.815 INFO:tasks.rados.rados.0.smithi100.stdout:1754: got expected ENOENT (src dne) 2024-04-07T16:22:28.815 INFO:tasks.rados.rados.0.smithi100.stdout:1754: finishing copy_from racing read to smithi10066867-14 2024-04-07T16:22:28.815 INFO:tasks.rados.rados.0.smithi100.stdout:1751: finishing rollback tid 1 to smithi10066867-30 2024-04-07T16:22:28.815 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 30 v 0 (ObjNum 161 snap 0 seq_num 0) dirty dne 2024-04-07T16:22:28.815 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 6 v 825 (ObjNum 527 snap 157 seq_num 527) dirty exists 2024-04-07T16:22:28.818 INFO:tasks.rados.rados.0.smithi100.stdout:1755: finishing write tid 1 to smithi10066867-13 2024-04-07T16:22:28.819 INFO:tasks.rados.rados.0.smithi100.stdout:1755: finishing write tid 2 to smithi10066867-13 2024-04-07T16:22:28.819 INFO:tasks.rados.rados.0.smithi100.stdout:1755: finishing write tid 3 to smithi10066867-13 2024-04-07T16:22:28.823 INFO:tasks.rados.rados.0.smithi100.stdout:1755: finishing write tid 4 to smithi10066867-13 2024-04-07T16:22:28.824 INFO:tasks.rados.rados.0.smithi100.stdout:1755: finishing write tid 5 to smithi10066867-13 2024-04-07T16:22:28.826 INFO:tasks.rados.rados.0.smithi100.stdout:1755: finishing write tid 6 to smithi10066867-13 2024-04-07T16:22:28.826 INFO:tasks.rados.rados.0.smithi100.stdout:1755: oid 13 updating version 0 to 720 2024-04-07T16:22:28.826 INFO:tasks.rados.rados.0.smithi100.stdout:1755: oid 13 updating version 720 to 721 2024-04-07T16:22:28.826 INFO:tasks.rados.rados.0.smithi100.stdout:1755: oid 13 version 721 is already newer than 717 2024-04-07T16:22:28.826 INFO:tasks.rados.rados.0.smithi100.stdout:1755: oid 13 version 721 is already newer than 719 2024-04-07T16:22:28.826 INFO:tasks.rados.rados.0.smithi100.stdout:1755: oid 13 version 721 is already newer than 718 2024-04-07T16:22:28.827 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 13 v 721 (ObjNum 537 snap 162 seq_num 537) dirty exists 2024-04-07T16:22:28.827 INFO:tasks.rados.rados.0.smithi100.stdout:1755: left oid 13 (ObjNum 537 snap 162 seq_num 537) 2024-04-07T16:22:29.151 INFO:teuthology.orchestra.run.smithi100.stdout:true 2024-04-07T16:22:29.520 INFO:teuthology.orchestra.run.smithi100.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-07T16:22:29.520 INFO:teuthology.orchestra.run.smithi100.stdout:alertmanager.a smithi100 *:9093,9094 running (8m) 13s ago 9m 14.1M - 0.25.0 c8568f914cd2 5143075e7d26 2024-04-07T16:22:29.520 INFO:teuthology.orchestra.run.smithi100.stdout:grafana.a smithi144 *:3000 running (8m) 71s ago 8m 79.3M - 9.4.7 954c08fa6188 7180c0708acd 2024-04-07T16:22:29.520 INFO:teuthology.orchestra.run.smithi100.stdout:mds.a smithi100 running (10m) 13s ago 10m 15.4M - 18.2.2-1171-g267c8ff5 8a4a01e7186c 92593b0c28ae 2024-04-07T16:22:29.520 INFO:teuthology.orchestra.run.smithi100.stdout:mds.b smithi144 running (10m) 71s ago 10m 18.3M - 18.2.2-1171-g267c8ff5 8a4a01e7186c 3df2ee3e17a4 2024-04-07T16:22:29.520 INFO:teuthology.orchestra.run.smithi100.stdout:mgr.x smithi144 *:8443,9283,8765 running (2m) 71s ago 17m 438M - 19.0.0-2785-gadce92d0 c0ecd4226ac9 233745c292a6 2024-04-07T16:22:29.520 INFO:teuthology.orchestra.run.smithi100.stdout:mgr.y smithi100 *:8443,9283,8765 running (3m) 13s ago 20m 500M - 19.0.0-2785-gadce92d0 c0ecd4226ac9 254dc65cdaa7 2024-04-07T16:22:29.520 INFO:teuthology.orchestra.run.smithi100.stdout:mon.a smithi100 running (2m) 13s ago 20m 47.9M 2048M 19.0.0-2785-gadce92d0 c0ecd4226ac9 718e12655c04 2024-04-07T16:22:29.520 INFO:teuthology.orchestra.run.smithi100.stdout:mon.b smithi144 running (80s) 71s ago 18m 18.2M 2048M 19.0.0-2785-gadce92d0 c0ecd4226ac9 aef817664bff 2024-04-07T16:22:29.520 INFO:teuthology.orchestra.run.smithi100.stdout:mon.c smithi100 running (108s) 13s ago 17m 40.8M 2048M 19.0.0-2785-gadce92d0 c0ecd4226ac9 63e889040f6e 2024-04-07T16:22:29.520 INFO:teuthology.orchestra.run.smithi100.stdout:node-exporter.a smithi100 *:9100 running (9m) 13s ago 9m 8643k - 1.5.0 0da6a335fe13 4bce264c1b26 2024-04-07T16:22:29.521 INFO:teuthology.orchestra.run.smithi100.stdout:node-exporter.b smithi144 *:9100 running (9m) 71s ago 9m 8443k - 1.5.0 0da6a335fe13 b6963413c560 2024-04-07T16:22:29.521 INFO:teuthology.orchestra.run.smithi100.stdout:osd.0 smithi100 running (17s) 13s ago 16m - 2524M 19.0.0-2785-gadce92d0 c0ecd4226ac9 a6e83c9d1abc 2024-04-07T16:22:29.521 INFO:teuthology.orchestra.run.smithi100.stdout:osd.1 smithi100 running (16m) 13s ago 16m 208M 2524M 18.2.2-1171-g267c8ff5 8a4a01e7186c 7f57ca90894c 2024-04-07T16:22:29.521 INFO:teuthology.orchestra.run.smithi100.stdout:osd.2 smithi100 running (15m) 13s ago 15m 273M 2524M 18.2.2-1171-g267c8ff5 8a4a01e7186c f3912ddcc8cc 2024-04-07T16:22:29.521 INFO:teuthology.orchestra.run.smithi100.stdout:osd.3 smithi100 running (14m) 13s ago 14m 196M 2524M 18.2.2-1171-g267c8ff5 8a4a01e7186c b41004090c63 2024-04-07T16:22:29.521 INFO:teuthology.orchestra.run.smithi100.stdout:osd.4 smithi144 running (13m) 71s ago 13m 257M 2524M 18.2.2-1171-g267c8ff5 8a4a01e7186c 8c73b6b4dc13 2024-04-07T16:22:29.521 INFO:teuthology.orchestra.run.smithi100.stdout:osd.5 smithi144 running (12m) 71s ago 12m 216M 2524M 18.2.2-1171-g267c8ff5 8a4a01e7186c 3bfa0cf6f6cb 2024-04-07T16:22:29.521 INFO:teuthology.orchestra.run.smithi100.stdout:osd.6 smithi144 running (11m) 71s ago 11m 98.7M 2524M 18.2.2-1171-g267c8ff5 8a4a01e7186c 32b7f95d77f0 2024-04-07T16:22:29.521 INFO:teuthology.orchestra.run.smithi100.stdout:osd.7 smithi144 running (11m) 71s ago 11m 156M 2524M 18.2.2-1171-g267c8ff5 8a4a01e7186c 45f7709e576a 2024-04-07T16:22:29.521 INFO:teuthology.orchestra.run.smithi100.stdout:prometheus.a smithi144 *:9095 running (2m) 71s ago 9m 46.5M - 2.43.0 a07b618ecd1d 2d0a8131f303 2024-04-07T16:22:29.790 INFO:tasks.rados.rados.0.smithi100.stdout:1751: done (6 left) 2024-04-07T16:22:29.791 INFO:tasks.rados.rados.0.smithi100.stdout:1752: done (5 left) 2024-04-07T16:22:29.791 INFO:tasks.rados.rados.0.smithi100.stdout:1753: done (4 left) 2024-04-07T16:22:29.791 INFO:tasks.rados.rados.0.smithi100.stdout:1754: done (3 left) 2024-04-07T16:22:29.791 INFO:tasks.rados.rados.0.smithi100.stdout:1755: done (2 left) 2024-04-07T16:22:29.791 INFO:tasks.rados.rados.0.smithi100.stdout:1756: done (1 left) 2024-04-07T16:22:29.791 INFO:tasks.rados.rados.0.smithi100.stdout:1757: done (0 left) 2024-04-07T16:22:29.791 INFO:tasks.rados.rados.0.smithi100.stdout:1758: delete oid 30 current snap is 163 2024-04-07T16:22:29.802 INFO:tasks.rados.rados.0.smithi100.stdout:1758: done (0 left) 2024-04-07T16:22:29.802 INFO:tasks.rados.rados.0.smithi100.stdout:1759: rollback oid 41 current snap is 163 2024-04-07T16:22:29.802 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 41 to 162 2024-04-07T16:22:29.802 INFO:tasks.rados.rados.0.smithi100.stdout:1760: read oid 42 snap -1 2024-04-07T16:22:29.802 INFO:tasks.rados.rados.0.smithi100.stdout:1760: expect (ObjNum 533 snap 161 seq_num 533) 2024-04-07T16:22:29.802 INFO:tasks.rados.rados.0.smithi100.stdout:1761: copy_from oid 31 from oid 50 current snap is 163 2024-04-07T16:22:29.802 INFO:tasks.rados.rados.0.smithi100.stdout:1762: snap_create 2024-04-07T16:22:29.807 INFO:tasks.rados.rados.0.smithi100.stdout:1759: finishing rollback tid 1 to smithi10066867-41 2024-04-07T16:22:29.807 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 41 v 852 (ObjNum 545 snap 0 seq_num 2702703744) dirty dne 2024-04-07T16:22:29.814 INFO:tasks.rados.rados.0.smithi100.stdout:1760: expect (ObjNum 533 snap 161 seq_num 533) 2024-04-07T16:22:29.888 INFO:tasks.rados.rados.0.smithi100.stdout:1761: finishing copy_from racing read to smithi10066867-31 2024-04-07T16:22:29.888 INFO:tasks.rados.rados.0.smithi100.stdout:1761: finishing copy_from to smithi10066867-31 2024-04-07T16:22:29.888 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 31 v 984 (ObjNum 518 snap 153 seq_num 518) dirty exists 2024-04-07T16:22:29.974 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-04-07T16:22:29.974 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": { 2024-04-07T16:22:29.974 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph version 19.0.0-2785-gadce92d0 (adce92d056533f22dc1ee37e3141d098b046ec1b) squid (dev)": 3 2024-04-07T16:22:29.974 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-07T16:22:29.974 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": { 2024-04-07T16:22:29.974 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph version 19.0.0-2785-gadce92d0 (adce92d056533f22dc1ee37e3141d098b046ec1b) squid (dev)": 2 2024-04-07T16:22:29.974 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-07T16:22:29.975 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": { 2024-04-07T16:22:29.975 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph version 18.2.2-1171-g267c8ff5 (267c8ff5b1ff5815581b3b2381d19360e081b37b) reef (stable)": 7, 2024-04-07T16:22:29.975 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph version 19.0.0-2785-gadce92d0 (adce92d056533f22dc1ee37e3141d098b046ec1b) squid (dev)": 1 2024-04-07T16:22:29.975 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-07T16:22:29.975 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": { 2024-04-07T16:22:29.975 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph version 18.2.2-1171-g267c8ff5 (267c8ff5b1ff5815581b3b2381d19360e081b37b) reef (stable)": 2 2024-04-07T16:22:29.975 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-07T16:22:29.975 INFO:teuthology.orchestra.run.smithi100.stdout: "overall": { 2024-04-07T16:22:29.975 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph version 18.2.2-1171-g267c8ff5 (267c8ff5b1ff5815581b3b2381d19360e081b37b) reef (stable)": 9, 2024-04-07T16:22:29.975 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph version 19.0.0-2785-gadce92d0 (adce92d056533f22dc1ee37e3141d098b046ec1b) squid (dev)": 6 2024-04-07T16:22:29.975 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-04-07T16:22:29.975 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-04-07T16:22:30.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:29 smithi144 bash[66654]: cluster 2024-04-07T16:22:28.652482+0000 mgr.y (mgr.25175) 131 : cluster [DBG] pgmap v187: 121 pgs: 121 active+clean; 145 MiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 2.3 MiB/s rd, 5.8 MiB/s wr, 19 op/s 2024-04-07T16:22:30.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:29 smithi144 bash[66654]: cluster 2024-04-07T16:22:28.805205+0000 mon.a (mon.0) 414 : cluster [DBG] osdmap e399: 8 total, 8 up, 8 in 2024-04-07T16:22:30.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:29 smithi144 bash[66654]: cephadm 2024-04-07T16:22:29.027523+0000 mgr.y (mgr.25175) 132 : cephadm [INF] Upgrade: Updating osd.1 2024-04-07T16:22:30.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:29 smithi144 bash[66654]: audit 2024-04-07T16:22:29.032035+0000 mon.a (mon.0) 415 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:22:30.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:29 smithi144 bash[66654]: audit 2024-04-07T16:22:29.034000+0000 mon.a (mon.0) 416 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-07T16:22:30.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:29 smithi144 bash[66654]: audit 2024-04-07T16:22:29.035542+0000 mon.a (mon.0) 417 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:22:30.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:29 smithi144 bash[66654]: cephadm 2024-04-07T16:22:29.040181+0000 mgr.y (mgr.25175) 133 : cephadm [INF] Deploying daemon osd.1 on smithi100 2024-04-07T16:22:30.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:29 smithi144 bash[66654]: audit 2024-04-07T16:22:29.152715+0000 mgr.y (mgr.25175) 134 : audit [DBG] from='client.44162 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:22:30.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:29 smithi100 bash[74942]: cluster 2024-04-07T16:22:28.652482+0000 mgr.y (mgr.25175) 131 : cluster [DBG] pgmap v187: 121 pgs: 121 active+clean; 145 MiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 2.3 MiB/s rd, 5.8 MiB/s wr, 19 op/s 2024-04-07T16:22:30.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:29 smithi100 bash[74942]: cluster 2024-04-07T16:22:28.805205+0000 mon.a (mon.0) 414 : cluster [DBG] osdmap e399: 8 total, 8 up, 8 in 2024-04-07T16:22:30.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:29 smithi100 bash[74942]: cephadm 2024-04-07T16:22:29.027523+0000 mgr.y (mgr.25175) 132 : cephadm [INF] Upgrade: Updating osd.1 2024-04-07T16:22:30.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:29 smithi100 bash[74942]: audit 2024-04-07T16:22:29.032035+0000 mon.a (mon.0) 415 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:22:30.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:29 smithi100 bash[74942]: audit 2024-04-07T16:22:29.034000+0000 mon.a (mon.0) 416 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-07T16:22:30.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:29 smithi100 bash[74942]: audit 2024-04-07T16:22:29.035542+0000 mon.a (mon.0) 417 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:22:30.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:29 smithi100 bash[74942]: cephadm 2024-04-07T16:22:29.040181+0000 mgr.y (mgr.25175) 133 : cephadm [INF] Deploying daemon osd.1 on smithi100 2024-04-07T16:22:30.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:29 smithi100 bash[74942]: audit 2024-04-07T16:22:29.152715+0000 mgr.y (mgr.25175) 134 : audit [DBG] from='client.44162 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:22:30.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:29 smithi100 bash[76202]: cluster 2024-04-07T16:22:28.652482+0000 mgr.y (mgr.25175) 131 : cluster [DBG] pgmap v187: 121 pgs: 121 active+clean; 145 MiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 2.3 MiB/s rd, 5.8 MiB/s wr, 19 op/s 2024-04-07T16:22:30.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:29 smithi100 bash[76202]: cluster 2024-04-07T16:22:28.805205+0000 mon.a (mon.0) 414 : cluster [DBG] osdmap e399: 8 total, 8 up, 8 in 2024-04-07T16:22:30.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:29 smithi100 bash[76202]: cephadm 2024-04-07T16:22:29.027523+0000 mgr.y (mgr.25175) 132 : cephadm [INF] Upgrade: Updating osd.1 2024-04-07T16:22:30.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:29 smithi100 bash[76202]: audit 2024-04-07T16:22:29.032035+0000 mon.a (mon.0) 415 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:22:30.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:29 smithi100 bash[76202]: audit 2024-04-07T16:22:29.034000+0000 mon.a (mon.0) 416 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-07T16:22:30.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:29 smithi100 bash[76202]: audit 2024-04-07T16:22:29.035542+0000 mon.a (mon.0) 417 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:22:30.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:29 smithi100 bash[76202]: cephadm 2024-04-07T16:22:29.040181+0000 mgr.y (mgr.25175) 133 : cephadm [INF] Deploying daemon osd.1 on smithi100 2024-04-07T16:22:30.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:29 smithi100 bash[76202]: audit 2024-04-07T16:22:29.152715+0000 mgr.y (mgr.25175) 134 : audit [DBG] from='client.44162 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:22:30.812 INFO:tasks.rados.rados.0.smithi100.stdout:1759: done (3 left) 2024-04-07T16:22:30.812 INFO:tasks.rados.rados.0.smithi100.stdout:1760: done (2 left) 2024-04-07T16:22:30.812 INFO:tasks.rados.rados.0.smithi100.stdout:1761: done (1 left) 2024-04-07T16:22:30.812 INFO:tasks.rados.rados.0.smithi100.stdout:1762: done (0 left) 2024-04-07T16:22:30.812 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 47 current snap is 164 2024-04-07T16:22:30.812 INFO:tasks.rados.rados.0.smithi100.stdout:1763: seq_num 539 ranges {3006464=630784,3637248=737280,4374528=466944} 2024-04-07T16:22:30.869 INFO:tasks.rados.rados.0.smithi100.stdout:1763: writing smithi10066867-47 from 3006464 to 3637248 tid 1 2024-04-07T16:22:30.878 INFO:tasks.rados.rados.0.smithi100.stdout:1763: writing smithi10066867-47 from 3637248 to 4374528 tid 2 2024-04-07T16:22:30.884 INFO:tasks.rados.rados.0.smithi100.stdout:1763: writing smithi10066867-47 from 4374528 to 4841472 tid 3 2024-04-07T16:22:30.884 INFO:tasks.rados.rados.0.smithi100.stdout:1764: read oid 37 snap -1 2024-04-07T16:22:30.884 INFO:tasks.rados.rados.0.smithi100.stdout:1764: expect (ObjNum 0 snap 0 seq_num 0) 2024-04-07T16:22:30.884 INFO:tasks.rados.rados.0.smithi100.stdout:1765: read oid 9 snap -1 2024-04-07T16:22:30.885 INFO:tasks.rados.rados.0.smithi100.stdout:1765: expect (ObjNum 481 snap 144 seq_num 481) 2024-04-07T16:22:30.885 INFO:tasks.rados.rados.0.smithi100.stdout:1766: read oid 24 snap -1 2024-04-07T16:22:30.885 INFO:tasks.rados.rados.0.smithi100.stdout:1766: expect (ObjNum 486 snap 147 seq_num 486) 2024-04-07T16:22:30.885 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 40 current snap is 164 2024-04-07T16:22:30.885 INFO:tasks.rados.rados.0.smithi100.stdout:1767: seq_num 540 ranges {3416064=647168,4063232=671744,4734976=532480} 2024-04-07T16:22:30.916 INFO:tasks.rados.rados.0.smithi100.stdout:1767: writing smithi10066867-40 from 3416064 to 4063232 tid 1 2024-04-07T16:22:30.921 INFO:tasks.rados.rados.0.smithi100.stdout:1767: writing smithi10066867-40 from 4063232 to 4734976 tid 2 2024-04-07T16:22:30.925 INFO:tasks.rados.rados.0.smithi100.stdout:1767: writing smithi10066867-40 from 4734976 to 5267456 tid 3 2024-04-07T16:22:30.925 INFO:tasks.rados.rados.0.smithi100.stdout:1768: copy_from oid 5 from oid 3 current snap is 164 2024-04-07T16:22:30.925 INFO:tasks.rados.rados.0.smithi100.stdout:1763: finishing write tid 1 to smithi10066867-47 2024-04-07T16:22:30.925 INFO:tasks.rados.rados.0.smithi100.stdout:1764: expect (ObjNum 0 snap 0 seq_num 0) 2024-04-07T16:22:30.928 INFO:tasks.rados.rados.0.smithi100.stdout:1763: finishing write tid 2 to smithi10066867-47 2024-04-07T16:22:30.928 INFO:tasks.rados.rados.0.smithi100.stdout:1763: finishing write tid 3 to smithi10066867-47 2024-04-07T16:22:30.928 INFO:tasks.rados.rados.0.smithi100.stdout:1763: finishing write tid 4 to smithi10066867-47 2024-04-07T16:22:30.928 INFO:tasks.rados.rados.0.smithi100.stdout:1763: finishing write tid 5 to smithi10066867-47 2024-04-07T16:22:30.928 INFO:tasks.rados.rados.0.smithi100.stdout:1763: oid 47 updating version 0 to 547 2024-04-07T16:22:30.928 INFO:tasks.rados.rados.0.smithi100.stdout:1763: oid 47 version 547 is already newer than 545 2024-04-07T16:22:30.928 INFO:tasks.rados.rados.0.smithi100.stdout:1763: oid 47 version 547 is already newer than 546 2024-04-07T16:22:30.929 INFO:tasks.rados.rados.0.smithi100.stdout:1763: oid 47 updating version 547 to 548 2024-04-07T16:22:30.929 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 47 v 548 (ObjNum 539 snap 164 seq_num 539) dirty exists 2024-04-07T16:22:30.929 INFO:tasks.rados.rados.0.smithi100.stdout:1763: left oid 47 (ObjNum 539 snap 164 seq_num 539) 2024-04-07T16:22:30.929 INFO:tasks.rados.rados.0.smithi100.stdout:1766: expect (ObjNum 486 snap 147 seq_num 486) 2024-04-07T16:22:30.934 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:30 smithi100 bash[76202]: audit 2024-04-07T16:22:29.523340+0000 mgr.y (mgr.25175) 135 : audit [DBG] from='client.34248 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:22:30.934 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:30 smithi100 bash[76202]: cluster 2024-04-07T16:22:29.807001+0000 mon.a (mon.0) 418 : cluster [DBG] osdmap e400: 8 total, 8 up, 8 in 2024-04-07T16:22:30.934 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:30 smithi100 bash[76202]: audit 2024-04-07T16:22:29.991592+0000 mon.c (mon.2) 6 : audit [DBG] from='client.? 172.21.15.100:0/880643273' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:22:30.935 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:30 smithi100 bash[74942]: audit 2024-04-07T16:22:29.523340+0000 mgr.y (mgr.25175) 135 : audit [DBG] from='client.34248 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:22:30.935 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:30 smithi100 bash[74942]: cluster 2024-04-07T16:22:29.807001+0000 mon.a (mon.0) 418 : cluster [DBG] osdmap e400: 8 total, 8 up, 8 in 2024-04-07T16:22:30.935 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:30 smithi100 bash[74942]: audit 2024-04-07T16:22:29.991592+0000 mon.c (mon.2) 6 : audit [DBG] from='client.? 172.21.15.100:0/880643273' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:22:30.969 INFO:tasks.rados.rados.0.smithi100.stdout:1765: expect (ObjNum 481 snap 144 seq_num 481) 2024-04-07T16:22:31.004 INFO:tasks.rados.rados.0.smithi100.stdout:1767: finishing write tid 1 to smithi10066867-40 2024-04-07T16:22:31.005 INFO:tasks.rados.rados.0.smithi100.stdout:1767: finishing write tid 2 to smithi10066867-40 2024-04-07T16:22:31.005 INFO:tasks.rados.rados.0.smithi100.stdout:1767: finishing write tid 3 to smithi10066867-40 2024-04-07T16:22:31.005 INFO:tasks.rados.rados.0.smithi100.stdout:1767: finishing write tid 4 to smithi10066867-40 2024-04-07T16:22:31.005 INFO:tasks.rados.rados.0.smithi100.stdout:1767: finishing write tid 5 to smithi10066867-40 2024-04-07T16:22:31.005 INFO:tasks.rados.rados.0.smithi100.stdout:1767: oid 40 updating version 0 to 829 2024-04-07T16:22:31.005 INFO:tasks.rados.rados.0.smithi100.stdout:1767: oid 40 updating version 829 to 830 2024-04-07T16:22:31.005 INFO:tasks.rados.rados.0.smithi100.stdout:1767: oid 40 version 830 is already newer than 827 2024-04-07T16:22:31.005 INFO:tasks.rados.rados.0.smithi100.stdout:1767: oid 40 version 830 is already newer than 828 2024-04-07T16:22:31.005 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 40 v 830 (ObjNum 540 snap 164 seq_num 540) dirty exists 2024-04-07T16:22:31.005 INFO:tasks.rados.rados.0.smithi100.stdout:1767: left oid 40 (ObjNum 540 snap 164 seq_num 540) 2024-04-07T16:22:31.005 INFO:tasks.rados.rados.0.smithi100.stdout:1768: finishing copy_from racing read to smithi10066867-5 2024-04-07T16:22:31.005 INFO:tasks.rados.rados.0.smithi100.stdout:1768: finishing copy_from to smithi10066867-5 2024-04-07T16:22:31.005 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 5 v 1008 (ObjNum 530 snap 161 seq_num 530) dirty exists 2024-04-07T16:22:31.005 INFO:tasks.rados.rados.0.smithi100.stdout:1763: done (5 left) 2024-04-07T16:22:31.005 INFO:tasks.rados.rados.0.smithi100.stdout:1764: done (4 left) 2024-04-07T16:22:31.006 INFO:tasks.rados.rados.0.smithi100.stdout:1765: done (3 left) 2024-04-07T16:22:31.006 INFO:tasks.rados.rados.0.smithi100.stdout:1766: done (2 left) 2024-04-07T16:22:31.006 INFO:tasks.rados.rados.0.smithi100.stdout:1767: done (1 left) 2024-04-07T16:22:31.006 INFO:tasks.rados.rados.0.smithi100.stdout:1768: done (0 left) 2024-04-07T16:22:31.006 INFO:tasks.rados.rados.0.smithi100.stdout:1769: read oid 43 snap 162 2024-04-07T16:22:31.006 INFO:tasks.rados.rados.0.smithi100.stdout:1769: expect deleted 2024-04-07T16:22:31.006 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 27 current snap is 164 2024-04-07T16:22:31.006 INFO:tasks.rados.rados.0.smithi100.stdout:1770: seq_num 541 ranges {3907584=663552,4571136=557056,5128192=475136,5603328=172032} 2024-04-07T16:22:31.045 INFO:tasks.rados.rados.0.smithi100.stdout:1770: writing smithi10066867-27 from 3907584 to 4571136 tid 1 2024-04-07T16:22:31.050 INFO:tasks.rados.rados.0.smithi100.stdout:1770: writing smithi10066867-27 from 4571136 to 5128192 tid 2 2024-04-07T16:22:31.055 INFO:tasks.rados.rados.0.smithi100.stdout:1770: writing smithi10066867-27 from 5128192 to 5603328 tid 3 2024-04-07T16:22:31.057 INFO:tasks.rados.rados.0.smithi100.stdout:1770: writing smithi10066867-27 from 5603328 to 5775360 tid 4 2024-04-07T16:22:31.057 INFO:tasks.rados.rados.0.smithi100.stdout:1771: rollback oid 11 current snap is 164 2024-04-07T16:22:31.057 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 11 to 163 2024-04-07T16:22:31.057 INFO:tasks.rados.rados.0.smithi100.stdout:1770: finishing write tid 1 to smithi10066867-27 2024-04-07T16:22:31.057 INFO:tasks.rados.rados.0.smithi100.stdout:1770: finishing write tid 2 to smithi10066867-27 2024-04-07T16:22:31.057 INFO:tasks.rados.rados.0.smithi100.stdout:1769: done (2 left) 2024-04-07T16:22:31.057 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 15 current snap is 164 2024-04-07T16:22:31.057 INFO:tasks.rados.rados.0.smithi100.stdout:1772: seq_num 542 ranges {1622016=679936,2301952=442368,2744320=679936,3424256=81920} 2024-04-07T16:22:31.074 INFO:tasks.rados.rados.0.smithi100.stdout:1772: writing smithi10066867-15 from 1622016 to 2301952 tid 1 2024-04-07T16:22:31.077 INFO:tasks.rados.rados.0.smithi100.stdout:1772: writing smithi10066867-15 from 2301952 to 2744320 tid 2 2024-04-07T16:22:31.081 INFO:tasks.rados.rados.0.smithi100.stdout:1772: writing smithi10066867-15 from 2744320 to 3424256 tid 3 2024-04-07T16:22:31.083 INFO:tasks.rados.rados.0.smithi100.stdout:1772: writing smithi10066867-15 from 3424256 to 3506176 tid 4 2024-04-07T16:22:31.083 INFO:tasks.rados.rados.0.smithi100.stdout:1773: setattr oid 18 current snap is 164 2024-04-07T16:22:31.083 INFO:tasks.rados.rados.0.smithi100.stdout:1770: finishing write tid 3 to smithi10066867-27 2024-04-07T16:22:31.083 INFO:tasks.rados.rados.0.smithi100.stdout:1770: finishing write tid 4 to smithi10066867-27 2024-04-07T16:22:31.083 INFO:tasks.rados.rados.0.smithi100.stdout:1770: finishing write tid 5 to smithi10066867-27 2024-04-07T16:22:31.083 INFO:tasks.rados.rados.0.smithi100.stdout:1771: finishing rollback tid 0 to smithi10066867-11 2024-04-07T16:22:31.083 INFO:tasks.rados.rados.0.smithi100.stdout:1770: finishing write tid 6 to smithi10066867-27 2024-04-07T16:22:31.083 INFO:tasks.rados.rados.0.smithi100.stdout:1770: oid 27 updating version 0 to 801 2024-04-07T16:22:31.083 INFO:tasks.rados.rados.0.smithi100.stdout:1770: oid 27 version 801 is already newer than 798 2024-04-07T16:22:31.083 INFO:tasks.rados.rados.0.smithi100.stdout:1770: oid 27 version 801 is already newer than 799 2024-04-07T16:22:31.083 INFO:tasks.rados.rados.0.smithi100.stdout:1770: oid 27 version 801 is already newer than 800 2024-04-07T16:22:31.083 INFO:tasks.rados.rados.0.smithi100.stdout:1770: oid 27 version 801 is already newer than 797 2024-04-07T16:22:31.084 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 27 v 801 (ObjNum 541 snap 164 seq_num 541) dirty exists 2024-04-07T16:22:31.084 INFO:tasks.rados.rados.0.smithi100.stdout:1770: left oid 27 (ObjNum 541 snap 164 seq_num 541) 2024-04-07T16:22:31.084 INFO:tasks.rados.rados.0.smithi100.stdout:1771: finishing rollback tid 1 to smithi10066867-11 2024-04-07T16:22:31.084 INFO:tasks.rados.rados.0.smithi100.stdout:1771: finishing rollback tid 2 to smithi10066867-11 2024-04-07T16:22:31.084 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 11 v 1045 (ObjNum 501 snap 150 seq_num 501) dirty exists 2024-04-07T16:22:31.084 INFO:tasks.rados.rados.0.smithi100.stdout:1772: finishing write tid 1 to smithi10066867-15 2024-04-07T16:22:31.084 INFO:tasks.rados.rados.0.smithi100.stdout:1772: finishing write tid 2 to smithi10066867-15 2024-04-07T16:22:31.085 INFO:tasks.rados.rados.0.smithi100.stdout:1770: done (3 left) 2024-04-07T16:22:31.085 INFO:tasks.rados.rados.0.smithi100.stdout:1771: done (2 left) 2024-04-07T16:22:31.085 INFO:tasks.rados.rados.0.smithi100.stdout:1774: copy_from oid 17 from oid 19 current snap is 164 2024-04-07T16:22:31.085 INFO:tasks.rados.rados.0.smithi100.stdout:1775: read oid 9 snap -1 2024-04-07T16:22:31.085 INFO:tasks.rados.rados.0.smithi100.stdout:1775: expect (ObjNum 481 snap 144 seq_num 481) 2024-04-07T16:22:31.085 INFO:tasks.rados.rados.0.smithi100.stdout:1776: snap_remove snap 155 2024-04-07T16:22:31.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:30 smithi144 bash[66654]: audit 2024-04-07T16:22:29.523340+0000 mgr.y (mgr.25175) 135 : audit [DBG] from='client.34248 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:22:31.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:30 smithi144 bash[66654]: cluster 2024-04-07T16:22:29.807001+0000 mon.a (mon.0) 418 : cluster [DBG] osdmap e400: 8 total, 8 up, 8 in 2024-04-07T16:22:31.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:30 smithi144 bash[66654]: audit 2024-04-07T16:22:29.991592+0000 mon.c (mon.2) 6 : audit [DBG] from='client.? 172.21.15.100:0/880643273' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:22:31.786 INFO:tasks.rados.rados.0.smithi100.stdout:1776: done (4 left) 2024-04-07T16:22:31.786 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 20 current snap is 164 2024-04-07T16:22:31.786 INFO:tasks.rados.rados.0.smithi100.stdout:1772: finishing write tid 3 to smithi10066867-15 2024-04-07T16:22:31.786 INFO:tasks.rados.rados.0.smithi100.stdout:1772: finishing write tid 4 to smithi10066867-15 2024-04-07T16:22:31.786 INFO:tasks.rados.rados.0.smithi100.stdout:1777: seq_num 543 ranges {7094272=704512,7798784=729088,8527872=475136} 2024-04-07T16:22:31.845 INFO:tasks.rados.rados.0.smithi100.stdout:1777: writing smithi10066867-20 from 7094272 to 7798784 tid 1 2024-04-07T16:22:31.850 INFO:tasks.rados.rados.0.smithi100.stdout:1777: writing smithi10066867-20 from 7798784 to 8527872 tid 2 2024-04-07T16:22:31.853 INFO:tasks.rados.rados.0.smithi100.stdout:1777: writing smithi10066867-20 from 8527872 to 9003008 tid 3 2024-04-07T16:22:31.853 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 24 current snap is 164 2024-04-07T16:22:31.853 INFO:tasks.rados.rados.0.smithi100.stdout:1778: seq_num 544 ranges {950272=720896,1671168=614400,2285568=589824} 2024-04-07T16:22:31.863 INFO:tasks.rados.rados.0.smithi100.stdout:1778: writing smithi10066867-24 from 950272 to 1671168 tid 1 2024-04-07T16:22:31.867 INFO:tasks.rados.rados.0.smithi100.stdout:1778: writing smithi10066867-24 from 1671168 to 2285568 tid 2 2024-04-07T16:22:31.872 INFO:tasks.rados.rados.0.smithi100.stdout:1778: writing smithi10066867-24 from 2285568 to 2875392 tid 3 2024-04-07T16:22:31.872 INFO:tasks.rados.rados.0.smithi100.stdout:1779: copy_from oid 6 from oid 26 current snap is 164 2024-04-07T16:22:31.872 INFO:tasks.rados.rados.0.smithi100.stdout:1772: finishing write tid 5 to smithi10066867-15 2024-04-07T16:22:31.872 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 18 v 1053 (ObjNum 473 snap 143 seq_num 473) dirty exists 2024-04-07T16:22:31.872 INFO:tasks.rados.rados.0.smithi100.stdout:1772: finishing write tid 6 to smithi10066867-15 2024-04-07T16:22:31.872 INFO:tasks.rados.rados.0.smithi100.stdout:1772: oid 15 updating version 0 to 1047 2024-04-07T16:22:31.872 INFO:tasks.rados.rados.0.smithi100.stdout:1772: oid 15 updating version 1047 to 1050 2024-04-07T16:22:31.872 INFO:tasks.rados.rados.0.smithi100.stdout:1772: oid 15 version 1050 is already newer than 1049 2024-04-07T16:22:31.872 INFO:tasks.rados.rados.0.smithi100.stdout:1772: oid 15 updating version 1050 to 1051 2024-04-07T16:22:31.872 INFO:tasks.rados.rados.0.smithi100.stdout:1772: oid 15 version 1051 is already newer than 1048 2024-04-07T16:22:31.873 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 15 v 1051 (ObjNum 542 snap 164 seq_num 542) dirty exists 2024-04-07T16:22:31.873 INFO:tasks.rados.rados.0.smithi100.stdout:1772: left oid 15 (ObjNum 542 snap 164 seq_num 542) 2024-04-07T16:22:31.873 INFO:tasks.rados.rados.0.smithi100.stdout:1775: expect (ObjNum 481 snap 144 seq_num 481) 2024-04-07T16:22:31.910 INFO:tasks.rados.rados.0.smithi100.stdout:1774: finishing copy_from racing read to smithi10066867-17 2024-04-07T16:22:31.910 INFO:tasks.rados.rados.0.smithi100.stdout:1774: finishing copy_from to smithi10066867-17 2024-04-07T16:22:31.910 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 17 v 712 (ObjNum 506 snap 151 seq_num 506) dirty exists 2024-04-07T16:22:31.910 INFO:tasks.rados.rados.0.smithi100.stdout:1777: finishing write tid 1 to smithi10066867-20 2024-04-07T16:22:31.910 INFO:tasks.rados.rados.0.smithi100.stdout:1777: finishing write tid 2 to smithi10066867-20 2024-04-07T16:22:31.910 INFO:tasks.rados.rados.0.smithi100.stdout:1772: done (6 left) 2024-04-07T16:22:31.910 INFO:tasks.rados.rados.0.smithi100.stdout:1773: done (5 left) 2024-04-07T16:22:31.911 INFO:tasks.rados.rados.0.smithi100.stdout:1774: done (4 left) 2024-04-07T16:22:31.911 INFO:tasks.rados.rados.0.smithi100.stdout:1775: done (3 left) 2024-04-07T16:22:31.911 INFO:tasks.rados.rados.0.smithi100.stdout:1780: snap_remove snap 162 2024-04-07T16:22:32.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:31 smithi144 bash[66654]: cluster 2024-04-07T16:22:30.653129+0000 mgr.y (mgr.25175) 136 : cluster [DBG] pgmap v190: 121 pgs: 121 active+clean; 145 MiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 2.3 MiB/s rd, 5.8 MiB/s wr, 19 op/s 2024-04-07T16:22:32.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:31 smithi144 bash[66654]: cluster 2024-04-07T16:22:30.827981+0000 mon.a (mon.0) 419 : cluster [DBG] osdmap e401: 8 total, 8 up, 8 in 2024-04-07T16:22:32.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:31 smithi144 bash[66654]: audit 2024-04-07T16:22:31.710943+0000 mon.a (mon.0) 420 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:22:32.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:31 smithi144 bash[66654]: audit 2024-04-07T16:22:31.712663+0000 mon.a (mon.0) 421 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:22:32.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:31 smithi144 bash[66654]: audit 2024-04-07T16:22:31.772437+0000 mon.a (mon.0) 422 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:22:32.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:31 smithi144 bash[66654]: cluster 2024-04-07T16:22:31.802941+0000 mon.a (mon.0) 423 : cluster [DBG] osdmap e402: 8 total, 8 up, 8 in 2024-04-07T16:22:32.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:31 smithi100 bash[76202]: cluster 2024-04-07T16:22:30.653129+0000 mgr.y (mgr.25175) 136 : cluster [DBG] pgmap v190: 121 pgs: 121 active+clean; 145 MiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 2.3 MiB/s rd, 5.8 MiB/s wr, 19 op/s 2024-04-07T16:22:32.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:31 smithi100 bash[76202]: cluster 2024-04-07T16:22:30.827981+0000 mon.a (mon.0) 419 : cluster [DBG] osdmap e401: 8 total, 8 up, 8 in 2024-04-07T16:22:32.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:31 smithi100 bash[76202]: audit 2024-04-07T16:22:31.710943+0000 mon.a (mon.0) 420 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:22:32.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:31 smithi100 bash[76202]: audit 2024-04-07T16:22:31.712663+0000 mon.a (mon.0) 421 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:22:32.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:31 smithi100 bash[76202]: audit 2024-04-07T16:22:31.772437+0000 mon.a (mon.0) 422 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:22:32.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:31 smithi100 bash[76202]: cluster 2024-04-07T16:22:31.802941+0000 mon.a (mon.0) 423 : cluster [DBG] osdmap e402: 8 total, 8 up, 8 in 2024-04-07T16:22:32.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:31 smithi100 bash[74942]: cluster 2024-04-07T16:22:30.653129+0000 mgr.y (mgr.25175) 136 : cluster [DBG] pgmap v190: 121 pgs: 121 active+clean; 145 MiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 2.3 MiB/s rd, 5.8 MiB/s wr, 19 op/s 2024-04-07T16:22:32.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:31 smithi100 bash[74942]: cluster 2024-04-07T16:22:30.827981+0000 mon.a (mon.0) 419 : cluster [DBG] osdmap e401: 8 total, 8 up, 8 in 2024-04-07T16:22:32.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:31 smithi100 bash[74942]: audit 2024-04-07T16:22:31.710943+0000 mon.a (mon.0) 420 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:22:32.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:31 smithi100 bash[74942]: audit 2024-04-07T16:22:31.712663+0000 mon.a (mon.0) 421 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:22:32.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:31 smithi100 bash[74942]: audit 2024-04-07T16:22:31.772437+0000 mon.a (mon.0) 422 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:22:32.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:31 smithi100 bash[74942]: cluster 2024-04-07T16:22:31.802941+0000 mon.a (mon.0) 423 : cluster [DBG] osdmap e402: 8 total, 8 up, 8 in 2024-04-07T16:22:32.606 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:22:32 smithi144 bash[59794]: ts=2024-04-07T16:22:32.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:22:32.830 INFO:tasks.rados.rados.0.smithi100.stdout:1780: done (3 left) 2024-04-07T16:22:32.830 INFO:tasks.rados.rados.0.smithi100.stdout:1781: read oid 8 snap -1 2024-04-07T16:22:32.830 INFO:tasks.rados.rados.0.smithi100.stdout:1781: expect (ObjNum 524 snap 156 seq_num 524) 2024-04-07T16:22:32.830 INFO:tasks.rados.rados.0.smithi100.stdout:1782: snap_create 2024-04-07T16:22:32.830 INFO:tasks.rados.rados.0.smithi100.stdout:1777: finishing write tid 3 to smithi10066867-20 2024-04-07T16:22:32.831 INFO:tasks.rados.rados.0.smithi100.stdout:1777: finishing write tid 4 to smithi10066867-20 2024-04-07T16:22:32.831 INFO:tasks.rados.rados.0.smithi100.stdout:1777: finishing write tid 5 to smithi10066867-20 2024-04-07T16:22:32.831 INFO:tasks.rados.rados.0.smithi100.stdout:1777: oid 20 updating version 0 to 763 2024-04-07T16:22:32.831 INFO:tasks.rados.rados.0.smithi100.stdout:1777: oid 20 updating version 763 to 765 2024-04-07T16:22:32.831 INFO:tasks.rados.rados.0.smithi100.stdout:1777: oid 20 updating version 765 to 766 2024-04-07T16:22:32.831 INFO:tasks.rados.rados.0.smithi100.stdout:1777: oid 20 version 766 is already newer than 764 2024-04-07T16:22:32.831 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 20 v 766 (ObjNum 543 snap 164 seq_num 543) dirty exists 2024-04-07T16:22:32.831 INFO:tasks.rados.rados.0.smithi100.stdout:1777: left oid 20 (ObjNum 543 snap 164 seq_num 543) 2024-04-07T16:22:32.831 INFO:tasks.rados.rados.0.smithi100.stdout:1778: finishing write tid 1 to smithi10066867-24 2024-04-07T16:22:32.831 INFO:tasks.rados.rados.0.smithi100.stdout:1778: finishing write tid 2 to smithi10066867-24 2024-04-07T16:22:32.831 INFO:tasks.rados.rados.0.smithi100.stdout:1778: finishing write tid 3 to smithi10066867-24 2024-04-07T16:22:32.831 INFO:tasks.rados.rados.0.smithi100.stdout:1778: finishing write tid 4 to smithi10066867-24 2024-04-07T16:22:32.831 INFO:tasks.rados.rados.0.smithi100.stdout:1778: finishing write tid 5 to smithi10066867-24 2024-04-07T16:22:32.831 INFO:tasks.rados.rados.0.smithi100.stdout:1778: oid 24 updating version 0 to 723 2024-04-07T16:22:32.831 INFO:tasks.rados.rados.0.smithi100.stdout:1778: oid 24 version 723 is already newer than 722 2024-04-07T16:22:32.832 INFO:tasks.rados.rados.0.smithi100.stdout:1778: oid 24 version 723 is already newer than 720 2024-04-07T16:22:32.832 INFO:tasks.rados.rados.0.smithi100.stdout:1778: oid 24 version 723 is already newer than 721 2024-04-07T16:22:32.832 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 24 v 723 (ObjNum 544 snap 164 seq_num 544) dirty exists 2024-04-07T16:22:32.832 INFO:tasks.rados.rados.0.smithi100.stdout:1778: left oid 24 (ObjNum 544 snap 164 seq_num 544) 2024-04-07T16:22:32.832 INFO:tasks.rados.rados.0.smithi100.stdout:1779: finishing copy_from racing read to smithi10066867-6 2024-04-07T16:22:32.832 INFO:tasks.rados.rados.0.smithi100.stdout:1779: finishing copy_from to smithi10066867-6 2024-04-07T16:22:32.832 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 6 v 710 (ObjNum 524 snap 156 seq_num 524) dirty exists 2024-04-07T16:22:32.882 INFO:tasks.rados.rados.0.smithi100.stdout:1781: expect (ObjNum 524 snap 156 seq_num 524) 2024-04-07T16:22:33.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:33 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:22:33.295 INFO:journalctl@ceph.osd.1.smithi100.stdout:Apr 07 16:22:33 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:22:33.295 INFO:journalctl@ceph.osd.2.smithi100.stdout:Apr 07 16:22:33 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:22:33.295 INFO:journalctl@ceph.osd.3.smithi100.stdout:Apr 07 16:22:33 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:22:33.295 INFO:journalctl@ceph.osd.0.smithi100.stdout:Apr 07 16:22:33 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:22:33.296 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 16:22:33 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:22:33.296 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:22:33 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:22:33.296 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Apr 07 16:22:33 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:22:33.297 INFO:journalctl@ceph.mds.a.smithi100.stdout:Apr 07 16:22:33 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:22:33.297 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:33 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:22:33.685 INFO:journalctl@ceph.osd.1.smithi100.stdout:Apr 07 16:22:33 smithi100 systemd[1]: Stopping Ceph osd.1 for 0702e3a8-f4f8-11ee-b649-cb9ed24678a4... 2024-04-07T16:22:33.685 INFO:journalctl@ceph.osd.1.smithi100.stdout:Apr 07 16:22:33 smithi100 bash[40038]: debug 2024-04-07T16:22:33.435+0000 7f55dfab9700 -1 received signal: Terminated from /sbin/docker-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug (PID: 1) UID: 0 2024-04-07T16:22:33.685 INFO:journalctl@ceph.osd.1.smithi100.stdout:Apr 07 16:22:33 smithi100 bash[40038]: debug 2024-04-07T16:22:33.435+0000 7f55dfab9700 -1 osd.1 403 *** Got signal Terminated *** 2024-04-07T16:22:33.685 INFO:journalctl@ceph.osd.1.smithi100.stdout:Apr 07 16:22:33 smithi100 bash[40038]: debug 2024-04-07T16:22:33.435+0000 7f55dfab9700 -1 osd.1 403 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-07T16:22:33.845 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:22:33 smithi144 bash[59794]: ts=2024-04-07T16:22:33.507Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=nodes name=CephNodeDiskspaceWarning index=4 msg="Evaluating rule failed" rule="alert: CephNodeDiskspaceWarning\nexpr: predict_linear(node_filesystem_free_bytes{device=~\"/.*\"}[2d], 3600 * 24 * 5)\n * on (instance) group_left (nodename) node_uname_info < 0\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.8.4\n severity: warning\n type: ceph_default\nannotations:\n description: Mountpoint {{ $labels.mountpoint }} on {{ $labels.nodename }} will\n be full in less than 5 days based on the 48 hour trailing fill rate.\n summary: Host filesystem free space is getting low\n" err="found duplicate series for the match group {instance=\"smithi100\"} on the right hand-side of the operation: [{__name__=\"node_uname_info\", domainname=\"(none)\", instance=\"smithi100\", job=\"node\", machine=\"x86_64\", nodename=\"smithi100\", release=\"5.15.0-101-generic\", sysname=\"Linux\", version=\"#111-Ubuntu SMP Tue Mar 5 20:16:58 UTC 2024\"}, {__name__=\"node_uname_info\", cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", domainname=\"(none)\", instance=\"smithi100\", job=\"node\", machine=\"x86_64\", nodename=\"smithi100\", release=\"5.15.0-101-generic\", sysname=\"Linux\", version=\"#111-Ubuntu SMP Tue Mar 5 20:16:58 UTC 2024\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:22:33.848 INFO:tasks.rados.rados.0.smithi100.stdout:1777: done (4 left) 2024-04-07T16:22:33.848 INFO:tasks.rados.rados.0.smithi100.stdout:1778: done (3 left) 2024-04-07T16:22:33.849 INFO:tasks.rados.rados.0.smithi100.stdout:1779: done (2 left) 2024-04-07T16:22:33.849 INFO:tasks.rados.rados.0.smithi100.stdout:1781: done (1 left) 2024-04-07T16:22:33.849 INFO:tasks.rados.rados.0.smithi100.stdout:1782: done (0 left) 2024-04-07T16:22:33.849 INFO:tasks.rados.rados.0.smithi100.stdout:1783: rollback oid 5 current snap is 165 2024-04-07T16:22:33.849 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 5 to 163 2024-04-07T16:22:33.849 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 46 current snap is 165 2024-04-07T16:22:33.849 INFO:tasks.rados.rados.0.smithi100.stdout:1784: seq_num 545 ranges {6103040=737280,6840320=491520,7331840=450560,7782400=262144} 2024-04-07T16:22:33.880 INFO:tasks.rados.rados.0.smithi100.stdout:1784: writing smithi10066867-46 from 6103040 to 6840320 tid 1 2024-04-07T16:22:33.883 INFO:tasks.rados.rados.0.smithi100.stdout:1784: writing smithi10066867-46 from 6840320 to 7331840 tid 2 2024-04-07T16:22:33.886 INFO:tasks.rados.rados.0.smithi100.stdout:1784: writing smithi10066867-46 from 7331840 to 7782400 tid 3 2024-04-07T16:22:33.887 INFO:tasks.rados.rados.0.smithi100.stdout:1784: writing smithi10066867-46 from 7782400 to 8044544 tid 4 2024-04-07T16:22:33.887 INFO:tasks.rados.rados.0.smithi100.stdout:1785: snap_create 2024-04-07T16:22:34.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:33 smithi144 bash[66654]: cluster 2024-04-07T16:22:32.654371+0000 mgr.y (mgr.25175) 137 : cluster [DBG] pgmap v193: 121 pgs: 121 active+clean; 162 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 2.0 MiB/s rd, 7.4 MiB/s wr, 26 op/s 2024-04-07T16:22:34.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:33 smithi144 bash[66654]: cluster 2024-04-07T16:22:32.846246+0000 mon.a (mon.0) 424 : cluster [DBG] osdmap e403: 8 total, 8 up, 8 in 2024-04-07T16:22:34.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:33 smithi144 bash[66654]: cluster 2024-04-07T16:22:33.441769+0000 mon.a (mon.0) 425 : cluster [INF] osd.1 marked itself down and dead 2024-04-07T16:22:34.184 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:33 smithi100 bash[76202]: cluster 2024-04-07T16:22:32.654371+0000 mgr.y (mgr.25175) 137 : cluster [DBG] pgmap v193: 121 pgs: 121 active+clean; 162 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 2.0 MiB/s rd, 7.4 MiB/s wr, 26 op/s 2024-04-07T16:22:34.184 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:33 smithi100 bash[76202]: cluster 2024-04-07T16:22:32.846246+0000 mon.a (mon.0) 424 : cluster [DBG] osdmap e403: 8 total, 8 up, 8 in 2024-04-07T16:22:34.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:33 smithi100 bash[76202]: cluster 2024-04-07T16:22:33.441769+0000 mon.a (mon.0) 425 : cluster [INF] osd.1 marked itself down and dead 2024-04-07T16:22:34.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:33 smithi100 bash[74942]: cluster 2024-04-07T16:22:32.654371+0000 mgr.y (mgr.25175) 137 : cluster [DBG] pgmap v193: 121 pgs: 121 active+clean; 162 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 2.0 MiB/s rd, 7.4 MiB/s wr, 26 op/s 2024-04-07T16:22:34.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:33 smithi100 bash[74942]: cluster 2024-04-07T16:22:32.846246+0000 mon.a (mon.0) 424 : cluster [DBG] osdmap e403: 8 total, 8 up, 8 in 2024-04-07T16:22:34.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:33 smithi100 bash[74942]: cluster 2024-04-07T16:22:33.441769+0000 mon.a (mon.0) 425 : cluster [INF] osd.1 marked itself down and dead 2024-04-07T16:22:34.831 INFO:tasks.rados.rados.0.smithi100.stdout:1785: done (2 left) 2024-04-07T16:22:34.831 INFO:tasks.rados.rados.0.smithi100.stdout:1786: snap_remove snap 165 2024-04-07T16:22:35.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:34 smithi144 bash[66654]: cluster 2024-04-07T16:22:33.836965+0000 mon.a (mon.0) 426 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN) 2024-04-07T16:22:35.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:34 smithi144 bash[66654]: cluster 2024-04-07T16:22:33.847064+0000 mon.a (mon.0) 427 : cluster [DBG] osdmap e404: 8 total, 7 up, 8 in 2024-04-07T16:22:35.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:34 smithi144 bash[66654]: cluster 2024-04-07T16:22:34.848489+0000 mon.a (mon.0) 428 : cluster [DBG] osdmap e405: 8 total, 7 up, 8 in 2024-04-07T16:22:35.185 INFO:journalctl@ceph.osd.1.smithi100.stdout:Apr 07 16:22:34 smithi100 bash[84796]: ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4-osd-1 2024-04-07T16:22:35.185 INFO:journalctl@ceph.osd.1.smithi100.stdout:Apr 07 16:22:35 smithi100 bash[84859]: Error response from daemon: No such container: ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4-osd-1 2024-04-07T16:22:35.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:34 smithi100 bash[74942]: cluster 2024-04-07T16:22:33.836965+0000 mon.a (mon.0) 426 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN) 2024-04-07T16:22:35.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:34 smithi100 bash[74942]: cluster 2024-04-07T16:22:33.847064+0000 mon.a (mon.0) 427 : cluster [DBG] osdmap e404: 8 total, 7 up, 8 in 2024-04-07T16:22:35.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:34 smithi100 bash[74942]: cluster 2024-04-07T16:22:34.848489+0000 mon.a (mon.0) 428 : cluster [DBG] osdmap e405: 8 total, 7 up, 8 in 2024-04-07T16:22:35.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:34 smithi100 bash[76202]: cluster 2024-04-07T16:22:33.836965+0000 mon.a (mon.0) 426 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN) 2024-04-07T16:22:35.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:34 smithi100 bash[76202]: cluster 2024-04-07T16:22:33.847064+0000 mon.a (mon.0) 427 : cluster [DBG] osdmap e404: 8 total, 7 up, 8 in 2024-04-07T16:22:35.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:34 smithi100 bash[76202]: cluster 2024-04-07T16:22:34.848489+0000 mon.a (mon.0) 428 : cluster [DBG] osdmap e405: 8 total, 7 up, 8 in 2024-04-07T16:22:35.829 INFO:tasks.rados.rados.0.smithi100.stdout:1786: done (2 left) 2024-04-07T16:22:35.829 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 9 current snap is 166 2024-04-07T16:22:35.829 INFO:tasks.rados.rados.0.smithi100.stdout:1783: finishing rollback tid 0 to smithi10066867-5 2024-04-07T16:22:35.830 INFO:tasks.rados.rados.0.smithi100.stdout:1787: seq_num 546 ranges {860160=753664,1613824=778240,2392064=425984} 2024-04-07T16:22:35.839 INFO:tasks.rados.rados.0.smithi100.stdout:1787: writing smithi10066867-9 from 860160 to 1613824 tid 1 2024-04-07T16:22:35.844 INFO:tasks.rados.rados.0.smithi100.stdout:1787: writing smithi10066867-9 from 1613824 to 2392064 tid 2 2024-04-07T16:22:35.847 INFO:tasks.rados.rados.0.smithi100.stdout:1787: writing smithi10066867-9 from 2392064 to 2818048 tid 3 2024-04-07T16:22:35.847 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 47 current snap is 166 2024-04-07T16:22:35.847 INFO:tasks.rados.rados.0.smithi100.stdout:1788: seq_num 547 ranges {4841472=770048,5611520=663552,6275072=466944,6742016=73728} 2024-04-07T16:22:35.879 INFO:tasks.rados.rados.0.smithi100.stdout:1788: writing smithi10066867-47 from 4841472 to 5611520 tid 1 2024-04-07T16:22:35.883 INFO:tasks.rados.rados.0.smithi100.stdout:1788: writing smithi10066867-47 from 5611520 to 6275072 tid 2 2024-04-07T16:22:35.886 INFO:tasks.rados.rados.0.smithi100.stdout:1788: writing smithi10066867-47 from 6275072 to 6742016 tid 3 2024-04-07T16:22:35.887 INFO:tasks.rados.rados.0.smithi100.stdout:1788: writing smithi10066867-47 from 6742016 to 6815744 tid 4 2024-04-07T16:22:35.887 INFO:tasks.rados.rados.0.smithi100.stdout:1789: copy_from oid 37 from oid 11 current snap is 166 2024-04-07T16:22:35.887 INFO:tasks.rados.rados.0.smithi100.stdout:1790: read oid 42 snap -1 2024-04-07T16:22:35.887 INFO:tasks.rados.rados.0.smithi100.stdout:1790: expect (ObjNum 533 snap 161 seq_num 533) 2024-04-07T16:22:35.887 INFO:tasks.rados.rados.0.smithi100.stdout:1791: setattr oid 6 current snap is 166 2024-04-07T16:22:35.888 INFO:tasks.rados.rados.0.smithi100.stdout:1783: finishing rollback tid 1 to smithi10066867-5 2024-04-07T16:22:35.888 INFO:tasks.rados.rados.0.smithi100.stdout:1783: finishing rollback tid 2 to smithi10066867-5 2024-04-07T16:22:35.888 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 5 v 1011 (ObjNum 468 snap 142 seq_num 468) dirty exists 2024-04-07T16:22:35.888 INFO:tasks.rados.rados.0.smithi100.stdout:1784: finishing write tid 1 to smithi10066867-46 2024-04-07T16:22:35.888 INFO:tasks.rados.rados.0.smithi100.stdout:1784: finishing write tid 2 to smithi10066867-46 2024-04-07T16:22:35.889 INFO:tasks.rados.rados.0.smithi100.stdout:1784: finishing write tid 3 to smithi10066867-46 2024-04-07T16:22:35.889 INFO:tasks.rados.rados.0.smithi100.stdout:1784: finishing write tid 4 to smithi10066867-46 2024-04-07T16:22:35.889 INFO:tasks.rados.rados.0.smithi100.stdout:1784: finishing write tid 5 to smithi10066867-46 2024-04-07T16:22:35.889 INFO:tasks.rados.rados.0.smithi100.stdout:1784: finishing write tid 6 to smithi10066867-46 2024-04-07T16:22:35.889 INFO:tasks.rados.rados.0.smithi100.stdout:1784: oid 46 updating version 0 to 724 2024-04-07T16:22:35.889 INFO:tasks.rados.rados.0.smithi100.stdout:1784: oid 46 updating version 724 to 726 2024-04-07T16:22:35.889 INFO:tasks.rados.rados.0.smithi100.stdout:1784: oid 46 version 726 is already newer than 723 2024-04-07T16:22:35.889 INFO:tasks.rados.rados.0.smithi100.stdout:1784: oid 46 version 726 is already newer than 725 2024-04-07T16:22:35.889 INFO:tasks.rados.rados.0.smithi100.stdout:1784: oid 46 version 726 is already newer than 722 2024-04-07T16:22:35.889 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 46 v 726 (ObjNum 545 snap 165 seq_num 545) dirty exists 2024-04-07T16:22:35.889 INFO:tasks.rados.rados.0.smithi100.stdout:1784: left oid 46 (ObjNum 545 snap 165 seq_num 545) 2024-04-07T16:22:35.889 INFO:tasks.rados.rados.0.smithi100.stdout:1787: finishing write tid 1 to smithi10066867-9 2024-04-07T16:22:35.889 INFO:tasks.rados.rados.0.smithi100.stdout:1787: finishing write tid 2 to smithi10066867-9 2024-04-07T16:22:35.889 INFO:tasks.rados.rados.0.smithi100.stdout:1787: finishing write tid 3 to smithi10066867-9 2024-04-07T16:22:35.889 INFO:tasks.rados.rados.0.smithi100.stdout:1787: finishing write tid 4 to smithi10066867-9 2024-04-07T16:22:35.890 INFO:tasks.rados.rados.0.smithi100.stdout:1787: finishing write tid 5 to smithi10066867-9 2024-04-07T16:22:35.890 INFO:tasks.rados.rados.0.smithi100.stdout:1787: oid 9 updating version 0 to 544 2024-04-07T16:22:35.890 INFO:tasks.rados.rados.0.smithi100.stdout:1787: oid 9 updating version 544 to 546 2024-04-07T16:22:35.890 INFO:tasks.rados.rados.0.smithi100.stdout:1787: oid 9 version 546 is already newer than 545 2024-04-07T16:22:35.890 INFO:tasks.rados.rados.0.smithi100.stdout:1787: oid 9 updating version 546 to 547 2024-04-07T16:22:35.890 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 9 v 547 (ObjNum 546 snap 166 seq_num 546) dirty exists 2024-04-07T16:22:35.890 INFO:tasks.rados.rados.0.smithi100.stdout:1787: left oid 9 (ObjNum 546 snap 166 seq_num 546) 2024-04-07T16:22:35.891 INFO:tasks.rados.rados.0.smithi100.stdout:1783: done (6 left) 2024-04-07T16:22:35.891 INFO:tasks.rados.rados.0.smithi100.stdout:1784: done (5 left) 2024-04-07T16:22:35.891 INFO:tasks.rados.rados.0.smithi100.stdout:1787: done (4 left) 2024-04-07T16:22:35.891 INFO:tasks.rados.rados.0.smithi100.stdout:1792: read oid 33 snap -1 2024-04-07T16:22:35.891 INFO:tasks.rados.rados.0.smithi100.stdout:1792: expect deleted 2024-04-07T16:22:35.891 INFO:tasks.rados.rados.0.smithi100.stdout:1793: delete oid 24 current snap is 166 2024-04-07T16:22:35.895 INFO:tasks.rados.rados.0.smithi100.stdout:1792: done (5 left) 2024-04-07T16:22:35.895 INFO:tasks.rados.rados.0.smithi100.stdout:1793: done (4 left) 2024-04-07T16:22:35.895 INFO:tasks.rados.rados.0.smithi100.stdout:1794: snap_remove snap 164 2024-04-07T16:22:36.184 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:35 smithi100 bash[74942]: cluster 2024-04-07T16:22:34.655085+0000 mgr.y (mgr.25175) 138 : cluster [DBG] pgmap v196: 121 pgs: 16 stale+active+clean, 105 active+clean; 159 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 3.2 MiB/s rd, 8.1 MiB/s wr, 28 op/s 2024-04-07T16:22:36.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:35 smithi100 bash[74942]: cluster 2024-04-07T16:22:35.847459+0000 mon.a (mon.0) 429 : cluster [DBG] osdmap e406: 8 total, 7 up, 8 in 2024-04-07T16:22:36.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:35 smithi100 bash[76202]: cluster 2024-04-07T16:22:34.655085+0000 mgr.y (mgr.25175) 138 : cluster [DBG] pgmap v196: 121 pgs: 16 stale+active+clean, 105 active+clean; 159 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 3.2 MiB/s rd, 8.1 MiB/s wr, 28 op/s 2024-04-07T16:22:36.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:35 smithi100 bash[76202]: cluster 2024-04-07T16:22:35.847459+0000 mon.a (mon.0) 429 : cluster [DBG] osdmap e406: 8 total, 7 up, 8 in 2024-04-07T16:22:36.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:35 smithi144 bash[66654]: cluster 2024-04-07T16:22:34.655085+0000 mgr.y (mgr.25175) 138 : cluster [DBG] pgmap v196: 121 pgs: 16 stale+active+clean, 105 active+clean; 159 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 3.2 MiB/s rd, 8.1 MiB/s wr, 28 op/s 2024-04-07T16:22:36.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:35 smithi144 bash[66654]: cluster 2024-04-07T16:22:35.847459+0000 mon.a (mon.0) 429 : cluster [DBG] osdmap e406: 8 total, 7 up, 8 in 2024-04-07T16:22:36.790 INFO:tasks.rados.rados.0.smithi100.stdout:1794: done (4 left) 2024-04-07T16:22:36.790 INFO:tasks.rados.rados.0.smithi100.stdout:1795: rollback oid 48 current snap is 166 2024-04-07T16:22:36.790 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 48 to 163 2024-04-07T16:22:36.790 INFO:tasks.rados.rados.0.smithi100.stdout:1790: expect (ObjNum 533 snap 161 seq_num 533) 2024-04-07T16:22:36.841 INFO:tasks.rados.rados.0.smithi100.stdout:1789: finishing copy_from racing read to smithi10066867-37 2024-04-07T16:22:36.841 INFO:tasks.rados.rados.0.smithi100.stdout:1788: finishing write tid 1 to smithi10066867-47 2024-04-07T16:22:36.841 INFO:tasks.rados.rados.0.smithi100.stdout:1788: finishing write tid 2 to smithi10066867-47 2024-04-07T16:22:36.841 INFO:tasks.rados.rados.0.smithi100.stdout:1790: done (4 left) 2024-04-07T16:22:36.841 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 21 current snap is 166 2024-04-07T16:22:36.841 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 6 v 842 (ObjNum 524 snap 156 seq_num 524) dirty exists 2024-04-07T16:22:36.841 INFO:tasks.rados.rados.0.smithi100.stdout:1788: finishing write tid 3 to smithi10066867-47 2024-04-07T16:22:36.841 INFO:tasks.rados.rados.0.smithi100.stdout:1796: seq_num 549 ranges {1761280=802816,2564096=483328,3047424=434176,3481600=286720} 2024-04-07T16:22:36.855 INFO:tasks.rados.rados.0.smithi100.stdout:1796: writing smithi10066867-21 from 1761280 to 2564096 tid 1 2024-04-07T16:22:36.858 INFO:tasks.rados.rados.0.smithi100.stdout:1796: writing smithi10066867-21 from 2564096 to 3047424 tid 2 2024-04-07T16:22:36.860 INFO:tasks.rados.rados.0.smithi100.stdout:1796: writing smithi10066867-21 from 3047424 to 3481600 tid 3 2024-04-07T16:22:36.862 INFO:tasks.rados.rados.0.smithi100.stdout:1796: writing smithi10066867-21 from 3481600 to 3768320 tid 4 2024-04-07T16:22:36.862 INFO:tasks.rados.rados.0.smithi100.stdout:1791: done (4 left) 2024-04-07T16:22:36.862 INFO:tasks.rados.rados.0.smithi100.stdout:1797: delete oid 43 current snap is 166 2024-04-07T16:22:36.862 INFO:tasks.rados.rados.0.smithi100.stdout:1788: finishing write tid 4 to smithi10066867-47 2024-04-07T16:22:36.863 INFO:tasks.rados.rados.0.smithi100.stdout:1788: finishing write tid 5 to smithi10066867-47 2024-04-07T16:22:36.863 INFO:tasks.rados.rados.0.smithi100.stdout:1789: finishing copy_from to smithi10066867-37 2024-04-07T16:22:36.863 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 37 v 1045 (ObjNum 501 snap 150 seq_num 501) dirty exists 2024-04-07T16:22:36.863 INFO:tasks.rados.rados.0.smithi100.stdout:1788: finishing write tid 6 to smithi10066867-47 2024-04-07T16:22:36.863 INFO:tasks.rados.rados.0.smithi100.stdout:1788: oid 47 updating version 0 to 553 2024-04-07T16:22:36.863 INFO:tasks.rados.rados.0.smithi100.stdout:1788: oid 47 version 553 is already newer than 549 2024-04-07T16:22:36.863 INFO:tasks.rados.rados.0.smithi100.stdout:1788: oid 47 version 553 is already newer than 551 2024-04-07T16:22:36.863 INFO:tasks.rados.rados.0.smithi100.stdout:1788: oid 47 version 553 is already newer than 552 2024-04-07T16:22:36.863 INFO:tasks.rados.rados.0.smithi100.stdout:1788: oid 47 version 553 is already newer than 550 2024-04-07T16:22:36.863 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 47 v 553 (ObjNum 547 snap 166 seq_num 547) dirty exists 2024-04-07T16:22:36.863 INFO:tasks.rados.rados.0.smithi100.stdout:1788: left oid 47 (ObjNum 547 snap 166 seq_num 547) 2024-04-07T16:22:36.863 INFO:tasks.rados.rados.0.smithi100.stdout:1795: finishing rollback tid 0 to smithi10066867-48 2024-04-07T16:22:36.863 INFO:tasks.rados.rados.0.smithi100.stdout:1795: finishing rollback tid 1 to smithi10066867-48 2024-04-07T16:22:36.863 INFO:tasks.rados.rados.0.smithi100.stdout:1795: finishing rollback tid 2 to smithi10066867-48 2024-04-07T16:22:36.863 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 48 v 998 (ObjNum 510 snap 152 seq_num 510) dirty exists 2024-04-07T16:22:36.867 INFO:tasks.rados.rados.0.smithi100.stdout:1788: done (4 left) 2024-04-07T16:22:36.867 INFO:tasks.rados.rados.0.smithi100.stdout:1789: done (3 left) 2024-04-07T16:22:36.867 INFO:tasks.rados.rados.0.smithi100.stdout:1795: done (2 left) 2024-04-07T16:22:36.867 INFO:tasks.rados.rados.0.smithi100.stdout:1797: done (1 left) 2024-04-07T16:22:36.867 INFO:tasks.rados.rados.0.smithi100.stdout:1798: delete oid 19 current snap is 166 2024-04-07T16:22:36.869 INFO:tasks.rados.rados.0.smithi100.stdout:1796: finishing write tid 1 to smithi10066867-21 2024-04-07T16:22:36.869 INFO:tasks.rados.rados.0.smithi100.stdout:1796: finishing write tid 2 to smithi10066867-21 2024-04-07T16:22:36.873 INFO:tasks.rados.rados.0.smithi100.stdout:1798: done (1 left) 2024-04-07T16:22:36.873 INFO:tasks.rados.rados.0.smithi100.stdout:1799: read oid 29 snap -1 2024-04-07T16:22:36.873 INFO:tasks.rados.rados.0.smithi100.stdout:1799: expect (ObjNum 509 snap 152 seq_num 509) 2024-04-07T16:22:36.873 INFO:tasks.rados.rados.0.smithi100.stdout:1800: copy_from oid 1 from oid 45 current snap is 166 2024-04-07T16:22:36.873 INFO:tasks.rados.rados.0.smithi100.stdout:1801: rollback oid 28 current snap is 166 2024-04-07T16:22:36.873 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 28 to 163 2024-04-07T16:22:36.873 INFO:tasks.rados.rados.0.smithi100.stdout:1802: snap_create 2024-04-07T16:22:36.873 INFO:tasks.rados.rados.0.smithi100.stdout:1796: finishing write tid 3 to smithi10066867-21 2024-04-07T16:22:36.876 INFO:tasks.rados.rados.0.smithi100.stdout:1800: finishing copy_from to smithi10066867-1 2024-04-07T16:22:36.876 INFO:tasks.rados.rados.0.smithi100.stdout:1800: got expected ENOENT (src dne) 2024-04-07T16:22:36.876 INFO:tasks.rados.rados.0.smithi100.stdout:1796: finishing write tid 4 to smithi10066867-21 2024-04-07T16:22:36.876 INFO:tasks.rados.rados.0.smithi100.stdout:1800: finishing copy_from racing read to smithi10066867-1 2024-04-07T16:22:36.876 INFO:tasks.rados.rados.0.smithi100.stdout:1796: finishing write tid 5 to smithi10066867-21 2024-04-07T16:22:36.879 INFO:tasks.rados.rados.0.smithi100.stdout:1796: finishing write tid 6 to smithi10066867-21 2024-04-07T16:22:36.880 INFO:tasks.rados.rados.0.smithi100.stdout:1796: oid 21 updating version 0 to 557 2024-04-07T16:22:36.880 INFO:tasks.rados.rados.0.smithi100.stdout:1796: oid 21 updating version 557 to 558 2024-04-07T16:22:36.880 INFO:tasks.rados.rados.0.smithi100.stdout:1796: oid 21 version 558 is already newer than 554 2024-04-07T16:22:36.880 INFO:tasks.rados.rados.0.smithi100.stdout:1796: oid 21 version 558 is already newer than 555 2024-04-07T16:22:36.880 INFO:tasks.rados.rados.0.smithi100.stdout:1796: oid 21 version 558 is already newer than 556 2024-04-07T16:22:36.880 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 21 v 558 (ObjNum 549 snap 166 seq_num 549) dirty exists 2024-04-07T16:22:36.880 INFO:tasks.rados.rados.0.smithi100.stdout:1796: left oid 21 (ObjNum 549 snap 166 seq_num 549) 2024-04-07T16:22:36.880 INFO:tasks.rados.rados.0.smithi100.stdout:1801: finishing rollback tid 0 to smithi10066867-28 2024-04-07T16:22:36.881 INFO:tasks.rados.rados.0.smithi100.stdout:1801: finishing rollback tid 1 to smithi10066867-28 2024-04-07T16:22:36.881 INFO:tasks.rados.rados.0.smithi100.stdout:1801: finishing rollback tid 2 to smithi10066867-28 2024-04-07T16:22:36.881 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 28 v 766 (ObjNum 461 snap 140 seq_num 461) dirty exists 2024-04-07T16:22:36.903 INFO:tasks.rados.rados.0.smithi100.stdout:1799: expect (ObjNum 509 snap 152 seq_num 509) 2024-04-07T16:22:37.152 INFO:journalctl@ceph.osd.2.smithi100.stdout:Apr 07 16:22:37 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:22:37.153 INFO:journalctl@ceph.osd.3.smithi100.stdout:Apr 07 16:22:37 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:22:37.153 INFO:journalctl@ceph.mds.a.smithi100.stdout:Apr 07 16:22:37 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:22:37.153 INFO:journalctl@ceph.osd.1.smithi100.stdout:Apr 07 16:22:36 smithi100 systemd[1]: ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@osd.1.service: Deactivated successfully. 2024-04-07T16:22:37.153 INFO:journalctl@ceph.osd.1.smithi100.stdout:Apr 07 16:22:36 smithi100 systemd[1]: Stopped Ceph osd.1 for 0702e3a8-f4f8-11ee-b649-cb9ed24678a4. 2024-04-07T16:22:37.153 INFO:journalctl@ceph.osd.1.smithi100.stdout:Apr 07 16:22:37 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:22:37.154 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:37 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:22:37.154 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:22:37 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:22:37.154 INFO:journalctl@ceph.osd.0.smithi100.stdout:Apr 07 16:22:37 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:22:37.155 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 16:22:37 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:22:37.155 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Apr 07 16:22:37 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:22:37.155 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:37 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:22:37.434 INFO:journalctl@ceph.osd.1.smithi100.stdout:Apr 07 16:22:37 smithi100 systemd[1]: Started Ceph osd.1 for 0702e3a8-f4f8-11ee-b649-cb9ed24678a4. 2024-04-07T16:22:37.872 INFO:tasks.rados.rados.0.smithi100.stdout:1796: done (4 left) 2024-04-07T16:22:37.872 INFO:tasks.rados.rados.0.smithi100.stdout:1799: done (3 left) 2024-04-07T16:22:37.873 INFO:tasks.rados.rados.0.smithi100.stdout:1800: done (2 left) 2024-04-07T16:22:37.873 INFO:tasks.rados.rados.0.smithi100.stdout:1801: done (1 left) 2024-04-07T16:22:37.873 INFO:tasks.rados.rados.0.smithi100.stdout:1802: done (0 left) 2024-04-07T16:22:37.873 INFO:tasks.rados.rados.0.smithi100.stdout:1803: rollback oid 7 current snap is 167 2024-04-07T16:22:37.873 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 7 to 166 2024-04-07T16:22:37.873 INFO:tasks.rados.rados.0.smithi100.stdout:1804: snap_remove snap 163 2024-04-07T16:22:38.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:37 smithi144 bash[66654]: cluster 2024-04-07T16:22:36.655780+0000 mgr.y (mgr.25175) 139 : cluster [DBG] pgmap v199: 121 pgs: 16 stale+active+clean, 105 active+clean; 159 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 1.2 MiB/s rd, 704 KiB/s wr, 2 op/s 2024-04-07T16:22:38.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:37 smithi144 bash[66654]: cluster 2024-04-07T16:22:36.806528+0000 mon.a (mon.0) 430 : cluster [DBG] osdmap e407: 8 total, 7 up, 8 in 2024-04-07T16:22:38.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:37 smithi144 bash[66654]: audit 2024-04-07T16:22:37.215676+0000 mon.a (mon.0) 431 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:22:38.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:37 smithi144 bash[66654]: audit 2024-04-07T16:22:37.223472+0000 mon.a (mon.0) 432 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:22:38.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:37 smithi144 bash[66654]: audit 2024-04-07T16:22:37.273020+0000 mon.a (mon.0) 433 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:22:38.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:37 smithi100 bash[74942]: cluster 2024-04-07T16:22:36.655780+0000 mgr.y (mgr.25175) 139 : cluster [DBG] pgmap v199: 121 pgs: 16 stale+active+clean, 105 active+clean; 159 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 1.2 MiB/s rd, 704 KiB/s wr, 2 op/s 2024-04-07T16:22:38.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:37 smithi100 bash[74942]: cluster 2024-04-07T16:22:36.806528+0000 mon.a (mon.0) 430 : cluster [DBG] osdmap e407: 8 total, 7 up, 8 in 2024-04-07T16:22:38.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:37 smithi100 bash[74942]: audit 2024-04-07T16:22:37.215676+0000 mon.a (mon.0) 431 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:22:38.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:37 smithi100 bash[74942]: audit 2024-04-07T16:22:37.223472+0000 mon.a (mon.0) 432 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:22:38.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:37 smithi100 bash[74942]: audit 2024-04-07T16:22:37.273020+0000 mon.a (mon.0) 433 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:22:38.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:37 smithi100 bash[76202]: cluster 2024-04-07T16:22:36.655780+0000 mgr.y (mgr.25175) 139 : cluster [DBG] pgmap v199: 121 pgs: 16 stale+active+clean, 105 active+clean; 159 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 1.2 MiB/s rd, 704 KiB/s wr, 2 op/s 2024-04-07T16:22:38.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:37 smithi100 bash[76202]: cluster 2024-04-07T16:22:36.806528+0000 mon.a (mon.0) 430 : cluster [DBG] osdmap e407: 8 total, 7 up, 8 in 2024-04-07T16:22:38.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:37 smithi100 bash[76202]: audit 2024-04-07T16:22:37.215676+0000 mon.a (mon.0) 431 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:22:38.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:37 smithi100 bash[76202]: audit 2024-04-07T16:22:37.223472+0000 mon.a (mon.0) 432 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:22:38.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:37 smithi100 bash[76202]: audit 2024-04-07T16:22:37.273020+0000 mon.a (mon.0) 433 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:22:38.684 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:22:38 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:22:38] "GET /metrics HTTP/1.1" 200 36147 "" "Prometheus/2.43.0" 2024-04-07T16:22:38.685 INFO:journalctl@ceph.osd.1.smithi100.stdout:Apr 07 16:22:38 smithi100 bash[85011]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-07T16:22:38.685 INFO:journalctl@ceph.osd.1.smithi100.stdout:Apr 07 16:22:38 smithi100 bash[85011]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-1 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-04-07T16:22:38.685 INFO:journalctl@ceph.osd.1.smithi100.stdout:Apr 07 16:22:38 smithi100 bash[85011]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-04-07T16:22:38.685 INFO:journalctl@ceph.osd.1.smithi100.stdout:Apr 07 16:22:38 smithi100 bash[85011]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-04-07T16:22:38.685 INFO:journalctl@ceph.osd.1.smithi100.stdout:Apr 07 16:22:38 smithi100 bash[85011]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-1/block 2024-04-07T16:22:38.685 INFO:journalctl@ceph.osd.1.smithi100.stdout:Apr 07 16:22:38 smithi100 bash[85011]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-07T16:22:38.685 INFO:journalctl@ceph.osd.1.smithi100.stdout:Apr 07 16:22:38 smithi100 bash[85011]: --> ceph-volume raw activate successful for osd ID: 1 2024-04-07T16:22:38.891 INFO:tasks.rados.rados.0.smithi100.stdout:1804: done (1 left) 2024-04-07T16:22:38.943 INFO:tasks.rados.rados.0.smithi100.stdout:1805: rollback oid 16 current snap is 167 2024-04-07T16:22:38.943 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 16 to 166 2024-04-07T16:22:38.943 INFO:tasks.rados.rados.0.smithi100.stdout:1803: finishing rollback tid 0 to smithi10066867-7 2024-04-07T16:22:38.943 INFO:tasks.rados.rados.0.smithi100.stdout:1803: finishing rollback tid 1 to smithi10066867-7 2024-04-07T16:22:38.943 INFO:tasks.rados.rados.0.smithi100.stdout:1803: finishing rollback tid 2 to smithi10066867-7 2024-04-07T16:22:38.943 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 7 v 804 (ObjNum 529 snap 160 seq_num 529) dirty exists 2024-04-07T16:22:38.944 INFO:tasks.rados.rados.0.smithi100.stdout:1803: done (1 left) 2024-04-07T16:22:38.944 INFO:tasks.rados.rados.0.smithi100.stdout:1806: rollback oid 17 current snap is 167 2024-04-07T16:22:38.944 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 17 to 166 2024-04-07T16:22:38.944 INFO:tasks.rados.rados.0.smithi100.stdout:1807: read oid 4 snap -1 2024-04-07T16:22:38.944 INFO:tasks.rados.rados.0.smithi100.stdout:1807: expect (ObjNum 521 snap 156 seq_num 521) 2024-04-07T16:22:38.944 INFO:tasks.rados.rados.0.smithi100.stdout:1808: snap_create 2024-04-07T16:22:38.945 INFO:tasks.rados.rados.0.smithi100.stdout:1806: finishing rollback tid 0 to smithi10066867-17 2024-04-07T16:22:38.945 INFO:tasks.rados.rados.0.smithi100.stdout:1806: finishing rollback tid 1 to smithi10066867-17 2024-04-07T16:22:38.945 INFO:tasks.rados.rados.0.smithi100.stdout:1806: finishing rollback tid 2 to smithi10066867-17 2024-04-07T16:22:38.945 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 17 v 728 (ObjNum 506 snap 151 seq_num 506) dirty exists 2024-04-07T16:22:38.945 INFO:tasks.rados.rados.0.smithi100.stdout:1805: finishing rollback tid 0 to smithi10066867-16 2024-04-07T16:22:38.945 INFO:tasks.rados.rados.0.smithi100.stdout:1805: finishing rollback tid 1 to smithi10066867-16 2024-04-07T16:22:38.945 INFO:tasks.rados.rados.0.smithi100.stdout:1805: finishing rollback tid 2 to smithi10066867-16 2024-04-07T16:22:38.945 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 16 v 1014 (ObjNum 475 snap 143 seq_num 475) dirty exists 2024-04-07T16:22:38.945 INFO:tasks.rados.rados.0.smithi100.stdout:1807: expect (ObjNum 521 snap 156 seq_num 521) 2024-04-07T16:22:39.184 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:38 smithi100 bash[74942]: cluster 2024-04-07T16:22:37.889691+0000 mon.a (mon.0) 434 : cluster [DBG] osdmap e408: 8 total, 7 up, 8 in 2024-04-07T16:22:39.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:38 smithi100 bash[76202]: cluster 2024-04-07T16:22:37.889691+0000 mon.a (mon.0) 434 : cluster [DBG] osdmap e408: 8 total, 7 up, 8 in 2024-04-07T16:22:39.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:38 smithi144 bash[66654]: cluster 2024-04-07T16:22:37.889691+0000 mon.a (mon.0) 434 : cluster [DBG] osdmap e408: 8 total, 7 up, 8 in 2024-04-07T16:22:39.896 INFO:tasks.rados.rados.0.smithi100.stdout:1805: done (3 left) 2024-04-07T16:22:39.896 INFO:tasks.rados.rados.0.smithi100.stdout:1806: done (2 left) 2024-04-07T16:22:39.896 INFO:tasks.rados.rados.0.smithi100.stdout:1807: done (1 left) 2024-04-07T16:22:39.896 INFO:tasks.rados.rados.0.smithi100.stdout:1808: done (0 left) 2024-04-07T16:22:39.896 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 16 current snap is 168 2024-04-07T16:22:39.896 INFO:tasks.rados.rados.0.smithi100.stdout:1809: seq_num 552 ranges {761856=450560,1212416=532480,1744896=655360,2400256=417792} 2024-04-07T16:22:39.916 INFO:tasks.rados.rados.0.smithi100.stdout:1809: writing smithi10066867-16 from 761856 to 1212416 tid 1 2024-04-07T16:22:39.925 INFO:tasks.rados.rados.0.smithi100.stdout:1809: writing smithi10066867-16 from 1212416 to 1744896 tid 2 2024-04-07T16:22:39.936 INFO:tasks.rados.rados.0.smithi100.stdout:1809: writing smithi10066867-16 from 1744896 to 2400256 tid 3 2024-04-07T16:22:39.943 INFO:tasks.rados.rados.0.smithi100.stdout:1809: writing smithi10066867-16 from 2400256 to 2818048 tid 4 2024-04-07T16:22:39.943 INFO:tasks.rados.rados.0.smithi100.stdout:1810: copy_from oid 50 from oid 48 current snap is 168 2024-04-07T16:22:39.943 INFO:tasks.rados.rados.0.smithi100.stdout:1809: finishing write tid 1 to smithi10066867-16 2024-04-07T16:22:39.943 INFO:tasks.rados.rados.0.smithi100.stdout:1809: finishing write tid 2 to smithi10066867-16 2024-04-07T16:22:39.943 INFO:tasks.rados.rados.0.smithi100.stdout:1809: finishing write tid 3 to smithi10066867-16 2024-04-07T16:22:39.943 INFO:tasks.rados.rados.0.smithi100.stdout:1811: copy_from oid 35 from oid 37 current snap is 168 2024-04-07T16:22:39.943 INFO:tasks.rados.rados.0.smithi100.stdout:1812: rollback oid 8 current snap is 168 2024-04-07T16:22:39.943 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 8 to 166 2024-04-07T16:22:39.943 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 12 current snap is 168 2024-04-07T16:22:39.943 INFO:tasks.rados.rados.0.smithi100.stdout:1813: seq_num 553 ranges {2301952=466944,2768896=417792,3186688=417792,3604480=516096,4120576=253952} 2024-04-07T16:22:39.972 INFO:tasks.rados.rados.0.smithi100.stdout:1813: writing smithi10066867-12 from 2301952 to 2768896 tid 1 2024-04-07T16:22:39.975 INFO:tasks.rados.rados.0.smithi100.stdout:1813: writing smithi10066867-12 from 2768896 to 3186688 tid 2 2024-04-07T16:22:39.979 INFO:tasks.rados.rados.0.smithi100.stdout:1813: writing smithi10066867-12 from 3186688 to 3604480 tid 3 2024-04-07T16:22:39.983 INFO:tasks.rados.rados.0.smithi100.stdout:1813: writing smithi10066867-12 from 3604480 to 4120576 tid 4 2024-04-07T16:22:39.986 INFO:tasks.rados.rados.0.smithi100.stdout:1813: writing smithi10066867-12 from 4120576 to 4374528 tid 5 2024-04-07T16:22:39.986 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 25 current snap is 168 2024-04-07T16:22:39.986 INFO:tasks.rados.rados.0.smithi100.stdout:1814: seq_num 554 ranges {0=483328,483328=704512,1187840=622592,1810432=278528} 2024-04-07T16:22:39.989 INFO:tasks.rados.rados.0.smithi100.stdout:1814: writing smithi10066867-25 from 0 to 483328 tid 1 2024-04-07T16:22:39.995 INFO:tasks.rados.rados.0.smithi100.stdout:1814: writing smithi10066867-25 from 483328 to 1187840 tid 2 2024-04-07T16:22:39.999 INFO:tasks.rados.rados.0.smithi100.stdout:1814: writing smithi10066867-25 from 1187840 to 1810432 tid 3 2024-04-07T16:22:40.002 INFO:tasks.rados.rados.0.smithi100.stdout:1814: writing smithi10066867-25 from 1810432 to 2088960 tid 4 2024-04-07T16:22:40.002 INFO:tasks.rados.rados.0.smithi100.stdout:1815: delete oid 41 current snap is 168 2024-04-07T16:22:40.002 INFO:tasks.rados.rados.0.smithi100.stdout:1809: finishing write tid 4 to smithi10066867-16 2024-04-07T16:22:40.002 INFO:tasks.rados.rados.0.smithi100.stdout:1809: finishing write tid 5 to smithi10066867-16 2024-04-07T16:22:40.002 INFO:tasks.rados.rados.0.smithi100.stdout:1809: finishing write tid 6 to smithi10066867-16 2024-04-07T16:22:40.003 INFO:tasks.rados.rados.0.smithi100.stdout:1809: oid 16 updating version 0 to 1015 2024-04-07T16:22:40.003 INFO:tasks.rados.rados.0.smithi100.stdout:1809: oid 16 updating version 1015 to 1018 2024-04-07T16:22:40.003 INFO:tasks.rados.rados.0.smithi100.stdout:1809: oid 16 updating version 1018 to 1019 2024-04-07T16:22:40.003 INFO:tasks.rados.rados.0.smithi100.stdout:1809: oid 16 version 1019 is already newer than 1017 2024-04-07T16:22:40.003 INFO:tasks.rados.rados.0.smithi100.stdout:1809: oid 16 version 1019 is already newer than 1016 2024-04-07T16:22:40.003 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 16 v 1019 (ObjNum 552 snap 168 seq_num 552) dirty exists 2024-04-07T16:22:40.003 INFO:tasks.rados.rados.0.smithi100.stdout:1809: left oid 16 (ObjNum 552 snap 168 seq_num 552) 2024-04-07T16:22:40.003 INFO:tasks.rados.rados.0.smithi100.stdout:1812: finishing rollback tid 0 to smithi10066867-8 2024-04-07T16:22:40.003 INFO:tasks.rados.rados.0.smithi100.stdout:1812: finishing rollback tid 1 to smithi10066867-8 2024-04-07T16:22:40.003 INFO:tasks.rados.rados.0.smithi100.stdout:1812: finishing rollback tid 2 to smithi10066867-8 2024-04-07T16:22:40.003 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 8 v 729 (ObjNum 524 snap 156 seq_num 524) dirty exists 2024-04-07T16:22:40.003 INFO:tasks.rados.rados.0.smithi100.stdout:1811: finishing copy_from racing read to smithi10066867-35 2024-04-07T16:22:40.003 INFO:tasks.rados.rados.0.smithi100.stdout:1811: finishing copy_from to smithi10066867-35 2024-04-07T16:22:40.003 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 35 v 1045 (ObjNum 501 snap 150 seq_num 501) dirty exists 2024-04-07T16:22:40.004 INFO:tasks.rados.rados.0.smithi100.stdout:1813: finishing write tid 1 to smithi10066867-12 2024-04-07T16:22:40.004 INFO:tasks.rados.rados.0.smithi100.stdout:1810: finishing copy_from racing read to smithi10066867-50 2024-04-07T16:22:40.004 INFO:tasks.rados.rados.0.smithi100.stdout:1810: finishing copy_from to smithi10066867-50 2024-04-07T16:22:40.004 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 50 v 998 (ObjNum 510 snap 152 seq_num 510) dirty exists 2024-04-07T16:22:40.004 INFO:tasks.rados.rados.0.smithi100.stdout:1813: finishing write tid 2 to smithi10066867-12 2024-04-07T16:22:40.004 INFO:tasks.rados.rados.0.smithi100.stdout:1813: finishing write tid 3 to smithi10066867-12 2024-04-07T16:22:40.004 INFO:tasks.rados.rados.0.smithi100.stdout:1813: finishing write tid 4 to smithi10066867-12 2024-04-07T16:22:40.004 INFO:tasks.rados.rados.0.smithi100.stdout:1813: finishing write tid 5 to smithi10066867-12 2024-04-07T16:22:40.004 INFO:tasks.rados.rados.0.smithi100.stdout:1813: finishing write tid 6 to smithi10066867-12 2024-04-07T16:22:40.004 INFO:tasks.rados.rados.0.smithi100.stdout:1814: finishing write tid 1 to smithi10066867-25 2024-04-07T16:22:40.004 INFO:tasks.rados.rados.0.smithi100.stdout:1813: finishing write tid 7 to smithi10066867-12 2024-04-07T16:22:40.004 INFO:tasks.rados.rados.0.smithi100.stdout:1813: oid 12 updating version 0 to 1021 2024-04-07T16:22:40.004 INFO:tasks.rados.rados.0.smithi100.stdout:1813: oid 12 version 1021 is already newer than 1020 2024-04-07T16:22:40.004 INFO:tasks.rados.rados.0.smithi100.stdout:1813: oid 12 updating version 1021 to 1022 2024-04-07T16:22:40.004 INFO:tasks.rados.rados.0.smithi100.stdout:1813: oid 12 updating version 1022 to 1023 2024-04-07T16:22:40.005 INFO:tasks.rados.rados.0.smithi100.stdout:1813: oid 12 updating version 1023 to 1025 2024-04-07T16:22:40.005 INFO:tasks.rados.rados.0.smithi100.stdout:1813: oid 12 version 1025 is already newer than 1024 2024-04-07T16:22:40.005 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 12 v 1025 (ObjNum 553 snap 168 seq_num 553) dirty exists 2024-04-07T16:22:40.005 INFO:tasks.rados.rados.0.smithi100.stdout:1813: left oid 12 (ObjNum 553 snap 168 seq_num 553) 2024-04-07T16:22:40.005 INFO:tasks.rados.rados.0.smithi100.stdout:1814: finishing write tid 2 to smithi10066867-25 2024-04-07T16:22:40.005 INFO:tasks.rados.rados.0.smithi100.stdout:1809: done (6 left) 2024-04-07T16:22:40.005 INFO:tasks.rados.rados.0.smithi100.stdout:1810: done (5 left) 2024-04-07T16:22:40.005 INFO:tasks.rados.rados.0.smithi100.stdout:1811: done (4 left) 2024-04-07T16:22:40.006 INFO:tasks.rados.rados.0.smithi100.stdout:1812: done (3 left) 2024-04-07T16:22:40.006 INFO:tasks.rados.rados.0.smithi100.stdout:1813: done (2 left) 2024-04-07T16:22:40.006 INFO:tasks.rados.rados.0.smithi100.stdout:1815: done (1 left) 2024-04-07T16:22:40.006 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 47 current snap is 168 2024-04-07T16:22:40.006 INFO:tasks.rados.rados.0.smithi100.stdout:1816: seq_num 556 ranges {6815744=516096,7331840=524288,7856128=638976,8495104=442368} 2024-04-07T16:22:40.049 INFO:tasks.rados.rados.0.smithi100.stdout:1816: writing smithi10066867-47 from 6815744 to 7331840 tid 1 2024-04-07T16:22:40.052 INFO:tasks.rados.rados.0.smithi100.stdout:1816: writing smithi10066867-47 from 7331840 to 7856128 tid 2 2024-04-07T16:22:40.056 INFO:tasks.rados.rados.0.smithi100.stdout:1816: writing smithi10066867-47 from 7856128 to 8495104 tid 3 2024-04-07T16:22:40.059 INFO:tasks.rados.rados.0.smithi100.stdout:1816: writing smithi10066867-47 from 8495104 to 8937472 tid 4 2024-04-07T16:22:40.059 INFO:tasks.rados.rados.0.smithi100.stdout:1817: rollback oid 26 current snap is 168 2024-04-07T16:22:40.059 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 26 to 167 2024-04-07T16:22:40.060 INFO:tasks.rados.rados.0.smithi100.stdout:1814: finishing write tid 3 to smithi10066867-25 2024-04-07T16:22:40.060 INFO:tasks.rados.rados.0.smithi100.stdout:1814: finishing write tid 4 to smithi10066867-25 2024-04-07T16:22:40.060 INFO:tasks.rados.rados.0.smithi100.stdout:1814: finishing write tid 5 to smithi10066867-25 2024-04-07T16:22:40.060 INFO:tasks.rados.rados.0.smithi100.stdout:1814: finishing write tid 6 to smithi10066867-25 2024-04-07T16:22:40.060 INFO:tasks.rados.rados.0.smithi100.stdout:1814: oid 25 updating version 0 to 767 2024-04-07T16:22:40.060 INFO:tasks.rados.rados.0.smithi100.stdout:1814: oid 25 updating version 767 to 771 2024-04-07T16:22:40.060 INFO:tasks.rados.rados.0.smithi100.stdout:1814: oid 25 version 771 is already newer than 768 2024-04-07T16:22:40.060 INFO:tasks.rados.rados.0.smithi100.stdout:1814: oid 25 version 771 is already newer than 769 2024-04-07T16:22:40.060 INFO:tasks.rados.rados.0.smithi100.stdout:1814: oid 25 version 771 is already newer than 770 2024-04-07T16:22:40.060 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 25 v 771 (ObjNum 554 snap 168 seq_num 554) dirty exists 2024-04-07T16:22:40.060 INFO:tasks.rados.rados.0.smithi100.stdout:1814: left oid 25 (ObjNum 554 snap 168 seq_num 554) 2024-04-07T16:22:40.060 INFO:tasks.rados.rados.0.smithi100.stdout:1816: finishing write tid 1 to smithi10066867-47 2024-04-07T16:22:40.060 INFO:tasks.rados.rados.0.smithi100.stdout:1814: done (2 left) 2024-04-07T16:22:40.060 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 45 current snap is 168 2024-04-07T16:22:40.060 INFO:tasks.rados.rados.0.smithi100.stdout:1818: seq_num 557 ranges {0=532480,532480=802816,1335296=794624,2129920=8192} 2024-04-07T16:22:40.063 INFO:tasks.rados.rados.0.smithi100.stdout:1818: writing smithi10066867-45 from 0 to 532480 tid 1 2024-04-07T16:22:40.068 INFO:tasks.rados.rados.0.smithi100.stdout:1818: writing smithi10066867-45 from 532480 to 1335296 tid 2 2024-04-07T16:22:40.074 INFO:tasks.rados.rados.0.smithi100.stdout:1818: writing smithi10066867-45 from 1335296 to 2129920 tid 3 2024-04-07T16:22:40.074 INFO:tasks.rados.rados.0.smithi100.stdout:1818: writing smithi10066867-45 from 2129920 to 2138112 tid 4 2024-04-07T16:22:40.074 INFO:tasks.rados.rados.0.smithi100.stdout:1819: setattr oid 13 current snap is 168 2024-04-07T16:22:40.074 INFO:tasks.rados.rados.0.smithi100.stdout:1816: finishing write tid 2 to smithi10066867-47 2024-04-07T16:22:40.074 INFO:tasks.rados.rados.0.smithi100.stdout:1817: finishing rollback tid 0 to smithi10066867-26 2024-04-07T16:22:40.074 INFO:tasks.rados.rados.0.smithi100.stdout:1816: finishing write tid 3 to smithi10066867-47 2024-04-07T16:22:40.074 INFO:tasks.rados.rados.0.smithi100.stdout:1817: finishing rollback tid 1 to smithi10066867-26 2024-04-07T16:22:40.074 INFO:tasks.rados.rados.0.smithi100.stdout:1816: finishing write tid 4 to smithi10066867-47 2024-04-07T16:22:40.074 INFO:tasks.rados.rados.0.smithi100.stdout:1817: finishing rollback tid 2 to smithi10066867-26 2024-04-07T16:22:40.074 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 26 v 774 (ObjNum 524 snap 156 seq_num 524) dirty exists 2024-04-07T16:22:40.074 INFO:tasks.rados.rados.0.smithi100.stdout:1816: finishing write tid 5 to smithi10066867-47 2024-04-07T16:22:40.075 INFO:tasks.rados.rados.0.smithi100.stdout:1818: finishing write tid 1 to smithi10066867-45 2024-04-07T16:22:40.075 INFO:tasks.rados.rados.0.smithi100.stdout:1816: finishing write tid 6 to smithi10066867-47 2024-04-07T16:22:40.075 INFO:tasks.rados.rados.0.smithi100.stdout:1816: oid 47 updating version 0 to 561 2024-04-07T16:22:40.075 INFO:tasks.rados.rados.0.smithi100.stdout:1816: oid 47 version 561 is already newer than 559 2024-04-07T16:22:40.075 INFO:tasks.rados.rados.0.smithi100.stdout:1816: oid 47 updating version 561 to 562 2024-04-07T16:22:40.075 INFO:tasks.rados.rados.0.smithi100.stdout:1816: oid 47 updating version 562 to 563 2024-04-07T16:22:40.075 INFO:tasks.rados.rados.0.smithi100.stdout:1816: oid 47 version 563 is already newer than 560 2024-04-07T16:22:40.075 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 47 v 563 (ObjNum 556 snap 168 seq_num 556) dirty exists 2024-04-07T16:22:40.075 INFO:tasks.rados.rados.0.smithi100.stdout:1816: left oid 47 (ObjNum 556 snap 168 seq_num 556) 2024-04-07T16:22:40.075 INFO:tasks.rados.rados.0.smithi100.stdout:1818: finishing write tid 2 to smithi10066867-45 2024-04-07T16:22:40.075 INFO:tasks.rados.rados.0.smithi100.stdout:1816: done (3 left) 2024-04-07T16:22:40.075 INFO:tasks.rados.rados.0.smithi100.stdout:1817: done (2 left) 2024-04-07T16:22:40.075 INFO:tasks.rados.rados.0.smithi100.stdout:1820: setattr oid 17 current snap is 168 2024-04-07T16:22:40.076 INFO:tasks.rados.rados.0.smithi100.stdout:1821: copy_from oid 43 from oid 39 current snap is 168 2024-04-07T16:22:40.076 INFO:tasks.rados.rados.0.smithi100.stdout:1822: read oid 38 snap -1 2024-04-07T16:22:40.076 INFO:tasks.rados.rados.0.smithi100.stdout:1822: expect (ObjNum 483 snap 144 seq_num 483) 2024-04-07T16:22:40.076 INFO:tasks.rados.rados.0.smithi100.stdout:1823: rollback oid 3 current snap is 168 2024-04-07T16:22:40.076 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 3 to 166 2024-04-07T16:22:40.076 INFO:tasks.rados.rados.0.smithi100.stdout:1824: copy_from oid 28 from oid 30 current snap is 168 2024-04-07T16:22:40.076 INFO:tasks.rados.rados.0.smithi100.stdout:1825: rollback oid 37 current snap is 168 2024-04-07T16:22:40.077 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 37 to 166 2024-04-07T16:22:40.077 INFO:tasks.rados.rados.0.smithi100.stdout:1826: snap_create 2024-04-07T16:22:40.084 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 13 v 730 (ObjNum 537 snap 162 seq_num 537) dirty exists 2024-04-07T16:22:40.084 INFO:tasks.rados.rados.0.smithi100.stdout:1824: finishing copy_from to smithi10066867-28 2024-04-07T16:22:40.084 INFO:tasks.rados.rados.0.smithi100.stdout:1824: got expected ENOENT (src dne) 2024-04-07T16:22:40.085 INFO:tasks.rados.rados.0.smithi100.stdout:1824: finishing copy_from racing read to smithi10066867-28 2024-04-07T16:22:40.086 INFO:tasks.rados.rados.0.smithi100.stdout:1818: finishing write tid 3 to smithi10066867-45 2024-04-07T16:22:40.086 INFO:tasks.rados.rados.0.smithi100.stdout:1818: finishing write tid 4 to smithi10066867-45 2024-04-07T16:22:40.086 INFO:tasks.rados.rados.0.smithi100.stdout:1818: finishing write tid 5 to smithi10066867-45 2024-04-07T16:22:40.086 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 17 v 729 (ObjNum 506 snap 151 seq_num 506) dirty exists 2024-04-07T16:22:40.088 INFO:tasks.rados.rados.0.smithi100.stdout:1825: finishing rollback tid 0 to smithi10066867-37 2024-04-07T16:22:40.088 INFO:tasks.rados.rados.0.smithi100.stdout:1825: finishing rollback tid 1 to smithi10066867-37 2024-04-07T16:22:40.088 INFO:tasks.rados.rados.0.smithi100.stdout:1825: finishing rollback tid 2 to smithi10066867-37 2024-04-07T16:22:40.088 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 37 v 1048 (ObjNum 501 snap 150 seq_num 501) dirty exists 2024-04-07T16:22:40.090 INFO:tasks.rados.rados.0.smithi100.stdout:1818: finishing write tid 6 to smithi10066867-45 2024-04-07T16:22:40.090 INFO:tasks.rados.rados.0.smithi100.stdout:1818: oid 45 updating version 0 to 1000 2024-04-07T16:22:40.090 INFO:tasks.rados.rados.0.smithi100.stdout:1818: oid 45 updating version 1000 to 1002 2024-04-07T16:22:40.090 INFO:tasks.rados.rados.0.smithi100.stdout:1818: oid 45 version 1002 is already newer than 999 2024-04-07T16:22:40.090 INFO:tasks.rados.rados.0.smithi100.stdout:1818: oid 45 updating version 1002 to 1003 2024-04-07T16:22:40.090 INFO:tasks.rados.rados.0.smithi100.stdout:1818: oid 45 version 1003 is already newer than 1001 2024-04-07T16:22:40.091 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 45 v 1003 (ObjNum 557 snap 168 seq_num 557) dirty exists 2024-04-07T16:22:40.091 INFO:tasks.rados.rados.0.smithi100.stdout:1818: left oid 45 (ObjNum 557 snap 168 seq_num 557) 2024-04-07T16:22:40.091 INFO:tasks.rados.rados.0.smithi100.stdout:1822: expect (ObjNum 483 snap 144 seq_num 483) 2024-04-07T16:22:40.114 INFO:tasks.rados.rados.0.smithi100.stdout:1823: finishing rollback tid 0 to smithi10066867-3 2024-04-07T16:22:40.114 INFO:tasks.rados.rados.0.smithi100.stdout:1823: finishing rollback tid 1 to smithi10066867-3 2024-04-07T16:22:40.114 INFO:tasks.rados.rados.0.smithi100.stdout:1823: finishing rollback tid 2 to smithi10066867-3 2024-04-07T16:22:40.114 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 3 v 1011 (ObjNum 530 snap 161 seq_num 530) dirty exists 2024-04-07T16:22:40.114 INFO:tasks.rados.rados.0.smithi100.stdout:1821: finishing copy_from racing read to smithi10066867-43 2024-04-07T16:22:40.114 INFO:tasks.rados.rados.0.smithi100.stdout:1821: finishing copy_from to smithi10066867-43 2024-04-07T16:22:40.115 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 43 v 624 (ObjNum 465 snap 142 seq_num 465) dirty exists 2024-04-07T16:22:40.184 INFO:journalctl@ceph.osd.1.smithi100.stdout:Apr 07 16:22:40 smithi100 bash[85238]: debug 2024-04-07T16:22:40.143+0000 7fbe933c5700 1 -- 172.21.15.100:0/4018241894 <== mon.2 v2:172.21.15.100:3301/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561501197680 con 0x5615011d4800 2024-04-07T16:22:40.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:39 smithi100 bash[74942]: cluster 2024-04-07T16:22:38.656370+0000 mgr.y (mgr.25175) 140 : cluster [DBG] pgmap v202: 121 pgs: 11 active+undersized+degraded, 34 active+undersized, 76 active+clean; 169 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 5.5 MiB/s rd, 2.2 MiB/s wr, 15 op/s; 36/258 objects degraded (13.953%) 2024-04-07T16:22:40.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:39 smithi100 bash[74942]: cluster 2024-04-07T16:22:38.884882+0000 mon.a (mon.0) 435 : cluster [WRN] Health check failed: Degraded data redundancy: 36/258 objects degraded (13.953%), 11 pgs degraded (PG_DEGRADED) 2024-04-07T16:22:40.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:39 smithi100 bash[74942]: cluster 2024-04-07T16:22:38.906316+0000 mon.a (mon.0) 436 : cluster [DBG] osdmap e409: 8 total, 7 up, 8 in 2024-04-07T16:22:40.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:39 smithi100 bash[76202]: cluster 2024-04-07T16:22:38.656370+0000 mgr.y (mgr.25175) 140 : cluster [DBG] pgmap v202: 121 pgs: 11 active+undersized+degraded, 34 active+undersized, 76 active+clean; 169 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 5.5 MiB/s rd, 2.2 MiB/s wr, 15 op/s; 36/258 objects degraded (13.953%) 2024-04-07T16:22:40.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:39 smithi100 bash[76202]: cluster 2024-04-07T16:22:38.884882+0000 mon.a (mon.0) 435 : cluster [WRN] Health check failed: Degraded data redundancy: 36/258 objects degraded (13.953%), 11 pgs degraded (PG_DEGRADED) 2024-04-07T16:22:40.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:39 smithi100 bash[76202]: cluster 2024-04-07T16:22:38.906316+0000 mon.a (mon.0) 436 : cluster [DBG] osdmap e409: 8 total, 7 up, 8 in 2024-04-07T16:22:40.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:39 smithi144 bash[66654]: cluster 2024-04-07T16:22:38.656370+0000 mgr.y (mgr.25175) 140 : cluster [DBG] pgmap v202: 121 pgs: 11 active+undersized+degraded, 34 active+undersized, 76 active+clean; 169 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 5.5 MiB/s rd, 2.2 MiB/s wr, 15 op/s; 36/258 objects degraded (13.953%) 2024-04-07T16:22:40.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:39 smithi144 bash[66654]: cluster 2024-04-07T16:22:38.884882+0000 mon.a (mon.0) 435 : cluster [WRN] Health check failed: Degraded data redundancy: 36/258 objects degraded (13.953%), 11 pgs degraded (PG_DEGRADED) 2024-04-07T16:22:40.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:39 smithi144 bash[66654]: cluster 2024-04-07T16:22:38.906316+0000 mon.a (mon.0) 436 : cluster [DBG] osdmap e409: 8 total, 7 up, 8 in 2024-04-07T16:22:40.907 INFO:tasks.rados.rados.0.smithi100.stdout:1818: done (8 left) 2024-04-07T16:22:40.907 INFO:tasks.rados.rados.0.smithi100.stdout:1819: done (7 left) 2024-04-07T16:22:40.907 INFO:tasks.rados.rados.0.smithi100.stdout:1820: done (6 left) 2024-04-07T16:22:40.907 INFO:tasks.rados.rados.0.smithi100.stdout:1821: done (5 left) 2024-04-07T16:22:40.907 INFO:tasks.rados.rados.0.smithi100.stdout:1822: done (4 left) 2024-04-07T16:22:40.907 INFO:tasks.rados.rados.0.smithi100.stdout:1823: done (3 left) 2024-04-07T16:22:40.907 INFO:tasks.rados.rados.0.smithi100.stdout:1824: done (2 left) 2024-04-07T16:22:40.907 INFO:tasks.rados.rados.0.smithi100.stdout:1825: done (1 left) 2024-04-07T16:22:40.907 INFO:tasks.rados.rados.0.smithi100.stdout:1826: done (0 left) 2024-04-07T16:22:40.907 INFO:tasks.rados.rados.0.smithi100.stdout:1827: snap_create 2024-04-07T16:22:40.934 INFO:journalctl@ceph.osd.1.smithi100.stdout:Apr 07 16:22:40 smithi100 bash[85238]: debug 2024-04-07T16:22:40.679+0000 7fbe9c55d7c0 -1 Falling back to public interface 2024-04-07T16:22:40.935 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:40 smithi100 bash[74942]: cluster 2024-04-07T16:22:39.913539+0000 mon.a (mon.0) 437 : cluster [DBG] osdmap e410: 8 total, 7 up, 8 in 2024-04-07T16:22:40.935 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:40 smithi100 bash[76202]: cluster 2024-04-07T16:22:39.913539+0000 mon.a (mon.0) 437 : cluster [DBG] osdmap e410: 8 total, 7 up, 8 in 2024-04-07T16:22:41.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:40 smithi144 bash[66654]: cluster 2024-04-07T16:22:39.913539+0000 mon.a (mon.0) 437 : cluster [DBG] osdmap e410: 8 total, 7 up, 8 in 2024-04-07T16:22:41.791 INFO:tasks.rados.rados.0.smithi100.stdout:1827: done (0 left) 2024-04-07T16:22:41.791 INFO:tasks.rados.rados.0.smithi100.stdout:1828: read oid 37 snap -1 2024-04-07T16:22:41.791 INFO:tasks.rados.rados.0.smithi100.stdout:1828: expect (ObjNum 501 snap 150 seq_num 501) 2024-04-07T16:22:41.791 INFO:tasks.rados.rados.0.smithi100.stdout:1829: read oid 17 snap -1 2024-04-07T16:22:41.791 INFO:tasks.rados.rados.0.smithi100.stdout:1829: expect (ObjNum 506 snap 151 seq_num 506) 2024-04-07T16:22:41.791 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 8 current snap is 170 2024-04-07T16:22:41.791 INFO:tasks.rados.rados.0.smithi100.stdout:1830: seq_num 558 ranges {7094272=548864,7643136=688128,8331264=606208,8937472=311296} 2024-04-07T16:22:41.836 INFO:tasks.rados.rados.0.smithi100.stdout:1830: writing smithi10066867-8 from 7094272 to 7643136 tid 1 2024-04-07T16:22:41.841 INFO:tasks.rados.rados.0.smithi100.stdout:1830: writing smithi10066867-8 from 7643136 to 8331264 tid 2 2024-04-07T16:22:41.844 INFO:tasks.rados.rados.0.smithi100.stdout:1830: writing smithi10066867-8 from 8331264 to 8937472 tid 3 2024-04-07T16:22:41.846 INFO:tasks.rados.rados.0.smithi100.stdout:1830: writing smithi10066867-8 from 8937472 to 9248768 tid 4 2024-04-07T16:22:41.846 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 29 current snap is 170 2024-04-07T16:22:41.846 INFO:tasks.rados.rados.0.smithi100.stdout:1831: seq_num 559 ranges {5980160=565248,6545408=573440,7118848=409600,7528448=622592} 2024-04-07T16:22:41.883 INFO:tasks.rados.rados.0.smithi100.stdout:1831: writing smithi10066867-29 from 5980160 to 6545408 tid 1 2024-04-07T16:22:41.886 INFO:tasks.rados.rados.0.smithi100.stdout:1831: writing smithi10066867-29 from 6545408 to 7118848 tid 2 2024-04-07T16:22:41.888 INFO:tasks.rados.rados.0.smithi100.stdout:1831: writing smithi10066867-29 from 7118848 to 7528448 tid 3 2024-04-07T16:22:41.893 INFO:tasks.rados.rados.0.smithi100.stdout:1831: writing smithi10066867-29 from 7528448 to 8151040 tid 4 2024-04-07T16:22:41.893 INFO:tasks.rados.rados.0.smithi100.stdout:1832: snap_create 2024-04-07T16:22:41.893 INFO:tasks.rados.rados.0.smithi100.stdout:1828: expect (ObjNum 501 snap 150 seq_num 501) 2024-04-07T16:22:41.943 INFO:tasks.rados.rados.0.smithi100.stdout:1829: expect (ObjNum 506 snap 151 seq_num 506) 2024-04-07T16:22:42.184 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:41 smithi100 bash[76202]: cluster 2024-04-07T16:22:40.657187+0000 mgr.y (mgr.25175) 141 : cluster [DBG] pgmap v205: 121 pgs: 11 active+undersized+degraded, 34 active+undersized, 76 active+clean; 169 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 5.5 MiB/s rd, 2.2 MiB/s wr, 15 op/s; 36/258 objects degraded (13.953%) 2024-04-07T16:22:42.184 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:41 smithi100 bash[76202]: cluster 2024-04-07T16:22:40.922867+0000 mon.a (mon.0) 438 : cluster [DBG] osdmap e411: 8 total, 7 up, 8 in 2024-04-07T16:22:42.184 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:41 smithi100 bash[76202]: cluster 2024-04-07T16:22:41.808408+0000 mon.a (mon.0) 439 : cluster [DBG] osdmap e412: 8 total, 7 up, 8 in 2024-04-07T16:22:42.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:41 smithi100 bash[74942]: cluster 2024-04-07T16:22:40.657187+0000 mgr.y (mgr.25175) 141 : cluster [DBG] pgmap v205: 121 pgs: 11 active+undersized+degraded, 34 active+undersized, 76 active+clean; 169 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 5.5 MiB/s rd, 2.2 MiB/s wr, 15 op/s; 36/258 objects degraded (13.953%) 2024-04-07T16:22:42.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:41 smithi100 bash[74942]: cluster 2024-04-07T16:22:40.922867+0000 mon.a (mon.0) 438 : cluster [DBG] osdmap e411: 8 total, 7 up, 8 in 2024-04-07T16:22:42.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:41 smithi100 bash[74942]: cluster 2024-04-07T16:22:41.808408+0000 mon.a (mon.0) 439 : cluster [DBG] osdmap e412: 8 total, 7 up, 8 in 2024-04-07T16:22:42.273 INFO:tasks.rados.rados.0.smithi100.stdout:1830: finishing write tid 1 to smithi10066867-8 2024-04-07T16:22:42.274 INFO:tasks.rados.rados.0.smithi100.stdout:1830: finishing write tid 2 to smithi10066867-8 2024-04-07T16:22:42.274 INFO:tasks.rados.rados.0.smithi100.stdout:1830: finishing write tid 3 to smithi10066867-8 2024-04-07T16:22:42.274 INFO:tasks.rados.rados.0.smithi100.stdout:1830: finishing write tid 4 to smithi10066867-8 2024-04-07T16:22:42.274 INFO:tasks.rados.rados.0.smithi100.stdout:1830: finishing write tid 5 to smithi10066867-8 2024-04-07T16:22:42.274 INFO:tasks.rados.rados.0.smithi100.stdout:1830: finishing write tid 6 to smithi10066867-8 2024-04-07T16:22:42.274 INFO:tasks.rados.rados.0.smithi100.stdout:1830: oid 8 updating version 0 to 731 2024-04-07T16:22:42.274 INFO:tasks.rados.rados.0.smithi100.stdout:1830: oid 8 updating version 731 to 735 2024-04-07T16:22:42.274 INFO:tasks.rados.rados.0.smithi100.stdout:1830: oid 8 version 735 is already newer than 732 2024-04-07T16:22:42.274 INFO:tasks.rados.rados.0.smithi100.stdout:1830: oid 8 version 735 is already newer than 733 2024-04-07T16:22:42.274 INFO:tasks.rados.rados.0.smithi100.stdout:1830: oid 8 version 735 is already newer than 734 2024-04-07T16:22:42.274 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 8 v 735 (ObjNum 558 snap 170 seq_num 558) dirty exists 2024-04-07T16:22:42.274 INFO:tasks.rados.rados.0.smithi100.stdout:1830: left oid 8 (ObjNum 558 snap 170 seq_num 558) 2024-04-07T16:22:42.274 INFO:tasks.rados.rados.0.smithi100.stdout:1831: finishing write tid 1 to smithi10066867-29 2024-04-07T16:22:42.275 INFO:tasks.rados.rados.0.smithi100.stdout:1831: finishing write tid 2 to smithi10066867-29 2024-04-07T16:22:42.275 INFO:tasks.rados.rados.0.smithi100.stdout:1831: finishing write tid 3 to smithi10066867-29 2024-04-07T16:22:42.275 INFO:tasks.rados.rados.0.smithi100.stdout:1831: finishing write tid 4 to smithi10066867-29 2024-04-07T16:22:42.275 INFO:tasks.rados.rados.0.smithi100.stdout:1831: finishing write tid 5 to smithi10066867-29 2024-04-07T16:22:42.275 INFO:tasks.rados.rados.0.smithi100.stdout:1831: finishing write tid 6 to smithi10066867-29 2024-04-07T16:22:42.275 INFO:tasks.rados.rados.0.smithi100.stdout:1831: oid 29 updating version 0 to 1052 2024-04-07T16:22:42.275 INFO:tasks.rados.rados.0.smithi100.stdout:1831: oid 29 updating version 1052 to 1053 2024-04-07T16:22:42.275 INFO:tasks.rados.rados.0.smithi100.stdout:1831: oid 29 version 1053 is already newer than 1051 2024-04-07T16:22:42.275 INFO:tasks.rados.rados.0.smithi100.stdout:1831: oid 29 version 1053 is already newer than 1049 2024-04-07T16:22:42.275 INFO:tasks.rados.rados.0.smithi100.stdout:1831: oid 29 version 1053 is already newer than 1050 2024-04-07T16:22:42.275 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 29 v 1053 (ObjNum 559 snap 170 seq_num 559) dirty exists 2024-04-07T16:22:42.275 INFO:tasks.rados.rados.0.smithi100.stdout:1831: left oid 29 (ObjNum 559 snap 170 seq_num 559) 2024-04-07T16:22:42.304 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:41 smithi144 bash[66654]: cluster 2024-04-07T16:22:40.657187+0000 mgr.y (mgr.25175) 141 : cluster [DBG] pgmap v205: 121 pgs: 11 active+undersized+degraded, 34 active+undersized, 76 active+clean; 169 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 5.5 MiB/s rd, 2.2 MiB/s wr, 15 op/s; 36/258 objects degraded (13.953%) 2024-04-07T16:22:42.304 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:41 smithi144 bash[66654]: cluster 2024-04-07T16:22:40.922867+0000 mon.a (mon.0) 438 : cluster [DBG] osdmap e411: 8 total, 7 up, 8 in 2024-04-07T16:22:42.304 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:41 smithi144 bash[66654]: cluster 2024-04-07T16:22:41.808408+0000 mon.a (mon.0) 439 : cluster [DBG] osdmap e412: 8 total, 7 up, 8 in 2024-04-07T16:22:42.606 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:22:42 smithi144 bash[59794]: ts=2024-04-07T16:22:42.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:22:42.793 INFO:tasks.rados.rados.0.smithi100.stdout:1828: done (4 left) 2024-04-07T16:22:42.793 INFO:tasks.rados.rados.0.smithi100.stdout:1829: done (3 left) 2024-04-07T16:22:42.793 INFO:tasks.rados.rados.0.smithi100.stdout:1830: done (2 left) 2024-04-07T16:22:42.793 INFO:tasks.rados.rados.0.smithi100.stdout:1831: done (1 left) 2024-04-07T16:22:42.793 INFO:tasks.rados.rados.0.smithi100.stdout:1832: done (0 left) 2024-04-07T16:22:42.794 INFO:tasks.rados.rados.0.smithi100.stdout:1833: rmattr oid 13 current snap is 171 2024-04-07T16:22:42.794 INFO:tasks.rados.rados.0.smithi100.stdout:1834: read oid 26 snap 166 2024-04-07T16:22:42.794 INFO:tasks.rados.rados.0.smithi100.stdout:1834: expect (ObjNum 524 snap 156 seq_num 524) 2024-04-07T16:22:42.794 INFO:tasks.rados.rados.0.smithi100.stdout:1835: rollback oid 22 current snap is 171 2024-04-07T16:22:42.794 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 22 to 170 2024-04-07T16:22:42.794 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 39 current snap is 171 2024-04-07T16:22:42.794 INFO:tasks.rados.rados.0.smithi100.stdout:1836: seq_num 560 ranges {3620864=581632,4202496=458752,4661248=614400,5275648=532480} 2024-04-07T16:22:42.820 INFO:tasks.rados.rados.0.smithi100.stdout:1836: writing smithi10066867-39 from 3620864 to 4202496 tid 1 2024-04-07T16:22:42.823 INFO:tasks.rados.rados.0.smithi100.stdout:1836: writing smithi10066867-39 from 4202496 to 4661248 tid 2 2024-04-07T16:22:42.827 INFO:tasks.rados.rados.0.smithi100.stdout:1836: writing smithi10066867-39 from 4661248 to 5275648 tid 3 2024-04-07T16:22:42.831 INFO:tasks.rados.rados.0.smithi100.stdout:1836: writing smithi10066867-39 from 5275648 to 5808128 tid 4 2024-04-07T16:22:42.831 INFO:tasks.rados.rados.0.smithi100.stdout:1837: snap_create 2024-04-07T16:22:42.831 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 13 v 736 (ObjNum 537 snap 162 seq_num 537) dirty exists 2024-04-07T16:22:42.831 INFO:tasks.rados.rados.0.smithi100.stdout:1835: finishing rollback tid 0 to smithi10066867-22 2024-04-07T16:22:42.831 INFO:tasks.rados.rados.0.smithi100.stdout:1836: finishing write tid 1 to smithi10066867-39 2024-04-07T16:22:42.831 INFO:tasks.rados.rados.0.smithi100.stdout:1836: finishing write tid 2 to smithi10066867-39 2024-04-07T16:22:42.833 INFO:tasks.rados.rados.0.smithi100.stdout:1834: expect (ObjNum 524 snap 156 seq_num 524) 2024-04-07T16:22:43.308 INFO:tasks.rados.rados.0.smithi100.stdout:1835: finishing rollback tid 1 to smithi10066867-22 2024-04-07T16:22:43.308 INFO:tasks.rados.rados.0.smithi100.stdout:1835: finishing rollback tid 2 to smithi10066867-22 2024-04-07T16:22:43.308 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 22 v 777 (ObjNum 512 snap 152 seq_num 512) dirty exists 2024-04-07T16:22:43.308 INFO:tasks.rados.rados.0.smithi100.stdout:1836: finishing write tid 3 to smithi10066867-39 2024-04-07T16:22:43.308 INFO:tasks.rados.rados.0.smithi100.stdout:1836: finishing write tid 4 to smithi10066867-39 2024-04-07T16:22:43.308 INFO:tasks.rados.rados.0.smithi100.stdout:1836: finishing write tid 5 to smithi10066867-39 2024-04-07T16:22:43.308 INFO:tasks.rados.rados.0.smithi100.stdout:1836: finishing write tid 6 to smithi10066867-39 2024-04-07T16:22:43.308 INFO:tasks.rados.rados.0.smithi100.stdout:1836: oid 39 updating version 0 to 1054 2024-04-07T16:22:43.308 INFO:tasks.rados.rados.0.smithi100.stdout:1836: oid 39 updating version 1054 to 1057 2024-04-07T16:22:43.308 INFO:tasks.rados.rados.0.smithi100.stdout:1836: oid 39 updating version 1057 to 1058 2024-04-07T16:22:43.308 INFO:tasks.rados.rados.0.smithi100.stdout:1836: oid 39 version 1058 is already newer than 1055 2024-04-07T16:22:43.309 INFO:tasks.rados.rados.0.smithi100.stdout:1836: oid 39 version 1058 is already newer than 1056 2024-04-07T16:22:43.309 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 39 v 1058 (ObjNum 560 snap 171 seq_num 560) dirty exists 2024-04-07T16:22:43.309 INFO:tasks.rados.rados.0.smithi100.stdout:1836: left oid 39 (ObjNum 560 snap 171 seq_num 560) 2024-04-07T16:22:43.789 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:22:43 smithi144 bash[59794]: ts=2024-04-07T16:22:43.507Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=nodes name=CephNodeDiskspaceWarning index=4 msg="Evaluating rule failed" rule="alert: CephNodeDiskspaceWarning\nexpr: predict_linear(node_filesystem_free_bytes{device=~\"/.*\"}[2d], 3600 * 24 * 5)\n * on (instance) group_left (nodename) node_uname_info < 0\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.8.4\n severity: warning\n type: ceph_default\nannotations:\n description: Mountpoint {{ $labels.mountpoint }} on {{ $labels.nodename }} will\n be full in less than 5 days based on the 48 hour trailing fill rate.\n summary: Host filesystem free space is getting low\n" err="found duplicate series for the match group {instance=\"smithi100\"} on the right hand-side of the operation: [{__name__=\"node_uname_info\", domainname=\"(none)\", instance=\"smithi100\", job=\"node\", machine=\"x86_64\", nodename=\"smithi100\", release=\"5.15.0-101-generic\", sysname=\"Linux\", version=\"#111-Ubuntu SMP Tue Mar 5 20:16:58 UTC 2024\"}, {__name__=\"node_uname_info\", cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", domainname=\"(none)\", instance=\"smithi100\", job=\"node\", machine=\"x86_64\", nodename=\"smithi100\", release=\"5.15.0-101-generic\", sysname=\"Linux\", version=\"#111-Ubuntu SMP Tue Mar 5 20:16:58 UTC 2024\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:22:43.802 INFO:tasks.rados.rados.0.smithi100.stdout:1833: done (4 left) 2024-04-07T16:22:43.802 INFO:tasks.rados.rados.0.smithi100.stdout:1834: done (3 left) 2024-04-07T16:22:43.802 INFO:tasks.rados.rados.0.smithi100.stdout:1835: done (2 left) 2024-04-07T16:22:43.803 INFO:tasks.rados.rados.0.smithi100.stdout:1836: done (1 left) 2024-04-07T16:22:43.803 INFO:tasks.rados.rados.0.smithi100.stdout:1837: done (0 left) 2024-04-07T16:22:43.803 INFO:tasks.rados.rados.0.smithi100.stdout:1838: read oid 12 snap -1 2024-04-07T16:22:43.803 INFO:tasks.rados.rados.0.smithi100.stdout:1838: expect (ObjNum 553 snap 168 seq_num 553) 2024-04-07T16:22:43.803 INFO:tasks.rados.rados.0.smithi100.stdout:1839: read oid 44 snap -1 2024-04-07T16:22:43.803 INFO:tasks.rados.rados.0.smithi100.stdout:1839: expect (ObjNum 509 snap 152 seq_num 509) 2024-04-07T16:22:43.803 INFO:tasks.rados.rados.0.smithi100.stdout:1840: read oid 20 snap -1 2024-04-07T16:22:43.803 INFO:tasks.rados.rados.0.smithi100.stdout:1840: expect (ObjNum 543 snap 164 seq_num 543) 2024-04-07T16:22:43.803 INFO:tasks.rados.rados.0.smithi100.stdout:1841: snap_create 2024-04-07T16:22:43.837 INFO:tasks.rados.rados.0.smithi100.stdout:1838: expect (ObjNum 553 snap 168 seq_num 553) 2024-04-07T16:22:44.041 INFO:tasks.rados.rados.0.smithi100.stdout:1840: expect (ObjNum 543 snap 164 seq_num 543) 2024-04-07T16:22:44.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:43 smithi144 bash[66654]: cluster 2024-04-07T16:22:42.658671+0000 mgr.y (mgr.25175) 142 : cluster [DBG] pgmap v208: 121 pgs: 11 active+undersized+degraded, 34 active+undersized, 76 active+clean; 181 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 7.7 MiB/s rd, 4.1 MiB/s wr, 23 op/s; 45/270 objects degraded (16.667%) 2024-04-07T16:22:44.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:43 smithi144 bash[66654]: cluster 2024-04-07T16:22:42.810403+0000 mon.a (mon.0) 440 : cluster [DBG] osdmap e413: 8 total, 7 up, 8 in 2024-04-07T16:22:44.184 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:43 smithi100 bash[74942]: cluster 2024-04-07T16:22:42.658671+0000 mgr.y (mgr.25175) 142 : cluster [DBG] pgmap v208: 121 pgs: 11 active+undersized+degraded, 34 active+undersized, 76 active+clean; 181 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 7.7 MiB/s rd, 4.1 MiB/s wr, 23 op/s; 45/270 objects degraded (16.667%) 2024-04-07T16:22:44.184 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:43 smithi100 bash[74942]: cluster 2024-04-07T16:22:42.810403+0000 mon.a (mon.0) 440 : cluster [DBG] osdmap e413: 8 total, 7 up, 8 in 2024-04-07T16:22:44.184 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:43 smithi100 bash[76202]: cluster 2024-04-07T16:22:42.658671+0000 mgr.y (mgr.25175) 142 : cluster [DBG] pgmap v208: 121 pgs: 11 active+undersized+degraded, 34 active+undersized, 76 active+clean; 181 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 7.7 MiB/s rd, 4.1 MiB/s wr, 23 op/s; 45/270 objects degraded (16.667%) 2024-04-07T16:22:44.184 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:43 smithi100 bash[76202]: cluster 2024-04-07T16:22:42.810403+0000 mon.a (mon.0) 440 : cluster [DBG] osdmap e413: 8 total, 7 up, 8 in 2024-04-07T16:22:44.684 INFO:journalctl@ceph.osd.1.smithi100.stdout:Apr 07 16:22:44 smithi100 bash[85238]: debug 2024-04-07T16:22:44.203+0000 7fbe9c55d7c0 -1 osd.1 403 log_to_monitors true 2024-04-07T16:22:44.685 INFO:tasks.rados.rados.0.smithi100.stdout:1839: expect (ObjNum 509 snap 152 seq_num 509) 2024-04-07T16:22:45.067 INFO:tasks.rados.rados.0.smithi100.stdout:1838: done (3 left) 2024-04-07T16:22:45.068 INFO:tasks.rados.rados.0.smithi100.stdout:1839: done (2 left) 2024-04-07T16:22:45.068 INFO:tasks.rados.rados.0.smithi100.stdout:1840: done (1 left) 2024-04-07T16:22:45.068 INFO:tasks.rados.rados.0.smithi100.stdout:1841: done (0 left) 2024-04-07T16:22:45.068 INFO:tasks.rados.rados.0.smithi100.stdout:1842: delete oid 18 current snap is 173 2024-04-07T16:22:45.077 INFO:tasks.rados.rados.0.smithi100.stdout:1842: done (0 left) 2024-04-07T16:22:45.077 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 38 current snap is 173 2024-04-07T16:22:45.077 INFO:tasks.rados.rados.0.smithi100.stdout:1843: seq_num 562 ranges {892928=622592,1515520=630784,2146304=581632,2727936=393216} 2024-04-07T16:22:45.086 INFO:tasks.rados.rados.0.smithi100.stdout:1843: writing smithi10066867-38 from 892928 to 1515520 tid 1 2024-04-07T16:22:45.090 INFO:tasks.rados.rados.0.smithi100.stdout:1843: writing smithi10066867-38 from 1515520 to 2146304 tid 2 2024-04-07T16:22:45.094 INFO:tasks.rados.rados.0.smithi100.stdout:1843: writing smithi10066867-38 from 2146304 to 2727936 tid 3 2024-04-07T16:22:45.097 INFO:tasks.rados.rados.0.smithi100.stdout:1843: writing smithi10066867-38 from 2727936 to 3121152 tid 4 2024-04-07T16:22:45.097 INFO:tasks.rados.rados.0.smithi100.stdout:1844: delete oid 32 current snap is 173 2024-04-07T16:22:45.103 INFO:tasks.rados.rados.0.smithi100.stdout:1844: done (1 left) 2024-04-07T16:22:45.103 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 47 current snap is 173 2024-04-07T16:22:45.103 INFO:tasks.rados.rados.0.smithi100.stdout:1845: seq_num 564 ranges {8937472=655360,9592832=442368,10035200=598016,10633216=565248} 2024-04-07T16:22:45.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:44 smithi144 bash[66654]: cluster 2024-04-07T16:22:43.820003+0000 mon.a (mon.0) 441 : cluster [DBG] osdmap e414: 8 total, 7 up, 8 in 2024-04-07T16:22:45.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:44 smithi144 bash[66654]: audit 2024-04-07T16:22:44.212591+0000 mon.c (mon.2) 7 : audit [INF] from='osd.1 [v2:172.21.15.100:6810/1469336877,v1:172.21.15.100:6811/1469336877]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-07T16:22:45.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:44 smithi144 bash[66654]: audit 2024-04-07T16:22:44.212978+0000 mon.a (mon.0) 442 : audit [INF] from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-07T16:22:45.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:44 smithi144 bash[66654]: audit 2024-04-07T16:22:44.622190+0000 mon.a (mon.0) 443 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:22:45.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:44 smithi144 bash[66654]: audit 2024-04-07T16:22:44.628182+0000 mon.a (mon.0) 444 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:22:45.157 INFO:tasks.rados.rados.0.smithi100.stdout:1845: writing smithi10066867-47 from 8937472 to 9592832 tid 1 2024-04-07T16:22:45.159 INFO:tasks.rados.rados.0.smithi100.stdout:1845: writing smithi10066867-47 from 9592832 to 10035200 tid 2 2024-04-07T16:22:45.163 INFO:tasks.rados.rados.0.smithi100.stdout:1845: writing smithi10066867-47 from 10035200 to 10633216 tid 3 2024-04-07T16:22:45.167 INFO:tasks.rados.rados.0.smithi100.stdout:1845: writing smithi10066867-47 from 10633216 to 11198464 tid 4 2024-04-07T16:22:45.167 INFO:tasks.rados.rados.0.smithi100.stdout:1846: rmattr oid 23 current snap is 173 2024-04-07T16:22:45.167 INFO:tasks.rados.rados.0.smithi100.stdout:1846: done (2 left) 2024-04-07T16:22:45.167 INFO:tasks.rados.rados.0.smithi100.stdout:1847: copy_from oid 30 from oid 2 current snap is 173 2024-04-07T16:22:45.167 INFO:tasks.rados.rados.0.smithi100.stdout:1843: finishing write tid 1 to smithi10066867-38 2024-04-07T16:22:45.167 INFO:tasks.rados.rados.0.smithi100.stdout:1843: finishing write tid 2 to smithi10066867-38 2024-04-07T16:22:45.167 INFO:tasks.rados.rados.0.smithi100.stdout:1843: finishing write tid 3 to smithi10066867-38 2024-04-07T16:22:45.167 INFO:tasks.rados.rados.0.smithi100.stdout:1843: finishing write tid 4 to smithi10066867-38 2024-04-07T16:22:45.168 INFO:tasks.rados.rados.0.smithi100.stdout:1843: finishing write tid 5 to smithi10066867-38 2024-04-07T16:22:45.168 INFO:tasks.rados.rados.0.smithi100.stdout:1843: finishing write tid 6 to smithi10066867-38 2024-04-07T16:22:45.168 INFO:tasks.rados.rados.0.smithi100.stdout:1843: oid 38 updating version 0 to 1012 2024-04-07T16:22:45.168 INFO:tasks.rados.rados.0.smithi100.stdout:1843: oid 38 updating version 1012 to 1015 2024-04-07T16:22:45.168 INFO:tasks.rados.rados.0.smithi100.stdout:1843: oid 38 version 1015 is already newer than 1013 2024-04-07T16:22:45.168 INFO:tasks.rados.rados.0.smithi100.stdout:1843: oid 38 version 1015 is already newer than 1014 2024-04-07T16:22:45.168 INFO:tasks.rados.rados.0.smithi100.stdout:1843: oid 38 updating version 1015 to 1016 2024-04-07T16:22:45.168 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 38 v 1016 (ObjNum 562 snap 173 seq_num 562) dirty exists 2024-04-07T16:22:45.168 INFO:tasks.rados.rados.0.smithi100.stdout:1843: left oid 38 (ObjNum 562 snap 173 seq_num 562) 2024-04-07T16:22:45.168 INFO:tasks.rados.rados.0.smithi100.stdout:1843: done (2 left) 2024-04-07T16:22:45.168 INFO:tasks.rados.rados.0.smithi100.stdout:1848: copy_from oid 18 from oid 44 current snap is 173 2024-04-07T16:22:45.168 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 7 current snap is 173 2024-04-07T16:22:45.168 INFO:tasks.rados.rados.0.smithi100.stdout:1849: seq_num 565 ranges {3407872=671744,4079616=729088,4808704=753664,5562368=122880} 2024-04-07T16:22:45.184 INFO:journalctl@ceph.osd.1.smithi100.stdout:Apr 07 16:22:44 smithi100 bash[85238]: debug 2024-04-07T16:22:44.859+0000 7fbe8d9b5700 -1 osd.1 403 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-07T16:22:45.184 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:44 smithi100 bash[76202]: cluster 2024-04-07T16:22:43.820003+0000 mon.a (mon.0) 441 : cluster [DBG] osdmap e414: 8 total, 7 up, 8 in 2024-04-07T16:22:45.184 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:44 smithi100 bash[76202]: audit 2024-04-07T16:22:44.212591+0000 mon.c (mon.2) 7 : audit [INF] from='osd.1 [v2:172.21.15.100:6810/1469336877,v1:172.21.15.100:6811/1469336877]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-07T16:22:45.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:44 smithi100 bash[76202]: audit 2024-04-07T16:22:44.212978+0000 mon.a (mon.0) 442 : audit [INF] from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-07T16:22:45.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:44 smithi100 bash[76202]: audit 2024-04-07T16:22:44.622190+0000 mon.a (mon.0) 443 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:22:45.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:44 smithi100 bash[76202]: audit 2024-04-07T16:22:44.628182+0000 mon.a (mon.0) 444 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:22:45.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:44 smithi100 bash[74942]: cluster 2024-04-07T16:22:43.820003+0000 mon.a (mon.0) 441 : cluster [DBG] osdmap e414: 8 total, 7 up, 8 in 2024-04-07T16:22:45.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:44 smithi100 bash[74942]: audit 2024-04-07T16:22:44.212591+0000 mon.c (mon.2) 7 : audit [INF] from='osd.1 [v2:172.21.15.100:6810/1469336877,v1:172.21.15.100:6811/1469336877]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-07T16:22:45.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:44 smithi100 bash[74942]: audit 2024-04-07T16:22:44.212978+0000 mon.a (mon.0) 442 : audit [INF] from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-07T16:22:45.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:44 smithi100 bash[74942]: audit 2024-04-07T16:22:44.622190+0000 mon.a (mon.0) 443 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:22:45.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:44 smithi100 bash[74942]: audit 2024-04-07T16:22:44.628182+0000 mon.a (mon.0) 444 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:22:45.191 INFO:tasks.rados.rados.0.smithi100.stdout:1849: writing smithi10066867-7 from 3407872 to 4079616 tid 1 2024-04-07T16:22:45.196 INFO:tasks.rados.rados.0.smithi100.stdout:1849: writing smithi10066867-7 from 4079616 to 4808704 tid 2 2024-04-07T16:22:45.201 INFO:tasks.rados.rados.0.smithi100.stdout:1849: writing smithi10066867-7 from 4808704 to 5562368 tid 3 2024-04-07T16:22:45.203 INFO:tasks.rados.rados.0.smithi100.stdout:1849: writing smithi10066867-7 from 5562368 to 5685248 tid 4 2024-04-07T16:22:45.203 INFO:tasks.rados.rados.0.smithi100.stdout:1850: delete oid 46 current snap is 173 2024-04-07T16:22:45.203 INFO:tasks.rados.rados.0.smithi100.stdout:1845: finishing write tid 1 to smithi10066867-47 2024-04-07T16:22:45.203 INFO:tasks.rados.rados.0.smithi100.stdout:1845: finishing write tid 2 to smithi10066867-47 2024-04-07T16:22:45.203 INFO:tasks.rados.rados.0.smithi100.stdout:1845: finishing write tid 3 to smithi10066867-47 2024-04-07T16:22:45.203 INFO:tasks.rados.rados.0.smithi100.stdout:1845: finishing write tid 4 to smithi10066867-47 2024-04-07T16:22:45.203 INFO:tasks.rados.rados.0.smithi100.stdout:1845: finishing write tid 5 to smithi10066867-47 2024-04-07T16:22:45.203 INFO:tasks.rados.rados.0.smithi100.stdout:1845: finishing write tid 6 to smithi10066867-47 2024-04-07T16:22:45.203 INFO:tasks.rados.rados.0.smithi100.stdout:1845: oid 47 updating version 0 to 567 2024-04-07T16:22:45.203 INFO:tasks.rados.rados.0.smithi100.stdout:1845: oid 47 version 567 is already newer than 564 2024-04-07T16:22:45.203 INFO:tasks.rados.rados.0.smithi100.stdout:1845: oid 47 updating version 567 to 568 2024-04-07T16:22:45.203 INFO:tasks.rados.rados.0.smithi100.stdout:1845: oid 47 version 568 is already newer than 565 2024-04-07T16:22:45.204 INFO:tasks.rados.rados.0.smithi100.stdout:1845: oid 47 version 568 is already newer than 566 2024-04-07T16:22:45.204 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 47 v 568 (ObjNum 564 snap 173 seq_num 564) dirty exists 2024-04-07T16:22:45.204 INFO:tasks.rados.rados.0.smithi100.stdout:1845: left oid 47 (ObjNum 564 snap 173 seq_num 564) 2024-04-07T16:22:45.204 INFO:tasks.rados.rados.0.smithi100.stdout:1847: finishing copy_from racing read to smithi10066867-30 2024-04-07T16:22:45.204 INFO:tasks.rados.rados.0.smithi100.stdout:1849: finishing write tid 1 to smithi10066867-7 2024-04-07T16:22:45.204 INFO:tasks.rados.rados.0.smithi100.stdout:1847: finishing copy_from to smithi10066867-30 2024-04-07T16:22:45.204 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 30 v 793 (ObjNum 505 snap 150 seq_num 505) dirty exists 2024-04-07T16:22:45.204 INFO:tasks.rados.rados.0.smithi100.stdout:1848: finishing copy_from racing read to smithi10066867-18 2024-04-07T16:22:45.204 INFO:tasks.rados.rados.0.smithi100.stdout:1849: finishing write tid 2 to smithi10066867-7 2024-04-07T16:22:45.206 INFO:tasks.rados.rados.0.smithi100.stdout:1848: finishing copy_from to smithi10066867-18 2024-04-07T16:22:45.206 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 18 v 798 (ObjNum 509 snap 152 seq_num 509) dirty exists 2024-04-07T16:22:45.209 INFO:tasks.rados.rados.0.smithi100.stdout:1845: done (4 left) 2024-04-07T16:22:45.210 INFO:tasks.rados.rados.0.smithi100.stdout:1847: done (3 left) 2024-04-07T16:22:45.210 INFO:tasks.rados.rados.0.smithi100.stdout:1848: done (2 left) 2024-04-07T16:22:45.210 INFO:tasks.rados.rados.0.smithi100.stdout:1850: done (1 left) 2024-04-07T16:22:45.210 INFO:tasks.rados.rados.0.smithi100.stdout:1851: read oid 29 snap -1 2024-04-07T16:22:45.210 INFO:tasks.rados.rados.0.smithi100.stdout:1851: expect (ObjNum 559 snap 170 seq_num 559) 2024-04-07T16:22:45.210 INFO:tasks.rados.rados.0.smithi100.stdout:1852: rmattr oid 48 current snap is 173 2024-04-07T16:22:45.210 INFO:tasks.rados.rados.0.smithi100.stdout:1852: done (2 left) 2024-04-07T16:22:45.210 INFO:tasks.rados.rados.0.smithi100.stdout:1853: snap_create 2024-04-07T16:22:45.210 INFO:tasks.rados.rados.0.smithi100.stdout:1849: finishing write tid 3 to smithi10066867-7 2024-04-07T16:22:45.211 INFO:tasks.rados.rados.0.smithi100.stdout:1849: finishing write tid 4 to smithi10066867-7 2024-04-07T16:22:45.212 INFO:tasks.rados.rados.0.smithi100.stdout:1849: finishing write tid 5 to smithi10066867-7 2024-04-07T16:22:45.215 INFO:tasks.rados.rados.0.smithi100.stdout:1849: finishing write tid 6 to smithi10066867-7 2024-04-07T16:22:45.215 INFO:tasks.rados.rados.0.smithi100.stdout:1849: oid 7 updating version 0 to 807 2024-04-07T16:22:45.215 INFO:tasks.rados.rados.0.smithi100.stdout:1849: oid 7 version 807 is already newer than 805 2024-04-07T16:22:45.215 INFO:tasks.rados.rados.0.smithi100.stdout:1849: oid 7 updating version 807 to 809 2024-04-07T16:22:45.215 INFO:tasks.rados.rados.0.smithi100.stdout:1849: oid 7 version 809 is already newer than 806 2024-04-07T16:22:45.215 INFO:tasks.rados.rados.0.smithi100.stdout:1849: oid 7 version 809 is already newer than 808 2024-04-07T16:22:45.215 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 7 v 809 (ObjNum 565 snap 173 seq_num 565) dirty exists 2024-04-07T16:22:45.215 INFO:tasks.rados.rados.0.smithi100.stdout:1849: left oid 7 (ObjNum 565 snap 173 seq_num 565) 2024-04-07T16:22:45.237 INFO:tasks.rados.rados.0.smithi100.stdout:1851: expect (ObjNum 559 snap 170 seq_num 559) 2024-04-07T16:22:45.841 INFO:tasks.rados.rados.0.smithi100.stdout:1849: done (2 left) 2024-04-07T16:22:45.841 INFO:tasks.rados.rados.0.smithi100.stdout:1851: done (1 left) 2024-04-07T16:22:45.841 INFO:tasks.rados.rados.0.smithi100.stdout:1853: done (0 left) 2024-04-07T16:22:45.841 INFO:tasks.rados.rados.0.smithi100.stdout:1854: rollback oid 11 current snap is 174 2024-04-07T16:22:45.841 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 11 to 167 2024-04-07T16:22:45.841 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 30 current snap is 174 2024-04-07T16:22:45.841 INFO:tasks.rados.rados.0.smithi100.stdout:1855: seq_num 567 ranges {3547136=704512,4251648=499712,4751360=770048,5521408=335872} 2024-04-07T16:22:45.857 INFO:tasks.rados.rados.0.smithi100.stdout:1855: writing smithi10066867-30 from 3547136 to 4251648 tid 1 2024-04-07T16:22:45.861 INFO:tasks.rados.rados.0.smithi100.stdout:1855: writing smithi10066867-30 from 4251648 to 4751360 tid 2 2024-04-07T16:22:45.866 INFO:tasks.rados.rados.0.smithi100.stdout:1855: writing smithi10066867-30 from 4751360 to 5521408 tid 3 2024-04-07T16:22:45.868 INFO:tasks.rados.rados.0.smithi100.stdout:1855: writing smithi10066867-30 from 5521408 to 5857280 tid 4 2024-04-07T16:22:45.869 INFO:tasks.rados.rados.0.smithi100.stdout:1856: read oid 25 snap -1 2024-04-07T16:22:45.869 INFO:tasks.rados.rados.0.smithi100.stdout:1856: expect (ObjNum 554 snap 168 seq_num 554) 2024-04-07T16:22:45.869 INFO:tasks.rados.rados.0.smithi100.stdout:1857: snap_create 2024-04-07T16:22:45.869 INFO:tasks.rados.rados.0.smithi100.stdout:1854: finishing rollback tid 0 to smithi10066867-11 2024-04-07T16:22:45.869 INFO:tasks.rados.rados.0.smithi100.stdout:1854: finishing rollback tid 1 to smithi10066867-11 2024-04-07T16:22:45.869 INFO:tasks.rados.rados.0.smithi100.stdout:1854: finishing rollback tid 2 to smithi10066867-11 2024-04-07T16:22:45.869 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 11 v 1078 (ObjNum 501 snap 150 seq_num 501) dirty exists 2024-04-07T16:22:45.869 INFO:tasks.rados.rados.0.smithi100.stdout:1855: finishing write tid 1 to smithi10066867-30 2024-04-07T16:22:45.869 INFO:tasks.rados.rados.0.smithi100.stdout:1855: finishing write tid 2 to smithi10066867-30 2024-04-07T16:22:45.874 INFO:tasks.rados.rados.0.smithi100.stdout:1855: finishing write tid 3 to smithi10066867-30 2024-04-07T16:22:45.879 INFO:tasks.rados.rados.0.smithi100.stdout:1856: expect (ObjNum 554 snap 168 seq_num 554) 2024-04-07T16:22:45.933 INFO:tasks.rados.rados.0.smithi100.stdout:1855: finishing write tid 4 to smithi10066867-30 2024-04-07T16:22:45.933 INFO:tasks.rados.rados.0.smithi100.stdout:1855: finishing write tid 5 to smithi10066867-30 2024-04-07T16:22:45.933 INFO:tasks.rados.rados.0.smithi100.stdout:1855: finishing write tid 6 to smithi10066867-30 2024-04-07T16:22:45.933 INFO:tasks.rados.rados.0.smithi100.stdout:1855: oid 30 updating version 0 to 1060 2024-04-07T16:22:45.933 INFO:tasks.rados.rados.0.smithi100.stdout:1855: oid 30 updating version 1060 to 1062 2024-04-07T16:22:45.933 INFO:tasks.rados.rados.0.smithi100.stdout:1855: oid 30 version 1062 is already newer than 1059 2024-04-07T16:22:45.933 INFO:tasks.rados.rados.0.smithi100.stdout:1855: oid 30 updating version 1062 to 1063 2024-04-07T16:22:45.933 INFO:tasks.rados.rados.0.smithi100.stdout:1855: oid 30 version 1063 is already newer than 1061 2024-04-07T16:22:45.933 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 30 v 1063 (ObjNum 567 snap 174 seq_num 567) dirty exists 2024-04-07T16:22:45.933 INFO:tasks.rados.rados.0.smithi100.stdout:1855: left oid 30 (ObjNum 567 snap 174 seq_num 567) 2024-04-07T16:22:46.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:45 smithi144 bash[66654]: cluster 2024-04-07T16:22:44.659263+0000 mgr.y (mgr.25175) 143 : cluster [DBG] pgmap v211: 121 pgs: 11 active+undersized+degraded, 34 active+undersized, 76 active+clean; 189 MiB data, 1.7 GiB used, 714 GiB / 715 GiB avail; 8.6 MiB/s rd, 6.1 MiB/s wr, 31 op/s; 48/285 objects degraded (16.842%) 2024-04-07T16:22:46.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:45 smithi144 bash[66654]: audit 2024-04-07T16:22:44.815782+0000 mon.a (mon.0) 445 : audit [INF] from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-07T16:22:46.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:45 smithi144 bash[66654]: cluster 2024-04-07T16:22:44.818835+0000 mon.a (mon.0) 446 : cluster [DBG] osdmap e415: 8 total, 7 up, 8 in 2024-04-07T16:22:46.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:45 smithi144 bash[66654]: audit 2024-04-07T16:22:44.822022+0000 mon.c (mon.2) 8 : audit [INF] from='osd.1 [v2:172.21.15.100:6810/1469336877,v1:172.21.15.100:6811/1469336877]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-04-07T16:22:46.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:45 smithi144 bash[66654]: audit 2024-04-07T16:22:44.831085+0000 mon.a (mon.0) 447 : audit [INF] from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-04-07T16:22:46.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:45 smithi144 bash[66654]: cluster 2024-04-07T16:22:45.625953+0000 mon.a (mon.0) 448 : cluster [WRN] Health check update: Degraded data redundancy: 48/285 objects degraded (16.842%), 11 pgs degraded (PG_DEGRADED) 2024-04-07T16:22:46.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:45 smithi144 bash[66654]: audit 2024-04-07T16:22:45.636784+0000 mon.a (mon.0) 449 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:22:46.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:45 smithi144 bash[66654]: audit 2024-04-07T16:22:45.643471+0000 mon.a (mon.0) 450 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:22:46.184 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:45 smithi100 bash[74942]: cluster 2024-04-07T16:22:44.659263+0000 mgr.y (mgr.25175) 143 : cluster [DBG] pgmap v211: 121 pgs: 11 active+undersized+degraded, 34 active+undersized, 76 active+clean; 189 MiB data, 1.7 GiB used, 714 GiB / 715 GiB avail; 8.6 MiB/s rd, 6.1 MiB/s wr, 31 op/s; 48/285 objects degraded (16.842%) 2024-04-07T16:22:46.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:45 smithi100 bash[74942]: audit 2024-04-07T16:22:44.815782+0000 mon.a (mon.0) 445 : audit [INF] from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-07T16:22:46.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:45 smithi100 bash[74942]: cluster 2024-04-07T16:22:44.818835+0000 mon.a (mon.0) 446 : cluster [DBG] osdmap e415: 8 total, 7 up, 8 in 2024-04-07T16:22:46.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:45 smithi100 bash[74942]: audit 2024-04-07T16:22:44.822022+0000 mon.c (mon.2) 8 : audit [INF] from='osd.1 [v2:172.21.15.100:6810/1469336877,v1:172.21.15.100:6811/1469336877]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-04-07T16:22:46.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:45 smithi100 bash[74942]: audit 2024-04-07T16:22:44.831085+0000 mon.a (mon.0) 447 : audit [INF] from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-04-07T16:22:46.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:45 smithi100 bash[74942]: cluster 2024-04-07T16:22:45.625953+0000 mon.a (mon.0) 448 : cluster [WRN] Health check update: Degraded data redundancy: 48/285 objects degraded (16.842%), 11 pgs degraded (PG_DEGRADED) 2024-04-07T16:22:46.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:45 smithi100 bash[74942]: audit 2024-04-07T16:22:45.636784+0000 mon.a (mon.0) 449 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:22:46.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:45 smithi100 bash[74942]: audit 2024-04-07T16:22:45.643471+0000 mon.a (mon.0) 450 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:22:46.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:45 smithi100 bash[76202]: cluster 2024-04-07T16:22:44.659263+0000 mgr.y (mgr.25175) 143 : cluster [DBG] pgmap v211: 121 pgs: 11 active+undersized+degraded, 34 active+undersized, 76 active+clean; 189 MiB data, 1.7 GiB used, 714 GiB / 715 GiB avail; 8.6 MiB/s rd, 6.1 MiB/s wr, 31 op/s; 48/285 objects degraded (16.842%) 2024-04-07T16:22:46.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:45 smithi100 bash[76202]: audit 2024-04-07T16:22:44.815782+0000 mon.a (mon.0) 445 : audit [INF] from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-07T16:22:46.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:45 smithi100 bash[76202]: cluster 2024-04-07T16:22:44.818835+0000 mon.a (mon.0) 446 : cluster [DBG] osdmap e415: 8 total, 7 up, 8 in 2024-04-07T16:22:46.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:45 smithi100 bash[76202]: audit 2024-04-07T16:22:44.822022+0000 mon.c (mon.2) 8 : audit [INF] from='osd.1 [v2:172.21.15.100:6810/1469336877,v1:172.21.15.100:6811/1469336877]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-04-07T16:22:46.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:45 smithi100 bash[76202]: audit 2024-04-07T16:22:44.831085+0000 mon.a (mon.0) 447 : audit [INF] from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-04-07T16:22:46.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:45 smithi100 bash[76202]: cluster 2024-04-07T16:22:45.625953+0000 mon.a (mon.0) 448 : cluster [WRN] Health check update: Degraded data redundancy: 48/285 objects degraded (16.842%), 11 pgs degraded (PG_DEGRADED) 2024-04-07T16:22:46.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:45 smithi100 bash[76202]: audit 2024-04-07T16:22:45.636784+0000 mon.a (mon.0) 449 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:22:46.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:45 smithi100 bash[76202]: audit 2024-04-07T16:22:45.643471+0000 mon.a (mon.0) 450 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:22:46.842 INFO:tasks.rados.rados.0.smithi100.stdout:1854: done (3 left) 2024-04-07T16:22:46.843 INFO:tasks.rados.rados.0.smithi100.stdout:1855: done (2 left) 2024-04-07T16:22:46.843 INFO:tasks.rados.rados.0.smithi100.stdout:1856: done (1 left) 2024-04-07T16:22:46.843 INFO:tasks.rados.rados.0.smithi100.stdout:1857: done (0 left) 2024-04-07T16:22:46.843 INFO:tasks.rados.rados.0.smithi100.stdout:1858: setattr oid 37 current snap is 175 2024-04-07T16:22:46.845 INFO:tasks.rados.rados.0.smithi100.stdout:1859: copy_from oid 34 from oid 33 current snap is 175 2024-04-07T16:22:46.846 INFO:tasks.rados.rados.0.smithi100.stdout:1860: copy_from oid 16 from oid 43 current snap is 175 2024-04-07T16:22:46.846 INFO:tasks.rados.rados.0.smithi100.stdout:1861: rollback oid 13 current snap is 175 2024-04-07T16:22:46.846 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 13 to 171 2024-04-07T16:22:46.846 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 24 current snap is 175 2024-04-07T16:22:46.846 INFO:tasks.rados.rados.0.smithi100.stdout:1862: seq_num 568 ranges {0=720896,720896=786432,1507328=581632,2088960=237568} 2024-04-07T16:22:46.850 INFO:tasks.rados.rados.0.smithi100.stdout:1862: writing smithi10066867-24 from 0 to 720896 tid 1 2024-04-07T16:22:46.856 INFO:tasks.rados.rados.0.smithi100.stdout:1862: writing smithi10066867-24 from 720896 to 1507328 tid 2 2024-04-07T16:22:46.861 INFO:tasks.rados.rados.0.smithi100.stdout:1862: writing smithi10066867-24 from 1507328 to 2088960 tid 3 2024-04-07T16:22:46.864 INFO:tasks.rados.rados.0.smithi100.stdout:1862: writing smithi10066867-24 from 2088960 to 2326528 tid 4 2024-04-07T16:22:46.864 INFO:tasks.rados.rados.0.smithi100.stdout:1863: copy_from oid 38 from oid 11 current snap is 175 2024-04-07T16:22:46.864 INFO:tasks.rados.rados.0.smithi100.stdout:1864: rollback oid 47 current snap is 175 2024-04-07T16:22:46.864 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 47 to 169 2024-04-07T16:22:46.865 INFO:tasks.rados.rados.0.smithi100.stdout:1865: snap_create 2024-04-07T16:22:46.865 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 37 v 1064 (ObjNum 501 snap 150 seq_num 501) dirty exists 2024-04-07T16:22:46.873 INFO:tasks.rados.rados.0.smithi100.stdout:1864: finishing rollback tid 0 to smithi10066867-47 2024-04-07T16:22:46.875 INFO:tasks.rados.rados.0.smithi100.stdout:1864: finishing rollback tid 1 to smithi10066867-47 2024-04-07T16:22:46.877 INFO:tasks.rados.rados.0.smithi100.stdout:1864: finishing rollback tid 2 to smithi10066867-47 2024-04-07T16:22:46.877 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 47 v 571 (ObjNum 556 snap 168 seq_num 556) dirty exists 2024-04-07T16:22:46.891 INFO:tasks.rados.rados.0.smithi100.stdout:1859: finishing copy_from to smithi10066867-34 2024-04-07T16:22:46.891 INFO:tasks.rados.rados.0.smithi100.stdout:1859: got expected ENOENT (src dne) 2024-04-07T16:22:46.892 INFO:tasks.rados.rados.0.smithi100.stdout:1859: finishing copy_from racing read to smithi10066867-34 2024-04-07T16:22:46.926 INFO:tasks.rados.rados.0.smithi100.stdout:1862: finishing write tid 1 to smithi10066867-24 2024-04-07T16:22:46.935 INFO:tasks.rados.rados.0.smithi100.stdout:1862: finishing write tid 2 to smithi10066867-24 2024-04-07T16:22:46.935 INFO:tasks.rados.rados.0.smithi100.stdout:1862: finishing write tid 3 to smithi10066867-24 2024-04-07T16:22:46.936 INFO:tasks.rados.rados.0.smithi100.stdout:1862: finishing write tid 4 to smithi10066867-24 2024-04-07T16:22:46.936 INFO:tasks.rados.rados.0.smithi100.stdout:1862: finishing write tid 5 to smithi10066867-24 2024-04-07T16:22:46.939 INFO:tasks.rados.rados.0.smithi100.stdout:1862: finishing write tid 6 to smithi10066867-24 2024-04-07T16:22:46.939 INFO:tasks.rados.rados.0.smithi100.stdout:1862: oid 24 updating version 0 to 734 2024-04-07T16:22:46.940 INFO:tasks.rados.rados.0.smithi100.stdout:1862: oid 24 version 734 is already newer than 732 2024-04-07T16:22:46.940 INFO:tasks.rados.rados.0.smithi100.stdout:1862: oid 24 version 734 is already newer than 731 2024-04-07T16:22:46.940 INFO:tasks.rados.rados.0.smithi100.stdout:1862: oid 24 version 734 is already newer than 733 2024-04-07T16:22:46.940 INFO:tasks.rados.rados.0.smithi100.stdout:1862: oid 24 version 734 is already newer than 730 2024-04-07T16:22:46.940 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 24 v 734 (ObjNum 568 snap 175 seq_num 568) dirty exists 2024-04-07T16:22:46.940 INFO:tasks.rados.rados.0.smithi100.stdout:1862: left oid 24 (ObjNum 568 snap 175 seq_num 568) 2024-04-07T16:22:46.941 INFO:tasks.rados.rados.0.smithi100.stdout:1861: finishing rollback tid 0 to smithi10066867-13 2024-04-07T16:22:46.941 INFO:tasks.rados.rados.0.smithi100.stdout:1861: finishing rollback tid 1 to smithi10066867-13 2024-04-07T16:22:46.941 INFO:tasks.rados.rados.0.smithi100.stdout:1861: finishing rollback tid 2 to smithi10066867-13 2024-04-07T16:22:46.941 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 13 v 740 (ObjNum 537 snap 162 seq_num 537) dirty exists 2024-04-07T16:22:46.947 INFO:tasks.rados.rados.0.smithi100.stdout:1863: finishing copy_from racing read to smithi10066867-38 2024-04-07T16:22:46.949 INFO:tasks.rados.rados.0.smithi100.stdout:1863: finishing copy_from to smithi10066867-38 2024-04-07T16:22:46.949 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 38 v 1078 (ObjNum 501 snap 150 seq_num 501) dirty exists 2024-04-07T16:22:46.972 INFO:tasks.rados.rados.0.smithi100.stdout:1860: finishing copy_from racing read to smithi10066867-16 2024-04-07T16:22:46.974 INFO:tasks.rados.rados.0.smithi100.stdout:1860: finishing copy_from to smithi10066867-16 2024-04-07T16:22:46.975 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 16 v 624 (ObjNum 465 snap 142 seq_num 465) dirty exists 2024-04-07T16:22:47.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:46 smithi144 bash[66654]: cluster 2024-04-07T16:22:45.825337+0000 mon.a (mon.0) 451 : cluster [INF] Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-07T16:22:47.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:46 smithi144 bash[66654]: cluster 2024-04-07T16:22:45.847450+0000 mon.a (mon.0) 452 : cluster [INF] osd.1 [v2:172.21.15.100:6810/1469336877,v1:172.21.15.100:6811/1469336877] boot 2024-04-07T16:22:47.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:46 smithi144 bash[66654]: cluster 2024-04-07T16:22:45.847642+0000 mon.a (mon.0) 453 : cluster [DBG] osdmap e416: 8 total, 8 up, 8 in 2024-04-07T16:22:47.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:46 smithi144 bash[66654]: audit 2024-04-07T16:22:45.849336+0000 mon.a (mon.0) 454 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-07T16:22:47.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:46 smithi144 bash[66654]: audit 2024-04-07T16:22:46.712024+0000 mon.a (mon.0) 455 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:22:47.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:46 smithi144 bash[66654]: audit 2024-04-07T16:22:46.714128+0000 mon.a (mon.0) 456 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:22:47.184 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:46 smithi100 bash[74942]: cluster 2024-04-07T16:22:45.825337+0000 mon.a (mon.0) 451 : cluster [INF] Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-07T16:22:47.184 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:46 smithi100 bash[74942]: cluster 2024-04-07T16:22:45.847450+0000 mon.a (mon.0) 452 : cluster [INF] osd.1 [v2:172.21.15.100:6810/1469336877,v1:172.21.15.100:6811/1469336877] boot 2024-04-07T16:22:47.184 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:46 smithi100 bash[74942]: cluster 2024-04-07T16:22:45.847642+0000 mon.a (mon.0) 453 : cluster [DBG] osdmap e416: 8 total, 8 up, 8 in 2024-04-07T16:22:47.184 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:46 smithi100 bash[74942]: audit 2024-04-07T16:22:45.849336+0000 mon.a (mon.0) 454 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-07T16:22:47.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:46 smithi100 bash[74942]: audit 2024-04-07T16:22:46.712024+0000 mon.a (mon.0) 455 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:22:47.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:46 smithi100 bash[74942]: audit 2024-04-07T16:22:46.714128+0000 mon.a (mon.0) 456 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:22:47.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:46 smithi100 bash[76202]: cluster 2024-04-07T16:22:45.825337+0000 mon.a (mon.0) 451 : cluster [INF] Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-07T16:22:47.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:46 smithi100 bash[76202]: cluster 2024-04-07T16:22:45.847450+0000 mon.a (mon.0) 452 : cluster [INF] osd.1 [v2:172.21.15.100:6810/1469336877,v1:172.21.15.100:6811/1469336877] boot 2024-04-07T16:22:47.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:46 smithi100 bash[76202]: cluster 2024-04-07T16:22:45.847642+0000 mon.a (mon.0) 453 : cluster [DBG] osdmap e416: 8 total, 8 up, 8 in 2024-04-07T16:22:47.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:46 smithi100 bash[76202]: audit 2024-04-07T16:22:45.849336+0000 mon.a (mon.0) 454 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-07T16:22:47.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:46 smithi100 bash[76202]: audit 2024-04-07T16:22:46.712024+0000 mon.a (mon.0) 455 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:22:47.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:46 smithi100 bash[76202]: audit 2024-04-07T16:22:46.714128+0000 mon.a (mon.0) 456 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:22:47.871 INFO:tasks.rados.rados.0.smithi100.stdout:1858: done (7 left) 2024-04-07T16:22:47.871 INFO:tasks.rados.rados.0.smithi100.stdout:1859: done (6 left) 2024-04-07T16:22:47.871 INFO:tasks.rados.rados.0.smithi100.stdout:1860: done (5 left) 2024-04-07T16:22:47.872 INFO:tasks.rados.rados.0.smithi100.stdout:1861: done (4 left) 2024-04-07T16:22:47.872 INFO:tasks.rados.rados.0.smithi100.stdout:1862: done (3 left) 2024-04-07T16:22:47.872 INFO:tasks.rados.rados.0.smithi100.stdout:1863: done (2 left) 2024-04-07T16:22:47.872 INFO:tasks.rados.rados.0.smithi100.stdout:1864: done (1 left) 2024-04-07T16:22:47.872 INFO:tasks.rados.rados.0.smithi100.stdout:1865: done (0 left) 2024-04-07T16:22:47.872 INFO:tasks.rados.rados.0.smithi100.stdout:1866: rmattr oid 9 current snap is 176 2024-04-07T16:22:47.872 INFO:tasks.rados.rados.0.smithi100.stdout:1867: copy_from oid 11 from oid 13 current snap is 176 2024-04-07T16:22:47.872 INFO:tasks.rados.rados.0.smithi100.stdout:1868: delete oid 35 current snap is 176 2024-04-07T16:22:47.876 INFO:tasks.rados.rados.0.smithi100.stdout:1868: done (2 left) 2024-04-07T16:22:47.876 INFO:tasks.rados.rados.0.smithi100.stdout:1869: rollback oid 49 current snap is 176 2024-04-07T16:22:47.877 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 49 to 173 2024-04-07T16:22:47.877 INFO:tasks.rados.rados.0.smithi100.stdout:1870: snap_create 2024-04-07T16:22:47.882 INFO:tasks.rados.rados.0.smithi100.stdout:1869: finishing rollback tid 1 to smithi10066867-49 2024-04-07T16:22:47.882 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 49 v 0 (ObjNum 97 snap 0 seq_num 4239996972) dirty dne 2024-04-07T16:22:47.887 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 9 v 548 (ObjNum 546 snap 166 seq_num 546) dirty exists 2024-04-07T16:22:47.903 INFO:tasks.rados.rados.0.smithi100.stdout:1867: finishing copy_from racing read to smithi10066867-11 2024-04-07T16:22:47.904 INFO:tasks.rados.rados.0.smithi100.stdout:1867: finishing copy_from to smithi10066867-11 2024-04-07T16:22:47.904 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 11 v 740 (ObjNum 537 snap 162 seq_num 537) dirty exists 2024-04-07T16:22:48.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:47 smithi144 bash[66654]: cluster 2024-04-07T16:22:46.660123+0000 mgr.y (mgr.25175) 144 : cluster [DBG] pgmap v214: 121 pgs: 11 active+undersized+degraded, 34 active+undersized, 76 active+clean; 189 MiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 877 KiB/s rd, 2.0 MiB/s wr, 8 op/s; 48/285 objects degraded (16.842%) 2024-04-07T16:22:48.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:47 smithi144 bash[66654]: cluster 2024-04-07T16:22:46.856029+0000 mon.a (mon.0) 457 : cluster [DBG] osdmap e417: 8 total, 8 up, 8 in 2024-04-07T16:22:48.184 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:47 smithi100 bash[74942]: cluster 2024-04-07T16:22:46.660123+0000 mgr.y (mgr.25175) 144 : cluster [DBG] pgmap v214: 121 pgs: 11 active+undersized+degraded, 34 active+undersized, 76 active+clean; 189 MiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 877 KiB/s rd, 2.0 MiB/s wr, 8 op/s; 48/285 objects degraded (16.842%) 2024-04-07T16:22:48.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:47 smithi100 bash[74942]: cluster 2024-04-07T16:22:46.856029+0000 mon.a (mon.0) 457 : cluster [DBG] osdmap e417: 8 total, 8 up, 8 in 2024-04-07T16:22:48.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:47 smithi100 bash[76202]: cluster 2024-04-07T16:22:46.660123+0000 mgr.y (mgr.25175) 144 : cluster [DBG] pgmap v214: 121 pgs: 11 active+undersized+degraded, 34 active+undersized, 76 active+clean; 189 MiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 877 KiB/s rd, 2.0 MiB/s wr, 8 op/s; 48/285 objects degraded (16.842%) 2024-04-07T16:22:48.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:47 smithi100 bash[76202]: cluster 2024-04-07T16:22:46.856029+0000 mon.a (mon.0) 457 : cluster [DBG] osdmap e417: 8 total, 8 up, 8 in 2024-04-07T16:22:48.684 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:22:48 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:22:48] "GET /metrics HTTP/1.1" 200 36067 "" "Prometheus/2.43.0" 2024-04-07T16:22:48.885 INFO:tasks.rados.rados.0.smithi100.stdout:1866: done (3 left) 2024-04-07T16:22:48.886 INFO:tasks.rados.rados.0.smithi100.stdout:1867: done (2 left) 2024-04-07T16:22:48.886 INFO:tasks.rados.rados.0.smithi100.stdout:1869: done (1 left) 2024-04-07T16:22:48.886 INFO:tasks.rados.rados.0.smithi100.stdout:1870: done (0 left) 2024-04-07T16:22:48.886 INFO:tasks.rados.rados.0.smithi100.stdout:1871: read oid 30 snap 169 2024-04-07T16:22:48.886 INFO:tasks.rados.rados.0.smithi100.stdout:1871: expect deleted 2024-04-07T16:22:48.886 INFO:tasks.rados.rados.0.smithi100.stdout:1872: rollback oid 17 current snap is 177 2024-04-07T16:22:48.886 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 17 to 167 2024-04-07T16:22:48.886 INFO:tasks.rados.rados.0.smithi100.stdout:1873: read oid 14 snap 169 2024-04-07T16:22:48.886 INFO:tasks.rados.rados.0.smithi100.stdout:1873: expect (ObjNum 536 snap 162 seq_num 536) 2024-04-07T16:22:48.886 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 13 current snap is 177 2024-04-07T16:22:48.886 INFO:tasks.rados.rados.0.smithi100.stdout:1874: seq_num 570 ranges {2564096=753664,3317760=548864,3866624=540672,4407296=516096} 2024-04-07T16:22:48.924 INFO:tasks.rados.rados.0.smithi100.stdout:1874: writing smithi10066867-13 from 2564096 to 3317760 tid 1 2024-04-07T16:22:48.930 INFO:tasks.rados.rados.0.smithi100.stdout:1874: writing smithi10066867-13 from 3317760 to 3866624 tid 2 2024-04-07T16:22:48.936 INFO:tasks.rados.rados.0.smithi100.stdout:1874: writing smithi10066867-13 from 3866624 to 4407296 tid 3 2024-04-07T16:22:48.941 INFO:tasks.rados.rados.0.smithi100.stdout:1874: writing smithi10066867-13 from 4407296 to 4923392 tid 4 2024-04-07T16:22:48.942 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 23 current snap is 177 2024-04-07T16:22:48.942 INFO:tasks.rados.rados.0.smithi100.stdout:1875: seq_num 571 ranges {2990080=770048,3760128=483328,4243456=753664,4997120=368640} 2024-04-07T16:22:48.969 INFO:tasks.rados.rados.0.smithi100.stdout:1875: writing smithi10066867-23 from 2990080 to 3760128 tid 1 2024-04-07T16:22:48.973 INFO:tasks.rados.rados.0.smithi100.stdout:1875: writing smithi10066867-23 from 3760128 to 4243456 tid 2 2024-04-07T16:22:48.978 INFO:tasks.rados.rados.0.smithi100.stdout:1875: writing smithi10066867-23 from 4243456 to 4997120 tid 3 2024-04-07T16:22:48.981 INFO:tasks.rados.rados.0.smithi100.stdout:1875: writing smithi10066867-23 from 4997120 to 5365760 tid 4 2024-04-07T16:22:48.982 INFO:tasks.rados.rados.0.smithi100.stdout:1876: rollback oid 32 current snap is 177 2024-04-07T16:22:48.982 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 32 to 169 2024-04-07T16:22:48.982 INFO:tasks.rados.rados.0.smithi100.stdout:1872: finishing rollback tid 0 to smithi10066867-17 2024-04-07T16:22:48.982 INFO:tasks.rados.rados.0.smithi100.stdout:1872: finishing rollback tid 1 to smithi10066867-17 2024-04-07T16:22:48.982 INFO:tasks.rados.rados.0.smithi100.stdout:1872: finishing rollback tid 2 to smithi10066867-17 2024-04-07T16:22:48.982 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 17 v 737 (ObjNum 506 snap 151 seq_num 506) dirty exists 2024-04-07T16:22:48.983 INFO:tasks.rados.rados.0.smithi100.stdout:1873: expect (ObjNum 536 snap 162 seq_num 536) 2024-04-07T16:22:49.076 INFO:tasks.rados.rados.0.smithi100.stdout:1874: finishing write tid 1 to smithi10066867-13 2024-04-07T16:22:49.076 INFO:tasks.rados.rados.0.smithi100.stdout:1874: finishing write tid 2 to smithi10066867-13 2024-04-07T16:22:49.077 INFO:tasks.rados.rados.0.smithi100.stdout:1874: finishing write tid 3 to smithi10066867-13 2024-04-07T16:22:49.077 INFO:tasks.rados.rados.0.smithi100.stdout:1874: finishing write tid 4 to smithi10066867-13 2024-04-07T16:22:49.077 INFO:tasks.rados.rados.0.smithi100.stdout:1874: finishing write tid 5 to smithi10066867-13 2024-04-07T16:22:49.077 INFO:tasks.rados.rados.0.smithi100.stdout:1874: finishing write tid 6 to smithi10066867-13 2024-04-07T16:22:49.077 INFO:tasks.rados.rados.0.smithi100.stdout:1874: oid 13 updating version 0 to 741 2024-04-07T16:22:49.077 INFO:tasks.rados.rados.0.smithi100.stdout:1874: oid 13 updating version 741 to 742 2024-04-07T16:22:49.077 INFO:tasks.rados.rados.0.smithi100.stdout:1874: oid 13 updating version 742 to 744 2024-04-07T16:22:49.077 INFO:tasks.rados.rados.0.smithi100.stdout:1874: oid 13 version 744 is already newer than 743 2024-04-07T16:22:49.077 INFO:tasks.rados.rados.0.smithi100.stdout:1874: oid 13 updating version 744 to 745 2024-04-07T16:22:49.077 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 13 v 745 (ObjNum 570 snap 177 seq_num 570) dirty exists 2024-04-07T16:22:49.077 INFO:tasks.rados.rados.0.smithi100.stdout:1874: left oid 13 (ObjNum 570 snap 177 seq_num 570) 2024-04-07T16:22:49.077 INFO:tasks.rados.rados.0.smithi100.stdout:1875: finishing write tid 1 to smithi10066867-23 2024-04-07T16:22:49.078 INFO:tasks.rados.rados.0.smithi100.stdout:1875: finishing write tid 2 to smithi10066867-23 2024-04-07T16:22:49.078 INFO:tasks.rados.rados.0.smithi100.stdout:1875: finishing write tid 3 to smithi10066867-23 2024-04-07T16:22:49.078 INFO:tasks.rados.rados.0.smithi100.stdout:1871: done (5 left) 2024-04-07T16:22:49.078 INFO:tasks.rados.rados.0.smithi100.stdout:1872: done (4 left) 2024-04-07T16:22:49.078 INFO:tasks.rados.rados.0.smithi100.stdout:1873: done (3 left) 2024-04-07T16:22:49.078 INFO:tasks.rados.rados.0.smithi100.stdout:1874: done (2 left) 2024-04-07T16:22:49.078 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 35 current snap is 177 2024-04-07T16:22:49.078 INFO:tasks.rados.rados.0.smithi100.stdout:1875: finishing write tid 4 to smithi10066867-23 2024-04-07T16:22:49.078 INFO:tasks.rados.rados.0.smithi100.stdout:1877: seq_num 572 ranges {0=786432,786432=770048,1556480=557056,2113536=278528} 2024-04-07T16:22:49.081 INFO:tasks.rados.rados.0.smithi100.stdout:1877: writing smithi10066867-35 from 0 to 786432 tid 1 2024-04-07T16:22:49.087 INFO:tasks.rados.rados.0.smithi100.stdout:1877: writing smithi10066867-35 from 786432 to 1556480 tid 2 2024-04-07T16:22:49.091 INFO:tasks.rados.rados.0.smithi100.stdout:1877: writing smithi10066867-35 from 1556480 to 2113536 tid 3 2024-04-07T16:22:49.093 INFO:tasks.rados.rados.0.smithi100.stdout:1877: writing smithi10066867-35 from 2113536 to 2392064 tid 4 2024-04-07T16:22:49.094 INFO:tasks.rados.rados.0.smithi100.stdout:1878: read oid 18 snap -1 2024-04-07T16:22:49.094 INFO:tasks.rados.rados.0.smithi100.stdout:1878: expect (ObjNum 509 snap 152 seq_num 509) 2024-04-07T16:22:49.094 INFO:tasks.rados.rados.0.smithi100.stdout:1879: snap_remove snap 173 2024-04-07T16:22:49.184 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:48 smithi100 bash[74942]: cluster 2024-04-07T16:22:47.887355+0000 mon.a (mon.0) 458 : cluster [DBG] osdmap e418: 8 total, 8 up, 8 in 2024-04-07T16:22:49.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:48 smithi100 bash[76202]: cluster 2024-04-07T16:22:47.887355+0000 mon.a (mon.0) 458 : cluster [DBG] osdmap e418: 8 total, 8 up, 8 in 2024-04-07T16:22:49.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:48 smithi144 bash[66654]: cluster 2024-04-07T16:22:47.887355+0000 mon.a (mon.0) 458 : cluster [DBG] osdmap e418: 8 total, 8 up, 8 in 2024-04-07T16:22:49.901 INFO:tasks.rados.rados.0.smithi100.stdout:1879: done (4 left) 2024-04-07T16:22:49.901 INFO:tasks.rados.rados.0.smithi100.stdout:1880: delete oid 9 current snap is 177 2024-04-07T16:22:49.901 INFO:tasks.rados.rados.0.smithi100.stdout:1875: finishing write tid 5 to smithi10066867-23 2024-04-07T16:22:49.901 INFO:tasks.rados.rados.0.smithi100.stdout:1876: finishing rollback tid 1 to smithi10066867-32 2024-04-07T16:22:49.901 INFO:tasks.rados.rados.0.smithi100.stdout:1876: finishing rollback tid 2 to smithi10066867-32 2024-04-07T16:22:49.902 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 32 v 769 (ObjNum 496 snap 149 seq_num 496) dirty exists 2024-04-07T16:22:49.902 INFO:tasks.rados.rados.0.smithi100.stdout:1875: finishing write tid 6 to smithi10066867-23 2024-04-07T16:22:49.902 INFO:tasks.rados.rados.0.smithi100.stdout:1875: oid 23 updating version 0 to 1082 2024-04-07T16:22:49.902 INFO:tasks.rados.rados.0.smithi100.stdout:1875: oid 23 updating version 1082 to 1084 2024-04-07T16:22:49.902 INFO:tasks.rados.rados.0.smithi100.stdout:1875: oid 23 version 1084 is already newer than 1083 2024-04-07T16:22:49.902 INFO:tasks.rados.rados.0.smithi100.stdout:1875: oid 23 updating version 1084 to 1086 2024-04-07T16:22:49.902 INFO:tasks.rados.rados.0.smithi100.stdout:1875: oid 23 version 1086 is already newer than 1085 2024-04-07T16:22:49.902 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 23 v 1086 (ObjNum 571 snap 177 seq_num 571) dirty exists 2024-04-07T16:22:49.902 INFO:tasks.rados.rados.0.smithi100.stdout:1875: left oid 23 (ObjNum 571 snap 177 seq_num 571) 2024-04-07T16:22:49.902 INFO:tasks.rados.rados.0.smithi100.stdout:1877: finishing write tid 1 to smithi10066867-35 2024-04-07T16:22:49.903 INFO:tasks.rados.rados.0.smithi100.stdout:1877: finishing write tid 2 to smithi10066867-35 2024-04-07T16:22:49.903 INFO:tasks.rados.rados.0.smithi100.stdout:1877: finishing write tid 3 to smithi10066867-35 2024-04-07T16:22:49.903 INFO:tasks.rados.rados.0.smithi100.stdout:1877: finishing write tid 4 to smithi10066867-35 2024-04-07T16:22:49.903 INFO:tasks.rados.rados.0.smithi100.stdout:1877: finishing write tid 5 to smithi10066867-35 2024-04-07T16:22:49.903 INFO:tasks.rados.rados.0.smithi100.stdout:1877: finishing write tid 6 to smithi10066867-35 2024-04-07T16:22:49.903 INFO:tasks.rados.rados.0.smithi100.stdout:1877: oid 35 updating version 0 to 1049 2024-04-07T16:22:49.903 INFO:tasks.rados.rados.0.smithi100.stdout:1877: oid 35 updating version 1049 to 1051 2024-04-07T16:22:49.903 INFO:tasks.rados.rados.0.smithi100.stdout:1877: oid 35 version 1051 is already newer than 1050 2024-04-07T16:22:49.903 INFO:tasks.rados.rados.0.smithi100.stdout:1877: oid 35 version 1051 is already newer than 1048 2024-04-07T16:22:49.903 INFO:tasks.rados.rados.0.smithi100.stdout:1877: oid 35 version 1051 is already newer than 1047 2024-04-07T16:22:49.903 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 35 v 1051 (ObjNum 572 snap 177 seq_num 572) dirty exists 2024-04-07T16:22:49.903 INFO:tasks.rados.rados.0.smithi100.stdout:1877: left oid 35 (ObjNum 572 snap 177 seq_num 572) 2024-04-07T16:22:49.904 INFO:tasks.rados.rados.0.smithi100.stdout:1878: expect (ObjNum 509 snap 152 seq_num 509) 2024-04-07T16:22:50.184 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:49 smithi100 bash[74942]: cluster 2024-04-07T16:22:48.660886+0000 mgr.y (mgr.25175) 145 : cluster [DBG] pgmap v217: 121 pgs: 4 active+undersized+degraded, 12 active+undersized, 105 active+clean; 194 MiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 26 MiB/s rd, 5.6 MiB/s wr, 24 op/s; 9/303 objects degraded (2.970%); 26 MiB/s, 7 objects/s recovering 2024-04-07T16:22:50.184 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:49 smithi100 bash[74942]: cluster 2024-04-07T16:22:48.901477+0000 mon.a (mon.0) 459 : cluster [DBG] osdmap e419: 8 total, 8 up, 8 in 2024-04-07T16:22:50.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:49 smithi100 bash[76202]: cluster 2024-04-07T16:22:48.660886+0000 mgr.y (mgr.25175) 145 : cluster [DBG] pgmap v217: 121 pgs: 4 active+undersized+degraded, 12 active+undersized, 105 active+clean; 194 MiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 26 MiB/s rd, 5.6 MiB/s wr, 24 op/s; 9/303 objects degraded (2.970%); 26 MiB/s, 7 objects/s recovering 2024-04-07T16:22:50.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:49 smithi100 bash[76202]: cluster 2024-04-07T16:22:48.901477+0000 mon.a (mon.0) 459 : cluster [DBG] osdmap e419: 8 total, 8 up, 8 in 2024-04-07T16:22:50.333 INFO:tasks.rados.rados.0.smithi100.stdout:1875: done (4 left) 2024-04-07T16:22:50.333 INFO:tasks.rados.rados.0.smithi100.stdout:1876: done (3 left) 2024-04-07T16:22:50.333 INFO:tasks.rados.rados.0.smithi100.stdout:1877: done (2 left) 2024-04-07T16:22:50.333 INFO:tasks.rados.rados.0.smithi100.stdout:1878: done (1 left) 2024-04-07T16:22:50.333 INFO:tasks.rados.rados.0.smithi100.stdout:1880: done (0 left) 2024-04-07T16:22:50.334 INFO:tasks.rados.rados.0.smithi100.stdout:1881: setattr oid 44 current snap is 177 2024-04-07T16:22:50.336 INFO:tasks.rados.rados.0.smithi100.stdout:1882: rollback oid 18 current snap is 177 2024-04-07T16:22:50.336 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 18 to 175 2024-04-07T16:22:50.336 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 48 current snap is 177 2024-04-07T16:22:50.336 INFO:tasks.rados.rados.0.smithi100.stdout:1883: seq_num 574 ranges {3801088=16384} 2024-04-07T16:22:50.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:49 smithi144 bash[66654]: cluster 2024-04-07T16:22:48.660886+0000 mgr.y (mgr.25175) 145 : cluster [DBG] pgmap v217: 121 pgs: 4 active+undersized+degraded, 12 active+undersized, 105 active+clean; 194 MiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 26 MiB/s rd, 5.6 MiB/s wr, 24 op/s; 9/303 objects degraded (2.970%); 26 MiB/s, 7 objects/s recovering 2024-04-07T16:22:50.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:49 smithi144 bash[66654]: cluster 2024-04-07T16:22:48.901477+0000 mon.a (mon.0) 459 : cluster [DBG] osdmap e419: 8 total, 8 up, 8 in 2024-04-07T16:22:50.366 INFO:tasks.rados.rados.0.smithi100.stdout:1883: writing smithi10066867-48 from 3801088 to 3817472 tid 1 2024-04-07T16:22:50.366 INFO:tasks.rados.rados.0.smithi100.stdout:1884: read oid 1 snap -1 2024-04-07T16:22:50.366 INFO:tasks.rados.rados.0.smithi100.stdout:1884: expect deleted 2024-04-07T16:22:50.367 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 15 current snap is 177 2024-04-07T16:22:50.367 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 44 v 850 (ObjNum 509 snap 152 seq_num 509) dirty exists 2024-04-07T16:22:50.367 INFO:tasks.rados.rados.0.smithi100.stdout:1882: finishing rollback tid 0 to smithi10066867-18 2024-04-07T16:22:50.367 INFO:tasks.rados.rados.0.smithi100.stdout:1882: finishing rollback tid 1 to smithi10066867-18 2024-04-07T16:22:50.367 INFO:tasks.rados.rados.0.smithi100.stdout:1885: seq_num 575 ranges {3506176=32768} 2024-04-07T16:22:50.398 INFO:tasks.rados.rados.0.smithi100.stdout:1885: writing smithi10066867-15 from 3506176 to 3538944 tid 1 2024-04-07T16:22:50.398 INFO:tasks.rados.rados.0.smithi100.stdout:1881: done (4 left) 2024-04-07T16:22:50.398 INFO:tasks.rados.rados.0.smithi100.stdout:1886: delete oid 30 current snap is 177 2024-04-07T16:22:50.399 INFO:tasks.rados.rados.0.smithi100.stdout:1882: finishing rollback tid 2 to smithi10066867-18 2024-04-07T16:22:50.399 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 18 v 1094 (ObjNum 509 snap 152 seq_num 509) dirty exists 2024-04-07T16:22:50.399 INFO:tasks.rados.rados.0.smithi100.stdout:1883: finishing write tid 1 to smithi10066867-48 2024-04-07T16:22:50.399 INFO:tasks.rados.rados.0.smithi100.stdout:1883: finishing write tid 2 to smithi10066867-48 2024-04-07T16:22:50.399 INFO:tasks.rados.rados.0.smithi100.stdout:1883: finishing write tid 3 to smithi10066867-48 2024-04-07T16:22:50.399 INFO:tasks.rados.rados.0.smithi100.stdout:1883: oid 48 updating version 0 to 1052 2024-04-07T16:22:50.399 INFO:tasks.rados.rados.0.smithi100.stdout:1883: oid 48 updating version 1052 to 1053 2024-04-07T16:22:50.399 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 48 v 1053 (ObjNum 574 snap 177 seq_num 574) dirty exists 2024-04-07T16:22:50.399 INFO:tasks.rados.rados.0.smithi100.stdout:1883: left oid 48 (ObjNum 574 snap 177 seq_num 574) 2024-04-07T16:22:50.403 INFO:tasks.rados.rados.0.smithi100.stdout:1882: done (4 left) 2024-04-07T16:22:50.403 INFO:tasks.rados.rados.0.smithi100.stdout:1883: done (3 left) 2024-04-07T16:22:50.403 INFO:tasks.rados.rados.0.smithi100.stdout:1884: done (2 left) 2024-04-07T16:22:50.403 INFO:tasks.rados.rados.0.smithi100.stdout:1886: done (1 left) 2024-04-07T16:22:50.403 INFO:tasks.rados.rados.0.smithi100.stdout:1887: snap_create 2024-04-07T16:22:50.404 INFO:tasks.rados.rados.0.smithi100.stdout:1885: finishing write tid 1 to smithi10066867-15 2024-04-07T16:22:50.404 INFO:tasks.rados.rados.0.smithi100.stdout:1885: finishing write tid 2 to smithi10066867-15 2024-04-07T16:22:50.407 INFO:tasks.rados.rados.0.smithi100.stdout:1885: finishing write tid 3 to smithi10066867-15 2024-04-07T16:22:50.407 INFO:tasks.rados.rados.0.smithi100.stdout:1885: oid 15 updating version 0 to 1096 2024-04-07T16:22:50.407 INFO:tasks.rados.rados.0.smithi100.stdout:1885: oid 15 updating version 1096 to 1097 2024-04-07T16:22:50.407 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 15 v 1097 (ObjNum 575 snap 177 seq_num 575) dirty exists 2024-04-07T16:22:50.407 INFO:tasks.rados.rados.0.smithi100.stdout:1885: left oid 15 (ObjNum 575 snap 177 seq_num 575) 2024-04-07T16:22:50.910 INFO:tasks.rados.rados.0.smithi100.stdout:1885: done (1 left) 2024-04-07T16:22:50.910 INFO:tasks.rados.rados.0.smithi100.stdout:1887: done (0 left) 2024-04-07T16:22:50.910 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 3 current snap is 178 2024-04-07T16:22:50.910 INFO:tasks.rados.rados.0.smithi100.stdout:1888: seq_num 577 ranges {2883584=65536} 2024-04-07T16:22:50.934 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:50 smithi100 bash[74942]: cluster 2024-04-07T16:22:49.911391+0000 mon.a (mon.0) 460 : cluster [DBG] osdmap e420: 8 total, 8 up, 8 in 2024-04-07T16:22:50.935 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:50 smithi100 bash[76202]: cluster 2024-04-07T16:22:49.911391+0000 mon.a (mon.0) 460 : cluster [DBG] osdmap e420: 8 total, 8 up, 8 in 2024-04-07T16:22:50.935 INFO:tasks.rados.rados.0.smithi100.stdout:1888: writing smithi10066867-3 from 2883584 to 2949120 tid 1 2024-04-07T16:22:50.935 INFO:tasks.rados.rados.0.smithi100.stdout:1889: rollback oid 11 current snap is 178 2024-04-07T16:22:50.936 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 11 to 177 2024-04-07T16:22:50.936 INFO:tasks.rados.rados.0.smithi100.stdout:1890: rollback oid 26 current snap is 178 2024-04-07T16:22:50.936 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 26 to 168 2024-04-07T16:22:50.936 INFO:tasks.rados.rados.0.smithi100.stdout:1891: rollback oid 19 current snap is 178 2024-04-07T16:22:50.936 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 19 to 177 2024-04-07T16:22:50.936 INFO:tasks.rados.rados.0.smithi100.stdout:1892: snap_create 2024-04-07T16:22:50.941 INFO:tasks.rados.rados.0.smithi100.stdout:1891: finishing rollback tid 1 to smithi10066867-19 2024-04-07T16:22:50.941 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 19 v 0 (ObjNum 33 snap 0 seq_num 2702958896) dirty dne 2024-04-07T16:22:50.950 INFO:tasks.rados.rados.0.smithi100.stdout:1888: finishing write tid 1 to smithi10066867-3 2024-04-07T16:22:50.950 INFO:tasks.rados.rados.0.smithi100.stdout:1888: finishing write tid 2 to smithi10066867-3 2024-04-07T16:22:50.952 INFO:tasks.rados.rados.0.smithi100.stdout:1889: finishing rollback tid 0 to smithi10066867-11 2024-04-07T16:22:50.953 INFO:tasks.rados.rados.0.smithi100.stdout:1888: finishing write tid 3 to smithi10066867-3 2024-04-07T16:22:50.953 INFO:tasks.rados.rados.0.smithi100.stdout:1888: oid 3 updating version 0 to 1080 2024-04-07T16:22:50.953 INFO:tasks.rados.rados.0.smithi100.stdout:1888: oid 3 version 1080 is already newer than 1079 2024-04-07T16:22:50.953 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 3 v 1080 (ObjNum 577 snap 178 seq_num 577) dirty exists 2024-04-07T16:22:50.953 INFO:tasks.rados.rados.0.smithi100.stdout:1888: left oid 3 (ObjNum 577 snap 178 seq_num 577) 2024-04-07T16:22:50.954 INFO:tasks.rados.rados.0.smithi100.stdout:1889: finishing rollback tid 1 to smithi10066867-11 2024-04-07T16:22:50.955 INFO:tasks.rados.rados.0.smithi100.stdout:1889: finishing rollback tid 2 to smithi10066867-11 2024-04-07T16:22:50.955 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 11 v 1101 (ObjNum 537 snap 162 seq_num 537) dirty exists 2024-04-07T16:22:50.957 INFO:tasks.rados.rados.0.smithi100.stdout:1890: finishing rollback tid 0 to smithi10066867-26 2024-04-07T16:22:50.957 INFO:tasks.rados.rados.0.smithi100.stdout:1890: finishing rollback tid 1 to smithi10066867-26 2024-04-07T16:22:50.957 INFO:tasks.rados.rados.0.smithi100.stdout:1890: finishing rollback tid 2 to smithi10066867-26 2024-04-07T16:22:50.957 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 26 v 780 (ObjNum 524 snap 156 seq_num 524) dirty exists 2024-04-07T16:22:51.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:50 smithi144 bash[66654]: cluster 2024-04-07T16:22:49.911391+0000 mon.a (mon.0) 460 : cluster [DBG] osdmap e420: 8 total, 8 up, 8 in 2024-04-07T16:22:51.790 INFO:tasks.rados.rados.0.smithi100.stdout:1888: done (4 left) 2024-04-07T16:22:51.791 INFO:tasks.rados.rados.0.smithi100.stdout:1889: done (3 left) 2024-04-07T16:22:51.791 INFO:tasks.rados.rados.0.smithi100.stdout:1890: done (2 left) 2024-04-07T16:22:51.791 INFO:tasks.rados.rados.0.smithi100.stdout:1891: done (1 left) 2024-04-07T16:22:51.791 INFO:tasks.rados.rados.0.smithi100.stdout:1892: done (0 left) 2024-04-07T16:22:51.791 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 38 current snap is 179 2024-04-07T16:22:51.791 INFO:tasks.rados.rados.0.smithi100.stdout:1893: seq_num 578 ranges {1196032=81920} 2024-04-07T16:22:51.799 INFO:tasks.rados.rados.0.smithi100.stdout:1893: writing smithi10066867-38 from 1196032 to 1277952 tid 1 2024-04-07T16:22:51.799 INFO:tasks.rados.rados.0.smithi100.stdout:1894: snap_create 2024-04-07T16:22:51.809 INFO:tasks.rados.rados.0.smithi100.stdout:1893: finishing write tid 1 to smithi10066867-38 2024-04-07T16:22:51.809 INFO:tasks.rados.rados.0.smithi100.stdout:1893: finishing write tid 2 to smithi10066867-38 2024-04-07T16:22:51.812 INFO:tasks.rados.rados.0.smithi100.stdout:1893: finishing write tid 3 to smithi10066867-38 2024-04-07T16:22:51.812 INFO:tasks.rados.rados.0.smithi100.stdout:1893: oid 38 updating version 0 to 1082 2024-04-07T16:22:51.812 INFO:tasks.rados.rados.0.smithi100.stdout:1893: oid 38 version 1082 is already newer than 1081 2024-04-07T16:22:51.812 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 38 v 1082 (ObjNum 578 snap 179 seq_num 578) dirty exists 2024-04-07T16:22:51.812 INFO:tasks.rados.rados.0.smithi100.stdout:1893: left oid 38 (ObjNum 578 snap 179 seq_num 578) 2024-04-07T16:22:52.007 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:51 smithi100 bash[74942]: cluster 2024-04-07T16:22:50.661591+0000 mgr.y (mgr.25175) 146 : cluster [DBG] pgmap v220: 121 pgs: 121 active+clean; 196 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 26 MiB/s rd, 6.1 MiB/s wr, 25 op/s; 30 MiB/s, 8 objects/s recovering 2024-04-07T16:22:52.007 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:51 smithi100 bash[74942]: cluster 2024-04-07T16:22:50.904011+0000 mon.a (mon.0) 461 : cluster [INF] Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 9/303 objects degraded (2.970%), 4 pgs degraded) 2024-04-07T16:22:52.007 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:51 smithi100 bash[74942]: cluster 2024-04-07T16:22:50.904089+0000 mon.a (mon.0) 462 : cluster [INF] Cluster is now healthy 2024-04-07T16:22:52.007 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:51 smithi100 bash[74942]: cluster 2024-04-07T16:22:50.927599+0000 mon.a (mon.0) 463 : cluster [DBG] osdmap e421: 8 total, 8 up, 8 in 2024-04-07T16:22:52.007 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:51 smithi100 bash[74942]: cluster 2024-04-07T16:22:51.808813+0000 mon.a (mon.0) 464 : cluster [DBG] osdmap e422: 8 total, 8 up, 8 in 2024-04-07T16:22:52.007 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:51 smithi100 bash[76202]: cluster 2024-04-07T16:22:50.661591+0000 mgr.y (mgr.25175) 146 : cluster [DBG] pgmap v220: 121 pgs: 121 active+clean; 196 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 26 MiB/s rd, 6.1 MiB/s wr, 25 op/s; 30 MiB/s, 8 objects/s recovering 2024-04-07T16:22:52.007 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:51 smithi100 bash[76202]: cluster 2024-04-07T16:22:50.904011+0000 mon.a (mon.0) 461 : cluster [INF] Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 9/303 objects degraded (2.970%), 4 pgs degraded) 2024-04-07T16:22:52.008 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:51 smithi100 bash[76202]: cluster 2024-04-07T16:22:50.904089+0000 mon.a (mon.0) 462 : cluster [INF] Cluster is now healthy 2024-04-07T16:22:52.008 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:51 smithi100 bash[76202]: cluster 2024-04-07T16:22:50.927599+0000 mon.a (mon.0) 463 : cluster [DBG] osdmap e421: 8 total, 8 up, 8 in 2024-04-07T16:22:52.008 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:51 smithi100 bash[76202]: cluster 2024-04-07T16:22:51.808813+0000 mon.a (mon.0) 464 : cluster [DBG] osdmap e422: 8 total, 8 up, 8 in 2024-04-07T16:22:52.305 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:51 smithi144 bash[66654]: cluster 2024-04-07T16:22:50.661591+0000 mgr.y (mgr.25175) 146 : cluster [DBG] pgmap v220: 121 pgs: 121 active+clean; 196 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 26 MiB/s rd, 6.1 MiB/s wr, 25 op/s; 30 MiB/s, 8 objects/s recovering 2024-04-07T16:22:52.305 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:51 smithi144 bash[66654]: cluster 2024-04-07T16:22:50.904011+0000 mon.a (mon.0) 461 : cluster [INF] Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 9/303 objects degraded (2.970%), 4 pgs degraded) 2024-04-07T16:22:52.305 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:51 smithi144 bash[66654]: cluster 2024-04-07T16:22:50.904089+0000 mon.a (mon.0) 462 : cluster [INF] Cluster is now healthy 2024-04-07T16:22:52.305 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:51 smithi144 bash[66654]: cluster 2024-04-07T16:22:50.927599+0000 mon.a (mon.0) 463 : cluster [DBG] osdmap e421: 8 total, 8 up, 8 in 2024-04-07T16:22:52.305 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:51 smithi144 bash[66654]: cluster 2024-04-07T16:22:51.808813+0000 mon.a (mon.0) 464 : cluster [DBG] osdmap e422: 8 total, 8 up, 8 in 2024-04-07T16:22:52.606 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:22:52 smithi144 bash[59794]: ts=2024-04-07T16:22:52.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:22:52.796 INFO:tasks.rados.rados.0.smithi100.stdout:1893: done (1 left) 2024-04-07T16:22:52.797 INFO:tasks.rados.rados.0.smithi100.stdout:1894: done (0 left) 2024-04-07T16:22:52.797 INFO:tasks.rados.rados.0.smithi100.stdout:1895: setattr oid 29 current snap is 180 2024-04-07T16:22:52.800 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 31 current snap is 180 2024-04-07T16:22:52.800 INFO:tasks.rados.rados.0.smithi100.stdout:1896: seq_num 579 ranges {3784704=98304} 2024-04-07T16:22:52.843 INFO:tasks.rados.rados.0.smithi100.stdout:1896: writing smithi10066867-31 from 3784704 to 3883008 tid 1 2024-04-07T16:22:52.843 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 46 current snap is 180 2024-04-07T16:22:52.843 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 29 v 1066 (ObjNum 559 snap 170 seq_num 559) dirty exists 2024-04-07T16:22:52.843 INFO:tasks.rados.rados.0.smithi100.stdout:1897: seq_num 580 ranges {0=114688} 2024-04-07T16:22:52.844 INFO:tasks.rados.rados.0.smithi100.stdout:1897: writing smithi10066867-46 from 0 to 114688 tid 1 2024-04-07T16:22:52.844 INFO:tasks.rados.rados.0.smithi100.stdout:1895: done (2 left) 2024-04-07T16:22:52.844 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 25 current snap is 180 2024-04-07T16:22:52.844 INFO:tasks.rados.rados.0.smithi100.stdout:1898: seq_num 581 ranges {2088960=139264} 2024-04-07T16:22:52.869 INFO:tasks.rados.rados.0.smithi100.stdout:1898: writing smithi10066867-25 from 2088960 to 2228224 tid 1 2024-04-07T16:22:52.869 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 2 current snap is 180 2024-04-07T16:22:52.869 INFO:tasks.rados.rados.0.smithi100.stdout:1899: seq_num 582 ranges {3547136=155648} 2024-04-07T16:22:52.902 INFO:tasks.rados.rados.0.smithi100.stdout:1899: writing smithi10066867-2 from 3547136 to 3702784 tid 1 2024-04-07T16:22:52.902 INFO:tasks.rados.rados.0.smithi100.stdout:1900: rmattr oid 12 current snap is 180 2024-04-07T16:22:52.902 INFO:tasks.rados.rados.0.smithi100.stdout:1896: finishing write tid 1 to smithi10066867-31 2024-04-07T16:22:52.902 INFO:tasks.rados.rados.0.smithi100.stdout:1896: finishing write tid 2 to smithi10066867-31 2024-04-07T16:22:52.902 INFO:tasks.rados.rados.0.smithi100.stdout:1897: finishing write tid 1 to smithi10066867-46 2024-04-07T16:22:52.902 INFO:tasks.rados.rados.0.smithi100.stdout:1896: finishing write tid 3 to smithi10066867-31 2024-04-07T16:22:52.902 INFO:tasks.rados.rados.0.smithi100.stdout:1896: oid 31 updating version 0 to 1055 2024-04-07T16:22:52.902 INFO:tasks.rados.rados.0.smithi100.stdout:1896: oid 31 version 1055 is already newer than 1054 2024-04-07T16:22:52.903 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 31 v 1055 (ObjNum 579 snap 180 seq_num 579) dirty exists 2024-04-07T16:22:52.903 INFO:tasks.rados.rados.0.smithi100.stdout:1896: left oid 31 (ObjNum 579 snap 180 seq_num 579) 2024-04-07T16:22:52.903 INFO:tasks.rados.rados.0.smithi100.stdout:1897: finishing write tid 2 to smithi10066867-46 2024-04-07T16:22:52.903 INFO:tasks.rados.rados.0.smithi100.stdout:1897: finishing write tid 3 to smithi10066867-46 2024-04-07T16:22:52.903 INFO:tasks.rados.rados.0.smithi100.stdout:1897: oid 46 updating version 0 to 746 2024-04-07T16:22:52.903 INFO:tasks.rados.rados.0.smithi100.stdout:1897: oid 46 updating version 746 to 747 2024-04-07T16:22:52.903 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 46 v 747 (ObjNum 580 snap 180 seq_num 580) dirty exists 2024-04-07T16:22:52.903 INFO:tasks.rados.rados.0.smithi100.stdout:1897: left oid 46 (ObjNum 580 snap 180 seq_num 580) 2024-04-07T16:22:52.903 INFO:tasks.rados.rados.0.smithi100.stdout:1898: finishing write tid 1 to smithi10066867-25 2024-04-07T16:22:52.903 INFO:tasks.rados.rados.0.smithi100.stdout:1898: finishing write tid 2 to smithi10066867-25 2024-04-07T16:22:52.903 INFO:tasks.rados.rados.0.smithi100.stdout:1898: finishing write tid 3 to smithi10066867-25 2024-04-07T16:22:52.903 INFO:tasks.rados.rados.0.smithi100.stdout:1898: oid 25 updating version 0 to 781 2024-04-07T16:22:52.903 INFO:tasks.rados.rados.0.smithi100.stdout:1898: oid 25 updating version 781 to 782 2024-04-07T16:22:52.903 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 25 v 782 (ObjNum 581 snap 180 seq_num 581) dirty exists 2024-04-07T16:22:52.903 INFO:tasks.rados.rados.0.smithi100.stdout:1898: left oid 25 (ObjNum 581 snap 180 seq_num 581) 2024-04-07T16:22:52.904 INFO:tasks.rados.rados.0.smithi100.stdout:1896: done (4 left) 2024-04-07T16:22:52.904 INFO:tasks.rados.rados.0.smithi100.stdout:1897: done (3 left) 2024-04-07T16:22:52.904 INFO:tasks.rados.rados.0.smithi100.stdout:1898: done (2 left) 2024-04-07T16:22:52.904 INFO:tasks.rados.rados.0.smithi100.stdout:1901: setattr oid 4 current snap is 180 2024-04-07T16:22:52.904 INFO:tasks.rados.rados.0.smithi100.stdout:1902: rollback oid 40 current snap is 180 2024-04-07T16:22:52.904 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 40 to 179 2024-04-07T16:22:52.905 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 8 current snap is 180 2024-04-07T16:22:52.905 INFO:tasks.rados.rados.0.smithi100.stdout:1903: seq_num 583 ranges {9248768=172032} 2024-04-07T16:22:52.964 INFO:tasks.rados.rados.0.smithi100.stdout:1903: writing smithi10066867-8 from 9248768 to 9420800 tid 1 2024-04-07T16:22:52.964 INFO:tasks.rados.rados.0.smithi100.stdout:1904: rollback oid 29 current snap is 180 2024-04-07T16:22:52.964 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 29 to 177 2024-04-07T16:22:52.964 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 12 v 1026 (ObjNum 553 snap 168 seq_num 553) dirty exists 2024-04-07T16:22:52.965 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 4 v 1103 (ObjNum 521 snap 156 seq_num 521) dirty exists 2024-04-07T16:22:52.965 INFO:tasks.rados.rados.0.smithi100.stdout:1899: finishing write tid 1 to smithi10066867-2 2024-04-07T16:22:52.965 INFO:tasks.rados.rados.0.smithi100.stdout:1899: finishing write tid 2 to smithi10066867-2 2024-04-07T16:22:52.965 INFO:tasks.rados.rados.0.smithi100.stdout:1902: finishing rollback tid 0 to smithi10066867-40 2024-04-07T16:22:52.965 INFO:tasks.rados.rados.0.smithi100.stdout:1902: finishing rollback tid 1 to smithi10066867-40 2024-04-07T16:22:52.965 INFO:tasks.rados.rados.0.smithi100.stdout:1902: finishing rollback tid 2 to smithi10066867-40 2024-04-07T16:22:52.965 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 40 v 857 (ObjNum 540 snap 164 seq_num 540) dirty exists 2024-04-07T16:22:52.965 INFO:tasks.rados.rados.0.smithi100.stdout:1899: finishing write tid 3 to smithi10066867-2 2024-04-07T16:22:52.965 INFO:tasks.rados.rados.0.smithi100.stdout:1899: oid 2 updating version 0 to 853 2024-04-07T16:22:52.965 INFO:tasks.rados.rados.0.smithi100.stdout:1899: oid 2 version 853 is already newer than 852 2024-04-07T16:22:52.965 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 2 v 853 (ObjNum 582 snap 180 seq_num 582) dirty exists 2024-04-07T16:22:52.965 INFO:tasks.rados.rados.0.smithi100.stdout:1899: left oid 2 (ObjNum 582 snap 180 seq_num 582) 2024-04-07T16:22:52.965 INFO:tasks.rados.rados.0.smithi100.stdout:1899: done (5 left) 2024-04-07T16:22:52.965 INFO:tasks.rados.rados.0.smithi100.stdout:1900: done (4 left) 2024-04-07T16:22:52.965 INFO:tasks.rados.rados.0.smithi100.stdout:1901: done (3 left) 2024-04-07T16:22:52.966 INFO:tasks.rados.rados.0.smithi100.stdout:1902: done (2 left) 2024-04-07T16:22:52.966 INFO:tasks.rados.rados.0.smithi100.stdout:1905: copy_from oid 35 from oid 50 current snap is 180 2024-04-07T16:22:52.966 INFO:tasks.rados.rados.0.smithi100.stdout:1906: delete oid 21 current snap is 180 2024-04-07T16:22:52.971 INFO:tasks.rados.rados.0.smithi100.stdout:1906: done (3 left) 2024-04-07T16:22:52.971 INFO:tasks.rados.rados.0.smithi100.stdout:1907: rmattr oid 26 current snap is 180 2024-04-07T16:22:52.971 INFO:tasks.rados.rados.0.smithi100.stdout:1908: copy_from oid 17 from oid 9 current snap is 180 2024-04-07T16:22:52.971 INFO:tasks.rados.rados.0.smithi100.stdout:1909: setattr oid 3 current snap is 180 2024-04-07T16:22:52.973 INFO:tasks.rados.rados.0.smithi100.stdout:1904: finishing rollback tid 0 to smithi10066867-29 2024-04-07T16:22:52.973 INFO:tasks.rados.rados.0.smithi100.stdout:1910: snap_create 2024-04-07T16:22:52.976 INFO:tasks.rados.rados.0.smithi100.stdout:1904: finishing rollback tid 1 to smithi10066867-29 2024-04-07T16:22:52.976 INFO:tasks.rados.rados.0.smithi100.stdout:1904: finishing rollback tid 2 to smithi10066867-29 2024-04-07T16:22:52.976 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 29 v 1069 (ObjNum 559 snap 170 seq_num 559) dirty exists 2024-04-07T16:22:52.976 INFO:tasks.rados.rados.0.smithi100.stdout:1903: finishing write tid 1 to smithi10066867-8 2024-04-07T16:22:52.977 INFO:tasks.rados.rados.0.smithi100.stdout:1903: finishing write tid 2 to smithi10066867-8 2024-04-07T16:22:52.977 INFO:tasks.rados.rados.0.smithi100.stdout:1908: finishing copy_from to smithi10066867-17 2024-04-07T16:22:52.977 INFO:tasks.rados.rados.0.smithi100.stdout:1908: got expected ENOENT (src dne) 2024-04-07T16:22:52.978 INFO:tasks.rados.rados.0.smithi100.stdout:1908: finishing copy_from racing read to smithi10066867-17 2024-04-07T16:22:52.982 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 26 v 783 (ObjNum 524 snap 156 seq_num 524) dirty exists 2024-04-07T16:22:52.982 INFO:tasks.rados.rados.0.smithi100.stdout:1903: finishing write tid 3 to smithi10066867-8 2024-04-07T16:22:52.982 INFO:tasks.rados.rados.0.smithi100.stdout:1903: oid 8 updating version 0 to 748 2024-04-07T16:22:52.982 INFO:tasks.rados.rados.0.smithi100.stdout:1903: oid 8 updating version 748 to 749 2024-04-07T16:22:52.982 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 8 v 749 (ObjNum 583 snap 180 seq_num 583) dirty exists 2024-04-07T16:22:52.982 INFO:tasks.rados.rados.0.smithi100.stdout:1903: left oid 8 (ObjNum 583 snap 180 seq_num 583) 2024-04-07T16:22:52.983 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 3 v 1083 (ObjNum 577 snap 178 seq_num 577) dirty exists 2024-04-07T16:22:52.992 INFO:tasks.rados.rados.0.smithi100.stdout:1905: finishing copy_from racing read to smithi10066867-35 2024-04-07T16:22:52.995 INFO:tasks.rados.rados.0.smithi100.stdout:1905: finishing copy_from to smithi10066867-35 2024-04-07T16:22:52.995 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 35 v 998 (ObjNum 510 snap 152 seq_num 510) dirty exists 2024-04-07T16:22:53.791 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:22:53 smithi144 bash[59794]: ts=2024-04-07T16:22:53.507Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=nodes name=CephNodeDiskspaceWarning index=4 msg="Evaluating rule failed" rule="alert: CephNodeDiskspaceWarning\nexpr: predict_linear(node_filesystem_free_bytes{device=~\"/.*\"}[2d], 3600 * 24 * 5)\n * on (instance) group_left (nodename) node_uname_info < 0\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.8.4\n severity: warning\n type: ceph_default\nannotations:\n description: Mountpoint {{ $labels.mountpoint }} on {{ $labels.nodename }} will\n be full in less than 5 days based on the 48 hour trailing fill rate.\n summary: Host filesystem free space is getting low\n" err="found duplicate series for the match group {instance=\"smithi100\"} on the right hand-side of the operation: [{__name__=\"node_uname_info\", domainname=\"(none)\", instance=\"smithi100\", job=\"node\", machine=\"x86_64\", nodename=\"smithi100\", release=\"5.15.0-101-generic\", sysname=\"Linux\", version=\"#111-Ubuntu SMP Tue Mar 5 20:16:58 UTC 2024\"}, {__name__=\"node_uname_info\", cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", domainname=\"(none)\", instance=\"smithi100\", job=\"node\", machine=\"x86_64\", nodename=\"smithi100\", release=\"5.15.0-101-generic\", sysname=\"Linux\", version=\"#111-Ubuntu SMP Tue Mar 5 20:16:58 UTC 2024\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:22:53.910 INFO:tasks.rados.rados.0.smithi100.stdout:1903: done (6 left) 2024-04-07T16:22:53.910 INFO:tasks.rados.rados.0.smithi100.stdout:1904: done (5 left) 2024-04-07T16:22:53.910 INFO:tasks.rados.rados.0.smithi100.stdout:1905: done (4 left) 2024-04-07T16:22:53.911 INFO:tasks.rados.rados.0.smithi100.stdout:1907: done (3 left) 2024-04-07T16:22:53.911 INFO:tasks.rados.rados.0.smithi100.stdout:1908: done (2 left) 2024-04-07T16:22:53.911 INFO:tasks.rados.rados.0.smithi100.stdout:1909: done (1 left) 2024-04-07T16:22:53.911 INFO:tasks.rados.rados.0.smithi100.stdout:1910: done (0 left) 2024-04-07T16:22:53.911 INFO:tasks.rados.rados.0.smithi100.stdout:1911: delete oid 24 current snap is 181 2024-04-07T16:22:53.914 INFO:tasks.rados.rados.0.smithi100.stdout:1911: done (0 left) 2024-04-07T16:22:53.914 INFO:tasks.rados.rados.0.smithi100.stdout:1912: rmattr oid 14 current snap is 181 2024-04-07T16:22:53.914 INFO:tasks.rados.rados.0.smithi100.stdout:1912: done (0 left) 2024-04-07T16:22:53.914 INFO:tasks.rados.rados.0.smithi100.stdout:1913: delete oid 15 current snap is 181 2024-04-07T16:22:53.921 INFO:tasks.rados.rados.0.smithi100.stdout:1913: done (0 left) 2024-04-07T16:22:53.921 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 4 current snap is 181 2024-04-07T16:22:53.922 INFO:tasks.rados.rados.0.smithi100.stdout:1914: seq_num 587 ranges {4521984=237568} 2024-04-07T16:22:53.949 INFO:tasks.rados.rados.0.smithi100.stdout:1914: writing smithi10066867-4 from 4521984 to 4759552 tid 1 2024-04-07T16:22:53.949 INFO:tasks.rados.rados.0.smithi100.stdout:1915: snap_create 2024-04-07T16:22:53.953 INFO:tasks.rados.rados.0.smithi100.stdout:1914: finishing write tid 1 to smithi10066867-4 2024-04-07T16:22:53.954 INFO:tasks.rados.rados.0.smithi100.stdout:1914: finishing write tid 2 to smithi10066867-4 2024-04-07T16:22:53.956 INFO:tasks.rados.rados.0.smithi100.stdout:1914: finishing write tid 3 to smithi10066867-4 2024-04-07T16:22:53.956 INFO:tasks.rados.rados.0.smithi100.stdout:1914: oid 4 updating version 0 to 1107 2024-04-07T16:22:53.956 INFO:tasks.rados.rados.0.smithi100.stdout:1914: oid 4 updating version 1107 to 1108 2024-04-07T16:22:53.956 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 4 v 1108 (ObjNum 587 snap 181 seq_num 587) dirty exists 2024-04-07T16:22:53.956 INFO:tasks.rados.rados.0.smithi100.stdout:1914: left oid 4 (ObjNum 587 snap 181 seq_num 587) 2024-04-07T16:22:54.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:53 smithi144 bash[66654]: cluster 2024-04-07T16:22:52.662784+0000 mgr.y (mgr.25175) 147 : cluster [DBG] pgmap v223: 121 pgs: 121 active+clean; 209 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 5.6 MiB/s rd, 2.9 MiB/s wr, 21 op/s; 3.8 MiB/s, 0 objects/s recovering 2024-04-07T16:22:54.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:53 smithi144 bash[66654]: cluster 2024-04-07T16:22:52.813521+0000 mon.a (mon.0) 465 : cluster [DBG] osdmap e423: 8 total, 8 up, 8 in 2024-04-07T16:22:54.184 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:53 smithi100 bash[74942]: cluster 2024-04-07T16:22:52.662784+0000 mgr.y (mgr.25175) 147 : cluster [DBG] pgmap v223: 121 pgs: 121 active+clean; 209 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 5.6 MiB/s rd, 2.9 MiB/s wr, 21 op/s; 3.8 MiB/s, 0 objects/s recovering 2024-04-07T16:22:54.184 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:53 smithi100 bash[74942]: cluster 2024-04-07T16:22:52.813521+0000 mon.a (mon.0) 465 : cluster [DBG] osdmap e423: 8 total, 8 up, 8 in 2024-04-07T16:22:54.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:53 smithi100 bash[76202]: cluster 2024-04-07T16:22:52.662784+0000 mgr.y (mgr.25175) 147 : cluster [DBG] pgmap v223: 121 pgs: 121 active+clean; 209 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 5.6 MiB/s rd, 2.9 MiB/s wr, 21 op/s; 3.8 MiB/s, 0 objects/s recovering 2024-04-07T16:22:54.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:53 smithi100 bash[76202]: cluster 2024-04-07T16:22:52.813521+0000 mon.a (mon.0) 465 : cluster [DBG] osdmap e423: 8 total, 8 up, 8 in 2024-04-07T16:22:54.922 INFO:tasks.rados.rados.0.smithi100.stdout:1914: done (1 left) 2024-04-07T16:22:54.923 INFO:tasks.rados.rados.0.smithi100.stdout:1915: done (0 left) 2024-04-07T16:22:54.923 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 45 current snap is 182 2024-04-07T16:22:54.923 INFO:tasks.rados.rados.0.smithi100.stdout:1916: seq_num 588 ranges {2138112=253952} 2024-04-07T16:22:54.947 INFO:tasks.rados.rados.0.smithi100.stdout:1916: writing smithi10066867-45 from 2138112 to 2392064 tid 1 2024-04-07T16:22:54.947 INFO:tasks.rados.rados.0.smithi100.stdout:1917: snap_remove snap 179 2024-04-07T16:22:55.184 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:54 smithi100 bash[74942]: cluster 2024-04-07T16:22:53.928803+0000 mon.a (mon.0) 466 : cluster [DBG] osdmap e424: 8 total, 8 up, 8 in 2024-04-07T16:22:55.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:54 smithi100 bash[76202]: cluster 2024-04-07T16:22:53.928803+0000 mon.a (mon.0) 466 : cluster [DBG] osdmap e424: 8 total, 8 up, 8 in 2024-04-07T16:22:55.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:54 smithi144 bash[66654]: cluster 2024-04-07T16:22:53.928803+0000 mon.a (mon.0) 466 : cluster [DBG] osdmap e424: 8 total, 8 up, 8 in 2024-04-07T16:22:55.936 INFO:tasks.rados.rados.0.smithi100.stdout:1917: done (1 left) 2024-04-07T16:22:55.936 INFO:tasks.rados.rados.0.smithi100.stdout:1918: copy_from oid 25 from oid 40 current snap is 182 2024-04-07T16:22:55.936 INFO:tasks.rados.rados.0.smithi100.stdout:1916: finishing write tid 1 to smithi10066867-45 2024-04-07T16:22:55.936 INFO:tasks.rados.rados.0.smithi100.stdout:1916: finishing write tid 2 to smithi10066867-45 2024-04-07T16:22:55.936 INFO:tasks.rados.rados.0.smithi100.stdout:1916: finishing write tid 3 to smithi10066867-45 2024-04-07T16:22:55.937 INFO:tasks.rados.rados.0.smithi100.stdout:1916: oid 45 updating version 0 to 1004 2024-04-07T16:22:55.937 INFO:tasks.rados.rados.0.smithi100.stdout:1916: oid 45 updating version 1004 to 1005 2024-04-07T16:22:55.937 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 45 v 1005 (ObjNum 588 snap 182 seq_num 588) dirty exists 2024-04-07T16:22:55.937 INFO:tasks.rados.rados.0.smithi100.stdout:1916: left oid 45 (ObjNum 588 snap 182 seq_num 588) 2024-04-07T16:22:55.937 INFO:tasks.rados.rados.0.smithi100.stdout:1916: done (1 left) 2024-04-07T16:22:55.937 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 32 current snap is 182 2024-04-07T16:22:55.937 INFO:tasks.rados.rados.0.smithi100.stdout:1919: seq_num 589 ranges {4104192=270336} 2024-04-07T16:22:55.986 INFO:tasks.rados.rados.0.smithi100.stdout:1919: writing smithi10066867-32 from 4104192 to 4374528 tid 1 2024-04-07T16:22:55.986 INFO:tasks.rados.rados.0.smithi100.stdout:1920: snap_remove snap 176 2024-04-07T16:22:56.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:55 smithi100 bash[74942]: cluster 2024-04-07T16:22:54.663553+0000 mgr.y (mgr.25175) 148 : cluster [DBG] pgmap v226: 121 pgs: 121 active+clean; 209 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 5.6 MiB/s rd, 2.4 MiB/s wr, 29 op/s 2024-04-07T16:22:56.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:55 smithi100 bash[74942]: cluster 2024-04-07T16:22:54.940275+0000 mon.a (mon.0) 467 : cluster [DBG] osdmap e425: 8 total, 8 up, 8 in 2024-04-07T16:22:56.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:55 smithi100 bash[74942]: audit 2024-04-07T16:22:55.240391+0000 mon.a (mon.0) 468 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:22:56.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:55 smithi100 bash[74942]: audit 2024-04-07T16:22:55.249007+0000 mon.a (mon.0) 469 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:22:56.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:55 smithi100 bash[74942]: audit 2024-04-07T16:22:55.250789+0000 mon.a (mon.0) 470 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:22:56.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:55 smithi100 bash[74942]: audit 2024-04-07T16:22:55.252798+0000 mon.a (mon.0) 471 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:22:56.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:55 smithi100 bash[74942]: audit 2024-04-07T16:22:55.262515+0000 mon.a (mon.0) 472 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:22:56.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:55 smithi100 bash[74942]: audit 2024-04-07T16:22:55.358061+0000 mon.a (mon.0) 473 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:22:56.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:55 smithi100 bash[74942]: audit 2024-04-07T16:22:55.363471+0000 mon.a (mon.0) 474 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:22:56.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:55 smithi100 bash[74942]: audit 2024-04-07T16:22:55.367714+0000 mon.a (mon.0) 475 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:22:56.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:55 smithi100 bash[74942]: audit 2024-04-07T16:22:55.370632+0000 mon.a (mon.0) 476 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:22:56.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:55 smithi100 bash[74942]: audit 2024-04-07T16:22:55.374472+0000 mon.a (mon.0) 477 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-04-07T16:22:56.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:55 smithi100 bash[76202]: cluster 2024-04-07T16:22:54.663553+0000 mgr.y (mgr.25175) 148 : cluster [DBG] pgmap v226: 121 pgs: 121 active+clean; 209 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 5.6 MiB/s rd, 2.4 MiB/s wr, 29 op/s 2024-04-07T16:22:56.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:55 smithi100 bash[76202]: cluster 2024-04-07T16:22:54.940275+0000 mon.a (mon.0) 467 : cluster [DBG] osdmap e425: 8 total, 8 up, 8 in 2024-04-07T16:22:56.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:55 smithi100 bash[76202]: audit 2024-04-07T16:22:55.240391+0000 mon.a (mon.0) 468 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:22:56.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:55 smithi100 bash[76202]: audit 2024-04-07T16:22:55.249007+0000 mon.a (mon.0) 469 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:22:56.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:55 smithi100 bash[76202]: audit 2024-04-07T16:22:55.250789+0000 mon.a (mon.0) 470 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:22:56.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:55 smithi100 bash[76202]: audit 2024-04-07T16:22:55.252798+0000 mon.a (mon.0) 471 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:22:56.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:55 smithi100 bash[76202]: audit 2024-04-07T16:22:55.262515+0000 mon.a (mon.0) 472 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:22:56.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:55 smithi100 bash[76202]: audit 2024-04-07T16:22:55.358061+0000 mon.a (mon.0) 473 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:22:56.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:55 smithi100 bash[76202]: audit 2024-04-07T16:22:55.363471+0000 mon.a (mon.0) 474 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:22:56.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:55 smithi100 bash[76202]: audit 2024-04-07T16:22:55.367714+0000 mon.a (mon.0) 475 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:22:56.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:55 smithi100 bash[76202]: audit 2024-04-07T16:22:55.370632+0000 mon.a (mon.0) 476 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:22:56.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:55 smithi100 bash[76202]: audit 2024-04-07T16:22:55.374472+0000 mon.a (mon.0) 477 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-04-07T16:22:56.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:55 smithi144 bash[66654]: cluster 2024-04-07T16:22:54.663553+0000 mgr.y (mgr.25175) 148 : cluster [DBG] pgmap v226: 121 pgs: 121 active+clean; 209 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 5.6 MiB/s rd, 2.4 MiB/s wr, 29 op/s 2024-04-07T16:22:56.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:55 smithi144 bash[66654]: cluster 2024-04-07T16:22:54.940275+0000 mon.a (mon.0) 467 : cluster [DBG] osdmap e425: 8 total, 8 up, 8 in 2024-04-07T16:22:56.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:55 smithi144 bash[66654]: audit 2024-04-07T16:22:55.240391+0000 mon.a (mon.0) 468 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:22:56.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:55 smithi144 bash[66654]: audit 2024-04-07T16:22:55.249007+0000 mon.a (mon.0) 469 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:22:56.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:55 smithi144 bash[66654]: audit 2024-04-07T16:22:55.250789+0000 mon.a (mon.0) 470 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:22:56.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:55 smithi144 bash[66654]: audit 2024-04-07T16:22:55.252798+0000 mon.a (mon.0) 471 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:22:56.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:55 smithi144 bash[66654]: audit 2024-04-07T16:22:55.262515+0000 mon.a (mon.0) 472 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:22:56.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:55 smithi144 bash[66654]: audit 2024-04-07T16:22:55.358061+0000 mon.a (mon.0) 473 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:22:56.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:55 smithi144 bash[66654]: audit 2024-04-07T16:22:55.363471+0000 mon.a (mon.0) 474 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:22:56.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:55 smithi144 bash[66654]: audit 2024-04-07T16:22:55.367714+0000 mon.a (mon.0) 475 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:22:56.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:55 smithi144 bash[66654]: audit 2024-04-07T16:22:55.370632+0000 mon.a (mon.0) 476 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:22:56.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:55 smithi144 bash[66654]: audit 2024-04-07T16:22:55.374472+0000 mon.a (mon.0) 477 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-04-07T16:22:56.940 INFO:tasks.rados.rados.0.smithi100.stdout:1920: done (2 left) 2024-04-07T16:22:56.940 INFO:tasks.rados.rados.0.smithi100.stdout:1921: delete oid 19 current snap is 182 2024-04-07T16:22:56.940 INFO:tasks.rados.rados.0.smithi100.stdout:1919: finishing write tid 1 to smithi10066867-32 2024-04-07T16:22:56.940 INFO:tasks.rados.rados.0.smithi100.stdout:1918: finishing copy_from racing read to smithi10066867-25 2024-04-07T16:22:56.940 INFO:tasks.rados.rados.0.smithi100.stdout:1919: finishing write tid 2 to smithi10066867-32 2024-04-07T16:22:56.940 INFO:tasks.rados.rados.0.smithi100.stdout:1919: finishing write tid 3 to smithi10066867-32 2024-04-07T16:22:56.940 INFO:tasks.rados.rados.0.smithi100.stdout:1919: oid 32 updating version 0 to 771 2024-04-07T16:22:56.941 INFO:tasks.rados.rados.0.smithi100.stdout:1919: oid 32 version 771 is already newer than 770 2024-04-07T16:22:56.941 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 32 v 771 (ObjNum 589 snap 182 seq_num 589) dirty exists 2024-04-07T16:22:56.941 INFO:tasks.rados.rados.0.smithi100.stdout:1919: left oid 32 (ObjNum 589 snap 182 seq_num 589) 2024-04-07T16:22:56.941 INFO:tasks.rados.rados.0.smithi100.stdout:1918: finishing copy_from to smithi10066867-25 2024-04-07T16:22:56.941 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 25 v 857 (ObjNum 540 snap 164 seq_num 540) dirty exists 2024-04-07T16:22:56.963 INFO:tasks.rados.rados.0.smithi100.stdout:1918: done (2 left) 2024-04-07T16:22:56.964 INFO:tasks.rados.rados.0.smithi100.stdout:1919: done (1 left) 2024-04-07T16:22:56.964 INFO:tasks.rados.rados.0.smithi100.stdout:1921: done (0 left) 2024-04-07T16:22:56.964 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 23 current snap is 182 2024-04-07T16:22:56.964 INFO:tasks.rados.rados.0.smithi100.stdout:1922: seq_num 591 ranges {5365760=303104} 2024-04-07T16:22:56.986 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:56 smithi100 bash[74942]: audit 2024-04-07T16:22:55.374831+0000 mgr.y (mgr.25175) 149 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-04-07T16:22:56.986 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:56 smithi100 bash[74942]: cephadm 2024-04-07T16:22:55.377076+0000 mgr.y (mgr.25175) 150 : cephadm [INF] Upgrade: osd.2 is safe to restart 2024-04-07T16:22:56.986 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:56 smithi100 bash[74942]: cluster 2024-04-07T16:22:55.953567+0000 mon.a (mon.0) 478 : cluster [DBG] osdmap e426: 8 total, 8 up, 8 in 2024-04-07T16:22:56.986 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:56 smithi100 bash[74942]: audit 2024-04-07T16:22:56.790714+0000 mon.a (mon.0) 479 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:22:56.986 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:56 smithi100 bash[76202]: audit 2024-04-07T16:22:55.374831+0000 mgr.y (mgr.25175) 149 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-04-07T16:22:56.986 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:56 smithi100 bash[76202]: cephadm 2024-04-07T16:22:55.377076+0000 mgr.y (mgr.25175) 150 : cephadm [INF] Upgrade: osd.2 is safe to restart 2024-04-07T16:22:56.986 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:56 smithi100 bash[76202]: cluster 2024-04-07T16:22:55.953567+0000 mon.a (mon.0) 478 : cluster [DBG] osdmap e426: 8 total, 8 up, 8 in 2024-04-07T16:22:56.986 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:56 smithi100 bash[76202]: audit 2024-04-07T16:22:56.790714+0000 mon.a (mon.0) 479 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:22:56.996 INFO:tasks.rados.rados.0.smithi100.stdout:1922: writing smithi10066867-23 from 5365760 to 5668864 tid 1 2024-04-07T16:22:56.996 INFO:tasks.rados.rados.0.smithi100.stdout:1923: snap_create 2024-04-07T16:22:57.000 INFO:tasks.rados.rados.0.smithi100.stdout:1922: finishing write tid 1 to smithi10066867-23 2024-04-07T16:22:57.000 INFO:tasks.rados.rados.0.smithi100.stdout:1922: finishing write tid 2 to smithi10066867-23 2024-04-07T16:22:57.002 INFO:tasks.rados.rados.0.smithi100.stdout:1922: finishing write tid 3 to smithi10066867-23 2024-04-07T16:22:57.002 INFO:tasks.rados.rados.0.smithi100.stdout:1922: oid 23 updating version 0 to 1125 2024-04-07T16:22:57.002 INFO:tasks.rados.rados.0.smithi100.stdout:1922: oid 23 version 1125 is already newer than 1124 2024-04-07T16:22:57.002 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 23 v 1125 (ObjNum 591 snap 182 seq_num 591) dirty exists 2024-04-07T16:22:57.002 INFO:tasks.rados.rados.0.smithi100.stdout:1922: left oid 23 (ObjNum 591 snap 182 seq_num 591) 2024-04-07T16:22:57.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:56 smithi144 bash[66654]: audit 2024-04-07T16:22:55.374831+0000 mgr.y (mgr.25175) 149 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-04-07T16:22:57.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:56 smithi144 bash[66654]: cephadm 2024-04-07T16:22:55.377076+0000 mgr.y (mgr.25175) 150 : cephadm [INF] Upgrade: osd.2 is safe to restart 2024-04-07T16:22:57.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:56 smithi144 bash[66654]: cluster 2024-04-07T16:22:55.953567+0000 mon.a (mon.0) 478 : cluster [DBG] osdmap e426: 8 total, 8 up, 8 in 2024-04-07T16:22:57.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:56 smithi144 bash[66654]: audit 2024-04-07T16:22:56.790714+0000 mon.a (mon.0) 479 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:22:57.956 INFO:tasks.rados.rados.0.smithi100.stdout:1922: done (1 left) 2024-04-07T16:22:57.957 INFO:tasks.rados.rados.0.smithi100.stdout:1923: done (0 left) 2024-04-07T16:22:57.957 INFO:tasks.rados.rados.0.smithi100.stdout:1924: read oid 33 snap -1 2024-04-07T16:22:57.957 INFO:tasks.rados.rados.0.smithi100.stdout:1924: expect deleted 2024-04-07T16:22:57.957 INFO:tasks.rados.rados.0.smithi100.stdout:1925: read oid 6 snap -1 2024-04-07T16:22:57.957 INFO:tasks.rados.rados.0.smithi100.stdout:1925: expect (ObjNum 524 snap 156 seq_num 524) 2024-04-07T16:22:57.957 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 12 current snap is 183 2024-04-07T16:22:57.957 INFO:tasks.rados.rados.0.smithi100.stdout:1926: seq_num 592 ranges {4374528=319488} 2024-04-07T16:22:57.984 INFO:tasks.rados.rados.0.smithi100.stdout:1926: writing smithi10066867-12 from 4374528 to 4694016 tid 1 2024-04-07T16:22:57.985 INFO:tasks.rados.rados.0.smithi100.stdout:1927: rmattr oid 40 current snap is 183 2024-04-07T16:22:57.985 INFO:tasks.rados.rados.0.smithi100.stdout:1927: done (3 left) 2024-04-07T16:22:57.985 INFO:tasks.rados.rados.0.smithi100.stdout:1928: copy_from oid 2 from oid 36 current snap is 183 2024-04-07T16:22:57.985 INFO:tasks.rados.rados.0.smithi100.stdout:1924: done (3 left) 2024-04-07T16:22:57.985 INFO:tasks.rados.rados.0.smithi100.stdout:1929: rollback oid 30 current snap is 183 2024-04-07T16:22:57.985 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 30 to 174 2024-04-07T16:22:57.985 INFO:tasks.rados.rados.0.smithi100.stdout:1930: read oid 13 snap -1 2024-04-07T16:22:57.985 INFO:tasks.rados.rados.0.smithi100.stdout:1930: expect (ObjNum 570 snap 177 seq_num 570) 2024-04-07T16:22:57.985 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 21 current snap is 183 2024-04-07T16:22:57.985 INFO:tasks.rados.rados.0.smithi100.stdout:1931: seq_num 593 ranges {0=335872} 2024-04-07T16:22:57.988 INFO:tasks.rados.rados.0.smithi100.stdout:1931: writing smithi10066867-21 from 0 to 335872 tid 1 2024-04-07T16:22:57.989 INFO:tasks.rados.rados.0.smithi100.stdout:1932: delete oid 4 current snap is 183 2024-04-07T16:22:57.996 INFO:tasks.rados.rados.0.smithi100.stdout:1931: finishing write tid 1 to smithi10066867-21 2024-04-07T16:22:57.997 INFO:tasks.rados.rados.0.smithi100.stdout:1931: finishing write tid 2 to smithi10066867-21 2024-04-07T16:22:58.000 INFO:tasks.rados.rados.0.smithi100.stdout:1931: finishing write tid 3 to smithi10066867-21 2024-04-07T16:22:58.000 INFO:tasks.rados.rados.0.smithi100.stdout:1931: oid 21 updating version 0 to 573 2024-04-07T16:22:58.000 INFO:tasks.rados.rados.0.smithi100.stdout:1931: oid 21 updating version 573 to 574 2024-04-07T16:22:58.000 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 21 v 574 (ObjNum 593 snap 183 seq_num 593) dirty exists 2024-04-07T16:22:58.000 INFO:tasks.rados.rados.0.smithi100.stdout:1931: left oid 21 (ObjNum 593 snap 183 seq_num 593) 2024-04-07T16:22:58.002 INFO:tasks.rados.rados.0.smithi100.stdout:1926: finishing write tid 1 to smithi10066867-12 2024-04-07T16:22:58.003 INFO:tasks.rados.rados.0.smithi100.stdout:1926: finishing write tid 2 to smithi10066867-12 2024-04-07T16:22:58.006 INFO:tasks.rados.rados.0.smithi100.stdout:1929: finishing rollback tid 1 to smithi10066867-30 2024-04-07T16:22:58.009 INFO:tasks.rados.rados.0.smithi100.stdout:1931: done (6 left) 2024-04-07T16:22:58.009 INFO:tasks.rados.rados.0.smithi100.stdout:1932: done (5 left) 2024-04-07T16:22:58.009 INFO:tasks.rados.rados.0.smithi100.stdout:1933: rollback oid 43 current snap is 183 2024-04-07T16:22:58.009 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 43 to 169 2024-04-07T16:22:58.009 INFO:tasks.rados.rados.0.smithi100.stdout:1934: read oid 8 snap -1 2024-04-07T16:22:58.009 INFO:tasks.rados.rados.0.smithi100.stdout:1934: expect (ObjNum 583 snap 180 seq_num 583) 2024-04-07T16:22:58.009 INFO:tasks.rados.rados.0.smithi100.stdout:1935: delete oid 39 current snap is 183 2024-04-07T16:22:58.014 INFO:tasks.rados.rados.0.smithi100.stdout:1926: finishing write tid 3 to smithi10066867-12 2024-04-07T16:22:58.014 INFO:tasks.rados.rados.0.smithi100.stdout:1926: oid 12 updating version 0 to 1027 2024-04-07T16:22:58.014 INFO:tasks.rados.rados.0.smithi100.stdout:1926: oid 12 updating version 1027 to 1028 2024-04-07T16:22:58.015 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 12 v 1028 (ObjNum 592 snap 183 seq_num 592) dirty exists 2024-04-07T16:22:58.015 INFO:tasks.rados.rados.0.smithi100.stdout:1926: left oid 12 (ObjNum 592 snap 183 seq_num 592) 2024-04-07T16:22:58.015 INFO:tasks.rados.rados.0.smithi100.stdout:1929: finishing rollback tid 2 to smithi10066867-30 2024-04-07T16:22:58.015 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 30 v 1071 (ObjNum 567 snap 174 seq_num 567) dirty exists 2024-04-07T16:22:58.015 INFO:tasks.rados.rados.0.smithi100.stdout:1925: expect (ObjNum 524 snap 156 seq_num 524) 2024-04-07T16:22:58.237 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:57 smithi100 bash[76202]: cluster 2024-04-07T16:22:56.664316+0000 mgr.y (mgr.25175) 151 : cluster [DBG] pgmap v229: 121 pgs: 121 active+clean; 209 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 0 B/s rd, 0 B/s wr, 8 op/s 2024-04-07T16:22:58.237 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:57 smithi100 bash[76202]: cluster 2024-04-07T16:22:56.957318+0000 mon.a (mon.0) 480 : cluster [DBG] osdmap e427: 8 total, 8 up, 8 in 2024-04-07T16:22:58.237 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:57 smithi100 bash[76202]: audit 2024-04-07T16:22:57.755020+0000 mon.a (mon.0) 481 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:22:58.237 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:57 smithi100 bash[76202]: audit 2024-04-07T16:22:57.759890+0000 mon.a (mon.0) 482 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-07T16:22:58.237 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:57 smithi100 bash[76202]: audit 2024-04-07T16:22:57.761607+0000 mon.a (mon.0) 483 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:22:58.237 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:57 smithi100 bash[74942]: cluster 2024-04-07T16:22:56.664316+0000 mgr.y (mgr.25175) 151 : cluster [DBG] pgmap v229: 121 pgs: 121 active+clean; 209 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 0 B/s rd, 0 B/s wr, 8 op/s 2024-04-07T16:22:58.237 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:57 smithi100 bash[74942]: cluster 2024-04-07T16:22:56.957318+0000 mon.a (mon.0) 480 : cluster [DBG] osdmap e427: 8 total, 8 up, 8 in 2024-04-07T16:22:58.238 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:57 smithi100 bash[74942]: audit 2024-04-07T16:22:57.755020+0000 mon.a (mon.0) 481 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:22:58.238 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:57 smithi100 bash[74942]: audit 2024-04-07T16:22:57.759890+0000 mon.a (mon.0) 482 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-07T16:22:58.238 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:57 smithi100 bash[74942]: audit 2024-04-07T16:22:57.761607+0000 mon.a (mon.0) 483 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:22:58.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:57 smithi144 bash[66654]: cluster 2024-04-07T16:22:56.664316+0000 mgr.y (mgr.25175) 151 : cluster [DBG] pgmap v229: 121 pgs: 121 active+clean; 209 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 0 B/s rd, 0 B/s wr, 8 op/s 2024-04-07T16:22:58.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:57 smithi144 bash[66654]: cluster 2024-04-07T16:22:56.957318+0000 mon.a (mon.0) 480 : cluster [DBG] osdmap e427: 8 total, 8 up, 8 in 2024-04-07T16:22:58.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:57 smithi144 bash[66654]: audit 2024-04-07T16:22:57.755020+0000 mon.a (mon.0) 481 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:22:58.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:57 smithi144 bash[66654]: audit 2024-04-07T16:22:57.759890+0000 mon.a (mon.0) 482 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-07T16:22:58.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:57 smithi144 bash[66654]: audit 2024-04-07T16:22:57.761607+0000 mon.a (mon.0) 483 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:22:58.537 INFO:tasks.rados.rados.0.smithi100.stdout:1928: finishing copy_from racing read to smithi10066867-2 2024-04-07T16:22:58.537 INFO:tasks.rados.rados.0.smithi100.stdout:1933: finishing rollback tid 0 to smithi10066867-43 2024-04-07T16:22:58.537 INFO:tasks.rados.rados.0.smithi100.stdout:1933: finishing rollback tid 1 to smithi10066867-43 2024-04-07T16:22:58.538 INFO:tasks.rados.rados.0.smithi100.stdout:1933: finishing rollback tid 2 to smithi10066867-43 2024-04-07T16:22:58.538 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 43 v 812 (ObjNum 465 snap 142 seq_num 465) dirty exists 2024-04-07T16:22:58.538 INFO:tasks.rados.rados.0.smithi100.stdout:1928: finishing copy_from to smithi10066867-2 2024-04-07T16:22:58.538 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 2 v 719 (ObjNum 492 snap 148 seq_num 492) dirty exists 2024-04-07T16:22:58.538 INFO:tasks.rados.rados.0.smithi100.stdout:1930: expect (ObjNum 570 snap 177 seq_num 570) 2024-04-07T16:22:58.685 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:22:58 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:22:58] "GET /metrics HTTP/1.1" 200 36067 "" "Prometheus/2.43.0" 2024-04-07T16:22:58.717 INFO:tasks.rados.rados.0.smithi100.stdout:1934: expect (ObjNum 583 snap 180 seq_num 583) 2024-04-07T16:22:59.223 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:58 smithi100 bash[74942]: cephadm 2024-04-07T16:22:57.747473+0000 mgr.y (mgr.25175) 152 : cephadm [INF] Upgrade: Updating osd.2 2024-04-07T16:22:59.223 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:58 smithi100 bash[74942]: cephadm 2024-04-07T16:22:57.767043+0000 mgr.y (mgr.25175) 153 : cephadm [INF] Deploying daemon osd.2 on smithi100 2024-04-07T16:22:59.223 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:58 smithi100 bash[74942]: cluster 2024-04-07T16:22:57.973933+0000 mon.a (mon.0) 484 : cluster [DBG] osdmap e428: 8 total, 8 up, 8 in 2024-04-07T16:22:59.223 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:58 smithi100 bash[76202]: cephadm 2024-04-07T16:22:57.747473+0000 mgr.y (mgr.25175) 152 : cephadm [INF] Upgrade: Updating osd.2 2024-04-07T16:22:59.223 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:58 smithi100 bash[76202]: cephadm 2024-04-07T16:22:57.767043+0000 mgr.y (mgr.25175) 153 : cephadm [INF] Deploying daemon osd.2 on smithi100 2024-04-07T16:22:59.224 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:58 smithi100 bash[76202]: cluster 2024-04-07T16:22:57.973933+0000 mon.a (mon.0) 484 : cluster [DBG] osdmap e428: 8 total, 8 up, 8 in 2024-04-07T16:22:59.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:58 smithi144 bash[66654]: cephadm 2024-04-07T16:22:57.747473+0000 mgr.y (mgr.25175) 152 : cephadm [INF] Upgrade: Updating osd.2 2024-04-07T16:22:59.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:58 smithi144 bash[66654]: cephadm 2024-04-07T16:22:57.767043+0000 mgr.y (mgr.25175) 153 : cephadm [INF] Deploying daemon osd.2 on smithi100 2024-04-07T16:22:59.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:58 smithi144 bash[66654]: cluster 2024-04-07T16:22:57.973933+0000 mon.a (mon.0) 484 : cluster [DBG] osdmap e428: 8 total, 8 up, 8 in 2024-04-07T16:22:59.479 INFO:tasks.rados.rados.0.smithi100.stdout:1925: done (7 left) 2024-04-07T16:22:59.479 INFO:tasks.rados.rados.0.smithi100.stdout:1926: done (6 left) 2024-04-07T16:22:59.479 INFO:tasks.rados.rados.0.smithi100.stdout:1928: done (5 left) 2024-04-07T16:22:59.479 INFO:tasks.rados.rados.0.smithi100.stdout:1929: done (4 left) 2024-04-07T16:22:59.479 INFO:tasks.rados.rados.0.smithi100.stdout:1930: done (3 left) 2024-04-07T16:22:59.479 INFO:tasks.rados.rados.0.smithi100.stdout:1933: done (2 left) 2024-04-07T16:22:59.479 INFO:tasks.rados.rados.0.smithi100.stdout:1934: done (1 left) 2024-04-07T16:22:59.479 INFO:tasks.rados.rados.0.smithi100.stdout:1935: done (0 left) 2024-04-07T16:22:59.479 INFO:tasks.rados.rados.0.smithi100.stdout:1936: snap_remove snap 180 2024-04-07T16:22:59.970 INFO:tasks.rados.rados.0.smithi100.stdout:1936: done (0 left) 2024-04-07T16:22:59.970 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 23 current snap is 183 2024-04-07T16:22:59.970 INFO:tasks.rados.rados.0.smithi100.stdout:1937: seq_num 596 ranges {5668864=385024} 2024-04-07T16:23:00.009 INFO:tasks.rados.rados.0.smithi100.stdout:1937: writing smithi10066867-23 from 5668864 to 6053888 tid 1 2024-04-07T16:23:00.009 INFO:tasks.rados.rados.0.smithi100.stdout:1938: read oid 15 snap -1 2024-04-07T16:23:00.009 INFO:tasks.rados.rados.0.smithi100.stdout:1938: expect deleted 2024-04-07T16:23:00.009 INFO:tasks.rados.rados.0.smithi100.stdout:1939: setattr oid 36 current snap is 183 2024-04-07T16:23:00.010 INFO:tasks.rados.rados.0.smithi100.stdout:1940: snap_create 2024-04-07T16:23:00.014 INFO:tasks.rados.rados.0.smithi100.stdout:1937: finishing write tid 1 to smithi10066867-23 2024-04-07T16:23:00.015 INFO:tasks.rados.rados.0.smithi100.stdout:1937: finishing write tid 2 to smithi10066867-23 2024-04-07T16:23:00.015 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 36 v 739 (ObjNum 492 snap 148 seq_num 492) dirty exists 2024-04-07T16:23:00.015 INFO:tasks.rados.rados.0.smithi100.stdout:1937: finishing write tid 3 to smithi10066867-23 2024-04-07T16:23:00.015 INFO:tasks.rados.rados.0.smithi100.stdout:1937: oid 23 updating version 0 to 1135 2024-04-07T16:23:00.016 INFO:tasks.rados.rados.0.smithi100.stdout:1937: oid 23 updating version 1135 to 1136 2024-04-07T16:23:00.016 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 23 v 1136 (ObjNum 596 snap 183 seq_num 596) dirty exists 2024-04-07T16:23:00.016 INFO:tasks.rados.rados.0.smithi100.stdout:1937: left oid 23 (ObjNum 596 snap 183 seq_num 596) 2024-04-07T16:23:00.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:22:59 smithi144 bash[66654]: cluster 2024-04-07T16:22:58.665131+0000 mgr.y (mgr.25175) 154 : cluster [DBG] pgmap v232: 121 pgs: 121 active+clean; 219 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 2.2 KiB/s rd, 2.6 MiB/s wr, 23 op/s 2024-04-07T16:23:00.357 INFO:teuthology.orchestra.run.smithi100.stdout:true 2024-04-07T16:23:00.434 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:22:59 smithi100 bash[74942]: cluster 2024-04-07T16:22:58.665131+0000 mgr.y (mgr.25175) 154 : cluster [DBG] pgmap v232: 121 pgs: 121 active+clean; 219 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 2.2 KiB/s rd, 2.6 MiB/s wr, 23 op/s 2024-04-07T16:23:00.434 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:22:59 smithi100 bash[76202]: cluster 2024-04-07T16:22:58.665131+0000 mgr.y (mgr.25175) 154 : cluster [DBG] pgmap v232: 121 pgs: 121 active+clean; 219 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 2.2 KiB/s rd, 2.6 MiB/s wr, 23 op/s 2024-04-07T16:23:00.720 INFO:teuthology.orchestra.run.smithi100.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-07T16:23:00.721 INFO:teuthology.orchestra.run.smithi100.stdout:alertmanager.a smithi100 *:9093,9094 running (9m) 16s ago 9m 14.4M - 0.25.0 c8568f914cd2 5143075e7d26 2024-04-07T16:23:00.721 INFO:teuthology.orchestra.run.smithi100.stdout:grafana.a smithi144 *:3000 running (9m) 102s ago 9m 79.3M - 9.4.7 954c08fa6188 7180c0708acd 2024-04-07T16:23:00.721 INFO:teuthology.orchestra.run.smithi100.stdout:mds.a smithi100 running (10m) 16s ago 10m 15.5M - 18.2.2-1171-g267c8ff5 8a4a01e7186c 92593b0c28ae 2024-04-07T16:23:00.721 INFO:teuthology.orchestra.run.smithi100.stdout:mds.b smithi144 running (10m) 102s ago 10m 18.3M - 18.2.2-1171-g267c8ff5 8a4a01e7186c 3df2ee3e17a4 2024-04-07T16:23:00.721 INFO:teuthology.orchestra.run.smithi100.stdout:mgr.x smithi144 *:8443,9283,8765 running (3m) 102s ago 18m 438M - 19.0.0-2785-gadce92d0 c0ecd4226ac9 233745c292a6 2024-04-07T16:23:00.721 INFO:teuthology.orchestra.run.smithi100.stdout:mgr.y smithi100 *:8443,9283,8765 running (4m) 16s ago 20m 501M - 19.0.0-2785-gadce92d0 c0ecd4226ac9 254dc65cdaa7 2024-04-07T16:23:00.721 INFO:teuthology.orchestra.run.smithi100.stdout:mon.a smithi100 running (2m) 16s ago 20m 50.6M 2048M 19.0.0-2785-gadce92d0 c0ecd4226ac9 718e12655c04 2024-04-07T16:23:00.721 INFO:teuthology.orchestra.run.smithi100.stdout:mon.b smithi144 running (111s) 102s ago 18m 18.2M 2048M 19.0.0-2785-gadce92d0 c0ecd4226ac9 aef817664bff 2024-04-07T16:23:00.721 INFO:teuthology.orchestra.run.smithi100.stdout:mon.c smithi100 running (2m) 16s ago 18m 43.1M 2048M 19.0.0-2785-gadce92d0 c0ecd4226ac9 63e889040f6e 2024-04-07T16:23:00.721 INFO:teuthology.orchestra.run.smithi100.stdout:node-exporter.a smithi100 *:9100 running (9m) 16s ago 9m 8427k - 1.5.0 0da6a335fe13 4bce264c1b26 2024-04-07T16:23:00.721 INFO:teuthology.orchestra.run.smithi100.stdout:node-exporter.b smithi144 *:9100 running (9m) 102s ago 9m 8443k - 1.5.0 0da6a335fe13 b6963413c560 2024-04-07T16:23:00.721 INFO:teuthology.orchestra.run.smithi100.stdout:osd.0 smithi100 running (48s) 16s ago 17m 149M 2524M 19.0.0-2785-gadce92d0 c0ecd4226ac9 a6e83c9d1abc 2024-04-07T16:23:00.721 INFO:teuthology.orchestra.run.smithi100.stdout:osd.1 smithi100 running (21s) 16s ago 16m 11.7M 2524M 19.0.0-2785-gadce92d0 c0ecd4226ac9 c50fe40079b6 2024-04-07T16:23:00.722 INFO:teuthology.orchestra.run.smithi100.stdout:osd.2 smithi100 running (15m) 16s ago 15m 303M 2524M 18.2.2-1171-g267c8ff5 8a4a01e7186c f3912ddcc8cc 2024-04-07T16:23:00.722 INFO:teuthology.orchestra.run.smithi100.stdout:osd.3 smithi100 running (14m) 16s ago 14m 206M 2524M 18.2.2-1171-g267c8ff5 8a4a01e7186c b41004090c63 2024-04-07T16:23:00.722 INFO:teuthology.orchestra.run.smithi100.stdout:osd.4 smithi144 running (14m) 102s ago 14m 257M 2524M 18.2.2-1171-g267c8ff5 8a4a01e7186c 8c73b6b4dc13 2024-04-07T16:23:00.722 INFO:teuthology.orchestra.run.smithi100.stdout:osd.5 smithi144 running (13m) 102s ago 13m 216M 2524M 18.2.2-1171-g267c8ff5 8a4a01e7186c 3bfa0cf6f6cb 2024-04-07T16:23:00.722 INFO:teuthology.orchestra.run.smithi100.stdout:osd.6 smithi144 running (12m) 102s ago 12m 98.7M 2524M 18.2.2-1171-g267c8ff5 8a4a01e7186c 32b7f95d77f0 2024-04-07T16:23:00.722 INFO:teuthology.orchestra.run.smithi100.stdout:osd.7 smithi144 running (11m) 102s ago 11m 156M 2524M 18.2.2-1171-g267c8ff5 8a4a01e7186c 45f7709e576a 2024-04-07T16:23:00.722 INFO:teuthology.orchestra.run.smithi100.stdout:prometheus.a smithi144 *:9095 running (3m) 102s ago 10m 46.5M - 2.43.0 a07b618ecd1d 2d0a8131f303 2024-04-07T16:23:00.984 INFO:tasks.rados.rados.0.smithi100.stdout:1937: done (3 left) 2024-04-07T16:23:00.985 INFO:tasks.rados.rados.0.smithi100.stdout:1938: done (2 left) 2024-04-07T16:23:00.985 INFO:tasks.rados.rados.0.smithi100.stdout:1939: done (1 left) 2024-04-07T16:23:00.985 INFO:tasks.rados.rados.0.smithi100.stdout:1940: done (0 left) 2024-04-07T16:23:00.985 INFO:tasks.rados.rados.0.smithi100.stdout:1941: delete oid 2 current snap is 184 2024-04-07T16:23:00.998 INFO:tasks.rados.rados.0.smithi100.stdout:1941: done (0 left) 2024-04-07T16:23:00.998 INFO:tasks.rados.rados.0.smithi100.stdout:1942: copy_from oid 29 from oid 30 current snap is 184 2024-04-07T16:23:00.998 INFO:tasks.rados.rados.0.smithi100.stdout:1943: read oid 1 snap 169 2024-04-07T16:23:00.998 INFO:tasks.rados.rados.0.smithi100.stdout:1943: expect deleted 2024-04-07T16:23:00.998 INFO:tasks.rados.rados.0.smithi100.stdout:1944: rollback oid 17 current snap is 184 2024-04-07T16:23:00.998 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 17 to 175 2024-04-07T16:23:00.998 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 49 current snap is 184 2024-04-07T16:23:00.998 INFO:tasks.rados.rados.0.smithi100.stdout:1945: seq_num 598 ranges {0=417792} 2024-04-07T16:23:01.002 INFO:tasks.rados.rados.0.smithi100.stdout:1945: writing smithi10066867-49 from 0 to 417792 tid 1 2024-04-07T16:23:01.002 INFO:tasks.rados.rados.0.smithi100.stdout:1946: setattr oid 5 current snap is 184 2024-04-07T16:23:01.004 INFO:tasks.rados.rados.0.smithi100.stdout:1943: done (4 left) 2024-04-07T16:23:01.004 INFO:tasks.rados.rados.0.smithi100.stdout:1947: read oid 3 snap 181 2024-04-07T16:23:01.004 INFO:tasks.rados.rados.0.smithi100.stdout:1947: expect (ObjNum 577 snap 178 seq_num 577) 2024-04-07T16:23:01.004 INFO:tasks.rados.rados.0.smithi100.stdout:1948: snap_create 2024-04-07T16:23:01.006 INFO:tasks.rados.rados.0.smithi100.stdout:1944: finishing rollback tid 0 to smithi10066867-17 2024-04-07T16:23:01.010 INFO:tasks.rados.rados.0.smithi100.stdout:1944: finishing rollback tid 1 to smithi10066867-17 2024-04-07T16:23:01.010 INFO:tasks.rados.rados.0.smithi100.stdout:1944: finishing rollback tid 2 to smithi10066867-17 2024-04-07T16:23:01.010 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 17 v 742 (ObjNum 506 snap 151 seq_num 506) dirty exists 2024-04-07T16:23:01.016 INFO:tasks.rados.rados.0.smithi100.stdout:1945: finishing write tid 1 to smithi10066867-49 2024-04-07T16:23:01.017 INFO:tasks.rados.rados.0.smithi100.stdout:1945: finishing write tid 2 to smithi10066867-49 2024-04-07T16:23:01.020 INFO:tasks.rados.rados.0.smithi100.stdout:1945: finishing write tid 3 to smithi10066867-49 2024-04-07T16:23:01.020 INFO:tasks.rados.rados.0.smithi100.stdout:1945: oid 49 updating version 0 to 1030 2024-04-07T16:23:01.020 INFO:tasks.rados.rados.0.smithi100.stdout:1945: oid 49 version 1030 is already newer than 1029 2024-04-07T16:23:01.020 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 49 v 1030 (ObjNum 598 snap 184 seq_num 598) dirty exists 2024-04-07T16:23:01.020 INFO:tasks.rados.rados.0.smithi100.stdout:1945: left oid 49 (ObjNum 598 snap 184 seq_num 598) 2024-04-07T16:23:01.020 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 5 v 1031 (ObjNum 468 snap 142 seq_num 468) dirty exists 2024-04-07T16:23:01.025 INFO:tasks.rados.rados.0.smithi100.stdout:1947: expect (ObjNum 577 snap 178 seq_num 577) 2024-04-07T16:23:01.164 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-04-07T16:23:01.165 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": { 2024-04-07T16:23:01.165 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph version 19.0.0-2785-gadce92d0 (adce92d056533f22dc1ee37e3141d098b046ec1b) squid (dev)": 3 2024-04-07T16:23:01.165 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-07T16:23:01.165 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": { 2024-04-07T16:23:01.165 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph version 19.0.0-2785-gadce92d0 (adce92d056533f22dc1ee37e3141d098b046ec1b) squid (dev)": 2 2024-04-07T16:23:01.165 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-07T16:23:01.165 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": { 2024-04-07T16:23:01.165 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph version 18.2.2-1171-g267c8ff5 (267c8ff5b1ff5815581b3b2381d19360e081b37b) reef (stable)": 6, 2024-04-07T16:23:01.165 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph version 19.0.0-2785-gadce92d0 (adce92d056533f22dc1ee37e3141d098b046ec1b) squid (dev)": 2 2024-04-07T16:23:01.165 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-07T16:23:01.165 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": { 2024-04-07T16:23:01.165 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph version 18.2.2-1171-g267c8ff5 (267c8ff5b1ff5815581b3b2381d19360e081b37b) reef (stable)": 2 2024-04-07T16:23:01.165 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-07T16:23:01.165 INFO:teuthology.orchestra.run.smithi100.stdout: "overall": { 2024-04-07T16:23:01.166 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph version 18.2.2-1171-g267c8ff5 (267c8ff5b1ff5815581b3b2381d19360e081b37b) reef (stable)": 8, 2024-04-07T16:23:01.166 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph version 19.0.0-2785-gadce92d0 (adce92d056533f22dc1ee37e3141d098b046ec1b) squid (dev)": 7 2024-04-07T16:23:01.166 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-04-07T16:23:01.166 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-04-07T16:23:01.177 INFO:tasks.rados.rados.0.smithi100.stdout:1942: finishing copy_from racing read to smithi10066867-29 2024-04-07T16:23:01.177 INFO:tasks.rados.rados.0.smithi100.stdout:1942: finishing copy_from to smithi10066867-29 2024-04-07T16:23:01.177 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 29 v 1071 (ObjNum 567 snap 174 seq_num 567) dirty exists 2024-04-07T16:23:01.255 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:00 smithi100 bash[74942]: cluster 2024-04-07T16:22:59.986238+0000 mon.a (mon.0) 485 : cluster [DBG] osdmap e429: 8 total, 8 up, 8 in 2024-04-07T16:23:01.255 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:00 smithi100 bash[76202]: cluster 2024-04-07T16:22:59.986238+0000 mon.a (mon.0) 485 : cluster [DBG] osdmap e429: 8 total, 8 up, 8 in 2024-04-07T16:23:01.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:00 smithi144 bash[66654]: cluster 2024-04-07T16:22:59.986238+0000 mon.a (mon.0) 485 : cluster [DBG] osdmap e429: 8 total, 8 up, 8 in 2024-04-07T16:23:01.792 INFO:tasks.rados.rados.0.smithi100.stdout:1942: done (5 left) 2024-04-07T16:23:01.792 INFO:tasks.rados.rados.0.smithi100.stdout:1944: done (4 left) 2024-04-07T16:23:01.793 INFO:tasks.rados.rados.0.smithi100.stdout:1945: done (3 left) 2024-04-07T16:23:01.793 INFO:tasks.rados.rados.0.smithi100.stdout:1946: done (2 left) 2024-04-07T16:23:01.793 INFO:tasks.rados.rados.0.smithi100.stdout:1947: done (1 left) 2024-04-07T16:23:01.793 INFO:tasks.rados.rados.0.smithi100.stdout:1948: done (0 left) 2024-04-07T16:23:01.793 INFO:tasks.rados.rados.0.smithi100.stdout:1949: copy_from oid 21 from oid 7 current snap is 185 2024-04-07T16:23:01.793 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 39 current snap is 185 2024-04-07T16:23:01.793 INFO:tasks.rados.rados.0.smithi100.stdout:1950: seq_num 599 ranges {0=434176} 2024-04-07T16:23:01.796 INFO:tasks.rados.rados.0.smithi100.stdout:1950: writing smithi10066867-39 from 0 to 434176 tid 1 2024-04-07T16:23:01.796 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 38 current snap is 185 2024-04-07T16:23:01.796 INFO:tasks.rados.rados.0.smithi100.stdout:1951: seq_num 600 ranges {1277952=450560} 2024-04-07T16:23:01.816 INFO:tasks.rados.rados.0.smithi100.stdout:1951: writing smithi10066867-38 from 1277952 to 1728512 tid 1 2024-04-07T16:23:01.816 INFO:tasks.rados.rados.0.smithi100.stdout:1952: copy_from oid 40 from oid 18 current snap is 185 2024-04-07T16:23:01.816 INFO:tasks.rados.rados.0.smithi100.stdout:1950: finishing write tid 1 to smithi10066867-39 2024-04-07T16:23:01.816 INFO:tasks.rados.rados.0.smithi100.stdout:1950: finishing write tid 2 to smithi10066867-39 2024-04-07T16:23:01.816 INFO:tasks.rados.rados.0.smithi100.stdout:1950: finishing write tid 3 to smithi10066867-39 2024-04-07T16:23:01.816 INFO:tasks.rados.rados.0.smithi100.stdout:1950: oid 39 updating version 0 to 1073 2024-04-07T16:23:01.816 INFO:tasks.rados.rados.0.smithi100.stdout:1950: oid 39 updating version 1073 to 1074 2024-04-07T16:23:01.816 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 39 v 1074 (ObjNum 599 snap 185 seq_num 599) dirty exists 2024-04-07T16:23:01.816 INFO:tasks.rados.rados.0.smithi100.stdout:1950: left oid 39 (ObjNum 599 snap 185 seq_num 599) 2024-04-07T16:23:01.816 INFO:tasks.rados.rados.0.smithi100.stdout:1950: done (3 left) 2024-04-07T16:23:01.816 INFO:tasks.rados.rados.0.smithi100.stdout:1953: delete oid 24 current snap is 185 2024-04-07T16:23:01.826 INFO:tasks.rados.rados.0.smithi100.stdout:1953: done (3 left) 2024-04-07T16:23:01.827 INFO:tasks.rados.rados.0.smithi100.stdout:1954: read oid 8 snap 175 2024-04-07T16:23:01.827 INFO:tasks.rados.rados.0.smithi100.stdout:1954: expect (ObjNum 558 snap 170 seq_num 558) 2024-04-07T16:23:01.827 INFO:tasks.rados.rados.0.smithi100.stdout:1955: copy_from oid 39 from oid 22 current snap is 185 2024-04-07T16:23:01.827 INFO:tasks.rados.rados.0.smithi100.stdout:1956: read oid 23 snap -1 2024-04-07T16:23:01.827 INFO:tasks.rados.rados.0.smithi100.stdout:1956: expect (ObjNum 596 snap 183 seq_num 596) 2024-04-07T16:23:01.827 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 28 current snap is 185 2024-04-07T16:23:01.827 INFO:tasks.rados.rados.0.smithi100.stdout:1957: seq_num 602 ranges {1662976=491520} 2024-04-07T16:23:01.841 INFO:tasks.rados.rados.0.smithi100.stdout:1957: writing smithi10066867-28 from 1662976 to 2154496 tid 1 2024-04-07T16:23:01.841 INFO:tasks.rados.rados.0.smithi100.stdout:1958: delete oid 45 current snap is 185 2024-04-07T16:23:01.841 INFO:tasks.rados.rados.0.smithi100.stdout:1951: finishing write tid 1 to smithi10066867-38 2024-04-07T16:23:01.841 INFO:tasks.rados.rados.0.smithi100.stdout:1951: finishing write tid 2 to smithi10066867-38 2024-04-07T16:23:01.841 INFO:tasks.rados.rados.0.smithi100.stdout:1951: finishing write tid 3 to smithi10066867-38 2024-04-07T16:23:01.841 INFO:tasks.rados.rados.0.smithi100.stdout:1951: oid 38 updating version 0 to 1085 2024-04-07T16:23:01.841 INFO:tasks.rados.rados.0.smithi100.stdout:1951: oid 38 version 1085 is already newer than 1084 2024-04-07T16:23:01.841 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 38 v 1085 (ObjNum 600 snap 185 seq_num 600) dirty exists 2024-04-07T16:23:01.841 INFO:tasks.rados.rados.0.smithi100.stdout:1951: left oid 38 (ObjNum 600 snap 185 seq_num 600) 2024-04-07T16:23:01.861 INFO:tasks.rados.rados.0.smithi100.stdout:1949: finishing copy_from racing read to smithi10066867-21 2024-04-07T16:23:01.862 INFO:tasks.rados.rados.0.smithi100.stdout:1955: finishing copy_from racing read to smithi10066867-39 2024-04-07T16:23:01.865 INFO:tasks.rados.rados.0.smithi100.stdout:1957: finishing write tid 1 to smithi10066867-28 2024-04-07T16:23:01.865 INFO:tasks.rados.rados.0.smithi100.stdout:1952: finishing copy_from racing read to smithi10066867-40 2024-04-07T16:23:01.866 INFO:tasks.rados.rados.0.smithi100.stdout:1957: finishing write tid 2 to smithi10066867-28 2024-04-07T16:23:01.866 INFO:tasks.rados.rados.0.smithi100.stdout:1949: finishing copy_from to smithi10066867-21 2024-04-07T16:23:01.866 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 21 v 809 (ObjNum 565 snap 173 seq_num 565) dirty exists 2024-04-07T16:23:01.866 INFO:tasks.rados.rados.0.smithi100.stdout:1949: done (7 left) 2024-04-07T16:23:01.866 INFO:tasks.rados.rados.0.smithi100.stdout:1951: done (6 left) 2024-04-07T16:23:01.866 INFO:tasks.rados.rados.0.smithi100.stdout:1958: done (5 left) 2024-04-07T16:23:01.866 INFO:tasks.rados.rados.0.smithi100.stdout:1959: delete oid 17 current snap is 185 2024-04-07T16:23:01.868 INFO:tasks.rados.rados.0.smithi100.stdout:1957: finishing write tid 3 to smithi10066867-28 2024-04-07T16:23:01.868 INFO:tasks.rados.rados.0.smithi100.stdout:1957: oid 28 updating version 0 to 773 2024-04-07T16:23:01.868 INFO:tasks.rados.rados.0.smithi100.stdout:1957: oid 28 version 773 is already newer than 772 2024-04-07T16:23:01.868 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 28 v 773 (ObjNum 602 snap 185 seq_num 602) dirty exists 2024-04-07T16:23:01.868 INFO:tasks.rados.rados.0.smithi100.stdout:1957: left oid 28 (ObjNum 602 snap 185 seq_num 602) 2024-04-07T16:23:01.869 INFO:tasks.rados.rados.0.smithi100.stdout:1955: finishing copy_from to smithi10066867-39 2024-04-07T16:23:01.869 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 39 v 777 (ObjNum 512 snap 152 seq_num 512) dirty exists 2024-04-07T16:23:01.870 INFO:tasks.rados.rados.0.smithi100.stdout:1956: expect (ObjNum 596 snap 183 seq_num 596) 2024-04-07T16:23:02.219 INFO:journalctl@ceph.osd.1.smithi100.stdout:Apr 07 16:23:02 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:23:02.220 INFO:journalctl@ceph.osd.2.smithi100.stdout:Apr 07 16:23:02 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:23:02.220 INFO:journalctl@ceph.osd.3.smithi100.stdout:Apr 07 16:23:02 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:23:02.220 INFO:journalctl@ceph.osd.0.smithi100.stdout:Apr 07 16:23:02 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:23:02.221 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 16:23:02 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:23:02.221 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:01 smithi100 bash[74942]: audit 2024-04-07T16:23:00.360437+0000 mgr.y (mgr.25175) 155 : audit [DBG] from='client.34266 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:23:02.221 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:01 smithi100 bash[74942]: cluster 2024-04-07T16:23:00.665649+0000 mgr.y (mgr.25175) 156 : cluster [DBG] pgmap v234: 121 pgs: 121 active+clean; 219 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 1.9 KiB/s rd, 2.2 MiB/s wr, 20 op/s 2024-04-07T16:23:02.221 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:01 smithi100 bash[74942]: audit 2024-04-07T16:23:00.721662+0000 mgr.y (mgr.25175) 157 : audit [DBG] from='client.44192 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:23:02.221 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:01 smithi100 bash[74942]: cluster 2024-04-07T16:23:01.002370+0000 mon.a (mon.0) 486 : cluster [DBG] osdmap e430: 8 total, 8 up, 8 in 2024-04-07T16:23:02.221 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:01 smithi100 bash[74942]: audit 2024-04-07T16:23:01.183132+0000 mon.a (mon.0) 487 : audit [DBG] from='client.? 172.21.15.100:0/3155357548' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:23:02.221 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:01 smithi100 bash[74942]: audit 2024-04-07T16:23:01.714323+0000 mon.a (mon.0) 488 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:23:02.221 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:01 smithi100 bash[74942]: audit 2024-04-07T16:23:01.716599+0000 mon.a (mon.0) 489 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:23:02.221 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:01 smithi100 bash[74942]: cluster 2024-04-07T16:23:01.810227+0000 mon.a (mon.0) 490 : cluster [DBG] osdmap e431: 8 total, 8 up, 8 in 2024-04-07T16:23:02.222 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:02 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:23:02.222 INFO:journalctl@ceph.mds.a.smithi100.stdout:Apr 07 16:23:02 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:23:02.222 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Apr 07 16:23:02 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:23:02.223 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:23:02 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:23:02.223 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:01 smithi100 bash[76202]: audit 2024-04-07T16:23:00.360437+0000 mgr.y (mgr.25175) 155 : audit [DBG] from='client.34266 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:23:02.223 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:01 smithi100 bash[76202]: cluster 2024-04-07T16:23:00.665649+0000 mgr.y (mgr.25175) 156 : cluster [DBG] pgmap v234: 121 pgs: 121 active+clean; 219 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 1.9 KiB/s rd, 2.2 MiB/s wr, 20 op/s 2024-04-07T16:23:02.223 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:01 smithi100 bash[76202]: audit 2024-04-07T16:23:00.721662+0000 mgr.y (mgr.25175) 157 : audit [DBG] from='client.44192 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:23:02.223 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:01 smithi100 bash[76202]: cluster 2024-04-07T16:23:01.002370+0000 mon.a (mon.0) 486 : cluster [DBG] osdmap e430: 8 total, 8 up, 8 in 2024-04-07T16:23:02.223 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:01 smithi100 bash[76202]: audit 2024-04-07T16:23:01.183132+0000 mon.a (mon.0) 487 : audit [DBG] from='client.? 172.21.15.100:0/3155357548' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:23:02.223 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:01 smithi100 bash[76202]: audit 2024-04-07T16:23:01.714323+0000 mon.a (mon.0) 488 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:23:02.223 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:01 smithi100 bash[76202]: audit 2024-04-07T16:23:01.716599+0000 mon.a (mon.0) 489 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:23:02.223 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:01 smithi100 bash[76202]: cluster 2024-04-07T16:23:01.810227+0000 mon.a (mon.0) 490 : cluster [DBG] osdmap e431: 8 total, 8 up, 8 in 2024-04-07T16:23:02.224 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:02 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:23:02.305 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:01 smithi144 bash[66654]: audit 2024-04-07T16:23:00.360437+0000 mgr.y (mgr.25175) 155 : audit [DBG] from='client.34266 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:23:02.305 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:01 smithi144 bash[66654]: cluster 2024-04-07T16:23:00.665649+0000 mgr.y (mgr.25175) 156 : cluster [DBG] pgmap v234: 121 pgs: 121 active+clean; 219 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 1.9 KiB/s rd, 2.2 MiB/s wr, 20 op/s 2024-04-07T16:23:02.305 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:01 smithi144 bash[66654]: audit 2024-04-07T16:23:00.721662+0000 mgr.y (mgr.25175) 157 : audit [DBG] from='client.44192 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:23:02.305 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:01 smithi144 bash[66654]: cluster 2024-04-07T16:23:01.002370+0000 mon.a (mon.0) 486 : cluster [DBG] osdmap e430: 8 total, 8 up, 8 in 2024-04-07T16:23:02.306 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:01 smithi144 bash[66654]: audit 2024-04-07T16:23:01.183132+0000 mon.a (mon.0) 487 : audit [DBG] from='client.? 172.21.15.100:0/3155357548' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:23:02.306 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:01 smithi144 bash[66654]: audit 2024-04-07T16:23:01.714323+0000 mon.a (mon.0) 488 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:23:02.306 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:01 smithi144 bash[66654]: audit 2024-04-07T16:23:01.716599+0000 mon.a (mon.0) 489 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:23:02.306 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:01 smithi144 bash[66654]: cluster 2024-04-07T16:23:01.810227+0000 mon.a (mon.0) 490 : cluster [DBG] osdmap e431: 8 total, 8 up, 8 in 2024-04-07T16:23:02.327 INFO:tasks.rados.rados.0.smithi100.stdout:1952: finishing copy_from to smithi10066867-40 2024-04-07T16:23:02.327 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 40 v 1094 (ObjNum 509 snap 152 seq_num 509) dirty exists 2024-04-07T16:23:02.327 INFO:tasks.rados.rados.0.smithi100.stdout:1954: expect (ObjNum 558 snap 170 seq_num 558) 2024-04-07T16:23:02.606 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:23:02 smithi144 bash[59794]: ts=2024-04-07T16:23:02.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:23:02.684 INFO:journalctl@ceph.osd.2.smithi100.stdout:Apr 07 16:23:02 smithi100 systemd[1]: Stopping Ceph osd.2 for 0702e3a8-f4f8-11ee-b649-cb9ed24678a4... 2024-04-07T16:23:02.684 INFO:journalctl@ceph.osd.2.smithi100.stdout:Apr 07 16:23:02 smithi100 bash[45799]: debug 2024-04-07T16:23:02.331+0000 7fce728de700 -1 received signal: Terminated from /sbin/docker-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug (PID: 1) UID: 0 2024-04-07T16:23:02.684 INFO:journalctl@ceph.osd.2.smithi100.stdout:Apr 07 16:23:02 smithi100 bash[45799]: debug 2024-04-07T16:23:02.331+0000 7fce728de700 -1 osd.2 431 *** Got signal Terminated *** 2024-04-07T16:23:02.685 INFO:journalctl@ceph.osd.2.smithi100.stdout:Apr 07 16:23:02 smithi100 bash[45799]: debug 2024-04-07T16:23:02.331+0000 7fce728de700 -1 osd.2 431 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-07T16:23:02.930 INFO:tasks.rados.rados.0.smithi100.stdout:1952: done (5 left) 2024-04-07T16:23:02.930 INFO:tasks.rados.rados.0.smithi100.stdout:1954: done (4 left) 2024-04-07T16:23:02.930 INFO:tasks.rados.rados.0.smithi100.stdout:1955: done (3 left) 2024-04-07T16:23:02.930 INFO:tasks.rados.rados.0.smithi100.stdout:1956: done (2 left) 2024-04-07T16:23:02.930 INFO:tasks.rados.rados.0.smithi100.stdout:1957: done (1 left) 2024-04-07T16:23:02.930 INFO:tasks.rados.rados.0.smithi100.stdout:1959: done (0 left) 2024-04-07T16:23:02.930 INFO:tasks.rados.rados.0.smithi100.stdout:1960: read oid 32 snap -1 2024-04-07T16:23:02.930 INFO:tasks.rados.rados.0.smithi100.stdout:1960: expect (ObjNum 589 snap 182 seq_num 589) 2024-04-07T16:23:02.930 INFO:tasks.rados.rados.0.smithi100.stdout:1961: copy_from oid 27 from oid 16 current snap is 185 2024-04-07T16:23:02.930 INFO:tasks.rados.rados.0.smithi100.stdout:1962: setattr oid 7 current snap is 185 2024-04-07T16:23:02.931 INFO:tasks.rados.rados.0.smithi100.stdout:1963: read oid 4 snap -1 2024-04-07T16:23:02.931 INFO:tasks.rados.rados.0.smithi100.stdout:1963: expect deleted 2024-04-07T16:23:02.932 INFO:tasks.rados.rados.0.smithi100.stdout:1964: read oid 34 snap -1 2024-04-07T16:23:02.932 INFO:tasks.rados.rados.0.smithi100.stdout:1964: expect (ObjNum 483 snap 144 seq_num 483) 2024-04-07T16:23:02.932 INFO:tasks.rados.rados.0.smithi100.stdout:1965: snap_create 2024-04-07T16:23:02.937 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 7 v 813 (ObjNum 565 snap 173 seq_num 565) dirty exists 2024-04-07T16:23:02.942 INFO:tasks.rados.rados.0.smithi100.stdout:1964: expect (ObjNum 483 snap 144 seq_num 483) 2024-04-07T16:23:02.966 INFO:tasks.rados.rados.0.smithi100.stdout:1960: expect (ObjNum 589 snap 182 seq_num 589) 2024-04-07T16:23:03.246 INFO:tasks.rados.rados.0.smithi100.stdout:1961: finishing copy_from racing read to smithi10066867-27 2024-04-07T16:23:03.246 INFO:tasks.rados.rados.0.smithi100.stdout:1961: finishing copy_from to smithi10066867-27 2024-04-07T16:23:03.246 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 27 v 624 (ObjNum 465 snap 142 seq_num 465) dirty exists 2024-04-07T16:23:03.246 INFO:tasks.rados.rados.0.smithi100.stdout:1960: done (5 left) 2024-04-07T16:23:03.246 INFO:tasks.rados.rados.0.smithi100.stdout:1961: done (4 left) 2024-04-07T16:23:03.246 INFO:tasks.rados.rados.0.smithi100.stdout:1962: done (3 left) 2024-04-07T16:23:03.247 INFO:tasks.rados.rados.0.smithi100.stdout:1963: done (2 left) 2024-04-07T16:23:03.247 INFO:tasks.rados.rados.0.smithi100.stdout:1964: done (1 left) 2024-04-07T16:23:03.247 INFO:tasks.rados.rados.0.smithi100.stdout:1965: done (0 left) 2024-04-07T16:23:03.247 INFO:tasks.rados.rados.0.smithi100.stdout:1966: copy_from oid 43 from oid 29 current snap is 186 2024-04-07T16:23:03.247 INFO:tasks.rados.rados.0.smithi100.stdout:1967: delete oid 37 current snap is 186 2024-04-07T16:23:03.258 INFO:tasks.rados.rados.0.smithi100.stdout:1967: done (1 left) 2024-04-07T16:23:03.258 INFO:tasks.rados.rados.0.smithi100.stdout:1968: read oid 5 snap -1 2024-04-07T16:23:03.258 INFO:tasks.rados.rados.0.smithi100.stdout:1968: expect (ObjNum 468 snap 142 seq_num 468) 2024-04-07T16:23:03.258 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 16 current snap is 186 2024-04-07T16:23:03.258 INFO:tasks.rados.rados.0.smithi100.stdout:1969: seq_num 606 ranges {3620864=557056} 2024-04-07T16:23:03.322 INFO:tasks.rados.rados.0.smithi100.stdout:1969: writing smithi10066867-16 from 3620864 to 4177920 tid 1 2024-04-07T16:23:03.323 INFO:tasks.rados.rados.0.smithi100.stdout:1970: rollback oid 22 current snap is 186 2024-04-07T16:23:03.323 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 22 to 171 2024-04-07T16:23:03.323 INFO:tasks.rados.rados.0.smithi100.stdout:1966: finishing copy_from racing read to smithi10066867-43 2024-04-07T16:23:03.323 INFO:tasks.rados.rados.0.smithi100.stdout:1966: finishing copy_from to smithi10066867-43 2024-04-07T16:23:03.323 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 43 v 1071 (ObjNum 567 snap 174 seq_num 567) dirty exists 2024-04-07T16:23:03.323 INFO:tasks.rados.rados.0.smithi100.stdout:1966: done (3 left) 2024-04-07T16:23:03.323 INFO:tasks.rados.rados.0.smithi100.stdout:1971: delete oid 10 current snap is 186 2024-04-07T16:23:03.323 INFO:tasks.rados.rados.0.smithi100.stdout:1968: expect (ObjNum 468 snap 142 seq_num 468) 2024-04-07T16:23:03.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:02 smithi144 bash[66654]: cluster 2024-04-07T16:23:02.336117+0000 mon.a (mon.0) 491 : cluster [INF] osd.2 marked itself down and dead 2024-04-07T16:23:03.434 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:02 smithi100 bash[74942]: cluster 2024-04-07T16:23:02.336117+0000 mon.a (mon.0) 491 : cluster [INF] osd.2 marked itself down and dead 2024-04-07T16:23:03.434 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:02 smithi100 bash[76202]: cluster 2024-04-07T16:23:02.336117+0000 mon.a (mon.0) 491 : cluster [INF] osd.2 marked itself down and dead 2024-04-07T16:23:03.857 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:23:03 smithi144 bash[59794]: ts=2024-04-07T16:23:03.509Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=nodes name=CephNodeDiskspaceWarning index=4 msg="Evaluating rule failed" rule="alert: CephNodeDiskspaceWarning\nexpr: predict_linear(node_filesystem_free_bytes{device=~\"/.*\"}[2d], 3600 * 24 * 5)\n * on (instance) group_left (nodename) node_uname_info < 0\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.8.4\n severity: warning\n type: ceph_default\nannotations:\n description: Mountpoint {{ $labels.mountpoint }} on {{ $labels.nodename }} will\n be full in less than 5 days based on the 48 hour trailing fill rate.\n summary: Host filesystem free space is getting low\n" err="found duplicate series for the match group {instance=\"smithi100\"} on the right hand-side of the operation: [{__name__=\"node_uname_info\", domainname=\"(none)\", instance=\"smithi100\", job=\"node\", machine=\"x86_64\", nodename=\"smithi100\", release=\"5.15.0-101-generic\", sysname=\"Linux\", version=\"#111-Ubuntu SMP Tue Mar 5 20:16:58 UTC 2024\"}, {__name__=\"node_uname_info\", cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", domainname=\"(none)\", instance=\"smithi100\", job=\"node\", machine=\"x86_64\", nodename=\"smithi100\", release=\"5.15.0-101-generic\", sysname=\"Linux\", version=\"#111-Ubuntu SMP Tue Mar 5 20:16:58 UTC 2024\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:23:03.863 INFO:tasks.rados.rados.0.smithi100.stdout:1969: finishing write tid 1 to smithi10066867-16 2024-04-07T16:23:03.863 INFO:tasks.rados.rados.0.smithi100.stdout:1969: finishing write tid 2 to smithi10066867-16 2024-04-07T16:23:03.864 INFO:tasks.rados.rados.0.smithi100.stdout:1969: finishing write tid 3 to smithi10066867-16 2024-04-07T16:23:03.864 INFO:tasks.rados.rados.0.smithi100.stdout:1969: oid 16 updating version 0 to 1033 2024-04-07T16:23:03.864 INFO:tasks.rados.rados.0.smithi100.stdout:1969: oid 16 version 1033 is already newer than 1032 2024-04-07T16:23:03.864 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 16 v 1033 (ObjNum 606 snap 186 seq_num 606) dirty exists 2024-04-07T16:23:03.864 INFO:tasks.rados.rados.0.smithi100.stdout:1969: left oid 16 (ObjNum 606 snap 186 seq_num 606) 2024-04-07T16:23:03.864 INFO:tasks.rados.rados.0.smithi100.stdout:1968: done (3 left) 2024-04-07T16:23:03.864 INFO:tasks.rados.rados.0.smithi100.stdout:1969: done (2 left) 2024-04-07T16:23:03.864 INFO:tasks.rados.rados.0.smithi100.stdout:1971: done (1 left) 2024-04-07T16:23:03.864 INFO:tasks.rados.rados.0.smithi100.stdout:1972: snap_remove snap 169 2024-04-07T16:23:04.007 INFO:tasks.rados.rados.0.smithi100.stdout:1972: done (1 left) 2024-04-07T16:23:04.007 INFO:tasks.rados.rados.0.smithi100.stdout:1973: snap_create 2024-04-07T16:23:04.032 INFO:tasks.rados.rados.0.smithi100.stdout:1970: finishing rollback tid 0 to smithi10066867-22 2024-04-07T16:23:04.035 INFO:tasks.rados.rados.0.smithi100.stdout:1970: finishing rollback tid 1 to smithi10066867-22 2024-04-07T16:23:04.035 INFO:tasks.rados.rados.0.smithi100.stdout:1970: finishing rollback tid 2 to smithi10066867-22 2024-04-07T16:23:04.035 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 22 v 860 (ObjNum 512 snap 152 seq_num 512) dirty exists 2024-04-07T16:23:04.314 INFO:journalctl@ceph.osd.2.smithi100.stdout:Apr 07 16:23:04 smithi100 bash[89228]: ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4-osd-2 2024-04-07T16:23:04.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:03 smithi100 bash[74942]: cluster 2024-04-07T16:23:02.666791+0000 mgr.y (mgr.25175) 158 : cluster [DBG] pgmap v237: 121 pgs: 121 active+clean; 234 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 24 MiB/s rd, 3.1 MiB/s wr, 15 op/s 2024-04-07T16:23:04.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:03 smithi100 bash[74942]: cluster 2024-04-07T16:23:02.995166+0000 mon.a (mon.0) 492 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN) 2024-04-07T16:23:04.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:03 smithi100 bash[74942]: cluster 2024-04-07T16:23:03.005842+0000 mon.a (mon.0) 493 : cluster [DBG] osdmap e432: 8 total, 7 up, 8 in 2024-04-07T16:23:04.315 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:03 smithi100 bash[76202]: cluster 2024-04-07T16:23:02.666791+0000 mgr.y (mgr.25175) 158 : cluster [DBG] pgmap v237: 121 pgs: 121 active+clean; 234 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 24 MiB/s rd, 3.1 MiB/s wr, 15 op/s 2024-04-07T16:23:04.315 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:03 smithi100 bash[76202]: cluster 2024-04-07T16:23:02.995166+0000 mon.a (mon.0) 492 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN) 2024-04-07T16:23:04.315 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:03 smithi100 bash[76202]: cluster 2024-04-07T16:23:03.005842+0000 mon.a (mon.0) 493 : cluster [DBG] osdmap e432: 8 total, 7 up, 8 in 2024-04-07T16:23:04.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:03 smithi144 bash[66654]: cluster 2024-04-07T16:23:02.666791+0000 mgr.y (mgr.25175) 158 : cluster [DBG] pgmap v237: 121 pgs: 121 active+clean; 234 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 24 MiB/s rd, 3.1 MiB/s wr, 15 op/s 2024-04-07T16:23:04.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:03 smithi144 bash[66654]: cluster 2024-04-07T16:23:02.995166+0000 mon.a (mon.0) 492 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN) 2024-04-07T16:23:04.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:03 smithi144 bash[66654]: cluster 2024-04-07T16:23:03.005842+0000 mon.a (mon.0) 493 : cluster [DBG] osdmap e432: 8 total, 7 up, 8 in 2024-04-07T16:23:04.684 INFO:journalctl@ceph.osd.2.smithi100.stdout:Apr 07 16:23:04 smithi100 bash[89290]: Error response from daemon: No such container: ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4-osd-2 2024-04-07T16:23:05.004 INFO:tasks.rados.rados.0.smithi100.stdout:1970: done (1 left) 2024-04-07T16:23:05.004 INFO:tasks.rados.rados.0.smithi100.stdout:1973: done (0 left) 2024-04-07T16:23:05.004 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 10 current snap is 187 2024-04-07T16:23:05.004 INFO:tasks.rados.rados.0.smithi100.stdout:1974: seq_num 608 ranges {0=589824} 2024-04-07T16:23:05.013 INFO:tasks.rados.rados.0.smithi100.stdout:1974: writing smithi10066867-10 from 0 to 589824 tid 1 2024-04-07T16:23:05.013 INFO:tasks.rados.rados.0.smithi100.stdout:1975: snap_create 2024-04-07T16:23:05.027 INFO:tasks.rados.rados.0.smithi100.stdout:1974: finishing write tid 1 to smithi10066867-10 2024-04-07T16:23:05.027 INFO:tasks.rados.rados.0.smithi100.stdout:1974: finishing write tid 2 to smithi10066867-10 2024-04-07T16:23:05.034 INFO:tasks.rados.rados.0.smithi100.stdout:1974: finishing write tid 3 to smithi10066867-10 2024-04-07T16:23:05.034 INFO:tasks.rados.rados.0.smithi100.stdout:1974: oid 10 updating version 0 to 982 2024-04-07T16:23:05.034 INFO:tasks.rados.rados.0.smithi100.stdout:1974: oid 10 updating version 982 to 983 2024-04-07T16:23:05.034 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 10 v 983 (ObjNum 608 snap 187 seq_num 608) dirty exists 2024-04-07T16:23:05.034 INFO:tasks.rados.rados.0.smithi100.stdout:1974: left oid 10 (ObjNum 608 snap 187 seq_num 608) 2024-04-07T16:23:05.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:05 smithi144 bash[66654]: cluster 2024-04-07T16:23:04.017937+0000 mon.a (mon.0) 494 : cluster [DBG] osdmap e433: 8 total, 7 up, 8 in 2024-04-07T16:23:05.434 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:05 smithi100 bash[74942]: cluster 2024-04-07T16:23:04.017937+0000 mon.a (mon.0) 494 : cluster [DBG] osdmap e433: 8 total, 7 up, 8 in 2024-04-07T16:23:05.434 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:05 smithi100 bash[76202]: cluster 2024-04-07T16:23:04.017937+0000 mon.a (mon.0) 494 : cluster [DBG] osdmap e433: 8 total, 7 up, 8 in 2024-04-07T16:23:06.005 INFO:tasks.rados.rados.0.smithi100.stdout:1974: done (1 left) 2024-04-07T16:23:06.005 INFO:tasks.rados.rados.0.smithi100.stdout:1975: done (0 left) 2024-04-07T16:23:06.005 INFO:tasks.rados.rados.0.smithi100.stdout:1976: rollback oid 27 current snap is 188 2024-04-07T16:23:06.005 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 27 to 184 2024-04-07T16:23:06.005 INFO:tasks.rados.rados.0.smithi100.stdout:1977: delete oid 39 current snap is 188 2024-04-07T16:23:06.015 INFO:tasks.rados.rados.0.smithi100.stdout:1976: finishing rollback tid 0 to smithi10066867-27 2024-04-07T16:23:06.017 INFO:tasks.rados.rados.0.smithi100.stdout:1976: finishing rollback tid 1 to smithi10066867-27 2024-04-07T16:23:06.018 INFO:tasks.rados.rados.0.smithi100.stdout:1976: finishing rollback tid 2 to smithi10066867-27 2024-04-07T16:23:06.018 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 27 v 1074 (ObjNum 541 snap 164 seq_num 541) dirty exists 2024-04-07T16:23:06.022 INFO:tasks.rados.rados.0.smithi100.stdout:1976: done (1 left) 2024-04-07T16:23:06.022 INFO:tasks.rados.rados.0.smithi100.stdout:1977: done (0 left) 2024-04-07T16:23:06.022 INFO:tasks.rados.rados.0.smithi100.stdout:1978: rollback oid 50 current snap is 188 2024-04-07T16:23:06.022 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 50 to 171 2024-04-07T16:23:06.022 INFO:tasks.rados.rados.0.smithi100.stdout:1979: copy_from oid 26 from oid 41 current snap is 188 2024-04-07T16:23:06.022 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 20 current snap is 188 2024-04-07T16:23:06.022 INFO:tasks.rados.rados.0.smithi100.stdout:1980: seq_num 610 ranges {9003008=622592} 2024-04-07T16:23:06.105 INFO:tasks.rados.rados.0.smithi100.stdout:1980: writing smithi10066867-20 from 9003008 to 9625600 tid 1 2024-04-07T16:23:06.106 INFO:tasks.rados.rados.0.smithi100.stdout:1981: snap_create 2024-04-07T16:23:06.106 INFO:tasks.rados.rados.0.smithi100.stdout:1979: finishing copy_from to smithi10066867-26 2024-04-07T16:23:06.106 INFO:tasks.rados.rados.0.smithi100.stdout:1979: got expected ENOENT (src dne) 2024-04-07T16:23:06.106 INFO:tasks.rados.rados.0.smithi100.stdout:1979: finishing copy_from racing read to smithi10066867-26 2024-04-07T16:23:06.106 INFO:tasks.rados.rados.0.smithi100.stdout:1978: finishing rollback tid 0 to smithi10066867-50 2024-04-07T16:23:06.106 INFO:tasks.rados.rados.0.smithi100.stdout:1978: finishing rollback tid 1 to smithi10066867-50 2024-04-07T16:23:06.106 INFO:tasks.rados.rados.0.smithi100.stdout:1978: finishing rollback tid 2 to smithi10066867-50 2024-04-07T16:23:06.106 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 50 v 1009 (ObjNum 510 snap 152 seq_num 510) dirty exists 2024-04-07T16:23:06.115 INFO:tasks.rados.rados.0.smithi100.stdout:1980: finishing write tid 1 to smithi10066867-20 2024-04-07T16:23:06.116 INFO:tasks.rados.rados.0.smithi100.stdout:1980: finishing write tid 2 to smithi10066867-20 2024-04-07T16:23:06.119 INFO:tasks.rados.rados.0.smithi100.stdout:1980: finishing write tid 3 to smithi10066867-20 2024-04-07T16:23:06.119 INFO:tasks.rados.rados.0.smithi100.stdout:1980: oid 20 updating version 0 to 861 2024-04-07T16:23:06.119 INFO:tasks.rados.rados.0.smithi100.stdout:1980: oid 20 updating version 861 to 862 2024-04-07T16:23:06.119 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 20 v 862 (ObjNum 610 snap 188 seq_num 610) dirty exists 2024-04-07T16:23:06.119 INFO:tasks.rados.rados.0.smithi100.stdout:1980: left oid 20 (ObjNum 610 snap 188 seq_num 610) 2024-04-07T16:23:06.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:06 smithi144 bash[66654]: cluster 2024-04-07T16:23:04.667491+0000 mgr.y (mgr.25175) 159 : cluster [DBG] pgmap v240: 121 pgs: 14 stale+active+clean, 107 active+clean; 250 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 29 MiB/s rd, 6.3 MiB/s wr, 26 op/s 2024-04-07T16:23:06.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:06 smithi144 bash[66654]: cluster 2024-04-07T16:23:05.021550+0000 mon.a (mon.0) 495 : cluster [DBG] osdmap e434: 8 total, 7 up, 8 in 2024-04-07T16:23:06.411 INFO:journalctl@ceph.osd.2.smithi100.stdout:Apr 07 16:23:06 smithi100 systemd[1]: ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@osd.2.service: Deactivated successfully. 2024-04-07T16:23:06.412 INFO:journalctl@ceph.osd.2.smithi100.stdout:Apr 07 16:23:06 smithi100 systemd[1]: Stopped Ceph osd.2 for 0702e3a8-f4f8-11ee-b649-cb9ed24678a4. 2024-04-07T16:23:06.412 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:06 smithi100 bash[74942]: cluster 2024-04-07T16:23:04.667491+0000 mgr.y (mgr.25175) 159 : cluster [DBG] pgmap v240: 121 pgs: 14 stale+active+clean, 107 active+clean; 250 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 29 MiB/s rd, 6.3 MiB/s wr, 26 op/s 2024-04-07T16:23:06.412 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:06 smithi100 bash[74942]: cluster 2024-04-07T16:23:05.021550+0000 mon.a (mon.0) 495 : cluster [DBG] osdmap e434: 8 total, 7 up, 8 in 2024-04-07T16:23:06.412 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:06 smithi100 bash[76202]: cluster 2024-04-07T16:23:04.667491+0000 mgr.y (mgr.25175) 159 : cluster [DBG] pgmap v240: 121 pgs: 14 stale+active+clean, 107 active+clean; 250 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 29 MiB/s rd, 6.3 MiB/s wr, 26 op/s 2024-04-07T16:23:06.412 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:06 smithi100 bash[76202]: cluster 2024-04-07T16:23:05.021550+0000 mon.a (mon.0) 495 : cluster [DBG] osdmap e434: 8 total, 7 up, 8 in 2024-04-07T16:23:06.684 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:06 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:23:06.684 INFO:journalctl@ceph.osd.1.smithi100.stdout:Apr 07 16:23:06 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:23:06.685 INFO:journalctl@ceph.mds.a.smithi100.stdout:Apr 07 16:23:06 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:23:06.685 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 16:23:06 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:23:06.685 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Apr 07 16:23:06 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:23:06.686 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:23:06 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:23:06.686 INFO:journalctl@ceph.osd.0.smithi100.stdout:Apr 07 16:23:06 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:23:06.686 INFO:journalctl@ceph.osd.3.smithi100.stdout:Apr 07 16:23:06 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:23:06.687 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:06 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:23:06.687 INFO:journalctl@ceph.osd.2.smithi100.stdout:Apr 07 16:23:06 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:23:06.687 INFO:journalctl@ceph.osd.2.smithi100.stdout:Apr 07 16:23:06 smithi100 systemd[1]: Started Ceph osd.2 for 0702e3a8-f4f8-11ee-b649-cb9ed24678a4. 2024-04-07T16:23:07.025 INFO:tasks.rados.rados.0.smithi100.stdout:1978: done (3 left) 2024-04-07T16:23:07.025 INFO:tasks.rados.rados.0.smithi100.stdout:1979: done (2 left) 2024-04-07T16:23:07.026 INFO:tasks.rados.rados.0.smithi100.stdout:1980: done (1 left) 2024-04-07T16:23:07.026 INFO:tasks.rados.rados.0.smithi100.stdout:1981: done (0 left) 2024-04-07T16:23:07.026 INFO:tasks.rados.rados.0.smithi100.stdout:1982: snap_remove snap 184 2024-04-07T16:23:07.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:07 smithi144 bash[66654]: cluster 2024-04-07T16:23:06.022292+0000 mon.a (mon.0) 496 : cluster [DBG] osdmap e435: 8 total, 7 up, 8 in 2024-04-07T16:23:07.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:07 smithi144 bash[66654]: audit 2024-04-07T16:23:06.593640+0000 mon.a (mon.0) 497 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:23:07.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:07 smithi144 bash[66654]: audit 2024-04-07T16:23:06.602250+0000 mon.a (mon.0) 498 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:23:07.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:07 smithi144 bash[66654]: audit 2024-04-07T16:23:06.604784+0000 mon.a (mon.0) 499 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:23:07.434 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:07 smithi100 bash[76202]: cluster 2024-04-07T16:23:06.022292+0000 mon.a (mon.0) 496 : cluster [DBG] osdmap e435: 8 total, 7 up, 8 in 2024-04-07T16:23:07.434 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:07 smithi100 bash[76202]: audit 2024-04-07T16:23:06.593640+0000 mon.a (mon.0) 497 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:23:07.434 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:07 smithi100 bash[76202]: audit 2024-04-07T16:23:06.602250+0000 mon.a (mon.0) 498 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:23:07.434 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:07 smithi100 bash[76202]: audit 2024-04-07T16:23:06.604784+0000 mon.a (mon.0) 499 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:23:07.435 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:07 smithi100 bash[74942]: cluster 2024-04-07T16:23:06.022292+0000 mon.a (mon.0) 496 : cluster [DBG] osdmap e435: 8 total, 7 up, 8 in 2024-04-07T16:23:07.435 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:07 smithi100 bash[74942]: audit 2024-04-07T16:23:06.593640+0000 mon.a (mon.0) 497 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:23:07.435 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:07 smithi100 bash[74942]: audit 2024-04-07T16:23:06.602250+0000 mon.a (mon.0) 498 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:23:07.435 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:07 smithi100 bash[74942]: audit 2024-04-07T16:23:06.604784+0000 mon.a (mon.0) 499 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:23:08.028 INFO:tasks.rados.rados.0.smithi100.stdout:1982: done (0 left) 2024-04-07T16:23:08.028 INFO:tasks.rados.rados.0.smithi100.stdout:1983: read oid 42 snap 175 2024-04-07T16:23:08.028 INFO:tasks.rados.rados.0.smithi100.stdout:1983: expect (ObjNum 533 snap 161 seq_num 533) 2024-04-07T16:23:08.028 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 33 current snap is 189 2024-04-07T16:23:08.029 INFO:tasks.rados.rados.0.smithi100.stdout:1984: seq_num 611 ranges {0=638976} 2024-04-07T16:23:08.033 INFO:tasks.rados.rados.0.smithi100.stdout:1984: writing smithi10066867-33 from 0 to 638976 tid 1 2024-04-07T16:23:08.033 INFO:tasks.rados.rados.0.smithi100.stdout:1985: delete oid 6 current snap is 189 2024-04-07T16:23:08.041 INFO:tasks.rados.rados.0.smithi100.stdout:1983: expect (ObjNum 533 snap 161 seq_num 533) 2024-04-07T16:23:08.087 INFO:tasks.rados.rados.0.smithi100.stdout:1984: finishing write tid 1 to smithi10066867-33 2024-04-07T16:23:08.087 INFO:tasks.rados.rados.0.smithi100.stdout:1984: finishing write tid 2 to smithi10066867-33 2024-04-07T16:23:08.087 INFO:tasks.rados.rados.0.smithi100.stdout:1984: finishing write tid 3 to smithi10066867-33 2024-04-07T16:23:08.087 INFO:tasks.rados.rados.0.smithi100.stdout:1984: oid 33 updating version 0 to 1138 2024-04-07T16:23:08.087 INFO:tasks.rados.rados.0.smithi100.stdout:1984: oid 33 version 1138 is already newer than 1137 2024-04-07T16:23:08.087 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 33 v 1138 (ObjNum 611 snap 189 seq_num 611) dirty exists 2024-04-07T16:23:08.087 INFO:tasks.rados.rados.0.smithi100.stdout:1984: left oid 33 (ObjNum 611 snap 189 seq_num 611) 2024-04-07T16:23:08.088 INFO:tasks.rados.rados.0.smithi100.stdout:1983: done (2 left) 2024-04-07T16:23:08.088 INFO:tasks.rados.rados.0.smithi100.stdout:1984: done (1 left) 2024-04-07T16:23:08.088 INFO:tasks.rados.rados.0.smithi100.stdout:1985: done (0 left) 2024-04-07T16:23:08.088 INFO:tasks.rados.rados.0.smithi100.stdout:1986: read oid 39 snap -1 2024-04-07T16:23:08.088 INFO:tasks.rados.rados.0.smithi100.stdout:1986: expect deleted 2024-04-07T16:23:08.088 INFO:tasks.rados.rados.0.smithi100.stdout:1987: read oid 27 snap -1 2024-04-07T16:23:08.088 INFO:tasks.rados.rados.0.smithi100.stdout:1987: expect (ObjNum 541 snap 164 seq_num 541) 2024-04-07T16:23:08.088 INFO:tasks.rados.rados.0.smithi100.stdout:1988: snap_remove snap 181 2024-04-07T16:23:08.324 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:08 smithi100 bash[74942]: cluster 2024-04-07T16:23:06.668029+0000 mgr.y (mgr.25175) 160 : cluster [DBG] pgmap v243: 121 pgs: 1 active+undersized+degraded, 13 stale+active+clean, 4 active+undersized, 103 active+clean; 259 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 4.9 MiB/s wr, 14 op/s; 4/462 objects degraded (0.866%) 2024-04-07T16:23:08.324 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:08 smithi100 bash[74942]: cluster 2024-04-07T16:23:07.044261+0000 mon.a (mon.0) 500 : cluster [DBG] osdmap e436: 8 total, 7 up, 8 in 2024-04-07T16:23:08.324 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:08 smithi100 bash[74942]: cluster 2024-04-07T16:23:07.599328+0000 mon.a (mon.0) 501 : cluster [WRN] Health check failed: Degraded data redundancy: 4/462 objects degraded (0.866%), 1 pg degraded (PG_DEGRADED) 2024-04-07T16:23:08.325 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:08 smithi100 bash[76202]: cluster 2024-04-07T16:23:06.668029+0000 mgr.y (mgr.25175) 160 : cluster [DBG] pgmap v243: 121 pgs: 1 active+undersized+degraded, 13 stale+active+clean, 4 active+undersized, 103 active+clean; 259 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 4.9 MiB/s wr, 14 op/s; 4/462 objects degraded (0.866%) 2024-04-07T16:23:08.325 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:08 smithi100 bash[76202]: cluster 2024-04-07T16:23:07.044261+0000 mon.a (mon.0) 500 : cluster [DBG] osdmap e436: 8 total, 7 up, 8 in 2024-04-07T16:23:08.325 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:08 smithi100 bash[76202]: cluster 2024-04-07T16:23:07.599328+0000 mon.a (mon.0) 501 : cluster [WRN] Health check failed: Degraded data redundancy: 4/462 objects degraded (0.866%), 1 pg degraded (PG_DEGRADED) 2024-04-07T16:23:08.325 INFO:journalctl@ceph.osd.2.smithi100.stdout:Apr 07 16:23:08 smithi100 bash[89441]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-07T16:23:08.325 INFO:journalctl@ceph.osd.2.smithi100.stdout:Apr 07 16:23:08 smithi100 bash[89441]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-2 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-04-07T16:23:08.325 INFO:journalctl@ceph.osd.2.smithi100.stdout:Apr 07 16:23:08 smithi100 bash[89441]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-04-07T16:23:08.325 INFO:journalctl@ceph.osd.2.smithi100.stdout:Apr 07 16:23:08 smithi100 bash[89441]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-04-07T16:23:08.325 INFO:journalctl@ceph.osd.2.smithi100.stdout:Apr 07 16:23:08 smithi100 bash[89441]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-2/block 2024-04-07T16:23:08.325 INFO:journalctl@ceph.osd.2.smithi100.stdout:Apr 07 16:23:08 smithi100 bash[89441]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-07T16:23:08.325 INFO:journalctl@ceph.osd.2.smithi100.stdout:Apr 07 16:23:08 smithi100 bash[89441]: --> ceph-volume raw activate successful for osd ID: 2 2024-04-07T16:23:08.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:08 smithi144 bash[66654]: cluster 2024-04-07T16:23:06.668029+0000 mgr.y (mgr.25175) 160 : cluster [DBG] pgmap v243: 121 pgs: 1 active+undersized+degraded, 13 stale+active+clean, 4 active+undersized, 103 active+clean; 259 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 4.9 MiB/s wr, 14 op/s; 4/462 objects degraded (0.866%) 2024-04-07T16:23:08.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:08 smithi144 bash[66654]: cluster 2024-04-07T16:23:07.044261+0000 mon.a (mon.0) 500 : cluster [DBG] osdmap e436: 8 total, 7 up, 8 in 2024-04-07T16:23:08.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:08 smithi144 bash[66654]: cluster 2024-04-07T16:23:07.599328+0000 mon.a (mon.0) 501 : cluster [WRN] Health check failed: Degraded data redundancy: 4/462 objects degraded (0.866%), 1 pg degraded (PG_DEGRADED) 2024-04-07T16:23:08.684 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:23:08 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:23:08] "GET /metrics HTTP/1.1" 200 36211 "" "Prometheus/2.43.0" 2024-04-07T16:23:09.057 INFO:tasks.rados.rados.0.smithi100.stdout:1988: done (2 left) 2024-04-07T16:23:09.057 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 9 current snap is 189 2024-04-07T16:23:09.057 INFO:tasks.rados.rados.0.smithi100.stdout:1989: seq_num 613 ranges {0=671744} 2024-04-07T16:23:09.060 INFO:tasks.rados.rados.0.smithi100.stdout:1989: writing smithi10066867-9 from 0 to 671744 tid 1 2024-04-07T16:23:09.060 INFO:tasks.rados.rados.0.smithi100.stdout:1990: snap_remove snap 171 2024-04-07T16:23:09.184 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:09 smithi100 bash[74942]: cluster 2024-04-07T16:23:08.045944+0000 mon.a (mon.0) 502 : cluster [DBG] osdmap e437: 8 total, 7 up, 8 in 2024-04-07T16:23:09.184 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:09 smithi100 bash[76202]: cluster 2024-04-07T16:23:08.045944+0000 mon.a (mon.0) 502 : cluster [DBG] osdmap e437: 8 total, 7 up, 8 in 2024-04-07T16:23:09.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:09 smithi144 bash[66654]: cluster 2024-04-07T16:23:08.045944+0000 mon.a (mon.0) 502 : cluster [DBG] osdmap e437: 8 total, 7 up, 8 in 2024-04-07T16:23:09.934 INFO:journalctl@ceph.osd.2.smithi100.stdout:Apr 07 16:23:09 smithi100 bash[89668]: debug 2024-04-07T16:23:09.623+0000 7f35319c4700 1 -- 172.21.15.100:0/3569491129 <== mon.2 v2:172.21.15.100:3301/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5596c8273680 con 0x5596c82b1000 2024-04-07T16:23:10.057 INFO:tasks.rados.rados.0.smithi100.stdout:1990: done (3 left) 2024-04-07T16:23:10.057 INFO:tasks.rados.rados.0.smithi100.stdout:1991: delete oid 10 current snap is 189 2024-04-07T16:23:10.057 INFO:tasks.rados.rados.0.smithi100.stdout:1987: expect (ObjNum 541 snap 164 seq_num 541) 2024-04-07T16:23:10.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:10 smithi144 bash[66654]: cluster 2024-04-07T16:23:08.668861+0000 mgr.y (mgr.25175) 161 : cluster [DBG] pgmap v246: 121 pgs: 10 active+undersized+degraded, 26 active+undersized, 85 active+clean; 261 MiB data, 1.7 GiB used, 714 GiB / 715 GiB avail; 952 KiB/s rd, 2.7 MiB/s wr, 10 op/s; 69/483 objects degraded (14.286%) 2024-04-07T16:23:10.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:10 smithi144 bash[66654]: cluster 2024-04-07T16:23:09.074607+0000 mon.a (mon.0) 503 : cluster [DBG] osdmap e438: 8 total, 7 up, 8 in 2024-04-07T16:23:10.399 INFO:tasks.rados.rados.0.smithi100.stdout:1989: finishing write tid 1 to smithi10066867-9 2024-04-07T16:23:10.399 INFO:tasks.rados.rados.0.smithi100.stdout:1989: finishing write tid 2 to smithi10066867-9 2024-04-07T16:23:10.399 INFO:tasks.rados.rados.0.smithi100.stdout:1989: finishing write tid 3 to smithi10066867-9 2024-04-07T16:23:10.399 INFO:tasks.rados.rados.0.smithi100.stdout:1989: oid 9 updating version 0 to 551 2024-04-07T16:23:10.399 INFO:tasks.rados.rados.0.smithi100.stdout:1989: oid 9 version 551 is already newer than 550 2024-04-07T16:23:10.399 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 9 v 551 (ObjNum 613 snap 189 seq_num 613) dirty exists 2024-04-07T16:23:10.399 INFO:tasks.rados.rados.0.smithi100.stdout:1989: left oid 9 (ObjNum 613 snap 189 seq_num 613) 2024-04-07T16:23:10.399 INFO:tasks.rados.rados.0.smithi100.stdout:1986: done (3 left) 2024-04-07T16:23:10.399 INFO:tasks.rados.rados.0.smithi100.stdout:1987: done (2 left) 2024-04-07T16:23:10.399 INFO:tasks.rados.rados.0.smithi100.stdout:1989: done (1 left) 2024-04-07T16:23:10.399 INFO:tasks.rados.rados.0.smithi100.stdout:1991: done (0 left) 2024-04-07T16:23:10.400 INFO:tasks.rados.rados.0.smithi100.stdout:1992: snap_create 2024-04-07T16:23:10.434 INFO:journalctl@ceph.osd.2.smithi100.stdout:Apr 07 16:23:10 smithi100 bash[89668]: debug 2024-04-07T16:23:10.159+0000 7f353ab5c7c0 -1 Falling back to public interface 2024-04-07T16:23:10.434 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:10 smithi100 bash[74942]: cluster 2024-04-07T16:23:08.668861+0000 mgr.y (mgr.25175) 161 : cluster [DBG] pgmap v246: 121 pgs: 10 active+undersized+degraded, 26 active+undersized, 85 active+clean; 261 MiB data, 1.7 GiB used, 714 GiB / 715 GiB avail; 952 KiB/s rd, 2.7 MiB/s wr, 10 op/s; 69/483 objects degraded (14.286%) 2024-04-07T16:23:10.434 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:10 smithi100 bash[74942]: cluster 2024-04-07T16:23:09.074607+0000 mon.a (mon.0) 503 : cluster [DBG] osdmap e438: 8 total, 7 up, 8 in 2024-04-07T16:23:10.434 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:10 smithi100 bash[76202]: cluster 2024-04-07T16:23:08.668861+0000 mgr.y (mgr.25175) 161 : cluster [DBG] pgmap v246: 121 pgs: 10 active+undersized+degraded, 26 active+undersized, 85 active+clean; 261 MiB data, 1.7 GiB used, 714 GiB / 715 GiB avail; 952 KiB/s rd, 2.7 MiB/s wr, 10 op/s; 69/483 objects degraded (14.286%) 2024-04-07T16:23:10.435 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:10 smithi100 bash[76202]: cluster 2024-04-07T16:23:09.074607+0000 mon.a (mon.0) 503 : cluster [DBG] osdmap e438: 8 total, 7 up, 8 in 2024-04-07T16:23:11.067 INFO:tasks.rados.rados.0.smithi100.stdout:1992: done (0 left) 2024-04-07T16:23:11.067 INFO:tasks.rados.rados.0.smithi100.stdout:1993: read oid 15 snap 178 2024-04-07T16:23:11.067 INFO:tasks.rados.rados.0.smithi100.stdout:1993: expect (ObjNum 575 snap 177 seq_num 575) 2024-04-07T16:23:11.067 INFO:tasks.rados.rados.0.smithi100.stdout:1994: snap_remove snap 185 2024-04-07T16:23:11.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:11 smithi144 bash[66654]: cluster 2024-04-07T16:23:10.074770+0000 mon.a (mon.0) 504 : cluster [DBG] osdmap e439: 8 total, 7 up, 8 in 2024-04-07T16:23:11.434 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:11 smithi100 bash[74942]: cluster 2024-04-07T16:23:10.074770+0000 mon.a (mon.0) 504 : cluster [DBG] osdmap e439: 8 total, 7 up, 8 in 2024-04-07T16:23:11.434 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:11 smithi100 bash[76202]: cluster 2024-04-07T16:23:10.074770+0000 mon.a (mon.0) 504 : cluster [DBG] osdmap e439: 8 total, 7 up, 8 in 2024-04-07T16:23:11.792 INFO:tasks.rados.rados.0.smithi100.stdout:1994: done (1 left) 2024-04-07T16:23:11.793 INFO:tasks.rados.rados.0.smithi100.stdout:1995: snap_remove snap 167 2024-04-07T16:23:12.356 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:23:12 smithi144 bash[59794]: ts=2024-04-07T16:23:12.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:23:12.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:12 smithi144 bash[66654]: cluster 2024-04-07T16:23:10.669745+0000 mgr.y (mgr.25175) 162 : cluster [DBG] pgmap v249: 121 pgs: 10 active+undersized+degraded, 26 active+undersized, 85 active+clean; 266 MiB data, 1.7 GiB used, 714 GiB / 715 GiB avail; 5.1 MiB/s rd, 432 KiB/s wr, 6 op/s; 69/486 objects degraded (14.198%) 2024-04-07T16:23:12.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:12 smithi144 bash[66654]: cluster 2024-04-07T16:23:11.084767+0000 mon.a (mon.0) 505 : cluster [DBG] osdmap e440: 8 total, 7 up, 8 in 2024-04-07T16:23:12.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:12 smithi144 bash[66654]: cluster 2024-04-07T16:23:11.811091+0000 mon.a (mon.0) 506 : cluster [DBG] osdmap e441: 8 total, 7 up, 8 in 2024-04-07T16:23:12.434 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:12 smithi100 bash[76202]: cluster 2024-04-07T16:23:10.669745+0000 mgr.y (mgr.25175) 162 : cluster [DBG] pgmap v249: 121 pgs: 10 active+undersized+degraded, 26 active+undersized, 85 active+clean; 266 MiB data, 1.7 GiB used, 714 GiB / 715 GiB avail; 5.1 MiB/s rd, 432 KiB/s wr, 6 op/s; 69/486 objects degraded (14.198%) 2024-04-07T16:23:12.434 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:12 smithi100 bash[76202]: cluster 2024-04-07T16:23:11.084767+0000 mon.a (mon.0) 505 : cluster [DBG] osdmap e440: 8 total, 7 up, 8 in 2024-04-07T16:23:12.434 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:12 smithi100 bash[76202]: cluster 2024-04-07T16:23:11.811091+0000 mon.a (mon.0) 506 : cluster [DBG] osdmap e441: 8 total, 7 up, 8 in 2024-04-07T16:23:12.435 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:12 smithi100 bash[74942]: cluster 2024-04-07T16:23:10.669745+0000 mgr.y (mgr.25175) 162 : cluster [DBG] pgmap v249: 121 pgs: 10 active+undersized+degraded, 26 active+undersized, 85 active+clean; 266 MiB data, 1.7 GiB used, 714 GiB / 715 GiB avail; 5.1 MiB/s rd, 432 KiB/s wr, 6 op/s; 69/486 objects degraded (14.198%) 2024-04-07T16:23:12.435 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:12 smithi100 bash[74942]: cluster 2024-04-07T16:23:11.084767+0000 mon.a (mon.0) 505 : cluster [DBG] osdmap e440: 8 total, 7 up, 8 in 2024-04-07T16:23:12.435 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:12 smithi100 bash[74942]: cluster 2024-04-07T16:23:11.811091+0000 mon.a (mon.0) 506 : cluster [DBG] osdmap e441: 8 total, 7 up, 8 in 2024-04-07T16:23:12.798 INFO:tasks.rados.rados.0.smithi100.stdout:1995: done (1 left) 2024-04-07T16:23:12.798 INFO:tasks.rados.rados.0.smithi100.stdout:1996: snap_create 2024-04-07T16:23:12.798 INFO:tasks.rados.rados.0.smithi100.stdout:1993: expect (ObjNum 575 snap 177 seq_num 575) 2024-04-07T16:23:13.606 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:23:13 smithi144 bash[59794]: ts=2024-04-07T16:23:13.508Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=nodes name=CephNodeDiskspaceWarning index=4 msg="Evaluating rule failed" rule="alert: CephNodeDiskspaceWarning\nexpr: predict_linear(node_filesystem_free_bytes{device=~\"/.*\"}[2d], 3600 * 24 * 5)\n * on (instance) group_left (nodename) node_uname_info < 0\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.8.4\n severity: warning\n type: ceph_default\nannotations:\n description: Mountpoint {{ $labels.mountpoint }} on {{ $labels.nodename }} will\n be full in less than 5 days based on the 48 hour trailing fill rate.\n summary: Host filesystem free space is getting low\n" err="found duplicate series for the match group {instance=\"smithi100\"} on the right hand-side of the operation: [{__name__=\"node_uname_info\", domainname=\"(none)\", instance=\"smithi100\", job=\"node\", machine=\"x86_64\", nodename=\"smithi100\", release=\"5.15.0-101-generic\", sysname=\"Linux\", version=\"#111-Ubuntu SMP Tue Mar 5 20:16:58 UTC 2024\"}, {__name__=\"node_uname_info\", cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", domainname=\"(none)\", instance=\"smithi100\", job=\"node\", machine=\"x86_64\", nodename=\"smithi100\", release=\"5.15.0-101-generic\", sysname=\"Linux\", version=\"#111-Ubuntu SMP Tue Mar 5 20:16:58 UTC 2024\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:23:13.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:13 smithi144 bash[66654]: audit 2024-04-07T16:23:12.302674+0000 mon.a (mon.0) 507 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:23:13.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:13 smithi144 bash[66654]: audit 2024-04-07T16:23:12.308037+0000 mon.a (mon.0) 508 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:23:13.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:13 smithi144 bash[66654]: cluster 2024-04-07T16:23:12.816420+0000 mon.a (mon.0) 509 : cluster [DBG] osdmap e442: 8 total, 7 up, 8 in 2024-04-07T16:23:13.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:13 smithi144 bash[66654]: audit 2024-04-07T16:23:13.299334+0000 mon.a (mon.0) 510 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:23:13.684 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:13 smithi100 bash[74942]: audit 2024-04-07T16:23:12.302674+0000 mon.a (mon.0) 507 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:23:13.684 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:13 smithi100 bash[74942]: audit 2024-04-07T16:23:12.308037+0000 mon.a (mon.0) 508 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:23:13.684 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:13 smithi100 bash[74942]: cluster 2024-04-07T16:23:12.816420+0000 mon.a (mon.0) 509 : cluster [DBG] osdmap e442: 8 total, 7 up, 8 in 2024-04-07T16:23:13.684 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:13 smithi100 bash[74942]: audit 2024-04-07T16:23:13.299334+0000 mon.a (mon.0) 510 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:23:13.684 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:13 smithi100 bash[76202]: audit 2024-04-07T16:23:12.302674+0000 mon.a (mon.0) 507 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:23:13.684 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:13 smithi100 bash[76202]: audit 2024-04-07T16:23:12.308037+0000 mon.a (mon.0) 508 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:23:13.684 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:13 smithi100 bash[76202]: cluster 2024-04-07T16:23:12.816420+0000 mon.a (mon.0) 509 : cluster [DBG] osdmap e442: 8 total, 7 up, 8 in 2024-04-07T16:23:13.684 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:13 smithi100 bash[76202]: audit 2024-04-07T16:23:13.299334+0000 mon.a (mon.0) 510 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:23:13.799 INFO:tasks.rados.rados.0.smithi100.stdout:1993: done (1 left) 2024-04-07T16:23:13.800 INFO:tasks.rados.rados.0.smithi100.stdout:1996: done (0 left) 2024-04-07T16:23:13.800 INFO:tasks.rados.rados.0.smithi100.stdout:1997: rollback oid 39 current snap is 191 2024-04-07T16:23:13.800 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 39 to 174 2024-04-07T16:23:13.800 INFO:tasks.rados.rados.0.smithi100.stdout:1998: snap_create 2024-04-07T16:23:13.808 INFO:tasks.rados.rados.0.smithi100.stdout:1997: finishing rollback tid 1 to smithi10066867-39 2024-04-07T16:23:13.808 INFO:tasks.rados.rados.0.smithi100.stdout:1997: finishing rollback tid 2 to smithi10066867-39 2024-04-07T16:23:13.808 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 39 v 1078 (ObjNum 560 snap 171 seq_num 560) dirty exists 2024-04-07T16:23:14.184 INFO:journalctl@ceph.osd.2.smithi100.stdout:Apr 07 16:23:13 smithi100 bash[89668]: debug 2024-04-07T16:23:13.795+0000 7f353ab5c7c0 -1 osd.2 431 log_to_monitors true 2024-04-07T16:23:14.184 INFO:journalctl@ceph.osd.2.smithi100.stdout:Apr 07 16:23:13 smithi100 bash[89668]: debug 2024-04-07T16:23:13.847+0000 7f3527fac700 -1 osd.2 431 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-07T16:23:14.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:14 smithi144 bash[66654]: cluster 2024-04-07T16:23:12.670615+0000 mgr.y (mgr.25175) 163 : cluster [DBG] pgmap v252: 121 pgs: 10 active+undersized+degraded, 26 active+undersized, 85 active+clean; 267 MiB data, 1.7 GiB used, 714 GiB / 715 GiB avail; 6.7 MiB/s rd, 320 KiB/s wr, 4 op/s; 71/495 objects degraded (14.343%) 2024-04-07T16:23:14.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:14 smithi144 bash[66654]: cluster 2024-04-07T16:23:13.306716+0000 mon.a (mon.0) 511 : cluster [WRN] Health check update: Degraded data redundancy: 71/495 objects degraded (14.343%), 10 pgs degraded (PG_DEGRADED) 2024-04-07T16:23:14.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:14 smithi144 bash[66654]: audit 2024-04-07T16:23:13.315877+0000 mon.a (mon.0) 512 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:23:14.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:14 smithi144 bash[66654]: audit 2024-04-07T16:23:13.801787+0000 mon.c (mon.2) 9 : audit [INF] from='osd.2 [v2:172.21.15.100:6818/2767460830,v1:172.21.15.100:6819/2767460830]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-07T16:23:14.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:14 smithi144 bash[66654]: audit 2024-04-07T16:23:13.802107+0000 mon.a (mon.0) 513 : audit [INF] from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-07T16:23:14.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:14 smithi144 bash[66654]: audit 2024-04-07T16:23:13.813357+0000 mon.a (mon.0) 514 : audit [INF] from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-07T16:23:14.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:14 smithi144 bash[66654]: cluster 2024-04-07T16:23:13.817958+0000 mon.a (mon.0) 515 : cluster [DBG] osdmap e443: 8 total, 7 up, 8 in 2024-04-07T16:23:14.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:14 smithi144 bash[66654]: audit 2024-04-07T16:23:13.821400+0000 mon.c (mon.2) 10 : audit [INF] from='osd.2 [v2:172.21.15.100:6818/2767460830,v1:172.21.15.100:6819/2767460830]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-04-07T16:23:14.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:14 smithi144 bash[66654]: audit 2024-04-07T16:23:13.821715+0000 mon.a (mon.0) 516 : audit [INF] from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-04-07T16:23:14.684 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:14 smithi100 bash[76202]: cluster 2024-04-07T16:23:12.670615+0000 mgr.y (mgr.25175) 163 : cluster [DBG] pgmap v252: 121 pgs: 10 active+undersized+degraded, 26 active+undersized, 85 active+clean; 267 MiB data, 1.7 GiB used, 714 GiB / 715 GiB avail; 6.7 MiB/s rd, 320 KiB/s wr, 4 op/s; 71/495 objects degraded (14.343%) 2024-04-07T16:23:14.684 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:14 smithi100 bash[76202]: cluster 2024-04-07T16:23:13.306716+0000 mon.a (mon.0) 511 : cluster [WRN] Health check update: Degraded data redundancy: 71/495 objects degraded (14.343%), 10 pgs degraded (PG_DEGRADED) 2024-04-07T16:23:14.684 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:14 smithi100 bash[76202]: audit 2024-04-07T16:23:13.315877+0000 mon.a (mon.0) 512 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:23:14.685 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:14 smithi100 bash[76202]: audit 2024-04-07T16:23:13.801787+0000 mon.c (mon.2) 9 : audit [INF] from='osd.2 [v2:172.21.15.100:6818/2767460830,v1:172.21.15.100:6819/2767460830]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-07T16:23:14.685 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:14 smithi100 bash[76202]: audit 2024-04-07T16:23:13.802107+0000 mon.a (mon.0) 513 : audit [INF] from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-07T16:23:14.685 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:14 smithi100 bash[76202]: audit 2024-04-07T16:23:13.813357+0000 mon.a (mon.0) 514 : audit [INF] from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-07T16:23:14.685 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:14 smithi100 bash[76202]: cluster 2024-04-07T16:23:13.817958+0000 mon.a (mon.0) 515 : cluster [DBG] osdmap e443: 8 total, 7 up, 8 in 2024-04-07T16:23:14.685 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:14 smithi100 bash[76202]: audit 2024-04-07T16:23:13.821400+0000 mon.c (mon.2) 10 : audit [INF] from='osd.2 [v2:172.21.15.100:6818/2767460830,v1:172.21.15.100:6819/2767460830]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-04-07T16:23:14.685 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:14 smithi100 bash[76202]: audit 2024-04-07T16:23:13.821715+0000 mon.a (mon.0) 516 : audit [INF] from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-04-07T16:23:14.685 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:14 smithi100 bash[74942]: cluster 2024-04-07T16:23:12.670615+0000 mgr.y (mgr.25175) 163 : cluster [DBG] pgmap v252: 121 pgs: 10 active+undersized+degraded, 26 active+undersized, 85 active+clean; 267 MiB data, 1.7 GiB used, 714 GiB / 715 GiB avail; 6.7 MiB/s rd, 320 KiB/s wr, 4 op/s; 71/495 objects degraded (14.343%) 2024-04-07T16:23:14.685 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:14 smithi100 bash[74942]: cluster 2024-04-07T16:23:13.306716+0000 mon.a (mon.0) 511 : cluster [WRN] Health check update: Degraded data redundancy: 71/495 objects degraded (14.343%), 10 pgs degraded (PG_DEGRADED) 2024-04-07T16:23:14.685 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:14 smithi100 bash[74942]: audit 2024-04-07T16:23:13.315877+0000 mon.a (mon.0) 512 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:23:14.685 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:14 smithi100 bash[74942]: audit 2024-04-07T16:23:13.801787+0000 mon.c (mon.2) 9 : audit [INF] from='osd.2 [v2:172.21.15.100:6818/2767460830,v1:172.21.15.100:6819/2767460830]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-07T16:23:14.685 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:14 smithi100 bash[74942]: audit 2024-04-07T16:23:13.802107+0000 mon.a (mon.0) 513 : audit [INF] from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-07T16:23:14.685 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:14 smithi100 bash[74942]: audit 2024-04-07T16:23:13.813357+0000 mon.a (mon.0) 514 : audit [INF] from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-07T16:23:14.685 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:14 smithi100 bash[74942]: cluster 2024-04-07T16:23:13.817958+0000 mon.a (mon.0) 515 : cluster [DBG] osdmap e443: 8 total, 7 up, 8 in 2024-04-07T16:23:14.686 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:14 smithi100 bash[74942]: audit 2024-04-07T16:23:13.821400+0000 mon.c (mon.2) 10 : audit [INF] from='osd.2 [v2:172.21.15.100:6818/2767460830,v1:172.21.15.100:6819/2767460830]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-04-07T16:23:14.686 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:14 smithi100 bash[74942]: audit 2024-04-07T16:23:13.821715+0000 mon.a (mon.0) 516 : audit [INF] from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-04-07T16:23:14.809 INFO:tasks.rados.rados.0.smithi100.stdout:1997: done (1 left) 2024-04-07T16:23:14.809 INFO:tasks.rados.rados.0.smithi100.stdout:1998: done (0 left) 2024-04-07T16:23:14.809 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 28 current snap is 192 2024-04-07T16:23:14.809 INFO:tasks.rados.rados.0.smithi100.stdout:1999: seq_num 615 ranges {2154496=704512} 2024-04-07T16:23:14.858 INFO:tasks.rados.rados.0.smithi100.stdout:1999: writing smithi10066867-28 from 2154496 to 2859008 tid 1 2024-04-07T16:23:14.858 INFO:tasks.rados.rados.0.smithi100.stdout:2000: snap_create 2024-04-07T16:23:15.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:15 smithi144 bash[66654]: cluster 2024-04-07T16:23:14.814728+0000 mon.a (mon.0) 517 : cluster [INF] Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-07T16:23:15.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:15 smithi144 bash[66654]: cluster 2024-04-07T16:23:14.826071+0000 mon.a (mon.0) 518 : cluster [INF] osd.2 [v2:172.21.15.100:6818/2767460830,v1:172.21.15.100:6819/2767460830] boot 2024-04-07T16:23:15.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:15 smithi144 bash[66654]: cluster 2024-04-07T16:23:14.826181+0000 mon.a (mon.0) 519 : cluster [DBG] osdmap e444: 8 total, 8 up, 8 in 2024-04-07T16:23:15.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:15 smithi144 bash[66654]: audit 2024-04-07T16:23:14.826775+0000 mon.a (mon.0) 520 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-07T16:23:15.684 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:15 smithi100 bash[76202]: cluster 2024-04-07T16:23:14.814728+0000 mon.a (mon.0) 517 : cluster [INF] Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-07T16:23:15.684 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:15 smithi100 bash[76202]: cluster 2024-04-07T16:23:14.826071+0000 mon.a (mon.0) 518 : cluster [INF] osd.2 [v2:172.21.15.100:6818/2767460830,v1:172.21.15.100:6819/2767460830] boot 2024-04-07T16:23:15.685 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:15 smithi100 bash[76202]: cluster 2024-04-07T16:23:14.826181+0000 mon.a (mon.0) 519 : cluster [DBG] osdmap e444: 8 total, 8 up, 8 in 2024-04-07T16:23:15.685 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:15 smithi100 bash[76202]: audit 2024-04-07T16:23:14.826775+0000 mon.a (mon.0) 520 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-07T16:23:15.685 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:15 smithi100 bash[74942]: cluster 2024-04-07T16:23:14.814728+0000 mon.a (mon.0) 517 : cluster [INF] Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-07T16:23:15.685 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:15 smithi100 bash[74942]: cluster 2024-04-07T16:23:14.826071+0000 mon.a (mon.0) 518 : cluster [INF] osd.2 [v2:172.21.15.100:6818/2767460830,v1:172.21.15.100:6819/2767460830] boot 2024-04-07T16:23:15.685 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:15 smithi100 bash[74942]: cluster 2024-04-07T16:23:14.826181+0000 mon.a (mon.0) 519 : cluster [DBG] osdmap e444: 8 total, 8 up, 8 in 2024-04-07T16:23:15.685 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:15 smithi100 bash[74942]: audit 2024-04-07T16:23:14.826775+0000 mon.a (mon.0) 520 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-07T16:23:15.810 INFO:tasks.rados.rados.0.smithi100.stdout:2000: done (1 left) 2024-04-07T16:23:15.810 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 1 current snap is 193 2024-04-07T16:23:15.810 INFO:tasks.rados.rados.0.smithi100.stdout:2001: seq_num 616 ranges {0=720896} 2024-04-07T16:23:15.818 INFO:tasks.rados.rados.0.smithi100.stdout:2001: writing smithi10066867-1 from 0 to 720896 tid 1 2024-04-07T16:23:15.818 INFO:tasks.rados.rados.0.smithi100.stdout:2002: copy_from oid 39 from oid 34 current snap is 193 2024-04-07T16:23:15.819 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 7 current snap is 193 2024-04-07T16:23:15.819 INFO:tasks.rados.rados.0.smithi100.stdout:2003: seq_num 617 ranges {5685248=737280} 2024-04-07T16:23:15.874 INFO:tasks.rados.rados.0.smithi100.stdout:2003: writing smithi10066867-7 from 5685248 to 6422528 tid 1 2024-04-07T16:23:15.874 INFO:tasks.rados.rados.0.smithi100.stdout:2004: copy_from oid 16 from oid 3 current snap is 193 2024-04-07T16:23:15.874 INFO:tasks.rados.rados.0.smithi100.stdout:2001: finishing write tid 1 to smithi10066867-1 2024-04-07T16:23:15.874 INFO:tasks.rados.rados.0.smithi100.stdout:2001: finishing write tid 2 to smithi10066867-1 2024-04-07T16:23:15.874 INFO:tasks.rados.rados.0.smithi100.stdout:2001: finishing write tid 3 to smithi10066867-1 2024-04-07T16:23:15.875 INFO:tasks.rados.rados.0.smithi100.stdout:2001: oid 1 updating version 0 to 986 2024-04-07T16:23:15.875 INFO:tasks.rados.rados.0.smithi100.stdout:2001: oid 1 version 986 is already newer than 985 2024-04-07T16:23:15.875 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 1 v 986 (ObjNum 616 snap 193 seq_num 616) dirty exists 2024-04-07T16:23:15.875 INFO:tasks.rados.rados.0.smithi100.stdout:2001: left oid 1 (ObjNum 616 snap 193 seq_num 616) 2024-04-07T16:23:15.875 INFO:tasks.rados.rados.0.smithi100.stdout:2002: finishing copy_from racing read to smithi10066867-39 2024-04-07T16:23:15.875 INFO:tasks.rados.rados.0.smithi100.stdout:2001: done (4 left) 2024-04-07T16:23:15.875 INFO:tasks.rados.rados.0.smithi100.stdout:2005: snap_create 2024-04-07T16:23:15.875 INFO:tasks.rados.rados.0.smithi100.stdout:2002: finishing copy_from to smithi10066867-39 2024-04-07T16:23:15.875 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 39 v 552 (ObjNum 483 snap 144 seq_num 483) dirty exists 2024-04-07T16:23:15.875 INFO:tasks.rados.rados.0.smithi100.stdout:1999: finishing write tid 1 to smithi10066867-28 2024-04-07T16:23:15.875 INFO:tasks.rados.rados.0.smithi100.stdout:1999: finishing write tid 2 to smithi10066867-28 2024-04-07T16:23:15.875 INFO:tasks.rados.rados.0.smithi100.stdout:1999: finishing write tid 3 to smithi10066867-28 2024-04-07T16:23:15.875 INFO:tasks.rados.rados.0.smithi100.stdout:1999: oid 28 updating version 0 to 775 2024-04-07T16:23:15.875 INFO:tasks.rados.rados.0.smithi100.stdout:1999: oid 28 version 775 is already newer than 774 2024-04-07T16:23:15.875 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 28 v 775 (ObjNum 615 snap 192 seq_num 615) dirty exists 2024-04-07T16:23:15.875 INFO:tasks.rados.rados.0.smithi100.stdout:1999: left oid 28 (ObjNum 615 snap 192 seq_num 615) 2024-04-07T16:23:15.881 INFO:tasks.rados.rados.0.smithi100.stdout:2003: finishing write tid 1 to smithi10066867-7 2024-04-07T16:23:15.881 INFO:tasks.rados.rados.0.smithi100.stdout:2003: finishing write tid 2 to smithi10066867-7 2024-04-07T16:23:15.884 INFO:tasks.rados.rados.0.smithi100.stdout:2003: finishing write tid 3 to smithi10066867-7 2024-04-07T16:23:15.884 INFO:tasks.rados.rados.0.smithi100.stdout:2003: oid 7 updating version 0 to 1075 2024-04-07T16:23:15.884 INFO:tasks.rados.rados.0.smithi100.stdout:2003: oid 7 updating version 1075 to 1076 2024-04-07T16:23:15.884 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 7 v 1076 (ObjNum 617 snap 193 seq_num 617) dirty exists 2024-04-07T16:23:15.884 INFO:tasks.rados.rados.0.smithi100.stdout:2003: left oid 7 (ObjNum 617 snap 193 seq_num 617) 2024-04-07T16:23:15.897 INFO:tasks.rados.rados.0.smithi100.stdout:2004: finishing copy_from racing read to smithi10066867-16 2024-04-07T16:23:15.899 INFO:tasks.rados.rados.0.smithi100.stdout:2004: finishing copy_from to smithi10066867-16 2024-04-07T16:23:15.899 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 16 v 1083 (ObjNum 577 snap 178 seq_num 577) dirty exists 2024-04-07T16:23:16.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:16 smithi144 bash[66654]: cluster 2024-04-07T16:23:14.671450+0000 mgr.y (mgr.25175) 164 : cluster [DBG] pgmap v255: 121 pgs: 10 active+undersized+degraded, 26 active+undersized, 85 active+clean; 267 MiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 3.8 MiB/s rd, 320 KiB/s wr, 4 op/s; 71/492 objects degraded (14.431%) 2024-04-07T16:23:16.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:16 smithi144 bash[66654]: cluster 2024-04-07T16:23:15.827819+0000 mon.a (mon.0) 521 : cluster [DBG] osdmap e445: 8 total, 8 up, 8 in 2024-04-07T16:23:16.684 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:16 smithi100 bash[74942]: cluster 2024-04-07T16:23:14.671450+0000 mgr.y (mgr.25175) 164 : cluster [DBG] pgmap v255: 121 pgs: 10 active+undersized+degraded, 26 active+undersized, 85 active+clean; 267 MiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 3.8 MiB/s rd, 320 KiB/s wr, 4 op/s; 71/492 objects degraded (14.431%) 2024-04-07T16:23:16.684 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:16 smithi100 bash[74942]: cluster 2024-04-07T16:23:15.827819+0000 mon.a (mon.0) 521 : cluster [DBG] osdmap e445: 8 total, 8 up, 8 in 2024-04-07T16:23:16.684 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:16 smithi100 bash[76202]: cluster 2024-04-07T16:23:14.671450+0000 mgr.y (mgr.25175) 164 : cluster [DBG] pgmap v255: 121 pgs: 10 active+undersized+degraded, 26 active+undersized, 85 active+clean; 267 MiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 3.8 MiB/s rd, 320 KiB/s wr, 4 op/s; 71/492 objects degraded (14.431%) 2024-04-07T16:23:16.684 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:16 smithi100 bash[76202]: cluster 2024-04-07T16:23:15.827819+0000 mon.a (mon.0) 521 : cluster [DBG] osdmap e445: 8 total, 8 up, 8 in 2024-04-07T16:23:16.813 INFO:tasks.rados.rados.0.smithi100.stdout:1999: done (4 left) 2024-04-07T16:23:16.813 INFO:tasks.rados.rados.0.smithi100.stdout:2002: done (3 left) 2024-04-07T16:23:16.813 INFO:tasks.rados.rados.0.smithi100.stdout:2003: done (2 left) 2024-04-07T16:23:16.813 INFO:tasks.rados.rados.0.smithi100.stdout:2004: done (1 left) 2024-04-07T16:23:16.813 INFO:tasks.rados.rados.0.smithi100.stdout:2005: done (0 left) 2024-04-07T16:23:16.813 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 43 current snap is 194 2024-04-07T16:23:16.813 INFO:tasks.rados.rados.0.smithi100.stdout:2006: seq_num 618 ranges {5857280=753664} 2024-04-07T16:23:16.861 INFO:tasks.rados.rados.0.smithi100.stdout:2006: writing smithi10066867-43 from 5857280 to 6610944 tid 1 2024-04-07T16:23:16.861 INFO:tasks.rados.rados.0.smithi100.stdout:2007: delete oid 18 current snap is 194 2024-04-07T16:23:16.874 INFO:tasks.rados.rados.0.smithi100.stdout:2006: finishing write tid 1 to smithi10066867-43 2024-04-07T16:23:16.874 INFO:tasks.rados.rados.0.smithi100.stdout:2006: finishing write tid 2 to smithi10066867-43 2024-04-07T16:23:16.877 INFO:tasks.rados.rados.0.smithi100.stdout:2006: finishing write tid 3 to smithi10066867-43 2024-04-07T16:23:16.877 INFO:tasks.rados.rados.0.smithi100.stdout:2006: oid 43 updating version 0 to 1078 2024-04-07T16:23:16.877 INFO:tasks.rados.rados.0.smithi100.stdout:2006: oid 43 version 1078 is already newer than 1077 2024-04-07T16:23:16.877 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 43 v 1078 (ObjNum 618 snap 194 seq_num 618) dirty exists 2024-04-07T16:23:16.877 INFO:tasks.rados.rados.0.smithi100.stdout:2006: left oid 43 (ObjNum 618 snap 194 seq_num 618) 2024-04-07T16:23:16.880 INFO:tasks.rados.rados.0.smithi100.stdout:2006: done (1 left) 2024-04-07T16:23:16.880 INFO:tasks.rados.rados.0.smithi100.stdout:2007: done (0 left) 2024-04-07T16:23:16.880 INFO:tasks.rados.rados.0.smithi100.stdout:2008: read oid 36 snap -1 2024-04-07T16:23:16.880 INFO:tasks.rados.rados.0.smithi100.stdout:2008: expect (ObjNum 492 snap 148 seq_num 492) 2024-04-07T16:23:16.880 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 18 current snap is 194 2024-04-07T16:23:16.880 INFO:tasks.rados.rados.0.smithi100.stdout:2009: seq_num 620 ranges {0=794624} 2024-04-07T16:23:16.887 INFO:tasks.rados.rados.0.smithi100.stdout:2009: writing smithi10066867-18 from 0 to 794624 tid 1 2024-04-07T16:23:16.887 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 44 current snap is 194 2024-04-07T16:23:16.887 INFO:tasks.rados.rados.0.smithi100.stdout:2010: seq_num 621 ranges {5980160=409600,6389760=401408} 2024-04-07T16:23:16.930 INFO:tasks.rados.rados.0.smithi100.stdout:2010: writing smithi10066867-44 from 5980160 to 6389760 tid 1 2024-04-07T16:23:16.932 INFO:tasks.rados.rados.0.smithi100.stdout:2010: writing smithi10066867-44 from 6389760 to 6791168 tid 2 2024-04-07T16:23:16.932 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 9 current snap is 194 2024-04-07T16:23:16.933 INFO:tasks.rados.rados.0.smithi100.stdout:2011: seq_num 622 ranges {671744=425984,1097728=401408} 2024-04-07T16:23:16.939 INFO:tasks.rados.rados.0.smithi100.stdout:2011: writing smithi10066867-9 from 671744 to 1097728 tid 1 2024-04-07T16:23:16.941 INFO:tasks.rados.rados.0.smithi100.stdout:2011: writing smithi10066867-9 from 1097728 to 1499136 tid 2 2024-04-07T16:23:16.942 INFO:tasks.rados.rados.0.smithi100.stdout:2012: read oid 10 snap -1 2024-04-07T16:23:16.942 INFO:tasks.rados.rados.0.smithi100.stdout:2012: expect deleted 2024-04-07T16:23:16.942 INFO:tasks.rados.rados.0.smithi100.stdout:2013: snap_remove snap 187 2024-04-07T16:23:17.818 INFO:tasks.rados.rados.0.smithi100.stdout:2013: done (5 left) 2024-04-07T16:23:17.818 INFO:tasks.rados.rados.0.smithi100.stdout:2014: rollback oid 16 current snap is 194 2024-04-07T16:23:17.818 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 16 to 190 2024-04-07T16:23:17.818 INFO:tasks.rados.rados.0.smithi100.stdout:2008: expect (ObjNum 492 snap 148 seq_num 492) 2024-04-07T16:23:17.925 INFO:tasks.rados.rados.0.smithi100.stdout:2009: finishing write tid 1 to smithi10066867-18 2024-04-07T16:23:17.925 INFO:tasks.rados.rados.0.smithi100.stdout:2009: finishing write tid 2 to smithi10066867-18 2024-04-07T16:23:17.925 INFO:tasks.rados.rados.0.smithi100.stdout:2009: finishing write tid 3 to smithi10066867-18 2024-04-07T16:23:17.925 INFO:tasks.rados.rados.0.smithi100.stdout:2009: oid 18 updating version 0 to 1156 2024-04-07T16:23:17.926 INFO:tasks.rados.rados.0.smithi100.stdout:2009: oid 18 version 1156 is already newer than 1155 2024-04-07T16:23:17.926 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 18 v 1156 (ObjNum 620 snap 194 seq_num 620) dirty exists 2024-04-07T16:23:17.926 INFO:tasks.rados.rados.0.smithi100.stdout:2009: left oid 18 (ObjNum 620 snap 194 seq_num 620) 2024-04-07T16:23:17.926 INFO:tasks.rados.rados.0.smithi100.stdout:2010: finishing write tid 1 to smithi10066867-44 2024-04-07T16:23:17.926 INFO:tasks.rados.rados.0.smithi100.stdout:2010: finishing write tid 2 to smithi10066867-44 2024-04-07T16:23:17.926 INFO:tasks.rados.rados.0.smithi100.stdout:2010: finishing write tid 3 to smithi10066867-44 2024-04-07T16:23:17.926 INFO:tasks.rados.rados.0.smithi100.stdout:2010: finishing write tid 4 to smithi10066867-44 2024-04-07T16:23:17.926 INFO:tasks.rados.rados.0.smithi100.stdout:2010: oid 44 updating version 0 to 1098 2024-04-07T16:23:17.926 INFO:tasks.rados.rados.0.smithi100.stdout:2010: oid 44 version 1098 is already newer than 1096 2024-04-07T16:23:17.926 INFO:tasks.rados.rados.0.smithi100.stdout:2010: oid 44 version 1098 is already newer than 1097 2024-04-07T16:23:17.926 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 44 v 1098 (ObjNum 621 snap 194 seq_num 621) dirty exists 2024-04-07T16:23:17.926 INFO:tasks.rados.rados.0.smithi100.stdout:2010: left oid 44 (ObjNum 621 snap 194 seq_num 621) 2024-04-07T16:23:17.927 INFO:tasks.rados.rados.0.smithi100.stdout:2011: finishing write tid 1 to smithi10066867-9 2024-04-07T16:23:17.927 INFO:tasks.rados.rados.0.smithi100.stdout:2011: finishing write tid 2 to smithi10066867-9 2024-04-07T16:23:17.927 INFO:tasks.rados.rados.0.smithi100.stdout:2011: finishing write tid 3 to smithi10066867-9 2024-04-07T16:23:17.927 INFO:tasks.rados.rados.0.smithi100.stdout:2011: finishing write tid 4 to smithi10066867-9 2024-04-07T16:23:17.927 INFO:tasks.rados.rados.0.smithi100.stdout:2011: oid 9 updating version 0 to 552 2024-04-07T16:23:17.927 INFO:tasks.rados.rados.0.smithi100.stdout:2011: oid 9 updating version 552 to 553 2024-04-07T16:23:17.927 INFO:tasks.rados.rados.0.smithi100.stdout:2011: oid 9 updating version 553 to 554 2024-04-07T16:23:17.927 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 9 v 554 (ObjNum 622 snap 194 seq_num 622) dirty exists 2024-04-07T16:23:17.927 INFO:tasks.rados.rados.0.smithi100.stdout:2011: left oid 9 (ObjNum 622 snap 194 seq_num 622) 2024-04-07T16:23:17.927 INFO:tasks.rados.rados.0.smithi100.stdout:2014: finishing rollback tid 0 to smithi10066867-16 2024-04-07T16:23:17.927 INFO:tasks.rados.rados.0.smithi100.stdout:2014: finishing rollback tid 1 to smithi10066867-16 2024-04-07T16:23:17.927 INFO:tasks.rados.rados.0.smithi100.stdout:2008: done (5 left) 2024-04-07T16:23:17.927 INFO:tasks.rados.rados.0.smithi100.stdout:2009: done (4 left) 2024-04-07T16:23:17.927 INFO:tasks.rados.rados.0.smithi100.stdout:2010: done (3 left) 2024-04-07T16:23:17.927 INFO:tasks.rados.rados.0.smithi100.stdout:2011: done (2 left) 2024-04-07T16:23:17.928 INFO:tasks.rados.rados.0.smithi100.stdout:2012: done (1 left) 2024-04-07T16:23:17.928 INFO:tasks.rados.rados.0.smithi100.stdout:2015: read oid 32 snap -1 2024-04-07T16:23:17.928 INFO:tasks.rados.rados.0.smithi100.stdout:2015: expect (ObjNum 589 snap 182 seq_num 589) 2024-04-07T16:23:17.928 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 17 current snap is 194 2024-04-07T16:23:17.928 INFO:tasks.rados.rados.0.smithi100.stdout:2014: finishing rollback tid 2 to smithi10066867-16 2024-04-07T16:23:17.928 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 16 v 1086 (ObjNum 606 snap 186 seq_num 606) dirty exists 2024-04-07T16:23:17.928 INFO:tasks.rados.rados.0.smithi100.stdout:2016: seq_num 623 ranges {0=442368,442368=401408} 2024-04-07T16:23:17.931 INFO:tasks.rados.rados.0.smithi100.stdout:2016: writing smithi10066867-17 from 0 to 442368 tid 1 2024-04-07T16:23:17.935 INFO:tasks.rados.rados.0.smithi100.stdout:2016: writing smithi10066867-17 from 442368 to 843776 tid 2 2024-04-07T16:23:17.935 INFO:tasks.rados.rados.0.smithi100.stdout:2014: done (2 left) 2024-04-07T16:23:17.935 INFO:tasks.rados.rados.0.smithi100.stdout:2017: delete oid 24 current snap is 194 2024-04-07T16:23:17.941 INFO:tasks.rados.rados.0.smithi100.stdout:2016: finishing write tid 1 to smithi10066867-17 2024-04-07T16:23:17.942 INFO:tasks.rados.rados.0.smithi100.stdout:2016: finishing write tid 2 to smithi10066867-17 2024-04-07T16:23:17.943 INFO:tasks.rados.rados.0.smithi100.stdout:2016: finishing write tid 3 to smithi10066867-17 2024-04-07T16:23:17.946 INFO:tasks.rados.rados.0.smithi100.stdout:2017: done (2 left) 2024-04-07T16:23:17.946 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 2 current snap is 194 2024-04-07T16:23:17.947 INFO:tasks.rados.rados.0.smithi100.stdout:2018: seq_num 625 ranges {0=475136,475136=401408} 2024-04-07T16:23:17.952 INFO:tasks.rados.rados.0.smithi100.stdout:2018: writing smithi10066867-2 from 0 to 475136 tid 1 2024-04-07T16:23:17.956 INFO:tasks.rados.rados.0.smithi100.stdout:2018: writing smithi10066867-2 from 475136 to 876544 tid 2 2024-04-07T16:23:17.956 INFO:tasks.rados.rados.0.smithi100.stdout:2019: read oid 37 snap -1 2024-04-07T16:23:17.957 INFO:tasks.rados.rados.0.smithi100.stdout:2019: expect deleted 2024-04-07T16:23:17.957 INFO:tasks.rados.rados.0.smithi100.stdout:2020: setattr oid 1 current snap is 194 2024-04-07T16:23:17.957 INFO:tasks.rados.rados.0.smithi100.stdout:2016: finishing write tid 4 to smithi10066867-17 2024-04-07T16:23:17.957 INFO:tasks.rados.rados.0.smithi100.stdout:2016: oid 17 updating version 0 to 744 2024-04-07T16:23:17.957 INFO:tasks.rados.rados.0.smithi100.stdout:2016: oid 17 updating version 744 to 745 2024-04-07T16:23:17.957 INFO:tasks.rados.rados.0.smithi100.stdout:2016: oid 17 updating version 745 to 746 2024-04-07T16:23:17.957 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 17 v 746 (ObjNum 623 snap 194 seq_num 623) dirty exists 2024-04-07T16:23:17.957 INFO:tasks.rados.rados.0.smithi100.stdout:2016: left oid 17 (ObjNum 623 snap 194 seq_num 623) 2024-04-07T16:23:17.959 INFO:tasks.rados.rados.0.smithi100.stdout:2016: done (4 left) 2024-04-07T16:23:17.959 INFO:tasks.rados.rados.0.smithi100.stdout:2019: done (3 left) 2024-04-07T16:23:17.959 INFO:tasks.rados.rados.0.smithi100.stdout:2021: delete oid 14 current snap is 194 2024-04-07T16:23:17.960 INFO:tasks.rados.rados.0.smithi100.stdout:2015: expect (ObjNum 589 snap 182 seq_num 589) 2024-04-07T16:23:18.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:17 smithi144 bash[66654]: cluster 2024-04-07T16:23:16.672289+0000 mgr.y (mgr.25175) 165 : cluster [DBG] pgmap v258: 121 pgs: 4 peering, 10 active+undersized+degraded, 22 active+undersized, 85 active+clean; 267 MiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 1.2 MiB/s rd, 0 op/s; 71/492 objects degraded (14.431%) 2024-04-07T16:23:18.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:17 smithi144 bash[66654]: audit 2024-04-07T16:23:16.720266+0000 mon.a (mon.0) 522 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:23:18.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:17 smithi144 bash[66654]: audit 2024-04-07T16:23:16.728291+0000 mon.a (mon.0) 523 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:23:18.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:17 smithi144 bash[66654]: audit 2024-04-07T16:23:16.820436+0000 mon.a (mon.0) 524 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:23:18.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:17 smithi144 bash[66654]: cluster 2024-04-07T16:23:16.828224+0000 mon.a (mon.0) 525 : cluster [DBG] osdmap e446: 8 total, 8 up, 8 in 2024-04-07T16:23:18.184 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:17 smithi100 bash[74942]: cluster 2024-04-07T16:23:16.672289+0000 mgr.y (mgr.25175) 165 : cluster [DBG] pgmap v258: 121 pgs: 4 peering, 10 active+undersized+degraded, 22 active+undersized, 85 active+clean; 267 MiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 1.2 MiB/s rd, 0 op/s; 71/492 objects degraded (14.431%) 2024-04-07T16:23:18.184 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:17 smithi100 bash[74942]: audit 2024-04-07T16:23:16.720266+0000 mon.a (mon.0) 522 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:23:18.184 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:17 smithi100 bash[74942]: audit 2024-04-07T16:23:16.728291+0000 mon.a (mon.0) 523 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:23:18.184 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:17 smithi100 bash[74942]: audit 2024-04-07T16:23:16.820436+0000 mon.a (mon.0) 524 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:23:18.184 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:17 smithi100 bash[74942]: cluster 2024-04-07T16:23:16.828224+0000 mon.a (mon.0) 525 : cluster [DBG] osdmap e446: 8 total, 8 up, 8 in 2024-04-07T16:23:18.184 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:17 smithi100 bash[76202]: cluster 2024-04-07T16:23:16.672289+0000 mgr.y (mgr.25175) 165 : cluster [DBG] pgmap v258: 121 pgs: 4 peering, 10 active+undersized+degraded, 22 active+undersized, 85 active+clean; 267 MiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 1.2 MiB/s rd, 0 op/s; 71/492 objects degraded (14.431%) 2024-04-07T16:23:18.184 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:17 smithi100 bash[76202]: audit 2024-04-07T16:23:16.720266+0000 mon.a (mon.0) 522 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:23:18.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:17 smithi100 bash[76202]: audit 2024-04-07T16:23:16.728291+0000 mon.a (mon.0) 523 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:23:18.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:17 smithi100 bash[76202]: audit 2024-04-07T16:23:16.820436+0000 mon.a (mon.0) 524 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:23:18.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:17 smithi100 bash[76202]: cluster 2024-04-07T16:23:16.828224+0000 mon.a (mon.0) 525 : cluster [DBG] osdmap e446: 8 total, 8 up, 8 in 2024-04-07T16:23:18.266 INFO:tasks.rados.rados.0.smithi100.stdout:2018: finishing write tid 1 to smithi10066867-2 2024-04-07T16:23:18.266 INFO:tasks.rados.rados.0.smithi100.stdout:2018: finishing write tid 2 to smithi10066867-2 2024-04-07T16:23:18.266 INFO:tasks.rados.rados.0.smithi100.stdout:2018: finishing write tid 3 to smithi10066867-2 2024-04-07T16:23:18.266 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 1 v 987 (ObjNum 616 snap 193 seq_num 616) dirty exists 2024-04-07T16:23:18.266 INFO:tasks.rados.rados.0.smithi100.stdout:2018: finishing write tid 4 to smithi10066867-2 2024-04-07T16:23:18.266 INFO:tasks.rados.rados.0.smithi100.stdout:2018: oid 2 updating version 0 to 1100 2024-04-07T16:23:18.267 INFO:tasks.rados.rados.0.smithi100.stdout:2018: oid 2 version 1100 is already newer than 1099 2024-04-07T16:23:18.267 INFO:tasks.rados.rados.0.smithi100.stdout:2018: oid 2 updating version 1100 to 1101 2024-04-07T16:23:18.267 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 2 v 1101 (ObjNum 625 snap 194 seq_num 625) dirty exists 2024-04-07T16:23:18.267 INFO:tasks.rados.rados.0.smithi100.stdout:2018: left oid 2 (ObjNum 625 snap 194 seq_num 625) 2024-04-07T16:23:18.267 INFO:tasks.rados.rados.0.smithi100.stdout:2015: done (3 left) 2024-04-07T16:23:18.267 INFO:tasks.rados.rados.0.smithi100.stdout:2018: done (2 left) 2024-04-07T16:23:18.267 INFO:tasks.rados.rados.0.smithi100.stdout:2020: done (1 left) 2024-04-07T16:23:18.267 INFO:tasks.rados.rados.0.smithi100.stdout:2021: done (0 left) 2024-04-07T16:23:18.267 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 10 current snap is 194 2024-04-07T16:23:18.267 INFO:tasks.rados.rados.0.smithi100.stdout:2022: seq_num 627 ranges {0=507904,507904=401408} 2024-04-07T16:23:18.272 INFO:tasks.rados.rados.0.smithi100.stdout:2022: writing smithi10066867-10 from 0 to 507904 tid 1 2024-04-07T16:23:18.280 INFO:tasks.rados.rados.0.smithi100.stdout:2022: writing smithi10066867-10 from 507904 to 909312 tid 2 2024-04-07T16:23:18.280 INFO:tasks.rados.rados.0.smithi100.stdout:2022: finishing write tid 1 to smithi10066867-10 2024-04-07T16:23:18.280 INFO:tasks.rados.rados.0.smithi100.stdout:2023: read oid 15 snap -1 2024-04-07T16:23:18.280 INFO:tasks.rados.rados.0.smithi100.stdout:2023: expect deleted 2024-04-07T16:23:18.280 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 13 current snap is 194 2024-04-07T16:23:18.280 INFO:tasks.rados.rados.0.smithi100.stdout:2024: seq_num 628 ranges {4923392=524288,5447680=401408} 2024-04-07T16:23:18.312 INFO:tasks.rados.rados.0.smithi100.stdout:2024: writing smithi10066867-13 from 4923392 to 5447680 tid 1 2024-04-07T16:23:18.314 INFO:tasks.rados.rados.0.smithi100.stdout:2024: writing smithi10066867-13 from 5447680 to 5849088 tid 2 2024-04-07T16:23:18.315 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 22 current snap is 194 2024-04-07T16:23:18.315 INFO:tasks.rados.rados.0.smithi100.stdout:2025: seq_num 629 ranges {4251648=540672,4792320=401408} 2024-04-07T16:23:18.341 INFO:tasks.rados.rados.0.smithi100.stdout:2025: writing smithi10066867-22 from 4251648 to 4792320 tid 1 2024-04-07T16:23:18.344 INFO:tasks.rados.rados.0.smithi100.stdout:2025: writing smithi10066867-22 from 4792320 to 5193728 tid 2 2024-04-07T16:23:18.344 INFO:tasks.rados.rados.0.smithi100.stdout:2026: snap_create 2024-04-07T16:23:18.344 INFO:tasks.rados.rados.0.smithi100.stdout:2022: finishing write tid 2 to smithi10066867-10 2024-04-07T16:23:18.344 INFO:tasks.rados.rados.0.smithi100.stdout:2022: finishing write tid 3 to smithi10066867-10 2024-04-07T16:23:18.344 INFO:tasks.rados.rados.0.smithi100.stdout:2022: finishing write tid 4 to smithi10066867-10 2024-04-07T16:23:18.344 INFO:tasks.rados.rados.0.smithi100.stdout:2022: oid 10 updating version 0 to 990 2024-04-07T16:23:18.345 INFO:tasks.rados.rados.0.smithi100.stdout:2022: oid 10 version 990 is already newer than 988 2024-04-07T16:23:18.345 INFO:tasks.rados.rados.0.smithi100.stdout:2022: oid 10 version 990 is already newer than 989 2024-04-07T16:23:18.345 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 10 v 990 (ObjNum 627 snap 194 seq_num 627) dirty exists 2024-04-07T16:23:18.345 INFO:tasks.rados.rados.0.smithi100.stdout:2022: left oid 10 (ObjNum 627 snap 194 seq_num 627) 2024-04-07T16:23:18.345 INFO:tasks.rados.rados.0.smithi100.stdout:2024: finishing write tid 1 to smithi10066867-13 2024-04-07T16:23:18.345 INFO:tasks.rados.rados.0.smithi100.stdout:2024: finishing write tid 2 to smithi10066867-13 2024-04-07T16:23:18.345 INFO:tasks.rados.rados.0.smithi100.stdout:2024: finishing write tid 3 to smithi10066867-13 2024-04-07T16:23:18.345 INFO:tasks.rados.rados.0.smithi100.stdout:2024: finishing write tid 4 to smithi10066867-13 2024-04-07T16:23:18.345 INFO:tasks.rados.rados.0.smithi100.stdout:2024: oid 13 updating version 0 to 757 2024-04-07T16:23:18.345 INFO:tasks.rados.rados.0.smithi100.stdout:2024: oid 13 version 757 is already newer than 756 2024-04-07T16:23:18.345 INFO:tasks.rados.rados.0.smithi100.stdout:2024: oid 13 version 757 is already newer than 755 2024-04-07T16:23:18.345 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 13 v 757 (ObjNum 628 snap 194 seq_num 628) dirty exists 2024-04-07T16:23:18.345 INFO:tasks.rados.rados.0.smithi100.stdout:2024: left oid 13 (ObjNum 628 snap 194 seq_num 628) 2024-04-07T16:23:18.348 INFO:tasks.rados.rados.0.smithi100.stdout:2025: finishing write tid 1 to smithi10066867-22 2024-04-07T16:23:18.348 INFO:tasks.rados.rados.0.smithi100.stdout:2025: finishing write tid 2 to smithi10066867-22 2024-04-07T16:23:18.348 INFO:tasks.rados.rados.0.smithi100.stdout:2025: finishing write tid 3 to smithi10066867-22 2024-04-07T16:23:18.350 INFO:tasks.rados.rados.0.smithi100.stdout:2025: finishing write tid 4 to smithi10066867-22 2024-04-07T16:23:18.350 INFO:tasks.rados.rados.0.smithi100.stdout:2025: oid 22 updating version 0 to 863 2024-04-07T16:23:18.350 INFO:tasks.rados.rados.0.smithi100.stdout:2025: oid 22 updating version 863 to 864 2024-04-07T16:23:18.350 INFO:tasks.rados.rados.0.smithi100.stdout:2025: oid 22 updating version 864 to 865 2024-04-07T16:23:18.350 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 22 v 865 (ObjNum 629 snap 194 seq_num 629) dirty exists 2024-04-07T16:23:18.350 INFO:tasks.rados.rados.0.smithi100.stdout:2025: left oid 22 (ObjNum 629 snap 194 seq_num 629) 2024-04-07T16:23:18.684 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:23:18 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:23:18] "GET /metrics HTTP/1.1" 200 36065 "" "Prometheus/2.43.0" 2024-04-07T16:23:18.827 INFO:tasks.rados.rados.0.smithi100.stdout:2022: done (4 left) 2024-04-07T16:23:18.827 INFO:tasks.rados.rados.0.smithi100.stdout:2023: done (3 left) 2024-04-07T16:23:18.827 INFO:tasks.rados.rados.0.smithi100.stdout:2024: done (2 left) 2024-04-07T16:23:18.827 INFO:tasks.rados.rados.0.smithi100.stdout:2025: done (1 left) 2024-04-07T16:23:18.827 INFO:tasks.rados.rados.0.smithi100.stdout:2026: done (0 left) 2024-04-07T16:23:18.827 INFO:tasks.rados.rados.0.smithi100.stdout:2027: read oid 22 snap -1 2024-04-07T16:23:18.827 INFO:tasks.rados.rados.0.smithi100.stdout:2027: expect (ObjNum 629 snap 194 seq_num 629) 2024-04-07T16:23:18.827 INFO:tasks.rados.rados.0.smithi100.stdout:2028: snap_create 2024-04-07T16:23:18.861 INFO:tasks.rados.rados.0.smithi100.stdout:2027: expect (ObjNum 629 snap 194 seq_num 629) 2024-04-07T16:23:19.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:18 smithi144 bash[66654]: cluster 2024-04-07T16:23:17.835970+0000 mon.a (mon.0) 526 : cluster [DBG] osdmap e447: 8 total, 8 up, 8 in 2024-04-07T16:23:19.184 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:18 smithi100 bash[74942]: cluster 2024-04-07T16:23:17.835970+0000 mon.a (mon.0) 526 : cluster [DBG] osdmap e447: 8 total, 8 up, 8 in 2024-04-07T16:23:19.184 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:18 smithi100 bash[76202]: cluster 2024-04-07T16:23:17.835970+0000 mon.a (mon.0) 526 : cluster [DBG] osdmap e447: 8 total, 8 up, 8 in 2024-04-07T16:23:19.834 INFO:tasks.rados.rados.0.smithi100.stdout:2027: done (1 left) 2024-04-07T16:23:19.834 INFO:tasks.rados.rados.0.smithi100.stdout:2028: done (0 left) 2024-04-07T16:23:19.834 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 49 current snap is 196 2024-04-07T16:23:19.834 INFO:tasks.rados.rados.0.smithi100.stdout:2029: seq_num 630 ranges {417792=557056,974848=401408} 2024-04-07T16:23:19.841 INFO:tasks.rados.rados.0.smithi100.stdout:2029: writing smithi10066867-49 from 417792 to 974848 tid 1 2024-04-07T16:23:19.844 INFO:tasks.rados.rados.0.smithi100.stdout:2029: writing smithi10066867-49 from 974848 to 1376256 tid 2 2024-04-07T16:23:19.844 INFO:tasks.rados.rados.0.smithi100.stdout:2030: read oid 38 snap -1 2024-04-07T16:23:19.844 INFO:tasks.rados.rados.0.smithi100.stdout:2030: expect (ObjNum 600 snap 185 seq_num 600) 2024-04-07T16:23:19.844 INFO:tasks.rados.rados.0.smithi100.stdout:2031: delete oid 33 current snap is 196 2024-04-07T16:23:19.861 INFO:tasks.rados.rados.0.smithi100.stdout:2030: expect (ObjNum 600 snap 185 seq_num 600) 2024-04-07T16:23:19.947 INFO:tasks.rados.rados.0.smithi100.stdout:2029: finishing write tid 1 to smithi10066867-49 2024-04-07T16:23:19.947 INFO:tasks.rados.rados.0.smithi100.stdout:2029: finishing write tid 2 to smithi10066867-49 2024-04-07T16:23:19.947 INFO:tasks.rados.rados.0.smithi100.stdout:2029: finishing write tid 3 to smithi10066867-49 2024-04-07T16:23:19.947 INFO:tasks.rados.rados.0.smithi100.stdout:2029: finishing write tid 4 to smithi10066867-49 2024-04-07T16:23:19.948 INFO:tasks.rados.rados.0.smithi100.stdout:2029: oid 49 updating version 0 to 1087 2024-04-07T16:23:19.948 INFO:tasks.rados.rados.0.smithi100.stdout:2029: oid 49 updating version 1087 to 1089 2024-04-07T16:23:19.948 INFO:tasks.rados.rados.0.smithi100.stdout:2029: oid 49 version 1089 is already newer than 1088 2024-04-07T16:23:19.948 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 49 v 1089 (ObjNum 630 snap 196 seq_num 630) dirty exists 2024-04-07T16:23:19.948 INFO:tasks.rados.rados.0.smithi100.stdout:2029: left oid 49 (ObjNum 630 snap 196 seq_num 630) 2024-04-07T16:23:19.948 INFO:tasks.rados.rados.0.smithi100.stdout:2029: done (2 left) 2024-04-07T16:23:19.948 INFO:tasks.rados.rados.0.smithi100.stdout:2030: done (1 left) 2024-04-07T16:23:19.948 INFO:tasks.rados.rados.0.smithi100.stdout:2031: done (0 left) 2024-04-07T16:23:19.948 INFO:tasks.rados.rados.0.smithi100.stdout:2032: snap_create 2024-04-07T16:23:20.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:19 smithi144 bash[66654]: cluster 2024-04-07T16:23:18.673232+0000 mgr.y (mgr.25175) 166 : cluster [DBG] pgmap v261: 121 pgs: 4 peering, 7 active+undersized+degraded, 7 active+undersized, 103 active+clean; 283 MiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 1.5 MiB/s wr, 4 op/s; 64/510 objects degraded (12.549%) 2024-04-07T16:23:20.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:19 smithi144 bash[66654]: cluster 2024-04-07T16:23:18.827262+0000 mon.a (mon.0) 527 : cluster [WRN] Health check update: Degraded data redundancy: 64/510 objects degraded (12.549%), 7 pgs degraded (PG_DEGRADED) 2024-04-07T16:23:20.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:19 smithi144 bash[66654]: cluster 2024-04-07T16:23:18.845104+0000 mon.a (mon.0) 528 : cluster [DBG] osdmap e448: 8 total, 8 up, 8 in 2024-04-07T16:23:20.184 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:19 smithi100 bash[74942]: cluster 2024-04-07T16:23:18.673232+0000 mgr.y (mgr.25175) 166 : cluster [DBG] pgmap v261: 121 pgs: 4 peering, 7 active+undersized+degraded, 7 active+undersized, 103 active+clean; 283 MiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 1.5 MiB/s wr, 4 op/s; 64/510 objects degraded (12.549%) 2024-04-07T16:23:20.184 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:19 smithi100 bash[74942]: cluster 2024-04-07T16:23:18.827262+0000 mon.a (mon.0) 527 : cluster [WRN] Health check update: Degraded data redundancy: 64/510 objects degraded (12.549%), 7 pgs degraded (PG_DEGRADED) 2024-04-07T16:23:20.184 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:19 smithi100 bash[74942]: cluster 2024-04-07T16:23:18.845104+0000 mon.a (mon.0) 528 : cluster [DBG] osdmap e448: 8 total, 8 up, 8 in 2024-04-07T16:23:20.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:19 smithi100 bash[76202]: cluster 2024-04-07T16:23:18.673232+0000 mgr.y (mgr.25175) 166 : cluster [DBG] pgmap v261: 121 pgs: 4 peering, 7 active+undersized+degraded, 7 active+undersized, 103 active+clean; 283 MiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 1.5 MiB/s wr, 4 op/s; 64/510 objects degraded (12.549%) 2024-04-07T16:23:20.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:19 smithi100 bash[76202]: cluster 2024-04-07T16:23:18.827262+0000 mon.a (mon.0) 527 : cluster [WRN] Health check update: Degraded data redundancy: 64/510 objects degraded (12.549%), 7 pgs degraded (PG_DEGRADED) 2024-04-07T16:23:20.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:19 smithi100 bash[76202]: cluster 2024-04-07T16:23:18.845104+0000 mon.a (mon.0) 528 : cluster [DBG] osdmap e448: 8 total, 8 up, 8 in 2024-04-07T16:23:20.846 INFO:tasks.rados.rados.0.smithi100.stdout:2032: done (0 left) 2024-04-07T16:23:20.846 INFO:tasks.rados.rados.0.smithi100.stdout:2033: snap_create 2024-04-07T16:23:21.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:20 smithi144 bash[66654]: cluster 2024-04-07T16:23:19.852032+0000 mon.a (mon.0) 529 : cluster [DBG] osdmap e449: 8 total, 8 up, 8 in 2024-04-07T16:23:21.161 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:20 smithi100 bash[74942]: cluster 2024-04-07T16:23:19.852032+0000 mon.a (mon.0) 529 : cluster [DBG] osdmap e449: 8 total, 8 up, 8 in 2024-04-07T16:23:21.162 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:20 smithi100 bash[76202]: cluster 2024-04-07T16:23:19.852032+0000 mon.a (mon.0) 529 : cluster [DBG] osdmap e449: 8 total, 8 up, 8 in 2024-04-07T16:23:21.799 INFO:tasks.rados.rados.0.smithi100.stdout:2033: done (0 left) 2024-04-07T16:23:21.799 INFO:tasks.rados.rados.0.smithi100.stdout:2034: read oid 37 snap 197 2024-04-07T16:23:21.799 INFO:tasks.rados.rados.0.smithi100.stdout:2034: expect deleted 2024-04-07T16:23:21.799 INFO:tasks.rados.rados.0.smithi100.stdout:2035: read oid 11 snap -1 2024-04-07T16:23:21.799 INFO:tasks.rados.rados.0.smithi100.stdout:2035: expect (ObjNum 537 snap 162 seq_num 537) 2024-04-07T16:23:21.799 INFO:tasks.rados.rados.0.smithi100.stdout:2036: read oid 24 snap 196 2024-04-07T16:23:21.799 INFO:tasks.rados.rados.0.smithi100.stdout:2036: expect deleted 2024-04-07T16:23:21.800 INFO:tasks.rados.rados.0.smithi100.stdout:2037: delete oid 39 current snap is 198 2024-04-07T16:23:21.816 INFO:tasks.rados.rados.0.smithi100.stdout:2034: done (3 left) 2024-04-07T16:23:21.816 INFO:tasks.rados.rados.0.smithi100.stdout:2036: done (2 left) 2024-04-07T16:23:21.817 INFO:tasks.rados.rados.0.smithi100.stdout:2037: done (1 left) 2024-04-07T16:23:21.817 INFO:tasks.rados.rados.0.smithi100.stdout:2038: snap_create 2024-04-07T16:23:21.824 INFO:tasks.rados.rados.0.smithi100.stdout:2035: expect (ObjNum 537 snap 162 seq_num 537) 2024-04-07T16:23:22.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:21 smithi144 bash[66654]: cluster 2024-04-07T16:23:20.674140+0000 mgr.y (mgr.25175) 167 : cluster [DBG] pgmap v264: 121 pgs: 121 active+clean; 288 MiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 0 B/s rd, 2.8 MiB/s wr, 10 op/s; 8.3 MiB/s, 1 objects/s recovering 2024-04-07T16:23:22.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:21 smithi144 bash[66654]: cluster 2024-04-07T16:23:20.846610+0000 mon.a (mon.0) 530 : cluster [INF] Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 64/510 objects degraded (12.549%), 7 pgs degraded) 2024-04-07T16:23:22.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:21 smithi144 bash[66654]: cluster 2024-04-07T16:23:20.846670+0000 mon.a (mon.0) 531 : cluster [INF] Cluster is now healthy 2024-04-07T16:23:22.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:21 smithi144 bash[66654]: cluster 2024-04-07T16:23:20.863967+0000 mon.a (mon.0) 532 : cluster [DBG] osdmap e450: 8 total, 8 up, 8 in 2024-04-07T16:23:22.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:21 smithi144 bash[66654]: cluster 2024-04-07T16:23:21.817195+0000 mon.a (mon.0) 533 : cluster [DBG] osdmap e451: 8 total, 8 up, 8 in 2024-04-07T16:23:22.184 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:21 smithi100 bash[74942]: cluster 2024-04-07T16:23:20.674140+0000 mgr.y (mgr.25175) 167 : cluster [DBG] pgmap v264: 121 pgs: 121 active+clean; 288 MiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 0 B/s rd, 2.8 MiB/s wr, 10 op/s; 8.3 MiB/s, 1 objects/s recovering 2024-04-07T16:23:22.184 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:21 smithi100 bash[74942]: cluster 2024-04-07T16:23:20.846610+0000 mon.a (mon.0) 530 : cluster [INF] Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 64/510 objects degraded (12.549%), 7 pgs degraded) 2024-04-07T16:23:22.184 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:21 smithi100 bash[74942]: cluster 2024-04-07T16:23:20.846670+0000 mon.a (mon.0) 531 : cluster [INF] Cluster is now healthy 2024-04-07T16:23:22.184 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:21 smithi100 bash[74942]: cluster 2024-04-07T16:23:20.863967+0000 mon.a (mon.0) 532 : cluster [DBG] osdmap e450: 8 total, 8 up, 8 in 2024-04-07T16:23:22.184 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:21 smithi100 bash[74942]: cluster 2024-04-07T16:23:21.817195+0000 mon.a (mon.0) 533 : cluster [DBG] osdmap e451: 8 total, 8 up, 8 in 2024-04-07T16:23:22.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:21 smithi100 bash[76202]: cluster 2024-04-07T16:23:20.674140+0000 mgr.y (mgr.25175) 167 : cluster [DBG] pgmap v264: 121 pgs: 121 active+clean; 288 MiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 0 B/s rd, 2.8 MiB/s wr, 10 op/s; 8.3 MiB/s, 1 objects/s recovering 2024-04-07T16:23:22.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:21 smithi100 bash[76202]: cluster 2024-04-07T16:23:20.846610+0000 mon.a (mon.0) 530 : cluster [INF] Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 64/510 objects degraded (12.549%), 7 pgs degraded) 2024-04-07T16:23:22.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:21 smithi100 bash[76202]: cluster 2024-04-07T16:23:20.846670+0000 mon.a (mon.0) 531 : cluster [INF] Cluster is now healthy 2024-04-07T16:23:22.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:21 smithi100 bash[76202]: cluster 2024-04-07T16:23:20.863967+0000 mon.a (mon.0) 532 : cluster [DBG] osdmap e450: 8 total, 8 up, 8 in 2024-04-07T16:23:22.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:21 smithi100 bash[76202]: cluster 2024-04-07T16:23:21.817195+0000 mon.a (mon.0) 533 : cluster [DBG] osdmap e451: 8 total, 8 up, 8 in 2024-04-07T16:23:22.606 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:23:22 smithi144 bash[59794]: ts=2024-04-07T16:23:22.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:23:22.802 INFO:tasks.rados.rados.0.smithi100.stdout:2035: done (1 left) 2024-04-07T16:23:22.803 INFO:tasks.rados.rados.0.smithi100.stdout:2038: done (0 left) 2024-04-07T16:23:22.803 INFO:tasks.rados.rados.0.smithi100.stdout:2039: delete oid 8 current snap is 199 2024-04-07T16:23:22.815 INFO:tasks.rados.rados.0.smithi100.stdout:2039: done (0 left) 2024-04-07T16:23:22.815 INFO:tasks.rados.rados.0.smithi100.stdout:2040: rollback oid 13 current snap is 199 2024-04-07T16:23:22.815 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 13 to 174 2024-04-07T16:23:22.815 INFO:tasks.rados.rados.0.smithi100.stdout:2041: snap_create 2024-04-07T16:23:22.823 INFO:tasks.rados.rados.0.smithi100.stdout:2040: finishing rollback tid 0 to smithi10066867-13 2024-04-07T16:23:22.823 INFO:tasks.rados.rados.0.smithi100.stdout:2040: finishing rollback tid 1 to smithi10066867-13 2024-04-07T16:23:22.824 INFO:tasks.rados.rados.0.smithi100.stdout:2040: finishing rollback tid 2 to smithi10066867-13 2024-04-07T16:23:22.824 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 13 v 763 (ObjNum 537 snap 162 seq_num 537) dirty exists 2024-04-07T16:23:23.799 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:23:23 smithi144 bash[59794]: ts=2024-04-07T16:23:23.507Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=nodes name=CephNodeDiskspaceWarning index=4 msg="Evaluating rule failed" rule="alert: CephNodeDiskspaceWarning\nexpr: predict_linear(node_filesystem_free_bytes{device=~\"/.*\"}[2d], 3600 * 24 * 5)\n * on (instance) group_left (nodename) node_uname_info < 0\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.8.4\n severity: warning\n type: ceph_default\nannotations:\n description: Mountpoint {{ $labels.mountpoint }} on {{ $labels.nodename }} will\n be full in less than 5 days based on the 48 hour trailing fill rate.\n summary: Host filesystem free space is getting low\n" err="found duplicate series for the match group {instance=\"smithi100\"} on the right hand-side of the operation: [{__name__=\"node_uname_info\", domainname=\"(none)\", instance=\"smithi100\", job=\"node\", machine=\"x86_64\", nodename=\"smithi100\", release=\"5.15.0-101-generic\", sysname=\"Linux\", version=\"#111-Ubuntu SMP Tue Mar 5 20:16:58 UTC 2024\"}, {__name__=\"node_uname_info\", cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", domainname=\"(none)\", instance=\"smithi100\", job=\"node\", machine=\"x86_64\", nodename=\"smithi100\", release=\"5.15.0-101-generic\", sysname=\"Linux\", version=\"#111-Ubuntu SMP Tue Mar 5 20:16:58 UTC 2024\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:23:23.813 INFO:tasks.rados.rados.0.smithi100.stdout:2040: done (1 left) 2024-04-07T16:23:23.813 INFO:tasks.rados.rados.0.smithi100.stdout:2041: done (0 left) 2024-04-07T16:23:23.813 INFO:tasks.rados.rados.0.smithi100.stdout:2042: snap_remove snap 189 2024-04-07T16:23:23.934 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:23 smithi100 bash[76202]: cluster 2024-04-07T16:23:22.675413+0000 mgr.y (mgr.25175) 168 : cluster [DBG] pgmap v267: 121 pgs: 121 active+clean; 290 MiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 3.3 MiB/s rd, 1.8 MiB/s wr, 15 op/s; 8.3 MiB/s, 1 objects/s recovering 2024-04-07T16:23:23.935 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:23 smithi100 bash[76202]: cluster 2024-04-07T16:23:22.820785+0000 mon.a (mon.0) 534 : cluster [DBG] osdmap e452: 8 total, 8 up, 8 in 2024-04-07T16:23:23.935 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:23 smithi100 bash[76202]: audit 2024-04-07T16:23:23.118935+0000 mon.a (mon.0) 535 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:23:23.935 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:23 smithi100 bash[76202]: audit 2024-04-07T16:23:23.132542+0000 mon.a (mon.0) 536 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:23:23.935 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:23 smithi100 bash[76202]: audit 2024-04-07T16:23:23.135092+0000 mon.a (mon.0) 537 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:23:23.935 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:23 smithi100 bash[76202]: audit 2024-04-07T16:23:23.136921+0000 mon.a (mon.0) 538 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:23:23.935 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:23 smithi100 bash[76202]: audit 2024-04-07T16:23:23.147105+0000 mon.a (mon.0) 539 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:23:23.935 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:23 smithi100 bash[76202]: audit 2024-04-07T16:23:23.231929+0000 mon.a (mon.0) 540 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:23:23.935 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:23 smithi100 bash[76202]: audit 2024-04-07T16:23:23.235771+0000 mon.a (mon.0) 541 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:23:23.935 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:23 smithi100 bash[76202]: audit 2024-04-07T16:23:23.238434+0000 mon.a (mon.0) 542 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:23:23.935 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:23 smithi100 bash[76202]: audit 2024-04-07T16:23:23.240516+0000 mon.a (mon.0) 543 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:23:23.935 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:23 smithi100 bash[76202]: audit 2024-04-07T16:23:23.243151+0000 mon.a (mon.0) 544 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-04-07T16:23:23.935 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:23 smithi100 bash[76202]: audit 2024-04-07T16:23:23.243523+0000 mgr.y (mgr.25175) 169 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-04-07T16:23:23.935 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:23 smithi100 bash[76202]: cephadm 2024-04-07T16:23:23.245263+0000 mgr.y (mgr.25175) 170 : cephadm [INF] Upgrade: osd.3 is safe to restart 2024-04-07T16:23:23.936 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:23 smithi100 bash[74942]: cluster 2024-04-07T16:23:22.675413+0000 mgr.y (mgr.25175) 168 : cluster [DBG] pgmap v267: 121 pgs: 121 active+clean; 290 MiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 3.3 MiB/s rd, 1.8 MiB/s wr, 15 op/s; 8.3 MiB/s, 1 objects/s recovering 2024-04-07T16:23:23.936 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:23 smithi100 bash[74942]: cluster 2024-04-07T16:23:22.820785+0000 mon.a (mon.0) 534 : cluster [DBG] osdmap e452: 8 total, 8 up, 8 in 2024-04-07T16:23:23.936 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:23 smithi100 bash[74942]: audit 2024-04-07T16:23:23.118935+0000 mon.a (mon.0) 535 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:23:23.936 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:23 smithi100 bash[74942]: audit 2024-04-07T16:23:23.132542+0000 mon.a (mon.0) 536 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:23:23.936 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:23 smithi100 bash[74942]: audit 2024-04-07T16:23:23.135092+0000 mon.a (mon.0) 537 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:23:23.936 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:23 smithi100 bash[74942]: audit 2024-04-07T16:23:23.136921+0000 mon.a (mon.0) 538 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:23:23.936 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:23 smithi100 bash[74942]: audit 2024-04-07T16:23:23.147105+0000 mon.a (mon.0) 539 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:23:23.936 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:23 smithi100 bash[74942]: audit 2024-04-07T16:23:23.231929+0000 mon.a (mon.0) 540 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:23:23.936 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:23 smithi100 bash[74942]: audit 2024-04-07T16:23:23.235771+0000 mon.a (mon.0) 541 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:23:23.936 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:23 smithi100 bash[74942]: audit 2024-04-07T16:23:23.238434+0000 mon.a (mon.0) 542 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:23:23.936 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:23 smithi100 bash[74942]: audit 2024-04-07T16:23:23.240516+0000 mon.a (mon.0) 543 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:23:23.936 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:23 smithi100 bash[74942]: audit 2024-04-07T16:23:23.243151+0000 mon.a (mon.0) 544 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-04-07T16:23:23.936 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:23 smithi100 bash[74942]: audit 2024-04-07T16:23:23.243523+0000 mgr.y (mgr.25175) 169 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-04-07T16:23:23.936 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:23 smithi100 bash[74942]: cephadm 2024-04-07T16:23:23.245263+0000 mgr.y (mgr.25175) 170 : cephadm [INF] Upgrade: osd.3 is safe to restart 2024-04-07T16:23:24.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:23 smithi144 bash[66654]: cluster 2024-04-07T16:23:22.675413+0000 mgr.y (mgr.25175) 168 : cluster [DBG] pgmap v267: 121 pgs: 121 active+clean; 290 MiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 3.3 MiB/s rd, 1.8 MiB/s wr, 15 op/s; 8.3 MiB/s, 1 objects/s recovering 2024-04-07T16:23:24.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:23 smithi144 bash[66654]: cluster 2024-04-07T16:23:22.820785+0000 mon.a (mon.0) 534 : cluster [DBG] osdmap e452: 8 total, 8 up, 8 in 2024-04-07T16:23:24.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:23 smithi144 bash[66654]: audit 2024-04-07T16:23:23.118935+0000 mon.a (mon.0) 535 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:23:24.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:23 smithi144 bash[66654]: audit 2024-04-07T16:23:23.132542+0000 mon.a (mon.0) 536 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:23:24.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:23 smithi144 bash[66654]: audit 2024-04-07T16:23:23.135092+0000 mon.a (mon.0) 537 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:23:24.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:23 smithi144 bash[66654]: audit 2024-04-07T16:23:23.136921+0000 mon.a (mon.0) 538 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:23:24.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:23 smithi144 bash[66654]: audit 2024-04-07T16:23:23.147105+0000 mon.a (mon.0) 539 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:23:24.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:23 smithi144 bash[66654]: audit 2024-04-07T16:23:23.231929+0000 mon.a (mon.0) 540 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:23:24.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:23 smithi144 bash[66654]: audit 2024-04-07T16:23:23.235771+0000 mon.a (mon.0) 541 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:23:24.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:23 smithi144 bash[66654]: audit 2024-04-07T16:23:23.238434+0000 mon.a (mon.0) 542 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:23:24.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:23 smithi144 bash[66654]: audit 2024-04-07T16:23:23.240516+0000 mon.a (mon.0) 543 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:23:24.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:23 smithi144 bash[66654]: audit 2024-04-07T16:23:23.243151+0000 mon.a (mon.0) 544 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-04-07T16:23:24.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:23 smithi144 bash[66654]: audit 2024-04-07T16:23:23.243523+0000 mgr.y (mgr.25175) 169 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-04-07T16:23:24.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:23 smithi144 bash[66654]: cephadm 2024-04-07T16:23:23.245263+0000 mgr.y (mgr.25175) 170 : cephadm [INF] Upgrade: osd.3 is safe to restart 2024-04-07T16:23:24.821 INFO:tasks.rados.rados.0.smithi100.stdout:2042: done (0 left) 2024-04-07T16:23:24.821 INFO:tasks.rados.rados.0.smithi100.stdout:2043: copy_from oid 10 from oid 48 current snap is 200 2024-04-07T16:23:24.822 INFO:tasks.rados.rados.0.smithi100.stdout:2044: snap_create 2024-04-07T16:23:24.853 INFO:tasks.rados.rados.0.smithi100.stdout:2043: finishing copy_from racing read to smithi10066867-10 2024-04-07T16:23:24.857 INFO:tasks.rados.rados.0.smithi100.stdout:2043: finishing copy_from to smithi10066867-10 2024-04-07T16:23:24.857 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 10 v 1053 (ObjNum 574 snap 177 seq_num 574) dirty exists 2024-04-07T16:23:24.897 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:24 smithi100 bash[74942]: cluster 2024-04-07T16:23:23.831674+0000 mon.a (mon.0) 545 : cluster [DBG] osdmap e453: 8 total, 8 up, 8 in 2024-04-07T16:23:24.898 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:24 smithi100 bash[76202]: cluster 2024-04-07T16:23:23.831674+0000 mon.a (mon.0) 545 : cluster [DBG] osdmap e453: 8 total, 8 up, 8 in 2024-04-07T16:23:25.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:24 smithi144 bash[66654]: cluster 2024-04-07T16:23:23.831674+0000 mon.a (mon.0) 545 : cluster [DBG] osdmap e453: 8 total, 8 up, 8 in 2024-04-07T16:23:25.833 INFO:tasks.rados.rados.0.smithi100.stdout:2043: done (1 left) 2024-04-07T16:23:25.833 INFO:tasks.rados.rados.0.smithi100.stdout:2044: done (0 left) 2024-04-07T16:23:25.833 INFO:tasks.rados.rados.0.smithi100.stdout:2045: read oid 11 snap -1 2024-04-07T16:23:25.833 INFO:tasks.rados.rados.0.smithi100.stdout:2045: expect (ObjNum 537 snap 162 seq_num 537) 2024-04-07T16:23:25.834 INFO:tasks.rados.rados.0.smithi100.stdout:2046: copy_from oid 48 from oid 47 current snap is 201 2024-04-07T16:23:25.834 INFO:tasks.rados.rados.0.smithi100.stdout:2047: rollback oid 28 current snap is 201 2024-04-07T16:23:25.834 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 28 to 174 2024-04-07T16:23:25.834 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 42 current snap is 201 2024-04-07T16:23:25.834 INFO:tasks.rados.rados.0.smithi100.stdout:2048: seq_num 634 ranges {1736704=622592,2359296=401408} 2024-04-07T16:23:25.874 INFO:tasks.rados.rados.0.smithi100.stdout:2048: writing smithi10066867-42 from 1736704 to 2359296 tid 1 2024-04-07T16:23:25.881 INFO:tasks.rados.rados.0.smithi100.stdout:2048: writing smithi10066867-42 from 2359296 to 2760704 tid 2 2024-04-07T16:23:25.881 INFO:tasks.rados.rados.0.smithi100.stdout:2049: setattr oid 43 current snap is 201 2024-04-07T16:23:25.881 INFO:tasks.rados.rados.0.smithi100.stdout:2047: finishing rollback tid 0 to smithi10066867-28 2024-04-07T16:23:25.881 INFO:tasks.rados.rados.0.smithi100.stdout:2047: finishing rollback tid 1 to smithi10066867-28 2024-04-07T16:23:25.881 INFO:tasks.rados.rados.0.smithi100.stdout:2047: finishing rollback tid 2 to smithi10066867-28 2024-04-07T16:23:25.881 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 28 v 778 (ObjNum 461 snap 140 seq_num 461) dirty exists 2024-04-07T16:23:25.881 INFO:tasks.rados.rados.0.smithi100.stdout:2045: expect (ObjNum 537 snap 162 seq_num 537) 2024-04-07T16:23:25.991 INFO:tasks.rados.rados.0.smithi100.stdout:2048: finishing write tid 1 to smithi10066867-42 2024-04-07T16:23:25.991 INFO:tasks.rados.rados.0.smithi100.stdout:2048: finishing write tid 2 to smithi10066867-42 2024-04-07T16:23:25.991 INFO:tasks.rados.rados.0.smithi100.stdout:2048: finishing write tid 3 to smithi10066867-42 2024-04-07T16:23:25.991 INFO:tasks.rados.rados.0.smithi100.stdout:2048: finishing write tid 4 to smithi10066867-42 2024-04-07T16:23:25.991 INFO:tasks.rados.rados.0.smithi100.stdout:2048: oid 42 updating version 0 to 1081 2024-04-07T16:23:25.991 INFO:tasks.rados.rados.0.smithi100.stdout:2048: oid 42 updating version 1081 to 1082 2024-04-07T16:23:25.991 INFO:tasks.rados.rados.0.smithi100.stdout:2048: oid 42 version 1082 is already newer than 1080 2024-04-07T16:23:25.991 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 42 v 1082 (ObjNum 634 snap 201 seq_num 634) dirty exists 2024-04-07T16:23:25.992 INFO:tasks.rados.rados.0.smithi100.stdout:2048: left oid 42 (ObjNum 634 snap 201 seq_num 634) 2024-04-07T16:23:25.992 INFO:tasks.rados.rados.0.smithi100.stdout:2046: finishing copy_from racing read to smithi10066867-48 2024-04-07T16:23:25.992 INFO:tasks.rados.rados.0.smithi100.stdout:2046: finishing copy_from to smithi10066867-48 2024-04-07T16:23:25.992 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 48 v 571 (ObjNum 556 snap 168 seq_num 556) dirty exists 2024-04-07T16:23:25.992 INFO:tasks.rados.rados.0.smithi100.stdout:2045: done (4 left) 2024-04-07T16:23:25.992 INFO:tasks.rados.rados.0.smithi100.stdout:2046: done (3 left) 2024-04-07T16:23:25.992 INFO:tasks.rados.rados.0.smithi100.stdout:2047: done (2 left) 2024-04-07T16:23:25.992 INFO:tasks.rados.rados.0.smithi100.stdout:2048: done (1 left) 2024-04-07T16:23:25.992 INFO:tasks.rados.rados.0.smithi100.stdout:2050: rollback oid 32 current snap is 201 2024-04-07T16:23:25.992 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 32 to 197 2024-04-07T16:23:25.992 INFO:tasks.rados.rados.0.smithi100.stdout:2051: read oid 16 snap -1 2024-04-07T16:23:25.992 INFO:tasks.rados.rados.0.smithi100.stdout:2051: expect (ObjNum 606 snap 186 seq_num 606) 2024-04-07T16:23:25.992 INFO:tasks.rados.rados.0.smithi100.stdout:2052: delete oid 17 current snap is 201 2024-04-07T16:23:25.996 INFO:tasks.rados.rados.0.smithi100.stdout:2052: done (3 left) 2024-04-07T16:23:25.996 INFO:tasks.rados.rados.0.smithi100.stdout:2053: read oid 40 snap -1 2024-04-07T16:23:25.997 INFO:tasks.rados.rados.0.smithi100.stdout:2053: expect (ObjNum 509 snap 152 seq_num 509) 2024-04-07T16:23:25.997 INFO:tasks.rados.rados.0.smithi100.stdout:2054: delete oid 34 current snap is 201 2024-04-07T16:23:25.998 INFO:tasks.rados.rados.0.smithi100.stdout:2050: finishing rollback tid 0 to smithi10066867-32 2024-04-07T16:23:25.999 INFO:tasks.rados.rados.0.smithi100.stdout:2050: finishing rollback tid 1 to smithi10066867-32 2024-04-07T16:23:25.999 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 43 v 1079 (ObjNum 618 snap 194 seq_num 618) dirty exists 2024-04-07T16:23:26.001 INFO:tasks.rados.rados.0.smithi100.stdout:2050: finishing rollback tid 2 to smithi10066867-32 2024-04-07T16:23:26.001 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 32 v 781 (ObjNum 589 snap 182 seq_num 589) dirty exists 2024-04-07T16:23:26.001 INFO:tasks.rados.rados.0.smithi100.stdout:2049: done (4 left) 2024-04-07T16:23:26.001 INFO:tasks.rados.rados.0.smithi100.stdout:2050: done (3 left) 2024-04-07T16:23:26.001 INFO:tasks.rados.rados.0.smithi100.stdout:2054: done (2 left) 2024-04-07T16:23:26.002 INFO:tasks.rados.rados.0.smithi100.stdout:2055: snap_remove snap 197 2024-04-07T16:23:26.102 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:25 smithi100 bash[74942]: cluster 2024-04-07T16:23:24.676160+0000 mgr.y (mgr.25175) 171 : cluster [DBG] pgmap v270: 121 pgs: 121 active+clean; 291 MiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 9.9 MiB/s rd, 682 KiB/s wr, 12 op/s 2024-04-07T16:23:26.102 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:25 smithi100 bash[74942]: cluster 2024-04-07T16:23:24.839538+0000 mon.a (mon.0) 546 : cluster [DBG] osdmap e454: 8 total, 8 up, 8 in 2024-04-07T16:23:26.102 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:25 smithi100 bash[74942]: audit 2024-04-07T16:23:25.622718+0000 mon.a (mon.0) 547 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:23:26.102 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:25 smithi100 bash[74942]: audit 2024-04-07T16:23:25.627257+0000 mon.a (mon.0) 548 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-07T16:23:26.103 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:25 smithi100 bash[74942]: audit 2024-04-07T16:23:25.628365+0000 mon.a (mon.0) 549 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:23:26.103 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:25 smithi100 bash[76202]: cluster 2024-04-07T16:23:24.676160+0000 mgr.y (mgr.25175) 171 : cluster [DBG] pgmap v270: 121 pgs: 121 active+clean; 291 MiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 9.9 MiB/s rd, 682 KiB/s wr, 12 op/s 2024-04-07T16:23:26.103 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:25 smithi100 bash[76202]: cluster 2024-04-07T16:23:24.839538+0000 mon.a (mon.0) 546 : cluster [DBG] osdmap e454: 8 total, 8 up, 8 in 2024-04-07T16:23:26.103 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:25 smithi100 bash[76202]: audit 2024-04-07T16:23:25.622718+0000 mon.a (mon.0) 547 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:23:26.103 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:25 smithi100 bash[76202]: audit 2024-04-07T16:23:25.627257+0000 mon.a (mon.0) 548 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-07T16:23:26.103 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:25 smithi100 bash[76202]: audit 2024-04-07T16:23:25.628365+0000 mon.a (mon.0) 549 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:23:26.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:25 smithi144 bash[66654]: cluster 2024-04-07T16:23:24.676160+0000 mgr.y (mgr.25175) 171 : cluster [DBG] pgmap v270: 121 pgs: 121 active+clean; 291 MiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 9.9 MiB/s rd, 682 KiB/s wr, 12 op/s 2024-04-07T16:23:26.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:25 smithi144 bash[66654]: cluster 2024-04-07T16:23:24.839538+0000 mon.a (mon.0) 546 : cluster [DBG] osdmap e454: 8 total, 8 up, 8 in 2024-04-07T16:23:26.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:25 smithi144 bash[66654]: audit 2024-04-07T16:23:25.622718+0000 mon.a (mon.0) 547 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:23:26.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:25 smithi144 bash[66654]: audit 2024-04-07T16:23:25.627257+0000 mon.a (mon.0) 548 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-07T16:23:26.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:25 smithi144 bash[66654]: audit 2024-04-07T16:23:25.628365+0000 mon.a (mon.0) 549 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:23:26.801 INFO:tasks.rados.rados.0.smithi100.stdout:2055: done (2 left) 2024-04-07T16:23:26.801 INFO:tasks.rados.rados.0.smithi100.stdout:2056: snap_remove snap 194 2024-04-07T16:23:27.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:26 smithi144 bash[66654]: cephadm 2024-04-07T16:23:25.615385+0000 mgr.y (mgr.25175) 172 : cephadm [INF] Upgrade: Updating osd.3 2024-04-07T16:23:27.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:26 smithi144 bash[66654]: cephadm 2024-04-07T16:23:25.632946+0000 mgr.y (mgr.25175) 173 : cephadm [INF] Deploying daemon osd.3 on smithi100 2024-04-07T16:23:27.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:26 smithi144 bash[66654]: cluster 2024-04-07T16:23:25.850820+0000 mon.a (mon.0) 550 : cluster [DBG] osdmap e455: 8 total, 8 up, 8 in 2024-04-07T16:23:27.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:26 smithi144 bash[66654]: cluster 2024-04-07T16:23:26.819739+0000 mon.a (mon.0) 551 : cluster [DBG] osdmap e456: 8 total, 8 up, 8 in 2024-04-07T16:23:27.184 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:26 smithi100 bash[74942]: cephadm 2024-04-07T16:23:25.615385+0000 mgr.y (mgr.25175) 172 : cephadm [INF] Upgrade: Updating osd.3 2024-04-07T16:23:27.184 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:26 smithi100 bash[74942]: cephadm 2024-04-07T16:23:25.632946+0000 mgr.y (mgr.25175) 173 : cephadm [INF] Deploying daemon osd.3 on smithi100 2024-04-07T16:23:27.184 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:26 smithi100 bash[74942]: cluster 2024-04-07T16:23:25.850820+0000 mon.a (mon.0) 550 : cluster [DBG] osdmap e455: 8 total, 8 up, 8 in 2024-04-07T16:23:27.184 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:26 smithi100 bash[74942]: cluster 2024-04-07T16:23:26.819739+0000 mon.a (mon.0) 551 : cluster [DBG] osdmap e456: 8 total, 8 up, 8 in 2024-04-07T16:23:27.184 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:26 smithi100 bash[76202]: cephadm 2024-04-07T16:23:25.615385+0000 mgr.y (mgr.25175) 172 : cephadm [INF] Upgrade: Updating osd.3 2024-04-07T16:23:27.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:26 smithi100 bash[76202]: cephadm 2024-04-07T16:23:25.632946+0000 mgr.y (mgr.25175) 173 : cephadm [INF] Deploying daemon osd.3 on smithi100 2024-04-07T16:23:27.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:26 smithi100 bash[76202]: cluster 2024-04-07T16:23:25.850820+0000 mon.a (mon.0) 550 : cluster [DBG] osdmap e455: 8 total, 8 up, 8 in 2024-04-07T16:23:27.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:26 smithi100 bash[76202]: cluster 2024-04-07T16:23:26.819739+0000 mon.a (mon.0) 551 : cluster [DBG] osdmap e456: 8 total, 8 up, 8 in 2024-04-07T16:23:27.800 INFO:tasks.rados.rados.0.smithi100.stdout:2056: done (2 left) 2024-04-07T16:23:27.801 INFO:tasks.rados.rados.0.smithi100.stdout:2057: rmattr oid 7 current snap is 201 2024-04-07T16:23:27.801 INFO:tasks.rados.rados.0.smithi100.stdout:2051: expect (ObjNum 606 snap 186 seq_num 606) 2024-04-07T16:23:28.045 INFO:tasks.rados.rados.0.smithi100.stdout:2053: expect (ObjNum 509 snap 152 seq_num 509) 2024-04-07T16:23:28.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:27 smithi144 bash[66654]: cluster 2024-04-07T16:23:26.676780+0000 mgr.y (mgr.25175) 174 : cluster [DBG] pgmap v273: 121 pgs: 121 active+clean; 291 MiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 5.6 MiB/s rd, 234 KiB/s wr, 3 op/s 2024-04-07T16:23:28.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:27 smithi144 bash[66654]: audit 2024-04-07T16:23:26.851589+0000 mon.a (mon.0) 552 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:23:28.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:27 smithi144 bash[66654]: cluster 2024-04-07T16:23:27.819360+0000 mon.a (mon.0) 553 : cluster [DBG] osdmap e457: 8 total, 8 up, 8 in 2024-04-07T16:23:28.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:27 smithi100 bash[76202]: cluster 2024-04-07T16:23:26.676780+0000 mgr.y (mgr.25175) 174 : cluster [DBG] pgmap v273: 121 pgs: 121 active+clean; 291 MiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 5.6 MiB/s rd, 234 KiB/s wr, 3 op/s 2024-04-07T16:23:28.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:27 smithi100 bash[76202]: audit 2024-04-07T16:23:26.851589+0000 mon.a (mon.0) 552 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:23:28.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:27 smithi100 bash[76202]: cluster 2024-04-07T16:23:27.819360+0000 mon.a (mon.0) 553 : cluster [DBG] osdmap e457: 8 total, 8 up, 8 in 2024-04-07T16:23:28.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:27 smithi100 bash[74942]: cluster 2024-04-07T16:23:26.676780+0000 mgr.y (mgr.25175) 174 : cluster [DBG] pgmap v273: 121 pgs: 121 active+clean; 291 MiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 5.6 MiB/s rd, 234 KiB/s wr, 3 op/s 2024-04-07T16:23:28.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:27 smithi100 bash[74942]: audit 2024-04-07T16:23:26.851589+0000 mon.a (mon.0) 552 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:23:28.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:27 smithi100 bash[74942]: cluster 2024-04-07T16:23:27.819360+0000 mon.a (mon.0) 553 : cluster [DBG] osdmap e457: 8 total, 8 up, 8 in 2024-04-07T16:23:28.416 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 7 v 1080 (ObjNum 617 snap 193 seq_num 617) dirty exists 2024-04-07T16:23:28.416 INFO:tasks.rados.rados.0.smithi100.stdout:2051: done (2 left) 2024-04-07T16:23:28.416 INFO:tasks.rados.rados.0.smithi100.stdout:2053: done (1 left) 2024-04-07T16:23:28.416 INFO:tasks.rados.rados.0.smithi100.stdout:2057: done (0 left) 2024-04-07T16:23:28.416 INFO:tasks.rados.rados.0.smithi100.stdout:2058: snap_remove snap 192 2024-04-07T16:23:28.683 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:23:28 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:23:28] "GET /metrics HTTP/1.1" 200 36065 "" "Prometheus/2.43.0" 2024-04-07T16:23:28.853 INFO:tasks.rados.rados.0.smithi100.stdout:2058: done (0 left) 2024-04-07T16:23:28.854 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 34 current snap is 201 2024-04-07T16:23:28.854 INFO:tasks.rados.rados.0.smithi100.stdout:2059: seq_num 637 ranges {0=671744,671744=401408} 2024-04-07T16:23:28.863 INFO:tasks.rados.rados.0.smithi100.stdout:2059: writing smithi10066867-34 from 0 to 671744 tid 1 2024-04-07T16:23:28.868 INFO:tasks.rados.rados.0.smithi100.stdout:2059: writing smithi10066867-34 from 671744 to 1073152 tid 2 2024-04-07T16:23:28.868 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 38 current snap is 201 2024-04-07T16:23:28.869 INFO:tasks.rados.rados.0.smithi100.stdout:2060: seq_num 638 ranges {1728512=688128,2416640=401408} 2024-04-07T16:23:28.891 INFO:tasks.rados.rados.0.smithi100.stdout:2060: writing smithi10066867-38 from 1728512 to 2416640 tid 1 2024-04-07T16:23:28.895 INFO:tasks.rados.rados.0.smithi100.stdout:2060: writing smithi10066867-38 from 2416640 to 2818048 tid 2 2024-04-07T16:23:28.895 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 1 current snap is 201 2024-04-07T16:23:28.895 INFO:tasks.rados.rados.0.smithi100.stdout:2059: finishing write tid 1 to smithi10066867-34 2024-04-07T16:23:28.895 INFO:tasks.rados.rados.0.smithi100.stdout:2059: finishing write tid 2 to smithi10066867-34 2024-04-07T16:23:28.895 INFO:tasks.rados.rados.0.smithi100.stdout:2059: finishing write tid 3 to smithi10066867-34 2024-04-07T16:23:28.895 INFO:tasks.rados.rados.0.smithi100.stdout:2059: finishing write tid 4 to smithi10066867-34 2024-04-07T16:23:28.896 INFO:tasks.rados.rados.0.smithi100.stdout:2059: oid 34 updating version 0 to 787 2024-04-07T16:23:28.896 INFO:tasks.rados.rados.0.smithi100.stdout:2059: oid 34 updating version 787 to 788 2024-04-07T16:23:28.896 INFO:tasks.rados.rados.0.smithi100.stdout:2059: oid 34 version 788 is already newer than 786 2024-04-07T16:23:28.896 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 34 v 788 (ObjNum 637 snap 201 seq_num 637) dirty exists 2024-04-07T16:23:28.896 INFO:tasks.rados.rados.0.smithi100.stdout:2059: left oid 34 (ObjNum 637 snap 201 seq_num 637) 2024-04-07T16:23:28.896 INFO:tasks.rados.rados.0.smithi100.stdout:2061: seq_num 639 ranges {720896=704512,1425408=401408} 2024-04-07T16:23:28.908 INFO:tasks.rados.rados.0.smithi100.stdout:2061: writing smithi10066867-1 from 720896 to 1425408 tid 1 2024-04-07T16:23:28.911 INFO:tasks.rados.rados.0.smithi100.stdout:2061: writing smithi10066867-1 from 1425408 to 1826816 tid 2 2024-04-07T16:23:28.911 INFO:tasks.rados.rados.0.smithi100.stdout:2059: done (2 left) 2024-04-07T16:23:28.912 INFO:tasks.rados.rados.0.smithi100.stdout:2062: rollback oid 48 current snap is 201 2024-04-07T16:23:28.912 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 48 to 198 2024-04-07T16:23:28.912 INFO:tasks.rados.rados.0.smithi100.stdout:2060: finishing write tid 1 to smithi10066867-38 2024-04-07T16:23:28.912 INFO:tasks.rados.rados.0.smithi100.stdout:2060: finishing write tid 2 to smithi10066867-38 2024-04-07T16:23:28.912 INFO:tasks.rados.rados.0.smithi100.stdout:2063: rollback oid 6 current snap is 201 2024-04-07T16:23:28.912 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 6 to 190 2024-04-07T16:23:28.912 INFO:tasks.rados.rados.0.smithi100.stdout:2064: copy_from oid 42 from oid 21 current snap is 201 2024-04-07T16:23:28.912 INFO:tasks.rados.rados.0.smithi100.stdout:2065: read oid 17 snap -1 2024-04-07T16:23:28.912 INFO:tasks.rados.rados.0.smithi100.stdout:2065: expect deleted 2024-04-07T16:23:28.912 INFO:tasks.rados.rados.0.smithi100.stdout:2066: delete oid 3 current snap is 201 2024-04-07T16:23:28.915 INFO:tasks.rados.rados.0.smithi100.stdout:2062: finishing rollback tid 0 to smithi10066867-48 2024-04-07T16:23:28.916 INFO:tasks.rados.rados.0.smithi100.stdout:2060: finishing write tid 3 to smithi10066867-38 2024-04-07T16:23:28.916 INFO:tasks.rados.rados.0.smithi100.stdout:2062: finishing rollback tid 1 to smithi10066867-48 2024-04-07T16:23:28.916 INFO:tasks.rados.rados.0.smithi100.stdout:2060: finishing write tid 4 to smithi10066867-38 2024-04-07T16:23:28.916 INFO:tasks.rados.rados.0.smithi100.stdout:2060: oid 38 updating version 0 to 1087 2024-04-07T16:23:28.916 INFO:tasks.rados.rados.0.smithi100.stdout:2060: oid 38 version 1087 is already newer than 1086 2024-04-07T16:23:28.916 INFO:tasks.rados.rados.0.smithi100.stdout:2060: oid 38 updating version 1087 to 1088 2024-04-07T16:23:28.916 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 38 v 1088 (ObjNum 638 snap 201 seq_num 638) dirty exists 2024-04-07T16:23:28.917 INFO:tasks.rados.rados.0.smithi100.stdout:2060: left oid 38 (ObjNum 638 snap 201 seq_num 638) 2024-04-07T16:23:28.917 INFO:tasks.rados.rados.0.smithi100.stdout:2062: finishing rollback tid 2 to smithi10066867-48 2024-04-07T16:23:28.917 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 48 v 1058 (ObjNum 574 snap 177 seq_num 574) dirty exists 2024-04-07T16:23:28.917 INFO:tasks.rados.rados.0.smithi100.stdout:2061: finishing write tid 1 to smithi10066867-1 2024-04-07T16:23:28.917 INFO:tasks.rados.rados.0.smithi100.stdout:2061: finishing write tid 2 to smithi10066867-1 2024-04-07T16:23:28.917 INFO:tasks.rados.rados.0.smithi100.stdout:2061: finishing write tid 3 to smithi10066867-1 2024-04-07T16:23:28.920 INFO:tasks.rados.rados.0.smithi100.stdout:2063: finishing rollback tid 1 to smithi10066867-6 2024-04-07T16:23:28.920 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 6 v 1095 (ObjNum 33 snap 0 seq_num 2702960288) dirty dne 2024-04-07T16:23:28.921 INFO:tasks.rados.rados.0.smithi100.stdout:2061: finishing write tid 4 to smithi10066867-1 2024-04-07T16:23:28.921 INFO:tasks.rados.rados.0.smithi100.stdout:2061: oid 1 updating version 0 to 1054 2024-04-07T16:23:28.921 INFO:tasks.rados.rados.0.smithi100.stdout:2061: oid 1 updating version 1054 to 1056 2024-04-07T16:23:28.922 INFO:tasks.rados.rados.0.smithi100.stdout:2061: oid 1 version 1056 is already newer than 1055 2024-04-07T16:23:28.922 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 1 v 1056 (ObjNum 639 snap 201 seq_num 639) dirty exists 2024-04-07T16:23:28.922 INFO:tasks.rados.rados.0.smithi100.stdout:2061: left oid 1 (ObjNum 639 snap 201 seq_num 639) 2024-04-07T16:23:28.922 INFO:tasks.rados.rados.0.smithi100.stdout:2060: done (6 left) 2024-04-07T16:23:28.922 INFO:tasks.rados.rados.0.smithi100.stdout:2061: done (5 left) 2024-04-07T16:23:28.922 INFO:tasks.rados.rados.0.smithi100.stdout:2062: done (4 left) 2024-04-07T16:23:28.922 INFO:tasks.rados.rados.0.smithi100.stdout:2063: done (3 left) 2024-04-07T16:23:28.922 INFO:tasks.rados.rados.0.smithi100.stdout:2065: done (2 left) 2024-04-07T16:23:28.922 INFO:tasks.rados.rados.0.smithi100.stdout:2066: done (1 left) 2024-04-07T16:23:28.922 INFO:tasks.rados.rados.0.smithi100.stdout:2067: read oid 28 snap -1 2024-04-07T16:23:28.922 INFO:tasks.rados.rados.0.smithi100.stdout:2067: expect (ObjNum 461 snap 140 seq_num 461) 2024-04-07T16:23:28.922 INFO:tasks.rados.rados.0.smithi100.stdout:2068: snap_create 2024-04-07T16:23:28.935 INFO:tasks.rados.rados.0.smithi100.stdout:2067: expect (ObjNum 461 snap 140 seq_num 461) 2024-04-07T16:23:29.009 INFO:tasks.rados.rados.0.smithi100.stdout:2064: finishing copy_from racing read to smithi10066867-42 2024-04-07T16:23:29.009 INFO:tasks.rados.rados.0.smithi100.stdout:2064: finishing copy_from to smithi10066867-42 2024-04-07T16:23:29.009 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 42 v 809 (ObjNum 565 snap 173 seq_num 565) dirty exists 2024-04-07T16:23:29.856 INFO:tasks.rados.rados.0.smithi100.stdout:2064: done (2 left) 2024-04-07T16:23:29.856 INFO:tasks.rados.rados.0.smithi100.stdout:2067: done (1 left) 2024-04-07T16:23:29.856 INFO:tasks.rados.rados.0.smithi100.stdout:2068: done (0 left) 2024-04-07T16:23:29.856 INFO:tasks.rados.rados.0.smithi100.stdout:2069: snap_remove snap 196 2024-04-07T16:23:30.066 INFO:journalctl@ceph.osd.2.smithi100.stdout:Apr 07 16:23:29 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:23:30.067 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 16:23:29 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:23:30.067 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:29 smithi100 bash[74942]: cluster 2024-04-07T16:23:28.677284+0000 mgr.y (mgr.25175) 175 : cluster [DBG] pgmap v276: 121 pgs: 121 active+clean; 295 MiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 3.0 MiB/s rd, 1.2 MiB/s wr, 4 op/s 2024-04-07T16:23:30.067 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:29 smithi100 bash[74942]: cluster 2024-04-07T16:23:28.871778+0000 mon.a (mon.0) 554 : cluster [DBG] osdmap e458: 8 total, 8 up, 8 in 2024-04-07T16:23:30.067 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:29 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:23:30.067 INFO:journalctl@ceph.osd.1.smithi100.stdout:Apr 07 16:23:29 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:23:30.068 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:23:29 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:23:30.068 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:29 smithi100 bash[76202]: cluster 2024-04-07T16:23:28.677284+0000 mgr.y (mgr.25175) 175 : cluster [DBG] pgmap v276: 121 pgs: 121 active+clean; 295 MiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 3.0 MiB/s rd, 1.2 MiB/s wr, 4 op/s 2024-04-07T16:23:30.068 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:29 smithi100 bash[76202]: cluster 2024-04-07T16:23:28.871778+0000 mon.a (mon.0) 554 : cluster [DBG] osdmap e458: 8 total, 8 up, 8 in 2024-04-07T16:23:30.068 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:29 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:23:30.068 INFO:journalctl@ceph.osd.0.smithi100.stdout:Apr 07 16:23:29 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:23:30.069 INFO:journalctl@ceph.osd.3.smithi100.stdout:Apr 07 16:23:29 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:23:30.069 INFO:journalctl@ceph.osd.3.smithi100.stdout:Apr 07 16:23:30 smithi100 systemd[1]: Stopping Ceph osd.3 for 0702e3a8-f4f8-11ee-b649-cb9ed24678a4... 2024-04-07T16:23:30.069 INFO:journalctl@ceph.mds.a.smithi100.stdout:Apr 07 16:23:29 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:23:30.069 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Apr 07 16:23:29 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:23:30.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:29 smithi144 bash[66654]: cluster 2024-04-07T16:23:28.677284+0000 mgr.y (mgr.25175) 175 : cluster [DBG] pgmap v276: 121 pgs: 121 active+clean; 295 MiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 3.0 MiB/s rd, 1.2 MiB/s wr, 4 op/s 2024-04-07T16:23:30.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:29 smithi144 bash[66654]: cluster 2024-04-07T16:23:28.871778+0000 mon.a (mon.0) 554 : cluster [DBG] osdmap e458: 8 total, 8 up, 8 in 2024-04-07T16:23:30.434 INFO:journalctl@ceph.osd.3.smithi100.stdout:Apr 07 16:23:30 smithi100 bash[51673]: debug 2024-04-07T16:23:30.203+0000 7f7bdb5da700 -1 received signal: Terminated from /sbin/docker-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug (PID: 1) UID: 0 2024-04-07T16:23:30.434 INFO:journalctl@ceph.osd.3.smithi100.stdout:Apr 07 16:23:30 smithi100 bash[51673]: debug 2024-04-07T16:23:30.203+0000 7f7bdb5da700 -1 osd.3 458 *** Got signal Terminated *** 2024-04-07T16:23:30.434 INFO:journalctl@ceph.osd.3.smithi100.stdout:Apr 07 16:23:30 smithi100 bash[51673]: debug 2024-04-07T16:23:30.203+0000 7f7bdb5da700 -1 osd.3 458 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-07T16:23:30.863 INFO:tasks.rados.rados.0.smithi100.stdout:2069: done (0 left) 2024-04-07T16:23:30.863 INFO:tasks.rados.rados.0.smithi100.stdout:2070: snap_remove snap 174 2024-04-07T16:23:31.184 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:30 smithi100 bash[74942]: cluster 2024-04-07T16:23:29.873267+0000 mon.a (mon.0) 555 : cluster [DBG] osdmap e459: 8 total, 8 up, 8 in 2024-04-07T16:23:31.184 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:30 smithi100 bash[74942]: cluster 2024-04-07T16:23:30.206031+0000 mon.a (mon.0) 556 : cluster [INF] osd.3 marked itself down and dead 2024-04-07T16:23:31.184 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:30 smithi100 bash[74942]: cluster 2024-04-07T16:23:30.869397+0000 mon.a (mon.0) 557 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN) 2024-04-07T16:23:31.184 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:30 smithi100 bash[76202]: cluster 2024-04-07T16:23:29.873267+0000 mon.a (mon.0) 555 : cluster [DBG] osdmap e459: 8 total, 8 up, 8 in 2024-04-07T16:23:31.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:30 smithi100 bash[76202]: cluster 2024-04-07T16:23:30.206031+0000 mon.a (mon.0) 556 : cluster [INF] osd.3 marked itself down and dead 2024-04-07T16:23:31.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:30 smithi100 bash[76202]: cluster 2024-04-07T16:23:30.869397+0000 mon.a (mon.0) 557 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN) 2024-04-07T16:23:31.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:30 smithi144 bash[66654]: cluster 2024-04-07T16:23:29.873267+0000 mon.a (mon.0) 555 : cluster [DBG] osdmap e459: 8 total, 8 up, 8 in 2024-04-07T16:23:31.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:30 smithi144 bash[66654]: cluster 2024-04-07T16:23:30.206031+0000 mon.a (mon.0) 556 : cluster [INF] osd.3 marked itself down and dead 2024-04-07T16:23:31.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:30 smithi144 bash[66654]: cluster 2024-04-07T16:23:30.869397+0000 mon.a (mon.0) 557 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN) 2024-04-07T16:23:31.565 INFO:teuthology.orchestra.run.smithi100.stdout:true 2024-04-07T16:23:31.822 INFO:tasks.rados.rados.0.smithi100.stdout:2070: done (0 left) 2024-04-07T16:23:31.822 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 8 current snap is 202 2024-04-07T16:23:31.822 INFO:tasks.rados.rados.0.smithi100.stdout:2071: seq_num 641 ranges {0=745472,745472=401408} 2024-04-07T16:23:31.822 INFO:tasks.rados.rados.0.smithi100.stdout:2071: writing smithi10066867-8 from 0 to 745472 tid 1 2024-04-07T16:23:31.824 INFO:tasks.rados.rados.0.smithi100.stdout:2071: writing smithi10066867-8 from 745472 to 1146880 tid 2 2024-04-07T16:23:31.824 INFO:tasks.rados.rados.0.smithi100.stdout:2072: rollback oid 45 current snap is 202 2024-04-07T16:23:31.824 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 45 to 200 2024-04-07T16:23:31.824 INFO:tasks.rados.rados.0.smithi100.stdout:2073: rollback oid 9 current snap is 202 2024-04-07T16:23:31.824 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 9 to 201 2024-04-07T16:23:31.824 INFO:tasks.rados.rados.0.smithi100.stdout:2074: read oid 36 snap -1 2024-04-07T16:23:31.824 INFO:tasks.rados.rados.0.smithi100.stdout:2074: expect (ObjNum 492 snap 148 seq_num 492) 2024-04-07T16:23:31.824 INFO:tasks.rados.rados.0.smithi100.stdout:2075: setattr oid 31 current snap is 202 2024-04-07T16:23:31.825 INFO:tasks.rados.rados.0.smithi100.stdout:2076: rmattr oid 39 current snap is 202 2024-04-07T16:23:31.825 INFO:tasks.rados.rados.0.smithi100.stdout:2076: done (5 left) 2024-04-07T16:23:31.826 INFO:tasks.rados.rados.0.smithi100.stdout:2077: copy_from oid 24 from oid 48 current snap is 202 2024-04-07T16:23:31.826 INFO:tasks.rados.rados.0.smithi100.stdout:2078: snap_remove snap 190 2024-04-07T16:23:31.955 INFO:teuthology.orchestra.run.smithi100.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-07T16:23:31.955 INFO:teuthology.orchestra.run.smithi100.stdout:alertmanager.a smithi100 *:9093,9094 running (9m) 19s ago 10m 14.4M - 0.25.0 c8568f914cd2 5143075e7d26 2024-04-07T16:23:31.955 INFO:teuthology.orchestra.run.smithi100.stdout:grafana.a smithi144 *:3000 running (9m) 2m ago 9m 79.3M - 9.4.7 954c08fa6188 7180c0708acd 2024-04-07T16:23:31.956 INFO:teuthology.orchestra.run.smithi100.stdout:mds.a smithi100 running (11m) 19s ago 11m 15.6M - 18.2.2-1171-g267c8ff5 8a4a01e7186c 92593b0c28ae 2024-04-07T16:23:31.956 INFO:teuthology.orchestra.run.smithi100.stdout:mds.b smithi144 running (11m) 2m ago 11m 18.3M - 18.2.2-1171-g267c8ff5 8a4a01e7186c 3df2ee3e17a4 2024-04-07T16:23:31.956 INFO:teuthology.orchestra.run.smithi100.stdout:mgr.x smithi144 *:8443,9283,8765 running (3m) 2m ago 18m 438M - 19.0.0-2785-gadce92d0 c0ecd4226ac9 233745c292a6 2024-04-07T16:23:31.956 INFO:teuthology.orchestra.run.smithi100.stdout:mgr.y smithi100 *:8443,9283,8765 running (4m) 19s ago 21m 502M - 19.0.0-2785-gadce92d0 c0ecd4226ac9 254dc65cdaa7 2024-04-07T16:23:31.956 INFO:teuthology.orchestra.run.smithi100.stdout:mon.a smithi100 running (3m) 19s ago 21m 52.7M 2048M 19.0.0-2785-gadce92d0 c0ecd4226ac9 718e12655c04 2024-04-07T16:23:31.956 INFO:teuthology.orchestra.run.smithi100.stdout:mon.b smithi144 running (2m) 2m ago 19m 18.2M 2048M 19.0.0-2785-gadce92d0 c0ecd4226ac9 aef817664bff 2024-04-07T16:23:31.956 INFO:teuthology.orchestra.run.smithi100.stdout:mon.c smithi100 running (2m) 19s ago 19m 42.8M 2048M 19.0.0-2785-gadce92d0 c0ecd4226ac9 63e889040f6e 2024-04-07T16:23:31.956 INFO:teuthology.orchestra.run.smithi100.stdout:node-exporter.a smithi100 *:9100 running (10m) 19s ago 10m 8867k - 1.5.0 0da6a335fe13 4bce264c1b26 2024-04-07T16:23:31.956 INFO:teuthology.orchestra.run.smithi100.stdout:node-exporter.b smithi144 *:9100 running (10m) 2m ago 10m 8443k - 1.5.0 0da6a335fe13 b6963413c560 2024-04-07T16:23:31.956 INFO:teuthology.orchestra.run.smithi100.stdout:osd.0 smithi100 running (79s) 19s ago 17m 181M 2524M 19.0.0-2785-gadce92d0 c0ecd4226ac9 a6e83c9d1abc 2024-04-07T16:23:31.956 INFO:teuthology.orchestra.run.smithi100.stdout:osd.1 smithi100 running (52s) 19s ago 17m 189M 2524M 19.0.0-2785-gadce92d0 c0ecd4226ac9 c50fe40079b6 2024-04-07T16:23:31.956 INFO:teuthology.orchestra.run.smithi100.stdout:osd.2 smithi100 running (23s) 19s ago 16m - 2524M 19.0.0-2785-gadce92d0 c0ecd4226ac9 8e631c43064b 2024-04-07T16:23:31.956 INFO:teuthology.orchestra.run.smithi100.stdout:osd.3 smithi100 running (15m) 19s ago 15m 216M 2524M 18.2.2-1171-g267c8ff5 8a4a01e7186c b41004090c63 2024-04-07T16:23:31.956 INFO:teuthology.orchestra.run.smithi100.stdout:osd.4 smithi144 running (14m) 2m ago 14m 257M 2524M 18.2.2-1171-g267c8ff5 8a4a01e7186c 8c73b6b4dc13 2024-04-07T16:23:31.957 INFO:teuthology.orchestra.run.smithi100.stdout:osd.5 smithi144 running (13m) 2m ago 13m 216M 2524M 18.2.2-1171-g267c8ff5 8a4a01e7186c 3bfa0cf6f6cb 2024-04-07T16:23:31.957 INFO:teuthology.orchestra.run.smithi100.stdout:osd.6 smithi144 running (12m) 2m ago 13m 98.7M 2524M 18.2.2-1171-g267c8ff5 8a4a01e7186c 32b7f95d77f0 2024-04-07T16:23:31.957 INFO:teuthology.orchestra.run.smithi100.stdout:osd.7 smithi144 running (12m) 2m ago 12m 156M 2524M 18.2.2-1171-g267c8ff5 8a4a01e7186c 45f7709e576a 2024-04-07T16:23:31.957 INFO:teuthology.orchestra.run.smithi100.stdout:prometheus.a smithi144 *:9095 running (3m) 2m ago 10m 46.5M - 2.43.0 a07b618ecd1d 2d0a8131f303 2024-04-07T16:23:32.123 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:31 smithi100 bash[74942]: cluster 2024-04-07T16:23:30.678006+0000 mgr.y (mgr.25175) 176 : cluster [DBG] pgmap v279: 121 pgs: 121 active+clean; 304 MiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 9.1 MiB/s rd, 3.3 MiB/s wr, 13 op/s 2024-04-07T16:23:32.123 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:31 smithi100 bash[74942]: cluster 2024-04-07T16:23:30.878911+0000 mon.a (mon.0) 558 : cluster [DBG] osdmap e460: 8 total, 7 up, 8 in 2024-04-07T16:23:32.123 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:31 smithi100 bash[74942]: audit 2024-04-07T16:23:31.716553+0000 mon.a (mon.0) 559 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:23:32.124 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:31 smithi100 bash[74942]: audit 2024-04-07T16:23:31.717499+0000 mon.a (mon.0) 560 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:23:32.124 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:31 smithi100 bash[74942]: cluster 2024-04-07T16:23:31.824105+0000 mon.a (mon.0) 561 : cluster [DBG] osdmap e461: 8 total, 7 up, 8 in 2024-04-07T16:23:32.124 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:31 smithi100 bash[76202]: cluster 2024-04-07T16:23:30.678006+0000 mgr.y (mgr.25175) 176 : cluster [DBG] pgmap v279: 121 pgs: 121 active+clean; 304 MiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 9.1 MiB/s rd, 3.3 MiB/s wr, 13 op/s 2024-04-07T16:23:32.124 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:31 smithi100 bash[76202]: cluster 2024-04-07T16:23:30.878911+0000 mon.a (mon.0) 558 : cluster [DBG] osdmap e460: 8 total, 7 up, 8 in 2024-04-07T16:23:32.124 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:31 smithi100 bash[76202]: audit 2024-04-07T16:23:31.716553+0000 mon.a (mon.0) 559 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:23:32.124 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:31 smithi100 bash[76202]: audit 2024-04-07T16:23:31.717499+0000 mon.a (mon.0) 560 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:23:32.124 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:31 smithi100 bash[76202]: cluster 2024-04-07T16:23:31.824105+0000 mon.a (mon.0) 561 : cluster [DBG] osdmap e461: 8 total, 7 up, 8 in 2024-04-07T16:23:32.304 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:31 smithi144 bash[66654]: cluster 2024-04-07T16:23:30.678006+0000 mgr.y (mgr.25175) 176 : cluster [DBG] pgmap v279: 121 pgs: 121 active+clean; 304 MiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 9.1 MiB/s rd, 3.3 MiB/s wr, 13 op/s 2024-04-07T16:23:32.304 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:31 smithi144 bash[66654]: cluster 2024-04-07T16:23:30.878911+0000 mon.a (mon.0) 558 : cluster [DBG] osdmap e460: 8 total, 7 up, 8 in 2024-04-07T16:23:32.304 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:31 smithi144 bash[66654]: audit 2024-04-07T16:23:31.716553+0000 mon.a (mon.0) 559 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:23:32.304 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:31 smithi144 bash[66654]: audit 2024-04-07T16:23:31.717499+0000 mon.a (mon.0) 560 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:23:32.304 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:31 smithi144 bash[66654]: cluster 2024-04-07T16:23:31.824105+0000 mon.a (mon.0) 561 : cluster [DBG] osdmap e461: 8 total, 7 up, 8 in 2024-04-07T16:23:32.400 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-04-07T16:23:32.400 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": { 2024-04-07T16:23:32.401 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph version 19.0.0-2785-gadce92d0 (adce92d056533f22dc1ee37e3141d098b046ec1b) squid (dev)": 3 2024-04-07T16:23:32.401 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-07T16:23:32.401 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": { 2024-04-07T16:23:32.401 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph version 19.0.0-2785-gadce92d0 (adce92d056533f22dc1ee37e3141d098b046ec1b) squid (dev)": 2 2024-04-07T16:23:32.401 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-07T16:23:32.401 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": { 2024-04-07T16:23:32.401 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph version 18.2.2-1171-g267c8ff5 (267c8ff5b1ff5815581b3b2381d19360e081b37b) reef (stable)": 4, 2024-04-07T16:23:32.401 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph version 19.0.0-2785-gadce92d0 (adce92d056533f22dc1ee37e3141d098b046ec1b) squid (dev)": 3 2024-04-07T16:23:32.401 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-07T16:23:32.401 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": { 2024-04-07T16:23:32.401 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph version 18.2.2-1171-g267c8ff5 (267c8ff5b1ff5815581b3b2381d19360e081b37b) reef (stable)": 2 2024-04-07T16:23:32.401 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-07T16:23:32.401 INFO:teuthology.orchestra.run.smithi100.stdout: "overall": { 2024-04-07T16:23:32.402 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph version 18.2.2-1171-g267c8ff5 (267c8ff5b1ff5815581b3b2381d19360e081b37b) reef (stable)": 6, 2024-04-07T16:23:32.402 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph version 19.0.0-2785-gadce92d0 (adce92d056533f22dc1ee37e3141d098b046ec1b) squid (dev)": 8 2024-04-07T16:23:32.402 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-04-07T16:23:32.402 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-04-07T16:23:32.433 INFO:journalctl@ceph.osd.3.smithi100.stdout:Apr 07 16:23:32 smithi100 bash[93470]: ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4-osd-3 2024-04-07T16:23:32.434 INFO:journalctl@ceph.osd.3.smithi100.stdout:Apr 07 16:23:32 smithi100 bash[93605]: Error response from daemon: No such container: ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4-osd-3 2024-04-07T16:23:32.606 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:23:32 smithi144 bash[59794]: ts=2024-04-07T16:23:32.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:23:32.808 INFO:tasks.rados.rados.0.smithi100.stdout:2078: done (6 left) 2024-04-07T16:23:32.808 INFO:tasks.rados.rados.0.smithi100.stdout:2079: delete oid 34 current snap is 202 2024-04-07T16:23:32.808 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 31 v 1059 (ObjNum 579 snap 180 seq_num 579) dirty exists 2024-04-07T16:23:32.808 INFO:tasks.rados.rados.0.smithi100.stdout:2073: finishing rollback tid 0 to smithi10066867-9 2024-04-07T16:23:32.808 INFO:tasks.rados.rados.0.smithi100.stdout:2072: finishing rollback tid 1 to smithi10066867-45 2024-04-07T16:23:32.808 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 45 v 1006 (ObjNum 33 snap 0 seq_num 2702118880) dirty dne 2024-04-07T16:23:32.808 INFO:tasks.rados.rados.0.smithi100.stdout:2074: expect (ObjNum 492 snap 148 seq_num 492) 2024-04-07T16:23:32.895 INFO:tasks.rados.rados.0.smithi100.stdout:2071: finishing write tid 1 to smithi10066867-8 2024-04-07T16:23:32.896 INFO:tasks.rados.rados.0.smithi100.stdout:2071: finishing write tid 2 to smithi10066867-8 2024-04-07T16:23:32.896 INFO:tasks.rados.rados.0.smithi100.stdout:2071: finishing write tid 3 to smithi10066867-8 2024-04-07T16:23:32.896 INFO:tasks.rados.rados.0.smithi100.stdout:2071: finishing write tid 4 to smithi10066867-8 2024-04-07T16:23:32.896 INFO:tasks.rados.rados.0.smithi100.stdout:2071: oid 8 updating version 0 to 797 2024-04-07T16:23:32.896 INFO:tasks.rados.rados.0.smithi100.stdout:2071: oid 8 updating version 797 to 798 2024-04-07T16:23:32.896 INFO:tasks.rados.rados.0.smithi100.stdout:2071: oid 8 updating version 798 to 799 2024-04-07T16:23:32.896 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 8 v 799 (ObjNum 641 snap 202 seq_num 641) dirty exists 2024-04-07T16:23:32.896 INFO:tasks.rados.rados.0.smithi100.stdout:2071: left oid 8 (ObjNum 641 snap 202 seq_num 641) 2024-04-07T16:23:32.896 INFO:tasks.rados.rados.0.smithi100.stdout:2077: finishing copy_from racing read to smithi10066867-24 2024-04-07T16:23:32.896 INFO:tasks.rados.rados.0.smithi100.stdout:2073: finishing rollback tid 1 to smithi10066867-9 2024-04-07T16:23:32.896 INFO:tasks.rados.rados.0.smithi100.stdout:2073: finishing rollback tid 2 to smithi10066867-9 2024-04-07T16:23:32.896 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 9 v 557 (ObjNum 622 snap 194 seq_num 622) dirty exists 2024-04-07T16:23:32.896 INFO:tasks.rados.rados.0.smithi100.stdout:2077: finishing copy_from to smithi10066867-24 2024-04-07T16:23:32.896 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 24 v 1058 (ObjNum 574 snap 177 seq_num 574) dirty exists 2024-04-07T16:23:32.896 INFO:tasks.rados.rados.0.smithi100.stdout:2071: done (6 left) 2024-04-07T16:23:32.897 INFO:tasks.rados.rados.0.smithi100.stdout:2072: done (5 left) 2024-04-07T16:23:32.897 INFO:tasks.rados.rados.0.smithi100.stdout:2073: done (4 left) 2024-04-07T16:23:32.897 INFO:tasks.rados.rados.0.smithi100.stdout:2074: done (3 left) 2024-04-07T16:23:32.897 INFO:tasks.rados.rados.0.smithi100.stdout:2075: done (2 left) 2024-04-07T16:23:32.897 INFO:tasks.rados.rados.0.smithi100.stdout:2077: done (1 left) 2024-04-07T16:23:32.897 INFO:tasks.rados.rados.0.smithi100.stdout:2079: done (0 left) 2024-04-07T16:23:32.897 INFO:tasks.rados.rados.0.smithi100.stdout:2080: snap_remove snap 201 2024-04-07T16:23:33.184 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:32 smithi100 bash[76202]: audit 2024-04-07T16:23:31.571118+0000 mgr.y (mgr.25175) 177 : audit [DBG] from='client.34290 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:23:33.184 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:32 smithi100 bash[76202]: audit 2024-04-07T16:23:31.958037+0000 mgr.y (mgr.25175) 178 : audit [DBG] from='client.44216 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:23:33.184 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:32 smithi100 bash[76202]: audit 2024-04-07T16:23:32.419435+0000 mon.a (mon.0) 562 : audit [DBG] from='client.? 172.21.15.100:0/1319102244' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:23:33.184 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:32 smithi100 bash[76202]: cluster 2024-04-07T16:23:32.825935+0000 mon.a (mon.0) 563 : cluster [DBG] osdmap e462: 8 total, 7 up, 8 in 2024-04-07T16:23:33.184 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:32 smithi100 bash[74942]: audit 2024-04-07T16:23:31.571118+0000 mgr.y (mgr.25175) 177 : audit [DBG] from='client.34290 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:23:33.184 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:32 smithi100 bash[74942]: audit 2024-04-07T16:23:31.958037+0000 mgr.y (mgr.25175) 178 : audit [DBG] from='client.44216 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:23:33.184 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:32 smithi100 bash[74942]: audit 2024-04-07T16:23:32.419435+0000 mon.a (mon.0) 562 : audit [DBG] from='client.? 172.21.15.100:0/1319102244' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:23:33.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:32 smithi100 bash[74942]: cluster 2024-04-07T16:23:32.825935+0000 mon.a (mon.0) 563 : cluster [DBG] osdmap e462: 8 total, 7 up, 8 in 2024-04-07T16:23:33.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:32 smithi144 bash[66654]: audit 2024-04-07T16:23:31.571118+0000 mgr.y (mgr.25175) 177 : audit [DBG] from='client.34290 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:23:33.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:32 smithi144 bash[66654]: audit 2024-04-07T16:23:31.958037+0000 mgr.y (mgr.25175) 178 : audit [DBG] from='client.44216 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:23:33.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:32 smithi144 bash[66654]: audit 2024-04-07T16:23:32.419435+0000 mon.a (mon.0) 562 : audit [DBG] from='client.? 172.21.15.100:0/1319102244' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:23:33.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:32 smithi144 bash[66654]: cluster 2024-04-07T16:23:32.825935+0000 mon.a (mon.0) 563 : cluster [DBG] osdmap e462: 8 total, 7 up, 8 in 2024-04-07T16:23:33.856 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:23:33 smithi144 bash[59794]: ts=2024-04-07T16:23:33.507Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=nodes name=CephNodeDiskspaceWarning index=4 msg="Evaluating rule failed" rule="alert: CephNodeDiskspaceWarning\nexpr: predict_linear(node_filesystem_free_bytes{device=~\"/.*\"}[2d], 3600 * 24 * 5)\n * on (instance) group_left (nodename) node_uname_info < 0\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.8.4\n severity: warning\n type: ceph_default\nannotations:\n description: Mountpoint {{ $labels.mountpoint }} on {{ $labels.nodename }} will\n be full in less than 5 days based on the 48 hour trailing fill rate.\n summary: Host filesystem free space is getting low\n" err="found duplicate series for the match group {instance=\"smithi100\"} on the right hand-side of the operation: [{__name__=\"node_uname_info\", domainname=\"(none)\", instance=\"smithi100\", job=\"node\", machine=\"x86_64\", nodename=\"smithi100\", release=\"5.15.0-101-generic\", sysname=\"Linux\", version=\"#111-Ubuntu SMP Tue Mar 5 20:16:58 UTC 2024\"}, {__name__=\"node_uname_info\", cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", domainname=\"(none)\", instance=\"smithi100\", job=\"node\", machine=\"x86_64\", nodename=\"smithi100\", release=\"5.15.0-101-generic\", sysname=\"Linux\", version=\"#111-Ubuntu SMP Tue Mar 5 20:16:58 UTC 2024\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:23:33.903 INFO:tasks.rados.rados.0.smithi100.stdout:2080: done (0 left) 2024-04-07T16:23:33.903 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 13 current snap is 202 2024-04-07T16:23:33.903 INFO:tasks.rados.rados.0.smithi100.stdout:2081: seq_num 643 ranges {2564096=778240,3342336=401408} 2024-04-07T16:23:33.958 INFO:tasks.rados.rados.0.smithi100.stdout:2081: writing smithi10066867-13 from 2564096 to 3342336 tid 1 2024-04-07T16:23:33.964 INFO:tasks.rados.rados.0.smithi100.stdout:2081: writing smithi10066867-13 from 3342336 to 3743744 tid 2 2024-04-07T16:23:33.965 INFO:tasks.rados.rados.0.smithi100.stdout:2082: copy_from oid 28 from oid 50 current snap is 202 2024-04-07T16:23:33.965 INFO:tasks.rados.rados.0.smithi100.stdout:2083: rollback oid 38 current snap is 202 2024-04-07T16:23:33.965 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 38 to 182 2024-04-07T16:23:33.965 INFO:tasks.rados.rados.0.smithi100.stdout:2084: read oid 20 snap -1 2024-04-07T16:23:33.965 INFO:tasks.rados.rados.0.smithi100.stdout:2084: expect (ObjNum 610 snap 188 seq_num 610) 2024-04-07T16:23:33.965 INFO:tasks.rados.rados.0.smithi100.stdout:2085: copy_from oid 37 from oid 49 current snap is 202 2024-04-07T16:23:33.965 INFO:tasks.rados.rados.0.smithi100.stdout:2086: read oid 11 snap -1 2024-04-07T16:23:33.965 INFO:tasks.rados.rados.0.smithi100.stdout:2086: expect (ObjNum 537 snap 162 seq_num 537) 2024-04-07T16:23:33.965 INFO:tasks.rados.rados.0.smithi100.stdout:2087: read oid 42 snap -1 2024-04-07T16:23:33.965 INFO:tasks.rados.rados.0.smithi100.stdout:2087: expect (ObjNum 565 snap 173 seq_num 565) 2024-04-07T16:23:33.965 INFO:tasks.rados.rados.0.smithi100.stdout:2088: rollback oid 30 current snap is 202 2024-04-07T16:23:33.965 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 30 to 172 2024-04-07T16:23:33.965 INFO:tasks.rados.rados.0.smithi100.stdout:2089: read oid 24 snap -1 2024-04-07T16:23:33.965 INFO:tasks.rados.rados.0.smithi100.stdout:2089: expect (ObjNum 574 snap 177 seq_num 574) 2024-04-07T16:23:33.965 INFO:tasks.rados.rados.0.smithi100.stdout:2090: read oid 48 snap -1 2024-04-07T16:23:33.966 INFO:tasks.rados.rados.0.smithi100.stdout:2090: expect (ObjNum 574 snap 177 seq_num 574) 2024-04-07T16:23:33.966 INFO:tasks.rados.rados.0.smithi100.stdout:2091: read oid 9 snap 182 2024-04-07T16:23:33.966 INFO:tasks.rados.rados.0.smithi100.stdout:2091: expect deleted 2024-04-07T16:23:33.966 INFO:tasks.rados.rados.0.smithi100.stdout:2092: rollback oid 17 current snap is 202 2024-04-07T16:23:33.966 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 17 to 191 2024-04-07T16:23:33.966 INFO:tasks.rados.rados.0.smithi100.stdout:2093: read oid 5 snap -1 2024-04-07T16:23:33.966 INFO:tasks.rados.rados.0.smithi100.stdout:2093: expect (ObjNum 468 snap 142 seq_num 468) 2024-04-07T16:23:33.966 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 27 current snap is 202 2024-04-07T16:23:33.966 INFO:tasks.rados.rados.0.smithi100.stdout:2094: seq_num 644 ranges {5775360=794624,6569984=401408} 2024-04-07T16:23:34.006 INFO:tasks.rados.rados.0.smithi100.stdout:2094: writing smithi10066867-27 from 5775360 to 6569984 tid 1 2024-04-07T16:23:34.010 INFO:tasks.rados.rados.0.smithi100.stdout:2094: writing smithi10066867-27 from 6569984 to 6971392 tid 2 2024-04-07T16:23:34.010 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 45 current snap is 202 2024-04-07T16:23:34.010 INFO:tasks.rados.rados.0.smithi100.stdout:2095: seq_num 645 ranges {0=409600,409600=778240,1187840=24576} 2024-04-07T16:23:34.012 INFO:tasks.rados.rados.0.smithi100.stdout:2095: writing smithi10066867-45 from 0 to 409600 tid 1 2024-04-07T16:23:34.018 INFO:tasks.rados.rados.0.smithi100.stdout:2095: writing smithi10066867-45 from 409600 to 1187840 tid 2 2024-04-07T16:23:34.019 INFO:tasks.rados.rados.0.smithi100.stdout:2095: writing smithi10066867-45 from 1187840 to 1212416 tid 3 2024-04-07T16:23:34.019 INFO:tasks.rados.rados.0.smithi100.stdout:2096: snap_create 2024-04-07T16:23:34.019 INFO:tasks.rados.rados.0.smithi100.stdout:2081: finishing write tid 1 to smithi10066867-13 2024-04-07T16:23:34.019 INFO:tasks.rados.rados.0.smithi100.stdout:2081: finishing write tid 2 to smithi10066867-13 2024-04-07T16:23:34.019 INFO:tasks.rados.rados.0.smithi100.stdout:2081: finishing write tid 3 to smithi10066867-13 2024-04-07T16:23:34.019 INFO:tasks.rados.rados.0.smithi100.stdout:2083: finishing rollback tid 0 to smithi10066867-38 2024-04-07T16:23:34.019 INFO:tasks.rados.rados.0.smithi100.stdout:2083: finishing rollback tid 1 to smithi10066867-38 2024-04-07T16:23:34.019 INFO:tasks.rados.rados.0.smithi100.stdout:2083: finishing rollback tid 2 to smithi10066867-38 2024-04-07T16:23:34.020 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 38 v 1092 (ObjNum 578 snap 179 seq_num 578) dirty exists 2024-04-07T16:23:34.020 INFO:tasks.rados.rados.0.smithi100.stdout:2081: finishing write tid 4 to smithi10066867-13 2024-04-07T16:23:34.020 INFO:tasks.rados.rados.0.smithi100.stdout:2081: oid 13 updating version 0 to 819 2024-04-07T16:23:34.020 INFO:tasks.rados.rados.0.smithi100.stdout:2081: oid 13 version 819 is already newer than 818 2024-04-07T16:23:34.020 INFO:tasks.rados.rados.0.smithi100.stdout:2081: oid 13 version 819 is already newer than 817 2024-04-07T16:23:34.020 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 13 v 819 (ObjNum 643 snap 202 seq_num 643) dirty exists 2024-04-07T16:23:34.020 INFO:tasks.rados.rados.0.smithi100.stdout:2081: left oid 13 (ObjNum 643 snap 202 seq_num 643) 2024-04-07T16:23:34.020 INFO:tasks.rados.rados.0.smithi100.stdout:2082: finishing copy_from racing read to smithi10066867-28 2024-04-07T16:23:34.020 INFO:tasks.rados.rados.0.smithi100.stdout:2086: expect (ObjNum 537 snap 162 seq_num 537) 2024-04-07T16:23:34.110 INFO:tasks.rados.rados.0.smithi100.stdout:2082: finishing copy_from to smithi10066867-28 2024-04-07T16:23:34.110 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 28 v 1009 (ObjNum 510 snap 152 seq_num 510) dirty exists 2024-04-07T16:23:34.111 INFO:tasks.rados.rados.0.smithi100.stdout:2092: finishing rollback tid 1 to smithi10066867-17 2024-04-07T16:23:34.111 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 17 v 747 (ObjNum 177 snap 0 seq_num 4238579567) dirty dne 2024-04-07T16:23:34.111 INFO:tasks.rados.rados.0.smithi100.stdout:2089: expect (ObjNum 574 snap 177 seq_num 574) 2024-04-07T16:23:34.184 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:33 smithi100 bash[74942]: cluster 2024-04-07T16:23:32.679200+0000 mgr.y (mgr.25175) 179 : cluster [DBG] pgmap v282: 121 pgs: 2 active+undersized+degraded, 5 stale+active+clean, 18 active+undersized, 96 active+clean; 305 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 7.3 MiB/s rd, 2.9 MiB/s wr, 14 op/s; 11/573 objects degraded (1.920%) 2024-04-07T16:23:34.184 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:33 smithi100 bash[74942]: cluster 2024-04-07T16:23:32.910736+0000 mon.a (mon.0) 564 : cluster [WRN] Health check failed: Degraded data redundancy: 11/573 objects degraded (1.920%), 2 pgs degraded (PG_DEGRADED) 2024-04-07T16:23:34.184 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:33 smithi100 bash[74942]: cluster 2024-04-07T16:23:33.920790+0000 mon.a (mon.0) 565 : cluster [DBG] osdmap e463: 8 total, 7 up, 8 in 2024-04-07T16:23:34.184 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:33 smithi100 bash[76202]: cluster 2024-04-07T16:23:32.679200+0000 mgr.y (mgr.25175) 179 : cluster [DBG] pgmap v282: 121 pgs: 2 active+undersized+degraded, 5 stale+active+clean, 18 active+undersized, 96 active+clean; 305 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 7.3 MiB/s rd, 2.9 MiB/s wr, 14 op/s; 11/573 objects degraded (1.920%) 2024-04-07T16:23:34.184 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:33 smithi100 bash[76202]: cluster 2024-04-07T16:23:32.910736+0000 mon.a (mon.0) 564 : cluster [WRN] Health check failed: Degraded data redundancy: 11/573 objects degraded (1.920%), 2 pgs degraded (PG_DEGRADED) 2024-04-07T16:23:34.184 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:33 smithi100 bash[76202]: cluster 2024-04-07T16:23:33.920790+0000 mon.a (mon.0) 565 : cluster [DBG] osdmap e463: 8 total, 7 up, 8 in 2024-04-07T16:23:34.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:33 smithi144 bash[66654]: cluster 2024-04-07T16:23:32.679200+0000 mgr.y (mgr.25175) 179 : cluster [DBG] pgmap v282: 121 pgs: 2 active+undersized+degraded, 5 stale+active+clean, 18 active+undersized, 96 active+clean; 305 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 7.3 MiB/s rd, 2.9 MiB/s wr, 14 op/s; 11/573 objects degraded (1.920%) 2024-04-07T16:23:34.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:33 smithi144 bash[66654]: cluster 2024-04-07T16:23:32.910736+0000 mon.a (mon.0) 564 : cluster [WRN] Health check failed: Degraded data redundancy: 11/573 objects degraded (1.920%), 2 pgs degraded (PG_DEGRADED) 2024-04-07T16:23:34.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:33 smithi144 bash[66654]: cluster 2024-04-07T16:23:33.920790+0000 mon.a (mon.0) 565 : cluster [DBG] osdmap e463: 8 total, 7 up, 8 in 2024-04-07T16:23:34.388 INFO:tasks.rados.rados.0.smithi100.stdout:2093: expect (ObjNum 468 snap 142 seq_num 468) 2024-04-07T16:23:34.582 INFO:journalctl@ceph.osd.0.smithi100.stdout:Apr 07 16:23:34 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:23:34.582 INFO:journalctl@ceph.osd.1.smithi100.stdout:Apr 07 16:23:34 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:23:34.582 INFO:journalctl@ceph.osd.2.smithi100.stdout:Apr 07 16:23:34 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:23:34.583 INFO:journalctl@ceph.mds.a.smithi100.stdout:Apr 07 16:23:34 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:23:34.583 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 16:23:34 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:23:34.583 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:34 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:23:34.584 INFO:journalctl@ceph.osd.3.smithi100.stdout:Apr 07 16:23:34 smithi100 systemd[1]: ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@osd.3.service: Deactivated successfully. 2024-04-07T16:23:34.584 INFO:journalctl@ceph.osd.3.smithi100.stdout:Apr 07 16:23:34 smithi100 systemd[1]: Stopped Ceph osd.3 for 0702e3a8-f4f8-11ee-b649-cb9ed24678a4. 2024-04-07T16:23:34.584 INFO:journalctl@ceph.osd.3.smithi100.stdout:Apr 07 16:23:34 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:23:34.584 INFO:journalctl@ceph.osd.3.smithi100.stdout:Apr 07 16:23:34 smithi100 systemd[1]: Started Ceph osd.3 for 0702e3a8-f4f8-11ee-b649-cb9ed24678a4. 2024-04-07T16:23:34.584 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Apr 07 16:23:34 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:23:34.584 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:23:34 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:23:34.585 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:34 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:23:34.923 INFO:tasks.rados.rados.0.smithi100.stdout:2095: finishing write tid 1 to smithi10066867-45 2024-04-07T16:23:34.923 INFO:tasks.rados.rados.0.smithi100.stdout:2095: finishing write tid 2 to smithi10066867-45 2024-04-07T16:23:34.923 INFO:tasks.rados.rados.0.smithi100.stdout:2095: finishing write tid 3 to smithi10066867-45 2024-04-07T16:23:34.923 INFO:tasks.rados.rados.0.smithi100.stdout:2095: finishing write tid 4 to smithi10066867-45 2024-04-07T16:23:34.923 INFO:tasks.rados.rados.0.smithi100.stdout:2095: finishing write tid 5 to smithi10066867-45 2024-04-07T16:23:34.923 INFO:tasks.rados.rados.0.smithi100.stdout:2095: oid 45 updating version 0 to 1011 2024-04-07T16:23:34.923 INFO:tasks.rados.rados.0.smithi100.stdout:2095: oid 45 updating version 1011 to 1013 2024-04-07T16:23:34.923 INFO:tasks.rados.rados.0.smithi100.stdout:2095: oid 45 updating version 1013 to 1014 2024-04-07T16:23:34.923 INFO:tasks.rados.rados.0.smithi100.stdout:2095: oid 45 version 1014 is already newer than 1012 2024-04-07T16:23:34.923 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 45 v 1014 (ObjNum 645 snap 202 seq_num 645) dirty exists 2024-04-07T16:23:34.923 INFO:tasks.rados.rados.0.smithi100.stdout:2095: left oid 45 (ObjNum 645 snap 202 seq_num 645) 2024-04-07T16:23:34.924 INFO:tasks.rados.rados.0.smithi100.stdout:2094: finishing write tid 1 to smithi10066867-27 2024-04-07T16:23:34.924 INFO:tasks.rados.rados.0.smithi100.stdout:2094: finishing write tid 2 to smithi10066867-27 2024-04-07T16:23:34.924 INFO:tasks.rados.rados.0.smithi100.stdout:2094: finishing write tid 3 to smithi10066867-27 2024-04-07T16:23:34.924 INFO:tasks.rados.rados.0.smithi100.stdout:2081: done (15 left) 2024-04-07T16:23:34.924 INFO:tasks.rados.rados.0.smithi100.stdout:2082: done (14 left) 2024-04-07T16:23:34.924 INFO:tasks.rados.rados.0.smithi100.stdout:2083: done (13 left) 2024-04-07T16:23:34.924 INFO:tasks.rados.rados.0.smithi100.stdout:2086: done (12 left) 2024-04-07T16:23:34.924 INFO:tasks.rados.rados.0.smithi100.stdout:2089: done (11 left) 2024-04-07T16:23:34.924 INFO:tasks.rados.rados.0.smithi100.stdout:2091: done (10 left) 2024-04-07T16:23:34.924 INFO:tasks.rados.rados.0.smithi100.stdout:2092: done (9 left) 2024-04-07T16:23:34.924 INFO:tasks.rados.rados.0.smithi100.stdout:2093: done (8 left) 2024-04-07T16:23:34.924 INFO:tasks.rados.rados.0.smithi100.stdout:2095: done (7 left) 2024-04-07T16:23:34.924 INFO:tasks.rados.rados.0.smithi100.stdout:2096: done (6 left) 2024-04-07T16:23:34.924 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 39 current snap is 203 2024-04-07T16:23:34.924 INFO:tasks.rados.rados.0.smithi100.stdout:2094: finishing write tid 4 to smithi10066867-27 2024-04-07T16:23:34.924 INFO:tasks.rados.rados.0.smithi100.stdout:2094: oid 27 updating version 0 to 1081 2024-04-07T16:23:34.924 INFO:tasks.rados.rados.0.smithi100.stdout:2094: oid 27 updating version 1081 to 1083 2024-04-07T16:23:34.925 INFO:tasks.rados.rados.0.smithi100.stdout:2094: oid 27 version 1083 is already newer than 1082 2024-04-07T16:23:34.925 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 27 v 1083 (ObjNum 644 snap 202 seq_num 644) dirty exists 2024-04-07T16:23:34.925 INFO:tasks.rados.rados.0.smithi100.stdout:2094: left oid 27 (ObjNum 644 snap 202 seq_num 644) 2024-04-07T16:23:34.925 INFO:tasks.rados.rados.0.smithi100.stdout:2085: finishing copy_from racing read to smithi10066867-37 2024-04-07T16:23:34.925 INFO:tasks.rados.rados.0.smithi100.stdout:2088: finishing rollback tid 0 to smithi10066867-30 2024-04-07T16:23:34.925 INFO:tasks.rados.rados.0.smithi100.stdout:2090: expect (ObjNum 574 snap 177 seq_num 574) 2024-04-07T16:23:35.192 INFO:tasks.rados.rados.0.smithi100.stdout:2087: expect (ObjNum 565 snap 173 seq_num 565) 2024-04-07T16:23:35.438 INFO:tasks.rados.rados.0.smithi100.stdout:2088: finishing rollback tid 1 to smithi10066867-30 2024-04-07T16:23:35.439 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 30 v 1084 (ObjNum 161 snap 0 seq_num 0) dirty dne 2024-04-07T16:23:35.439 INFO:tasks.rados.rados.0.smithi100.stdout:2085: finishing copy_from to smithi10066867-37 2024-04-07T16:23:35.439 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 37 v 1089 (ObjNum 630 snap 196 seq_num 630) dirty exists 2024-04-07T16:23:35.439 INFO:tasks.rados.rados.0.smithi100.stdout:2084: expect (ObjNum 610 snap 188 seq_num 610) 2024-04-07T16:23:35.684 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:35 smithi100 bash[74942]: audit 2024-04-07T16:23:34.639768+0000 mon.a (mon.0) 566 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:23:35.684 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:35 smithi100 bash[74942]: audit 2024-04-07T16:23:34.647577+0000 mon.a (mon.0) 567 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:23:35.684 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:35 smithi100 bash[74942]: audit 2024-04-07T16:23:34.666395+0000 mon.a (mon.0) 568 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:23:35.684 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:35 smithi100 bash[74942]: cluster 2024-04-07T16:23:34.679650+0000 mgr.y (mgr.25175) 180 : cluster [DBG] pgmap v285: 121 pgs: 6 active+undersized+degraded, 2 stale+active+clean, 26 active+undersized, 87 active+clean; 308 MiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 2.7 MiB/s rd, 3.1 MiB/s wr, 12 op/s; 52/576 objects degraded (9.028%) 2024-04-07T16:23:35.684 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:35 smithi100 bash[74942]: cluster 2024-04-07T16:23:34.931244+0000 mon.a (mon.0) 569 : cluster [DBG] osdmap e464: 8 total, 7 up, 8 in 2024-04-07T16:23:35.684 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:35 smithi100 bash[74942]: audit 2024-04-07T16:23:35.136990+0000 mon.a (mon.0) 570 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:23:35.684 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:35 smithi100 bash[74942]: audit 2024-04-07T16:23:35.143915+0000 mon.a (mon.0) 571 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:23:35.684 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:35 smithi100 bash[76202]: audit 2024-04-07T16:23:34.639768+0000 mon.a (mon.0) 566 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:23:35.685 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:35 smithi100 bash[76202]: audit 2024-04-07T16:23:34.647577+0000 mon.a (mon.0) 567 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:23:35.685 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:35 smithi100 bash[76202]: audit 2024-04-07T16:23:34.666395+0000 mon.a (mon.0) 568 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:23:35.685 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:35 smithi100 bash[76202]: cluster 2024-04-07T16:23:34.679650+0000 mgr.y (mgr.25175) 180 : cluster [DBG] pgmap v285: 121 pgs: 6 active+undersized+degraded, 2 stale+active+clean, 26 active+undersized, 87 active+clean; 308 MiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 2.7 MiB/s rd, 3.1 MiB/s wr, 12 op/s; 52/576 objects degraded (9.028%) 2024-04-07T16:23:35.685 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:35 smithi100 bash[76202]: cluster 2024-04-07T16:23:34.931244+0000 mon.a (mon.0) 569 : cluster [DBG] osdmap e464: 8 total, 7 up, 8 in 2024-04-07T16:23:35.685 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:35 smithi100 bash[76202]: audit 2024-04-07T16:23:35.136990+0000 mon.a (mon.0) 570 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:23:35.685 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:35 smithi100 bash[76202]: audit 2024-04-07T16:23:35.143915+0000 mon.a (mon.0) 571 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:23:36.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:35 smithi144 bash[66654]: audit 2024-04-07T16:23:34.639768+0000 mon.a (mon.0) 566 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:23:36.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:35 smithi144 bash[66654]: audit 2024-04-07T16:23:34.647577+0000 mon.a (mon.0) 567 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:23:36.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:35 smithi144 bash[66654]: audit 2024-04-07T16:23:34.666395+0000 mon.a (mon.0) 568 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:23:36.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:35 smithi144 bash[66654]: cluster 2024-04-07T16:23:34.679650+0000 mgr.y (mgr.25175) 180 : cluster [DBG] pgmap v285: 121 pgs: 6 active+undersized+degraded, 2 stale+active+clean, 26 active+undersized, 87 active+clean; 308 MiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 2.7 MiB/s rd, 3.1 MiB/s wr, 12 op/s; 52/576 objects degraded (9.028%) 2024-04-07T16:23:36.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:35 smithi144 bash[66654]: cluster 2024-04-07T16:23:34.931244+0000 mon.a (mon.0) 569 : cluster [DBG] osdmap e464: 8 total, 7 up, 8 in 2024-04-07T16:23:36.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:35 smithi144 bash[66654]: audit 2024-04-07T16:23:35.136990+0000 mon.a (mon.0) 570 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:23:36.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:35 smithi144 bash[66654]: audit 2024-04-07T16:23:35.143915+0000 mon.a (mon.0) 571 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:23:36.207 INFO:tasks.rados.rados.0.smithi100.stdout:2097: seq_num 646 ranges {0=425984,425984=655360,1081344=147456} 2024-04-07T16:23:36.212 INFO:tasks.rados.rados.0.smithi100.stdout:2097: writing smithi10066867-39 from 0 to 425984 tid 1 2024-04-07T16:23:36.221 INFO:tasks.rados.rados.0.smithi100.stdout:2097: writing smithi10066867-39 from 425984 to 1081344 tid 2 2024-04-07T16:23:36.223 INFO:tasks.rados.rados.0.smithi100.stdout:2097: writing smithi10066867-39 from 1081344 to 1228800 tid 3 2024-04-07T16:23:36.223 INFO:tasks.rados.rados.0.smithi100.stdout:2084: done (6 left) 2024-04-07T16:23:36.223 INFO:tasks.rados.rados.0.smithi100.stdout:2085: done (5 left) 2024-04-07T16:23:36.223 INFO:tasks.rados.rados.0.smithi100.stdout:2087: done (4 left) 2024-04-07T16:23:36.223 INFO:tasks.rados.rados.0.smithi100.stdout:2088: done (3 left) 2024-04-07T16:23:36.223 INFO:tasks.rados.rados.0.smithi100.stdout:2090: done (2 left) 2024-04-07T16:23:36.223 INFO:tasks.rados.rados.0.smithi100.stdout:2094: done (1 left) 2024-04-07T16:23:36.223 INFO:tasks.rados.rados.0.smithi100.stdout:2098: delete oid 41 current snap is 203 2024-04-07T16:23:36.223 INFO:tasks.rados.rados.0.smithi100.stdout:2097: finishing write tid 1 to smithi10066867-39 2024-04-07T16:23:36.227 INFO:tasks.rados.rados.0.smithi100.stdout:2098: done (1 left) 2024-04-07T16:23:36.228 INFO:tasks.rados.rados.0.smithi100.stdout:2099: read oid 6 snap 178 2024-04-07T16:23:36.228 INFO:tasks.rados.rados.0.smithi100.stdout:2099: expect (ObjNum 524 snap 156 seq_num 524) 2024-04-07T16:23:36.228 INFO:tasks.rados.rados.0.smithi100.stdout:2100: read oid 15 snap -1 2024-04-07T16:23:36.228 INFO:tasks.rados.rados.0.smithi100.stdout:2100: expect deleted 2024-04-07T16:23:36.228 INFO:tasks.rados.rados.0.smithi100.stdout:2101: setattr oid 48 current snap is 203 2024-04-07T16:23:36.228 INFO:tasks.rados.rados.0.smithi100.stdout:2097: finishing write tid 2 to smithi10066867-39 2024-04-07T16:23:36.228 INFO:tasks.rados.rados.0.smithi100.stdout:2097: finishing write tid 3 to smithi10066867-39 2024-04-07T16:23:36.228 INFO:tasks.rados.rados.0.smithi100.stdout:2097: finishing write tid 4 to smithi10066867-39 2024-04-07T16:23:36.231 INFO:tasks.rados.rados.0.smithi100.stdout:2102: rollback oid 46 current snap is 203 2024-04-07T16:23:36.231 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 46 to 168 2024-04-07T16:23:36.231 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 7 current snap is 203 2024-04-07T16:23:36.231 INFO:tasks.rados.rados.0.smithi100.stdout:2103: seq_num 648 ranges {6422528=458752,6881280=475136,7356416=327680} 2024-04-07T16:23:36.287 INFO:tasks.rados.rados.0.smithi100.stdout:2103: writing smithi10066867-7 from 6422528 to 6881280 tid 1 2024-04-07T16:23:36.290 INFO:tasks.rados.rados.0.smithi100.stdout:2103: writing smithi10066867-7 from 6881280 to 7356416 tid 2 2024-04-07T16:23:36.292 INFO:tasks.rados.rados.0.smithi100.stdout:2103: writing smithi10066867-7 from 7356416 to 7684096 tid 3 2024-04-07T16:23:36.293 INFO:tasks.rados.rados.0.smithi100.stdout:2104: copy_from oid 18 from oid 19 current snap is 203 2024-04-07T16:23:36.293 INFO:tasks.rados.rados.0.smithi100.stdout:2097: finishing write tid 5 to smithi10066867-39 2024-04-07T16:23:36.293 INFO:tasks.rados.rados.0.smithi100.stdout:2097: oid 39 updating version 0 to 1092 2024-04-07T16:23:36.293 INFO:tasks.rados.rados.0.smithi100.stdout:2097: oid 39 updating version 1092 to 1093 2024-04-07T16:23:36.293 INFO:tasks.rados.rados.0.smithi100.stdout:2097: oid 39 version 1093 is already newer than 1091 2024-04-07T16:23:36.293 INFO:tasks.rados.rados.0.smithi100.stdout:2097: oid 39 version 1093 is already newer than 1090 2024-04-07T16:23:36.293 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 39 v 1093 (ObjNum 646 snap 203 seq_num 646) dirty exists 2024-04-07T16:23:36.293 INFO:tasks.rados.rados.0.smithi100.stdout:2097: left oid 39 (ObjNum 646 snap 203 seq_num 646) 2024-04-07T16:23:36.293 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 48 v 1060 (ObjNum 574 snap 177 seq_num 574) dirty exists 2024-04-07T16:23:36.293 INFO:tasks.rados.rados.0.smithi100.stdout:2102: finishing rollback tid 0 to smithi10066867-46 2024-04-07T16:23:36.293 INFO:tasks.rados.rados.0.smithi100.stdout:2102: finishing rollback tid 1 to smithi10066867-46 2024-04-07T16:23:36.293 INFO:tasks.rados.rados.0.smithi100.stdout:2097: done (6 left) 2024-04-07T16:23:36.293 INFO:tasks.rados.rados.0.smithi100.stdout:2100: done (5 left) 2024-04-07T16:23:36.293 INFO:tasks.rados.rados.0.smithi100.stdout:2101: done (4 left) 2024-04-07T16:23:36.294 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 36 current snap is 203 2024-04-07T16:23:36.294 INFO:tasks.rados.rados.0.smithi100.stdout:2102: finishing rollback tid 2 to smithi10066867-46 2024-04-07T16:23:36.294 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 46 v 823 (ObjNum 545 snap 165 seq_num 545) dirty exists 2024-04-07T16:23:36.294 INFO:tasks.rados.rados.0.smithi100.stdout:2099: expect (ObjNum 524 snap 156 seq_num 524) 2024-04-07T16:23:36.434 INFO:journalctl@ceph.osd.3.smithi100.stdout:Apr 07 16:23:35 smithi100 bash[93756]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-07T16:23:36.434 INFO:journalctl@ceph.osd.3.smithi100.stdout:Apr 07 16:23:35 smithi100 bash[93756]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-3 --no-mon-config --dev /dev/mapper/vg_nvme-lv_1 2024-04-07T16:23:36.434 INFO:journalctl@ceph.osd.3.smithi100.stdout:Apr 07 16:23:35 smithi100 bash[93756]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_1 2024-04-07T16:23:36.434 INFO:journalctl@ceph.osd.3.smithi100.stdout:Apr 07 16:23:35 smithi100 bash[93756]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-0 2024-04-07T16:23:36.434 INFO:journalctl@ceph.osd.3.smithi100.stdout:Apr 07 16:23:35 smithi100 bash[93756]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-3/block 2024-04-07T16:23:36.434 INFO:journalctl@ceph.osd.3.smithi100.stdout:Apr 07 16:23:35 smithi100 bash[93756]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-07T16:23:36.434 INFO:journalctl@ceph.osd.3.smithi100.stdout:Apr 07 16:23:35 smithi100 bash[93756]: --> ceph-volume raw activate successful for osd ID: 3 2024-04-07T16:23:36.738 INFO:tasks.rados.rados.0.smithi100.stdout:2103: finishing write tid 1 to smithi10066867-7 2024-04-07T16:23:36.738 INFO:tasks.rados.rados.0.smithi100.stdout:2104: finishing copy_from to smithi10066867-18 2024-04-07T16:23:36.738 INFO:tasks.rados.rados.0.smithi100.stdout:2104: got expected ENOENT (src dne) 2024-04-07T16:23:36.738 INFO:tasks.rados.rados.0.smithi100.stdout:2104: finishing copy_from racing read to smithi10066867-18 2024-04-07T16:23:36.739 INFO:tasks.rados.rados.0.smithi100.stdout:2103: finishing write tid 2 to smithi10066867-7 2024-04-07T16:23:36.739 INFO:tasks.rados.rados.0.smithi100.stdout:2103: finishing write tid 3 to smithi10066867-7 2024-04-07T16:23:36.739 INFO:tasks.rados.rados.0.smithi100.stdout:2103: finishing write tid 4 to smithi10066867-7 2024-04-07T16:23:36.739 INFO:tasks.rados.rados.0.smithi100.stdout:2103: finishing write tid 5 to smithi10066867-7 2024-04-07T16:23:36.739 INFO:tasks.rados.rados.0.smithi100.stdout:2103: oid 7 updating version 0 to 1087 2024-04-07T16:23:36.739 INFO:tasks.rados.rados.0.smithi100.stdout:2103: oid 7 version 1087 is already newer than 1086 2024-04-07T16:23:36.739 INFO:tasks.rados.rados.0.smithi100.stdout:2103: oid 7 version 1087 is already newer than 1084 2024-04-07T16:23:36.739 INFO:tasks.rados.rados.0.smithi100.stdout:2103: oid 7 version 1087 is already newer than 1085 2024-04-07T16:23:36.739 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 7 v 1087 (ObjNum 648 snap 203 seq_num 648) dirty exists 2024-04-07T16:23:36.739 INFO:tasks.rados.rados.0.smithi100.stdout:2103: left oid 7 (ObjNum 648 snap 203 seq_num 648) 2024-04-07T16:23:36.739 INFO:tasks.rados.rados.0.smithi100.stdout:2105: seq_num 649 ranges {2031616=475136,2506752=761856,3268608=40960} 2024-04-07T16:23:36.752 INFO:tasks.rados.rados.0.smithi100.stdout:2105: writing smithi10066867-36 from 2031616 to 2506752 tid 1 2024-04-07T16:23:36.757 INFO:tasks.rados.rados.0.smithi100.stdout:2105: writing smithi10066867-36 from 2506752 to 3268608 tid 2 2024-04-07T16:23:36.757 INFO:tasks.rados.rados.0.smithi100.stdout:2105: writing smithi10066867-36 from 3268608 to 3309568 tid 3 2024-04-07T16:23:36.758 INFO:tasks.rados.rados.0.smithi100.stdout:2099: done (4 left) 2024-04-07T16:23:36.758 INFO:tasks.rados.rados.0.smithi100.stdout:2102: done (3 left) 2024-04-07T16:23:36.758 INFO:tasks.rados.rados.0.smithi100.stdout:2103: done (2 left) 2024-04-07T16:23:36.758 INFO:tasks.rados.rados.0.smithi100.stdout:2104: done (1 left) 2024-04-07T16:23:36.758 INFO:tasks.rados.rados.0.smithi100.stdout:2106: snap_remove snap 166 2024-04-07T16:23:36.803 INFO:tasks.rados.rados.0.smithi100.stdout:2106: done (1 left) 2024-04-07T16:23:36.803 INFO:tasks.rados.rados.0.smithi100.stdout:2107: snap_remove snap 182 2024-04-07T16:23:37.810 INFO:tasks.rados.rados.0.smithi100.stdout:2107: done (1 left) 2024-04-07T16:23:37.810 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 30 current snap is 203 2024-04-07T16:23:37.810 INFO:tasks.rados.rados.0.smithi100.stdout:2108: seq_num 650 ranges {0=491520,491520=647168,1138688=155648} 2024-04-07T16:23:37.811 INFO:journalctl@ceph.osd.3.smithi100.stdout:Apr 07 16:23:37 smithi100 bash[93998]: debug 2024-04-07T16:23:37.583+0000 7f6eae4c8700 1 -- 172.21.15.100:0/2359834140 <== mon.0 v2:172.21.15.100:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5597a5997680 con 0x5597a59d4c00 2024-04-07T16:23:37.817 INFO:tasks.rados.rados.0.smithi100.stdout:2108: writing smithi10066867-30 from 0 to 491520 tid 1 2024-04-07T16:23:37.827 INFO:tasks.rados.rados.0.smithi100.stdout:2108: writing smithi10066867-30 from 491520 to 1138688 tid 2 2024-04-07T16:23:37.829 INFO:tasks.rados.rados.0.smithi100.stdout:2108: writing smithi10066867-30 from 1138688 to 1294336 tid 3 2024-04-07T16:23:37.829 INFO:tasks.rados.rados.0.smithi100.stdout:2109: setattr oid 31 current snap is 203 2024-04-07T16:23:37.830 INFO:tasks.rados.rados.0.smithi100.stdout:2105: finishing write tid 1 to smithi10066867-36 2024-04-07T16:23:37.830 INFO:tasks.rados.rados.0.smithi100.stdout:2105: finishing write tid 2 to smithi10066867-36 2024-04-07T16:23:37.830 INFO:tasks.rados.rados.0.smithi100.stdout:2105: finishing write tid 3 to smithi10066867-36 2024-04-07T16:23:37.830 INFO:tasks.rados.rados.0.smithi100.stdout:2105: finishing write tid 4 to smithi10066867-36 2024-04-07T16:23:37.830 INFO:tasks.rados.rados.0.smithi100.stdout:2105: finishing write tid 5 to smithi10066867-36 2024-04-07T16:23:37.830 INFO:tasks.rados.rados.0.smithi100.stdout:2105: oid 36 updating version 0 to 1061 2024-04-07T16:23:37.830 INFO:tasks.rados.rados.0.smithi100.stdout:2105: oid 36 version 1061 is already newer than 1059 2024-04-07T16:23:37.830 INFO:tasks.rados.rados.0.smithi100.stdout:2105: oid 36 updating version 1061 to 1062 2024-04-07T16:23:37.830 INFO:tasks.rados.rados.0.smithi100.stdout:2105: oid 36 version 1062 is already newer than 1060 2024-04-07T16:23:37.830 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 36 v 1062 (ObjNum 649 snap 203 seq_num 649) dirty exists 2024-04-07T16:23:37.830 INFO:tasks.rados.rados.0.smithi100.stdout:2105: left oid 36 (ObjNum 649 snap 203 seq_num 649) 2024-04-07T16:23:37.833 INFO:tasks.rados.rados.0.smithi100.stdout:2108: finishing write tid 1 to smithi10066867-30 2024-04-07T16:23:37.833 INFO:tasks.rados.rados.0.smithi100.stdout:2105: done (2 left) 2024-04-07T16:23:37.833 INFO:tasks.rados.rados.0.smithi100.stdout:2110: rollback oid 11 current snap is 203 2024-04-07T16:23:37.833 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 11 to 195 2024-04-07T16:23:37.833 INFO:tasks.rados.rados.0.smithi100.stdout:2111: read oid 47 snap -1 2024-04-07T16:23:37.833 INFO:tasks.rados.rados.0.smithi100.stdout:2111: expect (ObjNum 556 snap 168 seq_num 556) 2024-04-07T16:23:37.833 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 16 current snap is 203 2024-04-07T16:23:37.833 INFO:tasks.rados.rados.0.smithi100.stdout:2112: seq_num 651 ranges {4177920=507904,4685824=532480,5218304=270336} 2024-04-07T16:23:37.883 INFO:tasks.rados.rados.0.smithi100.stdout:2112: writing smithi10066867-16 from 4177920 to 4685824 tid 1 2024-04-07T16:23:37.888 INFO:tasks.rados.rados.0.smithi100.stdout:2112: writing smithi10066867-16 from 4685824 to 5218304 tid 2 2024-04-07T16:23:37.890 INFO:tasks.rados.rados.0.smithi100.stdout:2112: writing smithi10066867-16 from 5218304 to 5488640 tid 3 2024-04-07T16:23:37.890 INFO:tasks.rados.rados.0.smithi100.stdout:2113: read oid 38 snap -1 2024-04-07T16:23:37.890 INFO:tasks.rados.rados.0.smithi100.stdout:2113: expect (ObjNum 578 snap 179 seq_num 578) 2024-04-07T16:23:37.890 INFO:tasks.rados.rados.0.smithi100.stdout:2114: read oid 4 snap -1 2024-04-07T16:23:37.891 INFO:tasks.rados.rados.0.smithi100.stdout:2114: expect deleted 2024-04-07T16:23:37.891 INFO:tasks.rados.rados.0.smithi100.stdout:2115: delete oid 29 current snap is 203 2024-04-07T16:23:37.891 INFO:tasks.rados.rados.0.smithi100.stdout:2108: finishing write tid 2 to smithi10066867-30 2024-04-07T16:23:37.891 INFO:tasks.rados.rados.0.smithi100.stdout:2108: finishing write tid 3 to smithi10066867-30 2024-04-07T16:23:37.891 INFO:tasks.rados.rados.0.smithi100.stdout:2108: finishing write tid 4 to smithi10066867-30 2024-04-07T16:23:37.891 INFO:tasks.rados.rados.0.smithi100.stdout:2110: finishing rollback tid 0 to smithi10066867-11 2024-04-07T16:23:37.891 INFO:tasks.rados.rados.0.smithi100.stdout:2110: finishing rollback tid 1 to smithi10066867-11 2024-04-07T16:23:37.891 INFO:tasks.rados.rados.0.smithi100.stdout:2110: finishing rollback tid 2 to smithi10066867-11 2024-04-07T16:23:37.891 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 11 v 1174 (ObjNum 537 snap 162 seq_num 537) dirty exists 2024-04-07T16:23:37.891 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 31 v 1061 (ObjNum 579 snap 180 seq_num 579) dirty exists 2024-04-07T16:23:37.891 INFO:tasks.rados.rados.0.smithi100.stdout:2108: finishing write tid 5 to smithi10066867-30 2024-04-07T16:23:37.891 INFO:tasks.rados.rados.0.smithi100.stdout:2108: oid 30 updating version 0 to 1097 2024-04-07T16:23:37.891 INFO:tasks.rados.rados.0.smithi100.stdout:2108: oid 30 version 1097 is already newer than 1094 2024-04-07T16:23:37.891 INFO:tasks.rados.rados.0.smithi100.stdout:2108: oid 30 version 1097 is already newer than 1095 2024-04-07T16:23:37.891 INFO:tasks.rados.rados.0.smithi100.stdout:2108: oid 30 version 1097 is already newer than 1096 2024-04-07T16:23:37.892 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 30 v 1097 (ObjNum 650 snap 203 seq_num 650) dirty exists 2024-04-07T16:23:37.892 INFO:tasks.rados.rados.0.smithi100.stdout:2108: left oid 30 (ObjNum 650 snap 203 seq_num 650) 2024-04-07T16:23:37.892 INFO:tasks.rados.rados.0.smithi100.stdout:2112: finishing write tid 1 to smithi10066867-16 2024-04-07T16:23:37.892 INFO:tasks.rados.rados.0.smithi100.stdout:2112: finishing write tid 2 to smithi10066867-16 2024-04-07T16:23:37.894 INFO:tasks.rados.rados.0.smithi100.stdout:2112: finishing write tid 3 to smithi10066867-16 2024-04-07T16:23:37.894 INFO:tasks.rados.rados.0.smithi100.stdout:2112: finishing write tid 4 to smithi10066867-16 2024-04-07T16:23:37.895 INFO:tasks.rados.rados.0.smithi100.stdout:2108: done (7 left) 2024-04-07T16:23:37.895 INFO:tasks.rados.rados.0.smithi100.stdout:2109: done (6 left) 2024-04-07T16:23:37.895 INFO:tasks.rados.rados.0.smithi100.stdout:2110: done (5 left) 2024-04-07T16:23:37.895 INFO:tasks.rados.rados.0.smithi100.stdout:2114: done (4 left) 2024-04-07T16:23:37.895 INFO:tasks.rados.rados.0.smithi100.stdout:2115: done (3 left) 2024-04-07T16:23:37.895 INFO:tasks.rados.rados.0.smithi100.stdout:2116: read oid 40 snap -1 2024-04-07T16:23:37.896 INFO:tasks.rados.rados.0.smithi100.stdout:2116: expect (ObjNum 509 snap 152 seq_num 509) 2024-04-07T16:23:37.896 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 9 current snap is 203 2024-04-07T16:23:37.896 INFO:tasks.rados.rados.0.smithi100.stdout:2117: seq_num 653 ranges {1499136=540672,2039808=696320,2736128=106496} 2024-04-07T16:23:37.908 INFO:tasks.rados.rados.0.smithi100.stdout:2117: writing smithi10066867-9 from 1499136 to 2039808 tid 1 2024-04-07T16:23:37.912 INFO:tasks.rados.rados.0.smithi100.stdout:2117: writing smithi10066867-9 from 2039808 to 2736128 tid 2 2024-04-07T16:23:37.914 INFO:tasks.rados.rados.0.smithi100.stdout:2117: writing smithi10066867-9 from 2736128 to 2842624 tid 3 2024-04-07T16:23:37.914 INFO:tasks.rados.rados.0.smithi100.stdout:2118: copy_from oid 5 from oid 35 current snap is 203 2024-04-07T16:23:37.914 INFO:tasks.rados.rados.0.smithi100.stdout:2112: finishing write tid 5 to smithi10066867-16 2024-04-07T16:23:37.914 INFO:tasks.rados.rados.0.smithi100.stdout:2112: oid 16 updating version 0 to 1093 2024-04-07T16:23:37.914 INFO:tasks.rados.rados.0.smithi100.stdout:2112: oid 16 version 1093 is already newer than 1090 2024-04-07T16:23:37.914 INFO:tasks.rados.rados.0.smithi100.stdout:2112: oid 16 version 1093 is already newer than 1092 2024-04-07T16:23:37.914 INFO:tasks.rados.rados.0.smithi100.stdout:2112: oid 16 version 1093 is already newer than 1091 2024-04-07T16:23:37.914 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 16 v 1093 (ObjNum 651 snap 203 seq_num 651) dirty exists 2024-04-07T16:23:37.914 INFO:tasks.rados.rados.0.smithi100.stdout:2112: left oid 16 (ObjNum 651 snap 203 seq_num 651) 2024-04-07T16:23:37.914 INFO:tasks.rados.rados.0.smithi100.stdout:2111: expect (ObjNum 556 snap 168 seq_num 556) 2024-04-07T16:23:38.106 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:37 smithi100 bash[74942]: cluster 2024-04-07T16:23:36.680077+0000 mgr.y (mgr.25175) 181 : cluster [DBG] pgmap v287: 121 pgs: 8 active+undersized+degraded, 32 active+undersized, 81 active+clean; 310 MiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 2.2 MiB/s rd, 3.0 MiB/s wr, 11 op/s; 60/582 objects degraded (10.309%) 2024-04-07T16:23:38.106 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:37 smithi100 bash[74942]: cluster 2024-04-07T16:23:36.822379+0000 mon.a (mon.0) 572 : cluster [DBG] osdmap e465: 8 total, 7 up, 8 in 2024-04-07T16:23:38.106 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:37 smithi100 bash[76202]: cluster 2024-04-07T16:23:36.680077+0000 mgr.y (mgr.25175) 181 : cluster [DBG] pgmap v287: 121 pgs: 8 active+undersized+degraded, 32 active+undersized, 81 active+clean; 310 MiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 2.2 MiB/s rd, 3.0 MiB/s wr, 11 op/s; 60/582 objects degraded (10.309%) 2024-04-07T16:23:38.107 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:37 smithi100 bash[76202]: cluster 2024-04-07T16:23:36.822379+0000 mon.a (mon.0) 572 : cluster [DBG] osdmap e465: 8 total, 7 up, 8 in 2024-04-07T16:23:38.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:37 smithi144 bash[66654]: cluster 2024-04-07T16:23:36.680077+0000 mgr.y (mgr.25175) 181 : cluster [DBG] pgmap v287: 121 pgs: 8 active+undersized+degraded, 32 active+undersized, 81 active+clean; 310 MiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 2.2 MiB/s rd, 3.0 MiB/s wr, 11 op/s; 60/582 objects degraded (10.309%) 2024-04-07T16:23:38.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:37 smithi144 bash[66654]: cluster 2024-04-07T16:23:36.822379+0000 mon.a (mon.0) 572 : cluster [DBG] osdmap e465: 8 total, 7 up, 8 in 2024-04-07T16:23:38.399 INFO:tasks.rados.rados.0.smithi100.stdout:2113: expect (ObjNum 578 snap 179 seq_num 578) 2024-04-07T16:23:38.416 INFO:journalctl@ceph.osd.3.smithi100.stdout:Apr 07 16:23:38 smithi100 bash[93998]: debug 2024-04-07T16:23:38.123+0000 7f6eb76607c0 -1 Falling back to public interface 2024-04-07T16:23:38.454 INFO:tasks.rados.rados.0.smithi100.stdout:2117: finishing write tid 1 to smithi10066867-9 2024-04-07T16:23:38.454 INFO:tasks.rados.rados.0.smithi100.stdout:2117: finishing write tid 2 to smithi10066867-9 2024-04-07T16:23:38.454 INFO:tasks.rados.rados.0.smithi100.stdout:2117: finishing write tid 3 to smithi10066867-9 2024-04-07T16:23:38.454 INFO:tasks.rados.rados.0.smithi100.stdout:2117: finishing write tid 4 to smithi10066867-9 2024-04-07T16:23:38.454 INFO:tasks.rados.rados.0.smithi100.stdout:2116: expect (ObjNum 509 snap 152 seq_num 509) 2024-04-07T16:23:38.684 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:23:38 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:23:38] "GET /metrics HTTP/1.1" 200 36208 "" "Prometheus/2.43.0" 2024-04-07T16:23:38.826 INFO:tasks.rados.rados.0.smithi100.stdout:2117: finishing write tid 5 to smithi10066867-9 2024-04-07T16:23:38.826 INFO:tasks.rados.rados.0.smithi100.stdout:2117: oid 9 updating version 0 to 559 2024-04-07T16:23:38.826 INFO:tasks.rados.rados.0.smithi100.stdout:2117: oid 9 version 559 is already newer than 558 2024-04-07T16:23:38.826 INFO:tasks.rados.rados.0.smithi100.stdout:2117: oid 9 updating version 559 to 561 2024-04-07T16:23:38.826 INFO:tasks.rados.rados.0.smithi100.stdout:2117: oid 9 version 561 is already newer than 560 2024-04-07T16:23:38.827 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 9 v 561 (ObjNum 653 snap 203 seq_num 653) dirty exists 2024-04-07T16:23:38.827 INFO:tasks.rados.rados.0.smithi100.stdout:2117: left oid 9 (ObjNum 653 snap 203 seq_num 653) 2024-04-07T16:23:38.827 INFO:tasks.rados.rados.0.smithi100.stdout:2118: finishing copy_from racing read to smithi10066867-5 2024-04-07T16:23:38.827 INFO:tasks.rados.rados.0.smithi100.stdout:2118: finishing copy_from to smithi10066867-5 2024-04-07T16:23:38.827 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 5 v 998 (ObjNum 510 snap 152 seq_num 510) dirty exists 2024-04-07T16:23:38.827 INFO:tasks.rados.rados.0.smithi100.stdout:2111: done (5 left) 2024-04-07T16:23:38.827 INFO:tasks.rados.rados.0.smithi100.stdout:2112: done (4 left) 2024-04-07T16:23:38.827 INFO:tasks.rados.rados.0.smithi100.stdout:2113: done (3 left) 2024-04-07T16:23:38.827 INFO:tasks.rados.rados.0.smithi100.stdout:2116: done (2 left) 2024-04-07T16:23:38.827 INFO:tasks.rados.rados.0.smithi100.stdout:2117: done (1 left) 2024-04-07T16:23:38.827 INFO:tasks.rados.rados.0.smithi100.stdout:2118: done (0 left) 2024-04-07T16:23:38.827 INFO:tasks.rados.rados.0.smithi100.stdout:2119: snap_remove snap 200 2024-04-07T16:23:39.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:38 smithi144 bash[66654]: cluster 2024-04-07T16:23:37.827576+0000 mon.a (mon.0) 573 : cluster [DBG] osdmap e466: 8 total, 7 up, 8 in 2024-04-07T16:23:39.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:38 smithi100 bash[74942]: cluster 2024-04-07T16:23:37.827576+0000 mon.a (mon.0) 573 : cluster [DBG] osdmap e466: 8 total, 7 up, 8 in 2024-04-07T16:23:39.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:38 smithi100 bash[76202]: cluster 2024-04-07T16:23:37.827576+0000 mon.a (mon.0) 573 : cluster [DBG] osdmap e466: 8 total, 7 up, 8 in 2024-04-07T16:23:39.829 INFO:tasks.rados.rados.0.smithi100.stdout:2119: done (0 left) 2024-04-07T16:23:39.829 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 33 current snap is 203 2024-04-07T16:23:39.829 INFO:tasks.rados.rados.0.smithi100.stdout:2120: seq_num 654 ranges {0=557056,557056=630784,1187840=172032} 2024-04-07T16:23:39.836 INFO:tasks.rados.rados.0.smithi100.stdout:2120: writing smithi10066867-33 from 0 to 557056 tid 1 2024-04-07T16:23:39.842 INFO:tasks.rados.rados.0.smithi100.stdout:2120: writing smithi10066867-33 from 557056 to 1187840 tid 2 2024-04-07T16:23:39.844 INFO:tasks.rados.rados.0.smithi100.stdout:2120: writing smithi10066867-33 from 1187840 to 1359872 tid 3 2024-04-07T16:23:39.844 INFO:tasks.rados.rados.0.smithi100.stdout:2121: snap_remove snap 168 2024-04-07T16:23:40.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:39 smithi144 bash[66654]: cluster 2024-04-07T16:23:38.680977+0000 mgr.y (mgr.25175) 182 : cluster [DBG] pgmap v290: 121 pgs: 8 active+undersized+degraded, 32 active+undersized, 81 active+clean; 327 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 7.3 MiB/s rd, 3.5 MiB/s wr, 11 op/s; 62/579 objects degraded (10.708%) 2024-04-07T16:23:40.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:39 smithi144 bash[66654]: cluster 2024-04-07T16:23:38.830281+0000 mon.a (mon.0) 574 : cluster [WRN] Health check update: Degraded data redundancy: 62/579 objects degraded (10.708%), 8 pgs degraded (PG_DEGRADED) 2024-04-07T16:23:40.184 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:39 smithi100 bash[74942]: cluster 2024-04-07T16:23:38.680977+0000 mgr.y (mgr.25175) 182 : cluster [DBG] pgmap v290: 121 pgs: 8 active+undersized+degraded, 32 active+undersized, 81 active+clean; 327 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 7.3 MiB/s rd, 3.5 MiB/s wr, 11 op/s; 62/579 objects degraded (10.708%) 2024-04-07T16:23:40.184 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:39 smithi100 bash[74942]: cluster 2024-04-07T16:23:38.830281+0000 mon.a (mon.0) 574 : cluster [WRN] Health check update: Degraded data redundancy: 62/579 objects degraded (10.708%), 8 pgs degraded (PG_DEGRADED) 2024-04-07T16:23:40.184 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:39 smithi100 bash[76202]: cluster 2024-04-07T16:23:38.680977+0000 mgr.y (mgr.25175) 182 : cluster [DBG] pgmap v290: 121 pgs: 8 active+undersized+degraded, 32 active+undersized, 81 active+clean; 327 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 7.3 MiB/s rd, 3.5 MiB/s wr, 11 op/s; 62/579 objects degraded (10.708%) 2024-04-07T16:23:40.184 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:39 smithi100 bash[76202]: cluster 2024-04-07T16:23:38.830281+0000 mon.a (mon.0) 574 : cluster [WRN] Health check update: Degraded data redundancy: 62/579 objects degraded (10.708%), 8 pgs degraded (PG_DEGRADED) 2024-04-07T16:23:40.847 INFO:tasks.rados.rados.0.smithi100.stdout:2121: done (1 left) 2024-04-07T16:23:40.847 INFO:tasks.rados.rados.0.smithi100.stdout:2122: read oid 16 snap -1 2024-04-07T16:23:40.848 INFO:tasks.rados.rados.0.smithi100.stdout:2122: expect (ObjNum 651 snap 203 seq_num 651) 2024-04-07T16:23:40.848 INFO:tasks.rados.rados.0.smithi100.stdout:2123: copy_from oid 22 from oid 49 current snap is 203 2024-04-07T16:23:40.848 INFO:tasks.rados.rados.0.smithi100.stdout:2120: finishing write tid 1 to smithi10066867-33 2024-04-07T16:23:40.848 INFO:tasks.rados.rados.0.smithi100.stdout:2120: finishing write tid 2 to smithi10066867-33 2024-04-07T16:23:40.848 INFO:tasks.rados.rados.0.smithi100.stdout:2120: finishing write tid 3 to smithi10066867-33 2024-04-07T16:23:40.848 INFO:tasks.rados.rados.0.smithi100.stdout:2120: finishing write tid 4 to smithi10066867-33 2024-04-07T16:23:40.848 INFO:tasks.rados.rados.0.smithi100.stdout:2120: finishing write tid 5 to smithi10066867-33 2024-04-07T16:23:40.848 INFO:tasks.rados.rados.0.smithi100.stdout:2120: oid 33 updating version 0 to 1178 2024-04-07T16:23:40.848 INFO:tasks.rados.rados.0.smithi100.stdout:2120: oid 33 version 1178 is already newer than 1177 2024-04-07T16:23:40.848 INFO:tasks.rados.rados.0.smithi100.stdout:2120: oid 33 version 1178 is already newer than 1175 2024-04-07T16:23:40.848 INFO:tasks.rados.rados.0.smithi100.stdout:2120: oid 33 version 1178 is already newer than 1176 2024-04-07T16:23:40.848 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 33 v 1178 (ObjNum 654 snap 203 seq_num 654) dirty exists 2024-04-07T16:23:40.848 INFO:tasks.rados.rados.0.smithi100.stdout:2120: left oid 33 (ObjNum 654 snap 203 seq_num 654) 2024-04-07T16:23:40.849 INFO:tasks.rados.rados.0.smithi100.stdout:2120: done (2 left) 2024-04-07T16:23:40.849 INFO:tasks.rados.rados.0.smithi100.stdout:2124: copy_from oid 20 from oid 14 current snap is 203 2024-04-07T16:23:40.849 INFO:tasks.rados.rados.0.smithi100.stdout:2125: read oid 12 snap 199 2024-04-07T16:23:40.849 INFO:tasks.rados.rados.0.smithi100.stdout:2125: expect (ObjNum 592 snap 183 seq_num 592) 2024-04-07T16:23:40.849 INFO:tasks.rados.rados.0.smithi100.stdout:2126: delete oid 31 current snap is 203 2024-04-07T16:23:40.861 INFO:tasks.rados.rados.0.smithi100.stdout:2126: done (4 left) 2024-04-07T16:23:40.861 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 47 current snap is 203 2024-04-07T16:23:40.861 INFO:tasks.rados.rados.0.smithi100.stdout:2127: seq_num 656 ranges {8937472=589824,9527296=802816} 2024-04-07T16:23:40.916 INFO:tasks.rados.rados.0.smithi100.stdout:2127: writing smithi10066867-47 from 8937472 to 9527296 tid 1 2024-04-07T16:23:40.920 INFO:tasks.rados.rados.0.smithi100.stdout:2127: writing smithi10066867-47 from 9527296 to 10330112 tid 2 2024-04-07T16:23:40.921 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 50 current snap is 203 2024-04-07T16:23:40.921 INFO:tasks.rados.rados.0.smithi100.stdout:2124: finishing copy_from to smithi10066867-20 2024-04-07T16:23:40.921 INFO:tasks.rados.rados.0.smithi100.stdout:2124: got expected ENOENT (src dne) 2024-04-07T16:23:40.921 INFO:tasks.rados.rados.0.smithi100.stdout:2128: seq_num 657 ranges {3801088=606208,4407296=688128,5095424=114688} 2024-04-07T16:23:40.945 INFO:tasks.rados.rados.0.smithi100.stdout:2128: writing smithi10066867-50 from 3801088 to 4407296 tid 1 2024-04-07T16:23:40.949 INFO:tasks.rados.rados.0.smithi100.stdout:2128: writing smithi10066867-50 from 4407296 to 5095424 tid 2 2024-04-07T16:23:40.950 INFO:tasks.rados.rados.0.smithi100.stdout:2128: writing smithi10066867-50 from 5095424 to 5210112 tid 3 2024-04-07T16:23:40.950 INFO:tasks.rados.rados.0.smithi100.stdout:2129: delete oid 9 current snap is 203 2024-04-07T16:23:40.950 INFO:tasks.rados.rados.0.smithi100.stdout:2124: finishing copy_from racing read to smithi10066867-20 2024-04-07T16:23:40.950 INFO:tasks.rados.rados.0.smithi100.stdout:2122: expect (ObjNum 651 snap 203 seq_num 651) 2024-04-07T16:23:41.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:40 smithi144 bash[66654]: cluster 2024-04-07T16:23:39.847222+0000 mon.a (mon.0) 575 : cluster [DBG] osdmap e467: 8 total, 7 up, 8 in 2024-04-07T16:23:41.184 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:40 smithi100 bash[74942]: cluster 2024-04-07T16:23:39.847222+0000 mon.a (mon.0) 575 : cluster [DBG] osdmap e467: 8 total, 7 up, 8 in 2024-04-07T16:23:41.184 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:40 smithi100 bash[76202]: cluster 2024-04-07T16:23:39.847222+0000 mon.a (mon.0) 575 : cluster [DBG] osdmap e467: 8 total, 7 up, 8 in 2024-04-07T16:23:41.314 INFO:tasks.rados.rados.0.smithi100.stdout:2125: expect (ObjNum 592 snap 183 seq_num 592) 2024-04-07T16:23:41.589 INFO:tasks.rados.rados.0.smithi100.stdout:2123: finishing copy_from racing read to smithi10066867-22 2024-04-07T16:23:41.589 INFO:tasks.rados.rados.0.smithi100.stdout:2123: finishing copy_from to smithi10066867-22 2024-04-07T16:23:41.589 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 22 v 1089 (ObjNum 630 snap 196 seq_num 630) dirty exists 2024-04-07T16:23:41.589 INFO:tasks.rados.rados.0.smithi100.stdout:2122: done (6 left) 2024-04-07T16:23:41.590 INFO:tasks.rados.rados.0.smithi100.stdout:2123: done (5 left) 2024-04-07T16:23:41.590 INFO:tasks.rados.rados.0.smithi100.stdout:2124: done (4 left) 2024-04-07T16:23:41.590 INFO:tasks.rados.rados.0.smithi100.stdout:2125: done (3 left) 2024-04-07T16:23:41.590 INFO:tasks.rados.rados.0.smithi100.stdout:2129: done (2 left) 2024-04-07T16:23:41.590 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 2 current snap is 203 2024-04-07T16:23:41.590 INFO:tasks.rados.rados.0.smithi100.stdout:2127: finishing write tid 1 to smithi10066867-47 2024-04-07T16:23:41.590 INFO:tasks.rados.rados.0.smithi100.stdout:2127: finishing write tid 2 to smithi10066867-47 2024-04-07T16:23:41.590 INFO:tasks.rados.rados.0.smithi100.stdout:2127: finishing write tid 3 to smithi10066867-47 2024-04-07T16:23:41.590 INFO:tasks.rados.rados.0.smithi100.stdout:2127: finishing write tid 4 to smithi10066867-47 2024-04-07T16:23:41.590 INFO:tasks.rados.rados.0.smithi100.stdout:2127: oid 47 updating version 0 to 810 2024-04-07T16:23:41.590 INFO:tasks.rados.rados.0.smithi100.stdout:2127: oid 47 updating version 810 to 812 2024-04-07T16:23:41.590 INFO:tasks.rados.rados.0.smithi100.stdout:2127: oid 47 version 812 is already newer than 811 2024-04-07T16:23:41.590 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 47 v 812 (ObjNum 656 snap 203 seq_num 656) dirty exists 2024-04-07T16:23:41.590 INFO:tasks.rados.rados.0.smithi100.stdout:2127: left oid 47 (ObjNum 656 snap 203 seq_num 656) 2024-04-07T16:23:41.590 INFO:tasks.rados.rados.0.smithi100.stdout:2128: finishing write tid 1 to smithi10066867-50 2024-04-07T16:23:41.591 INFO:tasks.rados.rados.0.smithi100.stdout:2128: finishing write tid 2 to smithi10066867-50 2024-04-07T16:23:41.591 INFO:tasks.rados.rados.0.smithi100.stdout:2128: finishing write tid 3 to smithi10066867-50 2024-04-07T16:23:41.591 INFO:tasks.rados.rados.0.smithi100.stdout:2130: seq_num 659 ranges {876544=647168,1523712=450560,1974272=352256} 2024-04-07T16:23:41.598 INFO:tasks.rados.rados.0.smithi100.stdout:2130: writing smithi10066867-2 from 876544 to 1523712 tid 1 2024-04-07T16:23:41.603 INFO:tasks.rados.rados.0.smithi100.stdout:2130: writing smithi10066867-2 from 1523712 to 1974272 tid 2 2024-04-07T16:23:41.606 INFO:tasks.rados.rados.0.smithi100.stdout:2130: writing smithi10066867-2 from 1974272 to 2326528 tid 3 2024-04-07T16:23:41.606 INFO:tasks.rados.rados.0.smithi100.stdout:2127: done (2 left) 2024-04-07T16:23:41.606 INFO:tasks.rados.rados.0.smithi100.stdout:2131: copy_from oid 32 from oid 22 current snap is 203 2024-04-07T16:23:41.606 INFO:tasks.rados.rados.0.smithi100.stdout:2128: finishing write tid 4 to smithi10066867-50 2024-04-07T16:23:41.606 INFO:tasks.rados.rados.0.smithi100.stdout:2128: finishing write tid 5 to smithi10066867-50 2024-04-07T16:23:41.606 INFO:tasks.rados.rados.0.smithi100.stdout:2128: oid 50 updating version 0 to 1016 2024-04-07T16:23:41.606 INFO:tasks.rados.rados.0.smithi100.stdout:2128: oid 50 version 1016 is already newer than 1015 2024-04-07T16:23:41.606 INFO:tasks.rados.rados.0.smithi100.stdout:2128: oid 50 updating version 1016 to 1017 2024-04-07T16:23:41.606 INFO:tasks.rados.rados.0.smithi100.stdout:2128: oid 50 updating version 1017 to 1018 2024-04-07T16:23:41.606 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 50 v 1018 (ObjNum 657 snap 203 seq_num 657) dirty exists 2024-04-07T16:23:41.606 INFO:tasks.rados.rados.0.smithi100.stdout:2128: left oid 50 (ObjNum 657 snap 203 seq_num 657) 2024-04-07T16:23:41.606 INFO:tasks.rados.rados.0.smithi100.stdout:2130: finishing write tid 1 to smithi10066867-2 2024-04-07T16:23:41.607 INFO:tasks.rados.rados.0.smithi100.stdout:2128: done (2 left) 2024-04-07T16:23:41.607 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 10 current snap is 203 2024-04-07T16:23:41.607 INFO:tasks.rados.rados.0.smithi100.stdout:2132: seq_num 660 ranges {3817472=663552,4481024=737280,5218304=65536} 2024-04-07T16:23:41.631 INFO:tasks.rados.rados.0.smithi100.stdout:2132: writing smithi10066867-10 from 3817472 to 4481024 tid 1 2024-04-07T16:23:41.636 INFO:tasks.rados.rados.0.smithi100.stdout:2132: writing smithi10066867-10 from 4481024 to 5218304 tid 2 2024-04-07T16:23:41.636 INFO:tasks.rados.rados.0.smithi100.stdout:2132: writing smithi10066867-10 from 5218304 to 5283840 tid 3 2024-04-07T16:23:41.636 INFO:tasks.rados.rados.0.smithi100.stdout:2133: copy_from oid 19 from oid 47 current snap is 203 2024-04-07T16:23:41.637 INFO:tasks.rados.rados.0.smithi100.stdout:2130: finishing write tid 2 to smithi10066867-2 2024-04-07T16:23:41.637 INFO:tasks.rados.rados.0.smithi100.stdout:2134: delete oid 13 current snap is 203 2024-04-07T16:23:41.637 INFO:tasks.rados.rados.0.smithi100.stdout:2130: finishing write tid 3 to smithi10066867-2 2024-04-07T16:23:41.637 INFO:tasks.rados.rados.0.smithi100.stdout:2130: finishing write tid 4 to smithi10066867-2 2024-04-07T16:23:41.637 INFO:tasks.rados.rados.0.smithi100.stdout:2130: finishing write tid 5 to smithi10066867-2 2024-04-07T16:23:41.637 INFO:tasks.rados.rados.0.smithi100.stdout:2130: oid 2 updating version 0 to 1105 2024-04-07T16:23:41.637 INFO:tasks.rados.rados.0.smithi100.stdout:2130: oid 2 version 1105 is already newer than 1102 2024-04-07T16:23:41.637 INFO:tasks.rados.rados.0.smithi100.stdout:2130: oid 2 version 1105 is already newer than 1103 2024-04-07T16:23:41.637 INFO:tasks.rados.rados.0.smithi100.stdout:2130: oid 2 version 1105 is already newer than 1104 2024-04-07T16:23:41.637 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 2 v 1105 (ObjNum 659 snap 203 seq_num 659) dirty exists 2024-04-07T16:23:41.637 INFO:tasks.rados.rados.0.smithi100.stdout:2130: left oid 2 (ObjNum 659 snap 203 seq_num 659) 2024-04-07T16:23:41.637 INFO:tasks.rados.rados.0.smithi100.stdout:2131: finishing copy_from racing read to smithi10066867-32 2024-04-07T16:23:41.637 INFO:tasks.rados.rados.0.smithi100.stdout:2131: finishing copy_from to smithi10066867-32 2024-04-07T16:23:41.638 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 32 v 1089 (ObjNum 630 snap 196 seq_num 630) dirty exists 2024-04-07T16:23:41.638 INFO:tasks.rados.rados.0.smithi100.stdout:2132: finishing write tid 1 to smithi10066867-10 2024-04-07T16:23:41.641 INFO:tasks.rados.rados.0.smithi100.stdout:2132: finishing write tid 2 to smithi10066867-10 2024-04-07T16:23:41.641 INFO:tasks.rados.rados.0.smithi100.stdout:2132: finishing write tid 3 to smithi10066867-10 2024-04-07T16:23:41.642 INFO:tasks.rados.rados.0.smithi100.stdout:2132: finishing write tid 4 to smithi10066867-10 2024-04-07T16:23:41.642 INFO:tasks.rados.rados.0.smithi100.stdout:2130: done (4 left) 2024-04-07T16:23:41.642 INFO:tasks.rados.rados.0.smithi100.stdout:2131: done (3 left) 2024-04-07T16:23:41.642 INFO:tasks.rados.rados.0.smithi100.stdout:2134: done (2 left) 2024-04-07T16:23:41.642 INFO:tasks.rados.rados.0.smithi100.stdout:2135: snap_remove snap 170 2024-04-07T16:23:41.851 INFO:tasks.rados.rados.0.smithi100.stdout:2135: done (2 left) 2024-04-07T16:23:41.851 INFO:tasks.rados.rados.0.smithi100.stdout:2136: snap_remove snap 198 2024-04-07T16:23:41.853 INFO:journalctl@ceph.osd.3.smithi100.stdout:Apr 07 16:23:41 smithi100 bash[93998]: debug 2024-04-07T16:23:41.591+0000 7f6eb76607c0 -1 osd.3 459 log_to_monitors true 2024-04-07T16:23:42.184 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:41 smithi100 bash[74942]: cluster 2024-04-07T16:23:40.681986+0000 mgr.y (mgr.25175) 183 : cluster [DBG] pgmap v292: 121 pgs: 8 active+undersized+degraded, 32 active+undersized, 81 active+clean; 328 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 22 MiB/s rd, 2.9 MiB/s wr, 21 op/s; 63/579 objects degraded (10.881%) 2024-04-07T16:23:42.184 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:41 smithi100 bash[74942]: cluster 2024-04-07T16:23:40.865361+0000 mon.a (mon.0) 576 : cluster [DBG] osdmap e468: 8 total, 7 up, 8 in 2024-04-07T16:23:42.184 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:41 smithi100 bash[74942]: audit 2024-04-07T16:23:41.595884+0000 mon.a (mon.0) 577 : audit [INF] from='osd.3 [v2:172.21.15.100:6826/269478984,v1:172.21.15.100:6827/269478984]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-07T16:23:42.184 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:41 smithi100 bash[76202]: cluster 2024-04-07T16:23:40.681986+0000 mgr.y (mgr.25175) 183 : cluster [DBG] pgmap v292: 121 pgs: 8 active+undersized+degraded, 32 active+undersized, 81 active+clean; 328 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 22 MiB/s rd, 2.9 MiB/s wr, 21 op/s; 63/579 objects degraded (10.881%) 2024-04-07T16:23:42.184 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:41 smithi100 bash[76202]: cluster 2024-04-07T16:23:40.865361+0000 mon.a (mon.0) 576 : cluster [DBG] osdmap e468: 8 total, 7 up, 8 in 2024-04-07T16:23:42.184 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:41 smithi100 bash[76202]: audit 2024-04-07T16:23:41.595884+0000 mon.a (mon.0) 577 : audit [INF] from='osd.3 [v2:172.21.15.100:6826/269478984,v1:172.21.15.100:6827/269478984]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-07T16:23:42.305 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:41 smithi144 bash[66654]: cluster 2024-04-07T16:23:40.681986+0000 mgr.y (mgr.25175) 183 : cluster [DBG] pgmap v292: 121 pgs: 8 active+undersized+degraded, 32 active+undersized, 81 active+clean; 328 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 22 MiB/s rd, 2.9 MiB/s wr, 21 op/s; 63/579 objects degraded (10.881%) 2024-04-07T16:23:42.305 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:41 smithi144 bash[66654]: cluster 2024-04-07T16:23:40.865361+0000 mon.a (mon.0) 576 : cluster [DBG] osdmap e468: 8 total, 7 up, 8 in 2024-04-07T16:23:42.305 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:41 smithi144 bash[66654]: audit 2024-04-07T16:23:41.595884+0000 mon.a (mon.0) 577 : audit [INF] from='osd.3 [v2:172.21.15.100:6826/269478984,v1:172.21.15.100:6827/269478984]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-07T16:23:42.606 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:23:42 smithi144 bash[59794]: ts=2024-04-07T16:23:42.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:23:42.683 INFO:journalctl@ceph.osd.3.smithi100.stdout:Apr 07 16:23:42 smithi100 bash[93998]: debug 2024-04-07T16:23:42.402+0000 7f6ea4ab0700 -1 osd.3 459 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-07T16:23:42.862 INFO:tasks.rados.rados.0.smithi100.stdout:2136: done (2 left) 2024-04-07T16:23:42.862 INFO:tasks.rados.rados.0.smithi100.stdout:2137: copy_from oid 30 from oid 31 current snap is 203 2024-04-07T16:23:42.862 INFO:tasks.rados.rados.0.smithi100.stdout:2132: finishing write tid 5 to smithi10066867-10 2024-04-07T16:23:42.862 INFO:tasks.rados.rados.0.smithi100.stdout:2132: oid 10 updating version 0 to 1057 2024-04-07T16:23:42.862 INFO:tasks.rados.rados.0.smithi100.stdout:2132: oid 10 updating version 1057 to 1059 2024-04-07T16:23:42.862 INFO:tasks.rados.rados.0.smithi100.stdout:2132: oid 10 updating version 1059 to 1060 2024-04-07T16:23:42.862 INFO:tasks.rados.rados.0.smithi100.stdout:2132: oid 10 version 1060 is already newer than 1058 2024-04-07T16:23:42.862 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 10 v 1060 (ObjNum 660 snap 203 seq_num 660) dirty exists 2024-04-07T16:23:42.862 INFO:tasks.rados.rados.0.smithi100.stdout:2132: left oid 10 (ObjNum 660 snap 203 seq_num 660) 2024-04-07T16:23:42.863 INFO:tasks.rados.rados.0.smithi100.stdout:2133: finishing copy_from racing read to smithi10066867-19 2024-04-07T16:23:42.863 INFO:tasks.rados.rados.0.smithi100.stdout:2133: finishing copy_from to smithi10066867-19 2024-04-07T16:23:42.863 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 19 v 812 (ObjNum 656 snap 203 seq_num 656) dirty exists 2024-04-07T16:23:42.864 INFO:tasks.rados.rados.0.smithi100.stdout:2132: done (2 left) 2024-04-07T16:23:42.864 INFO:tasks.rados.rados.0.smithi100.stdout:2133: done (1 left) 2024-04-07T16:23:42.864 INFO:tasks.rados.rados.0.smithi100.stdout:2138: read oid 21 snap -1 2024-04-07T16:23:42.864 INFO:tasks.rados.rados.0.smithi100.stdout:2138: expect (ObjNum 565 snap 173 seq_num 565) 2024-04-07T16:23:42.864 INFO:tasks.rados.rados.0.smithi100.stdout:2139: setattr oid 28 current snap is 203 2024-04-07T16:23:42.864 INFO:tasks.rados.rados.0.smithi100.stdout:2140: snap_remove snap 199 2024-04-07T16:23:43.184 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:42 smithi100 bash[74942]: audit 2024-04-07T16:23:41.863841+0000 mon.a (mon.0) 578 : audit [INF] from='osd.3 [v2:172.21.15.100:6826/269478984,v1:172.21.15.100:6827/269478984]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-07T16:23:43.184 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:42 smithi100 bash[74942]: cluster 2024-04-07T16:23:41.869628+0000 mon.a (mon.0) 579 : cluster [DBG] osdmap e469: 8 total, 7 up, 8 in 2024-04-07T16:23:43.184 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:42 smithi100 bash[74942]: audit 2024-04-07T16:23:41.870141+0000 mon.a (mon.0) 580 : audit [INF] from='osd.3 [v2:172.21.15.100:6826/269478984,v1:172.21.15.100:6827/269478984]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-04-07T16:23:43.184 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:42 smithi100 bash[74942]: audit 2024-04-07T16:23:42.120085+0000 mon.a (mon.0) 581 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:23:43.184 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:42 smithi100 bash[74942]: audit 2024-04-07T16:23:42.193721+0000 mon.a (mon.0) 582 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:23:43.184 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:42 smithi100 bash[74942]: audit 2024-04-07T16:23:42.383610+0000 mon.a (mon.0) 583 : audit [INF] from='osd.3 [v2:172.21.15.100:6826/269478984,v1:172.21.15.100:6827/269478984]' entity='osd.3' 2024-04-07T16:23:43.184 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:42 smithi100 bash[74942]: cluster 2024-04-07T16:23:42.864981+0000 mon.a (mon.0) 584 : cluster [INF] Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-07T16:23:43.184 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:42 smithi100 bash[74942]: cluster 2024-04-07T16:23:42.871412+0000 mon.a (mon.0) 585 : cluster [INF] osd.3 [v2:172.21.15.100:6826/269478984,v1:172.21.15.100:6827/269478984] boot 2024-04-07T16:23:43.184 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:42 smithi100 bash[74942]: cluster 2024-04-07T16:23:42.871526+0000 mon.a (mon.0) 586 : cluster [DBG] osdmap e470: 8 total, 8 up, 8 in 2024-04-07T16:23:43.184 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:42 smithi100 bash[74942]: audit 2024-04-07T16:23:42.871838+0000 mon.a (mon.0) 587 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-07T16:23:43.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:42 smithi100 bash[76202]: audit 2024-04-07T16:23:41.863841+0000 mon.a (mon.0) 578 : audit [INF] from='osd.3 [v2:172.21.15.100:6826/269478984,v1:172.21.15.100:6827/269478984]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-07T16:23:43.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:42 smithi100 bash[76202]: cluster 2024-04-07T16:23:41.869628+0000 mon.a (mon.0) 579 : cluster [DBG] osdmap e469: 8 total, 7 up, 8 in 2024-04-07T16:23:43.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:42 smithi100 bash[76202]: audit 2024-04-07T16:23:41.870141+0000 mon.a (mon.0) 580 : audit [INF] from='osd.3 [v2:172.21.15.100:6826/269478984,v1:172.21.15.100:6827/269478984]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-04-07T16:23:43.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:42 smithi100 bash[76202]: audit 2024-04-07T16:23:42.120085+0000 mon.a (mon.0) 581 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:23:43.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:42 smithi100 bash[76202]: audit 2024-04-07T16:23:42.193721+0000 mon.a (mon.0) 582 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:23:43.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:42 smithi100 bash[76202]: audit 2024-04-07T16:23:42.383610+0000 mon.a (mon.0) 583 : audit [INF] from='osd.3 [v2:172.21.15.100:6826/269478984,v1:172.21.15.100:6827/269478984]' entity='osd.3' 2024-04-07T16:23:43.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:42 smithi100 bash[76202]: cluster 2024-04-07T16:23:42.864981+0000 mon.a (mon.0) 584 : cluster [INF] Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-07T16:23:43.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:42 smithi100 bash[76202]: cluster 2024-04-07T16:23:42.871412+0000 mon.a (mon.0) 585 : cluster [INF] osd.3 [v2:172.21.15.100:6826/269478984,v1:172.21.15.100:6827/269478984] boot 2024-04-07T16:23:43.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:42 smithi100 bash[76202]: cluster 2024-04-07T16:23:42.871526+0000 mon.a (mon.0) 586 : cluster [DBG] osdmap e470: 8 total, 8 up, 8 in 2024-04-07T16:23:43.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:42 smithi100 bash[76202]: audit 2024-04-07T16:23:42.871838+0000 mon.a (mon.0) 587 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-07T16:23:43.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:42 smithi144 bash[66654]: audit 2024-04-07T16:23:41.863841+0000 mon.a (mon.0) 578 : audit [INF] from='osd.3 [v2:172.21.15.100:6826/269478984,v1:172.21.15.100:6827/269478984]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-07T16:23:43.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:42 smithi144 bash[66654]: cluster 2024-04-07T16:23:41.869628+0000 mon.a (mon.0) 579 : cluster [DBG] osdmap e469: 8 total, 7 up, 8 in 2024-04-07T16:23:43.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:42 smithi144 bash[66654]: audit 2024-04-07T16:23:41.870141+0000 mon.a (mon.0) 580 : audit [INF] from='osd.3 [v2:172.21.15.100:6826/269478984,v1:172.21.15.100:6827/269478984]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-04-07T16:23:43.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:42 smithi144 bash[66654]: audit 2024-04-07T16:23:42.120085+0000 mon.a (mon.0) 581 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:23:43.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:42 smithi144 bash[66654]: audit 2024-04-07T16:23:42.193721+0000 mon.a (mon.0) 582 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:23:43.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:42 smithi144 bash[66654]: audit 2024-04-07T16:23:42.383610+0000 mon.a (mon.0) 583 : audit [INF] from='osd.3 [v2:172.21.15.100:6826/269478984,v1:172.21.15.100:6827/269478984]' entity='osd.3' 2024-04-07T16:23:43.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:42 smithi144 bash[66654]: cluster 2024-04-07T16:23:42.864981+0000 mon.a (mon.0) 584 : cluster [INF] Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-07T16:23:43.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:42 smithi144 bash[66654]: cluster 2024-04-07T16:23:42.871412+0000 mon.a (mon.0) 585 : cluster [INF] osd.3 [v2:172.21.15.100:6826/269478984,v1:172.21.15.100:6827/269478984] boot 2024-04-07T16:23:43.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:42 smithi144 bash[66654]: cluster 2024-04-07T16:23:42.871526+0000 mon.a (mon.0) 586 : cluster [DBG] osdmap e470: 8 total, 8 up, 8 in 2024-04-07T16:23:43.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:42 smithi144 bash[66654]: audit 2024-04-07T16:23:42.871838+0000 mon.a (mon.0) 587 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-07T16:23:43.856 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:23:43 smithi144 bash[59794]: ts=2024-04-07T16:23:43.507Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=nodes name=CephNodeDiskspaceWarning index=4 msg="Evaluating rule failed" rule="alert: CephNodeDiskspaceWarning\nexpr: predict_linear(node_filesystem_free_bytes{device=~\"/.*\"}[2d], 3600 * 24 * 5)\n * on (instance) group_left (nodename) node_uname_info < 0\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.8.4\n severity: warning\n type: ceph_default\nannotations:\n description: Mountpoint {{ $labels.mountpoint }} on {{ $labels.nodename }} will\n be full in less than 5 days based on the 48 hour trailing fill rate.\n summary: Host filesystem free space is getting low\n" err="found duplicate series for the match group {instance=\"smithi100\"} on the right hand-side of the operation: [{__name__=\"node_uname_info\", domainname=\"(none)\", instance=\"smithi100\", job=\"node\", machine=\"x86_64\", nodename=\"smithi100\", release=\"5.15.0-101-generic\", sysname=\"Linux\", version=\"#111-Ubuntu SMP Tue Mar 5 20:16:58 UTC 2024\"}, {__name__=\"node_uname_info\", cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", domainname=\"(none)\", instance=\"smithi100\", job=\"node\", machine=\"x86_64\", nodename=\"smithi100\", release=\"5.15.0-101-generic\", sysname=\"Linux\", version=\"#111-Ubuntu SMP Tue Mar 5 20:16:58 UTC 2024\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:23:43.862 INFO:tasks.rados.rados.0.smithi100.stdout:2140: done (3 left) 2024-04-07T16:23:43.862 INFO:tasks.rados.rados.0.smithi100.stdout:2141: read oid 42 snap 175 2024-04-07T16:23:43.862 INFO:tasks.rados.rados.0.smithi100.stdout:2141: expect (ObjNum 533 snap 161 seq_num 533) 2024-04-07T16:23:43.862 INFO:tasks.rados.rados.0.smithi100.stdout:2142: rmattr oid 29 current snap is 203 2024-04-07T16:23:43.862 INFO:tasks.rados.rados.0.smithi100.stdout:2142: done (4 left) 2024-04-07T16:23:43.862 INFO:tasks.rados.rados.0.smithi100.stdout:2143: read oid 11 snap -1 2024-04-07T16:23:43.862 INFO:tasks.rados.rados.0.smithi100.stdout:2143: expect (ObjNum 537 snap 162 seq_num 537) 2024-04-07T16:23:43.862 INFO:tasks.rados.rados.0.smithi100.stdout:2144: copy_from oid 4 from oid 32 current snap is 203 2024-04-07T16:23:43.863 INFO:tasks.rados.rados.0.smithi100.stdout:2145: rollback oid 49 current snap is 203 2024-04-07T16:23:43.863 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 49 to 195 2024-04-07T16:23:43.863 INFO:tasks.rados.rados.0.smithi100.stdout:2146: delete oid 27 current snap is 203 2024-04-07T16:23:43.866 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 28 v 1090 (ObjNum 510 snap 152 seq_num 510) dirty exists 2024-04-07T16:23:43.895 INFO:tasks.rados.rados.0.smithi100.stdout:2139: done (7 left) 2024-04-07T16:23:43.895 INFO:tasks.rados.rados.0.smithi100.stdout:2146: done (6 left) 2024-04-07T16:23:43.895 INFO:tasks.rados.rados.0.smithi100.stdout:2147: read oid 24 snap -1 2024-04-07T16:23:43.895 INFO:tasks.rados.rados.0.smithi100.stdout:2147: expect (ObjNum 574 snap 177 seq_num 574) 2024-04-07T16:23:43.895 INFO:tasks.rados.rados.0.smithi100.stdout:2148: copy_from oid 40 from oid 34 current snap is 203 2024-04-07T16:23:43.895 INFO:tasks.rados.rados.0.smithi100.stdout:2149: rollback oid 19 current snap is 203 2024-04-07T16:23:43.895 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 19 to 188 2024-04-07T16:23:43.895 INFO:tasks.rados.rados.0.smithi100.stdout:2150: rollback oid 33 current snap is 203 2024-04-07T16:23:43.895 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 33 to 188 2024-04-07T16:23:43.896 INFO:tasks.rados.rados.0.smithi100.stdout:2151: snap_create 2024-04-07T16:23:43.909 INFO:tasks.rados.rados.0.smithi100.stdout:2148: finishing copy_from to smithi10066867-40 2024-04-07T16:23:43.910 INFO:tasks.rados.rados.0.smithi100.stdout:2148: got expected ENOENT (src dne) 2024-04-07T16:23:43.911 INFO:tasks.rados.rados.0.smithi100.stdout:2145: finishing rollback tid 0 to smithi10066867-49 2024-04-07T16:23:43.912 INFO:tasks.rados.rados.0.smithi100.stdout:2145: finishing rollback tid 1 to smithi10066867-49 2024-04-07T16:23:43.912 INFO:tasks.rados.rados.0.smithi100.stdout:2145: finishing rollback tid 2 to smithi10066867-49 2024-04-07T16:23:43.912 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 49 v 1096 (ObjNum 598 snap 184 seq_num 598) dirty exists 2024-04-07T16:23:43.918 INFO:tasks.rados.rados.0.smithi100.stdout:2148: finishing copy_from racing read to smithi10066867-40 2024-04-07T16:23:43.930 INFO:tasks.rados.rados.0.smithi100.stdout:2147: expect (ObjNum 574 snap 177 seq_num 574) 2024-04-07T16:23:44.236 INFO:tasks.rados.rados.0.smithi100.stdout:2149: finishing rollback tid 0 to smithi10066867-19 2024-04-07T16:23:44.237 INFO:tasks.rados.rados.0.smithi100.stdout:2149: finishing rollback tid 1 to smithi10066867-19 2024-04-07T16:23:44.237 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 19 v 1064 (ObjNum 289 snap 0 seq_num 1571426808) dirty dne 2024-04-07T16:23:44.237 INFO:tasks.rados.rados.0.smithi100.stdout:2141: expect (ObjNum 533 snap 161 seq_num 533) 2024-04-07T16:23:44.281 INFO:tasks.rados.rados.0.smithi100.stdout:2137: finishing copy_from to smithi10066867-30 2024-04-07T16:23:44.281 INFO:tasks.rados.rados.0.smithi100.stdout:2137: got expected ENOENT (src dne) 2024-04-07T16:23:44.281 INFO:tasks.rados.rados.0.smithi100.stdout:2137: finishing copy_from racing read to smithi10066867-30 2024-04-07T16:23:44.282 INFO:tasks.rados.rados.0.smithi100.stdout:2144: finishing copy_from racing read to smithi10066867-4 2024-04-07T16:23:44.282 INFO:tasks.rados.rados.0.smithi100.stdout:2143: expect (ObjNum 537 snap 162 seq_num 537) 2024-04-07T16:23:44.359 INFO:tasks.rados.rados.0.smithi100.stdout:2144: finishing copy_from to smithi10066867-4 2024-04-07T16:23:44.359 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 4 v 1089 (ObjNum 630 snap 196 seq_num 630) dirty exists 2024-04-07T16:23:44.359 INFO:tasks.rados.rados.0.smithi100.stdout:2150: finishing rollback tid 0 to smithi10066867-33 2024-04-07T16:23:44.359 INFO:tasks.rados.rados.0.smithi100.stdout:2150: finishing rollback tid 1 to smithi10066867-33 2024-04-07T16:23:44.360 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 33 v 1181 (ObjNum 161 snap 0 seq_num 0) dirty dne 2024-04-07T16:23:44.360 INFO:tasks.rados.rados.0.smithi100.stdout:2138: expect (ObjNum 565 snap 173 seq_num 565) 2024-04-07T16:23:44.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:44 smithi144 bash[66654]: cluster 2024-04-07T16:23:42.682698+0000 mgr.y (mgr.25175) 184 : cluster [DBG] pgmap v295: 121 pgs: 8 active+undersized+degraded, 32 active+undersized, 81 active+clean; 332 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 33 MiB/s rd, 3.4 MiB/s wr, 27 op/s; 64/588 objects degraded (10.884%) 2024-04-07T16:23:44.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:44 smithi144 bash[66654]: audit 2024-04-07T16:23:43.290506+0000 mon.a (mon.0) 588 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:23:44.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:44 smithi144 bash[66654]: audit 2024-04-07T16:23:43.297797+0000 mon.a (mon.0) 589 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:23:44.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:44 smithi144 bash[66654]: cluster 2024-04-07T16:23:43.878585+0000 mon.a (mon.0) 590 : cluster [DBG] osdmap e471: 8 total, 8 up, 8 in 2024-04-07T16:23:44.684 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:44 smithi100 bash[74942]: cluster 2024-04-07T16:23:42.682698+0000 mgr.y (mgr.25175) 184 : cluster [DBG] pgmap v295: 121 pgs: 8 active+undersized+degraded, 32 active+undersized, 81 active+clean; 332 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 33 MiB/s rd, 3.4 MiB/s wr, 27 op/s; 64/588 objects degraded (10.884%) 2024-04-07T16:23:44.684 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:44 smithi100 bash[74942]: audit 2024-04-07T16:23:43.290506+0000 mon.a (mon.0) 588 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:23:44.684 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:44 smithi100 bash[74942]: audit 2024-04-07T16:23:43.297797+0000 mon.a (mon.0) 589 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:23:44.684 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:44 smithi100 bash[74942]: cluster 2024-04-07T16:23:43.878585+0000 mon.a (mon.0) 590 : cluster [DBG] osdmap e471: 8 total, 8 up, 8 in 2024-04-07T16:23:44.684 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:44 smithi100 bash[76202]: cluster 2024-04-07T16:23:42.682698+0000 mgr.y (mgr.25175) 184 : cluster [DBG] pgmap v295: 121 pgs: 8 active+undersized+degraded, 32 active+undersized, 81 active+clean; 332 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 33 MiB/s rd, 3.4 MiB/s wr, 27 op/s; 64/588 objects degraded (10.884%) 2024-04-07T16:23:44.684 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:44 smithi100 bash[76202]: audit 2024-04-07T16:23:43.290506+0000 mon.a (mon.0) 588 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:23:44.684 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:44 smithi100 bash[76202]: audit 2024-04-07T16:23:43.297797+0000 mon.a (mon.0) 589 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:23:44.685 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:44 smithi100 bash[76202]: cluster 2024-04-07T16:23:43.878585+0000 mon.a (mon.0) 590 : cluster [DBG] osdmap e471: 8 total, 8 up, 8 in 2024-04-07T16:23:44.860 INFO:tasks.rados.rados.0.smithi100.stdout:2137: done (10 left) 2024-04-07T16:23:44.860 INFO:tasks.rados.rados.0.smithi100.stdout:2138: done (9 left) 2024-04-07T16:23:44.860 INFO:tasks.rados.rados.0.smithi100.stdout:2141: done (8 left) 2024-04-07T16:23:44.860 INFO:tasks.rados.rados.0.smithi100.stdout:2143: done (7 left) 2024-04-07T16:23:44.860 INFO:tasks.rados.rados.0.smithi100.stdout:2144: done (6 left) 2024-04-07T16:23:44.860 INFO:tasks.rados.rados.0.smithi100.stdout:2145: done (5 left) 2024-04-07T16:23:44.860 INFO:tasks.rados.rados.0.smithi100.stdout:2147: done (4 left) 2024-04-07T16:23:44.860 INFO:tasks.rados.rados.0.smithi100.stdout:2148: done (3 left) 2024-04-07T16:23:44.860 INFO:tasks.rados.rados.0.smithi100.stdout:2149: done (2 left) 2024-04-07T16:23:44.860 INFO:tasks.rados.rados.0.smithi100.stdout:2150: done (1 left) 2024-04-07T16:23:44.860 INFO:tasks.rados.rados.0.smithi100.stdout:2151: done (0 left) 2024-04-07T16:23:44.860 INFO:tasks.rados.rados.0.smithi100.stdout:2152: read oid 42 snap 186 2024-04-07T16:23:44.861 INFO:tasks.rados.rados.0.smithi100.stdout:2152: expect (ObjNum 533 snap 161 seq_num 533) 2024-04-07T16:23:44.861 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 12 current snap is 204 2024-04-07T16:23:44.861 INFO:tasks.rados.rados.0.smithi100.stdout:2153: seq_num 663 ranges {4694016=712704,5406720=442368,5849088=360448} 2024-04-07T16:23:44.935 INFO:tasks.rados.rados.0.smithi100.stdout:2153: writing smithi10066867-12 from 4694016 to 5406720 tid 1 2024-04-07T16:23:44.939 INFO:tasks.rados.rados.0.smithi100.stdout:2153: writing smithi10066867-12 from 5406720 to 5849088 tid 2 2024-04-07T16:23:44.943 INFO:tasks.rados.rados.0.smithi100.stdout:2153: writing smithi10066867-12 from 5849088 to 6209536 tid 3 2024-04-07T16:23:44.943 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 22 current snap is 204 2024-04-07T16:23:44.943 INFO:tasks.rados.rados.0.smithi100.stdout:2154: seq_num 664 ranges {1376256=729088,2105344=729088,2834432=73728} 2024-04-07T16:23:44.958 INFO:tasks.rados.rados.0.smithi100.stdout:2154: writing smithi10066867-22 from 1376256 to 2105344 tid 1 2024-04-07T16:23:44.964 INFO:tasks.rados.rados.0.smithi100.stdout:2154: writing smithi10066867-22 from 2105344 to 2834432 tid 2 2024-04-07T16:23:44.964 INFO:tasks.rados.rados.0.smithi100.stdout:2154: writing smithi10066867-22 from 2834432 to 2908160 tid 3 2024-04-07T16:23:44.965 INFO:tasks.rados.rados.0.smithi100.stdout:2155: rollback oid 35 current snap is 204 2024-04-07T16:23:44.965 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 35 to 177 2024-04-07T16:23:44.965 INFO:tasks.rados.rados.0.smithi100.stdout:2152: expect (ObjNum 533 snap 161 seq_num 533) 2024-04-07T16:23:45.012 INFO:tasks.rados.rados.0.smithi100.stdout:2153: finishing write tid 1 to smithi10066867-12 2024-04-07T16:23:45.012 INFO:tasks.rados.rados.0.smithi100.stdout:2153: finishing write tid 2 to smithi10066867-12 2024-04-07T16:23:45.012 INFO:tasks.rados.rados.0.smithi100.stdout:2153: finishing write tid 3 to smithi10066867-12 2024-04-07T16:23:45.012 INFO:tasks.rados.rados.0.smithi100.stdout:2153: finishing write tid 4 to smithi10066867-12 2024-04-07T16:23:45.012 INFO:tasks.rados.rados.0.smithi100.stdout:2153: finishing write tid 5 to smithi10066867-12 2024-04-07T16:23:45.012 INFO:tasks.rados.rados.0.smithi100.stdout:2153: oid 12 updating version 0 to 1100 2024-04-07T16:23:45.012 INFO:tasks.rados.rados.0.smithi100.stdout:2153: oid 12 version 1100 is already newer than 1099 2024-04-07T16:23:45.012 INFO:tasks.rados.rados.0.smithi100.stdout:2153: oid 12 version 1100 is already newer than 1097 2024-04-07T16:23:45.012 INFO:tasks.rados.rados.0.smithi100.stdout:2153: oid 12 version 1100 is already newer than 1098 2024-04-07T16:23:45.012 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 12 v 1100 (ObjNum 663 snap 204 seq_num 663) dirty exists 2024-04-07T16:23:45.012 INFO:tasks.rados.rados.0.smithi100.stdout:2153: left oid 12 (ObjNum 663 snap 204 seq_num 663) 2024-04-07T16:23:45.013 INFO:tasks.rados.rados.0.smithi100.stdout:2155: finishing rollback tid 0 to smithi10066867-35 2024-04-07T16:23:45.013 INFO:tasks.rados.rados.0.smithi100.stdout:2154: finishing write tid 1 to smithi10066867-22 2024-04-07T16:23:45.013 INFO:tasks.rados.rados.0.smithi100.stdout:2155: finishing rollback tid 1 to smithi10066867-35 2024-04-07T16:23:45.013 INFO:tasks.rados.rados.0.smithi100.stdout:2155: finishing rollback tid 2 to smithi10066867-35 2024-04-07T16:23:45.013 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 35 v 1065 (ObjNum 572 snap 177 seq_num 572) dirty exists 2024-04-07T16:23:45.013 INFO:tasks.rados.rados.0.smithi100.stdout:2154: finishing write tid 2 to smithi10066867-22 2024-04-07T16:23:45.013 INFO:tasks.rados.rados.0.smithi100.stdout:2154: finishing write tid 3 to smithi10066867-22 2024-04-07T16:23:45.013 INFO:tasks.rados.rados.0.smithi100.stdout:2154: finishing write tid 4 to smithi10066867-22 2024-04-07T16:23:45.013 INFO:tasks.rados.rados.0.smithi100.stdout:2154: finishing write tid 5 to smithi10066867-22 2024-04-07T16:23:45.013 INFO:tasks.rados.rados.0.smithi100.stdout:2154: oid 22 updating version 0 to 1091 2024-04-07T16:23:45.013 INFO:tasks.rados.rados.0.smithi100.stdout:2154: oid 22 updating version 1091 to 1092 2024-04-07T16:23:45.013 INFO:tasks.rados.rados.0.smithi100.stdout:2154: oid 22 updating version 1092 to 1093 2024-04-07T16:23:45.013 INFO:tasks.rados.rados.0.smithi100.stdout:2154: oid 22 version 1093 is already newer than 1090 2024-04-07T16:23:45.013 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 22 v 1093 (ObjNum 664 snap 204 seq_num 664) dirty exists 2024-04-07T16:23:45.013 INFO:tasks.rados.rados.0.smithi100.stdout:2154: left oid 22 (ObjNum 664 snap 204 seq_num 664) 2024-04-07T16:23:45.014 INFO:tasks.rados.rados.0.smithi100.stdout:2152: done (3 left) 2024-04-07T16:23:45.014 INFO:tasks.rados.rados.0.smithi100.stdout:2153: done (2 left) 2024-04-07T16:23:45.014 INFO:tasks.rados.rados.0.smithi100.stdout:2154: done (1 left) 2024-04-07T16:23:45.014 INFO:tasks.rados.rados.0.smithi100.stdout:2155: done (0 left) 2024-04-07T16:23:45.014 INFO:tasks.rados.rados.0.smithi100.stdout:2156: rollback oid 9 current snap is 204 2024-04-07T16:23:45.014 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 9 to 175 2024-04-07T16:23:45.014 INFO:tasks.rados.rados.0.smithi100.stdout:2157: read oid 35 snap -1 2024-04-07T16:23:45.014 INFO:tasks.rados.rados.0.smithi100.stdout:2157: expect (ObjNum 572 snap 177 seq_num 572) 2024-04-07T16:23:45.014 INFO:tasks.rados.rados.0.smithi100.stdout:2158: rmattr oid 36 current snap is 204 2024-04-07T16:23:45.014 INFO:tasks.rados.rados.0.smithi100.stdout:2159: setattr oid 43 current snap is 204 2024-04-07T16:23:45.015 INFO:tasks.rados.rados.0.smithi100.stdout:2160: delete oid 3 current snap is 204 2024-04-07T16:23:45.017 INFO:tasks.rados.rados.0.smithi100.stdout:2156: finishing rollback tid 1 to smithi10066867-9 2024-04-07T16:23:45.021 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 36 v 1065 (ObjNum 649 snap 203 seq_num 649) dirty exists 2024-04-07T16:23:45.023 INFO:tasks.rados.rados.0.smithi100.stdout:2158: done (4 left) 2024-04-07T16:23:45.023 INFO:tasks.rados.rados.0.smithi100.stdout:2160: done (3 left) 2024-04-07T16:23:45.023 INFO:tasks.rados.rados.0.smithi100.stdout:2161: copy_from oid 41 from oid 19 current snap is 204 2024-04-07T16:23:45.023 INFO:tasks.rados.rados.0.smithi100.stdout:2162: delete oid 23 current snap is 204 2024-04-07T16:23:45.023 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 43 v 1089 (ObjNum 618 snap 194 seq_num 618) dirty exists 2024-04-07T16:23:45.024 INFO:tasks.rados.rados.0.smithi100.stdout:2161: finishing copy_from to smithi10066867-41 2024-04-07T16:23:45.024 INFO:tasks.rados.rados.0.smithi100.stdout:2161: got expected ENOENT (src dne) 2024-04-07T16:23:45.024 INFO:tasks.rados.rados.0.smithi100.stdout:2161: finishing copy_from racing read to smithi10066867-41 2024-04-07T16:23:45.025 INFO:tasks.rados.rados.0.smithi100.stdout:2156: finishing rollback tid 2 to smithi10066867-9 2024-04-07T16:23:45.025 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 9 v 564 (ObjNum 546 snap 166 seq_num 546) dirty exists 2024-04-07T16:23:45.027 INFO:tasks.rados.rados.0.smithi100.stdout:2157: expect (ObjNum 572 snap 177 seq_num 572) 2024-04-07T16:23:45.088 INFO:tasks.rados.rados.0.smithi100.stdout:2156: done (4 left) 2024-04-07T16:23:45.088 INFO:tasks.rados.rados.0.smithi100.stdout:2157: done (3 left) 2024-04-07T16:23:45.088 INFO:tasks.rados.rados.0.smithi100.stdout:2159: done (2 left) 2024-04-07T16:23:45.088 INFO:tasks.rados.rados.0.smithi100.stdout:2161: done (1 left) 2024-04-07T16:23:45.089 INFO:tasks.rados.rados.0.smithi100.stdout:2162: done (0 left) 2024-04-07T16:23:45.089 INFO:tasks.rados.rados.0.smithi100.stdout:2163: read oid 39 snap -1 2024-04-07T16:23:45.089 INFO:tasks.rados.rados.0.smithi100.stdout:2163: expect (ObjNum 646 snap 203 seq_num 646) 2024-04-07T16:23:45.089 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 15 current snap is 204 2024-04-07T16:23:45.089 INFO:tasks.rados.rados.0.smithi100.stdout:2164: seq_num 667 ranges {0=778240,778240=778240,1556480=24576} 2024-04-07T16:23:45.099 INFO:tasks.rados.rados.0.smithi100.stdout:2164: writing smithi10066867-15 from 0 to 778240 tid 1 2024-04-07T16:23:45.109 INFO:tasks.rados.rados.0.smithi100.stdout:2164: writing smithi10066867-15 from 778240 to 1556480 tid 2 2024-04-07T16:23:45.109 INFO:tasks.rados.rados.0.smithi100.stdout:2164: writing smithi10066867-15 from 1556480 to 1581056 tid 3 2024-04-07T16:23:45.109 INFO:tasks.rados.rados.0.smithi100.stdout:2165: read oid 4 snap -1 2024-04-07T16:23:45.110 INFO:tasks.rados.rados.0.smithi100.stdout:2165: expect (ObjNum 630 snap 196 seq_num 630) 2024-04-07T16:23:45.110 INFO:tasks.rados.rados.0.smithi100.stdout:2166: snap_create 2024-04-07T16:23:45.110 INFO:tasks.rados.rados.0.smithi100.stdout:2163: expect (ObjNum 646 snap 203 seq_num 646) 2024-04-07T16:23:45.143 INFO:tasks.rados.rados.0.smithi100.stdout:2164: finishing write tid 1 to smithi10066867-15 2024-04-07T16:23:45.143 INFO:tasks.rados.rados.0.smithi100.stdout:2164: finishing write tid 2 to smithi10066867-15 2024-04-07T16:23:45.143 INFO:tasks.rados.rados.0.smithi100.stdout:2164: finishing write tid 3 to smithi10066867-15 2024-04-07T16:23:45.144 INFO:tasks.rados.rados.0.smithi100.stdout:2164: finishing write tid 4 to smithi10066867-15 2024-04-07T16:23:45.144 INFO:tasks.rados.rados.0.smithi100.stdout:2164: finishing write tid 5 to smithi10066867-15 2024-04-07T16:23:45.144 INFO:tasks.rados.rados.0.smithi100.stdout:2164: oid 15 updating version 0 to 1210 2024-04-07T16:23:45.144 INFO:tasks.rados.rados.0.smithi100.stdout:2164: oid 15 updating version 1210 to 1211 2024-04-07T16:23:45.144 INFO:tasks.rados.rados.0.smithi100.stdout:2164: oid 15 version 1211 is already newer than 1208 2024-04-07T16:23:45.144 INFO:tasks.rados.rados.0.smithi100.stdout:2164: oid 15 version 1211 is already newer than 1209 2024-04-07T16:23:45.144 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 15 v 1211 (ObjNum 667 snap 204 seq_num 667) dirty exists 2024-04-07T16:23:45.144 INFO:tasks.rados.rados.0.smithi100.stdout:2164: left oid 15 (ObjNum 667 snap 204 seq_num 667) 2024-04-07T16:23:45.144 INFO:tasks.rados.rados.0.smithi100.stdout:2165: expect (ObjNum 630 snap 196 seq_num 630) 2024-04-07T16:23:45.862 INFO:tasks.rados.rados.0.smithi100.stdout:2163: done (3 left) 2024-04-07T16:23:45.862 INFO:tasks.rados.rados.0.smithi100.stdout:2164: done (2 left) 2024-04-07T16:23:45.863 INFO:tasks.rados.rados.0.smithi100.stdout:2165: done (1 left) 2024-04-07T16:23:45.863 INFO:tasks.rados.rados.0.smithi100.stdout:2166: done (0 left) 2024-04-07T16:23:45.863 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 39 current snap is 205 2024-04-07T16:23:45.863 INFO:tasks.rados.rados.0.smithi100.stdout:2167: seq_num 668 ranges {1228800=794624,2023424=663552,2686976=139264} 2024-04-07T16:23:45.889 INFO:tasks.rados.rados.0.smithi100.stdout:2167: writing smithi10066867-39 from 1228800 to 2023424 tid 1 2024-04-07T16:23:45.900 INFO:tasks.rados.rados.0.smithi100.stdout:2167: writing smithi10066867-39 from 2023424 to 2686976 tid 2 2024-04-07T16:23:45.902 INFO:tasks.rados.rados.0.smithi100.stdout:2167: writing smithi10066867-39 from 2686976 to 2826240 tid 3 2024-04-07T16:23:45.902 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 38 current snap is 205 2024-04-07T16:23:45.902 INFO:tasks.rados.rados.0.smithi100.stdout:2168: seq_num 669 ranges {1277952=409600,1687552=548864,2236416=655360} 2024-04-07T16:23:45.922 INFO:tasks.rados.rados.0.smithi100.stdout:2168: writing smithi10066867-38 from 1277952 to 1687552 tid 1 2024-04-07T16:23:45.927 INFO:tasks.rados.rados.0.smithi100.stdout:2168: writing smithi10066867-38 from 1687552 to 2236416 tid 2 2024-04-07T16:23:45.933 INFO:tasks.rados.rados.0.smithi100.stdout:2168: writing smithi10066867-38 from 2236416 to 2891776 tid 3 2024-04-07T16:23:45.934 INFO:tasks.rados.rados.0.smithi100.stdout:2169: read oid 35 snap -1 2024-04-07T16:23:45.934 INFO:tasks.rados.rados.0.smithi100.stdout:2169: expect (ObjNum 572 snap 177 seq_num 572) 2024-04-07T16:23:45.934 INFO:tasks.rados.rados.0.smithi100.stdout:2170: copy_from oid 28 from oid 26 current snap is 205 2024-04-07T16:23:45.935 INFO:tasks.rados.rados.0.smithi100.stdout:2167: finishing write tid 1 to smithi10066867-39 2024-04-07T16:23:45.935 INFO:tasks.rados.rados.0.smithi100.stdout:2167: finishing write tid 2 to smithi10066867-39 2024-04-07T16:23:45.935 INFO:tasks.rados.rados.0.smithi100.stdout:2167: finishing write tid 3 to smithi10066867-39 2024-04-07T16:23:45.935 INFO:tasks.rados.rados.0.smithi100.stdout:2167: finishing write tid 4 to smithi10066867-39 2024-04-07T16:23:45.935 INFO:tasks.rados.rados.0.smithi100.stdout:2167: finishing write tid 5 to smithi10066867-39 2024-04-07T16:23:45.935 INFO:tasks.rados.rados.0.smithi100.stdout:2167: oid 39 updating version 0 to 1100 2024-04-07T16:23:45.935 INFO:tasks.rados.rados.0.smithi100.stdout:2167: oid 39 updating version 1100 to 1101 2024-04-07T16:23:45.936 INFO:tasks.rados.rados.0.smithi100.stdout:2167: oid 39 updating version 1101 to 1102 2024-04-07T16:23:45.936 INFO:tasks.rados.rados.0.smithi100.stdout:2167: oid 39 version 1102 is already newer than 1099 2024-04-07T16:23:45.936 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 39 v 1102 (ObjNum 668 snap 205 seq_num 668) dirty exists 2024-04-07T16:23:45.936 INFO:tasks.rados.rados.0.smithi100.stdout:2167: left oid 39 (ObjNum 668 snap 205 seq_num 668) 2024-04-07T16:23:45.936 INFO:tasks.rados.rados.0.smithi100.stdout:2167: done (3 left) 2024-04-07T16:23:45.936 INFO:tasks.rados.rados.0.smithi100.stdout:2171: snap_remove snap 175 2024-04-07T16:23:46.184 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:45 smithi100 bash[74942]: cluster 2024-04-07T16:23:44.683407+0000 mgr.y (mgr.25175) 185 : cluster [DBG] pgmap v298: 121 pgs: 9 peering, 4 active+undersized+degraded, 27 active+undersized, 81 active+clean; 346 MiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 15 MiB/s rd, 4.8 MiB/s wr, 12 op/s; 20/594 objects degraded (3.367%) 2024-04-07T16:23:46.184 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:45 smithi100 bash[74942]: cluster 2024-04-07T16:23:44.877852+0000 mon.a (mon.0) 591 : cluster [DBG] osdmap e472: 8 total, 8 up, 8 in 2024-04-07T16:23:46.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:45 smithi100 bash[74942]: cluster 2024-04-07T16:23:45.290100+0000 mon.a (mon.0) 592 : cluster [WRN] Health check update: Degraded data redundancy: 20/594 objects degraded (3.367%), 4 pgs degraded (PG_DEGRADED) 2024-04-07T16:23:46.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:45 smithi100 bash[76202]: cluster 2024-04-07T16:23:44.683407+0000 mgr.y (mgr.25175) 185 : cluster [DBG] pgmap v298: 121 pgs: 9 peering, 4 active+undersized+degraded, 27 active+undersized, 81 active+clean; 346 MiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 15 MiB/s rd, 4.8 MiB/s wr, 12 op/s; 20/594 objects degraded (3.367%) 2024-04-07T16:23:46.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:45 smithi100 bash[76202]: cluster 2024-04-07T16:23:44.877852+0000 mon.a (mon.0) 591 : cluster [DBG] osdmap e472: 8 total, 8 up, 8 in 2024-04-07T16:23:46.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:45 smithi100 bash[76202]: cluster 2024-04-07T16:23:45.290100+0000 mon.a (mon.0) 592 : cluster [WRN] Health check update: Degraded data redundancy: 20/594 objects degraded (3.367%), 4 pgs degraded (PG_DEGRADED) 2024-04-07T16:23:46.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:45 smithi144 bash[66654]: cluster 2024-04-07T16:23:44.683407+0000 mgr.y (mgr.25175) 185 : cluster [DBG] pgmap v298: 121 pgs: 9 peering, 4 active+undersized+degraded, 27 active+undersized, 81 active+clean; 346 MiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 15 MiB/s rd, 4.8 MiB/s wr, 12 op/s; 20/594 objects degraded (3.367%) 2024-04-07T16:23:46.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:45 smithi144 bash[66654]: cluster 2024-04-07T16:23:44.877852+0000 mon.a (mon.0) 591 : cluster [DBG] osdmap e472: 8 total, 8 up, 8 in 2024-04-07T16:23:46.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:45 smithi144 bash[66654]: cluster 2024-04-07T16:23:45.290100+0000 mon.a (mon.0) 592 : cluster [WRN] Health check update: Degraded data redundancy: 20/594 objects degraded (3.367%), 4 pgs degraded (PG_DEGRADED) 2024-04-07T16:23:46.892 INFO:tasks.rados.rados.0.smithi100.stdout:2171: done (3 left) 2024-04-07T16:23:46.892 INFO:tasks.rados.rados.0.smithi100.stdout:2172: read oid 39 snap -1 2024-04-07T16:23:46.893 INFO:tasks.rados.rados.0.smithi100.stdout:2172: expect (ObjNum 668 snap 205 seq_num 668) 2024-04-07T16:23:46.893 INFO:tasks.rados.rados.0.smithi100.stdout:2173: delete oid 27 current snap is 205 2024-04-07T16:23:46.899 INFO:tasks.rados.rados.0.smithi100.stdout:2168: finishing write tid 1 to smithi10066867-38 2024-04-07T16:23:46.900 INFO:tasks.rados.rados.0.smithi100.stdout:2168: finishing write tid 2 to smithi10066867-38 2024-04-07T16:23:46.900 INFO:tasks.rados.rados.0.smithi100.stdout:2168: finishing write tid 3 to smithi10066867-38 2024-04-07T16:23:46.900 INFO:tasks.rados.rados.0.smithi100.stdout:2168: finishing write tid 4 to smithi10066867-38 2024-04-07T16:23:46.900 INFO:tasks.rados.rados.0.smithi100.stdout:2168: finishing write tid 5 to smithi10066867-38 2024-04-07T16:23:46.900 INFO:tasks.rados.rados.0.smithi100.stdout:2168: oid 38 updating version 0 to 1093 2024-04-07T16:23:46.900 INFO:tasks.rados.rados.0.smithi100.stdout:2168: oid 38 updating version 1093 to 1094 2024-04-07T16:23:46.900 INFO:tasks.rados.rados.0.smithi100.stdout:2168: oid 38 updating version 1094 to 1096 2024-04-07T16:23:46.900 INFO:tasks.rados.rados.0.smithi100.stdout:2168: oid 38 version 1096 is already newer than 1095 2024-04-07T16:23:46.900 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 38 v 1096 (ObjNum 669 snap 205 seq_num 669) dirty exists 2024-04-07T16:23:46.900 INFO:tasks.rados.rados.0.smithi100.stdout:2168: left oid 38 (ObjNum 669 snap 205 seq_num 669) 2024-04-07T16:23:46.900 INFO:tasks.rados.rados.0.smithi100.stdout:2169: expect (ObjNum 572 snap 177 seq_num 572) 2024-04-07T16:23:46.985 INFO:tasks.rados.rados.0.smithi100.stdout:2170: finishing copy_from racing read to smithi10066867-28 2024-04-07T16:23:46.985 INFO:tasks.rados.rados.0.smithi100.stdout:2170: finishing copy_from to smithi10066867-28 2024-04-07T16:23:46.985 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 28 v 783 (ObjNum 524 snap 156 seq_num 524) dirty exists 2024-04-07T16:23:46.985 INFO:tasks.rados.rados.0.smithi100.stdout:2172: expect (ObjNum 668 snap 205 seq_num 668) 2024-04-07T16:23:47.076 INFO:tasks.rados.rados.0.smithi100.stdout:2168: done (4 left) 2024-04-07T16:23:47.077 INFO:tasks.rados.rados.0.smithi100.stdout:2169: done (3 left) 2024-04-07T16:23:47.077 INFO:tasks.rados.rados.0.smithi100.stdout:2170: done (2 left) 2024-04-07T16:23:47.077 INFO:tasks.rados.rados.0.smithi100.stdout:2172: done (1 left) 2024-04-07T16:23:47.077 INFO:tasks.rados.rados.0.smithi100.stdout:2173: done (0 left) 2024-04-07T16:23:47.077 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 26 current snap is 205 2024-04-07T16:23:47.077 INFO:tasks.rados.rados.0.smithi100.stdout:2174: seq_num 671 ranges {7094272=442368,7536640=761856,8298496=442368} 2024-04-07T16:23:47.165 INFO:tasks.rados.rados.0.smithi100.stdout:2174: writing smithi10066867-26 from 7094272 to 7536640 tid 1 2024-04-07T16:23:47.170 INFO:tasks.rados.rados.0.smithi100.stdout:2174: writing smithi10066867-26 from 7536640 to 8298496 tid 2 2024-04-07T16:23:47.174 INFO:tasks.rados.rados.0.smithi100.stdout:2174: writing smithi10066867-26 from 8298496 to 8740864 tid 3 2024-04-07T16:23:47.174 INFO:tasks.rados.rados.0.smithi100.stdout:2175: read oid 21 snap 172 2024-04-07T16:23:47.174 INFO:tasks.rados.rados.0.smithi100.stdout:2175: expect (ObjNum 549 snap 166 seq_num 549) 2024-04-07T16:23:47.174 INFO:tasks.rados.rados.0.smithi100.stdout:2176: read oid 25 snap 172 2024-04-07T16:23:47.174 INFO:tasks.rados.rados.0.smithi100.stdout:2176: expect (ObjNum 554 snap 168 seq_num 554) 2024-04-07T16:23:47.174 INFO:tasks.rados.rados.0.smithi100.stdout:2177: snap_create 2024-04-07T16:23:47.175 INFO:tasks.rados.rados.0.smithi100.stdout:2174: finishing write tid 1 to smithi10066867-26 2024-04-07T16:23:47.180 INFO:tasks.rados.rados.0.smithi100.stdout:2174: finishing write tid 2 to smithi10066867-26 2024-04-07T16:23:47.180 INFO:tasks.rados.rados.0.smithi100.stdout:2174: finishing write tid 3 to smithi10066867-26 2024-04-07T16:23:47.182 INFO:tasks.rados.rados.0.smithi100.stdout:2174: finishing write tid 4 to smithi10066867-26 2024-04-07T16:23:47.183 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:46 smithi100 bash[74942]: cluster 2024-04-07T16:23:45.880578+0000 mon.a (mon.0) 593 : cluster [DBG] osdmap e473: 8 total, 8 up, 8 in 2024-04-07T16:23:47.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:46 smithi100 bash[74942]: audit 2024-04-07T16:23:46.720725+0000 mon.a (mon.0) 594 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:23:47.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:46 smithi100 bash[74942]: audit 2024-04-07T16:23:46.722931+0000 mon.a (mon.0) 595 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:23:47.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:46 smithi100 bash[76202]: cluster 2024-04-07T16:23:45.880578+0000 mon.a (mon.0) 593 : cluster [DBG] osdmap e473: 8 total, 8 up, 8 in 2024-04-07T16:23:47.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:46 smithi100 bash[76202]: audit 2024-04-07T16:23:46.720725+0000 mon.a (mon.0) 594 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:23:47.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:46 smithi100 bash[76202]: audit 2024-04-07T16:23:46.722931+0000 mon.a (mon.0) 595 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:23:47.192 INFO:tasks.rados.rados.0.smithi100.stdout:2174: finishing write tid 5 to smithi10066867-26 2024-04-07T16:23:47.192 INFO:tasks.rados.rados.0.smithi100.stdout:2174: oid 26 updating version 0 to 1096 2024-04-07T16:23:47.192 INFO:tasks.rados.rados.0.smithi100.stdout:2174: oid 26 version 1096 is already newer than 1094 2024-04-07T16:23:47.192 INFO:tasks.rados.rados.0.smithi100.stdout:2174: oid 26 version 1096 is already newer than 1095 2024-04-07T16:23:47.192 INFO:tasks.rados.rados.0.smithi100.stdout:2174: oid 26 updating version 1096 to 1097 2024-04-07T16:23:47.192 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 26 v 1097 (ObjNum 671 snap 205 seq_num 671) dirty exists 2024-04-07T16:23:47.192 INFO:tasks.rados.rados.0.smithi100.stdout:2174: left oid 26 (ObjNum 671 snap 205 seq_num 671) 2024-04-07T16:23:47.193 INFO:tasks.rados.rados.0.smithi100.stdout:2176: expect (ObjNum 554 snap 168 seq_num 554) 2024-04-07T16:23:47.256 INFO:tasks.rados.rados.0.smithi100.stdout:2175: expect (ObjNum 549 snap 166 seq_num 549) 2024-04-07T16:23:47.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:46 smithi144 bash[66654]: cluster 2024-04-07T16:23:45.880578+0000 mon.a (mon.0) 593 : cluster [DBG] osdmap e473: 8 total, 8 up, 8 in 2024-04-07T16:23:47.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:46 smithi144 bash[66654]: audit 2024-04-07T16:23:46.720725+0000 mon.a (mon.0) 594 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:23:47.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:46 smithi144 bash[66654]: audit 2024-04-07T16:23:46.722931+0000 mon.a (mon.0) 595 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:23:47.902 INFO:tasks.rados.rados.0.smithi100.stdout:2174: done (3 left) 2024-04-07T16:23:47.902 INFO:tasks.rados.rados.0.smithi100.stdout:2175: done (2 left) 2024-04-07T16:23:47.902 INFO:tasks.rados.rados.0.smithi100.stdout:2176: done (1 left) 2024-04-07T16:23:47.902 INFO:tasks.rados.rados.0.smithi100.stdout:2177: done (0 left) 2024-04-07T16:23:47.902 INFO:tasks.rados.rados.0.smithi100.stdout:2178: read oid 27 snap 203 2024-04-07T16:23:47.902 INFO:tasks.rados.rados.0.smithi100.stdout:2178: expect deleted 2024-04-07T16:23:47.902 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 35 current snap is 206 2024-04-07T16:23:47.902 INFO:tasks.rados.rados.0.smithi100.stdout:2179: seq_num 672 ranges {2392064=458752,2850816=647168,3497984=466944,3964928=90112} 2024-04-07T16:23:47.920 INFO:tasks.rados.rados.0.smithi100.stdout:2179: writing smithi10066867-35 from 2392064 to 2850816 tid 1 2024-04-07T16:23:47.925 INFO:tasks.rados.rados.0.smithi100.stdout:2179: writing smithi10066867-35 from 2850816 to 3497984 tid 2 2024-04-07T16:23:47.929 INFO:tasks.rados.rados.0.smithi100.stdout:2179: writing smithi10066867-35 from 3497984 to 3964928 tid 3 2024-04-07T16:23:47.930 INFO:tasks.rados.rados.0.smithi100.stdout:2179: writing smithi10066867-35 from 3964928 to 4055040 tid 4 2024-04-07T16:23:47.930 INFO:tasks.rados.rados.0.smithi100.stdout:2180: rollback oid 40 current snap is 206 2024-04-07T16:23:47.930 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 40 to 202 2024-04-07T16:23:47.930 INFO:tasks.rados.rados.0.smithi100.stdout:2178: done (2 left) 2024-04-07T16:23:47.930 INFO:tasks.rados.rados.0.smithi100.stdout:2181: delete oid 14 current snap is 206 2024-04-07T16:23:47.932 INFO:tasks.rados.rados.0.smithi100.stdout:2179: finishing write tid 1 to smithi10066867-35 2024-04-07T16:23:47.933 INFO:tasks.rados.rados.0.smithi100.stdout:2179: finishing write tid 2 to smithi10066867-35 2024-04-07T16:23:47.937 INFO:tasks.rados.rados.0.smithi100.stdout:2179: finishing write tid 3 to smithi10066867-35 2024-04-07T16:23:47.938 INFO:tasks.rados.rados.0.smithi100.stdout:2179: finishing write tid 4 to smithi10066867-35 2024-04-07T16:23:47.938 INFO:tasks.rados.rados.0.smithi100.stdout:2179: finishing write tid 5 to smithi10066867-35 2024-04-07T16:23:47.939 INFO:tasks.rados.rados.0.smithi100.stdout:2179: finishing write tid 6 to smithi10066867-35 2024-04-07T16:23:47.939 INFO:tasks.rados.rados.0.smithi100.stdout:2179: oid 35 updating version 0 to 1067 2024-04-07T16:23:47.940 INFO:tasks.rados.rados.0.smithi100.stdout:2179: oid 35 updating version 1067 to 1069 2024-04-07T16:23:47.940 INFO:tasks.rados.rados.0.smithi100.stdout:2179: oid 35 updating version 1069 to 1070 2024-04-07T16:23:47.940 INFO:tasks.rados.rados.0.smithi100.stdout:2179: oid 35 version 1070 is already newer than 1068 2024-04-07T16:23:47.940 INFO:tasks.rados.rados.0.smithi100.stdout:2179: oid 35 version 1070 is already newer than 1066 2024-04-07T16:23:47.940 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 35 v 1070 (ObjNum 672 snap 206 seq_num 672) dirty exists 2024-04-07T16:23:47.940 INFO:tasks.rados.rados.0.smithi100.stdout:2179: left oid 35 (ObjNum 672 snap 206 seq_num 672) 2024-04-07T16:23:47.943 INFO:tasks.rados.rados.0.smithi100.stdout:2179: done (2 left) 2024-04-07T16:23:47.943 INFO:tasks.rados.rados.0.smithi100.stdout:2181: done (1 left) 2024-04-07T16:23:47.944 INFO:tasks.rados.rados.0.smithi100.stdout:2182: setattr oid 33 current snap is 206 2024-04-07T16:23:47.944 INFO:tasks.rados.rados.0.smithi100.stdout:2180: finishing rollback tid 0 to smithi10066867-40 2024-04-07T16:23:47.945 INFO:tasks.rados.rados.0.smithi100.stdout:2180: finishing rollback tid 1 to smithi10066867-40 2024-04-07T16:23:47.946 INFO:tasks.rados.rados.0.smithi100.stdout:2180: finishing rollback tid 2 to smithi10066867-40 2024-04-07T16:23:47.946 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 40 v 1108 (ObjNum 509 snap 152 seq_num 509) dirty exists 2024-04-07T16:23:47.946 INFO:tasks.rados.rados.0.smithi100.stdout:2180: done (1 left) 2024-04-07T16:23:47.946 INFO:tasks.rados.rados.0.smithi100.stdout:2183: rollback oid 29 current snap is 206 2024-04-07T16:23:47.946 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 29 to 202 2024-04-07T16:23:47.946 INFO:tasks.rados.rados.0.smithi100.stdout:2184: rollback oid 20 current snap is 206 2024-04-07T16:23:47.946 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 20 to 193 2024-04-07T16:23:47.946 INFO:tasks.rados.rados.0.smithi100.stdout:2185: read oid 43 snap 183 2024-04-07T16:23:47.946 INFO:tasks.rados.rados.0.smithi100.stdout:2185: expect (ObjNum 465 snap 142 seq_num 465) 2024-04-07T16:23:47.947 INFO:tasks.rados.rados.0.smithi100.stdout:2186: delete oid 8 current snap is 206 2024-04-07T16:23:47.954 INFO:tasks.rados.rados.0.smithi100.stdout:2184: finishing rollback tid 0 to smithi10066867-20 2024-04-07T16:23:47.955 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 33 v 1222 (ObjNum 161 snap 0 seq_num 0) dirty exists 2024-04-07T16:23:47.955 INFO:tasks.rados.rados.0.smithi100.stdout:2184: finishing rollback tid 1 to smithi10066867-20 2024-04-07T16:23:47.956 INFO:tasks.rados.rados.0.smithi100.stdout:2183: finishing rollback tid 1 to smithi10066867-29 2024-04-07T16:23:47.956 INFO:tasks.rados.rados.0.smithi100.stdout:2183: finishing rollback tid 2 to smithi10066867-29 2024-04-07T16:23:47.956 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 29 v 1104 (ObjNum 567 snap 174 seq_num 567) dirty exists 2024-04-07T16:23:47.956 INFO:tasks.rados.rados.0.smithi100.stdout:2182: done (4 left) 2024-04-07T16:23:47.956 INFO:tasks.rados.rados.0.smithi100.stdout:2183: done (3 left) 2024-04-07T16:23:47.956 INFO:tasks.rados.rados.0.smithi100.stdout:2186: done (2 left) 2024-04-07T16:23:47.956 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 44 current snap is 206 2024-04-07T16:23:47.956 INFO:tasks.rados.rados.0.smithi100.stdout:2187: seq_num 675 ranges {6791168=507904,7299072=704512,8003584=499712} 2024-04-07T16:23:48.001 INFO:tasks.rados.rados.0.smithi100.stdout:2187: writing smithi10066867-44 from 6791168 to 7299072 tid 1 2024-04-07T16:23:48.005 INFO:tasks.rados.rados.0.smithi100.stdout:2187: writing smithi10066867-44 from 7299072 to 8003584 tid 2 2024-04-07T16:23:48.009 INFO:tasks.rados.rados.0.smithi100.stdout:2187: writing smithi10066867-44 from 8003584 to 8503296 tid 3 2024-04-07T16:23:48.009 INFO:tasks.rados.rados.0.smithi100.stdout:2188: delete oid 47 current snap is 206 2024-04-07T16:23:48.009 INFO:tasks.rados.rados.0.smithi100.stdout:2184: finishing rollback tid 2 to smithi10066867-20 2024-04-07T16:23:48.009 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 20 v 1100 (ObjNum 610 snap 188 seq_num 610) dirty exists 2024-04-07T16:23:48.009 INFO:tasks.rados.rados.0.smithi100.stdout:2185: expect (ObjNum 465 snap 142 seq_num 465) 2024-04-07T16:23:48.172 INFO:tasks.rados.rados.0.smithi100.stdout:2187: finishing write tid 1 to smithi10066867-44 2024-04-07T16:23:48.172 INFO:tasks.rados.rados.0.smithi100.stdout:2187: finishing write tid 2 to smithi10066867-44 2024-04-07T16:23:48.172 INFO:tasks.rados.rados.0.smithi100.stdout:2187: finishing write tid 3 to smithi10066867-44 2024-04-07T16:23:48.172 INFO:tasks.rados.rados.0.smithi100.stdout:2187: finishing write tid 4 to smithi10066867-44 2024-04-07T16:23:48.172 INFO:tasks.rados.rados.0.smithi100.stdout:2187: finishing write tid 5 to smithi10066867-44 2024-04-07T16:23:48.172 INFO:tasks.rados.rados.0.smithi100.stdout:2187: oid 44 updating version 0 to 1110 2024-04-07T16:23:48.172 INFO:tasks.rados.rados.0.smithi100.stdout:2187: oid 44 version 1110 is already newer than 1109 2024-04-07T16:23:48.172 INFO:tasks.rados.rados.0.smithi100.stdout:2187: oid 44 updating version 1110 to 1112 2024-04-07T16:23:48.172 INFO:tasks.rados.rados.0.smithi100.stdout:2187: oid 44 version 1112 is already newer than 1111 2024-04-07T16:23:48.172 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 44 v 1112 (ObjNum 675 snap 206 seq_num 675) dirty exists 2024-04-07T16:23:48.173 INFO:tasks.rados.rados.0.smithi100.stdout:2187: left oid 44 (ObjNum 675 snap 206 seq_num 675) 2024-04-07T16:23:48.173 INFO:tasks.rados.rados.0.smithi100.stdout:2184: done (3 left) 2024-04-07T16:23:48.173 INFO:tasks.rados.rados.0.smithi100.stdout:2185: done (2 left) 2024-04-07T16:23:48.173 INFO:tasks.rados.rados.0.smithi100.stdout:2187: done (1 left) 2024-04-07T16:23:48.173 INFO:tasks.rados.rados.0.smithi100.stdout:2188: done (0 left) 2024-04-07T16:23:48.173 INFO:tasks.rados.rados.0.smithi100.stdout:2189: snap_remove snap 193 2024-04-07T16:23:48.183 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:47 smithi100 bash[76202]: cluster 2024-04-07T16:23:46.684238+0000 mgr.y (mgr.25175) 186 : cluster [DBG] pgmap v301: 121 pgs: 9 peering, 3 active+undersized+degraded, 22 active+undersized, 87 active+clean; 346 MiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 7.3 MiB/s rd, 3.5 MiB/s wr, 6 op/s; 19/597 objects degraded (3.183%) 2024-04-07T16:23:48.183 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:47 smithi100 bash[76202]: cluster 2024-04-07T16:23:46.910158+0000 mon.a (mon.0) 596 : cluster [DBG] osdmap e474: 8 total, 8 up, 8 in 2024-04-07T16:23:48.184 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:47 smithi100 bash[74942]: cluster 2024-04-07T16:23:46.684238+0000 mgr.y (mgr.25175) 186 : cluster [DBG] pgmap v301: 121 pgs: 9 peering, 3 active+undersized+degraded, 22 active+undersized, 87 active+clean; 346 MiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 7.3 MiB/s rd, 3.5 MiB/s wr, 6 op/s; 19/597 objects degraded (3.183%) 2024-04-07T16:23:48.184 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:47 smithi100 bash[74942]: cluster 2024-04-07T16:23:46.910158+0000 mon.a (mon.0) 596 : cluster [DBG] osdmap e474: 8 total, 8 up, 8 in 2024-04-07T16:23:48.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:47 smithi144 bash[66654]: cluster 2024-04-07T16:23:46.684238+0000 mgr.y (mgr.25175) 186 : cluster [DBG] pgmap v301: 121 pgs: 9 peering, 3 active+undersized+degraded, 22 active+undersized, 87 active+clean; 346 MiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 7.3 MiB/s rd, 3.5 MiB/s wr, 6 op/s; 19/597 objects degraded (3.183%) 2024-04-07T16:23:48.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:47 smithi144 bash[66654]: cluster 2024-04-07T16:23:46.910158+0000 mon.a (mon.0) 596 : cluster [DBG] osdmap e474: 8 total, 8 up, 8 in 2024-04-07T16:23:48.683 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:23:48 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:23:48] "GET /metrics HTTP/1.1" 200 36061 "" "Prometheus/2.43.0" 2024-04-07T16:23:48.916 INFO:tasks.rados.rados.0.smithi100.stdout:2189: done (0 left) 2024-04-07T16:23:48.916 INFO:tasks.rados.rados.0.smithi100.stdout:2190: snap_remove snap 203 2024-04-07T16:23:49.183 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:48 smithi100 bash[74942]: cluster 2024-04-07T16:23:47.918865+0000 mon.a (mon.0) 597 : cluster [DBG] osdmap e475: 8 total, 8 up, 8 in 2024-04-07T16:23:49.184 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:48 smithi100 bash[76202]: cluster 2024-04-07T16:23:47.918865+0000 mon.a (mon.0) 597 : cluster [DBG] osdmap e475: 8 total, 8 up, 8 in 2024-04-07T16:23:49.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:48 smithi144 bash[66654]: cluster 2024-04-07T16:23:47.918865+0000 mon.a (mon.0) 597 : cluster [DBG] osdmap e475: 8 total, 8 up, 8 in 2024-04-07T16:23:49.917 INFO:tasks.rados.rados.0.smithi100.stdout:2190: done (0 left) 2024-04-07T16:23:49.918 INFO:tasks.rados.rados.0.smithi100.stdout:2191: rollback oid 34 current snap is 206 2024-04-07T16:23:49.918 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 34 to 204 2024-04-07T16:23:49.918 INFO:tasks.rados.rados.0.smithi100.stdout:2192: snap_remove snap 202 2024-04-07T16:23:50.184 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:49 smithi100 bash[74942]: cluster 2024-04-07T16:23:48.685076+0000 mgr.y (mgr.25175) 187 : cluster [DBG] pgmap v304: 121 pgs: 6 peering, 115 active+clean; 353 MiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 12 MiB/s rd, 3.2 MiB/s wr, 17 op/s; 10 MiB/s, 2 objects/s recovering 2024-04-07T16:23:50.184 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:49 smithi100 bash[74942]: cluster 2024-04-07T16:23:48.911319+0000 mon.a (mon.0) 598 : cluster [INF] Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 19/597 objects degraded (3.183%), 3 pgs degraded) 2024-04-07T16:23:50.184 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:49 smithi100 bash[74942]: cluster 2024-04-07T16:23:48.911380+0000 mon.a (mon.0) 599 : cluster [INF] Cluster is now healthy 2024-04-07T16:23:50.184 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:49 smithi100 bash[74942]: cluster 2024-04-07T16:23:48.932670+0000 mon.a (mon.0) 600 : cluster [DBG] osdmap e476: 8 total, 8 up, 8 in 2024-04-07T16:23:50.184 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:49 smithi100 bash[76202]: cluster 2024-04-07T16:23:48.685076+0000 mgr.y (mgr.25175) 187 : cluster [DBG] pgmap v304: 121 pgs: 6 peering, 115 active+clean; 353 MiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 12 MiB/s rd, 3.2 MiB/s wr, 17 op/s; 10 MiB/s, 2 objects/s recovering 2024-04-07T16:23:50.184 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:49 smithi100 bash[76202]: cluster 2024-04-07T16:23:48.911319+0000 mon.a (mon.0) 598 : cluster [INF] Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 19/597 objects degraded (3.183%), 3 pgs degraded) 2024-04-07T16:23:50.184 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:49 smithi100 bash[76202]: cluster 2024-04-07T16:23:48.911380+0000 mon.a (mon.0) 599 : cluster [INF] Cluster is now healthy 2024-04-07T16:23:50.184 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:49 smithi100 bash[76202]: cluster 2024-04-07T16:23:48.932670+0000 mon.a (mon.0) 600 : cluster [DBG] osdmap e476: 8 total, 8 up, 8 in 2024-04-07T16:23:50.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:49 smithi144 bash[66654]: cluster 2024-04-07T16:23:48.685076+0000 mgr.y (mgr.25175) 187 : cluster [DBG] pgmap v304: 121 pgs: 6 peering, 115 active+clean; 353 MiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 12 MiB/s rd, 3.2 MiB/s wr, 17 op/s; 10 MiB/s, 2 objects/s recovering 2024-04-07T16:23:50.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:49 smithi144 bash[66654]: cluster 2024-04-07T16:23:48.911319+0000 mon.a (mon.0) 598 : cluster [INF] Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 19/597 objects degraded (3.183%), 3 pgs degraded) 2024-04-07T16:23:50.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:49 smithi144 bash[66654]: cluster 2024-04-07T16:23:48.911380+0000 mon.a (mon.0) 599 : cluster [INF] Cluster is now healthy 2024-04-07T16:23:50.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:49 smithi144 bash[66654]: cluster 2024-04-07T16:23:48.932670+0000 mon.a (mon.0) 600 : cluster [DBG] osdmap e476: 8 total, 8 up, 8 in 2024-04-07T16:23:50.931 INFO:tasks.rados.rados.0.smithi100.stdout:2192: done (1 left) 2024-04-07T16:23:50.931 INFO:tasks.rados.rados.0.smithi100.stdout:2193: copy_from oid 27 from oid 37 current snap is 206 2024-04-07T16:23:50.931 INFO:tasks.rados.rados.0.smithi100.stdout:2194: snap_remove snap 191 2024-04-07T16:23:51.184 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:50 smithi100 bash[74942]: cluster 2024-04-07T16:23:49.936171+0000 mon.a (mon.0) 601 : cluster [DBG] osdmap e477: 8 total, 8 up, 8 in 2024-04-07T16:23:51.184 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:50 smithi100 bash[76202]: cluster 2024-04-07T16:23:49.936171+0000 mon.a (mon.0) 601 : cluster [DBG] osdmap e477: 8 total, 8 up, 8 in 2024-04-07T16:23:51.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:50 smithi144 bash[66654]: cluster 2024-04-07T16:23:49.936171+0000 mon.a (mon.0) 601 : cluster [DBG] osdmap e477: 8 total, 8 up, 8 in 2024-04-07T16:23:51.812 INFO:tasks.rados.rados.0.smithi100.stdout:2194: done (2 left) 2024-04-07T16:23:51.812 INFO:tasks.rados.rados.0.smithi100.stdout:2195: delete oid 45 current snap is 206 2024-04-07T16:23:51.812 INFO:tasks.rados.rados.0.smithi100.stdout:2191: finishing rollback tid 1 to smithi10066867-34 2024-04-07T16:23:51.812 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 34 v 807 (ObjNum 11585 snap 0 seq_num 3067286480) dirty dne 2024-04-07T16:23:51.812 INFO:tasks.rados.rados.0.smithi100.stdout:2193: finishing copy_from racing read to smithi10066867-27 2024-04-07T16:23:51.812 INFO:tasks.rados.rados.0.smithi100.stdout:2193: finishing copy_from to smithi10066867-27 2024-04-07T16:23:51.812 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 27 v 1089 (ObjNum 630 snap 196 seq_num 630) dirty exists 2024-04-07T16:23:51.840 INFO:tasks.rados.rados.0.smithi100.stdout:2191: done (2 left) 2024-04-07T16:23:51.840 INFO:tasks.rados.rados.0.smithi100.stdout:2193: done (1 left) 2024-04-07T16:23:51.840 INFO:tasks.rados.rados.0.smithi100.stdout:2195: done (0 left) 2024-04-07T16:23:51.840 INFO:tasks.rados.rados.0.smithi100.stdout:2196: read oid 47 snap -1 2024-04-07T16:23:51.840 INFO:tasks.rados.rados.0.smithi100.stdout:2196: expect deleted 2024-04-07T16:23:51.840 INFO:tasks.rados.rados.0.smithi100.stdout:2197: read oid 2 snap -1 2024-04-07T16:23:51.840 INFO:tasks.rados.rados.0.smithi100.stdout:2197: expect (ObjNum 659 snap 203 seq_num 659) 2024-04-07T16:23:51.840 INFO:tasks.rados.rados.0.smithi100.stdout:2198: read oid 42 snap -1 2024-04-07T16:23:51.840 INFO:tasks.rados.rados.0.smithi100.stdout:2198: expect (ObjNum 565 snap 173 seq_num 565) 2024-04-07T16:23:51.840 INFO:tasks.rados.rados.0.smithi100.stdout:2199: copy_from oid 7 from oid 24 current snap is 206 2024-04-07T16:23:51.840 INFO:tasks.rados.rados.0.smithi100.stdout:2200: copy_from oid 1 from oid 14 current snap is 206 2024-04-07T16:23:51.841 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 33 current snap is 206 2024-04-07T16:23:51.841 INFO:tasks.rados.rados.0.smithi100.stdout:2201: seq_num 678 ranges {0=565248,565248=802816,1368064=401408} 2024-04-07T16:23:51.844 INFO:tasks.rados.rados.0.smithi100.stdout:2201: writing smithi10066867-33 from 0 to 565248 tid 1 2024-04-07T16:23:51.851 INFO:tasks.rados.rados.0.smithi100.stdout:2201: writing smithi10066867-33 from 565248 to 1368064 tid 2 2024-04-07T16:23:51.855 INFO:tasks.rados.rados.0.smithi100.stdout:2201: writing smithi10066867-33 from 1368064 to 1769472 tid 3 2024-04-07T16:23:51.855 INFO:tasks.rados.rados.0.smithi100.stdout:2202: read oid 41 snap -1 2024-04-07T16:23:51.855 INFO:tasks.rados.rados.0.smithi100.stdout:2202: expect deleted 2024-04-07T16:23:51.855 INFO:tasks.rados.rados.0.smithi100.stdout:2203: rollback oid 17 current snap is 206 2024-04-07T16:23:51.855 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 17 to 195 2024-04-07T16:23:51.855 INFO:tasks.rados.rados.0.smithi100.stdout:2204: delete oid 37 current snap is 206 2024-04-07T16:23:51.858 INFO:tasks.rados.rados.0.smithi100.stdout:2200: finishing copy_from to smithi10066867-1 2024-04-07T16:23:51.858 INFO:tasks.rados.rados.0.smithi100.stdout:2200: got expected ENOENT (src dne) 2024-04-07T16:23:51.858 INFO:tasks.rados.rados.0.smithi100.stdout:2200: finishing copy_from racing read to smithi10066867-1 2024-04-07T16:23:51.867 INFO:tasks.rados.rados.0.smithi100.stdout:2197: expect (ObjNum 659 snap 203 seq_num 659) 2024-04-07T16:23:51.954 INFO:tasks.rados.rados.0.smithi100.stdout:2201: finishing write tid 1 to smithi10066867-33 2024-04-07T16:23:51.954 INFO:tasks.rados.rados.0.smithi100.stdout:2201: finishing write tid 2 to smithi10066867-33 2024-04-07T16:23:51.954 INFO:tasks.rados.rados.0.smithi100.stdout:2196: done (8 left) 2024-04-07T16:23:51.954 INFO:tasks.rados.rados.0.smithi100.stdout:2197: done (7 left) 2024-04-07T16:23:51.954 INFO:tasks.rados.rados.0.smithi100.stdout:2200: done (6 left) 2024-04-07T16:23:51.954 INFO:tasks.rados.rados.0.smithi100.stdout:2202: done (5 left) 2024-04-07T16:23:51.954 INFO:tasks.rados.rados.0.smithi100.stdout:2204: done (4 left) 2024-04-07T16:23:51.954 INFO:tasks.rados.rados.0.smithi100.stdout:2205: snap_create 2024-04-07T16:23:51.954 INFO:tasks.rados.rados.0.smithi100.stdout:2201: finishing write tid 3 to smithi10066867-33 2024-04-07T16:23:51.955 INFO:tasks.rados.rados.0.smithi100.stdout:2203: finishing rollback tid 1 to smithi10066867-17 2024-04-07T16:23:51.955 INFO:tasks.rados.rados.0.smithi100.stdout:2201: finishing write tid 4 to smithi10066867-33 2024-04-07T16:23:51.955 INFO:tasks.rados.rados.0.smithi100.stdout:2201: finishing write tid 5 to smithi10066867-33 2024-04-07T16:23:51.955 INFO:tasks.rados.rados.0.smithi100.stdout:2201: oid 33 updating version 0 to 1246 2024-04-07T16:23:51.955 INFO:tasks.rados.rados.0.smithi100.stdout:2201: oid 33 updating version 1246 to 1248 2024-04-07T16:23:51.955 INFO:tasks.rados.rados.0.smithi100.stdout:2201: oid 33 version 1248 is already newer than 1245 2024-04-07T16:23:51.955 INFO:tasks.rados.rados.0.smithi100.stdout:2201: oid 33 version 1248 is already newer than 1247 2024-04-07T16:23:51.955 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 33 v 1248 (ObjNum 678 snap 206 seq_num 678) dirty exists 2024-04-07T16:23:51.955 INFO:tasks.rados.rados.0.smithi100.stdout:2201: left oid 33 (ObjNum 678 snap 206 seq_num 678) 2024-04-07T16:23:51.955 INFO:tasks.rados.rados.0.smithi100.stdout:2203: finishing rollback tid 2 to smithi10066867-17 2024-04-07T16:23:51.955 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 17 v 1067 (ObjNum 623 snap 194 seq_num 623) dirty exists 2024-04-07T16:23:51.955 INFO:tasks.rados.rados.0.smithi100.stdout:2198: expect (ObjNum 565 snap 173 seq_num 565) 2024-04-07T16:23:52.183 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:51 smithi100 bash[74942]: cluster 2024-04-07T16:23:50.685911+0000 mgr.y (mgr.25175) 188 : cluster [DBG] pgmap v307: 121 pgs: 121 active+clean; 353 MiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 25 MiB/s rd, 5.4 MiB/s wr, 46 op/s; 12 MiB/s, 3 objects/s recovering 2024-04-07T16:23:52.183 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:51 smithi100 bash[74942]: cluster 2024-04-07T16:23:50.949502+0000 mon.a (mon.0) 602 : cluster [DBG] osdmap e478: 8 total, 8 up, 8 in 2024-04-07T16:23:52.183 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:51 smithi100 bash[74942]: cluster 2024-04-07T16:23:51.827303+0000 mon.a (mon.0) 603 : cluster [DBG] osdmap e479: 8 total, 8 up, 8 in 2024-04-07T16:23:52.184 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:51 smithi100 bash[76202]: cluster 2024-04-07T16:23:50.685911+0000 mgr.y (mgr.25175) 188 : cluster [DBG] pgmap v307: 121 pgs: 121 active+clean; 353 MiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 25 MiB/s rd, 5.4 MiB/s wr, 46 op/s; 12 MiB/s, 3 objects/s recovering 2024-04-07T16:23:52.184 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:51 smithi100 bash[76202]: cluster 2024-04-07T16:23:50.949502+0000 mon.a (mon.0) 602 : cluster [DBG] osdmap e478: 8 total, 8 up, 8 in 2024-04-07T16:23:52.184 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:51 smithi100 bash[76202]: cluster 2024-04-07T16:23:51.827303+0000 mon.a (mon.0) 603 : cluster [DBG] osdmap e479: 8 total, 8 up, 8 in 2024-04-07T16:23:52.216 INFO:tasks.rados.rados.0.smithi100.stdout:2199: finishing copy_from racing read to smithi10066867-7 2024-04-07T16:23:52.216 INFO:tasks.rados.rados.0.smithi100.stdout:2199: finishing copy_from to smithi10066867-7 2024-04-07T16:23:52.216 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 7 v 1058 (ObjNum 574 snap 177 seq_num 574) dirty exists 2024-04-07T16:23:52.305 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:51 smithi144 bash[66654]: cluster 2024-04-07T16:23:50.685911+0000 mgr.y (mgr.25175) 188 : cluster [DBG] pgmap v307: 121 pgs: 121 active+clean; 353 MiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 25 MiB/s rd, 5.4 MiB/s wr, 46 op/s; 12 MiB/s, 3 objects/s recovering 2024-04-07T16:23:52.305 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:51 smithi144 bash[66654]: cluster 2024-04-07T16:23:50.949502+0000 mon.a (mon.0) 602 : cluster [DBG] osdmap e478: 8 total, 8 up, 8 in 2024-04-07T16:23:52.305 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:51 smithi144 bash[66654]: cluster 2024-04-07T16:23:51.827303+0000 mon.a (mon.0) 603 : cluster [DBG] osdmap e479: 8 total, 8 up, 8 in 2024-04-07T16:23:52.606 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:23:52 smithi144 bash[59794]: ts=2024-04-07T16:23:52.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:23:52.933 INFO:tasks.rados.rados.0.smithi100.stdout:2198: done (4 left) 2024-04-07T16:23:52.933 INFO:tasks.rados.rados.0.smithi100.stdout:2199: done (3 left) 2024-04-07T16:23:52.933 INFO:tasks.rados.rados.0.smithi100.stdout:2201: done (2 left) 2024-04-07T16:23:52.933 INFO:tasks.rados.rados.0.smithi100.stdout:2203: done (1 left) 2024-04-07T16:23:52.933 INFO:tasks.rados.rados.0.smithi100.stdout:2205: done (0 left) 2024-04-07T16:23:52.934 INFO:tasks.rados.rados.0.smithi100.stdout:2206: snap_create 2024-04-07T16:23:53.938 INFO:tasks.rados.rados.0.smithi100.stdout:2206: done (0 left) 2024-04-07T16:23:53.938 INFO:tasks.rados.rados.0.smithi100.stdout:2207: rmattr oid 40 current snap is 208 2024-04-07T16:23:53.938 INFO:tasks.rados.rados.0.smithi100.stdout:2208: rmattr oid 12 current snap is 208 2024-04-07T16:23:53.939 INFO:tasks.rados.rados.0.smithi100.stdout:2209: rollback oid 30 current snap is 208 2024-04-07T16:23:53.939 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 30 to 204 2024-04-07T16:23:53.939 INFO:tasks.rados.rados.0.smithi100.stdout:2210: snap_create 2024-04-07T16:23:53.960 INFO:tasks.rados.rados.0.smithi100.stdout:2209: finishing rollback tid 0 to smithi10066867-30 2024-04-07T16:23:53.960 INFO:tasks.rados.rados.0.smithi100.stdout:2209: finishing rollback tid 1 to smithi10066867-30 2024-04-07T16:23:53.961 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 12 v 1101 (ObjNum 663 snap 204 seq_num 663) dirty exists 2024-04-07T16:23:53.961 INFO:tasks.rados.rados.0.smithi100.stdout:2209: finishing rollback tid 2 to smithi10066867-30 2024-04-07T16:23:53.961 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 30 v 1108 (ObjNum 650 snap 203 seq_num 650) dirty exists 2024-04-07T16:23:53.968 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 40 v 1113 (ObjNum 509 snap 152 seq_num 509) dirty exists 2024-04-07T16:23:54.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:53 smithi144 bash[66654]: cluster 2024-04-07T16:23:52.687114+0000 mgr.y (mgr.25175) 189 : cluster [DBG] pgmap v310: 121 pgs: 121 active+clean; 352 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 13 MiB/s rd, 2.3 MiB/s wr, 30 op/s; 1.5 MiB/s, 0 objects/s recovering 2024-04-07T16:23:54.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:53 smithi144 bash[66654]: cluster 2024-04-07T16:23:52.950765+0000 mon.a (mon.0) 604 : cluster [DBG] osdmap e480: 8 total, 8 up, 8 in 2024-04-07T16:23:54.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:53 smithi144 bash[66654]: audit 2024-04-07T16:23:53.163643+0000 mon.a (mon.0) 605 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:23:54.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:53 smithi144 bash[66654]: audit 2024-04-07T16:23:53.177332+0000 mon.a (mon.0) 606 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:23:54.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:53 smithi144 bash[66654]: audit 2024-04-07T16:23:53.179934+0000 mon.a (mon.0) 607 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:23:54.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:53 smithi144 bash[66654]: audit 2024-04-07T16:23:53.181814+0000 mon.a (mon.0) 608 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:23:54.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:53 smithi144 bash[66654]: audit 2024-04-07T16:23:53.196263+0000 mon.a (mon.0) 609 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:23:54.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:53 smithi144 bash[66654]: audit 2024-04-07T16:23:53.293626+0000 mon.a (mon.0) 610 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:23:54.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:53 smithi144 bash[66654]: audit 2024-04-07T16:23:53.299039+0000 mon.a (mon.0) 611 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:23:54.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:53 smithi144 bash[66654]: audit 2024-04-07T16:23:53.303218+0000 mon.a (mon.0) 612 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:23:54.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:53 smithi144 bash[66654]: audit 2024-04-07T16:23:53.306266+0000 mon.a (mon.0) 613 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:23:54.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:53 smithi144 bash[66654]: audit 2024-04-07T16:23:53.310079+0000 mon.a (mon.0) 614 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-04-07T16:23:54.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:53 smithi144 bash[66654]: audit 2024-04-07T16:23:53.310425+0000 mgr.y (mgr.25175) 190 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-04-07T16:23:54.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:53 smithi144 bash[66654]: cephadm 2024-04-07T16:23:53.312702+0000 mgr.y (mgr.25175) 191 : cephadm [INF] Upgrade: osd.4 is safe to restart 2024-04-07T16:23:54.434 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:53 smithi100 bash[76202]: cluster 2024-04-07T16:23:52.687114+0000 mgr.y (mgr.25175) 189 : cluster [DBG] pgmap v310: 121 pgs: 121 active+clean; 352 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 13 MiB/s rd, 2.3 MiB/s wr, 30 op/s; 1.5 MiB/s, 0 objects/s recovering 2024-04-07T16:23:54.435 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:53 smithi100 bash[76202]: cluster 2024-04-07T16:23:52.950765+0000 mon.a (mon.0) 604 : cluster [DBG] osdmap e480: 8 total, 8 up, 8 in 2024-04-07T16:23:54.435 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:53 smithi100 bash[76202]: audit 2024-04-07T16:23:53.163643+0000 mon.a (mon.0) 605 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:23:54.435 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:53 smithi100 bash[76202]: audit 2024-04-07T16:23:53.177332+0000 mon.a (mon.0) 606 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:23:54.435 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:53 smithi100 bash[76202]: audit 2024-04-07T16:23:53.179934+0000 mon.a (mon.0) 607 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:23:54.435 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:53 smithi100 bash[76202]: audit 2024-04-07T16:23:53.181814+0000 mon.a (mon.0) 608 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:23:54.435 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:53 smithi100 bash[76202]: audit 2024-04-07T16:23:53.196263+0000 mon.a (mon.0) 609 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:23:54.435 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:53 smithi100 bash[76202]: audit 2024-04-07T16:23:53.293626+0000 mon.a (mon.0) 610 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:23:54.435 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:53 smithi100 bash[76202]: audit 2024-04-07T16:23:53.299039+0000 mon.a (mon.0) 611 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:23:54.435 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:53 smithi100 bash[76202]: audit 2024-04-07T16:23:53.303218+0000 mon.a (mon.0) 612 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:23:54.435 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:53 smithi100 bash[76202]: audit 2024-04-07T16:23:53.306266+0000 mon.a (mon.0) 613 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:23:54.435 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:53 smithi100 bash[76202]: audit 2024-04-07T16:23:53.310079+0000 mon.a (mon.0) 614 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-04-07T16:23:54.435 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:53 smithi100 bash[76202]: audit 2024-04-07T16:23:53.310425+0000 mgr.y (mgr.25175) 190 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-04-07T16:23:54.435 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:53 smithi100 bash[76202]: cephadm 2024-04-07T16:23:53.312702+0000 mgr.y (mgr.25175) 191 : cephadm [INF] Upgrade: osd.4 is safe to restart 2024-04-07T16:23:54.436 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:53 smithi100 bash[74942]: cluster 2024-04-07T16:23:52.687114+0000 mgr.y (mgr.25175) 189 : cluster [DBG] pgmap v310: 121 pgs: 121 active+clean; 352 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 13 MiB/s rd, 2.3 MiB/s wr, 30 op/s; 1.5 MiB/s, 0 objects/s recovering 2024-04-07T16:23:54.436 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:53 smithi100 bash[74942]: cluster 2024-04-07T16:23:52.950765+0000 mon.a (mon.0) 604 : cluster [DBG] osdmap e480: 8 total, 8 up, 8 in 2024-04-07T16:23:54.436 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:53 smithi100 bash[74942]: audit 2024-04-07T16:23:53.163643+0000 mon.a (mon.0) 605 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:23:54.436 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:53 smithi100 bash[74942]: audit 2024-04-07T16:23:53.177332+0000 mon.a (mon.0) 606 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:23:54.436 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:53 smithi100 bash[74942]: audit 2024-04-07T16:23:53.179934+0000 mon.a (mon.0) 607 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:23:54.436 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:53 smithi100 bash[74942]: audit 2024-04-07T16:23:53.181814+0000 mon.a (mon.0) 608 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:23:54.436 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:53 smithi100 bash[74942]: audit 2024-04-07T16:23:53.196263+0000 mon.a (mon.0) 609 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:23:54.436 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:53 smithi100 bash[74942]: audit 2024-04-07T16:23:53.293626+0000 mon.a (mon.0) 610 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:23:54.436 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:53 smithi100 bash[74942]: audit 2024-04-07T16:23:53.299039+0000 mon.a (mon.0) 611 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:23:54.436 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:53 smithi100 bash[74942]: audit 2024-04-07T16:23:53.303218+0000 mon.a (mon.0) 612 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:23:54.436 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:53 smithi100 bash[74942]: audit 2024-04-07T16:23:53.306266+0000 mon.a (mon.0) 613 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:23:54.436 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:53 smithi100 bash[74942]: audit 2024-04-07T16:23:53.310079+0000 mon.a (mon.0) 614 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-04-07T16:23:54.436 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:53 smithi100 bash[74942]: audit 2024-04-07T16:23:53.310425+0000 mgr.y (mgr.25175) 190 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-04-07T16:23:54.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:53 smithi100 bash[74942]: cephadm 2024-04-07T16:23:53.312702+0000 mgr.y (mgr.25175) 191 : cephadm [INF] Upgrade: osd.4 is safe to restart 2024-04-07T16:23:54.960 INFO:tasks.rados.rados.0.smithi100.stdout:2207: done (3 left) 2024-04-07T16:23:54.960 INFO:tasks.rados.rados.0.smithi100.stdout:2208: done (2 left) 2024-04-07T16:23:54.960 INFO:tasks.rados.rados.0.smithi100.stdout:2209: done (1 left) 2024-04-07T16:23:54.960 INFO:tasks.rados.rados.0.smithi100.stdout:2210: done (0 left) 2024-04-07T16:23:54.960 INFO:tasks.rados.rados.0.smithi100.stdout:2211: rollback oid 31 current snap is 209 2024-04-07T16:23:54.960 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 31 to 188 2024-04-07T16:23:54.960 INFO:tasks.rados.rados.0.smithi100.stdout:2212: snap_remove snap 172 2024-04-07T16:23:55.018 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:54 smithi144 bash[66654]: cluster 2024-04-07T16:23:53.956020+0000 mon.a (mon.0) 615 : cluster [DBG] osdmap e481: 8 total, 8 up, 8 in 2024-04-07T16:23:55.434 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:54 smithi100 bash[74942]: cluster 2024-04-07T16:23:53.956020+0000 mon.a (mon.0) 615 : cluster [DBG] osdmap e481: 8 total, 8 up, 8 in 2024-04-07T16:23:55.434 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:54 smithi100 bash[76202]: cluster 2024-04-07T16:23:53.956020+0000 mon.a (mon.0) 615 : cluster [DBG] osdmap e481: 8 total, 8 up, 8 in 2024-04-07T16:23:55.970 INFO:tasks.rados.rados.0.smithi100.stdout:2211: finishing rollback tid 1 to smithi10066867-31 2024-04-07T16:23:55.970 INFO:tasks.rados.rados.0.smithi100.stdout:2211: finishing rollback tid 2 to smithi10066867-31 2024-04-07T16:23:55.970 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 31 v 1072 (ObjNum 579 snap 180 seq_num 579) dirty exists 2024-04-07T16:23:55.970 INFO:tasks.rados.rados.0.smithi100.stdout:2211: done (1 left) 2024-04-07T16:23:55.970 INFO:tasks.rados.rados.0.smithi100.stdout:2212: done (0 left) 2024-04-07T16:23:55.970 INFO:tasks.rados.rados.0.smithi100.stdout:2213: rmattr oid 25 current snap is 209 2024-04-07T16:23:55.970 INFO:tasks.rados.rados.0.smithi100.stdout:2213: done (0 left) 2024-04-07T16:23:55.970 INFO:tasks.rados.rados.0.smithi100.stdout:2214: snap_remove snap 207 2024-04-07T16:23:56.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:55 smithi144 bash[66654]: cluster 2024-04-07T16:23:54.687857+0000 mgr.y (mgr.25175) 192 : cluster [DBG] pgmap v313: 121 pgs: 121 active+clean; 351 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 5.7 MiB/s rd, 432 KiB/s wr, 5 op/s 2024-04-07T16:23:56.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:55 smithi144 bash[66654]: cluster 2024-04-07T16:23:54.978458+0000 mon.a (mon.0) 616 : cluster [DBG] osdmap e482: 8 total, 8 up, 8 in 2024-04-07T16:23:56.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:55 smithi144 bash[66654]: audit 2024-04-07T16:23:55.691016+0000 mon.a (mon.0) 617 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:23:56.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:55 smithi144 bash[66654]: audit 2024-04-07T16:23:55.695617+0000 mon.a (mon.0) 618 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-07T16:23:56.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:55 smithi144 bash[66654]: audit 2024-04-07T16:23:55.697405+0000 mon.a (mon.0) 619 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:23:56.434 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:55 smithi100 bash[74942]: cluster 2024-04-07T16:23:54.687857+0000 mgr.y (mgr.25175) 192 : cluster [DBG] pgmap v313: 121 pgs: 121 active+clean; 351 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 5.7 MiB/s rd, 432 KiB/s wr, 5 op/s 2024-04-07T16:23:56.434 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:55 smithi100 bash[74942]: cluster 2024-04-07T16:23:54.978458+0000 mon.a (mon.0) 616 : cluster [DBG] osdmap e482: 8 total, 8 up, 8 in 2024-04-07T16:23:56.434 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:55 smithi100 bash[74942]: audit 2024-04-07T16:23:55.691016+0000 mon.a (mon.0) 617 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:23:56.434 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:55 smithi100 bash[74942]: audit 2024-04-07T16:23:55.695617+0000 mon.a (mon.0) 618 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-07T16:23:56.434 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:55 smithi100 bash[74942]: audit 2024-04-07T16:23:55.697405+0000 mon.a (mon.0) 619 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:23:56.434 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:55 smithi100 bash[76202]: cluster 2024-04-07T16:23:54.687857+0000 mgr.y (mgr.25175) 192 : cluster [DBG] pgmap v313: 121 pgs: 121 active+clean; 351 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 5.7 MiB/s rd, 432 KiB/s wr, 5 op/s 2024-04-07T16:23:56.434 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:55 smithi100 bash[76202]: cluster 2024-04-07T16:23:54.978458+0000 mon.a (mon.0) 616 : cluster [DBG] osdmap e482: 8 total, 8 up, 8 in 2024-04-07T16:23:56.435 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:55 smithi100 bash[76202]: audit 2024-04-07T16:23:55.691016+0000 mon.a (mon.0) 617 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:23:56.435 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:55 smithi100 bash[76202]: audit 2024-04-07T16:23:55.695617+0000 mon.a (mon.0) 618 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-07T16:23:56.435 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:55 smithi100 bash[76202]: audit 2024-04-07T16:23:55.697405+0000 mon.a (mon.0) 619 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:23:56.815 INFO:tasks.rados.rados.0.smithi100.stdout:2214: done (0 left) 2024-04-07T16:23:56.815 INFO:tasks.rados.rados.0.smithi100.stdout:2215: rollback oid 28 current snap is 209 2024-04-07T16:23:56.815 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 28 to 183 2024-04-07T16:23:56.815 INFO:tasks.rados.rados.0.smithi100.stdout:2216: read oid 11 snap -1 2024-04-07T16:23:56.815 INFO:tasks.rados.rados.0.smithi100.stdout:2216: expect (ObjNum 537 snap 162 seq_num 537) 2024-04-07T16:23:56.815 INFO:tasks.rados.rados.0.smithi100.stdout:2217: read oid 45 snap -1 2024-04-07T16:23:56.815 INFO:tasks.rados.rados.0.smithi100.stdout:2217: expect deleted 2024-04-07T16:23:56.815 INFO:tasks.rados.rados.0.smithi100.stdout:2218: snap_remove snap 177 2024-04-07T16:23:57.279 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:56 smithi144 bash[66654]: cephadm 2024-04-07T16:23:55.683651+0000 mgr.y (mgr.25175) 193 : cephadm [INF] Upgrade: Updating osd.4 2024-04-07T16:23:57.279 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:56 smithi144 bash[66654]: cephadm 2024-04-07T16:23:55.702769+0000 mgr.y (mgr.25175) 194 : cephadm [INF] Deploying daemon osd.4 on smithi144 2024-04-07T16:23:57.279 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:56 smithi144 bash[66654]: cluster 2024-04-07T16:23:55.981523+0000 mon.a (mon.0) 620 : cluster [DBG] osdmap e483: 8 total, 8 up, 8 in 2024-04-07T16:23:57.279 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:56 smithi144 bash[66654]: cluster 2024-04-07T16:23:56.832953+0000 mon.a (mon.0) 621 : cluster [DBG] osdmap e484: 8 total, 8 up, 8 in 2024-04-07T16:23:57.279 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:56 smithi144 bash[66654]: audit 2024-04-07T16:23:56.874502+0000 mon.a (mon.0) 622 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:23:57.434 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:56 smithi100 bash[74942]: cephadm 2024-04-07T16:23:55.683651+0000 mgr.y (mgr.25175) 193 : cephadm [INF] Upgrade: Updating osd.4 2024-04-07T16:23:57.434 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:56 smithi100 bash[74942]: cephadm 2024-04-07T16:23:55.702769+0000 mgr.y (mgr.25175) 194 : cephadm [INF] Deploying daemon osd.4 on smithi144 2024-04-07T16:23:57.434 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:56 smithi100 bash[74942]: cluster 2024-04-07T16:23:55.981523+0000 mon.a (mon.0) 620 : cluster [DBG] osdmap e483: 8 total, 8 up, 8 in 2024-04-07T16:23:57.434 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:56 smithi100 bash[74942]: cluster 2024-04-07T16:23:56.832953+0000 mon.a (mon.0) 621 : cluster [DBG] osdmap e484: 8 total, 8 up, 8 in 2024-04-07T16:23:57.434 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:56 smithi100 bash[74942]: audit 2024-04-07T16:23:56.874502+0000 mon.a (mon.0) 622 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:23:57.434 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:56 smithi100 bash[76202]: cephadm 2024-04-07T16:23:55.683651+0000 mgr.y (mgr.25175) 193 : cephadm [INF] Upgrade: Updating osd.4 2024-04-07T16:23:57.434 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:56 smithi100 bash[76202]: cephadm 2024-04-07T16:23:55.702769+0000 mgr.y (mgr.25175) 194 : cephadm [INF] Deploying daemon osd.4 on smithi144 2024-04-07T16:23:57.435 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:56 smithi100 bash[76202]: cluster 2024-04-07T16:23:55.981523+0000 mon.a (mon.0) 620 : cluster [DBG] osdmap e483: 8 total, 8 up, 8 in 2024-04-07T16:23:57.435 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:56 smithi100 bash[76202]: cluster 2024-04-07T16:23:56.832953+0000 mon.a (mon.0) 621 : cluster [DBG] osdmap e484: 8 total, 8 up, 8 in 2024-04-07T16:23:57.435 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:56 smithi100 bash[76202]: audit 2024-04-07T16:23:56.874502+0000 mon.a (mon.0) 622 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:23:57.815 INFO:tasks.rados.rados.0.smithi100.stdout:2218: done (3 left) 2024-04-07T16:23:57.815 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 46 current snap is 209 2024-04-07T16:23:57.816 INFO:tasks.rados.rados.0.smithi100.stdout:2215: finishing rollback tid 0 to smithi10066867-28 2024-04-07T16:23:57.816 INFO:tasks.rados.rados.0.smithi100.stdout:2215: finishing rollback tid 1 to smithi10066867-28 2024-04-07T16:23:57.816 INFO:tasks.rados.rados.0.smithi100.stdout:2219: seq_num 680 ranges {8044544=598016,8642560=573440,9216000=434176,9650176=196608} 2024-04-07T16:23:57.914 INFO:tasks.rados.rados.0.smithi100.stdout:2219: writing smithi10066867-46 from 8044544 to 8642560 tid 1 2024-04-07T16:23:57.918 INFO:tasks.rados.rados.0.smithi100.stdout:2219: writing smithi10066867-46 from 8642560 to 9216000 tid 2 2024-04-07T16:23:57.922 INFO:tasks.rados.rados.0.smithi100.stdout:2219: writing smithi10066867-46 from 9216000 to 9650176 tid 3 2024-04-07T16:23:57.923 INFO:tasks.rados.rados.0.smithi100.stdout:2219: writing smithi10066867-46 from 9650176 to 9846784 tid 4 2024-04-07T16:23:57.924 INFO:tasks.rados.rados.0.smithi100.stdout:2217: done (3 left) 2024-04-07T16:23:57.924 INFO:tasks.rados.rados.0.smithi100.stdout:2220: snap_create 2024-04-07T16:23:57.924 INFO:tasks.rados.rados.0.smithi100.stdout:2215: finishing rollback tid 2 to smithi10066867-28 2024-04-07T16:23:57.924 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 28 v 1093 (ObjNum 461 snap 140 seq_num 461) dirty exists 2024-04-07T16:23:57.924 INFO:tasks.rados.rados.0.smithi100.stdout:2216: expect (ObjNum 537 snap 162 seq_num 537) 2024-04-07T16:23:58.028 INFO:tasks.rados.rados.0.smithi100.stdout:2219: finishing write tid 1 to smithi10066867-46 2024-04-07T16:23:58.028 INFO:tasks.rados.rados.0.smithi100.stdout:2219: finishing write tid 2 to smithi10066867-46 2024-04-07T16:23:58.028 INFO:tasks.rados.rados.0.smithi100.stdout:2219: finishing write tid 3 to smithi10066867-46 2024-04-07T16:23:58.028 INFO:tasks.rados.rados.0.smithi100.stdout:2219: finishing write tid 4 to smithi10066867-46 2024-04-07T16:23:58.028 INFO:tasks.rados.rados.0.smithi100.stdout:2219: finishing write tid 5 to smithi10066867-46 2024-04-07T16:23:58.028 INFO:tasks.rados.rados.0.smithi100.stdout:2219: finishing write tid 6 to smithi10066867-46 2024-04-07T16:23:58.028 INFO:tasks.rados.rados.0.smithi100.stdout:2219: oid 46 updating version 0 to 927 2024-04-07T16:23:58.028 INFO:tasks.rados.rados.0.smithi100.stdout:2219: oid 46 updating version 927 to 929 2024-04-07T16:23:58.028 INFO:tasks.rados.rados.0.smithi100.stdout:2219: oid 46 version 929 is already newer than 926 2024-04-07T16:23:58.028 INFO:tasks.rados.rados.0.smithi100.stdout:2219: oid 46 version 929 is already newer than 928 2024-04-07T16:23:58.028 INFO:tasks.rados.rados.0.smithi100.stdout:2219: oid 46 updating version 929 to 930 2024-04-07T16:23:58.029 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 46 v 930 (ObjNum 680 snap 209 seq_num 680) dirty exists 2024-04-07T16:23:58.029 INFO:tasks.rados.rados.0.smithi100.stdout:2219: left oid 46 (ObjNum 680 snap 209 seq_num 680) 2024-04-07T16:23:58.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:57 smithi144 bash[66654]: cluster 2024-04-07T16:23:56.688514+0000 mgr.y (mgr.25175) 195 : cluster [DBG] pgmap v316: 121 pgs: 121 active+clean; 355 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 5.7 MiB/s rd, 1.7 MiB/s wr, 5 op/s 2024-04-07T16:23:58.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:57 smithi144 bash[66654]: cluster 2024-04-07T16:23:57.833345+0000 mon.a (mon.0) 623 : cluster [DBG] osdmap e485: 8 total, 8 up, 8 in 2024-04-07T16:23:58.416 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:58 smithi100 bash[76202]: cluster 2024-04-07T16:23:56.688514+0000 mgr.y (mgr.25175) 195 : cluster [DBG] pgmap v316: 121 pgs: 121 active+clean; 355 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 5.7 MiB/s rd, 1.7 MiB/s wr, 5 op/s 2024-04-07T16:23:58.417 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:58 smithi100 bash[76202]: cluster 2024-04-07T16:23:57.833345+0000 mon.a (mon.0) 623 : cluster [DBG] osdmap e485: 8 total, 8 up, 8 in 2024-04-07T16:23:58.417 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:57 smithi100 bash[74942]: cluster 2024-04-07T16:23:56.688514+0000 mgr.y (mgr.25175) 195 : cluster [DBG] pgmap v316: 121 pgs: 121 active+clean; 355 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 5.7 MiB/s rd, 1.7 MiB/s wr, 5 op/s 2024-04-07T16:23:58.417 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:57 smithi100 bash[74942]: cluster 2024-04-07T16:23:57.833345+0000 mon.a (mon.0) 623 : cluster [DBG] osdmap e485: 8 total, 8 up, 8 in 2024-04-07T16:23:58.683 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:23:58 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:23:58] "GET /metrics HTTP/1.1" 200 36061 "" "Prometheus/2.43.0" 2024-04-07T16:23:58.820 INFO:tasks.rados.rados.0.smithi100.stdout:2215: done (3 left) 2024-04-07T16:23:58.820 INFO:tasks.rados.rados.0.smithi100.stdout:2216: done (2 left) 2024-04-07T16:23:58.820 INFO:tasks.rados.rados.0.smithi100.stdout:2219: done (1 left) 2024-04-07T16:23:58.820 INFO:tasks.rados.rados.0.smithi100.stdout:2220: done (0 left) 2024-04-07T16:23:58.820 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 21 current snap is 210 2024-04-07T16:23:58.820 INFO:tasks.rados.rados.0.smithi100.stdout:2221: seq_num 681 ranges {5685248=614400,6299648=458752,6758400=638976,7397376=106496} 2024-04-07T16:23:58.857 INFO:tasks.rados.rados.0.smithi100.stdout:2221: writing smithi10066867-21 from 5685248 to 6299648 tid 1 2024-04-07T16:23:58.860 INFO:tasks.rados.rados.0.smithi100.stdout:2221: writing smithi10066867-21 from 6299648 to 6758400 tid 2 2024-04-07T16:23:58.864 INFO:tasks.rados.rados.0.smithi100.stdout:2221: writing smithi10066867-21 from 6758400 to 7397376 tid 3 2024-04-07T16:23:58.865 INFO:tasks.rados.rados.0.smithi100.stdout:2221: writing smithi10066867-21 from 7397376 to 7503872 tid 4 2024-04-07T16:23:58.865 INFO:tasks.rados.rados.0.smithi100.stdout:2222: snap_remove snap 208 2024-04-07T16:23:59.832 INFO:tasks.rados.rados.0.smithi100.stdout:2222: done (1 left) 2024-04-07T16:23:59.832 INFO:tasks.rados.rados.0.smithi100.stdout:2223: read oid 46 snap -1 2024-04-07T16:23:59.832 INFO:tasks.rados.rados.0.smithi100.stdout:2223: expect (ObjNum 680 snap 209 seq_num 680) 2024-04-07T16:23:59.832 INFO:tasks.rados.rados.0.smithi100.stdout:2224: read oid 14 snap -1 2024-04-07T16:23:59.832 INFO:tasks.rados.rados.0.smithi100.stdout:2224: expect deleted 2024-04-07T16:23:59.832 INFO:tasks.rados.rados.0.smithi100.stdout:2225: delete oid 1 current snap is 210 2024-04-07T16:23:59.833 INFO:tasks.rados.rados.0.smithi100.stdout:2221: finishing write tid 1 to smithi10066867-21 2024-04-07T16:23:59.833 INFO:tasks.rados.rados.0.smithi100.stdout:2221: finishing write tid 2 to smithi10066867-21 2024-04-07T16:23:59.833 INFO:tasks.rados.rados.0.smithi100.stdout:2221: finishing write tid 3 to smithi10066867-21 2024-04-07T16:23:59.833 INFO:tasks.rados.rados.0.smithi100.stdout:2221: finishing write tid 4 to smithi10066867-21 2024-04-07T16:23:59.833 INFO:tasks.rados.rados.0.smithi100.stdout:2221: finishing write tid 5 to smithi10066867-21 2024-04-07T16:23:59.833 INFO:tasks.rados.rados.0.smithi100.stdout:2221: finishing write tid 6 to smithi10066867-21 2024-04-07T16:23:59.833 INFO:tasks.rados.rados.0.smithi100.stdout:2221: oid 21 updating version 0 to 814 2024-04-07T16:23:59.833 INFO:tasks.rados.rados.0.smithi100.stdout:2221: oid 21 updating version 814 to 815 2024-04-07T16:23:59.833 INFO:tasks.rados.rados.0.smithi100.stdout:2221: oid 21 updating version 815 to 817 2024-04-07T16:23:59.833 INFO:tasks.rados.rados.0.smithi100.stdout:2221: oid 21 updating version 817 to 818 2024-04-07T16:23:59.833 INFO:tasks.rados.rados.0.smithi100.stdout:2221: oid 21 version 818 is already newer than 816 2024-04-07T16:23:59.833 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 21 v 818 (ObjNum 681 snap 210 seq_num 681) dirty exists 2024-04-07T16:23:59.833 INFO:tasks.rados.rados.0.smithi100.stdout:2221: left oid 21 (ObjNum 681 snap 210 seq_num 681) 2024-04-07T16:23:59.845 INFO:tasks.rados.rados.0.smithi100.stdout:2221: done (3 left) 2024-04-07T16:23:59.845 INFO:tasks.rados.rados.0.smithi100.stdout:2224: done (2 left) 2024-04-07T16:23:59.845 INFO:tasks.rados.rados.0.smithi100.stdout:2225: done (1 left) 2024-04-07T16:23:59.845 INFO:tasks.rados.rados.0.smithi100.stdout:2226: delete oid 42 current snap is 210 2024-04-07T16:23:59.850 INFO:tasks.rados.rados.0.smithi100.stdout:2226: done (1 left) 2024-04-07T16:23:59.850 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 47 current snap is 210 2024-04-07T16:23:59.851 INFO:tasks.rados.rados.0.smithi100.stdout:2227: seq_num 684 ranges {0=663552,663552=507904,1171456=409600,1581056=286720} 2024-04-07T16:23:59.857 INFO:tasks.rados.rados.0.smithi100.stdout:2227: writing smithi10066867-47 from 0 to 663552 tid 1 2024-04-07T16:23:59.862 INFO:tasks.rados.rados.0.smithi100.stdout:2227: writing smithi10066867-47 from 663552 to 1171456 tid 2 2024-04-07T16:23:59.866 INFO:tasks.rados.rados.0.smithi100.stdout:2227: writing smithi10066867-47 from 1171456 to 1581056 tid 3 2024-04-07T16:23:59.868 INFO:tasks.rados.rados.0.smithi100.stdout:2227: writing smithi10066867-47 from 1581056 to 1867776 tid 4 2024-04-07T16:23:59.868 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 1 current snap is 210 2024-04-07T16:23:59.868 INFO:tasks.rados.rados.0.smithi100.stdout:2227: finishing write tid 1 to smithi10066867-47 2024-04-07T16:23:59.869 INFO:tasks.rados.rados.0.smithi100.stdout:2227: finishing write tid 2 to smithi10066867-47 2024-04-07T16:23:59.869 INFO:tasks.rados.rados.0.smithi100.stdout:2228: seq_num 685 ranges {0=679936,679936=442368,1122304=622592,1744896=139264} 2024-04-07T16:23:59.875 INFO:tasks.rados.rados.0.smithi100.stdout:2228: writing smithi10066867-1 from 0 to 679936 tid 1 2024-04-07T16:23:59.882 INFO:tasks.rados.rados.0.smithi100.stdout:2228: writing smithi10066867-1 from 679936 to 1122304 tid 2 2024-04-07T16:23:59.889 INFO:tasks.rados.rados.0.smithi100.stdout:2228: writing smithi10066867-1 from 1122304 to 1744896 tid 3 2024-04-07T16:23:59.891 INFO:tasks.rados.rados.0.smithi100.stdout:2228: writing smithi10066867-1 from 1744896 to 1884160 tid 4 2024-04-07T16:23:59.891 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 49 current snap is 210 2024-04-07T16:23:59.891 INFO:tasks.rados.rados.0.smithi100.stdout:2227: finishing write tid 3 to smithi10066867-47 2024-04-07T16:23:59.891 INFO:tasks.rados.rados.0.smithi100.stdout:2227: finishing write tid 4 to smithi10066867-47 2024-04-07T16:23:59.891 INFO:tasks.rados.rados.0.smithi100.stdout:2227: finishing write tid 5 to smithi10066867-47 2024-04-07T16:23:59.892 INFO:tasks.rados.rados.0.smithi100.stdout:2227: finishing write tid 6 to smithi10066867-47 2024-04-07T16:23:59.892 INFO:tasks.rados.rados.0.smithi100.stdout:2227: oid 47 updating version 0 to 820 2024-04-07T16:23:59.892 INFO:tasks.rados.rados.0.smithi100.stdout:2227: oid 47 updating version 820 to 823 2024-04-07T16:23:59.892 INFO:tasks.rados.rados.0.smithi100.stdout:2227: oid 47 version 823 is already newer than 822 2024-04-07T16:23:59.892 INFO:tasks.rados.rados.0.smithi100.stdout:2227: oid 47 version 823 is already newer than 821 2024-04-07T16:23:59.892 INFO:tasks.rados.rados.0.smithi100.stdout:2227: oid 47 version 823 is already newer than 819 2024-04-07T16:23:59.892 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 47 v 823 (ObjNum 684 snap 210 seq_num 684) dirty exists 2024-04-07T16:23:59.892 INFO:tasks.rados.rados.0.smithi100.stdout:2227: left oid 47 (ObjNum 684 snap 210 seq_num 684) 2024-04-07T16:23:59.892 INFO:tasks.rados.rados.0.smithi100.stdout:2228: finishing write tid 1 to smithi10066867-1 2024-04-07T16:23:59.892 INFO:tasks.rados.rados.0.smithi100.stdout:2229: seq_num 686 ranges {417792=696320,1114112=729088,1843200=425984,2269184=49152} 2024-04-07T16:23:59.905 INFO:tasks.rados.rados.0.smithi100.stdout:2229: writing smithi10066867-49 from 417792 to 1114112 tid 1 2024-04-07T16:23:59.909 INFO:tasks.rados.rados.0.smithi100.stdout:2229: writing smithi10066867-49 from 1114112 to 1843200 tid 2 2024-04-07T16:23:59.913 INFO:tasks.rados.rados.0.smithi100.stdout:2229: writing smithi10066867-49 from 1843200 to 2269184 tid 3 2024-04-07T16:23:59.913 INFO:tasks.rados.rados.0.smithi100.stdout:2229: writing smithi10066867-49 from 2269184 to 2318336 tid 4 2024-04-07T16:23:59.914 INFO:tasks.rados.rados.0.smithi100.stdout:2227: done (3 left) 2024-04-07T16:23:59.914 INFO:tasks.rados.rados.0.smithi100.stdout:2230: setattr oid 2 current snap is 210 2024-04-07T16:23:59.914 INFO:tasks.rados.rados.0.smithi100.stdout:2228: finishing write tid 2 to smithi10066867-1 2024-04-07T16:23:59.914 INFO:tasks.rados.rados.0.smithi100.stdout:2228: finishing write tid 3 to smithi10066867-1 2024-04-07T16:23:59.914 INFO:tasks.rados.rados.0.smithi100.stdout:2223: expect (ObjNum 680 snap 209 seq_num 680) 2024-04-07T16:24:00.035 INFO:journalctl@ceph.osd.6.smithi144.stdout:Apr 07 16:23:59 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:24:00.035 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:23:59 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:24:00.035 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:23:59 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:24:00.036 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:59 smithi144 bash[66654]: cluster 2024-04-07T16:23:58.689361+0000 mgr.y (mgr.25175) 196 : cluster [DBG] pgmap v319: 121 pgs: 121 active+clean; 338 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 0 B/s rd, 1.2 MiB/s wr, 2 op/s 2024-04-07T16:24:00.036 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:59 smithi144 bash[66654]: cluster 2024-04-07T16:23:58.838276+0000 mon.a (mon.0) 624 : cluster [DBG] osdmap e486: 8 total, 8 up, 8 in 2024-04-07T16:24:00.036 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:23:59 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:24:00.036 INFO:journalctl@ceph.osd.4.smithi144.stdout:Apr 07 16:23:59 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:24:00.036 INFO:journalctl@ceph.osd.7.smithi144.stdout:Apr 07 16:23:59 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:24:00.037 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:23:59 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:24:00.037 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 16:23:59 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:24:00.037 INFO:journalctl@ceph.osd.5.smithi144.stdout:Apr 07 16:23:59 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:24:00.038 INFO:journalctl@ceph.mds.b.smithi144.stdout:Apr 07 16:23:59 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:24:00.183 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:59 smithi100 bash[74942]: cluster 2024-04-07T16:23:58.689361+0000 mgr.y (mgr.25175) 196 : cluster [DBG] pgmap v319: 121 pgs: 121 active+clean; 338 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 0 B/s rd, 1.2 MiB/s wr, 2 op/s 2024-04-07T16:24:00.183 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:23:59 smithi100 bash[74942]: cluster 2024-04-07T16:23:58.838276+0000 mon.a (mon.0) 624 : cluster [DBG] osdmap e486: 8 total, 8 up, 8 in 2024-04-07T16:24:00.184 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:59 smithi100 bash[76202]: cluster 2024-04-07T16:23:58.689361+0000 mgr.y (mgr.25175) 196 : cluster [DBG] pgmap v319: 121 pgs: 121 active+clean; 338 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 0 B/s rd, 1.2 MiB/s wr, 2 op/s 2024-04-07T16:24:00.184 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:23:59 smithi100 bash[76202]: cluster 2024-04-07T16:23:58.838276+0000 mon.a (mon.0) 624 : cluster [DBG] osdmap e486: 8 total, 8 up, 8 in 2024-04-07T16:24:00.356 INFO:journalctl@ceph.osd.4.smithi144.stdout:Apr 07 16:24:00 smithi144 systemd[1]: Stopping Ceph osd.4 for 0702e3a8-f4f8-11ee-b649-cb9ed24678a4... 2024-04-07T16:24:00.356 INFO:journalctl@ceph.osd.4.smithi144.stdout:Apr 07 16:24:00 smithi144 bash[27081]: debug 2024-04-07T16:24:00.142+0000 7fd618a36700 -1 received signal: Terminated from /sbin/docker-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug (PID: 1) UID: 0 2024-04-07T16:24:00.356 INFO:journalctl@ceph.osd.4.smithi144.stdout:Apr 07 16:24:00 smithi144 bash[27081]: debug 2024-04-07T16:24:00.142+0000 7fd618a36700 -1 osd.4 487 *** Got signal Terminated *** 2024-04-07T16:24:00.356 INFO:journalctl@ceph.osd.4.smithi144.stdout:Apr 07 16:24:00 smithi144 bash[27081]: debug 2024-04-07T16:24:00.142+0000 7fd618a36700 -1 osd.4 487 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-07T16:24:00.561 INFO:tasks.rados.rados.0.smithi100.stdout:2228: finishing write tid 4 to smithi10066867-1 2024-04-07T16:24:00.561 INFO:tasks.rados.rados.0.smithi100.stdout:2228: finishing write tid 5 to smithi10066867-1 2024-04-07T16:24:00.562 INFO:tasks.rados.rados.0.smithi100.stdout:2228: finishing write tid 6 to smithi10066867-1 2024-04-07T16:24:00.562 INFO:tasks.rados.rados.0.smithi100.stdout:2228: oid 1 updating version 0 to 1064 2024-04-07T16:24:00.562 INFO:tasks.rados.rados.0.smithi100.stdout:2228: oid 1 updating version 1064 to 1065 2024-04-07T16:24:00.562 INFO:tasks.rados.rados.0.smithi100.stdout:2228: oid 1 version 1065 is already newer than 1063 2024-04-07T16:24:00.562 INFO:tasks.rados.rados.0.smithi100.stdout:2228: oid 1 version 1065 is already newer than 1062 2024-04-07T16:24:00.562 INFO:tasks.rados.rados.0.smithi100.stdout:2228: oid 1 updating version 1065 to 1066 2024-04-07T16:24:00.562 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 1 v 1066 (ObjNum 685 snap 210 seq_num 685) dirty exists 2024-04-07T16:24:00.562 INFO:tasks.rados.rados.0.smithi100.stdout:2228: left oid 1 (ObjNum 685 snap 210 seq_num 685) 2024-04-07T16:24:00.562 INFO:tasks.rados.rados.0.smithi100.stdout:2229: finishing write tid 1 to smithi10066867-49 2024-04-07T16:24:00.562 INFO:tasks.rados.rados.0.smithi100.stdout:2229: finishing write tid 2 to smithi10066867-49 2024-04-07T16:24:00.562 INFO:tasks.rados.rados.0.smithi100.stdout:2229: finishing write tid 3 to smithi10066867-49 2024-04-07T16:24:00.562 INFO:tasks.rados.rados.0.smithi100.stdout:2229: finishing write tid 4 to smithi10066867-49 2024-04-07T16:24:00.562 INFO:tasks.rados.rados.0.smithi100.stdout:2229: finishing write tid 5 to smithi10066867-49 2024-04-07T16:24:00.563 INFO:tasks.rados.rados.0.smithi100.stdout:2229: finishing write tid 6 to smithi10066867-49 2024-04-07T16:24:00.563 INFO:tasks.rados.rados.0.smithi100.stdout:2229: oid 49 updating version 0 to 1104 2024-04-07T16:24:00.563 INFO:tasks.rados.rados.0.smithi100.stdout:2229: oid 49 version 1104 is already newer than 1103 2024-04-07T16:24:00.563 INFO:tasks.rados.rados.0.smithi100.stdout:2229: oid 49 version 1104 is already newer than 1102 2024-04-07T16:24:00.563 INFO:tasks.rados.rados.0.smithi100.stdout:2229: oid 49 updating version 1104 to 1105 2024-04-07T16:24:00.563 INFO:tasks.rados.rados.0.smithi100.stdout:2229: oid 49 updating version 1105 to 1106 2024-04-07T16:24:00.563 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 49 v 1106 (ObjNum 686 snap 210 seq_num 686) dirty exists 2024-04-07T16:24:00.563 INFO:tasks.rados.rados.0.smithi100.stdout:2229: left oid 49 (ObjNum 686 snap 210 seq_num 686) 2024-04-07T16:24:00.563 INFO:tasks.rados.rados.0.smithi100.stdout:2223: done (3 left) 2024-04-07T16:24:00.563 INFO:tasks.rados.rados.0.smithi100.stdout:2228: done (2 left) 2024-04-07T16:24:00.563 INFO:tasks.rados.rados.0.smithi100.stdout:2229: done (1 left) 2024-04-07T16:24:00.563 INFO:tasks.rados.rados.0.smithi100.stdout:2231: delete oid 9 current snap is 210 2024-04-07T16:24:00.568 INFO:tasks.rados.rados.0.smithi100.stdout:2231: done (1 left) 2024-04-07T16:24:00.568 INFO:tasks.rados.rados.0.smithi100.stdout:2232: read oid 40 snap -1 2024-04-07T16:24:00.568 INFO:tasks.rados.rados.0.smithi100.stdout:2232: expect (ObjNum 509 snap 152 seq_num 509) 2024-04-07T16:24:00.568 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 35 current snap is 210 2024-04-07T16:24:00.568 INFO:tasks.rados.rados.0.smithi100.stdout:2233: seq_num 688 ranges {4055040=729088,4784128=499712,5283840=704512} 2024-04-07T16:24:00.603 INFO:tasks.rados.rados.0.smithi100.stdout:2233: writing smithi10066867-35 from 4055040 to 4784128 tid 1 2024-04-07T16:24:00.607 INFO:tasks.rados.rados.0.smithi100.stdout:2233: writing smithi10066867-35 from 4784128 to 5283840 tid 2 2024-04-07T16:24:00.613 INFO:tasks.rados.rados.0.smithi100.stdout:2233: writing smithi10066867-35 from 5283840 to 5988352 tid 3 2024-04-07T16:24:00.613 INFO:tasks.rados.rados.0.smithi100.stdout:2234: snap_remove snap 209 2024-04-07T16:24:00.846 INFO:tasks.rados.rados.0.smithi100.stdout:2234: done (3 left) 2024-04-07T16:24:00.846 INFO:tasks.rados.rados.0.smithi100.stdout:2235: snap_create 2024-04-07T16:24:00.846 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 2 v 1114 (ObjNum 659 snap 203 seq_num 659) dirty exists 2024-04-07T16:24:00.846 INFO:tasks.rados.rados.0.smithi100.stdout:2232: expect (ObjNum 509 snap 152 seq_num 509) 2024-04-07T16:24:01.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:00 smithi144 bash[66654]: cluster 2024-04-07T16:23:59.848667+0000 mon.a (mon.0) 625 : cluster [DBG] osdmap e487: 8 total, 8 up, 8 in 2024-04-07T16:24:01.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:00 smithi144 bash[66654]: cluster 2024-04-07T16:24:00.155397+0000 mon.a (mon.0) 626 : cluster [INF] osd.4 marked itself down and dead 2024-04-07T16:24:01.184 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:00 smithi100 bash[74942]: cluster 2024-04-07T16:23:59.848667+0000 mon.a (mon.0) 625 : cluster [DBG] osdmap e487: 8 total, 8 up, 8 in 2024-04-07T16:24:01.184 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:00 smithi100 bash[74942]: cluster 2024-04-07T16:24:00.155397+0000 mon.a (mon.0) 626 : cluster [INF] osd.4 marked itself down and dead 2024-04-07T16:24:01.184 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:00 smithi100 bash[76202]: cluster 2024-04-07T16:23:59.848667+0000 mon.a (mon.0) 625 : cluster [DBG] osdmap e487: 8 total, 8 up, 8 in 2024-04-07T16:24:01.184 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:00 smithi100 bash[76202]: cluster 2024-04-07T16:24:00.155397+0000 mon.a (mon.0) 626 : cluster [INF] osd.4 marked itself down and dead 2024-04-07T16:24:01.292 INFO:tasks.rados.rados.0.smithi100.stdout:2233: finishing write tid 1 to smithi10066867-35 2024-04-07T16:24:01.292 INFO:tasks.rados.rados.0.smithi100.stdout:2233: finishing write tid 2 to smithi10066867-35 2024-04-07T16:24:01.292 INFO:tasks.rados.rados.0.smithi100.stdout:2233: finishing write tid 3 to smithi10066867-35 2024-04-07T16:24:01.292 INFO:tasks.rados.rados.0.smithi100.stdout:2233: finishing write tid 4 to smithi10066867-35 2024-04-07T16:24:01.292 INFO:tasks.rados.rados.0.smithi100.stdout:2233: finishing write tid 5 to smithi10066867-35 2024-04-07T16:24:01.292 INFO:tasks.rados.rados.0.smithi100.stdout:2233: oid 35 updating version 0 to 1074 2024-04-07T16:24:01.292 INFO:tasks.rados.rados.0.smithi100.stdout:2233: oid 35 version 1074 is already newer than 1073 2024-04-07T16:24:01.293 INFO:tasks.rados.rados.0.smithi100.stdout:2233: oid 35 updating version 1074 to 1075 2024-04-07T16:24:01.293 INFO:tasks.rados.rados.0.smithi100.stdout:2233: oid 35 updating version 1075 to 1076 2024-04-07T16:24:01.293 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 35 v 1076 (ObjNum 688 snap 210 seq_num 688) dirty exists 2024-04-07T16:24:01.293 INFO:tasks.rados.rados.0.smithi100.stdout:2233: left oid 35 (ObjNum 688 snap 210 seq_num 688) 2024-04-07T16:24:01.820 INFO:tasks.rados.rados.0.smithi100.stdout:2230: done (3 left) 2024-04-07T16:24:01.820 INFO:tasks.rados.rados.0.smithi100.stdout:2232: done (2 left) 2024-04-07T16:24:01.821 INFO:tasks.rados.rados.0.smithi100.stdout:2233: done (1 left) 2024-04-07T16:24:01.821 INFO:tasks.rados.rados.0.smithi100.stdout:2235: done (0 left) 2024-04-07T16:24:01.821 INFO:tasks.rados.rados.0.smithi100.stdout:2236: snap_remove snap 183 2024-04-07T16:24:02.099 INFO:journalctl@ceph.osd.4.smithi144.stdout:Apr 07 16:24:01 smithi144 bash[70786]: ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4-osd-4 2024-04-07T16:24:02.099 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:01 smithi144 bash[66654]: cluster 2024-04-07T16:24:00.690174+0000 mgr.y (mgr.25175) 197 : cluster [DBG] pgmap v322: 121 pgs: 121 active+clean; 336 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 1.8 MiB/s rd, 0 B/s wr, 5 op/s 2024-04-07T16:24:02.099 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:01 smithi144 bash[66654]: cluster 2024-04-07T16:24:00.844826+0000 mon.a (mon.0) 627 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN) 2024-04-07T16:24:02.099 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:01 smithi144 bash[66654]: cluster 2024-04-07T16:24:00.864035+0000 mon.a (mon.0) 628 : cluster [DBG] osdmap e488: 8 total, 7 up, 8 in 2024-04-07T16:24:02.099 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:01 smithi144 bash[66654]: audit 2024-04-07T16:24:01.721015+0000 mon.a (mon.0) 629 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:24:02.099 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:01 smithi144 bash[66654]: audit 2024-04-07T16:24:01.723175+0000 mon.a (mon.0) 630 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:24:02.099 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:01 smithi144 bash[66654]: cluster 2024-04-07T16:24:01.834806+0000 mon.a (mon.0) 631 : cluster [DBG] osdmap e489: 8 total, 7 up, 8 in 2024-04-07T16:24:02.184 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:01 smithi100 bash[76202]: cluster 2024-04-07T16:24:00.690174+0000 mgr.y (mgr.25175) 197 : cluster [DBG] pgmap v322: 121 pgs: 121 active+clean; 336 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 1.8 MiB/s rd, 0 B/s wr, 5 op/s 2024-04-07T16:24:02.184 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:01 smithi100 bash[76202]: cluster 2024-04-07T16:24:00.844826+0000 mon.a (mon.0) 627 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN) 2024-04-07T16:24:02.184 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:01 smithi100 bash[76202]: cluster 2024-04-07T16:24:00.864035+0000 mon.a (mon.0) 628 : cluster [DBG] osdmap e488: 8 total, 7 up, 8 in 2024-04-07T16:24:02.184 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:01 smithi100 bash[76202]: audit 2024-04-07T16:24:01.721015+0000 mon.a (mon.0) 629 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:24:02.184 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:01 smithi100 bash[76202]: audit 2024-04-07T16:24:01.723175+0000 mon.a (mon.0) 630 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:24:02.184 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:01 smithi100 bash[76202]: cluster 2024-04-07T16:24:01.834806+0000 mon.a (mon.0) 631 : cluster [DBG] osdmap e489: 8 total, 7 up, 8 in 2024-04-07T16:24:02.184 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:01 smithi100 bash[74942]: cluster 2024-04-07T16:24:00.690174+0000 mgr.y (mgr.25175) 197 : cluster [DBG] pgmap v322: 121 pgs: 121 active+clean; 336 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 1.8 MiB/s rd, 0 B/s wr, 5 op/s 2024-04-07T16:24:02.184 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:01 smithi100 bash[74942]: cluster 2024-04-07T16:24:00.844826+0000 mon.a (mon.0) 627 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN) 2024-04-07T16:24:02.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:01 smithi100 bash[74942]: cluster 2024-04-07T16:24:00.864035+0000 mon.a (mon.0) 628 : cluster [DBG] osdmap e488: 8 total, 7 up, 8 in 2024-04-07T16:24:02.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:01 smithi100 bash[74942]: audit 2024-04-07T16:24:01.721015+0000 mon.a (mon.0) 629 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:24:02.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:01 smithi100 bash[74942]: audit 2024-04-07T16:24:01.723175+0000 mon.a (mon.0) 630 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:24:02.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:01 smithi100 bash[74942]: cluster 2024-04-07T16:24:01.834806+0000 mon.a (mon.0) 631 : cluster [DBG] osdmap e489: 8 total, 7 up, 8 in 2024-04-07T16:24:02.356 INFO:journalctl@ceph.osd.4.smithi144.stdout:Apr 07 16:24:02 smithi144 bash[70851]: Error response from daemon: No such container: ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4-osd-4 2024-04-07T16:24:02.356 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:24:02 smithi144 bash[59794]: ts=2024-04-07T16:24:02.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:24:02.797 INFO:teuthology.orchestra.run.smithi100.stdout:true 2024-04-07T16:24:02.813 INFO:tasks.rados.rados.0.smithi100.stdout:2236: done (0 left) 2024-04-07T16:24:02.814 INFO:tasks.rados.rados.0.smithi100.stdout:2237: delete oid 21 current snap is 211 2024-04-07T16:24:02.818 INFO:tasks.rados.rados.0.smithi100.stdout:2237: done (0 left) 2024-04-07T16:24:02.818 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 23 current snap is 211 2024-04-07T16:24:02.818 INFO:tasks.rados.rados.0.smithi100.stdout:2238: seq_num 690 ranges {0=761856,761856=663552,1425408=409600,1835008=131072} 2024-04-07T16:24:02.825 INFO:tasks.rados.rados.0.smithi100.stdout:2238: writing smithi10066867-23 from 0 to 761856 tid 1 2024-04-07T16:24:02.832 INFO:tasks.rados.rados.0.smithi100.stdout:2238: writing smithi10066867-23 from 761856 to 1425408 tid 2 2024-04-07T16:24:02.836 INFO:tasks.rados.rados.0.smithi100.stdout:2238: writing smithi10066867-23 from 1425408 to 1835008 tid 3 2024-04-07T16:24:02.838 INFO:tasks.rados.rados.0.smithi100.stdout:2238: writing smithi10066867-23 from 1835008 to 1966080 tid 4 2024-04-07T16:24:02.838 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 43 current snap is 211 2024-04-07T16:24:02.838 INFO:tasks.rados.rados.0.smithi100.stdout:2239: seq_num 691 ranges {6610944=778240,7389184=548864,7938048=614400,8552448=40960} 2024-04-07T16:24:02.892 INFO:tasks.rados.rados.0.smithi100.stdout:2239: writing smithi10066867-43 from 6610944 to 7389184 tid 1 2024-04-07T16:24:02.896 INFO:tasks.rados.rados.0.smithi100.stdout:2239: writing smithi10066867-43 from 7389184 to 7938048 tid 2 2024-04-07T16:24:02.900 INFO:tasks.rados.rados.0.smithi100.stdout:2239: writing smithi10066867-43 from 7938048 to 8552448 tid 3 2024-04-07T16:24:02.900 INFO:tasks.rados.rados.0.smithi100.stdout:2239: writing smithi10066867-43 from 8552448 to 8593408 tid 4 2024-04-07T16:24:02.900 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 48 current snap is 211 2024-04-07T16:24:02.900 INFO:tasks.rados.rados.0.smithi100.stdout:2238: finishing write tid 1 to smithi10066867-23 2024-04-07T16:24:02.901 INFO:tasks.rados.rados.0.smithi100.stdout:2238: finishing write tid 2 to smithi10066867-23 2024-04-07T16:24:02.901 INFO:tasks.rados.rados.0.smithi100.stdout:2238: finishing write tid 3 to smithi10066867-23 2024-04-07T16:24:02.901 INFO:tasks.rados.rados.0.smithi100.stdout:2240: seq_num 692 ranges {3817472=794624,4612096=483328,5095424=720896} 2024-04-07T16:24:02.926 INFO:tasks.rados.rados.0.smithi100.stdout:2240: writing smithi10066867-48 from 3817472 to 4612096 tid 1 2024-04-07T16:24:02.929 INFO:tasks.rados.rados.0.smithi100.stdout:2240: writing smithi10066867-48 from 4612096 to 5095424 tid 2 2024-04-07T16:24:02.934 INFO:tasks.rados.rados.0.smithi100.stdout:2240: writing smithi10066867-48 from 5095424 to 5816320 tid 3 2024-04-07T16:24:02.934 INFO:tasks.rados.rados.0.smithi100.stdout:2238: finishing write tid 4 to smithi10066867-23 2024-04-07T16:24:02.935 INFO:tasks.rados.rados.0.smithi100.stdout:2238: finishing write tid 5 to smithi10066867-23 2024-04-07T16:24:02.935 INFO:tasks.rados.rados.0.smithi100.stdout:2238: finishing write tid 6 to smithi10066867-23 2024-04-07T16:24:02.935 INFO:tasks.rados.rados.0.smithi100.stdout:2238: oid 23 updating version 0 to 1271 2024-04-07T16:24:02.935 INFO:tasks.rados.rados.0.smithi100.stdout:2238: oid 23 version 1271 is already newer than 1270 2024-04-07T16:24:02.935 INFO:tasks.rados.rados.0.smithi100.stdout:2238: oid 23 updating version 1271 to 1273 2024-04-07T16:24:02.935 INFO:tasks.rados.rados.0.smithi100.stdout:2238: oid 23 version 1273 is already newer than 1272 2024-04-07T16:24:02.935 INFO:tasks.rados.rados.0.smithi100.stdout:2238: oid 23 version 1273 is already newer than 1269 2024-04-07T16:24:02.935 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 23 v 1273 (ObjNum 690 snap 211 seq_num 690) dirty exists 2024-04-07T16:24:02.935 INFO:tasks.rados.rados.0.smithi100.stdout:2238: left oid 23 (ObjNum 690 snap 211 seq_num 690) 2024-04-07T16:24:02.935 INFO:tasks.rados.rados.0.smithi100.stdout:2239: finishing write tid 1 to smithi10066867-43 2024-04-07T16:24:02.935 INFO:tasks.rados.rados.0.smithi100.stdout:2241: read oid 10 snap -1 2024-04-07T16:24:02.935 INFO:tasks.rados.rados.0.smithi100.stdout:2241: expect (ObjNum 660 snap 203 seq_num 660) 2024-04-07T16:24:02.935 INFO:tasks.rados.rados.0.smithi100.stdout:2238: done (3 left) 2024-04-07T16:24:02.935 INFO:tasks.rados.rados.0.smithi100.stdout:2242: snap_remove snap 188 2024-04-07T16:24:03.187 INFO:teuthology.orchestra.run.smithi100.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-07T16:24:03.187 INFO:teuthology.orchestra.run.smithi100.stdout:alertmanager.a smithi100 *:9093,9094 running (10m) 21s ago 10m 14.4M - 0.25.0 c8568f914cd2 5143075e7d26 2024-04-07T16:24:03.187 INFO:teuthology.orchestra.run.smithi100.stdout:grafana.a smithi144 *:3000 running (10m) 2m ago 10m 79.3M - 9.4.7 954c08fa6188 7180c0708acd 2024-04-07T16:24:03.187 INFO:teuthology.orchestra.run.smithi100.stdout:mds.a smithi100 running (11m) 21s ago 11m 15.7M - 18.2.2-1171-g267c8ff5 8a4a01e7186c 92593b0c28ae 2024-04-07T16:24:03.187 INFO:teuthology.orchestra.run.smithi100.stdout:mds.b smithi144 running (11m) 2m ago 11m 18.3M - 18.2.2-1171-g267c8ff5 8a4a01e7186c 3df2ee3e17a4 2024-04-07T16:24:03.188 INFO:teuthology.orchestra.run.smithi100.stdout:mgr.x smithi144 *:8443,9283,8765 running (4m) 2m ago 19m 438M - 19.0.0-2785-gadce92d0 c0ecd4226ac9 233745c292a6 2024-04-07T16:24:03.188 INFO:teuthology.orchestra.run.smithi100.stdout:mgr.y smithi100 *:8443,9283,8765 running (5m) 21s ago 21m 503M - 19.0.0-2785-gadce92d0 c0ecd4226ac9 254dc65cdaa7 2024-04-07T16:24:03.188 INFO:teuthology.orchestra.run.smithi100.stdout:mon.a smithi100 running (3m) 21s ago 21m 54.3M 2048M 19.0.0-2785-gadce92d0 c0ecd4226ac9 718e12655c04 2024-04-07T16:24:03.188 INFO:teuthology.orchestra.run.smithi100.stdout:mon.b smithi144 running (2m) 2m ago 19m 18.2M 2048M 19.0.0-2785-gadce92d0 c0ecd4226ac9 aef817664bff 2024-04-07T16:24:03.188 INFO:teuthology.orchestra.run.smithi100.stdout:mon.c smithi100 running (3m) 21s ago 19m 46.2M 2048M 19.0.0-2785-gadce92d0 c0ecd4226ac9 63e889040f6e 2024-04-07T16:24:03.188 INFO:teuthology.orchestra.run.smithi100.stdout:node-exporter.a smithi100 *:9100 running (10m) 21s ago 11m 8955k - 1.5.0 0da6a335fe13 4bce264c1b26 2024-04-07T16:24:03.188 INFO:teuthology.orchestra.run.smithi100.stdout:node-exporter.b smithi144 *:9100 running (10m) 2m ago 11m 8443k - 1.5.0 0da6a335fe13 b6963413c560 2024-04-07T16:24:03.188 INFO:teuthology.orchestra.run.smithi100.stdout:osd.0 smithi100 running (110s) 21s ago 18m 199M 2524M 19.0.0-2785-gadce92d0 c0ecd4226ac9 a6e83c9d1abc 2024-04-07T16:24:03.188 INFO:teuthology.orchestra.run.smithi100.stdout:osd.1 smithi100 running (83s) 21s ago 17m 197M 2524M 19.0.0-2785-gadce92d0 c0ecd4226ac9 c50fe40079b6 2024-04-07T16:24:03.188 INFO:teuthology.orchestra.run.smithi100.stdout:osd.2 smithi100 running (54s) 21s ago 16m 253M 2524M 19.0.0-2785-gadce92d0 c0ecd4226ac9 8e631c43064b 2024-04-07T16:24:03.188 INFO:teuthology.orchestra.run.smithi100.stdout:osd.3 smithi100 running (26s) 21s ago 15m 11.8M 2524M 19.0.0-2785-gadce92d0 c0ecd4226ac9 402931247c79 2024-04-07T16:24:03.188 INFO:teuthology.orchestra.run.smithi100.stdout:osd.4 smithi144 running (15m) 2m ago 15m 257M 2524M 18.2.2-1171-g267c8ff5 8a4a01e7186c 8c73b6b4dc13 2024-04-07T16:24:03.188 INFO:teuthology.orchestra.run.smithi100.stdout:osd.5 smithi144 running (14m) 2m ago 14m 216M 2524M 18.2.2-1171-g267c8ff5 8a4a01e7186c 3bfa0cf6f6cb 2024-04-07T16:24:03.188 INFO:teuthology.orchestra.run.smithi100.stdout:osd.6 smithi144 running (13m) 2m ago 13m 98.7M 2524M 18.2.2-1171-g267c8ff5 8a4a01e7186c 32b7f95d77f0 2024-04-07T16:24:03.189 INFO:teuthology.orchestra.run.smithi100.stdout:osd.7 smithi144 running (12m) 2m ago 12m 156M 2524M 18.2.2-1171-g267c8ff5 8a4a01e7186c 45f7709e576a 2024-04-07T16:24:03.189 INFO:teuthology.orchestra.run.smithi100.stdout:prometheus.a smithi144 *:9095 running (4m) 2m ago 11m 46.5M - 2.43.0 a07b618ecd1d 2d0a8131f303 2024-04-07T16:24:03.629 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-04-07T16:24:03.629 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": { 2024-04-07T16:24:03.629 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph version 19.0.0-2785-gadce92d0 (adce92d056533f22dc1ee37e3141d098b046ec1b) squid (dev)": 3 2024-04-07T16:24:03.629 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-07T16:24:03.629 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": { 2024-04-07T16:24:03.629 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph version 19.0.0-2785-gadce92d0 (adce92d056533f22dc1ee37e3141d098b046ec1b) squid (dev)": 2 2024-04-07T16:24:03.629 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-07T16:24:03.629 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": { 2024-04-07T16:24:03.629 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph version 18.2.2-1171-g267c8ff5 (267c8ff5b1ff5815581b3b2381d19360e081b37b) reef (stable)": 3, 2024-04-07T16:24:03.629 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph version 19.0.0-2785-gadce92d0 (adce92d056533f22dc1ee37e3141d098b046ec1b) squid (dev)": 4 2024-04-07T16:24:03.629 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-07T16:24:03.629 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": { 2024-04-07T16:24:03.630 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph version 18.2.2-1171-g267c8ff5 (267c8ff5b1ff5815581b3b2381d19360e081b37b) reef (stable)": 2 2024-04-07T16:24:03.630 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-07T16:24:03.630 INFO:teuthology.orchestra.run.smithi100.stdout: "overall": { 2024-04-07T16:24:03.630 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph version 18.2.2-1171-g267c8ff5 (267c8ff5b1ff5815581b3b2381d19360e081b37b) reef (stable)": 5, 2024-04-07T16:24:03.630 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph version 19.0.0-2785-gadce92d0 (adce92d056533f22dc1ee37e3141d098b046ec1b) squid (dev)": 9 2024-04-07T16:24:03.630 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-04-07T16:24:03.630 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-04-07T16:24:03.853 INFO:tasks.rados.rados.0.smithi100.stdout:2242: done (3 left) 2024-04-07T16:24:03.853 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 41 current snap is 211 2024-04-07T16:24:03.853 INFO:tasks.rados.rados.0.smithi100.stdout:2239: finishing write tid 2 to smithi10066867-43 2024-04-07T16:24:03.853 INFO:tasks.rados.rados.0.smithi100.stdout:2243: seq_num 693 ranges {0=409600,409600=770048,1179648=581632,1761280=253952} 2024-04-07T16:24:03.855 INFO:tasks.rados.rados.0.smithi100.stdout:2243: writing smithi10066867-41 from 0 to 409600 tid 1 2024-04-07T16:24:03.860 INFO:tasks.rados.rados.0.smithi100.stdout:2243: writing smithi10066867-41 from 409600 to 1179648 tid 2 2024-04-07T16:24:03.864 INFO:tasks.rados.rados.0.smithi100.stdout:2243: writing smithi10066867-41 from 1179648 to 1761280 tid 3 2024-04-07T16:24:03.866 INFO:tasks.rados.rados.0.smithi100.stdout:2243: writing smithi10066867-41 from 1761280 to 2015232 tid 4 2024-04-07T16:24:03.866 INFO:tasks.rados.rados.0.smithi100.stdout:2244: read oid 8 snap 178 2024-04-07T16:24:03.866 INFO:tasks.rados.rados.0.smithi100.stdout:2244: expect (ObjNum 558 snap 170 seq_num 558) 2024-04-07T16:24:03.867 INFO:tasks.rados.rados.0.smithi100.stdout:2245: delete oid 44 current snap is 211 2024-04-07T16:24:03.867 INFO:tasks.rados.rados.0.smithi100.stdout:2239: finishing write tid 3 to smithi10066867-43 2024-04-07T16:24:03.867 INFO:tasks.rados.rados.0.smithi100.stdout:2239: finishing write tid 4 to smithi10066867-43 2024-04-07T16:24:03.867 INFO:tasks.rados.rados.0.smithi100.stdout:2239: finishing write tid 5 to smithi10066867-43 2024-04-07T16:24:03.867 INFO:tasks.rados.rados.0.smithi100.stdout:2239: finishing write tid 6 to smithi10066867-43 2024-04-07T16:24:03.867 INFO:tasks.rados.rados.0.smithi100.stdout:2239: oid 43 updating version 0 to 1092 2024-04-07T16:24:03.867 INFO:tasks.rados.rados.0.smithi100.stdout:2239: oid 43 version 1092 is already newer than 1091 2024-04-07T16:24:03.867 INFO:tasks.rados.rados.0.smithi100.stdout:2239: oid 43 updating version 1092 to 1094 2024-04-07T16:24:03.867 INFO:tasks.rados.rados.0.smithi100.stdout:2239: oid 43 version 1094 is already newer than 1093 2024-04-07T16:24:03.867 INFO:tasks.rados.rados.0.smithi100.stdout:2239: oid 43 version 1094 is already newer than 1090 2024-04-07T16:24:03.867 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 43 v 1094 (ObjNum 691 snap 211 seq_num 691) dirty exists 2024-04-07T16:24:03.867 INFO:tasks.rados.rados.0.smithi100.stdout:2239: left oid 43 (ObjNum 691 snap 211 seq_num 691) 2024-04-07T16:24:03.867 INFO:tasks.rados.rados.0.smithi100.stdout:2240: finishing write tid 1 to smithi10066867-48 2024-04-07T16:24:03.867 INFO:tasks.rados.rados.0.smithi100.stdout:2240: finishing write tid 2 to smithi10066867-48 2024-04-07T16:24:03.867 INFO:tasks.rados.rados.0.smithi100.stdout:2240: finishing write tid 3 to smithi10066867-48 2024-04-07T16:24:03.868 INFO:tasks.rados.rados.0.smithi100.stdout:2240: finishing write tid 4 to smithi10066867-48 2024-04-07T16:24:03.868 INFO:tasks.rados.rados.0.smithi100.stdout:2240: finishing write tid 5 to smithi10066867-48 2024-04-07T16:24:03.868 INFO:tasks.rados.rados.0.smithi100.stdout:2240: oid 48 updating version 0 to 1077 2024-04-07T16:24:03.868 INFO:tasks.rados.rados.0.smithi100.stdout:2240: oid 48 updating version 1077 to 1079 2024-04-07T16:24:03.868 INFO:tasks.rados.rados.0.smithi100.stdout:2240: oid 48 version 1079 is already newer than 1078 2024-04-07T16:24:03.868 INFO:tasks.rados.rados.0.smithi100.stdout:2240: oid 48 updating version 1079 to 1080 2024-04-07T16:24:03.868 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 48 v 1080 (ObjNum 692 snap 211 seq_num 692) dirty exists 2024-04-07T16:24:03.868 INFO:tasks.rados.rados.0.smithi100.stdout:2240: left oid 48 (ObjNum 692 snap 211 seq_num 692) 2024-04-07T16:24:03.868 INFO:tasks.rados.rados.0.smithi100.stdout:2241: expect (ObjNum 660 snap 203 seq_num 660) 2024-04-07T16:24:04.070 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:03 smithi144 bash[66654]: cluster 2024-04-07T16:24:02.691034+0000 mgr.y (mgr.25175) 198 : cluster [DBG] pgmap v325: 121 pgs: 5 active+undersized+degraded, 4 stale+active+clean, 24 active+undersized, 88 active+clean; 343 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 8.9 MiB/s rd, 1.8 MiB/s wr, 11 op/s; 25/537 objects degraded (4.655%) 2024-04-07T16:24:04.070 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:03 smithi144 bash[66654]: audit 2024-04-07T16:24:02.801729+0000 mgr.y (mgr.25175) 199 : audit [DBG] from='client.44234 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:24:04.070 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:03 smithi144 bash[66654]: cluster 2024-04-07T16:24:02.832624+0000 mon.a (mon.0) 632 : cluster [DBG] osdmap e490: 8 total, 7 up, 8 in 2024-04-07T16:24:04.071 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:03 smithi144 bash[66654]: cluster 2024-04-07T16:24:02.857590+0000 mon.a (mon.0) 633 : cluster [WRN] Health check failed: Degraded data redundancy: 25/537 objects degraded (4.655%), 5 pgs degraded (PG_DEGRADED) 2024-04-07T16:24:04.071 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:03 smithi144 bash[66654]: audit 2024-04-07T16:24:03.186614+0000 mgr.y (mgr.25175) 200 : audit [DBG] from='client.44240 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:24:04.071 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:03 smithi144 bash[66654]: audit 2024-04-07T16:24:03.647348+0000 mon.a (mon.0) 634 : audit [DBG] from='client.? 172.21.15.100:0/2003896953' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:24:04.184 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:03 smithi100 bash[74942]: cluster 2024-04-07T16:24:02.691034+0000 mgr.y (mgr.25175) 198 : cluster [DBG] pgmap v325: 121 pgs: 5 active+undersized+degraded, 4 stale+active+clean, 24 active+undersized, 88 active+clean; 343 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 8.9 MiB/s rd, 1.8 MiB/s wr, 11 op/s; 25/537 objects degraded (4.655%) 2024-04-07T16:24:04.184 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:03 smithi100 bash[74942]: audit 2024-04-07T16:24:02.801729+0000 mgr.y (mgr.25175) 199 : audit [DBG] from='client.44234 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:24:04.184 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:03 smithi100 bash[74942]: cluster 2024-04-07T16:24:02.832624+0000 mon.a (mon.0) 632 : cluster [DBG] osdmap e490: 8 total, 7 up, 8 in 2024-04-07T16:24:04.184 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:03 smithi100 bash[74942]: cluster 2024-04-07T16:24:02.857590+0000 mon.a (mon.0) 633 : cluster [WRN] Health check failed: Degraded data redundancy: 25/537 objects degraded (4.655%), 5 pgs degraded (PG_DEGRADED) 2024-04-07T16:24:04.184 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:03 smithi100 bash[74942]: audit 2024-04-07T16:24:03.186614+0000 mgr.y (mgr.25175) 200 : audit [DBG] from='client.44240 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:24:04.184 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:03 smithi100 bash[74942]: audit 2024-04-07T16:24:03.647348+0000 mon.a (mon.0) 634 : audit [DBG] from='client.? 172.21.15.100:0/2003896953' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:24:04.184 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:03 smithi100 bash[76202]: cluster 2024-04-07T16:24:02.691034+0000 mgr.y (mgr.25175) 198 : cluster [DBG] pgmap v325: 121 pgs: 5 active+undersized+degraded, 4 stale+active+clean, 24 active+undersized, 88 active+clean; 343 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 8.9 MiB/s rd, 1.8 MiB/s wr, 11 op/s; 25/537 objects degraded (4.655%) 2024-04-07T16:24:04.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:03 smithi100 bash[76202]: audit 2024-04-07T16:24:02.801729+0000 mgr.y (mgr.25175) 199 : audit [DBG] from='client.44234 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:24:04.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:03 smithi100 bash[76202]: cluster 2024-04-07T16:24:02.832624+0000 mon.a (mon.0) 632 : cluster [DBG] osdmap e490: 8 total, 7 up, 8 in 2024-04-07T16:24:04.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:03 smithi100 bash[76202]: cluster 2024-04-07T16:24:02.857590+0000 mon.a (mon.0) 633 : cluster [WRN] Health check failed: Degraded data redundancy: 25/537 objects degraded (4.655%), 5 pgs degraded (PG_DEGRADED) 2024-04-07T16:24:04.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:03 smithi100 bash[76202]: audit 2024-04-07T16:24:03.186614+0000 mgr.y (mgr.25175) 200 : audit [DBG] from='client.44240 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:24:04.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:03 smithi100 bash[76202]: audit 2024-04-07T16:24:03.647348+0000 mon.a (mon.0) 634 : audit [DBG] from='client.? 172.21.15.100:0/2003896953' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:24:04.256 INFO:tasks.rados.rados.0.smithi100.stdout:2243: finishing write tid 1 to smithi10066867-41 2024-04-07T16:24:04.256 INFO:tasks.rados.rados.0.smithi100.stdout:2243: finishing write tid 2 to smithi10066867-41 2024-04-07T16:24:04.256 INFO:tasks.rados.rados.0.smithi100.stdout:2243: finishing write tid 3 to smithi10066867-41 2024-04-07T16:24:04.256 INFO:tasks.rados.rados.0.smithi100.stdout:2243: finishing write tid 4 to smithi10066867-41 2024-04-07T16:24:04.256 INFO:tasks.rados.rados.0.smithi100.stdout:2243: finishing write tid 5 to smithi10066867-41 2024-04-07T16:24:04.256 INFO:tasks.rados.rados.0.smithi100.stdout:2243: finishing write tid 6 to smithi10066867-41 2024-04-07T16:24:04.256 INFO:tasks.rados.rados.0.smithi100.stdout:2243: oid 41 updating version 0 to 1109 2024-04-07T16:24:04.256 INFO:tasks.rados.rados.0.smithi100.stdout:2243: oid 41 version 1109 is already newer than 1108 2024-04-07T16:24:04.256 INFO:tasks.rados.rados.0.smithi100.stdout:2243: oid 41 updating version 1109 to 1110 2024-04-07T16:24:04.257 INFO:tasks.rados.rados.0.smithi100.stdout:2243: oid 41 updating version 1110 to 1111 2024-04-07T16:24:04.257 INFO:tasks.rados.rados.0.smithi100.stdout:2243: oid 41 version 1111 is already newer than 1107 2024-04-07T16:24:04.257 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 41 v 1111 (ObjNum 693 snap 211 seq_num 693) dirty exists 2024-04-07T16:24:04.257 INFO:tasks.rados.rados.0.smithi100.stdout:2243: left oid 41 (ObjNum 693 snap 211 seq_num 693) 2024-04-07T16:24:04.257 INFO:tasks.rados.rados.0.smithi100.stdout:2244: expect (ObjNum 558 snap 170 seq_num 558) 2024-04-07T16:24:04.356 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:24:04 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:24:04.356 INFO:journalctl@ceph.osd.6.smithi144.stdout:Apr 07 16:24:04 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:24:04.357 INFO:journalctl@ceph.osd.7.smithi144.stdout:Apr 07 16:24:04 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:24:04.357 INFO:journalctl@ceph.mds.b.smithi144.stdout:Apr 07 16:24:04 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:24:04.357 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:24:04 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:24:04.358 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 16:24:04 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:24:04.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:04 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:24:04.358 INFO:journalctl@ceph.osd.4.smithi144.stdout:Apr 07 16:24:04 smithi144 systemd[1]: ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@osd.4.service: Deactivated successfully. 2024-04-07T16:24:04.358 INFO:journalctl@ceph.osd.4.smithi144.stdout:Apr 07 16:24:04 smithi144 systemd[1]: Stopped Ceph osd.4 for 0702e3a8-f4f8-11ee-b649-cb9ed24678a4. 2024-04-07T16:24:04.358 INFO:journalctl@ceph.osd.4.smithi144.stdout:Apr 07 16:24:04 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:24:04.359 INFO:journalctl@ceph.osd.5.smithi144.stdout:Apr 07 16:24:04 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:24:04.359 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:24:04 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:24:04.849 INFO:journalctl@ceph.osd.4.smithi144.stdout:Apr 07 16:24:04 smithi144 systemd[1]: Started Ceph osd.4 for 0702e3a8-f4f8-11ee-b649-cb9ed24678a4. 2024-04-07T16:24:04.858 INFO:tasks.rados.rados.0.smithi100.stdout:2239: done (5 left) 2024-04-07T16:24:04.858 INFO:tasks.rados.rados.0.smithi100.stdout:2240: done (4 left) 2024-04-07T16:24:04.858 INFO:tasks.rados.rados.0.smithi100.stdout:2241: done (3 left) 2024-04-07T16:24:04.858 INFO:tasks.rados.rados.0.smithi100.stdout:2243: done (2 left) 2024-04-07T16:24:04.858 INFO:tasks.rados.rados.0.smithi100.stdout:2244: done (1 left) 2024-04-07T16:24:04.858 INFO:tasks.rados.rados.0.smithi100.stdout:2245: done (0 left) 2024-04-07T16:24:04.858 INFO:tasks.rados.rados.0.smithi100.stdout:2246: rmattr oid 5 current snap is 211 2024-04-07T16:24:04.858 INFO:tasks.rados.rados.0.smithi100.stdout:2246: done (0 left) 2024-04-07T16:24:04.858 INFO:tasks.rados.rados.0.smithi100.stdout:2247: read oid 16 snap 206 2024-04-07T16:24:04.859 INFO:tasks.rados.rados.0.smithi100.stdout:2247: expect (ObjNum 651 snap 203 seq_num 651) 2024-04-07T16:24:04.859 INFO:tasks.rados.rados.0.smithi100.stdout:2248: rollback oid 33 current snap is 211 2024-04-07T16:24:04.859 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 33 to 205 2024-04-07T16:24:04.859 INFO:tasks.rados.rados.0.smithi100.stdout:2249: setattr oid 13 current snap is 211 2024-04-07T16:24:04.862 INFO:tasks.rados.rados.0.smithi100.stdout:2250: read oid 44 snap -1 2024-04-07T16:24:04.862 INFO:tasks.rados.rados.0.smithi100.stdout:2250: expect deleted 2024-04-07T16:24:04.862 INFO:tasks.rados.rados.0.smithi100.stdout:2251: snap_remove snap 204 2024-04-07T16:24:05.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:04 smithi144 bash[66654]: cluster 2024-04-07T16:24:03.871284+0000 mon.a (mon.0) 635 : cluster [DBG] osdmap e491: 8 total, 7 up, 8 in 2024-04-07T16:24:05.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:04 smithi144 bash[66654]: audit 2024-04-07T16:24:04.450791+0000 mon.a (mon.0) 636 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:24:05.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:04 smithi144 bash[66654]: audit 2024-04-07T16:24:04.461992+0000 mon.a (mon.0) 637 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:24:05.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:04 smithi144 bash[66654]: audit 2024-04-07T16:24:04.502363+0000 mon.a (mon.0) 638 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:24:05.184 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:04 smithi100 bash[76202]: cluster 2024-04-07T16:24:03.871284+0000 mon.a (mon.0) 635 : cluster [DBG] osdmap e491: 8 total, 7 up, 8 in 2024-04-07T16:24:05.184 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:04 smithi100 bash[76202]: audit 2024-04-07T16:24:04.450791+0000 mon.a (mon.0) 636 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:24:05.184 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:04 smithi100 bash[76202]: audit 2024-04-07T16:24:04.461992+0000 mon.a (mon.0) 637 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:24:05.184 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:04 smithi100 bash[76202]: audit 2024-04-07T16:24:04.502363+0000 mon.a (mon.0) 638 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:24:05.184 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:04 smithi100 bash[74942]: cluster 2024-04-07T16:24:03.871284+0000 mon.a (mon.0) 635 : cluster [DBG] osdmap e491: 8 total, 7 up, 8 in 2024-04-07T16:24:05.184 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:04 smithi100 bash[74942]: audit 2024-04-07T16:24:04.450791+0000 mon.a (mon.0) 636 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:24:05.184 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:04 smithi100 bash[74942]: audit 2024-04-07T16:24:04.461992+0000 mon.a (mon.0) 637 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:24:05.184 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:04 smithi100 bash[74942]: audit 2024-04-07T16:24:04.502363+0000 mon.a (mon.0) 638 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:24:05.864 INFO:tasks.rados.rados.0.smithi100.stdout:2251: done (4 left) 2024-04-07T16:24:05.864 INFO:tasks.rados.rados.0.smithi100.stdout:2252: copy_from oid 41 from oid 47 current snap is 211 2024-04-07T16:24:05.864 INFO:tasks.rados.rados.0.smithi100.stdout:2248: finishing rollback tid 0 to smithi10066867-33 2024-04-07T16:24:05.864 INFO:tasks.rados.rados.0.smithi100.stdout:2248: finishing rollback tid 1 to smithi10066867-33 2024-04-07T16:24:05.864 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 33 v 1276 (ObjNum 225 snap 0 seq_num 2704677696) dirty dne 2024-04-07T16:24:05.864 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 13 v 949 (ObjNum 289 snap 0 seq_num 1) dirty exists 2024-04-07T16:24:05.864 INFO:tasks.rados.rados.0.smithi100.stdout:2247: expect (ObjNum 651 snap 203 seq_num 651) 2024-04-07T16:24:06.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:05 smithi144 bash[66654]: cluster 2024-04-07T16:24:04.691855+0000 mgr.y (mgr.25175) 201 : cluster [DBG] pgmap v328: 121 pgs: 10 active+undersized+degraded, 1 stale+active+clean, 30 active+undersized, 80 active+clean; 349 MiB data, 1.7 GiB used, 714 GiB / 715 GiB avail; 11 MiB/s rd, 3.6 MiB/s wr, 18 op/s; 57/543 objects degraded (10.497%) 2024-04-07T16:24:06.106 INFO:journalctl@ceph.osd.4.smithi144.stdout:Apr 07 16:24:05 smithi144 bash[71004]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-07T16:24:06.106 INFO:journalctl@ceph.osd.4.smithi144.stdout:Apr 07 16:24:05 smithi144 bash[71004]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-4 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-04-07T16:24:06.106 INFO:journalctl@ceph.osd.4.smithi144.stdout:Apr 07 16:24:05 smithi144 bash[71004]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-04-07T16:24:06.107 INFO:journalctl@ceph.osd.4.smithi144.stdout:Apr 07 16:24:05 smithi144 bash[71004]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-04-07T16:24:06.107 INFO:journalctl@ceph.osd.4.smithi144.stdout:Apr 07 16:24:05 smithi144 bash[71004]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-4/block 2024-04-07T16:24:06.107 INFO:journalctl@ceph.osd.4.smithi144.stdout:Apr 07 16:24:05 smithi144 bash[71004]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-07T16:24:06.107 INFO:journalctl@ceph.osd.4.smithi144.stdout:Apr 07 16:24:05 smithi144 bash[71004]: --> ceph-volume raw activate successful for osd ID: 4 2024-04-07T16:24:06.183 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:05 smithi100 bash[74942]: cluster 2024-04-07T16:24:04.691855+0000 mgr.y (mgr.25175) 201 : cluster [DBG] pgmap v328: 121 pgs: 10 active+undersized+degraded, 1 stale+active+clean, 30 active+undersized, 80 active+clean; 349 MiB data, 1.7 GiB used, 714 GiB / 715 GiB avail; 11 MiB/s rd, 3.6 MiB/s wr, 18 op/s; 57/543 objects degraded (10.497%) 2024-04-07T16:24:06.184 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:05 smithi100 bash[76202]: cluster 2024-04-07T16:24:04.691855+0000 mgr.y (mgr.25175) 201 : cluster [DBG] pgmap v328: 121 pgs: 10 active+undersized+degraded, 1 stale+active+clean, 30 active+undersized, 80 active+clean; 349 MiB data, 1.7 GiB used, 714 GiB / 715 GiB avail; 11 MiB/s rd, 3.6 MiB/s wr, 18 op/s; 57/543 objects degraded (10.497%) 2024-04-07T16:24:06.212 INFO:tasks.rados.rados.0.smithi100.stdout:2252: finishing copy_from racing read to smithi10066867-41 2024-04-07T16:24:06.213 INFO:tasks.rados.rados.0.smithi100.stdout:2252: finishing copy_from to smithi10066867-41 2024-04-07T16:24:06.213 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 41 v 823 (ObjNum 684 snap 210 seq_num 684) dirty exists 2024-04-07T16:24:06.213 INFO:tasks.rados.rados.0.smithi100.stdout:2247: done (4 left) 2024-04-07T16:24:06.213 INFO:tasks.rados.rados.0.smithi100.stdout:2248: done (3 left) 2024-04-07T16:24:06.213 INFO:tasks.rados.rados.0.smithi100.stdout:2249: done (2 left) 2024-04-07T16:24:06.213 INFO:tasks.rados.rados.0.smithi100.stdout:2250: done (1 left) 2024-04-07T16:24:06.213 INFO:tasks.rados.rados.0.smithi100.stdout:2252: done (0 left) 2024-04-07T16:24:06.213 INFO:tasks.rados.rados.0.smithi100.stdout:2253: delete oid 4 current snap is 211 2024-04-07T16:24:06.216 INFO:tasks.rados.rados.0.smithi100.stdout:2253: done (0 left) 2024-04-07T16:24:06.216 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 23 current snap is 211 2024-04-07T16:24:06.216 INFO:tasks.rados.rados.0.smithi100.stdout:2254: seq_num 696 ranges {1966080=458752,2424832=417792,2842624=753664,3596288=434176} 2024-04-07T16:24:06.256 INFO:tasks.rados.rados.0.smithi100.stdout:2254: writing smithi10066867-23 from 1966080 to 2424832 tid 1 2024-04-07T16:24:06.263 INFO:tasks.rados.rados.0.smithi100.stdout:2254: writing smithi10066867-23 from 2424832 to 2842624 tid 2 2024-04-07T16:24:06.274 INFO:tasks.rados.rados.0.smithi100.stdout:2254: writing smithi10066867-23 from 2842624 to 3596288 tid 3 2024-04-07T16:24:06.279 INFO:tasks.rados.rados.0.smithi100.stdout:2254: writing smithi10066867-23 from 3596288 to 4030464 tid 4 2024-04-07T16:24:06.279 INFO:tasks.rados.rados.0.smithi100.stdout:2255: snap_remove snap 205 2024-04-07T16:24:06.279 INFO:tasks.rados.rados.0.smithi100.stdout:2254: finishing write tid 1 to smithi10066867-23 2024-04-07T16:24:06.279 INFO:tasks.rados.rados.0.smithi100.stdout:2254: finishing write tid 2 to smithi10066867-23 2024-04-07T16:24:06.279 INFO:tasks.rados.rados.0.smithi100.stdout:2254: finishing write tid 3 to smithi10066867-23 2024-04-07T16:24:06.824 INFO:tasks.rados.rados.0.smithi100.stdout:2255: done (1 left) 2024-04-07T16:24:06.824 INFO:tasks.rados.rados.0.smithi100.stdout:2256: delete oid 30 current snap is 211 2024-04-07T16:24:06.824 INFO:tasks.rados.rados.0.smithi100.stdout:2254: finishing write tid 4 to smithi10066867-23 2024-04-07T16:24:06.824 INFO:tasks.rados.rados.0.smithi100.stdout:2254: finishing write tid 5 to smithi10066867-23 2024-04-07T16:24:06.824 INFO:tasks.rados.rados.0.smithi100.stdout:2254: finishing write tid 6 to smithi10066867-23 2024-04-07T16:24:06.824 INFO:tasks.rados.rados.0.smithi100.stdout:2254: oid 23 updating version 0 to 1282 2024-04-07T16:24:06.824 INFO:tasks.rados.rados.0.smithi100.stdout:2254: oid 23 version 1282 is already newer than 1280 2024-04-07T16:24:06.824 INFO:tasks.rados.rados.0.smithi100.stdout:2254: oid 23 version 1282 is already newer than 1279 2024-04-07T16:24:06.824 INFO:tasks.rados.rados.0.smithi100.stdout:2254: oid 23 updating version 1282 to 1283 2024-04-07T16:24:06.825 INFO:tasks.rados.rados.0.smithi100.stdout:2254: oid 23 version 1283 is already newer than 1281 2024-04-07T16:24:06.825 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 23 v 1283 (ObjNum 696 snap 211 seq_num 696) dirty exists 2024-04-07T16:24:06.825 INFO:tasks.rados.rados.0.smithi100.stdout:2254: left oid 23 (ObjNum 696 snap 211 seq_num 696) 2024-04-07T16:24:06.840 INFO:tasks.rados.rados.0.smithi100.stdout:2254: done (1 left) 2024-04-07T16:24:06.840 INFO:tasks.rados.rados.0.smithi100.stdout:2256: done (0 left) 2024-04-07T16:24:06.840 INFO:tasks.rados.rados.0.smithi100.stdout:2257: setattr oid 22 current snap is 211 2024-04-07T16:24:06.843 INFO:tasks.rados.rados.0.smithi100.stdout:2258: snap_create 2024-04-07T16:24:06.849 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 22 v 1101 (ObjNum 664 snap 204 seq_num 664) dirty exists 2024-04-07T16:24:07.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:06 smithi144 bash[66654]: cluster 2024-04-07T16:24:05.882290+0000 mon.a (mon.0) 639 : cluster [DBG] osdmap e492: 8 total, 7 up, 8 in 2024-04-07T16:24:07.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:06 smithi144 bash[66654]: cluster 2024-04-07T16:24:06.837054+0000 mon.a (mon.0) 640 : cluster [DBG] osdmap e493: 8 total, 7 up, 8 in 2024-04-07T16:24:07.183 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:06 smithi100 bash[74942]: cluster 2024-04-07T16:24:05.882290+0000 mon.a (mon.0) 639 : cluster [DBG] osdmap e492: 8 total, 7 up, 8 in 2024-04-07T16:24:07.184 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:06 smithi100 bash[74942]: cluster 2024-04-07T16:24:06.837054+0000 mon.a (mon.0) 640 : cluster [DBG] osdmap e493: 8 total, 7 up, 8 in 2024-04-07T16:24:07.184 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:06 smithi100 bash[76202]: cluster 2024-04-07T16:24:05.882290+0000 mon.a (mon.0) 639 : cluster [DBG] osdmap e492: 8 total, 7 up, 8 in 2024-04-07T16:24:07.184 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:06 smithi100 bash[76202]: cluster 2024-04-07T16:24:06.837054+0000 mon.a (mon.0) 640 : cluster [DBG] osdmap e493: 8 total, 7 up, 8 in 2024-04-07T16:24:07.606 INFO:journalctl@ceph.osd.4.smithi144.stdout:Apr 07 16:24:07 smithi144 bash[71221]: debug 2024-04-07T16:24:07.362+0000 7f17b4190700 1 -- 172.21.15.144:0/2913386911 <== mon.1 v2:172.21.15.144:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b424b9f680 con 0x55b424bdc800 2024-04-07T16:24:07.819 INFO:tasks.rados.rados.0.smithi100.stdout:2257: done (1 left) 2024-04-07T16:24:07.819 INFO:tasks.rados.rados.0.smithi100.stdout:2258: done (0 left) 2024-04-07T16:24:07.819 INFO:tasks.rados.rados.0.smithi100.stdout:2259: snap_remove snap 195 2024-04-07T16:24:08.183 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:07 smithi100 bash[74942]: cluster 2024-04-07T16:24:06.692746+0000 mgr.y (mgr.25175) 202 : cluster [DBG] pgmap v330: 121 pgs: 13 active+undersized+degraded, 34 active+undersized, 74 active+clean; 350 MiB data, 1.7 GiB used, 714 GiB / 715 GiB avail; 9.3 MiB/s rd, 3.4 MiB/s wr, 16 op/s; 74/546 objects degraded (13.553%) 2024-04-07T16:24:08.183 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:07 smithi100 bash[74942]: cluster 2024-04-07T16:24:07.838045+0000 mon.a (mon.0) 641 : cluster [DBG] osdmap e494: 8 total, 7 up, 8 in 2024-04-07T16:24:08.184 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:07 smithi100 bash[76202]: cluster 2024-04-07T16:24:06.692746+0000 mgr.y (mgr.25175) 202 : cluster [DBG] pgmap v330: 121 pgs: 13 active+undersized+degraded, 34 active+undersized, 74 active+clean; 350 MiB data, 1.7 GiB used, 714 GiB / 715 GiB avail; 9.3 MiB/s rd, 3.4 MiB/s wr, 16 op/s; 74/546 objects degraded (13.553%) 2024-04-07T16:24:08.184 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:07 smithi100 bash[76202]: cluster 2024-04-07T16:24:07.838045+0000 mon.a (mon.0) 641 : cluster [DBG] osdmap e494: 8 total, 7 up, 8 in 2024-04-07T16:24:08.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:07 smithi144 bash[66654]: cluster 2024-04-07T16:24:06.692746+0000 mgr.y (mgr.25175) 202 : cluster [DBG] pgmap v330: 121 pgs: 13 active+undersized+degraded, 34 active+undersized, 74 active+clean; 350 MiB data, 1.7 GiB used, 714 GiB / 715 GiB avail; 9.3 MiB/s rd, 3.4 MiB/s wr, 16 op/s; 74/546 objects degraded (13.553%) 2024-04-07T16:24:08.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:07 smithi144 bash[66654]: cluster 2024-04-07T16:24:07.838045+0000 mon.a (mon.0) 641 : cluster [DBG] osdmap e494: 8 total, 7 up, 8 in 2024-04-07T16:24:08.356 INFO:journalctl@ceph.osd.4.smithi144.stdout:Apr 07 16:24:07 smithi144 bash[71221]: debug 2024-04-07T16:24:07.910+0000 7f17bd3287c0 -1 Falling back to public interface 2024-04-07T16:24:08.683 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:24:08 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:24:08] "GET /metrics HTTP/1.1" 200 36209 "" "Prometheus/2.43.0" 2024-04-07T16:24:08.824 INFO:tasks.rados.rados.0.smithi100.stdout:2259: done (0 left) 2024-04-07T16:24:08.825 INFO:tasks.rados.rados.0.smithi100.stdout:2260: read oid 8 snap -1 2024-04-07T16:24:08.825 INFO:tasks.rados.rados.0.smithi100.stdout:2260: expect deleted 2024-04-07T16:24:08.825 INFO:tasks.rados.rados.0.smithi100.stdout:2261: read oid 35 snap -1 2024-04-07T16:24:08.825 INFO:tasks.rados.rados.0.smithi100.stdout:2261: expect (ObjNum 688 snap 210 seq_num 688) 2024-04-07T16:24:08.825 INFO:tasks.rados.rados.0.smithi100.stdout:2262: read oid 6 snap -1 2024-04-07T16:24:08.825 INFO:tasks.rados.rados.0.smithi100.stdout:2262: expect deleted 2024-04-07T16:24:08.825 INFO:tasks.rados.rados.0.smithi100.stdout:2263: snap_remove snap 206 2024-04-07T16:24:09.835 INFO:tasks.rados.rados.0.smithi100.stdout:2263: done (3 left) 2024-04-07T16:24:09.835 INFO:tasks.rados.rados.0.smithi100.stdout:2264: snap_create 2024-04-07T16:24:09.836 INFO:tasks.rados.rados.0.smithi100.stdout:2261: expect (ObjNum 688 snap 210 seq_num 688) 2024-04-07T16:24:10.106 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:24:09 smithi144 bash[50386]: logger=cleanup t=2024-04-07T16:24:09.820797763Z level=info msg="Completed cleanup jobs" duration=78.455608ms 2024-04-07T16:24:10.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:09 smithi144 bash[66654]: cluster 2024-04-07T16:24:08.693777+0000 mgr.y (mgr.25175) 203 : cluster [DBG] pgmap v333: 121 pgs: 13 active+undersized+degraded, 34 active+undersized, 74 active+clean; 346 MiB data, 1.7 GiB used, 714 GiB / 715 GiB avail; 12 MiB/s rd, 1.2 MiB/s wr, 10 op/s; 76/537 objects degraded (14.153%) 2024-04-07T16:24:10.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:09 smithi144 bash[66654]: cluster 2024-04-07T16:24:08.842396+0000 mon.a (mon.0) 642 : cluster [DBG] osdmap e495: 8 total, 7 up, 8 in 2024-04-07T16:24:10.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:09 smithi144 bash[66654]: cluster 2024-04-07T16:24:08.887959+0000 mon.a (mon.0) 643 : cluster [WRN] Health check update: Degraded data redundancy: 76/537 objects degraded (14.153%), 13 pgs degraded (PG_DEGRADED) 2024-04-07T16:24:10.183 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:09 smithi100 bash[74942]: cluster 2024-04-07T16:24:08.693777+0000 mgr.y (mgr.25175) 203 : cluster [DBG] pgmap v333: 121 pgs: 13 active+undersized+degraded, 34 active+undersized, 74 active+clean; 346 MiB data, 1.7 GiB used, 714 GiB / 715 GiB avail; 12 MiB/s rd, 1.2 MiB/s wr, 10 op/s; 76/537 objects degraded (14.153%) 2024-04-07T16:24:10.184 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:09 smithi100 bash[74942]: cluster 2024-04-07T16:24:08.842396+0000 mon.a (mon.0) 642 : cluster [DBG] osdmap e495: 8 total, 7 up, 8 in 2024-04-07T16:24:10.184 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:09 smithi100 bash[74942]: cluster 2024-04-07T16:24:08.887959+0000 mon.a (mon.0) 643 : cluster [WRN] Health check update: Degraded data redundancy: 76/537 objects degraded (14.153%), 13 pgs degraded (PG_DEGRADED) 2024-04-07T16:24:10.184 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:09 smithi100 bash[76202]: cluster 2024-04-07T16:24:08.693777+0000 mgr.y (mgr.25175) 203 : cluster [DBG] pgmap v333: 121 pgs: 13 active+undersized+degraded, 34 active+undersized, 74 active+clean; 346 MiB data, 1.7 GiB used, 714 GiB / 715 GiB avail; 12 MiB/s rd, 1.2 MiB/s wr, 10 op/s; 76/537 objects degraded (14.153%) 2024-04-07T16:24:10.184 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:09 smithi100 bash[76202]: cluster 2024-04-07T16:24:08.842396+0000 mon.a (mon.0) 642 : cluster [DBG] osdmap e495: 8 total, 7 up, 8 in 2024-04-07T16:24:10.184 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:09 smithi100 bash[76202]: cluster 2024-04-07T16:24:08.887959+0000 mon.a (mon.0) 643 : cluster [WRN] Health check update: Degraded data redundancy: 76/537 objects degraded (14.153%), 13 pgs degraded (PG_DEGRADED) 2024-04-07T16:24:10.847 INFO:tasks.rados.rados.0.smithi100.stdout:2260: done (3 left) 2024-04-07T16:24:10.847 INFO:tasks.rados.rados.0.smithi100.stdout:2261: done (2 left) 2024-04-07T16:24:10.847 INFO:tasks.rados.rados.0.smithi100.stdout:2262: done (1 left) 2024-04-07T16:24:10.848 INFO:tasks.rados.rados.0.smithi100.stdout:2264: done (0 left) 2024-04-07T16:24:10.848 INFO:tasks.rados.rados.0.smithi100.stdout:2265: snap_remove snap 211 2024-04-07T16:24:11.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:10 smithi144 bash[66654]: cluster 2024-04-07T16:24:09.852838+0000 mon.a (mon.0) 644 : cluster [DBG] osdmap e496: 8 total, 7 up, 8 in 2024-04-07T16:24:11.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:10 smithi144 bash[66654]: audit 2024-04-07T16:24:10.548414+0000 mon.a (mon.0) 645 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:24:11.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:10 smithi144 bash[66654]: audit 2024-04-07T16:24:10.562012+0000 mon.a (mon.0) 646 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:24:11.183 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:10 smithi100 bash[74942]: cluster 2024-04-07T16:24:09.852838+0000 mon.a (mon.0) 644 : cluster [DBG] osdmap e496: 8 total, 7 up, 8 in 2024-04-07T16:24:11.183 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:10 smithi100 bash[74942]: audit 2024-04-07T16:24:10.548414+0000 mon.a (mon.0) 645 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:24:11.184 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:10 smithi100 bash[74942]: audit 2024-04-07T16:24:10.562012+0000 mon.a (mon.0) 646 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:24:11.184 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:10 smithi100 bash[76202]: cluster 2024-04-07T16:24:09.852838+0000 mon.a (mon.0) 644 : cluster [DBG] osdmap e496: 8 total, 7 up, 8 in 2024-04-07T16:24:11.184 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:10 smithi100 bash[76202]: audit 2024-04-07T16:24:10.548414+0000 mon.a (mon.0) 645 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:24:11.184 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:10 smithi100 bash[76202]: audit 2024-04-07T16:24:10.562012+0000 mon.a (mon.0) 646 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:24:11.841 INFO:journalctl@ceph.osd.4.smithi144.stdout:Apr 07 16:24:11 smithi144 bash[71221]: debug 2024-04-07T16:24:11.574+0000 7f17bd3287c0 -1 osd.4 487 log_to_monitors true 2024-04-07T16:24:11.858 INFO:tasks.rados.rados.0.smithi100.stdout:2265: done (0 left) 2024-04-07T16:24:11.858 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 6 current snap is 213 2024-04-07T16:24:11.859 INFO:tasks.rados.rados.0.smithi100.stdout:2266: seq_num 698 ranges {0=499712,499712=589824,1089536=770048,1859584=245760} 2024-04-07T16:24:11.862 INFO:tasks.rados.rados.0.smithi100.stdout:2266: writing smithi10066867-6 from 0 to 499712 tid 1 2024-04-07T16:24:11.866 INFO:tasks.rados.rados.0.smithi100.stdout:2266: writing smithi10066867-6 from 499712 to 1089536 tid 2 2024-04-07T16:24:11.871 INFO:tasks.rados.rados.0.smithi100.stdout:2266: writing smithi10066867-6 from 1089536 to 1859584 tid 3 2024-04-07T16:24:11.873 INFO:tasks.rados.rados.0.smithi100.stdout:2266: writing smithi10066867-6 from 1859584 to 2105344 tid 4 2024-04-07T16:24:11.873 INFO:tasks.rados.rados.0.smithi100.stdout:2267: copy_from oid 20 from oid 45 current snap is 213 2024-04-07T16:24:11.873 INFO:tasks.rados.rados.0.smithi100.stdout:2268: rollback oid 39 current snap is 213 2024-04-07T16:24:11.873 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 39 to 178 2024-04-07T16:24:11.873 INFO:tasks.rados.rados.0.smithi100.stdout:2269: read oid 30 snap 210 2024-04-07T16:24:11.874 INFO:tasks.rados.rados.0.smithi100.stdout:2269: expect (ObjNum 650 snap 203 seq_num 650) 2024-04-07T16:24:11.874 INFO:tasks.rados.rados.0.smithi100.stdout:2270: read oid 26 snap 186 2024-04-07T16:24:11.874 INFO:tasks.rados.rados.0.smithi100.stdout:2270: expect (ObjNum 524 snap 156 seq_num 524) 2024-04-07T16:24:11.874 INFO:tasks.rados.rados.0.smithi100.stdout:2271: copy_from oid 1 from oid 42 current snap is 213 2024-04-07T16:24:11.874 INFO:tasks.rados.rados.0.smithi100.stdout:2272: rollback oid 23 current snap is 213 2024-04-07T16:24:11.874 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 23 to 186 2024-04-07T16:24:11.874 INFO:tasks.rados.rados.0.smithi100.stdout:2273: copy_from oid 24 from oid 41 current snap is 213 2024-04-07T16:24:11.874 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 15 current snap is 213 2024-04-07T16:24:11.874 INFO:tasks.rados.rados.0.smithi100.stdout:2274: seq_num 699 ranges {1581056=516096,2097152=475136,2572288=573440,3145728=557056} 2024-04-07T16:24:11.888 INFO:tasks.rados.rados.0.smithi100.stdout:2274: writing smithi10066867-15 from 1581056 to 2097152 tid 1 2024-04-07T16:24:11.891 INFO:tasks.rados.rados.0.smithi100.stdout:2274: writing smithi10066867-15 from 2097152 to 2572288 tid 2 2024-04-07T16:24:11.895 INFO:tasks.rados.rados.0.smithi100.stdout:2274: writing smithi10066867-15 from 2572288 to 3145728 tid 3 2024-04-07T16:24:11.899 INFO:tasks.rados.rados.0.smithi100.stdout:2274: writing smithi10066867-15 from 3145728 to 3702784 tid 4 2024-04-07T16:24:11.899 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 48 current snap is 213 2024-04-07T16:24:11.899 INFO:tasks.rados.rados.0.smithi100.stdout:2275: seq_num 700 ranges {5816320=532480,6348800=409600,6758400=737280,7495680=458752} 2024-04-07T16:24:11.937 INFO:tasks.rados.rados.0.smithi100.stdout:2275: writing smithi10066867-48 from 5816320 to 6348800 tid 1 2024-04-07T16:24:11.939 INFO:tasks.rados.rados.0.smithi100.stdout:2275: writing smithi10066867-48 from 6348800 to 6758400 tid 2 2024-04-07T16:24:11.944 INFO:tasks.rados.rados.0.smithi100.stdout:2275: writing smithi10066867-48 from 6758400 to 7495680 tid 3 2024-04-07T16:24:11.948 INFO:tasks.rados.rados.0.smithi100.stdout:2275: writing smithi10066867-48 from 7495680 to 7954432 tid 4 2024-04-07T16:24:11.948 INFO:tasks.rados.rados.0.smithi100.stdout:2276: read oid 43 snap -1 2024-04-07T16:24:11.948 INFO:tasks.rados.rados.0.smithi100.stdout:2276: expect (ObjNum 691 snap 211 seq_num 691) 2024-04-07T16:24:11.948 INFO:tasks.rados.rados.0.smithi100.stdout:2277: delete oid 13 current snap is 213 2024-04-07T16:24:11.949 INFO:tasks.rados.rados.0.smithi100.stdout:2268: finishing rollback tid 0 to smithi10066867-39 2024-04-07T16:24:11.949 INFO:tasks.rados.rados.0.smithi100.stdout:2268: finishing rollback tid 1 to smithi10066867-39 2024-04-07T16:24:11.949 INFO:tasks.rados.rados.0.smithi100.stdout:2268: finishing rollback tid 2 to smithi10066867-39 2024-04-07T16:24:11.949 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 39 v 1113 (ObjNum 560 snap 171 seq_num 560) dirty exists 2024-04-07T16:24:11.949 INFO:tasks.rados.rados.0.smithi100.stdout:2267: finishing copy_from to smithi10066867-20 2024-04-07T16:24:11.949 INFO:tasks.rados.rados.0.smithi100.stdout:2267: got expected ENOENT (src dne) 2024-04-07T16:24:11.949 INFO:tasks.rados.rados.0.smithi100.stdout:2267: finishing copy_from racing read to smithi10066867-20 2024-04-07T16:24:11.949 INFO:tasks.rados.rados.0.smithi100.stdout:2266: finishing write tid 1 to smithi10066867-6 2024-04-07T16:24:11.949 INFO:tasks.rados.rados.0.smithi100.stdout:2273: finishing copy_from racing read to smithi10066867-24 2024-04-07T16:24:11.949 INFO:tasks.rados.rados.0.smithi100.stdout:2273: finishing copy_from to smithi10066867-24 2024-04-07T16:24:11.949 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 24 v 823 (ObjNum 684 snap 210 seq_num 684) dirty exists 2024-04-07T16:24:11.949 INFO:tasks.rados.rados.0.smithi100.stdout:2266: finishing write tid 2 to smithi10066867-6 2024-04-07T16:24:11.949 INFO:tasks.rados.rados.0.smithi100.stdout:2266: finishing write tid 3 to smithi10066867-6 2024-04-07T16:24:11.949 INFO:tasks.rados.rados.0.smithi100.stdout:2266: finishing write tid 4 to smithi10066867-6 2024-04-07T16:24:11.950 INFO:tasks.rados.rados.0.smithi100.stdout:2272: finishing rollback tid 0 to smithi10066867-23 2024-04-07T16:24:11.950 INFO:tasks.rados.rados.0.smithi100.stdout:2272: finishing rollback tid 1 to smithi10066867-23 2024-04-07T16:24:11.950 INFO:tasks.rados.rados.0.smithi100.stdout:2272: finishing rollback tid 2 to smithi10066867-23 2024-04-07T16:24:11.950 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 23 v 1287 (ObjNum 596 snap 183 seq_num 596) dirty exists 2024-04-07T16:24:11.950 INFO:tasks.rados.rados.0.smithi100.stdout:2271: finishing copy_from to smithi10066867-1 2024-04-07T16:24:11.950 INFO:tasks.rados.rados.0.smithi100.stdout:2271: got expected ENOENT (src dne) 2024-04-07T16:24:11.950 INFO:tasks.rados.rados.0.smithi100.stdout:2274: finishing write tid 1 to smithi10066867-15 2024-04-07T16:24:11.950 INFO:tasks.rados.rados.0.smithi100.stdout:2271: finishing copy_from racing read to smithi10066867-1 2024-04-07T16:24:11.950 INFO:tasks.rados.rados.0.smithi100.stdout:2269: expect (ObjNum 650 snap 203 seq_num 650) 2024-04-07T16:24:11.985 INFO:tasks.rados.rados.0.smithi100.stdout:2266: finishing write tid 5 to smithi10066867-6 2024-04-07T16:24:11.985 INFO:tasks.rados.rados.0.smithi100.stdout:2274: finishing write tid 2 to smithi10066867-15 2024-04-07T16:24:11.985 INFO:tasks.rados.rados.0.smithi100.stdout:2266: finishing write tid 6 to smithi10066867-6 2024-04-07T16:24:11.985 INFO:tasks.rados.rados.0.smithi100.stdout:2266: oid 6 updating version 0 to 1117 2024-04-07T16:24:11.986 INFO:tasks.rados.rados.0.smithi100.stdout:2266: oid 6 updating version 1117 to 1118 2024-04-07T16:24:11.986 INFO:tasks.rados.rados.0.smithi100.stdout:2266: oid 6 updating version 1118 to 1119 2024-04-07T16:24:11.986 INFO:tasks.rados.rados.0.smithi100.stdout:2266: oid 6 updating version 1119 to 1120 2024-04-07T16:24:11.986 INFO:tasks.rados.rados.0.smithi100.stdout:2266: oid 6 version 1120 is already newer than 1116 2024-04-07T16:24:11.986 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 6 v 1120 (ObjNum 698 snap 213 seq_num 698) dirty exists 2024-04-07T16:24:11.986 INFO:tasks.rados.rados.0.smithi100.stdout:2266: left oid 6 (ObjNum 698 snap 213 seq_num 698) 2024-04-07T16:24:11.986 INFO:tasks.rados.rados.0.smithi100.stdout:2274: finishing write tid 3 to smithi10066867-15 2024-04-07T16:24:11.986 INFO:tasks.rados.rados.0.smithi100.stdout:2266: done (11 left) 2024-04-07T16:24:11.986 INFO:tasks.rados.rados.0.smithi100.stdout:2267: done (10 left) 2024-04-07T16:24:11.986 INFO:tasks.rados.rados.0.smithi100.stdout:2268: done (9 left) 2024-04-07T16:24:11.986 INFO:tasks.rados.rados.0.smithi100.stdout:2269: done (8 left) 2024-04-07T16:24:11.986 INFO:tasks.rados.rados.0.smithi100.stdout:2271: done (7 left) 2024-04-07T16:24:11.986 INFO:tasks.rados.rados.0.smithi100.stdout:2272: done (6 left) 2024-04-07T16:24:11.986 INFO:tasks.rados.rados.0.smithi100.stdout:2273: done (5 left) 2024-04-07T16:24:11.987 INFO:tasks.rados.rados.0.smithi100.stdout:2277: done (4 left) 2024-04-07T16:24:11.987 INFO:tasks.rados.rados.0.smithi100.stdout:2278: read oid 40 snap -1 2024-04-07T16:24:11.987 INFO:tasks.rados.rados.0.smithi100.stdout:2278: expect (ObjNum 509 snap 152 seq_num 509) 2024-04-07T16:24:11.987 INFO:tasks.rados.rados.0.smithi100.stdout:2279: delete oid 21 current snap is 213 2024-04-07T16:24:11.987 INFO:tasks.rados.rados.0.smithi100.stdout:2274: finishing write tid 4 to smithi10066867-15 2024-04-07T16:24:11.987 INFO:tasks.rados.rados.0.smithi100.stdout:2274: finishing write tid 5 to smithi10066867-15 2024-04-07T16:24:11.987 INFO:tasks.rados.rados.0.smithi100.stdout:2274: finishing write tid 6 to smithi10066867-15 2024-04-07T16:24:11.987 INFO:tasks.rados.rados.0.smithi100.stdout:2274: oid 15 updating version 0 to 1289 2024-04-07T16:24:11.987 INFO:tasks.rados.rados.0.smithi100.stdout:2274: oid 15 updating version 1289 to 1290 2024-04-07T16:24:11.987 INFO:tasks.rados.rados.0.smithi100.stdout:2274: oid 15 updating version 1290 to 1293 2024-04-07T16:24:11.987 INFO:tasks.rados.rados.0.smithi100.stdout:2274: oid 15 version 1293 is already newer than 1291 2024-04-07T16:24:11.987 INFO:tasks.rados.rados.0.smithi100.stdout:2274: oid 15 version 1293 is already newer than 1292 2024-04-07T16:24:11.987 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 15 v 1293 (ObjNum 699 snap 213 seq_num 699) dirty exists 2024-04-07T16:24:11.987 INFO:tasks.rados.rados.0.smithi100.stdout:2274: left oid 15 (ObjNum 699 snap 213 seq_num 699) 2024-04-07T16:24:11.987 INFO:tasks.rados.rados.0.smithi100.stdout:2270: expect (ObjNum 524 snap 156 seq_num 524) 2024-04-07T16:24:12.106 INFO:journalctl@ceph.osd.4.smithi144.stdout:Apr 07 16:24:11 smithi144 bash[71221]: debug 2024-04-07T16:24:11.910+0000 7f17ae780700 -1 osd.4 487 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-07T16:24:12.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:11 smithi144 bash[66654]: cluster 2024-04-07T16:24:10.694830+0000 mgr.y (mgr.25175) 204 : cluster [DBG] pgmap v336: 121 pgs: 2 active+clean+snaptrim, 2 active+clean+snaptrim_wait, 13 active+undersized+degraded, 34 active+undersized, 70 active+clean; 348 MiB data, 1.7 GiB used, 714 GiB / 715 GiB avail; 14 MiB/s rd, 1.4 MiB/s wr, 18 op/s; 78/546 objects degraded (14.286%) 2024-04-07T16:24:12.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:11 smithi144 bash[66654]: cluster 2024-04-07T16:24:10.864529+0000 mon.a (mon.0) 647 : cluster [DBG] osdmap e497: 8 total, 7 up, 8 in 2024-04-07T16:24:12.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:11 smithi144 bash[66654]: audit 2024-04-07T16:24:11.582622+0000 mon.b (mon.1) 3 : audit [INF] from='osd.4 [v2:172.21.15.144:6800/1068767678,v1:172.21.15.144:6801/1068767678]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-07T16:24:12.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:11 smithi144 bash[66654]: audit 2024-04-07T16:24:11.592273+0000 mon.a (mon.0) 648 : audit [INF] from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-07T16:24:12.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:11 smithi144 bash[66654]: audit 2024-04-07T16:24:11.769570+0000 mon.a (mon.0) 649 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:24:12.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:11 smithi144 bash[66654]: audit 2024-04-07T16:24:11.781806+0000 mon.a (mon.0) 650 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:24:12.183 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:11 smithi100 bash[74942]: cluster 2024-04-07T16:24:10.694830+0000 mgr.y (mgr.25175) 204 : cluster [DBG] pgmap v336: 121 pgs: 2 active+clean+snaptrim, 2 active+clean+snaptrim_wait, 13 active+undersized+degraded, 34 active+undersized, 70 active+clean; 348 MiB data, 1.7 GiB used, 714 GiB / 715 GiB avail; 14 MiB/s rd, 1.4 MiB/s wr, 18 op/s; 78/546 objects degraded (14.286%) 2024-04-07T16:24:12.184 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:11 smithi100 bash[74942]: cluster 2024-04-07T16:24:10.864529+0000 mon.a (mon.0) 647 : cluster [DBG] osdmap e497: 8 total, 7 up, 8 in 2024-04-07T16:24:12.184 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:11 smithi100 bash[74942]: audit 2024-04-07T16:24:11.582622+0000 mon.b (mon.1) 3 : audit [INF] from='osd.4 [v2:172.21.15.144:6800/1068767678,v1:172.21.15.144:6801/1068767678]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-07T16:24:12.184 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:11 smithi100 bash[74942]: audit 2024-04-07T16:24:11.592273+0000 mon.a (mon.0) 648 : audit [INF] from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-07T16:24:12.184 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:11 smithi100 bash[74942]: audit 2024-04-07T16:24:11.769570+0000 mon.a (mon.0) 649 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:24:12.184 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:11 smithi100 bash[74942]: audit 2024-04-07T16:24:11.781806+0000 mon.a (mon.0) 650 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:24:12.184 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:11 smithi100 bash[76202]: cluster 2024-04-07T16:24:10.694830+0000 mgr.y (mgr.25175) 204 : cluster [DBG] pgmap v336: 121 pgs: 2 active+clean+snaptrim, 2 active+clean+snaptrim_wait, 13 active+undersized+degraded, 34 active+undersized, 70 active+clean; 348 MiB data, 1.7 GiB used, 714 GiB / 715 GiB avail; 14 MiB/s rd, 1.4 MiB/s wr, 18 op/s; 78/546 objects degraded (14.286%) 2024-04-07T16:24:12.184 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:11 smithi100 bash[76202]: cluster 2024-04-07T16:24:10.864529+0000 mon.a (mon.0) 647 : cluster [DBG] osdmap e497: 8 total, 7 up, 8 in 2024-04-07T16:24:12.184 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:11 smithi100 bash[76202]: audit 2024-04-07T16:24:11.582622+0000 mon.b (mon.1) 3 : audit [INF] from='osd.4 [v2:172.21.15.144:6800/1068767678,v1:172.21.15.144:6801/1068767678]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-07T16:24:12.184 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:11 smithi100 bash[76202]: audit 2024-04-07T16:24:11.592273+0000 mon.a (mon.0) 648 : audit [INF] from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-07T16:24:12.184 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:11 smithi100 bash[76202]: audit 2024-04-07T16:24:11.769570+0000 mon.a (mon.0) 649 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:24:12.184 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:11 smithi100 bash[76202]: audit 2024-04-07T16:24:11.781806+0000 mon.a (mon.0) 650 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:24:12.449 INFO:tasks.rados.rados.0.smithi100.stdout:2275: finishing write tid 1 to smithi10066867-48 2024-04-07T16:24:12.449 INFO:tasks.rados.rados.0.smithi100.stdout:2275: finishing write tid 2 to smithi10066867-48 2024-04-07T16:24:12.450 INFO:tasks.rados.rados.0.smithi100.stdout:2275: finishing write tid 3 to smithi10066867-48 2024-04-07T16:24:12.450 INFO:tasks.rados.rados.0.smithi100.stdout:2275: finishing write tid 4 to smithi10066867-48 2024-04-07T16:24:12.450 INFO:tasks.rados.rados.0.smithi100.stdout:2275: finishing write tid 5 to smithi10066867-48 2024-04-07T16:24:12.450 INFO:tasks.rados.rados.0.smithi100.stdout:2275: finishing write tid 6 to smithi10066867-48 2024-04-07T16:24:12.450 INFO:tasks.rados.rados.0.smithi100.stdout:2275: oid 48 updating version 0 to 1085 2024-04-07T16:24:12.450 INFO:tasks.rados.rados.0.smithi100.stdout:2275: oid 48 version 1085 is already newer than 1083 2024-04-07T16:24:12.450 INFO:tasks.rados.rados.0.smithi100.stdout:2275: oid 48 version 1085 is already newer than 1081 2024-04-07T16:24:12.450 INFO:tasks.rados.rados.0.smithi100.stdout:2275: oid 48 version 1085 is already newer than 1082 2024-04-07T16:24:12.450 INFO:tasks.rados.rados.0.smithi100.stdout:2275: oid 48 version 1085 is already newer than 1084 2024-04-07T16:24:12.450 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 48 v 1085 (ObjNum 700 snap 213 seq_num 700) dirty exists 2024-04-07T16:24:12.450 INFO:tasks.rados.rados.0.smithi100.stdout:2275: left oid 48 (ObjNum 700 snap 213 seq_num 700) 2024-04-07T16:24:12.450 INFO:tasks.rados.rados.0.smithi100.stdout:2276: expect (ObjNum 691 snap 211 seq_num 691) 2024-04-07T16:24:12.606 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:24:12 smithi144 bash[59794]: ts=2024-04-07T16:24:12.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:24:12.949 INFO:tasks.rados.rados.0.smithi100.stdout:2278: expect (ObjNum 509 snap 152 seq_num 509) 2024-04-07T16:24:13.183 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:12 smithi100 bash[74942]: audit 2024-04-07T16:24:11.868731+0000 mon.a (mon.0) 651 : audit [INF] from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-07T16:24:13.183 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:12 smithi100 bash[74942]: audit 2024-04-07T16:24:11.868991+0000 mon.b (mon.1) 4 : audit [INF] from='osd.4 [v2:172.21.15.144:6800/1068767678,v1:172.21.15.144:6801/1068767678]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-04-07T16:24:13.183 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:12 smithi100 bash[74942]: cluster 2024-04-07T16:24:11.876307+0000 mon.a (mon.0) 652 : cluster [DBG] osdmap e498: 8 total, 7 up, 8 in 2024-04-07T16:24:13.183 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:12 smithi100 bash[74942]: audit 2024-04-07T16:24:11.878809+0000 mon.a (mon.0) 653 : audit [INF] from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-04-07T16:24:13.184 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:12 smithi100 bash[76202]: audit 2024-04-07T16:24:11.868731+0000 mon.a (mon.0) 651 : audit [INF] from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-07T16:24:13.184 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:12 smithi100 bash[76202]: audit 2024-04-07T16:24:11.868991+0000 mon.b (mon.1) 4 : audit [INF] from='osd.4 [v2:172.21.15.144:6800/1068767678,v1:172.21.15.144:6801/1068767678]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-04-07T16:24:13.184 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:12 smithi100 bash[76202]: cluster 2024-04-07T16:24:11.876307+0000 mon.a (mon.0) 652 : cluster [DBG] osdmap e498: 8 total, 7 up, 8 in 2024-04-07T16:24:13.184 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:12 smithi100 bash[76202]: audit 2024-04-07T16:24:11.878809+0000 mon.a (mon.0) 653 : audit [INF] from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-04-07T16:24:13.313 INFO:tasks.rados.rados.0.smithi100.stdout:2270: done (5 left) 2024-04-07T16:24:13.313 INFO:tasks.rados.rados.0.smithi100.stdout:2274: done (4 left) 2024-04-07T16:24:13.314 INFO:tasks.rados.rados.0.smithi100.stdout:2275: done (3 left) 2024-04-07T16:24:13.314 INFO:tasks.rados.rados.0.smithi100.stdout:2276: done (2 left) 2024-04-07T16:24:13.314 INFO:tasks.rados.rados.0.smithi100.stdout:2278: done (1 left) 2024-04-07T16:24:13.314 INFO:tasks.rados.rados.0.smithi100.stdout:2279: done (0 left) 2024-04-07T16:24:13.314 INFO:tasks.rados.rados.0.smithi100.stdout:2280: snap_remove snap 212 2024-04-07T16:24:13.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:12 smithi144 bash[66654]: audit 2024-04-07T16:24:11.868731+0000 mon.a (mon.0) 651 : audit [INF] from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-07T16:24:13.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:12 smithi144 bash[66654]: audit 2024-04-07T16:24:11.868991+0000 mon.b (mon.1) 4 : audit [INF] from='osd.4 [v2:172.21.15.144:6800/1068767678,v1:172.21.15.144:6801/1068767678]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-04-07T16:24:13.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:12 smithi144 bash[66654]: cluster 2024-04-07T16:24:11.876307+0000 mon.a (mon.0) 652 : cluster [DBG] osdmap e498: 8 total, 7 up, 8 in 2024-04-07T16:24:13.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:12 smithi144 bash[66654]: audit 2024-04-07T16:24:11.878809+0000 mon.a (mon.0) 653 : audit [INF] from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-04-07T16:24:13.881 INFO:tasks.rados.rados.0.smithi100.stdout:2280: done (0 left) 2024-04-07T16:24:13.881 INFO:tasks.rados.rados.0.smithi100.stdout:2281: read oid 16 snap 178 2024-04-07T16:24:13.881 INFO:tasks.rados.rados.0.smithi100.stdout:2281: expect (ObjNum 465 snap 142 seq_num 465) 2024-04-07T16:24:13.881 INFO:tasks.rados.rados.0.smithi100.stdout:2282: delete oid 12 current snap is 213 2024-04-07T16:24:13.947 INFO:tasks.rados.rados.0.smithi100.stdout:2281: expect (ObjNum 465 snap 142 seq_num 465) 2024-04-07T16:24:14.124 INFO:tasks.rados.rados.0.smithi100.stdout:2281: done (1 left) 2024-04-07T16:24:14.125 INFO:tasks.rados.rados.0.smithi100.stdout:2282: done (0 left) 2024-04-07T16:24:14.125 INFO:tasks.rados.rados.0.smithi100.stdout:2283: read oid 7 snap -1 2024-04-07T16:24:14.125 INFO:tasks.rados.rados.0.smithi100.stdout:2283: expect (ObjNum 574 snap 177 seq_num 574) 2024-04-07T16:24:14.125 INFO:tasks.rados.rados.0.smithi100.stdout:2284: snap_create 2024-04-07T16:24:14.136 INFO:tasks.rados.rados.0.smithi100.stdout:2283: expect (ObjNum 574 snap 177 seq_num 574) 2024-04-07T16:24:14.183 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:13 smithi100 bash[74942]: cluster 2024-04-07T16:24:12.696132+0000 mgr.y (mgr.25175) 205 : cluster [DBG] pgmap v339: 121 pgs: 2 active+clean+snaptrim, 2 active+clean+snaptrim_wait, 13 active+undersized+degraded, 34 active+undersized, 70 active+clean; 336 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 255 B/s rd, 504 KiB/s wr, 7 op/s; 78/528 objects degraded (14.773%) 2024-04-07T16:24:14.183 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:13 smithi100 bash[74942]: cluster 2024-04-07T16:24:12.870295+0000 mon.a (mon.0) 654 : cluster [INF] Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-07T16:24:14.183 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:13 smithi100 bash[74942]: cluster 2024-04-07T16:24:12.890021+0000 mon.a (mon.0) 655 : cluster [INF] osd.4 [v2:172.21.15.144:6800/1068767678,v1:172.21.15.144:6801/1068767678] boot 2024-04-07T16:24:14.183 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:13 smithi100 bash[74942]: cluster 2024-04-07T16:24:12.890126+0000 mon.a (mon.0) 656 : cluster [DBG] osdmap e499: 8 total, 8 up, 8 in 2024-04-07T16:24:14.183 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:13 smithi100 bash[74942]: audit 2024-04-07T16:24:12.899092+0000 mon.a (mon.0) 657 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-07T16:24:14.184 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:13 smithi100 bash[76202]: cluster 2024-04-07T16:24:12.696132+0000 mgr.y (mgr.25175) 205 : cluster [DBG] pgmap v339: 121 pgs: 2 active+clean+snaptrim, 2 active+clean+snaptrim_wait, 13 active+undersized+degraded, 34 active+undersized, 70 active+clean; 336 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 255 B/s rd, 504 KiB/s wr, 7 op/s; 78/528 objects degraded (14.773%) 2024-04-07T16:24:14.184 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:13 smithi100 bash[76202]: cluster 2024-04-07T16:24:12.870295+0000 mon.a (mon.0) 654 : cluster [INF] Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-07T16:24:14.184 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:13 smithi100 bash[76202]: cluster 2024-04-07T16:24:12.890021+0000 mon.a (mon.0) 655 : cluster [INF] osd.4 [v2:172.21.15.144:6800/1068767678,v1:172.21.15.144:6801/1068767678] boot 2024-04-07T16:24:14.184 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:13 smithi100 bash[76202]: cluster 2024-04-07T16:24:12.890126+0000 mon.a (mon.0) 656 : cluster [DBG] osdmap e499: 8 total, 8 up, 8 in 2024-04-07T16:24:14.184 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:13 smithi100 bash[76202]: audit 2024-04-07T16:24:12.899092+0000 mon.a (mon.0) 657 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-07T16:24:14.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:13 smithi144 bash[66654]: cluster 2024-04-07T16:24:12.696132+0000 mgr.y (mgr.25175) 205 : cluster [DBG] pgmap v339: 121 pgs: 2 active+clean+snaptrim, 2 active+clean+snaptrim_wait, 13 active+undersized+degraded, 34 active+undersized, 70 active+clean; 336 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 255 B/s rd, 504 KiB/s wr, 7 op/s; 78/528 objects degraded (14.773%) 2024-04-07T16:24:14.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:13 smithi144 bash[66654]: cluster 2024-04-07T16:24:12.870295+0000 mon.a (mon.0) 654 : cluster [INF] Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-07T16:24:14.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:13 smithi144 bash[66654]: cluster 2024-04-07T16:24:12.890021+0000 mon.a (mon.0) 655 : cluster [INF] osd.4 [v2:172.21.15.144:6800/1068767678,v1:172.21.15.144:6801/1068767678] boot 2024-04-07T16:24:14.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:13 smithi144 bash[66654]: cluster 2024-04-07T16:24:12.890126+0000 mon.a (mon.0) 656 : cluster [DBG] osdmap e499: 8 total, 8 up, 8 in 2024-04-07T16:24:14.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:13 smithi144 bash[66654]: audit 2024-04-07T16:24:12.899092+0000 mon.a (mon.0) 657 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-07T16:24:14.890 INFO:tasks.rados.rados.0.smithi100.stdout:2283: done (1 left) 2024-04-07T16:24:14.890 INFO:tasks.rados.rados.0.smithi100.stdout:2284: done (0 left) 2024-04-07T16:24:14.890 INFO:tasks.rados.rados.0.smithi100.stdout:2285: read oid 46 snap 213 2024-04-07T16:24:14.891 INFO:tasks.rados.rados.0.smithi100.stdout:2285: expect (ObjNum 680 snap 209 seq_num 680) 2024-04-07T16:24:14.891 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 3 current snap is 214 2024-04-07T16:24:14.891 INFO:tasks.rados.rados.0.smithi100.stdout:2286: seq_num 704 ranges {0=598016,598016=745472,1343488=712704,2056192=147456} 2024-04-07T16:24:14.892 INFO:tasks.rados.rados.0.smithi100.stdout:2286: writing smithi10066867-3 from 0 to 598016 tid 1 2024-04-07T16:24:14.897 INFO:tasks.rados.rados.0.smithi100.stdout:2286: writing smithi10066867-3 from 598016 to 1343488 tid 2 2024-04-07T16:24:14.901 INFO:tasks.rados.rados.0.smithi100.stdout:2286: writing smithi10066867-3 from 1343488 to 2056192 tid 3 2024-04-07T16:24:14.902 INFO:tasks.rados.rados.0.smithi100.stdout:2286: writing smithi10066867-3 from 2056192 to 2203648 tid 4 2024-04-07T16:24:14.902 INFO:tasks.rados.rados.0.smithi100.stdout:2287: read oid 28 snap -1 2024-04-07T16:24:14.902 INFO:tasks.rados.rados.0.smithi100.stdout:2287: expect (ObjNum 461 snap 140 seq_num 461) 2024-04-07T16:24:14.902 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 7 current snap is 214 2024-04-07T16:24:14.902 INFO:tasks.rados.rados.0.smithi100.stdout:2288: seq_num 705 ranges {3817472=614400,4431872=630784,5062656=524288,5586944=450560} 2024-04-07T16:24:14.929 INFO:tasks.rados.rados.0.smithi100.stdout:2288: writing smithi10066867-7 from 3817472 to 4431872 tid 1 2024-04-07T16:24:14.932 INFO:tasks.rados.rados.0.smithi100.stdout:2288: writing smithi10066867-7 from 4431872 to 5062656 tid 2 2024-04-07T16:24:14.936 INFO:tasks.rados.rados.0.smithi100.stdout:2288: writing smithi10066867-7 from 5062656 to 5586944 tid 3 2024-04-07T16:24:14.939 INFO:tasks.rados.rados.0.smithi100.stdout:2288: writing smithi10066867-7 from 5586944 to 6037504 tid 4 2024-04-07T16:24:14.939 INFO:tasks.rados.rados.0.smithi100.stdout:2289: snap_create 2024-04-07T16:24:14.939 INFO:tasks.rados.rados.0.smithi100.stdout:2287: expect (ObjNum 461 snap 140 seq_num 461) 2024-04-07T16:24:15.020 INFO:tasks.rados.rados.0.smithi100.stdout:2286: finishing write tid 1 to smithi10066867-3 2024-04-07T16:24:15.020 INFO:tasks.rados.rados.0.smithi100.stdout:2286: finishing write tid 2 to smithi10066867-3 2024-04-07T16:24:15.020 INFO:tasks.rados.rados.0.smithi100.stdout:2286: finishing write tid 3 to smithi10066867-3 2024-04-07T16:24:15.020 INFO:tasks.rados.rados.0.smithi100.stdout:2286: finishing write tid 4 to smithi10066867-3 2024-04-07T16:24:15.020 INFO:tasks.rados.rados.0.smithi100.stdout:2286: finishing write tid 5 to smithi10066867-3 2024-04-07T16:24:15.020 INFO:tasks.rados.rados.0.smithi100.stdout:2286: finishing write tid 6 to smithi10066867-3 2024-04-07T16:24:15.021 INFO:tasks.rados.rados.0.smithi100.stdout:2286: oid 3 updating version 0 to 1097 2024-04-07T16:24:15.021 INFO:tasks.rados.rados.0.smithi100.stdout:2286: oid 3 updating version 1097 to 1098 2024-04-07T16:24:15.021 INFO:tasks.rados.rados.0.smithi100.stdout:2286: oid 3 updating version 1098 to 1099 2024-04-07T16:24:15.021 INFO:tasks.rados.rados.0.smithi100.stdout:2286: oid 3 updating version 1099 to 1101 2024-04-07T16:24:15.021 INFO:tasks.rados.rados.0.smithi100.stdout:2286: oid 3 version 1101 is already newer than 1100 2024-04-07T16:24:15.021 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 3 v 1101 (ObjNum 704 snap 214 seq_num 704) dirty exists 2024-04-07T16:24:15.021 INFO:tasks.rados.rados.0.smithi100.stdout:2286: left oid 3 (ObjNum 704 snap 214 seq_num 704) 2024-04-07T16:24:15.021 INFO:tasks.rados.rados.0.smithi100.stdout:2288: finishing write tid 1 to smithi10066867-7 2024-04-07T16:24:15.021 INFO:tasks.rados.rados.0.smithi100.stdout:2288: finishing write tid 2 to smithi10066867-7 2024-04-07T16:24:15.021 INFO:tasks.rados.rados.0.smithi100.stdout:2288: finishing write tid 3 to smithi10066867-7 2024-04-07T16:24:15.021 INFO:tasks.rados.rados.0.smithi100.stdout:2288: finishing write tid 4 to smithi10066867-7 2024-04-07T16:24:15.021 INFO:tasks.rados.rados.0.smithi100.stdout:2288: finishing write tid 5 to smithi10066867-7 2024-04-07T16:24:15.021 INFO:tasks.rados.rados.0.smithi100.stdout:2288: finishing write tid 6 to smithi10066867-7 2024-04-07T16:24:15.021 INFO:tasks.rados.rados.0.smithi100.stdout:2288: oid 7 updating version 0 to 1096 2024-04-07T16:24:15.021 INFO:tasks.rados.rados.0.smithi100.stdout:2288: oid 7 updating version 1096 to 1097 2024-04-07T16:24:15.021 INFO:tasks.rados.rados.0.smithi100.stdout:2288: oid 7 updating version 1097 to 1098 2024-04-07T16:24:15.021 INFO:tasks.rados.rados.0.smithi100.stdout:2288: oid 7 version 1098 is already newer than 1095 2024-04-07T16:24:15.022 INFO:tasks.rados.rados.0.smithi100.stdout:2288: oid 7 updating version 1098 to 1099 2024-04-07T16:24:15.022 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 7 v 1099 (ObjNum 705 snap 214 seq_num 705) dirty exists 2024-04-07T16:24:15.022 INFO:tasks.rados.rados.0.smithi100.stdout:2288: left oid 7 (ObjNum 705 snap 214 seq_num 705) 2024-04-07T16:24:15.022 INFO:tasks.rados.rados.0.smithi100.stdout:2285: expect (ObjNum 680 snap 209 seq_num 680) 2024-04-07T16:24:15.183 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:14 smithi100 bash[74942]: cluster 2024-04-07T16:24:13.897936+0000 mon.a (mon.0) 658 : cluster [DBG] osdmap e500: 8 total, 8 up, 8 in 2024-04-07T16:24:15.183 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:14 smithi100 bash[76202]: cluster 2024-04-07T16:24:13.897936+0000 mon.a (mon.0) 658 : cluster [DBG] osdmap e500: 8 total, 8 up, 8 in 2024-04-07T16:24:15.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:14 smithi144 bash[66654]: cluster 2024-04-07T16:24:13.897936+0000 mon.a (mon.0) 658 : cluster [DBG] osdmap e500: 8 total, 8 up, 8 in 2024-04-07T16:24:15.893 INFO:tasks.rados.rados.0.smithi100.stdout:2285: done (4 left) 2024-04-07T16:24:15.893 INFO:tasks.rados.rados.0.smithi100.stdout:2286: done (3 left) 2024-04-07T16:24:15.893 INFO:tasks.rados.rados.0.smithi100.stdout:2287: done (2 left) 2024-04-07T16:24:15.893 INFO:tasks.rados.rados.0.smithi100.stdout:2288: done (1 left) 2024-04-07T16:24:15.893 INFO:tasks.rados.rados.0.smithi100.stdout:2289: done (0 left) 2024-04-07T16:24:15.893 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 6 current snap is 215 2024-04-07T16:24:15.893 INFO:tasks.rados.rados.0.smithi100.stdout:2290: seq_num 706 ranges {2105344=630784,2736128=516096,3252224=729088,3981312=360448} 2024-04-07T16:24:15.910 INFO:tasks.rados.rados.0.smithi100.stdout:2290: writing smithi10066867-6 from 2105344 to 2736128 tid 1 2024-04-07T16:24:15.914 INFO:tasks.rados.rados.0.smithi100.stdout:2290: writing smithi10066867-6 from 2736128 to 3252224 tid 2 2024-04-07T16:24:15.919 INFO:tasks.rados.rados.0.smithi100.stdout:2290: writing smithi10066867-6 from 3252224 to 3981312 tid 3 2024-04-07T16:24:15.922 INFO:tasks.rados.rados.0.smithi100.stdout:2290: writing smithi10066867-6 from 3981312 to 4341760 tid 4 2024-04-07T16:24:15.922 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 36 current snap is 215 2024-04-07T16:24:15.922 INFO:tasks.rados.rados.0.smithi100.stdout:2291: seq_num 707 ranges {3309568=647168,3956736=802816,4759552=532480,5292032=270336} 2024-04-07T16:24:15.947 INFO:tasks.rados.rados.0.smithi100.stdout:2291: writing smithi10066867-36 from 3309568 to 3956736 tid 1 2024-04-07T16:24:15.953 INFO:tasks.rados.rados.0.smithi100.stdout:2291: writing smithi10066867-36 from 3956736 to 4759552 tid 2 2024-04-07T16:24:15.956 INFO:tasks.rados.rados.0.smithi100.stdout:2291: writing smithi10066867-36 from 4759552 to 5292032 tid 3 2024-04-07T16:24:15.958 INFO:tasks.rados.rados.0.smithi100.stdout:2291: writing smithi10066867-36 from 5292032 to 5562368 tid 4 2024-04-07T16:24:15.959 INFO:tasks.rados.rados.0.smithi100.stdout:2292: snap_create 2024-04-07T16:24:15.959 INFO:tasks.rados.rados.0.smithi100.stdout:2290: finishing write tid 1 to smithi10066867-6 2024-04-07T16:24:15.959 INFO:tasks.rados.rados.0.smithi100.stdout:2290: finishing write tid 2 to smithi10066867-6 2024-04-07T16:24:15.959 INFO:tasks.rados.rados.0.smithi100.stdout:2290: finishing write tid 3 to smithi10066867-6 2024-04-07T16:24:15.959 INFO:tasks.rados.rados.0.smithi100.stdout:2290: finishing write tid 4 to smithi10066867-6 2024-04-07T16:24:15.959 INFO:tasks.rados.rados.0.smithi100.stdout:2290: finishing write tid 5 to smithi10066867-6 2024-04-07T16:24:15.959 INFO:tasks.rados.rados.0.smithi100.stdout:2290: finishing write tid 6 to smithi10066867-6 2024-04-07T16:24:15.959 INFO:tasks.rados.rados.0.smithi100.stdout:2290: oid 6 updating version 0 to 1122 2024-04-07T16:24:15.959 INFO:tasks.rados.rados.0.smithi100.stdout:2290: oid 6 updating version 1122 to 1125 2024-04-07T16:24:15.959 INFO:tasks.rados.rados.0.smithi100.stdout:2290: oid 6 version 1125 is already newer than 1124 2024-04-07T16:24:15.959 INFO:tasks.rados.rados.0.smithi100.stdout:2290: oid 6 version 1125 is already newer than 1123 2024-04-07T16:24:15.959 INFO:tasks.rados.rados.0.smithi100.stdout:2290: oid 6 version 1125 is already newer than 1121 2024-04-07T16:24:15.959 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 6 v 1125 (ObjNum 706 snap 215 seq_num 706) dirty exists 2024-04-07T16:24:15.959 INFO:tasks.rados.rados.0.smithi100.stdout:2290: left oid 6 (ObjNum 706 snap 215 seq_num 706) 2024-04-07T16:24:15.959 INFO:tasks.rados.rados.0.smithi100.stdout:2291: finishing write tid 1 to smithi10066867-36 2024-04-07T16:24:15.961 INFO:tasks.rados.rados.0.smithi100.stdout:2291: finishing write tid 2 to smithi10066867-36 2024-04-07T16:24:15.965 INFO:tasks.rados.rados.0.smithi100.stdout:2291: finishing write tid 3 to smithi10066867-36 2024-04-07T16:24:15.967 INFO:tasks.rados.rados.0.smithi100.stdout:2291: finishing write tid 4 to smithi10066867-36 2024-04-07T16:24:15.967 INFO:tasks.rados.rados.0.smithi100.stdout:2291: finishing write tid 5 to smithi10066867-36 2024-04-07T16:24:15.969 INFO:tasks.rados.rados.0.smithi100.stdout:2291: finishing write tid 6 to smithi10066867-36 2024-04-07T16:24:15.969 INFO:tasks.rados.rados.0.smithi100.stdout:2291: oid 36 updating version 0 to 1068 2024-04-07T16:24:15.969 INFO:tasks.rados.rados.0.smithi100.stdout:2291: oid 36 updating version 1068 to 1070 2024-04-07T16:24:15.969 INFO:tasks.rados.rados.0.smithi100.stdout:2291: oid 36 updating version 1070 to 1071 2024-04-07T16:24:15.969 INFO:tasks.rados.rados.0.smithi100.stdout:2291: oid 36 version 1071 is already newer than 1069 2024-04-07T16:24:15.969 INFO:tasks.rados.rados.0.smithi100.stdout:2291: oid 36 updating version 1071 to 1072 2024-04-07T16:24:15.969 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 36 v 1072 (ObjNum 707 snap 215 seq_num 707) dirty exists 2024-04-07T16:24:15.969 INFO:tasks.rados.rados.0.smithi100.stdout:2291: left oid 36 (ObjNum 707 snap 215 seq_num 707) 2024-04-07T16:24:16.183 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:15 smithi100 bash[74942]: cluster 2024-04-07T16:24:14.697015+0000 mgr.y (mgr.25175) 206 : cluster [DBG] pgmap v342: 121 pgs: 8 peering, 9 active+undersized+degraded, 30 active+undersized, 74 active+clean; 347 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 15 MiB/s rd, 2.0 MiB/s wr, 10 op/s; 44/510 objects degraded (8.627%) 2024-04-07T16:24:16.183 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:15 smithi100 bash[74942]: cluster 2024-04-07T16:24:14.889679+0000 mon.a (mon.0) 659 : cluster [WRN] Health check update: Degraded data redundancy: 44/510 objects degraded (8.627%), 9 pgs degraded (PG_DEGRADED) 2024-04-07T16:24:16.183 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:15 smithi100 bash[74942]: cluster 2024-04-07T16:24:14.907025+0000 mon.a (mon.0) 660 : cluster [DBG] osdmap e501: 8 total, 8 up, 8 in 2024-04-07T16:24:16.184 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:15 smithi100 bash[76202]: cluster 2024-04-07T16:24:14.697015+0000 mgr.y (mgr.25175) 206 : cluster [DBG] pgmap v342: 121 pgs: 8 peering, 9 active+undersized+degraded, 30 active+undersized, 74 active+clean; 347 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 15 MiB/s rd, 2.0 MiB/s wr, 10 op/s; 44/510 objects degraded (8.627%) 2024-04-07T16:24:16.184 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:15 smithi100 bash[76202]: cluster 2024-04-07T16:24:14.889679+0000 mon.a (mon.0) 659 : cluster [WRN] Health check update: Degraded data redundancy: 44/510 objects degraded (8.627%), 9 pgs degraded (PG_DEGRADED) 2024-04-07T16:24:16.184 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:15 smithi100 bash[76202]: cluster 2024-04-07T16:24:14.907025+0000 mon.a (mon.0) 660 : cluster [DBG] osdmap e501: 8 total, 8 up, 8 in 2024-04-07T16:24:16.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:15 smithi144 bash[66654]: cluster 2024-04-07T16:24:14.697015+0000 mgr.y (mgr.25175) 206 : cluster [DBG] pgmap v342: 121 pgs: 8 peering, 9 active+undersized+degraded, 30 active+undersized, 74 active+clean; 347 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 15 MiB/s rd, 2.0 MiB/s wr, 10 op/s; 44/510 objects degraded (8.627%) 2024-04-07T16:24:16.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:15 smithi144 bash[66654]: cluster 2024-04-07T16:24:14.889679+0000 mon.a (mon.0) 659 : cluster [WRN] Health check update: Degraded data redundancy: 44/510 objects degraded (8.627%), 9 pgs degraded (PG_DEGRADED) 2024-04-07T16:24:16.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:15 smithi144 bash[66654]: cluster 2024-04-07T16:24:14.907025+0000 mon.a (mon.0) 660 : cluster [DBG] osdmap e501: 8 total, 8 up, 8 in 2024-04-07T16:24:16.920 INFO:tasks.rados.rados.0.smithi100.stdout:2290: done (2 left) 2024-04-07T16:24:16.920 INFO:tasks.rados.rados.0.smithi100.stdout:2291: done (1 left) 2024-04-07T16:24:16.920 INFO:tasks.rados.rados.0.smithi100.stdout:2292: done (0 left) 2024-04-07T16:24:16.920 INFO:tasks.rados.rados.0.smithi100.stdout:2293: rmattr oid 11 current snap is 216 2024-04-07T16:24:16.920 INFO:tasks.rados.rados.0.smithi100.stdout:2294: read oid 20 snap -1 2024-04-07T16:24:16.921 INFO:tasks.rados.rados.0.smithi100.stdout:2294: expect (ObjNum 610 snap 188 seq_num 610) 2024-04-07T16:24:16.921 INFO:tasks.rados.rados.0.smithi100.stdout:2295: snap_remove snap 213 2024-04-07T16:24:17.183 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:16 smithi100 bash[74942]: cluster 2024-04-07T16:24:15.910322+0000 mon.a (mon.0) 661 : cluster [DBG] osdmap e502: 8 total, 8 up, 8 in 2024-04-07T16:24:17.183 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:16 smithi100 bash[74942]: audit 2024-04-07T16:24:16.721347+0000 mon.a (mon.0) 662 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:24:17.184 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:16 smithi100 bash[74942]: audit 2024-04-07T16:24:16.723592+0000 mon.a (mon.0) 663 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:24:17.184 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:16 smithi100 bash[76202]: cluster 2024-04-07T16:24:15.910322+0000 mon.a (mon.0) 661 : cluster [DBG] osdmap e502: 8 total, 8 up, 8 in 2024-04-07T16:24:17.184 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:16 smithi100 bash[76202]: audit 2024-04-07T16:24:16.721347+0000 mon.a (mon.0) 662 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:24:17.184 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:16 smithi100 bash[76202]: audit 2024-04-07T16:24:16.723592+0000 mon.a (mon.0) 663 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:24:17.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:16 smithi144 bash[66654]: cluster 2024-04-07T16:24:15.910322+0000 mon.a (mon.0) 661 : cluster [DBG] osdmap e502: 8 total, 8 up, 8 in 2024-04-07T16:24:17.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:16 smithi144 bash[66654]: audit 2024-04-07T16:24:16.721347+0000 mon.a (mon.0) 662 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:24:17.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:16 smithi144 bash[66654]: audit 2024-04-07T16:24:16.723592+0000 mon.a (mon.0) 663 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:24:17.920 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 11 v 1311 (ObjNum 537 snap 162 seq_num 537) dirty exists 2024-04-07T16:24:17.920 INFO:tasks.rados.rados.0.smithi100.stdout:2294: expect (ObjNum 610 snap 188 seq_num 610) 2024-04-07T16:24:18.183 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:17 smithi100 bash[74942]: cluster 2024-04-07T16:24:16.697817+0000 mgr.y (mgr.25175) 207 : cluster [DBG] pgmap v345: 121 pgs: 8 peering, 6 active+undersized+degraded, 27 active+undersized, 80 active+clean; 339 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 17 MiB/s rd, 2.0 MiB/s wr, 10 op/s; 27/495 objects degraded (5.455%); 3.6 MiB/s, 0 objects/s recovering 2024-04-07T16:24:18.183 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:17 smithi100 bash[74942]: cluster 2024-04-07T16:24:16.938755+0000 mon.a (mon.0) 664 : cluster [DBG] osdmap e503: 8 total, 8 up, 8 in 2024-04-07T16:24:18.184 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:17 smithi100 bash[76202]: cluster 2024-04-07T16:24:16.697817+0000 mgr.y (mgr.25175) 207 : cluster [DBG] pgmap v345: 121 pgs: 8 peering, 6 active+undersized+degraded, 27 active+undersized, 80 active+clean; 339 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 17 MiB/s rd, 2.0 MiB/s wr, 10 op/s; 27/495 objects degraded (5.455%); 3.6 MiB/s, 0 objects/s recovering 2024-04-07T16:24:18.184 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:17 smithi100 bash[76202]: cluster 2024-04-07T16:24:16.938755+0000 mon.a (mon.0) 664 : cluster [DBG] osdmap e503: 8 total, 8 up, 8 in 2024-04-07T16:24:18.234 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:17 smithi144 bash[66654]: cluster 2024-04-07T16:24:16.697817+0000 mgr.y (mgr.25175) 207 : cluster [DBG] pgmap v345: 121 pgs: 8 peering, 6 active+undersized+degraded, 27 active+undersized, 80 active+clean; 339 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 17 MiB/s rd, 2.0 MiB/s wr, 10 op/s; 27/495 objects degraded (5.455%); 3.6 MiB/s, 0 objects/s recovering 2024-04-07T16:24:18.234 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:17 smithi144 bash[66654]: cluster 2024-04-07T16:24:16.938755+0000 mon.a (mon.0) 664 : cluster [DBG] osdmap e503: 8 total, 8 up, 8 in 2024-04-07T16:24:18.683 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:24:18 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:24:18] "GET /metrics HTTP/1.1" 200 36074 "" "Prometheus/2.43.0" 2024-04-07T16:24:18.722 INFO:tasks.rados.rados.0.smithi100.stdout:2293: done (2 left) 2024-04-07T16:24:18.722 INFO:tasks.rados.rados.0.smithi100.stdout:2294: done (1 left) 2024-04-07T16:24:18.722 INFO:tasks.rados.rados.0.smithi100.stdout:2295: done (0 left) 2024-04-07T16:24:18.723 INFO:tasks.rados.rados.0.smithi100.stdout:2296: copy_from oid 11 from oid 22 current snap is 216 2024-04-07T16:24:18.723 INFO:tasks.rados.rados.0.smithi100.stdout:2297: snap_create 2024-04-07T16:24:18.736 INFO:tasks.rados.rados.0.smithi100.stdout:2296: finishing copy_from racing read to smithi10066867-11 2024-04-07T16:24:18.739 INFO:tasks.rados.rados.0.smithi100.stdout:2296: finishing copy_from to smithi10066867-11 2024-04-07T16:24:18.739 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 11 v 1101 (ObjNum 664 snap 204 seq_num 664) dirty exists 2024-04-07T16:24:18.943 INFO:tasks.rados.rados.0.smithi100.stdout:2296: done (1 left) 2024-04-07T16:24:18.943 INFO:tasks.rados.rados.0.smithi100.stdout:2297: done (0 left) 2024-04-07T16:24:18.943 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 40 current snap is 217 2024-04-07T16:24:18.943 INFO:tasks.rados.rados.0.smithi100.stdout:2298: seq_num 708 ranges {5980160=663552,6643712=737280,7380992=696320,8077312=172032} 2024-04-07T16:24:19.020 INFO:tasks.rados.rados.0.smithi100.stdout:2298: writing smithi10066867-40 from 5980160 to 6643712 tid 1 2024-04-07T16:24:19.026 INFO:tasks.rados.rados.0.smithi100.stdout:2298: writing smithi10066867-40 from 6643712 to 7380992 tid 2 2024-04-07T16:24:19.031 INFO:tasks.rados.rados.0.smithi100.stdout:2298: writing smithi10066867-40 from 7380992 to 8077312 tid 3 2024-04-07T16:24:19.032 INFO:tasks.rados.rados.0.smithi100.stdout:2298: writing smithi10066867-40 from 8077312 to 8249344 tid 4 2024-04-07T16:24:19.032 INFO:tasks.rados.rados.0.smithi100.stdout:2299: setattr oid 23 current snap is 217 2024-04-07T16:24:19.033 INFO:tasks.rados.rados.0.smithi100.stdout:2300: delete oid 21 current snap is 217 2024-04-07T16:24:19.040 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 23 v 1316 (ObjNum 596 snap 183 seq_num 596) dirty exists 2024-04-07T16:24:19.040 INFO:tasks.rados.rados.0.smithi100.stdout:2298: finishing write tid 1 to smithi10066867-40 2024-04-07T16:24:19.043 INFO:tasks.rados.rados.0.smithi100.stdout:2298: finishing write tid 2 to smithi10066867-40 2024-04-07T16:24:19.044 INFO:tasks.rados.rados.0.smithi100.stdout:2299: done (2 left) 2024-04-07T16:24:19.044 INFO:tasks.rados.rados.0.smithi100.stdout:2300: done (1 left) 2024-04-07T16:24:19.044 INFO:tasks.rados.rados.0.smithi100.stdout:2301: read oid 18 snap -1 2024-04-07T16:24:19.044 INFO:tasks.rados.rados.0.smithi100.stdout:2301: expect (ObjNum 620 snap 194 seq_num 620) 2024-04-07T16:24:19.044 INFO:tasks.rados.rados.0.smithi100.stdout:2302: rollback oid 44 current snap is 217 2024-04-07T16:24:19.044 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 44 to 210 2024-04-07T16:24:19.044 INFO:tasks.rados.rados.0.smithi100.stdout:2303: setattr oid 32 current snap is 217 2024-04-07T16:24:19.045 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 13 current snap is 217 2024-04-07T16:24:19.045 INFO:tasks.rados.rados.0.smithi100.stdout:2304: seq_num 710 ranges {0=696320,696320=499712,1196032=712704,1908736=393216} 2024-04-07T16:24:19.050 INFO:tasks.rados.rados.0.smithi100.stdout:2304: writing smithi10066867-13 from 0 to 696320 tid 1 2024-04-07T16:24:19.054 INFO:tasks.rados.rados.0.smithi100.stdout:2304: writing smithi10066867-13 from 696320 to 1196032 tid 2 2024-04-07T16:24:19.060 INFO:tasks.rados.rados.0.smithi100.stdout:2304: writing smithi10066867-13 from 1196032 to 1908736 tid 3 2024-04-07T16:24:19.064 INFO:tasks.rados.rados.0.smithi100.stdout:2304: writing smithi10066867-13 from 1908736 to 2301952 tid 4 2024-04-07T16:24:19.064 INFO:tasks.rados.rados.0.smithi100.stdout:2305: copy_from oid 17 from oid 3 current snap is 217 2024-04-07T16:24:19.064 INFO:tasks.rados.rados.0.smithi100.stdout:2298: finishing write tid 3 to smithi10066867-40 2024-04-07T16:24:19.065 INFO:tasks.rados.rados.0.smithi100.stdout:2298: finishing write tid 4 to smithi10066867-40 2024-04-07T16:24:19.065 INFO:tasks.rados.rados.0.smithi100.stdout:2298: finishing write tid 5 to smithi10066867-40 2024-04-07T16:24:19.065 INFO:tasks.rados.rados.0.smithi100.stdout:2298: finishing write tid 6 to smithi10066867-40 2024-04-07T16:24:19.065 INFO:tasks.rados.rados.0.smithi100.stdout:2298: oid 40 updating version 0 to 1128 2024-04-07T16:24:19.065 INFO:tasks.rados.rados.0.smithi100.stdout:2298: oid 40 version 1128 is already newer than 1127 2024-04-07T16:24:19.065 INFO:tasks.rados.rados.0.smithi100.stdout:2298: oid 40 version 1128 is already newer than 1126 2024-04-07T16:24:19.065 INFO:tasks.rados.rados.0.smithi100.stdout:2298: oid 40 updating version 1128 to 1129 2024-04-07T16:24:19.065 INFO:tasks.rados.rados.0.smithi100.stdout:2298: oid 40 updating version 1129 to 1130 2024-04-07T16:24:19.065 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 40 v 1130 (ObjNum 708 snap 217 seq_num 708) dirty exists 2024-04-07T16:24:19.065 INFO:tasks.rados.rados.0.smithi100.stdout:2298: left oid 40 (ObjNum 708 snap 217 seq_num 708) 2024-04-07T16:24:19.065 INFO:tasks.rados.rados.0.smithi100.stdout:2302: finishing rollback tid 1 to smithi10066867-44 2024-04-07T16:24:19.065 INFO:tasks.rados.rados.0.smithi100.stdout:2301: expect (ObjNum 620 snap 194 seq_num 620) 2024-04-07T16:24:19.090 INFO:tasks.rados.rados.0.smithi100.stdout:2302: finishing rollback tid 2 to smithi10066867-44 2024-04-07T16:24:19.090 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 44 v 1132 (ObjNum 675 snap 206 seq_num 675) dirty exists 2024-04-07T16:24:19.090 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 32 v 1094 (ObjNum 630 snap 196 seq_num 630) dirty exists 2024-04-07T16:24:19.091 INFO:tasks.rados.rados.0.smithi100.stdout:2304: finishing write tid 1 to smithi10066867-13 2024-04-07T16:24:19.091 INFO:tasks.rados.rados.0.smithi100.stdout:2298: done (5 left) 2024-04-07T16:24:19.091 INFO:tasks.rados.rados.0.smithi100.stdout:2301: done (4 left) 2024-04-07T16:24:19.091 INFO:tasks.rados.rados.0.smithi100.stdout:2302: done (3 left) 2024-04-07T16:24:19.091 INFO:tasks.rados.rados.0.smithi100.stdout:2303: done (2 left) 2024-04-07T16:24:19.091 INFO:tasks.rados.rados.0.smithi100.stdout:2306: snap_remove snap 214 2024-04-07T16:24:19.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:18 smithi144 bash[66654]: cluster 2024-04-07T16:24:17.937228+0000 mon.a (mon.0) 665 : cluster [DBG] osdmap e504: 8 total, 8 up, 8 in 2024-04-07T16:24:19.433 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:18 smithi100 bash[74942]: cluster 2024-04-07T16:24:17.937228+0000 mon.a (mon.0) 665 : cluster [DBG] osdmap e504: 8 total, 8 up, 8 in 2024-04-07T16:24:19.434 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:18 smithi100 bash[76202]: cluster 2024-04-07T16:24:17.937228+0000 mon.a (mon.0) 665 : cluster [DBG] osdmap e504: 8 total, 8 up, 8 in 2024-04-07T16:24:19.955 INFO:tasks.rados.rados.0.smithi100.stdout:2306: done (2 left) 2024-04-07T16:24:19.955 INFO:tasks.rados.rados.0.smithi100.stdout:2307: snap_create 2024-04-07T16:24:19.956 INFO:tasks.rados.rados.0.smithi100.stdout:2304: finishing write tid 2 to smithi10066867-13 2024-04-07T16:24:19.956 INFO:tasks.rados.rados.0.smithi100.stdout:2304: finishing write tid 3 to smithi10066867-13 2024-04-07T16:24:19.956 INFO:tasks.rados.rados.0.smithi100.stdout:2304: finishing write tid 4 to smithi10066867-13 2024-04-07T16:24:19.956 INFO:tasks.rados.rados.0.smithi100.stdout:2304: finishing write tid 5 to smithi10066867-13 2024-04-07T16:24:19.956 INFO:tasks.rados.rados.0.smithi100.stdout:2304: finishing write tid 6 to smithi10066867-13 2024-04-07T16:24:19.956 INFO:tasks.rados.rados.0.smithi100.stdout:2304: oid 13 updating version 0 to 976 2024-04-07T16:24:19.956 INFO:tasks.rados.rados.0.smithi100.stdout:2304: oid 13 version 976 is already newer than 973 2024-04-07T16:24:19.956 INFO:tasks.rados.rados.0.smithi100.stdout:2304: oid 13 version 976 is already newer than 975 2024-04-07T16:24:19.956 INFO:tasks.rados.rados.0.smithi100.stdout:2304: oid 13 version 976 is already newer than 972 2024-04-07T16:24:19.956 INFO:tasks.rados.rados.0.smithi100.stdout:2304: oid 13 version 976 is already newer than 974 2024-04-07T16:24:19.956 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 13 v 976 (ObjNum 710 snap 217 seq_num 710) dirty exists 2024-04-07T16:24:19.956 INFO:tasks.rados.rados.0.smithi100.stdout:2304: left oid 13 (ObjNum 710 snap 217 seq_num 710) 2024-04-07T16:24:19.956 INFO:tasks.rados.rados.0.smithi100.stdout:2305: finishing copy_from racing read to smithi10066867-17 2024-04-07T16:24:19.956 INFO:tasks.rados.rados.0.smithi100.stdout:2305: finishing copy_from to smithi10066867-17 2024-04-07T16:24:19.956 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 17 v 1101 (ObjNum 704 snap 214 seq_num 704) dirty exists 2024-04-07T16:24:20.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:19 smithi144 bash[66654]: cluster 2024-04-07T16:24:18.698613+0000 mgr.y (mgr.25175) 208 : cluster [DBG] pgmap v348: 121 pgs: 1 active+clean+snaptrim, 2 peering, 118 active+clean; 335 MiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 14 MiB/s rd, 538 KiB/s wr, 5 op/s; 5.9 MiB/s, 1 objects/s recovering 2024-04-07T16:24:20.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:19 smithi144 bash[66654]: cluster 2024-04-07T16:24:18.940723+0000 mon.a (mon.0) 666 : cluster [INF] Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 27/495 objects degraded (5.455%), 6 pgs degraded) 2024-04-07T16:24:20.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:19 smithi144 bash[66654]: cluster 2024-04-07T16:24:18.940778+0000 mon.a (mon.0) 667 : cluster [INF] Cluster is now healthy 2024-04-07T16:24:20.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:19 smithi144 bash[66654]: cluster 2024-04-07T16:24:18.961629+0000 mon.a (mon.0) 668 : cluster [DBG] osdmap e505: 8 total, 8 up, 8 in 2024-04-07T16:24:20.433 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:19 smithi100 bash[74942]: cluster 2024-04-07T16:24:18.698613+0000 mgr.y (mgr.25175) 208 : cluster [DBG] pgmap v348: 121 pgs: 1 active+clean+snaptrim, 2 peering, 118 active+clean; 335 MiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 14 MiB/s rd, 538 KiB/s wr, 5 op/s; 5.9 MiB/s, 1 objects/s recovering 2024-04-07T16:24:20.433 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:19 smithi100 bash[74942]: cluster 2024-04-07T16:24:18.940723+0000 mon.a (mon.0) 666 : cluster [INF] Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 27/495 objects degraded (5.455%), 6 pgs degraded) 2024-04-07T16:24:20.434 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:19 smithi100 bash[74942]: cluster 2024-04-07T16:24:18.940778+0000 mon.a (mon.0) 667 : cluster [INF] Cluster is now healthy 2024-04-07T16:24:20.434 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:19 smithi100 bash[74942]: cluster 2024-04-07T16:24:18.961629+0000 mon.a (mon.0) 668 : cluster [DBG] osdmap e505: 8 total, 8 up, 8 in 2024-04-07T16:24:20.434 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:19 smithi100 bash[76202]: cluster 2024-04-07T16:24:18.698613+0000 mgr.y (mgr.25175) 208 : cluster [DBG] pgmap v348: 121 pgs: 1 active+clean+snaptrim, 2 peering, 118 active+clean; 335 MiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 14 MiB/s rd, 538 KiB/s wr, 5 op/s; 5.9 MiB/s, 1 objects/s recovering 2024-04-07T16:24:20.434 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:19 smithi100 bash[76202]: cluster 2024-04-07T16:24:18.940723+0000 mon.a (mon.0) 666 : cluster [INF] Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 27/495 objects degraded (5.455%), 6 pgs degraded) 2024-04-07T16:24:20.434 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:19 smithi100 bash[76202]: cluster 2024-04-07T16:24:18.940778+0000 mon.a (mon.0) 667 : cluster [INF] Cluster is now healthy 2024-04-07T16:24:20.434 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:19 smithi100 bash[76202]: cluster 2024-04-07T16:24:18.961629+0000 mon.a (mon.0) 668 : cluster [DBG] osdmap e505: 8 total, 8 up, 8 in 2024-04-07T16:24:20.960 INFO:tasks.rados.rados.0.smithi100.stdout:2304: done (2 left) 2024-04-07T16:24:20.960 INFO:tasks.rados.rados.0.smithi100.stdout:2305: done (1 left) 2024-04-07T16:24:20.960 INFO:tasks.rados.rados.0.smithi100.stdout:2307: done (0 left) 2024-04-07T16:24:20.960 INFO:tasks.rados.rados.0.smithi100.stdout:2308: rollback oid 29 current snap is 218 2024-04-07T16:24:20.960 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 29 to 210 2024-04-07T16:24:20.961 INFO:tasks.rados.rados.0.smithi100.stdout:2309: snap_remove snap 217 2024-04-07T16:24:21.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:20 smithi144 bash[66654]: cluster 2024-04-07T16:24:19.973302+0000 mon.a (mon.0) 669 : cluster [DBG] osdmap e506: 8 total, 8 up, 8 in 2024-04-07T16:24:21.433 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:20 smithi100 bash[74942]: cluster 2024-04-07T16:24:19.973302+0000 mon.a (mon.0) 669 : cluster [DBG] osdmap e506: 8 total, 8 up, 8 in 2024-04-07T16:24:21.433 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:20 smithi100 bash[76202]: cluster 2024-04-07T16:24:19.973302+0000 mon.a (mon.0) 669 : cluster [DBG] osdmap e506: 8 total, 8 up, 8 in 2024-04-07T16:24:21.829 INFO:tasks.rados.rados.0.smithi100.stdout:2308: finishing rollback tid 0 to smithi10066867-29 2024-04-07T16:24:21.829 INFO:tasks.rados.rados.0.smithi100.stdout:2308: finishing rollback tid 1 to smithi10066867-29 2024-04-07T16:24:21.829 INFO:tasks.rados.rados.0.smithi100.stdout:2308: finishing rollback tid 2 to smithi10066867-29 2024-04-07T16:24:21.829 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 29 v 1116 (ObjNum 567 snap 174 seq_num 567) dirty exists 2024-04-07T16:24:21.829 INFO:tasks.rados.rados.0.smithi100.stdout:2308: done (1 left) 2024-04-07T16:24:21.829 INFO:tasks.rados.rados.0.smithi100.stdout:2309: done (0 left) 2024-04-07T16:24:21.829 INFO:tasks.rados.rados.0.smithi100.stdout:2310: rollback oid 44 current snap is 218 2024-04-07T16:24:21.829 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 44 to 186 2024-04-07T16:24:21.829 INFO:tasks.rados.rados.0.smithi100.stdout:2311: delete oid 11 current snap is 218 2024-04-07T16:24:21.841 INFO:tasks.rados.rados.0.smithi100.stdout:2310: finishing rollback tid 0 to smithi10066867-44 2024-04-07T16:24:21.842 INFO:tasks.rados.rados.0.smithi100.stdout:2311: done (1 left) 2024-04-07T16:24:21.842 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 49 current snap is 218 2024-04-07T16:24:21.842 INFO:tasks.rados.rados.0.smithi100.stdout:2312: seq_num 712 ranges {2318336=729088,3047424=671744,3719168=671744,4390912=262144} 2024-04-07T16:24:21.877 INFO:tasks.rados.rados.0.smithi100.stdout:2312: writing smithi10066867-49 from 2318336 to 3047424 tid 1 2024-04-07T16:24:21.887 INFO:tasks.rados.rados.0.smithi100.stdout:2312: writing smithi10066867-49 from 3047424 to 3719168 tid 2 2024-04-07T16:24:21.896 INFO:tasks.rados.rados.0.smithi100.stdout:2312: writing smithi10066867-49 from 3719168 to 4390912 tid 3 2024-04-07T16:24:21.899 INFO:tasks.rados.rados.0.smithi100.stdout:2312: writing smithi10066867-49 from 4390912 to 4653056 tid 4 2024-04-07T16:24:21.899 INFO:tasks.rados.rados.0.smithi100.stdout:2313: rmattr oid 12 current snap is 218 2024-04-07T16:24:21.899 INFO:tasks.rados.rados.0.smithi100.stdout:2313: done (2 left) 2024-04-07T16:24:21.899 INFO:tasks.rados.rados.0.smithi100.stdout:2314: rmattr oid 10 current snap is 218 2024-04-07T16:24:21.899 INFO:tasks.rados.rados.0.smithi100.stdout:2314: done (2 left) 2024-04-07T16:24:21.899 INFO:tasks.rados.rados.0.smithi100.stdout:2315: copy_from oid 23 from oid 36 current snap is 218 2024-04-07T16:24:21.899 INFO:tasks.rados.rados.0.smithi100.stdout:2310: finishing rollback tid 1 to smithi10066867-44 2024-04-07T16:24:21.899 INFO:tasks.rados.rados.0.smithi100.stdout:2310: finishing rollback tid 2 to smithi10066867-44 2024-04-07T16:24:21.899 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 44 v 1135 (ObjNum 509 snap 152 seq_num 509) dirty exists 2024-04-07T16:24:21.899 INFO:tasks.rados.rados.0.smithi100.stdout:2312: finishing write tid 1 to smithi10066867-49 2024-04-07T16:24:21.900 INFO:tasks.rados.rados.0.smithi100.stdout:2310: done (2 left) 2024-04-07T16:24:21.900 INFO:tasks.rados.rados.0.smithi100.stdout:2316: copy_from oid 42 from oid 41 current snap is 218 2024-04-07T16:24:21.900 INFO:tasks.rados.rados.0.smithi100.stdout:2312: finishing write tid 2 to smithi10066867-49 2024-04-07T16:24:21.900 INFO:tasks.rados.rados.0.smithi100.stdout:2317: snap_remove snap 210 2024-04-07T16:24:22.305 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:21 smithi144 bash[66654]: cluster 2024-04-07T16:24:20.699363+0000 mgr.y (mgr.25175) 209 : cluster [DBG] pgmap v351: 121 pgs: 1 active+clean+snaptrim, 120 active+clean; 340 MiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 18 MiB/s rd, 2.8 MiB/s wr, 11 op/s; 6.6 MiB/s, 2 objects/s recovering 2024-04-07T16:24:22.305 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:21 smithi144 bash[66654]: cluster 2024-04-07T16:24:20.979127+0000 mon.a (mon.0) 670 : cluster [DBG] osdmap e507: 8 total, 8 up, 8 in 2024-04-07T16:24:22.305 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:21 smithi144 bash[66654]: audit 2024-04-07T16:24:21.299585+0000 mon.a (mon.0) 671 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:24:22.305 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:21 smithi144 bash[66654]: audit 2024-04-07T16:24:21.311856+0000 mon.a (mon.0) 672 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:24:22.305 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:21 smithi144 bash[66654]: audit 2024-04-07T16:24:21.314455+0000 mon.a (mon.0) 673 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:24:22.306 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:21 smithi144 bash[66654]: audit 2024-04-07T16:24:21.316465+0000 mon.a (mon.0) 674 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:24:22.306 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:21 smithi144 bash[66654]: audit 2024-04-07T16:24:21.326863+0000 mon.a (mon.0) 675 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:24:22.306 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:21 smithi144 bash[66654]: audit 2024-04-07T16:24:21.425267+0000 mon.a (mon.0) 676 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:24:22.306 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:21 smithi144 bash[66654]: audit 2024-04-07T16:24:21.430949+0000 mon.a (mon.0) 677 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:24:22.306 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:21 smithi144 bash[66654]: audit 2024-04-07T16:24:21.435274+0000 mon.a (mon.0) 678 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:24:22.306 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:21 smithi144 bash[66654]: audit 2024-04-07T16:24:21.438185+0000 mon.a (mon.0) 679 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:24:22.307 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:21 smithi144 bash[66654]: audit 2024-04-07T16:24:21.441970+0000 mon.a (mon.0) 680 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-04-07T16:24:22.307 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:21 smithi144 bash[66654]: cluster 2024-04-07T16:24:21.843258+0000 mon.a (mon.0) 681 : cluster [DBG] osdmap e508: 8 total, 8 up, 8 in 2024-04-07T16:24:22.434 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:21 smithi100 bash[74942]: cluster 2024-04-07T16:24:20.699363+0000 mgr.y (mgr.25175) 209 : cluster [DBG] pgmap v351: 121 pgs: 1 active+clean+snaptrim, 120 active+clean; 340 MiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 18 MiB/s rd, 2.8 MiB/s wr, 11 op/s; 6.6 MiB/s, 2 objects/s recovering 2024-04-07T16:24:22.434 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:21 smithi100 bash[74942]: cluster 2024-04-07T16:24:20.979127+0000 mon.a (mon.0) 670 : cluster [DBG] osdmap e507: 8 total, 8 up, 8 in 2024-04-07T16:24:22.434 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:21 smithi100 bash[74942]: audit 2024-04-07T16:24:21.299585+0000 mon.a (mon.0) 671 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:24:22.434 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:21 smithi100 bash[74942]: audit 2024-04-07T16:24:21.311856+0000 mon.a (mon.0) 672 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:24:22.434 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:21 smithi100 bash[74942]: audit 2024-04-07T16:24:21.314455+0000 mon.a (mon.0) 673 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:24:22.435 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:21 smithi100 bash[74942]: audit 2024-04-07T16:24:21.316465+0000 mon.a (mon.0) 674 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:24:22.435 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:21 smithi100 bash[74942]: audit 2024-04-07T16:24:21.326863+0000 mon.a (mon.0) 675 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:24:22.435 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:21 smithi100 bash[74942]: audit 2024-04-07T16:24:21.425267+0000 mon.a (mon.0) 676 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:24:22.435 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:21 smithi100 bash[74942]: audit 2024-04-07T16:24:21.430949+0000 mon.a (mon.0) 677 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:24:22.435 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:21 smithi100 bash[74942]: audit 2024-04-07T16:24:21.435274+0000 mon.a (mon.0) 678 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:24:22.435 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:21 smithi100 bash[74942]: audit 2024-04-07T16:24:21.438185+0000 mon.a (mon.0) 679 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:24:22.436 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:21 smithi100 bash[74942]: audit 2024-04-07T16:24:21.441970+0000 mon.a (mon.0) 680 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-04-07T16:24:22.436 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:21 smithi100 bash[74942]: cluster 2024-04-07T16:24:21.843258+0000 mon.a (mon.0) 681 : cluster [DBG] osdmap e508: 8 total, 8 up, 8 in 2024-04-07T16:24:22.436 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:21 smithi100 bash[76202]: cluster 2024-04-07T16:24:20.699363+0000 mgr.y (mgr.25175) 209 : cluster [DBG] pgmap v351: 121 pgs: 1 active+clean+snaptrim, 120 active+clean; 340 MiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 18 MiB/s rd, 2.8 MiB/s wr, 11 op/s; 6.6 MiB/s, 2 objects/s recovering 2024-04-07T16:24:22.436 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:21 smithi100 bash[76202]: cluster 2024-04-07T16:24:20.979127+0000 mon.a (mon.0) 670 : cluster [DBG] osdmap e507: 8 total, 8 up, 8 in 2024-04-07T16:24:22.436 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:21 smithi100 bash[76202]: audit 2024-04-07T16:24:21.299585+0000 mon.a (mon.0) 671 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:24:22.436 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:21 smithi100 bash[76202]: audit 2024-04-07T16:24:21.311856+0000 mon.a (mon.0) 672 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:24:22.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:21 smithi100 bash[76202]: audit 2024-04-07T16:24:21.314455+0000 mon.a (mon.0) 673 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:24:22.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:21 smithi100 bash[76202]: audit 2024-04-07T16:24:21.316465+0000 mon.a (mon.0) 674 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:24:22.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:21 smithi100 bash[76202]: audit 2024-04-07T16:24:21.326863+0000 mon.a (mon.0) 675 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:24:22.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:21 smithi100 bash[76202]: audit 2024-04-07T16:24:21.425267+0000 mon.a (mon.0) 676 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:24:22.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:21 smithi100 bash[76202]: audit 2024-04-07T16:24:21.430949+0000 mon.a (mon.0) 677 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:24:22.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:21 smithi100 bash[76202]: audit 2024-04-07T16:24:21.435274+0000 mon.a (mon.0) 678 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:24:22.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:21 smithi100 bash[76202]: audit 2024-04-07T16:24:21.438185+0000 mon.a (mon.0) 679 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:24:22.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:21 smithi100 bash[76202]: audit 2024-04-07T16:24:21.441970+0000 mon.a (mon.0) 680 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-04-07T16:24:22.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:21 smithi100 bash[76202]: cluster 2024-04-07T16:24:21.843258+0000 mon.a (mon.0) 681 : cluster [DBG] osdmap e508: 8 total, 8 up, 8 in 2024-04-07T16:24:22.606 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:24:22 smithi144 bash[59794]: ts=2024-04-07T16:24:22.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:24:22.845 INFO:tasks.rados.rados.0.smithi100.stdout:2317: done (3 left) 2024-04-07T16:24:22.845 INFO:tasks.rados.rados.0.smithi100.stdout:2318: rollback oid 27 current snap is 218 2024-04-07T16:24:22.846 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 27 to 178 2024-04-07T16:24:22.846 INFO:tasks.rados.rados.0.smithi100.stdout:2312: finishing write tid 3 to smithi10066867-49 2024-04-07T16:24:22.846 INFO:tasks.rados.rados.0.smithi100.stdout:2312: finishing write tid 4 to smithi10066867-49 2024-04-07T16:24:22.846 INFO:tasks.rados.rados.0.smithi100.stdout:2312: finishing write tid 5 to smithi10066867-49 2024-04-07T16:24:22.846 INFO:tasks.rados.rados.0.smithi100.stdout:2312: finishing write tid 6 to smithi10066867-49 2024-04-07T16:24:22.846 INFO:tasks.rados.rados.0.smithi100.stdout:2312: oid 49 updating version 0 to 1119 2024-04-07T16:24:22.847 INFO:tasks.rados.rados.0.smithi100.stdout:2312: oid 49 version 1119 is already newer than 1118 2024-04-07T16:24:22.847 INFO:tasks.rados.rados.0.smithi100.stdout:2312: oid 49 updating version 1119 to 1120 2024-04-07T16:24:22.847 INFO:tasks.rados.rados.0.smithi100.stdout:2312: oid 49 version 1120 is already newer than 1117 2024-04-07T16:24:22.847 INFO:tasks.rados.rados.0.smithi100.stdout:2312: oid 49 updating version 1120 to 1121 2024-04-07T16:24:22.847 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 49 v 1121 (ObjNum 712 snap 218 seq_num 712) dirty exists 2024-04-07T16:24:22.847 INFO:tasks.rados.rados.0.smithi100.stdout:2312: left oid 49 (ObjNum 712 snap 218 seq_num 712) 2024-04-07T16:24:22.848 INFO:tasks.rados.rados.0.smithi100.stdout:2316: finishing copy_from racing read to smithi10066867-42 2024-04-07T16:24:22.848 INFO:tasks.rados.rados.0.smithi100.stdout:2316: finishing copy_from to smithi10066867-42 2024-04-07T16:24:22.848 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 42 v 823 (ObjNum 684 snap 210 seq_num 684) dirty exists 2024-04-07T16:24:22.848 INFO:tasks.rados.rados.0.smithi100.stdout:2315: finishing copy_from racing read to smithi10066867-23 2024-04-07T16:24:22.848 INFO:tasks.rados.rados.0.smithi100.stdout:2315: finishing copy_from to smithi10066867-23 2024-04-07T16:24:22.848 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 23 v 1072 (ObjNum 707 snap 215 seq_num 707) dirty exists 2024-04-07T16:24:22.849 INFO:tasks.rados.rados.0.smithi100.stdout:2312: done (3 left) 2024-04-07T16:24:22.849 INFO:tasks.rados.rados.0.smithi100.stdout:2315: done (2 left) 2024-04-07T16:24:22.849 INFO:tasks.rados.rados.0.smithi100.stdout:2316: done (1 left) 2024-04-07T16:24:22.849 INFO:tasks.rados.rados.0.smithi100.stdout:2319: rollback oid 9 current snap is 218 2024-04-07T16:24:22.849 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 9 to 215 2024-04-07T16:24:22.849 INFO:tasks.rados.rados.0.smithi100.stdout:2320: read oid 48 snap -1 2024-04-07T16:24:22.850 INFO:tasks.rados.rados.0.smithi100.stdout:2320: expect (ObjNum 700 snap 213 seq_num 700) 2024-04-07T16:24:22.850 INFO:tasks.rados.rados.0.smithi100.stdout:2321: setattr oid 26 current snap is 218 2024-04-07T16:24:22.851 INFO:tasks.rados.rados.0.smithi100.stdout:2322: rollback oid 16 current snap is 218 2024-04-07T16:24:22.852 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 16 to 178 2024-04-07T16:24:22.852 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 36 current snap is 218 2024-04-07T16:24:22.852 INFO:tasks.rados.rados.0.smithi100.stdout:2323: seq_num 713 ranges {5562368=745472,6307840=557056,6864896=483328,7348224=565248} 2024-04-07T16:24:22.892 INFO:tasks.rados.rados.0.smithi100.stdout:2323: writing smithi10066867-36 from 5562368 to 6307840 tid 1 2024-04-07T16:24:22.896 INFO:tasks.rados.rados.0.smithi100.stdout:2323: writing smithi10066867-36 from 6307840 to 6864896 tid 2 2024-04-07T16:24:22.900 INFO:tasks.rados.rados.0.smithi100.stdout:2323: writing smithi10066867-36 from 6864896 to 7348224 tid 3 2024-04-07T16:24:22.905 INFO:tasks.rados.rados.0.smithi100.stdout:2323: writing smithi10066867-36 from 7348224 to 7913472 tid 4 2024-04-07T16:24:22.905 INFO:tasks.rados.rados.0.smithi100.stdout:2324: read oid 35 snap 216 2024-04-07T16:24:22.905 INFO:tasks.rados.rados.0.smithi100.stdout:2324: expect (ObjNum 688 snap 210 seq_num 688) 2024-04-07T16:24:22.905 INFO:tasks.rados.rados.0.smithi100.stdout:2325: copy_from oid 4 from oid 41 current snap is 218 2024-04-07T16:24:22.905 INFO:tasks.rados.rados.0.smithi100.stdout:2319: finishing rollback tid 1 to smithi10066867-9 2024-04-07T16:24:22.905 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 9 v 0 (ObjNum 129 snap 0 seq_num 2703072688) dirty dne 2024-04-07T16:24:22.906 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 26 v 1102 (ObjNum 671 snap 205 seq_num 671) dirty exists 2024-04-07T16:24:22.906 INFO:tasks.rados.rados.0.smithi100.stdout:2322: finishing rollback tid 0 to smithi10066867-16 2024-04-07T16:24:22.906 INFO:tasks.rados.rados.0.smithi100.stdout:2322: finishing rollback tid 1 to smithi10066867-16 2024-04-07T16:24:22.906 INFO:tasks.rados.rados.0.smithi100.stdout:2319: done (7 left) 2024-04-07T16:24:22.906 INFO:tasks.rados.rados.0.smithi100.stdout:2321: done (6 left) 2024-04-07T16:24:22.906 INFO:tasks.rados.rados.0.smithi100.stdout:2326: read oid 39 snap 178 2024-04-07T16:24:22.906 INFO:tasks.rados.rados.0.smithi100.stdout:2326: expect (ObjNum 560 snap 171 seq_num 560) 2024-04-07T16:24:22.906 INFO:tasks.rados.rados.0.smithi100.stdout:2327: snap_create 2024-04-07T16:24:22.906 INFO:tasks.rados.rados.0.smithi100.stdout:2322: finishing rollback tid 2 to smithi10066867-16 2024-04-07T16:24:22.906 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 16 v 1125 (ObjNum 465 snap 142 seq_num 465) dirty exists 2024-04-07T16:24:22.906 INFO:tasks.rados.rados.0.smithi100.stdout:2318: finishing rollback tid 0 to smithi10066867-27 2024-04-07T16:24:22.907 INFO:tasks.rados.rados.0.smithi100.stdout:2318: finishing rollback tid 1 to smithi10066867-27 2024-04-07T16:24:22.907 INFO:tasks.rados.rados.0.smithi100.stdout:2318: finishing rollback tid 2 to smithi10066867-27 2024-04-07T16:24:22.907 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 27 v 1102 (ObjNum 541 snap 164 seq_num 541) dirty exists 2024-04-07T16:24:22.907 INFO:tasks.rados.rados.0.smithi100.stdout:2320: expect (ObjNum 700 snap 213 seq_num 700) 2024-04-07T16:24:23.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:22 smithi144 bash[66654]: audit 2024-04-07T16:24:21.442343+0000 mgr.y (mgr.25175) 210 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-04-07T16:24:23.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:22 smithi144 bash[66654]: cephadm 2024-04-07T16:24:21.444481+0000 mgr.y (mgr.25175) 211 : cephadm [INF] Upgrade: osd.5 is safe to restart 2024-04-07T16:24:23.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:22 smithi144 bash[66654]: cluster 2024-04-07T16:24:22.857913+0000 mon.a (mon.0) 682 : cluster [DBG] osdmap e509: 8 total, 8 up, 8 in 2024-04-07T16:24:23.433 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:22 smithi100 bash[74942]: audit 2024-04-07T16:24:21.442343+0000 mgr.y (mgr.25175) 210 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-04-07T16:24:23.433 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:22 smithi100 bash[74942]: cephadm 2024-04-07T16:24:21.444481+0000 mgr.y (mgr.25175) 211 : cephadm [INF] Upgrade: osd.5 is safe to restart 2024-04-07T16:24:23.433 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:22 smithi100 bash[74942]: cluster 2024-04-07T16:24:22.857913+0000 mon.a (mon.0) 682 : cluster [DBG] osdmap e509: 8 total, 8 up, 8 in 2024-04-07T16:24:23.434 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:22 smithi100 bash[76202]: audit 2024-04-07T16:24:21.442343+0000 mgr.y (mgr.25175) 210 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-04-07T16:24:23.434 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:22 smithi100 bash[76202]: cephadm 2024-04-07T16:24:21.444481+0000 mgr.y (mgr.25175) 211 : cephadm [INF] Upgrade: osd.5 is safe to restart 2024-04-07T16:24:23.434 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:22 smithi100 bash[76202]: cluster 2024-04-07T16:24:22.857913+0000 mon.a (mon.0) 682 : cluster [DBG] osdmap e509: 8 total, 8 up, 8 in 2024-04-07T16:24:23.461 INFO:tasks.rados.rados.0.smithi100.stdout:2323: finishing write tid 1 to smithi10066867-36 2024-04-07T16:24:23.461 INFO:tasks.rados.rados.0.smithi100.stdout:2323: finishing write tid 2 to smithi10066867-36 2024-04-07T16:24:23.461 INFO:tasks.rados.rados.0.smithi100.stdout:2323: finishing write tid 3 to smithi10066867-36 2024-04-07T16:24:23.461 INFO:tasks.rados.rados.0.smithi100.stdout:2325: finishing copy_from racing read to smithi10066867-4 2024-04-07T16:24:23.461 INFO:tasks.rados.rados.0.smithi100.stdout:2325: finishing copy_from to smithi10066867-4 2024-04-07T16:24:23.461 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 4 v 823 (ObjNum 684 snap 210 seq_num 684) dirty exists 2024-04-07T16:24:23.461 INFO:tasks.rados.rados.0.smithi100.stdout:2323: finishing write tid 4 to smithi10066867-36 2024-04-07T16:24:23.461 INFO:tasks.rados.rados.0.smithi100.stdout:2323: finishing write tid 5 to smithi10066867-36 2024-04-07T16:24:23.461 INFO:tasks.rados.rados.0.smithi100.stdout:2323: finishing write tid 6 to smithi10066867-36 2024-04-07T16:24:23.461 INFO:tasks.rados.rados.0.smithi100.stdout:2323: oid 36 updating version 0 to 1103 2024-04-07T16:24:23.461 INFO:tasks.rados.rados.0.smithi100.stdout:2323: oid 36 updating version 1103 to 1104 2024-04-07T16:24:23.461 INFO:tasks.rados.rados.0.smithi100.stdout:2323: oid 36 updating version 1104 to 1106 2024-04-07T16:24:23.461 INFO:tasks.rados.rados.0.smithi100.stdout:2323: oid 36 version 1106 is already newer than 1105 2024-04-07T16:24:23.461 INFO:tasks.rados.rados.0.smithi100.stdout:2323: oid 36 version 1106 is already newer than 1102 2024-04-07T16:24:23.461 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 36 v 1106 (ObjNum 713 snap 218 seq_num 713) dirty exists 2024-04-07T16:24:23.462 INFO:tasks.rados.rados.0.smithi100.stdout:2323: left oid 36 (ObjNum 713 snap 218 seq_num 713) 2024-04-07T16:24:23.462 INFO:tasks.rados.rados.0.smithi100.stdout:2326: expect (ObjNum 560 snap 171 seq_num 560) 2024-04-07T16:24:23.723 INFO:tasks.rados.rados.0.smithi100.stdout:2324: expect (ObjNum 688 snap 210 seq_num 688) 2024-04-07T16:24:23.977 INFO:tasks.rados.rados.0.smithi100.stdout:2318: done (7 left) 2024-04-07T16:24:23.977 INFO:tasks.rados.rados.0.smithi100.stdout:2320: done (6 left) 2024-04-07T16:24:23.977 INFO:tasks.rados.rados.0.smithi100.stdout:2322: done (5 left) 2024-04-07T16:24:23.977 INFO:tasks.rados.rados.0.smithi100.stdout:2323: done (4 left) 2024-04-07T16:24:23.977 INFO:tasks.rados.rados.0.smithi100.stdout:2324: done (3 left) 2024-04-07T16:24:23.977 INFO:tasks.rados.rados.0.smithi100.stdout:2325: done (2 left) 2024-04-07T16:24:23.977 INFO:tasks.rados.rados.0.smithi100.stdout:2326: done (1 left) 2024-04-07T16:24:23.977 INFO:tasks.rados.rados.0.smithi100.stdout:2327: done (0 left) 2024-04-07T16:24:23.977 INFO:tasks.rados.rados.0.smithi100.stdout:2328: copy_from oid 23 from oid 16 current snap is 219 2024-04-07T16:24:23.977 INFO:tasks.rados.rados.0.smithi100.stdout:2329: setattr oid 43 current snap is 219 2024-04-07T16:24:23.978 INFO:tasks.rados.rados.0.smithi100.stdout:2330: snap_create 2024-04-07T16:24:23.988 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 43 v 1103 (ObjNum 691 snap 211 seq_num 691) dirty exists 2024-04-07T16:24:23.999 INFO:tasks.rados.rados.0.smithi100.stdout:2328: finishing copy_from racing read to smithi10066867-23 2024-04-07T16:24:24.003 INFO:tasks.rados.rados.0.smithi100.stdout:2328: finishing copy_from to smithi10066867-23 2024-04-07T16:24:24.003 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 23 v 1125 (ObjNum 465 snap 142 seq_num 465) dirty exists 2024-04-07T16:24:24.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:23 smithi144 bash[66654]: cluster 2024-04-07T16:24:22.700499+0000 mgr.y (mgr.25175) 212 : cluster [DBG] pgmap v354: 121 pgs: 1 active+clean+snaptrim, 120 active+clean; 342 MiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 6.9 MiB/s rd, 2.8 MiB/s wr, 13 op/s; 4.4 MiB/s, 1 objects/s recovering 2024-04-07T16:24:24.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:23 smithi144 bash[66654]: audit 2024-04-07T16:24:23.622929+0000 mon.a (mon.0) 683 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:24:24.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:23 smithi144 bash[66654]: audit 2024-04-07T16:24:23.627566+0000 mon.a (mon.0) 684 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-07T16:24:24.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:23 smithi144 bash[66654]: audit 2024-04-07T16:24:23.628983+0000 mon.a (mon.0) 685 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:24:24.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:23 smithi144 bash[66654]: cluster 2024-04-07T16:24:23.860490+0000 mon.a (mon.0) 686 : cluster [DBG] osdmap e510: 8 total, 8 up, 8 in 2024-04-07T16:24:24.433 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:23 smithi100 bash[74942]: cluster 2024-04-07T16:24:22.700499+0000 mgr.y (mgr.25175) 212 : cluster [DBG] pgmap v354: 121 pgs: 1 active+clean+snaptrim, 120 active+clean; 342 MiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 6.9 MiB/s rd, 2.8 MiB/s wr, 13 op/s; 4.4 MiB/s, 1 objects/s recovering 2024-04-07T16:24:24.434 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:23 smithi100 bash[74942]: audit 2024-04-07T16:24:23.622929+0000 mon.a (mon.0) 683 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:24:24.434 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:23 smithi100 bash[74942]: audit 2024-04-07T16:24:23.627566+0000 mon.a (mon.0) 684 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-07T16:24:24.434 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:23 smithi100 bash[74942]: audit 2024-04-07T16:24:23.628983+0000 mon.a (mon.0) 685 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:24:24.434 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:23 smithi100 bash[74942]: cluster 2024-04-07T16:24:23.860490+0000 mon.a (mon.0) 686 : cluster [DBG] osdmap e510: 8 total, 8 up, 8 in 2024-04-07T16:24:24.434 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:23 smithi100 bash[76202]: cluster 2024-04-07T16:24:22.700499+0000 mgr.y (mgr.25175) 212 : cluster [DBG] pgmap v354: 121 pgs: 1 active+clean+snaptrim, 120 active+clean; 342 MiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 6.9 MiB/s rd, 2.8 MiB/s wr, 13 op/s; 4.4 MiB/s, 1 objects/s recovering 2024-04-07T16:24:24.434 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:23 smithi100 bash[76202]: audit 2024-04-07T16:24:23.622929+0000 mon.a (mon.0) 683 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:24:24.434 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:23 smithi100 bash[76202]: audit 2024-04-07T16:24:23.627566+0000 mon.a (mon.0) 684 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-07T16:24:24.434 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:23 smithi100 bash[76202]: audit 2024-04-07T16:24:23.628983+0000 mon.a (mon.0) 685 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:24:24.434 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:23 smithi100 bash[76202]: cluster 2024-04-07T16:24:23.860490+0000 mon.a (mon.0) 686 : cluster [DBG] osdmap e510: 8 total, 8 up, 8 in 2024-04-07T16:24:24.981 INFO:tasks.rados.rados.0.smithi100.stdout:2328: done (2 left) 2024-04-07T16:24:24.981 INFO:tasks.rados.rados.0.smithi100.stdout:2329: done (1 left) 2024-04-07T16:24:24.981 INFO:tasks.rados.rados.0.smithi100.stdout:2330: done (0 left) 2024-04-07T16:24:24.981 INFO:tasks.rados.rados.0.smithi100.stdout:2331: copy_from oid 9 from oid 43 current snap is 220 2024-04-07T16:24:24.981 INFO:tasks.rados.rados.0.smithi100.stdout:2332: copy_from oid 20 from oid 18 current snap is 220 2024-04-07T16:24:24.981 INFO:tasks.rados.rados.0.smithi100.stdout:2333: read oid 15 snap 186 2024-04-07T16:24:24.981 INFO:tasks.rados.rados.0.smithi100.stdout:2333: expect deleted 2024-04-07T16:24:24.981 INFO:tasks.rados.rados.0.smithi100.stdout:2334: read oid 44 snap -1 2024-04-07T16:24:24.981 INFO:tasks.rados.rados.0.smithi100.stdout:2334: expect (ObjNum 509 snap 152 seq_num 509) 2024-04-07T16:24:24.981 INFO:tasks.rados.rados.0.smithi100.stdout:2335: read oid 32 snap -1 2024-04-07T16:24:24.981 INFO:tasks.rados.rados.0.smithi100.stdout:2335: expect (ObjNum 630 snap 196 seq_num 630) 2024-04-07T16:24:24.981 INFO:tasks.rados.rados.0.smithi100.stdout:2336: rollback oid 50 current snap is 220 2024-04-07T16:24:24.981 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 50 to 219 2024-04-07T16:24:24.981 INFO:tasks.rados.rados.0.smithi100.stdout:2337: read oid 19 snap -1 2024-04-07T16:24:24.982 INFO:tasks.rados.rados.0.smithi100.stdout:2337: expect deleted 2024-04-07T16:24:24.982 INFO:tasks.rados.rados.0.smithi100.stdout:2338: copy_from oid 33 from oid 37 current snap is 220 2024-04-07T16:24:24.982 INFO:tasks.rados.rados.0.smithi100.stdout:2339: read oid 12 snap -1 2024-04-07T16:24:24.982 INFO:tasks.rados.rados.0.smithi100.stdout:2339: expect deleted 2024-04-07T16:24:24.982 INFO:tasks.rados.rados.0.smithi100.stdout:2340: setattr oid 48 current snap is 220 2024-04-07T16:24:24.983 INFO:tasks.rados.rados.0.smithi100.stdout:2341: setattr oid 41 current snap is 220 2024-04-07T16:24:24.983 INFO:tasks.rados.rados.0.smithi100.stdout:2342: rmattr oid 47 current snap is 220 2024-04-07T16:24:24.983 INFO:tasks.rados.rados.0.smithi100.stdout:2342: done (11 left) 2024-04-07T16:24:24.983 INFO:tasks.rados.rados.0.smithi100.stdout:2343: read oid 47 snap -1 2024-04-07T16:24:24.983 INFO:tasks.rados.rados.0.smithi100.stdout:2343: expect (ObjNum 684 snap 210 seq_num 684) 2024-04-07T16:24:24.983 INFO:tasks.rados.rados.0.smithi100.stdout:2344: read oid 4 snap -1 2024-04-07T16:24:24.983 INFO:tasks.rados.rados.0.smithi100.stdout:2344: expect (ObjNum 684 snap 210 seq_num 684) 2024-04-07T16:24:24.983 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 27 current snap is 220 2024-04-07T16:24:24.983 INFO:tasks.rados.rados.0.smithi100.stdout:2345: seq_num 714 ranges {5775360=761856,6537216=442368,6979584=688128,7667712=475136} 2024-04-07T16:24:25.023 INFO:tasks.rados.rados.0.smithi100.stdout:2345: writing smithi10066867-27 from 5775360 to 6537216 tid 1 2024-04-07T16:24:25.026 INFO:tasks.rados.rados.0.smithi100.stdout:2345: writing smithi10066867-27 from 6537216 to 6979584 tid 2 2024-04-07T16:24:25.031 INFO:tasks.rados.rados.0.smithi100.stdout:2345: writing smithi10066867-27 from 6979584 to 7667712 tid 3 2024-04-07T16:24:25.035 INFO:tasks.rados.rados.0.smithi100.stdout:2345: writing smithi10066867-27 from 7667712 to 8142848 tid 4 2024-04-07T16:24:25.035 INFO:tasks.rados.rados.0.smithi100.stdout:2346: snap_create 2024-04-07T16:24:25.035 INFO:tasks.rados.rados.0.smithi100.stdout:2336: finishing rollback tid 0 to smithi10066867-50 2024-04-07T16:24:25.035 INFO:tasks.rados.rados.0.smithi100.stdout:2336: finishing rollback tid 1 to smithi10066867-50 2024-04-07T16:24:25.035 INFO:tasks.rados.rados.0.smithi100.stdout:2335: expect (ObjNum 630 snap 196 seq_num 630) 2024-04-07T16:24:25.080 INFO:tasks.rados.rados.0.smithi100.stdout:2343: expect (ObjNum 684 snap 210 seq_num 684) 2024-04-07T16:24:25.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:24 smithi144 bash[66654]: cephadm 2024-04-07T16:24:23.616206+0000 mgr.y (mgr.25175) 213 : cephadm [INF] Upgrade: Updating osd.5 2024-04-07T16:24:25.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:24 smithi144 bash[66654]: cephadm 2024-04-07T16:24:23.633981+0000 mgr.y (mgr.25175) 214 : cephadm [INF] Deploying daemon osd.5 on smithi144 2024-04-07T16:24:25.127 INFO:tasks.rados.rados.0.smithi100.stdout:2336: finishing rollback tid 2 to smithi10066867-50 2024-04-07T16:24:25.127 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 50 v 1022 (ObjNum 657 snap 203 seq_num 657) dirty exists 2024-04-07T16:24:25.127 INFO:tasks.rados.rados.0.smithi100.stdout:2338: finishing copy_from to smithi10066867-33 2024-04-07T16:24:25.127 INFO:tasks.rados.rados.0.smithi100.stdout:2338: got expected ENOENT (src dne) 2024-04-07T16:24:25.127 INFO:tasks.rados.rados.0.smithi100.stdout:2338: finishing copy_from racing read to smithi10066867-33 2024-04-07T16:24:25.127 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 41 v 1131 (ObjNum 684 snap 210 seq_num 684) dirty exists 2024-04-07T16:24:25.128 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 48 v 1086 (ObjNum 700 snap 213 seq_num 700) dirty exists 2024-04-07T16:24:25.128 INFO:tasks.rados.rados.0.smithi100.stdout:2344: expect (ObjNum 684 snap 210 seq_num 684) 2024-04-07T16:24:25.174 INFO:tasks.rados.rados.0.smithi100.stdout:2345: finishing write tid 1 to smithi10066867-27 2024-04-07T16:24:25.174 INFO:tasks.rados.rados.0.smithi100.stdout:2345: finishing write tid 2 to smithi10066867-27 2024-04-07T16:24:25.174 INFO:tasks.rados.rados.0.smithi100.stdout:2345: finishing write tid 3 to smithi10066867-27 2024-04-07T16:24:25.174 INFO:tasks.rados.rados.0.smithi100.stdout:2334: expect (ObjNum 509 snap 152 seq_num 509) 2024-04-07T16:24:25.433 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:24 smithi100 bash[74942]: cephadm 2024-04-07T16:24:23.616206+0000 mgr.y (mgr.25175) 213 : cephadm [INF] Upgrade: Updating osd.5 2024-04-07T16:24:25.433 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:24 smithi100 bash[74942]: cephadm 2024-04-07T16:24:23.633981+0000 mgr.y (mgr.25175) 214 : cephadm [INF] Deploying daemon osd.5 on smithi144 2024-04-07T16:24:25.434 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:24 smithi100 bash[76202]: cephadm 2024-04-07T16:24:23.616206+0000 mgr.y (mgr.25175) 213 : cephadm [INF] Upgrade: Updating osd.5 2024-04-07T16:24:25.434 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:24 smithi100 bash[76202]: cephadm 2024-04-07T16:24:23.633981+0000 mgr.y (mgr.25175) 214 : cephadm [INF] Deploying daemon osd.5 on smithi144 2024-04-07T16:24:25.567 INFO:tasks.rados.rados.0.smithi100.stdout:2332: finishing copy_from racing read to smithi10066867-20 2024-04-07T16:24:25.567 INFO:tasks.rados.rados.0.smithi100.stdout:2332: finishing copy_from to smithi10066867-20 2024-04-07T16:24:25.567 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 20 v 1156 (ObjNum 620 snap 194 seq_num 620) dirty exists 2024-04-07T16:24:25.567 INFO:tasks.rados.rados.0.smithi100.stdout:2345: finishing write tid 4 to smithi10066867-27 2024-04-07T16:24:25.568 INFO:tasks.rados.rados.0.smithi100.stdout:2345: finishing write tid 5 to smithi10066867-27 2024-04-07T16:24:25.568 INFO:tasks.rados.rados.0.smithi100.stdout:2345: finishing write tid 6 to smithi10066867-27 2024-04-07T16:24:25.568 INFO:tasks.rados.rados.0.smithi100.stdout:2345: oid 27 updating version 0 to 1107 2024-04-07T16:24:25.568 INFO:tasks.rados.rados.0.smithi100.stdout:2345: oid 27 version 1107 is already newer than 1106 2024-04-07T16:24:25.568 INFO:tasks.rados.rados.0.smithi100.stdout:2345: oid 27 updating version 1107 to 1108 2024-04-07T16:24:25.568 INFO:tasks.rados.rados.0.smithi100.stdout:2345: oid 27 version 1108 is already newer than 1105 2024-04-07T16:24:25.568 INFO:tasks.rados.rados.0.smithi100.stdout:2345: oid 27 version 1108 is already newer than 1104 2024-04-07T16:24:25.568 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 27 v 1108 (ObjNum 714 snap 220 seq_num 714) dirty exists 2024-04-07T16:24:25.568 INFO:tasks.rados.rados.0.smithi100.stdout:2345: left oid 27 (ObjNum 714 snap 220 seq_num 714) 2024-04-07T16:24:25.568 INFO:tasks.rados.rados.0.smithi100.stdout:2331: finishing copy_from racing read to smithi10066867-9 2024-04-07T16:24:25.568 INFO:tasks.rados.rados.0.smithi100.stdout:2331: finishing copy_from to smithi10066867-9 2024-04-07T16:24:25.568 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 9 v 1103 (ObjNum 691 snap 211 seq_num 691) dirty exists 2024-04-07T16:24:25.990 INFO:tasks.rados.rados.0.smithi100.stdout:2331: done (14 left) 2024-04-07T16:24:25.991 INFO:tasks.rados.rados.0.smithi100.stdout:2332: done (13 left) 2024-04-07T16:24:25.991 INFO:tasks.rados.rados.0.smithi100.stdout:2333: done (12 left) 2024-04-07T16:24:25.991 INFO:tasks.rados.rados.0.smithi100.stdout:2334: done (11 left) 2024-04-07T16:24:25.991 INFO:tasks.rados.rados.0.smithi100.stdout:2335: done (10 left) 2024-04-07T16:24:25.991 INFO:tasks.rados.rados.0.smithi100.stdout:2336: done (9 left) 2024-04-07T16:24:25.991 INFO:tasks.rados.rados.0.smithi100.stdout:2337: done (8 left) 2024-04-07T16:24:25.991 INFO:tasks.rados.rados.0.smithi100.stdout:2338: done (7 left) 2024-04-07T16:24:25.991 INFO:tasks.rados.rados.0.smithi100.stdout:2339: done (6 left) 2024-04-07T16:24:25.991 INFO:tasks.rados.rados.0.smithi100.stdout:2340: done (5 left) 2024-04-07T16:24:25.991 INFO:tasks.rados.rados.0.smithi100.stdout:2341: done (4 left) 2024-04-07T16:24:25.991 INFO:tasks.rados.rados.0.smithi100.stdout:2343: done (3 left) 2024-04-07T16:24:25.992 INFO:tasks.rados.rados.0.smithi100.stdout:2344: done (2 left) 2024-04-07T16:24:25.992 INFO:tasks.rados.rados.0.smithi100.stdout:2345: done (1 left) 2024-04-07T16:24:25.992 INFO:tasks.rados.rados.0.smithi100.stdout:2346: done (0 left) 2024-04-07T16:24:25.992 INFO:tasks.rados.rados.0.smithi100.stdout:2347: copy_from oid 27 from oid 9 current snap is 221 2024-04-07T16:24:25.992 INFO:tasks.rados.rados.0.smithi100.stdout:2348: read oid 16 snap 216 2024-04-07T16:24:25.992 INFO:tasks.rados.rados.0.smithi100.stdout:2348: expect (ObjNum 651 snap 203 seq_num 651) 2024-04-07T16:24:25.992 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 36 current snap is 221 2024-04-07T16:24:25.992 INFO:tasks.rados.rados.0.smithi100.stdout:2349: seq_num 715 ranges {7913472=778240,8691712=770048,9461760=499712,9961472=335872} 2024-04-07T16:24:26.047 INFO:tasks.rados.rados.0.smithi100.stdout:2349: writing smithi10066867-36 from 7913472 to 8691712 tid 1 2024-04-07T16:24:26.052 INFO:tasks.rados.rados.0.smithi100.stdout:2349: writing smithi10066867-36 from 8691712 to 9461760 tid 2 2024-04-07T16:24:26.055 INFO:tasks.rados.rados.0.smithi100.stdout:2349: writing smithi10066867-36 from 9461760 to 9961472 tid 3 2024-04-07T16:24:26.058 INFO:tasks.rados.rados.0.smithi100.stdout:2349: writing smithi10066867-36 from 9961472 to 10297344 tid 4 2024-04-07T16:24:26.058 INFO:tasks.rados.rados.0.smithi100.stdout:2350: setattr oid 44 current snap is 221 2024-04-07T16:24:26.058 INFO:tasks.rados.rados.0.smithi100.stdout:2348: expect (ObjNum 651 snap 203 seq_num 651) 2024-04-07T16:24:26.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:25 smithi144 bash[66654]: cluster 2024-04-07T16:24:24.701347+0000 mgr.y (mgr.25175) 215 : cluster [DBG] pgmap v357: 121 pgs: 121 active+clean; 355 MiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 15 MiB/s rd, 4.9 MiB/s wr, 28 op/s 2024-04-07T16:24:26.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:25 smithi144 bash[66654]: cluster 2024-04-07T16:24:24.998218+0000 mon.a (mon.0) 687 : cluster [DBG] osdmap e511: 8 total, 8 up, 8 in 2024-04-07T16:24:26.400 INFO:tasks.rados.rados.0.smithi100.stdout:2347: finishing copy_from racing read to smithi10066867-27 2024-04-07T16:24:26.400 INFO:tasks.rados.rados.0.smithi100.stdout:2349: finishing write tid 1 to smithi10066867-36 2024-04-07T16:24:26.400 INFO:tasks.rados.rados.0.smithi100.stdout:2349: finishing write tid 2 to smithi10066867-36 2024-04-07T16:24:26.401 INFO:tasks.rados.rados.0.smithi100.stdout:2349: finishing write tid 3 to smithi10066867-36 2024-04-07T16:24:26.401 INFO:tasks.rados.rados.0.smithi100.stdout:2349: finishing write tid 4 to smithi10066867-36 2024-04-07T16:24:26.401 INFO:tasks.rados.rados.0.smithi100.stdout:2349: finishing write tid 5 to smithi10066867-36 2024-04-07T16:24:26.401 INFO:tasks.rados.rados.0.smithi100.stdout:2347: finishing copy_from to smithi10066867-27 2024-04-07T16:24:26.401 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 27 v 1103 (ObjNum 691 snap 211 seq_num 691) dirty exists 2024-04-07T16:24:26.401 INFO:tasks.rados.rados.0.smithi100.stdout:2349: finishing write tid 6 to smithi10066867-36 2024-04-07T16:24:26.401 INFO:tasks.rados.rados.0.smithi100.stdout:2349: oid 36 updating version 0 to 1110 2024-04-07T16:24:26.401 INFO:tasks.rados.rados.0.smithi100.stdout:2349: oid 36 version 1110 is already newer than 1109 2024-04-07T16:24:26.401 INFO:tasks.rados.rados.0.smithi100.stdout:2349: oid 36 version 1110 is already newer than 1107 2024-04-07T16:24:26.401 INFO:tasks.rados.rados.0.smithi100.stdout:2349: oid 36 version 1110 is already newer than 1108 2024-04-07T16:24:26.402 INFO:tasks.rados.rados.0.smithi100.stdout:2349: oid 36 updating version 1110 to 1111 2024-04-07T16:24:26.402 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 36 v 1111 (ObjNum 715 snap 221 seq_num 715) dirty exists 2024-04-07T16:24:26.402 INFO:tasks.rados.rados.0.smithi100.stdout:2349: left oid 36 (ObjNum 715 snap 221 seq_num 715) 2024-04-07T16:24:26.402 INFO:tasks.rados.rados.0.smithi100.stdout:2347: done (3 left) 2024-04-07T16:24:26.402 INFO:tasks.rados.rados.0.smithi100.stdout:2348: done (2 left) 2024-04-07T16:24:26.402 INFO:tasks.rados.rados.0.smithi100.stdout:2349: done (1 left) 2024-04-07T16:24:26.402 INFO:tasks.rados.rados.0.smithi100.stdout:2351: copy_from oid 29 from oid 48 current snap is 221 2024-04-07T16:24:26.402 INFO:tasks.rados.rados.0.smithi100.stdout:2352: read oid 1 snap -1 2024-04-07T16:24:26.402 INFO:tasks.rados.rados.0.smithi100.stdout:2352: expect (ObjNum 685 snap 210 seq_num 685) 2024-04-07T16:24:26.402 INFO:tasks.rados.rados.0.smithi100.stdout:2353: read oid 19 snap 215 2024-04-07T16:24:26.402 INFO:tasks.rados.rados.0.smithi100.stdout:2353: expect deleted 2024-04-07T16:24:26.402 INFO:tasks.rados.rados.0.smithi100.stdout:2354: delete oid 42 current snap is 221 2024-04-07T16:24:26.407 INFO:tasks.rados.rados.0.smithi100.stdout:2353: done (4 left) 2024-04-07T16:24:26.408 INFO:tasks.rados.rados.0.smithi100.stdout:2354: done (3 left) 2024-04-07T16:24:26.408 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 19 current snap is 221 2024-04-07T16:24:26.408 INFO:tasks.rados.rados.0.smithi100.stdout:2355: seq_num 717 ranges {0=16384} 2024-04-07T16:24:26.408 INFO:tasks.rados.rados.0.smithi100.stdout:2355: writing smithi10066867-19 from 0 to 16384 tid 1 2024-04-07T16:24:26.408 INFO:tasks.rados.rados.0.smithi100.stdout:2356: rmattr oid 20 current snap is 221 2024-04-07T16:24:26.408 INFO:tasks.rados.rados.0.smithi100.stdout:2356: done (4 left) 2024-04-07T16:24:26.408 INFO:tasks.rados.rados.0.smithi100.stdout:2357: copy_from oid 9 from oid 47 current snap is 221 2024-04-07T16:24:26.408 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 11 current snap is 221 2024-04-07T16:24:26.408 INFO:tasks.rados.rados.0.smithi100.stdout:2358: seq_num 718 ranges {0=32768} 2024-04-07T16:24:26.408 INFO:tasks.rados.rados.0.smithi100.stdout:2358: writing smithi10066867-11 from 0 to 32768 tid 1 2024-04-07T16:24:26.408 INFO:tasks.rados.rados.0.smithi100.stdout:2359: snap_remove snap 178 2024-04-07T16:24:26.433 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:25 smithi100 bash[76202]: cluster 2024-04-07T16:24:24.701347+0000 mgr.y (mgr.25175) 215 : cluster [DBG] pgmap v357: 121 pgs: 121 active+clean; 355 MiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 15 MiB/s rd, 4.9 MiB/s wr, 28 op/s 2024-04-07T16:24:26.433 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:25 smithi100 bash[76202]: cluster 2024-04-07T16:24:24.998218+0000 mon.a (mon.0) 687 : cluster [DBG] osdmap e511: 8 total, 8 up, 8 in 2024-04-07T16:24:26.433 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:25 smithi100 bash[74942]: cluster 2024-04-07T16:24:24.701347+0000 mgr.y (mgr.25175) 215 : cluster [DBG] pgmap v357: 121 pgs: 121 active+clean; 355 MiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 15 MiB/s rd, 4.9 MiB/s wr, 28 op/s 2024-04-07T16:24:26.433 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:25 smithi100 bash[74942]: cluster 2024-04-07T16:24:24.998218+0000 mon.a (mon.0) 687 : cluster [DBG] osdmap e511: 8 total, 8 up, 8 in 2024-04-07T16:24:26.828 INFO:tasks.rados.rados.0.smithi100.stdout:2359: done (6 left) 2024-04-07T16:24:26.828 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 34 current snap is 221 2024-04-07T16:24:26.828 INFO:tasks.rados.rados.0.smithi100.stdout:2360: seq_num 719 ranges {0=49152} 2024-04-07T16:24:26.828 INFO:tasks.rados.rados.0.smithi100.stdout:2360: writing smithi10066867-34 from 0 to 49152 tid 1 2024-04-07T16:24:26.830 INFO:tasks.rados.rados.0.smithi100.stdout:2361: rollback oid 32 current snap is 221 2024-04-07T16:24:26.830 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 32 to 219 2024-04-07T16:24:26.830 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 44 v 1136 (ObjNum 509 snap 152 seq_num 509) dirty exists 2024-04-07T16:24:26.830 INFO:tasks.rados.rados.0.smithi100.stdout:2355: finishing write tid 1 to smithi10066867-19 2024-04-07T16:24:26.830 INFO:tasks.rados.rados.0.smithi100.stdout:2355: finishing write tid 2 to smithi10066867-19 2024-04-07T16:24:26.830 INFO:tasks.rados.rados.0.smithi100.stdout:2357: finishing copy_from racing read to smithi10066867-9 2024-04-07T16:24:26.831 INFO:tasks.rados.rados.0.smithi100.stdout:2358: finishing write tid 1 to smithi10066867-11 2024-04-07T16:24:26.831 INFO:tasks.rados.rados.0.smithi100.stdout:2358: finishing write tid 2 to smithi10066867-11 2024-04-07T16:24:26.831 INFO:tasks.rados.rados.0.smithi100.stdout:2355: finishing write tid 3 to smithi10066867-19 2024-04-07T16:24:26.831 INFO:tasks.rados.rados.0.smithi100.stdout:2355: oid 19 updating version 0 to 1113 2024-04-07T16:24:26.831 INFO:tasks.rados.rados.0.smithi100.stdout:2355: oid 19 version 1113 is already newer than 1112 2024-04-07T16:24:26.831 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 19 v 1113 (ObjNum 717 snap 221 seq_num 717) dirty exists 2024-04-07T16:24:26.831 INFO:tasks.rados.rados.0.smithi100.stdout:2355: left oid 19 (ObjNum 717 snap 221 seq_num 717) 2024-04-07T16:24:26.831 INFO:tasks.rados.rados.0.smithi100.stdout:2352: expect (ObjNum 685 snap 210 seq_num 685) 2024-04-07T16:24:26.902 INFO:tasks.rados.rados.0.smithi100.stdout:2357: finishing copy_from to smithi10066867-9 2024-04-07T16:24:26.902 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 9 v 823 (ObjNum 684 snap 210 seq_num 684) dirty exists 2024-04-07T16:24:26.902 INFO:tasks.rados.rados.0.smithi100.stdout:2358: finishing write tid 3 to smithi10066867-11 2024-04-07T16:24:26.903 INFO:tasks.rados.rados.0.smithi100.stdout:2358: oid 11 updating version 0 to 1336 2024-04-07T16:24:26.903 INFO:tasks.rados.rados.0.smithi100.stdout:2358: oid 11 version 1336 is already newer than 1335 2024-04-07T16:24:26.903 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 11 v 1336 (ObjNum 718 snap 221 seq_num 718) dirty exists 2024-04-07T16:24:26.903 INFO:tasks.rados.rados.0.smithi100.stdout:2358: left oid 11 (ObjNum 718 snap 221 seq_num 718) 2024-04-07T16:24:26.903 INFO:tasks.rados.rados.0.smithi100.stdout:2351: finishing copy_from racing read to smithi10066867-29 2024-04-07T16:24:26.903 INFO:tasks.rados.rados.0.smithi100.stdout:2351: finishing copy_from to smithi10066867-29 2024-04-07T16:24:26.903 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 29 v 1086 (ObjNum 700 snap 213 seq_num 700) dirty exists 2024-04-07T16:24:26.903 INFO:tasks.rados.rados.0.smithi100.stdout:2350: done (7 left) 2024-04-07T16:24:26.903 INFO:tasks.rados.rados.0.smithi100.stdout:2351: done (6 left) 2024-04-07T16:24:26.903 INFO:tasks.rados.rados.0.smithi100.stdout:2352: done (5 left) 2024-04-07T16:24:26.903 INFO:tasks.rados.rados.0.smithi100.stdout:2355: done (4 left) 2024-04-07T16:24:26.903 INFO:tasks.rados.rados.0.smithi100.stdout:2357: done (3 left) 2024-04-07T16:24:26.903 INFO:tasks.rados.rados.0.smithi100.stdout:2358: done (2 left) 2024-04-07T16:24:26.903 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 38 current snap is 221 2024-04-07T16:24:26.904 INFO:tasks.rados.rados.0.smithi100.stdout:2360: finishing write tid 1 to smithi10066867-34 2024-04-07T16:24:26.904 INFO:tasks.rados.rados.0.smithi100.stdout:2361: finishing rollback tid 0 to smithi10066867-32 2024-04-07T16:24:26.904 INFO:tasks.rados.rados.0.smithi100.stdout:2361: finishing rollback tid 1 to smithi10066867-32 2024-04-07T16:24:26.904 INFO:tasks.rados.rados.0.smithi100.stdout:2360: finishing write tid 2 to smithi10066867-34 2024-04-07T16:24:26.904 INFO:tasks.rados.rados.0.smithi100.stdout:2362: seq_num 720 ranges {2891776=65536} 2024-04-07T16:24:26.922 INFO:tasks.rados.rados.0.smithi100.stdout:2362: writing smithi10066867-38 from 2891776 to 2957312 tid 1 2024-04-07T16:24:26.923 INFO:tasks.rados.rados.0.smithi100.stdout:2363: snap_create 2024-04-07T16:24:26.923 INFO:tasks.rados.rados.0.smithi100.stdout:2361: finishing rollback tid 2 to smithi10066867-32 2024-04-07T16:24:26.923 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 32 v 1097 (ObjNum 630 snap 196 seq_num 630) dirty exists 2024-04-07T16:24:26.923 INFO:tasks.rados.rados.0.smithi100.stdout:2360: finishing write tid 3 to smithi10066867-34 2024-04-07T16:24:26.923 INFO:tasks.rados.rados.0.smithi100.stdout:2360: oid 34 updating version 0 to 978 2024-04-07T16:24:26.923 INFO:tasks.rados.rados.0.smithi100.stdout:2360: oid 34 version 978 is already newer than 977 2024-04-07T16:24:26.923 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 34 v 978 (ObjNum 719 snap 221 seq_num 719) dirty exists 2024-04-07T16:24:26.923 INFO:tasks.rados.rados.0.smithi100.stdout:2360: left oid 34 (ObjNum 719 snap 221 seq_num 719) 2024-04-07T16:24:26.927 INFO:tasks.rados.rados.0.smithi100.stdout:2362: finishing write tid 1 to smithi10066867-38 2024-04-07T16:24:26.927 INFO:tasks.rados.rados.0.smithi100.stdout:2362: finishing write tid 2 to smithi10066867-38 2024-04-07T16:24:26.929 INFO:tasks.rados.rados.0.smithi100.stdout:2362: finishing write tid 3 to smithi10066867-38 2024-04-07T16:24:26.929 INFO:tasks.rados.rados.0.smithi100.stdout:2362: oid 38 updating version 0 to 1103 2024-04-07T16:24:26.929 INFO:tasks.rados.rados.0.smithi100.stdout:2362: oid 38 version 1103 is already newer than 1102 2024-04-07T16:24:26.930 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 38 v 1103 (ObjNum 720 snap 221 seq_num 720) dirty exists 2024-04-07T16:24:26.930 INFO:tasks.rados.rados.0.smithi100.stdout:2362: left oid 38 (ObjNum 720 snap 221 seq_num 720) 2024-04-07T16:24:27.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:27 smithi144 bash[66654]: cluster 2024-04-07T16:24:26.008019+0000 mon.a (mon.0) 688 : cluster [DBG] osdmap e512: 8 total, 8 up, 8 in 2024-04-07T16:24:27.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:27 smithi144 bash[66654]: cluster 2024-04-07T16:24:26.845761+0000 mon.a (mon.0) 689 : cluster [DBG] osdmap e513: 8 total, 8 up, 8 in 2024-04-07T16:24:27.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:27 smithi144 bash[66654]: audit 2024-04-07T16:24:26.909814+0000 mon.a (mon.0) 690 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:24:27.433 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:27 smithi100 bash[74942]: cluster 2024-04-07T16:24:26.008019+0000 mon.a (mon.0) 688 : cluster [DBG] osdmap e512: 8 total, 8 up, 8 in 2024-04-07T16:24:27.434 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:27 smithi100 bash[74942]: cluster 2024-04-07T16:24:26.845761+0000 mon.a (mon.0) 689 : cluster [DBG] osdmap e513: 8 total, 8 up, 8 in 2024-04-07T16:24:27.434 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:27 smithi100 bash[74942]: audit 2024-04-07T16:24:26.909814+0000 mon.a (mon.0) 690 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:24:27.434 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:27 smithi100 bash[76202]: cluster 2024-04-07T16:24:26.008019+0000 mon.a (mon.0) 688 : cluster [DBG] osdmap e512: 8 total, 8 up, 8 in 2024-04-07T16:24:27.434 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:27 smithi100 bash[76202]: cluster 2024-04-07T16:24:26.845761+0000 mon.a (mon.0) 689 : cluster [DBG] osdmap e513: 8 total, 8 up, 8 in 2024-04-07T16:24:27.434 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:27 smithi100 bash[76202]: audit 2024-04-07T16:24:26.909814+0000 mon.a (mon.0) 690 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:24:27.715 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:27 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:24:27.715 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:24:27 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:24:27.715 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:24:27 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:24:27.716 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:24:27 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:24:27.716 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 16:24:27 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:24:27.716 INFO:journalctl@ceph.mds.b.smithi144.stdout:Apr 07 16:24:27 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:24:27.717 INFO:journalctl@ceph.osd.4.smithi144.stdout:Apr 07 16:24:27 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:24:27.717 INFO:journalctl@ceph.osd.6.smithi144.stdout:Apr 07 16:24:27 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:24:27.717 INFO:journalctl@ceph.osd.5.smithi144.stdout:Apr 07 16:24:27 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:24:27.717 INFO:journalctl@ceph.osd.5.smithi144.stdout:Apr 07 16:24:27 smithi144 systemd[1]: Stopping Ceph osd.5 for 0702e3a8-f4f8-11ee-b649-cb9ed24678a4... 2024-04-07T16:24:27.717 INFO:journalctl@ceph.osd.7.smithi144.stdout:Apr 07 16:24:27 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:24:27.908 INFO:tasks.rados.rados.0.smithi100.stdout:2360: done (3 left) 2024-04-07T16:24:27.909 INFO:tasks.rados.rados.0.smithi100.stdout:2361: done (2 left) 2024-04-07T16:24:27.909 INFO:tasks.rados.rados.0.smithi100.stdout:2362: done (1 left) 2024-04-07T16:24:27.909 INFO:tasks.rados.rados.0.smithi100.stdout:2363: done (0 left) 2024-04-07T16:24:27.909 INFO:tasks.rados.rados.0.smithi100.stdout:2364: snap_remove snap 221 2024-04-07T16:24:27.996 INFO:journalctl@ceph.osd.5.smithi144.stdout:Apr 07 16:24:27 smithi144 bash[32383]: debug 2024-04-07T16:24:27.846+0000 7fdd6f44b700 -1 received signal: Terminated from /sbin/docker-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug (PID: 1) UID: 0 2024-04-07T16:24:27.996 INFO:journalctl@ceph.osd.5.smithi144.stdout:Apr 07 16:24:27 smithi144 bash[32383]: debug 2024-04-07T16:24:27.846+0000 7fdd6f44b700 -1 osd.5 513 *** Got signal Terminated *** 2024-04-07T16:24:27.996 INFO:journalctl@ceph.osd.5.smithi144.stdout:Apr 07 16:24:27 smithi144 bash[32383]: debug 2024-04-07T16:24:27.846+0000 7fdd6f44b700 -1 osd.5 513 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-07T16:24:28.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:28 smithi144 bash[66654]: cluster 2024-04-07T16:24:26.702041+0000 mgr.y (mgr.25175) 216 : cluster [DBG] pgmap v360: 121 pgs: 121 active+clean; 360 MiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 15 MiB/s rd, 4.4 MiB/s wr, 27 op/s 2024-04-07T16:24:28.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:28 smithi144 bash[66654]: cluster 2024-04-07T16:24:27.859910+0000 mon.a (mon.0) 691 : cluster [INF] osd.5 marked itself down and dead 2024-04-07T16:24:28.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:28 smithi144 bash[66654]: cluster 2024-04-07T16:24:27.907951+0000 mon.a (mon.0) 692 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN) 2024-04-07T16:24:28.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:28 smithi144 bash[66654]: cluster 2024-04-07T16:24:27.920492+0000 mon.a (mon.0) 693 : cluster [DBG] osdmap e514: 8 total, 7 up, 8 in 2024-04-07T16:24:28.418 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:28 smithi100 bash[74942]: cluster 2024-04-07T16:24:26.702041+0000 mgr.y (mgr.25175) 216 : cluster [DBG] pgmap v360: 121 pgs: 121 active+clean; 360 MiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 15 MiB/s rd, 4.4 MiB/s wr, 27 op/s 2024-04-07T16:24:28.418 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:28 smithi100 bash[74942]: cluster 2024-04-07T16:24:27.859910+0000 mon.a (mon.0) 691 : cluster [INF] osd.5 marked itself down and dead 2024-04-07T16:24:28.418 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:28 smithi100 bash[74942]: cluster 2024-04-07T16:24:27.907951+0000 mon.a (mon.0) 692 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN) 2024-04-07T16:24:28.418 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:28 smithi100 bash[74942]: cluster 2024-04-07T16:24:27.920492+0000 mon.a (mon.0) 693 : cluster [DBG] osdmap e514: 8 total, 7 up, 8 in 2024-04-07T16:24:28.418 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:28 smithi100 bash[76202]: cluster 2024-04-07T16:24:26.702041+0000 mgr.y (mgr.25175) 216 : cluster [DBG] pgmap v360: 121 pgs: 121 active+clean; 360 MiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 15 MiB/s rd, 4.4 MiB/s wr, 27 op/s 2024-04-07T16:24:28.418 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:28 smithi100 bash[76202]: cluster 2024-04-07T16:24:27.859910+0000 mon.a (mon.0) 691 : cluster [INF] osd.5 marked itself down and dead 2024-04-07T16:24:28.418 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:28 smithi100 bash[76202]: cluster 2024-04-07T16:24:27.907951+0000 mon.a (mon.0) 692 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN) 2024-04-07T16:24:28.418 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:28 smithi100 bash[76202]: cluster 2024-04-07T16:24:27.920492+0000 mon.a (mon.0) 693 : cluster [DBG] osdmap e514: 8 total, 7 up, 8 in 2024-04-07T16:24:28.683 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:24:28 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:24:28] "GET /metrics HTTP/1.1" 200 36074 "" "Prometheus/2.43.0" 2024-04-07T16:24:28.909 INFO:tasks.rados.rados.0.smithi100.stdout:2364: done (0 left) 2024-04-07T16:24:28.909 INFO:tasks.rados.rados.0.smithi100.stdout:2365: read oid 16 snap -1 2024-04-07T16:24:28.909 INFO:tasks.rados.rados.0.smithi100.stdout:2365: expect (ObjNum 465 snap 142 seq_num 465) 2024-04-07T16:24:28.909 INFO:tasks.rados.rados.0.smithi100.stdout:2366: read oid 15 snap -1 2024-04-07T16:24:28.909 INFO:tasks.rados.rados.0.smithi100.stdout:2366: expect (ObjNum 699 snap 213 seq_num 699) 2024-04-07T16:24:28.909 INFO:tasks.rados.rados.0.smithi100.stdout:2367: snap_create 2024-04-07T16:24:28.959 INFO:tasks.rados.rados.0.smithi100.stdout:2366: expect (ObjNum 699 snap 213 seq_num 699) 2024-04-07T16:24:29.095 INFO:tasks.rados.rados.0.smithi100.stdout:2365: expect (ObjNum 465 snap 142 seq_num 465) 2024-04-07T16:24:29.356 INFO:journalctl@ceph.osd.5.smithi144.stdout:Apr 07 16:24:28 smithi144 bash[74917]: ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4-osd-5 2024-04-07T16:24:29.356 INFO:journalctl@ceph.osd.5.smithi144.stdout:Apr 07 16:24:29 smithi144 bash[74982]: Error response from daemon: No such container: ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4-osd-5 2024-04-07T16:24:29.903 INFO:tasks.rados.rados.0.smithi100.stdout:2365: done (2 left) 2024-04-07T16:24:29.903 INFO:tasks.rados.rados.0.smithi100.stdout:2366: done (1 left) 2024-04-07T16:24:29.903 INFO:tasks.rados.rados.0.smithi100.stdout:2367: done (0 left) 2024-04-07T16:24:29.903 INFO:tasks.rados.rados.0.smithi100.stdout:2368: setattr oid 33 current snap is 223 2024-04-07T16:24:29.908 INFO:tasks.rados.rados.0.smithi100.stdout:2369: delete oid 35 current snap is 223 2024-04-07T16:24:29.928 INFO:tasks.rados.rados.0.smithi100.stdout:2369: done (1 left) 2024-04-07T16:24:29.928 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 49 current snap is 223 2024-04-07T16:24:29.928 INFO:tasks.rados.rados.0.smithi100.stdout:2370: seq_num 722 ranges {4653056=98304} 2024-04-07T16:24:29.993 INFO:tasks.rados.rados.0.smithi100.stdout:2370: writing smithi10066867-49 from 4653056 to 4751360 tid 1 2024-04-07T16:24:29.993 INFO:tasks.rados.rados.0.smithi100.stdout:2371: snap_create 2024-04-07T16:24:29.993 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 33 v 1347 (ObjNum 881 snap 0 seq_num 2702780048) dirty exists 2024-04-07T16:24:29.998 INFO:tasks.rados.rados.0.smithi100.stdout:2370: finishing write tid 1 to smithi10066867-49 2024-04-07T16:24:29.998 INFO:tasks.rados.rados.0.smithi100.stdout:2370: finishing write tid 2 to smithi10066867-49 2024-04-07T16:24:30.000 INFO:tasks.rados.rados.0.smithi100.stdout:2370: finishing write tid 3 to smithi10066867-49 2024-04-07T16:24:30.001 INFO:tasks.rados.rados.0.smithi100.stdout:2370: oid 49 updating version 0 to 1140 2024-04-07T16:24:30.001 INFO:tasks.rados.rados.0.smithi100.stdout:2370: oid 49 version 1140 is already newer than 1139 2024-04-07T16:24:30.001 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 49 v 1140 (ObjNum 722 snap 223 seq_num 722) dirty exists 2024-04-07T16:24:30.001 INFO:tasks.rados.rados.0.smithi100.stdout:2370: left oid 49 (ObjNum 722 snap 223 seq_num 722) 2024-04-07T16:24:30.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:29 smithi144 bash[66654]: cluster 2024-04-07T16:24:28.703039+0000 mgr.y (mgr.25175) 217 : cluster [DBG] pgmap v363: 121 pgs: 5 peering, 17 stale+active+clean, 99 active+clean; 356 MiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 7.6 MiB/s rd, 4.4 MiB/s wr, 18 op/s 2024-04-07T16:24:30.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:29 smithi144 bash[66654]: cluster 2024-04-07T16:24:28.922099+0000 mon.a (mon.0) 694 : cluster [DBG] osdmap e515: 8 total, 7 up, 8 in 2024-04-07T16:24:30.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:29 smithi144 bash[66654]: cluster 2024-04-07T16:24:29.013668+0000 mon.a (mon.0) 695 : cluster [WRN] Health check failed: Reduced data availability: 2 pgs inactive, 2 pgs peering (PG_AVAILABILITY) 2024-04-07T16:24:30.183 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:29 smithi100 bash[74942]: cluster 2024-04-07T16:24:28.703039+0000 mgr.y (mgr.25175) 217 : cluster [DBG] pgmap v363: 121 pgs: 5 peering, 17 stale+active+clean, 99 active+clean; 356 MiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 7.6 MiB/s rd, 4.4 MiB/s wr, 18 op/s 2024-04-07T16:24:30.184 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:29 smithi100 bash[74942]: cluster 2024-04-07T16:24:28.922099+0000 mon.a (mon.0) 694 : cluster [DBG] osdmap e515: 8 total, 7 up, 8 in 2024-04-07T16:24:30.184 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:29 smithi100 bash[74942]: cluster 2024-04-07T16:24:29.013668+0000 mon.a (mon.0) 695 : cluster [WRN] Health check failed: Reduced data availability: 2 pgs inactive, 2 pgs peering (PG_AVAILABILITY) 2024-04-07T16:24:30.184 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:29 smithi100 bash[76202]: cluster 2024-04-07T16:24:28.703039+0000 mgr.y (mgr.25175) 217 : cluster [DBG] pgmap v363: 121 pgs: 5 peering, 17 stale+active+clean, 99 active+clean; 356 MiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 7.6 MiB/s rd, 4.4 MiB/s wr, 18 op/s 2024-04-07T16:24:30.184 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:29 smithi100 bash[76202]: cluster 2024-04-07T16:24:28.922099+0000 mon.a (mon.0) 694 : cluster [DBG] osdmap e515: 8 total, 7 up, 8 in 2024-04-07T16:24:30.184 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:29 smithi100 bash[76202]: cluster 2024-04-07T16:24:29.013668+0000 mon.a (mon.0) 695 : cluster [WRN] Health check failed: Reduced data availability: 2 pgs inactive, 2 pgs peering (PG_AVAILABILITY) 2024-04-07T16:24:30.925 INFO:tasks.rados.rados.0.smithi100.stdout:2368: done (2 left) 2024-04-07T16:24:30.925 INFO:tasks.rados.rados.0.smithi100.stdout:2370: done (1 left) 2024-04-07T16:24:30.925 INFO:tasks.rados.rados.0.smithi100.stdout:2371: done (0 left) 2024-04-07T16:24:30.925 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 32 current snap is 224 2024-04-07T16:24:30.925 INFO:tasks.rados.rados.0.smithi100.stdout:2372: seq_num 723 ranges {1376256=114688} 2024-04-07T16:24:30.940 INFO:tasks.rados.rados.0.smithi100.stdout:2372: writing smithi10066867-32 from 1376256 to 1490944 tid 1 2024-04-07T16:24:30.940 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 12 current snap is 224 2024-04-07T16:24:30.940 INFO:tasks.rados.rados.0.smithi100.stdout:2373: seq_num 724 ranges {0=131072} 2024-04-07T16:24:30.941 INFO:tasks.rados.rados.0.smithi100.stdout:2373: writing smithi10066867-12 from 0 to 131072 tid 1 2024-04-07T16:24:30.941 INFO:tasks.rados.rados.0.smithi100.stdout:2374: rmattr oid 2 current snap is 224 2024-04-07T16:24:30.942 INFO:tasks.rados.rados.0.smithi100.stdout:2375: rollback oid 17 current snap is 224 2024-04-07T16:24:30.942 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 17 to 223 2024-04-07T16:24:30.942 INFO:tasks.rados.rados.0.smithi100.stdout:2376: delete oid 35 current snap is 224 2024-04-07T16:24:30.951 INFO:tasks.rados.rados.0.smithi100.stdout:2372: finishing write tid 1 to smithi10066867-32 2024-04-07T16:24:30.951 INFO:tasks.rados.rados.0.smithi100.stdout:2372: finishing write tid 2 to smithi10066867-32 2024-04-07T16:24:30.954 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 2 v 1137 (ObjNum 659 snap 203 seq_num 659) dirty exists 2024-04-07T16:24:30.955 INFO:tasks.rados.rados.0.smithi100.stdout:2374: done (4 left) 2024-04-07T16:24:30.955 INFO:tasks.rados.rados.0.smithi100.stdout:2376: done (3 left) 2024-04-07T16:24:30.955 INFO:tasks.rados.rados.0.smithi100.stdout:2377: setattr oid 49 current snap is 224 2024-04-07T16:24:30.955 INFO:tasks.rados.rados.0.smithi100.stdout:2372: finishing write tid 3 to smithi10066867-32 2024-04-07T16:24:30.955 INFO:tasks.rados.rados.0.smithi100.stdout:2372: oid 32 updating version 0 to 1099 2024-04-07T16:24:30.955 INFO:tasks.rados.rados.0.smithi100.stdout:2372: oid 32 version 1099 is already newer than 1098 2024-04-07T16:24:30.955 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 32 v 1099 (ObjNum 723 snap 224 seq_num 723) dirty exists 2024-04-07T16:24:30.955 INFO:tasks.rados.rados.0.smithi100.stdout:2372: left oid 32 (ObjNum 723 snap 224 seq_num 723) 2024-04-07T16:24:30.956 INFO:tasks.rados.rados.0.smithi100.stdout:2375: finishing rollback tid 0 to smithi10066867-17 2024-04-07T16:24:30.956 INFO:tasks.rados.rados.0.smithi100.stdout:2375: finishing rollback tid 1 to smithi10066867-17 2024-04-07T16:24:30.956 INFO:tasks.rados.rados.0.smithi100.stdout:2375: finishing rollback tid 2 to smithi10066867-17 2024-04-07T16:24:30.957 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 17 v 1116 (ObjNum 704 snap 214 seq_num 704) dirty exists 2024-04-07T16:24:30.958 INFO:tasks.rados.rados.0.smithi100.stdout:2372: done (3 left) 2024-04-07T16:24:30.958 INFO:tasks.rados.rados.0.smithi100.stdout:2375: done (2 left) 2024-04-07T16:24:30.958 INFO:tasks.rados.rados.0.smithi100.stdout:2378: rmattr oid 47 current snap is 224 2024-04-07T16:24:30.958 INFO:tasks.rados.rados.0.smithi100.stdout:2378: done (2 left) 2024-04-07T16:24:30.958 INFO:tasks.rados.rados.0.smithi100.stdout:2379: read oid 14 snap -1 2024-04-07T16:24:30.958 INFO:tasks.rados.rados.0.smithi100.stdout:2379: expect deleted 2024-04-07T16:24:30.958 INFO:tasks.rados.rados.0.smithi100.stdout:2380: copy_from oid 21 from oid 37 current snap is 224 2024-04-07T16:24:30.958 INFO:tasks.rados.rados.0.smithi100.stdout:2373: finishing write tid 1 to smithi10066867-12 2024-04-07T16:24:30.958 INFO:tasks.rados.rados.0.smithi100.stdout:2381: setattr oid 8 current snap is 224 2024-04-07T16:24:30.958 INFO:tasks.rados.rados.0.smithi100.stdout:2373: finishing write tid 2 to smithi10066867-12 2024-04-07T16:24:30.960 INFO:tasks.rados.rados.0.smithi100.stdout:2379: done (4 left) 2024-04-07T16:24:30.960 INFO:tasks.rados.rados.0.smithi100.stdout:2382: rollback oid 9 current snap is 224 2024-04-07T16:24:30.960 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 9 to 219 2024-04-07T16:24:30.960 INFO:tasks.rados.rados.0.smithi100.stdout:2383: rollback oid 33 current snap is 224 2024-04-07T16:24:30.960 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 33 to 215 2024-04-07T16:24:30.960 INFO:tasks.rados.rados.0.smithi100.stdout:2384: copy_from oid 29 from oid 3 current snap is 224 2024-04-07T16:24:30.960 INFO:tasks.rados.rados.0.smithi100.stdout:2385: read oid 1 snap 220 2024-04-07T16:24:30.960 INFO:tasks.rados.rados.0.smithi100.stdout:2385: expect (ObjNum 685 snap 210 seq_num 685) 2024-04-07T16:24:30.960 INFO:tasks.rados.rados.0.smithi100.stdout:2386: delete oid 18 current snap is 224 2024-04-07T16:24:30.962 INFO:tasks.rados.rados.0.smithi100.stdout:2373: finishing write tid 3 to smithi10066867-12 2024-04-07T16:24:30.962 INFO:tasks.rados.rados.0.smithi100.stdout:2373: oid 12 updating version 0 to 1142 2024-04-07T16:24:30.962 INFO:tasks.rados.rados.0.smithi100.stdout:2373: oid 12 version 1142 is already newer than 1141 2024-04-07T16:24:30.962 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 12 v 1142 (ObjNum 724 snap 224 seq_num 724) dirty exists 2024-04-07T16:24:30.963 INFO:tasks.rados.rados.0.smithi100.stdout:2373: left oid 12 (ObjNum 724 snap 224 seq_num 724) 2024-04-07T16:24:30.964 INFO:tasks.rados.rados.0.smithi100.stdout:2380: finishing copy_from to smithi10066867-21 2024-04-07T16:24:30.964 INFO:tasks.rados.rados.0.smithi100.stdout:2380: got expected ENOENT (src dne) 2024-04-07T16:24:30.964 INFO:tasks.rados.rados.0.smithi100.stdout:2380: finishing copy_from racing read to smithi10066867-21 2024-04-07T16:24:30.965 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 8 v 985 (ObjNum 81 snap 0 seq_num 4238341353) dirty exists 2024-04-07T16:24:30.966 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 49 v 1144 (ObjNum 722 snap 223 seq_num 722) dirty exists 2024-04-07T16:24:30.968 INFO:tasks.rados.rados.0.smithi100.stdout:2382: finishing rollback tid 0 to smithi10066867-9 2024-04-07T16:24:30.972 INFO:tasks.rados.rados.0.smithi100.stdout:2383: finishing rollback tid 0 to smithi10066867-33 2024-04-07T16:24:30.972 INFO:tasks.rados.rados.0.smithi100.stdout:2382: finishing rollback tid 1 to smithi10066867-9 2024-04-07T16:24:30.972 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 9 v 1105 (ObjNum 273 snap 0 seq_num 0) dirty dne 2024-04-07T16:24:30.974 INFO:tasks.rados.rados.0.smithi100.stdout:2385: expect (ObjNum 685 snap 210 seq_num 685) 2024-04-07T16:24:31.035 INFO:tasks.rados.rados.0.smithi100.stdout:2383: finishing rollback tid 1 to smithi10066867-33 2024-04-07T16:24:31.036 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 33 v 1350 (ObjNum 113 snap 0 seq_num 2703004944) dirty dne 2024-04-07T16:24:31.036 INFO:tasks.rados.rados.0.smithi100.stdout:2384: finishing copy_from racing read to smithi10066867-29 2024-04-07T16:24:31.036 INFO:tasks.rados.rados.0.smithi100.stdout:2384: finishing copy_from to smithi10066867-29 2024-04-07T16:24:31.036 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 29 v 1101 (ObjNum 704 snap 214 seq_num 704) dirty exists 2024-04-07T16:24:31.036 INFO:tasks.rados.rados.0.smithi100.stdout:2373: done (8 left) 2024-04-07T16:24:31.036 INFO:tasks.rados.rados.0.smithi100.stdout:2377: done (7 left) 2024-04-07T16:24:31.036 INFO:tasks.rados.rados.0.smithi100.stdout:2380: done (6 left) 2024-04-07T16:24:31.036 INFO:tasks.rados.rados.0.smithi100.stdout:2381: done (5 left) 2024-04-07T16:24:31.036 INFO:tasks.rados.rados.0.smithi100.stdout:2382: done (4 left) 2024-04-07T16:24:31.036 INFO:tasks.rados.rados.0.smithi100.stdout:2383: done (3 left) 2024-04-07T16:24:31.036 INFO:tasks.rados.rados.0.smithi100.stdout:2384: done (2 left) 2024-04-07T16:24:31.036 INFO:tasks.rados.rados.0.smithi100.stdout:2385: done (1 left) 2024-04-07T16:24:31.036 INFO:tasks.rados.rados.0.smithi100.stdout:2386: done (0 left) 2024-04-07T16:24:31.036 INFO:tasks.rados.rados.0.smithi100.stdout:2387: rmattr oid 22 current snap is 224 2024-04-07T16:24:31.037 INFO:tasks.rados.rados.0.smithi100.stdout:2388: delete oid 2 current snap is 224 2024-04-07T16:24:31.040 INFO:tasks.rados.rados.0.smithi100.stdout:2388: done (1 left) 2024-04-07T16:24:31.040 INFO:tasks.rados.rados.0.smithi100.stdout:2389: copy_from oid 10 from oid 21 current snap is 224 2024-04-07T16:24:31.040 INFO:tasks.rados.rados.0.smithi100.stdout:2390: read oid 14 snap 223 2024-04-07T16:24:31.041 INFO:tasks.rados.rados.0.smithi100.stdout:2390: expect deleted 2024-04-07T16:24:31.041 INFO:tasks.rados.rados.0.smithi100.stdout:2391: read oid 23 snap -1 2024-04-07T16:24:31.041 INFO:tasks.rados.rados.0.smithi100.stdout:2391: expect (ObjNum 465 snap 142 seq_num 465) 2024-04-07T16:24:31.041 INFO:tasks.rados.rados.0.smithi100.stdout:2392: snap_create 2024-04-07T16:24:31.042 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 22 v 1157 (ObjNum 664 snap 204 seq_num 664) dirty exists 2024-04-07T16:24:31.043 INFO:tasks.rados.rados.0.smithi100.stdout:2389: finishing copy_from to smithi10066867-10 2024-04-07T16:24:31.043 INFO:tasks.rados.rados.0.smithi100.stdout:2389: got expected ENOENT (src dne) 2024-04-07T16:24:31.043 INFO:tasks.rados.rados.0.smithi100.stdout:2389: finishing copy_from racing read to smithi10066867-10 2024-04-07T16:24:31.058 INFO:tasks.rados.rados.0.smithi100.stdout:2391: expect (ObjNum 465 snap 142 seq_num 465) 2024-04-07T16:24:31.135 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:30 smithi144 bash[66654]: cluster 2024-04-07T16:24:29.921256+0000 mon.a (mon.0) 696 : cluster [DBG] osdmap e516: 8 total, 7 up, 8 in 2024-04-07T16:24:31.136 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:31 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:24:31.136 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:24:31 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:24:31.137 INFO:journalctl@ceph.osd.4.smithi144.stdout:Apr 07 16:24:31 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:24:31.137 INFO:journalctl@ceph.osd.6.smithi144.stdout:Apr 07 16:24:31 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:24:31.138 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:24:31 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:24:31.138 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:24:31 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:24:31.139 INFO:journalctl@ceph.osd.7.smithi144.stdout:Apr 07 16:24:31 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:24:31.139 INFO:journalctl@ceph.mds.b.smithi144.stdout:Apr 07 16:24:31 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:24:31.140 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 16:24:31 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:24:31.140 INFO:journalctl@ceph.osd.5.smithi144.stdout:Apr 07 16:24:30 smithi144 systemd[1]: ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@osd.5.service: Deactivated successfully. 2024-04-07T16:24:31.140 INFO:journalctl@ceph.osd.5.smithi144.stdout:Apr 07 16:24:30 smithi144 systemd[1]: Stopped Ceph osd.5 for 0702e3a8-f4f8-11ee-b649-cb9ed24678a4. 2024-04-07T16:24:31.141 INFO:journalctl@ceph.osd.5.smithi144.stdout:Apr 07 16:24:31 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:24:31.183 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:30 smithi100 bash[74942]: cluster 2024-04-07T16:24:29.921256+0000 mon.a (mon.0) 696 : cluster [DBG] osdmap e516: 8 total, 7 up, 8 in 2024-04-07T16:24:31.184 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:30 smithi100 bash[76202]: cluster 2024-04-07T16:24:29.921256+0000 mon.a (mon.0) 696 : cluster [DBG] osdmap e516: 8 total, 7 up, 8 in 2024-04-07T16:24:31.606 INFO:journalctl@ceph.osd.5.smithi144.stdout:Apr 07 16:24:31 smithi144 systemd[1]: Started Ceph osd.5 for 0702e3a8-f4f8-11ee-b649-cb9ed24678a4. 2024-04-07T16:24:31.830 INFO:tasks.rados.rados.0.smithi100.stdout:2387: done (4 left) 2024-04-07T16:24:31.830 INFO:tasks.rados.rados.0.smithi100.stdout:2389: done (3 left) 2024-04-07T16:24:31.830 INFO:tasks.rados.rados.0.smithi100.stdout:2390: done (2 left) 2024-04-07T16:24:31.830 INFO:tasks.rados.rados.0.smithi100.stdout:2391: done (1 left) 2024-04-07T16:24:31.830 INFO:tasks.rados.rados.0.smithi100.stdout:2392: done (0 left) 2024-04-07T16:24:31.831 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 24 current snap is 225 2024-04-07T16:24:31.831 INFO:tasks.rados.rados.0.smithi100.stdout:2393: seq_num 728 ranges {1867776=196608} 2024-04-07T16:24:31.870 INFO:tasks.rados.rados.0.smithi100.stdout:2393: writing smithi10066867-24 from 1867776 to 2064384 tid 1 2024-04-07T16:24:31.871 INFO:tasks.rados.rados.0.smithi100.stdout:2394: delete oid 9 current snap is 225 2024-04-07T16:24:31.874 INFO:tasks.rados.rados.0.smithi100.stdout:2394: done (1 left) 2024-04-07T16:24:31.874 INFO:tasks.rados.rados.0.smithi100.stdout:2395: read oid 27 snap -1 2024-04-07T16:24:31.874 INFO:tasks.rados.rados.0.smithi100.stdout:2395: expect (ObjNum 691 snap 211 seq_num 691) 2024-04-07T16:24:31.874 INFO:tasks.rados.rados.0.smithi100.stdout:2396: snap_remove snap 220 2024-04-07T16:24:32.184 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:31 smithi100 bash[74942]: cluster 2024-04-07T16:24:30.703941+0000 mgr.y (mgr.25175) 218 : cluster [DBG] pgmap v366: 121 pgs: 2 active+undersized+degraded, 12 peering, 13 stale+active+clean, 6 active+undersized, 88 active+clean; 336 MiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 13 MiB/s rd, 8.7 MiB/s wr, 31 op/s; 10/402 objects degraded (2.488%) 2024-04-07T16:24:32.184 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:31 smithi100 bash[74942]: cluster 2024-04-07T16:24:30.923865+0000 mon.a (mon.0) 697 : cluster [WRN] Health check failed: Degraded data redundancy: 10/402 objects degraded (2.488%), 2 pgs degraded (PG_DEGRADED) 2024-04-07T16:24:32.184 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:31 smithi100 bash[74942]: cluster 2024-04-07T16:24:30.943722+0000 mon.a (mon.0) 698 : cluster [DBG] osdmap e517: 8 total, 7 up, 8 in 2024-04-07T16:24:32.184 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:31 smithi100 bash[74942]: audit 2024-04-07T16:24:31.207281+0000 mon.a (mon.0) 699 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:24:32.184 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:31 smithi100 bash[74942]: audit 2024-04-07T16:24:31.218645+0000 mon.a (mon.0) 700 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:24:32.184 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:31 smithi100 bash[74942]: audit 2024-04-07T16:24:31.265505+0000 mon.a (mon.0) 701 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:24:32.184 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:31 smithi100 bash[74942]: audit 2024-04-07T16:24:31.722195+0000 mon.a (mon.0) 702 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:24:32.184 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:31 smithi100 bash[74942]: audit 2024-04-07T16:24:31.724499+0000 mon.a (mon.0) 703 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:24:32.184 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:31 smithi100 bash[74942]: cluster 2024-04-07T16:24:31.847529+0000 mon.a (mon.0) 704 : cluster [DBG] osdmap e518: 8 total, 7 up, 8 in 2024-04-07T16:24:32.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:31 smithi100 bash[74942]: audit 2024-04-07T16:24:31.859974+0000 mon.a (mon.0) 705 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:24:32.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:31 smithi100 bash[74942]: audit 2024-04-07T16:24:31.867969+0000 mon.a (mon.0) 706 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:24:32.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:31 smithi100 bash[76202]: cluster 2024-04-07T16:24:30.703941+0000 mgr.y (mgr.25175) 218 : cluster [DBG] pgmap v366: 121 pgs: 2 active+undersized+degraded, 12 peering, 13 stale+active+clean, 6 active+undersized, 88 active+clean; 336 MiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 13 MiB/s rd, 8.7 MiB/s wr, 31 op/s; 10/402 objects degraded (2.488%) 2024-04-07T16:24:32.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:31 smithi100 bash[76202]: cluster 2024-04-07T16:24:30.923865+0000 mon.a (mon.0) 697 : cluster [WRN] Health check failed: Degraded data redundancy: 10/402 objects degraded (2.488%), 2 pgs degraded (PG_DEGRADED) 2024-04-07T16:24:32.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:31 smithi100 bash[76202]: cluster 2024-04-07T16:24:30.943722+0000 mon.a (mon.0) 698 : cluster [DBG] osdmap e517: 8 total, 7 up, 8 in 2024-04-07T16:24:32.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:31 smithi100 bash[76202]: audit 2024-04-07T16:24:31.207281+0000 mon.a (mon.0) 699 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:24:32.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:31 smithi100 bash[76202]: audit 2024-04-07T16:24:31.218645+0000 mon.a (mon.0) 700 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:24:32.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:31 smithi100 bash[76202]: audit 2024-04-07T16:24:31.265505+0000 mon.a (mon.0) 701 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:24:32.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:31 smithi100 bash[76202]: audit 2024-04-07T16:24:31.722195+0000 mon.a (mon.0) 702 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:24:32.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:31 smithi100 bash[76202]: audit 2024-04-07T16:24:31.724499+0000 mon.a (mon.0) 703 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:24:32.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:31 smithi100 bash[76202]: cluster 2024-04-07T16:24:31.847529+0000 mon.a (mon.0) 704 : cluster [DBG] osdmap e518: 8 total, 7 up, 8 in 2024-04-07T16:24:32.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:31 smithi100 bash[76202]: audit 2024-04-07T16:24:31.859974+0000 mon.a (mon.0) 705 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:24:32.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:31 smithi100 bash[76202]: audit 2024-04-07T16:24:31.867969+0000 mon.a (mon.0) 706 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:24:32.304 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:31 smithi144 bash[66654]: cluster 2024-04-07T16:24:30.703941+0000 mgr.y (mgr.25175) 218 : cluster [DBG] pgmap v366: 121 pgs: 2 active+undersized+degraded, 12 peering, 13 stale+active+clean, 6 active+undersized, 88 active+clean; 336 MiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 13 MiB/s rd, 8.7 MiB/s wr, 31 op/s; 10/402 objects degraded (2.488%) 2024-04-07T16:24:32.304 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:31 smithi144 bash[66654]: cluster 2024-04-07T16:24:30.923865+0000 mon.a (mon.0) 697 : cluster [WRN] Health check failed: Degraded data redundancy: 10/402 objects degraded (2.488%), 2 pgs degraded (PG_DEGRADED) 2024-04-07T16:24:32.304 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:31 smithi144 bash[66654]: cluster 2024-04-07T16:24:30.943722+0000 mon.a (mon.0) 698 : cluster [DBG] osdmap e517: 8 total, 7 up, 8 in 2024-04-07T16:24:32.304 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:31 smithi144 bash[66654]: audit 2024-04-07T16:24:31.207281+0000 mon.a (mon.0) 699 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:24:32.304 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:31 smithi144 bash[66654]: audit 2024-04-07T16:24:31.218645+0000 mon.a (mon.0) 700 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:24:32.304 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:31 smithi144 bash[66654]: audit 2024-04-07T16:24:31.265505+0000 mon.a (mon.0) 701 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:24:32.304 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:31 smithi144 bash[66654]: audit 2024-04-07T16:24:31.722195+0000 mon.a (mon.0) 702 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:24:32.304 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:31 smithi144 bash[66654]: audit 2024-04-07T16:24:31.724499+0000 mon.a (mon.0) 703 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:24:32.304 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:31 smithi144 bash[66654]: cluster 2024-04-07T16:24:31.847529+0000 mon.a (mon.0) 704 : cluster [DBG] osdmap e518: 8 total, 7 up, 8 in 2024-04-07T16:24:32.304 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:31 smithi144 bash[66654]: audit 2024-04-07T16:24:31.859974+0000 mon.a (mon.0) 705 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:24:32.304 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:31 smithi144 bash[66654]: audit 2024-04-07T16:24:31.867969+0000 mon.a (mon.0) 706 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:24:32.606 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:24:32 smithi144 bash[59794]: ts=2024-04-07T16:24:32.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:24:32.606 INFO:journalctl@ceph.osd.5.smithi144.stdout:Apr 07 16:24:32 smithi144 bash[75131]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-07T16:24:32.606 INFO:journalctl@ceph.osd.5.smithi144.stdout:Apr 07 16:24:32 smithi144 bash[75131]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-5 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-04-07T16:24:32.606 INFO:journalctl@ceph.osd.5.smithi144.stdout:Apr 07 16:24:32 smithi144 bash[75131]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-04-07T16:24:32.606 INFO:journalctl@ceph.osd.5.smithi144.stdout:Apr 07 16:24:32 smithi144 bash[75131]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-04-07T16:24:32.606 INFO:journalctl@ceph.osd.5.smithi144.stdout:Apr 07 16:24:32 smithi144 bash[75131]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-5/block 2024-04-07T16:24:32.606 INFO:journalctl@ceph.osd.5.smithi144.stdout:Apr 07 16:24:32 smithi144 bash[75131]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-07T16:24:32.606 INFO:journalctl@ceph.osd.5.smithi144.stdout:Apr 07 16:24:32 smithi144 bash[75131]: --> ceph-volume raw activate successful for osd ID: 5 2024-04-07T16:24:32.859 INFO:tasks.rados.rados.0.smithi100.stdout:2396: done (2 left) 2024-04-07T16:24:32.859 INFO:tasks.rados.rados.0.smithi100.stdout:2397: delete oid 8 current snap is 225 2024-04-07T16:24:32.859 INFO:tasks.rados.rados.0.smithi100.stdout:2393: finishing write tid 1 to smithi10066867-24 2024-04-07T16:24:32.859 INFO:tasks.rados.rados.0.smithi100.stdout:2393: finishing write tid 2 to smithi10066867-24 2024-04-07T16:24:32.860 INFO:tasks.rados.rados.0.smithi100.stdout:2393: finishing write tid 3 to smithi10066867-24 2024-04-07T16:24:32.860 INFO:tasks.rados.rados.0.smithi100.stdout:2393: oid 24 updating version 0 to 1118 2024-04-07T16:24:32.860 INFO:tasks.rados.rados.0.smithi100.stdout:2393: oid 24 version 1118 is already newer than 1117 2024-04-07T16:24:32.860 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 24 v 1118 (ObjNum 728 snap 225 seq_num 728) dirty exists 2024-04-07T16:24:32.860 INFO:tasks.rados.rados.0.smithi100.stdout:2393: left oid 24 (ObjNum 728 snap 225 seq_num 728) 2024-04-07T16:24:32.860 INFO:tasks.rados.rados.0.smithi100.stdout:2395: expect (ObjNum 691 snap 211 seq_num 691) 2024-04-07T16:24:33.373 INFO:tasks.rados.rados.0.smithi100.stdout:2393: done (2 left) 2024-04-07T16:24:33.373 INFO:tasks.rados.rados.0.smithi100.stdout:2395: done (1 left) 2024-04-07T16:24:33.373 INFO:tasks.rados.rados.0.smithi100.stdout:2397: done (0 left) 2024-04-07T16:24:33.373 INFO:tasks.rados.rados.0.smithi100.stdout:2398: rollback oid 39 current snap is 225 2024-04-07T16:24:33.373 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 39 to 219 2024-04-07T16:24:33.373 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 21 current snap is 225 2024-04-07T16:24:33.373 INFO:tasks.rados.rados.0.smithi100.stdout:2399: seq_num 731 ranges {0=245760} 2024-04-07T16:24:33.378 INFO:tasks.rados.rados.0.smithi100.stdout:2399: writing smithi10066867-21 from 0 to 245760 tid 1 2024-04-07T16:24:33.378 INFO:tasks.rados.rados.0.smithi100.stdout:2400: read oid 19 snap 219 2024-04-07T16:24:33.378 INFO:tasks.rados.rados.0.smithi100.stdout:2400: expect deleted 2024-04-07T16:24:33.378 INFO:tasks.rados.rados.0.smithi100.stdout:2401: rmattr oid 35 current snap is 225 2024-04-07T16:24:33.379 INFO:tasks.rados.rados.0.smithi100.stdout:2401: done (3 left) 2024-04-07T16:24:33.379 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 24 current snap is 225 2024-04-07T16:24:33.379 INFO:tasks.rados.rados.0.smithi100.stdout:2402: seq_num 732 ranges {2064384=262144} 2024-04-07T16:24:33.417 INFO:tasks.rados.rados.0.smithi100.stdout:2402: writing smithi10066867-24 from 2064384 to 2326528 tid 1 2024-04-07T16:24:33.417 INFO:tasks.rados.rados.0.smithi100.stdout:2403: snap_create 2024-04-07T16:24:33.417 INFO:tasks.rados.rados.0.smithi100.stdout:2398: finishing rollback tid 0 to smithi10066867-39 2024-04-07T16:24:33.417 INFO:tasks.rados.rados.0.smithi100.stdout:2398: finishing rollback tid 1 to smithi10066867-39 2024-04-07T16:24:33.417 INFO:tasks.rados.rados.0.smithi100.stdout:2398: finishing rollback tid 2 to smithi10066867-39 2024-04-07T16:24:33.417 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 39 v 1120 (ObjNum 560 snap 171 seq_num 560) dirty exists 2024-04-07T16:24:33.417 INFO:tasks.rados.rados.0.smithi100.stdout:2399: finishing write tid 1 to smithi10066867-21 2024-04-07T16:24:33.417 INFO:tasks.rados.rados.0.smithi100.stdout:2399: finishing write tid 2 to smithi10066867-21 2024-04-07T16:24:33.417 INFO:tasks.rados.rados.0.smithi100.stdout:2399: finishing write tid 3 to smithi10066867-21 2024-04-07T16:24:33.417 INFO:tasks.rados.rados.0.smithi100.stdout:2399: oid 21 updating version 0 to 825 2024-04-07T16:24:33.417 INFO:tasks.rados.rados.0.smithi100.stdout:2399: oid 21 updating version 825 to 826 2024-04-07T16:24:33.418 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 21 v 826 (ObjNum 731 snap 225 seq_num 731) dirty exists 2024-04-07T16:24:33.418 INFO:tasks.rados.rados.0.smithi100.stdout:2399: left oid 21 (ObjNum 731 snap 225 seq_num 731) 2024-04-07T16:24:33.421 INFO:tasks.rados.rados.0.smithi100.stdout:2402: finishing write tid 1 to smithi10066867-24 2024-04-07T16:24:33.422 INFO:tasks.rados.rados.0.smithi100.stdout:2402: finishing write tid 2 to smithi10066867-24 2024-04-07T16:24:33.424 INFO:tasks.rados.rados.0.smithi100.stdout:2402: finishing write tid 3 to smithi10066867-24 2024-04-07T16:24:33.425 INFO:tasks.rados.rados.0.smithi100.stdout:2402: oid 24 updating version 0 to 1119 2024-04-07T16:24:33.425 INFO:tasks.rados.rados.0.smithi100.stdout:2402: oid 24 updating version 1119 to 1120 2024-04-07T16:24:33.425 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 24 v 1120 (ObjNum 732 snap 225 seq_num 732) dirty exists 2024-04-07T16:24:33.425 INFO:tasks.rados.rados.0.smithi100.stdout:2402: left oid 24 (ObjNum 732 snap 225 seq_num 732) 2024-04-07T16:24:33.929 INFO:tasks.rados.rados.0.smithi100.stdout:2398: done (4 left) 2024-04-07T16:24:33.929 INFO:tasks.rados.rados.0.smithi100.stdout:2399: done (3 left) 2024-04-07T16:24:33.929 INFO:tasks.rados.rados.0.smithi100.stdout:2400: done (2 left) 2024-04-07T16:24:33.929 INFO:tasks.rados.rados.0.smithi100.stdout:2402: done (1 left) 2024-04-07T16:24:33.929 INFO:tasks.rados.rados.0.smithi100.stdout:2403: done (0 left) 2024-04-07T16:24:33.929 INFO:tasks.rados.rados.0.smithi100.stdout:2404: read oid 13 snap 186 2024-04-07T16:24:33.929 INFO:tasks.rados.rados.0.smithi100.stdout:2404: expect (ObjNum 570 snap 177 seq_num 570) 2024-04-07T16:24:33.929 INFO:tasks.rados.rados.0.smithi100.stdout:2405: rmattr oid 45 current snap is 226 2024-04-07T16:24:33.929 INFO:tasks.rados.rados.0.smithi100.stdout:2405: done (1 left) 2024-04-07T16:24:33.929 INFO:tasks.rados.rados.0.smithi100.stdout:2406: read oid 39 snap -1 2024-04-07T16:24:33.929 INFO:tasks.rados.rados.0.smithi100.stdout:2406: expect (ObjNum 560 snap 171 seq_num 560) 2024-04-07T16:24:33.929 INFO:tasks.rados.rados.0.smithi100.stdout:2407: copy_from oid 22 from oid 18 current snap is 226 2024-04-07T16:24:33.929 INFO:tasks.rados.rados.0.smithi100.stdout:2408: rollback oid 41 current snap is 226 2024-04-07T16:24:33.929 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 41 to 225 2024-04-07T16:24:33.930 INFO:tasks.rados.rados.0.smithi100.stdout:2409: read oid 16 snap 223 2024-04-07T16:24:33.930 INFO:tasks.rados.rados.0.smithi100.stdout:2409: expect (ObjNum 465 snap 142 seq_num 465) 2024-04-07T16:24:33.930 INFO:tasks.rados.rados.0.smithi100.stdout:2410: rollback oid 20 current snap is 226 2024-04-07T16:24:33.930 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 20 to 218 2024-04-07T16:24:33.930 INFO:tasks.rados.rados.0.smithi100.stdout:2411: rollback oid 50 current snap is 226 2024-04-07T16:24:33.930 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 50 to 218 2024-04-07T16:24:33.930 INFO:tasks.rados.rados.0.smithi100.stdout:2412: rollback oid 2 current snap is 226 2024-04-07T16:24:33.930 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 2 to 186 2024-04-07T16:24:33.930 INFO:tasks.rados.rados.0.smithi100.stdout:2413: read oid 11 snap -1 2024-04-07T16:24:33.930 INFO:tasks.rados.rados.0.smithi100.stdout:2413: expect (ObjNum 718 snap 221 seq_num 718) 2024-04-07T16:24:33.930 INFO:tasks.rados.rados.0.smithi100.stdout:2414: rmattr oid 14 current snap is 226 2024-04-07T16:24:33.930 INFO:tasks.rados.rados.0.smithi100.stdout:2414: done (9 left) 2024-04-07T16:24:33.930 INFO:tasks.rados.rados.0.smithi100.stdout:2415: read oid 47 snap -1 2024-04-07T16:24:33.930 INFO:tasks.rados.rados.0.smithi100.stdout:2415: expect (ObjNum 684 snap 210 seq_num 684) 2024-04-07T16:24:33.930 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 3 current snap is 226 2024-04-07T16:24:33.931 INFO:tasks.rados.rados.0.smithi100.stdout:2416: seq_num 733 ranges {2203648=278528} 2024-04-07T16:24:33.968 INFO:tasks.rados.rados.0.smithi100.stdout:2416: writing smithi10066867-3 from 2203648 to 2482176 tid 1 2024-04-07T16:24:33.968 INFO:tasks.rados.rados.0.smithi100.stdout:2417: read oid 49 snap 215 2024-04-07T16:24:33.968 INFO:tasks.rados.rados.0.smithi100.stdout:2417: expect (ObjNum 686 snap 210 seq_num 686) 2024-04-07T16:24:33.968 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 1 current snap is 226 2024-04-07T16:24:33.968 INFO:tasks.rados.rados.0.smithi100.stdout:2418: seq_num 734 ranges {1884160=294912} 2024-04-07T16:24:33.995 INFO:tasks.rados.rados.0.smithi100.stdout:2418: writing smithi10066867-1 from 1884160 to 2179072 tid 1 2024-04-07T16:24:33.995 INFO:tasks.rados.rados.0.smithi100.stdout:2419: delete oid 31 current snap is 226 2024-04-07T16:24:33.995 INFO:tasks.rados.rados.0.smithi100.stdout:2415: expect (ObjNum 684 snap 210 seq_num 684) 2024-04-07T16:24:34.060 INFO:tasks.rados.rados.0.smithi100.stdout:2411: finishing rollback tid 0 to smithi10066867-50 2024-04-07T16:24:34.061 INFO:tasks.rados.rados.0.smithi100.stdout:2412: finishing rollback tid 1 to smithi10066867-2 2024-04-07T16:24:34.061 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 2 v 1138 (ObjNum 33 snap 0 seq_num 2670634328) dirty dne 2024-04-07T16:24:34.061 INFO:tasks.rados.rados.0.smithi100.stdout:2411: finishing rollback tid 1 to smithi10066867-50 2024-04-07T16:24:34.061 INFO:tasks.rados.rados.0.smithi100.stdout:2411: finishing rollback tid 2 to smithi10066867-50 2024-04-07T16:24:34.061 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 50 v 1025 (ObjNum 657 snap 203 seq_num 657) dirty exists 2024-04-07T16:24:34.061 INFO:tasks.rados.rados.0.smithi100.stdout:2411: done (13 left) 2024-04-07T16:24:34.061 INFO:tasks.rados.rados.0.smithi100.stdout:2412: done (12 left) 2024-04-07T16:24:34.061 INFO:tasks.rados.rados.0.smithi100.stdout:2415: done (11 left) 2024-04-07T16:24:34.061 INFO:tasks.rados.rados.0.smithi100.stdout:2419: done (10 left) 2024-04-07T16:24:34.061 INFO:tasks.rados.rados.0.smithi100.stdout:2420: snap_create 2024-04-07T16:24:34.061 INFO:tasks.rados.rados.0.smithi100.stdout:2407: finishing copy_from to smithi10066867-22 2024-04-07T16:24:34.061 INFO:tasks.rados.rados.0.smithi100.stdout:2407: got expected ENOENT (src dne) 2024-04-07T16:24:34.061 INFO:tasks.rados.rados.0.smithi100.stdout:2413: expect (ObjNum 718 snap 221 seq_num 718) 2024-04-07T16:24:34.062 INFO:teuthology.orchestra.run.smithi100.stdout:true 2024-04-07T16:24:34.062 INFO:tasks.rados.rados.0.smithi100.stdout:2407: finishing copy_from racing read to smithi10066867-22 2024-04-07T16:24:34.062 INFO:tasks.rados.rados.0.smithi100.stdout:2410: finishing rollback tid 0 to smithi10066867-20 2024-04-07T16:24:34.063 INFO:tasks.rados.rados.0.smithi100.stdout:2416: finishing write tid 1 to smithi10066867-3 2024-04-07T16:24:34.063 INFO:tasks.rados.rados.0.smithi100.stdout:2416: finishing write tid 2 to smithi10066867-3 2024-04-07T16:24:34.063 INFO:tasks.rados.rados.0.smithi100.stdout:2416: finishing write tid 3 to smithi10066867-3 2024-04-07T16:24:34.063 INFO:tasks.rados.rados.0.smithi100.stdout:2416: oid 3 updating version 0 to 1105 2024-04-07T16:24:34.063 INFO:tasks.rados.rados.0.smithi100.stdout:2416: oid 3 version 1105 is already newer than 1104 2024-04-07T16:24:34.063 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 3 v 1105 (ObjNum 733 snap 226 seq_num 733) dirty exists 2024-04-07T16:24:34.063 INFO:tasks.rados.rados.0.smithi100.stdout:2416: left oid 3 (ObjNum 733 snap 226 seq_num 733) 2024-04-07T16:24:34.063 INFO:tasks.rados.rados.0.smithi100.stdout:2408: finishing rollback tid 0 to smithi10066867-41 2024-04-07T16:24:34.063 INFO:tasks.rados.rados.0.smithi100.stdout:2408: finishing rollback tid 1 to smithi10066867-41 2024-04-07T16:24:34.063 INFO:tasks.rados.rados.0.smithi100.stdout:2408: finishing rollback tid 2 to smithi10066867-41 2024-04-07T16:24:34.063 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 41 v 1150 (ObjNum 684 snap 210 seq_num 684) dirty exists 2024-04-07T16:24:34.063 INFO:tasks.rados.rados.0.smithi100.stdout:2410: finishing rollback tid 1 to smithi10066867-20 2024-04-07T16:24:34.063 INFO:tasks.rados.rados.0.smithi100.stdout:2410: finishing rollback tid 2 to smithi10066867-20 2024-04-07T16:24:34.063 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 20 v 1160 (ObjNum 610 snap 188 seq_num 610) dirty exists 2024-04-07T16:24:34.063 INFO:tasks.rados.rados.0.smithi100.stdout:2409: expect (ObjNum 465 snap 142 seq_num 465) 2024-04-07T16:24:34.106 INFO:journalctl@ceph.osd.5.smithi144.stdout:Apr 07 16:24:33 smithi144 bash[75374]: debug 2024-04-07T16:24:33.986+0000 7f4affdf8700 1 -- 172.21.15.144:0/2904710426 <== mon.1 v2:172.21.15.144:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55dcfcad3680 con 0x55dcfcb11000 2024-04-07T16:24:34.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:33 smithi144 bash[66654]: cluster 2024-04-07T16:24:32.705348+0000 mgr.y (mgr.25175) 219 : cluster [DBG] pgmap v369: 121 pgs: 8 active+undersized+degraded, 12 peering, 30 active+undersized, 71 active+clean; 336 MiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 7.0 MiB/s rd, 4.3 MiB/s wr, 20 op/s; 31/408 objects degraded (7.598%) 2024-04-07T16:24:34.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:33 smithi144 bash[66654]: cluster 2024-04-07T16:24:32.877622+0000 mon.a (mon.0) 707 : cluster [DBG] osdmap e519: 8 total, 7 up, 8 in 2024-04-07T16:24:34.183 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:33 smithi100 bash[74942]: cluster 2024-04-07T16:24:32.705348+0000 mgr.y (mgr.25175) 219 : cluster [DBG] pgmap v369: 121 pgs: 8 active+undersized+degraded, 12 peering, 30 active+undersized, 71 active+clean; 336 MiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 7.0 MiB/s rd, 4.3 MiB/s wr, 20 op/s; 31/408 objects degraded (7.598%) 2024-04-07T16:24:34.184 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:33 smithi100 bash[74942]: cluster 2024-04-07T16:24:32.877622+0000 mon.a (mon.0) 707 : cluster [DBG] osdmap e519: 8 total, 7 up, 8 in 2024-04-07T16:24:34.184 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:33 smithi100 bash[76202]: cluster 2024-04-07T16:24:32.705348+0000 mgr.y (mgr.25175) 219 : cluster [DBG] pgmap v369: 121 pgs: 8 active+undersized+degraded, 12 peering, 30 active+undersized, 71 active+clean; 336 MiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 7.0 MiB/s rd, 4.3 MiB/s wr, 20 op/s; 31/408 objects degraded (7.598%) 2024-04-07T16:24:34.184 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:33 smithi100 bash[76202]: cluster 2024-04-07T16:24:32.877622+0000 mon.a (mon.0) 707 : cluster [DBG] osdmap e519: 8 total, 7 up, 8 in 2024-04-07T16:24:34.229 INFO:tasks.rados.rados.0.smithi100.stdout:2417: expect (ObjNum 686 snap 210 seq_num 686) 2024-04-07T16:24:34.291 INFO:tasks.rados.rados.0.smithi100.stdout:2404: expect (ObjNum 570 snap 177 seq_num 570) 2024-04-07T16:24:34.429 INFO:teuthology.orchestra.run.smithi100.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-07T16:24:34.429 INFO:teuthology.orchestra.run.smithi100.stdout:alertmanager.a smithi100 *:9093,9094 running (10m) 52s ago 11m 14.4M - 0.25.0 c8568f914cd2 5143075e7d26 2024-04-07T16:24:34.429 INFO:teuthology.orchestra.run.smithi100.stdout:grafana.a smithi144 *:3000 running (10m) 23s ago 10m 79.5M - 9.4.7 954c08fa6188 7180c0708acd 2024-04-07T16:24:34.429 INFO:teuthology.orchestra.run.smithi100.stdout:mds.a smithi100 running (12m) 52s ago 12m 15.7M - 18.2.2-1171-g267c8ff5 8a4a01e7186c 92593b0c28ae 2024-04-07T16:24:34.429 INFO:teuthology.orchestra.run.smithi100.stdout:mds.b smithi144 running (12m) 23s ago 12m 19.8M - 18.2.2-1171-g267c8ff5 8a4a01e7186c 3df2ee3e17a4 2024-04-07T16:24:34.430 INFO:teuthology.orchestra.run.smithi100.stdout:mgr.x smithi144 *:8443,9283,8765 running (4m) 23s ago 19m 440M - 19.0.0-2785-gadce92d0 c0ecd4226ac9 233745c292a6 2024-04-07T16:24:34.430 INFO:teuthology.orchestra.run.smithi100.stdout:mgr.y smithi100 *:8443,9283,8765 running (5m) 52s ago 22m 503M - 19.0.0-2785-gadce92d0 c0ecd4226ac9 254dc65cdaa7 2024-04-07T16:24:34.430 INFO:teuthology.orchestra.run.smithi100.stdout:mon.a smithi100 running (4m) 52s ago 22m 54.3M 2048M 19.0.0-2785-gadce92d0 c0ecd4226ac9 718e12655c04 2024-04-07T16:24:34.430 INFO:teuthology.orchestra.run.smithi100.stdout:mon.b smithi144 running (3m) 23s ago 20m 43.0M 2048M 19.0.0-2785-gadce92d0 c0ecd4226ac9 aef817664bff 2024-04-07T16:24:34.430 INFO:teuthology.orchestra.run.smithi100.stdout:mon.c smithi100 running (3m) 52s ago 20m 46.2M 2048M 19.0.0-2785-gadce92d0 c0ecd4226ac9 63e889040f6e 2024-04-07T16:24:34.430 INFO:teuthology.orchestra.run.smithi100.stdout:node-exporter.a smithi100 *:9100 running (11m) 52s ago 11m 8955k - 1.5.0 0da6a335fe13 4bce264c1b26 2024-04-07T16:24:34.430 INFO:teuthology.orchestra.run.smithi100.stdout:node-exporter.b smithi144 *:9100 running (11m) 23s ago 11m 8492k - 1.5.0 0da6a335fe13 b6963413c560 2024-04-07T16:24:34.430 INFO:teuthology.orchestra.run.smithi100.stdout:osd.0 smithi100 running (2m) 52s ago 18m 199M 2524M 19.0.0-2785-gadce92d0 c0ecd4226ac9 a6e83c9d1abc 2024-04-07T16:24:34.430 INFO:teuthology.orchestra.run.smithi100.stdout:osd.1 smithi100 running (115s) 52s ago 18m 197M 2524M 19.0.0-2785-gadce92d0 c0ecd4226ac9 c50fe40079b6 2024-04-07T16:24:34.430 INFO:teuthology.orchestra.run.smithi100.stdout:osd.2 smithi100 running (85s) 52s ago 17m 253M 2524M 19.0.0-2785-gadce92d0 c0ecd4226ac9 8e631c43064b 2024-04-07T16:24:34.430 INFO:teuthology.orchestra.run.smithi100.stdout:osd.3 smithi100 running (57s) 52s ago 16m 11.8M 2524M 19.0.0-2785-gadce92d0 c0ecd4226ac9 402931247c79 2024-04-07T16:24:34.430 INFO:teuthology.orchestra.run.smithi100.stdout:osd.4 smithi144 running (27s) 23s ago 15m - 2524M 19.0.0-2785-gadce92d0 c0ecd4226ac9 ebbb7be95d33 2024-04-07T16:24:34.430 INFO:teuthology.orchestra.run.smithi100.stdout:osd.5 smithi144 starting - - - 2524M 2024-04-07T16:24:34.431 INFO:teuthology.orchestra.run.smithi100.stdout:osd.6 smithi144 running (14m) 23s ago 14m 130M 2524M 18.2.2-1171-g267c8ff5 8a4a01e7186c 32b7f95d77f0 2024-04-07T16:24:34.431 INFO:teuthology.orchestra.run.smithi100.stdout:osd.7 smithi144 running (13m) 23s ago 13m 206M 2524M 18.2.2-1171-g267c8ff5 8a4a01e7186c 45f7709e576a 2024-04-07T16:24:34.431 INFO:teuthology.orchestra.run.smithi100.stdout:prometheus.a smithi144 *:9095 running (4m) 23s ago 11m 52.7M - 2.43.0 a07b618ecd1d 2d0a8131f303 2024-04-07T16:24:34.463 INFO:tasks.rados.rados.0.smithi100.stdout:2406: expect (ObjNum 560 snap 171 seq_num 560) 2024-04-07T16:24:34.727 INFO:tasks.rados.rados.0.smithi100.stdout:2418: finishing write tid 1 to smithi10066867-1 2024-04-07T16:24:34.727 INFO:tasks.rados.rados.0.smithi100.stdout:2418: finishing write tid 2 to smithi10066867-1 2024-04-07T16:24:34.727 INFO:tasks.rados.rados.0.smithi100.stdout:2418: finishing write tid 3 to smithi10066867-1 2024-04-07T16:24:34.727 INFO:tasks.rados.rados.0.smithi100.stdout:2418: oid 1 updating version 0 to 1067 2024-04-07T16:24:34.727 INFO:tasks.rados.rados.0.smithi100.stdout:2418: oid 1 updating version 1067 to 1068 2024-04-07T16:24:34.727 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 1 v 1068 (ObjNum 734 snap 226 seq_num 734) dirty exists 2024-04-07T16:24:34.728 INFO:tasks.rados.rados.0.smithi100.stdout:2418: left oid 1 (ObjNum 734 snap 226 seq_num 734) 2024-04-07T16:24:34.856 INFO:journalctl@ceph.osd.5.smithi144.stdout:Apr 07 16:24:34 smithi144 bash[75374]: debug 2024-04-07T16:24:34.530+0000 7f4b08f907c0 -1 Falling back to public interface 2024-04-07T16:24:34.864 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-04-07T16:24:34.864 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": { 2024-04-07T16:24:34.864 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph version 19.0.0-2785-gadce92d0 (adce92d056533f22dc1ee37e3141d098b046ec1b) squid (dev)": 3 2024-04-07T16:24:34.864 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-07T16:24:34.864 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": { 2024-04-07T16:24:34.864 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph version 19.0.0-2785-gadce92d0 (adce92d056533f22dc1ee37e3141d098b046ec1b) squid (dev)": 2 2024-04-07T16:24:34.864 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-07T16:24:34.864 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": { 2024-04-07T16:24:34.864 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph version 18.2.2-1171-g267c8ff5 (267c8ff5b1ff5815581b3b2381d19360e081b37b) reef (stable)": 2, 2024-04-07T16:24:34.864 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph version 19.0.0-2785-gadce92d0 (adce92d056533f22dc1ee37e3141d098b046ec1b) squid (dev)": 5 2024-04-07T16:24:34.865 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-07T16:24:34.865 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": { 2024-04-07T16:24:34.865 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph version 18.2.2-1171-g267c8ff5 (267c8ff5b1ff5815581b3b2381d19360e081b37b) reef (stable)": 2 2024-04-07T16:24:34.865 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-07T16:24:34.865 INFO:teuthology.orchestra.run.smithi100.stdout: "overall": { 2024-04-07T16:24:34.865 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph version 18.2.2-1171-g267c8ff5 (267c8ff5b1ff5815581b3b2381d19360e081b37b) reef (stable)": 4, 2024-04-07T16:24:34.865 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph version 19.0.0-2785-gadce92d0 (adce92d056533f22dc1ee37e3141d098b046ec1b) squid (dev)": 10 2024-04-07T16:24:34.865 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-04-07T16:24:34.865 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-04-07T16:24:34.943 INFO:tasks.rados.rados.0.smithi100.stdout:2404: done (10 left) 2024-04-07T16:24:34.943 INFO:tasks.rados.rados.0.smithi100.stdout:2406: done (9 left) 2024-04-07T16:24:34.943 INFO:tasks.rados.rados.0.smithi100.stdout:2407: done (8 left) 2024-04-07T16:24:34.944 INFO:tasks.rados.rados.0.smithi100.stdout:2408: done (7 left) 2024-04-07T16:24:34.944 INFO:tasks.rados.rados.0.smithi100.stdout:2409: done (6 left) 2024-04-07T16:24:34.944 INFO:tasks.rados.rados.0.smithi100.stdout:2410: done (5 left) 2024-04-07T16:24:34.944 INFO:tasks.rados.rados.0.smithi100.stdout:2413: done (4 left) 2024-04-07T16:24:34.944 INFO:tasks.rados.rados.0.smithi100.stdout:2416: done (3 left) 2024-04-07T16:24:34.944 INFO:tasks.rados.rados.0.smithi100.stdout:2417: done (2 left) 2024-04-07T16:24:34.944 INFO:tasks.rados.rados.0.smithi100.stdout:2418: done (1 left) 2024-04-07T16:24:34.944 INFO:tasks.rados.rados.0.smithi100.stdout:2420: done (0 left) 2024-04-07T16:24:34.944 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 13 current snap is 227 2024-04-07T16:24:34.944 INFO:tasks.rados.rados.0.smithi100.stdout:2421: seq_num 736 ranges {2301952=335872} 2024-04-07T16:24:34.958 INFO:tasks.rados.rados.0.smithi100.stdout:2421: writing smithi10066867-13 from 2301952 to 2637824 tid 1 2024-04-07T16:24:34.958 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 17 current snap is 227 2024-04-07T16:24:34.958 INFO:tasks.rados.rados.0.smithi100.stdout:2422: seq_num 737 ranges {2203648=352256} 2024-04-07T16:24:34.973 INFO:tasks.rados.rados.0.smithi100.stdout:2422: writing smithi10066867-17 from 2203648 to 2555904 tid 1 2024-04-07T16:24:34.973 INFO:tasks.rados.rados.0.smithi100.stdout:2423: read oid 30 snap -1 2024-04-07T16:24:34.973 INFO:tasks.rados.rados.0.smithi100.stdout:2423: expect deleted 2024-04-07T16:24:34.973 INFO:tasks.rados.rados.0.smithi100.stdout:2424: snap_remove snap 215 2024-04-07T16:24:35.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:34 smithi144 bash[66654]: cluster 2024-04-07T16:24:33.945988+0000 mon.a (mon.0) 708 : cluster [DBG] osdmap e520: 8 total, 7 up, 8 in 2024-04-07T16:24:35.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:34 smithi144 bash[66654]: audit 2024-04-07T16:24:34.066135+0000 mgr.y (mgr.25175) 220 : audit [DBG] from='client.44255 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:24:35.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:34 smithi144 bash[66654]: audit 2024-04-07T16:24:34.882617+0000 mon.c (mon.2) 11 : audit [DBG] from='client.? 172.21.15.100:0/3945575374' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:24:35.433 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:34 smithi100 bash[74942]: cluster 2024-04-07T16:24:33.945988+0000 mon.a (mon.0) 708 : cluster [DBG] osdmap e520: 8 total, 7 up, 8 in 2024-04-07T16:24:35.434 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:34 smithi100 bash[74942]: audit 2024-04-07T16:24:34.066135+0000 mgr.y (mgr.25175) 220 : audit [DBG] from='client.44255 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:24:35.434 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:34 smithi100 bash[74942]: audit 2024-04-07T16:24:34.882617+0000 mon.c (mon.2) 11 : audit [DBG] from='client.? 172.21.15.100:0/3945575374' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:24:35.434 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:34 smithi100 bash[76202]: cluster 2024-04-07T16:24:33.945988+0000 mon.a (mon.0) 708 : cluster [DBG] osdmap e520: 8 total, 7 up, 8 in 2024-04-07T16:24:35.434 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:34 smithi100 bash[76202]: audit 2024-04-07T16:24:34.066135+0000 mgr.y (mgr.25175) 220 : audit [DBG] from='client.44255 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:24:35.434 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:34 smithi100 bash[76202]: audit 2024-04-07T16:24:34.882617+0000 mon.c (mon.2) 11 : audit [DBG] from='client.? 172.21.15.100:0/3945575374' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:24:35.953 INFO:tasks.rados.rados.0.smithi100.stdout:2424: done (3 left) 2024-04-07T16:24:35.953 INFO:tasks.rados.rados.0.smithi100.stdout:2425: delete oid 47 current snap is 227 2024-04-07T16:24:35.953 INFO:tasks.rados.rados.0.smithi100.stdout:2421: finishing write tid 1 to smithi10066867-13 2024-04-07T16:24:35.953 INFO:tasks.rados.rados.0.smithi100.stdout:2421: finishing write tid 2 to smithi10066867-13 2024-04-07T16:24:35.953 INFO:tasks.rados.rados.0.smithi100.stdout:2421: finishing write tid 3 to smithi10066867-13 2024-04-07T16:24:35.953 INFO:tasks.rados.rados.0.smithi100.stdout:2421: oid 13 updating version 0 to 990 2024-04-07T16:24:35.953 INFO:tasks.rados.rados.0.smithi100.stdout:2421: oid 13 version 990 is already newer than 989 2024-04-07T16:24:35.953 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 13 v 990 (ObjNum 736 snap 227 seq_num 736) dirty exists 2024-04-07T16:24:35.954 INFO:tasks.rados.rados.0.smithi100.stdout:2421: left oid 13 (ObjNum 736 snap 227 seq_num 736) 2024-04-07T16:24:35.954 INFO:tasks.rados.rados.0.smithi100.stdout:2422: finishing write tid 1 to smithi10066867-17 2024-04-07T16:24:35.954 INFO:tasks.rados.rados.0.smithi100.stdout:2422: finishing write tid 2 to smithi10066867-17 2024-04-07T16:24:35.954 INFO:tasks.rados.rados.0.smithi100.stdout:2422: finishing write tid 3 to smithi10066867-17 2024-04-07T16:24:35.954 INFO:tasks.rados.rados.0.smithi100.stdout:2422: oid 17 updating version 0 to 1122 2024-04-07T16:24:35.954 INFO:tasks.rados.rados.0.smithi100.stdout:2422: oid 17 version 1122 is already newer than 1121 2024-04-07T16:24:35.954 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 17 v 1122 (ObjNum 737 snap 227 seq_num 737) dirty exists 2024-04-07T16:24:35.954 INFO:tasks.rados.rados.0.smithi100.stdout:2422: left oid 17 (ObjNum 737 snap 227 seq_num 737) 2024-04-07T16:24:35.960 INFO:tasks.rados.rados.0.smithi100.stdout:2421: done (3 left) 2024-04-07T16:24:35.960 INFO:tasks.rados.rados.0.smithi100.stdout:2422: done (2 left) 2024-04-07T16:24:35.960 INFO:tasks.rados.rados.0.smithi100.stdout:2423: done (1 left) 2024-04-07T16:24:35.960 INFO:tasks.rados.rados.0.smithi100.stdout:2425: done (0 left) 2024-04-07T16:24:35.961 INFO:tasks.rados.rados.0.smithi100.stdout:2426: snap_create 2024-04-07T16:24:36.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:35 smithi144 bash[66654]: audit 2024-04-07T16:24:34.429291+0000 mgr.y (mgr.25175) 221 : audit [DBG] from='client.34341 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:24:36.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:35 smithi144 bash[66654]: cluster 2024-04-07T16:24:34.706190+0000 mgr.y (mgr.25175) 222 : cluster [DBG] pgmap v372: 121 pgs: 11 active+undersized+degraded, 39 active+undersized, 71 active+clean; 339 MiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 9.2 MiB/s rd, 734 KiB/s wr, 24 op/s; 61/441 objects degraded (13.832%) 2024-04-07T16:24:36.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:35 smithi144 bash[66654]: cluster 2024-04-07T16:24:34.942906+0000 mon.a (mon.0) 709 : cluster [INF] Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 3 pgs inactive, 3 pgs peering) 2024-04-07T16:24:36.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:35 smithi144 bash[66654]: cluster 2024-04-07T16:24:34.962191+0000 mon.a (mon.0) 710 : cluster [DBG] osdmap e521: 8 total, 7 up, 8 in 2024-04-07T16:24:36.434 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:35 smithi100 bash[74942]: audit 2024-04-07T16:24:34.429291+0000 mgr.y (mgr.25175) 221 : audit [DBG] from='client.34341 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:24:36.434 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:35 smithi100 bash[74942]: cluster 2024-04-07T16:24:34.706190+0000 mgr.y (mgr.25175) 222 : cluster [DBG] pgmap v372: 121 pgs: 11 active+undersized+degraded, 39 active+undersized, 71 active+clean; 339 MiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 9.2 MiB/s rd, 734 KiB/s wr, 24 op/s; 61/441 objects degraded (13.832%) 2024-04-07T16:24:36.434 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:35 smithi100 bash[74942]: cluster 2024-04-07T16:24:34.942906+0000 mon.a (mon.0) 709 : cluster [INF] Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 3 pgs inactive, 3 pgs peering) 2024-04-07T16:24:36.434 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:35 smithi100 bash[74942]: cluster 2024-04-07T16:24:34.962191+0000 mon.a (mon.0) 710 : cluster [DBG] osdmap e521: 8 total, 7 up, 8 in 2024-04-07T16:24:36.434 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:35 smithi100 bash[76202]: audit 2024-04-07T16:24:34.429291+0000 mgr.y (mgr.25175) 221 : audit [DBG] from='client.34341 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:24:36.434 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:35 smithi100 bash[76202]: cluster 2024-04-07T16:24:34.706190+0000 mgr.y (mgr.25175) 222 : cluster [DBG] pgmap v372: 121 pgs: 11 active+undersized+degraded, 39 active+undersized, 71 active+clean; 339 MiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 9.2 MiB/s rd, 734 KiB/s wr, 24 op/s; 61/441 objects degraded (13.832%) 2024-04-07T16:24:36.434 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:35 smithi100 bash[76202]: cluster 2024-04-07T16:24:34.942906+0000 mon.a (mon.0) 709 : cluster [INF] Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 3 pgs inactive, 3 pgs peering) 2024-04-07T16:24:36.434 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:35 smithi100 bash[76202]: cluster 2024-04-07T16:24:34.962191+0000 mon.a (mon.0) 710 : cluster [DBG] osdmap e521: 8 total, 7 up, 8 in 2024-04-07T16:24:36.975 INFO:tasks.rados.rados.0.smithi100.stdout:2426: done (0 left) 2024-04-07T16:24:36.975 INFO:tasks.rados.rados.0.smithi100.stdout:2427: delete oid 28 current snap is 228 2024-04-07T16:24:36.984 INFO:tasks.rados.rados.0.smithi100.stdout:2427: done (0 left) 2024-04-07T16:24:36.984 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 39 current snap is 228 2024-04-07T16:24:36.984 INFO:tasks.rados.rados.0.smithi100.stdout:2428: seq_num 740 ranges {5808128=401408} 2024-04-07T16:24:37.052 INFO:tasks.rados.rados.0.smithi100.stdout:2428: writing smithi10066867-39 from 5808128 to 6209536 tid 1 2024-04-07T16:24:37.052 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 13 current snap is 228 2024-04-07T16:24:37.052 INFO:tasks.rados.rados.0.smithi100.stdout:2429: seq_num 741 ranges {2637824=417792} 2024-04-07T16:24:37.073 INFO:tasks.rados.rados.0.smithi100.stdout:2429: writing smithi10066867-13 from 2637824 to 3055616 tid 1 2024-04-07T16:24:37.074 INFO:tasks.rados.rados.0.smithi100.stdout:2430: snap_remove snap 219 2024-04-07T16:24:37.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:36 smithi144 bash[66654]: cluster 2024-04-07T16:24:35.971240+0000 mon.a (mon.0) 711 : cluster [DBG] osdmap e522: 8 total, 7 up, 8 in 2024-04-07T16:24:37.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:36 smithi144 bash[66654]: cluster 2024-04-07T16:24:36.840306+0000 mon.a (mon.0) 712 : cluster [WRN] Health check update: Degraded data redundancy: 61/441 objects degraded (13.832%), 11 pgs degraded (PG_DEGRADED) 2024-04-07T16:24:37.434 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:36 smithi100 bash[74942]: cluster 2024-04-07T16:24:35.971240+0000 mon.a (mon.0) 711 : cluster [DBG] osdmap e522: 8 total, 7 up, 8 in 2024-04-07T16:24:37.434 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:36 smithi100 bash[74942]: cluster 2024-04-07T16:24:36.840306+0000 mon.a (mon.0) 712 : cluster [WRN] Health check update: Degraded data redundancy: 61/441 objects degraded (13.832%), 11 pgs degraded (PG_DEGRADED) 2024-04-07T16:24:37.434 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:36 smithi100 bash[76202]: cluster 2024-04-07T16:24:35.971240+0000 mon.a (mon.0) 711 : cluster [DBG] osdmap e522: 8 total, 7 up, 8 in 2024-04-07T16:24:37.434 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:36 smithi100 bash[76202]: cluster 2024-04-07T16:24:36.840306+0000 mon.a (mon.0) 712 : cluster [WRN] Health check update: Degraded data redundancy: 61/441 objects degraded (13.832%), 11 pgs degraded (PG_DEGRADED) 2024-04-07T16:24:37.986 INFO:tasks.rados.rados.0.smithi100.stdout:2430: done (2 left) 2024-04-07T16:24:37.986 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 28 current snap is 228 2024-04-07T16:24:37.986 INFO:tasks.rados.rados.0.smithi100.stdout:2431: seq_num 742 ranges {0=434176} 2024-04-07T16:24:37.990 INFO:tasks.rados.rados.0.smithi100.stdout:2431: writing smithi10066867-28 from 0 to 434176 tid 1 2024-04-07T16:24:37.990 INFO:tasks.rados.rados.0.smithi100.stdout:2432: read oid 31 snap 186 2024-04-07T16:24:37.990 INFO:tasks.rados.rados.0.smithi100.stdout:2432: expect (ObjNum 579 snap 180 seq_num 579) 2024-04-07T16:24:37.990 INFO:tasks.rados.rados.0.smithi100.stdout:2433: copy_from oid 29 from oid 6 current snap is 228 2024-04-07T16:24:37.991 INFO:tasks.rados.rados.0.smithi100.stdout:2434: delete oid 1 current snap is 228 2024-04-07T16:24:37.991 INFO:tasks.rados.rados.0.smithi100.stdout:2428: finishing write tid 1 to smithi10066867-39 2024-04-07T16:24:37.991 INFO:tasks.rados.rados.0.smithi100.stdout:2428: finishing write tid 2 to smithi10066867-39 2024-04-07T16:24:37.991 INFO:tasks.rados.rados.0.smithi100.stdout:2428: finishing write tid 3 to smithi10066867-39 2024-04-07T16:24:37.991 INFO:tasks.rados.rados.0.smithi100.stdout:2428: oid 39 updating version 0 to 1121 2024-04-07T16:24:37.991 INFO:tasks.rados.rados.0.smithi100.stdout:2428: oid 39 updating version 1121 to 1122 2024-04-07T16:24:37.991 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 39 v 1122 (ObjNum 740 snap 228 seq_num 740) dirty exists 2024-04-07T16:24:37.991 INFO:tasks.rados.rados.0.smithi100.stdout:2428: left oid 39 (ObjNum 740 snap 228 seq_num 740) 2024-04-07T16:24:37.991 INFO:tasks.rados.rados.0.smithi100.stdout:2429: finishing write tid 1 to smithi10066867-13 2024-04-07T16:24:37.991 INFO:tasks.rados.rados.0.smithi100.stdout:2429: finishing write tid 2 to smithi10066867-13 2024-04-07T16:24:37.991 INFO:tasks.rados.rados.0.smithi100.stdout:2429: finishing write tid 3 to smithi10066867-13 2024-04-07T16:24:37.991 INFO:tasks.rados.rados.0.smithi100.stdout:2429: oid 13 updating version 0 to 992 2024-04-07T16:24:37.992 INFO:tasks.rados.rados.0.smithi100.stdout:2429: oid 13 updating version 992 to 993 2024-04-07T16:24:37.992 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 13 v 993 (ObjNum 741 snap 228 seq_num 741) dirty exists 2024-04-07T16:24:37.992 INFO:tasks.rados.rados.0.smithi100.stdout:2429: left oid 13 (ObjNum 741 snap 228 seq_num 741) 2024-04-07T16:24:37.997 INFO:tasks.rados.rados.0.smithi100.stdout:2431: finishing write tid 1 to smithi10066867-28 2024-04-07T16:24:37.997 INFO:tasks.rados.rados.0.smithi100.stdout:2431: finishing write tid 2 to smithi10066867-28 2024-04-07T16:24:38.000 INFO:tasks.rados.rados.0.smithi100.stdout:2428: done (5 left) 2024-04-07T16:24:38.000 INFO:tasks.rados.rados.0.smithi100.stdout:2429: done (4 left) 2024-04-07T16:24:38.000 INFO:tasks.rados.rados.0.smithi100.stdout:2434: done (3 left) 2024-04-07T16:24:38.000 INFO:tasks.rados.rados.0.smithi100.stdout:2435: read oid 50 snap -1 2024-04-07T16:24:38.000 INFO:tasks.rados.rados.0.smithi100.stdout:2435: expect (ObjNum 657 snap 203 seq_num 657) 2024-04-07T16:24:38.000 INFO:tasks.rados.rados.0.smithi100.stdout:2436: snap_remove snap 218 2024-04-07T16:24:38.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:37 smithi144 bash[66654]: cluster 2024-04-07T16:24:36.707059+0000 mgr.y (mgr.25175) 223 : cluster [DBG] pgmap v375: 121 pgs: 11 active+undersized+degraded, 39 active+undersized, 71 active+clean; 337 MiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 18 MiB/s rd, 706 KiB/s wr, 29 op/s; 62/441 objects degraded (14.059%) 2024-04-07T16:24:38.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:37 smithi144 bash[66654]: cluster 2024-04-07T16:24:36.993396+0000 mon.a (mon.0) 713 : cluster [DBG] osdmap e523: 8 total, 7 up, 8 in 2024-04-07T16:24:38.417 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:37 smithi100 bash[74942]: cluster 2024-04-07T16:24:36.707059+0000 mgr.y (mgr.25175) 223 : cluster [DBG] pgmap v375: 121 pgs: 11 active+undersized+degraded, 39 active+undersized, 71 active+clean; 337 MiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 18 MiB/s rd, 706 KiB/s wr, 29 op/s; 62/441 objects degraded (14.059%) 2024-04-07T16:24:38.417 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:37 smithi100 bash[74942]: cluster 2024-04-07T16:24:36.993396+0000 mon.a (mon.0) 713 : cluster [DBG] osdmap e523: 8 total, 7 up, 8 in 2024-04-07T16:24:38.417 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:37 smithi100 bash[76202]: cluster 2024-04-07T16:24:36.707059+0000 mgr.y (mgr.25175) 223 : cluster [DBG] pgmap v375: 121 pgs: 11 active+undersized+degraded, 39 active+undersized, 71 active+clean; 337 MiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 18 MiB/s rd, 706 KiB/s wr, 29 op/s; 62/441 objects degraded (14.059%) 2024-04-07T16:24:38.418 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:37 smithi100 bash[76202]: cluster 2024-04-07T16:24:36.993396+0000 mon.a (mon.0) 713 : cluster [DBG] osdmap e523: 8 total, 7 up, 8 in 2024-04-07T16:24:38.683 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:24:38 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:24:38] "GET /metrics HTTP/1.1" 200 36276 "" "Prometheus/2.43.0" 2024-04-07T16:24:38.856 INFO:journalctl@ceph.osd.5.smithi144.stdout:Apr 07 16:24:38 smithi144 bash[75374]: debug 2024-04-07T16:24:38.410+0000 7f4b08f907c0 -1 osd.5 513 log_to_monitors true 2024-04-07T16:24:38.994 INFO:tasks.rados.rados.0.smithi100.stdout:2436: done (4 left) 2024-04-07T16:24:38.994 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 22 current snap is 228 2024-04-07T16:24:38.994 INFO:tasks.rados.rados.0.smithi100.stdout:2431: finishing write tid 3 to smithi10066867-28 2024-04-07T16:24:38.994 INFO:tasks.rados.rados.0.smithi100.stdout:2431: oid 28 updating version 0 to 1101 2024-04-07T16:24:38.994 INFO:tasks.rados.rados.0.smithi100.stdout:2431: oid 28 updating version 1101 to 1102 2024-04-07T16:24:38.995 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 28 v 1102 (ObjNum 742 snap 228 seq_num 742) dirty exists 2024-04-07T16:24:38.995 INFO:tasks.rados.rados.0.smithi100.stdout:2431: left oid 28 (ObjNum 742 snap 228 seq_num 742) 2024-04-07T16:24:38.995 INFO:tasks.rados.rados.0.smithi100.stdout:2437: seq_num 744 ranges {2908160=466944} 2024-04-07T16:24:39.024 INFO:tasks.rados.rados.0.smithi100.stdout:2437: writing smithi10066867-22 from 2908160 to 3375104 tid 1 2024-04-07T16:24:39.025 INFO:tasks.rados.rados.0.smithi100.stdout:2431: done (4 left) 2024-04-07T16:24:39.025 INFO:tasks.rados.rados.0.smithi100.stdout:2438: rmattr oid 2 current snap is 228 2024-04-07T16:24:39.025 INFO:tasks.rados.rados.0.smithi100.stdout:2438: done (4 left) 2024-04-07T16:24:39.025 INFO:tasks.rados.rados.0.smithi100.stdout:2439: read oid 4 snap 223 2024-04-07T16:24:39.025 INFO:tasks.rados.rados.0.smithi100.stdout:2439: expect (ObjNum 684 snap 210 seq_num 684) 2024-04-07T16:24:39.025 INFO:tasks.rados.rados.0.smithi100.stdout:2440: read oid 14 snap 226 2024-04-07T16:24:39.025 INFO:tasks.rados.rados.0.smithi100.stdout:2440: expect deleted 2024-04-07T16:24:39.025 INFO:tasks.rados.rados.0.smithi100.stdout:2441: delete oid 26 current snap is 228 2024-04-07T16:24:39.025 INFO:tasks.rados.rados.0.smithi100.stdout:2432: expect (ObjNum 579 snap 180 seq_num 579) 2024-04-07T16:24:39.270 INFO:tasks.rados.rados.0.smithi100.stdout:2435: expect (ObjNum 657 snap 203 seq_num 657) 2024-04-07T16:24:39.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:39 smithi144 bash[66654]: cluster 2024-04-07T16:24:38.003591+0000 mon.a (mon.0) 714 : cluster [DBG] osdmap e524: 8 total, 7 up, 8 in 2024-04-07T16:24:39.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:39 smithi144 bash[66654]: audit 2024-04-07T16:24:38.417590+0000 mon.b (mon.1) 5 : audit [INF] from='osd.5 [v2:172.21.15.144:6808/1594566125,v1:172.21.15.144:6809/1594566125]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-07T16:24:39.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:39 smithi144 bash[66654]: audit 2024-04-07T16:24:38.426950+0000 mon.a (mon.0) 715 : audit [INF] from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-07T16:24:39.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:39 smithi144 bash[66654]: audit 2024-04-07T16:24:38.936384+0000 mon.a (mon.0) 716 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:24:39.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:39 smithi144 bash[66654]: audit 2024-04-07T16:24:38.950384+0000 mon.a (mon.0) 717 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:24:39.434 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:39 smithi100 bash[74942]: cluster 2024-04-07T16:24:38.003591+0000 mon.a (mon.0) 714 : cluster [DBG] osdmap e524: 8 total, 7 up, 8 in 2024-04-07T16:24:39.434 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:39 smithi100 bash[74942]: audit 2024-04-07T16:24:38.417590+0000 mon.b (mon.1) 5 : audit [INF] from='osd.5 [v2:172.21.15.144:6808/1594566125,v1:172.21.15.144:6809/1594566125]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-07T16:24:39.434 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:39 smithi100 bash[74942]: audit 2024-04-07T16:24:38.426950+0000 mon.a (mon.0) 715 : audit [INF] from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-07T16:24:39.434 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:39 smithi100 bash[74942]: audit 2024-04-07T16:24:38.936384+0000 mon.a (mon.0) 716 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:24:39.434 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:39 smithi100 bash[74942]: audit 2024-04-07T16:24:38.950384+0000 mon.a (mon.0) 717 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:24:39.434 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:39 smithi100 bash[76202]: cluster 2024-04-07T16:24:38.003591+0000 mon.a (mon.0) 714 : cluster [DBG] osdmap e524: 8 total, 7 up, 8 in 2024-04-07T16:24:39.434 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:39 smithi100 bash[76202]: audit 2024-04-07T16:24:38.417590+0000 mon.b (mon.1) 5 : audit [INF] from='osd.5 [v2:172.21.15.144:6808/1594566125,v1:172.21.15.144:6809/1594566125]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-07T16:24:39.434 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:39 smithi100 bash[76202]: audit 2024-04-07T16:24:38.426950+0000 mon.a (mon.0) 715 : audit [INF] from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-07T16:24:39.435 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:39 smithi100 bash[76202]: audit 2024-04-07T16:24:38.936384+0000 mon.a (mon.0) 716 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:24:39.435 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:39 smithi100 bash[76202]: audit 2024-04-07T16:24:38.950384+0000 mon.a (mon.0) 717 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:24:39.587 INFO:tasks.rados.rados.0.smithi100.stdout:2433: finishing copy_from racing read to smithi10066867-29 2024-04-07T16:24:39.588 INFO:tasks.rados.rados.0.smithi100.stdout:2433: finishing copy_from to smithi10066867-29 2024-04-07T16:24:39.588 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 29 v 1125 (ObjNum 706 snap 215 seq_num 706) dirty exists 2024-04-07T16:24:39.588 INFO:tasks.rados.rados.0.smithi100.stdout:2437: finishing write tid 1 to smithi10066867-22 2024-04-07T16:24:39.588 INFO:tasks.rados.rados.0.smithi100.stdout:2432: done (6 left) 2024-04-07T16:24:39.588 INFO:tasks.rados.rados.0.smithi100.stdout:2433: done (5 left) 2024-04-07T16:24:39.588 INFO:tasks.rados.rados.0.smithi100.stdout:2435: done (4 left) 2024-04-07T16:24:39.588 INFO:tasks.rados.rados.0.smithi100.stdout:2440: done (3 left) 2024-04-07T16:24:39.588 INFO:tasks.rados.rados.0.smithi100.stdout:2441: done (2 left) 2024-04-07T16:24:39.588 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 23 current snap is 228 2024-04-07T16:24:39.588 INFO:tasks.rados.rados.0.smithi100.stdout:2437: finishing write tid 2 to smithi10066867-22 2024-04-07T16:24:39.588 INFO:tasks.rados.rados.0.smithi100.stdout:2437: finishing write tid 3 to smithi10066867-22 2024-04-07T16:24:39.588 INFO:tasks.rados.rados.0.smithi100.stdout:2437: oid 22 updating version 0 to 1161 2024-04-07T16:24:39.588 INFO:tasks.rados.rados.0.smithi100.stdout:2437: oid 22 updating version 1161 to 1162 2024-04-07T16:24:39.589 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 22 v 1162 (ObjNum 744 snap 228 seq_num 744) dirty exists 2024-04-07T16:24:39.589 INFO:tasks.rados.rados.0.smithi100.stdout:2437: left oid 22 (ObjNum 744 snap 228 seq_num 744) 2024-04-07T16:24:39.589 INFO:tasks.rados.rados.0.smithi100.stdout:2439: expect (ObjNum 684 snap 210 seq_num 684) 2024-04-07T16:24:39.634 INFO:tasks.rados.rados.0.smithi100.stdout:2442: seq_num 746 ranges {3620864=499712} 2024-04-07T16:24:39.697 INFO:tasks.rados.rados.0.smithi100.stdout:2442: writing smithi10066867-23 from 3620864 to 4120576 tid 1 2024-04-07T16:24:39.697 INFO:tasks.rados.rados.0.smithi100.stdout:2437: done (2 left) 2024-04-07T16:24:39.697 INFO:tasks.rados.rados.0.smithi100.stdout:2439: done (1 left) 2024-04-07T16:24:39.698 INFO:tasks.rados.rados.0.smithi100.stdout:2443: delete oid 29 current snap is 228 2024-04-07T16:24:39.701 INFO:tasks.rados.rados.0.smithi100.stdout:2443: done (1 left) 2024-04-07T16:24:39.701 INFO:tasks.rados.rados.0.smithi100.stdout:2444: snap_create 2024-04-07T16:24:39.704 INFO:tasks.rados.rados.0.smithi100.stdout:2442: finishing write tid 1 to smithi10066867-23 2024-04-07T16:24:39.705 INFO:tasks.rados.rados.0.smithi100.stdout:2442: finishing write tid 2 to smithi10066867-23 2024-04-07T16:24:39.707 INFO:tasks.rados.rados.0.smithi100.stdout:2442: finishing write tid 3 to smithi10066867-23 2024-04-07T16:24:39.708 INFO:tasks.rados.rados.0.smithi100.stdout:2442: oid 23 updating version 0 to 1369 2024-04-07T16:24:39.708 INFO:tasks.rados.rados.0.smithi100.stdout:2442: oid 23 version 1369 is already newer than 1368 2024-04-07T16:24:39.708 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 23 v 1369 (ObjNum 746 snap 228 seq_num 746) dirty exists 2024-04-07T16:24:39.708 INFO:tasks.rados.rados.0.smithi100.stdout:2442: left oid 23 (ObjNum 746 snap 228 seq_num 746) 2024-04-07T16:24:39.856 INFO:journalctl@ceph.osd.5.smithi144.stdout:Apr 07 16:24:39 smithi144 bash[75374]: debug 2024-04-07T16:24:39.490+0000 7f4af63e0700 -1 osd.5 513 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-07T16:24:40.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:39 smithi144 bash[66654]: cluster 2024-04-07T16:24:38.708029+0000 mgr.y (mgr.25175) 224 : cluster [DBG] pgmap v378: 121 pgs: 11 active+undersized+degraded, 39 active+undersized, 71 active+clean; 338 MiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 19 MiB/s rd, 298 KiB/s wr, 16 op/s; 64/465 objects degraded (13.763%) 2024-04-07T16:24:40.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:39 smithi144 bash[66654]: audit 2024-04-07T16:24:39.004958+0000 mon.a (mon.0) 718 : audit [INF] from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-07T16:24:40.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:39 smithi144 bash[66654]: audit 2024-04-07T16:24:39.010094+0000 mon.b (mon.1) 6 : audit [INF] from='osd.5 [v2:172.21.15.144:6808/1594566125,v1:172.21.15.144:6809/1594566125]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-04-07T16:24:40.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:39 smithi144 bash[66654]: cluster 2024-04-07T16:24:39.012207+0000 mon.a (mon.0) 719 : cluster [DBG] osdmap e525: 8 total, 7 up, 8 in 2024-04-07T16:24:40.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:40 smithi144 bash[66654]: audit 2024-04-07T16:24:39.020749+0000 mon.a (mon.0) 720 : audit [INF] from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-04-07T16:24:40.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:40 smithi144 bash[66654]: audit 2024-04-07T16:24:39.494818+0000 mon.a (mon.0) 721 : audit [INF] from='osd.5 ' entity='osd.5' 2024-04-07T16:24:40.434 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:40 smithi100 bash[74942]: cluster 2024-04-07T16:24:38.708029+0000 mgr.y (mgr.25175) 224 : cluster [DBG] pgmap v378: 121 pgs: 11 active+undersized+degraded, 39 active+undersized, 71 active+clean; 338 MiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 19 MiB/s rd, 298 KiB/s wr, 16 op/s; 64/465 objects degraded (13.763%) 2024-04-07T16:24:40.434 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:40 smithi100 bash[74942]: audit 2024-04-07T16:24:39.004958+0000 mon.a (mon.0) 718 : audit [INF] from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-07T16:24:40.434 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:40 smithi100 bash[74942]: audit 2024-04-07T16:24:39.010094+0000 mon.b (mon.1) 6 : audit [INF] from='osd.5 [v2:172.21.15.144:6808/1594566125,v1:172.21.15.144:6809/1594566125]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-04-07T16:24:40.434 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:40 smithi100 bash[74942]: cluster 2024-04-07T16:24:39.012207+0000 mon.a (mon.0) 719 : cluster [DBG] osdmap e525: 8 total, 7 up, 8 in 2024-04-07T16:24:40.434 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:40 smithi100 bash[74942]: audit 2024-04-07T16:24:39.020749+0000 mon.a (mon.0) 720 : audit [INF] from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-04-07T16:24:40.434 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:40 smithi100 bash[74942]: audit 2024-04-07T16:24:39.494818+0000 mon.a (mon.0) 721 : audit [INF] from='osd.5 ' entity='osd.5' 2024-04-07T16:24:40.434 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:40 smithi100 bash[76202]: cluster 2024-04-07T16:24:38.708029+0000 mgr.y (mgr.25175) 224 : cluster [DBG] pgmap v378: 121 pgs: 11 active+undersized+degraded, 39 active+undersized, 71 active+clean; 338 MiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 19 MiB/s rd, 298 KiB/s wr, 16 op/s; 64/465 objects degraded (13.763%) 2024-04-07T16:24:40.435 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:40 smithi100 bash[76202]: audit 2024-04-07T16:24:39.004958+0000 mon.a (mon.0) 718 : audit [INF] from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-07T16:24:40.435 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:40 smithi100 bash[76202]: audit 2024-04-07T16:24:39.010094+0000 mon.b (mon.1) 6 : audit [INF] from='osd.5 [v2:172.21.15.144:6808/1594566125,v1:172.21.15.144:6809/1594566125]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-04-07T16:24:40.435 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:40 smithi100 bash[76202]: cluster 2024-04-07T16:24:39.012207+0000 mon.a (mon.0) 719 : cluster [DBG] osdmap e525: 8 total, 7 up, 8 in 2024-04-07T16:24:40.435 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:40 smithi100 bash[76202]: audit 2024-04-07T16:24:39.020749+0000 mon.a (mon.0) 720 : audit [INF] from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-04-07T16:24:40.435 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:40 smithi100 bash[76202]: audit 2024-04-07T16:24:39.494818+0000 mon.a (mon.0) 721 : audit [INF] from='osd.5 ' entity='osd.5' 2024-04-07T16:24:40.482 INFO:tasks.rados.rados.0.smithi100.stdout:2442: done (1 left) 2024-04-07T16:24:40.483 INFO:tasks.rados.rados.0.smithi100.stdout:2444: done (0 left) 2024-04-07T16:24:40.483 INFO:tasks.rados.rados.0.smithi100.stdout:2445: snap_remove snap 226 2024-04-07T16:24:41.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:41 smithi144 bash[66654]: audit 2024-04-07T16:24:40.114988+0000 mon.a (mon.0) 722 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:24:41.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:41 smithi144 bash[66654]: audit 2024-04-07T16:24:40.128670+0000 mon.a (mon.0) 723 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:24:41.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:41 smithi144 bash[66654]: cluster 2024-04-07T16:24:40.491913+0000 mon.a (mon.0) 724 : cluster [INF] Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-07T16:24:41.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:41 smithi144 bash[66654]: cluster 2024-04-07T16:24:40.499861+0000 mon.a (mon.0) 725 : cluster [INF] osd.5 [v2:172.21.15.144:6808/1594566125,v1:172.21.15.144:6809/1594566125] boot 2024-04-07T16:24:41.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:41 smithi144 bash[66654]: cluster 2024-04-07T16:24:40.500001+0000 mon.a (mon.0) 726 : cluster [DBG] osdmap e526: 8 total, 8 up, 8 in 2024-04-07T16:24:41.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:41 smithi144 bash[66654]: audit 2024-04-07T16:24:40.500940+0000 mon.a (mon.0) 727 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-07T16:24:41.434 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:41 smithi100 bash[74942]: audit 2024-04-07T16:24:40.114988+0000 mon.a (mon.0) 722 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:24:41.434 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:41 smithi100 bash[74942]: audit 2024-04-07T16:24:40.128670+0000 mon.a (mon.0) 723 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:24:41.434 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:41 smithi100 bash[74942]: cluster 2024-04-07T16:24:40.491913+0000 mon.a (mon.0) 724 : cluster [INF] Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-07T16:24:41.434 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:41 smithi100 bash[74942]: cluster 2024-04-07T16:24:40.499861+0000 mon.a (mon.0) 725 : cluster [INF] osd.5 [v2:172.21.15.144:6808/1594566125,v1:172.21.15.144:6809/1594566125] boot 2024-04-07T16:24:41.434 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:41 smithi100 bash[74942]: cluster 2024-04-07T16:24:40.500001+0000 mon.a (mon.0) 726 : cluster [DBG] osdmap e526: 8 total, 8 up, 8 in 2024-04-07T16:24:41.434 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:41 smithi100 bash[74942]: audit 2024-04-07T16:24:40.500940+0000 mon.a (mon.0) 727 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-07T16:24:41.435 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:41 smithi100 bash[76202]: audit 2024-04-07T16:24:40.114988+0000 mon.a (mon.0) 722 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:24:41.435 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:41 smithi100 bash[76202]: audit 2024-04-07T16:24:40.128670+0000 mon.a (mon.0) 723 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:24:41.435 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:41 smithi100 bash[76202]: cluster 2024-04-07T16:24:40.491913+0000 mon.a (mon.0) 724 : cluster [INF] Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-07T16:24:41.435 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:41 smithi100 bash[76202]: cluster 2024-04-07T16:24:40.499861+0000 mon.a (mon.0) 725 : cluster [INF] osd.5 [v2:172.21.15.144:6808/1594566125,v1:172.21.15.144:6809/1594566125] boot 2024-04-07T16:24:41.435 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:41 smithi100 bash[76202]: cluster 2024-04-07T16:24:40.500001+0000 mon.a (mon.0) 726 : cluster [DBG] osdmap e526: 8 total, 8 up, 8 in 2024-04-07T16:24:41.435 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:41 smithi100 bash[76202]: audit 2024-04-07T16:24:40.500940+0000 mon.a (mon.0) 727 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-07T16:24:41.493 INFO:tasks.rados.rados.0.smithi100.stdout:2445: done (0 left) 2024-04-07T16:24:41.493 INFO:tasks.rados.rados.0.smithi100.stdout:2446: copy_from oid 33 from oid 14 current snap is 229 2024-04-07T16:24:41.494 INFO:tasks.rados.rados.0.smithi100.stdout:2447: rmattr oid 20 current snap is 229 2024-04-07T16:24:41.494 INFO:tasks.rados.rados.0.smithi100.stdout:2448: snap_remove snap 224 2024-04-07T16:24:42.356 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:24:42 smithi144 bash[59794]: ts=2024-04-07T16:24:42.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:24:42.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:42 smithi144 bash[66654]: cluster 2024-04-07T16:24:40.708885+0000 mgr.y (mgr.25175) 225 : cluster [DBG] pgmap v381: 121 pgs: 11 active+undersized+degraded, 39 active+undersized, 71 active+clean; 343 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 13 MiB/s rd, 568 KiB/s wr, 14 op/s; 68/474 objects degraded (14.346%) 2024-04-07T16:24:42.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:42 smithi144 bash[66654]: cluster 2024-04-07T16:24:41.509811+0000 mon.a (mon.0) 728 : cluster [DBG] osdmap e527: 8 total, 8 up, 8 in 2024-04-07T16:24:42.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:42 smithi144 bash[66654]: cluster 2024-04-07T16:24:41.841461+0000 mon.a (mon.0) 729 : cluster [WRN] Health check update: Degraded data redundancy: 68/474 objects degraded (14.346%), 11 pgs degraded (PG_DEGRADED) 2024-04-07T16:24:42.434 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:42 smithi100 bash[74942]: cluster 2024-04-07T16:24:40.708885+0000 mgr.y (mgr.25175) 225 : cluster [DBG] pgmap v381: 121 pgs: 11 active+undersized+degraded, 39 active+undersized, 71 active+clean; 343 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 13 MiB/s rd, 568 KiB/s wr, 14 op/s; 68/474 objects degraded (14.346%) 2024-04-07T16:24:42.434 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:42 smithi100 bash[74942]: cluster 2024-04-07T16:24:41.509811+0000 mon.a (mon.0) 728 : cluster [DBG] osdmap e527: 8 total, 8 up, 8 in 2024-04-07T16:24:42.434 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:42 smithi100 bash[74942]: cluster 2024-04-07T16:24:41.841461+0000 mon.a (mon.0) 729 : cluster [WRN] Health check update: Degraded data redundancy: 68/474 objects degraded (14.346%), 11 pgs degraded (PG_DEGRADED) 2024-04-07T16:24:42.434 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:42 smithi100 bash[76202]: cluster 2024-04-07T16:24:40.708885+0000 mgr.y (mgr.25175) 225 : cluster [DBG] pgmap v381: 121 pgs: 11 active+undersized+degraded, 39 active+undersized, 71 active+clean; 343 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 13 MiB/s rd, 568 KiB/s wr, 14 op/s; 68/474 objects degraded (14.346%) 2024-04-07T16:24:42.434 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:42 smithi100 bash[76202]: cluster 2024-04-07T16:24:41.509811+0000 mon.a (mon.0) 728 : cluster [DBG] osdmap e527: 8 total, 8 up, 8 in 2024-04-07T16:24:42.435 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:42 smithi100 bash[76202]: cluster 2024-04-07T16:24:41.841461+0000 mon.a (mon.0) 729 : cluster [WRN] Health check update: Degraded data redundancy: 68/474 objects degraded (14.346%), 11 pgs degraded (PG_DEGRADED) 2024-04-07T16:24:42.494 INFO:tasks.rados.rados.0.smithi100.stdout:2448: done (2 left) 2024-04-07T16:24:42.495 INFO:tasks.rados.rados.0.smithi100.stdout:2449: snap_create 2024-04-07T16:24:42.495 INFO:tasks.rados.rados.0.smithi100.stdout:2446: finishing copy_from to smithi10066867-33 2024-04-07T16:24:42.495 INFO:tasks.rados.rados.0.smithi100.stdout:2446: got expected ENOENT (src dne) 2024-04-07T16:24:42.495 INFO:tasks.rados.rados.0.smithi100.stdout:2446: finishing copy_from racing read to smithi10066867-33 2024-04-07T16:24:42.495 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 20 v 1164 (ObjNum 610 snap 188 seq_num 610) dirty exists 2024-04-07T16:24:43.510 INFO:tasks.rados.rados.0.smithi100.stdout:2446: done (2 left) 2024-04-07T16:24:43.511 INFO:tasks.rados.rados.0.smithi100.stdout:2447: done (1 left) 2024-04-07T16:24:43.511 INFO:tasks.rados.rados.0.smithi100.stdout:2449: done (0 left) 2024-04-07T16:24:43.511 INFO:tasks.rados.rados.0.smithi100.stdout:2450: read oid 47 snap -1 2024-04-07T16:24:43.511 INFO:tasks.rados.rados.0.smithi100.stdout:2450: expect deleted 2024-04-07T16:24:43.511 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 12 current snap is 230 2024-04-07T16:24:43.511 INFO:tasks.rados.rados.0.smithi100.stdout:2451: seq_num 748 ranges {131072=532480} 2024-04-07T16:24:43.515 INFO:tasks.rados.rados.0.smithi100.stdout:2451: writing smithi10066867-12 from 131072 to 663552 tid 1 2024-04-07T16:24:43.515 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 8 current snap is 230 2024-04-07T16:24:43.516 INFO:tasks.rados.rados.0.smithi100.stdout:2452: seq_num 749 ranges {0=548864} 2024-04-07T16:24:43.520 INFO:tasks.rados.rados.0.smithi100.stdout:2452: writing smithi10066867-8 from 0 to 548864 tid 1 2024-04-07T16:24:43.520 INFO:tasks.rados.rados.0.smithi100.stdout:2453: delete oid 31 current snap is 230 2024-04-07T16:24:43.524 INFO:tasks.rados.rados.0.smithi100.stdout:2450: done (3 left) 2024-04-07T16:24:43.524 INFO:tasks.rados.rados.0.smithi100.stdout:2453: done (2 left) 2024-04-07T16:24:43.524 INFO:tasks.rados.rados.0.smithi100.stdout:2454: delete oid 10 current snap is 230 2024-04-07T16:24:43.529 INFO:tasks.rados.rados.0.smithi100.stdout:2451: finishing write tid 1 to smithi10066867-12 2024-04-07T16:24:43.532 INFO:tasks.rados.rados.0.smithi100.stdout:2454: done (2 left) 2024-04-07T16:24:43.532 INFO:tasks.rados.rados.0.smithi100.stdout:2455: read oid 16 snap -1 2024-04-07T16:24:43.532 INFO:tasks.rados.rados.0.smithi100.stdout:2455: expect (ObjNum 465 snap 142 seq_num 465) 2024-04-07T16:24:43.532 INFO:tasks.rados.rados.0.smithi100.stdout:2456: read oid 11 snap -1 2024-04-07T16:24:43.532 INFO:tasks.rados.rados.0.smithi100.stdout:2456: expect (ObjNum 718 snap 221 seq_num 718) 2024-04-07T16:24:43.532 INFO:tasks.rados.rados.0.smithi100.stdout:2457: snap_remove snap 227 2024-04-07T16:24:43.856 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:43 smithi144 bash[66654]: cluster 2024-04-07T16:24:42.512661+0000 mon.a (mon.0) 730 : cluster [DBG] osdmap e528: 8 total, 8 up, 8 in 2024-04-07T16:24:43.856 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:43 smithi144 bash[66654]: cluster 2024-04-07T16:24:42.709999+0000 mgr.y (mgr.25175) 226 : cluster [DBG] pgmap v384: 121 pgs: 5 active+undersized+degraded, 28 active+undersized, 88 active+clean; 335 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 6.5 MiB/s rd, 376 KiB/s wr, 7 op/s; 45/471 objects degraded (9.554%); 4.1 MiB/s, 1 objects/s recovering 2024-04-07T16:24:43.934 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:43 smithi100 bash[74942]: cluster 2024-04-07T16:24:42.512661+0000 mon.a (mon.0) 730 : cluster [DBG] osdmap e528: 8 total, 8 up, 8 in 2024-04-07T16:24:43.934 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:43 smithi100 bash[74942]: cluster 2024-04-07T16:24:42.709999+0000 mgr.y (mgr.25175) 226 : cluster [DBG] pgmap v384: 121 pgs: 5 active+undersized+degraded, 28 active+undersized, 88 active+clean; 335 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 6.5 MiB/s rd, 376 KiB/s wr, 7 op/s; 45/471 objects degraded (9.554%); 4.1 MiB/s, 1 objects/s recovering 2024-04-07T16:24:43.934 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:43 smithi100 bash[76202]: cluster 2024-04-07T16:24:42.512661+0000 mon.a (mon.0) 730 : cluster [DBG] osdmap e528: 8 total, 8 up, 8 in 2024-04-07T16:24:43.934 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:43 smithi100 bash[76202]: cluster 2024-04-07T16:24:42.709999+0000 mgr.y (mgr.25175) 226 : cluster [DBG] pgmap v384: 121 pgs: 5 active+undersized+degraded, 28 active+undersized, 88 active+clean; 335 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 6.5 MiB/s rd, 376 KiB/s wr, 7 op/s; 45/471 objects degraded (9.554%); 4.1 MiB/s, 1 objects/s recovering 2024-04-07T16:24:44.510 INFO:tasks.rados.rados.0.smithi100.stdout:2457: done (4 left) 2024-04-07T16:24:44.511 INFO:tasks.rados.rados.0.smithi100.stdout:2458: copy_from oid 13 from oid 2 current snap is 230 2024-04-07T16:24:44.511 INFO:tasks.rados.rados.0.smithi100.stdout:2451: finishing write tid 2 to smithi10066867-12 2024-04-07T16:24:44.511 INFO:tasks.rados.rados.0.smithi100.stdout:2456: expect (ObjNum 718 snap 221 seq_num 718) 2024-04-07T16:24:44.513 INFO:tasks.rados.rados.0.smithi100.stdout:2451: finishing write tid 3 to smithi10066867-12 2024-04-07T16:24:44.513 INFO:tasks.rados.rados.0.smithi100.stdout:2451: oid 12 updating version 0 to 1168 2024-04-07T16:24:44.513 INFO:tasks.rados.rados.0.smithi100.stdout:2451: oid 12 updating version 1168 to 1169 2024-04-07T16:24:44.513 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 12 v 1169 (ObjNum 748 snap 230 seq_num 748) dirty exists 2024-04-07T16:24:44.513 INFO:tasks.rados.rados.0.smithi100.stdout:2451: left oid 12 (ObjNum 748 snap 230 seq_num 748) 2024-04-07T16:24:44.513 INFO:tasks.rados.rados.0.smithi100.stdout:2452: finishing write tid 1 to smithi10066867-8 2024-04-07T16:24:44.513 INFO:tasks.rados.rados.0.smithi100.stdout:2452: finishing write tid 2 to smithi10066867-8 2024-04-07T16:24:44.513 INFO:tasks.rados.rados.0.smithi100.stdout:2452: finishing write tid 3 to smithi10066867-8 2024-04-07T16:24:44.513 INFO:tasks.rados.rados.0.smithi100.stdout:2452: oid 8 updating version 0 to 1000 2024-04-07T16:24:44.513 INFO:tasks.rados.rados.0.smithi100.stdout:2452: oid 8 updating version 1000 to 1001 2024-04-07T16:24:44.513 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 8 v 1001 (ObjNum 749 snap 230 seq_num 749) dirty exists 2024-04-07T16:24:44.514 INFO:tasks.rados.rados.0.smithi100.stdout:2452: left oid 8 (ObjNum 749 snap 230 seq_num 749) 2024-04-07T16:24:44.514 INFO:tasks.rados.rados.0.smithi100.stdout:2455: expect (ObjNum 465 snap 142 seq_num 465) 2024-04-07T16:24:44.702 INFO:tasks.rados.rados.0.smithi100.stdout:2458: finishing copy_from to smithi10066867-13 2024-04-07T16:24:44.702 INFO:tasks.rados.rados.0.smithi100.stdout:2458: got expected ENOENT (src dne) 2024-04-07T16:24:44.703 INFO:tasks.rados.rados.0.smithi100.stdout:2458: finishing copy_from racing read to smithi10066867-13 2024-04-07T16:24:44.703 INFO:tasks.rados.rados.0.smithi100.stdout:2451: done (4 left) 2024-04-07T16:24:44.703 INFO:tasks.rados.rados.0.smithi100.stdout:2452: done (3 left) 2024-04-07T16:24:44.703 INFO:tasks.rados.rados.0.smithi100.stdout:2455: done (2 left) 2024-04-07T16:24:44.703 INFO:tasks.rados.rados.0.smithi100.stdout:2456: done (1 left) 2024-04-07T16:24:44.703 INFO:tasks.rados.rados.0.smithi100.stdout:2458: done (0 left) 2024-04-07T16:24:44.703 INFO:tasks.rados.rados.0.smithi100.stdout:2459: snap_create 2024-04-07T16:24:44.856 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:44 smithi144 bash[66654]: cluster 2024-04-07T16:24:43.527630+0000 mon.a (mon.0) 731 : cluster [DBG] osdmap e529: 8 total, 8 up, 8 in 2024-04-07T16:24:44.934 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:44 smithi100 bash[74942]: cluster 2024-04-07T16:24:43.527630+0000 mon.a (mon.0) 731 : cluster [DBG] osdmap e529: 8 total, 8 up, 8 in 2024-04-07T16:24:44.934 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:44 smithi100 bash[76202]: cluster 2024-04-07T16:24:43.527630+0000 mon.a (mon.0) 731 : cluster [DBG] osdmap e529: 8 total, 8 up, 8 in 2024-04-07T16:24:45.540 INFO:tasks.rados.rados.0.smithi100.stdout:2459: done (0 left) 2024-04-07T16:24:45.540 INFO:tasks.rados.rados.0.smithi100.stdout:2460: setattr oid 23 current snap is 231 2024-04-07T16:24:45.546 INFO:tasks.rados.rados.0.smithi100.stdout:2461: snap_create 2024-04-07T16:24:45.554 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 23 v 1377 (ObjNum 746 snap 228 seq_num 746) dirty exists 2024-04-07T16:24:45.856 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:45 smithi144 bash[66654]: cluster 2024-04-07T16:24:44.526985+0000 mon.a (mon.0) 732 : cluster [DBG] osdmap e530: 8 total, 8 up, 8 in 2024-04-07T16:24:45.856 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:45 smithi144 bash[66654]: cluster 2024-04-07T16:24:44.710696+0000 mgr.y (mgr.25175) 227 : cluster [DBG] pgmap v387: 121 pgs: 1 active+undersized+degraded, 4 active+undersized, 116 active+clean; 329 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 5.1 MiB/s rd, 1.5 MiB/s wr, 10 op/s; 2/468 objects degraded (0.427%); 16 MiB/s, 6 objects/s recovering 2024-04-07T16:24:45.934 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:45 smithi100 bash[74942]: cluster 2024-04-07T16:24:44.526985+0000 mon.a (mon.0) 732 : cluster [DBG] osdmap e530: 8 total, 8 up, 8 in 2024-04-07T16:24:45.934 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:45 smithi100 bash[74942]: cluster 2024-04-07T16:24:44.710696+0000 mgr.y (mgr.25175) 227 : cluster [DBG] pgmap v387: 121 pgs: 1 active+undersized+degraded, 4 active+undersized, 116 active+clean; 329 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 5.1 MiB/s rd, 1.5 MiB/s wr, 10 op/s; 2/468 objects degraded (0.427%); 16 MiB/s, 6 objects/s recovering 2024-04-07T16:24:45.934 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:45 smithi100 bash[76202]: cluster 2024-04-07T16:24:44.526985+0000 mon.a (mon.0) 732 : cluster [DBG] osdmap e530: 8 total, 8 up, 8 in 2024-04-07T16:24:45.934 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:45 smithi100 bash[76202]: cluster 2024-04-07T16:24:44.710696+0000 mgr.y (mgr.25175) 227 : cluster [DBG] pgmap v387: 121 pgs: 1 active+undersized+degraded, 4 active+undersized, 116 active+clean; 329 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 5.1 MiB/s rd, 1.5 MiB/s wr, 10 op/s; 2/468 objects degraded (0.427%); 16 MiB/s, 6 objects/s recovering 2024-04-07T16:24:46.539 INFO:tasks.rados.rados.0.smithi100.stdout:2460: done (1 left) 2024-04-07T16:24:46.540 INFO:tasks.rados.rados.0.smithi100.stdout:2461: done (0 left) 2024-04-07T16:24:46.540 INFO:tasks.rados.rados.0.smithi100.stdout:2462: read oid 30 snap 230 2024-04-07T16:24:46.540 INFO:tasks.rados.rados.0.smithi100.stdout:2462: expect deleted 2024-04-07T16:24:46.540 INFO:tasks.rados.rados.0.smithi100.stdout:2463: read oid 25 snap -1 2024-04-07T16:24:46.540 INFO:tasks.rados.rados.0.smithi100.stdout:2463: expect (ObjNum 540 snap 164 seq_num 540) 2024-04-07T16:24:46.540 INFO:tasks.rados.rados.0.smithi100.stdout:2464: read oid 50 snap -1 2024-04-07T16:24:46.540 INFO:tasks.rados.rados.0.smithi100.stdout:2464: expect (ObjNum 657 snap 203 seq_num 657) 2024-04-07T16:24:46.540 INFO:tasks.rados.rados.0.smithi100.stdout:2465: snap_create 2024-04-07T16:24:46.578 INFO:tasks.rados.rados.0.smithi100.stdout:2464: expect (ObjNum 657 snap 203 seq_num 657) 2024-04-07T16:24:46.603 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:46 smithi144 bash[66654]: cluster 2024-04-07T16:24:45.558270+0000 mon.a (mon.0) 733 : cluster [DBG] osdmap e531: 8 total, 8 up, 8 in 2024-04-07T16:24:46.896 INFO:tasks.rados.rados.0.smithi100.stdout:2463: expect (ObjNum 540 snap 164 seq_num 540) 2024-04-07T16:24:46.962 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:46 smithi100 bash[74942]: cluster 2024-04-07T16:24:45.558270+0000 mon.a (mon.0) 733 : cluster [DBG] osdmap e531: 8 total, 8 up, 8 in 2024-04-07T16:24:46.963 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:46 smithi100 bash[76202]: cluster 2024-04-07T16:24:45.558270+0000 mon.a (mon.0) 733 : cluster [DBG] osdmap e531: 8 total, 8 up, 8 in 2024-04-07T16:24:47.101 INFO:tasks.rados.rados.0.smithi100.stdout:2462: done (3 left) 2024-04-07T16:24:47.101 INFO:tasks.rados.rados.0.smithi100.stdout:2463: done (2 left) 2024-04-07T16:24:47.101 INFO:tasks.rados.rados.0.smithi100.stdout:2464: done (1 left) 2024-04-07T16:24:47.102 INFO:tasks.rados.rados.0.smithi100.stdout:2465: done (0 left) 2024-04-07T16:24:47.102 INFO:tasks.rados.rados.0.smithi100.stdout:2466: snap_remove snap 231 2024-04-07T16:24:47.811 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:47 smithi144 bash[66654]: cluster 2024-04-07T16:24:46.554918+0000 mon.a (mon.0) 734 : cluster [DBG] osdmap e532: 8 total, 8 up, 8 in 2024-04-07T16:24:47.811 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:47 smithi144 bash[66654]: cluster 2024-04-07T16:24:46.711364+0000 mgr.y (mgr.25175) 228 : cluster [DBG] pgmap v390: 121 pgs: 121 active+clean; 329 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 8 op/s; 12 MiB/s, 4 objects/s recovering 2024-04-07T16:24:47.811 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:47 smithi144 bash[66654]: audit 2024-04-07T16:24:46.721974+0000 mon.a (mon.0) 735 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:24:47.811 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:47 smithi144 bash[66654]: audit 2024-04-07T16:24:46.724064+0000 mon.a (mon.0) 736 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:24:47.811 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:47 smithi144 bash[66654]: cluster 2024-04-07T16:24:46.842558+0000 mon.a (mon.0) 737 : cluster [WRN] Health check update: Degraded data redundancy: 2/468 objects degraded (0.427%), 1 pg degraded (PG_DEGRADED) 2024-04-07T16:24:47.812 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:47 smithi144 bash[66654]: cluster 2024-04-07T16:24:46.852867+0000 mon.a (mon.0) 738 : cluster [DBG] osdmap e533: 8 total, 8 up, 8 in 2024-04-07T16:24:47.839 INFO:tasks.rados.rados.0.smithi100.stdout:2466: done (0 left) 2024-04-07T16:24:47.839 INFO:tasks.rados.rados.0.smithi100.stdout:2467: rmattr oid 49 current snap is 233 2024-04-07T16:24:47.839 INFO:tasks.rados.rados.0.smithi100.stdout:2468: rollback oid 46 current snap is 233 2024-04-07T16:24:47.839 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 46 to 186 2024-04-07T16:24:47.839 INFO:tasks.rados.rados.0.smithi100.stdout:2469: snap_create 2024-04-07T16:24:47.848 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 49 v 1173 (ObjNum 722 snap 223 seq_num 722) dirty exists 2024-04-07T16:24:47.852 INFO:tasks.rados.rados.0.smithi100.stdout:2468: finishing rollback tid 0 to smithi10066867-46 2024-04-07T16:24:47.853 INFO:tasks.rados.rados.0.smithi100.stdout:2468: finishing rollback tid 1 to smithi10066867-46 2024-04-07T16:24:47.853 INFO:tasks.rados.rados.0.smithi100.stdout:2468: finishing rollback tid 2 to smithi10066867-46 2024-04-07T16:24:47.853 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 46 v 1007 (ObjNum 580 snap 180 seq_num 580) dirty exists 2024-04-07T16:24:47.934 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:47 smithi100 bash[74942]: cluster 2024-04-07T16:24:46.554918+0000 mon.a (mon.0) 734 : cluster [DBG] osdmap e532: 8 total, 8 up, 8 in 2024-04-07T16:24:47.934 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:47 smithi100 bash[74942]: cluster 2024-04-07T16:24:46.711364+0000 mgr.y (mgr.25175) 228 : cluster [DBG] pgmap v390: 121 pgs: 121 active+clean; 329 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 8 op/s; 12 MiB/s, 4 objects/s recovering 2024-04-07T16:24:47.934 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:47 smithi100 bash[74942]: audit 2024-04-07T16:24:46.721974+0000 mon.a (mon.0) 735 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:24:47.934 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:47 smithi100 bash[74942]: audit 2024-04-07T16:24:46.724064+0000 mon.a (mon.0) 736 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:24:47.934 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:47 smithi100 bash[74942]: cluster 2024-04-07T16:24:46.842558+0000 mon.a (mon.0) 737 : cluster [WRN] Health check update: Degraded data redundancy: 2/468 objects degraded (0.427%), 1 pg degraded (PG_DEGRADED) 2024-04-07T16:24:47.934 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:47 smithi100 bash[74942]: cluster 2024-04-07T16:24:46.852867+0000 mon.a (mon.0) 738 : cluster [DBG] osdmap e533: 8 total, 8 up, 8 in 2024-04-07T16:24:47.935 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:47 smithi100 bash[76202]: cluster 2024-04-07T16:24:46.554918+0000 mon.a (mon.0) 734 : cluster [DBG] osdmap e532: 8 total, 8 up, 8 in 2024-04-07T16:24:47.935 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:47 smithi100 bash[76202]: cluster 2024-04-07T16:24:46.711364+0000 mgr.y (mgr.25175) 228 : cluster [DBG] pgmap v390: 121 pgs: 121 active+clean; 329 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 8 op/s; 12 MiB/s, 4 objects/s recovering 2024-04-07T16:24:47.935 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:47 smithi100 bash[76202]: audit 2024-04-07T16:24:46.721974+0000 mon.a (mon.0) 735 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:24:47.935 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:47 smithi100 bash[76202]: audit 2024-04-07T16:24:46.724064+0000 mon.a (mon.0) 736 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:24:47.935 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:47 smithi100 bash[76202]: cluster 2024-04-07T16:24:46.842558+0000 mon.a (mon.0) 737 : cluster [WRN] Health check update: Degraded data redundancy: 2/468 objects degraded (0.427%), 1 pg degraded (PG_DEGRADED) 2024-04-07T16:24:47.935 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:47 smithi100 bash[76202]: cluster 2024-04-07T16:24:46.852867+0000 mon.a (mon.0) 738 : cluster [DBG] osdmap e533: 8 total, 8 up, 8 in 2024-04-07T16:24:48.684 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:24:48 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:24:48] "GET /metrics HTTP/1.1" 200 36281 "" "Prometheus/2.43.0" 2024-04-07T16:24:48.684 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:48 smithi100 bash[74942]: cluster 2024-04-07T16:24:47.559306+0000 mon.a (mon.0) 739 : cluster [INF] Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/468 objects degraded (0.427%), 1 pg degraded) 2024-04-07T16:24:48.684 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:48 smithi100 bash[74942]: cluster 2024-04-07T16:24:47.559373+0000 mon.a (mon.0) 740 : cluster [INF] Cluster is now healthy 2024-04-07T16:24:48.684 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:48 smithi100 bash[74942]: cluster 2024-04-07T16:24:47.856420+0000 mon.a (mon.0) 741 : cluster [DBG] osdmap e534: 8 total, 8 up, 8 in 2024-04-07T16:24:48.685 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:48 smithi100 bash[76202]: cluster 2024-04-07T16:24:47.559306+0000 mon.a (mon.0) 739 : cluster [INF] Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/468 objects degraded (0.427%), 1 pg degraded) 2024-04-07T16:24:48.685 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:48 smithi100 bash[76202]: cluster 2024-04-07T16:24:47.559373+0000 mon.a (mon.0) 740 : cluster [INF] Cluster is now healthy 2024-04-07T16:24:48.685 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:48 smithi100 bash[76202]: cluster 2024-04-07T16:24:47.856420+0000 mon.a (mon.0) 741 : cluster [DBG] osdmap e534: 8 total, 8 up, 8 in 2024-04-07T16:24:48.844 INFO:tasks.rados.rados.0.smithi100.stdout:2467: done (2 left) 2024-04-07T16:24:48.845 INFO:tasks.rados.rados.0.smithi100.stdout:2468: done (1 left) 2024-04-07T16:24:48.845 INFO:tasks.rados.rados.0.smithi100.stdout:2469: done (0 left) 2024-04-07T16:24:48.845 INFO:tasks.rados.rados.0.smithi100.stdout:2470: rmattr oid 32 current snap is 234 2024-04-07T16:24:48.845 INFO:tasks.rados.rados.0.smithi100.stdout:2471: rollback oid 9 current snap is 234 2024-04-07T16:24:48.845 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 9 to 186 2024-04-07T16:24:48.845 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 7 current snap is 234 2024-04-07T16:24:48.845 INFO:tasks.rados.rados.0.smithi100.stdout:2472: seq_num 752 ranges {6037504=598016} 2024-04-07T16:24:48.855 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:48 smithi144 bash[66654]: cluster 2024-04-07T16:24:47.559306+0000 mon.a (mon.0) 739 : cluster [INF] Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/468 objects degraded (0.427%), 1 pg degraded) 2024-04-07T16:24:48.856 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:48 smithi144 bash[66654]: cluster 2024-04-07T16:24:47.559373+0000 mon.a (mon.0) 740 : cluster [INF] Cluster is now healthy 2024-04-07T16:24:48.856 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:48 smithi144 bash[66654]: cluster 2024-04-07T16:24:47.856420+0000 mon.a (mon.0) 741 : cluster [DBG] osdmap e534: 8 total, 8 up, 8 in 2024-04-07T16:24:48.932 INFO:tasks.rados.rados.0.smithi100.stdout:2472: writing smithi10066867-7 from 6037504 to 6635520 tid 1 2024-04-07T16:24:48.932 INFO:tasks.rados.rados.0.smithi100.stdout:2473: snap_create 2024-04-07T16:24:48.932 INFO:tasks.rados.rados.0.smithi100.stdout:2471: finishing rollback tid 1 to smithi10066867-9 2024-04-07T16:24:48.932 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 9 v 1105 (ObjNum 145 snap 0 seq_num 2701969360) dirty dne 2024-04-07T16:24:48.932 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 32 v 1103 (ObjNum 723 snap 224 seq_num 723) dirty exists 2024-04-07T16:24:48.938 INFO:tasks.rados.rados.0.smithi100.stdout:2472: finishing write tid 1 to smithi10066867-7 2024-04-07T16:24:48.938 INFO:tasks.rados.rados.0.smithi100.stdout:2472: finishing write tid 2 to smithi10066867-7 2024-04-07T16:24:48.941 INFO:tasks.rados.rados.0.smithi100.stdout:2472: finishing write tid 3 to smithi10066867-7 2024-04-07T16:24:48.941 INFO:tasks.rados.rados.0.smithi100.stdout:2472: oid 7 updating version 0 to 1110 2024-04-07T16:24:48.941 INFO:tasks.rados.rados.0.smithi100.stdout:2472: oid 7 version 1110 is already newer than 1109 2024-04-07T16:24:48.941 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 7 v 1110 (ObjNum 752 snap 234 seq_num 752) dirty exists 2024-04-07T16:24:48.941 INFO:tasks.rados.rados.0.smithi100.stdout:2472: left oid 7 (ObjNum 752 snap 234 seq_num 752) 2024-04-07T16:24:49.854 INFO:tasks.rados.rados.0.smithi100.stdout:2470: done (3 left) 2024-04-07T16:24:49.855 INFO:tasks.rados.rados.0.smithi100.stdout:2471: done (2 left) 2024-04-07T16:24:49.855 INFO:tasks.rados.rados.0.smithi100.stdout:2472: done (1 left) 2024-04-07T16:24:49.855 INFO:tasks.rados.rados.0.smithi100.stdout:2473: done (0 left) 2024-04-07T16:24:49.855 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 33 current snap is 235 2024-04-07T16:24:49.855 INFO:tasks.rados.rados.0.smithi100.stdout:2474: seq_num 753 ranges {0=614400} 2024-04-07T16:24:49.860 INFO:tasks.rados.rados.0.smithi100.stdout:2474: writing smithi10066867-33 from 0 to 614400 tid 1 2024-04-07T16:24:49.860 INFO:tasks.rados.rados.0.smithi100.stdout:2475: snap_create 2024-04-07T16:24:49.870 INFO:tasks.rados.rados.0.smithi100.stdout:2474: finishing write tid 1 to smithi10066867-33 2024-04-07T16:24:49.870 INFO:tasks.rados.rados.0.smithi100.stdout:2474: finishing write tid 2 to smithi10066867-33 2024-04-07T16:24:49.873 INFO:tasks.rados.rados.0.smithi100.stdout:2474: finishing write tid 3 to smithi10066867-33 2024-04-07T16:24:49.873 INFO:tasks.rados.rados.0.smithi100.stdout:2474: oid 33 updating version 0 to 1379 2024-04-07T16:24:49.873 INFO:tasks.rados.rados.0.smithi100.stdout:2474: oid 33 version 1379 is already newer than 1378 2024-04-07T16:24:49.873 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 33 v 1379 (ObjNum 753 snap 235 seq_num 753) dirty exists 2024-04-07T16:24:49.873 INFO:tasks.rados.rados.0.smithi100.stdout:2474: left oid 33 (ObjNum 753 snap 235 seq_num 753) 2024-04-07T16:24:50.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:49 smithi144 bash[66654]: cluster 2024-04-07T16:24:48.712277+0000 mgr.y (mgr.25175) 229 : cluster [DBG] pgmap v393: 121 pgs: 121 active+clean; 330 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 6.3 MiB/s rd, 130 KiB/s wr, 5 op/s 2024-04-07T16:24:50.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:49 smithi144 bash[66654]: cluster 2024-04-07T16:24:48.862359+0000 mon.a (mon.0) 742 : cluster [DBG] osdmap e535: 8 total, 8 up, 8 in 2024-04-07T16:24:50.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:49 smithi144 bash[66654]: audit 2024-04-07T16:24:49.742574+0000 mon.a (mon.0) 743 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:24:50.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:49 smithi144 bash[66654]: audit 2024-04-07T16:24:49.756154+0000 mon.a (mon.0) 744 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:24:50.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:49 smithi144 bash[66654]: audit 2024-04-07T16:24:49.758789+0000 mon.a (mon.0) 745 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:24:50.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:49 smithi144 bash[66654]: audit 2024-04-07T16:24:49.760547+0000 mon.a (mon.0) 746 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:24:50.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:49 smithi144 bash[66654]: audit 2024-04-07T16:24:49.772907+0000 mon.a (mon.0) 747 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:24:50.184 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:49 smithi100 bash[74942]: cluster 2024-04-07T16:24:48.712277+0000 mgr.y (mgr.25175) 229 : cluster [DBG] pgmap v393: 121 pgs: 121 active+clean; 330 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 6.3 MiB/s rd, 130 KiB/s wr, 5 op/s 2024-04-07T16:24:50.184 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:49 smithi100 bash[74942]: cluster 2024-04-07T16:24:48.862359+0000 mon.a (mon.0) 742 : cluster [DBG] osdmap e535: 8 total, 8 up, 8 in 2024-04-07T16:24:50.184 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:49 smithi100 bash[74942]: audit 2024-04-07T16:24:49.742574+0000 mon.a (mon.0) 743 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:24:50.184 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:49 smithi100 bash[74942]: audit 2024-04-07T16:24:49.756154+0000 mon.a (mon.0) 744 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:24:50.184 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:49 smithi100 bash[74942]: audit 2024-04-07T16:24:49.758789+0000 mon.a (mon.0) 745 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:24:50.184 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:49 smithi100 bash[74942]: audit 2024-04-07T16:24:49.760547+0000 mon.a (mon.0) 746 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:24:50.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:49 smithi100 bash[74942]: audit 2024-04-07T16:24:49.772907+0000 mon.a (mon.0) 747 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:24:50.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:49 smithi100 bash[76202]: cluster 2024-04-07T16:24:48.712277+0000 mgr.y (mgr.25175) 229 : cluster [DBG] pgmap v393: 121 pgs: 121 active+clean; 330 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 6.3 MiB/s rd, 130 KiB/s wr, 5 op/s 2024-04-07T16:24:50.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:49 smithi100 bash[76202]: cluster 2024-04-07T16:24:48.862359+0000 mon.a (mon.0) 742 : cluster [DBG] osdmap e535: 8 total, 8 up, 8 in 2024-04-07T16:24:50.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:49 smithi100 bash[76202]: audit 2024-04-07T16:24:49.742574+0000 mon.a (mon.0) 743 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:24:50.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:49 smithi100 bash[76202]: audit 2024-04-07T16:24:49.756154+0000 mon.a (mon.0) 744 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:24:50.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:49 smithi100 bash[76202]: audit 2024-04-07T16:24:49.758789+0000 mon.a (mon.0) 745 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:24:50.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:49 smithi100 bash[76202]: audit 2024-04-07T16:24:49.760547+0000 mon.a (mon.0) 746 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:24:50.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:49 smithi100 bash[76202]: audit 2024-04-07T16:24:49.772907+0000 mon.a (mon.0) 747 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:24:50.860 INFO:tasks.rados.rados.0.smithi100.stdout:2474: done (1 left) 2024-04-07T16:24:50.860 INFO:tasks.rados.rados.0.smithi100.stdout:2475: done (0 left) 2024-04-07T16:24:50.860 INFO:tasks.rados.rados.0.smithi100.stdout:2476: delete oid 46 current snap is 236 2024-04-07T16:24:50.877 INFO:tasks.rados.rados.0.smithi100.stdout:2476: done (0 left) 2024-04-07T16:24:50.877 INFO:tasks.rados.rados.0.smithi100.stdout:2477: delete oid 40 current snap is 236 2024-04-07T16:24:50.888 INFO:tasks.rados.rados.0.smithi100.stdout:2477: done (0 left) 2024-04-07T16:24:50.888 INFO:tasks.rados.rados.0.smithi100.stdout:2478: snap_create 2024-04-07T16:24:51.095 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:50 smithi144 bash[66654]: audit 2024-04-07T16:24:49.863746+0000 mon.a (mon.0) 748 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:24:51.095 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:50 smithi144 bash[66654]: cluster 2024-04-07T16:24:49.872792+0000 mon.a (mon.0) 749 : cluster [DBG] osdmap e536: 8 total, 8 up, 8 in 2024-04-07T16:24:51.095 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:50 smithi144 bash[66654]: audit 2024-04-07T16:24:49.875673+0000 mon.a (mon.0) 750 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:24:51.095 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:50 smithi144 bash[66654]: audit 2024-04-07T16:24:49.878436+0000 mon.a (mon.0) 751 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:24:51.095 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:50 smithi144 bash[66654]: audit 2024-04-07T16:24:49.880459+0000 mon.a (mon.0) 752 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:24:51.095 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:50 smithi144 bash[66654]: audit 2024-04-07T16:24:49.883254+0000 mon.a (mon.0) 753 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd ok-to-stop", "ids": ["6"], "max": 16}]: dispatch 2024-04-07T16:24:51.095 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:50 smithi144 bash[66654]: audit 2024-04-07T16:24:49.883997+0000 mgr.y (mgr.25175) 230 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["6"], "max": 16}]: dispatch 2024-04-07T16:24:51.095 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:50 smithi144 bash[66654]: cephadm 2024-04-07T16:24:49.885794+0000 mgr.y (mgr.25175) 231 : cephadm [INF] Upgrade: osd.6 is safe to restart 2024-04-07T16:24:51.184 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:50 smithi100 bash[74942]: audit 2024-04-07T16:24:49.863746+0000 mon.a (mon.0) 748 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:24:51.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:50 smithi100 bash[74942]: cluster 2024-04-07T16:24:49.872792+0000 mon.a (mon.0) 749 : cluster [DBG] osdmap e536: 8 total, 8 up, 8 in 2024-04-07T16:24:51.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:50 smithi100 bash[74942]: audit 2024-04-07T16:24:49.875673+0000 mon.a (mon.0) 750 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:24:51.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:50 smithi100 bash[74942]: audit 2024-04-07T16:24:49.878436+0000 mon.a (mon.0) 751 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:24:51.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:50 smithi100 bash[74942]: audit 2024-04-07T16:24:49.880459+0000 mon.a (mon.0) 752 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:24:51.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:50 smithi100 bash[74942]: audit 2024-04-07T16:24:49.883254+0000 mon.a (mon.0) 753 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd ok-to-stop", "ids": ["6"], "max": 16}]: dispatch 2024-04-07T16:24:51.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:50 smithi100 bash[74942]: audit 2024-04-07T16:24:49.883997+0000 mgr.y (mgr.25175) 230 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["6"], "max": 16}]: dispatch 2024-04-07T16:24:51.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:50 smithi100 bash[74942]: cephadm 2024-04-07T16:24:49.885794+0000 mgr.y (mgr.25175) 231 : cephadm [INF] Upgrade: osd.6 is safe to restart 2024-04-07T16:24:51.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:50 smithi100 bash[76202]: audit 2024-04-07T16:24:49.863746+0000 mon.a (mon.0) 748 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:24:51.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:50 smithi100 bash[76202]: cluster 2024-04-07T16:24:49.872792+0000 mon.a (mon.0) 749 : cluster [DBG] osdmap e536: 8 total, 8 up, 8 in 2024-04-07T16:24:51.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:50 smithi100 bash[76202]: audit 2024-04-07T16:24:49.875673+0000 mon.a (mon.0) 750 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:24:51.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:50 smithi100 bash[76202]: audit 2024-04-07T16:24:49.878436+0000 mon.a (mon.0) 751 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:24:51.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:50 smithi100 bash[76202]: audit 2024-04-07T16:24:49.880459+0000 mon.a (mon.0) 752 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:24:51.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:50 smithi100 bash[76202]: audit 2024-04-07T16:24:49.883254+0000 mon.a (mon.0) 753 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd ok-to-stop", "ids": ["6"], "max": 16}]: dispatch 2024-04-07T16:24:51.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:50 smithi100 bash[76202]: audit 2024-04-07T16:24:49.883997+0000 mgr.y (mgr.25175) 230 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["6"], "max": 16}]: dispatch 2024-04-07T16:24:51.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:50 smithi100 bash[76202]: cephadm 2024-04-07T16:24:49.885794+0000 mgr.y (mgr.25175) 231 : cephadm [INF] Upgrade: osd.6 is safe to restart 2024-04-07T16:24:51.838 INFO:tasks.rados.rados.0.smithi100.stdout:2478: done (0 left) 2024-04-07T16:24:51.838 INFO:tasks.rados.rados.0.smithi100.stdout:2479: read oid 50 snap -1 2024-04-07T16:24:51.838 INFO:tasks.rados.rados.0.smithi100.stdout:2479: expect (ObjNum 657 snap 203 seq_num 657) 2024-04-07T16:24:51.838 INFO:tasks.rados.rados.0.smithi100.stdout:2480: read oid 18 snap -1 2024-04-07T16:24:51.839 INFO:tasks.rados.rados.0.smithi100.stdout:2480: expect deleted 2024-04-07T16:24:51.839 INFO:tasks.rados.rados.0.smithi100.stdout:2481: rmattr oid 13 current snap is 237 2024-04-07T16:24:51.839 INFO:tasks.rados.rados.0.smithi100.stdout:2481: done (2 left) 2024-04-07T16:24:51.839 INFO:tasks.rados.rados.0.smithi100.stdout:2482: rmattr oid 11 current snap is 237 2024-04-07T16:24:51.839 INFO:tasks.rados.rados.0.smithi100.stdout:2482: done (2 left) 2024-04-07T16:24:51.839 INFO:tasks.rados.rados.0.smithi100.stdout:2483: snap_create 2024-04-07T16:24:51.871 INFO:tasks.rados.rados.0.smithi100.stdout:2479: expect (ObjNum 657 snap 203 seq_num 657) 2024-04-07T16:24:52.184 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:51 smithi100 bash[74942]: cluster 2024-04-07T16:24:50.713259+0000 mgr.y (mgr.25175) 232 : cluster [DBG] pgmap v396: 121 pgs: 121 active+clean; 330 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 10 MiB/s rd, 276 KiB/s wr, 13 op/s 2024-04-07T16:24:52.184 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:51 smithi100 bash[74942]: cluster 2024-04-07T16:24:50.878231+0000 mon.a (mon.0) 754 : cluster [DBG] osdmap e537: 8 total, 8 up, 8 in 2024-04-07T16:24:52.184 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:51 smithi100 bash[74942]: cluster 2024-04-07T16:24:51.856091+0000 mon.a (mon.0) 755 : cluster [DBG] osdmap e538: 8 total, 8 up, 8 in 2024-04-07T16:24:52.184 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:51 smithi100 bash[76202]: cluster 2024-04-07T16:24:50.713259+0000 mgr.y (mgr.25175) 232 : cluster [DBG] pgmap v396: 121 pgs: 121 active+clean; 330 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 10 MiB/s rd, 276 KiB/s wr, 13 op/s 2024-04-07T16:24:52.184 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:51 smithi100 bash[76202]: cluster 2024-04-07T16:24:50.878231+0000 mon.a (mon.0) 754 : cluster [DBG] osdmap e537: 8 total, 8 up, 8 in 2024-04-07T16:24:52.184 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:51 smithi100 bash[76202]: cluster 2024-04-07T16:24:51.856091+0000 mon.a (mon.0) 755 : cluster [DBG] osdmap e538: 8 total, 8 up, 8 in 2024-04-07T16:24:52.305 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:51 smithi144 bash[66654]: cluster 2024-04-07T16:24:50.713259+0000 mgr.y (mgr.25175) 232 : cluster [DBG] pgmap v396: 121 pgs: 121 active+clean; 330 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 10 MiB/s rd, 276 KiB/s wr, 13 op/s 2024-04-07T16:24:52.305 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:51 smithi144 bash[66654]: cluster 2024-04-07T16:24:50.878231+0000 mon.a (mon.0) 754 : cluster [DBG] osdmap e537: 8 total, 8 up, 8 in 2024-04-07T16:24:52.305 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:51 smithi144 bash[66654]: cluster 2024-04-07T16:24:51.856091+0000 mon.a (mon.0) 755 : cluster [DBG] osdmap e538: 8 total, 8 up, 8 in 2024-04-07T16:24:52.606 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:24:52 smithi144 bash[59794]: ts=2024-04-07T16:24:52.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:24:52.845 INFO:tasks.rados.rados.0.smithi100.stdout:2479: done (2 left) 2024-04-07T16:24:52.845 INFO:tasks.rados.rados.0.smithi100.stdout:2480: done (1 left) 2024-04-07T16:24:52.845 INFO:tasks.rados.rados.0.smithi100.stdout:2483: done (0 left) 2024-04-07T16:24:52.845 INFO:tasks.rados.rados.0.smithi100.stdout:2484: copy_from oid 48 from oid 32 current snap is 238 2024-04-07T16:24:52.846 INFO:tasks.rados.rados.0.smithi100.stdout:2485: rollback oid 19 current snap is 238 2024-04-07T16:24:52.846 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 19 to 186 2024-04-07T16:24:52.846 INFO:tasks.rados.rados.0.smithi100.stdout:2486: copy_from oid 36 from oid 37 current snap is 238 2024-04-07T16:24:52.846 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 39 current snap is 238 2024-04-07T16:24:52.846 INFO:tasks.rados.rados.0.smithi100.stdout:2487: seq_num 756 ranges {6209536=671744} 2024-04-07T16:24:52.934 INFO:tasks.rados.rados.0.smithi100.stdout:2487: writing smithi10066867-39 from 6209536 to 6881280 tid 1 2024-04-07T16:24:52.934 INFO:tasks.rados.rados.0.smithi100.stdout:2488: snap_create 2024-04-07T16:24:52.934 INFO:tasks.rados.rados.0.smithi100.stdout:2485: finishing rollback tid 0 to smithi10066867-19 2024-04-07T16:24:52.934 INFO:tasks.rados.rados.0.smithi100.stdout:2485: finishing rollback tid 1 to smithi10066867-19 2024-04-07T16:24:52.934 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 19 v 1124 (ObjNum 145 snap 0 seq_num 2702639760) dirty dne 2024-04-07T16:24:52.934 INFO:tasks.rados.rados.0.smithi100.stdout:2486: finishing copy_from to smithi10066867-36 2024-04-07T16:24:52.934 INFO:tasks.rados.rados.0.smithi100.stdout:2486: got expected ENOENT (src dne) 2024-04-07T16:24:52.934 INFO:tasks.rados.rados.0.smithi100.stdout:2486: finishing copy_from racing read to smithi10066867-36 2024-04-07T16:24:52.934 INFO:tasks.rados.rados.0.smithi100.stdout:2484: finishing copy_from racing read to smithi10066867-48 2024-04-07T16:24:52.935 INFO:tasks.rados.rados.0.smithi100.stdout:2484: finishing copy_from to smithi10066867-48 2024-04-07T16:24:52.935 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 48 v 1103 (ObjNum 723 snap 224 seq_num 723) dirty exists 2024-04-07T16:24:52.937 INFO:tasks.rados.rados.0.smithi100.stdout:2487: finishing write tid 1 to smithi10066867-39 2024-04-07T16:24:52.937 INFO:tasks.rados.rados.0.smithi100.stdout:2487: finishing write tid 2 to smithi10066867-39 2024-04-07T16:24:52.940 INFO:tasks.rados.rados.0.smithi100.stdout:2487: finishing write tid 3 to smithi10066867-39 2024-04-07T16:24:52.940 INFO:tasks.rados.rados.0.smithi100.stdout:2487: oid 39 updating version 0 to 1128 2024-04-07T16:24:52.940 INFO:tasks.rados.rados.0.smithi100.stdout:2487: oid 39 version 1128 is already newer than 1127 2024-04-07T16:24:52.940 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 39 v 1128 (ObjNum 756 snap 238 seq_num 756) dirty exists 2024-04-07T16:24:52.940 INFO:tasks.rados.rados.0.smithi100.stdout:2487: left oid 39 (ObjNum 756 snap 238 seq_num 756) 2024-04-07T16:24:53.178 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:52 smithi144 bash[66654]: cephadm 2024-04-07T16:24:51.906348+0000 mgr.y (mgr.25175) 233 : cephadm [INF] Upgrade: Updating osd.6 2024-04-07T16:24:53.178 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:52 smithi144 bash[66654]: audit 2024-04-07T16:24:51.915892+0000 mon.a (mon.0) 756 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:24:53.178 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:52 smithi144 bash[66654]: audit 2024-04-07T16:24:51.921770+0000 mon.a (mon.0) 757 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-07T16:24:53.178 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:52 smithi144 bash[66654]: audit 2024-04-07T16:24:51.923226+0000 mon.a (mon.0) 758 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:24:53.178 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:52 smithi144 bash[66654]: cephadm 2024-04-07T16:24:51.928499+0000 mgr.y (mgr.25175) 234 : cephadm [INF] Deploying daemon osd.6 on smithi144 2024-04-07T16:24:53.178 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:52 smithi144 bash[66654]: audit 2024-04-07T16:24:51.945452+0000 mon.a (mon.0) 759 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:24:53.178 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:52 smithi144 bash[66654]: cluster 2024-04-07T16:24:52.862793+0000 mon.a (mon.0) 760 : cluster [DBG] osdmap e539: 8 total, 8 up, 8 in 2024-04-07T16:24:53.184 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:52 smithi100 bash[74942]: cephadm 2024-04-07T16:24:51.906348+0000 mgr.y (mgr.25175) 233 : cephadm [INF] Upgrade: Updating osd.6 2024-04-07T16:24:53.184 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:52 smithi100 bash[74942]: audit 2024-04-07T16:24:51.915892+0000 mon.a (mon.0) 756 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:24:53.184 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:52 smithi100 bash[74942]: audit 2024-04-07T16:24:51.921770+0000 mon.a (mon.0) 757 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-07T16:24:53.184 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:52 smithi100 bash[74942]: audit 2024-04-07T16:24:51.923226+0000 mon.a (mon.0) 758 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:24:53.184 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:52 smithi100 bash[74942]: cephadm 2024-04-07T16:24:51.928499+0000 mgr.y (mgr.25175) 234 : cephadm [INF] Deploying daemon osd.6 on smithi144 2024-04-07T16:24:53.184 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:52 smithi100 bash[74942]: audit 2024-04-07T16:24:51.945452+0000 mon.a (mon.0) 759 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:24:53.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:52 smithi100 bash[74942]: cluster 2024-04-07T16:24:52.862793+0000 mon.a (mon.0) 760 : cluster [DBG] osdmap e539: 8 total, 8 up, 8 in 2024-04-07T16:24:53.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:52 smithi100 bash[76202]: cephadm 2024-04-07T16:24:51.906348+0000 mgr.y (mgr.25175) 233 : cephadm [INF] Upgrade: Updating osd.6 2024-04-07T16:24:53.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:52 smithi100 bash[76202]: audit 2024-04-07T16:24:51.915892+0000 mon.a (mon.0) 756 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:24:53.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:52 smithi100 bash[76202]: audit 2024-04-07T16:24:51.921770+0000 mon.a (mon.0) 757 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-07T16:24:53.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:52 smithi100 bash[76202]: audit 2024-04-07T16:24:51.923226+0000 mon.a (mon.0) 758 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:24:53.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:52 smithi100 bash[76202]: cephadm 2024-04-07T16:24:51.928499+0000 mgr.y (mgr.25175) 234 : cephadm [INF] Deploying daemon osd.6 on smithi144 2024-04-07T16:24:53.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:52 smithi100 bash[76202]: audit 2024-04-07T16:24:51.945452+0000 mon.a (mon.0) 759 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:24:53.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:52 smithi100 bash[76202]: cluster 2024-04-07T16:24:52.862793+0000 mon.a (mon.0) 760 : cluster [DBG] osdmap e539: 8 total, 8 up, 8 in 2024-04-07T16:24:53.939 INFO:tasks.rados.rados.0.smithi100.stdout:2484: done (4 left) 2024-04-07T16:24:53.940 INFO:tasks.rados.rados.0.smithi100.stdout:2485: done (3 left) 2024-04-07T16:24:53.940 INFO:tasks.rados.rados.0.smithi100.stdout:2486: done (2 left) 2024-04-07T16:24:53.940 INFO:tasks.rados.rados.0.smithi100.stdout:2487: done (1 left) 2024-04-07T16:24:53.940 INFO:tasks.rados.rados.0.smithi100.stdout:2488: done (0 left) 2024-04-07T16:24:53.940 INFO:tasks.rados.rados.0.smithi100.stdout:2489: read oid 28 snap -1 2024-04-07T16:24:53.940 INFO:tasks.rados.rados.0.smithi100.stdout:2489: expect (ObjNum 742 snap 228 seq_num 742) 2024-04-07T16:24:53.940 INFO:tasks.rados.rados.0.smithi100.stdout:2490: read oid 50 snap -1 2024-04-07T16:24:53.940 INFO:tasks.rados.rados.0.smithi100.stdout:2490: expect (ObjNum 657 snap 203 seq_num 657) 2024-04-07T16:24:53.940 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 15 current snap is 239 2024-04-07T16:24:53.940 INFO:tasks.rados.rados.0.smithi100.stdout:2491: seq_num 757 ranges {3702784=688128} 2024-04-07T16:24:53.964 INFO:tasks.rados.rados.0.smithi100.stdout:2491: writing smithi10066867-15 from 3702784 to 4390912 tid 1 2024-04-07T16:24:53.964 INFO:tasks.rados.rados.0.smithi100.stdout:2492: snap_remove snap 232 2024-04-07T16:24:53.964 INFO:tasks.rados.rados.0.smithi100.stdout:2489: expect (ObjNum 742 snap 228 seq_num 742) 2024-04-07T16:24:53.980 INFO:tasks.rados.rados.0.smithi100.stdout:2490: expect (ObjNum 657 snap 203 seq_num 657) 2024-04-07T16:24:54.312 INFO:tasks.rados.rados.0.smithi100.stdout:2491: finishing write tid 1 to smithi10066867-15 2024-04-07T16:24:54.313 INFO:tasks.rados.rados.0.smithi100.stdout:2491: finishing write tid 2 to smithi10066867-15 2024-04-07T16:24:54.313 INFO:tasks.rados.rados.0.smithi100.stdout:2491: finishing write tid 3 to smithi10066867-15 2024-04-07T16:24:54.313 INFO:tasks.rados.rados.0.smithi100.stdout:2491: oid 15 updating version 0 to 1382 2024-04-07T16:24:54.313 INFO:tasks.rados.rados.0.smithi100.stdout:2491: oid 15 version 1382 is already newer than 1381 2024-04-07T16:24:54.313 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 15 v 1382 (ObjNum 757 snap 239 seq_num 757) dirty exists 2024-04-07T16:24:54.313 INFO:tasks.rados.rados.0.smithi100.stdout:2491: left oid 15 (ObjNum 757 snap 239 seq_num 757) 2024-04-07T16:24:54.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:53 smithi144 bash[66654]: cluster 2024-04-07T16:24:52.714401+0000 mgr.y (mgr.25175) 235 : cluster [DBG] pgmap v399: 121 pgs: 121 active+clean; 330 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 7.5 MiB/s rd, 146 KiB/s wr, 10 op/s 2024-04-07T16:24:54.434 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:53 smithi100 bash[74942]: cluster 2024-04-07T16:24:52.714401+0000 mgr.y (mgr.25175) 235 : cluster [DBG] pgmap v399: 121 pgs: 121 active+clean; 330 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 7.5 MiB/s rd, 146 KiB/s wr, 10 op/s 2024-04-07T16:24:54.434 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:53 smithi100 bash[76202]: cluster 2024-04-07T16:24:52.714401+0000 mgr.y (mgr.25175) 235 : cluster [DBG] pgmap v399: 121 pgs: 121 active+clean; 330 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 7.5 MiB/s rd, 146 KiB/s wr, 10 op/s 2024-04-07T16:24:54.962 INFO:tasks.rados.rados.0.smithi100.stdout:2489: done (3 left) 2024-04-07T16:24:54.962 INFO:tasks.rados.rados.0.smithi100.stdout:2490: done (2 left) 2024-04-07T16:24:54.962 INFO:tasks.rados.rados.0.smithi100.stdout:2491: done (1 left) 2024-04-07T16:24:54.962 INFO:tasks.rados.rados.0.smithi100.stdout:2492: done (0 left) 2024-04-07T16:24:54.962 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 41 current snap is 239 2024-04-07T16:24:54.963 INFO:tasks.rados.rados.0.smithi100.stdout:2493: seq_num 758 ranges {1867776=704512} 2024-04-07T16:24:54.998 INFO:tasks.rados.rados.0.smithi100.stdout:2493: writing smithi10066867-41 from 1867776 to 2572288 tid 1 2024-04-07T16:24:54.998 INFO:tasks.rados.rados.0.smithi100.stdout:2494: rollback oid 31 current snap is 239 2024-04-07T16:24:54.998 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 31 to 222 2024-04-07T16:24:54.998 INFO:tasks.rados.rados.0.smithi100.stdout:2495: read oid 24 snap 234 2024-04-07T16:24:54.998 INFO:tasks.rados.rados.0.smithi100.stdout:2495: expect (ObjNum 732 snap 225 seq_num 732) 2024-04-07T16:24:54.998 INFO:tasks.rados.rados.0.smithi100.stdout:2496: read oid 13 snap 216 2024-04-07T16:24:54.998 INFO:tasks.rados.rados.0.smithi100.stdout:2496: expect deleted 2024-04-07T16:24:54.998 INFO:tasks.rados.rados.0.smithi100.stdout:2497: read oid 44 snap -1 2024-04-07T16:24:54.999 INFO:tasks.rados.rados.0.smithi100.stdout:2497: expect (ObjNum 509 snap 152 seq_num 509) 2024-04-07T16:24:54.999 INFO:tasks.rados.rados.0.smithi100.stdout:2498: copy_from oid 35 from oid 18 current snap is 239 2024-04-07T16:24:54.999 INFO:tasks.rados.rados.0.smithi100.stdout:2499: read oid 29 snap -1 2024-04-07T16:24:54.999 INFO:tasks.rados.rados.0.smithi100.stdout:2499: expect deleted 2024-04-07T16:24:54.999 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 34 current snap is 239 2024-04-07T16:24:54.999 INFO:tasks.rados.rados.0.smithi100.stdout:2500: seq_num 759 ranges {49152=720896} 2024-04-07T16:24:55.007 INFO:tasks.rados.rados.0.smithi100.stdout:2500: writing smithi10066867-34 from 49152 to 770048 tid 1 2024-04-07T16:24:55.007 INFO:tasks.rados.rados.0.smithi100.stdout:2501: snap_create 2024-04-07T16:24:55.009 INFO:tasks.rados.rados.0.smithi100.stdout:2494: finishing rollback tid 1 to smithi10066867-31 2024-04-07T16:24:55.009 INFO:tasks.rados.rados.0.smithi100.stdout:2494: finishing rollback tid 2 to smithi10066867-31 2024-04-07T16:24:55.009 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 31 v 1105 (ObjNum 579 snap 180 seq_num 579) dirty exists 2024-04-07T16:24:55.010 INFO:tasks.rados.rados.0.smithi100.stdout:2493: finishing write tid 1 to smithi10066867-41 2024-04-07T16:24:55.010 INFO:tasks.rados.rados.0.smithi100.stdout:2498: finishing copy_from to smithi10066867-35 2024-04-07T16:24:55.011 INFO:tasks.rados.rados.0.smithi100.stdout:2498: got expected ENOENT (src dne) 2024-04-07T16:24:55.011 INFO:tasks.rados.rados.0.smithi100.stdout:2498: finishing copy_from racing read to smithi10066867-35 2024-04-07T16:24:55.011 INFO:tasks.rados.rados.0.smithi100.stdout:2493: finishing write tid 2 to smithi10066867-41 2024-04-07T16:24:55.018 INFO:tasks.rados.rados.0.smithi100.stdout:2493: finishing write tid 3 to smithi10066867-41 2024-04-07T16:24:55.018 INFO:tasks.rados.rados.0.smithi100.stdout:2493: oid 41 updating version 0 to 1178 2024-04-07T16:24:55.019 INFO:tasks.rados.rados.0.smithi100.stdout:2493: oid 41 version 1178 is already newer than 1177 2024-04-07T16:24:55.019 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 41 v 1178 (ObjNum 758 snap 239 seq_num 758) dirty exists 2024-04-07T16:24:55.019 INFO:tasks.rados.rados.0.smithi100.stdout:2493: left oid 41 (ObjNum 758 snap 239 seq_num 758) 2024-04-07T16:24:55.019 INFO:tasks.rados.rados.0.smithi100.stdout:2495: expect (ObjNum 732 snap 225 seq_num 732) 2024-04-07T16:24:55.073 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:54 smithi144 bash[66654]: cluster 2024-04-07T16:24:53.952440+0000 mon.a (mon.0) 761 : cluster [DBG] osdmap e540: 8 total, 8 up, 8 in 2024-04-07T16:24:55.158 INFO:tasks.rados.rados.0.smithi100.stdout:2500: finishing write tid 1 to smithi10066867-34 2024-04-07T16:24:55.158 INFO:tasks.rados.rados.0.smithi100.stdout:2500: finishing write tid 2 to smithi10066867-34 2024-04-07T16:24:55.158 INFO:tasks.rados.rados.0.smithi100.stdout:2500: finishing write tid 3 to smithi10066867-34 2024-04-07T16:24:55.158 INFO:tasks.rados.rados.0.smithi100.stdout:2500: oid 34 updating version 0 to 1014 2024-04-07T16:24:55.158 INFO:tasks.rados.rados.0.smithi100.stdout:2500: oid 34 version 1014 is already newer than 1013 2024-04-07T16:24:55.158 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 34 v 1014 (ObjNum 759 snap 239 seq_num 759) dirty exists 2024-04-07T16:24:55.158 INFO:tasks.rados.rados.0.smithi100.stdout:2500: left oid 34 (ObjNum 759 snap 239 seq_num 759) 2024-04-07T16:24:55.158 INFO:tasks.rados.rados.0.smithi100.stdout:2497: expect (ObjNum 509 snap 152 seq_num 509) 2024-04-07T16:24:55.434 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:54 smithi100 bash[74942]: cluster 2024-04-07T16:24:53.952440+0000 mon.a (mon.0) 761 : cluster [DBG] osdmap e540: 8 total, 8 up, 8 in 2024-04-07T16:24:55.434 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:54 smithi100 bash[76202]: cluster 2024-04-07T16:24:53.952440+0000 mon.a (mon.0) 761 : cluster [DBG] osdmap e540: 8 total, 8 up, 8 in 2024-04-07T16:24:55.968 INFO:tasks.rados.rados.0.smithi100.stdout:2493: done (8 left) 2024-04-07T16:24:55.968 INFO:tasks.rados.rados.0.smithi100.stdout:2494: done (7 left) 2024-04-07T16:24:55.968 INFO:tasks.rados.rados.0.smithi100.stdout:2495: done (6 left) 2024-04-07T16:24:55.968 INFO:tasks.rados.rados.0.smithi100.stdout:2496: done (5 left) 2024-04-07T16:24:55.968 INFO:tasks.rados.rados.0.smithi100.stdout:2497: done (4 left) 2024-04-07T16:24:55.969 INFO:tasks.rados.rados.0.smithi100.stdout:2498: done (3 left) 2024-04-07T16:24:55.969 INFO:tasks.rados.rados.0.smithi100.stdout:2499: done (2 left) 2024-04-07T16:24:55.969 INFO:tasks.rados.rados.0.smithi100.stdout:2500: done (1 left) 2024-04-07T16:24:55.969 INFO:tasks.rados.rados.0.smithi100.stdout:2501: done (0 left) 2024-04-07T16:24:55.969 INFO:tasks.rados.rados.0.smithi100.stdout:2502: copy_from oid 2 from oid 45 current snap is 240 2024-04-07T16:24:55.969 INFO:tasks.rados.rados.0.smithi100.stdout:2503: rmattr oid 23 current snap is 240 2024-04-07T16:24:55.969 INFO:tasks.rados.rados.0.smithi100.stdout:2504: rmattr oid 38 current snap is 240 2024-04-07T16:24:55.969 INFO:tasks.rados.rados.0.smithi100.stdout:2504: done (2 left) 2024-04-07T16:24:55.969 INFO:tasks.rados.rados.0.smithi100.stdout:2505: delete oid 48 current snap is 240 2024-04-07T16:24:55.975 INFO:tasks.rados.rados.0.smithi100.stdout:2502: finishing copy_from to smithi10066867-2 2024-04-07T16:24:55.976 INFO:tasks.rados.rados.0.smithi100.stdout:2502: got expected ENOENT (src dne) 2024-04-07T16:24:55.976 INFO:tasks.rados.rados.0.smithi100.stdout:2502: finishing copy_from racing read to smithi10066867-2 2024-04-07T16:24:55.979 INFO:tasks.rados.rados.0.smithi100.stdout:2502: done (2 left) 2024-04-07T16:24:55.979 INFO:tasks.rados.rados.0.smithi100.stdout:2505: done (1 left) 2024-04-07T16:24:55.979 INFO:tasks.rados.rados.0.smithi100.stdout:2506: snap_remove snap 229 2024-04-07T16:24:56.243 INFO:journalctl@ceph.mds.b.smithi144.stdout:Apr 07 16:24:56 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:24:56.243 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:24:56 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:24:56.244 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 16:24:56 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:24:56.244 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:24:56 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:24:56.244 INFO:journalctl@ceph.osd.5.smithi144.stdout:Apr 07 16:24:56 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:24:56.245 INFO:journalctl@ceph.osd.7.smithi144.stdout:Apr 07 16:24:56 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:24:56.245 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:55 smithi144 bash[66654]: cluster 2024-04-07T16:24:54.715293+0000 mgr.y (mgr.25175) 236 : cluster [DBG] pgmap v402: 121 pgs: 121 active+clean; 333 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 3.7 MiB/s rd, 678 KiB/s wr, 6 op/s 2024-04-07T16:24:56.245 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:55 smithi144 bash[66654]: cluster 2024-04-07T16:24:54.979147+0000 mon.a (mon.0) 762 : cluster [DBG] osdmap e541: 8 total, 8 up, 8 in 2024-04-07T16:24:56.245 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:56 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:24:56.245 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:24:56 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:24:56.246 INFO:journalctl@ceph.osd.4.smithi144.stdout:Apr 07 16:24:56 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:24:56.246 INFO:journalctl@ceph.osd.6.smithi144.stdout:Apr 07 16:24:56 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:24:56.246 INFO:journalctl@ceph.osd.6.smithi144.stdout:Apr 07 16:24:56 smithi144 systemd[1]: Stopping Ceph osd.6 for 0702e3a8-f4f8-11ee-b649-cb9ed24678a4... 2024-04-07T16:24:56.434 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:55 smithi100 bash[74942]: cluster 2024-04-07T16:24:54.715293+0000 mgr.y (mgr.25175) 236 : cluster [DBG] pgmap v402: 121 pgs: 121 active+clean; 333 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 3.7 MiB/s rd, 678 KiB/s wr, 6 op/s 2024-04-07T16:24:56.434 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:55 smithi100 bash[74942]: cluster 2024-04-07T16:24:54.979147+0000 mon.a (mon.0) 762 : cluster [DBG] osdmap e541: 8 total, 8 up, 8 in 2024-04-07T16:24:56.434 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:55 smithi100 bash[76202]: cluster 2024-04-07T16:24:54.715293+0000 mgr.y (mgr.25175) 236 : cluster [DBG] pgmap v402: 121 pgs: 121 active+clean; 333 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 3.7 MiB/s rd, 678 KiB/s wr, 6 op/s 2024-04-07T16:24:56.435 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:55 smithi100 bash[76202]: cluster 2024-04-07T16:24:54.979147+0000 mon.a (mon.0) 762 : cluster [DBG] osdmap e541: 8 total, 8 up, 8 in 2024-04-07T16:24:56.606 INFO:journalctl@ceph.osd.6.smithi144.stdout:Apr 07 16:24:56 smithi144 bash[38128]: debug 2024-04-07T16:24:56.353+0000 7f36dc41b700 -1 received signal: Terminated from /sbin/docker-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug (PID: 1) UID: 0 2024-04-07T16:24:56.606 INFO:journalctl@ceph.osd.6.smithi144.stdout:Apr 07 16:24:56 smithi144 bash[38128]: debug 2024-04-07T16:24:56.353+0000 7f36dc41b700 -1 osd.6 542 *** Got signal Terminated *** 2024-04-07T16:24:56.606 INFO:journalctl@ceph.osd.6.smithi144.stdout:Apr 07 16:24:56 smithi144 bash[38128]: debug 2024-04-07T16:24:56.353+0000 7f36dc41b700 -1 osd.6 542 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-07T16:24:56.847 INFO:tasks.rados.rados.0.smithi100.stdout:2506: done (1 left) 2024-04-07T16:24:56.848 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 10 current snap is 240 2024-04-07T16:24:56.848 INFO:tasks.rados.rados.0.smithi100.stdout:2507: seq_num 761 ranges {0=753664} 2024-04-07T16:24:56.853 INFO:tasks.rados.rados.0.smithi100.stdout:2507: writing smithi10066867-10 from 0 to 753664 tid 1 2024-04-07T16:24:56.854 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 23 v 1386 (ObjNum 746 snap 228 seq_num 746) dirty exists 2024-04-07T16:24:56.854 INFO:tasks.rados.rados.0.smithi100.stdout:2503: done (1 left) 2024-04-07T16:24:56.854 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 14 current snap is 240 2024-04-07T16:24:56.854 INFO:tasks.rados.rados.0.smithi100.stdout:2508: seq_num 762 ranges {0=770048} 2024-04-07T16:24:56.860 INFO:tasks.rados.rados.0.smithi100.stdout:2508: writing smithi10066867-14 from 0 to 770048 tid 1 2024-04-07T16:24:56.860 INFO:tasks.rados.rados.0.smithi100.stdout:2509: snap_remove snap 238 2024-04-07T16:24:57.184 INFO:journalctl@ceph.osd.3.smithi100.stdout:Apr 07 16:24:56 smithi100 bash[93998]: debug 2024-04-07T16:24:56.904+0000 7f6e92a8c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 144 after trimming was started 2024-04-07T16:24:57.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:56 smithi100 bash[74942]: cluster 2024-04-07T16:24:55.985677+0000 mon.a (mon.0) 763 : cluster [DBG] osdmap e542: 8 total, 8 up, 8 in 2024-04-07T16:24:57.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:56 smithi100 bash[74942]: cluster 2024-04-07T16:24:56.367825+0000 mon.a (mon.0) 764 : cluster [INF] osd.6 marked itself down and dead 2024-04-07T16:24:57.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:56 smithi100 bash[74942]: cluster 2024-04-07T16:24:56.849790+0000 mon.a (mon.0) 765 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN) 2024-04-07T16:24:57.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:56 smithi100 bash[74942]: cluster 2024-04-07T16:24:56.864222+0000 mon.a (mon.0) 766 : cluster [DBG] osdmap e543: 8 total, 7 up, 8 in 2024-04-07T16:24:57.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:56 smithi100 bash[76202]: cluster 2024-04-07T16:24:55.985677+0000 mon.a (mon.0) 763 : cluster [DBG] osdmap e542: 8 total, 8 up, 8 in 2024-04-07T16:24:57.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:56 smithi100 bash[76202]: cluster 2024-04-07T16:24:56.367825+0000 mon.a (mon.0) 764 : cluster [INF] osd.6 marked itself down and dead 2024-04-07T16:24:57.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:56 smithi100 bash[76202]: cluster 2024-04-07T16:24:56.849790+0000 mon.a (mon.0) 765 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN) 2024-04-07T16:24:57.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:56 smithi100 bash[76202]: cluster 2024-04-07T16:24:56.864222+0000 mon.a (mon.0) 766 : cluster [DBG] osdmap e543: 8 total, 7 up, 8 in 2024-04-07T16:24:57.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:56 smithi144 bash[66654]: cluster 2024-04-07T16:24:55.985677+0000 mon.a (mon.0) 763 : cluster [DBG] osdmap e542: 8 total, 8 up, 8 in 2024-04-07T16:24:57.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:56 smithi144 bash[66654]: cluster 2024-04-07T16:24:56.367825+0000 mon.a (mon.0) 764 : cluster [INF] osd.6 marked itself down and dead 2024-04-07T16:24:57.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:56 smithi144 bash[66654]: cluster 2024-04-07T16:24:56.849790+0000 mon.a (mon.0) 765 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN) 2024-04-07T16:24:57.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:56 smithi144 bash[66654]: cluster 2024-04-07T16:24:56.864222+0000 mon.a (mon.0) 766 : cluster [DBG] osdmap e543: 8 total, 7 up, 8 in 2024-04-07T16:24:57.851 INFO:tasks.rados.rados.0.smithi100.stdout:2507: finishing write tid 1 to smithi10066867-10 2024-04-07T16:24:57.851 INFO:tasks.rados.rados.0.smithi100.stdout:2508: finishing write tid 1 to smithi10066867-14 2024-04-07T16:24:57.851 INFO:tasks.rados.rados.0.smithi100.stdout:2508: finishing write tid 2 to smithi10066867-14 2024-04-07T16:24:57.851 INFO:tasks.rados.rados.0.smithi100.stdout:2507: finishing write tid 2 to smithi10066867-10 2024-04-07T16:24:57.852 INFO:tasks.rados.rados.0.smithi100.stdout:2508: finishing write tid 3 to smithi10066867-14 2024-04-07T16:24:57.852 INFO:tasks.rados.rados.0.smithi100.stdout:2508: oid 14 updating version 0 to 1027 2024-04-07T16:24:57.852 INFO:tasks.rados.rados.0.smithi100.stdout:2508: oid 14 version 1027 is already newer than 1026 2024-04-07T16:24:57.852 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 14 v 1027 (ObjNum 762 snap 240 seq_num 762) dirty exists 2024-04-07T16:24:57.852 INFO:tasks.rados.rados.0.smithi100.stdout:2508: left oid 14 (ObjNum 762 snap 240 seq_num 762) 2024-04-07T16:24:57.852 INFO:tasks.rados.rados.0.smithi100.stdout:2507: finishing write tid 3 to smithi10066867-10 2024-04-07T16:24:57.852 INFO:tasks.rados.rados.0.smithi100.stdout:2507: oid 10 updating version 0 to 1071 2024-04-07T16:24:57.852 INFO:tasks.rados.rados.0.smithi100.stdout:2507: oid 10 updating version 1071 to 1072 2024-04-07T16:24:57.852 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 10 v 1072 (ObjNum 761 snap 240 seq_num 761) dirty exists 2024-04-07T16:24:57.852 INFO:tasks.rados.rados.0.smithi100.stdout:2507: left oid 10 (ObjNum 761 snap 240 seq_num 761) 2024-04-07T16:24:57.852 INFO:tasks.rados.rados.0.smithi100.stdout:2507: done (2 left) 2024-04-07T16:24:57.852 INFO:tasks.rados.rados.0.smithi100.stdout:2508: done (1 left) 2024-04-07T16:24:57.852 INFO:tasks.rados.rados.0.smithi100.stdout:2509: done (0 left) 2024-04-07T16:24:57.852 INFO:tasks.rados.rados.0.smithi100.stdout:2510: setattr oid 27 current snap is 240 2024-04-07T16:24:57.853 INFO:tasks.rados.rados.0.smithi100.stdout:2511: rmattr oid 39 current snap is 240 2024-04-07T16:24:57.853 INFO:tasks.rados.rados.0.smithi100.stdout:2511: done (1 left) 2024-04-07T16:24:57.853 INFO:tasks.rados.rados.0.smithi100.stdout:2512: rollback oid 22 current snap is 240 2024-04-07T16:24:57.853 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 22 to 230 2024-04-07T16:24:57.853 INFO:tasks.rados.rados.0.smithi100.stdout:2513: snap_remove snap 216 2024-04-07T16:24:58.356 INFO:journalctl@ceph.osd.6.smithi144.stdout:Apr 07 16:24:57 smithi144 bash[79347]: ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4-osd-6 2024-04-07T16:24:58.356 INFO:journalctl@ceph.osd.6.smithi144.stdout:Apr 07 16:24:58 smithi144 bash[79413]: Error response from daemon: No such container: ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4-osd-6 2024-04-07T16:24:58.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:57 smithi144 bash[66654]: cluster 2024-04-07T16:24:56.716108+0000 mgr.y (mgr.25175) 237 : cluster [DBG] pgmap v405: 121 pgs: 121 active+clean; 333 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 511 B/s rd, 678 KiB/s wr, 3 op/s 2024-04-07T16:24:58.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:57 smithi144 bash[66654]: cluster 2024-04-07T16:24:57.867131+0000 mon.a (mon.0) 767 : cluster [DBG] osdmap e544: 8 total, 7 up, 8 in 2024-04-07T16:24:58.417 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:57 smithi100 bash[74942]: cluster 2024-04-07T16:24:56.716108+0000 mgr.y (mgr.25175) 237 : cluster [DBG] pgmap v405: 121 pgs: 121 active+clean; 333 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 511 B/s rd, 678 KiB/s wr, 3 op/s 2024-04-07T16:24:58.417 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:57 smithi100 bash[74942]: cluster 2024-04-07T16:24:57.867131+0000 mon.a (mon.0) 767 : cluster [DBG] osdmap e544: 8 total, 7 up, 8 in 2024-04-07T16:24:58.418 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:57 smithi100 bash[76202]: cluster 2024-04-07T16:24:56.716108+0000 mgr.y (mgr.25175) 237 : cluster [DBG] pgmap v405: 121 pgs: 121 active+clean; 333 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 511 B/s rd, 678 KiB/s wr, 3 op/s 2024-04-07T16:24:58.418 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:57 smithi100 bash[76202]: cluster 2024-04-07T16:24:57.867131+0000 mon.a (mon.0) 767 : cluster [DBG] osdmap e544: 8 total, 7 up, 8 in 2024-04-07T16:24:58.685 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:24:58 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:24:58] "GET /metrics HTTP/1.1" 200 36281 "" "Prometheus/2.43.0" 2024-04-07T16:24:58.856 INFO:tasks.rados.rados.0.smithi100.stdout:2512: finishing rollback tid 0 to smithi10066867-22 2024-04-07T16:24:58.856 INFO:tasks.rados.rados.0.smithi100.stdout:2512: finishing rollback tid 1 to smithi10066867-22 2024-04-07T16:24:58.857 INFO:tasks.rados.rados.0.smithi100.stdout:2512: finishing rollback tid 2 to smithi10066867-22 2024-04-07T16:24:58.857 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 22 v 1167 (ObjNum 744 snap 228 seq_num 744) dirty exists 2024-04-07T16:24:58.857 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 27 v 1111 (ObjNum 691 snap 211 seq_num 691) dirty exists 2024-04-07T16:24:58.857 INFO:tasks.rados.rados.0.smithi100.stdout:2510: done (2 left) 2024-04-07T16:24:58.857 INFO:tasks.rados.rados.0.smithi100.stdout:2512: done (1 left) 2024-04-07T16:24:58.857 INFO:tasks.rados.rados.0.smithi100.stdout:2513: done (0 left) 2024-04-07T16:24:58.857 INFO:tasks.rados.rados.0.smithi100.stdout:2514: delete oid 23 current snap is 240 2024-04-07T16:24:58.868 INFO:tasks.rados.rados.0.smithi100.stdout:2514: done (0 left) 2024-04-07T16:24:58.868 INFO:tasks.rados.rados.0.smithi100.stdout:2515: read oid 50 snap -1 2024-04-07T16:24:58.868 INFO:tasks.rados.rados.0.smithi100.stdout:2515: expect (ObjNum 657 snap 203 seq_num 657) 2024-04-07T16:24:58.869 INFO:tasks.rados.rados.0.smithi100.stdout:2516: rollback oid 21 current snap is 240 2024-04-07T16:24:58.869 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 21 to 236 2024-04-07T16:24:58.869 INFO:tasks.rados.rados.0.smithi100.stdout:2517: delete oid 35 current snap is 240 2024-04-07T16:24:58.878 INFO:tasks.rados.rados.0.smithi100.stdout:2516: finishing rollback tid 0 to smithi10066867-21 2024-04-07T16:24:58.878 INFO:tasks.rados.rados.0.smithi100.stdout:2516: finishing rollback tid 1 to smithi10066867-21 2024-04-07T16:24:58.879 INFO:tasks.rados.rados.0.smithi100.stdout:2516: finishing rollback tid 2 to smithi10066867-21 2024-04-07T16:24:58.879 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 21 v 830 (ObjNum 731 snap 225 seq_num 731) dirty exists 2024-04-07T16:24:58.884 INFO:tasks.rados.rados.0.smithi100.stdout:2516: done (2 left) 2024-04-07T16:24:58.884 INFO:tasks.rados.rados.0.smithi100.stdout:2517: done (1 left) 2024-04-07T16:24:58.884 INFO:tasks.rados.rados.0.smithi100.stdout:2518: delete oid 5 current snap is 240 2024-04-07T16:24:58.900 INFO:tasks.rados.rados.0.smithi100.stdout:2518: done (1 left) 2024-04-07T16:24:58.900 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 6 current snap is 240 2024-04-07T16:24:58.900 INFO:tasks.rados.rados.0.smithi100.stdout:2519: seq_num 766 ranges {4341760=434176,4775936=401408} 2024-04-07T16:24:58.947 INFO:tasks.rados.rados.0.smithi100.stdout:2519: writing smithi10066867-6 from 4341760 to 4775936 tid 1 2024-04-07T16:24:58.951 INFO:tasks.rados.rados.0.smithi100.stdout:2519: writing smithi10066867-6 from 4775936 to 5177344 tid 2 2024-04-07T16:24:58.951 INFO:tasks.rados.rados.0.smithi100.stdout:2520: delete oid 20 current snap is 240 2024-04-07T16:24:58.951 INFO:tasks.rados.rados.0.smithi100.stdout:2515: expect (ObjNum 657 snap 203 seq_num 657) 2024-04-07T16:24:59.282 INFO:tasks.rados.rados.0.smithi100.stdout:2519: finishing write tid 1 to smithi10066867-6 2024-04-07T16:24:59.283 INFO:tasks.rados.rados.0.smithi100.stdout:2519: finishing write tid 2 to smithi10066867-6 2024-04-07T16:24:59.283 INFO:tasks.rados.rados.0.smithi100.stdout:2519: finishing write tid 3 to smithi10066867-6 2024-04-07T16:24:59.283 INFO:tasks.rados.rados.0.smithi100.stdout:2519: finishing write tid 4 to smithi10066867-6 2024-04-07T16:24:59.283 INFO:tasks.rados.rados.0.smithi100.stdout:2519: oid 6 updating version 0 to 1141 2024-04-07T16:24:59.283 INFO:tasks.rados.rados.0.smithi100.stdout:2519: oid 6 updating version 1141 to 1142 2024-04-07T16:24:59.283 INFO:tasks.rados.rados.0.smithi100.stdout:2519: oid 6 version 1142 is already newer than 1140 2024-04-07T16:24:59.283 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 6 v 1142 (ObjNum 766 snap 240 seq_num 766) dirty exists 2024-04-07T16:24:59.283 INFO:tasks.rados.rados.0.smithi100.stdout:2519: left oid 6 (ObjNum 766 snap 240 seq_num 766) 2024-04-07T16:24:59.283 INFO:tasks.rados.rados.0.smithi100.stdout:2515: done (2 left) 2024-04-07T16:24:59.284 INFO:tasks.rados.rados.0.smithi100.stdout:2519: done (1 left) 2024-04-07T16:24:59.284 INFO:tasks.rados.rados.0.smithi100.stdout:2520: done (0 left) 2024-04-07T16:24:59.284 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 15 current snap is 240 2024-04-07T16:24:59.284 INFO:tasks.rados.rados.0.smithi100.stdout:2521: seq_num 768 ranges {4390912=466944,4857856=401408} 2024-04-07T16:24:59.332 INFO:tasks.rados.rados.0.smithi100.stdout:2521: writing smithi10066867-15 from 4390912 to 4857856 tid 1 2024-04-07T16:24:59.337 INFO:tasks.rados.rados.0.smithi100.stdout:2521: writing smithi10066867-15 from 4857856 to 5259264 tid 2 2024-04-07T16:24:59.337 INFO:tasks.rados.rados.0.smithi100.stdout:2522: snap_create 2024-04-07T16:24:59.344 INFO:tasks.rados.rados.0.smithi100.stdout:2521: finishing write tid 1 to smithi10066867-15 2024-04-07T16:24:59.344 INFO:tasks.rados.rados.0.smithi100.stdout:2521: finishing write tid 2 to smithi10066867-15 2024-04-07T16:24:59.344 INFO:tasks.rados.rados.0.smithi100.stdout:2521: finishing write tid 3 to smithi10066867-15 2024-04-07T16:24:59.347 INFO:tasks.rados.rados.0.smithi100.stdout:2521: finishing write tid 4 to smithi10066867-15 2024-04-07T16:24:59.348 INFO:tasks.rados.rados.0.smithi100.stdout:2521: oid 15 updating version 0 to 1405 2024-04-07T16:24:59.348 INFO:tasks.rados.rados.0.smithi100.stdout:2521: oid 15 updating version 1405 to 1406 2024-04-07T16:24:59.348 INFO:tasks.rados.rados.0.smithi100.stdout:2521: oid 15 updating version 1406 to 1407 2024-04-07T16:24:59.348 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 15 v 1407 (ObjNum 768 snap 240 seq_num 768) dirty exists 2024-04-07T16:24:59.348 INFO:tasks.rados.rados.0.smithi100.stdout:2521: left oid 15 (ObjNum 768 snap 240 seq_num 768) 2024-04-07T16:24:59.984 INFO:tasks.rados.rados.0.smithi100.stdout:2521: done (1 left) 2024-04-07T16:24:59.984 INFO:tasks.rados.rados.0.smithi100.stdout:2522: done (0 left) 2024-04-07T16:24:59.984 INFO:tasks.rados.rados.0.smithi100.stdout:2523: delete oid 47 current snap is 241 2024-04-07T16:24:59.993 INFO:tasks.rados.rados.0.smithi100.stdout:2523: done (0 left) 2024-04-07T16:24:59.993 INFO:tasks.rados.rados.0.smithi100.stdout:2524: copy_from oid 37 from oid 34 current snap is 241 2024-04-07T16:24:59.994 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 9 current snap is 241 2024-04-07T16:24:59.994 INFO:tasks.rados.rados.0.smithi100.stdout:2525: seq_num 770 ranges {0=499712,499712=401408} 2024-04-07T16:25:00.000 INFO:tasks.rados.rados.0.smithi100.stdout:2525: writing smithi10066867-9 from 0 to 499712 tid 1 2024-04-07T16:25:00.006 INFO:tasks.rados.rados.0.smithi100.stdout:2525: writing smithi10066867-9 from 499712 to 901120 tid 2 2024-04-07T16:25:00.006 INFO:tasks.rados.rados.0.smithi100.stdout:2526: rollback oid 27 current snap is 241 2024-04-07T16:25:00.007 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 27 to 237 2024-04-07T16:25:00.007 INFO:tasks.rados.rados.0.smithi100.stdout:2527: delete oid 28 current snap is 241 2024-04-07T16:25:00.007 INFO:tasks.rados.rados.0.smithi100.stdout:2525: finishing write tid 1 to smithi10066867-9 2024-04-07T16:25:00.016 INFO:tasks.rados.rados.0.smithi100.stdout:2525: finishing write tid 2 to smithi10066867-9 2024-04-07T16:25:00.016 INFO:tasks.rados.rados.0.smithi100.stdout:2525: finishing write tid 3 to smithi10066867-9 2024-04-07T16:25:00.017 INFO:tasks.rados.rados.0.smithi100.stdout:2524: finishing copy_from racing read to smithi10066867-37 2024-04-07T16:25:00.019 INFO:tasks.rados.rados.0.smithi100.stdout:2525: finishing write tid 4 to smithi10066867-9 2024-04-07T16:25:00.020 INFO:tasks.rados.rados.0.smithi100.stdout:2525: oid 9 updating version 0 to 1107 2024-04-07T16:25:00.020 INFO:tasks.rados.rados.0.smithi100.stdout:2525: oid 9 updating version 1107 to 1108 2024-04-07T16:25:00.020 INFO:tasks.rados.rados.0.smithi100.stdout:2525: oid 9 version 1108 is already newer than 1106 2024-04-07T16:25:00.020 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 9 v 1108 (ObjNum 770 snap 241 seq_num 770) dirty exists 2024-04-07T16:25:00.020 INFO:tasks.rados.rados.0.smithi100.stdout:2525: left oid 9 (ObjNum 770 snap 241 seq_num 770) 2024-04-07T16:25:00.021 INFO:tasks.rados.rados.0.smithi100.stdout:2524: finishing copy_from to smithi10066867-37 2024-04-07T16:25:00.021 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 37 v 1014 (ObjNum 759 snap 239 seq_num 759) dirty exists 2024-04-07T16:25:00.021 INFO:tasks.rados.rados.0.smithi100.stdout:2524: done (3 left) 2024-04-07T16:25:00.021 INFO:tasks.rados.rados.0.smithi100.stdout:2525: done (2 left) 2024-04-07T16:25:00.021 INFO:tasks.rados.rados.0.smithi100.stdout:2527: done (1 left) 2024-04-07T16:25:00.021 INFO:tasks.rados.rados.0.smithi100.stdout:2528: delete oid 23 current snap is 241 2024-04-07T16:25:00.021 INFO:tasks.rados.rados.0.smithi100.stdout:2526: finishing rollback tid 0 to smithi10066867-27 2024-04-07T16:25:00.021 INFO:tasks.rados.rados.0.smithi100.stdout:2526: finishing rollback tid 1 to smithi10066867-27 2024-04-07T16:25:00.022 INFO:tasks.rados.rados.0.smithi100.stdout:2526: finishing rollback tid 2 to smithi10066867-27 2024-04-07T16:25:00.022 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 27 v 1114 (ObjNum 691 snap 211 seq_num 691) dirty exists 2024-04-07T16:25:00.023 INFO:tasks.rados.rados.0.smithi100.stdout:2526: done (1 left) 2024-04-07T16:25:00.024 INFO:tasks.rados.rados.0.smithi100.stdout:2528: done (0 left) 2024-04-07T16:25:00.024 INFO:tasks.rados.rados.0.smithi100.stdout:2529: snap_create 2024-04-07T16:25:00.106 INFO:journalctl@ceph.osd.6.smithi144.stdout:Apr 07 16:24:59 smithi144 systemd[1]: ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@osd.6.service: Deactivated successfully. 2024-04-07T16:25:00.106 INFO:journalctl@ceph.osd.6.smithi144.stdout:Apr 07 16:24:59 smithi144 systemd[1]: Stopped Ceph osd.6 for 0702e3a8-f4f8-11ee-b649-cb9ed24678a4. 2024-04-07T16:25:00.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:59 smithi144 bash[66654]: cluster 2024-04-07T16:24:58.717171+0000 mgr.y (mgr.25175) 238 : cluster [DBG] pgmap v408: 121 pgs: 2 active+undersized+degraded, 25 peering, 3 stale+active+clean, 2 active+undersized, 89 active+clean; 335 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 4.0 MiB/s rd, 544 KiB/s wr, 4 op/s; 16/501 objects degraded (3.194%) 2024-04-07T16:25:00.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:59 smithi144 bash[66654]: cluster 2024-04-07T16:24:58.865456+0000 mon.a (mon.0) 768 : cluster [DBG] osdmap e545: 8 total, 7 up, 8 in 2024-04-07T16:25:00.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:59 smithi144 bash[66654]: cluster 2024-04-07T16:24:58.990957+0000 mon.a (mon.0) 769 : cluster [WRN] Health check failed: Reduced data availability: 2 pgs inactive, 3 pgs peering (PG_AVAILABILITY) 2024-04-07T16:25:00.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:24:59 smithi144 bash[66654]: cluster 2024-04-07T16:24:58.991032+0000 mon.a (mon.0) 770 : cluster [WRN] Health check failed: Degraded data redundancy: 16/501 objects degraded (3.194%), 2 pgs degraded (PG_DEGRADED) 2024-04-07T16:25:00.184 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:59 smithi100 bash[74942]: cluster 2024-04-07T16:24:58.717171+0000 mgr.y (mgr.25175) 238 : cluster [DBG] pgmap v408: 121 pgs: 2 active+undersized+degraded, 25 peering, 3 stale+active+clean, 2 active+undersized, 89 active+clean; 335 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 4.0 MiB/s rd, 544 KiB/s wr, 4 op/s; 16/501 objects degraded (3.194%) 2024-04-07T16:25:00.184 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:59 smithi100 bash[74942]: cluster 2024-04-07T16:24:58.865456+0000 mon.a (mon.0) 768 : cluster [DBG] osdmap e545: 8 total, 7 up, 8 in 2024-04-07T16:25:00.184 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:59 smithi100 bash[74942]: cluster 2024-04-07T16:24:58.990957+0000 mon.a (mon.0) 769 : cluster [WRN] Health check failed: Reduced data availability: 2 pgs inactive, 3 pgs peering (PG_AVAILABILITY) 2024-04-07T16:25:00.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:24:59 smithi100 bash[74942]: cluster 2024-04-07T16:24:58.991032+0000 mon.a (mon.0) 770 : cluster [WRN] Health check failed: Degraded data redundancy: 16/501 objects degraded (3.194%), 2 pgs degraded (PG_DEGRADED) 2024-04-07T16:25:00.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:59 smithi100 bash[76202]: cluster 2024-04-07T16:24:58.717171+0000 mgr.y (mgr.25175) 238 : cluster [DBG] pgmap v408: 121 pgs: 2 active+undersized+degraded, 25 peering, 3 stale+active+clean, 2 active+undersized, 89 active+clean; 335 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 4.0 MiB/s rd, 544 KiB/s wr, 4 op/s; 16/501 objects degraded (3.194%) 2024-04-07T16:25:00.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:59 smithi100 bash[76202]: cluster 2024-04-07T16:24:58.865456+0000 mon.a (mon.0) 768 : cluster [DBG] osdmap e545: 8 total, 7 up, 8 in 2024-04-07T16:25:00.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:59 smithi100 bash[76202]: cluster 2024-04-07T16:24:58.990957+0000 mon.a (mon.0) 769 : cluster [WRN] Health check failed: Reduced data availability: 2 pgs inactive, 3 pgs peering (PG_AVAILABILITY) 2024-04-07T16:25:00.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:24:59 smithi100 bash[76202]: cluster 2024-04-07T16:24:58.991032+0000 mon.a (mon.0) 770 : cluster [WRN] Health check failed: Degraded data redundancy: 16/501 objects degraded (3.194%), 2 pgs degraded (PG_DEGRADED) 2024-04-07T16:25:00.606 INFO:journalctl@ceph.osd.7.smithi144.stdout:Apr 07 16:25:00 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:25:00.606 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:25:00 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:25:00.606 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:25:00 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:25:00.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:00 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:25:00.607 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:25:00 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:25:00.607 INFO:journalctl@ceph.osd.4.smithi144.stdout:Apr 07 16:25:00 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:25:00.608 INFO:journalctl@ceph.mds.b.smithi144.stdout:Apr 07 16:25:00 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:25:00.608 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 16:25:00 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:25:00.608 INFO:journalctl@ceph.osd.5.smithi144.stdout:Apr 07 16:25:00 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:25:00.608 INFO:journalctl@ceph.osd.6.smithi144.stdout:Apr 07 16:25:00 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:25:00.609 INFO:journalctl@ceph.osd.6.smithi144.stdout:Apr 07 16:25:00 smithi144 systemd[1]: Started Ceph osd.6 for 0702e3a8-f4f8-11ee-b649-cb9ed24678a4. 2024-04-07T16:25:00.988 INFO:tasks.rados.rados.0.smithi100.stdout:2529: done (0 left) 2024-04-07T16:25:00.988 INFO:tasks.rados.rados.0.smithi100.stdout:2530: snap_remove snap 230 2024-04-07T16:25:01.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:01 smithi144 bash[66654]: cluster 2024-04-07T16:25:00.002061+0000 mon.a (mon.0) 771 : cluster [DBG] osdmap e546: 8 total, 7 up, 8 in 2024-04-07T16:25:01.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:01 smithi144 bash[66654]: audit 2024-04-07T16:25:00.303534+0000 mon.a (mon.0) 772 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:25:01.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:01 smithi144 bash[66654]: audit 2024-04-07T16:25:00.315352+0000 mon.a (mon.0) 773 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:25:01.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:01 smithi144 bash[66654]: audit 2024-04-07T16:25:00.365365+0000 mon.a (mon.0) 774 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:25:01.434 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:01 smithi100 bash[74942]: cluster 2024-04-07T16:25:00.002061+0000 mon.a (mon.0) 771 : cluster [DBG] osdmap e546: 8 total, 7 up, 8 in 2024-04-07T16:25:01.434 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:01 smithi100 bash[74942]: audit 2024-04-07T16:25:00.303534+0000 mon.a (mon.0) 772 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:25:01.434 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:01 smithi100 bash[74942]: audit 2024-04-07T16:25:00.315352+0000 mon.a (mon.0) 773 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:25:01.435 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:01 smithi100 bash[74942]: audit 2024-04-07T16:25:00.365365+0000 mon.a (mon.0) 774 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:25:01.435 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:01 smithi100 bash[76202]: cluster 2024-04-07T16:25:00.002061+0000 mon.a (mon.0) 771 : cluster [DBG] osdmap e546: 8 total, 7 up, 8 in 2024-04-07T16:25:01.435 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:01 smithi100 bash[76202]: audit 2024-04-07T16:25:00.303534+0000 mon.a (mon.0) 772 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:25:01.435 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:01 smithi100 bash[76202]: audit 2024-04-07T16:25:00.315352+0000 mon.a (mon.0) 773 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:25:01.435 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:01 smithi100 bash[76202]: audit 2024-04-07T16:25:00.365365+0000 mon.a (mon.0) 774 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:25:01.647 INFO:journalctl@ceph.osd.6.smithi144.stdout:Apr 07 16:25:01 smithi144 bash[79567]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-04-07T16:25:01.647 INFO:journalctl@ceph.osd.6.smithi144.stdout:Apr 07 16:25:01 smithi144 bash[79567]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-6 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-04-07T16:25:01.648 INFO:journalctl@ceph.osd.6.smithi144.stdout:Apr 07 16:25:01 smithi144 bash[79567]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-04-07T16:25:01.648 INFO:journalctl@ceph.osd.6.smithi144.stdout:Apr 07 16:25:01 smithi144 bash[79567]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-04-07T16:25:01.648 INFO:journalctl@ceph.osd.6.smithi144.stdout:Apr 07 16:25:01 smithi144 bash[79567]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-6/block 2024-04-07T16:25:01.648 INFO:journalctl@ceph.osd.6.smithi144.stdout:Apr 07 16:25:01 smithi144 bash[79567]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-04-07T16:25:01.648 INFO:journalctl@ceph.osd.6.smithi144.stdout:Apr 07 16:25:01 smithi144 bash[79567]: --> ceph-volume raw activate successful for osd ID: 6 2024-04-07T16:25:01.845 INFO:tasks.rados.rados.0.smithi100.stdout:2530: done (0 left) 2024-04-07T16:25:01.845 INFO:tasks.rados.rados.0.smithi100.stdout:2531: snap_create 2024-04-07T16:25:02.304 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:02 smithi144 bash[66654]: cluster 2024-04-07T16:25:00.718189+0000 mgr.y (mgr.25175) 239 : cluster [DBG] pgmap v411: 121 pgs: 2 active+clean+snaptrim, 3 active+clean+snaptrim_wait, 3 active+undersized+degraded, 25 peering, 15 active+undersized, 73 active+clean; 339 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 10 MiB/s rd, 888 KiB/s wr, 15 op/s; 17/513 objects degraded (3.314%) 2024-04-07T16:25:02.304 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:02 smithi144 bash[66654]: cluster 2024-04-07T16:25:01.006047+0000 mon.a (mon.0) 775 : cluster [DBG] osdmap e547: 8 total, 7 up, 8 in 2024-04-07T16:25:02.305 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:02 smithi144 bash[66654]: audit 2024-04-07T16:25:01.722723+0000 mon.a (mon.0) 776 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:25:02.305 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:02 smithi144 bash[66654]: audit 2024-04-07T16:25:01.724985+0000 mon.a (mon.0) 777 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:25:02.305 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:02 smithi144 bash[66654]: cluster 2024-04-07T16:25:01.862198+0000 mon.a (mon.0) 778 : cluster [DBG] osdmap e548: 8 total, 7 up, 8 in 2024-04-07T16:25:02.434 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:02 smithi100 bash[76202]: cluster 2024-04-07T16:25:00.718189+0000 mgr.y (mgr.25175) 239 : cluster [DBG] pgmap v411: 121 pgs: 2 active+clean+snaptrim, 3 active+clean+snaptrim_wait, 3 active+undersized+degraded, 25 peering, 15 active+undersized, 73 active+clean; 339 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 10 MiB/s rd, 888 KiB/s wr, 15 op/s; 17/513 objects degraded (3.314%) 2024-04-07T16:25:02.435 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:02 smithi100 bash[76202]: cluster 2024-04-07T16:25:01.006047+0000 mon.a (mon.0) 775 : cluster [DBG] osdmap e547: 8 total, 7 up, 8 in 2024-04-07T16:25:02.435 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:02 smithi100 bash[76202]: audit 2024-04-07T16:25:01.722723+0000 mon.a (mon.0) 776 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:25:02.435 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:02 smithi100 bash[76202]: audit 2024-04-07T16:25:01.724985+0000 mon.a (mon.0) 777 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:25:02.435 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:02 smithi100 bash[76202]: cluster 2024-04-07T16:25:01.862198+0000 mon.a (mon.0) 778 : cluster [DBG] osdmap e548: 8 total, 7 up, 8 in 2024-04-07T16:25:02.435 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:02 smithi100 bash[74942]: cluster 2024-04-07T16:25:00.718189+0000 mgr.y (mgr.25175) 239 : cluster [DBG] pgmap v411: 121 pgs: 2 active+clean+snaptrim, 3 active+clean+snaptrim_wait, 3 active+undersized+degraded, 25 peering, 15 active+undersized, 73 active+clean; 339 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 10 MiB/s rd, 888 KiB/s wr, 15 op/s; 17/513 objects degraded (3.314%) 2024-04-07T16:25:02.435 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:02 smithi100 bash[74942]: cluster 2024-04-07T16:25:01.006047+0000 mon.a (mon.0) 775 : cluster [DBG] osdmap e547: 8 total, 7 up, 8 in 2024-04-07T16:25:02.435 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:02 smithi100 bash[74942]: audit 2024-04-07T16:25:01.722723+0000 mon.a (mon.0) 776 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:25:02.435 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:02 smithi100 bash[74942]: audit 2024-04-07T16:25:01.724985+0000 mon.a (mon.0) 777 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:25:02.435 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:02 smithi100 bash[74942]: cluster 2024-04-07T16:25:01.862198+0000 mon.a (mon.0) 778 : cluster [DBG] osdmap e548: 8 total, 7 up, 8 in 2024-04-07T16:25:02.606 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:25:02 smithi144 bash[59794]: ts=2024-04-07T16:25:02.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:25:02.846 INFO:tasks.rados.rados.0.smithi100.stdout:2531: done (0 left) 2024-04-07T16:25:02.846 INFO:tasks.rados.rados.0.smithi100.stdout:2532: snap_remove snap 223 2024-04-07T16:25:03.356 INFO:journalctl@ceph.osd.6.smithi144.stdout:Apr 07 16:25:03 smithi144 bash[79803]: debug 2024-04-07T16:25:03.145+0000 7f8955b34700 1 -- 172.21.15.144:0/2788225532 <== mon.1 v2:172.21.15.144:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563dc4b09680 con 0x563dc4b46c00 2024-04-07T16:25:03.863 INFO:tasks.rados.rados.0.smithi100.stdout:2532: done (0 left) 2024-04-07T16:25:03.863 INFO:tasks.rados.rados.0.smithi100.stdout:2533: setattr oid 43 current snap is 243 2024-04-07T16:25:03.869 INFO:tasks.rados.rados.0.smithi100.stdout:2534: setattr oid 29 current snap is 243 2024-04-07T16:25:03.870 INFO:tasks.rados.rados.0.smithi100.stdout:2535: rmattr oid 12 current snap is 243 2024-04-07T16:25:03.870 INFO:tasks.rados.rados.0.smithi100.stdout:2535: done (2 left) 2024-04-07T16:25:03.870 INFO:tasks.rados.rados.0.smithi100.stdout:2536: read oid 3 snap -1 2024-04-07T16:25:03.870 INFO:tasks.rados.rados.0.smithi100.stdout:2536: expect (ObjNum 733 snap 226 seq_num 733) 2024-04-07T16:25:03.870 INFO:tasks.rados.rados.0.smithi100.stdout:2537: copy_from oid 15 from oid 24 current snap is 243 2024-04-07T16:25:03.871 INFO:tasks.rados.rados.0.smithi100.stdout:2538: read oid 32 snap 242 2024-04-07T16:25:03.871 INFO:tasks.rados.rados.0.smithi100.stdout:2538: expect (ObjNum 723 snap 224 seq_num 723) 2024-04-07T16:25:03.871 INFO:tasks.rados.rados.0.smithi100.stdout:2539: copy_from oid 4 from oid 20 current snap is 243 2024-04-07T16:25:03.872 INFO:tasks.rados.rados.0.smithi100.stdout:2540: snap_remove snap 222 2024-04-07T16:25:04.106 INFO:journalctl@ceph.osd.6.smithi144.stdout:Apr 07 16:25:03 smithi144 bash[79803]: debug 2024-04-07T16:25:03.689+0000 7f895eccc7c0 -1 Falling back to public interface 2024-04-07T16:25:04.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:03 smithi144 bash[66654]: cluster 2024-04-07T16:25:02.719405+0000 mgr.y (mgr.25175) 240 : cluster [DBG] pgmap v414: 121 pgs: 2 active+clean+snaptrim, 3 active+clean+snaptrim_wait, 8 active+undersized+degraded, 35 active+undersized, 73 active+clean; 339 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 9.7 MiB/s rd, 344 KiB/s wr, 13 op/s; 31/513 objects degraded (6.043%) 2024-04-07T16:25:04.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:03 smithi144 bash[66654]: cluster 2024-04-07T16:25:02.863314+0000 mon.a (mon.0) 779 : cluster [DBG] osdmap e549: 8 total, 7 up, 8 in 2024-04-07T16:25:04.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:03 smithi144 bash[66654]: cluster 2024-04-07T16:25:03.012041+0000 mon.a (mon.0) 780 : cluster [INF] Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs inactive, 3 pgs peering) 2024-04-07T16:25:04.184 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:03 smithi100 bash[74942]: cluster 2024-04-07T16:25:02.719405+0000 mgr.y (mgr.25175) 240 : cluster [DBG] pgmap v414: 121 pgs: 2 active+clean+snaptrim, 3 active+clean+snaptrim_wait, 8 active+undersized+degraded, 35 active+undersized, 73 active+clean; 339 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 9.7 MiB/s rd, 344 KiB/s wr, 13 op/s; 31/513 objects degraded (6.043%) 2024-04-07T16:25:04.184 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:03 smithi100 bash[74942]: cluster 2024-04-07T16:25:02.863314+0000 mon.a (mon.0) 779 : cluster [DBG] osdmap e549: 8 total, 7 up, 8 in 2024-04-07T16:25:04.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:03 smithi100 bash[74942]: cluster 2024-04-07T16:25:03.012041+0000 mon.a (mon.0) 780 : cluster [INF] Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs inactive, 3 pgs peering) 2024-04-07T16:25:04.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:03 smithi100 bash[76202]: cluster 2024-04-07T16:25:02.719405+0000 mgr.y (mgr.25175) 240 : cluster [DBG] pgmap v414: 121 pgs: 2 active+clean+snaptrim, 3 active+clean+snaptrim_wait, 8 active+undersized+degraded, 35 active+undersized, 73 active+clean; 339 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 9.7 MiB/s rd, 344 KiB/s wr, 13 op/s; 31/513 objects degraded (6.043%) 2024-04-07T16:25:04.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:03 smithi100 bash[76202]: cluster 2024-04-07T16:25:02.863314+0000 mon.a (mon.0) 779 : cluster [DBG] osdmap e549: 8 total, 7 up, 8 in 2024-04-07T16:25:04.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:03 smithi100 bash[76202]: cluster 2024-04-07T16:25:03.012041+0000 mon.a (mon.0) 780 : cluster [INF] Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs inactive, 3 pgs peering) 2024-04-07T16:25:04.871 INFO:tasks.rados.rados.0.smithi100.stdout:2540: done (6 left) 2024-04-07T16:25:04.871 INFO:tasks.rados.rados.0.smithi100.stdout:2541: rmattr oid 5 current snap is 243 2024-04-07T16:25:04.880 INFO:tasks.rados.rados.0.smithi100.stdout:2536: expect (ObjNum 733 snap 226 seq_num 733) 2024-04-07T16:25:05.013 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 29 v 1129 (ObjNum 609 snap 0 seq_num 2702434352) dirty exists 2024-04-07T16:25:05.013 INFO:tasks.rados.rados.0.smithi100.stdout:2537: finishing copy_from racing read to smithi10066867-15 2024-04-07T16:25:05.013 INFO:tasks.rados.rados.0.smithi100.stdout:2539: finishing copy_from to smithi10066867-4 2024-04-07T16:25:05.013 INFO:tasks.rados.rados.0.smithi100.stdout:2539: got expected ENOENT (src dne) 2024-04-07T16:25:05.013 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 43 v 1115 (ObjNum 691 snap 211 seq_num 691) dirty exists 2024-04-07T16:25:05.013 INFO:tasks.rados.rados.0.smithi100.stdout:2539: finishing copy_from racing read to smithi10066867-4 2024-04-07T16:25:05.014 INFO:tasks.rados.rados.0.smithi100.stdout:2537: finishing copy_from to smithi10066867-15 2024-04-07T16:25:05.014 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 15 v 1120 (ObjNum 732 snap 225 seq_num 732) dirty exists 2024-04-07T16:25:05.014 INFO:tasks.rados.rados.0.smithi100.stdout:2538: expect (ObjNum 723 snap 224 seq_num 723) 2024-04-07T16:25:05.082 INFO:tasks.rados.rados.0.smithi100.stdout:2533: done (6 left) 2024-04-07T16:25:05.082 INFO:tasks.rados.rados.0.smithi100.stdout:2534: done (5 left) 2024-04-07T16:25:05.082 INFO:tasks.rados.rados.0.smithi100.stdout:2536: done (4 left) 2024-04-07T16:25:05.083 INFO:tasks.rados.rados.0.smithi100.stdout:2537: done (3 left) 2024-04-07T16:25:05.083 INFO:tasks.rados.rados.0.smithi100.stdout:2538: done (2 left) 2024-04-07T16:25:05.083 INFO:tasks.rados.rados.0.smithi100.stdout:2539: done (1 left) 2024-04-07T16:25:05.083 INFO:tasks.rados.rados.0.smithi100.stdout:2541: done (0 left) 2024-04-07T16:25:05.083 INFO:tasks.rados.rados.0.smithi100.stdout:2542: copy_from oid 10 from oid 38 current snap is 243 2024-04-07T16:25:05.083 INFO:tasks.rados.rados.0.smithi100.stdout:2543: read oid 37 snap -1 2024-04-07T16:25:05.083 INFO:tasks.rados.rados.0.smithi100.stdout:2543: expect (ObjNum 759 snap 239 seq_num 759) 2024-04-07T16:25:05.083 INFO:tasks.rados.rados.0.smithi100.stdout:2544: read oid 19 snap -1 2024-04-07T16:25:05.083 INFO:tasks.rados.rados.0.smithi100.stdout:2544: expect deleted 2024-04-07T16:25:05.083 INFO:tasks.rados.rados.0.smithi100.stdout:2545: setattr oid 30 current snap is 243 2024-04-07T16:25:05.085 INFO:tasks.rados.rados.0.smithi100.stdout:2544: done (3 left) 2024-04-07T16:25:05.085 INFO:tasks.rados.rados.0.smithi100.stdout:2546: read oid 43 snap 234 2024-04-07T16:25:05.085 INFO:tasks.rados.rados.0.smithi100.stdout:2546: expect (ObjNum 691 snap 211 seq_num 691) 2024-04-07T16:25:05.085 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 26 current snap is 243 2024-04-07T16:25:05.085 INFO:tasks.rados.rados.0.smithi100.stdout:2547: seq_num 773 ranges {0=548864,548864=401408} 2024-04-07T16:25:05.093 INFO:tasks.rados.rados.0.smithi100.stdout:2547: writing smithi10066867-26 from 0 to 548864 tid 1 2024-04-07T16:25:05.103 INFO:tasks.rados.rados.0.smithi100.stdout:2547: writing smithi10066867-26 from 548864 to 950272 tid 2 2024-04-07T16:25:05.103 INFO:tasks.rados.rados.0.smithi100.stdout:2548: rollback oid 18 current snap is 243 2024-04-07T16:25:05.103 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 18 to 186 2024-04-07T16:25:05.103 INFO:tasks.rados.rados.0.smithi100.stdout:2543: expect (ObjNum 759 snap 239 seq_num 759) 2024-04-07T16:25:05.129 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 30 v 1130 (ObjNum 33 snap 0 seq_num 2703120272) dirty exists 2024-04-07T16:25:05.129 INFO:tasks.rados.rados.0.smithi100.stdout:2547: finishing write tid 1 to smithi10066867-26 2024-04-07T16:25:05.129 INFO:tasks.rados.rados.0.smithi100.stdout:2543: done (5 left) 2024-04-07T16:25:05.129 INFO:tasks.rados.rados.0.smithi100.stdout:2545: done (4 left) 2024-04-07T16:25:05.130 INFO:tasks.rados.rados.0.smithi100.stdout:2549: rmattr oid 9 current snap is 243 2024-04-07T16:25:05.130 INFO:tasks.rados.rados.0.smithi100.stdout:2549: done (4 left) 2024-04-07T16:25:05.130 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 6 current snap is 243 2024-04-07T16:25:05.130 INFO:tasks.rados.rados.0.smithi100.stdout:2542: finishing copy_from racing read to smithi10066867-10 2024-04-07T16:25:05.130 INFO:tasks.rados.rados.0.smithi100.stdout:2547: finishing write tid 2 to smithi10066867-26 2024-04-07T16:25:05.130 INFO:tasks.rados.rados.0.smithi100.stdout:2548: finishing rollback tid 1 to smithi10066867-18 2024-04-07T16:25:05.130 INFO:tasks.rados.rados.0.smithi100.stdout:2550: seq_num 774 ranges {5177344=565248,5742592=401408} 2024-04-07T16:25:05.165 INFO:tasks.rados.rados.0.smithi100.stdout:2550: writing smithi10066867-6 from 5177344 to 5742592 tid 1 2024-04-07T16:25:05.168 INFO:tasks.rados.rados.0.smithi100.stdout:2550: writing smithi10066867-6 from 5742592 to 6144000 tid 2 2024-04-07T16:25:05.168 INFO:tasks.rados.rados.0.smithi100.stdout:2551: read oid 31 snap -1 2024-04-07T16:25:05.168 INFO:tasks.rados.rados.0.smithi100.stdout:2551: expect (ObjNum 579 snap 180 seq_num 579) 2024-04-07T16:25:05.168 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 42 current snap is 243 2024-04-07T16:25:05.168 INFO:tasks.rados.rados.0.smithi100.stdout:2548: finishing rollback tid 2 to smithi10066867-18 2024-04-07T16:25:05.168 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 18 v 1430 (ObjNum 509 snap 152 seq_num 509) dirty exists 2024-04-07T16:25:05.168 INFO:tasks.rados.rados.0.smithi100.stdout:2552: seq_num 775 ranges {0=589824,589824=401408} 2024-04-07T16:25:05.172 INFO:tasks.rados.rados.0.smithi100.stdout:2552: writing smithi10066867-42 from 0 to 589824 tid 1 2024-04-07T16:25:05.175 INFO:tasks.rados.rados.0.smithi100.stdout:2552: writing smithi10066867-42 from 589824 to 991232 tid 2 2024-04-07T16:25:05.175 INFO:tasks.rados.rados.0.smithi100.stdout:2548: done (6 left) 2024-04-07T16:25:05.175 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 15 current snap is 243 2024-04-07T16:25:05.175 INFO:tasks.rados.rados.0.smithi100.stdout:2553: seq_num 776 ranges {2326528=606208,2932736=401408} 2024-04-07T16:25:05.184 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:04 smithi100 bash[74942]: cluster 2024-04-07T16:25:03.876248+0000 mon.a (mon.0) 781 : cluster [DBG] osdmap e550: 8 total, 7 up, 8 in 2024-04-07T16:25:05.184 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:04 smithi100 bash[76202]: cluster 2024-04-07T16:25:03.876248+0000 mon.a (mon.0) 781 : cluster [DBG] osdmap e550: 8 total, 7 up, 8 in 2024-04-07T16:25:05.192 INFO:tasks.rados.rados.0.smithi100.stdout:2553: writing smithi10066867-15 from 2326528 to 2932736 tid 1 2024-04-07T16:25:05.195 INFO:tasks.rados.rados.0.smithi100.stdout:2553: writing smithi10066867-15 from 2932736 to 3334144 tid 2 2024-04-07T16:25:05.195 INFO:tasks.rados.rados.0.smithi100.stdout:2554: copy_from oid 34 from oid 40 current snap is 243 2024-04-07T16:25:05.195 INFO:tasks.rados.rados.0.smithi100.stdout:2542: finishing copy_from to smithi10066867-10 2024-04-07T16:25:05.195 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 10 v 1103 (ObjNum 720 snap 221 seq_num 720) dirty exists 2024-04-07T16:25:05.195 INFO:tasks.rados.rados.0.smithi100.stdout:2547: finishing write tid 3 to smithi10066867-26 2024-04-07T16:25:05.195 INFO:tasks.rados.rados.0.smithi100.stdout:2547: finishing write tid 4 to smithi10066867-26 2024-04-07T16:25:05.195 INFO:tasks.rados.rados.0.smithi100.stdout:2547: oid 26 updating version 0 to 1171 2024-04-07T16:25:05.195 INFO:tasks.rados.rados.0.smithi100.stdout:2547: oid 26 version 1171 is already newer than 1170 2024-04-07T16:25:05.196 INFO:tasks.rados.rados.0.smithi100.stdout:2547: oid 26 version 1171 is already newer than 1169 2024-04-07T16:25:05.196 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 26 v 1171 (ObjNum 773 snap 243 seq_num 773) dirty exists 2024-04-07T16:25:05.196 INFO:tasks.rados.rados.0.smithi100.stdout:2547: left oid 26 (ObjNum 773 snap 243 seq_num 773) 2024-04-07T16:25:05.196 INFO:tasks.rados.rados.0.smithi100.stdout:2546: expect (ObjNum 691 snap 211 seq_num 691) 2024-04-07T16:25:05.335 INFO:teuthology.orchestra.run.smithi100.stdout:true 2024-04-07T16:25:05.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:04 smithi144 bash[66654]: cluster 2024-04-07T16:25:03.876248+0000 mon.a (mon.0) 781 : cluster [DBG] osdmap e550: 8 total, 7 up, 8 in 2024-04-07T16:25:05.719 INFO:teuthology.orchestra.run.smithi100.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-07T16:25:05.719 INFO:teuthology.orchestra.run.smithi100.stdout:alertmanager.a smithi100 *:9093,9094 running (11m) 83s ago 11m 14.4M - 0.25.0 c8568f914cd2 5143075e7d26 2024-04-07T16:25:05.719 INFO:teuthology.orchestra.run.smithi100.stdout:grafana.a smithi144 *:3000 running (11m) 26s ago 11m 79.7M - 9.4.7 954c08fa6188 7180c0708acd 2024-04-07T16:25:05.719 INFO:teuthology.orchestra.run.smithi100.stdout:mds.a smithi100 running (13m) 83s ago 13m 15.7M - 18.2.2-1171-g267c8ff5 8a4a01e7186c 92593b0c28ae 2024-04-07T16:25:05.719 INFO:teuthology.orchestra.run.smithi100.stdout:mds.b smithi144 running (12m) 26s ago 12m 20.1M - 18.2.2-1171-g267c8ff5 8a4a01e7186c 3df2ee3e17a4 2024-04-07T16:25:05.719 INFO:teuthology.orchestra.run.smithi100.stdout:mgr.x smithi144 *:8443,9283,8765 running (5m) 26s ago 20m 440M - 19.0.0-2785-gadce92d0 c0ecd4226ac9 233745c292a6 2024-04-07T16:25:05.719 INFO:teuthology.orchestra.run.smithi100.stdout:mgr.y smithi100 *:8443,9283,8765 running (6m) 83s ago 22m 503M - 19.0.0-2785-gadce92d0 c0ecd4226ac9 254dc65cdaa7 2024-04-07T16:25:05.719 INFO:teuthology.orchestra.run.smithi100.stdout:mon.a smithi100 running (4m) 83s ago 22m 54.3M 2048M 19.0.0-2785-gadce92d0 c0ecd4226ac9 718e12655c04 2024-04-07T16:25:05.720 INFO:teuthology.orchestra.run.smithi100.stdout:mon.b smithi144 running (3m) 26s ago 20m 43.5M 2048M 19.0.0-2785-gadce92d0 c0ecd4226ac9 aef817664bff 2024-04-07T16:25:05.720 INFO:teuthology.orchestra.run.smithi100.stdout:mon.c smithi100 running (4m) 83s ago 20m 46.2M 2048M 19.0.0-2785-gadce92d0 c0ecd4226ac9 63e889040f6e 2024-04-07T16:25:05.720 INFO:teuthology.orchestra.run.smithi100.stdout:node-exporter.a smithi100 *:9100 running (12m) 83s ago 12m 8955k - 1.5.0 0da6a335fe13 4bce264c1b26 2024-04-07T16:25:05.720 INFO:teuthology.orchestra.run.smithi100.stdout:node-exporter.b smithi144 *:9100 running (12m) 26s ago 12m 8676k - 1.5.0 0da6a335fe13 b6963413c560 2024-04-07T16:25:05.720 INFO:teuthology.orchestra.run.smithi100.stdout:osd.0 smithi100 running (2m) 83s ago 19m 199M 2524M 19.0.0-2785-gadce92d0 c0ecd4226ac9 a6e83c9d1abc 2024-04-07T16:25:05.720 INFO:teuthology.orchestra.run.smithi100.stdout:osd.1 smithi100 running (2m) 83s ago 18m 197M 2524M 19.0.0-2785-gadce92d0 c0ecd4226ac9 c50fe40079b6 2024-04-07T16:25:05.720 INFO:teuthology.orchestra.run.smithi100.stdout:osd.2 smithi100 running (116s) 83s ago 17m 253M 2524M 19.0.0-2785-gadce92d0 c0ecd4226ac9 8e631c43064b 2024-04-07T16:25:05.720 INFO:teuthology.orchestra.run.smithi100.stdout:osd.3 smithi100 running (88s) 83s ago 17m 11.8M 2524M 19.0.0-2785-gadce92d0 c0ecd4226ac9 402931247c79 2024-04-07T16:25:05.720 INFO:teuthology.orchestra.run.smithi100.stdout:osd.4 smithi144 running (59s) 26s ago 16m 243M 2524M 19.0.0-2785-gadce92d0 c0ecd4226ac9 ebbb7be95d33 2024-04-07T16:25:05.720 INFO:teuthology.orchestra.run.smithi100.stdout:osd.5 smithi144 running (32s) 26s ago 15m 11.7M 2524M 19.0.0-2785-gadce92d0 c0ecd4226ac9 ecce27bcb2f8 2024-04-07T16:25:05.720 INFO:teuthology.orchestra.run.smithi100.stdout:osd.6 smithi144 starting - - - 2524M 2024-04-07T16:25:05.720 INFO:teuthology.orchestra.run.smithi100.stdout:osd.7 smithi144 running (13m) 26s ago 13m 213M 2524M 18.2.2-1171-g267c8ff5 8a4a01e7186c 45f7709e576a 2024-04-07T16:25:05.720 INFO:teuthology.orchestra.run.smithi100.stdout:prometheus.a smithi144 *:9095 running (5m) 26s ago 12m 52.7M - 2.43.0 a07b618ecd1d 2d0a8131f303 2024-04-07T16:25:05.730 INFO:tasks.rados.rados.0.smithi100.stdout:2550: finishing write tid 1 to smithi10066867-6 2024-04-07T16:25:05.730 INFO:tasks.rados.rados.0.smithi100.stdout:2550: finishing write tid 2 to smithi10066867-6 2024-04-07T16:25:05.730 INFO:tasks.rados.rados.0.smithi100.stdout:2550: finishing write tid 3 to smithi10066867-6 2024-04-07T16:25:05.730 INFO:tasks.rados.rados.0.smithi100.stdout:2552: finishing write tid 1 to smithi10066867-42 2024-04-07T16:25:05.730 INFO:tasks.rados.rados.0.smithi100.stdout:2550: finishing write tid 4 to smithi10066867-6 2024-04-07T16:25:05.730 INFO:tasks.rados.rados.0.smithi100.stdout:2550: oid 6 updating version 0 to 1144 2024-04-07T16:25:05.730 INFO:tasks.rados.rados.0.smithi100.stdout:2550: oid 6 updating version 1144 to 1145 2024-04-07T16:25:05.730 INFO:tasks.rados.rados.0.smithi100.stdout:2550: oid 6 version 1145 is already newer than 1143 2024-04-07T16:25:05.730 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 6 v 1145 (ObjNum 774 snap 243 seq_num 774) dirty exists 2024-04-07T16:25:05.731 INFO:tasks.rados.rados.0.smithi100.stdout:2550: left oid 6 (ObjNum 774 snap 243 seq_num 774) 2024-04-07T16:25:05.731 INFO:tasks.rados.rados.0.smithi100.stdout:2552: finishing write tid 2 to smithi10066867-42 2024-04-07T16:25:05.731 INFO:tasks.rados.rados.0.smithi100.stdout:2552: finishing write tid 3 to smithi10066867-42 2024-04-07T16:25:05.731 INFO:tasks.rados.rados.0.smithi100.stdout:2552: finishing write tid 4 to smithi10066867-42 2024-04-07T16:25:05.731 INFO:tasks.rados.rados.0.smithi100.stdout:2552: oid 42 updating version 0 to 1131 2024-04-07T16:25:05.731 INFO:tasks.rados.rados.0.smithi100.stdout:2552: oid 42 updating version 1131 to 1132 2024-04-07T16:25:05.731 INFO:tasks.rados.rados.0.smithi100.stdout:2552: oid 42 updating version 1132 to 1133 2024-04-07T16:25:05.731 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 42 v 1133 (ObjNum 775 snap 243 seq_num 775) dirty exists 2024-04-07T16:25:05.731 INFO:tasks.rados.rados.0.smithi100.stdout:2552: left oid 42 (ObjNum 775 snap 243 seq_num 775) 2024-04-07T16:25:05.731 INFO:tasks.rados.rados.0.smithi100.stdout:2551: expect (ObjNum 579 snap 180 seq_num 579) 2024-04-07T16:25:05.959 INFO:tasks.rados.rados.0.smithi100.stdout:2554: finishing copy_from to smithi10066867-34 2024-04-07T16:25:05.959 INFO:tasks.rados.rados.0.smithi100.stdout:2554: got expected ENOENT (src dne) 2024-04-07T16:25:05.959 INFO:tasks.rados.rados.0.smithi100.stdout:2554: finishing copy_from racing read to smithi10066867-34 2024-04-07T16:25:05.959 INFO:tasks.rados.rados.0.smithi100.stdout:2553: finishing write tid 1 to smithi10066867-15 2024-04-07T16:25:05.960 INFO:tasks.rados.rados.0.smithi100.stdout:2553: finishing write tid 2 to smithi10066867-15 2024-04-07T16:25:05.960 INFO:tasks.rados.rados.0.smithi100.stdout:2553: finishing write tid 3 to smithi10066867-15 2024-04-07T16:25:05.960 INFO:tasks.rados.rados.0.smithi100.stdout:2553: finishing write tid 4 to smithi10066867-15 2024-04-07T16:25:05.960 INFO:tasks.rados.rados.0.smithi100.stdout:2553: oid 15 updating version 0 to 1432 2024-04-07T16:25:05.960 INFO:tasks.rados.rados.0.smithi100.stdout:2553: oid 15 updating version 1432 to 1433 2024-04-07T16:25:05.960 INFO:tasks.rados.rados.0.smithi100.stdout:2553: oid 15 version 1433 is already newer than 1431 2024-04-07T16:25:05.960 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 15 v 1433 (ObjNum 776 snap 243 seq_num 776) dirty exists 2024-04-07T16:25:05.960 INFO:tasks.rados.rados.0.smithi100.stdout:2553: left oid 15 (ObjNum 776 snap 243 seq_num 776) 2024-04-07T16:25:05.960 INFO:tasks.rados.rados.0.smithi100.stdout:2542: done (7 left) 2024-04-07T16:25:05.960 INFO:tasks.rados.rados.0.smithi100.stdout:2546: done (6 left) 2024-04-07T16:25:05.960 INFO:tasks.rados.rados.0.smithi100.stdout:2547: done (5 left) 2024-04-07T16:25:05.960 INFO:tasks.rados.rados.0.smithi100.stdout:2550: done (4 left) 2024-04-07T16:25:05.960 INFO:tasks.rados.rados.0.smithi100.stdout:2551: done (3 left) 2024-04-07T16:25:05.960 INFO:tasks.rados.rados.0.smithi100.stdout:2552: done (2 left) 2024-04-07T16:25:05.961 INFO:tasks.rados.rados.0.smithi100.stdout:2553: done (1 left) 2024-04-07T16:25:05.961 INFO:tasks.rados.rados.0.smithi100.stdout:2554: done (0 left) 2024-04-07T16:25:05.961 INFO:tasks.rados.rados.0.smithi100.stdout:2555: snap_remove snap 241 2024-04-07T16:25:06.145 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-04-07T16:25:06.145 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": { 2024-04-07T16:25:06.145 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph version 19.0.0-2785-gadce92d0 (adce92d056533f22dc1ee37e3141d098b046ec1b) squid (dev)": 3 2024-04-07T16:25:06.145 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-07T16:25:06.145 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": { 2024-04-07T16:25:06.145 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph version 19.0.0-2785-gadce92d0 (adce92d056533f22dc1ee37e3141d098b046ec1b) squid (dev)": 2 2024-04-07T16:25:06.145 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-07T16:25:06.145 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": { 2024-04-07T16:25:06.145 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph version 18.2.2-1171-g267c8ff5 (267c8ff5b1ff5815581b3b2381d19360e081b37b) reef (stable)": 1, 2024-04-07T16:25:06.145 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph version 19.0.0-2785-gadce92d0 (adce92d056533f22dc1ee37e3141d098b046ec1b) squid (dev)": 6 2024-04-07T16:25:06.145 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-07T16:25:06.145 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": { 2024-04-07T16:25:06.145 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph version 18.2.2-1171-g267c8ff5 (267c8ff5b1ff5815581b3b2381d19360e081b37b) reef (stable)": 2 2024-04-07T16:25:06.146 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-07T16:25:06.146 INFO:teuthology.orchestra.run.smithi100.stdout: "overall": { 2024-04-07T16:25:06.146 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph version 18.2.2-1171-g267c8ff5 (267c8ff5b1ff5815581b3b2381d19360e081b37b) reef (stable)": 3, 2024-04-07T16:25:06.146 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph version 19.0.0-2785-gadce92d0 (adce92d056533f22dc1ee37e3141d098b046ec1b) squid (dev)": 11 2024-04-07T16:25:06.146 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-04-07T16:25:06.146 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-04-07T16:25:06.184 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:05 smithi100 bash[74942]: cluster 2024-04-07T16:25:04.720514+0000 mgr.y (mgr.25175) 241 : cluster [DBG] pgmap v417: 121 pgs: 8 active+undersized+degraded, 35 active+undersized, 78 active+clean; 327 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 3.7 MiB/s rd, 824 KiB/s wr, 6 op/s; 32/495 objects degraded (6.465%) 2024-04-07T16:25:06.184 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:05 smithi100 bash[74942]: cluster 2024-04-07T16:25:04.869613+0000 mon.a (mon.0) 782 : cluster [WRN] Health check update: Degraded data redundancy: 32/495 objects degraded (6.465%), 8 pgs degraded (PG_DEGRADED) 2024-04-07T16:25:06.184 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:05 smithi100 bash[74942]: cluster 2024-04-07T16:25:04.886742+0000 mon.a (mon.0) 783 : cluster [DBG] osdmap e551: 8 total, 7 up, 8 in 2024-04-07T16:25:06.184 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:05 smithi100 bash[74942]: audit 2024-04-07T16:25:05.338943+0000 mgr.y (mgr.25175) 242 : audit [DBG] from='client.44276 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:25:06.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:05 smithi100 bash[76202]: cluster 2024-04-07T16:25:04.720514+0000 mgr.y (mgr.25175) 241 : cluster [DBG] pgmap v417: 121 pgs: 8 active+undersized+degraded, 35 active+undersized, 78 active+clean; 327 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 3.7 MiB/s rd, 824 KiB/s wr, 6 op/s; 32/495 objects degraded (6.465%) 2024-04-07T16:25:06.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:05 smithi100 bash[76202]: cluster 2024-04-07T16:25:04.869613+0000 mon.a (mon.0) 782 : cluster [WRN] Health check update: Degraded data redundancy: 32/495 objects degraded (6.465%), 8 pgs degraded (PG_DEGRADED) 2024-04-07T16:25:06.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:05 smithi100 bash[76202]: cluster 2024-04-07T16:25:04.886742+0000 mon.a (mon.0) 783 : cluster [DBG] osdmap e551: 8 total, 7 up, 8 in 2024-04-07T16:25:06.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:05 smithi100 bash[76202]: audit 2024-04-07T16:25:05.338943+0000 mgr.y (mgr.25175) 242 : audit [DBG] from='client.44276 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:25:06.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:05 smithi144 bash[66654]: cluster 2024-04-07T16:25:04.720514+0000 mgr.y (mgr.25175) 241 : cluster [DBG] pgmap v417: 121 pgs: 8 active+undersized+degraded, 35 active+undersized, 78 active+clean; 327 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 3.7 MiB/s rd, 824 KiB/s wr, 6 op/s; 32/495 objects degraded (6.465%) 2024-04-07T16:25:06.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:05 smithi144 bash[66654]: cluster 2024-04-07T16:25:04.869613+0000 mon.a (mon.0) 782 : cluster [WRN] Health check update: Degraded data redundancy: 32/495 objects degraded (6.465%), 8 pgs degraded (PG_DEGRADED) 2024-04-07T16:25:06.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:05 smithi144 bash[66654]: cluster 2024-04-07T16:25:04.886742+0000 mon.a (mon.0) 783 : cluster [DBG] osdmap e551: 8 total, 7 up, 8 in 2024-04-07T16:25:06.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:05 smithi144 bash[66654]: audit 2024-04-07T16:25:05.338943+0000 mgr.y (mgr.25175) 242 : audit [DBG] from='client.44276 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:25:06.891 INFO:tasks.rados.rados.0.smithi100.stdout:2555: done (0 left) 2024-04-07T16:25:06.891 INFO:tasks.rados.rados.0.smithi100.stdout:2556: read oid 3 snap -1 2024-04-07T16:25:06.891 INFO:tasks.rados.rados.0.smithi100.stdout:2556: expect (ObjNum 733 snap 226 seq_num 733) 2024-04-07T16:25:06.891 INFO:tasks.rados.rados.0.smithi100.stdout:2557: snap_remove snap 234 2024-04-07T16:25:07.105 INFO:journalctl@ceph.osd.6.smithi144.stdout:Apr 07 16:25:06 smithi144 bash[79803]: debug 2024-04-07T16:25:06.781+0000 7f895eccc7c0 -1 osd.6 542 log_to_monitors true 2024-04-07T16:25:07.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:06 smithi144 bash[66654]: audit 2024-04-07T16:25:05.701629+0000 mgr.y (mgr.25175) 243 : audit [DBG] from='client.34362 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:25:07.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:06 smithi144 bash[66654]: audit 2024-04-07T16:25:06.162817+0000 mon.c (mon.2) 12 : audit [DBG] from='client.? 172.21.15.100:0/414126112' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:25:07.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:06 smithi144 bash[66654]: audit 2024-04-07T16:25:06.791067+0000 mon.b (mon.1) 7 : audit [INF] from='osd.6 [v2:172.21.15.144:6816/2416344523,v1:172.21.15.144:6817/2416344523]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-07T16:25:07.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:06 smithi144 bash[66654]: audit 2024-04-07T16:25:06.799715+0000 mon.a (mon.0) 784 : audit [INF] from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-07T16:25:07.184 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:06 smithi100 bash[74942]: audit 2024-04-07T16:25:05.701629+0000 mgr.y (mgr.25175) 243 : audit [DBG] from='client.34362 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:25:07.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:06 smithi100 bash[74942]: audit 2024-04-07T16:25:06.162817+0000 mon.c (mon.2) 12 : audit [DBG] from='client.? 172.21.15.100:0/414126112' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:25:07.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:06 smithi100 bash[74942]: audit 2024-04-07T16:25:06.791067+0000 mon.b (mon.1) 7 : audit [INF] from='osd.6 [v2:172.21.15.144:6816/2416344523,v1:172.21.15.144:6817/2416344523]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-07T16:25:07.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:06 smithi100 bash[74942]: audit 2024-04-07T16:25:06.799715+0000 mon.a (mon.0) 784 : audit [INF] from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-07T16:25:07.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:06 smithi100 bash[76202]: audit 2024-04-07T16:25:05.701629+0000 mgr.y (mgr.25175) 243 : audit [DBG] from='client.34362 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:25:07.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:06 smithi100 bash[76202]: audit 2024-04-07T16:25:06.162817+0000 mon.c (mon.2) 12 : audit [DBG] from='client.? 172.21.15.100:0/414126112' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:25:07.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:06 smithi100 bash[76202]: audit 2024-04-07T16:25:06.791067+0000 mon.b (mon.1) 7 : audit [INF] from='osd.6 [v2:172.21.15.144:6816/2416344523,v1:172.21.15.144:6817/2416344523]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-07T16:25:07.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:06 smithi100 bash[76202]: audit 2024-04-07T16:25:06.799715+0000 mon.a (mon.0) 784 : audit [INF] from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-07T16:25:07.855 INFO:journalctl@ceph.osd.6.smithi144.stdout:Apr 07 16:25:07 smithi144 bash[79803]: debug 2024-04-07T16:25:07.437+0000 7f894c11c700 -1 osd.6 542 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-07T16:25:07.908 INFO:tasks.rados.rados.0.smithi100.stdout:2557: done (1 left) 2024-04-07T16:25:07.908 INFO:tasks.rados.rados.0.smithi100.stdout:2558: read oid 33 snap -1 2024-04-07T16:25:07.908 INFO:tasks.rados.rados.0.smithi100.stdout:2558: expect (ObjNum 753 snap 235 seq_num 753) 2024-04-07T16:25:07.908 INFO:tasks.rados.rados.0.smithi100.stdout:2559: rollback oid 50 current snap is 243 2024-04-07T16:25:07.908 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 50 to 240 2024-04-07T16:25:07.908 INFO:tasks.rados.rados.0.smithi100.stdout:2556: expect (ObjNum 733 snap 226 seq_num 733) 2024-04-07T16:25:08.024 INFO:tasks.rados.rados.0.smithi100.stdout:2559: finishing rollback tid 0 to smithi10066867-50 2024-04-07T16:25:08.024 INFO:tasks.rados.rados.0.smithi100.stdout:2559: finishing rollback tid 1 to smithi10066867-50 2024-04-07T16:25:08.024 INFO:tasks.rados.rados.0.smithi100.stdout:2559: finishing rollback tid 2 to smithi10066867-50 2024-04-07T16:25:08.024 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 50 v 1030 (ObjNum 657 snap 203 seq_num 657) dirty exists 2024-04-07T16:25:08.024 INFO:tasks.rados.rados.0.smithi100.stdout:2558: expect (ObjNum 753 snap 235 seq_num 753) 2024-04-07T16:25:08.039 INFO:tasks.rados.rados.0.smithi100.stdout:2556: done (2 left) 2024-04-07T16:25:08.039 INFO:tasks.rados.rados.0.smithi100.stdout:2558: done (1 left) 2024-04-07T16:25:08.039 INFO:tasks.rados.rados.0.smithi100.stdout:2559: done (0 left) 2024-04-07T16:25:08.039 INFO:tasks.rados.rados.0.smithi100.stdout:2560: copy_from oid 38 from oid 10 current snap is 243 2024-04-07T16:25:08.040 INFO:tasks.rados.rados.0.smithi100.stdout:2561: snap_remove snap 186 2024-04-07T16:25:08.184 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:07 smithi100 bash[74942]: cluster 2024-04-07T16:25:06.721510+0000 mgr.y (mgr.25175) 244 : cluster [DBG] pgmap v419: 121 pgs: 8 active+undersized+degraded, 35 active+undersized, 78 active+clean; 327 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 3.1 MiB/s rd, 677 KiB/s wr, 10 op/s; 32/501 objects degraded (6.387%) 2024-04-07T16:25:08.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:07 smithi100 bash[74942]: audit 2024-04-07T16:25:06.892167+0000 mon.a (mon.0) 785 : audit [INF] from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-07T16:25:08.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:07 smithi100 bash[74942]: audit 2024-04-07T16:25:06.893159+0000 mon.b (mon.1) 8 : audit [INF] from='osd.6 [v2:172.21.15.144:6816/2416344523,v1:172.21.15.144:6817/2416344523]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-04-07T16:25:08.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:07 smithi100 bash[74942]: cluster 2024-04-07T16:25:06.904483+0000 mon.a (mon.0) 786 : cluster [DBG] osdmap e552: 8 total, 7 up, 8 in 2024-04-07T16:25:08.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:07 smithi100 bash[74942]: audit 2024-04-07T16:25:06.908375+0000 mon.a (mon.0) 787 : audit [INF] from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-04-07T16:25:08.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:07 smithi100 bash[74942]: audit 2024-04-07T16:25:07.423565+0000 mon.a (mon.0) 788 : audit [INF] from='osd.6 ' entity='osd.6' 2024-04-07T16:25:08.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:07 smithi100 bash[76202]: cluster 2024-04-07T16:25:06.721510+0000 mgr.y (mgr.25175) 244 : cluster [DBG] pgmap v419: 121 pgs: 8 active+undersized+degraded, 35 active+undersized, 78 active+clean; 327 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 3.1 MiB/s rd, 677 KiB/s wr, 10 op/s; 32/501 objects degraded (6.387%) 2024-04-07T16:25:08.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:07 smithi100 bash[76202]: audit 2024-04-07T16:25:06.892167+0000 mon.a (mon.0) 785 : audit [INF] from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-07T16:25:08.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:07 smithi100 bash[76202]: audit 2024-04-07T16:25:06.893159+0000 mon.b (mon.1) 8 : audit [INF] from='osd.6 [v2:172.21.15.144:6816/2416344523,v1:172.21.15.144:6817/2416344523]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-04-07T16:25:08.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:07 smithi100 bash[76202]: cluster 2024-04-07T16:25:06.904483+0000 mon.a (mon.0) 786 : cluster [DBG] osdmap e552: 8 total, 7 up, 8 in 2024-04-07T16:25:08.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:07 smithi100 bash[76202]: audit 2024-04-07T16:25:06.908375+0000 mon.a (mon.0) 787 : audit [INF] from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-04-07T16:25:08.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:07 smithi100 bash[76202]: audit 2024-04-07T16:25:07.423565+0000 mon.a (mon.0) 788 : audit [INF] from='osd.6 ' entity='osd.6' 2024-04-07T16:25:08.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:07 smithi144 bash[66654]: cluster 2024-04-07T16:25:06.721510+0000 mgr.y (mgr.25175) 244 : cluster [DBG] pgmap v419: 121 pgs: 8 active+undersized+degraded, 35 active+undersized, 78 active+clean; 327 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 3.1 MiB/s rd, 677 KiB/s wr, 10 op/s; 32/501 objects degraded (6.387%) 2024-04-07T16:25:08.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:07 smithi144 bash[66654]: audit 2024-04-07T16:25:06.892167+0000 mon.a (mon.0) 785 : audit [INF] from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-07T16:25:08.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:07 smithi144 bash[66654]: audit 2024-04-07T16:25:06.893159+0000 mon.b (mon.1) 8 : audit [INF] from='osd.6 [v2:172.21.15.144:6816/2416344523,v1:172.21.15.144:6817/2416344523]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-04-07T16:25:08.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:07 smithi144 bash[66654]: cluster 2024-04-07T16:25:06.904483+0000 mon.a (mon.0) 786 : cluster [DBG] osdmap e552: 8 total, 7 up, 8 in 2024-04-07T16:25:08.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:07 smithi144 bash[66654]: audit 2024-04-07T16:25:06.908375+0000 mon.a (mon.0) 787 : audit [INF] from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-04-07T16:25:08.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:07 smithi144 bash[66654]: audit 2024-04-07T16:25:07.423565+0000 mon.a (mon.0) 788 : audit [INF] from='osd.6 ' entity='osd.6' 2024-04-07T16:25:08.684 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:25:08 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:25:08] "GET /metrics HTTP/1.1" 200 36276 "" "Prometheus/2.43.0" 2024-04-07T16:25:08.908 INFO:tasks.rados.rados.0.smithi100.stdout:2561: done (1 left) 2024-04-07T16:25:08.908 INFO:tasks.rados.rados.0.smithi100.stdout:2562: rollback oid 46 current snap is 243 2024-04-07T16:25:08.908 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 46 to 233 2024-04-07T16:25:08.908 INFO:tasks.rados.rados.0.smithi100.stdout:2563: read oid 42 snap 237 2024-04-07T16:25:08.908 INFO:tasks.rados.rados.0.smithi100.stdout:2563: expect deleted 2024-04-07T16:25:08.909 INFO:tasks.rados.rados.0.smithi100.stdout:2564: snap_create 2024-04-07T16:25:08.937 INFO:tasks.rados.rados.0.smithi100.stdout:2562: finishing rollback tid 1 to smithi10066867-46 2024-04-07T16:25:08.938 INFO:tasks.rados.rados.0.smithi100.stdout:2562: finishing rollback tid 2 to smithi10066867-46 2024-04-07T16:25:08.938 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 46 v 1044 (ObjNum 580 snap 180 seq_num 580) dirty exists 2024-04-07T16:25:08.975 INFO:tasks.rados.rados.0.smithi100.stdout:2560: finishing copy_from racing read to smithi10066867-38 2024-04-07T16:25:08.984 INFO:tasks.rados.rados.0.smithi100.stdout:2560: finishing copy_from to smithi10066867-38 2024-04-07T16:25:08.984 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 38 v 1103 (ObjNum 720 snap 221 seq_num 720) dirty exists 2024-04-07T16:25:09.184 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:08 smithi100 bash[74942]: audit 2024-04-07T16:25:07.893269+0000 mon.a (mon.0) 789 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:25:09.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:08 smithi100 bash[74942]: cluster 2024-04-07T16:25:07.896147+0000 mon.a (mon.0) 790 : cluster [INF] Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-07T16:25:09.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:08 smithi100 bash[74942]: cluster 2024-04-07T16:25:07.914778+0000 mon.a (mon.0) 791 : cluster [INF] osd.6 [v2:172.21.15.144:6816/2416344523,v1:172.21.15.144:6817/2416344523] boot 2024-04-07T16:25:09.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:08 smithi100 bash[74942]: cluster 2024-04-07T16:25:07.914895+0000 mon.a (mon.0) 792 : cluster [DBG] osdmap e553: 8 total, 8 up, 8 in 2024-04-07T16:25:09.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:08 smithi100 bash[74942]: audit 2024-04-07T16:25:07.924910+0000 mon.a (mon.0) 793 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-07T16:25:09.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:08 smithi100 bash[74942]: audit 2024-04-07T16:25:07.948800+0000 mon.a (mon.0) 794 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:25:09.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:08 smithi100 bash[76202]: audit 2024-04-07T16:25:07.893269+0000 mon.a (mon.0) 789 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:25:09.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:08 smithi100 bash[76202]: cluster 2024-04-07T16:25:07.896147+0000 mon.a (mon.0) 790 : cluster [INF] Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-07T16:25:09.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:08 smithi100 bash[76202]: cluster 2024-04-07T16:25:07.914778+0000 mon.a (mon.0) 791 : cluster [INF] osd.6 [v2:172.21.15.144:6816/2416344523,v1:172.21.15.144:6817/2416344523] boot 2024-04-07T16:25:09.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:08 smithi100 bash[76202]: cluster 2024-04-07T16:25:07.914895+0000 mon.a (mon.0) 792 : cluster [DBG] osdmap e553: 8 total, 8 up, 8 in 2024-04-07T16:25:09.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:08 smithi100 bash[76202]: audit 2024-04-07T16:25:07.924910+0000 mon.a (mon.0) 793 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-07T16:25:09.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:08 smithi100 bash[76202]: audit 2024-04-07T16:25:07.948800+0000 mon.a (mon.0) 794 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:25:09.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:08 smithi144 bash[66654]: audit 2024-04-07T16:25:07.893269+0000 mon.a (mon.0) 789 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:25:09.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:08 smithi144 bash[66654]: cluster 2024-04-07T16:25:07.896147+0000 mon.a (mon.0) 790 : cluster [INF] Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-07T16:25:09.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:08 smithi144 bash[66654]: cluster 2024-04-07T16:25:07.914778+0000 mon.a (mon.0) 791 : cluster [INF] osd.6 [v2:172.21.15.144:6816/2416344523,v1:172.21.15.144:6817/2416344523] boot 2024-04-07T16:25:09.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:08 smithi144 bash[66654]: cluster 2024-04-07T16:25:07.914895+0000 mon.a (mon.0) 792 : cluster [DBG] osdmap e553: 8 total, 8 up, 8 in 2024-04-07T16:25:09.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:08 smithi144 bash[66654]: audit 2024-04-07T16:25:07.924910+0000 mon.a (mon.0) 793 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-07T16:25:09.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:08 smithi144 bash[66654]: audit 2024-04-07T16:25:07.948800+0000 mon.a (mon.0) 794 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:25:09.912 INFO:tasks.rados.rados.0.smithi100.stdout:2560: done (3 left) 2024-04-07T16:25:09.912 INFO:tasks.rados.rados.0.smithi100.stdout:2562: done (2 left) 2024-04-07T16:25:09.912 INFO:tasks.rados.rados.0.smithi100.stdout:2563: done (1 left) 2024-04-07T16:25:09.912 INFO:tasks.rados.rados.0.smithi100.stdout:2564: done (0 left) 2024-04-07T16:25:09.912 INFO:tasks.rados.rados.0.smithi100.stdout:2565: setattr oid 9 current snap is 244 2024-04-07T16:25:09.916 INFO:tasks.rados.rados.0.smithi100.stdout:2566: read oid 10 snap -1 2024-04-07T16:25:09.916 INFO:tasks.rados.rados.0.smithi100.stdout:2566: expect (ObjNum 720 snap 221 seq_num 720) 2024-04-07T16:25:09.916 INFO:tasks.rados.rados.0.smithi100.stdout:2567: setattr oid 3 current snap is 244 2024-04-07T16:25:09.921 INFO:tasks.rados.rados.0.smithi100.stdout:2568: read oid 21 snap -1 2024-04-07T16:25:09.921 INFO:tasks.rados.rados.0.smithi100.stdout:2568: expect (ObjNum 731 snap 225 seq_num 731) 2024-04-07T16:25:09.921 INFO:tasks.rados.rados.0.smithi100.stdout:2569: setattr oid 7 current snap is 244 2024-04-07T16:25:09.927 INFO:tasks.rados.rados.0.smithi100.stdout:2570: read oid 23 snap -1 2024-04-07T16:25:09.927 INFO:tasks.rados.rados.0.smithi100.stdout:2570: expect deleted 2024-04-07T16:25:09.927 INFO:tasks.rados.rados.0.smithi100.stdout:2571: rmattr oid 5 current snap is 244 2024-04-07T16:25:09.927 INFO:tasks.rados.rados.0.smithi100.stdout:2571: done (6 left) 2024-04-07T16:25:09.928 INFO:tasks.rados.rados.0.smithi100.stdout:2572: snap_remove snap 240 2024-04-07T16:25:10.184 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:09 smithi100 bash[74942]: cluster 2024-04-07T16:25:08.722480+0000 mgr.y (mgr.25175) 245 : cluster [DBG] pgmap v422: 121 pgs: 1 peering, 7 active+undersized+degraded, 35 active+undersized, 78 active+clean; 331 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 4.3 MiB/s rd, 1.1 MiB/s wr, 19 op/s; 17/495 objects degraded (3.434%) 2024-04-07T16:25:10.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:09 smithi100 bash[74942]: cluster 2024-04-07T16:25:08.919674+0000 mon.a (mon.0) 795 : cluster [DBG] osdmap e554: 8 total, 8 up, 8 in 2024-04-07T16:25:10.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:09 smithi100 bash[74942]: audit 2024-04-07T16:25:09.153247+0000 mon.a (mon.0) 796 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:25:10.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:09 smithi100 bash[74942]: audit 2024-04-07T16:25:09.167037+0000 mon.a (mon.0) 797 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:25:10.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:09 smithi100 bash[76202]: cluster 2024-04-07T16:25:08.722480+0000 mgr.y (mgr.25175) 245 : cluster [DBG] pgmap v422: 121 pgs: 1 peering, 7 active+undersized+degraded, 35 active+undersized, 78 active+clean; 331 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 4.3 MiB/s rd, 1.1 MiB/s wr, 19 op/s; 17/495 objects degraded (3.434%) 2024-04-07T16:25:10.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:09 smithi100 bash[76202]: cluster 2024-04-07T16:25:08.919674+0000 mon.a (mon.0) 795 : cluster [DBG] osdmap e554: 8 total, 8 up, 8 in 2024-04-07T16:25:10.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:09 smithi100 bash[76202]: audit 2024-04-07T16:25:09.153247+0000 mon.a (mon.0) 796 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:25:10.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:09 smithi100 bash[76202]: audit 2024-04-07T16:25:09.167037+0000 mon.a (mon.0) 797 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:25:10.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:09 smithi144 bash[66654]: cluster 2024-04-07T16:25:08.722480+0000 mgr.y (mgr.25175) 245 : cluster [DBG] pgmap v422: 121 pgs: 1 peering, 7 active+undersized+degraded, 35 active+undersized, 78 active+clean; 331 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 4.3 MiB/s rd, 1.1 MiB/s wr, 19 op/s; 17/495 objects degraded (3.434%) 2024-04-07T16:25:10.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:09 smithi144 bash[66654]: cluster 2024-04-07T16:25:08.919674+0000 mon.a (mon.0) 795 : cluster [DBG] osdmap e554: 8 total, 8 up, 8 in 2024-04-07T16:25:10.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:09 smithi144 bash[66654]: audit 2024-04-07T16:25:09.153247+0000 mon.a (mon.0) 796 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:25:10.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:09 smithi144 bash[66654]: audit 2024-04-07T16:25:09.167037+0000 mon.a (mon.0) 797 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:25:10.926 INFO:tasks.rados.rados.0.smithi100.stdout:2572: done (6 left) 2024-04-07T16:25:10.926 INFO:tasks.rados.rados.0.smithi100.stdout:2573: copy_from oid 17 from oid 14 current snap is 244 2024-04-07T16:25:10.926 INFO:tasks.rados.rados.0.smithi100.stdout:2574: snap_create 2024-04-07T16:25:10.927 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 3 v 1106 (ObjNum 733 snap 226 seq_num 733) dirty exists 2024-04-07T16:25:10.927 INFO:tasks.rados.rados.0.smithi100.stdout:2568: expect (ObjNum 731 snap 225 seq_num 731) 2024-04-07T16:25:10.939 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 7 v 1116 (ObjNum 752 snap 234 seq_num 752) dirty exists 2024-04-07T16:25:10.939 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 9 v 1109 (ObjNum 770 snap 241 seq_num 770) dirty exists 2024-04-07T16:25:10.939 INFO:tasks.rados.rados.0.smithi100.stdout:2566: expect (ObjNum 720 snap 221 seq_num 720) 2024-04-07T16:25:11.122 INFO:tasks.rados.rados.0.smithi100.stdout:2573: finishing copy_from racing read to smithi10066867-17 2024-04-07T16:25:11.122 INFO:tasks.rados.rados.0.smithi100.stdout:2573: finishing copy_from to smithi10066867-17 2024-04-07T16:25:11.122 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 17 v 1027 (ObjNum 762 snap 240 seq_num 762) dirty exists 2024-04-07T16:25:11.184 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:10 smithi100 bash[74942]: cluster 2024-04-07T16:25:09.929830+0000 mon.a (mon.0) 798 : cluster [DBG] osdmap e555: 8 total, 8 up, 8 in 2024-04-07T16:25:11.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:10 smithi100 bash[76202]: cluster 2024-04-07T16:25:09.929830+0000 mon.a (mon.0) 798 : cluster [DBG] osdmap e555: 8 total, 8 up, 8 in 2024-04-07T16:25:11.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:10 smithi144 bash[66654]: cluster 2024-04-07T16:25:09.929830+0000 mon.a (mon.0) 798 : cluster [DBG] osdmap e555: 8 total, 8 up, 8 in 2024-04-07T16:25:11.933 INFO:tasks.rados.rados.0.smithi100.stdout:2565: done (7 left) 2024-04-07T16:25:11.934 INFO:tasks.rados.rados.0.smithi100.stdout:2566: done (6 left) 2024-04-07T16:25:11.934 INFO:tasks.rados.rados.0.smithi100.stdout:2567: done (5 left) 2024-04-07T16:25:11.934 INFO:tasks.rados.rados.0.smithi100.stdout:2568: done (4 left) 2024-04-07T16:25:11.934 INFO:tasks.rados.rados.0.smithi100.stdout:2569: done (3 left) 2024-04-07T16:25:11.934 INFO:tasks.rados.rados.0.smithi100.stdout:2570: done (2 left) 2024-04-07T16:25:11.934 INFO:tasks.rados.rados.0.smithi100.stdout:2573: done (1 left) 2024-04-07T16:25:11.934 INFO:tasks.rados.rados.0.smithi100.stdout:2574: done (0 left) 2024-04-07T16:25:11.934 INFO:tasks.rados.rados.0.smithi100.stdout:2575: snap_create 2024-04-07T16:25:12.184 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:11 smithi100 bash[74942]: cluster 2024-04-07T16:25:10.723305+0000 mgr.y (mgr.25175) 246 : cluster [DBG] pgmap v425: 121 pgs: 3 peering, 6 active+undersized+degraded, 25 active+undersized, 87 active+clean; 309 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 15 MiB/s rd, 2.2 MiB/s wr, 22 op/s; 16/456 objects degraded (3.509%) 2024-04-07T16:25:12.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:11 smithi100 bash[74942]: cluster 2024-04-07T16:25:10.922780+0000 mon.a (mon.0) 799 : cluster [WRN] Health check update: Degraded data redundancy: 16/456 objects degraded (3.509%), 6 pgs degraded (PG_DEGRADED) 2024-04-07T16:25:12.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:11 smithi100 bash[74942]: cluster 2024-04-07T16:25:10.941953+0000 mon.a (mon.0) 800 : cluster [DBG] osdmap e556: 8 total, 8 up, 8 in 2024-04-07T16:25:12.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:11 smithi100 bash[76202]: cluster 2024-04-07T16:25:10.723305+0000 mgr.y (mgr.25175) 246 : cluster [DBG] pgmap v425: 121 pgs: 3 peering, 6 active+undersized+degraded, 25 active+undersized, 87 active+clean; 309 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 15 MiB/s rd, 2.2 MiB/s wr, 22 op/s; 16/456 objects degraded (3.509%) 2024-04-07T16:25:12.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:11 smithi100 bash[76202]: cluster 2024-04-07T16:25:10.922780+0000 mon.a (mon.0) 799 : cluster [WRN] Health check update: Degraded data redundancy: 16/456 objects degraded (3.509%), 6 pgs degraded (PG_DEGRADED) 2024-04-07T16:25:12.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:11 smithi100 bash[76202]: cluster 2024-04-07T16:25:10.941953+0000 mon.a (mon.0) 800 : cluster [DBG] osdmap e556: 8 total, 8 up, 8 in 2024-04-07T16:25:12.304 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:11 smithi144 bash[66654]: cluster 2024-04-07T16:25:10.723305+0000 mgr.y (mgr.25175) 246 : cluster [DBG] pgmap v425: 121 pgs: 3 peering, 6 active+undersized+degraded, 25 active+undersized, 87 active+clean; 309 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 15 MiB/s rd, 2.2 MiB/s wr, 22 op/s; 16/456 objects degraded (3.509%) 2024-04-07T16:25:12.304 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:11 smithi144 bash[66654]: cluster 2024-04-07T16:25:10.922780+0000 mon.a (mon.0) 799 : cluster [WRN] Health check update: Degraded data redundancy: 16/456 objects degraded (3.509%), 6 pgs degraded (PG_DEGRADED) 2024-04-07T16:25:12.304 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:11 smithi144 bash[66654]: cluster 2024-04-07T16:25:10.941953+0000 mon.a (mon.0) 800 : cluster [DBG] osdmap e556: 8 total, 8 up, 8 in 2024-04-07T16:25:12.606 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:25:12 smithi144 bash[59794]: ts=2024-04-07T16:25:12.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:25:12.947 INFO:tasks.rados.rados.0.smithi100.stdout:2575: done (0 left) 2024-04-07T16:25:12.947 INFO:tasks.rados.rados.0.smithi100.stdout:2576: snap_remove snap 244 2024-04-07T16:25:13.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:12 smithi144 bash[66654]: cluster 2024-04-07T16:25:11.950621+0000 mon.a (mon.0) 801 : cluster [DBG] osdmap e557: 8 total, 8 up, 8 in 2024-04-07T16:25:13.434 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:12 smithi100 bash[74942]: cluster 2024-04-07T16:25:11.950621+0000 mon.a (mon.0) 801 : cluster [DBG] osdmap e557: 8 total, 8 up, 8 in 2024-04-07T16:25:13.434 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:12 smithi100 bash[76202]: cluster 2024-04-07T16:25:11.950621+0000 mon.a (mon.0) 801 : cluster [DBG] osdmap e557: 8 total, 8 up, 8 in 2024-04-07T16:25:13.949 INFO:tasks.rados.rados.0.smithi100.stdout:2576: done (0 left) 2024-04-07T16:25:13.949 INFO:tasks.rados.rados.0.smithi100.stdout:2577: rmattr oid 25 current snap is 246 2024-04-07T16:25:13.949 INFO:tasks.rados.rados.0.smithi100.stdout:2578: read oid 26 snap -1 2024-04-07T16:25:13.949 INFO:tasks.rados.rados.0.smithi100.stdout:2578: expect (ObjNum 773 snap 243 seq_num 773) 2024-04-07T16:25:13.949 INFO:tasks.rados.rados.0.smithi100.stdout:2579: delete oid 30 current snap is 246 2024-04-07T16:25:13.958 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 25 v 0 (ObjNum 540 snap 164 seq_num 540) dirty exists 2024-04-07T16:25:13.965 INFO:tasks.rados.rados.0.smithi100.stdout:2577: done (2 left) 2024-04-07T16:25:13.965 INFO:tasks.rados.rados.0.smithi100.stdout:2579: done (1 left) 2024-04-07T16:25:13.965 INFO:tasks.rados.rados.0.smithi100.stdout:2580: rollback oid 22 current snap is 246 2024-04-07T16:25:13.965 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 22 to 235 2024-04-07T16:25:13.965 INFO:tasks.rados.rados.0.smithi100.stdout:2581: snap_remove snap 233 2024-04-07T16:25:14.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:13 smithi144 bash[66654]: cluster 2024-04-07T16:25:12.724406+0000 mgr.y (mgr.25175) 247 : cluster [DBG] pgmap v428: 121 pgs: 3 peering, 118 active+clean; 285 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 9.9 MiB/s rd, 2.0 MiB/s wr, 19 op/s; 220 KiB/s, 0 objects/s recovering 2024-04-07T16:25:14.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:13 smithi144 bash[66654]: cluster 2024-04-07T16:25:12.943106+0000 mon.a (mon.0) 802 : cluster [INF] Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 16/456 objects degraded (3.509%), 6 pgs degraded) 2024-04-07T16:25:14.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:13 smithi144 bash[66654]: cluster 2024-04-07T16:25:12.943163+0000 mon.a (mon.0) 803 : cluster [INF] Cluster is now healthy 2024-04-07T16:25:14.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:13 smithi144 bash[66654]: cluster 2024-04-07T16:25:12.964360+0000 mon.a (mon.0) 804 : cluster [DBG] osdmap e558: 8 total, 8 up, 8 in 2024-04-07T16:25:14.435 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:13 smithi100 bash[74942]: cluster 2024-04-07T16:25:12.724406+0000 mgr.y (mgr.25175) 247 : cluster [DBG] pgmap v428: 121 pgs: 3 peering, 118 active+clean; 285 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 9.9 MiB/s rd, 2.0 MiB/s wr, 19 op/s; 220 KiB/s, 0 objects/s recovering 2024-04-07T16:25:14.435 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:13 smithi100 bash[74942]: cluster 2024-04-07T16:25:12.943106+0000 mon.a (mon.0) 802 : cluster [INF] Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 16/456 objects degraded (3.509%), 6 pgs degraded) 2024-04-07T16:25:14.435 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:13 smithi100 bash[74942]: cluster 2024-04-07T16:25:12.943163+0000 mon.a (mon.0) 803 : cluster [INF] Cluster is now healthy 2024-04-07T16:25:14.435 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:13 smithi100 bash[74942]: cluster 2024-04-07T16:25:12.964360+0000 mon.a (mon.0) 804 : cluster [DBG] osdmap e558: 8 total, 8 up, 8 in 2024-04-07T16:25:14.435 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:13 smithi100 bash[76202]: cluster 2024-04-07T16:25:12.724406+0000 mgr.y (mgr.25175) 247 : cluster [DBG] pgmap v428: 121 pgs: 3 peering, 118 active+clean; 285 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 9.9 MiB/s rd, 2.0 MiB/s wr, 19 op/s; 220 KiB/s, 0 objects/s recovering 2024-04-07T16:25:14.435 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:13 smithi100 bash[76202]: cluster 2024-04-07T16:25:12.943106+0000 mon.a (mon.0) 802 : cluster [INF] Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 16/456 objects degraded (3.509%), 6 pgs degraded) 2024-04-07T16:25:14.435 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:13 smithi100 bash[76202]: cluster 2024-04-07T16:25:12.943163+0000 mon.a (mon.0) 803 : cluster [INF] Cluster is now healthy 2024-04-07T16:25:14.435 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:13 smithi100 bash[76202]: cluster 2024-04-07T16:25:12.964360+0000 mon.a (mon.0) 804 : cluster [DBG] osdmap e558: 8 total, 8 up, 8 in 2024-04-07T16:25:14.985 INFO:tasks.rados.rados.0.smithi100.stdout:2581: done (2 left) 2024-04-07T16:25:14.985 INFO:tasks.rados.rados.0.smithi100.stdout:2582: copy_from oid 37 from oid 45 current snap is 246 2024-04-07T16:25:14.985 INFO:tasks.rados.rados.0.smithi100.stdout:2583: read oid 10 snap -1 2024-04-07T16:25:14.985 INFO:tasks.rados.rados.0.smithi100.stdout:2583: expect (ObjNum 720 snap 221 seq_num 720) 2024-04-07T16:25:14.985 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 40 current snap is 246 2024-04-07T16:25:14.985 INFO:tasks.rados.rados.0.smithi100.stdout:2584: seq_num 778 ranges {0=638976,638976=401408} 2024-04-07T16:25:14.991 INFO:tasks.rados.rados.0.smithi100.stdout:2584: writing smithi10066867-40 from 0 to 638976 tid 1 2024-04-07T16:25:15.003 INFO:tasks.rados.rados.0.smithi100.stdout:2584: writing smithi10066867-40 from 638976 to 1040384 tid 2 2024-04-07T16:25:15.010 INFO:tasks.rados.rados.0.smithi100.stdout:2578: expect (ObjNum 773 snap 243 seq_num 773) 2024-04-07T16:25:15.051 INFO:tasks.rados.rados.0.smithi100.stdout:2580: finishing rollback tid 0 to smithi10066867-22 2024-04-07T16:25:15.051 INFO:tasks.rados.rados.0.smithi100.stdout:2580: finishing rollback tid 1 to smithi10066867-22 2024-04-07T16:25:15.051 INFO:tasks.rados.rados.0.smithi100.stdout:2578: done (4 left) 2024-04-07T16:25:15.051 INFO:tasks.rados.rados.0.smithi100.stdout:2585: snap_remove snap 236 2024-04-07T16:25:15.220 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:14 smithi144 bash[66654]: cluster 2024-04-07T16:25:13.964961+0000 mon.a (mon.0) 805 : cluster [DBG] osdmap e559: 8 total, 8 up, 8 in 2024-04-07T16:25:15.434 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:14 smithi100 bash[74942]: cluster 2024-04-07T16:25:13.964961+0000 mon.a (mon.0) 805 : cluster [DBG] osdmap e559: 8 total, 8 up, 8 in 2024-04-07T16:25:15.435 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:14 smithi100 bash[76202]: cluster 2024-04-07T16:25:13.964961+0000 mon.a (mon.0) 805 : cluster [DBG] osdmap e559: 8 total, 8 up, 8 in 2024-04-07T16:25:15.974 INFO:tasks.rados.rados.0.smithi100.stdout:2585: done (4 left) 2024-04-07T16:25:15.975 INFO:tasks.rados.rados.0.smithi100.stdout:2586: read oid 3 snap -1 2024-04-07T16:25:15.975 INFO:tasks.rados.rados.0.smithi100.stdout:2586: expect (ObjNum 733 snap 226 seq_num 733) 2024-04-07T16:25:15.975 INFO:tasks.rados.rados.0.smithi100.stdout:2587: snap_remove snap 225 2024-04-07T16:25:16.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:15 smithi144 bash[66654]: cluster 2024-04-07T16:25:14.725302+0000 mgr.y (mgr.25175) 248 : cluster [DBG] pgmap v431: 121 pgs: 121 active+clean; 191 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 528 KiB/s rd, 910 KiB/s wr, 12 op/s; 650 KiB/s, 1 objects/s recovering 2024-04-07T16:25:16.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:15 smithi144 bash[66654]: cluster 2024-04-07T16:25:14.993933+0000 mon.a (mon.0) 806 : cluster [DBG] osdmap e560: 8 total, 8 up, 8 in 2024-04-07T16:25:16.434 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:16 smithi100 bash[74942]: cluster 2024-04-07T16:25:14.725302+0000 mgr.y (mgr.25175) 248 : cluster [DBG] pgmap v431: 121 pgs: 121 active+clean; 191 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 528 KiB/s rd, 910 KiB/s wr, 12 op/s; 650 KiB/s, 1 objects/s recovering 2024-04-07T16:25:16.435 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:16 smithi100 bash[74942]: cluster 2024-04-07T16:25:14.993933+0000 mon.a (mon.0) 806 : cluster [DBG] osdmap e560: 8 total, 8 up, 8 in 2024-04-07T16:25:16.435 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:16 smithi100 bash[76202]: cluster 2024-04-07T16:25:14.725302+0000 mgr.y (mgr.25175) 248 : cluster [DBG] pgmap v431: 121 pgs: 121 active+clean; 191 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 528 KiB/s rd, 910 KiB/s wr, 12 op/s; 650 KiB/s, 1 objects/s recovering 2024-04-07T16:25:16.435 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:16 smithi100 bash[76202]: cluster 2024-04-07T16:25:14.993933+0000 mon.a (mon.0) 806 : cluster [DBG] osdmap e560: 8 total, 8 up, 8 in 2024-04-07T16:25:16.856 INFO:tasks.rados.rados.0.smithi100.stdout:2587: done (5 left) 2024-04-07T16:25:16.856 INFO:tasks.rados.rados.0.smithi100.stdout:2588: copy_from oid 30 from oid 49 current snap is 246 2024-04-07T16:25:16.856 INFO:tasks.rados.rados.0.smithi100.stdout:2580: finishing rollback tid 2 to smithi10066867-22 2024-04-07T16:25:16.856 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 22 v 1174 (ObjNum 744 snap 228 seq_num 744) dirty exists 2024-04-07T16:25:16.856 INFO:tasks.rados.rados.0.smithi100.stdout:2580: done (5 left) 2024-04-07T16:25:16.856 INFO:tasks.rados.rados.0.smithi100.stdout:2589: read oid 38 snap -1 2024-04-07T16:25:16.856 INFO:tasks.rados.rados.0.smithi100.stdout:2589: expect (ObjNum 720 snap 221 seq_num 720) 2024-04-07T16:25:16.856 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 9 current snap is 246 2024-04-07T16:25:16.856 INFO:tasks.rados.rados.0.smithi100.stdout:2590: seq_num 779 ranges {901120=655360,1556480=401408} 2024-04-07T16:25:16.871 INFO:tasks.rados.rados.0.smithi100.stdout:2590: writing smithi10066867-9 from 901120 to 1556480 tid 1 2024-04-07T16:25:16.875 INFO:tasks.rados.rados.0.smithi100.stdout:2590: writing smithi10066867-9 from 1556480 to 1957888 tid 2 2024-04-07T16:25:16.876 INFO:tasks.rados.rados.0.smithi100.stdout:2591: rollback oid 44 current snap is 246 2024-04-07T16:25:16.876 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 44 to 235 2024-04-07T16:25:16.876 INFO:tasks.rados.rados.0.smithi100.stdout:2584: finishing write tid 1 to smithi10066867-40 2024-04-07T16:25:16.876 INFO:tasks.rados.rados.0.smithi100.stdout:2583: expect (ObjNum 720 snap 221 seq_num 720) 2024-04-07T16:25:17.057 INFO:tasks.rados.rados.0.smithi100.stdout:2582: finishing copy_from to smithi10066867-37 2024-04-07T16:25:17.058 INFO:tasks.rados.rados.0.smithi100.stdout:2582: got expected ENOENT (src dne) 2024-04-07T16:25:17.058 INFO:tasks.rados.rados.0.smithi100.stdout:2582: finishing copy_from racing read to smithi10066867-37 2024-04-07T16:25:17.058 INFO:tasks.rados.rados.0.smithi100.stdout:2584: finishing write tid 2 to smithi10066867-40 2024-04-07T16:25:17.058 INFO:tasks.rados.rados.0.smithi100.stdout:2584: finishing write tid 3 to smithi10066867-40 2024-04-07T16:25:17.058 INFO:tasks.rados.rados.0.smithi100.stdout:2584: finishing write tid 4 to smithi10066867-40 2024-04-07T16:25:17.058 INFO:tasks.rados.rados.0.smithi100.stdout:2584: oid 40 updating version 0 to 1156 2024-04-07T16:25:17.058 INFO:tasks.rados.rados.0.smithi100.stdout:2584: oid 40 updating version 1156 to 1157 2024-04-07T16:25:17.058 INFO:tasks.rados.rados.0.smithi100.stdout:2584: oid 40 updating version 1157 to 1158 2024-04-07T16:25:17.058 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 40 v 1158 (ObjNum 778 snap 246 seq_num 778) dirty exists 2024-04-07T16:25:17.058 INFO:tasks.rados.rados.0.smithi100.stdout:2584: left oid 40 (ObjNum 778 snap 246 seq_num 778) 2024-04-07T16:25:17.058 INFO:tasks.rados.rados.0.smithi100.stdout:2586: expect (ObjNum 733 snap 226 seq_num 733) 2024-04-07T16:25:17.158 INFO:tasks.rados.rados.0.smithi100.stdout:2590: finishing write tid 1 to smithi10066867-9 2024-04-07T16:25:17.158 INFO:tasks.rados.rados.0.smithi100.stdout:2590: finishing write tid 2 to smithi10066867-9 2024-04-07T16:25:17.158 INFO:tasks.rados.rados.0.smithi100.stdout:2590: finishing write tid 3 to smithi10066867-9 2024-04-07T16:25:17.158 INFO:tasks.rados.rados.0.smithi100.stdout:2591: finishing rollback tid 0 to smithi10066867-44 2024-04-07T16:25:17.159 INFO:tasks.rados.rados.0.smithi100.stdout:2590: finishing write tid 4 to smithi10066867-9 2024-04-07T16:25:17.159 INFO:tasks.rados.rados.0.smithi100.stdout:2590: oid 9 updating version 0 to 1112 2024-04-07T16:25:17.159 INFO:tasks.rados.rados.0.smithi100.stdout:2590: oid 9 version 1112 is already newer than 1110 2024-04-07T16:25:17.159 INFO:tasks.rados.rados.0.smithi100.stdout:2590: oid 9 version 1112 is already newer than 1111 2024-04-07T16:25:17.159 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 9 v 1112 (ObjNum 779 snap 246 seq_num 779) dirty exists 2024-04-07T16:25:17.159 INFO:tasks.rados.rados.0.smithi100.stdout:2590: left oid 9 (ObjNum 779 snap 246 seq_num 779) 2024-04-07T16:25:17.159 INFO:tasks.rados.rados.0.smithi100.stdout:2589: expect (ObjNum 720 snap 221 seq_num 720) 2024-04-07T16:25:17.323 INFO:tasks.rados.rados.0.smithi100.stdout:2591: finishing rollback tid 1 to smithi10066867-44 2024-04-07T16:25:17.323 INFO:tasks.rados.rados.0.smithi100.stdout:2591: finishing rollback tid 2 to smithi10066867-44 2024-04-07T16:25:17.323 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 44 v 1172 (ObjNum 509 snap 152 seq_num 509) dirty exists 2024-04-07T16:25:17.323 INFO:tasks.rados.rados.0.smithi100.stdout:2588: finishing copy_from racing read to smithi10066867-30 2024-04-07T16:25:17.323 INFO:tasks.rados.rados.0.smithi100.stdout:2588: finishing copy_from to smithi10066867-30 2024-04-07T16:25:17.323 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 30 v 1173 (ObjNum 722 snap 223 seq_num 722) dirty exists 2024-04-07T16:25:17.323 INFO:tasks.rados.rados.0.smithi100.stdout:2582: done (7 left) 2024-04-07T16:25:17.323 INFO:tasks.rados.rados.0.smithi100.stdout:2583: done (6 left) 2024-04-07T16:25:17.324 INFO:tasks.rados.rados.0.smithi100.stdout:2584: done (5 left) 2024-04-07T16:25:17.324 INFO:tasks.rados.rados.0.smithi100.stdout:2586: done (4 left) 2024-04-07T16:25:17.324 INFO:tasks.rados.rados.0.smithi100.stdout:2588: done (3 left) 2024-04-07T16:25:17.324 INFO:tasks.rados.rados.0.smithi100.stdout:2589: done (2 left) 2024-04-07T16:25:17.324 INFO:tasks.rados.rados.0.smithi100.stdout:2590: done (1 left) 2024-04-07T16:25:17.324 INFO:tasks.rados.rados.0.smithi100.stdout:2591: done (0 left) 2024-04-07T16:25:17.324 INFO:tasks.rados.rados.0.smithi100.stdout:2592: read oid 23 snap -1 2024-04-07T16:25:17.324 INFO:tasks.rados.rados.0.smithi100.stdout:2592: expect deleted 2024-04-07T16:25:17.324 INFO:tasks.rados.rados.0.smithi100.stdout:2593: copy_from oid 29 from oid 7 current snap is 246 2024-04-07T16:25:17.324 INFO:tasks.rados.rados.0.smithi100.stdout:2594: rollback oid 36 current snap is 246 2024-04-07T16:25:17.324 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 36 to 237 2024-04-07T16:25:17.324 INFO:tasks.rados.rados.0.smithi100.stdout:2595: setattr oid 20 current snap is 246 2024-04-07T16:25:17.327 INFO:tasks.rados.rados.0.smithi100.stdout:2592: done (3 left) 2024-04-07T16:25:17.327 INFO:tasks.rados.rados.0.smithi100.stdout:2596: setattr oid 12 current snap is 246 2024-04-07T16:25:17.330 INFO:tasks.rados.rados.0.smithi100.stdout:2594: finishing rollback tid 0 to smithi10066867-36 2024-04-07T16:25:17.333 INFO:tasks.rados.rados.0.smithi100.stdout:2594: finishing rollback tid 1 to smithi10066867-36 2024-04-07T16:25:17.333 INFO:tasks.rados.rados.0.smithi100.stdout:2597: read oid 27 snap -1 2024-04-07T16:25:17.334 INFO:tasks.rados.rados.0.smithi100.stdout:2597: expect (ObjNum 691 snap 211 seq_num 691) 2024-04-07T16:25:17.334 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 11 current snap is 246 2024-04-07T16:25:17.334 INFO:tasks.rados.rados.0.smithi100.stdout:2598: seq_num 780 ranges {32768=671744,704512=401408} 2024-04-07T16:25:17.342 INFO:tasks.rados.rados.0.smithi100.stdout:2598: writing smithi10066867-11 from 32768 to 704512 tid 1 2024-04-07T16:25:17.347 INFO:tasks.rados.rados.0.smithi100.stdout:2598: writing smithi10066867-11 from 704512 to 1105920 tid 2 2024-04-07T16:25:17.347 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 50 current snap is 246 2024-04-07T16:25:17.347 INFO:tasks.rados.rados.0.smithi100.stdout:2599: seq_num 781 ranges {5210112=688128,5898240=401408} 2024-04-07T16:25:17.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:16 smithi144 bash[66654]: cluster 2024-04-07T16:25:15.991841+0000 mon.a (mon.0) 807 : cluster [DBG] osdmap e561: 8 total, 8 up, 8 in 2024-04-07T16:25:17.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:16 smithi144 bash[66654]: audit 2024-04-07T16:25:16.725900+0000 mon.a (mon.0) 808 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:25:17.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:16 smithi144 bash[66654]: audit 2024-04-07T16:25:16.728247+0000 mon.a (mon.0) 809 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:25:17.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:16 smithi144 bash[66654]: cluster 2024-04-07T16:25:16.868584+0000 mon.a (mon.0) 810 : cluster [DBG] osdmap e562: 8 total, 8 up, 8 in 2024-04-07T16:25:17.400 INFO:tasks.rados.rados.0.smithi100.stdout:2599: writing smithi10066867-50 from 5210112 to 5898240 tid 1 2024-04-07T16:25:17.403 INFO:tasks.rados.rados.0.smithi100.stdout:2599: writing smithi10066867-50 from 5898240 to 6299648 tid 2 2024-04-07T16:25:17.403 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 26 current snap is 246 2024-04-07T16:25:17.403 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 20 v 1175 (ObjNum 129 snap 0 seq_num 2702518304) dirty exists 2024-04-07T16:25:17.404 INFO:tasks.rados.rados.0.smithi100.stdout:2594: finishing rollback tid 2 to smithi10066867-36 2024-04-07T16:25:17.404 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 36 v 1127 (ObjNum 715 snap 221 seq_num 715) dirty exists 2024-04-07T16:25:17.404 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 12 v 1242 (ObjNum 748 snap 230 seq_num 748) dirty exists 2024-04-07T16:25:17.404 INFO:tasks.rados.rados.0.smithi100.stdout:2598: finishing write tid 1 to smithi10066867-11 2024-04-07T16:25:17.404 INFO:tasks.rados.rados.0.smithi100.stdout:2593: finishing copy_from racing read to smithi10066867-29 2024-04-07T16:25:17.404 INFO:tasks.rados.rados.0.smithi100.stdout:2598: finishing write tid 2 to smithi10066867-11 2024-04-07T16:25:17.404 INFO:tasks.rados.rados.0.smithi100.stdout:2598: finishing write tid 3 to smithi10066867-11 2024-04-07T16:25:17.404 INFO:tasks.rados.rados.0.smithi100.stdout:2598: finishing write tid 4 to smithi10066867-11 2024-04-07T16:25:17.404 INFO:tasks.rados.rados.0.smithi100.stdout:2598: oid 11 updating version 0 to 1461 2024-04-07T16:25:17.404 INFO:tasks.rados.rados.0.smithi100.stdout:2598: oid 11 updating version 1461 to 1462 2024-04-07T16:25:17.404 INFO:tasks.rados.rados.0.smithi100.stdout:2598: oid 11 updating version 1462 to 1463 2024-04-07T16:25:17.404 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 11 v 1463 (ObjNum 780 snap 246 seq_num 780) dirty exists 2024-04-07T16:25:17.404 INFO:tasks.rados.rados.0.smithi100.stdout:2598: left oid 11 (ObjNum 780 snap 246 seq_num 780) 2024-04-07T16:25:17.404 INFO:tasks.rados.rados.0.smithi100.stdout:2593: finishing copy_from to smithi10066867-29 2024-04-07T16:25:17.405 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 29 v 1116 (ObjNum 752 snap 234 seq_num 752) dirty exists 2024-04-07T16:25:17.405 INFO:tasks.rados.rados.0.smithi100.stdout:2597: expect (ObjNum 691 snap 211 seq_num 691) 2024-04-07T16:25:17.434 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:17 smithi100 bash[76202]: cluster 2024-04-07T16:25:15.991841+0000 mon.a (mon.0) 807 : cluster [DBG] osdmap e561: 8 total, 8 up, 8 in 2024-04-07T16:25:17.434 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:17 smithi100 bash[76202]: audit 2024-04-07T16:25:16.725900+0000 mon.a (mon.0) 808 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:25:17.434 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:17 smithi100 bash[76202]: audit 2024-04-07T16:25:16.728247+0000 mon.a (mon.0) 809 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:25:17.434 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:17 smithi100 bash[76202]: cluster 2024-04-07T16:25:16.868584+0000 mon.a (mon.0) 810 : cluster [DBG] osdmap e562: 8 total, 8 up, 8 in 2024-04-07T16:25:17.435 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:17 smithi100 bash[74942]: cluster 2024-04-07T16:25:15.991841+0000 mon.a (mon.0) 807 : cluster [DBG] osdmap e561: 8 total, 8 up, 8 in 2024-04-07T16:25:17.435 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:17 smithi100 bash[74942]: audit 2024-04-07T16:25:16.725900+0000 mon.a (mon.0) 808 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:25:17.435 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:17 smithi100 bash[74942]: audit 2024-04-07T16:25:16.728247+0000 mon.a (mon.0) 809 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:25:17.435 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:17 smithi100 bash[74942]: cluster 2024-04-07T16:25:16.868584+0000 mon.a (mon.0) 810 : cluster [DBG] osdmap e562: 8 total, 8 up, 8 in 2024-04-07T16:25:17.927 INFO:tasks.rados.rados.0.smithi100.stdout:2599: finishing write tid 1 to smithi10066867-50 2024-04-07T16:25:17.927 INFO:tasks.rados.rados.0.smithi100.stdout:2599: finishing write tid 2 to smithi10066867-50 2024-04-07T16:25:17.927 INFO:tasks.rados.rados.0.smithi100.stdout:2599: finishing write tid 3 to smithi10066867-50 2024-04-07T16:25:17.927 INFO:tasks.rados.rados.0.smithi100.stdout:2600: seq_num 782 ranges {950272=704512,1654784=401408} 2024-04-07T16:25:17.939 INFO:tasks.rados.rados.0.smithi100.stdout:2600: writing smithi10066867-26 from 950272 to 1654784 tid 1 2024-04-07T16:25:17.946 INFO:tasks.rados.rados.0.smithi100.stdout:2600: writing smithi10066867-26 from 1654784 to 2056192 tid 2 2024-04-07T16:25:17.946 INFO:tasks.rados.rados.0.smithi100.stdout:2593: done (7 left) 2024-04-07T16:25:17.946 INFO:tasks.rados.rados.0.smithi100.stdout:2594: done (6 left) 2024-04-07T16:25:17.946 INFO:tasks.rados.rados.0.smithi100.stdout:2595: done (5 left) 2024-04-07T16:25:17.946 INFO:tasks.rados.rados.0.smithi100.stdout:2596: done (4 left) 2024-04-07T16:25:17.946 INFO:tasks.rados.rados.0.smithi100.stdout:2597: done (3 left) 2024-04-07T16:25:17.946 INFO:tasks.rados.rados.0.smithi100.stdout:2598: done (2 left) 2024-04-07T16:25:17.946 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 49 current snap is 246 2024-04-07T16:25:17.946 INFO:tasks.rados.rados.0.smithi100.stdout:2601: seq_num 783 ranges {4751360=720896,5472256=401408} 2024-04-07T16:25:17.984 INFO:tasks.rados.rados.0.smithi100.stdout:2601: writing smithi10066867-49 from 4751360 to 5472256 tid 1 2024-04-07T16:25:17.987 INFO:tasks.rados.rados.0.smithi100.stdout:2601: writing smithi10066867-49 from 5472256 to 5873664 tid 2 2024-04-07T16:25:17.987 INFO:tasks.rados.rados.0.smithi100.stdout:2602: read oid 43 snap 245 2024-04-07T16:25:17.988 INFO:tasks.rados.rados.0.smithi100.stdout:2602: expect (ObjNum 691 snap 211 seq_num 691) 2024-04-07T16:25:17.988 INFO:tasks.rados.rados.0.smithi100.stdout:2603: rollback oid 31 current snap is 246 2024-04-07T16:25:17.988 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 31 to 239 2024-04-07T16:25:17.988 INFO:tasks.rados.rados.0.smithi100.stdout:2604: snap_remove snap 228 2024-04-07T16:25:18.236 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:17 smithi144 bash[66654]: cluster 2024-04-07T16:25:16.726046+0000 mgr.y (mgr.25175) 249 : cluster [DBG] pgmap v434: 121 pgs: 121 active+clean; 191 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 0 B/s rd, 188 KiB/s wr, 5 op/s; 430 KiB/s, 0 objects/s recovering 2024-04-07T16:25:18.417 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:18 smithi100 bash[74942]: cluster 2024-04-07T16:25:16.726046+0000 mgr.y (mgr.25175) 249 : cluster [DBG] pgmap v434: 121 pgs: 121 active+clean; 191 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 0 B/s rd, 188 KiB/s wr, 5 op/s; 430 KiB/s, 0 objects/s recovering 2024-04-07T16:25:18.418 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:18 smithi100 bash[76202]: cluster 2024-04-07T16:25:16.726046+0000 mgr.y (mgr.25175) 249 : cluster [DBG] pgmap v434: 121 pgs: 121 active+clean; 191 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 0 B/s rd, 188 KiB/s wr, 5 op/s; 430 KiB/s, 0 objects/s recovering 2024-04-07T16:25:18.684 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:25:18 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:25:18] "GET /metrics HTTP/1.1" 200 36279 "" "Prometheus/2.43.0" 2024-04-07T16:25:19.002 INFO:tasks.rados.rados.0.smithi100.stdout:2604: done (5 left) 2024-04-07T16:25:19.002 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 25 current snap is 246 2024-04-07T16:25:19.002 INFO:tasks.rados.rados.0.smithi100.stdout:2599: finishing write tid 4 to smithi10066867-50 2024-04-07T16:25:19.002 INFO:tasks.rados.rados.0.smithi100.stdout:2599: oid 50 updating version 0 to 1033 2024-04-07T16:25:19.002 INFO:tasks.rados.rados.0.smithi100.stdout:2599: oid 50 version 1033 is already newer than 1031 2024-04-07T16:25:19.002 INFO:tasks.rados.rados.0.smithi100.stdout:2599: oid 50 version 1033 is already newer than 1032 2024-04-07T16:25:19.002 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 50 v 1033 (ObjNum 781 snap 246 seq_num 781) dirty exists 2024-04-07T16:25:19.002 INFO:tasks.rados.rados.0.smithi100.stdout:2599: left oid 50 (ObjNum 781 snap 246 seq_num 781) 2024-04-07T16:25:19.003 INFO:tasks.rados.rados.0.smithi100.stdout:2605: seq_num 784 ranges {5267456=737280,6004736=401408} 2024-04-07T16:25:19.054 INFO:tasks.rados.rados.0.smithi100.stdout:2605: writing smithi10066867-25 from 5267456 to 6004736 tid 1 2024-04-07T16:25:19.058 INFO:tasks.rados.rados.0.smithi100.stdout:2605: writing smithi10066867-25 from 6004736 to 6406144 tid 2 2024-04-07T16:25:19.058 INFO:tasks.rados.rados.0.smithi100.stdout:2599: done (5 left) 2024-04-07T16:25:19.058 INFO:tasks.rados.rados.0.smithi100.stdout:2606: snap_remove snap 235 2024-04-07T16:25:20.007 INFO:tasks.rados.rados.0.smithi100.stdout:2606: done (5 left) 2024-04-07T16:25:20.007 INFO:tasks.rados.rados.0.smithi100.stdout:2607: snap_remove snap 243 2024-04-07T16:25:20.082 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:19 smithi144 bash[66654]: cluster 2024-04-07T16:25:18.727229+0000 mgr.y (mgr.25175) 250 : cluster [DBG] pgmap v436: 121 pgs: 121 active+clean; 196 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 5.0 MiB/s rd, 2.5 MiB/s wr, 13 op/s 2024-04-07T16:25:20.082 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:19 smithi144 bash[66654]: audit 2024-04-07T16:25:18.797458+0000 mon.a (mon.0) 811 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:25:20.082 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:19 smithi144 bash[66654]: audit 2024-04-07T16:25:18.811135+0000 mon.a (mon.0) 812 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:25:20.082 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:19 smithi144 bash[66654]: audit 2024-04-07T16:25:18.813813+0000 mon.a (mon.0) 813 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:25:20.082 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:19 smithi144 bash[66654]: audit 2024-04-07T16:25:18.815646+0000 mon.a (mon.0) 814 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:25:20.082 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:19 smithi144 bash[66654]: audit 2024-04-07T16:25:18.825821+0000 mon.a (mon.0) 815 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:25:20.082 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:19 smithi144 bash[66654]: audit 2024-04-07T16:25:18.921047+0000 mon.a (mon.0) 816 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:25:20.082 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:19 smithi144 bash[66654]: audit 2024-04-07T16:25:18.926363+0000 mon.a (mon.0) 817 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:25:20.082 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:19 smithi144 bash[66654]: audit 2024-04-07T16:25:18.930549+0000 mon.a (mon.0) 818 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:25:20.082 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:19 smithi144 bash[66654]: audit 2024-04-07T16:25:18.933498+0000 mon.a (mon.0) 819 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:25:20.083 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:19 smithi144 bash[66654]: audit 2024-04-07T16:25:18.937609+0000 mon.a (mon.0) 820 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd ok-to-stop", "ids": ["7"], "max": 16}]: dispatch 2024-04-07T16:25:20.083 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:19 smithi144 bash[66654]: audit 2024-04-07T16:25:18.937941+0000 mgr.y (mgr.25175) 251 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["7"], "max": 16}]: dispatch 2024-04-07T16:25:20.083 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:19 smithi144 bash[66654]: cephadm 2024-04-07T16:25:18.940219+0000 mgr.y (mgr.25175) 252 : cephadm [INF] Upgrade: osd.7 is safe to restart 2024-04-07T16:25:20.083 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:19 smithi144 bash[66654]: cluster 2024-04-07T16:25:19.018468+0000 mon.a (mon.0) 821 : cluster [DBG] osdmap e563: 8 total, 8 up, 8 in 2024-04-07T16:25:20.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:19 smithi100 bash[74942]: cluster 2024-04-07T16:25:18.727229+0000 mgr.y (mgr.25175) 250 : cluster [DBG] pgmap v436: 121 pgs: 121 active+clean; 196 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 5.0 MiB/s rd, 2.5 MiB/s wr, 13 op/s 2024-04-07T16:25:20.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:19 smithi100 bash[74942]: audit 2024-04-07T16:25:18.797458+0000 mon.a (mon.0) 811 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:25:20.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:19 smithi100 bash[74942]: audit 2024-04-07T16:25:18.811135+0000 mon.a (mon.0) 812 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:25:20.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:19 smithi100 bash[74942]: audit 2024-04-07T16:25:18.813813+0000 mon.a (mon.0) 813 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:25:20.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:19 smithi100 bash[74942]: audit 2024-04-07T16:25:18.815646+0000 mon.a (mon.0) 814 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:25:20.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:19 smithi100 bash[74942]: audit 2024-04-07T16:25:18.825821+0000 mon.a (mon.0) 815 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:25:20.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:19 smithi100 bash[74942]: audit 2024-04-07T16:25:18.921047+0000 mon.a (mon.0) 816 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:25:20.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:19 smithi100 bash[74942]: audit 2024-04-07T16:25:18.926363+0000 mon.a (mon.0) 817 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:25:20.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:19 smithi100 bash[74942]: audit 2024-04-07T16:25:18.930549+0000 mon.a (mon.0) 818 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:25:20.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:19 smithi100 bash[74942]: audit 2024-04-07T16:25:18.933498+0000 mon.a (mon.0) 819 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:25:20.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:19 smithi100 bash[74942]: audit 2024-04-07T16:25:18.937609+0000 mon.a (mon.0) 820 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd ok-to-stop", "ids": ["7"], "max": 16}]: dispatch 2024-04-07T16:25:20.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:19 smithi100 bash[74942]: audit 2024-04-07T16:25:18.937941+0000 mgr.y (mgr.25175) 251 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["7"], "max": 16}]: dispatch 2024-04-07T16:25:20.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:19 smithi100 bash[74942]: cephadm 2024-04-07T16:25:18.940219+0000 mgr.y (mgr.25175) 252 : cephadm [INF] Upgrade: osd.7 is safe to restart 2024-04-07T16:25:20.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:19 smithi100 bash[74942]: cluster 2024-04-07T16:25:19.018468+0000 mon.a (mon.0) 821 : cluster [DBG] osdmap e563: 8 total, 8 up, 8 in 2024-04-07T16:25:20.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:19 smithi100 bash[76202]: cluster 2024-04-07T16:25:18.727229+0000 mgr.y (mgr.25175) 250 : cluster [DBG] pgmap v436: 121 pgs: 121 active+clean; 196 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 5.0 MiB/s rd, 2.5 MiB/s wr, 13 op/s 2024-04-07T16:25:20.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:19 smithi100 bash[76202]: audit 2024-04-07T16:25:18.797458+0000 mon.a (mon.0) 811 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:25:20.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:19 smithi100 bash[76202]: audit 2024-04-07T16:25:18.811135+0000 mon.a (mon.0) 812 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:25:20.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:19 smithi100 bash[76202]: audit 2024-04-07T16:25:18.813813+0000 mon.a (mon.0) 813 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:25:20.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:19 smithi100 bash[76202]: audit 2024-04-07T16:25:18.815646+0000 mon.a (mon.0) 814 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:25:20.188 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:19 smithi100 bash[76202]: audit 2024-04-07T16:25:18.825821+0000 mon.a (mon.0) 815 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:25:20.188 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:19 smithi100 bash[76202]: audit 2024-04-07T16:25:18.921047+0000 mon.a (mon.0) 816 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:25:20.188 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:19 smithi100 bash[76202]: audit 2024-04-07T16:25:18.926363+0000 mon.a (mon.0) 817 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:25:20.188 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:19 smithi100 bash[76202]: audit 2024-04-07T16:25:18.930549+0000 mon.a (mon.0) 818 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:25:20.188 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:19 smithi100 bash[76202]: audit 2024-04-07T16:25:18.933498+0000 mon.a (mon.0) 819 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:25:20.188 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:19 smithi100 bash[76202]: audit 2024-04-07T16:25:18.937609+0000 mon.a (mon.0) 820 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd ok-to-stop", "ids": ["7"], "max": 16}]: dispatch 2024-04-07T16:25:20.188 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:19 smithi100 bash[76202]: audit 2024-04-07T16:25:18.937941+0000 mgr.y (mgr.25175) 251 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["7"], "max": 16}]: dispatch 2024-04-07T16:25:20.189 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:19 smithi100 bash[76202]: cephadm 2024-04-07T16:25:18.940219+0000 mgr.y (mgr.25175) 252 : cephadm [INF] Upgrade: osd.7 is safe to restart 2024-04-07T16:25:20.189 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:19 smithi100 bash[76202]: cluster 2024-04-07T16:25:19.018468+0000 mon.a (mon.0) 821 : cluster [DBG] osdmap e563: 8 total, 8 up, 8 in 2024-04-07T16:25:21.026 INFO:tasks.rados.rados.0.smithi100.stdout:2607: done (5 left) 2024-04-07T16:25:21.026 INFO:tasks.rados.rados.0.smithi100.stdout:2608: read oid 29 snap 239 2024-04-07T16:25:21.026 INFO:tasks.rados.rados.0.smithi100.stdout:2608: expect deleted 2024-04-07T16:25:21.026 INFO:tasks.rados.rados.0.smithi100.stdout:2609: snap_create 2024-04-07T16:25:21.026 INFO:tasks.rados.rados.0.smithi100.stdout:2600: finishing write tid 1 to smithi10066867-26 2024-04-07T16:25:21.026 INFO:tasks.rados.rados.0.smithi100.stdout:2600: finishing write tid 2 to smithi10066867-26 2024-04-07T16:25:21.026 INFO:tasks.rados.rados.0.smithi100.stdout:2600: finishing write tid 3 to smithi10066867-26 2024-04-07T16:25:21.026 INFO:tasks.rados.rados.0.smithi100.stdout:2600: finishing write tid 4 to smithi10066867-26 2024-04-07T16:25:21.027 INFO:tasks.rados.rados.0.smithi100.stdout:2600: oid 26 updating version 0 to 1178 2024-04-07T16:25:21.027 INFO:tasks.rados.rados.0.smithi100.stdout:2600: oid 26 version 1178 is already newer than 1176 2024-04-07T16:25:21.027 INFO:tasks.rados.rados.0.smithi100.stdout:2600: oid 26 version 1178 is already newer than 1177 2024-04-07T16:25:21.027 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 26 v 1178 (ObjNum 782 snap 246 seq_num 782) dirty exists 2024-04-07T16:25:21.027 INFO:tasks.rados.rados.0.smithi100.stdout:2600: left oid 26 (ObjNum 782 snap 246 seq_num 782) 2024-04-07T16:25:21.027 INFO:tasks.rados.rados.0.smithi100.stdout:2601: finishing write tid 1 to smithi10066867-49 2024-04-07T16:25:21.027 INFO:tasks.rados.rados.0.smithi100.stdout:2603: finishing rollback tid 0 to smithi10066867-31 2024-04-07T16:25:21.027 INFO:tasks.rados.rados.0.smithi100.stdout:2603: finishing rollback tid 1 to smithi10066867-31 2024-04-07T16:25:21.027 INFO:tasks.rados.rados.0.smithi100.stdout:2603: finishing rollback tid 2 to smithi10066867-31 2024-04-07T16:25:21.027 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 31 v 1109 (ObjNum 579 snap 180 seq_num 579) dirty exists 2024-04-07T16:25:21.027 INFO:tasks.rados.rados.0.smithi100.stdout:2601: finishing write tid 2 to smithi10066867-49 2024-04-07T16:25:21.027 INFO:tasks.rados.rados.0.smithi100.stdout:2601: finishing write tid 3 to smithi10066867-49 2024-04-07T16:25:21.027 INFO:tasks.rados.rados.0.smithi100.stdout:2601: finishing write tid 4 to smithi10066867-49 2024-04-07T16:25:21.027 INFO:tasks.rados.rados.0.smithi100.stdout:2601: oid 49 updating version 0 to 1246 2024-04-07T16:25:21.028 INFO:tasks.rados.rados.0.smithi100.stdout:2601: oid 49 version 1246 is already newer than 1244 2024-04-07T16:25:21.028 INFO:tasks.rados.rados.0.smithi100.stdout:2601: oid 49 version 1246 is already newer than 1245 2024-04-07T16:25:21.028 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 49 v 1246 (ObjNum 783 snap 246 seq_num 783) dirty exists 2024-04-07T16:25:21.028 INFO:tasks.rados.rados.0.smithi100.stdout:2601: left oid 49 (ObjNum 783 snap 246 seq_num 783) 2024-04-07T16:25:21.028 INFO:tasks.rados.rados.0.smithi100.stdout:2602: expect (ObjNum 691 snap 211 seq_num 691) 2024-04-07T16:25:21.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:21 smithi144 bash[66654]: cluster 2024-04-07T16:25:20.022118+0000 mon.a (mon.0) 822 : cluster [DBG] osdmap e564: 8 total, 8 up, 8 in 2024-04-07T16:25:21.434 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:21 smithi100 bash[76202]: cluster 2024-04-07T16:25:20.022118+0000 mon.a (mon.0) 822 : cluster [DBG] osdmap e564: 8 total, 8 up, 8 in 2024-04-07T16:25:21.435 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:21 smithi100 bash[74942]: cluster 2024-04-07T16:25:20.022118+0000 mon.a (mon.0) 822 : cluster [DBG] osdmap e564: 8 total, 8 up, 8 in 2024-04-07T16:25:21.589 INFO:tasks.rados.rados.0.smithi100.stdout:2605: finishing write tid 1 to smithi10066867-25 2024-04-07T16:25:21.589 INFO:tasks.rados.rados.0.smithi100.stdout:2605: finishing write tid 2 to smithi10066867-25 2024-04-07T16:25:21.589 INFO:tasks.rados.rados.0.smithi100.stdout:2605: finishing write tid 3 to smithi10066867-25 2024-04-07T16:25:21.589 INFO:tasks.rados.rados.0.smithi100.stdout:2605: finishing write tid 4 to smithi10066867-25 2024-04-07T16:25:21.590 INFO:tasks.rados.rados.0.smithi100.stdout:2605: oid 25 updating version 0 to 1179 2024-04-07T16:25:21.590 INFO:tasks.rados.rados.0.smithi100.stdout:2605: oid 25 updating version 1179 to 1180 2024-04-07T16:25:21.590 INFO:tasks.rados.rados.0.smithi100.stdout:2605: oid 25 updating version 1180 to 1181 2024-04-07T16:25:21.590 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 25 v 1181 (ObjNum 784 snap 246 seq_num 784) dirty exists 2024-04-07T16:25:21.590 INFO:tasks.rados.rados.0.smithi100.stdout:2605: left oid 25 (ObjNum 784 snap 246 seq_num 784) 2024-04-07T16:25:21.852 INFO:tasks.rados.rados.0.smithi100.stdout:2600: done (6 left) 2024-04-07T16:25:21.852 INFO:tasks.rados.rados.0.smithi100.stdout:2601: done (5 left) 2024-04-07T16:25:21.852 INFO:tasks.rados.rados.0.smithi100.stdout:2602: done (4 left) 2024-04-07T16:25:21.852 INFO:tasks.rados.rados.0.smithi100.stdout:2603: done (3 left) 2024-04-07T16:25:21.853 INFO:tasks.rados.rados.0.smithi100.stdout:2605: done (2 left) 2024-04-07T16:25:21.853 INFO:tasks.rados.rados.0.smithi100.stdout:2608: done (1 left) 2024-04-07T16:25:21.853 INFO:tasks.rados.rados.0.smithi100.stdout:2609: done (0 left) 2024-04-07T16:25:21.853 INFO:tasks.rados.rados.0.smithi100.stdout:2610: setattr oid 44 current snap is 247 2024-04-07T16:25:21.857 INFO:tasks.rados.rados.0.smithi100.stdout:2611: rollback oid 17 current snap is 247 2024-04-07T16:25:21.857 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 17 to 242 2024-04-07T16:25:21.857 INFO:tasks.rados.rados.0.smithi100.stdout:2612: snap_remove snap 237 2024-04-07T16:25:22.289 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:22 smithi144 bash[66654]: cluster 2024-04-07T16:25:20.728037+0000 mgr.y (mgr.25175) 253 : cluster [DBG] pgmap v439: 121 pgs: 121 active+clean; 177 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 16 MiB/s rd, 3.1 MiB/s wr, 20 op/s 2024-04-07T16:25:22.289 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:22 smithi144 bash[66654]: cephadm 2024-04-07T16:25:21.010051+0000 mgr.y (mgr.25175) 254 : cephadm [INF] Upgrade: Updating osd.7 2024-04-07T16:25:22.290 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:22 smithi144 bash[66654]: audit 2024-04-07T16:25:21.017683+0000 mon.a (mon.0) 823 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:25:22.290 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:22 smithi144 bash[66654]: audit 2024-04-07T16:25:21.027935+0000 mon.a (mon.0) 824 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-07T16:25:22.290 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:22 smithi144 bash[66654]: audit 2024-04-07T16:25:21.029797+0000 mon.a (mon.0) 825 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:25:22.290 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:22 smithi144 bash[66654]: cluster 2024-04-07T16:25:21.038810+0000 mon.a (mon.0) 826 : cluster [DBG] osdmap e565: 8 total, 8 up, 8 in 2024-04-07T16:25:22.290 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:22 smithi144 bash[66654]: cephadm 2024-04-07T16:25:21.050812+0000 mgr.y (mgr.25175) 255 : cephadm [INF] Deploying daemon osd.7 on smithi144 2024-04-07T16:25:22.290 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:22 smithi144 bash[66654]: cluster 2024-04-07T16:25:21.869415+0000 mon.a (mon.0) 827 : cluster [DBG] osdmap e566: 8 total, 8 up, 8 in 2024-04-07T16:25:22.290 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:22 smithi144 bash[66654]: audit 2024-04-07T16:25:21.968451+0000 mon.a (mon.0) 828 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:25:22.435 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:22 smithi100 bash[74942]: cluster 2024-04-07T16:25:20.728037+0000 mgr.y (mgr.25175) 253 : cluster [DBG] pgmap v439: 121 pgs: 121 active+clean; 177 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 16 MiB/s rd, 3.1 MiB/s wr, 20 op/s 2024-04-07T16:25:22.435 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:22 smithi100 bash[74942]: cephadm 2024-04-07T16:25:21.010051+0000 mgr.y (mgr.25175) 254 : cephadm [INF] Upgrade: Updating osd.7 2024-04-07T16:25:22.435 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:22 smithi100 bash[74942]: audit 2024-04-07T16:25:21.017683+0000 mon.a (mon.0) 823 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:25:22.435 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:22 smithi100 bash[74942]: audit 2024-04-07T16:25:21.027935+0000 mon.a (mon.0) 824 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-07T16:25:22.435 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:22 smithi100 bash[74942]: audit 2024-04-07T16:25:21.029797+0000 mon.a (mon.0) 825 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:25:22.435 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:22 smithi100 bash[74942]: cluster 2024-04-07T16:25:21.038810+0000 mon.a (mon.0) 826 : cluster [DBG] osdmap e565: 8 total, 8 up, 8 in 2024-04-07T16:25:22.435 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:22 smithi100 bash[74942]: cephadm 2024-04-07T16:25:21.050812+0000 mgr.y (mgr.25175) 255 : cephadm [INF] Deploying daemon osd.7 on smithi144 2024-04-07T16:25:22.436 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:22 smithi100 bash[74942]: cluster 2024-04-07T16:25:21.869415+0000 mon.a (mon.0) 827 : cluster [DBG] osdmap e566: 8 total, 8 up, 8 in 2024-04-07T16:25:22.436 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:22 smithi100 bash[74942]: audit 2024-04-07T16:25:21.968451+0000 mon.a (mon.0) 828 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:25:22.436 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:22 smithi100 bash[76202]: cluster 2024-04-07T16:25:20.728037+0000 mgr.y (mgr.25175) 253 : cluster [DBG] pgmap v439: 121 pgs: 121 active+clean; 177 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 16 MiB/s rd, 3.1 MiB/s wr, 20 op/s 2024-04-07T16:25:22.436 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:22 smithi100 bash[76202]: cephadm 2024-04-07T16:25:21.010051+0000 mgr.y (mgr.25175) 254 : cephadm [INF] Upgrade: Updating osd.7 2024-04-07T16:25:22.436 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:22 smithi100 bash[76202]: audit 2024-04-07T16:25:21.017683+0000 mon.a (mon.0) 823 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:25:22.436 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:22 smithi100 bash[76202]: audit 2024-04-07T16:25:21.027935+0000 mon.a (mon.0) 824 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-07T16:25:22.436 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:22 smithi100 bash[76202]: audit 2024-04-07T16:25:21.029797+0000 mon.a (mon.0) 825 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:25:22.436 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:22 smithi100 bash[76202]: cluster 2024-04-07T16:25:21.038810+0000 mon.a (mon.0) 826 : cluster [DBG] osdmap e565: 8 total, 8 up, 8 in 2024-04-07T16:25:22.436 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:22 smithi100 bash[76202]: cephadm 2024-04-07T16:25:21.050812+0000 mgr.y (mgr.25175) 255 : cephadm [INF] Deploying daemon osd.7 on smithi144 2024-04-07T16:25:22.436 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:22 smithi100 bash[76202]: cluster 2024-04-07T16:25:21.869415+0000 mon.a (mon.0) 827 : cluster [DBG] osdmap e566: 8 total, 8 up, 8 in 2024-04-07T16:25:22.436 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:22 smithi100 bash[76202]: audit 2024-04-07T16:25:21.968451+0000 mon.a (mon.0) 828 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:25:22.606 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:25:22 smithi144 bash[59794]: ts=2024-04-07T16:25:22.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:25:22.858 INFO:tasks.rados.rados.0.smithi100.stdout:2612: done (2 left) 2024-04-07T16:25:22.858 INFO:tasks.rados.rados.0.smithi100.stdout:2613: delete oid 18 current snap is 247 2024-04-07T16:25:22.858 INFO:tasks.rados.rados.0.smithi100.stdout:2611: finishing rollback tid 0 to smithi10066867-17 2024-04-07T16:25:22.858 INFO:tasks.rados.rados.0.smithi100.stdout:2611: finishing rollback tid 1 to smithi10066867-17 2024-04-07T16:25:22.862 INFO:tasks.rados.rados.0.smithi100.stdout:2611: finishing rollback tid 2 to smithi10066867-17 2024-04-07T16:25:22.862 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 17 v 1130 (ObjNum 737 snap 227 seq_num 737) dirty exists 2024-04-07T16:25:22.862 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 44 v 1188 (ObjNum 509 snap 152 seq_num 509) dirty exists 2024-04-07T16:25:22.877 INFO:tasks.rados.rados.0.smithi100.stdout:2610: done (2 left) 2024-04-07T16:25:22.877 INFO:tasks.rados.rados.0.smithi100.stdout:2611: done (1 left) 2024-04-07T16:25:22.877 INFO:tasks.rados.rados.0.smithi100.stdout:2613: done (0 left) 2024-04-07T16:25:22.877 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 19 current snap is 247 2024-04-07T16:25:22.877 INFO:tasks.rados.rados.0.smithi100.stdout:2614: seq_num 786 ranges {0=770048,770048=401408} 2024-04-07T16:25:22.882 INFO:tasks.rados.rados.0.smithi100.stdout:2614: writing smithi10066867-19 from 0 to 770048 tid 1 2024-04-07T16:25:22.884 INFO:tasks.rados.rados.0.smithi100.stdout:2614: writing smithi10066867-19 from 770048 to 1171456 tid 2 2024-04-07T16:25:22.884 INFO:tasks.rados.rados.0.smithi100.stdout:2615: rmattr oid 12 current snap is 247 2024-04-07T16:25:22.886 INFO:tasks.rados.rados.0.smithi100.stdout:2616: read oid 6 snap 239 2024-04-07T16:25:22.886 INFO:tasks.rados.rados.0.smithi100.stdout:2616: expect (ObjNum 706 snap 215 seq_num 706) 2024-04-07T16:25:22.886 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 18 current snap is 247 2024-04-07T16:25:22.886 INFO:tasks.rados.rados.0.smithi100.stdout:2617: seq_num 787 ranges {0=786432,786432=401408} 2024-04-07T16:25:22.891 INFO:tasks.rados.rados.0.smithi100.stdout:2617: writing smithi10066867-18 from 0 to 786432 tid 1 2024-04-07T16:25:22.894 INFO:tasks.rados.rados.0.smithi100.stdout:2617: writing smithi10066867-18 from 786432 to 1187840 tid 2 2024-04-07T16:25:22.894 INFO:tasks.rados.rados.0.smithi100.stdout:2618: read oid 37 snap -1 2024-04-07T16:25:22.894 INFO:tasks.rados.rados.0.smithi100.stdout:2618: expect (ObjNum 759 snap 239 seq_num 759) 2024-04-07T16:25:22.894 INFO:tasks.rados.rados.0.smithi100.stdout:2619: snap_create 2024-04-07T16:25:22.905 INFO:tasks.rados.rados.0.smithi100.stdout:2617: finishing write tid 1 to smithi10066867-18 2024-04-07T16:25:22.905 INFO:tasks.rados.rados.0.smithi100.stdout:2617: finishing write tid 2 to smithi10066867-18 2024-04-07T16:25:22.905 INFO:tasks.rados.rados.0.smithi100.stdout:2614: finishing write tid 1 to smithi10066867-19 2024-04-07T16:25:22.909 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 12 v 1276 (ObjNum 748 snap 230 seq_num 748) dirty exists 2024-04-07T16:25:22.921 INFO:tasks.rados.rados.0.smithi100.stdout:2618: expect (ObjNum 759 snap 239 seq_num 759) 2024-04-07T16:25:22.943 INFO:tasks.rados.rados.0.smithi100.stdout:2614: finishing write tid 2 to smithi10066867-19 2024-04-07T16:25:22.943 INFO:tasks.rados.rados.0.smithi100.stdout:2614: finishing write tid 3 to smithi10066867-19 2024-04-07T16:25:22.944 INFO:tasks.rados.rados.0.smithi100.stdout:2617: finishing write tid 3 to smithi10066867-18 2024-04-07T16:25:22.944 INFO:tasks.rados.rados.0.smithi100.stdout:2614: finishing write tid 4 to smithi10066867-19 2024-04-07T16:25:22.944 INFO:tasks.rados.rados.0.smithi100.stdout:2614: oid 19 updating version 0 to 1131 2024-04-07T16:25:22.944 INFO:tasks.rados.rados.0.smithi100.stdout:2614: oid 19 updating version 1131 to 1132 2024-04-07T16:25:22.944 INFO:tasks.rados.rados.0.smithi100.stdout:2614: oid 19 updating version 1132 to 1133 2024-04-07T16:25:22.944 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 19 v 1133 (ObjNum 786 snap 247 seq_num 786) dirty exists 2024-04-07T16:25:22.944 INFO:tasks.rados.rados.0.smithi100.stdout:2614: left oid 19 (ObjNum 786 snap 247 seq_num 786) 2024-04-07T16:25:22.944 INFO:tasks.rados.rados.0.smithi100.stdout:2617: finishing write tid 4 to smithi10066867-18 2024-04-07T16:25:22.944 INFO:tasks.rados.rados.0.smithi100.stdout:2617: oid 18 updating version 0 to 1486 2024-04-07T16:25:22.944 INFO:tasks.rados.rados.0.smithi100.stdout:2617: oid 18 updating version 1486 to 1487 2024-04-07T16:25:22.945 INFO:tasks.rados.rados.0.smithi100.stdout:2617: oid 18 updating version 1487 to 1488 2024-04-07T16:25:22.945 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 18 v 1488 (ObjNum 787 snap 247 seq_num 787) dirty exists 2024-04-07T16:25:22.945 INFO:tasks.rados.rados.0.smithi100.stdout:2617: left oid 18 (ObjNum 787 snap 247 seq_num 787) 2024-04-07T16:25:22.945 INFO:tasks.rados.rados.0.smithi100.stdout:2616: expect (ObjNum 706 snap 215 seq_num 706) 2024-04-07T16:25:23.868 INFO:tasks.rados.rados.0.smithi100.stdout:2614: done (5 left) 2024-04-07T16:25:23.868 INFO:tasks.rados.rados.0.smithi100.stdout:2615: done (4 left) 2024-04-07T16:25:23.868 INFO:tasks.rados.rados.0.smithi100.stdout:2616: done (3 left) 2024-04-07T16:25:23.868 INFO:tasks.rados.rados.0.smithi100.stdout:2617: done (2 left) 2024-04-07T16:25:23.868 INFO:tasks.rados.rados.0.smithi100.stdout:2618: done (1 left) 2024-04-07T16:25:23.868 INFO:tasks.rados.rados.0.smithi100.stdout:2619: done (0 left) 2024-04-07T16:25:23.868 INFO:tasks.rados.rados.0.smithi100.stdout:2620: rmattr oid 7 current snap is 248 2024-04-07T16:25:23.868 INFO:tasks.rados.rados.0.smithi100.stdout:2621: read oid 45 snap -1 2024-04-07T16:25:23.868 INFO:tasks.rados.rados.0.smithi100.stdout:2621: expect deleted 2024-04-07T16:25:23.869 INFO:tasks.rados.rados.0.smithi100.stdout:2622: read oid 15 snap -1 2024-04-07T16:25:23.869 INFO:tasks.rados.rados.0.smithi100.stdout:2622: expect (ObjNum 776 snap 243 seq_num 776) 2024-04-07T16:25:23.869 INFO:tasks.rados.rados.0.smithi100.stdout:2623: snap_remove snap 245 2024-04-07T16:25:24.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:23 smithi144 bash[66654]: cluster 2024-04-07T16:25:22.729140+0000 mgr.y (mgr.25175) 256 : cluster [DBG] pgmap v442: 121 pgs: 121 active+clean; 173 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 13 MiB/s rd, 1.0 MiB/s wr, 13 op/s 2024-04-07T16:25:24.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:23 smithi144 bash[66654]: cluster 2024-04-07T16:25:22.873388+0000 mon.a (mon.0) 829 : cluster [DBG] osdmap e567: 8 total, 8 up, 8 in 2024-04-07T16:25:24.184 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:23 smithi100 bash[74942]: cluster 2024-04-07T16:25:22.729140+0000 mgr.y (mgr.25175) 256 : cluster [DBG] pgmap v442: 121 pgs: 121 active+clean; 173 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 13 MiB/s rd, 1.0 MiB/s wr, 13 op/s 2024-04-07T16:25:24.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:23 smithi100 bash[74942]: cluster 2024-04-07T16:25:22.873388+0000 mon.a (mon.0) 829 : cluster [DBG] osdmap e567: 8 total, 8 up, 8 in 2024-04-07T16:25:24.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:23 smithi100 bash[76202]: cluster 2024-04-07T16:25:22.729140+0000 mgr.y (mgr.25175) 256 : cluster [DBG] pgmap v442: 121 pgs: 121 active+clean; 173 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 13 MiB/s rd, 1.0 MiB/s wr, 13 op/s 2024-04-07T16:25:24.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:23 smithi100 bash[76202]: cluster 2024-04-07T16:25:22.873388+0000 mon.a (mon.0) 829 : cluster [DBG] osdmap e567: 8 total, 8 up, 8 in 2024-04-07T16:25:24.876 INFO:tasks.rados.rados.0.smithi100.stdout:2623: done (3 left) 2024-04-07T16:25:24.876 INFO:tasks.rados.rados.0.smithi100.stdout:2624: delete oid 16 current snap is 248 2024-04-07T16:25:24.876 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 7 v 1117 (ObjNum 752 snap 234 seq_num 752) dirty exists 2024-04-07T16:25:24.876 INFO:tasks.rados.rados.0.smithi100.stdout:2622: expect (ObjNum 776 snap 243 seq_num 776) 2024-04-07T16:25:25.077 INFO:tasks.rados.rados.0.smithi100.stdout:2620: done (3 left) 2024-04-07T16:25:25.077 INFO:tasks.rados.rados.0.smithi100.stdout:2621: done (2 left) 2024-04-07T16:25:25.077 INFO:tasks.rados.rados.0.smithi100.stdout:2622: done (1 left) 2024-04-07T16:25:25.077 INFO:tasks.rados.rados.0.smithi100.stdout:2624: done (0 left) 2024-04-07T16:25:25.077 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 29 current snap is 248 2024-04-07T16:25:25.077 INFO:tasks.rados.rados.0.smithi100.stdout:2625: seq_num 789 ranges {6635520=417792,7053312=663552,7716864=139264} 2024-04-07T16:25:25.166 INFO:tasks.rados.rados.0.smithi100.stdout:2625: writing smithi10066867-29 from 6635520 to 7053312 tid 1 2024-04-07T16:25:25.171 INFO:tasks.rados.rados.0.smithi100.stdout:2625: writing smithi10066867-29 from 7053312 to 7716864 tid 2 2024-04-07T16:25:25.172 INFO:tasks.rados.rados.0.smithi100.stdout:2625: writing smithi10066867-29 from 7716864 to 7856128 tid 3 2024-04-07T16:25:25.172 INFO:tasks.rados.rados.0.smithi100.stdout:2626: copy_from oid 21 from oid 20 current snap is 248 2024-04-07T16:25:25.172 INFO:tasks.rados.rados.0.smithi100.stdout:2627: snap_create 2024-04-07T16:25:25.173 INFO:tasks.rados.rados.0.smithi100.stdout:2625: finishing write tid 1 to smithi10066867-29 2024-04-07T16:25:25.177 INFO:tasks.rados.rados.0.smithi100.stdout:2626: finishing copy_from racing read to smithi10066867-21 2024-04-07T16:25:25.178 INFO:tasks.rados.rados.0.smithi100.stdout:2626: finishing copy_from to smithi10066867-21 2024-04-07T16:25:25.178 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 21 v 1175 (ObjNum 161 snap 0 seq_num 0) dirty exists 2024-04-07T16:25:25.179 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:24 smithi144 bash[66654]: cluster 2024-04-07T16:25:23.883736+0000 mon.a (mon.0) 830 : cluster [DBG] osdmap e568: 8 total, 8 up, 8 in 2024-04-07T16:25:25.181 INFO:tasks.rados.rados.0.smithi100.stdout:2625: finishing write tid 2 to smithi10066867-29 2024-04-07T16:25:25.182 INFO:tasks.rados.rados.0.smithi100.stdout:2625: finishing write tid 3 to smithi10066867-29 2024-04-07T16:25:25.182 INFO:tasks.rados.rados.0.smithi100.stdout:2625: finishing write tid 4 to smithi10066867-29 2024-04-07T16:25:25.184 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:24 smithi100 bash[74942]: cluster 2024-04-07T16:25:23.883736+0000 mon.a (mon.0) 830 : cluster [DBG] osdmap e568: 8 total, 8 up, 8 in 2024-04-07T16:25:25.184 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:24 smithi100 bash[76202]: cluster 2024-04-07T16:25:23.883736+0000 mon.a (mon.0) 830 : cluster [DBG] osdmap e568: 8 total, 8 up, 8 in 2024-04-07T16:25:25.185 INFO:tasks.rados.rados.0.smithi100.stdout:2625: finishing write tid 5 to smithi10066867-29 2024-04-07T16:25:25.186 INFO:tasks.rados.rados.0.smithi100.stdout:2625: oid 29 updating version 0 to 1175 2024-04-07T16:25:25.186 INFO:tasks.rados.rados.0.smithi100.stdout:2625: oid 29 updating version 1175 to 1177 2024-04-07T16:25:25.186 INFO:tasks.rados.rados.0.smithi100.stdout:2625: oid 29 version 1177 is already newer than 1174 2024-04-07T16:25:25.186 INFO:tasks.rados.rados.0.smithi100.stdout:2625: oid 29 version 1177 is already newer than 1176 2024-04-07T16:25:25.186 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 29 v 1177 (ObjNum 789 snap 248 seq_num 789) dirty exists 2024-04-07T16:25:25.186 INFO:tasks.rados.rados.0.smithi100.stdout:2625: left oid 29 (ObjNum 789 snap 248 seq_num 789) 2024-04-07T16:25:25.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:25 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:25:25.606 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:25:25 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:25:25.607 INFO:journalctl@ceph.osd.6.smithi144.stdout:Apr 07 16:25:25 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:25:25.607 INFO:journalctl@ceph.mds.b.smithi144.stdout:Apr 07 16:25:25 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:25:25.607 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:25:25 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:25:25.608 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 16:25:25 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:25:25.608 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:25:25 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:25:25.608 INFO:journalctl@ceph.osd.4.smithi144.stdout:Apr 07 16:25:25 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:25:25.609 INFO:journalctl@ceph.osd.5.smithi144.stdout:Apr 07 16:25:25 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:25:25.609 INFO:journalctl@ceph.osd.7.smithi144.stdout:Apr 07 16:25:25 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:25:25.609 INFO:journalctl@ceph.osd.7.smithi144.stdout:Apr 07 16:25:25 smithi144 systemd[1]: Stopping Ceph osd.7 for 0702e3a8-f4f8-11ee-b649-cb9ed24678a4... 2024-04-07T16:25:25.609 INFO:journalctl@ceph.osd.7.smithi144.stdout:Apr 07 16:25:25 smithi144 bash[43919]: debug 2024-04-07T16:25:25.409+0000 7f53866b0700 -1 received signal: Terminated from /sbin/docker-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug (PID: 1) UID: 0 2024-04-07T16:25:25.609 INFO:journalctl@ceph.osd.7.smithi144.stdout:Apr 07 16:25:25 smithi144 bash[43919]: debug 2024-04-07T16:25:25.409+0000 7f53866b0700 -1 osd.7 569 *** Got signal Terminated *** 2024-04-07T16:25:25.609 INFO:journalctl@ceph.osd.7.smithi144.stdout:Apr 07 16:25:25 smithi144 bash[43919]: debug 2024-04-07T16:25:25.409+0000 7f53866b0700 -1 osd.7 569 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-07T16:25:25.900 INFO:tasks.rados.rados.0.smithi100.stdout:2625: done (2 left) 2024-04-07T16:25:25.901 INFO:tasks.rados.rados.0.smithi100.stdout:2626: done (1 left) 2024-04-07T16:25:25.901 INFO:tasks.rados.rados.0.smithi100.stdout:2627: done (0 left) 2024-04-07T16:25:25.901 INFO:tasks.rados.rados.0.smithi100.stdout:2628: setattr oid 10 current snap is 249 2024-04-07T16:25:25.902 INFO:tasks.rados.rados.0.smithi100.stdout:2629: copy_from oid 36 from oid 20 current snap is 249 2024-04-07T16:25:25.902 INFO:tasks.rados.rados.0.smithi100.stdout:2630: copy_from oid 12 from oid 48 current snap is 249 2024-04-07T16:25:25.903 INFO:tasks.rados.rados.0.smithi100.stdout:2631: snap_create 2024-04-07T16:25:25.907 INFO:tasks.rados.rados.0.smithi100.stdout:2630: finishing copy_from to smithi10066867-12 2024-04-07T16:25:25.908 INFO:tasks.rados.rados.0.smithi100.stdout:2630: got expected ENOENT (src dne) 2024-04-07T16:25:25.908 INFO:tasks.rados.rados.0.smithi100.stdout:2630: finishing copy_from racing read to smithi10066867-12 2024-04-07T16:25:25.909 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 10 v 1104 (ObjNum 720 snap 221 seq_num 720) dirty exists 2024-04-07T16:25:25.912 INFO:tasks.rados.rados.0.smithi100.stdout:2629: finishing copy_from racing read to smithi10066867-36 2024-04-07T16:25:25.913 INFO:tasks.rados.rados.0.smithi100.stdout:2629: finishing copy_from to smithi10066867-36 2024-04-07T16:25:25.913 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 36 v 1175 (ObjNum 273 snap 0 seq_num 0) dirty exists 2024-04-07T16:25:26.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:25 smithi100 bash[74942]: cluster 2024-04-07T16:25:24.730105+0000 mgr.y (mgr.25175) 257 : cluster [DBG] pgmap v445: 121 pgs: 121 active+clean; 164 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 3.7 MiB/s rd, 1.4 MiB/s wr, 14 op/s 2024-04-07T16:25:26.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:25 smithi100 bash[74942]: cluster 2024-04-07T16:25:24.893027+0000 mon.a (mon.0) 831 : cluster [DBG] osdmap e569: 8 total, 8 up, 8 in 2024-04-07T16:25:26.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:25 smithi100 bash[74942]: cluster 2024-04-07T16:25:25.422416+0000 mon.a (mon.0) 832 : cluster [INF] osd.7 marked itself down and dead 2024-04-07T16:25:26.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:25 smithi100 bash[76202]: cluster 2024-04-07T16:25:24.730105+0000 mgr.y (mgr.25175) 257 : cluster [DBG] pgmap v445: 121 pgs: 121 active+clean; 164 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 3.7 MiB/s rd, 1.4 MiB/s wr, 14 op/s 2024-04-07T16:25:26.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:25 smithi100 bash[76202]: cluster 2024-04-07T16:25:24.893027+0000 mon.a (mon.0) 831 : cluster [DBG] osdmap e569: 8 total, 8 up, 8 in 2024-04-07T16:25:26.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:25 smithi100 bash[76202]: cluster 2024-04-07T16:25:25.422416+0000 mon.a (mon.0) 832 : cluster [INF] osd.7 marked itself down and dead 2024-04-07T16:25:26.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:25 smithi144 bash[66654]: cluster 2024-04-07T16:25:24.730105+0000 mgr.y (mgr.25175) 257 : cluster [DBG] pgmap v445: 121 pgs: 121 active+clean; 164 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 3.7 MiB/s rd, 1.4 MiB/s wr, 14 op/s 2024-04-07T16:25:26.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:25 smithi144 bash[66654]: cluster 2024-04-07T16:25:24.893027+0000 mon.a (mon.0) 831 : cluster [DBG] osdmap e569: 8 total, 8 up, 8 in 2024-04-07T16:25:26.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:25 smithi144 bash[66654]: cluster 2024-04-07T16:25:25.422416+0000 mon.a (mon.0) 832 : cluster [INF] osd.7 marked itself down and dead 2024-04-07T16:25:26.855 INFO:tasks.rados.rados.0.smithi100.stdout:2628: done (3 left) 2024-04-07T16:25:26.855 INFO:tasks.rados.rados.0.smithi100.stdout:2629: done (2 left) 2024-04-07T16:25:26.855 INFO:tasks.rados.rados.0.smithi100.stdout:2630: done (1 left) 2024-04-07T16:25:26.855 INFO:tasks.rados.rados.0.smithi100.stdout:2631: done (0 left) 2024-04-07T16:25:26.856 INFO:tasks.rados.rados.0.smithi100.stdout:2632: rmattr oid 47 current snap is 250 2024-04-07T16:25:26.856 INFO:tasks.rados.rados.0.smithi100.stdout:2632: done (0 left) 2024-04-07T16:25:26.856 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 50 current snap is 250 2024-04-07T16:25:26.856 INFO:tasks.rados.rados.0.smithi100.stdout:2633: seq_num 790 ranges {6299648=434176,6733824=548864,7282688=253952} 2024-04-07T16:25:26.901 INFO:tasks.rados.rados.0.smithi100.stdout:2633: writing smithi10066867-50 from 6299648 to 6733824 tid 1 2024-04-07T16:25:26.905 INFO:tasks.rados.rados.0.smithi100.stdout:2633: writing smithi10066867-50 from 6733824 to 7282688 tid 2 2024-04-07T16:25:26.907 INFO:tasks.rados.rados.0.smithi100.stdout:2633: writing smithi10066867-50 from 7282688 to 7536640 tid 3 2024-04-07T16:25:26.908 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 37 current snap is 250 2024-04-07T16:25:26.908 INFO:tasks.rados.rados.0.smithi100.stdout:2634: seq_num 791 ranges {770048=450560,1220608=475136,1695744=327680} 2024-04-07T16:25:26.916 INFO:tasks.rados.rados.0.smithi100.stdout:2634: writing smithi10066867-37 from 770048 to 1220608 tid 1 2024-04-07T16:25:26.919 INFO:tasks.rados.rados.0.smithi100.stdout:2634: writing smithi10066867-37 from 1220608 to 1695744 tid 2 2024-04-07T16:25:26.922 INFO:tasks.rados.rados.0.smithi100.stdout:2634: writing smithi10066867-37 from 1695744 to 2023424 tid 3 2024-04-07T16:25:26.922 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 47 current snap is 250 2024-04-07T16:25:26.922 INFO:tasks.rados.rados.0.smithi100.stdout:2633: finishing write tid 1 to smithi10066867-50 2024-04-07T16:25:26.922 INFO:tasks.rados.rados.0.smithi100.stdout:2635: seq_num 792 ranges {0=466944,466944=761856,1228800=40960} 2024-04-07T16:25:26.924 INFO:tasks.rados.rados.0.smithi100.stdout:2635: writing smithi10066867-47 from 0 to 466944 tid 1 2024-04-07T16:25:26.930 INFO:tasks.rados.rados.0.smithi100.stdout:2635: writing smithi10066867-47 from 466944 to 1228800 tid 2 2024-04-07T16:25:26.930 INFO:tasks.rados.rados.0.smithi100.stdout:2635: writing smithi10066867-47 from 1228800 to 1269760 tid 3 2024-04-07T16:25:26.930 INFO:tasks.rados.rados.0.smithi100.stdout:2636: delete oid 44 current snap is 250 2024-04-07T16:25:26.931 INFO:tasks.rados.rados.0.smithi100.stdout:2633: finishing write tid 2 to smithi10066867-50 2024-04-07T16:25:26.931 INFO:tasks.rados.rados.0.smithi100.stdout:2633: finishing write tid 3 to smithi10066867-50 2024-04-07T16:25:26.931 INFO:tasks.rados.rados.0.smithi100.stdout:2633: finishing write tid 4 to smithi10066867-50 2024-04-07T16:25:26.931 INFO:tasks.rados.rados.0.smithi100.stdout:2633: finishing write tid 5 to smithi10066867-50 2024-04-07T16:25:26.931 INFO:tasks.rados.rados.0.smithi100.stdout:2633: oid 50 updating version 0 to 1036 2024-04-07T16:25:26.931 INFO:tasks.rados.rados.0.smithi100.stdout:2633: oid 50 version 1036 is already newer than 1034 2024-04-07T16:25:26.931 INFO:tasks.rados.rados.0.smithi100.stdout:2633: oid 50 version 1036 is already newer than 1035 2024-04-07T16:25:26.931 INFO:tasks.rados.rados.0.smithi100.stdout:2633: oid 50 updating version 1036 to 1037 2024-04-07T16:25:26.931 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 50 v 1037 (ObjNum 790 snap 250 seq_num 790) dirty exists 2024-04-07T16:25:26.931 INFO:tasks.rados.rados.0.smithi100.stdout:2633: left oid 50 (ObjNum 790 snap 250 seq_num 790) 2024-04-07T16:25:26.931 INFO:tasks.rados.rados.0.smithi100.stdout:2634: finishing write tid 1 to smithi10066867-37 2024-04-07T16:25:26.931 INFO:tasks.rados.rados.0.smithi100.stdout:2634: finishing write tid 2 to smithi10066867-37 2024-04-07T16:25:26.931 INFO:tasks.rados.rados.0.smithi100.stdout:2635: finishing write tid 1 to smithi10066867-47 2024-04-07T16:25:26.932 INFO:tasks.rados.rados.0.smithi100.stdout:2634: finishing write tid 3 to smithi10066867-37 2024-04-07T16:25:26.932 INFO:tasks.rados.rados.0.smithi100.stdout:2634: finishing write tid 4 to smithi10066867-37 2024-04-07T16:25:26.934 INFO:tasks.rados.rados.0.smithi100.stdout:2634: finishing write tid 5 to smithi10066867-37 2024-04-07T16:25:26.934 INFO:tasks.rados.rados.0.smithi100.stdout:2634: oid 37 updating version 0 to 1181 2024-04-07T16:25:26.934 INFO:tasks.rados.rados.0.smithi100.stdout:2634: oid 37 version 1181 is already newer than 1178 2024-04-07T16:25:26.934 INFO:tasks.rados.rados.0.smithi100.stdout:2634: oid 37 version 1181 is already newer than 1180 2024-04-07T16:25:26.934 INFO:tasks.rados.rados.0.smithi100.stdout:2634: oid 37 version 1181 is already newer than 1179 2024-04-07T16:25:26.934 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 37 v 1181 (ObjNum 791 snap 250 seq_num 791) dirty exists 2024-04-07T16:25:26.934 INFO:tasks.rados.rados.0.smithi100.stdout:2634: left oid 37 (ObjNum 791 snap 250 seq_num 791) 2024-04-07T16:25:26.938 INFO:tasks.rados.rados.0.smithi100.stdout:2635: finishing write tid 2 to smithi10066867-47 2024-04-07T16:25:26.939 INFO:tasks.rados.rados.0.smithi100.stdout:2635: finishing write tid 3 to smithi10066867-47 2024-04-07T16:25:26.939 INFO:tasks.rados.rados.0.smithi100.stdout:2635: finishing write tid 4 to smithi10066867-47 2024-04-07T16:25:26.941 INFO:tasks.rados.rados.0.smithi100.stdout:2633: done (3 left) 2024-04-07T16:25:26.941 INFO:tasks.rados.rados.0.smithi100.stdout:2634: done (2 left) 2024-04-07T16:25:26.941 INFO:tasks.rados.rados.0.smithi100.stdout:2636: done (1 left) 2024-04-07T16:25:26.941 INFO:tasks.rados.rados.0.smithi100.stdout:2637: snap_create 2024-04-07T16:25:26.946 INFO:tasks.rados.rados.0.smithi100.stdout:2635: finishing write tid 5 to smithi10066867-47 2024-04-07T16:25:26.946 INFO:tasks.rados.rados.0.smithi100.stdout:2635: oid 47 updating version 0 to 1178 2024-04-07T16:25:26.946 INFO:tasks.rados.rados.0.smithi100.stdout:2635: oid 47 version 1178 is already newer than 1177 2024-04-07T16:25:26.946 INFO:tasks.rados.rados.0.smithi100.stdout:2635: oid 47 version 1178 is already newer than 1176 2024-04-07T16:25:26.946 INFO:tasks.rados.rados.0.smithi100.stdout:2635: oid 47 updating version 1178 to 1179 2024-04-07T16:25:26.946 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 47 v 1179 (ObjNum 792 snap 250 seq_num 792) dirty exists 2024-04-07T16:25:26.946 INFO:tasks.rados.rados.0.smithi100.stdout:2635: left oid 47 (ObjNum 792 snap 250 seq_num 792) 2024-04-07T16:25:27.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:26 smithi100 bash[74942]: cluster 2024-04-07T16:25:25.885098+0000 mon.a (mon.0) 833 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN) 2024-04-07T16:25:27.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:26 smithi100 bash[74942]: cluster 2024-04-07T16:25:25.885153+0000 mon.a (mon.0) 834 : cluster [WRN] Health check failed: all OSDs are running squid or later but require_osd_release < squid (OSD_UPGRADE_FINISHED) 2024-04-07T16:25:27.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:26 smithi100 bash[74942]: cluster 2024-04-07T16:25:25.908997+0000 mon.a (mon.0) 835 : cluster [DBG] osdmap e570: 8 total, 7 up, 8 in 2024-04-07T16:25:27.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:26 smithi100 bash[74942]: cluster 2024-04-07T16:25:26.871651+0000 mon.a (mon.0) 836 : cluster [DBG] osdmap e571: 8 total, 7 up, 8 in 2024-04-07T16:25:27.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:26 smithi100 bash[76202]: cluster 2024-04-07T16:25:25.885098+0000 mon.a (mon.0) 833 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN) 2024-04-07T16:25:27.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:26 smithi100 bash[76202]: cluster 2024-04-07T16:25:25.885153+0000 mon.a (mon.0) 834 : cluster [WRN] Health check failed: all OSDs are running squid or later but require_osd_release < squid (OSD_UPGRADE_FINISHED) 2024-04-07T16:25:27.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:26 smithi100 bash[76202]: cluster 2024-04-07T16:25:25.908997+0000 mon.a (mon.0) 835 : cluster [DBG] osdmap e570: 8 total, 7 up, 8 in 2024-04-07T16:25:27.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:26 smithi100 bash[76202]: cluster 2024-04-07T16:25:26.871651+0000 mon.a (mon.0) 836 : cluster [DBG] osdmap e571: 8 total, 7 up, 8 in 2024-04-07T16:25:27.323 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:26 smithi144 bash[66654]: cluster 2024-04-07T16:25:25.885098+0000 mon.a (mon.0) 833 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN) 2024-04-07T16:25:27.323 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:26 smithi144 bash[66654]: cluster 2024-04-07T16:25:25.885153+0000 mon.a (mon.0) 834 : cluster [WRN] Health check failed: all OSDs are running squid or later but require_osd_release < squid (OSD_UPGRADE_FINISHED) 2024-04-07T16:25:27.323 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:26 smithi144 bash[66654]: cluster 2024-04-07T16:25:25.908997+0000 mon.a (mon.0) 835 : cluster [DBG] osdmap e570: 8 total, 7 up, 8 in 2024-04-07T16:25:27.323 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:26 smithi144 bash[66654]: cluster 2024-04-07T16:25:26.871651+0000 mon.a (mon.0) 836 : cluster [DBG] osdmap e571: 8 total, 7 up, 8 in 2024-04-07T16:25:27.323 INFO:journalctl@ceph.osd.7.smithi144.stdout:Apr 07 16:25:27 smithi144 bash[83768]: ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4-osd-7 2024-04-07T16:25:27.606 INFO:journalctl@ceph.osd.7.smithi144.stdout:Apr 07 16:25:27 smithi144 bash[83832]: Error response from daemon: No such container: ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4-osd-7 2024-04-07T16:25:27.926 INFO:tasks.rados.rados.0.smithi100.stdout:2635: done (1 left) 2024-04-07T16:25:27.926 INFO:tasks.rados.rados.0.smithi100.stdout:2637: done (0 left) 2024-04-07T16:25:27.926 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 48 current snap is 251 2024-04-07T16:25:27.927 INFO:tasks.rados.rados.0.smithi100.stdout:2638: seq_num 794 ranges {0=499712,499712=532480,1032192=270336} 2024-04-07T16:25:27.930 INFO:tasks.rados.rados.0.smithi100.stdout:2638: writing smithi10066867-48 from 0 to 499712 tid 1 2024-04-07T16:25:27.933 INFO:tasks.rados.rados.0.smithi100.stdout:2638: writing smithi10066867-48 from 499712 to 1032192 tid 2 2024-04-07T16:25:27.935 INFO:tasks.rados.rados.0.smithi100.stdout:2638: writing smithi10066867-48 from 1032192 to 1302528 tid 3 2024-04-07T16:25:27.935 INFO:tasks.rados.rados.0.smithi100.stdout:2639: delete oid 9 current snap is 251 2024-04-07T16:25:27.942 INFO:tasks.rados.rados.0.smithi100.stdout:2638: finishing write tid 1 to smithi10066867-48 2024-04-07T16:25:27.943 INFO:tasks.rados.rados.0.smithi100.stdout:2638: finishing write tid 2 to smithi10066867-48 2024-04-07T16:25:27.946 INFO:tasks.rados.rados.0.smithi100.stdout:2638: finishing write tid 3 to smithi10066867-48 2024-04-07T16:25:27.946 INFO:tasks.rados.rados.0.smithi100.stdout:2639: done (1 left) 2024-04-07T16:25:27.946 INFO:tasks.rados.rados.0.smithi100.stdout:2640: read oid 25 snap -1 2024-04-07T16:25:27.946 INFO:tasks.rados.rados.0.smithi100.stdout:2640: expect (ObjNum 784 snap 246 seq_num 784) 2024-04-07T16:25:27.946 INFO:tasks.rados.rados.0.smithi100.stdout:2641: read oid 18 snap -1 2024-04-07T16:25:27.946 INFO:tasks.rados.rados.0.smithi100.stdout:2641: expect (ObjNum 787 snap 247 seq_num 787) 2024-04-07T16:25:27.946 INFO:tasks.rados.rados.0.smithi100.stdout:2642: copy_from oid 45 from oid 1 current snap is 251 2024-04-07T16:25:27.946 INFO:tasks.rados.rados.0.smithi100.stdout:2643: read oid 20 snap -1 2024-04-07T16:25:27.946 INFO:tasks.rados.rados.0.smithi100.stdout:2643: expect (ObjNum 0 snap 0 seq_num 0) 2024-04-07T16:25:27.947 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 11 current snap is 251 2024-04-07T16:25:27.947 INFO:tasks.rados.rados.0.smithi100.stdout:2644: seq_num 796 ranges {1105920=540672,1646592=704512,2351104=98304} 2024-04-07T16:25:27.957 INFO:tasks.rados.rados.0.smithi100.stdout:2644: writing smithi10066867-11 from 1105920 to 1646592 tid 1 2024-04-07T16:25:27.961 INFO:tasks.rados.rados.0.smithi100.stdout:2644: writing smithi10066867-11 from 1646592 to 2351104 tid 2 2024-04-07T16:25:27.962 INFO:tasks.rados.rados.0.smithi100.stdout:2644: writing smithi10066867-11 from 2351104 to 2449408 tid 3 2024-04-07T16:25:27.963 INFO:tasks.rados.rados.0.smithi100.stdout:2645: delete oid 44 current snap is 251 2024-04-07T16:25:27.963 INFO:tasks.rados.rados.0.smithi100.stdout:2638: finishing write tid 4 to smithi10066867-48 2024-04-07T16:25:27.963 INFO:tasks.rados.rados.0.smithi100.stdout:2638: finishing write tid 5 to smithi10066867-48 2024-04-07T16:25:27.963 INFO:tasks.rados.rados.0.smithi100.stdout:2638: oid 48 updating version 0 to 1112 2024-04-07T16:25:27.963 INFO:tasks.rados.rados.0.smithi100.stdout:2638: oid 48 updating version 1112 to 1113 2024-04-07T16:25:27.963 INFO:tasks.rados.rados.0.smithi100.stdout:2638: oid 48 version 1113 is already newer than 1110 2024-04-07T16:25:27.963 INFO:tasks.rados.rados.0.smithi100.stdout:2638: oid 48 version 1113 is already newer than 1111 2024-04-07T16:25:27.963 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 48 v 1113 (ObjNum 794 snap 251 seq_num 794) dirty exists 2024-04-07T16:25:27.963 INFO:tasks.rados.rados.0.smithi100.stdout:2638: left oid 48 (ObjNum 794 snap 251 seq_num 794) 2024-04-07T16:25:27.963 INFO:tasks.rados.rados.0.smithi100.stdout:2641: expect (ObjNum 787 snap 247 seq_num 787) 2024-04-07T16:25:28.032 INFO:tasks.rados.rados.0.smithi100.stdout:2642: finishing copy_from to smithi10066867-45 2024-04-07T16:25:28.032 INFO:tasks.rados.rados.0.smithi100.stdout:2642: got expected ENOENT (src dne) 2024-04-07T16:25:28.032 INFO:tasks.rados.rados.0.smithi100.stdout:2642: finishing copy_from racing read to smithi10066867-45 2024-04-07T16:25:28.032 INFO:tasks.rados.rados.0.smithi100.stdout:2640: expect (ObjNum 784 snap 246 seq_num 784) 2024-04-07T16:25:28.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:27 smithi100 bash[74942]: cluster 2024-04-07T16:25:26.730806+0000 mgr.y (mgr.25175) 258 : cluster [DBG] pgmap v448: 121 pgs: 20 stale+active+clean, 101 active+clean; 164 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 3.7 MiB/s rd, 1.1 MiB/s wr, 14 op/s 2024-04-07T16:25:28.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:27 smithi100 bash[76202]: cluster 2024-04-07T16:25:26.730806+0000 mgr.y (mgr.25175) 258 : cluster [DBG] pgmap v448: 121 pgs: 20 stale+active+clean, 101 active+clean; 164 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 3.7 MiB/s rd, 1.1 MiB/s wr, 14 op/s 2024-04-07T16:25:28.190 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:27 smithi144 bash[66654]: cluster 2024-04-07T16:25:26.730806+0000 mgr.y (mgr.25175) 258 : cluster [DBG] pgmap v448: 121 pgs: 20 stale+active+clean, 101 active+clean; 164 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 3.7 MiB/s rd, 1.1 MiB/s wr, 14 op/s 2024-04-07T16:25:28.372 INFO:tasks.rados.rados.0.smithi100.stdout:2643: expect (ObjNum 0 snap 0 seq_num 0) 2024-04-07T16:25:28.375 INFO:tasks.rados.rados.0.smithi100.stdout:2644: finishing write tid 1 to smithi10066867-11 2024-04-07T16:25:28.375 INFO:tasks.rados.rados.0.smithi100.stdout:2644: finishing write tid 2 to smithi10066867-11 2024-04-07T16:25:28.375 INFO:tasks.rados.rados.0.smithi100.stdout:2644: finishing write tid 3 to smithi10066867-11 2024-04-07T16:25:28.375 INFO:tasks.rados.rados.0.smithi100.stdout:2644: finishing write tid 4 to smithi10066867-11 2024-04-07T16:25:28.376 INFO:tasks.rados.rados.0.smithi100.stdout:2644: finishing write tid 5 to smithi10066867-11 2024-04-07T16:25:28.376 INFO:tasks.rados.rados.0.smithi100.stdout:2644: oid 11 updating version 0 to 1495 2024-04-07T16:25:28.376 INFO:tasks.rados.rados.0.smithi100.stdout:2644: oid 11 version 1495 is already newer than 1494 2024-04-07T16:25:28.376 INFO:tasks.rados.rados.0.smithi100.stdout:2644: oid 11 updating version 1495 to 1497 2024-04-07T16:25:28.376 INFO:tasks.rados.rados.0.smithi100.stdout:2644: oid 11 version 1497 is already newer than 1496 2024-04-07T16:25:28.376 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 11 v 1497 (ObjNum 796 snap 251 seq_num 796) dirty exists 2024-04-07T16:25:28.376 INFO:tasks.rados.rados.0.smithi100.stdout:2644: left oid 11 (ObjNum 796 snap 251 seq_num 796) 2024-04-07T16:25:28.376 INFO:tasks.rados.rados.0.smithi100.stdout:2638: done (6 left) 2024-04-07T16:25:28.376 INFO:tasks.rados.rados.0.smithi100.stdout:2640: done (5 left) 2024-04-07T16:25:28.376 INFO:tasks.rados.rados.0.smithi100.stdout:2641: done (4 left) 2024-04-07T16:25:28.376 INFO:tasks.rados.rados.0.smithi100.stdout:2642: done (3 left) 2024-04-07T16:25:28.376 INFO:tasks.rados.rados.0.smithi100.stdout:2643: done (2 left) 2024-04-07T16:25:28.376 INFO:tasks.rados.rados.0.smithi100.stdout:2644: done (1 left) 2024-04-07T16:25:28.376 INFO:tasks.rados.rados.0.smithi100.stdout:2645: done (0 left) 2024-04-07T16:25:28.376 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 46 current snap is 251 2024-04-07T16:25:28.377 INFO:tasks.rados.rados.0.smithi100.stdout:2646: seq_num 798 ranges {114688=573440,688128=466944,1155072=335872} 2024-04-07T16:25:28.382 INFO:tasks.rados.rados.0.smithi100.stdout:2646: writing smithi10066867-46 from 114688 to 688128 tid 1 2024-04-07T16:25:28.387 INFO:tasks.rados.rados.0.smithi100.stdout:2646: writing smithi10066867-46 from 688128 to 1155072 tid 2 2024-04-07T16:25:28.394 INFO:tasks.rados.rados.0.smithi100.stdout:2646: writing smithi10066867-46 from 1155072 to 1490944 tid 3 2024-04-07T16:25:28.394 INFO:tasks.rados.rados.0.smithi100.stdout:2647: setattr oid 11 current snap is 251 2024-04-07T16:25:28.394 INFO:tasks.rados.rados.0.smithi100.stdout:2646: finishing write tid 1 to smithi10066867-46 2024-04-07T16:25:28.394 INFO:tasks.rados.rados.0.smithi100.stdout:2646: finishing write tid 2 to smithi10066867-46 2024-04-07T16:25:28.395 INFO:tasks.rados.rados.0.smithi100.stdout:2648: read oid 33 snap -1 2024-04-07T16:25:28.395 INFO:tasks.rados.rados.0.smithi100.stdout:2648: expect (ObjNum 753 snap 235 seq_num 753) 2024-04-07T16:25:28.395 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 18 current snap is 251 2024-04-07T16:25:28.395 INFO:tasks.rados.rados.0.smithi100.stdout:2649: seq_num 799 ranges {1187840=589824,1777664=802816} 2024-04-07T16:25:28.414 INFO:tasks.rados.rados.0.smithi100.stdout:2649: writing smithi10066867-18 from 1187840 to 1777664 tid 1 2024-04-07T16:25:28.424 INFO:tasks.rados.rados.0.smithi100.stdout:2649: writing smithi10066867-18 from 1777664 to 2580480 tid 2 2024-04-07T16:25:28.424 INFO:tasks.rados.rados.0.smithi100.stdout:2650: setattr oid 36 current snap is 251 2024-04-07T16:25:28.424 INFO:tasks.rados.rados.0.smithi100.stdout:2646: finishing write tid 3 to smithi10066867-46 2024-04-07T16:25:28.424 INFO:tasks.rados.rados.0.smithi100.stdout:2646: finishing write tid 4 to smithi10066867-46 2024-04-07T16:25:28.424 INFO:tasks.rados.rados.0.smithi100.stdout:2646: finishing write tid 5 to smithi10066867-46 2024-04-07T16:25:28.424 INFO:tasks.rados.rados.0.smithi100.stdout:2646: oid 46 updating version 0 to 1077 2024-04-07T16:25:28.424 INFO:tasks.rados.rados.0.smithi100.stdout:2646: oid 46 version 1077 is already newer than 1076 2024-04-07T16:25:28.424 INFO:tasks.rados.rados.0.smithi100.stdout:2646: oid 46 updating version 1077 to 1079 2024-04-07T16:25:28.424 INFO:tasks.rados.rados.0.smithi100.stdout:2646: oid 46 version 1079 is already newer than 1078 2024-04-07T16:25:28.424 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 46 v 1079 (ObjNum 798 snap 251 seq_num 798) dirty exists 2024-04-07T16:25:28.424 INFO:tasks.rados.rados.0.smithi100.stdout:2646: left oid 46 (ObjNum 798 snap 251 seq_num 798) 2024-04-07T16:25:28.424 INFO:tasks.rados.rados.0.smithi100.stdout:2648: expect (ObjNum 753 snap 235 seq_num 753) 2024-04-07T16:25:28.453 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 11 v 1498 (ObjNum 796 snap 251 seq_num 796) dirty exists 2024-04-07T16:25:28.453 INFO:tasks.rados.rados.0.smithi100.stdout:2649: finishing write tid 1 to smithi10066867-18 2024-04-07T16:25:28.453 INFO:tasks.rados.rados.0.smithi100.stdout:2649: finishing write tid 2 to smithi10066867-18 2024-04-07T16:25:28.453 INFO:tasks.rados.rados.0.smithi100.stdout:2649: finishing write tid 3 to smithi10066867-18 2024-04-07T16:25:28.453 INFO:tasks.rados.rados.0.smithi100.stdout:2649: finishing write tid 4 to smithi10066867-18 2024-04-07T16:25:28.453 INFO:tasks.rados.rados.0.smithi100.stdout:2649: oid 18 updating version 0 to 1500 2024-04-07T16:25:28.453 INFO:tasks.rados.rados.0.smithi100.stdout:2649: oid 18 updating version 1500 to 1502 2024-04-07T16:25:28.453 INFO:tasks.rados.rados.0.smithi100.stdout:2649: oid 18 version 1502 is already newer than 1501 2024-04-07T16:25:28.453 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 18 v 1502 (ObjNum 799 snap 251 seq_num 799) dirty exists 2024-04-07T16:25:28.453 INFO:tasks.rados.rados.0.smithi100.stdout:2649: left oid 18 (ObjNum 799 snap 251 seq_num 799) 2024-04-07T16:25:28.453 INFO:tasks.rados.rados.0.smithi100.stdout:2646: done (4 left) 2024-04-07T16:25:28.453 INFO:tasks.rados.rados.0.smithi100.stdout:2647: done (3 left) 2024-04-07T16:25:28.454 INFO:tasks.rados.rados.0.smithi100.stdout:2648: done (2 left) 2024-04-07T16:25:28.454 INFO:tasks.rados.rados.0.smithi100.stdout:2649: done (1 left) 2024-04-07T16:25:28.454 INFO:tasks.rados.rados.0.smithi100.stdout:2651: read oid 5 snap 250 2024-04-07T16:25:28.454 INFO:tasks.rados.rados.0.smithi100.stdout:2651: expect deleted 2024-04-07T16:25:28.454 INFO:tasks.rados.rados.0.smithi100.stdout:2652: rmattr oid 9 current snap is 251 2024-04-07T16:25:28.454 INFO:tasks.rados.rados.0.smithi100.stdout:2652: done (2 left) 2024-04-07T16:25:28.454 INFO:tasks.rados.rados.0.smithi100.stdout:2653: copy_from oid 21 from oid 37 current snap is 251 2024-04-07T16:25:28.454 INFO:tasks.rados.rados.0.smithi100.stdout:2654: delete oid 24 current snap is 251 2024-04-07T16:25:28.458 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 36 v 1176 (ObjNum 943076912 snap 825046838 seq_num 304) dirty exists 2024-04-07T16:25:28.459 INFO:tasks.rados.rados.0.smithi100.stdout:2650: done (3 left) 2024-04-07T16:25:28.459 INFO:tasks.rados.rados.0.smithi100.stdout:2651: done (2 left) 2024-04-07T16:25:28.459 INFO:tasks.rados.rados.0.smithi100.stdout:2654: done (1 left) 2024-04-07T16:25:28.459 INFO:tasks.rados.rados.0.smithi100.stdout:2655: setattr oid 33 current snap is 251 2024-04-07T16:25:28.461 INFO:tasks.rados.rados.0.smithi100.stdout:2656: read oid 16 snap -1 2024-04-07T16:25:28.462 INFO:tasks.rados.rados.0.smithi100.stdout:2656: expect deleted 2024-04-07T16:25:28.462 INFO:tasks.rados.rados.0.smithi100.stdout:2657: setattr oid 45 current snap is 251 2024-04-07T16:25:28.464 INFO:tasks.rados.rados.0.smithi100.stdout:2656: done (3 left) 2024-04-07T16:25:28.464 INFO:tasks.rados.rados.0.smithi100.stdout:2658: rmattr oid 16 current snap is 251 2024-04-07T16:25:28.464 INFO:tasks.rados.rados.0.smithi100.stdout:2658: done (3 left) 2024-04-07T16:25:28.465 INFO:tasks.rados.rados.0.smithi100.stdout:2659: read oid 30 snap -1 2024-04-07T16:25:28.465 INFO:tasks.rados.rados.0.smithi100.stdout:2659: expect (ObjNum 722 snap 223 seq_num 722) 2024-04-07T16:25:28.465 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 29 current snap is 251 2024-04-07T16:25:28.465 INFO:tasks.rados.rados.0.smithi100.stdout:2660: seq_num 801 ranges {7856128=622592,8478720=573440,9052160=229376} 2024-04-07T16:25:28.534 INFO:tasks.rados.rados.0.smithi100.stdout:2660: writing smithi10066867-29 from 7856128 to 8478720 tid 1 2024-04-07T16:25:28.538 INFO:tasks.rados.rados.0.smithi100.stdout:2660: writing smithi10066867-29 from 8478720 to 9052160 tid 2 2024-04-07T16:25:28.540 INFO:tasks.rados.rados.0.smithi100.stdout:2660: writing smithi10066867-29 from 9052160 to 9281536 tid 3 2024-04-07T16:25:28.540 INFO:tasks.rados.rados.0.smithi100.stdout:2661: snap_create 2024-04-07T16:25:28.540 INFO:tasks.rados.rados.0.smithi100.stdout:2653: finishing copy_from racing read to smithi10066867-21 2024-04-07T16:25:28.540 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 33 v 1504 (ObjNum 753 snap 235 seq_num 753) dirty exists 2024-04-07T16:25:28.540 INFO:tasks.rados.rados.0.smithi100.stdout:2653: finishing copy_from to smithi10066867-21 2024-04-07T16:25:28.540 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 21 v 1181 (ObjNum 791 snap 250 seq_num 791) dirty exists 2024-04-07T16:25:28.541 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 45 v 1038 (ObjNum 33 snap 0 seq_num 2702494048) dirty exists 2024-04-07T16:25:28.541 INFO:tasks.rados.rados.0.smithi100.stdout:2659: expect (ObjNum 722 snap 223 seq_num 722) 2024-04-07T16:25:28.685 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:25:28 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:25:28] "GET /metrics HTTP/1.1" 200 36279 "" "Prometheus/2.43.0" 2024-04-07T16:25:28.807 INFO:tasks.rados.rados.0.smithi100.stdout:2660: finishing write tid 1 to smithi10066867-29 2024-04-07T16:25:28.807 INFO:tasks.rados.rados.0.smithi100.stdout:2660: finishing write tid 2 to smithi10066867-29 2024-04-07T16:25:28.807 INFO:tasks.rados.rados.0.smithi100.stdout:2660: finishing write tid 3 to smithi10066867-29 2024-04-07T16:25:28.807 INFO:tasks.rados.rados.0.smithi100.stdout:2660: finishing write tid 4 to smithi10066867-29 2024-04-07T16:25:28.807 INFO:tasks.rados.rados.0.smithi100.stdout:2660: finishing write tid 5 to smithi10066867-29 2024-04-07T16:25:28.807 INFO:tasks.rados.rados.0.smithi100.stdout:2660: oid 29 updating version 0 to 1182 2024-04-07T16:25:28.807 INFO:tasks.rados.rados.0.smithi100.stdout:2660: oid 29 updating version 1182 to 1185 2024-04-07T16:25:28.807 INFO:tasks.rados.rados.0.smithi100.stdout:2660: oid 29 version 1185 is already newer than 1184 2024-04-07T16:25:28.807 INFO:tasks.rados.rados.0.smithi100.stdout:2660: oid 29 version 1185 is already newer than 1183 2024-04-07T16:25:28.807 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 29 v 1185 (ObjNum 801 snap 251 seq_num 801) dirty exists 2024-04-07T16:25:28.807 INFO:tasks.rados.rados.0.smithi100.stdout:2660: left oid 29 (ObjNum 801 snap 251 seq_num 801) 2024-04-07T16:25:28.953 INFO:tasks.rados.rados.0.smithi100.stdout:2653: done (5 left) 2024-04-07T16:25:28.954 INFO:tasks.rados.rados.0.smithi100.stdout:2655: done (4 left) 2024-04-07T16:25:28.954 INFO:tasks.rados.rados.0.smithi100.stdout:2657: done (3 left) 2024-04-07T16:25:28.954 INFO:tasks.rados.rados.0.smithi100.stdout:2659: done (2 left) 2024-04-07T16:25:28.954 INFO:tasks.rados.rados.0.smithi100.stdout:2660: done (1 left) 2024-04-07T16:25:28.954 INFO:tasks.rados.rados.0.smithi100.stdout:2661: done (0 left) 2024-04-07T16:25:28.954 INFO:tasks.rados.rados.0.smithi100.stdout:2662: snap_remove snap 242 2024-04-07T16:25:29.205 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:28 smithi144 bash[66654]: cluster 2024-04-07T16:25:27.942297+0000 mon.a (mon.0) 837 : cluster [DBG] osdmap e572: 8 total, 7 up, 8 in 2024-04-07T16:25:29.206 INFO:journalctl@ceph.osd.7.smithi144.stdout:Apr 07 16:25:29 smithi144 systemd[1]: ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@osd.7.service: Deactivated successfully. 2024-04-07T16:25:29.206 INFO:journalctl@ceph.osd.7.smithi144.stdout:Apr 07 16:25:29 smithi144 systemd[1]: Stopped Ceph osd.7 for 0702e3a8-f4f8-11ee-b649-cb9ed24678a4. 2024-04-07T16:25:29.435 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:28 smithi100 bash[74942]: cluster 2024-04-07T16:25:27.942297+0000 mon.a (mon.0) 837 : cluster [DBG] osdmap e572: 8 total, 7 up, 8 in 2024-04-07T16:25:29.435 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:28 smithi100 bash[76202]: cluster 2024-04-07T16:25:27.942297+0000 mon.a (mon.0) 837 : cluster [DBG] osdmap e572: 8 total, 7 up, 8 in 2024-04-07T16:25:29.467 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:29 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:25:29.468 INFO:journalctl@ceph.osd.7.smithi144.stdout:Apr 07 16:25:29 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:25:29.468 INFO:journalctl@ceph.mds.b.smithi144.stdout:Apr 07 16:25:29 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:25:29.468 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:25:29 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:25:29.469 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:25:29 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:25:29.469 INFO:journalctl@ceph.osd.4.smithi144.stdout:Apr 07 16:25:29 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:25:29.469 INFO:journalctl@ceph.osd.5.smithi144.stdout:Apr 07 16:25:29 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:25:29.470 INFO:journalctl@ceph.osd.6.smithi144.stdout:Apr 07 16:25:29 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:25:29.470 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 16:25:29 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:25:29.470 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:25:29 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:25:29.856 INFO:journalctl@ceph.osd.7.smithi144.stdout:Apr 07 16:25:29 smithi144 systemd[1]: Started Ceph osd.7 for 0702e3a8-f4f8-11ee-b649-cb9ed24678a4. 2024-04-07T16:25:29.952 INFO:tasks.rados.rados.0.smithi100.stdout:2662: done (0 left) 2024-04-07T16:25:29.953 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 2 current snap is 252 2024-04-07T16:25:29.953 INFO:tasks.rados.rados.0.smithi100.stdout:2663: seq_num 802 ranges {0=638976,638976=458752,1097728=344064} 2024-04-07T16:25:29.957 INFO:tasks.rados.rados.0.smithi100.stdout:2663: writing smithi10066867-2 from 0 to 638976 tid 1 2024-04-07T16:25:29.961 INFO:tasks.rados.rados.0.smithi100.stdout:2663: writing smithi10066867-2 from 638976 to 1097728 tid 2 2024-04-07T16:25:29.963 INFO:tasks.rados.rados.0.smithi100.stdout:2663: writing smithi10066867-2 from 1097728 to 1441792 tid 3 2024-04-07T16:25:29.964 INFO:tasks.rados.rados.0.smithi100.stdout:2664: read oid 25 snap -1 2024-04-07T16:25:29.964 INFO:tasks.rados.rados.0.smithi100.stdout:2664: expect (ObjNum 784 snap 246 seq_num 784) 2024-04-07T16:25:29.964 INFO:tasks.rados.rados.0.smithi100.stdout:2665: read oid 48 snap 250 2024-04-07T16:25:29.964 INFO:tasks.rados.rados.0.smithi100.stdout:2665: expect deleted 2024-04-07T16:25:29.964 INFO:tasks.rados.rados.0.smithi100.stdout:2666: delete oid 27 current snap is 252 2024-04-07T16:25:29.991 INFO:tasks.rados.rados.0.smithi100.stdout:2665: done (3 left) 2024-04-07T16:25:29.992 INFO:tasks.rados.rados.0.smithi100.stdout:2666: done (2 left) 2024-04-07T16:25:29.992 INFO:tasks.rados.rados.0.smithi100.stdout:2667: snap_create 2024-04-07T16:25:29.994 INFO:tasks.rados.rados.0.smithi100.stdout:2663: finishing write tid 1 to smithi10066867-2 2024-04-07T16:25:29.994 INFO:tasks.rados.rados.0.smithi100.stdout:2663: finishing write tid 2 to smithi10066867-2 2024-04-07T16:25:29.994 INFO:tasks.rados.rados.0.smithi100.stdout:2663: finishing write tid 3 to smithi10066867-2 2024-04-07T16:25:29.995 INFO:tasks.rados.rados.0.smithi100.stdout:2663: finishing write tid 4 to smithi10066867-2 2024-04-07T16:25:29.998 INFO:tasks.rados.rados.0.smithi100.stdout:2663: finishing write tid 5 to smithi10066867-2 2024-04-07T16:25:29.998 INFO:tasks.rados.rados.0.smithi100.stdout:2663: oid 2 updating version 0 to 1198 2024-04-07T16:25:29.998 INFO:tasks.rados.rados.0.smithi100.stdout:2663: oid 2 updating version 1198 to 1201 2024-04-07T16:25:29.999 INFO:tasks.rados.rados.0.smithi100.stdout:2663: oid 2 version 1201 is already newer than 1199 2024-04-07T16:25:29.999 INFO:tasks.rados.rados.0.smithi100.stdout:2663: oid 2 version 1201 is already newer than 1200 2024-04-07T16:25:29.999 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 2 v 1201 (ObjNum 802 snap 252 seq_num 802) dirty exists 2024-04-07T16:25:29.999 INFO:tasks.rados.rados.0.smithi100.stdout:2663: left oid 2 (ObjNum 802 snap 252 seq_num 802) 2024-04-07T16:25:30.001 INFO:tasks.rados.rados.0.smithi100.stdout:2664: expect (ObjNum 784 snap 246 seq_num 784) 2024-04-07T16:25:30.306 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:29 smithi144 bash[66654]: cluster 2024-04-07T16:25:28.731782+0000 mgr.y (mgr.25175) 259 : cluster [DBG] pgmap v451: 121 pgs: 7 active+undersized+degraded, 7 stale+active+clean, 21 active+undersized, 86 active+clean; 166 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 604 KiB/s wr, 9 op/s; 14/324 objects degraded (4.321%) 2024-04-07T16:25:30.306 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:29 smithi144 bash[66654]: cluster 2024-04-07T16:25:28.945668+0000 mon.a (mon.0) 838 : cluster [WRN] Health check failed: Degraded data redundancy: 14/324 objects degraded (4.321%), 7 pgs degraded (PG_DEGRADED) 2024-04-07T16:25:30.306 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:29 smithi144 bash[66654]: cluster 2024-04-07T16:25:28.967271+0000 mon.a (mon.0) 839 : cluster [DBG] osdmap e573: 8 total, 7 up, 8 in 2024-04-07T16:25:30.306 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:29 smithi144 bash[66654]: audit 2024-04-07T16:25:29.538287+0000 mon.a (mon.0) 840 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:25:30.306 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:29 smithi144 bash[66654]: audit 2024-04-07T16:25:29.551461+0000 mon.a (mon.0) 841 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:25:30.306 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:29 smithi144 bash[66654]: audit 2024-04-07T16:25:29.596974+0000 mon.a (mon.0) 842 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:25:30.435 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:29 smithi100 bash[74942]: cluster 2024-04-07T16:25:28.731782+0000 mgr.y (mgr.25175) 259 : cluster [DBG] pgmap v451: 121 pgs: 7 active+undersized+degraded, 7 stale+active+clean, 21 active+undersized, 86 active+clean; 166 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 604 KiB/s wr, 9 op/s; 14/324 objects degraded (4.321%) 2024-04-07T16:25:30.435 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:29 smithi100 bash[74942]: cluster 2024-04-07T16:25:28.945668+0000 mon.a (mon.0) 838 : cluster [WRN] Health check failed: Degraded data redundancy: 14/324 objects degraded (4.321%), 7 pgs degraded (PG_DEGRADED) 2024-04-07T16:25:30.435 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:29 smithi100 bash[74942]: cluster 2024-04-07T16:25:28.967271+0000 mon.a (mon.0) 839 : cluster [DBG] osdmap e573: 8 total, 7 up, 8 in 2024-04-07T16:25:30.435 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:29 smithi100 bash[74942]: audit 2024-04-07T16:25:29.538287+0000 mon.a (mon.0) 840 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:25:30.435 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:29 smithi100 bash[74942]: audit 2024-04-07T16:25:29.551461+0000 mon.a (mon.0) 841 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:25:30.435 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:29 smithi100 bash[74942]: audit 2024-04-07T16:25:29.596974+0000 mon.a (mon.0) 842 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:25:30.435 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:29 smithi100 bash[76202]: cluster 2024-04-07T16:25:28.731782+0000 mgr.y (mgr.25175) 259 : cluster [DBG] pgmap v451: 121 pgs: 7 active+undersized+degraded, 7 stale+active+clean, 21 active+undersized, 86 active+clean; 166 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 604 KiB/s wr, 9 op/s; 14/324 objects degraded (4.321%) 2024-04-07T16:25:30.436 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:29 smithi100 bash[76202]: cluster 2024-04-07T16:25:28.945668+0000 mon.a (mon.0) 838 : cluster [WRN] Health check failed: Degraded data redundancy: 14/324 objects degraded (4.321%), 7 pgs degraded (PG_DEGRADED) 2024-04-07T16:25:30.436 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:29 smithi100 bash[76202]: cluster 2024-04-07T16:25:28.967271+0000 mon.a (mon.0) 839 : cluster [DBG] osdmap e573: 8 total, 7 up, 8 in 2024-04-07T16:25:30.436 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:29 smithi100 bash[76202]: audit 2024-04-07T16:25:29.538287+0000 mon.a (mon.0) 840 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:25:30.436 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:29 smithi100 bash[76202]: audit 2024-04-07T16:25:29.551461+0000 mon.a (mon.0) 841 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:25:30.436 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:29 smithi100 bash[76202]: audit 2024-04-07T16:25:29.596974+0000 mon.a (mon.0) 842 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:25:30.979 INFO:tasks.rados.rados.0.smithi100.stdout:2663: done (2 left) 2024-04-07T16:25:30.979 INFO:tasks.rados.rados.0.smithi100.stdout:2664: done (1 left) 2024-04-07T16:25:30.979 INFO:tasks.rados.rados.0.smithi100.stdout:2667: done (0 left) 2024-04-07T16:25:30.979 INFO:tasks.rados.rados.0.smithi100.stdout:2668: snap_remove snap 250 2024-04-07T16:25:31.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:30 smithi144 bash[66654]: cluster 2024-04-07T16:25:29.967383+0000 mon.a (mon.0) 843 : cluster [DBG] osdmap e574: 8 total, 7 up, 8 in 2024-04-07T16:25:31.356 INFO:journalctl@ceph.osd.7.smithi144.stdout:Apr 07 16:25:30 smithi144 bash[83982]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-04-07T16:25:31.356 INFO:journalctl@ceph.osd.7.smithi144.stdout:Apr 07 16:25:30 smithi144 bash[83982]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-7 --no-mon-config --dev /dev/mapper/vg_nvme-lv_1 2024-04-07T16:25:31.357 INFO:journalctl@ceph.osd.7.smithi144.stdout:Apr 07 16:25:30 smithi144 bash[83982]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_1 2024-04-07T16:25:31.357 INFO:journalctl@ceph.osd.7.smithi144.stdout:Apr 07 16:25:30 smithi144 bash[83982]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-0 2024-04-07T16:25:31.357 INFO:journalctl@ceph.osd.7.smithi144.stdout:Apr 07 16:25:30 smithi144 bash[83982]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-7/block 2024-04-07T16:25:31.357 INFO:journalctl@ceph.osd.7.smithi144.stdout:Apr 07 16:25:30 smithi144 bash[83982]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-04-07T16:25:31.357 INFO:journalctl@ceph.osd.7.smithi144.stdout:Apr 07 16:25:30 smithi144 bash[83982]: --> ceph-volume raw activate successful for osd ID: 7 2024-04-07T16:25:31.435 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:30 smithi100 bash[74942]: cluster 2024-04-07T16:25:29.967383+0000 mon.a (mon.0) 843 : cluster [DBG] osdmap e574: 8 total, 7 up, 8 in 2024-04-07T16:25:31.435 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:30 smithi100 bash[76202]: cluster 2024-04-07T16:25:29.967383+0000 mon.a (mon.0) 843 : cluster [DBG] osdmap e574: 8 total, 7 up, 8 in 2024-04-07T16:25:31.861 INFO:tasks.rados.rados.0.smithi100.stdout:2668: done (0 left) 2024-04-07T16:25:31.862 INFO:tasks.rados.rados.0.smithi100.stdout:2669: rmattr oid 26 current snap is 253 2024-04-07T16:25:31.862 INFO:tasks.rados.rados.0.smithi100.stdout:2669: done (0 left) 2024-04-07T16:25:31.862 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 1 current snap is 253 2024-04-07T16:25:31.862 INFO:tasks.rados.rados.0.smithi100.stdout:2670: seq_num 804 ranges {0=671744,671744=622592,1294336=180224} 2024-04-07T16:25:31.865 INFO:tasks.rados.rados.0.smithi100.stdout:2670: writing smithi10066867-1 from 0 to 671744 tid 1 2024-04-07T16:25:31.869 INFO:tasks.rados.rados.0.smithi100.stdout:2670: writing smithi10066867-1 from 671744 to 1294336 tid 2 2024-04-07T16:25:31.871 INFO:tasks.rados.rados.0.smithi100.stdout:2670: writing smithi10066867-1 from 1294336 to 1474560 tid 3 2024-04-07T16:25:31.871 INFO:tasks.rados.rados.0.smithi100.stdout:2671: copy_from oid 15 from oid 9 current snap is 253 2024-04-07T16:25:31.871 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 41 current snap is 253 2024-04-07T16:25:31.871 INFO:tasks.rados.rados.0.smithi100.stdout:2672: seq_num 805 ranges {2572288=688128,3260416=507904,3768320=294912} 2024-04-07T16:25:31.909 INFO:tasks.rados.rados.0.smithi100.stdout:2672: writing smithi10066867-41 from 2572288 to 3260416 tid 1 2024-04-07T16:25:31.911 INFO:tasks.rados.rados.0.smithi100.stdout:2672: writing smithi10066867-41 from 3260416 to 3768320 tid 2 2024-04-07T16:25:31.914 INFO:tasks.rados.rados.0.smithi100.stdout:2672: writing smithi10066867-41 from 3768320 to 4063232 tid 3 2024-04-07T16:25:31.914 INFO:tasks.rados.rados.0.smithi100.stdout:2673: delete oid 38 current snap is 253 2024-04-07T16:25:31.915 INFO:tasks.rados.rados.0.smithi100.stdout:2670: finishing write tid 1 to smithi10066867-1 2024-04-07T16:25:31.915 INFO:tasks.rados.rados.0.smithi100.stdout:2670: finishing write tid 2 to smithi10066867-1 2024-04-07T16:25:31.915 INFO:tasks.rados.rados.0.smithi100.stdout:2670: finishing write tid 3 to smithi10066867-1 2024-04-07T16:25:31.915 INFO:tasks.rados.rados.0.smithi100.stdout:2670: finishing write tid 4 to smithi10066867-1 2024-04-07T16:25:31.915 INFO:tasks.rados.rados.0.smithi100.stdout:2671: finishing copy_from to smithi10066867-15 2024-04-07T16:25:31.915 INFO:tasks.rados.rados.0.smithi100.stdout:2671: got expected ENOENT (src dne) 2024-04-07T16:25:31.915 INFO:tasks.rados.rados.0.smithi100.stdout:2671: finishing copy_from racing read to smithi10066867-15 2024-04-07T16:25:31.915 INFO:tasks.rados.rados.0.smithi100.stdout:2670: finishing write tid 5 to smithi10066867-1 2024-04-07T16:25:31.915 INFO:tasks.rados.rados.0.smithi100.stdout:2670: oid 1 updating version 0 to 1107 2024-04-07T16:25:31.915 INFO:tasks.rados.rados.0.smithi100.stdout:2670: oid 1 updating version 1107 to 1108 2024-04-07T16:25:31.915 INFO:tasks.rados.rados.0.smithi100.stdout:2670: oid 1 version 1108 is already newer than 1105 2024-04-07T16:25:31.915 INFO:tasks.rados.rados.0.smithi100.stdout:2670: oid 1 version 1108 is already newer than 1106 2024-04-07T16:25:31.915 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 1 v 1108 (ObjNum 804 snap 253 seq_num 804) dirty exists 2024-04-07T16:25:31.915 INFO:tasks.rados.rados.0.smithi100.stdout:2670: left oid 1 (ObjNum 804 snap 253 seq_num 804) 2024-04-07T16:25:31.918 INFO:tasks.rados.rados.0.smithi100.stdout:2672: finishing write tid 1 to smithi10066867-41 2024-04-07T16:25:31.918 INFO:tasks.rados.rados.0.smithi100.stdout:2670: done (3 left) 2024-04-07T16:25:31.918 INFO:tasks.rados.rados.0.smithi100.stdout:2671: done (2 left) 2024-04-07T16:25:31.918 INFO:tasks.rados.rados.0.smithi100.stdout:2673: done (1 left) 2024-04-07T16:25:31.918 INFO:tasks.rados.rados.0.smithi100.stdout:2674: rollback oid 42 current snap is 253 2024-04-07T16:25:31.918 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 42 to 252 2024-04-07T16:25:31.919 INFO:tasks.rados.rados.0.smithi100.stdout:2675: rollback oid 35 current snap is 253 2024-04-07T16:25:31.919 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 35 to 247 2024-04-07T16:25:31.919 INFO:tasks.rados.rados.0.smithi100.stdout:2676: rollback oid 19 current snap is 253 2024-04-07T16:25:31.919 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 19 to 247 2024-04-07T16:25:31.919 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 4 current snap is 253 2024-04-07T16:25:31.919 INFO:tasks.rados.rados.0.smithi100.stdout:2677: seq_num 807 ranges {1867776=720896,2588672=729088,3317760=73728} 2024-04-07T16:25:31.935 INFO:tasks.rados.rados.0.smithi100.stdout:2677: writing smithi10066867-4 from 1867776 to 2588672 tid 1 2024-04-07T16:25:31.939 INFO:tasks.rados.rados.0.smithi100.stdout:2677: writing smithi10066867-4 from 2588672 to 3317760 tid 2 2024-04-07T16:25:31.940 INFO:tasks.rados.rados.0.smithi100.stdout:2677: writing smithi10066867-4 from 3317760 to 3391488 tid 3 2024-04-07T16:25:31.940 INFO:tasks.rados.rados.0.smithi100.stdout:2678: snap_create 2024-04-07T16:25:31.940 INFO:tasks.rados.rados.0.smithi100.stdout:2675: finishing rollback tid 1 to smithi10066867-35 2024-04-07T16:25:31.941 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 35 v 0 (ObjNum 273 snap 0 seq_num 0) dirty dne 2024-04-07T16:25:31.941 INFO:tasks.rados.rados.0.smithi100.stdout:2672: finishing write tid 2 to smithi10066867-41 2024-04-07T16:25:31.941 INFO:tasks.rados.rados.0.smithi100.stdout:2676: finishing rollback tid 0 to smithi10066867-19 2024-04-07T16:25:31.941 INFO:tasks.rados.rados.0.smithi100.stdout:2674: finishing rollback tid 0 to smithi10066867-42 2024-04-07T16:25:31.941 INFO:tasks.rados.rados.0.smithi100.stdout:2674: finishing rollback tid 1 to smithi10066867-42 2024-04-07T16:25:31.941 INFO:tasks.rados.rados.0.smithi100.stdout:2672: finishing write tid 3 to smithi10066867-41 2024-04-07T16:25:31.941 INFO:tasks.rados.rados.0.smithi100.stdout:2674: finishing rollback tid 2 to smithi10066867-42 2024-04-07T16:25:31.941 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 42 v 1188 (ObjNum 775 snap 243 seq_num 775) dirty exists 2024-04-07T16:25:31.941 INFO:tasks.rados.rados.0.smithi100.stdout:2672: finishing write tid 4 to smithi10066867-41 2024-04-07T16:25:31.941 INFO:tasks.rados.rados.0.smithi100.stdout:2676: finishing rollback tid 1 to smithi10066867-19 2024-04-07T16:25:31.941 INFO:tasks.rados.rados.0.smithi100.stdout:2676: finishing rollback tid 2 to smithi10066867-19 2024-04-07T16:25:31.941 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 19 v 1180 (ObjNum 786 snap 247 seq_num 786) dirty exists 2024-04-07T16:25:31.941 INFO:tasks.rados.rados.0.smithi100.stdout:2672: finishing write tid 5 to smithi10066867-41 2024-04-07T16:25:31.941 INFO:tasks.rados.rados.0.smithi100.stdout:2672: oid 41 updating version 0 to 1290 2024-04-07T16:25:31.941 INFO:tasks.rados.rados.0.smithi100.stdout:2672: oid 41 updating version 1290 to 1292 2024-04-07T16:25:31.941 INFO:tasks.rados.rados.0.smithi100.stdout:2672: oid 41 version 1292 is already newer than 1291 2024-04-07T16:25:31.942 INFO:tasks.rados.rados.0.smithi100.stdout:2672: oid 41 updating version 1292 to 1293 2024-04-07T16:25:31.942 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 41 v 1293 (ObjNum 805 snap 253 seq_num 805) dirty exists 2024-04-07T16:25:31.942 INFO:tasks.rados.rados.0.smithi100.stdout:2672: left oid 41 (ObjNum 805 snap 253 seq_num 805) 2024-04-07T16:25:31.942 INFO:tasks.rados.rados.0.smithi100.stdout:2677: finishing write tid 1 to smithi10066867-4 2024-04-07T16:25:31.944 INFO:tasks.rados.rados.0.smithi100.stdout:2677: finishing write tid 2 to smithi10066867-4 2024-04-07T16:25:31.945 INFO:tasks.rados.rados.0.smithi100.stdout:2677: finishing write tid 3 to smithi10066867-4 2024-04-07T16:25:31.946 INFO:tasks.rados.rados.0.smithi100.stdout:2677: finishing write tid 4 to smithi10066867-4 2024-04-07T16:25:31.948 INFO:tasks.rados.rados.0.smithi100.stdout:2677: finishing write tid 5 to smithi10066867-4 2024-04-07T16:25:31.948 INFO:tasks.rados.rados.0.smithi100.stdout:2677: oid 4 updating version 0 to 1521 2024-04-07T16:25:31.948 INFO:tasks.rados.rados.0.smithi100.stdout:2677: oid 4 version 1521 is already newer than 1519 2024-04-07T16:25:31.948 INFO:tasks.rados.rados.0.smithi100.stdout:2677: oid 4 version 1521 is already newer than 1518 2024-04-07T16:25:31.948 INFO:tasks.rados.rados.0.smithi100.stdout:2677: oid 4 version 1521 is already newer than 1520 2024-04-07T16:25:31.948 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 4 v 1521 (ObjNum 807 snap 253 seq_num 807) dirty exists 2024-04-07T16:25:31.948 INFO:tasks.rados.rados.0.smithi100.stdout:2677: left oid 4 (ObjNum 807 snap 253 seq_num 807) 2024-04-07T16:25:32.312 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:31 smithi144 bash[66654]: cluster 2024-04-07T16:25:30.732665+0000 mgr.y (mgr.25175) 260 : cluster [DBG] pgmap v454: 121 pgs: 11 active+undersized+degraded, 39 active+undersized, 71 active+clean; 171 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 8.3 MiB/s rd, 1.9 MiB/s wr, 30 op/s; 27/345 objects degraded (7.826%) 2024-04-07T16:25:32.313 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:31 smithi144 bash[66654]: cluster 2024-04-07T16:25:30.996471+0000 mon.a (mon.0) 844 : cluster [DBG] osdmap e575: 8 total, 7 up, 8 in 2024-04-07T16:25:32.313 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:31 smithi144 bash[66654]: audit 2024-04-07T16:25:31.724379+0000 mon.a (mon.0) 845 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:25:32.313 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:31 smithi144 bash[66654]: audit 2024-04-07T16:25:31.726544+0000 mon.a (mon.0) 846 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:25:32.313 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:31 smithi144 bash[66654]: cluster 2024-04-07T16:25:31.877379+0000 mon.a (mon.0) 847 : cluster [DBG] osdmap e576: 8 total, 7 up, 8 in 2024-04-07T16:25:32.435 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:31 smithi100 bash[74942]: cluster 2024-04-07T16:25:30.732665+0000 mgr.y (mgr.25175) 260 : cluster [DBG] pgmap v454: 121 pgs: 11 active+undersized+degraded, 39 active+undersized, 71 active+clean; 171 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 8.3 MiB/s rd, 1.9 MiB/s wr, 30 op/s; 27/345 objects degraded (7.826%) 2024-04-07T16:25:32.435 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:31 smithi100 bash[74942]: cluster 2024-04-07T16:25:30.996471+0000 mon.a (mon.0) 844 : cluster [DBG] osdmap e575: 8 total, 7 up, 8 in 2024-04-07T16:25:32.435 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:31 smithi100 bash[74942]: audit 2024-04-07T16:25:31.724379+0000 mon.a (mon.0) 845 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:25:32.435 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:31 smithi100 bash[74942]: audit 2024-04-07T16:25:31.726544+0000 mon.a (mon.0) 846 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:25:32.435 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:31 smithi100 bash[74942]: cluster 2024-04-07T16:25:31.877379+0000 mon.a (mon.0) 847 : cluster [DBG] osdmap e576: 8 total, 7 up, 8 in 2024-04-07T16:25:32.436 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:31 smithi100 bash[76202]: cluster 2024-04-07T16:25:30.732665+0000 mgr.y (mgr.25175) 260 : cluster [DBG] pgmap v454: 121 pgs: 11 active+undersized+degraded, 39 active+undersized, 71 active+clean; 171 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 8.3 MiB/s rd, 1.9 MiB/s wr, 30 op/s; 27/345 objects degraded (7.826%) 2024-04-07T16:25:32.436 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:31 smithi100 bash[76202]: cluster 2024-04-07T16:25:30.996471+0000 mon.a (mon.0) 844 : cluster [DBG] osdmap e575: 8 total, 7 up, 8 in 2024-04-07T16:25:32.436 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:31 smithi100 bash[76202]: audit 2024-04-07T16:25:31.724379+0000 mon.a (mon.0) 845 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:25:32.436 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:31 smithi100 bash[76202]: audit 2024-04-07T16:25:31.726544+0000 mon.a (mon.0) 846 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:25:32.436 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:31 smithi100 bash[76202]: cluster 2024-04-07T16:25:31.877379+0000 mon.a (mon.0) 847 : cluster [DBG] osdmap e576: 8 total, 7 up, 8 in 2024-04-07T16:25:32.606 INFO:journalctl@ceph.osd.7.smithi144.stdout:Apr 07 16:25:32 smithi144 bash[84215]: debug 2024-04-07T16:25:32.449+0000 7f7873ed9700 1 -- 172.21.15.144:0/3073134506 <== mon.1 v2:172.21.15.144:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ed5b371680 con 0x55ed5b3aec00 2024-04-07T16:25:32.606 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:25:32 smithi144 bash[59794]: ts=2024-04-07T16:25:32.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:25:32.865 INFO:tasks.rados.rados.0.smithi100.stdout:2672: done (5 left) 2024-04-07T16:25:32.865 INFO:tasks.rados.rados.0.smithi100.stdout:2674: done (4 left) 2024-04-07T16:25:32.865 INFO:tasks.rados.rados.0.smithi100.stdout:2675: done (3 left) 2024-04-07T16:25:32.865 INFO:tasks.rados.rados.0.smithi100.stdout:2676: done (2 left) 2024-04-07T16:25:32.866 INFO:tasks.rados.rados.0.smithi100.stdout:2677: done (1 left) 2024-04-07T16:25:32.866 INFO:tasks.rados.rados.0.smithi100.stdout:2678: done (0 left) 2024-04-07T16:25:32.866 INFO:tasks.rados.rados.0.smithi100.stdout:2679: read oid 19 snap -1 2024-04-07T16:25:32.866 INFO:tasks.rados.rados.0.smithi100.stdout:2679: expect (ObjNum 786 snap 247 seq_num 786) 2024-04-07T16:25:32.866 INFO:tasks.rados.rados.0.smithi100.stdout:2680: snap_remove snap 239 2024-04-07T16:25:33.356 INFO:journalctl@ceph.osd.7.smithi144.stdout:Apr 07 16:25:32 smithi144 bash[84215]: debug 2024-04-07T16:25:32.985+0000 7f787d0717c0 -1 Falling back to public interface 2024-04-07T16:25:33.874 INFO:tasks.rados.rados.0.smithi100.stdout:2680: done (1 left) 2024-04-07T16:25:33.874 INFO:tasks.rados.rados.0.smithi100.stdout:2681: snap_create 2024-04-07T16:25:33.874 INFO:tasks.rados.rados.0.smithi100.stdout:2679: expect (ObjNum 786 snap 247 seq_num 786) 2024-04-07T16:25:34.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:33 smithi100 bash[74942]: cluster 2024-04-07T16:25:32.733869+0000 mgr.y (mgr.25175) 261 : cluster [DBG] pgmap v457: 121 pgs: 11 active+undersized+degraded, 39 active+undersized, 71 active+clean; 176 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 8.3 MiB/s rd, 2.7 MiB/s wr, 35 op/s; 30/360 objects degraded (8.333%) 2024-04-07T16:25:34.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:33 smithi100 bash[74942]: cluster 2024-04-07T16:25:32.880383+0000 mon.a (mon.0) 848 : cluster [DBG] osdmap e577: 8 total, 7 up, 8 in 2024-04-07T16:25:34.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:33 smithi100 bash[76202]: cluster 2024-04-07T16:25:32.733869+0000 mgr.y (mgr.25175) 261 : cluster [DBG] pgmap v457: 121 pgs: 11 active+undersized+degraded, 39 active+undersized, 71 active+clean; 176 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 8.3 MiB/s rd, 2.7 MiB/s wr, 35 op/s; 30/360 objects degraded (8.333%) 2024-04-07T16:25:34.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:33 smithi100 bash[76202]: cluster 2024-04-07T16:25:32.880383+0000 mon.a (mon.0) 848 : cluster [DBG] osdmap e577: 8 total, 7 up, 8 in 2024-04-07T16:25:34.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:33 smithi144 bash[66654]: cluster 2024-04-07T16:25:32.733869+0000 mgr.y (mgr.25175) 261 : cluster [DBG] pgmap v457: 121 pgs: 11 active+undersized+degraded, 39 active+undersized, 71 active+clean; 176 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 8.3 MiB/s rd, 2.7 MiB/s wr, 35 op/s; 30/360 objects degraded (8.333%) 2024-04-07T16:25:34.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:33 smithi144 bash[66654]: cluster 2024-04-07T16:25:32.880383+0000 mon.a (mon.0) 848 : cluster [DBG] osdmap e577: 8 total, 7 up, 8 in 2024-04-07T16:25:34.885 INFO:tasks.rados.rados.0.smithi100.stdout:2679: done (1 left) 2024-04-07T16:25:34.885 INFO:tasks.rados.rados.0.smithi100.stdout:2681: done (0 left) 2024-04-07T16:25:34.885 INFO:tasks.rados.rados.0.smithi100.stdout:2682: copy_from oid 50 from oid 43 current snap is 255 2024-04-07T16:25:34.885 INFO:tasks.rados.rados.0.smithi100.stdout:2683: snap_create 2024-04-07T16:25:34.958 INFO:tasks.rados.rados.0.smithi100.stdout:2682: finishing copy_from racing read to smithi10066867-50 2024-04-07T16:25:34.961 INFO:tasks.rados.rados.0.smithi100.stdout:2682: finishing copy_from to smithi10066867-50 2024-04-07T16:25:34.961 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 50 v 1115 (ObjNum 691 snap 211 seq_num 691) dirty exists 2024-04-07T16:25:35.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:34 smithi100 bash[74942]: cluster 2024-04-07T16:25:33.890500+0000 mon.a (mon.0) 849 : cluster [DBG] osdmap e578: 8 total, 7 up, 8 in 2024-04-07T16:25:35.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:34 smithi100 bash[76202]: cluster 2024-04-07T16:25:33.890500+0000 mon.a (mon.0) 849 : cluster [DBG] osdmap e578: 8 total, 7 up, 8 in 2024-04-07T16:25:35.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:34 smithi144 bash[66654]: cluster 2024-04-07T16:25:33.890500+0000 mon.a (mon.0) 849 : cluster [DBG] osdmap e578: 8 total, 7 up, 8 in 2024-04-07T16:25:35.886 INFO:tasks.rados.rados.0.smithi100.stdout:2682: done (1 left) 2024-04-07T16:25:35.886 INFO:tasks.rados.rados.0.smithi100.stdout:2683: done (0 left) 2024-04-07T16:25:35.886 INFO:tasks.rados.rados.0.smithi100.stdout:2684: rollback oid 44 current snap is 256 2024-04-07T16:25:35.886 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 44 to 247 2024-04-07T16:25:35.886 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 42 current snap is 256 2024-04-07T16:25:35.886 INFO:tasks.rados.rados.0.smithi100.stdout:2685: seq_num 808 ranges {991232=737280,1728512=614400,2342912=188416} 2024-04-07T16:25:35.907 INFO:tasks.rados.rados.0.smithi100.stdout:2685: writing smithi10066867-42 from 991232 to 1728512 tid 1 2024-04-07T16:25:35.914 INFO:tasks.rados.rados.0.smithi100.stdout:2685: writing smithi10066867-42 from 1728512 to 2342912 tid 2 2024-04-07T16:25:35.917 INFO:tasks.rados.rados.0.smithi100.stdout:2685: writing smithi10066867-42 from 2342912 to 2531328 tid 3 2024-04-07T16:25:35.917 INFO:tasks.rados.rados.0.smithi100.stdout:2686: rollback oid 5 current snap is 256 2024-04-07T16:25:35.917 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 5 to 249 2024-04-07T16:25:35.917 INFO:tasks.rados.rados.0.smithi100.stdout:2684: finishing rollback tid 1 to smithi10066867-44 2024-04-07T16:25:35.917 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 50 current snap is 256 2024-04-07T16:25:35.917 INFO:tasks.rados.rados.0.smithi100.stdout:2684: finishing rollback tid 2 to smithi10066867-44 2024-04-07T16:25:35.917 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 44 v 1203 (ObjNum 509 snap 152 seq_num 509) dirty exists 2024-04-07T16:25:35.917 INFO:tasks.rados.rados.0.smithi100.stdout:2687: seq_num 809 ranges {8593408=753664,9347072=499712,9846784=303104} 2024-04-07T16:25:35.991 INFO:tasks.rados.rados.0.smithi100.stdout:2687: writing smithi10066867-50 from 8593408 to 9347072 tid 1 2024-04-07T16:25:35.994 INFO:tasks.rados.rados.0.smithi100.stdout:2687: writing smithi10066867-50 from 9347072 to 9846784 tid 2 2024-04-07T16:25:35.996 INFO:tasks.rados.rados.0.smithi100.stdout:2687: writing smithi10066867-50 from 9846784 to 10149888 tid 3 2024-04-07T16:25:35.996 INFO:tasks.rados.rados.0.smithi100.stdout:2684: done (3 left) 2024-04-07T16:25:35.996 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 9 current snap is 256 2024-04-07T16:25:35.996 INFO:tasks.rados.rados.0.smithi100.stdout:2688: seq_num 810 ranges {0=770048,770048=778240,1548288=24576} 2024-04-07T16:25:36.000 INFO:tasks.rados.rados.0.smithi100.stdout:2688: writing smithi10066867-9 from 0 to 770048 tid 1 2024-04-07T16:25:36.005 INFO:tasks.rados.rados.0.smithi100.stdout:2688: writing smithi10066867-9 from 770048 to 1548288 tid 2 2024-04-07T16:25:36.005 INFO:tasks.rados.rados.0.smithi100.stdout:2688: writing smithi10066867-9 from 1548288 to 1572864 tid 3 2024-04-07T16:25:36.006 INFO:tasks.rados.rados.0.smithi100.stdout:2689: read oid 19 snap -1 2024-04-07T16:25:36.006 INFO:tasks.rados.rados.0.smithi100.stdout:2689: expect (ObjNum 786 snap 247 seq_num 786) 2024-04-07T16:25:36.006 INFO:tasks.rados.rados.0.smithi100.stdout:2690: delete oid 36 current snap is 256 2024-04-07T16:25:36.006 INFO:tasks.rados.rados.0.smithi100.stdout:2686: finishing rollback tid 1 to smithi10066867-5 2024-04-07T16:25:36.006 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 5 v 0 (ObjNum 129 snap 0 seq_num 2701868064) dirty dne 2024-04-07T16:25:36.006 INFO:tasks.rados.rados.0.smithi100.stdout:2685: finishing write tid 1 to smithi10066867-42 2024-04-07T16:25:36.006 INFO:tasks.rados.rados.0.smithi100.stdout:2685: finishing write tid 2 to smithi10066867-42 2024-04-07T16:25:36.006 INFO:tasks.rados.rados.0.smithi100.stdout:2685: finishing write tid 3 to smithi10066867-42 2024-04-07T16:25:36.006 INFO:tasks.rados.rados.0.smithi100.stdout:2685: finishing write tid 4 to smithi10066867-42 2024-04-07T16:25:36.006 INFO:tasks.rados.rados.0.smithi100.stdout:2685: finishing write tid 5 to smithi10066867-42 2024-04-07T16:25:36.006 INFO:tasks.rados.rados.0.smithi100.stdout:2685: oid 42 updating version 0 to 1192 2024-04-07T16:25:36.006 INFO:tasks.rados.rados.0.smithi100.stdout:2685: oid 42 version 1192 is already newer than 1190 2024-04-07T16:25:36.006 INFO:tasks.rados.rados.0.smithi100.stdout:2685: oid 42 version 1192 is already newer than 1191 2024-04-07T16:25:36.007 INFO:tasks.rados.rados.0.smithi100.stdout:2685: oid 42 version 1192 is already newer than 1189 2024-04-07T16:25:36.007 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 42 v 1192 (ObjNum 808 snap 256 seq_num 808) dirty exists 2024-04-07T16:25:36.007 INFO:tasks.rados.rados.0.smithi100.stdout:2685: left oid 42 (ObjNum 808 snap 256 seq_num 808) 2024-04-07T16:25:36.012 INFO:tasks.rados.rados.0.smithi100.stdout:2688: finishing write tid 1 to smithi10066867-9 2024-04-07T16:25:36.013 INFO:tasks.rados.rados.0.smithi100.stdout:2687: finishing write tid 1 to smithi10066867-50 2024-04-07T16:25:36.013 INFO:tasks.rados.rados.0.smithi100.stdout:2687: finishing write tid 2 to smithi10066867-50 2024-04-07T16:25:36.015 INFO:tasks.rados.rados.0.smithi100.stdout:2688: finishing write tid 2 to smithi10066867-9 2024-04-07T16:25:36.018 INFO:tasks.rados.rados.0.smithi100.stdout:2688: finishing write tid 3 to smithi10066867-9 2024-04-07T16:25:36.018 INFO:tasks.rados.rados.0.smithi100.stdout:2687: finishing write tid 3 to smithi10066867-50 2024-04-07T16:25:36.019 INFO:tasks.rados.rados.0.smithi100.stdout:2687: finishing write tid 4 to smithi10066867-50 2024-04-07T16:25:36.019 INFO:tasks.rados.rados.0.smithi100.stdout:2688: finishing write tid 4 to smithi10066867-9 2024-04-07T16:25:36.021 INFO:tasks.rados.rados.0.smithi100.stdout:2689: expect (ObjNum 786 snap 247 seq_num 786) 2024-04-07T16:25:36.068 INFO:tasks.rados.rados.0.smithi100.stdout:2687: finishing write tid 5 to smithi10066867-50 2024-04-07T16:25:36.068 INFO:tasks.rados.rados.0.smithi100.stdout:2687: oid 50 updating version 0 to 1118 2024-04-07T16:25:36.069 INFO:tasks.rados.rados.0.smithi100.stdout:2687: oid 50 version 1118 is already newer than 1116 2024-04-07T16:25:36.069 INFO:tasks.rados.rados.0.smithi100.stdout:2687: oid 50 updating version 1118 to 1119 2024-04-07T16:25:36.069 INFO:tasks.rados.rados.0.smithi100.stdout:2687: oid 50 version 1119 is already newer than 1117 2024-04-07T16:25:36.069 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 50 v 1119 (ObjNum 809 snap 256 seq_num 809) dirty exists 2024-04-07T16:25:36.069 INFO:tasks.rados.rados.0.smithi100.stdout:2687: left oid 50 (ObjNum 809 snap 256 seq_num 809) 2024-04-07T16:25:36.069 INFO:tasks.rados.rados.0.smithi100.stdout:2688: finishing write tid 5 to smithi10066867-9 2024-04-07T16:25:36.069 INFO:tasks.rados.rados.0.smithi100.stdout:2688: oid 9 updating version 0 to 1114 2024-04-07T16:25:36.069 INFO:tasks.rados.rados.0.smithi100.stdout:2688: oid 9 updating version 1114 to 1116 2024-04-07T16:25:36.069 INFO:tasks.rados.rados.0.smithi100.stdout:2688: oid 9 updating version 1116 to 1117 2024-04-07T16:25:36.069 INFO:tasks.rados.rados.0.smithi100.stdout:2688: oid 9 version 1117 is already newer than 1115 2024-04-07T16:25:36.069 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 9 v 1117 (ObjNum 810 snap 256 seq_num 810) dirty exists 2024-04-07T16:25:36.069 INFO:tasks.rados.rados.0.smithi100.stdout:2688: left oid 9 (ObjNum 810 snap 256 seq_num 810) 2024-04-07T16:25:36.069 INFO:tasks.rados.rados.0.smithi100.stdout:2685: done (5 left) 2024-04-07T16:25:36.069 INFO:tasks.rados.rados.0.smithi100.stdout:2686: done (4 left) 2024-04-07T16:25:36.069 INFO:tasks.rados.rados.0.smithi100.stdout:2687: done (3 left) 2024-04-07T16:25:36.070 INFO:tasks.rados.rados.0.smithi100.stdout:2688: done (2 left) 2024-04-07T16:25:36.070 INFO:tasks.rados.rados.0.smithi100.stdout:2689: done (1 left) 2024-04-07T16:25:36.070 INFO:tasks.rados.rados.0.smithi100.stdout:2690: done (0 left) 2024-04-07T16:25:36.070 INFO:tasks.rados.rados.0.smithi100.stdout:2691: read oid 30 snap -1 2024-04-07T16:25:36.070 INFO:tasks.rados.rados.0.smithi100.stdout:2691: expect (ObjNum 722 snap 223 seq_num 722) 2024-04-07T16:25:36.070 INFO:tasks.rados.rados.0.smithi100.stdout:2692: rollback oid 41 current snap is 256 2024-04-07T16:25:36.070 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 41 to 251 2024-04-07T16:25:36.070 INFO:tasks.rados.rados.0.smithi100.stdout:2693: read oid 31 snap -1 2024-04-07T16:25:36.070 INFO:tasks.rados.rados.0.smithi100.stdout:2693: expect (ObjNum 579 snap 180 seq_num 579) 2024-04-07T16:25:36.070 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 6 current snap is 256 2024-04-07T16:25:36.070 INFO:tasks.rados.rados.0.smithi100.stdout:2694: seq_num 812 ranges {6144000=802816,6946816=548864,7495680=253952} 2024-04-07T16:25:36.110 INFO:tasks.rados.rados.0.smithi100.stdout:2694: writing smithi10066867-6 from 6144000 to 6946816 tid 1 2024-04-07T16:25:36.114 INFO:tasks.rados.rados.0.smithi100.stdout:2694: writing smithi10066867-6 from 6946816 to 7495680 tid 2 2024-04-07T16:25:36.116 INFO:tasks.rados.rados.0.smithi100.stdout:2694: writing smithi10066867-6 from 7495680 to 7749632 tid 3 2024-04-07T16:25:36.116 INFO:tasks.rados.rados.0.smithi100.stdout:2695: delete oid 13 current snap is 256 2024-04-07T16:25:36.116 INFO:tasks.rados.rados.0.smithi100.stdout:2692: finishing rollback tid 0 to smithi10066867-41 2024-04-07T16:25:36.116 INFO:tasks.rados.rados.0.smithi100.stdout:2692: finishing rollback tid 1 to smithi10066867-41 2024-04-07T16:25:36.116 INFO:tasks.rados.rados.0.smithi100.stdout:2692: finishing rollback tid 2 to smithi10066867-41 2024-04-07T16:25:36.116 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 41 v 1307 (ObjNum 758 snap 239 seq_num 758) dirty exists 2024-04-07T16:25:36.116 INFO:tasks.rados.rados.0.smithi100.stdout:2693: expect (ObjNum 579 snap 180 seq_num 579) 2024-04-07T16:25:36.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:35 smithi100 bash[74942]: cluster 2024-04-07T16:25:34.734937+0000 mgr.y (mgr.25175) 262 : cluster [DBG] pgmap v460: 121 pgs: 11 active+undersized+degraded, 39 active+undersized, 71 active+clean; 181 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 8.8 MiB/s rd, 2.8 MiB/s wr, 19 op/s; 31/372 objects degraded (8.333%) 2024-04-07T16:25:36.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:35 smithi100 bash[74942]: cluster 2024-04-07T16:25:34.880757+0000 mon.a (mon.0) 850 : cluster [WRN] Health check update: Degraded data redundancy: 31/372 objects degraded (8.333%), 11 pgs degraded (PG_DEGRADED) 2024-04-07T16:25:36.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:35 smithi100 bash[74942]: cluster 2024-04-07T16:25:34.901275+0000 mon.a (mon.0) 851 : cluster [DBG] osdmap e579: 8 total, 7 up, 8 in 2024-04-07T16:25:36.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:35 smithi100 bash[76202]: cluster 2024-04-07T16:25:34.734937+0000 mgr.y (mgr.25175) 262 : cluster [DBG] pgmap v460: 121 pgs: 11 active+undersized+degraded, 39 active+undersized, 71 active+clean; 181 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 8.8 MiB/s rd, 2.8 MiB/s wr, 19 op/s; 31/372 objects degraded (8.333%) 2024-04-07T16:25:36.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:35 smithi100 bash[76202]: cluster 2024-04-07T16:25:34.880757+0000 mon.a (mon.0) 850 : cluster [WRN] Health check update: Degraded data redundancy: 31/372 objects degraded (8.333%), 11 pgs degraded (PG_DEGRADED) 2024-04-07T16:25:36.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:35 smithi100 bash[76202]: cluster 2024-04-07T16:25:34.901275+0000 mon.a (mon.0) 851 : cluster [DBG] osdmap e579: 8 total, 7 up, 8 in 2024-04-07T16:25:36.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:35 smithi144 bash[66654]: cluster 2024-04-07T16:25:34.734937+0000 mgr.y (mgr.25175) 262 : cluster [DBG] pgmap v460: 121 pgs: 11 active+undersized+degraded, 39 active+undersized, 71 active+clean; 181 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 8.8 MiB/s rd, 2.8 MiB/s wr, 19 op/s; 31/372 objects degraded (8.333%) 2024-04-07T16:25:36.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:35 smithi144 bash[66654]: cluster 2024-04-07T16:25:34.880757+0000 mon.a (mon.0) 850 : cluster [WRN] Health check update: Degraded data redundancy: 31/372 objects degraded (8.333%), 11 pgs degraded (PG_DEGRADED) 2024-04-07T16:25:36.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:35 smithi144 bash[66654]: cluster 2024-04-07T16:25:34.901275+0000 mon.a (mon.0) 851 : cluster [DBG] osdmap e579: 8 total, 7 up, 8 in 2024-04-07T16:25:36.409 INFO:tasks.rados.rados.0.smithi100.stdout:2691: expect (ObjNum 722 snap 223 seq_num 722) 2024-04-07T16:25:36.547 INFO:teuthology.orchestra.run.smithi100.stdout:true 2024-04-07T16:25:36.653 INFO:tasks.rados.rados.0.smithi100.stdout:2694: finishing write tid 1 to smithi10066867-6 2024-04-07T16:25:36.653 INFO:tasks.rados.rados.0.smithi100.stdout:2694: finishing write tid 2 to smithi10066867-6 2024-04-07T16:25:36.653 INFO:tasks.rados.rados.0.smithi100.stdout:2694: finishing write tid 3 to smithi10066867-6 2024-04-07T16:25:36.653 INFO:tasks.rados.rados.0.smithi100.stdout:2694: finishing write tid 4 to smithi10066867-6 2024-04-07T16:25:36.654 INFO:tasks.rados.rados.0.smithi100.stdout:2694: finishing write tid 5 to smithi10066867-6 2024-04-07T16:25:36.654 INFO:tasks.rados.rados.0.smithi100.stdout:2694: oid 6 updating version 0 to 1205 2024-04-07T16:25:36.654 INFO:tasks.rados.rados.0.smithi100.stdout:2694: oid 6 updating version 1205 to 1207 2024-04-07T16:25:36.654 INFO:tasks.rados.rados.0.smithi100.stdout:2694: oid 6 updating version 1207 to 1208 2024-04-07T16:25:36.654 INFO:tasks.rados.rados.0.smithi100.stdout:2694: oid 6 version 1208 is already newer than 1206 2024-04-07T16:25:36.654 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 6 v 1208 (ObjNum 812 snap 256 seq_num 812) dirty exists 2024-04-07T16:25:36.654 INFO:tasks.rados.rados.0.smithi100.stdout:2694: left oid 6 (ObjNum 812 snap 256 seq_num 812) 2024-04-07T16:25:36.654 INFO:tasks.rados.rados.0.smithi100.stdout:2691: done (4 left) 2024-04-07T16:25:36.654 INFO:tasks.rados.rados.0.smithi100.stdout:2692: done (3 left) 2024-04-07T16:25:36.654 INFO:tasks.rados.rados.0.smithi100.stdout:2693: done (2 left) 2024-04-07T16:25:36.654 INFO:tasks.rados.rados.0.smithi100.stdout:2694: done (1 left) 2024-04-07T16:25:36.654 INFO:tasks.rados.rados.0.smithi100.stdout:2695: done (0 left) 2024-04-07T16:25:36.654 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 30 current snap is 256 2024-04-07T16:25:36.654 INFO:tasks.rados.rados.0.smithi100.stdout:2696: seq_num 814 ranges {4751360=442368,5193728=770048,5963776=434176} 2024-04-07T16:25:36.684 INFO:tasks.rados.rados.0.smithi100.stdout:2696: writing smithi10066867-30 from 4751360 to 5193728 tid 1 2024-04-07T16:25:36.690 INFO:tasks.rados.rados.0.smithi100.stdout:2696: writing smithi10066867-30 from 5193728 to 5963776 tid 2 2024-04-07T16:25:36.693 INFO:tasks.rados.rados.0.smithi100.stdout:2696: writing smithi10066867-30 from 5963776 to 6397952 tid 3 2024-04-07T16:25:36.693 INFO:tasks.rados.rados.0.smithi100.stdout:2697: delete oid 35 current snap is 256 2024-04-07T16:25:36.693 INFO:tasks.rados.rados.0.smithi100.stdout:2696: finishing write tid 1 to smithi10066867-30 2024-04-07T16:25:36.693 INFO:tasks.rados.rados.0.smithi100.stdout:2696: finishing write tid 2 to smithi10066867-30 2024-04-07T16:25:36.695 INFO:tasks.rados.rados.0.smithi100.stdout:2697: done (1 left) 2024-04-07T16:25:36.695 INFO:tasks.rados.rados.0.smithi100.stdout:2698: rollback oid 39 current snap is 256 2024-04-07T16:25:36.695 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 39 to 247 2024-04-07T16:25:36.695 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 47 current snap is 256 2024-04-07T16:25:36.695 INFO:tasks.rados.rados.0.smithi100.stdout:2699: seq_num 816 ranges {1269760=475136,1744896=540672,2285568=614400,2899968=49152} 2024-04-07T16:25:36.705 INFO:tasks.rados.rados.0.smithi100.stdout:2699: writing smithi10066867-47 from 1269760 to 1744896 tid 1 2024-04-07T16:25:36.708 INFO:tasks.rados.rados.0.smithi100.stdout:2699: writing smithi10066867-47 from 1744896 to 2285568 tid 2 2024-04-07T16:25:36.711 INFO:tasks.rados.rados.0.smithi100.stdout:2699: writing smithi10066867-47 from 2285568 to 2899968 tid 3 2024-04-07T16:25:36.713 INFO:tasks.rados.rados.0.smithi100.stdout:2699: writing smithi10066867-47 from 2899968 to 2949120 tid 4 2024-04-07T16:25:36.713 INFO:tasks.rados.rados.0.smithi100.stdout:2700: read oid 7 snap -1 2024-04-07T16:25:36.713 INFO:tasks.rados.rados.0.smithi100.stdout:2700: expect (ObjNum 752 snap 234 seq_num 752) 2024-04-07T16:25:36.713 INFO:tasks.rados.rados.0.smithi100.stdout:2701: read oid 4 snap -1 2024-04-07T16:25:36.713 INFO:tasks.rados.rados.0.smithi100.stdout:2701: expect (ObjNum 807 snap 253 seq_num 807) 2024-04-07T16:25:36.713 INFO:tasks.rados.rados.0.smithi100.stdout:2702: copy_from oid 5 from oid 28 current snap is 256 2024-04-07T16:25:36.713 INFO:tasks.rados.rados.0.smithi100.stdout:2696: finishing write tid 3 to smithi10066867-30 2024-04-07T16:25:36.713 INFO:tasks.rados.rados.0.smithi100.stdout:2696: finishing write tid 4 to smithi10066867-30 2024-04-07T16:25:36.713 INFO:tasks.rados.rados.0.smithi100.stdout:2698: finishing rollback tid 0 to smithi10066867-39 2024-04-07T16:25:36.713 INFO:tasks.rados.rados.0.smithi100.stdout:2696: finishing write tid 5 to smithi10066867-30 2024-04-07T16:25:36.713 INFO:tasks.rados.rados.0.smithi100.stdout:2696: oid 30 updating version 0 to 1195 2024-04-07T16:25:36.713 INFO:tasks.rados.rados.0.smithi100.stdout:2696: oid 30 updating version 1195 to 1196 2024-04-07T16:25:36.714 INFO:tasks.rados.rados.0.smithi100.stdout:2696: oid 30 version 1196 is already newer than 1194 2024-04-07T16:25:36.714 INFO:tasks.rados.rados.0.smithi100.stdout:2696: oid 30 version 1196 is already newer than 1193 2024-04-07T16:25:36.714 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 30 v 1196 (ObjNum 814 snap 256 seq_num 814) dirty exists 2024-04-07T16:25:36.714 INFO:tasks.rados.rados.0.smithi100.stdout:2696: left oid 30 (ObjNum 814 snap 256 seq_num 814) 2024-04-07T16:25:36.714 INFO:tasks.rados.rados.0.smithi100.stdout:2698: finishing rollback tid 1 to smithi10066867-39 2024-04-07T16:25:36.714 INFO:tasks.rados.rados.0.smithi100.stdout:2698: finishing rollback tid 2 to smithi10066867-39 2024-04-07T16:25:36.714 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 39 v 1199 (ObjNum 756 snap 238 seq_num 756) dirty exists 2024-04-07T16:25:36.714 INFO:tasks.rados.rados.0.smithi100.stdout:2699: finishing write tid 1 to smithi10066867-47 2024-04-07T16:25:36.714 INFO:tasks.rados.rados.0.smithi100.stdout:2696: done (5 left) 2024-04-07T16:25:36.714 INFO:tasks.rados.rados.0.smithi100.stdout:2698: done (4 left) 2024-04-07T16:25:36.714 INFO:tasks.rados.rados.0.smithi100.stdout:2703: snap_create 2024-04-07T16:25:36.715 INFO:tasks.rados.rados.0.smithi100.stdout:2699: finishing write tid 2 to smithi10066867-47 2024-04-07T16:25:36.715 INFO:tasks.rados.rados.0.smithi100.stdout:2702: finishing copy_from to smithi10066867-5 2024-04-07T16:25:36.715 INFO:tasks.rados.rados.0.smithi100.stdout:2702: got expected ENOENT (src dne) 2024-04-07T16:25:36.715 INFO:tasks.rados.rados.0.smithi100.stdout:2702: finishing copy_from racing read to smithi10066867-5 2024-04-07T16:25:36.716 INFO:tasks.rados.rados.0.smithi100.stdout:2699: finishing write tid 3 to smithi10066867-47 2024-04-07T16:25:36.717 INFO:tasks.rados.rados.0.smithi100.stdout:2699: finishing write tid 4 to smithi10066867-47 2024-04-07T16:25:36.717 INFO:tasks.rados.rados.0.smithi100.stdout:2699: finishing write tid 5 to smithi10066867-47 2024-04-07T16:25:36.720 INFO:tasks.rados.rados.0.smithi100.stdout:2699: finishing write tid 6 to smithi10066867-47 2024-04-07T16:25:36.720 INFO:tasks.rados.rados.0.smithi100.stdout:2699: oid 47 updating version 0 to 1182 2024-04-07T16:25:36.720 INFO:tasks.rados.rados.0.smithi100.stdout:2699: oid 47 updating version 1182 to 1184 2024-04-07T16:25:36.720 INFO:tasks.rados.rados.0.smithi100.stdout:2699: oid 47 updating version 1184 to 1186 2024-04-07T16:25:36.720 INFO:tasks.rados.rados.0.smithi100.stdout:2699: oid 47 version 1186 is already newer than 1183 2024-04-07T16:25:36.720 INFO:tasks.rados.rados.0.smithi100.stdout:2699: oid 47 version 1186 is already newer than 1185 2024-04-07T16:25:36.720 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 47 v 1186 (ObjNum 816 snap 256 seq_num 816) dirty exists 2024-04-07T16:25:36.720 INFO:tasks.rados.rados.0.smithi100.stdout:2699: left oid 47 (ObjNum 816 snap 256 seq_num 816) 2024-04-07T16:25:36.727 INFO:tasks.rados.rados.0.smithi100.stdout:2701: expect (ObjNum 807 snap 253 seq_num 807) 2024-04-07T16:25:36.844 INFO:tasks.rados.rados.0.smithi100.stdout:2700: expect (ObjNum 752 snap 234 seq_num 752) 2024-04-07T16:25:36.855 INFO:journalctl@ceph.osd.7.smithi144.stdout:Apr 07 16:25:36 smithi144 bash[84215]: debug 2024-04-07T16:25:36.465+0000 7f787d0717c0 -1 osd.7 569 log_to_monitors true 2024-04-07T16:25:36.946 INFO:teuthology.orchestra.run.smithi100.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-07T16:25:36.946 INFO:teuthology.orchestra.run.smithi100.stdout:alertmanager.a smithi100 *:9093,9094 running (11m) 114s ago 12m 14.4M - 0.25.0 c8568f914cd2 5143075e7d26 2024-04-07T16:25:36.946 INFO:teuthology.orchestra.run.smithi100.stdout:grafana.a smithi144 *:3000 running (11m) 29s ago 11m 79.8M - 9.4.7 954c08fa6188 7180c0708acd 2024-04-07T16:25:36.946 INFO:teuthology.orchestra.run.smithi100.stdout:mds.a smithi100 running (13m) 114s ago 13m 15.7M - 18.2.2-1171-g267c8ff5 8a4a01e7186c 92593b0c28ae 2024-04-07T16:25:36.946 INFO:teuthology.orchestra.run.smithi100.stdout:mds.b smithi144 running (13m) 29s ago 13m 20.3M - 18.2.2-1171-g267c8ff5 8a4a01e7186c 3df2ee3e17a4 2024-04-07T16:25:36.947 INFO:teuthology.orchestra.run.smithi100.stdout:mgr.x smithi144 *:8443,9283,8765 running (5m) 29s ago 20m 440M - 19.0.0-2785-gadce92d0 c0ecd4226ac9 233745c292a6 2024-04-07T16:25:36.947 INFO:teuthology.orchestra.run.smithi100.stdout:mgr.y smithi100 *:8443,9283,8765 running (6m) 114s ago 23m 503M - 19.0.0-2785-gadce92d0 c0ecd4226ac9 254dc65cdaa7 2024-04-07T16:25:36.947 INFO:teuthology.orchestra.run.smithi100.stdout:mon.a smithi100 running (5m) 114s ago 23m 54.3M 2048M 19.0.0-2785-gadce92d0 c0ecd4226ac9 718e12655c04 2024-04-07T16:25:36.947 INFO:teuthology.orchestra.run.smithi100.stdout:mon.b smithi144 running (4m) 29s ago 21m 45.4M 2048M 19.0.0-2785-gadce92d0 c0ecd4226ac9 aef817664bff 2024-04-07T16:25:36.947 INFO:teuthology.orchestra.run.smithi100.stdout:mon.c smithi100 running (4m) 114s ago 21m 46.2M 2048M 19.0.0-2785-gadce92d0 c0ecd4226ac9 63e889040f6e 2024-04-07T16:25:36.947 INFO:teuthology.orchestra.run.smithi100.stdout:node-exporter.a smithi100 *:9100 running (12m) 114s ago 12m 8955k - 1.5.0 0da6a335fe13 4bce264c1b26 2024-04-07T16:25:36.947 INFO:teuthology.orchestra.run.smithi100.stdout:node-exporter.b smithi144 *:9100 running (12m) 29s ago 12m 8420k - 1.5.0 0da6a335fe13 b6963413c560 2024-04-07T16:25:36.947 INFO:teuthology.orchestra.run.smithi100.stdout:osd.0 smithi100 running (3m) 114s ago 19m 199M 2524M 19.0.0-2785-gadce92d0 c0ecd4226ac9 a6e83c9d1abc 2024-04-07T16:25:36.947 INFO:teuthology.orchestra.run.smithi100.stdout:osd.1 smithi100 running (2m) 114s ago 19m 197M 2524M 19.0.0-2785-gadce92d0 c0ecd4226ac9 c50fe40079b6 2024-04-07T16:25:36.947 INFO:teuthology.orchestra.run.smithi100.stdout:osd.2 smithi100 running (2m) 114s ago 18m 253M 2524M 19.0.0-2785-gadce92d0 c0ecd4226ac9 8e631c43064b 2024-04-07T16:25:36.947 INFO:teuthology.orchestra.run.smithi100.stdout:osd.3 smithi100 running (2m) 114s ago 17m 11.8M 2524M 19.0.0-2785-gadce92d0 c0ecd4226ac9 402931247c79 2024-04-07T16:25:36.947 INFO:teuthology.orchestra.run.smithi100.stdout:osd.4 smithi144 running (90s) 29s ago 16m 249M 2524M 19.0.0-2785-gadce92d0 c0ecd4226ac9 ebbb7be95d33 2024-04-07T16:25:36.947 INFO:teuthology.orchestra.run.smithi100.stdout:osd.5 smithi144 running (63s) 29s ago 15m 138M 2524M 19.0.0-2785-gadce92d0 c0ecd4226ac9 ecce27bcb2f8 2024-04-07T16:25:36.948 INFO:teuthology.orchestra.run.smithi100.stdout:osd.6 smithi144 running (34s) 29s ago 15m 11.7M 2524M 19.0.0-2785-gadce92d0 c0ecd4226ac9 3788c5c3f427 2024-04-07T16:25:36.948 INFO:teuthology.orchestra.run.smithi100.stdout:osd.7 smithi144 starting - - - 2524M 2024-04-07T16:25:36.948 INFO:teuthology.orchestra.run.smithi100.stdout:prometheus.a smithi144 *:9095 running (6m) 29s ago 12m 53.1M - 2.43.0 a07b618ecd1d 2d0a8131f303 2024-04-07T16:25:37.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:36 smithi100 bash[76202]: cluster 2024-04-07T16:25:35.901322+0000 mon.a (mon.0) 852 : cluster [DBG] osdmap e580: 8 total, 7 up, 8 in 2024-04-07T16:25:37.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:36 smithi100 bash[76202]: audit 2024-04-07T16:25:36.473789+0000 mon.b (mon.1) 9 : audit [INF] from='osd.7 [v2:172.21.15.144:6824/3641044519,v1:172.21.15.144:6825/3641044519]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-07T16:25:37.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:36 smithi100 bash[76202]: audit 2024-04-07T16:25:36.481653+0000 mon.a (mon.0) 853 : audit [INF] from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-07T16:25:37.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:36 smithi100 bash[74942]: cluster 2024-04-07T16:25:35.901322+0000 mon.a (mon.0) 852 : cluster [DBG] osdmap e580: 8 total, 7 up, 8 in 2024-04-07T16:25:37.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:36 smithi100 bash[74942]: audit 2024-04-07T16:25:36.473789+0000 mon.b (mon.1) 9 : audit [INF] from='osd.7 [v2:172.21.15.144:6824/3641044519,v1:172.21.15.144:6825/3641044519]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-07T16:25:37.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:36 smithi100 bash[74942]: audit 2024-04-07T16:25:36.481653+0000 mon.a (mon.0) 853 : audit [INF] from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-07T16:25:37.356 INFO:journalctl@ceph.osd.7.smithi144.stdout:Apr 07 16:25:36 smithi144 bash[84215]: debug 2024-04-07T16:25:36.945+0000 7f786a4c1700 -1 osd.7 569 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-07T16:25:37.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:36 smithi144 bash[66654]: cluster 2024-04-07T16:25:35.901322+0000 mon.a (mon.0) 852 : cluster [DBG] osdmap e580: 8 total, 7 up, 8 in 2024-04-07T16:25:37.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:36 smithi144 bash[66654]: audit 2024-04-07T16:25:36.473789+0000 mon.b (mon.1) 9 : audit [INF] from='osd.7 [v2:172.21.15.144:6824/3641044519,v1:172.21.15.144:6825/3641044519]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-07T16:25:37.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:36 smithi144 bash[66654]: audit 2024-04-07T16:25:36.481653+0000 mon.a (mon.0) 853 : audit [INF] from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-07T16:25:37.365 INFO:tasks.rados.rados.0.smithi100.stdout:2699: done (4 left) 2024-04-07T16:25:37.365 INFO:tasks.rados.rados.0.smithi100.stdout:2700: done (3 left) 2024-04-07T16:25:37.366 INFO:tasks.rados.rados.0.smithi100.stdout:2701: done (2 left) 2024-04-07T16:25:37.366 INFO:tasks.rados.rados.0.smithi100.stdout:2702: done (1 left) 2024-04-07T16:25:37.366 INFO:tasks.rados.rados.0.smithi100.stdout:2703: done (0 left) 2024-04-07T16:25:37.366 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 50 current snap is 257 2024-04-07T16:25:37.366 INFO:tasks.rados.rados.0.smithi100.stdout:2704: seq_num 817 ranges {10149888=491520,10641408=417792,11059200=425984,11485184=360448} 2024-04-07T16:25:37.397 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-04-07T16:25:37.397 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": { 2024-04-07T16:25:37.397 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph version 19.0.0-2785-gadce92d0 (adce92d056533f22dc1ee37e3141d098b046ec1b) squid (dev)": 3 2024-04-07T16:25:37.397 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-07T16:25:37.397 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": { 2024-04-07T16:25:37.397 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph version 19.0.0-2785-gadce92d0 (adce92d056533f22dc1ee37e3141d098b046ec1b) squid (dev)": 2 2024-04-07T16:25:37.397 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-07T16:25:37.397 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": { 2024-04-07T16:25:37.397 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph version 19.0.0-2785-gadce92d0 (adce92d056533f22dc1ee37e3141d098b046ec1b) squid (dev)": 7 2024-04-07T16:25:37.398 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-07T16:25:37.398 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": { 2024-04-07T16:25:37.398 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph version 18.2.2-1171-g267c8ff5 (267c8ff5b1ff5815581b3b2381d19360e081b37b) reef (stable)": 2 2024-04-07T16:25:37.398 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-07T16:25:37.398 INFO:teuthology.orchestra.run.smithi100.stdout: "overall": { 2024-04-07T16:25:37.398 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph version 18.2.2-1171-g267c8ff5 (267c8ff5b1ff5815581b3b2381d19360e081b37b) reef (stable)": 2, 2024-04-07T16:25:37.398 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph version 19.0.0-2785-gadce92d0 (adce92d056533f22dc1ee37e3141d098b046ec1b) squid (dev)": 12 2024-04-07T16:25:37.398 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-04-07T16:25:37.398 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-04-07T16:25:37.445 INFO:tasks.rados.rados.0.smithi100.stdout:2704: writing smithi10066867-50 from 10149888 to 10641408 tid 1 2024-04-07T16:25:37.446 INFO:tasks.rados.rados.0.smithi100.stdout:2704: writing smithi10066867-50 from 10641408 to 11059200 tid 2 2024-04-07T16:25:37.449 INFO:tasks.rados.rados.0.smithi100.stdout:2704: writing smithi10066867-50 from 11059200 to 11485184 tid 3 2024-04-07T16:25:37.452 INFO:tasks.rados.rados.0.smithi100.stdout:2704: writing smithi10066867-50 from 11485184 to 11845632 tid 4 2024-04-07T16:25:37.452 INFO:tasks.rados.rados.0.smithi100.stdout:2705: read oid 39 snap 247 2024-04-07T16:25:37.452 INFO:tasks.rados.rados.0.smithi100.stdout:2705: expect (ObjNum 756 snap 238 seq_num 756) 2024-04-07T16:25:37.452 INFO:tasks.rados.rados.0.smithi100.stdout:2706: read oid 14 snap -1 2024-04-07T16:25:37.452 INFO:tasks.rados.rados.0.smithi100.stdout:2706: expect (ObjNum 762 snap 240 seq_num 762) 2024-04-07T16:25:37.453 INFO:tasks.rados.rados.0.smithi100.stdout:2707: copy_from oid 7 from oid 3 current snap is 257 2024-04-07T16:25:37.453 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 8 current snap is 257 2024-04-07T16:25:37.453 INFO:tasks.rados.rados.0.smithi100.stdout:2708: seq_num 818 ranges {548864=507904,1056768=704512,1761280=499712} 2024-04-07T16:25:37.458 INFO:tasks.rados.rados.0.smithi100.stdout:2708: writing smithi10066867-8 from 548864 to 1056768 tid 1 2024-04-07T16:25:37.463 INFO:tasks.rados.rados.0.smithi100.stdout:2708: writing smithi10066867-8 from 1056768 to 1761280 tid 2 2024-04-07T16:25:37.466 INFO:tasks.rados.rados.0.smithi100.stdout:2708: writing smithi10066867-8 from 1761280 to 2260992 tid 3 2024-04-07T16:25:37.467 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 1 current snap is 257 2024-04-07T16:25:37.467 INFO:tasks.rados.rados.0.smithi100.stdout:2704: finishing write tid 1 to smithi10066867-50 2024-04-07T16:25:37.467 INFO:tasks.rados.rados.0.smithi100.stdout:2709: seq_num 819 ranges {1474560=524288,1998848=589824,2588672=614400} 2024-04-07T16:25:37.478 INFO:tasks.rados.rados.0.smithi100.stdout:2709: writing smithi10066867-1 from 1474560 to 1998848 tid 1 2024-04-07T16:25:37.482 INFO:tasks.rados.rados.0.smithi100.stdout:2709: writing smithi10066867-1 from 1998848 to 2588672 tid 2 2024-04-07T16:25:37.486 INFO:tasks.rados.rados.0.smithi100.stdout:2709: writing smithi10066867-1 from 2588672 to 3203072 tid 3 2024-04-07T16:25:37.486 INFO:tasks.rados.rados.0.smithi100.stdout:2710: delete oid 44 current snap is 257 2024-04-07T16:25:37.486 INFO:tasks.rados.rados.0.smithi100.stdout:2704: finishing write tid 2 to smithi10066867-50 2024-04-07T16:25:37.486 INFO:tasks.rados.rados.0.smithi100.stdout:2704: finishing write tid 3 to smithi10066867-50 2024-04-07T16:25:37.486 INFO:tasks.rados.rados.0.smithi100.stdout:2704: finishing write tid 4 to smithi10066867-50 2024-04-07T16:25:37.488 INFO:tasks.rados.rados.0.smithi100.stdout:2708: finishing write tid 1 to smithi10066867-8 2024-04-07T16:25:37.488 INFO:tasks.rados.rados.0.smithi100.stdout:2704: finishing write tid 5 to smithi10066867-50 2024-04-07T16:25:37.488 INFO:tasks.rados.rados.0.smithi100.stdout:2706: expect (ObjNum 762 snap 240 seq_num 762) 2024-04-07T16:25:37.521 INFO:tasks.rados.rados.0.smithi100.stdout:2704: finishing write tid 6 to smithi10066867-50 2024-04-07T16:25:37.521 INFO:tasks.rados.rados.0.smithi100.stdout:2704: oid 50 updating version 0 to 1121 2024-04-07T16:25:37.522 INFO:tasks.rados.rados.0.smithi100.stdout:2704: oid 50 updating version 1121 to 1123 2024-04-07T16:25:37.522 INFO:tasks.rados.rados.0.smithi100.stdout:2704: oid 50 version 1123 is already newer than 1120 2024-04-07T16:25:37.522 INFO:tasks.rados.rados.0.smithi100.stdout:2704: oid 50 updating version 1123 to 1124 2024-04-07T16:25:37.522 INFO:tasks.rados.rados.0.smithi100.stdout:2704: oid 50 version 1124 is already newer than 1122 2024-04-07T16:25:37.522 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 50 v 1124 (ObjNum 817 snap 257 seq_num 817) dirty exists 2024-04-07T16:25:37.522 INFO:tasks.rados.rados.0.smithi100.stdout:2704: left oid 50 (ObjNum 817 snap 257 seq_num 817) 2024-04-07T16:25:37.522 INFO:tasks.rados.rados.0.smithi100.stdout:2704: done (6 left) 2024-04-07T16:25:37.522 INFO:tasks.rados.rados.0.smithi100.stdout:2706: done (5 left) 2024-04-07T16:25:37.522 INFO:tasks.rados.rados.0.smithi100.stdout:2710: done (4 left) 2024-04-07T16:25:37.522 INFO:tasks.rados.rados.0.smithi100.stdout:2711: snap_remove snap 256 2024-04-07T16:25:37.921 INFO:tasks.rados.rados.0.smithi100.stdout:2711: done (4 left) 2024-04-07T16:25:37.921 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 26 current snap is 257 2024-04-07T16:25:37.921 INFO:tasks.rados.rados.0.smithi100.stdout:2712: seq_num 821 ranges {2056192=557056,2613248=761856,3375104=442368} 2024-04-07T16:25:37.932 INFO:tasks.rados.rados.0.smithi100.stdout:2712: writing smithi10066867-26 from 2056192 to 2613248 tid 1 2024-04-07T16:25:37.938 INFO:tasks.rados.rados.0.smithi100.stdout:2712: writing smithi10066867-26 from 2613248 to 3375104 tid 2 2024-04-07T16:25:37.941 INFO:tasks.rados.rados.0.smithi100.stdout:2712: writing smithi10066867-26 from 3375104 to 3817472 tid 3 2024-04-07T16:25:37.942 INFO:tasks.rados.rados.0.smithi100.stdout:2713: read oid 15 snap -1 2024-04-07T16:25:37.942 INFO:tasks.rados.rados.0.smithi100.stdout:2713: expect (ObjNum 776 snap 243 seq_num 776) 2024-04-07T16:25:37.942 INFO:tasks.rados.rados.0.smithi100.stdout:2714: snap_create 2024-04-07T16:25:37.942 INFO:tasks.rados.rados.0.smithi100.stdout:2707: finishing copy_from racing read to smithi10066867-7 2024-04-07T16:25:37.942 INFO:tasks.rados.rados.0.smithi100.stdout:2705: expect (ObjNum 756 snap 238 seq_num 756) 2024-04-07T16:25:38.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:37 smithi100 bash[76202]: audit 2024-04-07T16:25:36.550423+0000 mgr.y (mgr.25175) 263 : audit [DBG] from='client.44294 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:25:38.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:37 smithi100 bash[76202]: cluster 2024-04-07T16:25:36.735567+0000 mgr.y (mgr.25175) 264 : cluster [DBG] pgmap v463: 121 pgs: 11 active+undersized+degraded, 39 active+undersized, 71 active+clean; 181 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 8.8 MiB/s rd, 1.4 MiB/s wr, 9 op/s; 31/366 objects degraded (8.470%) 2024-04-07T16:25:38.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:37 smithi100 bash[76202]: audit 2024-04-07T16:25:36.912024+0000 mon.b (mon.1) 10 : audit [INF] from='osd.7 [v2:172.21.15.144:6824/3641044519,v1:172.21.15.144:6825/3641044519]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-04-07T16:25:38.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:37 smithi100 bash[76202]: audit 2024-04-07T16:25:36.915223+0000 mon.a (mon.0) 854 : audit [INF] from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-07T16:25:38.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:37 smithi100 bash[76202]: cluster 2024-04-07T16:25:36.924807+0000 mon.a (mon.0) 855 : cluster [DBG] osdmap e581: 8 total, 7 up, 8 in 2024-04-07T16:25:38.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:37 smithi100 bash[76202]: audit 2024-04-07T16:25:36.928116+0000 mon.a (mon.0) 856 : audit [INF] from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-04-07T16:25:38.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:37 smithi100 bash[76202]: audit 2024-04-07T16:25:36.935128+0000 mgr.y (mgr.25175) 265 : audit [DBG] from='client.44303 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:25:38.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:37 smithi100 bash[76202]: audit 2024-04-07T16:25:37.202730+0000 mon.a (mon.0) 857 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:25:38.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:37 smithi100 bash[76202]: audit 2024-04-07T16:25:37.214216+0000 mon.a (mon.0) 858 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:25:38.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:37 smithi100 bash[76202]: audit 2024-04-07T16:25:37.414827+0000 mon.a (mon.0) 859 : audit [DBG] from='client.? 172.21.15.100:0/1471499593' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:25:38.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:37 smithi100 bash[74942]: audit 2024-04-07T16:25:36.550423+0000 mgr.y (mgr.25175) 263 : audit [DBG] from='client.44294 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:25:38.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:37 smithi100 bash[74942]: cluster 2024-04-07T16:25:36.735567+0000 mgr.y (mgr.25175) 264 : cluster [DBG] pgmap v463: 121 pgs: 11 active+undersized+degraded, 39 active+undersized, 71 active+clean; 181 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 8.8 MiB/s rd, 1.4 MiB/s wr, 9 op/s; 31/366 objects degraded (8.470%) 2024-04-07T16:25:38.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:37 smithi100 bash[74942]: audit 2024-04-07T16:25:36.912024+0000 mon.b (mon.1) 10 : audit [INF] from='osd.7 [v2:172.21.15.144:6824/3641044519,v1:172.21.15.144:6825/3641044519]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-04-07T16:25:38.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:37 smithi100 bash[74942]: audit 2024-04-07T16:25:36.915223+0000 mon.a (mon.0) 854 : audit [INF] from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-07T16:25:38.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:37 smithi100 bash[74942]: cluster 2024-04-07T16:25:36.924807+0000 mon.a (mon.0) 855 : cluster [DBG] osdmap e581: 8 total, 7 up, 8 in 2024-04-07T16:25:38.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:37 smithi100 bash[74942]: audit 2024-04-07T16:25:36.928116+0000 mon.a (mon.0) 856 : audit [INF] from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-04-07T16:25:38.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:37 smithi100 bash[74942]: audit 2024-04-07T16:25:36.935128+0000 mgr.y (mgr.25175) 265 : audit [DBG] from='client.44303 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:25:38.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:37 smithi100 bash[74942]: audit 2024-04-07T16:25:37.202730+0000 mon.a (mon.0) 857 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:25:38.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:37 smithi100 bash[74942]: audit 2024-04-07T16:25:37.214216+0000 mon.a (mon.0) 858 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:25:38.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:37 smithi100 bash[74942]: audit 2024-04-07T16:25:37.414827+0000 mon.a (mon.0) 859 : audit [DBG] from='client.? 172.21.15.100:0/1471499593' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:25:38.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:37 smithi144 bash[66654]: audit 2024-04-07T16:25:36.550423+0000 mgr.y (mgr.25175) 263 : audit [DBG] from='client.44294 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:25:38.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:37 smithi144 bash[66654]: cluster 2024-04-07T16:25:36.735567+0000 mgr.y (mgr.25175) 264 : cluster [DBG] pgmap v463: 121 pgs: 11 active+undersized+degraded, 39 active+undersized, 71 active+clean; 181 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 8.8 MiB/s rd, 1.4 MiB/s wr, 9 op/s; 31/366 objects degraded (8.470%) 2024-04-07T16:25:38.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:37 smithi144 bash[66654]: audit 2024-04-07T16:25:36.912024+0000 mon.b (mon.1) 10 : audit [INF] from='osd.7 [v2:172.21.15.144:6824/3641044519,v1:172.21.15.144:6825/3641044519]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-04-07T16:25:38.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:37 smithi144 bash[66654]: audit 2024-04-07T16:25:36.915223+0000 mon.a (mon.0) 854 : audit [INF] from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-07T16:25:38.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:37 smithi144 bash[66654]: cluster 2024-04-07T16:25:36.924807+0000 mon.a (mon.0) 855 : cluster [DBG] osdmap e581: 8 total, 7 up, 8 in 2024-04-07T16:25:38.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:37 smithi144 bash[66654]: audit 2024-04-07T16:25:36.928116+0000 mon.a (mon.0) 856 : audit [INF] from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-04-07T16:25:38.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:37 smithi144 bash[66654]: audit 2024-04-07T16:25:36.935128+0000 mgr.y (mgr.25175) 265 : audit [DBG] from='client.44303 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:25:38.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:37 smithi144 bash[66654]: audit 2024-04-07T16:25:37.202730+0000 mon.a (mon.0) 857 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:25:38.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:37 smithi144 bash[66654]: audit 2024-04-07T16:25:37.214216+0000 mon.a (mon.0) 858 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:25:38.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:37 smithi144 bash[66654]: audit 2024-04-07T16:25:37.414827+0000 mon.a (mon.0) 859 : audit [DBG] from='client.? 172.21.15.100:0/1471499593' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:25:38.468 INFO:tasks.rados.rados.0.smithi100.stdout:2708: finishing write tid 2 to smithi10066867-8 2024-04-07T16:25:38.468 INFO:tasks.rados.rados.0.smithi100.stdout:2709: finishing write tid 1 to smithi10066867-1 2024-04-07T16:25:38.468 INFO:tasks.rados.rados.0.smithi100.stdout:2707: finishing copy_from to smithi10066867-7 2024-04-07T16:25:38.468 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 7 v 1106 (ObjNum 733 snap 226 seq_num 733) dirty exists 2024-04-07T16:25:38.468 INFO:tasks.rados.rados.0.smithi100.stdout:2709: finishing write tid 2 to smithi10066867-1 2024-04-07T16:25:38.468 INFO:tasks.rados.rados.0.smithi100.stdout:2708: finishing write tid 3 to smithi10066867-8 2024-04-07T16:25:38.468 INFO:tasks.rados.rados.0.smithi100.stdout:2708: finishing write tid 4 to smithi10066867-8 2024-04-07T16:25:38.468 INFO:tasks.rados.rados.0.smithi100.stdout:2709: finishing write tid 3 to smithi10066867-1 2024-04-07T16:25:38.468 INFO:tasks.rados.rados.0.smithi100.stdout:2709: finishing write tid 4 to smithi10066867-1 2024-04-07T16:25:38.468 INFO:tasks.rados.rados.0.smithi100.stdout:2709: finishing write tid 5 to smithi10066867-1 2024-04-07T16:25:38.468 INFO:tasks.rados.rados.0.smithi100.stdout:2709: oid 1 updating version 0 to 1112 2024-04-07T16:25:38.468 INFO:tasks.rados.rados.0.smithi100.stdout:2709: oid 1 version 1112 is already newer than 1109 2024-04-07T16:25:38.468 INFO:tasks.rados.rados.0.smithi100.stdout:2709: oid 1 version 1112 is already newer than 1111 2024-04-07T16:25:38.469 INFO:tasks.rados.rados.0.smithi100.stdout:2709: oid 1 version 1112 is already newer than 1110 2024-04-07T16:25:38.469 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 1 v 1112 (ObjNum 819 snap 257 seq_num 819) dirty exists 2024-04-07T16:25:38.469 INFO:tasks.rados.rados.0.smithi100.stdout:2709: left oid 1 (ObjNum 819 snap 257 seq_num 819) 2024-04-07T16:25:38.469 INFO:tasks.rados.rados.0.smithi100.stdout:2708: finishing write tid 5 to smithi10066867-8 2024-04-07T16:25:38.469 INFO:tasks.rados.rados.0.smithi100.stdout:2708: oid 8 updating version 0 to 1086 2024-04-07T16:25:38.469 INFO:tasks.rados.rados.0.smithi100.stdout:2708: oid 8 updating version 1086 to 1088 2024-04-07T16:25:38.469 INFO:tasks.rados.rados.0.smithi100.stdout:2708: oid 8 version 1088 is already newer than 1085 2024-04-07T16:25:38.469 INFO:tasks.rados.rados.0.smithi100.stdout:2708: oid 8 version 1088 is already newer than 1087 2024-04-07T16:25:38.469 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 8 v 1088 (ObjNum 818 snap 257 seq_num 818) dirty exists 2024-04-07T16:25:38.469 INFO:tasks.rados.rados.0.smithi100.stdout:2708: left oid 8 (ObjNum 818 snap 257 seq_num 818) 2024-04-07T16:25:38.469 INFO:tasks.rados.rados.0.smithi100.stdout:2712: finishing write tid 1 to smithi10066867-26 2024-04-07T16:25:38.469 INFO:tasks.rados.rados.0.smithi100.stdout:2712: finishing write tid 2 to smithi10066867-26 2024-04-07T16:25:38.469 INFO:tasks.rados.rados.0.smithi100.stdout:2712: finishing write tid 3 to smithi10066867-26 2024-04-07T16:25:38.469 INFO:tasks.rados.rados.0.smithi100.stdout:2712: finishing write tid 4 to smithi10066867-26 2024-04-07T16:25:38.469 INFO:tasks.rados.rados.0.smithi100.stdout:2712: finishing write tid 5 to smithi10066867-26 2024-04-07T16:25:38.469 INFO:tasks.rados.rados.0.smithi100.stdout:2712: oid 26 updating version 0 to 1184 2024-04-07T16:25:38.470 INFO:tasks.rados.rados.0.smithi100.stdout:2712: oid 26 updating version 1184 to 1185 2024-04-07T16:25:38.470 INFO:tasks.rados.rados.0.smithi100.stdout:2712: oid 26 version 1185 is already newer than 1183 2024-04-07T16:25:38.470 INFO:tasks.rados.rados.0.smithi100.stdout:2712: oid 26 version 1185 is already newer than 1182 2024-04-07T16:25:38.470 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 26 v 1185 (ObjNum 821 snap 257 seq_num 821) dirty exists 2024-04-07T16:25:38.470 INFO:tasks.rados.rados.0.smithi100.stdout:2712: left oid 26 (ObjNum 821 snap 257 seq_num 821) 2024-04-07T16:25:38.470 INFO:tasks.rados.rados.0.smithi100.stdout:2713: expect (ObjNum 776 snap 243 seq_num 776) 2024-04-07T16:25:38.685 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:25:38 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:25:38] "GET /metrics HTTP/1.1" 200 36344 "" "Prometheus/2.43.0" 2024-04-07T16:25:38.930 INFO:tasks.rados.rados.0.smithi100.stdout:2705: done (6 left) 2024-04-07T16:25:38.932 INFO:tasks.rados.rados.0.smithi100.stdout:2707: done (5 left) 2024-04-07T16:25:38.932 INFO:tasks.rados.rados.0.smithi100.stdout:2708: done (4 left) 2024-04-07T16:25:38.932 INFO:tasks.rados.rados.0.smithi100.stdout:2709: done (3 left) 2024-04-07T16:25:38.932 INFO:tasks.rados.rados.0.smithi100.stdout:2712: done (2 left) 2024-04-07T16:25:38.932 INFO:tasks.rados.rados.0.smithi100.stdout:2713: done (1 left) 2024-04-07T16:25:38.932 INFO:tasks.rados.rados.0.smithi100.stdout:2714: done (0 left) 2024-04-07T16:25:38.932 INFO:tasks.rados.rados.0.smithi100.stdout:2715: copy_from oid 10 from oid 15 current snap is 258 2024-04-07T16:25:38.932 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 37 current snap is 258 2024-04-07T16:25:38.932 INFO:tasks.rados.rados.0.smithi100.stdout:2716: seq_num 822 ranges {2023424=573440,2596864=696320,3293184=507904} 2024-04-07T16:25:38.949 INFO:tasks.rados.rados.0.smithi100.stdout:2716: writing smithi10066867-37 from 2023424 to 2596864 tid 1 2024-04-07T16:25:38.954 INFO:tasks.rados.rados.0.smithi100.stdout:2716: writing smithi10066867-37 from 2596864 to 3293184 tid 2 2024-04-07T16:25:38.959 INFO:tasks.rados.rados.0.smithi100.stdout:2716: writing smithi10066867-37 from 3293184 to 3801088 tid 3 2024-04-07T16:25:38.959 INFO:tasks.rados.rados.0.smithi100.stdout:2717: copy_from oid 6 from oid 9 current snap is 258 2024-04-07T16:25:38.959 INFO:tasks.rados.rados.0.smithi100.stdout:2718: snap_remove snap 257 2024-04-07T16:25:39.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:38 smithi144 bash[66654]: cluster 2024-04-07T16:25:37.916383+0000 mon.a (mon.0) 860 : cluster [INF] Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-07T16:25:39.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:38 smithi144 bash[66654]: cluster 2024-04-07T16:25:37.925264+0000 mon.a (mon.0) 861 : cluster [INF] osd.7 [v2:172.21.15.144:6824/3641044519,v1:172.21.15.144:6825/3641044519] boot 2024-04-07T16:25:39.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:38 smithi144 bash[66654]: cluster 2024-04-07T16:25:37.925385+0000 mon.a (mon.0) 862 : cluster [DBG] osdmap e582: 8 total, 8 up, 8 in 2024-04-07T16:25:39.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:38 smithi144 bash[66654]: audit 2024-04-07T16:25:37.931582+0000 mon.a (mon.0) 863 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-07T16:25:39.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:38 smithi144 bash[66654]: audit 2024-04-07T16:25:38.362034+0000 mon.a (mon.0) 864 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:25:39.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:38 smithi144 bash[66654]: audit 2024-04-07T16:25:38.373538+0000 mon.a (mon.0) 865 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:25:39.435 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:38 smithi100 bash[74942]: cluster 2024-04-07T16:25:37.916383+0000 mon.a (mon.0) 860 : cluster [INF] Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-07T16:25:39.435 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:38 smithi100 bash[74942]: cluster 2024-04-07T16:25:37.925264+0000 mon.a (mon.0) 861 : cluster [INF] osd.7 [v2:172.21.15.144:6824/3641044519,v1:172.21.15.144:6825/3641044519] boot 2024-04-07T16:25:39.435 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:38 smithi100 bash[74942]: cluster 2024-04-07T16:25:37.925385+0000 mon.a (mon.0) 862 : cluster [DBG] osdmap e582: 8 total, 8 up, 8 in 2024-04-07T16:25:39.435 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:38 smithi100 bash[74942]: audit 2024-04-07T16:25:37.931582+0000 mon.a (mon.0) 863 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-07T16:25:39.436 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:38 smithi100 bash[74942]: audit 2024-04-07T16:25:38.362034+0000 mon.a (mon.0) 864 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:25:39.436 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:38 smithi100 bash[74942]: audit 2024-04-07T16:25:38.373538+0000 mon.a (mon.0) 865 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:25:39.436 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:38 smithi100 bash[76202]: cluster 2024-04-07T16:25:37.916383+0000 mon.a (mon.0) 860 : cluster [INF] Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-07T16:25:39.436 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:38 smithi100 bash[76202]: cluster 2024-04-07T16:25:37.925264+0000 mon.a (mon.0) 861 : cluster [INF] osd.7 [v2:172.21.15.144:6824/3641044519,v1:172.21.15.144:6825/3641044519] boot 2024-04-07T16:25:39.436 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:38 smithi100 bash[76202]: cluster 2024-04-07T16:25:37.925385+0000 mon.a (mon.0) 862 : cluster [DBG] osdmap e582: 8 total, 8 up, 8 in 2024-04-07T16:25:39.436 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:38 smithi100 bash[76202]: audit 2024-04-07T16:25:37.931582+0000 mon.a (mon.0) 863 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-07T16:25:39.436 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:38 smithi100 bash[76202]: audit 2024-04-07T16:25:38.362034+0000 mon.a (mon.0) 864 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:25:39.436 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:38 smithi100 bash[76202]: audit 2024-04-07T16:25:38.373538+0000 mon.a (mon.0) 865 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:25:39.965 INFO:tasks.rados.rados.0.smithi100.stdout:2718: done (3 left) 2024-04-07T16:25:39.965 INFO:tasks.rados.rados.0.smithi100.stdout:2719: delete oid 7 current snap is 258 2024-04-07T16:25:39.966 INFO:tasks.rados.rados.0.smithi100.stdout:2716: finishing write tid 1 to smithi10066867-37 2024-04-07T16:25:39.966 INFO:tasks.rados.rados.0.smithi100.stdout:2716: finishing write tid 2 to smithi10066867-37 2024-04-07T16:25:39.966 INFO:tasks.rados.rados.0.smithi100.stdout:2716: finishing write tid 3 to smithi10066867-37 2024-04-07T16:25:39.966 INFO:tasks.rados.rados.0.smithi100.stdout:2716: finishing write tid 4 to smithi10066867-37 2024-04-07T16:25:39.966 INFO:tasks.rados.rados.0.smithi100.stdout:2716: finishing write tid 5 to smithi10066867-37 2024-04-07T16:25:39.966 INFO:tasks.rados.rados.0.smithi100.stdout:2716: oid 37 updating version 0 to 1200 2024-04-07T16:25:39.966 INFO:tasks.rados.rados.0.smithi100.stdout:2716: oid 37 updating version 1200 to 1203 2024-04-07T16:25:39.966 INFO:tasks.rados.rados.0.smithi100.stdout:2716: oid 37 version 1203 is already newer than 1202 2024-04-07T16:25:39.966 INFO:tasks.rados.rados.0.smithi100.stdout:2716: oid 37 version 1203 is already newer than 1201 2024-04-07T16:25:39.966 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 37 v 1203 (ObjNum 822 snap 258 seq_num 822) dirty exists 2024-04-07T16:25:39.966 INFO:tasks.rados.rados.0.smithi100.stdout:2716: left oid 37 (ObjNum 822 snap 258 seq_num 822) 2024-04-07T16:25:39.966 INFO:tasks.rados.rados.0.smithi100.stdout:2715: finishing copy_from racing read to smithi10066867-10 2024-04-07T16:25:39.966 INFO:tasks.rados.rados.0.smithi100.stdout:2715: finishing copy_from to smithi10066867-10 2024-04-07T16:25:39.967 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 10 v 1433 (ObjNum 776 snap 243 seq_num 776) dirty exists 2024-04-07T16:25:39.967 INFO:tasks.rados.rados.0.smithi100.stdout:2717: finishing copy_from racing read to smithi10066867-6 2024-04-07T16:25:39.967 INFO:tasks.rados.rados.0.smithi100.stdout:2717: finishing copy_from to smithi10066867-6 2024-04-07T16:25:39.967 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 6 v 1117 (ObjNum 810 snap 256 seq_num 810) dirty exists 2024-04-07T16:25:39.974 INFO:tasks.rados.rados.0.smithi100.stdout:2715: done (3 left) 2024-04-07T16:25:39.974 INFO:tasks.rados.rados.0.smithi100.stdout:2716: done (2 left) 2024-04-07T16:25:39.974 INFO:tasks.rados.rados.0.smithi100.stdout:2717: done (1 left) 2024-04-07T16:25:39.974 INFO:tasks.rados.rados.0.smithi100.stdout:2719: done (0 left) 2024-04-07T16:25:39.974 INFO:tasks.rados.rados.0.smithi100.stdout:2720: rmattr oid 13 current snap is 258 2024-04-07T16:25:39.974 INFO:tasks.rados.rados.0.smithi100.stdout:2720: done (0 left) 2024-04-07T16:25:39.974 INFO:tasks.rados.rados.0.smithi100.stdout:2721: setattr oid 39 current snap is 258 2024-04-07T16:25:39.977 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 43 current snap is 258 2024-04-07T16:25:39.977 INFO:tasks.rados.rados.0.smithi100.stdout:2722: seq_num 824 ranges {8593408=606208,9199616=458752,9658368=573440,10231808=172032} 2024-04-07T16:25:40.045 INFO:tasks.rados.rados.0.smithi100.stdout:2722: writing smithi10066867-43 from 8593408 to 9199616 tid 1 2024-04-07T16:25:40.048 INFO:tasks.rados.rados.0.smithi100.stdout:2722: writing smithi10066867-43 from 9199616 to 9658368 tid 2 2024-04-07T16:25:40.051 INFO:tasks.rados.rados.0.smithi100.stdout:2722: writing smithi10066867-43 from 9658368 to 10231808 tid 3 2024-04-07T16:25:40.053 INFO:tasks.rados.rados.0.smithi100.stdout:2722: writing smithi10066867-43 from 10231808 to 10403840 tid 4 2024-04-07T16:25:40.053 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 47 current snap is 258 2024-04-07T16:25:40.053 INFO:tasks.rados.rados.0.smithi100.stdout:2723: seq_num 825 ranges {2949120=622592,3571712=745472,4317184=458752} 2024-04-07T16:25:40.073 INFO:tasks.rados.rados.0.smithi100.stdout:2723: writing smithi10066867-47 from 2949120 to 3571712 tid 1 2024-04-07T16:25:40.077 INFO:tasks.rados.rados.0.smithi100.stdout:2723: writing smithi10066867-47 from 3571712 to 4317184 tid 2 2024-04-07T16:25:40.080 INFO:tasks.rados.rados.0.smithi100.stdout:2723: writing smithi10066867-47 from 4317184 to 4775936 tid 3 2024-04-07T16:25:40.080 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 9 current snap is 258 2024-04-07T16:25:40.080 INFO:tasks.rados.rados.0.smithi100.stdout:2724: seq_num 826 ranges {1572864=638976,2211840=630784,2842624=573440} 2024-04-07T16:25:40.092 INFO:tasks.rados.rados.0.smithi100.stdout:2724: writing smithi10066867-9 from 1572864 to 2211840 tid 1 2024-04-07T16:25:40.096 INFO:tasks.rados.rados.0.smithi100.stdout:2724: writing smithi10066867-9 from 2211840 to 2842624 tid 2 2024-04-07T16:25:40.100 INFO:tasks.rados.rados.0.smithi100.stdout:2724: writing smithi10066867-9 from 2842624 to 3416064 tid 3 2024-04-07T16:25:40.100 INFO:tasks.rados.rados.0.smithi100.stdout:2725: rollback oid 48 current snap is 258 2024-04-07T16:25:40.100 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 48 to 255 2024-04-07T16:25:40.100 INFO:tasks.rados.rados.0.smithi100.stdout:2726: delete oid 27 current snap is 258 2024-04-07T16:25:40.101 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 39 v 1204 (ObjNum 756 snap 238 seq_num 756) dirty exists 2024-04-07T16:25:40.101 INFO:tasks.rados.rados.0.smithi100.stdout:2722: finishing write tid 1 to smithi10066867-43 2024-04-07T16:25:40.101 INFO:tasks.rados.rados.0.smithi100.stdout:2722: finishing write tid 2 to smithi10066867-43 2024-04-07T16:25:40.101 INFO:tasks.rados.rados.0.smithi100.stdout:2722: finishing write tid 3 to smithi10066867-43 2024-04-07T16:25:40.101 INFO:tasks.rados.rados.0.smithi100.stdout:2722: finishing write tid 4 to smithi10066867-43 2024-04-07T16:25:40.101 INFO:tasks.rados.rados.0.smithi100.stdout:2722: finishing write tid 5 to smithi10066867-43 2024-04-07T16:25:40.101 INFO:tasks.rados.rados.0.smithi100.stdout:2722: finishing write tid 6 to smithi10066867-43 2024-04-07T16:25:40.101 INFO:tasks.rados.rados.0.smithi100.stdout:2722: oid 43 updating version 0 to 1124 2024-04-07T16:25:40.101 INFO:tasks.rados.rados.0.smithi100.stdout:2722: oid 43 version 1124 is already newer than 1123 2024-04-07T16:25:40.101 INFO:tasks.rados.rados.0.smithi100.stdout:2722: oid 43 version 1124 is already newer than 1122 2024-04-07T16:25:40.101 INFO:tasks.rados.rados.0.smithi100.stdout:2722: oid 43 version 1124 is already newer than 1121 2024-04-07T16:25:40.101 INFO:tasks.rados.rados.0.smithi100.stdout:2722: oid 43 version 1124 is already newer than 1120 2024-04-07T16:25:40.101 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 43 v 1124 (ObjNum 824 snap 258 seq_num 824) dirty exists 2024-04-07T16:25:40.101 INFO:tasks.rados.rados.0.smithi100.stdout:2722: left oid 43 (ObjNum 824 snap 258 seq_num 824) 2024-04-07T16:25:40.101 INFO:tasks.rados.rados.0.smithi100.stdout:2723: finishing write tid 1 to smithi10066867-47 2024-04-07T16:25:40.102 INFO:tasks.rados.rados.0.smithi100.stdout:2723: finishing write tid 2 to smithi10066867-47 2024-04-07T16:25:40.102 INFO:tasks.rados.rados.0.smithi100.stdout:2723: finishing write tid 3 to smithi10066867-47 2024-04-07T16:25:40.102 INFO:tasks.rados.rados.0.smithi100.stdout:2723: finishing write tid 4 to smithi10066867-47 2024-04-07T16:25:40.102 INFO:tasks.rados.rados.0.smithi100.stdout:2723: finishing write tid 5 to smithi10066867-47 2024-04-07T16:25:40.102 INFO:tasks.rados.rados.0.smithi100.stdout:2723: oid 47 updating version 0 to 1188 2024-04-07T16:25:40.102 INFO:tasks.rados.rados.0.smithi100.stdout:2723: oid 47 version 1188 is already newer than 1187 2024-04-07T16:25:40.102 INFO:tasks.rados.rados.0.smithi100.stdout:2723: oid 47 updating version 1188 to 1189 2024-04-07T16:25:40.102 INFO:tasks.rados.rados.0.smithi100.stdout:2723: oid 47 updating version 1189 to 1190 2024-04-07T16:25:40.102 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 47 v 1190 (ObjNum 825 snap 258 seq_num 825) dirty exists 2024-04-07T16:25:40.102 INFO:tasks.rados.rados.0.smithi100.stdout:2723: left oid 47 (ObjNum 825 snap 258 seq_num 825) 2024-04-07T16:25:40.103 INFO:tasks.rados.rados.0.smithi100.stdout:2724: finishing write tid 1 to smithi10066867-9 2024-04-07T16:25:40.103 INFO:tasks.rados.rados.0.smithi100.stdout:2721: done (5 left) 2024-04-07T16:25:40.103 INFO:tasks.rados.rados.0.smithi100.stdout:2722: done (4 left) 2024-04-07T16:25:40.103 INFO:tasks.rados.rados.0.smithi100.stdout:2723: done (3 left) 2024-04-07T16:25:40.103 INFO:tasks.rados.rados.0.smithi100.stdout:2726: done (2 left) 2024-04-07T16:25:40.103 INFO:tasks.rados.rados.0.smithi100.stdout:2727: snap_create 2024-04-07T16:25:40.104 INFO:tasks.rados.rados.0.smithi100.stdout:2725: finishing rollback tid 0 to smithi10066867-48 2024-04-07T16:25:40.104 INFO:tasks.rados.rados.0.smithi100.stdout:2725: finishing rollback tid 1 to smithi10066867-48 2024-04-07T16:25:40.104 INFO:tasks.rados.rados.0.smithi100.stdout:2725: finishing rollback tid 2 to smithi10066867-48 2024-04-07T16:25:40.104 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 48 v 1116 (ObjNum 794 snap 251 seq_num 794) dirty exists 2024-04-07T16:25:40.106 INFO:tasks.rados.rados.0.smithi100.stdout:2724: finishing write tid 2 to smithi10066867-9 2024-04-07T16:25:40.106 INFO:tasks.rados.rados.0.smithi100.stdout:2724: finishing write tid 3 to smithi10066867-9 2024-04-07T16:25:40.107 INFO:tasks.rados.rados.0.smithi100.stdout:2724: finishing write tid 4 to smithi10066867-9 2024-04-07T16:25:40.109 INFO:tasks.rados.rados.0.smithi100.stdout:2724: finishing write tid 5 to smithi10066867-9 2024-04-07T16:25:40.109 INFO:tasks.rados.rados.0.smithi100.stdout:2724: oid 9 updating version 0 to 1120 2024-04-07T16:25:40.109 INFO:tasks.rados.rados.0.smithi100.stdout:2724: oid 9 version 1120 is already newer than 1118 2024-04-07T16:25:40.110 INFO:tasks.rados.rados.0.smithi100.stdout:2724: oid 9 updating version 1120 to 1121 2024-04-07T16:25:40.110 INFO:tasks.rados.rados.0.smithi100.stdout:2724: oid 9 version 1121 is already newer than 1119 2024-04-07T16:25:40.110 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 9 v 1121 (ObjNum 826 snap 258 seq_num 826) dirty exists 2024-04-07T16:25:40.110 INFO:tasks.rados.rados.0.smithi100.stdout:2724: left oid 9 (ObjNum 826 snap 258 seq_num 826) 2024-04-07T16:25:40.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:39 smithi144 bash[66654]: cluster 2024-04-07T16:25:38.736500+0000 mgr.y (mgr.25175) 266 : cluster [DBG] pgmap v466: 121 pgs: 4 peering, 10 active+undersized+degraded, 36 active+undersized, 71 active+clean; 193 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 8.3 MiB/s rd, 4.0 MiB/s wr, 10 op/s; 30/372 objects degraded (8.065%) 2024-04-07T16:25:40.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:39 smithi144 bash[66654]: cluster 2024-04-07T16:25:38.945982+0000 mon.a (mon.0) 866 : cluster [DBG] osdmap e583: 8 total, 8 up, 8 in 2024-04-07T16:25:40.435 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:39 smithi100 bash[74942]: cluster 2024-04-07T16:25:38.736500+0000 mgr.y (mgr.25175) 266 : cluster [DBG] pgmap v466: 121 pgs: 4 peering, 10 active+undersized+degraded, 36 active+undersized, 71 active+clean; 193 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 8.3 MiB/s rd, 4.0 MiB/s wr, 10 op/s; 30/372 objects degraded (8.065%) 2024-04-07T16:25:40.435 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:39 smithi100 bash[74942]: cluster 2024-04-07T16:25:38.945982+0000 mon.a (mon.0) 866 : cluster [DBG] osdmap e583: 8 total, 8 up, 8 in 2024-04-07T16:25:40.435 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:39 smithi100 bash[76202]: cluster 2024-04-07T16:25:38.736500+0000 mgr.y (mgr.25175) 266 : cluster [DBG] pgmap v466: 121 pgs: 4 peering, 10 active+undersized+degraded, 36 active+undersized, 71 active+clean; 193 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 8.3 MiB/s rd, 4.0 MiB/s wr, 10 op/s; 30/372 objects degraded (8.065%) 2024-04-07T16:25:40.435 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:39 smithi100 bash[76202]: cluster 2024-04-07T16:25:38.945982+0000 mon.a (mon.0) 866 : cluster [DBG] osdmap e583: 8 total, 8 up, 8 in 2024-04-07T16:25:40.982 INFO:tasks.rados.rados.0.smithi100.stdout:2724: done (2 left) 2024-04-07T16:25:40.982 INFO:tasks.rados.rados.0.smithi100.stdout:2725: done (1 left) 2024-04-07T16:25:40.983 INFO:tasks.rados.rados.0.smithi100.stdout:2727: done (0 left) 2024-04-07T16:25:40.983 INFO:tasks.rados.rados.0.smithi100.stdout:2728: copy_from oid 8 from oid 3 current snap is 259 2024-04-07T16:25:40.983 INFO:tasks.rados.rados.0.smithi100.stdout:2729: setattr oid 13 current snap is 259 2024-04-07T16:25:40.985 INFO:tasks.rados.rados.0.smithi100.stdout:2730: delete oid 23 current snap is 259 2024-04-07T16:25:40.989 INFO:tasks.rados.rados.0.smithi100.stdout:2730: done (2 left) 2024-04-07T16:25:40.989 INFO:tasks.rados.rados.0.smithi100.stdout:2731: copy_from oid 2 from oid 24 current snap is 259 2024-04-07T16:25:40.989 INFO:tasks.rados.rados.0.smithi100.stdout:2732: snap_create 2024-04-07T16:25:40.990 INFO:tasks.rados.rados.0.smithi100.stdout:2731: finishing copy_from to smithi10066867-2 2024-04-07T16:25:40.990 INFO:tasks.rados.rados.0.smithi100.stdout:2731: got expected ENOENT (src dne) 2024-04-07T16:25:40.991 INFO:tasks.rados.rados.0.smithi100.stdout:2731: finishing copy_from racing read to smithi10066867-2 2024-04-07T16:25:40.995 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 13 v 1091 (ObjNum 33 snap 0 seq_num 4239396198) dirty exists 2024-04-07T16:25:41.021 INFO:tasks.rados.rados.0.smithi100.stdout:2728: finishing copy_from racing read to smithi10066867-8 2024-04-07T16:25:41.021 INFO:tasks.rados.rados.0.smithi100.stdout:2728: finishing copy_from to smithi10066867-8 2024-04-07T16:25:41.021 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 8 v 1106 (ObjNum 733 snap 226 seq_num 733) dirty exists 2024-04-07T16:25:41.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:40 smithi144 bash[66654]: cluster 2024-04-07T16:25:39.981786+0000 mon.a (mon.0) 867 : cluster [DBG] osdmap e584: 8 total, 8 up, 8 in 2024-04-07T16:25:41.435 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:40 smithi100 bash[74942]: cluster 2024-04-07T16:25:39.981786+0000 mon.a (mon.0) 867 : cluster [DBG] osdmap e584: 8 total, 8 up, 8 in 2024-04-07T16:25:41.435 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:40 smithi100 bash[76202]: cluster 2024-04-07T16:25:39.981786+0000 mon.a (mon.0) 867 : cluster [DBG] osdmap e584: 8 total, 8 up, 8 in 2024-04-07T16:25:41.982 INFO:tasks.rados.rados.0.smithi100.stdout:2728: done (3 left) 2024-04-07T16:25:41.982 INFO:tasks.rados.rados.0.smithi100.stdout:2729: done (2 left) 2024-04-07T16:25:41.982 INFO:tasks.rados.rados.0.smithi100.stdout:2731: done (1 left) 2024-04-07T16:25:41.982 INFO:tasks.rados.rados.0.smithi100.stdout:2732: done (0 left) 2024-04-07T16:25:41.982 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 12 current snap is 260 2024-04-07T16:25:41.982 INFO:tasks.rados.rados.0.smithi100.stdout:2733: seq_num 829 ranges {663552=688128,1351680=729088,2080768=475136} 2024-04-07T16:25:41.995 INFO:tasks.rados.rados.0.smithi100.stdout:2733: writing smithi10066867-12 from 663552 to 1351680 tid 1 2024-04-07T16:25:42.001 INFO:tasks.rados.rados.0.smithi100.stdout:2733: writing smithi10066867-12 from 1351680 to 2080768 tid 2 2024-04-07T16:25:42.006 INFO:tasks.rados.rados.0.smithi100.stdout:2733: writing smithi10066867-12 from 2080768 to 2555904 tid 3 2024-04-07T16:25:42.006 INFO:tasks.rados.rados.0.smithi100.stdout:2734: read oid 33 snap -1 2024-04-07T16:25:42.007 INFO:tasks.rados.rados.0.smithi100.stdout:2734: expect (ObjNum 753 snap 235 seq_num 753) 2024-04-07T16:25:42.007 INFO:tasks.rados.rados.0.smithi100.stdout:2735: read oid 50 snap 258 2024-04-07T16:25:42.008 INFO:tasks.rados.rados.0.smithi100.stdout:2735: expect (ObjNum 817 snap 257 seq_num 817) 2024-04-07T16:25:42.008 INFO:tasks.rados.rados.0.smithi100.stdout:2736: copy_from oid 30 from oid 45 current snap is 260 2024-04-07T16:25:42.008 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 36 current snap is 260 2024-04-07T16:25:42.008 INFO:tasks.rados.rados.0.smithi100.stdout:2737: seq_num 830 ranges {0=704512,704512=614400,1318912=573440,1892352=16384} 2024-04-07T16:25:42.014 INFO:tasks.rados.rados.0.smithi100.stdout:2737: writing smithi10066867-36 from 0 to 704512 tid 1 2024-04-07T16:25:42.020 INFO:tasks.rados.rados.0.smithi100.stdout:2737: writing smithi10066867-36 from 704512 to 1318912 tid 2 2024-04-07T16:25:42.026 INFO:tasks.rados.rados.0.smithi100.stdout:2737: writing smithi10066867-36 from 1318912 to 1892352 tid 3 2024-04-07T16:25:42.027 INFO:tasks.rados.rados.0.smithi100.stdout:2737: writing smithi10066867-36 from 1892352 to 1908736 tid 4 2024-04-07T16:25:42.027 INFO:tasks.rados.rados.0.smithi100.stdout:2738: delete oid 3 current snap is 260 2024-04-07T16:25:42.027 INFO:tasks.rados.rados.0.smithi100.stdout:2733: finishing write tid 1 to smithi10066867-12 2024-04-07T16:25:42.027 INFO:tasks.rados.rados.0.smithi100.stdout:2734: expect (ObjNum 753 snap 235 seq_num 753) 2024-04-07T16:25:42.085 INFO:tasks.rados.rados.0.smithi100.stdout:2733: finishing write tid 2 to smithi10066867-12 2024-04-07T16:25:42.085 INFO:tasks.rados.rados.0.smithi100.stdout:2733: finishing write tid 3 to smithi10066867-12 2024-04-07T16:25:42.085 INFO:tasks.rados.rados.0.smithi100.stdout:2733: finishing write tid 4 to smithi10066867-12 2024-04-07T16:25:42.085 INFO:tasks.rados.rados.0.smithi100.stdout:2733: finishing write tid 5 to smithi10066867-12 2024-04-07T16:25:42.086 INFO:tasks.rados.rados.0.smithi100.stdout:2733: oid 12 updating version 0 to 1311 2024-04-07T16:25:42.086 INFO:tasks.rados.rados.0.smithi100.stdout:2733: oid 12 version 1311 is already newer than 1309 2024-04-07T16:25:42.086 INFO:tasks.rados.rados.0.smithi100.stdout:2733: oid 12 updating version 1311 to 1312 2024-04-07T16:25:42.086 INFO:tasks.rados.rados.0.smithi100.stdout:2733: oid 12 version 1312 is already newer than 1310 2024-04-07T16:25:42.086 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 12 v 1312 (ObjNum 829 snap 260 seq_num 829) dirty exists 2024-04-07T16:25:42.086 INFO:tasks.rados.rados.0.smithi100.stdout:2733: left oid 12 (ObjNum 829 snap 260 seq_num 829) 2024-04-07T16:25:42.086 INFO:tasks.rados.rados.0.smithi100.stdout:2737: finishing write tid 1 to smithi10066867-36 2024-04-07T16:25:42.086 INFO:tasks.rados.rados.0.smithi100.stdout:2737: finishing write tid 2 to smithi10066867-36 2024-04-07T16:25:42.086 INFO:tasks.rados.rados.0.smithi100.stdout:2737: finishing write tid 3 to smithi10066867-36 2024-04-07T16:25:42.086 INFO:tasks.rados.rados.0.smithi100.stdout:2737: finishing write tid 4 to smithi10066867-36 2024-04-07T16:25:42.086 INFO:tasks.rados.rados.0.smithi100.stdout:2737: finishing write tid 5 to smithi10066867-36 2024-04-07T16:25:42.086 INFO:tasks.rados.rados.0.smithi100.stdout:2737: finishing write tid 6 to smithi10066867-36 2024-04-07T16:25:42.086 INFO:tasks.rados.rados.0.smithi100.stdout:2737: oid 36 updating version 0 to 1184 2024-04-07T16:25:42.087 INFO:tasks.rados.rados.0.smithi100.stdout:2737: oid 36 updating version 1184 to 1185 2024-04-07T16:25:42.087 INFO:tasks.rados.rados.0.smithi100.stdout:2737: oid 36 updating version 1185 to 1186 2024-04-07T16:25:42.087 INFO:tasks.rados.rados.0.smithi100.stdout:2737: oid 36 version 1186 is already newer than 1182 2024-04-07T16:25:42.087 INFO:tasks.rados.rados.0.smithi100.stdout:2737: oid 36 version 1186 is already newer than 1183 2024-04-07T16:25:42.087 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 36 v 1186 (ObjNum 830 snap 260 seq_num 830) dirty exists 2024-04-07T16:25:42.087 INFO:tasks.rados.rados.0.smithi100.stdout:2737: left oid 36 (ObjNum 830 snap 260 seq_num 830) 2024-04-07T16:25:42.087 INFO:tasks.rados.rados.0.smithi100.stdout:2733: done (5 left) 2024-04-07T16:25:42.087 INFO:tasks.rados.rados.0.smithi100.stdout:2734: done (4 left) 2024-04-07T16:25:42.087 INFO:tasks.rados.rados.0.smithi100.stdout:2737: done (3 left) 2024-04-07T16:25:42.087 INFO:tasks.rados.rados.0.smithi100.stdout:2738: done (2 left) 2024-04-07T16:25:42.087 INFO:tasks.rados.rados.0.smithi100.stdout:2739: setattr oid 6 current snap is 260 2024-04-07T16:25:42.087 INFO:tasks.rados.rados.0.smithi100.stdout:2736: finishing copy_from racing read to smithi10066867-30 2024-04-07T16:25:42.087 INFO:tasks.rados.rados.0.smithi100.stdout:2736: finishing copy_from to smithi10066867-30 2024-04-07T16:25:42.087 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 30 v 1038 (ObjNum 161 snap 0 seq_num 1) dirty exists 2024-04-07T16:25:42.088 INFO:tasks.rados.rados.0.smithi100.stdout:2736: done (2 left) 2024-04-07T16:25:42.088 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 26 current snap is 260 2024-04-07T16:25:42.088 INFO:tasks.rados.rados.0.smithi100.stdout:2740: seq_num 832 ranges {3817472=737280,4554752=786432,5341184=417792} 2024-04-07T16:25:42.112 INFO:tasks.rados.rados.0.smithi100.stdout:2740: writing smithi10066867-26 from 3817472 to 4554752 tid 1 2024-04-07T16:25:42.116 INFO:tasks.rados.rados.0.smithi100.stdout:2740: writing smithi10066867-26 from 4554752 to 5341184 tid 2 2024-04-07T16:25:42.119 INFO:tasks.rados.rados.0.smithi100.stdout:2740: writing smithi10066867-26 from 5341184 to 5758976 tid 3 2024-04-07T16:25:42.119 INFO:tasks.rados.rados.0.smithi100.stdout:2741: setattr oid 5 current snap is 260 2024-04-07T16:25:42.119 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 6 v 1224 (ObjNum 810 snap 256 seq_num 810) dirty exists 2024-04-07T16:25:42.119 INFO:tasks.rados.rados.0.smithi100.stdout:2735: expect (ObjNum 817 snap 257 seq_num 817) 2024-04-07T16:25:42.304 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:42 smithi144 bash[66654]: cluster 2024-04-07T16:25:40.737353+0000 mgr.y (mgr.25175) 267 : cluster [DBG] pgmap v469: 121 pgs: 8 peering, 8 active+undersized+degraded, 23 active+undersized, 82 active+clean; 185 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 22 MiB/s rd, 5.8 MiB/s wr, 20 op/s; 20/360 objects degraded (5.556%) 2024-04-07T16:25:42.304 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:42 smithi144 bash[66654]: cluster 2024-04-07T16:25:40.974138+0000 mon.a (mon.0) 868 : cluster [WRN] Health check update: Degraded data redundancy: 20/360 objects degraded (5.556%), 8 pgs degraded (PG_DEGRADED) 2024-04-07T16:25:42.305 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:42 smithi144 bash[66654]: cluster 2024-04-07T16:25:40.999088+0000 mon.a (mon.0) 869 : cluster [DBG] osdmap e585: 8 total, 8 up, 8 in 2024-04-07T16:25:42.435 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:42 smithi100 bash[74942]: cluster 2024-04-07T16:25:40.737353+0000 mgr.y (mgr.25175) 267 : cluster [DBG] pgmap v469: 121 pgs: 8 peering, 8 active+undersized+degraded, 23 active+undersized, 82 active+clean; 185 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 22 MiB/s rd, 5.8 MiB/s wr, 20 op/s; 20/360 objects degraded (5.556%) 2024-04-07T16:25:42.435 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:42 smithi100 bash[74942]: cluster 2024-04-07T16:25:40.974138+0000 mon.a (mon.0) 868 : cluster [WRN] Health check update: Degraded data redundancy: 20/360 objects degraded (5.556%), 8 pgs degraded (PG_DEGRADED) 2024-04-07T16:25:42.435 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:42 smithi100 bash[74942]: cluster 2024-04-07T16:25:40.999088+0000 mon.a (mon.0) 869 : cluster [DBG] osdmap e585: 8 total, 8 up, 8 in 2024-04-07T16:25:42.435 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:42 smithi100 bash[76202]: cluster 2024-04-07T16:25:40.737353+0000 mgr.y (mgr.25175) 267 : cluster [DBG] pgmap v469: 121 pgs: 8 peering, 8 active+undersized+degraded, 23 active+undersized, 82 active+clean; 185 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 22 MiB/s rd, 5.8 MiB/s wr, 20 op/s; 20/360 objects degraded (5.556%) 2024-04-07T16:25:42.435 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:42 smithi100 bash[76202]: cluster 2024-04-07T16:25:40.974138+0000 mon.a (mon.0) 868 : cluster [WRN] Health check update: Degraded data redundancy: 20/360 objects degraded (5.556%), 8 pgs degraded (PG_DEGRADED) 2024-04-07T16:25:42.436 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:42 smithi100 bash[76202]: cluster 2024-04-07T16:25:40.999088+0000 mon.a (mon.0) 869 : cluster [DBG] osdmap e585: 8 total, 8 up, 8 in 2024-04-07T16:25:42.606 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:25:42 smithi144 bash[59794]: ts=2024-04-07T16:25:42.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:25:43.062 INFO:tasks.rados.rados.0.smithi100.stdout:2740: finishing write tid 1 to smithi10066867-26 2024-04-07T16:25:43.062 INFO:tasks.rados.rados.0.smithi100.stdout:2740: finishing write tid 2 to smithi10066867-26 2024-04-07T16:25:43.062 INFO:tasks.rados.rados.0.smithi100.stdout:2740: finishing write tid 3 to smithi10066867-26 2024-04-07T16:25:43.062 INFO:tasks.rados.rados.0.smithi100.stdout:2740: finishing write tid 4 to smithi10066867-26 2024-04-07T16:25:43.063 INFO:tasks.rados.rados.0.smithi100.stdout:2740: finishing write tid 5 to smithi10066867-26 2024-04-07T16:25:43.063 INFO:tasks.rados.rados.0.smithi100.stdout:2740: oid 26 updating version 0 to 1189 2024-04-07T16:25:43.063 INFO:tasks.rados.rados.0.smithi100.stdout:2740: oid 26 version 1189 is already newer than 1188 2024-04-07T16:25:43.063 INFO:tasks.rados.rados.0.smithi100.stdout:2740: oid 26 version 1189 is already newer than 1186 2024-04-07T16:25:43.063 INFO:tasks.rados.rados.0.smithi100.stdout:2740: oid 26 version 1189 is already newer than 1187 2024-04-07T16:25:43.063 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 26 v 1189 (ObjNum 832 snap 260 seq_num 832) dirty exists 2024-04-07T16:25:43.063 INFO:tasks.rados.rados.0.smithi100.stdout:2740: left oid 26 (ObjNum 832 snap 260 seq_num 832) 2024-04-07T16:25:43.063 INFO:tasks.rados.rados.0.smithi100.stdout:2735: done (3 left) 2024-04-07T16:25:43.064 INFO:tasks.rados.rados.0.smithi100.stdout:2739: done (2 left) 2024-04-07T16:25:43.064 INFO:tasks.rados.rados.0.smithi100.stdout:2740: done (1 left) 2024-04-07T16:25:43.064 INFO:tasks.rados.rados.0.smithi100.stdout:2742: rollback oid 35 current snap is 260 2024-04-07T16:25:43.064 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 35 to 252 2024-04-07T16:25:43.064 INFO:tasks.rados.rados.0.smithi100.stdout:2743: read oid 22 snap -1 2024-04-07T16:25:43.064 INFO:tasks.rados.rados.0.smithi100.stdout:2743: expect (ObjNum 744 snap 228 seq_num 744) 2024-04-07T16:25:43.064 INFO:tasks.rados.rados.0.smithi100.stdout:2744: read oid 13 snap -1 2024-04-07T16:25:43.065 INFO:tasks.rados.rados.0.smithi100.stdout:2744: expect (ObjNum 0 snap 0 seq_num 0) 2024-04-07T16:25:43.065 INFO:tasks.rados.rados.0.smithi100.stdout:2745: delete oid 27 current snap is 260 2024-04-07T16:25:43.065 INFO:tasks.rados.rados.0.smithi100.stdout:2744: expect (ObjNum 0 snap 0 seq_num 0) 2024-04-07T16:25:43.070 INFO:tasks.rados.rados.0.smithi100.stdout:2742: finishing rollback tid 1 to smithi10066867-35 2024-04-07T16:25:43.071 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 35 v 0 (ObjNum 97 snap 0 seq_num 2670638160) dirty dne 2024-04-07T16:25:43.071 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 5 v 1313 (ObjNum 33 snap 0 seq_num 2702434368) dirty exists 2024-04-07T16:25:43.071 INFO:tasks.rados.rados.0.smithi100.stdout:2741: done (4 left) 2024-04-07T16:25:43.071 INFO:tasks.rados.rados.0.smithi100.stdout:2742: done (3 left) 2024-04-07T16:25:43.071 INFO:tasks.rados.rados.0.smithi100.stdout:2744: done (2 left) 2024-04-07T16:25:43.071 INFO:tasks.rados.rados.0.smithi100.stdout:2745: done (1 left) 2024-04-07T16:25:43.071 INFO:tasks.rados.rados.0.smithi100.stdout:2746: delete oid 29 current snap is 260 2024-04-07T16:25:43.075 INFO:tasks.rados.rados.0.smithi100.stdout:2746: done (1 left) 2024-04-07T16:25:43.075 INFO:tasks.rados.rados.0.smithi100.stdout:2747: snap_remove snap 247 2024-04-07T16:25:43.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:43 smithi144 bash[66654]: cluster 2024-04-07T16:25:41.997467+0000 mon.a (mon.0) 870 : cluster [DBG] osdmap e586: 8 total, 8 up, 8 in 2024-04-07T16:25:43.435 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:43 smithi100 bash[74942]: cluster 2024-04-07T16:25:41.997467+0000 mon.a (mon.0) 870 : cluster [DBG] osdmap e586: 8 total, 8 up, 8 in 2024-04-07T16:25:43.435 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:43 smithi100 bash[76202]: cluster 2024-04-07T16:25:41.997467+0000 mon.a (mon.0) 870 : cluster [DBG] osdmap e586: 8 total, 8 up, 8 in 2024-04-07T16:25:44.021 INFO:tasks.rados.rados.0.smithi100.stdout:2747: done (1 left) 2024-04-07T16:25:44.022 INFO:tasks.rados.rados.0.smithi100.stdout:2748: snap_create 2024-04-07T16:25:44.022 INFO:tasks.rados.rados.0.smithi100.stdout:2743: expect (ObjNum 744 snap 228 seq_num 744) 2024-04-07T16:25:44.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:44 smithi144 bash[66654]: cluster 2024-04-07T16:25:42.738114+0000 mgr.y (mgr.25175) 268 : cluster [DBG] pgmap v472: 121 pgs: 8 peering, 113 active+clean; 192 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 14 MiB/s rd, 4.3 MiB/s wr, 17 op/s; 2.2 MiB/s, 1 objects/s recovering 2024-04-07T16:25:44.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:44 smithi144 bash[66654]: cluster 2024-04-07T16:25:43.008286+0000 mon.a (mon.0) 871 : cluster [INF] Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 20/360 objects degraded (5.556%), 8 pgs degraded) 2024-04-07T16:25:44.435 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:44 smithi100 bash[74942]: cluster 2024-04-07T16:25:42.738114+0000 mgr.y (mgr.25175) 268 : cluster [DBG] pgmap v472: 121 pgs: 8 peering, 113 active+clean; 192 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 14 MiB/s rd, 4.3 MiB/s wr, 17 op/s; 2.2 MiB/s, 1 objects/s recovering 2024-04-07T16:25:44.435 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:44 smithi100 bash[74942]: cluster 2024-04-07T16:25:43.008286+0000 mon.a (mon.0) 871 : cluster [INF] Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 20/360 objects degraded (5.556%), 8 pgs degraded) 2024-04-07T16:25:44.435 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:44 smithi100 bash[76202]: cluster 2024-04-07T16:25:42.738114+0000 mgr.y (mgr.25175) 268 : cluster [DBG] pgmap v472: 121 pgs: 8 peering, 113 active+clean; 192 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 14 MiB/s rd, 4.3 MiB/s wr, 17 op/s; 2.2 MiB/s, 1 objects/s recovering 2024-04-07T16:25:44.435 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:44 smithi100 bash[76202]: cluster 2024-04-07T16:25:43.008286+0000 mon.a (mon.0) 871 : cluster [INF] Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 20/360 objects degraded (5.556%), 8 pgs degraded) 2024-04-07T16:25:45.035 INFO:tasks.rados.rados.0.smithi100.stdout:2743: done (1 left) 2024-04-07T16:25:45.036 INFO:tasks.rados.rados.0.smithi100.stdout:2748: done (0 left) 2024-04-07T16:25:45.036 INFO:tasks.rados.rados.0.smithi100.stdout:2749: rmattr oid 49 current snap is 261 2024-04-07T16:25:45.036 INFO:tasks.rados.rados.0.smithi100.stdout:2750: snap_remove snap 251 2024-04-07T16:25:45.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:45 smithi144 bash[66654]: cluster 2024-04-07T16:25:44.038325+0000 mon.a (mon.0) 872 : cluster [DBG] osdmap e587: 8 total, 8 up, 8 in 2024-04-07T16:25:45.435 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:45 smithi100 bash[74942]: cluster 2024-04-07T16:25:44.038325+0000 mon.a (mon.0) 872 : cluster [DBG] osdmap e587: 8 total, 8 up, 8 in 2024-04-07T16:25:45.435 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:45 smithi100 bash[76202]: cluster 2024-04-07T16:25:44.038325+0000 mon.a (mon.0) 872 : cluster [DBG] osdmap e587: 8 total, 8 up, 8 in 2024-04-07T16:25:46.050 INFO:tasks.rados.rados.0.smithi100.stdout:2750: done (1 left) 2024-04-07T16:25:46.050 INFO:tasks.rados.rados.0.smithi100.stdout:2751: rollback oid 14 current snap is 261 2024-04-07T16:25:46.050 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 14 to 258 2024-04-07T16:25:46.051 INFO:tasks.rados.rados.0.smithi100.stdout:2752: delete oid 47 current snap is 261 2024-04-07T16:25:46.051 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 49 v 1321 (ObjNum 783 snap 246 seq_num 783) dirty exists 2024-04-07T16:25:46.060 INFO:tasks.rados.rados.0.smithi100.stdout:2749: done (2 left) 2024-04-07T16:25:46.060 INFO:tasks.rados.rados.0.smithi100.stdout:2752: done (1 left) 2024-04-07T16:25:46.060 INFO:tasks.rados.rados.0.smithi100.stdout:2753: copy_from oid 1 from oid 9 current snap is 261 2024-04-07T16:25:46.060 INFO:tasks.rados.rados.0.smithi100.stdout:2754: snap_remove snap 249 2024-04-07T16:25:46.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:46 smithi144 bash[66654]: cluster 2024-04-07T16:25:44.739017+0000 mgr.y (mgr.25175) 269 : cluster [DBG] pgmap v474: 121 pgs: 121 active+clean; 196 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 2.9 MiB/s rd, 4.4 MiB/s wr, 27 op/s; 7.3 MiB/s, 2 objects/s recovering 2024-04-07T16:25:46.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:46 smithi144 bash[66654]: cluster 2024-04-07T16:25:45.052337+0000 mon.a (mon.0) 873 : cluster [DBG] osdmap e588: 8 total, 8 up, 8 in 2024-04-07T16:25:46.435 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:46 smithi100 bash[74942]: cluster 2024-04-07T16:25:44.739017+0000 mgr.y (mgr.25175) 269 : cluster [DBG] pgmap v474: 121 pgs: 121 active+clean; 196 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 2.9 MiB/s rd, 4.4 MiB/s wr, 27 op/s; 7.3 MiB/s, 2 objects/s recovering 2024-04-07T16:25:46.435 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:46 smithi100 bash[74942]: cluster 2024-04-07T16:25:45.052337+0000 mon.a (mon.0) 873 : cluster [DBG] osdmap e588: 8 total, 8 up, 8 in 2024-04-07T16:25:46.435 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:46 smithi100 bash[76202]: cluster 2024-04-07T16:25:44.739017+0000 mgr.y (mgr.25175) 269 : cluster [DBG] pgmap v474: 121 pgs: 121 active+clean; 196 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 2.9 MiB/s rd, 4.4 MiB/s wr, 27 op/s; 7.3 MiB/s, 2 objects/s recovering 2024-04-07T16:25:46.436 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:46 smithi100 bash[76202]: cluster 2024-04-07T16:25:45.052337+0000 mon.a (mon.0) 873 : cluster [DBG] osdmap e588: 8 total, 8 up, 8 in 2024-04-07T16:25:46.876 INFO:tasks.rados.rados.0.smithi100.stdout:2754: done (2 left) 2024-04-07T16:25:46.876 INFO:tasks.rados.rados.0.smithi100.stdout:2755: setattr oid 41 current snap is 261 2024-04-07T16:25:46.876 INFO:tasks.rados.rados.0.smithi100.stdout:2756: snap_remove snap 254 2024-04-07T16:25:47.335 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:47 smithi144 bash[66654]: cluster 2024-04-07T16:25:46.052322+0000 mon.a (mon.0) 874 : cluster [DBG] osdmap e589: 8 total, 8 up, 8 in 2024-04-07T16:25:47.335 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:47 smithi144 bash[66654]: audit 2024-04-07T16:25:46.724038+0000 mon.a (mon.0) 875 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:25:47.335 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:47 smithi144 bash[66654]: audit 2024-04-07T16:25:46.726224+0000 mon.a (mon.0) 876 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:25:47.335 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:47 smithi144 bash[66654]: cluster 2024-04-07T16:25:46.888473+0000 mon.a (mon.0) 877 : cluster [DBG] osdmap e590: 8 total, 8 up, 8 in 2024-04-07T16:25:47.435 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:47 smithi100 bash[74942]: cluster 2024-04-07T16:25:46.052322+0000 mon.a (mon.0) 874 : cluster [DBG] osdmap e589: 8 total, 8 up, 8 in 2024-04-07T16:25:47.435 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:47 smithi100 bash[74942]: audit 2024-04-07T16:25:46.724038+0000 mon.a (mon.0) 875 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:25:47.435 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:47 smithi100 bash[74942]: audit 2024-04-07T16:25:46.726224+0000 mon.a (mon.0) 876 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:25:47.435 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:47 smithi100 bash[74942]: cluster 2024-04-07T16:25:46.888473+0000 mon.a (mon.0) 877 : cluster [DBG] osdmap e590: 8 total, 8 up, 8 in 2024-04-07T16:25:47.436 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:47 smithi100 bash[76202]: cluster 2024-04-07T16:25:46.052322+0000 mon.a (mon.0) 874 : cluster [DBG] osdmap e589: 8 total, 8 up, 8 in 2024-04-07T16:25:47.436 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:47 smithi100 bash[76202]: audit 2024-04-07T16:25:46.724038+0000 mon.a (mon.0) 875 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:25:47.436 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:47 smithi100 bash[76202]: audit 2024-04-07T16:25:46.726224+0000 mon.a (mon.0) 876 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:25:47.436 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:47 smithi100 bash[76202]: cluster 2024-04-07T16:25:46.888473+0000 mon.a (mon.0) 877 : cluster [DBG] osdmap e590: 8 total, 8 up, 8 in 2024-04-07T16:25:47.890 INFO:tasks.rados.rados.0.smithi100.stdout:2751: finishing rollback tid 0 to smithi10066867-14 2024-04-07T16:25:47.890 INFO:tasks.rados.rados.0.smithi100.stdout:2751: finishing rollback tid 1 to smithi10066867-14 2024-04-07T16:25:47.890 INFO:tasks.rados.rados.0.smithi100.stdout:2751: finishing rollback tid 2 to smithi10066867-14 2024-04-07T16:25:47.890 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 14 v 1127 (ObjNum 762 snap 240 seq_num 762) dirty exists 2024-04-07T16:25:47.890 INFO:tasks.rados.rados.0.smithi100.stdout:2753: finishing copy_from racing read to smithi10066867-1 2024-04-07T16:25:47.890 INFO:tasks.rados.rados.0.smithi100.stdout:2753: finishing copy_from to smithi10066867-1 2024-04-07T16:25:47.890 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 1 v 1121 (ObjNum 826 snap 258 seq_num 826) dirty exists 2024-04-07T16:25:47.890 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 41 v 1329 (ObjNum 758 snap 239 seq_num 758) dirty exists 2024-04-07T16:25:47.890 INFO:tasks.rados.rados.0.smithi100.stdout:2751: done (3 left) 2024-04-07T16:25:47.890 INFO:tasks.rados.rados.0.smithi100.stdout:2753: done (2 left) 2024-04-07T16:25:47.890 INFO:tasks.rados.rados.0.smithi100.stdout:2755: done (1 left) 2024-04-07T16:25:47.891 INFO:tasks.rados.rados.0.smithi100.stdout:2756: done (0 left) 2024-04-07T16:25:47.891 INFO:tasks.rados.rados.0.smithi100.stdout:2757: delete oid 29 current snap is 261 2024-04-07T16:25:47.925 INFO:tasks.rados.rados.0.smithi100.stdout:2757: done (0 left) 2024-04-07T16:25:47.925 INFO:tasks.rados.rados.0.smithi100.stdout:2758: copy_from oid 14 from oid 22 current snap is 261 2024-04-07T16:25:47.925 INFO:tasks.rados.rados.0.smithi100.stdout:2759: delete oid 40 current snap is 261 2024-04-07T16:25:47.932 INFO:tasks.rados.rados.0.smithi100.stdout:2759: done (1 left) 2024-04-07T16:25:47.932 INFO:tasks.rados.rados.0.smithi100.stdout:2760: delete oid 21 current snap is 261 2024-04-07T16:25:47.936 INFO:tasks.rados.rados.0.smithi100.stdout:2760: done (1 left) 2024-04-07T16:25:47.936 INFO:tasks.rados.rados.0.smithi100.stdout:2761: rmattr oid 28 current snap is 261 2024-04-07T16:25:47.936 INFO:tasks.rados.rados.0.smithi100.stdout:2761: done (1 left) 2024-04-07T16:25:47.936 INFO:tasks.rados.rados.0.smithi100.stdout:2762: read oid 7 snap 255 2024-04-07T16:25:47.936 INFO:tasks.rados.rados.0.smithi100.stdout:2762: expect (ObjNum 752 snap 234 seq_num 752) 2024-04-07T16:25:47.936 INFO:tasks.rados.rados.0.smithi100.stdout:2763: delete oid 28 current snap is 261 2024-04-07T16:25:47.939 INFO:tasks.rados.rados.0.smithi100.stdout:2763: done (2 left) 2024-04-07T16:25:47.939 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 3 current snap is 261 2024-04-07T16:25:47.940 INFO:tasks.rados.rados.0.smithi100.stdout:2764: seq_num 840 ranges {0=475136,475136=712704,1187840=499712,1687552=393216} 2024-04-07T16:25:47.942 INFO:tasks.rados.rados.0.smithi100.stdout:2764: writing smithi10066867-3 from 0 to 475136 tid 1 2024-04-07T16:25:47.946 INFO:tasks.rados.rados.0.smithi100.stdout:2764: writing smithi10066867-3 from 475136 to 1187840 tid 2 2024-04-07T16:25:47.949 INFO:tasks.rados.rados.0.smithi100.stdout:2764: writing smithi10066867-3 from 1187840 to 1687552 tid 3 2024-04-07T16:25:47.952 INFO:tasks.rados.rados.0.smithi100.stdout:2764: writing smithi10066867-3 from 1687552 to 2080768 tid 4 2024-04-07T16:25:47.952 INFO:tasks.rados.rados.0.smithi100.stdout:2764: finishing write tid 1 to smithi10066867-3 2024-04-07T16:25:47.952 INFO:tasks.rados.rados.0.smithi100.stdout:2765: delete oid 27 current snap is 261 2024-04-07T16:25:47.953 INFO:tasks.rados.rados.0.smithi100.stdout:2758: finishing copy_from racing read to smithi10066867-14 2024-04-07T16:25:47.957 INFO:tasks.rados.rados.0.smithi100.stdout:2758: finishing copy_from to smithi10066867-14 2024-04-07T16:25:47.957 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 14 v 1174 (ObjNum 744 snap 228 seq_num 744) dirty exists 2024-04-07T16:25:47.965 INFO:tasks.rados.rados.0.smithi100.stdout:2764: finishing write tid 2 to smithi10066867-3 2024-04-07T16:25:47.966 INFO:tasks.rados.rados.0.smithi100.stdout:2764: finishing write tid 3 to smithi10066867-3 2024-04-07T16:25:47.966 INFO:tasks.rados.rados.0.smithi100.stdout:2764: finishing write tid 4 to smithi10066867-3 2024-04-07T16:25:47.966 INFO:tasks.rados.rados.0.smithi100.stdout:2764: finishing write tid 5 to smithi10066867-3 2024-04-07T16:25:47.968 INFO:tasks.rados.rados.0.smithi100.stdout:2764: finishing write tid 6 to smithi10066867-3 2024-04-07T16:25:47.968 INFO:tasks.rados.rados.0.smithi100.stdout:2764: oid 3 updating version 0 to 1111 2024-04-07T16:25:47.968 INFO:tasks.rados.rados.0.smithi100.stdout:2764: oid 3 updating version 1111 to 1112 2024-04-07T16:25:47.968 INFO:tasks.rados.rados.0.smithi100.stdout:2764: oid 3 version 1112 is already newer than 1109 2024-04-07T16:25:47.968 INFO:tasks.rados.rados.0.smithi100.stdout:2764: oid 3 updating version 1112 to 1113 2024-04-07T16:25:47.968 INFO:tasks.rados.rados.0.smithi100.stdout:2764: oid 3 version 1113 is already newer than 1110 2024-04-07T16:25:47.968 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 3 v 1113 (ObjNum 840 snap 261 seq_num 840) dirty exists 2024-04-07T16:25:47.968 INFO:tasks.rados.rados.0.smithi100.stdout:2764: left oid 3 (ObjNum 840 snap 261 seq_num 840) 2024-04-07T16:25:47.971 INFO:tasks.rados.rados.0.smithi100.stdout:2762: expect (ObjNum 752 snap 234 seq_num 752) 2024-04-07T16:25:48.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:48 smithi144 bash[66654]: cluster 2024-04-07T16:25:46.739697+0000 mgr.y (mgr.25175) 270 : cluster [DBG] pgmap v477: 121 pgs: 121 active+clean; 195 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 2.5 MiB/s rd, 3.1 MiB/s wr, 24 op/s; 5.5 MiB/s, 1 objects/s recovering 2024-04-07T16:25:48.375 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:48 smithi144 bash[66654]: audit 2024-04-07T16:25:47.889567+0000 mon.a (mon.0) 878 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:25:48.375 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:48 smithi144 bash[66654]: cluster 2024-04-07T16:25:47.901996+0000 mon.a (mon.0) 879 : cluster [DBG] osdmap e591: 8 total, 8 up, 8 in 2024-04-07T16:25:48.375 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:48 smithi144 bash[66654]: audit 2024-04-07T16:25:47.916873+0000 mon.a (mon.0) 880 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:25:48.375 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:48 smithi144 bash[66654]: audit 2024-04-07T16:25:47.920094+0000 mon.a (mon.0) 881 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:25:48.375 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:48 smithi144 bash[66654]: audit 2024-04-07T16:25:47.922629+0000 mon.a (mon.0) 882 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:25:48.375 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:48 smithi144 bash[66654]: audit 2024-04-07T16:25:47.937022+0000 mon.a (mon.0) 883 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:25:48.375 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:48 smithi144 bash[66654]: audit 2024-04-07T16:25:48.001592+0000 mon.a (mon.0) 884 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:25:48.375 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:48 smithi144 bash[66654]: audit 2024-04-07T16:25:48.003884+0000 mon.a (mon.0) 885 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:25:48.376 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:48 smithi144 bash[66654]: audit 2024-04-07T16:25:48.005748+0000 mon.a (mon.0) 886 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:25:48.376 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:48 smithi144 bash[66654]: audit 2024-04-07T16:25:48.006806+0000 mon.a (mon.0) 887 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:25:48.376 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:48 smithi144 bash[66654]: audit 2024-04-07T16:25:48.008895+0000 mon.a (mon.0) 888 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:25:48.376 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:48 smithi144 bash[66654]: audit 2024-04-07T16:25:48.013731+0000 mon.a (mon.0) 889 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:25:48.376 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:48 smithi144 bash[66654]: audit 2024-04-07T16:25:48.015892+0000 mon.a (mon.0) 890 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]: dispatch 2024-04-07T16:25:48.376 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:48 smithi144 bash[66654]: audit 2024-04-07T16:25:48.019896+0000 mon.a (mon.0) 891 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]': finished 2024-04-07T16:25:48.376 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:48 smithi144 bash[66654]: audit 2024-04-07T16:25:48.021753+0000 mon.a (mon.0) 892 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]: dispatch 2024-04-07T16:25:48.376 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:48 smithi144 bash[66654]: audit 2024-04-07T16:25:48.025782+0000 mon.a (mon.0) 893 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]': finished 2024-04-07T16:25:48.376 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:48 smithi144 bash[66654]: audit 2024-04-07T16:25:48.027598+0000 mon.a (mon.0) 894 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]: dispatch 2024-04-07T16:25:48.376 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:48 smithi144 bash[66654]: audit 2024-04-07T16:25:48.032432+0000 mon.a (mon.0) 895 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]': finished 2024-04-07T16:25:48.376 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:48 smithi144 bash[66654]: audit 2024-04-07T16:25:48.034172+0000 mon.a (mon.0) 896 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]: dispatch 2024-04-07T16:25:48.376 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:48 smithi144 bash[66654]: audit 2024-04-07T16:25:48.038344+0000 mon.a (mon.0) 897 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]': finished 2024-04-07T16:25:48.376 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:48 smithi144 bash[66654]: audit 2024-04-07T16:25:48.040017+0000 mon.a (mon.0) 898 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]: dispatch 2024-04-07T16:25:48.377 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:48 smithi144 bash[66654]: audit 2024-04-07T16:25:48.045411+0000 mon.a (mon.0) 899 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]': finished 2024-04-07T16:25:48.377 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:48 smithi144 bash[66654]: audit 2024-04-07T16:25:48.047149+0000 mon.a (mon.0) 900 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]: dispatch 2024-04-07T16:25:48.377 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:48 smithi144 bash[66654]: audit 2024-04-07T16:25:48.052570+0000 mon.a (mon.0) 901 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]': finished 2024-04-07T16:25:48.377 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:48 smithi144 bash[66654]: audit 2024-04-07T16:25:48.054422+0000 mon.a (mon.0) 902 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.6"}]: dispatch 2024-04-07T16:25:48.377 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:48 smithi144 bash[66654]: audit 2024-04-07T16:25:48.059668+0000 mon.a (mon.0) 903 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.6"}]': finished 2024-04-07T16:25:48.377 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:48 smithi144 bash[66654]: audit 2024-04-07T16:25:48.061585+0000 mon.a (mon.0) 904 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.7"}]: dispatch 2024-04-07T16:25:48.377 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:48 smithi144 bash[66654]: audit 2024-04-07T16:25:48.064109+0000 mon.a (mon.0) 905 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.7"}]': finished 2024-04-07T16:25:48.377 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:48 smithi144 bash[66654]: audit 2024-04-07T16:25:48.068476+0000 mon.a (mon.0) 906 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd require-osd-release", "release": "squid"}]: dispatch 2024-04-07T16:25:48.417 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:48 smithi100 bash[76202]: cluster 2024-04-07T16:25:46.739697+0000 mgr.y (mgr.25175) 270 : cluster [DBG] pgmap v477: 121 pgs: 121 active+clean; 195 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 2.5 MiB/s rd, 3.1 MiB/s wr, 24 op/s; 5.5 MiB/s, 1 objects/s recovering 2024-04-07T16:25:48.417 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:48 smithi100 bash[76202]: audit 2024-04-07T16:25:47.889567+0000 mon.a (mon.0) 878 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:25:48.417 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:48 smithi100 bash[76202]: cluster 2024-04-07T16:25:47.901996+0000 mon.a (mon.0) 879 : cluster [DBG] osdmap e591: 8 total, 8 up, 8 in 2024-04-07T16:25:48.417 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:48 smithi100 bash[76202]: audit 2024-04-07T16:25:47.916873+0000 mon.a (mon.0) 880 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:25:48.418 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:48 smithi100 bash[76202]: audit 2024-04-07T16:25:47.920094+0000 mon.a (mon.0) 881 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:25:48.418 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:48 smithi100 bash[76202]: audit 2024-04-07T16:25:47.922629+0000 mon.a (mon.0) 882 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:25:48.418 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:48 smithi100 bash[76202]: audit 2024-04-07T16:25:47.937022+0000 mon.a (mon.0) 883 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:25:48.418 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:48 smithi100 bash[76202]: audit 2024-04-07T16:25:48.001592+0000 mon.a (mon.0) 884 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:25:48.418 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:48 smithi100 bash[76202]: audit 2024-04-07T16:25:48.003884+0000 mon.a (mon.0) 885 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:25:48.418 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:48 smithi100 bash[76202]: audit 2024-04-07T16:25:48.005748+0000 mon.a (mon.0) 886 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:25:48.418 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:48 smithi100 bash[76202]: audit 2024-04-07T16:25:48.006806+0000 mon.a (mon.0) 887 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:25:48.418 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:48 smithi100 bash[76202]: audit 2024-04-07T16:25:48.008895+0000 mon.a (mon.0) 888 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:25:48.418 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:48 smithi100 bash[76202]: audit 2024-04-07T16:25:48.013731+0000 mon.a (mon.0) 889 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:25:48.418 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:48 smithi100 bash[76202]: audit 2024-04-07T16:25:48.015892+0000 mon.a (mon.0) 890 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]: dispatch 2024-04-07T16:25:48.418 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:48 smithi100 bash[76202]: audit 2024-04-07T16:25:48.019896+0000 mon.a (mon.0) 891 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]': finished 2024-04-07T16:25:48.418 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:48 smithi100 bash[76202]: audit 2024-04-07T16:25:48.021753+0000 mon.a (mon.0) 892 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]: dispatch 2024-04-07T16:25:48.418 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:48 smithi100 bash[76202]: audit 2024-04-07T16:25:48.025782+0000 mon.a (mon.0) 893 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]': finished 2024-04-07T16:25:48.419 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:48 smithi100 bash[76202]: audit 2024-04-07T16:25:48.027598+0000 mon.a (mon.0) 894 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]: dispatch 2024-04-07T16:25:48.419 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:48 smithi100 bash[76202]: audit 2024-04-07T16:25:48.032432+0000 mon.a (mon.0) 895 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]': finished 2024-04-07T16:25:48.419 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:48 smithi100 bash[76202]: audit 2024-04-07T16:25:48.034172+0000 mon.a (mon.0) 896 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]: dispatch 2024-04-07T16:25:48.419 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:48 smithi100 bash[76202]: audit 2024-04-07T16:25:48.038344+0000 mon.a (mon.0) 897 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]': finished 2024-04-07T16:25:48.419 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:48 smithi100 bash[76202]: audit 2024-04-07T16:25:48.040017+0000 mon.a (mon.0) 898 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]: dispatch 2024-04-07T16:25:48.419 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:48 smithi100 bash[76202]: audit 2024-04-07T16:25:48.045411+0000 mon.a (mon.0) 899 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]': finished 2024-04-07T16:25:48.419 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:48 smithi100 bash[76202]: audit 2024-04-07T16:25:48.047149+0000 mon.a (mon.0) 900 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]: dispatch 2024-04-07T16:25:48.419 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:48 smithi100 bash[76202]: audit 2024-04-07T16:25:48.052570+0000 mon.a (mon.0) 901 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]': finished 2024-04-07T16:25:48.419 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:48 smithi100 bash[76202]: audit 2024-04-07T16:25:48.054422+0000 mon.a (mon.0) 902 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.6"}]: dispatch 2024-04-07T16:25:48.419 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:48 smithi100 bash[76202]: audit 2024-04-07T16:25:48.059668+0000 mon.a (mon.0) 903 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.6"}]': finished 2024-04-07T16:25:48.419 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:48 smithi100 bash[76202]: audit 2024-04-07T16:25:48.061585+0000 mon.a (mon.0) 904 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.7"}]: dispatch 2024-04-07T16:25:48.419 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:48 smithi100 bash[76202]: audit 2024-04-07T16:25:48.064109+0000 mon.a (mon.0) 905 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.7"}]': finished 2024-04-07T16:25:48.419 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:48 smithi100 bash[76202]: audit 2024-04-07T16:25:48.068476+0000 mon.a (mon.0) 906 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd require-osd-release", "release": "squid"}]: dispatch 2024-04-07T16:25:48.420 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:48 smithi100 bash[74942]: cluster 2024-04-07T16:25:46.739697+0000 mgr.y (mgr.25175) 270 : cluster [DBG] pgmap v477: 121 pgs: 121 active+clean; 195 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 2.5 MiB/s rd, 3.1 MiB/s wr, 24 op/s; 5.5 MiB/s, 1 objects/s recovering 2024-04-07T16:25:48.420 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:48 smithi100 bash[74942]: audit 2024-04-07T16:25:47.889567+0000 mon.a (mon.0) 878 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:25:48.420 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:48 smithi100 bash[74942]: cluster 2024-04-07T16:25:47.901996+0000 mon.a (mon.0) 879 : cluster [DBG] osdmap e591: 8 total, 8 up, 8 in 2024-04-07T16:25:48.420 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:48 smithi100 bash[74942]: audit 2024-04-07T16:25:47.916873+0000 mon.a (mon.0) 880 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:25:48.420 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:48 smithi100 bash[74942]: audit 2024-04-07T16:25:47.920094+0000 mon.a (mon.0) 881 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:25:48.420 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:48 smithi100 bash[74942]: audit 2024-04-07T16:25:47.922629+0000 mon.a (mon.0) 882 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:25:48.420 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:48 smithi100 bash[74942]: audit 2024-04-07T16:25:47.937022+0000 mon.a (mon.0) 883 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:25:48.420 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:48 smithi100 bash[74942]: audit 2024-04-07T16:25:48.001592+0000 mon.a (mon.0) 884 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:25:48.420 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:48 smithi100 bash[74942]: audit 2024-04-07T16:25:48.003884+0000 mon.a (mon.0) 885 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:25:48.420 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:48 smithi100 bash[74942]: audit 2024-04-07T16:25:48.005748+0000 mon.a (mon.0) 886 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:25:48.420 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:48 smithi100 bash[74942]: audit 2024-04-07T16:25:48.006806+0000 mon.a (mon.0) 887 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:25:48.420 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:48 smithi100 bash[74942]: audit 2024-04-07T16:25:48.008895+0000 mon.a (mon.0) 888 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:25:48.421 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:48 smithi100 bash[74942]: audit 2024-04-07T16:25:48.013731+0000 mon.a (mon.0) 889 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:25:48.421 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:48 smithi100 bash[74942]: audit 2024-04-07T16:25:48.015892+0000 mon.a (mon.0) 890 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]: dispatch 2024-04-07T16:25:48.421 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:48 smithi100 bash[74942]: audit 2024-04-07T16:25:48.019896+0000 mon.a (mon.0) 891 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]': finished 2024-04-07T16:25:48.421 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:48 smithi100 bash[74942]: audit 2024-04-07T16:25:48.021753+0000 mon.a (mon.0) 892 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]: dispatch 2024-04-07T16:25:48.421 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:48 smithi100 bash[74942]: audit 2024-04-07T16:25:48.025782+0000 mon.a (mon.0) 893 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]': finished 2024-04-07T16:25:48.421 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:48 smithi100 bash[74942]: audit 2024-04-07T16:25:48.027598+0000 mon.a (mon.0) 894 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]: dispatch 2024-04-07T16:25:48.422 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:48 smithi100 bash[74942]: audit 2024-04-07T16:25:48.032432+0000 mon.a (mon.0) 895 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]': finished 2024-04-07T16:25:48.422 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:48 smithi100 bash[74942]: audit 2024-04-07T16:25:48.034172+0000 mon.a (mon.0) 896 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]: dispatch 2024-04-07T16:25:48.422 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:48 smithi100 bash[74942]: audit 2024-04-07T16:25:48.038344+0000 mon.a (mon.0) 897 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]': finished 2024-04-07T16:25:48.422 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:48 smithi100 bash[74942]: audit 2024-04-07T16:25:48.040017+0000 mon.a (mon.0) 898 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]: dispatch 2024-04-07T16:25:48.422 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:48 smithi100 bash[74942]: audit 2024-04-07T16:25:48.045411+0000 mon.a (mon.0) 899 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]': finished 2024-04-07T16:25:48.422 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:48 smithi100 bash[74942]: audit 2024-04-07T16:25:48.047149+0000 mon.a (mon.0) 900 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]: dispatch 2024-04-07T16:25:48.422 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:48 smithi100 bash[74942]: audit 2024-04-07T16:25:48.052570+0000 mon.a (mon.0) 901 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]': finished 2024-04-07T16:25:48.422 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:48 smithi100 bash[74942]: audit 2024-04-07T16:25:48.054422+0000 mon.a (mon.0) 902 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.6"}]: dispatch 2024-04-07T16:25:48.422 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:48 smithi100 bash[74942]: audit 2024-04-07T16:25:48.059668+0000 mon.a (mon.0) 903 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.6"}]': finished 2024-04-07T16:25:48.422 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:48 smithi100 bash[74942]: audit 2024-04-07T16:25:48.061585+0000 mon.a (mon.0) 904 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.7"}]: dispatch 2024-04-07T16:25:48.422 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:48 smithi100 bash[74942]: audit 2024-04-07T16:25:48.064109+0000 mon.a (mon.0) 905 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.7"}]': finished 2024-04-07T16:25:48.422 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:48 smithi100 bash[74942]: audit 2024-04-07T16:25:48.068476+0000 mon.a (mon.0) 906 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd require-osd-release", "release": "squid"}]: dispatch 2024-04-07T16:25:48.485 INFO:tasks.rados.rados.0.smithi100.stdout:2758: done (3 left) 2024-04-07T16:25:48.486 INFO:tasks.rados.rados.0.smithi100.stdout:2762: done (2 left) 2024-04-07T16:25:48.486 INFO:tasks.rados.rados.0.smithi100.stdout:2764: done (1 left) 2024-04-07T16:25:48.486 INFO:tasks.rados.rados.0.smithi100.stdout:2765: done (0 left) 2024-04-07T16:25:48.486 INFO:tasks.rados.rados.0.smithi100.stdout:2766: snap_remove snap 259 2024-04-07T16:25:48.685 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:25:48 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:25:48] "GET /metrics HTTP/1.1" 200 36353 "" "Prometheus/2.43.0" 2024-04-07T16:25:49.059 INFO:tasks.rados.rados.0.smithi100.stdout:2766: done (0 left) 2024-04-07T16:25:49.059 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 22 current snap is 261 2024-04-07T16:25:49.059 INFO:tasks.rados.rados.0.smithi100.stdout:2767: seq_num 842 ranges {3375104=507904,3883008=475136,4358144=516096,4874240=499712,5373952=114688} 2024-04-07T16:25:49.085 INFO:tasks.rados.rados.0.smithi100.stdout:2767: writing smithi10066867-22 from 3375104 to 3883008 tid 1 2024-04-07T16:25:49.088 INFO:tasks.rados.rados.0.smithi100.stdout:2767: writing smithi10066867-22 from 3883008 to 4358144 tid 2 2024-04-07T16:25:49.092 INFO:tasks.rados.rados.0.smithi100.stdout:2767: writing smithi10066867-22 from 4358144 to 4874240 tid 3 2024-04-07T16:25:49.096 INFO:tasks.rados.rados.0.smithi100.stdout:2767: writing smithi10066867-22 from 4874240 to 5373952 tid 4 2024-04-07T16:25:49.097 INFO:tasks.rados.rados.0.smithi100.stdout:2767: writing smithi10066867-22 from 5373952 to 5488640 tid 5 2024-04-07T16:25:49.098 INFO:tasks.rados.rados.0.smithi100.stdout:2768: read oid 40 snap -1 2024-04-07T16:25:49.098 INFO:tasks.rados.rados.0.smithi100.stdout:2768: expect deleted 2024-04-07T16:25:49.098 INFO:tasks.rados.rados.0.smithi100.stdout:2769: rollback oid 16 current snap is 261 2024-04-07T16:25:49.098 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 16 to 248 2024-04-07T16:25:49.098 INFO:tasks.rados.rados.0.smithi100.stdout:2770: read oid 15 snap -1 2024-04-07T16:25:49.098 INFO:tasks.rados.rados.0.smithi100.stdout:2770: expect (ObjNum 776 snap 243 seq_num 776) 2024-04-07T16:25:49.098 INFO:tasks.rados.rados.0.smithi100.stdout:2771: rmattr oid 3 current snap is 261 2024-04-07T16:25:49.098 INFO:tasks.rados.rados.0.smithi100.stdout:2771: done (4 left) 2024-04-07T16:25:49.098 INFO:tasks.rados.rados.0.smithi100.stdout:2772: copy_from oid 3 from oid 21 current snap is 261 2024-04-07T16:25:49.098 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 11 current snap is 261 2024-04-07T16:25:49.098 INFO:tasks.rados.rados.0.smithi100.stdout:2773: seq_num 843 ranges {2449408=524288,2973696=761856,3735552=671744,4407296=172032} 2024-04-07T16:25:49.116 INFO:tasks.rados.rados.0.smithi100.stdout:2773: writing smithi10066867-11 from 2449408 to 2973696 tid 1 2024-04-07T16:25:49.120 INFO:tasks.rados.rados.0.smithi100.stdout:2773: writing smithi10066867-11 from 2973696 to 3735552 tid 2 2024-04-07T16:25:49.125 INFO:tasks.rados.rados.0.smithi100.stdout:2773: writing smithi10066867-11 from 3735552 to 4407296 tid 3 2024-04-07T16:25:49.126 INFO:tasks.rados.rados.0.smithi100.stdout:2773: writing smithi10066867-11 from 4407296 to 4579328 tid 4 2024-04-07T16:25:49.127 INFO:tasks.rados.rados.0.smithi100.stdout:2774: delete oid 23 current snap is 261 2024-04-07T16:25:49.127 INFO:tasks.rados.rados.0.smithi100.stdout:2767: finishing write tid 1 to smithi10066867-22 2024-04-07T16:25:49.127 INFO:tasks.rados.rados.0.smithi100.stdout:2767: finishing write tid 2 to smithi10066867-22 2024-04-07T16:25:49.127 INFO:tasks.rados.rados.0.smithi100.stdout:2772: finishing copy_from to smithi10066867-3 2024-04-07T16:25:49.127 INFO:tasks.rados.rados.0.smithi100.stdout:2772: got expected ENOENT (src dne) 2024-04-07T16:25:49.127 INFO:tasks.rados.rados.0.smithi100.stdout:2772: finishing copy_from racing read to smithi10066867-3 2024-04-07T16:25:49.127 INFO:tasks.rados.rados.0.smithi100.stdout:2769: finishing rollback tid 1 to smithi10066867-16 2024-04-07T16:25:49.127 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 16 v 1278 (ObjNum 65 snap 0 seq_num 2670633904) dirty dne 2024-04-07T16:25:49.127 INFO:tasks.rados.rados.0.smithi100.stdout:2767: finishing write tid 3 to smithi10066867-22 2024-04-07T16:25:49.127 INFO:tasks.rados.rados.0.smithi100.stdout:2767: finishing write tid 4 to smithi10066867-22 2024-04-07T16:25:49.127 INFO:tasks.rados.rados.0.smithi100.stdout:2767: finishing write tid 5 to smithi10066867-22 2024-04-07T16:25:49.127 INFO:tasks.rados.rados.0.smithi100.stdout:2767: finishing write tid 6 to smithi10066867-22 2024-04-07T16:25:49.127 INFO:tasks.rados.rados.0.smithi100.stdout:2770: expect (ObjNum 776 snap 243 seq_num 776) 2024-04-07T16:25:49.310 INFO:tasks.rados.rados.0.smithi100.stdout:2767: finishing write tid 7 to smithi10066867-22 2024-04-07T16:25:49.310 INFO:tasks.rados.rados.0.smithi100.stdout:2767: oid 22 updating version 0 to 1191 2024-04-07T16:25:49.310 INFO:tasks.rados.rados.0.smithi100.stdout:2767: oid 22 updating version 1191 to 1192 2024-04-07T16:25:49.310 INFO:tasks.rados.rados.0.smithi100.stdout:2767: oid 22 updating version 1192 to 1195 2024-04-07T16:25:49.310 INFO:tasks.rados.rados.0.smithi100.stdout:2767: oid 22 version 1195 is already newer than 1190 2024-04-07T16:25:49.310 INFO:tasks.rados.rados.0.smithi100.stdout:2767: oid 22 version 1195 is already newer than 1194 2024-04-07T16:25:49.311 INFO:tasks.rados.rados.0.smithi100.stdout:2767: oid 22 version 1195 is already newer than 1193 2024-04-07T16:25:49.311 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 22 v 1195 (ObjNum 842 snap 261 seq_num 842) dirty exists 2024-04-07T16:25:49.311 INFO:tasks.rados.rados.0.smithi100.stdout:2767: left oid 22 (ObjNum 842 snap 261 seq_num 842) 2024-04-07T16:25:49.311 INFO:tasks.rados.rados.0.smithi100.stdout:2773: finishing write tid 1 to smithi10066867-11 2024-04-07T16:25:49.311 INFO:tasks.rados.rados.0.smithi100.stdout:2773: finishing write tid 2 to smithi10066867-11 2024-04-07T16:25:49.311 INFO:tasks.rados.rados.0.smithi100.stdout:2773: finishing write tid 3 to smithi10066867-11 2024-04-07T16:25:49.311 INFO:tasks.rados.rados.0.smithi100.stdout:2773: finishing write tid 4 to smithi10066867-11 2024-04-07T16:25:49.311 INFO:tasks.rados.rados.0.smithi100.stdout:2773: finishing write tid 5 to smithi10066867-11 2024-04-07T16:25:49.311 INFO:tasks.rados.rados.0.smithi100.stdout:2773: finishing write tid 6 to smithi10066867-11 2024-04-07T16:25:49.311 INFO:tasks.rados.rados.0.smithi100.stdout:2773: oid 11 updating version 0 to 1555 2024-04-07T16:25:49.311 INFO:tasks.rados.rados.0.smithi100.stdout:2773: oid 11 version 1555 is already newer than 1553 2024-04-07T16:25:49.311 INFO:tasks.rados.rados.0.smithi100.stdout:2773: oid 11 version 1555 is already newer than 1552 2024-04-07T16:25:49.311 INFO:tasks.rados.rados.0.smithi100.stdout:2773: oid 11 version 1555 is already newer than 1554 2024-04-07T16:25:49.311 INFO:tasks.rados.rados.0.smithi100.stdout:2773: oid 11 updating version 1555 to 1556 2024-04-07T16:25:49.311 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 11 v 1556 (ObjNum 843 snap 261 seq_num 843) dirty exists 2024-04-07T16:25:49.312 INFO:tasks.rados.rados.0.smithi100.stdout:2773: left oid 11 (ObjNum 843 snap 261 seq_num 843) 2024-04-07T16:25:49.312 INFO:tasks.rados.rados.0.smithi100.stdout:2767: done (6 left) 2024-04-07T16:25:49.312 INFO:tasks.rados.rados.0.smithi100.stdout:2768: done (5 left) 2024-04-07T16:25:49.312 INFO:tasks.rados.rados.0.smithi100.stdout:2769: done (4 left) 2024-04-07T16:25:49.312 INFO:tasks.rados.rados.0.smithi100.stdout:2770: done (3 left) 2024-04-07T16:25:49.312 INFO:tasks.rados.rados.0.smithi100.stdout:2772: done (2 left) 2024-04-07T16:25:49.312 INFO:tasks.rados.rados.0.smithi100.stdout:2773: done (1 left) 2024-04-07T16:25:49.312 INFO:tasks.rados.rados.0.smithi100.stdout:2774: done (0 left) 2024-04-07T16:25:49.312 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 2 current snap is 261 2024-04-07T16:25:49.312 INFO:tasks.rados.rados.0.smithi100.stdout:2775: seq_num 845 ranges {1441792=557056,1998848=581632,2580480=688128,3268608=335872} 2024-04-07T16:25:49.327 INFO:tasks.rados.rados.0.smithi100.stdout:2775: writing smithi10066867-2 from 1441792 to 1998848 tid 1 2024-04-07T16:25:49.331 INFO:tasks.rados.rados.0.smithi100.stdout:2775: writing smithi10066867-2 from 1998848 to 2580480 tid 2 2024-04-07T16:25:49.338 INFO:tasks.rados.rados.0.smithi100.stdout:2775: writing smithi10066867-2 from 2580480 to 3268608 tid 3 2024-04-07T16:25:49.340 INFO:tasks.rados.rados.0.smithi100.stdout:2775: writing smithi10066867-2 from 3268608 to 3604480 tid 4 2024-04-07T16:25:49.341 INFO:tasks.rados.rados.0.smithi100.stdout:2776: delete oid 14 current snap is 261 2024-04-07T16:25:49.341 INFO:tasks.rados.rados.0.smithi100.stdout:2775: finishing write tid 1 to smithi10066867-2 2024-04-07T16:25:49.341 INFO:tasks.rados.rados.0.smithi100.stdout:2775: finishing write tid 2 to smithi10066867-2 2024-04-07T16:25:49.343 INFO:tasks.rados.rados.0.smithi100.stdout:2775: finishing write tid 3 to smithi10066867-2 2024-04-07T16:25:49.345 INFO:tasks.rados.rados.0.smithi100.stdout:2776: done (1 left) 2024-04-07T16:25:49.345 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 38 current snap is 261 2024-04-07T16:25:49.345 INFO:tasks.rados.rados.0.smithi100.stdout:2777: seq_num 847 ranges {0=589824,589824=753664,1343488=655360,1998848=196608} 2024-04-07T16:25:49.349 INFO:tasks.rados.rados.0.smithi100.stdout:2777: writing smithi10066867-38 from 0 to 589824 tid 1 2024-04-07T16:25:49.353 INFO:tasks.rados.rados.0.smithi100.stdout:2777: writing smithi10066867-38 from 589824 to 1343488 tid 2 2024-04-07T16:25:49.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:49 smithi144 bash[66654]: cephadm 2024-04-07T16:25:48.009513+0000 mgr.y (mgr.25175) 271 : cephadm [INF] Upgrade: Setting container_image for all osd 2024-04-07T16:25:49.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:49 smithi144 bash[66654]: cephadm 2024-04-07T16:25:48.068164+0000 mgr.y (mgr.25175) 272 : cephadm [INF] Upgrade: Setting require_osd_release to 19 squid 2024-04-07T16:25:49.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:49 smithi144 bash[66654]: cluster 2024-04-07T16:25:49.063089+0000 mon.a (mon.0) 907 : cluster [INF] Health check cleared: OSD_UPGRADE_FINISHED (was: all OSDs are running squid or later but require_osd_release < squid) 2024-04-07T16:25:49.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:49 smithi144 bash[66654]: cluster 2024-04-07T16:25:49.063144+0000 mon.a (mon.0) 908 : cluster [INF] Cluster is now healthy 2024-04-07T16:25:49.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:49 smithi144 bash[66654]: audit 2024-04-07T16:25:49.065396+0000 mon.a (mon.0) 909 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "osd require-osd-release", "release": "squid"}]': finished 2024-04-07T16:25:49.356 INFO:tasks.rados.rados.0.smithi100.stdout:2777: writing smithi10066867-38 from 1343488 to 1998848 tid 3 2024-04-07T16:25:49.358 INFO:tasks.rados.rados.0.smithi100.stdout:2777: writing smithi10066867-38 from 1998848 to 2195456 tid 4 2024-04-07T16:25:49.358 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 50 current snap is 261 2024-04-07T16:25:49.358 INFO:tasks.rados.rados.0.smithi100.stdout:2775: finishing write tid 4 to smithi10066867-2 2024-04-07T16:25:49.358 INFO:tasks.rados.rados.0.smithi100.stdout:2778: seq_num 848 ranges {11845632=606208,12451840=630784,13082624=458752,13541376=516096} 2024-04-07T16:25:49.427 INFO:tasks.rados.rados.0.smithi100.stdout:2778: writing smithi10066867-50 from 11845632 to 12451840 tid 1 2024-04-07T16:25:49.431 INFO:tasks.rados.rados.0.smithi100.stdout:2778: writing smithi10066867-50 from 12451840 to 13082624 tid 2 2024-04-07T16:25:49.433 INFO:tasks.rados.rados.0.smithi100.stdout:2778: writing smithi10066867-50 from 13082624 to 13541376 tid 3 2024-04-07T16:25:49.434 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:49 smithi100 bash[74942]: cephadm 2024-04-07T16:25:48.009513+0000 mgr.y (mgr.25175) 271 : cephadm [INF] Upgrade: Setting container_image for all osd 2024-04-07T16:25:49.435 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:49 smithi100 bash[74942]: cephadm 2024-04-07T16:25:48.068164+0000 mgr.y (mgr.25175) 272 : cephadm [INF] Upgrade: Setting require_osd_release to 19 squid 2024-04-07T16:25:49.435 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:49 smithi100 bash[74942]: cluster 2024-04-07T16:25:49.063089+0000 mon.a (mon.0) 907 : cluster [INF] Health check cleared: OSD_UPGRADE_FINISHED (was: all OSDs are running squid or later but require_osd_release < squid) 2024-04-07T16:25:49.435 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:49 smithi100 bash[74942]: cluster 2024-04-07T16:25:49.063144+0000 mon.a (mon.0) 908 : cluster [INF] Cluster is now healthy 2024-04-07T16:25:49.435 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:49 smithi100 bash[74942]: audit 2024-04-07T16:25:49.065396+0000 mon.a (mon.0) 909 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "osd require-osd-release", "release": "squid"}]': finished 2024-04-07T16:25:49.435 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:49 smithi100 bash[76202]: cephadm 2024-04-07T16:25:48.009513+0000 mgr.y (mgr.25175) 271 : cephadm [INF] Upgrade: Setting container_image for all osd 2024-04-07T16:25:49.435 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:49 smithi100 bash[76202]: cephadm 2024-04-07T16:25:48.068164+0000 mgr.y (mgr.25175) 272 : cephadm [INF] Upgrade: Setting require_osd_release to 19 squid 2024-04-07T16:25:49.435 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:49 smithi100 bash[76202]: cluster 2024-04-07T16:25:49.063089+0000 mon.a (mon.0) 907 : cluster [INF] Health check cleared: OSD_UPGRADE_FINISHED (was: all OSDs are running squid or later but require_osd_release < squid) 2024-04-07T16:25:49.435 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:49 smithi100 bash[76202]: cluster 2024-04-07T16:25:49.063144+0000 mon.a (mon.0) 908 : cluster [INF] Cluster is now healthy 2024-04-07T16:25:49.435 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:49 smithi100 bash[76202]: audit 2024-04-07T16:25:49.065396+0000 mon.a (mon.0) 909 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "osd require-osd-release", "release": "squid"}]': finished 2024-04-07T16:25:49.437 INFO:tasks.rados.rados.0.smithi100.stdout:2778: writing smithi10066867-50 from 13541376 to 14057472 tid 4 2024-04-07T16:25:49.437 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 9 current snap is 261 2024-04-07T16:25:49.437 INFO:tasks.rados.rados.0.smithi100.stdout:2775: finishing write tid 5 to smithi10066867-2 2024-04-07T16:25:49.437 INFO:tasks.rados.rados.0.smithi100.stdout:2779: seq_num 849 ranges {3416064=622592,4038656=516096,4554752=663552,5218304=425984} 2024-04-07T16:25:49.460 INFO:tasks.rados.rados.0.smithi100.stdout:2779: writing smithi10066867-9 from 3416064 to 4038656 tid 1 2024-04-07T16:25:49.462 INFO:tasks.rados.rados.0.smithi100.stdout:2779: writing smithi10066867-9 from 4038656 to 4554752 tid 2 2024-04-07T16:25:49.466 INFO:tasks.rados.rados.0.smithi100.stdout:2779: writing smithi10066867-9 from 4554752 to 5218304 tid 3 2024-04-07T16:25:49.469 INFO:tasks.rados.rados.0.smithi100.stdout:2779: writing smithi10066867-9 from 5218304 to 5644288 tid 4 2024-04-07T16:25:49.469 INFO:tasks.rados.rados.0.smithi100.stdout:2780: read oid 24 snap -1 2024-04-07T16:25:49.469 INFO:tasks.rados.rados.0.smithi100.stdout:2780: expect deleted 2024-04-07T16:25:49.469 INFO:tasks.rados.rados.0.smithi100.stdout:2781: snap_remove snap 255 2024-04-07T16:25:50.104 INFO:tasks.rados.rados.0.smithi100.stdout:2781: done (5 left) 2024-04-07T16:25:50.104 INFO:tasks.rados.rados.0.smithi100.stdout:2782: delete oid 26 current snap is 261 2024-04-07T16:25:50.104 INFO:tasks.rados.rados.0.smithi100.stdout:2775: finishing write tid 6 to smithi10066867-2 2024-04-07T16:25:50.104 INFO:tasks.rados.rados.0.smithi100.stdout:2775: oid 2 updating version 0 to 1248 2024-04-07T16:25:50.105 INFO:tasks.rados.rados.0.smithi100.stdout:2775: oid 2 version 1248 is already newer than 1246 2024-04-07T16:25:50.105 INFO:tasks.rados.rados.0.smithi100.stdout:2775: oid 2 version 1248 is already newer than 1244 2024-04-07T16:25:50.105 INFO:tasks.rados.rados.0.smithi100.stdout:2775: oid 2 version 1248 is already newer than 1245 2024-04-07T16:25:50.105 INFO:tasks.rados.rados.0.smithi100.stdout:2775: oid 2 version 1248 is already newer than 1247 2024-04-07T16:25:50.105 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 2 v 1248 (ObjNum 845 snap 261 seq_num 845) dirty exists 2024-04-07T16:25:50.105 INFO:tasks.rados.rados.0.smithi100.stdout:2775: left oid 2 (ObjNum 845 snap 261 seq_num 845) 2024-04-07T16:25:50.105 INFO:tasks.rados.rados.0.smithi100.stdout:2777: finishing write tid 1 to smithi10066867-38 2024-04-07T16:25:50.105 INFO:tasks.rados.rados.0.smithi100.stdout:2777: finishing write tid 2 to smithi10066867-38 2024-04-07T16:25:50.106 INFO:tasks.rados.rados.0.smithi100.stdout:2777: finishing write tid 3 to smithi10066867-38 2024-04-07T16:25:50.106 INFO:tasks.rados.rados.0.smithi100.stdout:2777: finishing write tid 4 to smithi10066867-38 2024-04-07T16:25:50.106 INFO:tasks.rados.rados.0.smithi100.stdout:2777: finishing write tid 5 to smithi10066867-38 2024-04-07T16:25:50.106 INFO:tasks.rados.rados.0.smithi100.stdout:2777: finishing write tid 6 to smithi10066867-38 2024-04-07T16:25:50.106 INFO:tasks.rados.rados.0.smithi100.stdout:2777: oid 38 updating version 0 to 1114 2024-04-07T16:25:50.106 INFO:tasks.rados.rados.0.smithi100.stdout:2777: oid 38 updating version 1114 to 1117 2024-04-07T16:25:50.106 INFO:tasks.rados.rados.0.smithi100.stdout:2777: oid 38 version 1117 is already newer than 1116 2024-04-07T16:25:50.106 INFO:tasks.rados.rados.0.smithi100.stdout:2777: oid 38 version 1117 is already newer than 1115 2024-04-07T16:25:50.106 INFO:tasks.rados.rados.0.smithi100.stdout:2777: oid 38 updating version 1117 to 1118 2024-04-07T16:25:50.106 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 38 v 1118 (ObjNum 847 snap 261 seq_num 847) dirty exists 2024-04-07T16:25:50.106 INFO:tasks.rados.rados.0.smithi100.stdout:2777: left oid 38 (ObjNum 847 snap 261 seq_num 847) 2024-04-07T16:25:50.106 INFO:tasks.rados.rados.0.smithi100.stdout:2778: finishing write tid 1 to smithi10066867-50 2024-04-07T16:25:50.106 INFO:tasks.rados.rados.0.smithi100.stdout:2778: finishing write tid 2 to smithi10066867-50 2024-04-07T16:25:50.106 INFO:tasks.rados.rados.0.smithi100.stdout:2778: finishing write tid 3 to smithi10066867-50 2024-04-07T16:25:50.106 INFO:tasks.rados.rados.0.smithi100.stdout:2778: finishing write tid 4 to smithi10066867-50 2024-04-07T16:25:50.107 INFO:tasks.rados.rados.0.smithi100.stdout:2778: finishing write tid 5 to smithi10066867-50 2024-04-07T16:25:50.107 INFO:tasks.rados.rados.0.smithi100.stdout:2778: finishing write tid 6 to smithi10066867-50 2024-04-07T16:25:50.107 INFO:tasks.rados.rados.0.smithi100.stdout:2778: oid 50 updating version 0 to 1179 2024-04-07T16:25:50.107 INFO:tasks.rados.rados.0.smithi100.stdout:2778: oid 50 version 1179 is already newer than 1176 2024-04-07T16:25:50.107 INFO:tasks.rados.rados.0.smithi100.stdout:2778: oid 50 version 1179 is already newer than 1178 2024-04-07T16:25:50.107 INFO:tasks.rados.rados.0.smithi100.stdout:2778: oid 50 version 1179 is already newer than 1177 2024-04-07T16:25:50.107 INFO:tasks.rados.rados.0.smithi100.stdout:2778: oid 50 updating version 1179 to 1180 2024-04-07T16:25:50.107 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 50 v 1180 (ObjNum 848 snap 261 seq_num 848) dirty exists 2024-04-07T16:25:50.107 INFO:tasks.rados.rados.0.smithi100.stdout:2778: left oid 50 (ObjNum 848 snap 261 seq_num 848) 2024-04-07T16:25:50.107 INFO:tasks.rados.rados.0.smithi100.stdout:2779: finishing write tid 1 to smithi10066867-9 2024-04-07T16:25:50.107 INFO:tasks.rados.rados.0.smithi100.stdout:2779: finishing write tid 2 to smithi10066867-9 2024-04-07T16:25:50.107 INFO:tasks.rados.rados.0.smithi100.stdout:2779: finishing write tid 3 to smithi10066867-9 2024-04-07T16:25:50.107 INFO:tasks.rados.rados.0.smithi100.stdout:2779: finishing write tid 4 to smithi10066867-9 2024-04-07T16:25:50.107 INFO:tasks.rados.rados.0.smithi100.stdout:2779: finishing write tid 5 to smithi10066867-9 2024-04-07T16:25:50.107 INFO:tasks.rados.rados.0.smithi100.stdout:2779: finishing write tid 6 to smithi10066867-9 2024-04-07T16:25:50.107 INFO:tasks.rados.rados.0.smithi100.stdout:2779: oid 9 updating version 0 to 1125 2024-04-07T16:25:50.108 INFO:tasks.rados.rados.0.smithi100.stdout:2779: oid 9 version 1125 is already newer than 1122 2024-04-07T16:25:50.108 INFO:tasks.rados.rados.0.smithi100.stdout:2779: oid 9 version 1125 is already newer than 1123 2024-04-07T16:25:50.108 INFO:tasks.rados.rados.0.smithi100.stdout:2779: oid 9 updating version 1125 to 1126 2024-04-07T16:25:50.108 INFO:tasks.rados.rados.0.smithi100.stdout:2779: oid 9 version 1126 is already newer than 1124 2024-04-07T16:25:50.108 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 9 v 1126 (ObjNum 849 snap 261 seq_num 849) dirty exists 2024-04-07T16:25:50.108 INFO:tasks.rados.rados.0.smithi100.stdout:2779: left oid 9 (ObjNum 849 snap 261 seq_num 849) 2024-04-07T16:25:50.121 INFO:tasks.rados.rados.0.smithi100.stdout:2775: done (5 left) 2024-04-07T16:25:50.121 INFO:tasks.rados.rados.0.smithi100.stdout:2777: done (4 left) 2024-04-07T16:25:50.121 INFO:tasks.rados.rados.0.smithi100.stdout:2778: done (3 left) 2024-04-07T16:25:50.121 INFO:tasks.rados.rados.0.smithi100.stdout:2779: done (2 left) 2024-04-07T16:25:50.121 INFO:tasks.rados.rados.0.smithi100.stdout:2780: done (1 left) 2024-04-07T16:25:50.122 INFO:tasks.rados.rados.0.smithi100.stdout:2782: done (0 left) 2024-04-07T16:25:50.122 INFO:tasks.rados.rados.0.smithi100.stdout:2783: delete oid 39 current snap is 261 2024-04-07T16:25:50.137 INFO:tasks.rados.rados.0.smithi100.stdout:2783: done (0 left) 2024-04-07T16:25:50.137 INFO:tasks.rados.rados.0.smithi100.stdout:2784: copy_from oid 18 from oid 48 current snap is 261 2024-04-07T16:25:50.138 INFO:tasks.rados.rados.0.smithi100.stdout:2785: read oid 29 snap -1 2024-04-07T16:25:50.138 INFO:tasks.rados.rados.0.smithi100.stdout:2785: expect deleted 2024-04-07T16:25:50.138 INFO:tasks.rados.rados.0.smithi100.stdout:2786: read oid 12 snap -1 2024-04-07T16:25:50.138 INFO:tasks.rados.rados.0.smithi100.stdout:2786: expect (ObjNum 829 snap 260 seq_num 829) 2024-04-07T16:25:50.138 INFO:tasks.rados.rados.0.smithi100.stdout:2787: copy_from oid 34 from oid 33 current snap is 261 2024-04-07T16:25:50.138 INFO:tasks.rados.rados.0.smithi100.stdout:2788: setattr oid 43 current snap is 261 2024-04-07T16:25:50.139 INFO:tasks.rados.rados.0.smithi100.stdout:2785: done (4 left) 2024-04-07T16:25:50.139 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 37 current snap is 261 2024-04-07T16:25:50.139 INFO:tasks.rados.rados.0.smithi100.stdout:2789: seq_num 852 ranges {3801088=671744,4472832=622592,5095424=442368,5537792=540672} 2024-04-07T16:25:50.165 INFO:tasks.rados.rados.0.smithi100.stdout:2789: writing smithi10066867-37 from 3801088 to 4472832 tid 1 2024-04-07T16:25:50.169 INFO:tasks.rados.rados.0.smithi100.stdout:2789: writing smithi10066867-37 from 4472832 to 5095424 tid 2 2024-04-07T16:25:50.172 INFO:tasks.rados.rados.0.smithi100.stdout:2789: writing smithi10066867-37 from 5095424 to 5537792 tid 3 2024-04-07T16:25:50.175 INFO:tasks.rados.rados.0.smithi100.stdout:2789: writing smithi10066867-37 from 5537792 to 6078464 tid 4 2024-04-07T16:25:50.175 INFO:tasks.rados.rados.0.smithi100.stdout:2790: snap_remove snap 246 2024-04-07T16:25:50.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:50 smithi144 bash[66654]: cluster 2024-04-07T16:25:48.740559+0000 mgr.y (mgr.25175) 273 : cluster [DBG] pgmap v480: 121 pgs: 121 active+clean; 198 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 8.5 MiB/s rd, 1.2 MiB/s wr, 15 op/s 2024-04-07T16:25:50.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:50 smithi144 bash[66654]: cluster 2024-04-07T16:25:49.071383+0000 mon.a (mon.0) 910 : cluster [DBG] osdmap e592: 8 total, 8 up, 8 in 2024-04-07T16:25:50.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:50 smithi144 bash[66654]: audit 2024-04-07T16:25:49.090118+0000 mon.a (mon.0) 911 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mds ok-to-stop", "ids": ["a"]}]: dispatch 2024-04-07T16:25:50.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:50 smithi144 bash[66654]: cephadm 2024-04-07T16:25:49.091553+0000 mgr.y (mgr.25175) 274 : cephadm [INF] Upgrade: It appears safe to stop mds.a 2024-04-07T16:25:50.435 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:50 smithi100 bash[74942]: cluster 2024-04-07T16:25:48.740559+0000 mgr.y (mgr.25175) 273 : cluster [DBG] pgmap v480: 121 pgs: 121 active+clean; 198 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 8.5 MiB/s rd, 1.2 MiB/s wr, 15 op/s 2024-04-07T16:25:50.435 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:50 smithi100 bash[74942]: cluster 2024-04-07T16:25:49.071383+0000 mon.a (mon.0) 910 : cluster [DBG] osdmap e592: 8 total, 8 up, 8 in 2024-04-07T16:25:50.435 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:50 smithi100 bash[74942]: audit 2024-04-07T16:25:49.090118+0000 mon.a (mon.0) 911 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mds ok-to-stop", "ids": ["a"]}]: dispatch 2024-04-07T16:25:50.435 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:50 smithi100 bash[74942]: cephadm 2024-04-07T16:25:49.091553+0000 mgr.y (mgr.25175) 274 : cephadm [INF] Upgrade: It appears safe to stop mds.a 2024-04-07T16:25:50.435 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:50 smithi100 bash[76202]: cluster 2024-04-07T16:25:48.740559+0000 mgr.y (mgr.25175) 273 : cluster [DBG] pgmap v480: 121 pgs: 121 active+clean; 198 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 8.5 MiB/s rd, 1.2 MiB/s wr, 15 op/s 2024-04-07T16:25:50.436 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:50 smithi100 bash[76202]: cluster 2024-04-07T16:25:49.071383+0000 mon.a (mon.0) 910 : cluster [DBG] osdmap e592: 8 total, 8 up, 8 in 2024-04-07T16:25:50.436 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:50 smithi100 bash[76202]: audit 2024-04-07T16:25:49.090118+0000 mon.a (mon.0) 911 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mds ok-to-stop", "ids": ["a"]}]: dispatch 2024-04-07T16:25:50.436 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:50 smithi100 bash[76202]: cephadm 2024-04-07T16:25:49.091553+0000 mgr.y (mgr.25175) 274 : cephadm [INF] Upgrade: It appears safe to stop mds.a 2024-04-07T16:25:51.113 INFO:tasks.rados.rados.0.smithi100.stdout:2787: finishing copy_from racing read to smithi10066867-34 2024-04-07T16:25:51.113 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 43 v 1125 (ObjNum 824 snap 258 seq_num 824) dirty exists 2024-04-07T16:25:51.113 INFO:tasks.rados.rados.0.smithi100.stdout:2784: finishing copy_from racing read to smithi10066867-18 2024-04-07T16:25:51.113 INFO:tasks.rados.rados.0.smithi100.stdout:2784: finishing copy_from to smithi10066867-18 2024-04-07T16:25:51.113 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 18 v 1116 (ObjNum 794 snap 251 seq_num 794) dirty exists 2024-04-07T16:25:51.113 INFO:tasks.rados.rados.0.smithi100.stdout:2786: expect (ObjNum 829 snap 260 seq_num 829) 2024-04-07T16:25:51.196 INFO:tasks.rados.rados.0.smithi100.stdout:2787: finishing copy_from to smithi10066867-34 2024-04-07T16:25:51.196 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 34 v 1504 (ObjNum 753 snap 235 seq_num 753) dirty exists 2024-04-07T16:25:51.196 INFO:tasks.rados.rados.0.smithi100.stdout:2789: finishing write tid 1 to smithi10066867-37 2024-04-07T16:25:51.196 INFO:tasks.rados.rados.0.smithi100.stdout:2789: finishing write tid 2 to smithi10066867-37 2024-04-07T16:25:51.196 INFO:tasks.rados.rados.0.smithi100.stdout:2789: finishing write tid 3 to smithi10066867-37 2024-04-07T16:25:51.196 INFO:tasks.rados.rados.0.smithi100.stdout:2789: finishing write tid 4 to smithi10066867-37 2024-04-07T16:25:51.196 INFO:tasks.rados.rados.0.smithi100.stdout:2789: finishing write tid 5 to smithi10066867-37 2024-04-07T16:25:51.196 INFO:tasks.rados.rados.0.smithi100.stdout:2789: finishing write tid 6 to smithi10066867-37 2024-04-07T16:25:51.196 INFO:tasks.rados.rados.0.smithi100.stdout:2789: oid 37 updating version 0 to 1209 2024-04-07T16:25:51.196 INFO:tasks.rados.rados.0.smithi100.stdout:2789: oid 37 version 1209 is already newer than 1208 2024-04-07T16:25:51.197 INFO:tasks.rados.rados.0.smithi100.stdout:2789: oid 37 updating version 1209 to 1211 2024-04-07T16:25:51.197 INFO:tasks.rados.rados.0.smithi100.stdout:2789: oid 37 version 1211 is already newer than 1207 2024-04-07T16:25:51.197 INFO:tasks.rados.rados.0.smithi100.stdout:2789: oid 37 version 1211 is already newer than 1210 2024-04-07T16:25:51.197 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 37 v 1211 (ObjNum 852 snap 261 seq_num 852) dirty exists 2024-04-07T16:25:51.197 INFO:tasks.rados.rados.0.smithi100.stdout:2789: left oid 37 (ObjNum 852 snap 261 seq_num 852) 2024-04-07T16:25:51.197 INFO:tasks.rados.rados.0.smithi100.stdout:2784: done (5 left) 2024-04-07T16:25:51.197 INFO:tasks.rados.rados.0.smithi100.stdout:2786: done (4 left) 2024-04-07T16:25:51.197 INFO:tasks.rados.rados.0.smithi100.stdout:2787: done (3 left) 2024-04-07T16:25:51.197 INFO:tasks.rados.rados.0.smithi100.stdout:2788: done (2 left) 2024-04-07T16:25:51.197 INFO:tasks.rados.rados.0.smithi100.stdout:2789: done (1 left) 2024-04-07T16:25:51.197 INFO:tasks.rados.rados.0.smithi100.stdout:2790: done (0 left) 2024-04-07T16:25:51.197 INFO:tasks.rados.rados.0.smithi100.stdout:2791: snap_remove snap 258 2024-04-07T16:25:51.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:51 smithi144 bash[66654]: cluster 2024-04-07T16:25:50.112795+0000 mon.a (mon.0) 912 : cluster [DBG] osdmap e593: 8 total, 8 up, 8 in 2024-04-07T16:25:51.435 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:51 smithi100 bash[74942]: cluster 2024-04-07T16:25:50.112795+0000 mon.a (mon.0) 912 : cluster [DBG] osdmap e593: 8 total, 8 up, 8 in 2024-04-07T16:25:51.435 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:51 smithi100 bash[76202]: cluster 2024-04-07T16:25:50.112795+0000 mon.a (mon.0) 912 : cluster [DBG] osdmap e593: 8 total, 8 up, 8 in 2024-04-07T16:25:51.875 INFO:tasks.rados.rados.0.smithi100.stdout:2791: done (0 left) 2024-04-07T16:25:51.875 INFO:tasks.rados.rados.0.smithi100.stdout:2792: delete oid 28 current snap is 261 2024-04-07T16:25:51.903 INFO:tasks.rados.rados.0.smithi100.stdout:2792: done (0 left) 2024-04-07T16:25:51.904 INFO:tasks.rados.rados.0.smithi100.stdout:2793: delete oid 19 current snap is 261 2024-04-07T16:25:51.913 INFO:tasks.rados.rados.0.smithi100.stdout:2793: done (0 left) 2024-04-07T16:25:51.913 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 18 current snap is 261 2024-04-07T16:25:51.913 INFO:tasks.rados.rados.0.smithi100.stdout:2794: seq_num 855 ranges {1302528=729088,2031616=671744,2703360=663552,3366912=270336} 2024-04-07T16:25:51.931 INFO:tasks.rados.rados.0.smithi100.stdout:2794: writing smithi10066867-18 from 1302528 to 2031616 tid 1 2024-04-07T16:25:51.938 INFO:tasks.rados.rados.0.smithi100.stdout:2794: writing smithi10066867-18 from 2031616 to 2703360 tid 2 2024-04-07T16:25:51.944 INFO:tasks.rados.rados.0.smithi100.stdout:2794: writing smithi10066867-18 from 2703360 to 3366912 tid 3 2024-04-07T16:25:51.946 INFO:tasks.rados.rados.0.smithi100.stdout:2794: writing smithi10066867-18 from 3366912 to 3637248 tid 4 2024-04-07T16:25:51.946 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 14 current snap is 261 2024-04-07T16:25:51.946 INFO:tasks.rados.rados.0.smithi100.stdout:2794: finishing write tid 1 to smithi10066867-18 2024-04-07T16:25:51.946 INFO:tasks.rados.rados.0.smithi100.stdout:2794: finishing write tid 2 to smithi10066867-18 2024-04-07T16:25:51.946 INFO:tasks.rados.rados.0.smithi100.stdout:2795: seq_num 856 ranges {0=745472,745472=557056,1302528=417792,1720320=434176,2154496=196608} 2024-04-07T16:25:51.951 INFO:tasks.rados.rados.0.smithi100.stdout:2795: writing smithi10066867-14 from 0 to 745472 tid 1 2024-04-07T16:25:51.955 INFO:tasks.rados.rados.0.smithi100.stdout:2795: writing smithi10066867-14 from 745472 to 1302528 tid 2 2024-04-07T16:25:51.957 INFO:tasks.rados.rados.0.smithi100.stdout:2795: writing smithi10066867-14 from 1302528 to 1720320 tid 3 2024-04-07T16:25:51.959 INFO:tasks.rados.rados.0.smithi100.stdout:2795: writing smithi10066867-14 from 1720320 to 2154496 tid 4 2024-04-07T16:25:51.961 INFO:tasks.rados.rados.0.smithi100.stdout:2795: writing smithi10066867-14 from 2154496 to 2351104 tid 5 2024-04-07T16:25:51.961 INFO:tasks.rados.rados.0.smithi100.stdout:2796: delete oid 35 current snap is 261 2024-04-07T16:25:51.961 INFO:tasks.rados.rados.0.smithi100.stdout:2794: finishing write tid 3 to smithi10066867-18 2024-04-07T16:25:51.961 INFO:tasks.rados.rados.0.smithi100.stdout:2794: finishing write tid 4 to smithi10066867-18 2024-04-07T16:25:51.961 INFO:tasks.rados.rados.0.smithi100.stdout:2794: finishing write tid 5 to smithi10066867-18 2024-04-07T16:25:51.961 INFO:tasks.rados.rados.0.smithi100.stdout:2794: finishing write tid 6 to smithi10066867-18 2024-04-07T16:25:51.962 INFO:tasks.rados.rados.0.smithi100.stdout:2794: oid 18 updating version 0 to 1576 2024-04-07T16:25:51.962 INFO:tasks.rados.rados.0.smithi100.stdout:2794: oid 18 updating version 1576 to 1578 2024-04-07T16:25:51.962 INFO:tasks.rados.rados.0.smithi100.stdout:2794: oid 18 version 1578 is already newer than 1574 2024-04-07T16:25:51.962 INFO:tasks.rados.rados.0.smithi100.stdout:2794: oid 18 version 1578 is already newer than 1575 2024-04-07T16:25:51.962 INFO:tasks.rados.rados.0.smithi100.stdout:2794: oid 18 version 1578 is already newer than 1577 2024-04-07T16:25:51.962 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 18 v 1578 (ObjNum 855 snap 261 seq_num 855) dirty exists 2024-04-07T16:25:51.962 INFO:tasks.rados.rados.0.smithi100.stdout:2794: left oid 18 (ObjNum 855 snap 261 seq_num 855) 2024-04-07T16:25:51.962 INFO:tasks.rados.rados.0.smithi100.stdout:2795: finishing write tid 1 to smithi10066867-14 2024-04-07T16:25:51.962 INFO:tasks.rados.rados.0.smithi100.stdout:2795: finishing write tid 2 to smithi10066867-14 2024-04-07T16:25:51.962 INFO:tasks.rados.rados.0.smithi100.stdout:2794: done (2 left) 2024-04-07T16:25:51.963 INFO:tasks.rados.rados.0.smithi100.stdout:2796: done (1 left) 2024-04-07T16:25:51.963 INFO:tasks.rados.rados.0.smithi100.stdout:2797: setattr oid 35 current snap is 261 2024-04-07T16:25:51.963 INFO:tasks.rados.rados.0.smithi100.stdout:2795: finishing write tid 3 to smithi10066867-14 2024-04-07T16:25:51.964 INFO:tasks.rados.rados.0.smithi100.stdout:2798: copy_from oid 26 from oid 1 current snap is 261 2024-04-07T16:25:51.964 INFO:tasks.rados.rados.0.smithi100.stdout:2799: copy_from oid 17 from oid 29 current snap is 261 2024-04-07T16:25:51.964 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 27 current snap is 261 2024-04-07T16:25:51.964 INFO:tasks.rados.rados.0.smithi100.stdout:2800: seq_num 858 ranges {0=778240,778240=729088,1507328=434176,1941504=442368} 2024-04-07T16:25:51.969 INFO:tasks.rados.rados.0.smithi100.stdout:2800: writing smithi10066867-27 from 0 to 778240 tid 1 2024-04-07T16:25:51.974 INFO:tasks.rados.rados.0.smithi100.stdout:2800: writing smithi10066867-27 from 778240 to 1507328 tid 2 2024-04-07T16:25:51.977 INFO:tasks.rados.rados.0.smithi100.stdout:2800: writing smithi10066867-27 from 1507328 to 1941504 tid 3 2024-04-07T16:25:51.980 INFO:tasks.rados.rados.0.smithi100.stdout:2800: writing smithi10066867-27 from 1941504 to 2383872 tid 4 2024-04-07T16:25:51.980 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 2 current snap is 261 2024-04-07T16:25:51.980 INFO:tasks.rados.rados.0.smithi100.stdout:2795: finishing write tid 4 to smithi10066867-14 2024-04-07T16:25:51.980 INFO:tasks.rados.rados.0.smithi100.stdout:2799: finishing copy_from to smithi10066867-17 2024-04-07T16:25:51.980 INFO:tasks.rados.rados.0.smithi100.stdout:2799: got expected ENOENT (src dne) 2024-04-07T16:25:51.980 INFO:tasks.rados.rados.0.smithi100.stdout:2799: finishing copy_from racing read to smithi10066867-17 2024-04-07T16:25:51.980 INFO:tasks.rados.rados.0.smithi100.stdout:2795: finishing write tid 5 to smithi10066867-14 2024-04-07T16:25:51.980 INFO:tasks.rados.rados.0.smithi100.stdout:2795: finishing write tid 6 to smithi10066867-14 2024-04-07T16:25:51.981 INFO:tasks.rados.rados.0.smithi100.stdout:2795: finishing write tid 7 to smithi10066867-14 2024-04-07T16:25:51.981 INFO:tasks.rados.rados.0.smithi100.stdout:2795: oid 14 updating version 0 to 1185 2024-04-07T16:25:51.981 INFO:tasks.rados.rados.0.smithi100.stdout:2795: oid 14 version 1185 is already newer than 1182 2024-04-07T16:25:51.981 INFO:tasks.rados.rados.0.smithi100.stdout:2795: oid 14 version 1185 is already newer than 1183 2024-04-07T16:25:51.981 INFO:tasks.rados.rados.0.smithi100.stdout:2795: oid 14 version 1185 is already newer than 1181 2024-04-07T16:25:51.981 INFO:tasks.rados.rados.0.smithi100.stdout:2795: oid 14 version 1185 is already newer than 1184 2024-04-07T16:25:51.981 INFO:tasks.rados.rados.0.smithi100.stdout:2795: oid 14 updating version 1185 to 1186 2024-04-07T16:25:51.981 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 14 v 1186 (ObjNum 856 snap 261 seq_num 856) dirty exists 2024-04-07T16:25:51.981 INFO:tasks.rados.rados.0.smithi100.stdout:2795: left oid 14 (ObjNum 856 snap 261 seq_num 856) 2024-04-07T16:25:51.981 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 35 v 1117 (ObjNum 129 snap 0 seq_num 2704727776) dirty exists 2024-04-07T16:25:51.981 INFO:tasks.rados.rados.0.smithi100.stdout:2800: finishing write tid 1 to smithi10066867-27 2024-04-07T16:25:51.981 INFO:tasks.rados.rados.0.smithi100.stdout:2800: finishing write tid 2 to smithi10066867-27 2024-04-07T16:25:51.981 INFO:tasks.rados.rados.0.smithi100.stdout:2801: seq_num 859 ranges {3604480=794624,4399104=614400,5013504=638976,5652480=352256} 2024-04-07T16:25:52.005 INFO:tasks.rados.rados.0.smithi100.stdout:2801: writing smithi10066867-2 from 3604480 to 4399104 tid 1 2024-04-07T16:25:52.008 INFO:tasks.rados.rados.0.smithi100.stdout:2801: writing smithi10066867-2 from 4399104 to 5013504 tid 2 2024-04-07T16:25:52.012 INFO:tasks.rados.rados.0.smithi100.stdout:2801: writing smithi10066867-2 from 5013504 to 5652480 tid 3 2024-04-07T16:25:52.016 INFO:tasks.rados.rados.0.smithi100.stdout:2801: writing smithi10066867-2 from 5652480 to 6004736 tid 4 2024-04-07T16:25:52.016 INFO:tasks.rados.rados.0.smithi100.stdout:2795: done (5 left) 2024-04-07T16:25:52.016 INFO:tasks.rados.rados.0.smithi100.stdout:2797: done (4 left) 2024-04-07T16:25:52.016 INFO:tasks.rados.rados.0.smithi100.stdout:2799: done (3 left) 2024-04-07T16:25:52.016 INFO:tasks.rados.rados.0.smithi100.stdout:2802: read oid 45 snap -1 2024-04-07T16:25:52.016 INFO:tasks.rados.rados.0.smithi100.stdout:2802: expect (ObjNum 0 snap 0 seq_num 0) 2024-04-07T16:25:52.016 INFO:tasks.rados.rados.0.smithi100.stdout:2803: rollback oid 11 current snap is 261 2024-04-07T16:25:52.016 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 11 to 253 2024-04-07T16:25:52.016 INFO:tasks.rados.rados.0.smithi100.stdout:2800: finishing write tid 3 to smithi10066867-27 2024-04-07T16:25:52.016 INFO:tasks.rados.rados.0.smithi100.stdout:2798: finishing copy_from racing read to smithi10066867-26 2024-04-07T16:25:52.016 INFO:tasks.rados.rados.0.smithi100.stdout:2798: finishing copy_from to smithi10066867-26 2024-04-07T16:25:52.017 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 26 v 1121 (ObjNum 826 snap 258 seq_num 826) dirty exists 2024-04-07T16:25:52.017 INFO:tasks.rados.rados.0.smithi100.stdout:2800: finishing write tid 4 to smithi10066867-27 2024-04-07T16:25:52.017 INFO:tasks.rados.rados.0.smithi100.stdout:2800: finishing write tid 5 to smithi10066867-27 2024-04-07T16:25:52.017 INFO:tasks.rados.rados.0.smithi100.stdout:2800: finishing write tid 6 to smithi10066867-27 2024-04-07T16:25:52.017 INFO:tasks.rados.rados.0.smithi100.stdout:2800: oid 27 updating version 0 to 1128 2024-04-07T16:25:52.017 INFO:tasks.rados.rados.0.smithi100.stdout:2800: oid 27 updating version 1128 to 1130 2024-04-07T16:25:52.017 INFO:tasks.rados.rados.0.smithi100.stdout:2800: oid 27 version 1130 is already newer than 1126 2024-04-07T16:25:52.017 INFO:tasks.rados.rados.0.smithi100.stdout:2800: oid 27 version 1130 is already newer than 1127 2024-04-07T16:25:52.017 INFO:tasks.rados.rados.0.smithi100.stdout:2800: oid 27 version 1130 is already newer than 1129 2024-04-07T16:25:52.017 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 27 v 1130 (ObjNum 858 snap 261 seq_num 858) dirty exists 2024-04-07T16:25:52.017 INFO:tasks.rados.rados.0.smithi100.stdout:2800: left oid 27 (ObjNum 858 snap 261 seq_num 858) 2024-04-07T16:25:52.017 INFO:tasks.rados.rados.0.smithi100.stdout:2801: finishing write tid 1 to smithi10066867-2 2024-04-07T16:25:52.017 INFO:tasks.rados.rados.0.smithi100.stdout:2798: done (4 left) 2024-04-07T16:25:52.017 INFO:tasks.rados.rados.0.smithi100.stdout:2800: done (3 left) 2024-04-07T16:25:52.017 INFO:tasks.rados.rados.0.smithi100.stdout:2804: read oid 15 snap -1 2024-04-07T16:25:52.018 INFO:tasks.rados.rados.0.smithi100.stdout:2804: expect (ObjNum 776 snap 243 seq_num 776) 2024-04-07T16:25:52.018 INFO:tasks.rados.rados.0.smithi100.stdout:2805: read oid 7 snap -1 2024-04-07T16:25:52.018 INFO:tasks.rados.rados.0.smithi100.stdout:2805: expect deleted 2024-04-07T16:25:52.018 INFO:tasks.rados.rados.0.smithi100.stdout:2806: read oid 26 snap -1 2024-04-07T16:25:52.018 INFO:tasks.rados.rados.0.smithi100.stdout:2806: expect (ObjNum 826 snap 258 seq_num 826) 2024-04-07T16:25:52.018 INFO:tasks.rados.rados.0.smithi100.stdout:2807: delete oid 21 current snap is 261 2024-04-07T16:25:52.018 INFO:tasks.rados.rados.0.smithi100.stdout:2801: finishing write tid 2 to smithi10066867-2 2024-04-07T16:25:52.018 INFO:tasks.rados.rados.0.smithi100.stdout:2802: expect (ObjNum 0 snap 0 seq_num 0) 2024-04-07T16:25:52.021 INFO:tasks.rados.rados.0.smithi100.stdout:2803: finishing rollback tid 0 to smithi10066867-11 2024-04-07T16:25:52.022 INFO:tasks.rados.rados.0.smithi100.stdout:2802: done (6 left) 2024-04-07T16:25:52.022 INFO:tasks.rados.rados.0.smithi100.stdout:2805: done (5 left) 2024-04-07T16:25:52.022 INFO:tasks.rados.rados.0.smithi100.stdout:2807: done (4 left) 2024-04-07T16:25:52.022 INFO:tasks.rados.rados.0.smithi100.stdout:2808: copy_from oid 28 from oid 45 current snap is 261 2024-04-07T16:25:52.022 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 18 current snap is 261 2024-04-07T16:25:52.022 INFO:tasks.rados.rados.0.smithi100.stdout:2809: seq_num 861 ranges {3637248=24576} 2024-04-07T16:25:52.043 INFO:tasks.rados.rados.0.smithi100.stdout:2809: writing smithi10066867-18 from 3637248 to 3661824 tid 1 2024-04-07T16:25:52.043 INFO:tasks.rados.rados.0.smithi100.stdout:2810: delete oid 42 current snap is 261 2024-04-07T16:25:52.043 INFO:tasks.rados.rados.0.smithi100.stdout:2801: finishing write tid 3 to smithi10066867-2 2024-04-07T16:25:52.043 INFO:tasks.rados.rados.0.smithi100.stdout:2801: finishing write tid 4 to smithi10066867-2 2024-04-07T16:25:52.043 INFO:tasks.rados.rados.0.smithi100.stdout:2801: finishing write tid 5 to smithi10066867-2 2024-04-07T16:25:52.043 INFO:tasks.rados.rados.0.smithi100.stdout:2803: finishing rollback tid 1 to smithi10066867-11 2024-04-07T16:25:52.043 INFO:tasks.rados.rados.0.smithi100.stdout:2808: finishing copy_from racing read to smithi10066867-28 2024-04-07T16:25:52.043 INFO:tasks.rados.rados.0.smithi100.stdout:2801: finishing write tid 6 to smithi10066867-2 2024-04-07T16:25:52.044 INFO:tasks.rados.rados.0.smithi100.stdout:2801: oid 2 updating version 0 to 1267 2024-04-07T16:25:52.044 INFO:tasks.rados.rados.0.smithi100.stdout:2801: oid 2 updating version 1267 to 1268 2024-04-07T16:25:52.044 INFO:tasks.rados.rados.0.smithi100.stdout:2801: oid 2 updating version 1268 to 1269 2024-04-07T16:25:52.044 INFO:tasks.rados.rados.0.smithi100.stdout:2801: oid 2 updating version 1269 to 1271 2024-04-07T16:25:52.044 INFO:tasks.rados.rados.0.smithi100.stdout:2801: oid 2 version 1271 is already newer than 1270 2024-04-07T16:25:52.044 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 2 v 1271 (ObjNum 859 snap 261 seq_num 859) dirty exists 2024-04-07T16:25:52.044 INFO:tasks.rados.rados.0.smithi100.stdout:2801: left oid 2 (ObjNum 859 snap 261 seq_num 859) 2024-04-07T16:25:52.044 INFO:tasks.rados.rados.0.smithi100.stdout:2808: finishing copy_from to smithi10066867-28 2024-04-07T16:25:52.044 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 28 v 1038 (ObjNum 1345 snap 0 seq_num 3083533072) dirty exists 2024-04-07T16:25:52.044 INFO:tasks.rados.rados.0.smithi100.stdout:2806: expect (ObjNum 826 snap 258 seq_num 826) 2024-04-07T16:25:52.169 INFO:tasks.rados.rados.0.smithi100.stdout:2804: expect (ObjNum 776 snap 243 seq_num 776) 2024-04-07T16:25:52.356 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:25:52 smithi144 bash[59794]: ts=2024-04-07T16:25:52.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:25:52.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:52 smithi144 bash[66654]: cluster 2024-04-07T16:25:50.741094+0000 mgr.y (mgr.25175) 275 : cluster [DBG] pgmap v483: 121 pgs: 121 active+clean; 198 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 13 MiB/s rd, 834 KiB/s wr, 15 op/s 2024-04-07T16:25:52.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:52 smithi144 bash[66654]: cluster 2024-04-07T16:25:51.121842+0000 mon.a (mon.0) 913 : cluster [DBG] osdmap e594: 8 total, 8 up, 8 in 2024-04-07T16:25:52.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:52 smithi144 bash[66654]: audit 2024-04-07T16:25:51.521701+0000 mon.a (mon.0) 914 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:25:52.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:52 smithi144 bash[66654]: audit 2024-04-07T16:25:51.525969+0000 mon.a (mon.0) 915 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-07T16:25:52.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:52 smithi144 bash[66654]: audit 2024-04-07T16:25:51.527377+0000 mon.a (mon.0) 916 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:25:52.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:52 smithi144 bash[66654]: cluster 2024-04-07T16:25:51.892087+0000 mon.a (mon.0) 917 : cluster [DBG] osdmap e595: 8 total, 8 up, 8 in 2024-04-07T16:25:52.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:52 smithi144 bash[66654]: audit 2024-04-07T16:25:51.982200+0000 mon.a (mon.0) 918 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:25:52.357 INFO:tasks.rados.rados.0.smithi100.stdout:2803: finishing rollback tid 2 to smithi10066867-11 2024-04-07T16:25:52.358 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 11 v 1581 (ObjNum 796 snap 251 seq_num 796) dirty exists 2024-04-07T16:25:52.358 INFO:tasks.rados.rados.0.smithi100.stdout:2809: finishing write tid 1 to smithi10066867-18 2024-04-07T16:25:52.358 INFO:tasks.rados.rados.0.smithi100.stdout:2809: finishing write tid 2 to smithi10066867-18 2024-04-07T16:25:52.358 INFO:tasks.rados.rados.0.smithi100.stdout:2809: finishing write tid 3 to smithi10066867-18 2024-04-07T16:25:52.358 INFO:tasks.rados.rados.0.smithi100.stdout:2809: oid 18 updating version 0 to 1583 2024-04-07T16:25:52.358 INFO:tasks.rados.rados.0.smithi100.stdout:2809: oid 18 version 1583 is already newer than 1582 2024-04-07T16:25:52.358 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 18 v 1583 (ObjNum 861 snap 261 seq_num 861) dirty exists 2024-04-07T16:25:52.358 INFO:tasks.rados.rados.0.smithi100.stdout:2809: left oid 18 (ObjNum 861 snap 261 seq_num 861) 2024-04-07T16:25:52.358 INFO:tasks.rados.rados.0.smithi100.stdout:2801: done (6 left) 2024-04-07T16:25:52.358 INFO:tasks.rados.rados.0.smithi100.stdout:2803: done (5 left) 2024-04-07T16:25:52.358 INFO:tasks.rados.rados.0.smithi100.stdout:2804: done (4 left) 2024-04-07T16:25:52.359 INFO:tasks.rados.rados.0.smithi100.stdout:2806: done (3 left) 2024-04-07T16:25:52.359 INFO:tasks.rados.rados.0.smithi100.stdout:2808: done (2 left) 2024-04-07T16:25:52.359 INFO:tasks.rados.rados.0.smithi100.stdout:2809: done (1 left) 2024-04-07T16:25:52.359 INFO:tasks.rados.rados.0.smithi100.stdout:2810: done (0 left) 2024-04-07T16:25:52.359 INFO:tasks.rados.rados.0.smithi100.stdout:2811: read oid 33 snap 248 2024-04-07T16:25:52.359 INFO:tasks.rados.rados.0.smithi100.stdout:2811: expect (ObjNum 753 snap 235 seq_num 753) 2024-04-07T16:25:52.359 INFO:tasks.rados.rados.0.smithi100.stdout:2812: rmattr oid 11 current snap is 261 2024-04-07T16:25:52.359 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 27 current snap is 261 2024-04-07T16:25:52.359 INFO:tasks.rados.rados.0.smithi100.stdout:2813: seq_num 863 ranges {2383872=57344} 2024-04-07T16:25:52.374 INFO:tasks.rados.rados.0.smithi100.stdout:2813: writing smithi10066867-27 from 2383872 to 2441216 tid 1 2024-04-07T16:25:52.374 INFO:tasks.rados.rados.0.smithi100.stdout:2814: rollback oid 8 current snap is 261 2024-04-07T16:25:52.374 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 8 to 252 2024-04-07T16:25:52.374 INFO:tasks.rados.rados.0.smithi100.stdout:2815: setattr oid 41 current snap is 261 2024-04-07T16:25:52.374 INFO:tasks.rados.rados.0.smithi100.stdout:2811: expect (ObjNum 753 snap 235 seq_num 753) 2024-04-07T16:25:52.421 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 11 v 1584 (ObjNum 796 snap 251 seq_num 796) dirty exists 2024-04-07T16:25:52.422 INFO:tasks.rados.rados.0.smithi100.stdout:2813: finishing write tid 1 to smithi10066867-27 2024-04-07T16:25:52.422 INFO:tasks.rados.rados.0.smithi100.stdout:2813: finishing write tid 2 to smithi10066867-27 2024-04-07T16:25:52.422 INFO:tasks.rados.rados.0.smithi100.stdout:2813: finishing write tid 3 to smithi10066867-27 2024-04-07T16:25:52.422 INFO:tasks.rados.rados.0.smithi100.stdout:2813: oid 27 updating version 0 to 1131 2024-04-07T16:25:52.422 INFO:tasks.rados.rados.0.smithi100.stdout:2813: oid 27 updating version 1131 to 1132 2024-04-07T16:25:52.422 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 27 v 1132 (ObjNum 863 snap 261 seq_num 863) dirty exists 2024-04-07T16:25:52.422 INFO:tasks.rados.rados.0.smithi100.stdout:2813: left oid 27 (ObjNum 863 snap 261 seq_num 863) 2024-04-07T16:25:52.422 INFO:tasks.rados.rados.0.smithi100.stdout:2814: finishing rollback tid 0 to smithi10066867-8 2024-04-07T16:25:52.422 INFO:tasks.rados.rados.0.smithi100.stdout:2814: finishing rollback tid 1 to smithi10066867-8 2024-04-07T16:25:52.422 INFO:tasks.rados.rados.0.smithi100.stdout:2814: finishing rollback tid 2 to smithi10066867-8 2024-04-07T16:25:52.422 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 8 v 1507 (ObjNum 749 snap 230 seq_num 749) dirty exists 2024-04-07T16:25:52.422 INFO:tasks.rados.rados.0.smithi100.stdout:2811: done (4 left) 2024-04-07T16:25:52.423 INFO:tasks.rados.rados.0.smithi100.stdout:2812: done (3 left) 2024-04-07T16:25:52.423 INFO:tasks.rados.rados.0.smithi100.stdout:2813: done (2 left) 2024-04-07T16:25:52.423 INFO:tasks.rados.rados.0.smithi100.stdout:2814: done (1 left) 2024-04-07T16:25:52.423 INFO:tasks.rados.rados.0.smithi100.stdout:2816: rollback oid 3 current snap is 261 2024-04-07T16:25:52.423 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 3 to 252 2024-04-07T16:25:52.423 INFO:tasks.rados.rados.0.smithi100.stdout:2817: snap_create 2024-04-07T16:25:52.427 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 41 v 1369 (ObjNum 758 snap 239 seq_num 758) dirty exists 2024-04-07T16:25:52.429 INFO:tasks.rados.rados.0.smithi100.stdout:2816: finishing rollback tid 0 to smithi10066867-3 2024-04-07T16:25:52.430 INFO:tasks.rados.rados.0.smithi100.stdout:2816: finishing rollback tid 1 to smithi10066867-3 2024-04-07T16:25:52.430 INFO:tasks.rados.rados.0.smithi100.stdout:2816: finishing rollback tid 2 to smithi10066867-3 2024-04-07T16:25:52.430 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 3 v 1121 (ObjNum 733 snap 226 seq_num 733) dirty exists 2024-04-07T16:25:52.435 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:52 smithi100 bash[74942]: cluster 2024-04-07T16:25:50.741094+0000 mgr.y (mgr.25175) 275 : cluster [DBG] pgmap v483: 121 pgs: 121 active+clean; 198 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 13 MiB/s rd, 834 KiB/s wr, 15 op/s 2024-04-07T16:25:52.435 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:52 smithi100 bash[74942]: cluster 2024-04-07T16:25:51.121842+0000 mon.a (mon.0) 913 : cluster [DBG] osdmap e594: 8 total, 8 up, 8 in 2024-04-07T16:25:52.435 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:52 smithi100 bash[74942]: audit 2024-04-07T16:25:51.521701+0000 mon.a (mon.0) 914 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:25:52.435 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:52 smithi100 bash[74942]: audit 2024-04-07T16:25:51.525969+0000 mon.a (mon.0) 915 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-07T16:25:52.435 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:52 smithi100 bash[74942]: audit 2024-04-07T16:25:51.527377+0000 mon.a (mon.0) 916 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:25:52.435 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:52 smithi100 bash[74942]: cluster 2024-04-07T16:25:51.892087+0000 mon.a (mon.0) 917 : cluster [DBG] osdmap e595: 8 total, 8 up, 8 in 2024-04-07T16:25:52.435 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:52 smithi100 bash[74942]: audit 2024-04-07T16:25:51.982200+0000 mon.a (mon.0) 918 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:25:52.436 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:52 smithi100 bash[76202]: cluster 2024-04-07T16:25:50.741094+0000 mgr.y (mgr.25175) 275 : cluster [DBG] pgmap v483: 121 pgs: 121 active+clean; 198 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 13 MiB/s rd, 834 KiB/s wr, 15 op/s 2024-04-07T16:25:52.436 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:52 smithi100 bash[76202]: cluster 2024-04-07T16:25:51.121842+0000 mon.a (mon.0) 913 : cluster [DBG] osdmap e594: 8 total, 8 up, 8 in 2024-04-07T16:25:52.436 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:52 smithi100 bash[76202]: audit 2024-04-07T16:25:51.521701+0000 mon.a (mon.0) 914 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:25:52.436 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:52 smithi100 bash[76202]: audit 2024-04-07T16:25:51.525969+0000 mon.a (mon.0) 915 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-07T16:25:52.436 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:52 smithi100 bash[76202]: audit 2024-04-07T16:25:51.527377+0000 mon.a (mon.0) 916 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:25:52.436 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:52 smithi100 bash[76202]: cluster 2024-04-07T16:25:51.892087+0000 mon.a (mon.0) 917 : cluster [DBG] osdmap e595: 8 total, 8 up, 8 in 2024-04-07T16:25:52.436 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:52 smithi100 bash[76202]: audit 2024-04-07T16:25:51.982200+0000 mon.a (mon.0) 918 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:25:53.104 INFO:tasks.rados.rados.0.smithi100.stdout:2815: done (2 left) 2024-04-07T16:25:53.104 INFO:tasks.rados.rados.0.smithi100.stdout:2816: done (1 left) 2024-04-07T16:25:53.104 INFO:tasks.rados.rados.0.smithi100.stdout:2817: done (0 left) 2024-04-07T16:25:53.104 INFO:tasks.rados.rados.0.smithi100.stdout:2818: rollback oid 14 current snap is 262 2024-04-07T16:25:53.104 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 14 to 260 2024-04-07T16:25:53.104 INFO:tasks.rados.rados.0.smithi100.stdout:2819: snap_create 2024-04-07T16:25:53.121 INFO:tasks.rados.rados.0.smithi100.stdout:2818: finishing rollback tid 0 to smithi10066867-14 2024-04-07T16:25:53.123 INFO:tasks.rados.rados.0.smithi100.stdout:2818: finishing rollback tid 1 to smithi10066867-14 2024-04-07T16:25:53.124 INFO:tasks.rados.rados.0.smithi100.stdout:2818: finishing rollback tid 2 to smithi10066867-14 2024-04-07T16:25:53.124 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 14 v 1189 (ObjNum 762 snap 240 seq_num 762) dirty exists 2024-04-07T16:25:53.435 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:53 smithi100 bash[74942]: cephadm 2024-04-07T16:25:51.514543+0000 mgr.y (mgr.25175) 276 : cephadm [INF] Upgrade: Updating mds.a 2024-04-07T16:25:53.435 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:53 smithi100 bash[74942]: cephadm 2024-04-07T16:25:51.529002+0000 mgr.y (mgr.25175) 277 : cephadm [INF] Deploying daemon mds.a on smithi100 2024-04-07T16:25:53.435 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:53 smithi100 bash[76202]: cephadm 2024-04-07T16:25:51.514543+0000 mgr.y (mgr.25175) 276 : cephadm [INF] Upgrade: Updating mds.a 2024-04-07T16:25:53.435 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:53 smithi100 bash[76202]: cephadm 2024-04-07T16:25:51.529002+0000 mgr.y (mgr.25175) 277 : cephadm [INF] Deploying daemon mds.a on smithi100 2024-04-07T16:25:53.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:53 smithi144 bash[66654]: cephadm 2024-04-07T16:25:51.514543+0000 mgr.y (mgr.25175) 276 : cephadm [INF] Upgrade: Updating mds.a 2024-04-07T16:25:53.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:53 smithi144 bash[66654]: cephadm 2024-04-07T16:25:51.529002+0000 mgr.y (mgr.25175) 277 : cephadm [INF] Deploying daemon mds.a on smithi100 2024-04-07T16:25:53.935 INFO:journalctl@ceph.osd.3.smithi100.stdout:Apr 07 16:25:53 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:25:53.935 INFO:journalctl@ceph.osd.0.smithi100.stdout:Apr 07 16:25:53 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:25:53.936 INFO:journalctl@ceph.osd.2.smithi100.stdout:Apr 07 16:25:53 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:25:53.936 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:53 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:25:53.936 INFO:journalctl@ceph.osd.1.smithi100.stdout:Apr 07 16:25:53 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:25:53.937 INFO:journalctl@ceph.mds.a.smithi100.stdout:Apr 07 16:25:53 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:25:53.937 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:25:53 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:25:53.937 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:53 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:25:53.937 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 16:25:53 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:25:53.938 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Apr 07 16:25:53 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:25:54.119 INFO:tasks.rados.rados.0.smithi100.stdout:2818: done (1 left) 2024-04-07T16:25:54.119 INFO:tasks.rados.rados.0.smithi100.stdout:2819: done (0 left) 2024-04-07T16:25:54.119 INFO:tasks.rados.rados.0.smithi100.stdout:2820: read oid 49 snap -1 2024-04-07T16:25:54.119 INFO:tasks.rados.rados.0.smithi100.stdout:2820: expect (ObjNum 783 snap 246 seq_num 783) 2024-04-07T16:25:54.119 INFO:tasks.rados.rados.0.smithi100.stdout:2821: read oid 17 snap -1 2024-04-07T16:25:54.119 INFO:tasks.rados.rados.0.smithi100.stdout:2821: expect (ObjNum 737 snap 227 seq_num 737) 2024-04-07T16:25:54.120 INFO:tasks.rados.rados.0.smithi100.stdout:2822: rollback oid 10 current snap is 263 2024-04-07T16:25:54.120 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 10 to 262 2024-04-07T16:25:54.120 INFO:tasks.rados.rados.0.smithi100.stdout:2823: copy_from oid 32 from oid 4 current snap is 263 2024-04-07T16:25:54.120 INFO:tasks.rados.rados.0.smithi100.stdout:2824: copy_from oid 9 from oid 40 current snap is 263 2024-04-07T16:25:54.120 INFO:tasks.rados.rados.0.smithi100.stdout:2825: read oid 18 snap 253 2024-04-07T16:25:54.121 INFO:tasks.rados.rados.0.smithi100.stdout:2825: expect (ObjNum 799 snap 251 seq_num 799) 2024-04-07T16:25:54.121 INFO:tasks.rados.rados.0.smithi100.stdout:2826: rmattr oid 19 current snap is 263 2024-04-07T16:25:54.121 INFO:tasks.rados.rados.0.smithi100.stdout:2826: done (6 left) 2024-04-07T16:25:54.121 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 41 current snap is 263 2024-04-07T16:25:54.121 INFO:tasks.rados.rados.0.smithi100.stdout:2827: seq_num 864 ranges {2572288=73728} 2024-04-07T16:25:54.157 INFO:tasks.rados.rados.0.smithi100.stdout:2827: writing smithi10066867-41 from 2572288 to 2646016 tid 1 2024-04-07T16:25:54.157 INFO:tasks.rados.rados.0.smithi100.stdout:2828: copy_from oid 29 from oid 24 current snap is 263 2024-04-07T16:25:54.157 INFO:tasks.rados.rados.0.smithi100.stdout:2822: finishing rollback tid 0 to smithi10066867-10 2024-04-07T16:25:54.157 INFO:tasks.rados.rados.0.smithi100.stdout:2822: finishing rollback tid 1 to smithi10066867-10 2024-04-07T16:25:54.157 INFO:tasks.rados.rados.0.smithi100.stdout:2822: finishing rollback tid 2 to smithi10066867-10 2024-04-07T16:25:54.157 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 10 v 1436 (ObjNum 776 snap 243 seq_num 776) dirty exists 2024-04-07T16:25:54.157 INFO:tasks.rados.rados.0.smithi100.stdout:2824: finishing copy_from to smithi10066867-9 2024-04-07T16:25:54.157 INFO:tasks.rados.rados.0.smithi100.stdout:2824: got expected ENOENT (src dne) 2024-04-07T16:25:54.157 INFO:tasks.rados.rados.0.smithi100.stdout:2824: finishing copy_from racing read to smithi10066867-9 2024-04-07T16:25:54.157 INFO:tasks.rados.rados.0.smithi100.stdout:2821: expect (ObjNum 737 snap 227 seq_num 737) 2024-04-07T16:25:54.276 INFO:tasks.rados.rados.0.smithi100.stdout:2825: expect (ObjNum 799 snap 251 seq_num 799) 2024-04-07T16:25:54.360 INFO:tasks.rados.rados.0.smithi100.stdout:2828: finishing copy_from to smithi10066867-29 2024-04-07T16:25:54.360 INFO:tasks.rados.rados.0.smithi100.stdout:2828: got expected ENOENT (src dne) 2024-04-07T16:25:54.360 INFO:tasks.rados.rados.0.smithi100.stdout:2828: finishing copy_from racing read to smithi10066867-29 2024-04-07T16:25:54.360 INFO:tasks.rados.rados.0.smithi100.stdout:2820: expect (ObjNum 783 snap 246 seq_num 783) 2024-04-07T16:25:54.435 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:54 smithi100 bash[74942]: cluster 2024-04-07T16:25:52.742193+0000 mgr.y (mgr.25175) 278 : cluster [DBG] pgmap v486: 121 pgs: 1 active+clean+snaptrim, 120 active+clean; 206 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 4.8 MiB/s rd, 2.9 MiB/s wr, 9 op/s 2024-04-07T16:25:54.435 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:54 smithi100 bash[74942]: cluster 2024-04-07T16:25:53.119430+0000 mon.a (mon.0) 919 : cluster [DBG] osdmap e596: 8 total, 8 up, 8 in 2024-04-07T16:25:54.436 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:54 smithi100 bash[76202]: cluster 2024-04-07T16:25:52.742193+0000 mgr.y (mgr.25175) 278 : cluster [DBG] pgmap v486: 121 pgs: 1 active+clean+snaptrim, 120 active+clean; 206 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 4.8 MiB/s rd, 2.9 MiB/s wr, 9 op/s 2024-04-07T16:25:54.436 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:54 smithi100 bash[76202]: cluster 2024-04-07T16:25:53.119430+0000 mon.a (mon.0) 919 : cluster [DBG] osdmap e596: 8 total, 8 up, 8 in 2024-04-07T16:25:54.436 INFO:journalctl@ceph.mds.a.smithi100.stdout:Apr 07 16:25:53 smithi100 systemd[1]: Stopping Ceph mds.a for 0702e3a8-f4f8-11ee-b649-cb9ed24678a4... 2024-04-07T16:25:54.436 INFO:journalctl@ceph.mds.a.smithi100.stdout:Apr 07 16:25:54 smithi100 bash[56138]: debug 2024-04-07T16:25:54.066+0000 7fb0210f8700 -1 received signal: Terminated from /sbin/docker-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug (PID: 1) UID: 0 2024-04-07T16:25:54.436 INFO:journalctl@ceph.mds.a.smithi100.stdout:Apr 07 16:25:54 smithi100 bash[56138]: debug 2024-04-07T16:25:54.066+0000 7fb0210f8700 -1 mds.a *** got signal Terminated *** 2024-04-07T16:25:54.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:54 smithi144 bash[66654]: cluster 2024-04-07T16:25:52.742193+0000 mgr.y (mgr.25175) 278 : cluster [DBG] pgmap v486: 121 pgs: 1 active+clean+snaptrim, 120 active+clean; 206 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 4.8 MiB/s rd, 2.9 MiB/s wr, 9 op/s 2024-04-07T16:25:54.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:54 smithi144 bash[66654]: cluster 2024-04-07T16:25:53.119430+0000 mon.a (mon.0) 919 : cluster [DBG] osdmap e596: 8 total, 8 up, 8 in 2024-04-07T16:25:54.711 INFO:tasks.rados.rados.0.smithi100.stdout:2827: finishing write tid 1 to smithi10066867-41 2024-04-07T16:25:54.711 INFO:tasks.rados.rados.0.smithi100.stdout:2827: finishing write tid 2 to smithi10066867-41 2024-04-07T16:25:54.711 INFO:tasks.rados.rados.0.smithi100.stdout:2827: finishing write tid 3 to smithi10066867-41 2024-04-07T16:25:54.711 INFO:tasks.rados.rados.0.smithi100.stdout:2827: oid 41 updating version 0 to 1371 2024-04-07T16:25:54.711 INFO:tasks.rados.rados.0.smithi100.stdout:2827: oid 41 updating version 1371 to 1372 2024-04-07T16:25:54.711 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 41 v 1372 (ObjNum 864 snap 263 seq_num 864) dirty exists 2024-04-07T16:25:54.711 INFO:tasks.rados.rados.0.smithi100.stdout:2827: left oid 41 (ObjNum 864 snap 263 seq_num 864) 2024-04-07T16:25:54.711 INFO:tasks.rados.rados.0.smithi100.stdout:2823: finishing copy_from racing read to smithi10066867-32 2024-04-07T16:25:54.711 INFO:tasks.rados.rados.0.smithi100.stdout:2823: finishing copy_from to smithi10066867-32 2024-04-07T16:25:54.712 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 32 v 1521 (ObjNum 807 snap 253 seq_num 807) dirty exists 2024-04-07T16:25:54.712 INFO:tasks.rados.rados.0.smithi100.stdout:2820: done (7 left) 2024-04-07T16:25:54.712 INFO:tasks.rados.rados.0.smithi100.stdout:2821: done (6 left) 2024-04-07T16:25:54.712 INFO:tasks.rados.rados.0.smithi100.stdout:2822: done (5 left) 2024-04-07T16:25:54.712 INFO:tasks.rados.rados.0.smithi100.stdout:2823: done (4 left) 2024-04-07T16:25:54.712 INFO:tasks.rados.rados.0.smithi100.stdout:2824: done (3 left) 2024-04-07T16:25:54.712 INFO:tasks.rados.rados.0.smithi100.stdout:2825: done (2 left) 2024-04-07T16:25:54.712 INFO:tasks.rados.rados.0.smithi100.stdout:2827: done (1 left) 2024-04-07T16:25:54.712 INFO:tasks.rados.rados.0.smithi100.stdout:2828: done (0 left) 2024-04-07T16:25:54.712 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 3 current snap is 263 2024-04-07T16:25:54.712 INFO:tasks.rados.rados.0.smithi100.stdout:2829: seq_num 865 ranges {2482176=90112} 2024-04-07T16:25:54.753 INFO:tasks.rados.rados.0.smithi100.stdout:2829: writing smithi10066867-3 from 2482176 to 2572288 tid 1 2024-04-07T16:25:54.753 INFO:tasks.rados.rados.0.smithi100.stdout:2830: snap_remove snap 261 2024-04-07T16:25:55.143 INFO:tasks.rados.rados.0.smithi100.stdout:2829: finishing write tid 1 to smithi10066867-3 2024-04-07T16:25:55.143 INFO:tasks.rados.rados.0.smithi100.stdout:2829: finishing write tid 2 to smithi10066867-3 2024-04-07T16:25:55.143 INFO:tasks.rados.rados.0.smithi100.stdout:2829: finishing write tid 3 to smithi10066867-3 2024-04-07T16:25:55.143 INFO:tasks.rados.rados.0.smithi100.stdout:2829: oid 3 updating version 0 to 1122 2024-04-07T16:25:55.143 INFO:tasks.rados.rados.0.smithi100.stdout:2829: oid 3 updating version 1122 to 1123 2024-04-07T16:25:55.143 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 3 v 1123 (ObjNum 865 snap 263 seq_num 865) dirty exists 2024-04-07T16:25:55.143 INFO:tasks.rados.rados.0.smithi100.stdout:2829: left oid 3 (ObjNum 865 snap 263 seq_num 865) 2024-04-07T16:25:55.143 INFO:tasks.rados.rados.0.smithi100.stdout:2829: done (1 left) 2024-04-07T16:25:55.143 INFO:tasks.rados.rados.0.smithi100.stdout:2830: done (0 left) 2024-04-07T16:25:55.143 INFO:tasks.rados.rados.0.smithi100.stdout:2831: delete oid 39 current snap is 263 2024-04-07T16:25:55.153 INFO:tasks.rados.rados.0.smithi100.stdout:2831: done (0 left) 2024-04-07T16:25:55.153 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 43 current snap is 263 2024-04-07T16:25:55.153 INFO:tasks.rados.rados.0.smithi100.stdout:2832: seq_num 867 ranges {10403840=122880} 2024-04-07T16:25:55.212 INFO:tasks.rados.rados.0.smithi100.stdout:2832: writing smithi10066867-43 from 10403840 to 10526720 tid 1 2024-04-07T16:25:55.212 INFO:tasks.rados.rados.0.smithi100.stdout:2833: snap_remove snap 260 2024-04-07T16:25:55.435 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:55 smithi100 bash[74942]: cluster 2024-04-07T16:25:54.135594+0000 mon.a (mon.0) 920 : cluster [DBG] osdmap e597: 8 total, 8 up, 8 in 2024-04-07T16:25:55.435 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:55 smithi100 bash[74942]: cluster 2024-04-07T16:25:54.135907+0000 mon.a (mon.0) 921 : cluster [DBG] fsmap cephfs:1 {0=b=up:active} 2024-04-07T16:25:55.435 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:55 smithi100 bash[76202]: cluster 2024-04-07T16:25:54.135594+0000 mon.a (mon.0) 920 : cluster [DBG] osdmap e597: 8 total, 8 up, 8 in 2024-04-07T16:25:55.435 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:55 smithi100 bash[76202]: cluster 2024-04-07T16:25:54.135907+0000 mon.a (mon.0) 921 : cluster [DBG] fsmap cephfs:1 {0=b=up:active} 2024-04-07T16:25:55.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:55 smithi144 bash[66654]: cluster 2024-04-07T16:25:54.135594+0000 mon.a (mon.0) 920 : cluster [DBG] osdmap e597: 8 total, 8 up, 8 in 2024-04-07T16:25:55.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:55 smithi144 bash[66654]: cluster 2024-04-07T16:25:54.135907+0000 mon.a (mon.0) 921 : cluster [DBG] fsmap cephfs:1 {0=b=up:active} 2024-04-07T16:25:56.154 INFO:tasks.rados.rados.0.smithi100.stdout:2832: finishing write tid 1 to smithi10066867-43 2024-04-07T16:25:56.154 INFO:tasks.rados.rados.0.smithi100.stdout:2832: finishing write tid 2 to smithi10066867-43 2024-04-07T16:25:56.154 INFO:tasks.rados.rados.0.smithi100.stdout:2832: finishing write tid 3 to smithi10066867-43 2024-04-07T16:25:56.154 INFO:tasks.rados.rados.0.smithi100.stdout:2832: oid 43 updating version 0 to 1134 2024-04-07T16:25:56.154 INFO:tasks.rados.rados.0.smithi100.stdout:2832: oid 43 version 1134 is already newer than 1133 2024-04-07T16:25:56.155 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 43 v 1134 (ObjNum 867 snap 263 seq_num 867) dirty exists 2024-04-07T16:25:56.155 INFO:tasks.rados.rados.0.smithi100.stdout:2832: left oid 43 (ObjNum 867 snap 263 seq_num 867) 2024-04-07T16:25:56.155 INFO:tasks.rados.rados.0.smithi100.stdout:2832: done (1 left) 2024-04-07T16:25:56.155 INFO:tasks.rados.rados.0.smithi100.stdout:2833: done (0 left) 2024-04-07T16:25:56.155 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 26 current snap is 263 2024-04-07T16:25:56.155 INFO:tasks.rados.rados.0.smithi100.stdout:2834: seq_num 868 ranges {3416064=139264} 2024-04-07T16:25:56.177 INFO:tasks.rados.rados.0.smithi100.stdout:2834: writing smithi10066867-26 from 3416064 to 3555328 tid 1 2024-04-07T16:25:56.177 INFO:tasks.rados.rados.0.smithi100.stdout:2835: rmattr oid 36 current snap is 263 2024-04-07T16:25:56.177 INFO:tasks.rados.rados.0.smithi100.stdout:2835: done (1 left) 2024-04-07T16:25:56.177 INFO:tasks.rados.rados.0.smithi100.stdout:2836: snap_remove snap 252 2024-04-07T16:25:56.435 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:56 smithi100 bash[76202]: cluster 2024-04-07T16:25:54.742963+0000 mgr.y (mgr.25175) 279 : cluster [DBG] pgmap v489: 121 pgs: 1 active+clean+snaptrim, 120 active+clean; 210 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 9.5 MiB/s rd, 7.4 MiB/s wr, 39 op/s 2024-04-07T16:25:56.435 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:56 smithi100 bash[76202]: cluster 2024-04-07T16:25:55.156573+0000 mon.a (mon.0) 922 : cluster [DBG] osdmap e598: 8 total, 8 up, 8 in 2024-04-07T16:25:56.435 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:56 smithi100 bash[74942]: cluster 2024-04-07T16:25:54.742963+0000 mgr.y (mgr.25175) 279 : cluster [DBG] pgmap v489: 121 pgs: 1 active+clean+snaptrim, 120 active+clean; 210 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 9.5 MiB/s rd, 7.4 MiB/s wr, 39 op/s 2024-04-07T16:25:56.436 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:56 smithi100 bash[74942]: cluster 2024-04-07T16:25:55.156573+0000 mon.a (mon.0) 922 : cluster [DBG] osdmap e598: 8 total, 8 up, 8 in 2024-04-07T16:25:56.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:56 smithi144 bash[66654]: cluster 2024-04-07T16:25:54.742963+0000 mgr.y (mgr.25175) 279 : cluster [DBG] pgmap v489: 121 pgs: 1 active+clean+snaptrim, 120 active+clean; 210 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 9.5 MiB/s rd, 7.4 MiB/s wr, 39 op/s 2024-04-07T16:25:56.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:56 smithi144 bash[66654]: cluster 2024-04-07T16:25:55.156573+0000 mon.a (mon.0) 922 : cluster [DBG] osdmap e598: 8 total, 8 up, 8 in 2024-04-07T16:25:56.879 INFO:tasks.rados.rados.0.smithi100.stdout:2834: finishing write tid 1 to smithi10066867-26 2024-04-07T16:25:56.880 INFO:tasks.rados.rados.0.smithi100.stdout:2834: finishing write tid 2 to smithi10066867-26 2024-04-07T16:25:56.880 INFO:tasks.rados.rados.0.smithi100.stdout:2834: finishing write tid 3 to smithi10066867-26 2024-04-07T16:25:56.880 INFO:tasks.rados.rados.0.smithi100.stdout:2834: oid 26 updating version 0 to 1198 2024-04-07T16:25:56.880 INFO:tasks.rados.rados.0.smithi100.stdout:2834: oid 26 version 1198 is already newer than 1197 2024-04-07T16:25:56.880 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 26 v 1198 (ObjNum 868 snap 263 seq_num 868) dirty exists 2024-04-07T16:25:56.880 INFO:tasks.rados.rados.0.smithi100.stdout:2834: left oid 26 (ObjNum 868 snap 263 seq_num 868) 2024-04-07T16:25:56.880 INFO:tasks.rados.rados.0.smithi100.stdout:2834: done (1 left) 2024-04-07T16:25:56.880 INFO:tasks.rados.rados.0.smithi100.stdout:2836: done (0 left) 2024-04-07T16:25:56.880 INFO:tasks.rados.rados.0.smithi100.stdout:2837: snap_remove snap 248 2024-04-07T16:25:57.421 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:57 smithi100 bash[74942]: cluster 2024-04-07T16:25:56.170205+0000 mon.a (mon.0) 923 : cluster [DBG] osdmap e599: 8 total, 8 up, 8 in 2024-04-07T16:25:57.421 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:57 smithi100 bash[74942]: cluster 2024-04-07T16:25:56.895308+0000 mon.a (mon.0) 924 : cluster [DBG] osdmap e600: 8 total, 8 up, 8 in 2024-04-07T16:25:57.421 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:57 smithi100 bash[76202]: cluster 2024-04-07T16:25:56.170205+0000 mon.a (mon.0) 923 : cluster [DBG] osdmap e599: 8 total, 8 up, 8 in 2024-04-07T16:25:57.421 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:57 smithi100 bash[76202]: cluster 2024-04-07T16:25:56.895308+0000 mon.a (mon.0) 924 : cluster [DBG] osdmap e600: 8 total, 8 up, 8 in 2024-04-07T16:25:57.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:57 smithi144 bash[66654]: cluster 2024-04-07T16:25:56.170205+0000 mon.a (mon.0) 923 : cluster [DBG] osdmap e599: 8 total, 8 up, 8 in 2024-04-07T16:25:57.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:57 smithi144 bash[66654]: cluster 2024-04-07T16:25:56.895308+0000 mon.a (mon.0) 924 : cluster [DBG] osdmap e600: 8 total, 8 up, 8 in 2024-04-07T16:25:57.886 INFO:tasks.rados.rados.0.smithi100.stdout:2837: done (0 left) 2024-04-07T16:25:57.886 INFO:tasks.rados.rados.0.smithi100.stdout:2838: rmattr oid 22 current snap is 263 2024-04-07T16:25:57.886 INFO:tasks.rados.rados.0.smithi100.stdout:2839: read oid 37 snap -1 2024-04-07T16:25:57.886 INFO:tasks.rados.rados.0.smithi100.stdout:2839: expect (ObjNum 852 snap 261 seq_num 852) 2024-04-07T16:25:57.886 INFO:tasks.rados.rados.0.smithi100.stdout:2840: read oid 45 snap -1 2024-04-07T16:25:57.886 INFO:tasks.rados.rados.0.smithi100.stdout:2840: expect (ObjNum 0 snap 0 seq_num 0) 2024-04-07T16:25:57.886 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 10 current snap is 263 2024-04-07T16:25:57.886 INFO:tasks.rados.rados.0.smithi100.stdout:2841: seq_num 869 ranges {3334144=155648} 2024-04-07T16:25:57.911 INFO:tasks.rados.rados.0.smithi100.stdout:2841: writing smithi10066867-10 from 3334144 to 3489792 tid 1 2024-04-07T16:25:57.912 INFO:tasks.rados.rados.0.smithi100.stdout:2842: delete oid 43 current snap is 263 2024-04-07T16:25:57.912 INFO:tasks.rados.rados.0.smithi100.stdout:2840: expect (ObjNum 0 snap 0 seq_num 0) 2024-04-07T16:25:57.921 INFO:tasks.rados.rados.0.smithi100.stdout:2841: finishing write tid 1 to smithi10066867-10 2024-04-07T16:25:57.921 INFO:tasks.rados.rados.0.smithi100.stdout:2841: finishing write tid 2 to smithi10066867-10 2024-04-07T16:25:57.925 INFO:tasks.rados.rados.0.smithi100.stdout:2841: finishing write tid 3 to smithi10066867-10 2024-04-07T16:25:57.925 INFO:tasks.rados.rados.0.smithi100.stdout:2841: oid 10 updating version 0 to 1438 2024-04-07T16:25:57.925 INFO:tasks.rados.rados.0.smithi100.stdout:2841: oid 10 version 1438 is already newer than 1437 2024-04-07T16:25:57.925 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 10 v 1438 (ObjNum 869 snap 263 seq_num 869) dirty exists 2024-04-07T16:25:57.925 INFO:tasks.rados.rados.0.smithi100.stdout:2841: left oid 10 (ObjNum 869 snap 263 seq_num 869) 2024-04-07T16:25:57.934 INFO:tasks.rados.rados.0.smithi100.stdout:2840: done (4 left) 2024-04-07T16:25:57.935 INFO:tasks.rados.rados.0.smithi100.stdout:2841: done (3 left) 2024-04-07T16:25:57.935 INFO:tasks.rados.rados.0.smithi100.stdout:2842: done (2 left) 2024-04-07T16:25:57.935 INFO:tasks.rados.rados.0.smithi100.stdout:2843: setattr oid 15 current snap is 263 2024-04-07T16:25:57.937 INFO:tasks.rados.rados.0.smithi100.stdout:2844: rmattr oid 43 current snap is 263 2024-04-07T16:25:57.937 INFO:tasks.rados.rados.0.smithi100.stdout:2844: done (3 left) 2024-04-07T16:25:57.937 INFO:tasks.rados.rados.0.smithi100.stdout:2845: rmattr oid 1 current snap is 263 2024-04-07T16:25:57.937 INFO:tasks.rados.rados.0.smithi100.stdout:2845: done (3 left) 2024-04-07T16:25:57.937 INFO:tasks.rados.rados.0.smithi100.stdout:2846: read oid 2 snap 262 2024-04-07T16:25:57.937 INFO:tasks.rados.rados.0.smithi100.stdout:2846: expect (ObjNum 859 snap 261 seq_num 859) 2024-04-07T16:25:57.937 INFO:tasks.rados.rados.0.smithi100.stdout:2847: read oid 46 snap -1 2024-04-07T16:25:57.937 INFO:tasks.rados.rados.0.smithi100.stdout:2847: expect (ObjNum 798 snap 251 seq_num 798) 2024-04-07T16:25:57.937 INFO:tasks.rados.rados.0.smithi100.stdout:2848: delete oid 4 current snap is 263 2024-04-07T16:25:57.944 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 22 v 1199 (ObjNum 842 snap 261 seq_num 842) dirty exists 2024-04-07T16:25:57.952 INFO:tasks.rados.rados.0.smithi100.stdout:2839: expect (ObjNum 852 snap 261 seq_num 852) 2024-04-07T16:25:58.224 INFO:tasks.rados.rados.0.smithi100.stdout:2847: expect (ObjNum 798 snap 251 seq_num 798) 2024-04-07T16:25:58.263 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 15 v 1604 (ObjNum 776 snap 243 seq_num 776) dirty exists 2024-04-07T16:25:58.264 INFO:tasks.rados.rados.0.smithi100.stdout:2846: expect (ObjNum 859 snap 261 seq_num 859) 2024-04-07T16:25:58.415 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:58 smithi100 bash[74942]: cluster 2024-04-07T16:25:56.743718+0000 mgr.y (mgr.25175) 280 : cluster [DBG] pgmap v492: 121 pgs: 1 active+clean+snaptrim, 120 active+clean; 212 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 9.5 MiB/s rd, 5.1 MiB/s wr, 38 op/s 2024-04-07T16:25:58.416 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:58 smithi100 bash[74942]: cluster 2024-04-07T16:25:57.898242+0000 mon.a (mon.0) 925 : cluster [DBG] osdmap e601: 8 total, 8 up, 8 in 2024-04-07T16:25:58.416 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:58 smithi100 bash[76202]: cluster 2024-04-07T16:25:56.743718+0000 mgr.y (mgr.25175) 280 : cluster [DBG] pgmap v492: 121 pgs: 1 active+clean+snaptrim, 120 active+clean; 212 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 9.5 MiB/s rd, 5.1 MiB/s wr, 38 op/s 2024-04-07T16:25:58.416 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:58 smithi100 bash[76202]: cluster 2024-04-07T16:25:57.898242+0000 mon.a (mon.0) 925 : cluster [DBG] osdmap e601: 8 total, 8 up, 8 in 2024-04-07T16:25:58.416 INFO:journalctl@ceph.mds.a.smithi100.stdout:Apr 07 16:25:58 smithi100 bash[98167]: ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4-mds-a 2024-04-07T16:25:58.416 INFO:journalctl@ceph.mds.a.smithi100.stdout:Apr 07 16:25:58 smithi100 bash[98229]: Error response from daemon: No such container: ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4-mds-a 2024-04-07T16:25:58.416 INFO:journalctl@ceph.mds.a.smithi100.stdout:Apr 07 16:25:58 smithi100 systemd[1]: ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@mds.a.service: Deactivated successfully. 2024-04-07T16:25:58.416 INFO:journalctl@ceph.mds.a.smithi100.stdout:Apr 07 16:25:58 smithi100 systemd[1]: Stopped Ceph mds.a for 0702e3a8-f4f8-11ee-b649-cb9ed24678a4. 2024-04-07T16:25:58.504 INFO:tasks.rados.rados.0.smithi100.stdout:2838: done (5 left) 2024-04-07T16:25:58.504 INFO:tasks.rados.rados.0.smithi100.stdout:2839: done (4 left) 2024-04-07T16:25:58.504 INFO:tasks.rados.rados.0.smithi100.stdout:2843: done (3 left) 2024-04-07T16:25:58.504 INFO:tasks.rados.rados.0.smithi100.stdout:2846: done (2 left) 2024-04-07T16:25:58.504 INFO:tasks.rados.rados.0.smithi100.stdout:2847: done (1 left) 2024-04-07T16:25:58.504 INFO:tasks.rados.rados.0.smithi100.stdout:2848: done (0 left) 2024-04-07T16:25:58.504 INFO:tasks.rados.rados.0.smithi100.stdout:2849: snap_create 2024-04-07T16:25:58.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:58 smithi144 bash[66654]: cluster 2024-04-07T16:25:56.743718+0000 mgr.y (mgr.25175) 280 : cluster [DBG] pgmap v492: 121 pgs: 1 active+clean+snaptrim, 120 active+clean; 212 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 9.5 MiB/s rd, 5.1 MiB/s wr, 38 op/s 2024-04-07T16:25:58.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:58 smithi144 bash[66654]: cluster 2024-04-07T16:25:57.898242+0000 mon.a (mon.0) 925 : cluster [DBG] osdmap e601: 8 total, 8 up, 8 in 2024-04-07T16:25:58.685 INFO:journalctl@ceph.osd.0.smithi100.stdout:Apr 07 16:25:58 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:25:58.685 INFO:journalctl@ceph.osd.1.smithi100.stdout:Apr 07 16:25:58 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:25:58.686 INFO:journalctl@ceph.osd.2.smithi100.stdout:Apr 07 16:25:58 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:25:58.686 INFO:journalctl@ceph.osd.3.smithi100.stdout:Apr 07 16:25:58 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:25:58.686 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 16:25:58 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:25:58.687 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:25:58 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:25:58] "GET /metrics HTTP/1.1" 200 36353 "" "Prometheus/2.43.0" 2024-04-07T16:25:58.687 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:25:58 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:25:58.687 INFO:journalctl@ceph.mds.a.smithi100.stdout:Apr 07 16:25:58 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:25:58.687 INFO:journalctl@ceph.mds.a.smithi100.stdout:Apr 07 16:25:58 smithi100 systemd[1]: Started Ceph mds.a for 0702e3a8-f4f8-11ee-b649-cb9ed24678a4. 2024-04-07T16:25:58.687 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Apr 07 16:25:58 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:25:58.688 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:58 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:25:58.688 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:58 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:25:59.159 INFO:tasks.rados.rados.0.smithi100.stdout:2849: done (0 left) 2024-04-07T16:25:59.160 INFO:tasks.rados.rados.0.smithi100.stdout:2850: snap_remove snap 263 2024-04-07T16:25:59.935 INFO:journalctl@ceph.mds.a.smithi100.stdout:Apr 07 16:25:59 smithi100 bash[98288]: starting mds.a at 2024-04-07T16:25:59.936 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:59 smithi100 bash[74942]: audit 2024-04-07T16:25:58.687271+0000 mon.a (mon.0) 926 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:25:59.936 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:59 smithi100 bash[74942]: audit 2024-04-07T16:25:58.693145+0000 mon.a (mon.0) 927 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:25:59.936 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:59 smithi100 bash[74942]: audit 2024-04-07T16:25:58.708819+0000 mon.a (mon.0) 928 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:25:59.936 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:59 smithi100 bash[74942]: cluster 2024-04-07T16:25:58.744330+0000 mgr.y (mgr.25175) 281 : cluster [DBG] pgmap v495: 121 pgs: 1 active+clean+snaptrim, 120 active+clean; 210 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 8.6 MiB/s rd, 2.0 MiB/s wr, 16 op/s 2024-04-07T16:25:59.936 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:25:59 smithi100 bash[74942]: cluster 2024-04-07T16:25:59.176729+0000 mon.a (mon.0) 929 : cluster [DBG] osdmap e602: 8 total, 8 up, 8 in 2024-04-07T16:25:59.936 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:59 smithi100 bash[76202]: audit 2024-04-07T16:25:58.687271+0000 mon.a (mon.0) 926 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:25:59.936 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:59 smithi100 bash[76202]: audit 2024-04-07T16:25:58.693145+0000 mon.a (mon.0) 927 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:25:59.936 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:59 smithi100 bash[76202]: audit 2024-04-07T16:25:58.708819+0000 mon.a (mon.0) 928 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:25:59.936 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:59 smithi100 bash[76202]: cluster 2024-04-07T16:25:58.744330+0000 mgr.y (mgr.25175) 281 : cluster [DBG] pgmap v495: 121 pgs: 1 active+clean+snaptrim, 120 active+clean; 210 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 8.6 MiB/s rd, 2.0 MiB/s wr, 16 op/s 2024-04-07T16:25:59.936 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:25:59 smithi100 bash[76202]: cluster 2024-04-07T16:25:59.176729+0000 mon.a (mon.0) 929 : cluster [DBG] osdmap e602: 8 total, 8 up, 8 in 2024-04-07T16:26:00.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:59 smithi144 bash[66654]: audit 2024-04-07T16:25:58.687271+0000 mon.a (mon.0) 926 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:26:00.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:59 smithi144 bash[66654]: audit 2024-04-07T16:25:58.693145+0000 mon.a (mon.0) 927 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:26:00.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:59 smithi144 bash[66654]: audit 2024-04-07T16:25:58.708819+0000 mon.a (mon.0) 928 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:26:00.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:59 smithi144 bash[66654]: cluster 2024-04-07T16:25:58.744330+0000 mgr.y (mgr.25175) 281 : cluster [DBG] pgmap v495: 121 pgs: 1 active+clean+snaptrim, 120 active+clean; 210 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 8.6 MiB/s rd, 2.0 MiB/s wr, 16 op/s 2024-04-07T16:26:00.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:25:59 smithi144 bash[66654]: cluster 2024-04-07T16:25:59.176729+0000 mon.a (mon.0) 929 : cluster [DBG] osdmap e602: 8 total, 8 up, 8 in 2024-04-07T16:26:00.163 INFO:tasks.rados.rados.0.smithi100.stdout:2850: done (0 left) 2024-04-07T16:26:00.163 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 31 current snap is 264 2024-04-07T16:26:00.163 INFO:tasks.rados.rados.0.smithi100.stdout:2851: seq_num 872 ranges {3883008=212992} 2024-04-07T16:26:00.215 INFO:tasks.rados.rados.0.smithi100.stdout:2851: writing smithi10066867-31 from 3883008 to 4096000 tid 1 2024-04-07T16:26:00.215 INFO:tasks.rados.rados.0.smithi100.stdout:2852: delete oid 49 current snap is 264 2024-04-07T16:26:00.228 INFO:tasks.rados.rados.0.smithi100.stdout:2851: finishing write tid 1 to smithi10066867-31 2024-04-07T16:26:00.229 INFO:tasks.rados.rados.0.smithi100.stdout:2852: done (1 left) 2024-04-07T16:26:00.229 INFO:tasks.rados.rados.0.smithi100.stdout:2853: delete oid 34 current snap is 264 2024-04-07T16:26:00.229 INFO:tasks.rados.rados.0.smithi100.stdout:2851: finishing write tid 2 to smithi10066867-31 2024-04-07T16:26:00.232 INFO:tasks.rados.rados.0.smithi100.stdout:2851: finishing write tid 3 to smithi10066867-31 2024-04-07T16:26:00.232 INFO:tasks.rados.rados.0.smithi100.stdout:2851: oid 31 updating version 0 to 1118 2024-04-07T16:26:00.232 INFO:tasks.rados.rados.0.smithi100.stdout:2851: oid 31 updating version 1118 to 1119 2024-04-07T16:26:00.232 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 31 v 1119 (ObjNum 872 snap 264 seq_num 872) dirty exists 2024-04-07T16:26:00.232 INFO:tasks.rados.rados.0.smithi100.stdout:2851: left oid 31 (ObjNum 872 snap 264 seq_num 872) 2024-04-07T16:26:00.245 INFO:tasks.rados.rados.0.smithi100.stdout:2851: done (1 left) 2024-04-07T16:26:00.245 INFO:tasks.rados.rados.0.smithi100.stdout:2853: done (0 left) 2024-04-07T16:26:00.245 INFO:tasks.rados.rados.0.smithi100.stdout:2854: snap_remove snap 253 2024-04-07T16:26:01.172 INFO:tasks.rados.rados.0.smithi100.stdout:2854: done (0 left) 2024-04-07T16:26:01.173 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 45 current snap is 264 2024-04-07T16:26:01.173 INFO:tasks.rados.rados.0.smithi100.stdout:2855: seq_num 875 ranges {0=262144} 2024-04-07T16:26:01.175 INFO:tasks.rados.rados.0.smithi100.stdout:2855: writing smithi10066867-45 from 0 to 262144 tid 1 2024-04-07T16:26:01.175 INFO:tasks.rados.rados.0.smithi100.stdout:2856: read oid 30 snap -1 2024-04-07T16:26:01.175 INFO:tasks.rados.rados.0.smithi100.stdout:2856: expect (ObjNum 0 snap 0 seq_num 0) 2024-04-07T16:26:01.176 INFO:tasks.rados.rados.0.smithi100.stdout:2857: rmattr oid 8 current snap is 264 2024-04-07T16:26:01.176 INFO:tasks.rados.rados.0.smithi100.stdout:2857: done (2 left) 2024-04-07T16:26:01.176 INFO:tasks.rados.rados.0.smithi100.stdout:2858: copy_from oid 46 from oid 35 current snap is 264 2024-04-07T16:26:01.176 INFO:tasks.rados.rados.0.smithi100.stdout:2859: rmattr oid 44 current snap is 264 2024-04-07T16:26:01.176 INFO:tasks.rados.rados.0.smithi100.stdout:2859: done (3 left) 2024-04-07T16:26:01.176 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 38 current snap is 264 2024-04-07T16:26:01.176 INFO:tasks.rados.rados.0.smithi100.stdout:2860: seq_num 876 ranges {2195456=278528} 2024-04-07T16:26:01.196 INFO:tasks.rados.rados.0.smithi100.stdout:2860: writing smithi10066867-38 from 2195456 to 2473984 tid 1 2024-04-07T16:26:01.196 INFO:tasks.rados.rados.0.smithi100.stdout:2861: delete oid 21 current snap is 264 2024-04-07T16:26:01.196 INFO:tasks.rados.rados.0.smithi100.stdout:2856: expect (ObjNum 0 snap 0 seq_num 0) 2024-04-07T16:26:01.202 INFO:tasks.rados.rados.0.smithi100.stdout:2855: finishing write tid 1 to smithi10066867-45 2024-04-07T16:26:01.205 INFO:tasks.rados.rados.0.smithi100.stdout:2855: finishing write tid 2 to smithi10066867-45 2024-04-07T16:26:01.207 INFO:tasks.rados.rados.0.smithi100.stdout:2858: finishing copy_from racing read to smithi10066867-46 2024-04-07T16:26:01.212 INFO:tasks.rados.rados.0.smithi100.stdout:2858: finishing copy_from to smithi10066867-46 2024-04-07T16:26:01.212 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 46 v 1117 (ObjNum 33 snap 0 seq_num 4239542215) dirty exists 2024-04-07T16:26:01.213 INFO:tasks.rados.rados.0.smithi100.stdout:2855: finishing write tid 3 to smithi10066867-45 2024-04-07T16:26:01.213 INFO:tasks.rados.rados.0.smithi100.stdout:2855: oid 45 updating version 0 to 1190 2024-04-07T16:26:01.213 INFO:tasks.rados.rados.0.smithi100.stdout:2855: oid 45 updating version 1190 to 1191 2024-04-07T16:26:01.213 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 45 v 1191 (ObjNum 875 snap 264 seq_num 875) dirty exists 2024-04-07T16:26:01.213 INFO:tasks.rados.rados.0.smithi100.stdout:2855: left oid 45 (ObjNum 875 snap 264 seq_num 875) 2024-04-07T16:26:01.216 INFO:tasks.rados.rados.0.smithi100.stdout:2860: finishing write tid 1 to smithi10066867-38 2024-04-07T16:26:01.217 INFO:tasks.rados.rados.0.smithi100.stdout:2860: finishing write tid 2 to smithi10066867-38 2024-04-07T16:26:01.217 INFO:tasks.rados.rados.0.smithi100.stdout:2855: done (4 left) 2024-04-07T16:26:01.217 INFO:tasks.rados.rados.0.smithi100.stdout:2856: done (3 left) 2024-04-07T16:26:01.219 INFO:tasks.rados.rados.0.smithi100.stdout:2858: done (2 left) 2024-04-07T16:26:01.219 INFO:tasks.rados.rados.0.smithi100.stdout:2861: done (1 left) 2024-04-07T16:26:01.219 INFO:tasks.rados.rados.0.smithi100.stdout:2862: snap_remove snap 262 2024-04-07T16:26:01.435 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:01 smithi100 bash[74942]: cluster 2024-04-07T16:26:00.179000+0000 mon.a (mon.0) 930 : cluster [DBG] osdmap e603: 8 total, 8 up, 8 in 2024-04-07T16:26:01.435 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:01 smithi100 bash[74942]: cluster 2024-04-07T16:26:00.702906+0000 mon.a (mon.0) 931 : cluster [DBG] mds.? [v2:172.21.15.100:6834/2107704360,v1:172.21.15.100:6835/2107704360] up:boot 2024-04-07T16:26:01.436 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:01 smithi100 bash[74942]: cluster 2024-04-07T16:26:00.703111+0000 mon.a (mon.0) 932 : cluster [DBG] fsmap cephfs:1 {0=b=up:active} 1 up:standby 2024-04-07T16:26:01.436 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:01 smithi100 bash[74942]: audit 2024-04-07T16:26:00.703333+0000 mon.a (mon.0) 933 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-07T16:26:01.436 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:01 smithi100 bash[76202]: cluster 2024-04-07T16:26:00.179000+0000 mon.a (mon.0) 930 : cluster [DBG] osdmap e603: 8 total, 8 up, 8 in 2024-04-07T16:26:01.436 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:01 smithi100 bash[76202]: cluster 2024-04-07T16:26:00.702906+0000 mon.a (mon.0) 931 : cluster [DBG] mds.? [v2:172.21.15.100:6834/2107704360,v1:172.21.15.100:6835/2107704360] up:boot 2024-04-07T16:26:01.436 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:01 smithi100 bash[76202]: cluster 2024-04-07T16:26:00.703111+0000 mon.a (mon.0) 932 : cluster [DBG] fsmap cephfs:1 {0=b=up:active} 1 up:standby 2024-04-07T16:26:01.436 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:01 smithi100 bash[76202]: audit 2024-04-07T16:26:00.703333+0000 mon.a (mon.0) 933 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-07T16:26:01.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:01 smithi144 bash[66654]: cluster 2024-04-07T16:26:00.179000+0000 mon.a (mon.0) 930 : cluster [DBG] osdmap e603: 8 total, 8 up, 8 in 2024-04-07T16:26:01.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:01 smithi144 bash[66654]: cluster 2024-04-07T16:26:00.702906+0000 mon.a (mon.0) 931 : cluster [DBG] mds.? [v2:172.21.15.100:6834/2107704360,v1:172.21.15.100:6835/2107704360] up:boot 2024-04-07T16:26:01.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:01 smithi144 bash[66654]: cluster 2024-04-07T16:26:00.703111+0000 mon.a (mon.0) 932 : cluster [DBG] fsmap cephfs:1 {0=b=up:active} 1 up:standby 2024-04-07T16:26:01.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:01 smithi144 bash[66654]: audit 2024-04-07T16:26:00.703333+0000 mon.a (mon.0) 933 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-07T16:26:01.882 INFO:tasks.rados.rados.0.smithi100.stdout:2862: done (1 left) 2024-04-07T16:26:01.882 INFO:tasks.rados.rados.0.smithi100.stdout:2863: read oid 4 snap -1 2024-04-07T16:26:01.882 INFO:tasks.rados.rados.0.smithi100.stdout:2863: expect deleted 2024-04-07T16:26:01.889 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 19 current snap is 264 2024-04-07T16:26:01.889 INFO:tasks.rados.rados.0.smithi100.stdout:2864: seq_num 878 ranges {0=311296} 2024-04-07T16:26:01.889 INFO:tasks.rados.rados.0.smithi100.stdout:2864: writing smithi10066867-19 from 0 to 311296 tid 1 2024-04-07T16:26:01.890 INFO:tasks.rados.rados.0.smithi100.stdout:2865: rollback oid 25 current snap is 264 2024-04-07T16:26:01.890 INFO:tasks.rados.rados.0.smithi100.stdout:2865: done (3 left) 2024-04-07T16:26:01.890 INFO:tasks.rados.rados.0.smithi100.stdout:2866: copy_from oid 49 from oid 17 current snap is 264 2024-04-07T16:26:01.890 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 11 current snap is 264 2024-04-07T16:26:01.890 INFO:tasks.rados.rados.0.smithi100.stdout:2867: seq_num 879 ranges {2449408=327680} 2024-04-07T16:26:01.918 INFO:tasks.rados.rados.0.smithi100.stdout:2867: writing smithi10066867-11 from 2449408 to 2777088 tid 1 2024-04-07T16:26:01.919 INFO:tasks.rados.rados.0.smithi100.stdout:2860: finishing write tid 3 to smithi10066867-38 2024-04-07T16:26:01.919 INFO:tasks.rados.rados.0.smithi100.stdout:2860: oid 38 updating version 0 to 1124 2024-04-07T16:26:01.919 INFO:tasks.rados.rados.0.smithi100.stdout:2860: oid 38 updating version 1124 to 1125 2024-04-07T16:26:01.919 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 38 v 1125 (ObjNum 876 snap 264 seq_num 876) dirty exists 2024-04-07T16:26:01.919 INFO:tasks.rados.rados.0.smithi100.stdout:2860: left oid 38 (ObjNum 876 snap 264 seq_num 876) 2024-04-07T16:26:01.919 INFO:tasks.rados.rados.0.smithi100.stdout:2864: finishing write tid 1 to smithi10066867-19 2024-04-07T16:26:01.919 INFO:tasks.rados.rados.0.smithi100.stdout:2864: finishing write tid 2 to smithi10066867-19 2024-04-07T16:26:01.919 INFO:tasks.rados.rados.0.smithi100.stdout:2860: done (4 left) 2024-04-07T16:26:01.919 INFO:tasks.rados.rados.0.smithi100.stdout:2863: done (3 left) 2024-04-07T16:26:01.919 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 30 current snap is 264 2024-04-07T16:26:01.919 INFO:tasks.rados.rados.0.smithi100.stdout:2868: seq_num 880 ranges {0=344064} 2024-04-07T16:26:01.922 INFO:tasks.rados.rados.0.smithi100.stdout:2868: writing smithi10066867-30 from 0 to 344064 tid 1 2024-04-07T16:26:01.922 INFO:tasks.rados.rados.0.smithi100.stdout:2869: setattr oid 34 current snap is 264 2024-04-07T16:26:01.924 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 44 current snap is 264 2024-04-07T16:26:01.924 INFO:tasks.rados.rados.0.smithi100.stdout:2870: seq_num 881 ranges {0=360448} 2024-04-07T16:26:01.928 INFO:tasks.rados.rados.0.smithi100.stdout:2870: writing smithi10066867-44 from 0 to 360448 tid 1 2024-04-07T16:26:01.928 INFO:tasks.rados.rados.0.smithi100.stdout:2871: copy_from oid 1 from oid 27 current snap is 264 2024-04-07T16:26:01.929 INFO:tasks.rados.rados.0.smithi100.stdout:2864: finishing write tid 3 to smithi10066867-19 2024-04-07T16:26:01.929 INFO:tasks.rados.rados.0.smithi100.stdout:2864: oid 19 updating version 0 to 1189 2024-04-07T16:26:01.929 INFO:tasks.rados.rados.0.smithi100.stdout:2864: oid 19 version 1189 is already newer than 1188 2024-04-07T16:26:01.929 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 19 v 1189 (ObjNum 878 snap 264 seq_num 878) dirty exists 2024-04-07T16:26:01.929 INFO:tasks.rados.rados.0.smithi100.stdout:2864: left oid 19 (ObjNum 878 snap 264 seq_num 878) 2024-04-07T16:26:01.929 INFO:tasks.rados.rados.0.smithi100.stdout:2864: done (6 left) 2024-04-07T16:26:01.929 INFO:tasks.rados.rados.0.smithi100.stdout:2872: read oid 10 snap -1 2024-04-07T16:26:01.930 INFO:tasks.rados.rados.0.smithi100.stdout:2872: expect (ObjNum 869 snap 263 seq_num 869) 2024-04-07T16:26:01.930 INFO:tasks.rados.rados.0.smithi100.stdout:2873: read oid 4 snap -1 2024-04-07T16:26:01.930 INFO:tasks.rados.rados.0.smithi100.stdout:2873: expect deleted 2024-04-07T16:26:01.930 INFO:tasks.rados.rados.0.smithi100.stdout:2874: rollback oid 43 current snap is 264 2024-04-07T16:26:01.930 INFO:tasks.rados.rados.0.smithi100.stdout:2874: done (8 left) 2024-04-07T16:26:01.930 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 22 current snap is 264 2024-04-07T16:26:01.930 INFO:tasks.rados.rados.0.smithi100.stdout:2875: seq_num 882 ranges {5488640=376832} 2024-04-07T16:26:01.971 INFO:tasks.rados.rados.0.smithi100.stdout:2875: writing smithi10066867-22 from 5488640 to 5865472 tid 1 2024-04-07T16:26:01.971 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 50 current snap is 264 2024-04-07T16:26:01.971 INFO:tasks.rados.rados.0.smithi100.stdout:2876: seq_num 883 ranges {14057472=393216} 2024-04-07T16:26:02.052 INFO:tasks.rados.rados.0.smithi100.stdout:2876: writing smithi10066867-50 from 14057472 to 14450688 tid 1 2024-04-07T16:26:02.052 INFO:tasks.rados.rados.0.smithi100.stdout:2877: setattr oid 7 current snap is 264 2024-04-07T16:26:02.052 INFO:tasks.rados.rados.0.smithi100.stdout:2870: finishing write tid 1 to smithi10066867-44 2024-04-07T16:26:02.052 INFO:tasks.rados.rados.0.smithi100.stdout:2867: finishing write tid 1 to smithi10066867-11 2024-04-07T16:26:02.052 INFO:tasks.rados.rados.0.smithi100.stdout:2866: finishing copy_from racing read to smithi10066867-49 2024-04-07T16:26:02.052 INFO:tasks.rados.rados.0.smithi100.stdout:2870: finishing write tid 2 to smithi10066867-44 2024-04-07T16:26:02.052 INFO:tasks.rados.rados.0.smithi100.stdout:2868: finishing write tid 1 to smithi10066867-30 2024-04-07T16:26:02.053 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 34 v 1509 (ObjNum 705 snap 0 seq_num 3067285392) dirty exists 2024-04-07T16:26:02.053 INFO:tasks.rados.rados.0.smithi100.stdout:2868: finishing write tid 2 to smithi10066867-30 2024-04-07T16:26:02.053 INFO:tasks.rados.rados.0.smithi100.stdout:2867: finishing write tid 2 to smithi10066867-11 2024-04-07T16:26:02.053 INFO:tasks.rados.rados.0.smithi100.stdout:2866: finishing copy_from to smithi10066867-49 2024-04-07T16:26:02.053 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 49 v 1130 (ObjNum 737 snap 227 seq_num 737) dirty exists 2024-04-07T16:26:02.053 INFO:tasks.rados.rados.0.smithi100.stdout:2868: finishing write tid 3 to smithi10066867-30 2024-04-07T16:26:02.053 INFO:tasks.rados.rados.0.smithi100.stdout:2868: oid 30 updating version 0 to 1213 2024-04-07T16:26:02.053 INFO:tasks.rados.rados.0.smithi100.stdout:2868: oid 30 updating version 1213 to 1214 2024-04-07T16:26:02.053 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 30 v 1214 (ObjNum 880 snap 264 seq_num 880) dirty exists 2024-04-07T16:26:02.053 INFO:tasks.rados.rados.0.smithi100.stdout:2868: left oid 30 (ObjNum 880 snap 264 seq_num 880) 2024-04-07T16:26:02.053 INFO:tasks.rados.rados.0.smithi100.stdout:2867: finishing write tid 3 to smithi10066867-11 2024-04-07T16:26:02.053 INFO:tasks.rados.rados.0.smithi100.stdout:2867: oid 11 updating version 0 to 1620 2024-04-07T16:26:02.053 INFO:tasks.rados.rados.0.smithi100.stdout:2867: oid 11 version 1620 is already newer than 1619 2024-04-07T16:26:02.053 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 11 v 1620 (ObjNum 879 snap 264 seq_num 879) dirty exists 2024-04-07T16:26:02.054 INFO:tasks.rados.rados.0.smithi100.stdout:2867: left oid 11 (ObjNum 879 snap 264 seq_num 879) 2024-04-07T16:26:02.054 INFO:tasks.rados.rados.0.smithi100.stdout:2870: finishing write tid 3 to smithi10066867-44 2024-04-07T16:26:02.054 INFO:tasks.rados.rados.0.smithi100.stdout:2870: oid 44 updating version 0 to 1294 2024-04-07T16:26:02.054 INFO:tasks.rados.rados.0.smithi100.stdout:2870: oid 44 updating version 1294 to 1295 2024-04-07T16:26:02.054 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 44 v 1295 (ObjNum 881 snap 264 seq_num 881) dirty exists 2024-04-07T16:26:02.054 INFO:tasks.rados.rados.0.smithi100.stdout:2870: left oid 44 (ObjNum 881 snap 264 seq_num 881) 2024-04-07T16:26:02.054 INFO:tasks.rados.rados.0.smithi100.stdout:2872: expect (ObjNum 869 snap 263 seq_num 869) 2024-04-07T16:26:02.358 INFO:tasks.rados.rados.0.smithi100.stdout:2871: finishing copy_from racing read to smithi10066867-1 2024-04-07T16:26:02.358 INFO:tasks.rados.rados.0.smithi100.stdout:2871: finishing copy_from to smithi10066867-1 2024-04-07T16:26:02.358 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 1 v 1132 (ObjNum 863 snap 261 seq_num 863) dirty exists 2024-04-07T16:26:02.358 INFO:tasks.rados.rados.0.smithi100.stdout:2875: finishing write tid 1 to smithi10066867-22 2024-04-07T16:26:02.358 INFO:tasks.rados.rados.0.smithi100.stdout:2875: finishing write tid 2 to smithi10066867-22 2024-04-07T16:26:02.358 INFO:tasks.rados.rados.0.smithi100.stdout:2875: finishing write tid 3 to smithi10066867-22 2024-04-07T16:26:02.358 INFO:tasks.rados.rados.0.smithi100.stdout:2875: oid 22 updating version 0 to 1200 2024-04-07T16:26:02.358 INFO:tasks.rados.rados.0.smithi100.stdout:2875: oid 22 updating version 1200 to 1201 2024-04-07T16:26:02.358 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 22 v 1201 (ObjNum 882 snap 264 seq_num 882) dirty exists 2024-04-07T16:26:02.359 INFO:tasks.rados.rados.0.smithi100.stdout:2875: left oid 22 (ObjNum 882 snap 264 seq_num 882) 2024-04-07T16:26:02.359 INFO:tasks.rados.rados.0.smithi100.stdout:2876: finishing write tid 1 to smithi10066867-50 2024-04-07T16:26:02.359 INFO:tasks.rados.rados.0.smithi100.stdout:2876: finishing write tid 2 to smithi10066867-50 2024-04-07T16:26:02.359 INFO:tasks.rados.rados.0.smithi100.stdout:2876: finishing write tid 3 to smithi10066867-50 2024-04-07T16:26:02.359 INFO:tasks.rados.rados.0.smithi100.stdout:2876: oid 50 updating version 0 to 1192 2024-04-07T16:26:02.359 INFO:tasks.rados.rados.0.smithi100.stdout:2876: oid 50 updating version 1192 to 1193 2024-04-07T16:26:02.359 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 50 v 1193 (ObjNum 883 snap 264 seq_num 883) dirty exists 2024-04-07T16:26:02.359 INFO:tasks.rados.rados.0.smithi100.stdout:2876: left oid 50 (ObjNum 883 snap 264 seq_num 883) 2024-04-07T16:26:02.359 INFO:tasks.rados.rados.0.smithi100.stdout:2866: done (10 left) 2024-04-07T16:26:02.359 INFO:tasks.rados.rados.0.smithi100.stdout:2867: done (9 left) 2024-04-07T16:26:02.359 INFO:tasks.rados.rados.0.smithi100.stdout:2868: done (8 left) 2024-04-07T16:26:02.359 INFO:tasks.rados.rados.0.smithi100.stdout:2869: done (7 left) 2024-04-07T16:26:02.359 INFO:tasks.rados.rados.0.smithi100.stdout:2870: done (6 left) 2024-04-07T16:26:02.359 INFO:tasks.rados.rados.0.smithi100.stdout:2871: done (5 left) 2024-04-07T16:26:02.360 INFO:tasks.rados.rados.0.smithi100.stdout:2872: done (4 left) 2024-04-07T16:26:02.360 INFO:tasks.rados.rados.0.smithi100.stdout:2873: done (3 left) 2024-04-07T16:26:02.360 INFO:tasks.rados.rados.0.smithi100.stdout:2875: done (2 left) 2024-04-07T16:26:02.360 INFO:tasks.rados.rados.0.smithi100.stdout:2876: done (1 left) 2024-04-07T16:26:02.360 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 10 current snap is 264 2024-04-07T16:26:02.360 INFO:tasks.rados.rados.0.smithi100.stdout:2878: seq_num 884 ranges {3489792=409600} 2024-04-07T16:26:02.379 INFO:tasks.rados.rados.0.smithi100.stdout:2878: writing smithi10066867-10 from 3489792 to 3899392 tid 1 2024-04-07T16:26:02.380 INFO:tasks.rados.rados.0.smithi100.stdout:2879: read oid 21 snap -1 2024-04-07T16:26:02.380 INFO:tasks.rados.rados.0.smithi100.stdout:2879: expect deleted 2024-04-07T16:26:02.380 INFO:tasks.rados.rados.0.smithi100.stdout:2880: delete oid 29 current snap is 264 2024-04-07T16:26:02.380 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 7 v 1136 (ObjNum 33 snap 0 seq_num 2702429712) dirty exists 2024-04-07T16:26:02.384 INFO:tasks.rados.rados.0.smithi100.stdout:2877: done (3 left) 2024-04-07T16:26:02.384 INFO:tasks.rados.rados.0.smithi100.stdout:2879: done (2 left) 2024-04-07T16:26:02.384 INFO:tasks.rados.rados.0.smithi100.stdout:2880: done (1 left) 2024-04-07T16:26:02.384 INFO:tasks.rados.rados.0.smithi100.stdout:2881: read oid 6 snap -1 2024-04-07T16:26:02.385 INFO:tasks.rados.rados.0.smithi100.stdout:2881: expect (ObjNum 810 snap 256 seq_num 810) 2024-04-07T16:26:02.385 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 43 current snap is 264 2024-04-07T16:26:02.385 INFO:tasks.rados.rados.0.smithi100.stdout:2878: finishing write tid 1 to smithi10066867-10 2024-04-07T16:26:02.385 INFO:tasks.rados.rados.0.smithi100.stdout:2882: seq_num 886 ranges {0=442368} 2024-04-07T16:26:02.388 INFO:tasks.rados.rados.0.smithi100.stdout:2882: writing smithi10066867-43 from 0 to 442368 tid 1 2024-04-07T16:26:02.388 INFO:tasks.rados.rados.0.smithi100.stdout:2883: delete oid 19 current snap is 264 2024-04-07T16:26:02.388 INFO:tasks.rados.rados.0.smithi100.stdout:2878: finishing write tid 2 to smithi10066867-10 2024-04-07T16:26:02.388 INFO:tasks.rados.rados.0.smithi100.stdout:2878: finishing write tid 3 to smithi10066867-10 2024-04-07T16:26:02.388 INFO:tasks.rados.rados.0.smithi100.stdout:2878: oid 10 updating version 0 to 1439 2024-04-07T16:26:02.388 INFO:tasks.rados.rados.0.smithi100.stdout:2878: oid 10 updating version 1439 to 1440 2024-04-07T16:26:02.388 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 10 v 1440 (ObjNum 884 snap 264 seq_num 884) dirty exists 2024-04-07T16:26:02.388 INFO:tasks.rados.rados.0.smithi100.stdout:2878: left oid 10 (ObjNum 884 snap 264 seq_num 884) 2024-04-07T16:26:02.394 INFO:tasks.rados.rados.0.smithi100.stdout:2882: finishing write tid 1 to smithi10066867-43 2024-04-07T16:26:02.394 INFO:tasks.rados.rados.0.smithi100.stdout:2882: finishing write tid 2 to smithi10066867-43 2024-04-07T16:26:02.397 INFO:tasks.rados.rados.0.smithi100.stdout:2881: expect (ObjNum 810 snap 256 seq_num 810) 2024-04-07T16:26:02.435 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:02 smithi100 bash[76202]: cluster 2024-04-07T16:26:00.745040+0000 mgr.y (mgr.25175) 282 : cluster [DBG] pgmap v498: 121 pgs: 1 active+clean+snaptrim, 120 active+clean; 178 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 18 MiB/s rd, 1.5 MiB/s wr, 21 op/s 2024-04-07T16:26:02.435 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:02 smithi100 bash[76202]: cluster 2024-04-07T16:26:01.189244+0000 mon.a (mon.0) 934 : cluster [DBG] osdmap e604: 8 total, 8 up, 8 in 2024-04-07T16:26:02.435 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:02 smithi100 bash[76202]: audit 2024-04-07T16:26:01.721045+0000 mon.a (mon.0) 935 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:26:02.435 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:02 smithi100 bash[76202]: audit 2024-04-07T16:26:01.722060+0000 mon.a (mon.0) 936 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:26:02.435 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:02 smithi100 bash[76202]: cluster 2024-04-07T16:26:01.896903+0000 mon.a (mon.0) 937 : cluster [DBG] osdmap e605: 8 total, 8 up, 8 in 2024-04-07T16:26:02.436 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:02 smithi100 bash[74942]: cluster 2024-04-07T16:26:00.745040+0000 mgr.y (mgr.25175) 282 : cluster [DBG] pgmap v498: 121 pgs: 1 active+clean+snaptrim, 120 active+clean; 178 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 18 MiB/s rd, 1.5 MiB/s wr, 21 op/s 2024-04-07T16:26:02.436 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:02 smithi100 bash[74942]: cluster 2024-04-07T16:26:01.189244+0000 mon.a (mon.0) 934 : cluster [DBG] osdmap e604: 8 total, 8 up, 8 in 2024-04-07T16:26:02.436 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:02 smithi100 bash[74942]: audit 2024-04-07T16:26:01.721045+0000 mon.a (mon.0) 935 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:26:02.436 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:02 smithi100 bash[74942]: audit 2024-04-07T16:26:01.722060+0000 mon.a (mon.0) 936 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:26:02.436 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:02 smithi100 bash[74942]: cluster 2024-04-07T16:26:01.896903+0000 mon.a (mon.0) 937 : cluster [DBG] osdmap e605: 8 total, 8 up, 8 in 2024-04-07T16:26:02.438 INFO:tasks.rados.rados.0.smithi100.stdout:2882: finishing write tid 3 to smithi10066867-43 2024-04-07T16:26:02.438 INFO:tasks.rados.rados.0.smithi100.stdout:2882: oid 43 updating version 0 to 1137 2024-04-07T16:26:02.438 INFO:tasks.rados.rados.0.smithi100.stdout:2882: oid 43 updating version 1137 to 1138 2024-04-07T16:26:02.438 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 43 v 1138 (ObjNum 886 snap 264 seq_num 886) dirty exists 2024-04-07T16:26:02.438 INFO:tasks.rados.rados.0.smithi100.stdout:2882: left oid 43 (ObjNum 886 snap 264 seq_num 886) 2024-04-07T16:26:02.438 INFO:tasks.rados.rados.0.smithi100.stdout:2878: done (3 left) 2024-04-07T16:26:02.438 INFO:tasks.rados.rados.0.smithi100.stdout:2881: done (2 left) 2024-04-07T16:26:02.438 INFO:tasks.rados.rados.0.smithi100.stdout:2882: done (1 left) 2024-04-07T16:26:02.438 INFO:tasks.rados.rados.0.smithi100.stdout:2883: done (0 left) 2024-04-07T16:26:02.438 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 14 current snap is 264 2024-04-07T16:26:02.439 INFO:tasks.rados.rados.0.smithi100.stdout:2884: seq_num 888 ranges {770048=475136} 2024-04-07T16:26:02.451 INFO:tasks.rados.rados.0.smithi100.stdout:2884: writing smithi10066867-14 from 770048 to 1245184 tid 1 2024-04-07T16:26:02.451 INFO:tasks.rados.rados.0.smithi100.stdout:2885: copy_from oid 32 from oid 49 current snap is 264 2024-04-07T16:26:02.451 INFO:tasks.rados.rados.0.smithi100.stdout:2886: snap_create 2024-04-07T16:26:02.457 INFO:tasks.rados.rados.0.smithi100.stdout:2884: finishing write tid 1 to smithi10066867-14 2024-04-07T16:26:02.458 INFO:tasks.rados.rados.0.smithi100.stdout:2884: finishing write tid 2 to smithi10066867-14 2024-04-07T16:26:02.460 INFO:tasks.rados.rados.0.smithi100.stdout:2884: finishing write tid 3 to smithi10066867-14 2024-04-07T16:26:02.460 INFO:tasks.rados.rados.0.smithi100.stdout:2884: oid 14 updating version 0 to 1195 2024-04-07T16:26:02.460 INFO:tasks.rados.rados.0.smithi100.stdout:2884: oid 14 version 1195 is already newer than 1194 2024-04-07T16:26:02.460 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 14 v 1195 (ObjNum 888 snap 264 seq_num 888) dirty exists 2024-04-07T16:26:02.460 INFO:tasks.rados.rados.0.smithi100.stdout:2884: left oid 14 (ObjNum 888 snap 264 seq_num 888) 2024-04-07T16:26:02.474 INFO:tasks.rados.rados.0.smithi100.stdout:2885: finishing copy_from racing read to smithi10066867-32 2024-04-07T16:26:02.474 INFO:tasks.rados.rados.0.smithi100.stdout:2885: finishing copy_from to smithi10066867-32 2024-04-07T16:26:02.474 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 32 v 1130 (ObjNum 737 snap 227 seq_num 737) dirty exists 2024-04-07T16:26:02.606 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:26:02 smithi144 bash[59794]: ts=2024-04-07T16:26:02.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:26:02.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:02 smithi144 bash[66654]: cluster 2024-04-07T16:26:00.745040+0000 mgr.y (mgr.25175) 282 : cluster [DBG] pgmap v498: 121 pgs: 1 active+clean+snaptrim, 120 active+clean; 178 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 18 MiB/s rd, 1.5 MiB/s wr, 21 op/s 2024-04-07T16:26:02.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:02 smithi144 bash[66654]: cluster 2024-04-07T16:26:01.189244+0000 mon.a (mon.0) 934 : cluster [DBG] osdmap e604: 8 total, 8 up, 8 in 2024-04-07T16:26:02.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:02 smithi144 bash[66654]: audit 2024-04-07T16:26:01.721045+0000 mon.a (mon.0) 935 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:26:02.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:02 smithi144 bash[66654]: audit 2024-04-07T16:26:01.722060+0000 mon.a (mon.0) 936 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:26:02.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:02 smithi144 bash[66654]: cluster 2024-04-07T16:26:01.896903+0000 mon.a (mon.0) 937 : cluster [DBG] osdmap e605: 8 total, 8 up, 8 in 2024-04-07T16:26:03.184 INFO:tasks.rados.rados.0.smithi100.stdout:2884: done (2 left) 2024-04-07T16:26:03.184 INFO:tasks.rados.rados.0.smithi100.stdout:2885: done (1 left) 2024-04-07T16:26:03.184 INFO:tasks.rados.rados.0.smithi100.stdout:2886: done (0 left) 2024-04-07T16:26:03.184 INFO:tasks.rados.rados.0.smithi100.stdout:2887: copy_from oid 4 from oid 16 current snap is 265 2024-04-07T16:26:03.184 INFO:tasks.rados.rados.0.smithi100.stdout:2888: snap_create 2024-04-07T16:26:03.191 INFO:tasks.rados.rados.0.smithi100.stdout:2887: finishing copy_from to smithi10066867-4 2024-04-07T16:26:03.191 INFO:tasks.rados.rados.0.smithi100.stdout:2887: got expected ENOENT (src dne) 2024-04-07T16:26:03.191 INFO:tasks.rados.rados.0.smithi100.stdout:2887: finishing copy_from racing read to smithi10066867-4 2024-04-07T16:26:04.193 INFO:tasks.rados.rados.0.smithi100.stdout:2887: done (1 left) 2024-04-07T16:26:04.193 INFO:tasks.rados.rados.0.smithi100.stdout:2888: done (0 left) 2024-04-07T16:26:04.193 INFO:tasks.rados.rados.0.smithi100.stdout:2889: read oid 49 snap -1 2024-04-07T16:26:04.193 INFO:tasks.rados.rados.0.smithi100.stdout:2889: expect (ObjNum 737 snap 227 seq_num 737) 2024-04-07T16:26:04.193 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 22 current snap is 266 2024-04-07T16:26:04.193 INFO:tasks.rados.rados.0.smithi100.stdout:2890: seq_num 889 ranges {5865472=491520} 2024-04-07T16:26:04.234 INFO:tasks.rados.rados.0.smithi100.stdout:2890: writing smithi10066867-22 from 5865472 to 6356992 tid 1 2024-04-07T16:26:04.234 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 26 current snap is 266 2024-04-07T16:26:04.234 INFO:tasks.rados.rados.0.smithi100.stdout:2891: seq_num 890 ranges {3555328=507904} 2024-04-07T16:26:04.259 INFO:tasks.rados.rados.0.smithi100.stdout:2891: writing smithi10066867-26 from 3555328 to 4063232 tid 1 2024-04-07T16:26:04.259 INFO:tasks.rados.rados.0.smithi100.stdout:2892: copy_from oid 45 from oid 28 current snap is 266 2024-04-07T16:26:04.259 INFO:tasks.rados.rados.0.smithi100.stdout:2889: expect (ObjNum 737 snap 227 seq_num 737) 2024-04-07T16:26:04.363 INFO:tasks.rados.rados.0.smithi100.stdout:2890: finishing write tid 1 to smithi10066867-22 2024-04-07T16:26:04.363 INFO:tasks.rados.rados.0.smithi100.stdout:2890: finishing write tid 2 to smithi10066867-22 2024-04-07T16:26:04.363 INFO:tasks.rados.rados.0.smithi100.stdout:2890: finishing write tid 3 to smithi10066867-22 2024-04-07T16:26:04.363 INFO:tasks.rados.rados.0.smithi100.stdout:2890: oid 22 updating version 0 to 1202 2024-04-07T16:26:04.363 INFO:tasks.rados.rados.0.smithi100.stdout:2890: oid 22 updating version 1202 to 1203 2024-04-07T16:26:04.363 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 22 v 1203 (ObjNum 889 snap 266 seq_num 889) dirty exists 2024-04-07T16:26:04.364 INFO:tasks.rados.rados.0.smithi100.stdout:2890: left oid 22 (ObjNum 889 snap 266 seq_num 889) 2024-04-07T16:26:04.364 INFO:tasks.rados.rados.0.smithi100.stdout:2891: finishing write tid 1 to smithi10066867-26 2024-04-07T16:26:04.364 INFO:tasks.rados.rados.0.smithi100.stdout:2889: done (3 left) 2024-04-07T16:26:04.364 INFO:tasks.rados.rados.0.smithi100.stdout:2890: done (2 left) 2024-04-07T16:26:04.364 INFO:tasks.rados.rados.0.smithi100.stdout:2893: snap_create 2024-04-07T16:26:04.364 INFO:tasks.rados.rados.0.smithi100.stdout:2891: finishing write tid 2 to smithi10066867-26 2024-04-07T16:26:04.364 INFO:tasks.rados.rados.0.smithi100.stdout:2891: finishing write tid 3 to smithi10066867-26 2024-04-07T16:26:04.364 INFO:tasks.rados.rados.0.smithi100.stdout:2891: oid 26 updating version 0 to 1204 2024-04-07T16:26:04.364 INFO:tasks.rados.rados.0.smithi100.stdout:2891: oid 26 updating version 1204 to 1205 2024-04-07T16:26:04.364 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 26 v 1205 (ObjNum 890 snap 266 seq_num 890) dirty exists 2024-04-07T16:26:04.364 INFO:tasks.rados.rados.0.smithi100.stdout:2891: left oid 26 (ObjNum 890 snap 266 seq_num 890) 2024-04-07T16:26:04.364 INFO:tasks.rados.rados.0.smithi100.stdout:2892: finishing copy_from racing read to smithi10066867-45 2024-04-07T16:26:04.364 INFO:tasks.rados.rados.0.smithi100.stdout:2892: finishing copy_from to smithi10066867-45 2024-04-07T16:26:04.364 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 45 v 1038 (ObjNum 289 snap 0 seq_num 4239877261) dirty exists 2024-04-07T16:26:04.435 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:04 smithi100 bash[76202]: cluster 2024-04-07T16:26:02.746192+0000 mgr.y (mgr.25175) 283 : cluster [DBG] pgmap v501: 121 pgs: 1 active+clean+snaptrim, 120 active+clean; 157 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 9.0 MiB/s rd, 234 KiB/s wr, 15 op/s 2024-04-07T16:26:04.435 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:04 smithi100 bash[76202]: cluster 2024-04-07T16:26:03.200984+0000 mon.a (mon.0) 938 : cluster [DBG] osdmap e606: 8 total, 8 up, 8 in 2024-04-07T16:26:04.435 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:04 smithi100 bash[74942]: cluster 2024-04-07T16:26:02.746192+0000 mgr.y (mgr.25175) 283 : cluster [DBG] pgmap v501: 121 pgs: 1 active+clean+snaptrim, 120 active+clean; 157 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 9.0 MiB/s rd, 234 KiB/s wr, 15 op/s 2024-04-07T16:26:04.436 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:04 smithi100 bash[74942]: cluster 2024-04-07T16:26:03.200984+0000 mon.a (mon.0) 938 : cluster [DBG] osdmap e606: 8 total, 8 up, 8 in 2024-04-07T16:26:04.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:04 smithi144 bash[66654]: cluster 2024-04-07T16:26:02.746192+0000 mgr.y (mgr.25175) 283 : cluster [DBG] pgmap v501: 121 pgs: 1 active+clean+snaptrim, 120 active+clean; 157 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 9.0 MiB/s rd, 234 KiB/s wr, 15 op/s 2024-04-07T16:26:04.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:04 smithi144 bash[66654]: cluster 2024-04-07T16:26:03.200984+0000 mon.a (mon.0) 938 : cluster [DBG] osdmap e606: 8 total, 8 up, 8 in 2024-04-07T16:26:05.354 INFO:tasks.rados.rados.0.smithi100.stdout:2891: done (2 left) 2024-04-07T16:26:05.354 INFO:tasks.rados.rados.0.smithi100.stdout:2892: done (1 left) 2024-04-07T16:26:05.354 INFO:tasks.rados.rados.0.smithi100.stdout:2893: done (0 left) 2024-04-07T16:26:05.354 INFO:tasks.rados.rados.0.smithi100.stdout:2894: snap_remove snap 266 2024-04-07T16:26:05.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:05 smithi144 bash[66654]: cluster 2024-04-07T16:26:04.208269+0000 mon.a (mon.0) 939 : cluster [DBG] osdmap e607: 8 total, 8 up, 8 in 2024-04-07T16:26:05.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:05 smithi144 bash[66654]: audit 2024-04-07T16:26:04.346076+0000 mon.a (mon.0) 940 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:26:05.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:05 smithi144 bash[66654]: audit 2024-04-07T16:26:04.353415+0000 mon.a (mon.0) 941 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:26:05.685 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:05 smithi100 bash[74942]: cluster 2024-04-07T16:26:04.208269+0000 mon.a (mon.0) 939 : cluster [DBG] osdmap e607: 8 total, 8 up, 8 in 2024-04-07T16:26:05.685 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:05 smithi100 bash[74942]: audit 2024-04-07T16:26:04.346076+0000 mon.a (mon.0) 940 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:26:05.685 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:05 smithi100 bash[74942]: audit 2024-04-07T16:26:04.353415+0000 mon.a (mon.0) 941 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:26:05.686 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:05 smithi100 bash[76202]: cluster 2024-04-07T16:26:04.208269+0000 mon.a (mon.0) 939 : cluster [DBG] osdmap e607: 8 total, 8 up, 8 in 2024-04-07T16:26:05.686 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:05 smithi100 bash[76202]: audit 2024-04-07T16:26:04.346076+0000 mon.a (mon.0) 940 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:26:05.686 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:05 smithi100 bash[76202]: audit 2024-04-07T16:26:04.353415+0000 mon.a (mon.0) 941 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:26:06.363 INFO:tasks.rados.rados.0.smithi100.stdout:2894: done (0 left) 2024-04-07T16:26:06.363 INFO:tasks.rados.rados.0.smithi100.stdout:2895: snap_remove snap 265 2024-04-07T16:26:06.685 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:06 smithi100 bash[74942]: cluster 2024-04-07T16:26:04.746863+0000 mgr.y (mgr.25175) 284 : cluster [DBG] pgmap v504: 121 pgs: 121 active+clean; 111 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 16 op/s 2024-04-07T16:26:06.685 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:06 smithi100 bash[74942]: cluster 2024-04-07T16:26:05.370963+0000 mon.a (mon.0) 942 : cluster [DBG] osdmap e608: 8 total, 8 up, 8 in 2024-04-07T16:26:06.686 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:06 smithi100 bash[74942]: audit 2024-04-07T16:26:05.452606+0000 mon.a (mon.0) 943 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:26:06.686 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:06 smithi100 bash[74942]: audit 2024-04-07T16:26:05.459599+0000 mon.a (mon.0) 944 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:26:06.686 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:06 smithi100 bash[76202]: cluster 2024-04-07T16:26:04.746863+0000 mgr.y (mgr.25175) 284 : cluster [DBG] pgmap v504: 121 pgs: 121 active+clean; 111 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 16 op/s 2024-04-07T16:26:06.686 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:06 smithi100 bash[76202]: cluster 2024-04-07T16:26:05.370963+0000 mon.a (mon.0) 942 : cluster [DBG] osdmap e608: 8 total, 8 up, 8 in 2024-04-07T16:26:06.686 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:06 smithi100 bash[76202]: audit 2024-04-07T16:26:05.452606+0000 mon.a (mon.0) 943 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:26:06.686 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:06 smithi100 bash[76202]: audit 2024-04-07T16:26:05.459599+0000 mon.a (mon.0) 944 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:26:06.856 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:06 smithi144 bash[66654]: cluster 2024-04-07T16:26:04.746863+0000 mgr.y (mgr.25175) 284 : cluster [DBG] pgmap v504: 121 pgs: 121 active+clean; 111 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 16 op/s 2024-04-07T16:26:06.856 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:06 smithi144 bash[66654]: cluster 2024-04-07T16:26:05.370963+0000 mon.a (mon.0) 942 : cluster [DBG] osdmap e608: 8 total, 8 up, 8 in 2024-04-07T16:26:06.856 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:06 smithi144 bash[66654]: audit 2024-04-07T16:26:05.452606+0000 mon.a (mon.0) 943 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:26:06.856 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:06 smithi144 bash[66654]: audit 2024-04-07T16:26:05.459599+0000 mon.a (mon.0) 944 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:26:06.882 INFO:tasks.rados.rados.0.smithi100.stdout:2895: done (0 left) 2024-04-07T16:26:06.882 INFO:tasks.rados.rados.0.smithi100.stdout:2896: rollback oid 35 current snap is 267 2024-04-07T16:26:06.882 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 35 to 264 2024-04-07T16:26:06.882 INFO:tasks.rados.rados.0.smithi100.stdout:2897: copy_from oid 17 from oid 13 current snap is 267 2024-04-07T16:26:06.882 INFO:tasks.rados.rados.0.smithi100.stdout:2898: setattr oid 3 current snap is 267 2024-04-07T16:26:06.884 INFO:tasks.rados.rados.0.smithi100.stdout:2899: read oid 43 snap -1 2024-04-07T16:26:06.884 INFO:tasks.rados.rados.0.smithi100.stdout:2899: expect (ObjNum 886 snap 264 seq_num 886) 2024-04-07T16:26:06.884 INFO:tasks.rados.rados.0.smithi100.stdout:2900: rollback oid 27 current snap is 267 2024-04-07T16:26:06.884 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 27 to 264 2024-04-07T16:26:06.884 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 29 current snap is 267 2024-04-07T16:26:06.884 INFO:tasks.rados.rados.0.smithi100.stdout:2901: seq_num 891 ranges {0=532480} 2024-04-07T16:26:06.890 INFO:tasks.rados.rados.0.smithi100.stdout:2901: writing smithi10066867-29 from 0 to 532480 tid 1 2024-04-07T16:26:06.891 INFO:tasks.rados.rados.0.smithi100.stdout:2902: read oid 33 snap -1 2024-04-07T16:26:06.891 INFO:tasks.rados.rados.0.smithi100.stdout:2902: expect (ObjNum 753 snap 235 seq_num 753) 2024-04-07T16:26:06.891 INFO:tasks.rados.rados.0.smithi100.stdout:2903: read oid 18 snap 264 2024-04-07T16:26:06.891 INFO:tasks.rados.rados.0.smithi100.stdout:2903: expect (ObjNum 861 snap 261 seq_num 861) 2024-04-07T16:26:06.892 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 10 current snap is 267 2024-04-07T16:26:06.892 INFO:tasks.rados.rados.0.smithi100.stdout:2904: seq_num 892 ranges {3899392=548864} 2024-04-07T16:26:06.932 INFO:tasks.rados.rados.0.smithi100.stdout:2904: writing smithi10066867-10 from 3899392 to 4448256 tid 1 2024-04-07T16:26:06.932 INFO:tasks.rados.rados.0.smithi100.stdout:2905: snap_create 2024-04-07T16:26:06.932 INFO:tasks.rados.rados.0.smithi100.stdout:2899: expect (ObjNum 886 snap 264 seq_num 886) 2024-04-07T16:26:06.945 INFO:tasks.rados.rados.0.smithi100.stdout:2896: finishing rollback tid 0 to smithi10066867-35 2024-04-07T16:26:06.945 INFO:tasks.rados.rados.0.smithi100.stdout:2896: finishing rollback tid 1 to smithi10066867-35 2024-04-07T16:26:06.945 INFO:tasks.rados.rados.0.smithi100.stdout:2896: finishing rollback tid 2 to smithi10066867-35 2024-04-07T16:26:06.945 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 35 v 1122 (ObjNum 161 snap 0 seq_num 0) dirty exists 2024-04-07T16:26:06.945 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 3 v 1126 (ObjNum 865 snap 263 seq_num 865) dirty exists 2024-04-07T16:26:06.945 INFO:tasks.rados.rados.0.smithi100.stdout:2902: expect (ObjNum 753 snap 235 seq_num 753) 2024-04-07T16:26:06.962 INFO:tasks.rados.rados.0.smithi100.stdout:2900: finishing rollback tid 0 to smithi10066867-27 2024-04-07T16:26:06.963 INFO:tasks.rados.rados.0.smithi100.stdout:2900: finishing rollback tid 1 to smithi10066867-27 2024-04-07T16:26:06.963 INFO:tasks.rados.rados.0.smithi100.stdout:2900: finishing rollback tid 2 to smithi10066867-27 2024-04-07T16:26:06.963 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 27 v 1141 (ObjNum 863 snap 261 seq_num 863) dirty exists 2024-04-07T16:26:06.963 INFO:tasks.rados.rados.0.smithi100.stdout:2901: finishing write tid 1 to smithi10066867-29 2024-04-07T16:26:06.963 INFO:tasks.rados.rados.0.smithi100.stdout:2901: finishing write tid 2 to smithi10066867-29 2024-04-07T16:26:06.963 INFO:tasks.rados.rados.0.smithi100.stdout:2897: finishing copy_from racing read to smithi10066867-17 2024-04-07T16:26:06.963 INFO:tasks.rados.rados.0.smithi100.stdout:2901: finishing write tid 3 to smithi10066867-29 2024-04-07T16:26:06.963 INFO:tasks.rados.rados.0.smithi100.stdout:2901: oid 29 updating version 0 to 1216 2024-04-07T16:26:06.963 INFO:tasks.rados.rados.0.smithi100.stdout:2901: oid 29 version 1216 is already newer than 1215 2024-04-07T16:26:06.963 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 29 v 1216 (ObjNum 891 snap 267 seq_num 891) dirty exists 2024-04-07T16:26:06.963 INFO:tasks.rados.rados.0.smithi100.stdout:2901: left oid 29 (ObjNum 891 snap 267 seq_num 891) 2024-04-07T16:26:06.963 INFO:tasks.rados.rados.0.smithi100.stdout:2897: finishing copy_from to smithi10066867-17 2024-04-07T16:26:06.963 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 17 v 1091 (ObjNum 289 snap 0 seq_num 0) dirty exists 2024-04-07T16:26:06.964 INFO:tasks.rados.rados.0.smithi100.stdout:2903: expect (ObjNum 861 snap 261 seq_num 861) 2024-04-07T16:26:07.139 INFO:tasks.rados.rados.0.smithi100.stdout:2904: finishing write tid 1 to smithi10066867-10 2024-04-07T16:26:07.139 INFO:tasks.rados.rados.0.smithi100.stdout:2904: finishing write tid 2 to smithi10066867-10 2024-04-07T16:26:07.139 INFO:tasks.rados.rados.0.smithi100.stdout:2904: finishing write tid 3 to smithi10066867-10 2024-04-07T16:26:07.139 INFO:tasks.rados.rados.0.smithi100.stdout:2904: oid 10 updating version 0 to 1441 2024-04-07T16:26:07.139 INFO:tasks.rados.rados.0.smithi100.stdout:2904: oid 10 updating version 1441 to 1442 2024-04-07T16:26:07.139 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 10 v 1442 (ObjNum 892 snap 267 seq_num 892) dirty exists 2024-04-07T16:26:07.139 INFO:tasks.rados.rados.0.smithi100.stdout:2904: left oid 10 (ObjNum 892 snap 267 seq_num 892) 2024-04-07T16:26:07.685 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:07 smithi100 bash[74942]: cluster 2024-04-07T16:26:06.375662+0000 mon.a (mon.0) 945 : cluster [DBG] osdmap e609: 8 total, 8 up, 8 in 2024-04-07T16:26:07.685 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:07 smithi100 bash[74942]: cluster 2024-04-07T16:26:06.747622+0000 mgr.y (mgr.25175) 285 : cluster [DBG] pgmap v507: 121 pgs: 121 active+clean; 95 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 19 op/s 2024-04-07T16:26:07.685 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:07 smithi100 bash[74942]: cluster 2024-04-07T16:26:06.897724+0000 mon.a (mon.0) 946 : cluster [DBG] osdmap e610: 8 total, 8 up, 8 in 2024-04-07T16:26:07.686 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:07 smithi100 bash[76202]: cluster 2024-04-07T16:26:06.375662+0000 mon.a (mon.0) 945 : cluster [DBG] osdmap e609: 8 total, 8 up, 8 in 2024-04-07T16:26:07.686 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:07 smithi100 bash[76202]: cluster 2024-04-07T16:26:06.747622+0000 mgr.y (mgr.25175) 285 : cluster [DBG] pgmap v507: 121 pgs: 121 active+clean; 95 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 19 op/s 2024-04-07T16:26:07.686 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:07 smithi100 bash[76202]: cluster 2024-04-07T16:26:06.897724+0000 mon.a (mon.0) 946 : cluster [DBG] osdmap e610: 8 total, 8 up, 8 in 2024-04-07T16:26:07.793 INFO:teuthology.orchestra.run.smithi100.stdout:true 2024-04-07T16:26:07.856 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:07 smithi144 bash[66654]: cluster 2024-04-07T16:26:06.375662+0000 mon.a (mon.0) 945 : cluster [DBG] osdmap e609: 8 total, 8 up, 8 in 2024-04-07T16:26:07.856 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:07 smithi144 bash[66654]: cluster 2024-04-07T16:26:06.747622+0000 mgr.y (mgr.25175) 285 : cluster [DBG] pgmap v507: 121 pgs: 121 active+clean; 95 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 19 op/s 2024-04-07T16:26:07.856 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:07 smithi144 bash[66654]: cluster 2024-04-07T16:26:06.897724+0000 mon.a (mon.0) 946 : cluster [DBG] osdmap e610: 8 total, 8 up, 8 in 2024-04-07T16:26:07.885 INFO:tasks.rados.rados.0.smithi100.stdout:2896: done (9 left) 2024-04-07T16:26:07.885 INFO:tasks.rados.rados.0.smithi100.stdout:2897: done (8 left) 2024-04-07T16:26:07.885 INFO:tasks.rados.rados.0.smithi100.stdout:2898: done (7 left) 2024-04-07T16:26:07.885 INFO:tasks.rados.rados.0.smithi100.stdout:2899: done (6 left) 2024-04-07T16:26:07.885 INFO:tasks.rados.rados.0.smithi100.stdout:2900: done (5 left) 2024-04-07T16:26:07.885 INFO:tasks.rados.rados.0.smithi100.stdout:2901: done (4 left) 2024-04-07T16:26:07.885 INFO:tasks.rados.rados.0.smithi100.stdout:2902: done (3 left) 2024-04-07T16:26:07.885 INFO:tasks.rados.rados.0.smithi100.stdout:2903: done (2 left) 2024-04-07T16:26:07.886 INFO:tasks.rados.rados.0.smithi100.stdout:2904: done (1 left) 2024-04-07T16:26:07.886 INFO:tasks.rados.rados.0.smithi100.stdout:2905: done (0 left) 2024-04-07T16:26:07.886 INFO:tasks.rados.rados.0.smithi100.stdout:2906: rollback oid 5 current snap is 268 2024-04-07T16:26:07.886 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 5 to 264 2024-04-07T16:26:07.886 INFO:tasks.rados.rados.0.smithi100.stdout:2907: read oid 13 snap -1 2024-04-07T16:26:07.886 INFO:tasks.rados.rados.0.smithi100.stdout:2907: expect (ObjNum 0 snap 0 seq_num 0) 2024-04-07T16:26:07.886 INFO:tasks.rados.rados.0.smithi100.stdout:2908: read oid 36 snap -1 2024-04-07T16:26:07.886 INFO:tasks.rados.rados.0.smithi100.stdout:2908: expect (ObjNum 830 snap 260 seq_num 830) 2024-04-07T16:26:07.886 INFO:tasks.rados.rados.0.smithi100.stdout:2909: delete oid 10 current snap is 268 2024-04-07T16:26:07.892 INFO:tasks.rados.rados.0.smithi100.stdout:2907: expect (ObjNum 0 snap 0 seq_num 0) 2024-04-07T16:26:07.898 INFO:tasks.rados.rados.0.smithi100.stdout:2907: done (3 left) 2024-04-07T16:26:07.899 INFO:tasks.rados.rados.0.smithi100.stdout:2909: done (2 left) 2024-04-07T16:26:07.899 INFO:tasks.rados.rados.0.smithi100.stdout:2910: rmattr oid 18 current snap is 268 2024-04-07T16:26:07.899 INFO:tasks.rados.rados.0.smithi100.stdout:2910: done (2 left) 2024-04-07T16:26:07.899 INFO:tasks.rados.rados.0.smithi100.stdout:2911: snap_remove snap 267 2024-04-07T16:26:08.181 INFO:teuthology.orchestra.run.smithi100.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-07T16:26:08.181 INFO:teuthology.orchestra.run.smithi100.stdout:alertmanager.a smithi100 *:9093,9094 running (12m) 3s ago 12m 14.6M - 0.25.0 c8568f914cd2 5143075e7d26 2024-04-07T16:26:08.181 INFO:teuthology.orchestra.run.smithi100.stdout:grafana.a smithi144 *:3000 running (12m) 30s ago 12m 78.5M - 9.4.7 954c08fa6188 7180c0708acd 2024-04-07T16:26:08.181 INFO:teuthology.orchestra.run.smithi100.stdout:mds.a smithi100 running (9s) 3s ago 14m 11.6M - 19.0.0-2785-gadce92d0 c0ecd4226ac9 075552e67bd0 2024-04-07T16:26:08.181 INFO:teuthology.orchestra.run.smithi100.stdout:mds.b smithi144 running (14m) 30s ago 14m 20.6M - 18.2.2-1171-g267c8ff5 8a4a01e7186c 3df2ee3e17a4 2024-04-07T16:26:08.182 INFO:teuthology.orchestra.run.smithi100.stdout:mgr.x smithi144 *:8443,9283,8765 running (6m) 30s ago 21m 440M - 19.0.0-2785-gadce92d0 c0ecd4226ac9 233745c292a6 2024-04-07T16:26:08.182 INFO:teuthology.orchestra.run.smithi100.stdout:mgr.y smithi100 *:8443,9283,8765 running (7m) 3s ago 23m 506M - 19.0.0-2785-gadce92d0 c0ecd4226ac9 254dc65cdaa7 2024-04-07T16:26:08.182 INFO:teuthology.orchestra.run.smithi100.stdout:mon.a smithi100 running (6m) 3s ago 23m 64.3M 2048M 19.0.0-2785-gadce92d0 c0ecd4226ac9 718e12655c04 2024-04-07T16:26:08.182 INFO:teuthology.orchestra.run.smithi100.stdout:mon.b smithi144 running (4m) 30s ago 21m 47.6M 2048M 19.0.0-2785-gadce92d0 c0ecd4226ac9 aef817664bff 2024-04-07T16:26:08.182 INFO:teuthology.orchestra.run.smithi100.stdout:mon.c smithi100 running (5m) 3s ago 21m 53.3M 2048M 19.0.0-2785-gadce92d0 c0ecd4226ac9 63e889040f6e 2024-04-07T16:26:08.182 INFO:teuthology.orchestra.run.smithi100.stdout:node-exporter.a smithi100 *:9100 running (13m) 3s ago 13m 8496k - 1.5.0 0da6a335fe13 4bce264c1b26 2024-04-07T16:26:08.182 INFO:teuthology.orchestra.run.smithi100.stdout:node-exporter.b smithi144 *:9100 running (13m) 30s ago 13m 8735k - 1.5.0 0da6a335fe13 b6963413c560 2024-04-07T16:26:08.182 INFO:teuthology.orchestra.run.smithi100.stdout:osd.0 smithi100 running (3m) 3s ago 20m 321M 2524M 19.0.0-2785-gadce92d0 c0ecd4226ac9 a6e83c9d1abc 2024-04-07T16:26:08.182 INFO:teuthology.orchestra.run.smithi100.stdout:osd.1 smithi100 running (3m) 3s ago 19m 284M 2524M 19.0.0-2785-gadce92d0 c0ecd4226ac9 c50fe40079b6 2024-04-07T16:26:08.182 INFO:teuthology.orchestra.run.smithi100.stdout:osd.2 smithi100 running (2m) 3s ago 18m 340M 2524M 19.0.0-2785-gadce92d0 c0ecd4226ac9 8e631c43064b 2024-04-07T16:26:08.182 INFO:teuthology.orchestra.run.smithi100.stdout:osd.3 smithi100 running (2m) 3s ago 18m 148M 2524M 19.0.0-2785-gadce92d0 c0ecd4226ac9 402931247c79 2024-04-07T16:26:08.182 INFO:teuthology.orchestra.run.smithi100.stdout:osd.4 smithi144 running (2m) 30s ago 17m 276M 2524M 19.0.0-2785-gadce92d0 c0ecd4226ac9 ebbb7be95d33 2024-04-07T16:26:08.182 INFO:teuthology.orchestra.run.smithi100.stdout:osd.5 smithi144 running (94s) 30s ago 16m 152M 2524M 19.0.0-2785-gadce92d0 c0ecd4226ac9 ecce27bcb2f8 2024-04-07T16:26:08.183 INFO:teuthology.orchestra.run.smithi100.stdout:osd.6 smithi144 running (65s) 30s ago 15m 83.4M 2524M 19.0.0-2785-gadce92d0 c0ecd4226ac9 3788c5c3f427 2024-04-07T16:26:08.183 INFO:teuthology.orchestra.run.smithi100.stdout:osd.7 smithi144 running (36s) 30s ago 14m 11.8M 2524M 19.0.0-2785-gadce92d0 c0ecd4226ac9 9365aba97dc0 2024-04-07T16:26:08.183 INFO:teuthology.orchestra.run.smithi100.stdout:prometheus.a smithi144 *:9095 running (6m) 30s ago 13m 53.2M - 2.43.0 a07b618ecd1d 2d0a8131f303 2024-04-07T16:26:08.620 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-04-07T16:26:08.620 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": { 2024-04-07T16:26:08.620 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph version 19.0.0-2785-gadce92d0 (adce92d056533f22dc1ee37e3141d098b046ec1b) squid (dev)": 3 2024-04-07T16:26:08.620 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-07T16:26:08.620 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": { 2024-04-07T16:26:08.621 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph version 19.0.0-2785-gadce92d0 (adce92d056533f22dc1ee37e3141d098b046ec1b) squid (dev)": 2 2024-04-07T16:26:08.621 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-07T16:26:08.621 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": { 2024-04-07T16:26:08.621 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph version 19.0.0-2785-gadce92d0 (adce92d056533f22dc1ee37e3141d098b046ec1b) squid (dev)": 8 2024-04-07T16:26:08.621 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-07T16:26:08.621 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": { 2024-04-07T16:26:08.621 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph version 18.2.2-1171-g267c8ff5 (267c8ff5b1ff5815581b3b2381d19360e081b37b) reef (stable)": 1, 2024-04-07T16:26:08.621 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph version 19.0.0-2785-gadce92d0 (adce92d056533f22dc1ee37e3141d098b046ec1b) squid (dev)": 1 2024-04-07T16:26:08.621 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-07T16:26:08.621 INFO:teuthology.orchestra.run.smithi100.stdout: "overall": { 2024-04-07T16:26:08.621 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph version 18.2.2-1171-g267c8ff5 (267c8ff5b1ff5815581b3b2381d19360e081b37b) reef (stable)": 1, 2024-04-07T16:26:08.621 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph version 19.0.0-2785-gadce92d0 (adce92d056533f22dc1ee37e3141d098b046ec1b) squid (dev)": 14 2024-04-07T16:26:08.621 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-04-07T16:26:08.621 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-04-07T16:26:08.685 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:26:08 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:26:08] "GET /metrics HTTP/1.1" 200 36338 "" "Prometheus/2.43.0" 2024-04-07T16:26:08.898 INFO:tasks.rados.rados.0.smithi100.stdout:2911: done (2 left) 2024-04-07T16:26:08.901 INFO:tasks.rados.rados.0.smithi100.stdout:2912: delete oid 16 current snap is 268 2024-04-07T16:26:08.901 INFO:tasks.rados.rados.0.smithi100.stdout:2908: expect (ObjNum 830 snap 260 seq_num 830) 2024-04-07T16:26:08.995 INFO:tasks.rados.rados.0.smithi100.stdout:2906: finishing rollback tid 0 to smithi10066867-5 2024-04-07T16:26:08.995 INFO:tasks.rados.rados.0.smithi100.stdout:2906: finishing rollback tid 1 to smithi10066867-5 2024-04-07T16:26:08.996 INFO:tasks.rados.rados.0.smithi100.stdout:2906: finishing rollback tid 2 to smithi10066867-5 2024-04-07T16:26:08.996 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 5 v 1407 (ObjNum -1592847184 snap 21978 seq_num 304) dirty exists 2024-04-07T16:26:08.996 INFO:tasks.rados.rados.0.smithi100.stdout:2906: done (2 left) 2024-04-07T16:26:08.996 INFO:tasks.rados.rados.0.smithi100.stdout:2908: done (1 left) 2024-04-07T16:26:08.996 INFO:tasks.rados.rados.0.smithi100.stdout:2912: done (0 left) 2024-04-07T16:26:08.996 INFO:tasks.rados.rados.0.smithi100.stdout:2913: read oid 46 snap -1 2024-04-07T16:26:08.996 INFO:tasks.rados.rados.0.smithi100.stdout:2913: expect (ObjNum 0 snap 0 seq_num 0) 2024-04-07T16:26:08.996 INFO:tasks.rados.rados.0.smithi100.stdout:2914: snap_create 2024-04-07T16:26:09.003 INFO:tasks.rados.rados.0.smithi100.stdout:2913: expect (ObjNum 0 snap 0 seq_num 0) 2024-04-07T16:26:09.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:08 smithi100 bash[74942]: audit 2024-04-07T16:26:07.795677+0000 mgr.y (mgr.25175) 286 : audit [DBG] from='client.34407 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:26:09.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:08 smithi100 bash[74942]: cluster 2024-04-07T16:26:07.901351+0000 mon.a (mon.0) 947 : cluster [DBG] osdmap e611: 8 total, 8 up, 8 in 2024-04-07T16:26:09.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:08 smithi100 bash[74942]: audit 2024-04-07T16:26:08.158979+0000 mgr.y (mgr.25175) 287 : audit [DBG] from='client.34413 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:26:09.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:08 smithi100 bash[74942]: audit 2024-04-07T16:26:08.637013+0000 mon.c (mon.2) 13 : audit [DBG] from='client.? 172.21.15.100:0/3589338788' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:26:09.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:08 smithi100 bash[76202]: audit 2024-04-07T16:26:07.795677+0000 mgr.y (mgr.25175) 286 : audit [DBG] from='client.34407 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:26:09.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:08 smithi100 bash[76202]: cluster 2024-04-07T16:26:07.901351+0000 mon.a (mon.0) 947 : cluster [DBG] osdmap e611: 8 total, 8 up, 8 in 2024-04-07T16:26:09.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:08 smithi100 bash[76202]: audit 2024-04-07T16:26:08.158979+0000 mgr.y (mgr.25175) 287 : audit [DBG] from='client.34413 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:26:09.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:08 smithi100 bash[76202]: audit 2024-04-07T16:26:08.637013+0000 mon.c (mon.2) 13 : audit [DBG] from='client.? 172.21.15.100:0/3589338788' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:26:09.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:08 smithi144 bash[66654]: audit 2024-04-07T16:26:07.795677+0000 mgr.y (mgr.25175) 286 : audit [DBG] from='client.34407 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:26:09.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:08 smithi144 bash[66654]: cluster 2024-04-07T16:26:07.901351+0000 mon.a (mon.0) 947 : cluster [DBG] osdmap e611: 8 total, 8 up, 8 in 2024-04-07T16:26:09.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:08 smithi144 bash[66654]: audit 2024-04-07T16:26:08.158979+0000 mgr.y (mgr.25175) 287 : audit [DBG] from='client.34413 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:26:09.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:08 smithi144 bash[66654]: audit 2024-04-07T16:26:08.637013+0000 mon.c (mon.2) 13 : audit [DBG] from='client.? 172.21.15.100:0/3589338788' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:26:09.897 INFO:tasks.rados.rados.0.smithi100.stdout:2913: done (1 left) 2024-04-07T16:26:09.897 INFO:tasks.rados.rados.0.smithi100.stdout:2914: done (0 left) 2024-04-07T16:26:09.897 INFO:tasks.rados.rados.0.smithi100.stdout:2915: rollback oid 43 current snap is 269 2024-04-07T16:26:09.897 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 43 to 264 2024-04-07T16:26:09.897 INFO:tasks.rados.rados.0.smithi100.stdout:2916: copy_from oid 32 from oid 46 current snap is 269 2024-04-07T16:26:09.897 INFO:tasks.rados.rados.0.smithi100.stdout:2917: copy_from oid 33 from oid 40 current snap is 269 2024-04-07T16:26:09.897 INFO:tasks.rados.rados.0.smithi100.stdout:2918: snap_remove snap 268 2024-04-07T16:26:10.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:09 smithi100 bash[74942]: cluster 2024-04-07T16:26:08.748179+0000 mgr.y (mgr.25175) 288 : cluster [DBG] pgmap v510: 121 pgs: 121 active+clean; 94 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 4.3 MiB/s rd, 1.9 MiB/s wr, 18 op/s 2024-04-07T16:26:10.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:09 smithi100 bash[74942]: cluster 2024-04-07T16:26:08.914334+0000 mon.a (mon.0) 948 : cluster [DBG] osdmap e612: 8 total, 8 up, 8 in 2024-04-07T16:26:10.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:09 smithi100 bash[76202]: cluster 2024-04-07T16:26:08.748179+0000 mgr.y (mgr.25175) 288 : cluster [DBG] pgmap v510: 121 pgs: 121 active+clean; 94 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 4.3 MiB/s rd, 1.9 MiB/s wr, 18 op/s 2024-04-07T16:26:10.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:09 smithi100 bash[76202]: cluster 2024-04-07T16:26:08.914334+0000 mon.a (mon.0) 948 : cluster [DBG] osdmap e612: 8 total, 8 up, 8 in 2024-04-07T16:26:10.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:09 smithi144 bash[66654]: cluster 2024-04-07T16:26:08.748179+0000 mgr.y (mgr.25175) 288 : cluster [DBG] pgmap v510: 121 pgs: 121 active+clean; 94 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 4.3 MiB/s rd, 1.9 MiB/s wr, 18 op/s 2024-04-07T16:26:10.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:09 smithi144 bash[66654]: cluster 2024-04-07T16:26:08.914334+0000 mon.a (mon.0) 948 : cluster [DBG] osdmap e612: 8 total, 8 up, 8 in 2024-04-07T16:26:10.897 INFO:tasks.rados.rados.0.smithi100.stdout:2918: done (3 left) 2024-04-07T16:26:10.897 INFO:tasks.rados.rados.0.smithi100.stdout:2919: delete oid 29 current snap is 269 2024-04-07T16:26:10.897 INFO:tasks.rados.rados.0.smithi100.stdout:2917: finishing copy_from to smithi10066867-33 2024-04-07T16:26:10.897 INFO:tasks.rados.rados.0.smithi100.stdout:2917: got expected ENOENT (src dne) 2024-04-07T16:26:10.897 INFO:tasks.rados.rados.0.smithi100.stdout:2915: finishing rollback tid 0 to smithi10066867-43 2024-04-07T16:26:10.897 INFO:tasks.rados.rados.0.smithi100.stdout:2917: finishing copy_from racing read to smithi10066867-33 2024-04-07T16:26:10.898 INFO:tasks.rados.rados.0.smithi100.stdout:2915: finishing rollback tid 1 to smithi10066867-43 2024-04-07T16:26:10.898 INFO:tasks.rados.rados.0.smithi100.stdout:2915: finishing rollback tid 2 to smithi10066867-43 2024-04-07T16:26:10.898 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 43 v 1144 (ObjNum 886 snap 264 seq_num 886) dirty exists 2024-04-07T16:26:10.898 INFO:tasks.rados.rados.0.smithi100.stdout:2916: finishing copy_from racing read to smithi10066867-32 2024-04-07T16:26:10.898 INFO:tasks.rados.rados.0.smithi100.stdout:2916: finishing copy_from to smithi10066867-32 2024-04-07T16:26:10.898 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 32 v 1117 (ObjNum 1233 snap 0 seq_num 3067285760) dirty exists 2024-04-07T16:26:10.903 INFO:tasks.rados.rados.0.smithi100.stdout:2915: done (3 left) 2024-04-07T16:26:10.903 INFO:tasks.rados.rados.0.smithi100.stdout:2916: done (2 left) 2024-04-07T16:26:10.903 INFO:tasks.rados.rados.0.smithi100.stdout:2917: done (1 left) 2024-04-07T16:26:10.903 INFO:tasks.rados.rados.0.smithi100.stdout:2919: done (0 left) 2024-04-07T16:26:10.903 INFO:tasks.rados.rados.0.smithi100.stdout:2920: delete oid 21 current snap is 269 2024-04-07T16:26:10.908 INFO:tasks.rados.rados.0.smithi100.stdout:2920: done (0 left) 2024-04-07T16:26:10.908 INFO:tasks.rados.rados.0.smithi100.stdout:2921: copy_from oid 13 from oid 17 current snap is 269 2024-04-07T16:26:10.908 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 26 current snap is 269 2024-04-07T16:26:10.908 INFO:tasks.rados.rados.0.smithi100.stdout:2922: seq_num 897 ranges {4063232=630784} 2024-04-07T16:26:10.936 INFO:tasks.rados.rados.0.smithi100.stdout:2922: writing smithi10066867-26 from 4063232 to 4694016 tid 1 2024-04-07T16:26:10.936 INFO:tasks.rados.rados.0.smithi100.stdout:2923: snap_create 2024-04-07T16:26:10.936 INFO:tasks.rados.rados.0.smithi100.stdout:2921: finishing copy_from racing read to smithi10066867-13 2024-04-07T16:26:10.936 INFO:tasks.rados.rados.0.smithi100.stdout:2921: finishing copy_from to smithi10066867-13 2024-04-07T16:26:10.936 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 13 v 1091 (ObjNum 49 snap 0 seq_num 2702997808) dirty exists 2024-04-07T16:26:10.940 INFO:tasks.rados.rados.0.smithi100.stdout:2922: finishing write tid 1 to smithi10066867-26 2024-04-07T16:26:10.940 INFO:tasks.rados.rados.0.smithi100.stdout:2922: finishing write tid 2 to smithi10066867-26 2024-04-07T16:26:10.941 INFO:tasks.rados.rados.0.smithi100.stdout:2922: finishing write tid 3 to smithi10066867-26 2024-04-07T16:26:10.941 INFO:tasks.rados.rados.0.smithi100.stdout:2922: oid 26 updating version 0 to 1207 2024-04-07T16:26:10.941 INFO:tasks.rados.rados.0.smithi100.stdout:2922: oid 26 version 1207 is already newer than 1206 2024-04-07T16:26:10.941 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 26 v 1207 (ObjNum 897 snap 269 seq_num 897) dirty exists 2024-04-07T16:26:10.941 INFO:tasks.rados.rados.0.smithi100.stdout:2922: left oid 26 (ObjNum 897 snap 269 seq_num 897) 2024-04-07T16:26:11.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:10 smithi100 bash[74942]: cluster 2024-04-07T16:26:09.913066+0000 mon.a (mon.0) 949 : cluster [DBG] osdmap e613: 8 total, 8 up, 8 in 2024-04-07T16:26:11.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:10 smithi100 bash[74942]: cluster 2024-04-07T16:26:10.911925+0000 mon.a (mon.0) 950 : cluster [DBG] osdmap e614: 8 total, 8 up, 8 in 2024-04-07T16:26:11.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:10 smithi100 bash[76202]: cluster 2024-04-07T16:26:09.913066+0000 mon.a (mon.0) 949 : cluster [DBG] osdmap e613: 8 total, 8 up, 8 in 2024-04-07T16:26:11.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:10 smithi100 bash[76202]: cluster 2024-04-07T16:26:10.911925+0000 mon.a (mon.0) 950 : cluster [DBG] osdmap e614: 8 total, 8 up, 8 in 2024-04-07T16:26:11.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:10 smithi144 bash[66654]: cluster 2024-04-07T16:26:09.913066+0000 mon.a (mon.0) 949 : cluster [DBG] osdmap e613: 8 total, 8 up, 8 in 2024-04-07T16:26:11.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:10 smithi144 bash[66654]: cluster 2024-04-07T16:26:10.911925+0000 mon.a (mon.0) 950 : cluster [DBG] osdmap e614: 8 total, 8 up, 8 in 2024-04-07T16:26:11.887 INFO:tasks.rados.rados.0.smithi100.stdout:2921: done (2 left) 2024-04-07T16:26:11.887 INFO:tasks.rados.rados.0.smithi100.stdout:2922: done (1 left) 2024-04-07T16:26:11.887 INFO:tasks.rados.rados.0.smithi100.stdout:2923: done (0 left) 2024-04-07T16:26:11.887 INFO:tasks.rados.rados.0.smithi100.stdout:2924: copy_from oid 24 from oid 35 current snap is 270 2024-04-07T16:26:11.887 INFO:tasks.rados.rados.0.smithi100.stdout:2925: read oid 32 snap 269 2024-04-07T16:26:11.887 INFO:tasks.rados.rados.0.smithi100.stdout:2925: expect (ObjNum -1592985728 snap 21978 seq_num 2702530864) 2024-04-07T16:26:11.887 INFO:tasks.rados.rados.0.smithi100.stdout:2926: delete oid 31 current snap is 270 2024-04-07T16:26:11.895 INFO:tasks.rados.rados.0.smithi100.stdout:2925: expect (ObjNum -1592985728 snap 21978 seq_num 2702530864) 2024-04-07T16:26:11.899 INFO:tasks.rados.rados.0.smithi100.stdout:2924: finishing copy_from racing read to smithi10066867-24 2024-04-07T16:26:11.900 INFO:tasks.rados.rados.0.smithi100.stdout:2924: finishing copy_from to smithi10066867-24 2024-04-07T16:26:11.900 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 24 v 1122 (ObjNum 49 snap 0 seq_num 2670636496) dirty exists 2024-04-07T16:26:11.902 INFO:tasks.rados.rados.0.smithi100.stdout:2924: done (2 left) 2024-04-07T16:26:11.902 INFO:tasks.rados.rados.0.smithi100.stdout:2925: done (1 left) 2024-04-07T16:26:11.902 INFO:tasks.rados.rados.0.smithi100.stdout:2926: done (0 left) 2024-04-07T16:26:11.902 INFO:tasks.rados.rados.0.smithi100.stdout:2927: rollback oid 21 current snap is 270 2024-04-07T16:26:11.902 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 21 to 269 2024-04-07T16:26:11.903 INFO:tasks.rados.rados.0.smithi100.stdout:2928: delete oid 18 current snap is 270 2024-04-07T16:26:11.903 INFO:tasks.rados.rados.0.smithi100.stdout:2927: finishing rollback tid 1 to smithi10066867-21 2024-04-07T16:26:11.903 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 21 v 0 (ObjNum 65 snap 0 seq_num 1315133018) dirty dne 2024-04-07T16:26:11.906 INFO:tasks.rados.rados.0.smithi100.stdout:2927: done (1 left) 2024-04-07T16:26:11.906 INFO:tasks.rados.rados.0.smithi100.stdout:2928: done (0 left) 2024-04-07T16:26:11.906 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 42 current snap is 270 2024-04-07T16:26:11.906 INFO:tasks.rados.rados.0.smithi100.stdout:2929: seq_num 900 ranges {0=679936} 2024-04-07T16:26:11.912 INFO:tasks.rados.rados.0.smithi100.stdout:2929: writing smithi10066867-42 from 0 to 679936 tid 1 2024-04-07T16:26:11.912 INFO:tasks.rados.rados.0.smithi100.stdout:2930: snap_remove snap 269 2024-04-07T16:26:12.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:11 smithi100 bash[76202]: cluster 2024-04-07T16:26:10.748861+0000 mgr.y (mgr.25175) 289 : cluster [DBG] pgmap v513: 121 pgs: 121 active+clean; 95 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 9.1 MiB/s rd, 2.0 MiB/s wr, 21 op/s 2024-04-07T16:26:12.185 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:11 smithi100 bash[76202]: cluster 2024-04-07T16:26:11.902248+0000 mon.a (mon.0) 951 : cluster [DBG] osdmap e615: 8 total, 8 up, 8 in 2024-04-07T16:26:12.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:11 smithi100 bash[74942]: cluster 2024-04-07T16:26:10.748861+0000 mgr.y (mgr.25175) 289 : cluster [DBG] pgmap v513: 121 pgs: 121 active+clean; 95 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 9.1 MiB/s rd, 2.0 MiB/s wr, 21 op/s 2024-04-07T16:26:12.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:11 smithi100 bash[74942]: cluster 2024-04-07T16:26:11.902248+0000 mon.a (mon.0) 951 : cluster [DBG] osdmap e615: 8 total, 8 up, 8 in 2024-04-07T16:26:12.304 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:11 smithi144 bash[66654]: cluster 2024-04-07T16:26:10.748861+0000 mgr.y (mgr.25175) 289 : cluster [DBG] pgmap v513: 121 pgs: 121 active+clean; 95 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 9.1 MiB/s rd, 2.0 MiB/s wr, 21 op/s 2024-04-07T16:26:12.304 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:11 smithi144 bash[66654]: cluster 2024-04-07T16:26:11.902248+0000 mon.a (mon.0) 951 : cluster [DBG] osdmap e615: 8 total, 8 up, 8 in 2024-04-07T16:26:12.606 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:26:12 smithi144 bash[59794]: ts=2024-04-07T16:26:12.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:26:12.889 INFO:tasks.rados.rados.0.smithi100.stdout:2930: done (1 left) 2024-04-07T16:26:12.889 INFO:tasks.rados.rados.0.smithi100.stdout:2931: snap_remove snap 264 2024-04-07T16:26:13.890 INFO:tasks.rados.rados.0.smithi100.stdout:2931: done (1 left) 2024-04-07T16:26:13.891 INFO:tasks.rados.rados.0.smithi100.stdout:2932: delete oid 41 current snap is 270 2024-04-07T16:26:13.891 INFO:tasks.rados.rados.0.smithi100.stdout:2929: finishing write tid 1 to smithi10066867-42 2024-04-07T16:26:13.891 INFO:tasks.rados.rados.0.smithi100.stdout:2929: finishing write tid 2 to smithi10066867-42 2024-04-07T16:26:13.891 INFO:tasks.rados.rados.0.smithi100.stdout:2929: finishing write tid 3 to smithi10066867-42 2024-04-07T16:26:13.891 INFO:tasks.rados.rados.0.smithi100.stdout:2929: oid 42 updating version 0 to 1219 2024-04-07T16:26:13.891 INFO:tasks.rados.rados.0.smithi100.stdout:2929: oid 42 version 1219 is already newer than 1218 2024-04-07T16:26:13.891 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 42 v 1219 (ObjNum 900 snap 270 seq_num 900) dirty exists 2024-04-07T16:26:13.891 INFO:tasks.rados.rados.0.smithi100.stdout:2929: left oid 42 (ObjNum 900 snap 270 seq_num 900) 2024-04-07T16:26:13.905 INFO:tasks.rados.rados.0.smithi100.stdout:2929: done (1 left) 2024-04-07T16:26:13.905 INFO:tasks.rados.rados.0.smithi100.stdout:2932: done (0 left) 2024-04-07T16:26:13.905 INFO:tasks.rados.rados.0.smithi100.stdout:2933: rollback oid 18 current snap is 270 2024-04-07T16:26:13.906 INFO:tasks.rados.rados.0.smithi100.stdout:2933: done (0 left) 2024-04-07T16:26:13.906 INFO:tasks.rados.rados.0.smithi100.stdout:2934: rollback oid 19 current snap is 270 2024-04-07T16:26:13.906 INFO:tasks.rados.rados.0.smithi100.stdout:2934: done (0 left) 2024-04-07T16:26:13.906 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 11 current snap is 270 2024-04-07T16:26:13.906 INFO:tasks.rados.rados.0.smithi100.stdout:2935: seq_num 902 ranges {2777088=712704} 2024-04-07T16:26:13.921 INFO:tasks.rados.rados.0.smithi100.stdout:2935: writing smithi10066867-11 from 2777088 to 3489792 tid 1 2024-04-07T16:26:13.921 INFO:tasks.rados.rados.0.smithi100.stdout:2936: read oid 15 snap -1 2024-04-07T16:26:13.922 INFO:tasks.rados.rados.0.smithi100.stdout:2936: expect (ObjNum 776 snap 243 seq_num 776) 2024-04-07T16:26:13.922 INFO:tasks.rados.rados.0.smithi100.stdout:2937: snap_create 2024-04-07T16:26:13.931 INFO:tasks.rados.rados.0.smithi100.stdout:2935: finishing write tid 1 to smithi10066867-11 2024-04-07T16:26:13.941 INFO:tasks.rados.rados.0.smithi100.stdout:2935: finishing write tid 2 to smithi10066867-11 2024-04-07T16:26:13.945 INFO:tasks.rados.rados.0.smithi100.stdout:2936: expect (ObjNum 776 snap 243 seq_num 776) 2024-04-07T16:26:14.140 INFO:tasks.rados.rados.0.smithi100.stdout:2935: finishing write tid 3 to smithi10066867-11 2024-04-07T16:26:14.140 INFO:tasks.rados.rados.0.smithi100.stdout:2935: oid 11 updating version 0 to 1626 2024-04-07T16:26:14.140 INFO:tasks.rados.rados.0.smithi100.stdout:2935: oid 11 version 1626 is already newer than 1625 2024-04-07T16:26:14.140 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 11 v 1626 (ObjNum 902 snap 270 seq_num 902) dirty exists 2024-04-07T16:26:14.140 INFO:tasks.rados.rados.0.smithi100.stdout:2935: left oid 11 (ObjNum 902 snap 270 seq_num 902) 2024-04-07T16:26:14.155 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:13 smithi100 bash[76202]: cluster 2024-04-07T16:26:12.750004+0000 mgr.y (mgr.25175) 290 : cluster [DBG] pgmap v516: 121 pgs: 121 active+clean; 95 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 4.8 MiB/s rd, 244 KiB/s wr, 10 op/s 2024-04-07T16:26:14.155 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:13 smithi100 bash[76202]: cluster 2024-04-07T16:26:12.904637+0000 mon.a (mon.0) 952 : cluster [DBG] osdmap e616: 8 total, 8 up, 8 in 2024-04-07T16:26:14.155 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:13 smithi100 bash[74942]: cluster 2024-04-07T16:26:12.750004+0000 mgr.y (mgr.25175) 290 : cluster [DBG] pgmap v516: 121 pgs: 121 active+clean; 95 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 4.8 MiB/s rd, 244 KiB/s wr, 10 op/s 2024-04-07T16:26:14.155 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:13 smithi100 bash[74942]: cluster 2024-04-07T16:26:12.904637+0000 mon.a (mon.0) 952 : cluster [DBG] osdmap e616: 8 total, 8 up, 8 in 2024-04-07T16:26:14.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:13 smithi144 bash[66654]: cluster 2024-04-07T16:26:12.750004+0000 mgr.y (mgr.25175) 290 : cluster [DBG] pgmap v516: 121 pgs: 121 active+clean; 95 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 4.8 MiB/s rd, 244 KiB/s wr, 10 op/s 2024-04-07T16:26:14.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:13 smithi144 bash[66654]: cluster 2024-04-07T16:26:12.904637+0000 mon.a (mon.0) 952 : cluster [DBG] osdmap e616: 8 total, 8 up, 8 in 2024-04-07T16:26:14.922 INFO:tasks.rados.rados.0.smithi100.stdout:2935: done (2 left) 2024-04-07T16:26:14.923 INFO:tasks.rados.rados.0.smithi100.stdout:2936: done (1 left) 2024-04-07T16:26:14.923 INFO:tasks.rados.rados.0.smithi100.stdout:2937: done (0 left) 2024-04-07T16:26:14.923 INFO:tasks.rados.rados.0.smithi100.stdout:2938: snap_create 2024-04-07T16:26:15.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:14 smithi100 bash[74942]: cluster 2024-04-07T16:26:13.904545+0000 mon.a (mon.0) 953 : cluster [DBG] osdmap e617: 8 total, 8 up, 8 in 2024-04-07T16:26:15.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:14 smithi100 bash[76202]: cluster 2024-04-07T16:26:13.904545+0000 mon.a (mon.0) 953 : cluster [DBG] osdmap e617: 8 total, 8 up, 8 in 2024-04-07T16:26:15.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:14 smithi144 bash[66654]: cluster 2024-04-07T16:26:13.904545+0000 mon.a (mon.0) 953 : cluster [DBG] osdmap e617: 8 total, 8 up, 8 in 2024-04-07T16:26:15.921 INFO:tasks.rados.rados.0.smithi100.stdout:2938: done (0 left) 2024-04-07T16:26:15.921 INFO:tasks.rados.rados.0.smithi100.stdout:2939: snap_remove snap 270 2024-04-07T16:26:16.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:15 smithi100 bash[74942]: cluster 2024-04-07T16:26:14.750864+0000 mgr.y (mgr.25175) 291 : cluster [DBG] pgmap v519: 121 pgs: 121 active+clean; 95 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 14 KiB/s rd, 320 KiB/s wr, 5 op/s 2024-04-07T16:26:16.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:15 smithi100 bash[74942]: audit 2024-04-07T16:26:14.910686+0000 mon.a (mon.0) 954 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:26:16.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:15 smithi100 bash[74942]: audit 2024-04-07T16:26:14.937791+0000 mon.a (mon.0) 955 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:26:16.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:15 smithi100 bash[74942]: cluster 2024-04-07T16:26:14.939237+0000 mon.a (mon.0) 956 : cluster [DBG] osdmap e618: 8 total, 8 up, 8 in 2024-04-07T16:26:16.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:15 smithi100 bash[74942]: audit 2024-04-07T16:26:14.940320+0000 mon.a (mon.0) 957 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:26:16.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:15 smithi100 bash[74942]: audit 2024-04-07T16:26:14.941919+0000 mon.a (mon.0) 958 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:26:16.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:15 smithi100 bash[74942]: audit 2024-04-07T16:26:14.955929+0000 mon.a (mon.0) 959 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:26:16.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:15 smithi100 bash[74942]: audit 2024-04-07T16:26:15.050367+0000 mon.a (mon.0) 960 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:26:16.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:15 smithi100 bash[74942]: audit 2024-04-07T16:26:15.055640+0000 mon.a (mon.0) 961 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:26:16.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:15 smithi100 bash[74942]: audit 2024-04-07T16:26:15.059875+0000 mon.a (mon.0) 962 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:26:16.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:15 smithi100 bash[74942]: audit 2024-04-07T16:26:15.062967+0000 mon.a (mon.0) 963 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:26:16.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:15 smithi100 bash[74942]: audit 2024-04-07T16:26:15.068659+0000 mon.a (mon.0) 964 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:26:16.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:15 smithi100 bash[74942]: audit 2024-04-07T16:26:15.075460+0000 mon.a (mon.0) 965 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mds ok-to-stop", "ids": ["b"]}]: dispatch 2024-04-07T16:26:16.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:15 smithi100 bash[74942]: cephadm 2024-04-07T16:26:15.076905+0000 mgr.y (mgr.25175) 292 : cephadm [INF] Upgrade: It appears safe to stop mds.b 2024-04-07T16:26:16.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:15 smithi100 bash[76202]: cluster 2024-04-07T16:26:14.750864+0000 mgr.y (mgr.25175) 291 : cluster [DBG] pgmap v519: 121 pgs: 121 active+clean; 95 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 14 KiB/s rd, 320 KiB/s wr, 5 op/s 2024-04-07T16:26:16.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:15 smithi100 bash[76202]: audit 2024-04-07T16:26:14.910686+0000 mon.a (mon.0) 954 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:26:16.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:15 smithi100 bash[76202]: audit 2024-04-07T16:26:14.937791+0000 mon.a (mon.0) 955 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:26:16.188 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:15 smithi100 bash[76202]: cluster 2024-04-07T16:26:14.939237+0000 mon.a (mon.0) 956 : cluster [DBG] osdmap e618: 8 total, 8 up, 8 in 2024-04-07T16:26:16.188 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:15 smithi100 bash[76202]: audit 2024-04-07T16:26:14.940320+0000 mon.a (mon.0) 957 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:26:16.188 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:15 smithi100 bash[76202]: audit 2024-04-07T16:26:14.941919+0000 mon.a (mon.0) 958 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:26:16.188 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:15 smithi100 bash[76202]: audit 2024-04-07T16:26:14.955929+0000 mon.a (mon.0) 959 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:26:16.188 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:15 smithi100 bash[76202]: audit 2024-04-07T16:26:15.050367+0000 mon.a (mon.0) 960 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:26:16.188 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:15 smithi100 bash[76202]: audit 2024-04-07T16:26:15.055640+0000 mon.a (mon.0) 961 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:26:16.188 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:15 smithi100 bash[76202]: audit 2024-04-07T16:26:15.059875+0000 mon.a (mon.0) 962 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:26:16.188 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:15 smithi100 bash[76202]: audit 2024-04-07T16:26:15.062967+0000 mon.a (mon.0) 963 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:26:16.188 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:15 smithi100 bash[76202]: audit 2024-04-07T16:26:15.068659+0000 mon.a (mon.0) 964 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:26:16.188 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:15 smithi100 bash[76202]: audit 2024-04-07T16:26:15.075460+0000 mon.a (mon.0) 965 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mds ok-to-stop", "ids": ["b"]}]: dispatch 2024-04-07T16:26:16.188 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:15 smithi100 bash[76202]: cephadm 2024-04-07T16:26:15.076905+0000 mgr.y (mgr.25175) 292 : cephadm [INF] Upgrade: It appears safe to stop mds.b 2024-04-07T16:26:16.338 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:15 smithi144 bash[66654]: cluster 2024-04-07T16:26:14.750864+0000 mgr.y (mgr.25175) 291 : cluster [DBG] pgmap v519: 121 pgs: 121 active+clean; 95 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 14 KiB/s rd, 320 KiB/s wr, 5 op/s 2024-04-07T16:26:16.339 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:15 smithi144 bash[66654]: audit 2024-04-07T16:26:14.910686+0000 mon.a (mon.0) 954 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:26:16.339 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:15 smithi144 bash[66654]: audit 2024-04-07T16:26:14.937791+0000 mon.a (mon.0) 955 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:26:16.339 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:15 smithi144 bash[66654]: cluster 2024-04-07T16:26:14.939237+0000 mon.a (mon.0) 956 : cluster [DBG] osdmap e618: 8 total, 8 up, 8 in 2024-04-07T16:26:16.339 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:15 smithi144 bash[66654]: audit 2024-04-07T16:26:14.940320+0000 mon.a (mon.0) 957 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:26:16.339 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:15 smithi144 bash[66654]: audit 2024-04-07T16:26:14.941919+0000 mon.a (mon.0) 958 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:26:16.339 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:15 smithi144 bash[66654]: audit 2024-04-07T16:26:14.955929+0000 mon.a (mon.0) 959 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:26:16.339 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:15 smithi144 bash[66654]: audit 2024-04-07T16:26:15.050367+0000 mon.a (mon.0) 960 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:26:16.339 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:15 smithi144 bash[66654]: audit 2024-04-07T16:26:15.055640+0000 mon.a (mon.0) 961 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:26:16.339 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:15 smithi144 bash[66654]: audit 2024-04-07T16:26:15.059875+0000 mon.a (mon.0) 962 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:26:16.339 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:15 smithi144 bash[66654]: audit 2024-04-07T16:26:15.062967+0000 mon.a (mon.0) 963 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:26:16.339 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:15 smithi144 bash[66654]: audit 2024-04-07T16:26:15.068659+0000 mon.a (mon.0) 964 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:26:16.339 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:15 smithi144 bash[66654]: audit 2024-04-07T16:26:15.075460+0000 mon.a (mon.0) 965 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mds ok-to-stop", "ids": ["b"]}]: dispatch 2024-04-07T16:26:16.340 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:15 smithi144 bash[66654]: cephadm 2024-04-07T16:26:15.076905+0000 mgr.y (mgr.25175) 292 : cephadm [INF] Upgrade: It appears safe to stop mds.b 2024-04-07T16:26:16.897 INFO:tasks.rados.rados.0.smithi100.stdout:2939: done (0 left) 2024-04-07T16:26:16.898 INFO:tasks.rados.rados.0.smithi100.stdout:2940: copy_from oid 14 from oid 19 current snap is 272 2024-04-07T16:26:16.898 INFO:tasks.rados.rados.0.smithi100.stdout:2941: rollback oid 13 current snap is 272 2024-04-07T16:26:16.898 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 13 to 271 2024-04-07T16:26:16.898 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 3 current snap is 272 2024-04-07T16:26:16.898 INFO:tasks.rados.rados.0.smithi100.stdout:2942: seq_num 903 ranges {2572288=729088} 2024-04-07T16:26:16.930 INFO:tasks.rados.rados.0.smithi100.stdout:2942: writing smithi10066867-3 from 2572288 to 3301376 tid 1 2024-04-07T16:26:16.930 INFO:tasks.rados.rados.0.smithi100.stdout:2943: delete oid 43 current snap is 272 2024-04-07T16:26:16.930 INFO:tasks.rados.rados.0.smithi100.stdout:2940: finishing copy_from to smithi10066867-14 2024-04-07T16:26:16.930 INFO:tasks.rados.rados.0.smithi100.stdout:2940: got expected ENOENT (src dne) 2024-04-07T16:26:16.930 INFO:tasks.rados.rados.0.smithi100.stdout:2940: finishing copy_from racing read to smithi10066867-14 2024-04-07T16:26:16.930 INFO:tasks.rados.rados.0.smithi100.stdout:2941: finishing rollback tid 0 to smithi10066867-13 2024-04-07T16:26:16.930 INFO:tasks.rados.rados.0.smithi100.stdout:2941: finishing rollback tid 1 to smithi10066867-13 2024-04-07T16:26:16.930 INFO:tasks.rados.rados.0.smithi100.stdout:2941: finishing rollback tid 2 to smithi10066867-13 2024-04-07T16:26:16.930 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 13 v 1512 (ObjNum 305 snap 0 seq_num 3067284992) dirty exists 2024-04-07T16:26:16.933 INFO:tasks.rados.rados.0.smithi100.stdout:2940: done (3 left) 2024-04-07T16:26:16.933 INFO:tasks.rados.rados.0.smithi100.stdout:2941: done (2 left) 2024-04-07T16:26:16.934 INFO:tasks.rados.rados.0.smithi100.stdout:2943: done (1 left) 2024-04-07T16:26:16.934 INFO:tasks.rados.rados.0.smithi100.stdout:2944: snap_create 2024-04-07T16:26:16.941 INFO:tasks.rados.rados.0.smithi100.stdout:2942: finishing write tid 1 to smithi10066867-3 2024-04-07T16:26:16.943 INFO:tasks.rados.rados.0.smithi100.stdout:2942: finishing write tid 2 to smithi10066867-3 2024-04-07T16:26:16.945 INFO:tasks.rados.rados.0.smithi100.stdout:2942: finishing write tid 3 to smithi10066867-3 2024-04-07T16:26:16.945 INFO:tasks.rados.rados.0.smithi100.stdout:2942: oid 3 updating version 0 to 1127 2024-04-07T16:26:16.945 INFO:tasks.rados.rados.0.smithi100.stdout:2942: oid 3 updating version 1127 to 1128 2024-04-07T16:26:16.945 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 3 v 1128 (ObjNum 903 snap 272 seq_num 903) dirty exists 2024-04-07T16:26:16.945 INFO:tasks.rados.rados.0.smithi100.stdout:2942: left oid 3 (ObjNum 903 snap 272 seq_num 903) 2024-04-07T16:26:17.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:16 smithi144 bash[66654]: cluster 2024-04-07T16:26:15.936972+0000 mon.a (mon.0) 966 : cluster [DBG] osdmap e619: 8 total, 8 up, 8 in 2024-04-07T16:26:17.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:16 smithi144 bash[66654]: audit 2024-04-07T16:26:16.718709+0000 mon.a (mon.0) 967 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:26:17.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:16 smithi144 bash[66654]: cluster 2024-04-07T16:26:16.904849+0000 mon.a (mon.0) 968 : cluster [DBG] osdmap e620: 8 total, 8 up, 8 in 2024-04-07T16:26:17.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:16 smithi100 bash[74942]: cluster 2024-04-07T16:26:15.936972+0000 mon.a (mon.0) 966 : cluster [DBG] osdmap e619: 8 total, 8 up, 8 in 2024-04-07T16:26:17.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:16 smithi100 bash[74942]: audit 2024-04-07T16:26:16.718709+0000 mon.a (mon.0) 967 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:26:17.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:16 smithi100 bash[74942]: cluster 2024-04-07T16:26:16.904849+0000 mon.a (mon.0) 968 : cluster [DBG] osdmap e620: 8 total, 8 up, 8 in 2024-04-07T16:26:17.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:16 smithi100 bash[76202]: cluster 2024-04-07T16:26:15.936972+0000 mon.a (mon.0) 966 : cluster [DBG] osdmap e619: 8 total, 8 up, 8 in 2024-04-07T16:26:17.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:16 smithi100 bash[76202]: audit 2024-04-07T16:26:16.718709+0000 mon.a (mon.0) 967 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:26:17.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:16 smithi100 bash[76202]: cluster 2024-04-07T16:26:16.904849+0000 mon.a (mon.0) 968 : cluster [DBG] osdmap e620: 8 total, 8 up, 8 in 2024-04-07T16:26:17.947 INFO:tasks.rados.rados.0.smithi100.stdout:2942: done (1 left) 2024-04-07T16:26:17.947 INFO:tasks.rados.rados.0.smithi100.stdout:2944: done (0 left) 2024-04-07T16:26:17.947 INFO:tasks.rados.rados.0.smithi100.stdout:2945: read oid 11 snap -1 2024-04-07T16:26:17.947 INFO:tasks.rados.rados.0.smithi100.stdout:2945: expect (ObjNum 902 snap 270 seq_num 902) 2024-04-07T16:26:17.947 INFO:tasks.rados.rados.0.smithi100.stdout:2946: snap_create 2024-04-07T16:26:17.968 INFO:tasks.rados.rados.0.smithi100.stdout:2945: expect (ObjNum 902 snap 270 seq_num 902) 2024-04-07T16:26:18.464 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:18 smithi144 bash[66654]: cluster 2024-04-07T16:26:16.762704+0000 mgr.y (mgr.25175) 293 : cluster [DBG] pgmap v522: 121 pgs: 121 active+clean; 95 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 510 B/s rd, 319 KiB/s wr, 4 op/s 2024-04-07T16:26:18.464 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:18 smithi144 bash[66654]: cephadm 2024-04-07T16:26:17.204384+0000 mgr.y (mgr.25175) 294 : cephadm [INF] Upgrade: Updating mds.b 2024-04-07T16:26:18.464 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:18 smithi144 bash[66654]: audit 2024-04-07T16:26:17.211636+0000 mon.a (mon.0) 969 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:26:18.464 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:18 smithi144 bash[66654]: audit 2024-04-07T16:26:17.216643+0000 mon.a (mon.0) 970 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-07T16:26:18.464 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:18 smithi144 bash[66654]: audit 2024-04-07T16:26:17.218401+0000 mon.a (mon.0) 971 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:26:18.464 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:18 smithi144 bash[66654]: cephadm 2024-04-07T16:26:17.220399+0000 mgr.y (mgr.25175) 295 : cephadm [INF] Deploying daemon mds.b on smithi144 2024-04-07T16:26:18.464 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:18 smithi144 bash[66654]: cluster 2024-04-07T16:26:17.962387+0000 mon.a (mon.0) 972 : cluster [DBG] osdmap e621: 8 total, 8 up, 8 in 2024-04-07T16:26:18.685 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:26:18 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:26:18] "GET /metrics HTTP/1.1" 200 36338 "" "Prometheus/2.43.0" 2024-04-07T16:26:18.686 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:18 smithi100 bash[74942]: cluster 2024-04-07T16:26:16.762704+0000 mgr.y (mgr.25175) 293 : cluster [DBG] pgmap v522: 121 pgs: 121 active+clean; 95 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 510 B/s rd, 319 KiB/s wr, 4 op/s 2024-04-07T16:26:18.686 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:18 smithi100 bash[74942]: cephadm 2024-04-07T16:26:17.204384+0000 mgr.y (mgr.25175) 294 : cephadm [INF] Upgrade: Updating mds.b 2024-04-07T16:26:18.686 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:18 smithi100 bash[74942]: audit 2024-04-07T16:26:17.211636+0000 mon.a (mon.0) 969 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:26:18.686 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:18 smithi100 bash[74942]: audit 2024-04-07T16:26:17.216643+0000 mon.a (mon.0) 970 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-07T16:26:18.686 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:18 smithi100 bash[74942]: audit 2024-04-07T16:26:17.218401+0000 mon.a (mon.0) 971 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:26:18.686 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:18 smithi100 bash[74942]: cephadm 2024-04-07T16:26:17.220399+0000 mgr.y (mgr.25175) 295 : cephadm [INF] Deploying daemon mds.b on smithi144 2024-04-07T16:26:18.686 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:18 smithi100 bash[74942]: cluster 2024-04-07T16:26:17.962387+0000 mon.a (mon.0) 972 : cluster [DBG] osdmap e621: 8 total, 8 up, 8 in 2024-04-07T16:26:18.687 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:18 smithi100 bash[76202]: cluster 2024-04-07T16:26:16.762704+0000 mgr.y (mgr.25175) 293 : cluster [DBG] pgmap v522: 121 pgs: 121 active+clean; 95 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 510 B/s rd, 319 KiB/s wr, 4 op/s 2024-04-07T16:26:18.687 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:18 smithi100 bash[76202]: cephadm 2024-04-07T16:26:17.204384+0000 mgr.y (mgr.25175) 294 : cephadm [INF] Upgrade: Updating mds.b 2024-04-07T16:26:18.687 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:18 smithi100 bash[76202]: audit 2024-04-07T16:26:17.211636+0000 mon.a (mon.0) 969 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:26:18.687 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:18 smithi100 bash[76202]: audit 2024-04-07T16:26:17.216643+0000 mon.a (mon.0) 970 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-07T16:26:18.687 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:18 smithi100 bash[76202]: audit 2024-04-07T16:26:17.218401+0000 mon.a (mon.0) 971 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:26:18.687 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:18 smithi100 bash[76202]: cephadm 2024-04-07T16:26:17.220399+0000 mgr.y (mgr.25175) 295 : cephadm [INF] Deploying daemon mds.b on smithi144 2024-04-07T16:26:18.687 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:18 smithi100 bash[76202]: cluster 2024-04-07T16:26:17.962387+0000 mon.a (mon.0) 972 : cluster [DBG] osdmap e621: 8 total, 8 up, 8 in 2024-04-07T16:26:18.954 INFO:tasks.rados.rados.0.smithi100.stdout:2945: done (1 left) 2024-04-07T16:26:18.954 INFO:tasks.rados.rados.0.smithi100.stdout:2946: done (0 left) 2024-04-07T16:26:18.954 INFO:tasks.rados.rados.0.smithi100.stdout:2947: copy_from oid 48 from oid 43 current snap is 274 2024-04-07T16:26:18.954 INFO:tasks.rados.rados.0.smithi100.stdout:2948: setattr oid 7 current snap is 274 2024-04-07T16:26:18.956 INFO:tasks.rados.rados.0.smithi100.stdout:2949: read oid 36 snap 272 2024-04-07T16:26:18.956 INFO:tasks.rados.rados.0.smithi100.stdout:2949: expect (ObjNum 830 snap 260 seq_num 830) 2024-04-07T16:26:18.956 INFO:tasks.rados.rados.0.smithi100.stdout:2950: read oid 25 snap -1 2024-04-07T16:26:18.956 INFO:tasks.rados.rados.0.smithi100.stdout:2950: expect (ObjNum 784 snap 246 seq_num 784) 2024-04-07T16:26:18.956 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 44 current snap is 274 2024-04-07T16:26:18.956 INFO:tasks.rados.rados.0.smithi100.stdout:2951: seq_num 905 ranges {360448=761856} 2024-04-07T16:26:18.977 INFO:tasks.rados.rados.0.smithi100.stdout:2951: writing smithi10066867-44 from 360448 to 1122304 tid 1 2024-04-07T16:26:18.977 INFO:tasks.rados.rados.0.smithi100.stdout:2949: expect (ObjNum 830 snap 260 seq_num 830) 2024-04-07T16:26:19.030 INFO:tasks.rados.rados.0.smithi100.stdout:2947: finishing copy_from to smithi10066867-48 2024-04-07T16:26:19.031 INFO:tasks.rados.rados.0.smithi100.stdout:2947: got expected ENOENT (src dne) 2024-04-07T16:26:19.031 INFO:tasks.rados.rados.0.smithi100.stdout:2947: finishing copy_from racing read to smithi10066867-48 2024-04-07T16:26:19.031 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 7 v 1146 (ObjNum 385 snap 0 seq_num 2702521712) dirty exists 2024-04-07T16:26:19.031 INFO:tasks.rados.rados.0.smithi100.stdout:2951: finishing write tid 1 to smithi10066867-44 2024-04-07T16:26:19.031 INFO:tasks.rados.rados.0.smithi100.stdout:2951: finishing write tid 2 to smithi10066867-44 2024-04-07T16:26:19.031 INFO:tasks.rados.rados.0.smithi100.stdout:2950: expect (ObjNum 784 snap 246 seq_num 784) 2024-04-07T16:26:19.366 INFO:tasks.rados.rados.0.smithi100.stdout:2951: finishing write tid 3 to smithi10066867-44 2024-04-07T16:26:19.366 INFO:tasks.rados.rados.0.smithi100.stdout:2951: oid 44 updating version 0 to 1298 2024-04-07T16:26:19.367 INFO:tasks.rados.rados.0.smithi100.stdout:2951: oid 44 version 1298 is already newer than 1297 2024-04-07T16:26:19.367 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 44 v 1298 (ObjNum 905 snap 274 seq_num 905) dirty exists 2024-04-07T16:26:19.367 INFO:tasks.rados.rados.0.smithi100.stdout:2951: left oid 44 (ObjNum 905 snap 274 seq_num 905) 2024-04-07T16:26:19.367 INFO:tasks.rados.rados.0.smithi100.stdout:2947: done (4 left) 2024-04-07T16:26:19.367 INFO:tasks.rados.rados.0.smithi100.stdout:2948: done (3 left) 2024-04-07T16:26:19.367 INFO:tasks.rados.rados.0.smithi100.stdout:2949: done (2 left) 2024-04-07T16:26:19.367 INFO:tasks.rados.rados.0.smithi100.stdout:2950: done (1 left) 2024-04-07T16:26:19.367 INFO:tasks.rados.rados.0.smithi100.stdout:2951: done (0 left) 2024-04-07T16:26:19.367 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 22 current snap is 274 2024-04-07T16:26:19.367 INFO:tasks.rados.rados.0.smithi100.stdout:2952: seq_num 906 ranges {6356992=778240} 2024-04-07T16:26:19.436 INFO:tasks.rados.rados.0.smithi100.stdout:2952: writing smithi10066867-22 from 6356992 to 7135232 tid 1 2024-04-07T16:26:19.436 INFO:tasks.rados.rados.0.smithi100.stdout:2953: rmattr oid 45 current snap is 274 2024-04-07T16:26:19.437 INFO:tasks.rados.rados.0.smithi100.stdout:2954: rmattr oid 12 current snap is 274 2024-04-07T16:26:19.437 INFO:tasks.rados.rados.0.smithi100.stdout:2955: read oid 47 snap 272 2024-04-07T16:26:19.437 INFO:tasks.rados.rados.0.smithi100.stdout:2955: expect deleted 2024-04-07T16:26:19.437 INFO:tasks.rados.rados.0.smithi100.stdout:2956: delete oid 29 current snap is 274 2024-04-07T16:26:19.441 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 45 v 1196 (ObjNum 161 snap 0 seq_num 1) dirty exists 2024-04-07T16:26:19.441 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:19 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:26:19.442 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:26:19 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:26:19.442 INFO:journalctl@ceph.osd.4.smithi144.stdout:Apr 07 16:26:19 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:26:19.442 INFO:journalctl@ceph.osd.6.smithi144.stdout:Apr 07 16:26:19 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:26:19.443 INFO:journalctl@ceph.osd.7.smithi144.stdout:Apr 07 16:26:19 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:26:19.443 INFO:journalctl@ceph.mds.b.smithi144.stdout:Apr 07 16:26:19 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:26:19.443 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:26:19 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:26:19.444 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 16:26:19 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:26:19.444 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:26:19 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:26:19.444 INFO:journalctl@ceph.osd.5.smithi144.stdout:Apr 07 16:26:19 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:26:19.445 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 12 v 1415 (ObjNum 829 snap 260 seq_num 829) dirty exists 2024-04-07T16:26:19.445 INFO:tasks.rados.rados.0.smithi100.stdout:2953: done (4 left) 2024-04-07T16:26:19.445 INFO:tasks.rados.rados.0.smithi100.stdout:2954: done (3 left) 2024-04-07T16:26:19.445 INFO:tasks.rados.rados.0.smithi100.stdout:2955: done (2 left) 2024-04-07T16:26:19.445 INFO:tasks.rados.rados.0.smithi100.stdout:2956: done (1 left) 2024-04-07T16:26:19.445 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 7 current snap is 274 2024-04-07T16:26:19.445 INFO:tasks.rados.rados.0.smithi100.stdout:2957: seq_num 908 ranges {0=409600,409600=401408} 2024-04-07T16:26:19.446 INFO:tasks.rados.rados.0.smithi100.stdout:2957: writing smithi10066867-7 from 0 to 409600 tid 1 2024-04-07T16:26:19.450 INFO:tasks.rados.rados.0.smithi100.stdout:2957: writing smithi10066867-7 from 409600 to 811008 tid 2 2024-04-07T16:26:19.450 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 11 current snap is 274 2024-04-07T16:26:19.450 INFO:tasks.rados.rados.0.smithi100.stdout:2952: finishing write tid 1 to smithi10066867-22 2024-04-07T16:26:19.450 INFO:tasks.rados.rados.0.smithi100.stdout:2952: finishing write tid 2 to smithi10066867-22 2024-04-07T16:26:19.450 INFO:tasks.rados.rados.0.smithi100.stdout:2952: finishing write tid 3 to smithi10066867-22 2024-04-07T16:26:19.450 INFO:tasks.rados.rados.0.smithi100.stdout:2952: oid 22 updating version 0 to 1208 2024-04-07T16:26:19.451 INFO:tasks.rados.rados.0.smithi100.stdout:2952: oid 22 updating version 1208 to 1209 2024-04-07T16:26:19.451 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 22 v 1209 (ObjNum 906 snap 274 seq_num 906) dirty exists 2024-04-07T16:26:19.451 INFO:tasks.rados.rados.0.smithi100.stdout:2952: left oid 22 (ObjNum 906 snap 274 seq_num 906) 2024-04-07T16:26:19.451 INFO:tasks.rados.rados.0.smithi100.stdout:2958: seq_num 909 ranges {3489792=425984,3915776=401408} 2024-04-07T16:26:19.481 INFO:tasks.rados.rados.0.smithi100.stdout:2958: writing smithi10066867-11 from 3489792 to 3915776 tid 1 2024-04-07T16:26:19.484 INFO:tasks.rados.rados.0.smithi100.stdout:2958: writing smithi10066867-11 from 3915776 to 4317184 tid 2 2024-04-07T16:26:19.484 INFO:tasks.rados.rados.0.smithi100.stdout:2952: done (2 left) 2024-04-07T16:26:19.485 INFO:tasks.rados.rados.0.smithi100.stdout:2959: read oid 20 snap 271 2024-04-07T16:26:19.485 INFO:tasks.rados.rados.0.smithi100.stdout:2959: expect (ObjNum -1592226208 snap 21978 seq_num 2704731920) 2024-04-07T16:26:19.485 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 21 current snap is 274 2024-04-07T16:26:19.485 INFO:tasks.rados.rados.0.smithi100.stdout:2957: finishing write tid 1 to smithi10066867-7 2024-04-07T16:26:19.485 INFO:tasks.rados.rados.0.smithi100.stdout:2957: finishing write tid 2 to smithi10066867-7 2024-04-07T16:26:19.485 INFO:tasks.rados.rados.0.smithi100.stdout:2957: finishing write tid 3 to smithi10066867-7 2024-04-07T16:26:19.485 INFO:tasks.rados.rados.0.smithi100.stdout:2957: finishing write tid 4 to smithi10066867-7 2024-04-07T16:26:19.485 INFO:tasks.rados.rados.0.smithi100.stdout:2957: oid 7 updating version 0 to 1147 2024-04-07T16:26:19.485 INFO:tasks.rados.rados.0.smithi100.stdout:2957: oid 7 updating version 1147 to 1148 2024-04-07T16:26:19.485 INFO:tasks.rados.rados.0.smithi100.stdout:2957: oid 7 updating version 1148 to 1149 2024-04-07T16:26:19.485 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 7 v 1149 (ObjNum 908 snap 274 seq_num 908) dirty exists 2024-04-07T16:26:19.485 INFO:tasks.rados.rados.0.smithi100.stdout:2957: left oid 7 (ObjNum 908 snap 274 seq_num 908) 2024-04-07T16:26:19.485 INFO:tasks.rados.rados.0.smithi100.stdout:2960: seq_num 910 ranges {0=442368,442368=401408} 2024-04-07T16:26:19.488 INFO:tasks.rados.rados.0.smithi100.stdout:2960: writing smithi10066867-21 from 0 to 442368 tid 1 2024-04-07T16:26:19.491 INFO:tasks.rados.rados.0.smithi100.stdout:2960: writing smithi10066867-21 from 442368 to 843776 tid 2 2024-04-07T16:26:19.491 INFO:tasks.rados.rados.0.smithi100.stdout:2957: done (3 left) 2024-04-07T16:26:19.491 INFO:tasks.rados.rados.0.smithi100.stdout:2961: rollback oid 10 current snap is 274 2024-04-07T16:26:19.491 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 10 to 272 2024-04-07T16:26:19.491 INFO:tasks.rados.rados.0.smithi100.stdout:2959: expect (ObjNum -1592226208 snap 21978 seq_num 2704731920) 2024-04-07T16:26:19.500 INFO:tasks.rados.rados.0.smithi100.stdout:2958: finishing write tid 1 to smithi10066867-11 2024-04-07T16:26:19.500 INFO:tasks.rados.rados.0.smithi100.stdout:2961: finishing rollback tid 1 to smithi10066867-10 2024-04-07T16:26:19.500 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 10 v 0 (ObjNum 33 snap 0 seq_num 2702624672) dirty dne 2024-04-07T16:26:19.500 INFO:tasks.rados.rados.0.smithi100.stdout:2960: finishing write tid 1 to smithi10066867-21 2024-04-07T16:26:19.500 INFO:tasks.rados.rados.0.smithi100.stdout:2959: done (3 left) 2024-04-07T16:26:19.500 INFO:tasks.rados.rados.0.smithi100.stdout:2961: done (2 left) 2024-04-07T16:26:19.500 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 50 current snap is 274 2024-04-07T16:26:19.500 INFO:tasks.rados.rados.0.smithi100.stdout:2962: seq_num 911 ranges {14450688=466944,14917632=401408} 2024-04-07T16:26:19.588 INFO:tasks.rados.rados.0.smithi100.stdout:2962: writing smithi10066867-50 from 14450688 to 14917632 tid 1 2024-04-07T16:26:19.591 INFO:tasks.rados.rados.0.smithi100.stdout:2962: writing smithi10066867-50 from 14917632 to 15319040 tid 2 2024-04-07T16:26:19.591 INFO:tasks.rados.rados.0.smithi100.stdout:2963: delete oid 18 current snap is 274 2024-04-07T16:26:19.591 INFO:tasks.rados.rados.0.smithi100.stdout:2958: finishing write tid 2 to smithi10066867-11 2024-04-07T16:26:19.591 INFO:tasks.rados.rados.0.smithi100.stdout:2960: finishing write tid 2 to smithi10066867-21 2024-04-07T16:26:19.591 INFO:tasks.rados.rados.0.smithi100.stdout:2960: finishing write tid 3 to smithi10066867-21 2024-04-07T16:26:19.591 INFO:tasks.rados.rados.0.smithi100.stdout:2960: finishing write tid 4 to smithi10066867-21 2024-04-07T16:26:19.591 INFO:tasks.rados.rados.0.smithi100.stdout:2960: oid 21 updating version 0 to 1193 2024-04-07T16:26:19.591 INFO:tasks.rados.rados.0.smithi100.stdout:2960: oid 21 updating version 1193 to 1194 2024-04-07T16:26:19.591 INFO:tasks.rados.rados.0.smithi100.stdout:2960: oid 21 updating version 1194 to 1195 2024-04-07T16:26:19.591 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 21 v 1195 (ObjNum 910 snap 274 seq_num 910) dirty exists 2024-04-07T16:26:19.591 INFO:tasks.rados.rados.0.smithi100.stdout:2960: left oid 21 (ObjNum 910 snap 274 seq_num 910) 2024-04-07T16:26:19.592 INFO:tasks.rados.rados.0.smithi100.stdout:2958: finishing write tid 3 to smithi10066867-11 2024-04-07T16:26:19.592 INFO:tasks.rados.rados.0.smithi100.stdout:2958: finishing write tid 4 to smithi10066867-11 2024-04-07T16:26:19.592 INFO:tasks.rados.rados.0.smithi100.stdout:2958: oid 11 updating version 0 to 1631 2024-04-07T16:26:19.592 INFO:tasks.rados.rados.0.smithi100.stdout:2958: oid 11 version 1631 is already newer than 1630 2024-04-07T16:26:19.592 INFO:tasks.rados.rados.0.smithi100.stdout:2958: oid 11 updating version 1631 to 1632 2024-04-07T16:26:19.592 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 11 v 1632 (ObjNum 909 snap 274 seq_num 909) dirty exists 2024-04-07T16:26:19.592 INFO:tasks.rados.rados.0.smithi100.stdout:2958: left oid 11 (ObjNum 909 snap 274 seq_num 909) 2024-04-07T16:26:19.594 INFO:tasks.rados.rados.0.smithi100.stdout:2958: done (3 left) 2024-04-07T16:26:19.594 INFO:tasks.rados.rados.0.smithi100.stdout:2960: done (2 left) 2024-04-07T16:26:19.594 INFO:tasks.rados.rados.0.smithi100.stdout:2963: done (1 left) 2024-04-07T16:26:19.594 INFO:tasks.rados.rados.0.smithi100.stdout:2964: setattr oid 30 current snap is 274 2024-04-07T16:26:19.595 INFO:tasks.rados.rados.0.smithi100.stdout:2962: finishing write tid 1 to smithi10066867-50 2024-04-07T16:26:19.595 INFO:tasks.rados.rados.0.smithi100.stdout:2962: finishing write tid 2 to smithi10066867-50 2024-04-07T16:26:19.595 INFO:tasks.rados.rados.0.smithi100.stdout:2962: finishing write tid 3 to smithi10066867-50 2024-04-07T16:26:19.595 INFO:tasks.rados.rados.0.smithi100.stdout:2965: read oid 40 snap -1 2024-04-07T16:26:19.595 INFO:tasks.rados.rados.0.smithi100.stdout:2965: expect deleted 2024-04-07T16:26:19.596 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 1 current snap is 274 2024-04-07T16:26:19.596 INFO:tasks.rados.rados.0.smithi100.stdout:2966: seq_num 913 ranges {2441216=499712,2940928=401408} 2024-04-07T16:26:19.612 INFO:tasks.rados.rados.0.smithi100.stdout:2966: writing smithi10066867-1 from 2441216 to 2940928 tid 1 2024-04-07T16:26:19.614 INFO:tasks.rados.rados.0.smithi100.stdout:2966: writing smithi10066867-1 from 2940928 to 3342336 tid 2 2024-04-07T16:26:19.615 INFO:tasks.rados.rados.0.smithi100.stdout:2967: snap_create 2024-04-07T16:26:19.615 INFO:tasks.rados.rados.0.smithi100.stdout:2962: finishing write tid 4 to smithi10066867-50 2024-04-07T16:26:19.615 INFO:tasks.rados.rados.0.smithi100.stdout:2962: oid 50 updating version 0 to 1199 2024-04-07T16:26:19.615 INFO:tasks.rados.rados.0.smithi100.stdout:2962: oid 50 version 1199 is already newer than 1198 2024-04-07T16:26:19.615 INFO:tasks.rados.rados.0.smithi100.stdout:2962: oid 50 version 1199 is already newer than 1197 2024-04-07T16:26:19.615 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 50 v 1199 (ObjNum 911 snap 274 seq_num 911) dirty exists 2024-04-07T16:26:19.615 INFO:tasks.rados.rados.0.smithi100.stdout:2962: left oid 50 (ObjNum 911 snap 274 seq_num 911) 2024-04-07T16:26:19.615 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 30 v 1220 (ObjNum 880 snap 264 seq_num 880) dirty exists 2024-04-07T16:26:19.616 INFO:tasks.rados.rados.0.smithi100.stdout:2966: finishing write tid 1 to smithi10066867-1 2024-04-07T16:26:19.617 INFO:tasks.rados.rados.0.smithi100.stdout:2966: finishing write tid 2 to smithi10066867-1 2024-04-07T16:26:19.617 INFO:tasks.rados.rados.0.smithi100.stdout:2966: finishing write tid 3 to smithi10066867-1 2024-04-07T16:26:19.620 INFO:tasks.rados.rados.0.smithi100.stdout:2966: finishing write tid 4 to smithi10066867-1 2024-04-07T16:26:19.620 INFO:tasks.rados.rados.0.smithi100.stdout:2966: oid 1 updating version 0 to 1445 2024-04-07T16:26:19.621 INFO:tasks.rados.rados.0.smithi100.stdout:2966: oid 1 version 1445 is already newer than 1444 2024-04-07T16:26:19.621 INFO:tasks.rados.rados.0.smithi100.stdout:2966: oid 1 updating version 1445 to 1446 2024-04-07T16:26:19.621 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 1 v 1446 (ObjNum 913 snap 274 seq_num 913) dirty exists 2024-04-07T16:26:19.621 INFO:tasks.rados.rados.0.smithi100.stdout:2966: left oid 1 (ObjNum 913 snap 274 seq_num 913) 2024-04-07T16:26:19.856 INFO:journalctl@ceph.mds.b.smithi144.stdout:Apr 07 16:26:19 smithi144 systemd[1]: Stopping Ceph mds.b for 0702e3a8-f4f8-11ee-b649-cb9ed24678a4... 2024-04-07T16:26:19.856 INFO:journalctl@ceph.mds.b.smithi144.stdout:Apr 07 16:26:19 smithi144 bash[47566]: debug 2024-04-07T16:26:19.560+0000 7fd16e3cb700 -1 received signal: Terminated from /sbin/docker-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug (PID: 1) UID: 0 2024-04-07T16:26:19.856 INFO:journalctl@ceph.mds.b.smithi144.stdout:Apr 07 16:26:19 smithi144 bash[47566]: debug 2024-04-07T16:26:19.560+0000 7fd16e3cb700 -1 mds.b *** got signal Terminated *** 2024-04-07T16:26:19.935 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:19 smithi100 bash[74942]: debug 2024-04-07T16:26:19.569+0000 7f53b1d07700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-07T16:26:20.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:19 smithi144 bash[66654]: cluster 2024-04-07T16:26:18.763592+0000 mgr.y (mgr.25175) 296 : cluster [DBG] pgmap v525: 121 pgs: 121 active+clean; 87 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 0 B/s rd, 0 B/s wr, 0 op/s 2024-04-07T16:26:20.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:19 smithi144 bash[66654]: cluster 2024-04-07T16:26:18.968859+0000 mon.a (mon.0) 973 : cluster [DBG] osdmap e622: 8 total, 8 up, 8 in 2024-04-07T16:26:20.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:19 smithi144 bash[66654]: cluster 2024-04-07T16:26:19.574252+0000 mon.a (mon.0) 974 : cluster [WRN] Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-07T16:26:20.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:19 smithi144 bash[66654]: cluster 2024-04-07T16:26:19.574308+0000 mon.a (mon.0) 975 : cluster [ERR] Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-07T16:26:20.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:19 smithi144 bash[66654]: cluster 2024-04-07T16:26:19.582627+0000 mon.a (mon.0) 976 : cluster [DBG] osdmap e623: 8 total, 8 up, 8 in 2024-04-07T16:26:20.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:19 smithi144 bash[66654]: cluster 2024-04-07T16:26:19.583625+0000 mon.a (mon.0) 977 : cluster [INF] Standby daemon mds.a assigned to filesystem cephfs as rank 0 2024-04-07T16:26:20.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:19 smithi144 bash[66654]: cluster 2024-04-07T16:26:19.583733+0000 mon.a (mon.0) 978 : cluster [INF] Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-07T16:26:20.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:19 smithi144 bash[66654]: cluster 2024-04-07T16:26:19.584010+0000 mon.a (mon.0) 979 : cluster [DBG] fsmap cephfs:0/1 1 up:standby, 1 failed 2024-04-07T16:26:20.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:19 smithi144 bash[66654]: cluster 2024-04-07T16:26:19.588326+0000 mon.a (mon.0) 980 : cluster [DBG] fsmap cephfs:1/1 {0=a=up:replay} 2024-04-07T16:26:20.436 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:19 smithi100 bash[74942]: cluster 2024-04-07T16:26:18.763592+0000 mgr.y (mgr.25175) 296 : cluster [DBG] pgmap v525: 121 pgs: 121 active+clean; 87 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 0 B/s rd, 0 B/s wr, 0 op/s 2024-04-07T16:26:20.436 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:19 smithi100 bash[74942]: cluster 2024-04-07T16:26:18.968859+0000 mon.a (mon.0) 973 : cluster [DBG] osdmap e622: 8 total, 8 up, 8 in 2024-04-07T16:26:20.436 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:19 smithi100 bash[74942]: cluster 2024-04-07T16:26:19.574252+0000 mon.a (mon.0) 974 : cluster [WRN] Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-07T16:26:20.436 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:19 smithi100 bash[74942]: cluster 2024-04-07T16:26:19.574308+0000 mon.a (mon.0) 975 : cluster [ERR] Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-07T16:26:20.436 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:19 smithi100 bash[74942]: cluster 2024-04-07T16:26:19.582627+0000 mon.a (mon.0) 976 : cluster [DBG] osdmap e623: 8 total, 8 up, 8 in 2024-04-07T16:26:20.436 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:19 smithi100 bash[74942]: cluster 2024-04-07T16:26:19.583625+0000 mon.a (mon.0) 977 : cluster [INF] Standby daemon mds.a assigned to filesystem cephfs as rank 0 2024-04-07T16:26:20.436 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:19 smithi100 bash[74942]: cluster 2024-04-07T16:26:19.583733+0000 mon.a (mon.0) 978 : cluster [INF] Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-07T16:26:20.436 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:19 smithi100 bash[74942]: cluster 2024-04-07T16:26:19.584010+0000 mon.a (mon.0) 979 : cluster [DBG] fsmap cephfs:0/1 1 up:standby, 1 failed 2024-04-07T16:26:20.436 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:19 smithi100 bash[74942]: cluster 2024-04-07T16:26:19.588326+0000 mon.a (mon.0) 980 : cluster [DBG] fsmap cephfs:1/1 {0=a=up:replay} 2024-04-07T16:26:20.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:19 smithi100 bash[76202]: cluster 2024-04-07T16:26:18.763592+0000 mgr.y (mgr.25175) 296 : cluster [DBG] pgmap v525: 121 pgs: 121 active+clean; 87 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 0 B/s rd, 0 B/s wr, 0 op/s 2024-04-07T16:26:20.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:19 smithi100 bash[76202]: cluster 2024-04-07T16:26:18.968859+0000 mon.a (mon.0) 973 : cluster [DBG] osdmap e622: 8 total, 8 up, 8 in 2024-04-07T16:26:20.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:19 smithi100 bash[76202]: cluster 2024-04-07T16:26:19.574252+0000 mon.a (mon.0) 974 : cluster [WRN] Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-07T16:26:20.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:19 smithi100 bash[76202]: cluster 2024-04-07T16:26:19.574308+0000 mon.a (mon.0) 975 : cluster [ERR] Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-07T16:26:20.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:19 smithi100 bash[76202]: cluster 2024-04-07T16:26:19.582627+0000 mon.a (mon.0) 976 : cluster [DBG] osdmap e623: 8 total, 8 up, 8 in 2024-04-07T16:26:20.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:19 smithi100 bash[76202]: cluster 2024-04-07T16:26:19.583625+0000 mon.a (mon.0) 977 : cluster [INF] Standby daemon mds.a assigned to filesystem cephfs as rank 0 2024-04-07T16:26:20.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:19 smithi100 bash[76202]: cluster 2024-04-07T16:26:19.583733+0000 mon.a (mon.0) 978 : cluster [INF] Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-07T16:26:20.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:19 smithi100 bash[76202]: cluster 2024-04-07T16:26:19.584010+0000 mon.a (mon.0) 979 : cluster [DBG] fsmap cephfs:0/1 1 up:standby, 1 failed 2024-04-07T16:26:20.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:19 smithi100 bash[76202]: cluster 2024-04-07T16:26:19.588326+0000 mon.a (mon.0) 980 : cluster [DBG] fsmap cephfs:1/1 {0=a=up:replay} 2024-04-07T16:26:20.592 INFO:tasks.rados.rados.0.smithi100.stdout:2962: done (4 left) 2024-04-07T16:26:20.592 INFO:tasks.rados.rados.0.smithi100.stdout:2964: done (3 left) 2024-04-07T16:26:20.592 INFO:tasks.rados.rados.0.smithi100.stdout:2965: done (2 left) 2024-04-07T16:26:20.592 INFO:tasks.rados.rados.0.smithi100.stdout:2966: done (1 left) 2024-04-07T16:26:20.592 INFO:tasks.rados.rados.0.smithi100.stdout:2967: done (0 left) 2024-04-07T16:26:20.592 INFO:tasks.rados.rados.0.smithi100.stdout:2968: snap_create 2024-04-07T16:26:21.605 INFO:tasks.rados.rados.0.smithi100.stdout:2968: done (0 left) 2024-04-07T16:26:21.605 INFO:tasks.rados.rados.0.smithi100.stdout:2969: delete oid 14 current snap is 276 2024-04-07T16:26:21.610 INFO:tasks.rados.rados.0.smithi100.stdout:2969: done (0 left) 2024-04-07T16:26:21.610 INFO:tasks.rados.rados.0.smithi100.stdout:2970: snap_create 2024-04-07T16:26:21.856 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:21 smithi144 bash[66654]: cluster 2024-04-07T16:26:20.595007+0000 mon.a (mon.0) 981 : cluster [DBG] mds.? [v2:172.21.15.100:6834/2107704360,v1:172.21.15.100:6835/2107704360] up:reconnect 2024-04-07T16:26:21.856 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:21 smithi144 bash[66654]: cluster 2024-04-07T16:26:20.595168+0000 mon.a (mon.0) 982 : cluster [DBG] fsmap cephfs:1/1 {0=a=up:reconnect} 2024-04-07T16:26:21.856 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:21 smithi144 bash[66654]: cluster 2024-04-07T16:26:20.608624+0000 mon.a (mon.0) 983 : cluster [DBG] osdmap e624: 8 total, 8 up, 8 in 2024-04-07T16:26:21.856 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:21 smithi144 bash[66654]: cluster 2024-04-07T16:26:20.764293+0000 mgr.y (mgr.25175) 297 : cluster [DBG] pgmap v529: 121 pgs: 121 active+clean; 78 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 5.1 MiB/s rd, 385 KiB/s wr, 11 op/s 2024-04-07T16:26:21.892 INFO:tasks.rados.rados.0.smithi100.stdout:2970: done (0 left) 2024-04-07T16:26:21.892 INFO:tasks.rados.rados.0.smithi100.stdout:2971: copy_from oid 50 from oid 8 current snap is 277 2024-04-07T16:26:21.892 INFO:tasks.rados.rados.0.smithi100.stdout:2972: copy_from oid 29 from oid 43 current snap is 277 2024-04-07T16:26:21.892 INFO:tasks.rados.rados.0.smithi100.stdout:2973: read oid 40 snap -1 2024-04-07T16:26:21.892 INFO:tasks.rados.rados.0.smithi100.stdout:2973: expect deleted 2024-04-07T16:26:21.892 INFO:tasks.rados.rados.0.smithi100.stdout:2974: read oid 20 snap -1 2024-04-07T16:26:21.892 INFO:tasks.rados.rados.0.smithi100.stdout:2974: expect (ObjNum 0 snap 0 seq_num 0) 2024-04-07T16:26:21.892 INFO:tasks.rados.rados.0.smithi100.stdout:2975: copy_from oid 24 from oid 22 current snap is 277 2024-04-07T16:26:21.892 INFO:tasks.rados.rados.0.smithi100.stdout:2976: rollback oid 33 current snap is 277 2024-04-07T16:26:21.892 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 33 to 273 2024-04-07T16:26:21.892 INFO:tasks.rados.rados.0.smithi100.stdout:2977: rmattr oid 5 current snap is 277 2024-04-07T16:26:21.893 INFO:tasks.rados.rados.0.smithi100.stdout:2978: snap_remove snap 275 2024-04-07T16:26:21.936 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:21 smithi100 bash[74942]: cluster 2024-04-07T16:26:20.595007+0000 mon.a (mon.0) 981 : cluster [DBG] mds.? [v2:172.21.15.100:6834/2107704360,v1:172.21.15.100:6835/2107704360] up:reconnect 2024-04-07T16:26:21.936 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:21 smithi100 bash[74942]: cluster 2024-04-07T16:26:20.595168+0000 mon.a (mon.0) 982 : cluster [DBG] fsmap cephfs:1/1 {0=a=up:reconnect} 2024-04-07T16:26:21.936 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:21 smithi100 bash[74942]: cluster 2024-04-07T16:26:20.608624+0000 mon.a (mon.0) 983 : cluster [DBG] osdmap e624: 8 total, 8 up, 8 in 2024-04-07T16:26:21.936 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:21 smithi100 bash[74942]: cluster 2024-04-07T16:26:20.764293+0000 mgr.y (mgr.25175) 297 : cluster [DBG] pgmap v529: 121 pgs: 121 active+clean; 78 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 5.1 MiB/s rd, 385 KiB/s wr, 11 op/s 2024-04-07T16:26:21.936 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:21 smithi100 bash[76202]: cluster 2024-04-07T16:26:20.595007+0000 mon.a (mon.0) 981 : cluster [DBG] mds.? [v2:172.21.15.100:6834/2107704360,v1:172.21.15.100:6835/2107704360] up:reconnect 2024-04-07T16:26:21.936 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:21 smithi100 bash[76202]: cluster 2024-04-07T16:26:20.595168+0000 mon.a (mon.0) 982 : cluster [DBG] fsmap cephfs:1/1 {0=a=up:reconnect} 2024-04-07T16:26:21.936 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:21 smithi100 bash[76202]: cluster 2024-04-07T16:26:20.608624+0000 mon.a (mon.0) 983 : cluster [DBG] osdmap e624: 8 total, 8 up, 8 in 2024-04-07T16:26:21.936 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:21 smithi100 bash[76202]: cluster 2024-04-07T16:26:20.764293+0000 mgr.y (mgr.25175) 297 : cluster [DBG] pgmap v529: 121 pgs: 121 active+clean; 78 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 5.1 MiB/s rd, 385 KiB/s wr, 11 op/s 2024-04-07T16:26:22.585 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:26:22 smithi144 bash[59794]: ts=2024-04-07T16:26:22.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:26:22.856 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:22 smithi144 bash[66654]: cluster 2024-04-07T16:26:21.595015+0000 mon.a (mon.0) 984 : cluster [DBG] mds.? [v2:172.21.15.100:6834/2107704360,v1:172.21.15.100:6835/2107704360] up:rejoin 2024-04-07T16:26:22.856 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:22 smithi144 bash[66654]: cluster 2024-04-07T16:26:21.595120+0000 mon.a (mon.0) 985 : cluster [DBG] fsmap cephfs:1/1 {0=a=up:rejoin} 2024-04-07T16:26:22.856 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:22 smithi144 bash[66654]: cluster 2024-04-07T16:26:21.595921+0000 mon.a (mon.0) 986 : cluster [INF] daemon mds.a is now active in filesystem cephfs as rank 0 2024-04-07T16:26:22.856 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:22 smithi144 bash[66654]: cluster 2024-04-07T16:26:21.621428+0000 mon.a (mon.0) 987 : cluster [DBG] osdmap e625: 8 total, 8 up, 8 in 2024-04-07T16:26:22.856 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:22 smithi144 bash[66654]: cluster 2024-04-07T16:26:21.907313+0000 mon.a (mon.0) 988 : cluster [DBG] osdmap e626: 8 total, 8 up, 8 in 2024-04-07T16:26:22.856 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:22 smithi144 bash[66654]: cluster 2024-04-07T16:26:22.589750+0000 mon.a (mon.0) 989 : cluster [INF] Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-07T16:26:22.856 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:22 smithi144 bash[66654]: cluster 2024-04-07T16:26:22.589808+0000 mon.a (mon.0) 990 : cluster [INF] Cluster is now healthy 2024-04-07T16:26:22.896 INFO:tasks.rados.rados.0.smithi100.stdout:2978: done (7 left) 2024-04-07T16:26:22.896 INFO:tasks.rados.rados.0.smithi100.stdout:2979: snap_remove snap 274 2024-04-07T16:26:22.936 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:22 smithi100 bash[76202]: cluster 2024-04-07T16:26:21.595015+0000 mon.a (mon.0) 984 : cluster [DBG] mds.? [v2:172.21.15.100:6834/2107704360,v1:172.21.15.100:6835/2107704360] up:rejoin 2024-04-07T16:26:22.936 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:22 smithi100 bash[76202]: cluster 2024-04-07T16:26:21.595120+0000 mon.a (mon.0) 985 : cluster [DBG] fsmap cephfs:1/1 {0=a=up:rejoin} 2024-04-07T16:26:22.936 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:22 smithi100 bash[76202]: cluster 2024-04-07T16:26:21.595921+0000 mon.a (mon.0) 986 : cluster [INF] daemon mds.a is now active in filesystem cephfs as rank 0 2024-04-07T16:26:22.936 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:22 smithi100 bash[76202]: cluster 2024-04-07T16:26:21.621428+0000 mon.a (mon.0) 987 : cluster [DBG] osdmap e625: 8 total, 8 up, 8 in 2024-04-07T16:26:22.936 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:22 smithi100 bash[76202]: cluster 2024-04-07T16:26:21.907313+0000 mon.a (mon.0) 988 : cluster [DBG] osdmap e626: 8 total, 8 up, 8 in 2024-04-07T16:26:22.936 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:22 smithi100 bash[76202]: cluster 2024-04-07T16:26:22.589750+0000 mon.a (mon.0) 989 : cluster [INF] Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-07T16:26:22.936 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:22 smithi100 bash[76202]: cluster 2024-04-07T16:26:22.589808+0000 mon.a (mon.0) 990 : cluster [INF] Cluster is now healthy 2024-04-07T16:26:22.937 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:22 smithi100 bash[74942]: cluster 2024-04-07T16:26:21.595015+0000 mon.a (mon.0) 984 : cluster [DBG] mds.? [v2:172.21.15.100:6834/2107704360,v1:172.21.15.100:6835/2107704360] up:rejoin 2024-04-07T16:26:22.937 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:22 smithi100 bash[74942]: cluster 2024-04-07T16:26:21.595120+0000 mon.a (mon.0) 985 : cluster [DBG] fsmap cephfs:1/1 {0=a=up:rejoin} 2024-04-07T16:26:22.937 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:22 smithi100 bash[74942]: cluster 2024-04-07T16:26:21.595921+0000 mon.a (mon.0) 986 : cluster [INF] daemon mds.a is now active in filesystem cephfs as rank 0 2024-04-07T16:26:22.937 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:22 smithi100 bash[74942]: cluster 2024-04-07T16:26:21.621428+0000 mon.a (mon.0) 987 : cluster [DBG] osdmap e625: 8 total, 8 up, 8 in 2024-04-07T16:26:22.937 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:22 smithi100 bash[74942]: cluster 2024-04-07T16:26:21.907313+0000 mon.a (mon.0) 988 : cluster [DBG] osdmap e626: 8 total, 8 up, 8 in 2024-04-07T16:26:22.937 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:22 smithi100 bash[74942]: cluster 2024-04-07T16:26:22.589750+0000 mon.a (mon.0) 989 : cluster [INF] Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-07T16:26:22.937 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:22 smithi100 bash[74942]: cluster 2024-04-07T16:26:22.589808+0000 mon.a (mon.0) 990 : cluster [INF] Cluster is now healthy 2024-04-07T16:26:23.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:23 smithi144 bash[66654]: cluster 2024-04-07T16:26:22.597569+0000 mon.a (mon.0) 991 : cluster [DBG] mds.? [v2:172.21.15.100:6834/2107704360,v1:172.21.15.100:6835/2107704360] up:active 2024-04-07T16:26:23.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:23 smithi144 bash[66654]: cluster 2024-04-07T16:26:22.597732+0000 mon.a (mon.0) 992 : cluster [DBG] fsmap cephfs:1 {0=a=up:active} 2024-04-07T16:26:23.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:23 smithi144 bash[66654]: cluster 2024-04-07T16:26:22.765412+0000 mgr.y (mgr.25175) 298 : cluster [DBG] pgmap v532: 121 pgs: 121 active+clean; 82 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 5.3 KiB/s rd, 1.2 MiB/s wr, 15 op/s 2024-04-07T16:26:23.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:23 smithi144 bash[66654]: cluster 2024-04-07T16:26:22.911903+0000 mon.a (mon.0) 993 : cluster [DBG] osdmap e627: 8 total, 8 up, 8 in 2024-04-07T16:26:23.897 INFO:tasks.rados.rados.0.smithi100.stdout:2979: done (7 left) 2024-04-07T16:26:23.897 INFO:tasks.rados.rados.0.smithi100.stdout:2980: read oid 47 snap 271 2024-04-07T16:26:23.897 INFO:tasks.rados.rados.0.smithi100.stdout:2980: expect deleted 2024-04-07T16:26:23.897 INFO:tasks.rados.rados.0.smithi100.stdout:2981: copy_from oid 44 from oid 15 current snap is 277 2024-04-07T16:26:23.897 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 27 current snap is 277 2024-04-07T16:26:23.897 INFO:tasks.rados.rados.0.smithi100.stdout:2982: seq_num 915 ranges {2441216=532480,2973696=401408} 2024-04-07T16:26:23.921 INFO:tasks.rados.rados.0.smithi100.stdout:2982: writing smithi10066867-27 from 2441216 to 2973696 tid 1 2024-04-07T16:26:23.925 INFO:tasks.rados.rados.0.smithi100.stdout:2982: writing smithi10066867-27 from 2973696 to 3375104 tid 2 2024-04-07T16:26:23.925 INFO:tasks.rados.rados.0.smithi100.stdout:2983: snap_remove snap 276 2024-04-07T16:26:23.935 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:23 smithi100 bash[76202]: cluster 2024-04-07T16:26:22.597569+0000 mon.a (mon.0) 991 : cluster [DBG] mds.? [v2:172.21.15.100:6834/2107704360,v1:172.21.15.100:6835/2107704360] up:active 2024-04-07T16:26:23.935 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:23 smithi100 bash[76202]: cluster 2024-04-07T16:26:22.597732+0000 mon.a (mon.0) 992 : cluster [DBG] fsmap cephfs:1 {0=a=up:active} 2024-04-07T16:26:23.935 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:23 smithi100 bash[76202]: cluster 2024-04-07T16:26:22.765412+0000 mgr.y (mgr.25175) 298 : cluster [DBG] pgmap v532: 121 pgs: 121 active+clean; 82 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 5.3 KiB/s rd, 1.2 MiB/s wr, 15 op/s 2024-04-07T16:26:23.935 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:23 smithi100 bash[76202]: cluster 2024-04-07T16:26:22.911903+0000 mon.a (mon.0) 993 : cluster [DBG] osdmap e627: 8 total, 8 up, 8 in 2024-04-07T16:26:23.936 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:23 smithi100 bash[74942]: cluster 2024-04-07T16:26:22.597569+0000 mon.a (mon.0) 991 : cluster [DBG] mds.? [v2:172.21.15.100:6834/2107704360,v1:172.21.15.100:6835/2107704360] up:active 2024-04-07T16:26:23.936 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:23 smithi100 bash[74942]: cluster 2024-04-07T16:26:22.597732+0000 mon.a (mon.0) 992 : cluster [DBG] fsmap cephfs:1 {0=a=up:active} 2024-04-07T16:26:23.936 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:23 smithi100 bash[74942]: cluster 2024-04-07T16:26:22.765412+0000 mgr.y (mgr.25175) 298 : cluster [DBG] pgmap v532: 121 pgs: 121 active+clean; 82 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 5.3 KiB/s rd, 1.2 MiB/s wr, 15 op/s 2024-04-07T16:26:23.936 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:23 smithi100 bash[74942]: cluster 2024-04-07T16:26:22.911903+0000 mon.a (mon.0) 993 : cluster [DBG] osdmap e627: 8 total, 8 up, 8 in 2024-04-07T16:26:24.915 INFO:tasks.rados.rados.0.smithi100.stdout:2983: done (10 left) 2024-04-07T16:26:24.915 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 3 current snap is 277 2024-04-07T16:26:24.915 INFO:tasks.rados.rados.0.smithi100.stdout:2984: seq_num 916 ranges {3301376=548864,3850240=401408} 2024-04-07T16:26:24.941 INFO:tasks.rados.rados.0.smithi100.stdout:2984: writing smithi10066867-3 from 3301376 to 3850240 tid 1 2024-04-07T16:26:24.945 INFO:tasks.rados.rados.0.smithi100.stdout:2984: writing smithi10066867-3 from 3850240 to 4251648 tid 2 2024-04-07T16:26:24.945 INFO:tasks.rados.rados.0.smithi100.stdout:2985: read oid 48 snap 273 2024-04-07T16:26:24.945 INFO:tasks.rados.rados.0.smithi100.stdout:2985: expect (ObjNum 794 snap 251 seq_num 794) 2024-04-07T16:26:24.945 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 4 current snap is 277 2024-04-07T16:26:24.946 INFO:tasks.rados.rados.0.smithi100.stdout:2986: seq_num 917 ranges {0=565248,565248=401408} 2024-04-07T16:26:24.949 INFO:tasks.rados.rados.0.smithi100.stdout:2986: writing smithi10066867-4 from 0 to 565248 tid 1 2024-04-07T16:26:24.951 INFO:tasks.rados.rados.0.smithi100.stdout:2986: writing smithi10066867-4 from 565248 to 966656 tid 2 2024-04-07T16:26:24.951 INFO:tasks.rados.rados.0.smithi100.stdout:2987: copy_from oid 16 from oid 2 current snap is 277 2024-04-07T16:26:24.951 INFO:tasks.rados.rados.0.smithi100.stdout:2988: delete oid 45 current snap is 277 2024-04-07T16:26:24.951 INFO:tasks.rados.rados.0.smithi100.stdout:2974: expect (ObjNum 0 snap 0 seq_num 0) 2024-04-07T16:26:24.956 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 5 v 1417 (ObjNum 289 snap 0 seq_num 0) dirty exists 2024-04-07T16:26:24.956 INFO:tasks.rados.rados.0.smithi100.stdout:2972: finishing copy_from to smithi10066867-29 2024-04-07T16:26:24.956 INFO:tasks.rados.rados.0.smithi100.stdout:2972: got expected ENOENT (src dne) 2024-04-07T16:26:24.956 INFO:tasks.rados.rados.0.smithi100.stdout:2976: finishing rollback tid 0 to smithi10066867-33 2024-04-07T16:26:24.956 INFO:tasks.rados.rados.0.smithi100.stdout:2972: finishing copy_from racing read to smithi10066867-29 2024-04-07T16:26:24.957 INFO:tasks.rados.rados.0.smithi100.stdout:2976: finishing rollback tid 1 to smithi10066867-33 2024-04-07T16:26:24.957 INFO:tasks.rados.rados.0.smithi100.stdout:2976: finishing rollback tid 2 to smithi10066867-33 2024-04-07T16:26:24.957 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 33 v 1637 (ObjNum 753 snap 235 seq_num 753) dirty exists 2024-04-07T16:26:24.957 INFO:tasks.rados.rados.0.smithi100.stdout:2971: finishing copy_from racing read to smithi10066867-50 2024-04-07T16:26:24.957 INFO:tasks.rados.rados.0.smithi100.stdout:2971: finishing copy_from to smithi10066867-50 2024-04-07T16:26:24.957 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 50 v 1507 (ObjNum 749 snap 230 seq_num 749) dirty exists 2024-04-07T16:26:24.957 INFO:tasks.rados.rados.0.smithi100.stdout:2975: finishing copy_from racing read to smithi10066867-24 2024-04-07T16:26:24.957 INFO:tasks.rados.rados.0.smithi100.stdout:2975: finishing copy_from to smithi10066867-24 2024-04-07T16:26:24.957 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 24 v 1209 (ObjNum 906 snap 274 seq_num 906) dirty exists 2024-04-07T16:26:24.957 INFO:tasks.rados.rados.0.smithi100.stdout:2982: finishing write tid 1 to smithi10066867-27 2024-04-07T16:26:24.957 INFO:tasks.rados.rados.0.smithi100.stdout:2982: finishing write tid 2 to smithi10066867-27 2024-04-07T16:26:24.957 INFO:tasks.rados.rados.0.smithi100.stdout:2982: finishing write tid 3 to smithi10066867-27 2024-04-07T16:26:24.957 INFO:tasks.rados.rados.0.smithi100.stdout:2981: finishing copy_from racing read to smithi10066867-44 2024-04-07T16:26:24.957 INFO:tasks.rados.rados.0.smithi100.stdout:2982: finishing write tid 4 to smithi10066867-27 2024-04-07T16:26:24.958 INFO:tasks.rados.rados.0.smithi100.stdout:2982: oid 27 updating version 0 to 1151 2024-04-07T16:26:24.958 INFO:tasks.rados.rados.0.smithi100.stdout:2982: oid 27 updating version 1151 to 1152 2024-04-07T16:26:24.958 INFO:tasks.rados.rados.0.smithi100.stdout:2982: oid 27 version 1152 is already newer than 1150 2024-04-07T16:26:24.958 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 27 v 1152 (ObjNum 915 snap 277 seq_num 915) dirty exists 2024-04-07T16:26:24.958 INFO:tasks.rados.rados.0.smithi100.stdout:2982: left oid 27 (ObjNum 915 snap 277 seq_num 915) 2024-04-07T16:26:24.958 INFO:tasks.rados.rados.0.smithi100.stdout:2981: finishing copy_from to smithi10066867-44 2024-04-07T16:26:24.958 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 44 v 1604 (ObjNum 776 snap 243 seq_num 776) dirty exists 2024-04-07T16:26:24.963 INFO:tasks.rados.rados.0.smithi100.stdout:2971: done (14 left) 2024-04-07T16:26:24.963 INFO:tasks.rados.rados.0.smithi100.stdout:2972: done (13 left) 2024-04-07T16:26:24.963 INFO:tasks.rados.rados.0.smithi100.stdout:2973: done (12 left) 2024-04-07T16:26:24.964 INFO:tasks.rados.rados.0.smithi100.stdout:2974: done (11 left) 2024-04-07T16:26:24.964 INFO:tasks.rados.rados.0.smithi100.stdout:2975: done (10 left) 2024-04-07T16:26:24.964 INFO:tasks.rados.rados.0.smithi100.stdout:2976: done (9 left) 2024-04-07T16:26:24.964 INFO:tasks.rados.rados.0.smithi100.stdout:2977: done (8 left) 2024-04-07T16:26:24.964 INFO:tasks.rados.rados.0.smithi100.stdout:2980: done (7 left) 2024-04-07T16:26:24.964 INFO:tasks.rados.rados.0.smithi100.stdout:2981: done (6 left) 2024-04-07T16:26:24.964 INFO:tasks.rados.rados.0.smithi100.stdout:2982: done (5 left) 2024-04-07T16:26:24.964 INFO:tasks.rados.rados.0.smithi100.stdout:2988: done (4 left) 2024-04-07T16:26:24.964 INFO:tasks.rados.rados.0.smithi100.stdout:2989: read oid 18 snap -1 2024-04-07T16:26:24.964 INFO:tasks.rados.rados.0.smithi100.stdout:2989: expect deleted 2024-04-07T16:26:24.964 INFO:tasks.rados.rados.0.smithi100.stdout:2990: copy_from oid 28 from oid 12 current snap is 277 2024-04-07T16:26:24.964 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 50 current snap is 277 2024-04-07T16:26:24.964 INFO:tasks.rados.rados.0.smithi100.stdout:2991: seq_num 919 ranges {548864=598016,1146880=401408} 2024-04-07T16:26:24.973 INFO:tasks.rados.rados.0.smithi100.stdout:2991: writing smithi10066867-50 from 548864 to 1146880 tid 1 2024-04-07T16:26:24.974 INFO:tasks.rados.rados.0.smithi100.stdout:2991: writing smithi10066867-50 from 1146880 to 1548288 tid 2 2024-04-07T16:26:24.974 INFO:tasks.rados.rados.0.smithi100.stdout:2992: delete oid 6 current snap is 277 2024-04-07T16:26:24.974 INFO:tasks.rados.rados.0.smithi100.stdout:2984: finishing write tid 1 to smithi10066867-3 2024-04-07T16:26:24.974 INFO:tasks.rados.rados.0.smithi100.stdout:2984: finishing write tid 2 to smithi10066867-3 2024-04-07T16:26:24.974 INFO:tasks.rados.rados.0.smithi100.stdout:2985: expect (ObjNum 794 snap 251 seq_num 794) 2024-04-07T16:26:25.039 INFO:tasks.rados.rados.0.smithi100.stdout:2984: finishing write tid 3 to smithi10066867-3 2024-04-07T16:26:25.039 INFO:tasks.rados.rados.0.smithi100.stdout:2986: finishing write tid 1 to smithi10066867-4 2024-04-07T16:26:25.039 INFO:tasks.rados.rados.0.smithi100.stdout:2985: done (7 left) 2024-04-07T16:26:25.040 INFO:tasks.rados.rados.0.smithi100.stdout:2992: done (6 left) 2024-04-07T16:26:25.040 INFO:tasks.rados.rados.0.smithi100.stdout:2993: rollback oid 39 current snap is 277 2024-04-07T16:26:25.040 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 39 to 271 2024-04-07T16:26:25.040 INFO:tasks.rados.rados.0.smithi100.stdout:2984: finishing write tid 4 to smithi10066867-3 2024-04-07T16:26:25.040 INFO:tasks.rados.rados.0.smithi100.stdout:2984: oid 3 updating version 0 to 1130 2024-04-07T16:26:25.040 INFO:tasks.rados.rados.0.smithi100.stdout:2984: oid 3 version 1130 is already newer than 1129 2024-04-07T16:26:25.040 INFO:tasks.rados.rados.0.smithi100.stdout:2984: oid 3 updating version 1130 to 1131 2024-04-07T16:26:25.040 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 3 v 1131 (ObjNum 916 snap 277 seq_num 916) dirty exists 2024-04-07T16:26:25.040 INFO:tasks.rados.rados.0.smithi100.stdout:2984: left oid 3 (ObjNum 916 snap 277 seq_num 916) 2024-04-07T16:26:25.040 INFO:tasks.rados.rados.0.smithi100.stdout:2984: done (6 left) 2024-04-07T16:26:25.040 INFO:tasks.rados.rados.0.smithi100.stdout:2994: copy_from oid 10 from oid 32 current snap is 277 2024-04-07T16:26:25.040 INFO:tasks.rados.rados.0.smithi100.stdout:2986: finishing write tid 2 to smithi10066867-4 2024-04-07T16:26:25.040 INFO:tasks.rados.rados.0.smithi100.stdout:2986: finishing write tid 3 to smithi10066867-4 2024-04-07T16:26:25.040 INFO:tasks.rados.rados.0.smithi100.stdout:2991: finishing write tid 1 to smithi10066867-50 2024-04-07T16:26:25.041 INFO:tasks.rados.rados.0.smithi100.stdout:2991: finishing write tid 2 to smithi10066867-50 2024-04-07T16:26:25.041 INFO:tasks.rados.rados.0.smithi100.stdout:2991: finishing write tid 3 to smithi10066867-50 2024-04-07T16:26:25.041 INFO:tasks.rados.rados.0.smithi100.stdout:2986: finishing write tid 4 to smithi10066867-4 2024-04-07T16:26:25.041 INFO:tasks.rados.rados.0.smithi100.stdout:2986: oid 4 updating version 0 to 1645 2024-04-07T16:26:25.041 INFO:tasks.rados.rados.0.smithi100.stdout:2986: oid 4 updating version 1645 to 1646 2024-04-07T16:26:25.041 INFO:tasks.rados.rados.0.smithi100.stdout:2986: oid 4 version 1646 is already newer than 1644 2024-04-07T16:26:25.041 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 4 v 1646 (ObjNum 917 snap 277 seq_num 917) dirty exists 2024-04-07T16:26:25.041 INFO:tasks.rados.rados.0.smithi100.stdout:2986: left oid 4 (ObjNum 917 snap 277 seq_num 917) 2024-04-07T16:26:25.041 INFO:tasks.rados.rados.0.smithi100.stdout:2991: finishing write tid 4 to smithi10066867-50 2024-04-07T16:26:25.041 INFO:tasks.rados.rados.0.smithi100.stdout:2991: oid 50 updating version 0 to 1511 2024-04-07T16:26:25.041 INFO:tasks.rados.rados.0.smithi100.stdout:2991: oid 50 version 1511 is already newer than 1510 2024-04-07T16:26:25.041 INFO:tasks.rados.rados.0.smithi100.stdout:2991: oid 50 version 1511 is already newer than 1509 2024-04-07T16:26:25.041 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 50 v 1511 (ObjNum 919 snap 277 seq_num 919) dirty exists 2024-04-07T16:26:25.042 INFO:tasks.rados.rados.0.smithi100.stdout:2991: left oid 50 (ObjNum 919 snap 277 seq_num 919) 2024-04-07T16:26:25.042 INFO:tasks.rados.rados.0.smithi100.stdout:2990: finishing copy_from racing read to smithi10066867-28 2024-04-07T16:26:25.042 INFO:tasks.rados.rados.0.smithi100.stdout:2990: finishing copy_from to smithi10066867-28 2024-04-07T16:26:25.042 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 28 v 1415 (ObjNum 829 snap 260 seq_num 829) dirty exists 2024-04-07T16:26:25.042 INFO:tasks.rados.rados.0.smithi100.stdout:2987: finishing copy_from racing read to smithi10066867-16 2024-04-07T16:26:25.042 INFO:tasks.rados.rados.0.smithi100.stdout:2986: done (6 left) 2024-04-07T16:26:25.042 INFO:tasks.rados.rados.0.smithi100.stdout:2989: done (5 left) 2024-04-07T16:26:25.042 INFO:tasks.rados.rados.0.smithi100.stdout:2990: done (4 left) 2024-04-07T16:26:25.042 INFO:tasks.rados.rados.0.smithi100.stdout:2991: done (3 left) 2024-04-07T16:26:25.042 INFO:tasks.rados.rados.0.smithi100.stdout:2995: rollback oid 6 current snap is 277 2024-04-07T16:26:25.042 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 6 to 271 2024-04-07T16:26:25.042 INFO:tasks.rados.rados.0.smithi100.stdout:2987: finishing copy_from to smithi10066867-16 2024-04-07T16:26:25.042 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 16 v 1271 (ObjNum 859 snap 261 seq_num 859) dirty exists 2024-04-07T16:26:25.042 INFO:tasks.rados.rados.0.smithi100.stdout:2987: done (3 left) 2024-04-07T16:26:25.043 INFO:tasks.rados.rados.0.smithi100.stdout:2996: delete oid 44 current snap is 277 2024-04-07T16:26:25.043 INFO:tasks.rados.rados.0.smithi100.stdout:2993: finishing rollback tid 1 to smithi10066867-39 2024-04-07T16:26:25.043 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 39 v 0 (ObjNum 529 snap 0 seq_num 3067285216) dirty dne 2024-04-07T16:26:25.043 INFO:tasks.rados.rados.0.smithi100.stdout:2995: finishing rollback tid 1 to smithi10066867-6 2024-04-07T16:26:25.044 INFO:tasks.rados.rados.0.smithi100.stdout:2995: finishing rollback tid 2 to smithi10066867-6 2024-04-07T16:26:25.044 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 6 v 1607 (ObjNum 810 snap 256 seq_num 810) dirty exists 2024-04-07T16:26:25.044 INFO:tasks.rados.rados.0.smithi100.stdout:2994: finishing copy_from racing read to smithi10066867-10 2024-04-07T16:26:25.045 INFO:tasks.rados.rados.0.smithi100.stdout:2993: done (3 left) 2024-04-07T16:26:25.045 INFO:tasks.rados.rados.0.smithi100.stdout:2995: done (2 left) 2024-04-07T16:26:25.045 INFO:tasks.rados.rados.0.smithi100.stdout:2996: done (1 left) 2024-04-07T16:26:25.045 INFO:tasks.rados.rados.0.smithi100.stdout:2997: read oid 46 snap -1 2024-04-07T16:26:25.045 INFO:tasks.rados.rados.0.smithi100.stdout:2997: expect (ObjNum 0 snap 0 seq_num 0) 2024-04-07T16:26:25.045 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 18 current snap is 277 2024-04-07T16:26:25.045 INFO:tasks.rados.rados.0.smithi100.stdout:2998: seq_num 922 ranges {0=647168,647168=401408} 2024-04-07T16:26:25.051 INFO:tasks.rados.rados.0.smithi100.stdout:2998: writing smithi10066867-18 from 0 to 647168 tid 1 2024-04-07T16:26:25.055 INFO:tasks.rados.rados.0.smithi100.stdout:2998: writing smithi10066867-18 from 647168 to 1048576 tid 2 2024-04-07T16:26:25.055 INFO:tasks.rados.rados.0.smithi100.stdout:2999: rmattr oid 24 current snap is 277 2024-04-07T16:26:25.055 INFO:tasks.rados.rados.0.smithi100.stdout:2994: finishing copy_from to smithi10066867-10 2024-04-07T16:26:25.055 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 10 v 1117 (ObjNum 289 snap 0 seq_num 0) dirty exists 2024-04-07T16:26:25.055 INFO:tasks.rados.rados.0.smithi100.stdout:2997: expect (ObjNum 0 snap 0 seq_num 0) 2024-04-07T16:26:25.059 INFO:tasks.rados.rados.0.smithi100.stdout:2994: done (3 left) 2024-04-07T16:26:25.059 INFO:tasks.rados.rados.0.smithi100.stdout:2997: done (2 left) 2024-04-07T16:26:25.059 INFO:tasks.rados.rados.0.smithi100.stdout:3000: snap_remove snap 272 2024-04-07T16:26:25.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:24 smithi100 bash[74942]: cluster 2024-04-07T16:26:23.913149+0000 mon.a (mon.0) 994 : cluster [DBG] osdmap e628: 8 total, 8 up, 8 in 2024-04-07T16:26:25.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:24 smithi100 bash[76202]: cluster 2024-04-07T16:26:23.913149+0000 mon.a (mon.0) 994 : cluster [DBG] osdmap e628: 8 total, 8 up, 8 in 2024-04-07T16:26:25.242 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:24 smithi144 bash[66654]: cluster 2024-04-07T16:26:23.913149+0000 mon.a (mon.0) 994 : cluster [DBG] osdmap e628: 8 total, 8 up, 8 in 2024-04-07T16:26:25.590 INFO:journalctl@ceph.mds.b.smithi144.stdout:Apr 07 16:26:25 smithi144 bash[88086]: ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4-mds-b 2024-04-07T16:26:25.590 INFO:journalctl@ceph.mds.b.smithi144.stdout:Apr 07 16:26:25 smithi144 bash[88146]: Error response from daemon: No such container: ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4-mds-b 2024-04-07T16:26:25.590 INFO:journalctl@ceph.mds.b.smithi144.stdout:Apr 07 16:26:25 smithi144 systemd[1]: ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@mds.b.service: Deactivated successfully. 2024-04-07T16:26:25.590 INFO:journalctl@ceph.mds.b.smithi144.stdout:Apr 07 16:26:25 smithi144 systemd[1]: Stopped Ceph mds.b for 0702e3a8-f4f8-11ee-b649-cb9ed24678a4. 2024-04-07T16:26:25.856 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:26:25 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:26:25.856 INFO:journalctl@ceph.osd.5.smithi144.stdout:Apr 07 16:26:25 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:26:25.856 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:26:25 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:26:25.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:25 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:26:25.857 INFO:journalctl@ceph.osd.6.smithi144.stdout:Apr 07 16:26:25 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:26:25.857 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:26:25 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:26:25.858 INFO:journalctl@ceph.osd.7.smithi144.stdout:Apr 07 16:26:25 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:26:25.858 INFO:journalctl@ceph.mds.b.smithi144.stdout:Apr 07 16:26:25 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:26:25.858 INFO:journalctl@ceph.mds.b.smithi144.stdout:Apr 07 16:26:25 smithi144 systemd[1]: Started Ceph mds.b for 0702e3a8-f4f8-11ee-b649-cb9ed24678a4. 2024-04-07T16:26:25.858 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 16:26:25 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:26:25.858 INFO:journalctl@ceph.osd.4.smithi144.stdout:Apr 07 16:26:25 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:26:25.921 INFO:tasks.rados.rados.0.smithi100.stdout:3000: done (2 left) 2024-04-07T16:26:25.922 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 36 current snap is 277 2024-04-07T16:26:25.922 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 24 v 1210 (ObjNum 906 snap 274 seq_num 906) dirty exists 2024-04-07T16:26:25.922 INFO:tasks.rados.rados.0.smithi100.stdout:2998: finishing write tid 1 to smithi10066867-18 2024-04-07T16:26:25.922 INFO:tasks.rados.rados.0.smithi100.stdout:2998: finishing write tid 2 to smithi10066867-18 2024-04-07T16:26:25.922 INFO:tasks.rados.rados.0.smithi100.stdout:3001: seq_num 923 ranges {1908736=663552,2572288=401408} 2024-04-07T16:26:25.959 INFO:tasks.rados.rados.0.smithi100.stdout:3001: writing smithi10066867-36 from 1908736 to 2572288 tid 1 2024-04-07T16:26:25.965 INFO:tasks.rados.rados.0.smithi100.stdout:3001: writing smithi10066867-36 from 2572288 to 2973696 tid 2 2024-04-07T16:26:25.965 INFO:tasks.rados.rados.0.smithi100.stdout:2999: done (2 left) 2024-04-07T16:26:25.965 INFO:tasks.rados.rados.0.smithi100.stdout:3002: snap_create 2024-04-07T16:26:25.966 INFO:tasks.rados.rados.0.smithi100.stdout:2998: finishing write tid 3 to smithi10066867-18 2024-04-07T16:26:25.966 INFO:tasks.rados.rados.0.smithi100.stdout:2998: finishing write tid 4 to smithi10066867-18 2024-04-07T16:26:25.966 INFO:tasks.rados.rados.0.smithi100.stdout:2998: oid 18 updating version 0 to 1649 2024-04-07T16:26:25.966 INFO:tasks.rados.rados.0.smithi100.stdout:2998: oid 18 version 1649 is already newer than 1648 2024-04-07T16:26:25.966 INFO:tasks.rados.rados.0.smithi100.stdout:2998: oid 18 version 1649 is already newer than 1647 2024-04-07T16:26:25.966 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 18 v 1649 (ObjNum 922 snap 277 seq_num 922) dirty exists 2024-04-07T16:26:25.966 INFO:tasks.rados.rados.0.smithi100.stdout:2998: left oid 18 (ObjNum 922 snap 277 seq_num 922) 2024-04-07T16:26:25.978 INFO:tasks.rados.rados.0.smithi100.stdout:3001: finishing write tid 1 to smithi10066867-36 2024-04-07T16:26:25.979 INFO:tasks.rados.rados.0.smithi100.stdout:3001: finishing write tid 2 to smithi10066867-36 2024-04-07T16:26:25.984 INFO:tasks.rados.rados.0.smithi100.stdout:3001: finishing write tid 3 to smithi10066867-36 2024-04-07T16:26:25.987 INFO:tasks.rados.rados.0.smithi100.stdout:3001: finishing write tid 4 to smithi10066867-36 2024-04-07T16:26:25.987 INFO:tasks.rados.rados.0.smithi100.stdout:3001: oid 36 updating version 0 to 1212 2024-04-07T16:26:25.987 INFO:tasks.rados.rados.0.smithi100.stdout:3001: oid 36 updating version 1212 to 1213 2024-04-07T16:26:25.987 INFO:tasks.rados.rados.0.smithi100.stdout:3001: oid 36 version 1213 is already newer than 1211 2024-04-07T16:26:25.987 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 36 v 1213 (ObjNum 923 snap 277 seq_num 923) dirty exists 2024-04-07T16:26:25.987 INFO:tasks.rados.rados.0.smithi100.stdout:3001: left oid 36 (ObjNum 923 snap 277 seq_num 923) 2024-04-07T16:26:26.185 INFO:journalctl@ceph.mds.a.smithi100.stdout:Apr 07 16:26:25 smithi100 bash[98288]: debug 2024-04-07T16:26:25.704+0000 7fadc9db5700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-07T16:26:26.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:25 smithi100 bash[74942]: cluster 2024-04-07T16:26:24.766255+0000 mgr.y (mgr.25175) 299 : cluster [DBG] pgmap v535: 121 pgs: 121 active+clean; 91 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 6.0 MiB/s rd, 3.2 MiB/s wr, 32 op/s 2024-04-07T16:26:26.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:25 smithi100 bash[74942]: cluster 2024-04-07T16:26:24.930383+0000 mon.a (mon.0) 995 : cluster [DBG] osdmap e629: 8 total, 8 up, 8 in 2024-04-07T16:26:26.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:25 smithi100 bash[74942]: audit 2024-04-07T16:26:25.778335+0000 mon.a (mon.0) 996 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:26:26.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:25 smithi100 bash[74942]: audit 2024-04-07T16:26:25.790103+0000 mon.a (mon.0) 997 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:26:26.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:25 smithi100 bash[74942]: audit 2024-04-07T16:26:25.808720+0000 mon.a (mon.0) 998 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:26:26.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:25 smithi100 bash[76202]: cluster 2024-04-07T16:26:24.766255+0000 mgr.y (mgr.25175) 299 : cluster [DBG] pgmap v535: 121 pgs: 121 active+clean; 91 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 6.0 MiB/s rd, 3.2 MiB/s wr, 32 op/s 2024-04-07T16:26:26.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:25 smithi100 bash[76202]: cluster 2024-04-07T16:26:24.930383+0000 mon.a (mon.0) 995 : cluster [DBG] osdmap e629: 8 total, 8 up, 8 in 2024-04-07T16:26:26.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:25 smithi100 bash[76202]: audit 2024-04-07T16:26:25.778335+0000 mon.a (mon.0) 996 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:26:26.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:25 smithi100 bash[76202]: audit 2024-04-07T16:26:25.790103+0000 mon.a (mon.0) 997 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:26:26.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:25 smithi100 bash[76202]: audit 2024-04-07T16:26:25.808720+0000 mon.a (mon.0) 998 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:26:26.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:25 smithi144 bash[66654]: cluster 2024-04-07T16:26:24.766255+0000 mgr.y (mgr.25175) 299 : cluster [DBG] pgmap v535: 121 pgs: 121 active+clean; 91 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 6.0 MiB/s rd, 3.2 MiB/s wr, 32 op/s 2024-04-07T16:26:26.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:25 smithi144 bash[66654]: cluster 2024-04-07T16:26:24.930383+0000 mon.a (mon.0) 995 : cluster [DBG] osdmap e629: 8 total, 8 up, 8 in 2024-04-07T16:26:26.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:25 smithi144 bash[66654]: audit 2024-04-07T16:26:25.778335+0000 mon.a (mon.0) 996 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:26:26.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:25 smithi144 bash[66654]: audit 2024-04-07T16:26:25.790103+0000 mon.a (mon.0) 997 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:26:26.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:25 smithi144 bash[66654]: audit 2024-04-07T16:26:25.808720+0000 mon.a (mon.0) 998 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:26:26.935 INFO:tasks.rados.rados.0.smithi100.stdout:2998: done (2 left) 2024-04-07T16:26:26.935 INFO:tasks.rados.rados.0.smithi100.stdout:3001: done (1 left) 2024-04-07T16:26:26.935 INFO:tasks.rados.rados.0.smithi100.stdout:3002: done (0 left) 2024-04-07T16:26:26.935 INFO:tasks.rados.rados.0.smithi100.stdout:3003: rollback oid 12 current snap is 278 2024-04-07T16:26:26.935 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 12 to 277 2024-04-07T16:26:26.935 INFO:tasks.rados.rados.0.smithi100.stdout:3004: rmattr oid 23 current snap is 278 2024-04-07T16:26:26.935 INFO:tasks.rados.rados.0.smithi100.stdout:3004: done (1 left) 2024-04-07T16:26:26.935 INFO:tasks.rados.rados.0.smithi100.stdout:3005: read oid 2 snap -1 2024-04-07T16:26:26.935 INFO:tasks.rados.rados.0.smithi100.stdout:3005: expect (ObjNum 859 snap 261 seq_num 859) 2024-04-07T16:26:26.935 INFO:tasks.rados.rados.0.smithi100.stdout:3006: read oid 34 snap -1 2024-04-07T16:26:26.936 INFO:tasks.rados.rados.0.smithi100.stdout:3006: expect (ObjNum 0 snap 0 seq_num 0) 2024-04-07T16:26:26.936 INFO:tasks.rados.rados.0.smithi100.stdout:3007: setattr oid 30 current snap is 278 2024-04-07T16:26:26.941 INFO:tasks.rados.rados.0.smithi100.stdout:3008: rollback oid 9 current snap is 278 2024-04-07T16:26:26.941 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 9 to 271 2024-04-07T16:26:26.942 INFO:tasks.rados.rados.0.smithi100.stdout:3009: rmattr oid 44 current snap is 278 2024-04-07T16:26:26.942 INFO:tasks.rados.rados.0.smithi100.stdout:3009: done (5 left) 2024-04-07T16:26:26.942 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 39 current snap is 278 2024-04-07T16:26:26.942 INFO:tasks.rados.rados.0.smithi100.stdout:3010: seq_num 924 ranges {0=679936,679936=401408} 2024-04-07T16:26:26.952 INFO:tasks.rados.rados.0.smithi100.stdout:3010: writing smithi10066867-39 from 0 to 679936 tid 1 2024-04-07T16:26:26.958 INFO:tasks.rados.rados.0.smithi100.stdout:3010: writing smithi10066867-39 from 679936 to 1081344 tid 2 2024-04-07T16:26:26.958 INFO:tasks.rados.rados.0.smithi100.stdout:3011: delete oid 47 current snap is 278 2024-04-07T16:26:26.962 INFO:tasks.rados.rados.0.smithi100.stdout:3006: expect (ObjNum 0 snap 0 seq_num 0) 2024-04-07T16:26:26.966 INFO:tasks.rados.rados.0.smithi100.stdout:3008: finishing rollback tid 0 to smithi10066867-9 2024-04-07T16:26:26.966 INFO:tasks.rados.rados.0.smithi100.stdout:3003: finishing rollback tid 0 to smithi10066867-12 2024-04-07T16:26:26.966 INFO:tasks.rados.rados.0.smithi100.stdout:3003: finishing rollback tid 1 to smithi10066867-12 2024-04-07T16:26:26.966 INFO:tasks.rados.rados.0.smithi100.stdout:3003: finishing rollback tid 2 to smithi10066867-12 2024-04-07T16:26:26.966 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 12 v 1432 (ObjNum 829 snap 260 seq_num 829) dirty exists 2024-04-07T16:26:26.966 INFO:tasks.rados.rados.0.smithi100.stdout:3008: finishing rollback tid 1 to smithi10066867-9 2024-04-07T16:26:26.966 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 30 v 1221 (ObjNum 880 snap 264 seq_num 880) dirty exists 2024-04-07T16:26:26.966 INFO:tasks.rados.rados.0.smithi100.stdout:3008: finishing rollback tid 2 to smithi10066867-9 2024-04-07T16:26:26.966 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 9 v 1129 (ObjNum 849 snap 261 seq_num 849) dirty exists 2024-04-07T16:26:26.969 INFO:tasks.rados.rados.0.smithi100.stdout:3010: finishing write tid 1 to smithi10066867-39 2024-04-07T16:26:26.969 INFO:tasks.rados.rados.0.smithi100.stdout:3010: finishing write tid 2 to smithi10066867-39 2024-04-07T16:26:26.970 INFO:tasks.rados.rados.0.smithi100.stdout:3003: done (6 left) 2024-04-07T16:26:26.970 INFO:tasks.rados.rados.0.smithi100.stdout:3006: done (5 left) 2024-04-07T16:26:26.970 INFO:tasks.rados.rados.0.smithi100.stdout:3007: done (4 left) 2024-04-07T16:26:26.970 INFO:tasks.rados.rados.0.smithi100.stdout:3008: done (3 left) 2024-04-07T16:26:26.970 INFO:tasks.rados.rados.0.smithi100.stdout:3011: done (2 left) 2024-04-07T16:26:26.970 INFO:tasks.rados.rados.0.smithi100.stdout:3012: copy_from oid 33 from oid 31 current snap is 278 2024-04-07T16:26:26.970 INFO:tasks.rados.rados.0.smithi100.stdout:3013: delete oid 49 current snap is 278 2024-04-07T16:26:26.971 INFO:tasks.rados.rados.0.smithi100.stdout:3010: finishing write tid 3 to smithi10066867-39 2024-04-07T16:26:26.978 INFO:tasks.rados.rados.0.smithi100.stdout:3013: done (3 left) 2024-04-07T16:26:26.978 INFO:tasks.rados.rados.0.smithi100.stdout:3014: setattr oid 4 current snap is 278 2024-04-07T16:26:26.978 INFO:tasks.rados.rados.0.smithi100.stdout:3010: finishing write tid 4 to smithi10066867-39 2024-04-07T16:26:26.978 INFO:tasks.rados.rados.0.smithi100.stdout:3010: oid 39 updating version 0 to 1223 2024-04-07T16:26:26.978 INFO:tasks.rados.rados.0.smithi100.stdout:3010: oid 39 version 1223 is already newer than 1222 2024-04-07T16:26:26.978 INFO:tasks.rados.rados.0.smithi100.stdout:3010: oid 39 updating version 1223 to 1224 2024-04-07T16:26:26.978 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 39 v 1224 (ObjNum 924 snap 278 seq_num 924) dirty exists 2024-04-07T16:26:26.978 INFO:tasks.rados.rados.0.smithi100.stdout:3010: left oid 39 (ObjNum 924 snap 278 seq_num 924) 2024-04-07T16:26:26.979 INFO:tasks.rados.rados.0.smithi100.stdout:3010: done (3 left) 2024-04-07T16:26:26.979 INFO:tasks.rados.rados.0.smithi100.stdout:3015: rollback oid 27 current snap is 278 2024-04-07T16:26:26.979 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 27 to 273 2024-04-07T16:26:26.979 INFO:tasks.rados.rados.0.smithi100.stdout:3016: copy_from oid 21 from oid 48 current snap is 278 2024-04-07T16:26:26.979 INFO:tasks.rados.rados.0.smithi100.stdout:3017: rollback oid 23 current snap is 278 2024-04-07T16:26:26.979 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 23 to 277 2024-04-07T16:26:26.979 INFO:tasks.rados.rados.0.smithi100.stdout:3018: read oid 30 snap -1 2024-04-07T16:26:26.979 INFO:tasks.rados.rados.0.smithi100.stdout:3018: expect (ObjNum 880 snap 264 seq_num 880) 2024-04-07T16:26:26.979 INFO:tasks.rados.rados.0.smithi100.stdout:3019: rmattr oid 11 current snap is 278 2024-04-07T16:26:26.979 INFO:tasks.rados.rados.0.smithi100.stdout:3020: read oid 29 snap -1 2024-04-07T16:26:26.979 INFO:tasks.rados.rados.0.smithi100.stdout:3020: expect deleted 2024-04-07T16:26:26.979 INFO:tasks.rados.rados.0.smithi100.stdout:3021: read oid 25 snap 277 2024-04-07T16:26:26.979 INFO:tasks.rados.rados.0.smithi100.stdout:3021: expect (ObjNum 784 snap 246 seq_num 784) 2024-04-07T16:26:26.979 INFO:tasks.rados.rados.0.smithi100.stdout:3022: read oid 43 snap -1 2024-04-07T16:26:26.979 INFO:tasks.rados.rados.0.smithi100.stdout:3022: expect deleted 2024-04-07T16:26:26.979 INFO:tasks.rados.rados.0.smithi100.stdout:3023: rollback oid 3 current snap is 278 2024-04-07T16:26:26.979 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 3 to 273 2024-04-07T16:26:26.980 INFO:tasks.rados.rados.0.smithi100.stdout:3024: read oid 8 snap -1 2024-04-07T16:26:26.980 INFO:tasks.rados.rados.0.smithi100.stdout:3024: expect (ObjNum 749 snap 230 seq_num 749) 2024-04-07T16:26:26.980 INFO:tasks.rados.rados.0.smithi100.stdout:3025: snap_create 2024-04-07T16:26:26.985 INFO:tasks.rados.rados.0.smithi100.stdout:3018: expect (ObjNum 880 snap 264 seq_num 880) 2024-04-07T16:26:27.016 INFO:tasks.rados.rados.0.smithi100.stdout:3005: expect (ObjNum 859 snap 261 seq_num 859) 2024-04-07T16:26:27.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:26 smithi144 bash[66654]: cluster 2024-04-07T16:26:25.937196+0000 mon.a (mon.0) 999 : cluster [DBG] osdmap e630: 8 total, 8 up, 8 in 2024-04-07T16:26:27.106 INFO:journalctl@ceph.mds.b.smithi144.stdout:Apr 07 16:26:26 smithi144 bash[88204]: starting mds.b at 2024-04-07T16:26:27.185 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:26 smithi100 bash[74942]: cluster 2024-04-07T16:26:25.937196+0000 mon.a (mon.0) 999 : cluster [DBG] osdmap e630: 8 total, 8 up, 8 in 2024-04-07T16:26:27.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:26 smithi100 bash[76202]: cluster 2024-04-07T16:26:25.937196+0000 mon.a (mon.0) 999 : cluster [DBG] osdmap e630: 8 total, 8 up, 8 in 2024-04-07T16:26:27.263 INFO:tasks.rados.rados.0.smithi100.stdout:3024: expect (ObjNum 749 snap 230 seq_num 749) 2024-04-07T16:26:27.276 INFO:tasks.rados.rados.0.smithi100.stdout:3012: finishing copy_from to smithi10066867-33 2024-04-07T16:26:27.276 INFO:tasks.rados.rados.0.smithi100.stdout:3012: got expected ENOENT (src dne) 2024-04-07T16:26:27.276 INFO:tasks.rados.rados.0.smithi100.stdout:3012: finishing copy_from racing read to smithi10066867-33 2024-04-07T16:26:27.277 INFO:tasks.rados.rados.0.smithi100.stdout:3015: finishing rollback tid 0 to smithi10066867-27 2024-04-07T16:26:27.277 INFO:tasks.rados.rados.0.smithi100.stdout:3023: finishing rollback tid 0 to smithi10066867-3 2024-04-07T16:26:27.277 INFO:tasks.rados.rados.0.smithi100.stdout:3015: finishing rollback tid 1 to smithi10066867-27 2024-04-07T16:26:27.277 INFO:tasks.rados.rados.0.smithi100.stdout:3017: finishing rollback tid 1 to smithi10066867-23 2024-04-07T16:26:27.277 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 23 v 0 (ObjNum 33 snap 0 seq_num 2670634328) dirty dne 2024-04-07T16:26:27.277 INFO:tasks.rados.rados.0.smithi100.stdout:3015: finishing rollback tid 2 to smithi10066867-27 2024-04-07T16:26:27.277 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 27 v 1155 (ObjNum 863 snap 261 seq_num 863) dirty exists 2024-04-07T16:26:27.277 INFO:tasks.rados.rados.0.smithi100.stdout:3023: finishing rollback tid 1 to smithi10066867-3 2024-04-07T16:26:27.277 INFO:tasks.rados.rados.0.smithi100.stdout:3023: finishing rollback tid 2 to smithi10066867-3 2024-04-07T16:26:27.277 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 3 v 1134 (ObjNum 903 snap 272 seq_num 903) dirty exists 2024-04-07T16:26:27.277 INFO:tasks.rados.rados.0.smithi100.stdout:3016: finishing copy_from racing read to smithi10066867-21 2024-04-07T16:26:27.277 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 4 v 1655 (ObjNum 917 snap 277 seq_num 917) dirty exists 2024-04-07T16:26:27.277 INFO:tasks.rados.rados.0.smithi100.stdout:3016: finishing copy_from to smithi10066867-21 2024-04-07T16:26:27.277 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 21 v 1116 (ObjNum 794 snap 251 seq_num 794) dirty exists 2024-04-07T16:26:27.277 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 11 v 1657 (ObjNum 909 snap 274 seq_num 909) dirty exists 2024-04-07T16:26:27.278 INFO:tasks.rados.rados.0.smithi100.stdout:3021: expect (ObjNum 784 snap 246 seq_num 784) 2024-04-07T16:26:27.942 INFO:tasks.rados.rados.0.smithi100.stdout:3005: done (13 left) 2024-04-07T16:26:27.943 INFO:tasks.rados.rados.0.smithi100.stdout:3012: done (12 left) 2024-04-07T16:26:27.943 INFO:tasks.rados.rados.0.smithi100.stdout:3014: done (11 left) 2024-04-07T16:26:27.943 INFO:tasks.rados.rados.0.smithi100.stdout:3015: done (10 left) 2024-04-07T16:26:27.943 INFO:tasks.rados.rados.0.smithi100.stdout:3016: done (9 left) 2024-04-07T16:26:27.943 INFO:tasks.rados.rados.0.smithi100.stdout:3017: done (8 left) 2024-04-07T16:26:27.943 INFO:tasks.rados.rados.0.smithi100.stdout:3018: done (7 left) 2024-04-07T16:26:27.943 INFO:tasks.rados.rados.0.smithi100.stdout:3019: done (6 left) 2024-04-07T16:26:27.943 INFO:tasks.rados.rados.0.smithi100.stdout:3020: done (5 left) 2024-04-07T16:26:27.943 INFO:tasks.rados.rados.0.smithi100.stdout:3021: done (4 left) 2024-04-07T16:26:27.943 INFO:tasks.rados.rados.0.smithi100.stdout:3022: done (3 left) 2024-04-07T16:26:27.943 INFO:tasks.rados.rados.0.smithi100.stdout:3023: done (2 left) 2024-04-07T16:26:27.943 INFO:tasks.rados.rados.0.smithi100.stdout:3024: done (1 left) 2024-04-07T16:26:27.943 INFO:tasks.rados.rados.0.smithi100.stdout:3025: done (0 left) 2024-04-07T16:26:27.944 INFO:tasks.rados.rados.0.smithi100.stdout:3026: rollback oid 9 current snap is 279 2024-04-07T16:26:27.944 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 9 to 271 2024-04-07T16:26:27.944 INFO:tasks.rados.rados.0.smithi100.stdout:3027: setattr oid 48 current snap is 279 2024-04-07T16:26:27.944 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 18 current snap is 279 2024-04-07T16:26:27.944 INFO:tasks.rados.rados.0.smithi100.stdout:3028: seq_num 927 ranges {1048576=729088,1777664=401408} 2024-04-07T16:26:27.968 INFO:tasks.rados.rados.0.smithi100.stdout:3028: writing smithi10066867-18 from 1048576 to 1777664 tid 1 2024-04-07T16:26:27.975 INFO:tasks.rados.rados.0.smithi100.stdout:3028: writing smithi10066867-18 from 1777664 to 2179072 tid 2 2024-04-07T16:26:27.975 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 2 current snap is 279 2024-04-07T16:26:27.975 INFO:tasks.rados.rados.0.smithi100.stdout:3026: finishing rollback tid 0 to smithi10066867-9 2024-04-07T16:26:27.975 INFO:tasks.rados.rados.0.smithi100.stdout:3026: finishing rollback tid 1 to smithi10066867-9 2024-04-07T16:26:27.975 INFO:tasks.rados.rados.0.smithi100.stdout:3026: finishing rollback tid 2 to smithi10066867-9 2024-04-07T16:26:27.975 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 9 v 1132 (ObjNum 849 snap 261 seq_num 849) dirty exists 2024-04-07T16:26:27.975 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 48 v 1124 (ObjNum 794 snap 251 seq_num 794) dirty exists 2024-04-07T16:26:27.975 INFO:tasks.rados.rados.0.smithi100.stdout:3029: seq_num 928 ranges {6004736=745472,6750208=401408} 2024-04-07T16:26:28.036 INFO:tasks.rados.rados.0.smithi100.stdout:3029: writing smithi10066867-2 from 6004736 to 6750208 tid 1 2024-04-07T16:26:28.039 INFO:tasks.rados.rados.0.smithi100.stdout:3029: writing smithi10066867-2 from 6750208 to 7151616 tid 2 2024-04-07T16:26:28.039 INFO:tasks.rados.rados.0.smithi100.stdout:3026: done (3 left) 2024-04-07T16:26:28.039 INFO:tasks.rados.rados.0.smithi100.stdout:3027: done (2 left) 2024-04-07T16:26:28.039 INFO:tasks.rados.rados.0.smithi100.stdout:3030: snap_create 2024-04-07T16:26:28.039 INFO:tasks.rados.rados.0.smithi100.stdout:3028: finishing write tid 1 to smithi10066867-18 2024-04-07T16:26:28.039 INFO:tasks.rados.rados.0.smithi100.stdout:3028: finishing write tid 2 to smithi10066867-18 2024-04-07T16:26:28.039 INFO:tasks.rados.rados.0.smithi100.stdout:3028: finishing write tid 3 to smithi10066867-18 2024-04-07T16:26:28.039 INFO:tasks.rados.rados.0.smithi100.stdout:3028: finishing write tid 4 to smithi10066867-18 2024-04-07T16:26:28.039 INFO:tasks.rados.rados.0.smithi100.stdout:3028: oid 18 updating version 0 to 1661 2024-04-07T16:26:28.039 INFO:tasks.rados.rados.0.smithi100.stdout:3028: oid 18 version 1661 is already newer than 1660 2024-04-07T16:26:28.039 INFO:tasks.rados.rados.0.smithi100.stdout:3028: oid 18 version 1661 is already newer than 1659 2024-04-07T16:26:28.039 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 18 v 1661 (ObjNum 927 snap 279 seq_num 927) dirty exists 2024-04-07T16:26:28.039 INFO:tasks.rados.rados.0.smithi100.stdout:3028: left oid 18 (ObjNum 927 snap 279 seq_num 927) 2024-04-07T16:26:28.048 INFO:tasks.rados.rados.0.smithi100.stdout:3029: finishing write tid 1 to smithi10066867-2 2024-04-07T16:26:28.048 INFO:tasks.rados.rados.0.smithi100.stdout:3029: finishing write tid 2 to smithi10066867-2 2024-04-07T16:26:28.049 INFO:tasks.rados.rados.0.smithi100.stdout:3029: finishing write tid 3 to smithi10066867-2 2024-04-07T16:26:28.052 INFO:tasks.rados.rados.0.smithi100.stdout:3029: finishing write tid 4 to smithi10066867-2 2024-04-07T16:26:28.052 INFO:tasks.rados.rados.0.smithi100.stdout:3029: oid 2 updating version 0 to 1611 2024-04-07T16:26:28.052 INFO:tasks.rados.rados.0.smithi100.stdout:3029: oid 2 version 1611 is already newer than 1609 2024-04-07T16:26:28.052 INFO:tasks.rados.rados.0.smithi100.stdout:3029: oid 2 version 1611 is already newer than 1610 2024-04-07T16:26:28.052 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 2 v 1611 (ObjNum 928 snap 279 seq_num 928) dirty exists 2024-04-07T16:26:28.052 INFO:tasks.rados.rados.0.smithi100.stdout:3029: left oid 2 (ObjNum 928 snap 279 seq_num 928) 2024-04-07T16:26:28.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:27 smithi100 bash[74942]: cluster 2024-04-07T16:26:26.766934+0000 mgr.y (mgr.25175) 300 : cluster [DBG] pgmap v538: 121 pgs: 121 active+clean; 92 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 6.0 MiB/s rd, 2.5 MiB/s wr, 28 op/s 2024-04-07T16:26:28.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:27 smithi100 bash[74942]: cluster 2024-04-07T16:26:26.948912+0000 mon.a (mon.0) 1000 : cluster [DBG] mds.? [v2:172.21.15.144:6832/1569910340,v1:172.21.15.144:6833/1569910340] up:boot 2024-04-07T16:26:28.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:27 smithi100 bash[74942]: cluster 2024-04-07T16:26:26.949123+0000 mon.a (mon.0) 1001 : cluster [DBG] fsmap cephfs:1 {0=a=up:active} 1 up:standby 2024-04-07T16:26:28.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:27 smithi100 bash[74942]: cluster 2024-04-07T16:26:26.949603+0000 mon.a (mon.0) 1002 : cluster [DBG] osdmap e631: 8 total, 8 up, 8 in 2024-04-07T16:26:28.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:27 smithi100 bash[74942]: audit 2024-04-07T16:26:26.950407+0000 mon.a (mon.0) 1003 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-07T16:26:28.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:27 smithi100 bash[76202]: cluster 2024-04-07T16:26:26.766934+0000 mgr.y (mgr.25175) 300 : cluster [DBG] pgmap v538: 121 pgs: 121 active+clean; 92 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 6.0 MiB/s rd, 2.5 MiB/s wr, 28 op/s 2024-04-07T16:26:28.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:27 smithi100 bash[76202]: cluster 2024-04-07T16:26:26.948912+0000 mon.a (mon.0) 1000 : cluster [DBG] mds.? [v2:172.21.15.144:6832/1569910340,v1:172.21.15.144:6833/1569910340] up:boot 2024-04-07T16:26:28.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:27 smithi100 bash[76202]: cluster 2024-04-07T16:26:26.949123+0000 mon.a (mon.0) 1001 : cluster [DBG] fsmap cephfs:1 {0=a=up:active} 1 up:standby 2024-04-07T16:26:28.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:27 smithi100 bash[76202]: cluster 2024-04-07T16:26:26.949603+0000 mon.a (mon.0) 1002 : cluster [DBG] osdmap e631: 8 total, 8 up, 8 in 2024-04-07T16:26:28.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:27 smithi100 bash[76202]: audit 2024-04-07T16:26:26.950407+0000 mon.a (mon.0) 1003 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-07T16:26:28.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:27 smithi144 bash[66654]: cluster 2024-04-07T16:26:26.766934+0000 mgr.y (mgr.25175) 300 : cluster [DBG] pgmap v538: 121 pgs: 121 active+clean; 92 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 6.0 MiB/s rd, 2.5 MiB/s wr, 28 op/s 2024-04-07T16:26:28.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:27 smithi144 bash[66654]: cluster 2024-04-07T16:26:26.948912+0000 mon.a (mon.0) 1000 : cluster [DBG] mds.? [v2:172.21.15.144:6832/1569910340,v1:172.21.15.144:6833/1569910340] up:boot 2024-04-07T16:26:28.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:27 smithi144 bash[66654]: cluster 2024-04-07T16:26:26.949123+0000 mon.a (mon.0) 1001 : cluster [DBG] fsmap cephfs:1 {0=a=up:active} 1 up:standby 2024-04-07T16:26:28.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:27 smithi144 bash[66654]: cluster 2024-04-07T16:26:26.949603+0000 mon.a (mon.0) 1002 : cluster [DBG] osdmap e631: 8 total, 8 up, 8 in 2024-04-07T16:26:28.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:27 smithi144 bash[66654]: audit 2024-04-07T16:26:26.950407+0000 mon.a (mon.0) 1003 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-07T16:26:28.685 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:26:28 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:26:28] "GET /metrics HTTP/1.1" 200 36338 "" "Prometheus/2.43.0" 2024-04-07T16:26:28.955 INFO:tasks.rados.rados.0.smithi100.stdout:3028: done (2 left) 2024-04-07T16:26:28.955 INFO:tasks.rados.rados.0.smithi100.stdout:3029: done (1 left) 2024-04-07T16:26:28.955 INFO:tasks.rados.rados.0.smithi100.stdout:3030: done (0 left) 2024-04-07T16:26:28.955 INFO:tasks.rados.rados.0.smithi100.stdout:3031: read oid 9 snap -1 2024-04-07T16:26:28.955 INFO:tasks.rados.rados.0.smithi100.stdout:3031: expect (ObjNum 849 snap 261 seq_num 849) 2024-04-07T16:26:28.956 INFO:tasks.rados.rados.0.smithi100.stdout:3032: rmattr oid 1 current snap is 280 2024-04-07T16:26:28.956 INFO:tasks.rados.rados.0.smithi100.stdout:3032: done (1 left) 2024-04-07T16:26:28.956 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 32 current snap is 280 2024-04-07T16:26:28.956 INFO:tasks.rados.rados.0.smithi100.stdout:3033: seq_num 929 ranges {0=761856,761856=401408} 2024-04-07T16:26:28.960 INFO:tasks.rados.rados.0.smithi100.stdout:3033: writing smithi10066867-32 from 0 to 761856 tid 1 2024-04-07T16:26:28.963 INFO:tasks.rados.rados.0.smithi100.stdout:3033: writing smithi10066867-32 from 761856 to 1163264 tid 2 2024-04-07T16:26:28.963 INFO:tasks.rados.rados.0.smithi100.stdout:3034: delete oid 16 current snap is 280 2024-04-07T16:26:28.973 INFO:tasks.rados.rados.0.smithi100.stdout:3034: done (2 left) 2024-04-07T16:26:28.973 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 40 current snap is 280 2024-04-07T16:26:28.973 INFO:tasks.rados.rados.0.smithi100.stdout:3035: seq_num 931 ranges {0=802816,802816=401408} 2024-04-07T16:26:28.979 INFO:tasks.rados.rados.0.smithi100.stdout:3035: writing smithi10066867-40 from 0 to 802816 tid 1 2024-04-07T16:26:28.982 INFO:tasks.rados.rados.0.smithi100.stdout:3035: writing smithi10066867-40 from 802816 to 1204224 tid 2 2024-04-07T16:26:28.982 INFO:tasks.rados.rados.0.smithi100.stdout:3036: snap_remove snap 277 2024-04-07T16:26:29.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:28 smithi144 bash[66654]: cluster 2024-04-07T16:26:27.958445+0000 mon.a (mon.0) 1004 : cluster [DBG] osdmap e632: 8 total, 8 up, 8 in 2024-04-07T16:26:29.435 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:28 smithi100 bash[74942]: cluster 2024-04-07T16:26:27.958445+0000 mon.a (mon.0) 1004 : cluster [DBG] osdmap e632: 8 total, 8 up, 8 in 2024-04-07T16:26:29.436 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:28 smithi100 bash[76202]: cluster 2024-04-07T16:26:27.958445+0000 mon.a (mon.0) 1004 : cluster [DBG] osdmap e632: 8 total, 8 up, 8 in 2024-04-07T16:26:29.970 INFO:tasks.rados.rados.0.smithi100.stdout:3033: finishing write tid 1 to smithi10066867-32 2024-04-07T16:26:29.970 INFO:tasks.rados.rados.0.smithi100.stdout:3035: finishing write tid 1 to smithi10066867-40 2024-04-07T16:26:29.970 INFO:tasks.rados.rados.0.smithi100.stdout:3033: finishing write tid 2 to smithi10066867-32 2024-04-07T16:26:29.970 INFO:tasks.rados.rados.0.smithi100.stdout:3033: finishing write tid 3 to smithi10066867-32 2024-04-07T16:26:29.970 INFO:tasks.rados.rados.0.smithi100.stdout:3035: finishing write tid 2 to smithi10066867-40 2024-04-07T16:26:29.970 INFO:tasks.rados.rados.0.smithi100.stdout:3035: finishing write tid 3 to smithi10066867-40 2024-04-07T16:26:29.970 INFO:tasks.rados.rados.0.smithi100.stdout:3033: finishing write tid 4 to smithi10066867-32 2024-04-07T16:26:29.970 INFO:tasks.rados.rados.0.smithi100.stdout:3033: oid 32 updating version 0 to 1524 2024-04-07T16:26:29.970 INFO:tasks.rados.rados.0.smithi100.stdout:3033: oid 32 version 1524 is already newer than 1522 2024-04-07T16:26:29.970 INFO:tasks.rados.rados.0.smithi100.stdout:3033: oid 32 version 1524 is already newer than 1523 2024-04-07T16:26:29.970 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 32 v 1524 (ObjNum 929 snap 280 seq_num 929) dirty exists 2024-04-07T16:26:29.971 INFO:tasks.rados.rados.0.smithi100.stdout:3033: left oid 32 (ObjNum 929 snap 280 seq_num 929) 2024-04-07T16:26:29.971 INFO:tasks.rados.rados.0.smithi100.stdout:3035: finishing write tid 4 to smithi10066867-40 2024-04-07T16:26:29.971 INFO:tasks.rados.rados.0.smithi100.stdout:3035: oid 40 updating version 0 to 1613 2024-04-07T16:26:29.971 INFO:tasks.rados.rados.0.smithi100.stdout:3035: oid 40 version 1613 is already newer than 1612 2024-04-07T16:26:29.971 INFO:tasks.rados.rados.0.smithi100.stdout:3035: oid 40 updating version 1613 to 1614 2024-04-07T16:26:29.971 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 40 v 1614 (ObjNum 931 snap 280 seq_num 931) dirty exists 2024-04-07T16:26:29.971 INFO:tasks.rados.rados.0.smithi100.stdout:3035: left oid 40 (ObjNum 931 snap 280 seq_num 931) 2024-04-07T16:26:29.971 INFO:tasks.rados.rados.0.smithi100.stdout:3031: expect (ObjNum 849 snap 261 seq_num 849) 2024-04-07T16:26:30.208 INFO:tasks.rados.rados.0.smithi100.stdout:3031: done (3 left) 2024-04-07T16:26:30.209 INFO:tasks.rados.rados.0.smithi100.stdout:3033: done (2 left) 2024-04-07T16:26:30.209 INFO:tasks.rados.rados.0.smithi100.stdout:3035: done (1 left) 2024-04-07T16:26:30.209 INFO:tasks.rados.rados.0.smithi100.stdout:3036: done (0 left) 2024-04-07T16:26:30.209 INFO:tasks.rados.rados.0.smithi100.stdout:3037: rmattr oid 46 current snap is 280 2024-04-07T16:26:30.209 INFO:tasks.rados.rados.0.smithi100.stdout:3038: delete oid 36 current snap is 280 2024-04-07T16:26:30.212 INFO:tasks.rados.rados.0.smithi100.stdout:3038: done (1 left) 2024-04-07T16:26:30.212 INFO:tasks.rados.rados.0.smithi100.stdout:3039: rollback oid 27 current snap is 280 2024-04-07T16:26:30.212 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 27 to 279 2024-04-07T16:26:30.213 INFO:tasks.rados.rados.0.smithi100.stdout:3040: read oid 14 snap -1 2024-04-07T16:26:30.213 INFO:tasks.rados.rados.0.smithi100.stdout:3040: expect deleted 2024-04-07T16:26:30.213 INFO:tasks.rados.rados.0.smithi100.stdout:3041: read oid 20 snap 279 2024-04-07T16:26:30.213 INFO:tasks.rados.rados.0.smithi100.stdout:3041: expect (ObjNum -1592745616 snap 21978 seq_num 2885806432) 2024-04-07T16:26:30.213 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 45 current snap is 280 2024-04-07T16:26:30.213 INFO:tasks.rados.rados.0.smithi100.stdout:3042: seq_num 933 ranges {0=434176,434176=548864,983040=253952} 2024-04-07T16:26:30.215 INFO:tasks.rados.rados.0.smithi100.stdout:3042: writing smithi10066867-45 from 0 to 434176 tid 1 2024-04-07T16:26:30.219 INFO:tasks.rados.rados.0.smithi100.stdout:3042: writing smithi10066867-45 from 434176 to 983040 tid 2 2024-04-07T16:26:30.221 INFO:tasks.rados.rados.0.smithi100.stdout:3042: writing smithi10066867-45 from 983040 to 1236992 tid 3 2024-04-07T16:26:30.221 INFO:tasks.rados.rados.0.smithi100.stdout:3043: setattr oid 29 current snap is 280 2024-04-07T16:26:30.221 INFO:tasks.rados.rados.0.smithi100.stdout:3041: expect (ObjNum -1592745616 snap 21978 seq_num 2885806432) 2024-04-07T16:26:30.225 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 46 v 1513 (ObjNum 33 snap 0 seq_num 5) dirty exists 2024-04-07T16:26:30.225 INFO:tasks.rados.rados.0.smithi100.stdout:3039: finishing rollback tid 0 to smithi10066867-27 2024-04-07T16:26:30.225 INFO:tasks.rados.rados.0.smithi100.stdout:3039: finishing rollback tid 1 to smithi10066867-27 2024-04-07T16:26:30.225 INFO:tasks.rados.rados.0.smithi100.stdout:3039: finishing rollback tid 2 to smithi10066867-27 2024-04-07T16:26:30.225 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 27 v 1158 (ObjNum 863 snap 261 seq_num 863) dirty exists 2024-04-07T16:26:30.225 INFO:tasks.rados.rados.0.smithi100.stdout:3042: finishing write tid 1 to smithi10066867-45 2024-04-07T16:26:30.225 INFO:tasks.rados.rados.0.smithi100.stdout:3037: done (5 left) 2024-04-07T16:26:30.225 INFO:tasks.rados.rados.0.smithi100.stdout:3039: done (4 left) 2024-04-07T16:26:30.225 INFO:tasks.rados.rados.0.smithi100.stdout:3040: done (3 left) 2024-04-07T16:26:30.225 INFO:tasks.rados.rados.0.smithi100.stdout:3041: done (2 left) 2024-04-07T16:26:30.225 INFO:tasks.rados.rados.0.smithi100.stdout:3044: rmattr oid 9 current snap is 280 2024-04-07T16:26:30.226 INFO:tasks.rados.rados.0.smithi100.stdout:3044: done (2 left) 2024-04-07T16:26:30.226 INFO:tasks.rados.rados.0.smithi100.stdout:3045: copy_from oid 18 from oid 42 current snap is 280 2024-04-07T16:26:30.226 INFO:tasks.rados.rados.0.smithi100.stdout:3046: snap_remove snap 278 2024-04-07T16:26:30.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:29 smithi144 bash[66654]: cluster 2024-04-07T16:26:28.767797+0000 mgr.y (mgr.25175) 301 : cluster [DBG] pgmap v541: 121 pgs: 121 active+clean; 102 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 309 KiB/s rd, 3.0 MiB/s wr, 20 op/s 2024-04-07T16:26:30.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:29 smithi144 bash[66654]: cluster 2024-04-07T16:26:28.970802+0000 mon.a (mon.0) 1005 : cluster [DBG] osdmap e633: 8 total, 8 up, 8 in 2024-04-07T16:26:30.436 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:29 smithi100 bash[74942]: cluster 2024-04-07T16:26:28.767797+0000 mgr.y (mgr.25175) 301 : cluster [DBG] pgmap v541: 121 pgs: 121 active+clean; 102 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 309 KiB/s rd, 3.0 MiB/s wr, 20 op/s 2024-04-07T16:26:30.436 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:29 smithi100 bash[74942]: cluster 2024-04-07T16:26:28.970802+0000 mon.a (mon.0) 1005 : cluster [DBG] osdmap e633: 8 total, 8 up, 8 in 2024-04-07T16:26:30.436 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:29 smithi100 bash[76202]: cluster 2024-04-07T16:26:28.767797+0000 mgr.y (mgr.25175) 301 : cluster [DBG] pgmap v541: 121 pgs: 121 active+clean; 102 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 309 KiB/s rd, 3.0 MiB/s wr, 20 op/s 2024-04-07T16:26:30.436 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:29 smithi100 bash[76202]: cluster 2024-04-07T16:26:28.970802+0000 mon.a (mon.0) 1005 : cluster [DBG] osdmap e633: 8 total, 8 up, 8 in 2024-04-07T16:26:30.978 INFO:tasks.rados.rados.0.smithi100.stdout:3046: done (3 left) 2024-04-07T16:26:30.979 INFO:tasks.rados.rados.0.smithi100.stdout:3047: rollback oid 10 current snap is 280 2024-04-07T16:26:30.979 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 10 to 271 2024-04-07T16:26:30.979 INFO:tasks.rados.rados.0.smithi100.stdout:3048: rollback oid 21 current snap is 280 2024-04-07T16:26:30.979 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 21 to 279 2024-04-07T16:26:30.979 INFO:tasks.rados.rados.0.smithi100.stdout:3049: rollback oid 30 current snap is 280 2024-04-07T16:26:30.979 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 30 to 271 2024-04-07T16:26:30.979 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 8 current snap is 280 2024-04-07T16:26:30.979 INFO:tasks.rados.rados.0.smithi100.stdout:3050: seq_num 934 ranges {548864=450560,999424=434176,1433600=368640} 2024-04-07T16:26:30.987 INFO:tasks.rados.rados.0.smithi100.stdout:3050: writing smithi10066867-8 from 548864 to 999424 tid 1 2024-04-07T16:26:30.990 INFO:tasks.rados.rados.0.smithi100.stdout:3050: writing smithi10066867-8 from 999424 to 1433600 tid 2 2024-04-07T16:26:30.992 INFO:tasks.rados.rados.0.smithi100.stdout:3050: writing smithi10066867-8 from 1433600 to 1802240 tid 3 2024-04-07T16:26:30.993 INFO:tasks.rados.rados.0.smithi100.stdout:3042: finishing write tid 2 to smithi10066867-45 2024-04-07T16:26:30.993 INFO:tasks.rados.rados.0.smithi100.stdout:3042: finishing write tid 3 to smithi10066867-45 2024-04-07T16:26:30.993 INFO:tasks.rados.rados.0.smithi100.stdout:3042: finishing write tid 4 to smithi10066867-45 2024-04-07T16:26:30.993 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 29 v 1225 (ObjNum 625 snap 0 seq_num 1571426586) dirty exists 2024-04-07T16:26:30.993 INFO:tasks.rados.rados.0.smithi100.stdout:3042: finishing write tid 5 to smithi10066867-45 2024-04-07T16:26:30.993 INFO:tasks.rados.rados.0.smithi100.stdout:3042: oid 45 updating version 0 to 1513 2024-04-07T16:26:30.993 INFO:tasks.rados.rados.0.smithi100.stdout:3042: oid 45 updating version 1513 to 1514 2024-04-07T16:26:30.993 INFO:tasks.rados.rados.0.smithi100.stdout:3042: oid 45 updating version 1514 to 1515 2024-04-07T16:26:30.993 INFO:tasks.rados.rados.0.smithi100.stdout:3042: oid 45 version 1515 is already newer than 1512 2024-04-07T16:26:30.993 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 45 v 1515 (ObjNum 933 snap 280 seq_num 933) dirty exists 2024-04-07T16:26:30.993 INFO:tasks.rados.rados.0.smithi100.stdout:3042: left oid 45 (ObjNum 933 snap 280 seq_num 933) 2024-04-07T16:26:30.993 INFO:tasks.rados.rados.0.smithi100.stdout:3045: finishing copy_from racing read to smithi10066867-18 2024-04-07T16:26:30.993 INFO:tasks.rados.rados.0.smithi100.stdout:3045: finishing copy_from to smithi10066867-18 2024-04-07T16:26:30.993 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 18 v 1219 (ObjNum 900 snap 270 seq_num 900) dirty exists 2024-04-07T16:26:30.994 INFO:tasks.rados.rados.0.smithi100.stdout:3042: done (6 left) 2024-04-07T16:26:30.994 INFO:tasks.rados.rados.0.smithi100.stdout:3043: done (5 left) 2024-04-07T16:26:30.994 INFO:tasks.rados.rados.0.smithi100.stdout:3045: done (4 left) 2024-04-07T16:26:30.994 INFO:tasks.rados.rados.0.smithi100.stdout:3051: read oid 48 snap -1 2024-04-07T16:26:30.994 INFO:tasks.rados.rados.0.smithi100.stdout:3051: expect (ObjNum 794 snap 251 seq_num 794) 2024-04-07T16:26:30.994 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 7 current snap is 280 2024-04-07T16:26:30.994 INFO:tasks.rados.rados.0.smithi100.stdout:3052: seq_num 935 ranges {811008=466944,1277952=720896,1998848=81920} 2024-04-07T16:26:31.000 INFO:tasks.rados.rados.0.smithi100.stdout:3052: writing smithi10066867-7 from 811008 to 1277952 tid 1 2024-04-07T16:26:31.005 INFO:tasks.rados.rados.0.smithi100.stdout:3052: writing smithi10066867-7 from 1277952 to 1998848 tid 2 2024-04-07T16:26:31.006 INFO:tasks.rados.rados.0.smithi100.stdout:3052: writing smithi10066867-7 from 1998848 to 2080768 tid 3 2024-04-07T16:26:31.006 INFO:tasks.rados.rados.0.smithi100.stdout:3047: finishing rollback tid 0 to smithi10066867-10 2024-04-07T16:26:31.007 INFO:tasks.rados.rados.0.smithi100.stdout:3048: finishing rollback tid 0 to smithi10066867-21 2024-04-07T16:26:31.007 INFO:tasks.rados.rados.0.smithi100.stdout:3047: finishing rollback tid 1 to smithi10066867-10 2024-04-07T16:26:31.007 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 10 v 1448 (ObjNum 273 snap 0 seq_num 0) dirty dne 2024-04-07T16:26:31.007 INFO:tasks.rados.rados.0.smithi100.stdout:3047: done (5 left) 2024-04-07T16:26:31.007 INFO:tasks.rados.rados.0.smithi100.stdout:3053: setattr oid 50 current snap is 280 2024-04-07T16:26:31.008 INFO:tasks.rados.rados.0.smithi100.stdout:3048: finishing rollback tid 1 to smithi10066867-21 2024-04-07T16:26:31.008 INFO:tasks.rados.rados.0.smithi100.stdout:3048: finishing rollback tid 2 to smithi10066867-21 2024-04-07T16:26:31.008 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 21 v 1198 (ObjNum 794 snap 251 seq_num 794) dirty exists 2024-04-07T16:26:31.010 INFO:tasks.rados.rados.0.smithi100.stdout:3048: done (5 left) 2024-04-07T16:26:31.010 INFO:tasks.rados.rados.0.smithi100.stdout:3054: delete oid 25 current snap is 280 2024-04-07T16:26:31.013 INFO:tasks.rados.rados.0.smithi100.stdout:3051: expect (ObjNum 794 snap 251 seq_num 794) 2024-04-07T16:26:31.081 INFO:tasks.rados.rados.0.smithi100.stdout:3050: finishing write tid 1 to smithi10066867-8 2024-04-07T16:26:31.082 INFO:tasks.rados.rados.0.smithi100.stdout:3050: finishing write tid 2 to smithi10066867-8 2024-04-07T16:26:31.082 INFO:tasks.rados.rados.0.smithi100.stdout:3050: finishing write tid 3 to smithi10066867-8 2024-04-07T16:26:31.082 INFO:tasks.rados.rados.0.smithi100.stdout:3050: finishing write tid 4 to smithi10066867-8 2024-04-07T16:26:31.082 INFO:tasks.rados.rados.0.smithi100.stdout:3052: finishing write tid 1 to smithi10066867-7 2024-04-07T16:26:31.082 INFO:tasks.rados.rados.0.smithi100.stdout:3052: finishing write tid 2 to smithi10066867-7 2024-04-07T16:26:31.082 INFO:tasks.rados.rados.0.smithi100.stdout:3050: finishing write tid 5 to smithi10066867-8 2024-04-07T16:26:31.082 INFO:tasks.rados.rados.0.smithi100.stdout:3050: oid 8 updating version 0 to 1514 2024-04-07T16:26:31.082 INFO:tasks.rados.rados.0.smithi100.stdout:3050: oid 8 updating version 1514 to 1516 2024-04-07T16:26:31.082 INFO:tasks.rados.rados.0.smithi100.stdout:3050: oid 8 version 1516 is already newer than 1515 2024-04-07T16:26:31.082 INFO:tasks.rados.rados.0.smithi100.stdout:3050: oid 8 updating version 1516 to 1517 2024-04-07T16:26:31.083 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 8 v 1517 (ObjNum 934 snap 280 seq_num 934) dirty exists 2024-04-07T16:26:31.083 INFO:tasks.rados.rados.0.smithi100.stdout:3050: left oid 8 (ObjNum 934 snap 280 seq_num 934) 2024-04-07T16:26:31.083 INFO:tasks.rados.rados.0.smithi100.stdout:3049: finishing rollback tid 0 to smithi10066867-30 2024-04-07T16:26:31.083 INFO:tasks.rados.rados.0.smithi100.stdout:3050: done (5 left) 2024-04-07T16:26:31.083 INFO:tasks.rados.rados.0.smithi100.stdout:3051: done (4 left) 2024-04-07T16:26:31.083 INFO:tasks.rados.rados.0.smithi100.stdout:3054: done (3 left) 2024-04-07T16:26:31.083 INFO:tasks.rados.rados.0.smithi100.stdout:3055: snap_remove snap 273 2024-04-07T16:26:31.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:30 smithi144 bash[66654]: cluster 2024-04-07T16:26:29.979080+0000 mon.a (mon.0) 1006 : cluster [DBG] osdmap e634: 8 total, 8 up, 8 in 2024-04-07T16:26:31.436 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:30 smithi100 bash[74942]: cluster 2024-04-07T16:26:29.979080+0000 mon.a (mon.0) 1006 : cluster [DBG] osdmap e634: 8 total, 8 up, 8 in 2024-04-07T16:26:31.436 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:30 smithi100 bash[76202]: cluster 2024-04-07T16:26:29.979080+0000 mon.a (mon.0) 1006 : cluster [DBG] osdmap e634: 8 total, 8 up, 8 in 2024-04-07T16:26:31.902 INFO:tasks.rados.rados.0.smithi100.stdout:3055: done (3 left) 2024-04-07T16:26:31.902 INFO:tasks.rados.rados.0.smithi100.stdout:3056: read oid 22 snap -1 2024-04-07T16:26:31.902 INFO:tasks.rados.rados.0.smithi100.stdout:3056: expect (ObjNum 906 snap 274 seq_num 906) 2024-04-07T16:26:31.903 INFO:tasks.rados.rados.0.smithi100.stdout:3057: read oid 44 snap -1 2024-04-07T16:26:31.903 INFO:tasks.rados.rados.0.smithi100.stdout:3057: expect deleted 2024-04-07T16:26:31.903 INFO:tasks.rados.rados.0.smithi100.stdout:3058: read oid 29 snap -1 2024-04-07T16:26:31.903 INFO:tasks.rados.rados.0.smithi100.stdout:3058: expect (ObjNum 0 snap 0 seq_num 0) 2024-04-07T16:26:31.903 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 17 current snap is 280 2024-04-07T16:26:31.903 INFO:tasks.rados.rados.0.smithi100.stdout:3059: seq_num 937 ranges {0=499712,499712=532480,1032192=270336} 2024-04-07T16:26:31.905 INFO:tasks.rados.rados.0.smithi100.stdout:3059: writing smithi10066867-17 from 0 to 499712 tid 1 2024-04-07T16:26:31.909 INFO:tasks.rados.rados.0.smithi100.stdout:3059: writing smithi10066867-17 from 499712 to 1032192 tid 2 2024-04-07T16:26:31.911 INFO:tasks.rados.rados.0.smithi100.stdout:3059: writing smithi10066867-17 from 1032192 to 1302528 tid 3 2024-04-07T16:26:31.912 INFO:tasks.rados.rados.0.smithi100.stdout:3049: finishing rollback tid 1 to smithi10066867-30 2024-04-07T16:26:31.912 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 50 v 1516 (ObjNum 919 snap 277 seq_num 919) dirty exists 2024-04-07T16:26:31.912 INFO:tasks.rados.rados.0.smithi100.stdout:3049: finishing rollback tid 2 to smithi10066867-30 2024-04-07T16:26:31.912 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 30 v 1228 (ObjNum 880 snap 264 seq_num 880) dirty exists 2024-04-07T16:26:31.912 INFO:tasks.rados.rados.0.smithi100.stdout:3052: finishing write tid 3 to smithi10066867-7 2024-04-07T16:26:31.912 INFO:tasks.rados.rados.0.smithi100.stdout:3052: finishing write tid 4 to smithi10066867-7 2024-04-07T16:26:31.912 INFO:tasks.rados.rados.0.smithi100.stdout:3052: finishing write tid 5 to smithi10066867-7 2024-04-07T16:26:31.912 INFO:tasks.rados.rados.0.smithi100.stdout:3052: oid 7 updating version 0 to 1161 2024-04-07T16:26:31.912 INFO:tasks.rados.rados.0.smithi100.stdout:3052: oid 7 version 1161 is already newer than 1159 2024-04-07T16:26:31.912 INFO:tasks.rados.rados.0.smithi100.stdout:3052: oid 7 updating version 1161 to 1162 2024-04-07T16:26:31.912 INFO:tasks.rados.rados.0.smithi100.stdout:3052: oid 7 version 1162 is already newer than 1160 2024-04-07T16:26:31.912 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 7 v 1162 (ObjNum 935 snap 280 seq_num 935) dirty exists 2024-04-07T16:26:31.912 INFO:tasks.rados.rados.0.smithi100.stdout:3052: left oid 7 (ObjNum 935 snap 280 seq_num 935) 2024-04-07T16:26:31.913 INFO:tasks.rados.rados.0.smithi100.stdout:3049: done (6 left) 2024-04-07T16:26:31.913 INFO:tasks.rados.rados.0.smithi100.stdout:3052: done (5 left) 2024-04-07T16:26:31.913 INFO:tasks.rados.rados.0.smithi100.stdout:3053: done (4 left) 2024-04-07T16:26:31.913 INFO:tasks.rados.rados.0.smithi100.stdout:3060: snap_create 2024-04-07T16:26:31.913 INFO:tasks.rados.rados.0.smithi100.stdout:3058: expect (ObjNum 0 snap 0 seq_num 0) 2024-04-07T16:26:31.945 INFO:tasks.rados.rados.0.smithi100.stdout:3059: finishing write tid 1 to smithi10066867-17 2024-04-07T16:26:31.945 INFO:tasks.rados.rados.0.smithi100.stdout:3059: finishing write tid 2 to smithi10066867-17 2024-04-07T16:26:31.945 INFO:tasks.rados.rados.0.smithi100.stdout:3059: finishing write tid 3 to smithi10066867-17 2024-04-07T16:26:31.945 INFO:tasks.rados.rados.0.smithi100.stdout:3059: finishing write tid 4 to smithi10066867-17 2024-04-07T16:26:31.948 INFO:tasks.rados.rados.0.smithi100.stdout:3056: expect (ObjNum 906 snap 274 seq_num 906) 2024-04-07T16:26:32.304 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:31 smithi144 bash[66654]: cluster 2024-04-07T16:26:30.768455+0000 mgr.y (mgr.25175) 302 : cluster [DBG] pgmap v544: 121 pgs: 121 active+clean; 108 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 11 MiB/s rd, 4.8 MiB/s wr, 43 op/s 2024-04-07T16:26:32.304 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:31 smithi144 bash[66654]: cluster 2024-04-07T16:26:30.994051+0000 mon.a (mon.0) 1007 : cluster [DBG] osdmap e635: 8 total, 8 up, 8 in 2024-04-07T16:26:32.304 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:31 smithi144 bash[66654]: audit 2024-04-07T16:26:31.727122+0000 mon.a (mon.0) 1008 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:26:32.304 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:31 smithi144 bash[66654]: audit 2024-04-07T16:26:31.814812+0000 mon.a (mon.0) 1009 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:26:32.304 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:31 smithi144 bash[66654]: audit 2024-04-07T16:26:31.829083+0000 mon.a (mon.0) 1010 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:26:32.304 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:31 smithi144 bash[66654]: cluster 2024-04-07T16:26:31.915906+0000 mon.a (mon.0) 1011 : cluster [DBG] osdmap e636: 8 total, 8 up, 8 in 2024-04-07T16:26:32.436 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:31 smithi100 bash[74942]: cluster 2024-04-07T16:26:30.768455+0000 mgr.y (mgr.25175) 302 : cluster [DBG] pgmap v544: 121 pgs: 121 active+clean; 108 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 11 MiB/s rd, 4.8 MiB/s wr, 43 op/s 2024-04-07T16:26:32.436 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:31 smithi100 bash[74942]: cluster 2024-04-07T16:26:30.994051+0000 mon.a (mon.0) 1007 : cluster [DBG] osdmap e635: 8 total, 8 up, 8 in 2024-04-07T16:26:32.436 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:31 smithi100 bash[74942]: audit 2024-04-07T16:26:31.727122+0000 mon.a (mon.0) 1008 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:26:32.436 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:31 smithi100 bash[74942]: audit 2024-04-07T16:26:31.814812+0000 mon.a (mon.0) 1009 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:26:32.436 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:31 smithi100 bash[74942]: audit 2024-04-07T16:26:31.829083+0000 mon.a (mon.0) 1010 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:26:32.436 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:31 smithi100 bash[74942]: cluster 2024-04-07T16:26:31.915906+0000 mon.a (mon.0) 1011 : cluster [DBG] osdmap e636: 8 total, 8 up, 8 in 2024-04-07T16:26:32.436 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:31 smithi100 bash[76202]: cluster 2024-04-07T16:26:30.768455+0000 mgr.y (mgr.25175) 302 : cluster [DBG] pgmap v544: 121 pgs: 121 active+clean; 108 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 11 MiB/s rd, 4.8 MiB/s wr, 43 op/s 2024-04-07T16:26:32.436 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:31 smithi100 bash[76202]: cluster 2024-04-07T16:26:30.994051+0000 mon.a (mon.0) 1007 : cluster [DBG] osdmap e635: 8 total, 8 up, 8 in 2024-04-07T16:26:32.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:31 smithi100 bash[76202]: audit 2024-04-07T16:26:31.727122+0000 mon.a (mon.0) 1008 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:26:32.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:31 smithi100 bash[76202]: audit 2024-04-07T16:26:31.814812+0000 mon.a (mon.0) 1009 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:26:32.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:31 smithi100 bash[76202]: audit 2024-04-07T16:26:31.829083+0000 mon.a (mon.0) 1010 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:26:32.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:31 smithi100 bash[76202]: cluster 2024-04-07T16:26:31.915906+0000 mon.a (mon.0) 1011 : cluster [DBG] osdmap e636: 8 total, 8 up, 8 in 2024-04-07T16:26:32.574 INFO:tasks.rados.rados.0.smithi100.stdout:3059: finishing write tid 5 to smithi10066867-17 2024-04-07T16:26:32.575 INFO:tasks.rados.rados.0.smithi100.stdout:3059: oid 17 updating version 0 to 1218 2024-04-07T16:26:32.575 INFO:tasks.rados.rados.0.smithi100.stdout:3059: oid 17 version 1218 is already newer than 1216 2024-04-07T16:26:32.575 INFO:tasks.rados.rados.0.smithi100.stdout:3059: oid 17 version 1218 is already newer than 1217 2024-04-07T16:26:32.575 INFO:tasks.rados.rados.0.smithi100.stdout:3059: oid 17 version 1218 is already newer than 1215 2024-04-07T16:26:32.575 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 17 v 1218 (ObjNum 937 snap 280 seq_num 937) dirty exists 2024-04-07T16:26:32.575 INFO:tasks.rados.rados.0.smithi100.stdout:3059: left oid 17 (ObjNum 937 snap 280 seq_num 937) 2024-04-07T16:26:32.606 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:26:32 smithi144 bash[59794]: ts=2024-04-07T16:26:32.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:26:32.900 INFO:tasks.rados.rados.0.smithi100.stdout:3056: done (4 left) 2024-04-07T16:26:32.900 INFO:tasks.rados.rados.0.smithi100.stdout:3057: done (3 left) 2024-04-07T16:26:32.900 INFO:tasks.rados.rados.0.smithi100.stdout:3058: done (2 left) 2024-04-07T16:26:32.900 INFO:tasks.rados.rados.0.smithi100.stdout:3059: done (1 left) 2024-04-07T16:26:32.900 INFO:tasks.rados.rados.0.smithi100.stdout:3060: done (0 left) 2024-04-07T16:26:32.900 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 11 current snap is 281 2024-04-07T16:26:32.900 INFO:tasks.rados.rados.0.smithi100.stdout:3061: seq_num 938 ranges {4317184=516096,4833280=417792,5251072=385024} 2024-04-07T16:26:32.956 INFO:tasks.rados.rados.0.smithi100.stdout:3061: writing smithi10066867-11 from 4317184 to 4833280 tid 1 2024-04-07T16:26:32.961 INFO:tasks.rados.rados.0.smithi100.stdout:3061: writing smithi10066867-11 from 4833280 to 5251072 tid 2 2024-04-07T16:26:32.965 INFO:tasks.rados.rados.0.smithi100.stdout:3061: writing smithi10066867-11 from 5251072 to 5636096 tid 3 2024-04-07T16:26:32.965 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 31 current snap is 281 2024-04-07T16:26:32.965 INFO:tasks.rados.rados.0.smithi100.stdout:3062: seq_num 939 ranges {0=532480,532480=704512,1236992=98304} 2024-04-07T16:26:32.970 INFO:tasks.rados.rados.0.smithi100.stdout:3062: writing smithi10066867-31 from 0 to 532480 tid 1 2024-04-07T16:26:32.977 INFO:tasks.rados.rados.0.smithi100.stdout:3062: writing smithi10066867-31 from 532480 to 1236992 tid 2 2024-04-07T16:26:32.979 INFO:tasks.rados.rados.0.smithi100.stdout:3062: writing smithi10066867-31 from 1236992 to 1335296 tid 3 2024-04-07T16:26:32.979 INFO:tasks.rados.rados.0.smithi100.stdout:3063: rollback oid 42 current snap is 281 2024-04-07T16:26:32.979 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 42 to 280 2024-04-07T16:26:32.979 INFO:tasks.rados.rados.0.smithi100.stdout:3061: finishing write tid 1 to smithi10066867-11 2024-04-07T16:26:32.979 INFO:tasks.rados.rados.0.smithi100.stdout:3061: finishing write tid 2 to smithi10066867-11 2024-04-07T16:26:32.979 INFO:tasks.rados.rados.0.smithi100.stdout:3061: finishing write tid 3 to smithi10066867-11 2024-04-07T16:26:32.979 INFO:tasks.rados.rados.0.smithi100.stdout:3061: finishing write tid 4 to smithi10066867-11 2024-04-07T16:26:32.979 INFO:tasks.rados.rados.0.smithi100.stdout:3061: finishing write tid 5 to smithi10066867-11 2024-04-07T16:26:32.979 INFO:tasks.rados.rados.0.smithi100.stdout:3061: oid 11 updating version 0 to 1677 2024-04-07T16:26:32.979 INFO:tasks.rados.rados.0.smithi100.stdout:3061: oid 11 updating version 1677 to 1680 2024-04-07T16:26:32.980 INFO:tasks.rados.rados.0.smithi100.stdout:3061: oid 11 version 1680 is already newer than 1678 2024-04-07T16:26:32.980 INFO:tasks.rados.rados.0.smithi100.stdout:3061: oid 11 version 1680 is already newer than 1679 2024-04-07T16:26:32.980 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 11 v 1680 (ObjNum 938 snap 281 seq_num 938) dirty exists 2024-04-07T16:26:32.980 INFO:tasks.rados.rados.0.smithi100.stdout:3061: left oid 11 (ObjNum 938 snap 281 seq_num 938) 2024-04-07T16:26:32.980 INFO:tasks.rados.rados.0.smithi100.stdout:3062: finishing write tid 1 to smithi10066867-31 2024-04-07T16:26:32.980 INFO:tasks.rados.rados.0.smithi100.stdout:3061: done (2 left) 2024-04-07T16:26:32.980 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 3 current snap is 281 2024-04-07T16:26:32.980 INFO:tasks.rados.rados.0.smithi100.stdout:3064: seq_num 940 ranges {3301376=548864,3850240=589824,4440064=212992} 2024-04-07T16:26:33.002 INFO:tasks.rados.rados.0.smithi100.stdout:3064: writing smithi10066867-3 from 3301376 to 3850240 tid 1 2024-04-07T16:26:33.006 INFO:tasks.rados.rados.0.smithi100.stdout:3064: writing smithi10066867-3 from 3850240 to 4440064 tid 2 2024-04-07T16:26:33.007 INFO:tasks.rados.rados.0.smithi100.stdout:3064: writing smithi10066867-3 from 4440064 to 4653056 tid 3 2024-04-07T16:26:33.007 INFO:tasks.rados.rados.0.smithi100.stdout:3065: snap_create 2024-04-07T16:26:33.007 INFO:tasks.rados.rados.0.smithi100.stdout:3062: finishing write tid 2 to smithi10066867-31 2024-04-07T16:26:33.008 INFO:tasks.rados.rados.0.smithi100.stdout:3062: finishing write tid 3 to smithi10066867-31 2024-04-07T16:26:33.008 INFO:tasks.rados.rados.0.smithi100.stdout:3063: finishing rollback tid 0 to smithi10066867-42 2024-04-07T16:26:33.008 INFO:tasks.rados.rados.0.smithi100.stdout:3062: finishing write tid 4 to smithi10066867-31 2024-04-07T16:26:33.008 INFO:tasks.rados.rados.0.smithi100.stdout:3063: finishing rollback tid 1 to smithi10066867-42 2024-04-07T16:26:33.008 INFO:tasks.rados.rados.0.smithi100.stdout:3063: finishing rollback tid 2 to smithi10066867-42 2024-04-07T16:26:33.008 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 42 v 1232 (ObjNum 900 snap 270 seq_num 900) dirty exists 2024-04-07T16:26:33.008 INFO:tasks.rados.rados.0.smithi100.stdout:3062: finishing write tid 5 to smithi10066867-31 2024-04-07T16:26:33.008 INFO:tasks.rados.rados.0.smithi100.stdout:3062: oid 31 updating version 0 to 1126 2024-04-07T16:26:33.008 INFO:tasks.rados.rados.0.smithi100.stdout:3062: oid 31 version 1126 is already newer than 1125 2024-04-07T16:26:33.008 INFO:tasks.rados.rados.0.smithi100.stdout:3062: oid 31 updating version 1126 to 1128 2024-04-07T16:26:33.008 INFO:tasks.rados.rados.0.smithi100.stdout:3062: oid 31 version 1128 is already newer than 1127 2024-04-07T16:26:33.008 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 31 v 1128 (ObjNum 939 snap 281 seq_num 939) dirty exists 2024-04-07T16:26:33.008 INFO:tasks.rados.rados.0.smithi100.stdout:3062: left oid 31 (ObjNum 939 snap 281 seq_num 939) 2024-04-07T16:26:33.009 INFO:tasks.rados.rados.0.smithi100.stdout:3064: finishing write tid 1 to smithi10066867-3 2024-04-07T16:26:33.011 INFO:tasks.rados.rados.0.smithi100.stdout:3064: finishing write tid 2 to smithi10066867-3 2024-04-07T16:26:33.012 INFO:tasks.rados.rados.0.smithi100.stdout:3064: finishing write tid 3 to smithi10066867-3 2024-04-07T16:26:33.012 INFO:tasks.rados.rados.0.smithi100.stdout:3064: finishing write tid 4 to smithi10066867-3 2024-04-07T16:26:33.014 INFO:tasks.rados.rados.0.smithi100.stdout:3064: finishing write tid 5 to smithi10066867-3 2024-04-07T16:26:33.015 INFO:tasks.rados.rados.0.smithi100.stdout:3064: oid 3 updating version 0 to 1138 2024-04-07T16:26:33.015 INFO:tasks.rados.rados.0.smithi100.stdout:3064: oid 3 version 1138 is already newer than 1137 2024-04-07T16:26:33.015 INFO:tasks.rados.rados.0.smithi100.stdout:3064: oid 3 version 1138 is already newer than 1136 2024-04-07T16:26:33.015 INFO:tasks.rados.rados.0.smithi100.stdout:3064: oid 3 version 1138 is already newer than 1135 2024-04-07T16:26:33.015 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 3 v 1138 (ObjNum 940 snap 281 seq_num 940) dirty exists 2024-04-07T16:26:33.015 INFO:tasks.rados.rados.0.smithi100.stdout:3064: left oid 3 (ObjNum 940 snap 281 seq_num 940) 2024-04-07T16:26:33.981 INFO:tasks.rados.rados.0.smithi100.stdout:3062: done (3 left) 2024-04-07T16:26:33.982 INFO:tasks.rados.rados.0.smithi100.stdout:3063: done (2 left) 2024-04-07T16:26:33.982 INFO:tasks.rados.rados.0.smithi100.stdout:3064: done (1 left) 2024-04-07T16:26:33.982 INFO:tasks.rados.rados.0.smithi100.stdout:3065: done (0 left) 2024-04-07T16:26:33.982 INFO:tasks.rados.rados.0.smithi100.stdout:3066: snap_remove snap 280 2024-04-07T16:26:34.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:33 smithi100 bash[74942]: cluster 2024-04-07T16:26:32.769555+0000 mgr.y (mgr.25175) 303 : cluster [DBG] pgmap v547: 121 pgs: 1 active+clean+snaptrim, 120 active+clean; 110 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 14 MiB/s rd, 3.0 MiB/s wr, 36 op/s 2024-04-07T16:26:34.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:33 smithi100 bash[74942]: cluster 2024-04-07T16:26:32.915725+0000 mon.a (mon.0) 1012 : cluster [DBG] osdmap e637: 8 total, 8 up, 8 in 2024-04-07T16:26:34.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:33 smithi100 bash[74942]: audit 2024-04-07T16:26:33.031046+0000 mon.a (mon.0) 1013 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:26:34.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:33 smithi100 bash[74942]: audit 2024-04-07T16:26:33.041638+0000 mon.a (mon.0) 1014 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:26:34.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:33 smithi100 bash[76202]: cluster 2024-04-07T16:26:32.769555+0000 mgr.y (mgr.25175) 303 : cluster [DBG] pgmap v547: 121 pgs: 1 active+clean+snaptrim, 120 active+clean; 110 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 14 MiB/s rd, 3.0 MiB/s wr, 36 op/s 2024-04-07T16:26:34.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:33 smithi100 bash[76202]: cluster 2024-04-07T16:26:32.915725+0000 mon.a (mon.0) 1012 : cluster [DBG] osdmap e637: 8 total, 8 up, 8 in 2024-04-07T16:26:34.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:33 smithi100 bash[76202]: audit 2024-04-07T16:26:33.031046+0000 mon.a (mon.0) 1013 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:26:34.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:33 smithi100 bash[76202]: audit 2024-04-07T16:26:33.041638+0000 mon.a (mon.0) 1014 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:26:34.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:33 smithi144 bash[66654]: cluster 2024-04-07T16:26:32.769555+0000 mgr.y (mgr.25175) 303 : cluster [DBG] pgmap v547: 121 pgs: 1 active+clean+snaptrim, 120 active+clean; 110 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 14 MiB/s rd, 3.0 MiB/s wr, 36 op/s 2024-04-07T16:26:34.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:33 smithi144 bash[66654]: cluster 2024-04-07T16:26:32.915725+0000 mon.a (mon.0) 1012 : cluster [DBG] osdmap e637: 8 total, 8 up, 8 in 2024-04-07T16:26:34.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:33 smithi144 bash[66654]: audit 2024-04-07T16:26:33.031046+0000 mon.a (mon.0) 1013 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:26:34.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:33 smithi144 bash[66654]: audit 2024-04-07T16:26:33.041638+0000 mon.a (mon.0) 1014 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:26:35.000 INFO:tasks.rados.rados.0.smithi100.stdout:3066: done (0 left) 2024-04-07T16:26:35.000 INFO:tasks.rados.rados.0.smithi100.stdout:3067: setattr oid 40 current snap is 282 2024-04-07T16:26:35.005 INFO:tasks.rados.rados.0.smithi100.stdout:3068: read oid 41 snap 281 2024-04-07T16:26:35.005 INFO:tasks.rados.rados.0.smithi100.stdout:3068: expect deleted 2024-04-07T16:26:35.005 INFO:tasks.rados.rados.0.smithi100.stdout:3069: snap_remove snap 279 2024-04-07T16:26:35.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:34 smithi144 bash[66654]: cluster 2024-04-07T16:26:33.996448+0000 mon.a (mon.0) 1015 : cluster [DBG] osdmap e638: 8 total, 8 up, 8 in 2024-04-07T16:26:35.436 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:35 smithi100 bash[74942]: cluster 2024-04-07T16:26:33.996448+0000 mon.a (mon.0) 1015 : cluster [DBG] osdmap e638: 8 total, 8 up, 8 in 2024-04-07T16:26:35.436 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:35 smithi100 bash[76202]: cluster 2024-04-07T16:26:33.996448+0000 mon.a (mon.0) 1015 : cluster [DBG] osdmap e638: 8 total, 8 up, 8 in 2024-04-07T16:26:36.007 INFO:tasks.rados.rados.0.smithi100.stdout:3069: done (2 left) 2024-04-07T16:26:36.007 INFO:tasks.rados.rados.0.smithi100.stdout:3070: copy_from oid 27 from oid 44 current snap is 282 2024-04-07T16:26:36.007 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 40 v 1615 (ObjNum 931 snap 280 seq_num 931) dirty exists 2024-04-07T16:26:36.007 INFO:tasks.rados.rados.0.smithi100.stdout:3067: done (2 left) 2024-04-07T16:26:36.007 INFO:tasks.rados.rados.0.smithi100.stdout:3068: done (1 left) 2024-04-07T16:26:36.007 INFO:tasks.rados.rados.0.smithi100.stdout:3071: rmattr oid 48 current snap is 282 2024-04-07T16:26:36.007 INFO:tasks.rados.rados.0.smithi100.stdout:3072: rmattr oid 39 current snap is 282 2024-04-07T16:26:36.007 INFO:tasks.rados.rados.0.smithi100.stdout:3072: done (2 left) 2024-04-07T16:26:36.007 INFO:tasks.rados.rados.0.smithi100.stdout:3073: read oid 33 snap 271 2024-04-07T16:26:36.007 INFO:tasks.rados.rados.0.smithi100.stdout:3073: expect (ObjNum 753 snap 235 seq_num 753) 2024-04-07T16:26:36.007 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 23 current snap is 282 2024-04-07T16:26:36.007 INFO:tasks.rados.rados.0.smithi100.stdout:3074: seq_num 941 ranges {0=565248,565248=475136,1040384=327680} 2024-04-07T16:26:36.010 INFO:tasks.rados.rados.0.smithi100.stdout:3074: writing smithi10066867-23 from 0 to 565248 tid 1 2024-04-07T16:26:36.014 INFO:tasks.rados.rados.0.smithi100.stdout:3074: writing smithi10066867-23 from 565248 to 1040384 tid 2 2024-04-07T16:26:36.016 INFO:tasks.rados.rados.0.smithi100.stdout:3074: writing smithi10066867-23 from 1040384 to 1368064 tid 3 2024-04-07T16:26:36.016 INFO:tasks.rados.rados.0.smithi100.stdout:3075: snap_remove snap 281 2024-04-07T16:26:36.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:35 smithi144 bash[66654]: cluster 2024-04-07T16:26:34.770428+0000 mgr.y (mgr.25175) 304 : cluster [DBG] pgmap v550: 121 pgs: 1 active+clean+snaptrim, 120 active+clean; 116 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 10 MiB/s rd, 2.6 MiB/s wr, 29 op/s 2024-04-07T16:26:36.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:35 smithi144 bash[66654]: cluster 2024-04-07T16:26:35.008041+0000 mon.a (mon.0) 1016 : cluster [DBG] osdmap e639: 8 total, 8 up, 8 in 2024-04-07T16:26:36.436 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:36 smithi100 bash[74942]: cluster 2024-04-07T16:26:34.770428+0000 mgr.y (mgr.25175) 304 : cluster [DBG] pgmap v550: 121 pgs: 1 active+clean+snaptrim, 120 active+clean; 116 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 10 MiB/s rd, 2.6 MiB/s wr, 29 op/s 2024-04-07T16:26:36.436 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:36 smithi100 bash[74942]: cluster 2024-04-07T16:26:35.008041+0000 mon.a (mon.0) 1016 : cluster [DBG] osdmap e639: 8 total, 8 up, 8 in 2024-04-07T16:26:36.436 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:36 smithi100 bash[76202]: cluster 2024-04-07T16:26:34.770428+0000 mgr.y (mgr.25175) 304 : cluster [DBG] pgmap v550: 121 pgs: 1 active+clean+snaptrim, 120 active+clean; 116 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 10 MiB/s rd, 2.6 MiB/s wr, 29 op/s 2024-04-07T16:26:36.436 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:36 smithi100 bash[76202]: cluster 2024-04-07T16:26:35.008041+0000 mon.a (mon.0) 1016 : cluster [DBG] osdmap e639: 8 total, 8 up, 8 in 2024-04-07T16:26:37.009 INFO:tasks.rados.rados.0.smithi100.stdout:3075: done (4 left) 2024-04-07T16:26:37.009 INFO:tasks.rados.rados.0.smithi100.stdout:3076: rmattr oid 16 current snap is 282 2024-04-07T16:26:37.009 INFO:tasks.rados.rados.0.smithi100.stdout:3076: done (4 left) 2024-04-07T16:26:37.009 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 28 current snap is 282 2024-04-07T16:26:37.009 INFO:tasks.rados.rados.0.smithi100.stdout:3077: seq_num 942 ranges {2555904=581632,3137536=761856,3899392=40960} 2024-04-07T16:26:37.044 INFO:tasks.rados.rados.0.smithi100.stdout:3077: writing smithi10066867-28 from 2555904 to 3137536 tid 1 2024-04-07T16:26:37.052 INFO:tasks.rados.rados.0.smithi100.stdout:3077: writing smithi10066867-28 from 3137536 to 3899392 tid 2 2024-04-07T16:26:37.053 INFO:tasks.rados.rados.0.smithi100.stdout:3077: writing smithi10066867-28 from 3899392 to 3940352 tid 3 2024-04-07T16:26:37.053 INFO:tasks.rados.rados.0.smithi100.stdout:3078: read oid 21 snap -1 2024-04-07T16:26:37.053 INFO:tasks.rados.rados.0.smithi100.stdout:3078: expect (ObjNum 794 snap 251 seq_num 794) 2024-04-07T16:26:37.053 INFO:tasks.rados.rados.0.smithi100.stdout:3079: copy_from oid 13 from oid 5 current snap is 282 2024-04-07T16:26:37.053 INFO:tasks.rados.rados.0.smithi100.stdout:3070: finishing copy_from to smithi10066867-27 2024-04-07T16:26:37.053 INFO:tasks.rados.rados.0.smithi100.stdout:3070: got expected ENOENT (src dne) 2024-04-07T16:26:37.053 INFO:tasks.rados.rados.0.smithi100.stdout:3070: finishing copy_from racing read to smithi10066867-27 2024-04-07T16:26:37.053 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 48 v 1129 (ObjNum 794 snap 251 seq_num 794) dirty exists 2024-04-07T16:26:37.053 INFO:tasks.rados.rados.0.smithi100.stdout:3073: expect (ObjNum 753 snap 235 seq_num 753) 2024-04-07T16:26:37.087 INFO:tasks.rados.rados.0.smithi100.stdout:3074: finishing write tid 1 to smithi10066867-23 2024-04-07T16:26:37.087 INFO:tasks.rados.rados.0.smithi100.stdout:3074: finishing write tid 2 to smithi10066867-23 2024-04-07T16:26:37.087 INFO:tasks.rados.rados.0.smithi100.stdout:3070: done (6 left) 2024-04-07T16:26:37.087 INFO:tasks.rados.rados.0.smithi100.stdout:3071: done (5 left) 2024-04-07T16:26:37.087 INFO:tasks.rados.rados.0.smithi100.stdout:3073: done (4 left) 2024-04-07T16:26:37.087 INFO:tasks.rados.rados.0.smithi100.stdout:3080: copy_from oid 20 from oid 30 current snap is 282 2024-04-07T16:26:37.087 INFO:tasks.rados.rados.0.smithi100.stdout:3074: finishing write tid 3 to smithi10066867-23 2024-04-07T16:26:37.087 INFO:tasks.rados.rados.0.smithi100.stdout:3074: finishing write tid 4 to smithi10066867-23 2024-04-07T16:26:37.087 INFO:tasks.rados.rados.0.smithi100.stdout:3074: finishing write tid 5 to smithi10066867-23 2024-04-07T16:26:37.087 INFO:tasks.rados.rados.0.smithi100.stdout:3074: oid 23 updating version 0 to 1684 2024-04-07T16:26:37.087 INFO:tasks.rados.rados.0.smithi100.stdout:3074: oid 23 updating version 1684 to 1685 2024-04-07T16:26:37.088 INFO:tasks.rados.rados.0.smithi100.stdout:3074: oid 23 updating version 1685 to 1686 2024-04-07T16:26:37.088 INFO:tasks.rados.rados.0.smithi100.stdout:3074: oid 23 version 1686 is already newer than 1683 2024-04-07T16:26:37.088 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 23 v 1686 (ObjNum 941 snap 282 seq_num 941) dirty exists 2024-04-07T16:26:37.088 INFO:tasks.rados.rados.0.smithi100.stdout:3074: left oid 23 (ObjNum 941 snap 282 seq_num 941) 2024-04-07T16:26:37.088 INFO:tasks.rados.rados.0.smithi100.stdout:3077: finishing write tid 1 to smithi10066867-28 2024-04-07T16:26:37.088 INFO:tasks.rados.rados.0.smithi100.stdout:3079: finishing copy_from racing read to smithi10066867-13 2024-04-07T16:26:37.088 INFO:tasks.rados.rados.0.smithi100.stdout:3077: finishing write tid 2 to smithi10066867-28 2024-04-07T16:26:37.088 INFO:tasks.rados.rados.0.smithi100.stdout:3079: finishing copy_from to smithi10066867-13 2024-04-07T16:26:37.088 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 13 v 1417 (ObjNum 49 snap 0 seq_num 1953066355) dirty exists 2024-04-07T16:26:37.088 INFO:tasks.rados.rados.0.smithi100.stdout:3077: finishing write tid 3 to smithi10066867-28 2024-04-07T16:26:37.088 INFO:tasks.rados.rados.0.smithi100.stdout:3077: finishing write tid 4 to smithi10066867-28 2024-04-07T16:26:37.088 INFO:tasks.rados.rados.0.smithi100.stdout:3078: expect (ObjNum 794 snap 251 seq_num 794) 2024-04-07T16:26:37.139 INFO:tasks.rados.rados.0.smithi100.stdout:3077: finishing write tid 5 to smithi10066867-28 2024-04-07T16:26:37.139 INFO:tasks.rados.rados.0.smithi100.stdout:3077: oid 28 updating version 0 to 1526 2024-04-07T16:26:37.139 INFO:tasks.rados.rados.0.smithi100.stdout:3077: oid 28 updating version 1526 to 1527 2024-04-07T16:26:37.139 INFO:tasks.rados.rados.0.smithi100.stdout:3077: oid 28 version 1527 is already newer than 1525 2024-04-07T16:26:37.140 INFO:tasks.rados.rados.0.smithi100.stdout:3077: oid 28 updating version 1527 to 1528 2024-04-07T16:26:37.140 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 28 v 1528 (ObjNum 942 snap 282 seq_num 942) dirty exists 2024-04-07T16:26:37.140 INFO:tasks.rados.rados.0.smithi100.stdout:3077: left oid 28 (ObjNum 942 snap 282 seq_num 942) 2024-04-07T16:26:37.140 INFO:tasks.rados.rados.0.smithi100.stdout:3080: finishing copy_from racing read to smithi10066867-20 2024-04-07T16:26:37.140 INFO:tasks.rados.rados.0.smithi100.stdout:3080: finishing copy_from to smithi10066867-20 2024-04-07T16:26:37.140 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 20 v 1228 (ObjNum 880 snap 264 seq_num 880) dirty exists 2024-04-07T16:26:37.140 INFO:tasks.rados.rados.0.smithi100.stdout:3074: done (4 left) 2024-04-07T16:26:37.140 INFO:tasks.rados.rados.0.smithi100.stdout:3077: done (3 left) 2024-04-07T16:26:37.140 INFO:tasks.rados.rados.0.smithi100.stdout:3078: done (2 left) 2024-04-07T16:26:37.140 INFO:tasks.rados.rados.0.smithi100.stdout:3079: done (1 left) 2024-04-07T16:26:37.140 INFO:tasks.rados.rados.0.smithi100.stdout:3080: done (0 left) 2024-04-07T16:26:37.140 INFO:tasks.rados.rados.0.smithi100.stdout:3081: snap_remove snap 271 2024-04-07T16:26:37.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:37 smithi144 bash[66654]: cluster 2024-04-07T16:26:36.016176+0000 mon.a (mon.0) 1017 : cluster [DBG] osdmap e640: 8 total, 8 up, 8 in 2024-04-07T16:26:37.436 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:37 smithi100 bash[74942]: cluster 2024-04-07T16:26:36.016176+0000 mon.a (mon.0) 1017 : cluster [DBG] osdmap e640: 8 total, 8 up, 8 in 2024-04-07T16:26:37.436 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:37 smithi100 bash[76202]: cluster 2024-04-07T16:26:36.016176+0000 mon.a (mon.0) 1017 : cluster [DBG] osdmap e640: 8 total, 8 up, 8 in 2024-04-07T16:26:38.029 INFO:tasks.rados.rados.0.smithi100.stdout:3081: done (0 left) 2024-04-07T16:26:38.030 INFO:tasks.rados.rados.0.smithi100.stdout:3082: rmattr oid 40 current snap is 282 2024-04-07T16:26:38.030 INFO:tasks.rados.rados.0.smithi100.stdout:3083: read oid 45 snap -1 2024-04-07T16:26:38.030 INFO:tasks.rados.rados.0.smithi100.stdout:3083: expect (ObjNum 933 snap 280 seq_num 933) 2024-04-07T16:26:38.030 INFO:tasks.rados.rados.0.smithi100.stdout:3084: read oid 8 snap -1 2024-04-07T16:26:38.031 INFO:tasks.rados.rados.0.smithi100.stdout:3084: expect (ObjNum 934 snap 280 seq_num 934) 2024-04-07T16:26:38.031 INFO:tasks.rados.rados.0.smithi100.stdout:3085: read oid 30 snap -1 2024-04-07T16:26:38.031 INFO:tasks.rados.rados.0.smithi100.stdout:3085: expect (ObjNum 880 snap 264 seq_num 880) 2024-04-07T16:26:38.031 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 6 current snap is 282 2024-04-07T16:26:38.031 INFO:tasks.rados.rados.0.smithi100.stdout:3086: seq_num 943 ranges {1572864=598016,2170880=688128,2859008=114688} 2024-04-07T16:26:38.071 INFO:tasks.rados.rados.0.smithi100.stdout:3086: writing smithi10066867-6 from 1572864 to 2170880 tid 1 2024-04-07T16:26:38.082 INFO:tasks.rados.rados.0.smithi100.stdout:3086: writing smithi10066867-6 from 2170880 to 2859008 tid 2 2024-04-07T16:26:38.084 INFO:tasks.rados.rados.0.smithi100.stdout:3086: writing smithi10066867-6 from 2859008 to 2973696 tid 3 2024-04-07T16:26:38.085 INFO:tasks.rados.rados.0.smithi100.stdout:3087: read oid 39 snap -1 2024-04-07T16:26:38.085 INFO:tasks.rados.rados.0.smithi100.stdout:3087: expect (ObjNum 924 snap 278 seq_num 924) 2024-04-07T16:26:38.085 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 19 current snap is 282 2024-04-07T16:26:38.085 INFO:tasks.rados.rados.0.smithi100.stdout:3088: seq_num 944 ranges {0=614400,614400=573440,1187840=229376} 2024-04-07T16:26:38.089 INFO:tasks.rados.rados.0.smithi100.stdout:3088: writing smithi10066867-19 from 0 to 614400 tid 1 2024-04-07T16:26:38.094 INFO:tasks.rados.rados.0.smithi100.stdout:3088: writing smithi10066867-19 from 614400 to 1187840 tid 2 2024-04-07T16:26:38.096 INFO:tasks.rados.rados.0.smithi100.stdout:3088: writing smithi10066867-19 from 1187840 to 1417216 tid 3 2024-04-07T16:26:38.096 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 10 current snap is 282 2024-04-07T16:26:38.096 INFO:tasks.rados.rados.0.smithi100.stdout:3089: seq_num 945 ranges {0=630784,630784=458752,1089536=344064} 2024-04-07T16:26:38.100 INFO:tasks.rados.rados.0.smithi100.stdout:3089: writing smithi10066867-10 from 0 to 630784 tid 1 2024-04-07T16:26:38.103 INFO:tasks.rados.rados.0.smithi100.stdout:3089: writing smithi10066867-10 from 630784 to 1089536 tid 2 2024-04-07T16:26:38.106 INFO:tasks.rados.rados.0.smithi100.stdout:3089: writing smithi10066867-10 from 1089536 to 1433600 tid 3 2024-04-07T16:26:38.106 INFO:tasks.rados.rados.0.smithi100.stdout:3090: snap_create 2024-04-07T16:26:38.106 INFO:tasks.rados.rados.0.smithi100.stdout:3083: expect (ObjNum 933 snap 280 seq_num 933) 2024-04-07T16:26:38.154 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 40 v 1616 (ObjNum 931 snap 280 seq_num 931) dirty exists 2024-04-07T16:26:38.155 INFO:tasks.rados.rados.0.smithi100.stdout:3084: expect (ObjNum 934 snap 280 seq_num 934) 2024-04-07T16:26:38.213 INFO:tasks.rados.rados.0.smithi100.stdout:3085: expect (ObjNum 880 snap 264 seq_num 880) 2024-04-07T16:26:38.223 INFO:tasks.rados.rados.0.smithi100.stdout:3086: finishing write tid 1 to smithi10066867-6 2024-04-07T16:26:38.223 INFO:tasks.rados.rados.0.smithi100.stdout:3086: finishing write tid 2 to smithi10066867-6 2024-04-07T16:26:38.223 INFO:tasks.rados.rados.0.smithi100.stdout:3086: finishing write tid 3 to smithi10066867-6 2024-04-07T16:26:38.224 INFO:tasks.rados.rados.0.smithi100.stdout:3086: finishing write tid 4 to smithi10066867-6 2024-04-07T16:26:38.224 INFO:tasks.rados.rados.0.smithi100.stdout:3087: expect (ObjNum 924 snap 278 seq_num 924) 2024-04-07T16:26:38.250 INFO:tasks.rados.rados.0.smithi100.stdout:3086: finishing write tid 5 to smithi10066867-6 2024-04-07T16:26:38.251 INFO:tasks.rados.rados.0.smithi100.stdout:3086: oid 6 updating version 0 to 1618 2024-04-07T16:26:38.251 INFO:tasks.rados.rados.0.smithi100.stdout:3086: oid 6 version 1618 is already newer than 1617 2024-04-07T16:26:38.251 INFO:tasks.rados.rados.0.smithi100.stdout:3086: oid 6 updating version 1618 to 1620 2024-04-07T16:26:38.251 INFO:tasks.rados.rados.0.smithi100.stdout:3086: oid 6 version 1620 is already newer than 1619 2024-04-07T16:26:38.251 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 6 v 1620 (ObjNum 943 snap 282 seq_num 943) dirty exists 2024-04-07T16:26:38.251 INFO:tasks.rados.rados.0.smithi100.stdout:3086: left oid 6 (ObjNum 943 snap 282 seq_num 943) 2024-04-07T16:26:38.251 INFO:tasks.rados.rados.0.smithi100.stdout:3088: finishing write tid 1 to smithi10066867-19 2024-04-07T16:26:38.251 INFO:tasks.rados.rados.0.smithi100.stdout:3088: finishing write tid 2 to smithi10066867-19 2024-04-07T16:26:38.251 INFO:tasks.rados.rados.0.smithi100.stdout:3088: finishing write tid 3 to smithi10066867-19 2024-04-07T16:26:38.251 INFO:tasks.rados.rados.0.smithi100.stdout:3088: finishing write tid 4 to smithi10066867-19 2024-04-07T16:26:38.251 INFO:tasks.rados.rados.0.smithi100.stdout:3088: finishing write tid 5 to smithi10066867-19 2024-04-07T16:26:38.251 INFO:tasks.rados.rados.0.smithi100.stdout:3088: oid 19 updating version 0 to 1219 2024-04-07T16:26:38.251 INFO:tasks.rados.rados.0.smithi100.stdout:3088: oid 19 updating version 1219 to 1222 2024-04-07T16:26:38.251 INFO:tasks.rados.rados.0.smithi100.stdout:3088: oid 19 version 1222 is already newer than 1221 2024-04-07T16:26:38.251 INFO:tasks.rados.rados.0.smithi100.stdout:3088: oid 19 version 1222 is already newer than 1220 2024-04-07T16:26:38.252 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 19 v 1222 (ObjNum 944 snap 282 seq_num 944) dirty exists 2024-04-07T16:26:38.252 INFO:tasks.rados.rados.0.smithi100.stdout:3088: left oid 19 (ObjNum 944 snap 282 seq_num 944) 2024-04-07T16:26:38.252 INFO:tasks.rados.rados.0.smithi100.stdout:3089: finishing write tid 1 to smithi10066867-10 2024-04-07T16:26:38.252 INFO:tasks.rados.rados.0.smithi100.stdout:3089: finishing write tid 2 to smithi10066867-10 2024-04-07T16:26:38.252 INFO:tasks.rados.rados.0.smithi100.stdout:3089: finishing write tid 3 to smithi10066867-10 2024-04-07T16:26:38.252 INFO:tasks.rados.rados.0.smithi100.stdout:3089: finishing write tid 4 to smithi10066867-10 2024-04-07T16:26:38.252 INFO:tasks.rados.rados.0.smithi100.stdout:3089: finishing write tid 5 to smithi10066867-10 2024-04-07T16:26:38.252 INFO:tasks.rados.rados.0.smithi100.stdout:3089: oid 10 updating version 0 to 1449 2024-04-07T16:26:38.252 INFO:tasks.rados.rados.0.smithi100.stdout:3089: oid 10 updating version 1449 to 1450 2024-04-07T16:26:38.252 INFO:tasks.rados.rados.0.smithi100.stdout:3089: oid 10 updating version 1450 to 1451 2024-04-07T16:26:38.252 INFO:tasks.rados.rados.0.smithi100.stdout:3089: oid 10 updating version 1451 to 1452 2024-04-07T16:26:38.252 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 10 v 1452 (ObjNum 945 snap 282 seq_num 945) dirty exists 2024-04-07T16:26:38.252 INFO:tasks.rados.rados.0.smithi100.stdout:3089: left oid 10 (ObjNum 945 snap 282 seq_num 945) 2024-04-07T16:26:38.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:38 smithi144 bash[66654]: cluster 2024-04-07T16:26:36.771134+0000 mgr.y (mgr.25175) 305 : cluster [DBG] pgmap v553: 121 pgs: 1 active+clean+snaptrim, 120 active+clean; 117 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 6.1 MiB/s rd, 2.0 MiB/s wr, 20 op/s 2024-04-07T16:26:38.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:38 smithi144 bash[66654]: cluster 2024-04-07T16:26:37.024759+0000 mon.a (mon.0) 1018 : cluster [DBG] osdmap e641: 8 total, 8 up, 8 in 2024-04-07T16:26:38.417 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:38 smithi100 bash[74942]: cluster 2024-04-07T16:26:36.771134+0000 mgr.y (mgr.25175) 305 : cluster [DBG] pgmap v553: 121 pgs: 1 active+clean+snaptrim, 120 active+clean; 117 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 6.1 MiB/s rd, 2.0 MiB/s wr, 20 op/s 2024-04-07T16:26:38.417 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:38 smithi100 bash[74942]: cluster 2024-04-07T16:26:37.024759+0000 mon.a (mon.0) 1018 : cluster [DBG] osdmap e641: 8 total, 8 up, 8 in 2024-04-07T16:26:38.418 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:38 smithi100 bash[76202]: cluster 2024-04-07T16:26:36.771134+0000 mgr.y (mgr.25175) 305 : cluster [DBG] pgmap v553: 121 pgs: 1 active+clean+snaptrim, 120 active+clean; 117 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 6.1 MiB/s rd, 2.0 MiB/s wr, 20 op/s 2024-04-07T16:26:38.418 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:38 smithi100 bash[76202]: cluster 2024-04-07T16:26:37.024759+0000 mon.a (mon.0) 1018 : cluster [DBG] osdmap e641: 8 total, 8 up, 8 in 2024-04-07T16:26:38.686 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:26:38 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:26:38] "GET /metrics HTTP/1.1" 200 36343 "" "Prometheus/2.43.0" 2024-04-07T16:26:39.017 INFO:teuthology.orchestra.run.smithi100.stdout:true 2024-04-07T16:26:39.022 INFO:tasks.rados.rados.0.smithi100.stdout:3082: done (8 left) 2024-04-07T16:26:39.022 INFO:tasks.rados.rados.0.smithi100.stdout:3083: done (7 left) 2024-04-07T16:26:39.022 INFO:tasks.rados.rados.0.smithi100.stdout:3084: done (6 left) 2024-04-07T16:26:39.022 INFO:tasks.rados.rados.0.smithi100.stdout:3085: done (5 left) 2024-04-07T16:26:39.022 INFO:tasks.rados.rados.0.smithi100.stdout:3086: done (4 left) 2024-04-07T16:26:39.022 INFO:tasks.rados.rados.0.smithi100.stdout:3087: done (3 left) 2024-04-07T16:26:39.022 INFO:tasks.rados.rados.0.smithi100.stdout:3088: done (2 left) 2024-04-07T16:26:39.022 INFO:tasks.rados.rados.0.smithi100.stdout:3089: done (1 left) 2024-04-07T16:26:39.022 INFO:tasks.rados.rados.0.smithi100.stdout:3090: done (0 left) 2024-04-07T16:26:39.022 INFO:tasks.rados.rados.0.smithi100.stdout:3091: read oid 28 snap -1 2024-04-07T16:26:39.023 INFO:tasks.rados.rados.0.smithi100.stdout:3091: expect (ObjNum 942 snap 282 seq_num 942) 2024-04-07T16:26:39.023 INFO:tasks.rados.rados.0.smithi100.stdout:3092: delete oid 2 current snap is 283 2024-04-07T16:26:39.028 INFO:tasks.rados.rados.0.smithi100.stdout:3092: done (1 left) 2024-04-07T16:26:39.028 INFO:tasks.rados.rados.0.smithi100.stdout:3093: snap_create 2024-04-07T16:26:39.048 INFO:tasks.rados.rados.0.smithi100.stdout:3091: expect (ObjNum 942 snap 282 seq_num 942) 2024-04-07T16:26:39.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:39 smithi144 bash[66654]: cluster 2024-04-07T16:26:38.039577+0000 mon.a (mon.0) 1019 : cluster [DBG] osdmap e642: 8 total, 8 up, 8 in 2024-04-07T16:26:39.409 INFO:teuthology.orchestra.run.smithi100.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-07T16:26:39.409 INFO:teuthology.orchestra.run.smithi100.stdout:alertmanager.a smithi100 *:9093,9094 running (12m) 35s ago 13m 14.6M - 0.25.0 c8568f914cd2 5143075e7d26 2024-04-07T16:26:39.409 INFO:teuthology.orchestra.run.smithi100.stdout:grafana.a smithi144 *:3000 running (12m) 7s ago 12m 79.3M - 9.4.7 954c08fa6188 7180c0708acd 2024-04-07T16:26:39.409 INFO:teuthology.orchestra.run.smithi100.stdout:mds.a smithi100 running (40s) 35s ago 14m 11.6M - 19.0.0-2785-gadce92d0 c0ecd4226ac9 075552e67bd0 2024-04-07T16:26:39.409 INFO:teuthology.orchestra.run.smithi100.stdout:mds.b smithi144 running (13s) 7s ago 14m 11.5M - 19.0.0-2785-gadce92d0 c0ecd4226ac9 fc737f3b1edc 2024-04-07T16:26:39.409 INFO:teuthology.orchestra.run.smithi100.stdout:mgr.x smithi144 *:8443,9283,8765 running (6m) 7s ago 21m 441M - 19.0.0-2785-gadce92d0 c0ecd4226ac9 233745c292a6 2024-04-07T16:26:39.409 INFO:teuthology.orchestra.run.smithi100.stdout:mgr.y smithi100 *:8443,9283,8765 running (7m) 35s ago 24m 506M - 19.0.0-2785-gadce92d0 c0ecd4226ac9 254dc65cdaa7 2024-04-07T16:26:39.409 INFO:teuthology.orchestra.run.smithi100.stdout:mon.a smithi100 running (6m) 35s ago 24m 64.3M 2048M 19.0.0-2785-gadce92d0 c0ecd4226ac9 718e12655c04 2024-04-07T16:26:39.410 INFO:teuthology.orchestra.run.smithi100.stdout:mon.b smithi144 running (5m) 7s ago 22m 50.5M 2048M 19.0.0-2785-gadce92d0 c0ecd4226ac9 aef817664bff 2024-04-07T16:26:39.410 INFO:teuthology.orchestra.run.smithi100.stdout:mon.c smithi100 running (5m) 35s ago 22m 53.3M 2048M 19.0.0-2785-gadce92d0 c0ecd4226ac9 63e889040f6e 2024-04-07T16:26:39.410 INFO:teuthology.orchestra.run.smithi100.stdout:node-exporter.a smithi100 *:9100 running (13m) 35s ago 13m 8496k - 1.5.0 0da6a335fe13 4bce264c1b26 2024-04-07T16:26:39.410 INFO:teuthology.orchestra.run.smithi100.stdout:node-exporter.b smithi144 *:9100 running (13m) 7s ago 13m 9111k - 1.5.0 0da6a335fe13 b6963413c560 2024-04-07T16:26:39.410 INFO:teuthology.orchestra.run.smithi100.stdout:osd.0 smithi100 running (4m) 35s ago 21m 321M 2524M 19.0.0-2785-gadce92d0 c0ecd4226ac9 a6e83c9d1abc 2024-04-07T16:26:39.410 INFO:teuthology.orchestra.run.smithi100.stdout:osd.1 smithi100 running (4m) 35s ago 20m 284M 2524M 19.0.0-2785-gadce92d0 c0ecd4226ac9 c50fe40079b6 2024-04-07T16:26:39.410 INFO:teuthology.orchestra.run.smithi100.stdout:osd.2 smithi100 running (3m) 35s ago 19m 340M 2524M 19.0.0-2785-gadce92d0 c0ecd4226ac9 8e631c43064b 2024-04-07T16:26:39.410 INFO:teuthology.orchestra.run.smithi100.stdout:osd.3 smithi100 running (3m) 35s ago 18m 148M 2524M 19.0.0-2785-gadce92d0 c0ecd4226ac9 402931247c79 2024-04-07T16:26:39.410 INFO:teuthology.orchestra.run.smithi100.stdout:osd.4 smithi144 running (2m) 7s ago 17m 319M 2524M 19.0.0-2785-gadce92d0 c0ecd4226ac9 ebbb7be95d33 2024-04-07T16:26:39.410 INFO:teuthology.orchestra.run.smithi100.stdout:osd.5 smithi144 running (2m) 7s ago 16m 192M 2524M 19.0.0-2785-gadce92d0 c0ecd4226ac9 ecce27bcb2f8 2024-04-07T16:26:39.410 INFO:teuthology.orchestra.run.smithi100.stdout:osd.6 smithi144 running (97s) 7s ago 16m 96.8M 2524M 19.0.0-2785-gadce92d0 c0ecd4226ac9 3788c5c3f427 2024-04-07T16:26:39.410 INFO:teuthology.orchestra.run.smithi100.stdout:osd.7 smithi144 running (67s) 7s ago 15m 174M 2524M 19.0.0-2785-gadce92d0 c0ecd4226ac9 9365aba97dc0 2024-04-07T16:26:39.410 INFO:teuthology.orchestra.run.smithi100.stdout:prometheus.a smithi144 *:9095 running (7m) 7s ago 13m 53.3M - 2.43.0 a07b618ecd1d 2d0a8131f303 2024-04-07T16:26:39.436 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:39 smithi100 bash[74942]: cluster 2024-04-07T16:26:38.039577+0000 mon.a (mon.0) 1019 : cluster [DBG] osdmap e642: 8 total, 8 up, 8 in 2024-04-07T16:26:39.436 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:39 smithi100 bash[76202]: cluster 2024-04-07T16:26:38.039577+0000 mon.a (mon.0) 1019 : cluster [DBG] osdmap e642: 8 total, 8 up, 8 in 2024-04-07T16:26:39.842 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-04-07T16:26:39.842 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": { 2024-04-07T16:26:39.842 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph version 19.0.0-2785-gadce92d0 (adce92d056533f22dc1ee37e3141d098b046ec1b) squid (dev)": 3 2024-04-07T16:26:39.842 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-07T16:26:39.842 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": { 2024-04-07T16:26:39.843 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph version 19.0.0-2785-gadce92d0 (adce92d056533f22dc1ee37e3141d098b046ec1b) squid (dev)": 2 2024-04-07T16:26:39.843 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-07T16:26:39.843 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": { 2024-04-07T16:26:39.843 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph version 19.0.0-2785-gadce92d0 (adce92d056533f22dc1ee37e3141d098b046ec1b) squid (dev)": 8 2024-04-07T16:26:39.843 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-07T16:26:39.843 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": { 2024-04-07T16:26:39.843 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph version 19.0.0-2785-gadce92d0 (adce92d056533f22dc1ee37e3141d098b046ec1b) squid (dev)": 2 2024-04-07T16:26:39.843 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-07T16:26:39.843 INFO:teuthology.orchestra.run.smithi100.stdout: "overall": { 2024-04-07T16:26:39.843 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph version 19.0.0-2785-gadce92d0 (adce92d056533f22dc1ee37e3141d098b046ec1b) squid (dev)": 15 2024-04-07T16:26:39.843 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-04-07T16:26:39.843 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-04-07T16:26:40.041 INFO:tasks.rados.rados.0.smithi100.stdout:3091: done (1 left) 2024-04-07T16:26:40.042 INFO:tasks.rados.rados.0.smithi100.stdout:3093: done (0 left) 2024-04-07T16:26:40.042 INFO:tasks.rados.rados.0.smithi100.stdout:3094: setattr oid 28 current snap is 284 2024-04-07T16:26:40.044 INFO:tasks.rados.rados.0.smithi100.stdout:3095: read oid 7 snap 283 2024-04-07T16:26:40.044 INFO:tasks.rados.rados.0.smithi100.stdout:3095: expect (ObjNum 935 snap 280 seq_num 935) 2024-04-07T16:26:40.044 INFO:tasks.rados.rados.0.smithi100.stdout:3096: copy_from oid 38 from oid 3 current snap is 284 2024-04-07T16:26:40.044 INFO:tasks.rados.rados.0.smithi100.stdout:3097: read oid 49 snap -1 2024-04-07T16:26:40.044 INFO:tasks.rados.rados.0.smithi100.stdout:3097: expect deleted 2024-04-07T16:26:40.044 INFO:tasks.rados.rados.0.smithi100.stdout:3098: copy_from oid 36 from oid 42 current snap is 284 2024-04-07T16:26:40.044 INFO:tasks.rados.rados.0.smithi100.stdout:3099: read oid 27 snap -1 2024-04-07T16:26:40.044 INFO:tasks.rados.rados.0.smithi100.stdout:3099: expect (ObjNum 863 snap 261 seq_num 863) 2024-04-07T16:26:40.044 INFO:tasks.rados.rados.0.smithi100.stdout:3100: snap_remove snap 282 2024-04-07T16:26:40.105 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:40 smithi144 bash[66654]: cluster 2024-04-07T16:26:38.771640+0000 mgr.y (mgr.25175) 306 : cluster [DBG] pgmap v556: 121 pgs: 121 active+clean; 112 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 511 B/s rd, 640 KiB/s wr, 3 op/s 2024-04-07T16:26:40.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:40 smithi144 bash[66654]: audit 2024-04-07T16:26:39.018213+0000 mgr.y (mgr.25175) 307 : audit [DBG] from='client.34428 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:26:40.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:40 smithi144 bash[66654]: cluster 2024-04-07T16:26:39.038417+0000 mon.a (mon.0) 1020 : cluster [DBG] osdmap e643: 8 total, 8 up, 8 in 2024-04-07T16:26:40.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:40 smithi144 bash[66654]: audit 2024-04-07T16:26:39.858628+0000 mon.a (mon.0) 1021 : audit [DBG] from='client.? 172.21.15.100:0/3118950690' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:26:40.436 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:40 smithi100 bash[74942]: cluster 2024-04-07T16:26:38.771640+0000 mgr.y (mgr.25175) 306 : cluster [DBG] pgmap v556: 121 pgs: 121 active+clean; 112 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 511 B/s rd, 640 KiB/s wr, 3 op/s 2024-04-07T16:26:40.436 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:40 smithi100 bash[74942]: audit 2024-04-07T16:26:39.018213+0000 mgr.y (mgr.25175) 307 : audit [DBG] from='client.34428 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:26:40.436 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:40 smithi100 bash[74942]: cluster 2024-04-07T16:26:39.038417+0000 mon.a (mon.0) 1020 : cluster [DBG] osdmap e643: 8 total, 8 up, 8 in 2024-04-07T16:26:40.436 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:40 smithi100 bash[74942]: audit 2024-04-07T16:26:39.858628+0000 mon.a (mon.0) 1021 : audit [DBG] from='client.? 172.21.15.100:0/3118950690' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:26:40.436 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:40 smithi100 bash[76202]: cluster 2024-04-07T16:26:38.771640+0000 mgr.y (mgr.25175) 306 : cluster [DBG] pgmap v556: 121 pgs: 121 active+clean; 112 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 511 B/s rd, 640 KiB/s wr, 3 op/s 2024-04-07T16:26:40.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:40 smithi100 bash[76202]: audit 2024-04-07T16:26:39.018213+0000 mgr.y (mgr.25175) 307 : audit [DBG] from='client.34428 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:26:40.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:40 smithi100 bash[76202]: cluster 2024-04-07T16:26:39.038417+0000 mon.a (mon.0) 1020 : cluster [DBG] osdmap e643: 8 total, 8 up, 8 in 2024-04-07T16:26:40.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:40 smithi100 bash[76202]: audit 2024-04-07T16:26:39.858628+0000 mon.a (mon.0) 1021 : audit [DBG] from='client.? 172.21.15.100:0/3118950690' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:26:41.053 INFO:tasks.rados.rados.0.smithi100.stdout:3100: done (6 left) 2024-04-07T16:26:41.053 INFO:tasks.rados.rados.0.smithi100.stdout:3101: snap_create 2024-04-07T16:26:41.053 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 28 v 1529 (ObjNum 942 snap 282 seq_num 942) dirty exists 2024-04-07T16:26:41.053 INFO:tasks.rados.rados.0.smithi100.stdout:3098: finishing copy_from racing read to smithi10066867-36 2024-04-07T16:26:41.053 INFO:tasks.rados.rados.0.smithi100.stdout:3098: finishing copy_from to smithi10066867-36 2024-04-07T16:26:41.054 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 36 v 1232 (ObjNum 900 snap 270 seq_num 900) dirty exists 2024-04-07T16:26:41.054 INFO:tasks.rados.rados.0.smithi100.stdout:3095: expect (ObjNum 935 snap 280 seq_num 935) 2024-04-07T16:26:41.124 INFO:tasks.rados.rados.0.smithi100.stdout:3099: expect (ObjNum 863 snap 261 seq_num 863) 2024-04-07T16:26:41.225 INFO:tasks.rados.rados.0.smithi100.stdout:3096: finishing copy_from racing read to smithi10066867-38 2024-04-07T16:26:41.225 INFO:tasks.rados.rados.0.smithi100.stdout:3096: finishing copy_from to smithi10066867-38 2024-04-07T16:26:41.225 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 38 v 1138 (ObjNum 940 snap 281 seq_num 940) dirty exists 2024-04-07T16:26:41.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:41 smithi144 bash[66654]: audit 2024-04-07T16:26:39.385138+0000 mgr.y (mgr.25175) 308 : audit [DBG] from='client.44345 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:26:41.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:41 smithi144 bash[66654]: cluster 2024-04-07T16:26:40.057745+0000 mon.a (mon.0) 1022 : cluster [DBG] osdmap e644: 8 total, 8 up, 8 in 2024-04-07T16:26:41.436 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:41 smithi100 bash[74942]: audit 2024-04-07T16:26:39.385138+0000 mgr.y (mgr.25175) 308 : audit [DBG] from='client.44345 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:26:41.436 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:41 smithi100 bash[74942]: cluster 2024-04-07T16:26:40.057745+0000 mon.a (mon.0) 1022 : cluster [DBG] osdmap e644: 8 total, 8 up, 8 in 2024-04-07T16:26:41.436 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:41 smithi100 bash[76202]: audit 2024-04-07T16:26:39.385138+0000 mgr.y (mgr.25175) 308 : audit [DBG] from='client.44345 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:26:41.436 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:41 smithi100 bash[76202]: cluster 2024-04-07T16:26:40.057745+0000 mon.a (mon.0) 1022 : cluster [DBG] osdmap e644: 8 total, 8 up, 8 in 2024-04-07T16:26:41.898 INFO:tasks.rados.rados.0.smithi100.stdout:3094: done (6 left) 2024-04-07T16:26:41.898 INFO:tasks.rados.rados.0.smithi100.stdout:3095: done (5 left) 2024-04-07T16:26:41.898 INFO:tasks.rados.rados.0.smithi100.stdout:3096: done (4 left) 2024-04-07T16:26:41.898 INFO:tasks.rados.rados.0.smithi100.stdout:3097: done (3 left) 2024-04-07T16:26:41.899 INFO:tasks.rados.rados.0.smithi100.stdout:3098: done (2 left) 2024-04-07T16:26:41.899 INFO:tasks.rados.rados.0.smithi100.stdout:3099: done (1 left) 2024-04-07T16:26:41.899 INFO:tasks.rados.rados.0.smithi100.stdout:3101: done (0 left) 2024-04-07T16:26:41.899 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 48 current snap is 285 2024-04-07T16:26:41.899 INFO:tasks.rados.rados.0.smithi100.stdout:3102: seq_num 947 ranges {1302528=663552,1966080=630784,2596864=172032} 2024-04-07T16:26:41.930 INFO:tasks.rados.rados.0.smithi100.stdout:3102: writing smithi10066867-48 from 1302528 to 1966080 tid 1 2024-04-07T16:26:41.943 INFO:tasks.rados.rados.0.smithi100.stdout:3102: writing smithi10066867-48 from 1966080 to 2596864 tid 2 2024-04-07T16:26:41.947 INFO:tasks.rados.rados.0.smithi100.stdout:3102: writing smithi10066867-48 from 2596864 to 2768896 tid 3 2024-04-07T16:26:41.947 INFO:tasks.rados.rados.0.smithi100.stdout:3103: read oid 29 snap -1 2024-04-07T16:26:41.947 INFO:tasks.rados.rados.0.smithi100.stdout:3103: expect (ObjNum 0 snap 0 seq_num 0) 2024-04-07T16:26:41.947 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 24 current snap is 285 2024-04-07T16:26:41.948 INFO:tasks.rados.rados.0.smithi100.stdout:3104: seq_num 948 ranges {7135232=679936,7815168=516096,8331264=286720} 2024-04-07T16:26:42.012 INFO:tasks.rados.rados.0.smithi100.stdout:3104: writing smithi10066867-24 from 7135232 to 7815168 tid 1 2024-04-07T16:26:42.015 INFO:tasks.rados.rados.0.smithi100.stdout:3104: writing smithi10066867-24 from 7815168 to 8331264 tid 2 2024-04-07T16:26:42.017 INFO:tasks.rados.rados.0.smithi100.stdout:3104: writing smithi10066867-24 from 8331264 to 8617984 tid 3 2024-04-07T16:26:42.017 INFO:tasks.rados.rados.0.smithi100.stdout:3105: rmattr oid 47 current snap is 285 2024-04-07T16:26:42.017 INFO:tasks.rados.rados.0.smithi100.stdout:3105: done (3 left) 2024-04-07T16:26:42.017 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 36 current snap is 285 2024-04-07T16:26:42.017 INFO:tasks.rados.rados.0.smithi100.stdout:3106: seq_num 949 ranges {679936=704512,1384448=794624,2179072=8192} 2024-04-07T16:26:42.025 INFO:tasks.rados.rados.0.smithi100.stdout:3106: writing smithi10066867-36 from 679936 to 1384448 tid 1 2024-04-07T16:26:42.030 INFO:tasks.rados.rados.0.smithi100.stdout:3106: writing smithi10066867-36 from 1384448 to 2179072 tid 2 2024-04-07T16:26:42.031 INFO:tasks.rados.rados.0.smithi100.stdout:3106: writing smithi10066867-36 from 2179072 to 2187264 tid 3 2024-04-07T16:26:42.031 INFO:tasks.rados.rados.0.smithi100.stdout:3107: setattr oid 42 current snap is 285 2024-04-07T16:26:42.031 INFO:tasks.rados.rados.0.smithi100.stdout:3102: finishing write tid 1 to smithi10066867-48 2024-04-07T16:26:42.031 INFO:tasks.rados.rados.0.smithi100.stdout:3103: expect (ObjNum 0 snap 0 seq_num 0) 2024-04-07T16:26:42.035 INFO:tasks.rados.rados.0.smithi100.stdout:3102: finishing write tid 2 to smithi10066867-48 2024-04-07T16:26:42.035 INFO:tasks.rados.rados.0.smithi100.stdout:3102: finishing write tid 3 to smithi10066867-48 2024-04-07T16:26:42.035 INFO:tasks.rados.rados.0.smithi100.stdout:3102: finishing write tid 4 to smithi10066867-48 2024-04-07T16:26:42.035 INFO:tasks.rados.rados.0.smithi100.stdout:3102: finishing write tid 5 to smithi10066867-48 2024-04-07T16:26:42.035 INFO:tasks.rados.rados.0.smithi100.stdout:3102: oid 48 updating version 0 to 1130 2024-04-07T16:26:42.035 INFO:tasks.rados.rados.0.smithi100.stdout:3102: oid 48 updating version 1130 to 1133 2024-04-07T16:26:42.035 INFO:tasks.rados.rados.0.smithi100.stdout:3102: oid 48 version 1133 is already newer than 1132 2024-04-07T16:26:42.036 INFO:tasks.rados.rados.0.smithi100.stdout:3102: oid 48 version 1133 is already newer than 1131 2024-04-07T16:26:42.036 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 48 v 1133 (ObjNum 947 snap 285 seq_num 947) dirty exists 2024-04-07T16:26:42.036 INFO:tasks.rados.rados.0.smithi100.stdout:3102: left oid 48 (ObjNum 947 snap 285 seq_num 947) 2024-04-07T16:26:42.036 INFO:tasks.rados.rados.0.smithi100.stdout:3104: finishing write tid 1 to smithi10066867-24 2024-04-07T16:26:42.036 INFO:tasks.rados.rados.0.smithi100.stdout:3104: finishing write tid 2 to smithi10066867-24 2024-04-07T16:26:42.036 INFO:tasks.rados.rados.0.smithi100.stdout:3104: finishing write tid 3 to smithi10066867-24 2024-04-07T16:26:42.036 INFO:tasks.rados.rados.0.smithi100.stdout:3104: finishing write tid 4 to smithi10066867-24 2024-04-07T16:26:42.036 INFO:tasks.rados.rados.0.smithi100.stdout:3106: finishing write tid 1 to smithi10066867-36 2024-04-07T16:26:42.036 INFO:tasks.rados.rados.0.smithi100.stdout:3102: done (4 left) 2024-04-07T16:26:42.036 INFO:tasks.rados.rados.0.smithi100.stdout:3103: done (3 left) 2024-04-07T16:26:42.036 INFO:tasks.rados.rados.0.smithi100.stdout:3108: snap_remove snap 283 2024-04-07T16:26:42.304 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:42 smithi144 bash[66654]: cluster 2024-04-07T16:26:40.772428+0000 mgr.y (mgr.25175) 309 : cluster [DBG] pgmap v559: 121 pgs: 121 active+clean; 103 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 1.7 MiB/s rd, 1.4 MiB/s wr, 15 op/s 2024-04-07T16:26:42.304 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:42 smithi144 bash[66654]: cluster 2024-04-07T16:26:41.068689+0000 mon.a (mon.0) 1023 : cluster [DBG] osdmap e645: 8 total, 8 up, 8 in 2024-04-07T16:26:42.304 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:42 smithi144 bash[66654]: cluster 2024-04-07T16:26:41.914357+0000 mon.a (mon.0) 1024 : cluster [DBG] osdmap e646: 8 total, 8 up, 8 in 2024-04-07T16:26:42.436 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:42 smithi100 bash[74942]: cluster 2024-04-07T16:26:40.772428+0000 mgr.y (mgr.25175) 309 : cluster [DBG] pgmap v559: 121 pgs: 121 active+clean; 103 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 1.7 MiB/s rd, 1.4 MiB/s wr, 15 op/s 2024-04-07T16:26:42.436 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:42 smithi100 bash[74942]: cluster 2024-04-07T16:26:41.068689+0000 mon.a (mon.0) 1023 : cluster [DBG] osdmap e645: 8 total, 8 up, 8 in 2024-04-07T16:26:42.436 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:42 smithi100 bash[74942]: cluster 2024-04-07T16:26:41.914357+0000 mon.a (mon.0) 1024 : cluster [DBG] osdmap e646: 8 total, 8 up, 8 in 2024-04-07T16:26:42.436 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:42 smithi100 bash[76202]: cluster 2024-04-07T16:26:40.772428+0000 mgr.y (mgr.25175) 309 : cluster [DBG] pgmap v559: 121 pgs: 121 active+clean; 103 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 1.7 MiB/s rd, 1.4 MiB/s wr, 15 op/s 2024-04-07T16:26:42.436 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:42 smithi100 bash[76202]: cluster 2024-04-07T16:26:41.068689+0000 mon.a (mon.0) 1023 : cluster [DBG] osdmap e645: 8 total, 8 up, 8 in 2024-04-07T16:26:42.436 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:42 smithi100 bash[76202]: cluster 2024-04-07T16:26:41.914357+0000 mon.a (mon.0) 1024 : cluster [DBG] osdmap e646: 8 total, 8 up, 8 in 2024-04-07T16:26:42.605 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:26:42 smithi144 bash[59794]: ts=2024-04-07T16:26:42.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:26:42.910 INFO:tasks.rados.rados.0.smithi100.stdout:3108: done (3 left) 2024-04-07T16:26:42.910 INFO:tasks.rados.rados.0.smithi100.stdout:3109: snap_remove snap 284 2024-04-07T16:26:42.911 INFO:tasks.rados.rados.0.smithi100.stdout:3104: finishing write tid 5 to smithi10066867-24 2024-04-07T16:26:42.911 INFO:tasks.rados.rados.0.smithi100.stdout:3104: oid 24 updating version 0 to 1235 2024-04-07T16:26:42.911 INFO:tasks.rados.rados.0.smithi100.stdout:3104: oid 24 version 1235 is already newer than 1234 2024-04-07T16:26:42.911 INFO:tasks.rados.rados.0.smithi100.stdout:3104: oid 24 version 1235 is already newer than 1233 2024-04-07T16:26:42.911 INFO:tasks.rados.rados.0.smithi100.stdout:3104: oid 24 updating version 1235 to 1236 2024-04-07T16:26:42.911 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 24 v 1236 (ObjNum 948 snap 285 seq_num 948) dirty exists 2024-04-07T16:26:42.911 INFO:tasks.rados.rados.0.smithi100.stdout:3104: left oid 24 (ObjNum 948 snap 285 seq_num 948) 2024-04-07T16:26:42.911 INFO:tasks.rados.rados.0.smithi100.stdout:3106: finishing write tid 2 to smithi10066867-36 2024-04-07T16:26:42.911 INFO:tasks.rados.rados.0.smithi100.stdout:3106: finishing write tid 3 to smithi10066867-36 2024-04-07T16:26:42.911 INFO:tasks.rados.rados.0.smithi100.stdout:3106: finishing write tid 4 to smithi10066867-36 2024-04-07T16:26:42.911 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 42 v 1244 (ObjNum 900 snap 270 seq_num 900) dirty exists 2024-04-07T16:26:42.911 INFO:tasks.rados.rados.0.smithi100.stdout:3106: finishing write tid 5 to smithi10066867-36 2024-04-07T16:26:42.911 INFO:tasks.rados.rados.0.smithi100.stdout:3106: oid 36 updating version 0 to 1239 2024-04-07T16:26:42.911 INFO:tasks.rados.rados.0.smithi100.stdout:3106: oid 36 version 1239 is already newer than 1237 2024-04-07T16:26:42.912 INFO:tasks.rados.rados.0.smithi100.stdout:3106: oid 36 version 1239 is already newer than 1238 2024-04-07T16:26:42.912 INFO:tasks.rados.rados.0.smithi100.stdout:3106: oid 36 updating version 1239 to 1240 2024-04-07T16:26:42.912 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 36 v 1240 (ObjNum 949 snap 285 seq_num 949) dirty exists 2024-04-07T16:26:42.912 INFO:tasks.rados.rados.0.smithi100.stdout:3106: left oid 36 (ObjNum 949 snap 285 seq_num 949) 2024-04-07T16:26:43.431 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:43 smithi100 bash[74942]: audit 2024-04-07T16:26:42.445421+0000 mon.a (mon.0) 1025 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:26:43.431 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:43 smithi100 bash[74942]: audit 2024-04-07T16:26:42.457836+0000 mon.a (mon.0) 1026 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:26:43.431 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:43 smithi100 bash[74942]: audit 2024-04-07T16:26:42.460513+0000 mon.a (mon.0) 1027 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:26:43.431 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:43 smithi100 bash[74942]: audit 2024-04-07T16:26:42.462296+0000 mon.a (mon.0) 1028 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:26:43.431 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:43 smithi100 bash[74942]: audit 2024-04-07T16:26:42.473149+0000 mon.a (mon.0) 1029 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:26:43.431 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:43 smithi100 bash[74942]: audit 2024-04-07T16:26:42.571477+0000 mon.a (mon.0) 1030 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:26:43.431 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:43 smithi100 bash[74942]: audit 2024-04-07T16:26:42.576350+0000 mon.a (mon.0) 1031 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:26:43.432 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:43 smithi100 bash[74942]: audit 2024-04-07T16:26:42.580322+0000 mon.a (mon.0) 1032 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:26:43.432 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:43 smithi100 bash[74942]: audit 2024-04-07T16:26:42.583339+0000 mon.a (mon.0) 1033 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:26:43.687 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:43 smithi100 bash[74942]: audit 2024-04-07T16:26:42.589068+0000 mon.a (mon.0) 1034 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:26:43.687 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:43 smithi100 bash[74942]: audit 2024-04-07T16:26:42.596026+0000 mon.a (mon.0) 1035 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:26:43.687 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:43 smithi100 bash[74942]: cephadm 2024-04-07T16:26:42.597868+0000 mgr.y (mgr.25175) 310 : cephadm [INF] Upgrade: Setting container_image for all mds 2024-04-07T16:26:43.687 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:43 smithi100 bash[74942]: audit 2024-04-07T16:26:42.605197+0000 mon.a (mon.0) 1036 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:26:43.688 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:43 smithi100 bash[74942]: audit 2024-04-07T16:26:42.609924+0000 mon.a (mon.0) 1037 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.a"}]: dispatch 2024-04-07T16:26:43.688 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:43 smithi100 bash[74942]: audit 2024-04-07T16:26:42.616009+0000 mon.a (mon.0) 1038 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.a"}]': finished 2024-04-07T16:26:43.688 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:43 smithi100 bash[74942]: audit 2024-04-07T16:26:42.620314+0000 mon.a (mon.0) 1039 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.b"}]: dispatch 2024-04-07T16:26:43.688 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:43 smithi100 bash[74942]: audit 2024-04-07T16:26:42.626492+0000 mon.a (mon.0) 1040 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.b"}]': finished 2024-04-07T16:26:43.688 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:43 smithi100 bash[74942]: audit 2024-04-07T16:26:42.632335+0000 mon.a (mon.0) 1041 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:26:43.688 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:43 smithi100 bash[74942]: cephadm 2024-04-07T16:26:42.634121+0000 mgr.y (mgr.25175) 311 : cephadm [INF] Upgrade: Setting container_image for all rgw 2024-04-07T16:26:43.688 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:43 smithi100 bash[74942]: audit 2024-04-07T16:26:42.642838+0000 mon.a (mon.0) 1042 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:26:43.688 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:43 smithi100 bash[74942]: audit 2024-04-07T16:26:42.647727+0000 mon.a (mon.0) 1043 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:26:43.688 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:43 smithi100 bash[74942]: cephadm 2024-04-07T16:26:42.649136+0000 mgr.y (mgr.25175) 312 : cephadm [INF] Upgrade: Setting container_image for all rbd-mirror 2024-04-07T16:26:43.688 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:43 smithi100 bash[74942]: audit 2024-04-07T16:26:42.656708+0000 mon.a (mon.0) 1044 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:26:43.688 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:43 smithi100 bash[74942]: audit 2024-04-07T16:26:42.661563+0000 mon.a (mon.0) 1045 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:26:43.688 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:43 smithi100 bash[74942]: audit 2024-04-07T16:26:42.664577+0000 mon.a (mon.0) 1046 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:26:43.688 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:43 smithi100 bash[74942]: cephadm 2024-04-07T16:26:42.666258+0000 mgr.y (mgr.25175) 313 : cephadm [INF] Upgrade: Setting container_image for all ceph-exporter 2024-04-07T16:26:43.689 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:43 smithi100 bash[74942]: audit 2024-04-07T16:26:42.675010+0000 mon.a (mon.0) 1047 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:26:43.689 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:43 smithi100 bash[74942]: audit 2024-04-07T16:26:42.680339+0000 mon.a (mon.0) 1048 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:26:43.689 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:43 smithi100 bash[74942]: cephadm 2024-04-07T16:26:42.681913+0000 mgr.y (mgr.25175) 314 : cephadm [INF] Upgrade: Setting container_image for all iscsi 2024-04-07T16:26:43.689 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:43 smithi100 bash[74942]: audit 2024-04-07T16:26:42.690491+0000 mon.a (mon.0) 1049 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:26:43.689 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:43 smithi100 bash[74942]: audit 2024-04-07T16:26:42.695993+0000 mon.a (mon.0) 1050 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:26:43.689 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:43 smithi100 bash[74942]: cephadm 2024-04-07T16:26:42.697666+0000 mgr.y (mgr.25175) 315 : cephadm [INF] Upgrade: Setting container_image for all nfs 2024-04-07T16:26:43.689 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:43 smithi100 bash[74942]: audit 2024-04-07T16:26:42.706233+0000 mon.a (mon.0) 1051 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:26:43.689 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:43 smithi100 bash[74942]: audit 2024-04-07T16:26:42.712410+0000 mon.a (mon.0) 1052 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:26:43.689 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:43 smithi100 bash[74942]: cephadm 2024-04-07T16:26:42.714064+0000 mgr.y (mgr.25175) 316 : cephadm [INF] Upgrade: Setting container_image for all nvmeof 2024-04-07T16:26:43.689 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:43 smithi100 bash[74942]: audit 2024-04-07T16:26:42.723093+0000 mon.a (mon.0) 1053 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:26:43.689 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:43 smithi100 bash[74942]: cluster 2024-04-07T16:26:42.773181+0000 mgr.y (mgr.25175) 317 : cluster [DBG] pgmap v562: 121 pgs: 121 active+clean; 96 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 6.4 MiB/s rd, 2.9 MiB/s wr, 25 op/s 2024-04-07T16:26:43.689 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:43 smithi100 bash[74942]: cluster 2024-04-07T16:26:42.923643+0000 mon.a (mon.0) 1054 : cluster [DBG] osdmap e647: 8 total, 8 up, 8 in 2024-04-07T16:26:43.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:43 smithi100 bash[76202]: audit 2024-04-07T16:26:42.445421+0000 mon.a (mon.0) 1025 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:26:43.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:43 smithi100 bash[76202]: audit 2024-04-07T16:26:42.457836+0000 mon.a (mon.0) 1026 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:26:43.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:43 smithi100 bash[76202]: audit 2024-04-07T16:26:42.460513+0000 mon.a (mon.0) 1027 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:26:43.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:43 smithi100 bash[76202]: audit 2024-04-07T16:26:42.462296+0000 mon.a (mon.0) 1028 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:26:43.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:43 smithi100 bash[76202]: audit 2024-04-07T16:26:42.473149+0000 mon.a (mon.0) 1029 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:26:43.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:43 smithi100 bash[76202]: audit 2024-04-07T16:26:42.571477+0000 mon.a (mon.0) 1030 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:26:43.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:43 smithi100 bash[76202]: audit 2024-04-07T16:26:42.576350+0000 mon.a (mon.0) 1031 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:26:43.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:43 smithi100 bash[76202]: audit 2024-04-07T16:26:42.580322+0000 mon.a (mon.0) 1032 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:26:43.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:43 smithi100 bash[76202]: audit 2024-04-07T16:26:42.583339+0000 mon.a (mon.0) 1033 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:26:43.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:43 smithi100 bash[76202]: audit 2024-04-07T16:26:42.589068+0000 mon.a (mon.0) 1034 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:26:43.692 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:43 smithi100 bash[76202]: audit 2024-04-07T16:26:42.596026+0000 mon.a (mon.0) 1035 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:26:43.692 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:43 smithi100 bash[76202]: cephadm 2024-04-07T16:26:42.597868+0000 mgr.y (mgr.25175) 310 : cephadm [INF] Upgrade: Setting container_image for all mds 2024-04-07T16:26:43.692 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:43 smithi100 bash[76202]: audit 2024-04-07T16:26:42.605197+0000 mon.a (mon.0) 1036 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:26:43.692 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:43 smithi100 bash[76202]: audit 2024-04-07T16:26:42.609924+0000 mon.a (mon.0) 1037 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.a"}]: dispatch 2024-04-07T16:26:43.692 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:43 smithi100 bash[76202]: audit 2024-04-07T16:26:42.616009+0000 mon.a (mon.0) 1038 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.a"}]': finished 2024-04-07T16:26:43.692 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:43 smithi100 bash[76202]: audit 2024-04-07T16:26:42.620314+0000 mon.a (mon.0) 1039 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.b"}]: dispatch 2024-04-07T16:26:43.692 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:43 smithi100 bash[76202]: audit 2024-04-07T16:26:42.626492+0000 mon.a (mon.0) 1040 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.b"}]': finished 2024-04-07T16:26:43.692 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:43 smithi100 bash[76202]: audit 2024-04-07T16:26:42.632335+0000 mon.a (mon.0) 1041 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:26:43.692 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:43 smithi100 bash[76202]: cephadm 2024-04-07T16:26:42.634121+0000 mgr.y (mgr.25175) 311 : cephadm [INF] Upgrade: Setting container_image for all rgw 2024-04-07T16:26:43.692 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:43 smithi100 bash[76202]: audit 2024-04-07T16:26:42.642838+0000 mon.a (mon.0) 1042 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:26:43.692 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:43 smithi100 bash[76202]: audit 2024-04-07T16:26:42.647727+0000 mon.a (mon.0) 1043 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:26:43.692 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:43 smithi100 bash[76202]: cephadm 2024-04-07T16:26:42.649136+0000 mgr.y (mgr.25175) 312 : cephadm [INF] Upgrade: Setting container_image for all rbd-mirror 2024-04-07T16:26:43.692 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:43 smithi100 bash[76202]: audit 2024-04-07T16:26:42.656708+0000 mon.a (mon.0) 1044 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:26:43.692 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:43 smithi100 bash[76202]: audit 2024-04-07T16:26:42.661563+0000 mon.a (mon.0) 1045 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:26:43.692 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:43 smithi100 bash[76202]: audit 2024-04-07T16:26:42.664577+0000 mon.a (mon.0) 1046 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:26:43.693 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:43 smithi100 bash[76202]: cephadm 2024-04-07T16:26:42.666258+0000 mgr.y (mgr.25175) 313 : cephadm [INF] Upgrade: Setting container_image for all ceph-exporter 2024-04-07T16:26:43.693 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:43 smithi100 bash[76202]: audit 2024-04-07T16:26:42.675010+0000 mon.a (mon.0) 1047 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:26:43.693 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:43 smithi100 bash[76202]: audit 2024-04-07T16:26:42.680339+0000 mon.a (mon.0) 1048 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:26:43.693 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:43 smithi100 bash[76202]: cephadm 2024-04-07T16:26:42.681913+0000 mgr.y (mgr.25175) 314 : cephadm [INF] Upgrade: Setting container_image for all iscsi 2024-04-07T16:26:43.693 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:43 smithi100 bash[76202]: audit 2024-04-07T16:26:42.690491+0000 mon.a (mon.0) 1049 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:26:43.693 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:43 smithi100 bash[76202]: audit 2024-04-07T16:26:42.695993+0000 mon.a (mon.0) 1050 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:26:43.693 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:43 smithi100 bash[76202]: cephadm 2024-04-07T16:26:42.697666+0000 mgr.y (mgr.25175) 315 : cephadm [INF] Upgrade: Setting container_image for all nfs 2024-04-07T16:26:43.693 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:43 smithi100 bash[76202]: audit 2024-04-07T16:26:42.706233+0000 mon.a (mon.0) 1051 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:26:43.693 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:43 smithi100 bash[76202]: audit 2024-04-07T16:26:42.712410+0000 mon.a (mon.0) 1052 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:26:43.693 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:43 smithi100 bash[76202]: cephadm 2024-04-07T16:26:42.714064+0000 mgr.y (mgr.25175) 316 : cephadm [INF] Upgrade: Setting container_image for all nvmeof 2024-04-07T16:26:43.693 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:43 smithi100 bash[76202]: audit 2024-04-07T16:26:42.723093+0000 mon.a (mon.0) 1053 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:26:43.693 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:43 smithi100 bash[76202]: cluster 2024-04-07T16:26:42.773181+0000 mgr.y (mgr.25175) 317 : cluster [DBG] pgmap v562: 121 pgs: 121 active+clean; 96 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 6.4 MiB/s rd, 2.9 MiB/s wr, 25 op/s 2024-04-07T16:26:43.693 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:43 smithi100 bash[76202]: cluster 2024-04-07T16:26:42.923643+0000 mon.a (mon.0) 1054 : cluster [DBG] osdmap e647: 8 total, 8 up, 8 in 2024-04-07T16:26:43.856 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:43 smithi144 bash[66654]: audit 2024-04-07T16:26:42.445421+0000 mon.a (mon.0) 1025 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:26:43.856 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:43 smithi144 bash[66654]: audit 2024-04-07T16:26:42.457836+0000 mon.a (mon.0) 1026 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:26:43.856 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:43 smithi144 bash[66654]: audit 2024-04-07T16:26:42.460513+0000 mon.a (mon.0) 1027 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:26:43.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:43 smithi144 bash[66654]: audit 2024-04-07T16:26:42.462296+0000 mon.a (mon.0) 1028 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:26:43.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:43 smithi144 bash[66654]: audit 2024-04-07T16:26:42.473149+0000 mon.a (mon.0) 1029 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:26:43.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:43 smithi144 bash[66654]: audit 2024-04-07T16:26:42.571477+0000 mon.a (mon.0) 1030 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:26:43.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:43 smithi144 bash[66654]: audit 2024-04-07T16:26:42.576350+0000 mon.a (mon.0) 1031 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:26:43.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:43 smithi144 bash[66654]: audit 2024-04-07T16:26:42.580322+0000 mon.a (mon.0) 1032 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:26:43.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:43 smithi144 bash[66654]: audit 2024-04-07T16:26:42.583339+0000 mon.a (mon.0) 1033 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:26:43.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:43 smithi144 bash[66654]: audit 2024-04-07T16:26:42.589068+0000 mon.a (mon.0) 1034 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:26:43.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:43 smithi144 bash[66654]: audit 2024-04-07T16:26:42.596026+0000 mon.a (mon.0) 1035 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:26:43.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:43 smithi144 bash[66654]: cephadm 2024-04-07T16:26:42.597868+0000 mgr.y (mgr.25175) 310 : cephadm [INF] Upgrade: Setting container_image for all mds 2024-04-07T16:26:43.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:43 smithi144 bash[66654]: audit 2024-04-07T16:26:42.605197+0000 mon.a (mon.0) 1036 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:26:43.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:43 smithi144 bash[66654]: audit 2024-04-07T16:26:42.609924+0000 mon.a (mon.0) 1037 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.a"}]: dispatch 2024-04-07T16:26:43.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:43 smithi144 bash[66654]: audit 2024-04-07T16:26:42.616009+0000 mon.a (mon.0) 1038 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.a"}]': finished 2024-04-07T16:26:43.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:43 smithi144 bash[66654]: audit 2024-04-07T16:26:42.620314+0000 mon.a (mon.0) 1039 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.b"}]: dispatch 2024-04-07T16:26:43.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:43 smithi144 bash[66654]: audit 2024-04-07T16:26:42.626492+0000 mon.a (mon.0) 1040 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.b"}]': finished 2024-04-07T16:26:43.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:43 smithi144 bash[66654]: audit 2024-04-07T16:26:42.632335+0000 mon.a (mon.0) 1041 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:26:43.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:43 smithi144 bash[66654]: cephadm 2024-04-07T16:26:42.634121+0000 mgr.y (mgr.25175) 311 : cephadm [INF] Upgrade: Setting container_image for all rgw 2024-04-07T16:26:43.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:43 smithi144 bash[66654]: audit 2024-04-07T16:26:42.642838+0000 mon.a (mon.0) 1042 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:26:43.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:43 smithi144 bash[66654]: audit 2024-04-07T16:26:42.647727+0000 mon.a (mon.0) 1043 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:26:43.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:43 smithi144 bash[66654]: cephadm 2024-04-07T16:26:42.649136+0000 mgr.y (mgr.25175) 312 : cephadm [INF] Upgrade: Setting container_image for all rbd-mirror 2024-04-07T16:26:43.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:43 smithi144 bash[66654]: audit 2024-04-07T16:26:42.656708+0000 mon.a (mon.0) 1044 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:26:43.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:43 smithi144 bash[66654]: audit 2024-04-07T16:26:42.661563+0000 mon.a (mon.0) 1045 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:26:43.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:43 smithi144 bash[66654]: audit 2024-04-07T16:26:42.664577+0000 mon.a (mon.0) 1046 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:26:43.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:43 smithi144 bash[66654]: cephadm 2024-04-07T16:26:42.666258+0000 mgr.y (mgr.25175) 313 : cephadm [INF] Upgrade: Setting container_image for all ceph-exporter 2024-04-07T16:26:43.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:43 smithi144 bash[66654]: audit 2024-04-07T16:26:42.675010+0000 mon.a (mon.0) 1047 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:26:43.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:43 smithi144 bash[66654]: audit 2024-04-07T16:26:42.680339+0000 mon.a (mon.0) 1048 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:26:43.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:43 smithi144 bash[66654]: cephadm 2024-04-07T16:26:42.681913+0000 mgr.y (mgr.25175) 314 : cephadm [INF] Upgrade: Setting container_image for all iscsi 2024-04-07T16:26:43.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:43 smithi144 bash[66654]: audit 2024-04-07T16:26:42.690491+0000 mon.a (mon.0) 1049 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:26:43.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:43 smithi144 bash[66654]: audit 2024-04-07T16:26:42.695993+0000 mon.a (mon.0) 1050 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:26:43.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:43 smithi144 bash[66654]: cephadm 2024-04-07T16:26:42.697666+0000 mgr.y (mgr.25175) 315 : cephadm [INF] Upgrade: Setting container_image for all nfs 2024-04-07T16:26:43.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:43 smithi144 bash[66654]: audit 2024-04-07T16:26:42.706233+0000 mon.a (mon.0) 1051 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:26:43.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:43 smithi144 bash[66654]: audit 2024-04-07T16:26:42.712410+0000 mon.a (mon.0) 1052 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:26:43.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:43 smithi144 bash[66654]: cephadm 2024-04-07T16:26:42.714064+0000 mgr.y (mgr.25175) 316 : cephadm [INF] Upgrade: Setting container_image for all nvmeof 2024-04-07T16:26:43.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:43 smithi144 bash[66654]: audit 2024-04-07T16:26:42.723093+0000 mon.a (mon.0) 1053 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:26:43.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:43 smithi144 bash[66654]: cluster 2024-04-07T16:26:42.773181+0000 mgr.y (mgr.25175) 317 : cluster [DBG] pgmap v562: 121 pgs: 121 active+clean; 96 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 6.4 MiB/s rd, 2.9 MiB/s wr, 25 op/s 2024-04-07T16:26:43.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:43 smithi144 bash[66654]: cluster 2024-04-07T16:26:42.923643+0000 mon.a (mon.0) 1054 : cluster [DBG] osdmap e647: 8 total, 8 up, 8 in 2024-04-07T16:26:43.908 INFO:tasks.rados.rados.0.smithi100.stdout:3104: done (3 left) 2024-04-07T16:26:43.908 INFO:tasks.rados.rados.0.smithi100.stdout:3106: done (2 left) 2024-04-07T16:26:43.908 INFO:tasks.rados.rados.0.smithi100.stdout:3107: done (1 left) 2024-04-07T16:26:43.908 INFO:tasks.rados.rados.0.smithi100.stdout:3109: done (0 left) 2024-04-07T16:26:43.909 INFO:tasks.rados.rados.0.smithi100.stdout:3110: copy_from oid 32 from oid 23 current snap is 285 2024-04-07T16:26:43.909 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 38 current snap is 285 2024-04-07T16:26:43.909 INFO:tasks.rados.rados.0.smithi100.stdout:3111: seq_num 950 ranges {4653056=720896,5373952=679936,6053888=122880} 2024-04-07T16:26:43.939 INFO:tasks.rados.rados.0.smithi100.stdout:3111: writing smithi10066867-38 from 4653056 to 5373952 tid 1 2024-04-07T16:26:43.943 INFO:tasks.rados.rados.0.smithi100.stdout:3111: writing smithi10066867-38 from 5373952 to 6053888 tid 2 2024-04-07T16:26:43.944 INFO:tasks.rados.rados.0.smithi100.stdout:3111: writing smithi10066867-38 from 6053888 to 6176768 tid 3 2024-04-07T16:26:43.944 INFO:tasks.rados.rados.0.smithi100.stdout:3112: snap_create 2024-04-07T16:26:43.944 INFO:tasks.rados.rados.0.smithi100.stdout:3110: finishing copy_from racing read to smithi10066867-32 2024-04-07T16:26:43.944 INFO:tasks.rados.rados.0.smithi100.stdout:3110: finishing copy_from to smithi10066867-32 2024-04-07T16:26:43.944 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 32 v 1686 (ObjNum 941 snap 282 seq_num 941) dirty exists 2024-04-07T16:26:43.950 INFO:tasks.rados.rados.0.smithi100.stdout:3111: finishing write tid 1 to smithi10066867-38 2024-04-07T16:26:43.954 INFO:tasks.rados.rados.0.smithi100.stdout:3111: finishing write tid 2 to smithi10066867-38 2024-04-07T16:26:43.955 INFO:tasks.rados.rados.0.smithi100.stdout:3111: finishing write tid 3 to smithi10066867-38 2024-04-07T16:26:43.955 INFO:tasks.rados.rados.0.smithi100.stdout:3111: finishing write tid 4 to smithi10066867-38 2024-04-07T16:26:43.958 INFO:tasks.rados.rados.0.smithi100.stdout:3111: finishing write tid 5 to smithi10066867-38 2024-04-07T16:26:43.958 INFO:tasks.rados.rados.0.smithi100.stdout:3111: oid 38 updating version 0 to 1141 2024-04-07T16:26:43.958 INFO:tasks.rados.rados.0.smithi100.stdout:3111: oid 38 version 1141 is already newer than 1140 2024-04-07T16:26:43.959 INFO:tasks.rados.rados.0.smithi100.stdout:3111: oid 38 updating version 1141 to 1142 2024-04-07T16:26:43.959 INFO:tasks.rados.rados.0.smithi100.stdout:3111: oid 38 version 1142 is already newer than 1139 2024-04-07T16:26:43.959 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 38 v 1142 (ObjNum 950 snap 285 seq_num 950) dirty exists 2024-04-07T16:26:43.959 INFO:tasks.rados.rados.0.smithi100.stdout:3111: left oid 38 (ObjNum 950 snap 285 seq_num 950) 2024-04-07T16:26:44.922 INFO:tasks.rados.rados.0.smithi100.stdout:3110: done (2 left) 2024-04-07T16:26:44.922 INFO:tasks.rados.rados.0.smithi100.stdout:3111: done (1 left) 2024-04-07T16:26:44.922 INFO:tasks.rados.rados.0.smithi100.stdout:3112: done (0 left) 2024-04-07T16:26:44.922 INFO:tasks.rados.rados.0.smithi100.stdout:3113: snap_remove snap 285 2024-04-07T16:26:45.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:44 smithi100 bash[74942]: cluster 2024-04-07T16:26:43.923846+0000 mon.a (mon.0) 1055 : cluster [DBG] osdmap e648: 8 total, 8 up, 8 in 2024-04-07T16:26:45.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:44 smithi100 bash[76202]: cluster 2024-04-07T16:26:43.923846+0000 mon.a (mon.0) 1055 : cluster [DBG] osdmap e648: 8 total, 8 up, 8 in 2024-04-07T16:26:45.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:44 smithi144 bash[66654]: cluster 2024-04-07T16:26:43.923846+0000 mon.a (mon.0) 1055 : cluster [DBG] osdmap e648: 8 total, 8 up, 8 in 2024-04-07T16:26:45.920 INFO:tasks.rados.rados.0.smithi100.stdout:3113: done (0 left) 2024-04-07T16:26:45.921 INFO:tasks.rados.rados.0.smithi100.stdout:3114: copy_from oid 16 from oid 21 current snap is 286 2024-04-07T16:26:45.921 INFO:tasks.rados.rados.0.smithi100.stdout:3115: rmattr oid 13 current snap is 286 2024-04-07T16:26:45.921 INFO:tasks.rados.rados.0.smithi100.stdout:3116: delete oid 42 current snap is 286 2024-04-07T16:26:45.944 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 13 v 1518 (ObjNum 65 snap 0 seq_num 2701738528) dirty exists 2024-04-07T16:26:45.954 INFO:tasks.rados.rados.0.smithi100.stdout:3115: done (2 left) 2024-04-07T16:26:45.955 INFO:tasks.rados.rados.0.smithi100.stdout:3116: done (1 left) 2024-04-07T16:26:45.955 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 46 current snap is 286 2024-04-07T16:26:45.955 INFO:tasks.rados.rados.0.smithi100.stdout:3117: seq_num 952 ranges {0=753664,753664=499712,1253376=303104} 2024-04-07T16:26:45.962 INFO:tasks.rados.rados.0.smithi100.stdout:3117: writing smithi10066867-46 from 0 to 753664 tid 1 2024-04-07T16:26:45.966 INFO:journalctl@ceph.osd.0.smithi100.stdout:Apr 07 16:26:45 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:26:45.966 INFO:journalctl@ceph.osd.1.smithi100.stdout:Apr 07 16:26:45 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:26:45.967 INFO:journalctl@ceph.osd.2.smithi100.stdout:Apr 07 16:26:45 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:26:45.967 INFO:journalctl@ceph.osd.3.smithi100.stdout:Apr 07 16:26:45 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:26:45.967 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:45 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:26:45.967 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:45 smithi100 bash[74942]: cluster 2024-04-07T16:26:44.774077+0000 mgr.y (mgr.25175) 318 : cluster [DBG] pgmap v565: 121 pgs: 121 active+clean; 91 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 5.7 MiB/s rd, 3.0 MiB/s wr, 23 op/s 2024-04-07T16:26:45.967 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:45 smithi100 bash[74942]: cluster 2024-04-07T16:26:44.936194+0000 mon.a (mon.0) 1056 : cluster [DBG] osdmap e649: 8 total, 8 up, 8 in 2024-04-07T16:26:45.967 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:45 smithi100 bash[74942]: cephadm 2024-04-07T16:26:45.199841+0000 mgr.y (mgr.25175) 319 : cephadm [INF] Upgrade: Updating node-exporter.a (1/2) 2024-04-07T16:26:45.968 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:45 smithi100 bash[74942]: cephadm 2024-04-07T16:26:45.200631+0000 mgr.y (mgr.25175) 320 : cephadm [INF] Deploying daemon node-exporter.a on smithi100 2024-04-07T16:26:45.968 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:26:45 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:26:45.968 INFO:journalctl@ceph.mds.a.smithi100.stdout:Apr 07 16:26:45 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:26:45.968 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 16:26:45 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:26:45.969 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Apr 07 16:26:45 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:26:45.969 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:45 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:26:45.969 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:45 smithi100 bash[76202]: cluster 2024-04-07T16:26:44.774077+0000 mgr.y (mgr.25175) 318 : cluster [DBG] pgmap v565: 121 pgs: 121 active+clean; 91 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 5.7 MiB/s rd, 3.0 MiB/s wr, 23 op/s 2024-04-07T16:26:45.969 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:45 smithi100 bash[76202]: cluster 2024-04-07T16:26:44.936194+0000 mon.a (mon.0) 1056 : cluster [DBG] osdmap e649: 8 total, 8 up, 8 in 2024-04-07T16:26:45.969 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:45 smithi100 bash[76202]: cephadm 2024-04-07T16:26:45.199841+0000 mgr.y (mgr.25175) 319 : cephadm [INF] Upgrade: Updating node-exporter.a (1/2) 2024-04-07T16:26:45.969 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:45 smithi100 bash[76202]: cephadm 2024-04-07T16:26:45.200631+0000 mgr.y (mgr.25175) 320 : cephadm [INF] Deploying daemon node-exporter.a on smithi100 2024-04-07T16:26:45.969 INFO:tasks.rados.rados.0.smithi100.stdout:3117: writing smithi10066867-46 from 753664 to 1253376 tid 2 2024-04-07T16:26:45.970 INFO:tasks.rados.rados.0.smithi100.stdout:3117: writing smithi10066867-46 from 1253376 to 1556480 tid 3 2024-04-07T16:26:45.970 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 27 current snap is 286 2024-04-07T16:26:45.970 INFO:tasks.rados.rados.0.smithi100.stdout:3114: finishing copy_from racing read to smithi10066867-16 2024-04-07T16:26:45.970 INFO:tasks.rados.rados.0.smithi100.stdout:3114: finishing copy_from to smithi10066867-16 2024-04-07T16:26:45.970 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 16 v 1198 (ObjNum 794 snap 251 seq_num 794) dirty exists 2024-04-07T16:26:45.970 INFO:tasks.rados.rados.0.smithi100.stdout:3118: seq_num 953 ranges {2441216=770048,3211264=786432,3997696=16384} 2024-04-07T16:26:45.990 INFO:tasks.rados.rados.0.smithi100.stdout:3118: writing smithi10066867-27 from 2441216 to 3211264 tid 1 2024-04-07T16:26:45.996 INFO:tasks.rados.rados.0.smithi100.stdout:3118: writing smithi10066867-27 from 3211264 to 3997696 tid 2 2024-04-07T16:26:45.996 INFO:tasks.rados.rados.0.smithi100.stdout:3118: writing smithi10066867-27 from 3997696 to 4014080 tid 3 2024-04-07T16:26:45.996 INFO:tasks.rados.rados.0.smithi100.stdout:3114: done (2 left) 2024-04-07T16:26:45.996 INFO:tasks.rados.rados.0.smithi100.stdout:3119: setattr oid 28 current snap is 286 2024-04-07T16:26:45.996 INFO:tasks.rados.rados.0.smithi100.stdout:3117: finishing write tid 1 to smithi10066867-46 2024-04-07T16:26:45.996 INFO:tasks.rados.rados.0.smithi100.stdout:3117: finishing write tid 2 to smithi10066867-46 2024-04-07T16:26:45.996 INFO:tasks.rados.rados.0.smithi100.stdout:3117: finishing write tid 3 to smithi10066867-46 2024-04-07T16:26:45.996 INFO:tasks.rados.rados.0.smithi100.stdout:3117: finishing write tid 4 to smithi10066867-46 2024-04-07T16:26:45.997 INFO:tasks.rados.rados.0.smithi100.stdout:3117: finishing write tid 5 to smithi10066867-46 2024-04-07T16:26:45.997 INFO:tasks.rados.rados.0.smithi100.stdout:3117: oid 46 updating version 0 to 1522 2024-04-07T16:26:45.997 INFO:tasks.rados.rados.0.smithi100.stdout:3117: oid 46 version 1522 is already newer than 1519 2024-04-07T16:26:45.997 INFO:tasks.rados.rados.0.smithi100.stdout:3117: oid 46 version 1522 is already newer than 1520 2024-04-07T16:26:45.997 INFO:tasks.rados.rados.0.smithi100.stdout:3117: oid 46 version 1522 is already newer than 1521 2024-04-07T16:26:45.997 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 46 v 1522 (ObjNum 952 snap 286 seq_num 952) dirty exists 2024-04-07T16:26:45.997 INFO:tasks.rados.rados.0.smithi100.stdout:3117: left oid 46 (ObjNum 952 snap 286 seq_num 952) 2024-04-07T16:26:45.997 INFO:tasks.rados.rados.0.smithi100.stdout:3118: finishing write tid 1 to smithi10066867-27 2024-04-07T16:26:45.998 INFO:tasks.rados.rados.0.smithi100.stdout:3117: done (2 left) 2024-04-07T16:26:45.998 INFO:tasks.rados.rados.0.smithi100.stdout:3120: copy_from oid 15 from oid 35 current snap is 286 2024-04-07T16:26:45.998 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 37 current snap is 286 2024-04-07T16:26:45.998 INFO:tasks.rados.rados.0.smithi100.stdout:3121: seq_num 954 ranges {6078464=786432,6864896=671744,7536640=131072} 2024-04-07T16:26:46.036 INFO:tasks.rados.rados.0.smithi100.stdout:3121: writing smithi10066867-37 from 6078464 to 6864896 tid 1 2024-04-07T16:26:46.040 INFO:tasks.rados.rados.0.smithi100.stdout:3121: writing smithi10066867-37 from 6864896 to 7536640 tid 2 2024-04-07T16:26:46.041 INFO:tasks.rados.rados.0.smithi100.stdout:3121: writing smithi10066867-37 from 7536640 to 7667712 tid 3 2024-04-07T16:26:46.041 INFO:tasks.rados.rados.0.smithi100.stdout:3122: copy_from oid 40 from oid 5 current snap is 286 2024-04-07T16:26:46.042 INFO:tasks.rados.rados.0.smithi100.stdout:3118: finishing write tid 2 to smithi10066867-27 2024-04-07T16:26:46.042 INFO:tasks.rados.rados.0.smithi100.stdout:3118: finishing write tid 3 to smithi10066867-27 2024-04-07T16:26:46.042 INFO:tasks.rados.rados.0.smithi100.stdout:3118: finishing write tid 4 to smithi10066867-27 2024-04-07T16:26:46.042 INFO:tasks.rados.rados.0.smithi100.stdout:3120: finishing copy_from racing read to smithi10066867-15 2024-04-07T16:26:46.042 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 28 v 1687 (ObjNum 942 snap 282 seq_num 942) dirty exists 2024-04-07T16:26:46.042 INFO:tasks.rados.rados.0.smithi100.stdout:3118: finishing write tid 5 to smithi10066867-27 2024-04-07T16:26:46.042 INFO:tasks.rados.rados.0.smithi100.stdout:3118: oid 27 updating version 0 to 1165 2024-04-07T16:26:46.042 INFO:tasks.rados.rados.0.smithi100.stdout:3118: oid 27 version 1165 is already newer than 1164 2024-04-07T16:26:46.042 INFO:tasks.rados.rados.0.smithi100.stdout:3118: oid 27 updating version 1165 to 1166 2024-04-07T16:26:46.042 INFO:tasks.rados.rados.0.smithi100.stdout:3118: oid 27 version 1166 is already newer than 1163 2024-04-07T16:26:46.042 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 27 v 1166 (ObjNum 953 snap 286 seq_num 953) dirty exists 2024-04-07T16:26:46.042 INFO:tasks.rados.rados.0.smithi100.stdout:3118: left oid 27 (ObjNum 953 snap 286 seq_num 953) 2024-04-07T16:26:46.043 INFO:tasks.rados.rados.0.smithi100.stdout:3120: finishing copy_from to smithi10066867-15 2024-04-07T16:26:46.043 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 15 v 1122 (ObjNum 561 snap 0 seq_num 3067285248) dirty exists 2024-04-07T16:26:46.043 INFO:tasks.rados.rados.0.smithi100.stdout:3118: done (4 left) 2024-04-07T16:26:46.043 INFO:tasks.rados.rados.0.smithi100.stdout:3119: done (3 left) 2024-04-07T16:26:46.043 INFO:tasks.rados.rados.0.smithi100.stdout:3120: done (2 left) 2024-04-07T16:26:46.043 INFO:tasks.rados.rados.0.smithi100.stdout:3123: delete oid 47 current snap is 286 2024-04-07T16:26:46.043 INFO:tasks.rados.rados.0.smithi100.stdout:3121: finishing write tid 1 to smithi10066867-37 2024-04-07T16:26:46.043 INFO:tasks.rados.rados.0.smithi100.stdout:3122: finishing copy_from racing read to smithi10066867-40 2024-04-07T16:26:46.045 INFO:tasks.rados.rados.0.smithi100.stdout:3123: done (2 left) 2024-04-07T16:26:46.045 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 46 current snap is 286 2024-04-07T16:26:46.045 INFO:tasks.rados.rados.0.smithi100.stdout:3124: seq_num 956 ranges {1556480=417792,1974272=434176,2408448=770048} 2024-04-07T16:26:46.056 INFO:tasks.rados.rados.0.smithi100.stdout:3124: writing smithi10066867-46 from 1556480 to 1974272 tid 1 2024-04-07T16:26:46.059 INFO:tasks.rados.rados.0.smithi100.stdout:3124: writing smithi10066867-46 from 1974272 to 2408448 tid 2 2024-04-07T16:26:46.064 INFO:tasks.rados.rados.0.smithi100.stdout:3124: writing smithi10066867-46 from 2408448 to 3178496 tid 3 2024-04-07T16:26:46.065 INFO:tasks.rados.rados.0.smithi100.stdout:3125: read oid 28 snap -1 2024-04-07T16:26:46.065 INFO:tasks.rados.rados.0.smithi100.stdout:3125: expect (ObjNum 942 snap 282 seq_num 942) 2024-04-07T16:26:46.065 INFO:tasks.rados.rados.0.smithi100.stdout:3126: rollback oid 24 current snap is 286 2024-04-07T16:26:46.065 INFO:tasks.rados.rados.0.smithi100.stdout:3126: done (4 left) 2024-04-07T16:26:46.065 INFO:tasks.rados.rados.0.smithi100.stdout:3127: snap_create 2024-04-07T16:26:46.065 INFO:tasks.rados.rados.0.smithi100.stdout:3122: finishing copy_from to smithi10066867-40 2024-04-07T16:26:46.065 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 40 v 1417 (ObjNum 161 snap 0 seq_num 0) dirty exists 2024-04-07T16:26:46.065 INFO:tasks.rados.rados.0.smithi100.stdout:3121: finishing write tid 2 to smithi10066867-37 2024-04-07T16:26:46.065 INFO:tasks.rados.rados.0.smithi100.stdout:3121: finishing write tid 3 to smithi10066867-37 2024-04-07T16:26:46.065 INFO:tasks.rados.rados.0.smithi100.stdout:3121: finishing write tid 4 to smithi10066867-37 2024-04-07T16:26:46.065 INFO:tasks.rados.rados.0.smithi100.stdout:3121: finishing write tid 5 to smithi10066867-37 2024-04-07T16:26:46.066 INFO:tasks.rados.rados.0.smithi100.stdout:3121: oid 37 updating version 0 to 1253 2024-04-07T16:26:46.066 INFO:tasks.rados.rados.0.smithi100.stdout:3121: oid 37 version 1253 is already newer than 1251 2024-04-07T16:26:46.066 INFO:tasks.rados.rados.0.smithi100.stdout:3121: oid 37 version 1253 is already newer than 1250 2024-04-07T16:26:46.066 INFO:tasks.rados.rados.0.smithi100.stdout:3121: oid 37 version 1253 is already newer than 1252 2024-04-07T16:26:46.066 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 37 v 1253 (ObjNum 954 snap 286 seq_num 954) dirty exists 2024-04-07T16:26:46.066 INFO:tasks.rados.rados.0.smithi100.stdout:3121: left oid 37 (ObjNum 954 snap 286 seq_num 954) 2024-04-07T16:26:46.066 INFO:tasks.rados.rados.0.smithi100.stdout:3124: finishing write tid 1 to smithi10066867-46 2024-04-07T16:26:46.066 INFO:tasks.rados.rados.0.smithi100.stdout:3124: finishing write tid 2 to smithi10066867-46 2024-04-07T16:26:46.073 INFO:tasks.rados.rados.0.smithi100.stdout:3124: finishing write tid 3 to smithi10066867-46 2024-04-07T16:26:46.074 INFO:tasks.rados.rados.0.smithi100.stdout:3124: finishing write tid 4 to smithi10066867-46 2024-04-07T16:26:46.080 INFO:tasks.rados.rados.0.smithi100.stdout:3124: finishing write tid 5 to smithi10066867-46 2024-04-07T16:26:46.080 INFO:tasks.rados.rados.0.smithi100.stdout:3124: oid 46 updating version 0 to 1525 2024-04-07T16:26:46.080 INFO:tasks.rados.rados.0.smithi100.stdout:3124: oid 46 version 1525 is already newer than 1523 2024-04-07T16:26:46.080 INFO:tasks.rados.rados.0.smithi100.stdout:3124: oid 46 version 1525 is already newer than 1524 2024-04-07T16:26:46.080 INFO:tasks.rados.rados.0.smithi100.stdout:3124: oid 46 updating version 1525 to 1526 2024-04-07T16:26:46.080 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 46 v 1526 (ObjNum 956 snap 286 seq_num 956) dirty exists 2024-04-07T16:26:46.080 INFO:tasks.rados.rados.0.smithi100.stdout:3124: left oid 46 (ObjNum 956 snap 286 seq_num 956) 2024-04-07T16:26:46.089 INFO:tasks.rados.rados.0.smithi100.stdout:3125: expect (ObjNum 942 snap 282 seq_num 942) 2024-04-07T16:26:46.249 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 16:26:45 smithi100 systemd[1]: Stopping Ceph node-exporter.a for 0702e3a8-f4f8-11ee-b649-cb9ed24678a4... 2024-04-07T16:26:46.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:45 smithi144 bash[66654]: cluster 2024-04-07T16:26:44.774077+0000 mgr.y (mgr.25175) 318 : cluster [DBG] pgmap v565: 121 pgs: 121 active+clean; 91 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 5.7 MiB/s rd, 3.0 MiB/s wr, 23 op/s 2024-04-07T16:26:46.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:45 smithi144 bash[66654]: cluster 2024-04-07T16:26:44.936194+0000 mon.a (mon.0) 1056 : cluster [DBG] osdmap e649: 8 total, 8 up, 8 in 2024-04-07T16:26:46.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:45 smithi144 bash[66654]: cephadm 2024-04-07T16:26:45.199841+0000 mgr.y (mgr.25175) 319 : cephadm [INF] Upgrade: Updating node-exporter.a (1/2) 2024-04-07T16:26:46.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:45 smithi144 bash[66654]: cephadm 2024-04-07T16:26:45.200631+0000 mgr.y (mgr.25175) 320 : cephadm [INF] Deploying daemon node-exporter.a on smithi100 2024-04-07T16:26:46.902 INFO:tasks.rados.rados.0.smithi100.stdout:3121: done (4 left) 2024-04-07T16:26:46.903 INFO:tasks.rados.rados.0.smithi100.stdout:3122: done (3 left) 2024-04-07T16:26:46.903 INFO:tasks.rados.rados.0.smithi100.stdout:3124: done (2 left) 2024-04-07T16:26:46.903 INFO:tasks.rados.rados.0.smithi100.stdout:3125: done (1 left) 2024-04-07T16:26:46.903 INFO:tasks.rados.rados.0.smithi100.stdout:3127: done (0 left) 2024-04-07T16:26:46.903 INFO:tasks.rados.rados.0.smithi100.stdout:3128: rmattr oid 18 current snap is 287 2024-04-07T16:26:46.903 INFO:tasks.rados.rados.0.smithi100.stdout:3128: done (0 left) 2024-04-07T16:26:46.903 INFO:tasks.rados.rados.0.smithi100.stdout:3129: rmattr oid 17 current snap is 287 2024-04-07T16:26:46.903 INFO:tasks.rados.rados.0.smithi100.stdout:3130: snap_create 2024-04-07T16:26:46.918 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 17 v 1241 (ObjNum 937 snap 280 seq_num 937) dirty exists 2024-04-07T16:26:46.931 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 16:26:46 smithi100 bash[101364]: ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4-node-exporter-a 2024-04-07T16:26:46.931 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 16:26:46 smithi100 systemd[1]: ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@node-exporter.a.service: Main process exited, code=exited, status=143/n/a 2024-04-07T16:26:46.932 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 16:26:46 smithi100 bash[101424]: Error response from daemon: No such container: ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4-node-exporter-a 2024-04-07T16:26:46.932 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 16:26:46 smithi100 systemd[1]: ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@node-exporter.a.service: Failed with result 'exit-code'. 2024-04-07T16:26:46.932 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 16:26:46 smithi100 systemd[1]: Stopped Ceph node-exporter.a for 0702e3a8-f4f8-11ee-b649-cb9ed24678a4. 2024-04-07T16:26:46.932 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:46 smithi100 bash[76202]: cluster 2024-04-07T16:26:45.933020+0000 mon.a (mon.0) 1057 : cluster [DBG] osdmap e650: 8 total, 8 up, 8 in 2024-04-07T16:26:46.932 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:46 smithi100 bash[76202]: audit 2024-04-07T16:26:46.727634+0000 mon.a (mon.0) 1058 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:26:46.932 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:46 smithi100 bash[76202]: cluster 2024-04-07T16:26:46.918650+0000 mon.a (mon.0) 1059 : cluster [DBG] osdmap e651: 8 total, 8 up, 8 in 2024-04-07T16:26:47.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:46 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:26:47.186 INFO:journalctl@ceph.osd.0.smithi100.stdout:Apr 07 16:26:46 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:26:47.187 INFO:journalctl@ceph.osd.2.smithi100.stdout:Apr 07 16:26:46 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:26:47.187 INFO:journalctl@ceph.osd.3.smithi100.stdout:Apr 07 16:26:46 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:26:47.187 INFO:journalctl@ceph.mds.a.smithi100.stdout:Apr 07 16:26:46 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:26:47.187 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:26:46 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:26:47.188 INFO:journalctl@ceph.osd.1.smithi100.stdout:Apr 07 16:26:46 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:26:47.188 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 16:26:46 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:26:47.188 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 16:26:47 smithi100 systemd[1]: Started Ceph node-exporter.a for 0702e3a8-f4f8-11ee-b649-cb9ed24678a4. 2024-04-07T16:26:47.188 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Apr 07 16:26:46 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:26:47.189 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:46 smithi100 bash[74942]: cluster 2024-04-07T16:26:45.933020+0000 mon.a (mon.0) 1057 : cluster [DBG] osdmap e650: 8 total, 8 up, 8 in 2024-04-07T16:26:47.189 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:46 smithi100 bash[74942]: audit 2024-04-07T16:26:46.727634+0000 mon.a (mon.0) 1058 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:26:47.189 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:46 smithi100 bash[74942]: cluster 2024-04-07T16:26:46.918650+0000 mon.a (mon.0) 1059 : cluster [DBG] osdmap e651: 8 total, 8 up, 8 in 2024-04-07T16:26:47.189 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:46 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:26:47.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:46 smithi144 bash[66654]: cluster 2024-04-07T16:26:45.933020+0000 mon.a (mon.0) 1057 : cluster [DBG] osdmap e650: 8 total, 8 up, 8 in 2024-04-07T16:26:47.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:46 smithi144 bash[66654]: audit 2024-04-07T16:26:46.727634+0000 mon.a (mon.0) 1058 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:26:47.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:46 smithi144 bash[66654]: cluster 2024-04-07T16:26:46.918650+0000 mon.a (mon.0) 1059 : cluster [DBG] osdmap e651: 8 total, 8 up, 8 in 2024-04-07T16:26:47.904 INFO:tasks.rados.rados.0.smithi100.stdout:3129: done (1 left) 2024-04-07T16:26:47.904 INFO:tasks.rados.rados.0.smithi100.stdout:3130: done (0 left) 2024-04-07T16:26:47.904 INFO:tasks.rados.rados.0.smithi100.stdout:3131: delete oid 49 current snap is 288 2024-04-07T16:26:47.907 INFO:tasks.rados.rados.0.smithi100.stdout:3131: done (0 left) 2024-04-07T16:26:47.907 INFO:tasks.rados.rados.0.smithi100.stdout:3132: rmattr oid 20 current snap is 288 2024-04-07T16:26:47.907 INFO:tasks.rados.rados.0.smithi100.stdout:3133: copy_from oid 28 from oid 24 current snap is 288 2024-04-07T16:26:47.907 INFO:tasks.rados.rados.0.smithi100.stdout:3134: rollback oid 14 current snap is 288 2024-04-07T16:26:47.908 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 14 to 287 2024-04-07T16:26:47.908 INFO:tasks.rados.rados.0.smithi100.stdout:3135: read oid 36 snap -1 2024-04-07T16:26:47.908 INFO:tasks.rados.rados.0.smithi100.stdout:3135: expect (ObjNum 949 snap 285 seq_num 949) 2024-04-07T16:26:47.908 INFO:tasks.rados.rados.0.smithi100.stdout:3136: snap_create 2024-04-07T16:26:47.908 INFO:tasks.rados.rados.0.smithi100.stdout:3134: finishing rollback tid 1 to smithi10066867-14 2024-04-07T16:26:47.908 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 14 v 0 (ObjNum 289 snap 0 seq_num 4240428767) dirty dne 2024-04-07T16:26:47.922 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 20 v 1229 (ObjNum 880 snap 264 seq_num 880) dirty exists 2024-04-07T16:26:47.925 INFO:tasks.rados.rados.0.smithi100.stdout:3135: expect (ObjNum 949 snap 285 seq_num 949) 2024-04-07T16:26:47.994 INFO:tasks.rados.rados.0.smithi100.stdout:3133: finishing copy_from racing read to smithi10066867-28 2024-04-07T16:26:47.994 INFO:tasks.rados.rados.0.smithi100.stdout:3133: finishing copy_from to smithi10066867-28 2024-04-07T16:26:47.994 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 28 v 1236 (ObjNum 948 snap 285 seq_num 948) dirty exists 2024-04-07T16:26:48.417 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:48 smithi100 bash[74942]: cluster 2024-04-07T16:26:46.774909+0000 mgr.y (mgr.25175) 321 : cluster [DBG] pgmap v568: 121 pgs: 121 active+clean; 91 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 4.3 MiB/s rd, 1.2 MiB/s wr, 14 op/s 2024-04-07T16:26:48.417 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:48 smithi100 bash[74942]: audit 2024-04-07T16:26:47.141252+0000 mon.a (mon.0) 1060 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:26:48.417 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:48 smithi100 bash[74942]: audit 2024-04-07T16:26:47.151110+0000 mon.a (mon.0) 1061 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:26:48.417 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:48 smithi100 bash[74942]: cluster 2024-04-07T16:26:47.918424+0000 mon.a (mon.0) 1062 : cluster [DBG] osdmap e652: 8 total, 8 up, 8 in 2024-04-07T16:26:48.417 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:48 smithi100 bash[76202]: cluster 2024-04-07T16:26:46.774909+0000 mgr.y (mgr.25175) 321 : cluster [DBG] pgmap v568: 121 pgs: 121 active+clean; 91 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 4.3 MiB/s rd, 1.2 MiB/s wr, 14 op/s 2024-04-07T16:26:48.418 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:48 smithi100 bash[76202]: audit 2024-04-07T16:26:47.141252+0000 mon.a (mon.0) 1060 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:26:48.418 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:48 smithi100 bash[76202]: audit 2024-04-07T16:26:47.151110+0000 mon.a (mon.0) 1061 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:26:48.418 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:48 smithi100 bash[76202]: cluster 2024-04-07T16:26:47.918424+0000 mon.a (mon.0) 1062 : cluster [DBG] osdmap e652: 8 total, 8 up, 8 in 2024-04-07T16:26:48.418 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 16:26:48 smithi100 bash[101481]: ts=2024-04-07T16:26:48.109Z caller=node_exporter.go:180 level=info msg="Starting node_exporter" version="(version=1.5.0, branch=HEAD, revision=1b48970ffcf5630534fb00bb0687d73c66d1c959)" 2024-04-07T16:26:48.418 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 16:26:48 smithi100 bash[101481]: ts=2024-04-07T16:26:48.109Z caller=node_exporter.go:181 level=info msg="Build context" build_context="(go=go1.19.3, user=root@6e7732a7b81b, date=20221129-18:59:09)" 2024-04-07T16:26:48.418 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 16:26:48 smithi100 bash[101481]: ts=2024-04-07T16:26:48.109Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+|var/lib/containers/storage/.+)($|/) 2024-04-07T16:26:48.418 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 16:26:48 smithi100 bash[101481]: ts=2024-04-07T16:26:48.109Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2024-04-07T16:26:48.418 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 16:26:48 smithi100 bash[101481]: ts=2024-04-07T16:26:48.109Z caller=diskstats_common.go:111 level=info collector=diskstats msg="Parsed flag --collector.diskstats.device-exclude" flag=^(ram|loop|fd|(h|s|v|xv)d[a-z]|nvme\d+n\d+p)\d+$ 2024-04-07T16:26:48.418 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 16:26:48 smithi100 bash[101481]: ts=2024-04-07T16:26:48.109Z caller=diskstats_linux.go:264 level=error collector=diskstats msg="Failed to open directory, disabling udev device properties" path=/run/udev/data 2024-04-07T16:26:48.419 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 16:26:48 smithi100 bash[101481]: ts=2024-04-07T16:26:48.110Z caller=node_exporter.go:110 level=info msg="Enabled collectors" 2024-04-07T16:26:48.419 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 16:26:48 smithi100 bash[101481]: ts=2024-04-07T16:26:48.110Z caller=node_exporter.go:117 level=info collector=arp 2024-04-07T16:26:48.419 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 16:26:48 smithi100 bash[101481]: ts=2024-04-07T16:26:48.110Z caller=node_exporter.go:117 level=info collector=bcache 2024-04-07T16:26:48.419 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 16:26:48 smithi100 bash[101481]: ts=2024-04-07T16:26:48.110Z caller=node_exporter.go:117 level=info collector=bonding 2024-04-07T16:26:48.419 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 16:26:48 smithi100 bash[101481]: ts=2024-04-07T16:26:48.110Z caller=node_exporter.go:117 level=info collector=btrfs 2024-04-07T16:26:48.419 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 16:26:48 smithi100 bash[101481]: ts=2024-04-07T16:26:48.110Z caller=node_exporter.go:117 level=info collector=conntrack 2024-04-07T16:26:48.419 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 16:26:48 smithi100 bash[101481]: ts=2024-04-07T16:26:48.110Z caller=node_exporter.go:117 level=info collector=cpu 2024-04-07T16:26:48.419 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 16:26:48 smithi100 bash[101481]: ts=2024-04-07T16:26:48.110Z caller=node_exporter.go:117 level=info collector=cpufreq 2024-04-07T16:26:48.419 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 16:26:48 smithi100 bash[101481]: ts=2024-04-07T16:26:48.110Z caller=node_exporter.go:117 level=info collector=diskstats 2024-04-07T16:26:48.419 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 16:26:48 smithi100 bash[101481]: ts=2024-04-07T16:26:48.110Z caller=node_exporter.go:117 level=info collector=dmi 2024-04-07T16:26:48.419 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 16:26:48 smithi100 bash[101481]: ts=2024-04-07T16:26:48.110Z caller=node_exporter.go:117 level=info collector=edac 2024-04-07T16:26:48.419 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 16:26:48 smithi100 bash[101481]: ts=2024-04-07T16:26:48.110Z caller=node_exporter.go:117 level=info collector=entropy 2024-04-07T16:26:48.420 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 16:26:48 smithi100 bash[101481]: ts=2024-04-07T16:26:48.110Z caller=node_exporter.go:117 level=info collector=fibrechannel 2024-04-07T16:26:48.420 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 16:26:48 smithi100 bash[101481]: ts=2024-04-07T16:26:48.110Z caller=node_exporter.go:117 level=info collector=filefd 2024-04-07T16:26:48.420 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 16:26:48 smithi100 bash[101481]: ts=2024-04-07T16:26:48.110Z caller=node_exporter.go:117 level=info collector=filesystem 2024-04-07T16:26:48.420 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 16:26:48 smithi100 bash[101481]: ts=2024-04-07T16:26:48.110Z caller=node_exporter.go:117 level=info collector=hwmon 2024-04-07T16:26:48.420 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 16:26:48 smithi100 bash[101481]: ts=2024-04-07T16:26:48.110Z caller=node_exporter.go:117 level=info collector=infiniband 2024-04-07T16:26:48.420 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 16:26:48 smithi100 bash[101481]: ts=2024-04-07T16:26:48.110Z caller=node_exporter.go:117 level=info collector=ipvs 2024-04-07T16:26:48.420 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 16:26:48 smithi100 bash[101481]: ts=2024-04-07T16:26:48.110Z caller=node_exporter.go:117 level=info collector=loadavg 2024-04-07T16:26:48.420 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 16:26:48 smithi100 bash[101481]: ts=2024-04-07T16:26:48.110Z caller=node_exporter.go:117 level=info collector=mdadm 2024-04-07T16:26:48.421 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 16:26:48 smithi100 bash[101481]: ts=2024-04-07T16:26:48.110Z caller=node_exporter.go:117 level=info collector=meminfo 2024-04-07T16:26:48.421 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 16:26:48 smithi100 bash[101481]: ts=2024-04-07T16:26:48.110Z caller=node_exporter.go:117 level=info collector=netclass 2024-04-07T16:26:48.421 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 16:26:48 smithi100 bash[101481]: ts=2024-04-07T16:26:48.110Z caller=node_exporter.go:117 level=info collector=netdev 2024-04-07T16:26:48.421 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 16:26:48 smithi100 bash[101481]: ts=2024-04-07T16:26:48.110Z caller=node_exporter.go:117 level=info collector=netstat 2024-04-07T16:26:48.421 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 16:26:48 smithi100 bash[101481]: ts=2024-04-07T16:26:48.110Z caller=node_exporter.go:117 level=info collector=nfs 2024-04-07T16:26:48.421 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 16:26:48 smithi100 bash[101481]: ts=2024-04-07T16:26:48.110Z caller=node_exporter.go:117 level=info collector=nfsd 2024-04-07T16:26:48.421 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 16:26:48 smithi100 bash[101481]: ts=2024-04-07T16:26:48.110Z caller=node_exporter.go:117 level=info collector=nvme 2024-04-07T16:26:48.421 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 16:26:48 smithi100 bash[101481]: ts=2024-04-07T16:26:48.110Z caller=node_exporter.go:117 level=info collector=os 2024-04-07T16:26:48.421 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 16:26:48 smithi100 bash[101481]: ts=2024-04-07T16:26:48.110Z caller=node_exporter.go:117 level=info collector=powersupplyclass 2024-04-07T16:26:48.421 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 16:26:48 smithi100 bash[101481]: ts=2024-04-07T16:26:48.110Z caller=node_exporter.go:117 level=info collector=pressure 2024-04-07T16:26:48.421 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 16:26:48 smithi100 bash[101481]: ts=2024-04-07T16:26:48.110Z caller=node_exporter.go:117 level=info collector=rapl 2024-04-07T16:26:48.421 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 16:26:48 smithi100 bash[101481]: ts=2024-04-07T16:26:48.110Z caller=node_exporter.go:117 level=info collector=schedstat 2024-04-07T16:26:48.422 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 16:26:48 smithi100 bash[101481]: ts=2024-04-07T16:26:48.110Z caller=node_exporter.go:117 level=info collector=selinux 2024-04-07T16:26:48.422 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 16:26:48 smithi100 bash[101481]: ts=2024-04-07T16:26:48.110Z caller=node_exporter.go:117 level=info collector=sockstat 2024-04-07T16:26:48.422 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 16:26:48 smithi100 bash[101481]: ts=2024-04-07T16:26:48.110Z caller=node_exporter.go:117 level=info collector=softnet 2024-04-07T16:26:48.422 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 16:26:48 smithi100 bash[101481]: ts=2024-04-07T16:26:48.110Z caller=node_exporter.go:117 level=info collector=stat 2024-04-07T16:26:48.422 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 16:26:48 smithi100 bash[101481]: ts=2024-04-07T16:26:48.110Z caller=node_exporter.go:117 level=info collector=tapestats 2024-04-07T16:26:48.422 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 16:26:48 smithi100 bash[101481]: ts=2024-04-07T16:26:48.110Z caller=node_exporter.go:117 level=info collector=textfile 2024-04-07T16:26:48.422 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 16:26:48 smithi100 bash[101481]: ts=2024-04-07T16:26:48.110Z caller=node_exporter.go:117 level=info collector=thermal_zone 2024-04-07T16:26:48.422 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 16:26:48 smithi100 bash[101481]: ts=2024-04-07T16:26:48.110Z caller=node_exporter.go:117 level=info collector=time 2024-04-07T16:26:48.422 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 16:26:48 smithi100 bash[101481]: ts=2024-04-07T16:26:48.110Z caller=node_exporter.go:117 level=info collector=udp_queues 2024-04-07T16:26:48.422 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 16:26:48 smithi100 bash[101481]: ts=2024-04-07T16:26:48.110Z caller=node_exporter.go:117 level=info collector=uname 2024-04-07T16:26:48.422 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 16:26:48 smithi100 bash[101481]: ts=2024-04-07T16:26:48.110Z caller=node_exporter.go:117 level=info collector=vmstat 2024-04-07T16:26:48.423 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 16:26:48 smithi100 bash[101481]: ts=2024-04-07T16:26:48.110Z caller=node_exporter.go:117 level=info collector=xfs 2024-04-07T16:26:48.423 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 16:26:48 smithi100 bash[101481]: ts=2024-04-07T16:26:48.110Z caller=node_exporter.go:117 level=info collector=zfs 2024-04-07T16:26:48.423 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 16:26:48 smithi100 bash[101481]: ts=2024-04-07T16:26:48.111Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9100 2024-04-07T16:26:48.423 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 16:26:48 smithi100 bash[101481]: ts=2024-04-07T16:26:48.111Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9100 2024-04-07T16:26:48.499 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:48 smithi144 bash[66654]: cluster 2024-04-07T16:26:46.774909+0000 mgr.y (mgr.25175) 321 : cluster [DBG] pgmap v568: 121 pgs: 121 active+clean; 91 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 4.3 MiB/s rd, 1.2 MiB/s wr, 14 op/s 2024-04-07T16:26:48.499 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:48 smithi144 bash[66654]: audit 2024-04-07T16:26:47.141252+0000 mon.a (mon.0) 1060 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:26:48.499 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:48 smithi144 bash[66654]: audit 2024-04-07T16:26:47.151110+0000 mon.a (mon.0) 1061 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:26:48.499 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:48 smithi144 bash[66654]: cluster 2024-04-07T16:26:47.918424+0000 mon.a (mon.0) 1062 : cluster [DBG] osdmap e652: 8 total, 8 up, 8 in 2024-04-07T16:26:48.686 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:26:48 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:26:48] "GET /metrics HTTP/1.1" 200 36338 "" "Prometheus/2.43.0" 2024-04-07T16:26:48.907 INFO:tasks.rados.rados.0.smithi100.stdout:3132: done (4 left) 2024-04-07T16:26:48.908 INFO:tasks.rados.rados.0.smithi100.stdout:3133: done (3 left) 2024-04-07T16:26:48.908 INFO:tasks.rados.rados.0.smithi100.stdout:3134: done (2 left) 2024-04-07T16:26:48.908 INFO:tasks.rados.rados.0.smithi100.stdout:3135: done (1 left) 2024-04-07T16:26:48.908 INFO:tasks.rados.rados.0.smithi100.stdout:3136: done (0 left) 2024-04-07T16:26:48.908 INFO:tasks.rados.rados.0.smithi100.stdout:3137: delete oid 25 current snap is 289 2024-04-07T16:26:48.912 INFO:tasks.rados.rados.0.smithi100.stdout:3137: done (0 left) 2024-04-07T16:26:48.912 INFO:tasks.rados.rados.0.smithi100.stdout:3138: read oid 49 snap -1 2024-04-07T16:26:48.912 INFO:tasks.rados.rados.0.smithi100.stdout:3138: expect deleted 2024-04-07T16:26:48.912 INFO:tasks.rados.rados.0.smithi100.stdout:3139: delete oid 29 current snap is 289 2024-04-07T16:26:48.922 INFO:tasks.rados.rados.0.smithi100.stdout:3138: done (1 left) 2024-04-07T16:26:48.922 INFO:tasks.rados.rados.0.smithi100.stdout:3139: done (0 left) 2024-04-07T16:26:48.922 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 50 current snap is 289 2024-04-07T16:26:48.922 INFO:tasks.rados.rados.0.smithi100.stdout:3140: seq_num 960 ranges {1548288=483328,2031616=425984,2457600=761856,3219456=16384} 2024-04-07T16:26:48.940 INFO:tasks.rados.rados.0.smithi100.stdout:3140: writing smithi10066867-50 from 1548288 to 2031616 tid 1 2024-04-07T16:26:48.944 INFO:tasks.rados.rados.0.smithi100.stdout:3140: writing smithi10066867-50 from 2031616 to 2457600 tid 2 2024-04-07T16:26:48.951 INFO:tasks.rados.rados.0.smithi100.stdout:3140: writing smithi10066867-50 from 2457600 to 3219456 tid 3 2024-04-07T16:26:48.952 INFO:tasks.rados.rados.0.smithi100.stdout:3140: writing smithi10066867-50 from 3219456 to 3235840 tid 4 2024-04-07T16:26:48.952 INFO:tasks.rados.rados.0.smithi100.stdout:3141: copy_from oid 37 from oid 31 current snap is 289 2024-04-07T16:26:48.952 INFO:tasks.rados.rados.0.smithi100.stdout:3142: read oid 33 snap 287 2024-04-07T16:26:48.952 INFO:tasks.rados.rados.0.smithi100.stdout:3142: expect (ObjNum 753 snap 235 seq_num 753) 2024-04-07T16:26:48.952 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 17 current snap is 289 2024-04-07T16:26:48.952 INFO:tasks.rados.rados.0.smithi100.stdout:3143: seq_num 961 ranges {1302528=499712,1802240=712704,2514944=491520} 2024-04-07T16:26:48.969 INFO:tasks.rados.rados.0.smithi100.stdout:3143: writing smithi10066867-17 from 1302528 to 1802240 tid 1 2024-04-07T16:26:48.976 INFO:tasks.rados.rados.0.smithi100.stdout:3143: writing smithi10066867-17 from 1802240 to 2514944 tid 2 2024-04-07T16:26:48.981 INFO:tasks.rados.rados.0.smithi100.stdout:3143: writing smithi10066867-17 from 2514944 to 3006464 tid 3 2024-04-07T16:26:48.981 INFO:tasks.rados.rados.0.smithi100.stdout:3144: snap_remove snap 288 2024-04-07T16:26:49.914 INFO:tasks.rados.rados.0.smithi100.stdout:3140: finishing write tid 1 to smithi10066867-50 2024-04-07T16:26:49.914 INFO:tasks.rados.rados.0.smithi100.stdout:3140: finishing write tid 2 to smithi10066867-50 2024-04-07T16:26:49.915 INFO:tasks.rados.rados.0.smithi100.stdout:3140: finishing write tid 3 to smithi10066867-50 2024-04-07T16:26:49.915 INFO:tasks.rados.rados.0.smithi100.stdout:3142: expect (ObjNum 753 snap 235 seq_num 753) 2024-04-07T16:26:49.934 INFO:tasks.rados.rados.0.smithi100.stdout:3140: finishing write tid 4 to smithi10066867-50 2024-04-07T16:26:49.934 INFO:tasks.rados.rados.0.smithi100.stdout:3140: finishing write tid 5 to smithi10066867-50 2024-04-07T16:26:49.935 INFO:tasks.rados.rados.0.smithi100.stdout:3141: finishing copy_from racing read to smithi10066867-37 2024-04-07T16:26:49.935 INFO:tasks.rados.rados.0.smithi100.stdout:3140: finishing write tid 6 to smithi10066867-50 2024-04-07T16:26:49.935 INFO:tasks.rados.rados.0.smithi100.stdout:3140: oid 50 updating version 0 to 1518 2024-04-07T16:26:49.935 INFO:tasks.rados.rados.0.smithi100.stdout:3140: oid 50 updating version 1518 to 1519 2024-04-07T16:26:49.935 INFO:tasks.rados.rados.0.smithi100.stdout:3140: oid 50 updating version 1519 to 1520 2024-04-07T16:26:49.935 INFO:tasks.rados.rados.0.smithi100.stdout:3140: oid 50 updating version 1520 to 1521 2024-04-07T16:26:49.935 INFO:tasks.rados.rados.0.smithi100.stdout:3140: oid 50 version 1521 is already newer than 1517 2024-04-07T16:26:49.935 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 50 v 1521 (ObjNum 960 snap 289 seq_num 960) dirty exists 2024-04-07T16:26:49.935 INFO:tasks.rados.rados.0.smithi100.stdout:3140: left oid 50 (ObjNum 960 snap 289 seq_num 960) 2024-04-07T16:26:49.935 INFO:tasks.rados.rados.0.smithi100.stdout:3141: finishing copy_from to smithi10066867-37 2024-04-07T16:26:49.935 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 37 v 1128 (ObjNum 939 snap 281 seq_num 939) dirty exists 2024-04-07T16:26:49.935 INFO:tasks.rados.rados.0.smithi100.stdout:3143: finishing write tid 1 to smithi10066867-17 2024-04-07T16:26:49.936 INFO:tasks.rados.rados.0.smithi100.stdout:3143: finishing write tid 2 to smithi10066867-17 2024-04-07T16:26:49.936 INFO:tasks.rados.rados.0.smithi100.stdout:3143: finishing write tid 3 to smithi10066867-17 2024-04-07T16:26:49.936 INFO:tasks.rados.rados.0.smithi100.stdout:3140: done (4 left) 2024-04-07T16:26:49.936 INFO:tasks.rados.rados.0.smithi100.stdout:3141: done (3 left) 2024-04-07T16:26:49.936 INFO:tasks.rados.rados.0.smithi100.stdout:3142: done (2 left) 2024-04-07T16:26:49.936 INFO:tasks.rados.rados.0.smithi100.stdout:3144: done (1 left) 2024-04-07T16:26:49.936 INFO:tasks.rados.rados.0.smithi100.stdout:3145: snap_create 2024-04-07T16:26:49.936 INFO:tasks.rados.rados.0.smithi100.stdout:3143: finishing write tid 4 to smithi10066867-17 2024-04-07T16:26:49.936 INFO:tasks.rados.rados.0.smithi100.stdout:3143: finishing write tid 5 to smithi10066867-17 2024-04-07T16:26:49.936 INFO:tasks.rados.rados.0.smithi100.stdout:3143: oid 17 updating version 0 to 1244 2024-04-07T16:26:49.936 INFO:tasks.rados.rados.0.smithi100.stdout:3143: oid 17 updating version 1244 to 1245 2024-04-07T16:26:49.936 INFO:tasks.rados.rados.0.smithi100.stdout:3143: oid 17 version 1245 is already newer than 1243 2024-04-07T16:26:49.937 INFO:tasks.rados.rados.0.smithi100.stdout:3143: oid 17 version 1245 is already newer than 1242 2024-04-07T16:26:49.937 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 17 v 1245 (ObjNum 961 snap 289 seq_num 961) dirty exists 2024-04-07T16:26:49.937 INFO:tasks.rados.rados.0.smithi100.stdout:3143: left oid 17 (ObjNum 961 snap 289 seq_num 961) 2024-04-07T16:26:50.072 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:26:49 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:26:50.072 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:26:49 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:26:50.073 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:49 smithi144 bash[66654]: cluster 2024-04-07T16:26:48.775786+0000 mgr.y (mgr.25175) 322 : cluster [DBG] pgmap v571: 121 pgs: 121 active+clean; 93 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 6.1 MiB/s rd, 1.4 MiB/s wr, 11 op/s 2024-04-07T16:26:50.073 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:49 smithi144 bash[66654]: cluster 2024-04-07T16:26:48.922044+0000 mon.a (mon.0) 1063 : cluster [DBG] osdmap e653: 8 total, 8 up, 8 in 2024-04-07T16:26:50.073 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:49 smithi144 bash[66654]: cephadm 2024-04-07T16:26:49.296739+0000 mgr.y (mgr.25175) 323 : cephadm [INF] Upgrade: Updating node-exporter.b (2/2) 2024-04-07T16:26:50.073 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:49 smithi144 bash[66654]: cephadm 2024-04-07T16:26:49.297601+0000 mgr.y (mgr.25175) 324 : cephadm [INF] Deploying daemon node-exporter.b on smithi144 2024-04-07T16:26:50.073 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:49 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:26:50.073 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 16:26:49 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:26:50.073 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 16:26:50 smithi144 systemd[1]: Stopping Ceph node-exporter.b for 0702e3a8-f4f8-11ee-b649-cb9ed24678a4... 2024-04-07T16:26:50.074 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:26:49 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:26:50.074 INFO:journalctl@ceph.osd.4.smithi144.stdout:Apr 07 16:26:49 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:26:50.074 INFO:journalctl@ceph.osd.5.smithi144.stdout:Apr 07 16:26:49 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:26:50.074 INFO:journalctl@ceph.osd.6.smithi144.stdout:Apr 07 16:26:49 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:26:50.075 INFO:journalctl@ceph.osd.7.smithi144.stdout:Apr 07 16:26:49 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:26:50.075 INFO:journalctl@ceph.mds.b.smithi144.stdout:Apr 07 16:26:49 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:26:50.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:49 smithi100 bash[74942]: cluster 2024-04-07T16:26:48.775786+0000 mgr.y (mgr.25175) 322 : cluster [DBG] pgmap v571: 121 pgs: 121 active+clean; 93 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 6.1 MiB/s rd, 1.4 MiB/s wr, 11 op/s 2024-04-07T16:26:50.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:49 smithi100 bash[74942]: cluster 2024-04-07T16:26:48.922044+0000 mon.a (mon.0) 1063 : cluster [DBG] osdmap e653: 8 total, 8 up, 8 in 2024-04-07T16:26:50.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:49 smithi100 bash[74942]: cephadm 2024-04-07T16:26:49.296739+0000 mgr.y (mgr.25175) 323 : cephadm [INF] Upgrade: Updating node-exporter.b (2/2) 2024-04-07T16:26:50.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:49 smithi100 bash[74942]: cephadm 2024-04-07T16:26:49.297601+0000 mgr.y (mgr.25175) 324 : cephadm [INF] Deploying daemon node-exporter.b on smithi144 2024-04-07T16:26:50.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:49 smithi100 bash[76202]: cluster 2024-04-07T16:26:48.775786+0000 mgr.y (mgr.25175) 322 : cluster [DBG] pgmap v571: 121 pgs: 121 active+clean; 93 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 6.1 MiB/s rd, 1.4 MiB/s wr, 11 op/s 2024-04-07T16:26:50.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:49 smithi100 bash[76202]: cluster 2024-04-07T16:26:48.922044+0000 mon.a (mon.0) 1063 : cluster [DBG] osdmap e653: 8 total, 8 up, 8 in 2024-04-07T16:26:50.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:49 smithi100 bash[76202]: cephadm 2024-04-07T16:26:49.296739+0000 mgr.y (mgr.25175) 323 : cephadm [INF] Upgrade: Updating node-exporter.b (2/2) 2024-04-07T16:26:50.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:49 smithi100 bash[76202]: cephadm 2024-04-07T16:26:49.297601+0000 mgr.y (mgr.25175) 324 : cephadm [INF] Deploying daemon node-exporter.b on smithi144 2024-04-07T16:26:50.937 INFO:tasks.rados.rados.0.smithi100.stdout:3143: done (1 left) 2024-04-07T16:26:50.937 INFO:tasks.rados.rados.0.smithi100.stdout:3145: done (0 left) 2024-04-07T16:26:50.938 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 34 current snap is 290 2024-04-07T16:26:50.938 INFO:tasks.rados.rados.0.smithi100.stdout:3146: seq_num 962 ranges {0=516096,516096=589824,1105920=614400} 2024-04-07T16:26:50.945 INFO:tasks.rados.rados.0.smithi100.stdout:3146: writing smithi10066867-34 from 0 to 516096 tid 1 2024-04-07T16:26:50.953 INFO:tasks.rados.rados.0.smithi100.stdout:3146: writing smithi10066867-34 from 516096 to 1105920 tid 2 2024-04-07T16:26:50.964 INFO:tasks.rados.rados.0.smithi100.stdout:3146: writing smithi10066867-34 from 1105920 to 1720320 tid 3 2024-04-07T16:26:50.964 INFO:tasks.rados.rados.0.smithi100.stdout:3147: setattr oid 41 current snap is 290 2024-04-07T16:26:50.964 INFO:tasks.rados.rados.0.smithi100.stdout:3146: finishing write tid 1 to smithi10066867-34 2024-04-07T16:26:50.964 INFO:tasks.rados.rados.0.smithi100.stdout:3146: finishing write tid 2 to smithi10066867-34 2024-04-07T16:26:50.966 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 40 current snap is 290 2024-04-07T16:26:50.966 INFO:tasks.rados.rados.0.smithi100.stdout:3148: seq_num 963 ranges {0=532480,532480=475136,1007616=532480,1540096=196608} 2024-04-07T16:26:50.974 INFO:tasks.rados.rados.0.smithi100.stdout:3148: writing smithi10066867-40 from 0 to 532480 tid 1 2024-04-07T16:26:50.982 INFO:tasks.rados.rados.0.smithi100.stdout:3148: writing smithi10066867-40 from 532480 to 1007616 tid 2 2024-04-07T16:26:50.990 INFO:tasks.rados.rados.0.smithi100.stdout:3148: writing smithi10066867-40 from 1007616 to 1540096 tid 3 2024-04-07T16:26:50.993 INFO:tasks.rados.rados.0.smithi100.stdout:3148: writing smithi10066867-40 from 1540096 to 1736704 tid 4 2024-04-07T16:26:50.993 INFO:tasks.rados.rados.0.smithi100.stdout:3149: read oid 47 snap 286 2024-04-07T16:26:50.993 INFO:tasks.rados.rados.0.smithi100.stdout:3149: expect deleted 2024-04-07T16:26:50.993 INFO:tasks.rados.rados.0.smithi100.stdout:3150: read oid 12 snap -1 2024-04-07T16:26:50.993 INFO:tasks.rados.rados.0.smithi100.stdout:3150: expect (ObjNum 829 snap 260 seq_num 829) 2024-04-07T16:26:50.994 INFO:tasks.rados.rados.0.smithi100.stdout:3151: copy_from oid 35 from oid 23 current snap is 290 2024-04-07T16:26:50.994 INFO:tasks.rados.rados.0.smithi100.stdout:3146: finishing write tid 3 to smithi10066867-34 2024-04-07T16:26:50.994 INFO:tasks.rados.rados.0.smithi100.stdout:3146: finishing write tid 4 to smithi10066867-34 2024-04-07T16:26:50.994 INFO:tasks.rados.rados.0.smithi100.stdout:3152: snap_create 2024-04-07T16:26:50.994 INFO:tasks.rados.rados.0.smithi100.stdout:3146: finishing write tid 5 to smithi10066867-34 2024-04-07T16:26:50.994 INFO:tasks.rados.rados.0.smithi100.stdout:3146: oid 34 updating version 0 to 1529 2024-04-07T16:26:50.994 INFO:tasks.rados.rados.0.smithi100.stdout:3146: oid 34 updating version 1529 to 1530 2024-04-07T16:26:50.994 INFO:tasks.rados.rados.0.smithi100.stdout:3146: oid 34 version 1530 is already newer than 1527 2024-04-07T16:26:50.994 INFO:tasks.rados.rados.0.smithi100.stdout:3146: oid 34 version 1530 is already newer than 1528 2024-04-07T16:26:50.994 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 34 v 1530 (ObjNum 962 snap 290 seq_num 962) dirty exists 2024-04-07T16:26:50.994 INFO:tasks.rados.rados.0.smithi100.stdout:3146: left oid 34 (ObjNum 962 snap 290 seq_num 962) 2024-04-07T16:26:50.994 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 41 v 1461 (ObjNum 161 snap 0 seq_num 1) dirty exists 2024-04-07T16:26:50.994 INFO:tasks.rados.rados.0.smithi100.stdout:3148: finishing write tid 1 to smithi10066867-40 2024-04-07T16:26:50.994 INFO:tasks.rados.rados.0.smithi100.stdout:3148: finishing write tid 2 to smithi10066867-40 2024-04-07T16:26:51.002 INFO:tasks.rados.rados.0.smithi100.stdout:3148: finishing write tid 3 to smithi10066867-40 2024-04-07T16:26:51.002 INFO:tasks.rados.rados.0.smithi100.stdout:3148: finishing write tid 4 to smithi10066867-40 2024-04-07T16:26:51.003 INFO:tasks.rados.rados.0.smithi100.stdout:3148: finishing write tid 5 to smithi10066867-40 2024-04-07T16:26:51.007 INFO:tasks.rados.rados.0.smithi100.stdout:3151: finishing copy_from racing read to smithi10066867-35 2024-04-07T16:26:51.009 INFO:tasks.rados.rados.0.smithi100.stdout:3148: finishing write tid 6 to smithi10066867-40 2024-04-07T16:26:51.009 INFO:tasks.rados.rados.0.smithi100.stdout:3148: oid 40 updating version 0 to 1622 2024-04-07T16:26:51.009 INFO:tasks.rados.rados.0.smithi100.stdout:3148: oid 40 updating version 1622 to 1626 2024-04-07T16:26:51.010 INFO:tasks.rados.rados.0.smithi100.stdout:3148: oid 40 version 1626 is already newer than 1625 2024-04-07T16:26:51.010 INFO:tasks.rados.rados.0.smithi100.stdout:3148: oid 40 version 1626 is already newer than 1624 2024-04-07T16:26:51.010 INFO:tasks.rados.rados.0.smithi100.stdout:3148: oid 40 version 1626 is already newer than 1623 2024-04-07T16:26:51.010 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 40 v 1626 (ObjNum 963 snap 290 seq_num 963) dirty exists 2024-04-07T16:26:51.010 INFO:tasks.rados.rados.0.smithi100.stdout:3148: left oid 40 (ObjNum 963 snap 290 seq_num 963) 2024-04-07T16:26:51.010 INFO:tasks.rados.rados.0.smithi100.stdout:3150: expect (ObjNum 829 snap 260 seq_num 829) 2024-04-07T16:26:51.091 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:50 smithi144 bash[66654]: cluster 2024-04-07T16:26:49.924023+0000 mon.a (mon.0) 1064 : cluster [DBG] osdmap e654: 8 total, 8 up, 8 in 2024-04-07T16:26:51.092 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 16:26:50 smithi144 bash[91539]: ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4-node-exporter-b 2024-04-07T16:26:51.092 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 16:26:50 smithi144 bash[91601]: ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4-node-exporter-b 2024-04-07T16:26:51.092 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 16:26:50 smithi144 systemd[1]: ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@node-exporter.b.service: Main process exited, code=exited, status=143/n/a 2024-04-07T16:26:51.092 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 16:26:50 smithi144 systemd[1]: ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@node-exporter.b.service: Failed with result 'exit-code'. 2024-04-07T16:26:51.092 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 16:26:50 smithi144 systemd[1]: Stopped Ceph node-exporter.b for 0702e3a8-f4f8-11ee-b649-cb9ed24678a4. 2024-04-07T16:26:51.102 INFO:tasks.rados.rados.0.smithi100.stdout:3151: finishing copy_from to smithi10066867-35 2024-04-07T16:26:51.102 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 35 v 1686 (ObjNum 941 snap 282 seq_num 941) dirty exists 2024-04-07T16:26:51.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:50 smithi100 bash[74942]: cluster 2024-04-07T16:26:49.924023+0000 mon.a (mon.0) 1064 : cluster [DBG] osdmap e654: 8 total, 8 up, 8 in 2024-04-07T16:26:51.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:50 smithi100 bash[76202]: cluster 2024-04-07T16:26:49.924023+0000 mon.a (mon.0) 1064 : cluster [DBG] osdmap e654: 8 total, 8 up, 8 in 2024-04-07T16:26:51.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:51 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:26:51.356 INFO:journalctl@ceph.osd.7.smithi144.stdout:Apr 07 16:26:51 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:26:51.356 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:26:51 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:26:51.356 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:26:51 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:26:51.357 INFO:journalctl@ceph.osd.4.smithi144.stdout:Apr 07 16:26:51 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:26:51.357 INFO:journalctl@ceph.osd.5.smithi144.stdout:Apr 07 16:26:51 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:26:51.357 INFO:journalctl@ceph.osd.6.smithi144.stdout:Apr 07 16:26:51 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:26:51.358 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:26:51 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:26:51.358 INFO:journalctl@ceph.mds.b.smithi144.stdout:Apr 07 16:26:51 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:26:51.358 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 16:26:51 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:26:51.358 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 16:26:51 smithi144 systemd[1]: Started Ceph node-exporter.b for 0702e3a8-f4f8-11ee-b649-cb9ed24678a4. 2024-04-07T16:26:51.903 INFO:tasks.rados.rados.0.smithi100.stdout:3146: done (6 left) 2024-04-07T16:26:51.904 INFO:tasks.rados.rados.0.smithi100.stdout:3147: done (5 left) 2024-04-07T16:26:51.904 INFO:tasks.rados.rados.0.smithi100.stdout:3148: done (4 left) 2024-04-07T16:26:51.904 INFO:tasks.rados.rados.0.smithi100.stdout:3149: done (3 left) 2024-04-07T16:26:51.904 INFO:tasks.rados.rados.0.smithi100.stdout:3150: done (2 left) 2024-04-07T16:26:51.904 INFO:tasks.rados.rados.0.smithi100.stdout:3151: done (1 left) 2024-04-07T16:26:51.904 INFO:tasks.rados.rados.0.smithi100.stdout:3152: done (0 left) 2024-04-07T16:26:51.904 INFO:tasks.rados.rados.0.smithi100.stdout:3153: rmattr oid 44 current snap is 291 2024-04-07T16:26:51.904 INFO:tasks.rados.rados.0.smithi100.stdout:3153: done (0 left) 2024-04-07T16:26:51.904 INFO:tasks.rados.rados.0.smithi100.stdout:3154: rmattr oid 50 current snap is 291 2024-04-07T16:26:51.904 INFO:tasks.rados.rados.0.smithi100.stdout:3155: read oid 37 snap -1 2024-04-07T16:26:51.904 INFO:tasks.rados.rados.0.smithi100.stdout:3155: expect (ObjNum 939 snap 281 seq_num 939) 2024-04-07T16:26:51.904 INFO:tasks.rados.rados.0.smithi100.stdout:3156: rmattr oid 5 current snap is 291 2024-04-07T16:26:51.905 INFO:tasks.rados.rados.0.smithi100.stdout:3157: snap_create 2024-04-07T16:26:51.911 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 50 v 1522 (ObjNum 960 snap 289 seq_num 960) dirty exists 2024-04-07T16:26:51.918 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 5 v 1463 (ObjNum 33 snap 0 seq_num 2702284272) dirty exists 2024-04-07T16:26:51.923 INFO:tasks.rados.rados.0.smithi100.stdout:3155: expect (ObjNum 939 snap 281 seq_num 939) 2024-04-07T16:26:52.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:51 smithi100 bash[74942]: cluster 2024-04-07T16:26:50.776596+0000 mgr.y (mgr.25175) 325 : cluster [DBG] pgmap v574: 121 pgs: 121 active+clean; 93 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 4.4 MiB/s rd, 2.5 MiB/s wr, 19 op/s 2024-04-07T16:26:52.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:51 smithi100 bash[74942]: cluster 2024-04-07T16:26:50.953297+0000 mon.a (mon.0) 1065 : cluster [DBG] osdmap e655: 8 total, 8 up, 8 in 2024-04-07T16:26:52.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:51 smithi100 bash[74942]: audit 2024-04-07T16:26:51.283481+0000 mon.a (mon.0) 1066 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:26:52.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:51 smithi100 bash[74942]: audit 2024-04-07T16:26:51.295181+0000 mon.a (mon.0) 1067 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:26:52.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:51 smithi100 bash[74942]: audit 2024-04-07T16:26:51.317678+0000 mon.a (mon.0) 1068 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:26:52.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:51 smithi100 bash[74942]: cluster 2024-04-07T16:26:51.918293+0000 mon.a (mon.0) 1069 : cluster [DBG] osdmap e656: 8 total, 8 up, 8 in 2024-04-07T16:26:52.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:51 smithi100 bash[76202]: cluster 2024-04-07T16:26:50.776596+0000 mgr.y (mgr.25175) 325 : cluster [DBG] pgmap v574: 121 pgs: 121 active+clean; 93 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 4.4 MiB/s rd, 2.5 MiB/s wr, 19 op/s 2024-04-07T16:26:52.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:51 smithi100 bash[76202]: cluster 2024-04-07T16:26:50.953297+0000 mon.a (mon.0) 1065 : cluster [DBG] osdmap e655: 8 total, 8 up, 8 in 2024-04-07T16:26:52.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:51 smithi100 bash[76202]: audit 2024-04-07T16:26:51.283481+0000 mon.a (mon.0) 1066 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:26:52.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:51 smithi100 bash[76202]: audit 2024-04-07T16:26:51.295181+0000 mon.a (mon.0) 1067 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:26:52.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:51 smithi100 bash[76202]: audit 2024-04-07T16:26:51.317678+0000 mon.a (mon.0) 1068 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:26:52.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:51 smithi100 bash[76202]: cluster 2024-04-07T16:26:51.918293+0000 mon.a (mon.0) 1069 : cluster [DBG] osdmap e656: 8 total, 8 up, 8 in 2024-04-07T16:26:52.192 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:51 smithi144 bash[66654]: cluster 2024-04-07T16:26:50.776596+0000 mgr.y (mgr.25175) 325 : cluster [DBG] pgmap v574: 121 pgs: 121 active+clean; 93 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 4.4 MiB/s rd, 2.5 MiB/s wr, 19 op/s 2024-04-07T16:26:52.192 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:51 smithi144 bash[66654]: cluster 2024-04-07T16:26:50.953297+0000 mon.a (mon.0) 1065 : cluster [DBG] osdmap e655: 8 total, 8 up, 8 in 2024-04-07T16:26:52.192 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:51 smithi144 bash[66654]: audit 2024-04-07T16:26:51.283481+0000 mon.a (mon.0) 1066 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:26:52.192 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:51 smithi144 bash[66654]: audit 2024-04-07T16:26:51.295181+0000 mon.a (mon.0) 1067 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:26:52.192 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:51 smithi144 bash[66654]: audit 2024-04-07T16:26:51.317678+0000 mon.a (mon.0) 1068 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:26:52.192 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:51 smithi144 bash[66654]: cluster 2024-04-07T16:26:51.918293+0000 mon.a (mon.0) 1069 : cluster [DBG] osdmap e656: 8 total, 8 up, 8 in 2024-04-07T16:26:52.605 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:26:52 smithi144 bash[59794]: ts=2024-04-07T16:26:52.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:26:52.606 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 16:26:52 smithi144 bash[91657]: ts=2024-04-07T16:26:52.202Z caller=node_exporter.go:180 level=info msg="Starting node_exporter" version="(version=1.5.0, branch=HEAD, revision=1b48970ffcf5630534fb00bb0687d73c66d1c959)" 2024-04-07T16:26:52.606 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 16:26:52 smithi144 bash[91657]: ts=2024-04-07T16:26:52.202Z caller=node_exporter.go:181 level=info msg="Build context" build_context="(go=go1.19.3, user=root@6e7732a7b81b, date=20221129-18:59:09)" 2024-04-07T16:26:52.606 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 16:26:52 smithi144 bash[91657]: ts=2024-04-07T16:26:52.203Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+|var/lib/containers/storage/.+)($|/) 2024-04-07T16:26:52.606 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 16:26:52 smithi144 bash[91657]: ts=2024-04-07T16:26:52.203Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2024-04-07T16:26:52.606 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 16:26:52 smithi144 bash[91657]: ts=2024-04-07T16:26:52.203Z caller=diskstats_common.go:111 level=info collector=diskstats msg="Parsed flag --collector.diskstats.device-exclude" flag=^(ram|loop|fd|(h|s|v|xv)d[a-z]|nvme\d+n\d+p)\d+$ 2024-04-07T16:26:52.606 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 16:26:52 smithi144 bash[91657]: ts=2024-04-07T16:26:52.203Z caller=diskstats_linux.go:264 level=error collector=diskstats msg="Failed to open directory, disabling udev device properties" path=/run/udev/data 2024-04-07T16:26:52.606 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 16:26:52 smithi144 bash[91657]: ts=2024-04-07T16:26:52.204Z caller=node_exporter.go:110 level=info msg="Enabled collectors" 2024-04-07T16:26:52.606 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 16:26:52 smithi144 bash[91657]: ts=2024-04-07T16:26:52.204Z caller=node_exporter.go:117 level=info collector=arp 2024-04-07T16:26:52.607 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 16:26:52 smithi144 bash[91657]: ts=2024-04-07T16:26:52.204Z caller=node_exporter.go:117 level=info collector=bcache 2024-04-07T16:26:52.607 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 16:26:52 smithi144 bash[91657]: ts=2024-04-07T16:26:52.204Z caller=node_exporter.go:117 level=info collector=bonding 2024-04-07T16:26:52.607 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 16:26:52 smithi144 bash[91657]: ts=2024-04-07T16:26:52.204Z caller=node_exporter.go:117 level=info collector=btrfs 2024-04-07T16:26:52.607 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 16:26:52 smithi144 bash[91657]: ts=2024-04-07T16:26:52.204Z caller=node_exporter.go:117 level=info collector=conntrack 2024-04-07T16:26:52.607 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 16:26:52 smithi144 bash[91657]: ts=2024-04-07T16:26:52.204Z caller=node_exporter.go:117 level=info collector=cpu 2024-04-07T16:26:52.607 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 16:26:52 smithi144 bash[91657]: ts=2024-04-07T16:26:52.204Z caller=node_exporter.go:117 level=info collector=cpufreq 2024-04-07T16:26:52.607 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 16:26:52 smithi144 bash[91657]: ts=2024-04-07T16:26:52.204Z caller=node_exporter.go:117 level=info collector=diskstats 2024-04-07T16:26:52.607 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 16:26:52 smithi144 bash[91657]: ts=2024-04-07T16:26:52.204Z caller=node_exporter.go:117 level=info collector=dmi 2024-04-07T16:26:52.607 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 16:26:52 smithi144 bash[91657]: ts=2024-04-07T16:26:52.204Z caller=node_exporter.go:117 level=info collector=edac 2024-04-07T16:26:52.607 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 16:26:52 smithi144 bash[91657]: ts=2024-04-07T16:26:52.204Z caller=node_exporter.go:117 level=info collector=entropy 2024-04-07T16:26:52.607 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 16:26:52 smithi144 bash[91657]: ts=2024-04-07T16:26:52.204Z caller=node_exporter.go:117 level=info collector=fibrechannel 2024-04-07T16:26:52.607 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 16:26:52 smithi144 bash[91657]: ts=2024-04-07T16:26:52.204Z caller=node_exporter.go:117 level=info collector=filefd 2024-04-07T16:26:52.607 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 16:26:52 smithi144 bash[91657]: ts=2024-04-07T16:26:52.204Z caller=node_exporter.go:117 level=info collector=filesystem 2024-04-07T16:26:52.607 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 16:26:52 smithi144 bash[91657]: ts=2024-04-07T16:26:52.204Z caller=node_exporter.go:117 level=info collector=hwmon 2024-04-07T16:26:52.607 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 16:26:52 smithi144 bash[91657]: ts=2024-04-07T16:26:52.204Z caller=node_exporter.go:117 level=info collector=infiniband 2024-04-07T16:26:52.608 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 16:26:52 smithi144 bash[91657]: ts=2024-04-07T16:26:52.204Z caller=node_exporter.go:117 level=info collector=ipvs 2024-04-07T16:26:52.608 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 16:26:52 smithi144 bash[91657]: ts=2024-04-07T16:26:52.204Z caller=node_exporter.go:117 level=info collector=loadavg 2024-04-07T16:26:52.608 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 16:26:52 smithi144 bash[91657]: ts=2024-04-07T16:26:52.204Z caller=node_exporter.go:117 level=info collector=mdadm 2024-04-07T16:26:52.608 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 16:26:52 smithi144 bash[91657]: ts=2024-04-07T16:26:52.204Z caller=node_exporter.go:117 level=info collector=meminfo 2024-04-07T16:26:52.608 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 16:26:52 smithi144 bash[91657]: ts=2024-04-07T16:26:52.204Z caller=node_exporter.go:117 level=info collector=netclass 2024-04-07T16:26:52.608 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 16:26:52 smithi144 bash[91657]: ts=2024-04-07T16:26:52.204Z caller=node_exporter.go:117 level=info collector=netdev 2024-04-07T16:26:52.608 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 16:26:52 smithi144 bash[91657]: ts=2024-04-07T16:26:52.204Z caller=node_exporter.go:117 level=info collector=netstat 2024-04-07T16:26:52.608 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 16:26:52 smithi144 bash[91657]: ts=2024-04-07T16:26:52.204Z caller=node_exporter.go:117 level=info collector=nfs 2024-04-07T16:26:52.608 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 16:26:52 smithi144 bash[91657]: ts=2024-04-07T16:26:52.204Z caller=node_exporter.go:117 level=info collector=nfsd 2024-04-07T16:26:52.608 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 16:26:52 smithi144 bash[91657]: ts=2024-04-07T16:26:52.204Z caller=node_exporter.go:117 level=info collector=nvme 2024-04-07T16:26:52.608 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 16:26:52 smithi144 bash[91657]: ts=2024-04-07T16:26:52.204Z caller=node_exporter.go:117 level=info collector=os 2024-04-07T16:26:52.609 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 16:26:52 smithi144 bash[91657]: ts=2024-04-07T16:26:52.204Z caller=node_exporter.go:117 level=info collector=powersupplyclass 2024-04-07T16:26:52.609 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 16:26:52 smithi144 bash[91657]: ts=2024-04-07T16:26:52.204Z caller=node_exporter.go:117 level=info collector=pressure 2024-04-07T16:26:52.609 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 16:26:52 smithi144 bash[91657]: ts=2024-04-07T16:26:52.204Z caller=node_exporter.go:117 level=info collector=rapl 2024-04-07T16:26:52.609 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 16:26:52 smithi144 bash[91657]: ts=2024-04-07T16:26:52.204Z caller=node_exporter.go:117 level=info collector=schedstat 2024-04-07T16:26:52.609 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 16:26:52 smithi144 bash[91657]: ts=2024-04-07T16:26:52.204Z caller=node_exporter.go:117 level=info collector=selinux 2024-04-07T16:26:52.609 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 16:26:52 smithi144 bash[91657]: ts=2024-04-07T16:26:52.204Z caller=node_exporter.go:117 level=info collector=sockstat 2024-04-07T16:26:52.609 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 16:26:52 smithi144 bash[91657]: ts=2024-04-07T16:26:52.204Z caller=node_exporter.go:117 level=info collector=softnet 2024-04-07T16:26:52.609 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 16:26:52 smithi144 bash[91657]: ts=2024-04-07T16:26:52.204Z caller=node_exporter.go:117 level=info collector=stat 2024-04-07T16:26:52.609 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 16:26:52 smithi144 bash[91657]: ts=2024-04-07T16:26:52.204Z caller=node_exporter.go:117 level=info collector=tapestats 2024-04-07T16:26:52.609 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 16:26:52 smithi144 bash[91657]: ts=2024-04-07T16:26:52.204Z caller=node_exporter.go:117 level=info collector=textfile 2024-04-07T16:26:52.609 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 16:26:52 smithi144 bash[91657]: ts=2024-04-07T16:26:52.204Z caller=node_exporter.go:117 level=info collector=thermal_zone 2024-04-07T16:26:52.609 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 16:26:52 smithi144 bash[91657]: ts=2024-04-07T16:26:52.204Z caller=node_exporter.go:117 level=info collector=time 2024-04-07T16:26:52.609 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 16:26:52 smithi144 bash[91657]: ts=2024-04-07T16:26:52.204Z caller=node_exporter.go:117 level=info collector=udp_queues 2024-04-07T16:26:52.609 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 16:26:52 smithi144 bash[91657]: ts=2024-04-07T16:26:52.204Z caller=node_exporter.go:117 level=info collector=uname 2024-04-07T16:26:52.609 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 16:26:52 smithi144 bash[91657]: ts=2024-04-07T16:26:52.204Z caller=node_exporter.go:117 level=info collector=vmstat 2024-04-07T16:26:52.610 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 16:26:52 smithi144 bash[91657]: ts=2024-04-07T16:26:52.204Z caller=node_exporter.go:117 level=info collector=xfs 2024-04-07T16:26:52.610 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 16:26:52 smithi144 bash[91657]: ts=2024-04-07T16:26:52.204Z caller=node_exporter.go:117 level=info collector=zfs 2024-04-07T16:26:52.610 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 16:26:52 smithi144 bash[91657]: ts=2024-04-07T16:26:52.205Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9100 2024-04-07T16:26:52.610 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 16:26:52 smithi144 bash[91657]: ts=2024-04-07T16:26:52.205Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9100 2024-04-07T16:26:52.904 INFO:tasks.rados.rados.0.smithi100.stdout:3154: done (3 left) 2024-04-07T16:26:52.904 INFO:tasks.rados.rados.0.smithi100.stdout:3155: done (2 left) 2024-04-07T16:26:52.904 INFO:tasks.rados.rados.0.smithi100.stdout:3156: done (1 left) 2024-04-07T16:26:52.904 INFO:tasks.rados.rados.0.smithi100.stdout:3157: done (0 left) 2024-04-07T16:26:52.904 INFO:tasks.rados.rados.0.smithi100.stdout:3158: snap_remove snap 290 2024-04-07T16:26:53.910 INFO:tasks.rados.rados.0.smithi100.stdout:3158: done (0 left) 2024-04-07T16:26:53.911 INFO:tasks.rados.rados.0.smithi100.stdout:3159: read oid 1 snap -1 2024-04-07T16:26:53.911 INFO:tasks.rados.rados.0.smithi100.stdout:3159: expect (ObjNum 913 snap 274 seq_num 913) 2024-04-07T16:26:53.911 INFO:tasks.rados.rados.0.smithi100.stdout:3160: copy_from oid 12 from oid 10 current snap is 292 2024-04-07T16:26:53.911 INFO:tasks.rados.rados.0.smithi100.stdout:3161: read oid 45 snap -1 2024-04-07T16:26:53.911 INFO:tasks.rados.rados.0.smithi100.stdout:3161: expect (ObjNum 933 snap 280 seq_num 933) 2024-04-07T16:26:53.911 INFO:tasks.rados.rados.0.smithi100.stdout:3162: copy_from oid 28 from oid 16 current snap is 292 2024-04-07T16:26:53.911 INFO:tasks.rados.rados.0.smithi100.stdout:3163: read oid 2 snap 289 2024-04-07T16:26:53.911 INFO:tasks.rados.rados.0.smithi100.stdout:3163: expect deleted 2024-04-07T16:26:53.911 INFO:tasks.rados.rados.0.smithi100.stdout:3164: snap_create 2024-04-07T16:26:53.928 INFO:tasks.rados.rados.0.smithi100.stdout:3161: expect (ObjNum 933 snap 280 seq_num 933) 2024-04-07T16:26:53.973 INFO:tasks.rados.rados.0.smithi100.stdout:3159: expect (ObjNum 913 snap 274 seq_num 913) 2024-04-07T16:26:54.139 INFO:tasks.rados.rados.0.smithi100.stdout:3162: finishing copy_from racing read to smithi10066867-28 2024-04-07T16:26:54.140 INFO:tasks.rados.rados.0.smithi100.stdout:3162: finishing copy_from to smithi10066867-28 2024-04-07T16:26:54.140 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 28 v 1198 (ObjNum 794 snap 251 seq_num 794) dirty exists 2024-04-07T16:26:54.140 INFO:tasks.rados.rados.0.smithi100.stdout:3160: finishing copy_from racing read to smithi10066867-12 2024-04-07T16:26:54.140 INFO:tasks.rados.rados.0.smithi100.stdout:3160: finishing copy_from to smithi10066867-12 2024-04-07T16:26:54.140 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 12 v 1452 (ObjNum 945 snap 282 seq_num 945) dirty exists 2024-04-07T16:26:54.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:53 smithi100 bash[74942]: cluster 2024-04-07T16:26:52.777731+0000 mgr.y (mgr.25175) 326 : cluster [DBG] pgmap v577: 121 pgs: 121 active+clean; 103 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 1.6 MiB/s rd, 3.6 MiB/s wr, 12 op/s 2024-04-07T16:26:54.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:53 smithi100 bash[74942]: cluster 2024-04-07T16:26:52.920358+0000 mon.a (mon.0) 1070 : cluster [DBG] osdmap e657: 8 total, 8 up, 8 in 2024-04-07T16:26:54.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:53 smithi100 bash[76202]: cluster 2024-04-07T16:26:52.777731+0000 mgr.y (mgr.25175) 326 : cluster [DBG] pgmap v577: 121 pgs: 121 active+clean; 103 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 1.6 MiB/s rd, 3.6 MiB/s wr, 12 op/s 2024-04-07T16:26:54.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:53 smithi100 bash[76202]: cluster 2024-04-07T16:26:52.920358+0000 mon.a (mon.0) 1070 : cluster [DBG] osdmap e657: 8 total, 8 up, 8 in 2024-04-07T16:26:54.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:53 smithi144 bash[66654]: cluster 2024-04-07T16:26:52.777731+0000 mgr.y (mgr.25175) 326 : cluster [DBG] pgmap v577: 121 pgs: 121 active+clean; 103 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 1.6 MiB/s rd, 3.6 MiB/s wr, 12 op/s 2024-04-07T16:26:54.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:53 smithi144 bash[66654]: cluster 2024-04-07T16:26:52.920358+0000 mon.a (mon.0) 1070 : cluster [DBG] osdmap e657: 8 total, 8 up, 8 in 2024-04-07T16:26:54.912 INFO:tasks.rados.rados.0.smithi100.stdout:3159: done (5 left) 2024-04-07T16:26:54.912 INFO:tasks.rados.rados.0.smithi100.stdout:3160: done (4 left) 2024-04-07T16:26:54.912 INFO:tasks.rados.rados.0.smithi100.stdout:3161: done (3 left) 2024-04-07T16:26:54.912 INFO:tasks.rados.rados.0.smithi100.stdout:3162: done (2 left) 2024-04-07T16:26:54.912 INFO:tasks.rados.rados.0.smithi100.stdout:3163: done (1 left) 2024-04-07T16:26:54.912 INFO:tasks.rados.rados.0.smithi100.stdout:3164: done (0 left) 2024-04-07T16:26:54.912 INFO:tasks.rados.rados.0.smithi100.stdout:3165: setattr oid 18 current snap is 293 2024-04-07T16:26:54.917 INFO:tasks.rados.rados.0.smithi100.stdout:3166: copy_from oid 28 from oid 41 current snap is 293 2024-04-07T16:26:54.917 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 3 current snap is 293 2024-04-07T16:26:54.917 INFO:tasks.rados.rados.0.smithi100.stdout:3167: seq_num 964 ranges {4653056=548864,5201920=761856,5963776=442368} 2024-04-07T16:26:54.958 INFO:tasks.rados.rados.0.smithi100.stdout:3167: writing smithi10066867-3 from 4653056 to 5201920 tid 1 2024-04-07T16:26:54.962 INFO:tasks.rados.rados.0.smithi100.stdout:3167: writing smithi10066867-3 from 5201920 to 5963776 tid 2 2024-04-07T16:26:54.965 INFO:tasks.rados.rados.0.smithi100.stdout:3167: writing smithi10066867-3 from 5963776 to 6406144 tid 3 2024-04-07T16:26:54.965 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 5 current snap is 293 2024-04-07T16:26:54.965 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 18 v 1697 (ObjNum 900 snap 270 seq_num 900) dirty exists 2024-04-07T16:26:54.965 INFO:tasks.rados.rados.0.smithi100.stdout:3166: finishing copy_from racing read to smithi10066867-28 2024-04-07T16:26:54.966 INFO:tasks.rados.rados.0.smithi100.stdout:3166: finishing copy_from to smithi10066867-28 2024-04-07T16:26:54.966 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 28 v 1461 (ObjNum 161 snap 0 seq_num 1) dirty exists 2024-04-07T16:26:54.966 INFO:tasks.rados.rados.0.smithi100.stdout:3168: seq_num 965 ranges {0=565248,565248=647168,1212416=548864,1761280=8192} 2024-04-07T16:26:54.969 INFO:tasks.rados.rados.0.smithi100.stdout:3168: writing smithi10066867-5 from 0 to 565248 tid 1 2024-04-07T16:26:54.973 INFO:tasks.rados.rados.0.smithi100.stdout:3168: writing smithi10066867-5 from 565248 to 1212416 tid 2 2024-04-07T16:26:54.976 INFO:tasks.rados.rados.0.smithi100.stdout:3168: writing smithi10066867-5 from 1212416 to 1761280 tid 3 2024-04-07T16:26:54.976 INFO:tasks.rados.rados.0.smithi100.stdout:3168: writing smithi10066867-5 from 1761280 to 1769472 tid 4 2024-04-07T16:26:54.977 INFO:tasks.rados.rados.0.smithi100.stdout:3165: done (3 left) 2024-04-07T16:26:54.977 INFO:tasks.rados.rados.0.smithi100.stdout:3166: done (2 left) 2024-04-07T16:26:54.977 INFO:tasks.rados.rados.0.smithi100.stdout:3169: snap_remove snap 291 2024-04-07T16:26:55.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:54 smithi100 bash[74942]: cluster 2024-04-07T16:26:53.925539+0000 mon.a (mon.0) 1071 : cluster [DBG] osdmap e658: 8 total, 8 up, 8 in 2024-04-07T16:26:55.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:54 smithi100 bash[74942]: cluster 2024-04-07T16:26:54.926373+0000 mon.a (mon.0) 1072 : cluster [DBG] osdmap e659: 8 total, 8 up, 8 in 2024-04-07T16:26:55.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:54 smithi100 bash[76202]: cluster 2024-04-07T16:26:53.925539+0000 mon.a (mon.0) 1071 : cluster [DBG] osdmap e658: 8 total, 8 up, 8 in 2024-04-07T16:26:55.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:54 smithi100 bash[76202]: cluster 2024-04-07T16:26:54.926373+0000 mon.a (mon.0) 1072 : cluster [DBG] osdmap e659: 8 total, 8 up, 8 in 2024-04-07T16:26:55.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:54 smithi144 bash[66654]: cluster 2024-04-07T16:26:53.925539+0000 mon.a (mon.0) 1071 : cluster [DBG] osdmap e658: 8 total, 8 up, 8 in 2024-04-07T16:26:55.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:54 smithi144 bash[66654]: cluster 2024-04-07T16:26:54.926373+0000 mon.a (mon.0) 1072 : cluster [DBG] osdmap e659: 8 total, 8 up, 8 in 2024-04-07T16:26:55.942 INFO:tasks.rados.rados.0.smithi100.stdout:3169: done (2 left) 2024-04-07T16:26:55.942 INFO:tasks.rados.rados.0.smithi100.stdout:3170: rollback oid 28 current snap is 293 2024-04-07T16:26:55.943 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 28 to 292 2024-04-07T16:26:55.943 INFO:tasks.rados.rados.0.smithi100.stdout:3167: finishing write tid 1 to smithi10066867-3 2024-04-07T16:26:55.943 INFO:tasks.rados.rados.0.smithi100.stdout:3167: finishing write tid 2 to smithi10066867-3 2024-04-07T16:26:55.943 INFO:tasks.rados.rados.0.smithi100.stdout:3167: finishing write tid 3 to smithi10066867-3 2024-04-07T16:26:55.943 INFO:tasks.rados.rados.0.smithi100.stdout:3167: finishing write tid 4 to smithi10066867-3 2024-04-07T16:26:55.943 INFO:tasks.rados.rados.0.smithi100.stdout:3168: finishing write tid 1 to smithi10066867-5 2024-04-07T16:26:55.943 INFO:tasks.rados.rados.0.smithi100.stdout:3167: finishing write tid 5 to smithi10066867-3 2024-04-07T16:26:55.943 INFO:tasks.rados.rados.0.smithi100.stdout:3167: oid 3 updating version 0 to 1144 2024-04-07T16:26:55.943 INFO:tasks.rados.rados.0.smithi100.stdout:3167: oid 3 version 1144 is already newer than 1143 2024-04-07T16:26:55.943 INFO:tasks.rados.rados.0.smithi100.stdout:3167: oid 3 updating version 1144 to 1146 2024-04-07T16:26:55.943 INFO:tasks.rados.rados.0.smithi100.stdout:3167: oid 3 version 1146 is already newer than 1145 2024-04-07T16:26:55.943 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 3 v 1146 (ObjNum 964 snap 293 seq_num 964) dirty exists 2024-04-07T16:26:55.943 INFO:tasks.rados.rados.0.smithi100.stdout:3167: left oid 3 (ObjNum 964 snap 293 seq_num 964) 2024-04-07T16:26:55.943 INFO:tasks.rados.rados.0.smithi100.stdout:3168: finishing write tid 2 to smithi10066867-5 2024-04-07T16:26:55.943 INFO:tasks.rados.rados.0.smithi100.stdout:3167: done (2 left) 2024-04-07T16:26:55.944 INFO:tasks.rados.rados.0.smithi100.stdout:3171: setattr oid 43 current snap is 293 2024-04-07T16:26:55.944 INFO:tasks.rados.rados.0.smithi100.stdout:3168: finishing write tid 3 to smithi10066867-5 2024-04-07T16:26:55.944 INFO:tasks.rados.rados.0.smithi100.stdout:3168: finishing write tid 4 to smithi10066867-5 2024-04-07T16:26:55.944 INFO:tasks.rados.rados.0.smithi100.stdout:3168: finishing write tid 5 to smithi10066867-5 2024-04-07T16:26:55.944 INFO:tasks.rados.rados.0.smithi100.stdout:3168: finishing write tid 6 to smithi10066867-5 2024-04-07T16:26:55.944 INFO:tasks.rados.rados.0.smithi100.stdout:3168: oid 5 updating version 0 to 1471 2024-04-07T16:26:55.944 INFO:tasks.rados.rados.0.smithi100.stdout:3168: oid 5 updating version 1471 to 1473 2024-04-07T16:26:55.944 INFO:tasks.rados.rados.0.smithi100.stdout:3168: oid 5 version 1473 is already newer than 1470 2024-04-07T16:26:55.944 INFO:tasks.rados.rados.0.smithi100.stdout:3168: oid 5 version 1473 is already newer than 1469 2024-04-07T16:26:55.944 INFO:tasks.rados.rados.0.smithi100.stdout:3168: oid 5 version 1473 is already newer than 1472 2024-04-07T16:26:55.944 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 5 v 1473 (ObjNum 965 snap 293 seq_num 965) dirty exists 2024-04-07T16:26:55.944 INFO:tasks.rados.rados.0.smithi100.stdout:3168: left oid 5 (ObjNum 965 snap 293 seq_num 965) 2024-04-07T16:26:55.945 INFO:tasks.rados.rados.0.smithi100.stdout:3168: done (2 left) 2024-04-07T16:26:55.945 INFO:tasks.rados.rados.0.smithi100.stdout:3172: copy_from oid 45 from oid 34 current snap is 293 2024-04-07T16:26:55.945 INFO:tasks.rados.rados.0.smithi100.stdout:3173: read oid 15 snap -1 2024-04-07T16:26:55.945 INFO:tasks.rados.rados.0.smithi100.stdout:3173: expect (ObjNum 0 snap 0 seq_num 0) 2024-04-07T16:26:55.945 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 13 current snap is 293 2024-04-07T16:26:55.945 INFO:tasks.rados.rados.0.smithi100.stdout:3174: seq_num 966 ranges {0=581632,581632=581632,1163264=622592} 2024-04-07T16:26:55.949 INFO:tasks.rados.rados.0.smithi100.stdout:3174: writing smithi10066867-13 from 0 to 581632 tid 1 2024-04-07T16:26:55.955 INFO:tasks.rados.rados.0.smithi100.stdout:3174: writing smithi10066867-13 from 581632 to 1163264 tid 2 2024-04-07T16:26:55.961 INFO:tasks.rados.rados.0.smithi100.stdout:3174: writing smithi10066867-13 from 1163264 to 1785856 tid 3 2024-04-07T16:26:55.961 INFO:tasks.rados.rados.0.smithi100.stdout:3175: rollback oid 18 current snap is 293 2024-04-07T16:26:55.961 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 18 to 289 2024-04-07T16:26:55.961 INFO:tasks.rados.rados.0.smithi100.stdout:3173: expect (ObjNum 0 snap 0 seq_num 0) 2024-04-07T16:26:55.966 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 43 v 1167 (ObjNum 49 snap 0 seq_num 2) dirty exists 2024-04-07T16:26:55.966 INFO:tasks.rados.rados.0.smithi100.stdout:3171: done (5 left) 2024-04-07T16:26:55.966 INFO:tasks.rados.rados.0.smithi100.stdout:3173: done (4 left) 2024-04-07T16:26:55.966 INFO:tasks.rados.rados.0.smithi100.stdout:3176: read oid 26 snap -1 2024-04-07T16:26:55.966 INFO:tasks.rados.rados.0.smithi100.stdout:3176: expect (ObjNum 897 snap 269 seq_num 897) 2024-04-07T16:26:55.966 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 6 current snap is 293 2024-04-07T16:26:55.966 INFO:tasks.rados.rados.0.smithi100.stdout:3177: seq_num 967 ranges {2973696=598016,3571712=466944,4038656=516096,4554752=221184} 2024-04-07T16:26:56.001 INFO:tasks.rados.rados.0.smithi100.stdout:3177: writing smithi10066867-6 from 2973696 to 3571712 tid 1 2024-04-07T16:26:56.004 INFO:tasks.rados.rados.0.smithi100.stdout:3177: writing smithi10066867-6 from 3571712 to 4038656 tid 2 2024-04-07T16:26:56.008 INFO:tasks.rados.rados.0.smithi100.stdout:3177: writing smithi10066867-6 from 4038656 to 4554752 tid 3 2024-04-07T16:26:56.011 INFO:tasks.rados.rados.0.smithi100.stdout:3177: writing smithi10066867-6 from 4554752 to 4775936 tid 4 2024-04-07T16:26:56.011 INFO:tasks.rados.rados.0.smithi100.stdout:3178: copy_from oid 31 from oid 33 current snap is 293 2024-04-07T16:26:56.011 INFO:tasks.rados.rados.0.smithi100.stdout:3175: finishing rollback tid 0 to smithi10066867-18 2024-04-07T16:26:56.011 INFO:tasks.rados.rados.0.smithi100.stdout:3175: finishing rollback tid 1 to smithi10066867-18 2024-04-07T16:26:56.011 INFO:tasks.rados.rados.0.smithi100.stdout:3175: finishing rollback tid 2 to smithi10066867-18 2024-04-07T16:26:56.011 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 18 v 1702 (ObjNum 900 snap 270 seq_num 900) dirty exists 2024-04-07T16:26:56.011 INFO:tasks.rados.rados.0.smithi100.stdout:3170: finishing rollback tid 0 to smithi10066867-28 2024-04-07T16:26:56.012 INFO:tasks.rados.rados.0.smithi100.stdout:3170: finishing rollback tid 1 to smithi10066867-28 2024-04-07T16:26:56.012 INFO:tasks.rados.rados.0.smithi100.stdout:3175: done (6 left) 2024-04-07T16:26:56.012 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 8 current snap is 293 2024-04-07T16:26:56.012 INFO:tasks.rados.rados.0.smithi100.stdout:3170: finishing rollback tid 2 to smithi10066867-28 2024-04-07T16:26:56.012 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 28 v 1690 (ObjNum 794 snap 251 seq_num 794) dirty exists 2024-04-07T16:26:56.012 INFO:tasks.rados.rados.0.smithi100.stdout:3174: finishing write tid 1 to smithi10066867-13 2024-04-07T16:26:56.012 INFO:tasks.rados.rados.0.smithi100.stdout:3174: finishing write tid 2 to smithi10066867-13 2024-04-07T16:26:56.012 INFO:tasks.rados.rados.0.smithi100.stdout:3179: seq_num 968 ranges {1802240=614400,2416640=745472,3162112=458752} 2024-04-07T16:26:56.028 INFO:tasks.rados.rados.0.smithi100.stdout:3179: writing smithi10066867-8 from 1802240 to 2416640 tid 1 2024-04-07T16:26:56.033 INFO:tasks.rados.rados.0.smithi100.stdout:3179: writing smithi10066867-8 from 2416640 to 3162112 tid 2 2024-04-07T16:26:56.036 INFO:tasks.rados.rados.0.smithi100.stdout:3179: writing smithi10066867-8 from 3162112 to 3620864 tid 3 2024-04-07T16:26:56.037 INFO:tasks.rados.rados.0.smithi100.stdout:3170: done (6 left) 2024-04-07T16:26:56.037 INFO:tasks.rados.rados.0.smithi100.stdout:3180: read oid 1 snap -1 2024-04-07T16:26:56.037 INFO:tasks.rados.rados.0.smithi100.stdout:3180: expect (ObjNum 913 snap 274 seq_num 913) 2024-04-07T16:26:56.037 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 5 current snap is 293 2024-04-07T16:26:56.037 INFO:tasks.rados.rados.0.smithi100.stdout:3174: finishing write tid 3 to smithi10066867-13 2024-04-07T16:26:56.037 INFO:tasks.rados.rados.0.smithi100.stdout:3174: finishing write tid 4 to smithi10066867-13 2024-04-07T16:26:56.037 INFO:tasks.rados.rados.0.smithi100.stdout:3181: seq_num 969 ranges {1769472=638976,2408448=630784,3039232=532480,3571712=40960} 2024-04-07T16:26:56.053 INFO:tasks.rados.rados.0.smithi100.stdout:3181: writing smithi10066867-5 from 1769472 to 2408448 tid 1 2024-04-07T16:26:56.057 INFO:tasks.rados.rados.0.smithi100.stdout:3181: writing smithi10066867-5 from 2408448 to 3039232 tid 2 2024-04-07T16:26:56.061 INFO:tasks.rados.rados.0.smithi100.stdout:3181: writing smithi10066867-5 from 3039232 to 3571712 tid 3 2024-04-07T16:26:56.061 INFO:tasks.rados.rados.0.smithi100.stdout:3181: writing smithi10066867-5 from 3571712 to 3612672 tid 4 2024-04-07T16:26:56.061 INFO:tasks.rados.rados.0.smithi100.stdout:3182: rollback oid 24 current snap is 293 2024-04-07T16:26:56.061 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 24 to 287 2024-04-07T16:26:56.062 INFO:tasks.rados.rados.0.smithi100.stdout:3174: finishing write tid 5 to smithi10066867-13 2024-04-07T16:26:56.062 INFO:tasks.rados.rados.0.smithi100.stdout:3174: oid 13 updating version 0 to 1531 2024-04-07T16:26:56.062 INFO:tasks.rados.rados.0.smithi100.stdout:3174: oid 13 updating version 1531 to 1532 2024-04-07T16:26:56.062 INFO:tasks.rados.rados.0.smithi100.stdout:3174: oid 13 updating version 1532 to 1533 2024-04-07T16:26:56.062 INFO:tasks.rados.rados.0.smithi100.stdout:3174: oid 13 updating version 1533 to 1534 2024-04-07T16:26:56.062 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 13 v 1534 (ObjNum 966 snap 293 seq_num 966) dirty exists 2024-04-07T16:26:56.062 INFO:tasks.rados.rados.0.smithi100.stdout:3174: left oid 13 (ObjNum 966 snap 293 seq_num 966) 2024-04-07T16:26:56.062 INFO:tasks.rados.rados.0.smithi100.stdout:3172: finishing copy_from racing read to smithi10066867-45 2024-04-07T16:26:56.062 INFO:tasks.rados.rados.0.smithi100.stdout:3172: finishing copy_from to smithi10066867-45 2024-04-07T16:26:56.062 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 45 v 1530 (ObjNum 962 snap 290 seq_num 962) dirty exists 2024-04-07T16:26:56.062 INFO:tasks.rados.rados.0.smithi100.stdout:3176: expect (ObjNum 897 snap 269 seq_num 897) 2024-04-07T16:26:56.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:55 smithi100 bash[74942]: cluster 2024-04-07T16:26:54.778740+0000 mgr.y (mgr.25175) 327 : cluster [DBG] pgmap v580: 121 pgs: 121 active+clean; 110 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 3.2 MiB/s rd, 4.3 MiB/s wr, 17 op/s 2024-04-07T16:26:56.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:55 smithi100 bash[76202]: cluster 2024-04-07T16:26:54.778740+0000 mgr.y (mgr.25175) 327 : cluster [DBG] pgmap v580: 121 pgs: 121 active+clean; 110 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 3.2 MiB/s rd, 4.3 MiB/s wr, 17 op/s 2024-04-07T16:26:56.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:55 smithi144 bash[66654]: cluster 2024-04-07T16:26:54.778740+0000 mgr.y (mgr.25175) 327 : cluster [DBG] pgmap v580: 121 pgs: 121 active+clean; 110 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 3.2 MiB/s rd, 4.3 MiB/s wr, 17 op/s 2024-04-07T16:26:56.411 INFO:tasks.rados.rados.0.smithi100.stdout:3177: finishing write tid 1 to smithi10066867-6 2024-04-07T16:26:56.411 INFO:tasks.rados.rados.0.smithi100.stdout:3177: finishing write tid 2 to smithi10066867-6 2024-04-07T16:26:56.411 INFO:tasks.rados.rados.0.smithi100.stdout:3172: done (8 left) 2024-04-07T16:26:56.411 INFO:tasks.rados.rados.0.smithi100.stdout:3174: done (7 left) 2024-04-07T16:26:56.411 INFO:tasks.rados.rados.0.smithi100.stdout:3176: done (6 left) 2024-04-07T16:26:56.411 INFO:tasks.rados.rados.0.smithi100.stdout:3183: snap_create 2024-04-07T16:26:56.411 INFO:tasks.rados.rados.0.smithi100.stdout:3177: finishing write tid 3 to smithi10066867-6 2024-04-07T16:26:56.412 INFO:tasks.rados.rados.0.smithi100.stdout:3177: finishing write tid 4 to smithi10066867-6 2024-04-07T16:26:56.412 INFO:tasks.rados.rados.0.smithi100.stdout:3177: finishing write tid 5 to smithi10066867-6 2024-04-07T16:26:56.412 INFO:tasks.rados.rados.0.smithi100.stdout:3178: finishing copy_from racing read to smithi10066867-31 2024-04-07T16:26:56.412 INFO:tasks.rados.rados.0.smithi100.stdout:3177: finishing write tid 6 to smithi10066867-6 2024-04-07T16:26:56.412 INFO:tasks.rados.rados.0.smithi100.stdout:3177: oid 6 updating version 0 to 1628 2024-04-07T16:26:56.412 INFO:tasks.rados.rados.0.smithi100.stdout:3177: oid 6 updating version 1628 to 1630 2024-04-07T16:26:56.412 INFO:tasks.rados.rados.0.smithi100.stdout:3177: oid 6 updating version 1630 to 1631 2024-04-07T16:26:56.412 INFO:tasks.rados.rados.0.smithi100.stdout:3177: oid 6 version 1631 is already newer than 1627 2024-04-07T16:26:56.412 INFO:tasks.rados.rados.0.smithi100.stdout:3177: oid 6 version 1631 is already newer than 1629 2024-04-07T16:26:56.412 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 6 v 1631 (ObjNum 967 snap 293 seq_num 967) dirty exists 2024-04-07T16:26:56.412 INFO:tasks.rados.rados.0.smithi100.stdout:3177: left oid 6 (ObjNum 967 snap 293 seq_num 967) 2024-04-07T16:26:56.412 INFO:tasks.rados.rados.0.smithi100.stdout:3178: finishing copy_from to smithi10066867-31 2024-04-07T16:26:56.412 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 31 v 1637 (ObjNum 753 snap 235 seq_num 753) dirty exists 2024-04-07T16:26:56.412 INFO:tasks.rados.rados.0.smithi100.stdout:3179: finishing write tid 1 to smithi10066867-8 2024-04-07T16:26:56.413 INFO:tasks.rados.rados.0.smithi100.stdout:3179: finishing write tid 2 to smithi10066867-8 2024-04-07T16:26:56.413 INFO:tasks.rados.rados.0.smithi100.stdout:3179: finishing write tid 3 to smithi10066867-8 2024-04-07T16:26:56.413 INFO:tasks.rados.rados.0.smithi100.stdout:3179: finishing write tid 4 to smithi10066867-8 2024-04-07T16:26:56.413 INFO:tasks.rados.rados.0.smithi100.stdout:3179: finishing write tid 5 to smithi10066867-8 2024-04-07T16:26:56.413 INFO:tasks.rados.rados.0.smithi100.stdout:3179: oid 8 updating version 0 to 1538 2024-04-07T16:26:56.413 INFO:tasks.rados.rados.0.smithi100.stdout:3179: oid 8 version 1538 is already newer than 1535 2024-04-07T16:26:56.413 INFO:tasks.rados.rados.0.smithi100.stdout:3179: oid 8 version 1538 is already newer than 1537 2024-04-07T16:26:56.413 INFO:tasks.rados.rados.0.smithi100.stdout:3179: oid 8 version 1538 is already newer than 1536 2024-04-07T16:26:56.413 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 8 v 1538 (ObjNum 968 snap 293 seq_num 968) dirty exists 2024-04-07T16:26:56.413 INFO:tasks.rados.rados.0.smithi100.stdout:3179: left oid 8 (ObjNum 968 snap 293 seq_num 968) 2024-04-07T16:26:56.413 INFO:tasks.rados.rados.0.smithi100.stdout:3180: expect (ObjNum 913 snap 274 seq_num 913) 2024-04-07T16:26:56.576 INFO:tasks.rados.rados.0.smithi100.stdout:3181: finishing write tid 1 to smithi10066867-5 2024-04-07T16:26:56.577 INFO:tasks.rados.rados.0.smithi100.stdout:3182: finishing rollback tid 0 to smithi10066867-24 2024-04-07T16:26:56.577 INFO:tasks.rados.rados.0.smithi100.stdout:3181: finishing write tid 2 to smithi10066867-5 2024-04-07T16:26:56.577 INFO:tasks.rados.rados.0.smithi100.stdout:3182: finishing rollback tid 1 to smithi10066867-24 2024-04-07T16:26:56.577 INFO:tasks.rados.rados.0.smithi100.stdout:3181: finishing write tid 3 to smithi10066867-5 2024-04-07T16:26:56.577 INFO:tasks.rados.rados.0.smithi100.stdout:3181: finishing write tid 4 to smithi10066867-5 2024-04-07T16:26:56.577 INFO:tasks.rados.rados.0.smithi100.stdout:3181: finishing write tid 5 to smithi10066867-5 2024-04-07T16:26:56.577 INFO:tasks.rados.rados.0.smithi100.stdout:3182: finishing rollback tid 2 to smithi10066867-24 2024-04-07T16:26:56.577 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 24 v 1248 (ObjNum 948 snap 285 seq_num 948) dirty exists 2024-04-07T16:26:56.577 INFO:tasks.rados.rados.0.smithi100.stdout:3181: finishing write tid 6 to smithi10066867-5 2024-04-07T16:26:56.577 INFO:tasks.rados.rados.0.smithi100.stdout:3181: oid 5 updating version 0 to 1480 2024-04-07T16:26:56.577 INFO:tasks.rados.rados.0.smithi100.stdout:3181: oid 5 updating version 1480 to 1481 2024-04-07T16:26:56.577 INFO:tasks.rados.rados.0.smithi100.stdout:3181: oid 5 version 1481 is already newer than 1478 2024-04-07T16:26:56.578 INFO:tasks.rados.rados.0.smithi100.stdout:3181: oid 5 updating version 1481 to 1482 2024-04-07T16:26:56.578 INFO:tasks.rados.rados.0.smithi100.stdout:3181: oid 5 version 1482 is already newer than 1479 2024-04-07T16:26:56.578 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 5 v 1482 (ObjNum 969 snap 293 seq_num 969) dirty exists 2024-04-07T16:26:56.578 INFO:tasks.rados.rados.0.smithi100.stdout:3181: left oid 5 (ObjNum 969 snap 293 seq_num 969) 2024-04-07T16:26:56.946 INFO:tasks.rados.rados.0.smithi100.stdout:3177: done (6 left) 2024-04-07T16:26:56.946 INFO:tasks.rados.rados.0.smithi100.stdout:3178: done (5 left) 2024-04-07T16:26:56.946 INFO:tasks.rados.rados.0.smithi100.stdout:3179: done (4 left) 2024-04-07T16:26:56.947 INFO:tasks.rados.rados.0.smithi100.stdout:3180: done (3 left) 2024-04-07T16:26:56.947 INFO:tasks.rados.rados.0.smithi100.stdout:3181: done (2 left) 2024-04-07T16:26:56.947 INFO:tasks.rados.rados.0.smithi100.stdout:3182: done (1 left) 2024-04-07T16:26:56.947 INFO:tasks.rados.rados.0.smithi100.stdout:3183: done (0 left) 2024-04-07T16:26:56.947 INFO:tasks.rados.rados.0.smithi100.stdout:3184: snap_remove snap 289 2024-04-07T16:26:57.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:56 smithi144 bash[66654]: cluster 2024-04-07T16:26:55.956512+0000 mon.a (mon.0) 1073 : cluster [DBG] osdmap e660: 8 total, 8 up, 8 in 2024-04-07T16:26:57.436 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:56 smithi100 bash[74942]: cluster 2024-04-07T16:26:55.956512+0000 mon.a (mon.0) 1073 : cluster [DBG] osdmap e660: 8 total, 8 up, 8 in 2024-04-07T16:26:57.436 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:56 smithi100 bash[76202]: cluster 2024-04-07T16:26:55.956512+0000 mon.a (mon.0) 1073 : cluster [DBG] osdmap e660: 8 total, 8 up, 8 in 2024-04-07T16:26:57.957 INFO:tasks.rados.rados.0.smithi100.stdout:3184: done (0 left) 2024-04-07T16:26:57.957 INFO:tasks.rados.rados.0.smithi100.stdout:3185: delete oid 16 current snap is 294 2024-04-07T16:26:57.969 INFO:tasks.rados.rados.0.smithi100.stdout:3185: done (0 left) 2024-04-07T16:26:57.969 INFO:tasks.rados.rados.0.smithi100.stdout:3186: read oid 10 snap -1 2024-04-07T16:26:57.969 INFO:tasks.rados.rados.0.smithi100.stdout:3186: expect (ObjNum 945 snap 282 seq_num 945) 2024-04-07T16:26:57.969 INFO:tasks.rados.rados.0.smithi100.stdout:3187: rmattr oid 43 current snap is 294 2024-04-07T16:26:57.969 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 12 current snap is 294 2024-04-07T16:26:57.969 INFO:tasks.rados.rados.0.smithi100.stdout:3188: seq_num 971 ranges {1433600=671744,2105344=802816,2908160=401408} 2024-04-07T16:26:57.980 INFO:tasks.rados.rados.0.smithi100.stdout:3188: writing smithi10066867-12 from 1433600 to 2105344 tid 1 2024-04-07T16:26:57.985 INFO:tasks.rados.rados.0.smithi100.stdout:3188: writing smithi10066867-12 from 2105344 to 2908160 tid 2 2024-04-07T16:26:57.988 INFO:tasks.rados.rados.0.smithi100.stdout:3188: writing smithi10066867-12 from 2908160 to 3309568 tid 3 2024-04-07T16:26:57.988 INFO:tasks.rados.rados.0.smithi100.stdout:3189: rmattr oid 38 current snap is 294 2024-04-07T16:26:57.994 INFO:tasks.rados.rados.0.smithi100.stdout:3186: expect (ObjNum 945 snap 282 seq_num 945) 2024-04-07T16:26:58.032 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 43 v 1168 (ObjNum 65 snap 0 seq_num 1398496343) dirty exists 2024-04-07T16:26:58.032 INFO:tasks.rados.rados.0.smithi100.stdout:3188: finishing write tid 1 to smithi10066867-12 2024-04-07T16:26:58.032 INFO:tasks.rados.rados.0.smithi100.stdout:3188: finishing write tid 2 to smithi10066867-12 2024-04-07T16:26:58.033 INFO:tasks.rados.rados.0.smithi100.stdout:3188: finishing write tid 3 to smithi10066867-12 2024-04-07T16:26:58.033 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 38 v 1147 (ObjNum 950 snap 285 seq_num 950) dirty exists 2024-04-07T16:26:58.033 INFO:tasks.rados.rados.0.smithi100.stdout:3188: finishing write tid 4 to smithi10066867-12 2024-04-07T16:26:58.033 INFO:tasks.rados.rados.0.smithi100.stdout:3188: finishing write tid 5 to smithi10066867-12 2024-04-07T16:26:58.033 INFO:tasks.rados.rados.0.smithi100.stdout:3188: oid 12 updating version 0 to 1491 2024-04-07T16:26:58.033 INFO:tasks.rados.rados.0.smithi100.stdout:3188: oid 12 version 1491 is already newer than 1490 2024-04-07T16:26:58.033 INFO:tasks.rados.rados.0.smithi100.stdout:3188: oid 12 updating version 1491 to 1492 2024-04-07T16:26:58.033 INFO:tasks.rados.rados.0.smithi100.stdout:3188: oid 12 updating version 1492 to 1493 2024-04-07T16:26:58.033 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 12 v 1493 (ObjNum 971 snap 294 seq_num 971) dirty exists 2024-04-07T16:26:58.033 INFO:tasks.rados.rados.0.smithi100.stdout:3188: left oid 12 (ObjNum 971 snap 294 seq_num 971) 2024-04-07T16:26:58.033 INFO:tasks.rados.rados.0.smithi100.stdout:3186: done (3 left) 2024-04-07T16:26:58.033 INFO:tasks.rados.rados.0.smithi100.stdout:3187: done (2 left) 2024-04-07T16:26:58.033 INFO:tasks.rados.rados.0.smithi100.stdout:3188: done (1 left) 2024-04-07T16:26:58.033 INFO:tasks.rados.rados.0.smithi100.stdout:3189: done (0 left) 2024-04-07T16:26:58.033 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 26 current snap is 294 2024-04-07T16:26:58.033 INFO:tasks.rados.rados.0.smithi100.stdout:3190: seq_num 972 ranges {4694016=688128,5382144=688128,6070272=516096} 2024-04-07T16:26:58.061 INFO:tasks.rados.rados.0.smithi100.stdout:3190: writing smithi10066867-26 from 4694016 to 5382144 tid 1 2024-04-07T16:26:58.065 INFO:tasks.rados.rados.0.smithi100.stdout:3190: writing smithi10066867-26 from 5382144 to 6070272 tid 2 2024-04-07T16:26:58.068 INFO:tasks.rados.rados.0.smithi100.stdout:3190: writing smithi10066867-26 from 6070272 to 6586368 tid 3 2024-04-07T16:26:58.068 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 17 current snap is 294 2024-04-07T16:26:58.068 INFO:tasks.rados.rados.0.smithi100.stdout:3190: finishing write tid 1 to smithi10066867-26 2024-04-07T16:26:58.068 INFO:tasks.rados.rados.0.smithi100.stdout:3191: seq_num 973 ranges {3006464=704512,3710976=573440,4284416=507904,4792320=122880} 2024-04-07T16:26:58.088 INFO:tasks.rados.rados.0.smithi100.stdout:3191: writing smithi10066867-17 from 3006464 to 3710976 tid 1 2024-04-07T16:26:58.091 INFO:tasks.rados.rados.0.smithi100.stdout:3191: writing smithi10066867-17 from 3710976 to 4284416 tid 2 2024-04-07T16:26:58.094 INFO:tasks.rados.rados.0.smithi100.stdout:3191: writing smithi10066867-17 from 4284416 to 4792320 tid 3 2024-04-07T16:26:58.095 INFO:tasks.rados.rados.0.smithi100.stdout:3191: writing smithi10066867-17 from 4792320 to 4915200 tid 4 2024-04-07T16:26:58.095 INFO:tasks.rados.rados.0.smithi100.stdout:3192: delete oid 45 current snap is 294 2024-04-07T16:26:58.095 INFO:tasks.rados.rados.0.smithi100.stdout:3190: finishing write tid 2 to smithi10066867-26 2024-04-07T16:26:58.095 INFO:tasks.rados.rados.0.smithi100.stdout:3190: finishing write tid 3 to smithi10066867-26 2024-04-07T16:26:58.095 INFO:tasks.rados.rados.0.smithi100.stdout:3190: finishing write tid 4 to smithi10066867-26 2024-04-07T16:26:58.096 INFO:tasks.rados.rados.0.smithi100.stdout:3190: finishing write tid 5 to smithi10066867-26 2024-04-07T16:26:58.096 INFO:tasks.rados.rados.0.smithi100.stdout:3190: oid 26 updating version 0 to 1232 2024-04-07T16:26:58.096 INFO:tasks.rados.rados.0.smithi100.stdout:3190: oid 26 version 1232 is already newer than 1230 2024-04-07T16:26:58.096 INFO:tasks.rados.rados.0.smithi100.stdout:3190: oid 26 version 1232 is already newer than 1231 2024-04-07T16:26:58.096 INFO:tasks.rados.rados.0.smithi100.stdout:3190: oid 26 updating version 1232 to 1233 2024-04-07T16:26:58.096 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 26 v 1233 (ObjNum 972 snap 294 seq_num 972) dirty exists 2024-04-07T16:26:58.096 INFO:tasks.rados.rados.0.smithi100.stdout:3190: left oid 26 (ObjNum 972 snap 294 seq_num 972) 2024-04-07T16:26:58.096 INFO:tasks.rados.rados.0.smithi100.stdout:3191: finishing write tid 1 to smithi10066867-17 2024-04-07T16:26:58.096 INFO:tasks.rados.rados.0.smithi100.stdout:3191: finishing write tid 2 to smithi10066867-17 2024-04-07T16:26:58.100 INFO:tasks.rados.rados.0.smithi100.stdout:3191: finishing write tid 3 to smithi10066867-17 2024-04-07T16:26:58.100 INFO:tasks.rados.rados.0.smithi100.stdout:3191: finishing write tid 4 to smithi10066867-17 2024-04-07T16:26:58.101 INFO:tasks.rados.rados.0.smithi100.stdout:3190: done (2 left) 2024-04-07T16:26:58.101 INFO:tasks.rados.rados.0.smithi100.stdout:3192: done (1 left) 2024-04-07T16:26:58.101 INFO:tasks.rados.rados.0.smithi100.stdout:3193: read oid 9 snap -1 2024-04-07T16:26:58.101 INFO:tasks.rados.rados.0.smithi100.stdout:3193: expect (ObjNum 849 snap 261 seq_num 849) 2024-04-07T16:26:58.101 INFO:tasks.rados.rados.0.smithi100.stdout:3194: rmattr oid 47 current snap is 294 2024-04-07T16:26:58.101 INFO:tasks.rados.rados.0.smithi100.stdout:3194: done (2 left) 2024-04-07T16:26:58.101 INFO:tasks.rados.rados.0.smithi100.stdout:3195: snap_create 2024-04-07T16:26:58.101 INFO:tasks.rados.rados.0.smithi100.stdout:3191: finishing write tid 5 to smithi10066867-17 2024-04-07T16:26:58.102 INFO:tasks.rados.rados.0.smithi100.stdout:3191: finishing write tid 6 to smithi10066867-17 2024-04-07T16:26:58.102 INFO:tasks.rados.rados.0.smithi100.stdout:3191: oid 17 updating version 0 to 1253 2024-04-07T16:26:58.102 INFO:tasks.rados.rados.0.smithi100.stdout:3191: oid 17 version 1253 is already newer than 1252 2024-04-07T16:26:58.102 INFO:tasks.rados.rados.0.smithi100.stdout:3191: oid 17 version 1253 is already newer than 1249 2024-04-07T16:26:58.103 INFO:tasks.rados.rados.0.smithi100.stdout:3191: oid 17 version 1253 is already newer than 1250 2024-04-07T16:26:58.103 INFO:tasks.rados.rados.0.smithi100.stdout:3191: oid 17 version 1253 is already newer than 1251 2024-04-07T16:26:58.103 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 17 v 1253 (ObjNum 973 snap 294 seq_num 973) dirty exists 2024-04-07T16:26:58.103 INFO:tasks.rados.rados.0.smithi100.stdout:3191: left oid 17 (ObjNum 973 snap 294 seq_num 973) 2024-04-07T16:26:58.130 INFO:tasks.rados.rados.0.smithi100.stdout:3193: expect (ObjNum 849 snap 261 seq_num 849) 2024-04-07T16:26:58.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:57 smithi144 bash[66654]: cluster 2024-04-07T16:26:56.779229+0000 mgr.y (mgr.25175) 328 : cluster [DBG] pgmap v583: 121 pgs: 121 active+clean; 110 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 3.2 MiB/s rd, 1.9 MiB/s wr, 13 op/s 2024-04-07T16:26:58.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:57 smithi144 bash[66654]: cluster 2024-04-07T16:26:56.962673+0000 mon.a (mon.0) 1074 : cluster [DBG] osdmap e661: 8 total, 8 up, 8 in 2024-04-07T16:26:58.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:57 smithi144 bash[66654]: audit 2024-04-07T16:26:57.005708+0000 mon.a (mon.0) 1075 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:26:58.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:57 smithi144 bash[66654]: audit 2024-04-07T16:26:57.010421+0000 mon.a (mon.0) 1076 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:26:58.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:57 smithi144 bash[66654]: audit 2024-04-07T16:26:57.266529+0000 mon.a (mon.0) 1077 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:26:58.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:57 smithi144 bash[66654]: audit 2024-04-07T16:26:57.276851+0000 mon.a (mon.0) 1078 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:26:58.414 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:57 smithi100 bash[74942]: cluster 2024-04-07T16:26:56.779229+0000 mgr.y (mgr.25175) 328 : cluster [DBG] pgmap v583: 121 pgs: 121 active+clean; 110 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 3.2 MiB/s rd, 1.9 MiB/s wr, 13 op/s 2024-04-07T16:26:58.415 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:57 smithi100 bash[74942]: cluster 2024-04-07T16:26:56.962673+0000 mon.a (mon.0) 1074 : cluster [DBG] osdmap e661: 8 total, 8 up, 8 in 2024-04-07T16:26:58.415 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:57 smithi100 bash[74942]: audit 2024-04-07T16:26:57.005708+0000 mon.a (mon.0) 1075 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:26:58.415 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:57 smithi100 bash[74942]: audit 2024-04-07T16:26:57.010421+0000 mon.a (mon.0) 1076 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:26:58.415 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:57 smithi100 bash[74942]: audit 2024-04-07T16:26:57.266529+0000 mon.a (mon.0) 1077 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:26:58.415 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:57 smithi100 bash[74942]: audit 2024-04-07T16:26:57.276851+0000 mon.a (mon.0) 1078 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:26:58.415 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:57 smithi100 bash[76202]: cluster 2024-04-07T16:26:56.779229+0000 mgr.y (mgr.25175) 328 : cluster [DBG] pgmap v583: 121 pgs: 121 active+clean; 110 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 3.2 MiB/s rd, 1.9 MiB/s wr, 13 op/s 2024-04-07T16:26:58.415 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:57 smithi100 bash[76202]: cluster 2024-04-07T16:26:56.962673+0000 mon.a (mon.0) 1074 : cluster [DBG] osdmap e661: 8 total, 8 up, 8 in 2024-04-07T16:26:58.415 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:57 smithi100 bash[76202]: audit 2024-04-07T16:26:57.005708+0000 mon.a (mon.0) 1075 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:26:58.416 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:57 smithi100 bash[76202]: audit 2024-04-07T16:26:57.010421+0000 mon.a (mon.0) 1076 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:26:58.416 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:57 smithi100 bash[76202]: audit 2024-04-07T16:26:57.266529+0000 mon.a (mon.0) 1077 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:26:58.416 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:57 smithi100 bash[76202]: audit 2024-04-07T16:26:57.276851+0000 mon.a (mon.0) 1078 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:26:58.686 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:26:58 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:26:58] "GET /metrics HTTP/1.1" 200 36338 "" "Prometheus/2.43.0" 2024-04-07T16:26:59.024 INFO:tasks.rados.rados.0.smithi100.stdout:3191: done (2 left) 2024-04-07T16:26:59.024 INFO:tasks.rados.rados.0.smithi100.stdout:3193: done (1 left) 2024-04-07T16:26:59.024 INFO:tasks.rados.rados.0.smithi100.stdout:3195: done (0 left) 2024-04-07T16:26:59.024 INFO:tasks.rados.rados.0.smithi100.stdout:3196: read oid 49 snap -1 2024-04-07T16:26:59.024 INFO:tasks.rados.rados.0.smithi100.stdout:3196: expect deleted 2024-04-07T16:26:59.025 INFO:tasks.rados.rados.0.smithi100.stdout:3197: delete oid 13 current snap is 295 2024-04-07T16:26:59.034 INFO:tasks.rados.rados.0.smithi100.stdout:3196: done (1 left) 2024-04-07T16:26:59.034 INFO:tasks.rados.rados.0.smithi100.stdout:3197: done (0 left) 2024-04-07T16:26:59.034 INFO:tasks.rados.rados.0.smithi100.stdout:3198: snap_create 2024-04-07T16:26:59.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:58 smithi144 bash[66654]: cluster 2024-04-07T16:26:57.973100+0000 mon.a (mon.0) 1079 : cluster [DBG] osdmap e662: 8 total, 8 up, 8 in 2024-04-07T16:26:59.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:58 smithi144 bash[66654]: audit 2024-04-07T16:26:58.026586+0000 mon.a (mon.0) 1080 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:26:59.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:58 smithi144 bash[66654]: audit 2024-04-07T16:26:58.031526+0000 mon.a (mon.0) 1081 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:26:59.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:58 smithi144 bash[66654]: audit 2024-04-07T16:26:58.443143+0000 mon.a (mon.0) 1082 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:26:59.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:26:58 smithi144 bash[66654]: audit 2024-04-07T16:26:58.448297+0000 mon.a (mon.0) 1083 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:26:59.436 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:58 smithi100 bash[76202]: cluster 2024-04-07T16:26:57.973100+0000 mon.a (mon.0) 1079 : cluster [DBG] osdmap e662: 8 total, 8 up, 8 in 2024-04-07T16:26:59.436 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:58 smithi100 bash[76202]: audit 2024-04-07T16:26:58.026586+0000 mon.a (mon.0) 1080 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:26:59.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:58 smithi100 bash[76202]: audit 2024-04-07T16:26:58.031526+0000 mon.a (mon.0) 1081 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:26:59.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:58 smithi100 bash[76202]: audit 2024-04-07T16:26:58.443143+0000 mon.a (mon.0) 1082 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:26:59.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:26:58 smithi100 bash[76202]: audit 2024-04-07T16:26:58.448297+0000 mon.a (mon.0) 1083 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:26:59.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:58 smithi100 bash[74942]: cluster 2024-04-07T16:26:57.973100+0000 mon.a (mon.0) 1079 : cluster [DBG] osdmap e662: 8 total, 8 up, 8 in 2024-04-07T16:26:59.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:58 smithi100 bash[74942]: audit 2024-04-07T16:26:58.026586+0000 mon.a (mon.0) 1080 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:26:59.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:58 smithi100 bash[74942]: audit 2024-04-07T16:26:58.031526+0000 mon.a (mon.0) 1081 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:26:59.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:58 smithi100 bash[74942]: audit 2024-04-07T16:26:58.443143+0000 mon.a (mon.0) 1082 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:26:59.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:26:58 smithi100 bash[74942]: audit 2024-04-07T16:26:58.448297+0000 mon.a (mon.0) 1083 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:27:00.038 INFO:tasks.rados.rados.0.smithi100.stdout:3198: done (0 left) 2024-04-07T16:27:00.038 INFO:tasks.rados.rados.0.smithi100.stdout:3199: rollback oid 15 current snap is 296 2024-04-07T16:27:00.038 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 15 to 292 2024-04-07T16:27:00.038 INFO:tasks.rados.rados.0.smithi100.stdout:3200: read oid 40 snap -1 2024-04-07T16:27:00.039 INFO:tasks.rados.rados.0.smithi100.stdout:3200: expect (ObjNum 963 snap 290 seq_num 963) 2024-04-07T16:27:00.039 INFO:tasks.rados.rados.0.smithi100.stdout:3201: delete oid 9 current snap is 296 2024-04-07T16:27:00.051 INFO:tasks.rados.rados.0.smithi100.stdout:3201: done (2 left) 2024-04-07T16:27:00.051 INFO:tasks.rados.rados.0.smithi100.stdout:3202: delete oid 49 current snap is 296 2024-04-07T16:27:00.059 INFO:tasks.rados.rados.0.smithi100.stdout:3202: done (2 left) 2024-04-07T16:27:00.059 INFO:tasks.rados.rados.0.smithi100.stdout:3203: delete oid 17 current snap is 296 2024-04-07T16:27:00.063 INFO:tasks.rados.rados.0.smithi100.stdout:3200: expect (ObjNum 963 snap 290 seq_num 963) 2024-04-07T16:27:00.110 INFO:tasks.rados.rados.0.smithi100.stdout:3199: finishing rollback tid 0 to smithi10066867-15 2024-04-07T16:27:00.110 INFO:tasks.rados.rados.0.smithi100.stdout:3199: finishing rollback tid 1 to smithi10066867-15 2024-04-07T16:27:00.110 INFO:tasks.rados.rados.0.smithi100.stdout:3199: finishing rollback tid 2 to smithi10066867-15 2024-04-07T16:27:00.111 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 15 v 1708 (ObjNum 81 snap 0 seq_num 4239977162) dirty exists 2024-04-07T16:27:00.111 INFO:tasks.rados.rados.0.smithi100.stdout:3199: done (2 left) 2024-04-07T16:27:00.111 INFO:tasks.rados.rados.0.smithi100.stdout:3200: done (1 left) 2024-04-07T16:27:00.111 INFO:tasks.rados.rados.0.smithi100.stdout:3203: done (0 left) 2024-04-07T16:27:00.111 INFO:tasks.rados.rados.0.smithi100.stdout:3204: rmattr oid 32 current snap is 296 2024-04-07T16:27:00.111 INFO:tasks.rados.rados.0.smithi100.stdout:3204: done (0 left) 2024-04-07T16:27:00.111 INFO:tasks.rados.rados.0.smithi100.stdout:3205: snap_remove snap 287 2024-04-07T16:27:00.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:00 smithi144 bash[66654]: cluster 2024-04-07T16:26:58.779991+0000 mgr.y (mgr.25175) 329 : cluster [DBG] pgmap v586: 121 pgs: 1 active+clean+snaptrim, 120 active+clean; 121 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 5.7 MiB/s rd, 2.3 MiB/s wr, 9 op/s 2024-04-07T16:27:00.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:00 smithi144 bash[66654]: cluster 2024-04-07T16:26:59.039569+0000 mon.a (mon.0) 1084 : cluster [DBG] osdmap e663: 8 total, 8 up, 8 in 2024-04-07T16:27:00.436 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:00 smithi100 bash[74942]: cluster 2024-04-07T16:26:58.779991+0000 mgr.y (mgr.25175) 329 : cluster [DBG] pgmap v586: 121 pgs: 1 active+clean+snaptrim, 120 active+clean; 121 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 5.7 MiB/s rd, 2.3 MiB/s wr, 9 op/s 2024-04-07T16:27:00.436 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:00 smithi100 bash[74942]: cluster 2024-04-07T16:26:59.039569+0000 mon.a (mon.0) 1084 : cluster [DBG] osdmap e663: 8 total, 8 up, 8 in 2024-04-07T16:27:00.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:00 smithi100 bash[76202]: cluster 2024-04-07T16:26:58.779991+0000 mgr.y (mgr.25175) 329 : cluster [DBG] pgmap v586: 121 pgs: 1 active+clean+snaptrim, 120 active+clean; 121 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 5.7 MiB/s rd, 2.3 MiB/s wr, 9 op/s 2024-04-07T16:27:00.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:00 smithi100 bash[76202]: cluster 2024-04-07T16:26:59.039569+0000 mon.a (mon.0) 1084 : cluster [DBG] osdmap e663: 8 total, 8 up, 8 in 2024-04-07T16:27:01.042 INFO:tasks.rados.rados.0.smithi100.stdout:3205: done (0 left) 2024-04-07T16:27:01.042 INFO:tasks.rados.rados.0.smithi100.stdout:3206: read oid 29 snap -1 2024-04-07T16:27:01.042 INFO:tasks.rados.rados.0.smithi100.stdout:3206: expect deleted 2024-04-07T16:27:01.042 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 41 current snap is 296 2024-04-07T16:27:01.043 INFO:tasks.rados.rados.0.smithi100.stdout:3207: seq_num 979 ranges {0=802816,802816=729088,1531904=458752,1990656=16384} 2024-04-07T16:27:01.047 INFO:tasks.rados.rados.0.smithi100.stdout:3207: writing smithi10066867-41 from 0 to 802816 tid 1 2024-04-07T16:27:01.052 INFO:tasks.rados.rados.0.smithi100.stdout:3207: writing smithi10066867-41 from 802816 to 1531904 tid 2 2024-04-07T16:27:01.055 INFO:tasks.rados.rados.0.smithi100.stdout:3207: writing smithi10066867-41 from 1531904 to 1990656 tid 3 2024-04-07T16:27:01.055 INFO:tasks.rados.rados.0.smithi100.stdout:3207: writing smithi10066867-41 from 1990656 to 2007040 tid 4 2024-04-07T16:27:01.056 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 2 current snap is 296 2024-04-07T16:27:01.056 INFO:tasks.rados.rados.0.smithi100.stdout:3208: seq_num 980 ranges {0=417792,417792=614400,1032192=663552,1695744=327680} 2024-04-07T16:27:01.058 INFO:tasks.rados.rados.0.smithi100.stdout:3208: writing smithi10066867-2 from 0 to 417792 tid 1 2024-04-07T16:27:01.063 INFO:tasks.rados.rados.0.smithi100.stdout:3208: writing smithi10066867-2 from 417792 to 1032192 tid 2 2024-04-07T16:27:01.068 INFO:tasks.rados.rados.0.smithi100.stdout:3208: writing smithi10066867-2 from 1032192 to 1695744 tid 3 2024-04-07T16:27:01.070 INFO:tasks.rados.rados.0.smithi100.stdout:3208: writing smithi10066867-2 from 1695744 to 2023424 tid 4 2024-04-07T16:27:01.070 INFO:tasks.rados.rados.0.smithi100.stdout:3209: read oid 8 snap -1 2024-04-07T16:27:01.070 INFO:tasks.rados.rados.0.smithi100.stdout:3209: expect (ObjNum 968 snap 293 seq_num 968) 2024-04-07T16:27:01.071 INFO:tasks.rados.rados.0.smithi100.stdout:3210: setattr oid 22 current snap is 296 2024-04-07T16:27:01.073 INFO:tasks.rados.rados.0.smithi100.stdout:3207: finishing write tid 1 to smithi10066867-41 2024-04-07T16:27:01.073 INFO:tasks.rados.rados.0.smithi100.stdout:3206: done (4 left) 2024-04-07T16:27:01.074 INFO:tasks.rados.rados.0.smithi100.stdout:3211: snap_create 2024-04-07T16:27:01.074 INFO:tasks.rados.rados.0.smithi100.stdout:3207: finishing write tid 2 to smithi10066867-41 2024-04-07T16:27:01.074 INFO:tasks.rados.rados.0.smithi100.stdout:3207: finishing write tid 3 to smithi10066867-41 2024-04-07T16:27:01.074 INFO:tasks.rados.rados.0.smithi100.stdout:3207: finishing write tid 4 to smithi10066867-41 2024-04-07T16:27:01.074 INFO:tasks.rados.rados.0.smithi100.stdout:3207: finishing write tid 5 to smithi10066867-41 2024-04-07T16:27:01.074 INFO:tasks.rados.rados.0.smithi100.stdout:3208: finishing write tid 1 to smithi10066867-2 2024-04-07T16:27:01.077 INFO:tasks.rados.rados.0.smithi100.stdout:3207: finishing write tid 6 to smithi10066867-41 2024-04-07T16:27:01.077 INFO:tasks.rados.rados.0.smithi100.stdout:3207: oid 41 updating version 0 to 1503 2024-04-07T16:27:01.077 INFO:tasks.rados.rados.0.smithi100.stdout:3207: oid 41 version 1503 is already newer than 1500 2024-04-07T16:27:01.077 INFO:tasks.rados.rados.0.smithi100.stdout:3207: oid 41 version 1503 is already newer than 1502 2024-04-07T16:27:01.077 INFO:tasks.rados.rados.0.smithi100.stdout:3207: oid 41 updating version 1503 to 1504 2024-04-07T16:27:01.077 INFO:tasks.rados.rados.0.smithi100.stdout:3207: oid 41 version 1504 is already newer than 1501 2024-04-07T16:27:01.078 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 41 v 1504 (ObjNum 979 snap 296 seq_num 979) dirty exists 2024-04-07T16:27:01.078 INFO:tasks.rados.rados.0.smithi100.stdout:3207: left oid 41 (ObjNum 979 snap 296 seq_num 979) 2024-04-07T16:27:01.089 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 22 v 1234 (ObjNum 906 snap 274 seq_num 906) dirty exists 2024-04-07T16:27:01.090 INFO:tasks.rados.rados.0.smithi100.stdout:3208: finishing write tid 2 to smithi10066867-2 2024-04-07T16:27:01.090 INFO:tasks.rados.rados.0.smithi100.stdout:3208: finishing write tid 3 to smithi10066867-2 2024-04-07T16:27:01.090 INFO:tasks.rados.rados.0.smithi100.stdout:3208: finishing write tid 4 to smithi10066867-2 2024-04-07T16:27:01.090 INFO:tasks.rados.rados.0.smithi100.stdout:3208: finishing write tid 5 to smithi10066867-2 2024-04-07T16:27:01.094 INFO:tasks.rados.rados.0.smithi100.stdout:3208: finishing write tid 6 to smithi10066867-2 2024-04-07T16:27:01.094 INFO:tasks.rados.rados.0.smithi100.stdout:3208: oid 2 updating version 0 to 1635 2024-04-07T16:27:01.094 INFO:tasks.rados.rados.0.smithi100.stdout:3208: oid 2 version 1635 is already newer than 1632 2024-04-07T16:27:01.094 INFO:tasks.rados.rados.0.smithi100.stdout:3208: oid 2 updating version 1635 to 1636 2024-04-07T16:27:01.094 INFO:tasks.rados.rados.0.smithi100.stdout:3208: oid 2 version 1636 is already newer than 1633 2024-04-07T16:27:01.094 INFO:tasks.rados.rados.0.smithi100.stdout:3208: oid 2 version 1636 is already newer than 1634 2024-04-07T16:27:01.094 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 2 v 1636 (ObjNum 980 snap 296 seq_num 980) dirty exists 2024-04-07T16:27:01.094 INFO:tasks.rados.rados.0.smithi100.stdout:3208: left oid 2 (ObjNum 980 snap 296 seq_num 980) 2024-04-07T16:27:01.111 INFO:tasks.rados.rados.0.smithi100.stdout:3209: expect (ObjNum 968 snap 293 seq_num 968) 2024-04-07T16:27:01.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:01 smithi144 bash[66654]: cluster 2024-04-07T16:27:00.050457+0000 mon.a (mon.0) 1085 : cluster [DBG] osdmap e664: 8 total, 8 up, 8 in 2024-04-07T16:27:01.436 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:01 smithi100 bash[74942]: cluster 2024-04-07T16:27:00.050457+0000 mon.a (mon.0) 1085 : cluster [DBG] osdmap e664: 8 total, 8 up, 8 in 2024-04-07T16:27:01.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:01 smithi100 bash[76202]: cluster 2024-04-07T16:27:00.050457+0000 mon.a (mon.0) 1085 : cluster [DBG] osdmap e664: 8 total, 8 up, 8 in 2024-04-07T16:27:01.909 INFO:tasks.rados.rados.0.smithi100.stdout:3207: done (4 left) 2024-04-07T16:27:01.909 INFO:tasks.rados.rados.0.smithi100.stdout:3208: done (3 left) 2024-04-07T16:27:01.909 INFO:tasks.rados.rados.0.smithi100.stdout:3209: done (2 left) 2024-04-07T16:27:01.909 INFO:tasks.rados.rados.0.smithi100.stdout:3210: done (1 left) 2024-04-07T16:27:01.909 INFO:tasks.rados.rados.0.smithi100.stdout:3211: done (0 left) 2024-04-07T16:27:01.909 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 14 current snap is 297 2024-04-07T16:27:01.909 INFO:tasks.rados.rados.0.smithi100.stdout:3212: seq_num 981 ranges {0=434176,434176=540672,974848=466944,1441792=524288,1966080=73728} 2024-04-07T16:27:01.913 INFO:tasks.rados.rados.0.smithi100.stdout:3212: writing smithi10066867-14 from 0 to 434176 tid 1 2024-04-07T16:27:01.919 INFO:tasks.rados.rados.0.smithi100.stdout:3212: writing smithi10066867-14 from 434176 to 974848 tid 2 2024-04-07T16:27:01.924 INFO:tasks.rados.rados.0.smithi100.stdout:3212: writing smithi10066867-14 from 974848 to 1441792 tid 3 2024-04-07T16:27:01.929 INFO:tasks.rados.rados.0.smithi100.stdout:3212: writing smithi10066867-14 from 1441792 to 1966080 tid 4 2024-04-07T16:27:01.930 INFO:tasks.rados.rados.0.smithi100.stdout:3212: writing smithi10066867-14 from 1966080 to 2039808 tid 5 2024-04-07T16:27:01.930 INFO:tasks.rados.rados.0.smithi100.stdout:3213: rollback oid 15 current snap is 297 2024-04-07T16:27:01.930 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 15 to 296 2024-04-07T16:27:01.930 INFO:tasks.rados.rados.0.smithi100.stdout:3212: finishing write tid 1 to smithi10066867-14 2024-04-07T16:27:01.930 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 46 current snap is 297 2024-04-07T16:27:01.931 INFO:tasks.rados.rados.0.smithi100.stdout:3214: seq_num 982 ranges {3178496=450560,3629056=425984,4055040=679936,4734976=466944,5201920=32768} 2024-04-07T16:27:01.960 INFO:tasks.rados.rados.0.smithi100.stdout:3214: writing smithi10066867-46 from 3178496 to 3629056 tid 1 2024-04-07T16:27:01.963 INFO:tasks.rados.rados.0.smithi100.stdout:3214: writing smithi10066867-46 from 3629056 to 4055040 tid 2 2024-04-07T16:27:01.969 INFO:tasks.rados.rados.0.smithi100.stdout:3214: writing smithi10066867-46 from 4055040 to 4734976 tid 3 2024-04-07T16:27:01.974 INFO:tasks.rados.rados.0.smithi100.stdout:3214: writing smithi10066867-46 from 4734976 to 5201920 tid 4 2024-04-07T16:27:01.974 INFO:tasks.rados.rados.0.smithi100.stdout:3214: writing smithi10066867-46 from 5201920 to 5234688 tid 5 2024-04-07T16:27:01.974 INFO:tasks.rados.rados.0.smithi100.stdout:3215: read oid 49 snap 295 2024-04-07T16:27:01.974 INFO:tasks.rados.rados.0.smithi100.stdout:3215: expect deleted 2024-04-07T16:27:01.974 INFO:tasks.rados.rados.0.smithi100.stdout:3216: rmattr oid 8 current snap is 297 2024-04-07T16:27:01.974 INFO:tasks.rados.rados.0.smithi100.stdout:3216: done (4 left) 2024-04-07T16:27:01.974 INFO:tasks.rados.rados.0.smithi100.stdout:3217: snap_create 2024-04-07T16:27:01.974 INFO:tasks.rados.rados.0.smithi100.stdout:3212: finishing write tid 2 to smithi10066867-14 2024-04-07T16:27:01.974 INFO:tasks.rados.rados.0.smithi100.stdout:3212: finishing write tid 3 to smithi10066867-14 2024-04-07T16:27:01.974 INFO:tasks.rados.rados.0.smithi100.stdout:3212: finishing write tid 4 to smithi10066867-14 2024-04-07T16:27:01.974 INFO:tasks.rados.rados.0.smithi100.stdout:3212: finishing write tid 5 to smithi10066867-14 2024-04-07T16:27:01.974 INFO:tasks.rados.rados.0.smithi100.stdout:3212: finishing write tid 6 to smithi10066867-14 2024-04-07T16:27:01.975 INFO:tasks.rados.rados.0.smithi100.stdout:3212: finishing write tid 7 to smithi10066867-14 2024-04-07T16:27:01.975 INFO:tasks.rados.rados.0.smithi100.stdout:3212: oid 14 updating version 0 to 1536 2024-04-07T16:27:01.975 INFO:tasks.rados.rados.0.smithi100.stdout:3212: oid 14 version 1536 is already newer than 1535 2024-04-07T16:27:01.975 INFO:tasks.rados.rados.0.smithi100.stdout:3212: oid 14 updating version 1536 to 1537 2024-04-07T16:27:01.975 INFO:tasks.rados.rados.0.smithi100.stdout:3212: oid 14 version 1537 is already newer than 1532 2024-04-07T16:27:01.975 INFO:tasks.rados.rados.0.smithi100.stdout:3212: oid 14 version 1537 is already newer than 1533 2024-04-07T16:27:01.975 INFO:tasks.rados.rados.0.smithi100.stdout:3212: oid 14 version 1537 is already newer than 1534 2024-04-07T16:27:01.975 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 14 v 1537 (ObjNum 981 snap 297 seq_num 981) dirty exists 2024-04-07T16:27:01.975 INFO:tasks.rados.rados.0.smithi100.stdout:3212: left oid 14 (ObjNum 981 snap 297 seq_num 981) 2024-04-07T16:27:01.975 INFO:tasks.rados.rados.0.smithi100.stdout:3213: finishing rollback tid 0 to smithi10066867-15 2024-04-07T16:27:01.975 INFO:tasks.rados.rados.0.smithi100.stdout:3213: finishing rollback tid 1 to smithi10066867-15 2024-04-07T16:27:01.975 INFO:tasks.rados.rados.0.smithi100.stdout:3213: finishing rollback tid 2 to smithi10066867-15 2024-04-07T16:27:01.975 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 15 v 1716 (ObjNum 65 snap 0 seq_num 1582719585) dirty exists 2024-04-07T16:27:01.975 INFO:tasks.rados.rados.0.smithi100.stdout:3214: finishing write tid 1 to smithi10066867-46 2024-04-07T16:27:01.975 INFO:tasks.rados.rados.0.smithi100.stdout:3214: finishing write tid 2 to smithi10066867-46 2024-04-07T16:27:01.979 INFO:tasks.rados.rados.0.smithi100.stdout:3214: finishing write tid 3 to smithi10066867-46 2024-04-07T16:27:01.979 INFO:tasks.rados.rados.0.smithi100.stdout:3214: finishing write tid 4 to smithi10066867-46 2024-04-07T16:27:01.982 INFO:tasks.rados.rados.0.smithi100.stdout:3214: finishing write tid 5 to smithi10066867-46 2024-04-07T16:27:01.982 INFO:tasks.rados.rados.0.smithi100.stdout:3214: finishing write tid 6 to smithi10066867-46 2024-04-07T16:27:01.986 INFO:tasks.rados.rados.0.smithi100.stdout:3214: finishing write tid 7 to smithi10066867-46 2024-04-07T16:27:01.986 INFO:tasks.rados.rados.0.smithi100.stdout:3214: oid 46 updating version 0 to 1543 2024-04-07T16:27:01.986 INFO:tasks.rados.rados.0.smithi100.stdout:3214: oid 46 version 1543 is already newer than 1542 2024-04-07T16:27:01.987 INFO:tasks.rados.rados.0.smithi100.stdout:3214: oid 46 version 1543 is already newer than 1540 2024-04-07T16:27:01.987 INFO:tasks.rados.rados.0.smithi100.stdout:3214: oid 46 updating version 1543 to 1544 2024-04-07T16:27:01.987 INFO:tasks.rados.rados.0.smithi100.stdout:3214: oid 46 updating version 1544 to 1545 2024-04-07T16:27:01.987 INFO:tasks.rados.rados.0.smithi100.stdout:3214: oid 46 version 1545 is already newer than 1541 2024-04-07T16:27:01.987 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 46 v 1545 (ObjNum 982 snap 297 seq_num 982) dirty exists 2024-04-07T16:27:01.987 INFO:tasks.rados.rados.0.smithi100.stdout:3214: left oid 46 (ObjNum 982 snap 297 seq_num 982) 2024-04-07T16:27:02.305 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:02 smithi144 bash[66654]: cluster 2024-04-07T16:27:00.780690+0000 mgr.y (mgr.25175) 330 : cluster [DBG] pgmap v589: 121 pgs: 1 active+clean+snaptrim, 120 active+clean; 130 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 9.0 MiB/s rd, 4.7 MiB/s wr, 33 op/s 2024-04-07T16:27:02.305 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:02 smithi144 bash[66654]: cluster 2024-04-07T16:27:01.055659+0000 mon.a (mon.0) 1086 : cluster [DBG] osdmap e665: 8 total, 8 up, 8 in 2024-04-07T16:27:02.305 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:02 smithi144 bash[66654]: audit 2024-04-07T16:27:01.728029+0000 mon.a (mon.0) 1087 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:27:02.305 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:02 smithi144 bash[66654]: cluster 2024-04-07T16:27:01.923181+0000 mon.a (mon.0) 1088 : cluster [DBG] osdmap e666: 8 total, 8 up, 8 in 2024-04-07T16:27:02.436 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:02 smithi100 bash[76202]: cluster 2024-04-07T16:27:00.780690+0000 mgr.y (mgr.25175) 330 : cluster [DBG] pgmap v589: 121 pgs: 1 active+clean+snaptrim, 120 active+clean; 130 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 9.0 MiB/s rd, 4.7 MiB/s wr, 33 op/s 2024-04-07T16:27:02.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:02 smithi100 bash[76202]: cluster 2024-04-07T16:27:01.055659+0000 mon.a (mon.0) 1086 : cluster [DBG] osdmap e665: 8 total, 8 up, 8 in 2024-04-07T16:27:02.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:02 smithi100 bash[76202]: audit 2024-04-07T16:27:01.728029+0000 mon.a (mon.0) 1087 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:27:02.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:02 smithi100 bash[76202]: cluster 2024-04-07T16:27:01.923181+0000 mon.a (mon.0) 1088 : cluster [DBG] osdmap e666: 8 total, 8 up, 8 in 2024-04-07T16:27:02.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:02 smithi100 bash[74942]: cluster 2024-04-07T16:27:00.780690+0000 mgr.y (mgr.25175) 330 : cluster [DBG] pgmap v589: 121 pgs: 1 active+clean+snaptrim, 120 active+clean; 130 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 9.0 MiB/s rd, 4.7 MiB/s wr, 33 op/s 2024-04-07T16:27:02.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:02 smithi100 bash[74942]: cluster 2024-04-07T16:27:01.055659+0000 mon.a (mon.0) 1086 : cluster [DBG] osdmap e665: 8 total, 8 up, 8 in 2024-04-07T16:27:02.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:02 smithi100 bash[74942]: audit 2024-04-07T16:27:01.728029+0000 mon.a (mon.0) 1087 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:27:02.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:02 smithi100 bash[74942]: cluster 2024-04-07T16:27:01.923181+0000 mon.a (mon.0) 1088 : cluster [DBG] osdmap e666: 8 total, 8 up, 8 in 2024-04-07T16:27:02.605 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:27:02 smithi144 bash[59794]: ts=2024-04-07T16:27:02.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:27:02.909 INFO:tasks.rados.rados.0.smithi100.stdout:3212: done (4 left) 2024-04-07T16:27:02.909 INFO:tasks.rados.rados.0.smithi100.stdout:3213: done (3 left) 2024-04-07T16:27:02.909 INFO:tasks.rados.rados.0.smithi100.stdout:3214: done (2 left) 2024-04-07T16:27:02.909 INFO:tasks.rados.rados.0.smithi100.stdout:3215: done (1 left) 2024-04-07T16:27:02.909 INFO:tasks.rados.rados.0.smithi100.stdout:3217: done (0 left) 2024-04-07T16:27:02.909 INFO:tasks.rados.rados.0.smithi100.stdout:3218: read oid 5 snap -1 2024-04-07T16:27:02.909 INFO:tasks.rados.rados.0.smithi100.stdout:3218: expect (ObjNum 969 snap 293 seq_num 969) 2024-04-07T16:27:02.909 INFO:tasks.rados.rados.0.smithi100.stdout:3219: copy_from oid 43 from oid 13 current snap is 298 2024-04-07T16:27:02.910 INFO:tasks.rados.rados.0.smithi100.stdout:3220: copy_from oid 20 from oid 50 current snap is 298 2024-04-07T16:27:02.910 INFO:tasks.rados.rados.0.smithi100.stdout:3221: delete oid 45 current snap is 298 2024-04-07T16:27:02.919 INFO:tasks.rados.rados.0.smithi100.stdout:3219: finishing copy_from to smithi10066867-43 2024-04-07T16:27:02.919 INFO:tasks.rados.rados.0.smithi100.stdout:3219: got expected ENOENT (src dne) 2024-04-07T16:27:02.920 INFO:tasks.rados.rados.0.smithi100.stdout:3219: finishing copy_from racing read to smithi10066867-43 2024-04-07T16:27:02.922 INFO:tasks.rados.rados.0.smithi100.stdout:3219: done (3 left) 2024-04-07T16:27:02.922 INFO:tasks.rados.rados.0.smithi100.stdout:3221: done (2 left) 2024-04-07T16:27:02.922 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 41 current snap is 298 2024-04-07T16:27:02.922 INFO:tasks.rados.rados.0.smithi100.stdout:3222: seq_num 984 ranges {2007040=483328,2490368=598016,3088384=647168,3735552=360448} 2024-04-07T16:27:02.947 INFO:tasks.rados.rados.0.smithi100.stdout:3222: writing smithi10066867-41 from 2007040 to 2490368 tid 1 2024-04-07T16:27:02.954 INFO:tasks.rados.rados.0.smithi100.stdout:3222: writing smithi10066867-41 from 2490368 to 3088384 tid 2 2024-04-07T16:27:02.961 INFO:tasks.rados.rados.0.smithi100.stdout:3222: writing smithi10066867-41 from 3088384 to 3735552 tid 3 2024-04-07T16:27:02.965 INFO:tasks.rados.rados.0.smithi100.stdout:3222: writing smithi10066867-41 from 3735552 to 4096000 tid 4 2024-04-07T16:27:02.965 INFO:tasks.rados.rados.0.smithi100.stdout:3223: copy_from oid 18 from oid 35 current snap is 298 2024-04-07T16:27:02.965 INFO:tasks.rados.rados.0.smithi100.stdout:3218: expect (ObjNum 969 snap 293 seq_num 969) 2024-04-07T16:27:03.125 INFO:tasks.rados.rados.0.smithi100.stdout:3220: finishing copy_from racing read to smithi10066867-20 2024-04-07T16:27:03.126 INFO:tasks.rados.rados.0.smithi100.stdout:3220: finishing copy_from to smithi10066867-20 2024-04-07T16:27:03.126 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 20 v 1522 (ObjNum 960 snap 289 seq_num 960) dirty exists 2024-04-07T16:27:03.126 INFO:tasks.rados.rados.0.smithi100.stdout:3222: finishing write tid 1 to smithi10066867-41 2024-04-07T16:27:03.126 INFO:tasks.rados.rados.0.smithi100.stdout:3222: finishing write tid 2 to smithi10066867-41 2024-04-07T16:27:03.126 INFO:tasks.rados.rados.0.smithi100.stdout:3222: finishing write tid 3 to smithi10066867-41 2024-04-07T16:27:03.126 INFO:tasks.rados.rados.0.smithi100.stdout:3222: finishing write tid 4 to smithi10066867-41 2024-04-07T16:27:03.126 INFO:tasks.rados.rados.0.smithi100.stdout:3222: finishing write tid 5 to smithi10066867-41 2024-04-07T16:27:03.126 INFO:tasks.rados.rados.0.smithi100.stdout:3222: finishing write tid 6 to smithi10066867-41 2024-04-07T16:27:03.126 INFO:tasks.rados.rados.0.smithi100.stdout:3222: oid 41 updating version 0 to 1512 2024-04-07T16:27:03.126 INFO:tasks.rados.rados.0.smithi100.stdout:3222: oid 41 version 1512 is already newer than 1511 2024-04-07T16:27:03.126 INFO:tasks.rados.rados.0.smithi100.stdout:3222: oid 41 version 1512 is already newer than 1509 2024-04-07T16:27:03.126 INFO:tasks.rados.rados.0.smithi100.stdout:3222: oid 41 version 1512 is already newer than 1508 2024-04-07T16:27:03.127 INFO:tasks.rados.rados.0.smithi100.stdout:3222: oid 41 version 1512 is already newer than 1510 2024-04-07T16:27:03.127 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 41 v 1512 (ObjNum 984 snap 298 seq_num 984) dirty exists 2024-04-07T16:27:03.127 INFO:tasks.rados.rados.0.smithi100.stdout:3222: left oid 41 (ObjNum 984 snap 298 seq_num 984) 2024-04-07T16:27:03.127 INFO:tasks.rados.rados.0.smithi100.stdout:3223: finishing copy_from racing read to smithi10066867-18 2024-04-07T16:27:03.127 INFO:tasks.rados.rados.0.smithi100.stdout:3223: finishing copy_from to smithi10066867-18 2024-04-07T16:27:03.127 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 18 v 1686 (ObjNum 941 snap 282 seq_num 941) dirty exists 2024-04-07T16:27:03.127 INFO:tasks.rados.rados.0.smithi100.stdout:3218: done (3 left) 2024-04-07T16:27:03.127 INFO:tasks.rados.rados.0.smithi100.stdout:3220: done (2 left) 2024-04-07T16:27:03.127 INFO:tasks.rados.rados.0.smithi100.stdout:3222: done (1 left) 2024-04-07T16:27:03.127 INFO:tasks.rados.rados.0.smithi100.stdout:3223: done (0 left) 2024-04-07T16:27:03.127 INFO:tasks.rados.rados.0.smithi100.stdout:3224: copy_from oid 15 from oid 1 current snap is 298 2024-04-07T16:27:03.127 INFO:tasks.rados.rados.0.smithi100.stdout:3225: snap_remove snap 293 2024-04-07T16:27:04.050 INFO:tasks.rados.rados.0.smithi100.stdout:3225: done (1 left) 2024-04-07T16:27:04.050 INFO:tasks.rados.rados.0.smithi100.stdout:3226: copy_from oid 22 from oid 38 current snap is 298 2024-04-07T16:27:04.050 INFO:tasks.rados.rados.0.smithi100.stdout:3224: finishing copy_from racing read to smithi10066867-15 2024-04-07T16:27:04.050 INFO:tasks.rados.rados.0.smithi100.stdout:3224: finishing copy_from to smithi10066867-15 2024-04-07T16:27:04.050 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 15 v 1446 (ObjNum 913 snap 274 seq_num 913) dirty exists 2024-04-07T16:27:04.050 INFO:tasks.rados.rados.0.smithi100.stdout:3224: done (1 left) 2024-04-07T16:27:04.050 INFO:tasks.rados.rados.0.smithi100.stdout:3227: copy_from oid 12 from oid 28 current snap is 298 2024-04-07T16:27:04.050 INFO:tasks.rados.rados.0.smithi100.stdout:3228: delete oid 50 current snap is 298 2024-04-07T16:27:04.058 INFO:tasks.rados.rados.0.smithi100.stdout:3228: done (2 left) 2024-04-07T16:27:04.059 INFO:tasks.rados.rados.0.smithi100.stdout:3229: snap_remove snap 294 2024-04-07T16:27:04.186 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:03 smithi100 bash[76202]: cluster 2024-04-07T16:27:02.781794+0000 mgr.y (mgr.25175) 331 : cluster [DBG] pgmap v592: 121 pgs: 1 active+clean+snaptrim, 120 active+clean; 124 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 8.4 MiB/s rd, 2.8 MiB/s wr, 31 op/s 2024-04-07T16:27:04.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:03 smithi100 bash[76202]: cluster 2024-04-07T16:27:02.924249+0000 mon.a (mon.0) 1089 : cluster [DBG] osdmap e667: 8 total, 8 up, 8 in 2024-04-07T16:27:04.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:03 smithi100 bash[74942]: cluster 2024-04-07T16:27:02.781794+0000 mgr.y (mgr.25175) 331 : cluster [DBG] pgmap v592: 121 pgs: 1 active+clean+snaptrim, 120 active+clean; 124 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 8.4 MiB/s rd, 2.8 MiB/s wr, 31 op/s 2024-04-07T16:27:04.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:03 smithi100 bash[74942]: cluster 2024-04-07T16:27:02.924249+0000 mon.a (mon.0) 1089 : cluster [DBG] osdmap e667: 8 total, 8 up, 8 in 2024-04-07T16:27:04.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:03 smithi144 bash[66654]: cluster 2024-04-07T16:27:02.781794+0000 mgr.y (mgr.25175) 331 : cluster [DBG] pgmap v592: 121 pgs: 1 active+clean+snaptrim, 120 active+clean; 124 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 8.4 MiB/s rd, 2.8 MiB/s wr, 31 op/s 2024-04-07T16:27:04.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:03 smithi144 bash[66654]: cluster 2024-04-07T16:27:02.924249+0000 mon.a (mon.0) 1089 : cluster [DBG] osdmap e667: 8 total, 8 up, 8 in 2024-04-07T16:27:05.064 INFO:tasks.rados.rados.0.smithi100.stdout:3227: finishing copy_from racing read to smithi10066867-12 2024-04-07T16:27:05.064 INFO:tasks.rados.rados.0.smithi100.stdout:3227: finishing copy_from to smithi10066867-12 2024-04-07T16:27:05.064 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 12 v 1690 (ObjNum 794 snap 251 seq_num 794) dirty exists 2024-04-07T16:27:05.064 INFO:tasks.rados.rados.0.smithi100.stdout:3226: finishing copy_from racing read to smithi10066867-22 2024-04-07T16:27:05.064 INFO:tasks.rados.rados.0.smithi100.stdout:3226: finishing copy_from to smithi10066867-22 2024-04-07T16:27:05.064 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 22 v 1147 (ObjNum 950 snap 285 seq_num 950) dirty exists 2024-04-07T16:27:05.064 INFO:tasks.rados.rados.0.smithi100.stdout:3226: done (2 left) 2024-04-07T16:27:05.064 INFO:tasks.rados.rados.0.smithi100.stdout:3227: done (1 left) 2024-04-07T16:27:05.064 INFO:tasks.rados.rados.0.smithi100.stdout:3229: done (0 left) 2024-04-07T16:27:05.064 INFO:tasks.rados.rados.0.smithi100.stdout:3230: read oid 39 snap -1 2024-04-07T16:27:05.064 INFO:tasks.rados.rados.0.smithi100.stdout:3230: expect (ObjNum 924 snap 278 seq_num 924) 2024-04-07T16:27:05.064 INFO:tasks.rados.rados.0.smithi100.stdout:3231: setattr oid 34 current snap is 298 2024-04-07T16:27:05.065 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 38 current snap is 298 2024-04-07T16:27:05.065 INFO:tasks.rados.rados.0.smithi100.stdout:3232: seq_num 986 ranges {6176768=516096,6692864=770048,7462912=655360,8118272=180224} 2024-04-07T16:27:05.103 INFO:tasks.rados.rados.0.smithi100.stdout:3232: writing smithi10066867-38 from 6176768 to 6692864 tid 1 2024-04-07T16:27:05.108 INFO:tasks.rados.rados.0.smithi100.stdout:3232: writing smithi10066867-38 from 6692864 to 7462912 tid 2 2024-04-07T16:27:05.111 INFO:tasks.rados.rados.0.smithi100.stdout:3232: writing smithi10066867-38 from 7462912 to 8118272 tid 3 2024-04-07T16:27:05.112 INFO:tasks.rados.rados.0.smithi100.stdout:3232: writing smithi10066867-38 from 8118272 to 8298496 tid 4 2024-04-07T16:27:05.112 INFO:tasks.rados.rados.0.smithi100.stdout:3230: expect (ObjNum 924 snap 278 seq_num 924) 2024-04-07T16:27:05.142 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 34 v 1546 (ObjNum 962 snap 290 seq_num 962) dirty exists 2024-04-07T16:27:05.142 INFO:tasks.rados.rados.0.smithi100.stdout:3232: finishing write tid 1 to smithi10066867-38 2024-04-07T16:27:05.142 INFO:tasks.rados.rados.0.smithi100.stdout:3232: finishing write tid 2 to smithi10066867-38 2024-04-07T16:27:05.142 INFO:tasks.rados.rados.0.smithi100.stdout:3232: finishing write tid 3 to smithi10066867-38 2024-04-07T16:27:05.142 INFO:tasks.rados.rados.0.smithi100.stdout:3232: finishing write tid 4 to smithi10066867-38 2024-04-07T16:27:05.142 INFO:tasks.rados.rados.0.smithi100.stdout:3232: finishing write tid 5 to smithi10066867-38 2024-04-07T16:27:05.142 INFO:tasks.rados.rados.0.smithi100.stdout:3232: finishing write tid 6 to smithi10066867-38 2024-04-07T16:27:05.142 INFO:tasks.rados.rados.0.smithi100.stdout:3232: oid 38 updating version 0 to 1148 2024-04-07T16:27:05.142 INFO:tasks.rados.rados.0.smithi100.stdout:3232: oid 38 updating version 1148 to 1152 2024-04-07T16:27:05.142 INFO:tasks.rados.rados.0.smithi100.stdout:3232: oid 38 version 1152 is already newer than 1150 2024-04-07T16:27:05.143 INFO:tasks.rados.rados.0.smithi100.stdout:3232: oid 38 version 1152 is already newer than 1151 2024-04-07T16:27:05.143 INFO:tasks.rados.rados.0.smithi100.stdout:3232: oid 38 version 1152 is already newer than 1149 2024-04-07T16:27:05.143 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 38 v 1152 (ObjNum 986 snap 298 seq_num 986) dirty exists 2024-04-07T16:27:05.143 INFO:tasks.rados.rados.0.smithi100.stdout:3232: left oid 38 (ObjNum 986 snap 298 seq_num 986) 2024-04-07T16:27:05.143 INFO:tasks.rados.rados.0.smithi100.stdout:3230: done (2 left) 2024-04-07T16:27:05.143 INFO:tasks.rados.rados.0.smithi100.stdout:3231: done (1 left) 2024-04-07T16:27:05.143 INFO:tasks.rados.rados.0.smithi100.stdout:3232: done (0 left) 2024-04-07T16:27:05.143 INFO:tasks.rados.rados.0.smithi100.stdout:3233: read oid 46 snap -1 2024-04-07T16:27:05.143 INFO:tasks.rados.rados.0.smithi100.stdout:3233: expect (ObjNum 982 snap 297 seq_num 982) 2024-04-07T16:27:05.143 INFO:tasks.rados.rados.0.smithi100.stdout:3234: read oid 15 snap -1 2024-04-07T16:27:05.143 INFO:tasks.rados.rados.0.smithi100.stdout:3234: expect (ObjNum 913 snap 274 seq_num 913) 2024-04-07T16:27:05.143 INFO:tasks.rados.rados.0.smithi100.stdout:3235: rollback oid 27 current snap is 298 2024-04-07T16:27:05.143 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 27 to 296 2024-04-07T16:27:05.143 INFO:tasks.rados.rados.0.smithi100.stdout:3236: copy_from oid 45 from oid 41 current snap is 298 2024-04-07T16:27:05.143 INFO:tasks.rados.rados.0.smithi100.stdout:3237: snap_remove snap 292 2024-04-07T16:27:05.310 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:05 smithi144 bash[66654]: cluster 2024-04-07T16:27:04.065154+0000 mon.a (mon.0) 1090 : cluster [DBG] osdmap e668: 8 total, 8 up, 8 in 2024-04-07T16:27:05.436 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:05 smithi100 bash[74942]: cluster 2024-04-07T16:27:04.065154+0000 mon.a (mon.0) 1090 : cluster [DBG] osdmap e668: 8 total, 8 up, 8 in 2024-04-07T16:27:05.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:05 smithi100 bash[76202]: cluster 2024-04-07T16:27:04.065154+0000 mon.a (mon.0) 1090 : cluster [DBG] osdmap e668: 8 total, 8 up, 8 in 2024-04-07T16:27:06.073 INFO:tasks.rados.rados.0.smithi100.stdout:3237: done (4 left) 2024-04-07T16:27:06.073 INFO:tasks.rados.rados.0.smithi100.stdout:3238: rollback oid 30 current snap is 298 2024-04-07T16:27:06.073 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 30 to 297 2024-04-07T16:27:06.074 INFO:tasks.rados.rados.0.smithi100.stdout:3239: delete oid 6 current snap is 298 2024-04-07T16:27:06.074 INFO:tasks.rados.rados.0.smithi100.stdout:3235: finishing rollback tid 0 to smithi10066867-27 2024-04-07T16:27:06.074 INFO:tasks.rados.rados.0.smithi100.stdout:3235: finishing rollback tid 1 to smithi10066867-27 2024-04-07T16:27:06.074 INFO:tasks.rados.rados.0.smithi100.stdout:3235: finishing rollback tid 2 to smithi10066867-27 2024-04-07T16:27:06.074 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 27 v 1171 (ObjNum 953 snap 286 seq_num 953) dirty exists 2024-04-07T16:27:06.075 INFO:tasks.rados.rados.0.smithi100.stdout:3234: expect (ObjNum 913 snap 274 seq_num 913) 2024-04-07T16:27:06.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:06 smithi100 bash[74942]: cluster 2024-04-07T16:27:04.782687+0000 mgr.y (mgr.25175) 332 : cluster [DBG] pgmap v595: 121 pgs: 121 active+clean; 141 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 11 MiB/s rd, 4.8 MiB/s wr, 26 op/s 2024-04-07T16:27:06.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:06 smithi100 bash[74942]: cluster 2024-04-07T16:27:05.078551+0000 mon.a (mon.0) 1091 : cluster [DBG] osdmap e669: 8 total, 8 up, 8 in 2024-04-07T16:27:06.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:06 smithi100 bash[76202]: cluster 2024-04-07T16:27:04.782687+0000 mgr.y (mgr.25175) 332 : cluster [DBG] pgmap v595: 121 pgs: 121 active+clean; 141 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 11 MiB/s rd, 4.8 MiB/s wr, 26 op/s 2024-04-07T16:27:06.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:06 smithi100 bash[76202]: cluster 2024-04-07T16:27:05.078551+0000 mon.a (mon.0) 1091 : cluster [DBG] osdmap e669: 8 total, 8 up, 8 in 2024-04-07T16:27:06.252 INFO:tasks.rados.rados.0.smithi100.stdout:3233: expect (ObjNum 982 snap 297 seq_num 982) 2024-04-07T16:27:06.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:06 smithi144 bash[66654]: cluster 2024-04-07T16:27:04.782687+0000 mgr.y (mgr.25175) 332 : cluster [DBG] pgmap v595: 121 pgs: 121 active+clean; 141 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 11 MiB/s rd, 4.8 MiB/s wr, 26 op/s 2024-04-07T16:27:06.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:06 smithi144 bash[66654]: cluster 2024-04-07T16:27:05.078551+0000 mon.a (mon.0) 1091 : cluster [DBG] osdmap e669: 8 total, 8 up, 8 in 2024-04-07T16:27:06.457 INFO:tasks.rados.rados.0.smithi100.stdout:3236: finishing copy_from racing read to smithi10066867-45 2024-04-07T16:27:06.457 INFO:tasks.rados.rados.0.smithi100.stdout:3236: finishing copy_from to smithi10066867-45 2024-04-07T16:27:06.458 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 45 v 1512 (ObjNum 984 snap 298 seq_num 984) dirty exists 2024-04-07T16:27:06.458 INFO:tasks.rados.rados.0.smithi100.stdout:3238: finishing rollback tid 0 to smithi10066867-30 2024-04-07T16:27:06.458 INFO:tasks.rados.rados.0.smithi100.stdout:3238: finishing rollback tid 1 to smithi10066867-30 2024-04-07T16:27:06.458 INFO:tasks.rados.rados.0.smithi100.stdout:3238: finishing rollback tid 2 to smithi10066867-30 2024-04-07T16:27:06.458 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 30 v 1269 (ObjNum 880 snap 264 seq_num 880) dirty exists 2024-04-07T16:27:06.458 INFO:tasks.rados.rados.0.smithi100.stdout:3233: done (5 left) 2024-04-07T16:27:06.458 INFO:tasks.rados.rados.0.smithi100.stdout:3234: done (4 left) 2024-04-07T16:27:06.458 INFO:tasks.rados.rados.0.smithi100.stdout:3235: done (3 left) 2024-04-07T16:27:06.458 INFO:tasks.rados.rados.0.smithi100.stdout:3236: done (2 left) 2024-04-07T16:27:06.458 INFO:tasks.rados.rados.0.smithi100.stdout:3238: done (1 left) 2024-04-07T16:27:06.458 INFO:tasks.rados.rados.0.smithi100.stdout:3239: done (0 left) 2024-04-07T16:27:06.458 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 34 current snap is 298 2024-04-07T16:27:06.458 INFO:tasks.rados.rados.0.smithi100.stdout:3240: seq_num 988 ranges {1720320=557056,2277376=532480,2809856=622592,3432448=450560} 2024-04-07T16:27:06.470 INFO:tasks.rados.rados.0.smithi100.stdout:3240: writing smithi10066867-34 from 1720320 to 2277376 tid 1 2024-04-07T16:27:06.473 INFO:tasks.rados.rados.0.smithi100.stdout:3240: writing smithi10066867-34 from 2277376 to 2809856 tid 2 2024-04-07T16:27:06.477 INFO:tasks.rados.rados.0.smithi100.stdout:3240: writing smithi10066867-34 from 2809856 to 3432448 tid 3 2024-04-07T16:27:06.480 INFO:tasks.rados.rados.0.smithi100.stdout:3240: writing smithi10066867-34 from 3432448 to 3883008 tid 4 2024-04-07T16:27:06.481 INFO:tasks.rados.rados.0.smithi100.stdout:3241: read oid 41 snap 297 2024-04-07T16:27:06.481 INFO:tasks.rados.rados.0.smithi100.stdout:3241: expect (ObjNum 979 snap 296 seq_num 979) 2024-04-07T16:27:06.481 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 24 current snap is 298 2024-04-07T16:27:06.481 INFO:tasks.rados.rados.0.smithi100.stdout:3242: seq_num 989 ranges {8617984=573440,9191424=466944,9658368=434176,10092544=638976,10731520=65536} 2024-04-07T16:27:06.532 INFO:tasks.rados.rados.0.smithi100.stdout:3242: writing smithi10066867-24 from 8617984 to 9191424 tid 1 2024-04-07T16:27:06.535 INFO:tasks.rados.rados.0.smithi100.stdout:3242: writing smithi10066867-24 from 9191424 to 9658368 tid 2 2024-04-07T16:27:06.537 INFO:tasks.rados.rados.0.smithi100.stdout:3242: writing smithi10066867-24 from 9658368 to 10092544 tid 3 2024-04-07T16:27:06.541 INFO:tasks.rados.rados.0.smithi100.stdout:3242: writing smithi10066867-24 from 10092544 to 10731520 tid 4 2024-04-07T16:27:06.542 INFO:tasks.rados.rados.0.smithi100.stdout:3242: writing smithi10066867-24 from 10731520 to 10797056 tid 5 2024-04-07T16:27:06.542 INFO:tasks.rados.rados.0.smithi100.stdout:3243: snap_remove snap 286 2024-04-07T16:27:06.908 INFO:tasks.rados.rados.0.smithi100.stdout:3240: finishing write tid 1 to smithi10066867-34 2024-04-07T16:27:06.909 INFO:tasks.rados.rados.0.smithi100.stdout:3240: finishing write tid 2 to smithi10066867-34 2024-04-07T16:27:06.909 INFO:tasks.rados.rados.0.smithi100.stdout:3240: finishing write tid 3 to smithi10066867-34 2024-04-07T16:27:06.909 INFO:tasks.rados.rados.0.smithi100.stdout:3240: finishing write tid 4 to smithi10066867-34 2024-04-07T16:27:06.909 INFO:tasks.rados.rados.0.smithi100.stdout:3240: finishing write tid 5 to smithi10066867-34 2024-04-07T16:27:06.909 INFO:tasks.rados.rados.0.smithi100.stdout:3240: finishing write tid 6 to smithi10066867-34 2024-04-07T16:27:06.909 INFO:tasks.rados.rados.0.smithi100.stdout:3240: oid 34 updating version 0 to 1551 2024-04-07T16:27:06.909 INFO:tasks.rados.rados.0.smithi100.stdout:3240: oid 34 version 1551 is already newer than 1549 2024-04-07T16:27:06.909 INFO:tasks.rados.rados.0.smithi100.stdout:3240: oid 34 version 1551 is already newer than 1550 2024-04-07T16:27:06.909 INFO:tasks.rados.rados.0.smithi100.stdout:3240: oid 34 version 1551 is already newer than 1548 2024-04-07T16:27:06.909 INFO:tasks.rados.rados.0.smithi100.stdout:3240: oid 34 version 1551 is already newer than 1547 2024-04-07T16:27:06.909 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 34 v 1551 (ObjNum 988 snap 298 seq_num 988) dirty exists 2024-04-07T16:27:06.909 INFO:tasks.rados.rados.0.smithi100.stdout:3240: left oid 34 (ObjNum 988 snap 298 seq_num 988) 2024-04-07T16:27:06.909 INFO:tasks.rados.rados.0.smithi100.stdout:3241: expect (ObjNum 979 snap 296 seq_num 979) 2024-04-07T16:27:06.976 INFO:tasks.rados.rados.0.smithi100.stdout:3242: finishing write tid 1 to smithi10066867-24 2024-04-07T16:27:06.977 INFO:tasks.rados.rados.0.smithi100.stdout:3242: finishing write tid 2 to smithi10066867-24 2024-04-07T16:27:06.977 INFO:tasks.rados.rados.0.smithi100.stdout:3240: done (3 left) 2024-04-07T16:27:06.977 INFO:tasks.rados.rados.0.smithi100.stdout:3241: done (2 left) 2024-04-07T16:27:06.977 INFO:tasks.rados.rados.0.smithi100.stdout:3243: done (1 left) 2024-04-07T16:27:06.977 INFO:tasks.rados.rados.0.smithi100.stdout:3244: copy_from oid 38 from oid 18 current snap is 298 2024-04-07T16:27:06.977 INFO:tasks.rados.rados.0.smithi100.stdout:3242: finishing write tid 3 to smithi10066867-24 2024-04-07T16:27:06.977 INFO:tasks.rados.rados.0.smithi100.stdout:3242: finishing write tid 4 to smithi10066867-24 2024-04-07T16:27:06.977 INFO:tasks.rados.rados.0.smithi100.stdout:3242: finishing write tid 5 to smithi10066867-24 2024-04-07T16:27:06.977 INFO:tasks.rados.rados.0.smithi100.stdout:3242: finishing write tid 6 to smithi10066867-24 2024-04-07T16:27:06.977 INFO:tasks.rados.rados.0.smithi100.stdout:3242: finishing write tid 7 to smithi10066867-24 2024-04-07T16:27:06.977 INFO:tasks.rados.rados.0.smithi100.stdout:3242: oid 24 updating version 0 to 1257 2024-04-07T16:27:06.977 INFO:tasks.rados.rados.0.smithi100.stdout:3242: oid 24 updating version 1257 to 1260 2024-04-07T16:27:06.978 INFO:tasks.rados.rados.0.smithi100.stdout:3242: oid 24 version 1260 is already newer than 1256 2024-04-07T16:27:06.978 INFO:tasks.rados.rados.0.smithi100.stdout:3242: oid 24 version 1260 is already newer than 1255 2024-04-07T16:27:06.978 INFO:tasks.rados.rados.0.smithi100.stdout:3242: oid 24 version 1260 is already newer than 1258 2024-04-07T16:27:06.978 INFO:tasks.rados.rados.0.smithi100.stdout:3242: oid 24 version 1260 is already newer than 1259 2024-04-07T16:27:06.978 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 24 v 1260 (ObjNum 989 snap 298 seq_num 989) dirty exists 2024-04-07T16:27:06.978 INFO:tasks.rados.rados.0.smithi100.stdout:3242: left oid 24 (ObjNum 989 snap 298 seq_num 989) 2024-04-07T16:27:06.978 INFO:tasks.rados.rados.0.smithi100.stdout:3242: done (1 left) 2024-04-07T16:27:06.978 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 11 current snap is 298 2024-04-07T16:27:06.978 INFO:tasks.rados.rados.0.smithi100.stdout:3245: seq_num 990 ranges {5636096=589824,6225920=753664,6979584=638976,7618560=212992} 2024-04-07T16:27:07.011 INFO:tasks.rados.rados.0.smithi100.stdout:3245: writing smithi10066867-11 from 5636096 to 6225920 tid 1 2024-04-07T16:27:07.015 INFO:tasks.rados.rados.0.smithi100.stdout:3245: writing smithi10066867-11 from 6225920 to 6979584 tid 2 2024-04-07T16:27:07.019 INFO:tasks.rados.rados.0.smithi100.stdout:3245: writing smithi10066867-11 from 6979584 to 7618560 tid 3 2024-04-07T16:27:07.021 INFO:tasks.rados.rados.0.smithi100.stdout:3245: writing smithi10066867-11 from 7618560 to 7831552 tid 4 2024-04-07T16:27:07.021 INFO:tasks.rados.rados.0.smithi100.stdout:3246: rollback oid 17 current snap is 298 2024-04-07T16:27:07.021 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 17 to 296 2024-04-07T16:27:07.021 INFO:tasks.rados.rados.0.smithi100.stdout:3244: finishing copy_from racing read to smithi10066867-38 2024-04-07T16:27:07.021 INFO:tasks.rados.rados.0.smithi100.stdout:3247: rollback oid 46 current snap is 298 2024-04-07T16:27:07.021 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 46 to 297 2024-04-07T16:27:07.021 INFO:tasks.rados.rados.0.smithi100.stdout:3244: finishing copy_from to smithi10066867-38 2024-04-07T16:27:07.021 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 38 v 1686 (ObjNum 941 snap 282 seq_num 941) dirty exists 2024-04-07T16:27:07.021 INFO:tasks.rados.rados.0.smithi100.stdout:3245: finishing write tid 1 to smithi10066867-11 2024-04-07T16:27:07.022 INFO:tasks.rados.rados.0.smithi100.stdout:3245: finishing write tid 2 to smithi10066867-11 2024-04-07T16:27:07.022 INFO:tasks.rados.rados.0.smithi100.stdout:3244: done (3 left) 2024-04-07T16:27:07.022 INFO:tasks.rados.rados.0.smithi100.stdout:3248: snap_create 2024-04-07T16:27:07.025 INFO:tasks.rados.rados.0.smithi100.stdout:3246: finishing rollback tid 1 to smithi10066867-17 2024-04-07T16:27:07.026 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 17 v 1254 (ObjNum 65 snap 0 seq_num 2702049168) dirty dne 2024-04-07T16:27:07.026 INFO:tasks.rados.rados.0.smithi100.stdout:3247: finishing rollback tid 0 to smithi10066867-46 2024-04-07T16:27:07.026 INFO:tasks.rados.rados.0.smithi100.stdout:3245: finishing write tid 3 to smithi10066867-11 2024-04-07T16:27:07.026 INFO:tasks.rados.rados.0.smithi100.stdout:3247: finishing rollback tid 1 to smithi10066867-46 2024-04-07T16:27:07.027 INFO:tasks.rados.rados.0.smithi100.stdout:3245: finishing write tid 4 to smithi10066867-11 2024-04-07T16:27:07.027 INFO:tasks.rados.rados.0.smithi100.stdout:3247: finishing rollback tid 2 to smithi10066867-46 2024-04-07T16:27:07.027 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 46 v 1554 (ObjNum 982 snap 297 seq_num 982) dirty exists 2024-04-07T16:27:07.027 INFO:tasks.rados.rados.0.smithi100.stdout:3245: finishing write tid 5 to smithi10066867-11 2024-04-07T16:27:07.029 INFO:tasks.rados.rados.0.smithi100.stdout:3245: finishing write tid 6 to smithi10066867-11 2024-04-07T16:27:07.029 INFO:tasks.rados.rados.0.smithi100.stdout:3245: oid 11 updating version 0 to 1740 2024-04-07T16:27:07.029 INFO:tasks.rados.rados.0.smithi100.stdout:3245: oid 11 updating version 1740 to 1741 2024-04-07T16:27:07.029 INFO:tasks.rados.rados.0.smithi100.stdout:3245: oid 11 updating version 1741 to 1742 2024-04-07T16:27:07.029 INFO:tasks.rados.rados.0.smithi100.stdout:3245: oid 11 version 1742 is already newer than 1738 2024-04-07T16:27:07.029 INFO:tasks.rados.rados.0.smithi100.stdout:3245: oid 11 version 1742 is already newer than 1739 2024-04-07T16:27:07.030 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 11 v 1742 (ObjNum 990 snap 298 seq_num 990) dirty exists 2024-04-07T16:27:07.030 INFO:tasks.rados.rados.0.smithi100.stdout:3245: left oid 11 (ObjNum 990 snap 298 seq_num 990) 2024-04-07T16:27:07.322 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:07 smithi144 bash[66654]: cluster 2024-04-07T16:27:06.088195+0000 mon.a (mon.0) 1092 : cluster [DBG] osdmap e670: 8 total, 8 up, 8 in 2024-04-07T16:27:07.322 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:07 smithi144 bash[66654]: cluster 2024-04-07T16:27:06.922874+0000 mon.a (mon.0) 1093 : cluster [DBG] osdmap e671: 8 total, 8 up, 8 in 2024-04-07T16:27:07.436 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:07 smithi100 bash[74942]: cluster 2024-04-07T16:27:06.088195+0000 mon.a (mon.0) 1092 : cluster [DBG] osdmap e670: 8 total, 8 up, 8 in 2024-04-07T16:27:07.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:07 smithi100 bash[74942]: cluster 2024-04-07T16:27:06.922874+0000 mon.a (mon.0) 1093 : cluster [DBG] osdmap e671: 8 total, 8 up, 8 in 2024-04-07T16:27:07.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:07 smithi100 bash[76202]: cluster 2024-04-07T16:27:06.088195+0000 mon.a (mon.0) 1092 : cluster [DBG] osdmap e670: 8 total, 8 up, 8 in 2024-04-07T16:27:07.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:07 smithi100 bash[76202]: cluster 2024-04-07T16:27:06.922874+0000 mon.a (mon.0) 1093 : cluster [DBG] osdmap e671: 8 total, 8 up, 8 in 2024-04-07T16:27:07.911 INFO:tasks.rados.rados.0.smithi100.stdout:3245: done (3 left) 2024-04-07T16:27:07.911 INFO:tasks.rados.rados.0.smithi100.stdout:3246: done (2 left) 2024-04-07T16:27:07.911 INFO:tasks.rados.rados.0.smithi100.stdout:3247: done (1 left) 2024-04-07T16:27:07.911 INFO:tasks.rados.rados.0.smithi100.stdout:3248: done (0 left) 2024-04-07T16:27:07.911 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 30 current snap is 299 2024-04-07T16:27:07.911 INFO:tasks.rados.rados.0.smithi100.stdout:3249: seq_num 991 ranges {344064=606208,950272=638976,1589248=794624,2383872=172032} 2024-04-07T16:27:07.917 INFO:tasks.rados.rados.0.smithi100.stdout:3249: writing smithi10066867-30 from 344064 to 950272 tid 1 2024-04-07T16:27:07.922 INFO:tasks.rados.rados.0.smithi100.stdout:3249: writing smithi10066867-30 from 950272 to 1589248 tid 2 2024-04-07T16:27:07.928 INFO:tasks.rados.rados.0.smithi100.stdout:3249: writing smithi10066867-30 from 1589248 to 2383872 tid 3 2024-04-07T16:27:07.929 INFO:tasks.rados.rados.0.smithi100.stdout:3249: writing smithi10066867-30 from 2383872 to 2555904 tid 4 2024-04-07T16:27:07.930 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 38 current snap is 299 2024-04-07T16:27:07.930 INFO:tasks.rados.rados.0.smithi100.stdout:3250: seq_num 992 ranges {1368064=622592,1990656=524288,2514944=606208,3121152=475136} 2024-04-07T16:27:07.944 INFO:tasks.rados.rados.0.smithi100.stdout:3250: writing smithi10066867-38 from 1368064 to 1990656 tid 1 2024-04-07T16:27:07.948 INFO:tasks.rados.rados.0.smithi100.stdout:3250: writing smithi10066867-38 from 1990656 to 2514944 tid 2 2024-04-07T16:27:07.952 INFO:tasks.rados.rados.0.smithi100.stdout:3250: writing smithi10066867-38 from 2514944 to 3121152 tid 3 2024-04-07T16:27:07.955 INFO:tasks.rados.rados.0.smithi100.stdout:3250: writing smithi10066867-38 from 3121152 to 3596288 tid 4 2024-04-07T16:27:07.956 INFO:tasks.rados.rados.0.smithi100.stdout:3251: snap_create 2024-04-07T16:27:07.956 INFO:tasks.rados.rados.0.smithi100.stdout:3249: finishing write tid 1 to smithi10066867-30 2024-04-07T16:27:07.956 INFO:tasks.rados.rados.0.smithi100.stdout:3249: finishing write tid 2 to smithi10066867-30 2024-04-07T16:27:07.956 INFO:tasks.rados.rados.0.smithi100.stdout:3249: finishing write tid 3 to smithi10066867-30 2024-04-07T16:27:07.956 INFO:tasks.rados.rados.0.smithi100.stdout:3249: finishing write tid 4 to smithi10066867-30 2024-04-07T16:27:07.956 INFO:tasks.rados.rados.0.smithi100.stdout:3249: finishing write tid 5 to smithi10066867-30 2024-04-07T16:27:07.956 INFO:tasks.rados.rados.0.smithi100.stdout:3249: finishing write tid 6 to smithi10066867-30 2024-04-07T16:27:07.956 INFO:tasks.rados.rados.0.smithi100.stdout:3249: oid 30 updating version 0 to 1280 2024-04-07T16:27:07.956 INFO:tasks.rados.rados.0.smithi100.stdout:3249: oid 30 updating version 1280 to 1281 2024-04-07T16:27:07.956 INFO:tasks.rados.rados.0.smithi100.stdout:3249: oid 30 version 1281 is already newer than 1279 2024-04-07T16:27:07.956 INFO:tasks.rados.rados.0.smithi100.stdout:3249: oid 30 version 1281 is already newer than 1277 2024-04-07T16:27:07.956 INFO:tasks.rados.rados.0.smithi100.stdout:3249: oid 30 version 1281 is already newer than 1278 2024-04-07T16:27:07.956 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 30 v 1281 (ObjNum 991 snap 299 seq_num 991) dirty exists 2024-04-07T16:27:07.957 INFO:tasks.rados.rados.0.smithi100.stdout:3249: left oid 30 (ObjNum 991 snap 299 seq_num 991) 2024-04-07T16:27:07.957 INFO:tasks.rados.rados.0.smithi100.stdout:3250: finishing write tid 1 to smithi10066867-38 2024-04-07T16:27:07.957 INFO:tasks.rados.rados.0.smithi100.stdout:3250: finishing write tid 2 to smithi10066867-38 2024-04-07T16:27:07.960 INFO:tasks.rados.rados.0.smithi100.stdout:3250: finishing write tid 3 to smithi10066867-38 2024-04-07T16:27:07.961 INFO:tasks.rados.rados.0.smithi100.stdout:3250: finishing write tid 4 to smithi10066867-38 2024-04-07T16:27:07.961 INFO:tasks.rados.rados.0.smithi100.stdout:3250: finishing write tid 5 to smithi10066867-38 2024-04-07T16:27:07.964 INFO:tasks.rados.rados.0.smithi100.stdout:3250: finishing write tid 6 to smithi10066867-38 2024-04-07T16:27:07.965 INFO:tasks.rados.rados.0.smithi100.stdout:3250: oid 38 updating version 0 to 1687 2024-04-07T16:27:07.965 INFO:tasks.rados.rados.0.smithi100.stdout:3250: oid 38 updating version 1687 to 1688 2024-04-07T16:27:07.965 INFO:tasks.rados.rados.0.smithi100.stdout:3250: oid 38 updating version 1688 to 1689 2024-04-07T16:27:07.965 INFO:tasks.rados.rados.0.smithi100.stdout:3250: oid 38 updating version 1689 to 1691 2024-04-07T16:27:07.965 INFO:tasks.rados.rados.0.smithi100.stdout:3250: oid 38 version 1691 is already newer than 1690 2024-04-07T16:27:07.965 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 38 v 1691 (ObjNum 992 snap 299 seq_num 992) dirty exists 2024-04-07T16:27:07.965 INFO:tasks.rados.rados.0.smithi100.stdout:3250: left oid 38 (ObjNum 992 snap 299 seq_num 992) 2024-04-07T16:27:08.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:08 smithi144 bash[66654]: cluster 2024-04-07T16:27:06.783230+0000 mgr.y (mgr.25175) 333 : cluster [DBG] pgmap v598: 121 pgs: 121 active+clean; 141 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 6.4 MiB/s rd, 4.3 MiB/s wr, 18 op/s 2024-04-07T16:27:08.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:08 smithi144 bash[66654]: audit 2024-04-07T16:27:07.778994+0000 mon.a (mon.0) 1094 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:27:08.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:08 smithi144 bash[66654]: audit 2024-04-07T16:27:07.789614+0000 mon.a (mon.0) 1095 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:27:08.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:08 smithi144 bash[66654]: cluster 2024-04-07T16:27:07.925288+0000 mon.a (mon.0) 1096 : cluster [DBG] osdmap e672: 8 total, 8 up, 8 in 2024-04-07T16:27:08.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:08 smithi144 bash[66654]: audit 2024-04-07T16:27:08.053820+0000 mon.a (mon.0) 1097 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:27:08.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:08 smithi144 bash[66654]: audit 2024-04-07T16:27:08.065599+0000 mon.a (mon.0) 1098 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:27:08.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:08 smithi144 bash[66654]: audit 2024-04-07T16:27:08.068150+0000 mon.a (mon.0) 1099 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:27:08.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:08 smithi144 bash[66654]: audit 2024-04-07T16:27:08.070161+0000 mon.a (mon.0) 1100 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:27:08.417 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:08 smithi100 bash[74942]: cluster 2024-04-07T16:27:06.783230+0000 mgr.y (mgr.25175) 333 : cluster [DBG] pgmap v598: 121 pgs: 121 active+clean; 141 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 6.4 MiB/s rd, 4.3 MiB/s wr, 18 op/s 2024-04-07T16:27:08.417 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:08 smithi100 bash[74942]: audit 2024-04-07T16:27:07.778994+0000 mon.a (mon.0) 1094 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:27:08.417 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:08 smithi100 bash[74942]: audit 2024-04-07T16:27:07.789614+0000 mon.a (mon.0) 1095 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:27:08.417 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:08 smithi100 bash[74942]: cluster 2024-04-07T16:27:07.925288+0000 mon.a (mon.0) 1096 : cluster [DBG] osdmap e672: 8 total, 8 up, 8 in 2024-04-07T16:27:08.417 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:08 smithi100 bash[74942]: audit 2024-04-07T16:27:08.053820+0000 mon.a (mon.0) 1097 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:27:08.417 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:08 smithi100 bash[74942]: audit 2024-04-07T16:27:08.065599+0000 mon.a (mon.0) 1098 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:27:08.417 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:08 smithi100 bash[74942]: audit 2024-04-07T16:27:08.068150+0000 mon.a (mon.0) 1099 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:27:08.417 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:08 smithi100 bash[74942]: audit 2024-04-07T16:27:08.070161+0000 mon.a (mon.0) 1100 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:27:08.418 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:08 smithi100 bash[76202]: cluster 2024-04-07T16:27:06.783230+0000 mgr.y (mgr.25175) 333 : cluster [DBG] pgmap v598: 121 pgs: 121 active+clean; 141 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 6.4 MiB/s rd, 4.3 MiB/s wr, 18 op/s 2024-04-07T16:27:08.418 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:08 smithi100 bash[76202]: audit 2024-04-07T16:27:07.778994+0000 mon.a (mon.0) 1094 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:27:08.418 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:08 smithi100 bash[76202]: audit 2024-04-07T16:27:07.789614+0000 mon.a (mon.0) 1095 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:27:08.418 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:08 smithi100 bash[76202]: cluster 2024-04-07T16:27:07.925288+0000 mon.a (mon.0) 1096 : cluster [DBG] osdmap e672: 8 total, 8 up, 8 in 2024-04-07T16:27:08.418 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:08 smithi100 bash[76202]: audit 2024-04-07T16:27:08.053820+0000 mon.a (mon.0) 1097 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:27:08.418 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:08 smithi100 bash[76202]: audit 2024-04-07T16:27:08.065599+0000 mon.a (mon.0) 1098 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:27:08.418 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:08 smithi100 bash[76202]: audit 2024-04-07T16:27:08.068150+0000 mon.a (mon.0) 1099 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:27:08.418 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:08 smithi100 bash[76202]: audit 2024-04-07T16:27:08.070161+0000 mon.a (mon.0) 1100 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:27:08.686 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:27:08 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:27:08] "GET /metrics HTTP/1.1" 200 36332 "" "Prometheus/2.43.0" 2024-04-07T16:27:08.914 INFO:tasks.rados.rados.0.smithi100.stdout:3249: done (2 left) 2024-04-07T16:27:08.914 INFO:tasks.rados.rados.0.smithi100.stdout:3250: done (1 left) 2024-04-07T16:27:08.914 INFO:tasks.rados.rados.0.smithi100.stdout:3251: done (0 left) 2024-04-07T16:27:08.914 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 31 current snap is 300 2024-04-07T16:27:08.914 INFO:tasks.rados.rados.0.smithi100.stdout:3252: seq_num 993 ranges {614400=638976,1253376=409600,1662976=409600,2072576=688128,2760704=98304} 2024-04-07T16:27:08.923 INFO:tasks.rados.rados.0.smithi100.stdout:3252: writing smithi10066867-31 from 614400 to 1253376 tid 1 2024-04-07T16:27:08.925 INFO:tasks.rados.rados.0.smithi100.stdout:3252: writing smithi10066867-31 from 1253376 to 1662976 tid 2 2024-04-07T16:27:08.929 INFO:tasks.rados.rados.0.smithi100.stdout:3252: writing smithi10066867-31 from 1662976 to 2072576 tid 3 2024-04-07T16:27:08.936 INFO:tasks.rados.rados.0.smithi100.stdout:3252: writing smithi10066867-31 from 2072576 to 2760704 tid 4 2024-04-07T16:27:08.937 INFO:tasks.rados.rados.0.smithi100.stdout:3252: writing smithi10066867-31 from 2760704 to 2859008 tid 5 2024-04-07T16:27:08.937 INFO:tasks.rados.rados.0.smithi100.stdout:3253: rmattr oid 35 current snap is 300 2024-04-07T16:27:08.937 INFO:tasks.rados.rados.0.smithi100.stdout:3253: done (1 left) 2024-04-07T16:27:08.937 INFO:tasks.rados.rados.0.smithi100.stdout:3254: read oid 29 snap 296 2024-04-07T16:27:08.937 INFO:tasks.rados.rados.0.smithi100.stdout:3254: expect deleted 2024-04-07T16:27:08.938 INFO:tasks.rados.rados.0.smithi100.stdout:3255: copy_from oid 11 from oid 13 current snap is 300 2024-04-07T16:27:08.938 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 46 current snap is 300 2024-04-07T16:27:08.938 INFO:tasks.rados.rados.0.smithi100.stdout:3256: seq_num 994 ranges {5234688=655360,5890048=688128,6578176=622592,7200768=294912} 2024-04-07T16:27:08.976 INFO:tasks.rados.rados.0.smithi100.stdout:3256: writing smithi10066867-46 from 5234688 to 5890048 tid 1 2024-04-07T16:27:08.981 INFO:tasks.rados.rados.0.smithi100.stdout:3256: writing smithi10066867-46 from 5890048 to 6578176 tid 2 2024-04-07T16:27:08.985 INFO:tasks.rados.rados.0.smithi100.stdout:3256: writing smithi10066867-46 from 6578176 to 7200768 tid 3 2024-04-07T16:27:08.988 INFO:tasks.rados.rados.0.smithi100.stdout:3256: writing smithi10066867-46 from 7200768 to 7495680 tid 4 2024-04-07T16:27:08.988 INFO:tasks.rados.rados.0.smithi100.stdout:3257: delete oid 18 current snap is 300 2024-04-07T16:27:08.988 INFO:tasks.rados.rados.0.smithi100.stdout:3255: finishing copy_from to smithi10066867-11 2024-04-07T16:27:08.988 INFO:tasks.rados.rados.0.smithi100.stdout:3255: got expected ENOENT (src dne) 2024-04-07T16:27:08.988 INFO:tasks.rados.rados.0.smithi100.stdout:3255: finishing copy_from racing read to smithi10066867-11 2024-04-07T16:27:08.988 INFO:tasks.rados.rados.0.smithi100.stdout:3252: finishing write tid 1 to smithi10066867-31 2024-04-07T16:27:08.988 INFO:tasks.rados.rados.0.smithi100.stdout:3252: finishing write tid 2 to smithi10066867-31 2024-04-07T16:27:08.988 INFO:tasks.rados.rados.0.smithi100.stdout:3252: finishing write tid 3 to smithi10066867-31 2024-04-07T16:27:08.988 INFO:tasks.rados.rados.0.smithi100.stdout:3252: finishing write tid 4 to smithi10066867-31 2024-04-07T16:27:08.988 INFO:tasks.rados.rados.0.smithi100.stdout:3252: finishing write tid 5 to smithi10066867-31 2024-04-07T16:27:08.988 INFO:tasks.rados.rados.0.smithi100.stdout:3252: finishing write tid 6 to smithi10066867-31 2024-04-07T16:27:08.989 INFO:tasks.rados.rados.0.smithi100.stdout:3252: finishing write tid 7 to smithi10066867-31 2024-04-07T16:27:08.989 INFO:tasks.rados.rados.0.smithi100.stdout:3252: oid 31 updating version 0 to 1688 2024-04-07T16:27:08.989 INFO:tasks.rados.rados.0.smithi100.stdout:3252: oid 31 updating version 1688 to 1689 2024-04-07T16:27:08.989 INFO:tasks.rados.rados.0.smithi100.stdout:3252: oid 31 version 1689 is already newer than 1687 2024-04-07T16:27:08.989 INFO:tasks.rados.rados.0.smithi100.stdout:3252: oid 31 updating version 1689 to 1692 2024-04-07T16:27:08.989 INFO:tasks.rados.rados.0.smithi100.stdout:3252: oid 31 version 1692 is already newer than 1690 2024-04-07T16:27:08.989 INFO:tasks.rados.rados.0.smithi100.stdout:3252: oid 31 version 1692 is already newer than 1691 2024-04-07T16:27:08.989 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 31 v 1692 (ObjNum 993 snap 300 seq_num 993) dirty exists 2024-04-07T16:27:08.989 INFO:tasks.rados.rados.0.smithi100.stdout:3252: left oid 31 (ObjNum 993 snap 300 seq_num 993) 2024-04-07T16:27:08.989 INFO:tasks.rados.rados.0.smithi100.stdout:3256: finishing write tid 1 to smithi10066867-46 2024-04-07T16:27:08.992 INFO:tasks.rados.rados.0.smithi100.stdout:3252: done (4 left) 2024-04-07T16:27:08.992 INFO:tasks.rados.rados.0.smithi100.stdout:3254: done (3 left) 2024-04-07T16:27:08.992 INFO:tasks.rados.rados.0.smithi100.stdout:3255: done (2 left) 2024-04-07T16:27:08.992 INFO:tasks.rados.rados.0.smithi100.stdout:3257: done (1 left) 2024-04-07T16:27:08.992 INFO:tasks.rados.rados.0.smithi100.stdout:3258: read oid 33 snap -1 2024-04-07T16:27:08.993 INFO:tasks.rados.rados.0.smithi100.stdout:3258: expect (ObjNum 753 snap 235 seq_num 753) 2024-04-07T16:27:08.993 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 47 current snap is 300 2024-04-07T16:27:08.993 INFO:tasks.rados.rados.0.smithi100.stdout:3259: seq_num 996 ranges {0=688128,688128=507904,1196032=581632,1777664=516096} 2024-04-07T16:27:08.996 INFO:tasks.rados.rados.0.smithi100.stdout:3259: writing smithi10066867-47 from 0 to 688128 tid 1 2024-04-07T16:27:09.000 INFO:tasks.rados.rados.0.smithi100.stdout:3259: writing smithi10066867-47 from 688128 to 1196032 tid 2 2024-04-07T16:27:09.003 INFO:tasks.rados.rados.0.smithi100.stdout:3259: writing smithi10066867-47 from 1196032 to 1777664 tid 3 2024-04-07T16:27:09.007 INFO:tasks.rados.rados.0.smithi100.stdout:3259: writing smithi10066867-47 from 1777664 to 2293760 tid 4 2024-04-07T16:27:09.007 INFO:tasks.rados.rados.0.smithi100.stdout:3260: read oid 7 snap -1 2024-04-07T16:27:09.007 INFO:tasks.rados.rados.0.smithi100.stdout:3260: expect (ObjNum 935 snap 280 seq_num 935) 2024-04-07T16:27:09.007 INFO:tasks.rados.rados.0.smithi100.stdout:3261: delete oid 42 current snap is 300 2024-04-07T16:27:09.007 INFO:tasks.rados.rados.0.smithi100.stdout:3256: finishing write tid 2 to smithi10066867-46 2024-04-07T16:27:09.007 INFO:tasks.rados.rados.0.smithi100.stdout:3256: finishing write tid 3 to smithi10066867-46 2024-04-07T16:27:09.007 INFO:tasks.rados.rados.0.smithi100.stdout:3256: finishing write tid 4 to smithi10066867-46 2024-04-07T16:27:09.007 INFO:tasks.rados.rados.0.smithi100.stdout:3258: expect (ObjNum 753 snap 235 seq_num 753) 2024-04-07T16:27:09.036 INFO:tasks.rados.rados.0.smithi100.stdout:3256: finishing write tid 5 to smithi10066867-46 2024-04-07T16:27:09.037 INFO:tasks.rados.rados.0.smithi100.stdout:3256: finishing write tid 6 to smithi10066867-46 2024-04-07T16:27:09.037 INFO:tasks.rados.rados.0.smithi100.stdout:3256: oid 46 updating version 0 to 1557 2024-04-07T16:27:09.037 INFO:tasks.rados.rados.0.smithi100.stdout:3256: oid 46 updating version 1557 to 1559 2024-04-07T16:27:09.037 INFO:tasks.rados.rados.0.smithi100.stdout:3256: oid 46 version 1559 is already newer than 1555 2024-04-07T16:27:09.037 INFO:tasks.rados.rados.0.smithi100.stdout:3256: oid 46 version 1559 is already newer than 1556 2024-04-07T16:27:09.037 INFO:tasks.rados.rados.0.smithi100.stdout:3256: oid 46 version 1559 is already newer than 1558 2024-04-07T16:27:09.037 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 46 v 1559 (ObjNum 994 snap 300 seq_num 994) dirty exists 2024-04-07T16:27:09.037 INFO:tasks.rados.rados.0.smithi100.stdout:3256: left oid 46 (ObjNum 994 snap 300 seq_num 994) 2024-04-07T16:27:09.037 INFO:tasks.rados.rados.0.smithi100.stdout:3259: finishing write tid 1 to smithi10066867-47 2024-04-07T16:27:09.037 INFO:tasks.rados.rados.0.smithi100.stdout:3259: finishing write tid 2 to smithi10066867-47 2024-04-07T16:27:09.037 INFO:tasks.rados.rados.0.smithi100.stdout:3259: finishing write tid 3 to smithi10066867-47 2024-04-07T16:27:09.037 INFO:tasks.rados.rados.0.smithi100.stdout:3259: finishing write tid 4 to smithi10066867-47 2024-04-07T16:27:09.038 INFO:tasks.rados.rados.0.smithi100.stdout:3259: finishing write tid 5 to smithi10066867-47 2024-04-07T16:27:09.038 INFO:tasks.rados.rados.0.smithi100.stdout:3259: finishing write tid 6 to smithi10066867-47 2024-04-07T16:27:09.038 INFO:tasks.rados.rados.0.smithi100.stdout:3259: oid 47 updating version 0 to 1199 2024-04-07T16:27:09.038 INFO:tasks.rados.rados.0.smithi100.stdout:3259: oid 47 updating version 1199 to 1200 2024-04-07T16:27:09.038 INFO:tasks.rados.rados.0.smithi100.stdout:3259: oid 47 updating version 1200 to 1201 2024-04-07T16:27:09.038 INFO:tasks.rados.rados.0.smithi100.stdout:3259: oid 47 updating version 1201 to 1202 2024-04-07T16:27:09.038 INFO:tasks.rados.rados.0.smithi100.stdout:3259: oid 47 updating version 1202 to 1203 2024-04-07T16:27:09.038 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 47 v 1203 (ObjNum 996 snap 300 seq_num 996) dirty exists 2024-04-07T16:27:09.038 INFO:tasks.rados.rados.0.smithi100.stdout:3259: left oid 47 (ObjNum 996 snap 300 seq_num 996) 2024-04-07T16:27:09.038 INFO:tasks.rados.rados.0.smithi100.stdout:3256: done (4 left) 2024-04-07T16:27:09.038 INFO:tasks.rados.rados.0.smithi100.stdout:3258: done (3 left) 2024-04-07T16:27:09.038 INFO:tasks.rados.rados.0.smithi100.stdout:3259: done (2 left) 2024-04-07T16:27:09.039 INFO:tasks.rados.rados.0.smithi100.stdout:3261: done (1 left) 2024-04-07T16:27:09.039 INFO:tasks.rados.rados.0.smithi100.stdout:3262: read oid 9 snap 296 2024-04-07T16:27:09.039 INFO:tasks.rados.rados.0.smithi100.stdout:3262: expect deleted 2024-04-07T16:27:09.039 INFO:tasks.rados.rados.0.smithi100.stdout:3263: rmattr oid 47 current snap is 300 2024-04-07T16:27:09.039 INFO:tasks.rados.rados.0.smithi100.stdout:3260: expect (ObjNum 935 snap 280 seq_num 935) 2024-04-07T16:27:09.122 INFO:tasks.rados.rados.0.smithi100.stdout:3260: done (2 left) 2024-04-07T16:27:09.122 INFO:tasks.rados.rados.0.smithi100.stdout:3262: done (1 left) 2024-04-07T16:27:09.122 INFO:tasks.rados.rados.0.smithi100.stdout:3263: done (0 left) 2024-04-07T16:27:09.122 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 34 current snap is 300 2024-04-07T16:27:09.122 INFO:tasks.rados.rados.0.smithi100.stdout:3264: seq_num 998 ranges {3883008=720896,4603904=679936,5283840=598016,5881856=327680} 2024-04-07T16:27:09.154 INFO:tasks.rados.rados.0.smithi100.stdout:3264: writing smithi10066867-34 from 3883008 to 4603904 tid 1 2024-04-07T16:27:09.159 INFO:tasks.rados.rados.0.smithi100.stdout:3264: writing smithi10066867-34 from 4603904 to 5283840 tid 2 2024-04-07T16:27:09.164 INFO:tasks.rados.rados.0.smithi100.stdout:3264: writing smithi10066867-34 from 5283840 to 5881856 tid 3 2024-04-07T16:27:09.167 INFO:tasks.rados.rados.0.smithi100.stdout:3264: writing smithi10066867-34 from 5881856 to 6209536 tid 4 2024-04-07T16:27:09.167 INFO:tasks.rados.rados.0.smithi100.stdout:3265: read oid 24 snap -1 2024-04-07T16:27:09.167 INFO:tasks.rados.rados.0.smithi100.stdout:3265: expect (ObjNum 989 snap 298 seq_num 989) 2024-04-07T16:27:09.167 INFO:tasks.rados.rados.0.smithi100.stdout:3266: snap_remove snap 298 2024-04-07T16:27:09.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:09 smithi144 bash[66654]: audit 2024-04-07T16:27:08.083981+0000 mon.a (mon.0) 1101 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:27:09.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:09 smithi144 bash[66654]: audit 2024-04-07T16:27:08.169219+0000 mon.a (mon.0) 1102 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:27:09.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:09 smithi144 bash[66654]: audit 2024-04-07T16:27:08.172219+0000 mon.a (mon.0) 1103 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:27:09.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:09 smithi144 bash[66654]: audit 2024-04-07T16:27:08.174639+0000 mon.a (mon.0) 1104 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:27:09.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:09 smithi144 bash[66654]: audit 2024-04-07T16:27:08.176427+0000 mon.a (mon.0) 1105 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:27:09.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:09 smithi144 bash[66654]: audit 2024-04-07T16:27:08.179471+0000 mon.a (mon.0) 1106 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:27:09.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:09 smithi144 bash[66654]: audit 2024-04-07T16:27:08.184848+0000 mon.a (mon.0) 1107 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:27:09.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:09 smithi144 bash[66654]: audit 2024-04-07T16:27:08.187101+0000 mon.a (mon.0) 1108 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:27:09.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:09 smithi144 bash[66654]: audit 2024-04-07T16:27:08.189148+0000 mon.a (mon.0) 1109 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:27:09.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:09 smithi144 bash[66654]: audit 2024-04-07T16:27:08.191285+0000 mon.a (mon.0) 1110 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:27:09.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:09 smithi144 bash[66654]: audit 2024-04-07T16:27:08.193524+0000 mon.a (mon.0) 1111 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:27:09.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:09 smithi144 bash[66654]: audit 2024-04-07T16:27:08.195780+0000 mon.a (mon.0) 1112 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:27:09.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:09 smithi144 bash[66654]: audit 2024-04-07T16:27:08.197855+0000 mon.a (mon.0) 1113 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:27:09.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:09 smithi144 bash[66654]: audit 2024-04-07T16:27:08.200215+0000 mon.a (mon.0) 1114 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:27:09.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:09 smithi144 bash[66654]: audit 2024-04-07T16:27:08.203352+0000 mon.a (mon.0) 1115 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:27:09.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:09 smithi144 bash[66654]: cluster 2024-04-07T16:27:08.928420+0000 mon.a (mon.0) 1116 : cluster [DBG] osdmap e673: 8 total, 8 up, 8 in 2024-04-07T16:27:09.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:09 smithi100 bash[74942]: audit 2024-04-07T16:27:08.083981+0000 mon.a (mon.0) 1101 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:27:09.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:09 smithi100 bash[74942]: audit 2024-04-07T16:27:08.169219+0000 mon.a (mon.0) 1102 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:27:09.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:09 smithi100 bash[74942]: audit 2024-04-07T16:27:08.172219+0000 mon.a (mon.0) 1103 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:27:09.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:09 smithi100 bash[74942]: audit 2024-04-07T16:27:08.174639+0000 mon.a (mon.0) 1104 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:27:09.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:09 smithi100 bash[74942]: audit 2024-04-07T16:27:08.176427+0000 mon.a (mon.0) 1105 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:27:09.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:09 smithi100 bash[74942]: audit 2024-04-07T16:27:08.179471+0000 mon.a (mon.0) 1106 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:27:09.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:09 smithi100 bash[74942]: audit 2024-04-07T16:27:08.184848+0000 mon.a (mon.0) 1107 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:27:09.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:09 smithi100 bash[74942]: audit 2024-04-07T16:27:08.187101+0000 mon.a (mon.0) 1108 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:27:09.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:09 smithi100 bash[74942]: audit 2024-04-07T16:27:08.189148+0000 mon.a (mon.0) 1109 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:27:09.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:09 smithi100 bash[74942]: audit 2024-04-07T16:27:08.191285+0000 mon.a (mon.0) 1110 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:27:09.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:09 smithi100 bash[74942]: audit 2024-04-07T16:27:08.193524+0000 mon.a (mon.0) 1111 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:27:09.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:09 smithi100 bash[74942]: audit 2024-04-07T16:27:08.195780+0000 mon.a (mon.0) 1112 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:27:09.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:09 smithi100 bash[74942]: audit 2024-04-07T16:27:08.197855+0000 mon.a (mon.0) 1113 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:27:09.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:09 smithi100 bash[74942]: audit 2024-04-07T16:27:08.200215+0000 mon.a (mon.0) 1114 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:27:09.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:09 smithi100 bash[74942]: audit 2024-04-07T16:27:08.203352+0000 mon.a (mon.0) 1115 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:27:09.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:09 smithi100 bash[74942]: cluster 2024-04-07T16:27:08.928420+0000 mon.a (mon.0) 1116 : cluster [DBG] osdmap e673: 8 total, 8 up, 8 in 2024-04-07T16:27:09.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:09 smithi100 bash[76202]: audit 2024-04-07T16:27:08.083981+0000 mon.a (mon.0) 1101 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:27:09.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:09 smithi100 bash[76202]: audit 2024-04-07T16:27:08.169219+0000 mon.a (mon.0) 1102 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:27:09.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:09 smithi100 bash[76202]: audit 2024-04-07T16:27:08.172219+0000 mon.a (mon.0) 1103 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:27:09.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:09 smithi100 bash[76202]: audit 2024-04-07T16:27:08.174639+0000 mon.a (mon.0) 1104 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:27:09.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:09 smithi100 bash[76202]: audit 2024-04-07T16:27:08.176427+0000 mon.a (mon.0) 1105 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:27:09.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:09 smithi100 bash[76202]: audit 2024-04-07T16:27:08.179471+0000 mon.a (mon.0) 1106 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:27:09.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:09 smithi100 bash[76202]: audit 2024-04-07T16:27:08.184848+0000 mon.a (mon.0) 1107 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:27:09.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:09 smithi100 bash[76202]: audit 2024-04-07T16:27:08.187101+0000 mon.a (mon.0) 1108 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:27:09.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:09 smithi100 bash[76202]: audit 2024-04-07T16:27:08.189148+0000 mon.a (mon.0) 1109 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:27:09.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:09 smithi100 bash[76202]: audit 2024-04-07T16:27:08.191285+0000 mon.a (mon.0) 1110 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:27:09.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:09 smithi100 bash[76202]: audit 2024-04-07T16:27:08.193524+0000 mon.a (mon.0) 1111 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:27:09.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:09 smithi100 bash[76202]: audit 2024-04-07T16:27:08.195780+0000 mon.a (mon.0) 1112 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:27:09.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:09 smithi100 bash[76202]: audit 2024-04-07T16:27:08.197855+0000 mon.a (mon.0) 1113 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:27:09.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:09 smithi100 bash[76202]: audit 2024-04-07T16:27:08.200215+0000 mon.a (mon.0) 1114 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:27:09.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:09 smithi100 bash[76202]: audit 2024-04-07T16:27:08.203352+0000 mon.a (mon.0) 1115 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:27:09.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:09 smithi100 bash[76202]: cluster 2024-04-07T16:27:08.928420+0000 mon.a (mon.0) 1116 : cluster [DBG] osdmap e673: 8 total, 8 up, 8 in 2024-04-07T16:27:10.103 INFO:tasks.rados.rados.0.smithi100.stdout:3264: finishing write tid 1 to smithi10066867-34 2024-04-07T16:27:10.103 INFO:tasks.rados.rados.0.smithi100.stdout:3264: finishing write tid 2 to smithi10066867-34 2024-04-07T16:27:10.103 INFO:tasks.rados.rados.0.smithi100.stdout:3264: finishing write tid 3 to smithi10066867-34 2024-04-07T16:27:10.103 INFO:tasks.rados.rados.0.smithi100.stdout:3264: finishing write tid 4 to smithi10066867-34 2024-04-07T16:27:10.103 INFO:tasks.rados.rados.0.smithi100.stdout:3264: finishing write tid 5 to smithi10066867-34 2024-04-07T16:27:10.103 INFO:tasks.rados.rados.0.smithi100.stdout:3264: finishing write tid 6 to smithi10066867-34 2024-04-07T16:27:10.103 INFO:tasks.rados.rados.0.smithi100.stdout:3264: oid 34 updating version 0 to 1562 2024-04-07T16:27:10.103 INFO:tasks.rados.rados.0.smithi100.stdout:3264: oid 34 version 1562 is already newer than 1561 2024-04-07T16:27:10.104 INFO:tasks.rados.rados.0.smithi100.stdout:3264: oid 34 updating version 1562 to 1564 2024-04-07T16:27:10.104 INFO:tasks.rados.rados.0.smithi100.stdout:3264: oid 34 version 1564 is already newer than 1560 2024-04-07T16:27:10.104 INFO:tasks.rados.rados.0.smithi100.stdout:3264: oid 34 version 1564 is already newer than 1563 2024-04-07T16:27:10.104 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 34 v 1564 (ObjNum 998 snap 300 seq_num 998) dirty exists 2024-04-07T16:27:10.104 INFO:tasks.rados.rados.0.smithi100.stdout:3264: left oid 34 (ObjNum 998 snap 300 seq_num 998) 2024-04-07T16:27:10.104 INFO:tasks.rados.rados.0.smithi100.stdout:3265: expect (ObjNum 989 snap 298 seq_num 989) 2024-04-07T16:27:10.257 INFO:teuthology.orchestra.run.smithi100.stdout:true 2024-04-07T16:27:10.436 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:10 smithi100 bash[74942]: cluster 2024-04-07T16:27:08.784023+0000 mgr.y (mgr.25175) 334 : cluster [DBG] pgmap v601: 121 pgs: 121 active+clean; 131 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 2.2 MiB/s rd, 2.3 MiB/s wr, 7 op/s 2024-04-07T16:27:10.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:10 smithi100 bash[76202]: cluster 2024-04-07T16:27:08.784023+0000 mgr.y (mgr.25175) 334 : cluster [DBG] pgmap v601: 121 pgs: 121 active+clean; 131 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 2.2 MiB/s rd, 2.3 MiB/s wr, 7 op/s 2024-04-07T16:27:10.605 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:10 smithi144 bash[66654]: cluster 2024-04-07T16:27:08.784023+0000 mgr.y (mgr.25175) 334 : cluster [DBG] pgmap v601: 121 pgs: 121 active+clean; 131 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 2.2 MiB/s rd, 2.3 MiB/s wr, 7 op/s 2024-04-07T16:27:10.614 INFO:teuthology.orchestra.run.smithi100.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-07T16:27:10.615 INFO:teuthology.orchestra.run.smithi100.stdout:alertmanager.a smithi100 *:9093,9094 running (13m) 13s ago 13m 14.7M - 0.25.0 c8568f914cd2 5143075e7d26 2024-04-07T16:27:10.615 INFO:teuthology.orchestra.run.smithi100.stdout:grafana.a smithi144 *:3000 running (13m) 13s ago 13m 79.4M - 9.4.7 954c08fa6188 7180c0708acd 2024-04-07T16:27:10.615 INFO:teuthology.orchestra.run.smithi100.stdout:mds.a smithi100 running (71s) 13s ago 15m 13.4M - 19.0.0-2785-gadce92d0 c0ecd4226ac9 075552e67bd0 2024-04-07T16:27:10.615 INFO:teuthology.orchestra.run.smithi100.stdout:mds.b smithi144 running (44s) 13s ago 15m 11.6M - 19.0.0-2785-gadce92d0 c0ecd4226ac9 fc737f3b1edc 2024-04-07T16:27:10.615 INFO:teuthology.orchestra.run.smithi100.stdout:mgr.x smithi144 *:8443,9283,8765 running (7m) 13s ago 22m 441M - 19.0.0-2785-gadce92d0 c0ecd4226ac9 233745c292a6 2024-04-07T16:27:10.615 INFO:teuthology.orchestra.run.smithi100.stdout:mgr.y smithi100 *:8443,9283,8765 running (8m) 13s ago 24m 507M - 19.0.0-2785-gadce92d0 c0ecd4226ac9 254dc65cdaa7 2024-04-07T16:27:10.615 INFO:teuthology.orchestra.run.smithi100.stdout:mon.a smithi100 running (7m) 13s ago 24m 69.0M 2048M 19.0.0-2785-gadce92d0 c0ecd4226ac9 718e12655c04 2024-04-07T16:27:10.615 INFO:teuthology.orchestra.run.smithi100.stdout:mon.b smithi144 running (6m) 13s ago 22m 52.6M 2048M 19.0.0-2785-gadce92d0 c0ecd4226ac9 aef817664bff 2024-04-07T16:27:10.615 INFO:teuthology.orchestra.run.smithi100.stdout:mon.c smithi100 running (6m) 13s ago 22m 57.5M 2048M 19.0.0-2785-gadce92d0 c0ecd4226ac9 63e889040f6e 2024-04-07T16:27:10.615 INFO:teuthology.orchestra.run.smithi100.stdout:node-exporter.a smithi100 *:9100 running (23s) 13s ago 14m 6811k - 1.5.0 0da6a335fe13 9609d3a1c1ba 2024-04-07T16:27:10.615 INFO:teuthology.orchestra.run.smithi100.stdout:node-exporter.b smithi144 *:9100 running (19s) 13s ago 14m 3220k - 1.5.0 0da6a335fe13 abbe3a5b7ce6 2024-04-07T16:27:10.615 INFO:teuthology.orchestra.run.smithi100.stdout:osd.0 smithi100 running (4m) 13s ago 21m 341M 2524M 19.0.0-2785-gadce92d0 c0ecd4226ac9 a6e83c9d1abc 2024-04-07T16:27:10.616 INFO:teuthology.orchestra.run.smithi100.stdout:osd.1 smithi100 running (4m) 13s ago 20m 303M 2524M 19.0.0-2785-gadce92d0 c0ecd4226ac9 c50fe40079b6 2024-04-07T16:27:10.616 INFO:teuthology.orchestra.run.smithi100.stdout:osd.2 smithi100 running (4m) 13s ago 19m 362M 2524M 19.0.0-2785-gadce92d0 c0ecd4226ac9 8e631c43064b 2024-04-07T16:27:10.616 INFO:teuthology.orchestra.run.smithi100.stdout:osd.3 smithi100 running (3m) 13s ago 19m 151M 2524M 19.0.0-2785-gadce92d0 c0ecd4226ac9 402931247c79 2024-04-07T16:27:10.616 INFO:teuthology.orchestra.run.smithi100.stdout:osd.4 smithi144 running (3m) 13s ago 18m 328M 2524M 19.0.0-2785-gadce92d0 c0ecd4226ac9 ebbb7be95d33 2024-04-07T16:27:10.616 INFO:teuthology.orchestra.run.smithi100.stdout:osd.5 smithi144 running (2m) 13s ago 17m 206M 2524M 19.0.0-2785-gadce92d0 c0ecd4226ac9 ecce27bcb2f8 2024-04-07T16:27:10.616 INFO:teuthology.orchestra.run.smithi100.stdout:osd.6 smithi144 running (2m) 13s ago 16m 106M 2524M 19.0.0-2785-gadce92d0 c0ecd4226ac9 3788c5c3f427 2024-04-07T16:27:10.616 INFO:teuthology.orchestra.run.smithi100.stdout:osd.7 smithi144 running (99s) 13s ago 15m 183M 2524M 19.0.0-2785-gadce92d0 c0ecd4226ac9 9365aba97dc0 2024-04-07T16:27:10.616 INFO:teuthology.orchestra.run.smithi100.stdout:prometheus.a smithi144 *:9095 running (7m) 13s ago 14m 53.4M - 2.43.0 a07b618ecd1d 2d0a8131f303 2024-04-07T16:27:11.031 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-04-07T16:27:11.031 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": { 2024-04-07T16:27:11.031 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph version 19.0.0-2785-gadce92d0 (adce92d056533f22dc1ee37e3141d098b046ec1b) squid (dev)": 3 2024-04-07T16:27:11.031 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-07T16:27:11.031 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": { 2024-04-07T16:27:11.031 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph version 19.0.0-2785-gadce92d0 (adce92d056533f22dc1ee37e3141d098b046ec1b) squid (dev)": 2 2024-04-07T16:27:11.031 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-07T16:27:11.031 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": { 2024-04-07T16:27:11.031 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph version 19.0.0-2785-gadce92d0 (adce92d056533f22dc1ee37e3141d098b046ec1b) squid (dev)": 8 2024-04-07T16:27:11.031 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-07T16:27:11.031 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": { 2024-04-07T16:27:11.031 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph version 19.0.0-2785-gadce92d0 (adce92d056533f22dc1ee37e3141d098b046ec1b) squid (dev)": 2 2024-04-07T16:27:11.032 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-07T16:27:11.032 INFO:teuthology.orchestra.run.smithi100.stdout: "overall": { 2024-04-07T16:27:11.032 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph version 19.0.0-2785-gadce92d0 (adce92d056533f22dc1ee37e3141d098b046ec1b) squid (dev)": 15 2024-04-07T16:27:11.032 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-04-07T16:27:11.032 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-04-07T16:27:11.060 INFO:tasks.rados.rados.0.smithi100.stdout:3264: done (2 left) 2024-04-07T16:27:11.060 INFO:tasks.rados.rados.0.smithi100.stdout:3265: done (1 left) 2024-04-07T16:27:11.060 INFO:tasks.rados.rados.0.smithi100.stdout:3266: done (0 left) 2024-04-07T16:27:11.060 INFO:tasks.rados.rados.0.smithi100.stdout:3267: rollback oid 10 current snap is 300 2024-04-07T16:27:11.060 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 10 to 295 2024-04-07T16:27:11.060 INFO:tasks.rados.rados.0.smithi100.stdout:3268: read oid 12 snap -1 2024-04-07T16:27:11.060 INFO:tasks.rados.rados.0.smithi100.stdout:3268: expect (ObjNum 794 snap 251 seq_num 794) 2024-04-07T16:27:11.060 INFO:tasks.rados.rados.0.smithi100.stdout:3269: rollback oid 35 current snap is 300 2024-04-07T16:27:11.060 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 35 to 297 2024-04-07T16:27:11.061 INFO:tasks.rados.rados.0.smithi100.stdout:3270: read oid 39 snap -1 2024-04-07T16:27:11.061 INFO:tasks.rados.rados.0.smithi100.stdout:3270: expect (ObjNum 924 snap 278 seq_num 924) 2024-04-07T16:27:11.061 INFO:tasks.rados.rados.0.smithi100.stdout:3271: rollback oid 21 current snap is 300 2024-04-07T16:27:11.061 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 21 to 299 2024-04-07T16:27:11.061 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 45 current snap is 300 2024-04-07T16:27:11.061 INFO:tasks.rados.rados.0.smithi100.stdout:3272: seq_num 999 ranges {4096000=737280,4833280=565248,5398528=761856,6160384=278528} 2024-04-07T16:27:11.096 INFO:tasks.rados.rados.0.smithi100.stdout:3272: writing smithi10066867-45 from 4096000 to 4833280 tid 1 2024-04-07T16:27:11.099 INFO:tasks.rados.rados.0.smithi100.stdout:3272: writing smithi10066867-45 from 4833280 to 5398528 tid 2 2024-04-07T16:27:11.104 INFO:tasks.rados.rados.0.smithi100.stdout:3272: writing smithi10066867-45 from 5398528 to 6160384 tid 3 2024-04-07T16:27:11.106 INFO:tasks.rados.rados.0.smithi100.stdout:3272: writing smithi10066867-45 from 6160384 to 6438912 tid 4 2024-04-07T16:27:11.106 INFO:tasks.rados.rados.0.smithi100.stdout:3273: snap_remove snap 296 2024-04-07T16:27:11.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:11 smithi144 bash[66654]: cluster 2024-04-07T16:27:10.113923+0000 mon.a (mon.0) 1117 : cluster [DBG] osdmap e674: 8 total, 8 up, 8 in 2024-04-07T16:27:11.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:11 smithi144 bash[66654]: audit 2024-04-07T16:27:10.255595+0000 mgr.y (mgr.25175) 335 : audit [DBG] from='client.34446 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:27:11.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:11 smithi144 bash[66654]: cephadm 2024-04-07T16:27:10.324909+0000 mgr.y (mgr.25175) 336 : cephadm [INF] Upgrade: Updating prometheus.a 2024-04-07T16:27:11.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:11 smithi144 bash[66654]: audit 2024-04-07T16:27:11.046792+0000 mon.c (mon.2) 14 : audit [DBG] from='client.? 172.21.15.100:0/3342077722' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:27:11.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:11 smithi100 bash[74942]: cluster 2024-04-07T16:27:10.113923+0000 mon.a (mon.0) 1117 : cluster [DBG] osdmap e674: 8 total, 8 up, 8 in 2024-04-07T16:27:11.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:11 smithi100 bash[74942]: audit 2024-04-07T16:27:10.255595+0000 mgr.y (mgr.25175) 335 : audit [DBG] from='client.34446 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:27:11.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:11 smithi100 bash[74942]: cephadm 2024-04-07T16:27:10.324909+0000 mgr.y (mgr.25175) 336 : cephadm [INF] Upgrade: Updating prometheus.a 2024-04-07T16:27:11.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:11 smithi100 bash[74942]: audit 2024-04-07T16:27:11.046792+0000 mon.c (mon.2) 14 : audit [DBG] from='client.? 172.21.15.100:0/3342077722' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:27:11.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:11 smithi100 bash[76202]: cluster 2024-04-07T16:27:10.113923+0000 mon.a (mon.0) 1117 : cluster [DBG] osdmap e674: 8 total, 8 up, 8 in 2024-04-07T16:27:11.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:11 smithi100 bash[76202]: audit 2024-04-07T16:27:10.255595+0000 mgr.y (mgr.25175) 335 : audit [DBG] from='client.34446 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:27:11.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:11 smithi100 bash[76202]: cephadm 2024-04-07T16:27:10.324909+0000 mgr.y (mgr.25175) 336 : cephadm [INF] Upgrade: Updating prometheus.a 2024-04-07T16:27:11.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:11 smithi100 bash[76202]: audit 2024-04-07T16:27:11.046792+0000 mon.c (mon.2) 14 : audit [DBG] from='client.? 172.21.15.100:0/3342077722' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:27:11.922 INFO:tasks.rados.rados.0.smithi100.stdout:3273: done (6 left) 2024-04-07T16:27:11.922 INFO:tasks.rados.rados.0.smithi100.stdout:3274: setattr oid 4 current snap is 300 2024-04-07T16:27:11.924 INFO:tasks.rados.rados.0.smithi100.stdout:3271: finishing rollback tid 0 to smithi10066867-21 2024-04-07T16:27:11.924 INFO:tasks.rados.rados.0.smithi100.stdout:3271: finishing rollback tid 1 to smithi10066867-21 2024-04-07T16:27:11.924 INFO:tasks.rados.rados.0.smithi100.stdout:3271: finishing rollback tid 2 to smithi10066867-21 2024-04-07T16:27:11.924 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 21 v 1206 (ObjNum 794 snap 251 seq_num 794) dirty exists 2024-04-07T16:27:11.925 INFO:tasks.rados.rados.0.smithi100.stdout:3269: finishing rollback tid 0 to smithi10066867-35 2024-04-07T16:27:11.925 INFO:tasks.rados.rados.0.smithi100.stdout:3267: finishing rollback tid 0 to smithi10066867-10 2024-04-07T16:27:11.925 INFO:tasks.rados.rados.0.smithi100.stdout:3267: finishing rollback tid 1 to smithi10066867-10 2024-04-07T16:27:11.925 INFO:tasks.rados.rados.0.smithi100.stdout:3267: finishing rollback tid 2 to smithi10066867-10 2024-04-07T16:27:11.925 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 10 v 1455 (ObjNum 945 snap 282 seq_num 945) dirty exists 2024-04-07T16:27:11.925 INFO:tasks.rados.rados.0.smithi100.stdout:3268: expect (ObjNum 794 snap 251 seq_num 794) 2024-04-07T16:27:11.968 INFO:tasks.rados.rados.0.smithi100.stdout:3270: expect (ObjNum 924 snap 278 seq_num 924) 2024-04-07T16:27:11.996 INFO:tasks.rados.rados.0.smithi100.stdout:3269: finishing rollback tid 1 to smithi10066867-35 2024-04-07T16:27:11.997 INFO:tasks.rados.rados.0.smithi100.stdout:3269: finishing rollback tid 2 to smithi10066867-35 2024-04-07T16:27:11.997 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 35 v 1695 (ObjNum 941 snap 282 seq_num 941) dirty exists 2024-04-07T16:27:11.997 INFO:tasks.rados.rados.0.smithi100.stdout:3272: finishing write tid 1 to smithi10066867-45 2024-04-07T16:27:11.997 INFO:tasks.rados.rados.0.smithi100.stdout:3272: finishing write tid 2 to smithi10066867-45 2024-04-07T16:27:11.997 INFO:tasks.rados.rados.0.smithi100.stdout:3272: finishing write tid 3 to smithi10066867-45 2024-04-07T16:27:11.997 INFO:tasks.rados.rados.0.smithi100.stdout:3272: finishing write tid 4 to smithi10066867-45 2024-04-07T16:27:11.997 INFO:tasks.rados.rados.0.smithi100.stdout:3272: finishing write tid 5 to smithi10066867-45 2024-04-07T16:27:11.997 INFO:tasks.rados.rados.0.smithi100.stdout:3272: finishing write tid 6 to smithi10066867-45 2024-04-07T16:27:11.997 INFO:tasks.rados.rados.0.smithi100.stdout:3272: oid 45 updating version 0 to 1542 2024-04-07T16:27:11.997 INFO:tasks.rados.rados.0.smithi100.stdout:3272: oid 45 version 1542 is already newer than 1540 2024-04-07T16:27:11.997 INFO:tasks.rados.rados.0.smithi100.stdout:3272: oid 45 version 1542 is already newer than 1541 2024-04-07T16:27:11.997 INFO:tasks.rados.rados.0.smithi100.stdout:3272: oid 45 version 1542 is already newer than 1539 2024-04-07T16:27:11.997 INFO:tasks.rados.rados.0.smithi100.stdout:3272: oid 45 updating version 1542 to 1543 2024-04-07T16:27:11.998 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 45 v 1543 (ObjNum 999 snap 300 seq_num 999) dirty exists 2024-04-07T16:27:11.998 INFO:tasks.rados.rados.0.smithi100.stdout:3272: left oid 45 (ObjNum 999 snap 300 seq_num 999) 2024-04-07T16:27:11.998 INFO:tasks.rados.rados.0.smithi100.stdout:3267: done (6 left) 2024-04-07T16:27:11.998 INFO:tasks.rados.rados.0.smithi100.stdout:3268: done (5 left) 2024-04-07T16:27:11.998 INFO:tasks.rados.rados.0.smithi100.stdout:3269: done (4 left) 2024-04-07T16:27:11.998 INFO:tasks.rados.rados.0.smithi100.stdout:3270: done (3 left) 2024-04-07T16:27:11.998 INFO:tasks.rados.rados.0.smithi100.stdout:3271: done (2 left) 2024-04-07T16:27:11.998 INFO:tasks.rados.rados.0.smithi100.stdout:3272: done (1 left) 2024-04-07T16:27:11.998 INFO:tasks.rados.rados.0.smithi100.stdout:3275: read oid 32 snap -1 2024-04-07T16:27:11.998 INFO:tasks.rados.rados.0.smithi100.stdout:3275: expect (ObjNum 941 snap 282 seq_num 941) 2024-04-07T16:27:11.998 INFO:tasks.rados.rados.0.smithi100.stdout:3276: setattr oid 43 current snap is 300 2024-04-07T16:27:11.999 INFO:tasks.rados.rados.0.smithi100.stdout:3277: delete oid 46 current snap is 300 2024-04-07T16:27:12.003 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 4 v 1754 (ObjNum 917 snap 277 seq_num 917) dirty exists 2024-04-07T16:27:12.004 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 43 v 1172 (ObjNum 161 snap 0 seq_num 1) dirty exists 2024-04-07T16:27:12.008 INFO:tasks.rados.rados.0.smithi100.stdout:3275: expect (ObjNum 941 snap 282 seq_num 941) 2024-04-07T16:27:12.041 INFO:tasks.rados.rados.0.smithi100.stdout:3274: done (3 left) 2024-04-07T16:27:12.041 INFO:tasks.rados.rados.0.smithi100.stdout:3275: done (2 left) 2024-04-07T16:27:12.041 INFO:tasks.rados.rados.0.smithi100.stdout:3276: done (1 left) 2024-04-07T16:27:12.041 INFO:tasks.rados.rados.0.smithi100.stdout:3277: done (0 left) 2024-04-07T16:27:12.041 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 50 current snap is 300 2024-04-07T16:27:12.041 INFO:tasks.rados.rados.0.smithi100.stdout:3278: seq_num 1001 ranges {0=770048,770048=729088,1499136=770048,2269184=106496} 2024-04-07T16:27:12.047 INFO:tasks.rados.rados.0.smithi100.stdout:3278: writing smithi10066867-50 from 0 to 770048 tid 1 2024-04-07T16:27:12.053 INFO:tasks.rados.rados.0.smithi100.stdout:3278: writing smithi10066867-50 from 770048 to 1499136 tid 2 2024-04-07T16:27:12.058 INFO:tasks.rados.rados.0.smithi100.stdout:3278: writing smithi10066867-50 from 1499136 to 2269184 tid 3 2024-04-07T16:27:12.059 INFO:tasks.rados.rados.0.smithi100.stdout:3278: writing smithi10066867-50 from 2269184 to 2375680 tid 4 2024-04-07T16:27:12.059 INFO:tasks.rados.rados.0.smithi100.stdout:3279: delete oid 32 current snap is 300 2024-04-07T16:27:12.059 INFO:tasks.rados.rados.0.smithi100.stdout:3278: finishing write tid 1 to smithi10066867-50 2024-04-07T16:27:12.059 INFO:tasks.rados.rados.0.smithi100.stdout:3278: finishing write tid 2 to smithi10066867-50 2024-04-07T16:27:12.061 INFO:tasks.rados.rados.0.smithi100.stdout:3279: done (1 left) 2024-04-07T16:27:12.062 INFO:tasks.rados.rados.0.smithi100.stdout:3280: rollback oid 27 current snap is 300 2024-04-07T16:27:12.062 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 27 to 297 2024-04-07T16:27:12.062 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 26 current snap is 300 2024-04-07T16:27:12.062 INFO:tasks.rados.rados.0.smithi100.stdout:3281: seq_num 1003 ranges {6586368=802816,7389184=499712,7888896=737280,8626176=368640} 2024-04-07T16:27:12.103 INFO:tasks.rados.rados.0.smithi100.stdout:3281: writing smithi10066867-26 from 6586368 to 7389184 tid 1 2024-04-07T16:27:12.106 INFO:tasks.rados.rados.0.smithi100.stdout:3281: writing smithi10066867-26 from 7389184 to 7888896 tid 2 2024-04-07T16:27:12.111 INFO:tasks.rados.rados.0.smithi100.stdout:3281: writing smithi10066867-26 from 7888896 to 8626176 tid 3 2024-04-07T16:27:12.113 INFO:tasks.rados.rados.0.smithi100.stdout:3281: writing smithi10066867-26 from 8626176 to 8994816 tid 4 2024-04-07T16:27:12.113 INFO:tasks.rados.rados.0.smithi100.stdout:3282: read oid 8 snap -1 2024-04-07T16:27:12.113 INFO:tasks.rados.rados.0.smithi100.stdout:3282: expect (ObjNum 968 snap 293 seq_num 968) 2024-04-07T16:27:12.113 INFO:tasks.rados.rados.0.smithi100.stdout:3283: snap_remove snap 299 2024-04-07T16:27:12.294 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:12 smithi144 bash[66654]: cephadm 2024-04-07T16:27:10.602659+0000 mgr.y (mgr.25175) 337 : cephadm [INF] Deploying daemon prometheus.a on smithi144 2024-04-07T16:27:12.294 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:12 smithi144 bash[66654]: audit 2024-04-07T16:27:10.615662+0000 mgr.y (mgr.25175) 338 : audit [DBG] from='client.34452 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:27:12.294 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:12 smithi144 bash[66654]: cluster 2024-04-07T16:27:10.784443+0000 mgr.y (mgr.25175) 339 : cluster [DBG] pgmap v604: 121 pgs: 121 active+clean; 143 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 9.9 MiB/s rd, 5.9 MiB/s wr, 23 op/s 2024-04-07T16:27:12.294 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:12 smithi144 bash[66654]: cluster 2024-04-07T16:27:11.925736+0000 mon.a (mon.0) 1118 : cluster [DBG] osdmap e675: 8 total, 8 up, 8 in 2024-04-07T16:27:12.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:12 smithi100 bash[74942]: cephadm 2024-04-07T16:27:10.602659+0000 mgr.y (mgr.25175) 337 : cephadm [INF] Deploying daemon prometheus.a on smithi144 2024-04-07T16:27:12.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:12 smithi100 bash[74942]: audit 2024-04-07T16:27:10.615662+0000 mgr.y (mgr.25175) 338 : audit [DBG] from='client.34452 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:27:12.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:12 smithi100 bash[74942]: cluster 2024-04-07T16:27:10.784443+0000 mgr.y (mgr.25175) 339 : cluster [DBG] pgmap v604: 121 pgs: 121 active+clean; 143 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 9.9 MiB/s rd, 5.9 MiB/s wr, 23 op/s 2024-04-07T16:27:12.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:12 smithi100 bash[74942]: cluster 2024-04-07T16:27:11.925736+0000 mon.a (mon.0) 1118 : cluster [DBG] osdmap e675: 8 total, 8 up, 8 in 2024-04-07T16:27:12.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:12 smithi100 bash[76202]: cephadm 2024-04-07T16:27:10.602659+0000 mgr.y (mgr.25175) 337 : cephadm [INF] Deploying daemon prometheus.a on smithi144 2024-04-07T16:27:12.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:12 smithi100 bash[76202]: audit 2024-04-07T16:27:10.615662+0000 mgr.y (mgr.25175) 338 : audit [DBG] from='client.34452 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:27:12.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:12 smithi100 bash[76202]: cluster 2024-04-07T16:27:10.784443+0000 mgr.y (mgr.25175) 339 : cluster [DBG] pgmap v604: 121 pgs: 121 active+clean; 143 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 9.9 MiB/s rd, 5.9 MiB/s wr, 23 op/s 2024-04-07T16:27:12.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:12 smithi100 bash[76202]: cluster 2024-04-07T16:27:11.925736+0000 mon.a (mon.0) 1118 : cluster [DBG] osdmap e675: 8 total, 8 up, 8 in 2024-04-07T16:27:12.605 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:27:12 smithi144 bash[59794]: ts=2024-04-07T16:27:12.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:27:12.918 INFO:tasks.rados.rados.0.smithi100.stdout:3283: done (4 left) 2024-04-07T16:27:12.919 INFO:tasks.rados.rados.0.smithi100.stdout:3284: read oid 24 snap -1 2024-04-07T16:27:12.919 INFO:tasks.rados.rados.0.smithi100.stdout:3284: expect (ObjNum 989 snap 298 seq_num 989) 2024-04-07T16:27:12.919 INFO:tasks.rados.rados.0.smithi100.stdout:3285: snap_remove snap 295 2024-04-07T16:27:13.928 INFO:tasks.rados.rados.0.smithi100.stdout:3285: done (5 left) 2024-04-07T16:27:13.928 INFO:tasks.rados.rados.0.smithi100.stdout:3286: read oid 18 snap -1 2024-04-07T16:27:13.928 INFO:tasks.rados.rados.0.smithi100.stdout:3286: expect deleted 2024-04-07T16:27:13.928 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 40 current snap is 300 2024-04-07T16:27:13.928 INFO:tasks.rados.rados.0.smithi100.stdout:3287: seq_num 1004 ranges {1736704=16384} 2024-04-07T16:27:13.938 INFO:tasks.rados.rados.0.smithi100.stdout:3287: writing smithi10066867-40 from 1736704 to 1753088 tid 1 2024-04-07T16:27:13.939 INFO:tasks.rados.rados.0.smithi100.stdout:3278: finishing write tid 3 to smithi10066867-50 2024-04-07T16:27:13.939 INFO:tasks.rados.rados.0.smithi100.stdout:3278: finishing write tid 4 to smithi10066867-50 2024-04-07T16:27:13.939 INFO:tasks.rados.rados.0.smithi100.stdout:3278: finishing write tid 5 to smithi10066867-50 2024-04-07T16:27:13.939 INFO:tasks.rados.rados.0.smithi100.stdout:3280: finishing rollback tid 0 to smithi10066867-27 2024-04-07T16:27:13.939 INFO:tasks.rados.rados.0.smithi100.stdout:3280: finishing rollback tid 1 to smithi10066867-27 2024-04-07T16:27:13.939 INFO:tasks.rados.rados.0.smithi100.stdout:3280: finishing rollback tid 2 to smithi10066867-27 2024-04-07T16:27:13.939 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 27 v 1175 (ObjNum 953 snap 286 seq_num 953) dirty exists 2024-04-07T16:27:13.939 INFO:tasks.rados.rados.0.smithi100.stdout:3278: finishing write tid 6 to smithi10066867-50 2024-04-07T16:27:13.939 INFO:tasks.rados.rados.0.smithi100.stdout:3278: oid 50 updating version 0 to 1548 2024-04-07T16:27:13.939 INFO:tasks.rados.rados.0.smithi100.stdout:3278: oid 50 version 1548 is already newer than 1546 2024-04-07T16:27:13.939 INFO:tasks.rados.rados.0.smithi100.stdout:3278: oid 50 version 1548 is already newer than 1544 2024-04-07T16:27:13.939 INFO:tasks.rados.rados.0.smithi100.stdout:3278: oid 50 version 1548 is already newer than 1545 2024-04-07T16:27:13.939 INFO:tasks.rados.rados.0.smithi100.stdout:3278: oid 50 version 1548 is already newer than 1547 2024-04-07T16:27:13.939 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 50 v 1548 (ObjNum 1001 snap 300 seq_num 1001) dirty exists 2024-04-07T16:27:13.939 INFO:tasks.rados.rados.0.smithi100.stdout:3278: left oid 50 (ObjNum 1001 snap 300 seq_num 1001) 2024-04-07T16:27:13.940 INFO:tasks.rados.rados.0.smithi100.stdout:3281: finishing write tid 1 to smithi10066867-26 2024-04-07T16:27:13.940 INFO:tasks.rados.rados.0.smithi100.stdout:3281: finishing write tid 2 to smithi10066867-26 2024-04-07T16:27:13.940 INFO:tasks.rados.rados.0.smithi100.stdout:3281: finishing write tid 3 to smithi10066867-26 2024-04-07T16:27:13.940 INFO:tasks.rados.rados.0.smithi100.stdout:3281: finishing write tid 4 to smithi10066867-26 2024-04-07T16:27:13.940 INFO:tasks.rados.rados.0.smithi100.stdout:3281: finishing write tid 5 to smithi10066867-26 2024-04-07T16:27:13.940 INFO:tasks.rados.rados.0.smithi100.stdout:3281: finishing write tid 6 to smithi10066867-26 2024-04-07T16:27:13.940 INFO:tasks.rados.rados.0.smithi100.stdout:3281: oid 26 updating version 0 to 1526 2024-04-07T16:27:13.940 INFO:tasks.rados.rados.0.smithi100.stdout:3281: oid 26 updating version 1526 to 1527 2024-04-07T16:27:13.940 INFO:tasks.rados.rados.0.smithi100.stdout:3281: oid 26 version 1527 is already newer than 1523 2024-04-07T16:27:13.940 INFO:tasks.rados.rados.0.smithi100.stdout:3281: oid 26 version 1527 is already newer than 1525 2024-04-07T16:27:13.940 INFO:tasks.rados.rados.0.smithi100.stdout:3281: oid 26 version 1527 is already newer than 1524 2024-04-07T16:27:13.940 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 26 v 1527 (ObjNum 1003 snap 300 seq_num 1003) dirty exists 2024-04-07T16:27:13.940 INFO:tasks.rados.rados.0.smithi100.stdout:3281: left oid 26 (ObjNum 1003 snap 300 seq_num 1003) 2024-04-07T16:27:13.940 INFO:tasks.rados.rados.0.smithi100.stdout:3282: expect (ObjNum 968 snap 293 seq_num 968) 2024-04-07T16:27:14.078 INFO:tasks.rados.rados.0.smithi100.stdout:3284: expect (ObjNum 989 snap 298 seq_num 989) 2024-04-07T16:27:14.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:13 smithi144 bash[66654]: cluster 2024-04-07T16:27:12.785491+0000 mgr.y (mgr.25175) 340 : cluster [DBG] pgmap v606: 121 pgs: 121 active+clean; 149 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 8.1 MiB/s rd, 5.2 MiB/s wr, 22 op/s 2024-04-07T16:27:14.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:13 smithi144 bash[66654]: cluster 2024-04-07T16:27:12.928698+0000 mon.a (mon.0) 1119 : cluster [DBG] osdmap e676: 8 total, 8 up, 8 in 2024-04-07T16:27:14.186 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:13 smithi100 bash[74942]: cluster 2024-04-07T16:27:12.785491+0000 mgr.y (mgr.25175) 340 : cluster [DBG] pgmap v606: 121 pgs: 121 active+clean; 149 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 8.1 MiB/s rd, 5.2 MiB/s wr, 22 op/s 2024-04-07T16:27:14.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:13 smithi100 bash[74942]: cluster 2024-04-07T16:27:12.928698+0000 mon.a (mon.0) 1119 : cluster [DBG] osdmap e676: 8 total, 8 up, 8 in 2024-04-07T16:27:14.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:13 smithi100 bash[76202]: cluster 2024-04-07T16:27:12.785491+0000 mgr.y (mgr.25175) 340 : cluster [DBG] pgmap v606: 121 pgs: 121 active+clean; 149 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 8.1 MiB/s rd, 5.2 MiB/s wr, 22 op/s 2024-04-07T16:27:14.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:13 smithi100 bash[76202]: cluster 2024-04-07T16:27:12.928698+0000 mon.a (mon.0) 1119 : cluster [DBG] osdmap e676: 8 total, 8 up, 8 in 2024-04-07T16:27:14.626 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:14 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:27:14.626 INFO:journalctl@ceph.osd.4.smithi144.stdout:Apr 07 16:27:14 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:27:14.627 INFO:journalctl@ceph.osd.5.smithi144.stdout:Apr 07 16:27:14 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:27:14.627 INFO:journalctl@ceph.osd.7.smithi144.stdout:Apr 07 16:27:14 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:27:14.627 INFO:journalctl@ceph.mds.b.smithi144.stdout:Apr 07 16:27:14 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:27:14.628 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:27:14 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:27:14.628 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 16:27:14 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:27:14.628 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:27:14 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:27:14.628 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:27:14 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:27:14.629 INFO:journalctl@ceph.osd.6.smithi144.stdout:Apr 07 16:27:14 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:27:14.926 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:27:14 smithi144 systemd[1]: Stopping Ceph prometheus.a for 0702e3a8-f4f8-11ee-b649-cb9ed24678a4... 2024-04-07T16:27:14.926 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:27:14 smithi144 bash[59794]: ts=2024-04-07T16:27:14.758Z caller=main.go:840 level=warn msg="Received SIGTERM, exiting gracefully..." 2024-04-07T16:27:14.926 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:27:14 smithi144 bash[59794]: ts=2024-04-07T16:27:14.758Z caller=main.go:864 level=info msg="Stopping scrape discovery manager..." 2024-04-07T16:27:14.926 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:27:14 smithi144 bash[59794]: ts=2024-04-07T16:27:14.758Z caller=main.go:878 level=info msg="Stopping notify discovery manager..." 2024-04-07T16:27:14.926 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:27:14 smithi144 bash[59794]: ts=2024-04-07T16:27:14.758Z caller=manager.go:988 level=info component="rule manager" msg="Stopping rule manager..." 2024-04-07T16:27:14.926 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:27:14 smithi144 bash[59794]: ts=2024-04-07T16:27:14.758Z caller=main.go:860 level=info msg="Scrape discovery manager stopped" 2024-04-07T16:27:14.927 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:27:14 smithi144 bash[59794]: ts=2024-04-07T16:27:14.758Z caller=main.go:874 level=info msg="Notify discovery manager stopped" 2024-04-07T16:27:14.927 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:27:14 smithi144 bash[59794]: ts=2024-04-07T16:27:14.758Z caller=manager.go:998 level=info component="rule manager" msg="Rule manager stopped" 2024-04-07T16:27:14.927 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:27:14 smithi144 bash[59794]: ts=2024-04-07T16:27:14.758Z caller=main.go:915 level=info msg="Stopping scrape manager..." 2024-04-07T16:27:14.927 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:27:14 smithi144 bash[59794]: ts=2024-04-07T16:27:14.758Z caller=main.go:907 level=info msg="Scrape manager stopped" 2024-04-07T16:27:14.927 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:27:14 smithi144 bash[59794]: ts=2024-04-07T16:27:14.758Z caller=notifier.go:606 level=info component=notifier msg="Stopping notification manager..." 2024-04-07T16:27:14.927 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:27:14 smithi144 bash[59794]: ts=2024-04-07T16:27:14.759Z caller=main.go:1135 level=info msg="Notifier manager stopped" 2024-04-07T16:27:14.927 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:27:14 smithi144 bash[59794]: ts=2024-04-07T16:27:14.759Z caller=main.go:1147 level=info msg="See you next time!" 2024-04-07T16:27:14.992 INFO:tasks.rados.rados.0.smithi100.stdout:3287: finishing write tid 1 to smithi10066867-40 2024-04-07T16:27:14.992 INFO:tasks.rados.rados.0.smithi100.stdout:3287: finishing write tid 2 to smithi10066867-40 2024-04-07T16:27:14.992 INFO:tasks.rados.rados.0.smithi100.stdout:3287: finishing write tid 3 to smithi10066867-40 2024-04-07T16:27:14.992 INFO:tasks.rados.rados.0.smithi100.stdout:3287: oid 40 updating version 0 to 1639 2024-04-07T16:27:14.992 INFO:tasks.rados.rados.0.smithi100.stdout:3287: oid 40 version 1639 is already newer than 1638 2024-04-07T16:27:14.992 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 40 v 1639 (ObjNum 1004 snap 300 seq_num 1004) dirty exists 2024-04-07T16:27:14.992 INFO:tasks.rados.rados.0.smithi100.stdout:3287: left oid 40 (ObjNum 1004 snap 300 seq_num 1004) 2024-04-07T16:27:14.992 INFO:tasks.rados.rados.0.smithi100.stdout:3278: done (6 left) 2024-04-07T16:27:14.992 INFO:tasks.rados.rados.0.smithi100.stdout:3280: done (5 left) 2024-04-07T16:27:14.993 INFO:tasks.rados.rados.0.smithi100.stdout:3281: done (4 left) 2024-04-07T16:27:14.993 INFO:tasks.rados.rados.0.smithi100.stdout:3282: done (3 left) 2024-04-07T16:27:14.993 INFO:tasks.rados.rados.0.smithi100.stdout:3284: done (2 left) 2024-04-07T16:27:14.993 INFO:tasks.rados.rados.0.smithi100.stdout:3286: done (1 left) 2024-04-07T16:27:14.993 INFO:tasks.rados.rados.0.smithi100.stdout:3287: done (0 left) 2024-04-07T16:27:14.993 INFO:tasks.rados.rados.0.smithi100.stdout:3288: copy_from oid 39 from oid 8 current snap is 300 2024-04-07T16:27:14.993 INFO:tasks.rados.rados.0.smithi100.stdout:3289: read oid 34 snap -1 2024-04-07T16:27:14.993 INFO:tasks.rados.rados.0.smithi100.stdout:3289: expect (ObjNum 998 snap 300 seq_num 998) 2024-04-07T16:27:14.993 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 9 current snap is 300 2024-04-07T16:27:14.993 INFO:tasks.rados.rados.0.smithi100.stdout:3290: seq_num 1005 ranges {0=32768} 2024-04-07T16:27:14.993 INFO:tasks.rados.rados.0.smithi100.stdout:3290: writing smithi10066867-9 from 0 to 32768 tid 1 2024-04-07T16:27:14.993 INFO:tasks.rados.rados.0.smithi100.stdout:3291: rollback oid 43 current snap is 300 2024-04-07T16:27:14.993 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 43 to 297 2024-04-07T16:27:14.993 INFO:tasks.rados.rados.0.smithi100.stdout:3292: delete oid 2 current snap is 300 2024-04-07T16:27:14.998 INFO:tasks.rados.rados.0.smithi100.stdout:3290: finishing write tid 1 to smithi10066867-9 2024-04-07T16:27:14.998 INFO:tasks.rados.rados.0.smithi100.stdout:3292: done (4 left) 2024-04-07T16:27:14.998 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 20 current snap is 300 2024-04-07T16:27:14.999 INFO:tasks.rados.rados.0.smithi100.stdout:3293: seq_num 1007 ranges {3235840=65536} 2024-04-07T16:27:15.017 INFO:tasks.rados.rados.0.smithi100.stdout:3293: writing smithi10066867-20 from 3235840 to 3301376 tid 1 2024-04-07T16:27:15.017 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 33 current snap is 300 2024-04-07T16:27:15.017 INFO:tasks.rados.rados.0.smithi100.stdout:3290: finishing write tid 2 to smithi10066867-9 2024-04-07T16:27:15.017 INFO:tasks.rados.rados.0.smithi100.stdout:3294: seq_num 1008 ranges {614400=90112} 2024-04-07T16:27:15.021 INFO:tasks.rados.rados.0.smithi100.stdout:3294: writing smithi10066867-33 from 614400 to 704512 tid 1 2024-04-07T16:27:15.021 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 19 current snap is 300 2024-04-07T16:27:15.021 INFO:tasks.rados.rados.0.smithi100.stdout:3291: finishing rollback tid 0 to smithi10066867-43 2024-04-07T16:27:15.021 INFO:tasks.rados.rados.0.smithi100.stdout:3291: finishing rollback tid 1 to smithi10066867-43 2024-04-07T16:27:15.021 INFO:tasks.rados.rados.0.smithi100.stdout:3295: seq_num 1009 ranges {1417216=106496} 2024-04-07T16:27:15.031 INFO:tasks.rados.rados.0.smithi100.stdout:3295: writing smithi10066867-19 from 1417216 to 1523712 tid 1 2024-04-07T16:27:15.031 INFO:tasks.rados.rados.0.smithi100.stdout:3296: delete oid 12 current snap is 300 2024-04-07T16:27:15.031 INFO:tasks.rados.rados.0.smithi100.stdout:3291: finishing rollback tid 2 to smithi10066867-43 2024-04-07T16:27:15.031 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 43 v 1178 (ObjNum 161 snap 0 seq_num 0) dirty exists 2024-04-07T16:27:15.031 INFO:tasks.rados.rados.0.smithi100.stdout:3290: finishing write tid 3 to smithi10066867-9 2024-04-07T16:27:15.031 INFO:tasks.rados.rados.0.smithi100.stdout:3290: oid 9 updating version 0 to 1134 2024-04-07T16:27:15.031 INFO:tasks.rados.rados.0.smithi100.stdout:3290: oid 9 updating version 1134 to 1135 2024-04-07T16:27:15.032 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 9 v 1135 (ObjNum 1005 snap 300 seq_num 1005) dirty exists 2024-04-07T16:27:15.032 INFO:tasks.rados.rados.0.smithi100.stdout:3290: left oid 9 (ObjNum 1005 snap 300 seq_num 1005) 2024-04-07T16:27:15.032 INFO:tasks.rados.rados.0.smithi100.stdout:3293: finishing write tid 1 to smithi10066867-20 2024-04-07T16:27:15.032 INFO:tasks.rados.rados.0.smithi100.stdout:3293: finishing write tid 2 to smithi10066867-20 2024-04-07T16:27:15.032 INFO:tasks.rados.rados.0.smithi100.stdout:3293: finishing write tid 3 to smithi10066867-20 2024-04-07T16:27:15.032 INFO:tasks.rados.rados.0.smithi100.stdout:3293: oid 20 updating version 0 to 1529 2024-04-07T16:27:15.032 INFO:tasks.rados.rados.0.smithi100.stdout:3293: oid 20 version 1529 is already newer than 1528 2024-04-07T16:27:15.032 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 20 v 1529 (ObjNum 1007 snap 300 seq_num 1007) dirty exists 2024-04-07T16:27:15.032 INFO:tasks.rados.rados.0.smithi100.stdout:3293: left oid 20 (ObjNum 1007 snap 300 seq_num 1007) 2024-04-07T16:27:15.032 INFO:tasks.rados.rados.0.smithi100.stdout:3294: finishing write tid 1 to smithi10066867-33 2024-04-07T16:27:15.032 INFO:tasks.rados.rados.0.smithi100.stdout:3294: finishing write tid 2 to smithi10066867-33 2024-04-07T16:27:15.032 INFO:tasks.rados.rados.0.smithi100.stdout:3294: finishing write tid 3 to smithi10066867-33 2024-04-07T16:27:15.032 INFO:tasks.rados.rados.0.smithi100.stdout:3294: oid 33 updating version 0 to 1768 2024-04-07T16:27:15.032 INFO:tasks.rados.rados.0.smithi100.stdout:3294: oid 33 updating version 1768 to 1769 2024-04-07T16:27:15.033 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 33 v 1769 (ObjNum 1008 snap 300 seq_num 1008) dirty exists 2024-04-07T16:27:15.033 INFO:tasks.rados.rados.0.smithi100.stdout:3294: left oid 33 (ObjNum 1008 snap 300 seq_num 1008) 2024-04-07T16:27:15.034 INFO:tasks.rados.rados.0.smithi100.stdout:3290: done (7 left) 2024-04-07T16:27:15.035 INFO:tasks.rados.rados.0.smithi100.stdout:3291: done (6 left) 2024-04-07T16:27:15.035 INFO:tasks.rados.rados.0.smithi100.stdout:3293: done (5 left) 2024-04-07T16:27:15.035 INFO:tasks.rados.rados.0.smithi100.stdout:3294: done (4 left) 2024-04-07T16:27:15.035 INFO:tasks.rados.rados.0.smithi100.stdout:3296: done (3 left) 2024-04-07T16:27:15.035 INFO:tasks.rados.rados.0.smithi100.stdout:3297: rollback oid 46 current snap is 300 2024-04-07T16:27:15.035 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 46 to 297 2024-04-07T16:27:15.035 INFO:tasks.rados.rados.0.smithi100.stdout:3298: read oid 30 snap -1 2024-04-07T16:27:15.035 INFO:tasks.rados.rados.0.smithi100.stdout:3298: expect (ObjNum 991 snap 299 seq_num 991) 2024-04-07T16:27:15.035 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 35 current snap is 300 2024-04-07T16:27:15.035 INFO:tasks.rados.rados.0.smithi100.stdout:3299: seq_num 1011 ranges {1368064=139264} 2024-04-07T16:27:15.044 INFO:tasks.rados.rados.0.smithi100.stdout:3299: writing smithi10066867-35 from 1368064 to 1507328 tid 1 2024-04-07T16:27:15.044 INFO:tasks.rados.rados.0.smithi100.stdout:3300: rollback oid 43 current snap is 300 2024-04-07T16:27:15.044 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 43 to 297 2024-04-07T16:27:15.044 INFO:tasks.rados.rados.0.smithi100.stdout:3295: finishing write tid 1 to smithi10066867-19 2024-04-07T16:27:15.044 INFO:tasks.rados.rados.0.smithi100.stdout:3289: expect (ObjNum 998 snap 300 seq_num 998) 2024-04-07T16:27:15.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:14 smithi100 bash[74942]: cluster 2024-04-07T16:27:13.941904+0000 mon.a (mon.0) 1120 : cluster [DBG] osdmap e677: 8 total, 8 up, 8 in 2024-04-07T16:27:15.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:14 smithi100 bash[76202]: cluster 2024-04-07T16:27:13.941904+0000 mon.a (mon.0) 1120 : cluster [DBG] osdmap e677: 8 total, 8 up, 8 in 2024-04-07T16:27:15.299 INFO:tasks.rados.rados.0.smithi100.stdout:3295: finishing write tid 2 to smithi10066867-19 2024-04-07T16:27:15.299 INFO:tasks.rados.rados.0.smithi100.stdout:3295: finishing write tid 3 to smithi10066867-19 2024-04-07T16:27:15.299 INFO:tasks.rados.rados.0.smithi100.stdout:3295: oid 19 updating version 0 to 1261 2024-04-07T16:27:15.299 INFO:tasks.rados.rados.0.smithi100.stdout:3295: oid 19 updating version 1261 to 1262 2024-04-07T16:27:15.299 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 19 v 1262 (ObjNum 1009 snap 300 seq_num 1009) dirty exists 2024-04-07T16:27:15.299 INFO:tasks.rados.rados.0.smithi100.stdout:3295: left oid 19 (ObjNum 1009 snap 300 seq_num 1009) 2024-04-07T16:27:15.299 INFO:tasks.rados.rados.0.smithi100.stdout:3297: finishing rollback tid 1 to smithi10066867-46 2024-04-07T16:27:15.300 INFO:tasks.rados.rados.0.smithi100.stdout:3297: finishing rollback tid 2 to smithi10066867-46 2024-04-07T16:27:15.300 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 46 v 1567 (ObjNum 982 snap 297 seq_num 982) dirty exists 2024-04-07T16:27:15.300 INFO:tasks.rados.rados.0.smithi100.stdout:3300: finishing rollback tid 0 to smithi10066867-43 2024-04-07T16:27:15.300 INFO:tasks.rados.rados.0.smithi100.stdout:3289: done (6 left) 2024-04-07T16:27:15.300 INFO:tasks.rados.rados.0.smithi100.stdout:3295: done (5 left) 2024-04-07T16:27:15.300 INFO:tasks.rados.rados.0.smithi100.stdout:3297: done (4 left) 2024-04-07T16:27:15.300 INFO:tasks.rados.rados.0.smithi100.stdout:3301: rmattr oid 2 current snap is 300 2024-04-07T16:27:15.300 INFO:tasks.rados.rados.0.smithi100.stdout:3301: done (4 left) 2024-04-07T16:27:15.300 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 34 current snap is 300 2024-04-07T16:27:15.300 INFO:tasks.rados.rados.0.smithi100.stdout:3300: finishing rollback tid 1 to smithi10066867-43 2024-04-07T16:27:15.300 INFO:tasks.rados.rados.0.smithi100.stdout:3300: finishing rollback tid 2 to smithi10066867-43 2024-04-07T16:27:15.300 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 43 v 1181 (ObjNum 161 snap 0 seq_num 0) dirty exists 2024-04-07T16:27:15.300 INFO:tasks.rados.rados.0.smithi100.stdout:3299: finishing write tid 1 to smithi10066867-35 2024-04-07T16:27:15.300 INFO:tasks.rados.rados.0.smithi100.stdout:3299: finishing write tid 2 to smithi10066867-35 2024-04-07T16:27:15.301 INFO:tasks.rados.rados.0.smithi100.stdout:3299: finishing write tid 3 to smithi10066867-35 2024-04-07T16:27:15.301 INFO:tasks.rados.rados.0.smithi100.stdout:3299: oid 35 updating version 0 to 1697 2024-04-07T16:27:15.301 INFO:tasks.rados.rados.0.smithi100.stdout:3299: oid 35 version 1697 is already newer than 1696 2024-04-07T16:27:15.301 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 35 v 1697 (ObjNum 1011 snap 300 seq_num 1011) dirty exists 2024-04-07T16:27:15.301 INFO:tasks.rados.rados.0.smithi100.stdout:3299: left oid 35 (ObjNum 1011 snap 300 seq_num 1011) 2024-04-07T16:27:15.301 INFO:tasks.rados.rados.0.smithi100.stdout:3298: expect (ObjNum 991 snap 299 seq_num 991) 2024-04-07T16:27:15.327 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:14 smithi144 bash[66654]: cluster 2024-04-07T16:27:13.941904+0000 mon.a (mon.0) 1120 : cluster [DBG] osdmap e677: 8 total, 8 up, 8 in 2024-04-07T16:27:15.366 INFO:tasks.rados.rados.0.smithi100.stdout:3288: finishing copy_from racing read to smithi10066867-39 2024-04-07T16:27:15.366 INFO:tasks.rados.rados.0.smithi100.stdout:3288: finishing copy_from to smithi10066867-39 2024-04-07T16:27:15.367 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 39 v 1538 (ObjNum 968 snap 293 seq_num 968) dirty exists 2024-04-07T16:27:15.367 INFO:tasks.rados.rados.0.smithi100.stdout:3302: seq_num 1012 ranges {6209536=155648} 2024-04-07T16:27:15.400 INFO:tasks.rados.rados.0.smithi100.stdout:3302: writing smithi10066867-34 from 6209536 to 6365184 tid 1 2024-04-07T16:27:15.400 INFO:tasks.rados.rados.0.smithi100.stdout:3288: done (4 left) 2024-04-07T16:27:15.400 INFO:tasks.rados.rados.0.smithi100.stdout:3298: done (3 left) 2024-04-07T16:27:15.400 INFO:tasks.rados.rados.0.smithi100.stdout:3299: done (2 left) 2024-04-07T16:27:15.400 INFO:tasks.rados.rados.0.smithi100.stdout:3300: done (1 left) 2024-04-07T16:27:15.400 INFO:tasks.rados.rados.0.smithi100.stdout:3303: read oid 11 snap 297 2024-04-07T16:27:15.400 INFO:tasks.rados.rados.0.smithi100.stdout:3303: expect (ObjNum 938 snap 281 seq_num 938) 2024-04-07T16:27:15.400 INFO:tasks.rados.rados.0.smithi100.stdout:3304: read oid 23 snap -1 2024-04-07T16:27:15.400 INFO:tasks.rados.rados.0.smithi100.stdout:3304: expect (ObjNum 941 snap 282 seq_num 941) 2024-04-07T16:27:15.401 INFO:tasks.rados.rados.0.smithi100.stdout:3305: copy_from oid 25 from oid 29 current snap is 300 2024-04-07T16:27:15.401 INFO:tasks.rados.rados.0.smithi100.stdout:3306: snap_create 2024-04-07T16:27:15.402 INFO:tasks.rados.rados.0.smithi100.stdout:3305: finishing copy_from to smithi10066867-25 2024-04-07T16:27:15.402 INFO:tasks.rados.rados.0.smithi100.stdout:3305: got expected ENOENT (src dne) 2024-04-07T16:27:15.402 INFO:tasks.rados.rados.0.smithi100.stdout:3305: finishing copy_from racing read to smithi10066867-25 2024-04-07T16:27:15.404 INFO:tasks.rados.rados.0.smithi100.stdout:3302: finishing write tid 1 to smithi10066867-34 2024-04-07T16:27:15.404 INFO:tasks.rados.rados.0.smithi100.stdout:3302: finishing write tid 2 to smithi10066867-34 2024-04-07T16:27:15.406 INFO:tasks.rados.rados.0.smithi100.stdout:3302: finishing write tid 3 to smithi10066867-34 2024-04-07T16:27:15.406 INFO:tasks.rados.rados.0.smithi100.stdout:3302: oid 34 updating version 0 to 1569 2024-04-07T16:27:15.406 INFO:tasks.rados.rados.0.smithi100.stdout:3302: oid 34 version 1569 is already newer than 1568 2024-04-07T16:27:15.406 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 34 v 1569 (ObjNum 1012 snap 300 seq_num 1012) dirty exists 2024-04-07T16:27:15.406 INFO:tasks.rados.rados.0.smithi100.stdout:3302: left oid 34 (ObjNum 1012 snap 300 seq_num 1012) 2024-04-07T16:27:15.425 INFO:tasks.rados.rados.0.smithi100.stdout:3303: expect (ObjNum 938 snap 281 seq_num 938) 2024-04-07T16:27:15.605 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:27:15 smithi144 bash[95117]: ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4-prometheus-a 2024-04-07T16:27:15.605 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:27:15 smithi144 bash[95178]: Error response from daemon: No such container: ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4-prometheus-a 2024-04-07T16:27:15.605 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:27:15 smithi144 systemd[1]: ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@prometheus.a.service: Deactivated successfully. 2024-04-07T16:27:15.605 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:27:15 smithi144 systemd[1]: Stopped Ceph prometheus.a for 0702e3a8-f4f8-11ee-b649-cb9ed24678a4. 2024-04-07T16:27:15.804 INFO:tasks.rados.rados.0.smithi100.stdout:3304: expect (ObjNum 941 snap 282 seq_num 941) 2024-04-07T16:27:15.927 INFO:journalctl@ceph.osd.7.smithi144.stdout:Apr 07 16:27:15 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:27:15.928 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:15 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:27:15.928 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:27:15 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:27:15.928 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:27:15 smithi144 systemd[1]: Started Ceph prometheus.a for 0702e3a8-f4f8-11ee-b649-cb9ed24678a4. 2024-04-07T16:27:15.928 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:27:15 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:27:15.929 INFO:journalctl@ceph.mds.b.smithi144.stdout:Apr 07 16:27:15 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:27:15.929 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:27:15 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:27:15.929 INFO:journalctl@ceph.osd.4.smithi144.stdout:Apr 07 16:27:15 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:27:15.930 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 16:27:15 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:27:15.930 INFO:journalctl@ceph.osd.5.smithi144.stdout:Apr 07 16:27:15 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:27:15.930 INFO:journalctl@ceph.osd.6.smithi144.stdout:Apr 07 16:27:15 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:27:15.943 INFO:tasks.rados.rados.0.smithi100.stdout:3302: done (4 left) 2024-04-07T16:27:15.943 INFO:tasks.rados.rados.0.smithi100.stdout:3303: done (3 left) 2024-04-07T16:27:15.943 INFO:tasks.rados.rados.0.smithi100.stdout:3304: done (2 left) 2024-04-07T16:27:15.943 INFO:tasks.rados.rados.0.smithi100.stdout:3305: done (1 left) 2024-04-07T16:27:15.944 INFO:tasks.rados.rados.0.smithi100.stdout:3306: done (0 left) 2024-04-07T16:27:15.944 INFO:tasks.rados.rados.0.smithi100.stdout:3307: read oid 31 snap -1 2024-04-07T16:27:15.944 INFO:tasks.rados.rados.0.smithi100.stdout:3307: expect (ObjNum 993 snap 300 seq_num 993) 2024-04-07T16:27:15.944 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 41 current snap is 301 2024-04-07T16:27:15.944 INFO:tasks.rados.rados.0.smithi100.stdout:3308: seq_num 1013 ranges {4096000=172032} 2024-04-07T16:27:15.989 INFO:tasks.rados.rados.0.smithi100.stdout:3308: writing smithi10066867-41 from 4096000 to 4268032 tid 1 2024-04-07T16:27:15.989 INFO:tasks.rados.rados.0.smithi100.stdout:3309: copy_from oid 37 from oid 8 current snap is 301 2024-04-07T16:27:15.989 INFO:tasks.rados.rados.0.smithi100.stdout:3310: snap_create 2024-04-07T16:27:15.989 INFO:tasks.rados.rados.0.smithi100.stdout:3307: expect (ObjNum 993 snap 300 seq_num 993) 2024-04-07T16:27:16.073 INFO:tasks.rados.rados.0.smithi100.stdout:3308: finishing write tid 1 to smithi10066867-41 2024-04-07T16:27:16.073 INFO:tasks.rados.rados.0.smithi100.stdout:3308: finishing write tid 2 to smithi10066867-41 2024-04-07T16:27:16.073 INFO:tasks.rados.rados.0.smithi100.stdout:3308: finishing write tid 3 to smithi10066867-41 2024-04-07T16:27:16.074 INFO:tasks.rados.rados.0.smithi100.stdout:3308: oid 41 updating version 0 to 1692 2024-04-07T16:27:16.074 INFO:tasks.rados.rados.0.smithi100.stdout:3308: oid 41 updating version 1692 to 1693 2024-04-07T16:27:16.074 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 41 v 1693 (ObjNum 1013 snap 301 seq_num 1013) dirty exists 2024-04-07T16:27:16.074 INFO:tasks.rados.rados.0.smithi100.stdout:3308: left oid 41 (ObjNum 1013 snap 301 seq_num 1013) 2024-04-07T16:27:16.074 INFO:tasks.rados.rados.0.smithi100.stdout:3309: finishing copy_from racing read to smithi10066867-37 2024-04-07T16:27:16.074 INFO:tasks.rados.rados.0.smithi100.stdout:3309: finishing copy_from to smithi10066867-37 2024-04-07T16:27:16.074 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 37 v 1538 (ObjNum 968 snap 293 seq_num 968) dirty exists 2024-04-07T16:27:16.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:15 smithi100 bash[74942]: cluster 2024-04-07T16:27:14.786253+0000 mgr.y (mgr.25175) 341 : cluster [DBG] pgmap v609: 121 pgs: 121 active+clean; 152 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 17 MiB/s rd, 3.1 MiB/s wr, 19 op/s 2024-04-07T16:27:16.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:15 smithi100 bash[74942]: audit 2024-04-07T16:27:15.820556+0000 mon.a (mon.0) 1121 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:27:16.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:15 smithi100 bash[74942]: audit 2024-04-07T16:27:15.829190+0000 mon.a (mon.0) 1122 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:27:16.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:15 smithi100 bash[74942]: audit 2024-04-07T16:27:15.838592+0000 mon.a (mon.0) 1123 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:27:16.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:15 smithi100 bash[76202]: cluster 2024-04-07T16:27:14.786253+0000 mgr.y (mgr.25175) 341 : cluster [DBG] pgmap v609: 121 pgs: 121 active+clean; 152 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 17 MiB/s rd, 3.1 MiB/s wr, 19 op/s 2024-04-07T16:27:16.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:15 smithi100 bash[76202]: audit 2024-04-07T16:27:15.820556+0000 mon.a (mon.0) 1121 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:27:16.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:15 smithi100 bash[76202]: audit 2024-04-07T16:27:15.829190+0000 mon.a (mon.0) 1122 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:27:16.188 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:15 smithi100 bash[76202]: audit 2024-04-07T16:27:15.838592+0000 mon.a (mon.0) 1123 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:27:16.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:15 smithi144 bash[66654]: cluster 2024-04-07T16:27:14.786253+0000 mgr.y (mgr.25175) 341 : cluster [DBG] pgmap v609: 121 pgs: 121 active+clean; 152 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 17 MiB/s rd, 3.1 MiB/s wr, 19 op/s 2024-04-07T16:27:16.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:15 smithi144 bash[66654]: audit 2024-04-07T16:27:15.820556+0000 mon.a (mon.0) 1121 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:27:16.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:15 smithi144 bash[66654]: audit 2024-04-07T16:27:15.829190+0000 mon.a (mon.0) 1122 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:27:16.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:15 smithi144 bash[66654]: audit 2024-04-07T16:27:15.838592+0000 mon.a (mon.0) 1123 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:27:16.950 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:27:16 smithi144 bash[95233]: ts=2024-04-07T16:27:16.815Z caller=main.go:564 level=info msg="Starting Prometheus Server" mode=server version="(version=2.43.0, branch=HEAD, revision=edfc3bcd025dd6fe296c167a14a216cab1e552ee)" 2024-04-07T16:27:16.951 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:27:16 smithi144 bash[95233]: ts=2024-04-07T16:27:16.815Z caller=main.go:569 level=info build_context="(go=go1.19.7, platform=linux/amd64, user=root@8a0ee342e522, date=20230321-12:56:07, tags=netgo,builtinassets)" 2024-04-07T16:27:16.951 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:27:16 smithi144 bash[95233]: ts=2024-04-07T16:27:16.816Z caller=main.go:570 level=info host_details="(Linux 5.15.0-101-generic #111-Ubuntu SMP Tue Mar 5 20:16:58 UTC 2024 x86_64 smithi144 (none))" 2024-04-07T16:27:16.951 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:27:16 smithi144 bash[95233]: ts=2024-04-07T16:27:16.816Z caller=main.go:571 level=info fd_limits="(soft=1048576, hard=1048576)" 2024-04-07T16:27:16.951 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:27:16 smithi144 bash[95233]: ts=2024-04-07T16:27:16.816Z caller=main.go:572 level=info vm_limits="(soft=unlimited, hard=unlimited)" 2024-04-07T16:27:16.951 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:27:16 smithi144 bash[95233]: ts=2024-04-07T16:27:16.817Z caller=web.go:561 level=info component=web msg="Start listening for connections" address=:9095 2024-04-07T16:27:16.951 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:27:16 smithi144 bash[95233]: ts=2024-04-07T16:27:16.818Z caller=main.go:1005 level=info msg="Starting TSDB ..." 2024-04-07T16:27:16.951 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:27:16 smithi144 bash[95233]: ts=2024-04-07T16:27:16.820Z caller=tls_config.go:232 level=info component=web msg="Listening on" address=[::]:9095 2024-04-07T16:27:16.951 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:27:16 smithi144 bash[95233]: ts=2024-04-07T16:27:16.820Z caller=tls_config.go:235 level=info component=web msg="TLS is disabled." http2=false address=[::]:9095 2024-04-07T16:27:16.951 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:27:16 smithi144 bash[95233]: ts=2024-04-07T16:27:16.822Z caller=head.go:587 level=info component=tsdb msg="Replaying on-disk memory mappable chunks if any" 2024-04-07T16:27:16.951 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:27:16 smithi144 bash[95233]: ts=2024-04-07T16:27:16.822Z caller=head.go:658 level=info component=tsdb msg="On-disk memory mappable chunks replay completed" duration=1.723µs 2024-04-07T16:27:16.951 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:27:16 smithi144 bash[95233]: ts=2024-04-07T16:27:16.822Z caller=head.go:664 level=info component=tsdb msg="Replaying WAL, this may take a while" 2024-04-07T16:27:16.951 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:27:16 smithi144 bash[95233]: ts=2024-04-07T16:27:16.824Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=0 maxSegment=4 2024-04-07T16:27:16.952 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:27:16 smithi144 bash[95233]: ts=2024-04-07T16:27:16.834Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=1 maxSegment=4 2024-04-07T16:27:16.952 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:27:16 smithi144 bash[95233]: ts=2024-04-07T16:27:16.843Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=2 maxSegment=4 2024-04-07T16:27:16.952 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:27:16 smithi144 bash[95233]: ts=2024-04-07T16:27:16.850Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=3 maxSegment=4 2024-04-07T16:27:16.952 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:27:16 smithi144 bash[95233]: ts=2024-04-07T16:27:16.850Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=4 maxSegment=4 2024-04-07T16:27:16.952 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:27:16 smithi144 bash[95233]: ts=2024-04-07T16:27:16.850Z caller=head.go:772 level=info component=tsdb msg="WAL replay completed" checkpoint_replay_duration=29.109µs wal_replay_duration=28.472693ms wbl_replay_duration=143ns total_replay_duration=28.521274ms 2024-04-07T16:27:16.952 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:27:16 smithi144 bash[95233]: ts=2024-04-07T16:27:16.853Z caller=main.go:1026 level=info fs_type=XFS_SUPER_MAGIC 2024-04-07T16:27:16.952 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:27:16 smithi144 bash[95233]: ts=2024-04-07T16:27:16.853Z caller=main.go:1029 level=info msg="TSDB started" 2024-04-07T16:27:16.952 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:27:16 smithi144 bash[95233]: ts=2024-04-07T16:27:16.853Z caller=main.go:1209 level=info msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml 2024-04-07T16:27:16.952 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:27:16 smithi144 bash[95233]: ts=2024-04-07T16:27:16.879Z caller=main.go:1246 level=info msg="Completed loading of configuration file" filename=/etc/prometheus/prometheus.yml totalDuration=25.76951ms db_storage=925ns remote_storage=1.151µs web_handler=396ns query_engine=827ns scrape=249.404µs scrape_sd=108.838µs notify=11.09µs notify_sd=20.238µs rules=24.943419ms tracing=6.882µs 2024-04-07T16:27:16.952 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:27:16 smithi144 bash[95233]: ts=2024-04-07T16:27:16.879Z caller=main.go:990 level=info msg="Server is ready to receive web requests." 2024-04-07T16:27:16.952 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:27:16 smithi144 bash[95233]: ts=2024-04-07T16:27:16.879Z caller=manager.go:974 level=info component="rule manager" msg="Starting rule manager..." 2024-04-07T16:27:16.960 INFO:tasks.rados.rados.0.smithi100.stdout:3307: done (3 left) 2024-04-07T16:27:16.960 INFO:tasks.rados.rados.0.smithi100.stdout:3308: done (2 left) 2024-04-07T16:27:16.960 INFO:tasks.rados.rados.0.smithi100.stdout:3309: done (1 left) 2024-04-07T16:27:16.960 INFO:tasks.rados.rados.0.smithi100.stdout:3310: done (0 left) 2024-04-07T16:27:16.960 INFO:tasks.rados.rados.0.smithi100.stdout:3311: read oid 38 snap 297 2024-04-07T16:27:16.960 INFO:tasks.rados.rados.0.smithi100.stdout:3311: expect (ObjNum 950 snap 285 seq_num 950) 2024-04-07T16:27:16.960 INFO:tasks.rados.rados.0.smithi100.stdout:3312: read oid 13 snap -1 2024-04-07T16:27:16.960 INFO:tasks.rados.rados.0.smithi100.stdout:3312: expect deleted 2024-04-07T16:27:16.960 INFO:tasks.rados.rados.0.smithi100.stdout:3313: snap_remove snap 301 2024-04-07T16:27:17.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:16 smithi144 bash[66654]: cluster 2024-04-07T16:27:15.958317+0000 mon.a (mon.0) 1124 : cluster [DBG] osdmap e678: 8 total, 8 up, 8 in 2024-04-07T16:27:17.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:16 smithi144 bash[66654]: audit 2024-04-07T16:27:16.728505+0000 mon.a (mon.0) 1125 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:27:17.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:16 smithi100 bash[74942]: cluster 2024-04-07T16:27:15.958317+0000 mon.a (mon.0) 1124 : cluster [DBG] osdmap e678: 8 total, 8 up, 8 in 2024-04-07T16:27:17.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:16 smithi100 bash[74942]: audit 2024-04-07T16:27:16.728505+0000 mon.a (mon.0) 1125 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:27:17.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:16 smithi100 bash[76202]: cluster 2024-04-07T16:27:15.958317+0000 mon.a (mon.0) 1124 : cluster [DBG] osdmap e678: 8 total, 8 up, 8 in 2024-04-07T16:27:17.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:16 smithi100 bash[76202]: audit 2024-04-07T16:27:16.728505+0000 mon.a (mon.0) 1125 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:27:17.956 INFO:tasks.rados.rados.0.smithi100.stdout:3313: done (2 left) 2024-04-07T16:27:17.956 INFO:tasks.rados.rados.0.smithi100.stdout:3314: setattr oid 11 current snap is 302 2024-04-07T16:27:17.956 INFO:tasks.rados.rados.0.smithi100.stdout:3311: expect (ObjNum 950 snap 285 seq_num 950) 2024-04-07T16:27:18.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:17 smithi144 bash[66654]: cluster 2024-04-07T16:27:16.786926+0000 mgr.y (mgr.25175) 342 : cluster [DBG] pgmap v611: 121 pgs: 121 active+clean; 152 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 17 MiB/s rd, 3.0 MiB/s wr, 23 op/s 2024-04-07T16:27:18.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:17 smithi144 bash[66654]: cluster 2024-04-07T16:27:16.970489+0000 mon.a (mon.0) 1126 : cluster [DBG] osdmap e679: 8 total, 8 up, 8 in 2024-04-07T16:27:18.396 INFO:tasks.rados.rados.0.smithi100.stdout:3311: done (2 left) 2024-04-07T16:27:18.396 INFO:tasks.rados.rados.0.smithi100.stdout:3312: done (1 left) 2024-04-07T16:27:18.397 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 24 current snap is 302 2024-04-07T16:27:18.397 INFO:tasks.rados.rados.0.smithi100.stdout:3315: seq_num 1014 ranges {10797056=188416} 2024-04-07T16:27:18.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:17 smithi100 bash[74942]: cluster 2024-04-07T16:27:16.786926+0000 mgr.y (mgr.25175) 342 : cluster [DBG] pgmap v611: 121 pgs: 121 active+clean; 152 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 17 MiB/s rd, 3.0 MiB/s wr, 23 op/s 2024-04-07T16:27:18.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:17 smithi100 bash[74942]: cluster 2024-04-07T16:27:16.970489+0000 mon.a (mon.0) 1126 : cluster [DBG] osdmap e679: 8 total, 8 up, 8 in 2024-04-07T16:27:18.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:17 smithi100 bash[76202]: cluster 2024-04-07T16:27:16.786926+0000 mgr.y (mgr.25175) 342 : cluster [DBG] pgmap v611: 121 pgs: 121 active+clean; 152 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 17 MiB/s rd, 3.0 MiB/s wr, 23 op/s 2024-04-07T16:27:18.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:17 smithi100 bash[76202]: cluster 2024-04-07T16:27:16.970489+0000 mon.a (mon.0) 1126 : cluster [DBG] osdmap e679: 8 total, 8 up, 8 in 2024-04-07T16:27:18.506 INFO:tasks.rados.rados.0.smithi100.stdout:3315: writing smithi10066867-24 from 10797056 to 10985472 tid 1 2024-04-07T16:27:18.506 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 3 current snap is 302 2024-04-07T16:27:18.506 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 11 v 1771 (ObjNum 990 snap 298 seq_num 990) dirty exists 2024-04-07T16:27:18.506 INFO:tasks.rados.rados.0.smithi100.stdout:3316: seq_num 1015 ranges {6406144=204800} 2024-04-07T16:27:18.544 INFO:tasks.rados.rados.0.smithi100.stdout:3316: writing smithi10066867-3 from 6406144 to 6610944 tid 1 2024-04-07T16:27:18.544 INFO:tasks.rados.rados.0.smithi100.stdout:3314: done (2 left) 2024-04-07T16:27:18.544 INFO:tasks.rados.rados.0.smithi100.stdout:3317: copy_from oid 33 from oid 36 current snap is 302 2024-04-07T16:27:18.544 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 35 current snap is 302 2024-04-07T16:27:18.544 INFO:tasks.rados.rados.0.smithi100.stdout:3318: seq_num 1016 ranges {1507328=221184} 2024-04-07T16:27:18.555 INFO:tasks.rados.rados.0.smithi100.stdout:3318: writing smithi10066867-35 from 1507328 to 1728512 tid 1 2024-04-07T16:27:18.555 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 39 current snap is 302 2024-04-07T16:27:18.555 INFO:tasks.rados.rados.0.smithi100.stdout:3315: finishing write tid 1 to smithi10066867-24 2024-04-07T16:27:18.555 INFO:tasks.rados.rados.0.smithi100.stdout:3315: finishing write tid 2 to smithi10066867-24 2024-04-07T16:27:18.555 INFO:tasks.rados.rados.0.smithi100.stdout:3315: finishing write tid 3 to smithi10066867-24 2024-04-07T16:27:18.555 INFO:tasks.rados.rados.0.smithi100.stdout:3315: oid 24 updating version 0 to 1263 2024-04-07T16:27:18.555 INFO:tasks.rados.rados.0.smithi100.stdout:3315: oid 24 updating version 1263 to 1264 2024-04-07T16:27:18.555 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 24 v 1264 (ObjNum 1014 snap 302 seq_num 1014) dirty exists 2024-04-07T16:27:18.555 INFO:tasks.rados.rados.0.smithi100.stdout:3315: left oid 24 (ObjNum 1014 snap 302 seq_num 1014) 2024-04-07T16:27:18.555 INFO:tasks.rados.rados.0.smithi100.stdout:3316: finishing write tid 1 to smithi10066867-3 2024-04-07T16:27:18.555 INFO:tasks.rados.rados.0.smithi100.stdout:3316: finishing write tid 2 to smithi10066867-3 2024-04-07T16:27:18.555 INFO:tasks.rados.rados.0.smithi100.stdout:3319: seq_num 1017 ranges {3620864=237568} 2024-04-07T16:27:18.577 INFO:tasks.rados.rados.0.smithi100.stdout:3319: writing smithi10066867-39 from 3620864 to 3858432 tid 1 2024-04-07T16:27:18.577 INFO:tasks.rados.rados.0.smithi100.stdout:3315: done (4 left) 2024-04-07T16:27:18.577 INFO:tasks.rados.rados.0.smithi100.stdout:3320: snap_create 2024-04-07T16:27:18.577 INFO:tasks.rados.rados.0.smithi100.stdout:3316: finishing write tid 3 to smithi10066867-3 2024-04-07T16:27:18.577 INFO:tasks.rados.rados.0.smithi100.stdout:3316: oid 3 updating version 0 to 1692 2024-04-07T16:27:18.577 INFO:tasks.rados.rados.0.smithi100.stdout:3316: oid 3 updating version 1692 to 1693 2024-04-07T16:27:18.578 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 3 v 1693 (ObjNum 1015 snap 302 seq_num 1015) dirty exists 2024-04-07T16:27:18.578 INFO:tasks.rados.rados.0.smithi100.stdout:3316: left oid 3 (ObjNum 1015 snap 302 seq_num 1015) 2024-04-07T16:27:18.578 INFO:tasks.rados.rados.0.smithi100.stdout:3317: finishing copy_from racing read to smithi10066867-33 2024-04-07T16:27:18.578 INFO:tasks.rados.rados.0.smithi100.stdout:3318: finishing write tid 1 to smithi10066867-35 2024-04-07T16:27:18.578 INFO:tasks.rados.rados.0.smithi100.stdout:3318: finishing write tid 2 to smithi10066867-35 2024-04-07T16:27:18.578 INFO:tasks.rados.rados.0.smithi100.stdout:3318: finishing write tid 3 to smithi10066867-35 2024-04-07T16:27:18.578 INFO:tasks.rados.rados.0.smithi100.stdout:3318: oid 35 updating version 0 to 1698 2024-04-07T16:27:18.578 INFO:tasks.rados.rados.0.smithi100.stdout:3318: oid 35 updating version 1698 to 1699 2024-04-07T16:27:18.578 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 35 v 1699 (ObjNum 1016 snap 302 seq_num 1016) dirty exists 2024-04-07T16:27:18.578 INFO:tasks.rados.rados.0.smithi100.stdout:3318: left oid 35 (ObjNum 1016 snap 302 seq_num 1016) 2024-04-07T16:27:18.578 INFO:tasks.rados.rados.0.smithi100.stdout:3317: finishing copy_from to smithi10066867-33 2024-04-07T16:27:18.578 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 33 v 1240 (ObjNum 949 snap 285 seq_num 949) dirty exists 2024-04-07T16:27:18.581 INFO:tasks.rados.rados.0.smithi100.stdout:3319: finishing write tid 1 to smithi10066867-39 2024-04-07T16:27:18.581 INFO:tasks.rados.rados.0.smithi100.stdout:3319: finishing write tid 2 to smithi10066867-39 2024-04-07T16:27:18.584 INFO:tasks.rados.rados.0.smithi100.stdout:3319: finishing write tid 3 to smithi10066867-39 2024-04-07T16:27:18.584 INFO:tasks.rados.rados.0.smithi100.stdout:3319: oid 39 updating version 0 to 1539 2024-04-07T16:27:18.585 INFO:tasks.rados.rados.0.smithi100.stdout:3319: oid 39 updating version 1539 to 1540 2024-04-07T16:27:18.585 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 39 v 1540 (ObjNum 1017 snap 302 seq_num 1017) dirty exists 2024-04-07T16:27:18.585 INFO:tasks.rados.rados.0.smithi100.stdout:3319: left oid 39 (ObjNum 1017 snap 302 seq_num 1017) 2024-04-07T16:27:18.983 INFO:tasks.rados.rados.0.smithi100.stdout:3316: done (4 left) 2024-04-07T16:27:18.983 INFO:tasks.rados.rados.0.smithi100.stdout:3317: done (3 left) 2024-04-07T16:27:18.983 INFO:tasks.rados.rados.0.smithi100.stdout:3318: done (2 left) 2024-04-07T16:27:18.983 INFO:tasks.rados.rados.0.smithi100.stdout:3319: done (1 left) 2024-04-07T16:27:18.983 INFO:tasks.rados.rados.0.smithi100.stdout:3320: done (0 left) 2024-04-07T16:27:18.983 INFO:tasks.rados.rados.0.smithi100.stdout:3321: snap_create 2024-04-07T16:27:19.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:18 smithi144 bash[66654]: cluster 2024-04-07T16:27:17.970230+0000 mon.a (mon.0) 1127 : cluster [DBG] osdmap e680: 8 total, 8 up, 8 in 2024-04-07T16:27:19.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:18 smithi100 bash[74942]: cluster 2024-04-07T16:27:17.970230+0000 mon.a (mon.0) 1127 : cluster [DBG] osdmap e680: 8 total, 8 up, 8 in 2024-04-07T16:27:19.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:18 smithi100 bash[76202]: cluster 2024-04-07T16:27:17.970230+0000 mon.a (mon.0) 1127 : cluster [DBG] osdmap e680: 8 total, 8 up, 8 in 2024-04-07T16:27:19.986 INFO:tasks.rados.rados.0.smithi100.stdout:3321: done (0 left) 2024-04-07T16:27:19.987 INFO:tasks.rados.rados.0.smithi100.stdout:3322: snap_remove snap 297 2024-04-07T16:27:20.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:20 smithi144 bash[66654]: cluster 2024-04-07T16:27:18.787800+0000 mgr.y (mgr.25175) 343 : cluster [DBG] pgmap v614: 121 pgs: 121 active+clean; 155 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 4.5 MiB/s rd, 2.4 MiB/s wr, 12 op/s 2024-04-07T16:27:20.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:20 smithi144 bash[66654]: cluster 2024-04-07T16:27:18.998192+0000 mon.a (mon.0) 1128 : cluster [DBG] osdmap e681: 8 total, 8 up, 8 in 2024-04-07T16:27:20.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:20 smithi100 bash[74942]: cluster 2024-04-07T16:27:18.787800+0000 mgr.y (mgr.25175) 343 : cluster [DBG] pgmap v614: 121 pgs: 121 active+clean; 155 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 4.5 MiB/s rd, 2.4 MiB/s wr, 12 op/s 2024-04-07T16:27:20.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:20 smithi100 bash[74942]: cluster 2024-04-07T16:27:18.998192+0000 mon.a (mon.0) 1128 : cluster [DBG] osdmap e681: 8 total, 8 up, 8 in 2024-04-07T16:27:20.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:20 smithi100 bash[76202]: cluster 2024-04-07T16:27:18.787800+0000 mgr.y (mgr.25175) 343 : cluster [DBG] pgmap v614: 121 pgs: 121 active+clean; 155 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 4.5 MiB/s rd, 2.4 MiB/s wr, 12 op/s 2024-04-07T16:27:20.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:20 smithi100 bash[76202]: cluster 2024-04-07T16:27:18.998192+0000 mon.a (mon.0) 1128 : cluster [DBG] osdmap e681: 8 total, 8 up, 8 in 2024-04-07T16:27:21.007 INFO:tasks.rados.rados.0.smithi100.stdout:3322: done (0 left) 2024-04-07T16:27:21.007 INFO:tasks.rados.rados.0.smithi100.stdout:3323: rollback oid 24 current snap is 304 2024-04-07T16:27:21.007 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 24 to 300 2024-04-07T16:27:21.008 INFO:tasks.rados.rados.0.smithi100.stdout:3324: copy_from oid 33 from oid 15 current snap is 304 2024-04-07T16:27:21.008 INFO:tasks.rados.rados.0.smithi100.stdout:3325: snap_remove snap 302 2024-04-07T16:27:21.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:21 smithi144 bash[66654]: cluster 2024-04-07T16:27:19.997114+0000 mon.a (mon.0) 1129 : cluster [DBG] osdmap e682: 8 total, 8 up, 8 in 2024-04-07T16:27:21.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:21 smithi100 bash[74942]: cluster 2024-04-07T16:27:19.997114+0000 mon.a (mon.0) 1129 : cluster [DBG] osdmap e682: 8 total, 8 up, 8 in 2024-04-07T16:27:21.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:21 smithi100 bash[76202]: cluster 2024-04-07T16:27:19.997114+0000 mon.a (mon.0) 1129 : cluster [DBG] osdmap e682: 8 total, 8 up, 8 in 2024-04-07T16:27:21.916 INFO:tasks.rados.rados.0.smithi100.stdout:3325: done (2 left) 2024-04-07T16:27:21.916 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 40 current snap is 304 2024-04-07T16:27:21.916 INFO:tasks.rados.rados.0.smithi100.stdout:3323: finishing rollback tid 0 to smithi10066867-24 2024-04-07T16:27:21.916 INFO:tasks.rados.rados.0.smithi100.stdout:3326: seq_num 1018 ranges {1753088=253952} 2024-04-07T16:27:21.934 INFO:tasks.rados.rados.0.smithi100.stdout:3326: writing smithi10066867-40 from 1753088 to 2007040 tid 1 2024-04-07T16:27:21.934 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 21 current snap is 304 2024-04-07T16:27:21.934 INFO:tasks.rados.rados.0.smithi100.stdout:3323: finishing rollback tid 1 to smithi10066867-24 2024-04-07T16:27:21.934 INFO:tasks.rados.rados.0.smithi100.stdout:3323: finishing rollback tid 2 to smithi10066867-24 2024-04-07T16:27:21.934 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 24 v 1267 (ObjNum 989 snap 298 seq_num 989) dirty exists 2024-04-07T16:27:21.934 INFO:tasks.rados.rados.0.smithi100.stdout:3324: finishing copy_from racing read to smithi10066867-33 2024-04-07T16:27:21.934 INFO:tasks.rados.rados.0.smithi100.stdout:3324: finishing copy_from to smithi10066867-33 2024-04-07T16:27:21.934 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 33 v 1446 (ObjNum 913 snap 274 seq_num 913) dirty exists 2024-04-07T16:27:21.934 INFO:tasks.rados.rados.0.smithi100.stdout:3327: seq_num 1019 ranges {1302528=270336} 2024-04-07T16:27:21.947 INFO:tasks.rados.rados.0.smithi100.stdout:3327: writing smithi10066867-21 from 1302528 to 1572864 tid 1 2024-04-07T16:27:21.947 INFO:tasks.rados.rados.0.smithi100.stdout:3323: done (3 left) 2024-04-07T16:27:21.947 INFO:tasks.rados.rados.0.smithi100.stdout:3324: done (2 left) 2024-04-07T16:27:21.948 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 32 current snap is 304 2024-04-07T16:27:21.948 INFO:tasks.rados.rados.0.smithi100.stdout:3328: seq_num 1020 ranges {0=286720} 2024-04-07T16:27:21.950 INFO:tasks.rados.rados.0.smithi100.stdout:3328: writing smithi10066867-32 from 0 to 286720 tid 1 2024-04-07T16:27:21.950 INFO:tasks.rados.rados.0.smithi100.stdout:3329: delete oid 36 current snap is 304 2024-04-07T16:27:21.950 INFO:tasks.rados.rados.0.smithi100.stdout:3326: finishing write tid 1 to smithi10066867-40 2024-04-07T16:27:21.950 INFO:tasks.rados.rados.0.smithi100.stdout:3326: finishing write tid 2 to smithi10066867-40 2024-04-07T16:27:21.950 INFO:tasks.rados.rados.0.smithi100.stdout:3326: finishing write tid 3 to smithi10066867-40 2024-04-07T16:27:21.950 INFO:tasks.rados.rados.0.smithi100.stdout:3326: oid 40 updating version 0 to 1641 2024-04-07T16:27:21.950 INFO:tasks.rados.rados.0.smithi100.stdout:3326: oid 40 updating version 1641 to 1642 2024-04-07T16:27:21.950 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 40 v 1642 (ObjNum 1018 snap 304 seq_num 1018) dirty exists 2024-04-07T16:27:21.950 INFO:tasks.rados.rados.0.smithi100.stdout:3326: left oid 40 (ObjNum 1018 snap 304 seq_num 1018) 2024-04-07T16:27:21.955 INFO:tasks.rados.rados.0.smithi100.stdout:3327: finishing write tid 1 to smithi10066867-21 2024-04-07T16:27:21.956 INFO:tasks.rados.rados.0.smithi100.stdout:3327: finishing write tid 2 to smithi10066867-21 2024-04-07T16:27:21.958 INFO:tasks.rados.rados.0.smithi100.stdout:3328: finishing write tid 1 to smithi10066867-32 2024-04-07T16:27:21.958 INFO:tasks.rados.rados.0.smithi100.stdout:3326: done (3 left) 2024-04-07T16:27:21.958 INFO:tasks.rados.rados.0.smithi100.stdout:3329: done (2 left) 2024-04-07T16:27:21.958 INFO:tasks.rados.rados.0.smithi100.stdout:3330: snap_create 2024-04-07T16:27:21.959 INFO:tasks.rados.rados.0.smithi100.stdout:3328: finishing write tid 2 to smithi10066867-32 2024-04-07T16:27:21.961 INFO:tasks.rados.rados.0.smithi100.stdout:3328: finishing write tid 3 to smithi10066867-32 2024-04-07T16:27:21.961 INFO:tasks.rados.rados.0.smithi100.stdout:3328: oid 32 updating version 0 to 1692 2024-04-07T16:27:21.961 INFO:tasks.rados.rados.0.smithi100.stdout:3328: oid 32 updating version 1692 to 1693 2024-04-07T16:27:21.961 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 32 v 1693 (ObjNum 1020 snap 304 seq_num 1020) dirty exists 2024-04-07T16:27:21.961 INFO:tasks.rados.rados.0.smithi100.stdout:3328: left oid 32 (ObjNum 1020 snap 304 seq_num 1020) 2024-04-07T16:27:21.962 INFO:tasks.rados.rados.0.smithi100.stdout:3327: finishing write tid 3 to smithi10066867-21 2024-04-07T16:27:21.963 INFO:tasks.rados.rados.0.smithi100.stdout:3327: oid 21 updating version 0 to 1207 2024-04-07T16:27:21.963 INFO:tasks.rados.rados.0.smithi100.stdout:3327: oid 21 updating version 1207 to 1208 2024-04-07T16:27:21.963 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 21 v 1208 (ObjNum 1019 snap 304 seq_num 1019) dirty exists 2024-04-07T16:27:21.963 INFO:tasks.rados.rados.0.smithi100.stdout:3327: left oid 21 (ObjNum 1019 snap 304 seq_num 1019) 2024-04-07T16:27:22.303 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:22 smithi144 bash[66654]: cluster 2024-04-07T16:27:20.788566+0000 mgr.y (mgr.25175) 344 : cluster [DBG] pgmap v617: 121 pgs: 121 active+clean; 158 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 14 MiB/s rd, 3.1 MiB/s wr, 23 op/s 2024-04-07T16:27:22.303 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:22 smithi144 bash[66654]: cluster 2024-04-07T16:27:20.999623+0000 mon.a (mon.0) 1130 : cluster [DBG] osdmap e683: 8 total, 8 up, 8 in 2024-04-07T16:27:22.304 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:22 smithi144 bash[66654]: audit 2024-04-07T16:27:21.781021+0000 mon.a (mon.0) 1131 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:27:22.304 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:22 smithi144 bash[66654]: audit 2024-04-07T16:27:21.792775+0000 mon.a (mon.0) 1132 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:27:22.304 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:22 smithi144 bash[66654]: cluster 2024-04-07T16:27:21.930322+0000 mon.a (mon.0) 1133 : cluster [DBG] osdmap e684: 8 total, 8 up, 8 in 2024-04-07T16:27:22.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:22 smithi100 bash[74942]: cluster 2024-04-07T16:27:20.788566+0000 mgr.y (mgr.25175) 344 : cluster [DBG] pgmap v617: 121 pgs: 121 active+clean; 158 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 14 MiB/s rd, 3.1 MiB/s wr, 23 op/s 2024-04-07T16:27:22.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:22 smithi100 bash[74942]: cluster 2024-04-07T16:27:20.999623+0000 mon.a (mon.0) 1130 : cluster [DBG] osdmap e683: 8 total, 8 up, 8 in 2024-04-07T16:27:22.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:22 smithi100 bash[74942]: audit 2024-04-07T16:27:21.781021+0000 mon.a (mon.0) 1131 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:27:22.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:22 smithi100 bash[74942]: audit 2024-04-07T16:27:21.792775+0000 mon.a (mon.0) 1132 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:27:22.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:22 smithi100 bash[74942]: cluster 2024-04-07T16:27:21.930322+0000 mon.a (mon.0) 1133 : cluster [DBG] osdmap e684: 8 total, 8 up, 8 in 2024-04-07T16:27:22.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:22 smithi100 bash[76202]: cluster 2024-04-07T16:27:20.788566+0000 mgr.y (mgr.25175) 344 : cluster [DBG] pgmap v617: 121 pgs: 121 active+clean; 158 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 14 MiB/s rd, 3.1 MiB/s wr, 23 op/s 2024-04-07T16:27:22.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:22 smithi100 bash[76202]: cluster 2024-04-07T16:27:20.999623+0000 mon.a (mon.0) 1130 : cluster [DBG] osdmap e683: 8 total, 8 up, 8 in 2024-04-07T16:27:22.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:22 smithi100 bash[76202]: audit 2024-04-07T16:27:21.781021+0000 mon.a (mon.0) 1131 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:27:22.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:22 smithi100 bash[76202]: audit 2024-04-07T16:27:21.792775+0000 mon.a (mon.0) 1132 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:27:22.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:22 smithi100 bash[76202]: cluster 2024-04-07T16:27:21.930322+0000 mon.a (mon.0) 1133 : cluster [DBG] osdmap e684: 8 total, 8 up, 8 in 2024-04-07T16:27:22.605 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:27:22 smithi144 bash[95233]: ts=2024-04-07T16:27:22.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:27:22.923 INFO:tasks.rados.rados.0.smithi100.stdout:3327: done (2 left) 2024-04-07T16:27:22.923 INFO:tasks.rados.rados.0.smithi100.stdout:3328: done (1 left) 2024-04-07T16:27:22.924 INFO:tasks.rados.rados.0.smithi100.stdout:3330: done (0 left) 2024-04-07T16:27:22.924 INFO:tasks.rados.rados.0.smithi100.stdout:3331: read oid 21 snap 303 2024-04-07T16:27:22.924 INFO:tasks.rados.rados.0.smithi100.stdout:3331: expect (ObjNum 794 snap 251 seq_num 794) 2024-04-07T16:27:22.924 INFO:tasks.rados.rados.0.smithi100.stdout:3332: delete oid 3 current snap is 305 2024-04-07T16:27:22.934 INFO:tasks.rados.rados.0.smithi100.stdout:3331: expect (ObjNum 794 snap 251 seq_num 794) 2024-04-07T16:27:23.014 INFO:tasks.rados.rados.0.smithi100.stdout:3331: done (1 left) 2024-04-07T16:27:23.014 INFO:tasks.rados.rados.0.smithi100.stdout:3332: done (0 left) 2024-04-07T16:27:23.014 INFO:tasks.rados.rados.0.smithi100.stdout:3333: read oid 23 snap 300 2024-04-07T16:27:23.014 INFO:tasks.rados.rados.0.smithi100.stdout:3333: expect (ObjNum 941 snap 282 seq_num 941) 2024-04-07T16:27:23.014 INFO:tasks.rados.rados.0.smithi100.stdout:3334: setattr oid 29 current snap is 305 2024-04-07T16:27:23.016 INFO:tasks.rados.rados.0.smithi100.stdout:3335: copy_from oid 21 from oid 16 current snap is 305 2024-04-07T16:27:23.016 INFO:tasks.rados.rados.0.smithi100.stdout:3336: delete oid 4 current snap is 305 2024-04-07T16:27:23.024 INFO:tasks.rados.rados.0.smithi100.stdout:3333: expect (ObjNum 941 snap 282 seq_num 941) 2024-04-07T16:27:23.102 INFO:tasks.rados.rados.0.smithi100.stdout:3335: finishing copy_from to smithi10066867-21 2024-04-07T16:27:23.102 INFO:tasks.rados.rados.0.smithi100.stdout:3335: got expected ENOENT (src dne) 2024-04-07T16:27:23.103 INFO:tasks.rados.rados.0.smithi100.stdout:3335: finishing copy_from racing read to smithi10066867-21 2024-04-07T16:27:23.103 INFO:tasks.rados.rados.0.smithi100.stdout:3333: done (3 left) 2024-04-07T16:27:23.103 INFO:tasks.rados.rados.0.smithi100.stdout:3335: done (2 left) 2024-04-07T16:27:23.103 INFO:tasks.rados.rados.0.smithi100.stdout:3336: done (1 left) 2024-04-07T16:27:23.103 INFO:tasks.rados.rados.0.smithi100.stdout:3337: snap_create 2024-04-07T16:27:23.103 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 29 v 1541 (ObjNum 65 snap 0 seq_num 1650155862) dirty exists 2024-04-07T16:27:24.026 INFO:tasks.rados.rados.0.smithi100.stdout:3334: done (1 left) 2024-04-07T16:27:24.026 INFO:tasks.rados.rados.0.smithi100.stdout:3337: done (0 left) 2024-04-07T16:27:24.026 INFO:tasks.rados.rados.0.smithi100.stdout:3338: copy_from oid 14 from oid 19 current snap is 306 2024-04-07T16:27:24.026 INFO:tasks.rados.rados.0.smithi100.stdout:3339: rmattr oid 49 current snap is 306 2024-04-07T16:27:24.026 INFO:tasks.rados.rados.0.smithi100.stdout:3339: done (1 left) 2024-04-07T16:27:24.026 INFO:tasks.rados.rados.0.smithi100.stdout:3340: delete oid 29 current snap is 306 2024-04-07T16:27:24.032 INFO:tasks.rados.rados.0.smithi100.stdout:3340: done (1 left) 2024-04-07T16:27:24.032 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 39 current snap is 306 2024-04-07T16:27:24.032 INFO:tasks.rados.rados.0.smithi100.stdout:3341: seq_num 1025 ranges {3858432=368640} 2024-04-07T16:27:24.068 INFO:tasks.rados.rados.0.smithi100.stdout:3341: writing smithi10066867-39 from 3858432 to 4227072 tid 1 2024-04-07T16:27:24.068 INFO:tasks.rados.rados.0.smithi100.stdout:3342: delete oid 5 current snap is 306 2024-04-07T16:27:24.068 INFO:tasks.rados.rados.0.smithi100.stdout:3338: finishing copy_from racing read to smithi10066867-14 2024-04-07T16:27:24.068 INFO:tasks.rados.rados.0.smithi100.stdout:3338: finishing copy_from to smithi10066867-14 2024-04-07T16:27:24.068 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 14 v 1262 (ObjNum 1009 snap 300 seq_num 1009) dirty exists 2024-04-07T16:27:24.074 INFO:tasks.rados.rados.0.smithi100.stdout:3338: done (2 left) 2024-04-07T16:27:24.074 INFO:tasks.rados.rados.0.smithi100.stdout:3342: done (1 left) 2024-04-07T16:27:24.074 INFO:tasks.rados.rados.0.smithi100.stdout:3343: rmattr oid 47 current snap is 306 2024-04-07T16:27:24.074 INFO:tasks.rados.rados.0.smithi100.stdout:3343: done (1 left) 2024-04-07T16:27:24.075 INFO:tasks.rados.rados.0.smithi100.stdout:3344: read oid 47 snap 300 2024-04-07T16:27:24.075 INFO:tasks.rados.rados.0.smithi100.stdout:3344: expect (ObjNum 996 snap 300 seq_num 996) 2024-04-07T16:27:24.075 INFO:tasks.rados.rados.0.smithi100.stdout:3345: read oid 15 snap -1 2024-04-07T16:27:24.075 INFO:tasks.rados.rados.0.smithi100.stdout:3345: expect (ObjNum 913 snap 274 seq_num 913) 2024-04-07T16:27:24.075 INFO:tasks.rados.rados.0.smithi100.stdout:3346: rollback oid 37 current snap is 306 2024-04-07T16:27:24.075 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 37 to 303 2024-04-07T16:27:24.075 INFO:tasks.rados.rados.0.smithi100.stdout:3341: finishing write tid 1 to smithi10066867-39 2024-04-07T16:27:24.075 INFO:tasks.rados.rados.0.smithi100.stdout:3347: copy_from oid 49 from oid 14 current snap is 306 2024-04-07T16:27:24.075 INFO:tasks.rados.rados.0.smithi100.stdout:3348: read oid 40 snap 303 2024-04-07T16:27:24.075 INFO:tasks.rados.rados.0.smithi100.stdout:3348: expect (ObjNum 1004 snap 300 seq_num 1004) 2024-04-07T16:27:24.075 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 18 current snap is 306 2024-04-07T16:27:24.075 INFO:tasks.rados.rados.0.smithi100.stdout:3349: seq_num 1027 ranges {0=409600} 2024-04-07T16:27:24.079 INFO:tasks.rados.rados.0.smithi100.stdout:3349: writing smithi10066867-18 from 0 to 409600 tid 1 2024-04-07T16:27:24.079 INFO:tasks.rados.rados.0.smithi100.stdout:3350: read oid 21 snap -1 2024-04-07T16:27:24.079 INFO:tasks.rados.rados.0.smithi100.stdout:3350: expect (ObjNum 1019 snap 304 seq_num 1019) 2024-04-07T16:27:24.079 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 27 current snap is 306 2024-04-07T16:27:24.079 INFO:tasks.rados.rados.0.smithi100.stdout:3341: finishing write tid 2 to smithi10066867-39 2024-04-07T16:27:24.091 INFO:tasks.rados.rados.0.smithi100.stdout:3351: seq_num 1028 ranges {4014080=425984} 2024-04-07T16:27:24.112 INFO:tasks.rados.rados.0.smithi100.stdout:3351: writing smithi10066867-27 from 4014080 to 4440064 tid 1 2024-04-07T16:27:24.112 INFO:tasks.rados.rados.0.smithi100.stdout:3352: copy_from oid 33 from oid 8 current snap is 306 2024-04-07T16:27:24.112 INFO:tasks.rados.rados.0.smithi100.stdout:3344: expect (ObjNum 996 snap 300 seq_num 996) 2024-04-07T16:27:24.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:23 smithi100 bash[76202]: cluster 2024-04-07T16:27:22.789763+0000 mgr.y (mgr.25175) 345 : cluster [DBG] pgmap v620: 121 pgs: 121 active+clean; 148 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 16 MiB/s rd, 824 KiB/s wr, 18 op/s 2024-04-07T16:27:24.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:23 smithi100 bash[76202]: audit 2024-04-07T16:27:22.897949+0000 mon.a (mon.0) 1134 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:27:24.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:23 smithi100 bash[76202]: audit 2024-04-07T16:27:22.909624+0000 mon.a (mon.0) 1135 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:27:24.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:23 smithi100 bash[76202]: cluster 2024-04-07T16:27:22.937959+0000 mon.a (mon.0) 1136 : cluster [DBG] osdmap e685: 8 total, 8 up, 8 in 2024-04-07T16:27:24.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:23 smithi100 bash[74942]: cluster 2024-04-07T16:27:22.789763+0000 mgr.y (mgr.25175) 345 : cluster [DBG] pgmap v620: 121 pgs: 121 active+clean; 148 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 16 MiB/s rd, 824 KiB/s wr, 18 op/s 2024-04-07T16:27:24.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:23 smithi100 bash[74942]: audit 2024-04-07T16:27:22.897949+0000 mon.a (mon.0) 1134 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:27:24.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:23 smithi100 bash[74942]: audit 2024-04-07T16:27:22.909624+0000 mon.a (mon.0) 1135 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:27:24.188 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:23 smithi100 bash[74942]: cluster 2024-04-07T16:27:22.937959+0000 mon.a (mon.0) 1136 : cluster [DBG] osdmap e685: 8 total, 8 up, 8 in 2024-04-07T16:27:24.200 INFO:tasks.rados.rados.0.smithi100.stdout:3346: finishing rollback tid 0 to smithi10066867-37 2024-04-07T16:27:24.200 INFO:tasks.rados.rados.0.smithi100.stdout:3347: finishing copy_from racing read to smithi10066867-49 2024-04-07T16:27:24.200 INFO:tasks.rados.rados.0.smithi100.stdout:3341: finishing write tid 3 to smithi10066867-39 2024-04-07T16:27:24.200 INFO:tasks.rados.rados.0.smithi100.stdout:3341: oid 39 updating version 0 to 1544 2024-04-07T16:27:24.200 INFO:tasks.rados.rados.0.smithi100.stdout:3341: oid 39 version 1544 is already newer than 1543 2024-04-07T16:27:24.200 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 39 v 1544 (ObjNum 1025 snap 306 seq_num 1025) dirty exists 2024-04-07T16:27:24.200 INFO:tasks.rados.rados.0.smithi100.stdout:3341: left oid 39 (ObjNum 1025 snap 306 seq_num 1025) 2024-04-07T16:27:24.201 INFO:tasks.rados.rados.0.smithi100.stdout:3346: finishing rollback tid 1 to smithi10066867-37 2024-04-07T16:27:24.201 INFO:tasks.rados.rados.0.smithi100.stdout:3350: expect (ObjNum 1019 snap 304 seq_num 1019) 2024-04-07T16:27:24.262 INFO:tasks.rados.rados.0.smithi100.stdout:3346: finishing rollback tid 2 to smithi10066867-37 2024-04-07T16:27:24.262 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 37 v 1547 (ObjNum 968 snap 293 seq_num 968) dirty exists 2024-04-07T16:27:24.262 INFO:tasks.rados.rados.0.smithi100.stdout:3347: finishing copy_from to smithi10066867-49 2024-04-07T16:27:24.262 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 49 v 1262 (ObjNum 1009 snap 300 seq_num 1009) dirty exists 2024-04-07T16:27:24.262 INFO:tasks.rados.rados.0.smithi100.stdout:3345: expect (ObjNum 913 snap 274 seq_num 913) 2024-04-07T16:27:24.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:23 smithi144 bash[66654]: cluster 2024-04-07T16:27:22.789763+0000 mgr.y (mgr.25175) 345 : cluster [DBG] pgmap v620: 121 pgs: 121 active+clean; 148 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 16 MiB/s rd, 824 KiB/s wr, 18 op/s 2024-04-07T16:27:24.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:23 smithi144 bash[66654]: audit 2024-04-07T16:27:22.897949+0000 mon.a (mon.0) 1134 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:27:24.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:23 smithi144 bash[66654]: audit 2024-04-07T16:27:22.909624+0000 mon.a (mon.0) 1135 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:27:24.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:23 smithi144 bash[66654]: cluster 2024-04-07T16:27:22.937959+0000 mon.a (mon.0) 1136 : cluster [DBG] osdmap e685: 8 total, 8 up, 8 in 2024-04-07T16:27:24.426 INFO:tasks.rados.rados.0.smithi100.stdout:3348: expect (ObjNum 1004 snap 300 seq_num 1004) 2024-04-07T16:27:24.501 INFO:tasks.rados.rados.0.smithi100.stdout:3349: finishing write tid 1 to smithi10066867-18 2024-04-07T16:27:24.501 INFO:tasks.rados.rados.0.smithi100.stdout:3349: finishing write tid 2 to smithi10066867-18 2024-04-07T16:27:24.501 INFO:tasks.rados.rados.0.smithi100.stdout:3349: finishing write tid 3 to smithi10066867-18 2024-04-07T16:27:24.501 INFO:tasks.rados.rados.0.smithi100.stdout:3349: oid 18 updating version 0 to 1791 2024-04-07T16:27:24.501 INFO:tasks.rados.rados.0.smithi100.stdout:3349: oid 18 version 1791 is already newer than 1790 2024-04-07T16:27:24.501 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 18 v 1791 (ObjNum 1027 snap 306 seq_num 1027) dirty exists 2024-04-07T16:27:24.501 INFO:tasks.rados.rados.0.smithi100.stdout:3349: left oid 18 (ObjNum 1027 snap 306 seq_num 1027) 2024-04-07T16:27:24.501 INFO:tasks.rados.rados.0.smithi100.stdout:3351: finishing write tid 1 to smithi10066867-27 2024-04-07T16:27:24.501 INFO:tasks.rados.rados.0.smithi100.stdout:3351: finishing write tid 2 to smithi10066867-27 2024-04-07T16:27:24.501 INFO:tasks.rados.rados.0.smithi100.stdout:3351: finishing write tid 3 to smithi10066867-27 2024-04-07T16:27:24.501 INFO:tasks.rados.rados.0.smithi100.stdout:3351: oid 27 updating version 0 to 1182 2024-04-07T16:27:24.501 INFO:tasks.rados.rados.0.smithi100.stdout:3351: oid 27 updating version 1182 to 1183 2024-04-07T16:27:24.502 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 27 v 1183 (ObjNum 1028 snap 306 seq_num 1028) dirty exists 2024-04-07T16:27:24.502 INFO:tasks.rados.rados.0.smithi100.stdout:3351: left oid 27 (ObjNum 1028 snap 306 seq_num 1028) 2024-04-07T16:27:24.502 INFO:tasks.rados.rados.0.smithi100.stdout:3352: finishing copy_from racing read to smithi10066867-33 2024-04-07T16:27:24.502 INFO:tasks.rados.rados.0.smithi100.stdout:3352: finishing copy_from to smithi10066867-33 2024-04-07T16:27:24.502 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 33 v 1538 (ObjNum 968 snap 293 seq_num 968) dirty exists 2024-04-07T16:27:24.502 INFO:tasks.rados.rados.0.smithi100.stdout:3341: done (9 left) 2024-04-07T16:27:24.502 INFO:tasks.rados.rados.0.smithi100.stdout:3344: done (8 left) 2024-04-07T16:27:24.502 INFO:tasks.rados.rados.0.smithi100.stdout:3345: done (7 left) 2024-04-07T16:27:24.502 INFO:tasks.rados.rados.0.smithi100.stdout:3346: done (6 left) 2024-04-07T16:27:24.502 INFO:tasks.rados.rados.0.smithi100.stdout:3347: done (5 left) 2024-04-07T16:27:24.502 INFO:tasks.rados.rados.0.smithi100.stdout:3348: done (4 left) 2024-04-07T16:27:24.502 INFO:tasks.rados.rados.0.smithi100.stdout:3349: done (3 left) 2024-04-07T16:27:24.502 INFO:tasks.rados.rados.0.smithi100.stdout:3350: done (2 left) 2024-04-07T16:27:24.503 INFO:tasks.rados.rados.0.smithi100.stdout:3351: done (1 left) 2024-04-07T16:27:24.503 INFO:tasks.rados.rados.0.smithi100.stdout:3352: done (0 left) 2024-04-07T16:27:24.503 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 9 current snap is 306 2024-04-07T16:27:24.503 INFO:tasks.rados.rados.0.smithi100.stdout:3353: seq_num 1029 ranges {32768=442368} 2024-04-07T16:27:24.508 INFO:tasks.rados.rados.0.smithi100.stdout:3353: writing smithi10066867-9 from 32768 to 475136 tid 1 2024-04-07T16:27:24.508 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 43 current snap is 306 2024-04-07T16:27:24.508 INFO:tasks.rados.rados.0.smithi100.stdout:3354: seq_num 1030 ranges {0=458752} 2024-04-07T16:27:24.515 INFO:tasks.rados.rados.0.smithi100.stdout:3354: writing smithi10066867-43 from 0 to 458752 tid 1 2024-04-07T16:27:24.515 INFO:tasks.rados.rados.0.smithi100.stdout:3355: rollback oid 16 current snap is 306 2024-04-07T16:27:24.516 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 16 to 303 2024-04-07T16:27:24.516 INFO:tasks.rados.rados.0.smithi100.stdout:3353: finishing write tid 1 to smithi10066867-9 2024-04-07T16:27:24.516 INFO:tasks.rados.rados.0.smithi100.stdout:3356: rmattr oid 20 current snap is 306 2024-04-07T16:27:24.516 INFO:tasks.rados.rados.0.smithi100.stdout:3357: setattr oid 4 current snap is 306 2024-04-07T16:27:24.516 INFO:tasks.rados.rados.0.smithi100.stdout:3353: finishing write tid 2 to smithi10066867-9 2024-04-07T16:27:24.516 INFO:tasks.rados.rados.0.smithi100.stdout:3355: finishing rollback tid 1 to smithi10066867-16 2024-04-07T16:27:24.516 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 16 v 0 (ObjNum 49 snap 0 seq_num 400) dirty dne 2024-04-07T16:27:24.519 INFO:tasks.rados.rados.0.smithi100.stdout:3353: finishing write tid 3 to smithi10066867-9 2024-04-07T16:27:24.519 INFO:tasks.rados.rados.0.smithi100.stdout:3353: oid 9 updating version 0 to 1137 2024-04-07T16:27:24.519 INFO:tasks.rados.rados.0.smithi100.stdout:3353: oid 9 version 1137 is already newer than 1136 2024-04-07T16:27:24.519 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 9 v 1137 (ObjNum 1029 snap 306 seq_num 1029) dirty exists 2024-04-07T16:27:24.519 INFO:tasks.rados.rados.0.smithi100.stdout:3353: left oid 9 (ObjNum 1029 snap 306 seq_num 1029) 2024-04-07T16:27:24.521 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 20 v 1530 (ObjNum 1007 snap 300 seq_num 1007) dirty exists 2024-04-07T16:27:24.523 INFO:tasks.rados.rados.0.smithi100.stdout:3353: done (4 left) 2024-04-07T16:27:24.523 INFO:tasks.rados.rados.0.smithi100.stdout:3355: done (3 left) 2024-04-07T16:27:24.523 INFO:tasks.rados.rados.0.smithi100.stdout:3356: done (2 left) 2024-04-07T16:27:24.523 INFO:tasks.rados.rados.0.smithi100.stdout:3358: snap_create 2024-04-07T16:27:24.526 INFO:tasks.rados.rados.0.smithi100.stdout:3354: finishing write tid 1 to smithi10066867-43 2024-04-07T16:27:24.526 INFO:tasks.rados.rados.0.smithi100.stdout:3354: finishing write tid 2 to smithi10066867-43 2024-04-07T16:27:24.529 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 4 v 1794 (ObjNum 0 snap 0 seq_num 304) dirty exists 2024-04-07T16:27:24.529 INFO:tasks.rados.rados.0.smithi100.stdout:3354: finishing write tid 3 to smithi10066867-43 2024-04-07T16:27:24.529 INFO:tasks.rados.rados.0.smithi100.stdout:3354: oid 43 updating version 0 to 1185 2024-04-07T16:27:24.529 INFO:tasks.rados.rados.0.smithi100.stdout:3354: oid 43 version 1185 is already newer than 1184 2024-04-07T16:27:24.529 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 43 v 1185 (ObjNum 1030 snap 306 seq_num 1030) dirty exists 2024-04-07T16:27:24.529 INFO:tasks.rados.rados.0.smithi100.stdout:3354: left oid 43 (ObjNum 1030 snap 306 seq_num 1030) 2024-04-07T16:27:25.034 INFO:tasks.rados.rados.0.smithi100.stdout:3354: done (2 left) 2024-04-07T16:27:25.034 INFO:tasks.rados.rados.0.smithi100.stdout:3357: done (1 left) 2024-04-07T16:27:25.034 INFO:tasks.rados.rados.0.smithi100.stdout:3358: done (0 left) 2024-04-07T16:27:25.034 INFO:tasks.rados.rados.0.smithi100.stdout:3359: delete oid 42 current snap is 307 2024-04-07T16:27:25.041 INFO:tasks.rados.rados.0.smithi100.stdout:3359: done (0 left) 2024-04-07T16:27:25.041 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 26 current snap is 307 2024-04-07T16:27:25.041 INFO:tasks.rados.rados.0.smithi100.stdout:3360: seq_num 1032 ranges {8994816=491520} 2024-04-07T16:27:25.095 INFO:tasks.rados.rados.0.smithi100.stdout:3360: writing smithi10066867-26 from 8994816 to 9486336 tid 1 2024-04-07T16:27:25.095 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 42 current snap is 307 2024-04-07T16:27:25.096 INFO:tasks.rados.rados.0.smithi100.stdout:3361: seq_num 1033 ranges {0=507904} 2024-04-07T16:27:25.098 INFO:tasks.rados.rados.0.smithi100.stdout:3361: writing smithi10066867-42 from 0 to 507904 tid 1 2024-04-07T16:27:25.098 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 22 current snap is 307 2024-04-07T16:27:25.098 INFO:tasks.rados.rados.0.smithi100.stdout:3362: seq_num 1034 ranges {6176768=524288} 2024-04-07T16:27:25.137 INFO:tasks.rados.rados.0.smithi100.stdout:3362: writing smithi10066867-22 from 6176768 to 6701056 tid 1 2024-04-07T16:27:25.137 INFO:tasks.rados.rados.0.smithi100.stdout:3363: delete oid 44 current snap is 307 2024-04-07T16:27:25.137 INFO:tasks.rados.rados.0.smithi100.stdout:3360: finishing write tid 1 to smithi10066867-26 2024-04-07T16:27:25.137 INFO:tasks.rados.rados.0.smithi100.stdout:3360: finishing write tid 2 to smithi10066867-26 2024-04-07T16:27:25.137 INFO:tasks.rados.rados.0.smithi100.stdout:3361: finishing write tid 1 to smithi10066867-42 2024-04-07T16:27:25.137 INFO:tasks.rados.rados.0.smithi100.stdout:3361: finishing write tid 2 to smithi10066867-42 2024-04-07T16:27:25.137 INFO:tasks.rados.rados.0.smithi100.stdout:3360: finishing write tid 3 to smithi10066867-26 2024-04-07T16:27:25.137 INFO:tasks.rados.rados.0.smithi100.stdout:3360: oid 26 updating version 0 to 1531 2024-04-07T16:27:25.137 INFO:tasks.rados.rados.0.smithi100.stdout:3360: oid 26 updating version 1531 to 1532 2024-04-07T16:27:25.137 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 26 v 1532 (ObjNum 1032 snap 307 seq_num 1032) dirty exists 2024-04-07T16:27:25.137 INFO:tasks.rados.rados.0.smithi100.stdout:3360: left oid 26 (ObjNum 1032 snap 307 seq_num 1032) 2024-04-07T16:27:25.137 INFO:tasks.rados.rados.0.smithi100.stdout:3361: finishing write tid 3 to smithi10066867-42 2024-04-07T16:27:25.137 INFO:tasks.rados.rados.0.smithi100.stdout:3361: oid 42 updating version 0 to 1549 2024-04-07T16:27:25.138 INFO:tasks.rados.rados.0.smithi100.stdout:3361: oid 42 version 1549 is already newer than 1548 2024-04-07T16:27:25.138 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 42 v 1549 (ObjNum 1033 snap 307 seq_num 1033) dirty exists 2024-04-07T16:27:25.138 INFO:tasks.rados.rados.0.smithi100.stdout:3361: left oid 42 (ObjNum 1033 snap 307 seq_num 1033) 2024-04-07T16:27:25.141 INFO:tasks.rados.rados.0.smithi100.stdout:3360: done (3 left) 2024-04-07T16:27:25.141 INFO:tasks.rados.rados.0.smithi100.stdout:3361: done (2 left) 2024-04-07T16:27:25.141 INFO:tasks.rados.rados.0.smithi100.stdout:3363: done (1 left) 2024-04-07T16:27:25.141 INFO:tasks.rados.rados.0.smithi100.stdout:3364: read oid 18 snap -1 2024-04-07T16:27:25.141 INFO:tasks.rados.rados.0.smithi100.stdout:3364: expect (ObjNum 1027 snap 306 seq_num 1027) 2024-04-07T16:27:25.141 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 46 current snap is 307 2024-04-07T16:27:25.141 INFO:tasks.rados.rados.0.smithi100.stdout:3365: seq_num 1036 ranges {5234688=557056} 2024-04-07T16:27:25.174 INFO:tasks.rados.rados.0.smithi100.stdout:3365: writing smithi10066867-46 from 5234688 to 5791744 tid 1 2024-04-07T16:27:25.174 INFO:tasks.rados.rados.0.smithi100.stdout:3366: read oid 29 snap -1 2024-04-07T16:27:25.175 INFO:tasks.rados.rados.0.smithi100.stdout:3366: expect deleted 2024-04-07T16:27:25.175 INFO:tasks.rados.rados.0.smithi100.stdout:3367: read oid 34 snap -1 2024-04-07T16:27:25.175 INFO:tasks.rados.rados.0.smithi100.stdout:3367: expect (ObjNum 1012 snap 300 seq_num 1012) 2024-04-07T16:27:25.175 INFO:tasks.rados.rados.0.smithi100.stdout:3368: read oid 32 snap -1 2024-04-07T16:27:25.175 INFO:tasks.rados.rados.0.smithi100.stdout:3368: expect (ObjNum 1020 snap 304 seq_num 1020) 2024-04-07T16:27:25.175 INFO:tasks.rados.rados.0.smithi100.stdout:3369: copy_from oid 16 from oid 13 current snap is 307 2024-04-07T16:27:25.175 INFO:tasks.rados.rados.0.smithi100.stdout:3362: finishing write tid 1 to smithi10066867-22 2024-04-07T16:27:25.175 INFO:tasks.rados.rados.0.smithi100.stdout:3370: copy_from oid 24 from oid 43 current snap is 307 2024-04-07T16:27:25.175 INFO:tasks.rados.rados.0.smithi100.stdout:3362: finishing write tid 2 to smithi10066867-22 2024-04-07T16:27:25.175 INFO:tasks.rados.rados.0.smithi100.stdout:3364: expect (ObjNum 1027 snap 306 seq_num 1027) 2024-04-07T16:27:25.188 INFO:tasks.rados.rados.0.smithi100.stdout:3362: finishing write tid 3 to smithi10066867-22 2024-04-07T16:27:25.188 INFO:tasks.rados.rados.0.smithi100.stdout:3362: oid 22 updating version 0 to 1534 2024-04-07T16:27:25.188 INFO:tasks.rados.rados.0.smithi100.stdout:3362: oid 22 version 1534 is already newer than 1533 2024-04-07T16:27:25.188 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 22 v 1534 (ObjNum 1034 snap 307 seq_num 1034) dirty exists 2024-04-07T16:27:25.188 INFO:tasks.rados.rados.0.smithi100.stdout:3362: left oid 22 (ObjNum 1034 snap 307 seq_num 1034) 2024-04-07T16:27:25.188 INFO:tasks.rados.rados.0.smithi100.stdout:3362: done (7 left) 2024-04-07T16:27:25.188 INFO:tasks.rados.rados.0.smithi100.stdout:3364: done (6 left) 2024-04-07T16:27:25.188 INFO:tasks.rados.rados.0.smithi100.stdout:3371: setattr oid 39 current snap is 307 2024-04-07T16:27:25.188 INFO:tasks.rados.rados.0.smithi100.stdout:3368: expect (ObjNum 1020 snap 304 seq_num 1020) 2024-04-07T16:27:25.196 INFO:tasks.rados.rados.0.smithi100.stdout:3370: finishing copy_from racing read to smithi10066867-24 2024-04-07T16:27:25.196 INFO:tasks.rados.rados.0.smithi100.stdout:3370: finishing copy_from to smithi10066867-24 2024-04-07T16:27:25.196 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 24 v 1185 (ObjNum 1030 snap 306 seq_num 1030) dirty exists 2024-04-07T16:27:25.196 INFO:tasks.rados.rados.0.smithi100.stdout:3365: finishing write tid 1 to smithi10066867-46 2024-04-07T16:27:25.196 INFO:tasks.rados.rados.0.smithi100.stdout:3369: finishing copy_from to smithi10066867-16 2024-04-07T16:27:25.196 INFO:tasks.rados.rados.0.smithi100.stdout:3369: got expected ENOENT (src dne) 2024-04-07T16:27:25.196 INFO:tasks.rados.rados.0.smithi100.stdout:3369: finishing copy_from racing read to smithi10066867-16 2024-04-07T16:27:25.197 INFO:tasks.rados.rados.0.smithi100.stdout:3366: done (6 left) 2024-04-07T16:27:25.197 INFO:tasks.rados.rados.0.smithi100.stdout:3368: done (5 left) 2024-04-07T16:27:25.197 INFO:tasks.rados.rados.0.smithi100.stdout:3369: done (4 left) 2024-04-07T16:27:25.197 INFO:tasks.rados.rados.0.smithi100.stdout:3370: done (3 left) 2024-04-07T16:27:25.197 INFO:tasks.rados.rados.0.smithi100.stdout:3372: snap_remove snap 304 2024-04-07T16:27:25.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:25 smithi144 bash[66654]: cluster 2024-04-07T16:27:24.039590+0000 mon.a (mon.0) 1137 : cluster [DBG] osdmap e686: 8 total, 8 up, 8 in 2024-04-07T16:27:25.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:25 smithi100 bash[74942]: cluster 2024-04-07T16:27:24.039590+0000 mon.a (mon.0) 1137 : cluster [DBG] osdmap e686: 8 total, 8 up, 8 in 2024-04-07T16:27:25.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:25 smithi100 bash[76202]: cluster 2024-04-07T16:27:24.039590+0000 mon.a (mon.0) 1137 : cluster [DBG] osdmap e686: 8 total, 8 up, 8 in 2024-04-07T16:27:26.036 INFO:tasks.rados.rados.0.smithi100.stdout:3372: done (3 left) 2024-04-07T16:27:26.036 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 42 current snap is 307 2024-04-07T16:27:26.036 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 39 v 1550 (ObjNum 1025 snap 306 seq_num 1025) dirty exists 2024-04-07T16:27:26.036 INFO:tasks.rados.rados.0.smithi100.stdout:3373: seq_num 1037 ranges {507904=573440} 2024-04-07T16:27:26.046 INFO:tasks.rados.rados.0.smithi100.stdout:3373: writing smithi10066867-42 from 507904 to 1081344 tid 1 2024-04-07T16:27:26.046 INFO:tasks.rados.rados.0.smithi100.stdout:3365: finishing write tid 2 to smithi10066867-46 2024-04-07T16:27:26.046 INFO:tasks.rados.rados.0.smithi100.stdout:3367: expect (ObjNum 1012 snap 300 seq_num 1012) 2024-04-07T16:27:26.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:26 smithi144 bash[66654]: cluster 2024-04-07T16:27:24.790701+0000 mgr.y (mgr.25175) 346 : cluster [DBG] pgmap v623: 121 pgs: 121 active+clean; 128 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 5.4 MiB/s rd, 986 KiB/s wr, 5 op/s 2024-04-07T16:27:26.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:26 smithi144 bash[66654]: cluster 2024-04-07T16:27:25.048458+0000 mon.a (mon.0) 1138 : cluster [DBG] osdmap e687: 8 total, 8 up, 8 in 2024-04-07T16:27:26.429 INFO:tasks.rados.rados.0.smithi100.stdout:3365: finishing write tid 3 to smithi10066867-46 2024-04-07T16:27:26.429 INFO:tasks.rados.rados.0.smithi100.stdout:3365: oid 46 updating version 0 to 1570 2024-04-07T16:27:26.430 INFO:tasks.rados.rados.0.smithi100.stdout:3365: oid 46 updating version 1570 to 1571 2024-04-07T16:27:26.430 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 46 v 1571 (ObjNum 1036 snap 307 seq_num 1036) dirty exists 2024-04-07T16:27:26.430 INFO:tasks.rados.rados.0.smithi100.stdout:3365: left oid 46 (ObjNum 1036 snap 307 seq_num 1036) 2024-04-07T16:27:26.430 INFO:tasks.rados.rados.0.smithi100.stdout:3373: finishing write tid 1 to smithi10066867-42 2024-04-07T16:27:26.430 INFO:tasks.rados.rados.0.smithi100.stdout:3373: finishing write tid 2 to smithi10066867-42 2024-04-07T16:27:26.430 INFO:tasks.rados.rados.0.smithi100.stdout:3373: finishing write tid 3 to smithi10066867-42 2024-04-07T16:27:26.430 INFO:tasks.rados.rados.0.smithi100.stdout:3373: oid 42 updating version 0 to 1552 2024-04-07T16:27:26.430 INFO:tasks.rados.rados.0.smithi100.stdout:3373: oid 42 version 1552 is already newer than 1551 2024-04-07T16:27:26.430 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 42 v 1552 (ObjNum 1037 snap 307 seq_num 1037) dirty exists 2024-04-07T16:27:26.430 INFO:tasks.rados.rados.0.smithi100.stdout:3373: left oid 42 (ObjNum 1037 snap 307 seq_num 1037) 2024-04-07T16:27:26.430 INFO:tasks.rados.rados.0.smithi100.stdout:3365: done (3 left) 2024-04-07T16:27:26.430 INFO:tasks.rados.rados.0.smithi100.stdout:3367: done (2 left) 2024-04-07T16:27:26.430 INFO:tasks.rados.rados.0.smithi100.stdout:3371: done (1 left) 2024-04-07T16:27:26.430 INFO:tasks.rados.rados.0.smithi100.stdout:3373: done (0 left) 2024-04-07T16:27:26.430 INFO:tasks.rados.rados.0.smithi100.stdout:3374: rollback oid 30 current snap is 307 2024-04-07T16:27:26.431 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 30 to 303 2024-04-07T16:27:26.431 INFO:tasks.rados.rados.0.smithi100.stdout:3375: snap_remove snap 305 2024-04-07T16:27:26.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:26 smithi100 bash[74942]: cluster 2024-04-07T16:27:24.790701+0000 mgr.y (mgr.25175) 346 : cluster [DBG] pgmap v623: 121 pgs: 121 active+clean; 128 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 5.4 MiB/s rd, 986 KiB/s wr, 5 op/s 2024-04-07T16:27:26.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:26 smithi100 bash[74942]: cluster 2024-04-07T16:27:25.048458+0000 mon.a (mon.0) 1138 : cluster [DBG] osdmap e687: 8 total, 8 up, 8 in 2024-04-07T16:27:26.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:26 smithi100 bash[76202]: cluster 2024-04-07T16:27:24.790701+0000 mgr.y (mgr.25175) 346 : cluster [DBG] pgmap v623: 121 pgs: 121 active+clean; 128 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 5.4 MiB/s rd, 986 KiB/s wr, 5 op/s 2024-04-07T16:27:26.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:26 smithi100 bash[76202]: cluster 2024-04-07T16:27:25.048458+0000 mon.a (mon.0) 1138 : cluster [DBG] osdmap e687: 8 total, 8 up, 8 in 2024-04-07T16:27:26.922 INFO:tasks.rados.rados.0.smithi100.stdout:3375: done (1 left) 2024-04-07T16:27:26.922 INFO:tasks.rados.rados.0.smithi100.stdout:3376: delete oid 42 current snap is 307 2024-04-07T16:27:26.922 INFO:tasks.rados.rados.0.smithi100.stdout:3374: finishing rollback tid 0 to smithi10066867-30 2024-04-07T16:27:26.922 INFO:tasks.rados.rados.0.smithi100.stdout:3374: finishing rollback tid 1 to smithi10066867-30 2024-04-07T16:27:26.922 INFO:tasks.rados.rados.0.smithi100.stdout:3374: finishing rollback tid 2 to smithi10066867-30 2024-04-07T16:27:26.922 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 30 v 1555 (ObjNum 991 snap 299 seq_num 991) dirty exists 2024-04-07T16:27:26.955 INFO:tasks.rados.rados.0.smithi100.stdout:3374: done (1 left) 2024-04-07T16:27:26.955 INFO:tasks.rados.rados.0.smithi100.stdout:3376: done (0 left) 2024-04-07T16:27:26.955 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 41 current snap is 307 2024-04-07T16:27:26.955 INFO:tasks.rados.rados.0.smithi100.stdout:3377: seq_num 1039 ranges {4268032=606208} 2024-04-07T16:27:27.001 INFO:tasks.rados.rados.0.smithi100.stdout:3377: writing smithi10066867-41 from 4268032 to 4874240 tid 1 2024-04-07T16:27:27.001 INFO:tasks.rados.rados.0.smithi100.stdout:3378: delete oid 48 current snap is 307 2024-04-07T16:27:27.005 INFO:tasks.rados.rados.0.smithi100.stdout:3378: done (1 left) 2024-04-07T16:27:27.005 INFO:tasks.rados.rados.0.smithi100.stdout:3379: copy_from oid 34 from oid 3 current snap is 307 2024-04-07T16:27:27.005 INFO:tasks.rados.rados.0.smithi100.stdout:3380: copy_from oid 25 from oid 31 current snap is 307 2024-04-07T16:27:27.005 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 4 current snap is 307 2024-04-07T16:27:27.005 INFO:tasks.rados.rados.0.smithi100.stdout:3381: seq_num 1041 ranges {0=638976} 2024-04-07T16:27:27.012 INFO:tasks.rados.rados.0.smithi100.stdout:3381: writing smithi10066867-4 from 0 to 638976 tid 1 2024-04-07T16:27:27.012 INFO:tasks.rados.rados.0.smithi100.stdout:3382: rollback oid 12 current snap is 307 2024-04-07T16:27:27.012 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 12 to 306 2024-04-07T16:27:27.012 INFO:tasks.rados.rados.0.smithi100.stdout:3379: finishing copy_from to smithi10066867-34 2024-04-07T16:27:27.012 INFO:tasks.rados.rados.0.smithi100.stdout:3379: got expected ENOENT (src dne) 2024-04-07T16:27:27.012 INFO:tasks.rados.rados.0.smithi100.stdout:3379: finishing copy_from racing read to smithi10066867-34 2024-04-07T16:27:27.012 INFO:tasks.rados.rados.0.smithi100.stdout:3377: finishing write tid 1 to smithi10066867-41 2024-04-07T16:27:27.012 INFO:tasks.rados.rados.0.smithi100.stdout:3379: done (4 left) 2024-04-07T16:27:27.013 INFO:tasks.rados.rados.0.smithi100.stdout:3383: delete oid 6 current snap is 307 2024-04-07T16:27:27.013 INFO:tasks.rados.rados.0.smithi100.stdout:3382: finishing rollback tid 1 to smithi10066867-12 2024-04-07T16:27:27.013 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 12 v 0 (ObjNum 289 snap 0 seq_num 0) dirty dne 2024-04-07T16:27:27.013 INFO:tasks.rados.rados.0.smithi100.stdout:3377: finishing write tid 2 to smithi10066867-41 2024-04-07T16:27:27.019 INFO:tasks.rados.rados.0.smithi100.stdout:3377: finishing write tid 3 to smithi10066867-41 2024-04-07T16:27:27.019 INFO:tasks.rados.rados.0.smithi100.stdout:3377: oid 41 updating version 0 to 1696 2024-04-07T16:27:27.019 INFO:tasks.rados.rados.0.smithi100.stdout:3377: oid 41 version 1696 is already newer than 1695 2024-04-07T16:27:27.019 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 41 v 1696 (ObjNum 1039 snap 307 seq_num 1039) dirty exists 2024-04-07T16:27:27.019 INFO:tasks.rados.rados.0.smithi100.stdout:3377: left oid 41 (ObjNum 1039 snap 307 seq_num 1039) 2024-04-07T16:27:27.019 INFO:tasks.rados.rados.0.smithi100.stdout:3377: done (4 left) 2024-04-07T16:27:27.019 INFO:tasks.rados.rados.0.smithi100.stdout:3382: done (3 left) 2024-04-07T16:27:27.019 INFO:tasks.rados.rados.0.smithi100.stdout:3383: done (2 left) 2024-04-07T16:27:27.019 INFO:tasks.rados.rados.0.smithi100.stdout:3384: copy_from oid 35 from oid 7 current snap is 307 2024-04-07T16:27:27.019 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 50 current snap is 307 2024-04-07T16:27:27.019 INFO:tasks.rados.rados.0.smithi100.stdout:3385: seq_num 1043 ranges {2375680=671744} 2024-04-07T16:27:27.044 INFO:tasks.rados.rados.0.smithi100.stdout:3385: writing smithi10066867-50 from 2375680 to 3047424 tid 1 2024-04-07T16:27:27.044 INFO:tasks.rados.rados.0.smithi100.stdout:3386: rollback oid 3 current snap is 307 2024-04-07T16:27:27.044 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 3 to 306 2024-04-07T16:27:27.044 INFO:tasks.rados.rados.0.smithi100.stdout:3381: finishing write tid 1 to smithi10066867-4 2024-04-07T16:27:27.044 INFO:tasks.rados.rados.0.smithi100.stdout:3387: rollback oid 13 current snap is 307 2024-04-07T16:27:27.044 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 13 to 300 2024-04-07T16:27:27.044 INFO:tasks.rados.rados.0.smithi100.stdout:3381: finishing write tid 2 to smithi10066867-4 2024-04-07T16:27:27.044 INFO:tasks.rados.rados.0.smithi100.stdout:3380: finishing copy_from racing read to smithi10066867-25 2024-04-07T16:27:27.044 INFO:tasks.rados.rados.0.smithi100.stdout:3381: finishing write tid 3 to smithi10066867-4 2024-04-07T16:27:27.044 INFO:tasks.rados.rados.0.smithi100.stdout:3381: oid 4 updating version 0 to 1802 2024-04-07T16:27:27.044 INFO:tasks.rados.rados.0.smithi100.stdout:3381: oid 4 updating version 1802 to 1803 2024-04-07T16:27:27.044 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 4 v 1803 (ObjNum 1041 snap 307 seq_num 1041) dirty exists 2024-04-07T16:27:27.044 INFO:tasks.rados.rados.0.smithi100.stdout:3381: left oid 4 (ObjNum 1041 snap 307 seq_num 1041) 2024-04-07T16:27:27.045 INFO:tasks.rados.rados.0.smithi100.stdout:3380: finishing copy_from to smithi10066867-25 2024-04-07T16:27:27.045 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 25 v 1692 (ObjNum 993 snap 300 seq_num 993) dirty exists 2024-04-07T16:27:27.045 INFO:tasks.rados.rados.0.smithi100.stdout:3384: finishing copy_from racing read to smithi10066867-35 2024-04-07T16:27:27.045 INFO:tasks.rados.rados.0.smithi100.stdout:3380: done (5 left) 2024-04-07T16:27:27.045 INFO:tasks.rados.rados.0.smithi100.stdout:3381: done (4 left) 2024-04-07T16:27:27.045 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 27 current snap is 307 2024-04-07T16:27:27.045 INFO:tasks.rados.rados.0.smithi100.stdout:3384: finishing copy_from to smithi10066867-35 2024-04-07T16:27:27.045 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 35 v 1162 (ObjNum 935 snap 280 seq_num 935) dirty exists 2024-04-07T16:27:27.045 INFO:tasks.rados.rados.0.smithi100.stdout:3388: seq_num 1044 ranges {4440064=688128} 2024-04-07T16:27:27.075 INFO:tasks.rados.rados.0.smithi100.stdout:3388: writing smithi10066867-27 from 4440064 to 5128192 tid 1 2024-04-07T16:27:27.075 INFO:tasks.rados.rados.0.smithi100.stdout:3384: done (4 left) 2024-04-07T16:27:27.075 INFO:tasks.rados.rados.0.smithi100.stdout:3389: delete oid 26 current snap is 307 2024-04-07T16:27:27.075 INFO:tasks.rados.rados.0.smithi100.stdout:3387: finishing rollback tid 1 to smithi10066867-13 2024-04-07T16:27:27.075 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 13 v 0 (ObjNum 33 snap 0 seq_num 2701824448) dirty dne 2024-04-07T16:27:27.076 INFO:tasks.rados.rados.0.smithi100.stdout:3386: finishing rollback tid 1 to smithi10066867-3 2024-04-07T16:27:27.076 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 3 v 1694 (ObjNum 113 snap 0 seq_num 2701891008) dirty dne 2024-04-07T16:27:27.076 INFO:tasks.rados.rados.0.smithi100.stdout:3385: finishing write tid 1 to smithi10066867-50 2024-04-07T16:27:27.076 INFO:tasks.rados.rados.0.smithi100.stdout:3385: finishing write tid 2 to smithi10066867-50 2024-04-07T16:27:27.076 INFO:tasks.rados.rados.0.smithi100.stdout:3385: finishing write tid 3 to smithi10066867-50 2024-04-07T16:27:27.076 INFO:tasks.rados.rados.0.smithi100.stdout:3385: oid 50 updating version 0 to 1550 2024-04-07T16:27:27.076 INFO:tasks.rados.rados.0.smithi100.stdout:3385: oid 50 version 1550 is already newer than 1549 2024-04-07T16:27:27.076 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 50 v 1550 (ObjNum 1043 snap 307 seq_num 1043) dirty exists 2024-04-07T16:27:27.076 INFO:tasks.rados.rados.0.smithi100.stdout:3385: left oid 50 (ObjNum 1043 snap 307 seq_num 1043) 2024-04-07T16:27:27.079 INFO:tasks.rados.rados.0.smithi100.stdout:3385: done (4 left) 2024-04-07T16:27:27.079 INFO:tasks.rados.rados.0.smithi100.stdout:3386: done (3 left) 2024-04-07T16:27:27.079 INFO:tasks.rados.rados.0.smithi100.stdout:3387: done (2 left) 2024-04-07T16:27:27.079 INFO:tasks.rados.rados.0.smithi100.stdout:3389: done (1 left) 2024-04-07T16:27:27.079 INFO:tasks.rados.rados.0.smithi100.stdout:3390: read oid 16 snap -1 2024-04-07T16:27:27.079 INFO:tasks.rados.rados.0.smithi100.stdout:3390: expect deleted 2024-04-07T16:27:27.079 INFO:tasks.rados.rados.0.smithi100.stdout:3391: delete oid 31 current snap is 307 2024-04-07T16:27:27.080 INFO:tasks.rados.rados.0.smithi100.stdout:3388: finishing write tid 1 to smithi10066867-27 2024-04-07T16:27:27.080 INFO:tasks.rados.rados.0.smithi100.stdout:3388: finishing write tid 2 to smithi10066867-27 2024-04-07T16:27:27.082 INFO:tasks.rados.rados.0.smithi100.stdout:3388: finishing write tid 3 to smithi10066867-27 2024-04-07T16:27:27.082 INFO:tasks.rados.rados.0.smithi100.stdout:3388: oid 27 updating version 0 to 1186 2024-04-07T16:27:27.083 INFO:tasks.rados.rados.0.smithi100.stdout:3388: oid 27 updating version 1186 to 1187 2024-04-07T16:27:27.083 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 27 v 1187 (ObjNum 1044 snap 307 seq_num 1044) dirty exists 2024-04-07T16:27:27.083 INFO:tasks.rados.rados.0.smithi100.stdout:3388: left oid 27 (ObjNum 1044 snap 307 seq_num 1044) 2024-04-07T16:27:27.083 INFO:tasks.rados.rados.0.smithi100.stdout:3388: done (2 left) 2024-04-07T16:27:27.083 INFO:tasks.rados.rados.0.smithi100.stdout:3390: done (1 left) 2024-04-07T16:27:27.083 INFO:tasks.rados.rados.0.smithi100.stdout:3391: done (0 left) 2024-04-07T16:27:27.083 INFO:tasks.rados.rados.0.smithi100.stdout:3392: copy_from oid 16 from oid 14 current snap is 307 2024-04-07T16:27:27.084 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 1 current snap is 307 2024-04-07T16:27:27.084 INFO:tasks.rados.rados.0.smithi100.stdout:3393: seq_num 1047 ranges {3342336=745472} 2024-04-07T16:27:27.107 INFO:tasks.rados.rados.0.smithi100.stdout:3393: writing smithi10066867-1 from 3342336 to 4087808 tid 1 2024-04-07T16:27:27.107 INFO:tasks.rados.rados.0.smithi100.stdout:3392: finishing copy_from racing read to smithi10066867-16 2024-04-07T16:27:27.107 INFO:tasks.rados.rados.0.smithi100.stdout:3392: finishing copy_from to smithi10066867-16 2024-04-07T16:27:27.107 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 16 v 1262 (ObjNum 1009 snap 300 seq_num 1009) dirty exists 2024-04-07T16:27:27.107 INFO:tasks.rados.rados.0.smithi100.stdout:3394: read oid 27 snap -1 2024-04-07T16:27:27.107 INFO:tasks.rados.rados.0.smithi100.stdout:3394: expect (ObjNum 1044 snap 307 seq_num 1044) 2024-04-07T16:27:27.107 INFO:tasks.rados.rados.0.smithi100.stdout:3392: done (2 left) 2024-04-07T16:27:27.107 INFO:tasks.rados.rados.0.smithi100.stdout:3395: delete oid 18 current snap is 307 2024-04-07T16:27:27.112 INFO:tasks.rados.rados.0.smithi100.stdout:3395: done (2 left) 2024-04-07T16:27:27.112 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 6 current snap is 307 2024-04-07T16:27:27.113 INFO:tasks.rados.rados.0.smithi100.stdout:3396: seq_num 1049 ranges {0=778240} 2024-04-07T16:27:27.117 INFO:tasks.rados.rados.0.smithi100.stdout:3396: writing smithi10066867-6 from 0 to 778240 tid 1 2024-04-07T16:27:27.117 INFO:tasks.rados.rados.0.smithi100.stdout:3397: rollback oid 26 current snap is 307 2024-04-07T16:27:27.117 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 26 to 303 2024-04-07T16:27:27.117 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 40 current snap is 307 2024-04-07T16:27:27.117 INFO:tasks.rados.rados.0.smithi100.stdout:3398: seq_num 1050 ranges {2007040=794624} 2024-04-07T16:27:27.135 INFO:tasks.rados.rados.0.smithi100.stdout:3398: writing smithi10066867-40 from 2007040 to 2801664 tid 1 2024-04-07T16:27:27.135 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 7 current snap is 307 2024-04-07T16:27:27.135 INFO:tasks.rados.rados.0.smithi100.stdout:3393: finishing write tid 1 to smithi10066867-1 2024-04-07T16:27:27.135 INFO:tasks.rados.rados.0.smithi100.stdout:3399: seq_num 1051 ranges {2080768=409600,2490368=401408} 2024-04-07T16:27:27.148 INFO:tasks.rados.rados.0.smithi100.stdout:3399: writing smithi10066867-7 from 2080768 to 2490368 tid 1 2024-04-07T16:27:27.151 INFO:tasks.rados.rados.0.smithi100.stdout:3399: writing smithi10066867-7 from 2490368 to 2891776 tid 2 2024-04-07T16:27:27.151 INFO:tasks.rados.rados.0.smithi100.stdout:3400: snap_create 2024-04-07T16:27:27.151 INFO:tasks.rados.rados.0.smithi100.stdout:3393: finishing write tid 2 to smithi10066867-1 2024-04-07T16:27:27.151 INFO:tasks.rados.rados.0.smithi100.stdout:3393: finishing write tid 3 to smithi10066867-1 2024-04-07T16:27:27.151 INFO:tasks.rados.rados.0.smithi100.stdout:3393: oid 1 updating version 0 to 1456 2024-04-07T16:27:27.151 INFO:tasks.rados.rados.0.smithi100.stdout:3393: oid 1 updating version 1456 to 1457 2024-04-07T16:27:27.151 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 1 v 1457 (ObjNum 1047 snap 307 seq_num 1047) dirty exists 2024-04-07T16:27:27.152 INFO:tasks.rados.rados.0.smithi100.stdout:3393: left oid 1 (ObjNum 1047 snap 307 seq_num 1047) 2024-04-07T16:27:27.152 INFO:tasks.rados.rados.0.smithi100.stdout:3397: finishing rollback tid 1 to smithi10066867-26 2024-04-07T16:27:27.152 INFO:tasks.rados.rados.0.smithi100.stdout:3397: finishing rollback tid 2 to smithi10066867-26 2024-04-07T16:27:27.152 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 26 v 1695 (ObjNum 1003 snap 300 seq_num 1003) dirty exists 2024-04-07T16:27:27.152 INFO:tasks.rados.rados.0.smithi100.stdout:3396: finishing write tid 1 to smithi10066867-6 2024-04-07T16:27:27.152 INFO:tasks.rados.rados.0.smithi100.stdout:3396: finishing write tid 2 to smithi10066867-6 2024-04-07T16:27:27.152 INFO:tasks.rados.rados.0.smithi100.stdout:3396: finishing write tid 3 to smithi10066867-6 2024-04-07T16:27:27.152 INFO:tasks.rados.rados.0.smithi100.stdout:3396: oid 6 updating version 0 to 1644 2024-04-07T16:27:27.152 INFO:tasks.rados.rados.0.smithi100.stdout:3396: oid 6 version 1644 is already newer than 1643 2024-04-07T16:27:27.152 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 6 v 1644 (ObjNum 1049 snap 307 seq_num 1049) dirty exists 2024-04-07T16:27:27.152 INFO:tasks.rados.rados.0.smithi100.stdout:3396: left oid 6 (ObjNum 1049 snap 307 seq_num 1049) 2024-04-07T16:27:27.152 INFO:tasks.rados.rados.0.smithi100.stdout:3394: expect (ObjNum 1044 snap 307 seq_num 1044) 2024-04-07T16:27:27.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:27 smithi144 bash[66654]: cluster 2024-04-07T16:27:26.049383+0000 mon.a (mon.0) 1139 : cluster [DBG] osdmap e688: 8 total, 8 up, 8 in 2024-04-07T16:27:27.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:27 smithi144 bash[66654]: cluster 2024-04-07T16:27:26.933198+0000 mon.a (mon.0) 1140 : cluster [DBG] osdmap e689: 8 total, 8 up, 8 in 2024-04-07T16:27:27.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:27 smithi100 bash[74942]: cluster 2024-04-07T16:27:26.049383+0000 mon.a (mon.0) 1139 : cluster [DBG] osdmap e688: 8 total, 8 up, 8 in 2024-04-07T16:27:27.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:27 smithi100 bash[74942]: cluster 2024-04-07T16:27:26.933198+0000 mon.a (mon.0) 1140 : cluster [DBG] osdmap e689: 8 total, 8 up, 8 in 2024-04-07T16:27:27.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:27 smithi100 bash[76202]: cluster 2024-04-07T16:27:26.049383+0000 mon.a (mon.0) 1139 : cluster [DBG] osdmap e688: 8 total, 8 up, 8 in 2024-04-07T16:27:27.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:27 smithi100 bash[76202]: cluster 2024-04-07T16:27:26.933198+0000 mon.a (mon.0) 1140 : cluster [DBG] osdmap e689: 8 total, 8 up, 8 in 2024-04-07T16:27:27.543 INFO:tasks.rados.rados.0.smithi100.stdout:3398: finishing write tid 1 to smithi10066867-40 2024-04-07T16:27:27.543 INFO:tasks.rados.rados.0.smithi100.stdout:3398: finishing write tid 2 to smithi10066867-40 2024-04-07T16:27:27.543 INFO:tasks.rados.rados.0.smithi100.stdout:3398: finishing write tid 3 to smithi10066867-40 2024-04-07T16:27:27.544 INFO:tasks.rados.rados.0.smithi100.stdout:3398: oid 40 updating version 0 to 1646 2024-04-07T16:27:27.544 INFO:tasks.rados.rados.0.smithi100.stdout:3398: oid 40 version 1646 is already newer than 1645 2024-04-07T16:27:27.544 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 40 v 1646 (ObjNum 1050 snap 307 seq_num 1050) dirty exists 2024-04-07T16:27:27.544 INFO:tasks.rados.rados.0.smithi100.stdout:3398: left oid 40 (ObjNum 1050 snap 307 seq_num 1050) 2024-04-07T16:27:27.544 INFO:tasks.rados.rados.0.smithi100.stdout:3399: finishing write tid 1 to smithi10066867-7 2024-04-07T16:27:27.544 INFO:tasks.rados.rados.0.smithi100.stdout:3399: finishing write tid 2 to smithi10066867-7 2024-04-07T16:27:27.544 INFO:tasks.rados.rados.0.smithi100.stdout:3399: finishing write tid 3 to smithi10066867-7 2024-04-07T16:27:27.544 INFO:tasks.rados.rados.0.smithi100.stdout:3399: finishing write tid 4 to smithi10066867-7 2024-04-07T16:27:27.544 INFO:tasks.rados.rados.0.smithi100.stdout:3399: oid 7 updating version 0 to 1188 2024-04-07T16:27:27.544 INFO:tasks.rados.rados.0.smithi100.stdout:3399: oid 7 updating version 1188 to 1189 2024-04-07T16:27:27.544 INFO:tasks.rados.rados.0.smithi100.stdout:3399: oid 7 updating version 1189 to 1190 2024-04-07T16:27:27.544 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 7 v 1190 (ObjNum 1051 snap 307 seq_num 1051) dirty exists 2024-04-07T16:27:27.544 INFO:tasks.rados.rados.0.smithi100.stdout:3399: left oid 7 (ObjNum 1051 snap 307 seq_num 1051) 2024-04-07T16:27:28.068 INFO:tasks.rados.rados.0.smithi100.stdout:3393: done (6 left) 2024-04-07T16:27:28.068 INFO:tasks.rados.rados.0.smithi100.stdout:3394: done (5 left) 2024-04-07T16:27:28.068 INFO:tasks.rados.rados.0.smithi100.stdout:3396: done (4 left) 2024-04-07T16:27:28.068 INFO:tasks.rados.rados.0.smithi100.stdout:3397: done (3 left) 2024-04-07T16:27:28.068 INFO:tasks.rados.rados.0.smithi100.stdout:3398: done (2 left) 2024-04-07T16:27:28.068 INFO:tasks.rados.rados.0.smithi100.stdout:3399: done (1 left) 2024-04-07T16:27:28.069 INFO:tasks.rados.rados.0.smithi100.stdout:3400: done (0 left) 2024-04-07T16:27:28.069 INFO:tasks.rados.rados.0.smithi100.stdout:3401: delete oid 47 current snap is 308 2024-04-07T16:27:28.076 INFO:tasks.rados.rados.0.smithi100.stdout:3401: done (0 left) 2024-04-07T16:27:28.076 INFO:tasks.rados.rados.0.smithi100.stdout:3402: snap_create 2024-04-07T16:27:28.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:28 smithi144 bash[66654]: cluster 2024-04-07T16:27:26.791629+0000 mgr.y (mgr.25175) 347 : cluster [DBG] pgmap v626: 121 pgs: 121 active+clean; 129 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 1003 KiB/s rd, 1.1 MiB/s wr, 5 op/s 2024-04-07T16:27:28.418 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:28 smithi100 bash[74942]: cluster 2024-04-07T16:27:26.791629+0000 mgr.y (mgr.25175) 347 : cluster [DBG] pgmap v626: 121 pgs: 121 active+clean; 129 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 1003 KiB/s rd, 1.1 MiB/s wr, 5 op/s 2024-04-07T16:27:28.418 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:28 smithi100 bash[76202]: cluster 2024-04-07T16:27:26.791629+0000 mgr.y (mgr.25175) 347 : cluster [DBG] pgmap v626: 121 pgs: 121 active+clean; 129 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 1003 KiB/s rd, 1.1 MiB/s wr, 5 op/s 2024-04-07T16:27:28.687 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:27:28 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:27:28] "GET /metrics HTTP/1.1" 200 36344 "" "Prometheus/2.43.0" 2024-04-07T16:27:29.088 INFO:tasks.rados.rados.0.smithi100.stdout:3402: done (0 left) 2024-04-07T16:27:29.089 INFO:tasks.rados.rados.0.smithi100.stdout:3403: read oid 39 snap -1 2024-04-07T16:27:29.089 INFO:tasks.rados.rados.0.smithi100.stdout:3403: expect (ObjNum 1025 snap 306 seq_num 1025) 2024-04-07T16:27:29.089 INFO:tasks.rados.rados.0.smithi100.stdout:3404: read oid 43 snap -1 2024-04-07T16:27:29.089 INFO:tasks.rados.rados.0.smithi100.stdout:3404: expect (ObjNum 1030 snap 306 seq_num 1030) 2024-04-07T16:27:29.089 INFO:tasks.rados.rados.0.smithi100.stdout:3405: read oid 31 snap -1 2024-04-07T16:27:29.089 INFO:tasks.rados.rados.0.smithi100.stdout:3405: expect deleted 2024-04-07T16:27:29.089 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 15 current snap is 309 2024-04-07T16:27:29.089 INFO:tasks.rados.rados.0.smithi100.stdout:3406: seq_num 1053 ranges {3342336=442368,3784704=401408} 2024-04-07T16:27:29.110 INFO:tasks.rados.rados.0.smithi100.stdout:3406: writing smithi10066867-15 from 3342336 to 3784704 tid 1 2024-04-07T16:27:29.113 INFO:tasks.rados.rados.0.smithi100.stdout:3406: writing smithi10066867-15 from 3784704 to 4186112 tid 2 2024-04-07T16:27:29.113 INFO:tasks.rados.rados.0.smithi100.stdout:3407: snap_remove snap 303 2024-04-07T16:27:29.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:29 smithi144 bash[66654]: cluster 2024-04-07T16:27:28.082047+0000 mon.a (mon.0) 1141 : cluster [DBG] osdmap e690: 8 total, 8 up, 8 in 2024-04-07T16:27:29.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:29 smithi100 bash[74942]: cluster 2024-04-07T16:27:28.082047+0000 mon.a (mon.0) 1141 : cluster [DBG] osdmap e690: 8 total, 8 up, 8 in 2024-04-07T16:27:29.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:29 smithi100 bash[76202]: cluster 2024-04-07T16:27:28.082047+0000 mon.a (mon.0) 1141 : cluster [DBG] osdmap e690: 8 total, 8 up, 8 in 2024-04-07T16:27:30.101 INFO:tasks.rados.rados.0.smithi100.stdout:3407: done (4 left) 2024-04-07T16:27:30.101 INFO:tasks.rados.rados.0.smithi100.stdout:3408: delete oid 3 current snap is 309 2024-04-07T16:27:30.101 INFO:tasks.rados.rados.0.smithi100.stdout:3404: expect (ObjNum 1030 snap 306 seq_num 1030) 2024-04-07T16:27:30.114 INFO:tasks.rados.rados.0.smithi100.stdout:3404: done (4 left) 2024-04-07T16:27:30.114 INFO:tasks.rados.rados.0.smithi100.stdout:3405: done (3 left) 2024-04-07T16:27:30.115 INFO:tasks.rados.rados.0.smithi100.stdout:3408: done (2 left) 2024-04-07T16:27:30.115 INFO:tasks.rados.rados.0.smithi100.stdout:3409: read oid 33 snap 300 2024-04-07T16:27:30.115 INFO:tasks.rados.rados.0.smithi100.stdout:3409: expect (ObjNum 1008 snap 300 seq_num 1008) 2024-04-07T16:27:30.115 INFO:tasks.rados.rados.0.smithi100.stdout:3410: rollback oid 4 current snap is 309 2024-04-07T16:27:30.115 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 4 to 307 2024-04-07T16:27:30.115 INFO:tasks.rados.rados.0.smithi100.stdout:3403: expect (ObjNum 1025 snap 306 seq_num 1025) 2024-04-07T16:27:30.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:30 smithi144 bash[66654]: cluster 2024-04-07T16:27:28.792362+0000 mgr.y (mgr.25175) 348 : cluster [DBG] pgmap v629: 121 pgs: 1 active+clean+snaptrim, 120 active+clean; 135 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 3.9 MiB/s rd, 2.0 MiB/s wr, 24 op/s 2024-04-07T16:27:30.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:30 smithi144 bash[66654]: cluster 2024-04-07T16:27:29.102779+0000 mon.a (mon.0) 1142 : cluster [DBG] osdmap e691: 8 total, 8 up, 8 in 2024-04-07T16:27:30.396 INFO:tasks.rados.rados.0.smithi100.stdout:3406: finishing write tid 1 to smithi10066867-15 2024-04-07T16:27:30.397 INFO:tasks.rados.rados.0.smithi100.stdout:3406: finishing write tid 2 to smithi10066867-15 2024-04-07T16:27:30.397 INFO:tasks.rados.rados.0.smithi100.stdout:3406: finishing write tid 3 to smithi10066867-15 2024-04-07T16:27:30.397 INFO:tasks.rados.rados.0.smithi100.stdout:3406: finishing write tid 4 to smithi10066867-15 2024-04-07T16:27:30.397 INFO:tasks.rados.rados.0.smithi100.stdout:3406: oid 15 updating version 0 to 1809 2024-04-07T16:27:30.397 INFO:tasks.rados.rados.0.smithi100.stdout:3406: oid 15 version 1809 is already newer than 1808 2024-04-07T16:27:30.397 INFO:tasks.rados.rados.0.smithi100.stdout:3406: oid 15 version 1809 is already newer than 1807 2024-04-07T16:27:30.397 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 15 v 1809 (ObjNum 1053 snap 309 seq_num 1053) dirty exists 2024-04-07T16:27:30.397 INFO:tasks.rados.rados.0.smithi100.stdout:3406: left oid 15 (ObjNum 1053 snap 309 seq_num 1053) 2024-04-07T16:27:30.397 INFO:tasks.rados.rados.0.smithi100.stdout:3409: expect (ObjNum 1008 snap 300 seq_num 1008) 2024-04-07T16:27:30.426 INFO:tasks.rados.rados.0.smithi100.stdout:3410: finishing rollback tid 0 to smithi10066867-4 2024-04-07T16:27:30.426 INFO:tasks.rados.rados.0.smithi100.stdout:3410: finishing rollback tid 1 to smithi10066867-4 2024-04-07T16:27:30.427 INFO:tasks.rados.rados.0.smithi100.stdout:3410: finishing rollback tid 2 to smithi10066867-4 2024-04-07T16:27:30.427 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 4 v 1813 (ObjNum 1041 snap 307 seq_num 1041) dirty exists 2024-04-07T16:27:30.427 INFO:tasks.rados.rados.0.smithi100.stdout:3403: done (3 left) 2024-04-07T16:27:30.427 INFO:tasks.rados.rados.0.smithi100.stdout:3406: done (2 left) 2024-04-07T16:27:30.427 INFO:tasks.rados.rados.0.smithi100.stdout:3409: done (1 left) 2024-04-07T16:27:30.427 INFO:tasks.rados.rados.0.smithi100.stdout:3410: done (0 left) 2024-04-07T16:27:30.427 INFO:tasks.rados.rados.0.smithi100.stdout:3411: snap_remove snap 308 2024-04-07T16:27:30.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:30 smithi100 bash[74942]: cluster 2024-04-07T16:27:28.792362+0000 mgr.y (mgr.25175) 348 : cluster [DBG] pgmap v629: 121 pgs: 1 active+clean+snaptrim, 120 active+clean; 135 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 3.9 MiB/s rd, 2.0 MiB/s wr, 24 op/s 2024-04-07T16:27:30.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:30 smithi100 bash[74942]: cluster 2024-04-07T16:27:29.102779+0000 mon.a (mon.0) 1142 : cluster [DBG] osdmap e691: 8 total, 8 up, 8 in 2024-04-07T16:27:30.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:30 smithi100 bash[76202]: cluster 2024-04-07T16:27:28.792362+0000 mgr.y (mgr.25175) 348 : cluster [DBG] pgmap v629: 121 pgs: 1 active+clean+snaptrim, 120 active+clean; 135 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 3.9 MiB/s rd, 2.0 MiB/s wr, 24 op/s 2024-04-07T16:27:30.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:30 smithi100 bash[76202]: cluster 2024-04-07T16:27:29.102779+0000 mon.a (mon.0) 1142 : cluster [DBG] osdmap e691: 8 total, 8 up, 8 in 2024-04-07T16:27:31.107 INFO:tasks.rados.rados.0.smithi100.stdout:3411: done (0 left) 2024-04-07T16:27:31.107 INFO:tasks.rados.rados.0.smithi100.stdout:3412: rmattr oid 9 current snap is 309 2024-04-07T16:27:31.107 INFO:tasks.rados.rados.0.smithi100.stdout:3412: done (0 left) 2024-04-07T16:27:31.107 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 40 current snap is 309 2024-04-07T16:27:31.107 INFO:tasks.rados.rados.0.smithi100.stdout:3413: seq_num 1055 ranges {2801664=475136,3276800=401408} 2024-04-07T16:27:31.125 INFO:tasks.rados.rados.0.smithi100.stdout:3413: writing smithi10066867-40 from 2801664 to 3276800 tid 1 2024-04-07T16:27:31.130 INFO:tasks.rados.rados.0.smithi100.stdout:3413: writing smithi10066867-40 from 3276800 to 3678208 tid 2 2024-04-07T16:27:31.130 INFO:tasks.rados.rados.0.smithi100.stdout:3414: snap_remove snap 307 2024-04-07T16:27:31.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:31 smithi144 bash[66654]: cluster 2024-04-07T16:27:30.110435+0000 mon.a (mon.0) 1143 : cluster [DBG] osdmap e692: 8 total, 8 up, 8 in 2024-04-07T16:27:31.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:31 smithi100 bash[74942]: cluster 2024-04-07T16:27:30.110435+0000 mon.a (mon.0) 1143 : cluster [DBG] osdmap e692: 8 total, 8 up, 8 in 2024-04-07T16:27:31.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:31 smithi100 bash[76202]: cluster 2024-04-07T16:27:30.110435+0000 mon.a (mon.0) 1143 : cluster [DBG] osdmap e692: 8 total, 8 up, 8 in 2024-04-07T16:27:31.922 INFO:tasks.rados.rados.0.smithi100.stdout:3414: done (1 left) 2024-04-07T16:27:31.922 INFO:tasks.rados.rados.0.smithi100.stdout:3415: copy_from oid 6 from oid 44 current snap is 309 2024-04-07T16:27:31.922 INFO:tasks.rados.rados.0.smithi100.stdout:3416: copy_from oid 41 from oid 42 current snap is 309 2024-04-07T16:27:31.922 INFO:tasks.rados.rados.0.smithi100.stdout:3413: finishing write tid 1 to smithi10066867-40 2024-04-07T16:27:31.922 INFO:tasks.rados.rados.0.smithi100.stdout:3413: finishing write tid 2 to smithi10066867-40 2024-04-07T16:27:31.922 INFO:tasks.rados.rados.0.smithi100.stdout:3413: finishing write tid 3 to smithi10066867-40 2024-04-07T16:27:31.922 INFO:tasks.rados.rados.0.smithi100.stdout:3413: finishing write tid 4 to smithi10066867-40 2024-04-07T16:27:31.922 INFO:tasks.rados.rados.0.smithi100.stdout:3413: oid 40 updating version 0 to 1649 2024-04-07T16:27:31.922 INFO:tasks.rados.rados.0.smithi100.stdout:3413: oid 40 version 1649 is already newer than 1647 2024-04-07T16:27:31.923 INFO:tasks.rados.rados.0.smithi100.stdout:3413: oid 40 version 1649 is already newer than 1648 2024-04-07T16:27:31.923 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 40 v 1649 (ObjNum 1055 snap 309 seq_num 1055) dirty exists 2024-04-07T16:27:31.923 INFO:tasks.rados.rados.0.smithi100.stdout:3413: left oid 40 (ObjNum 1055 snap 309 seq_num 1055) 2024-04-07T16:27:31.923 INFO:tasks.rados.rados.0.smithi100.stdout:3413: done (2 left) 2024-04-07T16:27:31.923 INFO:tasks.rados.rados.0.smithi100.stdout:3417: setattr oid 12 current snap is 309 2024-04-07T16:27:31.924 INFO:tasks.rados.rados.0.smithi100.stdout:3418: delete oid 1 current snap is 309 2024-04-07T16:27:31.938 INFO:tasks.rados.rados.0.smithi100.stdout:3416: finishing copy_from to smithi10066867-41 2024-04-07T16:27:31.938 INFO:tasks.rados.rados.0.smithi100.stdout:3416: got expected ENOENT (src dne) 2024-04-07T16:27:31.938 INFO:tasks.rados.rados.0.smithi100.stdout:3416: finishing copy_from racing read to smithi10066867-41 2024-04-07T16:27:31.938 INFO:tasks.rados.rados.0.smithi100.stdout:3415: finishing copy_from to smithi10066867-6 2024-04-07T16:27:31.938 INFO:tasks.rados.rados.0.smithi100.stdout:3415: got expected ENOENT (src dne) 2024-04-07T16:27:31.939 INFO:tasks.rados.rados.0.smithi100.stdout:3416: done (3 left) 2024-04-07T16:27:31.939 INFO:tasks.rados.rados.0.smithi100.stdout:3418: done (2 left) 2024-04-07T16:27:31.939 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 46 current snap is 309 2024-04-07T16:27:31.939 INFO:tasks.rados.rados.0.smithi100.stdout:3419: seq_num 1057 ranges {5791744=507904,6299648=401408} 2024-04-07T16:27:31.974 INFO:tasks.rados.rados.0.smithi100.stdout:3419: writing smithi10066867-46 from 5791744 to 6299648 tid 1 2024-04-07T16:27:31.976 INFO:tasks.rados.rados.0.smithi100.stdout:3419: writing smithi10066867-46 from 6299648 to 6701056 tid 2 2024-04-07T16:27:31.976 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 5 current snap is 309 2024-04-07T16:27:31.977 INFO:tasks.rados.rados.0.smithi100.stdout:3415: finishing copy_from racing read to smithi10066867-6 2024-04-07T16:27:31.977 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 12 v 1697 (ObjNum 65 snap 0 seq_num 1902991445) dirty exists 2024-04-07T16:27:31.977 INFO:tasks.rados.rados.0.smithi100.stdout:3420: seq_num 1058 ranges {0=524288,524288=401408} 2024-04-07T16:27:31.979 INFO:tasks.rados.rados.0.smithi100.stdout:3420: writing smithi10066867-5 from 0 to 524288 tid 1 2024-04-07T16:27:31.982 INFO:tasks.rados.rados.0.smithi100.stdout:3420: writing smithi10066867-5 from 524288 to 925696 tid 2 2024-04-07T16:27:31.982 INFO:tasks.rados.rados.0.smithi100.stdout:3415: done (3 left) 2024-04-07T16:27:31.982 INFO:tasks.rados.rados.0.smithi100.stdout:3417: done (2 left) 2024-04-07T16:27:31.982 INFO:tasks.rados.rados.0.smithi100.stdout:3421: setattr oid 35 current snap is 309 2024-04-07T16:27:31.988 INFO:tasks.rados.rados.0.smithi100.stdout:3422: rollback oid 18 current snap is 309 2024-04-07T16:27:31.988 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 18 to 300 2024-04-07T16:27:31.988 INFO:tasks.rados.rados.0.smithi100.stdout:3423: setattr oid 50 current snap is 309 2024-04-07T16:27:31.988 INFO:tasks.rados.rados.0.smithi100.stdout:3419: finishing write tid 1 to smithi10066867-46 2024-04-07T16:27:31.988 INFO:tasks.rados.rados.0.smithi100.stdout:3419: finishing write tid 2 to smithi10066867-46 2024-04-07T16:27:31.988 INFO:tasks.rados.rados.0.smithi100.stdout:3420: finishing write tid 1 to smithi10066867-5 2024-04-07T16:27:31.988 INFO:tasks.rados.rados.0.smithi100.stdout:3419: finishing write tid 3 to smithi10066867-46 2024-04-07T16:27:31.989 INFO:tasks.rados.rados.0.smithi100.stdout:3420: finishing write tid 2 to smithi10066867-5 2024-04-07T16:27:31.989 INFO:tasks.rados.rados.0.smithi100.stdout:3420: finishing write tid 3 to smithi10066867-5 2024-04-07T16:27:31.989 INFO:tasks.rados.rados.0.smithi100.stdout:3424: snap_create 2024-04-07T16:27:31.989 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 35 v 1702 (ObjNum 935 snap 280 seq_num 935) dirty exists 2024-04-07T16:27:31.991 INFO:tasks.rados.rados.0.smithi100.stdout:3422: finishing rollback tid 1 to smithi10066867-18 2024-04-07T16:27:31.991 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 18 v 1805 (ObjNum 33 snap 0 seq_num 4239406439) dirty dne 2024-04-07T16:27:31.991 INFO:tasks.rados.rados.0.smithi100.stdout:3420: finishing write tid 4 to smithi10066867-5 2024-04-07T16:27:31.991 INFO:tasks.rados.rados.0.smithi100.stdout:3420: oid 5 updating version 0 to 1698 2024-04-07T16:27:31.991 INFO:tasks.rados.rados.0.smithi100.stdout:3420: oid 5 updating version 1698 to 1699 2024-04-07T16:27:31.991 INFO:tasks.rados.rados.0.smithi100.stdout:3420: oid 5 updating version 1699 to 1700 2024-04-07T16:27:31.991 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 5 v 1700 (ObjNum 1058 snap 309 seq_num 1058) dirty exists 2024-04-07T16:27:31.991 INFO:tasks.rados.rados.0.smithi100.stdout:3420: left oid 5 (ObjNum 1058 snap 309 seq_num 1058) 2024-04-07T16:27:31.991 INFO:tasks.rados.rados.0.smithi100.stdout:3419: finishing write tid 4 to smithi10066867-46 2024-04-07T16:27:31.992 INFO:tasks.rados.rados.0.smithi100.stdout:3419: oid 46 updating version 0 to 1574 2024-04-07T16:27:31.992 INFO:tasks.rados.rados.0.smithi100.stdout:3419: oid 46 version 1574 is already newer than 1573 2024-04-07T16:27:31.992 INFO:tasks.rados.rados.0.smithi100.stdout:3419: oid 46 version 1574 is already newer than 1572 2024-04-07T16:27:31.992 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 46 v 1574 (ObjNum 1057 snap 309 seq_num 1057) dirty exists 2024-04-07T16:27:31.992 INFO:tasks.rados.rados.0.smithi100.stdout:3419: left oid 46 (ObjNum 1057 snap 309 seq_num 1057) 2024-04-07T16:27:31.994 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 50 v 1551 (ObjNum 1043 snap 307 seq_num 1043) dirty exists 2024-04-07T16:27:32.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:32 smithi144 bash[66654]: cluster 2024-04-07T16:27:30.793133+0000 mgr.y (mgr.25175) 349 : cluster [DBG] pgmap v632: 121 pgs: 1 active+clean+snaptrim, 120 active+clean; 154 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 16 MiB/s rd, 5.3 MiB/s wr, 46 op/s 2024-04-07T16:27:32.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:32 smithi144 bash[66654]: cluster 2024-04-07T16:27:31.121856+0000 mon.a (mon.0) 1144 : cluster [DBG] osdmap e693: 8 total, 8 up, 8 in 2024-04-07T16:27:32.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:32 smithi144 bash[66654]: audit 2024-04-07T16:27:31.729186+0000 mon.a (mon.0) 1145 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:27:32.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:32 smithi144 bash[66654]: cluster 2024-04-07T16:27:31.934865+0000 mon.a (mon.0) 1146 : cluster [DBG] osdmap e694: 8 total, 8 up, 8 in 2024-04-07T16:27:32.356 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:27:32 smithi144 bash[95233]: ts=2024-04-07T16:27:32.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:27:32.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:32 smithi100 bash[76202]: cluster 2024-04-07T16:27:30.793133+0000 mgr.y (mgr.25175) 349 : cluster [DBG] pgmap v632: 121 pgs: 1 active+clean+snaptrim, 120 active+clean; 154 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 16 MiB/s rd, 5.3 MiB/s wr, 46 op/s 2024-04-07T16:27:32.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:32 smithi100 bash[76202]: cluster 2024-04-07T16:27:31.121856+0000 mon.a (mon.0) 1144 : cluster [DBG] osdmap e693: 8 total, 8 up, 8 in 2024-04-07T16:27:32.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:32 smithi100 bash[76202]: audit 2024-04-07T16:27:31.729186+0000 mon.a (mon.0) 1145 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:27:32.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:32 smithi100 bash[76202]: cluster 2024-04-07T16:27:31.934865+0000 mon.a (mon.0) 1146 : cluster [DBG] osdmap e694: 8 total, 8 up, 8 in 2024-04-07T16:27:32.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:32 smithi100 bash[74942]: cluster 2024-04-07T16:27:30.793133+0000 mgr.y (mgr.25175) 349 : cluster [DBG] pgmap v632: 121 pgs: 1 active+clean+snaptrim, 120 active+clean; 154 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 16 MiB/s rd, 5.3 MiB/s wr, 46 op/s 2024-04-07T16:27:32.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:32 smithi100 bash[74942]: cluster 2024-04-07T16:27:31.121856+0000 mon.a (mon.0) 1144 : cluster [DBG] osdmap e693: 8 total, 8 up, 8 in 2024-04-07T16:27:32.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:32 smithi100 bash[74942]: audit 2024-04-07T16:27:31.729186+0000 mon.a (mon.0) 1145 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:27:32.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:32 smithi100 bash[74942]: cluster 2024-04-07T16:27:31.934865+0000 mon.a (mon.0) 1146 : cluster [DBG] osdmap e694: 8 total, 8 up, 8 in 2024-04-07T16:27:32.921 INFO:tasks.rados.rados.0.smithi100.stdout:3419: done (5 left) 2024-04-07T16:27:32.922 INFO:tasks.rados.rados.0.smithi100.stdout:3420: done (4 left) 2024-04-07T16:27:32.922 INFO:tasks.rados.rados.0.smithi100.stdout:3421: done (3 left) 2024-04-07T16:27:32.922 INFO:tasks.rados.rados.0.smithi100.stdout:3422: done (2 left) 2024-04-07T16:27:32.922 INFO:tasks.rados.rados.0.smithi100.stdout:3423: done (1 left) 2024-04-07T16:27:32.922 INFO:tasks.rados.rados.0.smithi100.stdout:3424: done (0 left) 2024-04-07T16:27:32.922 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 45 current snap is 310 2024-04-07T16:27:32.923 INFO:tasks.rados.rados.0.smithi100.stdout:3425: seq_num 1059 ranges {6438912=540672,6979584=401408} 2024-04-07T16:27:32.960 INFO:tasks.rados.rados.0.smithi100.stdout:3425: writing smithi10066867-45 from 6438912 to 6979584 tid 1 2024-04-07T16:27:32.963 INFO:tasks.rados.rados.0.smithi100.stdout:3425: writing smithi10066867-45 from 6979584 to 7380992 tid 2 2024-04-07T16:27:32.963 INFO:tasks.rados.rados.0.smithi100.stdout:3426: read oid 41 snap 300 2024-04-07T16:27:32.963 INFO:tasks.rados.rados.0.smithi100.stdout:3426: expect (ObjNum 984 snap 298 seq_num 984) 2024-04-07T16:27:32.963 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 35 current snap is 310 2024-04-07T16:27:32.963 INFO:tasks.rados.rados.0.smithi100.stdout:3427: seq_num 1060 ranges {2080768=557056,2637824=401408} 2024-04-07T16:27:32.978 INFO:tasks.rados.rados.0.smithi100.stdout:3427: writing smithi10066867-35 from 2080768 to 2637824 tid 1 2024-04-07T16:27:32.980 INFO:tasks.rados.rados.0.smithi100.stdout:3427: writing smithi10066867-35 from 2637824 to 3039232 tid 2 2024-04-07T16:27:32.981 INFO:tasks.rados.rados.0.smithi100.stdout:3428: delete oid 40 current snap is 310 2024-04-07T16:27:32.981 INFO:tasks.rados.rados.0.smithi100.stdout:3425: finishing write tid 1 to smithi10066867-45 2024-04-07T16:27:32.981 INFO:tasks.rados.rados.0.smithi100.stdout:3425: finishing write tid 2 to smithi10066867-45 2024-04-07T16:27:32.981 INFO:tasks.rados.rados.0.smithi100.stdout:3425: finishing write tid 3 to smithi10066867-45 2024-04-07T16:27:32.981 INFO:tasks.rados.rados.0.smithi100.stdout:3425: finishing write tid 4 to smithi10066867-45 2024-04-07T16:27:32.981 INFO:tasks.rados.rados.0.smithi100.stdout:3425: oid 45 updating version 0 to 1552 2024-04-07T16:27:32.981 INFO:tasks.rados.rados.0.smithi100.stdout:3425: oid 45 updating version 1552 to 1553 2024-04-07T16:27:32.981 INFO:tasks.rados.rados.0.smithi100.stdout:3425: oid 45 updating version 1553 to 1554 2024-04-07T16:27:32.981 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 45 v 1554 (ObjNum 1059 snap 310 seq_num 1059) dirty exists 2024-04-07T16:27:32.981 INFO:tasks.rados.rados.0.smithi100.stdout:3425: left oid 45 (ObjNum 1059 snap 310 seq_num 1059) 2024-04-07T16:27:32.990 INFO:tasks.rados.rados.0.smithi100.stdout:3425: done (3 left) 2024-04-07T16:27:32.990 INFO:tasks.rados.rados.0.smithi100.stdout:3428: done (2 left) 2024-04-07T16:27:32.990 INFO:tasks.rados.rados.0.smithi100.stdout:3429: copy_from oid 10 from oid 50 current snap is 310 2024-04-07T16:27:32.990 INFO:tasks.rados.rados.0.smithi100.stdout:3430: setattr oid 26 current snap is 310 2024-04-07T16:27:33.000 INFO:tasks.rados.rados.0.smithi100.stdout:3431: rollback oid 31 current snap is 310 2024-04-07T16:27:33.000 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 31 to 306 2024-04-07T16:27:33.000 INFO:tasks.rados.rados.0.smithi100.stdout:3432: rmattr oid 33 current snap is 310 2024-04-07T16:27:33.000 INFO:tasks.rados.rados.0.smithi100.stdout:3432: done (5 left) 2024-04-07T16:27:33.001 INFO:tasks.rados.rados.0.smithi100.stdout:3433: rollback oid 25 current snap is 310 2024-04-07T16:27:33.001 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 25 to 300 2024-04-07T16:27:33.001 INFO:tasks.rados.rados.0.smithi100.stdout:3434: copy_from oid 5 from oid 47 current snap is 310 2024-04-07T16:27:33.001 INFO:tasks.rados.rados.0.smithi100.stdout:3435: rollback oid 2 current snap is 310 2024-04-07T16:27:33.001 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 2 to 300 2024-04-07T16:27:33.001 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 18 current snap is 310 2024-04-07T16:27:33.001 INFO:tasks.rados.rados.0.smithi100.stdout:3436: seq_num 1062 ranges {0=589824,589824=401408} 2024-04-07T16:27:33.002 INFO:tasks.rados.rados.0.smithi100.stdout:3436: writing smithi10066867-18 from 0 to 589824 tid 1 2024-04-07T16:27:33.002 INFO:tasks.rados.rados.0.smithi100.stdout:3436: writing smithi10066867-18 from 589824 to 991232 tid 2 2024-04-07T16:27:33.002 INFO:tasks.rados.rados.0.smithi100.stdout:3437: read oid 27 snap 300 2024-04-07T16:27:33.002 INFO:tasks.rados.rados.0.smithi100.stdout:3437: expect (ObjNum 953 snap 286 seq_num 953) 2024-04-07T16:27:33.002 INFO:tasks.rados.rados.0.smithi100.stdout:3438: read oid 20 snap -1 2024-04-07T16:27:33.002 INFO:tasks.rados.rados.0.smithi100.stdout:3438: expect (ObjNum 1007 snap 300 seq_num 1007) 2024-04-07T16:27:33.003 INFO:tasks.rados.rados.0.smithi100.stdout:3439: delete oid 42 current snap is 310 2024-04-07T16:27:33.003 INFO:tasks.rados.rados.0.smithi100.stdout:3435: finishing rollback tid 1 to smithi10066867-2 2024-04-07T16:27:33.003 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 2 v 0 (ObjNum 273 snap 0 seq_num 0) dirty dne 2024-04-07T16:27:33.019 INFO:tasks.rados.rados.0.smithi100.stdout:3426: expect (ObjNum 984 snap 298 seq_num 984) 2024-04-07T16:27:33.170 INFO:tasks.rados.rados.0.smithi100.stdout:3434: finishing copy_from to smithi10066867-5 2024-04-07T16:27:33.170 INFO:tasks.rados.rados.0.smithi100.stdout:3434: got expected ENOENT (src dne) 2024-04-07T16:27:33.170 INFO:tasks.rados.rados.0.smithi100.stdout:3434: finishing copy_from racing read to smithi10066867-5 2024-04-07T16:27:33.170 INFO:tasks.rados.rados.0.smithi100.stdout:3429: finishing copy_from racing read to smithi10066867-10 2024-04-07T16:27:33.171 INFO:tasks.rados.rados.0.smithi100.stdout:3436: finishing write tid 1 to smithi10066867-18 2024-04-07T16:27:33.171 INFO:tasks.rados.rados.0.smithi100.stdout:3436: finishing write tid 2 to smithi10066867-18 2024-04-07T16:27:33.171 INFO:tasks.rados.rados.0.smithi100.stdout:3436: finishing write tid 3 to smithi10066867-18 2024-04-07T16:27:33.171 INFO:tasks.rados.rados.0.smithi100.stdout:3427: finishing write tid 1 to smithi10066867-35 2024-04-07T16:27:33.171 INFO:tasks.rados.rados.0.smithi100.stdout:3427: finishing write tid 2 to smithi10066867-35 2024-04-07T16:27:33.171 INFO:tasks.rados.rados.0.smithi100.stdout:3427: finishing write tid 3 to smithi10066867-35 2024-04-07T16:27:33.171 INFO:tasks.rados.rados.0.smithi100.stdout:3431: finishing rollback tid 1 to smithi10066867-31 2024-04-07T16:27:33.171 INFO:tasks.rados.rados.0.smithi100.stdout:3431: finishing rollback tid 2 to smithi10066867-31 2024-04-07T16:27:33.171 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 31 v 1707 (ObjNum 993 snap 300 seq_num 993) dirty exists 2024-04-07T16:27:33.171 INFO:tasks.rados.rados.0.smithi100.stdout:3436: finishing write tid 4 to smithi10066867-18 2024-04-07T16:27:33.171 INFO:tasks.rados.rados.0.smithi100.stdout:3436: oid 18 updating version 0 to 1830 2024-04-07T16:27:33.171 INFO:tasks.rados.rados.0.smithi100.stdout:3436: oid 18 version 1830 is already newer than 1829 2024-04-07T16:27:33.171 INFO:tasks.rados.rados.0.smithi100.stdout:3436: oid 18 updating version 1830 to 1831 2024-04-07T16:27:33.171 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 18 v 1831 (ObjNum 1062 snap 310 seq_num 1062) dirty exists 2024-04-07T16:27:33.171 INFO:tasks.rados.rados.0.smithi100.stdout:3436: left oid 18 (ObjNum 1062 snap 310 seq_num 1062) 2024-04-07T16:27:33.172 INFO:tasks.rados.rados.0.smithi100.stdout:3437: expect (ObjNum 953 snap 286 seq_num 953) 2024-04-07T16:27:33.351 INFO:tasks.rados.rados.0.smithi100.stdout:3429: finishing copy_from to smithi10066867-10 2024-04-07T16:27:33.351 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 10 v 1551 (ObjNum 1043 snap 307 seq_num 1043) dirty exists 2024-04-07T16:27:33.351 INFO:tasks.rados.rados.0.smithi100.stdout:3427: finishing write tid 4 to smithi10066867-35 2024-04-07T16:27:33.351 INFO:tasks.rados.rados.0.smithi100.stdout:3427: oid 35 updating version 0 to 1703 2024-04-07T16:27:33.351 INFO:tasks.rados.rados.0.smithi100.stdout:3427: oid 35 updating version 1703 to 1704 2024-04-07T16:27:33.351 INFO:tasks.rados.rados.0.smithi100.stdout:3427: oid 35 updating version 1704 to 1705 2024-04-07T16:27:33.351 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 35 v 1705 (ObjNum 1060 snap 310 seq_num 1060) dirty exists 2024-04-07T16:27:33.352 INFO:tasks.rados.rados.0.smithi100.stdout:3427: left oid 35 (ObjNum 1060 snap 310 seq_num 1060) 2024-04-07T16:27:33.352 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 26 v 1696 (ObjNum 1003 snap 300 seq_num 1003) dirty exists 2024-04-07T16:27:33.352 INFO:tasks.rados.rados.0.smithi100.stdout:3433: finishing rollback tid 0 to smithi10066867-25 2024-04-07T16:27:33.352 INFO:tasks.rados.rados.0.smithi100.stdout:3433: finishing rollback tid 1 to smithi10066867-25 2024-04-07T16:27:33.352 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 25 v 1698 (ObjNum 33 snap 0 seq_num 4239518300) dirty dne 2024-04-07T16:27:33.352 INFO:tasks.rados.rados.0.smithi100.stdout:3438: expect (ObjNum 1007 snap 300 seq_num 1007) 2024-04-07T16:27:33.516 INFO:tasks.rados.rados.0.smithi100.stdout:3426: done (11 left) 2024-04-07T16:27:33.516 INFO:tasks.rados.rados.0.smithi100.stdout:3427: done (10 left) 2024-04-07T16:27:33.516 INFO:tasks.rados.rados.0.smithi100.stdout:3429: done (9 left) 2024-04-07T16:27:33.516 INFO:tasks.rados.rados.0.smithi100.stdout:3430: done (8 left) 2024-04-07T16:27:33.516 INFO:tasks.rados.rados.0.smithi100.stdout:3431: done (7 left) 2024-04-07T16:27:33.516 INFO:tasks.rados.rados.0.smithi100.stdout:3433: done (6 left) 2024-04-07T16:27:33.516 INFO:tasks.rados.rados.0.smithi100.stdout:3434: done (5 left) 2024-04-07T16:27:33.516 INFO:tasks.rados.rados.0.smithi100.stdout:3435: done (4 left) 2024-04-07T16:27:33.516 INFO:tasks.rados.rados.0.smithi100.stdout:3436: done (3 left) 2024-04-07T16:27:33.516 INFO:tasks.rados.rados.0.smithi100.stdout:3437: done (2 left) 2024-04-07T16:27:33.516 INFO:tasks.rados.rados.0.smithi100.stdout:3438: done (1 left) 2024-04-07T16:27:33.516 INFO:tasks.rados.rados.0.smithi100.stdout:3439: done (0 left) 2024-04-07T16:27:33.516 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 47 current snap is 310 2024-04-07T16:27:33.517 INFO:tasks.rados.rados.0.smithi100.stdout:3440: seq_num 1064 ranges {0=622592,622592=401408} 2024-04-07T16:27:33.519 INFO:tasks.rados.rados.0.smithi100.stdout:3440: writing smithi10066867-47 from 0 to 622592 tid 1 2024-04-07T16:27:33.522 INFO:tasks.rados.rados.0.smithi100.stdout:3440: writing smithi10066867-47 from 622592 to 1024000 tid 2 2024-04-07T16:27:33.522 INFO:tasks.rados.rados.0.smithi100.stdout:3441: setattr oid 7 current snap is 310 2024-04-07T16:27:33.524 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 2 current snap is 310 2024-04-07T16:27:33.524 INFO:tasks.rados.rados.0.smithi100.stdout:3442: seq_num 1065 ranges {0=638976,638976=401408} 2024-04-07T16:27:33.531 INFO:tasks.rados.rados.0.smithi100.stdout:3442: writing smithi10066867-2 from 0 to 638976 tid 1 2024-04-07T16:27:33.537 INFO:tasks.rados.rados.0.smithi100.stdout:3442: writing smithi10066867-2 from 638976 to 1040384 tid 2 2024-04-07T16:27:33.537 INFO:tasks.rados.rados.0.smithi100.stdout:3443: rmattr oid 28 current snap is 310 2024-04-07T16:27:33.537 INFO:tasks.rados.rados.0.smithi100.stdout:3443: done (3 left) 2024-04-07T16:27:33.537 INFO:tasks.rados.rados.0.smithi100.stdout:3444: snap_create 2024-04-07T16:27:33.538 INFO:tasks.rados.rados.0.smithi100.stdout:3440: finishing write tid 1 to smithi10066867-47 2024-04-07T16:27:33.538 INFO:tasks.rados.rados.0.smithi100.stdout:3440: finishing write tid 2 to smithi10066867-47 2024-04-07T16:27:33.538 INFO:tasks.rados.rados.0.smithi100.stdout:3440: finishing write tid 3 to smithi10066867-47 2024-04-07T16:27:33.538 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 7 v 1191 (ObjNum 1051 snap 307 seq_num 1051) dirty exists 2024-04-07T16:27:33.538 INFO:tasks.rados.rados.0.smithi100.stdout:3440: finishing write tid 4 to smithi10066867-47 2024-04-07T16:27:33.538 INFO:tasks.rados.rados.0.smithi100.stdout:3440: oid 47 updating version 0 to 1212 2024-04-07T16:27:33.538 INFO:tasks.rados.rados.0.smithi100.stdout:3440: oid 47 version 1212 is already newer than 1211 2024-04-07T16:27:33.538 INFO:tasks.rados.rados.0.smithi100.stdout:3440: oid 47 version 1212 is already newer than 1210 2024-04-07T16:27:33.538 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 47 v 1212 (ObjNum 1064 snap 310 seq_num 1064) dirty exists 2024-04-07T16:27:33.538 INFO:tasks.rados.rados.0.smithi100.stdout:3440: left oid 47 (ObjNum 1064 snap 310 seq_num 1064) 2024-04-07T16:27:33.544 INFO:tasks.rados.rados.0.smithi100.stdout:3442: finishing write tid 1 to smithi10066867-2 2024-04-07T16:27:33.544 INFO:tasks.rados.rados.0.smithi100.stdout:3442: finishing write tid 2 to smithi10066867-2 2024-04-07T16:27:33.544 INFO:tasks.rados.rados.0.smithi100.stdout:3442: finishing write tid 3 to smithi10066867-2 2024-04-07T16:27:33.546 INFO:tasks.rados.rados.0.smithi100.stdout:3442: finishing write tid 4 to smithi10066867-2 2024-04-07T16:27:33.546 INFO:tasks.rados.rados.0.smithi100.stdout:3442: oid 2 updating version 0 to 1653 2024-04-07T16:27:33.546 INFO:tasks.rados.rados.0.smithi100.stdout:3442: oid 2 version 1653 is already newer than 1652 2024-04-07T16:27:33.546 INFO:tasks.rados.rados.0.smithi100.stdout:3442: oid 2 version 1653 is already newer than 1651 2024-04-07T16:27:33.547 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 2 v 1653 (ObjNum 1065 snap 310 seq_num 1065) dirty exists 2024-04-07T16:27:33.547 INFO:tasks.rados.rados.0.smithi100.stdout:3442: left oid 2 (ObjNum 1065 snap 310 seq_num 1065) 2024-04-07T16:27:33.688 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:33 smithi100 bash[74942]: audit 2024-04-07T16:27:32.417736+0000 mon.a (mon.0) 1147 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:27:33.688 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:33 smithi100 bash[74942]: audit 2024-04-07T16:27:32.429886+0000 mon.a (mon.0) 1148 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:27:33.688 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:33 smithi100 bash[74942]: audit 2024-04-07T16:27:32.432617+0000 mon.a (mon.0) 1149 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:27:33.688 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:33 smithi100 bash[74942]: audit 2024-04-07T16:27:32.434681+0000 mon.a (mon.0) 1150 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:27:33.688 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:33 smithi100 bash[74942]: audit 2024-04-07T16:27:32.445175+0000 mon.a (mon.0) 1151 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:27:33.688 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:33 smithi100 bash[74942]: audit 2024-04-07T16:27:32.534091+0000 mon.a (mon.0) 1152 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-07T16:27:33.688 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:33 smithi100 bash[74942]: audit 2024-04-07T16:27:32.535194+0000 mgr.y (mgr.25175) 350 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-07T16:27:33.688 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:33 smithi100 bash[74942]: audit 2024-04-07T16:27:32.552374+0000 mon.a (mon.0) 1153 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:27:33.688 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:33 smithi100 bash[74942]: audit 2024-04-07T16:27:32.557119+0000 mon.a (mon.0) 1154 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:27:33.688 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:33 smithi100 bash[74942]: audit 2024-04-07T16:27:32.560728+0000 mon.a (mon.0) 1155 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:27:33.689 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:33 smithi100 bash[74942]: audit 2024-04-07T16:27:32.563161+0000 mon.a (mon.0) 1156 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:27:33.689 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:33 smithi100 bash[74942]: audit 2024-04-07T16:27:32.568599+0000 mon.a (mon.0) 1157 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:27:33.689 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:33 smithi100 bash[74942]: audit 2024-04-07T16:27:32.575017+0000 mon.a (mon.0) 1158 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:27:33.689 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:33 smithi100 bash[74942]: audit 2024-04-07T16:27:32.577414+0000 mon.a (mon.0) 1159 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:27:33.689 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:33 smithi100 bash[74942]: audit 2024-04-07T16:27:32.579771+0000 mon.a (mon.0) 1160 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:27:33.689 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:33 smithi100 bash[74942]: audit 2024-04-07T16:27:32.582127+0000 mon.a (mon.0) 1161 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:27:33.689 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:33 smithi100 bash[74942]: audit 2024-04-07T16:27:32.584540+0000 mon.a (mon.0) 1162 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:27:33.689 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:33 smithi100 bash[74942]: audit 2024-04-07T16:27:32.587003+0000 mon.a (mon.0) 1163 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:27:33.689 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:33 smithi100 bash[74942]: audit 2024-04-07T16:27:32.589879+0000 mon.a (mon.0) 1164 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:27:33.689 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:33 smithi100 bash[76202]: audit 2024-04-07T16:27:32.417736+0000 mon.a (mon.0) 1147 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:27:33.689 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:33 smithi100 bash[76202]: audit 2024-04-07T16:27:32.429886+0000 mon.a (mon.0) 1148 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:27:33.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:33 smithi100 bash[76202]: audit 2024-04-07T16:27:32.432617+0000 mon.a (mon.0) 1149 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:27:33.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:33 smithi100 bash[76202]: audit 2024-04-07T16:27:32.434681+0000 mon.a (mon.0) 1150 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:27:33.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:33 smithi100 bash[76202]: audit 2024-04-07T16:27:32.445175+0000 mon.a (mon.0) 1151 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:27:33.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:33 smithi100 bash[76202]: audit 2024-04-07T16:27:32.534091+0000 mon.a (mon.0) 1152 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-07T16:27:33.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:33 smithi100 bash[76202]: audit 2024-04-07T16:27:32.535194+0000 mgr.y (mgr.25175) 350 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-07T16:27:33.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:33 smithi100 bash[76202]: audit 2024-04-07T16:27:32.552374+0000 mon.a (mon.0) 1153 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:27:33.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:33 smithi100 bash[76202]: audit 2024-04-07T16:27:32.557119+0000 mon.a (mon.0) 1154 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:27:33.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:33 smithi100 bash[76202]: audit 2024-04-07T16:27:32.560728+0000 mon.a (mon.0) 1155 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:27:33.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:33 smithi100 bash[76202]: audit 2024-04-07T16:27:32.563161+0000 mon.a (mon.0) 1156 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:27:33.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:33 smithi100 bash[76202]: audit 2024-04-07T16:27:32.568599+0000 mon.a (mon.0) 1157 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:27:33.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:33 smithi100 bash[76202]: audit 2024-04-07T16:27:32.575017+0000 mon.a (mon.0) 1158 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:27:33.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:33 smithi100 bash[76202]: audit 2024-04-07T16:27:32.577414+0000 mon.a (mon.0) 1159 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:27:33.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:33 smithi100 bash[76202]: audit 2024-04-07T16:27:32.579771+0000 mon.a (mon.0) 1160 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:27:33.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:33 smithi100 bash[76202]: audit 2024-04-07T16:27:32.582127+0000 mon.a (mon.0) 1161 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:27:33.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:33 smithi100 bash[76202]: audit 2024-04-07T16:27:32.584540+0000 mon.a (mon.0) 1162 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:27:33.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:33 smithi100 bash[76202]: audit 2024-04-07T16:27:32.587003+0000 mon.a (mon.0) 1163 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:27:33.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:33 smithi100 bash[76202]: audit 2024-04-07T16:27:32.589879+0000 mon.a (mon.0) 1164 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:27:33.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:33 smithi100 bash[74942]: audit 2024-04-07T16:27:32.593111+0000 mon.a (mon.0) 1165 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:27:33.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:33 smithi100 bash[74942]: audit 2024-04-07T16:27:32.597286+0000 mon.a (mon.0) 1166 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:27:33.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:33 smithi100 bash[74942]: audit 2024-04-07T16:27:32.600941+0000 mon.a (mon.0) 1167 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:27:33.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:33 smithi100 bash[74942]: cluster 2024-04-07T16:27:32.794255+0000 mgr.y (mgr.25175) 351 : cluster [DBG] pgmap v635: 121 pgs: 121 active+clean; 154 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 13 MiB/s rd, 3.8 MiB/s wr, 25 op/s 2024-04-07T16:27:33.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:33 smithi100 bash[74942]: cluster 2024-04-07T16:27:32.936060+0000 mon.a (mon.0) 1168 : cluster [DBG] osdmap e695: 8 total, 8 up, 8 in 2024-04-07T16:27:33.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:33 smithi100 bash[76202]: audit 2024-04-07T16:27:32.593111+0000 mon.a (mon.0) 1165 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:27:33.692 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:33 smithi100 bash[76202]: audit 2024-04-07T16:27:32.597286+0000 mon.a (mon.0) 1166 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:27:33.692 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:33 smithi100 bash[76202]: audit 2024-04-07T16:27:32.600941+0000 mon.a (mon.0) 1167 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:27:33.692 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:33 smithi100 bash[76202]: cluster 2024-04-07T16:27:32.794255+0000 mgr.y (mgr.25175) 351 : cluster [DBG] pgmap v635: 121 pgs: 121 active+clean; 154 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 13 MiB/s rd, 3.8 MiB/s wr, 25 op/s 2024-04-07T16:27:33.692 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:33 smithi100 bash[76202]: cluster 2024-04-07T16:27:32.936060+0000 mon.a (mon.0) 1168 : cluster [DBG] osdmap e695: 8 total, 8 up, 8 in 2024-04-07T16:27:33.856 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:33 smithi144 bash[66654]: audit 2024-04-07T16:27:32.417736+0000 mon.a (mon.0) 1147 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:27:33.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:33 smithi144 bash[66654]: audit 2024-04-07T16:27:32.429886+0000 mon.a (mon.0) 1148 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:27:33.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:33 smithi144 bash[66654]: audit 2024-04-07T16:27:32.432617+0000 mon.a (mon.0) 1149 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:27:33.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:33 smithi144 bash[66654]: audit 2024-04-07T16:27:32.434681+0000 mon.a (mon.0) 1150 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:27:33.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:33 smithi144 bash[66654]: audit 2024-04-07T16:27:32.445175+0000 mon.a (mon.0) 1151 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:27:33.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:33 smithi144 bash[66654]: audit 2024-04-07T16:27:32.534091+0000 mon.a (mon.0) 1152 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-07T16:27:33.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:33 smithi144 bash[66654]: audit 2024-04-07T16:27:32.535194+0000 mgr.y (mgr.25175) 350 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-07T16:27:33.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:33 smithi144 bash[66654]: audit 2024-04-07T16:27:32.552374+0000 mon.a (mon.0) 1153 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:27:33.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:33 smithi144 bash[66654]: audit 2024-04-07T16:27:32.557119+0000 mon.a (mon.0) 1154 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:27:33.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:33 smithi144 bash[66654]: audit 2024-04-07T16:27:32.560728+0000 mon.a (mon.0) 1155 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:27:33.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:33 smithi144 bash[66654]: audit 2024-04-07T16:27:32.563161+0000 mon.a (mon.0) 1156 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:27:33.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:33 smithi144 bash[66654]: audit 2024-04-07T16:27:32.568599+0000 mon.a (mon.0) 1157 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:27:33.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:33 smithi144 bash[66654]: audit 2024-04-07T16:27:32.575017+0000 mon.a (mon.0) 1158 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:27:33.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:33 smithi144 bash[66654]: audit 2024-04-07T16:27:32.577414+0000 mon.a (mon.0) 1159 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:27:33.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:33 smithi144 bash[66654]: audit 2024-04-07T16:27:32.579771+0000 mon.a (mon.0) 1160 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:27:33.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:33 smithi144 bash[66654]: audit 2024-04-07T16:27:32.582127+0000 mon.a (mon.0) 1161 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:27:33.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:33 smithi144 bash[66654]: audit 2024-04-07T16:27:32.584540+0000 mon.a (mon.0) 1162 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:27:33.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:33 smithi144 bash[66654]: audit 2024-04-07T16:27:32.587003+0000 mon.a (mon.0) 1163 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:27:33.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:33 smithi144 bash[66654]: audit 2024-04-07T16:27:32.589879+0000 mon.a (mon.0) 1164 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:27:33.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:33 smithi144 bash[66654]: audit 2024-04-07T16:27:32.593111+0000 mon.a (mon.0) 1165 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:27:33.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:33 smithi144 bash[66654]: audit 2024-04-07T16:27:32.597286+0000 mon.a (mon.0) 1166 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:27:33.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:33 smithi144 bash[66654]: audit 2024-04-07T16:27:32.600941+0000 mon.a (mon.0) 1167 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:27:33.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:33 smithi144 bash[66654]: cluster 2024-04-07T16:27:32.794255+0000 mgr.y (mgr.25175) 351 : cluster [DBG] pgmap v635: 121 pgs: 121 active+clean; 154 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 13 MiB/s rd, 3.8 MiB/s wr, 25 op/s 2024-04-07T16:27:33.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:33 smithi144 bash[66654]: cluster 2024-04-07T16:27:32.936060+0000 mon.a (mon.0) 1168 : cluster [DBG] osdmap e695: 8 total, 8 up, 8 in 2024-04-07T16:27:34.438 INFO:tasks.rados.rados.0.smithi100.stdout:3440: done (3 left) 2024-04-07T16:27:34.438 INFO:tasks.rados.rados.0.smithi100.stdout:3441: done (2 left) 2024-04-07T16:27:34.438 INFO:tasks.rados.rados.0.smithi100.stdout:3442: done (1 left) 2024-04-07T16:27:34.438 INFO:tasks.rados.rados.0.smithi100.stdout:3444: done (0 left) 2024-04-07T16:27:34.438 INFO:tasks.rados.rados.0.smithi100.stdout:3445: copy_from oid 26 from oid 49 current snap is 311 2024-04-07T16:27:34.438 INFO:tasks.rados.rados.0.smithi100.stdout:3446: setattr oid 24 current snap is 311 2024-04-07T16:27:34.439 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 16 current snap is 311 2024-04-07T16:27:34.439 INFO:tasks.rados.rados.0.smithi100.stdout:3447: seq_num 1066 ranges {1523712=663552,2187264=401408} 2024-04-07T16:27:34.452 INFO:tasks.rados.rados.0.smithi100.stdout:3447: writing smithi10066867-16 from 1523712 to 2187264 tid 1 2024-04-07T16:27:34.455 INFO:tasks.rados.rados.0.smithi100.stdout:3447: writing smithi10066867-16 from 2187264 to 2588672 tid 2 2024-04-07T16:27:34.455 INFO:tasks.rados.rados.0.smithi100.stdout:3448: delete oid 41 current snap is 311 2024-04-07T16:27:34.455 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 24 v 1269 (ObjNum 1030 snap 306 seq_num 1030) dirty exists 2024-04-07T16:27:34.455 INFO:tasks.rados.rados.0.smithi100.stdout:3445: finishing copy_from racing read to smithi10066867-26 2024-04-07T16:27:34.456 INFO:tasks.rados.rados.0.smithi100.stdout:3445: finishing copy_from to smithi10066867-26 2024-04-07T16:27:34.456 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 26 v 1262 (ObjNum 1009 snap 300 seq_num 1009) dirty exists 2024-04-07T16:27:34.460 INFO:tasks.rados.rados.0.smithi100.stdout:3447: finishing write tid 1 to smithi10066867-16 2024-04-07T16:27:34.461 INFO:tasks.rados.rados.0.smithi100.stdout:3447: finishing write tid 2 to smithi10066867-16 2024-04-07T16:27:34.461 INFO:tasks.rados.rados.0.smithi100.stdout:3447: finishing write tid 3 to smithi10066867-16 2024-04-07T16:27:34.463 INFO:tasks.rados.rados.0.smithi100.stdout:3447: finishing write tid 4 to smithi10066867-16 2024-04-07T16:27:34.463 INFO:tasks.rados.rados.0.smithi100.stdout:3447: oid 16 updating version 0 to 1703 2024-04-07T16:27:34.463 INFO:tasks.rados.rados.0.smithi100.stdout:3447: oid 16 version 1703 is already newer than 1701 2024-04-07T16:27:34.463 INFO:tasks.rados.rados.0.smithi100.stdout:3447: oid 16 version 1703 is already newer than 1702 2024-04-07T16:27:34.463 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 16 v 1703 (ObjNum 1066 snap 311 seq_num 1066) dirty exists 2024-04-07T16:27:34.463 INFO:tasks.rados.rados.0.smithi100.stdout:3447: left oid 16 (ObjNum 1066 snap 311 seq_num 1066) 2024-04-07T16:27:34.463 INFO:tasks.rados.rados.0.smithi100.stdout:3445: done (3 left) 2024-04-07T16:27:34.463 INFO:tasks.rados.rados.0.smithi100.stdout:3446: done (2 left) 2024-04-07T16:27:34.463 INFO:tasks.rados.rados.0.smithi100.stdout:3447: done (1 left) 2024-04-07T16:27:34.464 INFO:tasks.rados.rados.0.smithi100.stdout:3448: done (0 left) 2024-04-07T16:27:34.464 INFO:tasks.rados.rados.0.smithi100.stdout:3449: rmattr oid 14 current snap is 311 2024-04-07T16:27:34.464 INFO:tasks.rados.rados.0.smithi100.stdout:3449: done (0 left) 2024-04-07T16:27:34.464 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 28 current snap is 311 2024-04-07T16:27:34.464 INFO:tasks.rados.rados.0.smithi100.stdout:3450: seq_num 1068 ranges {1302528=696320,1998848=401408} 2024-04-07T16:27:34.474 INFO:tasks.rados.rados.0.smithi100.stdout:3450: writing smithi10066867-28 from 1302528 to 1998848 tid 1 2024-04-07T16:27:34.476 INFO:tasks.rados.rados.0.smithi100.stdout:3450: writing smithi10066867-28 from 1998848 to 2400256 tid 2 2024-04-07T16:27:34.476 INFO:tasks.rados.rados.0.smithi100.stdout:3451: snap_create 2024-04-07T16:27:34.488 INFO:tasks.rados.rados.0.smithi100.stdout:3450: finishing write tid 1 to smithi10066867-28 2024-04-07T16:27:34.490 INFO:tasks.rados.rados.0.smithi100.stdout:3450: finishing write tid 2 to smithi10066867-28 2024-04-07T16:27:34.491 INFO:tasks.rados.rados.0.smithi100.stdout:3450: finishing write tid 3 to smithi10066867-28 2024-04-07T16:27:34.494 INFO:tasks.rados.rados.0.smithi100.stdout:3450: finishing write tid 4 to smithi10066867-28 2024-04-07T16:27:34.494 INFO:tasks.rados.rados.0.smithi100.stdout:3450: oid 28 updating version 0 to 1696 2024-04-07T16:27:34.494 INFO:tasks.rados.rados.0.smithi100.stdout:3450: oid 28 version 1696 is already newer than 1694 2024-04-07T16:27:34.494 INFO:tasks.rados.rados.0.smithi100.stdout:3450: oid 28 version 1696 is already newer than 1695 2024-04-07T16:27:34.494 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 28 v 1696 (ObjNum 1068 snap 311 seq_num 1068) dirty exists 2024-04-07T16:27:34.494 INFO:tasks.rados.rados.0.smithi100.stdout:3450: left oid 28 (ObjNum 1068 snap 311 seq_num 1068) 2024-04-07T16:27:35.446 INFO:tasks.rados.rados.0.smithi100.stdout:3450: done (1 left) 2024-04-07T16:27:35.446 INFO:tasks.rados.rados.0.smithi100.stdout:3451: done (0 left) 2024-04-07T16:27:35.446 INFO:tasks.rados.rados.0.smithi100.stdout:3452: delete oid 25 current snap is 312 2024-04-07T16:27:35.452 INFO:tasks.rados.rados.0.smithi100.stdout:3452: done (0 left) 2024-04-07T16:27:35.452 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 45 current snap is 312 2024-04-07T16:27:35.452 INFO:tasks.rados.rados.0.smithi100.stdout:3453: seq_num 1070 ranges {7380992=729088,8110080=401408} 2024-04-07T16:27:35.525 INFO:tasks.rados.rados.0.smithi100.stdout:3453: writing smithi10066867-45 from 7380992 to 8110080 tid 1 2024-04-07T16:27:35.528 INFO:tasks.rados.rados.0.smithi100.stdout:3453: writing smithi10066867-45 from 8110080 to 8511488 tid 2 2024-04-07T16:27:35.528 INFO:tasks.rados.rados.0.smithi100.stdout:3454: read oid 17 snap -1 2024-04-07T16:27:35.528 INFO:tasks.rados.rados.0.smithi100.stdout:3454: expect deleted 2024-04-07T16:27:35.528 INFO:tasks.rados.rados.0.smithi100.stdout:3455: delete oid 32 current snap is 312 2024-04-07T16:27:35.539 INFO:tasks.rados.rados.0.smithi100.stdout:3453: finishing write tid 1 to smithi10066867-45 2024-04-07T16:27:35.540 INFO:tasks.rados.rados.0.smithi100.stdout:3453: finishing write tid 2 to smithi10066867-45 2024-04-07T16:27:35.541 INFO:tasks.rados.rados.0.smithi100.stdout:3453: finishing write tid 3 to smithi10066867-45 2024-04-07T16:27:35.541 INFO:tasks.rados.rados.0.smithi100.stdout:3454: done (2 left) 2024-04-07T16:27:35.541 INFO:tasks.rados.rados.0.smithi100.stdout:3455: done (1 left) 2024-04-07T16:27:35.541 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 12 current snap is 312 2024-04-07T16:27:35.541 INFO:tasks.rados.rados.0.smithi100.stdout:3456: seq_num 1072 ranges {0=761856,761856=401408} 2024-04-07T16:27:35.546 INFO:tasks.rados.rados.0.smithi100.stdout:3456: writing smithi10066867-12 from 0 to 761856 tid 1 2024-04-07T16:27:35.549 INFO:tasks.rados.rados.0.smithi100.stdout:3456: writing smithi10066867-12 from 761856 to 1163264 tid 2 2024-04-07T16:27:35.549 INFO:tasks.rados.rados.0.smithi100.stdout:3457: snap_create 2024-04-07T16:27:35.550 INFO:tasks.rados.rados.0.smithi100.stdout:3453: finishing write tid 4 to smithi10066867-45 2024-04-07T16:27:35.550 INFO:tasks.rados.rados.0.smithi100.stdout:3453: oid 45 updating version 0 to 1556 2024-04-07T16:27:35.550 INFO:tasks.rados.rados.0.smithi100.stdout:3453: oid 45 version 1556 is already newer than 1555 2024-04-07T16:27:35.550 INFO:tasks.rados.rados.0.smithi100.stdout:3453: oid 45 updating version 1556 to 1557 2024-04-07T16:27:35.550 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 45 v 1557 (ObjNum 1070 snap 312 seq_num 1070) dirty exists 2024-04-07T16:27:35.550 INFO:tasks.rados.rados.0.smithi100.stdout:3453: left oid 45 (ObjNum 1070 snap 312 seq_num 1070) 2024-04-07T16:27:35.560 INFO:tasks.rados.rados.0.smithi100.stdout:3456: finishing write tid 1 to smithi10066867-12 2024-04-07T16:27:35.560 INFO:tasks.rados.rados.0.smithi100.stdout:3456: finishing write tid 2 to smithi10066867-12 2024-04-07T16:27:35.561 INFO:tasks.rados.rados.0.smithi100.stdout:3456: finishing write tid 3 to smithi10066867-12 2024-04-07T16:27:35.564 INFO:tasks.rados.rados.0.smithi100.stdout:3456: finishing write tid 4 to smithi10066867-12 2024-04-07T16:27:35.564 INFO:tasks.rados.rados.0.smithi100.stdout:3456: oid 12 updating version 0 to 1705 2024-04-07T16:27:35.564 INFO:tasks.rados.rados.0.smithi100.stdout:3456: oid 12 updating version 1705 to 1707 2024-04-07T16:27:35.565 INFO:tasks.rados.rados.0.smithi100.stdout:3456: oid 12 version 1707 is already newer than 1706 2024-04-07T16:27:35.565 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 12 v 1707 (ObjNum 1072 snap 312 seq_num 1072) dirty exists 2024-04-07T16:27:35.565 INFO:tasks.rados.rados.0.smithi100.stdout:3456: left oid 12 (ObjNum 1072 snap 312 seq_num 1072) 2024-04-07T16:27:35.687 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:35 smithi100 bash[74942]: cluster 2024-04-07T16:27:34.453087+0000 mon.a (mon.0) 1169 : cluster [DBG] osdmap e696: 8 total, 8 up, 8 in 2024-04-07T16:27:35.687 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:35 smithi100 bash[74942]: cluster 2024-04-07T16:27:34.795101+0000 mgr.y (mgr.25175) 352 : cluster [DBG] pgmap v638: 121 pgs: 121 active+clean; 161 MiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 7.9 MiB/s rd, 1.9 MiB/s wr, 30 op/s 2024-04-07T16:27:35.687 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:35 smithi100 bash[74942]: cephadm 2024-04-07T16:27:35.174531+0000 mgr.y (mgr.25175) 353 : cephadm [INF] Upgrade: Updating alertmanager.a 2024-04-07T16:27:35.687 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:35 smithi100 bash[74942]: cephadm 2024-04-07T16:27:35.195992+0000 mgr.y (mgr.25175) 354 : cephadm [INF] Deploying daemon alertmanager.a on smithi100 2024-04-07T16:27:35.688 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:35 smithi100 bash[76202]: cluster 2024-04-07T16:27:34.453087+0000 mon.a (mon.0) 1169 : cluster [DBG] osdmap e696: 8 total, 8 up, 8 in 2024-04-07T16:27:35.688 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:35 smithi100 bash[76202]: cluster 2024-04-07T16:27:34.795101+0000 mgr.y (mgr.25175) 352 : cluster [DBG] pgmap v638: 121 pgs: 121 active+clean; 161 MiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 7.9 MiB/s rd, 1.9 MiB/s wr, 30 op/s 2024-04-07T16:27:35.688 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:35 smithi100 bash[76202]: cephadm 2024-04-07T16:27:35.174531+0000 mgr.y (mgr.25175) 353 : cephadm [INF] Upgrade: Updating alertmanager.a 2024-04-07T16:27:35.688 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:35 smithi100 bash[76202]: cephadm 2024-04-07T16:27:35.195992+0000 mgr.y (mgr.25175) 354 : cephadm [INF] Deploying daemon alertmanager.a on smithi100 2024-04-07T16:27:35.855 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:35 smithi144 bash[66654]: cluster 2024-04-07T16:27:34.453087+0000 mon.a (mon.0) 1169 : cluster [DBG] osdmap e696: 8 total, 8 up, 8 in 2024-04-07T16:27:35.856 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:35 smithi144 bash[66654]: cluster 2024-04-07T16:27:34.795101+0000 mgr.y (mgr.25175) 352 : cluster [DBG] pgmap v638: 121 pgs: 121 active+clean; 161 MiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 7.9 MiB/s rd, 1.9 MiB/s wr, 30 op/s 2024-04-07T16:27:35.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:35 smithi144 bash[66654]: cephadm 2024-04-07T16:27:35.174531+0000 mgr.y (mgr.25175) 353 : cephadm [INF] Upgrade: Updating alertmanager.a 2024-04-07T16:27:35.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:35 smithi144 bash[66654]: cephadm 2024-04-07T16:27:35.195992+0000 mgr.y (mgr.25175) 354 : cephadm [INF] Deploying daemon alertmanager.a on smithi100 2024-04-07T16:27:36.461 INFO:tasks.rados.rados.0.smithi100.stdout:3453: done (2 left) 2024-04-07T16:27:36.461 INFO:tasks.rados.rados.0.smithi100.stdout:3456: done (1 left) 2024-04-07T16:27:36.461 INFO:tasks.rados.rados.0.smithi100.stdout:3457: done (0 left) 2024-04-07T16:27:36.461 INFO:tasks.rados.rados.0.smithi100.stdout:3458: delete oid 26 current snap is 313 2024-04-07T16:27:36.472 INFO:tasks.rados.rados.0.smithi100.stdout:3458: done (0 left) 2024-04-07T16:27:36.472 INFO:tasks.rados.rados.0.smithi100.stdout:3459: copy_from oid 40 from oid 42 current snap is 313 2024-04-07T16:27:36.472 INFO:tasks.rados.rados.0.smithi100.stdout:3460: rmattr oid 28 current snap is 313 2024-04-07T16:27:36.472 INFO:tasks.rados.rados.0.smithi100.stdout:3461: delete oid 50 current snap is 313 2024-04-07T16:27:36.473 INFO:tasks.rados.rados.0.smithi100.stdout:3459: finishing copy_from to smithi10066867-40 2024-04-07T16:27:36.473 INFO:tasks.rados.rados.0.smithi100.stdout:3459: got expected ENOENT (src dne) 2024-04-07T16:27:36.474 INFO:tasks.rados.rados.0.smithi100.stdout:3459: finishing copy_from racing read to smithi10066867-40 2024-04-07T16:27:36.479 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 28 v 0 (ObjNum 1068 snap 311 seq_num 1068) dirty exists 2024-04-07T16:27:36.480 INFO:tasks.rados.rados.0.smithi100.stdout:3459: done (2 left) 2024-04-07T16:27:36.480 INFO:tasks.rados.rados.0.smithi100.stdout:3460: done (1 left) 2024-04-07T16:27:36.480 INFO:tasks.rados.rados.0.smithi100.stdout:3461: done (0 left) 2024-04-07T16:27:36.480 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 27 current snap is 313 2024-04-07T16:27:36.480 INFO:tasks.rados.rados.0.smithi100.stdout:3462: seq_num 1075 ranges {5128192=409600,5537792=671744,6209536=131072} 2024-04-07T16:27:36.522 INFO:tasks.rados.rados.0.smithi100.stdout:3462: writing smithi10066867-27 from 5128192 to 5537792 tid 1 2024-04-07T16:27:36.527 INFO:tasks.rados.rados.0.smithi100.stdout:3462: writing smithi10066867-27 from 5537792 to 6209536 tid 2 2024-04-07T16:27:36.528 INFO:tasks.rados.rados.0.smithi100.stdout:3462: writing smithi10066867-27 from 6209536 to 6340608 tid 3 2024-04-07T16:27:36.529 INFO:tasks.rados.rados.0.smithi100.stdout:3463: read oid 46 snap -1 2024-04-07T16:27:36.529 INFO:tasks.rados.rados.0.smithi100.stdout:3463: expect (ObjNum 1057 snap 309 seq_num 1057) 2024-04-07T16:27:36.529 INFO:tasks.rados.rados.0.smithi100.stdout:3464: setattr oid 18 current snap is 313 2024-04-07T16:27:36.530 INFO:tasks.rados.rados.0.smithi100.stdout:3465: read oid 49 snap -1 2024-04-07T16:27:36.530 INFO:tasks.rados.rados.0.smithi100.stdout:3465: expect (ObjNum 1009 snap 300 seq_num 1009) 2024-04-07T16:27:36.531 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 9 current snap is 313 2024-04-07T16:27:36.531 INFO:tasks.rados.rados.0.smithi100.stdout:3466: seq_num 1076 ranges {475136=425984,901120=548864,1449984=253952} 2024-04-07T16:27:36.536 INFO:tasks.rados.rados.0.smithi100.stdout:3466: writing smithi10066867-9 from 475136 to 901120 tid 1 2024-04-07T16:27:36.540 INFO:tasks.rados.rados.0.smithi100.stdout:3466: writing smithi10066867-9 from 901120 to 1449984 tid 2 2024-04-07T16:27:36.542 INFO:tasks.rados.rados.0.smithi100.stdout:3466: writing smithi10066867-9 from 1449984 to 1703936 tid 3 2024-04-07T16:27:36.542 INFO:tasks.rados.rados.0.smithi100.stdout:3467: copy_from oid 8 from oid 24 current snap is 313 2024-04-07T16:27:36.542 INFO:tasks.rados.rados.0.smithi100.stdout:3462: finishing write tid 1 to smithi10066867-27 2024-04-07T16:27:36.542 INFO:tasks.rados.rados.0.smithi100.stdout:3462: finishing write tid 2 to smithi10066867-27 2024-04-07T16:27:36.543 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 18 v 1833 (ObjNum 1062 snap 310 seq_num 1062) dirty exists 2024-04-07T16:27:36.543 INFO:tasks.rados.rados.0.smithi100.stdout:3462: finishing write tid 3 to smithi10066867-27 2024-04-07T16:27:36.543 INFO:tasks.rados.rados.0.smithi100.stdout:3462: finishing write tid 4 to smithi10066867-27 2024-04-07T16:27:36.543 INFO:tasks.rados.rados.0.smithi100.stdout:3464: done (5 left) 2024-04-07T16:27:36.543 INFO:tasks.rados.rados.0.smithi100.stdout:3468: read oid 47 snap -1 2024-04-07T16:27:36.543 INFO:tasks.rados.rados.0.smithi100.stdout:3468: expect (ObjNum 1064 snap 310 seq_num 1064) 2024-04-07T16:27:36.543 INFO:tasks.rados.rados.0.smithi100.stdout:3469: delete oid 26 current snap is 313 2024-04-07T16:27:36.543 INFO:tasks.rados.rados.0.smithi100.stdout:3462: finishing write tid 5 to smithi10066867-27 2024-04-07T16:27:36.543 INFO:tasks.rados.rados.0.smithi100.stdout:3462: oid 27 updating version 0 to 1195 2024-04-07T16:27:36.543 INFO:tasks.rados.rados.0.smithi100.stdout:3462: oid 27 version 1195 is already newer than 1192 2024-04-07T16:27:36.543 INFO:tasks.rados.rados.0.smithi100.stdout:3462: oid 27 version 1195 is already newer than 1194 2024-04-07T16:27:36.543 INFO:tasks.rados.rados.0.smithi100.stdout:3462: oid 27 version 1195 is already newer than 1193 2024-04-07T16:27:36.543 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 27 v 1195 (ObjNum 1075 snap 313 seq_num 1075) dirty exists 2024-04-07T16:27:36.543 INFO:tasks.rados.rados.0.smithi100.stdout:3462: left oid 27 (ObjNum 1075 snap 313 seq_num 1075) 2024-04-07T16:27:36.544 INFO:tasks.rados.rados.0.smithi100.stdout:3465: expect (ObjNum 1009 snap 300 seq_num 1009) 2024-04-07T16:27:36.611 INFO:tasks.rados.rados.0.smithi100.stdout:3466: finishing write tid 1 to smithi10066867-9 2024-04-07T16:27:36.611 INFO:tasks.rados.rados.0.smithi100.stdout:3468: expect (ObjNum 1064 snap 310 seq_num 1064) 2024-04-07T16:27:36.637 INFO:tasks.rados.rados.0.smithi100.stdout:3466: finishing write tid 2 to smithi10066867-9 2024-04-07T16:27:36.637 INFO:tasks.rados.rados.0.smithi100.stdout:3466: finishing write tid 3 to smithi10066867-9 2024-04-07T16:27:36.637 INFO:tasks.rados.rados.0.smithi100.stdout:3466: finishing write tid 4 to smithi10066867-9 2024-04-07T16:27:36.637 INFO:tasks.rados.rados.0.smithi100.stdout:3466: finishing write tid 5 to smithi10066867-9 2024-04-07T16:27:36.637 INFO:tasks.rados.rados.0.smithi100.stdout:3466: oid 9 updating version 0 to 1139 2024-04-07T16:27:36.638 INFO:tasks.rados.rados.0.smithi100.stdout:3466: oid 9 updating version 1139 to 1141 2024-04-07T16:27:36.638 INFO:tasks.rados.rados.0.smithi100.stdout:3466: oid 9 version 1141 is already newer than 1140 2024-04-07T16:27:36.638 INFO:tasks.rados.rados.0.smithi100.stdout:3466: oid 9 version 1141 is already newer than 1138 2024-04-07T16:27:36.638 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 9 v 1141 (ObjNum 1076 snap 313 seq_num 1076) dirty exists 2024-04-07T16:27:36.638 INFO:tasks.rados.rados.0.smithi100.stdout:3466: left oid 9 (ObjNum 1076 snap 313 seq_num 1076) 2024-04-07T16:27:36.638 INFO:tasks.rados.rados.0.smithi100.stdout:3463: expect (ObjNum 1057 snap 309 seq_num 1057) 2024-04-07T16:27:36.825 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:36 smithi100 bash[74942]: cluster 2024-04-07T16:27:35.460972+0000 mon.a (mon.0) 1170 : cluster [DBG] osdmap e697: 8 total, 8 up, 8 in 2024-04-07T16:27:36.825 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:36 smithi100 bash[76202]: cluster 2024-04-07T16:27:35.460972+0000 mon.a (mon.0) 1170 : cluster [DBG] osdmap e697: 8 total, 8 up, 8 in 2024-04-07T16:27:36.855 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:36 smithi144 bash[66654]: cluster 2024-04-07T16:27:35.460972+0000 mon.a (mon.0) 1170 : cluster [DBG] osdmap e697: 8 total, 8 up, 8 in 2024-04-07T16:27:37.064 INFO:tasks.rados.rados.0.smithi100.stdout:3467: finishing copy_from racing read to smithi10066867-8 2024-04-07T16:27:37.064 INFO:tasks.rados.rados.0.smithi100.stdout:3467: finishing copy_from to smithi10066867-8 2024-04-07T16:27:37.064 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 8 v 1269 (ObjNum 1030 snap 306 seq_num 1030) dirty exists 2024-04-07T16:27:37.065 INFO:tasks.rados.rados.0.smithi100.stdout:3462: done (6 left) 2024-04-07T16:27:37.065 INFO:tasks.rados.rados.0.smithi100.stdout:3463: done (5 left) 2024-04-07T16:27:37.065 INFO:tasks.rados.rados.0.smithi100.stdout:3465: done (4 left) 2024-04-07T16:27:37.065 INFO:tasks.rados.rados.0.smithi100.stdout:3466: done (3 left) 2024-04-07T16:27:37.065 INFO:tasks.rados.rados.0.smithi100.stdout:3467: done (2 left) 2024-04-07T16:27:37.065 INFO:tasks.rados.rados.0.smithi100.stdout:3468: done (1 left) 2024-04-07T16:27:37.065 INFO:tasks.rados.rados.0.smithi100.stdout:3469: done (0 left) 2024-04-07T16:27:37.065 INFO:tasks.rados.rados.0.smithi100.stdout:3470: snap_create 2024-04-07T16:27:37.632 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:37 smithi100 bash[74942]: cluster 2024-04-07T16:27:36.476354+0000 mon.a (mon.0) 1171 : cluster [DBG] osdmap e698: 8 total, 8 up, 8 in 2024-04-07T16:27:37.632 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:37 smithi100 bash[74942]: cluster 2024-04-07T16:27:36.795852+0000 mgr.y (mgr.25175) 355 : cluster [DBG] pgmap v641: 121 pgs: 121 active+clean; 161 MiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 11 MiB/s rd, 1.6 MiB/s wr, 33 op/s 2024-04-07T16:27:37.632 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:37 smithi100 bash[74942]: cluster 2024-04-07T16:27:36.937892+0000 mon.a (mon.0) 1172 : cluster [DBG] osdmap e699: 8 total, 8 up, 8 in 2024-04-07T16:27:37.632 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:37 smithi100 bash[76202]: cluster 2024-04-07T16:27:36.476354+0000 mon.a (mon.0) 1171 : cluster [DBG] osdmap e698: 8 total, 8 up, 8 in 2024-04-07T16:27:37.632 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:37 smithi100 bash[76202]: cluster 2024-04-07T16:27:36.795852+0000 mgr.y (mgr.25175) 355 : cluster [DBG] pgmap v641: 121 pgs: 121 active+clean; 161 MiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 11 MiB/s rd, 1.6 MiB/s wr, 33 op/s 2024-04-07T16:27:37.633 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:37 smithi100 bash[76202]: cluster 2024-04-07T16:27:36.937892+0000 mon.a (mon.0) 1172 : cluster [DBG] osdmap e699: 8 total, 8 up, 8 in 2024-04-07T16:27:37.855 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:37 smithi144 bash[66654]: cluster 2024-04-07T16:27:36.476354+0000 mon.a (mon.0) 1171 : cluster [DBG] osdmap e698: 8 total, 8 up, 8 in 2024-04-07T16:27:37.856 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:37 smithi144 bash[66654]: cluster 2024-04-07T16:27:36.795852+0000 mgr.y (mgr.25175) 355 : cluster [DBG] pgmap v641: 121 pgs: 121 active+clean; 161 MiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 11 MiB/s rd, 1.6 MiB/s wr, 33 op/s 2024-04-07T16:27:37.856 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:37 smithi144 bash[66654]: cluster 2024-04-07T16:27:36.937892+0000 mon.a (mon.0) 1172 : cluster [DBG] osdmap e699: 8 total, 8 up, 8 in 2024-04-07T16:27:37.926 INFO:tasks.rados.rados.0.smithi100.stdout:3470: done (0 left) 2024-04-07T16:27:37.926 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 42 current snap is 314 2024-04-07T16:27:37.926 INFO:tasks.rados.rados.0.smithi100.stdout:3471: seq_num 1078 ranges {0=458752,458752=720896,1179648=81920} 2024-04-07T16:27:37.928 INFO:tasks.rados.rados.0.smithi100.stdout:3471: writing smithi10066867-42 from 0 to 458752 tid 1 2024-04-07T16:27:37.932 INFO:tasks.rados.rados.0.smithi100.stdout:3471: writing smithi10066867-42 from 458752 to 1179648 tid 2 2024-04-07T16:27:37.933 INFO:tasks.rados.rados.0.smithi100.stdout:3471: writing smithi10066867-42 from 1179648 to 1261568 tid 3 2024-04-07T16:27:37.933 INFO:tasks.rados.rados.0.smithi100.stdout:3472: read oid 13 snap -1 2024-04-07T16:27:37.934 INFO:tasks.rados.rados.0.smithi100.stdout:3472: expect deleted 2024-04-07T16:27:37.934 INFO:tasks.rados.rados.0.smithi100.stdout:3473: copy_from oid 10 from oid 18 current snap is 314 2024-04-07T16:27:37.934 INFO:tasks.rados.rados.0.smithi100.stdout:3474: read oid 31 snap -1 2024-04-07T16:27:37.934 INFO:tasks.rados.rados.0.smithi100.stdout:3474: expect (ObjNum 993 snap 300 seq_num 993) 2024-04-07T16:27:37.934 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 19 current snap is 314 2024-04-07T16:27:37.934 INFO:tasks.rados.rados.0.smithi100.stdout:3475: seq_num 1079 ranges {1523712=475136,1998848=606208,2605056=196608} 2024-04-07T16:27:37.945 INFO:tasks.rados.rados.0.smithi100.stdout:3475: writing smithi10066867-19 from 1523712 to 1998848 tid 1 2024-04-07T16:27:37.949 INFO:tasks.rados.rados.0.smithi100.stdout:3475: writing smithi10066867-19 from 1998848 to 2605056 tid 2 2024-04-07T16:27:37.951 INFO:tasks.rados.rados.0.smithi100.stdout:3475: writing smithi10066867-19 from 2605056 to 2801664 tid 3 2024-04-07T16:27:37.951 INFO:tasks.rados.rados.0.smithi100.stdout:3476: setattr oid 46 current snap is 314 2024-04-07T16:27:37.951 INFO:tasks.rados.rados.0.smithi100.stdout:3471: finishing write tid 1 to smithi10066867-42 2024-04-07T16:27:37.951 INFO:tasks.rados.rados.0.smithi100.stdout:3471: finishing write tid 2 to smithi10066867-42 2024-04-07T16:27:37.951 INFO:tasks.rados.rados.0.smithi100.stdout:3471: finishing write tid 3 to smithi10066867-42 2024-04-07T16:27:37.952 INFO:tasks.rados.rados.0.smithi100.stdout:3471: finishing write tid 4 to smithi10066867-42 2024-04-07T16:27:37.952 INFO:tasks.rados.rados.0.smithi100.stdout:3471: finishing write tid 5 to smithi10066867-42 2024-04-07T16:27:37.952 INFO:tasks.rados.rados.0.smithi100.stdout:3471: oid 42 updating version 0 to 1559 2024-04-07T16:27:37.952 INFO:tasks.rados.rados.0.smithi100.stdout:3471: oid 42 updating version 1559 to 1560 2024-04-07T16:27:37.952 INFO:tasks.rados.rados.0.smithi100.stdout:3471: oid 42 version 1560 is already newer than 1557 2024-04-07T16:27:37.952 INFO:tasks.rados.rados.0.smithi100.stdout:3471: oid 42 version 1560 is already newer than 1558 2024-04-07T16:27:37.952 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 42 v 1560 (ObjNum 1078 snap 314 seq_num 1078) dirty exists 2024-04-07T16:27:37.952 INFO:tasks.rados.rados.0.smithi100.stdout:3471: left oid 42 (ObjNum 1078 snap 314 seq_num 1078) 2024-04-07T16:27:37.952 INFO:tasks.rados.rados.0.smithi100.stdout:3473: finishing copy_from racing read to smithi10066867-10 2024-04-07T16:27:37.954 INFO:tasks.rados.rados.0.smithi100.stdout:3471: done (5 left) 2024-04-07T16:27:37.954 INFO:tasks.rados.rados.0.smithi100.stdout:3472: done (4 left) 2024-04-07T16:27:37.954 INFO:tasks.rados.rados.0.smithi100.stdout:3477: rmattr oid 45 current snap is 314 2024-04-07T16:27:37.954 INFO:tasks.rados.rados.0.smithi100.stdout:3478: copy_from oid 16 from oid 3 current snap is 314 2024-04-07T16:27:37.954 INFO:tasks.rados.rados.0.smithi100.stdout:3479: read oid 7 snap -1 2024-04-07T16:27:37.954 INFO:tasks.rados.rados.0.smithi100.stdout:3479: expect (ObjNum 1051 snap 307 seq_num 1051) 2024-04-07T16:27:37.954 INFO:tasks.rados.rados.0.smithi100.stdout:3480: read oid 32 snap 306 2024-04-07T16:27:37.954 INFO:tasks.rados.rados.0.smithi100.stdout:3480: expect (ObjNum 1020 snap 304 seq_num 1020) 2024-04-07T16:27:37.954 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 8 current snap is 314 2024-04-07T16:27:37.955 INFO:tasks.rados.rados.0.smithi100.stdout:3481: seq_num 1080 ranges {458752=491520,950272=540672,1490944=262144} 2024-04-07T16:27:37.960 INFO:tasks.rados.rados.0.smithi100.stdout:3481: writing smithi10066867-8 from 458752 to 950272 tid 1 2024-04-07T16:27:37.963 INFO:tasks.rados.rados.0.smithi100.stdout:3481: writing smithi10066867-8 from 950272 to 1490944 tid 2 2024-04-07T16:27:37.966 INFO:tasks.rados.rados.0.smithi100.stdout:3481: writing smithi10066867-8 from 1490944 to 1753088 tid 3 2024-04-07T16:27:37.966 INFO:tasks.rados.rados.0.smithi100.stdout:3482: read oid 4 snap -1 2024-04-07T16:27:37.966 INFO:tasks.rados.rados.0.smithi100.stdout:3482: expect (ObjNum 1041 snap 307 seq_num 1041) 2024-04-07T16:27:37.966 INFO:tasks.rados.rados.0.smithi100.stdout:3483: read oid 42 snap -1 2024-04-07T16:27:37.966 INFO:tasks.rados.rados.0.smithi100.stdout:3483: expect (ObjNum 1078 snap 314 seq_num 1078) 2024-04-07T16:27:37.966 INFO:tasks.rados.rados.0.smithi100.stdout:3484: copy_from oid 22 from oid 24 current snap is 314 2024-04-07T16:27:37.966 INFO:tasks.rados.rados.0.smithi100.stdout:3473: finishing copy_from to smithi10066867-10 2024-04-07T16:27:37.966 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 10 v 1833 (ObjNum 1062 snap 310 seq_num 1062) dirty exists 2024-04-07T16:27:37.966 INFO:tasks.rados.rados.0.smithi100.stdout:3474: expect (ObjNum 993 snap 300 seq_num 993) 2024-04-07T16:27:38.077 INFO:tasks.rados.rados.0.smithi100.stdout:3475: finishing write tid 1 to smithi10066867-19 2024-04-07T16:27:38.077 INFO:tasks.rados.rados.0.smithi100.stdout:3475: finishing write tid 2 to smithi10066867-19 2024-04-07T16:27:38.077 INFO:tasks.rados.rados.0.smithi100.stdout:3478: finishing copy_from to smithi10066867-16 2024-04-07T16:27:38.077 INFO:tasks.rados.rados.0.smithi100.stdout:3478: got expected ENOENT (src dne) 2024-04-07T16:27:38.077 INFO:tasks.rados.rados.0.smithi100.stdout:3478: finishing copy_from racing read to smithi10066867-16 2024-04-07T16:27:38.078 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 45 v 1559 (ObjNum 1070 snap 312 seq_num 1070) dirty exists 2024-04-07T16:27:38.078 INFO:tasks.rados.rados.0.smithi100.stdout:3475: finishing write tid 3 to smithi10066867-19 2024-04-07T16:27:38.078 INFO:tasks.rados.rados.0.smithi100.stdout:3475: finishing write tid 4 to smithi10066867-19 2024-04-07T16:27:38.078 INFO:tasks.rados.rados.0.smithi100.stdout:3475: finishing write tid 5 to smithi10066867-19 2024-04-07T16:27:38.078 INFO:tasks.rados.rados.0.smithi100.stdout:3475: oid 19 updating version 0 to 1272 2024-04-07T16:27:38.079 INFO:tasks.rados.rados.0.smithi100.stdout:3475: oid 19 version 1272 is already newer than 1271 2024-04-07T16:27:38.079 INFO:tasks.rados.rados.0.smithi100.stdout:3475: oid 19 updating version 1272 to 1273 2024-04-07T16:27:38.079 INFO:tasks.rados.rados.0.smithi100.stdout:3475: oid 19 version 1273 is already newer than 1270 2024-04-07T16:27:38.079 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 19 v 1273 (ObjNum 1079 snap 314 seq_num 1079) dirty exists 2024-04-07T16:27:38.079 INFO:tasks.rados.rados.0.smithi100.stdout:3475: left oid 19 (ObjNum 1079 snap 314 seq_num 1079) 2024-04-07T16:27:38.079 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 46 v 1575 (ObjNum 1057 snap 309 seq_num 1057) dirty exists 2024-04-07T16:27:38.080 INFO:tasks.rados.rados.0.smithi100.stdout:3473: done (11 left) 2024-04-07T16:27:38.080 INFO:tasks.rados.rados.0.smithi100.stdout:3474: done (10 left) 2024-04-07T16:27:38.080 INFO:tasks.rados.rados.0.smithi100.stdout:3475: done (9 left) 2024-04-07T16:27:38.080 INFO:tasks.rados.rados.0.smithi100.stdout:3476: done (8 left) 2024-04-07T16:27:38.080 INFO:tasks.rados.rados.0.smithi100.stdout:3477: done (7 left) 2024-04-07T16:27:38.080 INFO:tasks.rados.rados.0.smithi100.stdout:3478: done (6 left) 2024-04-07T16:27:38.081 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 30 current snap is 314 2024-04-07T16:27:38.081 INFO:tasks.rados.rados.0.smithi100.stdout:3480: expect (ObjNum 1020 snap 304 seq_num 1020) 2024-04-07T16:27:38.083 INFO:tasks.rados.rados.0.smithi100.stdout:3483: expect (ObjNum 1078 snap 314 seq_num 1078) 2024-04-07T16:27:38.115 INFO:tasks.rados.rados.0.smithi100.stdout:3484: finishing copy_from racing read to smithi10066867-22 2024-04-07T16:27:38.115 INFO:tasks.rados.rados.0.smithi100.stdout:3482: expect (ObjNum 1041 snap 307 seq_num 1041) 2024-04-07T16:27:38.134 INFO:tasks.rados.rados.0.smithi100.stdout:3484: finishing copy_from to smithi10066867-22 2024-04-07T16:27:38.134 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 22 v 1269 (ObjNum 1030 snap 306 seq_num 1030) dirty exists 2024-04-07T16:27:38.135 INFO:tasks.rados.rados.0.smithi100.stdout:3481: finishing write tid 1 to smithi10066867-8 2024-04-07T16:27:38.135 INFO:tasks.rados.rados.0.smithi100.stdout:3479: expect (ObjNum 1051 snap 307 seq_num 1051) 2024-04-07T16:27:38.263 INFO:tasks.rados.rados.0.smithi100.stdout:3481: finishing write tid 2 to smithi10066867-8 2024-04-07T16:27:38.263 INFO:tasks.rados.rados.0.smithi100.stdout:3481: finishing write tid 3 to smithi10066867-8 2024-04-07T16:27:38.263 INFO:tasks.rados.rados.0.smithi100.stdout:3481: finishing write tid 4 to smithi10066867-8 2024-04-07T16:27:38.263 INFO:tasks.rados.rados.0.smithi100.stdout:3481: finishing write tid 5 to smithi10066867-8 2024-04-07T16:27:38.263 INFO:tasks.rados.rados.0.smithi100.stdout:3481: oid 8 updating version 0 to 1578 2024-04-07T16:27:38.263 INFO:tasks.rados.rados.0.smithi100.stdout:3481: oid 8 updating version 1578 to 1579 2024-04-07T16:27:38.263 INFO:tasks.rados.rados.0.smithi100.stdout:3481: oid 8 version 1579 is already newer than 1577 2024-04-07T16:27:38.263 INFO:tasks.rados.rados.0.smithi100.stdout:3481: oid 8 version 1579 is already newer than 1576 2024-04-07T16:27:38.263 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 8 v 1579 (ObjNum 1080 snap 314 seq_num 1080) dirty exists 2024-04-07T16:27:38.263 INFO:tasks.rados.rados.0.smithi100.stdout:3481: left oid 8 (ObjNum 1080 snap 314 seq_num 1080) 2024-04-07T16:27:38.263 INFO:tasks.rados.rados.0.smithi100.stdout:3485: seq_num 1081 ranges {2555904=507904,3063808=425984,3489792=376832} 2024-04-07T16:27:38.279 INFO:tasks.rados.rados.0.smithi100.stdout:3485: writing smithi10066867-30 from 2555904 to 3063808 tid 1 2024-04-07T16:27:38.282 INFO:tasks.rados.rados.0.smithi100.stdout:3485: writing smithi10066867-30 from 3063808 to 3489792 tid 2 2024-04-07T16:27:38.285 INFO:tasks.rados.rados.0.smithi100.stdout:3485: writing smithi10066867-30 from 3489792 to 3866624 tid 3 2024-04-07T16:27:38.285 INFO:tasks.rados.rados.0.smithi100.stdout:3479: done (6 left) 2024-04-07T16:27:38.285 INFO:tasks.rados.rados.0.smithi100.stdout:3480: done (5 left) 2024-04-07T16:27:38.285 INFO:tasks.rados.rados.0.smithi100.stdout:3481: done (4 left) 2024-04-07T16:27:38.285 INFO:tasks.rados.rados.0.smithi100.stdout:3482: done (3 left) 2024-04-07T16:27:38.286 INFO:tasks.rados.rados.0.smithi100.stdout:3483: done (2 left) 2024-04-07T16:27:38.286 INFO:tasks.rados.rados.0.smithi100.stdout:3484: done (1 left) 2024-04-07T16:27:38.286 INFO:tasks.rados.rados.0.smithi100.stdout:3486: read oid 48 snap -1 2024-04-07T16:27:38.286 INFO:tasks.rados.rados.0.smithi100.stdout:3486: expect deleted 2024-04-07T16:27:38.286 INFO:tasks.rados.rados.0.smithi100.stdout:3487: copy_from oid 3 from oid 1 current snap is 314 2024-04-07T16:27:38.286 INFO:tasks.rados.rados.0.smithi100.stdout:3488: read oid 47 snap -1 2024-04-07T16:27:38.286 INFO:tasks.rados.rados.0.smithi100.stdout:3488: expect (ObjNum 1064 snap 310 seq_num 1064) 2024-04-07T16:27:38.286 INFO:tasks.rados.rados.0.smithi100.stdout:3489: copy_from oid 46 from oid 9 current snap is 314 2024-04-07T16:27:38.286 INFO:tasks.rados.rados.0.smithi100.stdout:3490: read oid 12 snap 311 2024-04-07T16:27:38.286 INFO:tasks.rados.rados.0.smithi100.stdout:3490: expect (ObjNum -1592981168 snap 21978 seq_num 2702665520) 2024-04-07T16:27:38.286 INFO:tasks.rados.rados.0.smithi100.stdout:3491: snap_remove snap 312 2024-04-07T16:27:38.517 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:27:38 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:27:38] "GET /metrics HTTP/1.1" 200 36343 "" "Prometheus/2.43.0" 2024-04-07T16:27:38.934 INFO:tasks.rados.rados.0.smithi100.stdout:3490: expect (ObjNum -1592981168 snap 21978 seq_num 2702665520) 2024-04-07T16:27:38.938 INFO:tasks.rados.rados.0.smithi100.stdout:3487: finishing copy_from to smithi10066867-3 2024-04-07T16:27:38.938 INFO:tasks.rados.rados.0.smithi100.stdout:3487: got expected ENOENT (src dne) 2024-04-07T16:27:38.938 INFO:tasks.rados.rados.0.smithi100.stdout:3487: finishing copy_from racing read to smithi10066867-3 2024-04-07T16:27:38.938 INFO:tasks.rados.rados.0.smithi100.stdout:3485: finishing write tid 1 to smithi10066867-30 2024-04-07T16:27:38.938 INFO:tasks.rados.rados.0.smithi100.stdout:3485: finishing write tid 2 to smithi10066867-30 2024-04-07T16:27:38.938 INFO:tasks.rados.rados.0.smithi100.stdout:3485: finishing write tid 3 to smithi10066867-30 2024-04-07T16:27:38.939 INFO:tasks.rados.rados.0.smithi100.stdout:3485: finishing write tid 4 to smithi10066867-30 2024-04-07T16:27:38.939 INFO:tasks.rados.rados.0.smithi100.stdout:3485: finishing write tid 5 to smithi10066867-30 2024-04-07T16:27:38.939 INFO:tasks.rados.rados.0.smithi100.stdout:3485: oid 30 updating version 0 to 1563 2024-04-07T16:27:38.939 INFO:tasks.rados.rados.0.smithi100.stdout:3485: oid 30 version 1563 is already newer than 1562 2024-04-07T16:27:38.939 INFO:tasks.rados.rados.0.smithi100.stdout:3485: oid 30 version 1563 is already newer than 1561 2024-04-07T16:27:38.939 INFO:tasks.rados.rados.0.smithi100.stdout:3485: oid 30 updating version 1563 to 1564 2024-04-07T16:27:38.939 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 30 v 1564 (ObjNum 1081 snap 314 seq_num 1081) dirty exists 2024-04-07T16:27:38.940 INFO:tasks.rados.rados.0.smithi100.stdout:3485: left oid 30 (ObjNum 1081 snap 314 seq_num 1081) 2024-04-07T16:27:38.940 INFO:tasks.rados.rados.0.smithi100.stdout:3488: expect (ObjNum 1064 snap 310 seq_num 1064) 2024-04-07T16:27:38.968 INFO:tasks.rados.rados.0.smithi100.stdout:3489: finishing copy_from racing read to smithi10066867-46 2024-04-07T16:27:38.968 INFO:tasks.rados.rados.0.smithi100.stdout:3489: finishing copy_from to smithi10066867-46 2024-04-07T16:27:38.968 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 46 v 1141 (ObjNum 1076 snap 313 seq_num 1076) dirty exists 2024-04-07T16:27:38.968 INFO:tasks.rados.rados.0.smithi100.stdout:3485: done (6 left) 2024-04-07T16:27:38.968 INFO:tasks.rados.rados.0.smithi100.stdout:3486: done (5 left) 2024-04-07T16:27:38.968 INFO:tasks.rados.rados.0.smithi100.stdout:3487: done (4 left) 2024-04-07T16:27:38.969 INFO:tasks.rados.rados.0.smithi100.stdout:3488: done (3 left) 2024-04-07T16:27:38.969 INFO:tasks.rados.rados.0.smithi100.stdout:3489: done (2 left) 2024-04-07T16:27:38.969 INFO:tasks.rados.rados.0.smithi100.stdout:3490: done (1 left) 2024-04-07T16:27:38.969 INFO:tasks.rados.rados.0.smithi100.stdout:3491: done (0 left) 2024-04-07T16:27:38.969 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 25 current snap is 314 2024-04-07T16:27:38.969 INFO:tasks.rados.rados.0.smithi100.stdout:3492: seq_num 1082 ranges {0=524288,524288=704512,1228800=98304} 2024-04-07T16:27:38.971 INFO:tasks.rados.rados.0.smithi100.stdout:3492: writing smithi10066867-25 from 0 to 524288 tid 1 2024-04-07T16:27:38.975 INFO:tasks.rados.rados.0.smithi100.stdout:3492: writing smithi10066867-25 from 524288 to 1228800 tid 2 2024-04-07T16:27:38.976 INFO:tasks.rados.rados.0.smithi100.stdout:3492: writing smithi10066867-25 from 1228800 to 1327104 tid 3 2024-04-07T16:27:38.976 INFO:tasks.rados.rados.0.smithi100.stdout:3493: setattr oid 17 current snap is 314 2024-04-07T16:27:38.978 INFO:tasks.rados.rados.0.smithi100.stdout:3494: snap_remove snap 313 2024-04-07T16:27:39.187 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:27:39 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:27:39.187 INFO:journalctl@ceph.osd.2.smithi100.stdout:Apr 07 16:27:39 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:27:39.188 INFO:journalctl@ceph.osd.3.smithi100.stdout:Apr 07 16:27:39 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:27:39.188 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Apr 07 16:27:39 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:27:39.188 INFO:journalctl@ceph.osd.0.smithi100.stdout:Apr 07 16:27:39 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:27:39.189 INFO:journalctl@ceph.osd.1.smithi100.stdout:Apr 07 16:27:39 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:27:39.189 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 16:27:39 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:27:39.189 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:38 smithi100 bash[74942]: cluster 2024-04-07T16:27:37.938066+0000 mon.a (mon.0) 1173 : cluster [DBG] osdmap e700: 8 total, 8 up, 8 in 2024-04-07T16:27:39.189 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:39 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:27:39.190 INFO:journalctl@ceph.mds.a.smithi100.stdout:Apr 07 16:27:39 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:27:39.190 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:38 smithi100 bash[76202]: cluster 2024-04-07T16:27:37.938066+0000 mon.a (mon.0) 1173 : cluster [DBG] osdmap e700: 8 total, 8 up, 8 in 2024-04-07T16:27:39.190 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:39 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:27:39.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:38 smithi144 bash[66654]: cluster 2024-04-07T16:27:37.938066+0000 mon.a (mon.0) 1173 : cluster [DBG] osdmap e700: 8 total, 8 up, 8 in 2024-04-07T16:27:39.616 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Apr 07 16:27:39 smithi100 systemd[1]: Stopping Ceph alertmanager.a for 0702e3a8-f4f8-11ee-b649-cb9ed24678a4... 2024-04-07T16:27:39.616 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Apr 07 16:27:39 smithi100 bash[60014]: ts=2024-04-07T16:27:39.389Z caller=main.go:583 level=info msg="Received SIGTERM, exiting gracefully..." 2024-04-07T16:27:39.947 INFO:tasks.rados.rados.0.smithi100.stdout:3492: finishing write tid 1 to smithi10066867-25 2024-04-07T16:27:39.947 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 17 v 1274 (ObjNum 289 snap 0 seq_num 3067284976) dirty exists 2024-04-07T16:27:39.947 INFO:tasks.rados.rados.0.smithi100.stdout:3492: finishing write tid 2 to smithi10066867-25 2024-04-07T16:27:39.948 INFO:tasks.rados.rados.0.smithi100.stdout:3492: finishing write tid 3 to smithi10066867-25 2024-04-07T16:27:39.948 INFO:tasks.rados.rados.0.smithi100.stdout:3492: finishing write tid 4 to smithi10066867-25 2024-04-07T16:27:39.948 INFO:tasks.rados.rados.0.smithi100.stdout:3492: finishing write tid 5 to smithi10066867-25 2024-04-07T16:27:39.948 INFO:tasks.rados.rados.0.smithi100.stdout:3492: oid 25 updating version 0 to 1701 2024-04-07T16:27:39.948 INFO:tasks.rados.rados.0.smithi100.stdout:3492: oid 25 updating version 1701 to 1702 2024-04-07T16:27:39.948 INFO:tasks.rados.rados.0.smithi100.stdout:3492: oid 25 version 1702 is already newer than 1700 2024-04-07T16:27:39.948 INFO:tasks.rados.rados.0.smithi100.stdout:3492: oid 25 updating version 1702 to 1703 2024-04-07T16:27:39.948 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 25 v 1703 (ObjNum 1082 snap 314 seq_num 1082) dirty exists 2024-04-07T16:27:39.948 INFO:tasks.rados.rados.0.smithi100.stdout:3492: left oid 25 (ObjNum 1082 snap 314 seq_num 1082) 2024-04-07T16:27:39.948 INFO:tasks.rados.rados.0.smithi100.stdout:3492: done (2 left) 2024-04-07T16:27:39.948 INFO:tasks.rados.rados.0.smithi100.stdout:3493: done (1 left) 2024-04-07T16:27:39.948 INFO:tasks.rados.rados.0.smithi100.stdout:3494: done (0 left) 2024-04-07T16:27:39.948 INFO:tasks.rados.rados.0.smithi100.stdout:3495: snap_remove snap 309 2024-04-07T16:27:40.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:39 smithi100 bash[76202]: cluster 2024-04-07T16:27:38.796715+0000 mgr.y (mgr.25175) 356 : cluster [DBG] pgmap v644: 121 pgs: 121 active+clean; 173 MiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 6.6 MiB/s rd, 2.9 MiB/s wr, 24 op/s 2024-04-07T16:27:40.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:39 smithi100 bash[76202]: cluster 2024-04-07T16:27:38.948824+0000 mon.a (mon.0) 1174 : cluster [DBG] osdmap e701: 8 total, 8 up, 8 in 2024-04-07T16:27:40.187 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Apr 07 16:27:40 smithi100 bash[105005]: ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4-alertmanager-a 2024-04-07T16:27:40.188 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:39 smithi100 bash[74942]: cluster 2024-04-07T16:27:38.796715+0000 mgr.y (mgr.25175) 356 : cluster [DBG] pgmap v644: 121 pgs: 121 active+clean; 173 MiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 6.6 MiB/s rd, 2.9 MiB/s wr, 24 op/s 2024-04-07T16:27:40.188 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:39 smithi100 bash[74942]: cluster 2024-04-07T16:27:38.948824+0000 mon.a (mon.0) 1174 : cluster [DBG] osdmap e701: 8 total, 8 up, 8 in 2024-04-07T16:27:40.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:39 smithi144 bash[66654]: cluster 2024-04-07T16:27:38.796715+0000 mgr.y (mgr.25175) 356 : cluster [DBG] pgmap v644: 121 pgs: 121 active+clean; 173 MiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 6.6 MiB/s rd, 2.9 MiB/s wr, 24 op/s 2024-04-07T16:27:40.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:39 smithi144 bash[66654]: cluster 2024-04-07T16:27:38.948824+0000 mon.a (mon.0) 1174 : cluster [DBG] osdmap e701: 8 total, 8 up, 8 in 2024-04-07T16:27:40.459 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:40 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:27:40.460 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:40 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:27:40.460 INFO:journalctl@ceph.osd.0.smithi100.stdout:Apr 07 16:27:40 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:27:40.460 INFO:journalctl@ceph.osd.1.smithi100.stdout:Apr 07 16:27:40 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:27:40.460 INFO:journalctl@ceph.osd.3.smithi100.stdout:Apr 07 16:27:40 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:27:40.461 INFO:journalctl@ceph.mds.a.smithi100.stdout:Apr 07 16:27:40 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:27:40.461 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 16:27:40 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:27:40.461 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:27:40 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:27:40.461 INFO:journalctl@ceph.osd.2.smithi100.stdout:Apr 07 16:27:40 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:27:40.462 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Apr 07 16:27:40 smithi100 bash[105065]: Error response from daemon: No such container: ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4-alertmanager-a 2024-04-07T16:27:40.462 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Apr 07 16:27:40 smithi100 systemd[1]: ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@alertmanager.a.service: Deactivated successfully. 2024-04-07T16:27:40.462 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Apr 07 16:27:40 smithi100 systemd[1]: Stopped Ceph alertmanager.a for 0702e3a8-f4f8-11ee-b649-cb9ed24678a4. 2024-04-07T16:27:40.462 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Apr 07 16:27:40 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:27:40.937 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Apr 07 16:27:40 smithi100 systemd[1]: Started Ceph alertmanager.a for 0702e3a8-f4f8-11ee-b649-cb9ed24678a4. 2024-04-07T16:27:40.962 INFO:tasks.rados.rados.0.smithi100.stdout:3495: done (0 left) 2024-04-07T16:27:40.962 INFO:tasks.rados.rados.0.smithi100.stdout:3496: snap_remove snap 306 2024-04-07T16:27:41.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:40 smithi144 bash[66654]: cluster 2024-04-07T16:27:39.960396+0000 mon.a (mon.0) 1175 : cluster [DBG] osdmap e702: 8 total, 8 up, 8 in 2024-04-07T16:27:41.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:40 smithi144 bash[66654]: audit 2024-04-07T16:27:40.519512+0000 mon.a (mon.0) 1176 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:27:41.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:40 smithi144 bash[66654]: audit 2024-04-07T16:27:40.528442+0000 mon.a (mon.0) 1177 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:27:41.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:40 smithi144 bash[66654]: audit 2024-04-07T16:27:40.574861+0000 mon.a (mon.0) 1178 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:27:41.422 INFO:teuthology.orchestra.run.smithi100.stdout:true 2024-04-07T16:27:41.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:40 smithi100 bash[76202]: cluster 2024-04-07T16:27:39.960396+0000 mon.a (mon.0) 1175 : cluster [DBG] osdmap e702: 8 total, 8 up, 8 in 2024-04-07T16:27:41.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:40 smithi100 bash[76202]: audit 2024-04-07T16:27:40.519512+0000 mon.a (mon.0) 1176 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:27:41.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:40 smithi100 bash[76202]: audit 2024-04-07T16:27:40.528442+0000 mon.a (mon.0) 1177 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:27:41.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:40 smithi100 bash[76202]: audit 2024-04-07T16:27:40.574861+0000 mon.a (mon.0) 1178 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:27:41.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:40 smithi100 bash[74942]: cluster 2024-04-07T16:27:39.960396+0000 mon.a (mon.0) 1175 : cluster [DBG] osdmap e702: 8 total, 8 up, 8 in 2024-04-07T16:27:41.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:40 smithi100 bash[74942]: audit 2024-04-07T16:27:40.519512+0000 mon.a (mon.0) 1176 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:27:41.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:40 smithi100 bash[74942]: audit 2024-04-07T16:27:40.528442+0000 mon.a (mon.0) 1177 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:27:41.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:40 smithi100 bash[74942]: audit 2024-04-07T16:27:40.574861+0000 mon.a (mon.0) 1178 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:27:41.776 INFO:teuthology.orchestra.run.smithi100.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-07T16:27:41.776 INFO:teuthology.orchestra.run.smithi100.stdout:alertmanager.a smithi100 *:9093,9094 starting - - - - 2024-04-07T16:27:41.776 INFO:teuthology.orchestra.run.smithi100.stdout:grafana.a smithi144 *:3000 running (13m) 20s ago 13m 79.4M - 9.4.7 954c08fa6188 7180c0708acd 2024-04-07T16:27:41.776 INFO:teuthology.orchestra.run.smithi100.stdout:mds.a smithi100 running (103s) 44s ago 15m 13.4M - 19.0.0-2785-gadce92d0 c0ecd4226ac9 075552e67bd0 2024-04-07T16:27:41.776 INFO:teuthology.orchestra.run.smithi100.stdout:mds.b smithi144 running (76s) 20s ago 15m 11.7M - 19.0.0-2785-gadce92d0 c0ecd4226ac9 fc737f3b1edc 2024-04-07T16:27:41.776 INFO:teuthology.orchestra.run.smithi100.stdout:mgr.x smithi144 *:8443,9283,8765 running (7m) 20s ago 22m 441M - 19.0.0-2785-gadce92d0 c0ecd4226ac9 233745c292a6 2024-04-07T16:27:41.777 INFO:teuthology.orchestra.run.smithi100.stdout:mgr.y smithi100 *:8443,9283,8765 running (8m) 44s ago 25m 507M - 19.0.0-2785-gadce92d0 c0ecd4226ac9 254dc65cdaa7 2024-04-07T16:27:41.777 INFO:teuthology.orchestra.run.smithi100.stdout:mon.a smithi100 running (7m) 44s ago 25m 69.0M 2048M 19.0.0-2785-gadce92d0 c0ecd4226ac9 718e12655c04 2024-04-07T16:27:41.777 INFO:teuthology.orchestra.run.smithi100.stdout:mon.b smithi144 running (6m) 20s ago 23m 54.6M 2048M 19.0.0-2785-gadce92d0 c0ecd4226ac9 aef817664bff 2024-04-07T16:27:41.777 INFO:teuthology.orchestra.run.smithi100.stdout:mon.c smithi100 running (7m) 44s ago 23m 57.5M 2048M 19.0.0-2785-gadce92d0 c0ecd4226ac9 63e889040f6e 2024-04-07T16:27:41.777 INFO:teuthology.orchestra.run.smithi100.stdout:node-exporter.a smithi100 *:9100 running (54s) 44s ago 14m 6811k - 1.5.0 0da6a335fe13 9609d3a1c1ba 2024-04-07T16:27:41.777 INFO:teuthology.orchestra.run.smithi100.stdout:node-exporter.b smithi144 *:9100 running (50s) 20s ago 14m 7348k - 1.5.0 0da6a335fe13 abbe3a5b7ce6 2024-04-07T16:27:41.777 INFO:teuthology.orchestra.run.smithi100.stdout:osd.0 smithi100 running (5m) 44s ago 22m 341M 2524M 19.0.0-2785-gadce92d0 c0ecd4226ac9 a6e83c9d1abc 2024-04-07T16:27:41.777 INFO:teuthology.orchestra.run.smithi100.stdout:osd.1 smithi100 running (5m) 44s ago 21m 303M 2524M 19.0.0-2785-gadce92d0 c0ecd4226ac9 c50fe40079b6 2024-04-07T16:27:41.777 INFO:teuthology.orchestra.run.smithi100.stdout:osd.2 smithi100 running (4m) 44s ago 20m 362M 2524M 19.0.0-2785-gadce92d0 c0ecd4226ac9 8e631c43064b 2024-04-07T16:27:41.777 INFO:teuthology.orchestra.run.smithi100.stdout:osd.3 smithi100 running (4m) 44s ago 19m 151M 2524M 19.0.0-2785-gadce92d0 c0ecd4226ac9 402931247c79 2024-04-07T16:27:41.777 INFO:teuthology.orchestra.run.smithi100.stdout:osd.4 smithi144 running (3m) 20s ago 18m 345M 2524M 19.0.0-2785-gadce92d0 c0ecd4226ac9 ebbb7be95d33 2024-04-07T16:27:41.777 INFO:teuthology.orchestra.run.smithi100.stdout:osd.5 smithi144 running (3m) 20s ago 17m 222M 2524M 19.0.0-2785-gadce92d0 c0ecd4226ac9 ecce27bcb2f8 2024-04-07T16:27:41.777 INFO:teuthology.orchestra.run.smithi100.stdout:osd.6 smithi144 running (2m) 20s ago 17m 111M 2524M 19.0.0-2785-gadce92d0 c0ecd4226ac9 3788c5c3f427 2024-04-07T16:27:41.777 INFO:teuthology.orchestra.run.smithi100.stdout:osd.7 smithi144 running (2m) 20s ago 16m 193M 2524M 19.0.0-2785-gadce92d0 c0ecd4226ac9 9365aba97dc0 2024-04-07T16:27:41.778 INFO:teuthology.orchestra.run.smithi100.stdout:prometheus.a smithi144 *:9095 running (25s) 20s ago 14m 35.9M - 2.43.0 a07b618ecd1d e7d72b7c75c4 2024-04-07T16:27:41.923 INFO:tasks.rados.rados.0.smithi100.stdout:3496: done (0 left) 2024-04-07T16:27:41.923 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 29 current snap is 314 2024-04-07T16:27:41.923 INFO:tasks.rados.rados.0.smithi100.stdout:3497: seq_num 1083 ranges {0=540672,540672=589824,1130496=212992} 2024-04-07T16:27:41.927 INFO:tasks.rados.rados.0.smithi100.stdout:3497: writing smithi10066867-29 from 0 to 540672 tid 1 2024-04-07T16:27:41.932 INFO:tasks.rados.rados.0.smithi100.stdout:3497: writing smithi10066867-29 from 540672 to 1130496 tid 2 2024-04-07T16:27:41.933 INFO:tasks.rados.rados.0.smithi100.stdout:3497: writing smithi10066867-29 from 1130496 to 1343488 tid 3 2024-04-07T16:27:41.933 INFO:tasks.rados.rados.0.smithi100.stdout:3498: snap_create 2024-04-07T16:27:41.937 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Apr 07 16:27:41 smithi100 bash[105121]: ts=2024-04-07T16:27:41.714Z caller=main.go:240 level=info msg="Starting Alertmanager" version="(version=0.25.0, branch=HEAD, revision=258fab7cdd551f2cf251ed0348f0ad7289aee789)" 2024-04-07T16:27:41.937 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Apr 07 16:27:41 smithi100 bash[105121]: ts=2024-04-07T16:27:41.714Z caller=main.go:241 level=info build_context="(go=go1.19.4, user=root@abe866dd5717, date=20221222-14:51:36)" 2024-04-07T16:27:41.937 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Apr 07 16:27:41 smithi100 bash[105121]: ts=2024-04-07T16:27:41.717Z caller=cluster.go:185 level=info component=cluster msg="setting advertise address explicitly" addr=172.21.15.100 port=9094 2024-04-07T16:27:41.937 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Apr 07 16:27:41 smithi100 bash[105121]: ts=2024-04-07T16:27:41.718Z caller=cluster.go:681 level=info component=cluster msg="Waiting for gossip to settle..." interval=2s 2024-04-07T16:27:41.937 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Apr 07 16:27:41 smithi100 bash[105121]: ts=2024-04-07T16:27:41.749Z caller=coordinator.go:113 level=info component=configuration msg="Loading configuration file" file=/etc/alertmanager/alertmanager.yml 2024-04-07T16:27:41.937 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Apr 07 16:27:41 smithi100 bash[105121]: ts=2024-04-07T16:27:41.750Z caller=coordinator.go:126 level=info component=configuration msg="Completed loading of configuration file" file=/etc/alertmanager/alertmanager.yml 2024-04-07T16:27:41.937 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Apr 07 16:27:41 smithi100 bash[105121]: ts=2024-04-07T16:27:41.753Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9093 2024-04-07T16:27:41.937 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Apr 07 16:27:41 smithi100 bash[105121]: ts=2024-04-07T16:27:41.753Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9093 2024-04-07T16:27:41.944 INFO:tasks.rados.rados.0.smithi100.stdout:3497: finishing write tid 1 to smithi10066867-29 2024-04-07T16:27:41.949 INFO:tasks.rados.rados.0.smithi100.stdout:3497: finishing write tid 2 to smithi10066867-29 2024-04-07T16:27:41.949 INFO:tasks.rados.rados.0.smithi100.stdout:3497: finishing write tid 3 to smithi10066867-29 2024-04-07T16:27:41.949 INFO:tasks.rados.rados.0.smithi100.stdout:3497: finishing write tid 4 to smithi10066867-29 2024-04-07T16:27:41.951 INFO:tasks.rados.rados.0.smithi100.stdout:3497: finishing write tid 5 to smithi10066867-29 2024-04-07T16:27:41.951 INFO:tasks.rados.rados.0.smithi100.stdout:3497: oid 29 updating version 0 to 1567 2024-04-07T16:27:41.951 INFO:tasks.rados.rados.0.smithi100.stdout:3497: oid 29 version 1567 is already newer than 1565 2024-04-07T16:27:41.952 INFO:tasks.rados.rados.0.smithi100.stdout:3497: oid 29 version 1567 is already newer than 1566 2024-04-07T16:27:41.952 INFO:tasks.rados.rados.0.smithi100.stdout:3497: oid 29 updating version 1567 to 1568 2024-04-07T16:27:41.952 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 29 v 1568 (ObjNum 1083 snap 314 seq_num 1083) dirty exists 2024-04-07T16:27:41.952 INFO:tasks.rados.rados.0.smithi100.stdout:3497: left oid 29 (ObjNum 1083 snap 314 seq_num 1083) 2024-04-07T16:27:42.210 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-04-07T16:27:42.210 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": { 2024-04-07T16:27:42.210 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph version 19.0.0-2785-gadce92d0 (adce92d056533f22dc1ee37e3141d098b046ec1b) squid (dev)": 3 2024-04-07T16:27:42.210 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-07T16:27:42.211 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": { 2024-04-07T16:27:42.211 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph version 19.0.0-2785-gadce92d0 (adce92d056533f22dc1ee37e3141d098b046ec1b) squid (dev)": 2 2024-04-07T16:27:42.211 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-07T16:27:42.211 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": { 2024-04-07T16:27:42.211 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph version 19.0.0-2785-gadce92d0 (adce92d056533f22dc1ee37e3141d098b046ec1b) squid (dev)": 8 2024-04-07T16:27:42.211 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-07T16:27:42.211 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": { 2024-04-07T16:27:42.211 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph version 19.0.0-2785-gadce92d0 (adce92d056533f22dc1ee37e3141d098b046ec1b) squid (dev)": 2 2024-04-07T16:27:42.211 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-07T16:27:42.211 INFO:teuthology.orchestra.run.smithi100.stdout: "overall": { 2024-04-07T16:27:42.211 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph version 19.0.0-2785-gadce92d0 (adce92d056533f22dc1ee37e3141d098b046ec1b) squid (dev)": 15 2024-04-07T16:27:42.211 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-04-07T16:27:42.211 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-04-07T16:27:42.305 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:41 smithi144 bash[66654]: cluster 2024-04-07T16:27:40.797397+0000 mgr.y (mgr.25175) 357 : cluster [DBG] pgmap v647: 121 pgs: 121 active+clean; 180 MiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 13 MiB/s rd, 4.7 MiB/s wr, 43 op/s 2024-04-07T16:27:42.305 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:41 smithi144 bash[66654]: cluster 2024-04-07T16:27:40.976757+0000 mon.a (mon.0) 1179 : cluster [DBG] osdmap e703: 8 total, 8 up, 8 in 2024-04-07T16:27:42.305 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:41 smithi144 bash[66654]: cluster 2024-04-07T16:27:41.936943+0000 mon.a (mon.0) 1180 : cluster [DBG] osdmap e704: 8 total, 8 up, 8 in 2024-04-07T16:27:42.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:41 smithi100 bash[74942]: cluster 2024-04-07T16:27:40.797397+0000 mgr.y (mgr.25175) 357 : cluster [DBG] pgmap v647: 121 pgs: 121 active+clean; 180 MiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 13 MiB/s rd, 4.7 MiB/s wr, 43 op/s 2024-04-07T16:27:42.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:41 smithi100 bash[74942]: cluster 2024-04-07T16:27:40.976757+0000 mon.a (mon.0) 1179 : cluster [DBG] osdmap e703: 8 total, 8 up, 8 in 2024-04-07T16:27:42.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:41 smithi100 bash[74942]: cluster 2024-04-07T16:27:41.936943+0000 mon.a (mon.0) 1180 : cluster [DBG] osdmap e704: 8 total, 8 up, 8 in 2024-04-07T16:27:42.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:41 smithi100 bash[76202]: cluster 2024-04-07T16:27:40.797397+0000 mgr.y (mgr.25175) 357 : cluster [DBG] pgmap v647: 121 pgs: 121 active+clean; 180 MiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 13 MiB/s rd, 4.7 MiB/s wr, 43 op/s 2024-04-07T16:27:42.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:41 smithi100 bash[76202]: cluster 2024-04-07T16:27:40.976757+0000 mon.a (mon.0) 1179 : cluster [DBG] osdmap e703: 8 total, 8 up, 8 in 2024-04-07T16:27:42.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:41 smithi100 bash[76202]: cluster 2024-04-07T16:27:41.936943+0000 mon.a (mon.0) 1180 : cluster [DBG] osdmap e704: 8 total, 8 up, 8 in 2024-04-07T16:27:42.605 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:27:42 smithi144 bash[95233]: ts=2024-04-07T16:27:42.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:27:42.927 INFO:tasks.rados.rados.0.smithi100.stdout:3497: done (1 left) 2024-04-07T16:27:42.927 INFO:tasks.rados.rados.0.smithi100.stdout:3498: done (0 left) 2024-04-07T16:27:42.927 INFO:tasks.rados.rados.0.smithi100.stdout:3499: rmattr oid 43 current snap is 315 2024-04-07T16:27:42.927 INFO:tasks.rados.rados.0.smithi100.stdout:3500: read oid 38 snap 300 2024-04-07T16:27:42.927 INFO:tasks.rados.rados.0.smithi100.stdout:3500: expect (ObjNum 992 snap 299 seq_num 992) 2024-04-07T16:27:42.928 INFO:tasks.rados.rados.0.smithi100.stdout:3501: delete oid 13 current snap is 315 2024-04-07T16:27:42.943 INFO:tasks.rados.rados.0.smithi100.stdout:3501: done (2 left) 2024-04-07T16:27:42.943 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 35 current snap is 315 2024-04-07T16:27:42.944 INFO:tasks.rados.rados.0.smithi100.stdout:3502: seq_num 1085 ranges {3039232=581632,3620864=761856,4382720=40960} 2024-04-07T16:27:43.000 INFO:tasks.rados.rados.0.smithi100.stdout:3502: writing smithi10066867-35 from 3039232 to 3620864 tid 1 2024-04-07T16:27:43.010 INFO:tasks.rados.rados.0.smithi100.stdout:3502: writing smithi10066867-35 from 3620864 to 4382720 tid 2 2024-04-07T16:27:43.011 INFO:tasks.rados.rados.0.smithi100.stdout:3502: writing smithi10066867-35 from 4382720 to 4423680 tid 3 2024-04-07T16:27:43.011 INFO:tasks.rados.rados.0.smithi100.stdout:3503: copy_from oid 40 from oid 33 current snap is 315 2024-04-07T16:27:43.011 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 43 v 1196 (ObjNum 1030 snap 306 seq_num 1030) dirty exists 2024-04-07T16:27:43.011 INFO:tasks.rados.rados.0.smithi100.stdout:3499: done (3 left) 2024-04-07T16:27:43.011 INFO:tasks.rados.rados.0.smithi100.stdout:3504: delete oid 49 current snap is 315 2024-04-07T16:27:43.011 INFO:tasks.rados.rados.0.smithi100.stdout:3500: expect (ObjNum 992 snap 299 seq_num 992) 2024-04-07T16:27:43.180 INFO:tasks.rados.rados.0.smithi100.stdout:3502: finishing write tid 1 to smithi10066867-35 2024-04-07T16:27:43.180 INFO:tasks.rados.rados.0.smithi100.stdout:3502: finishing write tid 2 to smithi10066867-35 2024-04-07T16:27:43.180 INFO:tasks.rados.rados.0.smithi100.stdout:3502: finishing write tid 3 to smithi10066867-35 2024-04-07T16:27:43.180 INFO:tasks.rados.rados.0.smithi100.stdout:3500: done (3 left) 2024-04-07T16:27:43.180 INFO:tasks.rados.rados.0.smithi100.stdout:3504: done (2 left) 2024-04-07T16:27:43.180 INFO:tasks.rados.rados.0.smithi100.stdout:3505: read oid 17 snap 311 2024-04-07T16:27:43.180 INFO:tasks.rados.rados.0.smithi100.stdout:3505: expect deleted 2024-04-07T16:27:43.180 INFO:tasks.rados.rados.0.smithi100.stdout:3506: snap_remove snap 314 2024-04-07T16:27:43.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:42 smithi144 bash[66654]: audit 2024-04-07T16:27:41.423530+0000 mgr.y (mgr.25175) 358 : audit [DBG] from='client.44375 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:27:43.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:42 smithi144 bash[66654]: audit 2024-04-07T16:27:41.778207+0000 mgr.y (mgr.25175) 359 : audit [DBG] from='client.34470 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:27:43.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:42 smithi144 bash[66654]: audit 2024-04-07T16:27:42.225352+0000 mon.c (mon.2) 15 : audit [DBG] from='client.? 172.21.15.100:0/1243318815' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:27:43.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:42 smithi144 bash[66654]: cluster 2024-04-07T16:27:42.940390+0000 mon.a (mon.0) 1181 : cluster [DBG] osdmap e705: 8 total, 8 up, 8 in 2024-04-07T16:27:43.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:42 smithi100 bash[74942]: audit 2024-04-07T16:27:41.423530+0000 mgr.y (mgr.25175) 358 : audit [DBG] from='client.44375 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:27:43.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:42 smithi100 bash[74942]: audit 2024-04-07T16:27:41.778207+0000 mgr.y (mgr.25175) 359 : audit [DBG] from='client.34470 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:27:43.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:42 smithi100 bash[74942]: audit 2024-04-07T16:27:42.225352+0000 mon.c (mon.2) 15 : audit [DBG] from='client.? 172.21.15.100:0/1243318815' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:27:43.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:42 smithi100 bash[74942]: cluster 2024-04-07T16:27:42.940390+0000 mon.a (mon.0) 1181 : cluster [DBG] osdmap e705: 8 total, 8 up, 8 in 2024-04-07T16:27:43.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:42 smithi100 bash[76202]: audit 2024-04-07T16:27:41.423530+0000 mgr.y (mgr.25175) 358 : audit [DBG] from='client.44375 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:27:43.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:42 smithi100 bash[76202]: audit 2024-04-07T16:27:41.778207+0000 mgr.y (mgr.25175) 359 : audit [DBG] from='client.34470 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:27:43.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:42 smithi100 bash[76202]: audit 2024-04-07T16:27:42.225352+0000 mon.c (mon.2) 15 : audit [DBG] from='client.? 172.21.15.100:0/1243318815' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:27:43.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:42 smithi100 bash[76202]: cluster 2024-04-07T16:27:42.940390+0000 mon.a (mon.0) 1181 : cluster [DBG] osdmap e705: 8 total, 8 up, 8 in 2024-04-07T16:27:43.968 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Apr 07 16:27:43 smithi100 bash[105121]: ts=2024-04-07T16:27:43.718Z caller=cluster.go:706 level=info component=cluster msg="gossip not settled" polls=0 before=0 now=1 elapsed=2.000496227s 2024-04-07T16:27:43.988 INFO:tasks.rados.rados.0.smithi100.stdout:3506: done (3 left) 2024-04-07T16:27:43.989 INFO:tasks.rados.rados.0.smithi100.stdout:3507: snap_create 2024-04-07T16:27:43.989 INFO:tasks.rados.rados.0.smithi100.stdout:3502: finishing write tid 4 to smithi10066867-35 2024-04-07T16:27:43.989 INFO:tasks.rados.rados.0.smithi100.stdout:3502: finishing write tid 5 to smithi10066867-35 2024-04-07T16:27:43.989 INFO:tasks.rados.rados.0.smithi100.stdout:3502: oid 35 updating version 0 to 1710 2024-04-07T16:27:43.989 INFO:tasks.rados.rados.0.smithi100.stdout:3502: oid 35 updating version 1710 to 1711 2024-04-07T16:27:43.989 INFO:tasks.rados.rados.0.smithi100.stdout:3502: oid 35 version 1711 is already newer than 1709 2024-04-07T16:27:43.989 INFO:tasks.rados.rados.0.smithi100.stdout:3502: oid 35 version 1711 is already newer than 1708 2024-04-07T16:27:43.989 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 35 v 1711 (ObjNum 1085 snap 315 seq_num 1085) dirty exists 2024-04-07T16:27:43.989 INFO:tasks.rados.rados.0.smithi100.stdout:3502: left oid 35 (ObjNum 1085 snap 315 seq_num 1085) 2024-04-07T16:27:43.989 INFO:tasks.rados.rados.0.smithi100.stdout:3503: finishing copy_from racing read to smithi10066867-40 2024-04-07T16:27:43.989 INFO:tasks.rados.rados.0.smithi100.stdout:3503: finishing copy_from to smithi10066867-40 2024-04-07T16:27:43.989 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 40 v 1538 (ObjNum 968 snap 293 seq_num 968) dirty exists 2024-04-07T16:27:44.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:43 smithi144 bash[66654]: cluster 2024-04-07T16:27:42.798500+0000 mgr.y (mgr.25175) 360 : cluster [DBG] pgmap v650: 121 pgs: 3 active+clean+snaptrim, 118 active+clean; 181 MiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 10 MiB/s rd, 2.1 MiB/s wr, 29 op/s 2024-04-07T16:27:44.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:43 smithi100 bash[74942]: cluster 2024-04-07T16:27:42.798500+0000 mgr.y (mgr.25175) 360 : cluster [DBG] pgmap v650: 121 pgs: 3 active+clean+snaptrim, 118 active+clean; 181 MiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 10 MiB/s rd, 2.1 MiB/s wr, 29 op/s 2024-04-07T16:27:44.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:43 smithi100 bash[76202]: cluster 2024-04-07T16:27:42.798500+0000 mgr.y (mgr.25175) 360 : cluster [DBG] pgmap v650: 121 pgs: 3 active+clean+snaptrim, 118 active+clean; 181 MiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 10 MiB/s rd, 2.1 MiB/s wr, 29 op/s 2024-04-07T16:27:44.992 INFO:tasks.rados.rados.0.smithi100.stdout:3502: done (3 left) 2024-04-07T16:27:44.993 INFO:tasks.rados.rados.0.smithi100.stdout:3503: done (2 left) 2024-04-07T16:27:44.993 INFO:tasks.rados.rados.0.smithi100.stdout:3505: done (1 left) 2024-04-07T16:27:44.993 INFO:tasks.rados.rados.0.smithi100.stdout:3507: done (0 left) 2024-04-07T16:27:44.993 INFO:tasks.rados.rados.0.smithi100.stdout:3508: read oid 37 snap -1 2024-04-07T16:27:44.993 INFO:tasks.rados.rados.0.smithi100.stdout:3508: expect (ObjNum 968 snap 293 seq_num 968) 2024-04-07T16:27:44.993 INFO:tasks.rados.rados.0.smithi100.stdout:3509: rmattr oid 21 current snap is 316 2024-04-07T16:27:44.993 INFO:tasks.rados.rados.0.smithi100.stdout:3509: done (1 left) 2024-04-07T16:27:44.993 INFO:tasks.rados.rados.0.smithi100.stdout:3510: snap_create 2024-04-07T16:27:45.019 INFO:tasks.rados.rados.0.smithi100.stdout:3508: expect (ObjNum 968 snap 293 seq_num 968) 2024-04-07T16:27:45.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:45 smithi144 bash[66654]: cluster 2024-04-07T16:27:44.003080+0000 mon.a (mon.0) 1182 : cluster [DBG] osdmap e706: 8 total, 8 up, 8 in 2024-04-07T16:27:45.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:45 smithi100 bash[74942]: cluster 2024-04-07T16:27:44.003080+0000 mon.a (mon.0) 1182 : cluster [DBG] osdmap e706: 8 total, 8 up, 8 in 2024-04-07T16:27:45.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:45 smithi100 bash[76202]: cluster 2024-04-07T16:27:44.003080+0000 mon.a (mon.0) 1182 : cluster [DBG] osdmap e706: 8 total, 8 up, 8 in 2024-04-07T16:27:45.993 INFO:tasks.rados.rados.0.smithi100.stdout:3508: done (1 left) 2024-04-07T16:27:45.993 INFO:tasks.rados.rados.0.smithi100.stdout:3510: done (0 left) 2024-04-07T16:27:45.993 INFO:tasks.rados.rados.0.smithi100.stdout:3511: read oid 43 snap -1 2024-04-07T16:27:45.993 INFO:tasks.rados.rados.0.smithi100.stdout:3511: expect (ObjNum 1030 snap 306 seq_num 1030) 2024-04-07T16:27:45.993 INFO:tasks.rados.rados.0.smithi100.stdout:3512: snap_create 2024-04-07T16:27:46.001 INFO:tasks.rados.rados.0.smithi100.stdout:3511: expect (ObjNum 1030 snap 306 seq_num 1030) 2024-04-07T16:27:46.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:46 smithi144 bash[66654]: cluster 2024-04-07T16:27:44.799498+0000 mgr.y (mgr.25175) 361 : cluster [DBG] pgmap v653: 121 pgs: 3 active+clean+snaptrim, 118 active+clean; 184 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 980 KiB/s rd, 2.4 MiB/s wr, 15 op/s 2024-04-07T16:27:46.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:46 smithi144 bash[66654]: cluster 2024-04-07T16:27:45.006110+0000 mon.a (mon.0) 1183 : cluster [DBG] osdmap e707: 8 total, 8 up, 8 in 2024-04-07T16:27:46.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:46 smithi144 bash[66654]: cluster 2024-04-07T16:27:46.007897+0000 mon.a (mon.0) 1184 : cluster [DBG] osdmap e708: 8 total, 8 up, 8 in 2024-04-07T16:27:46.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:46 smithi100 bash[76202]: cluster 2024-04-07T16:27:44.799498+0000 mgr.y (mgr.25175) 361 : cluster [DBG] pgmap v653: 121 pgs: 3 active+clean+snaptrim, 118 active+clean; 184 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 980 KiB/s rd, 2.4 MiB/s wr, 15 op/s 2024-04-07T16:27:46.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:46 smithi100 bash[76202]: cluster 2024-04-07T16:27:45.006110+0000 mon.a (mon.0) 1183 : cluster [DBG] osdmap e707: 8 total, 8 up, 8 in 2024-04-07T16:27:46.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:46 smithi100 bash[76202]: cluster 2024-04-07T16:27:46.007897+0000 mon.a (mon.0) 1184 : cluster [DBG] osdmap e708: 8 total, 8 up, 8 in 2024-04-07T16:27:46.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:46 smithi100 bash[74942]: cluster 2024-04-07T16:27:44.799498+0000 mgr.y (mgr.25175) 361 : cluster [DBG] pgmap v653: 121 pgs: 3 active+clean+snaptrim, 118 active+clean; 184 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 980 KiB/s rd, 2.4 MiB/s wr, 15 op/s 2024-04-07T16:27:46.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:46 smithi100 bash[74942]: cluster 2024-04-07T16:27:45.006110+0000 mon.a (mon.0) 1183 : cluster [DBG] osdmap e707: 8 total, 8 up, 8 in 2024-04-07T16:27:46.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:46 smithi100 bash[74942]: cluster 2024-04-07T16:27:46.007897+0000 mon.a (mon.0) 1184 : cluster [DBG] osdmap e708: 8 total, 8 up, 8 in 2024-04-07T16:27:46.925 INFO:tasks.rados.rados.0.smithi100.stdout:3511: done (1 left) 2024-04-07T16:27:46.925 INFO:tasks.rados.rados.0.smithi100.stdout:3512: done (0 left) 2024-04-07T16:27:46.925 INFO:tasks.rados.rados.0.smithi100.stdout:3513: read oid 27 snap 310 2024-04-07T16:27:46.925 INFO:tasks.rados.rados.0.smithi100.stdout:3513: expect (ObjNum 1044 snap 307 seq_num 1044) 2024-04-07T16:27:46.925 INFO:tasks.rados.rados.0.smithi100.stdout:3514: copy_from oid 15 from oid 36 current snap is 318 2024-04-07T16:27:46.925 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 7 current snap is 318 2024-04-07T16:27:46.931 INFO:tasks.rados.rados.0.smithi100.stdout:3515: seq_num 1087 ranges {2891776=614400,3506176=532480,4038656=270336} 2024-04-07T16:27:46.950 INFO:tasks.rados.rados.0.smithi100.stdout:3515: writing smithi10066867-7 from 2891776 to 3506176 tid 1 2024-04-07T16:27:46.954 INFO:tasks.rados.rados.0.smithi100.stdout:3515: writing smithi10066867-7 from 3506176 to 4038656 tid 2 2024-04-07T16:27:46.957 INFO:tasks.rados.rados.0.smithi100.stdout:3515: writing smithi10066867-7 from 4038656 to 4308992 tid 3 2024-04-07T16:27:46.957 INFO:tasks.rados.rados.0.smithi100.stdout:3514: finishing copy_from to smithi10066867-15 2024-04-07T16:27:46.957 INFO:tasks.rados.rados.0.smithi100.stdout:3514: got expected ENOENT (src dne) 2024-04-07T16:27:46.957 INFO:tasks.rados.rados.0.smithi100.stdout:3514: finishing copy_from racing read to smithi10066867-15 2024-04-07T16:27:46.957 INFO:tasks.rados.rados.0.smithi100.stdout:3513: expect (ObjNum 1044 snap 307 seq_num 1044) 2024-04-07T16:27:47.298 INFO:tasks.rados.rados.0.smithi100.stdout:3515: finishing write tid 1 to smithi10066867-7 2024-04-07T16:27:47.298 INFO:tasks.rados.rados.0.smithi100.stdout:3515: finishing write tid 2 to smithi10066867-7 2024-04-07T16:27:47.298 INFO:tasks.rados.rados.0.smithi100.stdout:3515: finishing write tid 3 to smithi10066867-7 2024-04-07T16:27:47.298 INFO:tasks.rados.rados.0.smithi100.stdout:3515: finishing write tid 4 to smithi10066867-7 2024-04-07T16:27:47.298 INFO:tasks.rados.rados.0.smithi100.stdout:3515: finishing write tid 5 to smithi10066867-7 2024-04-07T16:27:47.298 INFO:tasks.rados.rados.0.smithi100.stdout:3515: oid 7 updating version 0 to 1197 2024-04-07T16:27:47.298 INFO:tasks.rados.rados.0.smithi100.stdout:3515: oid 7 updating version 1197 to 1198 2024-04-07T16:27:47.298 INFO:tasks.rados.rados.0.smithi100.stdout:3515: oid 7 updating version 1198 to 1200 2024-04-07T16:27:47.298 INFO:tasks.rados.rados.0.smithi100.stdout:3515: oid 7 version 1200 is already newer than 1199 2024-04-07T16:27:47.298 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 7 v 1200 (ObjNum 1087 snap 318 seq_num 1087) dirty exists 2024-04-07T16:27:47.298 INFO:tasks.rados.rados.0.smithi100.stdout:3515: left oid 7 (ObjNum 1087 snap 318 seq_num 1087) 2024-04-07T16:27:47.298 INFO:tasks.rados.rados.0.smithi100.stdout:3513: done (2 left) 2024-04-07T16:27:47.299 INFO:tasks.rados.rados.0.smithi100.stdout:3514: done (1 left) 2024-04-07T16:27:47.299 INFO:tasks.rados.rados.0.smithi100.stdout:3515: done (0 left) 2024-04-07T16:27:47.299 INFO:tasks.rados.rados.0.smithi100.stdout:3516: copy_from oid 4 from oid 1 current snap is 318 2024-04-07T16:27:47.299 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 22 current snap is 318 2024-04-07T16:27:47.299 INFO:tasks.rados.rados.0.smithi100.stdout:3517: seq_num 1088 ranges {458752=630784,1089536=466944,1556480=335872} 2024-04-07T16:27:47.306 INFO:tasks.rados.rados.0.smithi100.stdout:3517: writing smithi10066867-22 from 458752 to 1089536 tid 1 2024-04-07T16:27:47.310 INFO:tasks.rados.rados.0.smithi100.stdout:3517: writing smithi10066867-22 from 1089536 to 1556480 tid 2 2024-04-07T16:27:47.313 INFO:tasks.rados.rados.0.smithi100.stdout:3517: writing smithi10066867-22 from 1556480 to 1892352 tid 3 2024-04-07T16:27:47.314 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 14 current snap is 318 2024-04-07T16:27:47.314 INFO:tasks.rados.rados.0.smithi100.stdout:3516: finishing copy_from to smithi10066867-4 2024-04-07T16:27:47.314 INFO:tasks.rados.rados.0.smithi100.stdout:3516: got expected ENOENT (src dne) 2024-04-07T16:27:47.314 INFO:tasks.rados.rados.0.smithi100.stdout:3518: seq_num 1089 ranges {1523712=647168,2170880=745472,2916352=57344} 2024-04-07T16:27:47.331 INFO:tasks.rados.rados.0.smithi100.stdout:3518: writing smithi10066867-14 from 1523712 to 2170880 tid 1 2024-04-07T16:27:47.336 INFO:tasks.rados.rados.0.smithi100.stdout:3518: writing smithi10066867-14 from 2170880 to 2916352 tid 2 2024-04-07T16:27:47.338 INFO:tasks.rados.rados.0.smithi100.stdout:3518: writing smithi10066867-14 from 2916352 to 2973696 tid 3 2024-04-07T16:27:47.338 INFO:tasks.rados.rados.0.smithi100.stdout:3519: rollback oid 13 current snap is 318 2024-04-07T16:27:47.338 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 13 to 315 2024-04-07T16:27:47.338 INFO:tasks.rados.rados.0.smithi100.stdout:3516: finishing copy_from racing read to smithi10066867-4 2024-04-07T16:27:47.338 INFO:tasks.rados.rados.0.smithi100.stdout:3517: finishing write tid 1 to smithi10066867-22 2024-04-07T16:27:47.338 INFO:tasks.rados.rados.0.smithi100.stdout:3517: finishing write tid 2 to smithi10066867-22 2024-04-07T16:27:47.339 INFO:tasks.rados.rados.0.smithi100.stdout:3517: finishing write tid 3 to smithi10066867-22 2024-04-07T16:27:47.339 INFO:tasks.rados.rados.0.smithi100.stdout:3517: finishing write tid 4 to smithi10066867-22 2024-04-07T16:27:47.339 INFO:tasks.rados.rados.0.smithi100.stdout:3517: finishing write tid 5 to smithi10066867-22 2024-04-07T16:27:47.339 INFO:tasks.rados.rados.0.smithi100.stdout:3517: oid 22 updating version 0 to 1705 2024-04-07T16:27:47.339 INFO:tasks.rados.rados.0.smithi100.stdout:3517: oid 22 updating version 1705 to 1707 2024-04-07T16:27:47.339 INFO:tasks.rados.rados.0.smithi100.stdout:3517: oid 22 version 1707 is already newer than 1706 2024-04-07T16:27:47.339 INFO:tasks.rados.rados.0.smithi100.stdout:3517: oid 22 version 1707 is already newer than 1704 2024-04-07T16:27:47.339 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 22 v 1707 (ObjNum 1088 snap 318 seq_num 1088) dirty exists 2024-04-07T16:27:47.339 INFO:tasks.rados.rados.0.smithi100.stdout:3517: left oid 22 (ObjNum 1088 snap 318 seq_num 1088) 2024-04-07T16:27:47.339 INFO:tasks.rados.rados.0.smithi100.stdout:3516: done (3 left) 2024-04-07T16:27:47.339 INFO:tasks.rados.rados.0.smithi100.stdout:3517: done (2 left) 2024-04-07T16:27:47.339 INFO:tasks.rados.rados.0.smithi100.stdout:3520: copy_from oid 18 from oid 47 current snap is 318 2024-04-07T16:27:47.339 INFO:tasks.rados.rados.0.smithi100.stdout:3521: snap_create 2024-04-07T16:27:47.340 INFO:tasks.rados.rados.0.smithi100.stdout:3518: finishing write tid 1 to smithi10066867-14 2024-04-07T16:27:47.341 INFO:tasks.rados.rados.0.smithi100.stdout:3519: finishing rollback tid 1 to smithi10066867-13 2024-04-07T16:27:47.341 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 13 v 0 (ObjNum 161 snap 0 seq_num 1) dirty dne 2024-04-07T16:27:47.347 INFO:tasks.rados.rados.0.smithi100.stdout:3518: finishing write tid 2 to smithi10066867-14 2024-04-07T16:27:47.347 INFO:tasks.rados.rados.0.smithi100.stdout:3520: finishing copy_from racing read to smithi10066867-18 2024-04-07T16:27:47.350 INFO:tasks.rados.rados.0.smithi100.stdout:3518: finishing write tid 3 to smithi10066867-14 2024-04-07T16:27:47.350 INFO:tasks.rados.rados.0.smithi100.stdout:3520: finishing copy_from to smithi10066867-18 2024-04-07T16:27:47.350 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 18 v 1212 (ObjNum 1064 snap 310 seq_num 1064) dirty exists 2024-04-07T16:27:47.351 INFO:tasks.rados.rados.0.smithi100.stdout:3518: finishing write tid 4 to smithi10066867-14 2024-04-07T16:27:47.354 INFO:tasks.rados.rados.0.smithi100.stdout:3518: finishing write tid 5 to smithi10066867-14 2024-04-07T16:27:47.354 INFO:tasks.rados.rados.0.smithi100.stdout:3518: oid 14 updating version 0 to 1560 2024-04-07T16:27:47.354 INFO:tasks.rados.rados.0.smithi100.stdout:3518: oid 14 updating version 1560 to 1562 2024-04-07T16:27:47.354 INFO:tasks.rados.rados.0.smithi100.stdout:3518: oid 14 updating version 1562 to 1563 2024-04-07T16:27:47.354 INFO:tasks.rados.rados.0.smithi100.stdout:3518: oid 14 version 1563 is already newer than 1561 2024-04-07T16:27:47.354 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 14 v 1563 (ObjNum 1089 snap 318 seq_num 1089) dirty exists 2024-04-07T16:27:47.354 INFO:tasks.rados.rados.0.smithi100.stdout:3518: left oid 14 (ObjNum 1089 snap 318 seq_num 1089) 2024-04-07T16:27:47.605 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:47 smithi144 bash[66654]: audit 2024-04-07T16:27:46.221517+0000 mon.a (mon.0) 1185 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:27:47.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:47 smithi144 bash[66654]: audit 2024-04-07T16:27:46.231438+0000 mon.a (mon.0) 1186 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:27:47.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:47 smithi144 bash[66654]: audit 2024-04-07T16:27:46.733519+0000 mon.a (mon.0) 1187 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:27:47.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:47 smithi144 bash[66654]: cluster 2024-04-07T16:27:46.938898+0000 mon.a (mon.0) 1188 : cluster [DBG] osdmap e709: 8 total, 8 up, 8 in 2024-04-07T16:27:47.687 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:47 smithi100 bash[74942]: audit 2024-04-07T16:27:46.221517+0000 mon.a (mon.0) 1185 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:27:47.687 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:47 smithi100 bash[74942]: audit 2024-04-07T16:27:46.231438+0000 mon.a (mon.0) 1186 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:27:47.687 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:47 smithi100 bash[74942]: audit 2024-04-07T16:27:46.733519+0000 mon.a (mon.0) 1187 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:27:47.687 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:47 smithi100 bash[74942]: cluster 2024-04-07T16:27:46.938898+0000 mon.a (mon.0) 1188 : cluster [DBG] osdmap e709: 8 total, 8 up, 8 in 2024-04-07T16:27:47.688 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:47 smithi100 bash[76202]: audit 2024-04-07T16:27:46.221517+0000 mon.a (mon.0) 1185 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:27:47.688 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:47 smithi100 bash[76202]: audit 2024-04-07T16:27:46.231438+0000 mon.a (mon.0) 1186 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:27:47.688 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:47 smithi100 bash[76202]: audit 2024-04-07T16:27:46.733519+0000 mon.a (mon.0) 1187 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:27:47.688 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:47 smithi100 bash[76202]: cluster 2024-04-07T16:27:46.938898+0000 mon.a (mon.0) 1188 : cluster [DBG] osdmap e709: 8 total, 8 up, 8 in 2024-04-07T16:27:48.331 INFO:tasks.rados.rados.0.smithi100.stdout:3518: done (3 left) 2024-04-07T16:27:48.331 INFO:tasks.rados.rados.0.smithi100.stdout:3519: done (2 left) 2024-04-07T16:27:48.332 INFO:tasks.rados.rados.0.smithi100.stdout:3520: done (1 left) 2024-04-07T16:27:48.332 INFO:tasks.rados.rados.0.smithi100.stdout:3521: done (0 left) 2024-04-07T16:27:48.332 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 2 current snap is 319 2024-04-07T16:27:48.332 INFO:tasks.rados.rados.0.smithi100.stdout:3522: seq_num 1090 ranges {1040384=663552,1703936=630784,2334720=172032} 2024-04-07T16:27:48.348 INFO:tasks.rados.rados.0.smithi100.stdout:3522: writing smithi10066867-2 from 1040384 to 1703936 tid 1 2024-04-07T16:27:48.354 INFO:tasks.rados.rados.0.smithi100.stdout:3522: writing smithi10066867-2 from 1703936 to 2334720 tid 2 2024-04-07T16:27:48.356 INFO:tasks.rados.rados.0.smithi100.stdout:3522: writing smithi10066867-2 from 2334720 to 2506752 tid 3 2024-04-07T16:27:48.356 INFO:tasks.rados.rados.0.smithi100.stdout:3523: setattr oid 41 current snap is 319 2024-04-07T16:27:48.359 INFO:tasks.rados.rados.0.smithi100.stdout:3524: read oid 44 snap -1 2024-04-07T16:27:48.359 INFO:tasks.rados.rados.0.smithi100.stdout:3524: expect deleted 2024-04-07T16:27:48.359 INFO:tasks.rados.rados.0.smithi100.stdout:3525: read oid 6 snap -1 2024-04-07T16:27:48.359 INFO:tasks.rados.rados.0.smithi100.stdout:3525: expect (ObjNum 1049 snap 307 seq_num 1049) 2024-04-07T16:27:48.359 INFO:tasks.rados.rados.0.smithi100.stdout:3526: read oid 35 snap -1 2024-04-07T16:27:48.359 INFO:tasks.rados.rados.0.smithi100.stdout:3526: expect (ObjNum 1085 snap 315 seq_num 1085) 2024-04-07T16:27:48.360 INFO:tasks.rados.rados.0.smithi100.stdout:3527: read oid 4 snap -1 2024-04-07T16:27:48.360 INFO:tasks.rados.rados.0.smithi100.stdout:3527: expect (ObjNum 1041 snap 307 seq_num 1041) 2024-04-07T16:27:48.360 INFO:tasks.rados.rados.0.smithi100.stdout:3528: copy_from oid 16 from oid 8 current snap is 319 2024-04-07T16:27:48.360 INFO:tasks.rados.rados.0.smithi100.stdout:3529: setattr oid 47 current snap is 319 2024-04-07T16:27:48.363 INFO:tasks.rados.rados.0.smithi100.stdout:3530: read oid 22 snap -1 2024-04-07T16:27:48.363 INFO:tasks.rados.rados.0.smithi100.stdout:3530: expect (ObjNum 1088 snap 318 seq_num 1088) 2024-04-07T16:27:48.363 INFO:tasks.rados.rados.0.smithi100.stdout:3531: snap_remove snap 317 2024-04-07T16:27:48.605 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:48 smithi144 bash[66654]: cluster 2024-04-07T16:27:46.800127+0000 mgr.y (mgr.25175) 362 : cluster [DBG] pgmap v656: 121 pgs: 3 active+clean+snaptrim, 118 active+clean; 184 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 19 KiB/s rd, 2.1 MiB/s wr, 12 op/s 2024-04-07T16:27:48.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:48 smithi144 bash[66654]: audit 2024-04-07T16:27:47.328338+0000 mon.a (mon.0) 1189 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:27:48.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:48 smithi144 bash[66654]: audit 2024-04-07T16:27:47.334670+0000 mon.a (mon.0) 1190 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:27:48.687 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:27:48 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:27:48] "GET /metrics HTTP/1.1" 200 36345 "" "Prometheus/2.43.0" 2024-04-07T16:27:48.687 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:48 smithi100 bash[74942]: cluster 2024-04-07T16:27:46.800127+0000 mgr.y (mgr.25175) 362 : cluster [DBG] pgmap v656: 121 pgs: 3 active+clean+snaptrim, 118 active+clean; 184 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 19 KiB/s rd, 2.1 MiB/s wr, 12 op/s 2024-04-07T16:27:48.688 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:48 smithi100 bash[74942]: audit 2024-04-07T16:27:47.328338+0000 mon.a (mon.0) 1189 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:27:48.688 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:48 smithi100 bash[74942]: audit 2024-04-07T16:27:47.334670+0000 mon.a (mon.0) 1190 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:27:48.688 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:48 smithi100 bash[76202]: cluster 2024-04-07T16:27:46.800127+0000 mgr.y (mgr.25175) 362 : cluster [DBG] pgmap v656: 121 pgs: 3 active+clean+snaptrim, 118 active+clean; 184 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 19 KiB/s rd, 2.1 MiB/s wr, 12 op/s 2024-04-07T16:27:48.688 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:48 smithi100 bash[76202]: audit 2024-04-07T16:27:47.328338+0000 mon.a (mon.0) 1189 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:27:48.688 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:48 smithi100 bash[76202]: audit 2024-04-07T16:27:47.334670+0000 mon.a (mon.0) 1190 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:27:49.342 INFO:tasks.rados.rados.0.smithi100.stdout:3531: done (9 left) 2024-04-07T16:27:49.342 INFO:tasks.rados.rados.0.smithi100.stdout:3532: rmattr oid 49 current snap is 319 2024-04-07T16:27:49.342 INFO:tasks.rados.rados.0.smithi100.stdout:3532: done (9 left) 2024-04-07T16:27:49.342 INFO:tasks.rados.rados.0.smithi100.stdout:3533: rollback oid 50 current snap is 319 2024-04-07T16:27:49.342 INFO:tasks.rados.rados.0.smithi100.stdout:3522: finishing write tid 1 to smithi10066867-2 2024-04-07T16:27:49.342 INFO:tasks.rados.rados.0.smithi100.stdout:3522: finishing write tid 2 to smithi10066867-2 2024-04-07T16:27:49.342 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 47 v 1213 (ObjNum 1064 snap 310 seq_num 1064) dirty exists 2024-04-07T16:27:49.342 INFO:tasks.rados.rados.0.smithi100.stdout:3527: expect (ObjNum 1041 snap 307 seq_num 1041) 2024-04-07T16:27:49.376 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 41 v 1709 (ObjNum 161 snap 0 seq_num 1) dirty exists 2024-04-07T16:27:49.376 INFO:tasks.rados.rados.0.smithi100.stdout:3522: finishing write tid 3 to smithi10066867-2 2024-04-07T16:27:49.376 INFO:tasks.rados.rados.0.smithi100.stdout:3522: finishing write tid 4 to smithi10066867-2 2024-04-07T16:27:49.376 INFO:tasks.rados.rados.0.smithi100.stdout:3525: expect (ObjNum 1049 snap 307 seq_num 1049) 2024-04-07T16:27:49.404 INFO:tasks.rados.rados.0.smithi100.stdout:3522: finishing write tid 5 to smithi10066867-2 2024-04-07T16:27:49.404 INFO:tasks.rados.rados.0.smithi100.stdout:3522: oid 2 updating version 0 to 1656 2024-04-07T16:27:49.404 INFO:tasks.rados.rados.0.smithi100.stdout:3522: oid 2 version 1656 is already newer than 1655 2024-04-07T16:27:49.404 INFO:tasks.rados.rados.0.smithi100.stdout:3522: oid 2 updating version 1656 to 1657 2024-04-07T16:27:49.404 INFO:tasks.rados.rados.0.smithi100.stdout:3522: oid 2 version 1657 is already newer than 1654 2024-04-07T16:27:49.404 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 2 v 1657 (ObjNum 1090 snap 319 seq_num 1090) dirty exists 2024-04-07T16:27:49.404 INFO:tasks.rados.rados.0.smithi100.stdout:3522: left oid 2 (ObjNum 1090 snap 319 seq_num 1090) 2024-04-07T16:27:49.404 INFO:tasks.rados.rados.0.smithi100.stdout:3528: finishing copy_from racing read to smithi10066867-16 2024-04-07T16:27:49.404 INFO:tasks.rados.rados.0.smithi100.stdout:3528: finishing copy_from to smithi10066867-16 2024-04-07T16:27:49.404 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 16 v 1579 (ObjNum 1080 snap 314 seq_num 1080) dirty exists 2024-04-07T16:27:49.405 INFO:tasks.rados.rados.0.smithi100.stdout:3530: expect (ObjNum 1088 snap 318 seq_num 1088) 2024-04-07T16:27:49.463 INFO:tasks.rados.rados.0.smithi100.stdout:3526: expect (ObjNum 1085 snap 315 seq_num 1085) 2024-04-07T16:27:49.605 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:49 smithi144 bash[66654]: cluster 2024-04-07T16:27:48.344879+0000 mon.a (mon.0) 1191 : cluster [DBG] osdmap e710: 8 total, 8 up, 8 in 2024-04-07T16:27:49.683 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 50 to 300 2024-04-07T16:27:49.683 INFO:tasks.rados.rados.0.smithi100.stdout:3522: done (9 left) 2024-04-07T16:27:49.683 INFO:tasks.rados.rados.0.smithi100.stdout:3523: done (8 left) 2024-04-07T16:27:49.683 INFO:tasks.rados.rados.0.smithi100.stdout:3524: done (7 left) 2024-04-07T16:27:49.683 INFO:tasks.rados.rados.0.smithi100.stdout:3525: done (6 left) 2024-04-07T16:27:49.683 INFO:tasks.rados.rados.0.smithi100.stdout:3526: done (5 left) 2024-04-07T16:27:49.683 INFO:tasks.rados.rados.0.smithi100.stdout:3527: done (4 left) 2024-04-07T16:27:49.683 INFO:tasks.rados.rados.0.smithi100.stdout:3528: done (3 left) 2024-04-07T16:27:49.683 INFO:tasks.rados.rados.0.smithi100.stdout:3529: done (2 left) 2024-04-07T16:27:49.683 INFO:tasks.rados.rados.0.smithi100.stdout:3530: done (1 left) 2024-04-07T16:27:49.683 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 26 current snap is 319 2024-04-07T16:27:49.683 INFO:tasks.rados.rados.0.smithi100.stdout:3534: seq_num 1091 ranges {0=679936,679936=516096,1196032=286720} 2024-04-07T16:27:49.687 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:49 smithi100 bash[74942]: cluster 2024-04-07T16:27:48.344879+0000 mon.a (mon.0) 1191 : cluster [DBG] osdmap e710: 8 total, 8 up, 8 in 2024-04-07T16:27:49.687 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:49 smithi100 bash[76202]: cluster 2024-04-07T16:27:48.344879+0000 mon.a (mon.0) 1191 : cluster [DBG] osdmap e710: 8 total, 8 up, 8 in 2024-04-07T16:27:49.694 INFO:tasks.rados.rados.0.smithi100.stdout:3534: writing smithi10066867-26 from 0 to 679936 tid 1 2024-04-07T16:27:49.704 INFO:tasks.rados.rados.0.smithi100.stdout:3534: writing smithi10066867-26 from 679936 to 1196032 tid 2 2024-04-07T16:27:49.709 INFO:tasks.rados.rados.0.smithi100.stdout:3534: writing smithi10066867-26 from 1196032 to 1482752 tid 3 2024-04-07T16:27:49.710 INFO:tasks.rados.rados.0.smithi100.stdout:3535: read oid 8 snap 311 2024-04-07T16:27:49.710 INFO:tasks.rados.rados.0.smithi100.stdout:3535: expect (ObjNum 968 snap 293 seq_num 968) 2024-04-07T16:27:49.710 INFO:tasks.rados.rados.0.smithi100.stdout:3536: setattr oid 12 current snap is 319 2024-04-07T16:27:49.710 INFO:tasks.rados.rados.0.smithi100.stdout:3533: finishing rollback tid 1 to smithi10066867-50 2024-04-07T16:27:49.710 INFO:tasks.rados.rados.0.smithi100.stdout:3533: finishing rollback tid 2 to smithi10066867-50 2024-04-07T16:27:49.710 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 50 v 1565 (ObjNum 1001 snap 300 seq_num 1001) dirty exists 2024-04-07T16:27:49.710 INFO:tasks.rados.rados.0.smithi100.stdout:3534: finishing write tid 1 to smithi10066867-26 2024-04-07T16:27:49.710 INFO:tasks.rados.rados.0.smithi100.stdout:3534: finishing write tid 2 to smithi10066867-26 2024-04-07T16:27:49.715 INFO:tasks.rados.rados.0.smithi100.stdout:3533: done (3 left) 2024-04-07T16:27:49.715 INFO:tasks.rados.rados.0.smithi100.stdout:3537: rollback oid 38 current snap is 319 2024-04-07T16:27:49.715 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 38 to 315 2024-04-07T16:27:49.715 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 36 current snap is 319 2024-04-07T16:27:49.715 INFO:tasks.rados.rados.0.smithi100.stdout:3538: seq_num 1092 ranges {0=696320,696320=802816} 2024-04-07T16:27:49.727 INFO:tasks.rados.rados.0.smithi100.stdout:3538: writing smithi10066867-36 from 0 to 696320 tid 1 2024-04-07T16:27:49.735 INFO:tasks.rados.rados.0.smithi100.stdout:3538: writing smithi10066867-36 from 696320 to 1499136 tid 2 2024-04-07T16:27:49.735 INFO:tasks.rados.rados.0.smithi100.stdout:3539: copy_from oid 9 from oid 18 current snap is 319 2024-04-07T16:27:49.735 INFO:tasks.rados.rados.0.smithi100.stdout:3534: finishing write tid 3 to smithi10066867-26 2024-04-07T16:27:49.735 INFO:tasks.rados.rados.0.smithi100.stdout:3534: finishing write tid 4 to smithi10066867-26 2024-04-07T16:27:49.735 INFO:tasks.rados.rados.0.smithi100.stdout:3534: finishing write tid 5 to smithi10066867-26 2024-04-07T16:27:49.735 INFO:tasks.rados.rados.0.smithi100.stdout:3534: oid 26 updating version 0 to 1711 2024-04-07T16:27:49.735 INFO:tasks.rados.rados.0.smithi100.stdout:3534: oid 26 version 1711 is already newer than 1709 2024-04-07T16:27:49.735 INFO:tasks.rados.rados.0.smithi100.stdout:3534: oid 26 version 1711 is already newer than 1708 2024-04-07T16:27:49.735 INFO:tasks.rados.rados.0.smithi100.stdout:3534: oid 26 version 1711 is already newer than 1710 2024-04-07T16:27:49.736 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 26 v 1711 (ObjNum 1091 snap 319 seq_num 1091) dirty exists 2024-04-07T16:27:49.736 INFO:tasks.rados.rados.0.smithi100.stdout:3534: left oid 26 (ObjNum 1091 snap 319 seq_num 1091) 2024-04-07T16:27:49.736 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 12 v 1710 (ObjNum 1072 snap 312 seq_num 1072) dirty exists 2024-04-07T16:27:49.736 INFO:tasks.rados.rados.0.smithi100.stdout:3537: finishing rollback tid 0 to smithi10066867-38 2024-04-07T16:27:49.736 INFO:tasks.rados.rados.0.smithi100.stdout:3537: finishing rollback tid 1 to smithi10066867-38 2024-04-07T16:27:49.736 INFO:tasks.rados.rados.0.smithi100.stdout:3537: finishing rollback tid 2 to smithi10066867-38 2024-04-07T16:27:49.736 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 38 v 1697 (ObjNum 992 snap 299 seq_num 992) dirty exists 2024-04-07T16:27:49.736 INFO:tasks.rados.rados.0.smithi100.stdout:3534: done (5 left) 2024-04-07T16:27:49.736 INFO:tasks.rados.rados.0.smithi100.stdout:3536: done (4 left) 2024-04-07T16:27:49.736 INFO:tasks.rados.rados.0.smithi100.stdout:3537: done (3 left) 2024-04-07T16:27:49.736 INFO:tasks.rados.rados.0.smithi100.stdout:3540: copy_from oid 27 from oid 32 current snap is 319 2024-04-07T16:27:49.736 INFO:tasks.rados.rados.0.smithi100.stdout:3541: read oid 15 snap 300 2024-04-07T16:27:49.736 INFO:tasks.rados.rados.0.smithi100.stdout:3541: expect (ObjNum 913 snap 274 seq_num 913) 2024-04-07T16:27:49.737 INFO:tasks.rados.rados.0.smithi100.stdout:3542: snap_remove snap 315 2024-04-07T16:27:50.353 INFO:tasks.rados.rados.0.smithi100.stdout:3542: done (5 left) 2024-04-07T16:27:50.353 INFO:tasks.rados.rados.0.smithi100.stdout:3543: delete oid 45 current snap is 319 2024-04-07T16:27:50.353 INFO:tasks.rados.rados.0.smithi100.stdout:3538: finishing write tid 1 to smithi10066867-36 2024-04-07T16:27:50.353 INFO:tasks.rados.rados.0.smithi100.stdout:3540: finishing copy_from to smithi10066867-27 2024-04-07T16:27:50.353 INFO:tasks.rados.rados.0.smithi100.stdout:3540: got expected ENOENT (src dne) 2024-04-07T16:27:50.353 INFO:tasks.rados.rados.0.smithi100.stdout:3540: finishing copy_from racing read to smithi10066867-27 2024-04-07T16:27:50.353 INFO:tasks.rados.rados.0.smithi100.stdout:3535: expect (ObjNum 968 snap 293 seq_num 968) 2024-04-07T16:27:50.499 INFO:tasks.rados.rados.0.smithi100.stdout:3538: finishing write tid 2 to smithi10066867-36 2024-04-07T16:27:50.499 INFO:tasks.rados.rados.0.smithi100.stdout:3538: finishing write tid 3 to smithi10066867-36 2024-04-07T16:27:50.499 INFO:tasks.rados.rados.0.smithi100.stdout:3538: finishing write tid 4 to smithi10066867-36 2024-04-07T16:27:50.499 INFO:tasks.rados.rados.0.smithi100.stdout:3538: oid 36 updating version 0 to 1275 2024-04-07T16:27:50.499 INFO:tasks.rados.rados.0.smithi100.stdout:3538: oid 36 updating version 1275 to 1277 2024-04-07T16:27:50.499 INFO:tasks.rados.rados.0.smithi100.stdout:3538: oid 36 version 1277 is already newer than 1276 2024-04-07T16:27:50.499 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 36 v 1277 (ObjNum 1092 snap 319 seq_num 1092) dirty exists 2024-04-07T16:27:50.499 INFO:tasks.rados.rados.0.smithi100.stdout:3538: left oid 36 (ObjNum 1092 snap 319 seq_num 1092) 2024-04-07T16:27:50.499 INFO:tasks.rados.rados.0.smithi100.stdout:3541: expect (ObjNum 913 snap 274 seq_num 913) 2024-04-07T16:27:50.605 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:50 smithi144 bash[66654]: cluster 2024-04-07T16:27:48.800785+0000 mgr.y (mgr.25175) 363 : cluster [DBG] pgmap v659: 121 pgs: 121 active+clean; 184 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 5.2 MiB/s rd, 0 B/s wr, 3 op/s 2024-04-07T16:27:50.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:50 smithi144 bash[66654]: cluster 2024-04-07T16:27:49.356049+0000 mon.a (mon.0) 1192 : cluster [DBG] osdmap e711: 8 total, 8 up, 8 in 2024-04-07T16:27:50.663 INFO:tasks.rados.rados.0.smithi100.stdout:3539: finishing copy_from racing read to smithi10066867-9 2024-04-07T16:27:50.663 INFO:tasks.rados.rados.0.smithi100.stdout:3539: finishing copy_from to smithi10066867-9 2024-04-07T16:27:50.663 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 9 v 1212 (ObjNum 1064 snap 310 seq_num 1064) dirty exists 2024-04-07T16:27:50.663 INFO:tasks.rados.rados.0.smithi100.stdout:3535: done (5 left) 2024-04-07T16:27:50.663 INFO:tasks.rados.rados.0.smithi100.stdout:3538: done (4 left) 2024-04-07T16:27:50.664 INFO:tasks.rados.rados.0.smithi100.stdout:3539: done (3 left) 2024-04-07T16:27:50.664 INFO:tasks.rados.rados.0.smithi100.stdout:3540: done (2 left) 2024-04-07T16:27:50.664 INFO:tasks.rados.rados.0.smithi100.stdout:3541: done (1 left) 2024-04-07T16:27:50.664 INFO:tasks.rados.rados.0.smithi100.stdout:3543: done (0 left) 2024-04-07T16:27:50.664 INFO:tasks.rados.rados.0.smithi100.stdout:3544: snap_remove snap 318 2024-04-07T16:27:50.687 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:50 smithi100 bash[74942]: cluster 2024-04-07T16:27:48.800785+0000 mgr.y (mgr.25175) 363 : cluster [DBG] pgmap v659: 121 pgs: 121 active+clean; 184 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 5.2 MiB/s rd, 0 B/s wr, 3 op/s 2024-04-07T16:27:50.687 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:50 smithi100 bash[74942]: cluster 2024-04-07T16:27:49.356049+0000 mon.a (mon.0) 1192 : cluster [DBG] osdmap e711: 8 total, 8 up, 8 in 2024-04-07T16:27:50.687 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:50 smithi100 bash[76202]: cluster 2024-04-07T16:27:48.800785+0000 mgr.y (mgr.25175) 363 : cluster [DBG] pgmap v659: 121 pgs: 121 active+clean; 184 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 5.2 MiB/s rd, 0 B/s wr, 3 op/s 2024-04-07T16:27:50.688 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:50 smithi100 bash[76202]: cluster 2024-04-07T16:27:49.356049+0000 mon.a (mon.0) 1192 : cluster [DBG] osdmap e711: 8 total, 8 up, 8 in 2024-04-07T16:27:51.360 INFO:tasks.rados.rados.0.smithi100.stdout:3544: done (0 left) 2024-04-07T16:27:51.360 INFO:tasks.rados.rados.0.smithi100.stdout:3545: setattr oid 49 current snap is 319 2024-04-07T16:27:51.366 INFO:tasks.rados.rados.0.smithi100.stdout:3546: snap_remove snap 300 2024-04-07T16:27:51.605 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:51 smithi144 bash[66654]: cluster 2024-04-07T16:27:50.365803+0000 mon.a (mon.0) 1193 : cluster [DBG] osdmap e712: 8 total, 8 up, 8 in 2024-04-07T16:27:51.687 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:51 smithi100 bash[74942]: cluster 2024-04-07T16:27:50.365803+0000 mon.a (mon.0) 1193 : cluster [DBG] osdmap e712: 8 total, 8 up, 8 in 2024-04-07T16:27:51.688 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:51 smithi100 bash[76202]: cluster 2024-04-07T16:27:50.365803+0000 mon.a (mon.0) 1193 : cluster [DBG] osdmap e712: 8 total, 8 up, 8 in 2024-04-07T16:27:51.929 INFO:tasks.rados.rados.0.smithi100.stdout:3546: done (1 left) 2024-04-07T16:27:51.929 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 47 current snap is 319 2024-04-07T16:27:51.929 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 49 v 1711 (ObjNum 321 snap 0 seq_num 2704666528) dirty exists 2024-04-07T16:27:51.929 INFO:tasks.rados.rados.0.smithi100.stdout:3547: seq_num 1094 ranges {1024000=729088,1753088=573440,2326528=229376} 2024-04-07T16:27:51.941 INFO:tasks.rados.rados.0.smithi100.stdout:3547: writing smithi10066867-47 from 1024000 to 1753088 tid 1 2024-04-07T16:27:51.945 INFO:tasks.rados.rados.0.smithi100.stdout:3547: writing smithi10066867-47 from 1753088 to 2326528 tid 2 2024-04-07T16:27:51.947 INFO:tasks.rados.rados.0.smithi100.stdout:3547: writing smithi10066867-47 from 2326528 to 2555904 tid 3 2024-04-07T16:27:51.947 INFO:tasks.rados.rados.0.smithi100.stdout:3545: done (1 left) 2024-04-07T16:27:51.947 INFO:tasks.rados.rados.0.smithi100.stdout:3548: snap_create 2024-04-07T16:27:51.969 INFO:tasks.rados.rados.0.smithi100.stdout:3547: finishing write tid 1 to smithi10066867-47 2024-04-07T16:27:51.969 INFO:tasks.rados.rados.0.smithi100.stdout:3547: finishing write tid 2 to smithi10066867-47 2024-04-07T16:27:51.972 INFO:tasks.rados.rados.0.smithi100.stdout:3547: finishing write tid 3 to smithi10066867-47 2024-04-07T16:27:51.974 INFO:tasks.rados.rados.0.smithi100.stdout:3547: finishing write tid 4 to smithi10066867-47 2024-04-07T16:27:51.979 INFO:tasks.rados.rados.0.smithi100.stdout:3547: finishing write tid 5 to smithi10066867-47 2024-04-07T16:27:51.979 INFO:tasks.rados.rados.0.smithi100.stdout:3547: oid 47 updating version 0 to 1214 2024-04-07T16:27:51.979 INFO:tasks.rados.rados.0.smithi100.stdout:3547: oid 47 updating version 1214 to 1216 2024-04-07T16:27:51.979 INFO:tasks.rados.rados.0.smithi100.stdout:3547: oid 47 version 1216 is already newer than 1215 2024-04-07T16:27:51.979 INFO:tasks.rados.rados.0.smithi100.stdout:3547: oid 47 updating version 1216 to 1217 2024-04-07T16:27:51.979 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 47 v 1217 (ObjNum 1094 snap 319 seq_num 1094) dirty exists 2024-04-07T16:27:51.979 INFO:tasks.rados.rados.0.smithi100.stdout:3547: left oid 47 (ObjNum 1094 snap 319 seq_num 1094) 2024-04-07T16:27:52.187 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Apr 07 16:27:51 smithi100 bash[105121]: ts=2024-04-07T16:27:51.720Z caller=cluster.go:698 level=info component=cluster msg="gossip settled; proceeding" elapsed=10.001900997s 2024-04-07T16:27:52.605 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:27:52 smithi144 bash[95233]: ts=2024-04-07T16:27:52.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:27:52.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:52 smithi144 bash[66654]: cluster 2024-04-07T16:27:50.801469+0000 mgr.y (mgr.25175) 364 : cluster [DBG] pgmap v662: 121 pgs: 121 active+clean; 190 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 15 MiB/s rd, 1.3 MiB/s wr, 13 op/s 2024-04-07T16:27:52.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:52 smithi144 bash[66654]: cluster 2024-04-07T16:27:51.373977+0000 mon.a (mon.0) 1194 : cluster [DBG] osdmap e713: 8 total, 8 up, 8 in 2024-04-07T16:27:52.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:52 smithi144 bash[66654]: cluster 2024-04-07T16:27:51.941856+0000 mon.a (mon.0) 1195 : cluster [DBG] osdmap e714: 8 total, 8 up, 8 in 2024-04-07T16:27:52.687 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:52 smithi100 bash[76202]: cluster 2024-04-07T16:27:50.801469+0000 mgr.y (mgr.25175) 364 : cluster [DBG] pgmap v662: 121 pgs: 121 active+clean; 190 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 15 MiB/s rd, 1.3 MiB/s wr, 13 op/s 2024-04-07T16:27:52.688 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:52 smithi100 bash[76202]: cluster 2024-04-07T16:27:51.373977+0000 mon.a (mon.0) 1194 : cluster [DBG] osdmap e713: 8 total, 8 up, 8 in 2024-04-07T16:27:52.688 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:52 smithi100 bash[76202]: cluster 2024-04-07T16:27:51.941856+0000 mon.a (mon.0) 1195 : cluster [DBG] osdmap e714: 8 total, 8 up, 8 in 2024-04-07T16:27:52.688 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:52 smithi100 bash[74942]: cluster 2024-04-07T16:27:50.801469+0000 mgr.y (mgr.25175) 364 : cluster [DBG] pgmap v662: 121 pgs: 121 active+clean; 190 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 15 MiB/s rd, 1.3 MiB/s wr, 13 op/s 2024-04-07T16:27:52.688 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:52 smithi100 bash[74942]: cluster 2024-04-07T16:27:51.373977+0000 mon.a (mon.0) 1194 : cluster [DBG] osdmap e713: 8 total, 8 up, 8 in 2024-04-07T16:27:52.688 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:52 smithi100 bash[74942]: cluster 2024-04-07T16:27:51.941856+0000 mon.a (mon.0) 1195 : cluster [DBG] osdmap e714: 8 total, 8 up, 8 in 2024-04-07T16:27:52.936 INFO:tasks.rados.rados.0.smithi100.stdout:3547: done (1 left) 2024-04-07T16:27:52.936 INFO:tasks.rados.rados.0.smithi100.stdout:3548: done (0 left) 2024-04-07T16:27:52.936 INFO:tasks.rados.rados.0.smithi100.stdout:3549: rmattr oid 26 current snap is 320 2024-04-07T16:27:52.936 INFO:tasks.rados.rados.0.smithi100.stdout:3549: done (0 left) 2024-04-07T16:27:52.936 INFO:tasks.rados.rados.0.smithi100.stdout:3550: rollback oid 26 current snap is 320 2024-04-07T16:27:52.936 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 26 to 311 2024-04-07T16:27:52.936 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 9 current snap is 320 2024-04-07T16:27:52.936 INFO:tasks.rados.rados.0.smithi100.stdout:3551: seq_num 1095 ranges {1024000=745472,1769472=499712,2269184=303104} 2024-04-07T16:27:52.960 INFO:tasks.rados.rados.0.smithi100.stdout:3551: writing smithi10066867-9 from 1024000 to 1769472 tid 1 2024-04-07T16:27:52.967 INFO:tasks.rados.rados.0.smithi100.stdout:3551: writing smithi10066867-9 from 1769472 to 2269184 tid 2 2024-04-07T16:27:52.971 INFO:tasks.rados.rados.0.smithi100.stdout:3551: writing smithi10066867-9 from 2269184 to 2572288 tid 3 2024-04-07T16:27:52.971 INFO:tasks.rados.rados.0.smithi100.stdout:3552: snap_remove snap 319 2024-04-07T16:27:53.942 INFO:tasks.rados.rados.0.smithi100.stdout:3552: done (2 left) 2024-04-07T16:27:53.942 INFO:tasks.rados.rados.0.smithi100.stdout:3553: setattr oid 5 current snap is 320 2024-04-07T16:27:53.942 INFO:tasks.rados.rados.0.smithi100.stdout:3550: finishing rollback tid 0 to smithi10066867-26 2024-04-07T16:27:53.943 INFO:tasks.rados.rados.0.smithi100.stdout:3550: finishing rollback tid 1 to smithi10066867-26 2024-04-07T16:27:53.943 INFO:tasks.rados.rados.0.smithi100.stdout:3550: finishing rollback tid 2 to smithi10066867-26 2024-04-07T16:27:53.943 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 26 v 1714 (ObjNum 1009 snap 300 seq_num 1009) dirty exists 2024-04-07T16:27:53.943 INFO:tasks.rados.rados.0.smithi100.stdout:3551: finishing write tid 1 to smithi10066867-9 2024-04-07T16:27:53.943 INFO:tasks.rados.rados.0.smithi100.stdout:3551: finishing write tid 2 to smithi10066867-9 2024-04-07T16:27:53.943 INFO:tasks.rados.rados.0.smithi100.stdout:3551: finishing write tid 3 to smithi10066867-9 2024-04-07T16:27:53.943 INFO:tasks.rados.rados.0.smithi100.stdout:3551: finishing write tid 4 to smithi10066867-9 2024-04-07T16:27:53.943 INFO:tasks.rados.rados.0.smithi100.stdout:3551: finishing write tid 5 to smithi10066867-9 2024-04-07T16:27:53.943 INFO:tasks.rados.rados.0.smithi100.stdout:3551: oid 9 updating version 0 to 1215 2024-04-07T16:27:53.943 INFO:tasks.rados.rados.0.smithi100.stdout:3551: oid 9 version 1215 is already newer than 1214 2024-04-07T16:27:53.943 INFO:tasks.rados.rados.0.smithi100.stdout:3551: oid 9 updating version 1215 to 1216 2024-04-07T16:27:53.943 INFO:tasks.rados.rados.0.smithi100.stdout:3551: oid 9 version 1216 is already newer than 1213 2024-04-07T16:27:53.943 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 9 v 1216 (ObjNum 1095 snap 320 seq_num 1095) dirty exists 2024-04-07T16:27:53.943 INFO:tasks.rados.rados.0.smithi100.stdout:3551: left oid 9 (ObjNum 1095 snap 320 seq_num 1095) 2024-04-07T16:27:53.946 INFO:tasks.rados.rados.0.smithi100.stdout:3550: done (2 left) 2024-04-07T16:27:53.946 INFO:tasks.rados.rados.0.smithi100.stdout:3551: done (1 left) 2024-04-07T16:27:53.946 INFO:tasks.rados.rados.0.smithi100.stdout:3554: read oid 43 snap -1 2024-04-07T16:27:53.946 INFO:tasks.rados.rados.0.smithi100.stdout:3554: expect (ObjNum 1030 snap 306 seq_num 1030) 2024-04-07T16:27:53.946 INFO:tasks.rados.rados.0.smithi100.stdout:3555: read oid 41 snap 316 2024-04-07T16:27:53.946 INFO:tasks.rados.rados.0.smithi100.stdout:3555: expect deleted 2024-04-07T16:27:53.946 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 47 current snap is 320 2024-04-07T16:27:53.946 INFO:tasks.rados.rados.0.smithi100.stdout:3556: seq_num 1096 ranges {2555904=761856,3317760=786432,4104192=16384} 2024-04-07T16:27:53.989 INFO:tasks.rados.rados.0.smithi100.stdout:3556: writing smithi10066867-47 from 2555904 to 3317760 tid 1 2024-04-07T16:27:53.998 INFO:tasks.rados.rados.0.smithi100.stdout:3556: writing smithi10066867-47 from 3317760 to 4104192 tid 2 2024-04-07T16:27:53.998 INFO:tasks.rados.rados.0.smithi100.stdout:3556: writing smithi10066867-47 from 4104192 to 4120576 tid 3 2024-04-07T16:27:53.998 INFO:tasks.rados.rados.0.smithi100.stdout:3557: snap_remove snap 311 2024-04-07T16:27:54.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:53 smithi100 bash[76202]: cluster 2024-04-07T16:27:52.802546+0000 mgr.y (mgr.25175) 365 : cluster [DBG] pgmap v665: 121 pgs: 2 active+clean+snaptrim, 119 active+clean; 187 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 9.6 MiB/s rd, 2.0 MiB/s wr, 18 op/s 2024-04-07T16:27:54.188 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:53 smithi100 bash[76202]: cluster 2024-04-07T16:27:52.949195+0000 mon.a (mon.0) 1196 : cluster [DBG] osdmap e715: 8 total, 8 up, 8 in 2024-04-07T16:27:54.188 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:53 smithi100 bash[74942]: cluster 2024-04-07T16:27:52.802546+0000 mgr.y (mgr.25175) 365 : cluster [DBG] pgmap v665: 121 pgs: 2 active+clean+snaptrim, 119 active+clean; 187 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 9.6 MiB/s rd, 2.0 MiB/s wr, 18 op/s 2024-04-07T16:27:54.188 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:53 smithi100 bash[74942]: cluster 2024-04-07T16:27:52.949195+0000 mon.a (mon.0) 1196 : cluster [DBG] osdmap e715: 8 total, 8 up, 8 in 2024-04-07T16:27:54.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:53 smithi144 bash[66654]: cluster 2024-04-07T16:27:52.802546+0000 mgr.y (mgr.25175) 365 : cluster [DBG] pgmap v665: 121 pgs: 2 active+clean+snaptrim, 119 active+clean; 187 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 9.6 MiB/s rd, 2.0 MiB/s wr, 18 op/s 2024-04-07T16:27:54.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:53 smithi144 bash[66654]: cluster 2024-04-07T16:27:52.949195+0000 mon.a (mon.0) 1196 : cluster [DBG] osdmap e715: 8 total, 8 up, 8 in 2024-04-07T16:27:54.952 INFO:tasks.rados.rados.0.smithi100.stdout:3557: done (4 left) 2024-04-07T16:27:54.953 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 46 current snap is 320 2024-04-07T16:27:54.953 INFO:tasks.rados.rados.0.smithi100.stdout:3558: seq_num 1097 ranges {1703936=778240,2482176=671744,3153920=131072} 2024-04-07T16:27:54.967 INFO:tasks.rados.rados.0.smithi100.stdout:3558: writing smithi10066867-46 from 1703936 to 2482176 tid 1 2024-04-07T16:27:54.972 INFO:tasks.rados.rados.0.smithi100.stdout:3558: writing smithi10066867-46 from 2482176 to 3153920 tid 2 2024-04-07T16:27:54.973 INFO:tasks.rados.rados.0.smithi100.stdout:3558: writing smithi10066867-46 from 3153920 to 3284992 tid 3 2024-04-07T16:27:54.973 INFO:tasks.rados.rados.0.smithi100.stdout:3559: read oid 24 snap -1 2024-04-07T16:27:54.973 INFO:tasks.rados.rados.0.smithi100.stdout:3559: expect (ObjNum 1030 snap 306 seq_num 1030) 2024-04-07T16:27:54.973 INFO:tasks.rados.rados.0.smithi100.stdout:3560: delete oid 35 current snap is 320 2024-04-07T16:27:54.973 INFO:tasks.rados.rados.0.smithi100.stdout:3554: expect (ObjNum 1030 snap 306 seq_num 1030) 2024-04-07T16:27:54.995 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 5 v 1712 (ObjNum 1058 snap 309 seq_num 1058) dirty exists 2024-04-07T16:27:54.995 INFO:tasks.rados.rados.0.smithi100.stdout:3556: finishing write tid 1 to smithi10066867-47 2024-04-07T16:27:54.995 INFO:tasks.rados.rados.0.smithi100.stdout:3556: finishing write tid 2 to smithi10066867-47 2024-04-07T16:27:54.995 INFO:tasks.rados.rados.0.smithi100.stdout:3556: finishing write tid 3 to smithi10066867-47 2024-04-07T16:27:54.995 INFO:tasks.rados.rados.0.smithi100.stdout:3556: finishing write tid 4 to smithi10066867-47 2024-04-07T16:27:54.995 INFO:tasks.rados.rados.0.smithi100.stdout:3556: finishing write tid 5 to smithi10066867-47 2024-04-07T16:27:54.995 INFO:tasks.rados.rados.0.smithi100.stdout:3556: oid 47 updating version 0 to 1218 2024-04-07T16:27:54.995 INFO:tasks.rados.rados.0.smithi100.stdout:3556: oid 47 updating version 1218 to 1219 2024-04-07T16:27:54.995 INFO:tasks.rados.rados.0.smithi100.stdout:3556: oid 47 updating version 1219 to 1221 2024-04-07T16:27:54.995 INFO:tasks.rados.rados.0.smithi100.stdout:3556: oid 47 version 1221 is already newer than 1220 2024-04-07T16:27:54.996 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 47 v 1221 (ObjNum 1096 snap 320 seq_num 1096) dirty exists 2024-04-07T16:27:54.996 INFO:tasks.rados.rados.0.smithi100.stdout:3556: left oid 47 (ObjNum 1096 snap 320 seq_num 1096) 2024-04-07T16:27:54.996 INFO:tasks.rados.rados.0.smithi100.stdout:3559: expect (ObjNum 1030 snap 306 seq_num 1030) 2024-04-07T16:27:55.016 INFO:tasks.rados.rados.0.smithi100.stdout:3558: finishing write tid 1 to smithi10066867-46 2024-04-07T16:27:55.016 INFO:tasks.rados.rados.0.smithi100.stdout:3558: finishing write tid 2 to smithi10066867-46 2024-04-07T16:27:55.016 INFO:tasks.rados.rados.0.smithi100.stdout:3558: finishing write tid 3 to smithi10066867-46 2024-04-07T16:27:55.016 INFO:tasks.rados.rados.0.smithi100.stdout:3558: finishing write tid 4 to smithi10066867-46 2024-04-07T16:27:55.017 INFO:tasks.rados.rados.0.smithi100.stdout:3558: finishing write tid 5 to smithi10066867-46 2024-04-07T16:27:55.017 INFO:tasks.rados.rados.0.smithi100.stdout:3558: oid 46 updating version 0 to 1583 2024-04-07T16:27:55.017 INFO:tasks.rados.rados.0.smithi100.stdout:3558: oid 46 version 1583 is already newer than 1581 2024-04-07T16:27:55.017 INFO:tasks.rados.rados.0.smithi100.stdout:3558: oid 46 version 1583 is already newer than 1582 2024-04-07T16:27:55.017 INFO:tasks.rados.rados.0.smithi100.stdout:3558: oid 46 version 1583 is already newer than 1580 2024-04-07T16:27:55.017 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 46 v 1583 (ObjNum 1097 snap 320 seq_num 1097) dirty exists 2024-04-07T16:27:55.017 INFO:tasks.rados.rados.0.smithi100.stdout:3558: left oid 46 (ObjNum 1097 snap 320 seq_num 1097) 2024-04-07T16:27:55.017 INFO:tasks.rados.rados.0.smithi100.stdout:3553: done (6 left) 2024-04-07T16:27:55.017 INFO:tasks.rados.rados.0.smithi100.stdout:3554: done (5 left) 2024-04-07T16:27:55.017 INFO:tasks.rados.rados.0.smithi100.stdout:3555: done (4 left) 2024-04-07T16:27:55.017 INFO:tasks.rados.rados.0.smithi100.stdout:3556: done (3 left) 2024-04-07T16:27:55.017 INFO:tasks.rados.rados.0.smithi100.stdout:3558: done (2 left) 2024-04-07T16:27:55.017 INFO:tasks.rados.rados.0.smithi100.stdout:3559: done (1 left) 2024-04-07T16:27:55.017 INFO:tasks.rados.rados.0.smithi100.stdout:3560: done (0 left) 2024-04-07T16:27:55.017 INFO:tasks.rados.rados.0.smithi100.stdout:3561: rmattr oid 14 current snap is 320 2024-04-07T16:27:55.018 INFO:tasks.rados.rados.0.smithi100.stdout:3561: done (0 left) 2024-04-07T16:27:55.018 INFO:tasks.rados.rados.0.smithi100.stdout:3562: copy_from oid 8 from oid 49 current snap is 320 2024-04-07T16:27:55.018 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 38 current snap is 320 2024-04-07T16:27:55.018 INFO:tasks.rados.rados.0.smithi100.stdout:3563: seq_num 1099 ranges {3596288=409600,4005888=442368,4448256=720896,5169152=40960} 2024-04-07T16:27:55.039 INFO:tasks.rados.rados.0.smithi100.stdout:3563: writing smithi10066867-38 from 3596288 to 4005888 tid 1 2024-04-07T16:27:55.041 INFO:tasks.rados.rados.0.smithi100.stdout:3563: writing smithi10066867-38 from 4005888 to 4448256 tid 2 2024-04-07T16:27:55.046 INFO:tasks.rados.rados.0.smithi100.stdout:3563: writing smithi10066867-38 from 4448256 to 5169152 tid 3 2024-04-07T16:27:55.046 INFO:tasks.rados.rados.0.smithi100.stdout:3563: writing smithi10066867-38 from 5169152 to 5210112 tid 4 2024-04-07T16:27:55.046 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 18 current snap is 320 2024-04-07T16:27:55.046 INFO:tasks.rados.rados.0.smithi100.stdout:3562: finishing copy_from racing read to smithi10066867-8 2024-04-07T16:27:55.046 INFO:tasks.rados.rados.0.smithi100.stdout:3562: finishing copy_from to smithi10066867-8 2024-04-07T16:27:55.047 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 8 v 1711 (ObjNum 33 snap 0 seq_num 2701876272) dirty exists 2024-04-07T16:27:55.047 INFO:tasks.rados.rados.0.smithi100.stdout:3563: finishing write tid 1 to smithi10066867-38 2024-04-07T16:27:55.047 INFO:tasks.rados.rados.0.smithi100.stdout:3563: finishing write tid 2 to smithi10066867-38 2024-04-07T16:27:55.047 INFO:tasks.rados.rados.0.smithi100.stdout:3564: seq_num 1100 ranges {1024000=425984,1449984=729088,2179072=475136} 2024-04-07T16:27:55.054 INFO:tasks.rados.rados.0.smithi100.stdout:3564: writing smithi10066867-18 from 1024000 to 1449984 tid 1 2024-04-07T16:27:55.059 INFO:tasks.rados.rados.0.smithi100.stdout:3564: writing smithi10066867-18 from 1449984 to 2179072 tid 2 2024-04-07T16:27:55.062 INFO:tasks.rados.rados.0.smithi100.stdout:3564: writing smithi10066867-18 from 2179072 to 2654208 tid 3 2024-04-07T16:27:55.062 INFO:tasks.rados.rados.0.smithi100.stdout:3562: done (2 left) 2024-04-07T16:27:55.062 INFO:tasks.rados.rados.0.smithi100.stdout:3565: snap_remove snap 316 2024-04-07T16:27:55.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:54 smithi144 bash[66654]: cluster 2024-04-07T16:27:53.954739+0000 mon.a (mon.0) 1197 : cluster [DBG] osdmap e716: 8 total, 8 up, 8 in 2024-04-07T16:27:55.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:54 smithi100 bash[76202]: cluster 2024-04-07T16:27:53.954739+0000 mon.a (mon.0) 1197 : cluster [DBG] osdmap e716: 8 total, 8 up, 8 in 2024-04-07T16:27:55.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:54 smithi100 bash[74942]: cluster 2024-04-07T16:27:53.954739+0000 mon.a (mon.0) 1197 : cluster [DBG] osdmap e716: 8 total, 8 up, 8 in 2024-04-07T16:27:55.951 INFO:tasks.rados.rados.0.smithi100.stdout:3565: done (2 left) 2024-04-07T16:27:55.952 INFO:tasks.rados.rados.0.smithi100.stdout:3566: snap_remove snap 310 2024-04-07T16:27:56.208 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:55 smithi100 bash[74942]: cluster 2024-04-07T16:27:54.803441+0000 mgr.y (mgr.25175) 366 : cluster [DBG] pgmap v668: 121 pgs: 2 active+clean+snaptrim, 119 active+clean; 157 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 5.0 MiB/s rd, 1.9 MiB/s wr, 28 op/s 2024-04-07T16:27:56.208 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:55 smithi100 bash[74942]: cluster 2024-04-07T16:27:54.966462+0000 mon.a (mon.0) 1198 : cluster [DBG] osdmap e717: 8 total, 8 up, 8 in 2024-04-07T16:27:56.208 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:55 smithi100 bash[76202]: cluster 2024-04-07T16:27:54.803441+0000 mgr.y (mgr.25175) 366 : cluster [DBG] pgmap v668: 121 pgs: 2 active+clean+snaptrim, 119 active+clean; 157 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 5.0 MiB/s rd, 1.9 MiB/s wr, 28 op/s 2024-04-07T16:27:56.208 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:55 smithi100 bash[76202]: cluster 2024-04-07T16:27:54.966462+0000 mon.a (mon.0) 1198 : cluster [DBG] osdmap e717: 8 total, 8 up, 8 in 2024-04-07T16:27:56.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:55 smithi144 bash[66654]: cluster 2024-04-07T16:27:54.803441+0000 mgr.y (mgr.25175) 366 : cluster [DBG] pgmap v668: 121 pgs: 2 active+clean+snaptrim, 119 active+clean; 157 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 5.0 MiB/s rd, 1.9 MiB/s wr, 28 op/s 2024-04-07T16:27:56.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:55 smithi144 bash[66654]: cluster 2024-04-07T16:27:54.966462+0000 mon.a (mon.0) 1198 : cluster [DBG] osdmap e717: 8 total, 8 up, 8 in 2024-04-07T16:27:56.928 INFO:tasks.rados.rados.0.smithi100.stdout:3566: done (2 left) 2024-04-07T16:27:56.928 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 2 current snap is 320 2024-04-07T16:27:56.928 INFO:tasks.rados.rados.0.smithi100.stdout:3567: seq_num 1101 ranges {2506752=442368,2949120=606208,3555328=598016} 2024-04-07T16:27:56.953 INFO:tasks.rados.rados.0.smithi100.stdout:3567: writing smithi10066867-2 from 2506752 to 2949120 tid 1 2024-04-07T16:27:56.959 INFO:tasks.rados.rados.0.smithi100.stdout:3567: writing smithi10066867-2 from 2949120 to 3555328 tid 2 2024-04-07T16:27:56.965 INFO:tasks.rados.rados.0.smithi100.stdout:3567: writing smithi10066867-2 from 3555328 to 4153344 tid 3 2024-04-07T16:27:56.965 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 35 current snap is 320 2024-04-07T16:27:56.965 INFO:tasks.rados.rados.0.smithi100.stdout:3563: finishing write tid 3 to smithi10066867-38 2024-04-07T16:27:56.965 INFO:tasks.rados.rados.0.smithi100.stdout:3563: finishing write tid 4 to smithi10066867-38 2024-04-07T16:27:56.965 INFO:tasks.rados.rados.0.smithi100.stdout:3563: finishing write tid 5 to smithi10066867-38 2024-04-07T16:27:56.965 INFO:tasks.rados.rados.0.smithi100.stdout:3564: finishing write tid 1 to smithi10066867-18 2024-04-07T16:27:56.965 INFO:tasks.rados.rados.0.smithi100.stdout:3563: finishing write tid 6 to smithi10066867-38 2024-04-07T16:27:56.965 INFO:tasks.rados.rados.0.smithi100.stdout:3563: oid 38 updating version 0 to 1699 2024-04-07T16:27:56.965 INFO:tasks.rados.rados.0.smithi100.stdout:3563: oid 38 updating version 1699 to 1702 2024-04-07T16:27:56.966 INFO:tasks.rados.rados.0.smithi100.stdout:3563: oid 38 version 1702 is already newer than 1698 2024-04-07T16:27:56.966 INFO:tasks.rados.rados.0.smithi100.stdout:3563: oid 38 version 1702 is already newer than 1700 2024-04-07T16:27:56.966 INFO:tasks.rados.rados.0.smithi100.stdout:3563: oid 38 version 1702 is already newer than 1701 2024-04-07T16:27:56.966 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 38 v 1702 (ObjNum 1099 snap 320 seq_num 1099) dirty exists 2024-04-07T16:27:56.966 INFO:tasks.rados.rados.0.smithi100.stdout:3563: left oid 38 (ObjNum 1099 snap 320 seq_num 1099) 2024-04-07T16:27:56.966 INFO:tasks.rados.rados.0.smithi100.stdout:3564: finishing write tid 2 to smithi10066867-18 2024-04-07T16:27:56.966 INFO:tasks.rados.rados.0.smithi100.stdout:3564: finishing write tid 3 to smithi10066867-18 2024-04-07T16:27:56.966 INFO:tasks.rados.rados.0.smithi100.stdout:3564: finishing write tid 4 to smithi10066867-18 2024-04-07T16:27:56.966 INFO:tasks.rados.rados.0.smithi100.stdout:3564: finishing write tid 5 to smithi10066867-18 2024-04-07T16:27:56.966 INFO:tasks.rados.rados.0.smithi100.stdout:3564: oid 18 updating version 0 to 1861 2024-04-07T16:27:56.966 INFO:tasks.rados.rados.0.smithi100.stdout:3564: oid 18 version 1861 is already newer than 1860 2024-04-07T16:27:56.966 INFO:tasks.rados.rados.0.smithi100.stdout:3564: oid 18 version 1861 is already newer than 1858 2024-04-07T16:27:56.966 INFO:tasks.rados.rados.0.smithi100.stdout:3564: oid 18 version 1861 is already newer than 1859 2024-04-07T16:27:56.967 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 18 v 1861 (ObjNum 1100 snap 320 seq_num 1100) dirty exists 2024-04-07T16:27:56.967 INFO:tasks.rados.rados.0.smithi100.stdout:3564: left oid 18 (ObjNum 1100 snap 320 seq_num 1100) 2024-04-07T16:27:56.967 INFO:tasks.rados.rados.0.smithi100.stdout:3568: seq_num 1102 ranges {0=458752,458752=491520,950272=491520,1441792=221184} 2024-04-07T16:27:56.969 INFO:tasks.rados.rados.0.smithi100.stdout:3568: writing smithi10066867-35 from 0 to 458752 tid 1 2024-04-07T16:27:56.974 INFO:tasks.rados.rados.0.smithi100.stdout:3568: writing smithi10066867-35 from 458752 to 950272 tid 2 2024-04-07T16:27:56.979 INFO:tasks.rados.rados.0.smithi100.stdout:3568: writing smithi10066867-35 from 950272 to 1441792 tid 3 2024-04-07T16:27:56.982 INFO:tasks.rados.rados.0.smithi100.stdout:3568: writing smithi10066867-35 from 1441792 to 1662976 tid 4 2024-04-07T16:27:56.982 INFO:tasks.rados.rados.0.smithi100.stdout:3563: done (3 left) 2024-04-07T16:27:56.982 INFO:tasks.rados.rados.0.smithi100.stdout:3564: done (2 left) 2024-04-07T16:27:56.982 INFO:tasks.rados.rados.0.smithi100.stdout:3569: read oid 19 snap -1 2024-04-07T16:27:56.982 INFO:tasks.rados.rados.0.smithi100.stdout:3569: expect (ObjNum 1079 snap 314 seq_num 1079) 2024-04-07T16:27:56.982 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 37 current snap is 320 2024-04-07T16:27:56.982 INFO:tasks.rados.rados.0.smithi100.stdout:3567: finishing write tid 1 to smithi10066867-2 2024-04-07T16:27:56.983 INFO:tasks.rados.rados.0.smithi100.stdout:3567: finishing write tid 2 to smithi10066867-2 2024-04-07T16:27:56.983 INFO:tasks.rados.rados.0.smithi100.stdout:3567: finishing write tid 3 to smithi10066867-2 2024-04-07T16:27:56.983 INFO:tasks.rados.rados.0.smithi100.stdout:3567: finishing write tid 4 to smithi10066867-2 2024-04-07T16:27:56.983 INFO:tasks.rados.rados.0.smithi100.stdout:3568: finishing write tid 1 to smithi10066867-35 2024-04-07T16:27:56.983 INFO:tasks.rados.rados.0.smithi100.stdout:3570: seq_num 1103 ranges {3620864=475136,4096000=425984,4521984=696320,5218304=81920} 2024-04-07T16:27:57.008 INFO:tasks.rados.rados.0.smithi100.stdout:3570: writing smithi10066867-37 from 3620864 to 4096000 tid 1 2024-04-07T16:27:57.010 INFO:tasks.rados.rados.0.smithi100.stdout:3570: writing smithi10066867-37 from 4096000 to 4521984 tid 2 2024-04-07T16:27:57.014 INFO:tasks.rados.rados.0.smithi100.stdout:3570: writing smithi10066867-37 from 4521984 to 5218304 tid 3 2024-04-07T16:27:57.015 INFO:tasks.rados.rados.0.smithi100.stdout:3570: writing smithi10066867-37 from 5218304 to 5300224 tid 4 2024-04-07T16:27:57.015 INFO:tasks.rados.rados.0.smithi100.stdout:3571: delete oid 12 current snap is 320 2024-04-07T16:27:57.015 INFO:tasks.rados.rados.0.smithi100.stdout:3567: finishing write tid 5 to smithi10066867-2 2024-04-07T16:27:57.016 INFO:tasks.rados.rados.0.smithi100.stdout:3567: oid 2 updating version 0 to 1660 2024-04-07T16:27:57.016 INFO:tasks.rados.rados.0.smithi100.stdout:3567: oid 2 updating version 1660 to 1661 2024-04-07T16:27:57.016 INFO:tasks.rados.rados.0.smithi100.stdout:3567: oid 2 version 1661 is already newer than 1659 2024-04-07T16:27:57.016 INFO:tasks.rados.rados.0.smithi100.stdout:3567: oid 2 version 1661 is already newer than 1658 2024-04-07T16:27:57.016 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 2 v 1661 (ObjNum 1101 snap 320 seq_num 1101) dirty exists 2024-04-07T16:27:57.016 INFO:tasks.rados.rados.0.smithi100.stdout:3567: left oid 2 (ObjNum 1101 snap 320 seq_num 1101) 2024-04-07T16:27:57.016 INFO:tasks.rados.rados.0.smithi100.stdout:3568: finishing write tid 2 to smithi10066867-35 2024-04-07T16:27:57.016 INFO:tasks.rados.rados.0.smithi100.stdout:3568: finishing write tid 3 to smithi10066867-35 2024-04-07T16:27:57.016 INFO:tasks.rados.rados.0.smithi100.stdout:3568: finishing write tid 4 to smithi10066867-35 2024-04-07T16:27:57.016 INFO:tasks.rados.rados.0.smithi100.stdout:3569: expect (ObjNum 1079 snap 314 seq_num 1079) 2024-04-07T16:27:57.154 INFO:tasks.rados.rados.0.smithi100.stdout:3568: finishing write tid 5 to smithi10066867-35 2024-04-07T16:27:57.154 INFO:tasks.rados.rados.0.smithi100.stdout:3568: finishing write tid 6 to smithi10066867-35 2024-04-07T16:27:57.154 INFO:tasks.rados.rados.0.smithi100.stdout:3568: oid 35 updating version 0 to 1716 2024-04-07T16:27:57.154 INFO:tasks.rados.rados.0.smithi100.stdout:3568: oid 35 updating version 1716 to 1717 2024-04-07T16:27:57.154 INFO:tasks.rados.rados.0.smithi100.stdout:3568: oid 35 version 1717 is already newer than 1713 2024-04-07T16:27:57.154 INFO:tasks.rados.rados.0.smithi100.stdout:3568: oid 35 version 1717 is already newer than 1714 2024-04-07T16:27:57.154 INFO:tasks.rados.rados.0.smithi100.stdout:3568: oid 35 version 1717 is already newer than 1715 2024-04-07T16:27:57.154 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 35 v 1717 (ObjNum 1102 snap 320 seq_num 1102) dirty exists 2024-04-07T16:27:57.154 INFO:tasks.rados.rados.0.smithi100.stdout:3568: left oid 35 (ObjNum 1102 snap 320 seq_num 1102) 2024-04-07T16:27:57.154 INFO:tasks.rados.rados.0.smithi100.stdout:3570: finishing write tid 1 to smithi10066867-37 2024-04-07T16:27:57.155 INFO:tasks.rados.rados.0.smithi100.stdout:3570: finishing write tid 2 to smithi10066867-37 2024-04-07T16:27:57.155 INFO:tasks.rados.rados.0.smithi100.stdout:3567: done (4 left) 2024-04-07T16:27:57.155 INFO:tasks.rados.rados.0.smithi100.stdout:3568: done (3 left) 2024-04-07T16:27:57.155 INFO:tasks.rados.rados.0.smithi100.stdout:3569: done (2 left) 2024-04-07T16:27:57.155 INFO:tasks.rados.rados.0.smithi100.stdout:3571: done (1 left) 2024-04-07T16:27:57.155 INFO:tasks.rados.rados.0.smithi100.stdout:3572: read oid 26 snap -1 2024-04-07T16:27:57.155 INFO:tasks.rados.rados.0.smithi100.stdout:3572: expect (ObjNum 1009 snap 300 seq_num 1009) 2024-04-07T16:27:57.155 INFO:tasks.rados.rados.0.smithi100.stdout:3573: delete oid 17 current snap is 320 2024-04-07T16:27:57.155 INFO:tasks.rados.rados.0.smithi100.stdout:3570: finishing write tid 3 to smithi10066867-37 2024-04-07T16:27:57.155 INFO:tasks.rados.rados.0.smithi100.stdout:3570: finishing write tid 4 to smithi10066867-37 2024-04-07T16:27:57.155 INFO:tasks.rados.rados.0.smithi100.stdout:3570: finishing write tid 5 to smithi10066867-37 2024-04-07T16:27:57.155 INFO:tasks.rados.rados.0.smithi100.stdout:3570: finishing write tid 6 to smithi10066867-37 2024-04-07T16:27:57.155 INFO:tasks.rados.rados.0.smithi100.stdout:3570: oid 37 updating version 0 to 1570 2024-04-07T16:27:57.155 INFO:tasks.rados.rados.0.smithi100.stdout:3570: oid 37 updating version 1570 to 1573 2024-04-07T16:27:57.155 INFO:tasks.rados.rados.0.smithi100.stdout:3570: oid 37 version 1573 is already newer than 1569 2024-04-07T16:27:57.156 INFO:tasks.rados.rados.0.smithi100.stdout:3570: oid 37 version 1573 is already newer than 1571 2024-04-07T16:27:57.156 INFO:tasks.rados.rados.0.smithi100.stdout:3570: oid 37 version 1573 is already newer than 1572 2024-04-07T16:27:57.156 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 37 v 1573 (ObjNum 1103 snap 320 seq_num 1103) dirty exists 2024-04-07T16:27:57.156 INFO:tasks.rados.rados.0.smithi100.stdout:3570: left oid 37 (ObjNum 1103 snap 320 seq_num 1103) 2024-04-07T16:27:57.158 INFO:tasks.rados.rados.0.smithi100.stdout:3570: done (2 left) 2024-04-07T16:27:57.158 INFO:tasks.rados.rados.0.smithi100.stdout:3573: done (1 left) 2024-04-07T16:27:57.158 INFO:tasks.rados.rados.0.smithi100.stdout:3574: copy_from oid 7 from oid 32 current snap is 320 2024-04-07T16:27:57.158 INFO:tasks.rados.rados.0.smithi100.stdout:3575: read oid 10 snap -1 2024-04-07T16:27:57.158 INFO:tasks.rados.rados.0.smithi100.stdout:3575: expect (ObjNum 1062 snap 310 seq_num 1062) 2024-04-07T16:27:57.158 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 15 current snap is 320 2024-04-07T16:27:57.158 INFO:tasks.rados.rados.0.smithi100.stdout:3576: seq_num 1106 ranges {4186112=532480,4718592=483328,5201920=475136,5677056=245760} 2024-04-07T16:27:57.190 INFO:tasks.rados.rados.0.smithi100.stdout:3576: writing smithi10066867-15 from 4186112 to 4718592 tid 1 2024-04-07T16:27:57.194 INFO:tasks.rados.rados.0.smithi100.stdout:3576: writing smithi10066867-15 from 4718592 to 5201920 tid 2 2024-04-07T16:27:57.197 INFO:tasks.rados.rados.0.smithi100.stdout:3576: writing smithi10066867-15 from 5201920 to 5677056 tid 3 2024-04-07T16:27:57.199 INFO:tasks.rados.rados.0.smithi100.stdout:3576: writing smithi10066867-15 from 5677056 to 5922816 tid 4 2024-04-07T16:27:57.199 INFO:tasks.rados.rados.0.smithi100.stdout:3577: read oid 43 snap -1 2024-04-07T16:27:57.199 INFO:tasks.rados.rados.0.smithi100.stdout:3577: expect (ObjNum 1030 snap 306 seq_num 1030) 2024-04-07T16:27:57.200 INFO:tasks.rados.rados.0.smithi100.stdout:3578: snap_create 2024-04-07T16:27:57.200 INFO:tasks.rados.rados.0.smithi100.stdout:3574: finishing copy_from to smithi10066867-7 2024-04-07T16:27:57.200 INFO:tasks.rados.rados.0.smithi100.stdout:3574: got expected ENOENT (src dne) 2024-04-07T16:27:57.200 INFO:tasks.rados.rados.0.smithi100.stdout:3574: finishing copy_from racing read to smithi10066867-7 2024-04-07T16:27:57.200 INFO:tasks.rados.rados.0.smithi100.stdout:3572: expect (ObjNum 1009 snap 300 seq_num 1009) 2024-04-07T16:27:57.282 INFO:tasks.rados.rados.0.smithi100.stdout:3575: expect (ObjNum 1062 snap 310 seq_num 1062) 2024-04-07T16:27:57.309 INFO:tasks.rados.rados.0.smithi100.stdout:3576: finishing write tid 1 to smithi10066867-15 2024-04-07T16:27:57.309 INFO:tasks.rados.rados.0.smithi100.stdout:3576: finishing write tid 2 to smithi10066867-15 2024-04-07T16:27:57.309 INFO:tasks.rados.rados.0.smithi100.stdout:3576: finishing write tid 3 to smithi10066867-15 2024-04-07T16:27:57.309 INFO:tasks.rados.rados.0.smithi100.stdout:3576: finishing write tid 4 to smithi10066867-15 2024-04-07T16:27:57.309 INFO:tasks.rados.rados.0.smithi100.stdout:3576: finishing write tid 5 to smithi10066867-15 2024-04-07T16:27:57.309 INFO:tasks.rados.rados.0.smithi100.stdout:3577: expect (ObjNum 1030 snap 306 seq_num 1030) 2024-04-07T16:27:57.321 INFO:tasks.rados.rados.0.smithi100.stdout:3576: finishing write tid 6 to smithi10066867-15 2024-04-07T16:27:57.321 INFO:tasks.rados.rados.0.smithi100.stdout:3576: oid 15 updating version 0 to 1869 2024-04-07T16:27:57.321 INFO:tasks.rados.rados.0.smithi100.stdout:3576: oid 15 version 1869 is already newer than 1867 2024-04-07T16:27:57.321 INFO:tasks.rados.rados.0.smithi100.stdout:3576: oid 15 version 1869 is already newer than 1866 2024-04-07T16:27:57.321 INFO:tasks.rados.rados.0.smithi100.stdout:3576: oid 15 updating version 1869 to 1870 2024-04-07T16:27:57.321 INFO:tasks.rados.rados.0.smithi100.stdout:3576: oid 15 version 1870 is already newer than 1868 2024-04-07T16:27:57.321 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 15 v 1870 (ObjNum 1106 snap 320 seq_num 1106) dirty exists 2024-04-07T16:27:57.321 INFO:tasks.rados.rados.0.smithi100.stdout:3576: left oid 15 (ObjNum 1106 snap 320 seq_num 1106) 2024-04-07T16:27:57.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:56 smithi144 bash[66654]: cluster 2024-04-07T16:27:55.965075+0000 mon.a (mon.0) 1199 : cluster [DBG] osdmap e718: 8 total, 8 up, 8 in 2024-04-07T16:27:57.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:56 smithi144 bash[66654]: cluster 2024-04-07T16:27:56.940950+0000 mon.a (mon.0) 1200 : cluster [DBG] osdmap e719: 8 total, 8 up, 8 in 2024-04-07T16:27:57.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:56 smithi100 bash[74942]: cluster 2024-04-07T16:27:55.965075+0000 mon.a (mon.0) 1199 : cluster [DBG] osdmap e718: 8 total, 8 up, 8 in 2024-04-07T16:27:57.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:56 smithi100 bash[74942]: cluster 2024-04-07T16:27:56.940950+0000 mon.a (mon.0) 1200 : cluster [DBG] osdmap e719: 8 total, 8 up, 8 in 2024-04-07T16:27:57.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:56 smithi100 bash[76202]: cluster 2024-04-07T16:27:55.965075+0000 mon.a (mon.0) 1199 : cluster [DBG] osdmap e718: 8 total, 8 up, 8 in 2024-04-07T16:27:57.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:56 smithi100 bash[76202]: cluster 2024-04-07T16:27:56.940950+0000 mon.a (mon.0) 1200 : cluster [DBG] osdmap e719: 8 total, 8 up, 8 in 2024-04-07T16:27:58.013 INFO:tasks.rados.rados.0.smithi100.stdout:3572: done (5 left) 2024-04-07T16:27:58.013 INFO:tasks.rados.rados.0.smithi100.stdout:3574: done (4 left) 2024-04-07T16:27:58.013 INFO:tasks.rados.rados.0.smithi100.stdout:3575: done (3 left) 2024-04-07T16:27:58.013 INFO:tasks.rados.rados.0.smithi100.stdout:3576: done (2 left) 2024-04-07T16:27:58.013 INFO:tasks.rados.rados.0.smithi100.stdout:3577: done (1 left) 2024-04-07T16:27:58.013 INFO:tasks.rados.rados.0.smithi100.stdout:3578: done (0 left) 2024-04-07T16:27:58.013 INFO:tasks.rados.rados.0.smithi100.stdout:3579: delete oid 27 current snap is 321 2024-04-07T16:27:58.025 INFO:tasks.rados.rados.0.smithi100.stdout:3579: done (0 left) 2024-04-07T16:27:58.025 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 27 current snap is 321 2024-04-07T16:27:58.025 INFO:tasks.rados.rados.0.smithi100.stdout:3580: seq_num 1108 ranges {0=565248,565248=647168,1212416=483328,1695744=73728} 2024-04-07T16:27:58.031 INFO:tasks.rados.rados.0.smithi100.stdout:3580: writing smithi10066867-27 from 0 to 565248 tid 1 2024-04-07T16:27:58.038 INFO:tasks.rados.rados.0.smithi100.stdout:3580: writing smithi10066867-27 from 565248 to 1212416 tid 2 2024-04-07T16:27:58.043 INFO:tasks.rados.rados.0.smithi100.stdout:3580: writing smithi10066867-27 from 1212416 to 1695744 tid 3 2024-04-07T16:27:58.044 INFO:tasks.rados.rados.0.smithi100.stdout:3580: writing smithi10066867-27 from 1695744 to 1769472 tid 4 2024-04-07T16:27:58.044 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 36 current snap is 321 2024-04-07T16:27:58.044 INFO:tasks.rados.rados.0.smithi100.stdout:3581: seq_num 1109 ranges {1499136=581632,2080768=532480,2613248=647168,3260416=24576} 2024-04-07T16:27:58.066 INFO:tasks.rados.rados.0.smithi100.stdout:3581: writing smithi10066867-36 from 1499136 to 2080768 tid 1 2024-04-07T16:27:58.071 INFO:tasks.rados.rados.0.smithi100.stdout:3581: writing smithi10066867-36 from 2080768 to 2613248 tid 2 2024-04-07T16:27:58.077 INFO:tasks.rados.rados.0.smithi100.stdout:3581: writing smithi10066867-36 from 2613248 to 3260416 tid 3 2024-04-07T16:27:58.078 INFO:tasks.rados.rados.0.smithi100.stdout:3581: writing smithi10066867-36 from 3260416 to 3284992 tid 4 2024-04-07T16:27:58.078 INFO:tasks.rados.rados.0.smithi100.stdout:3582: delete oid 13 current snap is 321 2024-04-07T16:27:58.078 INFO:tasks.rados.rados.0.smithi100.stdout:3580: finishing write tid 1 to smithi10066867-27 2024-04-07T16:27:58.078 INFO:tasks.rados.rados.0.smithi100.stdout:3580: finishing write tid 2 to smithi10066867-27 2024-04-07T16:27:58.078 INFO:tasks.rados.rados.0.smithi100.stdout:3580: finishing write tid 3 to smithi10066867-27 2024-04-07T16:27:58.078 INFO:tasks.rados.rados.0.smithi100.stdout:3580: finishing write tid 4 to smithi10066867-27 2024-04-07T16:27:58.078 INFO:tasks.rados.rados.0.smithi100.stdout:3580: finishing write tid 5 to smithi10066867-27 2024-04-07T16:27:58.078 INFO:tasks.rados.rados.0.smithi100.stdout:3580: finishing write tid 6 to smithi10066867-27 2024-04-07T16:27:58.078 INFO:tasks.rados.rados.0.smithi100.stdout:3580: oid 27 updating version 0 to 1204 2024-04-07T16:27:58.078 INFO:tasks.rados.rados.0.smithi100.stdout:3580: oid 27 version 1204 is already newer than 1202 2024-04-07T16:27:58.078 INFO:tasks.rados.rados.0.smithi100.stdout:3580: oid 27 updating version 1204 to 1206 2024-04-07T16:27:58.078 INFO:tasks.rados.rados.0.smithi100.stdout:3580: oid 27 version 1206 is already newer than 1203 2024-04-07T16:27:58.079 INFO:tasks.rados.rados.0.smithi100.stdout:3580: oid 27 version 1206 is already newer than 1205 2024-04-07T16:27:58.079 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 27 v 1206 (ObjNum 1108 snap 321 seq_num 1108) dirty exists 2024-04-07T16:27:58.079 INFO:tasks.rados.rados.0.smithi100.stdout:3580: left oid 27 (ObjNum 1108 snap 321 seq_num 1108) 2024-04-07T16:27:58.079 INFO:tasks.rados.rados.0.smithi100.stdout:3581: finishing write tid 1 to smithi10066867-36 2024-04-07T16:27:58.079 INFO:tasks.rados.rados.0.smithi100.stdout:3581: finishing write tid 2 to smithi10066867-36 2024-04-07T16:27:58.082 INFO:tasks.rados.rados.0.smithi100.stdout:3580: done (2 left) 2024-04-07T16:27:58.082 INFO:tasks.rados.rados.0.smithi100.stdout:3582: done (1 left) 2024-04-07T16:27:58.082 INFO:tasks.rados.rados.0.smithi100.stdout:3583: read oid 6 snap -1 2024-04-07T16:27:58.082 INFO:tasks.rados.rados.0.smithi100.stdout:3583: expect (ObjNum 1049 snap 307 seq_num 1049) 2024-04-07T16:27:58.082 INFO:tasks.rados.rados.0.smithi100.stdout:3584: read oid 27 snap -1 2024-04-07T16:27:58.082 INFO:tasks.rados.rados.0.smithi100.stdout:3584: expect (ObjNum 1108 snap 321 seq_num 1108) 2024-04-07T16:27:58.082 INFO:tasks.rados.rados.0.smithi100.stdout:3585: delete oid 3 current snap is 321 2024-04-07T16:27:58.085 INFO:tasks.rados.rados.0.smithi100.stdout:3581: finishing write tid 3 to smithi10066867-36 2024-04-07T16:27:58.088 INFO:tasks.rados.rados.0.smithi100.stdout:3581: finishing write tid 4 to smithi10066867-36 2024-04-07T16:27:58.089 INFO:tasks.rados.rados.0.smithi100.stdout:3583: expect (ObjNum 1049 snap 307 seq_num 1049) 2024-04-07T16:27:58.127 INFO:tasks.rados.rados.0.smithi100.stdout:3581: finishing write tid 5 to smithi10066867-36 2024-04-07T16:27:58.127 INFO:tasks.rados.rados.0.smithi100.stdout:3581: finishing write tid 6 to smithi10066867-36 2024-04-07T16:27:58.127 INFO:tasks.rados.rados.0.smithi100.stdout:3581: oid 36 updating version 0 to 1279 2024-04-07T16:27:58.127 INFO:tasks.rados.rados.0.smithi100.stdout:3581: oid 36 updating version 1279 to 1282 2024-04-07T16:27:58.127 INFO:tasks.rados.rados.0.smithi100.stdout:3581: oid 36 version 1282 is already newer than 1281 2024-04-07T16:27:58.127 INFO:tasks.rados.rados.0.smithi100.stdout:3581: oid 36 updating version 1282 to 1283 2024-04-07T16:27:58.127 INFO:tasks.rados.rados.0.smithi100.stdout:3581: oid 36 version 1283 is already newer than 1280 2024-04-07T16:27:58.127 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 36 v 1283 (ObjNum 1109 snap 321 seq_num 1109) dirty exists 2024-04-07T16:27:58.127 INFO:tasks.rados.rados.0.smithi100.stdout:3581: left oid 36 (ObjNum 1109 snap 321 seq_num 1109) 2024-04-07T16:27:58.127 INFO:tasks.rados.rados.0.smithi100.stdout:3584: expect (ObjNum 1108 snap 321 seq_num 1108) 2024-04-07T16:27:58.184 INFO:tasks.rados.rados.0.smithi100.stdout:3581: done (3 left) 2024-04-07T16:27:58.184 INFO:tasks.rados.rados.0.smithi100.stdout:3583: done (2 left) 2024-04-07T16:27:58.184 INFO:tasks.rados.rados.0.smithi100.stdout:3584: done (1 left) 2024-04-07T16:27:58.185 INFO:tasks.rados.rados.0.smithi100.stdout:3585: done (0 left) 2024-04-07T16:27:58.185 INFO:tasks.rados.rados.0.smithi100.stdout:3586: copy_from oid 21 from oid 46 current snap is 321 2024-04-07T16:27:58.185 INFO:tasks.rados.rados.0.smithi100.stdout:3587: copy_from oid 48 from oid 23 current snap is 321 2024-04-07T16:27:58.185 INFO:tasks.rados.rados.0.smithi100.stdout:3588: rollback oid 35 current snap is 321 2024-04-07T16:27:58.185 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 35 to 320 2024-04-07T16:27:58.185 INFO:tasks.rados.rados.0.smithi100.stdout:3589: copy_from oid 30 from oid 45 current snap is 321 2024-04-07T16:27:58.185 INFO:tasks.rados.rados.0.smithi100.stdout:3590: read oid 25 snap -1 2024-04-07T16:27:58.185 INFO:tasks.rados.rados.0.smithi100.stdout:3590: expect (ObjNum 1082 snap 314 seq_num 1082) 2024-04-07T16:27:58.185 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 31 current snap is 321 2024-04-07T16:27:58.185 INFO:tasks.rados.rados.0.smithi100.stdout:3591: seq_num 1112 ranges {2859008=630784,3489792=638976,4128768=466944,4595712=98304} 2024-04-07T16:27:58.238 INFO:tasks.rados.rados.0.smithi100.stdout:3591: writing smithi10066867-31 from 2859008 to 3489792 tid 1 2024-04-07T16:27:58.246 INFO:tasks.rados.rados.0.smithi100.stdout:3591: writing smithi10066867-31 from 3489792 to 4128768 tid 2 2024-04-07T16:27:58.251 INFO:tasks.rados.rados.0.smithi100.stdout:3591: writing smithi10066867-31 from 4128768 to 4595712 tid 3 2024-04-07T16:27:58.252 INFO:tasks.rados.rados.0.smithi100.stdout:3591: writing smithi10066867-31 from 4595712 to 4694016 tid 4 2024-04-07T16:27:58.252 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 11 current snap is 321 2024-04-07T16:27:58.253 INFO:tasks.rados.rados.0.smithi100.stdout:3589: finishing copy_from to smithi10066867-30 2024-04-07T16:27:58.253 INFO:tasks.rados.rados.0.smithi100.stdout:3589: got expected ENOENT (src dne) 2024-04-07T16:27:58.253 INFO:tasks.rados.rados.0.smithi100.stdout:3589: finishing copy_from racing read to smithi10066867-30 2024-04-07T16:27:58.253 INFO:tasks.rados.rados.0.smithi100.stdout:3590: expect (ObjNum 1082 snap 314 seq_num 1082) 2024-04-07T16:27:58.302 INFO:tasks.rados.rados.0.smithi100.stdout:3587: finishing copy_from racing read to smithi10066867-48 2024-04-07T16:27:58.302 INFO:tasks.rados.rados.0.smithi100.stdout:3588: finishing rollback tid 0 to smithi10066867-35 2024-04-07T16:27:58.302 INFO:tasks.rados.rados.0.smithi100.stdout:3592: seq_num 1113 ranges {7831552=647168,8478720=524288,9003008=622592,9625600=57344} 2024-04-07T16:27:58.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:57 smithi144 bash[66654]: cluster 2024-04-07T16:27:56.804279+0000 mgr.y (mgr.25175) 367 : cluster [DBG] pgmap v671: 121 pgs: 2 active+clean+snaptrim, 119 active+clean; 157 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 5.3 MiB/s rd, 1.1 MiB/s wr, 21 op/s 2024-04-07T16:27:58.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:57 smithi144 bash[66654]: audit 2024-04-07T16:27:56.999760+0000 mon.a (mon.0) 1201 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:27:58.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:57 smithi144 bash[66654]: audit 2024-04-07T16:27:57.009747+0000 mon.a (mon.0) 1202 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:27:58.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:57 smithi144 bash[66654]: audit 2024-04-07T16:27:57.011797+0000 mon.a (mon.0) 1203 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:27:58.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:57 smithi144 bash[66654]: audit 2024-04-07T16:27:57.013060+0000 mon.a (mon.0) 1204 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:27:58.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:57 smithi144 bash[66654]: audit 2024-04-07T16:27:57.019357+0000 mon.a (mon.0) 1205 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:27:58.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:57 smithi144 bash[66654]: audit 2024-04-07T16:27:57.092432+0000 mon.a (mon.0) 1206 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-07T16:27:58.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:57 smithi144 bash[66654]: audit 2024-04-07T16:27:57.092879+0000 mgr.y (mgr.25175) 368 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-07T16:27:58.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:57 smithi144 bash[66654]: audit 2024-04-07T16:27:57.099602+0000 mon.a (mon.0) 1207 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:27:58.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:57 smithi144 bash[66654]: audit 2024-04-07T16:27:57.101988+0000 mon.a (mon.0) 1208 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:27:58.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:57 smithi144 bash[66654]: audit 2024-04-07T16:27:57.103817+0000 mon.a (mon.0) 1209 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:27:58.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:57 smithi144 bash[66654]: audit 2024-04-07T16:27:57.105005+0000 mon.a (mon.0) 1210 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:27:58.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:57 smithi144 bash[66654]: audit 2024-04-07T16:27:57.107412+0000 mon.a (mon.0) 1211 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:27:58.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:57 smithi144 bash[66654]: audit 2024-04-07T16:27:57.110056+0000 mon.a (mon.0) 1212 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:27:58.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:57 smithi144 bash[66654]: audit 2024-04-07T16:27:57.111237+0000 mon.a (mon.0) 1213 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:27:58.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:57 smithi144 bash[66654]: audit 2024-04-07T16:27:57.112422+0000 mon.a (mon.0) 1214 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:27:58.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:57 smithi144 bash[66654]: audit 2024-04-07T16:27:57.113559+0000 mon.a (mon.0) 1215 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:27:58.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:57 smithi144 bash[66654]: audit 2024-04-07T16:27:57.114684+0000 mon.a (mon.0) 1216 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:27:58.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:57 smithi144 bash[66654]: audit 2024-04-07T16:27:57.115869+0000 mon.a (mon.0) 1217 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:27:58.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:57 smithi144 bash[66654]: audit 2024-04-07T16:27:57.117050+0000 mon.a (mon.0) 1218 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:27:58.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:57 smithi144 bash[66654]: audit 2024-04-07T16:27:57.118346+0000 mon.a (mon.0) 1219 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:27:58.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:57 smithi144 bash[66654]: audit 2024-04-07T16:27:57.120047+0000 mon.a (mon.0) 1220 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:27:58.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:57 smithi144 bash[66654]: audit 2024-04-07T16:27:57.121448+0000 mon.a (mon.0) 1221 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:27:58.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:57 smithi144 bash[66654]: audit 2024-04-07T16:27:57.122828+0000 mon.a (mon.0) 1222 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:27:58.376 INFO:tasks.rados.rados.0.smithi100.stdout:3592: writing smithi10066867-11 from 7831552 to 8478720 tid 1 2024-04-07T16:27:58.379 INFO:tasks.rados.rados.0.smithi100.stdout:3592: writing smithi10066867-11 from 8478720 to 9003008 tid 2 2024-04-07T16:27:58.383 INFO:tasks.rados.rados.0.smithi100.stdout:3592: writing smithi10066867-11 from 9003008 to 9625600 tid 3 2024-04-07T16:27:58.384 INFO:tasks.rados.rados.0.smithi100.stdout:3592: writing smithi10066867-11 from 9625600 to 9682944 tid 4 2024-04-07T16:27:58.384 INFO:tasks.rados.rados.0.smithi100.stdout:3589: done (6 left) 2024-04-07T16:27:58.384 INFO:tasks.rados.rados.0.smithi100.stdout:3590: done (5 left) 2024-04-07T16:27:58.384 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 47 current snap is 321 2024-04-07T16:27:58.384 INFO:tasks.rados.rados.0.smithi100.stdout:3593: seq_num 1114 ranges {4120576=663552,4784128=802816,5586944=401408} 2024-04-07T16:27:58.411 INFO:tasks.rados.rados.0.smithi100.stdout:3593: writing smithi10066867-47 from 4120576 to 4784128 tid 1 2024-04-07T16:27:58.416 INFO:tasks.rados.rados.0.smithi100.stdout:3593: writing smithi10066867-47 from 4784128 to 5586944 tid 2 2024-04-07T16:27:58.418 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:58 smithi100 bash[76202]: cluster 2024-04-07T16:27:56.804279+0000 mgr.y (mgr.25175) 367 : cluster [DBG] pgmap v671: 121 pgs: 2 active+clean+snaptrim, 119 active+clean; 157 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 5.3 MiB/s rd, 1.1 MiB/s wr, 21 op/s 2024-04-07T16:27:58.418 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:58 smithi100 bash[76202]: audit 2024-04-07T16:27:56.999760+0000 mon.a (mon.0) 1201 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:27:58.418 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:58 smithi100 bash[76202]: audit 2024-04-07T16:27:57.009747+0000 mon.a (mon.0) 1202 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:27:58.418 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:58 smithi100 bash[76202]: audit 2024-04-07T16:27:57.011797+0000 mon.a (mon.0) 1203 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:27:58.418 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:58 smithi100 bash[76202]: audit 2024-04-07T16:27:57.013060+0000 mon.a (mon.0) 1204 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:27:58.418 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:58 smithi100 bash[76202]: audit 2024-04-07T16:27:57.019357+0000 mon.a (mon.0) 1205 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:27:58.418 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:58 smithi100 bash[76202]: audit 2024-04-07T16:27:57.092432+0000 mon.a (mon.0) 1206 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-07T16:27:58.418 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:58 smithi100 bash[76202]: audit 2024-04-07T16:27:57.092879+0000 mgr.y (mgr.25175) 368 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-07T16:27:58.418 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:58 smithi100 bash[76202]: audit 2024-04-07T16:27:57.099602+0000 mon.a (mon.0) 1207 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:27:58.419 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:58 smithi100 bash[76202]: audit 2024-04-07T16:27:57.101988+0000 mon.a (mon.0) 1208 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:27:58.419 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:58 smithi100 bash[76202]: audit 2024-04-07T16:27:57.103817+0000 mon.a (mon.0) 1209 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:27:58.419 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:58 smithi100 bash[76202]: audit 2024-04-07T16:27:57.105005+0000 mon.a (mon.0) 1210 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:27:58.419 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:58 smithi100 bash[76202]: audit 2024-04-07T16:27:57.107412+0000 mon.a (mon.0) 1211 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:27:58.419 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:58 smithi100 bash[76202]: audit 2024-04-07T16:27:57.110056+0000 mon.a (mon.0) 1212 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:27:58.419 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:58 smithi100 bash[76202]: audit 2024-04-07T16:27:57.111237+0000 mon.a (mon.0) 1213 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:27:58.419 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:58 smithi100 bash[76202]: audit 2024-04-07T16:27:57.112422+0000 mon.a (mon.0) 1214 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:27:58.419 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:58 smithi100 bash[76202]: audit 2024-04-07T16:27:57.113559+0000 mon.a (mon.0) 1215 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:27:58.419 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:58 smithi100 bash[76202]: audit 2024-04-07T16:27:57.114684+0000 mon.a (mon.0) 1216 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:27:58.419 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:58 smithi100 bash[76202]: audit 2024-04-07T16:27:57.115869+0000 mon.a (mon.0) 1217 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:27:58.419 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:58 smithi100 bash[76202]: audit 2024-04-07T16:27:57.117050+0000 mon.a (mon.0) 1218 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:27:58.419 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:58 smithi100 bash[76202]: audit 2024-04-07T16:27:57.118346+0000 mon.a (mon.0) 1219 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:27:58.419 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:58 smithi100 bash[76202]: audit 2024-04-07T16:27:57.120047+0000 mon.a (mon.0) 1220 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:27:58.419 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:58 smithi100 bash[76202]: audit 2024-04-07T16:27:57.121448+0000 mon.a (mon.0) 1221 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:27:58.420 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:58 smithi100 bash[76202]: audit 2024-04-07T16:27:57.122828+0000 mon.a (mon.0) 1222 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:27:58.420 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:57 smithi100 bash[74942]: cluster 2024-04-07T16:27:56.804279+0000 mgr.y (mgr.25175) 367 : cluster [DBG] pgmap v671: 121 pgs: 2 active+clean+snaptrim, 119 active+clean; 157 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 5.3 MiB/s rd, 1.1 MiB/s wr, 21 op/s 2024-04-07T16:27:58.420 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:57 smithi100 bash[74942]: audit 2024-04-07T16:27:56.999760+0000 mon.a (mon.0) 1201 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:27:58.420 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:58 smithi100 bash[74942]: audit 2024-04-07T16:27:57.009747+0000 mon.a (mon.0) 1202 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:27:58.420 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:58 smithi100 bash[74942]: audit 2024-04-07T16:27:57.011797+0000 mon.a (mon.0) 1203 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:27:58.420 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:58 smithi100 bash[74942]: audit 2024-04-07T16:27:57.013060+0000 mon.a (mon.0) 1204 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:27:58.420 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:58 smithi100 bash[74942]: audit 2024-04-07T16:27:57.019357+0000 mon.a (mon.0) 1205 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:27:58.420 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:58 smithi100 bash[74942]: audit 2024-04-07T16:27:57.092432+0000 mon.a (mon.0) 1206 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-07T16:27:58.421 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:58 smithi100 bash[74942]: audit 2024-04-07T16:27:57.092879+0000 mgr.y (mgr.25175) 368 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-07T16:27:58.421 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:58 smithi100 bash[74942]: audit 2024-04-07T16:27:57.099602+0000 mon.a (mon.0) 1207 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:27:58.421 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:58 smithi100 bash[74942]: audit 2024-04-07T16:27:57.101988+0000 mon.a (mon.0) 1208 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:27:58.421 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:58 smithi100 bash[74942]: audit 2024-04-07T16:27:57.103817+0000 mon.a (mon.0) 1209 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:27:58.421 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:58 smithi100 bash[74942]: audit 2024-04-07T16:27:57.105005+0000 mon.a (mon.0) 1210 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:27:58.421 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:58 smithi100 bash[74942]: audit 2024-04-07T16:27:57.107412+0000 mon.a (mon.0) 1211 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:27:58.421 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:58 smithi100 bash[74942]: audit 2024-04-07T16:27:57.110056+0000 mon.a (mon.0) 1212 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:27:58.421 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:58 smithi100 bash[74942]: audit 2024-04-07T16:27:57.111237+0000 mon.a (mon.0) 1213 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:27:58.421 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:58 smithi100 bash[74942]: audit 2024-04-07T16:27:57.112422+0000 mon.a (mon.0) 1214 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:27:58.421 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:58 smithi100 bash[74942]: audit 2024-04-07T16:27:57.113559+0000 mon.a (mon.0) 1215 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:27:58.421 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:58 smithi100 bash[74942]: audit 2024-04-07T16:27:57.114684+0000 mon.a (mon.0) 1216 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:27:58.421 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:58 smithi100 bash[74942]: audit 2024-04-07T16:27:57.115869+0000 mon.a (mon.0) 1217 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:27:58.421 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:58 smithi100 bash[74942]: audit 2024-04-07T16:27:57.117050+0000 mon.a (mon.0) 1218 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:27:58.421 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:58 smithi100 bash[74942]: audit 2024-04-07T16:27:57.118346+0000 mon.a (mon.0) 1219 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:27:58.422 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:58 smithi100 bash[74942]: audit 2024-04-07T16:27:57.120047+0000 mon.a (mon.0) 1220 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:27:58.422 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:58 smithi100 bash[74942]: audit 2024-04-07T16:27:57.121448+0000 mon.a (mon.0) 1221 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:27:58.422 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:58 smithi100 bash[74942]: audit 2024-04-07T16:27:57.122828+0000 mon.a (mon.0) 1222 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:27:58.424 INFO:tasks.rados.rados.0.smithi100.stdout:3593: writing smithi10066867-47 from 5586944 to 5988352 tid 3 2024-04-07T16:27:58.425 INFO:tasks.rados.rados.0.smithi100.stdout:3594: delete oid 6 current snap is 321 2024-04-07T16:27:58.425 INFO:tasks.rados.rados.0.smithi100.stdout:3588: finishing rollback tid 1 to smithi10066867-35 2024-04-07T16:27:58.425 INFO:tasks.rados.rados.0.smithi100.stdout:3588: finishing rollback tid 2 to smithi10066867-35 2024-04-07T16:27:58.425 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 35 v 1720 (ObjNum 1102 snap 320 seq_num 1102) dirty exists 2024-04-07T16:27:58.425 INFO:tasks.rados.rados.0.smithi100.stdout:3587: finishing copy_from to smithi10066867-48 2024-04-07T16:27:58.425 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 48 v 1686 (ObjNum 941 snap 282 seq_num 941) dirty exists 2024-04-07T16:27:58.425 INFO:tasks.rados.rados.0.smithi100.stdout:3586: finishing copy_from racing read to smithi10066867-21 2024-04-07T16:27:58.425 INFO:tasks.rados.rados.0.smithi100.stdout:3586: finishing copy_from to smithi10066867-21 2024-04-07T16:27:58.425 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 21 v 1583 (ObjNum 1097 snap 320 seq_num 1097) dirty exists 2024-04-07T16:27:58.425 INFO:tasks.rados.rados.0.smithi100.stdout:3591: finishing write tid 1 to smithi10066867-31 2024-04-07T16:27:58.425 INFO:tasks.rados.rados.0.smithi100.stdout:3591: finishing write tid 2 to smithi10066867-31 2024-04-07T16:27:58.425 INFO:tasks.rados.rados.0.smithi100.stdout:3591: finishing write tid 3 to smithi10066867-31 2024-04-07T16:27:58.425 INFO:tasks.rados.rados.0.smithi100.stdout:3591: finishing write tid 4 to smithi10066867-31 2024-04-07T16:27:58.425 INFO:tasks.rados.rados.0.smithi100.stdout:3591: finishing write tid 5 to smithi10066867-31 2024-04-07T16:27:58.426 INFO:tasks.rados.rados.0.smithi100.stdout:3591: finishing write tid 6 to smithi10066867-31 2024-04-07T16:27:58.426 INFO:tasks.rados.rados.0.smithi100.stdout:3591: oid 31 updating version 0 to 1724 2024-04-07T16:27:58.426 INFO:tasks.rados.rados.0.smithi100.stdout:3591: oid 31 version 1724 is already newer than 1721 2024-04-07T16:27:58.426 INFO:tasks.rados.rados.0.smithi100.stdout:3591: oid 31 version 1724 is already newer than 1722 2024-04-07T16:27:58.426 INFO:tasks.rados.rados.0.smithi100.stdout:3591: oid 31 version 1724 is already newer than 1723 2024-04-07T16:27:58.426 INFO:tasks.rados.rados.0.smithi100.stdout:3591: oid 31 updating version 1724 to 1725 2024-04-07T16:27:58.426 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 31 v 1725 (ObjNum 1112 snap 321 seq_num 1112) dirty exists 2024-04-07T16:27:58.426 INFO:tasks.rados.rados.0.smithi100.stdout:3591: left oid 31 (ObjNum 1112 snap 321 seq_num 1112) 2024-04-07T16:27:58.426 INFO:tasks.rados.rados.0.smithi100.stdout:3592: finishing write tid 1 to smithi10066867-11 2024-04-07T16:27:58.426 INFO:tasks.rados.rados.0.smithi100.stdout:3592: finishing write tid 2 to smithi10066867-11 2024-04-07T16:27:58.426 INFO:tasks.rados.rados.0.smithi100.stdout:3592: finishing write tid 3 to smithi10066867-11 2024-04-07T16:27:58.426 INFO:tasks.rados.rados.0.smithi100.stdout:3592: finishing write tid 4 to smithi10066867-11 2024-04-07T16:27:58.426 INFO:tasks.rados.rados.0.smithi100.stdout:3592: finishing write tid 5 to smithi10066867-11 2024-04-07T16:27:58.426 INFO:tasks.rados.rados.0.smithi100.stdout:3592: finishing write tid 6 to smithi10066867-11 2024-04-07T16:27:58.427 INFO:tasks.rados.rados.0.smithi100.stdout:3592: oid 11 updating version 0 to 1873 2024-04-07T16:27:58.427 INFO:tasks.rados.rados.0.smithi100.stdout:3592: oid 11 updating version 1873 to 1875 2024-04-07T16:27:58.427 INFO:tasks.rados.rados.0.smithi100.stdout:3592: oid 11 version 1875 is already newer than 1874 2024-04-07T16:27:58.427 INFO:tasks.rados.rados.0.smithi100.stdout:3592: oid 11 updating version 1875 to 1876 2024-04-07T16:27:58.427 INFO:tasks.rados.rados.0.smithi100.stdout:3592: oid 11 version 1876 is already newer than 1872 2024-04-07T16:27:58.427 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 11 v 1876 (ObjNum 1113 snap 321 seq_num 1113) dirty exists 2024-04-07T16:27:58.427 INFO:tasks.rados.rados.0.smithi100.stdout:3592: left oid 11 (ObjNum 1113 snap 321 seq_num 1113) 2024-04-07T16:27:58.427 INFO:tasks.rados.rados.0.smithi100.stdout:3593: finishing write tid 1 to smithi10066867-47 2024-04-07T16:27:58.427 INFO:tasks.rados.rados.0.smithi100.stdout:3586: done (6 left) 2024-04-07T16:27:58.427 INFO:tasks.rados.rados.0.smithi100.stdout:3587: done (5 left) 2024-04-07T16:27:58.427 INFO:tasks.rados.rados.0.smithi100.stdout:3588: done (4 left) 2024-04-07T16:27:58.427 INFO:tasks.rados.rados.0.smithi100.stdout:3591: done (3 left) 2024-04-07T16:27:58.427 INFO:tasks.rados.rados.0.smithi100.stdout:3592: done (2 left) 2024-04-07T16:27:58.427 INFO:tasks.rados.rados.0.smithi100.stdout:3594: done (1 left) 2024-04-07T16:27:58.427 INFO:tasks.rados.rados.0.smithi100.stdout:3595: snap_create 2024-04-07T16:27:58.429 INFO:tasks.rados.rados.0.smithi100.stdout:3593: finishing write tid 2 to smithi10066867-47 2024-04-07T16:27:58.430 INFO:tasks.rados.rados.0.smithi100.stdout:3593: finishing write tid 3 to smithi10066867-47 2024-04-07T16:27:58.430 INFO:tasks.rados.rados.0.smithi100.stdout:3593: finishing write tid 4 to smithi10066867-47 2024-04-07T16:27:58.434 INFO:tasks.rados.rados.0.smithi100.stdout:3593: finishing write tid 5 to smithi10066867-47 2024-04-07T16:27:58.434 INFO:tasks.rados.rados.0.smithi100.stdout:3593: oid 47 updating version 0 to 1587 2024-04-07T16:27:58.434 INFO:tasks.rados.rados.0.smithi100.stdout:3593: oid 47 version 1587 is already newer than 1585 2024-04-07T16:27:58.434 INFO:tasks.rados.rados.0.smithi100.stdout:3593: oid 47 version 1587 is already newer than 1586 2024-04-07T16:27:58.434 INFO:tasks.rados.rados.0.smithi100.stdout:3593: oid 47 version 1587 is already newer than 1584 2024-04-07T16:27:58.434 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 47 v 1587 (ObjNum 1114 snap 321 seq_num 1114) dirty exists 2024-04-07T16:27:58.434 INFO:tasks.rados.rados.0.smithi100.stdout:3593: left oid 47 (ObjNum 1114 snap 321 seq_num 1114) 2024-04-07T16:27:58.687 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:27:58 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:27:58] "GET /metrics HTTP/1.1" 200 36345 "" "Prometheus/2.43.0" 2024-04-07T16:27:59.025 INFO:tasks.rados.rados.0.smithi100.stdout:3593: done (1 left) 2024-04-07T16:27:59.025 INFO:tasks.rados.rados.0.smithi100.stdout:3595: done (0 left) 2024-04-07T16:27:59.025 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 20 current snap is 322 2024-04-07T16:27:59.025 INFO:tasks.rados.rados.0.smithi100.stdout:3596: seq_num 1116 ranges {3301376=696320,3997696=573440,4571136=450560,5021696=180224} 2024-04-07T16:27:59.049 INFO:tasks.rados.rados.0.smithi100.stdout:3596: writing smithi10066867-20 from 3301376 to 3997696 tid 1 2024-04-07T16:27:59.052 INFO:tasks.rados.rados.0.smithi100.stdout:3596: writing smithi10066867-20 from 3997696 to 4571136 tid 2 2024-04-07T16:27:59.055 INFO:tasks.rados.rados.0.smithi100.stdout:3596: writing smithi10066867-20 from 4571136 to 5021696 tid 3 2024-04-07T16:27:59.057 INFO:tasks.rados.rados.0.smithi100.stdout:3596: writing smithi10066867-20 from 5021696 to 5201920 tid 4 2024-04-07T16:27:59.057 INFO:tasks.rados.rados.0.smithi100.stdout:3597: read oid 7 snap -1 2024-04-07T16:27:59.057 INFO:tasks.rados.rados.0.smithi100.stdout:3597: expect (ObjNum 1087 snap 318 seq_num 1087) 2024-04-07T16:27:59.058 INFO:tasks.rados.rados.0.smithi100.stdout:3598: rmattr oid 13 current snap is 322 2024-04-07T16:27:59.058 INFO:tasks.rados.rados.0.smithi100.stdout:3598: done (2 left) 2024-04-07T16:27:59.058 INFO:tasks.rados.rados.0.smithi100.stdout:3599: copy_from oid 15 from oid 13 current snap is 322 2024-04-07T16:27:59.058 INFO:tasks.rados.rados.0.smithi100.stdout:3600: setattr oid 2 current snap is 322 2024-04-07T16:27:59.059 INFO:tasks.rados.rados.0.smithi100.stdout:3601: snap_remove snap 320 2024-04-07T16:27:59.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:27:59 smithi144 bash[66654]: cluster 2024-04-07T16:27:58.026074+0000 mon.a (mon.0) 1223 : cluster [DBG] osdmap e720: 8 total, 8 up, 8 in 2024-04-07T16:27:59.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:27:59 smithi100 bash[74942]: cluster 2024-04-07T16:27:58.026074+0000 mon.a (mon.0) 1223 : cluster [DBG] osdmap e720: 8 total, 8 up, 8 in 2024-04-07T16:27:59.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:27:59 smithi100 bash[76202]: cluster 2024-04-07T16:27:58.026074+0000 mon.a (mon.0) 1223 : cluster [DBG] osdmap e720: 8 total, 8 up, 8 in 2024-04-07T16:28:00.032 INFO:tasks.rados.rados.0.smithi100.stdout:3601: done (4 left) 2024-04-07T16:28:00.032 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 12 current snap is 322 2024-04-07T16:28:00.032 INFO:tasks.rados.rados.0.smithi100.stdout:3602: seq_num 1117 ranges {0=712704,712704=458752,1171456=655360,1826816=90112} 2024-04-07T16:28:00.032 INFO:tasks.rados.rados.0.smithi100.stdout:3602: writing smithi10066867-12 from 0 to 712704 tid 1 2024-04-07T16:28:00.034 INFO:tasks.rados.rados.0.smithi100.stdout:3602: writing smithi10066867-12 from 712704 to 1171456 tid 2 2024-04-07T16:28:00.038 INFO:tasks.rados.rados.0.smithi100.stdout:3602: writing smithi10066867-12 from 1171456 to 1826816 tid 3 2024-04-07T16:28:00.038 INFO:tasks.rados.rados.0.smithi100.stdout:3602: writing smithi10066867-12 from 1826816 to 1916928 tid 4 2024-04-07T16:28:00.038 INFO:tasks.rados.rados.0.smithi100.stdout:3603: snap_remove snap 321 2024-04-07T16:28:00.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:00 smithi144 bash[66654]: cluster 2024-04-07T16:27:58.805132+0000 mgr.y (mgr.25175) 369 : cluster [DBG] pgmap v674: 121 pgs: 2 active+clean+snaptrim, 119 active+clean; 132 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 339 KiB/s rd, 1.7 MiB/s wr, 12 op/s 2024-04-07T16:28:00.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:00 smithi144 bash[66654]: cluster 2024-04-07T16:27:59.039124+0000 mon.a (mon.0) 1224 : cluster [DBG] osdmap e721: 8 total, 8 up, 8 in 2024-04-07T16:28:00.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:00 smithi100 bash[76202]: cluster 2024-04-07T16:27:58.805132+0000 mgr.y (mgr.25175) 369 : cluster [DBG] pgmap v674: 121 pgs: 2 active+clean+snaptrim, 119 active+clean; 132 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 339 KiB/s rd, 1.7 MiB/s wr, 12 op/s 2024-04-07T16:28:00.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:00 smithi100 bash[76202]: cluster 2024-04-07T16:27:59.039124+0000 mon.a (mon.0) 1224 : cluster [DBG] osdmap e721: 8 total, 8 up, 8 in 2024-04-07T16:28:00.439 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:00 smithi100 bash[74942]: cluster 2024-04-07T16:27:58.805132+0000 mgr.y (mgr.25175) 369 : cluster [DBG] pgmap v674: 121 pgs: 2 active+clean+snaptrim, 119 active+clean; 132 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 339 KiB/s rd, 1.7 MiB/s wr, 12 op/s 2024-04-07T16:28:00.439 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:00 smithi100 bash[74942]: cluster 2024-04-07T16:27:59.039124+0000 mon.a (mon.0) 1224 : cluster [DBG] osdmap e721: 8 total, 8 up, 8 in 2024-04-07T16:28:01.049 INFO:tasks.rados.rados.0.smithi100.stdout:3603: done (5 left) 2024-04-07T16:28:01.049 INFO:tasks.rados.rados.0.smithi100.stdout:3604: read oid 41 snap -1 2024-04-07T16:28:01.049 INFO:tasks.rados.rados.0.smithi100.stdout:3604: expect (ObjNum 0 snap 0 seq_num 0) 2024-04-07T16:28:01.049 INFO:tasks.rados.rados.0.smithi100.stdout:3605: copy_from oid 43 from oid 24 current snap is 322 2024-04-07T16:28:01.049 INFO:tasks.rados.rados.0.smithi100.stdout:3599: finishing copy_from to smithi10066867-15 2024-04-07T16:28:01.049 INFO:tasks.rados.rados.0.smithi100.stdout:3599: got expected ENOENT (src dne) 2024-04-07T16:28:01.049 INFO:tasks.rados.rados.0.smithi100.stdout:3599: finishing copy_from racing read to smithi10066867-15 2024-04-07T16:28:01.049 INFO:tasks.rados.rados.0.smithi100.stdout:3596: finishing write tid 1 to smithi10066867-20 2024-04-07T16:28:01.049 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 2 v 1663 (ObjNum 1101 snap 320 seq_num 1101) dirty exists 2024-04-07T16:28:01.049 INFO:tasks.rados.rados.0.smithi100.stdout:3596: finishing write tid 2 to smithi10066867-20 2024-04-07T16:28:01.050 INFO:tasks.rados.rados.0.smithi100.stdout:3596: finishing write tid 3 to smithi10066867-20 2024-04-07T16:28:01.050 INFO:tasks.rados.rados.0.smithi100.stdout:3596: finishing write tid 4 to smithi10066867-20 2024-04-07T16:28:01.050 INFO:tasks.rados.rados.0.smithi100.stdout:3596: finishing write tid 5 to smithi10066867-20 2024-04-07T16:28:01.050 INFO:tasks.rados.rados.0.smithi100.stdout:3597: expect (ObjNum 1087 snap 318 seq_num 1087) 2024-04-07T16:28:01.316 INFO:tasks.rados.rados.0.smithi100.stdout:3596: finishing write tid 6 to smithi10066867-20 2024-04-07T16:28:01.317 INFO:tasks.rados.rados.0.smithi100.stdout:3596: oid 20 updating version 0 to 1717 2024-04-07T16:28:01.317 INFO:tasks.rados.rados.0.smithi100.stdout:3596: oid 20 updating version 1717 to 1718 2024-04-07T16:28:01.317 INFO:tasks.rados.rados.0.smithi100.stdout:3596: oid 20 updating version 1718 to 1719 2024-04-07T16:28:01.317 INFO:tasks.rados.rados.0.smithi100.stdout:3596: oid 20 version 1719 is already newer than 1716 2024-04-07T16:28:01.317 INFO:tasks.rados.rados.0.smithi100.stdout:3596: oid 20 version 1719 is already newer than 1715 2024-04-07T16:28:01.317 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 20 v 1719 (ObjNum 1116 snap 322 seq_num 1116) dirty exists 2024-04-07T16:28:01.317 INFO:tasks.rados.rados.0.smithi100.stdout:3596: left oid 20 (ObjNum 1116 snap 322 seq_num 1116) 2024-04-07T16:28:01.317 INFO:tasks.rados.rados.0.smithi100.stdout:3602: finishing write tid 1 to smithi10066867-12 2024-04-07T16:28:01.317 INFO:tasks.rados.rados.0.smithi100.stdout:3602: finishing write tid 2 to smithi10066867-12 2024-04-07T16:28:01.317 INFO:tasks.rados.rados.0.smithi100.stdout:3602: finishing write tid 3 to smithi10066867-12 2024-04-07T16:28:01.317 INFO:tasks.rados.rados.0.smithi100.stdout:3602: finishing write tid 4 to smithi10066867-12 2024-04-07T16:28:01.317 INFO:tasks.rados.rados.0.smithi100.stdout:3602: finishing write tid 5 to smithi10066867-12 2024-04-07T16:28:01.317 INFO:tasks.rados.rados.0.smithi100.stdout:3602: finishing write tid 6 to smithi10066867-12 2024-04-07T16:28:01.318 INFO:tasks.rados.rados.0.smithi100.stdout:3602: oid 12 updating version 0 to 1716 2024-04-07T16:28:01.318 INFO:tasks.rados.rados.0.smithi100.stdout:3602: oid 12 version 1716 is already newer than 1715 2024-04-07T16:28:01.318 INFO:tasks.rados.rados.0.smithi100.stdout:3602: oid 12 updating version 1716 to 1717 2024-04-07T16:28:01.318 INFO:tasks.rados.rados.0.smithi100.stdout:3602: oid 12 version 1717 is already newer than 1714 2024-04-07T16:28:01.318 INFO:tasks.rados.rados.0.smithi100.stdout:3602: oid 12 updating version 1717 to 1718 2024-04-07T16:28:01.318 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 12 v 1718 (ObjNum 1117 snap 322 seq_num 1117) dirty exists 2024-04-07T16:28:01.318 INFO:tasks.rados.rados.0.smithi100.stdout:3602: left oid 12 (ObjNum 1117 snap 322 seq_num 1117) 2024-04-07T16:28:01.318 INFO:tasks.rados.rados.0.smithi100.stdout:3596: done (6 left) 2024-04-07T16:28:01.318 INFO:tasks.rados.rados.0.smithi100.stdout:3597: done (5 left) 2024-04-07T16:28:01.318 INFO:tasks.rados.rados.0.smithi100.stdout:3599: done (4 left) 2024-04-07T16:28:01.318 INFO:tasks.rados.rados.0.smithi100.stdout:3600: done (3 left) 2024-04-07T16:28:01.318 INFO:tasks.rados.rados.0.smithi100.stdout:3602: done (2 left) 2024-04-07T16:28:01.318 INFO:tasks.rados.rados.0.smithi100.stdout:3606: copy_from oid 21 from oid 4 current snap is 322 2024-04-07T16:28:01.319 INFO:tasks.rados.rados.0.smithi100.stdout:3604: expect (ObjNum 0 snap 0 seq_num 0) 2024-04-07T16:28:01.319 INFO:tasks.rados.rados.0.smithi100.stdout:3605: finishing copy_from racing read to smithi10066867-43 2024-04-07T16:28:01.319 INFO:tasks.rados.rados.0.smithi100.stdout:3605: finishing copy_from to smithi10066867-43 2024-04-07T16:28:01.319 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 43 v 1269 (ObjNum 1030 snap 306 seq_num 1030) dirty exists 2024-04-07T16:28:01.320 INFO:tasks.rados.rados.0.smithi100.stdout:3604: done (2 left) 2024-04-07T16:28:01.320 INFO:tasks.rados.rados.0.smithi100.stdout:3605: done (1 left) 2024-04-07T16:28:01.320 INFO:tasks.rados.rados.0.smithi100.stdout:3607: read oid 48 snap -1 2024-04-07T16:28:01.320 INFO:tasks.rados.rados.0.smithi100.stdout:3607: expect (ObjNum 941 snap 282 seq_num 941) 2024-04-07T16:28:01.320 INFO:tasks.rados.rados.0.smithi100.stdout:3608: rmattr oid 19 current snap is 322 2024-04-07T16:28:01.320 INFO:tasks.rados.rados.0.smithi100.stdout:3608: done (2 left) 2024-04-07T16:28:01.320 INFO:tasks.rados.rados.0.smithi100.stdout:3609: delete oid 25 current snap is 322 2024-04-07T16:28:01.322 INFO:tasks.rados.rados.0.smithi100.stdout:3609: done (2 left) 2024-04-07T16:28:01.322 INFO:tasks.rados.rados.0.smithi100.stdout:3610: delete oid 33 current snap is 322 2024-04-07T16:28:01.327 INFO:tasks.rados.rados.0.smithi100.stdout:3610: done (2 left) 2024-04-07T16:28:01.327 INFO:tasks.rados.rados.0.smithi100.stdout:3611: delete oid 8 current snap is 322 2024-04-07T16:28:01.328 INFO:tasks.rados.rados.0.smithi100.stdout:3606: finishing copy_from racing read to smithi10066867-21 2024-04-07T16:28:01.330 INFO:tasks.rados.rados.0.smithi100.stdout:3607: expect (ObjNum 941 snap 282 seq_num 941) 2024-04-07T16:28:01.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:01 smithi144 bash[66654]: cephadm 2024-04-07T16:27:59.543755+0000 mgr.y (mgr.25175) 370 : cephadm [INF] Upgrade: Updating grafana.a 2024-04-07T16:28:01.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:01 smithi144 bash[66654]: cephadm 2024-04-07T16:27:59.590398+0000 mgr.y (mgr.25175) 371 : cephadm [INF] Deploying daemon grafana.a on smithi144 2024-04-07T16:28:01.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:01 smithi144 bash[66654]: cluster 2024-04-07T16:28:00.038901+0000 mon.a (mon.0) 1225 : cluster [DBG] osdmap e722: 8 total, 8 up, 8 in 2024-04-07T16:28:01.366 INFO:tasks.rados.rados.0.smithi100.stdout:3606: finishing copy_from to smithi10066867-21 2024-04-07T16:28:01.366 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 21 v 1813 (ObjNum 1041 snap 307 seq_num 1041) dirty exists 2024-04-07T16:28:01.367 INFO:tasks.rados.rados.0.smithi100.stdout:3606: done (2 left) 2024-04-07T16:28:01.367 INFO:tasks.rados.rados.0.smithi100.stdout:3607: done (1 left) 2024-04-07T16:28:01.367 INFO:tasks.rados.rados.0.smithi100.stdout:3611: done (0 left) 2024-04-07T16:28:01.367 INFO:tasks.rados.rados.0.smithi100.stdout:3612: rmattr oid 49 current snap is 322 2024-04-07T16:28:01.367 INFO:tasks.rados.rados.0.smithi100.stdout:3613: setattr oid 14 current snap is 322 2024-04-07T16:28:01.370 INFO:tasks.rados.rados.0.smithi100.stdout:3614: rollback oid 41 current snap is 322 2024-04-07T16:28:01.370 INFO:tasks.rados.rados.0.smithi100.stdout:3614: done (2 left) 2024-04-07T16:28:01.370 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 36 current snap is 322 2024-04-07T16:28:01.370 INFO:tasks.rados.rados.0.smithi100.stdout:3615: seq_num 1121 ranges {3284992=778240,4063232=442368,4505600=638976,5144576=122880} 2024-04-07T16:28:01.429 INFO:tasks.rados.rados.0.smithi100.stdout:3615: writing smithi10066867-36 from 3284992 to 4063232 tid 1 2024-04-07T16:28:01.434 INFO:tasks.rados.rados.0.smithi100.stdout:3615: writing smithi10066867-36 from 4063232 to 4505600 tid 2 2024-04-07T16:28:01.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:01 smithi100 bash[74942]: cephadm 2024-04-07T16:27:59.543755+0000 mgr.y (mgr.25175) 370 : cephadm [INF] Upgrade: Updating grafana.a 2024-04-07T16:28:01.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:01 smithi100 bash[74942]: cephadm 2024-04-07T16:27:59.590398+0000 mgr.y (mgr.25175) 371 : cephadm [INF] Deploying daemon grafana.a on smithi144 2024-04-07T16:28:01.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:01 smithi100 bash[74942]: cluster 2024-04-07T16:28:00.038901+0000 mon.a (mon.0) 1225 : cluster [DBG] osdmap e722: 8 total, 8 up, 8 in 2024-04-07T16:28:01.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:01 smithi100 bash[76202]: cephadm 2024-04-07T16:27:59.543755+0000 mgr.y (mgr.25175) 370 : cephadm [INF] Upgrade: Updating grafana.a 2024-04-07T16:28:01.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:01 smithi100 bash[76202]: cephadm 2024-04-07T16:27:59.590398+0000 mgr.y (mgr.25175) 371 : cephadm [INF] Deploying daemon grafana.a on smithi144 2024-04-07T16:28:01.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:01 smithi100 bash[76202]: cluster 2024-04-07T16:28:00.038901+0000 mon.a (mon.0) 1225 : cluster [DBG] osdmap e722: 8 total, 8 up, 8 in 2024-04-07T16:28:01.441 INFO:tasks.rados.rados.0.smithi100.stdout:3615: writing smithi10066867-36 from 4505600 to 5144576 tid 3 2024-04-07T16:28:01.442 INFO:tasks.rados.rados.0.smithi100.stdout:3615: writing smithi10066867-36 from 5144576 to 5267456 tid 4 2024-04-07T16:28:01.442 INFO:tasks.rados.rados.0.smithi100.stdout:3616: rollback oid 43 current snap is 322 2024-04-07T16:28:01.442 INFO:tasks.rados.rados.0.smithi100.stdout:3616: done (3 left) 2024-04-07T16:28:01.443 INFO:tasks.rados.rados.0.smithi100.stdout:3617: setattr oid 30 current snap is 322 2024-04-07T16:28:01.443 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 49 v 1719 (ObjNum 81 snap 0 seq_num 4240222661) dirty exists 2024-04-07T16:28:01.443 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 14 v 1567 (ObjNum 1089 snap 318 seq_num 1089) dirty exists 2024-04-07T16:28:01.443 INFO:tasks.rados.rados.0.smithi100.stdout:3615: finishing write tid 1 to smithi10066867-36 2024-04-07T16:28:01.443 INFO:tasks.rados.rados.0.smithi100.stdout:3615: finishing write tid 2 to smithi10066867-36 2024-04-07T16:28:01.444 INFO:tasks.rados.rados.0.smithi100.stdout:3612: done (3 left) 2024-04-07T16:28:01.444 INFO:tasks.rados.rados.0.smithi100.stdout:3613: done (2 left) 2024-04-07T16:28:01.444 INFO:tasks.rados.rados.0.smithi100.stdout:3618: read oid 43 snap -1 2024-04-07T16:28:01.444 INFO:tasks.rados.rados.0.smithi100.stdout:3618: expect (ObjNum 1030 snap 306 seq_num 1030) 2024-04-07T16:28:01.444 INFO:tasks.rados.rados.0.smithi100.stdout:3619: snap_create 2024-04-07T16:28:01.450 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 30 v 1574 (ObjNum 1081 snap 314 seq_num 1081) dirty exists 2024-04-07T16:28:01.452 INFO:tasks.rados.rados.0.smithi100.stdout:3618: expect (ObjNum 1030 snap 306 seq_num 1030) 2024-04-07T16:28:01.471 INFO:tasks.rados.rados.0.smithi100.stdout:3615: finishing write tid 3 to smithi10066867-36 2024-04-07T16:28:01.471 INFO:tasks.rados.rados.0.smithi100.stdout:3615: finishing write tid 4 to smithi10066867-36 2024-04-07T16:28:01.471 INFO:tasks.rados.rados.0.smithi100.stdout:3615: finishing write tid 5 to smithi10066867-36 2024-04-07T16:28:01.471 INFO:tasks.rados.rados.0.smithi100.stdout:3615: finishing write tid 6 to smithi10066867-36 2024-04-07T16:28:01.471 INFO:tasks.rados.rados.0.smithi100.stdout:3615: oid 36 updating version 0 to 1287 2024-04-07T16:28:01.471 INFO:tasks.rados.rados.0.smithi100.stdout:3615: oid 36 version 1287 is already newer than 1285 2024-04-07T16:28:01.471 INFO:tasks.rados.rados.0.smithi100.stdout:3615: oid 36 version 1287 is already newer than 1284 2024-04-07T16:28:01.471 INFO:tasks.rados.rados.0.smithi100.stdout:3615: oid 36 version 1287 is already newer than 1286 2024-04-07T16:28:01.471 INFO:tasks.rados.rados.0.smithi100.stdout:3615: oid 36 updating version 1287 to 1288 2024-04-07T16:28:01.471 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 36 v 1288 (ObjNum 1121 snap 322 seq_num 1121) dirty exists 2024-04-07T16:28:01.471 INFO:tasks.rados.rados.0.smithi100.stdout:3615: left oid 36 (ObjNum 1121 snap 322 seq_num 1121) 2024-04-07T16:28:01.933 INFO:tasks.rados.rados.0.smithi100.stdout:3615: done (3 left) 2024-04-07T16:28:01.933 INFO:tasks.rados.rados.0.smithi100.stdout:3617: done (2 left) 2024-04-07T16:28:01.933 INFO:tasks.rados.rados.0.smithi100.stdout:3618: done (1 left) 2024-04-07T16:28:01.933 INFO:tasks.rados.rados.0.smithi100.stdout:3619: done (0 left) 2024-04-07T16:28:01.933 INFO:tasks.rados.rados.0.smithi100.stdout:3620: delete oid 9 current snap is 323 2024-04-07T16:28:01.938 INFO:tasks.rados.rados.0.smithi100.stdout:3620: done (0 left) 2024-04-07T16:28:01.938 INFO:tasks.rados.rados.0.smithi100.stdout:3621: snap_create 2024-04-07T16:28:02.304 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:02 smithi144 bash[66654]: cluster 2024-04-07T16:28:00.805896+0000 mgr.y (mgr.25175) 372 : cluster [DBG] pgmap v677: 121 pgs: 2 active+clean+snaptrim, 119 active+clean; 113 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 9.7 MiB/s rd, 5.7 MiB/s wr, 35 op/s 2024-04-07T16:28:02.304 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:02 smithi144 bash[66654]: cluster 2024-04-07T16:28:01.061941+0000 mon.a (mon.0) 1226 : cluster [DBG] osdmap e723: 8 total, 8 up, 8 in 2024-04-07T16:28:02.304 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:02 smithi144 bash[66654]: audit 2024-04-07T16:28:01.734875+0000 mon.a (mon.0) 1227 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:28:02.304 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:02 smithi144 bash[66654]: cluster 2024-04-07T16:28:01.946563+0000 mon.a (mon.0) 1228 : cluster [DBG] osdmap e724: 8 total, 8 up, 8 in 2024-04-07T16:28:02.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:02 smithi100 bash[74942]: cluster 2024-04-07T16:28:00.805896+0000 mgr.y (mgr.25175) 372 : cluster [DBG] pgmap v677: 121 pgs: 2 active+clean+snaptrim, 119 active+clean; 113 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 9.7 MiB/s rd, 5.7 MiB/s wr, 35 op/s 2024-04-07T16:28:02.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:02 smithi100 bash[74942]: cluster 2024-04-07T16:28:01.061941+0000 mon.a (mon.0) 1226 : cluster [DBG] osdmap e723: 8 total, 8 up, 8 in 2024-04-07T16:28:02.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:02 smithi100 bash[74942]: audit 2024-04-07T16:28:01.734875+0000 mon.a (mon.0) 1227 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:28:02.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:02 smithi100 bash[74942]: cluster 2024-04-07T16:28:01.946563+0000 mon.a (mon.0) 1228 : cluster [DBG] osdmap e724: 8 total, 8 up, 8 in 2024-04-07T16:28:02.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:02 smithi100 bash[76202]: cluster 2024-04-07T16:28:00.805896+0000 mgr.y (mgr.25175) 372 : cluster [DBG] pgmap v677: 121 pgs: 2 active+clean+snaptrim, 119 active+clean; 113 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 9.7 MiB/s rd, 5.7 MiB/s wr, 35 op/s 2024-04-07T16:28:02.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:02 smithi100 bash[76202]: cluster 2024-04-07T16:28:01.061941+0000 mon.a (mon.0) 1226 : cluster [DBG] osdmap e723: 8 total, 8 up, 8 in 2024-04-07T16:28:02.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:02 smithi100 bash[76202]: audit 2024-04-07T16:28:01.734875+0000 mon.a (mon.0) 1227 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:28:02.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:02 smithi100 bash[76202]: cluster 2024-04-07T16:28:01.946563+0000 mon.a (mon.0) 1228 : cluster [DBG] osdmap e724: 8 total, 8 up, 8 in 2024-04-07T16:28:02.605 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:28:02 smithi144 bash[95233]: ts=2024-04-07T16:28:02.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:28:02.940 INFO:tasks.rados.rados.0.smithi100.stdout:3621: done (0 left) 2024-04-07T16:28:02.940 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 1 current snap is 324 2024-04-07T16:28:02.940 INFO:tasks.rados.rados.0.smithi100.stdout:3622: seq_num 1123 ranges {0=409600,409600=614400,1024000=598016,1622016=393216} 2024-04-07T16:28:02.945 INFO:tasks.rados.rados.0.smithi100.stdout:3622: writing smithi10066867-1 from 0 to 409600 tid 1 2024-04-07T16:28:02.952 INFO:tasks.rados.rados.0.smithi100.stdout:3622: writing smithi10066867-1 from 409600 to 1024000 tid 2 2024-04-07T16:28:02.960 INFO:tasks.rados.rados.0.smithi100.stdout:3622: writing smithi10066867-1 from 1024000 to 1622016 tid 3 2024-04-07T16:28:02.965 INFO:tasks.rados.rados.0.smithi100.stdout:3622: writing smithi10066867-1 from 1622016 to 2015232 tid 4 2024-04-07T16:28:02.965 INFO:tasks.rados.rados.0.smithi100.stdout:3623: delete oid 50 current snap is 324 2024-04-07T16:28:02.965 INFO:tasks.rados.rados.0.smithi100.stdout:3622: finishing write tid 1 to smithi10066867-1 2024-04-07T16:28:02.965 INFO:tasks.rados.rados.0.smithi100.stdout:3622: finishing write tid 2 to smithi10066867-1 2024-04-07T16:28:02.965 INFO:tasks.rados.rados.0.smithi100.stdout:3622: finishing write tid 3 to smithi10066867-1 2024-04-07T16:28:02.970 INFO:tasks.rados.rados.0.smithi100.stdout:3623: done (1 left) 2024-04-07T16:28:02.970 INFO:tasks.rados.rados.0.smithi100.stdout:3624: snap_create 2024-04-07T16:28:02.972 INFO:tasks.rados.rados.0.smithi100.stdout:3622: finishing write tid 4 to smithi10066867-1 2024-04-07T16:28:02.973 INFO:tasks.rados.rados.0.smithi100.stdout:3622: finishing write tid 5 to smithi10066867-1 2024-04-07T16:28:02.975 INFO:tasks.rados.rados.0.smithi100.stdout:3622: finishing write tid 6 to smithi10066867-1 2024-04-07T16:28:02.975 INFO:tasks.rados.rados.0.smithi100.stdout:3622: oid 1 updating version 0 to 1834 2024-04-07T16:28:02.975 INFO:tasks.rados.rados.0.smithi100.stdout:3622: oid 1 updating version 1834 to 1836 2024-04-07T16:28:02.976 INFO:tasks.rados.rados.0.smithi100.stdout:3622: oid 1 updating version 1836 to 1837 2024-04-07T16:28:02.976 INFO:tasks.rados.rados.0.smithi100.stdout:3622: oid 1 updating version 1837 to 1838 2024-04-07T16:28:02.976 INFO:tasks.rados.rados.0.smithi100.stdout:3622: oid 1 version 1838 is already newer than 1835 2024-04-07T16:28:02.976 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 1 v 1838 (ObjNum 1123 snap 324 seq_num 1123) dirty exists 2024-04-07T16:28:02.976 INFO:tasks.rados.rados.0.smithi100.stdout:3622: left oid 1 (ObjNum 1123 snap 324 seq_num 1123) 2024-04-07T16:28:03.947 INFO:tasks.rados.rados.0.smithi100.stdout:3622: done (1 left) 2024-04-07T16:28:03.947 INFO:tasks.rados.rados.0.smithi100.stdout:3624: done (0 left) 2024-04-07T16:28:03.947 INFO:tasks.rados.rados.0.smithi100.stdout:3625: rmattr oid 15 current snap is 325 2024-04-07T16:28:03.948 INFO:tasks.rados.rados.0.smithi100.stdout:3625: done (0 left) 2024-04-07T16:28:03.948 INFO:tasks.rados.rados.0.smithi100.stdout:3626: read oid 37 snap -1 2024-04-07T16:28:03.948 INFO:tasks.rados.rados.0.smithi100.stdout:3626: expect (ObjNum 1103 snap 320 seq_num 1103) 2024-04-07T16:28:03.948 INFO:tasks.rados.rados.0.smithi100.stdout:3627: read oid 2 snap -1 2024-04-07T16:28:03.948 INFO:tasks.rados.rados.0.smithi100.stdout:3627: expect (ObjNum 1101 snap 320 seq_num 1101) 2024-04-07T16:28:03.948 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 6 current snap is 325 2024-04-07T16:28:03.948 INFO:tasks.rados.rados.0.smithi100.stdout:3628: seq_num 1125 ranges {0=450560,450560=786432,1236992=614400,1851392=204800} 2024-04-07T16:28:03.954 INFO:tasks.rados.rados.0.smithi100.stdout:3628: writing smithi10066867-6 from 0 to 450560 tid 1 2024-04-07T16:28:03.964 INFO:tasks.rados.rados.0.smithi100.stdout:3628: writing smithi10066867-6 from 450560 to 1236992 tid 2 2024-04-07T16:28:03.972 INFO:tasks.rados.rados.0.smithi100.stdout:3628: writing smithi10066867-6 from 1236992 to 1851392 tid 3 2024-04-07T16:28:03.975 INFO:tasks.rados.rados.0.smithi100.stdout:3628: writing smithi10066867-6 from 1851392 to 2056192 tid 4 2024-04-07T16:28:03.975 INFO:tasks.rados.rados.0.smithi100.stdout:3629: copy_from oid 38 from oid 19 current snap is 325 2024-04-07T16:28:03.975 INFO:tasks.rados.rados.0.smithi100.stdout:3630: snap_create 2024-04-07T16:28:03.991 INFO:tasks.rados.rados.0.smithi100.stdout:3627: expect (ObjNum 1101 snap 320 seq_num 1101) 2024-04-07T16:28:04.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:03 smithi100 bash[76202]: cluster 2024-04-07T16:28:02.807041+0000 mgr.y (mgr.25175) 373 : cluster [DBG] pgmap v680: 121 pgs: 121 active+clean; 114 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 10 MiB/s rd, 5.4 MiB/s wr, 32 op/s 2024-04-07T16:28:04.188 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:03 smithi100 bash[76202]: cluster 2024-04-07T16:28:02.954605+0000 mon.a (mon.0) 1229 : cluster [DBG] osdmap e725: 8 total, 8 up, 8 in 2024-04-07T16:28:04.188 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:03 smithi100 bash[74942]: cluster 2024-04-07T16:28:02.807041+0000 mgr.y (mgr.25175) 373 : cluster [DBG] pgmap v680: 121 pgs: 121 active+clean; 114 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 10 MiB/s rd, 5.4 MiB/s wr, 32 op/s 2024-04-07T16:28:04.188 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:03 smithi100 bash[74942]: cluster 2024-04-07T16:28:02.954605+0000 mon.a (mon.0) 1229 : cluster [DBG] osdmap e725: 8 total, 8 up, 8 in 2024-04-07T16:28:04.188 INFO:tasks.rados.rados.0.smithi100.stdout:3628: finishing write tid 1 to smithi10066867-6 2024-04-07T16:28:04.189 INFO:tasks.rados.rados.0.smithi100.stdout:3626: expect (ObjNum 1103 snap 320 seq_num 1103) 2024-04-07T16:28:04.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:03 smithi144 bash[66654]: cluster 2024-04-07T16:28:02.807041+0000 mgr.y (mgr.25175) 373 : cluster [DBG] pgmap v680: 121 pgs: 121 active+clean; 114 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 10 MiB/s rd, 5.4 MiB/s wr, 32 op/s 2024-04-07T16:28:04.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:03 smithi144 bash[66654]: cluster 2024-04-07T16:28:02.954605+0000 mon.a (mon.0) 1229 : cluster [DBG] osdmap e725: 8 total, 8 up, 8 in 2024-04-07T16:28:04.421 INFO:tasks.rados.rados.0.smithi100.stdout:3628: finishing write tid 2 to smithi10066867-6 2024-04-07T16:28:04.421 INFO:tasks.rados.rados.0.smithi100.stdout:3628: finishing write tid 3 to smithi10066867-6 2024-04-07T16:28:04.421 INFO:tasks.rados.rados.0.smithi100.stdout:3628: finishing write tid 4 to smithi10066867-6 2024-04-07T16:28:04.421 INFO:tasks.rados.rados.0.smithi100.stdout:3628: finishing write tid 5 to smithi10066867-6 2024-04-07T16:28:04.421 INFO:tasks.rados.rados.0.smithi100.stdout:3628: finishing write tid 6 to smithi10066867-6 2024-04-07T16:28:04.422 INFO:tasks.rados.rados.0.smithi100.stdout:3628: oid 6 updating version 0 to 1667 2024-04-07T16:28:04.422 INFO:tasks.rados.rados.0.smithi100.stdout:3628: oid 6 updating version 1667 to 1668 2024-04-07T16:28:04.422 INFO:tasks.rados.rados.0.smithi100.stdout:3628: oid 6 version 1668 is already newer than 1666 2024-04-07T16:28:04.422 INFO:tasks.rados.rados.0.smithi100.stdout:3628: oid 6 version 1668 is already newer than 1664 2024-04-07T16:28:04.422 INFO:tasks.rados.rados.0.smithi100.stdout:3628: oid 6 version 1668 is already newer than 1665 2024-04-07T16:28:04.422 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 6 v 1668 (ObjNum 1125 snap 325 seq_num 1125) dirty exists 2024-04-07T16:28:04.422 INFO:tasks.rados.rados.0.smithi100.stdout:3628: left oid 6 (ObjNum 1125 snap 325 seq_num 1125) 2024-04-07T16:28:04.422 INFO:tasks.rados.rados.0.smithi100.stdout:3629: finishing copy_from racing read to smithi10066867-38 2024-04-07T16:28:04.422 INFO:tasks.rados.rados.0.smithi100.stdout:3629: finishing copy_from to smithi10066867-38 2024-04-07T16:28:04.422 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 38 v 1273 (ObjNum 1079 snap 314 seq_num 1079) dirty exists 2024-04-07T16:28:04.957 INFO:tasks.rados.rados.0.smithi100.stdout:3626: done (4 left) 2024-04-07T16:28:04.957 INFO:tasks.rados.rados.0.smithi100.stdout:3627: done (3 left) 2024-04-07T16:28:04.958 INFO:tasks.rados.rados.0.smithi100.stdout:3628: done (2 left) 2024-04-07T16:28:04.958 INFO:tasks.rados.rados.0.smithi100.stdout:3629: done (1 left) 2024-04-07T16:28:04.958 INFO:tasks.rados.rados.0.smithi100.stdout:3630: done (0 left) 2024-04-07T16:28:04.958 INFO:tasks.rados.rados.0.smithi100.stdout:3631: delete oid 26 current snap is 326 2024-04-07T16:28:04.966 INFO:tasks.rados.rados.0.smithi100.stdout:3631: done (0 left) 2024-04-07T16:28:04.966 INFO:tasks.rados.rados.0.smithi100.stdout:3632: snap_remove snap 323 2024-04-07T16:28:05.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:04 smithi144 bash[66654]: cluster 2024-04-07T16:28:03.961211+0000 mon.a (mon.0) 1230 : cluster [DBG] osdmap e726: 8 total, 8 up, 8 in 2024-04-07T16:28:05.437 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:04 smithi100 bash[74942]: cluster 2024-04-07T16:28:03.961211+0000 mon.a (mon.0) 1230 : cluster [DBG] osdmap e726: 8 total, 8 up, 8 in 2024-04-07T16:28:05.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:04 smithi100 bash[76202]: cluster 2024-04-07T16:28:03.961211+0000 mon.a (mon.0) 1230 : cluster [DBG] osdmap e726: 8 total, 8 up, 8 in 2024-04-07T16:28:05.961 INFO:tasks.rados.rados.0.smithi100.stdout:3632: done (0 left) 2024-04-07T16:28:05.961 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 1 current snap is 326 2024-04-07T16:28:05.962 INFO:tasks.rados.rados.0.smithi100.stdout:3633: seq_num 1127 ranges {2015232=483328,2498560=598016,3096576=581632,3678208=425984} 2024-04-07T16:28:05.997 INFO:tasks.rados.rados.0.smithi100.stdout:3633: writing smithi10066867-1 from 2015232 to 2498560 tid 1 2024-04-07T16:28:06.007 INFO:tasks.rados.rados.0.smithi100.stdout:3633: writing smithi10066867-1 from 2498560 to 3096576 tid 2 2024-04-07T16:28:06.013 INFO:tasks.rados.rados.0.smithi100.stdout:3633: writing smithi10066867-1 from 3096576 to 3678208 tid 3 2024-04-07T16:28:06.018 INFO:tasks.rados.rados.0.smithi100.stdout:3633: writing smithi10066867-1 from 3678208 to 4104192 tid 4 2024-04-07T16:28:06.018 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 31 current snap is 326 2024-04-07T16:28:06.018 INFO:tasks.rados.rados.0.smithi100.stdout:3633: finishing write tid 1 to smithi10066867-1 2024-04-07T16:28:06.018 INFO:tasks.rados.rados.0.smithi100.stdout:3633: finishing write tid 2 to smithi10066867-1 2024-04-07T16:28:06.019 INFO:tasks.rados.rados.0.smithi100.stdout:3633: finishing write tid 3 to smithi10066867-1 2024-04-07T16:28:06.019 INFO:tasks.rados.rados.0.smithi100.stdout:3634: seq_num 1128 ranges {4694016=499712,5193728=483328,5677056=786432,6463488=335872} 2024-04-07T16:28:06.068 INFO:tasks.rados.rados.0.smithi100.stdout:3634: writing smithi10066867-31 from 4694016 to 5193728 tid 1 2024-04-07T16:28:06.072 INFO:tasks.rados.rados.0.smithi100.stdout:3634: writing smithi10066867-31 from 5193728 to 5677056 tid 2 2024-04-07T16:28:06.078 INFO:tasks.rados.rados.0.smithi100.stdout:3634: writing smithi10066867-31 from 5677056 to 6463488 tid 3 2024-04-07T16:28:06.081 INFO:tasks.rados.rados.0.smithi100.stdout:3634: writing smithi10066867-31 from 6463488 to 6799360 tid 4 2024-04-07T16:28:06.082 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 38 current snap is 326 2024-04-07T16:28:06.082 INFO:tasks.rados.rados.0.smithi100.stdout:3635: seq_num 1129 ranges {2801664=516096,3317760=770048,4087808=598016,4685824=237568} 2024-04-07T16:28:06.102 INFO:tasks.rados.rados.0.smithi100.stdout:3635: writing smithi10066867-38 from 2801664 to 3317760 tid 1 2024-04-07T16:28:06.106 INFO:tasks.rados.rados.0.smithi100.stdout:3635: writing smithi10066867-38 from 3317760 to 4087808 tid 2 2024-04-07T16:28:06.109 INFO:tasks.rados.rados.0.smithi100.stdout:3635: writing smithi10066867-38 from 4087808 to 4685824 tid 3 2024-04-07T16:28:06.111 INFO:tasks.rados.rados.0.smithi100.stdout:3635: writing smithi10066867-38 from 4685824 to 4923392 tid 4 2024-04-07T16:28:06.111 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 44 current snap is 326 2024-04-07T16:28:06.111 INFO:tasks.rados.rados.0.smithi100.stdout:3633: finishing write tid 4 to smithi10066867-1 2024-04-07T16:28:06.111 INFO:tasks.rados.rados.0.smithi100.stdout:3633: finishing write tid 5 to smithi10066867-1 2024-04-07T16:28:06.111 INFO:tasks.rados.rados.0.smithi100.stdout:3633: finishing write tid 6 to smithi10066867-1 2024-04-07T16:28:06.111 INFO:tasks.rados.rados.0.smithi100.stdout:3633: oid 1 updating version 0 to 1839 2024-04-07T16:28:06.112 INFO:tasks.rados.rados.0.smithi100.stdout:3633: oid 1 updating version 1839 to 1841 2024-04-07T16:28:06.112 INFO:tasks.rados.rados.0.smithi100.stdout:3633: oid 1 updating version 1841 to 1842 2024-04-07T16:28:06.112 INFO:tasks.rados.rados.0.smithi100.stdout:3633: oid 1 version 1842 is already newer than 1840 2024-04-07T16:28:06.112 INFO:tasks.rados.rados.0.smithi100.stdout:3633: oid 1 updating version 1842 to 1843 2024-04-07T16:28:06.112 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 1 v 1843 (ObjNum 1127 snap 326 seq_num 1127) dirty exists 2024-04-07T16:28:06.112 INFO:tasks.rados.rados.0.smithi100.stdout:3633: left oid 1 (ObjNum 1127 snap 326 seq_num 1127) 2024-04-07T16:28:06.112 INFO:tasks.rados.rados.0.smithi100.stdout:3634: finishing write tid 1 to smithi10066867-31 2024-04-07T16:28:06.112 INFO:tasks.rados.rados.0.smithi100.stdout:3634: finishing write tid 2 to smithi10066867-31 2024-04-07T16:28:06.112 INFO:tasks.rados.rados.0.smithi100.stdout:3634: finishing write tid 3 to smithi10066867-31 2024-04-07T16:28:06.112 INFO:tasks.rados.rados.0.smithi100.stdout:3634: finishing write tid 4 to smithi10066867-31 2024-04-07T16:28:06.112 INFO:tasks.rados.rados.0.smithi100.stdout:3634: finishing write tid 5 to smithi10066867-31 2024-04-07T16:28:06.112 INFO:tasks.rados.rados.0.smithi100.stdout:3634: finishing write tid 6 to smithi10066867-31 2024-04-07T16:28:06.112 INFO:tasks.rados.rados.0.smithi100.stdout:3634: oid 31 updating version 0 to 1728 2024-04-07T16:28:06.112 INFO:tasks.rados.rados.0.smithi100.stdout:3634: oid 31 version 1728 is already newer than 1726 2024-04-07T16:28:06.112 INFO:tasks.rados.rados.0.smithi100.stdout:3634: oid 31 updating version 1728 to 1729 2024-04-07T16:28:06.113 INFO:tasks.rados.rados.0.smithi100.stdout:3634: oid 31 updating version 1729 to 1730 2024-04-07T16:28:06.113 INFO:tasks.rados.rados.0.smithi100.stdout:3634: oid 31 version 1730 is already newer than 1727 2024-04-07T16:28:06.113 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 31 v 1730 (ObjNum 1128 snap 326 seq_num 1128) dirty exists 2024-04-07T16:28:06.113 INFO:tasks.rados.rados.0.smithi100.stdout:3634: left oid 31 (ObjNum 1128 snap 326 seq_num 1128) 2024-04-07T16:28:06.113 INFO:tasks.rados.rados.0.smithi100.stdout:3636: seq_num 1130 ranges {0=532480,532480=655360,1187840=753664,1941504=196608} 2024-04-07T16:28:06.114 INFO:tasks.rados.rados.0.smithi100.stdout:3636: writing smithi10066867-44 from 0 to 532480 tid 1 2024-04-07T16:28:06.118 INFO:tasks.rados.rados.0.smithi100.stdout:3636: writing smithi10066867-44 from 532480 to 1187840 tid 2 2024-04-07T16:28:06.122 INFO:tasks.rados.rados.0.smithi100.stdout:3636: writing smithi10066867-44 from 1187840 to 1941504 tid 3 2024-04-07T16:28:06.124 INFO:tasks.rados.rados.0.smithi100.stdout:3636: writing smithi10066867-44 from 1941504 to 2138112 tid 4 2024-04-07T16:28:06.124 INFO:tasks.rados.rados.0.smithi100.stdout:3633: done (3 left) 2024-04-07T16:28:06.124 INFO:tasks.rados.rados.0.smithi100.stdout:3634: done (2 left) 2024-04-07T16:28:06.124 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 48 current snap is 326 2024-04-07T16:28:06.124 INFO:tasks.rados.rados.0.smithi100.stdout:3635: finishing write tid 1 to smithi10066867-38 2024-04-07T16:28:06.124 INFO:tasks.rados.rados.0.smithi100.stdout:3637: seq_num 1131 ranges {1368064=548864,1916928=540672,2457600=565248,3022848=425984,3448832=73728} 2024-04-07T16:28:06.135 INFO:tasks.rados.rados.0.smithi100.stdout:3637: writing smithi10066867-48 from 1368064 to 1916928 tid 1 2024-04-07T16:28:06.138 INFO:tasks.rados.rados.0.smithi100.stdout:3637: writing smithi10066867-48 from 1916928 to 2457600 tid 2 2024-04-07T16:28:06.142 INFO:tasks.rados.rados.0.smithi100.stdout:3637: writing smithi10066867-48 from 2457600 to 3022848 tid 3 2024-04-07T16:28:06.145 INFO:tasks.rados.rados.0.smithi100.stdout:3637: writing smithi10066867-48 from 3022848 to 3448832 tid 4 2024-04-07T16:28:06.147 INFO:tasks.rados.rados.0.smithi100.stdout:3637: writing smithi10066867-48 from 3448832 to 3522560 tid 5 2024-04-07T16:28:06.147 INFO:tasks.rados.rados.0.smithi100.stdout:3638: rmattr oid 19 current snap is 326 2024-04-07T16:28:06.147 INFO:tasks.rados.rados.0.smithi100.stdout:3638: done (3 left) 2024-04-07T16:28:06.147 INFO:tasks.rados.rados.0.smithi100.stdout:3639: rollback oid 27 current snap is 326 2024-04-07T16:28:06.147 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 27 to 324 2024-04-07T16:28:06.147 INFO:tasks.rados.rados.0.smithi100.stdout:3635: finishing write tid 2 to smithi10066867-38 2024-04-07T16:28:06.147 INFO:tasks.rados.rados.0.smithi100.stdout:3635: finishing write tid 3 to smithi10066867-38 2024-04-07T16:28:06.147 INFO:tasks.rados.rados.0.smithi100.stdout:3635: finishing write tid 4 to smithi10066867-38 2024-04-07T16:28:06.147 INFO:tasks.rados.rados.0.smithi100.stdout:3635: finishing write tid 5 to smithi10066867-38 2024-04-07T16:28:06.148 INFO:tasks.rados.rados.0.smithi100.stdout:3636: finishing write tid 1 to smithi10066867-44 2024-04-07T16:28:06.148 INFO:tasks.rados.rados.0.smithi100.stdout:3640: rmattr oid 12 current snap is 326 2024-04-07T16:28:06.148 INFO:tasks.rados.rados.0.smithi100.stdout:3640: done (4 left) 2024-04-07T16:28:06.148 INFO:tasks.rados.rados.0.smithi100.stdout:3641: rmattr oid 9 current snap is 326 2024-04-07T16:28:06.148 INFO:tasks.rados.rados.0.smithi100.stdout:3641: done (4 left) 2024-04-07T16:28:06.148 INFO:tasks.rados.rados.0.smithi100.stdout:3642: read oid 9 snap -1 2024-04-07T16:28:06.148 INFO:tasks.rados.rados.0.smithi100.stdout:3642: expect deleted 2024-04-07T16:28:06.148 INFO:tasks.rados.rados.0.smithi100.stdout:3643: rollback oid 1 current snap is 326 2024-04-07T16:28:06.148 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 1 to 322 2024-04-07T16:28:06.148 INFO:tasks.rados.rados.0.smithi100.stdout:3635: finishing write tid 6 to smithi10066867-38 2024-04-07T16:28:06.148 INFO:tasks.rados.rados.0.smithi100.stdout:3635: oid 38 updating version 0 to 1706 2024-04-07T16:28:06.148 INFO:tasks.rados.rados.0.smithi100.stdout:3635: oid 38 version 1706 is already newer than 1704 2024-04-07T16:28:06.148 INFO:tasks.rados.rados.0.smithi100.stdout:3635: oid 38 updating version 1706 to 1707 2024-04-07T16:28:06.148 INFO:tasks.rados.rados.0.smithi100.stdout:3635: oid 38 version 1707 is already newer than 1703 2024-04-07T16:28:06.148 INFO:tasks.rados.rados.0.smithi100.stdout:3635: oid 38 version 1707 is already newer than 1705 2024-04-07T16:28:06.148 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 38 v 1707 (ObjNum 1129 snap 326 seq_num 1129) dirty exists 2024-04-07T16:28:06.148 INFO:tasks.rados.rados.0.smithi100.stdout:3635: left oid 38 (ObjNum 1129 snap 326 seq_num 1129) 2024-04-07T16:28:06.149 INFO:tasks.rados.rados.0.smithi100.stdout:3636: finishing write tid 2 to smithi10066867-44 2024-04-07T16:28:06.149 INFO:tasks.rados.rados.0.smithi100.stdout:3635: done (5 left) 2024-04-07T16:28:06.149 INFO:tasks.rados.rados.0.smithi100.stdout:3644: read oid 31 snap -1 2024-04-07T16:28:06.149 INFO:tasks.rados.rados.0.smithi100.stdout:3644: expect (ObjNum 1128 snap 326 seq_num 1128) 2024-04-07T16:28:06.149 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 33 current snap is 326 2024-04-07T16:28:06.149 INFO:tasks.rados.rados.0.smithi100.stdout:3645: seq_num 1132 ranges {0=565248,565248=425984,991232=770048,1761280=409600} 2024-04-07T16:28:06.160 INFO:tasks.rados.rados.0.smithi100.stdout:3645: writing smithi10066867-33 from 0 to 565248 tid 1 2024-04-07T16:28:06.160 INFO:tasks.rados.rados.0.smithi100.stdout:3645: writing smithi10066867-33 from 565248 to 991232 tid 2 2024-04-07T16:28:06.160 INFO:tasks.rados.rados.0.smithi100.stdout:3645: writing smithi10066867-33 from 991232 to 1761280 tid 3 2024-04-07T16:28:06.162 INFO:tasks.rados.rados.0.smithi100.stdout:3645: writing smithi10066867-33 from 1761280 to 2170880 tid 4 2024-04-07T16:28:06.162 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 37 current snap is 326 2024-04-07T16:28:06.162 INFO:tasks.rados.rados.0.smithi100.stdout:3636: finishing write tid 3 to smithi10066867-44 2024-04-07T16:28:06.162 INFO:tasks.rados.rados.0.smithi100.stdout:3636: finishing write tid 4 to smithi10066867-44 2024-04-07T16:28:06.162 INFO:tasks.rados.rados.0.smithi100.stdout:3636: finishing write tid 5 to smithi10066867-44 2024-04-07T16:28:06.162 INFO:tasks.rados.rados.0.smithi100.stdout:3636: finishing write tid 6 to smithi10066867-44 2024-04-07T16:28:06.162 INFO:tasks.rados.rados.0.smithi100.stdout:3636: oid 44 updating version 0 to 1671 2024-04-07T16:28:06.162 INFO:tasks.rados.rados.0.smithi100.stdout:3636: oid 44 updating version 1671 to 1672 2024-04-07T16:28:06.162 INFO:tasks.rados.rados.0.smithi100.stdout:3636: oid 44 updating version 1672 to 1673 2024-04-07T16:28:06.162 INFO:tasks.rados.rados.0.smithi100.stdout:3636: oid 44 version 1673 is already newer than 1670 2024-04-07T16:28:06.162 INFO:tasks.rados.rados.0.smithi100.stdout:3636: oid 44 version 1673 is already newer than 1669 2024-04-07T16:28:06.162 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 44 v 1673 (ObjNum 1130 snap 326 seq_num 1130) dirty exists 2024-04-07T16:28:06.162 INFO:tasks.rados.rados.0.smithi100.stdout:3636: left oid 44 (ObjNum 1130 snap 326 seq_num 1130) 2024-04-07T16:28:06.163 INFO:tasks.rados.rados.0.smithi100.stdout:3637: finishing write tid 1 to smithi10066867-48 2024-04-07T16:28:06.163 INFO:tasks.rados.rados.0.smithi100.stdout:3637: finishing write tid 2 to smithi10066867-48 2024-04-07T16:28:06.163 INFO:tasks.rados.rados.0.smithi100.stdout:3646: seq_num 1133 ranges {5300224=581632,5881856=753664,6635520=573440,7208960=278528} 2024-04-07T16:28:06.194 INFO:tasks.rados.rados.0.smithi100.stdout:3646: writing smithi10066867-37 from 5300224 to 5881856 tid 1 2024-04-07T16:28:06.198 INFO:tasks.rados.rados.0.smithi100.stdout:3646: writing smithi10066867-37 from 5881856 to 6635520 tid 2 2024-04-07T16:28:06.202 INFO:tasks.rados.rados.0.smithi100.stdout:3646: writing smithi10066867-37 from 6635520 to 7208960 tid 3 2024-04-07T16:28:06.204 INFO:tasks.rados.rados.0.smithi100.stdout:3646: writing smithi10066867-37 from 7208960 to 7487488 tid 4 2024-04-07T16:28:06.204 INFO:tasks.rados.rados.0.smithi100.stdout:3636: done (7 left) 2024-04-07T16:28:06.204 INFO:tasks.rados.rados.0.smithi100.stdout:3647: snap_remove snap 325 2024-04-07T16:28:06.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:05 smithi144 bash[66654]: cluster 2024-04-07T16:28:04.807930+0000 mgr.y (mgr.25175) 374 : cluster [DBG] pgmap v683: 121 pgs: 121 active+clean; 114 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 1.7 MiB/s rd, 2.8 MiB/s wr, 28 op/s 2024-04-07T16:28:06.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:05 smithi144 bash[66654]: cluster 2024-04-07T16:28:04.970581+0000 mon.a (mon.0) 1231 : cluster [DBG] osdmap e727: 8 total, 8 up, 8 in 2024-04-07T16:28:06.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:05 smithi100 bash[74942]: cluster 2024-04-07T16:28:04.807930+0000 mgr.y (mgr.25175) 374 : cluster [DBG] pgmap v683: 121 pgs: 121 active+clean; 114 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 1.7 MiB/s rd, 2.8 MiB/s wr, 28 op/s 2024-04-07T16:28:06.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:05 smithi100 bash[74942]: cluster 2024-04-07T16:28:04.970581+0000 mon.a (mon.0) 1231 : cluster [DBG] osdmap e727: 8 total, 8 up, 8 in 2024-04-07T16:28:06.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:05 smithi100 bash[76202]: cluster 2024-04-07T16:28:04.807930+0000 mgr.y (mgr.25175) 374 : cluster [DBG] pgmap v683: 121 pgs: 121 active+clean; 114 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 1.7 MiB/s rd, 2.8 MiB/s wr, 28 op/s 2024-04-07T16:28:06.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:05 smithi100 bash[76202]: cluster 2024-04-07T16:28:04.970581+0000 mon.a (mon.0) 1231 : cluster [DBG] osdmap e727: 8 total, 8 up, 8 in 2024-04-07T16:28:06.935 INFO:tasks.rados.rados.0.smithi100.stdout:3647: done (7 left) 2024-04-07T16:28:06.935 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 38 current snap is 326 2024-04-07T16:28:06.939 INFO:tasks.rados.rados.0.smithi100.stdout:3637: finishing write tid 3 to smithi10066867-48 2024-04-07T16:28:06.939 INFO:tasks.rados.rados.0.smithi100.stdout:3639: finishing rollback tid 0 to smithi10066867-27 2024-04-07T16:28:06.940 INFO:tasks.rados.rados.0.smithi100.stdout:3639: finishing rollback tid 1 to smithi10066867-27 2024-04-07T16:28:06.940 INFO:tasks.rados.rados.0.smithi100.stdout:3639: finishing rollback tid 2 to smithi10066867-27 2024-04-07T16:28:06.940 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 27 v 1272 (ObjNum 1108 snap 321 seq_num 1108) dirty exists 2024-04-07T16:28:06.940 INFO:tasks.rados.rados.0.smithi100.stdout:3643: finishing rollback tid 0 to smithi10066867-1 2024-04-07T16:28:06.940 INFO:tasks.rados.rados.0.smithi100.stdout:3643: finishing rollback tid 1 to smithi10066867-1 2024-04-07T16:28:06.940 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 1 v 1845 (ObjNum 433 snap 0 seq_num 4239677540) dirty dne 2024-04-07T16:28:06.940 INFO:tasks.rados.rados.0.smithi100.stdout:3645: finishing write tid 1 to smithi10066867-33 2024-04-07T16:28:06.940 INFO:tasks.rados.rados.0.smithi100.stdout:3637: finishing write tid 4 to smithi10066867-48 2024-04-07T16:28:06.940 INFO:tasks.rados.rados.0.smithi100.stdout:3637: finishing write tid 5 to smithi10066867-48 2024-04-07T16:28:06.940 INFO:tasks.rados.rados.0.smithi100.stdout:3637: finishing write tid 6 to smithi10066867-48 2024-04-07T16:28:06.940 INFO:tasks.rados.rados.0.smithi100.stdout:3645: finishing write tid 2 to smithi10066867-33 2024-04-07T16:28:06.941 INFO:tasks.rados.rados.0.smithi100.stdout:3645: finishing write tid 3 to smithi10066867-33 2024-04-07T16:28:06.941 INFO:tasks.rados.rados.0.smithi100.stdout:3645: finishing write tid 4 to smithi10066867-33 2024-04-07T16:28:06.941 INFO:tasks.rados.rados.0.smithi100.stdout:3645: finishing write tid 5 to smithi10066867-33 2024-04-07T16:28:06.941 INFO:tasks.rados.rados.0.smithi100.stdout:3645: finishing write tid 6 to smithi10066867-33 2024-04-07T16:28:06.941 INFO:tasks.rados.rados.0.smithi100.stdout:3645: oid 33 updating version 0 to 1882 2024-04-07T16:28:06.941 INFO:tasks.rados.rados.0.smithi100.stdout:3645: oid 33 updating version 1882 to 1884 2024-04-07T16:28:06.941 INFO:tasks.rados.rados.0.smithi100.stdout:3645: oid 33 version 1884 is already newer than 1883 2024-04-07T16:28:06.941 INFO:tasks.rados.rados.0.smithi100.stdout:3645: oid 33 version 1884 is already newer than 1881 2024-04-07T16:28:06.941 INFO:tasks.rados.rados.0.smithi100.stdout:3645: oid 33 version 1884 is already newer than 1880 2024-04-07T16:28:06.941 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 33 v 1884 (ObjNum 1132 snap 326 seq_num 1132) dirty exists 2024-04-07T16:28:06.941 INFO:tasks.rados.rados.0.smithi100.stdout:3645: left oid 33 (ObjNum 1132 snap 326 seq_num 1132) 2024-04-07T16:28:06.941 INFO:tasks.rados.rados.0.smithi100.stdout:3644: expect (ObjNum 1128 snap 326 seq_num 1128) 2024-04-07T16:28:07.250 INFO:tasks.rados.rados.0.smithi100.stdout:3637: finishing write tid 7 to smithi10066867-48 2024-04-07T16:28:07.250 INFO:tasks.rados.rados.0.smithi100.stdout:3637: oid 48 updating version 0 to 1735 2024-04-07T16:28:07.250 INFO:tasks.rados.rados.0.smithi100.stdout:3637: oid 48 version 1735 is already newer than 1733 2024-04-07T16:28:07.250 INFO:tasks.rados.rados.0.smithi100.stdout:3637: oid 48 version 1735 is already newer than 1734 2024-04-07T16:28:07.250 INFO:tasks.rados.rados.0.smithi100.stdout:3637: oid 48 updating version 1735 to 1736 2024-04-07T16:28:07.251 INFO:tasks.rados.rados.0.smithi100.stdout:3637: oid 48 version 1736 is already newer than 1732 2024-04-07T16:28:07.251 INFO:tasks.rados.rados.0.smithi100.stdout:3637: oid 48 version 1736 is already newer than 1731 2024-04-07T16:28:07.251 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 48 v 1736 (ObjNum 1131 snap 326 seq_num 1131) dirty exists 2024-04-07T16:28:07.251 INFO:tasks.rados.rados.0.smithi100.stdout:3637: left oid 48 (ObjNum 1131 snap 326 seq_num 1131) 2024-04-07T16:28:07.251 INFO:tasks.rados.rados.0.smithi100.stdout:3646: finishing write tid 1 to smithi10066867-37 2024-04-07T16:28:07.251 INFO:tasks.rados.rados.0.smithi100.stdout:3646: finishing write tid 2 to smithi10066867-37 2024-04-07T16:28:07.251 INFO:tasks.rados.rados.0.smithi100.stdout:3646: finishing write tid 3 to smithi10066867-37 2024-04-07T16:28:07.251 INFO:tasks.rados.rados.0.smithi100.stdout:3646: finishing write tid 4 to smithi10066867-37 2024-04-07T16:28:07.251 INFO:tasks.rados.rados.0.smithi100.stdout:3646: finishing write tid 5 to smithi10066867-37 2024-04-07T16:28:07.251 INFO:tasks.rados.rados.0.smithi100.stdout:3646: finishing write tid 6 to smithi10066867-37 2024-04-07T16:28:07.251 INFO:tasks.rados.rados.0.smithi100.stdout:3646: oid 37 updating version 0 to 1579 2024-04-07T16:28:07.251 INFO:tasks.rados.rados.0.smithi100.stdout:3646: oid 37 version 1579 is already newer than 1575 2024-04-07T16:28:07.251 INFO:tasks.rados.rados.0.smithi100.stdout:3646: oid 37 version 1579 is already newer than 1577 2024-04-07T16:28:07.251 INFO:tasks.rados.rados.0.smithi100.stdout:3646: oid 37 version 1579 is already newer than 1578 2024-04-07T16:28:07.251 INFO:tasks.rados.rados.0.smithi100.stdout:3646: oid 37 version 1579 is already newer than 1576 2024-04-07T16:28:07.252 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 37 v 1579 (ObjNum 1133 snap 326 seq_num 1133) dirty exists 2024-04-07T16:28:07.252 INFO:tasks.rados.rados.0.smithi100.stdout:3646: left oid 37 (ObjNum 1133 snap 326 seq_num 1133) 2024-04-07T16:28:07.252 INFO:tasks.rados.rados.0.smithi100.stdout:3648: seq_num 1134 ranges {4923392=598016,5521408=638976,6160384=737280,6897664=229376} 2024-04-07T16:28:07.292 INFO:tasks.rados.rados.0.smithi100.stdout:3648: writing smithi10066867-38 from 4923392 to 5521408 tid 1 2024-04-07T16:28:07.297 INFO:tasks.rados.rados.0.smithi100.stdout:3648: writing smithi10066867-38 from 5521408 to 6160384 tid 2 2024-04-07T16:28:07.303 INFO:tasks.rados.rados.0.smithi100.stdout:3648: writing smithi10066867-38 from 6160384 to 6897664 tid 3 2024-04-07T16:28:07.306 INFO:tasks.rados.rados.0.smithi100.stdout:3648: writing smithi10066867-38 from 6897664 to 7127040 tid 4 2024-04-07T16:28:07.306 INFO:tasks.rados.rados.0.smithi100.stdout:3637: done (7 left) 2024-04-07T16:28:07.306 INFO:tasks.rados.rados.0.smithi100.stdout:3639: done (6 left) 2024-04-07T16:28:07.306 INFO:tasks.rados.rados.0.smithi100.stdout:3642: done (5 left) 2024-04-07T16:28:07.306 INFO:tasks.rados.rados.0.smithi100.stdout:3643: done (4 left) 2024-04-07T16:28:07.306 INFO:tasks.rados.rados.0.smithi100.stdout:3644: done (3 left) 2024-04-07T16:28:07.306 INFO:tasks.rados.rados.0.smithi100.stdout:3645: done (2 left) 2024-04-07T16:28:07.306 INFO:tasks.rados.rados.0.smithi100.stdout:3646: done (1 left) 2024-04-07T16:28:07.306 INFO:tasks.rados.rados.0.smithi100.stdout:3649: read oid 5 snap -1 2024-04-07T16:28:07.306 INFO:tasks.rados.rados.0.smithi100.stdout:3649: expect (ObjNum 1058 snap 309 seq_num 1058) 2024-04-07T16:28:07.307 INFO:tasks.rados.rados.0.smithi100.stdout:3650: copy_from oid 40 from oid 26 current snap is 326 2024-04-07T16:28:07.307 INFO:tasks.rados.rados.0.smithi100.stdout:3651: copy_from oid 16 from oid 4 current snap is 326 2024-04-07T16:28:07.307 INFO:tasks.rados.rados.0.smithi100.stdout:3652: snap_remove snap 324 2024-04-07T16:28:07.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:06 smithi144 bash[66654]: cluster 2024-04-07T16:28:05.972789+0000 mon.a (mon.0) 1232 : cluster [DBG] osdmap e728: 8 total, 8 up, 8 in 2024-04-07T16:28:07.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:06 smithi144 bash[66654]: cluster 2024-04-07T16:28:06.947675+0000 mon.a (mon.0) 1233 : cluster [DBG] osdmap e729: 8 total, 8 up, 8 in 2024-04-07T16:28:07.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:06 smithi100 bash[74942]: cluster 2024-04-07T16:28:05.972789+0000 mon.a (mon.0) 1232 : cluster [DBG] osdmap e728: 8 total, 8 up, 8 in 2024-04-07T16:28:07.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:06 smithi100 bash[74942]: cluster 2024-04-07T16:28:06.947675+0000 mon.a (mon.0) 1233 : cluster [DBG] osdmap e729: 8 total, 8 up, 8 in 2024-04-07T16:28:07.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:06 smithi100 bash[76202]: cluster 2024-04-07T16:28:05.972789+0000 mon.a (mon.0) 1232 : cluster [DBG] osdmap e728: 8 total, 8 up, 8 in 2024-04-07T16:28:07.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:06 smithi100 bash[76202]: cluster 2024-04-07T16:28:06.947675+0000 mon.a (mon.0) 1233 : cluster [DBG] osdmap e729: 8 total, 8 up, 8 in 2024-04-07T16:28:07.987 INFO:tasks.rados.rados.0.smithi100.stdout:3652: done (4 left) 2024-04-07T16:28:07.987 INFO:tasks.rados.rados.0.smithi100.stdout:3653: rollback oid 43 current snap is 326 2024-04-07T16:28:07.987 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 43 to 322 2024-04-07T16:28:07.987 INFO:tasks.rados.rados.0.smithi100.stdout:3654: copy_from oid 45 from oid 7 current snap is 326 2024-04-07T16:28:07.987 INFO:tasks.rados.rados.0.smithi100.stdout:3655: rollback oid 1 current snap is 326 2024-04-07T16:28:07.987 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 1 to 322 2024-04-07T16:28:07.987 INFO:tasks.rados.rados.0.smithi100.stdout:3656: delete oid 10 current snap is 326 2024-04-07T16:28:07.987 INFO:tasks.rados.rados.0.smithi100.stdout:3648: finishing write tid 1 to smithi10066867-38 2024-04-07T16:28:07.988 INFO:tasks.rados.rados.0.smithi100.stdout:3648: finishing write tid 2 to smithi10066867-38 2024-04-07T16:28:07.988 INFO:tasks.rados.rados.0.smithi100.stdout:3649: expect (ObjNum 1058 snap 309 seq_num 1058) 2024-04-07T16:28:08.022 INFO:tasks.rados.rados.0.smithi100.stdout:3648: finishing write tid 3 to smithi10066867-38 2024-04-07T16:28:08.022 INFO:tasks.rados.rados.0.smithi100.stdout:3648: finishing write tid 4 to smithi10066867-38 2024-04-07T16:28:08.022 INFO:tasks.rados.rados.0.smithi100.stdout:3649: done (7 left) 2024-04-07T16:28:08.022 INFO:tasks.rados.rados.0.smithi100.stdout:3656: done (6 left) 2024-04-07T16:28:08.022 INFO:tasks.rados.rados.0.smithi100.stdout:3657: rollback oid 29 current snap is 326 2024-04-07T16:28:08.022 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 29 to 322 2024-04-07T16:28:08.022 INFO:tasks.rados.rados.0.smithi100.stdout:3648: finishing write tid 5 to smithi10066867-38 2024-04-07T16:28:08.023 INFO:tasks.rados.rados.0.smithi100.stdout:3650: finishing copy_from to smithi10066867-40 2024-04-07T16:28:08.023 INFO:tasks.rados.rados.0.smithi100.stdout:3650: got expected ENOENT (src dne) 2024-04-07T16:28:08.023 INFO:tasks.rados.rados.0.smithi100.stdout:3650: finishing copy_from racing read to smithi10066867-40 2024-04-07T16:28:08.023 INFO:tasks.rados.rados.0.smithi100.stdout:3648: finishing write tid 6 to smithi10066867-38 2024-04-07T16:28:08.023 INFO:tasks.rados.rados.0.smithi100.stdout:3648: oid 38 updating version 0 to 1708 2024-04-07T16:28:08.023 INFO:tasks.rados.rados.0.smithi100.stdout:3648: oid 38 updating version 1708 to 1711 2024-04-07T16:28:08.023 INFO:tasks.rados.rados.0.smithi100.stdout:3648: oid 38 version 1711 is already newer than 1709 2024-04-07T16:28:08.023 INFO:tasks.rados.rados.0.smithi100.stdout:3648: oid 38 version 1711 is already newer than 1710 2024-04-07T16:28:08.023 INFO:tasks.rados.rados.0.smithi100.stdout:3648: oid 38 updating version 1711 to 1712 2024-04-07T16:28:08.023 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 38 v 1712 (ObjNum 1134 snap 326 seq_num 1134) dirty exists 2024-04-07T16:28:08.023 INFO:tasks.rados.rados.0.smithi100.stdout:3648: left oid 38 (ObjNum 1134 snap 326 seq_num 1134) 2024-04-07T16:28:08.023 INFO:tasks.rados.rados.0.smithi100.stdout:3651: finishing copy_from racing read to smithi10066867-16 2024-04-07T16:28:08.023 INFO:tasks.rados.rados.0.smithi100.stdout:3651: finishing copy_from to smithi10066867-16 2024-04-07T16:28:08.023 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 16 v 1813 (ObjNum 1041 snap 307 seq_num 1041) dirty exists 2024-04-07T16:28:08.024 INFO:tasks.rados.rados.0.smithi100.stdout:3653: finishing rollback tid 0 to smithi10066867-43 2024-04-07T16:28:08.024 INFO:tasks.rados.rados.0.smithi100.stdout:3655: finishing rollback tid 1 to smithi10066867-1 2024-04-07T16:28:08.024 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 1 v 0 (ObjNum 433 snap 0 seq_num 4239677540) dirty dne 2024-04-07T16:28:08.024 INFO:tasks.rados.rados.0.smithi100.stdout:3653: finishing rollback tid 1 to smithi10066867-43 2024-04-07T16:28:08.024 INFO:tasks.rados.rados.0.smithi100.stdout:3653: finishing rollback tid 2 to smithi10066867-43 2024-04-07T16:28:08.024 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 43 v 1275 (ObjNum 1030 snap 306 seq_num 1030) dirty exists 2024-04-07T16:28:08.024 INFO:tasks.rados.rados.0.smithi100.stdout:3648: done (6 left) 2024-04-07T16:28:08.024 INFO:tasks.rados.rados.0.smithi100.stdout:3650: done (5 left) 2024-04-07T16:28:08.024 INFO:tasks.rados.rados.0.smithi100.stdout:3651: done (4 left) 2024-04-07T16:28:08.024 INFO:tasks.rados.rados.0.smithi100.stdout:3653: done (3 left) 2024-04-07T16:28:08.024 INFO:tasks.rados.rados.0.smithi100.stdout:3655: done (2 left) 2024-04-07T16:28:08.024 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 15 current snap is 326 2024-04-07T16:28:08.024 INFO:tasks.rados.rados.0.smithi100.stdout:3658: seq_num 1136 ranges {5922816=630784,6553600=409600,6963200=753664,7716864=417792,8134656=24576} 2024-04-07T16:28:08.065 INFO:tasks.rados.rados.0.smithi100.stdout:3658: writing smithi10066867-15 from 5922816 to 6553600 tid 1 2024-04-07T16:28:08.068 INFO:tasks.rados.rados.0.smithi100.stdout:3658: writing smithi10066867-15 from 6553600 to 6963200 tid 2 2024-04-07T16:28:08.074 INFO:tasks.rados.rados.0.smithi100.stdout:3658: writing smithi10066867-15 from 6963200 to 7716864 tid 3 2024-04-07T16:28:08.077 INFO:tasks.rados.rados.0.smithi100.stdout:3658: writing smithi10066867-15 from 7716864 to 8134656 tid 4 2024-04-07T16:28:08.078 INFO:tasks.rados.rados.0.smithi100.stdout:3658: writing smithi10066867-15 from 8134656 to 8159232 tid 5 2024-04-07T16:28:08.078 INFO:tasks.rados.rados.0.smithi100.stdout:3659: snap_create 2024-04-07T16:28:08.078 INFO:tasks.rados.rados.0.smithi100.stdout:3657: finishing rollback tid 0 to smithi10066867-29 2024-04-07T16:28:08.079 INFO:tasks.rados.rados.0.smithi100.stdout:3657: finishing rollback tid 1 to smithi10066867-29 2024-04-07T16:28:08.079 INFO:tasks.rados.rados.0.smithi100.stdout:3657: finishing rollback tid 2 to smithi10066867-29 2024-04-07T16:28:08.079 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 29 v 1582 (ObjNum 1083 snap 314 seq_num 1083) dirty exists 2024-04-07T16:28:08.079 INFO:tasks.rados.rados.0.smithi100.stdout:3654: finishing copy_from racing read to smithi10066867-45 2024-04-07T16:28:08.079 INFO:tasks.rados.rados.0.smithi100.stdout:3654: finishing copy_from to smithi10066867-45 2024-04-07T16:28:08.079 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 45 v 1200 (ObjNum 1087 snap 318 seq_num 1087) dirty exists 2024-04-07T16:28:08.079 INFO:tasks.rados.rados.0.smithi100.stdout:3658: finishing write tid 1 to smithi10066867-15 2024-04-07T16:28:08.084 INFO:tasks.rados.rados.0.smithi100.stdout:3658: finishing write tid 2 to smithi10066867-15 2024-04-07T16:28:08.088 INFO:tasks.rados.rados.0.smithi100.stdout:3658: finishing write tid 3 to smithi10066867-15 2024-04-07T16:28:08.088 INFO:tasks.rados.rados.0.smithi100.stdout:3658: finishing write tid 4 to smithi10066867-15 2024-04-07T16:28:08.088 INFO:tasks.rados.rados.0.smithi100.stdout:3658: finishing write tid 5 to smithi10066867-15 2024-04-07T16:28:08.088 INFO:tasks.rados.rados.0.smithi100.stdout:3658: finishing write tid 6 to smithi10066867-15 2024-04-07T16:28:08.091 INFO:tasks.rados.rados.0.smithi100.stdout:3658: finishing write tid 7 to smithi10066867-15 2024-04-07T16:28:08.091 INFO:tasks.rados.rados.0.smithi100.stdout:3658: oid 15 updating version 0 to 1890 2024-04-07T16:28:08.091 INFO:tasks.rados.rados.0.smithi100.stdout:3658: oid 15 updating version 1890 to 1891 2024-04-07T16:28:08.091 INFO:tasks.rados.rados.0.smithi100.stdout:3658: oid 15 version 1891 is already newer than 1886 2024-04-07T16:28:08.091 INFO:tasks.rados.rados.0.smithi100.stdout:3658: oid 15 version 1891 is already newer than 1889 2024-04-07T16:28:08.091 INFO:tasks.rados.rados.0.smithi100.stdout:3658: oid 15 version 1891 is already newer than 1887 2024-04-07T16:28:08.091 INFO:tasks.rados.rados.0.smithi100.stdout:3658: oid 15 version 1891 is already newer than 1888 2024-04-07T16:28:08.091 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 15 v 1891 (ObjNum 1136 snap 326 seq_num 1136) dirty exists 2024-04-07T16:28:08.091 INFO:tasks.rados.rados.0.smithi100.stdout:3658: left oid 15 (ObjNum 1136 snap 326 seq_num 1136) 2024-04-07T16:28:08.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:08 smithi144 bash[66654]: cluster 2024-04-07T16:28:06.808637+0000 mgr.y (mgr.25175) 375 : cluster [DBG] pgmap v686: 121 pgs: 121 active+clean; 108 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 21 op/s 2024-04-07T16:28:08.416 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:08 smithi100 bash[74942]: cluster 2024-04-07T16:28:06.808637+0000 mgr.y (mgr.25175) 375 : cluster [DBG] pgmap v686: 121 pgs: 121 active+clean; 108 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 21 op/s 2024-04-07T16:28:08.417 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:08 smithi100 bash[76202]: cluster 2024-04-07T16:28:06.808637+0000 mgr.y (mgr.25175) 375 : cluster [DBG] pgmap v686: 121 pgs: 121 active+clean; 108 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 21 op/s 2024-04-07T16:28:08.687 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:28:08 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:28:08] "GET /metrics HTTP/1.1" 200 36341 "" "Prometheus/2.43.0" 2024-04-07T16:28:09.014 INFO:tasks.rados.rados.0.smithi100.stdout:3654: done (3 left) 2024-04-07T16:28:09.014 INFO:tasks.rados.rados.0.smithi100.stdout:3657: done (2 left) 2024-04-07T16:28:09.015 INFO:tasks.rados.rados.0.smithi100.stdout:3658: done (1 left) 2024-04-07T16:28:09.015 INFO:tasks.rados.rados.0.smithi100.stdout:3659: done (0 left) 2024-04-07T16:28:09.015 INFO:tasks.rados.rados.0.smithi100.stdout:3660: delete oid 13 current snap is 327 2024-04-07T16:28:09.024 INFO:tasks.rados.rados.0.smithi100.stdout:3660: done (0 left) 2024-04-07T16:28:09.024 INFO:tasks.rados.rados.0.smithi100.stdout:3661: delete oid 2 current snap is 327 2024-04-07T16:28:09.036 INFO:tasks.rados.rados.0.smithi100.stdout:3661: done (0 left) 2024-04-07T16:28:09.036 INFO:tasks.rados.rados.0.smithi100.stdout:3662: snap_create 2024-04-07T16:28:09.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:09 smithi144 bash[66654]: cluster 2024-04-07T16:28:07.999337+0000 mon.a (mon.0) 1234 : cluster [DBG] osdmap e730: 8 total, 8 up, 8 in 2024-04-07T16:28:09.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:09 smithi100 bash[74942]: cluster 2024-04-07T16:28:07.999337+0000 mon.a (mon.0) 1234 : cluster [DBG] osdmap e730: 8 total, 8 up, 8 in 2024-04-07T16:28:09.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:09 smithi100 bash[76202]: cluster 2024-04-07T16:28:07.999337+0000 mon.a (mon.0) 1234 : cluster [DBG] osdmap e730: 8 total, 8 up, 8 in 2024-04-07T16:28:10.016 INFO:tasks.rados.rados.0.smithi100.stdout:3662: done (0 left) 2024-04-07T16:28:10.017 INFO:tasks.rados.rados.0.smithi100.stdout:3663: read oid 49 snap -1 2024-04-07T16:28:10.017 INFO:tasks.rados.rados.0.smithi100.stdout:3663: expect (ObjNum 0 snap 0 seq_num 0) 2024-04-07T16:28:10.017 INFO:tasks.rados.rados.0.smithi100.stdout:3664: rmattr oid 8 current snap is 328 2024-04-07T16:28:10.017 INFO:tasks.rados.rados.0.smithi100.stdout:3664: done (1 left) 2024-04-07T16:28:10.017 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 35 current snap is 328 2024-04-07T16:28:10.017 INFO:tasks.rados.rados.0.smithi100.stdout:3665: seq_num 1139 ranges {1662976=679936,2342912=458752,2801664=524288,3325952=540672,3866624=81920} 2024-04-07T16:28:10.036 INFO:tasks.rados.rados.0.smithi100.stdout:3665: writing smithi10066867-35 from 1662976 to 2342912 tid 1 2024-04-07T16:28:10.040 INFO:tasks.rados.rados.0.smithi100.stdout:3665: writing smithi10066867-35 from 2342912 to 2801664 tid 2 2024-04-07T16:28:10.046 INFO:tasks.rados.rados.0.smithi100.stdout:3665: writing smithi10066867-35 from 2801664 to 3325952 tid 3 2024-04-07T16:28:10.051 INFO:tasks.rados.rados.0.smithi100.stdout:3665: writing smithi10066867-35 from 3325952 to 3866624 tid 4 2024-04-07T16:28:10.052 INFO:tasks.rados.rados.0.smithi100.stdout:3665: writing smithi10066867-35 from 3866624 to 3948544 tid 5 2024-04-07T16:28:10.053 INFO:tasks.rados.rados.0.smithi100.stdout:3666: rollback oid 29 current snap is 328 2024-04-07T16:28:10.053 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 29 to 327 2024-04-07T16:28:10.053 INFO:tasks.rados.rados.0.smithi100.stdout:3663: expect (ObjNum 0 snap 0 seq_num 0) 2024-04-07T16:28:10.058 INFO:tasks.rados.rados.0.smithi100.stdout:3665: finishing write tid 1 to smithi10066867-35 2024-04-07T16:28:10.058 INFO:tasks.rados.rados.0.smithi100.stdout:3663: done (2 left) 2024-04-07T16:28:10.058 INFO:tasks.rados.rados.0.smithi100.stdout:3667: rmattr oid 6 current snap is 328 2024-04-07T16:28:10.058 INFO:tasks.rados.rados.0.smithi100.stdout:3667: done (2 left) 2024-04-07T16:28:10.058 INFO:tasks.rados.rados.0.smithi100.stdout:3668: read oid 17 snap -1 2024-04-07T16:28:10.059 INFO:tasks.rados.rados.0.smithi100.stdout:3668: expect deleted 2024-04-07T16:28:10.059 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 48 current snap is 328 2024-04-07T16:28:10.059 INFO:tasks.rados.rados.0.smithi100.stdout:3669: seq_num 1140 ranges {3522560=696320,4218880=745472,4964352=729088,5693440=131072} 2024-04-07T16:28:10.092 INFO:tasks.rados.rados.0.smithi100.stdout:3669: writing smithi10066867-48 from 3522560 to 4218880 tid 1 2024-04-07T16:28:10.097 INFO:tasks.rados.rados.0.smithi100.stdout:3669: writing smithi10066867-48 from 4218880 to 4964352 tid 2 2024-04-07T16:28:10.102 INFO:tasks.rados.rados.0.smithi100.stdout:3669: writing smithi10066867-48 from 4964352 to 5693440 tid 3 2024-04-07T16:28:10.104 INFO:tasks.rados.rados.0.smithi100.stdout:3669: writing smithi10066867-48 from 5693440 to 5824512 tid 4 2024-04-07T16:28:10.104 INFO:tasks.rados.rados.0.smithi100.stdout:3670: snap_create 2024-04-07T16:28:10.104 INFO:tasks.rados.rados.0.smithi100.stdout:3665: finishing write tid 2 to smithi10066867-35 2024-04-07T16:28:10.104 INFO:tasks.rados.rados.0.smithi100.stdout:3666: finishing rollback tid 0 to smithi10066867-29 2024-04-07T16:28:10.104 INFO:tasks.rados.rados.0.smithi100.stdout:3665: finishing write tid 3 to smithi10066867-35 2024-04-07T16:28:10.104 INFO:tasks.rados.rados.0.smithi100.stdout:3666: finishing rollback tid 1 to smithi10066867-29 2024-04-07T16:28:10.104 INFO:tasks.rados.rados.0.smithi100.stdout:3666: finishing rollback tid 2 to smithi10066867-29 2024-04-07T16:28:10.104 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 29 v 1585 (ObjNum 1083 snap 314 seq_num 1083) dirty exists 2024-04-07T16:28:10.104 INFO:tasks.rados.rados.0.smithi100.stdout:3665: finishing write tid 4 to smithi10066867-35 2024-04-07T16:28:10.104 INFO:tasks.rados.rados.0.smithi100.stdout:3665: finishing write tid 5 to smithi10066867-35 2024-04-07T16:28:10.105 INFO:tasks.rados.rados.0.smithi100.stdout:3665: finishing write tid 6 to smithi10066867-35 2024-04-07T16:28:10.105 INFO:tasks.rados.rados.0.smithi100.stdout:3665: finishing write tid 7 to smithi10066867-35 2024-04-07T16:28:10.105 INFO:tasks.rados.rados.0.smithi100.stdout:3665: oid 35 updating version 0 to 1740 2024-04-07T16:28:10.105 INFO:tasks.rados.rados.0.smithi100.stdout:3665: oid 35 version 1740 is already newer than 1738 2024-04-07T16:28:10.105 INFO:tasks.rados.rados.0.smithi100.stdout:3665: oid 35 updating version 1740 to 1741 2024-04-07T16:28:10.105 INFO:tasks.rados.rados.0.smithi100.stdout:3665: oid 35 version 1741 is already newer than 1737 2024-04-07T16:28:10.105 INFO:tasks.rados.rados.0.smithi100.stdout:3665: oid 35 version 1741 is already newer than 1739 2024-04-07T16:28:10.105 INFO:tasks.rados.rados.0.smithi100.stdout:3665: oid 35 updating version 1741 to 1742 2024-04-07T16:28:10.105 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 35 v 1742 (ObjNum 1139 snap 328 seq_num 1139) dirty exists 2024-04-07T16:28:10.105 INFO:tasks.rados.rados.0.smithi100.stdout:3665: left oid 35 (ObjNum 1139 snap 328 seq_num 1139) 2024-04-07T16:28:10.105 INFO:tasks.rados.rados.0.smithi100.stdout:3669: finishing write tid 1 to smithi10066867-48 2024-04-07T16:28:10.105 INFO:tasks.rados.rados.0.smithi100.stdout:3669: finishing write tid 2 to smithi10066867-48 2024-04-07T16:28:10.111 INFO:tasks.rados.rados.0.smithi100.stdout:3669: finishing write tid 3 to smithi10066867-48 2024-04-07T16:28:10.111 INFO:tasks.rados.rados.0.smithi100.stdout:3669: finishing write tid 4 to smithi10066867-48 2024-04-07T16:28:10.111 INFO:tasks.rados.rados.0.smithi100.stdout:3669: finishing write tid 5 to smithi10066867-48 2024-04-07T16:28:10.114 INFO:tasks.rados.rados.0.smithi100.stdout:3669: finishing write tid 6 to smithi10066867-48 2024-04-07T16:28:10.114 INFO:tasks.rados.rados.0.smithi100.stdout:3669: oid 48 updating version 0 to 1744 2024-04-07T16:28:10.114 INFO:tasks.rados.rados.0.smithi100.stdout:3669: oid 48 updating version 1744 to 1746 2024-04-07T16:28:10.114 INFO:tasks.rados.rados.0.smithi100.stdout:3669: oid 48 version 1746 is already newer than 1743 2024-04-07T16:28:10.114 INFO:tasks.rados.rados.0.smithi100.stdout:3669: oid 48 version 1746 is already newer than 1745 2024-04-07T16:28:10.114 INFO:tasks.rados.rados.0.smithi100.stdout:3669: oid 48 updating version 1746 to 1747 2024-04-07T16:28:10.114 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 48 v 1747 (ObjNum 1140 snap 328 seq_num 1140) dirty exists 2024-04-07T16:28:10.114 INFO:tasks.rados.rados.0.smithi100.stdout:3669: left oid 48 (ObjNum 1140 snap 328 seq_num 1140) 2024-04-07T16:28:10.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:10 smithi144 bash[66654]: cluster 2024-04-07T16:28:08.809497+0000 mgr.y (mgr.25175) 376 : cluster [DBG] pgmap v689: 121 pgs: 2 active+clean+snaptrim, 119 active+clean; 117 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 4.8 MiB/s rd, 2.9 MiB/s wr, 11 op/s 2024-04-07T16:28:10.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:10 smithi144 bash[66654]: cluster 2024-04-07T16:28:09.028460+0000 mon.a (mon.0) 1235 : cluster [DBG] osdmap e731: 8 total, 8 up, 8 in 2024-04-07T16:28:10.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:10 smithi100 bash[74942]: cluster 2024-04-07T16:28:08.809497+0000 mgr.y (mgr.25175) 376 : cluster [DBG] pgmap v689: 121 pgs: 2 active+clean+snaptrim, 119 active+clean; 117 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 4.8 MiB/s rd, 2.9 MiB/s wr, 11 op/s 2024-04-07T16:28:10.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:10 smithi100 bash[74942]: cluster 2024-04-07T16:28:09.028460+0000 mon.a (mon.0) 1235 : cluster [DBG] osdmap e731: 8 total, 8 up, 8 in 2024-04-07T16:28:10.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:10 smithi100 bash[76202]: cluster 2024-04-07T16:28:08.809497+0000 mgr.y (mgr.25175) 376 : cluster [DBG] pgmap v689: 121 pgs: 2 active+clean+snaptrim, 119 active+clean; 117 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 4.8 MiB/s rd, 2.9 MiB/s wr, 11 op/s 2024-04-07T16:28:10.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:10 smithi100 bash[76202]: cluster 2024-04-07T16:28:09.028460+0000 mon.a (mon.0) 1235 : cluster [DBG] osdmap e731: 8 total, 8 up, 8 in 2024-04-07T16:28:11.039 INFO:tasks.rados.rados.0.smithi100.stdout:3665: done (4 left) 2024-04-07T16:28:11.039 INFO:tasks.rados.rados.0.smithi100.stdout:3666: done (3 left) 2024-04-07T16:28:11.039 INFO:tasks.rados.rados.0.smithi100.stdout:3668: done (2 left) 2024-04-07T16:28:11.039 INFO:tasks.rados.rados.0.smithi100.stdout:3669: done (1 left) 2024-04-07T16:28:11.039 INFO:tasks.rados.rados.0.smithi100.stdout:3670: done (0 left) 2024-04-07T16:28:11.040 INFO:tasks.rados.rados.0.smithi100.stdout:3671: snap_create 2024-04-07T16:28:11.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:11 smithi144 bash[66654]: cluster 2024-04-07T16:28:10.031020+0000 mon.a (mon.0) 1236 : cluster [DBG] osdmap e732: 8 total, 8 up, 8 in 2024-04-07T16:28:11.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:11 smithi100 bash[74942]: cluster 2024-04-07T16:28:10.031020+0000 mon.a (mon.0) 1236 : cluster [DBG] osdmap e732: 8 total, 8 up, 8 in 2024-04-07T16:28:11.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:11 smithi100 bash[76202]: cluster 2024-04-07T16:28:10.031020+0000 mon.a (mon.0) 1236 : cluster [DBG] osdmap e732: 8 total, 8 up, 8 in 2024-04-07T16:28:11.938 INFO:tasks.rados.rados.0.smithi100.stdout:3671: done (0 left) 2024-04-07T16:28:11.938 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 32 current snap is 330 2024-04-07T16:28:11.938 INFO:tasks.rados.rados.0.smithi100.stdout:3672: seq_num 1141 ranges {0=712704,712704=679936,1392640=540672,1933312=385024} 2024-04-07T16:28:11.945 INFO:tasks.rados.rados.0.smithi100.stdout:3672: writing smithi10066867-32 from 0 to 712704 tid 1 2024-04-07T16:28:11.951 INFO:tasks.rados.rados.0.smithi100.stdout:3672: writing smithi10066867-32 from 712704 to 1392640 tid 2 2024-04-07T16:28:11.956 INFO:tasks.rados.rados.0.smithi100.stdout:3672: writing smithi10066867-32 from 1392640 to 1933312 tid 3 2024-04-07T16:28:11.960 INFO:tasks.rados.rados.0.smithi100.stdout:3672: writing smithi10066867-32 from 1933312 to 2318336 tid 4 2024-04-07T16:28:11.961 INFO:tasks.rados.rados.0.smithi100.stdout:3673: read oid 50 snap 326 2024-04-07T16:28:11.961 INFO:tasks.rados.rados.0.smithi100.stdout:3673: expect deleted 2024-04-07T16:28:11.961 INFO:tasks.rados.rados.0.smithi100.stdout:3674: copy_from oid 45 from oid 16 current snap is 330 2024-04-07T16:28:11.961 INFO:tasks.rados.rados.0.smithi100.stdout:3675: snap_create 2024-04-07T16:28:11.965 INFO:tasks.rados.rados.0.smithi100.stdout:3672: finishing write tid 1 to smithi10066867-32 2024-04-07T16:28:11.967 INFO:tasks.rados.rados.0.smithi100.stdout:3672: finishing write tid 2 to smithi10066867-32 2024-04-07T16:28:11.970 INFO:tasks.rados.rados.0.smithi100.stdout:3672: finishing write tid 3 to smithi10066867-32 2024-04-07T16:28:11.971 INFO:tasks.rados.rados.0.smithi100.stdout:3672: finishing write tid 4 to smithi10066867-32 2024-04-07T16:28:11.972 INFO:tasks.rados.rados.0.smithi100.stdout:3672: finishing write tid 5 to smithi10066867-32 2024-04-07T16:28:11.974 INFO:tasks.rados.rados.0.smithi100.stdout:3672: finishing write tid 6 to smithi10066867-32 2024-04-07T16:28:11.974 INFO:tasks.rados.rados.0.smithi100.stdout:3672: oid 32 updating version 0 to 1701 2024-04-07T16:28:11.974 INFO:tasks.rados.rados.0.smithi100.stdout:3672: oid 32 updating version 1701 to 1702 2024-04-07T16:28:11.974 INFO:tasks.rados.rados.0.smithi100.stdout:3672: oid 32 version 1702 is already newer than 1700 2024-04-07T16:28:11.974 INFO:tasks.rados.rados.0.smithi100.stdout:3672: oid 32 version 1702 is already newer than 1699 2024-04-07T16:28:11.974 INFO:tasks.rados.rados.0.smithi100.stdout:3672: oid 32 version 1702 is already newer than 1698 2024-04-07T16:28:11.974 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 32 v 1702 (ObjNum 1141 snap 330 seq_num 1141) dirty exists 2024-04-07T16:28:11.974 INFO:tasks.rados.rados.0.smithi100.stdout:3672: left oid 32 (ObjNum 1141 snap 330 seq_num 1141) 2024-04-07T16:28:11.975 INFO:tasks.rados.rados.0.smithi100.stdout:3674: finishing copy_from racing read to smithi10066867-45 2024-04-07T16:28:11.979 INFO:tasks.rados.rados.0.smithi100.stdout:3674: finishing copy_from to smithi10066867-45 2024-04-07T16:28:11.979 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 45 v 1813 (ObjNum 1041 snap 307 seq_num 1041) dirty exists 2024-04-07T16:28:12.304 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:12 smithi144 bash[66654]: cluster 2024-04-07T16:28:10.810196+0000 mgr.y (mgr.25175) 377 : cluster [DBG] pgmap v692: 121 pgs: 2 active+clean+snaptrim, 119 active+clean; 130 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 12 MiB/s rd, 5.9 MiB/s wr, 23 op/s 2024-04-07T16:28:12.304 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:12 smithi144 bash[66654]: cluster 2024-04-07T16:28:11.052324+0000 mon.a (mon.0) 1237 : cluster [DBG] osdmap e733: 8 total, 8 up, 8 in 2024-04-07T16:28:12.304 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:12 smithi144 bash[66654]: cluster 2024-04-07T16:28:11.951644+0000 mon.a (mon.0) 1238 : cluster [DBG] osdmap e734: 8 total, 8 up, 8 in 2024-04-07T16:28:12.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:12 smithi100 bash[74942]: cluster 2024-04-07T16:28:10.810196+0000 mgr.y (mgr.25175) 377 : cluster [DBG] pgmap v692: 121 pgs: 2 active+clean+snaptrim, 119 active+clean; 130 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 12 MiB/s rd, 5.9 MiB/s wr, 23 op/s 2024-04-07T16:28:12.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:12 smithi100 bash[74942]: cluster 2024-04-07T16:28:11.052324+0000 mon.a (mon.0) 1237 : cluster [DBG] osdmap e733: 8 total, 8 up, 8 in 2024-04-07T16:28:12.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:12 smithi100 bash[74942]: cluster 2024-04-07T16:28:11.951644+0000 mon.a (mon.0) 1238 : cluster [DBG] osdmap e734: 8 total, 8 up, 8 in 2024-04-07T16:28:12.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:12 smithi100 bash[76202]: cluster 2024-04-07T16:28:10.810196+0000 mgr.y (mgr.25175) 377 : cluster [DBG] pgmap v692: 121 pgs: 2 active+clean+snaptrim, 119 active+clean; 130 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 12 MiB/s rd, 5.9 MiB/s wr, 23 op/s 2024-04-07T16:28:12.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:12 smithi100 bash[76202]: cluster 2024-04-07T16:28:11.052324+0000 mon.a (mon.0) 1237 : cluster [DBG] osdmap e733: 8 total, 8 up, 8 in 2024-04-07T16:28:12.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:12 smithi100 bash[76202]: cluster 2024-04-07T16:28:11.951644+0000 mon.a (mon.0) 1238 : cluster [DBG] osdmap e734: 8 total, 8 up, 8 in 2024-04-07T16:28:12.566 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:28:12 smithi144 bash[95233]: ts=2024-04-07T16:28:12.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:28:12.607 INFO:teuthology.orchestra.run.smithi100.stdout:true 2024-04-07T16:28:12.941 INFO:tasks.rados.rados.0.smithi100.stdout:3672: done (3 left) 2024-04-07T16:28:12.941 INFO:tasks.rados.rados.0.smithi100.stdout:3673: done (2 left) 2024-04-07T16:28:12.941 INFO:tasks.rados.rados.0.smithi100.stdout:3674: done (1 left) 2024-04-07T16:28:12.941 INFO:tasks.rados.rados.0.smithi100.stdout:3675: done (0 left) 2024-04-07T16:28:12.941 INFO:tasks.rados.rados.0.smithi100.stdout:3676: read oid 34 snap -1 2024-04-07T16:28:12.941 INFO:tasks.rados.rados.0.smithi100.stdout:3676: expect (ObjNum 1012 snap 300 seq_num 1012) 2024-04-07T16:28:12.941 INFO:tasks.rados.rados.0.smithi100.stdout:3677: copy_from oid 6 from oid 50 current snap is 331 2024-04-07T16:28:12.941 INFO:tasks.rados.rados.0.smithi100.stdout:3678: snap_create 2024-04-07T16:28:12.946 INFO:tasks.rados.rados.0.smithi100.stdout:3677: finishing copy_from to smithi10066867-6 2024-04-07T16:28:12.946 INFO:tasks.rados.rados.0.smithi100.stdout:3677: got expected ENOENT (src dne) 2024-04-07T16:28:12.947 INFO:tasks.rados.rados.0.smithi100.stdout:3677: finishing copy_from racing read to smithi10066867-6 2024-04-07T16:28:12.988 INFO:tasks.rados.rados.0.smithi100.stdout:3676: expect (ObjNum 1012 snap 300 seq_num 1012) 2024-04-07T16:28:13.004 INFO:teuthology.orchestra.run.smithi100.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-07T16:28:13.004 INFO:teuthology.orchestra.run.smithi100.stdout:alertmanager.a smithi100 *:9093,9094 running (32s) 26s ago 15m 13.9M - 0.25.0 c8568f914cd2 67b26d69aca6 2024-04-07T16:28:13.004 INFO:teuthology.orchestra.run.smithi100.stdout:grafana.a smithi144 *:3000 running (14m) 51s ago 14m 79.4M - 9.4.7 954c08fa6188 7180c0708acd 2024-04-07T16:28:13.004 INFO:teuthology.orchestra.run.smithi100.stdout:mds.a smithi100 running (2m) 26s ago 16m 13.8M - 19.0.0-2785-gadce92d0 c0ecd4226ac9 075552e67bd0 2024-04-07T16:28:13.004 INFO:teuthology.orchestra.run.smithi100.stdout:mds.b smithi144 running (107s) 51s ago 16m 11.7M - 19.0.0-2785-gadce92d0 c0ecd4226ac9 fc737f3b1edc 2024-04-07T16:28:13.004 INFO:teuthology.orchestra.run.smithi100.stdout:mgr.x smithi144 *:8443,9283,8765 running (8m) 51s ago 23m 441M - 19.0.0-2785-gadce92d0 c0ecd4226ac9 233745c292a6 2024-04-07T16:28:13.004 INFO:teuthology.orchestra.run.smithi100.stdout:mgr.y smithi100 *:8443,9283,8765 running (9m) 26s ago 25m 509M - 19.0.0-2785-gadce92d0 c0ecd4226ac9 254dc65cdaa7 2024-04-07T16:28:13.004 INFO:teuthology.orchestra.run.smithi100.stdout:mon.a smithi100 running (8m) 26s ago 25m 70.9M 2048M 19.0.0-2785-gadce92d0 c0ecd4226ac9 718e12655c04 2024-04-07T16:28:13.005 INFO:teuthology.orchestra.run.smithi100.stdout:mon.b smithi144 running (7m) 51s ago 23m 54.6M 2048M 19.0.0-2785-gadce92d0 c0ecd4226ac9 aef817664bff 2024-04-07T16:28:13.005 INFO:teuthology.orchestra.run.smithi100.stdout:mon.c smithi100 running (7m) 26s ago 23m 58.9M 2048M 19.0.0-2785-gadce92d0 c0ecd4226ac9 63e889040f6e 2024-04-07T16:28:13.005 INFO:teuthology.orchestra.run.smithi100.stdout:node-exporter.a smithi100 *:9100 running (85s) 26s ago 15m 8043k - 1.5.0 0da6a335fe13 9609d3a1c1ba 2024-04-07T16:28:13.005 INFO:teuthology.orchestra.run.smithi100.stdout:node-exporter.b smithi144 *:9100 running (81s) 51s ago 15m 7348k - 1.5.0 0da6a335fe13 abbe3a5b7ce6 2024-04-07T16:28:13.005 INFO:teuthology.orchestra.run.smithi100.stdout:osd.0 smithi100 running (6m) 26s ago 22m 380M 2524M 19.0.0-2785-gadce92d0 c0ecd4226ac9 a6e83c9d1abc 2024-04-07T16:28:13.005 INFO:teuthology.orchestra.run.smithi100.stdout:osd.1 smithi100 running (5m) 26s ago 21m 342M 2524M 19.0.0-2785-gadce92d0 c0ecd4226ac9 c50fe40079b6 2024-04-07T16:28:13.005 INFO:teuthology.orchestra.run.smithi100.stdout:osd.2 smithi100 running (5m) 26s ago 20m 388M 2524M 19.0.0-2785-gadce92d0 c0ecd4226ac9 8e631c43064b 2024-04-07T16:28:13.005 INFO:teuthology.orchestra.run.smithi100.stdout:osd.3 smithi100 running (4m) 26s ago 20m 169M 2524M 19.0.0-2785-gadce92d0 c0ecd4226ac9 402931247c79 2024-04-07T16:28:13.005 INFO:teuthology.orchestra.run.smithi100.stdout:osd.4 smithi144 running (4m) 51s ago 19m 345M 2524M 19.0.0-2785-gadce92d0 c0ecd4226ac9 ebbb7be95d33 2024-04-07T16:28:13.005 INFO:teuthology.orchestra.run.smithi100.stdout:osd.5 smithi144 running (3m) 51s ago 18m 222M 2524M 19.0.0-2785-gadce92d0 c0ecd4226ac9 ecce27bcb2f8 2024-04-07T16:28:13.005 INFO:teuthology.orchestra.run.smithi100.stdout:osd.6 smithi144 running (3m) 51s ago 17m 111M 2524M 19.0.0-2785-gadce92d0 c0ecd4226ac9 3788c5c3f427 2024-04-07T16:28:13.005 INFO:teuthology.orchestra.run.smithi100.stdout:osd.7 smithi144 running (2m) 51s ago 16m 193M 2524M 19.0.0-2785-gadce92d0 c0ecd4226ac9 9365aba97dc0 2024-04-07T16:28:13.005 INFO:teuthology.orchestra.run.smithi100.stdout:prometheus.a smithi144 *:9095 running (57s) 51s ago 15m 35.9M - 2.43.0 a07b618ecd1d e7d72b7c75c4 2024-04-07T16:28:13.444 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-04-07T16:28:13.444 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": { 2024-04-07T16:28:13.445 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph version 19.0.0-2785-gadce92d0 (adce92d056533f22dc1ee37e3141d098b046ec1b) squid (dev)": 3 2024-04-07T16:28:13.445 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-07T16:28:13.445 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": { 2024-04-07T16:28:13.445 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph version 19.0.0-2785-gadce92d0 (adce92d056533f22dc1ee37e3141d098b046ec1b) squid (dev)": 2 2024-04-07T16:28:13.445 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-07T16:28:13.445 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": { 2024-04-07T16:28:13.445 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph version 19.0.0-2785-gadce92d0 (adce92d056533f22dc1ee37e3141d098b046ec1b) squid (dev)": 8 2024-04-07T16:28:13.445 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-07T16:28:13.445 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": { 2024-04-07T16:28:13.445 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph version 19.0.0-2785-gadce92d0 (adce92d056533f22dc1ee37e3141d098b046ec1b) squid (dev)": 2 2024-04-07T16:28:13.445 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-07T16:28:13.445 INFO:teuthology.orchestra.run.smithi100.stdout: "overall": { 2024-04-07T16:28:13.445 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph version 19.0.0-2785-gadce92d0 (adce92d056533f22dc1ee37e3141d098b046ec1b) squid (dev)": 15 2024-04-07T16:28:13.445 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-04-07T16:28:13.446 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-04-07T16:28:13.947 INFO:tasks.rados.rados.0.smithi100.stdout:3676: done (2 left) 2024-04-07T16:28:13.947 INFO:tasks.rados.rados.0.smithi100.stdout:3677: done (1 left) 2024-04-07T16:28:13.947 INFO:tasks.rados.rados.0.smithi100.stdout:3678: done (0 left) 2024-04-07T16:28:13.947 INFO:tasks.rados.rados.0.smithi100.stdout:3679: snap_remove snap 328 2024-04-07T16:28:14.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:13 smithi144 bash[66654]: audit 2024-04-07T16:28:12.607348+0000 mgr.y (mgr.25175) 378 : audit [DBG] from='client.34482 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:28:14.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:13 smithi144 bash[66654]: cluster 2024-04-07T16:28:12.811304+0000 mgr.y (mgr.25175) 379 : cluster [DBG] pgmap v695: 121 pgs: 2 active+clean+snaptrim, 119 active+clean; 136 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 7.9 MiB/s rd, 5.2 MiB/s wr, 16 op/s 2024-04-07T16:28:14.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:13 smithi144 bash[66654]: cluster 2024-04-07T16:28:12.953720+0000 mon.a (mon.0) 1239 : cluster [DBG] osdmap e735: 8 total, 8 up, 8 in 2024-04-07T16:28:14.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:13 smithi144 bash[66654]: audit 2024-04-07T16:28:12.974123+0000 mgr.y (mgr.25175) 380 : audit [DBG] from='client.34488 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:28:14.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:13 smithi144 bash[66654]: audit 2024-04-07T16:28:13.459146+0000 mon.a (mon.0) 1240 : audit [DBG] from='client.? 172.21.15.100:0/3965038655' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:28:14.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:13 smithi100 bash[74942]: audit 2024-04-07T16:28:12.607348+0000 mgr.y (mgr.25175) 378 : audit [DBG] from='client.34482 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:28:14.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:13 smithi100 bash[74942]: cluster 2024-04-07T16:28:12.811304+0000 mgr.y (mgr.25175) 379 : cluster [DBG] pgmap v695: 121 pgs: 2 active+clean+snaptrim, 119 active+clean; 136 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 7.9 MiB/s rd, 5.2 MiB/s wr, 16 op/s 2024-04-07T16:28:14.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:13 smithi100 bash[74942]: cluster 2024-04-07T16:28:12.953720+0000 mon.a (mon.0) 1239 : cluster [DBG] osdmap e735: 8 total, 8 up, 8 in 2024-04-07T16:28:14.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:13 smithi100 bash[74942]: audit 2024-04-07T16:28:12.974123+0000 mgr.y (mgr.25175) 380 : audit [DBG] from='client.34488 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:28:14.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:13 smithi100 bash[74942]: audit 2024-04-07T16:28:13.459146+0000 mon.a (mon.0) 1240 : audit [DBG] from='client.? 172.21.15.100:0/3965038655' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:28:14.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:13 smithi100 bash[76202]: audit 2024-04-07T16:28:12.607348+0000 mgr.y (mgr.25175) 378 : audit [DBG] from='client.34482 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:28:14.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:13 smithi100 bash[76202]: cluster 2024-04-07T16:28:12.811304+0000 mgr.y (mgr.25175) 379 : cluster [DBG] pgmap v695: 121 pgs: 2 active+clean+snaptrim, 119 active+clean; 136 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 7.9 MiB/s rd, 5.2 MiB/s wr, 16 op/s 2024-04-07T16:28:14.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:13 smithi100 bash[76202]: cluster 2024-04-07T16:28:12.953720+0000 mon.a (mon.0) 1239 : cluster [DBG] osdmap e735: 8 total, 8 up, 8 in 2024-04-07T16:28:14.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:13 smithi100 bash[76202]: audit 2024-04-07T16:28:12.974123+0000 mgr.y (mgr.25175) 380 : audit [DBG] from='client.34488 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:28:14.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:13 smithi100 bash[76202]: audit 2024-04-07T16:28:13.459146+0000 mon.a (mon.0) 1240 : audit [DBG] from='client.? 172.21.15.100:0/3965038655' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:28:14.855 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:28:14 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:28:14.856 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:14 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:28:14.856 INFO:journalctl@ceph.osd.7.smithi144.stdout:Apr 07 16:28:14 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:28:14.856 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:28:14 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:28:14.857 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:28:14 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:28:14.857 INFO:journalctl@ceph.mds.b.smithi144.stdout:Apr 07 16:28:14 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:28:14.857 INFO:journalctl@ceph.osd.6.smithi144.stdout:Apr 07 16:28:14 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:28:14.858 INFO:journalctl@ceph.osd.5.smithi144.stdout:Apr 07 16:28:14 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:28:14.858 INFO:journalctl@ceph.osd.4.smithi144.stdout:Apr 07 16:28:14 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:28:14.858 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 16:28:14 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:28:14.945 INFO:tasks.rados.rados.0.smithi100.stdout:3679: done (0 left) 2024-04-07T16:28:14.945 INFO:tasks.rados.rados.0.smithi100.stdout:3680: snap_create 2024-04-07T16:28:15.173 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:14 smithi144 bash[66654]: cluster 2024-04-07T16:28:13.957508+0000 mon.a (mon.0) 1241 : cluster [DBG] osdmap e736: 8 total, 8 up, 8 in 2024-04-07T16:28:15.174 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:14 smithi144 bash[66654]: cluster 2024-04-07T16:28:14.958506+0000 mon.a (mon.0) 1242 : cluster [DBG] osdmap e737: 8 total, 8 up, 8 in 2024-04-07T16:28:15.174 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:28:14 smithi144 systemd[1]: Stopping Ceph grafana.a for 0702e3a8-f4f8-11ee-b649-cb9ed24678a4... 2024-04-07T16:28:15.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:14 smithi100 bash[74942]: cluster 2024-04-07T16:28:13.957508+0000 mon.a (mon.0) 1241 : cluster [DBG] osdmap e736: 8 total, 8 up, 8 in 2024-04-07T16:28:15.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:14 smithi100 bash[74942]: cluster 2024-04-07T16:28:14.958506+0000 mon.a (mon.0) 1242 : cluster [DBG] osdmap e737: 8 total, 8 up, 8 in 2024-04-07T16:28:15.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:14 smithi100 bash[76202]: cluster 2024-04-07T16:28:13.957508+0000 mon.a (mon.0) 1241 : cluster [DBG] osdmap e736: 8 total, 8 up, 8 in 2024-04-07T16:28:15.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:14 smithi100 bash[76202]: cluster 2024-04-07T16:28:14.958506+0000 mon.a (mon.0) 1242 : cluster [DBG] osdmap e737: 8 total, 8 up, 8 in 2024-04-07T16:28:15.846 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:28:15 smithi144 bash[98751]: ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4-grafana-a 2024-04-07T16:28:15.846 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:28:15 smithi144 bash[98811]: ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4-grafana-a 2024-04-07T16:28:15.847 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:28:15 smithi144 systemd[1]: ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@grafana.a.service: Main process exited, code=exited, status=143/n/a 2024-04-07T16:28:15.847 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:28:15 smithi144 systemd[1]: ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@grafana.a.service: Failed with result 'exit-code'. 2024-04-07T16:28:15.847 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:28:15 smithi144 systemd[1]: Stopped Ceph grafana.a for 0702e3a8-f4f8-11ee-b649-cb9ed24678a4. 2024-04-07T16:28:15.947 INFO:tasks.rados.rados.0.smithi100.stdout:3680: done (0 left) 2024-04-07T16:28:15.947 INFO:tasks.rados.rados.0.smithi100.stdout:3681: snap_create 2024-04-07T16:28:16.105 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 16:28:15 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:28:16.106 INFO:journalctl@ceph.osd.7.smithi144.stdout:Apr 07 16:28:15 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:28:16.106 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:28:15 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:28:16.106 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:28:15 smithi144 systemd[1]: Started Ceph grafana.a for 0702e3a8-f4f8-11ee-b649-cb9ed24678a4. 2024-04-07T16:28:16.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:15 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:28:16.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:15 smithi144 bash[66654]: cluster 2024-04-07T16:28:14.812143+0000 mgr.y (mgr.25175) 381 : cluster [DBG] pgmap v698: 121 pgs: 121 active+clean; 141 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 4.6 MiB/s rd, 3.2 MiB/s wr, 10 op/s 2024-04-07T16:28:16.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:15 smithi144 bash[66654]: cluster 2024-04-07T16:28:15.960509+0000 mon.a (mon.0) 1243 : cluster [DBG] osdmap e738: 8 total, 8 up, 8 in 2024-04-07T16:28:16.107 INFO:journalctl@ceph.osd.6.smithi144.stdout:Apr 07 16:28:15 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:28:16.107 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:28:15 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:28:16.107 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 16:28:15 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:28:16.108 INFO:journalctl@ceph.osd.4.smithi144.stdout:Apr 07 16:28:15 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:28:16.108 INFO:journalctl@ceph.mds.b.smithi144.stdout:Apr 07 16:28:15 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:28:16.108 INFO:journalctl@ceph.osd.5.smithi144.stdout:Apr 07 16:28:15 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T16:28:16.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:15 smithi100 bash[74942]: cluster 2024-04-07T16:28:14.812143+0000 mgr.y (mgr.25175) 381 : cluster [DBG] pgmap v698: 121 pgs: 121 active+clean; 141 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 4.6 MiB/s rd, 3.2 MiB/s wr, 10 op/s 2024-04-07T16:28:16.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:15 smithi100 bash[74942]: cluster 2024-04-07T16:28:15.960509+0000 mon.a (mon.0) 1243 : cluster [DBG] osdmap e738: 8 total, 8 up, 8 in 2024-04-07T16:28:16.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:15 smithi100 bash[76202]: cluster 2024-04-07T16:28:14.812143+0000 mgr.y (mgr.25175) 381 : cluster [DBG] pgmap v698: 121 pgs: 121 active+clean; 141 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 4.6 MiB/s rd, 3.2 MiB/s wr, 10 op/s 2024-04-07T16:28:16.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:15 smithi100 bash[76202]: cluster 2024-04-07T16:28:15.960509+0000 mon.a (mon.0) 1243 : cluster [DBG] osdmap e738: 8 total, 8 up, 8 in 2024-04-07T16:28:16.943 INFO:tasks.rados.rados.0.smithi100.stdout:3681: done (0 left) 2024-04-07T16:28:16.943 INFO:tasks.rados.rados.0.smithi100.stdout:3682: copy_from oid 28 from oid 26 current snap is 334 2024-04-07T16:28:16.943 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 20 current snap is 334 2024-04-07T16:28:16.943 INFO:tasks.rados.rados.0.smithi100.stdout:3683: seq_num 1142 ranges {5201920=729088,5931008=565248,6496256=696320,7192576=344064} 2024-04-07T16:28:16.975 INFO:tasks.rados.rados.0.smithi100.stdout:3683: writing smithi10066867-20 from 5201920 to 5931008 tid 1 2024-04-07T16:28:16.978 INFO:tasks.rados.rados.0.smithi100.stdout:3683: writing smithi10066867-20 from 5931008 to 6496256 tid 2 2024-04-07T16:28:16.982 INFO:tasks.rados.rados.0.smithi100.stdout:3683: writing smithi10066867-20 from 6496256 to 7192576 tid 3 2024-04-07T16:28:16.985 INFO:tasks.rados.rados.0.smithi100.stdout:3683: writing smithi10066867-20 from 7192576 to 7536640 tid 4 2024-04-07T16:28:16.985 INFO:tasks.rados.rados.0.smithi100.stdout:3684: snap_create 2024-04-07T16:28:16.985 INFO:tasks.rados.rados.0.smithi100.stdout:3682: finishing copy_from to smithi10066867-28 2024-04-07T16:28:16.985 INFO:tasks.rados.rados.0.smithi100.stdout:3682: got expected ENOENT (src dne) 2024-04-07T16:28:16.985 INFO:tasks.rados.rados.0.smithi100.stdout:3682: finishing copy_from racing read to smithi10066867-28 2024-04-07T16:28:16.986 INFO:tasks.rados.rados.0.smithi100.stdout:3683: finishing write tid 1 to smithi10066867-20 2024-04-07T16:28:16.991 INFO:tasks.rados.rados.0.smithi100.stdout:3683: finishing write tid 2 to smithi10066867-20 2024-04-07T16:28:16.992 INFO:tasks.rados.rados.0.smithi100.stdout:3683: finishing write tid 3 to smithi10066867-20 2024-04-07T16:28:16.993 INFO:tasks.rados.rados.0.smithi100.stdout:3683: finishing write tid 4 to smithi10066867-20 2024-04-07T16:28:16.997 INFO:tasks.rados.rados.0.smithi100.stdout:3683: finishing write tid 5 to smithi10066867-20 2024-04-07T16:28:17.001 INFO:tasks.rados.rados.0.smithi100.stdout:3683: finishing write tid 6 to smithi10066867-20 2024-04-07T16:28:17.001 INFO:tasks.rados.rados.0.smithi100.stdout:3683: oid 20 updating version 0 to 1724 2024-04-07T16:28:17.001 INFO:tasks.rados.rados.0.smithi100.stdout:3683: oid 20 updating version 1724 to 1725 2024-04-07T16:28:17.001 INFO:tasks.rados.rados.0.smithi100.stdout:3683: oid 20 updating version 1725 to 1726 2024-04-07T16:28:17.001 INFO:tasks.rados.rados.0.smithi100.stdout:3683: oid 20 version 1726 is already newer than 1723 2024-04-07T16:28:17.001 INFO:tasks.rados.rados.0.smithi100.stdout:3683: oid 20 version 1726 is already newer than 1722 2024-04-07T16:28:17.001 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 20 v 1726 (ObjNum 1142 snap 334 seq_num 1142) dirty exists 2024-04-07T16:28:17.001 INFO:tasks.rados.rados.0.smithi100.stdout:3683: left oid 20 (ObjNum 1142 snap 334 seq_num 1142) 2024-04-07T16:28:17.146 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:17 smithi144 bash[66654]: audit 2024-04-07T16:28:16.030227+0000 mon.a (mon.0) 1244 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:28:17.147 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:17 smithi144 bash[66654]: audit 2024-04-07T16:28:16.041720+0000 mon.a (mon.0) 1245 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:28:17.147 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:17 smithi144 bash[66654]: audit 2024-04-07T16:28:16.077821+0000 mon.a (mon.0) 1246 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:28:17.147 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:17 smithi144 bash[66654]: audit 2024-04-07T16:28:16.735312+0000 mon.a (mon.0) 1247 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:28:17.147 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:17 smithi144 bash[66654]: cluster 2024-04-07T16:28:16.954491+0000 mon.a (mon.0) 1248 : cluster [DBG] osdmap e739: 8 total, 8 up, 8 in 2024-04-07T16:28:17.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:17 smithi100 bash[74942]: audit 2024-04-07T16:28:16.030227+0000 mon.a (mon.0) 1244 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:28:17.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:17 smithi100 bash[74942]: audit 2024-04-07T16:28:16.041720+0000 mon.a (mon.0) 1245 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:28:17.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:17 smithi100 bash[74942]: audit 2024-04-07T16:28:16.077821+0000 mon.a (mon.0) 1246 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:28:17.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:17 smithi100 bash[74942]: audit 2024-04-07T16:28:16.735312+0000 mon.a (mon.0) 1247 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:28:17.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:17 smithi100 bash[74942]: cluster 2024-04-07T16:28:16.954491+0000 mon.a (mon.0) 1248 : cluster [DBG] osdmap e739: 8 total, 8 up, 8 in 2024-04-07T16:28:17.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:17 smithi100 bash[76202]: audit 2024-04-07T16:28:16.030227+0000 mon.a (mon.0) 1244 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:28:17.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:17 smithi100 bash[76202]: audit 2024-04-07T16:28:16.041720+0000 mon.a (mon.0) 1245 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:28:17.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:17 smithi100 bash[76202]: audit 2024-04-07T16:28:16.077821+0000 mon.a (mon.0) 1246 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:28:17.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:17 smithi100 bash[76202]: audit 2024-04-07T16:28:16.735312+0000 mon.a (mon.0) 1247 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:28:17.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:17 smithi100 bash[76202]: cluster 2024-04-07T16:28:16.954491+0000 mon.a (mon.0) 1248 : cluster [DBG] osdmap e739: 8 total, 8 up, 8 in 2024-04-07T16:28:17.606 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:28:17 smithi144 bash[98866]: logger=settings t=2024-04-07T16:28:17.156796581Z level=info msg="Starting Grafana" version=9.4.12 commit=37ee25ba46 branch=HEAD compiled=2023-05-22T15:58:05Z 2024-04-07T16:28:17.607 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:28:17 smithi144 bash[98866]: logger=settings t=2024-04-07T16:28:17.157035046Z level=warn msg="\"sentry\" frontend logging provider is deprecated and will be removed in the next major version. Use \"grafana\" provider instead." 2024-04-07T16:28:17.607 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:28:17 smithi144 bash[98866]: logger=settings t=2024-04-07T16:28:17.157055063Z level=info msg="Config loaded from" file=/usr/share/grafana/conf/defaults.ini 2024-04-07T16:28:17.607 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:28:17 smithi144 bash[98866]: logger=settings t=2024-04-07T16:28:17.157062302Z level=info msg="Config loaded from" file=/etc/grafana/grafana.ini 2024-04-07T16:28:17.607 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:28:17 smithi144 bash[98866]: logger=settings t=2024-04-07T16:28:17.157068225Z level=info msg="Config overridden from command line" arg="default.paths.data=/var/lib/grafana" 2024-04-07T16:28:17.607 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:28:17 smithi144 bash[98866]: logger=settings t=2024-04-07T16:28:17.15707406Z level=info msg="Config overridden from command line" arg="default.paths.logs=/var/log/grafana" 2024-04-07T16:28:17.607 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:28:17 smithi144 bash[98866]: logger=settings t=2024-04-07T16:28:17.157079737Z level=info msg="Config overridden from command line" arg="default.paths.plugins=/var/lib/grafana/plugins" 2024-04-07T16:28:17.607 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:28:17 smithi144 bash[98866]: logger=settings t=2024-04-07T16:28:17.157085808Z level=info msg="Config overridden from command line" arg="default.paths.provisioning=/etc/grafana/provisioning" 2024-04-07T16:28:17.607 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:28:17 smithi144 bash[98866]: logger=settings t=2024-04-07T16:28:17.157091219Z level=info msg="Config overridden from command line" arg="default.log.mode=console" 2024-04-07T16:28:17.607 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:28:17 smithi144 bash[98866]: logger=settings t=2024-04-07T16:28:17.15709772Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_DATA=/var/lib/grafana" 2024-04-07T16:28:17.607 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:28:17 smithi144 bash[98866]: logger=settings t=2024-04-07T16:28:17.157106866Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_LOGS=/var/log/grafana" 2024-04-07T16:28:17.607 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:28:17 smithi144 bash[98866]: logger=settings t=2024-04-07T16:28:17.157113159Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_PLUGINS=/var/lib/grafana/plugins" 2024-04-07T16:28:17.607 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:28:17 smithi144 bash[98866]: logger=settings t=2024-04-07T16:28:17.157118681Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_PROVISIONING=/etc/grafana/provisioning" 2024-04-07T16:28:17.608 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:28:17 smithi144 bash[98866]: logger=settings t=2024-04-07T16:28:17.15712378Z level=info msg="Path Home" path=/usr/share/grafana 2024-04-07T16:28:17.608 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:28:17 smithi144 bash[98866]: logger=settings t=2024-04-07T16:28:17.157129548Z level=info msg="Path Data" path=/var/lib/grafana 2024-04-07T16:28:17.608 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:28:17 smithi144 bash[98866]: logger=settings t=2024-04-07T16:28:17.157139471Z level=info msg="Path Logs" path=/var/log/grafana 2024-04-07T16:28:17.608 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:28:17 smithi144 bash[98866]: logger=settings t=2024-04-07T16:28:17.157145559Z level=info msg="Path Plugins" path=/var/lib/grafana/plugins 2024-04-07T16:28:17.608 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:28:17 smithi144 bash[98866]: logger=settings t=2024-04-07T16:28:17.157155017Z level=info msg="Path Provisioning" path=/etc/grafana/provisioning 2024-04-07T16:28:17.608 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:28:17 smithi144 bash[98866]: logger=settings t=2024-04-07T16:28:17.157160422Z level=info msg="App mode production" 2024-04-07T16:28:17.608 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:28:17 smithi144 bash[98866]: logger=sqlstore t=2024-04-07T16:28:17.157221237Z level=info msg="Connecting to DB" dbtype=sqlite3 2024-04-07T16:28:17.608 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:28:17 smithi144 bash[98866]: logger=sqlstore t=2024-04-07T16:28:17.157250073Z level=warn msg="SQLite database file has broader permissions than it should" path=/var/lib/grafana/grafana.db mode=-rw-r--r-- expected=-rw-r----- 2024-04-07T16:28:17.608 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:28:17 smithi144 bash[98866]: logger=migrator t=2024-04-07T16:28:17.174205367Z level=info msg="Starting DB migrations" 2024-04-07T16:28:17.608 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:28:17 smithi144 bash[98866]: logger=migrator t=2024-04-07T16:28:17.183821809Z level=info msg="migrations completed" performed=0 skipped=481 duration=1.215894ms 2024-04-07T16:28:17.608 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:28:17 smithi144 bash[98866]: logger=sqlstore t=2024-04-07T16:28:17.184864496Z level=info msg="Created default organization" 2024-04-07T16:28:17.608 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:28:17 smithi144 bash[98866]: logger=plugin.finder t=2024-04-07T16:28:17.257974718Z level=warn msg="Skipping finding plugins as directory does not exist" path=/usr/share/grafana/plugins-bundled 2024-04-07T16:28:17.608 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:28:17 smithi144 bash[98866]: logger=secrets t=2024-04-07T16:28:17.258141678Z level=info msg="Envelope encryption state" enabled=true currentprovider=secretKey.v1 2024-04-07T16:28:17.608 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:28:17 smithi144 bash[98866]: logger=query_data t=2024-04-07T16:28:17.26224253Z level=info msg="Query Service initialization" 2024-04-07T16:28:17.609 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:28:17 smithi144 bash[98866]: logger=live.push_http t=2024-04-07T16:28:17.26749689Z level=info msg="Live Push Gateway initialization" 2024-04-07T16:28:17.941 INFO:tasks.rados.rados.0.smithi100.stdout:3682: done (2 left) 2024-04-07T16:28:17.941 INFO:tasks.rados.rados.0.smithi100.stdout:3683: done (1 left) 2024-04-07T16:28:17.941 INFO:tasks.rados.rados.0.smithi100.stdout:3684: done (0 left) 2024-04-07T16:28:17.941 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 5 current snap is 335 2024-04-07T16:28:17.942 INFO:tasks.rados.rados.0.smithi100.stdout:3685: seq_num 1143 ranges {925696=753664,1679360=450560,2129920=507904,2637824=589824,3227648=57344} 2024-04-07T16:28:17.954 INFO:tasks.rados.rados.0.smithi100.stdout:3685: writing smithi10066867-5 from 925696 to 1679360 tid 1 2024-04-07T16:28:17.957 INFO:tasks.rados.rados.0.smithi100.stdout:3685: writing smithi10066867-5 from 1679360 to 2129920 tid 2 2024-04-07T16:28:17.962 INFO:tasks.rados.rados.0.smithi100.stdout:3685: writing smithi10066867-5 from 2129920 to 2637824 tid 3 2024-04-07T16:28:17.966 INFO:tasks.rados.rados.0.smithi100.stdout:3685: writing smithi10066867-5 from 2637824 to 3227648 tid 4 2024-04-07T16:28:17.967 INFO:tasks.rados.rados.0.smithi100.stdout:3685: writing smithi10066867-5 from 3227648 to 3284992 tid 5 2024-04-07T16:28:17.967 INFO:tasks.rados.rados.0.smithi100.stdout:3686: delete oid 35 current snap is 335 2024-04-07T16:28:17.971 INFO:tasks.rados.rados.0.smithi100.stdout:3685: finishing write tid 1 to smithi10066867-5 2024-04-07T16:28:17.974 INFO:tasks.rados.rados.0.smithi100.stdout:3685: finishing write tid 2 to smithi10066867-5 2024-04-07T16:28:17.976 INFO:tasks.rados.rados.0.smithi100.stdout:3685: finishing write tid 3 to smithi10066867-5 2024-04-07T16:28:17.977 INFO:tasks.rados.rados.0.smithi100.stdout:3686: done (1 left) 2024-04-07T16:28:17.977 INFO:tasks.rados.rados.0.smithi100.stdout:3687: rmattr oid 45 current snap is 335 2024-04-07T16:28:17.977 INFO:tasks.rados.rados.0.smithi100.stdout:3688: read oid 20 snap -1 2024-04-07T16:28:17.977 INFO:tasks.rados.rados.0.smithi100.stdout:3688: expect (ObjNum 1142 snap 334 seq_num 1142) 2024-04-07T16:28:17.977 INFO:tasks.rados.rados.0.smithi100.stdout:3689: read oid 44 snap 322 2024-04-07T16:28:17.977 INFO:tasks.rados.rados.0.smithi100.stdout:3689: expect deleted 2024-04-07T16:28:17.978 INFO:tasks.rados.rados.0.smithi100.stdout:3690: snap_create 2024-04-07T16:28:17.979 INFO:tasks.rados.rados.0.smithi100.stdout:3685: finishing write tid 4 to smithi10066867-5 2024-04-07T16:28:17.981 INFO:tasks.rados.rados.0.smithi100.stdout:3685: finishing write tid 5 to smithi10066867-5 2024-04-07T16:28:17.982 INFO:tasks.rados.rados.0.smithi100.stdout:3685: finishing write tid 6 to smithi10066867-5 2024-04-07T16:28:17.984 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 45 v 1814 (ObjNum 1041 snap 307 seq_num 1041) dirty exists 2024-04-07T16:28:17.987 INFO:tasks.rados.rados.0.smithi100.stdout:3685: finishing write tid 7 to smithi10066867-5 2024-04-07T16:28:17.987 INFO:tasks.rados.rados.0.smithi100.stdout:3685: oid 5 updating version 0 to 1817 2024-04-07T16:28:17.987 INFO:tasks.rados.rados.0.smithi100.stdout:3685: oid 5 updating version 1817 to 1818 2024-04-07T16:28:17.987 INFO:tasks.rados.rados.0.smithi100.stdout:3685: oid 5 version 1818 is already newer than 1815 2024-04-07T16:28:17.987 INFO:tasks.rados.rados.0.smithi100.stdout:3685: oid 5 updating version 1818 to 1819 2024-04-07T16:28:17.987 INFO:tasks.rados.rados.0.smithi100.stdout:3685: oid 5 version 1819 is already newer than 1816 2024-04-07T16:28:17.987 INFO:tasks.rados.rados.0.smithi100.stdout:3685: oid 5 version 1819 is already newer than 1814 2024-04-07T16:28:17.987 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 5 v 1819 (ObjNum 1143 snap 335 seq_num 1143) dirty exists 2024-04-07T16:28:17.987 INFO:tasks.rados.rados.0.smithi100.stdout:3685: left oid 5 (ObjNum 1143 snap 335 seq_num 1143) 2024-04-07T16:28:18.010 INFO:tasks.rados.rados.0.smithi100.stdout:3688: expect (ObjNum 1142 snap 334 seq_num 1142) 2024-04-07T16:28:18.301 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:18 smithi144 bash[66654]: cluster 2024-04-07T16:28:16.812795+0000 mgr.y (mgr.25175) 382 : cluster [DBG] pgmap v701: 121 pgs: 121 active+clean; 141 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 4.6 MiB/s rd, 1.1 MiB/s wr, 6 op/s 2024-04-07T16:28:18.301 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:18 smithi144 bash[66654]: cluster 2024-04-07T16:28:17.955362+0000 mon.a (mon.0) 1249 : cluster [DBG] osdmap e740: 8 total, 8 up, 8 in 2024-04-07T16:28:18.416 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:18 smithi100 bash[76202]: cluster 2024-04-07T16:28:16.812795+0000 mgr.y (mgr.25175) 382 : cluster [DBG] pgmap v701: 121 pgs: 121 active+clean; 141 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 4.6 MiB/s rd, 1.1 MiB/s wr, 6 op/s 2024-04-07T16:28:18.416 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:18 smithi100 bash[76202]: cluster 2024-04-07T16:28:17.955362+0000 mon.a (mon.0) 1249 : cluster [DBG] osdmap e740: 8 total, 8 up, 8 in 2024-04-07T16:28:18.416 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:18 smithi100 bash[74942]: cluster 2024-04-07T16:28:16.812795+0000 mgr.y (mgr.25175) 382 : cluster [DBG] pgmap v701: 121 pgs: 121 active+clean; 141 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 4.6 MiB/s rd, 1.1 MiB/s wr, 6 op/s 2024-04-07T16:28:18.417 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:18 smithi100 bash[74942]: cluster 2024-04-07T16:28:17.955362+0000 mon.a (mon.0) 1249 : cluster [DBG] osdmap e740: 8 total, 8 up, 8 in 2024-04-07T16:28:18.606 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:28:18 smithi144 bash[98866]: logger=infra.usagestats.collector t=2024-04-07T16:28:18.311829326Z level=info msg="registering usage stat providers" usageStatsProvidersLen=2 2024-04-07T16:28:18.606 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:28:18 smithi144 bash[98866]: logger=provisioning.datasources t=2024-04-07T16:28:18.342393998Z level=info msg="deleted correlations based on configuration" ds_name=Dashboard1 2024-04-07T16:28:18.606 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:28:18 smithi144 bash[98866]: logger=provisioning.datasources t=2024-04-07T16:28:18.342407171Z level=info msg="deleted datasource based on configuration" name=Dashboard1 2024-04-07T16:28:18.606 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:28:18 smithi144 bash[98866]: logger=provisioning.datasources t=2024-04-07T16:28:18.342536857Z level=info msg="inserting datasource from configuration " name=Dashboard1 uid=P43CA22E17D0F9596 2024-04-07T16:28:18.606 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:28:18 smithi144 bash[98866]: logger=provisioning.alerting t=2024-04-07T16:28:18.417843425Z level=info msg="starting to provision alerting" 2024-04-07T16:28:18.606 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:28:18 smithi144 bash[98866]: logger=provisioning.alerting t=2024-04-07T16:28:18.417867451Z level=info msg="finished to provision alerting" 2024-04-07T16:28:18.606 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:28:18 smithi144 bash[98866]: logger=ngalert.state.manager t=2024-04-07T16:28:18.418078094Z level=info msg="Warming state cache for startup" 2024-04-07T16:28:18.606 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:28:18 smithi144 bash[98866]: logger=grafanaStorageLogger t=2024-04-07T16:28:18.418617337Z level=info msg="storage starting" 2024-04-07T16:28:18.606 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:28:18 smithi144 bash[98866]: logger=http.server t=2024-04-07T16:28:18.42230223Z level=info msg="HTTP Server Listen" address=[::]:3000 protocol=https subUrl= socket= 2024-04-07T16:28:18.688 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:28:18 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:28:18] "GET /metrics HTTP/1.1" 200 36341 "" "Prometheus/2.43.0" 2024-04-07T16:28:18.949 INFO:tasks.rados.rados.0.smithi100.stdout:3685: done (4 left) 2024-04-07T16:28:18.949 INFO:tasks.rados.rados.0.smithi100.stdout:3687: done (3 left) 2024-04-07T16:28:18.949 INFO:tasks.rados.rados.0.smithi100.stdout:3688: done (2 left) 2024-04-07T16:28:18.949 INFO:tasks.rados.rados.0.smithi100.stdout:3689: done (1 left) 2024-04-07T16:28:18.949 INFO:tasks.rados.rados.0.smithi100.stdout:3690: done (0 left) 2024-04-07T16:28:18.949 INFO:tasks.rados.rados.0.smithi100.stdout:3691: snap_create 2024-04-07T16:28:19.356 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:28:18 smithi144 bash[98866]: logger=ngalert.state.manager t=2024-04-07T16:28:18.9667123Z level=info msg="State cache has been initialized" states=0 duration=548.627003ms 2024-04-07T16:28:19.356 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:28:18 smithi144 bash[98866]: logger=ngalert.multiorg.alertmanager t=2024-04-07T16:28:18.966806864Z level=info msg="starting MultiOrg Alertmanager" 2024-04-07T16:28:19.356 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:28:18 smithi144 bash[98866]: logger=ticker t=2024-04-07T16:28:18.96685813Z level=info msg=starting first_tick=2024-04-07T16:28:20Z 2024-04-07T16:28:19.953 INFO:tasks.rados.rados.0.smithi100.stdout:3691: done (0 left) 2024-04-07T16:28:19.953 INFO:tasks.rados.rados.0.smithi100.stdout:3692: rollback oid 9 current snap is 337 2024-04-07T16:28:19.953 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 9 to 335 2024-04-07T16:28:19.953 INFO:tasks.rados.rados.0.smithi100.stdout:3693: copy_from oid 11 from oid 31 current snap is 337 2024-04-07T16:28:19.953 INFO:tasks.rados.rados.0.smithi100.stdout:3694: read oid 13 snap -1 2024-04-07T16:28:19.953 INFO:tasks.rados.rados.0.smithi100.stdout:3694: expect deleted 2024-04-07T16:28:19.953 INFO:tasks.rados.rados.0.smithi100.stdout:3695: snap_create 2024-04-07T16:28:19.962 INFO:tasks.rados.rados.0.smithi100.stdout:3692: finishing rollback tid 1 to smithi10066867-9 2024-04-07T16:28:19.963 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 9 v 1217 (ObjNum 97 snap 0 seq_num 1571426163) dirty dne 2024-04-07T16:28:19.986 INFO:tasks.rados.rados.0.smithi100.stdout:3693: finishing copy_from racing read to smithi10066867-11 2024-04-07T16:28:19.995 INFO:tasks.rados.rados.0.smithi100.stdout:3693: finishing copy_from to smithi10066867-11 2024-04-07T16:28:19.996 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 11 v 1730 (ObjNum 1128 snap 326 seq_num 1128) dirty exists 2024-04-07T16:28:20.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:19 smithi144 bash[66654]: cluster 2024-04-07T16:28:18.813610+0000 mgr.y (mgr.25175) 383 : cluster [DBG] pgmap v704: 121 pgs: 121 active+clean; 143 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 511 B/s rd, 732 KiB/s wr, 2 op/s 2024-04-07T16:28:20.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:19 smithi144 bash[66654]: cluster 2024-04-07T16:28:18.959543+0000 mon.a (mon.0) 1250 : cluster [DBG] osdmap e741: 8 total, 8 up, 8 in 2024-04-07T16:28:20.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:19 smithi100 bash[74942]: cluster 2024-04-07T16:28:18.813610+0000 mgr.y (mgr.25175) 383 : cluster [DBG] pgmap v704: 121 pgs: 121 active+clean; 143 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 511 B/s rd, 732 KiB/s wr, 2 op/s 2024-04-07T16:28:20.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:19 smithi100 bash[74942]: cluster 2024-04-07T16:28:18.959543+0000 mon.a (mon.0) 1250 : cluster [DBG] osdmap e741: 8 total, 8 up, 8 in 2024-04-07T16:28:20.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:19 smithi100 bash[76202]: cluster 2024-04-07T16:28:18.813610+0000 mgr.y (mgr.25175) 383 : cluster [DBG] pgmap v704: 121 pgs: 121 active+clean; 143 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 511 B/s rd, 732 KiB/s wr, 2 op/s 2024-04-07T16:28:20.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:19 smithi100 bash[76202]: cluster 2024-04-07T16:28:18.959543+0000 mon.a (mon.0) 1250 : cluster [DBG] osdmap e741: 8 total, 8 up, 8 in 2024-04-07T16:28:20.951 INFO:tasks.rados.rados.0.smithi100.stdout:3692: done (3 left) 2024-04-07T16:28:20.952 INFO:tasks.rados.rados.0.smithi100.stdout:3693: done (2 left) 2024-04-07T16:28:20.952 INFO:tasks.rados.rados.0.smithi100.stdout:3694: done (1 left) 2024-04-07T16:28:20.952 INFO:tasks.rados.rados.0.smithi100.stdout:3695: done (0 left) 2024-04-07T16:28:20.952 INFO:tasks.rados.rados.0.smithi100.stdout:3696: rollback oid 20 current snap is 338 2024-04-07T16:28:20.952 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 20 to 327 2024-04-07T16:28:20.952 INFO:tasks.rados.rados.0.smithi100.stdout:3697: rmattr oid 23 current snap is 338 2024-04-07T16:28:20.952 INFO:tasks.rados.rados.0.smithi100.stdout:3697: done (1 left) 2024-04-07T16:28:20.952 INFO:tasks.rados.rados.0.smithi100.stdout:3698: rollback oid 31 current snap is 338 2024-04-07T16:28:20.952 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 31 to 335 2024-04-07T16:28:20.952 INFO:tasks.rados.rados.0.smithi100.stdout:3699: delete oid 39 current snap is 338 2024-04-07T16:28:20.979 INFO:tasks.rados.rados.0.smithi100.stdout:3699: done (2 left) 2024-04-07T16:28:20.979 INFO:tasks.rados.rados.0.smithi100.stdout:3700: delete oid 11 current snap is 338 2024-04-07T16:28:20.987 INFO:tasks.rados.rados.0.smithi100.stdout:3698: finishing rollback tid 0 to smithi10066867-31 2024-04-07T16:28:20.988 INFO:tasks.rados.rados.0.smithi100.stdout:3696: finishing rollback tid 0 to smithi10066867-20 2024-04-07T16:28:20.988 INFO:tasks.rados.rados.0.smithi100.stdout:3698: finishing rollback tid 1 to smithi10066867-31 2024-04-07T16:28:20.988 INFO:tasks.rados.rados.0.smithi100.stdout:3696: finishing rollback tid 1 to smithi10066867-20 2024-04-07T16:28:20.988 INFO:tasks.rados.rados.0.smithi100.stdout:3698: finishing rollback tid 2 to smithi10066867-31 2024-04-07T16:28:20.988 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 31 v 1751 (ObjNum 1128 snap 326 seq_num 1128) dirty exists 2024-04-07T16:28:20.988 INFO:tasks.rados.rados.0.smithi100.stdout:3696: finishing rollback tid 2 to smithi10066867-20 2024-04-07T16:28:20.988 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 20 v 1729 (ObjNum 1116 snap 322 seq_num 1116) dirty exists 2024-04-07T16:28:20.989 INFO:tasks.rados.rados.0.smithi100.stdout:3696: done (2 left) 2024-04-07T16:28:20.989 INFO:tasks.rados.rados.0.smithi100.stdout:3698: done (1 left) 2024-04-07T16:28:20.989 INFO:tasks.rados.rados.0.smithi100.stdout:3700: done (0 left) 2024-04-07T16:28:20.989 INFO:tasks.rados.rados.0.smithi100.stdout:3701: snap_remove snap 336 2024-04-07T16:28:21.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:20 smithi144 bash[66654]: cluster 2024-04-07T16:28:19.965686+0000 mon.a (mon.0) 1251 : cluster [DBG] osdmap e742: 8 total, 8 up, 8 in 2024-04-07T16:28:21.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:20 smithi144 bash[66654]: cluster 2024-04-07T16:28:20.964658+0000 mon.a (mon.0) 1252 : cluster [DBG] osdmap e743: 8 total, 8 up, 8 in 2024-04-07T16:28:21.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:20 smithi100 bash[74942]: cluster 2024-04-07T16:28:19.965686+0000 mon.a (mon.0) 1251 : cluster [DBG] osdmap e742: 8 total, 8 up, 8 in 2024-04-07T16:28:21.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:20 smithi100 bash[74942]: cluster 2024-04-07T16:28:20.964658+0000 mon.a (mon.0) 1252 : cluster [DBG] osdmap e743: 8 total, 8 up, 8 in 2024-04-07T16:28:21.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:20 smithi100 bash[76202]: cluster 2024-04-07T16:28:19.965686+0000 mon.a (mon.0) 1251 : cluster [DBG] osdmap e742: 8 total, 8 up, 8 in 2024-04-07T16:28:21.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:20 smithi100 bash[76202]: cluster 2024-04-07T16:28:20.964658+0000 mon.a (mon.0) 1252 : cluster [DBG] osdmap e743: 8 total, 8 up, 8 in 2024-04-07T16:28:21.944 INFO:tasks.rados.rados.0.smithi100.stdout:3701: done (0 left) 2024-04-07T16:28:21.944 INFO:tasks.rados.rados.0.smithi100.stdout:3702: delete oid 17 current snap is 338 2024-04-07T16:28:21.958 INFO:tasks.rados.rados.0.smithi100.stdout:3702: done (0 left) 2024-04-07T16:28:21.958 INFO:tasks.rados.rados.0.smithi100.stdout:3703: snap_create 2024-04-07T16:28:22.305 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:21 smithi144 bash[66654]: cluster 2024-04-07T16:28:20.814310+0000 mgr.y (mgr.25175) 384 : cluster [DBG] pgmap v707: 121 pgs: 121 active+clean; 146 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 5.4 MiB/s rd, 1.3 MiB/s wr, 5 op/s 2024-04-07T16:28:22.305 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:21 smithi144 bash[66654]: cluster 2024-04-07T16:28:21.957708+0000 mon.a (mon.0) 1253 : cluster [DBG] osdmap e744: 8 total, 8 up, 8 in 2024-04-07T16:28:22.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:21 smithi100 bash[74942]: cluster 2024-04-07T16:28:20.814310+0000 mgr.y (mgr.25175) 384 : cluster [DBG] pgmap v707: 121 pgs: 121 active+clean; 146 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 5.4 MiB/s rd, 1.3 MiB/s wr, 5 op/s 2024-04-07T16:28:22.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:21 smithi100 bash[74942]: cluster 2024-04-07T16:28:21.957708+0000 mon.a (mon.0) 1253 : cluster [DBG] osdmap e744: 8 total, 8 up, 8 in 2024-04-07T16:28:22.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:21 smithi100 bash[76202]: cluster 2024-04-07T16:28:20.814310+0000 mgr.y (mgr.25175) 384 : cluster [DBG] pgmap v707: 121 pgs: 121 active+clean; 146 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 5.4 MiB/s rd, 1.3 MiB/s wr, 5 op/s 2024-04-07T16:28:22.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:21 smithi100 bash[76202]: cluster 2024-04-07T16:28:21.957708+0000 mon.a (mon.0) 1253 : cluster [DBG] osdmap e744: 8 total, 8 up, 8 in 2024-04-07T16:28:22.605 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:28:22 smithi144 bash[95233]: ts=2024-04-07T16:28:22.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:28:22.945 INFO:tasks.rados.rados.0.smithi100.stdout:3703: done (0 left) 2024-04-07T16:28:22.945 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 8 current snap is 339 2024-04-07T16:28:22.946 INFO:tasks.rados.rados.0.smithi100.stdout:3704: seq_num 1148 ranges {0=32768} 2024-04-07T16:28:22.946 INFO:tasks.rados.rados.0.smithi100.stdout:3704: writing smithi10066867-8 from 0 to 32768 tid 1 2024-04-07T16:28:22.946 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 29 current snap is 339 2024-04-07T16:28:22.946 INFO:tasks.rados.rados.0.smithi100.stdout:3705: seq_num 1149 ranges {1343488=49152} 2024-04-07T16:28:22.970 INFO:tasks.rados.rados.0.smithi100.stdout:3705: writing smithi10066867-29 from 1343488 to 1392640 tid 1 2024-04-07T16:28:22.970 INFO:tasks.rados.rados.0.smithi100.stdout:3706: copy_from oid 12 from oid 17 current snap is 339 2024-04-07T16:28:22.970 INFO:tasks.rados.rados.0.smithi100.stdout:3704: finishing write tid 1 to smithi10066867-8 2024-04-07T16:28:22.970 INFO:tasks.rados.rados.0.smithi100.stdout:3704: finishing write tid 2 to smithi10066867-8 2024-04-07T16:28:22.970 INFO:tasks.rados.rados.0.smithi100.stdout:3704: finishing write tid 3 to smithi10066867-8 2024-04-07T16:28:22.970 INFO:tasks.rados.rados.0.smithi100.stdout:3704: oid 8 updating version 0 to 1714 2024-04-07T16:28:22.970 INFO:tasks.rados.rados.0.smithi100.stdout:3704: oid 8 version 1714 is already newer than 1713 2024-04-07T16:28:22.970 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 8 v 1714 (ObjNum 1148 snap 339 seq_num 1148) dirty exists 2024-04-07T16:28:22.970 INFO:tasks.rados.rados.0.smithi100.stdout:3704: left oid 8 (ObjNum 1148 snap 339 seq_num 1148) 2024-04-07T16:28:22.970 INFO:tasks.rados.rados.0.smithi100.stdout:3704: done (2 left) 2024-04-07T16:28:22.970 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 22 current snap is 339 2024-04-07T16:28:22.970 INFO:tasks.rados.rados.0.smithi100.stdout:3707: seq_num 1150 ranges {1892352=65536} 2024-04-07T16:28:23.002 INFO:tasks.rados.rados.0.smithi100.stdout:3707: writing smithi10066867-22 from 1892352 to 1957888 tid 1 2024-04-07T16:28:23.002 INFO:tasks.rados.rados.0.smithi100.stdout:3708: snap_remove snap 334 2024-04-07T16:28:23.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:22 smithi144 bash[66654]: audit 2024-04-07T16:28:21.987524+0000 mon.a (mon.0) 1254 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:28:23.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:22 smithi144 bash[66654]: audit 2024-04-07T16:28:22.003311+0000 mon.a (mon.0) 1255 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:28:23.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:22 smithi144 bash[66654]: cluster 2024-04-07T16:28:22.958490+0000 mon.a (mon.0) 1256 : cluster [DBG] osdmap e745: 8 total, 8 up, 8 in 2024-04-07T16:28:23.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:22 smithi100 bash[74942]: audit 2024-04-07T16:28:21.987524+0000 mon.a (mon.0) 1254 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:28:23.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:22 smithi100 bash[74942]: audit 2024-04-07T16:28:22.003311+0000 mon.a (mon.0) 1255 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:28:23.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:22 smithi100 bash[74942]: cluster 2024-04-07T16:28:22.958490+0000 mon.a (mon.0) 1256 : cluster [DBG] osdmap e745: 8 total, 8 up, 8 in 2024-04-07T16:28:23.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:22 smithi100 bash[76202]: audit 2024-04-07T16:28:21.987524+0000 mon.a (mon.0) 1254 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:28:23.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:22 smithi100 bash[76202]: audit 2024-04-07T16:28:22.003311+0000 mon.a (mon.0) 1255 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:28:23.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:22 smithi100 bash[76202]: cluster 2024-04-07T16:28:22.958490+0000 mon.a (mon.0) 1256 : cluster [DBG] osdmap e745: 8 total, 8 up, 8 in 2024-04-07T16:28:24.002 INFO:tasks.rados.rados.0.smithi100.stdout:3708: done (3 left) 2024-04-07T16:28:24.003 INFO:tasks.rados.rados.0.smithi100.stdout:3709: rollback oid 40 current snap is 339 2024-04-07T16:28:24.003 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 40 to 335 2024-04-07T16:28:24.003 INFO:tasks.rados.rados.0.smithi100.stdout:3705: finishing write tid 1 to smithi10066867-29 2024-04-07T16:28:24.003 INFO:tasks.rados.rados.0.smithi100.stdout:3706: finishing copy_from to smithi10066867-12 2024-04-07T16:28:24.003 INFO:tasks.rados.rados.0.smithi100.stdout:3706: got expected ENOENT (src dne) 2024-04-07T16:28:24.003 INFO:tasks.rados.rados.0.smithi100.stdout:3706: finishing copy_from racing read to smithi10066867-12 2024-04-07T16:28:24.003 INFO:tasks.rados.rados.0.smithi100.stdout:3705: finishing write tid 2 to smithi10066867-29 2024-04-07T16:28:24.003 INFO:tasks.rados.rados.0.smithi100.stdout:3705: finishing write tid 3 to smithi10066867-29 2024-04-07T16:28:24.003 INFO:tasks.rados.rados.0.smithi100.stdout:3705: oid 29 updating version 0 to 1587 2024-04-07T16:28:24.003 INFO:tasks.rados.rados.0.smithi100.stdout:3705: oid 29 updating version 1587 to 1588 2024-04-07T16:28:24.003 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 29 v 1588 (ObjNum 1149 snap 339 seq_num 1149) dirty exists 2024-04-07T16:28:24.003 INFO:tasks.rados.rados.0.smithi100.stdout:3705: left oid 29 (ObjNum 1149 snap 339 seq_num 1149) 2024-04-07T16:28:24.003 INFO:tasks.rados.rados.0.smithi100.stdout:3707: finishing write tid 1 to smithi10066867-22 2024-04-07T16:28:24.004 INFO:tasks.rados.rados.0.smithi100.stdout:3707: finishing write tid 2 to smithi10066867-22 2024-04-07T16:28:24.004 INFO:tasks.rados.rados.0.smithi100.stdout:3707: finishing write tid 3 to smithi10066867-22 2024-04-07T16:28:24.004 INFO:tasks.rados.rados.0.smithi100.stdout:3707: oid 22 updating version 0 to 1730 2024-04-07T16:28:24.004 INFO:tasks.rados.rados.0.smithi100.stdout:3707: oid 22 updating version 1730 to 1731 2024-04-07T16:28:24.004 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 22 v 1731 (ObjNum 1150 snap 339 seq_num 1150) dirty exists 2024-04-07T16:28:24.004 INFO:tasks.rados.rados.0.smithi100.stdout:3707: left oid 22 (ObjNum 1150 snap 339 seq_num 1150) 2024-04-07T16:28:24.004 INFO:tasks.rados.rados.0.smithi100.stdout:3705: done (3 left) 2024-04-07T16:28:24.004 INFO:tasks.rados.rados.0.smithi100.stdout:3706: done (2 left) 2024-04-07T16:28:24.004 INFO:tasks.rados.rados.0.smithi100.stdout:3707: done (1 left) 2024-04-07T16:28:24.004 INFO:tasks.rados.rados.0.smithi100.stdout:3710: snap_remove snap 326 2024-04-07T16:28:24.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:24 smithi144 bash[66654]: cluster 2024-04-07T16:28:22.815517+0000 mgr.y (mgr.25175) 385 : cluster [DBG] pgmap v710: 121 pgs: 121 active+clean; 146 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 5.4 MiB/s rd, 570 KiB/s wr, 3 op/s 2024-04-07T16:28:24.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:24 smithi144 bash[66654]: audit 2024-04-07T16:28:23.103803+0000 mon.a (mon.0) 1257 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:28:24.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:24 smithi144 bash[66654]: audit 2024-04-07T16:28:23.114574+0000 mon.a (mon.0) 1258 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:28:24.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:24 smithi144 bash[66654]: cluster 2024-04-07T16:28:24.015193+0000 mon.a (mon.0) 1259 : cluster [DBG] osdmap e746: 8 total, 8 up, 8 in 2024-04-07T16:28:24.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:24 smithi100 bash[74942]: cluster 2024-04-07T16:28:22.815517+0000 mgr.y (mgr.25175) 385 : cluster [DBG] pgmap v710: 121 pgs: 121 active+clean; 146 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 5.4 MiB/s rd, 570 KiB/s wr, 3 op/s 2024-04-07T16:28:24.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:24 smithi100 bash[74942]: audit 2024-04-07T16:28:23.103803+0000 mon.a (mon.0) 1257 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:28:24.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:24 smithi100 bash[74942]: audit 2024-04-07T16:28:23.114574+0000 mon.a (mon.0) 1258 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:28:24.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:24 smithi100 bash[74942]: cluster 2024-04-07T16:28:24.015193+0000 mon.a (mon.0) 1259 : cluster [DBG] osdmap e746: 8 total, 8 up, 8 in 2024-04-07T16:28:24.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:24 smithi100 bash[76202]: cluster 2024-04-07T16:28:22.815517+0000 mgr.y (mgr.25175) 385 : cluster [DBG] pgmap v710: 121 pgs: 121 active+clean; 146 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 5.4 MiB/s rd, 570 KiB/s wr, 3 op/s 2024-04-07T16:28:24.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:24 smithi100 bash[76202]: audit 2024-04-07T16:28:23.103803+0000 mon.a (mon.0) 1257 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:28:24.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:24 smithi100 bash[76202]: audit 2024-04-07T16:28:23.114574+0000 mon.a (mon.0) 1258 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:28:24.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:24 smithi100 bash[76202]: cluster 2024-04-07T16:28:24.015193+0000 mon.a (mon.0) 1259 : cluster [DBG] osdmap e746: 8 total, 8 up, 8 in 2024-04-07T16:28:25.004 INFO:tasks.rados.rados.0.smithi100.stdout:3710: done (1 left) 2024-04-07T16:28:25.004 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 9 current snap is 339 2024-04-07T16:28:25.004 INFO:tasks.rados.rados.0.smithi100.stdout:3709: finishing rollback tid 0 to smithi10066867-40 2024-04-07T16:28:25.004 INFO:tasks.rados.rados.0.smithi100.stdout:3709: finishing rollback tid 1 to smithi10066867-40 2024-04-07T16:28:25.005 INFO:tasks.rados.rados.0.smithi100.stdout:3709: finishing rollback tid 2 to smithi10066867-40 2024-04-07T16:28:25.005 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 40 v 1677 (ObjNum 968 snap 293 seq_num 968) dirty exists 2024-04-07T16:28:25.005 INFO:tasks.rados.rados.0.smithi100.stdout:3711: seq_num 1151 ranges {0=81920} 2024-04-07T16:28:25.005 INFO:tasks.rados.rados.0.smithi100.stdout:3711: writing smithi10066867-9 from 0 to 81920 tid 1 2024-04-07T16:28:25.005 INFO:tasks.rados.rados.0.smithi100.stdout:3709: done (1 left) 2024-04-07T16:28:25.006 INFO:tasks.rados.rados.0.smithi100.stdout:3712: snap_create 2024-04-07T16:28:25.023 INFO:tasks.rados.rados.0.smithi100.stdout:3711: finishing write tid 1 to smithi10066867-9 2024-04-07T16:28:25.026 INFO:tasks.rados.rados.0.smithi100.stdout:3711: finishing write tid 2 to smithi10066867-9 2024-04-07T16:28:25.032 INFO:tasks.rados.rados.0.smithi100.stdout:3711: finishing write tid 3 to smithi10066867-9 2024-04-07T16:28:25.032 INFO:tasks.rados.rados.0.smithi100.stdout:3711: oid 9 updating version 0 to 1219 2024-04-07T16:28:25.032 INFO:tasks.rados.rados.0.smithi100.stdout:3711: oid 9 version 1219 is already newer than 1218 2024-04-07T16:28:25.032 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 9 v 1219 (ObjNum 1151 snap 339 seq_num 1151) dirty exists 2024-04-07T16:28:25.032 INFO:tasks.rados.rados.0.smithi100.stdout:3711: left oid 9 (ObjNum 1151 snap 339 seq_num 1151) 2024-04-07T16:28:26.009 INFO:tasks.rados.rados.0.smithi100.stdout:3711: done (1 left) 2024-04-07T16:28:26.010 INFO:tasks.rados.rados.0.smithi100.stdout:3712: done (0 left) 2024-04-07T16:28:26.010 INFO:tasks.rados.rados.0.smithi100.stdout:3713: read oid 31 snap -1 2024-04-07T16:28:26.010 INFO:tasks.rados.rados.0.smithi100.stdout:3713: expect (ObjNum 1128 snap 326 seq_num 1128) 2024-04-07T16:28:26.010 INFO:tasks.rados.rados.0.smithi100.stdout:3714: delete oid 23 current snap is 340 2024-04-07T16:28:26.018 INFO:tasks.rados.rados.0.smithi100.stdout:3714: done (1 left) 2024-04-07T16:28:26.018 INFO:tasks.rados.rados.0.smithi100.stdout:3715: read oid 20 snap 338 2024-04-07T16:28:26.018 INFO:tasks.rados.rados.0.smithi100.stdout:3715: expect (ObjNum 1116 snap 322 seq_num 1116) 2024-04-07T16:28:26.023 INFO:tasks.rados.rados.0.smithi100.stdout:3716: rmattr oid 3 current snap is 340 2024-04-07T16:28:26.023 INFO:tasks.rados.rados.0.smithi100.stdout:3716: done (2 left) 2024-04-07T16:28:26.023 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 9 current snap is 340 2024-04-07T16:28:26.023 INFO:tasks.rados.rados.0.smithi100.stdout:3717: seq_num 1153 ranges {81920=114688} 2024-04-07T16:28:26.023 INFO:tasks.rados.rados.0.smithi100.stdout:3717: writing smithi10066867-9 from 81920 to 196608 tid 1 2024-04-07T16:28:26.025 INFO:tasks.rados.rados.0.smithi100.stdout:3718: setattr oid 3 current snap is 340 2024-04-07T16:28:26.027 INFO:tasks.rados.rados.0.smithi100.stdout:3719: setattr oid 21 current snap is 340 2024-04-07T16:28:26.032 INFO:tasks.rados.rados.0.smithi100.stdout:3720: copy_from oid 8 from oid 18 current snap is 340 2024-04-07T16:28:26.032 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 44 current snap is 340 2024-04-07T16:28:26.032 INFO:tasks.rados.rados.0.smithi100.stdout:3721: seq_num 1154 ranges {2138112=131072} 2024-04-07T16:28:26.073 INFO:tasks.rados.rados.0.smithi100.stdout:3721: writing smithi10066867-44 from 2138112 to 2269184 tid 1 2024-04-07T16:28:26.073 INFO:tasks.rados.rados.0.smithi100.stdout:3722: snap_create 2024-04-07T16:28:26.073 INFO:tasks.rados.rados.0.smithi100.stdout:3717: finishing write tid 1 to smithi10066867-9 2024-04-07T16:28:26.073 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 3 v 1713 (ObjNum 161 snap 0 seq_num 1) dirty exists 2024-04-07T16:28:26.073 INFO:tasks.rados.rados.0.smithi100.stdout:3717: finishing write tid 2 to smithi10066867-9 2024-04-07T16:28:26.073 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 21 v 1814 (ObjNum 1041 snap 307 seq_num 1041) dirty exists 2024-04-07T16:28:26.074 INFO:tasks.rados.rados.0.smithi100.stdout:3717: finishing write tid 3 to smithi10066867-9 2024-04-07T16:28:26.074 INFO:tasks.rados.rados.0.smithi100.stdout:3717: oid 9 updating version 0 to 1221 2024-04-07T16:28:26.074 INFO:tasks.rados.rados.0.smithi100.stdout:3717: oid 9 version 1221 is already newer than 1220 2024-04-07T16:28:26.074 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 9 v 1221 (ObjNum 1153 snap 340 seq_num 1153) dirty exists 2024-04-07T16:28:26.074 INFO:tasks.rados.rados.0.smithi100.stdout:3717: left oid 9 (ObjNum 1153 snap 340 seq_num 1153) 2024-04-07T16:28:26.074 INFO:tasks.rados.rados.0.smithi100.stdout:3713: expect (ObjNum 1128 snap 326 seq_num 1128) 2024-04-07T16:28:26.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:26 smithi144 bash[66654]: cluster 2024-04-07T16:28:24.816353+0000 mgr.y (mgr.25175) 386 : cluster [DBG] pgmap v713: 121 pgs: 121 active+clean; 152 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 1.7 MiB/s wr, 5 op/s 2024-04-07T16:28:26.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:26 smithi144 bash[66654]: cluster 2024-04-07T16:28:25.017138+0000 mon.a (mon.0) 1260 : cluster [DBG] osdmap e747: 8 total, 8 up, 8 in 2024-04-07T16:28:26.420 INFO:tasks.rados.rados.0.smithi100.stdout:3715: expect (ObjNum 1116 snap 322 seq_num 1116) 2024-04-07T16:28:26.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:26 smithi100 bash[74942]: cluster 2024-04-07T16:28:24.816353+0000 mgr.y (mgr.25175) 386 : cluster [DBG] pgmap v713: 121 pgs: 121 active+clean; 152 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 1.7 MiB/s wr, 5 op/s 2024-04-07T16:28:26.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:26 smithi100 bash[74942]: cluster 2024-04-07T16:28:25.017138+0000 mon.a (mon.0) 1260 : cluster [DBG] osdmap e747: 8 total, 8 up, 8 in 2024-04-07T16:28:26.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:26 smithi100 bash[76202]: cluster 2024-04-07T16:28:24.816353+0000 mgr.y (mgr.25175) 386 : cluster [DBG] pgmap v713: 121 pgs: 121 active+clean; 152 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 1.7 MiB/s wr, 5 op/s 2024-04-07T16:28:26.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:26 smithi100 bash[76202]: cluster 2024-04-07T16:28:25.017138+0000 mon.a (mon.0) 1260 : cluster [DBG] osdmap e747: 8 total, 8 up, 8 in 2024-04-07T16:28:26.696 INFO:tasks.rados.rados.0.smithi100.stdout:3721: finishing write tid 1 to smithi10066867-44 2024-04-07T16:28:26.696 INFO:tasks.rados.rados.0.smithi100.stdout:3721: finishing write tid 2 to smithi10066867-44 2024-04-07T16:28:26.696 INFO:tasks.rados.rados.0.smithi100.stdout:3721: finishing write tid 3 to smithi10066867-44 2024-04-07T16:28:26.696 INFO:tasks.rados.rados.0.smithi100.stdout:3721: oid 44 updating version 0 to 1679 2024-04-07T16:28:26.696 INFO:tasks.rados.rados.0.smithi100.stdout:3721: oid 44 version 1679 is already newer than 1678 2024-04-07T16:28:26.696 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 44 v 1679 (ObjNum 1154 snap 340 seq_num 1154) dirty exists 2024-04-07T16:28:26.696 INFO:tasks.rados.rados.0.smithi100.stdout:3721: left oid 44 (ObjNum 1154 snap 340 seq_num 1154) 2024-04-07T16:28:26.696 INFO:tasks.rados.rados.0.smithi100.stdout:3720: finishing copy_from racing read to smithi10066867-8 2024-04-07T16:28:26.696 INFO:tasks.rados.rados.0.smithi100.stdout:3720: finishing copy_from to smithi10066867-8 2024-04-07T16:28:26.696 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 8 v 1861 (ObjNum 1100 snap 320 seq_num 1100) dirty exists 2024-04-07T16:28:27.031 INFO:tasks.rados.rados.0.smithi100.stdout:3713: done (7 left) 2024-04-07T16:28:27.031 INFO:tasks.rados.rados.0.smithi100.stdout:3715: done (6 left) 2024-04-07T16:28:27.031 INFO:tasks.rados.rados.0.smithi100.stdout:3717: done (5 left) 2024-04-07T16:28:27.031 INFO:tasks.rados.rados.0.smithi100.stdout:3718: done (4 left) 2024-04-07T16:28:27.031 INFO:tasks.rados.rados.0.smithi100.stdout:3719: done (3 left) 2024-04-07T16:28:27.031 INFO:tasks.rados.rados.0.smithi100.stdout:3720: done (2 left) 2024-04-07T16:28:27.031 INFO:tasks.rados.rados.0.smithi100.stdout:3721: done (1 left) 2024-04-07T16:28:27.031 INFO:tasks.rados.rados.0.smithi100.stdout:3722: done (0 left) 2024-04-07T16:28:27.031 INFO:tasks.rados.rados.0.smithi100.stdout:3723: read oid 10 snap -1 2024-04-07T16:28:27.032 INFO:tasks.rados.rados.0.smithi100.stdout:3723: expect deleted 2024-04-07T16:28:27.032 INFO:tasks.rados.rados.0.smithi100.stdout:3724: snap_create 2024-04-07T16:28:27.355 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:27 smithi144 bash[66654]: cluster 2024-04-07T16:28:26.022614+0000 mon.a (mon.0) 1261 : cluster [DBG] osdmap e748: 8 total, 8 up, 8 in 2024-04-07T16:28:27.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:27 smithi100 bash[74942]: cluster 2024-04-07T16:28:26.022614+0000 mon.a (mon.0) 1261 : cluster [DBG] osdmap e748: 8 total, 8 up, 8 in 2024-04-07T16:28:27.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:27 smithi100 bash[76202]: cluster 2024-04-07T16:28:26.022614+0000 mon.a (mon.0) 1261 : cluster [DBG] osdmap e748: 8 total, 8 up, 8 in 2024-04-07T16:28:28.031 INFO:tasks.rados.rados.0.smithi100.stdout:3723: done (1 left) 2024-04-07T16:28:28.031 INFO:tasks.rados.rados.0.smithi100.stdout:3724: done (0 left) 2024-04-07T16:28:28.031 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 42 current snap is 342 2024-04-07T16:28:28.032 INFO:tasks.rados.rados.0.smithi100.stdout:3725: seq_num 1155 ranges {1261568=147456} 2024-04-07T16:28:28.050 INFO:tasks.rados.rados.0.smithi100.stdout:3725: writing smithi10066867-42 from 1261568 to 1409024 tid 1 2024-04-07T16:28:28.050 INFO:tasks.rados.rados.0.smithi100.stdout:3726: setattr oid 12 current snap is 342 2024-04-07T16:28:28.057 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 25 current snap is 342 2024-04-07T16:28:28.057 INFO:tasks.rados.rados.0.smithi100.stdout:3727: seq_num 1156 ranges {0=163840} 2024-04-07T16:28:28.059 INFO:tasks.rados.rados.0.smithi100.stdout:3727: writing smithi10066867-25 from 0 to 163840 tid 1 2024-04-07T16:28:28.059 INFO:tasks.rados.rados.0.smithi100.stdout:3728: read oid 23 snap -1 2024-04-07T16:28:28.059 INFO:tasks.rados.rados.0.smithi100.stdout:3728: expect deleted 2024-04-07T16:28:28.060 INFO:tasks.rados.rados.0.smithi100.stdout:3729: delete oid 17 current snap is 342 2024-04-07T16:28:28.068 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 12 v 1820 (ObjNum 1117 snap 322 seq_num 1117) dirty exists 2024-04-07T16:28:28.068 INFO:tasks.rados.rados.0.smithi100.stdout:3725: finishing write tid 1 to smithi10066867-42 2024-04-07T16:28:28.073 INFO:tasks.rados.rados.0.smithi100.stdout:3725: finishing write tid 2 to smithi10066867-42 2024-04-07T16:28:28.073 INFO:tasks.rados.rados.0.smithi100.stdout:3726: done (4 left) 2024-04-07T16:28:28.073 INFO:tasks.rados.rados.0.smithi100.stdout:3728: done (3 left) 2024-04-07T16:28:28.074 INFO:tasks.rados.rados.0.smithi100.stdout:3729: done (2 left) 2024-04-07T16:28:28.074 INFO:tasks.rados.rados.0.smithi100.stdout:3730: setattr oid 46 current snap is 342 2024-04-07T16:28:28.074 INFO:tasks.rados.rados.0.smithi100.stdout:3731: delete oid 32 current snap is 342 2024-04-07T16:28:28.074 INFO:tasks.rados.rados.0.smithi100.stdout:3727: finishing write tid 1 to smithi10066867-25 2024-04-07T16:28:28.074 INFO:tasks.rados.rados.0.smithi100.stdout:3727: finishing write tid 2 to smithi10066867-25 2024-04-07T16:28:28.074 INFO:tasks.rados.rados.0.smithi100.stdout:3725: finishing write tid 3 to smithi10066867-42 2024-04-07T16:28:28.075 INFO:tasks.rados.rados.0.smithi100.stdout:3725: oid 42 updating version 0 to 1590 2024-04-07T16:28:28.075 INFO:tasks.rados.rados.0.smithi100.stdout:3725: oid 42 version 1590 is already newer than 1589 2024-04-07T16:28:28.075 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 42 v 1590 (ObjNum 1155 snap 342 seq_num 1155) dirty exists 2024-04-07T16:28:28.075 INFO:tasks.rados.rados.0.smithi100.stdout:3725: left oid 42 (ObjNum 1155 snap 342 seq_num 1155) 2024-04-07T16:28:28.075 INFO:tasks.rados.rados.0.smithi100.stdout:3727: finishing write tid 3 to smithi10066867-25 2024-04-07T16:28:28.075 INFO:tasks.rados.rados.0.smithi100.stdout:3727: oid 25 updating version 0 to 1732 2024-04-07T16:28:28.075 INFO:tasks.rados.rados.0.smithi100.stdout:3727: oid 25 updating version 1732 to 1733 2024-04-07T16:28:28.076 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 25 v 1733 (ObjNum 1156 snap 342 seq_num 1156) dirty exists 2024-04-07T16:28:28.076 INFO:tasks.rados.rados.0.smithi100.stdout:3727: left oid 25 (ObjNum 1156 snap 342 seq_num 1156) 2024-04-07T16:28:28.082 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 46 v 1862 (ObjNum 1097 snap 320 seq_num 1097) dirty exists 2024-04-07T16:28:28.086 INFO:tasks.rados.rados.0.smithi100.stdout:3725: done (3 left) 2024-04-07T16:28:28.086 INFO:tasks.rados.rados.0.smithi100.stdout:3727: done (2 left) 2024-04-07T16:28:28.086 INFO:tasks.rados.rados.0.smithi100.stdout:3730: done (1 left) 2024-04-07T16:28:28.086 INFO:tasks.rados.rados.0.smithi100.stdout:3731: done (0 left) 2024-04-07T16:28:28.086 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 12 current snap is 342 2024-04-07T16:28:28.086 INFO:tasks.rados.rados.0.smithi100.stdout:3732: seq_num 1159 ranges {1916928=212992} 2024-04-07T16:28:28.097 INFO:tasks.rados.rados.0.smithi100.stdout:3732: writing smithi10066867-12 from 1916928 to 2129920 tid 1 2024-04-07T16:28:28.097 INFO:tasks.rados.rados.0.smithi100.stdout:3733: snap_remove snap 330 2024-04-07T16:28:28.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:28 smithi144 bash[66654]: cluster 2024-04-07T16:28:26.816977+0000 mgr.y (mgr.25175) 387 : cluster [DBG] pgmap v716: 121 pgs: 121 active+clean; 152 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 1.7 MiB/s wr, 4 op/s 2024-04-07T16:28:28.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:28 smithi144 bash[66654]: cluster 2024-04-07T16:28:27.043566+0000 mon.a (mon.0) 1262 : cluster [DBG] osdmap e749: 8 total, 8 up, 8 in 2024-04-07T16:28:28.416 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:28 smithi100 bash[74942]: cluster 2024-04-07T16:28:26.816977+0000 mgr.y (mgr.25175) 387 : cluster [DBG] pgmap v716: 121 pgs: 121 active+clean; 152 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 1.7 MiB/s wr, 4 op/s 2024-04-07T16:28:28.417 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:28 smithi100 bash[74942]: cluster 2024-04-07T16:28:27.043566+0000 mon.a (mon.0) 1262 : cluster [DBG] osdmap e749: 8 total, 8 up, 8 in 2024-04-07T16:28:28.417 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:28 smithi100 bash[76202]: cluster 2024-04-07T16:28:26.816977+0000 mgr.y (mgr.25175) 387 : cluster [DBG] pgmap v716: 121 pgs: 121 active+clean; 152 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 1.7 MiB/s wr, 4 op/s 2024-04-07T16:28:28.417 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:28 smithi100 bash[76202]: cluster 2024-04-07T16:28:27.043566+0000 mon.a (mon.0) 1262 : cluster [DBG] osdmap e749: 8 total, 8 up, 8 in 2024-04-07T16:28:28.688 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:28:28 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:28:28] "GET /metrics HTTP/1.1" 200 36341 "" "Prometheus/2.43.0" 2024-04-07T16:28:29.059 INFO:tasks.rados.rados.0.smithi100.stdout:3733: done (1 left) 2024-04-07T16:28:29.059 INFO:tasks.rados.rados.0.smithi100.stdout:3734: copy_from oid 28 from oid 33 current snap is 342 2024-04-07T16:28:29.059 INFO:tasks.rados.rados.0.smithi100.stdout:3735: rollback oid 2 current snap is 342 2024-04-07T16:28:29.059 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 2 to 331 2024-04-07T16:28:29.059 INFO:tasks.rados.rados.0.smithi100.stdout:3732: finishing write tid 1 to smithi10066867-12 2024-04-07T16:28:29.059 INFO:tasks.rados.rados.0.smithi100.stdout:3732: finishing write tid 2 to smithi10066867-12 2024-04-07T16:28:29.059 INFO:tasks.rados.rados.0.smithi100.stdout:3732: finishing write tid 3 to smithi10066867-12 2024-04-07T16:28:29.059 INFO:tasks.rados.rados.0.smithi100.stdout:3732: oid 12 updating version 0 to 1822 2024-04-07T16:28:29.059 INFO:tasks.rados.rados.0.smithi100.stdout:3732: oid 12 version 1822 is already newer than 1821 2024-04-07T16:28:29.059 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 12 v 1822 (ObjNum 1159 snap 342 seq_num 1159) dirty exists 2024-04-07T16:28:29.059 INFO:tasks.rados.rados.0.smithi100.stdout:3732: left oid 12 (ObjNum 1159 snap 342 seq_num 1159) 2024-04-07T16:28:29.059 INFO:tasks.rados.rados.0.smithi100.stdout:3732: done (2 left) 2024-04-07T16:28:29.060 INFO:tasks.rados.rados.0.smithi100.stdout:3736: snap_create 2024-04-07T16:28:29.075 INFO:tasks.rados.rados.0.smithi100.stdout:3735: finishing rollback tid 1 to smithi10066867-2 2024-04-07T16:28:29.075 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 2 v 1674 (ObjNum 273 snap 0 seq_num 0) dirty dne 2024-04-07T16:28:29.094 INFO:tasks.rados.rados.0.smithi100.stdout:3734: finishing copy_from racing read to smithi10066867-28 2024-04-07T16:28:29.099 INFO:tasks.rados.rados.0.smithi100.stdout:3734: finishing copy_from to smithi10066867-28 2024-04-07T16:28:29.099 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 28 v 1884 (ObjNum 1132 snap 326 seq_num 1132) dirty exists 2024-04-07T16:28:29.288 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:29 smithi144 bash[66654]: cluster 2024-04-07T16:28:28.043799+0000 mon.a (mon.0) 1263 : cluster [DBG] osdmap e750: 8 total, 8 up, 8 in 2024-04-07T16:28:29.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:29 smithi100 bash[74942]: cluster 2024-04-07T16:28:28.043799+0000 mon.a (mon.0) 1263 : cluster [DBG] osdmap e750: 8 total, 8 up, 8 in 2024-04-07T16:28:29.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:29 smithi100 bash[76202]: cluster 2024-04-07T16:28:28.043799+0000 mon.a (mon.0) 1263 : cluster [DBG] osdmap e750: 8 total, 8 up, 8 in 2024-04-07T16:28:30.060 INFO:tasks.rados.rados.0.smithi100.stdout:3734: done (2 left) 2024-04-07T16:28:30.060 INFO:tasks.rados.rados.0.smithi100.stdout:3735: done (1 left) 2024-04-07T16:28:30.060 INFO:tasks.rados.rados.0.smithi100.stdout:3736: done (0 left) 2024-04-07T16:28:30.060 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 4 current snap is 343 2024-04-07T16:28:30.060 INFO:tasks.rados.rados.0.smithi100.stdout:3737: seq_num 1160 ranges {638976=229376} 2024-04-07T16:28:30.071 INFO:tasks.rados.rados.0.smithi100.stdout:3737: writing smithi10066867-4 from 638976 to 868352 tid 1 2024-04-07T16:28:30.071 INFO:tasks.rados.rados.0.smithi100.stdout:3738: snap_remove snap 338 2024-04-07T16:28:30.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:30 smithi144 bash[66654]: cluster 2024-04-07T16:28:28.817760+0000 mgr.y (mgr.25175) 388 : cluster [DBG] pgmap v719: 121 pgs: 121 active+clean; 153 MiB data, 1.7 GiB used, 714 GiB / 715 GiB avail; 511 B/s rd, 48 KiB/s wr, 11 op/s 2024-04-07T16:28:30.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:30 smithi144 bash[66654]: cluster 2024-04-07T16:28:29.070569+0000 mon.a (mon.0) 1264 : cluster [DBG] osdmap e751: 8 total, 8 up, 8 in 2024-04-07T16:28:30.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:30 smithi100 bash[74942]: cluster 2024-04-07T16:28:28.817760+0000 mgr.y (mgr.25175) 388 : cluster [DBG] pgmap v719: 121 pgs: 121 active+clean; 153 MiB data, 1.7 GiB used, 714 GiB / 715 GiB avail; 511 B/s rd, 48 KiB/s wr, 11 op/s 2024-04-07T16:28:30.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:30 smithi100 bash[74942]: cluster 2024-04-07T16:28:29.070569+0000 mon.a (mon.0) 1264 : cluster [DBG] osdmap e751: 8 total, 8 up, 8 in 2024-04-07T16:28:30.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:30 smithi100 bash[76202]: cluster 2024-04-07T16:28:28.817760+0000 mgr.y (mgr.25175) 388 : cluster [DBG] pgmap v719: 121 pgs: 121 active+clean; 153 MiB data, 1.7 GiB used, 714 GiB / 715 GiB avail; 511 B/s rd, 48 KiB/s wr, 11 op/s 2024-04-07T16:28:30.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:30 smithi100 bash[76202]: cluster 2024-04-07T16:28:29.070569+0000 mon.a (mon.0) 1264 : cluster [DBG] osdmap e751: 8 total, 8 up, 8 in 2024-04-07T16:28:31.084 INFO:tasks.rados.rados.0.smithi100.stdout:3738: done (1 left) 2024-04-07T16:28:31.084 INFO:tasks.rados.rados.0.smithi100.stdout:3739: snap_remove snap 340 2024-04-07T16:28:31.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:31 smithi144 bash[66654]: cluster 2024-04-07T16:28:30.071542+0000 mon.a (mon.0) 1265 : cluster [DBG] osdmap e752: 8 total, 8 up, 8 in 2024-04-07T16:28:31.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:31 smithi100 bash[74942]: cluster 2024-04-07T16:28:30.071542+0000 mon.a (mon.0) 1265 : cluster [DBG] osdmap e752: 8 total, 8 up, 8 in 2024-04-07T16:28:31.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:31 smithi100 bash[76202]: cluster 2024-04-07T16:28:30.071542+0000 mon.a (mon.0) 1265 : cluster [DBG] osdmap e752: 8 total, 8 up, 8 in 2024-04-07T16:28:31.945 INFO:tasks.rados.rados.0.smithi100.stdout:3737: finishing write tid 1 to smithi10066867-4 2024-04-07T16:28:31.945 INFO:tasks.rados.rados.0.smithi100.stdout:3737: finishing write tid 2 to smithi10066867-4 2024-04-07T16:28:31.945 INFO:tasks.rados.rados.0.smithi100.stdout:3737: finishing write tid 3 to smithi10066867-4 2024-04-07T16:28:31.945 INFO:tasks.rados.rados.0.smithi100.stdout:3737: oid 4 updating version 0 to 1910 2024-04-07T16:28:31.945 INFO:tasks.rados.rados.0.smithi100.stdout:3737: oid 4 version 1910 is already newer than 1909 2024-04-07T16:28:31.945 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 4 v 1910 (ObjNum 1160 snap 343 seq_num 1160) dirty exists 2024-04-07T16:28:31.945 INFO:tasks.rados.rados.0.smithi100.stdout:3737: left oid 4 (ObjNum 1160 snap 343 seq_num 1160) 2024-04-07T16:28:31.945 INFO:tasks.rados.rados.0.smithi100.stdout:3737: done (1 left) 2024-04-07T16:28:31.945 INFO:tasks.rados.rados.0.smithi100.stdout:3739: done (0 left) 2024-04-07T16:28:31.945 INFO:tasks.rados.rados.0.smithi100.stdout:3740: delete oid 49 current snap is 343 2024-04-07T16:28:31.962 INFO:tasks.rados.rados.0.smithi100.stdout:3740: done (0 left) 2024-04-07T16:28:31.962 INFO:tasks.rados.rados.0.smithi100.stdout:3741: snap_remove snap 341 2024-04-07T16:28:32.356 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:28:32 smithi144 bash[95233]: ts=2024-04-07T16:28:32.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:28:32.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:32 smithi144 bash[66654]: cluster 2024-04-07T16:28:30.818544+0000 mgr.y (mgr.25175) 389 : cluster [DBG] pgmap v722: 121 pgs: 121 active+clean; 155 MiB data, 1.7 GiB used, 714 GiB / 715 GiB avail; 8.6 MiB/s rd, 768 KiB/s wr, 21 op/s 2024-04-07T16:28:32.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:32 smithi144 bash[66654]: cluster 2024-04-07T16:28:31.097023+0000 mon.a (mon.0) 1266 : cluster [DBG] osdmap e753: 8 total, 8 up, 8 in 2024-04-07T16:28:32.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:32 smithi144 bash[66654]: audit 2024-04-07T16:28:31.735726+0000 mon.a (mon.0) 1267 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:28:32.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:32 smithi144 bash[66654]: cluster 2024-04-07T16:28:31.958756+0000 mon.a (mon.0) 1268 : cluster [DBG] osdmap e754: 8 total, 8 up, 8 in 2024-04-07T16:28:32.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:32 smithi100 bash[74942]: cluster 2024-04-07T16:28:30.818544+0000 mgr.y (mgr.25175) 389 : cluster [DBG] pgmap v722: 121 pgs: 121 active+clean; 155 MiB data, 1.7 GiB used, 714 GiB / 715 GiB avail; 8.6 MiB/s rd, 768 KiB/s wr, 21 op/s 2024-04-07T16:28:32.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:32 smithi100 bash[74942]: cluster 2024-04-07T16:28:31.097023+0000 mon.a (mon.0) 1266 : cluster [DBG] osdmap e753: 8 total, 8 up, 8 in 2024-04-07T16:28:32.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:32 smithi100 bash[74942]: audit 2024-04-07T16:28:31.735726+0000 mon.a (mon.0) 1267 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:28:32.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:32 smithi100 bash[74942]: cluster 2024-04-07T16:28:31.958756+0000 mon.a (mon.0) 1268 : cluster [DBG] osdmap e754: 8 total, 8 up, 8 in 2024-04-07T16:28:32.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:32 smithi100 bash[76202]: cluster 2024-04-07T16:28:30.818544+0000 mgr.y (mgr.25175) 389 : cluster [DBG] pgmap v722: 121 pgs: 121 active+clean; 155 MiB data, 1.7 GiB used, 714 GiB / 715 GiB avail; 8.6 MiB/s rd, 768 KiB/s wr, 21 op/s 2024-04-07T16:28:32.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:32 smithi100 bash[76202]: cluster 2024-04-07T16:28:31.097023+0000 mon.a (mon.0) 1266 : cluster [DBG] osdmap e753: 8 total, 8 up, 8 in 2024-04-07T16:28:32.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:32 smithi100 bash[76202]: audit 2024-04-07T16:28:31.735726+0000 mon.a (mon.0) 1267 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:28:32.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:32 smithi100 bash[76202]: cluster 2024-04-07T16:28:31.958756+0000 mon.a (mon.0) 1268 : cluster [DBG] osdmap e754: 8 total, 8 up, 8 in 2024-04-07T16:28:32.950 INFO:tasks.rados.rados.0.smithi100.stdout:3741: done (0 left) 2024-04-07T16:28:32.950 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 50 current snap is 343 2024-04-07T16:28:32.950 INFO:tasks.rados.rados.0.smithi100.stdout:3742: seq_num 1162 ranges {0=270336} 2024-04-07T16:28:32.952 INFO:tasks.rados.rados.0.smithi100.stdout:3742: writing smithi10066867-50 from 0 to 270336 tid 1 2024-04-07T16:28:32.952 INFO:tasks.rados.rados.0.smithi100.stdout:3743: delete oid 42 current snap is 343 2024-04-07T16:28:32.959 INFO:tasks.rados.rados.0.smithi100.stdout:3742: finishing write tid 1 to smithi10066867-50 2024-04-07T16:28:32.966 INFO:tasks.rados.rados.0.smithi100.stdout:3742: finishing write tid 2 to smithi10066867-50 2024-04-07T16:28:32.968 INFO:tasks.rados.rados.0.smithi100.stdout:3743: done (1 left) 2024-04-07T16:28:32.968 INFO:tasks.rados.rados.0.smithi100.stdout:3744: snap_create 2024-04-07T16:28:32.969 INFO:tasks.rados.rados.0.smithi100.stdout:3742: finishing write tid 3 to smithi10066867-50 2024-04-07T16:28:32.969 INFO:tasks.rados.rados.0.smithi100.stdout:3742: oid 50 updating version 0 to 1816 2024-04-07T16:28:32.969 INFO:tasks.rados.rados.0.smithi100.stdout:3742: oid 50 version 1816 is already newer than 1815 2024-04-07T16:28:32.969 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 50 v 1816 (ObjNum 1162 snap 343 seq_num 1162) dirty exists 2024-04-07T16:28:32.969 INFO:tasks.rados.rados.0.smithi100.stdout:3742: left oid 50 (ObjNum 1162 snap 343 seq_num 1162) 2024-04-07T16:28:33.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:33 smithi144 bash[66654]: audit 2024-04-07T16:28:32.544851+0000 mon.a (mon.0) 1269 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:28:33.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:33 smithi144 bash[66654]: audit 2024-04-07T16:28:32.556983+0000 mon.a (mon.0) 1270 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:28:33.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:33 smithi144 bash[66654]: audit 2024-04-07T16:28:32.559786+0000 mon.a (mon.0) 1271 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:28:33.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:33 smithi144 bash[66654]: audit 2024-04-07T16:28:32.561927+0000 mon.a (mon.0) 1272 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:28:33.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:33 smithi144 bash[66654]: audit 2024-04-07T16:28:32.572922+0000 mon.a (mon.0) 1273 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:28:33.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:33 smithi144 bash[66654]: audit 2024-04-07T16:28:32.661461+0000 mon.a (mon.0) 1274 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-07T16:28:33.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:33 smithi144 bash[66654]: audit 2024-04-07T16:28:32.662506+0000 mgr.y (mgr.25175) 390 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-07T16:28:33.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:33 smithi144 bash[66654]: audit 2024-04-07T16:28:32.679978+0000 mon.a (mon.0) 1275 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:28:33.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:33 smithi144 bash[66654]: audit 2024-04-07T16:28:32.685122+0000 mon.a (mon.0) 1276 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:28:33.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:33 smithi144 bash[66654]: audit 2024-04-07T16:28:32.689093+0000 mon.a (mon.0) 1277 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:28:33.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:33 smithi144 bash[66654]: audit 2024-04-07T16:28:32.691954+0000 mon.a (mon.0) 1278 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:28:33.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:33 smithi144 bash[66654]: audit 2024-04-07T16:28:32.697635+0000 mon.a (mon.0) 1279 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:28:33.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:33 smithi144 bash[66654]: audit 2024-04-07T16:28:32.704506+0000 mon.a (mon.0) 1280 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:28:33.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:33 smithi144 bash[66654]: audit 2024-04-07T16:28:32.707370+0000 mon.a (mon.0) 1281 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:28:33.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:33 smithi144 bash[66654]: audit 2024-04-07T16:28:32.710198+0000 mon.a (mon.0) 1282 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:28:33.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:33 smithi144 bash[66654]: audit 2024-04-07T16:28:32.713049+0000 mon.a (mon.0) 1283 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:28:33.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:33 smithi144 bash[66654]: audit 2024-04-07T16:28:32.715885+0000 mon.a (mon.0) 1284 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:28:33.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:33 smithi144 bash[66654]: audit 2024-04-07T16:28:32.718649+0000 mon.a (mon.0) 1285 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:28:33.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:33 smithi144 bash[66654]: audit 2024-04-07T16:28:32.721404+0000 mon.a (mon.0) 1286 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:28:33.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:33 smithi144 bash[66654]: audit 2024-04-07T16:28:32.724525+0000 mon.a (mon.0) 1287 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:28:33.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:33 smithi144 bash[66654]: audit 2024-04-07T16:28:32.728853+0000 mon.a (mon.0) 1288 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:28:33.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:33 smithi144 bash[66654]: audit 2024-04-07T16:28:32.732702+0000 mon.a (mon.0) 1289 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:28:33.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:33 smithi144 bash[66654]: audit 2024-04-07T16:28:32.736378+0000 mon.a (mon.0) 1290 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:28:33.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:33 smithi144 bash[66654]: audit 2024-04-07T16:28:32.740177+0000 mon.a (mon.0) 1291 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:28:33.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:33 smithi144 bash[66654]: audit 2024-04-07T16:28:32.743177+0000 mon.a (mon.0) 1292 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:28:33.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:33 smithi144 bash[66654]: audit 2024-04-07T16:28:32.746277+0000 mon.a (mon.0) 1293 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:28:33.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:33 smithi144 bash[66654]: cephadm 2024-04-07T16:28:32.747895+0000 mgr.y (mgr.25175) 391 : cephadm [INF] Upgrade: Finalizing container_image settings 2024-04-07T16:28:33.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:33 smithi144 bash[66654]: audit 2024-04-07T16:28:32.755245+0000 mon.a (mon.0) 1294 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:28:33.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:33 smithi144 bash[66654]: audit 2024-04-07T16:28:32.759799+0000 mon.a (mon.0) 1295 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-04-07T16:28:33.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:33 smithi144 bash[66654]: audit 2024-04-07T16:28:32.764422+0000 mon.a (mon.0) 1296 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-04-07T16:28:33.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:33 smithi144 bash[66654]: audit 2024-04-07T16:28:32.767640+0000 mon.a (mon.0) 1297 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-07T16:28:33.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:33 smithi144 bash[66654]: audit 2024-04-07T16:28:32.774021+0000 mon.a (mon.0) 1298 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-04-07T16:28:33.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:33 smithi144 bash[66654]: audit 2024-04-07T16:28:32.777456+0000 mon.a (mon.0) 1299 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-04-07T16:28:33.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:33 smithi144 bash[66654]: audit 2024-04-07T16:28:32.785070+0000 mon.a (mon.0) 1300 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]': finished 2024-04-07T16:28:33.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:33 smithi144 bash[66654]: audit 2024-04-07T16:28:32.788174+0000 mon.a (mon.0) 1301 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-04-07T16:28:33.860 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:33 smithi144 bash[66654]: audit 2024-04-07T16:28:32.794478+0000 mon.a (mon.0) 1302 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd"}]': finished 2024-04-07T16:28:33.860 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:33 smithi144 bash[66654]: audit 2024-04-07T16:28:32.797502+0000 mon.a (mon.0) 1303 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-04-07T16:28:33.860 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:33 smithi144 bash[66654]: audit 2024-04-07T16:28:32.801782+0000 mon.a (mon.0) 1304 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds"}]': finished 2024-04-07T16:28:33.860 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:33 smithi144 bash[66654]: audit 2024-04-07T16:28:32.804981+0000 mon.a (mon.0) 1305 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-04-07T16:28:33.860 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:33 smithi144 bash[66654]: audit 2024-04-07T16:28:32.809763+0000 mon.a (mon.0) 1306 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-04-07T16:28:33.860 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:33 smithi144 bash[66654]: audit 2024-04-07T16:28:32.812850+0000 mon.a (mon.0) 1307 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-04-07T16:28:33.861 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:33 smithi144 bash[66654]: audit 2024-04-07T16:28:32.817075+0000 mon.a (mon.0) 1308 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-04-07T16:28:33.861 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:33 smithi144 bash[66654]: cluster 2024-04-07T16:28:32.819845+0000 mgr.y (mgr.25175) 392 : cluster [DBG] pgmap v725: 121 pgs: 121 active+clean; 157 MiB data, 1.7 GiB used, 714 GiB / 715 GiB avail; 8.6 MiB/s rd, 1.2 MiB/s wr, 10 op/s 2024-04-07T16:28:33.861 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:33 smithi144 bash[66654]: audit 2024-04-07T16:28:32.821009+0000 mon.a (mon.0) 1309 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-07T16:28:33.861 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:33 smithi144 bash[66654]: audit 2024-04-07T16:28:32.822538+0000 mon.a (mon.0) 1310 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-04-07T16:28:33.861 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:33 smithi144 bash[66654]: audit 2024-04-07T16:28:32.827054+0000 mon.a (mon.0) 1311 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2024-04-07T16:28:33.861 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:33 smithi144 bash[66654]: audit 2024-04-07T16:28:32.830091+0000 mon.a (mon.0) 1312 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-04-07T16:28:33.861 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:33 smithi144 bash[66654]: audit 2024-04-07T16:28:32.834506+0000 mon.a (mon.0) 1313 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-04-07T16:28:33.861 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:33 smithi144 bash[66654]: audit 2024-04-07T16:28:32.837484+0000 mon.a (mon.0) 1314 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-04-07T16:28:33.861 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:33 smithi144 bash[66654]: audit 2024-04-07T16:28:32.841876+0000 mon.a (mon.0) 1315 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-04-07T16:28:33.861 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:33 smithi144 bash[66654]: audit 2024-04-07T16:28:32.844883+0000 mon.a (mon.0) 1316 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-04-07T16:28:33.861 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:33 smithi144 bash[66654]: audit 2024-04-07T16:28:32.849103+0000 mon.a (mon.0) 1317 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-04-07T16:28:33.861 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:33 smithi144 bash[66654]: audit 2024-04-07T16:28:32.851835+0000 mon.a (mon.0) 1318 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-07T16:28:33.861 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:33 smithi144 bash[66654]: audit 2024-04-07T16:28:32.852869+0000 mon.a (mon.0) 1319 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-07T16:28:33.862 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:33 smithi144 bash[66654]: audit 2024-04-07T16:28:32.853816+0000 mon.a (mon.0) 1320 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-07T16:28:33.862 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:33 smithi144 bash[66654]: audit 2024-04-07T16:28:32.854950+0000 mon.a (mon.0) 1321 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-07T16:28:33.862 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:33 smithi144 bash[66654]: audit 2024-04-07T16:28:32.855974+0000 mon.a (mon.0) 1322 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-07T16:28:33.862 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:33 smithi144 bash[66654]: audit 2024-04-07T16:28:32.857224+0000 mon.a (mon.0) 1323 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-07T16:28:33.862 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:33 smithi144 bash[66654]: cephadm 2024-04-07T16:28:32.858037+0000 mgr.y (mgr.25175) 393 : cephadm [INF] Upgrade: Complete! 2024-04-07T16:28:33.862 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:33 smithi144 bash[66654]: audit 2024-04-07T16:28:32.858617+0000 mon.a (mon.0) 1324 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-04-07T16:28:33.862 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:33 smithi144 bash[66654]: audit 2024-04-07T16:28:32.862817+0000 mon.a (mon.0) 1325 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-04-07T16:28:33.862 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:33 smithi144 bash[66654]: audit 2024-04-07T16:28:32.863799+0000 mon.a (mon.0) 1326 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:28:33.862 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:33 smithi144 bash[66654]: audit 2024-04-07T16:28:32.866732+0000 mon.a (mon.0) 1327 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:28:33.862 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:33 smithi144 bash[66654]: audit 2024-04-07T16:28:32.868105+0000 mon.a (mon.0) 1328 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:28:33.862 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:33 smithi144 bash[66654]: audit 2024-04-07T16:28:32.877298+0000 mon.a (mon.0) 1329 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:28:33.862 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:33 smithi144 bash[66654]: cluster 2024-04-07T16:28:32.961854+0000 mon.a (mon.0) 1330 : cluster [DBG] osdmap e755: 8 total, 8 up, 8 in 2024-04-07T16:28:33.863 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:33 smithi144 bash[66654]: audit 2024-04-07T16:28:32.974693+0000 mon.a (mon.0) 1331 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:28:33.863 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:33 smithi144 bash[66654]: audit 2024-04-07T16:28:32.979383+0000 mon.a (mon.0) 1332 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:28:33.863 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:33 smithi144 bash[66654]: audit 2024-04-07T16:28:32.981461+0000 mon.a (mon.0) 1333 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:28:33.863 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:33 smithi144 bash[66654]: audit 2024-04-07T16:28:32.992810+0000 mon.a (mon.0) 1334 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:28:33.939 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:33 smithi100 bash[74942]: audit 2024-04-07T16:28:32.544851+0000 mon.a (mon.0) 1269 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:28:33.939 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:33 smithi100 bash[74942]: audit 2024-04-07T16:28:32.556983+0000 mon.a (mon.0) 1270 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:28:33.939 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:33 smithi100 bash[74942]: audit 2024-04-07T16:28:32.559786+0000 mon.a (mon.0) 1271 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:28:33.939 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:33 smithi100 bash[74942]: audit 2024-04-07T16:28:32.561927+0000 mon.a (mon.0) 1272 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:28:33.939 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:33 smithi100 bash[74942]: audit 2024-04-07T16:28:32.572922+0000 mon.a (mon.0) 1273 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:28:33.939 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:33 smithi100 bash[74942]: audit 2024-04-07T16:28:32.661461+0000 mon.a (mon.0) 1274 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-07T16:28:33.939 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:33 smithi100 bash[74942]: audit 2024-04-07T16:28:32.662506+0000 mgr.y (mgr.25175) 390 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-07T16:28:33.939 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:33 smithi100 bash[74942]: audit 2024-04-07T16:28:32.679978+0000 mon.a (mon.0) 1275 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:28:33.939 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:33 smithi100 bash[74942]: audit 2024-04-07T16:28:32.685122+0000 mon.a (mon.0) 1276 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:28:33.939 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:33 smithi100 bash[74942]: audit 2024-04-07T16:28:32.689093+0000 mon.a (mon.0) 1277 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:28:33.939 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:33 smithi100 bash[74942]: audit 2024-04-07T16:28:32.691954+0000 mon.a (mon.0) 1278 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:28:33.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:33 smithi100 bash[74942]: audit 2024-04-07T16:28:32.697635+0000 mon.a (mon.0) 1279 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:28:33.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:33 smithi100 bash[74942]: audit 2024-04-07T16:28:32.704506+0000 mon.a (mon.0) 1280 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:28:33.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:33 smithi100 bash[74942]: audit 2024-04-07T16:28:32.707370+0000 mon.a (mon.0) 1281 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:28:33.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:33 smithi100 bash[74942]: audit 2024-04-07T16:28:32.710198+0000 mon.a (mon.0) 1282 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:28:33.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:33 smithi100 bash[74942]: audit 2024-04-07T16:28:32.713049+0000 mon.a (mon.0) 1283 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:28:33.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:33 smithi100 bash[74942]: audit 2024-04-07T16:28:32.715885+0000 mon.a (mon.0) 1284 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:28:33.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:33 smithi100 bash[74942]: audit 2024-04-07T16:28:32.718649+0000 mon.a (mon.0) 1285 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:28:33.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:33 smithi100 bash[74942]: audit 2024-04-07T16:28:32.721404+0000 mon.a (mon.0) 1286 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:28:33.941 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:33 smithi100 bash[76202]: audit 2024-04-07T16:28:32.544851+0000 mon.a (mon.0) 1269 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:28:33.941 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:33 smithi100 bash[76202]: audit 2024-04-07T16:28:32.556983+0000 mon.a (mon.0) 1270 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:28:33.941 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:33 smithi100 bash[76202]: audit 2024-04-07T16:28:32.559786+0000 mon.a (mon.0) 1271 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:28:33.941 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:33 smithi100 bash[76202]: audit 2024-04-07T16:28:32.561927+0000 mon.a (mon.0) 1272 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:28:33.941 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:33 smithi100 bash[76202]: audit 2024-04-07T16:28:32.572922+0000 mon.a (mon.0) 1273 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:28:33.941 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:33 smithi100 bash[76202]: audit 2024-04-07T16:28:32.661461+0000 mon.a (mon.0) 1274 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-07T16:28:33.941 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:33 smithi100 bash[76202]: audit 2024-04-07T16:28:32.662506+0000 mgr.y (mgr.25175) 390 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-07T16:28:33.941 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:33 smithi100 bash[76202]: audit 2024-04-07T16:28:32.679978+0000 mon.a (mon.0) 1275 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:28:33.941 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:33 smithi100 bash[76202]: audit 2024-04-07T16:28:32.685122+0000 mon.a (mon.0) 1276 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:28:33.941 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:33 smithi100 bash[76202]: audit 2024-04-07T16:28:32.689093+0000 mon.a (mon.0) 1277 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:28:33.941 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:33 smithi100 bash[76202]: audit 2024-04-07T16:28:32.691954+0000 mon.a (mon.0) 1278 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:28:33.941 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:33 smithi100 bash[76202]: audit 2024-04-07T16:28:32.697635+0000 mon.a (mon.0) 1279 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:28:33.941 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:33 smithi100 bash[76202]: audit 2024-04-07T16:28:32.704506+0000 mon.a (mon.0) 1280 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:28:33.941 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:33 smithi100 bash[76202]: audit 2024-04-07T16:28:32.707370+0000 mon.a (mon.0) 1281 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:28:33.942 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:33 smithi100 bash[76202]: audit 2024-04-07T16:28:32.710198+0000 mon.a (mon.0) 1282 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:28:33.942 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:33 smithi100 bash[76202]: audit 2024-04-07T16:28:32.713049+0000 mon.a (mon.0) 1283 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:28:33.942 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:33 smithi100 bash[76202]: audit 2024-04-07T16:28:32.715885+0000 mon.a (mon.0) 1284 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:28:33.942 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:33 smithi100 bash[76202]: audit 2024-04-07T16:28:32.718649+0000 mon.a (mon.0) 1285 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:28:33.942 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:33 smithi100 bash[76202]: audit 2024-04-07T16:28:32.721404+0000 mon.a (mon.0) 1286 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:28:33.942 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:33 smithi100 bash[76202]: audit 2024-04-07T16:28:32.724525+0000 mon.a (mon.0) 1287 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:28:33.942 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:33 smithi100 bash[76202]: audit 2024-04-07T16:28:32.728853+0000 mon.a (mon.0) 1288 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:28:33.942 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:33 smithi100 bash[76202]: audit 2024-04-07T16:28:32.732702+0000 mon.a (mon.0) 1289 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:28:33.942 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:33 smithi100 bash[76202]: audit 2024-04-07T16:28:32.736378+0000 mon.a (mon.0) 1290 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:28:33.942 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:33 smithi100 bash[76202]: audit 2024-04-07T16:28:32.740177+0000 mon.a (mon.0) 1291 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:28:33.942 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:33 smithi100 bash[76202]: audit 2024-04-07T16:28:32.743177+0000 mon.a (mon.0) 1292 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:28:33.942 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:33 smithi100 bash[76202]: audit 2024-04-07T16:28:32.746277+0000 mon.a (mon.0) 1293 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:28:33.942 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:33 smithi100 bash[76202]: cephadm 2024-04-07T16:28:32.747895+0000 mgr.y (mgr.25175) 391 : cephadm [INF] Upgrade: Finalizing container_image settings 2024-04-07T16:28:33.942 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:33 smithi100 bash[76202]: audit 2024-04-07T16:28:32.755245+0000 mon.a (mon.0) 1294 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:28:33.942 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:33 smithi100 bash[76202]: audit 2024-04-07T16:28:32.759799+0000 mon.a (mon.0) 1295 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-04-07T16:28:33.942 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:33 smithi100 bash[76202]: audit 2024-04-07T16:28:32.764422+0000 mon.a (mon.0) 1296 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-04-07T16:28:33.943 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:33 smithi100 bash[76202]: audit 2024-04-07T16:28:32.767640+0000 mon.a (mon.0) 1297 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-07T16:28:33.943 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:33 smithi100 bash[76202]: audit 2024-04-07T16:28:32.774021+0000 mon.a (mon.0) 1298 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-04-07T16:28:33.943 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:33 smithi100 bash[76202]: audit 2024-04-07T16:28:32.777456+0000 mon.a (mon.0) 1299 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-04-07T16:28:33.943 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:33 smithi100 bash[76202]: audit 2024-04-07T16:28:32.785070+0000 mon.a (mon.0) 1300 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]': finished 2024-04-07T16:28:33.943 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:33 smithi100 bash[76202]: audit 2024-04-07T16:28:32.788174+0000 mon.a (mon.0) 1301 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-04-07T16:28:33.943 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:33 smithi100 bash[76202]: audit 2024-04-07T16:28:32.794478+0000 mon.a (mon.0) 1302 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd"}]': finished 2024-04-07T16:28:33.943 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:33 smithi100 bash[76202]: audit 2024-04-07T16:28:32.797502+0000 mon.a (mon.0) 1303 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-04-07T16:28:33.943 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:33 smithi100 bash[74942]: audit 2024-04-07T16:28:32.724525+0000 mon.a (mon.0) 1287 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:28:33.943 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:33 smithi100 bash[74942]: audit 2024-04-07T16:28:32.728853+0000 mon.a (mon.0) 1288 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:28:33.943 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:33 smithi100 bash[74942]: audit 2024-04-07T16:28:32.732702+0000 mon.a (mon.0) 1289 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:28:33.943 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:33 smithi100 bash[74942]: audit 2024-04-07T16:28:32.736378+0000 mon.a (mon.0) 1290 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:28:33.944 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:33 smithi100 bash[74942]: audit 2024-04-07T16:28:32.740177+0000 mon.a (mon.0) 1291 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:28:33.944 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:33 smithi100 bash[74942]: audit 2024-04-07T16:28:32.743177+0000 mon.a (mon.0) 1292 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:28:33.944 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:33 smithi100 bash[74942]: audit 2024-04-07T16:28:32.746277+0000 mon.a (mon.0) 1293 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:28:33.944 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:33 smithi100 bash[74942]: cephadm 2024-04-07T16:28:32.747895+0000 mgr.y (mgr.25175) 391 : cephadm [INF] Upgrade: Finalizing container_image settings 2024-04-07T16:28:33.944 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:33 smithi100 bash[74942]: audit 2024-04-07T16:28:32.755245+0000 mon.a (mon.0) 1294 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:28:33.944 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:33 smithi100 bash[74942]: audit 2024-04-07T16:28:32.759799+0000 mon.a (mon.0) 1295 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-04-07T16:28:33.944 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:33 smithi100 bash[74942]: audit 2024-04-07T16:28:32.764422+0000 mon.a (mon.0) 1296 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-04-07T16:28:33.944 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:33 smithi100 bash[74942]: audit 2024-04-07T16:28:32.767640+0000 mon.a (mon.0) 1297 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-07T16:28:33.944 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:33 smithi100 bash[74942]: audit 2024-04-07T16:28:32.774021+0000 mon.a (mon.0) 1298 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-04-07T16:28:33.944 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:33 smithi100 bash[74942]: audit 2024-04-07T16:28:32.777456+0000 mon.a (mon.0) 1299 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-04-07T16:28:33.944 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:33 smithi100 bash[74942]: audit 2024-04-07T16:28:32.785070+0000 mon.a (mon.0) 1300 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]': finished 2024-04-07T16:28:33.944 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:33 smithi100 bash[74942]: audit 2024-04-07T16:28:32.788174+0000 mon.a (mon.0) 1301 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-04-07T16:28:33.944 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:33 smithi100 bash[74942]: audit 2024-04-07T16:28:32.794478+0000 mon.a (mon.0) 1302 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd"}]': finished 2024-04-07T16:28:33.944 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:33 smithi100 bash[74942]: audit 2024-04-07T16:28:32.797502+0000 mon.a (mon.0) 1303 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-04-07T16:28:33.945 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:33 smithi100 bash[76202]: audit 2024-04-07T16:28:32.801782+0000 mon.a (mon.0) 1304 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds"}]': finished 2024-04-07T16:28:33.945 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:33 smithi100 bash[76202]: audit 2024-04-07T16:28:32.804981+0000 mon.a (mon.0) 1305 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-04-07T16:28:33.945 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:33 smithi100 bash[76202]: audit 2024-04-07T16:28:32.809763+0000 mon.a (mon.0) 1306 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-04-07T16:28:33.945 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:33 smithi100 bash[76202]: audit 2024-04-07T16:28:32.812850+0000 mon.a (mon.0) 1307 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-04-07T16:28:33.945 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:33 smithi100 bash[76202]: audit 2024-04-07T16:28:32.817075+0000 mon.a (mon.0) 1308 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-04-07T16:28:33.945 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:33 smithi100 bash[76202]: cluster 2024-04-07T16:28:32.819845+0000 mgr.y (mgr.25175) 392 : cluster [DBG] pgmap v725: 121 pgs: 121 active+clean; 157 MiB data, 1.7 GiB used, 714 GiB / 715 GiB avail; 8.6 MiB/s rd, 1.2 MiB/s wr, 10 op/s 2024-04-07T16:28:33.945 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:33 smithi100 bash[76202]: audit 2024-04-07T16:28:32.821009+0000 mon.a (mon.0) 1309 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-07T16:28:33.945 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:33 smithi100 bash[76202]: audit 2024-04-07T16:28:32.822538+0000 mon.a (mon.0) 1310 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-04-07T16:28:33.945 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:33 smithi100 bash[76202]: audit 2024-04-07T16:28:32.827054+0000 mon.a (mon.0) 1311 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2024-04-07T16:28:33.945 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:33 smithi100 bash[76202]: audit 2024-04-07T16:28:32.830091+0000 mon.a (mon.0) 1312 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-04-07T16:28:33.945 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:33 smithi100 bash[76202]: audit 2024-04-07T16:28:32.834506+0000 mon.a (mon.0) 1313 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-04-07T16:28:33.945 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:33 smithi100 bash[76202]: audit 2024-04-07T16:28:32.837484+0000 mon.a (mon.0) 1314 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-04-07T16:28:33.946 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:33 smithi100 bash[76202]: audit 2024-04-07T16:28:32.841876+0000 mon.a (mon.0) 1315 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-04-07T16:28:33.946 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:33 smithi100 bash[76202]: audit 2024-04-07T16:28:32.844883+0000 mon.a (mon.0) 1316 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-04-07T16:28:33.946 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:33 smithi100 bash[76202]: audit 2024-04-07T16:28:32.849103+0000 mon.a (mon.0) 1317 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-04-07T16:28:33.946 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:33 smithi100 bash[76202]: audit 2024-04-07T16:28:32.851835+0000 mon.a (mon.0) 1318 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-07T16:28:33.946 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:33 smithi100 bash[74942]: audit 2024-04-07T16:28:32.801782+0000 mon.a (mon.0) 1304 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds"}]': finished 2024-04-07T16:28:33.946 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:33 smithi100 bash[74942]: audit 2024-04-07T16:28:32.804981+0000 mon.a (mon.0) 1305 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-04-07T16:28:33.946 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:33 smithi100 bash[74942]: audit 2024-04-07T16:28:32.809763+0000 mon.a (mon.0) 1306 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-04-07T16:28:33.946 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:33 smithi100 bash[74942]: audit 2024-04-07T16:28:32.812850+0000 mon.a (mon.0) 1307 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-04-07T16:28:33.946 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:33 smithi100 bash[74942]: audit 2024-04-07T16:28:32.817075+0000 mon.a (mon.0) 1308 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-04-07T16:28:33.946 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:33 smithi100 bash[74942]: cluster 2024-04-07T16:28:32.819845+0000 mgr.y (mgr.25175) 392 : cluster [DBG] pgmap v725: 121 pgs: 121 active+clean; 157 MiB data, 1.7 GiB used, 714 GiB / 715 GiB avail; 8.6 MiB/s rd, 1.2 MiB/s wr, 10 op/s 2024-04-07T16:28:33.946 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:33 smithi100 bash[74942]: audit 2024-04-07T16:28:32.821009+0000 mon.a (mon.0) 1309 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-07T16:28:33.947 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:33 smithi100 bash[74942]: audit 2024-04-07T16:28:32.822538+0000 mon.a (mon.0) 1310 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-04-07T16:28:33.947 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:33 smithi100 bash[74942]: audit 2024-04-07T16:28:32.827054+0000 mon.a (mon.0) 1311 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2024-04-07T16:28:33.947 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:33 smithi100 bash[74942]: audit 2024-04-07T16:28:32.830091+0000 mon.a (mon.0) 1312 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-04-07T16:28:33.947 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:33 smithi100 bash[74942]: audit 2024-04-07T16:28:32.834506+0000 mon.a (mon.0) 1313 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-04-07T16:28:33.947 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:33 smithi100 bash[74942]: audit 2024-04-07T16:28:32.837484+0000 mon.a (mon.0) 1314 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-04-07T16:28:33.947 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:33 smithi100 bash[74942]: audit 2024-04-07T16:28:32.841876+0000 mon.a (mon.0) 1315 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-04-07T16:28:33.947 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:33 smithi100 bash[74942]: audit 2024-04-07T16:28:32.844883+0000 mon.a (mon.0) 1316 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-04-07T16:28:33.947 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:33 smithi100 bash[74942]: audit 2024-04-07T16:28:32.849103+0000 mon.a (mon.0) 1317 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-04-07T16:28:33.947 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:33 smithi100 bash[74942]: audit 2024-04-07T16:28:32.851835+0000 mon.a (mon.0) 1318 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-07T16:28:33.947 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:33 smithi100 bash[76202]: audit 2024-04-07T16:28:32.852869+0000 mon.a (mon.0) 1319 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-07T16:28:33.947 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:33 smithi100 bash[76202]: audit 2024-04-07T16:28:32.853816+0000 mon.a (mon.0) 1320 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-07T16:28:33.947 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:33 smithi100 bash[76202]: audit 2024-04-07T16:28:32.854950+0000 mon.a (mon.0) 1321 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-07T16:28:33.948 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:33 smithi100 bash[76202]: audit 2024-04-07T16:28:32.855974+0000 mon.a (mon.0) 1322 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-07T16:28:33.948 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:33 smithi100 bash[76202]: audit 2024-04-07T16:28:32.857224+0000 mon.a (mon.0) 1323 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-07T16:28:33.948 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:33 smithi100 bash[76202]: cephadm 2024-04-07T16:28:32.858037+0000 mgr.y (mgr.25175) 393 : cephadm [INF] Upgrade: Complete! 2024-04-07T16:28:33.948 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:33 smithi100 bash[76202]: audit 2024-04-07T16:28:32.858617+0000 mon.a (mon.0) 1324 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-04-07T16:28:33.948 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:33 smithi100 bash[76202]: audit 2024-04-07T16:28:32.862817+0000 mon.a (mon.0) 1325 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-04-07T16:28:33.948 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:33 smithi100 bash[76202]: audit 2024-04-07T16:28:32.863799+0000 mon.a (mon.0) 1326 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:28:33.948 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:33 smithi100 bash[76202]: audit 2024-04-07T16:28:32.866732+0000 mon.a (mon.0) 1327 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:28:33.948 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:33 smithi100 bash[76202]: audit 2024-04-07T16:28:32.868105+0000 mon.a (mon.0) 1328 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:28:33.948 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:33 smithi100 bash[76202]: audit 2024-04-07T16:28:32.877298+0000 mon.a (mon.0) 1329 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:28:33.948 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:33 smithi100 bash[76202]: cluster 2024-04-07T16:28:32.961854+0000 mon.a (mon.0) 1330 : cluster [DBG] osdmap e755: 8 total, 8 up, 8 in 2024-04-07T16:28:33.948 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:33 smithi100 bash[76202]: audit 2024-04-07T16:28:32.974693+0000 mon.a (mon.0) 1331 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:28:33.948 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:33 smithi100 bash[76202]: audit 2024-04-07T16:28:32.979383+0000 mon.a (mon.0) 1332 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:28:33.948 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:33 smithi100 bash[76202]: audit 2024-04-07T16:28:32.981461+0000 mon.a (mon.0) 1333 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:28:33.948 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:33 smithi100 bash[76202]: audit 2024-04-07T16:28:32.992810+0000 mon.a (mon.0) 1334 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:28:33.948 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:33 smithi100 bash[74942]: audit 2024-04-07T16:28:32.852869+0000 mon.a (mon.0) 1319 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-07T16:28:33.949 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:33 smithi100 bash[74942]: audit 2024-04-07T16:28:32.853816+0000 mon.a (mon.0) 1320 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-07T16:28:33.949 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:33 smithi100 bash[74942]: audit 2024-04-07T16:28:32.854950+0000 mon.a (mon.0) 1321 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-07T16:28:33.949 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:33 smithi100 bash[74942]: audit 2024-04-07T16:28:32.855974+0000 mon.a (mon.0) 1322 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-07T16:28:33.949 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:33 smithi100 bash[74942]: audit 2024-04-07T16:28:32.857224+0000 mon.a (mon.0) 1323 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-07T16:28:33.949 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:33 smithi100 bash[74942]: cephadm 2024-04-07T16:28:32.858037+0000 mgr.y (mgr.25175) 393 : cephadm [INF] Upgrade: Complete! 2024-04-07T16:28:33.949 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:33 smithi100 bash[74942]: audit 2024-04-07T16:28:32.858617+0000 mon.a (mon.0) 1324 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-04-07T16:28:33.949 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:33 smithi100 bash[74942]: audit 2024-04-07T16:28:32.862817+0000 mon.a (mon.0) 1325 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-04-07T16:28:33.949 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:33 smithi100 bash[74942]: audit 2024-04-07T16:28:32.863799+0000 mon.a (mon.0) 1326 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:28:33.949 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:33 smithi100 bash[74942]: audit 2024-04-07T16:28:32.866732+0000 mon.a (mon.0) 1327 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:28:33.949 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:33 smithi100 bash[74942]: audit 2024-04-07T16:28:32.868105+0000 mon.a (mon.0) 1328 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:28:33.949 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:33 smithi100 bash[74942]: audit 2024-04-07T16:28:32.877298+0000 mon.a (mon.0) 1329 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:28:33.949 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:33 smithi100 bash[74942]: cluster 2024-04-07T16:28:32.961854+0000 mon.a (mon.0) 1330 : cluster [DBG] osdmap e755: 8 total, 8 up, 8 in 2024-04-07T16:28:33.949 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:33 smithi100 bash[74942]: audit 2024-04-07T16:28:32.974693+0000 mon.a (mon.0) 1331 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:28:33.949 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:33 smithi100 bash[74942]: audit 2024-04-07T16:28:32.979383+0000 mon.a (mon.0) 1332 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:28:33.949 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:33 smithi100 bash[74942]: audit 2024-04-07T16:28:32.981461+0000 mon.a (mon.0) 1333 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:28:33.950 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:33 smithi100 bash[74942]: audit 2024-04-07T16:28:32.992810+0000 mon.a (mon.0) 1334 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:28:33.958 INFO:tasks.rados.rados.0.smithi100.stdout:3742: done (1 left) 2024-04-07T16:28:33.958 INFO:tasks.rados.rados.0.smithi100.stdout:3744: done (0 left) 2024-04-07T16:28:33.958 INFO:tasks.rados.rados.0.smithi100.stdout:3745: snap_create 2024-04-07T16:28:34.962 INFO:tasks.rados.rados.0.smithi100.stdout:3745: done (0 left) 2024-04-07T16:28:34.962 INFO:tasks.rados.rados.0.smithi100.stdout:3746: rollback oid 11 current snap is 345 2024-04-07T16:28:34.962 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 11 to 327 2024-04-07T16:28:34.962 INFO:tasks.rados.rados.0.smithi100.stdout:3747: rollback oid 32 current snap is 345 2024-04-07T16:28:34.962 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 32 to 329 2024-04-07T16:28:34.962 INFO:tasks.rados.rados.0.smithi100.stdout:3748: delete oid 15 current snap is 345 2024-04-07T16:28:34.978 INFO:tasks.rados.rados.0.smithi100.stdout:3747: finishing rollback tid 1 to smithi10066867-32 2024-04-07T16:28:34.978 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 32 v 1703 (ObjNum 337 snap 0 seq_num 3067285024) dirty dne 2024-04-07T16:28:34.979 INFO:tasks.rados.rados.0.smithi100.stdout:3746: finishing rollback tid 1 to smithi10066867-11 2024-04-07T16:28:34.980 INFO:tasks.rados.rados.0.smithi100.stdout:3746: finishing rollback tid 2 to smithi10066867-11 2024-04-07T16:28:34.980 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 11 v 1920 (ObjNum 1113 snap 321 seq_num 1113) dirty exists 2024-04-07T16:28:34.982 INFO:tasks.rados.rados.0.smithi100.stdout:3746: done (2 left) 2024-04-07T16:28:34.982 INFO:tasks.rados.rados.0.smithi100.stdout:3747: done (1 left) 2024-04-07T16:28:34.982 INFO:tasks.rados.rados.0.smithi100.stdout:3748: done (0 left) 2024-04-07T16:28:34.982 INFO:tasks.rados.rados.0.smithi100.stdout:3749: rollback oid 45 current snap is 345 2024-04-07T16:28:34.982 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 45 to 332 2024-04-07T16:28:34.982 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 40 current snap is 345 2024-04-07T16:28:34.982 INFO:tasks.rados.rados.0.smithi100.stdout:3750: seq_num 1165 ranges {3620864=319488} 2024-04-07T16:28:35.041 INFO:tasks.rados.rados.0.smithi100.stdout:3750: writing smithi10066867-40 from 3620864 to 3940352 tid 1 2024-04-07T16:28:35.041 INFO:tasks.rados.rados.0.smithi100.stdout:3751: snap_create 2024-04-07T16:28:35.041 INFO:tasks.rados.rados.0.smithi100.stdout:3749: finishing rollback tid 0 to smithi10066867-45 2024-04-07T16:28:35.041 INFO:tasks.rados.rados.0.smithi100.stdout:3749: finishing rollback tid 1 to smithi10066867-45 2024-04-07T16:28:35.041 INFO:tasks.rados.rados.0.smithi100.stdout:3749: finishing rollback tid 2 to smithi10066867-45 2024-04-07T16:28:35.041 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 45 v 1819 (ObjNum 1041 snap 307 seq_num 1041) dirty exists 2024-04-07T16:28:35.048 INFO:tasks.rados.rados.0.smithi100.stdout:3750: finishing write tid 1 to smithi10066867-40 2024-04-07T16:28:35.048 INFO:tasks.rados.rados.0.smithi100.stdout:3750: finishing write tid 2 to smithi10066867-40 2024-04-07T16:28:35.051 INFO:tasks.rados.rados.0.smithi100.stdout:3750: finishing write tid 3 to smithi10066867-40 2024-04-07T16:28:35.051 INFO:tasks.rados.rados.0.smithi100.stdout:3750: oid 40 updating version 0 to 1681 2024-04-07T16:28:35.051 INFO:tasks.rados.rados.0.smithi100.stdout:3750: oid 40 version 1681 is already newer than 1680 2024-04-07T16:28:35.051 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 40 v 1681 (ObjNum 1165 snap 345 seq_num 1165) dirty exists 2024-04-07T16:28:35.051 INFO:tasks.rados.rados.0.smithi100.stdout:3750: left oid 40 (ObjNum 1165 snap 345 seq_num 1165) 2024-04-07T16:28:35.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:34 smithi144 bash[66654]: cluster 2024-04-07T16:28:33.971238+0000 mon.a (mon.0) 1335 : cluster [DBG] osdmap e756: 8 total, 8 up, 8 in 2024-04-07T16:28:35.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:34 smithi100 bash[74942]: cluster 2024-04-07T16:28:33.971238+0000 mon.a (mon.0) 1335 : cluster [DBG] osdmap e756: 8 total, 8 up, 8 in 2024-04-07T16:28:35.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:34 smithi100 bash[76202]: cluster 2024-04-07T16:28:33.971238+0000 mon.a (mon.0) 1335 : cluster [DBG] osdmap e756: 8 total, 8 up, 8 in 2024-04-07T16:28:35.964 INFO:tasks.rados.rados.0.smithi100.stdout:3749: done (2 left) 2024-04-07T16:28:35.964 INFO:tasks.rados.rados.0.smithi100.stdout:3750: done (1 left) 2024-04-07T16:28:35.964 INFO:tasks.rados.rados.0.smithi100.stdout:3751: done (0 left) 2024-04-07T16:28:35.964 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 13 current snap is 346 2024-04-07T16:28:35.965 INFO:tasks.rados.rados.0.smithi100.stdout:3752: seq_num 1166 ranges {0=335872} 2024-04-07T16:28:35.968 INFO:tasks.rados.rados.0.smithi100.stdout:3752: writing smithi10066867-13 from 0 to 335872 tid 1 2024-04-07T16:28:35.969 INFO:tasks.rados.rados.0.smithi100.stdout:3753: rollback oid 30 current snap is 346 2024-04-07T16:28:35.969 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 30 to 329 2024-04-07T16:28:35.969 INFO:tasks.rados.rados.0.smithi100.stdout:3754: snap_create 2024-04-07T16:28:35.987 INFO:tasks.rados.rados.0.smithi100.stdout:3752: finishing write tid 1 to smithi10066867-13 2024-04-07T16:28:35.988 INFO:tasks.rados.rados.0.smithi100.stdout:3752: finishing write tid 2 to smithi10066867-13 2024-04-07T16:28:35.991 INFO:tasks.rados.rados.0.smithi100.stdout:3753: finishing rollback tid 0 to smithi10066867-30 2024-04-07T16:28:35.991 INFO:tasks.rados.rados.0.smithi100.stdout:3753: finishing rollback tid 1 to smithi10066867-30 2024-04-07T16:28:35.992 INFO:tasks.rados.rados.0.smithi100.stdout:3752: finishing write tid 3 to smithi10066867-13 2024-04-07T16:28:35.992 INFO:tasks.rados.rados.0.smithi100.stdout:3752: oid 13 updating version 0 to 1864 2024-04-07T16:28:35.992 INFO:tasks.rados.rados.0.smithi100.stdout:3752: oid 13 version 1864 is already newer than 1863 2024-04-07T16:28:35.992 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 13 v 1864 (ObjNum 1166 snap 346 seq_num 1166) dirty exists 2024-04-07T16:28:35.992 INFO:tasks.rados.rados.0.smithi100.stdout:3752: left oid 13 (ObjNum 1166 snap 346 seq_num 1166) 2024-04-07T16:28:35.993 INFO:tasks.rados.rados.0.smithi100.stdout:3753: finishing rollback tid 2 to smithi10066867-30 2024-04-07T16:28:35.993 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 30 v 1594 (ObjNum 1081 snap 314 seq_num 1081) dirty exists 2024-04-07T16:28:36.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:35 smithi144 bash[66654]: cluster 2024-04-07T16:28:34.820714+0000 mgr.y (mgr.25175) 394 : cluster [DBG] pgmap v728: 121 pgs: 121 active+clean; 158 MiB data, 1.7 GiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 674 KiB/s wr, 9 op/s 2024-04-07T16:28:36.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:35 smithi144 bash[66654]: cluster 2024-04-07T16:28:34.975395+0000 mon.a (mon.0) 1336 : cluster [DBG] osdmap e757: 8 total, 8 up, 8 in 2024-04-07T16:28:36.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:35 smithi100 bash[74942]: cluster 2024-04-07T16:28:34.820714+0000 mgr.y (mgr.25175) 394 : cluster [DBG] pgmap v728: 121 pgs: 121 active+clean; 158 MiB data, 1.7 GiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 674 KiB/s wr, 9 op/s 2024-04-07T16:28:36.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:35 smithi100 bash[74942]: cluster 2024-04-07T16:28:34.975395+0000 mon.a (mon.0) 1336 : cluster [DBG] osdmap e757: 8 total, 8 up, 8 in 2024-04-07T16:28:36.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:36 smithi100 bash[76202]: cluster 2024-04-07T16:28:34.820714+0000 mgr.y (mgr.25175) 394 : cluster [DBG] pgmap v728: 121 pgs: 121 active+clean; 158 MiB data, 1.7 GiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 674 KiB/s wr, 9 op/s 2024-04-07T16:28:36.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:36 smithi100 bash[76202]: cluster 2024-04-07T16:28:34.975395+0000 mon.a (mon.0) 1336 : cluster [DBG] osdmap e757: 8 total, 8 up, 8 in 2024-04-07T16:28:36.950 INFO:tasks.rados.rados.0.smithi100.stdout:3752: done (2 left) 2024-04-07T16:28:36.950 INFO:tasks.rados.rados.0.smithi100.stdout:3753: done (1 left) 2024-04-07T16:28:36.950 INFO:tasks.rados.rados.0.smithi100.stdout:3754: done (0 left) 2024-04-07T16:28:36.950 INFO:tasks.rados.rados.0.smithi100.stdout:3755: read oid 41 snap -1 2024-04-07T16:28:36.950 INFO:tasks.rados.rados.0.smithi100.stdout:3755: expect (ObjNum 0 snap 0 seq_num 0) 2024-04-07T16:28:36.950 INFO:tasks.rados.rados.0.smithi100.stdout:3756: delete oid 28 current snap is 347 2024-04-07T16:28:36.959 INFO:tasks.rados.rados.0.smithi100.stdout:3755: expect (ObjNum 0 snap 0 seq_num 0) 2024-04-07T16:28:36.967 INFO:tasks.rados.rados.0.smithi100.stdout:3755: done (1 left) 2024-04-07T16:28:36.967 INFO:tasks.rados.rados.0.smithi100.stdout:3756: done (0 left) 2024-04-07T16:28:36.967 INFO:tasks.rados.rados.0.smithi100.stdout:3757: snap_create 2024-04-07T16:28:37.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:36 smithi144 bash[66654]: cluster 2024-04-07T16:28:35.976839+0000 mon.a (mon.0) 1337 : cluster [DBG] osdmap e758: 8 total, 8 up, 8 in 2024-04-07T16:28:37.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:36 smithi144 bash[66654]: cluster 2024-04-07T16:28:36.963728+0000 mon.a (mon.0) 1338 : cluster [DBG] osdmap e759: 8 total, 8 up, 8 in 2024-04-07T16:28:37.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:36 smithi100 bash[74942]: cluster 2024-04-07T16:28:35.976839+0000 mon.a (mon.0) 1337 : cluster [DBG] osdmap e758: 8 total, 8 up, 8 in 2024-04-07T16:28:37.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:36 smithi100 bash[74942]: cluster 2024-04-07T16:28:36.963728+0000 mon.a (mon.0) 1338 : cluster [DBG] osdmap e759: 8 total, 8 up, 8 in 2024-04-07T16:28:37.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:36 smithi100 bash[76202]: cluster 2024-04-07T16:28:35.976839+0000 mon.a (mon.0) 1337 : cluster [DBG] osdmap e758: 8 total, 8 up, 8 in 2024-04-07T16:28:37.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:36 smithi100 bash[76202]: cluster 2024-04-07T16:28:36.963728+0000 mon.a (mon.0) 1338 : cluster [DBG] osdmap e759: 8 total, 8 up, 8 in 2024-04-07T16:28:37.951 INFO:tasks.rados.rados.0.smithi100.stdout:3757: done (0 left) 2024-04-07T16:28:37.951 INFO:tasks.rados.rados.0.smithi100.stdout:3758: snap_remove snap 337 2024-04-07T16:28:38.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:38 smithi144 bash[66654]: cluster 2024-04-07T16:28:36.821341+0000 mgr.y (mgr.25175) 395 : cluster [DBG] pgmap v731: 121 pgs: 121 active+clean; 158 MiB data, 1.7 GiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 144 KiB/s wr, 8 op/s 2024-04-07T16:28:38.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:38 smithi144 bash[66654]: audit 2024-04-07T16:28:37.035706+0000 mon.a (mon.0) 1339 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:28:38.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:38 smithi144 bash[66654]: cluster 2024-04-07T16:28:37.964938+0000 mon.a (mon.0) 1340 : cluster [DBG] osdmap e760: 8 total, 8 up, 8 in 2024-04-07T16:28:38.417 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:38 smithi100 bash[76202]: cluster 2024-04-07T16:28:36.821341+0000 mgr.y (mgr.25175) 395 : cluster [DBG] pgmap v731: 121 pgs: 121 active+clean; 158 MiB data, 1.7 GiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 144 KiB/s wr, 8 op/s 2024-04-07T16:28:38.417 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:38 smithi100 bash[76202]: audit 2024-04-07T16:28:37.035706+0000 mon.a (mon.0) 1339 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:28:38.417 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:38 smithi100 bash[76202]: cluster 2024-04-07T16:28:37.964938+0000 mon.a (mon.0) 1340 : cluster [DBG] osdmap e760: 8 total, 8 up, 8 in 2024-04-07T16:28:38.417 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:38 smithi100 bash[74942]: cluster 2024-04-07T16:28:36.821341+0000 mgr.y (mgr.25175) 395 : cluster [DBG] pgmap v731: 121 pgs: 121 active+clean; 158 MiB data, 1.7 GiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 144 KiB/s wr, 8 op/s 2024-04-07T16:28:38.418 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:38 smithi100 bash[74942]: audit 2024-04-07T16:28:37.035706+0000 mon.a (mon.0) 1339 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:28:38.418 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:38 smithi100 bash[74942]: cluster 2024-04-07T16:28:37.964938+0000 mon.a (mon.0) 1340 : cluster [DBG] osdmap e760: 8 total, 8 up, 8 in 2024-04-07T16:28:38.688 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:28:38 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:28:38] "GET /metrics HTTP/1.1" 200 36338 "" "Prometheus/2.43.0" 2024-04-07T16:28:38.955 INFO:tasks.rados.rados.0.smithi100.stdout:3758: done (0 left) 2024-04-07T16:28:38.956 INFO:tasks.rados.rados.0.smithi100.stdout:3759: snap_create 2024-04-07T16:28:39.955 INFO:tasks.rados.rados.0.smithi100.stdout:3759: done (0 left) 2024-04-07T16:28:39.955 INFO:tasks.rados.rados.0.smithi100.stdout:3760: copy_from oid 13 from oid 23 current snap is 349 2024-04-07T16:28:39.955 INFO:tasks.rados.rados.0.smithi100.stdout:3761: delete oid 7 current snap is 349 2024-04-07T16:28:39.969 INFO:tasks.rados.rados.0.smithi100.stdout:3760: finishing copy_from to smithi10066867-13 2024-04-07T16:28:39.969 INFO:tasks.rados.rados.0.smithi100.stdout:3760: got expected ENOENT (src dne) 2024-04-07T16:28:39.969 INFO:tasks.rados.rados.0.smithi100.stdout:3760: finishing copy_from racing read to smithi10066867-13 2024-04-07T16:28:39.979 INFO:tasks.rados.rados.0.smithi100.stdout:3760: done (1 left) 2024-04-07T16:28:39.979 INFO:tasks.rados.rados.0.smithi100.stdout:3761: done (0 left) 2024-04-07T16:28:39.979 INFO:tasks.rados.rados.0.smithi100.stdout:3762: snap_create 2024-04-07T16:28:40.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:39 smithi144 bash[66654]: cluster 2024-04-07T16:28:38.822178+0000 mgr.y (mgr.25175) 396 : cluster [DBG] pgmap v734: 121 pgs: 121 active+clean; 158 MiB data, 1.7 GiB used, 714 GiB / 715 GiB avail; 14 KiB/s rd, 66 KiB/s wr, 3 op/s 2024-04-07T16:28:40.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:39 smithi144 bash[66654]: cluster 2024-04-07T16:28:38.966995+0000 mon.a (mon.0) 1341 : cluster [DBG] osdmap e761: 8 total, 8 up, 8 in 2024-04-07T16:28:40.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:39 smithi100 bash[74942]: cluster 2024-04-07T16:28:38.822178+0000 mgr.y (mgr.25175) 396 : cluster [DBG] pgmap v734: 121 pgs: 121 active+clean; 158 MiB data, 1.7 GiB used, 714 GiB / 715 GiB avail; 14 KiB/s rd, 66 KiB/s wr, 3 op/s 2024-04-07T16:28:40.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:39 smithi100 bash[74942]: cluster 2024-04-07T16:28:38.966995+0000 mon.a (mon.0) 1341 : cluster [DBG] osdmap e761: 8 total, 8 up, 8 in 2024-04-07T16:28:40.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:39 smithi100 bash[76202]: cluster 2024-04-07T16:28:38.822178+0000 mgr.y (mgr.25175) 396 : cluster [DBG] pgmap v734: 121 pgs: 121 active+clean; 158 MiB data, 1.7 GiB used, 714 GiB / 715 GiB avail; 14 KiB/s rd, 66 KiB/s wr, 3 op/s 2024-04-07T16:28:40.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:39 smithi100 bash[76202]: cluster 2024-04-07T16:28:38.966995+0000 mon.a (mon.0) 1341 : cluster [DBG] osdmap e761: 8 total, 8 up, 8 in 2024-04-07T16:28:40.981 INFO:tasks.rados.rados.0.smithi100.stdout:3762: done (0 left) 2024-04-07T16:28:40.981 INFO:tasks.rados.rados.0.smithi100.stdout:3763: snap_create 2024-04-07T16:28:41.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:40 smithi144 bash[66654]: cluster 2024-04-07T16:28:39.968448+0000 mon.a (mon.0) 1342 : cluster [DBG] osdmap e762: 8 total, 8 up, 8 in 2024-04-07T16:28:41.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:40 smithi100 bash[74942]: cluster 2024-04-07T16:28:39.968448+0000 mon.a (mon.0) 1342 : cluster [DBG] osdmap e762: 8 total, 8 up, 8 in 2024-04-07T16:28:41.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:40 smithi100 bash[76202]: cluster 2024-04-07T16:28:39.968448+0000 mon.a (mon.0) 1342 : cluster [DBG] osdmap e762: 8 total, 8 up, 8 in 2024-04-07T16:28:41.952 INFO:tasks.rados.rados.0.smithi100.stdout:3763: done (0 left) 2024-04-07T16:28:41.953 INFO:tasks.rados.rados.0.smithi100.stdout:3764: snap_create 2024-04-07T16:28:42.304 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:41 smithi144 bash[66654]: cluster 2024-04-07T16:28:40.822932+0000 mgr.y (mgr.25175) 397 : cluster [DBG] pgmap v737: 121 pgs: 121 active+clean; 168 MiB data, 1.7 GiB used, 714 GiB / 715 GiB avail; 15 KiB/s rd, 226 KiB/s wr, 5 op/s 2024-04-07T16:28:42.304 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:41 smithi144 bash[66654]: cluster 2024-04-07T16:28:40.994382+0000 mon.a (mon.0) 1343 : cluster [DBG] osdmap e763: 8 total, 8 up, 8 in 2024-04-07T16:28:42.305 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:41 smithi144 bash[66654]: cluster 2024-04-07T16:28:41.964818+0000 mon.a (mon.0) 1344 : cluster [DBG] osdmap e764: 8 total, 8 up, 8 in 2024-04-07T16:28:42.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:41 smithi100 bash[74942]: cluster 2024-04-07T16:28:40.822932+0000 mgr.y (mgr.25175) 397 : cluster [DBG] pgmap v737: 121 pgs: 121 active+clean; 168 MiB data, 1.7 GiB used, 714 GiB / 715 GiB avail; 15 KiB/s rd, 226 KiB/s wr, 5 op/s 2024-04-07T16:28:42.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:41 smithi100 bash[74942]: cluster 2024-04-07T16:28:40.994382+0000 mon.a (mon.0) 1343 : cluster [DBG] osdmap e763: 8 total, 8 up, 8 in 2024-04-07T16:28:42.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:41 smithi100 bash[74942]: cluster 2024-04-07T16:28:41.964818+0000 mon.a (mon.0) 1344 : cluster [DBG] osdmap e764: 8 total, 8 up, 8 in 2024-04-07T16:28:42.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:41 smithi100 bash[76202]: cluster 2024-04-07T16:28:40.822932+0000 mgr.y (mgr.25175) 397 : cluster [DBG] pgmap v737: 121 pgs: 121 active+clean; 168 MiB data, 1.7 GiB used, 714 GiB / 715 GiB avail; 15 KiB/s rd, 226 KiB/s wr, 5 op/s 2024-04-07T16:28:42.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:41 smithi100 bash[76202]: cluster 2024-04-07T16:28:40.994382+0000 mon.a (mon.0) 1343 : cluster [DBG] osdmap e763: 8 total, 8 up, 8 in 2024-04-07T16:28:42.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:41 smithi100 bash[76202]: cluster 2024-04-07T16:28:41.964818+0000 mon.a (mon.0) 1344 : cluster [DBG] osdmap e764: 8 total, 8 up, 8 in 2024-04-07T16:28:42.606 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:28:42 smithi144 bash[95233]: ts=2024-04-07T16:28:42.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:28:42.956 INFO:tasks.rados.rados.0.smithi100.stdout:3764: done (0 left) 2024-04-07T16:28:42.956 INFO:tasks.rados.rados.0.smithi100.stdout:3765: snap_create 2024-04-07T16:28:43.952 INFO:tasks.rados.rados.0.smithi100.stdout:3765: done (0 left) 2024-04-07T16:28:43.952 INFO:tasks.rados.rados.0.smithi100.stdout:3766: read oid 3 snap 343 2024-04-07T16:28:43.952 INFO:tasks.rados.rados.0.smithi100.stdout:3766: expect (ObjNum -1592697488 snap 21978 seq_num 2702290400) 2024-04-07T16:28:43.952 INFO:tasks.rados.rados.0.smithi100.stdout:3767: snap_create 2024-04-07T16:28:43.958 INFO:tasks.rados.rados.0.smithi100.stdout:3766: expect (ObjNum -1592697488 snap 21978 seq_num 2702290400) 2024-04-07T16:28:44.188 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:43 smithi100 bash[74942]: cluster 2024-04-07T16:28:42.824051+0000 mgr.y (mgr.25175) 398 : cluster [DBG] pgmap v740: 121 pgs: 121 active+clean; 168 MiB data, 1.7 GiB used, 714 GiB / 715 GiB avail; 511 B/s rd, 160 KiB/s wr, 2 op/s 2024-04-07T16:28:44.188 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:43 smithi100 bash[74942]: cluster 2024-04-07T16:28:42.969147+0000 mon.a (mon.0) 1345 : cluster [DBG] osdmap e765: 8 total, 8 up, 8 in 2024-04-07T16:28:44.189 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:43 smithi100 bash[76202]: cluster 2024-04-07T16:28:42.824051+0000 mgr.y (mgr.25175) 398 : cluster [DBG] pgmap v740: 121 pgs: 121 active+clean; 168 MiB data, 1.7 GiB used, 714 GiB / 715 GiB avail; 511 B/s rd, 160 KiB/s wr, 2 op/s 2024-04-07T16:28:44.189 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:43 smithi100 bash[76202]: cluster 2024-04-07T16:28:42.969147+0000 mon.a (mon.0) 1345 : cluster [DBG] osdmap e765: 8 total, 8 up, 8 in 2024-04-07T16:28:44.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:43 smithi144 bash[66654]: cluster 2024-04-07T16:28:42.824051+0000 mgr.y (mgr.25175) 398 : cluster [DBG] pgmap v740: 121 pgs: 121 active+clean; 168 MiB data, 1.7 GiB used, 714 GiB / 715 GiB avail; 511 B/s rd, 160 KiB/s wr, 2 op/s 2024-04-07T16:28:44.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:43 smithi144 bash[66654]: cluster 2024-04-07T16:28:42.969147+0000 mon.a (mon.0) 1345 : cluster [DBG] osdmap e765: 8 total, 8 up, 8 in 2024-04-07T16:28:44.694 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0702e3a8-f4f8-11ee-b649-cb9ed24678a4 -e sha1=adce92d056533f22dc1ee37e3141d098b046ec1b -- bash -c 'ceph orch ps' 2024-04-07T16:28:44.960 INFO:tasks.rados.rados.0.smithi100.stdout:3766: done (1 left) 2024-04-07T16:28:44.960 INFO:tasks.rados.rados.0.smithi100.stdout:3767: done (0 left) 2024-04-07T16:28:44.960 INFO:tasks.rados.rados.0.smithi100.stdout:3768: read oid 48 snap 322 2024-04-07T16:28:44.960 INFO:tasks.rados.rados.0.smithi100.stdout:3768: expect (ObjNum 941 snap 282 seq_num 941) 2024-04-07T16:28:44.960 INFO:tasks.rados.rados.0.smithi100.stdout:3769: read oid 39 snap 344 2024-04-07T16:28:44.960 INFO:tasks.rados.rados.0.smithi100.stdout:3769: expect deleted 2024-04-07T16:28:44.960 INFO:tasks.rados.rados.0.smithi100.stdout:3770: delete oid 36 current snap is 354 2024-04-07T16:28:44.988 INFO:tasks.rados.rados.0.smithi100.stdout:3768: expect (ObjNum 941 snap 282 seq_num 941) 2024-04-07T16:28:45.062 INFO:tasks.rados.rados.0.smithi100.stdout:3768: done (2 left) 2024-04-07T16:28:45.062 INFO:tasks.rados.rados.0.smithi100.stdout:3769: done (1 left) 2024-04-07T16:28:45.062 INFO:tasks.rados.rados.0.smithi100.stdout:3770: done (0 left) 2024-04-07T16:28:45.062 INFO:tasks.rados.rados.0.smithi100.stdout:3771: snap_remove snap 322 2024-04-07T16:28:45.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:44 smithi144 bash[66654]: audit 2024-04-07T16:28:43.842377+0000 mgr.y (mgr.25175) 399 : audit [DBG] from='client.34500 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:28:45.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:44 smithi144 bash[66654]: cluster 2024-04-07T16:28:43.963598+0000 mon.a (mon.0) 1346 : cluster [DBG] osdmap e766: 8 total, 8 up, 8 in 2024-04-07T16:28:45.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:44 smithi100 bash[74942]: audit 2024-04-07T16:28:43.842377+0000 mgr.y (mgr.25175) 399 : audit [DBG] from='client.34500 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:28:45.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:44 smithi100 bash[74942]: cluster 2024-04-07T16:28:43.963598+0000 mon.a (mon.0) 1346 : cluster [DBG] osdmap e766: 8 total, 8 up, 8 in 2024-04-07T16:28:45.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:44 smithi100 bash[76202]: audit 2024-04-07T16:28:43.842377+0000 mgr.y (mgr.25175) 399 : audit [DBG] from='client.34500 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:28:45.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:44 smithi100 bash[76202]: cluster 2024-04-07T16:28:43.963598+0000 mon.a (mon.0) 1346 : cluster [DBG] osdmap e766: 8 total, 8 up, 8 in 2024-04-07T16:28:45.983 INFO:tasks.rados.rados.0.smithi100.stdout:3771: done (0 left) 2024-04-07T16:28:45.984 INFO:tasks.rados.rados.0.smithi100.stdout:3772: read oid 9 snap -1 2024-04-07T16:28:45.984 INFO:tasks.rados.rados.0.smithi100.stdout:3772: expect (ObjNum 1153 snap 340 seq_num 1153) 2024-04-07T16:28:45.984 INFO:tasks.rados.rados.0.smithi100.stdout:3773: snap_create 2024-04-07T16:28:46.009 INFO:tasks.rados.rados.0.smithi100.stdout:3772: expect (ObjNum 1153 snap 340 seq_num 1153) 2024-04-07T16:28:46.356 INFO:journalctl@ceph.osd.6.smithi144.stdout:Apr 07 16:28:46 smithi144 bash[79803]: debug 2024-04-07T16:28:46.017+0000 7f89390f6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 1a1 after trimming was started 2024-04-07T16:28:46.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:45 smithi144 bash[66654]: cluster 2024-04-07T16:28:44.824890+0000 mgr.y (mgr.25175) 400 : cluster [DBG] pgmap v743: 121 pgs: 121 active+clean; 162 MiB data, 1.7 GiB used, 714 GiB / 715 GiB avail; 0 B/s rd, 0 op/s 2024-04-07T16:28:46.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:45 smithi144 bash[66654]: cluster 2024-04-07T16:28:44.970247+0000 mon.a (mon.0) 1347 : cluster [DBG] osdmap e767: 8 total, 8 up, 8 in 2024-04-07T16:28:46.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:45 smithi100 bash[74942]: cluster 2024-04-07T16:28:44.824890+0000 mgr.y (mgr.25175) 400 : cluster [DBG] pgmap v743: 121 pgs: 121 active+clean; 162 MiB data, 1.7 GiB used, 714 GiB / 715 GiB avail; 0 B/s rd, 0 op/s 2024-04-07T16:28:46.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:45 smithi100 bash[74942]: cluster 2024-04-07T16:28:44.970247+0000 mon.a (mon.0) 1347 : cluster [DBG] osdmap e767: 8 total, 8 up, 8 in 2024-04-07T16:28:46.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:45 smithi100 bash[76202]: cluster 2024-04-07T16:28:44.824890+0000 mgr.y (mgr.25175) 400 : cluster [DBG] pgmap v743: 121 pgs: 121 active+clean; 162 MiB data, 1.7 GiB used, 714 GiB / 715 GiB avail; 0 B/s rd, 0 op/s 2024-04-07T16:28:46.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:45 smithi100 bash[76202]: cluster 2024-04-07T16:28:44.970247+0000 mon.a (mon.0) 1347 : cluster [DBG] osdmap e767: 8 total, 8 up, 8 in 2024-04-07T16:28:46.953 INFO:tasks.rados.rados.0.smithi100.stdout:3772: done (1 left) 2024-04-07T16:28:46.954 INFO:tasks.rados.rados.0.smithi100.stdout:3773: done (0 left) 2024-04-07T16:28:46.954 INFO:tasks.rados.rados.0.smithi100.stdout:3774: read oid 48 snap -1 2024-04-07T16:28:46.954 INFO:tasks.rados.rados.0.smithi100.stdout:3774: expect (ObjNum 1140 snap 328 seq_num 1140) 2024-04-07T16:28:46.954 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 31 current snap is 355 2024-04-07T16:28:46.954 INFO:tasks.rados.rados.0.smithi100.stdout:3775: seq_num 1170 ranges {6799360=401408} 2024-04-07T16:28:47.044 INFO:tasks.rados.rados.0.smithi100.stdout:3775: writing smithi10066867-31 from 6799360 to 7200768 tid 1 2024-04-07T16:28:47.044 INFO:tasks.rados.rados.0.smithi100.stdout:3776: read oid 49 snap -1 2024-04-07T16:28:47.045 INFO:tasks.rados.rados.0.smithi100.stdout:3776: expect deleted 2024-04-07T16:28:47.045 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 10 current snap is 355 2024-04-07T16:28:47.045 INFO:tasks.rados.rados.0.smithi100.stdout:3777: seq_num 1171 ranges {0=417792} 2024-04-07T16:28:47.048 INFO:tasks.rados.rados.0.smithi100.stdout:3777: writing smithi10066867-10 from 0 to 417792 tid 1 2024-04-07T16:28:47.048 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 7 current snap is 355 2024-04-07T16:28:47.048 INFO:tasks.rados.rados.0.smithi100.stdout:3778: seq_num 1172 ranges {0=434176} 2024-04-07T16:28:47.052 INFO:tasks.rados.rados.0.smithi100.stdout:3778: writing smithi10066867-7 from 0 to 434176 tid 1 2024-04-07T16:28:47.052 INFO:tasks.rados.rados.0.smithi100.stdout:3779: read oid 23 snap -1 2024-04-07T16:28:47.052 INFO:tasks.rados.rados.0.smithi100.stdout:3779: expect deleted 2024-04-07T16:28:47.052 INFO:tasks.rados.rados.0.smithi100.stdout:3780: copy_from oid 30 from oid 17 current snap is 355 2024-04-07T16:28:47.052 INFO:tasks.rados.rados.0.smithi100.stdout:3774: expect (ObjNum 1140 snap 328 seq_num 1140) 2024-04-07T16:28:47.339 INFO:tasks.rados.rados.0.smithi100.stdout:3780: finishing copy_from to smithi10066867-30 2024-04-07T16:28:47.339 INFO:tasks.rados.rados.0.smithi100.stdout:3780: got expected ENOENT (src dne) 2024-04-07T16:28:47.339 INFO:tasks.rados.rados.0.smithi100.stdout:3780: finishing copy_from racing read to smithi10066867-30 2024-04-07T16:28:47.339 INFO:tasks.rados.rados.0.smithi100.stdout:3777: finishing write tid 1 to smithi10066867-10 2024-04-07T16:28:47.340 INFO:tasks.rados.rados.0.smithi100.stdout:3775: finishing write tid 1 to smithi10066867-31 2024-04-07T16:28:47.340 INFO:tasks.rados.rados.0.smithi100.stdout:3777: finishing write tid 2 to smithi10066867-10 2024-04-07T16:28:47.340 INFO:tasks.rados.rados.0.smithi100.stdout:3775: finishing write tid 2 to smithi10066867-31 2024-04-07T16:28:47.340 INFO:tasks.rados.rados.0.smithi100.stdout:3775: finishing write tid 3 to smithi10066867-31 2024-04-07T16:28:47.340 INFO:tasks.rados.rados.0.smithi100.stdout:3775: oid 31 updating version 0 to 1752 2024-04-07T16:28:47.340 INFO:tasks.rados.rados.0.smithi100.stdout:3775: oid 31 updating version 1752 to 1753 2024-04-07T16:28:47.340 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 31 v 1753 (ObjNum 1170 snap 355 seq_num 1170) dirty exists 2024-04-07T16:28:47.340 INFO:tasks.rados.rados.0.smithi100.stdout:3775: left oid 31 (ObjNum 1170 snap 355 seq_num 1170) 2024-04-07T16:28:47.340 INFO:tasks.rados.rados.0.smithi100.stdout:3778: finishing write tid 1 to smithi10066867-7 2024-04-07T16:28:47.340 INFO:tasks.rados.rados.0.smithi100.stdout:3778: finishing write tid 2 to smithi10066867-7 2024-04-07T16:28:47.340 INFO:tasks.rados.rados.0.smithi100.stdout:3777: finishing write tid 3 to smithi10066867-10 2024-04-07T16:28:47.340 INFO:tasks.rados.rados.0.smithi100.stdout:3777: oid 10 updating version 0 to 1847 2024-04-07T16:28:47.340 INFO:tasks.rados.rados.0.smithi100.stdout:3777: oid 10 updating version 1847 to 1848 2024-04-07T16:28:47.340 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 10 v 1848 (ObjNum 1171 snap 355 seq_num 1171) dirty exists 2024-04-07T16:28:47.340 INFO:tasks.rados.rados.0.smithi100.stdout:3777: left oid 10 (ObjNum 1171 snap 355 seq_num 1171) 2024-04-07T16:28:47.340 INFO:tasks.rados.rados.0.smithi100.stdout:3778: finishing write tid 3 to smithi10066867-7 2024-04-07T16:28:47.341 INFO:tasks.rados.rados.0.smithi100.stdout:3778: oid 7 updating version 0 to 1278 2024-04-07T16:28:47.341 INFO:tasks.rados.rados.0.smithi100.stdout:3778: oid 7 version 1278 is already newer than 1277 2024-04-07T16:28:47.341 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 7 v 1278 (ObjNum 1172 snap 355 seq_num 1172) dirty exists 2024-04-07T16:28:47.341 INFO:tasks.rados.rados.0.smithi100.stdout:3778: left oid 7 (ObjNum 1172 snap 355 seq_num 1172) 2024-04-07T16:28:47.341 INFO:tasks.rados.rados.0.smithi100.stdout:3774: done (6 left) 2024-04-07T16:28:47.341 INFO:tasks.rados.rados.0.smithi100.stdout:3775: done (5 left) 2024-04-07T16:28:47.341 INFO:tasks.rados.rados.0.smithi100.stdout:3776: done (4 left) 2024-04-07T16:28:47.341 INFO:tasks.rados.rados.0.smithi100.stdout:3777: done (3 left) 2024-04-07T16:28:47.341 INFO:tasks.rados.rados.0.smithi100.stdout:3778: done (2 left) 2024-04-07T16:28:47.341 INFO:tasks.rados.rados.0.smithi100.stdout:3779: done (1 left) 2024-04-07T16:28:47.341 INFO:tasks.rados.rados.0.smithi100.stdout:3780: done (0 left) 2024-04-07T16:28:47.341 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 12 current snap is 355 2024-04-07T16:28:47.341 INFO:tasks.rados.rados.0.smithi100.stdout:3781: seq_num 1173 ranges {2129920=450560} 2024-04-07T16:28:47.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:46 smithi144 bash[66654]: cluster 2024-04-07T16:28:45.996247+0000 mon.a (mon.0) 1348 : cluster [DBG] osdmap e768: 8 total, 8 up, 8 in 2024-04-07T16:28:47.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:46 smithi144 bash[66654]: audit 2024-04-07T16:28:46.736178+0000 mon.a (mon.0) 1349 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:28:47.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:46 smithi144 bash[66654]: cluster 2024-04-07T16:28:46.966303+0000 mon.a (mon.0) 1350 : cluster [DBG] osdmap e769: 8 total, 8 up, 8 in 2024-04-07T16:28:47.360 INFO:tasks.rados.rados.0.smithi100.stdout:3781: writing smithi10066867-12 from 2129920 to 2580480 tid 1 2024-04-07T16:28:47.360 INFO:tasks.rados.rados.0.smithi100.stdout:3782: snap_remove snap 329 2024-04-07T16:28:47.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:47 smithi100 bash[74942]: cluster 2024-04-07T16:28:45.996247+0000 mon.a (mon.0) 1348 : cluster [DBG] osdmap e768: 8 total, 8 up, 8 in 2024-04-07T16:28:47.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:47 smithi100 bash[74942]: audit 2024-04-07T16:28:46.736178+0000 mon.a (mon.0) 1349 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:28:47.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:47 smithi100 bash[74942]: cluster 2024-04-07T16:28:46.966303+0000 mon.a (mon.0) 1350 : cluster [DBG] osdmap e769: 8 total, 8 up, 8 in 2024-04-07T16:28:47.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:47 smithi100 bash[76202]: cluster 2024-04-07T16:28:45.996247+0000 mon.a (mon.0) 1348 : cluster [DBG] osdmap e768: 8 total, 8 up, 8 in 2024-04-07T16:28:47.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:47 smithi100 bash[76202]: audit 2024-04-07T16:28:46.736178+0000 mon.a (mon.0) 1349 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:28:47.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:47 smithi100 bash[76202]: cluster 2024-04-07T16:28:46.966303+0000 mon.a (mon.0) 1350 : cluster [DBG] osdmap e769: 8 total, 8 up, 8 in 2024-04-07T16:28:48.002 INFO:tasks.rados.rados.0.smithi100.stdout:3782: done (1 left) 2024-04-07T16:28:48.002 INFO:tasks.rados.rados.0.smithi100.stdout:3783: snap_create 2024-04-07T16:28:48.003 INFO:tasks.rados.rados.0.smithi100.stdout:3781: finishing write tid 1 to smithi10066867-12 2024-04-07T16:28:48.003 INFO:tasks.rados.rados.0.smithi100.stdout:3781: finishing write tid 2 to smithi10066867-12 2024-04-07T16:28:48.003 INFO:tasks.rados.rados.0.smithi100.stdout:3781: finishing write tid 3 to smithi10066867-12 2024-04-07T16:28:48.003 INFO:tasks.rados.rados.0.smithi100.stdout:3781: oid 12 updating version 0 to 1824 2024-04-07T16:28:48.003 INFO:tasks.rados.rados.0.smithi100.stdout:3781: oid 12 updating version 1824 to 1825 2024-04-07T16:28:48.003 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 12 v 1825 (ObjNum 1173 snap 355 seq_num 1173) dirty exists 2024-04-07T16:28:48.003 INFO:tasks.rados.rados.0.smithi100.stdout:3781: left oid 12 (ObjNum 1173 snap 355 seq_num 1173) 2024-04-07T16:28:48.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:48 smithi144 bash[66654]: cluster 2024-04-07T16:28:46.825539+0000 mgr.y (mgr.25175) 401 : cluster [DBG] pgmap v746: 121 pgs: 121 active+clean; 162 MiB data, 1.7 GiB used, 714 GiB / 715 GiB avail; 0 B/s rd, 0 B/s wr, 0 op/s 2024-04-07T16:28:48.417 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:48 smithi100 bash[74942]: cluster 2024-04-07T16:28:46.825539+0000 mgr.y (mgr.25175) 401 : cluster [DBG] pgmap v746: 121 pgs: 121 active+clean; 162 MiB data, 1.7 GiB used, 714 GiB / 715 GiB avail; 0 B/s rd, 0 B/s wr, 0 op/s 2024-04-07T16:28:48.417 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:48 smithi100 bash[76202]: cluster 2024-04-07T16:28:46.825539+0000 mgr.y (mgr.25175) 401 : cluster [DBG] pgmap v746: 121 pgs: 121 active+clean; 162 MiB data, 1.7 GiB used, 714 GiB / 715 GiB avail; 0 B/s rd, 0 B/s wr, 0 op/s 2024-04-07T16:28:48.688 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:28:48 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:28:48] "GET /metrics HTTP/1.1" 200 36344 "" "Prometheus/2.43.0" 2024-04-07T16:28:49.000 INFO:tasks.rados.rados.0.smithi100.stdout:3781: done (1 left) 2024-04-07T16:28:49.000 INFO:tasks.rados.rados.0.smithi100.stdout:3783: done (0 left) 2024-04-07T16:28:49.000 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 45 current snap is 356 2024-04-07T16:28:49.000 INFO:tasks.rados.rados.0.smithi100.stdout:3784: seq_num 1174 ranges {638976=466944} 2024-04-07T16:28:49.014 INFO:tasks.rados.rados.0.smithi100.stdout:3784: writing smithi10066867-45 from 638976 to 1105920 tid 1 2024-04-07T16:28:49.014 INFO:tasks.rados.rados.0.smithi100.stdout:3785: rollback oid 38 current snap is 356 2024-04-07T16:28:49.014 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 38 to 347 2024-04-07T16:28:49.014 INFO:tasks.rados.rados.0.smithi100.stdout:3786: rollback oid 1 current snap is 356 2024-04-07T16:28:49.014 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 1 to 347 2024-04-07T16:28:49.014 INFO:tasks.rados.rados.0.smithi100.stdout:3787: setattr oid 22 current snap is 356 2024-04-07T16:28:49.014 INFO:tasks.rados.rados.0.smithi100.stdout:3786: finishing rollback tid 1 to smithi10066867-1 2024-04-07T16:28:49.015 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 1 v 0 (ObjNum 81 snap 0 seq_num 2702735344) dirty dne 2024-04-07T16:28:49.016 INFO:tasks.rados.rados.0.smithi100.stdout:3786: done (3 left) 2024-04-07T16:28:49.017 INFO:tasks.rados.rados.0.smithi100.stdout:3788: snap_create 2024-04-07T16:28:49.034 INFO:tasks.rados.rados.0.smithi100.stdout:3784: finishing write tid 1 to smithi10066867-45 2024-04-07T16:28:49.035 INFO:tasks.rados.rados.0.smithi100.stdout:3784: finishing write tid 2 to smithi10066867-45 2024-04-07T16:28:49.036 INFO:tasks.rados.rados.0.smithi100.stdout:3785: finishing rollback tid 0 to smithi10066867-38 2024-04-07T16:28:49.036 INFO:tasks.rados.rados.0.smithi100.stdout:3785: finishing rollback tid 1 to smithi10066867-38 2024-04-07T16:28:49.037 INFO:tasks.rados.rados.0.smithi100.stdout:3785: finishing rollback tid 2 to smithi10066867-38 2024-04-07T16:28:49.037 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 38 v 1716 (ObjNum 1134 snap 326 seq_num 1134) dirty exists 2024-04-07T16:28:49.038 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 22 v 1734 (ObjNum 1150 snap 339 seq_num 1150) dirty exists 2024-04-07T16:28:49.040 INFO:tasks.rados.rados.0.smithi100.stdout:3784: finishing write tid 3 to smithi10066867-45 2024-04-07T16:28:49.040 INFO:tasks.rados.rados.0.smithi100.stdout:3784: oid 45 updating version 0 to 1821 2024-04-07T16:28:49.040 INFO:tasks.rados.rados.0.smithi100.stdout:3784: oid 45 version 1821 is already newer than 1820 2024-04-07T16:28:49.041 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 45 v 1821 (ObjNum 1174 snap 356 seq_num 1174) dirty exists 2024-04-07T16:28:49.041 INFO:tasks.rados.rados.0.smithi100.stdout:3784: left oid 45 (ObjNum 1174 snap 356 seq_num 1174) 2024-04-07T16:28:49.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:49 smithi144 bash[66654]: cluster 2024-04-07T16:28:48.013175+0000 mon.a (mon.0) 1351 : cluster [DBG] osdmap e770: 8 total, 8 up, 8 in 2024-04-07T16:28:49.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:49 smithi144 bash[66654]: cluster 2024-04-07T16:28:49.013954+0000 mon.a (mon.0) 1352 : cluster [DBG] osdmap e771: 8 total, 8 up, 8 in 2024-04-07T16:28:49.434 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/mon.a/config 2024-04-07T16:28:49.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:49 smithi100 bash[74942]: cluster 2024-04-07T16:28:48.013175+0000 mon.a (mon.0) 1351 : cluster [DBG] osdmap e770: 8 total, 8 up, 8 in 2024-04-07T16:28:49.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:49 smithi100 bash[74942]: cluster 2024-04-07T16:28:49.013954+0000 mon.a (mon.0) 1352 : cluster [DBG] osdmap e771: 8 total, 8 up, 8 in 2024-04-07T16:28:49.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:49 smithi100 bash[76202]: cluster 2024-04-07T16:28:48.013175+0000 mon.a (mon.0) 1351 : cluster [DBG] osdmap e770: 8 total, 8 up, 8 in 2024-04-07T16:28:49.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:49 smithi100 bash[76202]: cluster 2024-04-07T16:28:49.013954+0000 mon.a (mon.0) 1352 : cluster [DBG] osdmap e771: 8 total, 8 up, 8 in 2024-04-07T16:28:50.027 INFO:tasks.rados.rados.0.smithi100.stdout:3784: done (3 left) 2024-04-07T16:28:50.027 INFO:tasks.rados.rados.0.smithi100.stdout:3785: done (2 left) 2024-04-07T16:28:50.027 INFO:tasks.rados.rados.0.smithi100.stdout:3787: done (1 left) 2024-04-07T16:28:50.027 INFO:tasks.rados.rados.0.smithi100.stdout:3788: done (0 left) 2024-04-07T16:28:50.027 INFO:tasks.rados.rados.0.smithi100.stdout:3789: rollback oid 29 current snap is 357 2024-04-07T16:28:50.027 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 29 to 345 2024-04-07T16:28:50.027 INFO:tasks.rados.rados.0.smithi100.stdout:3790: snap_create 2024-04-07T16:28:50.044 INFO:tasks.rados.rados.0.smithi100.stdout:3789: finishing rollback tid 0 to smithi10066867-29 2024-04-07T16:28:50.045 INFO:tasks.rados.rados.0.smithi100.stdout:3789: finishing rollback tid 1 to smithi10066867-29 2024-04-07T16:28:50.045 INFO:tasks.rados.rados.0.smithi100.stdout:3789: finishing rollback tid 2 to smithi10066867-29 2024-04-07T16:28:50.045 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 29 v 1597 (ObjNum 1149 snap 339 seq_num 1149) dirty exists 2024-04-07T16:28:50.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:50 smithi144 bash[66654]: cluster 2024-04-07T16:28:48.826361+0000 mgr.y (mgr.25175) 402 : cluster [DBG] pgmap v749: 121 pgs: 121 active+clean; 150 MiB data, 1.7 GiB used, 714 GiB / 715 GiB avail; 157 KiB/s rd, 110 KiB/s wr, 3 op/s 2024-04-07T16:28:50.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:50 smithi100 bash[74942]: cluster 2024-04-07T16:28:48.826361+0000 mgr.y (mgr.25175) 402 : cluster [DBG] pgmap v749: 121 pgs: 121 active+clean; 150 MiB data, 1.7 GiB used, 714 GiB / 715 GiB avail; 157 KiB/s rd, 110 KiB/s wr, 3 op/s 2024-04-07T16:28:50.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:50 smithi100 bash[76202]: cluster 2024-04-07T16:28:48.826361+0000 mgr.y (mgr.25175) 402 : cluster [DBG] pgmap v749: 121 pgs: 121 active+clean; 150 MiB data, 1.7 GiB used, 714 GiB / 715 GiB avail; 157 KiB/s rd, 110 KiB/s wr, 3 op/s 2024-04-07T16:28:51.033 INFO:tasks.rados.rados.0.smithi100.stdout:3789: done (1 left) 2024-04-07T16:28:51.034 INFO:tasks.rados.rados.0.smithi100.stdout:3790: done (0 left) 2024-04-07T16:28:51.034 INFO:tasks.rados.rados.0.smithi100.stdout:3791: delete oid 19 current snap is 358 2024-04-07T16:28:51.044 INFO:tasks.rados.rados.0.smithi100.stdout:3791: done (0 left) 2024-04-07T16:28:51.044 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 44 current snap is 358 2024-04-07T16:28:51.044 INFO:tasks.rados.rados.0.smithi100.stdout:3792: seq_num 1176 ranges {2269184=499712} 2024-04-07T16:28:51.083 INFO:tasks.rados.rados.0.smithi100.stdout:3792: writing smithi10066867-44 from 2269184 to 2768896 tid 1 2024-04-07T16:28:51.083 INFO:tasks.rados.rados.0.smithi100.stdout:3793: snap_create 2024-04-07T16:28:51.096 INFO:tasks.rados.rados.0.smithi100.stdout:3792: finishing write tid 1 to smithi10066867-44 2024-04-07T16:28:51.097 INFO:tasks.rados.rados.0.smithi100.stdout:3792: finishing write tid 2 to smithi10066867-44 2024-04-07T16:28:51.100 INFO:tasks.rados.rados.0.smithi100.stdout:3792: finishing write tid 3 to smithi10066867-44 2024-04-07T16:28:51.100 INFO:tasks.rados.rados.0.smithi100.stdout:3792: oid 44 updating version 0 to 1682 2024-04-07T16:28:51.100 INFO:tasks.rados.rados.0.smithi100.stdout:3792: oid 44 updating version 1682 to 1683 2024-04-07T16:28:51.100 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 44 v 1683 (ObjNum 1176 snap 358 seq_num 1176) dirty exists 2024-04-07T16:28:51.100 INFO:tasks.rados.rados.0.smithi100.stdout:3792: left oid 44 (ObjNum 1176 snap 358 seq_num 1176) 2024-04-07T16:28:51.188 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:51 smithi100 bash[74942]: cluster 2024-04-07T16:28:50.039974+0000 mon.a (mon.0) 1353 : cluster [DBG] osdmap e772: 8 total, 8 up, 8 in 2024-04-07T16:28:51.188 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:51 smithi100 bash[76202]: cluster 2024-04-07T16:28:50.039974+0000 mon.a (mon.0) 1353 : cluster [DBG] osdmap e772: 8 total, 8 up, 8 in 2024-04-07T16:28:51.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:51 smithi144 bash[66654]: cluster 2024-04-07T16:28:50.039974+0000 mon.a (mon.0) 1353 : cluster [DBG] osdmap e772: 8 total, 8 up, 8 in 2024-04-07T16:28:51.955 INFO:tasks.rados.rados.0.smithi100.stdout:3792: done (1 left) 2024-04-07T16:28:51.955 INFO:tasks.rados.rados.0.smithi100.stdout:3793: done (0 left) 2024-04-07T16:28:51.955 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 23 current snap is 359 2024-04-07T16:28:51.955 INFO:tasks.rados.rados.0.smithi100.stdout:3794: seq_num 1177 ranges {0=516096} 2024-04-07T16:28:51.963 INFO:tasks.rados.rados.0.smithi100.stdout:3794: writing smithi10066867-23 from 0 to 516096 tid 1 2024-04-07T16:28:51.963 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 3 current snap is 359 2024-04-07T16:28:51.963 INFO:tasks.rados.rados.0.smithi100.stdout:3795: seq_num 1178 ranges {0=532480} 2024-04-07T16:28:51.973 INFO:tasks.rados.rados.0.smithi100.stdout:3795: writing smithi10066867-3 from 0 to 532480 tid 1 2024-04-07T16:28:51.973 INFO:tasks.rados.rados.0.smithi100.stdout:3796: read oid 34 snap -1 2024-04-07T16:28:51.973 INFO:tasks.rados.rados.0.smithi100.stdout:3796: expect (ObjNum 1012 snap 300 seq_num 1012) 2024-04-07T16:28:51.974 INFO:tasks.rados.rados.0.smithi100.stdout:3797: delete oid 29 current snap is 359 2024-04-07T16:28:51.979 INFO:tasks.rados.rados.0.smithi100.stdout:3794: finishing write tid 1 to smithi10066867-23 2024-04-07T16:28:51.979 INFO:tasks.rados.rados.0.smithi100.stdout:3794: finishing write tid 2 to smithi10066867-23 2024-04-07T16:28:51.985 INFO:tasks.rados.rados.0.smithi100.stdout:3794: finishing write tid 3 to smithi10066867-23 2024-04-07T16:28:51.985 INFO:tasks.rados.rados.0.smithi100.stdout:3794: oid 23 updating version 0 to 1947 2024-04-07T16:28:51.985 INFO:tasks.rados.rados.0.smithi100.stdout:3794: oid 23 updating version 1947 to 1948 2024-04-07T16:28:51.985 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 23 v 1948 (ObjNum 1177 snap 359 seq_num 1177) dirty exists 2024-04-07T16:28:51.985 INFO:tasks.rados.rados.0.smithi100.stdout:3794: left oid 23 (ObjNum 1177 snap 359 seq_num 1177) 2024-04-07T16:28:51.987 INFO:tasks.rados.rados.0.smithi100.stdout:3794: done (3 left) 2024-04-07T16:28:51.987 INFO:tasks.rados.rados.0.smithi100.stdout:3797: done (2 left) 2024-04-07T16:28:51.987 INFO:tasks.rados.rados.0.smithi100.stdout:3798: setattr oid 25 current snap is 359 2024-04-07T16:28:51.990 INFO:tasks.rados.rados.0.smithi100.stdout:3799: setattr oid 35 current snap is 359 2024-04-07T16:28:51.994 INFO:tasks.rados.rados.0.smithi100.stdout:3800: read oid 5 snap -1 2024-04-07T16:28:51.994 INFO:tasks.rados.rados.0.smithi100.stdout:3800: expect (ObjNum 1143 snap 335 seq_num 1143) 2024-04-07T16:28:51.994 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 30 current snap is 359 2024-04-07T16:28:51.994 INFO:tasks.rados.rados.0.smithi100.stdout:3801: seq_num 1180 ranges {3866624=565248} 2024-04-07T16:28:52.039 INFO:tasks.rados.rados.0.smithi100.stdout:3801: writing smithi10066867-30 from 3866624 to 4431872 tid 1 2024-04-07T16:28:52.039 INFO:tasks.rados.rados.0.smithi100.stdout:3802: copy_from oid 4 from oid 17 current snap is 359 2024-04-07T16:28:52.039 INFO:tasks.rados.rados.0.smithi100.stdout:3803: delete oid 27 current snap is 359 2024-04-07T16:28:52.039 INFO:tasks.rados.rados.0.smithi100.stdout:3795: finishing write tid 1 to smithi10066867-3 2024-04-07T16:28:52.039 INFO:tasks.rados.rados.0.smithi100.stdout:3795: finishing write tid 2 to smithi10066867-3 2024-04-07T16:28:52.039 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 35 v 1754 (ObjNum 65 snap 0 seq_num 1) dirty exists 2024-04-07T16:28:52.039 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 25 v 1735 (ObjNum 1156 snap 342 seq_num 1156) dirty exists 2024-04-07T16:28:52.039 INFO:tasks.rados.rados.0.smithi100.stdout:3795: finishing write tid 3 to smithi10066867-3 2024-04-07T16:28:52.039 INFO:tasks.rados.rados.0.smithi100.stdout:3795: oid 3 updating version 0 to 1717 2024-04-07T16:28:52.039 INFO:tasks.rados.rados.0.smithi100.stdout:3795: oid 3 updating version 1717 to 1718 2024-04-07T16:28:52.039 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 3 v 1718 (ObjNum 1178 snap 359 seq_num 1178) dirty exists 2024-04-07T16:28:52.040 INFO:tasks.rados.rados.0.smithi100.stdout:3795: left oid 3 (ObjNum 1178 snap 359 seq_num 1178) 2024-04-07T16:28:52.040 INFO:tasks.rados.rados.0.smithi100.stdout:3800: expect (ObjNum 1143 snap 335 seq_num 1143) 2024-04-07T16:28:52.182 INFO:tasks.rados.rados.0.smithi100.stdout:3796: expect (ObjNum 1012 snap 300 seq_num 1012) 2024-04-07T16:28:52.304 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:52 smithi144 bash[66654]: cluster 2024-04-07T16:28:50.826790+0000 mgr.y (mgr.25175) 403 : cluster [DBG] pgmap v752: 121 pgs: 121 active+clean; 145 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 5.3 MiB/s rd, 314 KiB/s wr, 6 op/s 2024-04-07T16:28:52.304 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:52 smithi144 bash[66654]: cluster 2024-04-07T16:28:51.045568+0000 mon.a (mon.0) 1354 : cluster [DBG] osdmap e773: 8 total, 8 up, 8 in 2024-04-07T16:28:52.305 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:52 smithi144 bash[66654]: cluster 2024-04-07T16:28:51.968648+0000 mon.a (mon.0) 1355 : cluster [DBG] osdmap e774: 8 total, 8 up, 8 in 2024-04-07T16:28:52.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:52 smithi100 bash[74942]: cluster 2024-04-07T16:28:50.826790+0000 mgr.y (mgr.25175) 403 : cluster [DBG] pgmap v752: 121 pgs: 121 active+clean; 145 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 5.3 MiB/s rd, 314 KiB/s wr, 6 op/s 2024-04-07T16:28:52.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:52 smithi100 bash[74942]: cluster 2024-04-07T16:28:51.045568+0000 mon.a (mon.0) 1354 : cluster [DBG] osdmap e773: 8 total, 8 up, 8 in 2024-04-07T16:28:52.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:52 smithi100 bash[74942]: cluster 2024-04-07T16:28:51.968648+0000 mon.a (mon.0) 1355 : cluster [DBG] osdmap e774: 8 total, 8 up, 8 in 2024-04-07T16:28:52.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:52 smithi100 bash[76202]: cluster 2024-04-07T16:28:50.826790+0000 mgr.y (mgr.25175) 403 : cluster [DBG] pgmap v752: 121 pgs: 121 active+clean; 145 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 5.3 MiB/s rd, 314 KiB/s wr, 6 op/s 2024-04-07T16:28:52.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:52 smithi100 bash[76202]: cluster 2024-04-07T16:28:51.045568+0000 mon.a (mon.0) 1354 : cluster [DBG] osdmap e773: 8 total, 8 up, 8 in 2024-04-07T16:28:52.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:52 smithi100 bash[76202]: cluster 2024-04-07T16:28:51.968648+0000 mon.a (mon.0) 1355 : cluster [DBG] osdmap e774: 8 total, 8 up, 8 in 2024-04-07T16:28:52.556 INFO:tasks.rados.rados.0.smithi100.stdout:3802: finishing copy_from to smithi10066867-4 2024-04-07T16:28:52.556 INFO:tasks.rados.rados.0.smithi100.stdout:3802: got expected ENOENT (src dne) 2024-04-07T16:28:52.557 INFO:tasks.rados.rados.0.smithi100.stdout:3802: finishing copy_from racing read to smithi10066867-4 2024-04-07T16:28:52.557 INFO:tasks.rados.rados.0.smithi100.stdout:3801: finishing write tid 1 to smithi10066867-30 2024-04-07T16:28:52.557 INFO:tasks.rados.rados.0.smithi100.stdout:3801: finishing write tid 2 to smithi10066867-30 2024-04-07T16:28:52.557 INFO:tasks.rados.rados.0.smithi100.stdout:3801: finishing write tid 3 to smithi10066867-30 2024-04-07T16:28:52.557 INFO:tasks.rados.rados.0.smithi100.stdout:3801: oid 30 updating version 0 to 1599 2024-04-07T16:28:52.557 INFO:tasks.rados.rados.0.smithi100.stdout:3801: oid 30 updating version 1599 to 1600 2024-04-07T16:28:52.557 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 30 v 1600 (ObjNum 1180 snap 359 seq_num 1180) dirty exists 2024-04-07T16:28:52.557 INFO:tasks.rados.rados.0.smithi100.stdout:3801: left oid 30 (ObjNum 1180 snap 359 seq_num 1180) 2024-04-07T16:28:52.557 INFO:tasks.rados.rados.0.smithi100.stdout:3795: done (7 left) 2024-04-07T16:28:52.557 INFO:tasks.rados.rados.0.smithi100.stdout:3796: done (6 left) 2024-04-07T16:28:52.557 INFO:tasks.rados.rados.0.smithi100.stdout:3798: done (5 left) 2024-04-07T16:28:52.557 INFO:tasks.rados.rados.0.smithi100.stdout:3799: done (4 left) 2024-04-07T16:28:52.557 INFO:tasks.rados.rados.0.smithi100.stdout:3800: done (3 left) 2024-04-07T16:28:52.557 INFO:tasks.rados.rados.0.smithi100.stdout:3801: done (2 left) 2024-04-07T16:28:52.557 INFO:tasks.rados.rados.0.smithi100.stdout:3802: done (1 left) 2024-04-07T16:28:52.558 INFO:tasks.rados.rados.0.smithi100.stdout:3803: done (0 left) 2024-04-07T16:28:52.558 INFO:tasks.rados.rados.0.smithi100.stdout:3804: read oid 26 snap -1 2024-04-07T16:28:52.558 INFO:tasks.rados.rados.0.smithi100.stdout:3804: expect deleted 2024-04-07T16:28:52.558 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 28 current snap is 359 2024-04-07T16:28:52.558 INFO:tasks.rados.rados.0.smithi100.stdout:3805: seq_num 1182 ranges {0=606208} 2024-04-07T16:28:52.562 INFO:tasks.rados.rados.0.smithi100.stdout:3805: writing smithi10066867-28 from 0 to 606208 tid 1 2024-04-07T16:28:52.562 INFO:tasks.rados.rados.0.smithi100.stdout:3806: rmattr oid 25 current snap is 359 2024-04-07T16:28:52.562 INFO:tasks.rados.rados.0.smithi100.stdout:3804: done (2 left) 2024-04-07T16:28:52.563 INFO:tasks.rados.rados.0.smithi100.stdout:3807: delete oid 47 current snap is 359 2024-04-07T16:28:52.566 INFO:teuthology.orchestra.run.smithi100.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-07T16:28:52.566 INFO:teuthology.orchestra.run.smithi100.stdout:alertmanager.a smithi100 *:9093,9094 running (72s) 66s ago 15m 13.9M - 0.25.0 c8568f914cd2 67b26d69aca6 2024-04-07T16:28:52.566 INFO:teuthology.orchestra.run.smithi100.stdout:grafana.a smithi144 *:3000 running (36s) 30s ago 15m 59.3M - 9.4.12 2bacad6d85d8 1421a5110839 2024-04-07T16:28:52.567 INFO:teuthology.orchestra.run.smithi100.stdout:mds.a smithi100 running (2m) 66s ago 16m 13.8M - 19.0.0-2785-gadce92d0 c0ecd4226ac9 075552e67bd0 2024-04-07T16:28:52.567 INFO:teuthology.orchestra.run.smithi100.stdout:mds.b smithi144 running (2m) 30s ago 16m 12.0M - 19.0.0-2785-gadce92d0 c0ecd4226ac9 fc737f3b1edc 2024-04-07T16:28:52.567 INFO:teuthology.orchestra.run.smithi100.stdout:mgr.x smithi144 *:8443,9283,8765 running (9m) 30s ago 23m 441M - 19.0.0-2785-gadce92d0 c0ecd4226ac9 233745c292a6 2024-04-07T16:28:52.567 INFO:teuthology.orchestra.run.smithi100.stdout:mgr.y smithi100 *:8443,9283,8765 running (9m) 66s ago 26m 509M - 19.0.0-2785-gadce92d0 c0ecd4226ac9 254dc65cdaa7 2024-04-07T16:28:52.567 INFO:teuthology.orchestra.run.smithi100.stdout:mon.a smithi100 running (8m) 66s ago 26m 70.9M 2048M 19.0.0-2785-gadce92d0 c0ecd4226ac9 718e12655c04 2024-04-07T16:28:52.567 INFO:teuthology.orchestra.run.smithi100.stdout:mon.b smithi144 running (7m) 30s ago 24m 55.6M 2048M 19.0.0-2785-gadce92d0 c0ecd4226ac9 aef817664bff 2024-04-07T16:28:52.567 INFO:teuthology.orchestra.run.smithi100.stdout:mon.c smithi100 running (8m) 66s ago 24m 58.9M 2048M 19.0.0-2785-gadce92d0 c0ecd4226ac9 63e889040f6e 2024-04-07T16:28:52.567 INFO:teuthology.orchestra.run.smithi100.stdout:node-exporter.a smithi100 *:9100 running (2m) 66s ago 15m 8043k - 1.5.0 0da6a335fe13 9609d3a1c1ba 2024-04-07T16:28:52.567 INFO:teuthology.orchestra.run.smithi100.stdout:node-exporter.b smithi144 *:9100 running (2m) 30s ago 15m 7908k - 1.5.0 0da6a335fe13 abbe3a5b7ce6 2024-04-07T16:28:52.567 INFO:teuthology.orchestra.run.smithi100.stdout:osd.0 smithi100 running (6m) 66s ago 23m 380M 2524M 19.0.0-2785-gadce92d0 c0ecd4226ac9 a6e83c9d1abc 2024-04-07T16:28:52.567 INFO:teuthology.orchestra.run.smithi100.stdout:osd.1 smithi100 running (6m) 66s ago 22m 342M 2524M 19.0.0-2785-gadce92d0 c0ecd4226ac9 c50fe40079b6 2024-04-07T16:28:52.567 INFO:teuthology.orchestra.run.smithi100.stdout:osd.2 smithi100 running (5m) 66s ago 21m 388M 2524M 19.0.0-2785-gadce92d0 c0ecd4226ac9 8e631c43064b 2024-04-07T16:28:52.567 INFO:teuthology.orchestra.run.smithi100.stdout:osd.3 smithi100 running (5m) 66s ago 20m 169M 2524M 19.0.0-2785-gadce92d0 c0ecd4226ac9 402931247c79 2024-04-07T16:28:52.567 INFO:teuthology.orchestra.run.smithi100.stdout:osd.4 smithi144 running (4m) 30s ago 19m 373M 2524M 19.0.0-2785-gadce92d0 c0ecd4226ac9 ebbb7be95d33 2024-04-07T16:28:52.567 INFO:teuthology.orchestra.run.smithi100.stdout:osd.5 smithi144 running (4m) 30s ago 19m 240M 2524M 19.0.0-2785-gadce92d0 c0ecd4226ac9 ecce27bcb2f8 2024-04-07T16:28:52.567 INFO:teuthology.orchestra.run.smithi100.stdout:osd.6 smithi144 running (3m) 30s ago 18m 116M 2524M 19.0.0-2785-gadce92d0 c0ecd4226ac9 3788c5c3f427 2024-04-07T16:28:52.568 INFO:teuthology.orchestra.run.smithi100.stdout:osd.7 smithi144 running (3m) 30s ago 17m 195M 2524M 19.0.0-2785-gadce92d0 c0ecd4226ac9 9365aba97dc0 2024-04-07T16:28:52.568 INFO:teuthology.orchestra.run.smithi100.stdout:prometheus.a smithi144 *:9095 running (96s) 30s ago 16m 42.4M - 2.43.0 a07b618ecd1d e7d72b7c75c4 2024-04-07T16:28:52.568 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 25 v 1736 (ObjNum 1156 snap 342 seq_num 1156) dirty exists 2024-04-07T16:28:52.571 INFO:tasks.rados.rados.0.smithi100.stdout:3806: done (2 left) 2024-04-07T16:28:52.571 INFO:tasks.rados.rados.0.smithi100.stdout:3807: done (1 left) 2024-04-07T16:28:52.571 INFO:tasks.rados.rados.0.smithi100.stdout:3808: rmattr oid 3 current snap is 359 2024-04-07T16:28:52.571 INFO:tasks.rados.rados.0.smithi100.stdout:3809: copy_from oid 11 from oid 12 current snap is 359 2024-04-07T16:28:52.571 INFO:tasks.rados.rados.0.smithi100.stdout:3810: rollback oid 16 current snap is 359 2024-04-07T16:28:52.571 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 16 to 347 2024-04-07T16:28:52.571 INFO:tasks.rados.rados.0.smithi100.stdout:3811: delete oid 22 current snap is 359 2024-04-07T16:28:52.572 INFO:tasks.rados.rados.0.smithi100.stdout:3805: finishing write tid 1 to smithi10066867-28 2024-04-07T16:28:52.572 INFO:tasks.rados.rados.0.smithi100.stdout:3805: finishing write tid 2 to smithi10066867-28 2024-04-07T16:28:52.575 INFO:tasks.rados.rados.0.smithi100.stdout:3805: finishing write tid 3 to smithi10066867-28 2024-04-07T16:28:52.575 INFO:tasks.rados.rados.0.smithi100.stdout:3805: oid 28 updating version 0 to 1887 2024-04-07T16:28:52.575 INFO:tasks.rados.rados.0.smithi100.stdout:3805: oid 28 version 1887 is already newer than 1886 2024-04-07T16:28:52.575 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 28 v 1887 (ObjNum 1182 snap 359 seq_num 1182) dirty exists 2024-04-07T16:28:52.575 INFO:tasks.rados.rados.0.smithi100.stdout:3805: left oid 28 (ObjNum 1182 snap 359 seq_num 1182) 2024-04-07T16:28:52.577 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 3 v 1719 (ObjNum 1178 snap 359 seq_num 1178) dirty exists 2024-04-07T16:28:52.578 INFO:tasks.rados.rados.0.smithi100.stdout:3805: done (4 left) 2024-04-07T16:28:52.578 INFO:tasks.rados.rados.0.smithi100.stdout:3808: done (3 left) 2024-04-07T16:28:52.578 INFO:tasks.rados.rados.0.smithi100.stdout:3811: done (2 left) 2024-04-07T16:28:52.578 INFO:tasks.rados.rados.0.smithi100.stdout:3812: read oid 40 snap 345 2024-04-07T16:28:52.579 INFO:tasks.rados.rados.0.smithi100.stdout:3812: expect (ObjNum 1165 snap 345 seq_num 1165) 2024-04-07T16:28:52.579 INFO:tasks.rados.rados.0.smithi100.stdout:3813: setattr oid 43 current snap is 359 2024-04-07T16:28:52.580 INFO:tasks.rados.rados.0.smithi100.stdout:3814: snap_create 2024-04-07T16:28:52.583 INFO:tasks.rados.rados.0.smithi100.stdout:3810: finishing rollback tid 0 to smithi10066867-16 2024-04-07T16:28:52.583 INFO:tasks.rados.rados.0.smithi100.stdout:3810: finishing rollback tid 1 to smithi10066867-16 2024-04-07T16:28:52.584 INFO:tasks.rados.rados.0.smithi100.stdout:3810: finishing rollback tid 2 to smithi10066867-16 2024-04-07T16:28:52.584 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 16 v 1828 (ObjNum 1041 snap 307 seq_num 1041) dirty exists 2024-04-07T16:28:52.587 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 43 v 1280 (ObjNum 1030 snap 306 seq_num 1030) dirty exists 2024-04-07T16:28:52.600 INFO:tasks.rados.rados.0.smithi100.stdout:3812: expect (ObjNum 1165 snap 345 seq_num 1165) 2024-04-07T16:28:52.606 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:28:52 smithi144 bash[95233]: ts=2024-04-07T16:28:52.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:28:52.802 INFO:tasks.rados.rados.0.smithi100.stdout:3809: finishing copy_from racing read to smithi10066867-11 2024-04-07T16:28:52.802 INFO:tasks.rados.rados.0.smithi100.stdout:3809: finishing copy_from to smithi10066867-11 2024-04-07T16:28:52.802 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 11 v 1825 (ObjNum 1173 snap 355 seq_num 1173) dirty exists 2024-04-07T16:28:53.066 INFO:tasks.rados.rados.0.smithi100.stdout:3809: done (4 left) 2024-04-07T16:28:53.066 INFO:tasks.rados.rados.0.smithi100.stdout:3810: done (3 left) 2024-04-07T16:28:53.066 INFO:tasks.rados.rados.0.smithi100.stdout:3812: done (2 left) 2024-04-07T16:28:53.066 INFO:tasks.rados.rados.0.smithi100.stdout:3813: done (1 left) 2024-04-07T16:28:53.066 INFO:tasks.rados.rados.0.smithi100.stdout:3814: done (0 left) 2024-04-07T16:28:53.066 INFO:tasks.rados.rados.0.smithi100.stdout:3815: rmattr oid 41 current snap is 360 2024-04-07T16:28:53.066 INFO:tasks.rados.rados.0.smithi100.stdout:3816: read oid 24 snap -1 2024-04-07T16:28:53.066 INFO:tasks.rados.rados.0.smithi100.stdout:3816: expect (ObjNum 1030 snap 306 seq_num 1030) 2024-04-07T16:28:53.067 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 15 current snap is 360 2024-04-07T16:28:53.067 INFO:tasks.rados.rados.0.smithi100.stdout:3817: seq_num 1185 ranges {0=655360} 2024-04-07T16:28:53.080 INFO:tasks.rados.rados.0.smithi100.stdout:3817: writing smithi10066867-15 from 0 to 655360 tid 1 2024-04-07T16:28:53.080 INFO:tasks.rados.rados.0.smithi100.stdout:3818: rollback oid 14 current snap is 360 2024-04-07T16:28:53.080 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 14 to 349 2024-04-07T16:28:53.080 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 41 v 1829 (ObjNum 33 snap 0 seq_num 2670638992) dirty exists 2024-04-07T16:28:53.080 INFO:tasks.rados.rados.0.smithi100.stdout:3815: done (3 left) 2024-04-07T16:28:53.080 INFO:tasks.rados.rados.0.smithi100.stdout:3819: rollback oid 25 current snap is 360 2024-04-07T16:28:53.080 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 25 to 339 2024-04-07T16:28:53.080 INFO:tasks.rados.rados.0.smithi100.stdout:3820: snap_create 2024-04-07T16:28:53.083 INFO:tasks.rados.rados.0.smithi100.stdout:3816: expect (ObjNum 1030 snap 306 seq_num 1030) 2024-04-07T16:28:53.103 INFO:tasks.rados.rados.0.smithi100.stdout:3819: finishing rollback tid 0 to smithi10066867-25 2024-04-07T16:28:53.103 INFO:tasks.rados.rados.0.smithi100.stdout:3819: finishing rollback tid 1 to smithi10066867-25 2024-04-07T16:28:53.103 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 25 v 1739 (ObjNum 33 snap 0 seq_num 4240230614) dirty dne 2024-04-07T16:28:53.103 INFO:tasks.rados.rados.0.smithi100.stdout:3817: finishing write tid 1 to smithi10066867-15 2024-04-07T16:28:53.103 INFO:tasks.rados.rados.0.smithi100.stdout:3818: finishing rollback tid 0 to smithi10066867-14 2024-04-07T16:28:53.103 INFO:tasks.rados.rados.0.smithi100.stdout:3817: finishing write tid 2 to smithi10066867-15 2024-04-07T16:28:53.103 INFO:tasks.rados.rados.0.smithi100.stdout:3818: finishing rollback tid 1 to smithi10066867-14 2024-04-07T16:28:53.103 INFO:tasks.rados.rados.0.smithi100.stdout:3818: finishing rollback tid 2 to smithi10066867-14 2024-04-07T16:28:53.103 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 14 v 1824 (ObjNum 1089 snap 318 seq_num 1089) dirty exists 2024-04-07T16:28:53.103 INFO:tasks.rados.rados.0.smithi100.stdout:3817: finishing write tid 3 to smithi10066867-15 2024-04-07T16:28:53.103 INFO:tasks.rados.rados.0.smithi100.stdout:3817: oid 15 updating version 0 to 1952 2024-04-07T16:28:53.104 INFO:tasks.rados.rados.0.smithi100.stdout:3817: oid 15 version 1952 is already newer than 1951 2024-04-07T16:28:53.104 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 15 v 1952 (ObjNum 1185 snap 360 seq_num 1185) dirty exists 2024-04-07T16:28:53.104 INFO:tasks.rados.rados.0.smithi100.stdout:3817: left oid 15 (ObjNum 1185 snap 360 seq_num 1185) 2024-04-07T16:28:53.291 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0702e3a8-f4f8-11ee-b649-cb9ed24678a4 -e sha1=adce92d056533f22dc1ee37e3141d098b046ec1b -- bash -c 'ceph versions' 2024-04-07T16:28:54.078 INFO:tasks.rados.rados.0.smithi100.stdout:3816: done (4 left) 2024-04-07T16:28:54.078 INFO:tasks.rados.rados.0.smithi100.stdout:3817: done (3 left) 2024-04-07T16:28:54.078 INFO:tasks.rados.rados.0.smithi100.stdout:3818: done (2 left) 2024-04-07T16:28:54.078 INFO:tasks.rados.rados.0.smithi100.stdout:3819: done (1 left) 2024-04-07T16:28:54.078 INFO:tasks.rados.rados.0.smithi100.stdout:3820: done (0 left) 2024-04-07T16:28:54.078 INFO:tasks.rados.rados.0.smithi100.stdout:3821: read oid 5 snap -1 2024-04-07T16:28:54.078 INFO:tasks.rados.rados.0.smithi100.stdout:3821: expect (ObjNum 1143 snap 335 seq_num 1143) 2024-04-07T16:28:54.078 INFO:tasks.rados.rados.0.smithi100.stdout:3822: snap_remove snap 354 2024-04-07T16:28:54.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:54 smithi144 bash[66654]: audit 2024-04-07T16:28:52.557949+0000 mgr.y (mgr.25175) 404 : audit [DBG] from='client.34506 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:28:54.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:54 smithi144 bash[66654]: cluster 2024-04-07T16:28:52.827476+0000 mgr.y (mgr.25175) 405 : cluster [DBG] pgmap v755: 121 pgs: 121 active+clean; 146 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 5.1 MiB/s rd, 420 KiB/s wr, 5 op/s 2024-04-07T16:28:54.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:54 smithi144 bash[66654]: cluster 2024-04-07T16:28:53.078843+0000 mon.a (mon.0) 1356 : cluster [DBG] osdmap e775: 8 total, 8 up, 8 in 2024-04-07T16:28:54.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:54 smithi100 bash[74942]: audit 2024-04-07T16:28:52.557949+0000 mgr.y (mgr.25175) 404 : audit [DBG] from='client.34506 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:28:54.439 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:54 smithi100 bash[74942]: cluster 2024-04-07T16:28:52.827476+0000 mgr.y (mgr.25175) 405 : cluster [DBG] pgmap v755: 121 pgs: 121 active+clean; 146 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 5.1 MiB/s rd, 420 KiB/s wr, 5 op/s 2024-04-07T16:28:54.439 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:54 smithi100 bash[74942]: cluster 2024-04-07T16:28:53.078843+0000 mon.a (mon.0) 1356 : cluster [DBG] osdmap e775: 8 total, 8 up, 8 in 2024-04-07T16:28:54.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:54 smithi100 bash[76202]: audit 2024-04-07T16:28:52.557949+0000 mgr.y (mgr.25175) 404 : audit [DBG] from='client.34506 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T16:28:54.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:54 smithi100 bash[76202]: cluster 2024-04-07T16:28:52.827476+0000 mgr.y (mgr.25175) 405 : cluster [DBG] pgmap v755: 121 pgs: 121 active+clean; 146 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 5.1 MiB/s rd, 420 KiB/s wr, 5 op/s 2024-04-07T16:28:54.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:54 smithi100 bash[76202]: cluster 2024-04-07T16:28:53.078843+0000 mon.a (mon.0) 1356 : cluster [DBG] osdmap e775: 8 total, 8 up, 8 in 2024-04-07T16:28:55.088 INFO:tasks.rados.rados.0.smithi100.stdout:3822: done (1 left) 2024-04-07T16:28:55.088 INFO:tasks.rados.rados.0.smithi100.stdout:3823: rollback oid 17 current snap is 361 2024-04-07T16:28:55.088 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 17 to 342 2024-04-07T16:28:55.089 INFO:tasks.rados.rados.0.smithi100.stdout:3824: copy_from oid 6 from oid 39 current snap is 361 2024-04-07T16:28:55.089 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 14 current snap is 361 2024-04-07T16:28:55.089 INFO:tasks.rados.rados.0.smithi100.stdout:3825: seq_num 1186 ranges {2973696=671744} 2024-04-07T16:28:55.110 INFO:tasks.rados.rados.0.smithi100.stdout:3825: writing smithi10066867-14 from 2973696 to 3645440 tid 1 2024-04-07T16:28:55.110 INFO:tasks.rados.rados.0.smithi100.stdout:3826: read oid 38 snap 333 2024-04-07T16:28:55.110 INFO:tasks.rados.rados.0.smithi100.stdout:3826: expect (ObjNum 1134 snap 326 seq_num 1134) 2024-04-07T16:28:55.110 INFO:tasks.rados.rados.0.smithi100.stdout:3827: snap_create 2024-04-07T16:28:55.110 INFO:tasks.rados.rados.0.smithi100.stdout:3821: expect (ObjNum 1143 snap 335 seq_num 1143) 2024-04-07T16:28:55.214 INFO:tasks.rados.rados.0.smithi100.stdout:3824: finishing copy_from to smithi10066867-6 2024-04-07T16:28:55.215 INFO:tasks.rados.rados.0.smithi100.stdout:3824: got expected ENOENT (src dne) 2024-04-07T16:28:55.215 INFO:tasks.rados.rados.0.smithi100.stdout:3824: finishing copy_from racing read to smithi10066867-6 2024-04-07T16:28:55.215 INFO:tasks.rados.rados.0.smithi100.stdout:3823: finishing rollback tid 1 to smithi10066867-17 2024-04-07T16:28:55.215 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 17 v 0 (ObjNum 65 snap 0 seq_num 2670633904) dirty dne 2024-04-07T16:28:55.215 INFO:tasks.rados.rados.0.smithi100.stdout:3825: finishing write tid 1 to smithi10066867-14 2024-04-07T16:28:55.215 INFO:tasks.rados.rados.0.smithi100.stdout:3825: finishing write tid 2 to smithi10066867-14 2024-04-07T16:28:55.215 INFO:tasks.rados.rados.0.smithi100.stdout:3825: finishing write tid 3 to smithi10066867-14 2024-04-07T16:28:55.215 INFO:tasks.rados.rados.0.smithi100.stdout:3825: oid 14 updating version 0 to 1825 2024-04-07T16:28:55.215 INFO:tasks.rados.rados.0.smithi100.stdout:3825: oid 14 updating version 1825 to 1826 2024-04-07T16:28:55.215 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 14 v 1826 (ObjNum 1186 snap 361 seq_num 1186) dirty exists 2024-04-07T16:28:55.215 INFO:tasks.rados.rados.0.smithi100.stdout:3825: left oid 14 (ObjNum 1186 snap 361 seq_num 1186) 2024-04-07T16:28:55.215 INFO:tasks.rados.rados.0.smithi100.stdout:3826: expect (ObjNum 1134 snap 326 seq_num 1134) 2024-04-07T16:28:55.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:55 smithi144 bash[66654]: cluster 2024-04-07T16:28:54.090476+0000 mon.a (mon.0) 1357 : cluster [DBG] osdmap e776: 8 total, 8 up, 8 in 2024-04-07T16:28:55.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:55 smithi100 bash[74942]: cluster 2024-04-07T16:28:54.090476+0000 mon.a (mon.0) 1357 : cluster [DBG] osdmap e776: 8 total, 8 up, 8 in 2024-04-07T16:28:55.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:55 smithi100 bash[76202]: cluster 2024-04-07T16:28:54.090476+0000 mon.a (mon.0) 1357 : cluster [DBG] osdmap e776: 8 total, 8 up, 8 in 2024-04-07T16:28:56.090 INFO:tasks.rados.rados.0.smithi100.stdout:3821: done (5 left) 2024-04-07T16:28:56.090 INFO:tasks.rados.rados.0.smithi100.stdout:3823: done (4 left) 2024-04-07T16:28:56.090 INFO:tasks.rados.rados.0.smithi100.stdout:3824: done (3 left) 2024-04-07T16:28:56.090 INFO:tasks.rados.rados.0.smithi100.stdout:3825: done (2 left) 2024-04-07T16:28:56.090 INFO:tasks.rados.rados.0.smithi100.stdout:3826: done (1 left) 2024-04-07T16:28:56.090 INFO:tasks.rados.rados.0.smithi100.stdout:3827: done (0 left) 2024-04-07T16:28:56.090 INFO:tasks.rados.rados.0.smithi100.stdout:3828: rmattr oid 27 current snap is 362 2024-04-07T16:28:56.090 INFO:tasks.rados.rados.0.smithi100.stdout:3828: done (0 left) 2024-04-07T16:28:56.091 INFO:tasks.rados.rados.0.smithi100.stdout:3829: copy_from oid 33 from oid 29 current snap is 362 2024-04-07T16:28:56.092 INFO:tasks.rados.rados.0.smithi100.stdout:3830: setattr oid 37 current snap is 362 2024-04-07T16:28:56.092 INFO:tasks.rados.rados.0.smithi100.stdout:3831: read oid 23 snap -1 2024-04-07T16:28:56.093 INFO:tasks.rados.rados.0.smithi100.stdout:3831: expect (ObjNum 1177 snap 359 seq_num 1177) 2024-04-07T16:28:56.093 INFO:tasks.rados.rados.0.smithi100.stdout:3832: rollback oid 27 current snap is 362 2024-04-07T16:28:56.093 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 27 to 351 2024-04-07T16:28:56.093 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 26 current snap is 362 2024-04-07T16:28:56.093 INFO:tasks.rados.rados.0.smithi100.stdout:3833: seq_num 1187 ranges {0=688128} 2024-04-07T16:28:56.103 INFO:tasks.rados.rados.0.smithi100.stdout:3833: writing smithi10066867-26 from 0 to 688128 tid 1 2024-04-07T16:28:56.103 INFO:tasks.rados.rados.0.smithi100.stdout:3834: rollback oid 45 current snap is 362 2024-04-07T16:28:56.103 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 45 to 347 2024-04-07T16:28:56.103 INFO:tasks.rados.rados.0.smithi100.stdout:3835: setattr oid 49 current snap is 362 2024-04-07T16:28:56.105 INFO:tasks.rados.rados.0.smithi100.stdout:3832: finishing rollback tid 1 to smithi10066867-27 2024-04-07T16:28:56.108 INFO:tasks.rados.rados.0.smithi100.stdout:3836: snap_create 2024-04-07T16:28:56.120 INFO:tasks.rados.rados.0.smithi100.stdout:3831: expect (ObjNum 1177 snap 359 seq_num 1177) 2024-04-07T16:28:56.141 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 37 v 1601 (ObjNum 1133 snap 326 seq_num 1133) dirty exists 2024-04-07T16:28:56.141 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 49 v 1830 (ObjNum 113 snap 0 seq_num 2702389728) dirty exists 2024-04-07T16:28:56.141 INFO:tasks.rados.rados.0.smithi100.stdout:3832: finishing rollback tid 2 to smithi10066867-27 2024-04-07T16:28:56.141 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 27 v 1282 (ObjNum 1108 snap 321 seq_num 1108) dirty exists 2024-04-07T16:28:56.141 INFO:tasks.rados.rados.0.smithi100.stdout:3829: finishing copy_from to smithi10066867-33 2024-04-07T16:28:56.141 INFO:tasks.rados.rados.0.smithi100.stdout:3829: got expected ENOENT (src dne) 2024-04-07T16:28:56.141 INFO:tasks.rados.rados.0.smithi100.stdout:3829: finishing copy_from racing read to smithi10066867-33 2024-04-07T16:28:56.142 INFO:tasks.rados.rados.0.smithi100.stdout:3833: finishing write tid 1 to smithi10066867-26 2024-04-07T16:28:56.142 INFO:tasks.rados.rados.0.smithi100.stdout:3833: finishing write tid 2 to smithi10066867-26 2024-04-07T16:28:56.142 INFO:tasks.rados.rados.0.smithi100.stdout:3833: finishing write tid 3 to smithi10066867-26 2024-04-07T16:28:56.142 INFO:tasks.rados.rados.0.smithi100.stdout:3833: oid 26 updating version 0 to 1741 2024-04-07T16:28:56.142 INFO:tasks.rados.rados.0.smithi100.stdout:3833: oid 26 version 1741 is already newer than 1740 2024-04-07T16:28:56.142 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 26 v 1741 (ObjNum 1187 snap 362 seq_num 1187) dirty exists 2024-04-07T16:28:56.142 INFO:tasks.rados.rados.0.smithi100.stdout:3833: left oid 26 (ObjNum 1187 snap 362 seq_num 1187) 2024-04-07T16:28:56.142 INFO:tasks.rados.rados.0.smithi100.stdout:3834: finishing rollback tid 0 to smithi10066867-45 2024-04-07T16:28:56.142 INFO:tasks.rados.rados.0.smithi100.stdout:3834: finishing rollback tid 1 to smithi10066867-45 2024-04-07T16:28:56.142 INFO:tasks.rados.rados.0.smithi100.stdout:3834: finishing rollback tid 2 to smithi10066867-45 2024-04-07T16:28:56.142 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 45 v 1829 (ObjNum 1041 snap 307 seq_num 1041) dirty exists 2024-04-07T16:28:56.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:56 smithi144 bash[66654]: cluster 2024-04-07T16:28:54.828286+0000 mgr.y (mgr.25175) 406 : cluster [DBG] pgmap v758: 121 pgs: 121 active+clean; 151 MiB data, 1.7 GiB used, 714 GiB / 715 GiB avail; 10 MiB/s rd, 1.4 MiB/s wr, 28 op/s 2024-04-07T16:28:56.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:56 smithi144 bash[66654]: cluster 2024-04-07T16:28:55.099661+0000 mon.a (mon.0) 1358 : cluster [DBG] osdmap e777: 8 total, 8 up, 8 in 2024-04-07T16:28:56.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:56 smithi100 bash[74942]: cluster 2024-04-07T16:28:54.828286+0000 mgr.y (mgr.25175) 406 : cluster [DBG] pgmap v758: 121 pgs: 121 active+clean; 151 MiB data, 1.7 GiB used, 714 GiB / 715 GiB avail; 10 MiB/s rd, 1.4 MiB/s wr, 28 op/s 2024-04-07T16:28:56.439 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:56 smithi100 bash[74942]: cluster 2024-04-07T16:28:55.099661+0000 mon.a (mon.0) 1358 : cluster [DBG] osdmap e777: 8 total, 8 up, 8 in 2024-04-07T16:28:56.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:56 smithi100 bash[76202]: cluster 2024-04-07T16:28:54.828286+0000 mgr.y (mgr.25175) 406 : cluster [DBG] pgmap v758: 121 pgs: 121 active+clean; 151 MiB data, 1.7 GiB used, 714 GiB / 715 GiB avail; 10 MiB/s rd, 1.4 MiB/s wr, 28 op/s 2024-04-07T16:28:56.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:56 smithi100 bash[76202]: cluster 2024-04-07T16:28:55.099661+0000 mon.a (mon.0) 1358 : cluster [DBG] osdmap e777: 8 total, 8 up, 8 in 2024-04-07T16:28:56.956 INFO:tasks.rados.rados.0.smithi100.stdout:3829: done (7 left) 2024-04-07T16:28:56.956 INFO:tasks.rados.rados.0.smithi100.stdout:3830: done (6 left) 2024-04-07T16:28:56.956 INFO:tasks.rados.rados.0.smithi100.stdout:3831: done (5 left) 2024-04-07T16:28:56.957 INFO:tasks.rados.rados.0.smithi100.stdout:3832: done (4 left) 2024-04-07T16:28:56.957 INFO:tasks.rados.rados.0.smithi100.stdout:3833: done (3 left) 2024-04-07T16:28:56.957 INFO:tasks.rados.rados.0.smithi100.stdout:3834: done (2 left) 2024-04-07T16:28:56.957 INFO:tasks.rados.rados.0.smithi100.stdout:3835: done (1 left) 2024-04-07T16:28:56.957 INFO:tasks.rados.rados.0.smithi100.stdout:3836: done (0 left) 2024-04-07T16:28:56.957 INFO:tasks.rados.rados.0.smithi100.stdout:3837: read oid 20 snap -1 2024-04-07T16:28:56.957 INFO:tasks.rados.rados.0.smithi100.stdout:3837: expect (ObjNum 1116 snap 322 seq_num 1116) 2024-04-07T16:28:56.957 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 24 current snap is 363 2024-04-07T16:28:56.957 INFO:tasks.rados.rados.0.smithi100.stdout:3838: seq_num 1188 ranges {458752=704512} 2024-04-07T16:28:56.970 INFO:tasks.rados.rados.0.smithi100.stdout:3838: writing smithi10066867-24 from 458752 to 1163264 tid 1 2024-04-07T16:28:56.970 INFO:tasks.rados.rados.0.smithi100.stdout:3839: snap_remove snap 352 2024-04-07T16:28:57.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:57 smithi144 bash[66654]: cluster 2024-04-07T16:28:56.102271+0000 mon.a (mon.0) 1359 : cluster [DBG] osdmap e778: 8 total, 8 up, 8 in 2024-04-07T16:28:57.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:57 smithi144 bash[66654]: cluster 2024-04-07T16:28:56.969780+0000 mon.a (mon.0) 1360 : cluster [DBG] osdmap e779: 8 total, 8 up, 8 in 2024-04-07T16:28:57.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:57 smithi100 bash[74942]: cluster 2024-04-07T16:28:56.102271+0000 mon.a (mon.0) 1359 : cluster [DBG] osdmap e778: 8 total, 8 up, 8 in 2024-04-07T16:28:57.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:57 smithi100 bash[74942]: cluster 2024-04-07T16:28:56.969780+0000 mon.a (mon.0) 1360 : cluster [DBG] osdmap e779: 8 total, 8 up, 8 in 2024-04-07T16:28:57.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:57 smithi100 bash[76202]: cluster 2024-04-07T16:28:56.102271+0000 mon.a (mon.0) 1359 : cluster [DBG] osdmap e778: 8 total, 8 up, 8 in 2024-04-07T16:28:57.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:57 smithi100 bash[76202]: cluster 2024-04-07T16:28:56.969780+0000 mon.a (mon.0) 1360 : cluster [DBG] osdmap e779: 8 total, 8 up, 8 in 2024-04-07T16:28:57.966 INFO:tasks.rados.rados.0.smithi100.stdout:3839: done (2 left) 2024-04-07T16:28:57.966 INFO:tasks.rados.rados.0.smithi100.stdout:3840: read oid 50 snap 349 2024-04-07T16:28:57.967 INFO:tasks.rados.rados.0.smithi100.stdout:3840: expect (ObjNum 1162 snap 343 seq_num 1162) 2024-04-07T16:28:57.967 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 28 current snap is 363 2024-04-07T16:28:57.967 INFO:tasks.rados.rados.0.smithi100.stdout:3841: seq_num 1189 ranges {606208=720896} 2024-04-07T16:28:57.975 INFO:tasks.rados.rados.0.smithi100.stdout:3841: writing smithi10066867-28 from 606208 to 1327104 tid 1 2024-04-07T16:28:57.984 INFO:tasks.rados.rados.0.smithi100.stdout:3838: finishing write tid 1 to smithi10066867-24 2024-04-07T16:28:57.984 INFO:tasks.rados.rados.0.smithi100.stdout:3838: finishing write tid 2 to smithi10066867-24 2024-04-07T16:28:57.985 INFO:tasks.rados.rados.0.smithi100.stdout:3838: finishing write tid 3 to smithi10066867-24 2024-04-07T16:28:57.985 INFO:tasks.rados.rados.0.smithi100.stdout:3838: oid 24 updating version 0 to 1292 2024-04-07T16:28:57.985 INFO:tasks.rados.rados.0.smithi100.stdout:3838: oid 24 version 1292 is already newer than 1291 2024-04-07T16:28:57.985 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 24 v 1292 (ObjNum 1188 snap 363 seq_num 1188) dirty exists 2024-04-07T16:28:57.985 INFO:tasks.rados.rados.0.smithi100.stdout:3838: left oid 24 (ObjNum 1188 snap 363 seq_num 1188) 2024-04-07T16:28:57.985 INFO:tasks.rados.rados.0.smithi100.stdout:3837: expect (ObjNum 1116 snap 322 seq_num 1116) 2024-04-07T16:28:58.024 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/mon.a/config 2024-04-07T16:28:58.262 INFO:tasks.rados.rados.0.smithi100.stdout:3841: finishing write tid 1 to smithi10066867-28 2024-04-07T16:28:58.262 INFO:tasks.rados.rados.0.smithi100.stdout:3841: finishing write tid 2 to smithi10066867-28 2024-04-07T16:28:58.263 INFO:tasks.rados.rados.0.smithi100.stdout:3840: expect (ObjNum 1162 snap 343 seq_num 1162) 2024-04-07T16:28:58.269 INFO:tasks.rados.rados.0.smithi100.stdout:3841: finishing write tid 3 to smithi10066867-28 2024-04-07T16:28:58.269 INFO:tasks.rados.rados.0.smithi100.stdout:3841: oid 28 updating version 0 to 1889 2024-04-07T16:28:58.269 INFO:tasks.rados.rados.0.smithi100.stdout:3841: oid 28 version 1889 is already newer than 1888 2024-04-07T16:28:58.269 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 28 v 1889 (ObjNum 1189 snap 363 seq_num 1189) dirty exists 2024-04-07T16:28:58.269 INFO:tasks.rados.rados.0.smithi100.stdout:3841: left oid 28 (ObjNum 1189 snap 363 seq_num 1189) 2024-04-07T16:28:58.269 INFO:tasks.rados.rados.0.smithi100.stdout:3837: done (3 left) 2024-04-07T16:28:58.270 INFO:tasks.rados.rados.0.smithi100.stdout:3838: done (2 left) 2024-04-07T16:28:58.270 INFO:tasks.rados.rados.0.smithi100.stdout:3840: done (1 left) 2024-04-07T16:28:58.270 INFO:tasks.rados.rados.0.smithi100.stdout:3841: done (0 left) 2024-04-07T16:28:58.270 INFO:tasks.rados.rados.0.smithi100.stdout:3842: snap_remove snap 360 2024-04-07T16:28:58.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:58 smithi144 bash[66654]: cluster 2024-04-07T16:28:56.829000+0000 mgr.y (mgr.25175) 407 : cluster [DBG] pgmap v761: 121 pgs: 121 active+clean; 151 MiB data, 1.7 GiB used, 714 GiB / 715 GiB avail; 10 MiB/s rd, 1.1 MiB/s wr, 32 op/s 2024-04-07T16:28:58.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:28:58 smithi144 bash[66654]: cluster 2024-04-07T16:28:57.975915+0000 mon.a (mon.0) 1361 : cluster [DBG] osdmap e780: 8 total, 8 up, 8 in 2024-04-07T16:28:58.420 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:58 smithi100 bash[74942]: cluster 2024-04-07T16:28:56.829000+0000 mgr.y (mgr.25175) 407 : cluster [DBG] pgmap v761: 121 pgs: 121 active+clean; 151 MiB data, 1.7 GiB used, 714 GiB / 715 GiB avail; 10 MiB/s rd, 1.1 MiB/s wr, 32 op/s 2024-04-07T16:28:58.420 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:28:58 smithi100 bash[74942]: cluster 2024-04-07T16:28:57.975915+0000 mon.a (mon.0) 1361 : cluster [DBG] osdmap e780: 8 total, 8 up, 8 in 2024-04-07T16:28:58.421 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:58 smithi100 bash[76202]: cluster 2024-04-07T16:28:56.829000+0000 mgr.y (mgr.25175) 407 : cluster [DBG] pgmap v761: 121 pgs: 121 active+clean; 151 MiB data, 1.7 GiB used, 714 GiB / 715 GiB avail; 10 MiB/s rd, 1.1 MiB/s wr, 32 op/s 2024-04-07T16:28:58.421 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:28:58 smithi100 bash[76202]: cluster 2024-04-07T16:28:57.975915+0000 mon.a (mon.0) 1361 : cluster [DBG] osdmap e780: 8 total, 8 up, 8 in 2024-04-07T16:28:58.688 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:28:58 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:28:58] "GET /metrics HTTP/1.1" 200 36344 "" "Prometheus/2.43.0" 2024-04-07T16:28:59.114 INFO:tasks.rados.rados.0.smithi100.stdout:3842: done (0 left) 2024-04-07T16:28:59.114 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 18 current snap is 363 2024-04-07T16:28:59.114 INFO:tasks.rados.rados.0.smithi100.stdout:3843: seq_num 1190 ranges {2654208=737280} 2024-04-07T16:28:59.133 INFO:tasks.rados.rados.0.smithi100.stdout:3843: writing smithi10066867-18 from 2654208 to 3391488 tid 1 2024-04-07T16:28:59.133 INFO:tasks.rados.rados.0.smithi100.stdout:3844: snap_create 2024-04-07T16:28:59.142 INFO:tasks.rados.rados.0.smithi100.stdout:3843: finishing write tid 1 to smithi10066867-18 2024-04-07T16:28:59.142 INFO:tasks.rados.rados.0.smithi100.stdout:3843: finishing write tid 2 to smithi10066867-18 2024-04-07T16:28:59.144 INFO:tasks.rados.rados.0.smithi100.stdout:3843: finishing write tid 3 to smithi10066867-18 2024-04-07T16:28:59.144 INFO:tasks.rados.rados.0.smithi100.stdout:3843: oid 18 updating version 0 to 1958 2024-04-07T16:28:59.144 INFO:tasks.rados.rados.0.smithi100.stdout:3843: oid 18 updating version 1958 to 1959 2024-04-07T16:28:59.144 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 18 v 1959 (ObjNum 1190 snap 363 seq_num 1190) dirty exists 2024-04-07T16:28:59.144 INFO:tasks.rados.rados.0.smithi100.stdout:3843: left oid 18 (ObjNum 1190 snap 363 seq_num 1190) 2024-04-07T16:29:00.129 INFO:tasks.rados.rados.0.smithi100.stdout:3843: done (1 left) 2024-04-07T16:29:00.129 INFO:tasks.rados.rados.0.smithi100.stdout:3844: done (0 left) 2024-04-07T16:29:00.129 INFO:tasks.rados.rados.0.smithi100.stdout:3845: snap_create 2024-04-07T16:29:00.188 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:29:00 smithi100 bash[74942]: cluster 2024-04-07T16:28:58.829775+0000 mgr.y (mgr.25175) 408 : cluster [DBG] pgmap v764: 121 pgs: 1 active+clean+snaptrim, 120 active+clean; 152 MiB data, 1.7 GiB used, 714 GiB / 715 GiB avail; 7.5 MiB/s rd, 442 KiB/s wr, 23 op/s 2024-04-07T16:29:00.188 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:29:00 smithi100 bash[74942]: cluster 2024-04-07T16:28:59.127116+0000 mon.a (mon.0) 1362 : cluster [DBG] osdmap e781: 8 total, 8 up, 8 in 2024-04-07T16:29:00.189 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:29:00 smithi100 bash[76202]: cluster 2024-04-07T16:28:58.829775+0000 mgr.y (mgr.25175) 408 : cluster [DBG] pgmap v764: 121 pgs: 1 active+clean+snaptrim, 120 active+clean; 152 MiB data, 1.7 GiB used, 714 GiB / 715 GiB avail; 7.5 MiB/s rd, 442 KiB/s wr, 23 op/s 2024-04-07T16:29:00.189 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:29:00 smithi100 bash[76202]: cluster 2024-04-07T16:28:59.127116+0000 mon.a (mon.0) 1362 : cluster [DBG] osdmap e781: 8 total, 8 up, 8 in 2024-04-07T16:29:00.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:29:00 smithi144 bash[66654]: cluster 2024-04-07T16:28:58.829775+0000 mgr.y (mgr.25175) 408 : cluster [DBG] pgmap v764: 121 pgs: 1 active+clean+snaptrim, 120 active+clean; 152 MiB data, 1.7 GiB used, 714 GiB / 715 GiB avail; 7.5 MiB/s rd, 442 KiB/s wr, 23 op/s 2024-04-07T16:29:00.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:29:00 smithi144 bash[66654]: cluster 2024-04-07T16:28:59.127116+0000 mon.a (mon.0) 1362 : cluster [DBG] osdmap e781: 8 total, 8 up, 8 in 2024-04-07T16:29:01.139 INFO:tasks.rados.rados.0.smithi100.stdout:3845: done (0 left) 2024-04-07T16:29:01.140 INFO:tasks.rados.rados.0.smithi100.stdout:3846: rollback oid 29 current snap is 365 2024-04-07T16:29:01.140 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 29 to 362 2024-04-07T16:29:01.140 INFO:tasks.rados.rados.0.smithi100.stdout:3847: rmattr oid 15 current snap is 365 2024-04-07T16:29:01.140 INFO:tasks.rados.rados.0.smithi100.stdout:3847: done (1 left) 2024-04-07T16:29:01.140 INFO:tasks.rados.rados.0.smithi100.stdout:3848: rmattr oid 40 current snap is 365 2024-04-07T16:29:01.140 INFO:tasks.rados.rados.0.smithi100.stdout:3848: done (1 left) 2024-04-07T16:29:01.140 INFO:tasks.rados.rados.0.smithi100.stdout:3849: snap_remove snap 353 2024-04-07T16:29:01.166 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-04-07T16:29:01.166 INFO:teuthology.orchestra.run.smithi100.stdout: "mon": { 2024-04-07T16:29:01.166 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph version 19.0.0-2785-gadce92d0 (adce92d056533f22dc1ee37e3141d098b046ec1b) squid (dev)": 3 2024-04-07T16:29:01.166 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-07T16:29:01.166 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": { 2024-04-07T16:29:01.166 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph version 19.0.0-2785-gadce92d0 (adce92d056533f22dc1ee37e3141d098b046ec1b) squid (dev)": 2 2024-04-07T16:29:01.166 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-07T16:29:01.167 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": { 2024-04-07T16:29:01.167 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph version 19.0.0-2785-gadce92d0 (adce92d056533f22dc1ee37e3141d098b046ec1b) squid (dev)": 8 2024-04-07T16:29:01.167 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-07T16:29:01.167 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": { 2024-04-07T16:29:01.167 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph version 19.0.0-2785-gadce92d0 (adce92d056533f22dc1ee37e3141d098b046ec1b) squid (dev)": 2 2024-04-07T16:29:01.167 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-07T16:29:01.167 INFO:teuthology.orchestra.run.smithi100.stdout: "overall": { 2024-04-07T16:29:01.167 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph version 19.0.0-2785-gadce92d0 (adce92d056533f22dc1ee37e3141d098b046ec1b) squid (dev)": 15 2024-04-07T16:29:01.167 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-04-07T16:29:01.167 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-04-07T16:29:01.387 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:29:01 smithi100 bash[74942]: cluster 2024-04-07T16:29:00.143379+0000 mon.a (mon.0) 1363 : cluster [DBG] osdmap e782: 8 total, 8 up, 8 in 2024-04-07T16:29:01.387 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:29:01 smithi100 bash[76202]: cluster 2024-04-07T16:29:00.143379+0000 mon.a (mon.0) 1363 : cluster [DBG] osdmap e782: 8 total, 8 up, 8 in 2024-04-07T16:29:01.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:29:01 smithi144 bash[66654]: cluster 2024-04-07T16:29:00.143379+0000 mon.a (mon.0) 1363 : cluster [DBG] osdmap e782: 8 total, 8 up, 8 in 2024-04-07T16:29:01.912 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0702e3a8-f4f8-11ee-b649-cb9ed24678a4 -e sha1=adce92d056533f22dc1ee37e3141d098b046ec1b -- bash -c 'ceph versions | jq -e '"'"'.overall | length == 1'"'"'' 2024-04-07T16:29:01.958 INFO:tasks.rados.rados.0.smithi100.stdout:3849: done (1 left) 2024-04-07T16:29:01.958 INFO:tasks.rados.rados.0.smithi100.stdout:3850: delete oid 12 current snap is 365 2024-04-07T16:29:01.958 INFO:tasks.rados.rados.0.smithi100.stdout:3846: finishing rollback tid 1 to smithi10066867-29 2024-04-07T16:29:01.958 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 29 v 1598 (ObjNum 33 snap 0 seq_num 2703201088) dirty dne 2024-04-07T16:29:01.995 INFO:tasks.rados.rados.0.smithi100.stdout:3846: done (1 left) 2024-04-07T16:29:01.995 INFO:tasks.rados.rados.0.smithi100.stdout:3850: done (0 left) 2024-04-07T16:29:01.995 INFO:tasks.rados.rados.0.smithi100.stdout:3851: snap_create 2024-04-07T16:29:02.188 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:29:02 smithi100 bash[76202]: cluster 2024-04-07T16:29:00.830217+0000 mgr.y (mgr.25175) 409 : cluster [DBG] pgmap v767: 121 pgs: 1 active+clean+snaptrim, 120 active+clean; 155 MiB data, 1.7 GiB used, 714 GiB / 715 GiB avail; 12 MiB/s rd, 782 KiB/s wr, 22 op/s 2024-04-07T16:29:02.188 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:29:02 smithi100 bash[76202]: cluster 2024-04-07T16:29:01.152442+0000 mon.a (mon.0) 1364 : cluster [DBG] osdmap e783: 8 total, 8 up, 8 in 2024-04-07T16:29:02.189 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:29:02 smithi100 bash[76202]: audit 2024-04-07T16:29:01.180468+0000 mon.c (mon.2) 16 : audit [DBG] from='client.? 172.21.15.100:0/1815981225' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:29:02.189 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:29:02 smithi100 bash[76202]: audit 2024-04-07T16:29:01.736057+0000 mon.a (mon.0) 1365 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:29:02.189 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:29:02 smithi100 bash[76202]: cluster 2024-04-07T16:29:01.967898+0000 mon.a (mon.0) 1366 : cluster [DBG] osdmap e784: 8 total, 8 up, 8 in 2024-04-07T16:29:02.189 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:29:02 smithi100 bash[74942]: cluster 2024-04-07T16:29:00.830217+0000 mgr.y (mgr.25175) 409 : cluster [DBG] pgmap v767: 121 pgs: 1 active+clean+snaptrim, 120 active+clean; 155 MiB data, 1.7 GiB used, 714 GiB / 715 GiB avail; 12 MiB/s rd, 782 KiB/s wr, 22 op/s 2024-04-07T16:29:02.189 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:29:02 smithi100 bash[74942]: cluster 2024-04-07T16:29:01.152442+0000 mon.a (mon.0) 1364 : cluster [DBG] osdmap e783: 8 total, 8 up, 8 in 2024-04-07T16:29:02.189 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:29:02 smithi100 bash[74942]: audit 2024-04-07T16:29:01.180468+0000 mon.c (mon.2) 16 : audit [DBG] from='client.? 172.21.15.100:0/1815981225' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:29:02.189 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:29:02 smithi100 bash[74942]: audit 2024-04-07T16:29:01.736057+0000 mon.a (mon.0) 1365 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:29:02.189 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:29:02 smithi100 bash[74942]: cluster 2024-04-07T16:29:01.967898+0000 mon.a (mon.0) 1366 : cluster [DBG] osdmap e784: 8 total, 8 up, 8 in 2024-04-07T16:29:02.606 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:29:02 smithi144 bash[95233]: ts=2024-04-07T16:29:02.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:29:02.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:29:02 smithi144 bash[66654]: cluster 2024-04-07T16:29:00.830217+0000 mgr.y (mgr.25175) 409 : cluster [DBG] pgmap v767: 121 pgs: 1 active+clean+snaptrim, 120 active+clean; 155 MiB data, 1.7 GiB used, 714 GiB / 715 GiB avail; 12 MiB/s rd, 782 KiB/s wr, 22 op/s 2024-04-07T16:29:02.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:29:02 smithi144 bash[66654]: cluster 2024-04-07T16:29:01.152442+0000 mon.a (mon.0) 1364 : cluster [DBG] osdmap e783: 8 total, 8 up, 8 in 2024-04-07T16:29:02.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:29:02 smithi144 bash[66654]: audit 2024-04-07T16:29:01.180468+0000 mon.c (mon.2) 16 : audit [DBG] from='client.? 172.21.15.100:0/1815981225' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:29:02.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:29:02 smithi144 bash[66654]: audit 2024-04-07T16:29:01.736057+0000 mon.a (mon.0) 1365 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:29:02.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:29:02 smithi144 bash[66654]: cluster 2024-04-07T16:29:01.967898+0000 mon.a (mon.0) 1366 : cluster [DBG] osdmap e784: 8 total, 8 up, 8 in 2024-04-07T16:29:02.962 INFO:tasks.rados.rados.0.smithi100.stdout:3851: done (0 left) 2024-04-07T16:29:02.962 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 19 current snap is 366 2024-04-07T16:29:02.962 INFO:tasks.rados.rados.0.smithi100.stdout:3852: seq_num 1192 ranges {0=770048} 2024-04-07T16:29:02.968 INFO:tasks.rados.rados.0.smithi100.stdout:3852: writing smithi10066867-19 from 0 to 770048 tid 1 2024-04-07T16:29:02.968 INFO:tasks.rados.rados.0.smithi100.stdout:3853: copy_from oid 20 from oid 49 current snap is 366 2024-04-07T16:29:02.968 INFO:tasks.rados.rados.0.smithi100.stdout:3854: snap_remove snap 359 2024-04-07T16:29:03.974 INFO:tasks.rados.rados.0.smithi100.stdout:3852: finishing write tid 1 to smithi10066867-19 2024-04-07T16:29:03.974 INFO:tasks.rados.rados.0.smithi100.stdout:3852: finishing write tid 2 to smithi10066867-19 2024-04-07T16:29:03.974 INFO:tasks.rados.rados.0.smithi100.stdout:3853: finishing copy_from racing read to smithi10066867-20 2024-04-07T16:29:03.974 INFO:tasks.rados.rados.0.smithi100.stdout:3852: finishing write tid 3 to smithi10066867-19 2024-04-07T16:29:03.974 INFO:tasks.rados.rados.0.smithi100.stdout:3852: oid 19 updating version 0 to 1293 2024-04-07T16:29:03.975 INFO:tasks.rados.rados.0.smithi100.stdout:3852: oid 19 updating version 1293 to 1294 2024-04-07T16:29:03.975 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 19 v 1294 (ObjNum 1192 snap 366 seq_num 1192) dirty exists 2024-04-07T16:29:03.975 INFO:tasks.rados.rados.0.smithi100.stdout:3852: left oid 19 (ObjNum 1192 snap 366 seq_num 1192) 2024-04-07T16:29:03.975 INFO:tasks.rados.rados.0.smithi100.stdout:3853: finishing copy_from to smithi10066867-20 2024-04-07T16:29:03.975 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 20 v 1830 (ObjNum 161 snap 0 seq_num 1) dirty exists 2024-04-07T16:29:03.975 INFO:tasks.rados.rados.0.smithi100.stdout:3852: done (2 left) 2024-04-07T16:29:03.975 INFO:tasks.rados.rados.0.smithi100.stdout:3853: done (1 left) 2024-04-07T16:29:03.975 INFO:tasks.rados.rados.0.smithi100.stdout:3854: done (0 left) 2024-04-07T16:29:03.975 INFO:tasks.rados.rados.0.smithi100.stdout:3855: rollback oid 48 current snap is 366 2024-04-07T16:29:03.975 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 48 to 332 2024-04-07T16:29:03.975 INFO:tasks.rados.rados.0.smithi100.stdout:3856: copy_from oid 27 from oid 45 current snap is 366 2024-04-07T16:29:03.975 INFO:tasks.rados.rados.0.smithi100.stdout:3857: delete oid 39 current snap is 366 2024-04-07T16:29:03.995 INFO:tasks.rados.rados.0.smithi100.stdout:3857: done (2 left) 2024-04-07T16:29:03.995 INFO:tasks.rados.rados.0.smithi100.stdout:3858: rollback oid 38 current snap is 366 2024-04-07T16:29:03.995 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 38 to 357 2024-04-07T16:29:03.995 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 23 current snap is 366 2024-04-07T16:29:03.995 INFO:tasks.rados.rados.0.smithi100.stdout:3859: seq_num 1194 ranges {516096=802816} 2024-04-07T16:29:04.019 INFO:tasks.rados.rados.0.smithi100.stdout:3859: writing smithi10066867-23 from 516096 to 1318912 tid 1 2024-04-07T16:29:04.019 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 42 current snap is 366 2024-04-07T16:29:04.019 INFO:tasks.rados.rados.0.smithi100.stdout:3860: seq_num 1195 ranges {0=417792,417792=401408} 2024-04-07T16:29:04.026 INFO:tasks.rados.rados.0.smithi100.stdout:3860: writing smithi10066867-42 from 0 to 417792 tid 1 2024-04-07T16:29:04.033 INFO:tasks.rados.rados.0.smithi100.stdout:3860: writing smithi10066867-42 from 417792 to 819200 tid 2 2024-04-07T16:29:04.033 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 22 current snap is 366 2024-04-07T16:29:04.033 INFO:tasks.rados.rados.0.smithi100.stdout:3861: seq_num 1196 ranges {0=434176,434176=401408} 2024-04-07T16:29:04.040 INFO:tasks.rados.rados.0.smithi100.stdout:3861: writing smithi10066867-22 from 0 to 434176 tid 1 2024-04-07T16:29:04.046 INFO:tasks.rados.rados.0.smithi100.stdout:3861: writing smithi10066867-22 from 434176 to 835584 tid 2 2024-04-07T16:29:04.046 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 10 current snap is 366 2024-04-07T16:29:04.046 INFO:tasks.rados.rados.0.smithi100.stdout:3855: finishing rollback tid 0 to smithi10066867-48 2024-04-07T16:29:04.046 INFO:tasks.rados.rados.0.smithi100.stdout:3855: finishing rollback tid 1 to smithi10066867-48 2024-04-07T16:29:04.046 INFO:tasks.rados.rados.0.smithi100.stdout:3855: finishing rollback tid 2 to smithi10066867-48 2024-04-07T16:29:04.046 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 48 v 1757 (ObjNum 1140 snap 328 seq_num 1140) dirty exists 2024-04-07T16:29:04.046 INFO:tasks.rados.rados.0.smithi100.stdout:3856: finishing copy_from racing read to smithi10066867-27 2024-04-07T16:29:04.046 INFO:tasks.rados.rados.0.smithi100.stdout:3856: finishing copy_from to smithi10066867-27 2024-04-07T16:29:04.046 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 27 v 1829 (ObjNum 1041 snap 307 seq_num 1041) dirty exists 2024-04-07T16:29:04.046 INFO:tasks.rados.rados.0.smithi100.stdout:3858: finishing rollback tid 0 to smithi10066867-38 2024-04-07T16:29:04.047 INFO:tasks.rados.rados.0.smithi100.stdout:3862: seq_num 1197 ranges {417792=450560,868352=401408} 2024-04-07T16:29:04.056 INFO:tasks.rados.rados.0.smithi100.stdout:3862: writing smithi10066867-10 from 417792 to 868352 tid 1 2024-04-07T16:29:04.062 INFO:tasks.rados.rados.0.smithi100.stdout:3862: writing smithi10066867-10 from 868352 to 1269760 tid 2 2024-04-07T16:29:04.062 INFO:tasks.rados.rados.0.smithi100.stdout:3855: done (6 left) 2024-04-07T16:29:04.062 INFO:tasks.rados.rados.0.smithi100.stdout:3856: done (5 left) 2024-04-07T16:29:04.062 INFO:tasks.rados.rados.0.smithi100.stdout:3863: delete oid 49 current snap is 366 2024-04-07T16:29:04.062 INFO:tasks.rados.rados.0.smithi100.stdout:3858: finishing rollback tid 1 to smithi10066867-38 2024-04-07T16:29:04.062 INFO:tasks.rados.rados.0.smithi100.stdout:3858: finishing rollback tid 2 to smithi10066867-38 2024-04-07T16:29:04.062 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 38 v 1722 (ObjNum 1134 snap 326 seq_num 1134) dirty exists 2024-04-07T16:29:04.062 INFO:tasks.rados.rados.0.smithi100.stdout:3859: finishing write tid 1 to smithi10066867-23 2024-04-07T16:29:04.063 INFO:tasks.rados.rados.0.smithi100.stdout:3859: finishing write tid 2 to smithi10066867-23 2024-04-07T16:29:04.063 INFO:tasks.rados.rados.0.smithi100.stdout:3859: finishing write tid 3 to smithi10066867-23 2024-04-07T16:29:04.063 INFO:tasks.rados.rados.0.smithi100.stdout:3859: oid 23 updating version 0 to 1968 2024-04-07T16:29:04.063 INFO:tasks.rados.rados.0.smithi100.stdout:3859: oid 23 version 1968 is already newer than 1967 2024-04-07T16:29:04.063 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 23 v 1968 (ObjNum 1194 snap 366 seq_num 1194) dirty exists 2024-04-07T16:29:04.063 INFO:tasks.rados.rados.0.smithi100.stdout:3859: left oid 23 (ObjNum 1194 snap 366 seq_num 1194) 2024-04-07T16:29:04.063 INFO:tasks.rados.rados.0.smithi100.stdout:3860: finishing write tid 1 to smithi10066867-42 2024-04-07T16:29:04.063 INFO:tasks.rados.rados.0.smithi100.stdout:3860: finishing write tid 2 to smithi10066867-42 2024-04-07T16:29:04.063 INFO:tasks.rados.rados.0.smithi100.stdout:3860: finishing write tid 3 to smithi10066867-42 2024-04-07T16:29:04.063 INFO:tasks.rados.rados.0.smithi100.stdout:3860: finishing write tid 4 to smithi10066867-42 2024-04-07T16:29:04.063 INFO:tasks.rados.rados.0.smithi100.stdout:3860: oid 42 updating version 0 to 1603 2024-04-07T16:29:04.063 INFO:tasks.rados.rados.0.smithi100.stdout:3860: oid 42 version 1603 is already newer than 1602 2024-04-07T16:29:04.063 INFO:tasks.rados.rados.0.smithi100.stdout:3860: oid 42 updating version 1603 to 1604 2024-04-07T16:29:04.063 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 42 v 1604 (ObjNum 1195 snap 366 seq_num 1195) dirty exists 2024-04-07T16:29:04.064 INFO:tasks.rados.rados.0.smithi100.stdout:3860: left oid 42 (ObjNum 1195 snap 366 seq_num 1195) 2024-04-07T16:29:04.064 INFO:tasks.rados.rados.0.smithi100.stdout:3861: finishing write tid 1 to smithi10066867-22 2024-04-07T16:29:04.064 INFO:tasks.rados.rados.0.smithi100.stdout:3861: finishing write tid 2 to smithi10066867-22 2024-04-07T16:29:04.064 INFO:tasks.rados.rados.0.smithi100.stdout:3861: finishing write tid 3 to smithi10066867-22 2024-04-07T16:29:04.064 INFO:tasks.rados.rados.0.smithi100.stdout:3861: finishing write tid 4 to smithi10066867-22 2024-04-07T16:29:04.064 INFO:tasks.rados.rados.0.smithi100.stdout:3861: oid 22 updating version 0 to 1832 2024-04-07T16:29:04.064 INFO:tasks.rados.rados.0.smithi100.stdout:3861: oid 22 updating version 1832 to 1833 2024-04-07T16:29:04.064 INFO:tasks.rados.rados.0.smithi100.stdout:3861: oid 22 version 1833 is already newer than 1831 2024-04-07T16:29:04.064 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 22 v 1833 (ObjNum 1196 snap 366 seq_num 1196) dirty exists 2024-04-07T16:29:04.064 INFO:tasks.rados.rados.0.smithi100.stdout:3861: left oid 22 (ObjNum 1196 snap 366 seq_num 1196) 2024-04-07T16:29:04.064 INFO:tasks.rados.rados.0.smithi100.stdout:3862: finishing write tid 1 to smithi10066867-10 2024-04-07T16:29:04.065 INFO:tasks.rados.rados.0.smithi100.stdout:3862: finishing write tid 2 to smithi10066867-10 2024-04-07T16:29:04.066 INFO:tasks.rados.rados.0.smithi100.stdout:3862: finishing write tid 3 to smithi10066867-10 2024-04-07T16:29:04.067 INFO:tasks.rados.rados.0.smithi100.stdout:3858: done (5 left) 2024-04-07T16:29:04.067 INFO:tasks.rados.rados.0.smithi100.stdout:3859: done (4 left) 2024-04-07T16:29:04.067 INFO:tasks.rados.rados.0.smithi100.stdout:3860: done (3 left) 2024-04-07T16:29:04.067 INFO:tasks.rados.rados.0.smithi100.stdout:3861: done (2 left) 2024-04-07T16:29:04.067 INFO:tasks.rados.rados.0.smithi100.stdout:3863: done (1 left) 2024-04-07T16:29:04.067 INFO:tasks.rados.rados.0.smithi100.stdout:3864: delete oid 1 current snap is 366 2024-04-07T16:29:04.070 INFO:tasks.rados.rados.0.smithi100.stdout:3862: finishing write tid 4 to smithi10066867-10 2024-04-07T16:29:04.070 INFO:tasks.rados.rados.0.smithi100.stdout:3862: oid 10 updating version 0 to 1850 2024-04-07T16:29:04.070 INFO:tasks.rados.rados.0.smithi100.stdout:3862: oid 10 version 1850 is already newer than 1849 2024-04-07T16:29:04.070 INFO:tasks.rados.rados.0.smithi100.stdout:3862: oid 10 updating version 1850 to 1851 2024-04-07T16:29:04.070 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 10 v 1851 (ObjNum 1197 snap 366 seq_num 1197) dirty exists 2024-04-07T16:29:04.070 INFO:tasks.rados.rados.0.smithi100.stdout:3862: left oid 10 (ObjNum 1197 snap 366 seq_num 1197) 2024-04-07T16:29:04.071 INFO:tasks.rados.rados.0.smithi100.stdout:3862: done (1 left) 2024-04-07T16:29:04.071 INFO:tasks.rados.rados.0.smithi100.stdout:3864: done (0 left) 2024-04-07T16:29:04.071 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 37 current snap is 366 2024-04-07T16:29:04.071 INFO:tasks.rados.rados.0.smithi100.stdout:3865: seq_num 1200 ranges {7487488=499712,7987200=401408} 2024-04-07T16:29:04.126 INFO:tasks.rados.rados.0.smithi100.stdout:3865: writing smithi10066867-37 from 7487488 to 7987200 tid 1 2024-04-07T16:29:04.129 INFO:tasks.rados.rados.0.smithi100.stdout:3865: writing smithi10066867-37 from 7987200 to 8388608 tid 2 2024-04-07T16:29:04.129 INFO:tasks.rados.rados.0.smithi100.stdout:3866: snap_create 2024-04-07T16:29:04.134 INFO:tasks.rados.rados.0.smithi100.stdout:3865: finishing write tid 1 to smithi10066867-37 2024-04-07T16:29:04.135 INFO:tasks.rados.rados.0.smithi100.stdout:3865: finishing write tid 2 to smithi10066867-37 2024-04-07T16:29:04.135 INFO:tasks.rados.rados.0.smithi100.stdout:3865: finishing write tid 3 to smithi10066867-37 2024-04-07T16:29:04.137 INFO:tasks.rados.rados.0.smithi100.stdout:3865: finishing write tid 4 to smithi10066867-37 2024-04-07T16:29:04.137 INFO:tasks.rados.rados.0.smithi100.stdout:3865: oid 37 updating version 0 to 1607 2024-04-07T16:29:04.137 INFO:tasks.rados.rados.0.smithi100.stdout:3865: oid 37 version 1607 is already newer than 1606 2024-04-07T16:29:04.137 INFO:tasks.rados.rados.0.smithi100.stdout:3865: oid 37 version 1607 is already newer than 1605 2024-04-07T16:29:04.137 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 37 v 1607 (ObjNum 1200 snap 366 seq_num 1200) dirty exists 2024-04-07T16:29:04.137 INFO:tasks.rados.rados.0.smithi100.stdout:3865: left oid 37 (ObjNum 1200 snap 366 seq_num 1200) 2024-04-07T16:29:04.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:29:03 smithi144 bash[66654]: cluster 2024-04-07T16:29:02.831272+0000 mgr.y (mgr.25175) 410 : cluster [DBG] pgmap v770: 121 pgs: 1 active+clean+snaptrim, 120 active+clean; 156 MiB data, 1.7 GiB used, 714 GiB / 715 GiB avail; 4.3 MiB/s rd, 516 KiB/s wr, 5 op/s 2024-04-07T16:29:04.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:29:03 smithi144 bash[66654]: cluster 2024-04-07T16:29:02.975500+0000 mon.a (mon.0) 1367 : cluster [DBG] osdmap e785: 8 total, 8 up, 8 in 2024-04-07T16:29:04.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:29:03 smithi100 bash[74942]: cluster 2024-04-07T16:29:02.831272+0000 mgr.y (mgr.25175) 410 : cluster [DBG] pgmap v770: 121 pgs: 1 active+clean+snaptrim, 120 active+clean; 156 MiB data, 1.7 GiB used, 714 GiB / 715 GiB avail; 4.3 MiB/s rd, 516 KiB/s wr, 5 op/s 2024-04-07T16:29:04.439 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:29:03 smithi100 bash[74942]: cluster 2024-04-07T16:29:02.975500+0000 mon.a (mon.0) 1367 : cluster [DBG] osdmap e785: 8 total, 8 up, 8 in 2024-04-07T16:29:04.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:29:03 smithi100 bash[76202]: cluster 2024-04-07T16:29:02.831272+0000 mgr.y (mgr.25175) 410 : cluster [DBG] pgmap v770: 121 pgs: 1 active+clean+snaptrim, 120 active+clean; 156 MiB data, 1.7 GiB used, 714 GiB / 715 GiB avail; 4.3 MiB/s rd, 516 KiB/s wr, 5 op/s 2024-04-07T16:29:04.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:29:03 smithi100 bash[76202]: cluster 2024-04-07T16:29:02.975500+0000 mon.a (mon.0) 1367 : cluster [DBG] osdmap e785: 8 total, 8 up, 8 in 2024-04-07T16:29:05.005 INFO:tasks.rados.rados.0.smithi100.stdout:3865: done (1 left) 2024-04-07T16:29:05.005 INFO:tasks.rados.rados.0.smithi100.stdout:3866: done (0 left) 2024-04-07T16:29:05.005 INFO:tasks.rados.rados.0.smithi100.stdout:3867: read oid 25 snap -1 2024-04-07T16:29:05.006 INFO:tasks.rados.rados.0.smithi100.stdout:3867: expect deleted 2024-04-07T16:29:05.006 INFO:tasks.rados.rados.0.smithi100.stdout:3868: rollback oid 21 current snap is 367 2024-04-07T16:29:05.006 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 21 to 349 2024-04-07T16:29:05.006 INFO:tasks.rados.rados.0.smithi100.stdout:3869: read oid 35 snap 358 2024-04-07T16:29:05.006 INFO:tasks.rados.rados.0.smithi100.stdout:3869: expect deleted 2024-04-07T16:29:05.006 INFO:tasks.rados.rados.0.smithi100.stdout:3870: snap_create 2024-04-07T16:29:05.018 INFO:tasks.rados.rados.0.smithi100.stdout:3868: finishing rollback tid 0 to smithi10066867-21 2024-04-07T16:29:05.018 INFO:tasks.rados.rados.0.smithi100.stdout:3868: finishing rollback tid 1 to smithi10066867-21 2024-04-07T16:29:05.019 INFO:tasks.rados.rados.0.smithi100.stdout:3868: finishing rollback tid 2 to smithi10066867-21 2024-04-07T16:29:05.019 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 21 v 1818 (ObjNum 1041 snap 307 seq_num 1041) dirty exists 2024-04-07T16:29:05.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:29:05 smithi144 bash[66654]: cluster 2024-04-07T16:29:03.978867+0000 mon.a (mon.0) 1368 : cluster [DBG] osdmap e786: 8 total, 8 up, 8 in 2024-04-07T16:29:05.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:29:05 smithi100 bash[74942]: cluster 2024-04-07T16:29:03.978867+0000 mon.a (mon.0) 1368 : cluster [DBG] osdmap e786: 8 total, 8 up, 8 in 2024-04-07T16:29:05.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:29:05 smithi100 bash[76202]: cluster 2024-04-07T16:29:03.978867+0000 mon.a (mon.0) 1368 : cluster [DBG] osdmap e786: 8 total, 8 up, 8 in 2024-04-07T16:29:06.001 INFO:tasks.rados.rados.0.smithi100.stdout:3867: done (3 left) 2024-04-07T16:29:06.002 INFO:tasks.rados.rados.0.smithi100.stdout:3868: done (2 left) 2024-04-07T16:29:06.002 INFO:tasks.rados.rados.0.smithi100.stdout:3869: done (1 left) 2024-04-07T16:29:06.002 INFO:tasks.rados.rados.0.smithi100.stdout:3870: done (0 left) 2024-04-07T16:29:06.002 INFO:tasks.rados.rados.0.smithi100.stdout:3871: setattr oid 4 current snap is 368 2024-04-07T16:29:06.003 INFO:tasks.rados.rados.0.smithi100.stdout:3872: setattr oid 39 current snap is 368 2024-04-07T16:29:06.005 INFO:tasks.rados.rados.0.smithi100.stdout:3873: snap_create 2024-04-07T16:29:06.020 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 39 v 1608 (ObjNum 33 snap 0 seq_num 2703284224) dirty exists 2024-04-07T16:29:06.026 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 4 v 1970 (ObjNum 1160 snap 343 seq_num 1160) dirty exists 2024-04-07T16:29:06.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:29:06 smithi144 bash[66654]: cluster 2024-04-07T16:29:04.832171+0000 mgr.y (mgr.25175) 411 : cluster [DBG] pgmap v773: 121 pgs: 121 active+clean; 157 MiB data, 1.7 GiB used, 714 GiB / 715 GiB avail; 199 KiB/s rd, 544 KiB/s wr, 3 op/s 2024-04-07T16:29:06.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:29:06 smithi144 bash[66654]: cluster 2024-04-07T16:29:05.014710+0000 mon.a (mon.0) 1369 : cluster [DBG] osdmap e787: 8 total, 8 up, 8 in 2024-04-07T16:29:06.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:29:06 smithi100 bash[74942]: cluster 2024-04-07T16:29:04.832171+0000 mgr.y (mgr.25175) 411 : cluster [DBG] pgmap v773: 121 pgs: 121 active+clean; 157 MiB data, 1.7 GiB used, 714 GiB / 715 GiB avail; 199 KiB/s rd, 544 KiB/s wr, 3 op/s 2024-04-07T16:29:06.439 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:29:06 smithi100 bash[74942]: cluster 2024-04-07T16:29:05.014710+0000 mon.a (mon.0) 1369 : cluster [DBG] osdmap e787: 8 total, 8 up, 8 in 2024-04-07T16:29:06.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:29:06 smithi100 bash[76202]: cluster 2024-04-07T16:29:04.832171+0000 mgr.y (mgr.25175) 411 : cluster [DBG] pgmap v773: 121 pgs: 121 active+clean; 157 MiB data, 1.7 GiB used, 714 GiB / 715 GiB avail; 199 KiB/s rd, 544 KiB/s wr, 3 op/s 2024-04-07T16:29:06.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:29:06 smithi100 bash[76202]: cluster 2024-04-07T16:29:05.014710+0000 mon.a (mon.0) 1369 : cluster [DBG] osdmap e787: 8 total, 8 up, 8 in 2024-04-07T16:29:06.661 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/mon.a/config 2024-04-07T16:29:06.963 INFO:tasks.rados.rados.0.smithi100.stdout:3871: done (2 left) 2024-04-07T16:29:06.964 INFO:tasks.rados.rados.0.smithi100.stdout:3872: done (1 left) 2024-04-07T16:29:06.964 INFO:tasks.rados.rados.0.smithi100.stdout:3873: done (0 left) 2024-04-07T16:29:06.964 INFO:tasks.rados.rados.0.smithi100.stdout:3874: rollback oid 29 current snap is 369 2024-04-07T16:29:06.964 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 29 to 351 2024-04-07T16:29:06.964 INFO:tasks.rados.rados.0.smithi100.stdout:3875: rollback oid 14 current snap is 369 2024-04-07T16:29:06.964 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 14 to 355 2024-04-07T16:29:06.964 INFO:tasks.rados.rados.0.smithi100.stdout:3876: snap_remove snap 363 2024-04-07T16:29:07.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:29:07 smithi144 bash[66654]: cluster 2024-04-07T16:29:06.011378+0000 mon.a (mon.0) 1370 : cluster [DBG] osdmap e788: 8 total, 8 up, 8 in 2024-04-07T16:29:07.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:29:07 smithi144 bash[66654]: cluster 2024-04-07T16:29:06.977008+0000 mon.a (mon.0) 1371 : cluster [DBG] osdmap e789: 8 total, 8 up, 8 in 2024-04-07T16:29:07.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:29:07 smithi100 bash[74942]: cluster 2024-04-07T16:29:06.011378+0000 mon.a (mon.0) 1370 : cluster [DBG] osdmap e788: 8 total, 8 up, 8 in 2024-04-07T16:29:07.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:29:07 smithi100 bash[74942]: cluster 2024-04-07T16:29:06.977008+0000 mon.a (mon.0) 1371 : cluster [DBG] osdmap e789: 8 total, 8 up, 8 in 2024-04-07T16:29:07.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:29:07 smithi100 bash[76202]: cluster 2024-04-07T16:29:06.011378+0000 mon.a (mon.0) 1370 : cluster [DBG] osdmap e788: 8 total, 8 up, 8 in 2024-04-07T16:29:07.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:29:07 smithi100 bash[76202]: cluster 2024-04-07T16:29:06.977008+0000 mon.a (mon.0) 1371 : cluster [DBG] osdmap e789: 8 total, 8 up, 8 in 2024-04-07T16:29:07.969 INFO:tasks.rados.rados.0.smithi100.stdout:3876: done (2 left) 2024-04-07T16:29:07.969 INFO:tasks.rados.rados.0.smithi100.stdout:3877: rmattr oid 43 current snap is 369 2024-04-07T16:29:07.969 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 41 current snap is 369 2024-04-07T16:29:07.969 INFO:tasks.rados.rados.0.smithi100.stdout:3878: seq_num 1201 ranges {0=524288,524288=401408} 2024-04-07T16:29:07.969 INFO:tasks.rados.rados.0.smithi100.stdout:3878: writing smithi10066867-41 from 0 to 524288 tid 1 2024-04-07T16:29:07.969 INFO:tasks.rados.rados.0.smithi100.stdout:3878: writing smithi10066867-41 from 524288 to 925696 tid 2 2024-04-07T16:29:07.969 INFO:tasks.rados.rados.0.smithi100.stdout:3879: read oid 32 snap -1 2024-04-07T16:29:07.969 INFO:tasks.rados.rados.0.smithi100.stdout:3879: expect deleted 2024-04-07T16:29:07.969 INFO:tasks.rados.rados.0.smithi100.stdout:3880: copy_from oid 34 from oid 38 current snap is 369 2024-04-07T16:29:07.970 INFO:tasks.rados.rados.0.smithi100.stdout:3874: finishing rollback tid 1 to smithi10066867-29 2024-04-07T16:29:07.970 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 49 current snap is 369 2024-04-07T16:29:07.970 INFO:tasks.rados.rados.0.smithi100.stdout:3875: finishing rollback tid 0 to smithi10066867-14 2024-04-07T16:29:07.970 INFO:tasks.rados.rados.0.smithi100.stdout:3875: finishing rollback tid 1 to smithi10066867-14 2024-04-07T16:29:07.970 INFO:tasks.rados.rados.0.smithi100.stdout:3874: finishing rollback tid 2 to smithi10066867-29 2024-04-07T16:29:07.970 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 29 v 1610 (ObjNum 1149 snap 339 seq_num 1149) dirty exists 2024-04-07T16:29:07.970 INFO:tasks.rados.rados.0.smithi100.stdout:3875: finishing rollback tid 2 to smithi10066867-14 2024-04-07T16:29:07.970 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 14 v 1832 (ObjNum 1089 snap 318 seq_num 1089) dirty exists 2024-04-07T16:29:07.970 INFO:tasks.rados.rados.0.smithi100.stdout:3881: seq_num 1202 ranges {0=540672,540672=401408} 2024-04-07T16:29:07.973 INFO:tasks.rados.rados.0.smithi100.stdout:3881: writing smithi10066867-49 from 0 to 540672 tid 1 2024-04-07T16:29:07.976 INFO:tasks.rados.rados.0.smithi100.stdout:3881: writing smithi10066867-49 from 540672 to 942080 tid 2 2024-04-07T16:29:07.977 INFO:tasks.rados.rados.0.smithi100.stdout:3874: done (6 left) 2024-04-07T16:29:07.977 INFO:tasks.rados.rados.0.smithi100.stdout:3875: done (5 left) 2024-04-07T16:29:07.977 INFO:tasks.rados.rados.0.smithi100.stdout:3882: copy_from oid 15 from oid 23 current snap is 369 2024-04-07T16:29:07.977 INFO:tasks.rados.rados.0.smithi100.stdout:3879: done (5 left) 2024-04-07T16:29:07.977 INFO:tasks.rados.rados.0.smithi100.stdout:3883: read oid 8 snap -1 2024-04-07T16:29:07.977 INFO:tasks.rados.rados.0.smithi100.stdout:3883: expect (ObjNum 1100 snap 320 seq_num 1100) 2024-04-07T16:29:07.977 INFO:tasks.rados.rados.0.smithi100.stdout:3884: setattr oid 27 current snap is 369 2024-04-07T16:29:07.979 INFO:tasks.rados.rados.0.smithi100.stdout:3885: delete oid 7 current snap is 369 2024-04-07T16:29:07.997 INFO:tasks.rados.rados.0.smithi100.stdout:3878: finishing write tid 1 to smithi10066867-41 2024-04-07T16:29:07.997 INFO:tasks.rados.rados.0.smithi100.stdout:3878: finishing write tid 2 to smithi10066867-41 2024-04-07T16:29:07.997 INFO:tasks.rados.rados.0.smithi100.stdout:3878: finishing write tid 3 to smithi10066867-41 2024-04-07T16:29:07.999 INFO:tasks.rados.rados.0.smithi100.stdout:3878: finishing write tid 4 to smithi10066867-41 2024-04-07T16:29:08.000 INFO:tasks.rados.rados.0.smithi100.stdout:3878: oid 41 updating version 0 to 1834 2024-04-07T16:29:08.000 INFO:tasks.rados.rados.0.smithi100.stdout:3878: oid 41 version 1834 is already newer than 1833 2024-04-07T16:29:08.000 INFO:tasks.rados.rados.0.smithi100.stdout:3878: oid 41 updating version 1834 to 1835 2024-04-07T16:29:08.000 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 41 v 1835 (ObjNum 1201 snap 369 seq_num 1201) dirty exists 2024-04-07T16:29:08.000 INFO:tasks.rados.rados.0.smithi100.stdout:3878: left oid 41 (ObjNum 1201 snap 369 seq_num 1201) 2024-04-07T16:29:08.000 INFO:tasks.rados.rados.0.smithi100.stdout:3881: finishing write tid 1 to smithi10066867-49 2024-04-07T16:29:08.001 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 43 v 1830 (ObjNum 1030 snap 306 seq_num 1030) dirty exists 2024-04-07T16:29:08.001 INFO:tasks.rados.rados.0.smithi100.stdout:3881: finishing write tid 2 to smithi10066867-49 2024-04-07T16:29:08.001 INFO:tasks.rados.rados.0.smithi100.stdout:3881: finishing write tid 3 to smithi10066867-49 2024-04-07T16:29:08.002 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 27 v 1831 (ObjNum 1041 snap 307 seq_num 1041) dirty exists 2024-04-07T16:29:08.003 INFO:tasks.rados.rados.0.smithi100.stdout:3877: done (7 left) 2024-04-07T16:29:08.158 INFO:tasks.rados.rados.0.smithi100.stdout:3878: done (6 left) 2024-04-07T16:29:08.158 INFO:tasks.rados.rados.0.smithi100.stdout:3884: done (5 left) 2024-04-07T16:29:08.158 INFO:tasks.rados.rados.0.smithi100.stdout:3885: done (4 left) 2024-04-07T16:29:08.158 INFO:tasks.rados.rados.0.smithi100.stdout:3886: read oid 42 snap -1 2024-04-07T16:29:08.159 INFO:tasks.rados.rados.0.smithi100.stdout:3886: expect (ObjNum 1195 snap 366 seq_num 1195) 2024-04-07T16:29:08.159 INFO:tasks.rados.rados.0.smithi100.stdout:3887: copy_from oid 19 from oid 9 current snap is 369 2024-04-07T16:29:08.159 INFO:tasks.rados.rados.0.smithi100.stdout:3882: finishing copy_from racing read to smithi10066867-15 2024-04-07T16:29:08.159 INFO:tasks.rados.rados.0.smithi100.stdout:3888: read oid 30 snap 333 2024-04-07T16:29:08.159 INFO:tasks.rados.rados.0.smithi100.stdout:3888: expect (ObjNum 1081 snap 314 seq_num 1081) 2024-04-07T16:29:08.159 INFO:tasks.rados.rados.0.smithi100.stdout:3889: rollback oid 12 current snap is 369 2024-04-07T16:29:08.159 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 12 to 365 2024-04-07T16:29:08.159 INFO:tasks.rados.rados.0.smithi100.stdout:3890: rollback oid 50 current snap is 369 2024-04-07T16:29:08.159 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 50 to 332 2024-04-07T16:29:08.159 INFO:tasks.rados.rados.0.smithi100.stdout:3891: delete oid 48 current snap is 369 2024-04-07T16:29:08.161 INFO:tasks.rados.rados.0.smithi100.stdout:3882: finishing copy_from to smithi10066867-15 2024-04-07T16:29:08.161 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 15 v 1968 (ObjNum 1194 snap 366 seq_num 1194) dirty exists 2024-04-07T16:29:08.161 INFO:tasks.rados.rados.0.smithi100.stdout:3881: finishing write tid 4 to smithi10066867-49 2024-04-07T16:29:08.161 INFO:tasks.rados.rados.0.smithi100.stdout:3881: oid 49 updating version 0 to 1837 2024-04-07T16:29:08.161 INFO:tasks.rados.rados.0.smithi100.stdout:3881: oid 49 version 1837 is already newer than 1836 2024-04-07T16:29:08.162 INFO:tasks.rados.rados.0.smithi100.stdout:3881: oid 49 updating version 1837 to 1838 2024-04-07T16:29:08.162 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 49 v 1838 (ObjNum 1202 snap 369 seq_num 1202) dirty exists 2024-04-07T16:29:08.162 INFO:tasks.rados.rados.0.smithi100.stdout:3881: left oid 49 (ObjNum 1202 snap 369 seq_num 1202) 2024-04-07T16:29:08.162 INFO:tasks.rados.rados.0.smithi100.stdout:3881: done (9 left) 2024-04-07T16:29:08.162 INFO:tasks.rados.rados.0.smithi100.stdout:3882: done (8 left) 2024-04-07T16:29:08.162 INFO:tasks.rados.rados.0.smithi100.stdout:3891: done (7 left) 2024-04-07T16:29:08.162 INFO:tasks.rados.rados.0.smithi100.stdout:3892: delete oid 18 current snap is 369 2024-04-07T16:29:08.162 INFO:tasks.rados.rados.0.smithi100.stdout:3890: finishing rollback tid 0 to smithi10066867-50 2024-04-07T16:29:08.162 INFO:tasks.rados.rados.0.smithi100.stdout:3890: finishing rollback tid 1 to smithi10066867-50 2024-04-07T16:29:08.162 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 50 v 1834 (ObjNum 33 snap 0 seq_num 2703145872) dirty dne 2024-04-07T16:29:08.162 INFO:tasks.rados.rados.0.smithi100.stdout:3887: finishing copy_from racing read to smithi10066867-19 2024-04-07T16:29:08.162 INFO:tasks.rados.rados.0.smithi100.stdout:3890: done (7 left) 2024-04-07T16:29:08.162 INFO:tasks.rados.rados.0.smithi100.stdout:3892: done (6 left) 2024-04-07T16:29:08.162 INFO:tasks.rados.rados.0.smithi100.stdout:3893: rmattr oid 35 current snap is 369 2024-04-07T16:29:08.163 INFO:tasks.rados.rados.0.smithi100.stdout:3894: read oid 29 snap -1 2024-04-07T16:29:08.163 INFO:tasks.rados.rados.0.smithi100.stdout:3894: expect (ObjNum 1149 snap 339 seq_num 1149) 2024-04-07T16:29:08.163 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 14 current snap is 369 2024-04-07T16:29:08.163 INFO:tasks.rados.rados.0.smithi100.stdout:3895: seq_num 1206 ranges {2973696=606208,3579904=401408} 2024-04-07T16:29:08.163 INFO:tasks.rados.rados.0.smithi100.stdout:3895: writing smithi10066867-14 from 2973696 to 3579904 tid 1 2024-04-07T16:29:08.163 INFO:tasks.rados.rados.0.smithi100.stdout:3895: writing smithi10066867-14 from 3579904 to 3981312 tid 2 2024-04-07T16:29:08.163 INFO:tasks.rados.rados.0.smithi100.stdout:3896: copy_from oid 28 from oid 20 current snap is 369 2024-04-07T16:29:08.163 INFO:tasks.rados.rados.0.smithi100.stdout:3883: expect (ObjNum 1100 snap 320 seq_num 1100) 2024-04-07T16:29:08.163 INFO:tasks.rados.rados.0.smithi100.stdout:3886: expect (ObjNum 1195 snap 366 seq_num 1195) 2024-04-07T16:29:08.163 INFO:tasks.rados.rados.0.smithi100.stdout:3889: finishing rollback tid 1 to smithi10066867-12 2024-04-07T16:29:08.163 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 12 v 1831 (ObjNum 273 snap 0 seq_num 0) dirty dne 2024-04-07T16:29:08.163 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 35 v 1759 (ObjNum 65 snap 0 seq_num 1936487018) dirty exists 2024-04-07T16:29:08.163 INFO:tasks.rados.rados.0.smithi100.stdout:3887: finishing copy_from to smithi10066867-19 2024-04-07T16:29:08.163 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 19 v 1221 (ObjNum 1153 snap 340 seq_num 1153) dirty exists 2024-04-07T16:29:08.163 INFO:tasks.rados.rados.0.smithi100.stdout:3888: expect (ObjNum 1081 snap 314 seq_num 1081) 2024-04-07T16:29:08.296 INFO:tasks.rados.rados.0.smithi100.stdout:3880: finishing copy_from racing read to smithi10066867-34 2024-04-07T16:29:08.297 INFO:tasks.rados.rados.0.smithi100.stdout:3894: expect (ObjNum 1149 snap 339 seq_num 1149) 2024-04-07T16:29:08.354 INFO:tasks.rados.rados.0.smithi100.stdout:3896: finishing copy_from racing read to smithi10066867-28 2024-04-07T16:29:08.355 INFO:tasks.rados.rados.0.smithi100.stdout:3896: finishing copy_from to smithi10066867-28 2024-04-07T16:29:08.355 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 28 v 1830 (ObjNum 177 snap 0 seq_num 1) dirty exists 2024-04-07T16:29:08.355 INFO:tasks.rados.rados.0.smithi100.stdout:3880: finishing copy_from to smithi10066867-34 2024-04-07T16:29:08.355 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 34 v 1722 (ObjNum 1134 snap 326 seq_num 1134) dirty exists 2024-04-07T16:29:08.355 INFO:tasks.rados.rados.0.smithi100.stdout:3895: finishing write tid 1 to smithi10066867-14 2024-04-07T16:29:08.355 INFO:tasks.rados.rados.0.smithi100.stdout:3895: finishing write tid 2 to smithi10066867-14 2024-04-07T16:29:08.355 INFO:tasks.rados.rados.0.smithi100.stdout:3895: finishing write tid 3 to smithi10066867-14 2024-04-07T16:29:08.355 INFO:tasks.rados.rados.0.smithi100.stdout:3895: finishing write tid 4 to smithi10066867-14 2024-04-07T16:29:08.355 INFO:tasks.rados.rados.0.smithi100.stdout:3895: oid 14 updating version 0 to 1835 2024-04-07T16:29:08.355 INFO:tasks.rados.rados.0.smithi100.stdout:3895: oid 14 updating version 1835 to 1837 2024-04-07T16:29:08.355 INFO:tasks.rados.rados.0.smithi100.stdout:3895: oid 14 version 1837 is already newer than 1836 2024-04-07T16:29:08.355 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 14 v 1837 (ObjNum 1206 snap 369 seq_num 1206) dirty exists 2024-04-07T16:29:08.355 INFO:tasks.rados.rados.0.smithi100.stdout:3895: left oid 14 (ObjNum 1206 snap 369 seq_num 1206) 2024-04-07T16:29:08.355 INFO:tasks.rados.rados.0.smithi100.stdout:3880: done (9 left) 2024-04-07T16:29:08.355 INFO:tasks.rados.rados.0.smithi100.stdout:3883: done (8 left) 2024-04-07T16:29:08.355 INFO:tasks.rados.rados.0.smithi100.stdout:3886: done (7 left) 2024-04-07T16:29:08.356 INFO:tasks.rados.rados.0.smithi100.stdout:3887: done (6 left) 2024-04-07T16:29:08.356 INFO:tasks.rados.rados.0.smithi100.stdout:3888: done (5 left) 2024-04-07T16:29:08.356 INFO:tasks.rados.rados.0.smithi100.stdout:3889: done (4 left) 2024-04-07T16:29:08.356 INFO:tasks.rados.rados.0.smithi100.stdout:3893: done (3 left) 2024-04-07T16:29:08.356 INFO:tasks.rados.rados.0.smithi100.stdout:3894: done (2 left) 2024-04-07T16:29:08.356 INFO:tasks.rados.rados.0.smithi100.stdout:3895: done (1 left) 2024-04-07T16:29:08.356 INFO:tasks.rados.rados.0.smithi100.stdout:3896: done (0 left) 2024-04-07T16:29:08.356 INFO:tasks.rados.rados.0.smithi100.stdout:3897: read oid 25 snap -1 2024-04-07T16:29:08.356 INFO:tasks.rados.rados.0.smithi100.stdout:3897: expect deleted 2024-04-07T16:29:08.356 INFO:tasks.rados.rados.0.smithi100.stdout:3898: read oid 3 snap -1 2024-04-07T16:29:08.356 INFO:tasks.rados.rados.0.smithi100.stdout:3898: expect (ObjNum 1178 snap 359 seq_num 1178) 2024-04-07T16:29:08.356 INFO:tasks.rados.rados.0.smithi100.stdout:3899: snap_create 2024-04-07T16:29:08.357 INFO:journalctl@ceph.osd.7.smithi144.stdout:Apr 07 16:29:08 smithi144 bash[84215]: debug 2024-04-07T16:29:08.009+0000 7f7853c94700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 1ca after trimming was started 2024-04-07T16:29:08.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:29:08 smithi144 bash[66654]: cluster 2024-04-07T16:29:06.832875+0000 mgr.y (mgr.25175) 412 : cluster [DBG] pgmap v776: 121 pgs: 121 active+clean; 158 MiB data, 1.7 GiB used, 714 GiB / 715 GiB avail; 511 B/s rd, 524 KiB/s wr, 2 op/s 2024-04-07T16:29:08.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:29:08 smithi144 bash[66654]: cluster 2024-04-07T16:29:07.973094+0000 mon.a (mon.0) 1372 : cluster [DBG] osdmap e790: 8 total, 8 up, 8 in 2024-04-07T16:29:08.364 INFO:tasks.rados.rados.0.smithi100.stdout:3898: expect (ObjNum 1178 snap 359 seq_num 1178) 2024-04-07T16:29:08.416 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:29:08 smithi100 bash[74942]: cluster 2024-04-07T16:29:06.832875+0000 mgr.y (mgr.25175) 412 : cluster [DBG] pgmap v776: 121 pgs: 121 active+clean; 158 MiB data, 1.7 GiB used, 714 GiB / 715 GiB avail; 511 B/s rd, 524 KiB/s wr, 2 op/s 2024-04-07T16:29:08.416 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:29:08 smithi100 bash[74942]: cluster 2024-04-07T16:29:07.973094+0000 mon.a (mon.0) 1372 : cluster [DBG] osdmap e790: 8 total, 8 up, 8 in 2024-04-07T16:29:08.416 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:29:08 smithi100 bash[76202]: cluster 2024-04-07T16:29:06.832875+0000 mgr.y (mgr.25175) 412 : cluster [DBG] pgmap v776: 121 pgs: 121 active+clean; 158 MiB data, 1.7 GiB used, 714 GiB / 715 GiB avail; 511 B/s rd, 524 KiB/s wr, 2 op/s 2024-04-07T16:29:08.416 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:29:08 smithi100 bash[76202]: cluster 2024-04-07T16:29:07.973094+0000 mon.a (mon.0) 1372 : cluster [DBG] osdmap e790: 8 total, 8 up, 8 in 2024-04-07T16:29:08.688 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:29:08 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:29:08] "GET /metrics HTTP/1.1" 200 36339 "" "Prometheus/2.43.0" 2024-04-07T16:29:09.032 INFO:tasks.rados.rados.0.smithi100.stdout:3897: done (2 left) 2024-04-07T16:29:09.032 INFO:tasks.rados.rados.0.smithi100.stdout:3898: done (1 left) 2024-04-07T16:29:09.032 INFO:tasks.rados.rados.0.smithi100.stdout:3899: done (0 left) 2024-04-07T16:29:09.032 INFO:tasks.rados.rados.0.smithi100.stdout:3900: delete oid 10 current snap is 370 2024-04-07T16:29:09.036 INFO:tasks.rados.rados.0.smithi100.stdout:3900: done (0 left) 2024-04-07T16:29:09.036 INFO:tasks.rados.rados.0.smithi100.stdout:3901: copy_from oid 32 from oid 4 current snap is 370 2024-04-07T16:29:09.036 INFO:tasks.rados.rados.0.smithi100.stdout:3902: read oid 44 snap -1 2024-04-07T16:29:09.036 INFO:tasks.rados.rados.0.smithi100.stdout:3902: expect (ObjNum 1176 snap 358 seq_num 1176) 2024-04-07T16:29:09.036 INFO:tasks.rados.rados.0.smithi100.stdout:3903: copy_from oid 20 from oid 2 current snap is 370 2024-04-07T16:29:09.036 INFO:tasks.rados.rados.0.smithi100.stdout:3904: snap_remove snap 362 2024-04-07T16:29:09.753 INFO:teuthology.orchestra.run.smithi100.stdout:true 2024-04-07T16:29:10.052 INFO:tasks.rados.rados.0.smithi100.stdout:3903: finishing copy_from to smithi10066867-20 2024-04-07T16:29:10.052 INFO:tasks.rados.rados.0.smithi100.stdout:3903: got expected ENOENT (src dne) 2024-04-07T16:29:10.052 INFO:tasks.rados.rados.0.smithi100.stdout:3904: done (3 left) 2024-04-07T16:29:10.052 INFO:tasks.rados.rados.0.smithi100.stdout:3905: read oid 37 snap -1 2024-04-07T16:29:10.052 INFO:tasks.rados.rados.0.smithi100.stdout:3905: expect (ObjNum 1200 snap 366 seq_num 1200) 2024-04-07T16:29:10.052 INFO:tasks.rados.rados.0.smithi100.stdout:3906: rollback oid 8 current snap is 370 2024-04-07T16:29:10.052 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 8 to 355 2024-04-07T16:29:10.052 INFO:tasks.rados.rados.0.smithi100.stdout:3907: delete oid 3 current snap is 370 2024-04-07T16:29:10.052 INFO:tasks.rados.rados.0.smithi100.stdout:3903: finishing copy_from racing read to smithi10066867-20 2024-04-07T16:29:10.053 INFO:tasks.rados.rados.0.smithi100.stdout:3902: expect (ObjNum 1176 snap 358 seq_num 1176) 2024-04-07T16:29:10.208 INFO:tasks.rados.rados.0.smithi100.stdout:3901: finishing copy_from racing read to smithi10066867-32 2024-04-07T16:29:10.208 INFO:tasks.rados.rados.0.smithi100.stdout:3901: finishing copy_from to smithi10066867-32 2024-04-07T16:29:10.208 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 32 v 1970 (ObjNum 1160 snap 343 seq_num 1160) dirty exists 2024-04-07T16:29:10.208 INFO:tasks.rados.rados.0.smithi100.stdout:3906: finishing rollback tid 0 to smithi10066867-8 2024-04-07T16:29:10.208 INFO:tasks.rados.rados.0.smithi100.stdout:3906: finishing rollback tid 1 to smithi10066867-8 2024-04-07T16:29:10.208 INFO:tasks.rados.rados.0.smithi100.stdout:3906: finishing rollback tid 2 to smithi10066867-8 2024-04-07T16:29:10.208 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 8 v 1867 (ObjNum 1100 snap 320 seq_num 1100) dirty exists 2024-04-07T16:29:10.208 INFO:tasks.rados.rados.0.smithi100.stdout:3905: expect (ObjNum 1200 snap 366 seq_num 1200) 2024-04-07T16:29:10.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:29:10 smithi144 bash[66654]: cluster 2024-04-07T16:29:08.833487+0000 mgr.y (mgr.25175) 413 : cluster [DBG] pgmap v779: 121 pgs: 1 active+clean+snaptrim, 120 active+clean; 164 MiB data, 1.7 GiB used, 714 GiB / 715 GiB avail; 401 KiB/s rd, 1.2 MiB/s wr, 12 op/s 2024-04-07T16:29:10.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:29:10 smithi144 bash[66654]: cluster 2024-04-07T16:29:09.045677+0000 mon.a (mon.0) 1373 : cluster [DBG] osdmap e791: 8 total, 8 up, 8 in 2024-04-07T16:29:10.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:29:10 smithi144 bash[66654]: audit 2024-04-07T16:29:09.753924+0000 mon.c (mon.2) 17 : audit [DBG] from='client.? 172.21.15.100:0/3655958196' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:29:10.382 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:29:10 smithi100 bash[74942]: cluster 2024-04-07T16:29:08.833487+0000 mgr.y (mgr.25175) 413 : cluster [DBG] pgmap v779: 121 pgs: 1 active+clean+snaptrim, 120 active+clean; 164 MiB data, 1.7 GiB used, 714 GiB / 715 GiB avail; 401 KiB/s rd, 1.2 MiB/s wr, 12 op/s 2024-04-07T16:29:10.383 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:29:10 smithi100 bash[74942]: cluster 2024-04-07T16:29:09.045677+0000 mon.a (mon.0) 1373 : cluster [DBG] osdmap e791: 8 total, 8 up, 8 in 2024-04-07T16:29:10.383 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:29:10 smithi100 bash[74942]: audit 2024-04-07T16:29:09.753924+0000 mon.c (mon.2) 17 : audit [DBG] from='client.? 172.21.15.100:0/3655958196' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:29:10.383 INFO:journalctl@ceph.osd.0.smithi100.stdout:Apr 07 16:29:10 smithi100 bash[80932]: debug 2024-04-07T16:29:10.109+0000 7f990d895700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 1c9 after trimming was started 2024-04-07T16:29:10.383 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:29:10 smithi100 bash[76202]: cluster 2024-04-07T16:29:08.833487+0000 mgr.y (mgr.25175) 413 : cluster [DBG] pgmap v779: 121 pgs: 1 active+clean+snaptrim, 120 active+clean; 164 MiB data, 1.7 GiB used, 714 GiB / 715 GiB avail; 401 KiB/s rd, 1.2 MiB/s wr, 12 op/s 2024-04-07T16:29:10.383 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:29:10 smithi100 bash[76202]: cluster 2024-04-07T16:29:09.045677+0000 mon.a (mon.0) 1373 : cluster [DBG] osdmap e791: 8 total, 8 up, 8 in 2024-04-07T16:29:10.383 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:29:10 smithi100 bash[76202]: audit 2024-04-07T16:29:09.753924+0000 mon.c (mon.2) 17 : audit [DBG] from='client.? 172.21.15.100:0/3655958196' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:29:10.384 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0702e3a8-f4f8-11ee-b649-cb9ed24678a4 -e sha1=adce92d056533f22dc1ee37e3141d098b046ec1b -- bash -c 'ceph versions | jq -e '"'"'.overall | keys'"'"' | grep $sha1' 2024-04-07T16:29:10.706 INFO:tasks.rados.rados.0.smithi100.stdout:3901: done (5 left) 2024-04-07T16:29:10.706 INFO:tasks.rados.rados.0.smithi100.stdout:3902: done (4 left) 2024-04-07T16:29:10.707 INFO:tasks.rados.rados.0.smithi100.stdout:3903: done (3 left) 2024-04-07T16:29:10.707 INFO:tasks.rados.rados.0.smithi100.stdout:3905: done (2 left) 2024-04-07T16:29:10.707 INFO:tasks.rados.rados.0.smithi100.stdout:3906: done (1 left) 2024-04-07T16:29:10.707 INFO:tasks.rados.rados.0.smithi100.stdout:3907: done (0 left) 2024-04-07T16:29:10.707 INFO:tasks.rados.rados.0.smithi100.stdout:3908: read oid 4 snap -1 2024-04-07T16:29:10.707 INFO:tasks.rados.rados.0.smithi100.stdout:3908: expect (ObjNum 1160 snap 343 seq_num 1160) 2024-04-07T16:29:10.707 INFO:tasks.rados.rados.0.smithi100.stdout:3909: delete oid 47 current snap is 370 2024-04-07T16:29:10.710 INFO:tasks.rados.rados.0.smithi100.stdout:3909: done (1 left) 2024-04-07T16:29:10.710 INFO:tasks.rados.rados.0.smithi100.stdout:3910: setattr oid 10 current snap is 370 2024-04-07T16:29:10.714 INFO:tasks.rados.rados.0.smithi100.stdout:3908: expect (ObjNum 1160 snap 343 seq_num 1160) 2024-04-07T16:29:10.799 INFO:tasks.rados.rados.0.smithi100.stdout:3908: done (1 left) 2024-04-07T16:29:10.799 INFO:tasks.rados.rados.0.smithi100.stdout:3911: rollback oid 27 current snap is 370 2024-04-07T16:29:10.799 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 27 to 335 2024-04-07T16:29:10.799 INFO:tasks.rados.rados.0.smithi100.stdout:3912: snap_create 2024-04-07T16:29:10.806 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 10 v 1853 (ObjNum 4113 snap 0 seq_num 3048699844) dirty exists 2024-04-07T16:29:10.806 INFO:tasks.rados.rados.0.smithi100.stdout:3911: finishing rollback tid 0 to smithi10066867-27 2024-04-07T16:29:10.807 INFO:tasks.rados.rados.0.smithi100.stdout:3911: finishing rollback tid 1 to smithi10066867-27 2024-04-07T16:29:10.807 INFO:tasks.rados.rados.0.smithi100.stdout:3911: finishing rollback tid 2 to smithi10066867-27 2024-04-07T16:29:10.808 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 27 v 1835 (ObjNum 1108 snap 321 seq_num 1108) dirty exists 2024-04-07T16:29:11.062 INFO:tasks.rados.rados.0.smithi100.stdout:3910: done (2 left) 2024-04-07T16:29:11.062 INFO:tasks.rados.rados.0.smithi100.stdout:3911: done (1 left) 2024-04-07T16:29:11.063 INFO:tasks.rados.rados.0.smithi100.stdout:3912: done (0 left) 2024-04-07T16:29:11.063 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 23 current snap is 371 2024-04-07T16:29:11.063 INFO:tasks.rados.rados.0.smithi100.stdout:3913: seq_num 1210 ranges {1318912=671744,1990656=401408} 2024-04-07T16:29:11.075 INFO:tasks.rados.rados.0.smithi100.stdout:3913: writing smithi10066867-23 from 1318912 to 1990656 tid 1 2024-04-07T16:29:11.078 INFO:tasks.rados.rados.0.smithi100.stdout:3913: writing smithi10066867-23 from 1990656 to 2392064 tid 2 2024-04-07T16:29:11.078 INFO:tasks.rados.rados.0.smithi100.stdout:3914: delete oid 13 current snap is 371 2024-04-07T16:29:11.089 INFO:tasks.rados.rados.0.smithi100.stdout:3913: finishing write tid 1 to smithi10066867-23 2024-04-07T16:29:11.091 INFO:tasks.rados.rados.0.smithi100.stdout:3913: finishing write tid 2 to smithi10066867-23 2024-04-07T16:29:11.094 INFO:tasks.rados.rados.0.smithi100.stdout:3913: finishing write tid 3 to smithi10066867-23 2024-04-07T16:29:11.095 INFO:tasks.rados.rados.0.smithi100.stdout:3914: done (1 left) 2024-04-07T16:29:11.095 INFO:tasks.rados.rados.0.smithi100.stdout:3915: copy_from oid 14 from oid 37 current snap is 371 2024-04-07T16:29:11.095 INFO:tasks.rados.rados.0.smithi100.stdout:3916: snap_remove snap 370 2024-04-07T16:29:11.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:29:11 smithi144 bash[66654]: cluster 2024-04-07T16:29:10.065608+0000 mon.a (mon.0) 1374 : cluster [DBG] osdmap e792: 8 total, 8 up, 8 in 2024-04-07T16:29:11.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:29:11 smithi100 bash[74942]: cluster 2024-04-07T16:29:10.065608+0000 mon.a (mon.0) 1374 : cluster [DBG] osdmap e792: 8 total, 8 up, 8 in 2024-04-07T16:29:11.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:29:11 smithi100 bash[76202]: cluster 2024-04-07T16:29:10.065608+0000 mon.a (mon.0) 1374 : cluster [DBG] osdmap e792: 8 total, 8 up, 8 in 2024-04-07T16:29:11.965 INFO:tasks.rados.rados.0.smithi100.stdout:3916: done (2 left) 2024-04-07T16:29:11.965 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 19 current snap is 371 2024-04-07T16:29:11.965 INFO:tasks.rados.rados.0.smithi100.stdout:3917: seq_num 1212 ranges {196608=704512,901120=401408} 2024-04-07T16:29:11.972 INFO:tasks.rados.rados.0.smithi100.stdout:3917: writing smithi10066867-19 from 196608 to 901120 tid 1 2024-04-07T16:29:11.976 INFO:tasks.rados.rados.0.smithi100.stdout:3917: writing smithi10066867-19 from 901120 to 1302528 tid 2 2024-04-07T16:29:11.977 INFO:tasks.rados.rados.0.smithi100.stdout:3918: snap_create 2024-04-07T16:29:11.977 INFO:tasks.rados.rados.0.smithi100.stdout:3913: finishing write tid 4 to smithi10066867-23 2024-04-07T16:29:11.977 INFO:tasks.rados.rados.0.smithi100.stdout:3913: oid 23 updating version 0 to 1990 2024-04-07T16:29:11.977 INFO:tasks.rados.rados.0.smithi100.stdout:3913: oid 23 version 1990 is already newer than 1988 2024-04-07T16:29:11.977 INFO:tasks.rados.rados.0.smithi100.stdout:3913: oid 23 version 1990 is already newer than 1989 2024-04-07T16:29:11.977 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 23 v 1990 (ObjNum 1210 snap 371 seq_num 1210) dirty exists 2024-04-07T16:29:11.977 INFO:tasks.rados.rados.0.smithi100.stdout:3913: left oid 23 (ObjNum 1210 snap 371 seq_num 1210) 2024-04-07T16:29:11.977 INFO:tasks.rados.rados.0.smithi100.stdout:3915: finishing copy_from racing read to smithi10066867-14 2024-04-07T16:29:11.977 INFO:tasks.rados.rados.0.smithi100.stdout:3915: finishing copy_from to smithi10066867-14 2024-04-07T16:29:11.977 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 14 v 1607 (ObjNum 1200 snap 366 seq_num 1200) dirty exists 2024-04-07T16:29:11.990 INFO:tasks.rados.rados.0.smithi100.stdout:3917: finishing write tid 1 to smithi10066867-19 2024-04-07T16:29:11.990 INFO:tasks.rados.rados.0.smithi100.stdout:3917: finishing write tid 2 to smithi10066867-19 2024-04-07T16:29:11.992 INFO:tasks.rados.rados.0.smithi100.stdout:3917: finishing write tid 3 to smithi10066867-19 2024-04-07T16:29:11.994 INFO:tasks.rados.rados.0.smithi100.stdout:3917: finishing write tid 4 to smithi10066867-19 2024-04-07T16:29:11.994 INFO:tasks.rados.rados.0.smithi100.stdout:3917: oid 19 updating version 0 to 1297 2024-04-07T16:29:11.995 INFO:tasks.rados.rados.0.smithi100.stdout:3917: oid 19 version 1297 is already newer than 1295 2024-04-07T16:29:11.995 INFO:tasks.rados.rados.0.smithi100.stdout:3917: oid 19 version 1297 is already newer than 1296 2024-04-07T16:29:11.995 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 19 v 1297 (ObjNum 1212 snap 371 seq_num 1212) dirty exists 2024-04-07T16:29:11.995 INFO:tasks.rados.rados.0.smithi100.stdout:3917: left oid 19 (ObjNum 1212 snap 371 seq_num 1212) 2024-04-07T16:29:12.305 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:29:12 smithi144 bash[66654]: cluster 2024-04-07T16:29:10.834235+0000 mgr.y (mgr.25175) 414 : cluster [DBG] pgmap v782: 121 pgs: 1 active+clean+snaptrim, 120 active+clean; 173 MiB data, 1.7 GiB used, 713 GiB / 715 GiB avail; 2.3 MiB/s rd, 3.5 MiB/s wr, 34 op/s 2024-04-07T16:29:12.305 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:29:12 smithi144 bash[66654]: cluster 2024-04-07T16:29:11.075908+0000 mon.a (mon.0) 1375 : cluster [DBG] osdmap e793: 8 total, 8 up, 8 in 2024-04-07T16:29:12.305 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:29:12 smithi144 bash[66654]: cluster 2024-04-07T16:29:11.977293+0000 mon.a (mon.0) 1376 : cluster [DBG] osdmap e794: 8 total, 8 up, 8 in 2024-04-07T16:29:12.439 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:29:12 smithi100 bash[74942]: cluster 2024-04-07T16:29:10.834235+0000 mgr.y (mgr.25175) 414 : cluster [DBG] pgmap v782: 121 pgs: 1 active+clean+snaptrim, 120 active+clean; 173 MiB data, 1.7 GiB used, 713 GiB / 715 GiB avail; 2.3 MiB/s rd, 3.5 MiB/s wr, 34 op/s 2024-04-07T16:29:12.439 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:29:12 smithi100 bash[74942]: cluster 2024-04-07T16:29:11.075908+0000 mon.a (mon.0) 1375 : cluster [DBG] osdmap e793: 8 total, 8 up, 8 in 2024-04-07T16:29:12.439 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:29:12 smithi100 bash[74942]: cluster 2024-04-07T16:29:11.977293+0000 mon.a (mon.0) 1376 : cluster [DBG] osdmap e794: 8 total, 8 up, 8 in 2024-04-07T16:29:12.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:29:12 smithi100 bash[76202]: cluster 2024-04-07T16:29:10.834235+0000 mgr.y (mgr.25175) 414 : cluster [DBG] pgmap v782: 121 pgs: 1 active+clean+snaptrim, 120 active+clean; 173 MiB data, 1.7 GiB used, 713 GiB / 715 GiB avail; 2.3 MiB/s rd, 3.5 MiB/s wr, 34 op/s 2024-04-07T16:29:12.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:29:12 smithi100 bash[76202]: cluster 2024-04-07T16:29:11.075908+0000 mon.a (mon.0) 1375 : cluster [DBG] osdmap e793: 8 total, 8 up, 8 in 2024-04-07T16:29:12.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:29:12 smithi100 bash[76202]: cluster 2024-04-07T16:29:11.977293+0000 mon.a (mon.0) 1376 : cluster [DBG] osdmap e794: 8 total, 8 up, 8 in 2024-04-07T16:29:12.606 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:29:12 smithi144 bash[95233]: ts=2024-04-07T16:29:12.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:29:12.967 INFO:tasks.rados.rados.0.smithi100.stdout:3913: done (3 left) 2024-04-07T16:29:12.967 INFO:tasks.rados.rados.0.smithi100.stdout:3915: done (2 left) 2024-04-07T16:29:12.967 INFO:tasks.rados.rados.0.smithi100.stdout:3917: done (1 left) 2024-04-07T16:29:12.967 INFO:tasks.rados.rados.0.smithi100.stdout:3918: done (0 left) 2024-04-07T16:29:12.967 INFO:tasks.rados.rados.0.smithi100.stdout:3919: copy_from oid 7 from oid 49 current snap is 372 2024-04-07T16:29:12.967 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 37 current snap is 372 2024-04-07T16:29:12.967 INFO:tasks.rados.rados.0.smithi100.stdout:3920: seq_num 1213 ranges {8388608=720896,9109504=401408} 2024-04-07T16:29:13.066 INFO:tasks.rados.rados.0.smithi100.stdout:3920: writing smithi10066867-37 from 8388608 to 9109504 tid 1 2024-04-07T16:29:13.069 INFO:tasks.rados.rados.0.smithi100.stdout:3920: writing smithi10066867-37 from 9109504 to 9510912 tid 2 2024-04-07T16:29:13.069 INFO:tasks.rados.rados.0.smithi100.stdout:3921: copy_from oid 5 from oid 6 current snap is 372 2024-04-07T16:29:13.069 INFO:tasks.rados.rados.0.smithi100.stdout:3922: rollback oid 30 current snap is 372 2024-04-07T16:29:13.069 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 30 to 344 2024-04-07T16:29:13.069 INFO:tasks.rados.rados.0.smithi100.stdout:3923: rollback oid 44 current snap is 372 2024-04-07T16:29:13.069 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 44 to 332 2024-04-07T16:29:13.069 INFO:tasks.rados.rados.0.smithi100.stdout:3924: copy_from oid 25 from oid 18 current snap is 372 2024-04-07T16:29:13.069 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 10 current snap is 372 2024-04-07T16:29:13.070 INFO:tasks.rados.rados.0.smithi100.stdout:3925: seq_num 1214 ranges {0=737280,737280=401408} 2024-04-07T16:29:13.074 INFO:tasks.rados.rados.0.smithi100.stdout:3925: writing smithi10066867-10 from 0 to 737280 tid 1 2024-04-07T16:29:13.077 INFO:tasks.rados.rados.0.smithi100.stdout:3925: writing smithi10066867-10 from 737280 to 1138688 tid 2 2024-04-07T16:29:13.079 INFO:tasks.rados.rados.0.smithi100.stdout:3919: finishing copy_from racing read to smithi10066867-7 2024-04-07T16:29:13.079 INFO:tasks.rados.rados.0.smithi100.stdout:3919: finishing copy_from to smithi10066867-7 2024-04-07T16:29:13.079 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 7 v 1838 (ObjNum 1202 snap 369 seq_num 1202) dirty exists 2024-04-07T16:29:13.080 INFO:tasks.rados.rados.0.smithi100.stdout:3924: finishing copy_from to smithi10066867-25 2024-04-07T16:29:13.080 INFO:tasks.rados.rados.0.smithi100.stdout:3924: got expected ENOENT (src dne) 2024-04-07T16:29:13.080 INFO:tasks.rados.rados.0.smithi100.stdout:3924: finishing copy_from racing read to smithi10066867-25 2024-04-07T16:29:13.080 INFO:tasks.rados.rados.0.smithi100.stdout:3919: done (6 left) 2024-04-07T16:29:13.080 INFO:tasks.rados.rados.0.smithi100.stdout:3924: done (5 left) 2024-04-07T16:29:13.080 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 40 current snap is 372 2024-04-07T16:29:13.080 INFO:tasks.rados.rados.0.smithi100.stdout:3926: seq_num 1215 ranges {3940352=753664,4694016=401408} 2024-04-07T16:29:13.105 INFO:tasks.rados.rados.0.smithi100.stdout:3926: writing smithi10066867-40 from 3940352 to 4694016 tid 1 2024-04-07T16:29:13.108 INFO:tasks.rados.rados.0.smithi100.stdout:3926: writing smithi10066867-40 from 4694016 to 5095424 tid 2 2024-04-07T16:29:13.108 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 25 current snap is 372 2024-04-07T16:29:13.108 INFO:tasks.rados.rados.0.smithi100.stdout:3920: finishing write tid 1 to smithi10066867-37 2024-04-07T16:29:13.108 INFO:tasks.rados.rados.0.smithi100.stdout:3920: finishing write tid 2 to smithi10066867-37 2024-04-07T16:29:13.108 INFO:tasks.rados.rados.0.smithi100.stdout:3920: finishing write tid 3 to smithi10066867-37 2024-04-07T16:29:13.108 INFO:tasks.rados.rados.0.smithi100.stdout:3927: seq_num 1216 ranges {0=770048,770048=401408} 2024-04-07T16:29:13.112 INFO:tasks.rados.rados.0.smithi100.stdout:3927: writing smithi10066867-25 from 0 to 770048 tid 1 2024-04-07T16:29:13.116 INFO:tasks.rados.rados.0.smithi100.stdout:3927: writing smithi10066867-25 from 770048 to 1171456 tid 2 2024-04-07T16:29:13.116 INFO:tasks.rados.rados.0.smithi100.stdout:3920: finishing write tid 4 to smithi10066867-37 2024-04-07T16:29:13.116 INFO:tasks.rados.rados.0.smithi100.stdout:3920: oid 37 updating version 0 to 1611 2024-04-07T16:29:13.116 INFO:tasks.rados.rados.0.smithi100.stdout:3920: oid 37 updating version 1611 to 1612 2024-04-07T16:29:13.116 INFO:tasks.rados.rados.0.smithi100.stdout:3920: oid 37 updating version 1612 to 1613 2024-04-07T16:29:13.116 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 37 v 1613 (ObjNum 1213 snap 372 seq_num 1213) dirty exists 2024-04-07T16:29:13.116 INFO:tasks.rados.rados.0.smithi100.stdout:3920: left oid 37 (ObjNum 1213 snap 372 seq_num 1213) 2024-04-07T16:29:13.116 INFO:tasks.rados.rados.0.smithi100.stdout:3923: finishing rollback tid 0 to smithi10066867-44 2024-04-07T16:29:13.116 INFO:tasks.rados.rados.0.smithi100.stdout:3925: finishing write tid 1 to smithi10066867-10 2024-04-07T16:29:13.116 INFO:tasks.rados.rados.0.smithi100.stdout:3922: finishing rollback tid 0 to smithi10066867-30 2024-04-07T16:29:13.116 INFO:tasks.rados.rados.0.smithi100.stdout:3925: finishing write tid 2 to smithi10066867-10 2024-04-07T16:29:13.116 INFO:tasks.rados.rados.0.smithi100.stdout:3923: finishing rollback tid 1 to smithi10066867-44 2024-04-07T16:29:13.116 INFO:tasks.rados.rados.0.smithi100.stdout:3923: finishing rollback tid 2 to smithi10066867-44 2024-04-07T16:29:13.117 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 44 v 1686 (ObjNum 1130 snap 326 seq_num 1130) dirty exists 2024-04-07T16:29:13.117 INFO:tasks.rados.rados.0.smithi100.stdout:3925: finishing write tid 3 to smithi10066867-10 2024-04-07T16:29:13.117 INFO:tasks.rados.rados.0.smithi100.stdout:3922: finishing rollback tid 1 to smithi10066867-30 2024-04-07T16:29:13.117 INFO:tasks.rados.rados.0.smithi100.stdout:3922: finishing rollback tid 2 to smithi10066867-30 2024-04-07T16:29:13.117 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 30 v 1616 (ObjNum 1081 snap 314 seq_num 1081) dirty exists 2024-04-07T16:29:13.117 INFO:tasks.rados.rados.0.smithi100.stdout:3925: finishing write tid 4 to smithi10066867-10 2024-04-07T16:29:13.117 INFO:tasks.rados.rados.0.smithi100.stdout:3925: oid 10 updating version 0 to 1855 2024-04-07T16:29:13.117 INFO:tasks.rados.rados.0.smithi100.stdout:3925: oid 10 version 1855 is already newer than 1854 2024-04-07T16:29:13.117 INFO:tasks.rados.rados.0.smithi100.stdout:3925: oid 10 updating version 1855 to 1856 2024-04-07T16:29:13.117 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 10 v 1856 (ObjNum 1214 snap 372 seq_num 1214) dirty exists 2024-04-07T16:29:13.117 INFO:tasks.rados.rados.0.smithi100.stdout:3925: left oid 10 (ObjNum 1214 snap 372 seq_num 1214) 2024-04-07T16:29:13.117 INFO:tasks.rados.rados.0.smithi100.stdout:3921: finishing copy_from racing read to smithi10066867-5 2024-04-07T16:29:13.117 INFO:tasks.rados.rados.0.smithi100.stdout:3921: finishing copy_from to smithi10066867-5 2024-04-07T16:29:13.117 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 5 v 1668 (ObjNum 1125 snap 325 seq_num 1125) dirty exists 2024-04-07T16:29:13.117 INFO:tasks.rados.rados.0.smithi100.stdout:3926: finishing write tid 1 to smithi10066867-40 2024-04-07T16:29:13.118 INFO:tasks.rados.rados.0.smithi100.stdout:3926: finishing write tid 2 to smithi10066867-40 2024-04-07T16:29:13.118 INFO:tasks.rados.rados.0.smithi100.stdout:3926: finishing write tid 3 to smithi10066867-40 2024-04-07T16:29:13.118 INFO:tasks.rados.rados.0.smithi100.stdout:3920: done (6 left) 2024-04-07T16:29:13.118 INFO:tasks.rados.rados.0.smithi100.stdout:3921: done (5 left) 2024-04-07T16:29:13.118 INFO:tasks.rados.rados.0.smithi100.stdout:3922: done (4 left) 2024-04-07T16:29:13.118 INFO:tasks.rados.rados.0.smithi100.stdout:3923: done (3 left) 2024-04-07T16:29:13.118 INFO:tasks.rados.rados.0.smithi100.stdout:3925: done (2 left) 2024-04-07T16:29:13.118 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 2 current snap is 372 2024-04-07T16:29:13.118 INFO:tasks.rados.rados.0.smithi100.stdout:3928: seq_num 1217 ranges {0=786432,786432=401408} 2024-04-07T16:29:13.119 INFO:tasks.rados.rados.0.smithi100.stdout:3928: writing smithi10066867-2 from 0 to 786432 tid 1 2024-04-07T16:29:13.122 INFO:tasks.rados.rados.0.smithi100.stdout:3928: writing smithi10066867-2 from 786432 to 1187840 tid 2 2024-04-07T16:29:13.123 INFO:tasks.rados.rados.0.smithi100.stdout:3929: read oid 24 snap -1 2024-04-07T16:29:13.123 INFO:tasks.rados.rados.0.smithi100.stdout:3929: expect (ObjNum 1188 snap 363 seq_num 1188) 2024-04-07T16:29:13.123 INFO:tasks.rados.rados.0.smithi100.stdout:3930: rmattr oid 33 current snap is 372 2024-04-07T16:29:13.123 INFO:tasks.rados.rados.0.smithi100.stdout:3930: done (4 left) 2024-04-07T16:29:13.123 INFO:tasks.rados.rados.0.smithi100.stdout:3931: read oid 37 snap -1 2024-04-07T16:29:13.123 INFO:tasks.rados.rados.0.smithi100.stdout:3931: expect (ObjNum 1213 snap 372 seq_num 1213) 2024-04-07T16:29:13.123 INFO:tasks.rados.rados.0.smithi100.stdout:3932: setattr oid 43 current snap is 372 2024-04-07T16:29:13.123 INFO:tasks.rados.rados.0.smithi100.stdout:3927: finishing write tid 1 to smithi10066867-25 2024-04-07T16:29:13.123 INFO:tasks.rados.rados.0.smithi100.stdout:3926: finishing write tid 4 to smithi10066867-40 2024-04-07T16:29:13.123 INFO:tasks.rados.rados.0.smithi100.stdout:3926: oid 40 updating version 0 to 1689 2024-04-07T16:29:13.123 INFO:tasks.rados.rados.0.smithi100.stdout:3926: oid 40 version 1689 is already newer than 1687 2024-04-07T16:29:13.123 INFO:tasks.rados.rados.0.smithi100.stdout:3926: oid 40 version 1689 is already newer than 1688 2024-04-07T16:29:13.123 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 40 v 1689 (ObjNum 1215 snap 372 seq_num 1215) dirty exists 2024-04-07T16:29:13.123 INFO:tasks.rados.rados.0.smithi100.stdout:3926: left oid 40 (ObjNum 1215 snap 372 seq_num 1215) 2024-04-07T16:29:13.124 INFO:tasks.rados.rados.0.smithi100.stdout:3927: finishing write tid 2 to smithi10066867-25 2024-04-07T16:29:13.124 INFO:tasks.rados.rados.0.smithi100.stdout:3927: finishing write tid 3 to smithi10066867-25 2024-04-07T16:29:13.124 INFO:tasks.rados.rados.0.smithi100.stdout:3927: finishing write tid 4 to smithi10066867-25 2024-04-07T16:29:13.124 INFO:tasks.rados.rados.0.smithi100.stdout:3927: oid 25 updating version 0 to 1834 2024-04-07T16:29:13.124 INFO:tasks.rados.rados.0.smithi100.stdout:3927: oid 25 updating version 1834 to 1835 2024-04-07T16:29:13.124 INFO:tasks.rados.rados.0.smithi100.stdout:3927: oid 25 updating version 1835 to 1836 2024-04-07T16:29:13.124 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 25 v 1836 (ObjNum 1216 snap 372 seq_num 1216) dirty exists 2024-04-07T16:29:13.124 INFO:tasks.rados.rados.0.smithi100.stdout:3927: left oid 25 (ObjNum 1216 snap 372 seq_num 1216) 2024-04-07T16:29:13.124 INFO:tasks.rados.rados.0.smithi100.stdout:3926: done (5 left) 2024-04-07T16:29:13.125 INFO:tasks.rados.rados.0.smithi100.stdout:3927: done (4 left) 2024-04-07T16:29:13.125 INFO:tasks.rados.rados.0.smithi100.stdout:3933: rmattr oid 18 current snap is 372 2024-04-07T16:29:13.125 INFO:tasks.rados.rados.0.smithi100.stdout:3933: done (4 left) 2024-04-07T16:29:13.125 INFO:tasks.rados.rados.0.smithi100.stdout:3934: read oid 15 snap -1 2024-04-07T16:29:13.125 INFO:tasks.rados.rados.0.smithi100.stdout:3934: expect (ObjNum 1194 snap 366 seq_num 1194) 2024-04-07T16:29:13.125 INFO:tasks.rados.rados.0.smithi100.stdout:3935: snap_create 2024-04-07T16:29:13.128 INFO:tasks.rados.rados.0.smithi100.stdout:3928: finishing write tid 1 to smithi10066867-2 2024-04-07T16:29:13.128 INFO:tasks.rados.rados.0.smithi100.stdout:3928: finishing write tid 2 to smithi10066867-2 2024-04-07T16:29:13.128 INFO:tasks.rados.rados.0.smithi100.stdout:3928: finishing write tid 3 to smithi10066867-2 2024-04-07T16:29:13.132 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 43 v 1839 (ObjNum 1030 snap 306 seq_num 1030) dirty exists 2024-04-07T16:29:13.133 INFO:tasks.rados.rados.0.smithi100.stdout:3929: expect (ObjNum 1188 snap 363 seq_num 1188) 2024-04-07T16:29:13.176 INFO:tasks.rados.rados.0.smithi100.stdout:3928: finishing write tid 4 to smithi10066867-2 2024-04-07T16:29:13.176 INFO:tasks.rados.rados.0.smithi100.stdout:3928: oid 2 updating version 0 to 1691 2024-04-07T16:29:13.176 INFO:tasks.rados.rados.0.smithi100.stdout:3928: oid 2 updating version 1691 to 1692 2024-04-07T16:29:13.177 INFO:tasks.rados.rados.0.smithi100.stdout:3928: oid 2 version 1692 is already newer than 1690 2024-04-07T16:29:13.177 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 2 v 1692 (ObjNum 1217 snap 372 seq_num 1217) dirty exists 2024-04-07T16:29:13.177 INFO:tasks.rados.rados.0.smithi100.stdout:3928: left oid 2 (ObjNum 1217 snap 372 seq_num 1217) 2024-04-07T16:29:13.177 INFO:tasks.rados.rados.0.smithi100.stdout:3934: expect (ObjNum 1194 snap 366 seq_num 1194) 2024-04-07T16:29:13.218 INFO:tasks.rados.rados.0.smithi100.stdout:3931: expect (ObjNum 1213 snap 372 seq_num 1213) 2024-04-07T16:29:14.064 INFO:tasks.rados.rados.0.smithi100.stdout:3928: done (5 left) 2024-04-07T16:29:14.064 INFO:tasks.rados.rados.0.smithi100.stdout:3929: done (4 left) 2024-04-07T16:29:14.064 INFO:tasks.rados.rados.0.smithi100.stdout:3931: done (3 left) 2024-04-07T16:29:14.064 INFO:tasks.rados.rados.0.smithi100.stdout:3932: done (2 left) 2024-04-07T16:29:14.064 INFO:tasks.rados.rados.0.smithi100.stdout:3934: done (1 left) 2024-04-07T16:29:14.064 INFO:tasks.rados.rados.0.smithi100.stdout:3935: done (0 left) 2024-04-07T16:29:14.064 INFO:tasks.rados.rados.0.smithi100.stdout:3936: delete oid 17 current snap is 373 2024-04-07T16:29:14.072 INFO:tasks.rados.rados.0.smithi100.stdout:3936: done (0 left) 2024-04-07T16:29:14.072 INFO:tasks.rados.rados.0.smithi100.stdout:3937: rollback oid 23 current snap is 373 2024-04-07T16:29:14.072 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 23 to 344 2024-04-07T16:29:14.072 INFO:tasks.rados.rados.0.smithi100.stdout:3938: snap_remove snap 339 2024-04-07T16:29:14.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:29:13 smithi144 bash[66654]: cluster 2024-04-07T16:29:12.835404+0000 mgr.y (mgr.25175) 415 : cluster [DBG] pgmap v785: 121 pgs: 1 active+clean+snaptrim, 120 active+clean; 183 MiB data, 1.7 GiB used, 713 GiB / 715 GiB avail; 2.3 MiB/s rd, 4.9 MiB/s wr, 31 op/s 2024-04-07T16:29:14.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:29:13 smithi144 bash[66654]: cluster 2024-04-07T16:29:12.979690+0000 mon.a (mon.0) 1377 : cluster [DBG] osdmap e795: 8 total, 8 up, 8 in 2024-04-07T16:29:14.439 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:29:13 smithi100 bash[74942]: cluster 2024-04-07T16:29:12.835404+0000 mgr.y (mgr.25175) 415 : cluster [DBG] pgmap v785: 121 pgs: 1 active+clean+snaptrim, 120 active+clean; 183 MiB data, 1.7 GiB used, 713 GiB / 715 GiB avail; 2.3 MiB/s rd, 4.9 MiB/s wr, 31 op/s 2024-04-07T16:29:14.439 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:29:13 smithi100 bash[74942]: cluster 2024-04-07T16:29:12.979690+0000 mon.a (mon.0) 1377 : cluster [DBG] osdmap e795: 8 total, 8 up, 8 in 2024-04-07T16:29:14.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:29:13 smithi100 bash[76202]: cluster 2024-04-07T16:29:12.835404+0000 mgr.y (mgr.25175) 415 : cluster [DBG] pgmap v785: 121 pgs: 1 active+clean+snaptrim, 120 active+clean; 183 MiB data, 1.7 GiB used, 713 GiB / 715 GiB avail; 2.3 MiB/s rd, 4.9 MiB/s wr, 31 op/s 2024-04-07T16:29:14.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:29:13 smithi100 bash[76202]: cluster 2024-04-07T16:29:12.979690+0000 mon.a (mon.0) 1377 : cluster [DBG] osdmap e795: 8 total, 8 up, 8 in 2024-04-07T16:29:15.081 INFO:tasks.rados.rados.0.smithi100.stdout:3938: done (1 left) 2024-04-07T16:29:15.082 INFO:tasks.rados.rados.0.smithi100.stdout:3939: read oid 15 snap -1 2024-04-07T16:29:15.082 INFO:tasks.rados.rados.0.smithi100.stdout:3939: expect (ObjNum 1194 snap 366 seq_num 1194) 2024-04-07T16:29:15.082 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 16 current snap is 373 2024-04-07T16:29:15.082 INFO:tasks.rados.rados.0.smithi100.stdout:3940: seq_num 1219 ranges {638976=417792,1056768=557056,1613824=245760} 2024-04-07T16:29:15.088 INFO:tasks.rados.rados.0.smithi100.stdout:3940: writing smithi10066867-16 from 638976 to 1056768 tid 1 2024-04-07T16:29:15.091 INFO:tasks.rados.rados.0.smithi100.stdout:3940: writing smithi10066867-16 from 1056768 to 1613824 tid 2 2024-04-07T16:29:15.093 INFO:tasks.rados.rados.0.smithi100.stdout:3940: writing smithi10066867-16 from 1613824 to 1859584 tid 3 2024-04-07T16:29:15.093 INFO:tasks.rados.rados.0.smithi100.stdout:3941: snap_remove snap 369 2024-04-07T16:29:15.106 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/mon.a/config 2024-04-07T16:29:15.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:29:15 smithi144 bash[66654]: cluster 2024-04-07T16:29:14.077634+0000 mon.a (mon.0) 1378 : cluster [DBG] osdmap e796: 8 total, 8 up, 8 in 2024-04-07T16:29:15.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:29:15 smithi100 bash[74942]: cluster 2024-04-07T16:29:14.077634+0000 mon.a (mon.0) 1378 : cluster [DBG] osdmap e796: 8 total, 8 up, 8 in 2024-04-07T16:29:15.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:29:15 smithi100 bash[76202]: cluster 2024-04-07T16:29:14.077634+0000 mon.a (mon.0) 1378 : cluster [DBG] osdmap e796: 8 total, 8 up, 8 in 2024-04-07T16:29:16.077 INFO:tasks.rados.rados.0.smithi100.stdout:3941: done (3 left) 2024-04-07T16:29:16.077 INFO:tasks.rados.rados.0.smithi100.stdout:3942: read oid 24 snap -1 2024-04-07T16:29:16.077 INFO:tasks.rados.rados.0.smithi100.stdout:3942: expect (ObjNum 1188 snap 363 seq_num 1188) 2024-04-07T16:29:16.077 INFO:tasks.rados.rados.0.smithi100.stdout:3937: finishing rollback tid 0 to smithi10066867-23 2024-04-07T16:29:16.077 INFO:tasks.rados.rados.0.smithi100.stdout:3937: finishing rollback tid 1 to smithi10066867-23 2024-04-07T16:29:16.077 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 23 v 1995 (ObjNum 273 snap 0 seq_num 0) dirty dne 2024-04-07T16:29:16.077 INFO:tasks.rados.rados.0.smithi100.stdout:3939: expect (ObjNum 1194 snap 366 seq_num 1194) 2024-04-07T16:29:16.129 INFO:tasks.rados.rados.0.smithi100.stdout:3940: finishing write tid 1 to smithi10066867-16 2024-04-07T16:29:16.129 INFO:tasks.rados.rados.0.smithi100.stdout:3940: finishing write tid 2 to smithi10066867-16 2024-04-07T16:29:16.129 INFO:tasks.rados.rados.0.smithi100.stdout:3943: read oid 22 snap -1 2024-04-07T16:29:16.129 INFO:tasks.rados.rados.0.smithi100.stdout:3943: expect (ObjNum 1196 snap 366 seq_num 1196) 2024-04-07T16:29:16.129 INFO:tasks.rados.rados.0.smithi100.stdout:3937: done (4 left) 2024-04-07T16:29:16.129 INFO:tasks.rados.rados.0.smithi100.stdout:3939: done (3 left) 2024-04-07T16:29:16.129 INFO:tasks.rados.rados.0.smithi100.stdout:3944: setattr oid 47 current snap is 373 2024-04-07T16:29:16.129 INFO:tasks.rados.rados.0.smithi100.stdout:3940: finishing write tid 3 to smithi10066867-16 2024-04-07T16:29:16.129 INFO:tasks.rados.rados.0.smithi100.stdout:3940: finishing write tid 4 to smithi10066867-16 2024-04-07T16:29:16.129 INFO:tasks.rados.rados.0.smithi100.stdout:3940: finishing write tid 5 to smithi10066867-16 2024-04-07T16:29:16.129 INFO:tasks.rados.rados.0.smithi100.stdout:3940: oid 16 updating version 0 to 1842 2024-04-07T16:29:16.129 INFO:tasks.rados.rados.0.smithi100.stdout:3940: oid 16 version 1842 is already newer than 1839 2024-04-07T16:29:16.129 INFO:tasks.rados.rados.0.smithi100.stdout:3940: oid 16 version 1842 is already newer than 1841 2024-04-07T16:29:16.130 INFO:tasks.rados.rados.0.smithi100.stdout:3940: oid 16 version 1842 is already newer than 1840 2024-04-07T16:29:16.130 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 16 v 1842 (ObjNum 1219 snap 373 seq_num 1219) dirty exists 2024-04-07T16:29:16.130 INFO:tasks.rados.rados.0.smithi100.stdout:3940: left oid 16 (ObjNum 1219 snap 373 seq_num 1219) 2024-04-07T16:29:16.130 INFO:tasks.rados.rados.0.smithi100.stdout:3942: expect (ObjNum 1188 snap 363 seq_num 1188) 2024-04-07T16:29:16.169 INFO:tasks.rados.rados.0.smithi100.stdout:3943: expect (ObjNum 1196 snap 366 seq_num 1196) 2024-04-07T16:29:16.191 INFO:tasks.rados.rados.0.smithi100.stdout:3940: done (3 left) 2024-04-07T16:29:16.191 INFO:tasks.rados.rados.0.smithi100.stdout:3942: done (2 left) 2024-04-07T16:29:16.191 INFO:tasks.rados.rados.0.smithi100.stdout:3943: done (1 left) 2024-04-07T16:29:16.191 INFO:tasks.rados.rados.0.smithi100.stdout:3945: copy_from oid 3 from oid 19 current snap is 373 2024-04-07T16:29:16.191 INFO:tasks.rados.rados.0.smithi100.stdout:3946: read oid 43 snap -1 2024-04-07T16:29:16.191 INFO:tasks.rados.rados.0.smithi100.stdout:3946: expect (ObjNum 1030 snap 306 seq_num 1030) 2024-04-07T16:29:16.191 INFO:tasks.rados.rados.0.smithi100.stdout:3947: copy_from oid 2 from oid 39 current snap is 373 2024-04-07T16:29:16.191 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 4 current snap is 373 2024-04-07T16:29:16.191 INFO:tasks.rados.rados.0.smithi100.stdout:3948: seq_num 1220 ranges {868352=434176,1302528=442368,1744896=360448} 2024-04-07T16:29:16.198 INFO:tasks.rados.rados.0.smithi100.stdout:3948: writing smithi10066867-4 from 868352 to 1302528 tid 1 2024-04-07T16:29:16.201 INFO:tasks.rados.rados.0.smithi100.stdout:3948: writing smithi10066867-4 from 1302528 to 1744896 tid 2 2024-04-07T16:29:16.203 INFO:tasks.rados.rados.0.smithi100.stdout:3948: writing smithi10066867-4 from 1744896 to 2105344 tid 3 2024-04-07T16:29:16.203 INFO:tasks.rados.rados.0.smithi100.stdout:3949: snap_remove snap 333 2024-04-07T16:29:16.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:29:16 smithi144 bash[66654]: cluster 2024-04-07T16:29:14.836275+0000 mgr.y (mgr.25175) 416 : cluster [DBG] pgmap v788: 121 pgs: 121 active+clean; 189 MiB data, 1.7 GiB used, 713 GiB / 715 GiB avail; 15 MiB/s rd, 3.8 MiB/s wr, 23 op/s 2024-04-07T16:29:16.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:29:16 smithi144 bash[66654]: cluster 2024-04-07T16:29:15.087884+0000 mon.a (mon.0) 1379 : cluster [DBG] osdmap e797: 8 total, 8 up, 8 in 2024-04-07T16:29:16.367 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:29:16 smithi100 bash[74942]: cluster 2024-04-07T16:29:14.836275+0000 mgr.y (mgr.25175) 416 : cluster [DBG] pgmap v788: 121 pgs: 121 active+clean; 189 MiB data, 1.7 GiB used, 713 GiB / 715 GiB avail; 15 MiB/s rd, 3.8 MiB/s wr, 23 op/s 2024-04-07T16:29:16.367 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:29:16 smithi100 bash[74942]: cluster 2024-04-07T16:29:15.087884+0000 mon.a (mon.0) 1379 : cluster [DBG] osdmap e797: 8 total, 8 up, 8 in 2024-04-07T16:29:16.367 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:29:16 smithi100 bash[76202]: cluster 2024-04-07T16:29:14.836275+0000 mgr.y (mgr.25175) 416 : cluster [DBG] pgmap v788: 121 pgs: 121 active+clean; 189 MiB data, 1.7 GiB used, 713 GiB / 715 GiB avail; 15 MiB/s rd, 3.8 MiB/s wr, 23 op/s 2024-04-07T16:29:16.367 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:29:16 smithi100 bash[76202]: cluster 2024-04-07T16:29:15.087884+0000 mon.a (mon.0) 1379 : cluster [DBG] osdmap e797: 8 total, 8 up, 8 in 2024-04-07T16:29:16.970 INFO:tasks.rados.rados.0.smithi100.stdout:3949: done (5 left) 2024-04-07T16:29:16.970 INFO:tasks.rados.rados.0.smithi100.stdout:3950: copy_from oid 40 from oid 26 current snap is 373 2024-04-07T16:29:16.971 INFO:tasks.rados.rados.0.smithi100.stdout:3947: finishing copy_from racing read to smithi10066867-2 2024-04-07T16:29:16.971 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 47 v 1819 (ObjNum 4113 snap 0 seq_num 2702870624) dirty exists 2024-04-07T16:29:16.971 INFO:tasks.rados.rados.0.smithi100.stdout:3947: finishing copy_from to smithi10066867-2 2024-04-07T16:29:16.971 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 2 v 1608 (ObjNum 65 snap 0 seq_num 1903388502) dirty exists 2024-04-07T16:29:16.971 INFO:tasks.rados.rados.0.smithi100.stdout:3946: expect (ObjNum 1030 snap 306 seq_num 1030) 2024-04-07T16:29:17.032 INFO:tasks.rados.rados.0.smithi100.stdout:3945: finishing copy_from racing read to smithi10066867-3 2024-04-07T16:29:17.032 INFO:tasks.rados.rados.0.smithi100.stdout:3945: finishing copy_from to smithi10066867-3 2024-04-07T16:29:17.032 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 3 v 1297 (ObjNum 1212 snap 371 seq_num 1212) dirty exists 2024-04-07T16:29:17.032 INFO:tasks.rados.rados.0.smithi100.stdout:3948: finishing write tid 1 to smithi10066867-4 2024-04-07T16:29:17.032 INFO:tasks.rados.rados.0.smithi100.stdout:3948: finishing write tid 2 to smithi10066867-4 2024-04-07T16:29:17.032 INFO:tasks.rados.rados.0.smithi100.stdout:3948: finishing write tid 3 to smithi10066867-4 2024-04-07T16:29:17.032 INFO:tasks.rados.rados.0.smithi100.stdout:3944: done (5 left) 2024-04-07T16:29:17.032 INFO:tasks.rados.rados.0.smithi100.stdout:3945: done (4 left) 2024-04-07T16:29:17.032 INFO:tasks.rados.rados.0.smithi100.stdout:3946: done (3 left) 2024-04-07T16:29:17.032 INFO:tasks.rados.rados.0.smithi100.stdout:3947: done (2 left) 2024-04-07T16:29:17.033 INFO:tasks.rados.rados.0.smithi100.stdout:3951: read oid 38 snap -1 2024-04-07T16:29:17.033 INFO:tasks.rados.rados.0.smithi100.stdout:3951: expect (ObjNum 1134 snap 326 seq_num 1134) 2024-04-07T16:29:17.033 INFO:tasks.rados.rados.0.smithi100.stdout:3952: copy_from oid 35 from oid 14 current snap is 373 2024-04-07T16:29:17.033 INFO:tasks.rados.rados.0.smithi100.stdout:3948: finishing write tid 4 to smithi10066867-4 2024-04-07T16:29:17.033 INFO:tasks.rados.rados.0.smithi100.stdout:3948: finishing write tid 5 to smithi10066867-4 2024-04-07T16:29:17.033 INFO:tasks.rados.rados.0.smithi100.stdout:3948: oid 4 updating version 0 to 2010 2024-04-07T16:29:17.033 INFO:tasks.rados.rados.0.smithi100.stdout:3948: oid 4 version 2010 is already newer than 2007 2024-04-07T16:29:17.033 INFO:tasks.rados.rados.0.smithi100.stdout:3948: oid 4 version 2010 is already newer than 2008 2024-04-07T16:29:17.033 INFO:tasks.rados.rados.0.smithi100.stdout:3948: oid 4 version 2010 is already newer than 2009 2024-04-07T16:29:17.033 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 4 v 2010 (ObjNum 1220 snap 373 seq_num 1220) dirty exists 2024-04-07T16:29:17.033 INFO:tasks.rados.rados.0.smithi100.stdout:3948: left oid 4 (ObjNum 1220 snap 373 seq_num 1220) 2024-04-07T16:29:17.033 INFO:tasks.rados.rados.0.smithi100.stdout:3950: finishing copy_from racing read to smithi10066867-40 2024-04-07T16:29:17.033 INFO:tasks.rados.rados.0.smithi100.stdout:3950: finishing copy_from to smithi10066867-40 2024-04-07T16:29:17.033 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 40 v 1741 (ObjNum 1187 snap 362 seq_num 1187) dirty exists 2024-04-07T16:29:17.033 INFO:tasks.rados.rados.0.smithi100.stdout:3948: done (3 left) 2024-04-07T16:29:17.034 INFO:tasks.rados.rados.0.smithi100.stdout:3950: done (2 left) 2024-04-07T16:29:17.034 INFO:tasks.rados.rados.0.smithi100.stdout:3953: rmattr oid 30 current snap is 373 2024-04-07T16:29:17.034 INFO:tasks.rados.rados.0.smithi100.stdout:3954: read oid 1 snap -1 2024-04-07T16:29:17.034 INFO:tasks.rados.rados.0.smithi100.stdout:3954: expect deleted 2024-04-07T16:29:17.034 INFO:tasks.rados.rados.0.smithi100.stdout:3955: read oid 13 snap -1 2024-04-07T16:29:17.034 INFO:tasks.rados.rados.0.smithi100.stdout:3955: expect deleted 2024-04-07T16:29:17.034 INFO:tasks.rados.rados.0.smithi100.stdout:3956: rollback oid 7 current snap is 373 2024-04-07T16:29:17.034 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 7 to 351 2024-04-07T16:29:17.034 INFO:tasks.rados.rados.0.smithi100.stdout:3957: rmattr oid 28 current snap is 373 2024-04-07T16:29:17.034 INFO:tasks.rados.rados.0.smithi100.stdout:3958: read oid 39 snap -1 2024-04-07T16:29:17.034 INFO:tasks.rados.rados.0.smithi100.stdout:3958: expect (ObjNum 0 snap 0 seq_num 0) 2024-04-07T16:29:17.034 INFO:tasks.rados.rados.0.smithi100.stdout:3959: snap_remove snap 342 2024-04-07T16:29:17.188 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:29:17 smithi100 bash[74942]: cluster 2024-04-07T16:29:16.089226+0000 mon.a (mon.0) 1380 : cluster [DBG] osdmap e798: 8 total, 8 up, 8 in 2024-04-07T16:29:17.189 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:29:17 smithi100 bash[74942]: audit 2024-04-07T16:29:16.737038+0000 mon.a (mon.0) 1381 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:29:17.189 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:29:17 smithi100 bash[74942]: cluster 2024-04-07T16:29:16.983874+0000 mon.a (mon.0) 1382 : cluster [DBG] osdmap e799: 8 total, 8 up, 8 in 2024-04-07T16:29:17.189 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:29:17 smithi100 bash[76202]: cluster 2024-04-07T16:29:16.089226+0000 mon.a (mon.0) 1380 : cluster [DBG] osdmap e798: 8 total, 8 up, 8 in 2024-04-07T16:29:17.189 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:29:17 smithi100 bash[76202]: audit 2024-04-07T16:29:16.737038+0000 mon.a (mon.0) 1381 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:29:17.189 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:29:17 smithi100 bash[76202]: cluster 2024-04-07T16:29:16.983874+0000 mon.a (mon.0) 1382 : cluster [DBG] osdmap e799: 8 total, 8 up, 8 in 2024-04-07T16:29:17.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:29:17 smithi144 bash[66654]: cluster 2024-04-07T16:29:16.089226+0000 mon.a (mon.0) 1380 : cluster [DBG] osdmap e798: 8 total, 8 up, 8 in 2024-04-07T16:29:17.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:29:17 smithi144 bash[66654]: audit 2024-04-07T16:29:16.737038+0000 mon.a (mon.0) 1381 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:29:17.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:29:17 smithi144 bash[66654]: cluster 2024-04-07T16:29:16.983874+0000 mon.a (mon.0) 1382 : cluster [DBG] osdmap e799: 8 total, 8 up, 8 in 2024-04-07T16:29:17.981 INFO:tasks.rados.rados.0.smithi100.stdout:3959: done (8 left) 2024-04-07T16:29:17.981 INFO:tasks.rados.rados.0.smithi100.stdout:3960: setattr oid 6 current snap is 373 2024-04-07T16:29:17.981 INFO:tasks.rados.rados.0.smithi100.stdout:3961: delete oid 34 current snap is 373 2024-04-07T16:29:17.981 INFO:tasks.rados.rados.0.smithi100.stdout:3958: expect (ObjNum 0 snap 0 seq_num 0) 2024-04-07T16:29:17.982 INFO:tasks.rados.rados.0.smithi100.stdout:3956: finishing rollback tid 0 to smithi10066867-7 2024-04-07T16:29:17.983 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 30 v 1617 (ObjNum 1081 snap 314 seq_num 1081) dirty exists 2024-04-07T16:29:17.983 INFO:tasks.rados.rados.0.smithi100.stdout:3956: finishing rollback tid 1 to smithi10066867-7 2024-04-07T16:29:17.983 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 7 v 1841 (ObjNum 33 snap 0 seq_num 4237004915) dirty dne 2024-04-07T16:29:17.983 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 28 v 1971 (ObjNum 161 snap 0 seq_num 0) dirty exists 2024-04-07T16:29:17.983 INFO:tasks.rados.rados.0.smithi100.stdout:3951: expect (ObjNum 1134 snap 326 seq_num 1134) 2024-04-07T16:29:18.133 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph version 19.0.0-2785-gadce92d0 (adce92d056533f22dc1ee37e3141d098b046ec1b) squid (dev)" 2024-04-07T16:29:18.340 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:29:18 smithi100 bash[74942]: cluster 2024-04-07T16:29:16.836946+0000 mgr.y (mgr.25175) 417 : cluster [DBG] pgmap v791: 121 pgs: 121 active+clean; 191 MiB data, 1.8 GiB used, 713 GiB / 715 GiB avail; 14 MiB/s rd, 1.6 MiB/s wr, 19 op/s 2024-04-07T16:29:18.341 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:29:18 smithi100 bash[74942]: cluster 2024-04-07T16:29:17.985378+0000 mon.a (mon.0) 1383 : cluster [DBG] osdmap e800: 8 total, 8 up, 8 in 2024-04-07T16:29:18.341 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:29:18 smithi100 bash[76202]: cluster 2024-04-07T16:29:16.836946+0000 mgr.y (mgr.25175) 417 : cluster [DBG] pgmap v791: 121 pgs: 121 active+clean; 191 MiB data, 1.8 GiB used, 713 GiB / 715 GiB avail; 14 MiB/s rd, 1.6 MiB/s wr, 19 op/s 2024-04-07T16:29:18.341 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:29:18 smithi100 bash[76202]: cluster 2024-04-07T16:29:17.985378+0000 mon.a (mon.0) 1383 : cluster [DBG] osdmap e800: 8 total, 8 up, 8 in 2024-04-07T16:29:18.348 INFO:tasks.rados.rados.0.smithi100.stdout:3952: finishing copy_from racing read to smithi10066867-35 2024-04-07T16:29:18.348 INFO:tasks.rados.rados.0.smithi100.stdout:3952: finishing copy_from to smithi10066867-35 2024-04-07T16:29:18.348 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 35 v 1607 (ObjNum 1200 snap 366 seq_num 1200) dirty exists 2024-04-07T16:29:18.348 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 6 v 1742 (ObjNum 1125 snap 325 seq_num 1125) dirty exists 2024-04-07T16:29:18.348 INFO:tasks.rados.rados.0.smithi100.stdout:3951: done (9 left) 2024-04-07T16:29:18.348 INFO:tasks.rados.rados.0.smithi100.stdout:3952: done (8 left) 2024-04-07T16:29:18.348 INFO:tasks.rados.rados.0.smithi100.stdout:3953: done (7 left) 2024-04-07T16:29:18.348 INFO:tasks.rados.rados.0.smithi100.stdout:3954: done (6 left) 2024-04-07T16:29:18.348 INFO:tasks.rados.rados.0.smithi100.stdout:3955: done (5 left) 2024-04-07T16:29:18.348 INFO:tasks.rados.rados.0.smithi100.stdout:3956: done (4 left) 2024-04-07T16:29:18.348 INFO:tasks.rados.rados.0.smithi100.stdout:3957: done (3 left) 2024-04-07T16:29:18.348 INFO:tasks.rados.rados.0.smithi100.stdout:3958: done (2 left) 2024-04-07T16:29:18.348 INFO:tasks.rados.rados.0.smithi100.stdout:3960: done (1 left) 2024-04-07T16:29:18.349 INFO:tasks.rados.rados.0.smithi100.stdout:3961: done (0 left) 2024-04-07T16:29:18.349 INFO:tasks.rados.rados.0.smithi100.stdout:3962: read oid 23 snap -1 2024-04-07T16:29:18.349 INFO:tasks.rados.rados.0.smithi100.stdout:3962: expect deleted 2024-04-07T16:29:18.349 INFO:tasks.rados.rados.0.smithi100.stdout:3963: rollback oid 39 current snap is 373 2024-04-07T16:29:18.349 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 39 to 345 2024-04-07T16:29:18.349 INFO:tasks.rados.rados.0.smithi100.stdout:3964: read oid 29 snap -1 2024-04-07T16:29:18.349 INFO:tasks.rados.rados.0.smithi100.stdout:3964: expect (ObjNum 1149 snap 339 seq_num 1149) 2024-04-07T16:29:18.349 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 5 current snap is 373 2024-04-07T16:29:18.349 INFO:tasks.rados.rados.0.smithi100.stdout:3965: seq_num 1222 ranges {2056192=475136,2531328=606208,3137536=196608} 2024-04-07T16:29:18.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:29:18 smithi144 bash[66654]: cluster 2024-04-07T16:29:16.836946+0000 mgr.y (mgr.25175) 417 : cluster [DBG] pgmap v791: 121 pgs: 121 active+clean; 191 MiB data, 1.8 GiB used, 713 GiB / 715 GiB avail; 14 MiB/s rd, 1.6 MiB/s wr, 19 op/s 2024-04-07T16:29:18.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:29:18 smithi144 bash[66654]: cluster 2024-04-07T16:29:17.985378+0000 mon.a (mon.0) 1383 : cluster [DBG] osdmap e800: 8 total, 8 up, 8 in 2024-04-07T16:29:18.389 INFO:tasks.rados.rados.0.smithi100.stdout:3965: writing smithi10066867-5 from 2056192 to 2531328 tid 1 2024-04-07T16:29:18.402 INFO:tasks.rados.rados.0.smithi100.stdout:3965: writing smithi10066867-5 from 2531328 to 3137536 tid 2 2024-04-07T16:29:18.402 INFO:tasks.rados.rados.0.smithi100.stdout:3965: writing smithi10066867-5 from 3137536 to 3334144 tid 3 2024-04-07T16:29:18.402 INFO:tasks.rados.rados.0.smithi100.stdout:3966: rollback oid 10 current snap is 373 2024-04-07T16:29:18.402 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 10 to 368 2024-04-07T16:29:18.402 INFO:tasks.rados.rados.0.smithi100.stdout:3963: finishing rollback tid 0 to smithi10066867-39 2024-04-07T16:29:18.402 INFO:tasks.rados.rados.0.smithi100.stdout:3963: finishing rollback tid 1 to smithi10066867-39 2024-04-07T16:29:18.402 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 39 v 1619 (ObjNum 97 snap 0 seq_num 0) dirty dne 2024-04-07T16:29:18.402 INFO:tasks.rados.rados.0.smithi100.stdout:3964: expect (ObjNum 1149 snap 339 seq_num 1149) 2024-04-07T16:29:18.506 INFO:tasks.rados.rados.0.smithi100.stdout:3965: finishing write tid 1 to smithi10066867-5 2024-04-07T16:29:18.507 INFO:tasks.rados.rados.0.smithi100.stdout:3965: finishing write tid 2 to smithi10066867-5 2024-04-07T16:29:18.507 INFO:tasks.rados.rados.0.smithi100.stdout:3966: finishing rollback tid 0 to smithi10066867-10 2024-04-07T16:29:18.507 INFO:tasks.rados.rados.0.smithi100.stdout:3966: finishing rollback tid 1 to smithi10066867-10 2024-04-07T16:29:18.507 INFO:tasks.rados.rados.0.smithi100.stdout:3962: done (4 left) 2024-04-07T16:29:18.507 INFO:tasks.rados.rados.0.smithi100.stdout:3963: done (3 left) 2024-04-07T16:29:18.507 INFO:tasks.rados.rados.0.smithi100.stdout:3964: done (2 left) 2024-04-07T16:29:18.507 INFO:tasks.rados.rados.0.smithi100.stdout:3967: copy_from oid 3 from oid 9 current snap is 373 2024-04-07T16:29:18.507 INFO:tasks.rados.rados.0.smithi100.stdout:3965: finishing write tid 3 to smithi10066867-5 2024-04-07T16:29:18.507 INFO:tasks.rados.rados.0.smithi100.stdout:3966: finishing rollback tid 2 to smithi10066867-10 2024-04-07T16:29:18.507 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 10 v 1859 (ObjNum 1197 snap 366 seq_num 1197) dirty exists 2024-04-07T16:29:18.507 INFO:tasks.rados.rados.0.smithi100.stdout:3965: finishing write tid 4 to smithi10066867-5 2024-04-07T16:29:18.507 INFO:tasks.rados.rados.0.smithi100.stdout:3965: finishing write tid 5 to smithi10066867-5 2024-04-07T16:29:18.507 INFO:tasks.rados.rados.0.smithi100.stdout:3965: oid 5 updating version 0 to 1844 2024-04-07T16:29:18.507 INFO:tasks.rados.rados.0.smithi100.stdout:3965: oid 5 updating version 1844 to 1845 2024-04-07T16:29:18.508 INFO:tasks.rados.rados.0.smithi100.stdout:3965: oid 5 version 1845 is already newer than 1843 2024-04-07T16:29:18.508 INFO:tasks.rados.rados.0.smithi100.stdout:3965: oid 5 updating version 1845 to 1846 2024-04-07T16:29:18.508 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 5 v 1846 (ObjNum 1222 snap 373 seq_num 1222) dirty exists 2024-04-07T16:29:18.508 INFO:tasks.rados.rados.0.smithi100.stdout:3965: left oid 5 (ObjNum 1222 snap 373 seq_num 1222) 2024-04-07T16:29:18.508 INFO:tasks.rados.rados.0.smithi100.stdout:3965: done (2 left) 2024-04-07T16:29:18.508 INFO:tasks.rados.rados.0.smithi100.stdout:3966: done (1 left) 2024-04-07T16:29:18.508 INFO:tasks.rados.rados.0.smithi100.stdout:3968: read oid 42 snap -1 2024-04-07T16:29:18.508 INFO:tasks.rados.rados.0.smithi100.stdout:3968: expect (ObjNum 1195 snap 366 seq_num 1195) 2024-04-07T16:29:18.508 INFO:tasks.rados.rados.0.smithi100.stdout:3969: copy_from oid 49 from oid 17 current snap is 373 2024-04-07T16:29:18.508 INFO:tasks.rados.rados.0.smithi100.stdout:3970: copy_from oid 4 from oid 41 current snap is 373 2024-04-07T16:29:18.508 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 40 current snap is 373 2024-04-07T16:29:18.508 INFO:tasks.rados.rados.0.smithi100.stdout:3971: seq_num 1223 ranges {688128=491520,1179648=491520,1671168=311296} 2024-04-07T16:29:18.522 INFO:tasks.rados.rados.0.smithi100.stdout:3971: writing smithi10066867-40 from 688128 to 1179648 tid 1 2024-04-07T16:29:18.530 INFO:tasks.rados.rados.0.smithi100.stdout:3971: writing smithi10066867-40 from 1179648 to 1671168 tid 2 2024-04-07T16:29:18.535 INFO:tasks.rados.rados.0.smithi100.stdout:3971: writing smithi10066867-40 from 1671168 to 1982464 tid 3 2024-04-07T16:29:18.535 INFO:tasks.rados.rados.0.smithi100.stdout:3972: read oid 33 snap -1 2024-04-07T16:29:18.535 INFO:tasks.rados.rados.0.smithi100.stdout:3972: expect (ObjNum 1132 snap 326 seq_num 1132) 2024-04-07T16:29:18.535 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 23 current snap is 373 2024-04-07T16:29:18.535 INFO:tasks.rados.rados.0.smithi100.stdout:3969: finishing copy_from to smithi10066867-49 2024-04-07T16:29:18.535 INFO:tasks.rados.rados.0.smithi100.stdout:3969: got expected ENOENT (src dne) 2024-04-07T16:29:18.535 INFO:tasks.rados.rados.0.smithi100.stdout:3969: finishing copy_from racing read to smithi10066867-49 2024-04-07T16:29:18.535 INFO:tasks.rados.rados.0.smithi100.stdout:3967: finishing copy_from racing read to smithi10066867-3 2024-04-07T16:29:18.535 INFO:tasks.rados.rados.0.smithi100.stdout:3968: expect (ObjNum 1195 snap 366 seq_num 1195) 2024-04-07T16:29:18.558 INFO:tasks.rados.rados.0.smithi100.stdout:3967: finishing copy_from to smithi10066867-3 2024-04-07T16:29:18.558 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 3 v 1221 (ObjNum 1153 snap 340 seq_num 1153) dirty exists 2024-04-07T16:29:18.559 INFO:tasks.rados.rados.0.smithi100.stdout:3970: finishing copy_from racing read to smithi10066867-4 2024-04-07T16:29:18.559 INFO:tasks.rados.rados.0.smithi100.stdout:3970: finishing copy_from to smithi10066867-4 2024-04-07T16:29:18.559 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 4 v 1835 (ObjNum 1201 snap 369 seq_num 1201) dirty exists 2024-04-07T16:29:18.559 INFO:tasks.rados.rados.0.smithi100.stdout:3971: finishing write tid 1 to smithi10066867-40 2024-04-07T16:29:18.559 INFO:tasks.rados.rados.0.smithi100.stdout:3971: finishing write tid 2 to smithi10066867-40 2024-04-07T16:29:18.559 INFO:tasks.rados.rados.0.smithi100.stdout:3971: finishing write tid 3 to smithi10066867-40 2024-04-07T16:29:18.559 INFO:tasks.rados.rados.0.smithi100.stdout:3971: finishing write tid 4 to smithi10066867-40 2024-04-07T16:29:18.559 INFO:tasks.rados.rados.0.smithi100.stdout:3972: expect (ObjNum 1132 snap 326 seq_num 1132) 2024-04-07T16:29:18.613 INFO:tasks.rados.rados.0.smithi100.stdout:3971: finishing write tid 5 to smithi10066867-40 2024-04-07T16:29:18.613 INFO:tasks.rados.rados.0.smithi100.stdout:3971: oid 40 updating version 0 to 1746 2024-04-07T16:29:18.613 INFO:tasks.rados.rados.0.smithi100.stdout:3971: oid 40 version 1746 is already newer than 1744 2024-04-07T16:29:18.613 INFO:tasks.rados.rados.0.smithi100.stdout:3971: oid 40 version 1746 is already newer than 1745 2024-04-07T16:29:18.613 INFO:tasks.rados.rados.0.smithi100.stdout:3971: oid 40 version 1746 is already newer than 1743 2024-04-07T16:29:18.613 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 40 v 1746 (ObjNum 1223 snap 373 seq_num 1223) dirty exists 2024-04-07T16:29:18.613 INFO:tasks.rados.rados.0.smithi100.stdout:3971: left oid 40 (ObjNum 1223 snap 373 seq_num 1223) 2024-04-07T16:29:18.613 INFO:tasks.rados.rados.0.smithi100.stdout:3973: seq_num 1224 ranges {0=507904,507904=425984,933888=376832} 2024-04-07T16:29:18.621 INFO:tasks.rados.rados.0.smithi100.stdout:3973: writing smithi10066867-23 from 0 to 507904 tid 1 2024-04-07T16:29:18.628 INFO:tasks.rados.rados.0.smithi100.stdout:3973: writing smithi10066867-23 from 507904 to 933888 tid 2 2024-04-07T16:29:18.635 INFO:tasks.rados.rados.0.smithi100.stdout:3973: writing smithi10066867-23 from 933888 to 1310720 tid 3 2024-04-07T16:29:18.635 INFO:tasks.rados.rados.0.smithi100.stdout:3967: done (6 left) 2024-04-07T16:29:18.635 INFO:tasks.rados.rados.0.smithi100.stdout:3968: done (5 left) 2024-04-07T16:29:18.635 INFO:tasks.rados.rados.0.smithi100.stdout:3969: done (4 left) 2024-04-07T16:29:18.636 INFO:tasks.rados.rados.0.smithi100.stdout:3970: done (3 left) 2024-04-07T16:29:18.636 INFO:tasks.rados.rados.0.smithi100.stdout:3971: done (2 left) 2024-04-07T16:29:18.636 INFO:tasks.rados.rados.0.smithi100.stdout:3972: done (1 left) 2024-04-07T16:29:18.636 INFO:tasks.rados.rados.0.smithi100.stdout:3974: copy_from oid 7 from oid 30 current snap is 373 2024-04-07T16:29:18.636 INFO:tasks.rados.rados.0.smithi100.stdout:3973: finishing write tid 1 to smithi10066867-23 2024-04-07T16:29:18.636 INFO:tasks.rados.rados.0.smithi100.stdout:3973: finishing write tid 2 to smithi10066867-23 2024-04-07T16:29:18.636 INFO:tasks.rados.rados.0.smithi100.stdout:3975: copy_from oid 2 from oid 20 current snap is 373 2024-04-07T16:29:18.636 INFO:tasks.rados.rados.0.smithi100.stdout:3976: snap_create 2024-04-07T16:29:18.638 INFO:tasks.rados.rados.0.smithi100.stdout:3975: finishing copy_from racing read to smithi10066867-2 2024-04-07T16:29:18.641 INFO:tasks.rados.rados.0.smithi100.stdout:3975: finishing copy_from to smithi10066867-2 2024-04-07T16:29:18.641 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 2 v 1830 (ObjNum 65 snap 0 seq_num 1632916340) dirty exists 2024-04-07T16:29:18.641 INFO:tasks.rados.rados.0.smithi100.stdout:3973: finishing write tid 3 to smithi10066867-23 2024-04-07T16:29:18.642 INFO:tasks.rados.rados.0.smithi100.stdout:3973: finishing write tid 4 to smithi10066867-23 2024-04-07T16:29:18.644 INFO:tasks.rados.rados.0.smithi100.stdout:3973: finishing write tid 5 to smithi10066867-23 2024-04-07T16:29:18.644 INFO:tasks.rados.rados.0.smithi100.stdout:3973: oid 23 updating version 0 to 2031 2024-04-07T16:29:18.645 INFO:tasks.rados.rados.0.smithi100.stdout:3973: oid 23 version 2031 is already newer than 2029 2024-04-07T16:29:18.645 INFO:tasks.rados.rados.0.smithi100.stdout:3973: oid 23 version 2031 is already newer than 2030 2024-04-07T16:29:18.645 INFO:tasks.rados.rados.0.smithi100.stdout:3973: oid 23 version 2031 is already newer than 2028 2024-04-07T16:29:18.645 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 23 v 2031 (ObjNum 1224 snap 373 seq_num 1224) dirty exists 2024-04-07T16:29:18.645 INFO:tasks.rados.rados.0.smithi100.stdout:3973: left oid 23 (ObjNum 1224 snap 373 seq_num 1224) 2024-04-07T16:29:18.657 INFO:tasks.rados.rados.0.smithi100.stdout:3974: finishing copy_from racing read to smithi10066867-7 2024-04-07T16:29:18.661 INFO:tasks.rados.rados.0.smithi100.stdout:3974: finishing copy_from to smithi10066867-7 2024-04-07T16:29:18.661 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 7 v 1617 (ObjNum 1081 snap 314 seq_num 1081) dirty exists 2024-04-07T16:29:18.688 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:29:18 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:29:18] "GET /metrics HTTP/1.1" 200 36343 "" "Prometheus/2.43.0" 2024-04-07T16:29:18.882 INFO:teuthology.task.sequential:In sequential, running task print... 2024-04-07T16:29:18.882 INFO:teuthology.task.print:**** done end upgrade, wait... 2024-04-07T16:29:18.883 DEBUG:teuthology.parallel:result is None 2024-04-07T16:29:19.101 INFO:tasks.rados.rados.0.smithi100.stdout:3973: done (3 left) 2024-04-07T16:29:19.102 INFO:tasks.rados.rados.0.smithi100.stdout:3974: done (2 left) 2024-04-07T16:29:19.102 INFO:tasks.rados.rados.0.smithi100.stdout:3975: done (1 left) 2024-04-07T16:29:19.102 INFO:tasks.rados.rados.0.smithi100.stdout:3976: done (0 left) 2024-04-07T16:29:19.102 INFO:tasks.rados.rados.0.smithi100.stdout:3977: rollback oid 36 current snap is 374 2024-04-07T16:29:19.102 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 36 to 346 2024-04-07T16:29:19.102 INFO:tasks.rados.rados.0.smithi100.stdout:3978: copy_from oid 39 from oid 26 current snap is 374 2024-04-07T16:29:19.103 INFO:tasks.rados.rados.0.smithi100.stdout:3979: rmattr oid 20 current snap is 374 2024-04-07T16:29:19.103 INFO:tasks.rados.rados.0.smithi100.stdout:3980: read oid 5 snap 350 2024-04-07T16:29:19.103 INFO:tasks.rados.rados.0.smithi100.stdout:3980: expect (ObjNum 1143 snap 335 seq_num 1143) 2024-04-07T16:29:19.103 INFO:tasks.rados.rados.0.smithi100.stdout:3981: setattr oid 43 current snap is 374 2024-04-07T16:29:19.108 INFO:tasks.rados.rados.0.smithi100.stdout:3982: copy_from oid 9 from oid 34 current snap is 374 2024-04-07T16:29:19.109 INFO:tasks.rados.rados.0.smithi100.stdout:3983: rmattr oid 23 current snap is 374 2024-04-07T16:29:19.109 INFO:tasks.rados.rados.0.smithi100.stdout:3983: done (6 left) 2024-04-07T16:29:19.109 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 45 current snap is 374 2024-04-07T16:29:19.109 INFO:tasks.rados.rados.0.smithi100.stdout:3984: seq_num 1225 ranges {638976=524288,1163264=712704,1875968=90112} 2024-04-07T16:29:19.130 INFO:tasks.rados.rados.0.smithi100.stdout:3984: writing smithi10066867-45 from 638976 to 1163264 tid 1 2024-04-07T16:29:19.145 INFO:tasks.rados.rados.0.smithi100.stdout:3984: writing smithi10066867-45 from 1163264 to 1875968 tid 2 2024-04-07T16:29:19.146 INFO:tasks.rados.rados.0.smithi100.stdout:3984: writing smithi10066867-45 from 1875968 to 1966080 tid 3 2024-04-07T16:29:19.146 INFO:tasks.rados.rados.0.smithi100.stdout:3985: copy_from oid 10 from oid 37 current snap is 374 2024-04-07T16:29:19.146 INFO:tasks.rados.rados.0.smithi100.stdout:3982: finishing copy_from to smithi10066867-9 2024-04-07T16:29:19.146 INFO:tasks.rados.rados.0.smithi100.stdout:3982: got expected ENOENT (src dne) 2024-04-07T16:29:19.147 INFO:tasks.rados.rados.0.smithi100.stdout:3982: finishing copy_from racing read to smithi10066867-9 2024-04-07T16:29:19.147 INFO:tasks.rados.rados.0.smithi100.stdout:3977: finishing rollback tid 1 to smithi10066867-36 2024-04-07T16:29:19.147 INFO:tasks.rados.rados.0.smithi100.stdout:3977: finishing rollback tid 2 to smithi10066867-36 2024-04-07T16:29:19.147 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 36 v 1299 (ObjNum 1121 snap 322 seq_num 1121) dirty exists 2024-04-07T16:29:19.147 INFO:tasks.rados.rados.0.smithi100.stdout:3978: finishing copy_from racing read to smithi10066867-39 2024-04-07T16:29:19.147 INFO:tasks.rados.rados.0.smithi100.stdout:3978: finishing copy_from to smithi10066867-39 2024-04-07T16:29:19.147 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 39 v 1741 (ObjNum 1187 snap 362 seq_num 1187) dirty exists 2024-04-07T16:29:19.147 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 20 v 1837 (ObjNum 65 snap 0 seq_num 1666142051) dirty exists 2024-04-07T16:29:19.147 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 43 v 1842 (ObjNum 1030 snap 306 seq_num 1030) dirty exists 2024-04-07T16:29:19.147 INFO:tasks.rados.rados.0.smithi100.stdout:3980: expect (ObjNum 1143 snap 335 seq_num 1143) 2024-04-07T16:29:19.188 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:29:19 smithi100 bash[74942]: audit 2024-04-07T16:29:18.134611+0000 mon.a (mon.0) 1384 : audit [DBG] from='client.? 172.21.15.100:0/4013622992' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:29:19.188 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:29:19 smithi100 bash[76202]: audit 2024-04-07T16:29:18.134611+0000 mon.a (mon.0) 1384 : audit [DBG] from='client.? 172.21.15.100:0/4013622992' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:29:19.269 INFO:tasks.rados.rados.0.smithi100.stdout:3984: finishing write tid 1 to smithi10066867-45 2024-04-07T16:29:19.269 INFO:tasks.rados.rados.0.smithi100.stdout:3984: finishing write tid 2 to smithi10066867-45 2024-04-07T16:29:19.269 INFO:tasks.rados.rados.0.smithi100.stdout:3984: finishing write tid 3 to smithi10066867-45 2024-04-07T16:29:19.269 INFO:tasks.rados.rados.0.smithi100.stdout:3984: finishing write tid 4 to smithi10066867-45 2024-04-07T16:29:19.269 INFO:tasks.rados.rados.0.smithi100.stdout:3984: finishing write tid 5 to smithi10066867-45 2024-04-07T16:29:19.269 INFO:tasks.rados.rados.0.smithi100.stdout:3984: oid 45 updating version 0 to 1838 2024-04-07T16:29:19.269 INFO:tasks.rados.rados.0.smithi100.stdout:3984: oid 45 updating version 1838 to 1839 2024-04-07T16:29:19.269 INFO:tasks.rados.rados.0.smithi100.stdout:3984: oid 45 updating version 1839 to 1840 2024-04-07T16:29:19.270 INFO:tasks.rados.rados.0.smithi100.stdout:3984: oid 45 updating version 1840 to 1841 2024-04-07T16:29:19.270 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 45 v 1841 (ObjNum 1225 snap 374 seq_num 1225) dirty exists 2024-04-07T16:29:19.270 INFO:tasks.rados.rados.0.smithi100.stdout:3984: left oid 45 (ObjNum 1225 snap 374 seq_num 1225) 2024-04-07T16:29:19.270 INFO:tasks.rados.rados.0.smithi100.stdout:3985: finishing copy_from racing read to smithi10066867-10 2024-04-07T16:29:19.270 INFO:tasks.rados.rados.0.smithi100.stdout:3985: finishing copy_from to smithi10066867-10 2024-04-07T16:29:19.270 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 10 v 1613 (ObjNum 1213 snap 372 seq_num 1213) dirty exists 2024-04-07T16:29:19.270 INFO:tasks.rados.rados.0.smithi100.stdout:3977: done (7 left) 2024-04-07T16:29:19.270 INFO:tasks.rados.rados.0.smithi100.stdout:3978: done (6 left) 2024-04-07T16:29:19.270 INFO:tasks.rados.rados.0.smithi100.stdout:3979: done (5 left) 2024-04-07T16:29:19.270 INFO:tasks.rados.rados.0.smithi100.stdout:3980: done (4 left) 2024-04-07T16:29:19.270 INFO:tasks.rados.rados.0.smithi100.stdout:3981: done (3 left) 2024-04-07T16:29:19.270 INFO:tasks.rados.rados.0.smithi100.stdout:3982: done (2 left) 2024-04-07T16:29:19.270 INFO:tasks.rados.rados.0.smithi100.stdout:3984: done (1 left) 2024-04-07T16:29:19.270 INFO:tasks.rados.rados.0.smithi100.stdout:3985: done (0 left) 2024-04-07T16:29:19.270 INFO:tasks.rados.rados.0.smithi100.stdout:3986: setattr oid 22 current snap is 374 2024-04-07T16:29:19.274 INFO:tasks.rados.rados.0.smithi100.stdout:3987: rollback oid 49 current snap is 374 2024-04-07T16:29:19.275 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 49 to 349 2024-04-07T16:29:19.275 INFO:tasks.rados.rados.0.smithi100.stdout:3988: read oid 16 snap 355 2024-04-07T16:29:19.275 INFO:tasks.rados.rados.0.smithi100.stdout:3988: expect (ObjNum 1041 snap 307 seq_num 1041) 2024-04-07T16:29:19.275 INFO:tasks.rados.rados.0.smithi100.stdout:3989: delete oid 32 current snap is 374 2024-04-07T16:29:19.280 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 22 v 1838 (ObjNum 1196 snap 366 seq_num 1196) dirty exists 2024-04-07T16:29:19.282 INFO:tasks.rados.rados.0.smithi100.stdout:3986: done (3 left) 2024-04-07T16:29:19.282 INFO:tasks.rados.rados.0.smithi100.stdout:3989: done (2 left) 2024-04-07T16:29:19.282 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 18 current snap is 374 2024-04-07T16:29:19.282 INFO:tasks.rados.rados.0.smithi100.stdout:3990: seq_num 1227 ranges {0=557056,557056=483328,1040384=319488} 2024-04-07T16:29:19.292 INFO:tasks.rados.rados.0.smithi100.stdout:3990: writing smithi10066867-18 from 0 to 557056 tid 1 2024-04-07T16:29:19.302 INFO:tasks.rados.rados.0.smithi100.stdout:3990: writing smithi10066867-18 from 557056 to 1040384 tid 2 2024-04-07T16:29:19.308 INFO:tasks.rados.rados.0.smithi100.stdout:3990: writing smithi10066867-18 from 1040384 to 1359872 tid 3 2024-04-07T16:29:19.308 INFO:tasks.rados.rados.0.smithi100.stdout:3991: rollback oid 36 current snap is 374 2024-04-07T16:29:19.309 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 36 to 372 2024-04-07T16:29:19.309 INFO:tasks.rados.rados.0.smithi100.stdout:3987: finishing rollback tid 0 to smithi10066867-49 2024-04-07T16:29:19.309 INFO:tasks.rados.rados.0.smithi100.stdout:3987: finishing rollback tid 1 to smithi10066867-49 2024-04-07T16:29:19.309 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 49 v 1848 (ObjNum 273 snap 0 seq_num 0) dirty dne 2024-04-07T16:29:19.309 INFO:tasks.rados.rados.0.smithi100.stdout:3988: expect (ObjNum 1041 snap 307 seq_num 1041) 2024-04-07T16:29:19.331 INFO:tasks.rados.rados.0.smithi100.stdout:3990: finishing write tid 1 to smithi10066867-18 2024-04-07T16:29:19.331 INFO:tasks.rados.rados.0.smithi100.stdout:3990: finishing write tid 2 to smithi10066867-18 2024-04-07T16:29:19.331 INFO:tasks.rados.rados.0.smithi100.stdout:3990: finishing write tid 3 to smithi10066867-18 2024-04-07T16:29:19.331 INFO:tasks.rados.rados.0.smithi100.stdout:3991: finishing rollback tid 0 to smithi10066867-36 2024-04-07T16:29:19.331 INFO:tasks.rados.rados.0.smithi100.stdout:3991: finishing rollback tid 1 to smithi10066867-36 2024-04-07T16:29:19.331 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 36 v 1301 (ObjNum 161 snap 0 seq_num 1) dirty dne 2024-04-07T16:29:19.331 INFO:tasks.rados.rados.0.smithi100.stdout:3990: finishing write tid 4 to smithi10066867-18 2024-04-07T16:29:19.332 INFO:tasks.rados.rados.0.smithi100.stdout:3990: finishing write tid 5 to smithi10066867-18 2024-04-07T16:29:19.332 INFO:tasks.rados.rados.0.smithi100.stdout:3990: oid 18 updating version 0 to 2035 2024-04-07T16:29:19.332 INFO:tasks.rados.rados.0.smithi100.stdout:3990: oid 18 version 2035 is already newer than 2034 2024-04-07T16:29:19.332 INFO:tasks.rados.rados.0.smithi100.stdout:3990: oid 18 version 2035 is already newer than 2033 2024-04-07T16:29:19.332 INFO:tasks.rados.rados.0.smithi100.stdout:3990: oid 18 version 2035 is already newer than 2032 2024-04-07T16:29:19.332 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 18 v 2035 (ObjNum 1227 snap 374 seq_num 1227) dirty exists 2024-04-07T16:29:19.332 INFO:tasks.rados.rados.0.smithi100.stdout:3990: left oid 18 (ObjNum 1227 snap 374 seq_num 1227) 2024-04-07T16:29:19.332 INFO:tasks.rados.rados.0.smithi100.stdout:3987: done (3 left) 2024-04-07T16:29:19.332 INFO:tasks.rados.rados.0.smithi100.stdout:3988: done (2 left) 2024-04-07T16:29:19.332 INFO:tasks.rados.rados.0.smithi100.stdout:3990: done (1 left) 2024-04-07T16:29:19.332 INFO:tasks.rados.rados.0.smithi100.stdout:3991: done (0 left) 2024-04-07T16:29:19.332 INFO:tasks.rados.rados.0.smithi100.stdout:3992: read oid 34 snap 335 2024-04-07T16:29:19.332 INFO:tasks.rados.rados.0.smithi100.stdout:3992: expect (ObjNum 1012 snap 300 seq_num 1012) 2024-04-07T16:29:19.332 INFO:tasks.rados.rados.0.smithi100.stdout:3993: snap_create 2024-04-07T16:29:19.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:29:19 smithi144 bash[66654]: audit 2024-04-07T16:29:18.134611+0000 mon.a (mon.0) 1384 : audit [DBG] from='client.? 172.21.15.100:0/4013622992' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-07T16:29:19.369 INFO:tasks.rados.rados.0.smithi100.stdout:3992: expect (ObjNum 1012 snap 300 seq_num 1012) 2024-04-07T16:29:20.111 INFO:tasks.rados.rados.0.smithi100.stdout:3992: done (1 left) 2024-04-07T16:29:20.111 INFO:tasks.rados.rados.0.smithi100.stdout:3993: done (0 left) 2024-04-07T16:29:20.111 INFO:tasks.rados.rados.0.smithi100.stdout:3994: read oid 4 snap -1 2024-04-07T16:29:20.111 INFO:tasks.rados.rados.0.smithi100.stdout:3994: expect (ObjNum 1201 snap 369 seq_num 1201) 2024-04-07T16:29:20.112 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 2 current snap is 375 2024-04-07T16:29:20.112 INFO:tasks.rados.rados.0.smithi100.stdout:3995: seq_num 1228 ranges {0=573440,573440=761856,1335296=40960} 2024-04-07T16:29:20.116 INFO:tasks.rados.rados.0.smithi100.stdout:3995: writing smithi10066867-2 from 0 to 573440 tid 1 2024-04-07T16:29:20.123 INFO:tasks.rados.rados.0.smithi100.stdout:3995: writing smithi10066867-2 from 573440 to 1335296 tid 2 2024-04-07T16:29:20.123 INFO:tasks.rados.rados.0.smithi100.stdout:3995: writing smithi10066867-2 from 1335296 to 1376256 tid 3 2024-04-07T16:29:20.123 INFO:tasks.rados.rados.0.smithi100.stdout:3996: snap_remove snap 351 2024-04-07T16:29:20.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:29:20 smithi144 bash[66654]: cluster 2024-04-07T16:29:18.837697+0000 mgr.y (mgr.25175) 418 : cluster [DBG] pgmap v794: 121 pgs: 5 active+clean+snaptrim, 116 active+clean; 197 MiB data, 1.8 GiB used, 713 GiB / 715 GiB avail; 6.8 MiB/s rd, 1.9 MiB/s wr, 21 op/s 2024-04-07T16:29:20.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:29:20 smithi144 bash[66654]: cluster 2024-04-07T16:29:19.113438+0000 mon.a (mon.0) 1385 : cluster [DBG] osdmap e801: 8 total, 8 up, 8 in 2024-04-07T16:29:20.441 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:29:20 smithi100 bash[74942]: cluster 2024-04-07T16:29:18.837697+0000 mgr.y (mgr.25175) 418 : cluster [DBG] pgmap v794: 121 pgs: 5 active+clean+snaptrim, 116 active+clean; 197 MiB data, 1.8 GiB used, 713 GiB / 715 GiB avail; 6.8 MiB/s rd, 1.9 MiB/s wr, 21 op/s 2024-04-07T16:29:20.442 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:29:20 smithi100 bash[74942]: cluster 2024-04-07T16:29:19.113438+0000 mon.a (mon.0) 1385 : cluster [DBG] osdmap e801: 8 total, 8 up, 8 in 2024-04-07T16:29:20.442 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:29:20 smithi100 bash[76202]: cluster 2024-04-07T16:29:18.837697+0000 mgr.y (mgr.25175) 418 : cluster [DBG] pgmap v794: 121 pgs: 5 active+clean+snaptrim, 116 active+clean; 197 MiB data, 1.8 GiB used, 713 GiB / 715 GiB avail; 6.8 MiB/s rd, 1.9 MiB/s wr, 21 op/s 2024-04-07T16:29:20.442 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:29:20 smithi100 bash[76202]: cluster 2024-04-07T16:29:19.113438+0000 mon.a (mon.0) 1385 : cluster [DBG] osdmap e801: 8 total, 8 up, 8 in 2024-04-07T16:29:21.125 INFO:tasks.rados.rados.0.smithi100.stdout:3996: done (2 left) 2024-04-07T16:29:21.125 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 34 current snap is 375 2024-04-07T16:29:21.125 INFO:tasks.rados.rados.0.smithi100.stdout:3997: seq_num 1229 ranges {0=589824,589824=647168,1236992=155648} 2024-04-07T16:29:21.129 INFO:tasks.rados.rados.0.smithi100.stdout:3997: writing smithi10066867-34 from 0 to 589824 tid 1 2024-04-07T16:29:21.135 INFO:tasks.rados.rados.0.smithi100.stdout:3997: writing smithi10066867-34 from 589824 to 1236992 tid 2 2024-04-07T16:29:21.137 INFO:tasks.rados.rados.0.smithi100.stdout:3997: writing smithi10066867-34 from 1236992 to 1392640 tid 3 2024-04-07T16:29:21.137 INFO:tasks.rados.rados.0.smithi100.stdout:3998: rollback oid 3 current snap is 375 2024-04-07T16:29:21.137 INFO:tasks.rados.rados.0.smithi100.stdout:rollback oid 3 to 368 2024-04-07T16:29:21.137 INFO:tasks.rados.rados.0.smithi100.stdout:3994: expect (ObjNum 1201 snap 369 seq_num 1201) 2024-04-07T16:29:21.170 INFO:tasks.rados.rados.0.smithi100.stdout:3995: finishing write tid 1 to smithi10066867-2 2024-04-07T16:29:21.170 INFO:tasks.rados.rados.0.smithi100.stdout:3995: finishing write tid 2 to smithi10066867-2 2024-04-07T16:29:21.170 INFO:tasks.rados.rados.0.smithi100.stdout:3995: finishing write tid 3 to smithi10066867-2 2024-04-07T16:29:21.170 INFO:tasks.rados.rados.0.smithi100.stdout:3995: finishing write tid 4 to smithi10066867-2 2024-04-07T16:29:21.170 INFO:tasks.rados.rados.0.smithi100.stdout:3995: finishing write tid 5 to smithi10066867-2 2024-04-07T16:29:21.170 INFO:tasks.rados.rados.0.smithi100.stdout:3995: oid 2 updating version 0 to 1831 2024-04-07T16:29:21.170 INFO:tasks.rados.rados.0.smithi100.stdout:3995: oid 2 updating version 1831 to 1832 2024-04-07T16:29:21.170 INFO:tasks.rados.rados.0.smithi100.stdout:3995: oid 2 updating version 1832 to 1834 2024-04-07T16:29:21.170 INFO:tasks.rados.rados.0.smithi100.stdout:3995: oid 2 version 1834 is already newer than 1833 2024-04-07T16:29:21.171 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 2 v 1834 (ObjNum 1228 snap 375 seq_num 1228) dirty exists 2024-04-07T16:29:21.171 INFO:tasks.rados.rados.0.smithi100.stdout:3995: left oid 2 (ObjNum 1228 snap 375 seq_num 1228) 2024-04-07T16:29:21.171 INFO:tasks.rados.rados.0.smithi100.stdout:3998: finishing rollback tid 0 to smithi10066867-3 2024-04-07T16:29:21.171 INFO:tasks.rados.rados.0.smithi100.stdout:3994: done (3 left) 2024-04-07T16:29:21.171 INFO:tasks.rados.rados.0.smithi100.stdout:3995: done (2 left) 2024-04-07T16:29:21.171 INFO:tasks.rados.rados.0.smithi100.stdout:append oid 43 current snap is 375 2024-04-07T16:29:21.171 INFO:tasks.rados.rados.0.smithi100.stdout:3998: finishing rollback tid 1 to smithi10066867-3 2024-04-07T16:29:21.171 INFO:tasks.rados.rados.0.smithi100.stdout:3997: finishing write tid 1 to smithi10066867-34 2024-04-07T16:29:21.171 INFO:tasks.rados.rados.0.smithi100.stdout:3997: finishing write tid 2 to smithi10066867-34 2024-04-07T16:29:21.171 INFO:tasks.rados.rados.0.smithi100.stdout:3999: seq_num 1230 ranges {458752=606208,1064960=532480,1597440=270336} 2024-04-07T16:29:21.180 INFO:tasks.rados.rados.0.smithi100.stdout:3999: writing smithi10066867-43 from 458752 to 1064960 tid 1 2024-04-07T16:29:21.186 INFO:tasks.rados.rados.0.smithi100.stdout:3999: writing smithi10066867-43 from 1064960 to 1597440 tid 2 2024-04-07T16:29:21.190 INFO:tasks.rados.rados.0.smithi100.stdout:3999: writing smithi10066867-43 from 1597440 to 1867776 tid 3 2024-04-07T16:29:21.190 INFO:tasks.rados.rados.0.smithi100.stdout: waiting on 3 2024-04-07T16:29:21.190 INFO:tasks.rados.rados.0.smithi100.stdout:3997: finishing write tid 3 to smithi10066867-34 2024-04-07T16:29:21.190 INFO:tasks.rados.rados.0.smithi100.stdout:3998: finishing rollback tid 2 to smithi10066867-3 2024-04-07T16:29:21.190 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 3 v 1726 (ObjNum 1178 snap 359 seq_num 1178) dirty exists 2024-04-07T16:29:21.191 INFO:tasks.rados.rados.0.smithi100.stdout:3997: finishing write tid 4 to smithi10066867-34 2024-04-07T16:29:21.191 INFO:tasks.rados.rados.0.smithi100.stdout:3997: finishing write tid 5 to smithi10066867-34 2024-04-07T16:29:21.191 INFO:tasks.rados.rados.0.smithi100.stdout:3997: oid 34 updating version 0 to 1870 2024-04-07T16:29:21.191 INFO:tasks.rados.rados.0.smithi100.stdout:3997: oid 34 updating version 1870 to 1871 2024-04-07T16:29:21.191 INFO:tasks.rados.rados.0.smithi100.stdout:3997: oid 34 updating version 1871 to 1872 2024-04-07T16:29:21.191 INFO:tasks.rados.rados.0.smithi100.stdout:3997: oid 34 updating version 1872 to 1873 2024-04-07T16:29:21.191 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 34 v 1873 (ObjNum 1229 snap 375 seq_num 1229) dirty exists 2024-04-07T16:29:21.191 INFO:tasks.rados.rados.0.smithi100.stdout:3997: left oid 34 (ObjNum 1229 snap 375 seq_num 1229) 2024-04-07T16:29:21.191 INFO:tasks.rados.rados.0.smithi100.stdout:3999: finishing write tid 1 to smithi10066867-43 2024-04-07T16:29:21.191 INFO:tasks.rados.rados.0.smithi100.stdout:3997: done (2 left) 2024-04-07T16:29:21.191 INFO:tasks.rados.rados.0.smithi100.stdout:3998: done (1 left) 2024-04-07T16:29:21.191 INFO:tasks.rados.rados.0.smithi100.stdout: waiting on 1 2024-04-07T16:29:21.193 INFO:tasks.rados.rados.0.smithi100.stdout:3999: finishing write tid 2 to smithi10066867-43 2024-04-07T16:29:21.194 INFO:tasks.rados.rados.0.smithi100.stdout:3999: finishing write tid 3 to smithi10066867-43 2024-04-07T16:29:21.195 INFO:tasks.rados.rados.0.smithi100.stdout:3999: finishing write tid 4 to smithi10066867-43 2024-04-07T16:29:21.198 INFO:tasks.rados.rados.0.smithi100.stdout:3999: finishing write tid 5 to smithi10066867-43 2024-04-07T16:29:21.198 INFO:tasks.rados.rados.0.smithi100.stdout:3999: oid 43 updating version 0 to 1844 2024-04-07T16:29:21.198 INFO:tasks.rados.rados.0.smithi100.stdout:3999: oid 43 updating version 1844 to 1846 2024-04-07T16:29:21.198 INFO:tasks.rados.rados.0.smithi100.stdout:3999: oid 43 version 1846 is already newer than 1843 2024-04-07T16:29:21.198 INFO:tasks.rados.rados.0.smithi100.stdout:3999: oid 43 version 1846 is already newer than 1845 2024-04-07T16:29:21.198 INFO:tasks.rados.rados.0.smithi100.stdout:update_object_version oid 43 v 1846 (ObjNum 1230 snap 375 seq_num 1230) dirty exists 2024-04-07T16:29:21.198 INFO:tasks.rados.rados.0.smithi100.stdout:3999: left oid 43 (ObjNum 1230 snap 375 seq_num 1230) 2024-04-07T16:29:21.198 INFO:tasks.rados.rados.0.smithi100.stdout:3999: done (0 left) 2024-04-07T16:29:21.208 INFO:tasks.rados.rados.0.smithi100.stderr:0 errors. 2024-04-07T16:29:21.208 INFO:tasks.rados.rados.0.smithi100.stderr: 2024-04-07T16:29:21.228 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 0702e3a8-f4f8-11ee-b649-cb9ed24678a4 -- ceph osd dump --format=json 2024-04-07T16:29:21.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:29:21 smithi100 bash[74942]: cluster 2024-04-07T16:29:20.124234+0000 mon.a (mon.0) 1386 : cluster [DBG] osdmap e802: 8 total, 8 up, 8 in 2024-04-07T16:29:21.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:29:21 smithi100 bash[76202]: cluster 2024-04-07T16:29:20.124234+0000 mon.a (mon.0) 1386 : cluster [DBG] osdmap e802: 8 total, 8 up, 8 in 2024-04-07T16:29:21.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:29:21 smithi144 bash[66654]: cluster 2024-04-07T16:29:20.124234+0000 mon.a (mon.0) 1386 : cluster [DBG] osdmap e802: 8 total, 8 up, 8 in 2024-04-07T16:29:22.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:29:22 smithi100 bash[74942]: cluster 2024-04-07T16:29:20.838419+0000 mgr.y (mgr.25175) 419 : cluster [DBG] pgmap v797: 121 pgs: 5 active+clean+snaptrim, 116 active+clean; 212 MiB data, 1.9 GiB used, 713 GiB / 715 GiB avail; 11 MiB/s rd, 5.9 MiB/s wr, 39 op/s 2024-04-07T16:29:22.439 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:29:22 smithi100 bash[74942]: cluster 2024-04-07T16:29:21.136964+0000 mon.a (mon.0) 1387 : cluster [DBG] osdmap e803: 8 total, 8 up, 8 in 2024-04-07T16:29:22.439 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:29:22 smithi100 bash[74942]: cluster 2024-04-07T16:29:21.981367+0000 mon.a (mon.0) 1388 : cluster [DBG] osdmap e804: 8 total, 8 up, 8 in 2024-04-07T16:29:22.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:29:22 smithi100 bash[76202]: cluster 2024-04-07T16:29:20.838419+0000 mgr.y (mgr.25175) 419 : cluster [DBG] pgmap v797: 121 pgs: 5 active+clean+snaptrim, 116 active+clean; 212 MiB data, 1.9 GiB used, 713 GiB / 715 GiB avail; 11 MiB/s rd, 5.9 MiB/s wr, 39 op/s 2024-04-07T16:29:22.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:29:22 smithi100 bash[76202]: cluster 2024-04-07T16:29:21.136964+0000 mon.a (mon.0) 1387 : cluster [DBG] osdmap e803: 8 total, 8 up, 8 in 2024-04-07T16:29:22.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:29:22 smithi100 bash[76202]: cluster 2024-04-07T16:29:21.981367+0000 mon.a (mon.0) 1388 : cluster [DBG] osdmap e804: 8 total, 8 up, 8 in 2024-04-07T16:29:22.606 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:29:22 smithi144 bash[95233]: ts=2024-04-07T16:29:22.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:29:22.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:29:22 smithi144 bash[66654]: cluster 2024-04-07T16:29:20.838419+0000 mgr.y (mgr.25175) 419 : cluster [DBG] pgmap v797: 121 pgs: 5 active+clean+snaptrim, 116 active+clean; 212 MiB data, 1.9 GiB used, 713 GiB / 715 GiB avail; 11 MiB/s rd, 5.9 MiB/s wr, 39 op/s 2024-04-07T16:29:22.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:29:22 smithi144 bash[66654]: cluster 2024-04-07T16:29:21.136964+0000 mon.a (mon.0) 1387 : cluster [DBG] osdmap e803: 8 total, 8 up, 8 in 2024-04-07T16:29:22.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:29:22 smithi144 bash[66654]: cluster 2024-04-07T16:29:21.981367+0000 mon.a (mon.0) 1388 : cluster [DBG] osdmap e804: 8 total, 8 up, 8 in 2024-04-07T16:29:24.439 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:29:24 smithi100 bash[74942]: cluster 2024-04-07T16:29:22.839600+0000 mgr.y (mgr.25175) 420 : cluster [DBG] pgmap v800: 121 pgs: 1 active+clean+snaptrim, 120 active+clean; 223 MiB data, 1.9 GiB used, 713 GiB / 715 GiB avail; 9.4 MiB/s rd, 6.9 MiB/s wr, 29 op/s 2024-04-07T16:29:24.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:29:24 smithi100 bash[76202]: cluster 2024-04-07T16:29:22.839600+0000 mgr.y (mgr.25175) 420 : cluster [DBG] pgmap v800: 121 pgs: 1 active+clean+snaptrim, 120 active+clean; 223 MiB data, 1.9 GiB used, 713 GiB / 715 GiB avail; 9.4 MiB/s rd, 6.9 MiB/s wr, 29 op/s 2024-04-07T16:29:24.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:29:24 smithi144 bash[66654]: cluster 2024-04-07T16:29:22.839600+0000 mgr.y (mgr.25175) 420 : cluster [DBG] pgmap v800: 121 pgs: 1 active+clean+snaptrim, 120 active+clean; 223 MiB data, 1.9 GiB used, 713 GiB / 715 GiB avail; 9.4 MiB/s rd, 6.9 MiB/s wr, 29 op/s 2024-04-07T16:29:26.008 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/mon.a/config 2024-04-07T16:29:26.439 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:29:26 smithi100 bash[74942]: cluster 2024-04-07T16:29:24.840646+0000 mgr.y (mgr.25175) 421 : cluster [DBG] pgmap v801: 121 pgs: 121 active+clean; 229 MiB data, 2.0 GiB used, 713 GiB / 715 GiB avail; 11 MiB/s rd, 3.5 MiB/s wr, 23 op/s 2024-04-07T16:29:26.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:29:26 smithi100 bash[76202]: cluster 2024-04-07T16:29:24.840646+0000 mgr.y (mgr.25175) 421 : cluster [DBG] pgmap v801: 121 pgs: 121 active+clean; 229 MiB data, 2.0 GiB used, 713 GiB / 715 GiB avail; 11 MiB/s rd, 3.5 MiB/s wr, 23 op/s 2024-04-07T16:29:26.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:29:26 smithi144 bash[66654]: cluster 2024-04-07T16:29:24.840646+0000 mgr.y (mgr.25175) 421 : cluster [DBG] pgmap v801: 121 pgs: 121 active+clean; 229 MiB data, 2.0 GiB used, 713 GiB / 715 GiB avail; 11 MiB/s rd, 3.5 MiB/s wr, 23 op/s 2024-04-07T16:29:28.189 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:29:27 smithi100 bash[74942]: cluster 2024-04-07T16:29:26.841488+0000 mgr.y (mgr.25175) 422 : cluster [DBG] pgmap v802: 121 pgs: 121 active+clean; 230 MiB data, 2.0 GiB used, 713 GiB / 715 GiB avail; 8.8 MiB/s rd, 2.6 MiB/s wr, 16 op/s 2024-04-07T16:29:28.189 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:29:27 smithi100 bash[74942]: cluster 2024-04-07T16:29:26.988204+0000 mon.a (mon.0) 1389 : cluster [DBG] osdmap e805: 8 total, 8 up, 8 in 2024-04-07T16:29:28.189 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:29:27 smithi100 bash[76202]: cluster 2024-04-07T16:29:26.841488+0000 mgr.y (mgr.25175) 422 : cluster [DBG] pgmap v802: 121 pgs: 121 active+clean; 230 MiB data, 2.0 GiB used, 713 GiB / 715 GiB avail; 8.8 MiB/s rd, 2.6 MiB/s wr, 16 op/s 2024-04-07T16:29:28.189 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:29:27 smithi100 bash[76202]: cluster 2024-04-07T16:29:26.988204+0000 mon.a (mon.0) 1389 : cluster [DBG] osdmap e805: 8 total, 8 up, 8 in 2024-04-07T16:29:28.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:29:27 smithi144 bash[66654]: cluster 2024-04-07T16:29:26.841488+0000 mgr.y (mgr.25175) 422 : cluster [DBG] pgmap v802: 121 pgs: 121 active+clean; 230 MiB data, 2.0 GiB used, 713 GiB / 715 GiB avail; 8.8 MiB/s rd, 2.6 MiB/s wr, 16 op/s 2024-04-07T16:29:28.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:29:27 smithi144 bash[66654]: cluster 2024-04-07T16:29:26.988204+0000 mon.a (mon.0) 1389 : cluster [DBG] osdmap e805: 8 total, 8 up, 8 in 2024-04-07T16:29:28.678 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:29:28 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:29:28] "GET /metrics HTTP/1.1" 200 36343 "" "Prometheus/2.43.0" 2024-04-07T16:29:29.054 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-07T16:29:29.054 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":805,"fsid":"0702e3a8-f4f8-11ee-b649-cb9ed24678a4","created":"2024-04-07T16:02:21.629692+0000","modified":"2024-04-07T16:29:26.973998+0000","last_up_change":"2024-04-07T16:25:37.915048+0000","last_in_change":"2024-04-07T16:11:04.793434+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-07T16:07:26.784071+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":7.8899998664855957,"score_stable":7.8899998664855957,"optimal_score":0.37999999523162842,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-07T16:12:11.856505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-07T16:12:12.737709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"rbd","create_time":"2024-04-07T16:13:28.913394+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":3,"score_stable":3,"optimal_score":1,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"unique_pool_0","create_time":"2024-04-07T16:16:37.584649+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":3,"size":3,"min_size":2,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":16,"pg_placement_num":16,"pg_placement_num_target":16,"pg_num_target":16,"pg_num_pending":16,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"803","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":469,"snap_epoch":803,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"teuthologyprofile","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rados":{}}}],"osds":[{"osd":0,"uuid":"8facd48e-0a0c-4fda-aee2-f001b60a3811","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":8,"last_clean_end":376,"up_from":389,"up_thru":582,"down_at":377,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":165581137},{"type":"v1","addr":"172.21.15.100:6803","nonce":165581137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":165581137},{"type":"v1","addr":"172.21.15.100:6805","nonce":165581137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":165581137},{"type":"v1","addr":"172.21.15.100:6809","nonce":165581137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":165581137},{"type":"v1","addr":"172.21.15.100:6807","nonce":165581137}]},"public_addr":"172.21.15.100:6803/165581137","cluster_addr":"172.21.15.100:6805/165581137","heartbeat_back_addr":"172.21.15.100:6809/165581137","heartbeat_front_addr":"172.21.15.100:6807/165581137","state":["exists","up"]},{"osd":1,"uuid":"cb5f6c23-6764-48b6-8b21-97a36b004786","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":13,"last_clean_end":403,"up_from":416,"up_thru":582,"down_at":404,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":1469336877},{"type":"v1","addr":"172.21.15.100:6811","nonce":1469336877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1469336877},{"type":"v1","addr":"172.21.15.100:6813","nonce":1469336877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1469336877},{"type":"v1","addr":"172.21.15.100:6817","nonce":1469336877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1469336877},{"type":"v1","addr":"172.21.15.100:6815","nonce":1469336877}]},"public_addr":"172.21.15.100:6811/1469336877","cluster_addr":"172.21.15.100:6813/1469336877","heartbeat_back_addr":"172.21.15.100:6817/1469336877","heartbeat_front_addr":"172.21.15.100:6815/1469336877","state":["exists","up"]},{"osd":2,"uuid":"1650aedb-9ce9-4c14-a059-e1f089e5e63f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":18,"last_clean_end":431,"up_from":444,"up_thru":582,"down_at":432,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":2767460830},{"type":"v1","addr":"172.21.15.100:6819","nonce":2767460830}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2767460830},{"type":"v1","addr":"172.21.15.100:6821","nonce":2767460830}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":2767460830},{"type":"v1","addr":"172.21.15.100:6825","nonce":2767460830}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":2767460830},{"type":"v1","addr":"172.21.15.100:6823","nonce":2767460830}]},"public_addr":"172.21.15.100:6819/2767460830","cluster_addr":"172.21.15.100:6821/2767460830","heartbeat_back_addr":"172.21.15.100:6825/2767460830","heartbeat_front_addr":"172.21.15.100:6823/2767460830","state":["exists","up"]},{"osd":3,"uuid":"17591c6b-862b-43b2-858b-02f4c3199247","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":25,"last_clean_end":459,"up_from":470,"up_thru":582,"down_at":460,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":269478984},{"type":"v1","addr":"172.21.15.100:6827","nonce":269478984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":269478984},{"type":"v1","addr":"172.21.15.100:6829","nonce":269478984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":269478984},{"type":"v1","addr":"172.21.15.100:6833","nonce":269478984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":269478984},{"type":"v1","addr":"172.21.15.100:6831","nonce":269478984}]},"public_addr":"172.21.15.100:6827/269478984","cluster_addr":"172.21.15.100:6829/269478984","heartbeat_back_addr":"172.21.15.100:6833/269478984","heartbeat_front_addr":"172.21.15.100:6831/269478984","state":["exists","up"]},{"osd":4,"uuid":"6ecc1205-7c78-461e-8037-640136cb3658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":30,"last_clean_end":487,"up_from":499,"up_thru":582,"down_at":488,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":1068767678},{"type":"v1","addr":"172.21.15.144:6801","nonce":1068767678}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":1068767678},{"type":"v1","addr":"172.21.15.144:6803","nonce":1068767678}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1068767678},{"type":"v1","addr":"172.21.15.144:6807","nonce":1068767678}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":1068767678},{"type":"v1","addr":"172.21.15.144:6805","nonce":1068767678}]},"public_addr":"172.21.15.144:6801/1068767678","cluster_addr":"172.21.15.144:6803/1068767678","heartbeat_back_addr":"172.21.15.144:6807/1068767678","heartbeat_front_addr":"172.21.15.144:6805/1068767678","state":["exists","up"]},{"osd":5,"uuid":"54789cc8-1c3c-46a4-a92e-8881564338db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":38,"last_clean_end":513,"up_from":526,"up_thru":582,"down_at":514,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1594566125},{"type":"v1","addr":"172.21.15.144:6809","nonce":1594566125}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1594566125},{"type":"v1","addr":"172.21.15.144:6811","nonce":1594566125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1594566125},{"type":"v1","addr":"172.21.15.144:6815","nonce":1594566125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1594566125},{"type":"v1","addr":"172.21.15.144:6813","nonce":1594566125}]},"public_addr":"172.21.15.144:6809/1594566125","cluster_addr":"172.21.15.144:6811/1594566125","heartbeat_back_addr":"172.21.15.144:6815/1594566125","heartbeat_front_addr":"172.21.15.144:6813/1594566125","state":["exists","up"]},{"osd":6,"uuid":"ac619497-7605-4ca6-84bf-3abb3f6f32d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":46,"last_clean_end":542,"up_from":553,"up_thru":582,"down_at":543,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2416344523},{"type":"v1","addr":"172.21.15.144:6817","nonce":2416344523}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2416344523},{"type":"v1","addr":"172.21.15.144:6819","nonce":2416344523}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2416344523},{"type":"v1","addr":"172.21.15.144:6823","nonce":2416344523}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2416344523},{"type":"v1","addr":"172.21.15.144:6821","nonce":2416344523}]},"public_addr":"172.21.15.144:6817/2416344523","cluster_addr":"172.21.15.144:6819/2416344523","heartbeat_back_addr":"172.21.15.144:6823/2416344523","heartbeat_front_addr":"172.21.15.144:6821/2416344523","state":["exists","up"]},{"osd":7,"uuid":"b85b3422-9da2-4605-88e7-fe9c3b2020cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":54,"last_clean_end":569,"up_from":582,"up_thru":582,"down_at":570,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":3641044519},{"type":"v1","addr":"172.21.15.144:6825","nonce":3641044519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":3641044519},{"type":"v1","addr":"172.21.15.144:6827","nonce":3641044519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":3641044519},{"type":"v1","addr":"172.21.15.144:6831","nonce":3641044519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":3641044519},{"type":"v1","addr":"172.21.15.144:6829","nonce":3641044519}]},"public_addr":"172.21.15.144:6825/3641044519","cluster_addr":"172.21.15.144:6827/3641044519","heartbeat_back_addr":"172.21.15.144:6831/3641044519","heartbeat_front_addr":"172.21.15.144:6829/3641044519","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-07T16:05:44.696671+0000","dead_epoch":376},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-07T16:06:33.787321+0000","dead_epoch":403},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-07T16:07:22.983158+0000","dead_epoch":431},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-07T16:08:12.935606+0000","dead_epoch":458},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-07T16:09:00.218619+0000","dead_epoch":487},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-07T16:09:50.222185+0000","dead_epoch":513},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-07T16:10:37.856158+0000","dead_epoch":542},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-07T16:11:28.267877+0000","dead_epoch":569}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.7","mappings":[{"from":7,"to":4}]},{"pgid":"3.1","mappings":[{"from":0,"to":2}]},{"pgid":"3.12","mappings":[{"from":0,"to":2}]},{"pgid":"3.15","mappings":[{"from":4,"to":1}]},{"pgid":"3.17","mappings":[{"from":0,"to":2}]},{"pgid":"3.1d","mappings":[{"from":4,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.144:6833/226244023":"2024-04-08T16:26:19.573335+0000","172.21.15.100:0/2558802424":"2024-04-08T16:18:20.617755+0000","172.21.15.100:6800/3429225708":"2024-04-08T16:18:20.617755+0000","172.21.15.100:0/258495078":"2024-04-08T16:18:20.617755+0000","172.21.15.100:0/2754245222":"2024-04-08T16:20:16.558634+0000","172.21.15.144:0/2599140395":"2024-04-08T16:19:13.981948+0000","172.21.15.144:0/653654312":"2024-04-08T16:19:13.981948+0000","172.21.15.144:6835/3360802682":"2024-04-08T16:19:13.981948+0000","172.21.15.100:6801/190788518":"2024-04-08T16:12:52.577534+0000","172.21.15.144:0/1073632838":"2024-04-08T16:19:13.981948+0000","172.21.15.100:6800/2710911757":"2024-04-08T16:03:24.146096+0000","172.21.15.100:0/1631130639":"2024-04-08T16:18:20.617755+0000","172.21.15.100:0/3273267396":"2024-04-08T16:12:52.577534+0000","172.21.15.100:0/4121992264":"2024-04-08T16:02:53.269514+0000","172.21.15.100:0/218414173":"2024-04-08T16:18:20.617755+0000","172.21.15.100:6801/2710911757":"2024-04-08T16:03:24.146096+0000","172.21.15.100:6800/3440287045":"2024-04-08T16:02:53.269514+0000","172.21.15.100:0/3316052690":"2024-04-08T16:20:16.558634+0000","172.21.15.100:0/4162615562":"2024-04-08T16:02:53.269514+0000","172.21.15.100:6801/3440287045":"2024-04-08T16:02:53.269514+0000","172.21.15.100:0/1466792171":"2024-04-08T16:20:16.558634+0000","172.21.15.100:0/48519977":"2024-04-08T16:20:16.558634+0000","172.21.15.144:6832/226244023":"2024-04-08T16:26:19.573335+0000","172.21.15.100:0/2474331937":"2024-04-08T16:03:24.146096+0000","172.21.15.144:0/493790447":"2024-04-08T16:19:13.981948+0000","172.21.15.100:6801/3429225708":"2024-04-08T16:18:20.617755+0000","172.21.15.100:0/2159245581":"2024-04-08T16:20:16.558634+0000","172.21.15.100:6801/3273770395":"2024-04-08T16:20:16.558634+0000","172.21.15.100:0/639120693":"2024-04-08T16:03:24.146096+0000","172.21.15.144:0/3723623887":"2024-04-08T16:19:13.981948+0000","172.21.15.100:0/1472671368":"2024-04-08T16:03:24.146096+0000","172.21.15.144:6834/3360802682":"2024-04-08T16:19:13.981948+0000","172.21.15.100:0/1309290877":"2024-04-08T16:12:52.577534+0000","172.21.15.100:0/1822703232":"2024-04-08T16:20:16.558634+0000","172.21.15.100:6800/3273770395":"2024-04-08T16:20:16.558634+0000","172.21.15.100:6800/190788518":"2024-04-08T16:12:52.577534+0000","172.21.15.100:0/367509160":"2024-04-08T16:20:16.558634+0000","172.21.15.100:0/2535336108":"2024-04-08T16:12:52.577534+0000","172.21.15.144:0/1989982380":"2024-04-08T16:19:13.981948+0000","172.21.15.100:0/109071285":"2024-04-08T16:02:53.269514+0000","172.21.15.100:0/353657034":"2024-04-08T16:18:20.617755+0000","172.21.15.144:0/115503816":"2024-04-08T16:19:13.981948+0000","172.21.15.100:0/74983397":"2024-04-08T16:12:52.577534+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"},"teuthologyprofile":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van","w":"8"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":446,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[{"pool":5,"snaps":[{"begin":428,"length":1},{"begin":437,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-07T16:29:29.700 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 0702e3a8-f4f8-11ee-b649-cb9ed24678a4 -- ceph pg dump --format=json 2024-04-07T16:29:30.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:29:29 smithi144 bash[66654]: cluster 2024-04-07T16:29:28.842394+0000 mgr.y (mgr.25175) 423 : cluster [DBG] pgmap v804: 121 pgs: 121 active+clean; 230 MiB data, 2.0 GiB used, 713 GiB / 715 GiB avail; 7.7 MiB/s rd, 2.3 MiB/s wr, 14 op/s 2024-04-07T16:29:30.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:29:29 smithi144 bash[66654]: audit 2024-04-07T16:29:29.065680+0000 mon.c (mon.2) 18 : audit [DBG] from='client.? 172.21.15.100:0/3497759336' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-07T16:29:30.439 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:29:29 smithi100 bash[74942]: cluster 2024-04-07T16:29:28.842394+0000 mgr.y (mgr.25175) 423 : cluster [DBG] pgmap v804: 121 pgs: 121 active+clean; 230 MiB data, 2.0 GiB used, 713 GiB / 715 GiB avail; 7.7 MiB/s rd, 2.3 MiB/s wr, 14 op/s 2024-04-07T16:29:30.439 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:29:29 smithi100 bash[74942]: audit 2024-04-07T16:29:29.065680+0000 mon.c (mon.2) 18 : audit [DBG] from='client.? 172.21.15.100:0/3497759336' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-07T16:29:30.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:29:29 smithi100 bash[76202]: cluster 2024-04-07T16:29:28.842394+0000 mgr.y (mgr.25175) 423 : cluster [DBG] pgmap v804: 121 pgs: 121 active+clean; 230 MiB data, 2.0 GiB used, 713 GiB / 715 GiB avail; 7.7 MiB/s rd, 2.3 MiB/s wr, 14 op/s 2024-04-07T16:29:30.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:29:29 smithi100 bash[76202]: audit 2024-04-07T16:29:29.065680+0000 mon.c (mon.2) 18 : audit [DBG] from='client.? 172.21.15.100:0/3497759336' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-07T16:29:32.305 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:29:31 smithi144 bash[66654]: cluster 2024-04-07T16:29:30.843417+0000 mgr.y (mgr.25175) 424 : cluster [DBG] pgmap v805: 121 pgs: 121 active+clean; 230 MiB data, 2.0 GiB used, 713 GiB / 715 GiB avail; 6.7 MiB/s rd, 2.0 MiB/s wr, 12 op/s 2024-04-07T16:29:32.306 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:29:31 smithi144 bash[66654]: audit 2024-04-07T16:29:31.737486+0000 mon.a (mon.0) 1390 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:29:32.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:29:31 smithi100 bash[74942]: cluster 2024-04-07T16:29:30.843417+0000 mgr.y (mgr.25175) 424 : cluster [DBG] pgmap v805: 121 pgs: 121 active+clean; 230 MiB data, 2.0 GiB used, 713 GiB / 715 GiB avail; 6.7 MiB/s rd, 2.0 MiB/s wr, 12 op/s 2024-04-07T16:29:32.439 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:29:31 smithi100 bash[74942]: audit 2024-04-07T16:29:31.737486+0000 mon.a (mon.0) 1390 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:29:32.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:29:31 smithi100 bash[76202]: cluster 2024-04-07T16:29:30.843417+0000 mgr.y (mgr.25175) 424 : cluster [DBG] pgmap v805: 121 pgs: 121 active+clean; 230 MiB data, 2.0 GiB used, 713 GiB / 715 GiB avail; 6.7 MiB/s rd, 2.0 MiB/s wr, 12 op/s 2024-04-07T16:29:32.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:29:31 smithi100 bash[76202]: audit 2024-04-07T16:29:31.737486+0000 mon.a (mon.0) 1390 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:29:32.606 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:29:32 smithi144 bash[95233]: ts=2024-04-07T16:29:32.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:29:33.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:29:33 smithi144 bash[66654]: cluster 2024-04-07T16:29:31.990741+0000 mon.a (mon.0) 1391 : cluster [DBG] osdmap e806: 8 total, 8 up, 8 in 2024-04-07T16:29:33.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:29:33 smithi100 bash[74942]: cluster 2024-04-07T16:29:31.990741+0000 mon.a (mon.0) 1391 : cluster [DBG] osdmap e806: 8 total, 8 up, 8 in 2024-04-07T16:29:33.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:29:33 smithi100 bash[76202]: cluster 2024-04-07T16:29:31.990741+0000 mon.a (mon.0) 1391 : cluster [DBG] osdmap e806: 8 total, 8 up, 8 in 2024-04-07T16:29:34.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:29:34 smithi144 bash[66654]: cluster 2024-04-07T16:29:32.844516+0000 mgr.y (mgr.25175) 425 : cluster [DBG] pgmap v807: 121 pgs: 121 active+clean; 230 MiB data, 2.0 GiB used, 713 GiB / 715 GiB avail; 127 B/s rd, 172 KiB/s wr, 0 op/s 2024-04-07T16:29:34.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:29:34 smithi144 bash[66654]: audit 2024-04-07T16:29:33.075651+0000 mon.a (mon.0) 1392 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:29:34.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:29:34 smithi144 bash[66654]: audit 2024-04-07T16:29:33.808528+0000 mon.a (mon.0) 1393 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:29:34.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:29:34 smithi144 bash[66654]: audit 2024-04-07T16:29:33.810293+0000 mon.a (mon.0) 1394 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:29:34.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:29:34 smithi144 bash[66654]: audit 2024-04-07T16:29:33.819946+0000 mon.a (mon.0) 1395 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:29:34.439 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:29:34 smithi100 bash[74942]: cluster 2024-04-07T16:29:32.844516+0000 mgr.y (mgr.25175) 425 : cluster [DBG] pgmap v807: 121 pgs: 121 active+clean; 230 MiB data, 2.0 GiB used, 713 GiB / 715 GiB avail; 127 B/s rd, 172 KiB/s wr, 0 op/s 2024-04-07T16:29:34.439 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:29:34 smithi100 bash[74942]: audit 2024-04-07T16:29:33.075651+0000 mon.a (mon.0) 1392 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:29:34.439 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:29:34 smithi100 bash[74942]: audit 2024-04-07T16:29:33.808528+0000 mon.a (mon.0) 1393 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:29:34.439 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:29:34 smithi100 bash[74942]: audit 2024-04-07T16:29:33.810293+0000 mon.a (mon.0) 1394 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:29:34.439 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:29:34 smithi100 bash[74942]: audit 2024-04-07T16:29:33.819946+0000 mon.a (mon.0) 1395 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:29:34.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:29:34 smithi100 bash[76202]: cluster 2024-04-07T16:29:32.844516+0000 mgr.y (mgr.25175) 425 : cluster [DBG] pgmap v807: 121 pgs: 121 active+clean; 230 MiB data, 2.0 GiB used, 713 GiB / 715 GiB avail; 127 B/s rd, 172 KiB/s wr, 0 op/s 2024-04-07T16:29:34.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:29:34 smithi100 bash[76202]: audit 2024-04-07T16:29:33.075651+0000 mon.a (mon.0) 1392 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:29:34.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:29:34 smithi100 bash[76202]: audit 2024-04-07T16:29:33.808528+0000 mon.a (mon.0) 1393 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:29:34.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:29:34 smithi100 bash[76202]: audit 2024-04-07T16:29:33.810293+0000 mon.a (mon.0) 1394 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:29:34.440 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:29:34 smithi100 bash[76202]: audit 2024-04-07T16:29:33.819946+0000 mon.a (mon.0) 1395 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:29:34.445 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/mon.a/config 2024-04-07T16:29:36.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:29:36 smithi144 bash[66654]: cluster 2024-04-07T16:29:34.845531+0000 mgr.y (mgr.25175) 426 : cluster [DBG] pgmap v808: 121 pgs: 121 active+clean; 230 MiB data, 2.0 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:29:36.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:29:36 smithi100 bash[76202]: cluster 2024-04-07T16:29:34.845531+0000 mgr.y (mgr.25175) 426 : cluster [DBG] pgmap v808: 121 pgs: 121 active+clean; 230 MiB data, 2.0 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:29:36.439 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:29:36 smithi100 bash[74942]: cluster 2024-04-07T16:29:34.845531+0000 mgr.y (mgr.25175) 426 : cluster [DBG] pgmap v808: 121 pgs: 121 active+clean; 230 MiB data, 2.0 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:29:37.432 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-07T16:29:37.434 INFO:teuthology.orchestra.run.smithi100.stderr:dumped all 2024-04-07T16:29:38.066 INFO:teuthology.orchestra.run.smithi100.stdout:{"pg_ready":true,"pg_map":{"version":809,"stamp":"2024-04-07T16:29:36.845790+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":241662974,"num_objects":189,"num_object_clones":117,"num_object_copies":567,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":189,"num_whiteouts":7,"num_read":4949,"num_read_kb":5029121,"num_write":9323,"num_write_kb":1749684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":146,"num_bytes_recovered":444335712,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17642,"ondisk_log_size":17642,"up":363,"acting":363,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":363,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":2068592,"kb_used_data":425876,"kb_used_omap":292,"kb_used_meta":1642331,"kb_avail":747892624,"statfs":{"total":767960285184,"available":765842046976,"internally_reserved":0,"allocated":436097024,"data_stored":642179954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":299418,"internal_metadata":1681747558},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.864135"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":978,"reported_epoch":805,"state":"active+clean","last_fresh":"2024-04-07T16:29:27.870429+0000","last_change":"2024-04-07T16:25:38.969419+0000","last_active":"2024-04-07T16:29:27.870429+0000","last_peered":"2024-04-07T16:29:27.870429+0000","last_clean":"2024-04-07T16:29:27.870429+0000","last_became_active":"2024-04-07T16:25:38.966876+0000","last_became_peered":"2024-04-07T16:25:38.966876+0000","last_unstale":"2024-04-07T16:29:27.870429+0000","last_undegraded":"2024-04-07T16:29:27.870429+0000","last_fullsized":"2024-04-07T16:29:27.870429+0000","mapping_epoch":582,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":583,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T01:03:39.405808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":1148,"reported_epoch":806,"state":"active+clean","last_fresh":"2024-04-07T16:29:31.994283+0000","last_change":"2024-04-07T16:25:08.962703+0000","last_active":"2024-04-07T16:29:31.994283+0000","last_peered":"2024-04-07T16:29:31.994283+0000","last_clean":"2024-04-07T16:29:31.994283+0000","last_became_active":"2024-04-07T16:25:08.962145+0000","last_became_peered":"2024-04-07T16:25:08.962145+0000","last_unstale":"2024-04-07T16:29:31.994283+0000","last_undegraded":"2024-04-07T16:29:31.994283+0000","last_fullsized":"2024-04-07T16:29:31.994283+0000","mapping_epoch":553,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":554,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T23:09:45.491615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,6],"acting":[1,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":1159,"reported_epoch":806,"state":"active+clean","last_fresh":"2024-04-07T16:29:32.096029+0000","last_change":"2024-04-07T16:23:15.876856+0000","last_active":"2024-04-07T16:29:32.096029+0000","last_peered":"2024-04-07T16:29:32.096029+0000","last_clean":"2024-04-07T16:29:32.096029+0000","last_became_active":"2024-04-07T16:23:15.876439+0000","last_became_peered":"2024-04-07T16:23:15.876439+0000","last_unstale":"2024-04-07T16:29:32.096029+0000","last_undegraded":"2024-04-07T16:29:32.096029+0000","last_fullsized":"2024-04-07T16:29:32.096029+0000","mapping_epoch":444,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":445,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T03:48:36.975141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":1053,"reported_epoch":806,"state":"active+clean","last_fresh":"2024-04-07T16:29:32.035756+0000","last_change":"2024-04-07T16:25:38.987454+0000","last_active":"2024-04-07T16:29:32.035756+0000","last_peered":"2024-04-07T16:29:32.035756+0000","last_clean":"2024-04-07T16:29:32.035756+0000","last_became_active":"2024-04-07T16:25:38.986923+0000","last_became_peered":"2024-04-07T16:25:38.986923+0000","last_unstale":"2024-04-07T16:29:32.035756+0000","last_undegraded":"2024-04-07T16:29:32.035756+0000","last_fullsized":"2024-04-07T16:29:32.035756+0000","mapping_epoch":582,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":583,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T02:20:58.654636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":965,"reported_epoch":805,"state":"active+clean","last_fresh":"2024-04-07T16:29:27.869278+0000","last_change":"2024-04-07T16:25:38.990756+0000","last_active":"2024-04-07T16:29:27.869278+0000","last_peered":"2024-04-07T16:29:27.869278+0000","last_clean":"2024-04-07T16:29:27.869278+0000","last_became_active":"2024-04-07T16:25:38.985715+0000","last_became_peered":"2024-04-07T16:25:38.985715+0000","last_unstale":"2024-04-07T16:29:27.869278+0000","last_undegraded":"2024-04-07T16:29:27.869278+0000","last_fullsized":"2024-04-07T16:29:27.869278+0000","mapping_epoch":582,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":583,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T17:50:57.375885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":1043,"reported_epoch":806,"state":"active+clean","last_fresh":"2024-04-07T16:29:32.036259+0000","last_change":"2024-04-07T16:24:41.551443+0000","last_active":"2024-04-07T16:29:32.036259+0000","last_peered":"2024-04-07T16:29:32.036259+0000","last_clean":"2024-04-07T16:29:32.036259+0000","last_became_active":"2024-04-07T16:24:41.548397+0000","last_became_peered":"2024-04-07T16:24:41.548397+0000","last_unstale":"2024-04-07T16:29:32.036259+0000","last_undegraded":"2024-04-07T16:29:32.036259+0000","last_fullsized":"2024-04-07T16:29:32.036259+0000","mapping_epoch":526,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":527,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T20:38:47.135272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":965,"reported_epoch":805,"state":"active+clean","last_fresh":"2024-04-07T16:29:27.868829+0000","last_change":"2024-04-07T16:25:39.000120+0000","last_active":"2024-04-07T16:29:27.868829+0000","last_peered":"2024-04-07T16:29:27.868829+0000","last_clean":"2024-04-07T16:29:27.868829+0000","last_became_active":"2024-04-07T16:25:38.996215+0000","last_became_peered":"2024-04-07T16:25:38.996215+0000","last_unstale":"2024-04-07T16:29:27.868829+0000","last_undegraded":"2024-04-07T16:29:27.868829+0000","last_fullsized":"2024-04-07T16:29:27.868829+0000","mapping_epoch":582,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":583,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T22:36:35.618662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":1082,"reported_epoch":806,"state":"active+clean","last_fresh":"2024-04-07T16:29:31.986767+0000","last_change":"2024-04-07T16:25:08.978090+0000","last_active":"2024-04-07T16:29:31.986767+0000","last_peered":"2024-04-07T16:29:31.986767+0000","last_clean":"2024-04-07T16:29:31.986767+0000","last_became_active":"2024-04-07T16:25:08.977725+0000","last_became_peered":"2024-04-07T16:25:08.977725+0000","last_unstale":"2024-04-07T16:29:31.986767+0000","last_undegraded":"2024-04-07T16:29:31.986767+0000","last_fullsized":"2024-04-07T16:29:31.986767+0000","mapping_epoch":553,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":554,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T18:02:53.604912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":992,"reported_epoch":805,"state":"active+clean","last_fresh":"2024-04-07T16:29:27.870117+0000","last_change":"2024-04-07T16:25:38.976137+0000","last_active":"2024-04-07T16:29:27.870117+0000","last_peered":"2024-04-07T16:29:27.870117+0000","last_clean":"2024-04-07T16:29:27.870117+0000","last_became_active":"2024-04-07T16:25:38.968877+0000","last_became_peered":"2024-04-07T16:25:38.968877+0000","last_unstale":"2024-04-07T16:29:27.870117+0000","last_undegraded":"2024-04-07T16:29:27.870117+0000","last_fullsized":"2024-04-07T16:29:27.870117+0000","mapping_epoch":582,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":583,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T17:38:48.549583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,0],"acting":[7,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":1000,"reported_epoch":805,"state":"active+clean","last_fresh":"2024-04-07T16:29:27.848494+0000","last_change":"2024-04-07T16:25:08.978144+0000","last_active":"2024-04-07T16:29:27.848494+0000","last_peered":"2024-04-07T16:29:27.848494+0000","last_clean":"2024-04-07T16:29:27.848494+0000","last_became_active":"2024-04-07T16:25:08.977646+0000","last_became_peered":"2024-04-07T16:25:08.977646+0000","last_unstale":"2024-04-07T16:29:27.848494+0000","last_undegraded":"2024-04-07T16:29:27.848494+0000","last_fullsized":"2024-04-07T16:29:27.848494+0000","mapping_epoch":553,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":554,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T01:10:43.702742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":1000,"reported_epoch":805,"state":"active+clean","last_fresh":"2024-04-07T16:29:27.847843+0000","last_change":"2024-04-07T16:25:08.979235+0000","last_active":"2024-04-07T16:29:27.847843+0000","last_peered":"2024-04-07T16:29:27.847843+0000","last_clean":"2024-04-07T16:29:27.847843+0000","last_became_active":"2024-04-07T16:25:08.978750+0000","last_became_peered":"2024-04-07T16:25:08.978750+0000","last_unstale":"2024-04-07T16:29:27.847843+0000","last_undegraded":"2024-04-07T16:29:27.847843+0000","last_fullsized":"2024-04-07T16:29:27.847843+0000","mapping_epoch":553,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":554,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T03:27:45.790517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,0],"acting":[6,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":1073,"reported_epoch":806,"state":"active+clean","last_fresh":"2024-04-07T16:29:31.984322+0000","last_change":"2024-04-07T16:25:08.956220+0000","last_active":"2024-04-07T16:29:31.984322+0000","last_peered":"2024-04-07T16:29:31.984322+0000","last_clean":"2024-04-07T16:29:31.984322+0000","last_became_active":"2024-04-07T16:25:08.955654+0000","last_became_peered":"2024-04-07T16:25:08.955654+0000","last_unstale":"2024-04-07T16:29:31.984322+0000","last_undegraded":"2024-04-07T16:29:31.984322+0000","last_fullsized":"2024-04-07T16:29:31.984322+0000","mapping_epoch":553,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":554,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T00:20:14.081274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,0],"acting":[4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":1115,"reported_epoch":806,"state":"active+clean","last_fresh":"2024-04-07T16:29:31.987692+0000","last_change":"2024-04-07T16:25:08.976499+0000","last_active":"2024-04-07T16:29:31.987692+0000","last_peered":"2024-04-07T16:29:31.987692+0000","last_clean":"2024-04-07T16:29:31.987692+0000","last_became_active":"2024-04-07T16:25:08.976145+0000","last_became_peered":"2024-04-07T16:25:08.976145+0000","last_unstale":"2024-04-07T16:29:31.987692+0000","last_undegraded":"2024-04-07T16:29:31.987692+0000","last_fullsized":"2024-04-07T16:29:31.987692+0000","mapping_epoch":553,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":554,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T04:02:25.054838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":1158,"reported_epoch":806,"state":"active+clean","last_fresh":"2024-04-07T16:29:32.095376+0000","last_change":"2024-04-07T16:24:41.538718+0000","last_active":"2024-04-07T16:29:32.095376+0000","last_peered":"2024-04-07T16:29:32.095376+0000","last_clean":"2024-04-07T16:29:32.095376+0000","last_became_active":"2024-04-07T16:24:41.538233+0000","last_became_peered":"2024-04-07T16:24:41.538233+0000","last_unstale":"2024-04-07T16:29:32.095376+0000","last_undegraded":"2024-04-07T16:29:32.095376+0000","last_fullsized":"2024-04-07T16:29:32.095376+0000","mapping_epoch":526,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":527,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T22:18:55.330487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":1141,"reported_epoch":806,"state":"active+clean","last_fresh":"2024-04-07T16:29:31.990120+0000","last_change":"2024-04-07T16:24:41.534090+0000","last_active":"2024-04-07T16:29:31.990120+0000","last_peered":"2024-04-07T16:29:31.990120+0000","last_clean":"2024-04-07T16:29:31.990120+0000","last_became_active":"2024-04-07T16:24:41.533626+0000","last_became_peered":"2024-04-07T16:24:41.533626+0000","last_unstale":"2024-04-07T16:29:31.990120+0000","last_undegraded":"2024-04-07T16:29:31.990120+0000","last_fullsized":"2024-04-07T16:29:31.990120+0000","mapping_epoch":526,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":527,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T16:42:51.811301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,5],"acting":[1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":1052,"reported_epoch":806,"state":"active+clean","last_fresh":"2024-04-07T16:29:32.035015+0000","last_change":"2024-04-07T16:25:39.006138+0000","last_active":"2024-04-07T16:29:32.035015+0000","last_peered":"2024-04-07T16:29:32.035015+0000","last_clean":"2024-04-07T16:29:32.035015+0000","last_became_active":"2024-04-07T16:25:39.005541+0000","last_became_peered":"2024-04-07T16:25:39.005541+0000","last_unstale":"2024-04-07T16:29:32.035015+0000","last_undegraded":"2024-04-07T16:29:32.035015+0000","last_fullsized":"2024-04-07T16:29:32.035015+0000","mapping_epoch":582,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":583,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T20:40:58.093262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":1039,"reported_epoch":806,"state":"active+clean","last_fresh":"2024-04-07T16:29:32.037607+0000","last_change":"2024-04-07T16:25:38.971858+0000","last_active":"2024-04-07T16:29:32.037607+0000","last_peered":"2024-04-07T16:29:32.037607+0000","last_clean":"2024-04-07T16:29:32.037607+0000","last_became_active":"2024-04-07T16:25:38.971115+0000","last_became_peered":"2024-04-07T16:25:38.971115+0000","last_unstale":"2024-04-07T16:29:32.037607+0000","last_undegraded":"2024-04-07T16:29:32.037607+0000","last_fullsized":"2024-04-07T16:29:32.037607+0000","mapping_epoch":582,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":583,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T22:01:10.276598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,4],"acting":[5,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":1129,"reported_epoch":806,"state":"active+clean","last_fresh":"2024-04-07T16:29:32.153100+0000","last_change":"2024-04-07T16:25:38.967737+0000","last_active":"2024-04-07T16:29:32.153100+0000","last_peered":"2024-04-07T16:29:32.153100+0000","last_clean":"2024-04-07T16:29:32.153100+0000","last_became_active":"2024-04-07T16:25:38.967369+0000","last_became_peered":"2024-04-07T16:25:38.967369+0000","last_unstale":"2024-04-07T16:29:32.153100+0000","last_undegraded":"2024-04-07T16:29:32.153100+0000","last_fullsized":"2024-04-07T16:29:32.153100+0000","mapping_epoch":582,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":583,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T22:15:36.602436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,7],"acting":[2,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":977,"reported_epoch":805,"state":"active+clean","last_fresh":"2024-04-07T16:29:27.870873+0000","last_change":"2024-04-07T16:25:39.002704+0000","last_active":"2024-04-07T16:29:27.870873+0000","last_peered":"2024-04-07T16:29:27.870873+0000","last_clean":"2024-04-07T16:29:27.870873+0000","last_became_active":"2024-04-07T16:25:38.996053+0000","last_became_peered":"2024-04-07T16:25:38.996053+0000","last_unstale":"2024-04-07T16:29:27.870873+0000","last_undegraded":"2024-04-07T16:29:27.870873+0000","last_fullsized":"2024-04-07T16:29:27.870873+0000","mapping_epoch":582,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":583,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T17:45:19.934847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,6],"acting":[7,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1f","version":"623'5","reported_seq":1167,"reported_epoch":806,"state":"active+clean","last_fresh":"2024-04-07T16:29:32.097246+0000","last_change":"2024-04-07T16:25:38.997712+0000","last_active":"2024-04-07T16:29:32.097246+0000","last_peered":"2024-04-07T16:29:32.097246+0000","last_clean":"2024-04-07T16:29:32.097246+0000","last_became_active":"2024-04-07T16:25:38.997317+0000","last_became_peered":"2024-04-07T16:25:38.997317+0000","last_unstale":"2024-04-07T16:29:32.097246+0000","last_undegraded":"2024-04-07T16:29:32.097246+0000","last_fullsized":"2024-04-07T16:29:32.097246+0000","mapping_epoch":582,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":583,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T18:23:18.184011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":1102,"reported_epoch":806,"state":"active+clean","last_fresh":"2024-04-07T16:29:31.988512+0000","last_change":"2024-04-07T16:25:08.978435+0000","last_active":"2024-04-07T16:29:31.988512+0000","last_peered":"2024-04-07T16:29:31.988512+0000","last_clean":"2024-04-07T16:29:31.988512+0000","last_became_active":"2024-04-07T16:25:08.977988+0000","last_became_peered":"2024-04-07T16:25:08.977988+0000","last_unstale":"2024-04-07T16:29:31.988512+0000","last_undegraded":"2024-04-07T16:29:31.988512+0000","last_fullsized":"2024-04-07T16:29:31.988512+0000","mapping_epoch":553,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":554,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T21:46:59.340658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"623'2","reported_seq":1092,"reported_epoch":806,"state":"active+clean","last_fresh":"2024-04-07T16:29:31.988942+0000","last_change":"2024-04-07T16:24:41.546952+0000","last_active":"2024-04-07T16:29:31.988942+0000","last_peered":"2024-04-07T16:29:31.988942+0000","last_clean":"2024-04-07T16:29:31.988942+0000","last_became_active":"2024-04-07T16:24:41.546310+0000","last_became_peered":"2024-04-07T16:24:41.546310+0000","last_unstale":"2024-04-07T16:29:31.988942+0000","last_undegraded":"2024-04-07T16:29:31.988942+0000","last_fullsized":"2024-04-07T16:29:31.988942+0000","mapping_epoch":526,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":527,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T21:45:55.182027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":1159,"reported_epoch":806,"state":"active+clean","last_fresh":"2024-04-07T16:29:32.096495+0000","last_change":"2024-04-07T16:24:41.565093+0000","last_active":"2024-04-07T16:29:32.096495+0000","last_peered":"2024-04-07T16:29:32.096495+0000","last_clean":"2024-04-07T16:29:32.096495+0000","last_became_active":"2024-04-07T16:24:41.564700+0000","last_became_peered":"2024-04-07T16:24:41.564700+0000","last_unstale":"2024-04-07T16:29:32.096495+0000","last_undegraded":"2024-04-07T16:29:32.096495+0000","last_fullsized":"2024-04-07T16:29:32.096495+0000","mapping_epoch":526,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":527,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T01:20:29.103128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"61'2","reported_seq":982,"reported_epoch":805,"state":"active+clean","last_fresh":"2024-04-07T16:29:27.870784+0000","last_change":"2024-04-07T16:25:39.004508+0000","last_active":"2024-04-07T16:29:27.870784+0000","last_peered":"2024-04-07T16:29:27.870784+0000","last_clean":"2024-04-07T16:29:27.870784+0000","last_became_active":"2024-04-07T16:25:39.001898+0000","last_became_peered":"2024-04-07T16:25:39.001898+0000","last_unstale":"2024-04-07T16:29:27.870784+0000","last_undegraded":"2024-04-07T16:29:27.870784+0000","last_fullsized":"2024-04-07T16:29:27.870784+0000","mapping_epoch":582,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":583,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T01:47:40.701645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":1043,"reported_epoch":806,"state":"active+clean","last_fresh":"2024-04-07T16:29:32.036854+0000","last_change":"2024-04-07T16:24:41.543356+0000","last_active":"2024-04-07T16:29:32.036854+0000","last_peered":"2024-04-07T16:29:32.036854+0000","last_clean":"2024-04-07T16:29:32.036854+0000","last_became_active":"2024-04-07T16:24:41.542687+0000","last_became_peered":"2024-04-07T16:24:41.542687+0000","last_unstale":"2024-04-07T16:29:32.036854+0000","last_undegraded":"2024-04-07T16:29:32.036854+0000","last_fullsized":"2024-04-07T16:29:32.036854+0000","mapping_epoch":526,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":527,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T21:43:40.340691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"61'2","reported_seq":997,"reported_epoch":805,"state":"active+clean","last_fresh":"2024-04-07T16:29:27.869960+0000","last_change":"2024-04-07T16:25:38.991127+0000","last_active":"2024-04-07T16:29:27.869960+0000","last_peered":"2024-04-07T16:29:27.869960+0000","last_clean":"2024-04-07T16:29:27.869960+0000","last_became_active":"2024-04-07T16:25:38.975920+0000","last_became_peered":"2024-04-07T16:25:38.975920+0000","last_unstale":"2024-04-07T16:29:27.869960+0000","last_undegraded":"2024-04-07T16:29:27.869960+0000","last_fullsized":"2024-04-07T16:29:27.869960+0000","mapping_epoch":582,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":583,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T22:48:18.986526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":1046,"reported_epoch":806,"state":"active+clean","last_fresh":"2024-04-07T16:29:32.036543+0000","last_change":"2024-04-07T16:25:08.975369+0000","last_active":"2024-04-07T16:29:32.036543+0000","last_peered":"2024-04-07T16:29:32.036543+0000","last_clean":"2024-04-07T16:29:32.036543+0000","last_became_active":"2024-04-07T16:25:08.974827+0000","last_became_peered":"2024-04-07T16:25:08.974827+0000","last_unstale":"2024-04-07T16:29:32.036543+0000","last_undegraded":"2024-04-07T16:29:32.036543+0000","last_fullsized":"2024-04-07T16:29:32.036543+0000","mapping_epoch":553,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":554,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T19:16:10.778146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"623'2","reported_seq":1085,"reported_epoch":806,"state":"active+clean","last_fresh":"2024-04-07T16:29:31.988354+0000","last_change":"2024-04-07T16:25:38.998434+0000","last_active":"2024-04-07T16:29:31.988354+0000","last_peered":"2024-04-07T16:29:31.988354+0000","last_clean":"2024-04-07T16:29:31.988354+0000","last_became_active":"2024-04-07T16:25:38.997910+0000","last_became_peered":"2024-04-07T16:25:38.997910+0000","last_unstale":"2024-04-07T16:29:31.988354+0000","last_undegraded":"2024-04-07T16:29:31.988354+0000","last_fullsized":"2024-04-07T16:29:31.988354+0000","mapping_epoch":582,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":583,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T02:18:46.594583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":1092,"reported_epoch":806,"state":"active+clean","last_fresh":"2024-04-07T16:29:31.983833+0000","last_change":"2024-04-07T16:24:13.937948+0000","last_active":"2024-04-07T16:29:31.983833+0000","last_peered":"2024-04-07T16:29:31.983833+0000","last_clean":"2024-04-07T16:29:31.983833+0000","last_became_active":"2024-04-07T16:24:13.936610+0000","last_became_peered":"2024-04-07T16:24:13.936610+0000","last_unstale":"2024-04-07T16:29:31.983833+0000","last_undegraded":"2024-04-07T16:29:31.983833+0000","last_fullsized":"2024-04-07T16:29:31.983833+0000","mapping_epoch":499,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":500,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T02:25:15.032887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"623'2","reported_seq":1012,"reported_epoch":805,"state":"active+clean","last_fresh":"2024-04-07T16:29:27.847424+0000","last_change":"2024-04-07T16:25:38.960896+0000","last_active":"2024-04-07T16:29:27.847424+0000","last_peered":"2024-04-07T16:29:27.847424+0000","last_clean":"2024-04-07T16:29:27.847424+0000","last_became_active":"2024-04-07T16:25:38.960251+0000","last_became_peered":"2024-04-07T16:25:38.960251+0000","last_unstale":"2024-04-07T16:29:27.847424+0000","last_undegraded":"2024-04-07T16:29:27.847424+0000","last_fullsized":"2024-04-07T16:29:27.847424+0000","mapping_epoch":582,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":583,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T18:24:54.929614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":1157,"reported_epoch":806,"state":"active+clean","last_fresh":"2024-04-07T16:29:32.097199+0000","last_change":"2024-04-07T16:25:39.004810+0000","last_active":"2024-04-07T16:29:32.097199+0000","last_peered":"2024-04-07T16:29:32.097199+0000","last_clean":"2024-04-07T16:29:32.097199+0000","last_became_active":"2024-04-07T16:25:39.004366+0000","last_became_peered":"2024-04-07T16:25:39.004366+0000","last_unstale":"2024-04-07T16:29:32.097199+0000","last_undegraded":"2024-04-07T16:29:32.097199+0000","last_fullsized":"2024-04-07T16:29:32.097199+0000","mapping_epoch":582,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":583,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T19:31:04.334052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":1089,"reported_epoch":806,"state":"active+clean","last_fresh":"2024-04-07T16:29:31.986941+0000","last_change":"2024-04-07T16:25:08.976353+0000","last_active":"2024-04-07T16:29:31.986941+0000","last_peered":"2024-04-07T16:29:31.986941+0000","last_clean":"2024-04-07T16:29:31.986941+0000","last_became_active":"2024-04-07T16:25:08.976003+0000","last_became_peered":"2024-04-07T16:25:08.976003+0000","last_unstale":"2024-04-07T16:29:31.986941+0000","last_undegraded":"2024-04-07T16:29:31.986941+0000","last_fullsized":"2024-04-07T16:29:31.986941+0000","mapping_epoch":553,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":554,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T01:23:38.086617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":1121,"reported_epoch":806,"state":"active+clean","last_fresh":"2024-04-07T16:29:31.986985+0000","last_change":"2024-04-07T16:23:43.904215+0000","last_active":"2024-04-07T16:29:31.986985+0000","last_peered":"2024-04-07T16:29:31.986985+0000","last_clean":"2024-04-07T16:29:31.986985+0000","last_became_active":"2024-04-07T16:23:43.903341+0000","last_became_peered":"2024-04-07T16:23:43.903341+0000","last_unstale":"2024-04-07T16:29:31.986985+0000","last_undegraded":"2024-04-07T16:29:31.986985+0000","last_fullsized":"2024-04-07T16:29:31.986985+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T23:54:23.660433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":1039,"reported_epoch":806,"state":"active+clean","last_fresh":"2024-04-07T16:29:32.035574+0000","last_change":"2024-04-07T16:25:38.990885+0000","last_active":"2024-04-07T16:29:32.035574+0000","last_peered":"2024-04-07T16:29:32.035574+0000","last_clean":"2024-04-07T16:29:32.035574+0000","last_became_active":"2024-04-07T16:25:38.990134+0000","last_became_peered":"2024-04-07T16:25:38.990134+0000","last_unstale":"2024-04-07T16:29:32.035574+0000","last_undegraded":"2024-04-07T16:29:32.035574+0000","last_fullsized":"2024-04-07T16:29:32.035574+0000","mapping_epoch":582,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":583,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T19:33:03.967565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":1141,"reported_epoch":806,"state":"active+clean","last_fresh":"2024-04-07T16:29:31.990613+0000","last_change":"2024-04-07T16:24:13.933594+0000","last_active":"2024-04-07T16:29:31.990613+0000","last_peered":"2024-04-07T16:29:31.990613+0000","last_clean":"2024-04-07T16:29:31.990613+0000","last_became_active":"2024-04-07T16:24:13.932694+0000","last_became_peered":"2024-04-07T16:24:13.932694+0000","last_unstale":"2024-04-07T16:29:31.990613+0000","last_undegraded":"2024-04-07T16:29:31.990613+0000","last_fullsized":"2024-04-07T16:29:31.990613+0000","mapping_epoch":499,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":500,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T17:08:30.909773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":1001,"reported_epoch":805,"state":"active+clean","last_fresh":"2024-04-07T16:29:27.849072+0000","last_change":"2024-04-07T16:25:08.978095+0000","last_active":"2024-04-07T16:29:27.849072+0000","last_peered":"2024-04-07T16:29:27.849072+0000","last_clean":"2024-04-07T16:29:27.849072+0000","last_became_active":"2024-04-07T16:25:08.977704+0000","last_became_peered":"2024-04-07T16:25:08.977704+0000","last_unstale":"2024-04-07T16:29:27.849072+0000","last_undegraded":"2024-04-07T16:29:27.849072+0000","last_fullsized":"2024-04-07T16:29:27.849072+0000","mapping_epoch":553,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":554,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T18:47:28.916356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":1038,"reported_epoch":806,"state":"active+clean","last_fresh":"2024-04-07T16:29:32.036543+0000","last_change":"2024-04-07T16:25:38.978447+0000","last_active":"2024-04-07T16:29:32.036543+0000","last_peered":"2024-04-07T16:29:32.036543+0000","last_clean":"2024-04-07T16:29:32.036543+0000","last_became_active":"2024-04-07T16:25:38.977859+0000","last_became_peered":"2024-04-07T16:25:38.977859+0000","last_unstale":"2024-04-07T16:29:32.036543+0000","last_undegraded":"2024-04-07T16:29:32.036543+0000","last_fullsized":"2024-04-07T16:29:32.036543+0000","mapping_epoch":582,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":583,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T16:22:51.136501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":1026,"reported_epoch":806,"state":"active+clean","last_fresh":"2024-04-07T16:29:32.036178+0000","last_change":"2024-04-07T16:25:08.976930+0000","last_active":"2024-04-07T16:29:32.036178+0000","last_peered":"2024-04-07T16:29:32.036178+0000","last_clean":"2024-04-07T16:29:32.036178+0000","last_became_active":"2024-04-07T16:25:08.971659+0000","last_became_peered":"2024-04-07T16:25:08.971659+0000","last_unstale":"2024-04-07T16:29:32.036178+0000","last_undegraded":"2024-04-07T16:29:32.036178+0000","last_fullsized":"2024-04-07T16:29:32.036178+0000","mapping_epoch":553,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":554,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T02:13:26.493523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":1089,"reported_epoch":806,"state":"active+clean","last_fresh":"2024-04-07T16:29:32.153353+0000","last_change":"2024-04-07T16:24:41.558648+0000","last_active":"2024-04-07T16:29:32.153353+0000","last_peered":"2024-04-07T16:29:32.153353+0000","last_clean":"2024-04-07T16:29:32.153353+0000","last_became_active":"2024-04-07T16:24:41.558300+0000","last_became_peered":"2024-04-07T16:24:41.558300+0000","last_unstale":"2024-04-07T16:29:32.153353+0000","last_undegraded":"2024-04-07T16:29:32.153353+0000","last_fullsized":"2024-04-07T16:29:32.153353+0000","mapping_epoch":526,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":527,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T01:44:12.981988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3],"acting":[2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.15","version":"61'3","reported_seq":1154,"reported_epoch":806,"state":"active+clean","last_fresh":"2024-04-07T16:29:31.994058+0000","last_change":"2024-04-07T16:24:41.530429+0000","last_active":"2024-04-07T16:29:31.994058+0000","last_peered":"2024-04-07T16:29:31.994058+0000","last_clean":"2024-04-07T16:29:31.994058+0000","last_became_active":"2024-04-07T16:24:41.527553+0000","last_became_peered":"2024-04-07T16:24:41.527553+0000","last_unstale":"2024-04-07T16:29:31.994058+0000","last_undegraded":"2024-04-07T16:29:31.994058+0000","last_fullsized":"2024-04-07T16:29:31.994058+0000","mapping_epoch":526,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":527,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T16:34:47.703047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":1066,"reported_epoch":806,"state":"active+clean","last_fresh":"2024-04-07T16:29:31.985711+0000","last_change":"2024-04-07T16:25:38.986153+0000","last_active":"2024-04-07T16:29:31.985711+0000","last_peered":"2024-04-07T16:29:31.985711+0000","last_clean":"2024-04-07T16:29:31.985711+0000","last_became_active":"2024-04-07T16:25:38.985283+0000","last_became_peered":"2024-04-07T16:25:38.985283+0000","last_unstale":"2024-04-07T16:29:31.985711+0000","last_undegraded":"2024-04-07T16:29:31.985711+0000","last_fullsized":"2024-04-07T16:29:31.985711+0000","mapping_epoch":582,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":583,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T22:16:21.117705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"623'29","reported_seq":1036,"reported_epoch":805,"state":"active+clean","last_fresh":"2024-04-07T16:29:27.848272+0000","last_change":"2024-04-07T16:25:08.977082+0000","last_active":"2024-04-07T16:29:27.848272+0000","last_peered":"2024-04-07T16:29:27.848272+0000","last_clean":"2024-04-07T16:29:27.848272+0000","last_became_active":"2024-04-07T16:25:08.975701+0000","last_became_peered":"2024-04-07T16:25:08.975701+0000","last_unstale":"2024-04-07T16:29:27.848272+0000","last_undegraded":"2024-04-07T16:29:27.848272+0000","last_fullsized":"2024-04-07T16:29:27.848272+0000","mapping_epoch":553,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":554,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T20:48:03.443222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5824,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":10,"num_read_kb":14,"num_write":28,"num_write_kb":25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":986,"reported_epoch":805,"state":"active+clean","last_fresh":"2024-04-07T16:29:27.869702+0000","last_change":"2024-04-07T16:25:38.999803+0000","last_active":"2024-04-07T16:29:27.869702+0000","last_peered":"2024-04-07T16:29:27.869702+0000","last_clean":"2024-04-07T16:29:27.869702+0000","last_became_active":"2024-04-07T16:25:38.987197+0000","last_became_peered":"2024-04-07T16:25:38.987197+0000","last_unstale":"2024-04-07T16:29:27.869702+0000","last_undegraded":"2024-04-07T16:29:27.869702+0000","last_fullsized":"2024-04-07T16:29:27.869702+0000","mapping_epoch":582,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":583,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T03:29:56.940037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"61'2","reported_seq":1163,"reported_epoch":806,"state":"active+clean","last_fresh":"2024-04-07T16:29:32.097665+0000","last_change":"2024-04-07T16:24:13.964689+0000","last_active":"2024-04-07T16:29:32.097665+0000","last_peered":"2024-04-07T16:29:32.097665+0000","last_clean":"2024-04-07T16:29:32.097665+0000","last_became_active":"2024-04-07T16:24:13.963512+0000","last_became_peered":"2024-04-07T16:24:13.963512+0000","last_unstale":"2024-04-07T16:29:32.097665+0000","last_undegraded":"2024-04-07T16:29:32.097665+0000","last_fullsized":"2024-04-07T16:29:32.097665+0000","mapping_epoch":499,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":500,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T16:15:37.955732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":8,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":1089,"reported_epoch":806,"state":"active+clean","last_fresh":"2024-04-07T16:29:32.152915+0000","last_change":"2024-04-07T16:25:38.970054+0000","last_active":"2024-04-07T16:29:32.152915+0000","last_peered":"2024-04-07T16:29:32.152915+0000","last_clean":"2024-04-07T16:29:32.152915+0000","last_became_active":"2024-04-07T16:25:38.969402+0000","last_became_peered":"2024-04-07T16:25:38.969402+0000","last_unstale":"2024-04-07T16:29:32.152915+0000","last_undegraded":"2024-04-07T16:29:32.152915+0000","last_fullsized":"2024-04-07T16:29:32.152915+0000","mapping_epoch":582,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":583,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T23:11:21.401536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,3],"acting":[2,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":1053,"reported_epoch":806,"state":"active+clean","last_fresh":"2024-04-07T16:29:32.035905+0000","last_change":"2024-04-07T16:25:38.979026+0000","last_active":"2024-04-07T16:29:32.035905+0000","last_peered":"2024-04-07T16:29:32.035905+0000","last_clean":"2024-04-07T16:29:32.035905+0000","last_became_active":"2024-04-07T16:25:38.978180+0000","last_became_peered":"2024-04-07T16:25:38.978180+0000","last_unstale":"2024-04-07T16:29:32.035905+0000","last_undegraded":"2024-04-07T16:29:32.035905+0000","last_fullsized":"2024-04-07T16:29:32.035905+0000","mapping_epoch":582,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":583,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T01:41:45.196090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":965,"reported_epoch":805,"state":"active+clean","last_fresh":"2024-04-07T16:29:27.870187+0000","last_change":"2024-04-07T16:25:38.991515+0000","last_active":"2024-04-07T16:29:27.870187+0000","last_peered":"2024-04-07T16:29:27.870187+0000","last_clean":"2024-04-07T16:29:27.870187+0000","last_became_active":"2024-04-07T16:25:38.987994+0000","last_became_peered":"2024-04-07T16:25:38.987994+0000","last_unstale":"2024-04-07T16:29:27.870187+0000","last_undegraded":"2024-04-07T16:29:27.870187+0000","last_fullsized":"2024-04-07T16:29:27.870187+0000","mapping_epoch":582,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":583,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T20:51:08.365772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":1016,"reported_epoch":805,"state":"active+clean","last_fresh":"2024-04-07T16:29:27.847225+0000","last_change":"2024-04-07T16:25:08.966301+0000","last_active":"2024-04-07T16:29:27.847225+0000","last_peered":"2024-04-07T16:29:27.847225+0000","last_clean":"2024-04-07T16:29:27.847225+0000","last_became_active":"2024-04-07T16:25:08.965799+0000","last_became_peered":"2024-04-07T16:25:08.965799+0000","last_unstale":"2024-04-07T16:29:27.847225+0000","last_undegraded":"2024-04-07T16:29:27.847225+0000","last_fullsized":"2024-04-07T16:29:27.847225+0000","mapping_epoch":553,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":554,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T21:12:09.705565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":1000,"reported_epoch":805,"state":"active+clean","last_fresh":"2024-04-07T16:29:27.846984+0000","last_change":"2024-04-07T16:25:08.964055+0000","last_active":"2024-04-07T16:29:27.846984+0000","last_peered":"2024-04-07T16:29:27.846984+0000","last_clean":"2024-04-07T16:29:27.846984+0000","last_became_active":"2024-04-07T16:25:08.963259+0000","last_became_peered":"2024-04-07T16:25:08.963259+0000","last_unstale":"2024-04-07T16:29:27.846984+0000","last_undegraded":"2024-04-07T16:29:27.846984+0000","last_fullsized":"2024-04-07T16:29:27.846984+0000","mapping_epoch":553,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":554,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T23:13:34.886609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"61'8","reported_seq":1144,"reported_epoch":806,"state":"active+clean","last_fresh":"2024-04-07T16:29:32.151388+0000","last_change":"2024-04-07T16:23:15.880710+0000","last_active":"2024-04-07T16:29:32.151388+0000","last_peered":"2024-04-07T16:29:32.151388+0000","last_clean":"2024-04-07T16:29:32.151388+0000","last_became_active":"2024-04-07T16:23:15.875351+0000","last_became_peered":"2024-04-07T16:23:15.875351+0000","last_unstale":"2024-04-07T16:29:32.151388+0000","last_undegraded":"2024-04-07T16:29:32.151388+0000","last_fullsized":"2024-04-07T16:29:32.151388+0000","mapping_epoch":444,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":445,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T17:16:20.268788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":977,"reported_epoch":805,"state":"active+clean","last_fresh":"2024-04-07T16:29:27.868925+0000","last_change":"2024-04-07T16:25:38.989241+0000","last_active":"2024-04-07T16:29:27.868925+0000","last_peered":"2024-04-07T16:29:27.868925+0000","last_clean":"2024-04-07T16:29:27.868925+0000","last_became_active":"2024-04-07T16:25:38.987996+0000","last_became_peered":"2024-04-07T16:25:38.987996+0000","last_unstale":"2024-04-07T16:29:27.868925+0000","last_undegraded":"2024-04-07T16:29:27.868925+0000","last_fullsized":"2024-04-07T16:29:27.868925+0000","mapping_epoch":582,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":583,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T16:46:00.763122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"803'1026","reported_seq":3983,"reported_epoch":806,"state":"active+clean","last_fresh":"2024-04-07T16:29:32.151468+0000","last_change":"2024-04-07T16:29:21.563457+0000","last_active":"2024-04-07T16:29:32.151468+0000","last_peered":"2024-04-07T16:29:32.151468+0000","last_clean":"2024-04-07T16:29:32.151468+0000","last_became_active":"2024-04-07T16:23:15.866572+0000","last_became_peered":"2024-04-07T16:23:15.866572+0000","last_unstale":"2024-04-07T16:29:32.151468+0000","last_undegraded":"2024-04-07T16:29:32.151468+0000","last_fullsized":"2024-04-07T16:29:32.151468+0000","mapping_epoch":444,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":445,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:16:38.108540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:16:38.108540+0000","last_clean_scrub_stamp":"2024-04-07T16:16:38.108540+0000","objects_scrubbed":0,"log_size":1026,"log_dups_size":0,"ondisk_log_size":1026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T16:23:21.822706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.011323646,"stat_sum":{"num_bytes":25362432,"num_objects":12,"num_object_clones":9,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":1,"num_read":216,"num_read_kb":343801,"num_write":491,"num_write_kb":108664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":18046976,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.f","version":"61'1","reported_seq":1077,"reported_epoch":806,"state":"active+clean","last_fresh":"2024-04-07T16:29:31.985050+0000","last_change":"2024-04-07T16:25:38.968805+0000","last_active":"2024-04-07T16:29:31.985050+0000","last_peered":"2024-04-07T16:29:31.985050+0000","last_clean":"2024-04-07T16:29:31.985050+0000","last_became_active":"2024-04-07T16:25:38.967913+0000","last_became_peered":"2024-04-07T16:25:38.967913+0000","last_unstale":"2024-04-07T16:29:31.985050+0000","last_undegraded":"2024-04-07T16:29:31.985050+0000","last_fullsized":"2024-04-07T16:29:31.985050+0000","mapping_epoch":582,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":583,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T19:43:23.875589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":965,"reported_epoch":805,"state":"active+clean","last_fresh":"2024-04-07T16:29:27.870204+0000","last_change":"2024-04-07T16:25:38.970807+0000","last_active":"2024-04-07T16:29:27.870204+0000","last_peered":"2024-04-07T16:29:27.870204+0000","last_clean":"2024-04-07T16:29:27.870204+0000","last_became_active":"2024-04-07T16:25:38.968892+0000","last_became_peered":"2024-04-07T16:25:38.968892+0000","last_unstale":"2024-04-07T16:29:27.870204+0000","last_undegraded":"2024-04-07T16:29:27.870204+0000","last_fullsized":"2024-04-07T16:29:27.870204+0000","mapping_epoch":582,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":583,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T21:17:23.974869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3","version":"803'1560","reported_seq":4636,"reported_epoch":806,"state":"active+clean","last_fresh":"2024-04-07T16:29:32.096769+0000","last_change":"2024-04-07T16:29:21.173454+0000","last_active":"2024-04-07T16:29:32.096769+0000","last_peered":"2024-04-07T16:29:32.096769+0000","last_clean":"2024-04-07T16:29:32.096769+0000","last_became_active":"2024-04-07T16:25:08.999028+0000","last_became_peered":"2024-04-07T16:25:08.999028+0000","last_unstale":"2024-04-07T16:29:32.096769+0000","last_undegraded":"2024-04-07T16:29:32.096769+0000","last_fullsized":"2024-04-07T16:29:32.096769+0000","mapping_epoch":553,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":554,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:16:38.108540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:16:38.108540+0000","last_clean_scrub_stamp":"2024-04-07T16:16:38.108540+0000","objects_scrubbed":0,"log_size":1560,"log_dups_size":0,"ondisk_log_size":1560,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T02:51:35.423200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.029950878,"stat_sum":{"num_bytes":23724032,"num_objects":19,"num_object_clones":14,"num_object_copies":57,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":416,"num_read_kb":574484,"num_write":815,"num_write_kb":156560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":24616960,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":1142,"reported_epoch":806,"state":"active+clean","last_fresh":"2024-04-07T16:29:31.991358+0000","last_change":"2024-04-07T16:24:41.537907+0000","last_active":"2024-04-07T16:29:31.991358+0000","last_peered":"2024-04-07T16:29:31.991358+0000","last_clean":"2024-04-07T16:29:31.991358+0000","last_became_active":"2024-04-07T16:24:41.537435+0000","last_became_peered":"2024-04-07T16:24:41.537435+0000","last_unstale":"2024-04-07T16:29:31.991358+0000","last_undegraded":"2024-04-07T16:29:31.991358+0000","last_fullsized":"2024-04-07T16:29:31.991358+0000","mapping_epoch":526,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":527,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:13:28.963534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:13:28.963534+0000","last_clean_scrub_stamp":"2024-04-07T16:13:28.963534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T18:25:59.495066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1149,"reported_epoch":806,"state":"active+clean","last_fresh":"2024-04-07T16:29:31.993616+0000","last_change":"2024-04-07T16:25:39.000176+0000","last_active":"2024-04-07T16:29:31.993616+0000","last_peered":"2024-04-07T16:29:31.993616+0000","last_clean":"2024-04-07T16:29:31.993616+0000","last_became_active":"2024-04-07T16:25:38.999666+0000","last_became_peered":"2024-04-07T16:25:38.999666+0000","last_unstale":"2024-04-07T16:29:31.993616+0000","last_undegraded":"2024-04-07T16:29:31.993616+0000","last_fullsized":"2024-04-07T16:29:31.993616+0000","mapping_epoch":582,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":583,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T19:36:37.701087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":1058,"reported_epoch":806,"state":"active+clean","last_fresh":"2024-04-07T16:29:32.036090+0000","last_change":"2024-04-07T16:24:41.557859+0000","last_active":"2024-04-07T16:29:32.036090+0000","last_peered":"2024-04-07T16:29:32.036090+0000","last_clean":"2024-04-07T16:29:32.036090+0000","last_became_active":"2024-04-07T16:24:41.554485+0000","last_became_peered":"2024-04-07T16:24:41.554485+0000","last_unstale":"2024-04-07T16:29:32.036090+0000","last_undegraded":"2024-04-07T16:29:32.036090+0000","last_fullsized":"2024-04-07T16:29:32.036090+0000","mapping_epoch":526,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":527,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T18:53:57.243053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"803'1810","reported_seq":4817,"reported_epoch":806,"state":"active+clean","last_fresh":"2024-04-07T16:29:32.096890+0000","last_change":"2024-04-07T16:29:21.177780+0000","last_active":"2024-04-07T16:29:32.096890+0000","last_peered":"2024-04-07T16:29:32.096890+0000","last_clean":"2024-04-07T16:29:32.096890+0000","last_became_active":"2024-04-07T16:24:13.938477+0000","last_became_peered":"2024-04-07T16:24:13.938477+0000","last_unstale":"2024-04-07T16:29:32.096890+0000","last_undegraded":"2024-04-07T16:29:32.096890+0000","last_fullsized":"2024-04-07T16:29:32.096890+0000","mapping_epoch":499,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":500,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:16:38.108540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:16:38.108540+0000","last_clean_scrub_stamp":"2024-04-07T16:16:38.108540+0000","objects_scrubbed":0,"log_size":1810,"log_dups_size":0,"ondisk_log_size":1810,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T03:16:22.905235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.035064373000000003,"stat_sum":{"num_bytes":12926976,"num_objects":18,"num_object_clones":13,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":2,"num_read":361,"num_read_kb":433977,"num_write":965,"num_write_kb":151816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":36438016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":1142,"reported_epoch":806,"state":"active+clean","last_fresh":"2024-04-07T16:29:31.990001+0000","last_change":"2024-04-07T16:23:43.924954+0000","last_active":"2024-04-07T16:29:31.990001+0000","last_peered":"2024-04-07T16:29:31.990001+0000","last_clean":"2024-04-07T16:29:31.990001+0000","last_became_active":"2024-04-07T16:23:43.924443+0000","last_became_peered":"2024-04-07T16:23:43.924443+0000","last_unstale":"2024-04-07T16:29:31.990001+0000","last_undegraded":"2024-04-07T16:29:31.990001+0000","last_fullsized":"2024-04-07T16:29:31.990001+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:13:28.963534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:13:28.963534+0000","last_clean_scrub_stamp":"2024-04-07T16:13:28.963534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T22:32:37.638847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":1040,"reported_epoch":806,"state":"active+clean","last_fresh":"2024-04-07T16:29:32.035522+0000","last_change":"2024-04-07T16:25:08.973250+0000","last_active":"2024-04-07T16:29:32.035522+0000","last_peered":"2024-04-07T16:29:32.035522+0000","last_clean":"2024-04-07T16:29:32.035522+0000","last_became_active":"2024-04-07T16:25:08.972663+0000","last_became_peered":"2024-04-07T16:25:08.972663+0000","last_unstale":"2024-04-07T16:29:32.035522+0000","last_undegraded":"2024-04-07T16:29:32.035522+0000","last_fullsized":"2024-04-07T16:29:32.035522+0000","mapping_epoch":553,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":554,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T22:42:44.603307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":1074,"reported_epoch":806,"state":"active+clean","last_fresh":"2024-04-07T16:29:31.984581+0000","last_change":"2024-04-07T16:25:08.952832+0000","last_active":"2024-04-07T16:29:31.984581+0000","last_peered":"2024-04-07T16:29:31.984581+0000","last_clean":"2024-04-07T16:29:31.984581+0000","last_became_active":"2024-04-07T16:25:08.952350+0000","last_became_peered":"2024-04-07T16:25:08.952350+0000","last_unstale":"2024-04-07T16:29:31.984581+0000","last_undegraded":"2024-04-07T16:29:31.984581+0000","last_fullsized":"2024-04-07T16:29:31.984581+0000","mapping_epoch":553,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":554,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T22:57:40.507778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.6","version":"803'1261","reported_seq":4228,"reported_epoch":806,"state":"active+clean","last_fresh":"2024-04-07T16:29:32.153357+0000","last_change":"2024-04-07T16:29:21.585439+0000","last_active":"2024-04-07T16:29:32.153357+0000","last_peered":"2024-04-07T16:29:32.153357+0000","last_clean":"2024-04-07T16:29:32.153357+0000","last_became_active":"2024-04-07T16:24:41.565201+0000","last_became_peered":"2024-04-07T16:24:41.565201+0000","last_unstale":"2024-04-07T16:29:32.153357+0000","last_undegraded":"2024-04-07T16:29:32.153357+0000","last_fullsized":"2024-04-07T16:29:32.153357+0000","mapping_epoch":526,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":527,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:16:38.108540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:16:38.108540+0000","last_clean_scrub_stamp":"2024-04-07T16:16:38.108540+0000","objects_scrubbed":0,"log_size":1261,"log_dups_size":0,"ondisk_log_size":1261,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T16:48:28.297879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.032462800999999999,"stat_sum":{"num_bytes":11304960,"num_objects":7,"num_object_clones":4,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":1,"num_read":274,"num_read_kb":410464,"num_write":556,"num_write_kb":129496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":35332096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,1],"acting":[2,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":1141,"reported_epoch":806,"state":"active+clean","last_fresh":"2024-04-07T16:29:31.993793+0000","last_change":"2024-04-07T16:24:41.570886+0000","last_active":"2024-04-07T16:29:31.993793+0000","last_peered":"2024-04-07T16:29:31.993793+0000","last_clean":"2024-04-07T16:29:31.993793+0000","last_became_active":"2024-04-07T16:24:41.570251+0000","last_became_peered":"2024-04-07T16:24:41.570251+0000","last_unstale":"2024-04-07T16:29:31.993793+0000","last_undegraded":"2024-04-07T16:29:31.993793+0000","last_fullsized":"2024-04-07T16:29:31.993793+0000","mapping_epoch":526,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":527,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:13:28.963534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:13:28.963534+0000","last_clean_scrub_stamp":"2024-04-07T16:13:28.963534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T21:59:24.421795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":1142,"reported_epoch":806,"state":"active+clean","last_fresh":"2024-04-07T16:29:31.989994+0000","last_change":"2024-04-07T16:25:08.962269+0000","last_active":"2024-04-07T16:29:31.989994+0000","last_peered":"2024-04-07T16:29:31.989994+0000","last_clean":"2024-04-07T16:29:31.989994+0000","last_became_active":"2024-04-07T16:25:08.961818+0000","last_became_peered":"2024-04-07T16:25:08.961818+0000","last_unstale":"2024-04-07T16:29:31.989994+0000","last_undegraded":"2024-04-07T16:29:31.989994+0000","last_fullsized":"2024-04-07T16:29:31.989994+0000","mapping_epoch":553,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":554,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T01:33:47.318091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1138,"reported_epoch":806,"state":"active+clean","last_fresh":"2024-04-07T16:29:32.151575+0000","last_change":"2024-04-07T16:23:43.930960+0000","last_active":"2024-04-07T16:29:32.151575+0000","last_peered":"2024-04-07T16:29:32.151575+0000","last_clean":"2024-04-07T16:29:32.151575+0000","last_became_active":"2024-04-07T16:23:43.929900+0000","last_became_peered":"2024-04-07T16:23:43.929900+0000","last_unstale":"2024-04-07T16:29:32.151575+0000","last_undegraded":"2024-04-07T16:29:32.151575+0000","last_fullsized":"2024-04-07T16:29:32.151575+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T20:05:04.268255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.4","version":"801'656","reported_seq":3366,"reported_epoch":805,"state":"active+clean","last_fresh":"2024-04-07T16:29:27.870698+0000","last_change":"2024-04-07T16:29:21.143161+0000","last_active":"2024-04-07T16:29:27.870698+0000","last_peered":"2024-04-07T16:29:27.870698+0000","last_clean":"2024-04-07T16:29:27.870698+0000","last_became_active":"2024-04-07T16:25:38.976755+0000","last_became_peered":"2024-04-07T16:25:38.976755+0000","last_unstale":"2024-04-07T16:29:27.870698+0000","last_undegraded":"2024-04-07T16:29:27.870698+0000","last_fullsized":"2024-04-07T16:29:27.870698+0000","mapping_epoch":582,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":583,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:16:38.108540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:16:38.108540+0000","last_clean_scrub_stamp":"2024-04-07T16:16:38.108540+0000","objects_scrubbed":0,"log_size":656,"log_dups_size":0,"ondisk_log_size":656,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T19:16:27.035622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003636233,"stat_sum":{"num_bytes":9084928,"num_objects":9,"num_object_clones":7,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":1,"num_read":173,"num_read_kb":190725,"num_write":288,"num_write_kb":64216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4964352,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"1be","length":"1"}]},{"pgid":"4.5","version":"0'0","reported_seq":1004,"reported_epoch":805,"state":"active+clean","last_fresh":"2024-04-07T16:29:27.848543+0000","last_change":"2024-04-07T16:25:39.000913+0000","last_active":"2024-04-07T16:29:27.848543+0000","last_peered":"2024-04-07T16:29:27.848543+0000","last_clean":"2024-04-07T16:29:27.848543+0000","last_became_active":"2024-04-07T16:25:39.000185+0000","last_became_peered":"2024-04-07T16:25:39.000185+0000","last_unstale":"2024-04-07T16:29:27.848543+0000","last_undegraded":"2024-04-07T16:29:27.848543+0000","last_fullsized":"2024-04-07T16:29:27.848543+0000","mapping_epoch":582,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":583,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:13:28.963534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:13:28.963534+0000","last_clean_scrub_stamp":"2024-04-07T16:13:28.963534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T00:32:30.890680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":1083,"reported_epoch":806,"state":"active+clean","last_fresh":"2024-04-07T16:29:31.990375+0000","last_change":"2024-04-07T16:25:08.978231+0000","last_active":"2024-04-07T16:29:31.990375+0000","last_peered":"2024-04-07T16:29:31.990375+0000","last_clean":"2024-04-07T16:29:31.990375+0000","last_became_active":"2024-04-07T16:25:08.977825+0000","last_became_peered":"2024-04-07T16:25:08.977825+0000","last_unstale":"2024-04-07T16:29:31.990375+0000","last_undegraded":"2024-04-07T16:29:31.990375+0000","last_fullsized":"2024-04-07T16:29:31.990375+0000","mapping_epoch":553,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":554,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T01:34:08.840626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":1027,"reported_epoch":806,"state":"active+clean","last_fresh":"2024-04-07T16:29:32.035561+0000","last_change":"2024-04-07T16:25:38.986647+0000","last_active":"2024-04-07T16:29:32.035561+0000","last_peered":"2024-04-07T16:29:32.035561+0000","last_clean":"2024-04-07T16:29:32.035561+0000","last_became_active":"2024-04-07T16:25:38.985921+0000","last_became_peered":"2024-04-07T16:25:38.985921+0000","last_unstale":"2024-04-07T16:29:32.035561+0000","last_undegraded":"2024-04-07T16:29:32.035561+0000","last_fullsized":"2024-04-07T16:29:32.035561+0000","mapping_epoch":582,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":583,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T02:44:42.906581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"308'221","reported_seq":1560,"reported_epoch":805,"state":"active+clean","last_fresh":"2024-04-07T16:29:27.868727+0000","last_change":"2024-04-07T16:25:39.004716+0000","last_active":"2024-04-07T16:29:27.868727+0000","last_peered":"2024-04-07T16:29:27.868727+0000","last_clean":"2024-04-07T16:29:27.868727+0000","last_became_active":"2024-04-07T16:25:39.000587+0000","last_became_peered":"2024-04-07T16:25:39.000587+0000","last_unstale":"2024-04-07T16:29:27.868727+0000","last_undegraded":"2024-04-07T16:29:27.868727+0000","last_fullsized":"2024-04-07T16:29:27.868727+0000","mapping_epoch":582,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":583,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:07:26.866003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:07:26.866003+0000","last_clean_scrub_stamp":"2024-04-07T16:07:26.866003+0000","objects_scrubbed":0,"log_size":221,"log_dups_size":0,"ondisk_log_size":221,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T21:59:54.730545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1034,"num_read_kb":1874,"num_write":359,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"803'1447","reported_seq":4249,"reported_epoch":806,"state":"active+clean","last_fresh":"2024-04-07T16:29:32.037782+0000","last_change":"2024-04-07T16:29:21.171917+0000","last_active":"2024-04-07T16:29:32.037782+0000","last_peered":"2024-04-07T16:29:32.037782+0000","last_clean":"2024-04-07T16:29:32.037782+0000","last_became_active":"2024-04-07T16:24:41.557966+0000","last_became_peered":"2024-04-07T16:24:41.557966+0000","last_unstale":"2024-04-07T16:29:32.037782+0000","last_undegraded":"2024-04-07T16:29:32.037782+0000","last_fullsized":"2024-04-07T16:29:32.037782+0000","mapping_epoch":526,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":527,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:16:38.108540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:16:38.108540+0000","last_clean_scrub_stamp":"2024-04-07T16:16:38.108540+0000","objects_scrubbed":0,"log_size":1447,"log_dups_size":0,"ondisk_log_size":1447,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T16:27:09.232154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.031757543999999999,"stat_sum":{"num_bytes":21159936,"num_objects":9,"num_object_clones":5,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":1,"num_read":317,"num_read_kb":560655,"num_write":708,"num_write_kb":145344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":69910528,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":1158,"reported_epoch":806,"state":"active+clean","last_fresh":"2024-04-07T16:29:32.096597+0000","last_change":"2024-04-07T16:23:15.883356+0000","last_active":"2024-04-07T16:29:32.096597+0000","last_peered":"2024-04-07T16:29:32.096597+0000","last_clean":"2024-04-07T16:29:32.096597+0000","last_became_active":"2024-04-07T16:23:15.882881+0000","last_became_peered":"2024-04-07T16:23:15.882881+0000","last_unstale":"2024-04-07T16:29:32.096597+0000","last_undegraded":"2024-04-07T16:29:32.096597+0000","last_fullsized":"2024-04-07T16:29:32.096597+0000","mapping_epoch":444,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":445,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:13:28.963534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:13:28.963534+0000","last_clean_scrub_stamp":"2024-04-07T16:13:28.963534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T03:46:29.223596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":1090,"reported_epoch":806,"state":"active+clean","last_fresh":"2024-04-07T16:29:32.151221+0000","last_change":"2024-04-07T16:24:13.931259+0000","last_active":"2024-04-07T16:29:32.151221+0000","last_peered":"2024-04-07T16:29:32.151221+0000","last_clean":"2024-04-07T16:29:32.151221+0000","last_became_active":"2024-04-07T16:24:13.930517+0000","last_became_peered":"2024-04-07T16:24:13.930517+0000","last_unstale":"2024-04-07T16:29:32.151221+0000","last_undegraded":"2024-04-07T16:29:32.151221+0000","last_fullsized":"2024-04-07T16:29:32.151221+0000","mapping_epoch":499,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":500,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T01:43:11.885075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":966,"reported_epoch":805,"state":"active+clean","last_fresh":"2024-04-07T16:29:27.868685+0000","last_change":"2024-04-07T16:25:39.003944+0000","last_active":"2024-04-07T16:29:27.868685+0000","last_peered":"2024-04-07T16:29:27.868685+0000","last_clean":"2024-04-07T16:29:27.868685+0000","last_became_active":"2024-04-07T16:25:39.000134+0000","last_became_peered":"2024-04-07T16:25:39.000134+0000","last_unstale":"2024-04-07T16:29:27.868685+0000","last_undegraded":"2024-04-07T16:29:27.868685+0000","last_fullsized":"2024-04-07T16:29:27.868685+0000","mapping_epoch":582,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":583,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T22:28:12.376172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"797'379","reported_seq":3137,"reported_epoch":805,"state":"active+clean","last_fresh":"2024-04-07T16:29:27.847865+0000","last_change":"2024-04-07T16:29:21.151093+0000","last_active":"2024-04-07T16:29:27.847865+0000","last_peered":"2024-04-07T16:29:27.847865+0000","last_clean":"2024-04-07T16:29:27.847865+0000","last_became_active":"2024-04-07T16:25:08.959021+0000","last_became_peered":"2024-04-07T16:25:08.959021+0000","last_unstale":"2024-04-07T16:29:27.847865+0000","last_undegraded":"2024-04-07T16:29:27.847865+0000","last_fullsized":"2024-04-07T16:29:27.847865+0000","mapping_epoch":553,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":554,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:16:38.108540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:16:38.108540+0000","last_clean_scrub_stamp":"2024-04-07T16:16:38.108540+0000","objects_scrubbed":0,"log_size":379,"log_dups_size":0,"ondisk_log_size":379,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T22:54:47.893034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001430345,"stat_sum":{"num_bytes":196608,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":94,"num_read_kb":71471,"num_write":180,"num_write_kb":37880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":2768896,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"1be","length":"1"}]},{"pgid":"4.3","version":"0'0","reported_seq":1158,"reported_epoch":806,"state":"active+clean","last_fresh":"2024-04-07T16:29:32.097050+0000","last_change":"2024-04-07T16:25:39.004276+0000","last_active":"2024-04-07T16:29:32.097050+0000","last_peered":"2024-04-07T16:29:32.097050+0000","last_clean":"2024-04-07T16:29:32.097050+0000","last_became_active":"2024-04-07T16:25:39.003851+0000","last_became_peered":"2024-04-07T16:25:39.003851+0000","last_unstale":"2024-04-07T16:29:32.097050+0000","last_undegraded":"2024-04-07T16:29:32.097050+0000","last_fullsized":"2024-04-07T16:29:32.097050+0000","mapping_epoch":582,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":583,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:13:28.963534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:13:28.963534+0000","last_clean_scrub_stamp":"2024-04-07T16:13:28.963534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T16:54:52.299834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":967,"reported_epoch":805,"state":"active+clean","last_fresh":"2024-04-07T16:29:27.870348+0000","last_change":"2024-04-07T16:25:39.003607+0000","last_active":"2024-04-07T16:29:27.870348+0000","last_peered":"2024-04-07T16:29:27.870348+0000","last_clean":"2024-04-07T16:29:27.870348+0000","last_became_active":"2024-04-07T16:25:39.001835+0000","last_became_peered":"2024-04-07T16:25:39.001835+0000","last_unstale":"2024-04-07T16:29:27.870348+0000","last_undegraded":"2024-04-07T16:29:27.870348+0000","last_fullsized":"2024-04-07T16:29:27.870348+0000","mapping_epoch":582,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":583,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T21:54:36.255484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":1142,"reported_epoch":806,"state":"active+clean","last_fresh":"2024-04-07T16:29:31.993302+0000","last_change":"2024-04-07T16:24:41.553129+0000","last_active":"2024-04-07T16:29:31.993302+0000","last_peered":"2024-04-07T16:29:31.993302+0000","last_clean":"2024-04-07T16:29:31.993302+0000","last_became_active":"2024-04-07T16:24:41.552588+0000","last_became_peered":"2024-04-07T16:24:41.552588+0000","last_unstale":"2024-04-07T16:29:31.993302+0000","last_undegraded":"2024-04-07T16:29:31.993302+0000","last_fullsized":"2024-04-07T16:29:31.993302+0000","mapping_epoch":526,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":527,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T18:17:15.023877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"803'710","reported_seq":3587,"reported_epoch":806,"state":"active+clean","last_fresh":"2024-04-07T16:29:31.985049+0000","last_change":"2024-04-07T16:29:21.164947+0000","last_active":"2024-04-07T16:29:31.985049+0000","last_peered":"2024-04-07T16:29:31.985049+0000","last_clean":"2024-04-07T16:29:31.985049+0000","last_became_active":"2024-04-07T16:25:38.990987+0000","last_became_peered":"2024-04-07T16:25:38.990987+0000","last_unstale":"2024-04-07T16:29:31.985049+0000","last_undegraded":"2024-04-07T16:29:31.985049+0000","last_fullsized":"2024-04-07T16:29:31.985049+0000","mapping_epoch":582,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":583,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:16:38.108540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:16:38.108540+0000","last_clean_scrub_stamp":"2024-04-07T16:16:38.108540+0000","objects_scrubbed":0,"log_size":710,"log_dups_size":0,"ondisk_log_size":710,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T19:55:07.584473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.035436525000000003,"stat_sum":{"num_bytes":6627328,"num_objects":5,"num_object_clones":3,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":153,"num_read_kb":138138,"num_write":369,"num_write_kb":65584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":33013760,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":1082,"reported_epoch":806,"state":"active+clean","last_fresh":"2024-04-07T16:29:31.987276+0000","last_change":"2024-04-07T16:25:38.998655+0000","last_active":"2024-04-07T16:29:31.987276+0000","last_peered":"2024-04-07T16:29:31.987276+0000","last_clean":"2024-04-07T16:29:31.987276+0000","last_became_active":"2024-04-07T16:25:38.997956+0000","last_became_peered":"2024-04-07T16:25:38.997956+0000","last_unstale":"2024-04-07T16:29:31.987276+0000","last_undegraded":"2024-04-07T16:29:31.987276+0000","last_fullsized":"2024-04-07T16:29:31.987276+0000","mapping_epoch":582,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":583,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:13:28.963534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:13:28.963534+0000","last_clean_scrub_stamp":"2024-04-07T16:13:28.963534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T21:31:44.631562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"627'2","reported_seq":1147,"reported_epoch":806,"state":"active+clean","last_fresh":"2024-04-07T16:29:31.995450+0000","last_change":"2024-04-07T16:25:08.963320+0000","last_active":"2024-04-07T16:29:31.995450+0000","last_peered":"2024-04-07T16:29:31.995450+0000","last_clean":"2024-04-07T16:29:31.995450+0000","last_became_active":"2024-04-07T16:25:08.962773+0000","last_became_peered":"2024-04-07T16:25:08.962773+0000","last_unstale":"2024-04-07T16:29:31.995450+0000","last_undegraded":"2024-04-07T16:29:31.995450+0000","last_fullsized":"2024-04-07T16:29:31.995450+0000","mapping_epoch":553,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":554,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T18:43:13.545482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":1088,"reported_epoch":806,"state":"active+clean","last_fresh":"2024-04-07T16:29:31.988382+0000","last_change":"2024-04-07T16:25:38.974763+0000","last_active":"2024-04-07T16:29:31.988382+0000","last_peered":"2024-04-07T16:29:31.988382+0000","last_clean":"2024-04-07T16:29:31.988382+0000","last_became_active":"2024-04-07T16:25:38.974388+0000","last_became_peered":"2024-04-07T16:25:38.974388+0000","last_unstale":"2024-04-07T16:29:31.988382+0000","last_undegraded":"2024-04-07T16:29:31.988382+0000","last_fullsized":"2024-04-07T16:29:31.988382+0000","mapping_epoch":582,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":583,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T03:43:59.412072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"801'674","reported_seq":3573,"reported_epoch":806,"state":"active+clean","last_fresh":"2024-04-07T16:29:31.987302+0000","last_change":"2024-04-07T16:29:21.134078+0000","last_active":"2024-04-07T16:29:31.987302+0000","last_peered":"2024-04-07T16:29:31.987302+0000","last_clean":"2024-04-07T16:29:31.987302+0000","last_became_active":"2024-04-07T16:24:13.910496+0000","last_became_peered":"2024-04-07T16:24:13.910496+0000","last_unstale":"2024-04-07T16:29:31.987302+0000","last_undegraded":"2024-04-07T16:29:31.987302+0000","last_fullsized":"2024-04-07T16:29:31.987302+0000","mapping_epoch":499,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":500,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:16:38.108540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:16:38.108540+0000","last_clean_scrub_stamp":"2024-04-07T16:16:38.108540+0000","objects_scrubbed":0,"log_size":674,"log_dups_size":0,"ondisk_log_size":674,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T03:08:07.522748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022801029999999999,"stat_sum":{"num_bytes":13189120,"num_objects":6,"num_object_clones":5,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":132,"num_read_kb":123124,"num_write":334,"num_write_kb":93440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":9101312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":1066,"reported_epoch":806,"state":"active+clean","last_fresh":"2024-04-07T16:29:31.983644+0000","last_change":"2024-04-07T16:25:08.952896+0000","last_active":"2024-04-07T16:29:31.983644+0000","last_peered":"2024-04-07T16:29:31.983644+0000","last_clean":"2024-04-07T16:29:31.983644+0000","last_became_active":"2024-04-07T16:25:08.952433+0000","last_became_peered":"2024-04-07T16:25:08.952433+0000","last_unstale":"2024-04-07T16:29:31.983644+0000","last_undegraded":"2024-04-07T16:29:31.983644+0000","last_fullsized":"2024-04-07T16:29:31.983644+0000","mapping_epoch":553,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":554,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:13:28.963534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:13:28.963534+0000","last_clean_scrub_stamp":"2024-04-07T16:13:28.963534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T03:12:45.918416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"61'2","reported_seq":1034,"reported_epoch":805,"state":"active+clean","last_fresh":"2024-04-07T16:29:27.848732+0000","last_change":"2024-04-07T16:25:08.975487+0000","last_active":"2024-04-07T16:29:27.848732+0000","last_peered":"2024-04-07T16:29:27.848732+0000","last_clean":"2024-04-07T16:29:27.848732+0000","last_became_active":"2024-04-07T16:25:08.974754+0000","last_became_peered":"2024-04-07T16:25:08.974754+0000","last_unstale":"2024-04-07T16:29:27.848732+0000","last_undegraded":"2024-04-07T16:29:27.848732+0000","last_fullsized":"2024-04-07T16:29:27.848732+0000","mapping_epoch":553,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":554,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T01:37:46.529471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":1159,"reported_epoch":806,"state":"active+clean","last_fresh":"2024-04-07T16:29:32.096282+0000","last_change":"2024-04-07T16:24:13.963599+0000","last_active":"2024-04-07T16:29:32.096282+0000","last_peered":"2024-04-07T16:29:32.096282+0000","last_clean":"2024-04-07T16:29:32.096282+0000","last_became_active":"2024-04-07T16:24:13.962953+0000","last_became_peered":"2024-04-07T16:24:13.962953+0000","last_unstale":"2024-04-07T16:29:32.096282+0000","last_undegraded":"2024-04-07T16:29:32.096282+0000","last_fullsized":"2024-04-07T16:29:32.096282+0000","mapping_epoch":499,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":500,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T22:29:41.083754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":2762,"reported_epoch":806,"state":"active+clean","last_fresh":"2024-04-07T16:29:32.037140+0000","last_change":"2024-04-07T16:29:21.140990+0000","last_active":"2024-04-07T16:29:32.037140+0000","last_peered":"2024-04-07T16:29:32.037140+0000","last_clean":"2024-04-07T16:29:32.037140+0000","last_became_active":"2024-04-07T16:25:08.978348+0000","last_became_peered":"2024-04-07T16:25:08.978348+0000","last_unstale":"2024-04-07T16:29:32.037140+0000","last_undegraded":"2024-04-07T16:29:32.037140+0000","last_fullsized":"2024-04-07T16:29:32.037140+0000","mapping_epoch":553,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":554,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:16:38.108540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:16:38.108540+0000","last_clean_scrub_stamp":"2024-04-07T16:16:38.108540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T03:45:16.100864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002110986,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":965,"reported_epoch":805,"state":"active+clean","last_fresh":"2024-04-07T16:29:27.869423+0000","last_change":"2024-04-07T16:25:39.001597+0000","last_active":"2024-04-07T16:29:27.869423+0000","last_peered":"2024-04-07T16:29:27.869423+0000","last_clean":"2024-04-07T16:29:27.869423+0000","last_became_active":"2024-04-07T16:25:38.996008+0000","last_became_peered":"2024-04-07T16:25:38.996008+0000","last_unstale":"2024-04-07T16:29:27.869423+0000","last_undegraded":"2024-04-07T16:29:27.869423+0000","last_fullsized":"2024-04-07T16:29:27.869423+0000","mapping_epoch":582,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":583,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T19:53:15.918325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":1087,"reported_epoch":806,"state":"active+clean","last_fresh":"2024-04-07T16:29:31.983611+0000","last_change":"2024-04-07T16:25:38.985745+0000","last_active":"2024-04-07T16:29:31.983611+0000","last_peered":"2024-04-07T16:29:31.983611+0000","last_clean":"2024-04-07T16:29:31.983611+0000","last_became_active":"2024-04-07T16:25:38.985206+0000","last_became_peered":"2024-04-07T16:25:38.985206+0000","last_unstale":"2024-04-07T16:29:31.983611+0000","last_undegraded":"2024-04-07T16:29:31.983611+0000","last_fullsized":"2024-04-07T16:29:31.983611+0000","mapping_epoch":582,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":583,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T00:51:54.981164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.e","version":"803'1055","reported_seq":3899,"reported_epoch":806,"state":"active+clean","last_fresh":"2024-04-07T16:29:31.985510+0000","last_change":"2024-04-07T16:29:21.170362+0000","last_active":"2024-04-07T16:29:31.985510+0000","last_peered":"2024-04-07T16:29:31.985510+0000","last_clean":"2024-04-07T16:29:31.985510+0000","last_became_active":"2024-04-07T16:24:41.528952+0000","last_became_peered":"2024-04-07T16:24:41.528952+0000","last_unstale":"2024-04-07T16:29:31.985510+0000","last_undegraded":"2024-04-07T16:29:31.985510+0000","last_fullsized":"2024-04-07T16:29:31.985510+0000","mapping_epoch":526,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":527,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:16:38.108540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:16:38.108540+0000","last_clean_scrub_stamp":"2024-04-07T16:16:38.108540+0000","objects_scrubbed":0,"log_size":1055,"log_dups_size":0,"ondisk_log_size":1055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T03:12:31.704558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.041098705999999999,"stat_sum":{"num_bytes":19046400,"num_objects":12,"num_object_clones":9,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":1,"num_read":203,"num_read_kb":210084,"num_write":524,"num_write_kb":114032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":31473664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":1142,"reported_epoch":806,"state":"active+clean","last_fresh":"2024-04-07T16:29:31.992688+0000","last_change":"2024-04-07T16:25:38.992779+0000","last_active":"2024-04-07T16:29:31.992688+0000","last_peered":"2024-04-07T16:29:31.992688+0000","last_clean":"2024-04-07T16:29:31.992688+0000","last_became_active":"2024-04-07T16:25:38.992340+0000","last_became_peered":"2024-04-07T16:25:38.992340+0000","last_unstale":"2024-04-07T16:29:31.992688+0000","last_undegraded":"2024-04-07T16:29:31.992688+0000","last_fullsized":"2024-04-07T16:29:31.992688+0000","mapping_epoch":582,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":583,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T21:25:43.908090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":1116,"reported_epoch":806,"state":"active+clean","last_fresh":"2024-04-07T16:29:31.987581+0000","last_change":"2024-04-07T16:25:38.998502+0000","last_active":"2024-04-07T16:29:31.987581+0000","last_peered":"2024-04-07T16:29:31.987581+0000","last_clean":"2024-04-07T16:29:31.987581+0000","last_became_active":"2024-04-07T16:25:38.997582+0000","last_became_peered":"2024-04-07T16:25:38.997582+0000","last_unstale":"2024-04-07T16:29:31.987581+0000","last_undegraded":"2024-04-07T16:29:31.987581+0000","last_fullsized":"2024-04-07T16:29:31.987581+0000","mapping_epoch":582,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":583,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T23:09:31.581874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"803'1562","reported_seq":4472,"reported_epoch":806,"state":"active+clean","last_fresh":"2024-04-07T16:29:32.152122+0000","last_change":"2024-04-07T16:29:21.592496+0000","last_active":"2024-04-07T16:29:32.152122+0000","last_peered":"2024-04-07T16:29:32.152122+0000","last_clean":"2024-04-07T16:29:32.152122+0000","last_became_active":"2024-04-07T16:25:39.015159+0000","last_became_peered":"2024-04-07T16:25:39.015159+0000","last_unstale":"2024-04-07T16:29:32.152122+0000","last_undegraded":"2024-04-07T16:29:32.152122+0000","last_fullsized":"2024-04-07T16:29:32.152122+0000","mapping_epoch":582,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":583,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:16:38.108540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:16:38.108540+0000","last_clean_scrub_stamp":"2024-04-07T16:16:38.108540+0000","objects_scrubbed":0,"log_size":1562,"log_dups_size":0,"ondisk_log_size":1562,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T00:51:29.160735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.039045152,"stat_sum":{"num_bytes":14467072,"num_objects":14,"num_object_clones":10,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":320,"num_read_kb":556273,"num_write":801,"num_write_kb":145328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":36716544,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":1142,"reported_epoch":806,"state":"active+clean","last_fresh":"2024-04-07T16:29:31.990783+0000","last_change":"2024-04-07T16:25:38.976248+0000","last_active":"2024-04-07T16:29:31.990783+0000","last_peered":"2024-04-07T16:29:31.990783+0000","last_clean":"2024-04-07T16:29:31.990783+0000","last_became_active":"2024-04-07T16:25:38.975734+0000","last_became_peered":"2024-04-07T16:25:38.975734+0000","last_unstale":"2024-04-07T16:29:31.990783+0000","last_undegraded":"2024-04-07T16:29:31.990783+0000","last_fullsized":"2024-04-07T16:29:31.990783+0000","mapping_epoch":582,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":583,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T19:45:35.140357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":1088,"reported_epoch":806,"state":"active+clean","last_fresh":"2024-04-07T16:29:31.987381+0000","last_change":"2024-04-07T16:24:13.934742+0000","last_active":"2024-04-07T16:29:31.987381+0000","last_peered":"2024-04-07T16:29:31.987381+0000","last_clean":"2024-04-07T16:29:31.987381+0000","last_became_active":"2024-04-07T16:24:13.933754+0000","last_became_peered":"2024-04-07T16:24:13.933754+0000","last_unstale":"2024-04-07T16:29:31.987381+0000","last_undegraded":"2024-04-07T16:29:31.987381+0000","last_fullsized":"2024-04-07T16:29:31.987381+0000","mapping_epoch":499,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":500,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T02:43:13.536050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.c","version":"803'1137","reported_seq":4105,"reported_epoch":806,"state":"active+clean","last_fresh":"2024-04-07T16:29:31.993626+0000","last_change":"2024-04-07T16:29:21.164477+0000","last_active":"2024-04-07T16:29:31.993626+0000","last_peered":"2024-04-07T16:29:31.993626+0000","last_clean":"2024-04-07T16:29:31.993626+0000","last_became_active":"2024-04-07T16:24:13.944895+0000","last_became_peered":"2024-04-07T16:24:13.944895+0000","last_unstale":"2024-04-07T16:29:31.993626+0000","last_undegraded":"2024-04-07T16:29:31.993626+0000","last_fullsized":"2024-04-07T16:29:31.993626+0000","mapping_epoch":499,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":500,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:16:38.108540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:16:38.108540+0000","last_clean_scrub_stamp":"2024-04-07T16:16:38.108540+0000","objects_scrubbed":0,"log_size":1137,"log_dups_size":0,"ondisk_log_size":1137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T16:51:51.371511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.018391074,"stat_sum":{"num_bytes":17367040,"num_objects":14,"num_object_clones":11,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":225,"num_read_kb":316676,"num_write":715,"num_write_kb":98840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":37535744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.b","version":"623'3","reported_seq":997,"reported_epoch":805,"state":"active+clean","last_fresh":"2024-04-07T16:29:27.869622+0000","last_change":"2024-04-07T16:25:38.992642+0000","last_active":"2024-04-07T16:29:27.869622+0000","last_peered":"2024-04-07T16:29:27.869622+0000","last_clean":"2024-04-07T16:29:27.869622+0000","last_became_active":"2024-04-07T16:25:38.990309+0000","last_became_peered":"2024-04-07T16:25:38.990309+0000","last_unstale":"2024-04-07T16:29:27.869622+0000","last_undegraded":"2024-04-07T16:29:27.869622+0000","last_fullsized":"2024-04-07T16:29:27.869622+0000","mapping_epoch":582,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":583,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T17:09:27.535561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":1016,"reported_epoch":805,"state":"active+clean","last_fresh":"2024-04-07T16:29:27.847351+0000","last_change":"2024-04-07T16:25:08.966628+0000","last_active":"2024-04-07T16:29:27.847351+0000","last_peered":"2024-04-07T16:29:27.847351+0000","last_clean":"2024-04-07T16:29:27.847351+0000","last_became_active":"2024-04-07T16:25:08.966124+0000","last_became_peered":"2024-04-07T16:25:08.966124+0000","last_unstale":"2024-04-07T16:29:27.847351+0000","last_undegraded":"2024-04-07T16:29:27.847351+0000","last_fullsized":"2024-04-07T16:29:27.847351+0000","mapping_epoch":553,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":554,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T19:57:28.644470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.b","version":"803'1231","reported_seq":4185,"reported_epoch":806,"state":"active+clean","last_fresh":"2024-04-07T16:29:32.152269+0000","last_change":"2024-04-07T16:29:21.570202+0000","last_active":"2024-04-07T16:29:32.152269+0000","last_peered":"2024-04-07T16:29:32.152269+0000","last_clean":"2024-04-07T16:29:32.152269+0000","last_became_active":"2024-04-07T16:24:41.564012+0000","last_became_peered":"2024-04-07T16:24:41.564012+0000","last_unstale":"2024-04-07T16:29:32.152269+0000","last_undegraded":"2024-04-07T16:29:32.152269+0000","last_fullsized":"2024-04-07T16:29:32.152269+0000","mapping_epoch":526,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":527,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:16:38.108540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:16:38.108540+0000","last_clean_scrub_stamp":"2024-04-07T16:16:38.108540+0000","objects_scrubbed":0,"log_size":1231,"log_dups_size":0,"ondisk_log_size":1231,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T21:19:39.236990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017868973999999999,"stat_sum":{"num_bytes":12353536,"num_objects":13,"num_object_clones":9,"num_object_copies":39,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":287,"num_read_kb":438753,"num_write":649,"num_write_kb":109120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":64266240,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":1136,"reported_epoch":806,"state":"active+clean","last_fresh":"2024-04-07T16:29:32.152629+0000","last_change":"2024-04-07T16:24:41.550837+0000","last_active":"2024-04-07T16:29:32.152629+0000","last_peered":"2024-04-07T16:29:32.152629+0000","last_clean":"2024-04-07T16:29:32.152629+0000","last_became_active":"2024-04-07T16:24:41.549188+0000","last_became_peered":"2024-04-07T16:24:41.549188+0000","last_unstale":"2024-04-07T16:29:32.152629+0000","last_undegraded":"2024-04-07T16:29:32.152629+0000","last_fullsized":"2024-04-07T16:29:32.152629+0000","mapping_epoch":526,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":527,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T18:56:44.543940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":977,"reported_epoch":805,"state":"active+clean","last_fresh":"2024-04-07T16:29:27.870698+0000","last_change":"2024-04-07T16:25:39.000662+0000","last_active":"2024-04-07T16:29:27.870698+0000","last_peered":"2024-04-07T16:29:27.870698+0000","last_clean":"2024-04-07T16:29:27.870698+0000","last_became_active":"2024-04-07T16:25:38.989051+0000","last_became_peered":"2024-04-07T16:25:38.989051+0000","last_unstale":"2024-04-07T16:29:27.870698+0000","last_undegraded":"2024-04-07T16:29:27.870698+0000","last_fullsized":"2024-04-07T16:29:27.870698+0000","mapping_epoch":582,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":583,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T23:45:21.166051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.a","version":"803'2041","reported_seq":5012,"reported_epoch":806,"state":"active+clean","last_fresh":"2024-04-07T16:29:32.151681+0000","last_change":"2024-04-07T16:29:21.581521+0000","last_active":"2024-04-07T16:29:32.151681+0000","last_peered":"2024-04-07T16:29:32.151681+0000","last_clean":"2024-04-07T16:29:32.151681+0000","last_became_active":"2024-04-07T16:24:13.955741+0000","last_became_peered":"2024-04-07T16:24:13.955741+0000","last_unstale":"2024-04-07T16:29:32.151681+0000","last_undegraded":"2024-04-07T16:29:32.151681+0000","last_fullsized":"2024-04-07T16:29:32.151681+0000","mapping_epoch":499,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":500,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:16:38.108540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:16:38.108540+0000","last_clean_scrub_stamp":"2024-04-07T16:16:38.108540+0000","objects_scrubbed":0,"log_size":2041,"log_dups_size":0,"ondisk_log_size":2041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T16:23:05.940515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028996694,"stat_sum":{"num_bytes":45867008,"num_objects":19,"num_object_clones":13,"num_object_copies":57,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":520,"num_read_kb":510676,"num_write":1169,"num_write_kb":245256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":25640960,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"61'1","reported_seq":1145,"reported_epoch":806,"state":"active+clean","last_fresh":"2024-04-07T16:29:31.994904+0000","last_change":"2024-04-07T16:24:13.933216+0000","last_active":"2024-04-07T16:29:31.994904+0000","last_peered":"2024-04-07T16:29:31.994904+0000","last_clean":"2024-04-07T16:29:31.994904+0000","last_became_active":"2024-04-07T16:24:13.932662+0000","last_became_peered":"2024-04-07T16:24:13.932662+0000","last_unstale":"2024-04-07T16:29:31.994904+0000","last_undegraded":"2024-04-07T16:29:31.994904+0000","last_fullsized":"2024-04-07T16:29:31.994904+0000","mapping_epoch":499,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":500,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T17:17:11.134012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":1052,"reported_epoch":806,"state":"active+clean","last_fresh":"2024-04-07T16:29:32.036460+0000","last_change":"2024-04-07T16:25:08.973166+0000","last_active":"2024-04-07T16:29:32.036460+0000","last_peered":"2024-04-07T16:29:32.036460+0000","last_clean":"2024-04-07T16:29:32.036460+0000","last_became_active":"2024-04-07T16:25:08.971755+0000","last_became_peered":"2024-04-07T16:25:08.971755+0000","last_unstale":"2024-04-07T16:29:32.036460+0000","last_undegraded":"2024-04-07T16:29:32.036460+0000","last_fullsized":"2024-04-07T16:29:32.036460+0000","mapping_epoch":553,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":554,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T02:59:49.583756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"803'804","reported_seq":3476,"reported_epoch":805,"state":"active+clean","last_fresh":"2024-04-07T16:29:27.870494+0000","last_change":"2024-04-07T16:29:21.165741+0000","last_active":"2024-04-07T16:29:27.870494+0000","last_peered":"2024-04-07T16:29:27.870494+0000","last_clean":"2024-04-07T16:29:27.870494+0000","last_became_active":"2024-04-07T16:25:38.992766+0000","last_became_peered":"2024-04-07T16:25:38.992766+0000","last_unstale":"2024-04-07T16:29:27.870494+0000","last_undegraded":"2024-04-07T16:29:27.870494+0000","last_fullsized":"2024-04-07T16:29:27.870494+0000","mapping_epoch":582,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":583,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:16:38.108540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:16:38.108540+0000","last_clean_scrub_stamp":"2024-04-07T16:16:38.108540+0000","objects_scrubbed":0,"log_size":804,"log_dups_size":0,"ondisk_log_size":804,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T22:28:34.073808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.026693801,"stat_sum":{"num_bytes":8388608,"num_objects":7,"num_object_clones":5,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":145,"num_read_kb":147859,"num_write":332,"num_write_kb":81656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":12738560,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"1be","length":"1"}]},{"pgid":"2.e","version":"61'2","reported_seq":1134,"reported_epoch":806,"state":"active+clean","last_fresh":"2024-04-07T16:29:32.154062+0000","last_change":"2024-04-07T16:25:38.968390+0000","last_active":"2024-04-07T16:29:32.154062+0000","last_peered":"2024-04-07T16:29:32.154062+0000","last_clean":"2024-04-07T16:29:32.154062+0000","last_became_active":"2024-04-07T16:25:38.967914+0000","last_became_peered":"2024-04-07T16:25:38.967914+0000","last_unstale":"2024-04-07T16:29:32.154062+0000","last_undegraded":"2024-04-07T16:29:32.154062+0000","last_fullsized":"2024-04-07T16:29:32.154062+0000","mapping_epoch":582,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":583,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:11.994648+0000","last_clean_scrub_stamp":"2024-04-07T16:12:11.994648+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T21:28:08.591091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":965,"reported_epoch":805,"state":"active+clean","last_fresh":"2024-04-07T16:29:27.870884+0000","last_change":"2024-04-07T16:25:38.994496+0000","last_active":"2024-04-07T16:29:27.870884+0000","last_peered":"2024-04-07T16:29:27.870884+0000","last_clean":"2024-04-07T16:29:27.870884+0000","last_became_active":"2024-04-07T16:25:38.976388+0000","last_became_peered":"2024-04-07T16:25:38.976388+0000","last_unstale":"2024-04-07T16:29:27.870884+0000","last_undegraded":"2024-04-07T16:29:27.870884+0000","last_fullsized":"2024-04-07T16:29:27.870884+0000","mapping_epoch":582,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":583,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T17:59:57.662112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":1001,"reported_epoch":805,"state":"active+clean","last_fresh":"2024-04-07T16:29:27.846971+0000","last_change":"2024-04-07T16:25:08.976755+0000","last_active":"2024-04-07T16:29:27.846971+0000","last_peered":"2024-04-07T16:29:27.846971+0000","last_clean":"2024-04-07T16:29:27.846971+0000","last_became_active":"2024-04-07T16:25:08.975767+0000","last_became_peered":"2024-04-07T16:25:08.975767+0000","last_unstale":"2024-04-07T16:29:27.846971+0000","last_undegraded":"2024-04-07T16:29:27.846971+0000","last_fullsized":"2024-04-07T16:29:27.846971+0000","mapping_epoch":553,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":554,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T16:33:00.288075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":1131,"reported_epoch":806,"state":"active+clean","last_fresh":"2024-04-07T16:29:32.151672+0000","last_change":"2024-04-07T16:25:08.976897+0000","last_active":"2024-04-07T16:29:32.151672+0000","last_peered":"2024-04-07T16:29:32.151672+0000","last_clean":"2024-04-07T16:29:32.151672+0000","last_became_active":"2024-04-07T16:25:08.976322+0000","last_became_peered":"2024-04-07T16:25:08.976322+0000","last_unstale":"2024-04-07T16:29:32.151672+0000","last_undegraded":"2024-04-07T16:29:32.151672+0000","last_fullsized":"2024-04-07T16:29:32.151672+0000","mapping_epoch":553,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":554,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T18:02:01.757959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":1058,"reported_epoch":806,"state":"active+clean","last_fresh":"2024-04-07T16:29:32.036030+0000","last_change":"2024-04-07T16:24:41.564435+0000","last_active":"2024-04-07T16:29:32.036030+0000","last_peered":"2024-04-07T16:29:32.036030+0000","last_clean":"2024-04-07T16:29:32.036030+0000","last_became_active":"2024-04-07T16:24:41.561766+0000","last_became_peered":"2024-04-07T16:24:41.561766+0000","last_unstale":"2024-04-07T16:29:32.036030+0000","last_undegraded":"2024-04-07T16:29:32.036030+0000","last_fullsized":"2024-04-07T16:29:32.036030+0000","mapping_epoch":526,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":527,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T17:08:34.046549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,0],"acting":[5,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":1159,"reported_epoch":806,"state":"active+clean","last_fresh":"2024-04-07T16:29:32.096188+0000","last_change":"2024-04-07T16:24:13.964515+0000","last_active":"2024-04-07T16:29:32.096188+0000","last_peered":"2024-04-07T16:29:32.096188+0000","last_clean":"2024-04-07T16:29:32.096188+0000","last_became_active":"2024-04-07T16:24:13.962893+0000","last_became_peered":"2024-04-07T16:24:13.962893+0000","last_unstale":"2024-04-07T16:29:32.096188+0000","last_undegraded":"2024-04-07T16:29:32.096188+0000","last_fullsized":"2024-04-07T16:29:32.096188+0000","mapping_epoch":499,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":500,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T17:30:33.589121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":1088,"reported_epoch":806,"state":"active+clean","last_fresh":"2024-04-07T16:29:31.987349+0000","last_change":"2024-04-07T16:24:13.931275+0000","last_active":"2024-04-07T16:29:31.987349+0000","last_peered":"2024-04-07T16:29:31.987349+0000","last_clean":"2024-04-07T16:29:31.987349+0000","last_became_active":"2024-04-07T16:24:13.930420+0000","last_became_peered":"2024-04-07T16:24:13.930420+0000","last_unstale":"2024-04-07T16:29:31.987349+0000","last_undegraded":"2024-04-07T16:29:31.987349+0000","last_fullsized":"2024-04-07T16:29:31.987349+0000","mapping_epoch":499,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":500,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T00:26:09.028937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,0],"acting":[3,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":1142,"reported_epoch":806,"state":"active+clean","last_fresh":"2024-04-07T16:29:31.995119+0000","last_change":"2024-04-07T16:23:43.926295+0000","last_active":"2024-04-07T16:29:31.995119+0000","last_peered":"2024-04-07T16:29:31.995119+0000","last_clean":"2024-04-07T16:29:31.995119+0000","last_became_active":"2024-04-07T16:23:43.924498+0000","last_became_peered":"2024-04-07T16:23:43.924498+0000","last_unstale":"2024-04-07T16:29:31.995119+0000","last_undegraded":"2024-04-07T16:29:31.995119+0000","last_fullsized":"2024-04-07T16:29:31.995119+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T17:58:42.938076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":1010,"reported_epoch":805,"state":"active+clean","last_fresh":"2024-04-07T16:29:27.848977+0000","last_change":"2024-04-07T16:25:38.962091+0000","last_active":"2024-04-07T16:29:27.848977+0000","last_peered":"2024-04-07T16:29:27.848977+0000","last_clean":"2024-04-07T16:29:27.848977+0000","last_became_active":"2024-04-07T16:25:38.960595+0000","last_became_peered":"2024-04-07T16:25:38.960595+0000","last_unstale":"2024-04-07T16:29:27.848977+0000","last_undegraded":"2024-04-07T16:29:27.848977+0000","last_fullsized":"2024-04-07T16:29:27.848977+0000","mapping_epoch":582,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":583,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T19:11:01.462021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,4],"acting":[6,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":1102,"reported_epoch":806,"state":"active+clean","last_fresh":"2024-04-07T16:29:31.988776+0000","last_change":"2024-04-07T16:24:13.934646+0000","last_active":"2024-04-07T16:29:31.988776+0000","last_peered":"2024-04-07T16:29:31.988776+0000","last_clean":"2024-04-07T16:29:31.988776+0000","last_became_active":"2024-04-07T16:24:13.933691+0000","last_became_peered":"2024-04-07T16:24:13.933691+0000","last_unstale":"2024-04-07T16:29:31.988776+0000","last_undegraded":"2024-04-07T16:29:31.988776+0000","last_fullsized":"2024-04-07T16:29:31.988776+0000","mapping_epoch":499,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":500,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T19:33:33.960559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":1058,"reported_epoch":806,"state":"active+clean","last_fresh":"2024-04-07T16:29:32.035341+0000","last_change":"2024-04-07T16:24:41.557281+0000","last_active":"2024-04-07T16:29:32.035341+0000","last_peered":"2024-04-07T16:29:32.035341+0000","last_clean":"2024-04-07T16:29:32.035341+0000","last_became_active":"2024-04-07T16:24:41.553450+0000","last_became_peered":"2024-04-07T16:24:41.553450+0000","last_unstale":"2024-04-07T16:29:32.035341+0000","last_undegraded":"2024-04-07T16:29:32.035341+0000","last_fullsized":"2024-04-07T16:29:32.035341+0000","mapping_epoch":526,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":527,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T19:49:37.795766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":1116,"reported_epoch":806,"state":"active+clean","last_fresh":"2024-04-07T16:29:31.988105+0000","last_change":"2024-04-07T16:24:13.933359+0000","last_active":"2024-04-07T16:29:31.988105+0000","last_peered":"2024-04-07T16:29:31.988105+0000","last_clean":"2024-04-07T16:29:31.988105+0000","last_became_active":"2024-04-07T16:24:13.933027+0000","last_became_peered":"2024-04-07T16:24:13.933027+0000","last_unstale":"2024-04-07T16:29:31.988105+0000","last_undegraded":"2024-04-07T16:29:31.988105+0000","last_fullsized":"2024-04-07T16:29:31.988105+0000","mapping_epoch":499,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":500,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T00:08:14.265846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":1046,"reported_epoch":806,"state":"active+clean","last_fresh":"2024-04-07T16:29:32.035381+0000","last_change":"2024-04-07T16:25:38.978928+0000","last_active":"2024-04-07T16:29:32.035381+0000","last_peered":"2024-04-07T16:29:32.035381+0000","last_clean":"2024-04-07T16:29:32.035381+0000","last_became_active":"2024-04-07T16:25:38.977861+0000","last_became_peered":"2024-04-07T16:25:38.977861+0000","last_unstale":"2024-04-07T16:29:32.035381+0000","last_undegraded":"2024-04-07T16:29:32.035381+0000","last_fullsized":"2024-04-07T16:29:32.035381+0000","mapping_epoch":582,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":583,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T01:59:28.849179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":1159,"reported_epoch":806,"state":"active+clean","last_fresh":"2024-04-07T16:29:32.095891+0000","last_change":"2024-04-07T16:25:39.004542+0000","last_active":"2024-04-07T16:29:32.095891+0000","last_peered":"2024-04-07T16:29:32.095891+0000","last_clean":"2024-04-07T16:29:32.095891+0000","last_became_active":"2024-04-07T16:25:39.003945+0000","last_became_peered":"2024-04-07T16:25:39.003945+0000","last_unstale":"2024-04-07T16:29:32.095891+0000","last_undegraded":"2024-04-07T16:29:32.095891+0000","last_fullsized":"2024-04-07T16:29:32.095891+0000","mapping_epoch":582,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":583,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-08T18:56:04.984002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1],"acting":[0,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":1015,"reported_epoch":805,"state":"active+clean","last_fresh":"2024-04-07T16:29:27.847808+0000","last_change":"2024-04-07T16:25:08.963061+0000","last_active":"2024-04-07T16:29:27.847808+0000","last_peered":"2024-04-07T16:29:27.847808+0000","last_clean":"2024-04-07T16:29:27.847808+0000","last_became_active":"2024-04-07T16:25:08.962622+0000","last_became_peered":"2024-04-07T16:25:08.962622+0000","last_unstale":"2024-04-07T16:29:27.847808+0000","last_undegraded":"2024-04-07T16:29:27.847808+0000","last_fullsized":"2024-04-07T16:29:27.847808+0000","mapping_epoch":553,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":554,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-07T16:12:12.996208+0000","last_clean_scrub_stamp":"2024-04-07T16:12:12.996208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-09T02:53:55.780425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":16,"stat_sum":{"num_bytes":241065984,"num_objects":165,"num_object_clones":117,"num_object_copies":495,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":165,"num_whiteouts":7,"num_read":3836,"num_read_kb":5027160,"num_write":8896,"num_write_kb":1747232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":140,"num_bytes_recovered":442564608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":336973824,"data_stored":583766016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17353,"ondisk_log_size":17353,"up":48,"acting":48,"num_store_stats":8},{"poolid":4,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1034,"num_read_kb":1874,"num_write":359,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":221,"ondisk_log_size":221,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":6622,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":79,"num_read_kb":87,"num_write":68,"num_write_kb":36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":19866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":48581,"internal_metadata":0},"log_size":68,"ondisk_log_size":68,"up":96,"acting":96,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":582,"seq":2499670966320,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":107708,"kb_used_data":31024,"kb_used_omap":28,"kb_used_meta":76643,"kb_avail":93637444,"statfs":{"total":95995035648,"available":95884742656,"internally_reserved":0,"allocated":31768576,"data_stored":44910661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":29588,"internal_metadata":78482540},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":553,"seq":2375116914742,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":82284,"kb_used_data":27296,"kb_used_omap":21,"kb_used_meta":54954,"kb_avail":93662868,"statfs":{"total":95995035648,"available":95910776832,"internally_reserved":0,"allocated":27951104,"data_stored":52407465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":22383,"internal_metadata":56273041},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":416,"seq":1786706395219,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":374532,"kb_used_data":62840,"kb_used_omap":33,"kb_used_meta":311646,"kb_avail":93370620,"statfs":{"total":95995035648,"available":95611514880,"internally_reserved":0,"allocated":64348160,"data_stored":90071098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":34587,"internal_metadata":319125733},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":389,"seq":1670742278232,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":513556,"kb_used_data":74572,"kb_used_omap":36,"kb_used_meta":438939,"kb_avail":93231596,"statfs":{"total":95995035648,"available":95469154304,"internally_reserved":0,"allocated":76361728,"data_stored":126392513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":37735,"internal_metadata":449473689},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":444,"seq":1906965479501,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":288484,"kb_used_data":64984,"kb_used_omap":44,"kb_used_meta":223443,"kb_avail":93456668,"statfs":{"total":95995035648,"available":95699628032,"internally_reserved":0,"allocated":66543616,"data_stored":99575582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":45175,"internal_metadata":228806537},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":470,"seq":2018634629191,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":173176,"kb_used_data":49516,"kb_used_omap":34,"kb_used_meta":123613,"kb_avail":93571976,"statfs":{"total":95995035648,"available":95817703424,"internally_reserved":0,"allocated":50704384,"data_stored":77085425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":35023,"internal_metadata":126580529},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":499,"seq":2143188680769,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":282996,"kb_used_data":60968,"kb_used_omap":46,"kb_used_meta":221969,"kb_avail":93462156,"statfs":{"total":95995035648,"available":95705247744,"internally_reserved":0,"allocated":62431232,"data_stored":76818061,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":47602,"internal_metadata":227296782},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":526,"seq":2259152797756,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":245856,"kb_used_data":54676,"kb_used_omap":46,"kb_used_meta":191121,"kb_avail":93499296,"statfs":{"total":95995035648,"available":95743279104,"internally_reserved":0,"allocated":55988224,"data_stored":74919149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":47325,"internal_metadata":195708707},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5862,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4736,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6174,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":5928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6309,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6691,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":5914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5829,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":5824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6317,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6663,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":63602688,"data_stored":118726656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":52178944,"data_stored":82989056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":54378496,"data_stored":92491776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":38531072,"data_stored":70004736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":50266112,"data_stored":69742592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":43806720,"data_stored":67829760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":15192064,"data_stored":44736512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":19017728,"data_stored":37244928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-07T16:29:38.072 INFO:tasks.cephadm.ceph_manager.ceph:removing pool_name unique_pool_0 2024-04-07T16:29:38.072 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 0702e3a8-f4f8-11ee-b649-cb9ed24678a4 -- ceph osd pool rm unique_pool_0 unique_pool_0 --yes-i-really-really-mean-it 2024-04-07T16:29:38.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:29:38 smithi144 bash[66654]: cluster 2024-04-07T16:29:36.846301+0000 mgr.y (mgr.25175) 427 : cluster [DBG] pgmap v809: 121 pgs: 121 active+clean; 230 MiB data, 2.0 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:29:38.417 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:29:38 smithi100 bash[74942]: cluster 2024-04-07T16:29:36.846301+0000 mgr.y (mgr.25175) 427 : cluster [DBG] pgmap v809: 121 pgs: 121 active+clean; 230 MiB data, 2.0 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:29:38.418 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:29:38 smithi100 bash[76202]: cluster 2024-04-07T16:29:36.846301+0000 mgr.y (mgr.25175) 427 : cluster [DBG] pgmap v809: 121 pgs: 121 active+clean; 230 MiB data, 2.0 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:29:38.689 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:29:38 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:29:38] "GET /metrics HTTP/1.1" 200 36334 "" "Prometheus/2.43.0" 2024-04-07T16:29:39.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:29:39 smithi144 bash[66654]: audit 2024-04-07T16:29:37.410746+0000 mgr.y (mgr.25175) 428 : audit [DBG] from='client.44447 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-07T16:29:39.439 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:29:39 smithi100 bash[74942]: audit 2024-04-07T16:29:37.410746+0000 mgr.y (mgr.25175) 428 : audit [DBG] from='client.44447 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-07T16:29:39.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:29:39 smithi100 bash[76202]: audit 2024-04-07T16:29:37.410746+0000 mgr.y (mgr.25175) 428 : audit [DBG] from='client.44447 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-07T16:29:40.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:29:40 smithi144 bash[66654]: cluster 2024-04-07T16:29:38.847454+0000 mgr.y (mgr.25175) 429 : cluster [DBG] pgmap v810: 121 pgs: 121 active+clean; 230 MiB data, 2.0 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:29:40.439 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:29:40 smithi100 bash[74942]: cluster 2024-04-07T16:29:38.847454+0000 mgr.y (mgr.25175) 429 : cluster [DBG] pgmap v810: 121 pgs: 121 active+clean; 230 MiB data, 2.0 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:29:40.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:29:40 smithi100 bash[76202]: cluster 2024-04-07T16:29:38.847454+0000 mgr.y (mgr.25175) 429 : cluster [DBG] pgmap v810: 121 pgs: 121 active+clean; 230 MiB data, 2.0 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:29:42.304 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:29:42 smithi144 bash[66654]: cluster 2024-04-07T16:29:40.848458+0000 mgr.y (mgr.25175) 430 : cluster [DBG] pgmap v811: 121 pgs: 121 active+clean; 230 MiB data, 2.0 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:29:42.439 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:29:42 smithi100 bash[74942]: cluster 2024-04-07T16:29:40.848458+0000 mgr.y (mgr.25175) 430 : cluster [DBG] pgmap v811: 121 pgs: 121 active+clean; 230 MiB data, 2.0 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:29:42.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:29:42 smithi100 bash[76202]: cluster 2024-04-07T16:29:40.848458+0000 mgr.y (mgr.25175) 430 : cluster [DBG] pgmap v811: 121 pgs: 121 active+clean; 230 MiB data, 2.0 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:29:42.606 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:29:42 smithi144 bash[95233]: ts=2024-04-07T16:29:42.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:29:42.801 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/mon.a/config 2024-04-07T16:29:44.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:29:44 smithi144 bash[66654]: cluster 2024-04-07T16:29:42.849500+0000 mgr.y (mgr.25175) 431 : cluster [DBG] pgmap v812: 121 pgs: 121 active+clean; 230 MiB data, 2.0 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:29:44.439 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:29:44 smithi100 bash[74942]: cluster 2024-04-07T16:29:42.849500+0000 mgr.y (mgr.25175) 431 : cluster [DBG] pgmap v812: 121 pgs: 121 active+clean; 230 MiB data, 2.0 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:29:44.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:29:44 smithi100 bash[76202]: cluster 2024-04-07T16:29:42.849500+0000 mgr.y (mgr.25175) 431 : cluster [DBG] pgmap v812: 121 pgs: 121 active+clean; 230 MiB data, 2.0 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:29:46.033 INFO:teuthology.orchestra.run.smithi100.stderr:pool 'unique_pool_0' removed 2024-04-07T16:29:46.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:29:46 smithi144 bash[66654]: cluster 2024-04-07T16:29:44.850537+0000 mgr.y (mgr.25175) 432 : cluster [DBG] pgmap v813: 121 pgs: 121 active+clean; 230 MiB data, 2.0 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:29:46.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:29:46 smithi144 bash[66654]: audit 2024-04-07T16:29:45.654902+0000 mon.a (mon.0) 1396 : audit [INF] from='client.? 172.21.15.100:0/2426276561' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_0", "pool2": "unique_pool_0", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-07T16:29:46.439 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:29:46 smithi100 bash[74942]: cluster 2024-04-07T16:29:44.850537+0000 mgr.y (mgr.25175) 432 : cluster [DBG] pgmap v813: 121 pgs: 121 active+clean; 230 MiB data, 2.0 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:29:46.439 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:29:46 smithi100 bash[74942]: audit 2024-04-07T16:29:45.654902+0000 mon.a (mon.0) 1396 : audit [INF] from='client.? 172.21.15.100:0/2426276561' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_0", "pool2": "unique_pool_0", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-07T16:29:46.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:29:46 smithi100 bash[76202]: cluster 2024-04-07T16:29:44.850537+0000 mgr.y (mgr.25175) 432 : cluster [DBG] pgmap v813: 121 pgs: 121 active+clean; 230 MiB data, 2.0 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:29:46.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:29:46 smithi100 bash[76202]: audit 2024-04-07T16:29:45.654902+0000 mon.a (mon.0) 1396 : audit [INF] from='client.? 172.21.15.100:0/2426276561' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_0", "pool2": "unique_pool_0", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-07T16:29:46.786 INFO:teuthology.task.full_sequential:In full_sequential, running task print... 2024-04-07T16:29:46.787 INFO:teuthology.task.print:**** done end ec-rados-default.yaml 2024-04-07T16:29:46.787 INFO:teuthology.task.full_sequential:In full_sequential, running task print... 2024-04-07T16:29:46.787 INFO:teuthology.task.print:**** done start rados_api.yaml 2024-04-07T16:29:46.787 INFO:teuthology.task.full_sequential:In full_sequential, running task workunit... 2024-04-07T16:29:46.787 INFO:tasks.workunit:Pulling workunits from ref reef 2024-04-07T16:29:46.788 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-04-07T16:29:46.788 DEBUG:teuthology.orchestra.run.smithi100:> stat -- /home/ubuntu/cephtest/mnt.0 2024-04-07T16:29:46.793 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-07T16:29:46.794 INFO:teuthology.orchestra.run.smithi100.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-04-07T16:29:46.794 DEBUG:teuthology.orchestra.run.smithi100:> mkdir -- /home/ubuntu/cephtest/mnt.0 2024-04-07T16:29:46.842 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2024-04-07T16:29:46.842 DEBUG:teuthology.orchestra.run.smithi100:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2024-04-07T16:29:46.889 INFO:tasks.workunit:timeout=3h 2024-04-07T16:29:46.889 INFO:tasks.workunit:cleanup=True 2024-04-07T16:29:46.890 DEBUG:teuthology.orchestra.run.smithi100:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone --depth 1 --branch reef https://github.com/ceph/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 2024-04-07T16:29:46.940 INFO:tasks.workunit.client.0.smithi100.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-04-07T16:29:47.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:29:47 smithi144 bash[66654]: audit 2024-04-07T16:29:46.041904+0000 mon.a (mon.0) 1397 : audit [INF] from='client.? 172.21.15.100:0/2426276561' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_0", "pool2": "unique_pool_0", "yes_i_really_really_mean_it": true}]': finished 2024-04-07T16:29:47.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:29:47 smithi144 bash[66654]: cluster 2024-04-07T16:29:46.050370+0000 mon.a (mon.0) 1398 : cluster [DBG] osdmap e807: 8 total, 8 up, 8 in 2024-04-07T16:29:47.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:29:47 smithi144 bash[66654]: audit 2024-04-07T16:29:46.737838+0000 mon.a (mon.0) 1399 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:29:47.439 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:29:47 smithi100 bash[74942]: audit 2024-04-07T16:29:46.041904+0000 mon.a (mon.0) 1397 : audit [INF] from='client.? 172.21.15.100:0/2426276561' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_0", "pool2": "unique_pool_0", "yes_i_really_really_mean_it": true}]': finished 2024-04-07T16:29:47.439 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:29:47 smithi100 bash[74942]: cluster 2024-04-07T16:29:46.050370+0000 mon.a (mon.0) 1398 : cluster [DBG] osdmap e807: 8 total, 8 up, 8 in 2024-04-07T16:29:47.439 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:29:47 smithi100 bash[74942]: audit 2024-04-07T16:29:46.737838+0000 mon.a (mon.0) 1399 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:29:47.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:29:47 smithi100 bash[76202]: audit 2024-04-07T16:29:46.041904+0000 mon.a (mon.0) 1397 : audit [INF] from='client.? 172.21.15.100:0/2426276561' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_0", "pool2": "unique_pool_0", "yes_i_really_really_mean_it": true}]': finished 2024-04-07T16:29:47.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:29:47 smithi100 bash[76202]: cluster 2024-04-07T16:29:46.050370+0000 mon.a (mon.0) 1398 : cluster [DBG] osdmap e807: 8 total, 8 up, 8 in 2024-04-07T16:29:47.440 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:29:47 smithi100 bash[76202]: audit 2024-04-07T16:29:46.737838+0000 mon.a (mon.0) 1399 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:29:48.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:29:48 smithi144 bash[66654]: cluster 2024-04-07T16:29:46.851239+0000 mgr.y (mgr.25175) 433 : cluster [DBG] pgmap v815: 105 pgs: 105 active+clean; 583 KiB data, 2.0 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:29:48.417 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:29:48 smithi100 bash[74942]: cluster 2024-04-07T16:29:46.851239+0000 mgr.y (mgr.25175) 433 : cluster [DBG] pgmap v815: 105 pgs: 105 active+clean; 583 KiB data, 2.0 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:29:48.417 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:29:48 smithi100 bash[76202]: cluster 2024-04-07T16:29:46.851239+0000 mgr.y (mgr.25175) 433 : cluster [DBG] pgmap v815: 105 pgs: 105 active+clean; 583 KiB data, 2.0 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:29:48.689 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:29:48 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:29:48] "GET /metrics HTTP/1.1" 200 33991 "" "Prometheus/2.43.0" 2024-04-07T16:29:50.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:29:50 smithi144 bash[66654]: cluster 2024-04-07T16:29:48.852458+0000 mgr.y (mgr.25175) 434 : cluster [DBG] pgmap v816: 105 pgs: 105 active+clean; 583 KiB data, 1.8 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:29:50.439 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:29:50 smithi100 bash[74942]: cluster 2024-04-07T16:29:48.852458+0000 mgr.y (mgr.25175) 434 : cluster [DBG] pgmap v816: 105 pgs: 105 active+clean; 583 KiB data, 1.8 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:29:50.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:29:50 smithi100 bash[76202]: cluster 2024-04-07T16:29:48.852458+0000 mgr.y (mgr.25175) 434 : cluster [DBG] pgmap v816: 105 pgs: 105 active+clean; 583 KiB data, 1.8 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:29:52.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:29:52 smithi144 bash[66654]: cluster 2024-04-07T16:29:50.853412+0000 mgr.y (mgr.25175) 435 : cluster [DBG] pgmap v817: 105 pgs: 105 active+clean; 583 KiB data, 1.7 GiB used, 714 GiB / 715 GiB avail 2024-04-07T16:29:52.357 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:29:52 smithi144 bash[95233]: ts=2024-04-07T16:29:52.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:29:52.439 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:29:52 smithi100 bash[74942]: cluster 2024-04-07T16:29:50.853412+0000 mgr.y (mgr.25175) 435 : cluster [DBG] pgmap v817: 105 pgs: 105 active+clean; 583 KiB data, 1.7 GiB used, 714 GiB / 715 GiB avail 2024-04-07T16:29:52.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:29:52 smithi100 bash[76202]: cluster 2024-04-07T16:29:50.853412+0000 mgr.y (mgr.25175) 435 : cluster [DBG] pgmap v817: 105 pgs: 105 active+clean; 583 KiB data, 1.7 GiB used, 714 GiB / 715 GiB avail 2024-04-07T16:29:53.083 DEBUG:teuthology.orchestra.run.smithi100:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-04-07T16:29:53.131 INFO:tasks.workunit.client.0.smithi100.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-04-07T16:29:53.134 INFO:tasks.workunit.client.0.smithi100.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-07T16:29:53.134 INFO:tasks.workunit.client.0.smithi100.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-04-07T16:29:53.189 INFO:tasks.workunit.client.0.smithi100.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-04-07T16:29:53.238 INFO:tasks.workunit.client.0.smithi100.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-04-07T16:29:53.280 INFO:tasks.workunit.client.0.smithi100.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-07T16:29:53.282 INFO:tasks.workunit.client.0.smithi100.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-07T16:29:53.282 INFO:tasks.workunit.client.0.smithi100.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-04-07T16:29:53.326 INFO:tasks.workunit.client.0.smithi100.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-07T16:29:53.330 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-07T16:29:53.330 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-04-07T16:29:53.381 INFO:tasks.workunit:Running workunits matching cls on client.0... 2024-04-07T16:29:53.382 INFO:tasks.workunit:Running workunit cls/test_cls_2pc_queue.sh... 2024-04-07T16:29:53.382 DEBUG:teuthology.orchestra.run.smithi100:workunit test cls/test_cls_2pc_queue.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=reef TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_2pc_queue.sh 2024-04-07T16:29:53.445 INFO:tasks.workunit.client.0.smithi100.stdout:Running main() from gmock_main.cc 2024-04-07T16:29:53.445 INFO:tasks.workunit.client.0.smithi100.stdout:[==========] Running 22 tests from 1 test suite. 2024-04-07T16:29:53.445 INFO:tasks.workunit.client.0.smithi100.stdout:[----------] Global test environment set-up. 2024-04-07T16:29:53.445 INFO:tasks.workunit.client.0.smithi100.stdout:[----------] 22 tests from TestCls2PCQueue 2024-04-07T16:29:53.445 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestCls2PCQueue.GetCapacity 2024-04-07T16:29:54.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:29:54 smithi144 bash[66654]: cluster 2024-04-07T16:29:52.854421+0000 mgr.y (mgr.25175) 436 : cluster [DBG] pgmap v818: 105 pgs: 105 active+clean; 583 KiB data, 1.7 GiB used, 714 GiB / 715 GiB avail 2024-04-07T16:29:54.439 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:29:54 smithi100 bash[74942]: cluster 2024-04-07T16:29:52.854421+0000 mgr.y (mgr.25175) 436 : cluster [DBG] pgmap v818: 105 pgs: 105 active+clean; 583 KiB data, 1.7 GiB used, 714 GiB / 715 GiB avail 2024-04-07T16:29:54.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:29:54 smithi100 bash[76202]: cluster 2024-04-07T16:29:52.854421+0000 mgr.y (mgr.25175) 436 : cluster [DBG] pgmap v818: 105 pgs: 105 active+clean; 583 KiB data, 1.7 GiB used, 714 GiB / 715 GiB avail 2024-04-07T16:29:55.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:29:55 smithi144 bash[66654]: cluster 2024-04-07T16:29:54.105198+0000 mon.a (mon.0) 1400 : cluster [DBG] osdmap e808: 8 total, 8 up, 8 in 2024-04-07T16:29:55.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:29:55 smithi144 bash[66654]: audit 2024-04-07T16:29:54.109218+0000 mon.a (mon.0) 1401 : audit [INF] from='client.? 172.21.15.100:0/816276451' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:29:55.439 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:29:55 smithi100 bash[74942]: cluster 2024-04-07T16:29:54.105198+0000 mon.a (mon.0) 1400 : cluster [DBG] osdmap e808: 8 total, 8 up, 8 in 2024-04-07T16:29:55.439 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:29:55 smithi100 bash[74942]: audit 2024-04-07T16:29:54.109218+0000 mon.a (mon.0) 1401 : audit [INF] from='client.? 172.21.15.100:0/816276451' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:29:55.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:29:55 smithi100 bash[76202]: cluster 2024-04-07T16:29:54.105198+0000 mon.a (mon.0) 1400 : cluster [DBG] osdmap e808: 8 total, 8 up, 8 in 2024-04-07T16:29:55.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:29:55 smithi100 bash[76202]: audit 2024-04-07T16:29:54.109218+0000 mon.a (mon.0) 1401 : audit [INF] from='client.? 172.21.15.100:0/816276451' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:29:56.122 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestCls2PCQueue.GetCapacity (2677 ms) 2024-04-07T16:29:56.123 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestCls2PCQueue.AsyncGetCapacity 2024-04-07T16:29:56.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:29:56 smithi144 bash[66654]: cluster 2024-04-07T16:29:54.855382+0000 mgr.y (mgr.25175) 437 : cluster [DBG] pgmap v820: 137 pgs: 4 creating+peering, 28 unknown, 105 active+clean; 583 KiB data, 1.7 GiB used, 714 GiB / 715 GiB avail 2024-04-07T16:29:56.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:29:56 smithi144 bash[66654]: cluster 2024-04-07T16:29:55.098233+0000 mon.a (mon.0) 1402 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T16:29:56.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:29:56 smithi144 bash[66654]: audit 2024-04-07T16:29:55.107658+0000 mon.a (mon.0) 1403 : audit [INF] from='client.? 172.21.15.100:0/816276451' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:29:56.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:29:56 smithi144 bash[66654]: cluster 2024-04-07T16:29:55.115700+0000 mon.a (mon.0) 1404 : cluster [DBG] osdmap e809: 8 total, 8 up, 8 in 2024-04-07T16:29:56.439 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:29:56 smithi100 bash[74942]: cluster 2024-04-07T16:29:54.855382+0000 mgr.y (mgr.25175) 437 : cluster [DBG] pgmap v820: 137 pgs: 4 creating+peering, 28 unknown, 105 active+clean; 583 KiB data, 1.7 GiB used, 714 GiB / 715 GiB avail 2024-04-07T16:29:56.439 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:29:56 smithi100 bash[74942]: cluster 2024-04-07T16:29:55.098233+0000 mon.a (mon.0) 1402 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T16:29:56.439 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:29:56 smithi100 bash[74942]: audit 2024-04-07T16:29:55.107658+0000 mon.a (mon.0) 1403 : audit [INF] from='client.? 172.21.15.100:0/816276451' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:29:56.439 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:29:56 smithi100 bash[74942]: cluster 2024-04-07T16:29:55.115700+0000 mon.a (mon.0) 1404 : cluster [DBG] osdmap e809: 8 total, 8 up, 8 in 2024-04-07T16:29:56.440 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:29:56 smithi100 bash[76202]: cluster 2024-04-07T16:29:54.855382+0000 mgr.y (mgr.25175) 437 : cluster [DBG] pgmap v820: 137 pgs: 4 creating+peering, 28 unknown, 105 active+clean; 583 KiB data, 1.7 GiB used, 714 GiB / 715 GiB avail 2024-04-07T16:29:56.440 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:29:56 smithi100 bash[76202]: cluster 2024-04-07T16:29:55.098233+0000 mon.a (mon.0) 1402 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T16:29:56.440 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:29:56 smithi100 bash[76202]: audit 2024-04-07T16:29:55.107658+0000 mon.a (mon.0) 1403 : audit [INF] from='client.? 172.21.15.100:0/816276451' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:29:56.440 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:29:56 smithi100 bash[76202]: cluster 2024-04-07T16:29:55.115700+0000 mon.a (mon.0) 1404 : cluster [DBG] osdmap e809: 8 total, 8 up, 8 in 2024-04-07T16:29:57.439 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:29:57 smithi100 bash[74942]: cluster 2024-04-07T16:29:56.126202+0000 mon.a (mon.0) 1405 : cluster [DBG] osdmap e810: 8 total, 8 up, 8 in 2024-04-07T16:29:57.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:29:57 smithi100 bash[76202]: cluster 2024-04-07T16:29:56.126202+0000 mon.a (mon.0) 1405 : cluster [DBG] osdmap e810: 8 total, 8 up, 8 in 2024-04-07T16:29:57.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:29:57 smithi144 bash[66654]: cluster 2024-04-07T16:29:56.126202+0000 mon.a (mon.0) 1405 : cluster [DBG] osdmap e810: 8 total, 8 up, 8 in 2024-04-07T16:29:58.418 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:29:58 smithi100 bash[74942]: cluster 2024-04-07T16:29:56.856029+0000 mgr.y (mgr.25175) 438 : cluster [DBG] pgmap v823: 105 pgs: 105 active+clean; 583 KiB data, 1.7 GiB used, 714 GiB / 715 GiB avail 2024-04-07T16:29:58.418 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:29:58 smithi100 bash[74942]: cluster 2024-04-07T16:29:57.118699+0000 mon.a (mon.0) 1406 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T16:29:58.418 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:29:58 smithi100 bash[74942]: cluster 2024-04-07T16:29:57.118776+0000 mon.a (mon.0) 1407 : cluster [INF] Cluster is now healthy 2024-04-07T16:29:58.418 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:29:58 smithi100 bash[74942]: cluster 2024-04-07T16:29:57.143866+0000 mon.a (mon.0) 1408 : cluster [DBG] osdmap e811: 8 total, 8 up, 8 in 2024-04-07T16:29:58.419 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:29:58 smithi100 bash[74942]: audit 2024-04-07T16:29:57.147095+0000 mon.a (mon.0) 1409 : audit [INF] from='client.? 172.21.15.100:0/2877949032' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:29:58.419 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:29:58 smithi100 bash[76202]: cluster 2024-04-07T16:29:56.856029+0000 mgr.y (mgr.25175) 438 : cluster [DBG] pgmap v823: 105 pgs: 105 active+clean; 583 KiB data, 1.7 GiB used, 714 GiB / 715 GiB avail 2024-04-07T16:29:58.419 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:29:58 smithi100 bash[76202]: cluster 2024-04-07T16:29:57.118699+0000 mon.a (mon.0) 1406 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T16:29:58.419 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:29:58 smithi100 bash[76202]: cluster 2024-04-07T16:29:57.118776+0000 mon.a (mon.0) 1407 : cluster [INF] Cluster is now healthy 2024-04-07T16:29:58.419 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:29:58 smithi100 bash[76202]: cluster 2024-04-07T16:29:57.143866+0000 mon.a (mon.0) 1408 : cluster [DBG] osdmap e811: 8 total, 8 up, 8 in 2024-04-07T16:29:58.419 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:29:58 smithi100 bash[76202]: audit 2024-04-07T16:29:57.147095+0000 mon.a (mon.0) 1409 : audit [INF] from='client.? 172.21.15.100:0/2877949032' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:29:58.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:29:58 smithi144 bash[66654]: cluster 2024-04-07T16:29:56.856029+0000 mgr.y (mgr.25175) 438 : cluster [DBG] pgmap v823: 105 pgs: 105 active+clean; 583 KiB data, 1.7 GiB used, 714 GiB / 715 GiB avail 2024-04-07T16:29:58.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:29:58 smithi144 bash[66654]: cluster 2024-04-07T16:29:57.118699+0000 mon.a (mon.0) 1406 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T16:29:58.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:29:58 smithi144 bash[66654]: cluster 2024-04-07T16:29:57.118776+0000 mon.a (mon.0) 1407 : cluster [INF] Cluster is now healthy 2024-04-07T16:29:58.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:29:58 smithi144 bash[66654]: cluster 2024-04-07T16:29:57.143866+0000 mon.a (mon.0) 1408 : cluster [DBG] osdmap e811: 8 total, 8 up, 8 in 2024-04-07T16:29:58.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:29:58 smithi144 bash[66654]: audit 2024-04-07T16:29:57.147095+0000 mon.a (mon.0) 1409 : audit [INF] from='client.? 172.21.15.100:0/2877949032' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:29:58.689 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:29:58 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:29:58] "GET /metrics HTTP/1.1" 200 33991 "" "Prometheus/2.43.0" 2024-04-07T16:29:59.171 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestCls2PCQueue.AsyncGetCapacity (3049 ms) 2024-04-07T16:29:59.171 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestCls2PCQueue.Reserve 2024-04-07T16:29:59.439 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:29:59 smithi100 bash[74942]: audit 2024-04-07T16:29:58.133455+0000 mon.a (mon.0) 1410 : audit [INF] from='client.? 172.21.15.100:0/2877949032' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:29:59.439 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:29:59 smithi100 bash[74942]: cluster 2024-04-07T16:29:58.140375+0000 mon.a (mon.0) 1411 : cluster [DBG] osdmap e812: 8 total, 8 up, 8 in 2024-04-07T16:29:59.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:29:59 smithi100 bash[76202]: audit 2024-04-07T16:29:58.133455+0000 mon.a (mon.0) 1410 : audit [INF] from='client.? 172.21.15.100:0/2877949032' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:29:59.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:29:59 smithi100 bash[76202]: cluster 2024-04-07T16:29:58.140375+0000 mon.a (mon.0) 1411 : cluster [DBG] osdmap e812: 8 total, 8 up, 8 in 2024-04-07T16:29:59.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:29:59 smithi144 bash[66654]: audit 2024-04-07T16:29:58.133455+0000 mon.a (mon.0) 1410 : audit [INF] from='client.? 172.21.15.100:0/2877949032' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:29:59.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:29:59 smithi144 bash[66654]: cluster 2024-04-07T16:29:58.140375+0000 mon.a (mon.0) 1411 : cluster [DBG] osdmap e812: 8 total, 8 up, 8 in 2024-04-07T16:30:00.439 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:30:00 smithi100 bash[74942]: cluster 2024-04-07T16:29:58.856726+0000 mgr.y (mgr.25175) 439 : cluster [DBG] pgmap v826: 137 pgs: 7 creating+peering, 25 unknown, 105 active+clean; 583 KiB data, 1.7 GiB used, 714 GiB / 715 GiB avail 2024-04-07T16:30:00.439 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:30:00 smithi100 bash[74942]: cluster 2024-04-07T16:29:59.179909+0000 mon.a (mon.0) 1412 : cluster [DBG] osdmap e813: 8 total, 8 up, 8 in 2024-04-07T16:30:00.439 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:30:00 smithi100 bash[74942]: cluster 2024-04-07T16:30:00.000162+0000 mon.a (mon.0) 1413 : cluster [INF] overall HEALTH_OK 2024-04-07T16:30:00.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:30:00 smithi100 bash[76202]: cluster 2024-04-07T16:29:58.856726+0000 mgr.y (mgr.25175) 439 : cluster [DBG] pgmap v826: 137 pgs: 7 creating+peering, 25 unknown, 105 active+clean; 583 KiB data, 1.7 GiB used, 714 GiB / 715 GiB avail 2024-04-07T16:30:00.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:30:00 smithi100 bash[76202]: cluster 2024-04-07T16:29:59.179909+0000 mon.a (mon.0) 1412 : cluster [DBG] osdmap e813: 8 total, 8 up, 8 in 2024-04-07T16:30:00.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:30:00 smithi100 bash[76202]: cluster 2024-04-07T16:30:00.000162+0000 mon.a (mon.0) 1413 : cluster [INF] overall HEALTH_OK 2024-04-07T16:30:00.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:30:00 smithi144 bash[66654]: cluster 2024-04-07T16:29:58.856726+0000 mgr.y (mgr.25175) 439 : cluster [DBG] pgmap v826: 137 pgs: 7 creating+peering, 25 unknown, 105 active+clean; 583 KiB data, 1.7 GiB used, 714 GiB / 715 GiB avail 2024-04-07T16:30:00.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:30:00 smithi144 bash[66654]: cluster 2024-04-07T16:29:59.179909+0000 mon.a (mon.0) 1412 : cluster [DBG] osdmap e813: 8 total, 8 up, 8 in 2024-04-07T16:30:00.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:30:00 smithi144 bash[66654]: cluster 2024-04-07T16:30:00.000162+0000 mon.a (mon.0) 1413 : cluster [INF] overall HEALTH_OK 2024-04-07T16:30:01.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:30:01 smithi144 bash[66654]: cluster 2024-04-07T16:30:00.186827+0000 mon.a (mon.0) 1414 : cluster [DBG] osdmap e814: 8 total, 8 up, 8 in 2024-04-07T16:30:01.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:30:01 smithi144 bash[66654]: audit 2024-04-07T16:30:00.189287+0000 mon.c (mon.2) 19 : audit [INF] from='client.? 172.21.15.100:0/3411839906' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:30:01.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:30:01 smithi144 bash[66654]: audit 2024-04-07T16:30:00.200130+0000 mon.a (mon.0) 1415 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:30:01.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:30:01 smithi144 bash[66654]: audit 2024-04-07T16:30:01.181074+0000 mon.a (mon.0) 1416 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:30:01.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:30:01 smithi144 bash[66654]: cluster 2024-04-07T16:30:01.188374+0000 mon.a (mon.0) 1417 : cluster [DBG] osdmap e815: 8 total, 8 up, 8 in 2024-04-07T16:30:01.689 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:30:01 smithi100 bash[74942]: cluster 2024-04-07T16:30:00.186827+0000 mon.a (mon.0) 1414 : cluster [DBG] osdmap e814: 8 total, 8 up, 8 in 2024-04-07T16:30:01.689 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:30:01 smithi100 bash[74942]: audit 2024-04-07T16:30:00.189287+0000 mon.c (mon.2) 19 : audit [INF] from='client.? 172.21.15.100:0/3411839906' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:30:01.689 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:30:01 smithi100 bash[74942]: audit 2024-04-07T16:30:00.200130+0000 mon.a (mon.0) 1415 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:30:01.689 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:30:01 smithi100 bash[74942]: audit 2024-04-07T16:30:01.181074+0000 mon.a (mon.0) 1416 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:30:01.689 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:30:01 smithi100 bash[74942]: cluster 2024-04-07T16:30:01.188374+0000 mon.a (mon.0) 1417 : cluster [DBG] osdmap e815: 8 total, 8 up, 8 in 2024-04-07T16:30:01.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:30:01 smithi100 bash[76202]: cluster 2024-04-07T16:30:00.186827+0000 mon.a (mon.0) 1414 : cluster [DBG] osdmap e814: 8 total, 8 up, 8 in 2024-04-07T16:30:01.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:30:01 smithi100 bash[76202]: audit 2024-04-07T16:30:00.189287+0000 mon.c (mon.2) 19 : audit [INF] from='client.? 172.21.15.100:0/3411839906' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:30:01.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:30:01 smithi100 bash[76202]: audit 2024-04-07T16:30:00.200130+0000 mon.a (mon.0) 1415 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:30:01.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:30:01 smithi100 bash[76202]: audit 2024-04-07T16:30:01.181074+0000 mon.a (mon.0) 1416 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:30:01.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:30:01 smithi100 bash[76202]: cluster 2024-04-07T16:30:01.188374+0000 mon.a (mon.0) 1417 : cluster [DBG] osdmap e815: 8 total, 8 up, 8 in 2024-04-07T16:30:02.205 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestCls2PCQueue.Reserve (3033 ms) 2024-04-07T16:30:02.205 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestCls2PCQueue.AsyncReserve 2024-04-07T16:30:02.606 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:30:02 smithi144 bash[95233]: ts=2024-04-07T16:30:02.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:30:02.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:30:02 smithi144 bash[66654]: cluster 2024-04-07T16:30:00.857415+0000 mgr.y (mgr.25175) 440 : cluster [DBG] pgmap v829: 137 pgs: 32 unknown, 105 active+clean; 583 KiB data, 1.7 GiB used, 714 GiB / 715 GiB avail 2024-04-07T16:30:02.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:30:02 smithi144 bash[66654]: cluster 2024-04-07T16:30:01.193276+0000 mon.a (mon.0) 1418 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T16:30:02.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:30:02 smithi144 bash[66654]: audit 2024-04-07T16:30:01.746609+0000 mon.a (mon.0) 1419 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:30:02.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:30:02 smithi144 bash[66654]: audit 2024-04-07T16:30:01.749007+0000 mon.a (mon.0) 1420 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:30:02.689 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:30:02 smithi100 bash[74942]: cluster 2024-04-07T16:30:00.857415+0000 mgr.y (mgr.25175) 440 : cluster [DBG] pgmap v829: 137 pgs: 32 unknown, 105 active+clean; 583 KiB data, 1.7 GiB used, 714 GiB / 715 GiB avail 2024-04-07T16:30:02.689 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:30:02 smithi100 bash[74942]: cluster 2024-04-07T16:30:01.193276+0000 mon.a (mon.0) 1418 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T16:30:02.689 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:30:02 smithi100 bash[74942]: audit 2024-04-07T16:30:01.746609+0000 mon.a (mon.0) 1419 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:30:02.689 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:30:02 smithi100 bash[74942]: audit 2024-04-07T16:30:01.749007+0000 mon.a (mon.0) 1420 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:30:02.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:30:02 smithi100 bash[76202]: cluster 2024-04-07T16:30:00.857415+0000 mgr.y (mgr.25175) 440 : cluster [DBG] pgmap v829: 137 pgs: 32 unknown, 105 active+clean; 583 KiB data, 1.7 GiB used, 714 GiB / 715 GiB avail 2024-04-07T16:30:02.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:30:02 smithi100 bash[76202]: cluster 2024-04-07T16:30:01.193276+0000 mon.a (mon.0) 1418 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T16:30:02.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:30:02 smithi100 bash[76202]: audit 2024-04-07T16:30:01.746609+0000 mon.a (mon.0) 1419 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:30:02.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:30:02 smithi100 bash[76202]: audit 2024-04-07T16:30:01.749007+0000 mon.a (mon.0) 1420 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:30:03.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:30:03 smithi144 bash[66654]: cluster 2024-04-07T16:30:02.209699+0000 mon.a (mon.0) 1421 : cluster [DBG] osdmap e816: 8 total, 8 up, 8 in 2024-04-07T16:30:03.689 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:30:03 smithi100 bash[74942]: cluster 2024-04-07T16:30:02.209699+0000 mon.a (mon.0) 1421 : cluster [DBG] osdmap e816: 8 total, 8 up, 8 in 2024-04-07T16:30:03.689 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:30:03 smithi100 bash[76202]: cluster 2024-04-07T16:30:02.209699+0000 mon.a (mon.0) 1421 : cluster [DBG] osdmap e816: 8 total, 8 up, 8 in 2024-04-07T16:30:04.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:30:04 smithi144 bash[66654]: cluster 2024-04-07T16:30:02.858053+0000 mgr.y (mgr.25175) 441 : cluster [DBG] pgmap v832: 105 pgs: 105 active+clean; 583 KiB data, 1.7 GiB used, 714 GiB / 715 GiB avail 2024-04-07T16:30:04.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:30:04 smithi144 bash[66654]: cluster 2024-04-07T16:30:03.217030+0000 mon.a (mon.0) 1422 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T16:30:04.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:30:04 smithi144 bash[66654]: cluster 2024-04-07T16:30:03.217118+0000 mon.a (mon.0) 1423 : cluster [INF] Cluster is now healthy 2024-04-07T16:30:04.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:30:04 smithi144 bash[66654]: cluster 2024-04-07T16:30:03.239183+0000 mon.a (mon.0) 1424 : cluster [DBG] osdmap e817: 8 total, 8 up, 8 in 2024-04-07T16:30:04.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:30:04 smithi144 bash[66654]: audit 2024-04-07T16:30:03.242608+0000 mon.c (mon.2) 20 : audit [INF] from='client.? 172.21.15.100:0/91462843' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:30:04.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:30:04 smithi144 bash[66654]: audit 2024-04-07T16:30:03.243844+0000 mon.a (mon.0) 1425 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:30:04.689 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:30:04 smithi100 bash[74942]: cluster 2024-04-07T16:30:02.858053+0000 mgr.y (mgr.25175) 441 : cluster [DBG] pgmap v832: 105 pgs: 105 active+clean; 583 KiB data, 1.7 GiB used, 714 GiB / 715 GiB avail 2024-04-07T16:30:04.689 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:30:04 smithi100 bash[74942]: cluster 2024-04-07T16:30:03.217030+0000 mon.a (mon.0) 1422 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T16:30:04.689 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:30:04 smithi100 bash[74942]: cluster 2024-04-07T16:30:03.217118+0000 mon.a (mon.0) 1423 : cluster [INF] Cluster is now healthy 2024-04-07T16:30:04.689 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:30:04 smithi100 bash[74942]: cluster 2024-04-07T16:30:03.239183+0000 mon.a (mon.0) 1424 : cluster [DBG] osdmap e817: 8 total, 8 up, 8 in 2024-04-07T16:30:04.690 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:30:04 smithi100 bash[74942]: audit 2024-04-07T16:30:03.242608+0000 mon.c (mon.2) 20 : audit [INF] from='client.? 172.21.15.100:0/91462843' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:30:04.690 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:30:04 smithi100 bash[74942]: audit 2024-04-07T16:30:03.243844+0000 mon.a (mon.0) 1425 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:30:04.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:30:04 smithi100 bash[76202]: cluster 2024-04-07T16:30:02.858053+0000 mgr.y (mgr.25175) 441 : cluster [DBG] pgmap v832: 105 pgs: 105 active+clean; 583 KiB data, 1.7 GiB used, 714 GiB / 715 GiB avail 2024-04-07T16:30:04.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:30:04 smithi100 bash[76202]: cluster 2024-04-07T16:30:03.217030+0000 mon.a (mon.0) 1422 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T16:30:04.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:30:04 smithi100 bash[76202]: cluster 2024-04-07T16:30:03.217118+0000 mon.a (mon.0) 1423 : cluster [INF] Cluster is now healthy 2024-04-07T16:30:04.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:30:04 smithi100 bash[76202]: cluster 2024-04-07T16:30:03.239183+0000 mon.a (mon.0) 1424 : cluster [DBG] osdmap e817: 8 total, 8 up, 8 in 2024-04-07T16:30:04.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:30:04 smithi100 bash[76202]: audit 2024-04-07T16:30:03.242608+0000 mon.c (mon.2) 20 : audit [INF] from='client.? 172.21.15.100:0/91462843' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:30:04.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:30:04 smithi100 bash[76202]: audit 2024-04-07T16:30:03.243844+0000 mon.a (mon.0) 1425 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:30:05.254 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestCls2PCQueue.AsyncReserve (3050 ms) 2024-04-07T16:30:05.254 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestCls2PCQueue.Commit 2024-04-07T16:30:05.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:30:05 smithi144 bash[66654]: audit 2024-04-07T16:30:04.232273+0000 mon.a (mon.0) 1426 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:30:05.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:30:05 smithi144 bash[66654]: cluster 2024-04-07T16:30:04.240395+0000 mon.a (mon.0) 1427 : cluster [DBG] osdmap e818: 8 total, 8 up, 8 in 2024-04-07T16:30:05.689 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:30:05 smithi100 bash[74942]: audit 2024-04-07T16:30:04.232273+0000 mon.a (mon.0) 1426 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:30:05.689 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:30:05 smithi100 bash[74942]: cluster 2024-04-07T16:30:04.240395+0000 mon.a (mon.0) 1427 : cluster [DBG] osdmap e818: 8 total, 8 up, 8 in 2024-04-07T16:30:05.689 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:30:05 smithi100 bash[76202]: audit 2024-04-07T16:30:04.232273+0000 mon.a (mon.0) 1426 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:30:05.689 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:30:05 smithi100 bash[76202]: cluster 2024-04-07T16:30:04.240395+0000 mon.a (mon.0) 1427 : cluster [DBG] osdmap e818: 8 total, 8 up, 8 in 2024-04-07T16:30:06.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:30:06 smithi144 bash[66654]: cluster 2024-04-07T16:30:04.858740+0000 mgr.y (mgr.25175) 442 : cluster [DBG] pgmap v835: 137 pgs: 5 creating+peering, 27 unknown, 105 active+clean; 583 KiB data, 1.7 GiB used, 714 GiB / 715 GiB avail 2024-04-07T16:30:06.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:30:06 smithi144 bash[66654]: cluster 2024-04-07T16:30:05.263037+0000 mon.a (mon.0) 1428 : cluster [DBG] osdmap e819: 8 total, 8 up, 8 in 2024-04-07T16:30:06.689 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:30:06 smithi100 bash[74942]: cluster 2024-04-07T16:30:04.858740+0000 mgr.y (mgr.25175) 442 : cluster [DBG] pgmap v835: 137 pgs: 5 creating+peering, 27 unknown, 105 active+clean; 583 KiB data, 1.7 GiB used, 714 GiB / 715 GiB avail 2024-04-07T16:30:06.689 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:30:06 smithi100 bash[74942]: cluster 2024-04-07T16:30:05.263037+0000 mon.a (mon.0) 1428 : cluster [DBG] osdmap e819: 8 total, 8 up, 8 in 2024-04-07T16:30:06.689 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:30:06 smithi100 bash[76202]: cluster 2024-04-07T16:30:04.858740+0000 mgr.y (mgr.25175) 442 : cluster [DBG] pgmap v835: 137 pgs: 5 creating+peering, 27 unknown, 105 active+clean; 583 KiB data, 1.7 GiB used, 714 GiB / 715 GiB avail 2024-04-07T16:30:06.689 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:30:06 smithi100 bash[76202]: cluster 2024-04-07T16:30:05.263037+0000 mon.a (mon.0) 1428 : cluster [DBG] osdmap e819: 8 total, 8 up, 8 in 2024-04-07T16:30:07.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:30:07 smithi144 bash[66654]: cluster 2024-04-07T16:30:06.267529+0000 mon.a (mon.0) 1429 : cluster [DBG] osdmap e820: 8 total, 8 up, 8 in 2024-04-07T16:30:07.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:30:07 smithi144 bash[66654]: audit 2024-04-07T16:30:06.273615+0000 mon.c (mon.2) 21 : audit [INF] from='client.? 172.21.15.100:0/2638225203' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:30:07.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:30:07 smithi144 bash[66654]: audit 2024-04-07T16:30:06.283597+0000 mon.a (mon.0) 1430 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:30:07.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:30:07 smithi144 bash[66654]: audit 2024-04-07T16:30:07.261850+0000 mon.a (mon.0) 1431 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:30:07.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:30:07 smithi144 bash[66654]: cluster 2024-04-07T16:30:07.270970+0000 mon.a (mon.0) 1432 : cluster [DBG] osdmap e821: 8 total, 8 up, 8 in 2024-04-07T16:30:07.689 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:30:07 smithi100 bash[74942]: cluster 2024-04-07T16:30:06.267529+0000 mon.a (mon.0) 1429 : cluster [DBG] osdmap e820: 8 total, 8 up, 8 in 2024-04-07T16:30:07.689 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:30:07 smithi100 bash[74942]: audit 2024-04-07T16:30:06.273615+0000 mon.c (mon.2) 21 : audit [INF] from='client.? 172.21.15.100:0/2638225203' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:30:07.689 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:30:07 smithi100 bash[74942]: audit 2024-04-07T16:30:06.283597+0000 mon.a (mon.0) 1430 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:30:07.689 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:30:07 smithi100 bash[74942]: audit 2024-04-07T16:30:07.261850+0000 mon.a (mon.0) 1431 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:30:07.689 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:30:07 smithi100 bash[74942]: cluster 2024-04-07T16:30:07.270970+0000 mon.a (mon.0) 1432 : cluster [DBG] osdmap e821: 8 total, 8 up, 8 in 2024-04-07T16:30:07.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:30:07 smithi100 bash[76202]: cluster 2024-04-07T16:30:06.267529+0000 mon.a (mon.0) 1429 : cluster [DBG] osdmap e820: 8 total, 8 up, 8 in 2024-04-07T16:30:07.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:30:07 smithi100 bash[76202]: audit 2024-04-07T16:30:06.273615+0000 mon.c (mon.2) 21 : audit [INF] from='client.? 172.21.15.100:0/2638225203' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:30:07.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:30:07 smithi100 bash[76202]: audit 2024-04-07T16:30:06.283597+0000 mon.a (mon.0) 1430 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:30:07.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:30:07 smithi100 bash[76202]: audit 2024-04-07T16:30:07.261850+0000 mon.a (mon.0) 1431 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:30:07.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:30:07 smithi100 bash[76202]: cluster 2024-04-07T16:30:07.270970+0000 mon.a (mon.0) 1432 : cluster [DBG] osdmap e821: 8 total, 8 up, 8 in 2024-04-07T16:30:08.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:30:08 smithi144 bash[66654]: cluster 2024-04-07T16:30:06.859525+0000 mgr.y (mgr.25175) 443 : cluster [DBG] pgmap v838: 137 pgs: 32 unknown, 105 active+clean; 583 KiB data, 1.7 GiB used, 714 GiB / 715 GiB avail 2024-04-07T16:30:08.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:30:08 smithi144 bash[66654]: cluster 2024-04-07T16:30:07.279076+0000 mon.a (mon.0) 1433 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T16:30:08.689 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:30:08 smithi100 bash[74942]: cluster 2024-04-07T16:30:06.859525+0000 mgr.y (mgr.25175) 443 : cluster [DBG] pgmap v838: 137 pgs: 32 unknown, 105 active+clean; 583 KiB data, 1.7 GiB used, 714 GiB / 715 GiB avail 2024-04-07T16:30:08.689 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:30:08 smithi100 bash[74942]: cluster 2024-04-07T16:30:07.279076+0000 mon.a (mon.0) 1433 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T16:30:08.689 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:30:08 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:30:08] "GET /metrics HTTP/1.1" 200 36298 "" "Prometheus/2.43.0" 2024-04-07T16:30:08.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:30:08 smithi100 bash[76202]: cluster 2024-04-07T16:30:06.859525+0000 mgr.y (mgr.25175) 443 : cluster [DBG] pgmap v838: 137 pgs: 32 unknown, 105 active+clean; 583 KiB data, 1.7 GiB used, 714 GiB / 715 GiB avail 2024-04-07T16:30:08.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:30:08 smithi100 bash[76202]: cluster 2024-04-07T16:30:07.279076+0000 mon.a (mon.0) 1433 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T16:30:09.316 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestCls2PCQueue.Commit (4062 ms) 2024-04-07T16:30:09.316 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestCls2PCQueue.Abort 2024-04-07T16:30:09.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:30:09 smithi144 bash[66654]: cluster 2024-04-07T16:30:08.297607+0000 mon.a (mon.0) 1434 : cluster [DBG] osdmap e822: 8 total, 8 up, 8 in 2024-04-07T16:30:09.689 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:30:09 smithi100 bash[74942]: cluster 2024-04-07T16:30:08.297607+0000 mon.a (mon.0) 1434 : cluster [DBG] osdmap e822: 8 total, 8 up, 8 in 2024-04-07T16:30:09.689 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:30:09 smithi100 bash[76202]: cluster 2024-04-07T16:30:08.297607+0000 mon.a (mon.0) 1434 : cluster [DBG] osdmap e822: 8 total, 8 up, 8 in 2024-04-07T16:30:10.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:30:10 smithi144 bash[66654]: cluster 2024-04-07T16:30:08.860230+0000 mgr.y (mgr.25175) 444 : cluster [DBG] pgmap v841: 137 pgs: 19 creating+peering, 10 unknown, 108 active+clean; 583 KiB data, 1.7 GiB used, 714 GiB / 715 GiB avail 2024-04-07T16:30:10.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:30:10 smithi144 bash[66654]: cluster 2024-04-07T16:30:09.300709+0000 mon.a (mon.0) 1435 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T16:30:10.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:30:10 smithi144 bash[66654]: cluster 2024-04-07T16:30:09.300766+0000 mon.a (mon.0) 1436 : cluster [INF] Cluster is now healthy 2024-04-07T16:30:10.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:30:10 smithi144 bash[66654]: cluster 2024-04-07T16:30:09.319047+0000 mon.a (mon.0) 1437 : cluster [DBG] osdmap e823: 8 total, 8 up, 8 in 2024-04-07T16:30:10.689 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:30:10 smithi100 bash[74942]: cluster 2024-04-07T16:30:08.860230+0000 mgr.y (mgr.25175) 444 : cluster [DBG] pgmap v841: 137 pgs: 19 creating+peering, 10 unknown, 108 active+clean; 583 KiB data, 1.7 GiB used, 714 GiB / 715 GiB avail 2024-04-07T16:30:10.689 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:30:10 smithi100 bash[74942]: cluster 2024-04-07T16:30:09.300709+0000 mon.a (mon.0) 1435 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T16:30:10.689 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:30:10 smithi100 bash[74942]: cluster 2024-04-07T16:30:09.300766+0000 mon.a (mon.0) 1436 : cluster [INF] Cluster is now healthy 2024-04-07T16:30:10.689 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:30:10 smithi100 bash[74942]: cluster 2024-04-07T16:30:09.319047+0000 mon.a (mon.0) 1437 : cluster [DBG] osdmap e823: 8 total, 8 up, 8 in 2024-04-07T16:30:10.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:30:10 smithi100 bash[76202]: cluster 2024-04-07T16:30:08.860230+0000 mgr.y (mgr.25175) 444 : cluster [DBG] pgmap v841: 137 pgs: 19 creating+peering, 10 unknown, 108 active+clean; 583 KiB data, 1.7 GiB used, 714 GiB / 715 GiB avail 2024-04-07T16:30:10.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:30:10 smithi100 bash[76202]: cluster 2024-04-07T16:30:09.300709+0000 mon.a (mon.0) 1435 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T16:30:10.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:30:10 smithi100 bash[76202]: cluster 2024-04-07T16:30:09.300766+0000 mon.a (mon.0) 1436 : cluster [INF] Cluster is now healthy 2024-04-07T16:30:10.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:30:10 smithi100 bash[76202]: cluster 2024-04-07T16:30:09.319047+0000 mon.a (mon.0) 1437 : cluster [DBG] osdmap e823: 8 total, 8 up, 8 in 2024-04-07T16:30:11.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:30:11 smithi144 bash[66654]: cluster 2024-04-07T16:30:10.335137+0000 mon.a (mon.0) 1438 : cluster [DBG] osdmap e824: 8 total, 8 up, 8 in 2024-04-07T16:30:11.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:30:11 smithi144 bash[66654]: audit 2024-04-07T16:30:10.344004+0000 mon.c (mon.2) 22 : audit [INF] from='client.? 172.21.15.100:0/368574230' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:30:11.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:30:11 smithi144 bash[66654]: audit 2024-04-07T16:30:10.344899+0000 mon.a (mon.0) 1439 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:30:11.689 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:30:11 smithi100 bash[74942]: cluster 2024-04-07T16:30:10.335137+0000 mon.a (mon.0) 1438 : cluster [DBG] osdmap e824: 8 total, 8 up, 8 in 2024-04-07T16:30:11.689 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:30:11 smithi100 bash[74942]: audit 2024-04-07T16:30:10.344004+0000 mon.c (mon.2) 22 : audit [INF] from='client.? 172.21.15.100:0/368574230' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:30:11.689 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:30:11 smithi100 bash[74942]: audit 2024-04-07T16:30:10.344899+0000 mon.a (mon.0) 1439 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:30:11.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:30:11 smithi100 bash[76202]: cluster 2024-04-07T16:30:10.335137+0000 mon.a (mon.0) 1438 : cluster [DBG] osdmap e824: 8 total, 8 up, 8 in 2024-04-07T16:30:11.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:30:11 smithi100 bash[76202]: audit 2024-04-07T16:30:10.344004+0000 mon.c (mon.2) 22 : audit [INF] from='client.? 172.21.15.100:0/368574230' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:30:11.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:30:11 smithi100 bash[76202]: audit 2024-04-07T16:30:10.344899+0000 mon.a (mon.0) 1439 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:30:12.359 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestCls2PCQueue.Abort (3042 ms) 2024-04-07T16:30:12.359 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestCls2PCQueue.ReserveError 2024-04-07T16:30:12.606 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:30:12 smithi144 bash[95233]: ts=2024-04-07T16:30:12.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:30:12.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:30:12 smithi144 bash[66654]: cluster 2024-04-07T16:30:10.860979+0000 mgr.y (mgr.25175) 445 : cluster [DBG] pgmap v844: 137 pgs: 32 unknown, 105 active+clean; 583 KiB data, 1.7 GiB used, 714 GiB / 715 GiB avail 2024-04-07T16:30:12.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:30:12 smithi144 bash[66654]: cluster 2024-04-07T16:30:11.326910+0000 mon.a (mon.0) 1440 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T16:30:12.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:30:12 smithi144 bash[66654]: audit 2024-04-07T16:30:11.336349+0000 mon.a (mon.0) 1441 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:30:12.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:30:12 smithi144 bash[66654]: cluster 2024-04-07T16:30:11.346233+0000 mon.a (mon.0) 1442 : cluster [DBG] osdmap e825: 8 total, 8 up, 8 in 2024-04-07T16:30:12.689 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:30:12 smithi100 bash[74942]: cluster 2024-04-07T16:30:10.860979+0000 mgr.y (mgr.25175) 445 : cluster [DBG] pgmap v844: 137 pgs: 32 unknown, 105 active+clean; 583 KiB data, 1.7 GiB used, 714 GiB / 715 GiB avail 2024-04-07T16:30:12.689 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:30:12 smithi100 bash[74942]: cluster 2024-04-07T16:30:11.326910+0000 mon.a (mon.0) 1440 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T16:30:12.689 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:30:12 smithi100 bash[74942]: audit 2024-04-07T16:30:11.336349+0000 mon.a (mon.0) 1441 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:30:12.689 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:30:12 smithi100 bash[74942]: cluster 2024-04-07T16:30:11.346233+0000 mon.a (mon.0) 1442 : cluster [DBG] osdmap e825: 8 total, 8 up, 8 in 2024-04-07T16:30:12.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:30:12 smithi100 bash[76202]: cluster 2024-04-07T16:30:10.860979+0000 mgr.y (mgr.25175) 445 : cluster [DBG] pgmap v844: 137 pgs: 32 unknown, 105 active+clean; 583 KiB data, 1.7 GiB used, 714 GiB / 715 GiB avail 2024-04-07T16:30:12.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:30:12 smithi100 bash[76202]: cluster 2024-04-07T16:30:11.326910+0000 mon.a (mon.0) 1440 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T16:30:12.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:30:12 smithi100 bash[76202]: audit 2024-04-07T16:30:11.336349+0000 mon.a (mon.0) 1441 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:30:12.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:30:12 smithi100 bash[76202]: cluster 2024-04-07T16:30:11.346233+0000 mon.a (mon.0) 1442 : cluster [DBG] osdmap e825: 8 total, 8 up, 8 in 2024-04-07T16:30:13.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:30:13 smithi144 bash[66654]: cluster 2024-04-07T16:30:12.355584+0000 mon.a (mon.0) 1443 : cluster [DBG] osdmap e826: 8 total, 8 up, 8 in 2024-04-07T16:30:13.689 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:30:13 smithi100 bash[74942]: cluster 2024-04-07T16:30:12.355584+0000 mon.a (mon.0) 1443 : cluster [DBG] osdmap e826: 8 total, 8 up, 8 in 2024-04-07T16:30:13.689 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:30:13 smithi100 bash[76202]: cluster 2024-04-07T16:30:12.355584+0000 mon.a (mon.0) 1443 : cluster [DBG] osdmap e826: 8 total, 8 up, 8 in 2024-04-07T16:30:14.689 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:30:14 smithi100 bash[76202]: cluster 2024-04-07T16:30:12.861629+0000 mgr.y (mgr.25175) 446 : cluster [DBG] pgmap v847: 105 pgs: 105 active+clean; 583 KiB data, 1.7 GiB used, 714 GiB / 715 GiB avail 2024-04-07T16:30:14.689 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:30:14 smithi100 bash[76202]: cluster 2024-04-07T16:30:13.349444+0000 mon.a (mon.0) 1444 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T16:30:14.689 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:30:14 smithi100 bash[76202]: cluster 2024-04-07T16:30:13.349521+0000 mon.a (mon.0) 1445 : cluster [INF] Cluster is now healthy 2024-04-07T16:30:14.689 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:30:14 smithi100 bash[76202]: cluster 2024-04-07T16:30:13.385409+0000 mon.a (mon.0) 1446 : cluster [DBG] osdmap e827: 8 total, 8 up, 8 in 2024-04-07T16:30:14.689 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:30:14 smithi100 bash[76202]: audit 2024-04-07T16:30:13.388516+0000 mon.a (mon.0) 1447 : audit [INF] from='client.? 172.21.15.100:0/875113546' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:30:14.690 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:30:14 smithi100 bash[74942]: cluster 2024-04-07T16:30:12.861629+0000 mgr.y (mgr.25175) 446 : cluster [DBG] pgmap v847: 105 pgs: 105 active+clean; 583 KiB data, 1.7 GiB used, 714 GiB / 715 GiB avail 2024-04-07T16:30:14.690 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:30:14 smithi100 bash[74942]: cluster 2024-04-07T16:30:13.349444+0000 mon.a (mon.0) 1444 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T16:30:14.690 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:30:14 smithi100 bash[74942]: cluster 2024-04-07T16:30:13.349521+0000 mon.a (mon.0) 1445 : cluster [INF] Cluster is now healthy 2024-04-07T16:30:14.690 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:30:14 smithi100 bash[74942]: cluster 2024-04-07T16:30:13.385409+0000 mon.a (mon.0) 1446 : cluster [DBG] osdmap e827: 8 total, 8 up, 8 in 2024-04-07T16:30:14.690 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:30:14 smithi100 bash[74942]: audit 2024-04-07T16:30:13.388516+0000 mon.a (mon.0) 1447 : audit [INF] from='client.? 172.21.15.100:0/875113546' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:30:14.856 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:30:14 smithi144 bash[66654]: cluster 2024-04-07T16:30:12.861629+0000 mgr.y (mgr.25175) 446 : cluster [DBG] pgmap v847: 105 pgs: 105 active+clean; 583 KiB data, 1.7 GiB used, 714 GiB / 715 GiB avail 2024-04-07T16:30:14.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:30:14 smithi144 bash[66654]: cluster 2024-04-07T16:30:13.349444+0000 mon.a (mon.0) 1444 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T16:30:14.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:30:14 smithi144 bash[66654]: cluster 2024-04-07T16:30:13.349521+0000 mon.a (mon.0) 1445 : cluster [INF] Cluster is now healthy 2024-04-07T16:30:14.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:30:14 smithi144 bash[66654]: cluster 2024-04-07T16:30:13.385409+0000 mon.a (mon.0) 1446 : cluster [DBG] osdmap e827: 8 total, 8 up, 8 in 2024-04-07T16:30:14.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:30:14 smithi144 bash[66654]: audit 2024-04-07T16:30:13.388516+0000 mon.a (mon.0) 1447 : audit [INF] from='client.? 172.21.15.100:0/875113546' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:30:15.392 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestCls2PCQueue.ReserveError (3034 ms) 2024-04-07T16:30:15.392 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestCls2PCQueue.CommitError 2024-04-07T16:30:15.689 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:30:15 smithi100 bash[74942]: audit 2024-04-07T16:30:14.366940+0000 mon.a (mon.0) 1448 : audit [INF] from='client.? 172.21.15.100:0/875113546' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:30:15.689 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:30:15 smithi100 bash[74942]: cluster 2024-04-07T16:30:14.375811+0000 mon.a (mon.0) 1449 : cluster [DBG] osdmap e828: 8 total, 8 up, 8 in 2024-04-07T16:30:15.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:30:15 smithi100 bash[76202]: audit 2024-04-07T16:30:14.366940+0000 mon.a (mon.0) 1448 : audit [INF] from='client.? 172.21.15.100:0/875113546' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:30:15.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:30:15 smithi100 bash[76202]: cluster 2024-04-07T16:30:14.375811+0000 mon.a (mon.0) 1449 : cluster [DBG] osdmap e828: 8 total, 8 up, 8 in 2024-04-07T16:30:15.856 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:30:15 smithi144 bash[66654]: audit 2024-04-07T16:30:14.366940+0000 mon.a (mon.0) 1448 : audit [INF] from='client.? 172.21.15.100:0/875113546' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:30:15.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:30:15 smithi144 bash[66654]: cluster 2024-04-07T16:30:14.375811+0000 mon.a (mon.0) 1449 : cluster [DBG] osdmap e828: 8 total, 8 up, 8 in 2024-04-07T16:30:16.689 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:30:16 smithi100 bash[74942]: cluster 2024-04-07T16:30:14.862211+0000 mgr.y (mgr.25175) 447 : cluster [DBG] pgmap v850: 137 pgs: 4 creating+peering, 28 unknown, 105 active+clean; 583 KiB data, 1.7 GiB used, 714 GiB / 715 GiB avail 2024-04-07T16:30:16.689 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:30:16 smithi100 bash[74942]: cluster 2024-04-07T16:30:15.401885+0000 mon.a (mon.0) 1450 : cluster [DBG] osdmap e829: 8 total, 8 up, 8 in 2024-04-07T16:30:16.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:30:16 smithi100 bash[76202]: cluster 2024-04-07T16:30:14.862211+0000 mgr.y (mgr.25175) 447 : cluster [DBG] pgmap v850: 137 pgs: 4 creating+peering, 28 unknown, 105 active+clean; 583 KiB data, 1.7 GiB used, 714 GiB / 715 GiB avail 2024-04-07T16:30:16.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:30:16 smithi100 bash[76202]: cluster 2024-04-07T16:30:15.401885+0000 mon.a (mon.0) 1450 : cluster [DBG] osdmap e829: 8 total, 8 up, 8 in 2024-04-07T16:30:16.856 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:30:16 smithi144 bash[66654]: cluster 2024-04-07T16:30:14.862211+0000 mgr.y (mgr.25175) 447 : cluster [DBG] pgmap v850: 137 pgs: 4 creating+peering, 28 unknown, 105 active+clean; 583 KiB data, 1.7 GiB used, 714 GiB / 715 GiB avail 2024-04-07T16:30:16.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:30:16 smithi144 bash[66654]: cluster 2024-04-07T16:30:15.401885+0000 mon.a (mon.0) 1450 : cluster [DBG] osdmap e829: 8 total, 8 up, 8 in 2024-04-07T16:30:17.689 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:30:17 smithi100 bash[74942]: cluster 2024-04-07T16:30:16.408923+0000 mon.a (mon.0) 1451 : cluster [DBG] osdmap e830: 8 total, 8 up, 8 in 2024-04-07T16:30:17.689 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:30:17 smithi100 bash[74942]: audit 2024-04-07T16:30:16.413168+0000 mon.c (mon.2) 23 : audit [INF] from='client.? 172.21.15.100:0/1690623690' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:30:17.689 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:30:17 smithi100 bash[74942]: audit 2024-04-07T16:30:16.420031+0000 mon.a (mon.0) 1452 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:30:17.690 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:30:17 smithi100 bash[74942]: audit 2024-04-07T16:30:16.745350+0000 mon.a (mon.0) 1453 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:30:17.690 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:30:17 smithi100 bash[74942]: audit 2024-04-07T16:30:16.747608+0000 mon.a (mon.0) 1454 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:30:17.690 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:30:17 smithi100 bash[74942]: audit 2024-04-07T16:30:17.401499+0000 mon.a (mon.0) 1455 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:30:17.690 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:30:17 smithi100 bash[74942]: cluster 2024-04-07T16:30:17.410169+0000 mon.a (mon.0) 1456 : cluster [DBG] osdmap e831: 8 total, 8 up, 8 in 2024-04-07T16:30:17.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:30:17 smithi100 bash[76202]: cluster 2024-04-07T16:30:16.408923+0000 mon.a (mon.0) 1451 : cluster [DBG] osdmap e830: 8 total, 8 up, 8 in 2024-04-07T16:30:17.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:30:17 smithi100 bash[76202]: audit 2024-04-07T16:30:16.413168+0000 mon.c (mon.2) 23 : audit [INF] from='client.? 172.21.15.100:0/1690623690' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:30:17.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:30:17 smithi100 bash[76202]: audit 2024-04-07T16:30:16.420031+0000 mon.a (mon.0) 1452 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:30:17.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:30:17 smithi100 bash[76202]: audit 2024-04-07T16:30:16.745350+0000 mon.a (mon.0) 1453 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:30:17.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:30:17 smithi100 bash[76202]: audit 2024-04-07T16:30:16.747608+0000 mon.a (mon.0) 1454 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:30:17.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:30:17 smithi100 bash[76202]: audit 2024-04-07T16:30:17.401499+0000 mon.a (mon.0) 1455 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:30:17.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:30:17 smithi100 bash[76202]: cluster 2024-04-07T16:30:17.410169+0000 mon.a (mon.0) 1456 : cluster [DBG] osdmap e831: 8 total, 8 up, 8 in 2024-04-07T16:30:17.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:30:17 smithi144 bash[66654]: cluster 2024-04-07T16:30:16.408923+0000 mon.a (mon.0) 1451 : cluster [DBG] osdmap e830: 8 total, 8 up, 8 in 2024-04-07T16:30:17.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:30:17 smithi144 bash[66654]: audit 2024-04-07T16:30:16.413168+0000 mon.c (mon.2) 23 : audit [INF] from='client.? 172.21.15.100:0/1690623690' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:30:17.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:30:17 smithi144 bash[66654]: audit 2024-04-07T16:30:16.420031+0000 mon.a (mon.0) 1452 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:30:17.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:30:17 smithi144 bash[66654]: audit 2024-04-07T16:30:16.745350+0000 mon.a (mon.0) 1453 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:30:17.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:30:17 smithi144 bash[66654]: audit 2024-04-07T16:30:16.747608+0000 mon.a (mon.0) 1454 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:30:17.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:30:17 smithi144 bash[66654]: audit 2024-04-07T16:30:17.401499+0000 mon.a (mon.0) 1455 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:30:17.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:30:17 smithi144 bash[66654]: cluster 2024-04-07T16:30:17.410169+0000 mon.a (mon.0) 1456 : cluster [DBG] osdmap e831: 8 total, 8 up, 8 in 2024-04-07T16:30:18.439 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestCls2PCQueue.CommitError (3047 ms) 2024-04-07T16:30:18.439 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestCls2PCQueue.AbortError 2024-04-07T16:30:18.689 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:30:18 smithi100 bash[74942]: cluster 2024-04-07T16:30:16.862917+0000 mgr.y (mgr.25175) 448 : cluster [DBG] pgmap v853: 137 pgs: 32 unknown, 105 active+clean; 583 KiB data, 1.7 GiB used, 714 GiB / 715 GiB avail 2024-04-07T16:30:18.689 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:30:18 smithi100 bash[74942]: cluster 2024-04-07T16:30:17.740612+0000 mon.a (mon.0) 1457 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T16:30:18.689 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:30:18 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:30:18] "GET /metrics HTTP/1.1" 200 36355 "" "Prometheus/2.43.0" 2024-04-07T16:30:18.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:30:18 smithi100 bash[76202]: cluster 2024-04-07T16:30:16.862917+0000 mgr.y (mgr.25175) 448 : cluster [DBG] pgmap v853: 137 pgs: 32 unknown, 105 active+clean; 583 KiB data, 1.7 GiB used, 714 GiB / 715 GiB avail 2024-04-07T16:30:18.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:30:18 smithi100 bash[76202]: cluster 2024-04-07T16:30:17.740612+0000 mon.a (mon.0) 1457 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T16:30:18.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:30:18 smithi144 bash[66654]: cluster 2024-04-07T16:30:16.862917+0000 mgr.y (mgr.25175) 448 : cluster [DBG] pgmap v853: 137 pgs: 32 unknown, 105 active+clean; 583 KiB data, 1.7 GiB used, 714 GiB / 715 GiB avail 2024-04-07T16:30:18.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:30:18 smithi144 bash[66654]: cluster 2024-04-07T16:30:17.740612+0000 mon.a (mon.0) 1457 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T16:30:19.856 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:30:19 smithi144 bash[66654]: cluster 2024-04-07T16:30:18.446024+0000 mon.a (mon.0) 1458 : cluster [DBG] osdmap e832: 8 total, 8 up, 8 in 2024-04-07T16:30:19.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:30:19 smithi144 bash[66654]: cluster 2024-04-07T16:30:18.863722+0000 mgr.y (mgr.25175) 449 : cluster [DBG] pgmap v856: 105 pgs: 105 active+clean; 583 KiB data, 1.7 GiB used, 714 GiB / 715 GiB avail 2024-04-07T16:30:19.939 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:30:19 smithi100 bash[74942]: cluster 2024-04-07T16:30:18.446024+0000 mon.a (mon.0) 1458 : cluster [DBG] osdmap e832: 8 total, 8 up, 8 in 2024-04-07T16:30:19.939 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:30:19 smithi100 bash[74942]: cluster 2024-04-07T16:30:18.863722+0000 mgr.y (mgr.25175) 449 : cluster [DBG] pgmap v856: 105 pgs: 105 active+clean; 583 KiB data, 1.7 GiB used, 714 GiB / 715 GiB avail 2024-04-07T16:30:19.939 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:30:19 smithi100 bash[76202]: cluster 2024-04-07T16:30:18.446024+0000 mon.a (mon.0) 1458 : cluster [DBG] osdmap e832: 8 total, 8 up, 8 in 2024-04-07T16:30:19.940 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:30:19 smithi100 bash[76202]: cluster 2024-04-07T16:30:18.863722+0000 mgr.y (mgr.25175) 449 : cluster [DBG] pgmap v856: 105 pgs: 105 active+clean; 583 KiB data, 1.7 GiB used, 714 GiB / 715 GiB avail 2024-04-07T16:30:20.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:30:20 smithi144 bash[66654]: cluster 2024-04-07T16:30:19.440505+0000 mon.a (mon.0) 1459 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T16:30:20.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:30:20 smithi144 bash[66654]: cluster 2024-04-07T16:30:19.440573+0000 mon.a (mon.0) 1460 : cluster [INF] Cluster is now healthy 2024-04-07T16:30:20.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:30:20 smithi144 bash[66654]: cluster 2024-04-07T16:30:19.456353+0000 mon.a (mon.0) 1461 : cluster [DBG] osdmap e833: 8 total, 8 up, 8 in 2024-04-07T16:30:20.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:30:20 smithi144 bash[66654]: audit 2024-04-07T16:30:19.465059+0000 mon.a (mon.0) 1462 : audit [INF] from='client.? 172.21.15.100:0/1912698430' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:30:20.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:30:20 smithi144 bash[66654]: audit 2024-04-07T16:30:20.445756+0000 mon.a (mon.0) 1463 : audit [INF] from='client.? 172.21.15.100:0/1912698430' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:30:20.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:30:20 smithi144 bash[66654]: cluster 2024-04-07T16:30:20.455188+0000 mon.a (mon.0) 1464 : cluster [DBG] osdmap e834: 8 total, 8 up, 8 in 2024-04-07T16:30:20.939 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:30:20 smithi100 bash[74942]: cluster 2024-04-07T16:30:19.440505+0000 mon.a (mon.0) 1459 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T16:30:20.939 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:30:20 smithi100 bash[74942]: cluster 2024-04-07T16:30:19.440573+0000 mon.a (mon.0) 1460 : cluster [INF] Cluster is now healthy 2024-04-07T16:30:20.939 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:30:20 smithi100 bash[74942]: cluster 2024-04-07T16:30:19.456353+0000 mon.a (mon.0) 1461 : cluster [DBG] osdmap e833: 8 total, 8 up, 8 in 2024-04-07T16:30:20.939 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:30:20 smithi100 bash[74942]: audit 2024-04-07T16:30:19.465059+0000 mon.a (mon.0) 1462 : audit [INF] from='client.? 172.21.15.100:0/1912698430' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:30:20.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:30:20 smithi100 bash[74942]: audit 2024-04-07T16:30:20.445756+0000 mon.a (mon.0) 1463 : audit [INF] from='client.? 172.21.15.100:0/1912698430' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:30:20.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:30:20 smithi100 bash[74942]: cluster 2024-04-07T16:30:20.455188+0000 mon.a (mon.0) 1464 : cluster [DBG] osdmap e834: 8 total, 8 up, 8 in 2024-04-07T16:30:20.940 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:30:20 smithi100 bash[76202]: cluster 2024-04-07T16:30:19.440505+0000 mon.a (mon.0) 1459 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T16:30:20.940 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:30:20 smithi100 bash[76202]: cluster 2024-04-07T16:30:19.440573+0000 mon.a (mon.0) 1460 : cluster [INF] Cluster is now healthy 2024-04-07T16:30:20.940 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:30:20 smithi100 bash[76202]: cluster 2024-04-07T16:30:19.456353+0000 mon.a (mon.0) 1461 : cluster [DBG] osdmap e833: 8 total, 8 up, 8 in 2024-04-07T16:30:20.940 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:30:20 smithi100 bash[76202]: audit 2024-04-07T16:30:19.465059+0000 mon.a (mon.0) 1462 : audit [INF] from='client.? 172.21.15.100:0/1912698430' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:30:20.940 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:30:20 smithi100 bash[76202]: audit 2024-04-07T16:30:20.445756+0000 mon.a (mon.0) 1463 : audit [INF] from='client.? 172.21.15.100:0/1912698430' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:30:20.940 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:30:20 smithi100 bash[76202]: cluster 2024-04-07T16:30:20.455188+0000 mon.a (mon.0) 1464 : cluster [DBG] osdmap e834: 8 total, 8 up, 8 in 2024-04-07T16:30:21.477 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestCls2PCQueue.AbortError (3037 ms) 2024-04-07T16:30:21.477 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestCls2PCQueue.MultiReserve 2024-04-07T16:30:21.856 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:30:21 smithi144 bash[66654]: cluster 2024-04-07T16:30:20.864427+0000 mgr.y (mgr.25175) 450 : cluster [DBG] pgmap v859: 137 pgs: 4 creating+peering, 28 unknown, 105 active+clean; 583 KiB data, 1.7 GiB used, 714 GiB / 715 GiB avail 2024-04-07T16:30:21.939 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:30:21 smithi100 bash[74942]: cluster 2024-04-07T16:30:20.864427+0000 mgr.y (mgr.25175) 450 : cluster [DBG] pgmap v859: 137 pgs: 4 creating+peering, 28 unknown, 105 active+clean; 583 KiB data, 1.7 GiB used, 714 GiB / 715 GiB avail 2024-04-07T16:30:21.939 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:30:21 smithi100 bash[76202]: cluster 2024-04-07T16:30:20.864427+0000 mgr.y (mgr.25175) 450 : cluster [DBG] pgmap v859: 137 pgs: 4 creating+peering, 28 unknown, 105 active+clean; 583 KiB data, 1.7 GiB used, 714 GiB / 715 GiB avail 2024-04-07T16:30:22.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:30:22 smithi144 bash[66654]: cluster 2024-04-07T16:30:21.487756+0000 mon.a (mon.0) 1465 : cluster [DBG] osdmap e835: 8 total, 8 up, 8 in 2024-04-07T16:30:22.607 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:30:22 smithi144 bash[95233]: ts=2024-04-07T16:30:22.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:30:22.939 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:30:22 smithi100 bash[74942]: cluster 2024-04-07T16:30:21.487756+0000 mon.a (mon.0) 1465 : cluster [DBG] osdmap e835: 8 total, 8 up, 8 in 2024-04-07T16:30:22.939 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:30:22 smithi100 bash[76202]: cluster 2024-04-07T16:30:21.487756+0000 mon.a (mon.0) 1465 : cluster [DBG] osdmap e835: 8 total, 8 up, 8 in 2024-04-07T16:30:23.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:30:23 smithi144 bash[66654]: cluster 2024-04-07T16:30:22.495679+0000 mon.a (mon.0) 1466 : cluster [DBG] osdmap e836: 8 total, 8 up, 8 in 2024-04-07T16:30:23.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:30:23 smithi144 bash[66654]: audit 2024-04-07T16:30:22.498500+0000 mon.c (mon.2) 24 : audit [INF] from='client.? 172.21.15.100:0/3927790429' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:30:23.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:30:23 smithi144 bash[66654]: audit 2024-04-07T16:30:22.503249+0000 mon.a (mon.0) 1467 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:30:23.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:30:23 smithi144 bash[66654]: cluster 2024-04-07T16:30:22.865187+0000 mgr.y (mgr.25175) 451 : cluster [DBG] pgmap v862: 137 pgs: 32 unknown, 105 active+clean; 583 KiB data, 1.7 GiB used, 714 GiB / 715 GiB avail 2024-04-07T16:30:23.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:30:23 smithi144 bash[66654]: audit 2024-04-07T16:30:23.486016+0000 mon.a (mon.0) 1468 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:30:23.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:30:23 smithi144 bash[66654]: cluster 2024-04-07T16:30:23.494662+0000 mon.a (mon.0) 1469 : cluster [DBG] osdmap e837: 8 total, 8 up, 8 in 2024-04-07T16:30:23.939 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:30:23 smithi100 bash[74942]: cluster 2024-04-07T16:30:22.495679+0000 mon.a (mon.0) 1466 : cluster [DBG] osdmap e836: 8 total, 8 up, 8 in 2024-04-07T16:30:23.939 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:30:23 smithi100 bash[74942]: audit 2024-04-07T16:30:22.498500+0000 mon.c (mon.2) 24 : audit [INF] from='client.? 172.21.15.100:0/3927790429' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:30:23.939 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:30:23 smithi100 bash[74942]: audit 2024-04-07T16:30:22.503249+0000 mon.a (mon.0) 1467 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:30:23.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:30:23 smithi100 bash[74942]: cluster 2024-04-07T16:30:22.865187+0000 mgr.y (mgr.25175) 451 : cluster [DBG] pgmap v862: 137 pgs: 32 unknown, 105 active+clean; 583 KiB data, 1.7 GiB used, 714 GiB / 715 GiB avail 2024-04-07T16:30:23.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:30:23 smithi100 bash[74942]: audit 2024-04-07T16:30:23.486016+0000 mon.a (mon.0) 1468 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:30:23.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:30:23 smithi100 bash[74942]: cluster 2024-04-07T16:30:23.494662+0000 mon.a (mon.0) 1469 : cluster [DBG] osdmap e837: 8 total, 8 up, 8 in 2024-04-07T16:30:23.940 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:30:23 smithi100 bash[76202]: cluster 2024-04-07T16:30:22.495679+0000 mon.a (mon.0) 1466 : cluster [DBG] osdmap e836: 8 total, 8 up, 8 in 2024-04-07T16:30:23.940 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:30:23 smithi100 bash[76202]: audit 2024-04-07T16:30:22.498500+0000 mon.c (mon.2) 24 : audit [INF] from='client.? 172.21.15.100:0/3927790429' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:30:23.940 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:30:23 smithi100 bash[76202]: audit 2024-04-07T16:30:22.503249+0000 mon.a (mon.0) 1467 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:30:23.940 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:30:23 smithi100 bash[76202]: cluster 2024-04-07T16:30:22.865187+0000 mgr.y (mgr.25175) 451 : cluster [DBG] pgmap v862: 137 pgs: 32 unknown, 105 active+clean; 583 KiB data, 1.7 GiB used, 714 GiB / 715 GiB avail 2024-04-07T16:30:23.940 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:30:23 smithi100 bash[76202]: audit 2024-04-07T16:30:23.486016+0000 mon.a (mon.0) 1468 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:30:23.940 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:30:23 smithi100 bash[76202]: cluster 2024-04-07T16:30:23.494662+0000 mon.a (mon.0) 1469 : cluster [DBG] osdmap e837: 8 total, 8 up, 8 in 2024-04-07T16:30:24.507 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestCls2PCQueue.MultiReserve (3029 ms) 2024-04-07T16:30:24.507 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestCls2PCQueue.MultiCommit 2024-04-07T16:30:24.856 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:30:24 smithi144 bash[66654]: cluster 2024-04-07T16:30:23.503956+0000 mon.a (mon.0) 1470 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T16:30:24.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:30:24 smithi144 bash[66654]: cluster 2024-04-07T16:30:24.516797+0000 mon.a (mon.0) 1471 : cluster [DBG] osdmap e838: 8 total, 8 up, 8 in 2024-04-07T16:30:24.939 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:30:24 smithi100 bash[74942]: cluster 2024-04-07T16:30:23.503956+0000 mon.a (mon.0) 1470 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T16:30:24.939 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:30:24 smithi100 bash[74942]: cluster 2024-04-07T16:30:24.516797+0000 mon.a (mon.0) 1471 : cluster [DBG] osdmap e838: 8 total, 8 up, 8 in 2024-04-07T16:30:24.940 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:30:24 smithi100 bash[76202]: cluster 2024-04-07T16:30:23.503956+0000 mon.a (mon.0) 1470 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T16:30:24.940 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:30:24 smithi100 bash[76202]: cluster 2024-04-07T16:30:24.516797+0000 mon.a (mon.0) 1471 : cluster [DBG] osdmap e838: 8 total, 8 up, 8 in 2024-04-07T16:30:25.856 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:30:25 smithi144 bash[66654]: cluster 2024-04-07T16:30:24.865810+0000 mgr.y (mgr.25175) 452 : cluster [DBG] pgmap v865: 105 pgs: 105 active+clean; 583 KiB data, 1.7 GiB used, 714 GiB / 715 GiB avail 2024-04-07T16:30:25.939 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:30:25 smithi100 bash[74942]: cluster 2024-04-07T16:30:24.865810+0000 mgr.y (mgr.25175) 452 : cluster [DBG] pgmap v865: 105 pgs: 105 active+clean; 583 KiB data, 1.7 GiB used, 714 GiB / 715 GiB avail 2024-04-07T16:30:25.939 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:30:25 smithi100 bash[76202]: cluster 2024-04-07T16:30:24.865810+0000 mgr.y (mgr.25175) 452 : cluster [DBG] pgmap v865: 105 pgs: 105 active+clean; 583 KiB data, 1.7 GiB used, 714 GiB / 715 GiB avail 2024-04-07T16:30:26.856 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:30:26 smithi144 bash[66654]: cluster 2024-04-07T16:30:25.522103+0000 mon.a (mon.0) 1472 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T16:30:26.856 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:30:26 smithi144 bash[66654]: cluster 2024-04-07T16:30:25.522191+0000 mon.a (mon.0) 1473 : cluster [INF] Cluster is now healthy 2024-04-07T16:30:26.856 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:30:26 smithi144 bash[66654]: cluster 2024-04-07T16:30:25.545030+0000 mon.a (mon.0) 1474 : cluster [DBG] osdmap e839: 8 total, 8 up, 8 in 2024-04-07T16:30:26.856 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:30:26 smithi144 bash[66654]: audit 2024-04-07T16:30:25.547726+0000 mon.c (mon.2) 25 : audit [INF] from='client.? 172.21.15.100:0/761930268' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:30:26.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:30:26 smithi144 bash[66654]: audit 2024-04-07T16:30:25.549002+0000 mon.a (mon.0) 1475 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:30:26.939 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:30:26 smithi100 bash[74942]: cluster 2024-04-07T16:30:25.522103+0000 mon.a (mon.0) 1472 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T16:30:26.939 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:30:26 smithi100 bash[74942]: cluster 2024-04-07T16:30:25.522191+0000 mon.a (mon.0) 1473 : cluster [INF] Cluster is now healthy 2024-04-07T16:30:26.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:30:26 smithi100 bash[74942]: cluster 2024-04-07T16:30:25.545030+0000 mon.a (mon.0) 1474 : cluster [DBG] osdmap e839: 8 total, 8 up, 8 in 2024-04-07T16:30:26.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:30:26 smithi100 bash[74942]: audit 2024-04-07T16:30:25.547726+0000 mon.c (mon.2) 25 : audit [INF] from='client.? 172.21.15.100:0/761930268' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:30:26.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:30:26 smithi100 bash[74942]: audit 2024-04-07T16:30:25.549002+0000 mon.a (mon.0) 1475 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:30:26.940 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:30:26 smithi100 bash[76202]: cluster 2024-04-07T16:30:25.522103+0000 mon.a (mon.0) 1472 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T16:30:26.940 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:30:26 smithi100 bash[76202]: cluster 2024-04-07T16:30:25.522191+0000 mon.a (mon.0) 1473 : cluster [INF] Cluster is now healthy 2024-04-07T16:30:26.940 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:30:26 smithi100 bash[76202]: cluster 2024-04-07T16:30:25.545030+0000 mon.a (mon.0) 1474 : cluster [DBG] osdmap e839: 8 total, 8 up, 8 in 2024-04-07T16:30:26.940 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:30:26 smithi100 bash[76202]: audit 2024-04-07T16:30:25.547726+0000 mon.c (mon.2) 25 : audit [INF] from='client.? 172.21.15.100:0/761930268' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:30:26.940 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:30:26 smithi100 bash[76202]: audit 2024-04-07T16:30:25.549002+0000 mon.a (mon.0) 1475 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:30:27.570 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestCls2PCQueue.MultiCommit (3063 ms) 2024-04-07T16:30:27.570 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestCls2PCQueue.MultiAbort 2024-04-07T16:30:27.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:30:27 smithi144 bash[66654]: audit 2024-04-07T16:30:26.538394+0000 mon.a (mon.0) 1476 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:30:27.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:30:27 smithi144 bash[66654]: cluster 2024-04-07T16:30:26.546653+0000 mon.a (mon.0) 1477 : cluster [DBG] osdmap e840: 8 total, 8 up, 8 in 2024-04-07T16:30:27.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:30:27 smithi144 bash[66654]: cluster 2024-04-07T16:30:26.866656+0000 mgr.y (mgr.25175) 453 : cluster [DBG] pgmap v868: 137 pgs: 32 unknown, 105 active+clean; 583 KiB data, 1.7 GiB used, 714 GiB / 715 GiB avail 2024-04-07T16:30:27.939 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:30:27 smithi100 bash[74942]: audit 2024-04-07T16:30:26.538394+0000 mon.a (mon.0) 1476 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:30:27.939 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:30:27 smithi100 bash[74942]: cluster 2024-04-07T16:30:26.546653+0000 mon.a (mon.0) 1477 : cluster [DBG] osdmap e840: 8 total, 8 up, 8 in 2024-04-07T16:30:27.939 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:30:27 smithi100 bash[74942]: cluster 2024-04-07T16:30:26.866656+0000 mgr.y (mgr.25175) 453 : cluster [DBG] pgmap v868: 137 pgs: 32 unknown, 105 active+clean; 583 KiB data, 1.7 GiB used, 714 GiB / 715 GiB avail 2024-04-07T16:30:27.939 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:30:27 smithi100 bash[76202]: audit 2024-04-07T16:30:26.538394+0000 mon.a (mon.0) 1476 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:30:27.940 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:30:27 smithi100 bash[76202]: cluster 2024-04-07T16:30:26.546653+0000 mon.a (mon.0) 1477 : cluster [DBG] osdmap e840: 8 total, 8 up, 8 in 2024-04-07T16:30:27.940 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:30:27 smithi100 bash[76202]: cluster 2024-04-07T16:30:26.866656+0000 mgr.y (mgr.25175) 453 : cluster [DBG] pgmap v868: 137 pgs: 32 unknown, 105 active+clean; 583 KiB data, 1.7 GiB used, 714 GiB / 715 GiB avail 2024-04-07T16:30:28.689 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:30:28 smithi100 bash[74942]: cluster 2024-04-07T16:30:27.578892+0000 mon.a (mon.0) 1478 : cluster [DBG] osdmap e841: 8 total, 8 up, 8 in 2024-04-07T16:30:28.689 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:30:28 smithi100 bash[76202]: cluster 2024-04-07T16:30:27.578892+0000 mon.a (mon.0) 1478 : cluster [DBG] osdmap e841: 8 total, 8 up, 8 in 2024-04-07T16:30:28.690 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:30:28 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:30:28] "GET /metrics HTTP/1.1" 200 36355 "" "Prometheus/2.43.0" 2024-04-07T16:30:28.856 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:30:28 smithi144 bash[66654]: cluster 2024-04-07T16:30:27.578892+0000 mon.a (mon.0) 1478 : cluster [DBG] osdmap e841: 8 total, 8 up, 8 in 2024-04-07T16:30:29.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:30:29 smithi144 bash[66654]: cluster 2024-04-07T16:30:28.580547+0000 mon.a (mon.0) 1479 : cluster [DBG] osdmap e842: 8 total, 8 up, 8 in 2024-04-07T16:30:29.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:30:29 smithi144 bash[66654]: audit 2024-04-07T16:30:28.584268+0000 mon.c (mon.2) 26 : audit [INF] from='client.? 172.21.15.100:0/2096464765' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:30:29.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:30:29 smithi144 bash[66654]: audit 2024-04-07T16:30:28.598792+0000 mon.a (mon.0) 1480 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:30:29.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:30:29 smithi144 bash[66654]: cluster 2024-04-07T16:30:28.867484+0000 mgr.y (mgr.25175) 454 : cluster [DBG] pgmap v871: 137 pgs: 32 unknown, 105 active+clean; 583 KiB data, 1.7 GiB used, 714 GiB / 715 GiB avail 2024-04-07T16:30:29.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:30:29 smithi144 bash[66654]: audit 2024-04-07T16:30:29.573571+0000 mon.a (mon.0) 1481 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:30:29.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:30:29 smithi144 bash[66654]: cluster 2024-04-07T16:30:29.582831+0000 mon.a (mon.0) 1482 : cluster [DBG] osdmap e843: 8 total, 8 up, 8 in 2024-04-07T16:30:29.939 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:30:29 smithi100 bash[74942]: cluster 2024-04-07T16:30:28.580547+0000 mon.a (mon.0) 1479 : cluster [DBG] osdmap e842: 8 total, 8 up, 8 in 2024-04-07T16:30:29.939 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:30:29 smithi100 bash[74942]: audit 2024-04-07T16:30:28.584268+0000 mon.c (mon.2) 26 : audit [INF] from='client.? 172.21.15.100:0/2096464765' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:30:29.939 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:30:29 smithi100 bash[74942]: audit 2024-04-07T16:30:28.598792+0000 mon.a (mon.0) 1480 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:30:29.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:30:29 smithi100 bash[74942]: cluster 2024-04-07T16:30:28.867484+0000 mgr.y (mgr.25175) 454 : cluster [DBG] pgmap v871: 137 pgs: 32 unknown, 105 active+clean; 583 KiB data, 1.7 GiB used, 714 GiB / 715 GiB avail 2024-04-07T16:30:29.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:30:29 smithi100 bash[74942]: audit 2024-04-07T16:30:29.573571+0000 mon.a (mon.0) 1481 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:30:29.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:30:29 smithi100 bash[74942]: cluster 2024-04-07T16:30:29.582831+0000 mon.a (mon.0) 1482 : cluster [DBG] osdmap e843: 8 total, 8 up, 8 in 2024-04-07T16:30:29.940 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:30:29 smithi100 bash[76202]: cluster 2024-04-07T16:30:28.580547+0000 mon.a (mon.0) 1479 : cluster [DBG] osdmap e842: 8 total, 8 up, 8 in 2024-04-07T16:30:29.940 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:30:29 smithi100 bash[76202]: audit 2024-04-07T16:30:28.584268+0000 mon.c (mon.2) 26 : audit [INF] from='client.? 172.21.15.100:0/2096464765' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:30:29.940 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:30:29 smithi100 bash[76202]: audit 2024-04-07T16:30:28.598792+0000 mon.a (mon.0) 1480 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:30:29.940 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:30:29 smithi100 bash[76202]: cluster 2024-04-07T16:30:28.867484+0000 mgr.y (mgr.25175) 454 : cluster [DBG] pgmap v871: 137 pgs: 32 unknown, 105 active+clean; 583 KiB data, 1.7 GiB used, 714 GiB / 715 GiB avail 2024-04-07T16:30:29.940 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:30:29 smithi100 bash[76202]: audit 2024-04-07T16:30:29.573571+0000 mon.a (mon.0) 1481 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:30:29.940 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:30:29 smithi100 bash[76202]: cluster 2024-04-07T16:30:29.582831+0000 mon.a (mon.0) 1482 : cluster [DBG] osdmap e843: 8 total, 8 up, 8 in 2024-04-07T16:30:30.598 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestCls2PCQueue.MultiAbort (3028 ms) 2024-04-07T16:30:30.598 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestCls2PCQueue.ReserveCommit 2024-04-07T16:30:30.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:30:30 smithi144 bash[66654]: cluster 2024-04-07T16:30:29.589553+0000 mon.a (mon.0) 1483 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T16:30:30.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:30:30 smithi144 bash[66654]: cluster 2024-04-07T16:30:30.605953+0000 mon.a (mon.0) 1484 : cluster [DBG] osdmap e844: 8 total, 8 up, 8 in 2024-04-07T16:30:30.939 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:30:30 smithi100 bash[76202]: cluster 2024-04-07T16:30:29.589553+0000 mon.a (mon.0) 1483 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T16:30:30.939 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:30:30 smithi100 bash[76202]: cluster 2024-04-07T16:30:30.605953+0000 mon.a (mon.0) 1484 : cluster [DBG] osdmap e844: 8 total, 8 up, 8 in 2024-04-07T16:30:30.939 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:30:30 smithi100 bash[74942]: cluster 2024-04-07T16:30:29.589553+0000 mon.a (mon.0) 1483 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T16:30:30.939 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:30:30 smithi100 bash[74942]: cluster 2024-04-07T16:30:30.605953+0000 mon.a (mon.0) 1484 : cluster [DBG] osdmap e844: 8 total, 8 up, 8 in 2024-04-07T16:30:31.939 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:30:31 smithi100 bash[76202]: cluster 2024-04-07T16:30:30.868087+0000 mgr.y (mgr.25175) 455 : cluster [DBG] pgmap v874: 105 pgs: 105 active+clean; 583 KiB data, 1.7 GiB used, 714 GiB / 715 GiB avail 2024-04-07T16:30:31.939 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:30:31 smithi100 bash[74942]: cluster 2024-04-07T16:30:30.868087+0000 mgr.y (mgr.25175) 455 : cluster [DBG] pgmap v874: 105 pgs: 105 active+clean; 583 KiB data, 1.7 GiB used, 714 GiB / 715 GiB avail 2024-04-07T16:30:32.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:30:31 smithi144 bash[66654]: cluster 2024-04-07T16:30:30.868087+0000 mgr.y (mgr.25175) 455 : cluster [DBG] pgmap v874: 105 pgs: 105 active+clean; 583 KiB data, 1.7 GiB used, 714 GiB / 715 GiB avail 2024-04-07T16:30:32.606 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:30:32 smithi144 bash[95233]: ts=2024-04-07T16:30:32.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:30:32.939 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:30:32 smithi100 bash[76202]: cluster 2024-04-07T16:30:31.615031+0000 mon.a (mon.0) 1485 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T16:30:32.939 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:30:32 smithi100 bash[76202]: cluster 2024-04-07T16:30:31.615111+0000 mon.a (mon.0) 1486 : cluster [INF] Cluster is now healthy 2024-04-07T16:30:32.939 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:30:32 smithi100 bash[76202]: cluster 2024-04-07T16:30:31.643856+0000 mon.a (mon.0) 1487 : cluster [DBG] osdmap e845: 8 total, 8 up, 8 in 2024-04-07T16:30:32.939 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:30:32 smithi100 bash[76202]: audit 2024-04-07T16:30:31.650791+0000 mon.c (mon.2) 27 : audit [INF] from='client.? 172.21.15.100:0/1314315385' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:30:32.939 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:30:32 smithi100 bash[76202]: audit 2024-04-07T16:30:31.652663+0000 mon.a (mon.0) 1488 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:30:32.940 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:30:32 smithi100 bash[76202]: audit 2024-04-07T16:30:31.739238+0000 mon.a (mon.0) 1489 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:30:32.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:30:32 smithi100 bash[74942]: cluster 2024-04-07T16:30:31.615031+0000 mon.a (mon.0) 1485 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T16:30:32.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:30:32 smithi100 bash[74942]: cluster 2024-04-07T16:30:31.615111+0000 mon.a (mon.0) 1486 : cluster [INF] Cluster is now healthy 2024-04-07T16:30:32.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:30:32 smithi100 bash[74942]: cluster 2024-04-07T16:30:31.643856+0000 mon.a (mon.0) 1487 : cluster [DBG] osdmap e845: 8 total, 8 up, 8 in 2024-04-07T16:30:32.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:30:32 smithi100 bash[74942]: audit 2024-04-07T16:30:31.650791+0000 mon.c (mon.2) 27 : audit [INF] from='client.? 172.21.15.100:0/1314315385' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:30:32.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:30:32 smithi100 bash[74942]: audit 2024-04-07T16:30:31.652663+0000 mon.a (mon.0) 1488 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:30:32.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:30:32 smithi100 bash[74942]: audit 2024-04-07T16:30:31.739238+0000 mon.a (mon.0) 1489 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:30:33.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:30:32 smithi144 bash[66654]: cluster 2024-04-07T16:30:31.615031+0000 mon.a (mon.0) 1485 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T16:30:33.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:30:32 smithi144 bash[66654]: cluster 2024-04-07T16:30:31.615111+0000 mon.a (mon.0) 1486 : cluster [INF] Cluster is now healthy 2024-04-07T16:30:33.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:30:32 smithi144 bash[66654]: cluster 2024-04-07T16:30:31.643856+0000 mon.a (mon.0) 1487 : cluster [DBG] osdmap e845: 8 total, 8 up, 8 in 2024-04-07T16:30:33.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:30:32 smithi144 bash[66654]: audit 2024-04-07T16:30:31.650791+0000 mon.c (mon.2) 27 : audit [INF] from='client.? 172.21.15.100:0/1314315385' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:30:33.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:30:32 smithi144 bash[66654]: audit 2024-04-07T16:30:31.652663+0000 mon.a (mon.0) 1488 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:30:33.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:30:32 smithi144 bash[66654]: audit 2024-04-07T16:30:31.739238+0000 mon.a (mon.0) 1489 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:30:33.658 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestCls2PCQueue.ReserveCommit (3060 ms) 2024-04-07T16:30:33.658 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestCls2PCQueue.ReserveAbort 2024-04-07T16:30:33.939 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:30:33 smithi100 bash[74942]: audit 2024-04-07T16:30:32.645934+0000 mon.a (mon.0) 1490 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:30:33.939 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:30:33 smithi100 bash[74942]: cluster 2024-04-07T16:30:32.655940+0000 mon.a (mon.0) 1491 : cluster [DBG] osdmap e846: 8 total, 8 up, 8 in 2024-04-07T16:30:33.939 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:30:33 smithi100 bash[74942]: cluster 2024-04-07T16:30:32.868880+0000 mgr.y (mgr.25175) 456 : cluster [DBG] pgmap v877: 137 pgs: 13 creating+peering, 19 unknown, 105 active+clean; 583 KiB data, 1.7 GiB used, 714 GiB / 715 GiB avail 2024-04-07T16:30:33.939 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:30:33 smithi100 bash[76202]: audit 2024-04-07T16:30:32.645934+0000 mon.a (mon.0) 1490 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:30:33.939 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:30:33 smithi100 bash[76202]: cluster 2024-04-07T16:30:32.655940+0000 mon.a (mon.0) 1491 : cluster [DBG] osdmap e846: 8 total, 8 up, 8 in 2024-04-07T16:30:33.939 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:30:33 smithi100 bash[76202]: cluster 2024-04-07T16:30:32.868880+0000 mgr.y (mgr.25175) 456 : cluster [DBG] pgmap v877: 137 pgs: 13 creating+peering, 19 unknown, 105 active+clean; 583 KiB data, 1.7 GiB used, 714 GiB / 715 GiB avail 2024-04-07T16:30:34.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:30:33 smithi144 bash[66654]: audit 2024-04-07T16:30:32.645934+0000 mon.a (mon.0) 1490 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:30:34.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:30:33 smithi144 bash[66654]: cluster 2024-04-07T16:30:32.655940+0000 mon.a (mon.0) 1491 : cluster [DBG] osdmap e846: 8 total, 8 up, 8 in 2024-04-07T16:30:34.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:30:33 smithi144 bash[66654]: cluster 2024-04-07T16:30:32.868880+0000 mgr.y (mgr.25175) 456 : cluster [DBG] pgmap v877: 137 pgs: 13 creating+peering, 19 unknown, 105 active+clean; 583 KiB data, 1.7 GiB used, 714 GiB / 715 GiB avail 2024-04-07T16:30:34.939 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:30:34 smithi100 bash[74942]: cluster 2024-04-07T16:30:33.664144+0000 mon.a (mon.0) 1492 : cluster [DBG] osdmap e847: 8 total, 8 up, 8 in 2024-04-07T16:30:34.939 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:30:34 smithi100 bash[74942]: audit 2024-04-07T16:30:33.910628+0000 mon.a (mon.0) 1493 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:30:34.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:30:34 smithi100 bash[74942]: audit 2024-04-07T16:30:34.640110+0000 mon.a (mon.0) 1494 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:30:34.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:30:34 smithi100 bash[74942]: audit 2024-04-07T16:30:34.641936+0000 mon.a (mon.0) 1495 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:30:34.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:30:34 smithi100 bash[74942]: audit 2024-04-07T16:30:34.653505+0000 mon.a (mon.0) 1496 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:30:34.940 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:30:34 smithi100 bash[76202]: cluster 2024-04-07T16:30:33.664144+0000 mon.a (mon.0) 1492 : cluster [DBG] osdmap e847: 8 total, 8 up, 8 in 2024-04-07T16:30:34.940 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:30:34 smithi100 bash[76202]: audit 2024-04-07T16:30:33.910628+0000 mon.a (mon.0) 1493 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:30:34.940 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:30:34 smithi100 bash[76202]: audit 2024-04-07T16:30:34.640110+0000 mon.a (mon.0) 1494 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:30:34.940 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:30:34 smithi100 bash[76202]: audit 2024-04-07T16:30:34.641936+0000 mon.a (mon.0) 1495 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:30:34.940 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:30:34 smithi100 bash[76202]: audit 2024-04-07T16:30:34.653505+0000 mon.a (mon.0) 1496 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:30:35.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:30:34 smithi144 bash[66654]: cluster 2024-04-07T16:30:33.664144+0000 mon.a (mon.0) 1492 : cluster [DBG] osdmap e847: 8 total, 8 up, 8 in 2024-04-07T16:30:35.125 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:30:34 smithi144 bash[66654]: audit 2024-04-07T16:30:33.910628+0000 mon.a (mon.0) 1493 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:30:35.125 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:30:34 smithi144 bash[66654]: audit 2024-04-07T16:30:34.640110+0000 mon.a (mon.0) 1494 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:30:35.125 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:30:34 smithi144 bash[66654]: audit 2024-04-07T16:30:34.641936+0000 mon.a (mon.0) 1495 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:30:35.125 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:30:34 smithi144 bash[66654]: audit 2024-04-07T16:30:34.653505+0000 mon.a (mon.0) 1496 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:30:35.939 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:30:35 smithi100 bash[76202]: cluster 2024-04-07T16:30:34.667855+0000 mon.a (mon.0) 1497 : cluster [DBG] osdmap e848: 8 total, 8 up, 8 in 2024-04-07T16:30:35.939 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:30:35 smithi100 bash[76202]: audit 2024-04-07T16:30:34.682935+0000 mon.a (mon.0) 1498 : audit [INF] from='client.? 172.21.15.100:0/190050032' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:30:35.939 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:30:35 smithi100 bash[76202]: cluster 2024-04-07T16:30:34.869622+0000 mgr.y (mgr.25175) 457 : cluster [DBG] pgmap v880: 137 pgs: 32 unknown, 105 active+clean; 583 KiB data, 1.7 GiB used, 714 GiB / 715 GiB avail 2024-04-07T16:30:35.939 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:30:35 smithi100 bash[76202]: audit 2024-04-07T16:30:35.660825+0000 mon.a (mon.0) 1499 : audit [INF] from='client.? 172.21.15.100:0/190050032' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:30:35.939 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:30:35 smithi100 bash[76202]: cluster 2024-04-07T16:30:35.670579+0000 mon.a (mon.0) 1500 : cluster [DBG] osdmap e849: 8 total, 8 up, 8 in 2024-04-07T16:30:35.939 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:30:35 smithi100 bash[74942]: cluster 2024-04-07T16:30:34.667855+0000 mon.a (mon.0) 1497 : cluster [DBG] osdmap e848: 8 total, 8 up, 8 in 2024-04-07T16:30:35.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:30:35 smithi100 bash[74942]: audit 2024-04-07T16:30:34.682935+0000 mon.a (mon.0) 1498 : audit [INF] from='client.? 172.21.15.100:0/190050032' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:30:35.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:30:35 smithi100 bash[74942]: cluster 2024-04-07T16:30:34.869622+0000 mgr.y (mgr.25175) 457 : cluster [DBG] pgmap v880: 137 pgs: 32 unknown, 105 active+clean; 583 KiB data, 1.7 GiB used, 714 GiB / 715 GiB avail 2024-04-07T16:30:35.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:30:35 smithi100 bash[74942]: audit 2024-04-07T16:30:35.660825+0000 mon.a (mon.0) 1499 : audit [INF] from='client.? 172.21.15.100:0/190050032' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:30:35.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:30:35 smithi100 bash[74942]: cluster 2024-04-07T16:30:35.670579+0000 mon.a (mon.0) 1500 : cluster [DBG] osdmap e849: 8 total, 8 up, 8 in 2024-04-07T16:30:36.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:30:35 smithi144 bash[66654]: cluster 2024-04-07T16:30:34.667855+0000 mon.a (mon.0) 1497 : cluster [DBG] osdmap e848: 8 total, 8 up, 8 in 2024-04-07T16:30:36.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:30:35 smithi144 bash[66654]: audit 2024-04-07T16:30:34.682935+0000 mon.a (mon.0) 1498 : audit [INF] from='client.? 172.21.15.100:0/190050032' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:30:36.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:30:35 smithi144 bash[66654]: cluster 2024-04-07T16:30:34.869622+0000 mgr.y (mgr.25175) 457 : cluster [DBG] pgmap v880: 137 pgs: 32 unknown, 105 active+clean; 583 KiB data, 1.7 GiB used, 714 GiB / 715 GiB avail 2024-04-07T16:30:36.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:30:35 smithi144 bash[66654]: audit 2024-04-07T16:30:35.660825+0000 mon.a (mon.0) 1499 : audit [INF] from='client.? 172.21.15.100:0/190050032' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:30:36.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:30:35 smithi144 bash[66654]: cluster 2024-04-07T16:30:35.670579+0000 mon.a (mon.0) 1500 : cluster [DBG] osdmap e849: 8 total, 8 up, 8 in 2024-04-07T16:30:36.683 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestCls2PCQueue.ReserveAbort (3025 ms) 2024-04-07T16:30:36.683 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestCls2PCQueue.ManualCleanup 2024-04-07T16:30:37.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:30:36 smithi144 bash[66654]: cluster 2024-04-07T16:30:35.676315+0000 mon.a (mon.0) 1501 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T16:30:37.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:30:36 smithi144 bash[66654]: cluster 2024-04-07T16:30:36.693014+0000 mon.a (mon.0) 1502 : cluster [DBG] osdmap e850: 8 total, 8 up, 8 in 2024-04-07T16:30:37.189 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:30:36 smithi100 bash[74942]: cluster 2024-04-07T16:30:35.676315+0000 mon.a (mon.0) 1501 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T16:30:37.189 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:30:36 smithi100 bash[74942]: cluster 2024-04-07T16:30:36.693014+0000 mon.a (mon.0) 1502 : cluster [DBG] osdmap e850: 8 total, 8 up, 8 in 2024-04-07T16:30:37.189 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:30:36 smithi100 bash[76202]: cluster 2024-04-07T16:30:35.676315+0000 mon.a (mon.0) 1501 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T16:30:37.190 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:30:36 smithi100 bash[76202]: cluster 2024-04-07T16:30:36.693014+0000 mon.a (mon.0) 1502 : cluster [DBG] osdmap e850: 8 total, 8 up, 8 in 2024-04-07T16:30:38.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:30:37 smithi144 bash[66654]: cluster 2024-04-07T16:30:36.870265+0000 mgr.y (mgr.25175) 458 : cluster [DBG] pgmap v883: 105 pgs: 105 active+clean; 583 KiB data, 1.7 GiB used, 714 GiB / 715 GiB avail 2024-04-07T16:30:38.189 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:30:37 smithi100 bash[74942]: cluster 2024-04-07T16:30:36.870265+0000 mgr.y (mgr.25175) 458 : cluster [DBG] pgmap v883: 105 pgs: 105 active+clean; 583 KiB data, 1.7 GiB used, 714 GiB / 715 GiB avail 2024-04-07T16:30:38.189 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:30:37 smithi100 bash[76202]: cluster 2024-04-07T16:30:36.870265+0000 mgr.y (mgr.25175) 458 : cluster [DBG] pgmap v883: 105 pgs: 105 active+clean; 583 KiB data, 1.7 GiB used, 714 GiB / 715 GiB avail 2024-04-07T16:30:38.689 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:30:38 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:30:38] "GET /metrics HTTP/1.1" 200 36359 "" "Prometheus/2.43.0" 2024-04-07T16:30:39.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:30:38 smithi144 bash[66654]: cluster 2024-04-07T16:30:37.715890+0000 mon.a (mon.0) 1503 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T16:30:39.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:30:38 smithi144 bash[66654]: cluster 2024-04-07T16:30:37.715958+0000 mon.a (mon.0) 1504 : cluster [INF] Cluster is now healthy 2024-04-07T16:30:39.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:30:38 smithi144 bash[66654]: cluster 2024-04-07T16:30:37.744093+0000 mon.a (mon.0) 1505 : cluster [DBG] osdmap e851: 8 total, 8 up, 8 in 2024-04-07T16:30:39.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:30:38 smithi144 bash[66654]: audit 2024-04-07T16:30:37.749421+0000 mon.c (mon.2) 28 : audit [INF] from='client.? 172.21.15.100:0/3941992345' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:30:39.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:30:38 smithi144 bash[66654]: audit 2024-04-07T16:30:37.751113+0000 mon.a (mon.0) 1506 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:30:39.189 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:30:38 smithi100 bash[74942]: cluster 2024-04-07T16:30:37.715890+0000 mon.a (mon.0) 1503 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T16:30:39.189 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:30:38 smithi100 bash[74942]: cluster 2024-04-07T16:30:37.715958+0000 mon.a (mon.0) 1504 : cluster [INF] Cluster is now healthy 2024-04-07T16:30:39.189 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:30:38 smithi100 bash[74942]: cluster 2024-04-07T16:30:37.744093+0000 mon.a (mon.0) 1505 : cluster [DBG] osdmap e851: 8 total, 8 up, 8 in 2024-04-07T16:30:39.189 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:30:38 smithi100 bash[74942]: audit 2024-04-07T16:30:37.749421+0000 mon.c (mon.2) 28 : audit [INF] from='client.? 172.21.15.100:0/3941992345' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:30:39.190 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:30:38 smithi100 bash[74942]: audit 2024-04-07T16:30:37.751113+0000 mon.a (mon.0) 1506 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:30:39.190 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:30:38 smithi100 bash[76202]: cluster 2024-04-07T16:30:37.715890+0000 mon.a (mon.0) 1503 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T16:30:39.190 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:30:38 smithi100 bash[76202]: cluster 2024-04-07T16:30:37.715958+0000 mon.a (mon.0) 1504 : cluster [INF] Cluster is now healthy 2024-04-07T16:30:39.190 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:30:38 smithi100 bash[76202]: cluster 2024-04-07T16:30:37.744093+0000 mon.a (mon.0) 1505 : cluster [DBG] osdmap e851: 8 total, 8 up, 8 in 2024-04-07T16:30:39.190 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:30:38 smithi100 bash[76202]: audit 2024-04-07T16:30:37.749421+0000 mon.c (mon.2) 28 : audit [INF] from='client.? 172.21.15.100:0/3941992345' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:30:39.190 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:30:38 smithi100 bash[76202]: audit 2024-04-07T16:30:37.751113+0000 mon.a (mon.0) 1506 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:30:39.753 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestCls2PCQueue.ManualCleanup (3067 ms) 2024-04-07T16:30:39.753 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestCls2PCQueue.Cleanup 2024-04-07T16:30:40.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:30:39 smithi144 bash[66654]: audit 2024-04-07T16:30:38.741282+0000 mon.a (mon.0) 1507 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:30:40.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:30:39 smithi144 bash[66654]: cluster 2024-04-07T16:30:38.749805+0000 mon.a (mon.0) 1508 : cluster [DBG] osdmap e852: 8 total, 8 up, 8 in 2024-04-07T16:30:40.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:30:39 smithi144 bash[66654]: cluster 2024-04-07T16:30:38.870781+0000 mgr.y (mgr.25175) 459 : cluster [DBG] pgmap v886: 137 pgs: 3 creating+peering, 29 unknown, 105 active+clean; 583 KiB data, 1.7 GiB used, 714 GiB / 715 GiB avail 2024-04-07T16:30:40.189 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:30:39 smithi100 bash[74942]: audit 2024-04-07T16:30:38.741282+0000 mon.a (mon.0) 1507 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:30:40.189 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:30:39 smithi100 bash[74942]: cluster 2024-04-07T16:30:38.749805+0000 mon.a (mon.0) 1508 : cluster [DBG] osdmap e852: 8 total, 8 up, 8 in 2024-04-07T16:30:40.189 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:30:39 smithi100 bash[74942]: cluster 2024-04-07T16:30:38.870781+0000 mgr.y (mgr.25175) 459 : cluster [DBG] pgmap v886: 137 pgs: 3 creating+peering, 29 unknown, 105 active+clean; 583 KiB data, 1.7 GiB used, 714 GiB / 715 GiB avail 2024-04-07T16:30:40.190 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:30:39 smithi100 bash[76202]: audit 2024-04-07T16:30:38.741282+0000 mon.a (mon.0) 1507 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:30:40.190 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:30:39 smithi100 bash[76202]: cluster 2024-04-07T16:30:38.749805+0000 mon.a (mon.0) 1508 : cluster [DBG] osdmap e852: 8 total, 8 up, 8 in 2024-04-07T16:30:40.190 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:30:39 smithi100 bash[76202]: cluster 2024-04-07T16:30:38.870781+0000 mgr.y (mgr.25175) 459 : cluster [DBG] pgmap v886: 137 pgs: 3 creating+peering, 29 unknown, 105 active+clean; 583 KiB data, 1.7 GiB used, 714 GiB / 715 GiB avail 2024-04-07T16:30:41.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:30:40 smithi144 bash[66654]: cluster 2024-04-07T16:30:39.760073+0000 mon.a (mon.0) 1509 : cluster [DBG] osdmap e853: 8 total, 8 up, 8 in 2024-04-07T16:30:41.189 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:30:40 smithi100 bash[74942]: cluster 2024-04-07T16:30:39.760073+0000 mon.a (mon.0) 1509 : cluster [DBG] osdmap e853: 8 total, 8 up, 8 in 2024-04-07T16:30:41.189 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:30:40 smithi100 bash[76202]: cluster 2024-04-07T16:30:39.760073+0000 mon.a (mon.0) 1509 : cluster [DBG] osdmap e853: 8 total, 8 up, 8 in 2024-04-07T16:30:42.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:30:41 smithi144 bash[66654]: cluster 2024-04-07T16:30:40.765035+0000 mon.a (mon.0) 1510 : cluster [DBG] osdmap e854: 8 total, 8 up, 8 in 2024-04-07T16:30:42.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:30:41 smithi144 bash[66654]: audit 2024-04-07T16:30:40.769038+0000 mon.c (mon.2) 29 : audit [INF] from='client.? 172.21.15.100:0/3214619960' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:30:42.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:30:41 smithi144 bash[66654]: audit 2024-04-07T16:30:40.780513+0000 mon.a (mon.0) 1511 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:30:42.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:30:41 smithi144 bash[66654]: cluster 2024-04-07T16:30:40.871530+0000 mgr.y (mgr.25175) 460 : cluster [DBG] pgmap v889: 137 pgs: 32 unknown, 105 active+clean; 583 KiB data, 1.7 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:30:42.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:30:41 smithi144 bash[66654]: audit 2024-04-07T16:30:41.757268+0000 mon.a (mon.0) 1512 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:30:42.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:30:41 smithi144 bash[66654]: cluster 2024-04-07T16:30:41.767551+0000 mon.a (mon.0) 1513 : cluster [DBG] osdmap e855: 8 total, 8 up, 8 in 2024-04-07T16:30:42.190 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:30:41 smithi100 bash[74942]: cluster 2024-04-07T16:30:40.765035+0000 mon.a (mon.0) 1510 : cluster [DBG] osdmap e854: 8 total, 8 up, 8 in 2024-04-07T16:30:42.190 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:30:41 smithi100 bash[74942]: audit 2024-04-07T16:30:40.769038+0000 mon.c (mon.2) 29 : audit [INF] from='client.? 172.21.15.100:0/3214619960' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:30:42.190 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:30:41 smithi100 bash[74942]: audit 2024-04-07T16:30:40.780513+0000 mon.a (mon.0) 1511 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:30:42.190 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:30:41 smithi100 bash[74942]: cluster 2024-04-07T16:30:40.871530+0000 mgr.y (mgr.25175) 460 : cluster [DBG] pgmap v889: 137 pgs: 32 unknown, 105 active+clean; 583 KiB data, 1.7 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:30:42.190 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:30:41 smithi100 bash[74942]: audit 2024-04-07T16:30:41.757268+0000 mon.a (mon.0) 1512 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:30:42.190 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:30:41 smithi100 bash[74942]: cluster 2024-04-07T16:30:41.767551+0000 mon.a (mon.0) 1513 : cluster [DBG] osdmap e855: 8 total, 8 up, 8 in 2024-04-07T16:30:42.190 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:30:41 smithi100 bash[76202]: cluster 2024-04-07T16:30:40.765035+0000 mon.a (mon.0) 1510 : cluster [DBG] osdmap e854: 8 total, 8 up, 8 in 2024-04-07T16:30:42.190 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:30:41 smithi100 bash[76202]: audit 2024-04-07T16:30:40.769038+0000 mon.c (mon.2) 29 : audit [INF] from='client.? 172.21.15.100:0/3214619960' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:30:42.191 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:30:41 smithi100 bash[76202]: audit 2024-04-07T16:30:40.780513+0000 mon.a (mon.0) 1511 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:30:42.191 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:30:41 smithi100 bash[76202]: cluster 2024-04-07T16:30:40.871530+0000 mgr.y (mgr.25175) 460 : cluster [DBG] pgmap v889: 137 pgs: 32 unknown, 105 active+clean; 583 KiB data, 1.7 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:30:42.191 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:30:41 smithi100 bash[76202]: audit 2024-04-07T16:30:41.757268+0000 mon.a (mon.0) 1512 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:30:42.191 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:30:41 smithi100 bash[76202]: cluster 2024-04-07T16:30:41.767551+0000 mon.a (mon.0) 1513 : cluster [DBG] osdmap e855: 8 total, 8 up, 8 in 2024-04-07T16:30:42.606 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:30:42 smithi144 bash[95233]: ts=2024-04-07T16:30:42.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:30:42.792 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestCls2PCQueue.Cleanup (3041 ms) 2024-04-07T16:30:42.792 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestCls2PCQueue.MultiProducer 2024-04-07T16:30:43.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:30:42 smithi144 bash[66654]: cluster 2024-04-07T16:30:41.770375+0000 mon.a (mon.0) 1514 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T16:30:43.189 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:30:42 smithi100 bash[74942]: cluster 2024-04-07T16:30:41.770375+0000 mon.a (mon.0) 1514 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T16:30:43.190 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:30:42 smithi100 bash[76202]: cluster 2024-04-07T16:30:41.770375+0000 mon.a (mon.0) 1514 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T16:30:44.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:30:43 smithi144 bash[66654]: cluster 2024-04-07T16:30:42.797086+0000 mon.a (mon.0) 1515 : cluster [DBG] osdmap e856: 8 total, 8 up, 8 in 2024-04-07T16:30:44.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:30:43 smithi144 bash[66654]: cluster 2024-04-07T16:30:42.872134+0000 mgr.y (mgr.25175) 461 : cluster [DBG] pgmap v892: 105 pgs: 105 active+clean; 583 KiB data, 1.7 GiB used, 714 GiB / 715 GiB avail 2024-04-07T16:30:44.189 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:30:43 smithi100 bash[74942]: cluster 2024-04-07T16:30:42.797086+0000 mon.a (mon.0) 1515 : cluster [DBG] osdmap e856: 8 total, 8 up, 8 in 2024-04-07T16:30:44.189 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:30:43 smithi100 bash[74942]: cluster 2024-04-07T16:30:42.872134+0000 mgr.y (mgr.25175) 461 : cluster [DBG] pgmap v892: 105 pgs: 105 active+clean; 583 KiB data, 1.7 GiB used, 714 GiB / 715 GiB avail 2024-04-07T16:30:44.189 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:30:43 smithi100 bash[76202]: cluster 2024-04-07T16:30:42.797086+0000 mon.a (mon.0) 1515 : cluster [DBG] osdmap e856: 8 total, 8 up, 8 in 2024-04-07T16:30:44.190 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:30:43 smithi100 bash[76202]: cluster 2024-04-07T16:30:42.872134+0000 mgr.y (mgr.25175) 461 : cluster [DBG] pgmap v892: 105 pgs: 105 active+clean; 583 KiB data, 1.7 GiB used, 714 GiB / 715 GiB avail 2024-04-07T16:30:45.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:30:44 smithi144 bash[66654]: cluster 2024-04-07T16:30:43.788813+0000 mon.a (mon.0) 1516 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T16:30:45.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:30:44 smithi144 bash[66654]: cluster 2024-04-07T16:30:43.788891+0000 mon.a (mon.0) 1517 : cluster [INF] Cluster is now healthy 2024-04-07T16:30:45.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:30:44 smithi144 bash[66654]: cluster 2024-04-07T16:30:43.816100+0000 mon.a (mon.0) 1518 : cluster [DBG] osdmap e857: 8 total, 8 up, 8 in 2024-04-07T16:30:45.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:30:44 smithi144 bash[66654]: audit 2024-04-07T16:30:43.823051+0000 mon.c (mon.2) 30 : audit [INF] from='client.? 172.21.15.100:0/2476972246' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:30:45.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:30:44 smithi144 bash[66654]: audit 2024-04-07T16:30:43.824368+0000 mon.a (mon.0) 1519 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:30:45.189 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:30:44 smithi100 bash[74942]: cluster 2024-04-07T16:30:43.788813+0000 mon.a (mon.0) 1516 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T16:30:45.189 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:30:44 smithi100 bash[74942]: cluster 2024-04-07T16:30:43.788891+0000 mon.a (mon.0) 1517 : cluster [INF] Cluster is now healthy 2024-04-07T16:30:45.189 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:30:44 smithi100 bash[74942]: cluster 2024-04-07T16:30:43.816100+0000 mon.a (mon.0) 1518 : cluster [DBG] osdmap e857: 8 total, 8 up, 8 in 2024-04-07T16:30:45.189 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:30:44 smithi100 bash[74942]: audit 2024-04-07T16:30:43.823051+0000 mon.c (mon.2) 30 : audit [INF] from='client.? 172.21.15.100:0/2476972246' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:30:45.189 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:30:44 smithi100 bash[74942]: audit 2024-04-07T16:30:43.824368+0000 mon.a (mon.0) 1519 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:30:45.189 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:30:44 smithi100 bash[76202]: cluster 2024-04-07T16:30:43.788813+0000 mon.a (mon.0) 1516 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T16:30:45.190 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:30:44 smithi100 bash[76202]: cluster 2024-04-07T16:30:43.788891+0000 mon.a (mon.0) 1517 : cluster [INF] Cluster is now healthy 2024-04-07T16:30:45.190 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:30:44 smithi100 bash[76202]: cluster 2024-04-07T16:30:43.816100+0000 mon.a (mon.0) 1518 : cluster [DBG] osdmap e857: 8 total, 8 up, 8 in 2024-04-07T16:30:45.190 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:30:44 smithi100 bash[76202]: audit 2024-04-07T16:30:43.823051+0000 mon.c (mon.2) 30 : audit [INF] from='client.? 172.21.15.100:0/2476972246' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:30:45.190 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:30:44 smithi100 bash[76202]: audit 2024-04-07T16:30:43.824368+0000 mon.a (mon.0) 1519 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:30:46.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:30:45 smithi144 bash[66654]: audit 2024-04-07T16:30:44.804892+0000 mon.a (mon.0) 1520 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:30:46.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:30:45 smithi144 bash[66654]: cluster 2024-04-07T16:30:44.814189+0000 mon.a (mon.0) 1521 : cluster [DBG] osdmap e858: 8 total, 8 up, 8 in 2024-04-07T16:30:46.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:30:45 smithi144 bash[66654]: cluster 2024-04-07T16:30:44.872662+0000 mgr.y (mgr.25175) 462 : cluster [DBG] pgmap v895: 137 pgs: 6 creating+peering, 26 unknown, 105 active+clean; 583 KiB data, 1.7 GiB used, 714 GiB / 715 GiB avail 2024-04-07T16:30:46.189 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:30:45 smithi100 bash[74942]: audit 2024-04-07T16:30:44.804892+0000 mon.a (mon.0) 1520 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:30:46.189 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:30:45 smithi100 bash[74942]: cluster 2024-04-07T16:30:44.814189+0000 mon.a (mon.0) 1521 : cluster [DBG] osdmap e858: 8 total, 8 up, 8 in 2024-04-07T16:30:46.189 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:30:45 smithi100 bash[74942]: cluster 2024-04-07T16:30:44.872662+0000 mgr.y (mgr.25175) 462 : cluster [DBG] pgmap v895: 137 pgs: 6 creating+peering, 26 unknown, 105 active+clean; 583 KiB data, 1.7 GiB used, 714 GiB / 715 GiB avail 2024-04-07T16:30:46.189 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:30:45 smithi100 bash[76202]: audit 2024-04-07T16:30:44.804892+0000 mon.a (mon.0) 1520 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:30:46.189 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:30:45 smithi100 bash[76202]: cluster 2024-04-07T16:30:44.814189+0000 mon.a (mon.0) 1521 : cluster [DBG] osdmap e858: 8 total, 8 up, 8 in 2024-04-07T16:30:46.189 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:30:45 smithi100 bash[76202]: cluster 2024-04-07T16:30:44.872662+0000 mgr.y (mgr.25175) 462 : cluster [DBG] pgmap v895: 137 pgs: 6 creating+peering, 26 unknown, 105 active+clean; 583 KiB data, 1.7 GiB used, 714 GiB / 715 GiB avail 2024-04-07T16:30:47.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:30:46 smithi144 bash[66654]: cluster 2024-04-07T16:30:45.842224+0000 mon.a (mon.0) 1522 : cluster [DBG] osdmap e859: 8 total, 8 up, 8 in 2024-04-07T16:30:47.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:30:46 smithi144 bash[66654]: audit 2024-04-07T16:30:46.738721+0000 mon.a (mon.0) 1523 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:30:47.189 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:30:46 smithi100 bash[74942]: cluster 2024-04-07T16:30:45.842224+0000 mon.a (mon.0) 1522 : cluster [DBG] osdmap e859: 8 total, 8 up, 8 in 2024-04-07T16:30:47.189 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:30:46 smithi100 bash[74942]: audit 2024-04-07T16:30:46.738721+0000 mon.a (mon.0) 1523 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:30:47.189 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:30:46 smithi100 bash[76202]: cluster 2024-04-07T16:30:45.842224+0000 mon.a (mon.0) 1522 : cluster [DBG] osdmap e859: 8 total, 8 up, 8 in 2024-04-07T16:30:47.189 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:30:46 smithi100 bash[76202]: audit 2024-04-07T16:30:46.738721+0000 mon.a (mon.0) 1523 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:30:48.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:30:47 smithi144 bash[66654]: cluster 2024-04-07T16:30:46.873107+0000 mgr.y (mgr.25175) 463 : cluster [DBG] pgmap v897: 137 pgs: 6 creating+peering, 18 unknown, 113 active+clean; 628 KiB data, 1.7 GiB used, 714 GiB / 715 GiB avail; 90 KiB/s rd, 234 KiB/s wr, 257 op/s 2024-04-07T16:30:48.189 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:30:47 smithi100 bash[74942]: cluster 2024-04-07T16:30:46.873107+0000 mgr.y (mgr.25175) 463 : cluster [DBG] pgmap v897: 137 pgs: 6 creating+peering, 18 unknown, 113 active+clean; 628 KiB data, 1.7 GiB used, 714 GiB / 715 GiB avail; 90 KiB/s rd, 234 KiB/s wr, 257 op/s 2024-04-07T16:30:48.190 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:30:47 smithi100 bash[76202]: cluster 2024-04-07T16:30:46.873107+0000 mgr.y (mgr.25175) 463 : cluster [DBG] pgmap v897: 137 pgs: 6 creating+peering, 18 unknown, 113 active+clean; 628 KiB data, 1.7 GiB used, 714 GiB / 715 GiB avail; 90 KiB/s rd, 234 KiB/s wr, 257 op/s 2024-04-07T16:30:48.689 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:30:48 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:30:48] "GET /metrics HTTP/1.1" 200 36395 "" "Prometheus/2.43.0" 2024-04-07T16:30:50.189 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:30:49 smithi100 bash[74942]: cluster 2024-04-07T16:30:48.874375+0000 mgr.y (mgr.25175) 464 : cluster [DBG] pgmap v898: 137 pgs: 6 creating+peering, 131 active+clean; 628 KiB data, 1.8 GiB used, 713 GiB / 715 GiB avail; 61 KiB/s rd, 159 KiB/s wr, 174 op/s 2024-04-07T16:30:50.189 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:30:49 smithi100 bash[76202]: cluster 2024-04-07T16:30:48.874375+0000 mgr.y (mgr.25175) 464 : cluster [DBG] pgmap v898: 137 pgs: 6 creating+peering, 131 active+clean; 628 KiB data, 1.8 GiB used, 713 GiB / 715 GiB avail; 61 KiB/s rd, 159 KiB/s wr, 174 op/s 2024-04-07T16:30:50.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:30:49 smithi144 bash[66654]: cluster 2024-04-07T16:30:48.874375+0000 mgr.y (mgr.25175) 464 : cluster [DBG] pgmap v898: 137 pgs: 6 creating+peering, 131 active+clean; 628 KiB data, 1.8 GiB used, 713 GiB / 715 GiB avail; 61 KiB/s rd, 159 KiB/s wr, 174 op/s 2024-04-07T16:30:52.189 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:30:51 smithi100 bash[74942]: cluster 2024-04-07T16:30:50.875604+0000 mgr.y (mgr.25175) 465 : cluster [DBG] pgmap v899: 137 pgs: 137 active+clean; 1.4 MiB data, 1.8 GiB used, 713 GiB / 715 GiB avail; 15 MiB/s rd, 4.9 MiB/s wr, 5.54k op/s 2024-04-07T16:30:52.189 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:30:51 smithi100 bash[76202]: cluster 2024-04-07T16:30:50.875604+0000 mgr.y (mgr.25175) 465 : cluster [DBG] pgmap v899: 137 pgs: 137 active+clean; 1.4 MiB data, 1.8 GiB used, 713 GiB / 715 GiB avail; 15 MiB/s rd, 4.9 MiB/s wr, 5.54k op/s 2024-04-07T16:30:52.306 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:30:51 smithi144 bash[66654]: cluster 2024-04-07T16:30:50.875604+0000 mgr.y (mgr.25175) 465 : cluster [DBG] pgmap v899: 137 pgs: 137 active+clean; 1.4 MiB data, 1.8 GiB used, 713 GiB / 715 GiB avail; 15 MiB/s rd, 4.9 MiB/s wr, 5.54k op/s 2024-04-07T16:30:52.606 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:30:52 smithi144 bash[95233]: ts=2024-04-07T16:30:52.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:30:54.189 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:30:53 smithi100 bash[74942]: cluster 2024-04-07T16:30:52.876750+0000 mgr.y (mgr.25175) 466 : cluster [DBG] pgmap v900: 137 pgs: 137 active+clean; 1.4 MiB data, 1.8 GiB used, 713 GiB / 715 GiB avail; 13 MiB/s rd, 4.3 MiB/s wr, 4.85k op/s 2024-04-07T16:30:54.189 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:30:53 smithi100 bash[76202]: cluster 2024-04-07T16:30:52.876750+0000 mgr.y (mgr.25175) 466 : cluster [DBG] pgmap v900: 137 pgs: 137 active+clean; 1.4 MiB data, 1.8 GiB used, 713 GiB / 715 GiB avail; 13 MiB/s rd, 4.3 MiB/s wr, 4.85k op/s 2024-04-07T16:30:54.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:30:53 smithi144 bash[66654]: cluster 2024-04-07T16:30:52.876750+0000 mgr.y (mgr.25175) 466 : cluster [DBG] pgmap v900: 137 pgs: 137 active+clean; 1.4 MiB data, 1.8 GiB used, 713 GiB / 715 GiB avail; 13 MiB/s rd, 4.3 MiB/s wr, 4.85k op/s 2024-04-07T16:30:56.189 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:30:55 smithi100 bash[74942]: cluster 2024-04-07T16:30:54.877803+0000 mgr.y (mgr.25175) 467 : cluster [DBG] pgmap v901: 137 pgs: 137 active+clean; 1.4 MiB data, 1.8 GiB used, 713 GiB / 715 GiB avail; 11 MiB/s rd, 3.5 MiB/s wr, 3.91k op/s 2024-04-07T16:30:56.190 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:30:55 smithi100 bash[76202]: cluster 2024-04-07T16:30:54.877803+0000 mgr.y (mgr.25175) 467 : cluster [DBG] pgmap v901: 137 pgs: 137 active+clean; 1.4 MiB data, 1.8 GiB used, 713 GiB / 715 GiB avail; 11 MiB/s rd, 3.5 MiB/s wr, 3.91k op/s 2024-04-07T16:30:56.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:30:55 smithi144 bash[66654]: cluster 2024-04-07T16:30:54.877803+0000 mgr.y (mgr.25175) 467 : cluster [DBG] pgmap v901: 137 pgs: 137 active+clean; 1.4 MiB data, 1.8 GiB used, 713 GiB / 715 GiB avail; 11 MiB/s rd, 3.5 MiB/s wr, 3.91k op/s 2024-04-07T16:30:58.189 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:30:57 smithi100 bash[74942]: cluster 2024-04-07T16:30:56.878544+0000 mgr.y (mgr.25175) 468 : cluster [DBG] pgmap v902: 137 pgs: 137 active+clean; 1.8 MiB data, 1.8 GiB used, 713 GiB / 715 GiB avail; 19 MiB/s rd, 4.4 MiB/s wr, 5.00k op/s 2024-04-07T16:30:58.189 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:30:57 smithi100 bash[76202]: cluster 2024-04-07T16:30:56.878544+0000 mgr.y (mgr.25175) 468 : cluster [DBG] pgmap v902: 137 pgs: 137 active+clean; 1.8 MiB data, 1.8 GiB used, 713 GiB / 715 GiB avail; 19 MiB/s rd, 4.4 MiB/s wr, 5.00k op/s 2024-04-07T16:30:58.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:30:57 smithi144 bash[66654]: cluster 2024-04-07T16:30:56.878544+0000 mgr.y (mgr.25175) 468 : cluster [DBG] pgmap v902: 137 pgs: 137 active+clean; 1.8 MiB data, 1.8 GiB used, 713 GiB / 715 GiB avail; 19 MiB/s rd, 4.4 MiB/s wr, 5.00k op/s 2024-04-07T16:30:58.689 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:30:58 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:30:58] "GET /metrics HTTP/1.1" 200 36395 "" "Prometheus/2.43.0" 2024-04-07T16:31:00.189 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:30:59 smithi100 bash[74942]: cluster 2024-04-07T16:30:58.879880+0000 mgr.y (mgr.25175) 469 : cluster [DBG] pgmap v903: 137 pgs: 137 active+clean; 1.8 MiB data, 1.9 GiB used, 713 GiB / 715 GiB avail; 18 MiB/s rd, 4.1 MiB/s wr, 4.60k op/s 2024-04-07T16:31:00.189 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:30:59 smithi100 bash[76202]: cluster 2024-04-07T16:30:58.879880+0000 mgr.y (mgr.25175) 469 : cluster [DBG] pgmap v903: 137 pgs: 137 active+clean; 1.8 MiB data, 1.9 GiB used, 713 GiB / 715 GiB avail; 18 MiB/s rd, 4.1 MiB/s wr, 4.60k op/s 2024-04-07T16:31:00.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:30:59 smithi144 bash[66654]: cluster 2024-04-07T16:30:58.879880+0000 mgr.y (mgr.25175) 469 : cluster [DBG] pgmap v903: 137 pgs: 137 active+clean; 1.8 MiB data, 1.9 GiB used, 713 GiB / 715 GiB avail; 18 MiB/s rd, 4.1 MiB/s wr, 4.60k op/s 2024-04-07T16:31:02.305 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:31:01 smithi144 bash[66654]: cluster 2024-04-07T16:31:00.880985+0000 mgr.y (mgr.25175) 470 : cluster [DBG] pgmap v904: 137 pgs: 137 active+clean; 2.1 MiB data, 2.0 GiB used, 713 GiB / 715 GiB avail; 28 MiB/s rd, 5.2 MiB/s wr, 5.85k op/s 2024-04-07T16:31:02.306 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:31:01 smithi144 bash[66654]: audit 2024-04-07T16:31:01.740057+0000 mon.a (mon.0) 1524 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:31:02.439 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:31:01 smithi100 bash[74942]: cluster 2024-04-07T16:31:00.880985+0000 mgr.y (mgr.25175) 470 : cluster [DBG] pgmap v904: 137 pgs: 137 active+clean; 2.1 MiB data, 2.0 GiB used, 713 GiB / 715 GiB avail; 28 MiB/s rd, 5.2 MiB/s wr, 5.85k op/s 2024-04-07T16:31:02.439 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:31:01 smithi100 bash[74942]: audit 2024-04-07T16:31:01.740057+0000 mon.a (mon.0) 1524 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:31:02.440 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:31:01 smithi100 bash[76202]: cluster 2024-04-07T16:31:00.880985+0000 mgr.y (mgr.25175) 470 : cluster [DBG] pgmap v904: 137 pgs: 137 active+clean; 2.1 MiB data, 2.0 GiB used, 713 GiB / 715 GiB avail; 28 MiB/s rd, 5.2 MiB/s wr, 5.85k op/s 2024-04-07T16:31:02.440 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:31:01 smithi100 bash[76202]: audit 2024-04-07T16:31:01.740057+0000 mon.a (mon.0) 1524 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:31:02.607 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:31:02 smithi144 bash[95233]: ts=2024-04-07T16:31:02.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:31:04.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:31:03 smithi144 bash[66654]: cluster 2024-04-07T16:31:02.882206+0000 mgr.y (mgr.25175) 471 : cluster [DBG] pgmap v905: 137 pgs: 137 active+clean; 2.1 MiB data, 2.0 GiB used, 713 GiB / 715 GiB avail; 19 MiB/s rd, 2.4 MiB/s wr, 2.67k op/s 2024-04-07T16:31:04.439 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:31:03 smithi100 bash[74942]: cluster 2024-04-07T16:31:02.882206+0000 mgr.y (mgr.25175) 471 : cluster [DBG] pgmap v905: 137 pgs: 137 active+clean; 2.1 MiB data, 2.0 GiB used, 713 GiB / 715 GiB avail; 19 MiB/s rd, 2.4 MiB/s wr, 2.67k op/s 2024-04-07T16:31:04.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:31:03 smithi100 bash[76202]: cluster 2024-04-07T16:31:02.882206+0000 mgr.y (mgr.25175) 471 : cluster [DBG] pgmap v905: 137 pgs: 137 active+clean; 2.1 MiB data, 2.0 GiB used, 713 GiB / 715 GiB avail; 19 MiB/s rd, 2.4 MiB/s wr, 2.67k op/s 2024-04-07T16:31:06.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:31:05 smithi144 bash[66654]: cluster 2024-04-07T16:31:04.883366+0000 mgr.y (mgr.25175) 472 : cluster [DBG] pgmap v906: 137 pgs: 137 active+clean; 2.1 MiB data, 2.0 GiB used, 713 GiB / 715 GiB avail; 19 MiB/s rd, 2.4 MiB/s wr, 2.67k op/s 2024-04-07T16:31:06.439 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:31:05 smithi100 bash[74942]: cluster 2024-04-07T16:31:04.883366+0000 mgr.y (mgr.25175) 472 : cluster [DBG] pgmap v906: 137 pgs: 137 active+clean; 2.1 MiB data, 2.0 GiB used, 713 GiB / 715 GiB avail; 19 MiB/s rd, 2.4 MiB/s wr, 2.67k op/s 2024-04-07T16:31:06.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:31:05 smithi100 bash[76202]: cluster 2024-04-07T16:31:04.883366+0000 mgr.y (mgr.25175) 472 : cluster [DBG] pgmap v906: 137 pgs: 137 active+clean; 2.1 MiB data, 2.0 GiB used, 713 GiB / 715 GiB avail; 19 MiB/s rd, 2.4 MiB/s wr, 2.67k op/s 2024-04-07T16:31:08.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:31:07 smithi144 bash[66654]: cluster 2024-04-07T16:31:06.884285+0000 mgr.y (mgr.25175) 473 : cluster [DBG] pgmap v907: 137 pgs: 137 active+clean; 2.3 MiB data, 2.0 GiB used, 713 GiB / 715 GiB avail; 49 MiB/s rd, 3.2 MiB/s wr, 3.68k op/s 2024-04-07T16:31:08.418 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:31:07 smithi100 bash[74942]: cluster 2024-04-07T16:31:06.884285+0000 mgr.y (mgr.25175) 473 : cluster [DBG] pgmap v907: 137 pgs: 137 active+clean; 2.3 MiB data, 2.0 GiB used, 713 GiB / 715 GiB avail; 49 MiB/s rd, 3.2 MiB/s wr, 3.68k op/s 2024-04-07T16:31:08.418 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:31:07 smithi100 bash[76202]: cluster 2024-04-07T16:31:06.884285+0000 mgr.y (mgr.25175) 473 : cluster [DBG] pgmap v907: 137 pgs: 137 active+clean; 2.3 MiB data, 2.0 GiB used, 713 GiB / 715 GiB avail; 49 MiB/s rd, 3.2 MiB/s wr, 3.68k op/s 2024-04-07T16:31:08.689 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:31:08 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:31:08] "GET /metrics HTTP/1.1" 200 36412 "" "Prometheus/2.43.0" 2024-04-07T16:31:10.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:31:09 smithi144 bash[66654]: cluster 2024-04-07T16:31:08.885571+0000 mgr.y (mgr.25175) 474 : cluster [DBG] pgmap v908: 137 pgs: 137 active+clean; 2.3 MiB data, 2.0 GiB used, 713 GiB / 715 GiB avail; 40 MiB/s rd, 2.0 MiB/s wr, 2.25k op/s 2024-04-07T16:31:10.439 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:31:09 smithi100 bash[74942]: cluster 2024-04-07T16:31:08.885571+0000 mgr.y (mgr.25175) 474 : cluster [DBG] pgmap v908: 137 pgs: 137 active+clean; 2.3 MiB data, 2.0 GiB used, 713 GiB / 715 GiB avail; 40 MiB/s rd, 2.0 MiB/s wr, 2.25k op/s 2024-04-07T16:31:10.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:31:09 smithi100 bash[76202]: cluster 2024-04-07T16:31:08.885571+0000 mgr.y (mgr.25175) 474 : cluster [DBG] pgmap v908: 137 pgs: 137 active+clean; 2.3 MiB data, 2.0 GiB used, 713 GiB / 715 GiB avail; 40 MiB/s rd, 2.0 MiB/s wr, 2.25k op/s 2024-04-07T16:31:12.305 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:31:11 smithi144 bash[66654]: cluster 2024-04-07T16:31:10.886667+0000 mgr.y (mgr.25175) 475 : cluster [DBG] pgmap v909: 137 pgs: 137 active+clean; 2.3 MiB data, 2.0 GiB used, 713 GiB / 715 GiB avail; 137 MiB/s rd, 2.0 MiB/s wr, 2.53k op/s 2024-04-07T16:31:12.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:31:11 smithi100 bash[76202]: cluster 2024-04-07T16:31:10.886667+0000 mgr.y (mgr.25175) 475 : cluster [DBG] pgmap v909: 137 pgs: 137 active+clean; 2.3 MiB data, 2.0 GiB used, 713 GiB / 715 GiB avail; 137 MiB/s rd, 2.0 MiB/s wr, 2.53k op/s 2024-04-07T16:31:12.439 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:31:11 smithi100 bash[74942]: cluster 2024-04-07T16:31:10.886667+0000 mgr.y (mgr.25175) 475 : cluster [DBG] pgmap v909: 137 pgs: 137 active+clean; 2.3 MiB data, 2.0 GiB used, 713 GiB / 715 GiB avail; 137 MiB/s rd, 2.0 MiB/s wr, 2.53k op/s 2024-04-07T16:31:12.607 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:31:12 smithi144 bash[95233]: ts=2024-04-07T16:31:12.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:31:14.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:31:13 smithi144 bash[66654]: cluster 2024-04-07T16:31:12.887895+0000 mgr.y (mgr.25175) 476 : cluster [DBG] pgmap v910: 137 pgs: 137 active+clean; 2.3 MiB data, 2.0 GiB used, 713 GiB / 715 GiB avail; 127 MiB/s rd, 927 KiB/s wr, 1.28k op/s 2024-04-07T16:31:14.439 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:31:13 smithi100 bash[74942]: cluster 2024-04-07T16:31:12.887895+0000 mgr.y (mgr.25175) 476 : cluster [DBG] pgmap v910: 137 pgs: 137 active+clean; 2.3 MiB data, 2.0 GiB used, 713 GiB / 715 GiB avail; 127 MiB/s rd, 927 KiB/s wr, 1.28k op/s 2024-04-07T16:31:14.440 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:31:13 smithi100 bash[76202]: cluster 2024-04-07T16:31:12.887895+0000 mgr.y (mgr.25175) 476 : cluster [DBG] pgmap v910: 137 pgs: 137 active+clean; 2.3 MiB data, 2.0 GiB used, 713 GiB / 715 GiB avail; 127 MiB/s rd, 927 KiB/s wr, 1.28k op/s 2024-04-07T16:31:15.992 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestCls2PCQueue.MultiProducer (33200 ms) 2024-04-07T16:31:15.992 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestCls2PCQueue.AsyncConsumer 2024-04-07T16:31:16.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:31:15 smithi144 bash[66654]: cluster 2024-04-07T16:31:14.889008+0000 mgr.y (mgr.25175) 477 : cluster [DBG] pgmap v911: 137 pgs: 137 active+clean; 2.3 MiB data, 2.0 GiB used, 713 GiB / 715 GiB avail; 127 MiB/s rd, 927 KiB/s wr, 1.28k op/s 2024-04-07T16:31:16.439 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:31:15 smithi100 bash[74942]: cluster 2024-04-07T16:31:14.889008+0000 mgr.y (mgr.25175) 477 : cluster [DBG] pgmap v911: 137 pgs: 137 active+clean; 2.3 MiB data, 2.0 GiB used, 713 GiB / 715 GiB avail; 127 MiB/s rd, 927 KiB/s wr, 1.28k op/s 2024-04-07T16:31:16.440 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:31:15 smithi100 bash[76202]: cluster 2024-04-07T16:31:14.889008+0000 mgr.y (mgr.25175) 477 : cluster [DBG] pgmap v911: 137 pgs: 137 active+clean; 2.3 MiB data, 2.0 GiB used, 713 GiB / 715 GiB avail; 127 MiB/s rd, 927 KiB/s wr, 1.28k op/s 2024-04-07T16:31:17.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:31:17 smithi144 bash[66654]: cluster 2024-04-07T16:31:15.999488+0000 mon.a (mon.0) 1525 : cluster [DBG] osdmap e860: 8 total, 8 up, 8 in 2024-04-07T16:31:17.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:31:17 smithi144 bash[66654]: audit 2024-04-07T16:31:16.740989+0000 mon.a (mon.0) 1526 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:31:17.439 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:31:17 smithi100 bash[74942]: cluster 2024-04-07T16:31:15.999488+0000 mon.a (mon.0) 1525 : cluster [DBG] osdmap e860: 8 total, 8 up, 8 in 2024-04-07T16:31:17.439 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:31:17 smithi100 bash[74942]: audit 2024-04-07T16:31:16.740989+0000 mon.a (mon.0) 1526 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:31:17.440 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:31:17 smithi100 bash[76202]: cluster 2024-04-07T16:31:15.999488+0000 mon.a (mon.0) 1525 : cluster [DBG] osdmap e860: 8 total, 8 up, 8 in 2024-04-07T16:31:17.440 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:31:17 smithi100 bash[76202]: audit 2024-04-07T16:31:16.740989+0000 mon.a (mon.0) 1526 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:31:18.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:31:18 smithi144 bash[66654]: cluster 2024-04-07T16:31:16.889656+0000 mgr.y (mgr.25175) 478 : cluster [DBG] pgmap v913: 105 pgs: 105 active+clean; 583 KiB data, 2.0 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:31:18.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:31:18 smithi144 bash[66654]: cluster 2024-04-07T16:31:17.014671+0000 mon.a (mon.0) 1527 : cluster [DBG] osdmap e861: 8 total, 8 up, 8 in 2024-04-07T16:31:18.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:31:18 smithi144 bash[66654]: audit 2024-04-07T16:31:17.019261+0000 mon.c (mon.2) 31 : audit [INF] from='client.? 172.21.15.100:0/2432822803' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:31:18.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:31:18 smithi144 bash[66654]: audit 2024-04-07T16:31:17.019992+0000 mon.a (mon.0) 1528 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:31:18.417 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:31:18 smithi100 bash[74942]: cluster 2024-04-07T16:31:16.889656+0000 mgr.y (mgr.25175) 478 : cluster [DBG] pgmap v913: 105 pgs: 105 active+clean; 583 KiB data, 2.0 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:31:18.417 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:31:18 smithi100 bash[74942]: cluster 2024-04-07T16:31:17.014671+0000 mon.a (mon.0) 1527 : cluster [DBG] osdmap e861: 8 total, 8 up, 8 in 2024-04-07T16:31:18.417 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:31:18 smithi100 bash[74942]: audit 2024-04-07T16:31:17.019261+0000 mon.c (mon.2) 31 : audit [INF] from='client.? 172.21.15.100:0/2432822803' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:31:18.417 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:31:18 smithi100 bash[74942]: audit 2024-04-07T16:31:17.019992+0000 mon.a (mon.0) 1528 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:31:18.417 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:31:18 smithi100 bash[76202]: cluster 2024-04-07T16:31:16.889656+0000 mgr.y (mgr.25175) 478 : cluster [DBG] pgmap v913: 105 pgs: 105 active+clean; 583 KiB data, 2.0 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:31:18.417 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:31:18 smithi100 bash[76202]: cluster 2024-04-07T16:31:17.014671+0000 mon.a (mon.0) 1527 : cluster [DBG] osdmap e861: 8 total, 8 up, 8 in 2024-04-07T16:31:18.417 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:31:18 smithi100 bash[76202]: audit 2024-04-07T16:31:17.019261+0000 mon.c (mon.2) 31 : audit [INF] from='client.? 172.21.15.100:0/2432822803' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:31:18.417 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:31:18 smithi100 bash[76202]: audit 2024-04-07T16:31:17.019992+0000 mon.a (mon.0) 1528 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:31:18.689 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:31:18 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:31:18] "GET /metrics HTTP/1.1" 200 33985 "" "Prometheus/2.43.0" 2024-04-07T16:31:19.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:31:19 smithi144 bash[66654]: audit 2024-04-07T16:31:18.017997+0000 mon.a (mon.0) 1529 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:31:19.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:31:19 smithi144 bash[66654]: cluster 2024-04-07T16:31:18.027637+0000 mon.a (mon.0) 1530 : cluster [DBG] osdmap e862: 8 total, 8 up, 8 in 2024-04-07T16:31:19.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:31:19 smithi100 bash[76202]: audit 2024-04-07T16:31:18.017997+0000 mon.a (mon.0) 1529 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:31:19.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:31:19 smithi100 bash[76202]: cluster 2024-04-07T16:31:18.027637+0000 mon.a (mon.0) 1530 : cluster [DBG] osdmap e862: 8 total, 8 up, 8 in 2024-04-07T16:31:19.439 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:31:19 smithi100 bash[74942]: audit 2024-04-07T16:31:18.017997+0000 mon.a (mon.0) 1529 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:31:19.439 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:31:19 smithi100 bash[74942]: cluster 2024-04-07T16:31:18.027637+0000 mon.a (mon.0) 1530 : cluster [DBG] osdmap e862: 8 total, 8 up, 8 in 2024-04-07T16:31:20.050 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestCls2PCQueue.AsyncConsumer (4059 ms) 2024-04-07T16:31:20.051 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestCls2PCQueue.MultiProducerConsumer 2024-04-07T16:31:20.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:31:20 smithi144 bash[66654]: cluster 2024-04-07T16:31:18.890299+0000 mgr.y (mgr.25175) 479 : cluster [DBG] pgmap v916: 137 pgs: 7 creating+activating, 4 creating+peering, 21 unknown, 105 active+clean; 583 KiB data, 2.1 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:31:20.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:31:20 smithi144 bash[66654]: cluster 2024-04-07T16:31:19.031305+0000 mon.a (mon.0) 1531 : cluster [DBG] osdmap e863: 8 total, 8 up, 8 in 2024-04-07T16:31:20.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:31:20 smithi100 bash[76202]: cluster 2024-04-07T16:31:18.890299+0000 mgr.y (mgr.25175) 479 : cluster [DBG] pgmap v916: 137 pgs: 7 creating+activating, 4 creating+peering, 21 unknown, 105 active+clean; 583 KiB data, 2.1 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:31:20.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:31:20 smithi100 bash[76202]: cluster 2024-04-07T16:31:19.031305+0000 mon.a (mon.0) 1531 : cluster [DBG] osdmap e863: 8 total, 8 up, 8 in 2024-04-07T16:31:20.440 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:31:20 smithi100 bash[74942]: cluster 2024-04-07T16:31:18.890299+0000 mgr.y (mgr.25175) 479 : cluster [DBG] pgmap v916: 137 pgs: 7 creating+activating, 4 creating+peering, 21 unknown, 105 active+clean; 583 KiB data, 2.1 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:31:20.440 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:31:20 smithi100 bash[74942]: cluster 2024-04-07T16:31:19.031305+0000 mon.a (mon.0) 1531 : cluster [DBG] osdmap e863: 8 total, 8 up, 8 in 2024-04-07T16:31:21.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:31:21 smithi144 bash[66654]: cluster 2024-04-07T16:31:20.051318+0000 mon.a (mon.0) 1532 : cluster [DBG] osdmap e864: 8 total, 8 up, 8 in 2024-04-07T16:31:21.439 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:31:21 smithi100 bash[74942]: cluster 2024-04-07T16:31:20.051318+0000 mon.a (mon.0) 1532 : cluster [DBG] osdmap e864: 8 total, 8 up, 8 in 2024-04-07T16:31:21.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:31:21 smithi100 bash[76202]: cluster 2024-04-07T16:31:20.051318+0000 mon.a (mon.0) 1532 : cluster [DBG] osdmap e864: 8 total, 8 up, 8 in 2024-04-07T16:31:22.305 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:31:22 smithi144 bash[66654]: cluster 2024-04-07T16:31:20.890918+0000 mgr.y (mgr.25175) 480 : cluster [DBG] pgmap v919: 105 pgs: 105 active+clean; 583 KiB data, 2.1 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:31:22.305 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:31:22 smithi144 bash[66654]: cluster 2024-04-07T16:31:21.057510+0000 mon.a (mon.0) 1533 : cluster [DBG] osdmap e865: 8 total, 8 up, 8 in 2024-04-07T16:31:22.305 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:31:22 smithi144 bash[66654]: audit 2024-04-07T16:31:21.061940+0000 mon.a (mon.0) 1534 : audit [INF] from='client.? 172.21.15.100:0/35976784' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:31:22.439 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:31:22 smithi100 bash[74942]: cluster 2024-04-07T16:31:20.890918+0000 mgr.y (mgr.25175) 480 : cluster [DBG] pgmap v919: 105 pgs: 105 active+clean; 583 KiB data, 2.1 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:31:22.439 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:31:22 smithi100 bash[74942]: cluster 2024-04-07T16:31:21.057510+0000 mon.a (mon.0) 1533 : cluster [DBG] osdmap e865: 8 total, 8 up, 8 in 2024-04-07T16:31:22.439 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:31:22 smithi100 bash[74942]: audit 2024-04-07T16:31:21.061940+0000 mon.a (mon.0) 1534 : audit [INF] from='client.? 172.21.15.100:0/35976784' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:31:22.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:31:22 smithi100 bash[76202]: cluster 2024-04-07T16:31:20.890918+0000 mgr.y (mgr.25175) 480 : cluster [DBG] pgmap v919: 105 pgs: 105 active+clean; 583 KiB data, 2.1 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:31:22.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:31:22 smithi100 bash[76202]: cluster 2024-04-07T16:31:21.057510+0000 mon.a (mon.0) 1533 : cluster [DBG] osdmap e865: 8 total, 8 up, 8 in 2024-04-07T16:31:22.440 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:31:22 smithi100 bash[76202]: audit 2024-04-07T16:31:21.061940+0000 mon.a (mon.0) 1534 : audit [INF] from='client.? 172.21.15.100:0/35976784' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:31:22.607 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:31:22 smithi144 bash[95233]: ts=2024-04-07T16:31:22.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:31:23.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:31:23 smithi144 bash[66654]: audit 2024-04-07T16:31:22.059358+0000 mon.a (mon.0) 1535 : audit [INF] from='client.? 172.21.15.100:0/35976784' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:31:23.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:31:23 smithi144 bash[66654]: cluster 2024-04-07T16:31:22.067116+0000 mon.a (mon.0) 1536 : cluster [DBG] osdmap e866: 8 total, 8 up, 8 in 2024-04-07T16:31:23.439 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:31:23 smithi100 bash[74942]: audit 2024-04-07T16:31:22.059358+0000 mon.a (mon.0) 1535 : audit [INF] from='client.? 172.21.15.100:0/35976784' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:31:23.439 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:31:23 smithi100 bash[74942]: cluster 2024-04-07T16:31:22.067116+0000 mon.a (mon.0) 1536 : cluster [DBG] osdmap e866: 8 total, 8 up, 8 in 2024-04-07T16:31:23.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:31:23 smithi100 bash[76202]: audit 2024-04-07T16:31:22.059358+0000 mon.a (mon.0) 1535 : audit [INF] from='client.? 172.21.15.100:0/35976784' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:31:23.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:31:23 smithi100 bash[76202]: cluster 2024-04-07T16:31:22.067116+0000 mon.a (mon.0) 1536 : cluster [DBG] osdmap e866: 8 total, 8 up, 8 in 2024-04-07T16:31:24.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:31:24 smithi144 bash[66654]: cluster 2024-04-07T16:31:22.891400+0000 mgr.y (mgr.25175) 481 : cluster [DBG] pgmap v922: 137 pgs: 14 creating+peering, 18 unknown, 105 active+clean; 583 KiB data, 2.1 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:31:24.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:31:24 smithi144 bash[66654]: cluster 2024-04-07T16:31:23.070511+0000 mon.a (mon.0) 1537 : cluster [DBG] osdmap e867: 8 total, 8 up, 8 in 2024-04-07T16:31:24.439 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:31:24 smithi100 bash[74942]: cluster 2024-04-07T16:31:22.891400+0000 mgr.y (mgr.25175) 481 : cluster [DBG] pgmap v922: 137 pgs: 14 creating+peering, 18 unknown, 105 active+clean; 583 KiB data, 2.1 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:31:24.439 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:31:24 smithi100 bash[74942]: cluster 2024-04-07T16:31:23.070511+0000 mon.a (mon.0) 1537 : cluster [DBG] osdmap e867: 8 total, 8 up, 8 in 2024-04-07T16:31:24.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:31:24 smithi100 bash[76202]: cluster 2024-04-07T16:31:22.891400+0000 mgr.y (mgr.25175) 481 : cluster [DBG] pgmap v922: 137 pgs: 14 creating+peering, 18 unknown, 105 active+clean; 583 KiB data, 2.1 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:31:24.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:31:24 smithi100 bash[76202]: cluster 2024-04-07T16:31:23.070511+0000 mon.a (mon.0) 1537 : cluster [DBG] osdmap e867: 8 total, 8 up, 8 in 2024-04-07T16:31:26.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:31:26 smithi144 bash[66654]: cluster 2024-04-07T16:31:24.891883+0000 mgr.y (mgr.25175) 482 : cluster [DBG] pgmap v924: 137 pgs: 14 creating+peering, 5 unknown, 118 active+clean; 583 KiB data, 2.1 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:31:26.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:31:26 smithi100 bash[76202]: cluster 2024-04-07T16:31:24.891883+0000 mgr.y (mgr.25175) 482 : cluster [DBG] pgmap v924: 137 pgs: 14 creating+peering, 5 unknown, 118 active+clean; 583 KiB data, 2.1 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:31:26.439 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:31:26 smithi100 bash[74942]: cluster 2024-04-07T16:31:24.891883+0000 mgr.y (mgr.25175) 482 : cluster [DBG] pgmap v924: 137 pgs: 14 creating+peering, 5 unknown, 118 active+clean; 583 KiB data, 2.1 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:31:28.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:31:28 smithi144 bash[66654]: cluster 2024-04-07T16:31:26.892368+0000 mgr.y (mgr.25175) 483 : cluster [DBG] pgmap v925: 137 pgs: 14 creating+peering, 123 active+clean; 583 KiB data, 2.1 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:31:28.417 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:31:28 smithi100 bash[74942]: cluster 2024-04-07T16:31:26.892368+0000 mgr.y (mgr.25175) 483 : cluster [DBG] pgmap v925: 137 pgs: 14 creating+peering, 123 active+clean; 583 KiB data, 2.1 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:31:28.417 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:31:28 smithi100 bash[76202]: cluster 2024-04-07T16:31:26.892368+0000 mgr.y (mgr.25175) 483 : cluster [DBG] pgmap v925: 137 pgs: 14 creating+peering, 123 active+clean; 583 KiB data, 2.1 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:31:28.689 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:31:28 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:31:28] "GET /metrics HTTP/1.1" 200 33985 "" "Prometheus/2.43.0" 2024-04-07T16:31:30.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:31:30 smithi144 bash[66654]: cluster 2024-04-07T16:31:28.893158+0000 mgr.y (mgr.25175) 484 : cluster [DBG] pgmap v926: 137 pgs: 137 active+clean; 1.3 MiB data, 2.2 GiB used, 713 GiB / 715 GiB avail; 28 MiB/s rd, 4.0 MiB/s wr, 4.79k op/s 2024-04-07T16:31:30.439 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:31:30 smithi100 bash[74942]: cluster 2024-04-07T16:31:28.893158+0000 mgr.y (mgr.25175) 484 : cluster [DBG] pgmap v926: 137 pgs: 137 active+clean; 1.3 MiB data, 2.2 GiB used, 713 GiB / 715 GiB avail; 28 MiB/s rd, 4.0 MiB/s wr, 4.79k op/s 2024-04-07T16:31:30.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:31:30 smithi100 bash[76202]: cluster 2024-04-07T16:31:28.893158+0000 mgr.y (mgr.25175) 484 : cluster [DBG] pgmap v926: 137 pgs: 137 active+clean; 1.3 MiB data, 2.2 GiB used, 713 GiB / 715 GiB avail; 28 MiB/s rd, 4.0 MiB/s wr, 4.79k op/s 2024-04-07T16:31:32.357 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:31:32 smithi144 bash[95233]: ts=2024-04-07T16:31:32.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:31:32.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:31:32 smithi144 bash[66654]: cluster 2024-04-07T16:31:30.893735+0000 mgr.y (mgr.25175) 485 : cluster [DBG] pgmap v927: 137 pgs: 137 active+clean; 1.3 MiB data, 2.2 GiB used, 713 GiB / 715 GiB avail; 25 MiB/s rd, 3.6 MiB/s wr, 4.25k op/s 2024-04-07T16:31:32.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:31:32 smithi144 bash[66654]: audit 2024-04-07T16:31:31.741044+0000 mon.a (mon.0) 1538 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:31:32.439 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:31:32 smithi100 bash[74942]: cluster 2024-04-07T16:31:30.893735+0000 mgr.y (mgr.25175) 485 : cluster [DBG] pgmap v927: 137 pgs: 137 active+clean; 1.3 MiB data, 2.2 GiB used, 713 GiB / 715 GiB avail; 25 MiB/s rd, 3.6 MiB/s wr, 4.25k op/s 2024-04-07T16:31:32.439 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:31:32 smithi100 bash[74942]: audit 2024-04-07T16:31:31.741044+0000 mon.a (mon.0) 1538 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:31:32.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:31:32 smithi100 bash[76202]: cluster 2024-04-07T16:31:30.893735+0000 mgr.y (mgr.25175) 485 : cluster [DBG] pgmap v927: 137 pgs: 137 active+clean; 1.3 MiB data, 2.2 GiB used, 713 GiB / 715 GiB avail; 25 MiB/s rd, 3.6 MiB/s wr, 4.25k op/s 2024-04-07T16:31:32.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:31:32 smithi100 bash[76202]: audit 2024-04-07T16:31:31.741044+0000 mon.a (mon.0) 1538 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:31:34.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:31:34 smithi144 bash[66654]: cluster 2024-04-07T16:31:32.894651+0000 mgr.y (mgr.25175) 486 : cluster [DBG] pgmap v928: 137 pgs: 137 active+clean; 1.6 MiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 53 MiB/s rd, 5.0 MiB/s wr, 5.98k op/s 2024-04-07T16:31:34.439 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:31:34 smithi100 bash[74942]: cluster 2024-04-07T16:31:32.894651+0000 mgr.y (mgr.25175) 486 : cluster [DBG] pgmap v928: 137 pgs: 137 active+clean; 1.6 MiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 53 MiB/s rd, 5.0 MiB/s wr, 5.98k op/s 2024-04-07T16:31:34.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:31:34 smithi100 bash[76202]: cluster 2024-04-07T16:31:32.894651+0000 mgr.y (mgr.25175) 486 : cluster [DBG] pgmap v928: 137 pgs: 137 active+clean; 1.6 MiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 53 MiB/s rd, 5.0 MiB/s wr, 5.98k op/s 2024-04-07T16:31:35.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:31:35 smithi144 bash[66654]: audit 2024-04-07T16:31:34.748183+0000 mon.a (mon.0) 1539 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:31:35.441 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:31:35 smithi100 bash[74942]: audit 2024-04-07T16:31:34.748183+0000 mon.a (mon.0) 1539 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:31:35.442 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:31:35 smithi100 bash[76202]: audit 2024-04-07T16:31:34.748183+0000 mon.a (mon.0) 1539 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:31:36.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:31:36 smithi144 bash[66654]: cluster 2024-04-07T16:31:34.895225+0000 mgr.y (mgr.25175) 487 : cluster [DBG] pgmap v929: 137 pgs: 137 active+clean; 1.6 MiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 44 MiB/s rd, 4.3 MiB/s wr, 5.06k op/s 2024-04-07T16:31:36.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:31:36 smithi144 bash[66654]: audit 2024-04-07T16:31:35.482028+0000 mon.a (mon.0) 1540 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:31:36.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:31:36 smithi144 bash[66654]: audit 2024-04-07T16:31:35.482749+0000 mon.a (mon.0) 1541 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:31:36.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:31:36 smithi144 bash[66654]: audit 2024-04-07T16:31:35.487748+0000 mon.a (mon.0) 1542 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:31:36.439 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:31:36 smithi100 bash[74942]: cluster 2024-04-07T16:31:34.895225+0000 mgr.y (mgr.25175) 487 : cluster [DBG] pgmap v929: 137 pgs: 137 active+clean; 1.6 MiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 44 MiB/s rd, 4.3 MiB/s wr, 5.06k op/s 2024-04-07T16:31:36.439 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:31:36 smithi100 bash[74942]: audit 2024-04-07T16:31:35.482028+0000 mon.a (mon.0) 1540 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:31:36.439 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:31:36 smithi100 bash[74942]: audit 2024-04-07T16:31:35.482749+0000 mon.a (mon.0) 1541 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:31:36.439 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:31:36 smithi100 bash[74942]: audit 2024-04-07T16:31:35.487748+0000 mon.a (mon.0) 1542 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:31:36.440 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:31:36 smithi100 bash[76202]: cluster 2024-04-07T16:31:34.895225+0000 mgr.y (mgr.25175) 487 : cluster [DBG] pgmap v929: 137 pgs: 137 active+clean; 1.6 MiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 44 MiB/s rd, 4.3 MiB/s wr, 5.06k op/s 2024-04-07T16:31:36.440 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:31:36 smithi100 bash[76202]: audit 2024-04-07T16:31:35.482028+0000 mon.a (mon.0) 1540 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:31:36.440 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:31:36 smithi100 bash[76202]: audit 2024-04-07T16:31:35.482749+0000 mon.a (mon.0) 1541 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:31:36.440 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:31:36 smithi100 bash[76202]: audit 2024-04-07T16:31:35.487748+0000 mon.a (mon.0) 1542 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:31:38.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:31:38 smithi144 bash[66654]: cluster 2024-04-07T16:31:36.895694+0000 mgr.y (mgr.25175) 488 : cluster [DBG] pgmap v930: 137 pgs: 137 active+clean; 1.6 MiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 44 MiB/s rd, 4.2 MiB/s wr, 4.98k op/s 2024-04-07T16:31:38.417 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:31:38 smithi100 bash[74942]: cluster 2024-04-07T16:31:36.895694+0000 mgr.y (mgr.25175) 488 : cluster [DBG] pgmap v930: 137 pgs: 137 active+clean; 1.6 MiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 44 MiB/s rd, 4.2 MiB/s wr, 4.98k op/s 2024-04-07T16:31:38.417 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:31:38 smithi100 bash[76202]: cluster 2024-04-07T16:31:36.895694+0000 mgr.y (mgr.25175) 488 : cluster [DBG] pgmap v930: 137 pgs: 137 active+clean; 1.6 MiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 44 MiB/s rd, 4.2 MiB/s wr, 4.98k op/s 2024-04-07T16:31:38.689 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:31:38 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:31:38] "GET /metrics HTTP/1.1" 200 36418 "" "Prometheus/2.43.0" 2024-04-07T16:31:40.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:31:40 smithi144 bash[66654]: cluster 2024-04-07T16:31:38.896331+0000 mgr.y (mgr.25175) 489 : cluster [DBG] pgmap v931: 137 pgs: 137 active+clean; 1.6 MiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 60 MiB/s rd, 4.9 MiB/s wr, 5.92k op/s 2024-04-07T16:31:40.439 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:31:40 smithi100 bash[74942]: cluster 2024-04-07T16:31:38.896331+0000 mgr.y (mgr.25175) 489 : cluster [DBG] pgmap v931: 137 pgs: 137 active+clean; 1.6 MiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 60 MiB/s rd, 4.9 MiB/s wr, 5.92k op/s 2024-04-07T16:31:40.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:31:40 smithi100 bash[76202]: cluster 2024-04-07T16:31:38.896331+0000 mgr.y (mgr.25175) 489 : cluster [DBG] pgmap v931: 137 pgs: 137 active+clean; 1.6 MiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 60 MiB/s rd, 4.9 MiB/s wr, 5.92k op/s 2024-04-07T16:31:42.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:31:42 smithi144 bash[66654]: cluster 2024-04-07T16:31:40.896945+0000 mgr.y (mgr.25175) 490 : cluster [DBG] pgmap v932: 137 pgs: 137 active+clean; 1.6 MiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 42 MiB/s rd, 2.3 MiB/s wr, 2.79k op/s 2024-04-07T16:31:42.357 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:31:42 smithi144 bash[95233]: ts=2024-04-07T16:31:42.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:31:42.439 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:31:42 smithi100 bash[74942]: cluster 2024-04-07T16:31:40.896945+0000 mgr.y (mgr.25175) 490 : cluster [DBG] pgmap v932: 137 pgs: 137 active+clean; 1.6 MiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 42 MiB/s rd, 2.3 MiB/s wr, 2.79k op/s 2024-04-07T16:31:42.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:31:42 smithi100 bash[76202]: cluster 2024-04-07T16:31:40.896945+0000 mgr.y (mgr.25175) 490 : cluster [DBG] pgmap v932: 137 pgs: 137 active+clean; 1.6 MiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 42 MiB/s rd, 2.3 MiB/s wr, 2.79k op/s 2024-04-07T16:31:44.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:31:44 smithi144 bash[66654]: cluster 2024-04-07T16:31:42.897526+0000 mgr.y (mgr.25175) 491 : cluster [DBG] pgmap v933: 137 pgs: 137 active+clean; 1.6 MiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 59 MiB/s rd, 3.0 MiB/s wr, 3.68k op/s 2024-04-07T16:31:44.439 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:31:44 smithi100 bash[74942]: cluster 2024-04-07T16:31:42.897526+0000 mgr.y (mgr.25175) 491 : cluster [DBG] pgmap v933: 137 pgs: 137 active+clean; 1.6 MiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 59 MiB/s rd, 3.0 MiB/s wr, 3.68k op/s 2024-04-07T16:31:44.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:31:44 smithi100 bash[76202]: cluster 2024-04-07T16:31:42.897526+0000 mgr.y (mgr.25175) 491 : cluster [DBG] pgmap v933: 137 pgs: 137 active+clean; 1.6 MiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 59 MiB/s rd, 3.0 MiB/s wr, 3.68k op/s 2024-04-07T16:31:46.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:31:46 smithi144 bash[66654]: cluster 2024-04-07T16:31:44.898112+0000 mgr.y (mgr.25175) 492 : cluster [DBG] pgmap v934: 137 pgs: 137 active+clean; 1.6 MiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 33 MiB/s rd, 1.4 MiB/s wr, 1.82k op/s 2024-04-07T16:31:46.439 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:31:46 smithi100 bash[74942]: cluster 2024-04-07T16:31:44.898112+0000 mgr.y (mgr.25175) 492 : cluster [DBG] pgmap v934: 137 pgs: 137 active+clean; 1.6 MiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 33 MiB/s rd, 1.4 MiB/s wr, 1.82k op/s 2024-04-07T16:31:46.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:31:46 smithi100 bash[76202]: cluster 2024-04-07T16:31:44.898112+0000 mgr.y (mgr.25175) 492 : cluster [DBG] pgmap v934: 137 pgs: 137 active+clean; 1.6 MiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 33 MiB/s rd, 1.4 MiB/s wr, 1.82k op/s 2024-04-07T16:31:47.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:31:47 smithi144 bash[66654]: audit 2024-04-07T16:31:46.741449+0000 mon.a (mon.0) 1543 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:31:47.439 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:31:47 smithi100 bash[74942]: audit 2024-04-07T16:31:46.741449+0000 mon.a (mon.0) 1543 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:31:47.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:31:47 smithi100 bash[76202]: audit 2024-04-07T16:31:46.741449+0000 mon.a (mon.0) 1543 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:31:48.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:31:48 smithi144 bash[66654]: cluster 2024-04-07T16:31:46.898583+0000 mgr.y (mgr.25175) 493 : cluster [DBG] pgmap v935: 137 pgs: 137 active+clean; 1.6 MiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 33 MiB/s rd, 1.4 MiB/s wr, 1.82k op/s 2024-04-07T16:31:48.416 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:31:48 smithi100 bash[74942]: cluster 2024-04-07T16:31:46.898583+0000 mgr.y (mgr.25175) 493 : cluster [DBG] pgmap v935: 137 pgs: 137 active+clean; 1.6 MiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 33 MiB/s rd, 1.4 MiB/s wr, 1.82k op/s 2024-04-07T16:31:48.417 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:31:48 smithi100 bash[76202]: cluster 2024-04-07T16:31:46.898583+0000 mgr.y (mgr.25175) 493 : cluster [DBG] pgmap v935: 137 pgs: 137 active+clean; 1.6 MiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 33 MiB/s rd, 1.4 MiB/s wr, 1.82k op/s 2024-04-07T16:31:48.689 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:31:48 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:31:48] "GET /metrics HTTP/1.1" 200 36417 "" "Prometheus/2.43.0" 2024-04-07T16:31:50.125 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestCls2PCQueue.MultiProducerConsumer (30069 ms) 2024-04-07T16:31:50.126 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestCls2PCQueue.ReserveSpillover 2024-04-07T16:31:50.439 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:31:50 smithi100 bash[74942]: cluster 2024-04-07T16:31:48.899561+0000 mgr.y (mgr.25175) 494 : cluster [DBG] pgmap v936: 137 pgs: 137 active+clean; 1.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 59 MiB/s rd, 2.1 MiB/s wr, 2.71k op/s 2024-04-07T16:31:50.440 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:31:50 smithi100 bash[76202]: cluster 2024-04-07T16:31:48.899561+0000 mgr.y (mgr.25175) 494 : cluster [DBG] pgmap v936: 137 pgs: 137 active+clean; 1.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 59 MiB/s rd, 2.1 MiB/s wr, 2.71k op/s 2024-04-07T16:31:50.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:31:50 smithi144 bash[66654]: cluster 2024-04-07T16:31:48.899561+0000 mgr.y (mgr.25175) 494 : cluster [DBG] pgmap v936: 137 pgs: 137 active+clean; 1.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 59 MiB/s rd, 2.1 MiB/s wr, 2.71k op/s 2024-04-07T16:31:51.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:31:51 smithi100 bash[76202]: cluster 2024-04-07T16:31:50.127709+0000 mon.a (mon.0) 1544 : cluster [DBG] osdmap e868: 8 total, 8 up, 8 in 2024-04-07T16:31:51.439 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:31:51 smithi100 bash[74942]: cluster 2024-04-07T16:31:50.127709+0000 mon.a (mon.0) 1544 : cluster [DBG] osdmap e868: 8 total, 8 up, 8 in 2024-04-07T16:31:51.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:31:51 smithi144 bash[66654]: cluster 2024-04-07T16:31:50.127709+0000 mon.a (mon.0) 1544 : cluster [DBG] osdmap e868: 8 total, 8 up, 8 in 2024-04-07T16:31:52.439 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:31:52 smithi100 bash[74942]: cluster 2024-04-07T16:31:50.900173+0000 mgr.y (mgr.25175) 495 : cluster [DBG] pgmap v938: 105 pgs: 105 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:31:52.439 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:31:52 smithi100 bash[74942]: cluster 2024-04-07T16:31:51.152790+0000 mon.a (mon.0) 1545 : cluster [DBG] osdmap e869: 8 total, 8 up, 8 in 2024-04-07T16:31:52.439 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:31:52 smithi100 bash[74942]: audit 2024-04-07T16:31:51.154643+0000 mon.a (mon.0) 1546 : audit [INF] from='client.? 172.21.15.100:0/3132644671' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:31:52.440 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:31:52 smithi100 bash[76202]: cluster 2024-04-07T16:31:50.900173+0000 mgr.y (mgr.25175) 495 : cluster [DBG] pgmap v938: 105 pgs: 105 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:31:52.440 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:31:52 smithi100 bash[76202]: cluster 2024-04-07T16:31:51.152790+0000 mon.a (mon.0) 1545 : cluster [DBG] osdmap e869: 8 total, 8 up, 8 in 2024-04-07T16:31:52.440 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:31:52 smithi100 bash[76202]: audit 2024-04-07T16:31:51.154643+0000 mon.a (mon.0) 1546 : audit [INF] from='client.? 172.21.15.100:0/3132644671' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:31:52.607 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:31:52 smithi144 bash[95233]: ts=2024-04-07T16:31:52.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:31:52.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:31:52 smithi144 bash[66654]: cluster 2024-04-07T16:31:50.900173+0000 mgr.y (mgr.25175) 495 : cluster [DBG] pgmap v938: 105 pgs: 105 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:31:52.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:31:52 smithi144 bash[66654]: cluster 2024-04-07T16:31:51.152790+0000 mon.a (mon.0) 1545 : cluster [DBG] osdmap e869: 8 total, 8 up, 8 in 2024-04-07T16:31:52.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:31:52 smithi144 bash[66654]: audit 2024-04-07T16:31:51.154643+0000 mon.a (mon.0) 1546 : audit [INF] from='client.? 172.21.15.100:0/3132644671' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:31:53.439 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:31:53 smithi100 bash[74942]: audit 2024-04-07T16:31:52.155747+0000 mon.a (mon.0) 1547 : audit [INF] from='client.? 172.21.15.100:0/3132644671' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:31:53.439 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:31:53 smithi100 bash[74942]: cluster 2024-04-07T16:31:52.163571+0000 mon.a (mon.0) 1548 : cluster [DBG] osdmap e870: 8 total, 8 up, 8 in 2024-04-07T16:31:53.440 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:31:53 smithi100 bash[76202]: audit 2024-04-07T16:31:52.155747+0000 mon.a (mon.0) 1547 : audit [INF] from='client.? 172.21.15.100:0/3132644671' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:31:53.440 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:31:53 smithi100 bash[76202]: cluster 2024-04-07T16:31:52.163571+0000 mon.a (mon.0) 1548 : cluster [DBG] osdmap e870: 8 total, 8 up, 8 in 2024-04-07T16:31:53.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:31:53 smithi144 bash[66654]: audit 2024-04-07T16:31:52.155747+0000 mon.a (mon.0) 1547 : audit [INF] from='client.? 172.21.15.100:0/3132644671' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:31:53.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:31:53 smithi144 bash[66654]: cluster 2024-04-07T16:31:52.163571+0000 mon.a (mon.0) 1548 : cluster [DBG] osdmap e870: 8 total, 8 up, 8 in 2024-04-07T16:31:54.440 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:31:54 smithi100 bash[74942]: cluster 2024-04-07T16:31:52.900993+0000 mgr.y (mgr.25175) 496 : cluster [DBG] pgmap v941: 137 pgs: 15 creating+peering, 17 unknown, 105 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:31:54.440 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:31:54 smithi100 bash[74942]: cluster 2024-04-07T16:31:53.178545+0000 mon.a (mon.0) 1549 : cluster [DBG] osdmap e871: 8 total, 8 up, 8 in 2024-04-07T16:31:54.440 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:31:54 smithi100 bash[76202]: cluster 2024-04-07T16:31:52.900993+0000 mgr.y (mgr.25175) 496 : cluster [DBG] pgmap v941: 137 pgs: 15 creating+peering, 17 unknown, 105 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:31:54.440 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:31:54 smithi100 bash[76202]: cluster 2024-04-07T16:31:53.178545+0000 mon.a (mon.0) 1549 : cluster [DBG] osdmap e871: 8 total, 8 up, 8 in 2024-04-07T16:31:54.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:31:54 smithi144 bash[66654]: cluster 2024-04-07T16:31:52.900993+0000 mgr.y (mgr.25175) 496 : cluster [DBG] pgmap v941: 137 pgs: 15 creating+peering, 17 unknown, 105 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:31:54.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:31:54 smithi144 bash[66654]: cluster 2024-04-07T16:31:53.178545+0000 mon.a (mon.0) 1549 : cluster [DBG] osdmap e871: 8 total, 8 up, 8 in 2024-04-07T16:31:56.183 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestCls2PCQueue.ReserveSpillover (6064 ms) 2024-04-07T16:31:56.184 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestCls2PCQueue.CommitSpillover 2024-04-07T16:31:56.439 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:31:56 smithi100 bash[74942]: cluster 2024-04-07T16:31:54.901999+0000 mgr.y (mgr.25175) 497 : cluster [DBG] pgmap v943: 137 pgs: 15 creating+peering, 2 unknown, 120 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:31:56.440 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:31:56 smithi100 bash[76202]: cluster 2024-04-07T16:31:54.901999+0000 mgr.y (mgr.25175) 497 : cluster [DBG] pgmap v943: 137 pgs: 15 creating+peering, 2 unknown, 120 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:31:56.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:31:56 smithi144 bash[66654]: cluster 2024-04-07T16:31:54.901999+0000 mgr.y (mgr.25175) 497 : cluster [DBG] pgmap v943: 137 pgs: 15 creating+peering, 2 unknown, 120 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:31:57.439 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:31:57 smithi100 bash[74942]: cluster 2024-04-07T16:31:56.192250+0000 mon.a (mon.0) 1550 : cluster [DBG] osdmap e872: 8 total, 8 up, 8 in 2024-04-07T16:31:57.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:31:57 smithi100 bash[76202]: cluster 2024-04-07T16:31:56.192250+0000 mon.a (mon.0) 1550 : cluster [DBG] osdmap e872: 8 total, 8 up, 8 in 2024-04-07T16:31:57.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:31:57 smithi144 bash[66654]: cluster 2024-04-07T16:31:56.192250+0000 mon.a (mon.0) 1550 : cluster [DBG] osdmap e872: 8 total, 8 up, 8 in 2024-04-07T16:31:58.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:31:58 smithi144 bash[66654]: cluster 2024-04-07T16:31:56.902724+0000 mgr.y (mgr.25175) 498 : cluster [DBG] pgmap v945: 105 pgs: 105 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:31:58.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:31:58 smithi144 bash[66654]: cluster 2024-04-07T16:31:57.203086+0000 mon.a (mon.0) 1551 : cluster [DBG] osdmap e873: 8 total, 8 up, 8 in 2024-04-07T16:31:58.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:31:58 smithi144 bash[66654]: audit 2024-04-07T16:31:57.208676+0000 mon.c (mon.2) 32 : audit [INF] from='client.? 172.21.15.100:0/1940107960' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:31:58.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:31:58 smithi144 bash[66654]: audit 2024-04-07T16:31:57.211496+0000 mon.a (mon.0) 1552 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:31:58.689 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:31:58 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:31:58] "GET /metrics HTTP/1.1" 200 36417 "" "Prometheus/2.43.0" 2024-04-07T16:31:58.690 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:31:58 smithi100 bash[74942]: cluster 2024-04-07T16:31:56.902724+0000 mgr.y (mgr.25175) 498 : cluster [DBG] pgmap v945: 105 pgs: 105 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:31:58.690 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:31:58 smithi100 bash[74942]: cluster 2024-04-07T16:31:57.203086+0000 mon.a (mon.0) 1551 : cluster [DBG] osdmap e873: 8 total, 8 up, 8 in 2024-04-07T16:31:58.690 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:31:58 smithi100 bash[74942]: audit 2024-04-07T16:31:57.208676+0000 mon.c (mon.2) 32 : audit [INF] from='client.? 172.21.15.100:0/1940107960' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:31:58.690 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:31:58 smithi100 bash[74942]: audit 2024-04-07T16:31:57.211496+0000 mon.a (mon.0) 1552 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:31:58.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:31:58 smithi100 bash[76202]: cluster 2024-04-07T16:31:56.902724+0000 mgr.y (mgr.25175) 498 : cluster [DBG] pgmap v945: 105 pgs: 105 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:31:58.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:31:58 smithi100 bash[76202]: cluster 2024-04-07T16:31:57.203086+0000 mon.a (mon.0) 1551 : cluster [DBG] osdmap e873: 8 total, 8 up, 8 in 2024-04-07T16:31:58.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:31:58 smithi100 bash[76202]: audit 2024-04-07T16:31:57.208676+0000 mon.c (mon.2) 32 : audit [INF] from='client.? 172.21.15.100:0/1940107960' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:31:58.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:31:58 smithi100 bash[76202]: audit 2024-04-07T16:31:57.211496+0000 mon.a (mon.0) 1552 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:31:59.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:31:59 smithi144 bash[66654]: audit 2024-04-07T16:31:58.198041+0000 mon.a (mon.0) 1553 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:31:59.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:31:59 smithi144 bash[66654]: cluster 2024-04-07T16:31:58.208920+0000 mon.a (mon.0) 1554 : cluster [DBG] osdmap e874: 8 total, 8 up, 8 in 2024-04-07T16:31:59.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:31:59 smithi144 bash[66654]: cluster 2024-04-07T16:31:59.208450+0000 mon.a (mon.0) 1555 : cluster [DBG] osdmap e875: 8 total, 8 up, 8 in 2024-04-07T16:31:59.689 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:31:59 smithi100 bash[74942]: audit 2024-04-07T16:31:58.198041+0000 mon.a (mon.0) 1553 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:31:59.689 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:31:59 smithi100 bash[74942]: cluster 2024-04-07T16:31:58.208920+0000 mon.a (mon.0) 1554 : cluster [DBG] osdmap e874: 8 total, 8 up, 8 in 2024-04-07T16:31:59.690 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:31:59 smithi100 bash[74942]: cluster 2024-04-07T16:31:59.208450+0000 mon.a (mon.0) 1555 : cluster [DBG] osdmap e875: 8 total, 8 up, 8 in 2024-04-07T16:31:59.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:31:59 smithi100 bash[76202]: audit 2024-04-07T16:31:58.198041+0000 mon.a (mon.0) 1553 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:31:59.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:31:59 smithi100 bash[76202]: cluster 2024-04-07T16:31:58.208920+0000 mon.a (mon.0) 1554 : cluster [DBG] osdmap e874: 8 total, 8 up, 8 in 2024-04-07T16:31:59.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:31:59 smithi100 bash[76202]: cluster 2024-04-07T16:31:59.208450+0000 mon.a (mon.0) 1555 : cluster [DBG] osdmap e875: 8 total, 8 up, 8 in 2024-04-07T16:32:00.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:32:00 smithi144 bash[66654]: cluster 2024-04-07T16:31:58.903397+0000 mgr.y (mgr.25175) 499 : cluster [DBG] pgmap v948: 137 pgs: 2 creating+peering, 30 unknown, 105 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:32:00.689 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:32:00 smithi100 bash[74942]: cluster 2024-04-07T16:31:58.903397+0000 mgr.y (mgr.25175) 499 : cluster [DBG] pgmap v948: 137 pgs: 2 creating+peering, 30 unknown, 105 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:32:00.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:32:00 smithi100 bash[76202]: cluster 2024-04-07T16:31:58.903397+0000 mgr.y (mgr.25175) 499 : cluster [DBG] pgmap v948: 137 pgs: 2 creating+peering, 30 unknown, 105 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:32:02.257 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestCls2PCQueue.CommitSpillover (6074 ms) 2024-04-07T16:32:02.258 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestCls2PCQueue.AbortSpillover 2024-04-07T16:32:02.607 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:32:02 smithi144 bash[95233]: ts=2024-04-07T16:32:02.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:32:02.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:32:02 smithi144 bash[66654]: cluster 2024-04-07T16:32:00.904260+0000 mgr.y (mgr.25175) 500 : cluster [DBG] pgmap v950: 137 pgs: 2 creating+peering, 16 unknown, 119 active+clean; 599 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 325 op/s 2024-04-07T16:32:02.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:32:02 smithi144 bash[66654]: audit 2024-04-07T16:32:01.742630+0000 mon.a (mon.0) 1556 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:32:02.689 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:32:02 smithi100 bash[74942]: cluster 2024-04-07T16:32:00.904260+0000 mgr.y (mgr.25175) 500 : cluster [DBG] pgmap v950: 137 pgs: 2 creating+peering, 16 unknown, 119 active+clean; 599 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 325 op/s 2024-04-07T16:32:02.689 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:32:02 smithi100 bash[74942]: audit 2024-04-07T16:32:01.742630+0000 mon.a (mon.0) 1556 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:32:02.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:32:02 smithi100 bash[76202]: cluster 2024-04-07T16:32:00.904260+0000 mgr.y (mgr.25175) 500 : cluster [DBG] pgmap v950: 137 pgs: 2 creating+peering, 16 unknown, 119 active+clean; 599 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 325 op/s 2024-04-07T16:32:02.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:32:02 smithi100 bash[76202]: audit 2024-04-07T16:32:01.742630+0000 mon.a (mon.0) 1556 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:32:03.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:32:03 smithi144 bash[66654]: cluster 2024-04-07T16:32:02.260184+0000 mon.a (mon.0) 1557 : cluster [DBG] osdmap e876: 8 total, 8 up, 8 in 2024-04-07T16:32:03.689 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:32:03 smithi100 bash[74942]: cluster 2024-04-07T16:32:02.260184+0000 mon.a (mon.0) 1557 : cluster [DBG] osdmap e876: 8 total, 8 up, 8 in 2024-04-07T16:32:03.689 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:32:03 smithi100 bash[76202]: cluster 2024-04-07T16:32:02.260184+0000 mon.a (mon.0) 1557 : cluster [DBG] osdmap e876: 8 total, 8 up, 8 in 2024-04-07T16:32:04.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:32:04 smithi144 bash[66654]: cluster 2024-04-07T16:32:02.904948+0000 mgr.y (mgr.25175) 501 : cluster [DBG] pgmap v952: 105 pgs: 105 active+clean; 583 KiB data, 2.5 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:32:04.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:32:04 smithi144 bash[66654]: cluster 2024-04-07T16:32:03.291004+0000 mon.a (mon.0) 1558 : cluster [DBG] osdmap e877: 8 total, 8 up, 8 in 2024-04-07T16:32:04.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:32:04 smithi144 bash[66654]: audit 2024-04-07T16:32:03.293464+0000 mon.a (mon.0) 1559 : audit [INF] from='client.? 172.21.15.100:0/1873320617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:32:04.689 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:32:04 smithi100 bash[76202]: cluster 2024-04-07T16:32:02.904948+0000 mgr.y (mgr.25175) 501 : cluster [DBG] pgmap v952: 105 pgs: 105 active+clean; 583 KiB data, 2.5 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:32:04.689 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:32:04 smithi100 bash[76202]: cluster 2024-04-07T16:32:03.291004+0000 mon.a (mon.0) 1558 : cluster [DBG] osdmap e877: 8 total, 8 up, 8 in 2024-04-07T16:32:04.689 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:32:04 smithi100 bash[76202]: audit 2024-04-07T16:32:03.293464+0000 mon.a (mon.0) 1559 : audit [INF] from='client.? 172.21.15.100:0/1873320617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:32:04.690 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:32:04 smithi100 bash[74942]: cluster 2024-04-07T16:32:02.904948+0000 mgr.y (mgr.25175) 501 : cluster [DBG] pgmap v952: 105 pgs: 105 active+clean; 583 KiB data, 2.5 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:32:04.690 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:32:04 smithi100 bash[74942]: cluster 2024-04-07T16:32:03.291004+0000 mon.a (mon.0) 1558 : cluster [DBG] osdmap e877: 8 total, 8 up, 8 in 2024-04-07T16:32:04.690 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:32:04 smithi100 bash[74942]: audit 2024-04-07T16:32:03.293464+0000 mon.a (mon.0) 1559 : audit [INF] from='client.? 172.21.15.100:0/1873320617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:32:05.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:32:05 smithi144 bash[66654]: audit 2024-04-07T16:32:04.286277+0000 mon.a (mon.0) 1560 : audit [INF] from='client.? 172.21.15.100:0/1873320617' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:32:05.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:32:05 smithi144 bash[66654]: cluster 2024-04-07T16:32:04.294870+0000 mon.a (mon.0) 1561 : cluster [DBG] osdmap e878: 8 total, 8 up, 8 in 2024-04-07T16:32:05.689 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:32:05 smithi100 bash[74942]: audit 2024-04-07T16:32:04.286277+0000 mon.a (mon.0) 1560 : audit [INF] from='client.? 172.21.15.100:0/1873320617' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:32:05.689 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:32:05 smithi100 bash[74942]: cluster 2024-04-07T16:32:04.294870+0000 mon.a (mon.0) 1561 : cluster [DBG] osdmap e878: 8 total, 8 up, 8 in 2024-04-07T16:32:05.689 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:32:05 smithi100 bash[76202]: audit 2024-04-07T16:32:04.286277+0000 mon.a (mon.0) 1560 : audit [INF] from='client.? 172.21.15.100:0/1873320617' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-109975-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:32:05.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:32:05 smithi100 bash[76202]: cluster 2024-04-07T16:32:04.294870+0000 mon.a (mon.0) 1561 : cluster [DBG] osdmap e878: 8 total, 8 up, 8 in 2024-04-07T16:32:06.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:32:06 smithi144 bash[66654]: cluster 2024-04-07T16:32:04.905545+0000 mgr.y (mgr.25175) 502 : cluster [DBG] pgmap v955: 137 pgs: 11 creating+peering, 21 unknown, 105 active+clean; 583 KiB data, 2.5 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:32:06.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:32:06 smithi144 bash[66654]: cluster 2024-04-07T16:32:05.319744+0000 mon.a (mon.0) 1562 : cluster [DBG] osdmap e879: 8 total, 8 up, 8 in 2024-04-07T16:32:06.689 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:32:06 smithi100 bash[74942]: cluster 2024-04-07T16:32:04.905545+0000 mgr.y (mgr.25175) 502 : cluster [DBG] pgmap v955: 137 pgs: 11 creating+peering, 21 unknown, 105 active+clean; 583 KiB data, 2.5 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:32:06.689 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:32:06 smithi100 bash[74942]: cluster 2024-04-07T16:32:05.319744+0000 mon.a (mon.0) 1562 : cluster [DBG] osdmap e879: 8 total, 8 up, 8 in 2024-04-07T16:32:06.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:32:06 smithi100 bash[76202]: cluster 2024-04-07T16:32:04.905545+0000 mgr.y (mgr.25175) 502 : cluster [DBG] pgmap v955: 137 pgs: 11 creating+peering, 21 unknown, 105 active+clean; 583 KiB data, 2.5 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:32:06.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:32:06 smithi100 bash[76202]: cluster 2024-04-07T16:32:05.319744+0000 mon.a (mon.0) 1562 : cluster [DBG] osdmap e879: 8 total, 8 up, 8 in 2024-04-07T16:32:08.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:32:08 smithi144 bash[66654]: cluster 2024-04-07T16:32:06.906184+0000 mgr.y (mgr.25175) 503 : cluster [DBG] pgmap v957: 137 pgs: 11 creating+peering, 19 unknown, 107 active+clean; 583 KiB data, 2.5 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:32:08.689 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:32:08 smithi100 bash[74942]: cluster 2024-04-07T16:32:06.906184+0000 mgr.y (mgr.25175) 503 : cluster [DBG] pgmap v957: 137 pgs: 11 creating+peering, 19 unknown, 107 active+clean; 583 KiB data, 2.5 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:32:08.690 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:32:08 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:32:08] "GET /metrics HTTP/1.1" 200 36408 "" "Prometheus/2.43.0" 2024-04-07T16:32:08.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:32:08 smithi100 bash[76202]: cluster 2024-04-07T16:32:06.906184+0000 mgr.y (mgr.25175) 503 : cluster [DBG] pgmap v957: 137 pgs: 11 creating+peering, 19 unknown, 107 active+clean; 583 KiB data, 2.5 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:32:10.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:32:10 smithi144 bash[66654]: cluster 2024-04-07T16:32:08.907346+0000 mgr.y (mgr.25175) 504 : cluster [DBG] pgmap v958: 137 pgs: 11 creating+peering, 126 active+clean; 606 KiB data, 2.5 GiB used, 713 GiB / 715 GiB avail; 3.3 MiB/s rd, 3.2 MiB/s wr, 675 op/s 2024-04-07T16:32:10.689 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:32:10 smithi100 bash[74942]: cluster 2024-04-07T16:32:08.907346+0000 mgr.y (mgr.25175) 504 : cluster [DBG] pgmap v958: 137 pgs: 11 creating+peering, 126 active+clean; 606 KiB data, 2.5 GiB used, 713 GiB / 715 GiB avail; 3.3 MiB/s rd, 3.2 MiB/s wr, 675 op/s 2024-04-07T16:32:10.689 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:32:10 smithi100 bash[76202]: cluster 2024-04-07T16:32:08.907346+0000 mgr.y (mgr.25175) 504 : cluster [DBG] pgmap v958: 137 pgs: 11 creating+peering, 126 active+clean; 606 KiB data, 2.5 GiB used, 713 GiB / 715 GiB avail; 3.3 MiB/s rd, 3.2 MiB/s wr, 675 op/s 2024-04-07T16:32:11.332 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestCls2PCQueue.AbortSpillover (9074 ms) 2024-04-07T16:32:11.332 INFO:tasks.workunit.client.0.smithi100.stdout:[----------] 22 tests from TestCls2PCQueue (137886 ms total) 2024-04-07T16:32:11.332 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T16:32:11.332 INFO:tasks.workunit.client.0.smithi100.stdout:[----------] Global test environment tear-down 2024-04-07T16:32:11.332 INFO:tasks.workunit.client.0.smithi100.stdout:[==========] 22 tests from 1 test suite ran. (137886 ms total) 2024-04-07T16:32:11.332 INFO:tasks.workunit.client.0.smithi100.stdout:[ PASSED ] 22 tests. 2024-04-07T16:32:11.338 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-07T16:32:11.339 DEBUG:teuthology.orchestra.run.smithi100:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-07T16:32:11.395 INFO:tasks.workunit:Running workunit cls/test_cls_cas.sh... 2024-04-07T16:32:11.395 DEBUG:teuthology.orchestra.run.smithi100:workunit test cls/test_cls_cas.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=reef TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_cas.sh 2024-04-07T16:32:11.476 INFO:tasks.workunit.client.0.smithi100.stdout:Running main() from gmock_main.cc 2024-04-07T16:32:11.476 INFO:tasks.workunit.client.0.smithi100.stdout:[==========] Running 6 tests from 2 test suites. 2024-04-07T16:32:11.476 INFO:tasks.workunit.client.0.smithi100.stdout:[----------] Global test environment set-up. 2024-04-07T16:32:11.476 INFO:tasks.workunit.client.0.smithi100.stdout:[----------] 5 tests from cls_cas 2024-04-07T16:32:11.476 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] cls_cas.get_put 2024-04-07T16:32:12.607 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:32:12 smithi144 bash[95233]: ts=2024-04-07T16:32:12.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:32:12.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:32:12 smithi144 bash[66654]: cluster 2024-04-07T16:32:10.908472+0000 mgr.y (mgr.25175) 505 : cluster [DBG] pgmap v959: 137 pgs: 137 active+clean; 606 KiB data, 2.5 GiB used, 713 GiB / 715 GiB avail; 2.6 MiB/s rd, 2.5 MiB/s wr, 531 op/s 2024-04-07T16:32:12.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:32:12 smithi144 bash[66654]: cluster 2024-04-07T16:32:11.341471+0000 mon.a (mon.0) 1563 : cluster [DBG] osdmap e880: 8 total, 8 up, 8 in 2024-04-07T16:32:12.689 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:32:12 smithi100 bash[74942]: cluster 2024-04-07T16:32:10.908472+0000 mgr.y (mgr.25175) 505 : cluster [DBG] pgmap v959: 137 pgs: 137 active+clean; 606 KiB data, 2.5 GiB used, 713 GiB / 715 GiB avail; 2.6 MiB/s rd, 2.5 MiB/s wr, 531 op/s 2024-04-07T16:32:12.689 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:32:12 smithi100 bash[74942]: cluster 2024-04-07T16:32:11.341471+0000 mon.a (mon.0) 1563 : cluster [DBG] osdmap e880: 8 total, 8 up, 8 in 2024-04-07T16:32:12.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:32:12 smithi100 bash[76202]: cluster 2024-04-07T16:32:10.908472+0000 mgr.y (mgr.25175) 505 : cluster [DBG] pgmap v959: 137 pgs: 137 active+clean; 606 KiB data, 2.5 GiB used, 713 GiB / 715 GiB avail; 2.6 MiB/s rd, 2.5 MiB/s wr, 531 op/s 2024-04-07T16:32:12.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:32:12 smithi100 bash[76202]: cluster 2024-04-07T16:32:11.341471+0000 mon.a (mon.0) 1563 : cluster [DBG] osdmap e880: 8 total, 8 up, 8 in 2024-04-07T16:32:13.689 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:32:13 smithi100 bash[74942]: cluster 2024-04-07T16:32:12.346566+0000 mon.a (mon.0) 1564 : cluster [DBG] osdmap e881: 8 total, 8 up, 8 in 2024-04-07T16:32:13.689 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:32:13 smithi100 bash[74942]: audit 2024-04-07T16:32:12.348659+0000 mon.a (mon.0) 1565 : audit [INF] from='client.? 172.21.15.100:0/502920528' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110482-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:32:13.690 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:32:13 smithi100 bash[74942]: audit 2024-04-07T16:32:13.340845+0000 mon.a (mon.0) 1566 : audit [INF] from='client.? 172.21.15.100:0/502920528' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110482-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:32:13.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:32:13 smithi100 bash[76202]: cluster 2024-04-07T16:32:12.346566+0000 mon.a (mon.0) 1564 : cluster [DBG] osdmap e881: 8 total, 8 up, 8 in 2024-04-07T16:32:13.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:32:13 smithi100 bash[76202]: audit 2024-04-07T16:32:12.348659+0000 mon.a (mon.0) 1565 : audit [INF] from='client.? 172.21.15.100:0/502920528' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110482-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:32:13.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:32:13 smithi100 bash[76202]: audit 2024-04-07T16:32:13.340845+0000 mon.a (mon.0) 1566 : audit [INF] from='client.? 172.21.15.100:0/502920528' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110482-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:32:13.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:32:13 smithi144 bash[66654]: cluster 2024-04-07T16:32:12.346566+0000 mon.a (mon.0) 1564 : cluster [DBG] osdmap e881: 8 total, 8 up, 8 in 2024-04-07T16:32:13.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:32:13 smithi144 bash[66654]: audit 2024-04-07T16:32:12.348659+0000 mon.a (mon.0) 1565 : audit [INF] from='client.? 172.21.15.100:0/502920528' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110482-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:32:13.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:32:13 smithi144 bash[66654]: audit 2024-04-07T16:32:13.340845+0000 mon.a (mon.0) 1566 : audit [INF] from='client.? 172.21.15.100:0/502920528' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110482-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:32:14.344 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] cls_cas.get_put (2868 ms) 2024-04-07T16:32:14.344 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] cls_cas.wrong_put 2024-04-07T16:32:14.689 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:32:14 smithi100 bash[74942]: cluster 2024-04-07T16:32:12.909196+0000 mgr.y (mgr.25175) 506 : cluster [DBG] pgmap v962: 137 pgs: 32 unknown, 105 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:32:14.690 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:32:14 smithi100 bash[74942]: cluster 2024-04-07T16:32:13.354768+0000 mon.a (mon.0) 1567 : cluster [DBG] osdmap e882: 8 total, 8 up, 8 in 2024-04-07T16:32:14.690 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:32:14 smithi100 bash[74942]: cluster 2024-04-07T16:32:13.363316+0000 mon.a (mon.0) 1568 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T16:32:14.690 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:32:14 smithi100 bash[74942]: cluster 2024-04-07T16:32:14.351995+0000 mon.a (mon.0) 1569 : cluster [DBG] osdmap e883: 8 total, 8 up, 8 in 2024-04-07T16:32:14.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:32:14 smithi100 bash[76202]: cluster 2024-04-07T16:32:12.909196+0000 mgr.y (mgr.25175) 506 : cluster [DBG] pgmap v962: 137 pgs: 32 unknown, 105 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:32:14.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:32:14 smithi100 bash[76202]: cluster 2024-04-07T16:32:13.354768+0000 mon.a (mon.0) 1567 : cluster [DBG] osdmap e882: 8 total, 8 up, 8 in 2024-04-07T16:32:14.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:32:14 smithi100 bash[76202]: cluster 2024-04-07T16:32:13.363316+0000 mon.a (mon.0) 1568 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T16:32:14.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:32:14 smithi100 bash[76202]: cluster 2024-04-07T16:32:14.351995+0000 mon.a (mon.0) 1569 : cluster [DBG] osdmap e883: 8 total, 8 up, 8 in 2024-04-07T16:32:14.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:32:14 smithi144 bash[66654]: cluster 2024-04-07T16:32:12.909196+0000 mgr.y (mgr.25175) 506 : cluster [DBG] pgmap v962: 137 pgs: 32 unknown, 105 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:32:14.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:32:14 smithi144 bash[66654]: cluster 2024-04-07T16:32:13.354768+0000 mon.a (mon.0) 1567 : cluster [DBG] osdmap e882: 8 total, 8 up, 8 in 2024-04-07T16:32:14.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:32:14 smithi144 bash[66654]: cluster 2024-04-07T16:32:13.363316+0000 mon.a (mon.0) 1568 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T16:32:14.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:32:14 smithi144 bash[66654]: cluster 2024-04-07T16:32:14.351995+0000 mon.a (mon.0) 1569 : cluster [DBG] osdmap e883: 8 total, 8 up, 8 in 2024-04-07T16:32:15.689 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:32:15 smithi100 bash[74942]: cluster 2024-04-07T16:32:15.373398+0000 mon.a (mon.0) 1570 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T16:32:15.690 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:32:15 smithi100 bash[74942]: cluster 2024-04-07T16:32:15.373461+0000 mon.a (mon.0) 1571 : cluster [INF] Cluster is now healthy 2024-04-07T16:32:15.690 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:32:15 smithi100 bash[74942]: cluster 2024-04-07T16:32:15.394466+0000 mon.a (mon.0) 1572 : cluster [DBG] osdmap e884: 8 total, 8 up, 8 in 2024-04-07T16:32:15.690 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:32:15 smithi100 bash[74942]: audit 2024-04-07T16:32:15.401447+0000 mon.a (mon.0) 1573 : audit [INF] from='client.? 172.21.15.100:0/3001067879' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110482-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:32:15.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:32:15 smithi100 bash[76202]: cluster 2024-04-07T16:32:15.373398+0000 mon.a (mon.0) 1570 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T16:32:15.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:32:15 smithi100 bash[76202]: cluster 2024-04-07T16:32:15.373461+0000 mon.a (mon.0) 1571 : cluster [INF] Cluster is now healthy 2024-04-07T16:32:15.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:32:15 smithi100 bash[76202]: cluster 2024-04-07T16:32:15.394466+0000 mon.a (mon.0) 1572 : cluster [DBG] osdmap e884: 8 total, 8 up, 8 in 2024-04-07T16:32:15.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:32:15 smithi100 bash[76202]: audit 2024-04-07T16:32:15.401447+0000 mon.a (mon.0) 1573 : audit [INF] from='client.? 172.21.15.100:0/3001067879' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110482-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:32:15.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:32:15 smithi144 bash[66654]: cluster 2024-04-07T16:32:15.373398+0000 mon.a (mon.0) 1570 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T16:32:15.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:32:15 smithi144 bash[66654]: cluster 2024-04-07T16:32:15.373461+0000 mon.a (mon.0) 1571 : cluster [INF] Cluster is now healthy 2024-04-07T16:32:15.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:32:15 smithi144 bash[66654]: cluster 2024-04-07T16:32:15.394466+0000 mon.a (mon.0) 1572 : cluster [DBG] osdmap e884: 8 total, 8 up, 8 in 2024-04-07T16:32:15.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:32:15 smithi144 bash[66654]: audit 2024-04-07T16:32:15.401447+0000 mon.a (mon.0) 1573 : audit [INF] from='client.? 172.21.15.100:0/3001067879' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110482-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:32:16.689 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:32:16 smithi100 bash[74942]: cluster 2024-04-07T16:32:14.909964+0000 mgr.y (mgr.25175) 507 : cluster [DBG] pgmap v965: 105 pgs: 105 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:32:16.689 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:32:16 smithi100 bash[74942]: audit 2024-04-07T16:32:16.388910+0000 mon.a (mon.0) 1574 : audit [INF] from='client.? 172.21.15.100:0/3001067879' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110482-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:32:16.690 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:32:16 smithi100 bash[74942]: cluster 2024-04-07T16:32:16.396480+0000 mon.a (mon.0) 1575 : cluster [DBG] osdmap e885: 8 total, 8 up, 8 in 2024-04-07T16:32:16.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:32:16 smithi100 bash[76202]: cluster 2024-04-07T16:32:14.909964+0000 mgr.y (mgr.25175) 507 : cluster [DBG] pgmap v965: 105 pgs: 105 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:32:16.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:32:16 smithi100 bash[76202]: audit 2024-04-07T16:32:16.388910+0000 mon.a (mon.0) 1574 : audit [INF] from='client.? 172.21.15.100:0/3001067879' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110482-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:32:16.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:32:16 smithi100 bash[76202]: cluster 2024-04-07T16:32:16.396480+0000 mon.a (mon.0) 1575 : cluster [DBG] osdmap e885: 8 total, 8 up, 8 in 2024-04-07T16:32:16.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:32:16 smithi144 bash[66654]: cluster 2024-04-07T16:32:14.909964+0000 mgr.y (mgr.25175) 507 : cluster [DBG] pgmap v965: 105 pgs: 105 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:32:16.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:32:16 smithi144 bash[66654]: audit 2024-04-07T16:32:16.388910+0000 mon.a (mon.0) 1574 : audit [INF] from='client.? 172.21.15.100:0/3001067879' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110482-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:32:16.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:32:16 smithi144 bash[66654]: cluster 2024-04-07T16:32:16.396480+0000 mon.a (mon.0) 1575 : cluster [DBG] osdmap e885: 8 total, 8 up, 8 in 2024-04-07T16:32:17.389 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] cls_cas.wrong_put (3045 ms) 2024-04-07T16:32:17.390 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] cls_cas.dup_get 2024-04-07T16:32:17.689 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:32:17 smithi100 bash[74942]: audit 2024-04-07T16:32:16.743354+0000 mon.a (mon.0) 1576 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:32:17.690 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:32:17 smithi100 bash[74942]: cluster 2024-04-07T16:32:17.400064+0000 mon.a (mon.0) 1577 : cluster [DBG] osdmap e886: 8 total, 8 up, 8 in 2024-04-07T16:32:17.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:32:17 smithi100 bash[76202]: audit 2024-04-07T16:32:16.743354+0000 mon.a (mon.0) 1576 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:32:17.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:32:17 smithi100 bash[76202]: cluster 2024-04-07T16:32:17.400064+0000 mon.a (mon.0) 1577 : cluster [DBG] osdmap e886: 8 total, 8 up, 8 in 2024-04-07T16:32:17.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:32:17 smithi144 bash[66654]: audit 2024-04-07T16:32:16.743354+0000 mon.a (mon.0) 1576 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:32:17.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:32:17 smithi144 bash[66654]: cluster 2024-04-07T16:32:17.400064+0000 mon.a (mon.0) 1577 : cluster [DBG] osdmap e886: 8 total, 8 up, 8 in 2024-04-07T16:32:18.689 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:32:18 smithi100 bash[74942]: cluster 2024-04-07T16:32:16.910814+0000 mgr.y (mgr.25175) 508 : cluster [DBG] pgmap v968: 137 pgs: 32 unknown, 105 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:32:18.690 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:32:18 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:32:18] "GET /metrics HTTP/1.1" 200 36359 "" "Prometheus/2.43.0" 2024-04-07T16:32:18.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:32:18 smithi100 bash[76202]: cluster 2024-04-07T16:32:16.910814+0000 mgr.y (mgr.25175) 508 : cluster [DBG] pgmap v968: 137 pgs: 32 unknown, 105 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:32:18.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:32:18 smithi144 bash[66654]: cluster 2024-04-07T16:32:16.910814+0000 mgr.y (mgr.25175) 508 : cluster [DBG] pgmap v968: 137 pgs: 32 unknown, 105 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:32:19.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:32:19 smithi144 bash[66654]: cluster 2024-04-07T16:32:18.463757+0000 mon.a (mon.0) 1578 : cluster [DBG] osdmap e887: 8 total, 8 up, 8 in 2024-04-07T16:32:19.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:32:19 smithi144 bash[66654]: audit 2024-04-07T16:32:18.465670+0000 mon.c (mon.2) 33 : audit [INF] from='client.? 172.21.15.100:0/3472094739' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110482-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:32:19.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:32:19 smithi144 bash[66654]: audit 2024-04-07T16:32:18.466389+0000 mon.a (mon.0) 1579 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110482-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:32:19.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:32:19 smithi144 bash[66654]: cluster 2024-04-07T16:32:18.911727+0000 mgr.y (mgr.25175) 509 : cluster [DBG] pgmap v971: 137 pgs: 32 unknown, 105 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:32:19.939 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:32:19 smithi100 bash[74942]: cluster 2024-04-07T16:32:18.463757+0000 mon.a (mon.0) 1578 : cluster [DBG] osdmap e887: 8 total, 8 up, 8 in 2024-04-07T16:32:19.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:32:19 smithi100 bash[74942]: audit 2024-04-07T16:32:18.465670+0000 mon.c (mon.2) 33 : audit [INF] from='client.? 172.21.15.100:0/3472094739' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110482-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:32:19.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:32:19 smithi100 bash[74942]: audit 2024-04-07T16:32:18.466389+0000 mon.a (mon.0) 1579 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110482-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:32:19.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:32:19 smithi100 bash[74942]: cluster 2024-04-07T16:32:18.911727+0000 mgr.y (mgr.25175) 509 : cluster [DBG] pgmap v971: 137 pgs: 32 unknown, 105 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:32:19.940 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:32:19 smithi100 bash[76202]: cluster 2024-04-07T16:32:18.463757+0000 mon.a (mon.0) 1578 : cluster [DBG] osdmap e887: 8 total, 8 up, 8 in 2024-04-07T16:32:19.940 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:32:19 smithi100 bash[76202]: audit 2024-04-07T16:32:18.465670+0000 mon.c (mon.2) 33 : audit [INF] from='client.? 172.21.15.100:0/3472094739' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110482-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:32:19.940 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:32:19 smithi100 bash[76202]: audit 2024-04-07T16:32:18.466389+0000 mon.a (mon.0) 1579 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110482-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:32:19.940 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:32:19 smithi100 bash[76202]: cluster 2024-04-07T16:32:18.911727+0000 mgr.y (mgr.25175) 509 : cluster [DBG] pgmap v971: 137 pgs: 32 unknown, 105 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:32:20.483 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] cls_cas.dup_get (3094 ms) 2024-04-07T16:32:20.483 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] cls_cas.dup_put 2024-04-07T16:32:20.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:32:20 smithi144 bash[66654]: cluster 2024-04-07T16:32:19.456472+0000 mon.a (mon.0) 1580 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T16:32:20.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:32:20 smithi144 bash[66654]: audit 2024-04-07T16:32:19.458022+0000 mon.a (mon.0) 1581 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110482-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:32:20.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:32:20 smithi144 bash[66654]: cluster 2024-04-07T16:32:19.467106+0000 mon.a (mon.0) 1582 : cluster [DBG] osdmap e888: 8 total, 8 up, 8 in 2024-04-07T16:32:20.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:32:20 smithi100 bash[74942]: cluster 2024-04-07T16:32:19.456472+0000 mon.a (mon.0) 1580 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T16:32:20.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:32:20 smithi100 bash[74942]: audit 2024-04-07T16:32:19.458022+0000 mon.a (mon.0) 1581 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110482-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:32:20.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:32:20 smithi100 bash[74942]: cluster 2024-04-07T16:32:19.467106+0000 mon.a (mon.0) 1582 : cluster [DBG] osdmap e888: 8 total, 8 up, 8 in 2024-04-07T16:32:20.940 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:32:20 smithi100 bash[76202]: cluster 2024-04-07T16:32:19.456472+0000 mon.a (mon.0) 1580 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T16:32:20.940 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:32:20 smithi100 bash[76202]: audit 2024-04-07T16:32:19.458022+0000 mon.a (mon.0) 1581 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110482-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:32:20.940 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:32:20 smithi100 bash[76202]: cluster 2024-04-07T16:32:19.467106+0000 mon.a (mon.0) 1582 : cluster [DBG] osdmap e888: 8 total, 8 up, 8 in 2024-04-07T16:32:21.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:32:21 smithi144 bash[66654]: cluster 2024-04-07T16:32:20.492394+0000 mon.a (mon.0) 1583 : cluster [DBG] osdmap e889: 8 total, 8 up, 8 in 2024-04-07T16:32:21.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:32:21 smithi144 bash[66654]: cluster 2024-04-07T16:32:20.912355+0000 mgr.y (mgr.25175) 510 : cluster [DBG] pgmap v974: 105 pgs: 105 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:32:21.939 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:32:21 smithi100 bash[74942]: cluster 2024-04-07T16:32:20.492394+0000 mon.a (mon.0) 1583 : cluster [DBG] osdmap e889: 8 total, 8 up, 8 in 2024-04-07T16:32:21.939 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:32:21 smithi100 bash[74942]: cluster 2024-04-07T16:32:20.912355+0000 mgr.y (mgr.25175) 510 : cluster [DBG] pgmap v974: 105 pgs: 105 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:32:21.940 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:32:21 smithi100 bash[76202]: cluster 2024-04-07T16:32:20.492394+0000 mon.a (mon.0) 1583 : cluster [DBG] osdmap e889: 8 total, 8 up, 8 in 2024-04-07T16:32:21.940 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:32:21 smithi100 bash[76202]: cluster 2024-04-07T16:32:20.912355+0000 mgr.y (mgr.25175) 510 : cluster [DBG] pgmap v974: 105 pgs: 105 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:32:22.607 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:32:22 smithi144 bash[95233]: ts=2024-04-07T16:32:22.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:32:22.676 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:32:22 smithi144 bash[66654]: cluster 2024-04-07T16:32:21.485282+0000 mon.a (mon.0) 1584 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T16:32:22.676 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:32:22 smithi144 bash[66654]: cluster 2024-04-07T16:32:21.485361+0000 mon.a (mon.0) 1585 : cluster [INF] Cluster is now healthy 2024-04-07T16:32:22.676 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:32:22 smithi144 bash[66654]: cluster 2024-04-07T16:32:21.508780+0000 mon.a (mon.0) 1586 : cluster [DBG] osdmap e890: 8 total, 8 up, 8 in 2024-04-07T16:32:22.676 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:32:22 smithi144 bash[66654]: audit 2024-04-07T16:32:21.515094+0000 mon.c (mon.2) 34 : audit [INF] from='client.? 172.21.15.100:0/2681360268' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110482-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:32:22.676 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:32:22 smithi144 bash[66654]: audit 2024-04-07T16:32:21.516133+0000 mon.a (mon.0) 1587 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110482-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:32:22.939 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:32:22 smithi100 bash[74942]: cluster 2024-04-07T16:32:21.485282+0000 mon.a (mon.0) 1584 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T16:32:22.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:32:22 smithi100 bash[74942]: cluster 2024-04-07T16:32:21.485361+0000 mon.a (mon.0) 1585 : cluster [INF] Cluster is now healthy 2024-04-07T16:32:22.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:32:22 smithi100 bash[74942]: cluster 2024-04-07T16:32:21.508780+0000 mon.a (mon.0) 1586 : cluster [DBG] osdmap e890: 8 total, 8 up, 8 in 2024-04-07T16:32:22.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:32:22 smithi100 bash[74942]: audit 2024-04-07T16:32:21.515094+0000 mon.c (mon.2) 34 : audit [INF] from='client.? 172.21.15.100:0/2681360268' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110482-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:32:22.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:32:22 smithi100 bash[74942]: audit 2024-04-07T16:32:21.516133+0000 mon.a (mon.0) 1587 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110482-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:32:22.940 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:32:22 smithi100 bash[76202]: cluster 2024-04-07T16:32:21.485282+0000 mon.a (mon.0) 1584 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T16:32:22.940 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:32:22 smithi100 bash[76202]: cluster 2024-04-07T16:32:21.485361+0000 mon.a (mon.0) 1585 : cluster [INF] Cluster is now healthy 2024-04-07T16:32:22.940 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:32:22 smithi100 bash[76202]: cluster 2024-04-07T16:32:21.508780+0000 mon.a (mon.0) 1586 : cluster [DBG] osdmap e890: 8 total, 8 up, 8 in 2024-04-07T16:32:22.940 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:32:22 smithi100 bash[76202]: audit 2024-04-07T16:32:21.515094+0000 mon.c (mon.2) 34 : audit [INF] from='client.? 172.21.15.100:0/2681360268' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110482-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:32:22.940 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:32:22 smithi100 bash[76202]: audit 2024-04-07T16:32:21.516133+0000 mon.a (mon.0) 1587 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110482-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:32:23.526 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] cls_cas.dup_put (3042 ms) 2024-04-07T16:32:23.526 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] cls_cas.get_wrong_data 2024-04-07T16:32:23.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:32:23 smithi144 bash[66654]: audit 2024-04-07T16:32:22.513225+0000 mon.a (mon.0) 1588 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110482-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:32:23.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:32:23 smithi144 bash[66654]: cluster 2024-04-07T16:32:22.520880+0000 mon.a (mon.0) 1589 : cluster [DBG] osdmap e891: 8 total, 8 up, 8 in 2024-04-07T16:32:23.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:32:23 smithi144 bash[66654]: cluster 2024-04-07T16:32:22.913019+0000 mgr.y (mgr.25175) 511 : cluster [DBG] pgmap v977: 137 pgs: 18 creating+peering, 14 unknown, 105 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:32:23.939 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:32:23 smithi100 bash[76202]: audit 2024-04-07T16:32:22.513225+0000 mon.a (mon.0) 1588 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110482-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:32:23.939 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:32:23 smithi100 bash[76202]: cluster 2024-04-07T16:32:22.520880+0000 mon.a (mon.0) 1589 : cluster [DBG] osdmap e891: 8 total, 8 up, 8 in 2024-04-07T16:32:23.940 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:32:23 smithi100 bash[76202]: cluster 2024-04-07T16:32:22.913019+0000 mgr.y (mgr.25175) 511 : cluster [DBG] pgmap v977: 137 pgs: 18 creating+peering, 14 unknown, 105 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:32:23.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:32:23 smithi100 bash[74942]: audit 2024-04-07T16:32:22.513225+0000 mon.a (mon.0) 1588 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110482-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:32:23.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:32:23 smithi100 bash[74942]: cluster 2024-04-07T16:32:22.520880+0000 mon.a (mon.0) 1589 : cluster [DBG] osdmap e891: 8 total, 8 up, 8 in 2024-04-07T16:32:23.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:32:23 smithi100 bash[74942]: cluster 2024-04-07T16:32:22.913019+0000 mgr.y (mgr.25175) 511 : cluster [DBG] pgmap v977: 137 pgs: 18 creating+peering, 14 unknown, 105 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:32:24.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:32:24 smithi144 bash[66654]: cluster 2024-04-07T16:32:23.530687+0000 mon.a (mon.0) 1590 : cluster [DBG] osdmap e892: 8 total, 8 up, 8 in 2024-04-07T16:32:24.939 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:32:24 smithi100 bash[74942]: cluster 2024-04-07T16:32:23.530687+0000 mon.a (mon.0) 1590 : cluster [DBG] osdmap e892: 8 total, 8 up, 8 in 2024-04-07T16:32:24.939 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:32:24 smithi100 bash[76202]: cluster 2024-04-07T16:32:23.530687+0000 mon.a (mon.0) 1590 : cluster [DBG] osdmap e892: 8 total, 8 up, 8 in 2024-04-07T16:32:25.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:32:25 smithi144 bash[66654]: cluster 2024-04-07T16:32:24.542869+0000 mon.a (mon.0) 1591 : cluster [DBG] osdmap e893: 8 total, 8 up, 8 in 2024-04-07T16:32:25.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:32:25 smithi144 bash[66654]: audit 2024-04-07T16:32:24.550692+0000 mon.c (mon.2) 35 : audit [INF] from='client.? 172.21.15.100:0/4240870200' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110482-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:32:25.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:32:25 smithi144 bash[66654]: audit 2024-04-07T16:32:24.556097+0000 mon.a (mon.0) 1592 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110482-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:32:25.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:32:25 smithi144 bash[66654]: cluster 2024-04-07T16:32:24.913831+0000 mgr.y (mgr.25175) 512 : cluster [DBG] pgmap v980: 137 pgs: 32 unknown, 105 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:32:25.939 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:32:25 smithi100 bash[76202]: cluster 2024-04-07T16:32:24.542869+0000 mon.a (mon.0) 1591 : cluster [DBG] osdmap e893: 8 total, 8 up, 8 in 2024-04-07T16:32:25.940 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:32:25 smithi100 bash[76202]: audit 2024-04-07T16:32:24.550692+0000 mon.c (mon.2) 35 : audit [INF] from='client.? 172.21.15.100:0/4240870200' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110482-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:32:25.940 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:32:25 smithi100 bash[76202]: audit 2024-04-07T16:32:24.556097+0000 mon.a (mon.0) 1592 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110482-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:32:25.940 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:32:25 smithi100 bash[76202]: cluster 2024-04-07T16:32:24.913831+0000 mgr.y (mgr.25175) 512 : cluster [DBG] pgmap v980: 137 pgs: 32 unknown, 105 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:32:25.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:32:25 smithi100 bash[74942]: cluster 2024-04-07T16:32:24.542869+0000 mon.a (mon.0) 1591 : cluster [DBG] osdmap e893: 8 total, 8 up, 8 in 2024-04-07T16:32:25.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:32:25 smithi100 bash[74942]: audit 2024-04-07T16:32:24.550692+0000 mon.c (mon.2) 35 : audit [INF] from='client.? 172.21.15.100:0/4240870200' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110482-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:32:25.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:32:25 smithi100 bash[74942]: audit 2024-04-07T16:32:24.556097+0000 mon.a (mon.0) 1592 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110482-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:32:25.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:32:25 smithi100 bash[74942]: cluster 2024-04-07T16:32:24.913831+0000 mgr.y (mgr.25175) 512 : cluster [DBG] pgmap v980: 137 pgs: 32 unknown, 105 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:32:26.548 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] cls_cas.get_wrong_data (3022 ms) 2024-04-07T16:32:26.548 INFO:tasks.workunit.client.0.smithi100.stdout:[----------] 5 tests from cls_cas (15072 ms total) 2024-04-07T16:32:26.548 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T16:32:26.548 INFO:tasks.workunit.client.0.smithi100.stdout:[----------] 1 test from chunk_refs_t 2024-04-07T16:32:26.548 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] chunk_refs_t.size 2024-04-07T16:32:26.548 INFO:tasks.workunit.client.0.smithi100.stdout:1 60 by_object 2024-04-07T16:32:26.548 INFO:tasks.workunit.client.0.smithi100.stdout:2 103 by_object 2024-04-07T16:32:26.548 INFO:tasks.workunit.client.0.smithi100.stdout:4 189 by_object 2024-04-07T16:32:26.549 INFO:tasks.workunit.client.0.smithi100.stdout:8 361 by_object 2024-04-07T16:32:26.549 INFO:tasks.workunit.client.0.smithi100.stdout:16 112 by_hash(32 bits) 2024-04-07T16:32:26.549 INFO:tasks.workunit.client.0.smithi100.stdout:32 208 by_hash(32 bits) 2024-04-07T16:32:26.549 INFO:tasks.workunit.client.0.smithi100.stdout:64 400 by_hash(32 bits) 2024-04-07T16:32:26.549 INFO:tasks.workunit.client.0.smithi100.stdout:128 509 by_hash(10 bits) 2024-04-07T16:32:26.550 INFO:tasks.workunit.client.0.smithi100.stdout:256 499 by_hash(7 bits) 2024-04-07T16:32:26.550 INFO:tasks.workunit.client.0.smithi100.stdout:512 462 by_hash(5 bits) 2024-04-07T16:32:26.551 INFO:tasks.workunit.client.0.smithi100.stdout:1024 466 by_hash(5 bits) 2024-04-07T16:32:26.552 INFO:tasks.workunit.client.0.smithi100.stdout:2048 466 by_hash(5 bits) 2024-04-07T16:32:26.553 INFO:tasks.workunit.client.0.smithi100.stdout:4096 470 by_hash(5 bits) 2024-04-07T16:32:26.556 INFO:tasks.workunit.client.0.smithi100.stdout:8192 498 by_hash(4 bits) 2024-04-07T16:32:26.560 INFO:tasks.workunit.client.0.smithi100.stdout:16384 278 by_hash(3 bits) 2024-04-07T16:32:26.566 INFO:tasks.workunit.client.0.smithi100.stdout:32768 337 by_hash(2 bits) 2024-04-07T16:32:26.580 INFO:tasks.workunit.client.0.smithi100.stdout:65536 367 by_hash(2 bits) 2024-04-07T16:32:26.605 INFO:tasks.workunit.client.0.smithi100.stdout:131072 387 by_hash(1 bits) 2024-04-07T16:32:26.629 INFO:tasks.workunit.client.0.smithi100.stdout:262144 331 by_pool 2024-04-07T16:32:26.663 INFO:tasks.workunit.client.0.smithi100.stdout:524288 21 count 2024-04-07T16:32:26.721 INFO:tasks.workunit.client.0.smithi100.stdout:1048576 21 count 2024-04-07T16:32:26.831 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] chunk_refs_t.size (284 ms) 2024-04-07T16:32:26.832 INFO:tasks.workunit.client.0.smithi100.stdout:[----------] 1 test from chunk_refs_t (284 ms total) 2024-04-07T16:32:26.832 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T16:32:26.832 INFO:tasks.workunit.client.0.smithi100.stdout:[----------] Global test environment tear-down 2024-04-07T16:32:26.832 INFO:tasks.workunit.client.0.smithi100.stdout:[==========] 6 tests from 2 test suites ran. (15356 ms total) 2024-04-07T16:32:26.832 INFO:tasks.workunit.client.0.smithi100.stdout:[ PASSED ] 6 tests. 2024-04-07T16:32:26.834 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-07T16:32:26.835 DEBUG:teuthology.orchestra.run.smithi100:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-07T16:32:26.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:32:26 smithi144 bash[66654]: cluster 2024-04-07T16:32:25.536256+0000 mon.a (mon.0) 1593 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T16:32:26.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:32:26 smithi144 bash[66654]: audit 2024-04-07T16:32:25.545431+0000 mon.a (mon.0) 1594 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110482-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:32:26.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:32:26 smithi144 bash[66654]: cluster 2024-04-07T16:32:25.555044+0000 mon.a (mon.0) 1595 : cluster [DBG] osdmap e894: 8 total, 8 up, 8 in 2024-04-07T16:32:26.885 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:32:26 smithi100 bash[74942]: cluster 2024-04-07T16:32:25.536256+0000 mon.a (mon.0) 1593 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T16:32:26.885 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:32:26 smithi100 bash[74942]: audit 2024-04-07T16:32:25.545431+0000 mon.a (mon.0) 1594 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110482-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:32:26.885 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:32:26 smithi100 bash[74942]: cluster 2024-04-07T16:32:25.555044+0000 mon.a (mon.0) 1595 : cluster [DBG] osdmap e894: 8 total, 8 up, 8 in 2024-04-07T16:32:26.885 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:32:26 smithi100 bash[76202]: cluster 2024-04-07T16:32:25.536256+0000 mon.a (mon.0) 1593 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T16:32:26.885 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:32:26 smithi100 bash[76202]: audit 2024-04-07T16:32:25.545431+0000 mon.a (mon.0) 1594 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110482-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:32:26.885 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:32:26 smithi100 bash[76202]: cluster 2024-04-07T16:32:25.555044+0000 mon.a (mon.0) 1595 : cluster [DBG] osdmap e894: 8 total, 8 up, 8 in 2024-04-07T16:32:26.887 INFO:tasks.workunit:Running workunit cls/test_cls_cmpomap.sh... 2024-04-07T16:32:26.887 DEBUG:teuthology.orchestra.run.smithi100:workunit test cls/test_cls_cmpomap.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=reef TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_cmpomap.sh 2024-04-07T16:32:26.951 INFO:tasks.workunit.client.0.smithi100.stdout:Running main() from gmock_main.cc 2024-04-07T16:32:26.951 INFO:tasks.workunit.client.0.smithi100.stdout:[==========] Running 31 tests from 1 test suite. 2024-04-07T16:32:26.951 INFO:tasks.workunit.client.0.smithi100.stdout:[----------] Global test environment set-up. 2024-04-07T16:32:27.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:32:27 smithi144 bash[66654]: cluster 2024-04-07T16:32:26.555443+0000 mon.a (mon.0) 1596 : cluster [DBG] osdmap e895: 8 total, 8 up, 8 in 2024-04-07T16:32:27.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:32:27 smithi144 bash[66654]: cluster 2024-04-07T16:32:26.914617+0000 mgr.y (mgr.25175) 513 : cluster [DBG] pgmap v983: 105 pgs: 105 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:32:27.939 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:32:27 smithi100 bash[74942]: cluster 2024-04-07T16:32:26.555443+0000 mon.a (mon.0) 1596 : cluster [DBG] osdmap e895: 8 total, 8 up, 8 in 2024-04-07T16:32:27.939 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:32:27 smithi100 bash[74942]: cluster 2024-04-07T16:32:26.914617+0000 mgr.y (mgr.25175) 513 : cluster [DBG] pgmap v983: 105 pgs: 105 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:32:27.940 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:32:27 smithi100 bash[76202]: cluster 2024-04-07T16:32:26.555443+0000 mon.a (mon.0) 1596 : cluster [DBG] osdmap e895: 8 total, 8 up, 8 in 2024-04-07T16:32:27.940 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:32:27 smithi100 bash[76202]: cluster 2024-04-07T16:32:26.914617+0000 mgr.y (mgr.25175) 513 : cluster [DBG] pgmap v983: 105 pgs: 105 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:32:28.575 INFO:tasks.workunit.client.0.smithi100.stdout:[----------] 31 tests from CmpOmap 2024-04-07T16:32:28.575 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] CmpOmap.cmp_vals_noexist_str 2024-04-07T16:32:28.611 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] CmpOmap.cmp_vals_noexist_str (36 ms) 2024-04-07T16:32:28.611 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] CmpOmap.cmp_vals_noexist_str_default 2024-04-07T16:32:28.615 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] CmpOmap.cmp_vals_noexist_str_default (4 ms) 2024-04-07T16:32:28.615 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] CmpOmap.cmp_vals_noexist_u64 2024-04-07T16:32:28.619 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] CmpOmap.cmp_vals_noexist_u64 (4 ms) 2024-04-07T16:32:28.619 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] CmpOmap.cmp_vals_noexist_u64_default 2024-04-07T16:32:28.623 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] CmpOmap.cmp_vals_noexist_u64_default (4 ms) 2024-04-07T16:32:28.623 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] CmpOmap.cmp_vals_str 2024-04-07T16:32:28.630 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] CmpOmap.cmp_vals_str (7 ms) 2024-04-07T16:32:28.630 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] CmpOmap.cmp_vals_u64 2024-04-07T16:32:28.643 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] CmpOmap.cmp_vals_u64 (13 ms) 2024-04-07T16:32:28.643 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] CmpOmap.cmp_vals_u64_invalid_input 2024-04-07T16:32:28.648 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] CmpOmap.cmp_vals_u64_invalid_input (5 ms) 2024-04-07T16:32:28.648 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] CmpOmap.cmp_vals_u64_invalid_default 2024-04-07T16:32:28.652 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] CmpOmap.cmp_vals_u64_invalid_default (4 ms) 2024-04-07T16:32:28.652 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] CmpOmap.cmp_vals_u64_empty_default 2024-04-07T16:32:28.656 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] CmpOmap.cmp_vals_u64_empty_default (4 ms) 2024-04-07T16:32:28.656 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] CmpOmap.cmp_vals_u64_invalid_value 2024-04-07T16:32:28.663 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] CmpOmap.cmp_vals_u64_invalid_value (6 ms) 2024-04-07T16:32:28.663 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] CmpOmap.cmp_vals_at_max_keys 2024-04-07T16:32:28.664 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] CmpOmap.cmp_vals_at_max_keys (2 ms) 2024-04-07T16:32:28.665 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] CmpOmap.cmp_vals_over_max_keys 2024-04-07T16:32:28.666 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] CmpOmap.cmp_vals_over_max_keys (2 ms) 2024-04-07T16:32:28.666 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] CmpOmap.cmp_set_vals_noexist_str 2024-04-07T16:32:28.678 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] CmpOmap.cmp_set_vals_noexist_str (11 ms) 2024-04-07T16:32:28.678 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] CmpOmap.cmp_set_vals_noexist_str_default 2024-04-07T16:32:28.689 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:32:28 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:32:28] "GET /metrics HTTP/1.1" 200 36359 "" "Prometheus/2.43.0" 2024-04-07T16:32:28.689 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:32:28 smithi100 bash[74942]: cluster 2024-04-07T16:32:27.560327+0000 mon.a (mon.0) 1597 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T16:32:28.689 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:32:28 smithi100 bash[74942]: cluster 2024-04-07T16:32:27.560396+0000 mon.a (mon.0) 1598 : cluster [INF] Cluster is now healthy 2024-04-07T16:32:28.689 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:32:28 smithi100 bash[74942]: cluster 2024-04-07T16:32:27.581786+0000 mon.a (mon.0) 1599 : cluster [DBG] osdmap e896: 8 total, 8 up, 8 in 2024-04-07T16:32:28.690 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:32:28 smithi100 bash[74942]: audit 2024-04-07T16:32:27.583753+0000 mon.a (mon.0) 1600 : audit [INF] from='client.? 172.21.15.100:0/3262394495' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110574-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:32:28.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:32:28 smithi100 bash[76202]: cluster 2024-04-07T16:32:27.560327+0000 mon.a (mon.0) 1597 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T16:32:28.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:32:28 smithi100 bash[76202]: cluster 2024-04-07T16:32:27.560396+0000 mon.a (mon.0) 1598 : cluster [INF] Cluster is now healthy 2024-04-07T16:32:28.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:32:28 smithi100 bash[76202]: cluster 2024-04-07T16:32:27.581786+0000 mon.a (mon.0) 1599 : cluster [DBG] osdmap e896: 8 total, 8 up, 8 in 2024-04-07T16:32:28.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:32:28 smithi100 bash[76202]: audit 2024-04-07T16:32:27.583753+0000 mon.a (mon.0) 1600 : audit [INF] from='client.? 172.21.15.100:0/3262394495' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110574-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:32:28.690 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] CmpOmap.cmp_set_vals_noexist_str_default (12 ms) 2024-04-07T16:32:28.690 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] CmpOmap.cmp_set_vals_noexist_u64 2024-04-07T16:32:28.700 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] CmpOmap.cmp_set_vals_noexist_u64 (11 ms) 2024-04-07T16:32:28.700 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] CmpOmap.cmp_set_vals_noexist_u64_default 2024-04-07T16:32:28.718 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] CmpOmap.cmp_set_vals_noexist_u64_default (18 ms) 2024-04-07T16:32:28.718 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] CmpOmap.cmp_set_vals_str 2024-04-07T16:32:28.733 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] CmpOmap.cmp_set_vals_str (15 ms) 2024-04-07T16:32:28.733 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] CmpOmap.cmp_set_vals_u64 2024-04-07T16:32:28.750 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] CmpOmap.cmp_set_vals_u64 (16 ms) 2024-04-07T16:32:28.750 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] CmpOmap.cmp_set_vals_u64_einval 2024-04-07T16:32:28.755 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] CmpOmap.cmp_set_vals_u64_einval (5 ms) 2024-04-07T16:32:28.755 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] CmpOmap.cmp_set_vals_u64_eio 2024-04-07T16:32:28.760 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] CmpOmap.cmp_set_vals_u64_eio (5 ms) 2024-04-07T16:32:28.760 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] CmpOmap.cmp_set_vals_at_max_keys 2024-04-07T16:32:28.763 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] CmpOmap.cmp_set_vals_at_max_keys (3 ms) 2024-04-07T16:32:28.763 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] CmpOmap.cmp_set_vals_over_max_keys 2024-04-07T16:32:28.765 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] CmpOmap.cmp_set_vals_over_max_keys (2 ms) 2024-04-07T16:32:28.765 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] CmpOmap.cmp_rm_keys_noexist_str 2024-04-07T16:32:28.785 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] CmpOmap.cmp_rm_keys_noexist_str (20 ms) 2024-04-07T16:32:28.785 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] CmpOmap.cmp_rm_keys_noexist_u64 2024-04-07T16:32:28.799 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] CmpOmap.cmp_rm_keys_noexist_u64 (14 ms) 2024-04-07T16:32:28.799 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] CmpOmap.cmp_rm_keys_str 2024-04-07T16:32:28.816 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] CmpOmap.cmp_rm_keys_str (17 ms) 2024-04-07T16:32:28.816 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] CmpOmap.cmp_rm_keys_u64 2024-04-07T16:32:28.835 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] CmpOmap.cmp_rm_keys_u64 (19 ms) 2024-04-07T16:32:28.835 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] CmpOmap.cmp_rm_keys_u64_einval 2024-04-07T16:32:28.841 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] CmpOmap.cmp_rm_keys_u64_einval (5 ms) 2024-04-07T16:32:28.841 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] CmpOmap.cmp_rm_keys_u64_eio 2024-04-07T16:32:28.846 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] CmpOmap.cmp_rm_keys_u64_eio (6 ms) 2024-04-07T16:32:28.847 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] CmpOmap.cmp_rm_keys_at_max_keys 2024-04-07T16:32:28.849 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] CmpOmap.cmp_rm_keys_at_max_keys (3 ms) 2024-04-07T16:32:28.849 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] CmpOmap.cmp_rm_keys_over_max_keys 2024-04-07T16:32:28.851 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] CmpOmap.cmp_rm_keys_over_max_keys (2 ms) 2024-04-07T16:32:28.851 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] CmpOmap.cmp_rm_keys_u64_empty 2024-04-07T16:32:28.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:32:28 smithi144 bash[66654]: cluster 2024-04-07T16:32:27.560327+0000 mon.a (mon.0) 1597 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T16:32:28.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:32:28 smithi144 bash[66654]: cluster 2024-04-07T16:32:27.560396+0000 mon.a (mon.0) 1598 : cluster [INF] Cluster is now healthy 2024-04-07T16:32:28.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:32:28 smithi144 bash[66654]: cluster 2024-04-07T16:32:27.581786+0000 mon.a (mon.0) 1599 : cluster [DBG] osdmap e896: 8 total, 8 up, 8 in 2024-04-07T16:32:28.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:32:28 smithi144 bash[66654]: audit 2024-04-07T16:32:27.583753+0000 mon.a (mon.0) 1600 : audit [INF] from='client.? 172.21.15.100:0/3262394495' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110574-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:32:28.872 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] CmpOmap.cmp_rm_keys_u64_empty (21 ms) 2024-04-07T16:32:28.872 INFO:tasks.workunit.client.0.smithi100.stdout:[----------] 31 tests from CmpOmap (297 ms total) 2024-04-07T16:32:28.872 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T16:32:28.872 INFO:tasks.workunit.client.0.smithi100.stdout:[----------] Global test environment tear-down 2024-04-07T16:32:29.605 INFO:tasks.workunit.client.0.smithi100.stdout:[==========] 31 tests from 1 test suite ran. (2653 ms total) 2024-04-07T16:32:29.605 INFO:tasks.workunit.client.0.smithi100.stdout:[ PASSED ] 31 tests. 2024-04-07T16:32:29.612 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-07T16:32:29.612 DEBUG:teuthology.orchestra.run.smithi100:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-07T16:32:29.664 INFO:tasks.workunit:Running workunit cls/test_cls_hello.sh... 2024-04-07T16:32:29.664 DEBUG:teuthology.orchestra.run.smithi100:workunit test cls/test_cls_hello.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=reef TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_hello.sh 2024-04-07T16:32:29.725 INFO:tasks.workunit.client.0.smithi100.stdout:Running main() from gmock_main.cc 2024-04-07T16:32:29.725 INFO:tasks.workunit.client.0.smithi100.stdout:[==========] Running 6 tests from 1 test suite. 2024-04-07T16:32:29.725 INFO:tasks.workunit.client.0.smithi100.stdout:[----------] Global test environment set-up. 2024-04-07T16:32:29.726 INFO:tasks.workunit.client.0.smithi100.stdout:[----------] 6 tests from ClsHello 2024-04-07T16:32:29.726 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] ClsHello.SayHello 2024-04-07T16:32:29.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:32:29 smithi144 bash[66654]: audit 2024-04-07T16:32:28.575978+0000 mon.a (mon.0) 1601 : audit [INF] from='client.? 172.21.15.100:0/3262394495' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110574-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:32:29.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:32:29 smithi144 bash[66654]: cluster 2024-04-07T16:32:28.584231+0000 mon.a (mon.0) 1602 : cluster [DBG] osdmap e897: 8 total, 8 up, 8 in 2024-04-07T16:32:29.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:32:29 smithi144 bash[66654]: cluster 2024-04-07T16:32:28.915311+0000 mgr.y (mgr.25175) 514 : cluster [DBG] pgmap v986: 137 pgs: 2 creating+peering, 30 unknown, 105 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:32:29.939 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:32:29 smithi100 bash[74942]: audit 2024-04-07T16:32:28.575978+0000 mon.a (mon.0) 1601 : audit [INF] from='client.? 172.21.15.100:0/3262394495' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110574-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:32:29.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:32:29 smithi100 bash[74942]: cluster 2024-04-07T16:32:28.584231+0000 mon.a (mon.0) 1602 : cluster [DBG] osdmap e897: 8 total, 8 up, 8 in 2024-04-07T16:32:29.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:32:29 smithi100 bash[74942]: cluster 2024-04-07T16:32:28.915311+0000 mgr.y (mgr.25175) 514 : cluster [DBG] pgmap v986: 137 pgs: 2 creating+peering, 30 unknown, 105 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:32:29.940 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:32:29 smithi100 bash[76202]: audit 2024-04-07T16:32:28.575978+0000 mon.a (mon.0) 1601 : audit [INF] from='client.? 172.21.15.100:0/3262394495' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110574-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:32:29.940 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:32:29 smithi100 bash[76202]: cluster 2024-04-07T16:32:28.584231+0000 mon.a (mon.0) 1602 : cluster [DBG] osdmap e897: 8 total, 8 up, 8 in 2024-04-07T16:32:29.940 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:32:29 smithi100 bash[76202]: cluster 2024-04-07T16:32:28.915311+0000 mgr.y (mgr.25175) 514 : cluster [DBG] pgmap v986: 137 pgs: 2 creating+peering, 30 unknown, 105 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:32:30.939 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:32:30 smithi100 bash[74942]: cluster 2024-04-07T16:32:29.609768+0000 mon.a (mon.0) 1603 : cluster [DBG] osdmap e898: 8 total, 8 up, 8 in 2024-04-07T16:32:30.940 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:32:30 smithi100 bash[76202]: cluster 2024-04-07T16:32:29.609768+0000 mon.a (mon.0) 1603 : cluster [DBG] osdmap e898: 8 total, 8 up, 8 in 2024-04-07T16:32:31.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:32:30 smithi144 bash[66654]: cluster 2024-04-07T16:32:29.609768+0000 mon.a (mon.0) 1603 : cluster [DBG] osdmap e898: 8 total, 8 up, 8 in 2024-04-07T16:32:31.939 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:32:31 smithi100 bash[76202]: cluster 2024-04-07T16:32:30.614435+0000 mon.a (mon.0) 1604 : cluster [DBG] osdmap e899: 8 total, 8 up, 8 in 2024-04-07T16:32:31.940 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:32:31 smithi100 bash[76202]: audit 2024-04-07T16:32:30.618955+0000 mon.c (mon.2) 36 : audit [INF] from='client.? 172.21.15.100:0/3522708994' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110598-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:32:31.940 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:32:31 smithi100 bash[76202]: audit 2024-04-07T16:32:30.629110+0000 mon.a (mon.0) 1605 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110598-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:32:31.940 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:32:31 smithi100 bash[76202]: cluster 2024-04-07T16:32:30.916172+0000 mgr.y (mgr.25175) 515 : cluster [DBG] pgmap v989: 137 pgs: 32 unknown, 105 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:32:31.940 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:32:31 smithi100 bash[76202]: audit 2024-04-07T16:32:31.606570+0000 mon.a (mon.0) 1606 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110598-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:32:31.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:32:31 smithi100 bash[74942]: cluster 2024-04-07T16:32:30.614435+0000 mon.a (mon.0) 1604 : cluster [DBG] osdmap e899: 8 total, 8 up, 8 in 2024-04-07T16:32:31.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:32:31 smithi100 bash[74942]: audit 2024-04-07T16:32:30.618955+0000 mon.c (mon.2) 36 : audit [INF] from='client.? 172.21.15.100:0/3522708994' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110598-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:32:31.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:32:31 smithi100 bash[74942]: audit 2024-04-07T16:32:30.629110+0000 mon.a (mon.0) 1605 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110598-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:32:31.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:32:31 smithi100 bash[74942]: cluster 2024-04-07T16:32:30.916172+0000 mgr.y (mgr.25175) 515 : cluster [DBG] pgmap v989: 137 pgs: 32 unknown, 105 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:32:31.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:32:31 smithi100 bash[74942]: audit 2024-04-07T16:32:31.606570+0000 mon.a (mon.0) 1606 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110598-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:32:32.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:32:31 smithi144 bash[66654]: cluster 2024-04-07T16:32:30.614435+0000 mon.a (mon.0) 1604 : cluster [DBG] osdmap e899: 8 total, 8 up, 8 in 2024-04-07T16:32:32.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:32:31 smithi144 bash[66654]: audit 2024-04-07T16:32:30.618955+0000 mon.c (mon.2) 36 : audit [INF] from='client.? 172.21.15.100:0/3522708994' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110598-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:32:32.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:32:31 smithi144 bash[66654]: audit 2024-04-07T16:32:30.629110+0000 mon.a (mon.0) 1605 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110598-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:32:32.108 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:32:31 smithi144 bash[66654]: cluster 2024-04-07T16:32:30.916172+0000 mgr.y (mgr.25175) 515 : cluster [DBG] pgmap v989: 137 pgs: 32 unknown, 105 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:32:32.108 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:32:31 smithi144 bash[66654]: audit 2024-04-07T16:32:31.606570+0000 mon.a (mon.0) 1606 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110598-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:32:32.607 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:32:32 smithi144 bash[95233]: ts=2024-04-07T16:32:32.317Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:32:32.611 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] ClsHello.SayHello (2885 ms) 2024-04-07T16:32:32.611 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] ClsHello.RecordHello 2024-04-07T16:32:32.939 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:32:32 smithi100 bash[74942]: cluster 2024-04-07T16:32:31.616882+0000 mon.a (mon.0) 1607 : cluster [DBG] osdmap e900: 8 total, 8 up, 8 in 2024-04-07T16:32:32.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:32:32 smithi100 bash[74942]: cluster 2024-04-07T16:32:31.621347+0000 mon.a (mon.0) 1608 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T16:32:32.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:32:32 smithi100 bash[74942]: audit 2024-04-07T16:32:31.750095+0000 mon.a (mon.0) 1609 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:32:32.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:32:32 smithi100 bash[74942]: audit 2024-04-07T16:32:31.752356+0000 mon.a (mon.0) 1610 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:32:32.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:32:32 smithi100 bash[74942]: cluster 2024-04-07T16:32:32.616915+0000 mon.a (mon.0) 1611 : cluster [DBG] osdmap e901: 8 total, 8 up, 8 in 2024-04-07T16:32:32.940 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:32:32 smithi100 bash[76202]: cluster 2024-04-07T16:32:31.616882+0000 mon.a (mon.0) 1607 : cluster [DBG] osdmap e900: 8 total, 8 up, 8 in 2024-04-07T16:32:32.940 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:32:32 smithi100 bash[76202]: cluster 2024-04-07T16:32:31.621347+0000 mon.a (mon.0) 1608 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T16:32:32.940 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:32:32 smithi100 bash[76202]: audit 2024-04-07T16:32:31.750095+0000 mon.a (mon.0) 1609 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:32:32.940 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:32:32 smithi100 bash[76202]: audit 2024-04-07T16:32:31.752356+0000 mon.a (mon.0) 1610 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:32:32.940 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:32:32 smithi100 bash[76202]: cluster 2024-04-07T16:32:32.616915+0000 mon.a (mon.0) 1611 : cluster [DBG] osdmap e901: 8 total, 8 up, 8 in 2024-04-07T16:32:33.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:32:32 smithi144 bash[66654]: cluster 2024-04-07T16:32:31.616882+0000 mon.a (mon.0) 1607 : cluster [DBG] osdmap e900: 8 total, 8 up, 8 in 2024-04-07T16:32:33.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:32:32 smithi144 bash[66654]: cluster 2024-04-07T16:32:31.621347+0000 mon.a (mon.0) 1608 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T16:32:33.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:32:32 smithi144 bash[66654]: audit 2024-04-07T16:32:31.750095+0000 mon.a (mon.0) 1609 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:32:33.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:32:32 smithi144 bash[66654]: audit 2024-04-07T16:32:31.752356+0000 mon.a (mon.0) 1610 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:32:33.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:32:32 smithi144 bash[66654]: cluster 2024-04-07T16:32:32.616915+0000 mon.a (mon.0) 1611 : cluster [DBG] osdmap e901: 8 total, 8 up, 8 in 2024-04-07T16:32:33.939 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:32:33 smithi100 bash[74942]: cluster 2024-04-07T16:32:32.916893+0000 mgr.y (mgr.25175) 516 : cluster [DBG] pgmap v992: 105 pgs: 105 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:32:33.939 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:32:33 smithi100 bash[76202]: cluster 2024-04-07T16:32:32.916893+0000 mgr.y (mgr.25175) 516 : cluster [DBG] pgmap v992: 105 pgs: 105 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:32:34.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:32:33 smithi144 bash[66654]: cluster 2024-04-07T16:32:32.916893+0000 mgr.y (mgr.25175) 516 : cluster [DBG] pgmap v992: 105 pgs: 105 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:32:34.939 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:32:34 smithi100 bash[74942]: cluster 2024-04-07T16:32:33.638154+0000 mon.a (mon.0) 1612 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T16:32:34.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:32:34 smithi100 bash[74942]: cluster 2024-04-07T16:32:33.638222+0000 mon.a (mon.0) 1613 : cluster [INF] Cluster is now healthy 2024-04-07T16:32:34.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:32:34 smithi100 bash[74942]: cluster 2024-04-07T16:32:33.660969+0000 mon.a (mon.0) 1614 : cluster [DBG] osdmap e902: 8 total, 8 up, 8 in 2024-04-07T16:32:34.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:32:34 smithi100 bash[74942]: audit 2024-04-07T16:32:33.672428+0000 mon.a (mon.0) 1615 : audit [INF] from='client.? 172.21.15.100:0/4147772542' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110598-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:32:34.940 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:32:34 smithi100 bash[76202]: cluster 2024-04-07T16:32:33.638154+0000 mon.a (mon.0) 1612 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T16:32:34.940 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:32:34 smithi100 bash[76202]: cluster 2024-04-07T16:32:33.638222+0000 mon.a (mon.0) 1613 : cluster [INF] Cluster is now healthy 2024-04-07T16:32:34.940 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:32:34 smithi100 bash[76202]: cluster 2024-04-07T16:32:33.660969+0000 mon.a (mon.0) 1614 : cluster [DBG] osdmap e902: 8 total, 8 up, 8 in 2024-04-07T16:32:34.940 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:32:34 smithi100 bash[76202]: audit 2024-04-07T16:32:33.672428+0000 mon.a (mon.0) 1615 : audit [INF] from='client.? 172.21.15.100:0/4147772542' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110598-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:32:35.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:32:34 smithi144 bash[66654]: cluster 2024-04-07T16:32:33.638154+0000 mon.a (mon.0) 1612 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T16:32:35.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:32:34 smithi144 bash[66654]: cluster 2024-04-07T16:32:33.638222+0000 mon.a (mon.0) 1613 : cluster [INF] Cluster is now healthy 2024-04-07T16:32:35.108 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:32:34 smithi144 bash[66654]: cluster 2024-04-07T16:32:33.660969+0000 mon.a (mon.0) 1614 : cluster [DBG] osdmap e902: 8 total, 8 up, 8 in 2024-04-07T16:32:35.108 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:32:34 smithi144 bash[66654]: audit 2024-04-07T16:32:33.672428+0000 mon.a (mon.0) 1615 : audit [INF] from='client.? 172.21.15.100:0/4147772542' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110598-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:32:35.672 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] ClsHello.RecordHello (3061 ms) 2024-04-07T16:32:35.673 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] ClsHello.WriteReturnData 2024-04-07T16:32:35.939 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:32:35 smithi100 bash[74942]: audit 2024-04-07T16:32:34.663495+0000 mon.a (mon.0) 1616 : audit [INF] from='client.? 172.21.15.100:0/4147772542' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110598-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:32:35.939 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:32:35 smithi100 bash[74942]: cluster 2024-04-07T16:32:34.675525+0000 mon.a (mon.0) 1617 : cluster [DBG] osdmap e903: 8 total, 8 up, 8 in 2024-04-07T16:32:35.939 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:32:35 smithi100 bash[74942]: cluster 2024-04-07T16:32:34.917582+0000 mgr.y (mgr.25175) 517 : cluster [DBG] pgmap v995: 137 pgs: 12 creating+peering, 20 unknown, 105 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:32:35.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:32:35 smithi100 bash[74942]: audit 2024-04-07T16:32:35.546634+0000 mon.a (mon.0) 1618 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:32:35.940 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:32:35 smithi100 bash[76202]: audit 2024-04-07T16:32:34.663495+0000 mon.a (mon.0) 1616 : audit [INF] from='client.? 172.21.15.100:0/4147772542' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110598-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:32:35.940 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:32:35 smithi100 bash[76202]: cluster 2024-04-07T16:32:34.675525+0000 mon.a (mon.0) 1617 : cluster [DBG] osdmap e903: 8 total, 8 up, 8 in 2024-04-07T16:32:35.940 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:32:35 smithi100 bash[76202]: cluster 2024-04-07T16:32:34.917582+0000 mgr.y (mgr.25175) 517 : cluster [DBG] pgmap v995: 137 pgs: 12 creating+peering, 20 unknown, 105 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:32:35.940 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:32:35 smithi100 bash[76202]: audit 2024-04-07T16:32:35.546634+0000 mon.a (mon.0) 1618 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:32:36.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:32:35 smithi144 bash[66654]: audit 2024-04-07T16:32:34.663495+0000 mon.a (mon.0) 1616 : audit [INF] from='client.? 172.21.15.100:0/4147772542' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110598-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:32:36.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:32:35 smithi144 bash[66654]: cluster 2024-04-07T16:32:34.675525+0000 mon.a (mon.0) 1617 : cluster [DBG] osdmap e903: 8 total, 8 up, 8 in 2024-04-07T16:32:36.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:32:35 smithi144 bash[66654]: cluster 2024-04-07T16:32:34.917582+0000 mgr.y (mgr.25175) 517 : cluster [DBG] pgmap v995: 137 pgs: 12 creating+peering, 20 unknown, 105 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:32:36.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:32:35 smithi144 bash[66654]: audit 2024-04-07T16:32:35.546634+0000 mon.a (mon.0) 1618 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:32:36.939 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:32:36 smithi100 bash[74942]: cluster 2024-04-07T16:32:35.681492+0000 mon.a (mon.0) 1619 : cluster [DBG] osdmap e904: 8 total, 8 up, 8 in 2024-04-07T16:32:36.939 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:32:36 smithi100 bash[74942]: audit 2024-04-07T16:32:36.336858+0000 mon.a (mon.0) 1620 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:32:36.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:32:36 smithi100 bash[74942]: audit 2024-04-07T16:32:36.338835+0000 mon.a (mon.0) 1621 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:32:36.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:32:36 smithi100 bash[74942]: audit 2024-04-07T16:32:36.351073+0000 mon.a (mon.0) 1622 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:32:36.940 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:32:36 smithi100 bash[76202]: cluster 2024-04-07T16:32:35.681492+0000 mon.a (mon.0) 1619 : cluster [DBG] osdmap e904: 8 total, 8 up, 8 in 2024-04-07T16:32:36.940 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:32:36 smithi100 bash[76202]: audit 2024-04-07T16:32:36.336858+0000 mon.a (mon.0) 1620 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:32:36.940 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:32:36 smithi100 bash[76202]: audit 2024-04-07T16:32:36.338835+0000 mon.a (mon.0) 1621 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:32:36.940 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:32:36 smithi100 bash[76202]: audit 2024-04-07T16:32:36.351073+0000 mon.a (mon.0) 1622 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:32:37.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:32:36 smithi144 bash[66654]: cluster 2024-04-07T16:32:35.681492+0000 mon.a (mon.0) 1619 : cluster [DBG] osdmap e904: 8 total, 8 up, 8 in 2024-04-07T16:32:37.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:32:36 smithi144 bash[66654]: audit 2024-04-07T16:32:36.336858+0000 mon.a (mon.0) 1620 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:32:37.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:32:36 smithi144 bash[66654]: audit 2024-04-07T16:32:36.338835+0000 mon.a (mon.0) 1621 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:32:37.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:32:36 smithi144 bash[66654]: audit 2024-04-07T16:32:36.351073+0000 mon.a (mon.0) 1622 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:32:37.702 INFO:tasks.workunit.client.0.smithi100.stdout:require_osd_release = squid 2024-04-07T16:32:37.725 INFO:tasks.workunit.client.0.smithi100.stdout:0 2024-04-07T16:32:37.939 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:32:37 smithi100 bash[74942]: cluster 2024-04-07T16:32:36.695557+0000 mon.a (mon.0) 1623 : cluster [DBG] osdmap e905: 8 total, 8 up, 8 in 2024-04-07T16:32:37.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:32:37 smithi100 bash[74942]: audit 2024-04-07T16:32:36.698669+0000 mon.c (mon.2) 37 : audit [INF] from='client.? 172.21.15.100:0/2668902816' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110598-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:32:37.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:32:37 smithi100 bash[74942]: audit 2024-04-07T16:32:36.699976+0000 mon.a (mon.0) 1624 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110598-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:32:37.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:32:37 smithi100 bash[74942]: cluster 2024-04-07T16:32:36.918318+0000 mgr.y (mgr.25175) 518 : cluster [DBG] pgmap v998: 137 pgs: 32 unknown, 105 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:32:37.940 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:32:37 smithi100 bash[76202]: cluster 2024-04-07T16:32:36.695557+0000 mon.a (mon.0) 1623 : cluster [DBG] osdmap e905: 8 total, 8 up, 8 in 2024-04-07T16:32:37.940 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:32:37 smithi100 bash[76202]: audit 2024-04-07T16:32:36.698669+0000 mon.c (mon.2) 37 : audit [INF] from='client.? 172.21.15.100:0/2668902816' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110598-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:32:37.940 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:32:37 smithi100 bash[76202]: audit 2024-04-07T16:32:36.699976+0000 mon.a (mon.0) 1624 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110598-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:32:37.940 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:32:37 smithi100 bash[76202]: cluster 2024-04-07T16:32:36.918318+0000 mgr.y (mgr.25175) 518 : cluster [DBG] pgmap v998: 137 pgs: 32 unknown, 105 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:32:38.095 INFO:tasks.workunit.client.0.smithi100.stdout:1 2024-04-07T16:32:38.097 INFO:tasks.workunit.client.0.smithi100.stdout:2 2024-04-07T16:32:38.100 INFO:tasks.workunit.client.0.smithi100.stdout:3 2024-04-07T16:32:38.102 INFO:tasks.workunit.client.0.smithi100.stdout:4 2024-04-07T16:32:38.105 INFO:tasks.workunit.client.0.smithi100.stdout:5 2024-04-07T16:32:38.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:32:37 smithi144 bash[66654]: cluster 2024-04-07T16:32:36.695557+0000 mon.a (mon.0) 1623 : cluster [DBG] osdmap e905: 8 total, 8 up, 8 in 2024-04-07T16:32:38.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:32:37 smithi144 bash[66654]: audit 2024-04-07T16:32:36.698669+0000 mon.c (mon.2) 37 : audit [INF] from='client.? 172.21.15.100:0/2668902816' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110598-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:32:38.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:32:37 smithi144 bash[66654]: audit 2024-04-07T16:32:36.699976+0000 mon.a (mon.0) 1624 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110598-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:32:38.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:32:37 smithi144 bash[66654]: cluster 2024-04-07T16:32:36.918318+0000 mgr.y (mgr.25175) 518 : cluster [DBG] pgmap v998: 137 pgs: 32 unknown, 105 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:32:38.107 INFO:tasks.workunit.client.0.smithi100.stdout:6 2024-04-07T16:32:38.110 INFO:tasks.workunit.client.0.smithi100.stdout:7 2024-04-07T16:32:38.112 INFO:tasks.workunit.client.0.smithi100.stdout:8 2024-04-07T16:32:38.114 INFO:tasks.workunit.client.0.smithi100.stdout:9 2024-04-07T16:32:38.117 INFO:tasks.workunit.client.0.smithi100.stdout:10 2024-04-07T16:32:38.119 INFO:tasks.workunit.client.0.smithi100.stdout:11 2024-04-07T16:32:38.121 INFO:tasks.workunit.client.0.smithi100.stdout:12 2024-04-07T16:32:38.124 INFO:tasks.workunit.client.0.smithi100.stdout:13 2024-04-07T16:32:38.126 INFO:tasks.workunit.client.0.smithi100.stdout:14 2024-04-07T16:32:38.128 INFO:tasks.workunit.client.0.smithi100.stdout:15 2024-04-07T16:32:38.131 INFO:tasks.workunit.client.0.smithi100.stdout:16 2024-04-07T16:32:38.133 INFO:tasks.workunit.client.0.smithi100.stdout:17 2024-04-07T16:32:38.135 INFO:tasks.workunit.client.0.smithi100.stdout:18 2024-04-07T16:32:38.137 INFO:tasks.workunit.client.0.smithi100.stdout:19 2024-04-07T16:32:38.140 INFO:tasks.workunit.client.0.smithi100.stdout:20 2024-04-07T16:32:38.142 INFO:tasks.workunit.client.0.smithi100.stdout:21 2024-04-07T16:32:38.154 INFO:tasks.workunit.client.0.smithi100.stdout:22 2024-04-07T16:32:38.157 INFO:tasks.workunit.client.0.smithi100.stdout:23 2024-04-07T16:32:38.159 INFO:tasks.workunit.client.0.smithi100.stdout:24 2024-04-07T16:32:38.162 INFO:tasks.workunit.client.0.smithi100.stdout:25 2024-04-07T16:32:38.164 INFO:tasks.workunit.client.0.smithi100.stdout:26 2024-04-07T16:32:38.167 INFO:tasks.workunit.client.0.smithi100.stdout:27 2024-04-07T16:32:38.170 INFO:tasks.workunit.client.0.smithi100.stdout:28 2024-04-07T16:32:38.172 INFO:tasks.workunit.client.0.smithi100.stdout:29 2024-04-07T16:32:38.174 INFO:tasks.workunit.client.0.smithi100.stdout:30 2024-04-07T16:32:38.177 INFO:tasks.workunit.client.0.smithi100.stdout:31 2024-04-07T16:32:38.179 INFO:tasks.workunit.client.0.smithi100.stdout:32 2024-04-07T16:32:38.181 INFO:tasks.workunit.client.0.smithi100.stdout:33 2024-04-07T16:32:38.184 INFO:tasks.workunit.client.0.smithi100.stdout:34 2024-04-07T16:32:38.186 INFO:tasks.workunit.client.0.smithi100.stdout:35 2024-04-07T16:32:38.188 INFO:tasks.workunit.client.0.smithi100.stdout:36 2024-04-07T16:32:38.191 INFO:tasks.workunit.client.0.smithi100.stdout:37 2024-04-07T16:32:38.193 INFO:tasks.workunit.client.0.smithi100.stdout:38 2024-04-07T16:32:38.195 INFO:tasks.workunit.client.0.smithi100.stdout:39 2024-04-07T16:32:38.197 INFO:tasks.workunit.client.0.smithi100.stdout:40 2024-04-07T16:32:38.200 INFO:tasks.workunit.client.0.smithi100.stdout:41 2024-04-07T16:32:38.202 INFO:tasks.workunit.client.0.smithi100.stdout:42 2024-04-07T16:32:38.204 INFO:tasks.workunit.client.0.smithi100.stdout:43 2024-04-07T16:32:38.207 INFO:tasks.workunit.client.0.smithi100.stdout:44 2024-04-07T16:32:38.209 INFO:tasks.workunit.client.0.smithi100.stdout:45 2024-04-07T16:32:38.211 INFO:tasks.workunit.client.0.smithi100.stdout:46 2024-04-07T16:32:38.214 INFO:tasks.workunit.client.0.smithi100.stdout:47 2024-04-07T16:32:38.216 INFO:tasks.workunit.client.0.smithi100.stdout:48 2024-04-07T16:32:38.219 INFO:tasks.workunit.client.0.smithi100.stdout:49 2024-04-07T16:32:38.221 INFO:tasks.workunit.client.0.smithi100.stdout:50 2024-04-07T16:32:38.223 INFO:tasks.workunit.client.0.smithi100.stdout:51 2024-04-07T16:32:38.226 INFO:tasks.workunit.client.0.smithi100.stdout:52 2024-04-07T16:32:38.228 INFO:tasks.workunit.client.0.smithi100.stdout:53 2024-04-07T16:32:38.230 INFO:tasks.workunit.client.0.smithi100.stdout:54 2024-04-07T16:32:38.233 INFO:tasks.workunit.client.0.smithi100.stdout:55 2024-04-07T16:32:38.235 INFO:tasks.workunit.client.0.smithi100.stdout:56 2024-04-07T16:32:38.237 INFO:tasks.workunit.client.0.smithi100.stdout:57 2024-04-07T16:32:38.240 INFO:tasks.workunit.client.0.smithi100.stdout:58 2024-04-07T16:32:38.242 INFO:tasks.workunit.client.0.smithi100.stdout:59 2024-04-07T16:32:38.244 INFO:tasks.workunit.client.0.smithi100.stdout:60 2024-04-07T16:32:38.247 INFO:tasks.workunit.client.0.smithi100.stdout:61 2024-04-07T16:32:38.249 INFO:tasks.workunit.client.0.smithi100.stdout:62 2024-04-07T16:32:38.252 INFO:tasks.workunit.client.0.smithi100.stdout:63 2024-04-07T16:32:38.254 INFO:tasks.workunit.client.0.smithi100.stdout:64 2024-04-07T16:32:38.257 INFO:tasks.workunit.client.0.smithi100.stdout:65 2024-04-07T16:32:38.259 INFO:tasks.workunit.client.0.smithi100.stdout:66 2024-04-07T16:32:38.262 INFO:tasks.workunit.client.0.smithi100.stdout:67 2024-04-07T16:32:38.265 INFO:tasks.workunit.client.0.smithi100.stdout:68 2024-04-07T16:32:38.267 INFO:tasks.workunit.client.0.smithi100.stdout:69 2024-04-07T16:32:38.269 INFO:tasks.workunit.client.0.smithi100.stdout:70 2024-04-07T16:32:38.272 INFO:tasks.workunit.client.0.smithi100.stdout:71 2024-04-07T16:32:38.274 INFO:tasks.workunit.client.0.smithi100.stdout:72 2024-04-07T16:32:38.277 INFO:tasks.workunit.client.0.smithi100.stdout:73 2024-04-07T16:32:38.279 INFO:tasks.workunit.client.0.smithi100.stdout:74 2024-04-07T16:32:38.281 INFO:tasks.workunit.client.0.smithi100.stdout:75 2024-04-07T16:32:38.284 INFO:tasks.workunit.client.0.smithi100.stdout:76 2024-04-07T16:32:38.286 INFO:tasks.workunit.client.0.smithi100.stdout:77 2024-04-07T16:32:38.288 INFO:tasks.workunit.client.0.smithi100.stdout:78 2024-04-07T16:32:38.291 INFO:tasks.workunit.client.0.smithi100.stdout:79 2024-04-07T16:32:38.294 INFO:tasks.workunit.client.0.smithi100.stdout:80 2024-04-07T16:32:38.296 INFO:tasks.workunit.client.0.smithi100.stdout:81 2024-04-07T16:32:38.299 INFO:tasks.workunit.client.0.smithi100.stdout:82 2024-04-07T16:32:38.301 INFO:tasks.workunit.client.0.smithi100.stdout:83 2024-04-07T16:32:38.304 INFO:tasks.workunit.client.0.smithi100.stdout:84 2024-04-07T16:32:38.306 INFO:tasks.workunit.client.0.smithi100.stdout:85 2024-04-07T16:32:38.309 INFO:tasks.workunit.client.0.smithi100.stdout:86 2024-04-07T16:32:38.311 INFO:tasks.workunit.client.0.smithi100.stdout:87 2024-04-07T16:32:38.314 INFO:tasks.workunit.client.0.smithi100.stdout:88 2024-04-07T16:32:38.317 INFO:tasks.workunit.client.0.smithi100.stdout:89 2024-04-07T16:32:38.319 INFO:tasks.workunit.client.0.smithi100.stdout:90 2024-04-07T16:32:38.321 INFO:tasks.workunit.client.0.smithi100.stdout:91 2024-04-07T16:32:38.324 INFO:tasks.workunit.client.0.smithi100.stdout:92 2024-04-07T16:32:38.327 INFO:tasks.workunit.client.0.smithi100.stdout:93 2024-04-07T16:32:38.329 INFO:tasks.workunit.client.0.smithi100.stdout:94 2024-04-07T16:32:38.331 INFO:tasks.workunit.client.0.smithi100.stdout:95 2024-04-07T16:32:38.334 INFO:tasks.workunit.client.0.smithi100.stdout:96 2024-04-07T16:32:38.337 INFO:tasks.workunit.client.0.smithi100.stdout:97 2024-04-07T16:32:38.339 INFO:tasks.workunit.client.0.smithi100.stdout:98 2024-04-07T16:32:38.342 INFO:tasks.workunit.client.0.smithi100.stdout:99 2024-04-07T16:32:38.344 INFO:tasks.workunit.client.0.smithi100.stdout:100 2024-04-07T16:32:38.347 INFO:tasks.workunit.client.0.smithi100.stdout:101 2024-04-07T16:32:38.350 INFO:tasks.workunit.client.0.smithi100.stdout:102 2024-04-07T16:32:38.353 INFO:tasks.workunit.client.0.smithi100.stdout:103 2024-04-07T16:32:38.356 INFO:tasks.workunit.client.0.smithi100.stdout:104 2024-04-07T16:32:38.359 INFO:tasks.workunit.client.0.smithi100.stdout:105 2024-04-07T16:32:38.362 INFO:tasks.workunit.client.0.smithi100.stdout:106 2024-04-07T16:32:38.365 INFO:tasks.workunit.client.0.smithi100.stdout:107 2024-04-07T16:32:38.370 INFO:tasks.workunit.client.0.smithi100.stdout:108 2024-04-07T16:32:38.370 INFO:tasks.workunit.client.0.smithi100.stdout:109 2024-04-07T16:32:38.372 INFO:tasks.workunit.client.0.smithi100.stdout:110 2024-04-07T16:32:38.375 INFO:tasks.workunit.client.0.smithi100.stdout:111 2024-04-07T16:32:38.377 INFO:tasks.workunit.client.0.smithi100.stdout:112 2024-04-07T16:32:38.380 INFO:tasks.workunit.client.0.smithi100.stdout:113 2024-04-07T16:32:38.382 INFO:tasks.workunit.client.0.smithi100.stdout:114 2024-04-07T16:32:38.385 INFO:tasks.workunit.client.0.smithi100.stdout:115 2024-04-07T16:32:38.388 INFO:tasks.workunit.client.0.smithi100.stdout:116 2024-04-07T16:32:38.391 INFO:tasks.workunit.client.0.smithi100.stdout:117 2024-04-07T16:32:38.394 INFO:tasks.workunit.client.0.smithi100.stdout:118 2024-04-07T16:32:38.397 INFO:tasks.workunit.client.0.smithi100.stdout:119 2024-04-07T16:32:38.399 INFO:tasks.workunit.client.0.smithi100.stdout:120 2024-04-07T16:32:38.402 INFO:tasks.workunit.client.0.smithi100.stdout:121 2024-04-07T16:32:38.404 INFO:tasks.workunit.client.0.smithi100.stdout:122 2024-04-07T16:32:38.407 INFO:tasks.workunit.client.0.smithi100.stdout:123 2024-04-07T16:32:38.409 INFO:tasks.workunit.client.0.smithi100.stdout:124 2024-04-07T16:32:38.412 INFO:tasks.workunit.client.0.smithi100.stdout:125 2024-04-07T16:32:38.414 INFO:tasks.workunit.client.0.smithi100.stdout:126 2024-04-07T16:32:38.417 INFO:tasks.workunit.client.0.smithi100.stdout:127 2024-04-07T16:32:38.424 INFO:tasks.workunit.client.0.smithi100.stdout:128 2024-04-07T16:32:38.426 INFO:tasks.workunit.client.0.smithi100.stdout:129 2024-04-07T16:32:38.430 INFO:tasks.workunit.client.0.smithi100.stdout:130 2024-04-07T16:32:38.432 INFO:tasks.workunit.client.0.smithi100.stdout:131 2024-04-07T16:32:38.435 INFO:tasks.workunit.client.0.smithi100.stdout:132 2024-04-07T16:32:38.437 INFO:tasks.workunit.client.0.smithi100.stdout:133 2024-04-07T16:32:38.440 INFO:tasks.workunit.client.0.smithi100.stdout:134 2024-04-07T16:32:38.443 INFO:tasks.workunit.client.0.smithi100.stdout:135 2024-04-07T16:32:38.446 INFO:tasks.workunit.client.0.smithi100.stdout:136 2024-04-07T16:32:38.448 INFO:tasks.workunit.client.0.smithi100.stdout:137 2024-04-07T16:32:38.452 INFO:tasks.workunit.client.0.smithi100.stdout:138 2024-04-07T16:32:38.454 INFO:tasks.workunit.client.0.smithi100.stdout:139 2024-04-07T16:32:38.457 INFO:tasks.workunit.client.0.smithi100.stdout:140 2024-04-07T16:32:38.460 INFO:tasks.workunit.client.0.smithi100.stdout:141 2024-04-07T16:32:38.463 INFO:tasks.workunit.client.0.smithi100.stdout:142 2024-04-07T16:32:38.466 INFO:tasks.workunit.client.0.smithi100.stdout:143 2024-04-07T16:32:38.469 INFO:tasks.workunit.client.0.smithi100.stdout:144 2024-04-07T16:32:38.472 INFO:tasks.workunit.client.0.smithi100.stdout:145 2024-04-07T16:32:38.475 INFO:tasks.workunit.client.0.smithi100.stdout:146 2024-04-07T16:32:38.478 INFO:tasks.workunit.client.0.smithi100.stdout:147 2024-04-07T16:32:38.481 INFO:tasks.workunit.client.0.smithi100.stdout:148 2024-04-07T16:32:38.483 INFO:tasks.workunit.client.0.smithi100.stdout:149 2024-04-07T16:32:38.486 INFO:tasks.workunit.client.0.smithi100.stdout:150 2024-04-07T16:32:38.489 INFO:tasks.workunit.client.0.smithi100.stdout:151 2024-04-07T16:32:38.492 INFO:tasks.workunit.client.0.smithi100.stdout:152 2024-04-07T16:32:38.495 INFO:tasks.workunit.client.0.smithi100.stdout:153 2024-04-07T16:32:38.497 INFO:tasks.workunit.client.0.smithi100.stdout:154 2024-04-07T16:32:38.501 INFO:tasks.workunit.client.0.smithi100.stdout:155 2024-04-07T16:32:38.503 INFO:tasks.workunit.client.0.smithi100.stdout:156 2024-04-07T16:32:38.506 INFO:tasks.workunit.client.0.smithi100.stdout:157 2024-04-07T16:32:38.509 INFO:tasks.workunit.client.0.smithi100.stdout:158 2024-04-07T16:32:38.512 INFO:tasks.workunit.client.0.smithi100.stdout:159 2024-04-07T16:32:38.514 INFO:tasks.workunit.client.0.smithi100.stdout:160 2024-04-07T16:32:38.517 INFO:tasks.workunit.client.0.smithi100.stdout:161 2024-04-07T16:32:38.519 INFO:tasks.workunit.client.0.smithi100.stdout:162 2024-04-07T16:32:38.522 INFO:tasks.workunit.client.0.smithi100.stdout:163 2024-04-07T16:32:38.525 INFO:tasks.workunit.client.0.smithi100.stdout:164 2024-04-07T16:32:38.528 INFO:tasks.workunit.client.0.smithi100.stdout:165 2024-04-07T16:32:38.530 INFO:tasks.workunit.client.0.smithi100.stdout:166 2024-04-07T16:32:38.533 INFO:tasks.workunit.client.0.smithi100.stdout:167 2024-04-07T16:32:38.536 INFO:tasks.workunit.client.0.smithi100.stdout:168 2024-04-07T16:32:38.538 INFO:tasks.workunit.client.0.smithi100.stdout:169 2024-04-07T16:32:38.541 INFO:tasks.workunit.client.0.smithi100.stdout:170 2024-04-07T16:32:38.544 INFO:tasks.workunit.client.0.smithi100.stdout:171 2024-04-07T16:32:38.546 INFO:tasks.workunit.client.0.smithi100.stdout:172 2024-04-07T16:32:38.549 INFO:tasks.workunit.client.0.smithi100.stdout:173 2024-04-07T16:32:38.551 INFO:tasks.workunit.client.0.smithi100.stdout:174 2024-04-07T16:32:38.554 INFO:tasks.workunit.client.0.smithi100.stdout:175 2024-04-07T16:32:38.557 INFO:tasks.workunit.client.0.smithi100.stdout:176 2024-04-07T16:32:38.559 INFO:tasks.workunit.client.0.smithi100.stdout:177 2024-04-07T16:32:38.562 INFO:tasks.workunit.client.0.smithi100.stdout:178 2024-04-07T16:32:38.565 INFO:tasks.workunit.client.0.smithi100.stdout:179 2024-04-07T16:32:38.567 INFO:tasks.workunit.client.0.smithi100.stdout:180 2024-04-07T16:32:38.570 INFO:tasks.workunit.client.0.smithi100.stdout:181 2024-04-07T16:32:38.573 INFO:tasks.workunit.client.0.smithi100.stdout:182 2024-04-07T16:32:38.575 INFO:tasks.workunit.client.0.smithi100.stdout:183 2024-04-07T16:32:38.578 INFO:tasks.workunit.client.0.smithi100.stdout:184 2024-04-07T16:32:38.580 INFO:tasks.workunit.client.0.smithi100.stdout:185 2024-04-07T16:32:38.583 INFO:tasks.workunit.client.0.smithi100.stdout:186 2024-04-07T16:32:38.585 INFO:tasks.workunit.client.0.smithi100.stdout:187 2024-04-07T16:32:38.588 INFO:tasks.workunit.client.0.smithi100.stdout:188 2024-04-07T16:32:38.590 INFO:tasks.workunit.client.0.smithi100.stdout:189 2024-04-07T16:32:38.593 INFO:tasks.workunit.client.0.smithi100.stdout:190 2024-04-07T16:32:38.596 INFO:tasks.workunit.client.0.smithi100.stdout:191 2024-04-07T16:32:38.598 INFO:tasks.workunit.client.0.smithi100.stdout:192 2024-04-07T16:32:38.601 INFO:tasks.workunit.client.0.smithi100.stdout:193 2024-04-07T16:32:38.603 INFO:tasks.workunit.client.0.smithi100.stdout:194 2024-04-07T16:32:38.606 INFO:tasks.workunit.client.0.smithi100.stdout:195 2024-04-07T16:32:38.609 INFO:tasks.workunit.client.0.smithi100.stdout:196 2024-04-07T16:32:38.611 INFO:tasks.workunit.client.0.smithi100.stdout:197 2024-04-07T16:32:38.614 INFO:tasks.workunit.client.0.smithi100.stdout:198 2024-04-07T16:32:38.616 INFO:tasks.workunit.client.0.smithi100.stdout:199 2024-04-07T16:32:38.619 INFO:tasks.workunit.client.0.smithi100.stdout:200 2024-04-07T16:32:38.621 INFO:tasks.workunit.client.0.smithi100.stdout:201 2024-04-07T16:32:38.624 INFO:tasks.workunit.client.0.smithi100.stdout:202 2024-04-07T16:32:38.626 INFO:tasks.workunit.client.0.smithi100.stdout:203 2024-04-07T16:32:38.629 INFO:tasks.workunit.client.0.smithi100.stdout:204 2024-04-07T16:32:38.632 INFO:tasks.workunit.client.0.smithi100.stdout:205 2024-04-07T16:32:38.634 INFO:tasks.workunit.client.0.smithi100.stdout:206 2024-04-07T16:32:38.637 INFO:tasks.workunit.client.0.smithi100.stdout:207 2024-04-07T16:32:38.640 INFO:tasks.workunit.client.0.smithi100.stdout:208 2024-04-07T16:32:38.642 INFO:tasks.workunit.client.0.smithi100.stdout:209 2024-04-07T16:32:38.645 INFO:tasks.workunit.client.0.smithi100.stdout:210 2024-04-07T16:32:38.648 INFO:tasks.workunit.client.0.smithi100.stdout:211 2024-04-07T16:32:38.651 INFO:tasks.workunit.client.0.smithi100.stdout:212 2024-04-07T16:32:38.654 INFO:tasks.workunit.client.0.smithi100.stdout:213 2024-04-07T16:32:38.656 INFO:tasks.workunit.client.0.smithi100.stdout:214 2024-04-07T16:32:38.659 INFO:tasks.workunit.client.0.smithi100.stdout:215 2024-04-07T16:32:38.661 INFO:tasks.workunit.client.0.smithi100.stdout:216 2024-04-07T16:32:38.664 INFO:tasks.workunit.client.0.smithi100.stdout:217 2024-04-07T16:32:38.667 INFO:tasks.workunit.client.0.smithi100.stdout:218 2024-04-07T16:32:38.669 INFO:tasks.workunit.client.0.smithi100.stdout:219 2024-04-07T16:32:38.672 INFO:tasks.workunit.client.0.smithi100.stdout:220 2024-04-07T16:32:38.675 INFO:tasks.workunit.client.0.smithi100.stdout:221 2024-04-07T16:32:38.678 INFO:tasks.workunit.client.0.smithi100.stdout:222 2024-04-07T16:32:38.679 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:32:38 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:32:38] "GET /metrics HTTP/1.1" 200 36357 "" "Prometheus/2.43.0" 2024-04-07T16:32:38.683 INFO:tasks.workunit.client.0.smithi100.stdout:223 2024-04-07T16:32:38.686 INFO:tasks.workunit.client.0.smithi100.stdout:224 2024-04-07T16:32:38.689 INFO:tasks.workunit.client.0.smithi100.stdout:225 2024-04-07T16:32:38.699 INFO:tasks.workunit.client.0.smithi100.stdout:226 2024-04-07T16:32:38.702 INFO:tasks.workunit.client.0.smithi100.stdout:227 2024-04-07T16:32:38.704 INFO:tasks.workunit.client.0.smithi100.stdout:228 2024-04-07T16:32:38.707 INFO:tasks.workunit.client.0.smithi100.stdout:229 2024-04-07T16:32:38.709 INFO:tasks.workunit.client.0.smithi100.stdout:230 2024-04-07T16:32:38.712 INFO:tasks.workunit.client.0.smithi100.stdout:231 2024-04-07T16:32:38.715 INFO:tasks.workunit.client.0.smithi100.stdout:232 2024-04-07T16:32:38.717 INFO:tasks.workunit.client.0.smithi100.stdout:233 2024-04-07T16:32:38.720 INFO:tasks.workunit.client.0.smithi100.stdout:234 2024-04-07T16:32:38.722 INFO:tasks.workunit.client.0.smithi100.stdout:235 2024-04-07T16:32:38.725 INFO:tasks.workunit.client.0.smithi100.stdout:236 2024-04-07T16:32:38.728 INFO:tasks.workunit.client.0.smithi100.stdout:237 2024-04-07T16:32:38.730 INFO:tasks.workunit.client.0.smithi100.stdout:238 2024-04-07T16:32:38.733 INFO:tasks.workunit.client.0.smithi100.stdout:239 2024-04-07T16:32:38.735 INFO:tasks.workunit.client.0.smithi100.stdout:240 2024-04-07T16:32:38.738 INFO:tasks.workunit.client.0.smithi100.stdout:241 2024-04-07T16:32:38.741 INFO:tasks.workunit.client.0.smithi100.stdout:242 2024-04-07T16:32:38.743 INFO:tasks.workunit.client.0.smithi100.stdout:243 2024-04-07T16:32:38.746 INFO:tasks.workunit.client.0.smithi100.stdout:244 2024-04-07T16:32:38.748 INFO:tasks.workunit.client.0.smithi100.stdout:245 2024-04-07T16:32:38.751 INFO:tasks.workunit.client.0.smithi100.stdout:246 2024-04-07T16:32:38.754 INFO:tasks.workunit.client.0.smithi100.stdout:247 2024-04-07T16:32:38.757 INFO:tasks.workunit.client.0.smithi100.stdout:248 2024-04-07T16:32:38.760 INFO:tasks.workunit.client.0.smithi100.stdout:249 2024-04-07T16:32:38.763 INFO:tasks.workunit.client.0.smithi100.stdout:250 2024-04-07T16:32:38.766 INFO:tasks.workunit.client.0.smithi100.stdout:251 2024-04-07T16:32:38.769 INFO:tasks.workunit.client.0.smithi100.stdout:252 2024-04-07T16:32:38.772 INFO:tasks.workunit.client.0.smithi100.stdout:253 2024-04-07T16:32:38.775 INFO:tasks.workunit.client.0.smithi100.stdout:254 2024-04-07T16:32:38.778 INFO:tasks.workunit.client.0.smithi100.stdout:255 2024-04-07T16:32:38.781 INFO:tasks.workunit.client.0.smithi100.stdout:256 2024-04-07T16:32:38.784 INFO:tasks.workunit.client.0.smithi100.stdout:257 2024-04-07T16:32:38.787 INFO:tasks.workunit.client.0.smithi100.stdout:258 2024-04-07T16:32:38.789 INFO:tasks.workunit.client.0.smithi100.stdout:259 2024-04-07T16:32:38.792 INFO:tasks.workunit.client.0.smithi100.stdout:260 2024-04-07T16:32:38.795 INFO:tasks.workunit.client.0.smithi100.stdout:261 2024-04-07T16:32:38.798 INFO:tasks.workunit.client.0.smithi100.stdout:262 2024-04-07T16:32:38.800 INFO:tasks.workunit.client.0.smithi100.stdout:263 2024-04-07T16:32:38.803 INFO:tasks.workunit.client.0.smithi100.stdout:264 2024-04-07T16:32:38.806 INFO:tasks.workunit.client.0.smithi100.stdout:265 2024-04-07T16:32:38.808 INFO:tasks.workunit.client.0.smithi100.stdout:266 2024-04-07T16:32:38.811 INFO:tasks.workunit.client.0.smithi100.stdout:267 2024-04-07T16:32:38.813 INFO:tasks.workunit.client.0.smithi100.stdout:268 2024-04-07T16:32:38.816 INFO:tasks.workunit.client.0.smithi100.stdout:269 2024-04-07T16:32:38.819 INFO:tasks.workunit.client.0.smithi100.stdout:270 2024-04-07T16:32:38.822 INFO:tasks.workunit.client.0.smithi100.stdout:271 2024-04-07T16:32:38.824 INFO:tasks.workunit.client.0.smithi100.stdout:272 2024-04-07T16:32:38.827 INFO:tasks.workunit.client.0.smithi100.stdout:273 2024-04-07T16:32:38.830 INFO:tasks.workunit.client.0.smithi100.stdout:274 2024-04-07T16:32:38.832 INFO:tasks.workunit.client.0.smithi100.stdout:275 2024-04-07T16:32:38.835 INFO:tasks.workunit.client.0.smithi100.stdout:276 2024-04-07T16:32:38.838 INFO:tasks.workunit.client.0.smithi100.stdout:277 2024-04-07T16:32:38.841 INFO:tasks.workunit.client.0.smithi100.stdout:278 2024-04-07T16:32:38.844 INFO:tasks.workunit.client.0.smithi100.stdout:279 2024-04-07T16:32:38.846 INFO:tasks.workunit.client.0.smithi100.stdout:280 2024-04-07T16:32:38.849 INFO:tasks.workunit.client.0.smithi100.stdout:281 2024-04-07T16:32:38.852 INFO:tasks.workunit.client.0.smithi100.stdout:282 2024-04-07T16:32:38.855 INFO:tasks.workunit.client.0.smithi100.stdout:283 2024-04-07T16:32:38.857 INFO:tasks.workunit.client.0.smithi100.stdout:284 2024-04-07T16:32:38.860 INFO:tasks.workunit.client.0.smithi100.stdout:285 2024-04-07T16:32:38.863 INFO:tasks.workunit.client.0.smithi100.stdout:286 2024-04-07T16:32:38.866 INFO:tasks.workunit.client.0.smithi100.stdout:287 2024-04-07T16:32:38.868 INFO:tasks.workunit.client.0.smithi100.stdout:288 2024-04-07T16:32:38.871 INFO:tasks.workunit.client.0.smithi100.stdout:289 2024-04-07T16:32:38.874 INFO:tasks.workunit.client.0.smithi100.stdout:290 2024-04-07T16:32:38.876 INFO:tasks.workunit.client.0.smithi100.stdout:291 2024-04-07T16:32:38.879 INFO:tasks.workunit.client.0.smithi100.stdout:292 2024-04-07T16:32:38.882 INFO:tasks.workunit.client.0.smithi100.stdout:293 2024-04-07T16:32:38.885 INFO:tasks.workunit.client.0.smithi100.stdout:294 2024-04-07T16:32:38.887 INFO:tasks.workunit.client.0.smithi100.stdout:295 2024-04-07T16:32:38.890 INFO:tasks.workunit.client.0.smithi100.stdout:296 2024-04-07T16:32:38.893 INFO:tasks.workunit.client.0.smithi100.stdout:297 2024-04-07T16:32:38.896 INFO:tasks.workunit.client.0.smithi100.stdout:298 2024-04-07T16:32:38.900 INFO:tasks.workunit.client.0.smithi100.stdout:299 2024-04-07T16:32:38.902 INFO:tasks.workunit.client.0.smithi100.stdout:300 2024-04-07T16:32:38.905 INFO:tasks.workunit.client.0.smithi100.stdout:301 2024-04-07T16:32:38.908 INFO:tasks.workunit.client.0.smithi100.stdout:302 2024-04-07T16:32:38.910 INFO:tasks.workunit.client.0.smithi100.stdout:303 2024-04-07T16:32:38.914 INFO:tasks.workunit.client.0.smithi100.stdout:304 2024-04-07T16:32:38.917 INFO:tasks.workunit.client.0.smithi100.stdout:305 2024-04-07T16:32:38.920 INFO:tasks.workunit.client.0.smithi100.stdout:306 2024-04-07T16:32:38.923 INFO:tasks.workunit.client.0.smithi100.stdout:307 2024-04-07T16:32:38.926 INFO:tasks.workunit.client.0.smithi100.stdout:308 2024-04-07T16:32:38.928 INFO:tasks.workunit.client.0.smithi100.stdout:309 2024-04-07T16:32:38.932 INFO:tasks.workunit.client.0.smithi100.stdout:310 2024-04-07T16:32:38.935 INFO:tasks.workunit.client.0.smithi100.stdout:311 2024-04-07T16:32:38.937 INFO:tasks.workunit.client.0.smithi100.stdout:312 2024-04-07T16:32:38.939 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:32:38 smithi100 bash[74942]: cluster 2024-04-07T16:32:37.688047+0000 mon.a (mon.0) 1625 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T16:32:38.939 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:32:38 smithi100 bash[74942]: audit 2024-04-07T16:32:37.701436+0000 mon.a (mon.0) 1626 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110598-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:32:38.939 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:32:38 smithi100 bash[74942]: cluster 2024-04-07T16:32:37.709621+0000 mon.a (mon.0) 1627 : cluster [DBG] osdmap e906: 8 total, 8 up, 8 in 2024-04-07T16:32:38.939 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:32:38 smithi100 bash[74942]: audit 2024-04-07T16:32:37.711477+0000 mon.c (mon.2) 38 : audit [DBG] from='client.? 172.21.15.100:0/2668902816' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-04-07T16:32:38.940 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:32:38 smithi100 bash[76202]: cluster 2024-04-07T16:32:37.688047+0000 mon.a (mon.0) 1625 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T16:32:38.940 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:32:38 smithi100 bash[76202]: audit 2024-04-07T16:32:37.701436+0000 mon.a (mon.0) 1626 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110598-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:32:38.940 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:32:38 smithi100 bash[76202]: cluster 2024-04-07T16:32:37.709621+0000 mon.a (mon.0) 1627 : cluster [DBG] osdmap e906: 8 total, 8 up, 8 in 2024-04-07T16:32:38.940 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:32:38 smithi100 bash[76202]: audit 2024-04-07T16:32:37.711477+0000 mon.c (mon.2) 38 : audit [DBG] from='client.? 172.21.15.100:0/2668902816' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-04-07T16:32:38.940 INFO:tasks.workunit.client.0.smithi100.stdout:313 2024-04-07T16:32:38.943 INFO:tasks.workunit.client.0.smithi100.stdout:314 2024-04-07T16:32:38.945 INFO:tasks.workunit.client.0.smithi100.stdout:315 2024-04-07T16:32:38.948 INFO:tasks.workunit.client.0.smithi100.stdout:316 2024-04-07T16:32:38.950 INFO:tasks.workunit.client.0.smithi100.stdout:317 2024-04-07T16:32:38.953 INFO:tasks.workunit.client.0.smithi100.stdout:318 2024-04-07T16:32:38.955 INFO:tasks.workunit.client.0.smithi100.stdout:319 2024-04-07T16:32:38.958 INFO:tasks.workunit.client.0.smithi100.stdout:320 2024-04-07T16:32:38.961 INFO:tasks.workunit.client.0.smithi100.stdout:321 2024-04-07T16:32:38.963 INFO:tasks.workunit.client.0.smithi100.stdout:322 2024-04-07T16:32:38.966 INFO:tasks.workunit.client.0.smithi100.stdout:323 2024-04-07T16:32:38.969 INFO:tasks.workunit.client.0.smithi100.stdout:324 2024-04-07T16:32:38.972 INFO:tasks.workunit.client.0.smithi100.stdout:325 2024-04-07T16:32:38.974 INFO:tasks.workunit.client.0.smithi100.stdout:326 2024-04-07T16:32:38.977 INFO:tasks.workunit.client.0.smithi100.stdout:327 2024-04-07T16:32:38.980 INFO:tasks.workunit.client.0.smithi100.stdout:328 2024-04-07T16:32:38.983 INFO:tasks.workunit.client.0.smithi100.stdout:329 2024-04-07T16:32:38.986 INFO:tasks.workunit.client.0.smithi100.stdout:330 2024-04-07T16:32:38.988 INFO:tasks.workunit.client.0.smithi100.stdout:331 2024-04-07T16:32:38.991 INFO:tasks.workunit.client.0.smithi100.stdout:332 2024-04-07T16:32:38.993 INFO:tasks.workunit.client.0.smithi100.stdout:333 2024-04-07T16:32:38.996 INFO:tasks.workunit.client.0.smithi100.stdout:334 2024-04-07T16:32:38.999 INFO:tasks.workunit.client.0.smithi100.stdout:335 2024-04-07T16:32:39.002 INFO:tasks.workunit.client.0.smithi100.stdout:336 2024-04-07T16:32:39.005 INFO:tasks.workunit.client.0.smithi100.stdout:337 2024-04-07T16:32:39.008 INFO:tasks.workunit.client.0.smithi100.stdout:338 2024-04-07T16:32:39.017 INFO:tasks.workunit.client.0.smithi100.stdout:339 2024-04-07T16:32:39.020 INFO:tasks.workunit.client.0.smithi100.stdout:340 2024-04-07T16:32:39.023 INFO:tasks.workunit.client.0.smithi100.stdout:341 2024-04-07T16:32:39.025 INFO:tasks.workunit.client.0.smithi100.stdout:342 2024-04-07T16:32:39.028 INFO:tasks.workunit.client.0.smithi100.stdout:343 2024-04-07T16:32:39.030 INFO:tasks.workunit.client.0.smithi100.stdout:344 2024-04-07T16:32:39.033 INFO:tasks.workunit.client.0.smithi100.stdout:345 2024-04-07T16:32:39.036 INFO:tasks.workunit.client.0.smithi100.stdout:346 2024-04-07T16:32:39.038 INFO:tasks.workunit.client.0.smithi100.stdout:347 2024-04-07T16:32:39.041 INFO:tasks.workunit.client.0.smithi100.stdout:348 2024-04-07T16:32:39.044 INFO:tasks.workunit.client.0.smithi100.stdout:349 2024-04-07T16:32:39.046 INFO:tasks.workunit.client.0.smithi100.stdout:350 2024-04-07T16:32:39.049 INFO:tasks.workunit.client.0.smithi100.stdout:351 2024-04-07T16:32:39.052 INFO:tasks.workunit.client.0.smithi100.stdout:352 2024-04-07T16:32:39.055 INFO:tasks.workunit.client.0.smithi100.stdout:353 2024-04-07T16:32:39.057 INFO:tasks.workunit.client.0.smithi100.stdout:354 2024-04-07T16:32:39.060 INFO:tasks.workunit.client.0.smithi100.stdout:355 2024-04-07T16:32:39.063 INFO:tasks.workunit.client.0.smithi100.stdout:356 2024-04-07T16:32:39.065 INFO:tasks.workunit.client.0.smithi100.stdout:357 2024-04-07T16:32:39.068 INFO:tasks.workunit.client.0.smithi100.stdout:358 2024-04-07T16:32:39.071 INFO:tasks.workunit.client.0.smithi100.stdout:359 2024-04-07T16:32:39.073 INFO:tasks.workunit.client.0.smithi100.stdout:360 2024-04-07T16:32:39.076 INFO:tasks.workunit.client.0.smithi100.stdout:361 2024-04-07T16:32:39.079 INFO:tasks.workunit.client.0.smithi100.stdout:362 2024-04-07T16:32:39.081 INFO:tasks.workunit.client.0.smithi100.stdout:363 2024-04-07T16:32:39.084 INFO:tasks.workunit.client.0.smithi100.stdout:364 2024-04-07T16:32:39.087 INFO:tasks.workunit.client.0.smithi100.stdout:365 2024-04-07T16:32:39.090 INFO:tasks.workunit.client.0.smithi100.stdout:366 2024-04-07T16:32:39.093 INFO:tasks.workunit.client.0.smithi100.stdout:367 2024-04-07T16:32:39.095 INFO:tasks.workunit.client.0.smithi100.stdout:368 2024-04-07T16:32:39.098 INFO:tasks.workunit.client.0.smithi100.stdout:369 2024-04-07T16:32:39.101 INFO:tasks.workunit.client.0.smithi100.stdout:370 2024-04-07T16:32:39.104 INFO:tasks.workunit.client.0.smithi100.stdout:371 2024-04-07T16:32:39.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:32:38 smithi144 bash[66654]: cluster 2024-04-07T16:32:37.688047+0000 mon.a (mon.0) 1625 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T16:32:39.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:32:38 smithi144 bash[66654]: audit 2024-04-07T16:32:37.701436+0000 mon.a (mon.0) 1626 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110598-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:32:39.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:32:38 smithi144 bash[66654]: cluster 2024-04-07T16:32:37.709621+0000 mon.a (mon.0) 1627 : cluster [DBG] osdmap e906: 8 total, 8 up, 8 in 2024-04-07T16:32:39.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:32:38 smithi144 bash[66654]: audit 2024-04-07T16:32:37.711477+0000 mon.c (mon.2) 38 : audit [DBG] from='client.? 172.21.15.100:0/2668902816' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-04-07T16:32:39.107 INFO:tasks.workunit.client.0.smithi100.stdout:372 2024-04-07T16:32:39.110 INFO:tasks.workunit.client.0.smithi100.stdout:373 2024-04-07T16:32:39.112 INFO:tasks.workunit.client.0.smithi100.stdout:374 2024-04-07T16:32:39.115 INFO:tasks.workunit.client.0.smithi100.stdout:375 2024-04-07T16:32:39.118 INFO:tasks.workunit.client.0.smithi100.stdout:376 2024-04-07T16:32:39.120 INFO:tasks.workunit.client.0.smithi100.stdout:377 2024-04-07T16:32:39.123 INFO:tasks.workunit.client.0.smithi100.stdout:378 2024-04-07T16:32:39.126 INFO:tasks.workunit.client.0.smithi100.stdout:379 2024-04-07T16:32:39.129 INFO:tasks.workunit.client.0.smithi100.stdout:380 2024-04-07T16:32:39.131 INFO:tasks.workunit.client.0.smithi100.stdout:381 2024-04-07T16:32:39.134 INFO:tasks.workunit.client.0.smithi100.stdout:382 2024-04-07T16:32:39.137 INFO:tasks.workunit.client.0.smithi100.stdout:383 2024-04-07T16:32:39.140 INFO:tasks.workunit.client.0.smithi100.stdout:384 2024-04-07T16:32:39.143 INFO:tasks.workunit.client.0.smithi100.stdout:385 2024-04-07T16:32:39.145 INFO:tasks.workunit.client.0.smithi100.stdout:386 2024-04-07T16:32:39.148 INFO:tasks.workunit.client.0.smithi100.stdout:387 2024-04-07T16:32:39.151 INFO:tasks.workunit.client.0.smithi100.stdout:388 2024-04-07T16:32:39.154 INFO:tasks.workunit.client.0.smithi100.stdout:389 2024-04-07T16:32:39.157 INFO:tasks.workunit.client.0.smithi100.stdout:390 2024-04-07T16:32:39.160 INFO:tasks.workunit.client.0.smithi100.stdout:391 2024-04-07T16:32:39.163 INFO:tasks.workunit.client.0.smithi100.stdout:392 2024-04-07T16:32:39.166 INFO:tasks.workunit.client.0.smithi100.stdout:393 2024-04-07T16:32:39.169 INFO:tasks.workunit.client.0.smithi100.stdout:394 2024-04-07T16:32:39.172 INFO:tasks.workunit.client.0.smithi100.stdout:395 2024-04-07T16:32:39.175 INFO:tasks.workunit.client.0.smithi100.stdout:396 2024-04-07T16:32:39.178 INFO:tasks.workunit.client.0.smithi100.stdout:397 2024-04-07T16:32:39.182 INFO:tasks.workunit.client.0.smithi100.stdout:398 2024-04-07T16:32:39.185 INFO:tasks.workunit.client.0.smithi100.stdout:399 2024-04-07T16:32:39.189 INFO:tasks.workunit.client.0.smithi100.stdout:400 2024-04-07T16:32:39.192 INFO:tasks.workunit.client.0.smithi100.stdout:401 2024-04-07T16:32:39.194 INFO:tasks.workunit.client.0.smithi100.stdout:402 2024-04-07T16:32:39.197 INFO:tasks.workunit.client.0.smithi100.stdout:403 2024-04-07T16:32:39.200 INFO:tasks.workunit.client.0.smithi100.stdout:404 2024-04-07T16:32:39.203 INFO:tasks.workunit.client.0.smithi100.stdout:405 2024-04-07T16:32:39.206 INFO:tasks.workunit.client.0.smithi100.stdout:406 2024-04-07T16:32:39.209 INFO:tasks.workunit.client.0.smithi100.stdout:407 2024-04-07T16:32:39.211 INFO:tasks.workunit.client.0.smithi100.stdout:408 2024-04-07T16:32:39.214 INFO:tasks.workunit.client.0.smithi100.stdout:409 2024-04-07T16:32:39.217 INFO:tasks.workunit.client.0.smithi100.stdout:410 2024-04-07T16:32:39.220 INFO:tasks.workunit.client.0.smithi100.stdout:411 2024-04-07T16:32:39.223 INFO:tasks.workunit.client.0.smithi100.stdout:412 2024-04-07T16:32:39.225 INFO:tasks.workunit.client.0.smithi100.stdout:413 2024-04-07T16:32:39.228 INFO:tasks.workunit.client.0.smithi100.stdout:414 2024-04-07T16:32:39.231 INFO:tasks.workunit.client.0.smithi100.stdout:415 2024-04-07T16:32:39.234 INFO:tasks.workunit.client.0.smithi100.stdout:416 2024-04-07T16:32:39.237 INFO:tasks.workunit.client.0.smithi100.stdout:417 2024-04-07T16:32:39.240 INFO:tasks.workunit.client.0.smithi100.stdout:418 2024-04-07T16:32:39.243 INFO:tasks.workunit.client.0.smithi100.stdout:419 2024-04-07T16:32:39.245 INFO:tasks.workunit.client.0.smithi100.stdout:420 2024-04-07T16:32:39.248 INFO:tasks.workunit.client.0.smithi100.stdout:421 2024-04-07T16:32:39.251 INFO:tasks.workunit.client.0.smithi100.stdout:422 2024-04-07T16:32:39.254 INFO:tasks.workunit.client.0.smithi100.stdout:423 2024-04-07T16:32:39.257 INFO:tasks.workunit.client.0.smithi100.stdout:424 2024-04-07T16:32:39.260 INFO:tasks.workunit.client.0.smithi100.stdout:425 2024-04-07T16:32:39.263 INFO:tasks.workunit.client.0.smithi100.stdout:426 2024-04-07T16:32:39.266 INFO:tasks.workunit.client.0.smithi100.stdout:427 2024-04-07T16:32:39.269 INFO:tasks.workunit.client.0.smithi100.stdout:428 2024-04-07T16:32:39.272 INFO:tasks.workunit.client.0.smithi100.stdout:429 2024-04-07T16:32:39.275 INFO:tasks.workunit.client.0.smithi100.stdout:430 2024-04-07T16:32:39.277 INFO:tasks.workunit.client.0.smithi100.stdout:431 2024-04-07T16:32:39.280 INFO:tasks.workunit.client.0.smithi100.stdout:432 2024-04-07T16:32:39.283 INFO:tasks.workunit.client.0.smithi100.stdout:433 2024-04-07T16:32:39.286 INFO:tasks.workunit.client.0.smithi100.stdout:434 2024-04-07T16:32:39.289 INFO:tasks.workunit.client.0.smithi100.stdout:435 2024-04-07T16:32:39.292 INFO:tasks.workunit.client.0.smithi100.stdout:436 2024-04-07T16:32:39.295 INFO:tasks.workunit.client.0.smithi100.stdout:437 2024-04-07T16:32:39.298 INFO:tasks.workunit.client.0.smithi100.stdout:438 2024-04-07T16:32:39.301 INFO:tasks.workunit.client.0.smithi100.stdout:439 2024-04-07T16:32:39.304 INFO:tasks.workunit.client.0.smithi100.stdout:440 2024-04-07T16:32:39.307 INFO:tasks.workunit.client.0.smithi100.stdout:441 2024-04-07T16:32:39.310 INFO:tasks.workunit.client.0.smithi100.stdout:442 2024-04-07T16:32:39.312 INFO:tasks.workunit.client.0.smithi100.stdout:443 2024-04-07T16:32:39.316 INFO:tasks.workunit.client.0.smithi100.stdout:444 2024-04-07T16:32:39.319 INFO:tasks.workunit.client.0.smithi100.stdout:445 2024-04-07T16:32:39.322 INFO:tasks.workunit.client.0.smithi100.stdout:446 2024-04-07T16:32:39.325 INFO:tasks.workunit.client.0.smithi100.stdout:447 2024-04-07T16:32:39.327 INFO:tasks.workunit.client.0.smithi100.stdout:448 2024-04-07T16:32:39.330 INFO:tasks.workunit.client.0.smithi100.stdout:449 2024-04-07T16:32:39.333 INFO:tasks.workunit.client.0.smithi100.stdout:450 2024-04-07T16:32:39.337 INFO:tasks.workunit.client.0.smithi100.stdout:451 2024-04-07T16:32:39.339 INFO:tasks.workunit.client.0.smithi100.stdout:452 2024-04-07T16:32:39.342 INFO:tasks.workunit.client.0.smithi100.stdout:453 2024-04-07T16:32:39.345 INFO:tasks.workunit.client.0.smithi100.stdout:454 2024-04-07T16:32:39.347 INFO:tasks.workunit.client.0.smithi100.stdout:455 2024-04-07T16:32:39.350 INFO:tasks.workunit.client.0.smithi100.stdout:456 2024-04-07T16:32:39.353 INFO:tasks.workunit.client.0.smithi100.stdout:457 2024-04-07T16:32:39.356 INFO:tasks.workunit.client.0.smithi100.stdout:458 2024-04-07T16:32:39.359 INFO:tasks.workunit.client.0.smithi100.stdout:459 2024-04-07T16:32:39.361 INFO:tasks.workunit.client.0.smithi100.stdout:460 2024-04-07T16:32:39.364 INFO:tasks.workunit.client.0.smithi100.stdout:461 2024-04-07T16:32:39.367 INFO:tasks.workunit.client.0.smithi100.stdout:462 2024-04-07T16:32:39.370 INFO:tasks.workunit.client.0.smithi100.stdout:463 2024-04-07T16:32:39.373 INFO:tasks.workunit.client.0.smithi100.stdout:464 2024-04-07T16:32:39.376 INFO:tasks.workunit.client.0.smithi100.stdout:465 2024-04-07T16:32:39.379 INFO:tasks.workunit.client.0.smithi100.stdout:466 2024-04-07T16:32:39.382 INFO:tasks.workunit.client.0.smithi100.stdout:467 2024-04-07T16:32:39.385 INFO:tasks.workunit.client.0.smithi100.stdout:468 2024-04-07T16:32:39.387 INFO:tasks.workunit.client.0.smithi100.stdout:469 2024-04-07T16:32:39.390 INFO:tasks.workunit.client.0.smithi100.stdout:470 2024-04-07T16:32:39.393 INFO:tasks.workunit.client.0.smithi100.stdout:471 2024-04-07T16:32:39.396 INFO:tasks.workunit.client.0.smithi100.stdout:472 2024-04-07T16:32:39.399 INFO:tasks.workunit.client.0.smithi100.stdout:473 2024-04-07T16:32:39.402 INFO:tasks.workunit.client.0.smithi100.stdout:474 2024-04-07T16:32:39.405 INFO:tasks.workunit.client.0.smithi100.stdout:475 2024-04-07T16:32:39.408 INFO:tasks.workunit.client.0.smithi100.stdout:476 2024-04-07T16:32:39.411 INFO:tasks.workunit.client.0.smithi100.stdout:477 2024-04-07T16:32:39.413 INFO:tasks.workunit.client.0.smithi100.stdout:478 2024-04-07T16:32:39.416 INFO:tasks.workunit.client.0.smithi100.stdout:479 2024-04-07T16:32:39.419 INFO:tasks.workunit.client.0.smithi100.stdout:480 2024-04-07T16:32:39.422 INFO:tasks.workunit.client.0.smithi100.stdout:481 2024-04-07T16:32:39.425 INFO:tasks.workunit.client.0.smithi100.stdout:482 2024-04-07T16:32:39.428 INFO:tasks.workunit.client.0.smithi100.stdout:483 2024-04-07T16:32:39.431 INFO:tasks.workunit.client.0.smithi100.stdout:484 2024-04-07T16:32:39.434 INFO:tasks.workunit.client.0.smithi100.stdout:485 2024-04-07T16:32:39.437 INFO:tasks.workunit.client.0.smithi100.stdout:486 2024-04-07T16:32:39.440 INFO:tasks.workunit.client.0.smithi100.stdout:487 2024-04-07T16:32:39.442 INFO:tasks.workunit.client.0.smithi100.stdout:488 2024-04-07T16:32:39.445 INFO:tasks.workunit.client.0.smithi100.stdout:489 2024-04-07T16:32:39.448 INFO:tasks.workunit.client.0.smithi100.stdout:490 2024-04-07T16:32:39.451 INFO:tasks.workunit.client.0.smithi100.stdout:491 2024-04-07T16:32:39.454 INFO:tasks.workunit.client.0.smithi100.stdout:492 2024-04-07T16:32:39.457 INFO:tasks.workunit.client.0.smithi100.stdout:493 2024-04-07T16:32:39.460 INFO:tasks.workunit.client.0.smithi100.stdout:494 2024-04-07T16:32:39.463 INFO:tasks.workunit.client.0.smithi100.stdout:495 2024-04-07T16:32:39.466 INFO:tasks.workunit.client.0.smithi100.stdout:496 2024-04-07T16:32:39.469 INFO:tasks.workunit.client.0.smithi100.stdout:497 2024-04-07T16:32:39.472 INFO:tasks.workunit.client.0.smithi100.stdout:498 2024-04-07T16:32:39.475 INFO:tasks.workunit.client.0.smithi100.stdout:499 2024-04-07T16:32:39.478 INFO:tasks.workunit.client.0.smithi100.stdout:500 2024-04-07T16:32:39.481 INFO:tasks.workunit.client.0.smithi100.stdout:501 2024-04-07T16:32:39.484 INFO:tasks.workunit.client.0.smithi100.stdout:502 2024-04-07T16:32:39.487 INFO:tasks.workunit.client.0.smithi100.stdout:503 2024-04-07T16:32:39.490 INFO:tasks.workunit.client.0.smithi100.stdout:504 2024-04-07T16:32:39.493 INFO:tasks.workunit.client.0.smithi100.stdout:505 2024-04-07T16:32:39.495 INFO:tasks.workunit.client.0.smithi100.stdout:506 2024-04-07T16:32:39.499 INFO:tasks.workunit.client.0.smithi100.stdout:507 2024-04-07T16:32:39.502 INFO:tasks.workunit.client.0.smithi100.stdout:508 2024-04-07T16:32:39.505 INFO:tasks.workunit.client.0.smithi100.stdout:509 2024-04-07T16:32:39.508 INFO:tasks.workunit.client.0.smithi100.stdout:510 2024-04-07T16:32:39.511 INFO:tasks.workunit.client.0.smithi100.stdout:511 2024-04-07T16:32:39.514 INFO:tasks.workunit.client.0.smithi100.stdout:512 2024-04-07T16:32:39.517 INFO:tasks.workunit.client.0.smithi100.stdout:513 2024-04-07T16:32:39.520 INFO:tasks.workunit.client.0.smithi100.stdout:514 2024-04-07T16:32:39.523 INFO:tasks.workunit.client.0.smithi100.stdout:515 2024-04-07T16:32:39.526 INFO:tasks.workunit.client.0.smithi100.stdout:516 2024-04-07T16:32:39.529 INFO:tasks.workunit.client.0.smithi100.stdout:517 2024-04-07T16:32:39.532 INFO:tasks.workunit.client.0.smithi100.stdout:518 2024-04-07T16:32:39.535 INFO:tasks.workunit.client.0.smithi100.stdout:519 2024-04-07T16:32:39.537 INFO:tasks.workunit.client.0.smithi100.stdout:520 2024-04-07T16:32:39.540 INFO:tasks.workunit.client.0.smithi100.stdout:521 2024-04-07T16:32:39.543 INFO:tasks.workunit.client.0.smithi100.stdout:522 2024-04-07T16:32:39.546 INFO:tasks.workunit.client.0.smithi100.stdout:523 2024-04-07T16:32:39.549 INFO:tasks.workunit.client.0.smithi100.stdout:524 2024-04-07T16:32:39.552 INFO:tasks.workunit.client.0.smithi100.stdout:525 2024-04-07T16:32:39.554 INFO:tasks.workunit.client.0.smithi100.stdout:526 2024-04-07T16:32:39.557 INFO:tasks.workunit.client.0.smithi100.stdout:527 2024-04-07T16:32:39.560 INFO:tasks.workunit.client.0.smithi100.stdout:528 2024-04-07T16:32:39.563 INFO:tasks.workunit.client.0.smithi100.stdout:529 2024-04-07T16:32:39.566 INFO:tasks.workunit.client.0.smithi100.stdout:530 2024-04-07T16:32:39.568 INFO:tasks.workunit.client.0.smithi100.stdout:531 2024-04-07T16:32:39.571 INFO:tasks.workunit.client.0.smithi100.stdout:532 2024-04-07T16:32:39.574 INFO:tasks.workunit.client.0.smithi100.stdout:533 2024-04-07T16:32:39.577 INFO:tasks.workunit.client.0.smithi100.stdout:534 2024-04-07T16:32:39.580 INFO:tasks.workunit.client.0.smithi100.stdout:535 2024-04-07T16:32:39.583 INFO:tasks.workunit.client.0.smithi100.stdout:536 2024-04-07T16:32:39.586 INFO:tasks.workunit.client.0.smithi100.stdout:537 2024-04-07T16:32:39.589 INFO:tasks.workunit.client.0.smithi100.stdout:538 2024-04-07T16:32:39.592 INFO:tasks.workunit.client.0.smithi100.stdout:539 2024-04-07T16:32:39.595 INFO:tasks.workunit.client.0.smithi100.stdout:540 2024-04-07T16:32:39.597 INFO:tasks.workunit.client.0.smithi100.stdout:541 2024-04-07T16:32:39.600 INFO:tasks.workunit.client.0.smithi100.stdout:542 2024-04-07T16:32:39.603 INFO:tasks.workunit.client.0.smithi100.stdout:543 2024-04-07T16:32:39.606 INFO:tasks.workunit.client.0.smithi100.stdout:544 2024-04-07T16:32:39.609 INFO:tasks.workunit.client.0.smithi100.stdout:545 2024-04-07T16:32:39.612 INFO:tasks.workunit.client.0.smithi100.stdout:546 2024-04-07T16:32:39.615 INFO:tasks.workunit.client.0.smithi100.stdout:547 2024-04-07T16:32:39.618 INFO:tasks.workunit.client.0.smithi100.stdout:548 2024-04-07T16:32:39.621 INFO:tasks.workunit.client.0.smithi100.stdout:549 2024-04-07T16:32:39.623 INFO:tasks.workunit.client.0.smithi100.stdout:550 2024-04-07T16:32:39.626 INFO:tasks.workunit.client.0.smithi100.stdout:551 2024-04-07T16:32:39.629 INFO:tasks.workunit.client.0.smithi100.stdout:552 2024-04-07T16:32:39.632 INFO:tasks.workunit.client.0.smithi100.stdout:553 2024-04-07T16:32:39.635 INFO:tasks.workunit.client.0.smithi100.stdout:554 2024-04-07T16:32:39.638 INFO:tasks.workunit.client.0.smithi100.stdout:555 2024-04-07T16:32:39.641 INFO:tasks.workunit.client.0.smithi100.stdout:556 2024-04-07T16:32:39.643 INFO:tasks.workunit.client.0.smithi100.stdout:557 2024-04-07T16:32:39.646 INFO:tasks.workunit.client.0.smithi100.stdout:558 2024-04-07T16:32:39.649 INFO:tasks.workunit.client.0.smithi100.stdout:559 2024-04-07T16:32:39.652 INFO:tasks.workunit.client.0.smithi100.stdout:560 2024-04-07T16:32:39.655 INFO:tasks.workunit.client.0.smithi100.stdout:561 2024-04-07T16:32:39.658 INFO:tasks.workunit.client.0.smithi100.stdout:562 2024-04-07T16:32:39.661 INFO:tasks.workunit.client.0.smithi100.stdout:563 2024-04-07T16:32:39.664 INFO:tasks.workunit.client.0.smithi100.stdout:564 2024-04-07T16:32:39.667 INFO:tasks.workunit.client.0.smithi100.stdout:565 2024-04-07T16:32:39.671 INFO:tasks.workunit.client.0.smithi100.stdout:566 2024-04-07T16:32:39.674 INFO:tasks.workunit.client.0.smithi100.stdout:567 2024-04-07T16:32:39.677 INFO:tasks.workunit.client.0.smithi100.stdout:568 2024-04-07T16:32:39.680 INFO:tasks.workunit.client.0.smithi100.stdout:569 2024-04-07T16:32:39.683 INFO:tasks.workunit.client.0.smithi100.stdout:570 2024-04-07T16:32:39.686 INFO:tasks.workunit.client.0.smithi100.stdout:571 2024-04-07T16:32:39.689 INFO:tasks.workunit.client.0.smithi100.stdout:572 2024-04-07T16:32:39.693 INFO:tasks.workunit.client.0.smithi100.stdout:573 2024-04-07T16:32:39.697 INFO:tasks.workunit.client.0.smithi100.stdout:574 2024-04-07T16:32:39.700 INFO:tasks.workunit.client.0.smithi100.stdout:575 2024-04-07T16:32:39.703 INFO:tasks.workunit.client.0.smithi100.stdout:576 2024-04-07T16:32:39.707 INFO:tasks.workunit.client.0.smithi100.stdout:577 2024-04-07T16:32:39.710 INFO:tasks.workunit.client.0.smithi100.stdout:578 2024-04-07T16:32:39.713 INFO:tasks.workunit.client.0.smithi100.stdout:579 2024-04-07T16:32:39.716 INFO:tasks.workunit.client.0.smithi100.stdout:580 2024-04-07T16:32:39.719 INFO:tasks.workunit.client.0.smithi100.stdout:581 2024-04-07T16:32:39.722 INFO:tasks.workunit.client.0.smithi100.stdout:582 2024-04-07T16:32:39.725 INFO:tasks.workunit.client.0.smithi100.stdout:583 2024-04-07T16:32:39.728 INFO:tasks.workunit.client.0.smithi100.stdout:584 2024-04-07T16:32:39.732 INFO:tasks.workunit.client.0.smithi100.stdout:585 2024-04-07T16:32:39.735 INFO:tasks.workunit.client.0.smithi100.stdout:586 2024-04-07T16:32:39.738 INFO:tasks.workunit.client.0.smithi100.stdout:587 2024-04-07T16:32:39.741 INFO:tasks.workunit.client.0.smithi100.stdout:588 2024-04-07T16:32:39.744 INFO:tasks.workunit.client.0.smithi100.stdout:589 2024-04-07T16:32:39.747 INFO:tasks.workunit.client.0.smithi100.stdout:590 2024-04-07T16:32:39.750 INFO:tasks.workunit.client.0.smithi100.stdout:591 2024-04-07T16:32:39.752 INFO:tasks.workunit.client.0.smithi100.stdout:592 2024-04-07T16:32:39.756 INFO:tasks.workunit.client.0.smithi100.stdout:593 2024-04-07T16:32:39.759 INFO:tasks.workunit.client.0.smithi100.stdout:594 2024-04-07T16:32:39.762 INFO:tasks.workunit.client.0.smithi100.stdout:595 2024-04-07T16:32:39.765 INFO:tasks.workunit.client.0.smithi100.stdout:596 2024-04-07T16:32:39.768 INFO:tasks.workunit.client.0.smithi100.stdout:597 2024-04-07T16:32:39.771 INFO:tasks.workunit.client.0.smithi100.stdout:598 2024-04-07T16:32:39.774 INFO:tasks.workunit.client.0.smithi100.stdout:599 2024-04-07T16:32:39.777 INFO:tasks.workunit.client.0.smithi100.stdout:600 2024-04-07T16:32:39.781 INFO:tasks.workunit.client.0.smithi100.stdout:601 2024-04-07T16:32:39.784 INFO:tasks.workunit.client.0.smithi100.stdout:602 2024-04-07T16:32:39.787 INFO:tasks.workunit.client.0.smithi100.stdout:603 2024-04-07T16:32:39.791 INFO:tasks.workunit.client.0.smithi100.stdout:604 2024-04-07T16:32:39.794 INFO:tasks.workunit.client.0.smithi100.stdout:605 2024-04-07T16:32:39.797 INFO:tasks.workunit.client.0.smithi100.stdout:606 2024-04-07T16:32:39.800 INFO:tasks.workunit.client.0.smithi100.stdout:607 2024-04-07T16:32:39.805 INFO:tasks.workunit.client.0.smithi100.stdout:608 2024-04-07T16:32:39.806 INFO:tasks.workunit.client.0.smithi100.stdout:609 2024-04-07T16:32:39.809 INFO:tasks.workunit.client.0.smithi100.stdout:610 2024-04-07T16:32:39.812 INFO:tasks.workunit.client.0.smithi100.stdout:611 2024-04-07T16:32:39.815 INFO:tasks.workunit.client.0.smithi100.stdout:612 2024-04-07T16:32:39.818 INFO:tasks.workunit.client.0.smithi100.stdout:613 2024-04-07T16:32:39.820 INFO:tasks.workunit.client.0.smithi100.stdout:614 2024-04-07T16:32:39.823 INFO:tasks.workunit.client.0.smithi100.stdout:615 2024-04-07T16:32:39.826 INFO:tasks.workunit.client.0.smithi100.stdout:616 2024-04-07T16:32:39.829 INFO:tasks.workunit.client.0.smithi100.stdout:617 2024-04-07T16:32:39.832 INFO:tasks.workunit.client.0.smithi100.stdout:618 2024-04-07T16:32:39.835 INFO:tasks.workunit.client.0.smithi100.stdout:619 2024-04-07T16:32:39.838 INFO:tasks.workunit.client.0.smithi100.stdout:620 2024-04-07T16:32:39.841 INFO:tasks.workunit.client.0.smithi100.stdout:621 2024-04-07T16:32:39.844 INFO:tasks.workunit.client.0.smithi100.stdout:622 2024-04-07T16:32:39.847 INFO:tasks.workunit.client.0.smithi100.stdout:623 2024-04-07T16:32:39.850 INFO:tasks.workunit.client.0.smithi100.stdout:624 2024-04-07T16:32:39.853 INFO:tasks.workunit.client.0.smithi100.stdout:625 2024-04-07T16:32:39.856 INFO:tasks.workunit.client.0.smithi100.stdout:626 2024-04-07T16:32:39.859 INFO:tasks.workunit.client.0.smithi100.stdout:627 2024-04-07T16:32:39.862 INFO:tasks.workunit.client.0.smithi100.stdout:628 2024-04-07T16:32:39.865 INFO:tasks.workunit.client.0.smithi100.stdout:629 2024-04-07T16:32:39.869 INFO:tasks.workunit.client.0.smithi100.stdout:630 2024-04-07T16:32:39.872 INFO:tasks.workunit.client.0.smithi100.stdout:631 2024-04-07T16:32:39.875 INFO:tasks.workunit.client.0.smithi100.stdout:632 2024-04-07T16:32:39.878 INFO:tasks.workunit.client.0.smithi100.stdout:633 2024-04-07T16:32:39.881 INFO:tasks.workunit.client.0.smithi100.stdout:634 2024-04-07T16:32:39.884 INFO:tasks.workunit.client.0.smithi100.stdout:635 2024-04-07T16:32:39.887 INFO:tasks.workunit.client.0.smithi100.stdout:636 2024-04-07T16:32:39.890 INFO:tasks.workunit.client.0.smithi100.stdout:637 2024-04-07T16:32:39.893 INFO:tasks.workunit.client.0.smithi100.stdout:638 2024-04-07T16:32:39.896 INFO:tasks.workunit.client.0.smithi100.stdout:639 2024-04-07T16:32:39.899 INFO:tasks.workunit.client.0.smithi100.stdout:640 2024-04-07T16:32:39.902 INFO:tasks.workunit.client.0.smithi100.stdout:641 2024-04-07T16:32:39.905 INFO:tasks.workunit.client.0.smithi100.stdout:642 2024-04-07T16:32:39.908 INFO:tasks.workunit.client.0.smithi100.stdout:643 2024-04-07T16:32:39.911 INFO:tasks.workunit.client.0.smithi100.stdout:644 2024-04-07T16:32:39.914 INFO:tasks.workunit.client.0.smithi100.stdout:645 2024-04-07T16:32:39.917 INFO:tasks.workunit.client.0.smithi100.stdout:646 2024-04-07T16:32:39.920 INFO:tasks.workunit.client.0.smithi100.stdout:647 2024-04-07T16:32:39.923 INFO:tasks.workunit.client.0.smithi100.stdout:648 2024-04-07T16:32:39.926 INFO:tasks.workunit.client.0.smithi100.stdout:649 2024-04-07T16:32:39.929 INFO:tasks.workunit.client.0.smithi100.stdout:650 2024-04-07T16:32:39.932 INFO:tasks.workunit.client.0.smithi100.stdout:651 2024-04-07T16:32:39.935 INFO:tasks.workunit.client.0.smithi100.stdout:652 2024-04-07T16:32:39.938 INFO:tasks.workunit.client.0.smithi100.stdout:653 2024-04-07T16:32:39.942 INFO:tasks.workunit.client.0.smithi100.stdout:654 2024-04-07T16:32:39.946 INFO:tasks.workunit.client.0.smithi100.stdout:655 2024-04-07T16:32:39.949 INFO:tasks.workunit.client.0.smithi100.stdout:656 2024-04-07T16:32:39.952 INFO:tasks.workunit.client.0.smithi100.stdout:657 2024-04-07T16:32:39.955 INFO:tasks.workunit.client.0.smithi100.stdout:658 2024-04-07T16:32:39.959 INFO:tasks.workunit.client.0.smithi100.stdout:659 2024-04-07T16:32:39.963 INFO:tasks.workunit.client.0.smithi100.stdout:660 2024-04-07T16:32:39.966 INFO:tasks.workunit.client.0.smithi100.stdout:661 2024-04-07T16:32:39.969 INFO:tasks.workunit.client.0.smithi100.stdout:662 2024-04-07T16:32:39.972 INFO:tasks.workunit.client.0.smithi100.stdout:663 2024-04-07T16:32:39.976 INFO:tasks.workunit.client.0.smithi100.stdout:664 2024-04-07T16:32:39.978 INFO:tasks.workunit.client.0.smithi100.stdout:665 2024-04-07T16:32:39.981 INFO:tasks.workunit.client.0.smithi100.stdout:666 2024-04-07T16:32:39.984 INFO:tasks.workunit.client.0.smithi100.stdout:667 2024-04-07T16:32:39.987 INFO:tasks.workunit.client.0.smithi100.stdout:668 2024-04-07T16:32:39.990 INFO:tasks.workunit.client.0.smithi100.stdout:669 2024-04-07T16:32:39.994 INFO:tasks.workunit.client.0.smithi100.stdout:670 2024-04-07T16:32:39.997 INFO:tasks.workunit.client.0.smithi100.stdout:671 2024-04-07T16:32:40.000 INFO:tasks.workunit.client.0.smithi100.stdout:672 2024-04-07T16:32:40.003 INFO:tasks.workunit.client.0.smithi100.stdout:673 2024-04-07T16:32:40.006 INFO:tasks.workunit.client.0.smithi100.stdout:674 2024-04-07T16:32:40.009 INFO:tasks.workunit.client.0.smithi100.stdout:675 2024-04-07T16:32:40.012 INFO:tasks.workunit.client.0.smithi100.stdout:676 2024-04-07T16:32:40.015 INFO:tasks.workunit.client.0.smithi100.stdout:677 2024-04-07T16:32:40.018 INFO:tasks.workunit.client.0.smithi100.stdout:678 2024-04-07T16:32:40.021 INFO:tasks.workunit.client.0.smithi100.stdout:679 2024-04-07T16:32:40.024 INFO:tasks.workunit.client.0.smithi100.stdout:680 2024-04-07T16:32:40.027 INFO:tasks.workunit.client.0.smithi100.stdout:681 2024-04-07T16:32:40.030 INFO:tasks.workunit.client.0.smithi100.stdout:682 2024-04-07T16:32:40.033 INFO:tasks.workunit.client.0.smithi100.stdout:683 2024-04-07T16:32:40.036 INFO:tasks.workunit.client.0.smithi100.stdout:684 2024-04-07T16:32:40.040 INFO:tasks.workunit.client.0.smithi100.stdout:685 2024-04-07T16:32:40.043 INFO:tasks.workunit.client.0.smithi100.stdout:686 2024-04-07T16:32:40.046 INFO:tasks.workunit.client.0.smithi100.stdout:687 2024-04-07T16:32:40.049 INFO:tasks.workunit.client.0.smithi100.stdout:688 2024-04-07T16:32:40.053 INFO:tasks.workunit.client.0.smithi100.stdout:689 2024-04-07T16:32:40.056 INFO:tasks.workunit.client.0.smithi100.stdout:690 2024-04-07T16:32:40.060 INFO:tasks.workunit.client.0.smithi100.stdout:691 2024-04-07T16:32:40.063 INFO:tasks.workunit.client.0.smithi100.stdout:692 2024-04-07T16:32:40.066 INFO:tasks.workunit.client.0.smithi100.stdout:693 2024-04-07T16:32:40.070 INFO:tasks.workunit.client.0.smithi100.stdout:694 2024-04-07T16:32:40.073 INFO:tasks.workunit.client.0.smithi100.stdout:695 2024-04-07T16:32:40.076 INFO:tasks.workunit.client.0.smithi100.stdout:696 2024-04-07T16:32:40.079 INFO:tasks.workunit.client.0.smithi100.stdout:697 2024-04-07T16:32:40.083 INFO:tasks.workunit.client.0.smithi100.stdout:698 2024-04-07T16:32:40.086 INFO:tasks.workunit.client.0.smithi100.stdout:699 2024-04-07T16:32:40.089 INFO:tasks.workunit.client.0.smithi100.stdout:700 2024-04-07T16:32:40.092 INFO:tasks.workunit.client.0.smithi100.stdout:701 2024-04-07T16:32:40.095 INFO:tasks.workunit.client.0.smithi100.stdout:702 2024-04-07T16:32:40.098 INFO:tasks.workunit.client.0.smithi100.stdout:703 2024-04-07T16:32:40.101 INFO:tasks.workunit.client.0.smithi100.stdout:704 2024-04-07T16:32:40.104 INFO:tasks.workunit.client.0.smithi100.stdout:705 2024-04-07T16:32:40.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:32:39 smithi144 bash[66654]: cluster 2024-04-07T16:32:38.710978+0000 mon.a (mon.0) 1628 : cluster [DBG] osdmap e907: 8 total, 8 up, 8 in 2024-04-07T16:32:40.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:32:39 smithi144 bash[66654]: cluster 2024-04-07T16:32:38.919101+0000 mgr.y (mgr.25175) 519 : cluster [DBG] pgmap v1001: 137 pgs: 2 creating+activating, 12 creating+peering, 14 unknown, 109 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:32:40.107 INFO:tasks.workunit.client.0.smithi100.stdout:706 2024-04-07T16:32:40.111 INFO:tasks.workunit.client.0.smithi100.stdout:707 2024-04-07T16:32:40.113 INFO:tasks.workunit.client.0.smithi100.stdout:708 2024-04-07T16:32:40.117 INFO:tasks.workunit.client.0.smithi100.stdout:709 2024-04-07T16:32:40.120 INFO:tasks.workunit.client.0.smithi100.stdout:710 2024-04-07T16:32:40.123 INFO:tasks.workunit.client.0.smithi100.stdout:711 2024-04-07T16:32:40.126 INFO:tasks.workunit.client.0.smithi100.stdout:712 2024-04-07T16:32:40.130 INFO:tasks.workunit.client.0.smithi100.stdout:713 2024-04-07T16:32:40.133 INFO:tasks.workunit.client.0.smithi100.stdout:714 2024-04-07T16:32:40.137 INFO:tasks.workunit.client.0.smithi100.stdout:715 2024-04-07T16:32:40.139 INFO:tasks.workunit.client.0.smithi100.stdout:716 2024-04-07T16:32:40.143 INFO:tasks.workunit.client.0.smithi100.stdout:717 2024-04-07T16:32:40.146 INFO:tasks.workunit.client.0.smithi100.stdout:718 2024-04-07T16:32:40.149 INFO:tasks.workunit.client.0.smithi100.stdout:719 2024-04-07T16:32:40.152 INFO:tasks.workunit.client.0.smithi100.stdout:720 2024-04-07T16:32:40.155 INFO:tasks.workunit.client.0.smithi100.stdout:721 2024-04-07T16:32:40.158 INFO:tasks.workunit.client.0.smithi100.stdout:722 2024-04-07T16:32:40.161 INFO:tasks.workunit.client.0.smithi100.stdout:723 2024-04-07T16:32:40.164 INFO:tasks.workunit.client.0.smithi100.stdout:724 2024-04-07T16:32:40.167 INFO:tasks.workunit.client.0.smithi100.stdout:725 2024-04-07T16:32:40.170 INFO:tasks.workunit.client.0.smithi100.stdout:726 2024-04-07T16:32:40.173 INFO:tasks.workunit.client.0.smithi100.stdout:727 2024-04-07T16:32:40.176 INFO:tasks.workunit.client.0.smithi100.stdout:728 2024-04-07T16:32:40.179 INFO:tasks.workunit.client.0.smithi100.stdout:729 2024-04-07T16:32:40.182 INFO:tasks.workunit.client.0.smithi100.stdout:730 2024-04-07T16:32:40.185 INFO:tasks.workunit.client.0.smithi100.stdout:731 2024-04-07T16:32:40.188 INFO:tasks.workunit.client.0.smithi100.stdout:732 2024-04-07T16:32:40.189 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:32:39 smithi100 bash[74942]: cluster 2024-04-07T16:32:38.710978+0000 mon.a (mon.0) 1628 : cluster [DBG] osdmap e907: 8 total, 8 up, 8 in 2024-04-07T16:32:40.189 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:32:39 smithi100 bash[74942]: cluster 2024-04-07T16:32:38.919101+0000 mgr.y (mgr.25175) 519 : cluster [DBG] pgmap v1001: 137 pgs: 2 creating+activating, 12 creating+peering, 14 unknown, 109 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:32:40.189 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:32:39 smithi100 bash[76202]: cluster 2024-04-07T16:32:38.710978+0000 mon.a (mon.0) 1628 : cluster [DBG] osdmap e907: 8 total, 8 up, 8 in 2024-04-07T16:32:40.189 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:32:39 smithi100 bash[76202]: cluster 2024-04-07T16:32:38.919101+0000 mgr.y (mgr.25175) 519 : cluster [DBG] pgmap v1001: 137 pgs: 2 creating+activating, 12 creating+peering, 14 unknown, 109 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:32:40.192 INFO:tasks.workunit.client.0.smithi100.stdout:733 2024-04-07T16:32:40.195 INFO:tasks.workunit.client.0.smithi100.stdout:734 2024-04-07T16:32:40.198 INFO:tasks.workunit.client.0.smithi100.stdout:735 2024-04-07T16:32:40.201 INFO:tasks.workunit.client.0.smithi100.stdout:736 2024-04-07T16:32:40.204 INFO:tasks.workunit.client.0.smithi100.stdout:737 2024-04-07T16:32:40.208 INFO:tasks.workunit.client.0.smithi100.stdout:738 2024-04-07T16:32:40.211 INFO:tasks.workunit.client.0.smithi100.stdout:739 2024-04-07T16:32:40.214 INFO:tasks.workunit.client.0.smithi100.stdout:740 2024-04-07T16:32:40.217 INFO:tasks.workunit.client.0.smithi100.stdout:741 2024-04-07T16:32:40.220 INFO:tasks.workunit.client.0.smithi100.stdout:742 2024-04-07T16:32:40.224 INFO:tasks.workunit.client.0.smithi100.stdout:743 2024-04-07T16:32:40.228 INFO:tasks.workunit.client.0.smithi100.stdout:744 2024-04-07T16:32:40.231 INFO:tasks.workunit.client.0.smithi100.stdout:745 2024-04-07T16:32:40.234 INFO:tasks.workunit.client.0.smithi100.stdout:746 2024-04-07T16:32:40.237 INFO:tasks.workunit.client.0.smithi100.stdout:747 2024-04-07T16:32:40.241 INFO:tasks.workunit.client.0.smithi100.stdout:748 2024-04-07T16:32:40.244 INFO:tasks.workunit.client.0.smithi100.stdout:749 2024-04-07T16:32:40.247 INFO:tasks.workunit.client.0.smithi100.stdout:750 2024-04-07T16:32:40.251 INFO:tasks.workunit.client.0.smithi100.stdout:751 2024-04-07T16:32:40.254 INFO:tasks.workunit.client.0.smithi100.stdout:752 2024-04-07T16:32:40.257 INFO:tasks.workunit.client.0.smithi100.stdout:753 2024-04-07T16:32:40.260 INFO:tasks.workunit.client.0.smithi100.stdout:754 2024-04-07T16:32:40.263 INFO:tasks.workunit.client.0.smithi100.stdout:755 2024-04-07T16:32:40.266 INFO:tasks.workunit.client.0.smithi100.stdout:756 2024-04-07T16:32:40.269 INFO:tasks.workunit.client.0.smithi100.stdout:757 2024-04-07T16:32:40.272 INFO:tasks.workunit.client.0.smithi100.stdout:758 2024-04-07T16:32:40.275 INFO:tasks.workunit.client.0.smithi100.stdout:759 2024-04-07T16:32:40.279 INFO:tasks.workunit.client.0.smithi100.stdout:760 2024-04-07T16:32:40.282 INFO:tasks.workunit.client.0.smithi100.stdout:761 2024-04-07T16:32:40.286 INFO:tasks.workunit.client.0.smithi100.stdout:762 2024-04-07T16:32:40.289 INFO:tasks.workunit.client.0.smithi100.stdout:763 2024-04-07T16:32:40.292 INFO:tasks.workunit.client.0.smithi100.stdout:764 2024-04-07T16:32:40.296 INFO:tasks.workunit.client.0.smithi100.stdout:765 2024-04-07T16:32:40.299 INFO:tasks.workunit.client.0.smithi100.stdout:766 2024-04-07T16:32:40.302 INFO:tasks.workunit.client.0.smithi100.stdout:767 2024-04-07T16:32:40.306 INFO:tasks.workunit.client.0.smithi100.stdout:768 2024-04-07T16:32:40.309 INFO:tasks.workunit.client.0.smithi100.stdout:769 2024-04-07T16:32:40.313 INFO:tasks.workunit.client.0.smithi100.stdout:770 2024-04-07T16:32:40.316 INFO:tasks.workunit.client.0.smithi100.stdout:771 2024-04-07T16:32:40.320 INFO:tasks.workunit.client.0.smithi100.stdout:772 2024-04-07T16:32:40.323 INFO:tasks.workunit.client.0.smithi100.stdout:773 2024-04-07T16:32:40.326 INFO:tasks.workunit.client.0.smithi100.stdout:774 2024-04-07T16:32:40.329 INFO:tasks.workunit.client.0.smithi100.stdout:775 2024-04-07T16:32:40.332 INFO:tasks.workunit.client.0.smithi100.stdout:776 2024-04-07T16:32:40.336 INFO:tasks.workunit.client.0.smithi100.stdout:777 2024-04-07T16:32:40.339 INFO:tasks.workunit.client.0.smithi100.stdout:778 2024-04-07T16:32:40.343 INFO:tasks.workunit.client.0.smithi100.stdout:779 2024-04-07T16:32:40.346 INFO:tasks.workunit.client.0.smithi100.stdout:780 2024-04-07T16:32:40.350 INFO:tasks.workunit.client.0.smithi100.stdout:781 2024-04-07T16:32:40.353 INFO:tasks.workunit.client.0.smithi100.stdout:782 2024-04-07T16:32:40.357 INFO:tasks.workunit.client.0.smithi100.stdout:783 2024-04-07T16:32:40.360 INFO:tasks.workunit.client.0.smithi100.stdout:784 2024-04-07T16:32:40.364 INFO:tasks.workunit.client.0.smithi100.stdout:785 2024-04-07T16:32:40.367 INFO:tasks.workunit.client.0.smithi100.stdout:786 2024-04-07T16:32:40.370 INFO:tasks.workunit.client.0.smithi100.stdout:787 2024-04-07T16:32:40.373 INFO:tasks.workunit.client.0.smithi100.stdout:788 2024-04-07T16:32:40.376 INFO:tasks.workunit.client.0.smithi100.stdout:789 2024-04-07T16:32:40.380 INFO:tasks.workunit.client.0.smithi100.stdout:790 2024-04-07T16:32:40.385 INFO:tasks.workunit.client.0.smithi100.stdout:791 2024-04-07T16:32:40.387 INFO:tasks.workunit.client.0.smithi100.stdout:792 2024-04-07T16:32:40.390 INFO:tasks.workunit.client.0.smithi100.stdout:793 2024-04-07T16:32:40.393 INFO:tasks.workunit.client.0.smithi100.stdout:794 2024-04-07T16:32:40.397 INFO:tasks.workunit.client.0.smithi100.stdout:795 2024-04-07T16:32:40.400 INFO:tasks.workunit.client.0.smithi100.stdout:796 2024-04-07T16:32:40.404 INFO:tasks.workunit.client.0.smithi100.stdout:797 2024-04-07T16:32:40.407 INFO:tasks.workunit.client.0.smithi100.stdout:798 2024-04-07T16:32:40.411 INFO:tasks.workunit.client.0.smithi100.stdout:799 2024-04-07T16:32:40.414 INFO:tasks.workunit.client.0.smithi100.stdout:800 2024-04-07T16:32:40.417 INFO:tasks.workunit.client.0.smithi100.stdout:801 2024-04-07T16:32:40.421 INFO:tasks.workunit.client.0.smithi100.stdout:802 2024-04-07T16:32:40.425 INFO:tasks.workunit.client.0.smithi100.stdout:803 2024-04-07T16:32:40.428 INFO:tasks.workunit.client.0.smithi100.stdout:804 2024-04-07T16:32:40.432 INFO:tasks.workunit.client.0.smithi100.stdout:805 2024-04-07T16:32:40.435 INFO:tasks.workunit.client.0.smithi100.stdout:806 2024-04-07T16:32:40.438 INFO:tasks.workunit.client.0.smithi100.stdout:807 2024-04-07T16:32:40.442 INFO:tasks.workunit.client.0.smithi100.stdout:808 2024-04-07T16:32:40.445 INFO:tasks.workunit.client.0.smithi100.stdout:809 2024-04-07T16:32:40.449 INFO:tasks.workunit.client.0.smithi100.stdout:810 2024-04-07T16:32:40.453 INFO:tasks.workunit.client.0.smithi100.stdout:811 2024-04-07T16:32:40.456 INFO:tasks.workunit.client.0.smithi100.stdout:812 2024-04-07T16:32:40.460 INFO:tasks.workunit.client.0.smithi100.stdout:813 2024-04-07T16:32:40.463 INFO:tasks.workunit.client.0.smithi100.stdout:814 2024-04-07T16:32:40.467 INFO:tasks.workunit.client.0.smithi100.stdout:815 2024-04-07T16:32:40.471 INFO:tasks.workunit.client.0.smithi100.stdout:816 2024-04-07T16:32:40.475 INFO:tasks.workunit.client.0.smithi100.stdout:817 2024-04-07T16:32:40.478 INFO:tasks.workunit.client.0.smithi100.stdout:818 2024-04-07T16:32:40.481 INFO:tasks.workunit.client.0.smithi100.stdout:819 2024-04-07T16:32:40.485 INFO:tasks.workunit.client.0.smithi100.stdout:820 2024-04-07T16:32:40.488 INFO:tasks.workunit.client.0.smithi100.stdout:821 2024-04-07T16:32:40.491 INFO:tasks.workunit.client.0.smithi100.stdout:822 2024-04-07T16:32:40.495 INFO:tasks.workunit.client.0.smithi100.stdout:823 2024-04-07T16:32:40.498 INFO:tasks.workunit.client.0.smithi100.stdout:824 2024-04-07T16:32:40.501 INFO:tasks.workunit.client.0.smithi100.stdout:825 2024-04-07T16:32:40.505 INFO:tasks.workunit.client.0.smithi100.stdout:826 2024-04-07T16:32:40.508 INFO:tasks.workunit.client.0.smithi100.stdout:827 2024-04-07T16:32:40.512 INFO:tasks.workunit.client.0.smithi100.stdout:828 2024-04-07T16:32:40.515 INFO:tasks.workunit.client.0.smithi100.stdout:829 2024-04-07T16:32:40.519 INFO:tasks.workunit.client.0.smithi100.stdout:830 2024-04-07T16:32:40.522 INFO:tasks.workunit.client.0.smithi100.stdout:831 2024-04-07T16:32:40.525 INFO:tasks.workunit.client.0.smithi100.stdout:832 2024-04-07T16:32:40.529 INFO:tasks.workunit.client.0.smithi100.stdout:833 2024-04-07T16:32:40.532 INFO:tasks.workunit.client.0.smithi100.stdout:834 2024-04-07T16:32:40.537 INFO:tasks.workunit.client.0.smithi100.stdout:835 2024-04-07T16:32:40.539 INFO:tasks.workunit.client.0.smithi100.stdout:836 2024-04-07T16:32:40.542 INFO:tasks.workunit.client.0.smithi100.stdout:837 2024-04-07T16:32:40.545 INFO:tasks.workunit.client.0.smithi100.stdout:838 2024-04-07T16:32:40.549 INFO:tasks.workunit.client.0.smithi100.stdout:839 2024-04-07T16:32:40.552 INFO:tasks.workunit.client.0.smithi100.stdout:840 2024-04-07T16:32:40.555 INFO:tasks.workunit.client.0.smithi100.stdout:841 2024-04-07T16:32:40.558 INFO:tasks.workunit.client.0.smithi100.stdout:842 2024-04-07T16:32:40.562 INFO:tasks.workunit.client.0.smithi100.stdout:843 2024-04-07T16:32:40.565 INFO:tasks.workunit.client.0.smithi100.stdout:844 2024-04-07T16:32:40.569 INFO:tasks.workunit.client.0.smithi100.stdout:845 2024-04-07T16:32:40.573 INFO:tasks.workunit.client.0.smithi100.stdout:846 2024-04-07T16:32:40.577 INFO:tasks.workunit.client.0.smithi100.stdout:847 2024-04-07T16:32:40.580 INFO:tasks.workunit.client.0.smithi100.stdout:848 2024-04-07T16:32:40.584 INFO:tasks.workunit.client.0.smithi100.stdout:849 2024-04-07T16:32:40.587 INFO:tasks.workunit.client.0.smithi100.stdout:850 2024-04-07T16:32:40.591 INFO:tasks.workunit.client.0.smithi100.stdout:851 2024-04-07T16:32:40.595 INFO:tasks.workunit.client.0.smithi100.stdout:852 2024-04-07T16:32:40.598 INFO:tasks.workunit.client.0.smithi100.stdout:853 2024-04-07T16:32:40.601 INFO:tasks.workunit.client.0.smithi100.stdout:854 2024-04-07T16:32:40.605 INFO:tasks.workunit.client.0.smithi100.stdout:855 2024-04-07T16:32:40.608 INFO:tasks.workunit.client.0.smithi100.stdout:856 2024-04-07T16:32:40.611 INFO:tasks.workunit.client.0.smithi100.stdout:857 2024-04-07T16:32:40.614 INFO:tasks.workunit.client.0.smithi100.stdout:858 2024-04-07T16:32:40.617 INFO:tasks.workunit.client.0.smithi100.stdout:859 2024-04-07T16:32:40.620 INFO:tasks.workunit.client.0.smithi100.stdout:860 2024-04-07T16:32:40.623 INFO:tasks.workunit.client.0.smithi100.stdout:861 2024-04-07T16:32:40.627 INFO:tasks.workunit.client.0.smithi100.stdout:862 2024-04-07T16:32:40.630 INFO:tasks.workunit.client.0.smithi100.stdout:863 2024-04-07T16:32:40.634 INFO:tasks.workunit.client.0.smithi100.stdout:864 2024-04-07T16:32:40.637 INFO:tasks.workunit.client.0.smithi100.stdout:865 2024-04-07T16:32:40.640 INFO:tasks.workunit.client.0.smithi100.stdout:866 2024-04-07T16:32:40.644 INFO:tasks.workunit.client.0.smithi100.stdout:867 2024-04-07T16:32:40.648 INFO:tasks.workunit.client.0.smithi100.stdout:868 2024-04-07T16:32:40.651 INFO:tasks.workunit.client.0.smithi100.stdout:869 2024-04-07T16:32:40.655 INFO:tasks.workunit.client.0.smithi100.stdout:870 2024-04-07T16:32:40.659 INFO:tasks.workunit.client.0.smithi100.stdout:871 2024-04-07T16:32:40.662 INFO:tasks.workunit.client.0.smithi100.stdout:872 2024-04-07T16:32:40.665 INFO:tasks.workunit.client.0.smithi100.stdout:873 2024-04-07T16:32:40.669 INFO:tasks.workunit.client.0.smithi100.stdout:874 2024-04-07T16:32:40.672 INFO:tasks.workunit.client.0.smithi100.stdout:875 2024-04-07T16:32:40.676 INFO:tasks.workunit.client.0.smithi100.stdout:876 2024-04-07T16:32:40.679 INFO:tasks.workunit.client.0.smithi100.stdout:877 2024-04-07T16:32:40.682 INFO:tasks.workunit.client.0.smithi100.stdout:878 2024-04-07T16:32:40.686 INFO:tasks.workunit.client.0.smithi100.stdout:879 2024-04-07T16:32:40.689 INFO:tasks.workunit.client.0.smithi100.stdout:880 2024-04-07T16:32:40.692 INFO:tasks.workunit.client.0.smithi100.stdout:881 2024-04-07T16:32:40.696 INFO:tasks.workunit.client.0.smithi100.stdout:882 2024-04-07T16:32:40.700 INFO:tasks.workunit.client.0.smithi100.stdout:883 2024-04-07T16:32:40.704 INFO:tasks.workunit.client.0.smithi100.stdout:884 2024-04-07T16:32:40.708 INFO:tasks.workunit.client.0.smithi100.stdout:885 2024-04-07T16:32:40.711 INFO:tasks.workunit.client.0.smithi100.stdout:886 2024-04-07T16:32:40.714 INFO:tasks.workunit.client.0.smithi100.stdout:887 2024-04-07T16:32:40.717 INFO:tasks.workunit.client.0.smithi100.stdout:888 2024-04-07T16:32:40.720 INFO:tasks.workunit.client.0.smithi100.stdout:889 2024-04-07T16:32:40.724 INFO:tasks.workunit.client.0.smithi100.stdout:890 2024-04-07T16:32:40.727 INFO:tasks.workunit.client.0.smithi100.stdout:891 2024-04-07T16:32:40.730 INFO:tasks.workunit.client.0.smithi100.stdout:892 2024-04-07T16:32:40.734 INFO:tasks.workunit.client.0.smithi100.stdout:893 2024-04-07T16:32:40.737 INFO:tasks.workunit.client.0.smithi100.stdout:894 2024-04-07T16:32:40.740 INFO:tasks.workunit.client.0.smithi100.stdout:895 2024-04-07T16:32:40.744 INFO:tasks.workunit.client.0.smithi100.stdout:896 2024-04-07T16:32:40.747 INFO:tasks.workunit.client.0.smithi100.stdout:897 2024-04-07T16:32:40.751 INFO:tasks.workunit.client.0.smithi100.stdout:898 2024-04-07T16:32:40.755 INFO:tasks.workunit.client.0.smithi100.stdout:899 2024-04-07T16:32:40.758 INFO:tasks.workunit.client.0.smithi100.stdout:900 2024-04-07T16:32:40.762 INFO:tasks.workunit.client.0.smithi100.stdout:901 2024-04-07T16:32:40.765 INFO:tasks.workunit.client.0.smithi100.stdout:902 2024-04-07T16:32:40.769 INFO:tasks.workunit.client.0.smithi100.stdout:903 2024-04-07T16:32:40.772 INFO:tasks.workunit.client.0.smithi100.stdout:904 2024-04-07T16:32:40.775 INFO:tasks.workunit.client.0.smithi100.stdout:905 2024-04-07T16:32:40.779 INFO:tasks.workunit.client.0.smithi100.stdout:906 2024-04-07T16:32:40.782 INFO:tasks.workunit.client.0.smithi100.stdout:907 2024-04-07T16:32:40.785 INFO:tasks.workunit.client.0.smithi100.stdout:908 2024-04-07T16:32:40.789 INFO:tasks.workunit.client.0.smithi100.stdout:909 2024-04-07T16:32:40.792 INFO:tasks.workunit.client.0.smithi100.stdout:910 2024-04-07T16:32:40.795 INFO:tasks.workunit.client.0.smithi100.stdout:911 2024-04-07T16:32:40.798 INFO:tasks.workunit.client.0.smithi100.stdout:912 2024-04-07T16:32:40.801 INFO:tasks.workunit.client.0.smithi100.stdout:913 2024-04-07T16:32:40.804 INFO:tasks.workunit.client.0.smithi100.stdout:914 2024-04-07T16:32:40.808 INFO:tasks.workunit.client.0.smithi100.stdout:915 2024-04-07T16:32:40.811 INFO:tasks.workunit.client.0.smithi100.stdout:916 2024-04-07T16:32:40.814 INFO:tasks.workunit.client.0.smithi100.stdout:917 2024-04-07T16:32:40.818 INFO:tasks.workunit.client.0.smithi100.stdout:918 2024-04-07T16:32:40.822 INFO:tasks.workunit.client.0.smithi100.stdout:919 2024-04-07T16:32:40.825 INFO:tasks.workunit.client.0.smithi100.stdout:920 2024-04-07T16:32:40.829 INFO:tasks.workunit.client.0.smithi100.stdout:921 2024-04-07T16:32:40.832 INFO:tasks.workunit.client.0.smithi100.stdout:922 2024-04-07T16:32:40.835 INFO:tasks.workunit.client.0.smithi100.stdout:923 2024-04-07T16:32:40.839 INFO:tasks.workunit.client.0.smithi100.stdout:924 2024-04-07T16:32:40.843 INFO:tasks.workunit.client.0.smithi100.stdout:925 2024-04-07T16:32:40.847 INFO:tasks.workunit.client.0.smithi100.stdout:926 2024-04-07T16:32:40.850 INFO:tasks.workunit.client.0.smithi100.stdout:927 2024-04-07T16:32:40.853 INFO:tasks.workunit.client.0.smithi100.stdout:928 2024-04-07T16:32:40.857 INFO:tasks.workunit.client.0.smithi100.stdout:929 2024-04-07T16:32:40.860 INFO:tasks.workunit.client.0.smithi100.stdout:930 2024-04-07T16:32:40.863 INFO:tasks.workunit.client.0.smithi100.stdout:931 2024-04-07T16:32:40.866 INFO:tasks.workunit.client.0.smithi100.stdout:932 2024-04-07T16:32:40.870 INFO:tasks.workunit.client.0.smithi100.stdout:933 2024-04-07T16:32:40.873 INFO:tasks.workunit.client.0.smithi100.stdout:934 2024-04-07T16:32:40.876 INFO:tasks.workunit.client.0.smithi100.stdout:935 2024-04-07T16:32:40.879 INFO:tasks.workunit.client.0.smithi100.stdout:936 2024-04-07T16:32:40.883 INFO:tasks.workunit.client.0.smithi100.stdout:937 2024-04-07T16:32:40.886 INFO:tasks.workunit.client.0.smithi100.stdout:938 2024-04-07T16:32:40.889 INFO:tasks.workunit.client.0.smithi100.stdout:939 2024-04-07T16:32:40.893 INFO:tasks.workunit.client.0.smithi100.stdout:940 2024-04-07T16:32:40.896 INFO:tasks.workunit.client.0.smithi100.stdout:941 2024-04-07T16:32:40.900 INFO:tasks.workunit.client.0.smithi100.stdout:942 2024-04-07T16:32:40.903 INFO:tasks.workunit.client.0.smithi100.stdout:943 2024-04-07T16:32:40.907 INFO:tasks.workunit.client.0.smithi100.stdout:944 2024-04-07T16:32:40.912 INFO:tasks.workunit.client.0.smithi100.stdout:945 2024-04-07T16:32:40.915 INFO:tasks.workunit.client.0.smithi100.stdout:946 2024-04-07T16:32:40.918 INFO:tasks.workunit.client.0.smithi100.stdout:947 2024-04-07T16:32:40.922 INFO:tasks.workunit.client.0.smithi100.stdout:948 2024-04-07T16:32:40.925 INFO:tasks.workunit.client.0.smithi100.stdout:949 2024-04-07T16:32:40.928 INFO:tasks.workunit.client.0.smithi100.stdout:950 2024-04-07T16:32:40.931 INFO:tasks.workunit.client.0.smithi100.stdout:951 2024-04-07T16:32:40.935 INFO:tasks.workunit.client.0.smithi100.stdout:952 2024-04-07T16:32:40.938 INFO:tasks.workunit.client.0.smithi100.stdout:953 2024-04-07T16:32:40.941 INFO:tasks.workunit.client.0.smithi100.stdout:954 2024-04-07T16:32:40.944 INFO:tasks.workunit.client.0.smithi100.stdout:955 2024-04-07T16:32:40.948 INFO:tasks.workunit.client.0.smithi100.stdout:956 2024-04-07T16:32:40.951 INFO:tasks.workunit.client.0.smithi100.stdout:957 2024-04-07T16:32:40.954 INFO:tasks.workunit.client.0.smithi100.stdout:958 2024-04-07T16:32:40.957 INFO:tasks.workunit.client.0.smithi100.stdout:959 2024-04-07T16:32:40.960 INFO:tasks.workunit.client.0.smithi100.stdout:960 2024-04-07T16:32:40.963 INFO:tasks.workunit.client.0.smithi100.stdout:961 2024-04-07T16:32:40.966 INFO:tasks.workunit.client.0.smithi100.stdout:962 2024-04-07T16:32:40.970 INFO:tasks.workunit.client.0.smithi100.stdout:963 2024-04-07T16:32:40.973 INFO:tasks.workunit.client.0.smithi100.stdout:964 2024-04-07T16:32:40.976 INFO:tasks.workunit.client.0.smithi100.stdout:965 2024-04-07T16:32:40.980 INFO:tasks.workunit.client.0.smithi100.stdout:966 2024-04-07T16:32:40.984 INFO:tasks.workunit.client.0.smithi100.stdout:967 2024-04-07T16:32:40.987 INFO:tasks.workunit.client.0.smithi100.stdout:968 2024-04-07T16:32:40.991 INFO:tasks.workunit.client.0.smithi100.stdout:969 2024-04-07T16:32:40.994 INFO:tasks.workunit.client.0.smithi100.stdout:970 2024-04-07T16:32:40.998 INFO:tasks.workunit.client.0.smithi100.stdout:971 2024-04-07T16:32:41.001 INFO:tasks.workunit.client.0.smithi100.stdout:972 2024-04-07T16:32:41.005 INFO:tasks.workunit.client.0.smithi100.stdout:973 2024-04-07T16:32:41.009 INFO:tasks.workunit.client.0.smithi100.stdout:974 2024-04-07T16:32:41.012 INFO:tasks.workunit.client.0.smithi100.stdout:975 2024-04-07T16:32:41.015 INFO:tasks.workunit.client.0.smithi100.stdout:976 2024-04-07T16:32:41.019 INFO:tasks.workunit.client.0.smithi100.stdout:977 2024-04-07T16:32:41.023 INFO:tasks.workunit.client.0.smithi100.stdout:978 2024-04-07T16:32:41.026 INFO:tasks.workunit.client.0.smithi100.stdout:979 2024-04-07T16:32:41.029 INFO:tasks.workunit.client.0.smithi100.stdout:980 2024-04-07T16:32:41.033 INFO:tasks.workunit.client.0.smithi100.stdout:981 2024-04-07T16:32:41.037 INFO:tasks.workunit.client.0.smithi100.stdout:982 2024-04-07T16:32:41.041 INFO:tasks.workunit.client.0.smithi100.stdout:983 2024-04-07T16:32:41.044 INFO:tasks.workunit.client.0.smithi100.stdout:984 2024-04-07T16:32:41.048 INFO:tasks.workunit.client.0.smithi100.stdout:985 2024-04-07T16:32:41.052 INFO:tasks.workunit.client.0.smithi100.stdout:986 2024-04-07T16:32:41.056 INFO:tasks.workunit.client.0.smithi100.stdout:987 2024-04-07T16:32:41.059 INFO:tasks.workunit.client.0.smithi100.stdout:988 2024-04-07T16:32:41.062 INFO:tasks.workunit.client.0.smithi100.stdout:989 2024-04-07T16:32:41.066 INFO:tasks.workunit.client.0.smithi100.stdout:990 2024-04-07T16:32:41.069 INFO:tasks.workunit.client.0.smithi100.stdout:991 2024-04-07T16:32:41.073 INFO:tasks.workunit.client.0.smithi100.stdout:992 2024-04-07T16:32:41.076 INFO:tasks.workunit.client.0.smithi100.stdout:993 2024-04-07T16:32:41.080 INFO:tasks.workunit.client.0.smithi100.stdout:994 2024-04-07T16:32:41.084 INFO:tasks.workunit.client.0.smithi100.stdout:995 2024-04-07T16:32:41.088 INFO:tasks.workunit.client.0.smithi100.stdout:996 2024-04-07T16:32:41.092 INFO:tasks.workunit.client.0.smithi100.stdout:997 2024-04-07T16:32:41.095 INFO:tasks.workunit.client.0.smithi100.stdout:998 2024-04-07T16:32:41.098 INFO:tasks.workunit.client.0.smithi100.stdout:999 2024-04-07T16:32:41.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:32:40 smithi144 bash[66654]: cluster 2024-04-07T16:32:39.704473+0000 mon.a (mon.0) 1629 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T16:32:41.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:32:40 smithi144 bash[66654]: cluster 2024-04-07T16:32:39.704542+0000 mon.a (mon.0) 1630 : cluster [INF] Cluster is now healthy 2024-04-07T16:32:41.189 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:32:40 smithi100 bash[74942]: cluster 2024-04-07T16:32:39.704473+0000 mon.a (mon.0) 1629 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T16:32:41.189 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:32:40 smithi100 bash[74942]: cluster 2024-04-07T16:32:39.704542+0000 mon.a (mon.0) 1630 : cluster [INF] Cluster is now healthy 2024-04-07T16:32:41.190 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:32:40 smithi100 bash[76202]: cluster 2024-04-07T16:32:39.704473+0000 mon.a (mon.0) 1629 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T16:32:41.190 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:32:40 smithi100 bash[76202]: cluster 2024-04-07T16:32:39.704542+0000 mon.a (mon.0) 1630 : cluster [INF] Cluster is now healthy 2024-04-07T16:32:41.718 INFO:tasks.workunit.client.0.smithi100.stdout:00000000 79 6f 75 20 6d 69 67 68 74 20 73 65 65 20 74 68 |you might see th| 2024-04-07T16:32:41.718 INFO:tasks.workunit.client.0.smithi100.stdout:00000010 69 73 |is| 2024-04-07T16:32:41.718 INFO:tasks.workunit.client.0.smithi100.stdout:00000012 2024-04-07T16:32:41.718 INFO:tasks.workunit.client.0.smithi100.stdout:00000000 79 6f 75 20 6d 69 67 68 74 20 73 65 65 20 74 68 |you might see th| 2024-04-07T16:32:41.718 INFO:tasks.workunit.client.0.smithi100.stdout:00000010 69 73 |is| 2024-04-07T16:32:41.718 INFO:tasks.workunit.client.0.smithi100.stdout:00000012 2024-04-07T16:32:41.718 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] ClsHello.WriteReturnData (6045 ms) 2024-04-07T16:32:41.718 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] ClsHello.Loud 2024-04-07T16:32:42.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:32:41 smithi144 bash[66654]: cluster 2024-04-07T16:32:40.920194+0000 mgr.y (mgr.25175) 520 : cluster [DBG] pgmap v1002: 137 pgs: 2 creating+activating, 12 creating+peering, 123 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 0 B/s wr, 0 op/s 2024-04-07T16:32:42.189 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:32:41 smithi100 bash[74942]: cluster 2024-04-07T16:32:40.920194+0000 mgr.y (mgr.25175) 520 : cluster [DBG] pgmap v1002: 137 pgs: 2 creating+activating, 12 creating+peering, 123 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 0 B/s wr, 0 op/s 2024-04-07T16:32:42.189 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:32:41 smithi100 bash[76202]: cluster 2024-04-07T16:32:40.920194+0000 mgr.y (mgr.25175) 520 : cluster [DBG] pgmap v1002: 137 pgs: 2 creating+activating, 12 creating+peering, 123 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 0 B/s wr, 0 op/s 2024-04-07T16:32:42.607 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:32:42 smithi144 bash[95233]: ts=2024-04-07T16:32:42.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:32:43.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:32:42 smithi144 bash[66654]: cluster 2024-04-07T16:32:41.726565+0000 mon.a (mon.0) 1631 : cluster [DBG] osdmap e908: 8 total, 8 up, 8 in 2024-04-07T16:32:43.189 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:32:42 smithi100 bash[74942]: cluster 2024-04-07T16:32:41.726565+0000 mon.a (mon.0) 1631 : cluster [DBG] osdmap e908: 8 total, 8 up, 8 in 2024-04-07T16:32:43.189 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:32:42 smithi100 bash[76202]: cluster 2024-04-07T16:32:41.726565+0000 mon.a (mon.0) 1631 : cluster [DBG] osdmap e908: 8 total, 8 up, 8 in 2024-04-07T16:32:44.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:32:43 smithi144 bash[66654]: cluster 2024-04-07T16:32:42.738008+0000 mon.a (mon.0) 1632 : cluster [DBG] osdmap e909: 8 total, 8 up, 8 in 2024-04-07T16:32:44.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:32:43 smithi144 bash[66654]: audit 2024-04-07T16:32:42.743034+0000 mon.c (mon.2) 39 : audit [INF] from='client.? 172.21.15.100:0/3641533639' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110598-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:32:44.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:32:43 smithi144 bash[66654]: audit 2024-04-07T16:32:42.744582+0000 mon.a (mon.0) 1633 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110598-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:32:44.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:32:43 smithi144 bash[66654]: cluster 2024-04-07T16:32:42.920910+0000 mgr.y (mgr.25175) 521 : cluster [DBG] pgmap v1005: 137 pgs: 32 unknown, 105 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:32:44.189 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:32:43 smithi100 bash[74942]: cluster 2024-04-07T16:32:42.738008+0000 mon.a (mon.0) 1632 : cluster [DBG] osdmap e909: 8 total, 8 up, 8 in 2024-04-07T16:32:44.189 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:32:43 smithi100 bash[74942]: audit 2024-04-07T16:32:42.743034+0000 mon.c (mon.2) 39 : audit [INF] from='client.? 172.21.15.100:0/3641533639' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110598-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:32:44.190 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:32:43 smithi100 bash[74942]: audit 2024-04-07T16:32:42.744582+0000 mon.a (mon.0) 1633 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110598-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:32:44.190 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:32:43 smithi100 bash[74942]: cluster 2024-04-07T16:32:42.920910+0000 mgr.y (mgr.25175) 521 : cluster [DBG] pgmap v1005: 137 pgs: 32 unknown, 105 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:32:44.190 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:32:43 smithi100 bash[76202]: cluster 2024-04-07T16:32:42.738008+0000 mon.a (mon.0) 1632 : cluster [DBG] osdmap e909: 8 total, 8 up, 8 in 2024-04-07T16:32:44.190 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:32:43 smithi100 bash[76202]: audit 2024-04-07T16:32:42.743034+0000 mon.c (mon.2) 39 : audit [INF] from='client.? 172.21.15.100:0/3641533639' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110598-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:32:44.190 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:32:43 smithi100 bash[76202]: audit 2024-04-07T16:32:42.744582+0000 mon.a (mon.0) 1633 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110598-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:32:44.190 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:32:43 smithi100 bash[76202]: cluster 2024-04-07T16:32:42.920910+0000 mgr.y (mgr.25175) 521 : cluster [DBG] pgmap v1005: 137 pgs: 32 unknown, 105 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:32:44.745 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] ClsHello.Loud (3027 ms) 2024-04-07T16:32:44.745 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] ClsHello.BadMethods 2024-04-07T16:32:45.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:32:44 smithi144 bash[66654]: cluster 2024-04-07T16:32:43.730657+0000 mon.a (mon.0) 1634 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T16:32:45.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:32:44 smithi144 bash[66654]: audit 2024-04-07T16:32:43.740591+0000 mon.a (mon.0) 1635 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110598-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:32:45.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:32:44 smithi144 bash[66654]: cluster 2024-04-07T16:32:43.750812+0000 mon.a (mon.0) 1636 : cluster [DBG] osdmap e910: 8 total, 8 up, 8 in 2024-04-07T16:32:45.189 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:32:44 smithi100 bash[74942]: cluster 2024-04-07T16:32:43.730657+0000 mon.a (mon.0) 1634 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T16:32:45.189 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:32:44 smithi100 bash[74942]: audit 2024-04-07T16:32:43.740591+0000 mon.a (mon.0) 1635 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110598-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:32:45.190 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:32:44 smithi100 bash[74942]: cluster 2024-04-07T16:32:43.750812+0000 mon.a (mon.0) 1636 : cluster [DBG] osdmap e910: 8 total, 8 up, 8 in 2024-04-07T16:32:45.190 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:32:44 smithi100 bash[76202]: cluster 2024-04-07T16:32:43.730657+0000 mon.a (mon.0) 1634 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T16:32:45.190 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:32:44 smithi100 bash[76202]: audit 2024-04-07T16:32:43.740591+0000 mon.a (mon.0) 1635 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110598-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:32:45.190 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:32:44 smithi100 bash[76202]: cluster 2024-04-07T16:32:43.750812+0000 mon.a (mon.0) 1636 : cluster [DBG] osdmap e910: 8 total, 8 up, 8 in 2024-04-07T16:32:46.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:32:45 smithi144 bash[66654]: cluster 2024-04-07T16:32:44.753293+0000 mon.a (mon.0) 1637 : cluster [DBG] osdmap e911: 8 total, 8 up, 8 in 2024-04-07T16:32:46.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:32:45 smithi144 bash[66654]: cluster 2024-04-07T16:32:44.921538+0000 mgr.y (mgr.25175) 522 : cluster [DBG] pgmap v1008: 105 pgs: 105 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:32:46.189 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:32:45 smithi100 bash[74942]: cluster 2024-04-07T16:32:44.753293+0000 mon.a (mon.0) 1637 : cluster [DBG] osdmap e911: 8 total, 8 up, 8 in 2024-04-07T16:32:46.189 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:32:45 smithi100 bash[74942]: cluster 2024-04-07T16:32:44.921538+0000 mgr.y (mgr.25175) 522 : cluster [DBG] pgmap v1008: 105 pgs: 105 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:32:46.190 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:32:45 smithi100 bash[76202]: cluster 2024-04-07T16:32:44.753293+0000 mon.a (mon.0) 1637 : cluster [DBG] osdmap e911: 8 total, 8 up, 8 in 2024-04-07T16:32:46.190 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:32:45 smithi100 bash[76202]: cluster 2024-04-07T16:32:44.921538+0000 mgr.y (mgr.25175) 522 : cluster [DBG] pgmap v1008: 105 pgs: 105 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:32:47.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:32:46 smithi144 bash[66654]: cluster 2024-04-07T16:32:45.765944+0000 mon.a (mon.0) 1638 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T16:32:47.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:32:46 smithi144 bash[66654]: cluster 2024-04-07T16:32:45.766007+0000 mon.a (mon.0) 1639 : cluster [INF] Cluster is now healthy 2024-04-07T16:32:47.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:32:46 smithi144 bash[66654]: cluster 2024-04-07T16:32:45.786114+0000 mon.a (mon.0) 1640 : cluster [DBG] osdmap e912: 8 total, 8 up, 8 in 2024-04-07T16:32:47.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:32:46 smithi144 bash[66654]: audit 2024-04-07T16:32:45.789744+0000 mon.a (mon.0) 1641 : audit [INF] from='client.? 172.21.15.100:0/2786192756' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110598-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:32:47.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:32:46 smithi144 bash[66654]: audit 2024-04-07T16:32:46.750162+0000 mon.a (mon.0) 1642 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:32:47.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:32:46 smithi144 bash[66654]: audit 2024-04-07T16:32:46.752434+0000 mon.a (mon.0) 1643 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:32:47.189 INFO:journalctl@ceph.osd.3.smithi100.stdout:Apr 07 16:32:46 smithi100 bash[93998]: debug 2024-04-07T16:32:46.826+0000 7f6e91a8a700 -1 osd.3 pg_epoch: 913 pg[38.2( v 913'1 (0'0,913'1] local-lis/les=912/913 n=1 ec=912/912 lis/c=912/912 les/c/f=913/913/0 sis=912) [3,7,5] r=0 lpr=912 crt=913'1 lcod 0'0 mlcod 0'0 active+clean] method hello.bad_reader tried to read object but is not marked RD 2024-04-07T16:32:47.189 INFO:journalctl@ceph.osd.3.smithi100.stdout:Apr 07 16:32:46 smithi100 bash[93998]: debug 2024-04-07T16:32:46.830+0000 7f6e91a8a700 -1 osd.3 pg_epoch: 913 pg[38.2( v 913'2 (0'0,913'2] local-lis/les=912/913 n=1 ec=912/912 lis/c=912/912 les/c/f=913/913/0 sis=912) [3,7,5] r=0 lpr=912 crt=913'1 lcod 913'1 mlcod 913'1 active+clean] method hello.bad_writer tried to update object but is not marked WR 2024-04-07T16:32:47.190 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:32:46 smithi100 bash[74942]: cluster 2024-04-07T16:32:45.765944+0000 mon.a (mon.0) 1638 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T16:32:47.190 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:32:46 smithi100 bash[74942]: cluster 2024-04-07T16:32:45.766007+0000 mon.a (mon.0) 1639 : cluster [INF] Cluster is now healthy 2024-04-07T16:32:47.190 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:32:46 smithi100 bash[74942]: cluster 2024-04-07T16:32:45.786114+0000 mon.a (mon.0) 1640 : cluster [DBG] osdmap e912: 8 total, 8 up, 8 in 2024-04-07T16:32:47.190 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:32:46 smithi100 bash[74942]: audit 2024-04-07T16:32:45.789744+0000 mon.a (mon.0) 1641 : audit [INF] from='client.? 172.21.15.100:0/2786192756' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110598-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:32:47.190 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:32:46 smithi100 bash[74942]: audit 2024-04-07T16:32:46.750162+0000 mon.a (mon.0) 1642 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:32:47.190 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:32:46 smithi100 bash[74942]: audit 2024-04-07T16:32:46.752434+0000 mon.a (mon.0) 1643 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:32:47.190 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:32:46 smithi100 bash[76202]: cluster 2024-04-07T16:32:45.765944+0000 mon.a (mon.0) 1638 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T16:32:47.191 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:32:46 smithi100 bash[76202]: cluster 2024-04-07T16:32:45.766007+0000 mon.a (mon.0) 1639 : cluster [INF] Cluster is now healthy 2024-04-07T16:32:47.191 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:32:46 smithi100 bash[76202]: cluster 2024-04-07T16:32:45.786114+0000 mon.a (mon.0) 1640 : cluster [DBG] osdmap e912: 8 total, 8 up, 8 in 2024-04-07T16:32:47.191 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:32:46 smithi100 bash[76202]: audit 2024-04-07T16:32:45.789744+0000 mon.a (mon.0) 1641 : audit [INF] from='client.? 172.21.15.100:0/2786192756' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110598-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:32:47.191 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:32:46 smithi100 bash[76202]: audit 2024-04-07T16:32:46.750162+0000 mon.a (mon.0) 1642 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:32:47.191 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:32:46 smithi100 bash[76202]: audit 2024-04-07T16:32:46.752434+0000 mon.a (mon.0) 1643 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:32:47.793 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] ClsHello.BadMethods (3048 ms) 2024-04-07T16:32:47.793 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] ClsHello.Filter 2024-04-07T16:32:48.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:32:47 smithi144 bash[66654]: audit 2024-04-07T16:32:46.782403+0000 mon.a (mon.0) 1644 : audit [INF] from='client.? 172.21.15.100:0/2786192756' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110598-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:32:48.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:32:47 smithi144 bash[66654]: cluster 2024-04-07T16:32:46.793419+0000 mon.a (mon.0) 1645 : cluster [DBG] osdmap e913: 8 total, 8 up, 8 in 2024-04-07T16:32:48.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:32:47 smithi144 bash[66654]: cluster 2024-04-07T16:32:46.922168+0000 mgr.y (mgr.25175) 523 : cluster [DBG] pgmap v1011: 137 pgs: 32 unknown, 105 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:32:48.189 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:32:47 smithi100 bash[74942]: audit 2024-04-07T16:32:46.782403+0000 mon.a (mon.0) 1644 : audit [INF] from='client.? 172.21.15.100:0/2786192756' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110598-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:32:48.214 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:32:47 smithi100 bash[74942]: cluster 2024-04-07T16:32:46.793419+0000 mon.a (mon.0) 1645 : cluster [DBG] osdmap e913: 8 total, 8 up, 8 in 2024-04-07T16:32:48.214 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:32:47 smithi100 bash[74942]: cluster 2024-04-07T16:32:46.922168+0000 mgr.y (mgr.25175) 523 : cluster [DBG] pgmap v1011: 137 pgs: 32 unknown, 105 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:32:48.214 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:32:47 smithi100 bash[76202]: audit 2024-04-07T16:32:46.782403+0000 mon.a (mon.0) 1644 : audit [INF] from='client.? 172.21.15.100:0/2786192756' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110598-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:32:48.214 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:32:47 smithi100 bash[76202]: cluster 2024-04-07T16:32:46.793419+0000 mon.a (mon.0) 1645 : cluster [DBG] osdmap e913: 8 total, 8 up, 8 in 2024-04-07T16:32:48.214 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:32:47 smithi100 bash[76202]: cluster 2024-04-07T16:32:46.922168+0000 mgr.y (mgr.25175) 523 : cluster [DBG] pgmap v1011: 137 pgs: 32 unknown, 105 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:32:48.689 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:32:48 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:32:48] "GET /metrics HTTP/1.1" 200 36357 "" "Prometheus/2.43.0" 2024-04-07T16:32:49.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:32:48 smithi144 bash[66654]: cluster 2024-04-07T16:32:47.803109+0000 mon.a (mon.0) 1646 : cluster [DBG] osdmap e914: 8 total, 8 up, 8 in 2024-04-07T16:32:49.189 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:32:48 smithi100 bash[74942]: cluster 2024-04-07T16:32:47.803109+0000 mon.a (mon.0) 1646 : cluster [DBG] osdmap e914: 8 total, 8 up, 8 in 2024-04-07T16:32:49.189 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:32:48 smithi100 bash[76202]: cluster 2024-04-07T16:32:47.803109+0000 mon.a (mon.0) 1646 : cluster [DBG] osdmap e914: 8 total, 8 up, 8 in 2024-04-07T16:32:50.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:32:49 smithi144 bash[66654]: cluster 2024-04-07T16:32:48.812916+0000 mon.a (mon.0) 1647 : cluster [DBG] osdmap e915: 8 total, 8 up, 8 in 2024-04-07T16:32:50.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:32:49 smithi144 bash[66654]: audit 2024-04-07T16:32:48.817292+0000 mon.a (mon.0) 1648 : audit [INF] from='client.? 172.21.15.100:0/3739952214' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110598-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:32:50.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:32:49 smithi144 bash[66654]: cluster 2024-04-07T16:32:48.922946+0000 mgr.y (mgr.25175) 524 : cluster [DBG] pgmap v1014: 137 pgs: 32 unknown, 105 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:32:50.189 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:32:49 smithi100 bash[74942]: cluster 2024-04-07T16:32:48.812916+0000 mon.a (mon.0) 1647 : cluster [DBG] osdmap e915: 8 total, 8 up, 8 in 2024-04-07T16:32:50.190 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:32:49 smithi100 bash[74942]: audit 2024-04-07T16:32:48.817292+0000 mon.a (mon.0) 1648 : audit [INF] from='client.? 172.21.15.100:0/3739952214' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110598-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:32:50.190 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:32:49 smithi100 bash[74942]: cluster 2024-04-07T16:32:48.922946+0000 mgr.y (mgr.25175) 524 : cluster [DBG] pgmap v1014: 137 pgs: 32 unknown, 105 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:32:50.190 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:32:49 smithi100 bash[76202]: cluster 2024-04-07T16:32:48.812916+0000 mon.a (mon.0) 1647 : cluster [DBG] osdmap e915: 8 total, 8 up, 8 in 2024-04-07T16:32:50.190 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:32:49 smithi100 bash[76202]: audit 2024-04-07T16:32:48.817292+0000 mon.a (mon.0) 1648 : audit [INF] from='client.? 172.21.15.100:0/3739952214' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110598-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:32:50.190 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:32:49 smithi100 bash[76202]: cluster 2024-04-07T16:32:48.922946+0000 mgr.y (mgr.25175) 524 : cluster [DBG] pgmap v1014: 137 pgs: 32 unknown, 105 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:32:50.825 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] ClsHello.Filter (3032 ms) 2024-04-07T16:32:50.826 INFO:tasks.workunit.client.0.smithi100.stdout:[----------] 6 tests from ClsHello (21100 ms total) 2024-04-07T16:32:50.826 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T16:32:50.826 INFO:tasks.workunit.client.0.smithi100.stdout:[----------] Global test environment tear-down 2024-04-07T16:32:50.826 INFO:tasks.workunit.client.0.smithi100.stdout:[==========] 6 tests from 1 test suite ran. (21100 ms total) 2024-04-07T16:32:50.826 INFO:tasks.workunit.client.0.smithi100.stdout:[ PASSED ] 6 tests. 2024-04-07T16:32:50.832 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-07T16:32:50.832 DEBUG:teuthology.orchestra.run.smithi100:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-07T16:32:50.888 INFO:tasks.workunit:Running workunit cls/test_cls_journal.sh... 2024-04-07T16:32:50.888 DEBUG:teuthology.orchestra.run.smithi100:workunit test cls/test_cls_journal.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=reef TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_journal.sh 2024-04-07T16:32:50.954 INFO:tasks.workunit.client.0.smithi100.stdout:Running main() from gmock_main.cc 2024-04-07T16:32:50.954 INFO:tasks.workunit.client.0.smithi100.stdout:[==========] Running 27 tests from 1 test suite. 2024-04-07T16:32:50.954 INFO:tasks.workunit.client.0.smithi100.stdout:[----------] Global test environment set-up. 2024-04-07T16:32:50.954 INFO:tasks.workunit.client.0.smithi100.stdout:[----------] 27 tests from TestClsJournal 2024-04-07T16:32:51.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:32:50 smithi144 bash[66654]: cluster 2024-04-07T16:32:49.805564+0000 mon.a (mon.0) 1649 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T16:32:51.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:32:50 smithi144 bash[66654]: audit 2024-04-07T16:32:49.814923+0000 mon.a (mon.0) 1650 : audit [INF] from='client.? 172.21.15.100:0/3739952214' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110598-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:32:51.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:32:50 smithi144 bash[66654]: cluster 2024-04-07T16:32:49.824434+0000 mon.a (mon.0) 1651 : cluster [DBG] osdmap e916: 8 total, 8 up, 8 in 2024-04-07T16:32:51.189 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:32:50 smithi100 bash[74942]: cluster 2024-04-07T16:32:49.805564+0000 mon.a (mon.0) 1649 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T16:32:51.190 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:32:50 smithi100 bash[74942]: audit 2024-04-07T16:32:49.814923+0000 mon.a (mon.0) 1650 : audit [INF] from='client.? 172.21.15.100:0/3739952214' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110598-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:32:51.190 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:32:50 smithi100 bash[74942]: cluster 2024-04-07T16:32:49.824434+0000 mon.a (mon.0) 1651 : cluster [DBG] osdmap e916: 8 total, 8 up, 8 in 2024-04-07T16:32:51.190 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:32:50 smithi100 bash[76202]: cluster 2024-04-07T16:32:49.805564+0000 mon.a (mon.0) 1649 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T16:32:51.190 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:32:50 smithi100 bash[76202]: audit 2024-04-07T16:32:49.814923+0000 mon.a (mon.0) 1650 : audit [INF] from='client.? 172.21.15.100:0/3739952214' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110598-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:32:51.190 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:32:50 smithi100 bash[76202]: cluster 2024-04-07T16:32:49.824434+0000 mon.a (mon.0) 1651 : cluster [DBG] osdmap e916: 8 total, 8 up, 8 in 2024-04-07T16:32:52.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:32:51 smithi144 bash[66654]: cluster 2024-04-07T16:32:50.834256+0000 mon.a (mon.0) 1652 : cluster [DBG] osdmap e917: 8 total, 8 up, 8 in 2024-04-07T16:32:52.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:32:51 smithi144 bash[66654]: cluster 2024-04-07T16:32:50.923389+0000 mgr.y (mgr.25175) 525 : cluster [DBG] pgmap v1017: 105 pgs: 105 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:32:52.189 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:32:51 smithi100 bash[74942]: cluster 2024-04-07T16:32:50.834256+0000 mon.a (mon.0) 1652 : cluster [DBG] osdmap e917: 8 total, 8 up, 8 in 2024-04-07T16:32:52.189 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:32:51 smithi100 bash[74942]: cluster 2024-04-07T16:32:50.923389+0000 mgr.y (mgr.25175) 525 : cluster [DBG] pgmap v1017: 105 pgs: 105 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:32:52.190 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:32:51 smithi100 bash[76202]: cluster 2024-04-07T16:32:50.834256+0000 mon.a (mon.0) 1652 : cluster [DBG] osdmap e917: 8 total, 8 up, 8 in 2024-04-07T16:32:52.190 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:32:51 smithi100 bash[76202]: cluster 2024-04-07T16:32:50.923389+0000 mgr.y (mgr.25175) 525 : cluster [DBG] pgmap v1017: 105 pgs: 105 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:32:52.607 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:32:52 smithi144 bash[95233]: ts=2024-04-07T16:32:52.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:32:52.847 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestClsJournal.Create 2024-04-07T16:32:52.869 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestClsJournal.Create (22 ms) 2024-04-07T16:32:52.869 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestClsJournal.MinimumSet 2024-04-07T16:32:52.890 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestClsJournal.MinimumSet (21 ms) 2024-04-07T16:32:52.890 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestClsJournal.MinimumSetStale 2024-04-07T16:32:52.897 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestClsJournal.MinimumSetStale (7 ms) 2024-04-07T16:32:52.897 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestClsJournal.MinimumSetOrderConstraint 2024-04-07T16:32:52.906 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestClsJournal.MinimumSetOrderConstraint (6 ms) 2024-04-07T16:32:52.906 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestClsJournal.ActiveSet 2024-04-07T16:32:52.907 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestClsJournal.ActiveSet (4 ms) 2024-04-07T16:32:52.907 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestClsJournal.ActiveSetStale 2024-04-07T16:32:52.913 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestClsJournal.ActiveSetStale (6 ms) 2024-04-07T16:32:52.913 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestClsJournal.CreateDuplicate 2024-04-07T16:32:52.919 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestClsJournal.CreateDuplicate (6 ms) 2024-04-07T16:32:52.919 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestClsJournal.GetClient 2024-04-07T16:32:52.924 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestClsJournal.GetClient (5 ms) 2024-04-07T16:32:52.924 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestClsJournal.ClientRegister 2024-04-07T16:32:52.927 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestClsJournal.ClientRegister (3 ms) 2024-04-07T16:32:52.927 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestClsJournal.ClientRegisterDuplicate 2024-04-07T16:32:52.935 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestClsJournal.ClientRegisterDuplicate (8 ms) 2024-04-07T16:32:52.935 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestClsJournal.ClientUpdateData 2024-04-07T16:32:52.942 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestClsJournal.ClientUpdateData (7 ms) 2024-04-07T16:32:52.942 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestClsJournal.ClientUpdateState 2024-04-07T16:32:52.952 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestClsJournal.ClientUpdateState (10 ms) 2024-04-07T16:32:52.953 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestClsJournal.ClientUnregister 2024-04-07T16:32:52.959 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestClsJournal.ClientUnregister (7 ms) 2024-04-07T16:32:52.959 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestClsJournal.ClientUnregisterDNE 2024-04-07T16:32:52.968 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestClsJournal.ClientUnregisterDNE (9 ms) 2024-04-07T16:32:52.969 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestClsJournal.ClientUnregisterPruneTags 2024-04-07T16:32:53.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:32:52 smithi144 bash[66654]: cluster 2024-04-07T16:32:51.826990+0000 mon.a (mon.0) 1653 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T16:32:53.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:32:52 smithi144 bash[66654]: cluster 2024-04-07T16:32:51.827057+0000 mon.a (mon.0) 1654 : cluster [INF] Cluster is now healthy 2024-04-07T16:32:53.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:32:52 smithi144 bash[66654]: cluster 2024-04-07T16:32:51.846902+0000 mon.a (mon.0) 1655 : cluster [DBG] osdmap e918: 8 total, 8 up, 8 in 2024-04-07T16:32:53.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:32:52 smithi144 bash[66654]: audit 2024-04-07T16:32:51.849269+0000 mon.a (mon.0) 1656 : audit [INF] from='client.? 172.21.15.100:0/3564172327' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110721-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:32:53.189 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:32:52 smithi100 bash[74942]: cluster 2024-04-07T16:32:51.826990+0000 mon.a (mon.0) 1653 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T16:32:53.189 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:32:52 smithi100 bash[74942]: cluster 2024-04-07T16:32:51.827057+0000 mon.a (mon.0) 1654 : cluster [INF] Cluster is now healthy 2024-04-07T16:32:53.189 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:32:52 smithi100 bash[74942]: cluster 2024-04-07T16:32:51.846902+0000 mon.a (mon.0) 1655 : cluster [DBG] osdmap e918: 8 total, 8 up, 8 in 2024-04-07T16:32:53.189 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:32:52 smithi100 bash[74942]: audit 2024-04-07T16:32:51.849269+0000 mon.a (mon.0) 1656 : audit [INF] from='client.? 172.21.15.100:0/3564172327' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110721-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:32:53.190 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:32:52 smithi100 bash[76202]: cluster 2024-04-07T16:32:51.826990+0000 mon.a (mon.0) 1653 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T16:32:53.190 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:32:52 smithi100 bash[76202]: cluster 2024-04-07T16:32:51.827057+0000 mon.a (mon.0) 1654 : cluster [INF] Cluster is now healthy 2024-04-07T16:32:53.190 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:32:52 smithi100 bash[76202]: cluster 2024-04-07T16:32:51.846902+0000 mon.a (mon.0) 1655 : cluster [DBG] osdmap e918: 8 total, 8 up, 8 in 2024-04-07T16:32:53.190 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:32:52 smithi100 bash[76202]: audit 2024-04-07T16:32:51.849269+0000 mon.a (mon.0) 1656 : audit [INF] from='client.? 172.21.15.100:0/3564172327' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110721-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:32:53.215 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestClsJournal.ClientUnregisterPruneTags (247 ms) 2024-04-07T16:32:53.216 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestClsJournal.ClientCommit 2024-04-07T16:32:53.225 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestClsJournal.ClientCommit (10 ms) 2024-04-07T16:32:53.225 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestClsJournal.ClientCommitInvalid 2024-04-07T16:32:53.232 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestClsJournal.ClientCommitInvalid (6 ms) 2024-04-07T16:32:53.232 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestClsJournal.ClientCommitDNE 2024-04-07T16:32:53.234 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestClsJournal.ClientCommitDNE (3 ms) 2024-04-07T16:32:53.234 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestClsJournal.ClientList 2024-04-07T16:32:53.327 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestClsJournal.ClientList (92 ms) 2024-04-07T16:32:53.327 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestClsJournal.GetNextTagTid 2024-04-07T16:32:53.338 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestClsJournal.GetNextTagTid (12 ms) 2024-04-07T16:32:53.338 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestClsJournal.TagCreate 2024-04-07T16:32:53.365 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestClsJournal.TagCreate (26 ms) 2024-04-07T16:32:53.365 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestClsJournal.TagCreatePrunesTags 2024-04-07T16:32:53.385 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestClsJournal.TagCreatePrunesTags (21 ms) 2024-04-07T16:32:53.385 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestClsJournal.TagList 2024-04-07T16:32:53.666 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestClsJournal.TagList (280 ms) 2024-04-07T16:32:53.666 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestClsJournal.GuardAppend 2024-04-07T16:32:53.671 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestClsJournal.GuardAppend (6 ms) 2024-04-07T16:32:53.671 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestClsJournal.GuardAppendDNE 2024-04-07T16:32:53.674 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestClsJournal.GuardAppendDNE (3 ms) 2024-04-07T16:32:53.674 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestClsJournal.GuardAppendOverflow 2024-04-07T16:32:53.679 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestClsJournal.GuardAppendOverflow (4 ms) 2024-04-07T16:32:53.679 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestClsJournal.Append 2024-04-07T16:32:53.703 INFO:tasks.workunit.client.0.smithi100.stdout:padding is not enabled 2024-04-07T16:32:53.703 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestClsJournal.Append (24 ms) 2024-04-07T16:32:53.867 INFO:tasks.workunit.client.0.smithi100.stdout:[----------] 27 tests from TestClsJournal (857 ms total) 2024-04-07T16:32:53.867 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T16:32:53.868 INFO:tasks.workunit.client.0.smithi100.stdout:[----------] Global test environment tear-down 2024-04-07T16:32:53.868 INFO:tasks.workunit.client.0.smithi100.stdout:[==========] 27 tests from 1 test suite ran. (2914 ms total) 2024-04-07T16:32:53.868 INFO:tasks.workunit.client.0.smithi100.stdout:[ PASSED ] 27 tests. 2024-04-07T16:32:53.874 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-07T16:32:53.874 DEBUG:teuthology.orchestra.run.smithi100:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-07T16:32:53.929 INFO:tasks.workunit:Running workunit cls/test_cls_lock.sh... 2024-04-07T16:32:53.930 DEBUG:teuthology.orchestra.run.smithi100:workunit test cls/test_cls_lock.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=reef TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_lock.sh 2024-04-07T16:32:53.999 INFO:tasks.workunit.client.0.smithi100.stdout:Running main() from gmock_main.cc 2024-04-07T16:32:54.000 INFO:tasks.workunit.client.0.smithi100.stdout:[==========] Running 11 tests from 1 test suite. 2024-04-07T16:32:54.000 INFO:tasks.workunit.client.0.smithi100.stdout:[----------] Global test environment set-up. 2024-04-07T16:32:54.000 INFO:tasks.workunit.client.0.smithi100.stdout:[----------] 11 tests from ClsLock 2024-04-07T16:32:54.000 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] ClsLock.TestMultiLocking 2024-04-07T16:32:54.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:32:53 smithi144 bash[66654]: audit 2024-04-07T16:32:52.851180+0000 mon.a (mon.0) 1657 : audit [INF] from='client.? 172.21.15.100:0/3564172327' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110721-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:32:54.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:32:53 smithi144 bash[66654]: cluster 2024-04-07T16:32:52.857619+0000 mon.a (mon.0) 1658 : cluster [DBG] osdmap e919: 8 total, 8 up, 8 in 2024-04-07T16:32:54.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:32:53 smithi144 bash[66654]: cluster 2024-04-07T16:32:52.923864+0000 mgr.y (mgr.25175) 526 : cluster [DBG] pgmap v1020: 137 pgs: 16 creating+peering, 16 unknown, 105 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:32:54.190 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:32:53 smithi100 bash[74942]: audit 2024-04-07T16:32:52.851180+0000 mon.a (mon.0) 1657 : audit [INF] from='client.? 172.21.15.100:0/3564172327' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110721-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:32:54.190 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:32:53 smithi100 bash[74942]: cluster 2024-04-07T16:32:52.857619+0000 mon.a (mon.0) 1658 : cluster [DBG] osdmap e919: 8 total, 8 up, 8 in 2024-04-07T16:32:54.190 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:32:53 smithi100 bash[74942]: cluster 2024-04-07T16:32:52.923864+0000 mgr.y (mgr.25175) 526 : cluster [DBG] pgmap v1020: 137 pgs: 16 creating+peering, 16 unknown, 105 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:32:54.190 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:32:53 smithi100 bash[76202]: audit 2024-04-07T16:32:52.851180+0000 mon.a (mon.0) 1657 : audit [INF] from='client.? 172.21.15.100:0/3564172327' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110721-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:32:54.190 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:32:53 smithi100 bash[76202]: cluster 2024-04-07T16:32:52.857619+0000 mon.a (mon.0) 1658 : cluster [DBG] osdmap e919: 8 total, 8 up, 8 in 2024-04-07T16:32:54.190 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:32:53 smithi100 bash[76202]: cluster 2024-04-07T16:32:52.923864+0000 mgr.y (mgr.25175) 526 : cluster [DBG] pgmap v1020: 137 pgs: 16 creating+peering, 16 unknown, 105 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:32:55.189 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:32:54 smithi100 bash[74942]: cluster 2024-04-07T16:32:53.875158+0000 mon.a (mon.0) 1659 : cluster [DBG] osdmap e920: 8 total, 8 up, 8 in 2024-04-07T16:32:55.189 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:32:54 smithi100 bash[76202]: cluster 2024-04-07T16:32:53.875158+0000 mon.a (mon.0) 1659 : cluster [DBG] osdmap e920: 8 total, 8 up, 8 in 2024-04-07T16:32:55.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:32:54 smithi144 bash[66654]: cluster 2024-04-07T16:32:53.875158+0000 mon.a (mon.0) 1659 : cluster [DBG] osdmap e920: 8 total, 8 up, 8 in 2024-04-07T16:32:55.919 INFO:tasks.workunit.client.0.smithi100.stdout:lock: mylock 2024-04-07T16:32:55.919 INFO:tasks.workunit.client.0.smithi100.stdout: lock_type: exclusive 2024-04-07T16:32:55.919 INFO:tasks.workunit.client.0.smithi100.stdout: tag: 2024-04-07T16:32:55.919 INFO:tasks.workunit.client.0.smithi100.stdout: lockers: 2024-04-07T16:32:55.919 INFO:tasks.workunit.client.0.smithi100.stdout: client.44651 expiration=2024-04-07T16:34:55.922232+0000 addr=v1:172.21.15.100:0/4275210414 cookie= 2024-04-07T16:32:55.925 INFO:tasks.workunit.client.0.smithi100.stdout:lock: mylock 2024-04-07T16:32:55.925 INFO:tasks.workunit.client.0.smithi100.stdout: lock_type: shared 2024-04-07T16:32:55.925 INFO:tasks.workunit.client.0.smithi100.stdout: tag: 2024-04-07T16:32:55.926 INFO:tasks.workunit.client.0.smithi100.stdout: lockers: 2024-04-07T16:32:55.926 INFO:tasks.workunit.client.0.smithi100.stdout: client.44651 expiration=2024-04-07T16:34:55.933515+0000 addr=v1:172.21.15.100:0/4275210414 cookie= 2024-04-07T16:32:55.926 INFO:tasks.workunit.client.0.smithi100.stdout: client.44657 expiration=0.000000 addr=v1:172.21.15.100:0/477315252 cookie= 2024-04-07T16:32:55.928 INFO:tasks.workunit.client.0.smithi100.stdout:lock: mylock 2024-04-07T16:32:55.928 INFO:tasks.workunit.client.0.smithi100.stdout: lock_type: shared 2024-04-07T16:32:55.928 INFO:tasks.workunit.client.0.smithi100.stdout: tag: 2024-04-07T16:32:55.928 INFO:tasks.workunit.client.0.smithi100.stdout: lockers: 2024-04-07T16:32:55.928 INFO:tasks.workunit.client.0.smithi100.stdout: client.44657 expiration=0.000000 addr=v1:172.21.15.100:0/477315252 cookie= 2024-04-07T16:32:56.190 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:32:55 smithi100 bash[74942]: cluster 2024-04-07T16:32:54.877409+0000 mon.a (mon.0) 1660 : cluster [DBG] osdmap e921: 8 total, 8 up, 8 in 2024-04-07T16:32:56.190 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:32:55 smithi100 bash[74942]: audit 2024-04-07T16:32:54.881858+0000 mon.c (mon.2) 40 : audit [INF] from='client.? 172.21.15.100:0/4275210414' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110745-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:32:56.190 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:32:55 smithi100 bash[74942]: audit 2024-04-07T16:32:54.892082+0000 mon.a (mon.0) 1661 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110745-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:32:56.190 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:32:55 smithi100 bash[74942]: cluster 2024-04-07T16:32:54.924537+0000 mgr.y (mgr.25175) 527 : cluster [DBG] pgmap v1023: 137 pgs: 32 unknown, 105 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:32:56.190 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:32:55 smithi100 bash[74942]: audit 2024-04-07T16:32:55.870091+0000 mon.a (mon.0) 1662 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110745-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:32:56.190 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:32:55 smithi100 bash[74942]: cluster 2024-04-07T16:32:55.879159+0000 mon.a (mon.0) 1663 : cluster [DBG] osdmap e922: 8 total, 8 up, 8 in 2024-04-07T16:32:56.191 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:32:55 smithi100 bash[76202]: cluster 2024-04-07T16:32:54.877409+0000 mon.a (mon.0) 1660 : cluster [DBG] osdmap e921: 8 total, 8 up, 8 in 2024-04-07T16:32:56.191 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:32:55 smithi100 bash[76202]: audit 2024-04-07T16:32:54.881858+0000 mon.c (mon.2) 40 : audit [INF] from='client.? 172.21.15.100:0/4275210414' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110745-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:32:56.191 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:32:55 smithi100 bash[76202]: audit 2024-04-07T16:32:54.892082+0000 mon.a (mon.0) 1661 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110745-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:32:56.191 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:32:55 smithi100 bash[76202]: cluster 2024-04-07T16:32:54.924537+0000 mgr.y (mgr.25175) 527 : cluster [DBG] pgmap v1023: 137 pgs: 32 unknown, 105 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:32:56.191 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:32:55 smithi100 bash[76202]: audit 2024-04-07T16:32:55.870091+0000 mon.a (mon.0) 1662 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110745-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:32:56.191 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:32:55 smithi100 bash[76202]: cluster 2024-04-07T16:32:55.879159+0000 mon.a (mon.0) 1663 : cluster [DBG] osdmap e922: 8 total, 8 up, 8 in 2024-04-07T16:32:56.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:32:55 smithi144 bash[66654]: cluster 2024-04-07T16:32:54.877409+0000 mon.a (mon.0) 1660 : cluster [DBG] osdmap e921: 8 total, 8 up, 8 in 2024-04-07T16:32:56.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:32:55 smithi144 bash[66654]: audit 2024-04-07T16:32:54.881858+0000 mon.c (mon.2) 40 : audit [INF] from='client.? 172.21.15.100:0/4275210414' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110745-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:32:56.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:32:55 smithi144 bash[66654]: audit 2024-04-07T16:32:54.892082+0000 mon.a (mon.0) 1661 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110745-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:32:56.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:32:55 smithi144 bash[66654]: cluster 2024-04-07T16:32:54.924537+0000 mgr.y (mgr.25175) 527 : cluster [DBG] pgmap v1023: 137 pgs: 32 unknown, 105 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:32:56.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:32:55 smithi144 bash[66654]: audit 2024-04-07T16:32:55.870091+0000 mon.a (mon.0) 1662 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110745-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:32:56.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:32:55 smithi144 bash[66654]: cluster 2024-04-07T16:32:55.879159+0000 mon.a (mon.0) 1663 : cluster [DBG] osdmap e922: 8 total, 8 up, 8 in 2024-04-07T16:32:56.909 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] ClsLock.TestMultiLocking (2909 ms) 2024-04-07T16:32:56.909 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] ClsLock.TestMeta 2024-04-07T16:32:57.189 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:32:56 smithi100 bash[74942]: cluster 2024-04-07T16:32:55.886200+0000 mon.a (mon.0) 1664 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T16:32:57.189 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:32:56 smithi100 bash[76202]: cluster 2024-04-07T16:32:55.886200+0000 mon.a (mon.0) 1664 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T16:32:57.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:32:56 smithi144 bash[66654]: cluster 2024-04-07T16:32:55.886200+0000 mon.a (mon.0) 1664 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T16:32:58.189 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:32:57 smithi100 bash[74942]: cluster 2024-04-07T16:32:56.908995+0000 mon.a (mon.0) 1665 : cluster [DBG] osdmap e923: 8 total, 8 up, 8 in 2024-04-07T16:32:58.190 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:32:57 smithi100 bash[74942]: cluster 2024-04-07T16:32:56.925339+0000 mgr.y (mgr.25175) 528 : cluster [DBG] pgmap v1026: 105 pgs: 105 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:32:58.190 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:32:57 smithi100 bash[76202]: cluster 2024-04-07T16:32:56.908995+0000 mon.a (mon.0) 1665 : cluster [DBG] osdmap e923: 8 total, 8 up, 8 in 2024-04-07T16:32:58.190 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:32:57 smithi100 bash[76202]: cluster 2024-04-07T16:32:56.925339+0000 mgr.y (mgr.25175) 528 : cluster [DBG] pgmap v1026: 105 pgs: 105 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:32:58.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:32:57 smithi144 bash[66654]: cluster 2024-04-07T16:32:56.908995+0000 mon.a (mon.0) 1665 : cluster [DBG] osdmap e923: 8 total, 8 up, 8 in 2024-04-07T16:32:58.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:32:57 smithi144 bash[66654]: cluster 2024-04-07T16:32:56.925339+0000 mgr.y (mgr.25175) 528 : cluster [DBG] pgmap v1026: 105 pgs: 105 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:32:58.689 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:32:58 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:32:58] "GET /metrics HTTP/1.1" 200 36357 "" "Prometheus/2.43.0" 2024-04-07T16:32:58.981 INFO:tasks.workunit.client.0.smithi100.stdout:lock: mylock 2024-04-07T16:32:58.981 INFO:tasks.workunit.client.0.smithi100.stdout: lock_type: shared 2024-04-07T16:32:58.981 INFO:tasks.workunit.client.0.smithi100.stdout: tag: 2024-04-07T16:32:58.981 INFO:tasks.workunit.client.0.smithi100.stdout: lockers: 2024-04-07T16:32:58.981 INFO:tasks.workunit.client.0.smithi100.stdout: client.34761 expiration=0.000000 addr=v1:172.21.15.100:0/3809912294 cookie= 2024-04-07T16:32:58.986 INFO:tasks.workunit.client.0.smithi100.stdout:lock: mylock 2024-04-07T16:32:58.986 INFO:tasks.workunit.client.0.smithi100.stdout: lock_type: exclusive 2024-04-07T16:32:58.986 INFO:tasks.workunit.client.0.smithi100.stdout: tag: new_tag 2024-04-07T16:32:58.986 INFO:tasks.workunit.client.0.smithi100.stdout: lockers: 2024-04-07T16:32:58.986 INFO:tasks.workunit.client.0.smithi100.stdout: client.44663 expiration=0.000000 addr=v1:172.21.15.100:0/759002283 cookie= 2024-04-07T16:32:59.189 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:32:58 smithi100 bash[74942]: cluster 2024-04-07T16:32:57.900456+0000 mon.a (mon.0) 1666 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T16:32:59.190 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:32:58 smithi100 bash[74942]: cluster 2024-04-07T16:32:57.900526+0000 mon.a (mon.0) 1667 : cluster [INF] Cluster is now healthy 2024-04-07T16:32:59.190 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:32:58 smithi100 bash[74942]: cluster 2024-04-07T16:32:57.921923+0000 mon.a (mon.0) 1668 : cluster [DBG] osdmap e924: 8 total, 8 up, 8 in 2024-04-07T16:32:59.190 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:32:58 smithi100 bash[74942]: audit 2024-04-07T16:32:57.926956+0000 mon.c (mon.2) 41 : audit [INF] from='client.? 172.21.15.100:0/759002283' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110745-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:32:59.190 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:32:58 smithi100 bash[74942]: audit 2024-04-07T16:32:57.928587+0000 mon.a (mon.0) 1669 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110745-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:32:59.190 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:32:58 smithi100 bash[76202]: cluster 2024-04-07T16:32:57.900456+0000 mon.a (mon.0) 1666 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T16:32:59.190 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:32:58 smithi100 bash[76202]: cluster 2024-04-07T16:32:57.900526+0000 mon.a (mon.0) 1667 : cluster [INF] Cluster is now healthy 2024-04-07T16:32:59.190 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:32:58 smithi100 bash[76202]: cluster 2024-04-07T16:32:57.921923+0000 mon.a (mon.0) 1668 : cluster [DBG] osdmap e924: 8 total, 8 up, 8 in 2024-04-07T16:32:59.190 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:32:58 smithi100 bash[76202]: audit 2024-04-07T16:32:57.926956+0000 mon.c (mon.2) 41 : audit [INF] from='client.? 172.21.15.100:0/759002283' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110745-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:32:59.190 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:32:58 smithi100 bash[76202]: audit 2024-04-07T16:32:57.928587+0000 mon.a (mon.0) 1669 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110745-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:32:59.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:32:58 smithi144 bash[66654]: cluster 2024-04-07T16:32:57.900456+0000 mon.a (mon.0) 1666 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T16:32:59.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:32:58 smithi144 bash[66654]: cluster 2024-04-07T16:32:57.900526+0000 mon.a (mon.0) 1667 : cluster [INF] Cluster is now healthy 2024-04-07T16:32:59.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:32:58 smithi144 bash[66654]: cluster 2024-04-07T16:32:57.921923+0000 mon.a (mon.0) 1668 : cluster [DBG] osdmap e924: 8 total, 8 up, 8 in 2024-04-07T16:32:59.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:32:58 smithi144 bash[66654]: audit 2024-04-07T16:32:57.926956+0000 mon.c (mon.2) 41 : audit [INF] from='client.? 172.21.15.100:0/759002283' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110745-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:32:59.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:32:58 smithi144 bash[66654]: audit 2024-04-07T16:32:57.928587+0000 mon.a (mon.0) 1669 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110745-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:32:59.933 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] ClsLock.TestMeta (3024 ms) 2024-04-07T16:32:59.933 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] ClsLock.TestCookie 2024-04-07T16:33:00.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:32:59 smithi144 bash[66654]: cluster 2024-04-07T16:32:58.926185+0000 mgr.y (mgr.25175) 529 : cluster [DBG] pgmap v1028: 137 pgs: 1 creating+peering, 31 unknown, 105 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:33:00.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:32:59 smithi144 bash[66654]: audit 2024-04-07T16:32:58.928414+0000 mon.a (mon.0) 1670 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110745-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:33:00.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:32:59 smithi144 bash[66654]: cluster 2024-04-07T16:32:58.935694+0000 mon.a (mon.0) 1671 : cluster [DBG] osdmap e925: 8 total, 8 up, 8 in 2024-04-07T16:33:00.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:32:59 smithi144 bash[66654]: cluster 2024-04-07T16:32:59.916774+0000 mon.a (mon.0) 1672 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T16:33:00.439 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:32:59 smithi100 bash[74942]: cluster 2024-04-07T16:32:58.926185+0000 mgr.y (mgr.25175) 529 : cluster [DBG] pgmap v1028: 137 pgs: 1 creating+peering, 31 unknown, 105 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:33:00.439 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:32:59 smithi100 bash[74942]: audit 2024-04-07T16:32:58.928414+0000 mon.a (mon.0) 1670 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110745-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:33:00.439 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:32:59 smithi100 bash[74942]: cluster 2024-04-07T16:32:58.935694+0000 mon.a (mon.0) 1671 : cluster [DBG] osdmap e925: 8 total, 8 up, 8 in 2024-04-07T16:33:00.439 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:32:59 smithi100 bash[74942]: cluster 2024-04-07T16:32:59.916774+0000 mon.a (mon.0) 1672 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T16:33:00.440 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:32:59 smithi100 bash[76202]: cluster 2024-04-07T16:32:58.926185+0000 mgr.y (mgr.25175) 529 : cluster [DBG] pgmap v1028: 137 pgs: 1 creating+peering, 31 unknown, 105 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:33:00.440 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:32:59 smithi100 bash[76202]: audit 2024-04-07T16:32:58.928414+0000 mon.a (mon.0) 1670 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110745-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:33:00.440 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:32:59 smithi100 bash[76202]: cluster 2024-04-07T16:32:58.935694+0000 mon.a (mon.0) 1671 : cluster [DBG] osdmap e925: 8 total, 8 up, 8 in 2024-04-07T16:33:00.440 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:32:59 smithi100 bash[76202]: cluster 2024-04-07T16:32:59.916774+0000 mon.a (mon.0) 1672 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T16:33:01.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:33:00 smithi144 bash[66654]: cluster 2024-04-07T16:32:59.937392+0000 mon.a (mon.0) 1673 : cluster [DBG] osdmap e926: 8 total, 8 up, 8 in 2024-04-07T16:33:01.439 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:33:00 smithi100 bash[74942]: cluster 2024-04-07T16:32:59.937392+0000 mon.a (mon.0) 1673 : cluster [DBG] osdmap e926: 8 total, 8 up, 8 in 2024-04-07T16:33:01.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:33:00 smithi100 bash[76202]: cluster 2024-04-07T16:32:59.937392+0000 mon.a (mon.0) 1673 : cluster [DBG] osdmap e926: 8 total, 8 up, 8 in 2024-04-07T16:33:02.104 INFO:tasks.workunit.client.0.smithi100.stdout:lock: mylock 2024-04-07T16:33:02.105 INFO:tasks.workunit.client.0.smithi100.stdout: lock_type: exclusive 2024-04-07T16:33:02.105 INFO:tasks.workunit.client.0.smithi100.stdout: tag: 2024-04-07T16:33:02.105 INFO:tasks.workunit.client.0.smithi100.stdout: lockers: 2024-04-07T16:33:02.110 INFO:tasks.workunit.client.0.smithi100.stdout:lock: mylock 2024-04-07T16:33:02.110 INFO:tasks.workunit.client.0.smithi100.stdout: lock_type: shared 2024-04-07T16:33:02.110 INFO:tasks.workunit.client.0.smithi100.stdout: tag: 2024-04-07T16:33:02.110 INFO:tasks.workunit.client.0.smithi100.stdout: lockers: 2024-04-07T16:33:02.110 INFO:tasks.workunit.client.0.smithi100.stdout: client.34767 expiration=0.000000 addr=v1:172.21.15.100:0/3385436676 cookie= 2024-04-07T16:33:02.110 INFO:tasks.workunit.client.0.smithi100.stdout: client.34767 expiration=0.000000 addr=v1:172.21.15.100:0/3385436676 cookie=new cookie 2024-04-07T16:33:02.306 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:33:01 smithi144 bash[66654]: cluster 2024-04-07T16:33:00.926964+0000 mgr.y (mgr.25175) 530 : cluster [DBG] pgmap v1031: 105 pgs: 105 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:33:02.307 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:33:01 smithi144 bash[66654]: cluster 2024-04-07T16:33:00.954000+0000 mon.a (mon.0) 1674 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T16:33:02.307 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:33:01 smithi144 bash[66654]: cluster 2024-04-07T16:33:00.954104+0000 mon.a (mon.0) 1675 : cluster [INF] Cluster is now healthy 2024-04-07T16:33:02.307 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:33:01 smithi144 bash[66654]: cluster 2024-04-07T16:33:00.975732+0000 mon.a (mon.0) 1676 : cluster [DBG] osdmap e927: 8 total, 8 up, 8 in 2024-04-07T16:33:02.307 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:33:01 smithi144 bash[66654]: audit 2024-04-07T16:33:00.979438+0000 mon.a (mon.0) 1677 : audit [INF] from='client.? 172.21.15.100:0/3385436676' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110745-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:33:02.307 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:33:01 smithi144 bash[66654]: audit 2024-04-07T16:33:01.744307+0000 mon.a (mon.0) 1678 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:33:02.439 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:33:01 smithi100 bash[74942]: cluster 2024-04-07T16:33:00.926964+0000 mgr.y (mgr.25175) 530 : cluster [DBG] pgmap v1031: 105 pgs: 105 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:33:02.440 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:33:01 smithi100 bash[74942]: cluster 2024-04-07T16:33:00.954000+0000 mon.a (mon.0) 1674 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T16:33:02.440 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:33:01 smithi100 bash[74942]: cluster 2024-04-07T16:33:00.954104+0000 mon.a (mon.0) 1675 : cluster [INF] Cluster is now healthy 2024-04-07T16:33:02.440 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:33:01 smithi100 bash[74942]: cluster 2024-04-07T16:33:00.975732+0000 mon.a (mon.0) 1676 : cluster [DBG] osdmap e927: 8 total, 8 up, 8 in 2024-04-07T16:33:02.440 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:33:01 smithi100 bash[74942]: audit 2024-04-07T16:33:00.979438+0000 mon.a (mon.0) 1677 : audit [INF] from='client.? 172.21.15.100:0/3385436676' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110745-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:33:02.440 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:33:01 smithi100 bash[74942]: audit 2024-04-07T16:33:01.744307+0000 mon.a (mon.0) 1678 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:33:02.440 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:33:01 smithi100 bash[76202]: cluster 2024-04-07T16:33:00.926964+0000 mgr.y (mgr.25175) 530 : cluster [DBG] pgmap v1031: 105 pgs: 105 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:33:02.440 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:33:01 smithi100 bash[76202]: cluster 2024-04-07T16:33:00.954000+0000 mon.a (mon.0) 1674 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T16:33:02.440 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:33:01 smithi100 bash[76202]: cluster 2024-04-07T16:33:00.954104+0000 mon.a (mon.0) 1675 : cluster [INF] Cluster is now healthy 2024-04-07T16:33:02.440 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:33:01 smithi100 bash[76202]: cluster 2024-04-07T16:33:00.975732+0000 mon.a (mon.0) 1676 : cluster [DBG] osdmap e927: 8 total, 8 up, 8 in 2024-04-07T16:33:02.440 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:33:01 smithi100 bash[76202]: audit 2024-04-07T16:33:00.979438+0000 mon.a (mon.0) 1677 : audit [INF] from='client.? 172.21.15.100:0/3385436676' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110745-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:33:02.440 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:33:01 smithi100 bash[76202]: audit 2024-04-07T16:33:01.744307+0000 mon.a (mon.0) 1678 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:33:02.607 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:33:02 smithi144 bash[95233]: ts=2024-04-07T16:33:02.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:33:02.982 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] ClsLock.TestCookie (3049 ms) 2024-04-07T16:33:02.982 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] ClsLock.TestMultipleLocks 2024-04-07T16:33:03.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:33:02 smithi144 bash[66654]: audit 2024-04-07T16:33:01.970174+0000 mon.a (mon.0) 1679 : audit [INF] from='client.? 172.21.15.100:0/3385436676' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110745-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:33:03.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:33:02 smithi144 bash[66654]: cluster 2024-04-07T16:33:01.975889+0000 mon.a (mon.0) 1680 : cluster [DBG] osdmap e928: 8 total, 8 up, 8 in 2024-04-07T16:33:03.439 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:33:02 smithi100 bash[74942]: audit 2024-04-07T16:33:01.970174+0000 mon.a (mon.0) 1679 : audit [INF] from='client.? 172.21.15.100:0/3385436676' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110745-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:33:03.439 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:33:02 smithi100 bash[74942]: cluster 2024-04-07T16:33:01.975889+0000 mon.a (mon.0) 1680 : cluster [DBG] osdmap e928: 8 total, 8 up, 8 in 2024-04-07T16:33:03.440 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:33:02 smithi100 bash[76202]: audit 2024-04-07T16:33:01.970174+0000 mon.a (mon.0) 1679 : audit [INF] from='client.? 172.21.15.100:0/3385436676' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110745-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:33:03.440 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:33:02 smithi100 bash[76202]: cluster 2024-04-07T16:33:01.975889+0000 mon.a (mon.0) 1680 : cluster [DBG] osdmap e928: 8 total, 8 up, 8 in 2024-04-07T16:33:04.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:33:04 smithi144 bash[66654]: cluster 2024-04-07T16:33:02.928205+0000 mgr.y (mgr.25175) 531 : cluster [DBG] pgmap v1034: 137 pgs: 8 creating+activating, 12 unknown, 117 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:33:04.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:33:04 smithi144 bash[66654]: cluster 2024-04-07T16:33:02.992098+0000 mon.a (mon.0) 1681 : cluster [DBG] osdmap e929: 8 total, 8 up, 8 in 2024-04-07T16:33:04.439 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:33:04 smithi100 bash[74942]: cluster 2024-04-07T16:33:02.928205+0000 mgr.y (mgr.25175) 531 : cluster [DBG] pgmap v1034: 137 pgs: 8 creating+activating, 12 unknown, 117 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:33:04.439 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:33:04 smithi100 bash[74942]: cluster 2024-04-07T16:33:02.992098+0000 mon.a (mon.0) 1681 : cluster [DBG] osdmap e929: 8 total, 8 up, 8 in 2024-04-07T16:33:04.440 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:33:04 smithi100 bash[76202]: cluster 2024-04-07T16:33:02.928205+0000 mgr.y (mgr.25175) 531 : cluster [DBG] pgmap v1034: 137 pgs: 8 creating+activating, 12 unknown, 117 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:33:04.440 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:33:04 smithi100 bash[76202]: cluster 2024-04-07T16:33:02.992098+0000 mon.a (mon.0) 1681 : cluster [DBG] osdmap e929: 8 total, 8 up, 8 in 2024-04-07T16:33:05.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:33:05 smithi144 bash[66654]: cluster 2024-04-07T16:33:03.996155+0000 mon.a (mon.0) 1682 : cluster [DBG] osdmap e930: 8 total, 8 up, 8 in 2024-04-07T16:33:05.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:33:05 smithi144 bash[66654]: audit 2024-04-07T16:33:04.000963+0000 mon.a (mon.0) 1683 : audit [INF] from='client.? 172.21.15.100:0/3814596298' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110745-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:33:05.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:33:05 smithi144 bash[66654]: audit 2024-04-07T16:33:04.989844+0000 mon.a (mon.0) 1684 : audit [INF] from='client.? 172.21.15.100:0/3814596298' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110745-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:33:05.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:33:05 smithi144 bash[66654]: cluster 2024-04-07T16:33:04.998674+0000 mon.a (mon.0) 1685 : cluster [DBG] osdmap e931: 8 total, 8 up, 8 in 2024-04-07T16:33:05.439 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:33:05 smithi100 bash[74942]: cluster 2024-04-07T16:33:03.996155+0000 mon.a (mon.0) 1682 : cluster [DBG] osdmap e930: 8 total, 8 up, 8 in 2024-04-07T16:33:05.440 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:33:05 smithi100 bash[74942]: audit 2024-04-07T16:33:04.000963+0000 mon.a (mon.0) 1683 : audit [INF] from='client.? 172.21.15.100:0/3814596298' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110745-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:33:05.440 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:33:05 smithi100 bash[74942]: audit 2024-04-07T16:33:04.989844+0000 mon.a (mon.0) 1684 : audit [INF] from='client.? 172.21.15.100:0/3814596298' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110745-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:33:05.440 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:33:05 smithi100 bash[74942]: cluster 2024-04-07T16:33:04.998674+0000 mon.a (mon.0) 1685 : cluster [DBG] osdmap e931: 8 total, 8 up, 8 in 2024-04-07T16:33:05.440 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:33:05 smithi100 bash[76202]: cluster 2024-04-07T16:33:03.996155+0000 mon.a (mon.0) 1682 : cluster [DBG] osdmap e930: 8 total, 8 up, 8 in 2024-04-07T16:33:05.440 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:33:05 smithi100 bash[76202]: audit 2024-04-07T16:33:04.000963+0000 mon.a (mon.0) 1683 : audit [INF] from='client.? 172.21.15.100:0/3814596298' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110745-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:33:05.440 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:33:05 smithi100 bash[76202]: audit 2024-04-07T16:33:04.989844+0000 mon.a (mon.0) 1684 : audit [INF] from='client.? 172.21.15.100:0/3814596298' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110745-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:33:05.440 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:33:05 smithi100 bash[76202]: cluster 2024-04-07T16:33:04.998674+0000 mon.a (mon.0) 1685 : cluster [DBG] osdmap e931: 8 total, 8 up, 8 in 2024-04-07T16:33:06.016 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] ClsLock.TestMultipleLocks (3034 ms) 2024-04-07T16:33:06.017 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] ClsLock.TestLockDuration 2024-04-07T16:33:06.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:33:06 smithi144 bash[66654]: cluster 2024-04-07T16:33:04.929100+0000 mgr.y (mgr.25175) 532 : cluster [DBG] pgmap v1037: 137 pgs: 3 creating+peering, 29 unknown, 105 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:33:06.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:33:06 smithi144 bash[66654]: cluster 2024-04-07T16:33:05.005458+0000 mon.a (mon.0) 1686 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T16:33:06.439 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:33:06 smithi100 bash[74942]: cluster 2024-04-07T16:33:04.929100+0000 mgr.y (mgr.25175) 532 : cluster [DBG] pgmap v1037: 137 pgs: 3 creating+peering, 29 unknown, 105 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:33:06.439 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:33:06 smithi100 bash[74942]: cluster 2024-04-07T16:33:05.005458+0000 mon.a (mon.0) 1686 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T16:33:06.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:33:06 smithi100 bash[76202]: cluster 2024-04-07T16:33:04.929100+0000 mgr.y (mgr.25175) 532 : cluster [DBG] pgmap v1037: 137 pgs: 3 creating+peering, 29 unknown, 105 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:33:06.440 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:33:06 smithi100 bash[76202]: cluster 2024-04-07T16:33:05.005458+0000 mon.a (mon.0) 1686 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T16:33:07.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:33:07 smithi144 bash[66654]: cluster 2024-04-07T16:33:06.025967+0000 mon.a (mon.0) 1687 : cluster [DBG] osdmap e932: 8 total, 8 up, 8 in 2024-04-07T16:33:07.439 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:33:07 smithi100 bash[74942]: cluster 2024-04-07T16:33:06.025967+0000 mon.a (mon.0) 1687 : cluster [DBG] osdmap e932: 8 total, 8 up, 8 in 2024-04-07T16:33:07.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:33:07 smithi100 bash[76202]: cluster 2024-04-07T16:33:06.025967+0000 mon.a (mon.0) 1687 : cluster [DBG] osdmap e932: 8 total, 8 up, 8 in 2024-04-07T16:33:08.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:33:08 smithi144 bash[66654]: cluster 2024-04-07T16:33:06.929887+0000 mgr.y (mgr.25175) 533 : cluster [DBG] pgmap v1040: 105 pgs: 105 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:33:08.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:33:08 smithi144 bash[66654]: cluster 2024-04-07T16:33:07.019201+0000 mon.a (mon.0) 1688 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T16:33:08.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:33:08 smithi144 bash[66654]: cluster 2024-04-07T16:33:07.019300+0000 mon.a (mon.0) 1689 : cluster [INF] Cluster is now healthy 2024-04-07T16:33:08.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:33:08 smithi144 bash[66654]: cluster 2024-04-07T16:33:07.047860+0000 mon.a (mon.0) 1690 : cluster [DBG] osdmap e933: 8 total, 8 up, 8 in 2024-04-07T16:33:08.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:33:08 smithi144 bash[66654]: audit 2024-04-07T16:33:07.054521+0000 mon.a (mon.0) 1691 : audit [INF] from='client.? 172.21.15.100:0/3660147321' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110745-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:33:08.418 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:33:08 smithi100 bash[74942]: cluster 2024-04-07T16:33:06.929887+0000 mgr.y (mgr.25175) 533 : cluster [DBG] pgmap v1040: 105 pgs: 105 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:33:08.418 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:33:08 smithi100 bash[74942]: cluster 2024-04-07T16:33:07.019201+0000 mon.a (mon.0) 1688 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T16:33:08.418 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:33:08 smithi100 bash[74942]: cluster 2024-04-07T16:33:07.019300+0000 mon.a (mon.0) 1689 : cluster [INF] Cluster is now healthy 2024-04-07T16:33:08.418 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:33:08 smithi100 bash[74942]: cluster 2024-04-07T16:33:07.047860+0000 mon.a (mon.0) 1690 : cluster [DBG] osdmap e933: 8 total, 8 up, 8 in 2024-04-07T16:33:08.418 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:33:08 smithi100 bash[74942]: audit 2024-04-07T16:33:07.054521+0000 mon.a (mon.0) 1691 : audit [INF] from='client.? 172.21.15.100:0/3660147321' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110745-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:33:08.419 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:33:08 smithi100 bash[76202]: cluster 2024-04-07T16:33:06.929887+0000 mgr.y (mgr.25175) 533 : cluster [DBG] pgmap v1040: 105 pgs: 105 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:33:08.419 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:33:08 smithi100 bash[76202]: cluster 2024-04-07T16:33:07.019201+0000 mon.a (mon.0) 1688 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T16:33:08.419 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:33:08 smithi100 bash[76202]: cluster 2024-04-07T16:33:07.019300+0000 mon.a (mon.0) 1689 : cluster [INF] Cluster is now healthy 2024-04-07T16:33:08.419 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:33:08 smithi100 bash[76202]: cluster 2024-04-07T16:33:07.047860+0000 mon.a (mon.0) 1690 : cluster [DBG] osdmap e933: 8 total, 8 up, 8 in 2024-04-07T16:33:08.419 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:33:08 smithi100 bash[76202]: audit 2024-04-07T16:33:07.054521+0000 mon.a (mon.0) 1691 : audit [INF] from='client.? 172.21.15.100:0/3660147321' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110745-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:33:08.689 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:33:08 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:33:08] "GET /metrics HTTP/1.1" 200 36357 "" "Prometheus/2.43.0" 2024-04-07T16:33:09.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:33:09 smithi144 bash[66654]: audit 2024-04-07T16:33:08.038362+0000 mon.a (mon.0) 1692 : audit [INF] from='client.? 172.21.15.100:0/3660147321' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110745-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:33:09.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:33:09 smithi144 bash[66654]: cluster 2024-04-07T16:33:08.047322+0000 mon.a (mon.0) 1693 : cluster [DBG] osdmap e934: 8 total, 8 up, 8 in 2024-04-07T16:33:09.439 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:33:09 smithi100 bash[74942]: audit 2024-04-07T16:33:08.038362+0000 mon.a (mon.0) 1692 : audit [INF] from='client.? 172.21.15.100:0/3660147321' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110745-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:33:09.439 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:33:09 smithi100 bash[74942]: cluster 2024-04-07T16:33:08.047322+0000 mon.a (mon.0) 1693 : cluster [DBG] osdmap e934: 8 total, 8 up, 8 in 2024-04-07T16:33:09.440 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:33:09 smithi100 bash[76202]: audit 2024-04-07T16:33:08.038362+0000 mon.a (mon.0) 1692 : audit [INF] from='client.? 172.21.15.100:0/3660147321' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110745-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:33:09.440 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:33:09 smithi100 bash[76202]: cluster 2024-04-07T16:33:08.047322+0000 mon.a (mon.0) 1693 : cluster [DBG] osdmap e934: 8 total, 8 up, 8 in 2024-04-07T16:33:10.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:33:10 smithi144 bash[66654]: cluster 2024-04-07T16:33:08.930752+0000 mgr.y (mgr.25175) 534 : cluster [DBG] pgmap v1043: 137 pgs: 2 creating+activating, 15 creating+peering, 14 unknown, 106 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:33:10.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:33:10 smithi144 bash[66654]: cluster 2024-04-07T16:33:09.076440+0000 mon.a (mon.0) 1694 : cluster [DBG] osdmap e935: 8 total, 8 up, 8 in 2024-04-07T16:33:10.439 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:33:10 smithi100 bash[74942]: cluster 2024-04-07T16:33:08.930752+0000 mgr.y (mgr.25175) 534 : cluster [DBG] pgmap v1043: 137 pgs: 2 creating+activating, 15 creating+peering, 14 unknown, 106 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:33:10.439 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:33:10 smithi100 bash[74942]: cluster 2024-04-07T16:33:09.076440+0000 mon.a (mon.0) 1694 : cluster [DBG] osdmap e935: 8 total, 8 up, 8 in 2024-04-07T16:33:10.440 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:33:10 smithi100 bash[76202]: cluster 2024-04-07T16:33:08.930752+0000 mgr.y (mgr.25175) 534 : cluster [DBG] pgmap v1043: 137 pgs: 2 creating+activating, 15 creating+peering, 14 unknown, 106 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:33:10.440 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:33:10 smithi100 bash[76202]: cluster 2024-04-07T16:33:09.076440+0000 mon.a (mon.0) 1694 : cluster [DBG] osdmap e935: 8 total, 8 up, 8 in 2024-04-07T16:33:12.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:33:12 smithi144 bash[66654]: cluster 2024-04-07T16:33:10.931730+0000 mgr.y (mgr.25175) 535 : cluster [DBG] pgmap v1045: 137 pgs: 2 creating+activating, 15 creating+peering, 3 unknown, 117 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:33:12.357 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:33:12 smithi144 bash[95233]: ts=2024-04-07T16:33:12.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:33:12.439 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:33:12 smithi100 bash[74942]: cluster 2024-04-07T16:33:10.931730+0000 mgr.y (mgr.25175) 535 : cluster [DBG] pgmap v1045: 137 pgs: 2 creating+activating, 15 creating+peering, 3 unknown, 117 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:33:12.440 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:33:12 smithi100 bash[76202]: cluster 2024-04-07T16:33:10.931730+0000 mgr.y (mgr.25175) 535 : cluster [DBG] pgmap v1045: 137 pgs: 2 creating+activating, 15 creating+peering, 3 unknown, 117 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:33:14.082 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] ClsLock.TestLockDuration (8065 ms) 2024-04-07T16:33:14.082 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] ClsLock.TestAssertLocked 2024-04-07T16:33:14.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:33:14 smithi144 bash[66654]: cluster 2024-04-07T16:33:12.932973+0000 mgr.y (mgr.25175) 536 : cluster [DBG] pgmap v1046: 137 pgs: 2 creating+activating, 135 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 0 B/s rd, 0 B/s wr, 0 op/s 2024-04-07T16:33:14.439 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:33:14 smithi100 bash[74942]: cluster 2024-04-07T16:33:12.932973+0000 mgr.y (mgr.25175) 536 : cluster [DBG] pgmap v1046: 137 pgs: 2 creating+activating, 135 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 0 B/s rd, 0 B/s wr, 0 op/s 2024-04-07T16:33:14.440 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:33:14 smithi100 bash[76202]: cluster 2024-04-07T16:33:12.932973+0000 mgr.y (mgr.25175) 536 : cluster [DBG] pgmap v1046: 137 pgs: 2 creating+activating, 135 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 0 B/s rd, 0 B/s wr, 0 op/s 2024-04-07T16:33:15.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:33:15 smithi144 bash[66654]: cluster 2024-04-07T16:33:14.088069+0000 mon.a (mon.0) 1695 : cluster [DBG] osdmap e936: 8 total, 8 up, 8 in 2024-04-07T16:33:15.439 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:33:15 smithi100 bash[74942]: cluster 2024-04-07T16:33:14.088069+0000 mon.a (mon.0) 1695 : cluster [DBG] osdmap e936: 8 total, 8 up, 8 in 2024-04-07T16:33:15.440 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:33:15 smithi100 bash[76202]: cluster 2024-04-07T16:33:14.088069+0000 mon.a (mon.0) 1695 : cluster [DBG] osdmap e936: 8 total, 8 up, 8 in 2024-04-07T16:33:16.442 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:33:16 smithi100 bash[74942]: cluster 2024-04-07T16:33:14.933772+0000 mgr.y (mgr.25175) 537 : cluster [DBG] pgmap v1048: 105 pgs: 105 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:33:16.442 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:33:16 smithi100 bash[74942]: cluster 2024-04-07T16:33:15.117772+0000 mon.a (mon.0) 1696 : cluster [DBG] osdmap e937: 8 total, 8 up, 8 in 2024-04-07T16:33:16.442 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:33:16 smithi100 bash[74942]: audit 2024-04-07T16:33:15.123483+0000 mon.a (mon.0) 1697 : audit [INF] from='client.? 172.21.15.100:0/2004342264' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110745-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:33:16.443 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:33:16 smithi100 bash[76202]: cluster 2024-04-07T16:33:14.933772+0000 mgr.y (mgr.25175) 537 : cluster [DBG] pgmap v1048: 105 pgs: 105 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:33:16.443 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:33:16 smithi100 bash[76202]: cluster 2024-04-07T16:33:15.117772+0000 mon.a (mon.0) 1696 : cluster [DBG] osdmap e937: 8 total, 8 up, 8 in 2024-04-07T16:33:16.443 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:33:16 smithi100 bash[76202]: audit 2024-04-07T16:33:15.123483+0000 mon.a (mon.0) 1697 : audit [INF] from='client.? 172.21.15.100:0/2004342264' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110745-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:33:16.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:33:16 smithi144 bash[66654]: cluster 2024-04-07T16:33:14.933772+0000 mgr.y (mgr.25175) 537 : cluster [DBG] pgmap v1048: 105 pgs: 105 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:33:16.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:33:16 smithi144 bash[66654]: cluster 2024-04-07T16:33:15.117772+0000 mon.a (mon.0) 1696 : cluster [DBG] osdmap e937: 8 total, 8 up, 8 in 2024-04-07T16:33:16.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:33:16 smithi144 bash[66654]: audit 2024-04-07T16:33:15.123483+0000 mon.a (mon.0) 1697 : audit [INF] from='client.? 172.21.15.100:0/2004342264' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110745-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:33:17.145 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] ClsLock.TestAssertLocked (3063 ms) 2024-04-07T16:33:17.145 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] ClsLock.TestSetCookie 2024-04-07T16:33:17.439 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:33:17 smithi100 bash[74942]: audit 2024-04-07T16:33:16.111821+0000 mon.a (mon.0) 1698 : audit [INF] from='client.? 172.21.15.100:0/2004342264' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110745-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:33:17.440 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:33:17 smithi100 bash[74942]: cluster 2024-04-07T16:33:16.120324+0000 mon.a (mon.0) 1699 : cluster [DBG] osdmap e938: 8 total, 8 up, 8 in 2024-04-07T16:33:17.440 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:33:17 smithi100 bash[74942]: audit 2024-04-07T16:33:16.744744+0000 mon.a (mon.0) 1700 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:33:17.440 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:33:17 smithi100 bash[76202]: audit 2024-04-07T16:33:16.111821+0000 mon.a (mon.0) 1698 : audit [INF] from='client.? 172.21.15.100:0/2004342264' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110745-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:33:17.440 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:33:17 smithi100 bash[76202]: cluster 2024-04-07T16:33:16.120324+0000 mon.a (mon.0) 1699 : cluster [DBG] osdmap e938: 8 total, 8 up, 8 in 2024-04-07T16:33:17.440 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:33:17 smithi100 bash[76202]: audit 2024-04-07T16:33:16.744744+0000 mon.a (mon.0) 1700 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:33:17.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:33:17 smithi144 bash[66654]: audit 2024-04-07T16:33:16.111821+0000 mon.a (mon.0) 1698 : audit [INF] from='client.? 172.21.15.100:0/2004342264' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110745-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:33:17.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:33:17 smithi144 bash[66654]: cluster 2024-04-07T16:33:16.120324+0000 mon.a (mon.0) 1699 : cluster [DBG] osdmap e938: 8 total, 8 up, 8 in 2024-04-07T16:33:17.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:33:17 smithi144 bash[66654]: audit 2024-04-07T16:33:16.744744+0000 mon.a (mon.0) 1700 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:33:18.417 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:33:18 smithi100 bash[74942]: cluster 2024-04-07T16:33:16.934445+0000 mgr.y (mgr.25175) 538 : cluster [DBG] pgmap v1051: 137 pgs: 32 unknown, 105 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:33:18.418 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:33:18 smithi100 bash[74942]: cluster 2024-04-07T16:33:17.147752+0000 mon.a (mon.0) 1701 : cluster [DBG] osdmap e939: 8 total, 8 up, 8 in 2024-04-07T16:33:18.418 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:33:18 smithi100 bash[76202]: cluster 2024-04-07T16:33:16.934445+0000 mgr.y (mgr.25175) 538 : cluster [DBG] pgmap v1051: 137 pgs: 32 unknown, 105 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:33:18.418 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:33:18 smithi100 bash[76202]: cluster 2024-04-07T16:33:17.147752+0000 mon.a (mon.0) 1701 : cluster [DBG] osdmap e939: 8 total, 8 up, 8 in 2024-04-07T16:33:18.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:33:18 smithi144 bash[66654]: cluster 2024-04-07T16:33:16.934445+0000 mgr.y (mgr.25175) 538 : cluster [DBG] pgmap v1051: 137 pgs: 32 unknown, 105 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:33:18.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:33:18 smithi144 bash[66654]: cluster 2024-04-07T16:33:17.147752+0000 mon.a (mon.0) 1701 : cluster [DBG] osdmap e939: 8 total, 8 up, 8 in 2024-04-07T16:33:18.689 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:33:18 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:33:18] "GET /metrics HTTP/1.1" 200 36356 "" "Prometheus/2.43.0" 2024-04-07T16:33:19.439 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:33:19 smithi100 bash[74942]: cluster 2024-04-07T16:33:18.148045+0000 mon.a (mon.0) 1702 : cluster [DBG] osdmap e940: 8 total, 8 up, 8 in 2024-04-07T16:33:19.440 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:33:19 smithi100 bash[74942]: audit 2024-04-07T16:33:18.162693+0000 mon.c (mon.2) 42 : audit [INF] from='client.? 172.21.15.100:0/1270375952' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110745-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:33:19.440 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:33:19 smithi100 bash[74942]: audit 2024-04-07T16:33:18.163113+0000 mon.a (mon.0) 1703 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110745-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:33:19.440 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:33:19 smithi100 bash[74942]: audit 2024-04-07T16:33:19.143199+0000 mon.a (mon.0) 1704 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110745-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:33:19.440 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:33:19 smithi100 bash[74942]: cluster 2024-04-07T16:33:19.151327+0000 mon.a (mon.0) 1705 : cluster [DBG] osdmap e941: 8 total, 8 up, 8 in 2024-04-07T16:33:19.440 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:33:19 smithi100 bash[76202]: cluster 2024-04-07T16:33:18.148045+0000 mon.a (mon.0) 1702 : cluster [DBG] osdmap e940: 8 total, 8 up, 8 in 2024-04-07T16:33:19.440 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:33:19 smithi100 bash[76202]: audit 2024-04-07T16:33:18.162693+0000 mon.c (mon.2) 42 : audit [INF] from='client.? 172.21.15.100:0/1270375952' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110745-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:33:19.440 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:33:19 smithi100 bash[76202]: audit 2024-04-07T16:33:18.163113+0000 mon.a (mon.0) 1703 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110745-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:33:19.440 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:33:19 smithi100 bash[76202]: audit 2024-04-07T16:33:19.143199+0000 mon.a (mon.0) 1704 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110745-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:33:19.440 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:33:19 smithi100 bash[76202]: cluster 2024-04-07T16:33:19.151327+0000 mon.a (mon.0) 1705 : cluster [DBG] osdmap e941: 8 total, 8 up, 8 in 2024-04-07T16:33:19.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:33:19 smithi144 bash[66654]: cluster 2024-04-07T16:33:18.148045+0000 mon.a (mon.0) 1702 : cluster [DBG] osdmap e940: 8 total, 8 up, 8 in 2024-04-07T16:33:19.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:33:19 smithi144 bash[66654]: audit 2024-04-07T16:33:18.162693+0000 mon.c (mon.2) 42 : audit [INF] from='client.? 172.21.15.100:0/1270375952' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110745-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:33:19.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:33:19 smithi144 bash[66654]: audit 2024-04-07T16:33:18.163113+0000 mon.a (mon.0) 1703 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110745-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:33:19.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:33:19 smithi144 bash[66654]: audit 2024-04-07T16:33:19.143199+0000 mon.a (mon.0) 1704 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110745-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:33:19.608 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:33:19 smithi144 bash[66654]: cluster 2024-04-07T16:33:19.151327+0000 mon.a (mon.0) 1705 : cluster [DBG] osdmap e941: 8 total, 8 up, 8 in 2024-04-07T16:33:20.190 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] ClsLock.TestSetCookie (3046 ms) 2024-04-07T16:33:20.190 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] ClsLock.TestRenew 2024-04-07T16:33:20.439 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:33:20 smithi100 bash[74942]: cluster 2024-04-07T16:33:18.935277+0000 mgr.y (mgr.25175) 539 : cluster [DBG] pgmap v1054: 137 pgs: 32 unknown, 105 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:33:20.440 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:33:20 smithi100 bash[74942]: cluster 2024-04-07T16:33:19.154947+0000 mon.a (mon.0) 1706 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T16:33:20.440 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:33:20 smithi100 bash[76202]: cluster 2024-04-07T16:33:18.935277+0000 mgr.y (mgr.25175) 539 : cluster [DBG] pgmap v1054: 137 pgs: 32 unknown, 105 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:33:20.440 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:33:20 smithi100 bash[76202]: cluster 2024-04-07T16:33:19.154947+0000 mon.a (mon.0) 1706 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T16:33:20.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:33:20 smithi144 bash[66654]: cluster 2024-04-07T16:33:18.935277+0000 mgr.y (mgr.25175) 539 : cluster [DBG] pgmap v1054: 137 pgs: 32 unknown, 105 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:33:20.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:33:20 smithi144 bash[66654]: cluster 2024-04-07T16:33:19.154947+0000 mon.a (mon.0) 1706 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T16:33:21.439 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:33:21 smithi100 bash[74942]: cluster 2024-04-07T16:33:20.188691+0000 mon.a (mon.0) 1707 : cluster [DBG] osdmap e942: 8 total, 8 up, 8 in 2024-04-07T16:33:21.440 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:33:21 smithi100 bash[76202]: cluster 2024-04-07T16:33:20.188691+0000 mon.a (mon.0) 1707 : cluster [DBG] osdmap e942: 8 total, 8 up, 8 in 2024-04-07T16:33:21.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:33:21 smithi144 bash[66654]: cluster 2024-04-07T16:33:20.188691+0000 mon.a (mon.0) 1707 : cluster [DBG] osdmap e942: 8 total, 8 up, 8 in 2024-04-07T16:33:22.440 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:33:22 smithi100 bash[74942]: cluster 2024-04-07T16:33:20.935954+0000 mgr.y (mgr.25175) 540 : cluster [DBG] pgmap v1057: 105 pgs: 105 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:33:22.440 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:33:22 smithi100 bash[74942]: cluster 2024-04-07T16:33:21.181889+0000 mon.a (mon.0) 1708 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T16:33:22.440 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:33:22 smithi100 bash[74942]: cluster 2024-04-07T16:33:21.182005+0000 mon.a (mon.0) 1709 : cluster [INF] Cluster is now healthy 2024-04-07T16:33:22.440 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:33:22 smithi100 bash[74942]: cluster 2024-04-07T16:33:21.203795+0000 mon.a (mon.0) 1710 : cluster [DBG] osdmap e943: 8 total, 8 up, 8 in 2024-04-07T16:33:22.440 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:33:22 smithi100 bash[74942]: audit 2024-04-07T16:33:21.209225+0000 mon.c (mon.2) 43 : audit [INF] from='client.? 172.21.15.100:0/1269380384' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110745-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:33:22.440 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:33:22 smithi100 bash[74942]: audit 2024-04-07T16:33:21.210957+0000 mon.a (mon.0) 1711 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110745-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:33:22.440 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:33:22 smithi100 bash[76202]: cluster 2024-04-07T16:33:20.935954+0000 mgr.y (mgr.25175) 540 : cluster [DBG] pgmap v1057: 105 pgs: 105 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:33:22.440 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:33:22 smithi100 bash[76202]: cluster 2024-04-07T16:33:21.181889+0000 mon.a (mon.0) 1708 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T16:33:22.440 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:33:22 smithi100 bash[76202]: cluster 2024-04-07T16:33:21.182005+0000 mon.a (mon.0) 1709 : cluster [INF] Cluster is now healthy 2024-04-07T16:33:22.441 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:33:22 smithi100 bash[76202]: cluster 2024-04-07T16:33:21.203795+0000 mon.a (mon.0) 1710 : cluster [DBG] osdmap e943: 8 total, 8 up, 8 in 2024-04-07T16:33:22.441 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:33:22 smithi100 bash[76202]: audit 2024-04-07T16:33:21.209225+0000 mon.c (mon.2) 43 : audit [INF] from='client.? 172.21.15.100:0/1269380384' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110745-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:33:22.441 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:33:22 smithi100 bash[76202]: audit 2024-04-07T16:33:21.210957+0000 mon.a (mon.0) 1711 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110745-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:33:22.607 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:33:22 smithi144 bash[95233]: ts=2024-04-07T16:33:22.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:33:22.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:33:22 smithi144 bash[66654]: cluster 2024-04-07T16:33:20.935954+0000 mgr.y (mgr.25175) 540 : cluster [DBG] pgmap v1057: 105 pgs: 105 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:33:22.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:33:22 smithi144 bash[66654]: cluster 2024-04-07T16:33:21.181889+0000 mon.a (mon.0) 1708 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T16:33:22.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:33:22 smithi144 bash[66654]: cluster 2024-04-07T16:33:21.182005+0000 mon.a (mon.0) 1709 : cluster [INF] Cluster is now healthy 2024-04-07T16:33:22.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:33:22 smithi144 bash[66654]: cluster 2024-04-07T16:33:21.203795+0000 mon.a (mon.0) 1710 : cluster [DBG] osdmap e943: 8 total, 8 up, 8 in 2024-04-07T16:33:22.608 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:33:22 smithi144 bash[66654]: audit 2024-04-07T16:33:21.209225+0000 mon.c (mon.2) 43 : audit [INF] from='client.? 172.21.15.100:0/1269380384' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110745-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:33:22.608 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:33:22 smithi144 bash[66654]: audit 2024-04-07T16:33:21.210957+0000 mon.a (mon.0) 1711 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110745-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:33:23.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:33:23 smithi144 bash[66654]: audit 2024-04-07T16:33:22.205503+0000 mon.a (mon.0) 1712 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110745-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:33:23.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:33:23 smithi144 bash[66654]: cluster 2024-04-07T16:33:22.214129+0000 mon.a (mon.0) 1713 : cluster [DBG] osdmap e944: 8 total, 8 up, 8 in 2024-04-07T16:33:23.689 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:33:23 smithi100 bash[74942]: audit 2024-04-07T16:33:22.205503+0000 mon.a (mon.0) 1712 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110745-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:33:23.690 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:33:23 smithi100 bash[74942]: cluster 2024-04-07T16:33:22.214129+0000 mon.a (mon.0) 1713 : cluster [DBG] osdmap e944: 8 total, 8 up, 8 in 2024-04-07T16:33:23.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:33:23 smithi100 bash[76202]: audit 2024-04-07T16:33:22.205503+0000 mon.a (mon.0) 1712 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110745-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:33:23.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:33:23 smithi100 bash[76202]: cluster 2024-04-07T16:33:22.214129+0000 mon.a (mon.0) 1713 : cluster [DBG] osdmap e944: 8 total, 8 up, 8 in 2024-04-07T16:33:24.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:33:24 smithi144 bash[66654]: cluster 2024-04-07T16:33:22.936666+0000 mgr.y (mgr.25175) 541 : cluster [DBG] pgmap v1060: 137 pgs: 17 creating+peering, 15 unknown, 105 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:33:24.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:33:24 smithi144 bash[66654]: cluster 2024-04-07T16:33:23.226894+0000 mon.a (mon.0) 1714 : cluster [DBG] osdmap e945: 8 total, 8 up, 8 in 2024-04-07T16:33:24.689 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:33:24 smithi100 bash[74942]: cluster 2024-04-07T16:33:22.936666+0000 mgr.y (mgr.25175) 541 : cluster [DBG] pgmap v1060: 137 pgs: 17 creating+peering, 15 unknown, 105 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:33:24.690 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:33:24 smithi100 bash[74942]: cluster 2024-04-07T16:33:23.226894+0000 mon.a (mon.0) 1714 : cluster [DBG] osdmap e945: 8 total, 8 up, 8 in 2024-04-07T16:33:24.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:33:24 smithi100 bash[76202]: cluster 2024-04-07T16:33:22.936666+0000 mgr.y (mgr.25175) 541 : cluster [DBG] pgmap v1060: 137 pgs: 17 creating+peering, 15 unknown, 105 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:33:24.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:33:24 smithi100 bash[76202]: cluster 2024-04-07T16:33:23.226894+0000 mon.a (mon.0) 1714 : cluster [DBG] osdmap e945: 8 total, 8 up, 8 in 2024-04-07T16:33:26.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:33:26 smithi144 bash[66654]: cluster 2024-04-07T16:33:24.937632+0000 mgr.y (mgr.25175) 542 : cluster [DBG] pgmap v1062: 137 pgs: 17 creating+peering, 4 unknown, 116 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 0 B/s rd, 0 B/s wr, 0 op/s 2024-04-07T16:33:26.689 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:33:26 smithi100 bash[74942]: cluster 2024-04-07T16:33:24.937632+0000 mgr.y (mgr.25175) 542 : cluster [DBG] pgmap v1062: 137 pgs: 17 creating+peering, 4 unknown, 116 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 0 B/s rd, 0 B/s wr, 0 op/s 2024-04-07T16:33:26.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:33:26 smithi100 bash[76202]: cluster 2024-04-07T16:33:24.937632+0000 mgr.y (mgr.25175) 542 : cluster [DBG] pgmap v1062: 137 pgs: 17 creating+peering, 4 unknown, 116 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 0 B/s rd, 0 B/s wr, 0 op/s 2024-04-07T16:33:28.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:33:28 smithi144 bash[66654]: cluster 2024-04-07T16:33:26.938571+0000 mgr.y (mgr.25175) 543 : cluster [DBG] pgmap v1063: 137 pgs: 17 creating+peering, 120 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 0 B/s rd, 0 B/s wr, 0 op/s 2024-04-07T16:33:28.689 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:33:28 smithi100 bash[74942]: cluster 2024-04-07T16:33:26.938571+0000 mgr.y (mgr.25175) 543 : cluster [DBG] pgmap v1063: 137 pgs: 17 creating+peering, 120 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 0 B/s rd, 0 B/s wr, 0 op/s 2024-04-07T16:33:28.690 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:33:28 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:33:28] "GET /metrics HTTP/1.1" 200 36356 "" "Prometheus/2.43.0" 2024-04-07T16:33:28.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:33:28 smithi100 bash[76202]: cluster 2024-04-07T16:33:26.938571+0000 mgr.y (mgr.25175) 543 : cluster [DBG] pgmap v1063: 137 pgs: 17 creating+peering, 120 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 0 B/s rd, 0 B/s wr, 0 op/s 2024-04-07T16:33:30.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:33:30 smithi144 bash[66654]: cluster 2024-04-07T16:33:28.940009+0000 mgr.y (mgr.25175) 544 : cluster [DBG] pgmap v1064: 137 pgs: 137 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 132 B/s rd, 0 B/s wr, 0 op/s 2024-04-07T16:33:30.689 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:33:30 smithi100 bash[74942]: cluster 2024-04-07T16:33:28.940009+0000 mgr.y (mgr.25175) 544 : cluster [DBG] pgmap v1064: 137 pgs: 137 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 132 B/s rd, 0 B/s wr, 0 op/s 2024-04-07T16:33:30.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:33:30 smithi100 bash[76202]: cluster 2024-04-07T16:33:28.940009+0000 mgr.y (mgr.25175) 544 : cluster [DBG] pgmap v1064: 137 pgs: 137 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 132 B/s rd, 0 B/s wr, 0 op/s 2024-04-07T16:33:32.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:33:32 smithi144 bash[66654]: cluster 2024-04-07T16:33:30.941229+0000 mgr.y (mgr.25175) 545 : cluster [DBG] pgmap v1065: 137 pgs: 137 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 117 B/s rd, 0 B/s wr, 0 op/s 2024-04-07T16:33:32.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:33:32 smithi144 bash[66654]: audit 2024-04-07T16:33:31.745162+0000 mon.a (mon.0) 1715 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:33:32.607 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:33:32 smithi144 bash[95233]: ts=2024-04-07T16:33:32.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:33:32.690 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:33:32 smithi100 bash[74942]: cluster 2024-04-07T16:33:30.941229+0000 mgr.y (mgr.25175) 545 : cluster [DBG] pgmap v1065: 137 pgs: 137 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 117 B/s rd, 0 B/s wr, 0 op/s 2024-04-07T16:33:32.690 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:33:32 smithi100 bash[74942]: audit 2024-04-07T16:33:31.745162+0000 mon.a (mon.0) 1715 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:33:32.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:33:32 smithi100 bash[76202]: cluster 2024-04-07T16:33:30.941229+0000 mgr.y (mgr.25175) 545 : cluster [DBG] pgmap v1065: 137 pgs: 137 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 117 B/s rd, 0 B/s wr, 0 op/s 2024-04-07T16:33:32.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:33:32 smithi100 bash[76202]: audit 2024-04-07T16:33:31.745162+0000 mon.a (mon.0) 1715 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:33:34.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:33:34 smithi144 bash[66654]: cluster 2024-04-07T16:33:32.942554+0000 mgr.y (mgr.25175) 546 : cluster [DBG] pgmap v1066: 137 pgs: 137 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 102 B/s rd, 0 B/s wr, 0 op/s 2024-04-07T16:33:34.690 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:33:34 smithi100 bash[74942]: cluster 2024-04-07T16:33:32.942554+0000 mgr.y (mgr.25175) 546 : cluster [DBG] pgmap v1066: 137 pgs: 137 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 102 B/s rd, 0 B/s wr, 0 op/s 2024-04-07T16:33:34.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:33:34 smithi100 bash[76202]: cluster 2024-04-07T16:33:32.942554+0000 mgr.y (mgr.25175) 546 : cluster [DBG] pgmap v1066: 137 pgs: 137 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 102 B/s rd, 0 B/s wr, 0 op/s 2024-04-07T16:33:36.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:33:36 smithi144 bash[66654]: cluster 2024-04-07T16:33:34.943720+0000 mgr.y (mgr.25175) 547 : cluster [DBG] pgmap v1067: 137 pgs: 137 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 262 B/s rd, 0 B/s wr, 0 op/s 2024-04-07T16:33:36.689 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:33:36 smithi100 bash[74942]: cluster 2024-04-07T16:33:34.943720+0000 mgr.y (mgr.25175) 547 : cluster [DBG] pgmap v1067: 137 pgs: 137 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 262 B/s rd, 0 B/s wr, 0 op/s 2024-04-07T16:33:36.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:33:36 smithi100 bash[76202]: cluster 2024-04-07T16:33:34.943720+0000 mgr.y (mgr.25175) 547 : cluster [DBG] pgmap v1067: 137 pgs: 137 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 262 B/s rd, 0 B/s wr, 0 op/s 2024-04-07T16:33:37.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:33:37 smithi144 bash[66654]: audit 2024-04-07T16:33:36.441004+0000 mon.a (mon.0) 1716 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:33:37.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:33:37 smithi144 bash[66654]: audit 2024-04-07T16:33:36.967068+0000 mon.a (mon.0) 1717 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:33:37.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:33:37 smithi144 bash[66654]: audit 2024-04-07T16:33:36.974679+0000 mon.a (mon.0) 1718 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:33:37.690 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:33:37 smithi100 bash[74942]: audit 2024-04-07T16:33:36.441004+0000 mon.a (mon.0) 1716 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:33:37.690 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:33:37 smithi100 bash[74942]: audit 2024-04-07T16:33:36.967068+0000 mon.a (mon.0) 1717 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:33:37.690 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:33:37 smithi100 bash[74942]: audit 2024-04-07T16:33:36.974679+0000 mon.a (mon.0) 1718 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:33:37.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:33:37 smithi100 bash[76202]: audit 2024-04-07T16:33:36.441004+0000 mon.a (mon.0) 1716 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:33:37.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:33:37 smithi100 bash[76202]: audit 2024-04-07T16:33:36.967068+0000 mon.a (mon.0) 1717 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:33:37.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:33:37 smithi100 bash[76202]: audit 2024-04-07T16:33:36.974679+0000 mon.a (mon.0) 1718 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:33:38.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:33:38 smithi144 bash[66654]: cluster 2024-04-07T16:33:36.944283+0000 mgr.y (mgr.25175) 548 : cluster [DBG] pgmap v1068: 137 pgs: 137 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 255 B/s rd, 0 B/s wr, 0 op/s 2024-04-07T16:33:38.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:33:38 smithi144 bash[66654]: audit 2024-04-07T16:33:37.703753+0000 mon.a (mon.0) 1719 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:33:38.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:33:38 smithi144 bash[66654]: audit 2024-04-07T16:33:37.704997+0000 mon.a (mon.0) 1720 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:33:38.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:33:38 smithi144 bash[66654]: audit 2024-04-07T16:33:37.713719+0000 mon.a (mon.0) 1721 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:33:38.690 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:33:38 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:33:38] "GET /metrics HTTP/1.1" 200 36361 "" "Prometheus/2.43.0" 2024-04-07T16:33:38.690 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:33:38 smithi100 bash[74942]: cluster 2024-04-07T16:33:36.944283+0000 mgr.y (mgr.25175) 548 : cluster [DBG] pgmap v1068: 137 pgs: 137 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 255 B/s rd, 0 B/s wr, 0 op/s 2024-04-07T16:33:38.690 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:33:38 smithi100 bash[74942]: audit 2024-04-07T16:33:37.703753+0000 mon.a (mon.0) 1719 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:33:38.690 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:33:38 smithi100 bash[74942]: audit 2024-04-07T16:33:37.704997+0000 mon.a (mon.0) 1720 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:33:38.690 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:33:38 smithi100 bash[74942]: audit 2024-04-07T16:33:37.713719+0000 mon.a (mon.0) 1721 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:33:38.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:33:38 smithi100 bash[76202]: cluster 2024-04-07T16:33:36.944283+0000 mgr.y (mgr.25175) 548 : cluster [DBG] pgmap v1068: 137 pgs: 137 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 255 B/s rd, 0 B/s wr, 0 op/s 2024-04-07T16:33:38.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:33:38 smithi100 bash[76202]: audit 2024-04-07T16:33:37.703753+0000 mon.a (mon.0) 1719 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:33:38.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:33:38 smithi100 bash[76202]: audit 2024-04-07T16:33:37.704997+0000 mon.a (mon.0) 1720 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:33:38.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:33:38 smithi100 bash[76202]: audit 2024-04-07T16:33:37.713719+0000 mon.a (mon.0) 1721 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:33:40.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:33:40 smithi144 bash[66654]: cluster 2024-04-07T16:33:38.945538+0000 mgr.y (mgr.25175) 549 : cluster [DBG] pgmap v1069: 137 pgs: 137 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 341 B/s rd, 0 B/s wr, 0 op/s 2024-04-07T16:33:40.690 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:33:40 smithi100 bash[74942]: cluster 2024-04-07T16:33:38.945538+0000 mgr.y (mgr.25175) 549 : cluster [DBG] pgmap v1069: 137 pgs: 137 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 341 B/s rd, 0 B/s wr, 0 op/s 2024-04-07T16:33:40.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:33:40 smithi100 bash[76202]: cluster 2024-04-07T16:33:38.945538+0000 mgr.y (mgr.25175) 549 : cluster [DBG] pgmap v1069: 137 pgs: 137 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 341 B/s rd, 0 B/s wr, 0 op/s 2024-04-07T16:33:41.278 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] ClsLock.TestRenew (21079 ms) 2024-04-07T16:33:41.278 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] ClsLock.TestExclusiveEphemeralBasic 2024-04-07T16:33:42.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:33:42 smithi144 bash[66654]: cluster 2024-04-07T16:33:40.946635+0000 mgr.y (mgr.25175) 550 : cluster [DBG] pgmap v1070: 137 pgs: 137 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 255 B/s rd, 0 B/s wr, 0 op/s 2024-04-07T16:33:42.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:33:42 smithi144 bash[66654]: cluster 2024-04-07T16:33:41.275627+0000 mon.a (mon.0) 1722 : cluster [DBG] osdmap e946: 8 total, 8 up, 8 in 2024-04-07T16:33:42.607 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:33:42 smithi144 bash[95233]: ts=2024-04-07T16:33:42.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:33:42.690 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:33:42 smithi100 bash[74942]: cluster 2024-04-07T16:33:40.946635+0000 mgr.y (mgr.25175) 550 : cluster [DBG] pgmap v1070: 137 pgs: 137 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 255 B/s rd, 0 B/s wr, 0 op/s 2024-04-07T16:33:42.690 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:33:42 smithi100 bash[74942]: cluster 2024-04-07T16:33:41.275627+0000 mon.a (mon.0) 1722 : cluster [DBG] osdmap e946: 8 total, 8 up, 8 in 2024-04-07T16:33:42.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:33:42 smithi100 bash[76202]: cluster 2024-04-07T16:33:40.946635+0000 mgr.y (mgr.25175) 550 : cluster [DBG] pgmap v1070: 137 pgs: 137 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 255 B/s rd, 0 B/s wr, 0 op/s 2024-04-07T16:33:42.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:33:42 smithi100 bash[76202]: cluster 2024-04-07T16:33:41.275627+0000 mon.a (mon.0) 1722 : cluster [DBG] osdmap e946: 8 total, 8 up, 8 in 2024-04-07T16:33:43.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:33:43 smithi144 bash[66654]: cluster 2024-04-07T16:33:42.282596+0000 mon.a (mon.0) 1723 : cluster [DBG] osdmap e947: 8 total, 8 up, 8 in 2024-04-07T16:33:43.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:33:43 smithi144 bash[66654]: audit 2024-04-07T16:33:42.290725+0000 mon.a (mon.0) 1724 : audit [INF] from='client.? 172.21.15.100:0/4200585966' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110745-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:33:43.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:33:43 smithi144 bash[66654]: audit 2024-04-07T16:33:43.275122+0000 mon.a (mon.0) 1725 : audit [INF] from='client.? 172.21.15.100:0/4200585966' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110745-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:33:43.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:33:43 smithi144 bash[66654]: cluster 2024-04-07T16:33:43.284277+0000 mon.a (mon.0) 1726 : cluster [DBG] osdmap e948: 8 total, 8 up, 8 in 2024-04-07T16:33:43.690 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:33:43 smithi100 bash[74942]: cluster 2024-04-07T16:33:42.282596+0000 mon.a (mon.0) 1723 : cluster [DBG] osdmap e947: 8 total, 8 up, 8 in 2024-04-07T16:33:43.690 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:33:43 smithi100 bash[74942]: audit 2024-04-07T16:33:42.290725+0000 mon.a (mon.0) 1724 : audit [INF] from='client.? 172.21.15.100:0/4200585966' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110745-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:33:43.690 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:33:43 smithi100 bash[74942]: audit 2024-04-07T16:33:43.275122+0000 mon.a (mon.0) 1725 : audit [INF] from='client.? 172.21.15.100:0/4200585966' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110745-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:33:43.690 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:33:43 smithi100 bash[74942]: cluster 2024-04-07T16:33:43.284277+0000 mon.a (mon.0) 1726 : cluster [DBG] osdmap e948: 8 total, 8 up, 8 in 2024-04-07T16:33:43.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:33:43 smithi100 bash[76202]: cluster 2024-04-07T16:33:42.282596+0000 mon.a (mon.0) 1723 : cluster [DBG] osdmap e947: 8 total, 8 up, 8 in 2024-04-07T16:33:43.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:33:43 smithi100 bash[76202]: audit 2024-04-07T16:33:42.290725+0000 mon.a (mon.0) 1724 : audit [INF] from='client.? 172.21.15.100:0/4200585966' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110745-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:33:43.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:33:43 smithi100 bash[76202]: audit 2024-04-07T16:33:43.275122+0000 mon.a (mon.0) 1725 : audit [INF] from='client.? 172.21.15.100:0/4200585966' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110745-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:33:43.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:33:43 smithi100 bash[76202]: cluster 2024-04-07T16:33:43.284277+0000 mon.a (mon.0) 1726 : cluster [DBG] osdmap e948: 8 total, 8 up, 8 in 2024-04-07T16:33:44.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:33:44 smithi144 bash[66654]: cluster 2024-04-07T16:33:42.947397+0000 mgr.y (mgr.25175) 551 : cluster [DBG] pgmap v1073: 137 pgs: 32 unknown, 105 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:33:44.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:33:44 smithi144 bash[66654]: cluster 2024-04-07T16:33:43.296153+0000 mon.a (mon.0) 1727 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T16:33:44.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:33:44 smithi144 bash[66654]: cluster 2024-04-07T16:33:44.285167+0000 mon.a (mon.0) 1728 : cluster [DBG] osdmap e949: 8 total, 8 up, 8 in 2024-04-07T16:33:44.690 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:33:44 smithi100 bash[74942]: cluster 2024-04-07T16:33:42.947397+0000 mgr.y (mgr.25175) 551 : cluster [DBG] pgmap v1073: 137 pgs: 32 unknown, 105 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:33:44.690 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:33:44 smithi100 bash[74942]: cluster 2024-04-07T16:33:43.296153+0000 mon.a (mon.0) 1727 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T16:33:44.690 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:33:44 smithi100 bash[74942]: cluster 2024-04-07T16:33:44.285167+0000 mon.a (mon.0) 1728 : cluster [DBG] osdmap e949: 8 total, 8 up, 8 in 2024-04-07T16:33:44.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:33:44 smithi100 bash[76202]: cluster 2024-04-07T16:33:42.947397+0000 mgr.y (mgr.25175) 551 : cluster [DBG] pgmap v1073: 137 pgs: 32 unknown, 105 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:33:44.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:33:44 smithi100 bash[76202]: cluster 2024-04-07T16:33:43.296153+0000 mon.a (mon.0) 1727 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T16:33:44.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:33:44 smithi100 bash[76202]: cluster 2024-04-07T16:33:44.285167+0000 mon.a (mon.0) 1728 : cluster [DBG] osdmap e949: 8 total, 8 up, 8 in 2024-04-07T16:33:45.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:33:45 smithi144 bash[66654]: cluster 2024-04-07T16:33:45.301711+0000 mon.a (mon.0) 1729 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T16:33:45.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:33:45 smithi144 bash[66654]: cluster 2024-04-07T16:33:45.301774+0000 mon.a (mon.0) 1730 : cluster [INF] Cluster is now healthy 2024-04-07T16:33:45.690 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:33:45 smithi100 bash[74942]: cluster 2024-04-07T16:33:45.301711+0000 mon.a (mon.0) 1729 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T16:33:45.690 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:33:45 smithi100 bash[74942]: cluster 2024-04-07T16:33:45.301774+0000 mon.a (mon.0) 1730 : cluster [INF] Cluster is now healthy 2024-04-07T16:33:45.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:33:45 smithi100 bash[76202]: cluster 2024-04-07T16:33:45.301711+0000 mon.a (mon.0) 1729 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T16:33:45.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:33:45 smithi100 bash[76202]: cluster 2024-04-07T16:33:45.301774+0000 mon.a (mon.0) 1730 : cluster [INF] Cluster is now healthy 2024-04-07T16:33:46.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:33:46 smithi144 bash[66654]: cluster 2024-04-07T16:33:44.948103+0000 mgr.y (mgr.25175) 552 : cluster [DBG] pgmap v1076: 137 pgs: 1 creating+peering, 23 unknown, 113 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:33:46.690 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:33:46 smithi100 bash[74942]: cluster 2024-04-07T16:33:44.948103+0000 mgr.y (mgr.25175) 552 : cluster [DBG] pgmap v1076: 137 pgs: 1 creating+peering, 23 unknown, 113 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:33:46.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:33:46 smithi100 bash[76202]: cluster 2024-04-07T16:33:44.948103+0000 mgr.y (mgr.25175) 552 : cluster [DBG] pgmap v1076: 137 pgs: 1 creating+peering, 23 unknown, 113 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:33:47.376 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] ClsLock.TestExclusiveEphemeralBasic (6106 ms) 2024-04-07T16:33:47.377 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] ClsLock.TestExclusiveEphemeralStealEphemeral 2024-04-07T16:33:47.690 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:33:47 smithi100 bash[74942]: audit 2024-04-07T16:33:46.745523+0000 mon.a (mon.0) 1731 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:33:47.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:33:47 smithi100 bash[76202]: audit 2024-04-07T16:33:46.745523+0000 mon.a (mon.0) 1731 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:33:47.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:33:47 smithi144 bash[66654]: audit 2024-04-07T16:33:46.745523+0000 mon.a (mon.0) 1731 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:33:48.690 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:33:48 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:33:48] "GET /metrics HTTP/1.1" 200 36355 "" "Prometheus/2.43.0" 2024-04-07T16:33:48.690 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:33:48 smithi100 bash[74942]: cluster 2024-04-07T16:33:46.948940+0000 mgr.y (mgr.25175) 553 : cluster [DBG] pgmap v1077: 137 pgs: 1 creating+peering, 18 unknown, 118 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 0 B/s rd, 0 B/s wr, 0 op/s 2024-04-07T16:33:48.690 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:33:48 smithi100 bash[74942]: cluster 2024-04-07T16:33:47.384041+0000 mon.a (mon.0) 1732 : cluster [DBG] osdmap e950: 8 total, 8 up, 8 in 2024-04-07T16:33:48.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:33:48 smithi100 bash[76202]: cluster 2024-04-07T16:33:46.948940+0000 mgr.y (mgr.25175) 553 : cluster [DBG] pgmap v1077: 137 pgs: 1 creating+peering, 18 unknown, 118 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 0 B/s rd, 0 B/s wr, 0 op/s 2024-04-07T16:33:48.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:33:48 smithi100 bash[76202]: cluster 2024-04-07T16:33:47.384041+0000 mon.a (mon.0) 1732 : cluster [DBG] osdmap e950: 8 total, 8 up, 8 in 2024-04-07T16:33:48.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:33:48 smithi144 bash[66654]: cluster 2024-04-07T16:33:46.948940+0000 mgr.y (mgr.25175) 553 : cluster [DBG] pgmap v1077: 137 pgs: 1 creating+peering, 18 unknown, 118 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 0 B/s rd, 0 B/s wr, 0 op/s 2024-04-07T16:33:48.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:33:48 smithi144 bash[66654]: cluster 2024-04-07T16:33:47.384041+0000 mon.a (mon.0) 1732 : cluster [DBG] osdmap e950: 8 total, 8 up, 8 in 2024-04-07T16:33:49.690 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:33:49 smithi100 bash[74942]: cluster 2024-04-07T16:33:48.388625+0000 mon.a (mon.0) 1733 : cluster [DBG] osdmap e951: 8 total, 8 up, 8 in 2024-04-07T16:33:49.690 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:33:49 smithi100 bash[74942]: audit 2024-04-07T16:33:48.392082+0000 mon.a (mon.0) 1734 : audit [INF] from='client.? 172.21.15.100:0/3120357972' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110745-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:33:49.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:33:49 smithi100 bash[76202]: cluster 2024-04-07T16:33:48.388625+0000 mon.a (mon.0) 1733 : cluster [DBG] osdmap e951: 8 total, 8 up, 8 in 2024-04-07T16:33:49.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:33:49 smithi100 bash[76202]: audit 2024-04-07T16:33:48.392082+0000 mon.a (mon.0) 1734 : audit [INF] from='client.? 172.21.15.100:0/3120357972' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110745-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:33:49.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:33:49 smithi144 bash[66654]: cluster 2024-04-07T16:33:48.388625+0000 mon.a (mon.0) 1733 : cluster [DBG] osdmap e951: 8 total, 8 up, 8 in 2024-04-07T16:33:49.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:33:49 smithi144 bash[66654]: audit 2024-04-07T16:33:48.392082+0000 mon.a (mon.0) 1734 : audit [INF] from='client.? 172.21.15.100:0/3120357972' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110745-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:33:50.690 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:33:50 smithi100 bash[74942]: cluster 2024-04-07T16:33:48.949659+0000 mgr.y (mgr.25175) 554 : cluster [DBG] pgmap v1080: 137 pgs: 32 unknown, 105 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:33:50.690 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:33:50 smithi100 bash[74942]: cluster 2024-04-07T16:33:49.383125+0000 mon.a (mon.0) 1735 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T16:33:50.690 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:33:50 smithi100 bash[74942]: audit 2024-04-07T16:33:49.384787+0000 mon.a (mon.0) 1736 : audit [INF] from='client.? 172.21.15.100:0/3120357972' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110745-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:33:50.690 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:33:50 smithi100 bash[74942]: cluster 2024-04-07T16:33:49.394597+0000 mon.a (mon.0) 1737 : cluster [DBG] osdmap e952: 8 total, 8 up, 8 in 2024-04-07T16:33:50.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:33:50 smithi100 bash[76202]: cluster 2024-04-07T16:33:48.949659+0000 mgr.y (mgr.25175) 554 : cluster [DBG] pgmap v1080: 137 pgs: 32 unknown, 105 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:33:50.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:33:50 smithi100 bash[76202]: cluster 2024-04-07T16:33:49.383125+0000 mon.a (mon.0) 1735 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T16:33:50.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:33:50 smithi100 bash[76202]: audit 2024-04-07T16:33:49.384787+0000 mon.a (mon.0) 1736 : audit [INF] from='client.? 172.21.15.100:0/3120357972' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110745-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:33:50.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:33:50 smithi100 bash[76202]: cluster 2024-04-07T16:33:49.394597+0000 mon.a (mon.0) 1737 : cluster [DBG] osdmap e952: 8 total, 8 up, 8 in 2024-04-07T16:33:50.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:33:50 smithi144 bash[66654]: cluster 2024-04-07T16:33:48.949659+0000 mgr.y (mgr.25175) 554 : cluster [DBG] pgmap v1080: 137 pgs: 32 unknown, 105 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:33:50.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:33:50 smithi144 bash[66654]: cluster 2024-04-07T16:33:49.383125+0000 mon.a (mon.0) 1735 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T16:33:50.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:33:50 smithi144 bash[66654]: audit 2024-04-07T16:33:49.384787+0000 mon.a (mon.0) 1736 : audit [INF] from='client.? 172.21.15.100:0/3120357972' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110745-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:33:50.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:33:50 smithi144 bash[66654]: cluster 2024-04-07T16:33:49.394597+0000 mon.a (mon.0) 1737 : cluster [DBG] osdmap e952: 8 total, 8 up, 8 in 2024-04-07T16:33:51.690 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:33:51 smithi100 bash[74942]: cluster 2024-04-07T16:33:50.410885+0000 mon.a (mon.0) 1738 : cluster [DBG] osdmap e953: 8 total, 8 up, 8 in 2024-04-07T16:33:51.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:33:51 smithi100 bash[76202]: cluster 2024-04-07T16:33:50.410885+0000 mon.a (mon.0) 1738 : cluster [DBG] osdmap e953: 8 total, 8 up, 8 in 2024-04-07T16:33:51.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:33:51 smithi144 bash[66654]: cluster 2024-04-07T16:33:50.410885+0000 mon.a (mon.0) 1738 : cluster [DBG] osdmap e953: 8 total, 8 up, 8 in 2024-04-07T16:33:52.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:33:52 smithi144 bash[66654]: cluster 2024-04-07T16:33:50.950385+0000 mgr.y (mgr.25175) 555 : cluster [DBG] pgmap v1083: 137 pgs: 11 creating+peering, 20 unknown, 106 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:33:52.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:33:52 smithi144 bash[66654]: cluster 2024-04-07T16:33:51.414185+0000 mon.a (mon.0) 1739 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T16:33:52.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:33:52 smithi144 bash[66654]: cluster 2024-04-07T16:33:51.414253+0000 mon.a (mon.0) 1740 : cluster [INF] Cluster is now healthy 2024-04-07T16:33:52.607 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:33:52 smithi144 bash[95233]: ts=2024-04-07T16:33:52.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:33:52.690 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:33:52 smithi100 bash[74942]: cluster 2024-04-07T16:33:50.950385+0000 mgr.y (mgr.25175) 555 : cluster [DBG] pgmap v1083: 137 pgs: 11 creating+peering, 20 unknown, 106 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:33:52.690 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:33:52 smithi100 bash[74942]: cluster 2024-04-07T16:33:51.414185+0000 mon.a (mon.0) 1739 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T16:33:52.690 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:33:52 smithi100 bash[74942]: cluster 2024-04-07T16:33:51.414253+0000 mon.a (mon.0) 1740 : cluster [INF] Cluster is now healthy 2024-04-07T16:33:52.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:33:52 smithi100 bash[76202]: cluster 2024-04-07T16:33:50.950385+0000 mgr.y (mgr.25175) 555 : cluster [DBG] pgmap v1083: 137 pgs: 11 creating+peering, 20 unknown, 106 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:33:52.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:33:52 smithi100 bash[76202]: cluster 2024-04-07T16:33:51.414185+0000 mon.a (mon.0) 1739 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T16:33:52.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:33:52 smithi100 bash[76202]: cluster 2024-04-07T16:33:51.414253+0000 mon.a (mon.0) 1740 : cluster [INF] Cluster is now healthy 2024-04-07T16:33:54.690 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:33:54 smithi100 bash[74942]: cluster 2024-04-07T16:33:52.951553+0000 mgr.y (mgr.25175) 556 : cluster [DBG] pgmap v1084: 137 pgs: 11 creating+peering, 2 unknown, 124 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:33:54.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:33:54 smithi100 bash[76202]: cluster 2024-04-07T16:33:52.951553+0000 mgr.y (mgr.25175) 556 : cluster [DBG] pgmap v1084: 137 pgs: 11 creating+peering, 2 unknown, 124 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:33:54.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:33:54 smithi144 bash[66654]: cluster 2024-04-07T16:33:52.951553+0000 mgr.y (mgr.25175) 556 : cluster [DBG] pgmap v1084: 137 pgs: 11 creating+peering, 2 unknown, 124 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:33:55.440 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] ClsLock.TestExclusiveEphemeralStealEphemeral (8064 ms) 2024-04-07T16:33:55.440 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] ClsLock.TestExclusiveEphemeralStealExclusive 2024-04-07T16:33:56.690 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:33:56 smithi100 bash[74942]: cluster 2024-04-07T16:33:54.952579+0000 mgr.y (mgr.25175) 557 : cluster [DBG] pgmap v1085: 137 pgs: 137 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 155 B/s rd, 0 B/s wr, 0 op/s 2024-04-07T16:33:56.690 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:33:56 smithi100 bash[74942]: cluster 2024-04-07T16:33:55.448048+0000 mon.a (mon.0) 1741 : cluster [DBG] osdmap e954: 8 total, 8 up, 8 in 2024-04-07T16:33:56.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:33:56 smithi100 bash[76202]: cluster 2024-04-07T16:33:54.952579+0000 mgr.y (mgr.25175) 557 : cluster [DBG] pgmap v1085: 137 pgs: 137 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 155 B/s rd, 0 B/s wr, 0 op/s 2024-04-07T16:33:56.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:33:56 smithi100 bash[76202]: cluster 2024-04-07T16:33:55.448048+0000 mon.a (mon.0) 1741 : cluster [DBG] osdmap e954: 8 total, 8 up, 8 in 2024-04-07T16:33:56.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:33:56 smithi144 bash[66654]: cluster 2024-04-07T16:33:54.952579+0000 mgr.y (mgr.25175) 557 : cluster [DBG] pgmap v1085: 137 pgs: 137 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 155 B/s rd, 0 B/s wr, 0 op/s 2024-04-07T16:33:56.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:33:56 smithi144 bash[66654]: cluster 2024-04-07T16:33:55.448048+0000 mon.a (mon.0) 1741 : cluster [DBG] osdmap e954: 8 total, 8 up, 8 in 2024-04-07T16:33:57.690 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:33:57 smithi100 bash[74942]: cluster 2024-04-07T16:33:56.456805+0000 mon.a (mon.0) 1742 : cluster [DBG] osdmap e955: 8 total, 8 up, 8 in 2024-04-07T16:33:57.690 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:33:57 smithi100 bash[74942]: audit 2024-04-07T16:33:56.465136+0000 mon.a (mon.0) 1743 : audit [INF] from='client.? 172.21.15.100:0/2561817170' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110745-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:33:57.690 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:33:57 smithi100 bash[76202]: cluster 2024-04-07T16:33:56.456805+0000 mon.a (mon.0) 1742 : cluster [DBG] osdmap e955: 8 total, 8 up, 8 in 2024-04-07T16:33:57.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:33:57 smithi100 bash[76202]: audit 2024-04-07T16:33:56.465136+0000 mon.a (mon.0) 1743 : audit [INF] from='client.? 172.21.15.100:0/2561817170' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110745-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:33:57.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:33:57 smithi144 bash[66654]: cluster 2024-04-07T16:33:56.456805+0000 mon.a (mon.0) 1742 : cluster [DBG] osdmap e955: 8 total, 8 up, 8 in 2024-04-07T16:33:57.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:33:57 smithi144 bash[66654]: audit 2024-04-07T16:33:56.465136+0000 mon.a (mon.0) 1743 : audit [INF] from='client.? 172.21.15.100:0/2561817170' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110745-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:33:58.690 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:33:58 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:33:58] "GET /metrics HTTP/1.1" 200 36355 "" "Prometheus/2.43.0" 2024-04-07T16:33:58.690 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:33:58 smithi100 bash[74942]: cluster 2024-04-07T16:33:56.953279+0000 mgr.y (mgr.25175) 558 : cluster [DBG] pgmap v1088: 137 pgs: 32 unknown, 105 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:33:58.690 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:33:58 smithi100 bash[74942]: audit 2024-04-07T16:33:57.462012+0000 mon.a (mon.0) 1744 : audit [INF] from='client.? 172.21.15.100:0/2561817170' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110745-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:33:58.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:33:58 smithi100 bash[74942]: cluster 2024-04-07T16:33:57.468362+0000 mon.a (mon.0) 1745 : cluster [DBG] osdmap e956: 8 total, 8 up, 8 in 2024-04-07T16:33:58.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:33:58 smithi100 bash[74942]: cluster 2024-04-07T16:33:57.470717+0000 mon.a (mon.0) 1746 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T16:33:58.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:33:58 smithi100 bash[76202]: cluster 2024-04-07T16:33:56.953279+0000 mgr.y (mgr.25175) 558 : cluster [DBG] pgmap v1088: 137 pgs: 32 unknown, 105 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:33:58.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:33:58 smithi100 bash[76202]: audit 2024-04-07T16:33:57.462012+0000 mon.a (mon.0) 1744 : audit [INF] from='client.? 172.21.15.100:0/2561817170' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110745-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:33:58.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:33:58 smithi100 bash[76202]: cluster 2024-04-07T16:33:57.468362+0000 mon.a (mon.0) 1745 : cluster [DBG] osdmap e956: 8 total, 8 up, 8 in 2024-04-07T16:33:58.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:33:58 smithi100 bash[76202]: cluster 2024-04-07T16:33:57.470717+0000 mon.a (mon.0) 1746 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T16:33:58.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:33:58 smithi144 bash[66654]: cluster 2024-04-07T16:33:56.953279+0000 mgr.y (mgr.25175) 558 : cluster [DBG] pgmap v1088: 137 pgs: 32 unknown, 105 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:33:58.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:33:58 smithi144 bash[66654]: audit 2024-04-07T16:33:57.462012+0000 mon.a (mon.0) 1744 : audit [INF] from='client.? 172.21.15.100:0/2561817170' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-110745-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:33:58.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:33:58 smithi144 bash[66654]: cluster 2024-04-07T16:33:57.468362+0000 mon.a (mon.0) 1745 : cluster [DBG] osdmap e956: 8 total, 8 up, 8 in 2024-04-07T16:33:58.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:33:58 smithi144 bash[66654]: cluster 2024-04-07T16:33:57.470717+0000 mon.a (mon.0) 1746 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T16:33:59.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:33:59 smithi144 bash[66654]: cluster 2024-04-07T16:33:58.486865+0000 mon.a (mon.0) 1747 : cluster [DBG] osdmap e957: 8 total, 8 up, 8 in 2024-04-07T16:33:59.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:33:59 smithi144 bash[66654]: cluster 2024-04-07T16:33:58.953994+0000 mgr.y (mgr.25175) 559 : cluster [DBG] pgmap v1091: 137 pgs: 19 creating+peering, 8 unknown, 110 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:33:59.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:33:59 smithi100 bash[74942]: cluster 2024-04-07T16:33:58.486865+0000 mon.a (mon.0) 1747 : cluster [DBG] osdmap e957: 8 total, 8 up, 8 in 2024-04-07T16:33:59.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:33:59 smithi100 bash[74942]: cluster 2024-04-07T16:33:58.953994+0000 mgr.y (mgr.25175) 559 : cluster [DBG] pgmap v1091: 137 pgs: 19 creating+peering, 8 unknown, 110 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:33:59.940 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:33:59 smithi100 bash[76202]: cluster 2024-04-07T16:33:58.486865+0000 mon.a (mon.0) 1747 : cluster [DBG] osdmap e957: 8 total, 8 up, 8 in 2024-04-07T16:33:59.940 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:33:59 smithi100 bash[76202]: cluster 2024-04-07T16:33:58.953994+0000 mgr.y (mgr.25175) 559 : cluster [DBG] pgmap v1091: 137 pgs: 19 creating+peering, 8 unknown, 110 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:34:00.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:34:00 smithi144 bash[66654]: cluster 2024-04-07T16:33:59.479458+0000 mon.a (mon.0) 1748 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T16:34:00.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:34:00 smithi144 bash[66654]: cluster 2024-04-07T16:33:59.479529+0000 mon.a (mon.0) 1749 : cluster [INF] Cluster is now healthy 2024-04-07T16:34:00.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:34:00 smithi100 bash[74942]: cluster 2024-04-07T16:33:59.479458+0000 mon.a (mon.0) 1748 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T16:34:00.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:34:00 smithi100 bash[74942]: cluster 2024-04-07T16:33:59.479529+0000 mon.a (mon.0) 1749 : cluster [INF] Cluster is now healthy 2024-04-07T16:34:00.940 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:34:00 smithi100 bash[76202]: cluster 2024-04-07T16:33:59.479458+0000 mon.a (mon.0) 1748 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T16:34:00.940 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:34:00 smithi100 bash[76202]: cluster 2024-04-07T16:33:59.479529+0000 mon.a (mon.0) 1749 : cluster [INF] Cluster is now healthy 2024-04-07T16:34:01.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:34:01 smithi144 bash[66654]: cluster 2024-04-07T16:34:00.955042+0000 mgr.y (mgr.25175) 560 : cluster [DBG] pgmap v1092: 137 pgs: 19 creating+peering, 118 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 0 B/s rd, 0 B/s wr, 0 op/s 2024-04-07T16:34:01.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:34:01 smithi100 bash[74942]: cluster 2024-04-07T16:34:00.955042+0000 mgr.y (mgr.25175) 560 : cluster [DBG] pgmap v1092: 137 pgs: 19 creating+peering, 118 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 0 B/s rd, 0 B/s wr, 0 op/s 2024-04-07T16:34:01.940 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:34:01 smithi100 bash[76202]: cluster 2024-04-07T16:34:00.955042+0000 mgr.y (mgr.25175) 560 : cluster [DBG] pgmap v1092: 137 pgs: 19 creating+peering, 118 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 0 B/s rd, 0 B/s wr, 0 op/s 2024-04-07T16:34:02.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:34:02 smithi144 bash[66654]: audit 2024-04-07T16:34:01.745953+0000 mon.a (mon.0) 1750 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:34:02.607 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:34:02 smithi144 bash[95233]: ts=2024-04-07T16:34:02.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:34:02.942 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:34:02 smithi100 bash[74942]: audit 2024-04-07T16:34:01.745953+0000 mon.a (mon.0) 1750 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:34:02.942 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:34:02 smithi100 bash[76202]: audit 2024-04-07T16:34:01.745953+0000 mon.a (mon.0) 1750 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:34:03.513 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] ClsLock.TestExclusiveEphemeralStealExclusive (8073 ms) 2024-04-07T16:34:03.513 INFO:tasks.workunit.client.0.smithi100.stdout:[----------] 11 tests from ClsLock (69513 ms total) 2024-04-07T16:34:03.514 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T16:34:03.514 INFO:tasks.workunit.client.0.smithi100.stdout:[----------] Global test environment tear-down 2024-04-07T16:34:03.514 INFO:tasks.workunit.client.0.smithi100.stdout:[==========] 11 tests from 1 test suite ran. (69513 ms total) 2024-04-07T16:34:03.514 INFO:tasks.workunit.client.0.smithi100.stdout:[ PASSED ] 11 tests. 2024-04-07T16:34:03.520 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-07T16:34:03.520 DEBUG:teuthology.orchestra.run.smithi100:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-07T16:34:03.535 INFO:tasks.workunit:Running workunit cls/test_cls_log.sh... 2024-04-07T16:34:03.536 DEBUG:teuthology.orchestra.run.smithi100:workunit test cls/test_cls_log.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=reef TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_log.sh 2024-04-07T16:34:03.595 INFO:tasks.workunit.client.0.smithi100.stdout:Running main() from gmock_main.cc 2024-04-07T16:34:03.596 INFO:tasks.workunit.client.0.smithi100.stdout:[==========] Running 4 tests from 1 test suite. 2024-04-07T16:34:03.596 INFO:tasks.workunit.client.0.smithi100.stdout:[----------] Global test environment set-up. 2024-04-07T16:34:03.596 INFO:tasks.workunit.client.0.smithi100.stdout:[----------] 4 tests from cls_log 2024-04-07T16:34:03.596 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] cls_log.test_log_add_same_time 2024-04-07T16:34:03.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:34:03 smithi144 bash[66654]: cluster 2024-04-07T16:34:02.956285+0000 mgr.y (mgr.25175) 561 : cluster [DBG] pgmap v1093: 137 pgs: 137 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 0 B/s rd, 0 B/s wr, 0 op/s 2024-04-07T16:34:03.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:34:03 smithi100 bash[74942]: cluster 2024-04-07T16:34:02.956285+0000 mgr.y (mgr.25175) 561 : cluster [DBG] pgmap v1093: 137 pgs: 137 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 0 B/s rd, 0 B/s wr, 0 op/s 2024-04-07T16:34:03.940 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:34:03 smithi100 bash[76202]: cluster 2024-04-07T16:34:02.956285+0000 mgr.y (mgr.25175) 561 : cluster [DBG] pgmap v1093: 137 pgs: 137 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 0 B/s rd, 0 B/s wr, 0 op/s 2024-04-07T16:34:04.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:34:04 smithi144 bash[66654]: cluster 2024-04-07T16:34:03.521474+0000 mon.a (mon.0) 1751 : cluster [DBG] osdmap e958: 8 total, 8 up, 8 in 2024-04-07T16:34:04.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:34:04 smithi100 bash[74942]: cluster 2024-04-07T16:34:03.521474+0000 mon.a (mon.0) 1751 : cluster [DBG] osdmap e958: 8 total, 8 up, 8 in 2024-04-07T16:34:04.940 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:34:04 smithi100 bash[76202]: cluster 2024-04-07T16:34:03.521474+0000 mon.a (mon.0) 1751 : cluster [DBG] osdmap e958: 8 total, 8 up, 8 in 2024-04-07T16:34:05.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:34:05 smithi144 bash[66654]: cluster 2024-04-07T16:34:04.524621+0000 mon.a (mon.0) 1752 : cluster [DBG] osdmap e959: 8 total, 8 up, 8 in 2024-04-07T16:34:05.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:34:05 smithi144 bash[66654]: audit 2024-04-07T16:34:04.527497+0000 mon.c (mon.2) 44 : audit [INF] from='client.? 172.21.15.100:0/2356968633' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111008-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:34:05.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:34:05 smithi144 bash[66654]: audit 2024-04-07T16:34:04.533673+0000 mon.a (mon.0) 1753 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111008-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:34:05.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:34:05 smithi144 bash[66654]: cluster 2024-04-07T16:34:04.957011+0000 mgr.y (mgr.25175) 562 : cluster [DBG] pgmap v1096: 137 pgs: 32 unknown, 105 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:34:05.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:34:05 smithi144 bash[66654]: audit 2024-04-07T16:34:05.516381+0000 mon.a (mon.0) 1754 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111008-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:34:05.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:34:05 smithi144 bash[66654]: cluster 2024-04-07T16:34:05.527789+0000 mon.a (mon.0) 1755 : cluster [DBG] osdmap e960: 8 total, 8 up, 8 in 2024-04-07T16:34:05.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:34:05 smithi100 bash[74942]: cluster 2024-04-07T16:34:04.524621+0000 mon.a (mon.0) 1752 : cluster [DBG] osdmap e959: 8 total, 8 up, 8 in 2024-04-07T16:34:05.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:34:05 smithi100 bash[74942]: audit 2024-04-07T16:34:04.527497+0000 mon.c (mon.2) 44 : audit [INF] from='client.? 172.21.15.100:0/2356968633' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111008-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:34:05.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:34:05 smithi100 bash[74942]: audit 2024-04-07T16:34:04.533673+0000 mon.a (mon.0) 1753 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111008-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:34:05.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:34:05 smithi100 bash[74942]: cluster 2024-04-07T16:34:04.957011+0000 mgr.y (mgr.25175) 562 : cluster [DBG] pgmap v1096: 137 pgs: 32 unknown, 105 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:34:05.941 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:34:05 smithi100 bash[74942]: audit 2024-04-07T16:34:05.516381+0000 mon.a (mon.0) 1754 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111008-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:34:05.941 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:34:05 smithi100 bash[74942]: cluster 2024-04-07T16:34:05.527789+0000 mon.a (mon.0) 1755 : cluster [DBG] osdmap e960: 8 total, 8 up, 8 in 2024-04-07T16:34:05.941 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:34:05 smithi100 bash[76202]: cluster 2024-04-07T16:34:04.524621+0000 mon.a (mon.0) 1752 : cluster [DBG] osdmap e959: 8 total, 8 up, 8 in 2024-04-07T16:34:05.941 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:34:05 smithi100 bash[76202]: audit 2024-04-07T16:34:04.527497+0000 mon.c (mon.2) 44 : audit [INF] from='client.? 172.21.15.100:0/2356968633' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111008-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:34:05.941 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:34:05 smithi100 bash[76202]: audit 2024-04-07T16:34:04.533673+0000 mon.a (mon.0) 1753 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111008-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:34:05.941 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:34:05 smithi100 bash[76202]: cluster 2024-04-07T16:34:04.957011+0000 mgr.y (mgr.25175) 562 : cluster [DBG] pgmap v1096: 137 pgs: 32 unknown, 105 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:34:05.941 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:34:05 smithi100 bash[76202]: audit 2024-04-07T16:34:05.516381+0000 mon.a (mon.0) 1754 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111008-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:34:05.941 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:34:05 smithi100 bash[76202]: cluster 2024-04-07T16:34:05.527789+0000 mon.a (mon.0) 1755 : cluster [DBG] osdmap e960: 8 total, 8 up, 8 in 2024-04-07T16:34:06.522 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] cls_log.test_log_add_same_time (2926 ms) 2024-04-07T16:34:06.523 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] cls_log.test_log_add_different_time 2024-04-07T16:34:06.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:34:06 smithi144 bash[66654]: cluster 2024-04-07T16:34:05.534998+0000 mon.a (mon.0) 1756 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T16:34:06.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:34:06 smithi144 bash[66654]: cluster 2024-04-07T16:34:06.529290+0000 mon.a (mon.0) 1757 : cluster [DBG] osdmap e961: 8 total, 8 up, 8 in 2024-04-07T16:34:06.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:34:06 smithi100 bash[74942]: cluster 2024-04-07T16:34:05.534998+0000 mon.a (mon.0) 1756 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T16:34:06.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:34:06 smithi100 bash[74942]: cluster 2024-04-07T16:34:06.529290+0000 mon.a (mon.0) 1757 : cluster [DBG] osdmap e961: 8 total, 8 up, 8 in 2024-04-07T16:34:06.940 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:34:06 smithi100 bash[76202]: cluster 2024-04-07T16:34:05.534998+0000 mon.a (mon.0) 1756 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T16:34:06.940 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:34:06 smithi100 bash[76202]: cluster 2024-04-07T16:34:06.529290+0000 mon.a (mon.0) 1757 : cluster [DBG] osdmap e961: 8 total, 8 up, 8 in 2024-04-07T16:34:07.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:34:07 smithi144 bash[66654]: cluster 2024-04-07T16:34:06.957691+0000 mgr.y (mgr.25175) 563 : cluster [DBG] pgmap v1099: 105 pgs: 105 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:34:07.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:34:07 smithi100 bash[74942]: cluster 2024-04-07T16:34:06.957691+0000 mgr.y (mgr.25175) 563 : cluster [DBG] pgmap v1099: 105 pgs: 105 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:34:07.940 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:34:07 smithi100 bash[76202]: cluster 2024-04-07T16:34:06.957691+0000 mgr.y (mgr.25175) 563 : cluster [DBG] pgmap v1099: 105 pgs: 105 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:34:08.690 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:34:08 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:34:08] "GET /metrics HTTP/1.1" 200 36366 "" "Prometheus/2.43.0" 2024-04-07T16:34:08.690 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:34:08 smithi100 bash[74942]: cluster 2024-04-07T16:34:07.552736+0000 mon.a (mon.0) 1758 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T16:34:08.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:34:08 smithi100 bash[74942]: cluster 2024-04-07T16:34:07.552799+0000 mon.a (mon.0) 1759 : cluster [INF] Cluster is now healthy 2024-04-07T16:34:08.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:34:08 smithi100 bash[74942]: cluster 2024-04-07T16:34:07.575513+0000 mon.a (mon.0) 1760 : cluster [DBG] osdmap e962: 8 total, 8 up, 8 in 2024-04-07T16:34:08.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:34:08 smithi100 bash[74942]: audit 2024-04-07T16:34:07.578678+0000 mon.a (mon.0) 1761 : audit [INF] from='client.? 172.21.15.100:0/3061525362' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111008-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:34:08.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:34:08 smithi100 bash[76202]: cluster 2024-04-07T16:34:07.552736+0000 mon.a (mon.0) 1758 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T16:34:08.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:34:08 smithi100 bash[76202]: cluster 2024-04-07T16:34:07.552799+0000 mon.a (mon.0) 1759 : cluster [INF] Cluster is now healthy 2024-04-07T16:34:08.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:34:08 smithi100 bash[76202]: cluster 2024-04-07T16:34:07.575513+0000 mon.a (mon.0) 1760 : cluster [DBG] osdmap e962: 8 total, 8 up, 8 in 2024-04-07T16:34:08.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:34:08 smithi100 bash[76202]: audit 2024-04-07T16:34:07.578678+0000 mon.a (mon.0) 1761 : audit [INF] from='client.? 172.21.15.100:0/3061525362' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111008-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:34:08.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:34:08 smithi144 bash[66654]: cluster 2024-04-07T16:34:07.552736+0000 mon.a (mon.0) 1758 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T16:34:08.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:34:08 smithi144 bash[66654]: cluster 2024-04-07T16:34:07.552799+0000 mon.a (mon.0) 1759 : cluster [INF] Cluster is now healthy 2024-04-07T16:34:08.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:34:08 smithi144 bash[66654]: cluster 2024-04-07T16:34:07.575513+0000 mon.a (mon.0) 1760 : cluster [DBG] osdmap e962: 8 total, 8 up, 8 in 2024-04-07T16:34:08.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:34:08 smithi144 bash[66654]: audit 2024-04-07T16:34:07.578678+0000 mon.a (mon.0) 1761 : audit [INF] from='client.? 172.21.15.100:0/3061525362' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111008-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:34:09.588 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] cls_log.test_log_add_different_time (3065 ms) 2024-04-07T16:34:09.588 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] cls_log.trim_by_time 2024-04-07T16:34:09.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:34:09 smithi144 bash[66654]: audit 2024-04-07T16:34:08.576326+0000 mon.a (mon.0) 1762 : audit [INF] from='client.? 172.21.15.100:0/3061525362' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111008-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:34:09.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:34:09 smithi144 bash[66654]: cluster 2024-04-07T16:34:08.583816+0000 mon.a (mon.0) 1763 : cluster [DBG] osdmap e963: 8 total, 8 up, 8 in 2024-04-07T16:34:09.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:34:09 smithi144 bash[66654]: cluster 2024-04-07T16:34:08.958419+0000 mgr.y (mgr.25175) 564 : cluster [DBG] pgmap v1102: 137 pgs: 8 creating+peering, 24 unknown, 105 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:34:09.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:34:09 smithi100 bash[74942]: audit 2024-04-07T16:34:08.576326+0000 mon.a (mon.0) 1762 : audit [INF] from='client.? 172.21.15.100:0/3061525362' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111008-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:34:09.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:34:09 smithi100 bash[74942]: cluster 2024-04-07T16:34:08.583816+0000 mon.a (mon.0) 1763 : cluster [DBG] osdmap e963: 8 total, 8 up, 8 in 2024-04-07T16:34:09.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:34:09 smithi100 bash[74942]: cluster 2024-04-07T16:34:08.958419+0000 mgr.y (mgr.25175) 564 : cluster [DBG] pgmap v1102: 137 pgs: 8 creating+peering, 24 unknown, 105 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:34:09.941 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:34:09 smithi100 bash[76202]: audit 2024-04-07T16:34:08.576326+0000 mon.a (mon.0) 1762 : audit [INF] from='client.? 172.21.15.100:0/3061525362' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111008-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:34:09.941 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:34:09 smithi100 bash[76202]: cluster 2024-04-07T16:34:08.583816+0000 mon.a (mon.0) 1763 : cluster [DBG] osdmap e963: 8 total, 8 up, 8 in 2024-04-07T16:34:09.941 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:34:09 smithi100 bash[76202]: cluster 2024-04-07T16:34:08.958419+0000 mgr.y (mgr.25175) 564 : cluster [DBG] pgmap v1102: 137 pgs: 8 creating+peering, 24 unknown, 105 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:34:10.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:34:10 smithi144 bash[66654]: cluster 2024-04-07T16:34:09.596724+0000 mon.a (mon.0) 1764 : cluster [DBG] osdmap e964: 8 total, 8 up, 8 in 2024-04-07T16:34:10.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:34:10 smithi100 bash[74942]: cluster 2024-04-07T16:34:09.596724+0000 mon.a (mon.0) 1764 : cluster [DBG] osdmap e964: 8 total, 8 up, 8 in 2024-04-07T16:34:10.940 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:34:10 smithi100 bash[76202]: cluster 2024-04-07T16:34:09.596724+0000 mon.a (mon.0) 1764 : cluster [DBG] osdmap e964: 8 total, 8 up, 8 in 2024-04-07T16:34:11.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:34:11 smithi100 bash[74942]: cluster 2024-04-07T16:34:10.609008+0000 mon.a (mon.0) 1765 : cluster [DBG] osdmap e965: 8 total, 8 up, 8 in 2024-04-07T16:34:11.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:34:11 smithi100 bash[74942]: audit 2024-04-07T16:34:10.611969+0000 mon.c (mon.2) 45 : audit [INF] from='client.? 172.21.15.100:0/1873845134' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111008-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:34:11.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:34:11 smithi100 bash[74942]: audit 2024-04-07T16:34:10.613397+0000 mon.a (mon.0) 1766 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111008-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:34:11.941 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:34:11 smithi100 bash[74942]: cluster 2024-04-07T16:34:10.959306+0000 mgr.y (mgr.25175) 565 : cluster [DBG] pgmap v1105: 137 pgs: 32 unknown, 105 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:34:11.941 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:34:11 smithi100 bash[76202]: cluster 2024-04-07T16:34:10.609008+0000 mon.a (mon.0) 1765 : cluster [DBG] osdmap e965: 8 total, 8 up, 8 in 2024-04-07T16:34:11.941 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:34:11 smithi100 bash[76202]: audit 2024-04-07T16:34:10.611969+0000 mon.c (mon.2) 45 : audit [INF] from='client.? 172.21.15.100:0/1873845134' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111008-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:34:11.941 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:34:11 smithi100 bash[76202]: audit 2024-04-07T16:34:10.613397+0000 mon.a (mon.0) 1766 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111008-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:34:11.941 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:34:11 smithi100 bash[76202]: cluster 2024-04-07T16:34:10.959306+0000 mgr.y (mgr.25175) 565 : cluster [DBG] pgmap v1105: 137 pgs: 32 unknown, 105 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:34:12.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:34:11 smithi144 bash[66654]: cluster 2024-04-07T16:34:10.609008+0000 mon.a (mon.0) 1765 : cluster [DBG] osdmap e965: 8 total, 8 up, 8 in 2024-04-07T16:34:12.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:34:11 smithi144 bash[66654]: audit 2024-04-07T16:34:10.611969+0000 mon.c (mon.2) 45 : audit [INF] from='client.? 172.21.15.100:0/1873845134' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111008-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:34:12.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:34:11 smithi144 bash[66654]: audit 2024-04-07T16:34:10.613397+0000 mon.a (mon.0) 1766 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111008-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:34:12.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:34:11 smithi144 bash[66654]: cluster 2024-04-07T16:34:10.959306+0000 mgr.y (mgr.25175) 565 : cluster [DBG] pgmap v1105: 137 pgs: 32 unknown, 105 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:34:12.607 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:34:12 smithi144 bash[95233]: ts=2024-04-07T16:34:12.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:34:12.611 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] cls_log.trim_by_time (3023 ms) 2024-04-07T16:34:12.611 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] cls_log.trim_by_marker 2024-04-07T16:34:12.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:34:12 smithi100 bash[74942]: cluster 2024-04-07T16:34:11.603000+0000 mon.a (mon.0) 1767 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T16:34:12.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:34:12 smithi100 bash[74942]: audit 2024-04-07T16:34:11.604717+0000 mon.a (mon.0) 1768 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111008-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:34:12.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:34:12 smithi100 bash[74942]: cluster 2024-04-07T16:34:11.614030+0000 mon.a (mon.0) 1769 : cluster [DBG] osdmap e966: 8 total, 8 up, 8 in 2024-04-07T16:34:12.941 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:34:12 smithi100 bash[74942]: cluster 2024-04-07T16:34:12.616428+0000 mon.a (mon.0) 1770 : cluster [DBG] osdmap e967: 8 total, 8 up, 8 in 2024-04-07T16:34:12.941 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:34:12 smithi100 bash[76202]: cluster 2024-04-07T16:34:11.603000+0000 mon.a (mon.0) 1767 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T16:34:12.941 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:34:12 smithi100 bash[76202]: audit 2024-04-07T16:34:11.604717+0000 mon.a (mon.0) 1768 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111008-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:34:12.941 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:34:12 smithi100 bash[76202]: cluster 2024-04-07T16:34:11.614030+0000 mon.a (mon.0) 1769 : cluster [DBG] osdmap e966: 8 total, 8 up, 8 in 2024-04-07T16:34:12.941 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:34:12 smithi100 bash[76202]: cluster 2024-04-07T16:34:12.616428+0000 mon.a (mon.0) 1770 : cluster [DBG] osdmap e967: 8 total, 8 up, 8 in 2024-04-07T16:34:13.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:34:12 smithi144 bash[66654]: cluster 2024-04-07T16:34:11.603000+0000 mon.a (mon.0) 1767 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T16:34:13.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:34:12 smithi144 bash[66654]: audit 2024-04-07T16:34:11.604717+0000 mon.a (mon.0) 1768 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111008-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:34:13.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:34:12 smithi144 bash[66654]: cluster 2024-04-07T16:34:11.614030+0000 mon.a (mon.0) 1769 : cluster [DBG] osdmap e966: 8 total, 8 up, 8 in 2024-04-07T16:34:13.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:34:12 smithi144 bash[66654]: cluster 2024-04-07T16:34:12.616428+0000 mon.a (mon.0) 1770 : cluster [DBG] osdmap e967: 8 total, 8 up, 8 in 2024-04-07T16:34:13.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:34:13 smithi100 bash[74942]: cluster 2024-04-07T16:34:12.959989+0000 mgr.y (mgr.25175) 566 : cluster [DBG] pgmap v1108: 105 pgs: 105 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:34:13.940 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:34:13 smithi100 bash[76202]: cluster 2024-04-07T16:34:12.959989+0000 mgr.y (mgr.25175) 566 : cluster [DBG] pgmap v1108: 105 pgs: 105 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:34:14.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:34:13 smithi144 bash[66654]: cluster 2024-04-07T16:34:12.959989+0000 mgr.y (mgr.25175) 566 : cluster [DBG] pgmap v1108: 105 pgs: 105 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:34:14.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:34:14 smithi100 bash[74942]: cluster 2024-04-07T16:34:13.637622+0000 mon.a (mon.0) 1771 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T16:34:14.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:34:14 smithi100 bash[74942]: cluster 2024-04-07T16:34:13.637725+0000 mon.a (mon.0) 1772 : cluster [INF] Cluster is now healthy 2024-04-07T16:34:14.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:34:14 smithi100 bash[74942]: cluster 2024-04-07T16:34:13.658565+0000 mon.a (mon.0) 1773 : cluster [DBG] osdmap e968: 8 total, 8 up, 8 in 2024-04-07T16:34:14.941 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:34:14 smithi100 bash[74942]: audit 2024-04-07T16:34:13.666777+0000 mon.c (mon.2) 46 : audit [INF] from='client.? 172.21.15.100:0/3022740124' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111008-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:34:14.941 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:34:14 smithi100 bash[74942]: audit 2024-04-07T16:34:13.668804+0000 mon.a (mon.0) 1774 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111008-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:34:14.941 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:34:14 smithi100 bash[76202]: cluster 2024-04-07T16:34:13.637622+0000 mon.a (mon.0) 1771 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T16:34:14.941 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:34:14 smithi100 bash[76202]: cluster 2024-04-07T16:34:13.637725+0000 mon.a (mon.0) 1772 : cluster [INF] Cluster is now healthy 2024-04-07T16:34:14.941 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:34:14 smithi100 bash[76202]: cluster 2024-04-07T16:34:13.658565+0000 mon.a (mon.0) 1773 : cluster [DBG] osdmap e968: 8 total, 8 up, 8 in 2024-04-07T16:34:14.941 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:34:14 smithi100 bash[76202]: audit 2024-04-07T16:34:13.666777+0000 mon.c (mon.2) 46 : audit [INF] from='client.? 172.21.15.100:0/3022740124' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111008-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:34:14.941 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:34:14 smithi100 bash[76202]: audit 2024-04-07T16:34:13.668804+0000 mon.a (mon.0) 1774 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111008-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:34:15.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:34:14 smithi144 bash[66654]: cluster 2024-04-07T16:34:13.637622+0000 mon.a (mon.0) 1771 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T16:34:15.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:34:14 smithi144 bash[66654]: cluster 2024-04-07T16:34:13.637725+0000 mon.a (mon.0) 1772 : cluster [INF] Cluster is now healthy 2024-04-07T16:34:15.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:34:14 smithi144 bash[66654]: cluster 2024-04-07T16:34:13.658565+0000 mon.a (mon.0) 1773 : cluster [DBG] osdmap e968: 8 total, 8 up, 8 in 2024-04-07T16:34:15.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:34:14 smithi144 bash[66654]: audit 2024-04-07T16:34:13.666777+0000 mon.c (mon.2) 46 : audit [INF] from='client.? 172.21.15.100:0/3022740124' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111008-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:34:15.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:34:14 smithi144 bash[66654]: audit 2024-04-07T16:34:13.668804+0000 mon.a (mon.0) 1774 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111008-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:34:15.685 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] cls_log.trim_by_marker (3073 ms) 2024-04-07T16:34:15.685 INFO:tasks.workunit.client.0.smithi100.stdout:[----------] 4 tests from cls_log (12088 ms total) 2024-04-07T16:34:15.685 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T16:34:15.685 INFO:tasks.workunit.client.0.smithi100.stdout:[----------] Global test environment tear-down 2024-04-07T16:34:15.685 INFO:tasks.workunit.client.0.smithi100.stdout:[==========] 4 tests from 1 test suite ran. (12088 ms total) 2024-04-07T16:34:15.685 INFO:tasks.workunit.client.0.smithi100.stdout:[ PASSED ] 4 tests. 2024-04-07T16:34:15.689 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-07T16:34:15.689 DEBUG:teuthology.orchestra.run.smithi100:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-07T16:34:15.742 INFO:tasks.workunit:Running workunit cls/test_cls_numops.sh... 2024-04-07T16:34:15.742 DEBUG:teuthology.orchestra.run.smithi100:workunit test cls/test_cls_numops.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=reef TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_numops.sh 2024-04-07T16:34:15.809 INFO:tasks.workunit.client.0.smithi100.stdout:Running main() from gmock_main.cc 2024-04-07T16:34:15.810 INFO:tasks.workunit.client.0.smithi100.stdout:[==========] Running 4 tests from 1 test suite. 2024-04-07T16:34:15.810 INFO:tasks.workunit.client.0.smithi100.stdout:[----------] Global test environment set-up. 2024-04-07T16:34:15.810 INFO:tasks.workunit.client.0.smithi100.stdout:[----------] 4 tests from ClsNumOps 2024-04-07T16:34:15.810 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] ClsNumOps.Add 2024-04-07T16:34:15.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:34:15 smithi100 bash[74942]: audit 2024-04-07T16:34:14.654610+0000 mon.a (mon.0) 1775 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111008-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:34:15.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:34:15 smithi100 bash[74942]: cluster 2024-04-07T16:34:14.664575+0000 mon.a (mon.0) 1776 : cluster [DBG] osdmap e969: 8 total, 8 up, 8 in 2024-04-07T16:34:15.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:34:15 smithi100 bash[74942]: cluster 2024-04-07T16:34:14.960739+0000 mgr.y (mgr.25175) 567 : cluster [DBG] pgmap v1111: 137 pgs: 8 creating+peering, 24 unknown, 105 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:34:15.941 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:34:15 smithi100 bash[76202]: audit 2024-04-07T16:34:14.654610+0000 mon.a (mon.0) 1775 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111008-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:34:15.941 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:34:15 smithi100 bash[76202]: cluster 2024-04-07T16:34:14.664575+0000 mon.a (mon.0) 1776 : cluster [DBG] osdmap e969: 8 total, 8 up, 8 in 2024-04-07T16:34:15.941 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:34:15 smithi100 bash[76202]: cluster 2024-04-07T16:34:14.960739+0000 mgr.y (mgr.25175) 567 : cluster [DBG] pgmap v1111: 137 pgs: 8 creating+peering, 24 unknown, 105 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:34:16.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:34:15 smithi144 bash[66654]: audit 2024-04-07T16:34:14.654610+0000 mon.a (mon.0) 1775 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111008-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:34:16.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:34:15 smithi144 bash[66654]: cluster 2024-04-07T16:34:14.664575+0000 mon.a (mon.0) 1776 : cluster [DBG] osdmap e969: 8 total, 8 up, 8 in 2024-04-07T16:34:16.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:34:15 smithi144 bash[66654]: cluster 2024-04-07T16:34:14.960739+0000 mgr.y (mgr.25175) 567 : cluster [DBG] pgmap v1111: 137 pgs: 8 creating+peering, 24 unknown, 105 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:34:17.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:34:16 smithi144 bash[66654]: cluster 2024-04-07T16:34:15.692533+0000 mon.a (mon.0) 1777 : cluster [DBG] osdmap e970: 8 total, 8 up, 8 in 2024-04-07T16:34:17.190 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:34:16 smithi100 bash[74942]: cluster 2024-04-07T16:34:15.692533+0000 mon.a (mon.0) 1777 : cluster [DBG] osdmap e970: 8 total, 8 up, 8 in 2024-04-07T16:34:17.190 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:34:16 smithi100 bash[76202]: cluster 2024-04-07T16:34:15.692533+0000 mon.a (mon.0) 1777 : cluster [DBG] osdmap e970: 8 total, 8 up, 8 in 2024-04-07T16:34:18.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:34:17 smithi144 bash[66654]: cluster 2024-04-07T16:34:16.699800+0000 mon.a (mon.0) 1778 : cluster [DBG] osdmap e971: 8 total, 8 up, 8 in 2024-04-07T16:34:18.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:34:17 smithi144 bash[66654]: audit 2024-04-07T16:34:16.705796+0000 mon.a (mon.0) 1779 : audit [INF] from='client.? 172.21.15.100:0/2275974463' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111084-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:34:18.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:34:17 smithi144 bash[66654]: audit 2024-04-07T16:34:16.745690+0000 mon.a (mon.0) 1780 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:34:18.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:34:17 smithi144 bash[66654]: cluster 2024-04-07T16:34:16.961435+0000 mgr.y (mgr.25175) 568 : cluster [DBG] pgmap v1114: 137 pgs: 32 unknown, 105 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:34:18.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:34:17 smithi144 bash[66654]: audit 2024-04-07T16:34:17.689268+0000 mon.a (mon.0) 1781 : audit [INF] from='client.? 172.21.15.100:0/2275974463' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111084-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:34:18.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:34:17 smithi144 bash[66654]: cluster 2024-04-07T16:34:17.699011+0000 mon.a (mon.0) 1782 : cluster [DBG] osdmap e972: 8 total, 8 up, 8 in 2024-04-07T16:34:18.190 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:34:17 smithi100 bash[74942]: cluster 2024-04-07T16:34:16.699800+0000 mon.a (mon.0) 1778 : cluster [DBG] osdmap e971: 8 total, 8 up, 8 in 2024-04-07T16:34:18.191 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:34:17 smithi100 bash[74942]: audit 2024-04-07T16:34:16.705796+0000 mon.a (mon.0) 1779 : audit [INF] from='client.? 172.21.15.100:0/2275974463' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111084-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:34:18.191 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:34:17 smithi100 bash[74942]: audit 2024-04-07T16:34:16.745690+0000 mon.a (mon.0) 1780 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:34:18.191 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:34:17 smithi100 bash[74942]: cluster 2024-04-07T16:34:16.961435+0000 mgr.y (mgr.25175) 568 : cluster [DBG] pgmap v1114: 137 pgs: 32 unknown, 105 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:34:18.191 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:34:17 smithi100 bash[74942]: audit 2024-04-07T16:34:17.689268+0000 mon.a (mon.0) 1781 : audit [INF] from='client.? 172.21.15.100:0/2275974463' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111084-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:34:18.191 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:34:17 smithi100 bash[74942]: cluster 2024-04-07T16:34:17.699011+0000 mon.a (mon.0) 1782 : cluster [DBG] osdmap e972: 8 total, 8 up, 8 in 2024-04-07T16:34:18.191 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:34:17 smithi100 bash[76202]: cluster 2024-04-07T16:34:16.699800+0000 mon.a (mon.0) 1778 : cluster [DBG] osdmap e971: 8 total, 8 up, 8 in 2024-04-07T16:34:18.191 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:34:17 smithi100 bash[76202]: audit 2024-04-07T16:34:16.705796+0000 mon.a (mon.0) 1779 : audit [INF] from='client.? 172.21.15.100:0/2275974463' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111084-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:34:18.191 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:34:17 smithi100 bash[76202]: audit 2024-04-07T16:34:16.745690+0000 mon.a (mon.0) 1780 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:34:18.191 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:34:17 smithi100 bash[76202]: cluster 2024-04-07T16:34:16.961435+0000 mgr.y (mgr.25175) 568 : cluster [DBG] pgmap v1114: 137 pgs: 32 unknown, 105 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:34:18.191 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:34:17 smithi100 bash[76202]: audit 2024-04-07T16:34:17.689268+0000 mon.a (mon.0) 1781 : audit [INF] from='client.? 172.21.15.100:0/2275974463' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111084-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:34:18.191 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:34:17 smithi100 bash[76202]: cluster 2024-04-07T16:34:17.699011+0000 mon.a (mon.0) 1782 : cluster [DBG] osdmap e972: 8 total, 8 up, 8 in 2024-04-07T16:34:18.689 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] ClsNumOps.Add (2880 ms) 2024-04-07T16:34:18.689 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] ClsNumOps.Sub 2024-04-07T16:34:18.690 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:34:18 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:34:18] "GET /metrics HTTP/1.1" 200 36358 "" "Prometheus/2.43.0" 2024-04-07T16:34:19.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:34:18 smithi144 bash[66654]: cluster 2024-04-07T16:34:17.711356+0000 mon.a (mon.0) 1783 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T16:34:19.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:34:18 smithi144 bash[66654]: cluster 2024-04-07T16:34:18.697930+0000 mon.a (mon.0) 1784 : cluster [DBG] osdmap e973: 8 total, 8 up, 8 in 2024-04-07T16:34:19.190 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:34:18 smithi100 bash[74942]: cluster 2024-04-07T16:34:17.711356+0000 mon.a (mon.0) 1783 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T16:34:19.191 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:34:18 smithi100 bash[74942]: cluster 2024-04-07T16:34:18.697930+0000 mon.a (mon.0) 1784 : cluster [DBG] osdmap e973: 8 total, 8 up, 8 in 2024-04-07T16:34:19.191 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:34:18 smithi100 bash[76202]: cluster 2024-04-07T16:34:17.711356+0000 mon.a (mon.0) 1783 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T16:34:19.191 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:34:18 smithi100 bash[76202]: cluster 2024-04-07T16:34:18.697930+0000 mon.a (mon.0) 1784 : cluster [DBG] osdmap e973: 8 total, 8 up, 8 in 2024-04-07T16:34:20.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:34:19 smithi144 bash[66654]: cluster 2024-04-07T16:34:18.962085+0000 mgr.y (mgr.25175) 569 : cluster [DBG] pgmap v1117: 105 pgs: 105 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:34:20.190 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:34:19 smithi100 bash[74942]: cluster 2024-04-07T16:34:18.962085+0000 mgr.y (mgr.25175) 569 : cluster [DBG] pgmap v1117: 105 pgs: 105 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:34:20.190 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:34:19 smithi100 bash[76202]: cluster 2024-04-07T16:34:18.962085+0000 mgr.y (mgr.25175) 569 : cluster [DBG] pgmap v1117: 105 pgs: 105 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:34:21.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:34:20 smithi144 bash[66654]: cluster 2024-04-07T16:34:19.727265+0000 mon.a (mon.0) 1785 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T16:34:21.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:34:20 smithi144 bash[66654]: cluster 2024-04-07T16:34:19.727386+0000 mon.a (mon.0) 1786 : cluster [INF] Cluster is now healthy 2024-04-07T16:34:21.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:34:20 smithi144 bash[66654]: cluster 2024-04-07T16:34:19.748669+0000 mon.a (mon.0) 1787 : cluster [DBG] osdmap e974: 8 total, 8 up, 8 in 2024-04-07T16:34:21.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:34:20 smithi144 bash[66654]: audit 2024-04-07T16:34:19.756314+0000 mon.a (mon.0) 1788 : audit [INF] from='client.? 172.21.15.100:0/713824668' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111084-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:34:21.190 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:34:20 smithi100 bash[74942]: cluster 2024-04-07T16:34:19.727265+0000 mon.a (mon.0) 1785 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T16:34:21.190 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:34:20 smithi100 bash[74942]: cluster 2024-04-07T16:34:19.727386+0000 mon.a (mon.0) 1786 : cluster [INF] Cluster is now healthy 2024-04-07T16:34:21.191 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:34:20 smithi100 bash[74942]: cluster 2024-04-07T16:34:19.748669+0000 mon.a (mon.0) 1787 : cluster [DBG] osdmap e974: 8 total, 8 up, 8 in 2024-04-07T16:34:21.191 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:34:20 smithi100 bash[74942]: audit 2024-04-07T16:34:19.756314+0000 mon.a (mon.0) 1788 : audit [INF] from='client.? 172.21.15.100:0/713824668' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111084-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:34:21.191 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:34:20 smithi100 bash[76202]: cluster 2024-04-07T16:34:19.727265+0000 mon.a (mon.0) 1785 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T16:34:21.191 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:34:20 smithi100 bash[76202]: cluster 2024-04-07T16:34:19.727386+0000 mon.a (mon.0) 1786 : cluster [INF] Cluster is now healthy 2024-04-07T16:34:21.191 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:34:20 smithi100 bash[76202]: cluster 2024-04-07T16:34:19.748669+0000 mon.a (mon.0) 1787 : cluster [DBG] osdmap e974: 8 total, 8 up, 8 in 2024-04-07T16:34:21.191 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:34:20 smithi100 bash[76202]: audit 2024-04-07T16:34:19.756314+0000 mon.a (mon.0) 1788 : audit [INF] from='client.? 172.21.15.100:0/713824668' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111084-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:34:21.762 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] ClsNumOps.Sub (3067 ms) 2024-04-07T16:34:21.762 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] ClsNumOps.Mul 2024-04-07T16:34:22.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:34:21 smithi144 bash[66654]: audit 2024-04-07T16:34:20.742396+0000 mon.a (mon.0) 1789 : audit [INF] from='client.? 172.21.15.100:0/713824668' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111084-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:34:22.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:34:21 smithi144 bash[66654]: cluster 2024-04-07T16:34:20.750465+0000 mon.a (mon.0) 1790 : cluster [DBG] osdmap e975: 8 total, 8 up, 8 in 2024-04-07T16:34:22.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:34:21 smithi144 bash[66654]: cluster 2024-04-07T16:34:20.962780+0000 mgr.y (mgr.25175) 570 : cluster [DBG] pgmap v1120: 137 pgs: 2 creating+peering, 30 unknown, 105 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:34:22.190 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:34:21 smithi100 bash[74942]: audit 2024-04-07T16:34:20.742396+0000 mon.a (mon.0) 1789 : audit [INF] from='client.? 172.21.15.100:0/713824668' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111084-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:34:22.190 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:34:21 smithi100 bash[74942]: cluster 2024-04-07T16:34:20.750465+0000 mon.a (mon.0) 1790 : cluster [DBG] osdmap e975: 8 total, 8 up, 8 in 2024-04-07T16:34:22.191 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:34:21 smithi100 bash[74942]: cluster 2024-04-07T16:34:20.962780+0000 mgr.y (mgr.25175) 570 : cluster [DBG] pgmap v1120: 137 pgs: 2 creating+peering, 30 unknown, 105 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:34:22.191 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:34:21 smithi100 bash[76202]: audit 2024-04-07T16:34:20.742396+0000 mon.a (mon.0) 1789 : audit [INF] from='client.? 172.21.15.100:0/713824668' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111084-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:34:22.191 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:34:21 smithi100 bash[76202]: cluster 2024-04-07T16:34:20.750465+0000 mon.a (mon.0) 1790 : cluster [DBG] osdmap e975: 8 total, 8 up, 8 in 2024-04-07T16:34:22.191 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:34:21 smithi100 bash[76202]: cluster 2024-04-07T16:34:20.962780+0000 mgr.y (mgr.25175) 570 : cluster [DBG] pgmap v1120: 137 pgs: 2 creating+peering, 30 unknown, 105 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:34:22.607 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:34:22 smithi144 bash[95233]: ts=2024-04-07T16:34:22.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:34:23.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:34:22 smithi144 bash[66654]: cluster 2024-04-07T16:34:21.761465+0000 mon.a (mon.0) 1791 : cluster [DBG] osdmap e976: 8 total, 8 up, 8 in 2024-04-07T16:34:23.190 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:34:22 smithi100 bash[74942]: cluster 2024-04-07T16:34:21.761465+0000 mon.a (mon.0) 1791 : cluster [DBG] osdmap e976: 8 total, 8 up, 8 in 2024-04-07T16:34:23.191 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:34:22 smithi100 bash[76202]: cluster 2024-04-07T16:34:21.761465+0000 mon.a (mon.0) 1791 : cluster [DBG] osdmap e976: 8 total, 8 up, 8 in 2024-04-07T16:34:24.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:34:23 smithi144 bash[66654]: cluster 2024-04-07T16:34:22.776073+0000 mon.a (mon.0) 1792 : cluster [DBG] osdmap e977: 8 total, 8 up, 8 in 2024-04-07T16:34:24.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:34:23 smithi144 bash[66654]: audit 2024-04-07T16:34:22.781994+0000 mon.c (mon.2) 47 : audit [INF] from='client.? 172.21.15.100:0/2784443128' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111084-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:34:24.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:34:23 smithi144 bash[66654]: audit 2024-04-07T16:34:22.782891+0000 mon.a (mon.0) 1793 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111084-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:34:24.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:34:23 smithi144 bash[66654]: cluster 2024-04-07T16:34:22.963553+0000 mgr.y (mgr.25175) 571 : cluster [DBG] pgmap v1123: 137 pgs: 32 unknown, 105 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:34:24.190 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:34:23 smithi100 bash[74942]: cluster 2024-04-07T16:34:22.776073+0000 mon.a (mon.0) 1792 : cluster [DBG] osdmap e977: 8 total, 8 up, 8 in 2024-04-07T16:34:24.190 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:34:23 smithi100 bash[74942]: audit 2024-04-07T16:34:22.781994+0000 mon.c (mon.2) 47 : audit [INF] from='client.? 172.21.15.100:0/2784443128' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111084-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:34:24.191 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:34:23 smithi100 bash[74942]: audit 2024-04-07T16:34:22.782891+0000 mon.a (mon.0) 1793 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111084-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:34:24.191 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:34:23 smithi100 bash[74942]: cluster 2024-04-07T16:34:22.963553+0000 mgr.y (mgr.25175) 571 : cluster [DBG] pgmap v1123: 137 pgs: 32 unknown, 105 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:34:24.191 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:34:23 smithi100 bash[76202]: cluster 2024-04-07T16:34:22.776073+0000 mon.a (mon.0) 1792 : cluster [DBG] osdmap e977: 8 total, 8 up, 8 in 2024-04-07T16:34:24.191 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:34:23 smithi100 bash[76202]: audit 2024-04-07T16:34:22.781994+0000 mon.c (mon.2) 47 : audit [INF] from='client.? 172.21.15.100:0/2784443128' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111084-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:34:24.191 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:34:23 smithi100 bash[76202]: audit 2024-04-07T16:34:22.782891+0000 mon.a (mon.0) 1793 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111084-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:34:24.191 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:34:23 smithi100 bash[76202]: cluster 2024-04-07T16:34:22.963553+0000 mgr.y (mgr.25175) 571 : cluster [DBG] pgmap v1123: 137 pgs: 32 unknown, 105 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:34:24.804 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] ClsNumOps.Mul (3048 ms) 2024-04-07T16:34:24.804 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] ClsNumOps.Div 2024-04-07T16:34:25.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:34:24 smithi144 bash[66654]: cluster 2024-04-07T16:34:23.768983+0000 mon.a (mon.0) 1794 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T16:34:25.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:34:24 smithi144 bash[66654]: audit 2024-04-07T16:34:23.770858+0000 mon.a (mon.0) 1795 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111084-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:34:25.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:34:24 smithi144 bash[66654]: cluster 2024-04-07T16:34:23.780639+0000 mon.a (mon.0) 1796 : cluster [DBG] osdmap e978: 8 total, 8 up, 8 in 2024-04-07T16:34:25.190 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:34:24 smithi100 bash[74942]: cluster 2024-04-07T16:34:23.768983+0000 mon.a (mon.0) 1794 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T16:34:25.190 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:34:24 smithi100 bash[74942]: audit 2024-04-07T16:34:23.770858+0000 mon.a (mon.0) 1795 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111084-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:34:25.191 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:34:24 smithi100 bash[74942]: cluster 2024-04-07T16:34:23.780639+0000 mon.a (mon.0) 1796 : cluster [DBG] osdmap e978: 8 total, 8 up, 8 in 2024-04-07T16:34:25.191 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:34:24 smithi100 bash[76202]: cluster 2024-04-07T16:34:23.768983+0000 mon.a (mon.0) 1794 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T16:34:25.191 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:34:24 smithi100 bash[76202]: audit 2024-04-07T16:34:23.770858+0000 mon.a (mon.0) 1795 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111084-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:34:25.191 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:34:24 smithi100 bash[76202]: cluster 2024-04-07T16:34:23.780639+0000 mon.a (mon.0) 1796 : cluster [DBG] osdmap e978: 8 total, 8 up, 8 in 2024-04-07T16:34:26.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:34:25 smithi144 bash[66654]: cluster 2024-04-07T16:34:24.812666+0000 mon.a (mon.0) 1797 : cluster [DBG] osdmap e979: 8 total, 8 up, 8 in 2024-04-07T16:34:26.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:34:25 smithi144 bash[66654]: cluster 2024-04-07T16:34:24.964161+0000 mgr.y (mgr.25175) 572 : cluster [DBG] pgmap v1126: 105 pgs: 105 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:34:26.190 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:34:25 smithi100 bash[74942]: cluster 2024-04-07T16:34:24.812666+0000 mon.a (mon.0) 1797 : cluster [DBG] osdmap e979: 8 total, 8 up, 8 in 2024-04-07T16:34:26.191 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:34:25 smithi100 bash[74942]: cluster 2024-04-07T16:34:24.964161+0000 mgr.y (mgr.25175) 572 : cluster [DBG] pgmap v1126: 105 pgs: 105 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:34:26.191 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:34:25 smithi100 bash[76202]: cluster 2024-04-07T16:34:24.812666+0000 mon.a (mon.0) 1797 : cluster [DBG] osdmap e979: 8 total, 8 up, 8 in 2024-04-07T16:34:26.191 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:34:25 smithi100 bash[76202]: cluster 2024-04-07T16:34:24.964161+0000 mgr.y (mgr.25175) 572 : cluster [DBG] pgmap v1126: 105 pgs: 105 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:34:27.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:34:26 smithi144 bash[66654]: cluster 2024-04-07T16:34:25.816487+0000 mon.a (mon.0) 1798 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T16:34:27.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:34:26 smithi144 bash[66654]: cluster 2024-04-07T16:34:25.816563+0000 mon.a (mon.0) 1799 : cluster [INF] Cluster is now healthy 2024-04-07T16:34:27.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:34:26 smithi144 bash[66654]: cluster 2024-04-07T16:34:25.843821+0000 mon.a (mon.0) 1800 : cluster [DBG] osdmap e980: 8 total, 8 up, 8 in 2024-04-07T16:34:27.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:34:26 smithi144 bash[66654]: audit 2024-04-07T16:34:25.846244+0000 mon.c (mon.2) 48 : audit [INF] from='client.? 172.21.15.100:0/2471771765' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111084-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:34:27.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:34:26 smithi144 bash[66654]: audit 2024-04-07T16:34:25.853414+0000 mon.a (mon.0) 1801 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111084-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:34:27.190 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:34:26 smithi100 bash[76202]: cluster 2024-04-07T16:34:25.816487+0000 mon.a (mon.0) 1798 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T16:34:27.190 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:34:26 smithi100 bash[76202]: cluster 2024-04-07T16:34:25.816563+0000 mon.a (mon.0) 1799 : cluster [INF] Cluster is now healthy 2024-04-07T16:34:27.191 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:34:26 smithi100 bash[76202]: cluster 2024-04-07T16:34:25.843821+0000 mon.a (mon.0) 1800 : cluster [DBG] osdmap e980: 8 total, 8 up, 8 in 2024-04-07T16:34:27.191 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:34:26 smithi100 bash[76202]: audit 2024-04-07T16:34:25.846244+0000 mon.c (mon.2) 48 : audit [INF] from='client.? 172.21.15.100:0/2471771765' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111084-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:34:27.191 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:34:26 smithi100 bash[76202]: audit 2024-04-07T16:34:25.853414+0000 mon.a (mon.0) 1801 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111084-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:34:27.191 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:34:26 smithi100 bash[74942]: cluster 2024-04-07T16:34:25.816487+0000 mon.a (mon.0) 1798 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T16:34:27.191 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:34:26 smithi100 bash[74942]: cluster 2024-04-07T16:34:25.816563+0000 mon.a (mon.0) 1799 : cluster [INF] Cluster is now healthy 2024-04-07T16:34:27.191 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:34:26 smithi100 bash[74942]: cluster 2024-04-07T16:34:25.843821+0000 mon.a (mon.0) 1800 : cluster [DBG] osdmap e980: 8 total, 8 up, 8 in 2024-04-07T16:34:27.191 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:34:26 smithi100 bash[74942]: audit 2024-04-07T16:34:25.846244+0000 mon.c (mon.2) 48 : audit [INF] from='client.? 172.21.15.100:0/2471771765' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111084-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:34:27.191 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:34:26 smithi100 bash[74942]: audit 2024-04-07T16:34:25.853414+0000 mon.a (mon.0) 1801 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111084-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:34:27.858 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] ClsNumOps.Div (3053 ms) 2024-04-07T16:34:27.858 INFO:tasks.workunit.client.0.smithi100.stdout:[----------] 4 tests from ClsNumOps (12048 ms total) 2024-04-07T16:34:27.858 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T16:34:27.858 INFO:tasks.workunit.client.0.smithi100.stdout:[----------] Global test environment tear-down 2024-04-07T16:34:27.858 INFO:tasks.workunit.client.0.smithi100.stdout:[==========] 4 tests from 1 test suite ran. (12048 ms total) 2024-04-07T16:34:27.858 INFO:tasks.workunit.client.0.smithi100.stdout:[ PASSED ] 4 tests. 2024-04-07T16:34:27.862 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-07T16:34:27.863 DEBUG:teuthology.orchestra.run.smithi100:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-07T16:34:27.920 INFO:tasks.workunit:Running workunit cls/test_cls_rbd.sh... 2024-04-07T16:34:27.920 DEBUG:teuthology.orchestra.run.smithi100:workunit test cls/test_cls_rbd.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=reef TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_rbd.sh 2024-04-07T16:34:27.981 INFO:tasks.workunit.client.0.smithi100.stdout:Running main() from gmock_main.cc 2024-04-07T16:34:27.981 INFO:tasks.workunit.client.0.smithi100.stdout:[==========] Running 70 tests from 1 test suite. 2024-04-07T16:34:27.981 INFO:tasks.workunit.client.0.smithi100.stdout:[----------] Global test environment set-up. 2024-04-07T16:34:27.981 INFO:tasks.workunit.client.0.smithi100.stdout:[----------] 70 tests from TestClsRbd 2024-04-07T16:34:28.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:34:27 smithi144 bash[66654]: audit 2024-04-07T16:34:26.843625+0000 mon.a (mon.0) 1802 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111084-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:34:28.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:34:27 smithi144 bash[66654]: cluster 2024-04-07T16:34:26.850473+0000 mon.a (mon.0) 1803 : cluster [DBG] osdmap e981: 8 total, 8 up, 8 in 2024-04-07T16:34:28.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:34:27 smithi144 bash[66654]: cluster 2024-04-07T16:34:26.964816+0000 mgr.y (mgr.25175) 573 : cluster [DBG] pgmap v1129: 137 pgs: 32 unknown, 105 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:34:28.190 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:34:27 smithi100 bash[74942]: audit 2024-04-07T16:34:26.843625+0000 mon.a (mon.0) 1802 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111084-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:34:28.191 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:34:27 smithi100 bash[74942]: cluster 2024-04-07T16:34:26.850473+0000 mon.a (mon.0) 1803 : cluster [DBG] osdmap e981: 8 total, 8 up, 8 in 2024-04-07T16:34:28.191 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:34:27 smithi100 bash[74942]: cluster 2024-04-07T16:34:26.964816+0000 mgr.y (mgr.25175) 573 : cluster [DBG] pgmap v1129: 137 pgs: 32 unknown, 105 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:34:28.191 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:34:27 smithi100 bash[76202]: audit 2024-04-07T16:34:26.843625+0000 mon.a (mon.0) 1802 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111084-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:34:28.191 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:34:27 smithi100 bash[76202]: cluster 2024-04-07T16:34:26.850473+0000 mon.a (mon.0) 1803 : cluster [DBG] osdmap e981: 8 total, 8 up, 8 in 2024-04-07T16:34:28.191 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:34:27 smithi100 bash[76202]: cluster 2024-04-07T16:34:26.964816+0000 mgr.y (mgr.25175) 573 : cluster [DBG] pgmap v1129: 137 pgs: 32 unknown, 105 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:34:28.690 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:34:28 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:34:28] "GET /metrics HTTP/1.1" 200 36358 "" "Prometheus/2.43.0" 2024-04-07T16:34:29.190 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:34:28 smithi100 bash[74942]: cluster 2024-04-07T16:34:27.865068+0000 mon.a (mon.0) 1804 : cluster [DBG] osdmap e982: 8 total, 8 up, 8 in 2024-04-07T16:34:29.191 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:34:28 smithi100 bash[76202]: cluster 2024-04-07T16:34:27.865068+0000 mon.a (mon.0) 1804 : cluster [DBG] osdmap e982: 8 total, 8 up, 8 in 2024-04-07T16:34:29.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:34:28 smithi144 bash[66654]: cluster 2024-04-07T16:34:27.865068+0000 mon.a (mon.0) 1804 : cluster [DBG] osdmap e982: 8 total, 8 up, 8 in 2024-04-07T16:34:29.861 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestClsRbd.get_all_features 2024-04-07T16:34:29.900 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestClsRbd.get_all_features (39 ms) 2024-04-07T16:34:29.900 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestClsRbd.copyup 2024-04-07T16:34:30.142 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestClsRbd.copyup (242 ms) 2024-04-07T16:34:30.143 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestClsRbd.sparse_copyup 2024-04-07T16:34:30.172 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestClsRbd.sparse_copyup (29 ms) 2024-04-07T16:34:30.172 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestClsRbd.get_and_set_id 2024-04-07T16:34:30.189 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestClsRbd.get_and_set_id (17 ms) 2024-04-07T16:34:30.189 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestClsRbd.add_remove_child 2024-04-07T16:34:30.190 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:34:29 smithi100 bash[74942]: cluster 2024-04-07T16:34:28.866511+0000 mon.a (mon.0) 1805 : cluster [DBG] osdmap e983: 8 total, 8 up, 8 in 2024-04-07T16:34:30.190 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:34:29 smithi100 bash[74942]: audit 2024-04-07T16:34:28.869364+0000 mon.a (mon.0) 1806 : audit [INF] from='client.? 172.21.15.100:0/305988345' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111159-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:34:30.190 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:34:29 smithi100 bash[74942]: cluster 2024-04-07T16:34:28.965555+0000 mgr.y (mgr.25175) 574 : cluster [DBG] pgmap v1132: 137 pgs: 32 unknown, 105 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:34:30.190 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:34:29 smithi100 bash[74942]: audit 2024-04-07T16:34:29.861437+0000 mon.a (mon.0) 1807 : audit [INF] from='client.? 172.21.15.100:0/305988345' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111159-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:34:30.190 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:34:29 smithi100 bash[76202]: cluster 2024-04-07T16:34:28.866511+0000 mon.a (mon.0) 1805 : cluster [DBG] osdmap e983: 8 total, 8 up, 8 in 2024-04-07T16:34:30.191 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:34:29 smithi100 bash[76202]: audit 2024-04-07T16:34:28.869364+0000 mon.a (mon.0) 1806 : audit [INF] from='client.? 172.21.15.100:0/305988345' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111159-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:34:30.191 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:34:29 smithi100 bash[76202]: cluster 2024-04-07T16:34:28.965555+0000 mgr.y (mgr.25175) 574 : cluster [DBG] pgmap v1132: 137 pgs: 32 unknown, 105 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:34:30.191 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:34:29 smithi100 bash[76202]: audit 2024-04-07T16:34:29.861437+0000 mon.a (mon.0) 1807 : audit [INF] from='client.? 172.21.15.100:0/305988345' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111159-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:34:30.220 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestClsRbd.add_remove_child (32 ms) 2024-04-07T16:34:30.220 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestClsRbd.directory_methods 2024-04-07T16:34:30.299 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestClsRbd.directory_methods (79 ms) 2024-04-07T16:34:30.299 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestClsRbd.create 2024-04-07T16:34:30.337 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestClsRbd.create (38 ms) 2024-04-07T16:34:30.337 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestClsRbd.get_features 2024-04-07T16:34:30.341 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestClsRbd.get_features (4 ms) 2024-04-07T16:34:30.341 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestClsRbd.get_object_prefix 2024-04-07T16:34:30.345 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestClsRbd.get_object_prefix (4 ms) 2024-04-07T16:34:30.345 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestClsRbd.get_create_timestamp 2024-04-07T16:34:30.350 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestClsRbd.get_create_timestamp (5 ms) 2024-04-07T16:34:30.350 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestClsRbd.get_access_timestamp 2024-04-07T16:34:30.355 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestClsRbd.get_access_timestamp (5 ms) 2024-04-07T16:34:30.355 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestClsRbd.get_modify_timestamp 2024-04-07T16:34:30.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:34:29 smithi144 bash[66654]: cluster 2024-04-07T16:34:28.866511+0000 mon.a (mon.0) 1805 : cluster [DBG] osdmap e983: 8 total, 8 up, 8 in 2024-04-07T16:34:30.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:34:29 smithi144 bash[66654]: audit 2024-04-07T16:34:28.869364+0000 mon.a (mon.0) 1806 : audit [INF] from='client.? 172.21.15.100:0/305988345' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111159-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:34:30.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:34:29 smithi144 bash[66654]: cluster 2024-04-07T16:34:28.965555+0000 mgr.y (mgr.25175) 574 : cluster [DBG] pgmap v1132: 137 pgs: 32 unknown, 105 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:34:30.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:34:29 smithi144 bash[66654]: audit 2024-04-07T16:34:29.861437+0000 mon.a (mon.0) 1807 : audit [INF] from='client.? 172.21.15.100:0/305988345' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111159-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:34:30.359 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestClsRbd.get_modify_timestamp (4 ms) 2024-04-07T16:34:30.359 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestClsRbd.get_data_pool 2024-04-07T16:34:30.369 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestClsRbd.get_data_pool (10 ms) 2024-04-07T16:34:30.369 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestClsRbd.get_size 2024-04-07T16:34:30.381 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestClsRbd.get_size (11 ms) 2024-04-07T16:34:30.381 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestClsRbd.set_size 2024-04-07T16:34:30.393 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestClsRbd.set_size (13 ms) 2024-04-07T16:34:30.393 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestClsRbd.protection_status 2024-04-07T16:34:30.450 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestClsRbd.protection_status (56 ms) 2024-04-07T16:34:30.450 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestClsRbd.snapshot_limits 2024-04-07T16:34:30.475 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestClsRbd.snapshot_limits (25 ms) 2024-04-07T16:34:30.475 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestClsRbd.parents_v1 2024-04-07T16:34:30.596 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestClsRbd.parents_v1 (122 ms) 2024-04-07T16:34:30.597 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestClsRbd.parents_v2 2024-04-07T16:34:30.687 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestClsRbd.parents_v2 (91 ms) 2024-04-07T16:34:30.687 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestClsRbd.snapshots 2024-04-07T16:34:30.747 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestClsRbd.snapshots (60 ms) 2024-04-07T16:34:30.747 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestClsRbd.snapid_race 2024-04-07T16:34:30.760 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestClsRbd.snapid_race (13 ms) 2024-04-07T16:34:30.760 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestClsRbd.stripingv2 2024-04-07T16:34:30.786 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestClsRbd.stripingv2 (26 ms) 2024-04-07T16:34:30.787 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestClsRbd.object_map_save 2024-04-07T16:34:30.791 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestClsRbd.object_map_save (5 ms) 2024-04-07T16:34:30.791 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestClsRbd.object_map_resize 2024-04-07T16:34:30.807 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestClsRbd.object_map_resize (15 ms) 2024-04-07T16:34:30.807 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestClsRbd.object_map_update 2024-04-07T16:34:30.821 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestClsRbd.object_map_update (15 ms) 2024-04-07T16:34:30.822 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestClsRbd.object_map_load_enoent 2024-04-07T16:34:30.822 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestClsRbd.object_map_load_enoent (1 ms) 2024-04-07T16:34:30.822 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestClsRbd.object_map_snap_add 2024-04-07T16:34:30.839 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestClsRbd.object_map_snap_add (16 ms) 2024-04-07T16:34:30.839 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestClsRbd.object_map_snap_remove 2024-04-07T16:34:30.853 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestClsRbd.object_map_snap_remove (15 ms) 2024-04-07T16:34:30.853 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestClsRbd.flags 2024-04-07T16:34:30.868 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestClsRbd.flags (15 ms) 2024-04-07T16:34:30.869 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestClsRbd.metadata 2024-04-07T16:34:30.961 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestClsRbd.metadata (92 ms) 2024-04-07T16:34:30.961 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestClsRbd.set_features 2024-04-07T16:34:30.981 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestClsRbd.set_features (21 ms) 2024-04-07T16:34:30.981 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestClsRbd.mirror 2024-04-07T16:34:31.092 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestClsRbd.mirror (111 ms) 2024-04-07T16:34:31.092 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestClsRbd.mirror_image 2024-04-07T16:34:31.141 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestClsRbd.mirror_image (49 ms) 2024-04-07T16:34:31.141 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestClsRbd.mirror_image_status 2024-04-07T16:34:31.190 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:34:30 smithi100 bash[74942]: cluster 2024-04-07T16:34:29.871349+0000 mon.a (mon.0) 1808 : cluster [DBG] osdmap e984: 8 total, 8 up, 8 in 2024-04-07T16:34:31.190 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:34:30 smithi100 bash[74942]: cluster 2024-04-07T16:34:29.872719+0000 mon.a (mon.0) 1809 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T16:34:31.191 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:34:30 smithi100 bash[76202]: cluster 2024-04-07T16:34:29.871349+0000 mon.a (mon.0) 1808 : cluster [DBG] osdmap e984: 8 total, 8 up, 8 in 2024-04-07T16:34:31.191 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:34:30 smithi100 bash[76202]: cluster 2024-04-07T16:34:29.872719+0000 mon.a (mon.0) 1809 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T16:34:31.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:34:30 smithi144 bash[66654]: cluster 2024-04-07T16:34:29.871349+0000 mon.a (mon.0) 1808 : cluster [DBG] osdmap e984: 8 total, 8 up, 8 in 2024-04-07T16:34:31.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:34:30 smithi144 bash[66654]: cluster 2024-04-07T16:34:29.872719+0000 mon.a (mon.0) 1809 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T16:34:32.190 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:34:31 smithi100 bash[74942]: cluster 2024-04-07T16:34:30.888611+0000 mon.a (mon.0) 1810 : cluster [DBG] osdmap e985: 8 total, 8 up, 8 in 2024-04-07T16:34:32.191 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:34:31 smithi100 bash[74942]: cluster 2024-04-07T16:34:30.966275+0000 mgr.y (mgr.25175) 575 : cluster [DBG] pgmap v1135: 137 pgs: 8 creating+peering, 21 unknown, 108 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:34:32.191 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:34:31 smithi100 bash[74942]: audit 2024-04-07T16:34:31.753833+0000 mon.a (mon.0) 1811 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:34:32.191 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:34:31 smithi100 bash[74942]: audit 2024-04-07T16:34:31.756548+0000 mon.a (mon.0) 1812 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:34:32.191 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:34:31 smithi100 bash[76202]: cluster 2024-04-07T16:34:30.888611+0000 mon.a (mon.0) 1810 : cluster [DBG] osdmap e985: 8 total, 8 up, 8 in 2024-04-07T16:34:32.191 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:34:31 smithi100 bash[76202]: cluster 2024-04-07T16:34:30.966275+0000 mgr.y (mgr.25175) 575 : cluster [DBG] pgmap v1135: 137 pgs: 8 creating+peering, 21 unknown, 108 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:34:32.191 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:34:31 smithi100 bash[76202]: audit 2024-04-07T16:34:31.753833+0000 mon.a (mon.0) 1811 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:34:32.191 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:34:31 smithi100 bash[76202]: audit 2024-04-07T16:34:31.756548+0000 mon.a (mon.0) 1812 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:34:32.306 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:34:31 smithi144 bash[66654]: cluster 2024-04-07T16:34:30.888611+0000 mon.a (mon.0) 1810 : cluster [DBG] osdmap e985: 8 total, 8 up, 8 in 2024-04-07T16:34:32.306 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:34:31 smithi144 bash[66654]: cluster 2024-04-07T16:34:30.966275+0000 mgr.y (mgr.25175) 575 : cluster [DBG] pgmap v1135: 137 pgs: 8 creating+peering, 21 unknown, 108 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:34:32.306 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:34:31 smithi144 bash[66654]: audit 2024-04-07T16:34:31.753833+0000 mon.a (mon.0) 1811 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:34:32.306 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:34:31 smithi144 bash[66654]: audit 2024-04-07T16:34:31.756548+0000 mon.a (mon.0) 1812 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:34:32.607 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:34:32 smithi144 bash[95233]: ts=2024-04-07T16:34:32.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:34:33.190 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:34:32 smithi100 bash[74942]: cluster 2024-04-07T16:34:31.891505+0000 mon.a (mon.0) 1813 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T16:34:33.190 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:34:32 smithi100 bash[74942]: cluster 2024-04-07T16:34:31.891564+0000 mon.a (mon.0) 1814 : cluster [INF] Cluster is now healthy 2024-04-07T16:34:33.191 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:34:32 smithi100 bash[76202]: cluster 2024-04-07T16:34:31.891505+0000 mon.a (mon.0) 1813 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T16:34:33.191 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:34:32 smithi100 bash[76202]: cluster 2024-04-07T16:34:31.891564+0000 mon.a (mon.0) 1814 : cluster [INF] Cluster is now healthy 2024-04-07T16:34:33.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:34:32 smithi144 bash[66654]: cluster 2024-04-07T16:34:31.891505+0000 mon.a (mon.0) 1813 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T16:34:33.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:34:32 smithi144 bash[66654]: cluster 2024-04-07T16:34:31.891564+0000 mon.a (mon.0) 1814 : cluster [INF] Cluster is now healthy 2024-04-07T16:34:34.190 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:34:33 smithi100 bash[74942]: cluster 2024-04-07T16:34:32.967619+0000 mgr.y (mgr.25175) 576 : cluster [DBG] pgmap v1136: 137 pgs: 8 creating+peering, 8 unknown, 121 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 1.6 MiB/s rd, 813 KiB/s wr, 65 op/s 2024-04-07T16:34:34.190 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:34:33 smithi100 bash[76202]: cluster 2024-04-07T16:34:32.967619+0000 mgr.y (mgr.25175) 576 : cluster [DBG] pgmap v1136: 137 pgs: 8 creating+peering, 8 unknown, 121 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 1.6 MiB/s rd, 813 KiB/s wr, 65 op/s 2024-04-07T16:34:34.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:34:33 smithi144 bash[66654]: cluster 2024-04-07T16:34:32.967619+0000 mgr.y (mgr.25175) 576 : cluster [DBG] pgmap v1136: 137 pgs: 8 creating+peering, 8 unknown, 121 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 1.6 MiB/s rd, 813 KiB/s wr, 65 op/s 2024-04-07T16:34:36.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:34:36 smithi144 bash[66654]: cluster 2024-04-07T16:34:34.968883+0000 mgr.y (mgr.25175) 577 : cluster [DBG] pgmap v1137: 137 pgs: 137 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 1.7 MiB/s rd, 934 KiB/s wr, 728 op/s 2024-04-07T16:34:36.440 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:34:36 smithi100 bash[74942]: cluster 2024-04-07T16:34:34.968883+0000 mgr.y (mgr.25175) 577 : cluster [DBG] pgmap v1137: 137 pgs: 137 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 1.7 MiB/s rd, 934 KiB/s wr, 728 op/s 2024-04-07T16:34:36.440 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:34:36 smithi100 bash[76202]: cluster 2024-04-07T16:34:34.968883+0000 mgr.y (mgr.25175) 577 : cluster [DBG] pgmap v1137: 137 pgs: 137 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 1.7 MiB/s rd, 934 KiB/s wr, 728 op/s 2024-04-07T16:34:37.856 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestClsRbd.mirror_image_status (6714 ms) 2024-04-07T16:34:37.856 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestClsRbd.mirror_image_map 2024-04-07T16:34:38.016 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestClsRbd.mirror_image_map (160 ms) 2024-04-07T16:34:38.016 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestClsRbd.mirror_instances 2024-04-07T16:34:38.033 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestClsRbd.mirror_instances (17 ms) 2024-04-07T16:34:38.033 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestClsRbd.mirror_snapshot 2024-04-07T16:34:38.058 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestClsRbd.mirror_snapshot (25 ms) 2024-04-07T16:34:38.058 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestClsRbd.group_dir_list 2024-04-07T16:34:38.064 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestClsRbd.group_dir_list (6 ms) 2024-04-07T16:34:38.064 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestClsRbd.group_dir_add 2024-04-07T16:34:38.068 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestClsRbd.group_dir_add (4 ms) 2024-04-07T16:34:38.069 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestClsRbd.dir_add_already_existing 2024-04-07T16:34:38.076 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestClsRbd.dir_add_already_existing (7 ms) 2024-04-07T16:34:38.076 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestClsRbd.group_dir_rename 2024-04-07T16:34:38.090 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestClsRbd.group_dir_rename (15 ms) 2024-04-07T16:34:38.090 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestClsRbd.group_dir_remove 2024-04-07T16:34:38.098 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestClsRbd.group_dir_remove (8 ms) 2024-04-07T16:34:38.098 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestClsRbd.group_dir_remove_missing 2024-04-07T16:34:38.109 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestClsRbd.group_dir_remove_missing (10 ms) 2024-04-07T16:34:38.109 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestClsRbd.group_image_add 2024-04-07T16:34:38.113 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestClsRbd.group_image_add (5 ms) 2024-04-07T16:34:38.113 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestClsRbd.group_image_remove 2024-04-07T16:34:38.120 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestClsRbd.group_image_remove (6 ms) 2024-04-07T16:34:38.120 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestClsRbd.group_image_list 2024-04-07T16:34:38.126 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestClsRbd.group_image_list (6 ms) 2024-04-07T16:34:38.126 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestClsRbd.group_image_clean 2024-04-07T16:34:38.135 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestClsRbd.group_image_clean (10 ms) 2024-04-07T16:34:38.136 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestClsRbd.image_group_add 2024-04-07T16:34:38.140 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestClsRbd.image_group_add (5 ms) 2024-04-07T16:34:38.141 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestClsRbd.image_group_remove 2024-04-07T16:34:38.147 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestClsRbd.image_group_remove (7 ms) 2024-04-07T16:34:38.147 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestClsRbd.image_group_get 2024-04-07T16:34:38.152 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestClsRbd.image_group_get (4 ms) 2024-04-07T16:34:38.152 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestClsRbd.group_snap_set_empty_name 2024-04-07T16:34:38.155 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestClsRbd.group_snap_set_empty_name (4 ms) 2024-04-07T16:34:38.155 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestClsRbd.group_snap_set_empty_id 2024-04-07T16:34:38.165 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestClsRbd.group_snap_set_empty_id (10 ms) 2024-04-07T16:34:38.165 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestClsRbd.group_snap_set_duplicate_id 2024-04-07T16:34:38.171 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestClsRbd.group_snap_set_duplicate_id (6 ms) 2024-04-07T16:34:38.171 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestClsRbd.group_snap_set_duplicate_name 2024-04-07T16:34:38.178 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestClsRbd.group_snap_set_duplicate_name (5 ms) 2024-04-07T16:34:38.178 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestClsRbd.group_snap_set 2024-04-07T16:34:38.180 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestClsRbd.group_snap_set (4 ms) 2024-04-07T16:34:38.180 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestClsRbd.group_snap_list 2024-04-07T16:34:38.187 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestClsRbd.group_snap_list (6 ms) 2024-04-07T16:34:38.187 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestClsRbd.group_snap_list_max_return 2024-04-07T16:34:38.215 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestClsRbd.group_snap_list_max_return (29 ms) 2024-04-07T16:34:38.215 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestClsRbd.group_snap_list_max_read 2024-04-07T16:34:38.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:34:38 smithi144 bash[66654]: cluster 2024-04-07T16:34:36.969865+0000 mgr.y (mgr.25175) 578 : cluster [DBG] pgmap v1138: 137 pgs: 137 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 1.3 MiB/s rd, 713 KiB/s wr, 558 op/s 2024-04-07T16:34:38.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:34:38 smithi144 bash[66654]: audit 2024-04-07T16:34:37.805859+0000 mon.a (mon.0) 1815 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:34:38.431 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:34:38 smithi100 bash[74942]: cluster 2024-04-07T16:34:36.969865+0000 mgr.y (mgr.25175) 578 : cluster [DBG] pgmap v1138: 137 pgs: 137 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 1.3 MiB/s rd, 713 KiB/s wr, 558 op/s 2024-04-07T16:34:38.431 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:34:38 smithi100 bash[74942]: audit 2024-04-07T16:34:37.805859+0000 mon.a (mon.0) 1815 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:34:38.431 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:34:38 smithi100 bash[76202]: cluster 2024-04-07T16:34:36.969865+0000 mgr.y (mgr.25175) 578 : cluster [DBG] pgmap v1138: 137 pgs: 137 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 1.3 MiB/s rd, 713 KiB/s wr, 558 op/s 2024-04-07T16:34:38.431 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:34:38 smithi100 bash[76202]: audit 2024-04-07T16:34:37.805859+0000 mon.a (mon.0) 1815 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:34:38.484 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestClsRbd.group_snap_list_max_read (269 ms) 2024-04-07T16:34:38.485 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestClsRbd.group_snap_remove 2024-04-07T16:34:38.488 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestClsRbd.group_snap_remove (4 ms) 2024-04-07T16:34:38.488 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestClsRbd.group_snap_get_by_id 2024-04-07T16:34:38.490 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestClsRbd.group_snap_get_by_id (2 ms) 2024-04-07T16:34:38.490 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestClsRbd.trash_methods 2024-04-07T16:34:38.500 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestClsRbd.trash_methods (10 ms) 2024-04-07T16:34:38.500 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestClsRbd.op_features 2024-04-07T16:34:38.509 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestClsRbd.op_features (8 ms) 2024-04-07T16:34:38.509 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestClsRbd.clone_parent 2024-04-07T16:34:38.538 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestClsRbd.clone_parent (30 ms) 2024-04-07T16:34:38.538 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestClsRbd.clone_parent_ns 2024-04-07T16:34:38.550 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestClsRbd.clone_parent_ns (12 ms) 2024-04-07T16:34:38.550 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestClsRbd.clone_child 2024-04-07T16:34:38.567 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestClsRbd.clone_child (17 ms) 2024-04-07T16:34:38.567 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestClsRbd.namespace_methods 2024-04-07T16:34:38.575 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestClsRbd.namespace_methods (8 ms) 2024-04-07T16:34:38.575 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestClsRbd.migration 2024-04-07T16:34:38.590 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestClsRbd.migration (14 ms) 2024-04-07T16:34:38.590 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestClsRbd.migration_v1 2024-04-07T16:34:38.605 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestClsRbd.migration_v1 (15 ms) 2024-04-07T16:34:38.605 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestClsRbd.assert_snapc_seq 2024-04-07T16:34:38.690 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:34:38 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:34:38] "GET /metrics HTTP/1.1" 200 36393 "" "Prometheus/2.43.0" 2024-04-07T16:34:39.690 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:34:39 smithi100 bash[74942]: audit 2024-04-07T16:34:38.381002+0000 mon.a (mon.0) 1816 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:34:39.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:34:39 smithi100 bash[74942]: audit 2024-04-07T16:34:38.387954+0000 mon.a (mon.0) 1817 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:34:39.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:34:39 smithi100 bash[74942]: audit 2024-04-07T16:34:39.014290+0000 mon.a (mon.0) 1818 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:34:39.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:34:39 smithi100 bash[74942]: audit 2024-04-07T16:34:39.016275+0000 mon.a (mon.0) 1819 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:34:39.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:34:39 smithi100 bash[74942]: audit 2024-04-07T16:34:39.026389+0000 mon.a (mon.0) 1820 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:34:39.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:34:39 smithi100 bash[76202]: audit 2024-04-07T16:34:38.381002+0000 mon.a (mon.0) 1816 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:34:39.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:34:39 smithi100 bash[76202]: audit 2024-04-07T16:34:38.387954+0000 mon.a (mon.0) 1817 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:34:39.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:34:39 smithi100 bash[76202]: audit 2024-04-07T16:34:39.014290+0000 mon.a (mon.0) 1818 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:34:39.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:34:39 smithi100 bash[76202]: audit 2024-04-07T16:34:39.016275+0000 mon.a (mon.0) 1819 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:34:39.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:34:39 smithi100 bash[76202]: audit 2024-04-07T16:34:39.026389+0000 mon.a (mon.0) 1820 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:34:39.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:34:39 smithi144 bash[66654]: audit 2024-04-07T16:34:38.381002+0000 mon.a (mon.0) 1816 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:34:39.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:34:39 smithi144 bash[66654]: audit 2024-04-07T16:34:38.387954+0000 mon.a (mon.0) 1817 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:34:39.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:34:39 smithi144 bash[66654]: audit 2024-04-07T16:34:39.014290+0000 mon.a (mon.0) 1818 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:34:39.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:34:39 smithi144 bash[66654]: audit 2024-04-07T16:34:39.016275+0000 mon.a (mon.0) 1819 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:34:39.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:34:39 smithi144 bash[66654]: audit 2024-04-07T16:34:39.026389+0000 mon.a (mon.0) 1820 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:34:40.401 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestClsRbd.assert_snapc_seq (1795 ms) 2024-04-07T16:34:40.401 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestClsRbd.sparsify 2024-04-07T16:34:40.488 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestClsRbd.sparsify (86 ms) 2024-04-07T16:34:40.690 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:34:40 smithi100 bash[74942]: cluster 2024-04-07T16:34:38.971270+0000 mgr.y (mgr.25175) 579 : cluster [DBG] pgmap v1139: 137 pgs: 137 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 1.2 MiB/s rd, 628 KiB/s wr, 495 op/s 2024-04-07T16:34:40.690 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:34:40 smithi100 bash[74942]: cluster 2024-04-07T16:34:39.402628+0000 mon.a (mon.0) 1821 : cluster [DBG] osdmap e986: 8 total, 8 up, 8 in 2024-04-07T16:34:40.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:34:40 smithi100 bash[76202]: cluster 2024-04-07T16:34:38.971270+0000 mgr.y (mgr.25175) 579 : cluster [DBG] pgmap v1139: 137 pgs: 137 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 1.2 MiB/s rd, 628 KiB/s wr, 495 op/s 2024-04-07T16:34:40.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:34:40 smithi100 bash[76202]: cluster 2024-04-07T16:34:39.402628+0000 mon.a (mon.0) 1821 : cluster [DBG] osdmap e986: 8 total, 8 up, 8 in 2024-04-07T16:34:40.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:34:40 smithi144 bash[66654]: cluster 2024-04-07T16:34:38.971270+0000 mgr.y (mgr.25175) 579 : cluster [DBG] pgmap v1139: 137 pgs: 137 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 1.2 MiB/s rd, 628 KiB/s wr, 495 op/s 2024-04-07T16:34:40.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:34:40 smithi144 bash[66654]: cluster 2024-04-07T16:34:39.402628+0000 mon.a (mon.0) 1821 : cluster [DBG] osdmap e986: 8 total, 8 up, 8 in 2024-04-07T16:34:41.414 INFO:tasks.workunit.client.0.smithi100.stdout:[----------] 70 tests from TestClsRbd (10626 ms total) 2024-04-07T16:34:41.414 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T16:34:41.414 INFO:tasks.workunit.client.0.smithi100.stdout:[----------] Global test environment tear-down 2024-04-07T16:34:41.414 INFO:tasks.workunit.client.0.smithi100.stdout:[==========] 70 tests from 1 test suite ran. (13432 ms total) 2024-04-07T16:34:41.414 INFO:tasks.workunit.client.0.smithi100.stdout:[ PASSED ] 70 tests. 2024-04-07T16:34:41.420 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-07T16:34:41.420 DEBUG:teuthology.orchestra.run.smithi100:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-07T16:34:41.474 INFO:tasks.workunit:Running workunit cls/test_cls_refcount.sh... 2024-04-07T16:34:41.474 DEBUG:teuthology.orchestra.run.smithi100:workunit test cls/test_cls_refcount.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=reef TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_refcount.sh 2024-04-07T16:34:41.536 INFO:tasks.workunit.client.0.smithi100.stdout:Running main() from gmock_main.cc 2024-04-07T16:34:41.536 INFO:tasks.workunit.client.0.smithi100.stdout:[==========] Running 10 tests from 1 test suite. 2024-04-07T16:34:41.536 INFO:tasks.workunit.client.0.smithi100.stdout:[----------] Global test environment set-up. 2024-04-07T16:34:41.536 INFO:tasks.workunit.client.0.smithi100.stdout:[----------] 10 tests from cls_rgw 2024-04-07T16:34:41.536 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] cls_rgw.test_implicit 2024-04-07T16:34:41.690 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:34:41 smithi100 bash[74942]: cluster 2024-04-07T16:34:40.411669+0000 mon.a (mon.0) 1822 : cluster [DBG] osdmap e987: 8 total, 8 up, 8 in 2024-04-07T16:34:41.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:34:41 smithi100 bash[76202]: cluster 2024-04-07T16:34:40.411669+0000 mon.a (mon.0) 1822 : cluster [DBG] osdmap e987: 8 total, 8 up, 8 in 2024-04-07T16:34:41.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:34:41 smithi144 bash[66654]: cluster 2024-04-07T16:34:40.411669+0000 mon.a (mon.0) 1822 : cluster [DBG] osdmap e987: 8 total, 8 up, 8 in 2024-04-07T16:34:42.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:34:42 smithi144 bash[66654]: cluster 2024-04-07T16:34:40.971988+0000 mgr.y (mgr.25175) 580 : cluster [DBG] pgmap v1142: 137 pgs: 137 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 1.1 MiB/s rd, 539 KiB/s wr, 1.68k op/s 2024-04-07T16:34:42.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:34:42 smithi144 bash[66654]: cluster 2024-04-07T16:34:41.422166+0000 mon.a (mon.0) 1823 : cluster [DBG] osdmap e988: 8 total, 8 up, 8 in 2024-04-07T16:34:42.607 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:34:42 smithi144 bash[95233]: ts=2024-04-07T16:34:42.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:34:42.693 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:34:42 smithi100 bash[74942]: cluster 2024-04-07T16:34:40.971988+0000 mgr.y (mgr.25175) 580 : cluster [DBG] pgmap v1142: 137 pgs: 137 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 1.1 MiB/s rd, 539 KiB/s wr, 1.68k op/s 2024-04-07T16:34:42.693 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:34:42 smithi100 bash[74942]: cluster 2024-04-07T16:34:41.422166+0000 mon.a (mon.0) 1823 : cluster [DBG] osdmap e988: 8 total, 8 up, 8 in 2024-04-07T16:34:42.694 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:34:42 smithi100 bash[76202]: cluster 2024-04-07T16:34:40.971988+0000 mgr.y (mgr.25175) 580 : cluster [DBG] pgmap v1142: 137 pgs: 137 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 1.1 MiB/s rd, 539 KiB/s wr, 1.68k op/s 2024-04-07T16:34:42.694 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:34:42 smithi100 bash[76202]: cluster 2024-04-07T16:34:41.422166+0000 mon.a (mon.0) 1823 : cluster [DBG] osdmap e988: 8 total, 8 up, 8 in 2024-04-07T16:34:43.690 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:34:43 smithi100 bash[74942]: cluster 2024-04-07T16:34:42.433555+0000 mon.a (mon.0) 1824 : cluster [DBG] osdmap e989: 8 total, 8 up, 8 in 2024-04-07T16:34:43.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:34:43 smithi100 bash[74942]: audit 2024-04-07T16:34:42.439760+0000 mon.c (mon.2) 49 : audit [INF] from='client.? 172.21.15.100:0/1772300682' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111196-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:34:43.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:34:43 smithi100 bash[74942]: audit 2024-04-07T16:34:42.440738+0000 mon.a (mon.0) 1825 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111196-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:34:43.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:34:43 smithi100 bash[76202]: cluster 2024-04-07T16:34:42.433555+0000 mon.a (mon.0) 1824 : cluster [DBG] osdmap e989: 8 total, 8 up, 8 in 2024-04-07T16:34:43.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:34:43 smithi100 bash[76202]: audit 2024-04-07T16:34:42.439760+0000 mon.c (mon.2) 49 : audit [INF] from='client.? 172.21.15.100:0/1772300682' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111196-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:34:43.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:34:43 smithi100 bash[76202]: audit 2024-04-07T16:34:42.440738+0000 mon.a (mon.0) 1825 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111196-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:34:43.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:34:43 smithi144 bash[66654]: cluster 2024-04-07T16:34:42.433555+0000 mon.a (mon.0) 1824 : cluster [DBG] osdmap e989: 8 total, 8 up, 8 in 2024-04-07T16:34:43.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:34:43 smithi144 bash[66654]: audit 2024-04-07T16:34:42.439760+0000 mon.c (mon.2) 49 : audit [INF] from='client.? 172.21.15.100:0/1772300682' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111196-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:34:43.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:34:43 smithi144 bash[66654]: audit 2024-04-07T16:34:42.440738+0000 mon.a (mon.0) 1825 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111196-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:34:44.444 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] cls_rgw.test_implicit (2907 ms) 2024-04-07T16:34:44.444 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] cls_rgw.test_implicit_idempotent 2024-04-07T16:34:44.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:34:44 smithi100 bash[74942]: cluster 2024-04-07T16:34:42.972719+0000 mgr.y (mgr.25175) 581 : cluster [DBG] pgmap v1145: 137 pgs: 32 unknown, 105 active+clean; 583 KiB data, 2.7 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:34:44.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:34:44 smithi100 bash[74942]: cluster 2024-04-07T16:34:43.425680+0000 mon.a (mon.0) 1826 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T16:34:44.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:34:44 smithi100 bash[74942]: audit 2024-04-07T16:34:43.434947+0000 mon.a (mon.0) 1827 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111196-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:34:44.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:34:44 smithi100 bash[74942]: cluster 2024-04-07T16:34:43.444668+0000 mon.a (mon.0) 1828 : cluster [DBG] osdmap e990: 8 total, 8 up, 8 in 2024-04-07T16:34:44.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:34:44 smithi100 bash[76202]: cluster 2024-04-07T16:34:42.972719+0000 mgr.y (mgr.25175) 581 : cluster [DBG] pgmap v1145: 137 pgs: 32 unknown, 105 active+clean; 583 KiB data, 2.7 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:34:44.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:34:44 smithi100 bash[76202]: cluster 2024-04-07T16:34:43.425680+0000 mon.a (mon.0) 1826 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T16:34:44.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:34:44 smithi100 bash[76202]: audit 2024-04-07T16:34:43.434947+0000 mon.a (mon.0) 1827 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111196-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:34:44.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:34:44 smithi100 bash[76202]: cluster 2024-04-07T16:34:43.444668+0000 mon.a (mon.0) 1828 : cluster [DBG] osdmap e990: 8 total, 8 up, 8 in 2024-04-07T16:34:44.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:34:44 smithi144 bash[66654]: cluster 2024-04-07T16:34:42.972719+0000 mgr.y (mgr.25175) 581 : cluster [DBG] pgmap v1145: 137 pgs: 32 unknown, 105 active+clean; 583 KiB data, 2.7 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:34:44.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:34:44 smithi144 bash[66654]: cluster 2024-04-07T16:34:43.425680+0000 mon.a (mon.0) 1826 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T16:34:44.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:34:44 smithi144 bash[66654]: audit 2024-04-07T16:34:43.434947+0000 mon.a (mon.0) 1827 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111196-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:34:44.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:34:44 smithi144 bash[66654]: cluster 2024-04-07T16:34:43.444668+0000 mon.a (mon.0) 1828 : cluster [DBG] osdmap e990: 8 total, 8 up, 8 in 2024-04-07T16:34:45.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:34:45 smithi144 bash[66654]: cluster 2024-04-07T16:34:44.451737+0000 mon.a (mon.0) 1829 : cluster [DBG] osdmap e991: 8 total, 8 up, 8 in 2024-04-07T16:34:45.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:34:45 smithi100 bash[74942]: cluster 2024-04-07T16:34:44.451737+0000 mon.a (mon.0) 1829 : cluster [DBG] osdmap e991: 8 total, 8 up, 8 in 2024-04-07T16:34:45.941 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:34:45 smithi100 bash[76202]: cluster 2024-04-07T16:34:44.451737+0000 mon.a (mon.0) 1829 : cluster [DBG] osdmap e991: 8 total, 8 up, 8 in 2024-04-07T16:34:46.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:34:46 smithi144 bash[66654]: cluster 2024-04-07T16:34:44.973394+0000 mgr.y (mgr.25175) 582 : cluster [DBG] pgmap v1148: 105 pgs: 105 active+clean; 583 KiB data, 2.7 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:34:46.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:34:46 smithi144 bash[66654]: cluster 2024-04-07T16:34:45.444975+0000 mon.a (mon.0) 1830 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T16:34:46.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:34:46 smithi144 bash[66654]: cluster 2024-04-07T16:34:45.445043+0000 mon.a (mon.0) 1831 : cluster [INF] Cluster is now healthy 2024-04-07T16:34:46.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:34:46 smithi144 bash[66654]: cluster 2024-04-07T16:34:45.470544+0000 mon.a (mon.0) 1832 : cluster [DBG] osdmap e992: 8 total, 8 up, 8 in 2024-04-07T16:34:46.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:34:46 smithi144 bash[66654]: audit 2024-04-07T16:34:45.481080+0000 mon.c (mon.2) 50 : audit [INF] from='client.? 172.21.15.100:0/2864161537' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111196-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:34:46.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:34:46 smithi144 bash[66654]: audit 2024-04-07T16:34:45.482536+0000 mon.a (mon.0) 1833 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111196-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:34:46.941 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:34:46 smithi100 bash[74942]: cluster 2024-04-07T16:34:44.973394+0000 mgr.y (mgr.25175) 582 : cluster [DBG] pgmap v1148: 105 pgs: 105 active+clean; 583 KiB data, 2.7 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:34:46.941 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:34:46 smithi100 bash[74942]: cluster 2024-04-07T16:34:45.444975+0000 mon.a (mon.0) 1830 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T16:34:46.941 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:34:46 smithi100 bash[74942]: cluster 2024-04-07T16:34:45.445043+0000 mon.a (mon.0) 1831 : cluster [INF] Cluster is now healthy 2024-04-07T16:34:46.941 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:34:46 smithi100 bash[74942]: cluster 2024-04-07T16:34:45.470544+0000 mon.a (mon.0) 1832 : cluster [DBG] osdmap e992: 8 total, 8 up, 8 in 2024-04-07T16:34:46.941 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:34:46 smithi100 bash[74942]: audit 2024-04-07T16:34:45.481080+0000 mon.c (mon.2) 50 : audit [INF] from='client.? 172.21.15.100:0/2864161537' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111196-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:34:46.941 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:34:46 smithi100 bash[74942]: audit 2024-04-07T16:34:45.482536+0000 mon.a (mon.0) 1833 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111196-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:34:46.941 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:34:46 smithi100 bash[76202]: cluster 2024-04-07T16:34:44.973394+0000 mgr.y (mgr.25175) 582 : cluster [DBG] pgmap v1148: 105 pgs: 105 active+clean; 583 KiB data, 2.7 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:34:46.941 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:34:46 smithi100 bash[76202]: cluster 2024-04-07T16:34:45.444975+0000 mon.a (mon.0) 1830 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T16:34:46.941 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:34:46 smithi100 bash[76202]: cluster 2024-04-07T16:34:45.445043+0000 mon.a (mon.0) 1831 : cluster [INF] Cluster is now healthy 2024-04-07T16:34:46.942 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:34:46 smithi100 bash[76202]: cluster 2024-04-07T16:34:45.470544+0000 mon.a (mon.0) 1832 : cluster [DBG] osdmap e992: 8 total, 8 up, 8 in 2024-04-07T16:34:46.942 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:34:46 smithi100 bash[76202]: audit 2024-04-07T16:34:45.481080+0000 mon.c (mon.2) 50 : audit [INF] from='client.? 172.21.15.100:0/2864161537' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111196-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:34:46.942 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:34:46 smithi100 bash[76202]: audit 2024-04-07T16:34:45.482536+0000 mon.a (mon.0) 1833 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111196-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:34:47.490 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] cls_rgw.test_implicit_idempotent (3047 ms) 2024-04-07T16:34:47.491 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] cls_rgw.test_put_snap 2024-04-07T16:34:47.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:34:47 smithi144 bash[66654]: audit 2024-04-07T16:34:46.462866+0000 mon.a (mon.0) 1834 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111196-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:34:47.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:34:47 smithi144 bash[66654]: cluster 2024-04-07T16:34:46.471895+0000 mon.a (mon.0) 1835 : cluster [DBG] osdmap e993: 8 total, 8 up, 8 in 2024-04-07T16:34:47.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:34:47 smithi144 bash[66654]: audit 2024-04-07T16:34:46.753932+0000 mon.a (mon.0) 1836 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:34:47.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:34:47 smithi144 bash[66654]: audit 2024-04-07T16:34:46.756296+0000 mon.a (mon.0) 1837 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:34:47.940 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:34:47 smithi100 bash[76202]: audit 2024-04-07T16:34:46.462866+0000 mon.a (mon.0) 1834 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111196-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:34:47.940 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:34:47 smithi100 bash[76202]: cluster 2024-04-07T16:34:46.471895+0000 mon.a (mon.0) 1835 : cluster [DBG] osdmap e993: 8 total, 8 up, 8 in 2024-04-07T16:34:47.941 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:34:47 smithi100 bash[76202]: audit 2024-04-07T16:34:46.753932+0000 mon.a (mon.0) 1836 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:34:47.941 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:34:47 smithi100 bash[76202]: audit 2024-04-07T16:34:46.756296+0000 mon.a (mon.0) 1837 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:34:47.941 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:34:47 smithi100 bash[74942]: audit 2024-04-07T16:34:46.462866+0000 mon.a (mon.0) 1834 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111196-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:34:47.941 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:34:47 smithi100 bash[74942]: cluster 2024-04-07T16:34:46.471895+0000 mon.a (mon.0) 1835 : cluster [DBG] osdmap e993: 8 total, 8 up, 8 in 2024-04-07T16:34:47.941 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:34:47 smithi100 bash[74942]: audit 2024-04-07T16:34:46.753932+0000 mon.a (mon.0) 1836 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:34:47.941 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:34:47 smithi100 bash[74942]: audit 2024-04-07T16:34:46.756296+0000 mon.a (mon.0) 1837 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:34:48.690 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:34:48 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:34:48] "GET /metrics HTTP/1.1" 200 36354 "" "Prometheus/2.43.0" 2024-04-07T16:34:48.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:34:48 smithi100 bash[74942]: cluster 2024-04-07T16:34:46.974060+0000 mgr.y (mgr.25175) 583 : cluster [DBG] pgmap v1151: 137 pgs: 32 unknown, 105 active+clean; 583 KiB data, 2.7 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:34:48.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:34:48 smithi100 bash[74942]: cluster 2024-04-07T16:34:47.497210+0000 mon.a (mon.0) 1838 : cluster [DBG] osdmap e994: 8 total, 8 up, 8 in 2024-04-07T16:34:48.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:34:48 smithi100 bash[76202]: cluster 2024-04-07T16:34:46.974060+0000 mgr.y (mgr.25175) 583 : cluster [DBG] pgmap v1151: 137 pgs: 32 unknown, 105 active+clean; 583 KiB data, 2.7 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:34:48.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:34:48 smithi100 bash[76202]: cluster 2024-04-07T16:34:47.497210+0000 mon.a (mon.0) 1838 : cluster [DBG] osdmap e994: 8 total, 8 up, 8 in 2024-04-07T16:34:48.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:34:48 smithi144 bash[66654]: cluster 2024-04-07T16:34:46.974060+0000 mgr.y (mgr.25175) 583 : cluster [DBG] pgmap v1151: 137 pgs: 32 unknown, 105 active+clean; 583 KiB data, 2.7 GiB used, 713 GiB / 715 GiB avail 2024-04-07T16:34:48.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:34:48 smithi144 bash[66654]: cluster 2024-04-07T16:34:47.497210+0000 mon.a (mon.0) 1838 : cluster [DBG] osdmap e994: 8 total, 8 up, 8 in 2024-04-07T16:34:49.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:34:49 smithi144 bash[66654]: cluster 2024-04-07T16:34:48.500897+0000 mon.a (mon.0) 1839 : cluster [DBG] osdmap e995: 8 total, 8 up, 8 in 2024-04-07T16:34:49.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:34:49 smithi144 bash[66654]: audit 2024-04-07T16:34:48.508120+0000 mon.c (mon.2) 51 : audit [INF] from='client.? 172.21.15.100:0/4288213640' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111196-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:34:49.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:34:49 smithi144 bash[66654]: audit 2024-04-07T16:34:48.508757+0000 mon.a (mon.0) 1840 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111196-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:34:49.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:34:49 smithi144 bash[66654]: cluster 2024-04-07T16:34:48.975007+0000 mgr.y (mgr.25175) 584 : cluster [DBG] pgmap v1154: 137 pgs: 3 creating+peering, 29 unknown, 105 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail 2024-04-07T16:34:49.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:34:49 smithi100 bash[74942]: cluster 2024-04-07T16:34:48.500897+0000 mon.a (mon.0) 1839 : cluster [DBG] osdmap e995: 8 total, 8 up, 8 in 2024-04-07T16:34:49.941 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:34:49 smithi100 bash[74942]: audit 2024-04-07T16:34:48.508120+0000 mon.c (mon.2) 51 : audit [INF] from='client.? 172.21.15.100:0/4288213640' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111196-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:34:49.941 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:34:49 smithi100 bash[74942]: audit 2024-04-07T16:34:48.508757+0000 mon.a (mon.0) 1840 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111196-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:34:49.941 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:34:49 smithi100 bash[74942]: cluster 2024-04-07T16:34:48.975007+0000 mgr.y (mgr.25175) 584 : cluster [DBG] pgmap v1154: 137 pgs: 3 creating+peering, 29 unknown, 105 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail 2024-04-07T16:34:49.941 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:34:49 smithi100 bash[76202]: cluster 2024-04-07T16:34:48.500897+0000 mon.a (mon.0) 1839 : cluster [DBG] osdmap e995: 8 total, 8 up, 8 in 2024-04-07T16:34:49.941 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:34:49 smithi100 bash[76202]: audit 2024-04-07T16:34:48.508120+0000 mon.c (mon.2) 51 : audit [INF] from='client.? 172.21.15.100:0/4288213640' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111196-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:34:49.941 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:34:49 smithi100 bash[76202]: audit 2024-04-07T16:34:48.508757+0000 mon.a (mon.0) 1840 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111196-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:34:49.941 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:34:49 smithi100 bash[76202]: cluster 2024-04-07T16:34:48.975007+0000 mgr.y (mgr.25175) 584 : cluster [DBG] pgmap v1154: 137 pgs: 3 creating+peering, 29 unknown, 105 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail 2024-04-07T16:34:50.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:34:50 smithi144 bash[66654]: cluster 2024-04-07T16:34:49.492710+0000 mon.a (mon.0) 1841 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T16:34:50.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:34:50 smithi144 bash[66654]: audit 2024-04-07T16:34:49.502149+0000 mon.a (mon.0) 1842 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111196-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:34:50.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:34:50 smithi144 bash[66654]: cluster 2024-04-07T16:34:49.512577+0000 mon.a (mon.0) 1843 : cluster [DBG] osdmap e996: 8 total, 8 up, 8 in 2024-04-07T16:34:50.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:34:50 smithi100 bash[74942]: cluster 2024-04-07T16:34:49.492710+0000 mon.a (mon.0) 1841 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T16:34:50.941 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:34:50 smithi100 bash[74942]: audit 2024-04-07T16:34:49.502149+0000 mon.a (mon.0) 1842 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111196-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:34:50.941 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:34:50 smithi100 bash[74942]: cluster 2024-04-07T16:34:49.512577+0000 mon.a (mon.0) 1843 : cluster [DBG] osdmap e996: 8 total, 8 up, 8 in 2024-04-07T16:34:50.941 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:34:50 smithi100 bash[76202]: cluster 2024-04-07T16:34:49.492710+0000 mon.a (mon.0) 1841 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T16:34:50.941 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:34:50 smithi100 bash[76202]: audit 2024-04-07T16:34:49.502149+0000 mon.a (mon.0) 1842 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111196-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:34:50.941 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:34:50 smithi100 bash[76202]: cluster 2024-04-07T16:34:49.512577+0000 mon.a (mon.0) 1843 : cluster [DBG] osdmap e996: 8 total, 8 up, 8 in 2024-04-07T16:34:51.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:34:51 smithi144 bash[66654]: cluster 2024-04-07T16:34:50.522895+0000 mon.a (mon.0) 1844 : cluster [DBG] osdmap e997: 8 total, 8 up, 8 in 2024-04-07T16:34:51.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:34:51 smithi144 bash[66654]: cluster 2024-04-07T16:34:50.975723+0000 mgr.y (mgr.25175) 585 : cluster [DBG] pgmap v1157: 137 pgs: 11 creating+peering, 13 unknown, 113 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail 2024-04-07T16:34:51.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:34:51 smithi100 bash[74942]: cluster 2024-04-07T16:34:50.522895+0000 mon.a (mon.0) 1844 : cluster [DBG] osdmap e997: 8 total, 8 up, 8 in 2024-04-07T16:34:51.941 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:34:51 smithi100 bash[74942]: cluster 2024-04-07T16:34:50.975723+0000 mgr.y (mgr.25175) 585 : cluster [DBG] pgmap v1157: 137 pgs: 11 creating+peering, 13 unknown, 113 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail 2024-04-07T16:34:51.941 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:34:51 smithi100 bash[76202]: cluster 2024-04-07T16:34:50.522895+0000 mon.a (mon.0) 1844 : cluster [DBG] osdmap e997: 8 total, 8 up, 8 in 2024-04-07T16:34:51.941 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:34:51 smithi100 bash[76202]: cluster 2024-04-07T16:34:50.975723+0000 mgr.y (mgr.25175) 585 : cluster [DBG] pgmap v1157: 137 pgs: 11 creating+peering, 13 unknown, 113 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail 2024-04-07T16:34:52.607 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:34:52 smithi144 bash[95233]: ts=2024-04-07T16:34:52.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:34:52.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:34:52 smithi144 bash[66654]: cluster 2024-04-07T16:34:51.516029+0000 mon.a (mon.0) 1845 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T16:34:52.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:34:52 smithi144 bash[66654]: cluster 2024-04-07T16:34:51.516108+0000 mon.a (mon.0) 1846 : cluster [INF] Cluster is now healthy 2024-04-07T16:34:52.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:34:52 smithi100 bash[74942]: cluster 2024-04-07T16:34:51.516029+0000 mon.a (mon.0) 1845 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T16:34:52.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:34:52 smithi100 bash[74942]: cluster 2024-04-07T16:34:51.516108+0000 mon.a (mon.0) 1846 : cluster [INF] Cluster is now healthy 2024-04-07T16:34:52.941 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:34:52 smithi100 bash[76202]: cluster 2024-04-07T16:34:51.516029+0000 mon.a (mon.0) 1845 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T16:34:52.941 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:34:52 smithi100 bash[76202]: cluster 2024-04-07T16:34:51.516108+0000 mon.a (mon.0) 1846 : cluster [INF] Cluster is now healthy 2024-04-07T16:34:53.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:34:53 smithi144 bash[66654]: cluster 2024-04-07T16:34:52.976888+0000 mgr.y (mgr.25175) 586 : cluster [DBG] pgmap v1158: 137 pgs: 11 creating+peering, 126 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail 2024-04-07T16:34:53.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:34:53 smithi100 bash[74942]: cluster 2024-04-07T16:34:52.976888+0000 mgr.y (mgr.25175) 586 : cluster [DBG] pgmap v1158: 137 pgs: 11 creating+peering, 126 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail 2024-04-07T16:34:53.941 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:34:53 smithi100 bash[76202]: cluster 2024-04-07T16:34:52.976888+0000 mgr.y (mgr.25175) 586 : cluster [DBG] pgmap v1158: 137 pgs: 11 creating+peering, 126 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail 2024-04-07T16:34:54.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:34:54 smithi144 bash[66654]: cluster 2024-04-07T16:34:53.551224+0000 mon.a (mon.0) 1847 : cluster [DBG] osdmap e998: 8 total, 8 up, 8 in 2024-04-07T16:34:54.940 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:34:54 smithi100 bash[76202]: cluster 2024-04-07T16:34:53.551224+0000 mon.a (mon.0) 1847 : cluster [DBG] osdmap e998: 8 total, 8 up, 8 in 2024-04-07T16:34:54.941 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:34:54 smithi100 bash[74942]: cluster 2024-04-07T16:34:53.551224+0000 mon.a (mon.0) 1847 : cluster [DBG] osdmap e998: 8 total, 8 up, 8 in 2024-04-07T16:34:55.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:34:55 smithi144 bash[66654]: cluster 2024-04-07T16:34:54.548949+0000 mon.a (mon.0) 1848 : cluster [DBG] osdmap e999: 8 total, 8 up, 8 in 2024-04-07T16:34:55.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:34:55 smithi144 bash[66654]: cluster 2024-04-07T16:34:54.977571+0000 mgr.y (mgr.25175) 587 : cluster [DBG] pgmap v1161: 137 pgs: 137 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 187 B/s wr, 0 op/s 2024-04-07T16:34:55.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:34:55 smithi144 bash[66654]: cluster 2024-04-07T16:34:55.550389+0000 mon.a (mon.0) 1849 : cluster [DBG] osdmap e1000: 8 total, 8 up, 8 in 2024-04-07T16:34:55.941 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:34:55 smithi100 bash[74942]: cluster 2024-04-07T16:34:54.548949+0000 mon.a (mon.0) 1848 : cluster [DBG] osdmap e999: 8 total, 8 up, 8 in 2024-04-07T16:34:55.941 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:34:55 smithi100 bash[74942]: cluster 2024-04-07T16:34:54.977571+0000 mgr.y (mgr.25175) 587 : cluster [DBG] pgmap v1161: 137 pgs: 137 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 187 B/s wr, 0 op/s 2024-04-07T16:34:55.941 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:34:55 smithi100 bash[74942]: cluster 2024-04-07T16:34:55.550389+0000 mon.a (mon.0) 1849 : cluster [DBG] osdmap e1000: 8 total, 8 up, 8 in 2024-04-07T16:34:55.941 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:34:55 smithi100 bash[76202]: cluster 2024-04-07T16:34:54.548949+0000 mon.a (mon.0) 1848 : cluster [DBG] osdmap e999: 8 total, 8 up, 8 in 2024-04-07T16:34:55.941 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:34:55 smithi100 bash[76202]: cluster 2024-04-07T16:34:54.977571+0000 mgr.y (mgr.25175) 587 : cluster [DBG] pgmap v1161: 137 pgs: 137 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 187 B/s wr, 0 op/s 2024-04-07T16:34:55.941 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:34:55 smithi100 bash[76202]: cluster 2024-04-07T16:34:55.550389+0000 mon.a (mon.0) 1849 : cluster [DBG] osdmap e1000: 8 total, 8 up, 8 in 2024-04-07T16:34:56.547 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] cls_rgw.test_put_snap (9055 ms) 2024-04-07T16:34:56.547 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] cls_rgw.test_explicit 2024-04-07T16:34:57.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:34:57 smithi144 bash[66654]: cluster 2024-04-07T16:34:56.553187+0000 mon.a (mon.0) 1850 : cluster [DBG] osdmap e1001: 8 total, 8 up, 8 in 2024-04-07T16:34:57.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:34:57 smithi144 bash[66654]: cluster 2024-04-07T16:34:56.978439+0000 mgr.y (mgr.25175) 588 : cluster [DBG] pgmap v1164: 105 pgs: 105 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail 2024-04-07T16:34:57.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:34:57 smithi100 bash[74942]: cluster 2024-04-07T16:34:56.553187+0000 mon.a (mon.0) 1850 : cluster [DBG] osdmap e1001: 8 total, 8 up, 8 in 2024-04-07T16:34:57.941 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:34:57 smithi100 bash[74942]: cluster 2024-04-07T16:34:56.978439+0000 mgr.y (mgr.25175) 588 : cluster [DBG] pgmap v1164: 105 pgs: 105 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail 2024-04-07T16:34:57.941 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:34:57 smithi100 bash[76202]: cluster 2024-04-07T16:34:56.553187+0000 mon.a (mon.0) 1850 : cluster [DBG] osdmap e1001: 8 total, 8 up, 8 in 2024-04-07T16:34:57.941 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:34:57 smithi100 bash[76202]: cluster 2024-04-07T16:34:56.978439+0000 mgr.y (mgr.25175) 588 : cluster [DBG] pgmap v1164: 105 pgs: 105 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail 2024-04-07T16:34:58.690 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:34:58 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:34:58] "GET /metrics HTTP/1.1" 200 36354 "" "Prometheus/2.43.0" 2024-04-07T16:34:58.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:34:58 smithi100 bash[74942]: cluster 2024-04-07T16:34:57.566414+0000 mon.a (mon.0) 1851 : cluster [DBG] osdmap e1002: 8 total, 8 up, 8 in 2024-04-07T16:34:58.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:34:58 smithi100 bash[74942]: audit 2024-04-07T16:34:57.572843+0000 mon.c (mon.2) 52 : audit [INF] from='client.? 172.21.15.100:0/14747841' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111196-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:34:58.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:34:58 smithi100 bash[74942]: audit 2024-04-07T16:34:57.573819+0000 mon.a (mon.0) 1852 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111196-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:34:58.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:34:58 smithi100 bash[76202]: cluster 2024-04-07T16:34:57.566414+0000 mon.a (mon.0) 1851 : cluster [DBG] osdmap e1002: 8 total, 8 up, 8 in 2024-04-07T16:34:58.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:34:58 smithi100 bash[76202]: audit 2024-04-07T16:34:57.572843+0000 mon.c (mon.2) 52 : audit [INF] from='client.? 172.21.15.100:0/14747841' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111196-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:34:58.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:34:58 smithi100 bash[76202]: audit 2024-04-07T16:34:57.573819+0000 mon.a (mon.0) 1852 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111196-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:34:58.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:34:58 smithi144 bash[66654]: cluster 2024-04-07T16:34:57.566414+0000 mon.a (mon.0) 1851 : cluster [DBG] osdmap e1002: 8 total, 8 up, 8 in 2024-04-07T16:34:58.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:34:58 smithi144 bash[66654]: audit 2024-04-07T16:34:57.572843+0000 mon.c (mon.2) 52 : audit [INF] from='client.? 172.21.15.100:0/14747841' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111196-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:34:58.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:34:58 smithi144 bash[66654]: audit 2024-04-07T16:34:57.573819+0000 mon.a (mon.0) 1852 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111196-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:34:59.591 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] cls_rgw.test_explicit (3046 ms) 2024-04-07T16:34:59.591 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] cls_rgw.set 2024-04-07T16:34:59.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:34:59 smithi144 bash[66654]: audit 2024-04-07T16:34:58.562305+0000 mon.a (mon.0) 1853 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111196-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:34:59.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:34:59 smithi144 bash[66654]: cluster 2024-04-07T16:34:58.573113+0000 mon.a (mon.0) 1854 : cluster [DBG] osdmap e1003: 8 total, 8 up, 8 in 2024-04-07T16:34:59.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:34:59 smithi144 bash[66654]: cluster 2024-04-07T16:34:58.979303+0000 mgr.y (mgr.25175) 589 : cluster [DBG] pgmap v1167: 137 pgs: 2 creating+peering, 24 unknown, 111 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail 2024-04-07T16:34:59.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:34:59 smithi100 bash[74942]: audit 2024-04-07T16:34:58.562305+0000 mon.a (mon.0) 1853 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111196-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:34:59.941 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:34:59 smithi100 bash[74942]: cluster 2024-04-07T16:34:58.573113+0000 mon.a (mon.0) 1854 : cluster [DBG] osdmap e1003: 8 total, 8 up, 8 in 2024-04-07T16:34:59.941 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:34:59 smithi100 bash[74942]: cluster 2024-04-07T16:34:58.979303+0000 mgr.y (mgr.25175) 589 : cluster [DBG] pgmap v1167: 137 pgs: 2 creating+peering, 24 unknown, 111 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail 2024-04-07T16:34:59.941 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:34:59 smithi100 bash[76202]: audit 2024-04-07T16:34:58.562305+0000 mon.a (mon.0) 1853 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111196-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:34:59.941 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:34:59 smithi100 bash[76202]: cluster 2024-04-07T16:34:58.573113+0000 mon.a (mon.0) 1854 : cluster [DBG] osdmap e1003: 8 total, 8 up, 8 in 2024-04-07T16:34:59.941 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:34:59 smithi100 bash[76202]: cluster 2024-04-07T16:34:58.979303+0000 mgr.y (mgr.25175) 589 : cluster [DBG] pgmap v1167: 137 pgs: 2 creating+peering, 24 unknown, 111 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail 2024-04-07T16:35:00.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:35:00 smithi144 bash[66654]: cluster 2024-04-07T16:34:59.597264+0000 mon.a (mon.0) 1855 : cluster [DBG] osdmap e1004: 8 total, 8 up, 8 in 2024-04-07T16:35:00.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:35:00 smithi100 bash[74942]: cluster 2024-04-07T16:34:59.597264+0000 mon.a (mon.0) 1855 : cluster [DBG] osdmap e1004: 8 total, 8 up, 8 in 2024-04-07T16:35:00.941 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:35:00 smithi100 bash[76202]: cluster 2024-04-07T16:34:59.597264+0000 mon.a (mon.0) 1855 : cluster [DBG] osdmap e1004: 8 total, 8 up, 8 in 2024-04-07T16:35:01.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:35:01 smithi144 bash[66654]: cluster 2024-04-07T16:35:00.609993+0000 mon.a (mon.0) 1856 : cluster [DBG] osdmap e1005: 8 total, 8 up, 8 in 2024-04-07T16:35:01.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:35:01 smithi144 bash[66654]: audit 2024-04-07T16:35:00.615727+0000 mon.c (mon.2) 53 : audit [INF] from='client.? 172.21.15.100:0/1777123637' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111196-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:35:01.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:35:01 smithi144 bash[66654]: audit 2024-04-07T16:35:00.618811+0000 mon.a (mon.0) 1857 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111196-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:35:01.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:35:01 smithi144 bash[66654]: cluster 2024-04-07T16:35:00.980085+0000 mgr.y (mgr.25175) 590 : cluster [DBG] pgmap v1170: 137 pgs: 32 unknown, 105 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail 2024-04-07T16:35:01.941 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:35:01 smithi100 bash[74942]: cluster 2024-04-07T16:35:00.609993+0000 mon.a (mon.0) 1856 : cluster [DBG] osdmap e1005: 8 total, 8 up, 8 in 2024-04-07T16:35:01.941 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:35:01 smithi100 bash[74942]: audit 2024-04-07T16:35:00.615727+0000 mon.c (mon.2) 53 : audit [INF] from='client.? 172.21.15.100:0/1777123637' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111196-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:35:01.941 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:35:01 smithi100 bash[74942]: audit 2024-04-07T16:35:00.618811+0000 mon.a (mon.0) 1857 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111196-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:35:01.941 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:35:01 smithi100 bash[74942]: cluster 2024-04-07T16:35:00.980085+0000 mgr.y (mgr.25175) 590 : cluster [DBG] pgmap v1170: 137 pgs: 32 unknown, 105 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail 2024-04-07T16:35:01.941 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:35:01 smithi100 bash[76202]: cluster 2024-04-07T16:35:00.609993+0000 mon.a (mon.0) 1856 : cluster [DBG] osdmap e1005: 8 total, 8 up, 8 in 2024-04-07T16:35:01.941 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:35:01 smithi100 bash[76202]: audit 2024-04-07T16:35:00.615727+0000 mon.c (mon.2) 53 : audit [INF] from='client.? 172.21.15.100:0/1777123637' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111196-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:35:01.941 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:35:01 smithi100 bash[76202]: audit 2024-04-07T16:35:00.618811+0000 mon.a (mon.0) 1857 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111196-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:35:01.941 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:35:01 smithi100 bash[76202]: cluster 2024-04-07T16:35:00.980085+0000 mgr.y (mgr.25175) 590 : cluster [DBG] pgmap v1170: 137 pgs: 32 unknown, 105 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail 2024-04-07T16:35:02.607 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:35:02 smithi144 bash[95233]: ts=2024-04-07T16:35:02.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:35:02.623 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] cls_rgw.set (3031 ms) 2024-04-07T16:35:02.623 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] cls_rgw.test_implicit_ec 2024-04-07T16:35:02.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:35:02 smithi100 bash[74942]: cluster 2024-04-07T16:35:01.602901+0000 mon.a (mon.0) 1858 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T16:35:02.941 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:35:02 smithi100 bash[74942]: audit 2024-04-07T16:35:01.614472+0000 mon.a (mon.0) 1859 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111196-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:35:02.941 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:35:02 smithi100 bash[74942]: cluster 2024-04-07T16:35:01.631209+0000 mon.a (mon.0) 1860 : cluster [DBG] osdmap e1006: 8 total, 8 up, 8 in 2024-04-07T16:35:02.941 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:35:02 smithi100 bash[74942]: audit 2024-04-07T16:35:01.754383+0000 mon.a (mon.0) 1861 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:35:02.941 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:35:02 smithi100 bash[74942]: audit 2024-04-07T16:35:01.756716+0000 mon.a (mon.0) 1862 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:35:02.941 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:35:02 smithi100 bash[76202]: cluster 2024-04-07T16:35:01.602901+0000 mon.a (mon.0) 1858 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T16:35:02.941 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:35:02 smithi100 bash[76202]: audit 2024-04-07T16:35:01.614472+0000 mon.a (mon.0) 1859 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111196-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:35:02.941 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:35:02 smithi100 bash[76202]: cluster 2024-04-07T16:35:01.631209+0000 mon.a (mon.0) 1860 : cluster [DBG] osdmap e1006: 8 total, 8 up, 8 in 2024-04-07T16:35:02.941 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:35:02 smithi100 bash[76202]: audit 2024-04-07T16:35:01.754383+0000 mon.a (mon.0) 1861 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:35:02.941 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:35:02 smithi100 bash[76202]: audit 2024-04-07T16:35:01.756716+0000 mon.a (mon.0) 1862 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:35:03.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:35:02 smithi144 bash[66654]: cluster 2024-04-07T16:35:01.602901+0000 mon.a (mon.0) 1858 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T16:35:03.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:35:02 smithi144 bash[66654]: audit 2024-04-07T16:35:01.614472+0000 mon.a (mon.0) 1859 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111196-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:35:03.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:35:02 smithi144 bash[66654]: cluster 2024-04-07T16:35:01.631209+0000 mon.a (mon.0) 1860 : cluster [DBG] osdmap e1006: 8 total, 8 up, 8 in 2024-04-07T16:35:03.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:35:02 smithi144 bash[66654]: audit 2024-04-07T16:35:01.754383+0000 mon.a (mon.0) 1861 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:35:03.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:35:02 smithi144 bash[66654]: audit 2024-04-07T16:35:01.756716+0000 mon.a (mon.0) 1862 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:35:03.941 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:35:03 smithi100 bash[74942]: cluster 2024-04-07T16:35:02.625571+0000 mon.a (mon.0) 1863 : cluster [DBG] osdmap e1007: 8 total, 8 up, 8 in 2024-04-07T16:35:03.941 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:35:03 smithi100 bash[74942]: audit 2024-04-07T16:35:02.670856+0000 mon.a (mon.0) 1864 : audit [INF] from='client.? 172.21.15.100:0/1975575884' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi100-111196-6"}]: dispatch 2024-04-07T16:35:03.941 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:35:03 smithi100 bash[74942]: audit 2024-04-07T16:35:02.671882+0000 mon.a (mon.0) 1865 : audit [INF] from='client.? 172.21.15.100:0/1975575884' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi100-111196-6"}]: dispatch 2024-04-07T16:35:03.941 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:35:03 smithi100 bash[74942]: audit 2024-04-07T16:35:02.672328+0000 mon.a (mon.0) 1866 : audit [INF] from='client.? 172.21.15.100:0/1975575884' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi100-111196-6", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-07T16:35:03.941 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:35:03 smithi100 bash[74942]: cluster 2024-04-07T16:35:02.980757+0000 mgr.y (mgr.25175) 591 : cluster [DBG] pgmap v1173: 105 pgs: 105 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail 2024-04-07T16:35:03.941 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:35:03 smithi100 bash[76202]: cluster 2024-04-07T16:35:02.625571+0000 mon.a (mon.0) 1863 : cluster [DBG] osdmap e1007: 8 total, 8 up, 8 in 2024-04-07T16:35:03.941 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:35:03 smithi100 bash[76202]: audit 2024-04-07T16:35:02.670856+0000 mon.a (mon.0) 1864 : audit [INF] from='client.? 172.21.15.100:0/1975575884' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi100-111196-6"}]: dispatch 2024-04-07T16:35:03.941 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:35:03 smithi100 bash[76202]: audit 2024-04-07T16:35:02.671882+0000 mon.a (mon.0) 1865 : audit [INF] from='client.? 172.21.15.100:0/1975575884' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi100-111196-6"}]: dispatch 2024-04-07T16:35:03.941 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:35:03 smithi100 bash[76202]: audit 2024-04-07T16:35:02.672328+0000 mon.a (mon.0) 1866 : audit [INF] from='client.? 172.21.15.100:0/1975575884' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi100-111196-6", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-07T16:35:03.941 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:35:03 smithi100 bash[76202]: cluster 2024-04-07T16:35:02.980757+0000 mgr.y (mgr.25175) 591 : cluster [DBG] pgmap v1173: 105 pgs: 105 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail 2024-04-07T16:35:04.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:35:03 smithi144 bash[66654]: cluster 2024-04-07T16:35:02.625571+0000 mon.a (mon.0) 1863 : cluster [DBG] osdmap e1007: 8 total, 8 up, 8 in 2024-04-07T16:35:04.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:35:03 smithi144 bash[66654]: audit 2024-04-07T16:35:02.670856+0000 mon.a (mon.0) 1864 : audit [INF] from='client.? 172.21.15.100:0/1975575884' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi100-111196-6"}]: dispatch 2024-04-07T16:35:04.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:35:03 smithi144 bash[66654]: audit 2024-04-07T16:35:02.671882+0000 mon.a (mon.0) 1865 : audit [INF] from='client.? 172.21.15.100:0/1975575884' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi100-111196-6"}]: dispatch 2024-04-07T16:35:04.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:35:03 smithi144 bash[66654]: audit 2024-04-07T16:35:02.672328+0000 mon.a (mon.0) 1866 : audit [INF] from='client.? 172.21.15.100:0/1975575884' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi100-111196-6", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-07T16:35:04.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:35:03 smithi144 bash[66654]: cluster 2024-04-07T16:35:02.980757+0000 mgr.y (mgr.25175) 591 : cluster [DBG] pgmap v1173: 105 pgs: 105 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail 2024-04-07T16:35:04.941 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:35:04 smithi100 bash[74942]: cluster 2024-04-07T16:35:03.632325+0000 mon.a (mon.0) 1867 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T16:35:04.941 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:35:04 smithi100 bash[74942]: cluster 2024-04-07T16:35:03.632392+0000 mon.a (mon.0) 1868 : cluster [INF] Cluster is now healthy 2024-04-07T16:35:04.941 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:35:04 smithi100 bash[74942]: audit 2024-04-07T16:35:03.641943+0000 mon.a (mon.0) 1869 : audit [INF] from='client.? 172.21.15.100:0/1975575884' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi100-111196-6", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-07T16:35:04.941 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:35:04 smithi100 bash[74942]: cluster 2024-04-07T16:35:03.651378+0000 mon.a (mon.0) 1870 : cluster [DBG] osdmap e1008: 8 total, 8 up, 8 in 2024-04-07T16:35:04.941 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:35:04 smithi100 bash[74942]: audit 2024-04-07T16:35:03.653348+0000 mon.a (mon.0) 1871 : audit [INF] from='client.? 172.21.15.100:0/1975575884' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi100-111196-6", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi100-111196-6"}]: dispatch 2024-04-07T16:35:04.941 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:35:04 smithi100 bash[76202]: cluster 2024-04-07T16:35:03.632325+0000 mon.a (mon.0) 1867 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T16:35:04.941 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:35:04 smithi100 bash[76202]: cluster 2024-04-07T16:35:03.632392+0000 mon.a (mon.0) 1868 : cluster [INF] Cluster is now healthy 2024-04-07T16:35:04.942 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:35:04 smithi100 bash[76202]: audit 2024-04-07T16:35:03.641943+0000 mon.a (mon.0) 1869 : audit [INF] from='client.? 172.21.15.100:0/1975575884' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi100-111196-6", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-07T16:35:04.942 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:35:04 smithi100 bash[76202]: cluster 2024-04-07T16:35:03.651378+0000 mon.a (mon.0) 1870 : cluster [DBG] osdmap e1008: 8 total, 8 up, 8 in 2024-04-07T16:35:04.942 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:35:04 smithi100 bash[76202]: audit 2024-04-07T16:35:03.653348+0000 mon.a (mon.0) 1871 : audit [INF] from='client.? 172.21.15.100:0/1975575884' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi100-111196-6", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi100-111196-6"}]: dispatch 2024-04-07T16:35:05.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:35:04 smithi144 bash[66654]: cluster 2024-04-07T16:35:03.632325+0000 mon.a (mon.0) 1867 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T16:35:05.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:35:04 smithi144 bash[66654]: cluster 2024-04-07T16:35:03.632392+0000 mon.a (mon.0) 1868 : cluster [INF] Cluster is now healthy 2024-04-07T16:35:05.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:35:04 smithi144 bash[66654]: audit 2024-04-07T16:35:03.641943+0000 mon.a (mon.0) 1869 : audit [INF] from='client.? 172.21.15.100:0/1975575884' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi100-111196-6", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-07T16:35:05.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:35:04 smithi144 bash[66654]: cluster 2024-04-07T16:35:03.651378+0000 mon.a (mon.0) 1870 : cluster [DBG] osdmap e1008: 8 total, 8 up, 8 in 2024-04-07T16:35:05.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:35:04 smithi144 bash[66654]: audit 2024-04-07T16:35:03.653348+0000 mon.a (mon.0) 1871 : audit [INF] from='client.? 172.21.15.100:0/1975575884' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi100-111196-6", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi100-111196-6"}]: dispatch 2024-04-07T16:35:05.941 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:35:05 smithi100 bash[74942]: cluster 2024-04-07T16:35:04.674826+0000 mon.a (mon.0) 1872 : cluster [DBG] osdmap e1009: 8 total, 8 up, 8 in 2024-04-07T16:35:05.941 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:35:05 smithi100 bash[74942]: cluster 2024-04-07T16:35:04.981546+0000 mgr.y (mgr.25175) 592 : cluster [DBG] pgmap v1176: 105 pgs: 105 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail 2024-04-07T16:35:05.941 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:35:05 smithi100 bash[76202]: cluster 2024-04-07T16:35:04.674826+0000 mon.a (mon.0) 1872 : cluster [DBG] osdmap e1009: 8 total, 8 up, 8 in 2024-04-07T16:35:05.941 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:35:05 smithi100 bash[76202]: cluster 2024-04-07T16:35:04.981546+0000 mgr.y (mgr.25175) 592 : cluster [DBG] pgmap v1176: 105 pgs: 105 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail 2024-04-07T16:35:06.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:35:05 smithi144 bash[66654]: cluster 2024-04-07T16:35:04.674826+0000 mon.a (mon.0) 1872 : cluster [DBG] osdmap e1009: 8 total, 8 up, 8 in 2024-04-07T16:35:06.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:35:05 smithi144 bash[66654]: cluster 2024-04-07T16:35:04.981546+0000 mgr.y (mgr.25175) 592 : cluster [DBG] pgmap v1176: 105 pgs: 105 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail 2024-04-07T16:35:06.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:35:06 smithi100 bash[74942]: audit 2024-04-07T16:35:05.670771+0000 mon.a (mon.0) 1873 : audit [INF] from='client.? 172.21.15.100:0/1975575884' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi100-111196-6", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi100-111196-6"}]': finished 2024-04-07T16:35:06.941 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:35:06 smithi100 bash[74942]: cluster 2024-04-07T16:35:05.680977+0000 mon.a (mon.0) 1874 : cluster [DBG] osdmap e1010: 8 total, 8 up, 8 in 2024-04-07T16:35:06.941 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:35:06 smithi100 bash[76202]: audit 2024-04-07T16:35:05.670771+0000 mon.a (mon.0) 1873 : audit [INF] from='client.? 172.21.15.100:0/1975575884' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi100-111196-6", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi100-111196-6"}]': finished 2024-04-07T16:35:06.941 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:35:06 smithi100 bash[76202]: cluster 2024-04-07T16:35:05.680977+0000 mon.a (mon.0) 1874 : cluster [DBG] osdmap e1010: 8 total, 8 up, 8 in 2024-04-07T16:35:07.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:35:06 smithi144 bash[66654]: audit 2024-04-07T16:35:05.670771+0000 mon.a (mon.0) 1873 : audit [INF] from='client.? 172.21.15.100:0/1975575884' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi100-111196-6", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi100-111196-6"}]': finished 2024-04-07T16:35:07.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:35:06 smithi144 bash[66654]: cluster 2024-04-07T16:35:05.680977+0000 mon.a (mon.0) 1874 : cluster [DBG] osdmap e1010: 8 total, 8 up, 8 in 2024-04-07T16:35:08.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:35:07 smithi144 bash[66654]: cluster 2024-04-07T16:35:06.680897+0000 mon.a (mon.0) 1875 : cluster [DBG] osdmap e1011: 8 total, 8 up, 8 in 2024-04-07T16:35:08.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:35:07 smithi144 bash[66654]: cluster 2024-04-07T16:35:06.982308+0000 mgr.y (mgr.25175) 593 : cluster [DBG] pgmap v1179: 113 pgs: 8 unknown, 105 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail 2024-04-07T16:35:08.190 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:35:07 smithi100 bash[74942]: cluster 2024-04-07T16:35:06.680897+0000 mon.a (mon.0) 1875 : cluster [DBG] osdmap e1011: 8 total, 8 up, 8 in 2024-04-07T16:35:08.191 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:35:07 smithi100 bash[74942]: cluster 2024-04-07T16:35:06.982308+0000 mgr.y (mgr.25175) 593 : cluster [DBG] pgmap v1179: 113 pgs: 8 unknown, 105 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail 2024-04-07T16:35:08.191 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:35:07 smithi100 bash[76202]: cluster 2024-04-07T16:35:06.680897+0000 mon.a (mon.0) 1875 : cluster [DBG] osdmap e1011: 8 total, 8 up, 8 in 2024-04-07T16:35:08.191 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:35:07 smithi100 bash[76202]: cluster 2024-04-07T16:35:06.982308+0000 mgr.y (mgr.25175) 593 : cluster [DBG] pgmap v1179: 113 pgs: 8 unknown, 105 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail 2024-04-07T16:35:08.690 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:35:08 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:35:08] "GET /metrics HTTP/1.1" 200 36359 "" "Prometheus/2.43.0" 2024-04-07T16:35:09.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:35:08 smithi144 bash[66654]: cluster 2024-04-07T16:35:07.692493+0000 mon.a (mon.0) 1876 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T16:35:09.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:35:08 smithi144 bash[66654]: cluster 2024-04-07T16:35:07.708382+0000 mon.a (mon.0) 1877 : cluster [DBG] osdmap e1012: 8 total, 8 up, 8 in 2024-04-07T16:35:09.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:35:08 smithi144 bash[66654]: audit 2024-04-07T16:35:07.716607+0000 mon.a (mon.0) 1878 : audit [INF] from='client.? 172.21.15.100:0/1975575884' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi100-111196-6"}]: dispatch 2024-04-07T16:35:09.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:35:08 smithi144 bash[66654]: audit 2024-04-07T16:35:08.697058+0000 mon.a (mon.0) 1879 : audit [INF] from='client.? 172.21.15.100:0/1975575884' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi100-111196-6"}]': finished 2024-04-07T16:35:09.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:35:08 smithi144 bash[66654]: cluster 2024-04-07T16:35:08.705357+0000 mon.a (mon.0) 1880 : cluster [DBG] osdmap e1013: 8 total, 8 up, 8 in 2024-04-07T16:35:09.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:35:08 smithi144 bash[66654]: audit 2024-04-07T16:35:08.708087+0000 mon.a (mon.0) 1881 : audit [INF] from='client.? 172.21.15.100:0/1975575884' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi100-111196-6"}]: dispatch 2024-04-07T16:35:09.191 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:35:08 smithi100 bash[74942]: cluster 2024-04-07T16:35:07.692493+0000 mon.a (mon.0) 1876 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T16:35:09.191 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:35:08 smithi100 bash[74942]: cluster 2024-04-07T16:35:07.708382+0000 mon.a (mon.0) 1877 : cluster [DBG] osdmap e1012: 8 total, 8 up, 8 in 2024-04-07T16:35:09.191 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:35:08 smithi100 bash[74942]: audit 2024-04-07T16:35:07.716607+0000 mon.a (mon.0) 1878 : audit [INF] from='client.? 172.21.15.100:0/1975575884' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi100-111196-6"}]: dispatch 2024-04-07T16:35:09.191 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:35:08 smithi100 bash[74942]: audit 2024-04-07T16:35:08.697058+0000 mon.a (mon.0) 1879 : audit [INF] from='client.? 172.21.15.100:0/1975575884' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi100-111196-6"}]': finished 2024-04-07T16:35:09.191 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:35:08 smithi100 bash[74942]: cluster 2024-04-07T16:35:08.705357+0000 mon.a (mon.0) 1880 : cluster [DBG] osdmap e1013: 8 total, 8 up, 8 in 2024-04-07T16:35:09.191 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:35:08 smithi100 bash[74942]: audit 2024-04-07T16:35:08.708087+0000 mon.a (mon.0) 1881 : audit [INF] from='client.? 172.21.15.100:0/1975575884' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi100-111196-6"}]: dispatch 2024-04-07T16:35:09.191 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:35:08 smithi100 bash[76202]: cluster 2024-04-07T16:35:07.692493+0000 mon.a (mon.0) 1876 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T16:35:09.191 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:35:08 smithi100 bash[76202]: cluster 2024-04-07T16:35:07.708382+0000 mon.a (mon.0) 1877 : cluster [DBG] osdmap e1012: 8 total, 8 up, 8 in 2024-04-07T16:35:09.191 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:35:08 smithi100 bash[76202]: audit 2024-04-07T16:35:07.716607+0000 mon.a (mon.0) 1878 : audit [INF] from='client.? 172.21.15.100:0/1975575884' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi100-111196-6"}]: dispatch 2024-04-07T16:35:09.191 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:35:08 smithi100 bash[76202]: audit 2024-04-07T16:35:08.697058+0000 mon.a (mon.0) 1879 : audit [INF] from='client.? 172.21.15.100:0/1975575884' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi100-111196-6"}]': finished 2024-04-07T16:35:09.192 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:35:08 smithi100 bash[76202]: cluster 2024-04-07T16:35:08.705357+0000 mon.a (mon.0) 1880 : cluster [DBG] osdmap e1013: 8 total, 8 up, 8 in 2024-04-07T16:35:09.192 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:35:08 smithi100 bash[76202]: audit 2024-04-07T16:35:08.708087+0000 mon.a (mon.0) 1881 : audit [INF] from='client.? 172.21.15.100:0/1975575884' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi100-111196-6"}]: dispatch 2024-04-07T16:35:09.701 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] cls_rgw.test_implicit_ec (7078 ms) 2024-04-07T16:35:09.701 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] cls_rgw.test_implicit_idempotent_ec 2024-04-07T16:35:10.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:35:09 smithi144 bash[66654]: cluster 2024-04-07T16:35:08.983129+0000 mgr.y (mgr.25175) 594 : cluster [DBG] pgmap v1182: 105 pgs: 105 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail 2024-04-07T16:35:10.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:35:09 smithi144 bash[66654]: audit 2024-04-07T16:35:09.700423+0000 mon.a (mon.0) 1882 : audit [INF] from='client.? 172.21.15.100:0/1975575884' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi100-111196-6"}]': finished 2024-04-07T16:35:10.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:35:09 smithi144 bash[66654]: cluster 2024-04-07T16:35:09.708937+0000 mon.a (mon.0) 1883 : cluster [DBG] osdmap e1014: 8 total, 8 up, 8 in 2024-04-07T16:35:10.191 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:35:09 smithi100 bash[74942]: cluster 2024-04-07T16:35:08.983129+0000 mgr.y (mgr.25175) 594 : cluster [DBG] pgmap v1182: 105 pgs: 105 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail 2024-04-07T16:35:10.191 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:35:09 smithi100 bash[74942]: audit 2024-04-07T16:35:09.700423+0000 mon.a (mon.0) 1882 : audit [INF] from='client.? 172.21.15.100:0/1975575884' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi100-111196-6"}]': finished 2024-04-07T16:35:10.191 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:35:09 smithi100 bash[74942]: cluster 2024-04-07T16:35:09.708937+0000 mon.a (mon.0) 1883 : cluster [DBG] osdmap e1014: 8 total, 8 up, 8 in 2024-04-07T16:35:10.191 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:35:09 smithi100 bash[76202]: cluster 2024-04-07T16:35:08.983129+0000 mgr.y (mgr.25175) 594 : cluster [DBG] pgmap v1182: 105 pgs: 105 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail 2024-04-07T16:35:10.191 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:35:09 smithi100 bash[76202]: audit 2024-04-07T16:35:09.700423+0000 mon.a (mon.0) 1882 : audit [INF] from='client.? 172.21.15.100:0/1975575884' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi100-111196-6"}]': finished 2024-04-07T16:35:10.191 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:35:09 smithi100 bash[76202]: cluster 2024-04-07T16:35:09.708937+0000 mon.a (mon.0) 1883 : cluster [DBG] osdmap e1014: 8 total, 8 up, 8 in 2024-04-07T16:35:11.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:35:10 smithi144 bash[66654]: cluster 2024-04-07T16:35:09.723635+0000 mon.a (mon.0) 1884 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T16:35:11.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:35:10 smithi144 bash[66654]: cluster 2024-04-07T16:35:09.723684+0000 mon.a (mon.0) 1885 : cluster [INF] Cluster is now healthy 2024-04-07T16:35:11.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:35:10 smithi144 bash[66654]: audit 2024-04-07T16:35:09.752001+0000 mon.a (mon.0) 1886 : audit [INF] from='client.? 172.21.15.100:0/1196843104' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi100-111196-7"}]: dispatch 2024-04-07T16:35:11.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:35:10 smithi144 bash[66654]: audit 2024-04-07T16:35:09.753569+0000 mon.a (mon.0) 1887 : audit [INF] from='client.? 172.21.15.100:0/1196843104' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi100-111196-7"}]: dispatch 2024-04-07T16:35:11.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:35:10 smithi144 bash[66654]: audit 2024-04-07T16:35:09.754048+0000 mon.a (mon.0) 1888 : audit [INF] from='client.? 172.21.15.100:0/1196843104' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi100-111196-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-07T16:35:11.191 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:35:10 smithi100 bash[74942]: cluster 2024-04-07T16:35:09.723635+0000 mon.a (mon.0) 1884 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T16:35:11.191 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:35:10 smithi100 bash[74942]: cluster 2024-04-07T16:35:09.723684+0000 mon.a (mon.0) 1885 : cluster [INF] Cluster is now healthy 2024-04-07T16:35:11.191 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:35:10 smithi100 bash[74942]: audit 2024-04-07T16:35:09.752001+0000 mon.a (mon.0) 1886 : audit [INF] from='client.? 172.21.15.100:0/1196843104' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi100-111196-7"}]: dispatch 2024-04-07T16:35:11.191 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:35:10 smithi100 bash[74942]: audit 2024-04-07T16:35:09.753569+0000 mon.a (mon.0) 1887 : audit [INF] from='client.? 172.21.15.100:0/1196843104' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi100-111196-7"}]: dispatch 2024-04-07T16:35:11.191 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:35:10 smithi100 bash[74942]: audit 2024-04-07T16:35:09.754048+0000 mon.a (mon.0) 1888 : audit [INF] from='client.? 172.21.15.100:0/1196843104' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi100-111196-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-07T16:35:11.191 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:35:10 smithi100 bash[76202]: cluster 2024-04-07T16:35:09.723635+0000 mon.a (mon.0) 1884 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T16:35:11.191 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:35:10 smithi100 bash[76202]: cluster 2024-04-07T16:35:09.723684+0000 mon.a (mon.0) 1885 : cluster [INF] Cluster is now healthy 2024-04-07T16:35:11.191 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:35:10 smithi100 bash[76202]: audit 2024-04-07T16:35:09.752001+0000 mon.a (mon.0) 1886 : audit [INF] from='client.? 172.21.15.100:0/1196843104' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi100-111196-7"}]: dispatch 2024-04-07T16:35:11.192 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:35:10 smithi100 bash[76202]: audit 2024-04-07T16:35:09.753569+0000 mon.a (mon.0) 1887 : audit [INF] from='client.? 172.21.15.100:0/1196843104' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi100-111196-7"}]: dispatch 2024-04-07T16:35:11.192 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:35:10 smithi100 bash[76202]: audit 2024-04-07T16:35:09.754048+0000 mon.a (mon.0) 1888 : audit [INF] from='client.? 172.21.15.100:0/1196843104' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi100-111196-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-07T16:35:12.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:35:11 smithi144 bash[66654]: audit 2024-04-07T16:35:10.735105+0000 mon.a (mon.0) 1889 : audit [INF] from='client.? 172.21.15.100:0/1196843104' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi100-111196-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-07T16:35:12.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:35:11 smithi144 bash[66654]: cluster 2024-04-07T16:35:10.742134+0000 mon.a (mon.0) 1890 : cluster [DBG] osdmap e1015: 8 total, 8 up, 8 in 2024-04-07T16:35:12.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:35:11 smithi144 bash[66654]: audit 2024-04-07T16:35:10.743544+0000 mon.a (mon.0) 1891 : audit [INF] from='client.? 172.21.15.100:0/1196843104' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi100-111196-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi100-111196-7"}]: dispatch 2024-04-07T16:35:12.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:35:11 smithi144 bash[66654]: cluster 2024-04-07T16:35:10.983843+0000 mgr.y (mgr.25175) 595 : cluster [DBG] pgmap v1185: 105 pgs: 105 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail 2024-04-07T16:35:12.191 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:35:11 smithi100 bash[74942]: audit 2024-04-07T16:35:10.735105+0000 mon.a (mon.0) 1889 : audit [INF] from='client.? 172.21.15.100:0/1196843104' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi100-111196-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-07T16:35:12.191 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:35:11 smithi100 bash[74942]: cluster 2024-04-07T16:35:10.742134+0000 mon.a (mon.0) 1890 : cluster [DBG] osdmap e1015: 8 total, 8 up, 8 in 2024-04-07T16:35:12.191 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:35:11 smithi100 bash[74942]: audit 2024-04-07T16:35:10.743544+0000 mon.a (mon.0) 1891 : audit [INF] from='client.? 172.21.15.100:0/1196843104' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi100-111196-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi100-111196-7"}]: dispatch 2024-04-07T16:35:12.193 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:35:11 smithi100 bash[74942]: cluster 2024-04-07T16:35:10.983843+0000 mgr.y (mgr.25175) 595 : cluster [DBG] pgmap v1185: 105 pgs: 105 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail 2024-04-07T16:35:12.193 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:35:11 smithi100 bash[76202]: audit 2024-04-07T16:35:10.735105+0000 mon.a (mon.0) 1889 : audit [INF] from='client.? 172.21.15.100:0/1196843104' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi100-111196-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-07T16:35:12.194 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:35:11 smithi100 bash[76202]: cluster 2024-04-07T16:35:10.742134+0000 mon.a (mon.0) 1890 : cluster [DBG] osdmap e1015: 8 total, 8 up, 8 in 2024-04-07T16:35:12.194 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:35:11 smithi100 bash[76202]: audit 2024-04-07T16:35:10.743544+0000 mon.a (mon.0) 1891 : audit [INF] from='client.? 172.21.15.100:0/1196843104' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi100-111196-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi100-111196-7"}]: dispatch 2024-04-07T16:35:12.194 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:35:11 smithi100 bash[76202]: cluster 2024-04-07T16:35:10.983843+0000 mgr.y (mgr.25175) 595 : cluster [DBG] pgmap v1185: 105 pgs: 105 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail 2024-04-07T16:35:12.607 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:35:12 smithi144 bash[95233]: ts=2024-04-07T16:35:12.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:35:13.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:35:12 smithi144 bash[66654]: cluster 2024-04-07T16:35:11.759127+0000 mon.a (mon.0) 1892 : cluster [DBG] osdmap e1016: 8 total, 8 up, 8 in 2024-04-07T16:35:13.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:35:12 smithi144 bash[66654]: audit 2024-04-07T16:35:12.741706+0000 mon.a (mon.0) 1893 : audit [INF] from='client.? 172.21.15.100:0/1196843104' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi100-111196-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi100-111196-7"}]': finished 2024-04-07T16:35:13.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:35:12 smithi144 bash[66654]: cluster 2024-04-07T16:35:12.752281+0000 mon.a (mon.0) 1894 : cluster [DBG] osdmap e1017: 8 total, 8 up, 8 in 2024-04-07T16:35:13.191 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:35:12 smithi100 bash[76202]: cluster 2024-04-07T16:35:11.759127+0000 mon.a (mon.0) 1892 : cluster [DBG] osdmap e1016: 8 total, 8 up, 8 in 2024-04-07T16:35:13.191 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:35:12 smithi100 bash[76202]: audit 2024-04-07T16:35:12.741706+0000 mon.a (mon.0) 1893 : audit [INF] from='client.? 172.21.15.100:0/1196843104' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi100-111196-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi100-111196-7"}]': finished 2024-04-07T16:35:13.191 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:35:12 smithi100 bash[76202]: cluster 2024-04-07T16:35:12.752281+0000 mon.a (mon.0) 1894 : cluster [DBG] osdmap e1017: 8 total, 8 up, 8 in 2024-04-07T16:35:13.191 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:35:12 smithi100 bash[74942]: cluster 2024-04-07T16:35:11.759127+0000 mon.a (mon.0) 1892 : cluster [DBG] osdmap e1016: 8 total, 8 up, 8 in 2024-04-07T16:35:13.191 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:35:12 smithi100 bash[74942]: audit 2024-04-07T16:35:12.741706+0000 mon.a (mon.0) 1893 : audit [INF] from='client.? 172.21.15.100:0/1196843104' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi100-111196-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi100-111196-7"}]': finished 2024-04-07T16:35:13.191 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:35:12 smithi100 bash[74942]: cluster 2024-04-07T16:35:12.752281+0000 mon.a (mon.0) 1894 : cluster [DBG] osdmap e1017: 8 total, 8 up, 8 in 2024-04-07T16:35:14.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:35:13 smithi144 bash[66654]: cluster 2024-04-07T16:35:12.984533+0000 mgr.y (mgr.25175) 596 : cluster [DBG] pgmap v1188: 113 pgs: 8 unknown, 105 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail 2024-04-07T16:35:14.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:35:13 smithi144 bash[66654]: cluster 2024-04-07T16:35:13.754561+0000 mon.a (mon.0) 1895 : cluster [DBG] osdmap e1018: 8 total, 8 up, 8 in 2024-04-07T16:35:14.191 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:35:13 smithi100 bash[74942]: cluster 2024-04-07T16:35:12.984533+0000 mgr.y (mgr.25175) 596 : cluster [DBG] pgmap v1188: 113 pgs: 8 unknown, 105 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail 2024-04-07T16:35:14.191 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:35:13 smithi100 bash[74942]: cluster 2024-04-07T16:35:13.754561+0000 mon.a (mon.0) 1895 : cluster [DBG] osdmap e1018: 8 total, 8 up, 8 in 2024-04-07T16:35:14.191 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:35:13 smithi100 bash[76202]: cluster 2024-04-07T16:35:12.984533+0000 mgr.y (mgr.25175) 596 : cluster [DBG] pgmap v1188: 113 pgs: 8 unknown, 105 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail 2024-04-07T16:35:14.191 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:35:13 smithi100 bash[76202]: cluster 2024-04-07T16:35:13.754561+0000 mon.a (mon.0) 1895 : cluster [DBG] osdmap e1018: 8 total, 8 up, 8 in 2024-04-07T16:35:15.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:35:14 smithi144 bash[66654]: cluster 2024-04-07T16:35:13.763123+0000 mon.a (mon.0) 1896 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T16:35:15.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:35:14 smithi144 bash[66654]: cluster 2024-04-07T16:35:14.781516+0000 mon.a (mon.0) 1897 : cluster [DBG] osdmap e1019: 8 total, 8 up, 8 in 2024-04-07T16:35:15.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:35:14 smithi144 bash[66654]: audit 2024-04-07T16:35:14.783101+0000 mon.a (mon.0) 1898 : audit [INF] from='client.? 172.21.15.100:0/1196843104' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi100-111196-7"}]: dispatch 2024-04-07T16:35:15.191 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:35:14 smithi100 bash[74942]: cluster 2024-04-07T16:35:13.763123+0000 mon.a (mon.0) 1896 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T16:35:15.191 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:35:14 smithi100 bash[74942]: cluster 2024-04-07T16:35:14.781516+0000 mon.a (mon.0) 1897 : cluster [DBG] osdmap e1019: 8 total, 8 up, 8 in 2024-04-07T16:35:15.191 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:35:14 smithi100 bash[74942]: audit 2024-04-07T16:35:14.783101+0000 mon.a (mon.0) 1898 : audit [INF] from='client.? 172.21.15.100:0/1196843104' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi100-111196-7"}]: dispatch 2024-04-07T16:35:15.191 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:35:14 smithi100 bash[76202]: cluster 2024-04-07T16:35:13.763123+0000 mon.a (mon.0) 1896 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T16:35:15.191 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:35:14 smithi100 bash[76202]: cluster 2024-04-07T16:35:14.781516+0000 mon.a (mon.0) 1897 : cluster [DBG] osdmap e1019: 8 total, 8 up, 8 in 2024-04-07T16:35:15.191 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:35:14 smithi100 bash[76202]: audit 2024-04-07T16:35:14.783101+0000 mon.a (mon.0) 1898 : audit [INF] from='client.? 172.21.15.100:0/1196843104' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi100-111196-7"}]: dispatch 2024-04-07T16:35:16.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:35:15 smithi144 bash[66654]: cluster 2024-04-07T16:35:14.985279+0000 mgr.y (mgr.25175) 597 : cluster [DBG] pgmap v1191: 105 pgs: 105 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail 2024-04-07T16:35:16.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:35:15 smithi144 bash[66654]: audit 2024-04-07T16:35:15.773196+0000 mon.a (mon.0) 1899 : audit [INF] from='client.? 172.21.15.100:0/1196843104' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi100-111196-7"}]': finished 2024-04-07T16:35:16.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:35:15 smithi144 bash[66654]: cluster 2024-04-07T16:35:15.780099+0000 mon.a (mon.0) 1900 : cluster [DBG] osdmap e1020: 8 total, 8 up, 8 in 2024-04-07T16:35:16.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:35:15 smithi144 bash[66654]: audit 2024-04-07T16:35:15.781489+0000 mon.a (mon.0) 1901 : audit [INF] from='client.? 172.21.15.100:0/1196843104' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi100-111196-7"}]: dispatch 2024-04-07T16:35:16.191 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:35:15 smithi100 bash[74942]: cluster 2024-04-07T16:35:14.985279+0000 mgr.y (mgr.25175) 597 : cluster [DBG] pgmap v1191: 105 pgs: 105 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail 2024-04-07T16:35:16.191 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:35:15 smithi100 bash[74942]: audit 2024-04-07T16:35:15.773196+0000 mon.a (mon.0) 1899 : audit [INF] from='client.? 172.21.15.100:0/1196843104' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi100-111196-7"}]': finished 2024-04-07T16:35:16.191 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:35:15 smithi100 bash[74942]: cluster 2024-04-07T16:35:15.780099+0000 mon.a (mon.0) 1900 : cluster [DBG] osdmap e1020: 8 total, 8 up, 8 in 2024-04-07T16:35:16.191 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:35:15 smithi100 bash[74942]: audit 2024-04-07T16:35:15.781489+0000 mon.a (mon.0) 1901 : audit [INF] from='client.? 172.21.15.100:0/1196843104' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi100-111196-7"}]: dispatch 2024-04-07T16:35:16.192 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:35:15 smithi100 bash[76202]: cluster 2024-04-07T16:35:14.985279+0000 mgr.y (mgr.25175) 597 : cluster [DBG] pgmap v1191: 105 pgs: 105 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail 2024-04-07T16:35:16.192 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:35:15 smithi100 bash[76202]: audit 2024-04-07T16:35:15.773196+0000 mon.a (mon.0) 1899 : audit [INF] from='client.? 172.21.15.100:0/1196843104' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi100-111196-7"}]': finished 2024-04-07T16:35:16.192 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:35:15 smithi100 bash[76202]: cluster 2024-04-07T16:35:15.780099+0000 mon.a (mon.0) 1900 : cluster [DBG] osdmap e1020: 8 total, 8 up, 8 in 2024-04-07T16:35:16.192 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:35:15 smithi100 bash[76202]: audit 2024-04-07T16:35:15.781489+0000 mon.a (mon.0) 1901 : audit [INF] from='client.? 172.21.15.100:0/1196843104' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi100-111196-7"}]: dispatch 2024-04-07T16:35:16.785 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] cls_rgw.test_implicit_idempotent_ec (7084 ms) 2024-04-07T16:35:16.785 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] cls_rgw.test_put_snap_ec 2024-04-07T16:35:17.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:35:16 smithi144 bash[66654]: cluster 2024-04-07T16:35:15.786002+0000 mon.a (mon.0) 1902 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T16:35:17.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:35:16 smithi144 bash[66654]: cluster 2024-04-07T16:35:15.786082+0000 mon.a (mon.0) 1903 : cluster [INF] Cluster is now healthy 2024-04-07T16:35:17.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:35:16 smithi144 bash[66654]: audit 2024-04-07T16:35:16.754617+0000 mon.a (mon.0) 1904 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:35:17.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:35:16 smithi144 bash[66654]: audit 2024-04-07T16:35:16.756839+0000 mon.a (mon.0) 1905 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:35:17.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:35:16 smithi144 bash[66654]: audit 2024-04-07T16:35:16.778208+0000 mon.a (mon.0) 1906 : audit [INF] from='client.? 172.21.15.100:0/1196843104' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi100-111196-7"}]': finished 2024-04-07T16:35:17.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:35:16 smithi144 bash[66654]: cluster 2024-04-07T16:35:16.793014+0000 mon.a (mon.0) 1907 : cluster [DBG] osdmap e1021: 8 total, 8 up, 8 in 2024-04-07T16:35:17.191 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:35:16 smithi100 bash[74942]: cluster 2024-04-07T16:35:15.786002+0000 mon.a (mon.0) 1902 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T16:35:17.191 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:35:16 smithi100 bash[74942]: cluster 2024-04-07T16:35:15.786082+0000 mon.a (mon.0) 1903 : cluster [INF] Cluster is now healthy 2024-04-07T16:35:17.191 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:35:16 smithi100 bash[74942]: audit 2024-04-07T16:35:16.754617+0000 mon.a (mon.0) 1904 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:35:17.191 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:35:16 smithi100 bash[74942]: audit 2024-04-07T16:35:16.756839+0000 mon.a (mon.0) 1905 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:35:17.191 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:35:16 smithi100 bash[74942]: audit 2024-04-07T16:35:16.778208+0000 mon.a (mon.0) 1906 : audit [INF] from='client.? 172.21.15.100:0/1196843104' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi100-111196-7"}]': finished 2024-04-07T16:35:17.191 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:35:16 smithi100 bash[74942]: cluster 2024-04-07T16:35:16.793014+0000 mon.a (mon.0) 1907 : cluster [DBG] osdmap e1021: 8 total, 8 up, 8 in 2024-04-07T16:35:17.191 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:35:16 smithi100 bash[76202]: cluster 2024-04-07T16:35:15.786002+0000 mon.a (mon.0) 1902 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T16:35:17.191 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:35:16 smithi100 bash[76202]: cluster 2024-04-07T16:35:15.786082+0000 mon.a (mon.0) 1903 : cluster [INF] Cluster is now healthy 2024-04-07T16:35:17.192 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:35:16 smithi100 bash[76202]: audit 2024-04-07T16:35:16.754617+0000 mon.a (mon.0) 1904 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:35:17.192 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:35:16 smithi100 bash[76202]: audit 2024-04-07T16:35:16.756839+0000 mon.a (mon.0) 1905 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:35:17.192 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:35:16 smithi100 bash[76202]: audit 2024-04-07T16:35:16.778208+0000 mon.a (mon.0) 1906 : audit [INF] from='client.? 172.21.15.100:0/1196843104' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi100-111196-7"}]': finished 2024-04-07T16:35:17.192 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:35:16 smithi100 bash[76202]: cluster 2024-04-07T16:35:16.793014+0000 mon.a (mon.0) 1907 : cluster [DBG] osdmap e1021: 8 total, 8 up, 8 in 2024-04-07T16:35:18.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:35:17 smithi144 bash[66654]: audit 2024-04-07T16:35:16.837247+0000 mon.a (mon.0) 1908 : audit [INF] from='client.? 172.21.15.100:0/2137139960' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi100-111196-8"}]: dispatch 2024-04-07T16:35:18.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:35:17 smithi144 bash[66654]: audit 2024-04-07T16:35:16.838172+0000 mon.a (mon.0) 1909 : audit [INF] from='client.? 172.21.15.100:0/2137139960' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi100-111196-8"}]: dispatch 2024-04-07T16:35:18.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:35:17 smithi144 bash[66654]: audit 2024-04-07T16:35:16.838627+0000 mon.a (mon.0) 1910 : audit [INF] from='client.? 172.21.15.100:0/2137139960' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi100-111196-8", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-07T16:35:18.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:35:17 smithi144 bash[66654]: cluster 2024-04-07T16:35:16.985986+0000 mgr.y (mgr.25175) 598 : cluster [DBG] pgmap v1194: 105 pgs: 105 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail 2024-04-07T16:35:18.191 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:35:17 smithi100 bash[74942]: audit 2024-04-07T16:35:16.837247+0000 mon.a (mon.0) 1908 : audit [INF] from='client.? 172.21.15.100:0/2137139960' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi100-111196-8"}]: dispatch 2024-04-07T16:35:18.191 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:35:17 smithi100 bash[74942]: audit 2024-04-07T16:35:16.838172+0000 mon.a (mon.0) 1909 : audit [INF] from='client.? 172.21.15.100:0/2137139960' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi100-111196-8"}]: dispatch 2024-04-07T16:35:18.191 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:35:17 smithi100 bash[74942]: audit 2024-04-07T16:35:16.838627+0000 mon.a (mon.0) 1910 : audit [INF] from='client.? 172.21.15.100:0/2137139960' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi100-111196-8", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-07T16:35:18.191 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:35:17 smithi100 bash[74942]: cluster 2024-04-07T16:35:16.985986+0000 mgr.y (mgr.25175) 598 : cluster [DBG] pgmap v1194: 105 pgs: 105 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail 2024-04-07T16:35:18.191 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:35:17 smithi100 bash[76202]: audit 2024-04-07T16:35:16.837247+0000 mon.a (mon.0) 1908 : audit [INF] from='client.? 172.21.15.100:0/2137139960' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi100-111196-8"}]: dispatch 2024-04-07T16:35:18.191 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:35:17 smithi100 bash[76202]: audit 2024-04-07T16:35:16.838172+0000 mon.a (mon.0) 1909 : audit [INF] from='client.? 172.21.15.100:0/2137139960' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi100-111196-8"}]: dispatch 2024-04-07T16:35:18.191 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:35:17 smithi100 bash[76202]: audit 2024-04-07T16:35:16.838627+0000 mon.a (mon.0) 1910 : audit [INF] from='client.? 172.21.15.100:0/2137139960' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi100-111196-8", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-07T16:35:18.191 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:35:17 smithi100 bash[76202]: cluster 2024-04-07T16:35:16.985986+0000 mgr.y (mgr.25175) 598 : cluster [DBG] pgmap v1194: 105 pgs: 105 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail 2024-04-07T16:35:18.690 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:35:18 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:35:18] "GET /metrics HTTP/1.1" 200 33990 "" "Prometheus/2.43.0" 2024-04-07T16:35:19.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:35:18 smithi144 bash[66654]: audit 2024-04-07T16:35:17.808848+0000 mon.a (mon.0) 1911 : audit [INF] from='client.? 172.21.15.100:0/2137139960' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi100-111196-8", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-07T16:35:19.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:35:18 smithi144 bash[66654]: cluster 2024-04-07T16:35:17.820272+0000 mon.a (mon.0) 1912 : cluster [DBG] osdmap e1022: 8 total, 8 up, 8 in 2024-04-07T16:35:19.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:35:18 smithi144 bash[66654]: audit 2024-04-07T16:35:17.820917+0000 mon.a (mon.0) 1913 : audit [INF] from='client.? 172.21.15.100:0/2137139960' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi100-111196-8", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi100-111196-8"}]: dispatch 2024-04-07T16:35:19.191 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:35:18 smithi100 bash[74942]: audit 2024-04-07T16:35:17.808848+0000 mon.a (mon.0) 1911 : audit [INF] from='client.? 172.21.15.100:0/2137139960' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi100-111196-8", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-07T16:35:19.191 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:35:18 smithi100 bash[74942]: cluster 2024-04-07T16:35:17.820272+0000 mon.a (mon.0) 1912 : cluster [DBG] osdmap e1022: 8 total, 8 up, 8 in 2024-04-07T16:35:19.191 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:35:18 smithi100 bash[74942]: audit 2024-04-07T16:35:17.820917+0000 mon.a (mon.0) 1913 : audit [INF] from='client.? 172.21.15.100:0/2137139960' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi100-111196-8", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi100-111196-8"}]: dispatch 2024-04-07T16:35:19.191 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:35:18 smithi100 bash[76202]: audit 2024-04-07T16:35:17.808848+0000 mon.a (mon.0) 1911 : audit [INF] from='client.? 172.21.15.100:0/2137139960' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi100-111196-8", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-07T16:35:19.191 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:35:18 smithi100 bash[76202]: cluster 2024-04-07T16:35:17.820272+0000 mon.a (mon.0) 1912 : cluster [DBG] osdmap e1022: 8 total, 8 up, 8 in 2024-04-07T16:35:19.191 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:35:18 smithi100 bash[76202]: audit 2024-04-07T16:35:17.820917+0000 mon.a (mon.0) 1913 : audit [INF] from='client.? 172.21.15.100:0/2137139960' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi100-111196-8", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi100-111196-8"}]: dispatch 2024-04-07T16:35:20.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:35:19 smithi144 bash[66654]: cluster 2024-04-07T16:35:18.830394+0000 mon.a (mon.0) 1914 : cluster [DBG] osdmap e1023: 8 total, 8 up, 8 in 2024-04-07T16:35:20.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:35:19 smithi144 bash[66654]: cluster 2024-04-07T16:35:18.986744+0000 mgr.y (mgr.25175) 599 : cluster [DBG] pgmap v1197: 105 pgs: 105 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail 2024-04-07T16:35:20.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:35:19 smithi144 bash[66654]: audit 2024-04-07T16:35:19.815263+0000 mon.a (mon.0) 1915 : audit [INF] from='client.? 172.21.15.100:0/2137139960' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi100-111196-8", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi100-111196-8"}]': finished 2024-04-07T16:35:20.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:35:19 smithi144 bash[66654]: cluster 2024-04-07T16:35:19.823083+0000 mon.a (mon.0) 1916 : cluster [DBG] osdmap e1024: 8 total, 8 up, 8 in 2024-04-07T16:35:20.191 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:35:19 smithi100 bash[74942]: cluster 2024-04-07T16:35:18.830394+0000 mon.a (mon.0) 1914 : cluster [DBG] osdmap e1023: 8 total, 8 up, 8 in 2024-04-07T16:35:20.191 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:35:19 smithi100 bash[74942]: cluster 2024-04-07T16:35:18.986744+0000 mgr.y (mgr.25175) 599 : cluster [DBG] pgmap v1197: 105 pgs: 105 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail 2024-04-07T16:35:20.191 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:35:19 smithi100 bash[74942]: audit 2024-04-07T16:35:19.815263+0000 mon.a (mon.0) 1915 : audit [INF] from='client.? 172.21.15.100:0/2137139960' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi100-111196-8", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi100-111196-8"}]': finished 2024-04-07T16:35:20.191 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:35:19 smithi100 bash[74942]: cluster 2024-04-07T16:35:19.823083+0000 mon.a (mon.0) 1916 : cluster [DBG] osdmap e1024: 8 total, 8 up, 8 in 2024-04-07T16:35:20.191 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:35:19 smithi100 bash[76202]: cluster 2024-04-07T16:35:18.830394+0000 mon.a (mon.0) 1914 : cluster [DBG] osdmap e1023: 8 total, 8 up, 8 in 2024-04-07T16:35:20.191 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:35:19 smithi100 bash[76202]: cluster 2024-04-07T16:35:18.986744+0000 mgr.y (mgr.25175) 599 : cluster [DBG] pgmap v1197: 105 pgs: 105 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail 2024-04-07T16:35:20.191 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:35:19 smithi100 bash[76202]: audit 2024-04-07T16:35:19.815263+0000 mon.a (mon.0) 1915 : audit [INF] from='client.? 172.21.15.100:0/2137139960' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi100-111196-8", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi100-111196-8"}]': finished 2024-04-07T16:35:20.191 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:35:19 smithi100 bash[76202]: cluster 2024-04-07T16:35:19.823083+0000 mon.a (mon.0) 1916 : cluster [DBG] osdmap e1024: 8 total, 8 up, 8 in 2024-04-07T16:35:22.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:35:21 smithi144 bash[66654]: cluster 2024-04-07T16:35:20.828376+0000 mon.a (mon.0) 1917 : cluster [DBG] osdmap e1025: 8 total, 8 up, 8 in 2024-04-07T16:35:22.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:35:21 smithi144 bash[66654]: cluster 2024-04-07T16:35:20.987517+0000 mgr.y (mgr.25175) 600 : cluster [DBG] pgmap v1200: 113 pgs: 1 creating+peering, 7 unknown, 105 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail 2024-04-07T16:35:22.191 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:35:21 smithi100 bash[74942]: cluster 2024-04-07T16:35:20.828376+0000 mon.a (mon.0) 1917 : cluster [DBG] osdmap e1025: 8 total, 8 up, 8 in 2024-04-07T16:35:22.191 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:35:21 smithi100 bash[74942]: cluster 2024-04-07T16:35:20.987517+0000 mgr.y (mgr.25175) 600 : cluster [DBG] pgmap v1200: 113 pgs: 1 creating+peering, 7 unknown, 105 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail 2024-04-07T16:35:22.191 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:35:21 smithi100 bash[76202]: cluster 2024-04-07T16:35:20.828376+0000 mon.a (mon.0) 1917 : cluster [DBG] osdmap e1025: 8 total, 8 up, 8 in 2024-04-07T16:35:22.191 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:35:21 smithi100 bash[76202]: cluster 2024-04-07T16:35:20.987517+0000 mgr.y (mgr.25175) 600 : cluster [DBG] pgmap v1200: 113 pgs: 1 creating+peering, 7 unknown, 105 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail 2024-04-07T16:35:22.607 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:35:22 smithi144 bash[95233]: ts=2024-04-07T16:35:22.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:35:23.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:35:22 smithi144 bash[66654]: cluster 2024-04-07T16:35:21.818008+0000 mon.a (mon.0) 1918 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T16:35:23.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:35:22 smithi144 bash[66654]: cluster 2024-04-07T16:35:21.836661+0000 mon.a (mon.0) 1919 : cluster [DBG] osdmap e1026: 8 total, 8 up, 8 in 2024-04-07T16:35:23.191 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:35:22 smithi100 bash[74942]: cluster 2024-04-07T16:35:21.818008+0000 mon.a (mon.0) 1918 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T16:35:23.191 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:35:22 smithi100 bash[74942]: cluster 2024-04-07T16:35:21.836661+0000 mon.a (mon.0) 1919 : cluster [DBG] osdmap e1026: 8 total, 8 up, 8 in 2024-04-07T16:35:23.191 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:35:22 smithi100 bash[76202]: cluster 2024-04-07T16:35:21.818008+0000 mon.a (mon.0) 1918 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T16:35:23.191 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:35:22 smithi100 bash[76202]: cluster 2024-04-07T16:35:21.836661+0000 mon.a (mon.0) 1919 : cluster [DBG] osdmap e1026: 8 total, 8 up, 8 in 2024-04-07T16:35:24.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:35:23 smithi144 bash[66654]: cluster 2024-04-07T16:35:22.988589+0000 mgr.y (mgr.25175) 601 : cluster [DBG] pgmap v1202: 113 pgs: 1 creating+peering, 4 unknown, 108 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail 2024-04-07T16:35:24.191 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:35:23 smithi100 bash[74942]: cluster 2024-04-07T16:35:22.988589+0000 mgr.y (mgr.25175) 601 : cluster [DBG] pgmap v1202: 113 pgs: 1 creating+peering, 4 unknown, 108 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail 2024-04-07T16:35:24.191 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:35:23 smithi100 bash[76202]: cluster 2024-04-07T16:35:22.988589+0000 mgr.y (mgr.25175) 601 : cluster [DBG] pgmap v1202: 113 pgs: 1 creating+peering, 4 unknown, 108 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail 2024-04-07T16:35:26.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:35:25 smithi144 bash[66654]: cluster 2024-04-07T16:35:24.850326+0000 mon.a (mon.0) 1920 : cluster [DBG] osdmap e1027: 8 total, 8 up, 8 in 2024-04-07T16:35:26.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:35:25 smithi144 bash[66654]: cluster 2024-04-07T16:35:24.989325+0000 mgr.y (mgr.25175) 602 : cluster [DBG] pgmap v1204: 113 pgs: 1 creating+peering, 112 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 197 B/s wr, 0 op/s 2024-04-07T16:35:26.191 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:35:25 smithi100 bash[74942]: cluster 2024-04-07T16:35:24.850326+0000 mon.a (mon.0) 1920 : cluster [DBG] osdmap e1027: 8 total, 8 up, 8 in 2024-04-07T16:35:26.191 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:35:25 smithi100 bash[74942]: cluster 2024-04-07T16:35:24.989325+0000 mgr.y (mgr.25175) 602 : cluster [DBG] pgmap v1204: 113 pgs: 1 creating+peering, 112 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 197 B/s wr, 0 op/s 2024-04-07T16:35:26.191 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:35:25 smithi100 bash[76202]: cluster 2024-04-07T16:35:24.850326+0000 mon.a (mon.0) 1920 : cluster [DBG] osdmap e1027: 8 total, 8 up, 8 in 2024-04-07T16:35:26.191 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:35:25 smithi100 bash[76202]: cluster 2024-04-07T16:35:24.989325+0000 mgr.y (mgr.25175) 602 : cluster [DBG] pgmap v1204: 113 pgs: 1 creating+peering, 112 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 197 B/s wr, 0 op/s 2024-04-07T16:35:27.106 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:35:26 smithi144 bash[66654]: cluster 2024-04-07T16:35:25.869791+0000 mon.a (mon.0) 1921 : cluster [DBG] osdmap e1028: 8 total, 8 up, 8 in 2024-04-07T16:35:27.190 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:35:26 smithi100 bash[74942]: cluster 2024-04-07T16:35:25.869791+0000 mon.a (mon.0) 1921 : cluster [DBG] osdmap e1028: 8 total, 8 up, 8 in 2024-04-07T16:35:27.191 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:35:26 smithi100 bash[76202]: cluster 2024-04-07T16:35:25.869791+0000 mon.a (mon.0) 1921 : cluster [DBG] osdmap e1028: 8 total, 8 up, 8 in 2024-04-07T16:35:28.191 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:35:27 smithi100 bash[76202]: cluster 2024-04-07T16:35:26.876622+0000 mon.a (mon.0) 1922 : cluster [DBG] osdmap e1029: 8 total, 8 up, 8 in 2024-04-07T16:35:28.191 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:35:27 smithi100 bash[76202]: cluster 2024-04-07T16:35:26.990083+0000 mgr.y (mgr.25175) 603 : cluster [DBG] pgmap v1207: 113 pgs: 113 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 198 B/s wr, 0 op/s 2024-04-07T16:35:28.191 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:35:27 smithi100 bash[74942]: cluster 2024-04-07T16:35:26.876622+0000 mon.a (mon.0) 1922 : cluster [DBG] osdmap e1029: 8 total, 8 up, 8 in 2024-04-07T16:35:28.191 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:35:27 smithi100 bash[74942]: cluster 2024-04-07T16:35:26.990083+0000 mgr.y (mgr.25175) 603 : cluster [DBG] pgmap v1207: 113 pgs: 113 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 198 B/s wr, 0 op/s 2024-04-07T16:35:28.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:35:27 smithi144 bash[66654]: cluster 2024-04-07T16:35:26.876622+0000 mon.a (mon.0) 1922 : cluster [DBG] osdmap e1029: 8 total, 8 up, 8 in 2024-04-07T16:35:28.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:35:27 smithi144 bash[66654]: cluster 2024-04-07T16:35:26.990083+0000 mgr.y (mgr.25175) 603 : cluster [DBG] pgmap v1207: 113 pgs: 113 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 198 B/s wr, 0 op/s 2024-04-07T16:35:28.690 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:35:28 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:35:28] "GET /metrics HTTP/1.1" 200 33990 "" "Prometheus/2.43.0" 2024-04-07T16:35:29.191 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:35:28 smithi100 bash[74942]: cluster 2024-04-07T16:35:27.887137+0000 mon.a (mon.0) 1923 : cluster [DBG] osdmap e1030: 8 total, 8 up, 8 in 2024-04-07T16:35:29.191 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:35:28 smithi100 bash[74942]: audit 2024-04-07T16:35:27.889102+0000 mon.a (mon.0) 1924 : audit [INF] from='client.? 172.21.15.100:0/2137139960' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi100-111196-8"}]: dispatch 2024-04-07T16:35:29.191 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:35:28 smithi100 bash[76202]: cluster 2024-04-07T16:35:27.887137+0000 mon.a (mon.0) 1923 : cluster [DBG] osdmap e1030: 8 total, 8 up, 8 in 2024-04-07T16:35:29.191 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:35:28 smithi100 bash[76202]: audit 2024-04-07T16:35:27.889102+0000 mon.a (mon.0) 1924 : audit [INF] from='client.? 172.21.15.100:0/2137139960' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi100-111196-8"}]: dispatch 2024-04-07T16:35:29.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:35:28 smithi144 bash[66654]: cluster 2024-04-07T16:35:27.887137+0000 mon.a (mon.0) 1923 : cluster [DBG] osdmap e1030: 8 total, 8 up, 8 in 2024-04-07T16:35:29.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:35:28 smithi144 bash[66654]: audit 2024-04-07T16:35:27.889102+0000 mon.a (mon.0) 1924 : audit [INF] from='client.? 172.21.15.100:0/2137139960' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi100-111196-8"}]: dispatch 2024-04-07T16:35:29.904 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] cls_rgw.test_put_snap_ec (13118 ms) 2024-04-07T16:35:29.904 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] cls_rgw.test_explicit_ec 2024-04-07T16:35:30.191 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:35:29 smithi100 bash[74942]: audit 2024-04-07T16:35:28.890549+0000 mon.a (mon.0) 1925 : audit [INF] from='client.? 172.21.15.100:0/2137139960' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi100-111196-8"}]': finished 2024-04-07T16:35:30.191 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:35:29 smithi100 bash[74942]: cluster 2024-04-07T16:35:28.898967+0000 mon.a (mon.0) 1926 : cluster [DBG] osdmap e1031: 8 total, 8 up, 8 in 2024-04-07T16:35:30.191 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:35:29 smithi100 bash[74942]: audit 2024-04-07T16:35:28.899880+0000 mon.a (mon.0) 1927 : audit [INF] from='client.? 172.21.15.100:0/2137139960' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi100-111196-8"}]: dispatch 2024-04-07T16:35:30.191 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:35:29 smithi100 bash[74942]: cluster 2024-04-07T16:35:28.990820+0000 mgr.y (mgr.25175) 604 : cluster [DBG] pgmap v1210: 105 pgs: 105 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail 2024-04-07T16:35:30.191 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:35:29 smithi100 bash[76202]: audit 2024-04-07T16:35:28.890549+0000 mon.a (mon.0) 1925 : audit [INF] from='client.? 172.21.15.100:0/2137139960' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi100-111196-8"}]': finished 2024-04-07T16:35:30.191 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:35:29 smithi100 bash[76202]: cluster 2024-04-07T16:35:28.898967+0000 mon.a (mon.0) 1926 : cluster [DBG] osdmap e1031: 8 total, 8 up, 8 in 2024-04-07T16:35:30.191 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:35:29 smithi100 bash[76202]: audit 2024-04-07T16:35:28.899880+0000 mon.a (mon.0) 1927 : audit [INF] from='client.? 172.21.15.100:0/2137139960' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi100-111196-8"}]: dispatch 2024-04-07T16:35:30.192 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:35:29 smithi100 bash[76202]: cluster 2024-04-07T16:35:28.990820+0000 mgr.y (mgr.25175) 604 : cluster [DBG] pgmap v1210: 105 pgs: 105 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail 2024-04-07T16:35:30.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:35:29 smithi144 bash[66654]: audit 2024-04-07T16:35:28.890549+0000 mon.a (mon.0) 1925 : audit [INF] from='client.? 172.21.15.100:0/2137139960' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi100-111196-8"}]': finished 2024-04-07T16:35:30.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:35:29 smithi144 bash[66654]: cluster 2024-04-07T16:35:28.898967+0000 mon.a (mon.0) 1926 : cluster [DBG] osdmap e1031: 8 total, 8 up, 8 in 2024-04-07T16:35:30.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:35:29 smithi144 bash[66654]: audit 2024-04-07T16:35:28.899880+0000 mon.a (mon.0) 1927 : audit [INF] from='client.? 172.21.15.100:0/2137139960' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi100-111196-8"}]: dispatch 2024-04-07T16:35:30.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:35:29 smithi144 bash[66654]: cluster 2024-04-07T16:35:28.990820+0000 mgr.y (mgr.25175) 604 : cluster [DBG] pgmap v1210: 105 pgs: 105 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail 2024-04-07T16:35:31.191 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:35:30 smithi100 bash[76202]: cluster 2024-04-07T16:35:29.890554+0000 mon.a (mon.0) 1928 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T16:35:31.191 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:35:30 smithi100 bash[76202]: cluster 2024-04-07T16:35:29.890612+0000 mon.a (mon.0) 1929 : cluster [INF] Cluster is now healthy 2024-04-07T16:35:31.191 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:35:30 smithi100 bash[76202]: audit 2024-04-07T16:35:29.902637+0000 mon.a (mon.0) 1930 : audit [INF] from='client.? 172.21.15.100:0/2137139960' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi100-111196-8"}]': finished 2024-04-07T16:35:31.191 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:35:30 smithi100 bash[76202]: cluster 2024-04-07T16:35:29.912277+0000 mon.a (mon.0) 1931 : cluster [DBG] osdmap e1032: 8 total, 8 up, 8 in 2024-04-07T16:35:31.191 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:35:30 smithi100 bash[76202]: audit 2024-04-07T16:35:29.938832+0000 mon.a (mon.0) 1932 : audit [INF] from='client.? 172.21.15.100:0/2286475932' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi100-111196-9"}]: dispatch 2024-04-07T16:35:31.191 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:35:30 smithi100 bash[76202]: audit 2024-04-07T16:35:29.939813+0000 mon.a (mon.0) 1933 : audit [INF] from='client.? 172.21.15.100:0/2286475932' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi100-111196-9"}]: dispatch 2024-04-07T16:35:31.191 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:35:30 smithi100 bash[76202]: audit 2024-04-07T16:35:29.940265+0000 mon.a (mon.0) 1934 : audit [INF] from='client.? 172.21.15.100:0/2286475932' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi100-111196-9", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-07T16:35:31.192 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:35:30 smithi100 bash[74942]: cluster 2024-04-07T16:35:29.890554+0000 mon.a (mon.0) 1928 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T16:35:31.192 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:35:30 smithi100 bash[74942]: cluster 2024-04-07T16:35:29.890612+0000 mon.a (mon.0) 1929 : cluster [INF] Cluster is now healthy 2024-04-07T16:35:31.192 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:35:30 smithi100 bash[74942]: audit 2024-04-07T16:35:29.902637+0000 mon.a (mon.0) 1930 : audit [INF] from='client.? 172.21.15.100:0/2137139960' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi100-111196-8"}]': finished 2024-04-07T16:35:31.192 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:35:30 smithi100 bash[74942]: cluster 2024-04-07T16:35:29.912277+0000 mon.a (mon.0) 1931 : cluster [DBG] osdmap e1032: 8 total, 8 up, 8 in 2024-04-07T16:35:31.192 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:35:30 smithi100 bash[74942]: audit 2024-04-07T16:35:29.938832+0000 mon.a (mon.0) 1932 : audit [INF] from='client.? 172.21.15.100:0/2286475932' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi100-111196-9"}]: dispatch 2024-04-07T16:35:31.192 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:35:30 smithi100 bash[74942]: audit 2024-04-07T16:35:29.939813+0000 mon.a (mon.0) 1933 : audit [INF] from='client.? 172.21.15.100:0/2286475932' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi100-111196-9"}]: dispatch 2024-04-07T16:35:31.192 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:35:30 smithi100 bash[74942]: audit 2024-04-07T16:35:29.940265+0000 mon.a (mon.0) 1934 : audit [INF] from='client.? 172.21.15.100:0/2286475932' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi100-111196-9", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-07T16:35:31.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:35:30 smithi144 bash[66654]: cluster 2024-04-07T16:35:29.890554+0000 mon.a (mon.0) 1928 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T16:35:31.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:35:30 smithi144 bash[66654]: cluster 2024-04-07T16:35:29.890612+0000 mon.a (mon.0) 1929 : cluster [INF] Cluster is now healthy 2024-04-07T16:35:31.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:35:30 smithi144 bash[66654]: audit 2024-04-07T16:35:29.902637+0000 mon.a (mon.0) 1930 : audit [INF] from='client.? 172.21.15.100:0/2137139960' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi100-111196-8"}]': finished 2024-04-07T16:35:31.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:35:30 smithi144 bash[66654]: cluster 2024-04-07T16:35:29.912277+0000 mon.a (mon.0) 1931 : cluster [DBG] osdmap e1032: 8 total, 8 up, 8 in 2024-04-07T16:35:31.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:35:30 smithi144 bash[66654]: audit 2024-04-07T16:35:29.938832+0000 mon.a (mon.0) 1932 : audit [INF] from='client.? 172.21.15.100:0/2286475932' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi100-111196-9"}]: dispatch 2024-04-07T16:35:31.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:35:30 smithi144 bash[66654]: audit 2024-04-07T16:35:29.939813+0000 mon.a (mon.0) 1933 : audit [INF] from='client.? 172.21.15.100:0/2286475932' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi100-111196-9"}]: dispatch 2024-04-07T16:35:31.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:35:30 smithi144 bash[66654]: audit 2024-04-07T16:35:29.940265+0000 mon.a (mon.0) 1934 : audit [INF] from='client.? 172.21.15.100:0/2286475932' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi100-111196-9", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-07T16:35:32.191 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:35:31 smithi100 bash[74942]: audit 2024-04-07T16:35:30.905200+0000 mon.a (mon.0) 1935 : audit [INF] from='client.? 172.21.15.100:0/2286475932' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi100-111196-9", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-07T16:35:32.191 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:35:31 smithi100 bash[74942]: cluster 2024-04-07T16:35:30.913003+0000 mon.a (mon.0) 1936 : cluster [DBG] osdmap e1033: 8 total, 8 up, 8 in 2024-04-07T16:35:32.191 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:35:31 smithi100 bash[74942]: audit 2024-04-07T16:35:30.914204+0000 mon.a (mon.0) 1937 : audit [INF] from='client.? 172.21.15.100:0/2286475932' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi100-111196-9", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi100-111196-9"}]: dispatch 2024-04-07T16:35:32.191 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:35:31 smithi100 bash[74942]: cluster 2024-04-07T16:35:30.991513+0000 mgr.y (mgr.25175) 605 : cluster [DBG] pgmap v1213: 105 pgs: 105 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail 2024-04-07T16:35:32.191 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:35:31 smithi100 bash[74942]: audit 2024-04-07T16:35:31.748879+0000 mon.a (mon.0) 1938 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:35:32.191 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:35:31 smithi100 bash[76202]: audit 2024-04-07T16:35:30.905200+0000 mon.a (mon.0) 1935 : audit [INF] from='client.? 172.21.15.100:0/2286475932' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi100-111196-9", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-07T16:35:32.191 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:35:31 smithi100 bash[76202]: cluster 2024-04-07T16:35:30.913003+0000 mon.a (mon.0) 1936 : cluster [DBG] osdmap e1033: 8 total, 8 up, 8 in 2024-04-07T16:35:32.192 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:35:31 smithi100 bash[76202]: audit 2024-04-07T16:35:30.914204+0000 mon.a (mon.0) 1937 : audit [INF] from='client.? 172.21.15.100:0/2286475932' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi100-111196-9", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi100-111196-9"}]: dispatch 2024-04-07T16:35:32.192 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:35:31 smithi100 bash[76202]: cluster 2024-04-07T16:35:30.991513+0000 mgr.y (mgr.25175) 605 : cluster [DBG] pgmap v1213: 105 pgs: 105 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail 2024-04-07T16:35:32.192 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:35:31 smithi100 bash[76202]: audit 2024-04-07T16:35:31.748879+0000 mon.a (mon.0) 1938 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:35:32.307 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:35:31 smithi144 bash[66654]: audit 2024-04-07T16:35:30.905200+0000 mon.a (mon.0) 1935 : audit [INF] from='client.? 172.21.15.100:0/2286475932' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi100-111196-9", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-07T16:35:32.307 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:35:31 smithi144 bash[66654]: cluster 2024-04-07T16:35:30.913003+0000 mon.a (mon.0) 1936 : cluster [DBG] osdmap e1033: 8 total, 8 up, 8 in 2024-04-07T16:35:32.307 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:35:31 smithi144 bash[66654]: audit 2024-04-07T16:35:30.914204+0000 mon.a (mon.0) 1937 : audit [INF] from='client.? 172.21.15.100:0/2286475932' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi100-111196-9", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi100-111196-9"}]: dispatch 2024-04-07T16:35:32.307 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:35:31 smithi144 bash[66654]: cluster 2024-04-07T16:35:30.991513+0000 mgr.y (mgr.25175) 605 : cluster [DBG] pgmap v1213: 105 pgs: 105 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail 2024-04-07T16:35:32.307 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:35:31 smithi144 bash[66654]: audit 2024-04-07T16:35:31.748879+0000 mon.a (mon.0) 1938 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:35:32.607 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:35:32 smithi144 bash[95233]: ts=2024-04-07T16:35:32.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:35:33.191 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:35:32 smithi100 bash[74942]: cluster 2024-04-07T16:35:31.929447+0000 mon.a (mon.0) 1939 : cluster [DBG] osdmap e1034: 8 total, 8 up, 8 in 2024-04-07T16:35:33.191 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:35:32 smithi100 bash[74942]: audit 2024-04-07T16:35:32.911793+0000 mon.a (mon.0) 1940 : audit [INF] from='client.? 172.21.15.100:0/2286475932' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi100-111196-9", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi100-111196-9"}]': finished 2024-04-07T16:35:33.191 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:35:32 smithi100 bash[74942]: cluster 2024-04-07T16:35:32.919605+0000 mon.a (mon.0) 1941 : cluster [DBG] osdmap e1035: 8 total, 8 up, 8 in 2024-04-07T16:35:33.191 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:35:32 smithi100 bash[76202]: cluster 2024-04-07T16:35:31.929447+0000 mon.a (mon.0) 1939 : cluster [DBG] osdmap e1034: 8 total, 8 up, 8 in 2024-04-07T16:35:33.191 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:35:32 smithi100 bash[76202]: audit 2024-04-07T16:35:32.911793+0000 mon.a (mon.0) 1940 : audit [INF] from='client.? 172.21.15.100:0/2286475932' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi100-111196-9", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi100-111196-9"}]': finished 2024-04-07T16:35:33.192 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:35:32 smithi100 bash[76202]: cluster 2024-04-07T16:35:32.919605+0000 mon.a (mon.0) 1941 : cluster [DBG] osdmap e1035: 8 total, 8 up, 8 in 2024-04-07T16:35:33.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:35:32 smithi144 bash[66654]: cluster 2024-04-07T16:35:31.929447+0000 mon.a (mon.0) 1939 : cluster [DBG] osdmap e1034: 8 total, 8 up, 8 in 2024-04-07T16:35:33.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:35:32 smithi144 bash[66654]: audit 2024-04-07T16:35:32.911793+0000 mon.a (mon.0) 1940 : audit [INF] from='client.? 172.21.15.100:0/2286475932' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi100-111196-9", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi100-111196-9"}]': finished 2024-04-07T16:35:33.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:35:32 smithi144 bash[66654]: cluster 2024-04-07T16:35:32.919605+0000 mon.a (mon.0) 1941 : cluster [DBG] osdmap e1035: 8 total, 8 up, 8 in 2024-04-07T16:35:34.191 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:35:33 smithi100 bash[74942]: cluster 2024-04-07T16:35:32.992219+0000 mgr.y (mgr.25175) 606 : cluster [DBG] pgmap v1216: 113 pgs: 8 unknown, 105 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail 2024-04-07T16:35:34.191 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:35:33 smithi100 bash[74942]: cluster 2024-04-07T16:35:33.922806+0000 mon.a (mon.0) 1942 : cluster [DBG] osdmap e1036: 8 total, 8 up, 8 in 2024-04-07T16:35:34.191 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:35:33 smithi100 bash[76202]: cluster 2024-04-07T16:35:32.992219+0000 mgr.y (mgr.25175) 606 : cluster [DBG] pgmap v1216: 113 pgs: 8 unknown, 105 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail 2024-04-07T16:35:34.191 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:35:33 smithi100 bash[76202]: cluster 2024-04-07T16:35:33.922806+0000 mon.a (mon.0) 1942 : cluster [DBG] osdmap e1036: 8 total, 8 up, 8 in 2024-04-07T16:35:34.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:35:33 smithi144 bash[66654]: cluster 2024-04-07T16:35:32.992219+0000 mgr.y (mgr.25175) 606 : cluster [DBG] pgmap v1216: 113 pgs: 8 unknown, 105 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail 2024-04-07T16:35:34.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:35:33 smithi144 bash[66654]: cluster 2024-04-07T16:35:33.922806+0000 mon.a (mon.0) 1942 : cluster [DBG] osdmap e1036: 8 total, 8 up, 8 in 2024-04-07T16:35:35.191 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:35:34 smithi100 bash[74942]: cluster 2024-04-07T16:35:33.933222+0000 mon.a (mon.0) 1943 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T16:35:35.191 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:35:34 smithi100 bash[74942]: cluster 2024-04-07T16:35:34.924589+0000 mon.a (mon.0) 1944 : cluster [DBG] osdmap e1037: 8 total, 8 up, 8 in 2024-04-07T16:35:35.191 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:35:34 smithi100 bash[74942]: audit 2024-04-07T16:35:34.929307+0000 mon.a (mon.0) 1945 : audit [INF] from='client.? 172.21.15.100:0/2286475932' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi100-111196-9"}]: dispatch 2024-04-07T16:35:35.191 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:35:34 smithi100 bash[76202]: cluster 2024-04-07T16:35:33.933222+0000 mon.a (mon.0) 1943 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T16:35:35.191 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:35:34 smithi100 bash[76202]: cluster 2024-04-07T16:35:34.924589+0000 mon.a (mon.0) 1944 : cluster [DBG] osdmap e1037: 8 total, 8 up, 8 in 2024-04-07T16:35:35.191 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:35:34 smithi100 bash[76202]: audit 2024-04-07T16:35:34.929307+0000 mon.a (mon.0) 1945 : audit [INF] from='client.? 172.21.15.100:0/2286475932' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi100-111196-9"}]: dispatch 2024-04-07T16:35:35.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:35:34 smithi144 bash[66654]: cluster 2024-04-07T16:35:33.933222+0000 mon.a (mon.0) 1943 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T16:35:35.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:35:34 smithi144 bash[66654]: cluster 2024-04-07T16:35:34.924589+0000 mon.a (mon.0) 1944 : cluster [DBG] osdmap e1037: 8 total, 8 up, 8 in 2024-04-07T16:35:35.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:35:34 smithi144 bash[66654]: audit 2024-04-07T16:35:34.929307+0000 mon.a (mon.0) 1945 : audit [INF] from='client.? 172.21.15.100:0/2286475932' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi100-111196-9"}]: dispatch 2024-04-07T16:35:36.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:35:35 smithi144 bash[66654]: cluster 2024-04-07T16:35:34.992809+0000 mgr.y (mgr.25175) 607 : cluster [DBG] pgmap v1219: 105 pgs: 105 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail 2024-04-07T16:35:36.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:35:35 smithi144 bash[66654]: audit 2024-04-07T16:35:35.920043+0000 mon.a (mon.0) 1946 : audit [INF] from='client.? 172.21.15.100:0/2286475932' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi100-111196-9"}]': finished 2024-04-07T16:35:36.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:35:35 smithi144 bash[66654]: cluster 2024-04-07T16:35:35.931241+0000 mon.a (mon.0) 1947 : cluster [DBG] osdmap e1038: 8 total, 8 up, 8 in 2024-04-07T16:35:36.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:35:35 smithi144 bash[66654]: audit 2024-04-07T16:35:35.932099+0000 mon.a (mon.0) 1948 : audit [INF] from='client.? 172.21.15.100:0/2286475932' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi100-111196-9"}]: dispatch 2024-04-07T16:35:36.441 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:35:35 smithi100 bash[74942]: cluster 2024-04-07T16:35:34.992809+0000 mgr.y (mgr.25175) 607 : cluster [DBG] pgmap v1219: 105 pgs: 105 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail 2024-04-07T16:35:36.441 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:35:35 smithi100 bash[74942]: audit 2024-04-07T16:35:35.920043+0000 mon.a (mon.0) 1946 : audit [INF] from='client.? 172.21.15.100:0/2286475932' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi100-111196-9"}]': finished 2024-04-07T16:35:36.441 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:35:35 smithi100 bash[74942]: cluster 2024-04-07T16:35:35.931241+0000 mon.a (mon.0) 1947 : cluster [DBG] osdmap e1038: 8 total, 8 up, 8 in 2024-04-07T16:35:36.441 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:35:35 smithi100 bash[74942]: audit 2024-04-07T16:35:35.932099+0000 mon.a (mon.0) 1948 : audit [INF] from='client.? 172.21.15.100:0/2286475932' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi100-111196-9"}]: dispatch 2024-04-07T16:35:36.441 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:35:35 smithi100 bash[76202]: cluster 2024-04-07T16:35:34.992809+0000 mgr.y (mgr.25175) 607 : cluster [DBG] pgmap v1219: 105 pgs: 105 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail 2024-04-07T16:35:36.441 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:35:35 smithi100 bash[76202]: audit 2024-04-07T16:35:35.920043+0000 mon.a (mon.0) 1946 : audit [INF] from='client.? 172.21.15.100:0/2286475932' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi100-111196-9"}]': finished 2024-04-07T16:35:36.442 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:35:35 smithi100 bash[76202]: cluster 2024-04-07T16:35:35.931241+0000 mon.a (mon.0) 1947 : cluster [DBG] osdmap e1038: 8 total, 8 up, 8 in 2024-04-07T16:35:36.442 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:35:35 smithi100 bash[76202]: audit 2024-04-07T16:35:35.932099+0000 mon.a (mon.0) 1948 : audit [INF] from='client.? 172.21.15.100:0/2286475932' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi100-111196-9"}]: dispatch 2024-04-07T16:35:36.923 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] cls_rgw.test_explicit_ec (7019 ms) 2024-04-07T16:35:36.923 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] cls_rgw.set_ec 2024-04-07T16:35:37.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:35:36 smithi144 bash[66654]: cluster 2024-04-07T16:35:35.948993+0000 mon.a (mon.0) 1949 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T16:35:37.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:35:36 smithi144 bash[66654]: cluster 2024-04-07T16:35:35.949043+0000 mon.a (mon.0) 1950 : cluster [INF] Cluster is now healthy 2024-04-07T16:35:37.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:35:36 smithi144 bash[66654]: audit 2024-04-07T16:35:36.923492+0000 mon.a (mon.0) 1951 : audit [INF] from='client.? 172.21.15.100:0/2286475932' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi100-111196-9"}]': finished 2024-04-07T16:35:37.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:35:36 smithi144 bash[66654]: cluster 2024-04-07T16:35:36.931049+0000 mon.a (mon.0) 1952 : cluster [DBG] osdmap e1039: 8 total, 8 up, 8 in 2024-04-07T16:35:37.441 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:35:36 smithi100 bash[74942]: cluster 2024-04-07T16:35:35.948993+0000 mon.a (mon.0) 1949 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T16:35:37.441 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:35:36 smithi100 bash[74942]: cluster 2024-04-07T16:35:35.949043+0000 mon.a (mon.0) 1950 : cluster [INF] Cluster is now healthy 2024-04-07T16:35:37.441 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:35:36 smithi100 bash[74942]: audit 2024-04-07T16:35:36.923492+0000 mon.a (mon.0) 1951 : audit [INF] from='client.? 172.21.15.100:0/2286475932' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi100-111196-9"}]': finished 2024-04-07T16:35:37.441 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:35:36 smithi100 bash[74942]: cluster 2024-04-07T16:35:36.931049+0000 mon.a (mon.0) 1952 : cluster [DBG] osdmap e1039: 8 total, 8 up, 8 in 2024-04-07T16:35:37.441 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:35:36 smithi100 bash[76202]: cluster 2024-04-07T16:35:35.948993+0000 mon.a (mon.0) 1949 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T16:35:37.441 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:35:36 smithi100 bash[76202]: cluster 2024-04-07T16:35:35.949043+0000 mon.a (mon.0) 1950 : cluster [INF] Cluster is now healthy 2024-04-07T16:35:37.442 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:35:36 smithi100 bash[76202]: audit 2024-04-07T16:35:36.923492+0000 mon.a (mon.0) 1951 : audit [INF] from='client.? 172.21.15.100:0/2286475932' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi100-111196-9"}]': finished 2024-04-07T16:35:37.442 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:35:36 smithi100 bash[76202]: cluster 2024-04-07T16:35:36.931049+0000 mon.a (mon.0) 1952 : cluster [DBG] osdmap e1039: 8 total, 8 up, 8 in 2024-04-07T16:35:38.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:35:37 smithi144 bash[66654]: audit 2024-04-07T16:35:36.972145+0000 mon.a (mon.0) 1953 : audit [INF] from='client.? 172.21.15.100:0/3358779172' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi100-111196-10"}]: dispatch 2024-04-07T16:35:38.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:35:37 smithi144 bash[66654]: audit 2024-04-07T16:35:36.973097+0000 mon.a (mon.0) 1954 : audit [INF] from='client.? 172.21.15.100:0/3358779172' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi100-111196-10"}]: dispatch 2024-04-07T16:35:38.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:35:37 smithi144 bash[66654]: audit 2024-04-07T16:35:36.973466+0000 mon.a (mon.0) 1955 : audit [INF] from='client.? 172.21.15.100:0/3358779172' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi100-111196-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-07T16:35:38.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:35:37 smithi144 bash[66654]: cluster 2024-04-07T16:35:36.993326+0000 mgr.y (mgr.25175) 608 : cluster [DBG] pgmap v1222: 105 pgs: 105 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail 2024-04-07T16:35:38.418 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:35:37 smithi100 bash[74942]: audit 2024-04-07T16:35:36.972145+0000 mon.a (mon.0) 1953 : audit [INF] from='client.? 172.21.15.100:0/3358779172' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi100-111196-10"}]: dispatch 2024-04-07T16:35:38.418 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:35:37 smithi100 bash[74942]: audit 2024-04-07T16:35:36.973097+0000 mon.a (mon.0) 1954 : audit [INF] from='client.? 172.21.15.100:0/3358779172' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi100-111196-10"}]: dispatch 2024-04-07T16:35:38.418 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:35:37 smithi100 bash[74942]: audit 2024-04-07T16:35:36.973466+0000 mon.a (mon.0) 1955 : audit [INF] from='client.? 172.21.15.100:0/3358779172' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi100-111196-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-07T16:35:38.418 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:35:37 smithi100 bash[74942]: cluster 2024-04-07T16:35:36.993326+0000 mgr.y (mgr.25175) 608 : cluster [DBG] pgmap v1222: 105 pgs: 105 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail 2024-04-07T16:35:38.418 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:35:37 smithi100 bash[76202]: audit 2024-04-07T16:35:36.972145+0000 mon.a (mon.0) 1953 : audit [INF] from='client.? 172.21.15.100:0/3358779172' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi100-111196-10"}]: dispatch 2024-04-07T16:35:38.419 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:35:37 smithi100 bash[76202]: audit 2024-04-07T16:35:36.973097+0000 mon.a (mon.0) 1954 : audit [INF] from='client.? 172.21.15.100:0/3358779172' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi100-111196-10"}]: dispatch 2024-04-07T16:35:38.419 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:35:37 smithi100 bash[76202]: audit 2024-04-07T16:35:36.973466+0000 mon.a (mon.0) 1955 : audit [INF] from='client.? 172.21.15.100:0/3358779172' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi100-111196-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-07T16:35:38.419 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:35:37 smithi100 bash[76202]: cluster 2024-04-07T16:35:36.993326+0000 mgr.y (mgr.25175) 608 : cluster [DBG] pgmap v1222: 105 pgs: 105 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail 2024-04-07T16:35:38.691 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:35:38 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:35:38] "GET /metrics HTTP/1.1" 200 33985 "" "Prometheus/2.43.0" 2024-04-07T16:35:39.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:35:38 smithi144 bash[66654]: audit 2024-04-07T16:35:37.972216+0000 mon.a (mon.0) 1956 : audit [INF] from='client.? 172.21.15.100:0/3358779172' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi100-111196-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-07T16:35:39.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:35:38 smithi144 bash[66654]: cluster 2024-04-07T16:35:37.979892+0000 mon.a (mon.0) 1957 : cluster [DBG] osdmap e1040: 8 total, 8 up, 8 in 2024-04-07T16:35:39.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:35:38 smithi144 bash[66654]: audit 2024-04-07T16:35:37.980507+0000 mon.a (mon.0) 1958 : audit [INF] from='client.? 172.21.15.100:0/3358779172' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi100-111196-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi100-111196-10"}]: dispatch 2024-04-07T16:35:39.441 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:35:39 smithi100 bash[76202]: audit 2024-04-07T16:35:37.972216+0000 mon.a (mon.0) 1956 : audit [INF] from='client.? 172.21.15.100:0/3358779172' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi100-111196-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-07T16:35:39.441 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:35:39 smithi100 bash[76202]: cluster 2024-04-07T16:35:37.979892+0000 mon.a (mon.0) 1957 : cluster [DBG] osdmap e1040: 8 total, 8 up, 8 in 2024-04-07T16:35:39.441 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:35:39 smithi100 bash[76202]: audit 2024-04-07T16:35:37.980507+0000 mon.a (mon.0) 1958 : audit [INF] from='client.? 172.21.15.100:0/3358779172' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi100-111196-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi100-111196-10"}]: dispatch 2024-04-07T16:35:39.441 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:35:38 smithi100 bash[74942]: audit 2024-04-07T16:35:37.972216+0000 mon.a (mon.0) 1956 : audit [INF] from='client.? 172.21.15.100:0/3358779172' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi100-111196-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-07T16:35:39.441 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:35:38 smithi100 bash[74942]: cluster 2024-04-07T16:35:37.979892+0000 mon.a (mon.0) 1957 : cluster [DBG] osdmap e1040: 8 total, 8 up, 8 in 2024-04-07T16:35:39.441 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:35:38 smithi100 bash[74942]: audit 2024-04-07T16:35:37.980507+0000 mon.a (mon.0) 1958 : audit [INF] from='client.? 172.21.15.100:0/3358779172' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi100-111196-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi100-111196-10"}]: dispatch 2024-04-07T16:35:40.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:35:39 smithi144 bash[66654]: cluster 2024-04-07T16:35:38.993837+0000 mgr.y (mgr.25175) 609 : cluster [DBG] pgmap v1225: 105 pgs: 105 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail 2024-04-07T16:35:40.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:35:40 smithi144 bash[66654]: cluster 2024-04-07T16:35:38.995558+0000 mon.a (mon.0) 1959 : cluster [DBG] osdmap e1041: 8 total, 8 up, 8 in 2024-04-07T16:35:40.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:35:40 smithi144 bash[66654]: audit 2024-04-07T16:35:39.118729+0000 mon.a (mon.0) 1960 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:35:40.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:35:40 smithi144 bash[66654]: audit 2024-04-07T16:35:39.858427+0000 mon.a (mon.0) 1961 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:35:40.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:35:40 smithi144 bash[66654]: audit 2024-04-07T16:35:39.860227+0000 mon.a (mon.0) 1962 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:35:40.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:35:40 smithi144 bash[66654]: audit 2024-04-07T16:35:39.870274+0000 mon.a (mon.0) 1963 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:35:40.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:35:40 smithi144 bash[66654]: audit 2024-04-07T16:35:39.984628+0000 mon.a (mon.0) 1964 : audit [INF] from='client.? 172.21.15.100:0/3358779172' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi100-111196-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi100-111196-10"}]': finished 2024-04-07T16:35:40.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:35:40 smithi144 bash[66654]: cluster 2024-04-07T16:35:39.992392+0000 mon.a (mon.0) 1965 : cluster [DBG] osdmap e1042: 8 total, 8 up, 8 in 2024-04-07T16:35:40.441 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:35:40 smithi100 bash[74942]: cluster 2024-04-07T16:35:38.993837+0000 mgr.y (mgr.25175) 609 : cluster [DBG] pgmap v1225: 105 pgs: 105 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail 2024-04-07T16:35:40.441 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:35:40 smithi100 bash[74942]: cluster 2024-04-07T16:35:38.995558+0000 mon.a (mon.0) 1959 : cluster [DBG] osdmap e1041: 8 total, 8 up, 8 in 2024-04-07T16:35:40.441 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:35:40 smithi100 bash[74942]: audit 2024-04-07T16:35:39.118729+0000 mon.a (mon.0) 1960 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:35:40.441 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:35:40 smithi100 bash[74942]: audit 2024-04-07T16:35:39.858427+0000 mon.a (mon.0) 1961 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:35:40.441 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:35:40 smithi100 bash[74942]: audit 2024-04-07T16:35:39.860227+0000 mon.a (mon.0) 1962 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:35:40.441 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:35:40 smithi100 bash[74942]: audit 2024-04-07T16:35:39.870274+0000 mon.a (mon.0) 1963 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:35:40.442 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:35:40 smithi100 bash[74942]: audit 2024-04-07T16:35:39.984628+0000 mon.a (mon.0) 1964 : audit [INF] from='client.? 172.21.15.100:0/3358779172' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi100-111196-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi100-111196-10"}]': finished 2024-04-07T16:35:40.442 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:35:40 smithi100 bash[74942]: cluster 2024-04-07T16:35:39.992392+0000 mon.a (mon.0) 1965 : cluster [DBG] osdmap e1042: 8 total, 8 up, 8 in 2024-04-07T16:35:40.442 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:35:40 smithi100 bash[76202]: cluster 2024-04-07T16:35:38.993837+0000 mgr.y (mgr.25175) 609 : cluster [DBG] pgmap v1225: 105 pgs: 105 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail 2024-04-07T16:35:40.442 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:35:40 smithi100 bash[76202]: cluster 2024-04-07T16:35:38.995558+0000 mon.a (mon.0) 1959 : cluster [DBG] osdmap e1041: 8 total, 8 up, 8 in 2024-04-07T16:35:40.442 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:35:40 smithi100 bash[76202]: audit 2024-04-07T16:35:39.118729+0000 mon.a (mon.0) 1960 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:35:40.442 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:35:40 smithi100 bash[76202]: audit 2024-04-07T16:35:39.858427+0000 mon.a (mon.0) 1961 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:35:40.442 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:35:40 smithi100 bash[76202]: audit 2024-04-07T16:35:39.860227+0000 mon.a (mon.0) 1962 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:35:40.442 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:35:40 smithi100 bash[76202]: audit 2024-04-07T16:35:39.870274+0000 mon.a (mon.0) 1963 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:35:40.442 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:35:40 smithi100 bash[76202]: audit 2024-04-07T16:35:39.984628+0000 mon.a (mon.0) 1964 : audit [INF] from='client.? 172.21.15.100:0/3358779172' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi100-111196-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi100-111196-10"}]': finished 2024-04-07T16:35:40.442 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:35:40 smithi100 bash[76202]: cluster 2024-04-07T16:35:39.992392+0000 mon.a (mon.0) 1965 : cluster [DBG] osdmap e1042: 8 total, 8 up, 8 in 2024-04-07T16:35:42.306 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:35:41 smithi144 bash[66654]: cluster 2024-04-07T16:35:40.994748+0000 mgr.y (mgr.25175) 610 : cluster [DBG] pgmap v1228: 113 pgs: 1 creating+peering, 7 unknown, 105 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail 2024-04-07T16:35:42.306 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:35:41 smithi144 bash[66654]: cluster 2024-04-07T16:35:40.994772+0000 mon.a (mon.0) 1966 : cluster [DBG] osdmap e1043: 8 total, 8 up, 8 in 2024-04-07T16:35:42.441 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:35:41 smithi100 bash[74942]: cluster 2024-04-07T16:35:40.994748+0000 mgr.y (mgr.25175) 610 : cluster [DBG] pgmap v1228: 113 pgs: 1 creating+peering, 7 unknown, 105 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail 2024-04-07T16:35:42.441 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:35:41 smithi100 bash[74942]: cluster 2024-04-07T16:35:40.994772+0000 mon.a (mon.0) 1966 : cluster [DBG] osdmap e1043: 8 total, 8 up, 8 in 2024-04-07T16:35:42.441 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:35:41 smithi100 bash[76202]: cluster 2024-04-07T16:35:40.994748+0000 mgr.y (mgr.25175) 610 : cluster [DBG] pgmap v1228: 113 pgs: 1 creating+peering, 7 unknown, 105 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail 2024-04-07T16:35:42.441 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:35:41 smithi100 bash[76202]: cluster 2024-04-07T16:35:40.994772+0000 mon.a (mon.0) 1966 : cluster [DBG] osdmap e1043: 8 total, 8 up, 8 in 2024-04-07T16:35:42.607 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:35:42 smithi144 bash[95233]: ts=2024-04-07T16:35:42.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:35:43.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:35:42 smithi144 bash[66654]: cluster 2024-04-07T16:35:41.988564+0000 mon.a (mon.0) 1967 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T16:35:43.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:35:42 smithi144 bash[66654]: cluster 2024-04-07T16:35:42.000883+0000 mon.a (mon.0) 1968 : cluster [DBG] osdmap e1044: 8 total, 8 up, 8 in 2024-04-07T16:35:43.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:35:42 smithi144 bash[66654]: audit 2024-04-07T16:35:42.003127+0000 mon.a (mon.0) 1969 : audit [INF] from='client.? 172.21.15.100:0/3358779172' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi100-111196-10"}]: dispatch 2024-04-07T16:35:43.441 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:35:42 smithi100 bash[74942]: cluster 2024-04-07T16:35:41.988564+0000 mon.a (mon.0) 1967 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T16:35:43.441 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:35:42 smithi100 bash[74942]: cluster 2024-04-07T16:35:42.000883+0000 mon.a (mon.0) 1968 : cluster [DBG] osdmap e1044: 8 total, 8 up, 8 in 2024-04-07T16:35:43.441 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:35:42 smithi100 bash[74942]: audit 2024-04-07T16:35:42.003127+0000 mon.a (mon.0) 1969 : audit [INF] from='client.? 172.21.15.100:0/3358779172' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi100-111196-10"}]: dispatch 2024-04-07T16:35:43.441 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:35:42 smithi100 bash[76202]: cluster 2024-04-07T16:35:41.988564+0000 mon.a (mon.0) 1967 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T16:35:43.441 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:35:42 smithi100 bash[76202]: cluster 2024-04-07T16:35:42.000883+0000 mon.a (mon.0) 1968 : cluster [DBG] osdmap e1044: 8 total, 8 up, 8 in 2024-04-07T16:35:43.441 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:35:42 smithi100 bash[76202]: audit 2024-04-07T16:35:42.003127+0000 mon.a (mon.0) 1969 : audit [INF] from='client.? 172.21.15.100:0/3358779172' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi100-111196-10"}]: dispatch 2024-04-07T16:35:44.011 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] cls_rgw.set_ec (7087 ms) 2024-04-07T16:35:44.011 INFO:tasks.workunit.client.0.smithi100.stdout:[----------] 10 tests from cls_rgw (62474 ms total) 2024-04-07T16:35:44.011 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T16:35:44.011 INFO:tasks.workunit.client.0.smithi100.stdout:[----------] Global test environment tear-down 2024-04-07T16:35:44.011 INFO:tasks.workunit.client.0.smithi100.stdout:[==========] 10 tests from 1 test suite ran. (62474 ms total) 2024-04-07T16:35:44.011 INFO:tasks.workunit.client.0.smithi100.stdout:[ PASSED ] 10 tests. 2024-04-07T16:35:44.015 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-07T16:35:44.016 DEBUG:teuthology.orchestra.run.smithi100:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-07T16:35:44.071 INFO:tasks.workunit:Running workunit cls/test_cls_rgw.sh... 2024-04-07T16:35:44.071 DEBUG:teuthology.orchestra.run.smithi100:workunit test cls/test_cls_rgw.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=reef TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_rgw.sh 2024-04-07T16:35:44.139 INFO:tasks.workunit.client.0.smithi100.stdout:Running main() from gmock_main.cc 2024-04-07T16:35:44.139 INFO:tasks.workunit.client.0.smithi100.stdout:[==========] Running 16 tests from 1 test suite. 2024-04-07T16:35:44.139 INFO:tasks.workunit.client.0.smithi100.stdout:[----------] Global test environment set-up. 2024-04-07T16:35:44.139 INFO:tasks.workunit.client.0.smithi100.stdout:[----------] 16 tests from cls_rgw 2024-04-07T16:35:44.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:35:44 smithi144 bash[66654]: cluster 2024-04-07T16:35:42.995820+0000 mgr.y (mgr.25175) 611 : cluster [DBG] pgmap v1230: 105 pgs: 105 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail 2024-04-07T16:35:44.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:35:44 smithi144 bash[66654]: audit 2024-04-07T16:35:43.000418+0000 mon.a (mon.0) 1970 : audit [INF] from='client.? 172.21.15.100:0/3358779172' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi100-111196-10"}]': finished 2024-04-07T16:35:44.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:35:44 smithi144 bash[66654]: cluster 2024-04-07T16:35:43.007643+0000 mon.a (mon.0) 1971 : cluster [DBG] osdmap e1045: 8 total, 8 up, 8 in 2024-04-07T16:35:44.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:35:44 smithi144 bash[66654]: audit 2024-04-07T16:35:43.008377+0000 mon.a (mon.0) 1972 : audit [INF] from='client.? 172.21.15.100:0/3358779172' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi100-111196-10"}]: dispatch 2024-04-07T16:35:44.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:35:44 smithi144 bash[66654]: cluster 2024-04-07T16:35:43.991188+0000 mon.a (mon.0) 1973 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T16:35:44.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:35:44 smithi144 bash[66654]: cluster 2024-04-07T16:35:43.991247+0000 mon.a (mon.0) 1974 : cluster [INF] Cluster is now healthy 2024-04-07T16:35:44.441 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:35:44 smithi100 bash[74942]: cluster 2024-04-07T16:35:42.995820+0000 mgr.y (mgr.25175) 611 : cluster [DBG] pgmap v1230: 105 pgs: 105 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail 2024-04-07T16:35:44.441 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:35:44 smithi100 bash[74942]: audit 2024-04-07T16:35:43.000418+0000 mon.a (mon.0) 1970 : audit [INF] from='client.? 172.21.15.100:0/3358779172' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi100-111196-10"}]': finished 2024-04-07T16:35:44.441 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:35:44 smithi100 bash[74942]: cluster 2024-04-07T16:35:43.007643+0000 mon.a (mon.0) 1971 : cluster [DBG] osdmap e1045: 8 total, 8 up, 8 in 2024-04-07T16:35:44.441 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:35:44 smithi100 bash[74942]: audit 2024-04-07T16:35:43.008377+0000 mon.a (mon.0) 1972 : audit [INF] from='client.? 172.21.15.100:0/3358779172' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi100-111196-10"}]: dispatch 2024-04-07T16:35:44.441 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:35:44 smithi100 bash[74942]: cluster 2024-04-07T16:35:43.991188+0000 mon.a (mon.0) 1973 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T16:35:44.441 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:35:44 smithi100 bash[74942]: cluster 2024-04-07T16:35:43.991247+0000 mon.a (mon.0) 1974 : cluster [INF] Cluster is now healthy 2024-04-07T16:35:44.441 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:35:44 smithi100 bash[76202]: cluster 2024-04-07T16:35:42.995820+0000 mgr.y (mgr.25175) 611 : cluster [DBG] pgmap v1230: 105 pgs: 105 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail 2024-04-07T16:35:44.441 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:35:44 smithi100 bash[76202]: audit 2024-04-07T16:35:43.000418+0000 mon.a (mon.0) 1970 : audit [INF] from='client.? 172.21.15.100:0/3358779172' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi100-111196-10"}]': finished 2024-04-07T16:35:44.441 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:35:44 smithi100 bash[76202]: cluster 2024-04-07T16:35:43.007643+0000 mon.a (mon.0) 1971 : cluster [DBG] osdmap e1045: 8 total, 8 up, 8 in 2024-04-07T16:35:44.442 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:35:44 smithi100 bash[76202]: audit 2024-04-07T16:35:43.008377+0000 mon.a (mon.0) 1972 : audit [INF] from='client.? 172.21.15.100:0/3358779172' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi100-111196-10"}]: dispatch 2024-04-07T16:35:44.442 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:35:44 smithi100 bash[76202]: cluster 2024-04-07T16:35:43.991188+0000 mon.a (mon.0) 1973 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T16:35:44.442 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:35:44 smithi100 bash[76202]: cluster 2024-04-07T16:35:43.991247+0000 mon.a (mon.0) 1974 : cluster [INF] Cluster is now healthy 2024-04-07T16:35:45.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:35:45 smithi144 bash[66654]: audit 2024-04-07T16:35:44.011270+0000 mon.a (mon.0) 1975 : audit [INF] from='client.? 172.21.15.100:0/3358779172' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi100-111196-10"}]': finished 2024-04-07T16:35:45.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:35:45 smithi144 bash[66654]: cluster 2024-04-07T16:35:44.018263+0000 mon.a (mon.0) 1976 : cluster [DBG] osdmap e1046: 8 total, 8 up, 8 in 2024-04-07T16:35:45.441 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:35:45 smithi100 bash[74942]: audit 2024-04-07T16:35:44.011270+0000 mon.a (mon.0) 1975 : audit [INF] from='client.? 172.21.15.100:0/3358779172' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi100-111196-10"}]': finished 2024-04-07T16:35:45.441 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:35:45 smithi100 bash[74942]: cluster 2024-04-07T16:35:44.018263+0000 mon.a (mon.0) 1976 : cluster [DBG] osdmap e1046: 8 total, 8 up, 8 in 2024-04-07T16:35:45.441 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:35:45 smithi100 bash[76202]: audit 2024-04-07T16:35:44.011270+0000 mon.a (mon.0) 1975 : audit [INF] from='client.? 172.21.15.100:0/3358779172' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi100-111196-10"}]': finished 2024-04-07T16:35:45.441 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:35:45 smithi100 bash[76202]: cluster 2024-04-07T16:35:44.018263+0000 mon.a (mon.0) 1976 : cluster [DBG] osdmap e1046: 8 total, 8 up, 8 in 2024-04-07T16:35:46.021 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] cls_rgw.index_basic 2024-04-07T16:35:46.102 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] cls_rgw.index_basic (82 ms) 2024-04-07T16:35:46.103 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] cls_rgw.index_multiple_obj_writers 2024-04-07T16:35:46.164 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] cls_rgw.index_multiple_obj_writers (61 ms) 2024-04-07T16:35:46.164 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] cls_rgw.index_remove_object 2024-04-07T16:35:46.257 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] cls_rgw.index_remove_object (93 ms) 2024-04-07T16:35:46.257 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] cls_rgw.index_suggest 2024-04-07T16:35:46.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:35:46 smithi144 bash[66654]: cluster 2024-04-07T16:35:44.996648+0000 mgr.y (mgr.25175) 612 : cluster [DBG] pgmap v1233: 105 pgs: 105 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail 2024-04-07T16:35:46.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:35:46 smithi144 bash[66654]: cluster 2024-04-07T16:35:45.024683+0000 mon.a (mon.0) 1977 : cluster [DBG] osdmap e1047: 8 total, 8 up, 8 in 2024-04-07T16:35:46.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:35:46 smithi144 bash[66654]: audit 2024-04-07T16:35:45.037159+0000 mon.c (mon.2) 54 : audit [INF] from='client.? 172.21.15.100:0/900529352' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111390-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:35:46.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:35:46 smithi144 bash[66654]: audit 2024-04-07T16:35:45.047348+0000 mon.a (mon.0) 1978 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111390-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:35:46.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:35:46 smithi144 bash[66654]: audit 2024-04-07T16:35:46.018869+0000 mon.a (mon.0) 1979 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111390-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:35:46.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:35:46 smithi144 bash[66654]: cluster 2024-04-07T16:35:46.026803+0000 mon.a (mon.0) 1980 : cluster [DBG] osdmap e1048: 8 total, 8 up, 8 in 2024-04-07T16:35:46.441 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:35:46 smithi100 bash[74942]: cluster 2024-04-07T16:35:44.996648+0000 mgr.y (mgr.25175) 612 : cluster [DBG] pgmap v1233: 105 pgs: 105 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail 2024-04-07T16:35:46.441 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:35:46 smithi100 bash[74942]: cluster 2024-04-07T16:35:45.024683+0000 mon.a (mon.0) 1977 : cluster [DBG] osdmap e1047: 8 total, 8 up, 8 in 2024-04-07T16:35:46.441 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:35:46 smithi100 bash[74942]: audit 2024-04-07T16:35:45.037159+0000 mon.c (mon.2) 54 : audit [INF] from='client.? 172.21.15.100:0/900529352' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111390-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:35:46.441 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:35:46 smithi100 bash[74942]: audit 2024-04-07T16:35:45.047348+0000 mon.a (mon.0) 1978 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111390-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:35:46.441 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:35:46 smithi100 bash[74942]: audit 2024-04-07T16:35:46.018869+0000 mon.a (mon.0) 1979 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111390-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:35:46.441 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:35:46 smithi100 bash[74942]: cluster 2024-04-07T16:35:46.026803+0000 mon.a (mon.0) 1980 : cluster [DBG] osdmap e1048: 8 total, 8 up, 8 in 2024-04-07T16:35:46.441 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:35:46 smithi100 bash[76202]: cluster 2024-04-07T16:35:44.996648+0000 mgr.y (mgr.25175) 612 : cluster [DBG] pgmap v1233: 105 pgs: 105 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail 2024-04-07T16:35:46.441 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:35:46 smithi100 bash[76202]: cluster 2024-04-07T16:35:45.024683+0000 mon.a (mon.0) 1977 : cluster [DBG] osdmap e1047: 8 total, 8 up, 8 in 2024-04-07T16:35:46.442 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:35:46 smithi100 bash[76202]: audit 2024-04-07T16:35:45.037159+0000 mon.c (mon.2) 54 : audit [INF] from='client.? 172.21.15.100:0/900529352' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111390-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:35:46.442 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:35:46 smithi100 bash[76202]: audit 2024-04-07T16:35:45.047348+0000 mon.a (mon.0) 1978 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111390-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:35:46.442 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:35:46 smithi100 bash[76202]: audit 2024-04-07T16:35:46.018869+0000 mon.a (mon.0) 1979 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111390-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:35:46.442 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:35:46 smithi100 bash[76202]: cluster 2024-04-07T16:35:46.026803+0000 mon.a (mon.0) 1980 : cluster [DBG] osdmap e1048: 8 total, 8 up, 8 in 2024-04-07T16:35:47.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:35:47 smithi144 bash[66654]: audit 2024-04-07T16:35:46.749200+0000 mon.a (mon.0) 1981 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:35:47.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:35:47 smithi144 bash[66654]: cluster 2024-04-07T16:35:47.034185+0000 mon.a (mon.0) 1982 : cluster [DBG] osdmap e1049: 8 total, 8 up, 8 in 2024-04-07T16:35:47.440 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:35:47 smithi100 bash[74942]: audit 2024-04-07T16:35:46.749200+0000 mon.a (mon.0) 1981 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:35:47.441 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:35:47 smithi100 bash[74942]: cluster 2024-04-07T16:35:47.034185+0000 mon.a (mon.0) 1982 : cluster [DBG] osdmap e1049: 8 total, 8 up, 8 in 2024-04-07T16:35:47.441 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:35:47 smithi100 bash[76202]: audit 2024-04-07T16:35:46.749200+0000 mon.a (mon.0) 1981 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:35:47.441 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:35:47 smithi100 bash[76202]: cluster 2024-04-07T16:35:47.034185+0000 mon.a (mon.0) 1982 : cluster [DBG] osdmap e1049: 8 total, 8 up, 8 in 2024-04-07T16:35:48.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:35:48 smithi144 bash[66654]: cluster 2024-04-07T16:35:46.997337+0000 mgr.y (mgr.25175) 613 : cluster [DBG] pgmap v1236: 137 pgs: 4 creating+peering, 28 unknown, 105 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail 2024-04-07T16:35:48.418 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:35:48 smithi100 bash[74942]: cluster 2024-04-07T16:35:46.997337+0000 mgr.y (mgr.25175) 613 : cluster [DBG] pgmap v1236: 137 pgs: 4 creating+peering, 28 unknown, 105 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail 2024-04-07T16:35:48.418 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:35:48 smithi100 bash[76202]: cluster 2024-04-07T16:35:46.997337+0000 mgr.y (mgr.25175) 613 : cluster [DBG] pgmap v1236: 137 pgs: 4 creating+peering, 28 unknown, 105 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail 2024-04-07T16:35:48.502 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] cls_rgw.index_suggest (2245 ms) 2024-04-07T16:35:48.502 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] cls_rgw.index_suggest_complete 2024-04-07T16:35:48.517 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] cls_rgw.index_suggest_complete (15 ms) 2024-04-07T16:35:48.517 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] cls_rgw.index_list 2024-04-07T16:35:48.565 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] cls_rgw.index_list (48 ms) 2024-04-07T16:35:48.565 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] cls_rgw.index_list_delimited 2024-04-07T16:35:48.690 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:35:48 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:35:48] "GET /metrics HTTP/1.1" 200 36354 "" "Prometheus/2.43.0" 2024-04-07T16:35:50.356 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:35:50 smithi144 bash[66654]: cluster 2024-04-07T16:35:48.998433+0000 mgr.y (mgr.25175) 614 : cluster [DBG] pgmap v1238: 137 pgs: 4 creating+peering, 3 unknown, 130 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 179 KiB/s rd, 109 KiB/s wr, 326 op/s 2024-04-07T16:35:50.440 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:35:50 smithi100 bash[74942]: cluster 2024-04-07T16:35:48.998433+0000 mgr.y (mgr.25175) 614 : cluster [DBG] pgmap v1238: 137 pgs: 4 creating+peering, 3 unknown, 130 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 179 KiB/s rd, 109 KiB/s wr, 326 op/s 2024-04-07T16:35:50.441 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:35:50 smithi100 bash[76202]: cluster 2024-04-07T16:35:48.998433+0000 mgr.y (mgr.25175) 614 : cluster [DBG] pgmap v1238: 137 pgs: 4 creating+peering, 3 unknown, 130 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 179 KiB/s rd, 109 KiB/s wr, 326 op/s 2024-04-07T16:35:52.357 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:35:52 smithi144 bash[95233]: ts=2024-04-07T16:35:52.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:35:52.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:35:52 smithi144 bash[66654]: cluster 2024-04-07T16:35:50.999774+0000 mgr.y (mgr.25175) 615 : cluster [DBG] pgmap v1239: 137 pgs: 137 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 163 KiB/s rd, 99 KiB/s wr, 298 op/s 2024-04-07T16:35:52.441 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:35:52 smithi100 bash[74942]: cluster 2024-04-07T16:35:50.999774+0000 mgr.y (mgr.25175) 615 : cluster [DBG] pgmap v1239: 137 pgs: 137 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 163 KiB/s rd, 99 KiB/s wr, 298 op/s 2024-04-07T16:35:52.441 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:35:52 smithi100 bash[76202]: cluster 2024-04-07T16:35:50.999774+0000 mgr.y (mgr.25175) 615 : cluster [DBG] pgmap v1239: 137 pgs: 137 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 163 KiB/s rd, 99 KiB/s wr, 298 op/s 2024-04-07T16:35:54.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:35:54 smithi144 bash[66654]: cluster 2024-04-07T16:35:53.001045+0000 mgr.y (mgr.25175) 616 : cluster [DBG] pgmap v1240: 137 pgs: 137 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 174 KiB/s rd, 94 KiB/s wr, 301 op/s 2024-04-07T16:35:54.440 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:35:54 smithi100 bash[74942]: cluster 2024-04-07T16:35:53.001045+0000 mgr.y (mgr.25175) 616 : cluster [DBG] pgmap v1240: 137 pgs: 137 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 174 KiB/s rd, 94 KiB/s wr, 301 op/s 2024-04-07T16:35:54.441 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:35:54 smithi100 bash[76202]: cluster 2024-04-07T16:35:53.001045+0000 mgr.y (mgr.25175) 616 : cluster [DBG] pgmap v1240: 137 pgs: 137 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 174 KiB/s rd, 94 KiB/s wr, 301 op/s 2024-04-07T16:35:56.441 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:35:56 smithi100 bash[74942]: cluster 2024-04-07T16:35:55.002188+0000 mgr.y (mgr.25175) 617 : cluster [DBG] pgmap v1241: 137 pgs: 137 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 465 KiB/s rd, 291 KiB/s wr, 889 op/s 2024-04-07T16:35:56.441 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:35:56 smithi100 bash[76202]: cluster 2024-04-07T16:35:55.002188+0000 mgr.y (mgr.25175) 617 : cluster [DBG] pgmap v1241: 137 pgs: 137 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 465 KiB/s rd, 291 KiB/s wr, 889 op/s 2024-04-07T16:35:56.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:35:56 smithi144 bash[66654]: cluster 2024-04-07T16:35:55.002188+0000 mgr.y (mgr.25175) 617 : cluster [DBG] pgmap v1241: 137 pgs: 137 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 465 KiB/s rd, 291 KiB/s wr, 889 op/s 2024-04-07T16:35:58.417 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:35:58 smithi100 bash[74942]: cluster 2024-04-07T16:35:57.002966+0000 mgr.y (mgr.25175) 618 : cluster [DBG] pgmap v1242: 137 pgs: 137 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 418 KiB/s rd, 261 KiB/s wr, 798 op/s 2024-04-07T16:35:58.417 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:35:58 smithi100 bash[76202]: cluster 2024-04-07T16:35:57.002966+0000 mgr.y (mgr.25175) 618 : cluster [DBG] pgmap v1242: 137 pgs: 137 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 418 KiB/s rd, 261 KiB/s wr, 798 op/s 2024-04-07T16:35:58.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:35:58 smithi144 bash[66654]: cluster 2024-04-07T16:35:57.002966+0000 mgr.y (mgr.25175) 618 : cluster [DBG] pgmap v1242: 137 pgs: 137 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 418 KiB/s rd, 261 KiB/s wr, 798 op/s 2024-04-07T16:35:58.691 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:35:58 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:35:58] "GET /metrics HTTP/1.1" 200 36354 "" "Prometheus/2.43.0" 2024-04-07T16:36:00.441 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:36:00 smithi100 bash[74942]: cluster 2024-04-07T16:35:59.004308+0000 mgr.y (mgr.25175) 619 : cluster [DBG] pgmap v1243: 137 pgs: 137 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 525 KiB/s rd, 336 KiB/s wr, 1.02k op/s 2024-04-07T16:36:00.441 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:36:00 smithi100 bash[76202]: cluster 2024-04-07T16:35:59.004308+0000 mgr.y (mgr.25175) 619 : cluster [DBG] pgmap v1243: 137 pgs: 137 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 525 KiB/s rd, 336 KiB/s wr, 1.02k op/s 2024-04-07T16:36:00.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:36:00 smithi144 bash[66654]: cluster 2024-04-07T16:35:59.004308+0000 mgr.y (mgr.25175) 619 : cluster [DBG] pgmap v1243: 137 pgs: 137 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 525 KiB/s rd, 336 KiB/s wr, 1.02k op/s 2024-04-07T16:36:02.441 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:36:02 smithi100 bash[74942]: cluster 2024-04-07T16:36:01.005081+0000 mgr.y (mgr.25175) 620 : cluster [DBG] pgmap v1244: 137 pgs: 137 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 450 KiB/s rd, 290 KiB/s wr, 881 op/s 2024-04-07T16:36:02.441 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:36:02 smithi100 bash[74942]: audit 2024-04-07T16:36:01.749476+0000 mon.a (mon.0) 1983 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:36:02.441 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:36:02 smithi100 bash[76202]: cluster 2024-04-07T16:36:01.005081+0000 mgr.y (mgr.25175) 620 : cluster [DBG] pgmap v1244: 137 pgs: 137 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 450 KiB/s rd, 290 KiB/s wr, 881 op/s 2024-04-07T16:36:02.441 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:36:02 smithi100 bash[76202]: audit 2024-04-07T16:36:01.749476+0000 mon.a (mon.0) 1983 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:36:02.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:36:02 smithi144 bash[66654]: cluster 2024-04-07T16:36:01.005081+0000 mgr.y (mgr.25175) 620 : cluster [DBG] pgmap v1244: 137 pgs: 137 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 450 KiB/s rd, 290 KiB/s wr, 881 op/s 2024-04-07T16:36:02.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:36:02 smithi144 bash[66654]: audit 2024-04-07T16:36:01.749476+0000 mon.a (mon.0) 1983 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:36:02.607 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:36:02 smithi144 bash[95233]: ts=2024-04-07T16:36:02.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:36:04.441 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:36:04 smithi100 bash[74942]: cluster 2024-04-07T16:36:03.006274+0000 mgr.y (mgr.25175) 621 : cluster [DBG] pgmap v1245: 137 pgs: 137 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 442 KiB/s rd, 286 KiB/s wr, 868 op/s 2024-04-07T16:36:04.441 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:36:04 smithi100 bash[76202]: cluster 2024-04-07T16:36:03.006274+0000 mgr.y (mgr.25175) 621 : cluster [DBG] pgmap v1245: 137 pgs: 137 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 442 KiB/s rd, 286 KiB/s wr, 868 op/s 2024-04-07T16:36:04.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:36:04 smithi144 bash[66654]: cluster 2024-04-07T16:36:03.006274+0000 mgr.y (mgr.25175) 621 : cluster [DBG] pgmap v1245: 137 pgs: 137 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 442 KiB/s rd, 286 KiB/s wr, 868 op/s 2024-04-07T16:36:06.441 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:36:06 smithi100 bash[74942]: cluster 2024-04-07T16:36:05.007269+0000 mgr.y (mgr.25175) 622 : cluster [DBG] pgmap v1246: 137 pgs: 137 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 561 KiB/s rd, 374 KiB/s wr, 1.12k op/s 2024-04-07T16:36:06.441 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:36:06 smithi100 bash[76202]: cluster 2024-04-07T16:36:05.007269+0000 mgr.y (mgr.25175) 622 : cluster [DBG] pgmap v1246: 137 pgs: 137 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 561 KiB/s rd, 374 KiB/s wr, 1.12k op/s 2024-04-07T16:36:06.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:36:06 smithi144 bash[66654]: cluster 2024-04-07T16:36:05.007269+0000 mgr.y (mgr.25175) 622 : cluster [DBG] pgmap v1246: 137 pgs: 137 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 561 KiB/s rd, 374 KiB/s wr, 1.12k op/s 2024-04-07T16:36:08.418 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:36:08 smithi100 bash[74942]: cluster 2024-04-07T16:36:07.007952+0000 mgr.y (mgr.25175) 623 : cluster [DBG] pgmap v1247: 137 pgs: 137 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 328 KiB/s rd, 219 KiB/s wr, 656 op/s 2024-04-07T16:36:08.418 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:36:08 smithi100 bash[76202]: cluster 2024-04-07T16:36:07.007952+0000 mgr.y (mgr.25175) 623 : cluster [DBG] pgmap v1247: 137 pgs: 137 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 328 KiB/s rd, 219 KiB/s wr, 656 op/s 2024-04-07T16:36:08.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:36:08 smithi144 bash[66654]: cluster 2024-04-07T16:36:07.007952+0000 mgr.y (mgr.25175) 623 : cluster [DBG] pgmap v1247: 137 pgs: 137 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 328 KiB/s rd, 219 KiB/s wr, 656 op/s 2024-04-07T16:36:08.691 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:36:08 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:36:08] "GET /metrics HTTP/1.1" 200 36373 "" "Prometheus/2.43.0" 2024-04-07T16:36:10.441 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:36:10 smithi100 bash[74942]: cluster 2024-04-07T16:36:09.009263+0000 mgr.y (mgr.25175) 624 : cluster [DBG] pgmap v1248: 137 pgs: 137 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 464 KiB/s rd, 309 KiB/s wr, 928 op/s 2024-04-07T16:36:10.441 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:36:10 smithi100 bash[76202]: cluster 2024-04-07T16:36:09.009263+0000 mgr.y (mgr.25175) 624 : cluster [DBG] pgmap v1248: 137 pgs: 137 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 464 KiB/s rd, 309 KiB/s wr, 928 op/s 2024-04-07T16:36:10.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:36:10 smithi144 bash[66654]: cluster 2024-04-07T16:36:09.009263+0000 mgr.y (mgr.25175) 624 : cluster [DBG] pgmap v1248: 137 pgs: 137 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 464 KiB/s rd, 309 KiB/s wr, 928 op/s 2024-04-07T16:36:12.441 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:36:12 smithi100 bash[74942]: cluster 2024-04-07T16:36:11.010088+0000 mgr.y (mgr.25175) 625 : cluster [DBG] pgmap v1249: 137 pgs: 137 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 288 KiB/s rd, 192 KiB/s wr, 576 op/s 2024-04-07T16:36:12.441 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:36:12 smithi100 bash[76202]: cluster 2024-04-07T16:36:11.010088+0000 mgr.y (mgr.25175) 625 : cluster [DBG] pgmap v1249: 137 pgs: 137 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 288 KiB/s rd, 192 KiB/s wr, 576 op/s 2024-04-07T16:36:12.607 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:36:12 smithi144 bash[95233]: ts=2024-04-07T16:36:12.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:36:12.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:36:12 smithi144 bash[66654]: cluster 2024-04-07T16:36:11.010088+0000 mgr.y (mgr.25175) 625 : cluster [DBG] pgmap v1249: 137 pgs: 137 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 288 KiB/s rd, 192 KiB/s wr, 576 op/s 2024-04-07T16:36:14.441 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:36:14 smithi100 bash[74942]: cluster 2024-04-07T16:36:13.011363+0000 mgr.y (mgr.25175) 626 : cluster [DBG] pgmap v1250: 137 pgs: 137 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 288 KiB/s rd, 192 KiB/s wr, 576 op/s 2024-04-07T16:36:14.441 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:36:14 smithi100 bash[76202]: cluster 2024-04-07T16:36:13.011363+0000 mgr.y (mgr.25175) 626 : cluster [DBG] pgmap v1250: 137 pgs: 137 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 288 KiB/s rd, 192 KiB/s wr, 576 op/s 2024-04-07T16:36:14.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:36:14 smithi144 bash[66654]: cluster 2024-04-07T16:36:13.011363+0000 mgr.y (mgr.25175) 626 : cluster [DBG] pgmap v1250: 137 pgs: 137 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 288 KiB/s rd, 192 KiB/s wr, 576 op/s 2024-04-07T16:36:16.441 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:36:16 smithi100 bash[74942]: cluster 2024-04-07T16:36:15.012361+0000 mgr.y (mgr.25175) 627 : cluster [DBG] pgmap v1251: 137 pgs: 137 active+clean; 583 KiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 413 KiB/s rd, 275 KiB/s wr, 825 op/s 2024-04-07T16:36:16.441 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:36:16 smithi100 bash[76202]: cluster 2024-04-07T16:36:15.012361+0000 mgr.y (mgr.25175) 627 : cluster [DBG] pgmap v1251: 137 pgs: 137 active+clean; 583 KiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 413 KiB/s rd, 275 KiB/s wr, 825 op/s 2024-04-07T16:36:16.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:36:16 smithi144 bash[66654]: cluster 2024-04-07T16:36:15.012361+0000 mgr.y (mgr.25175) 627 : cluster [DBG] pgmap v1251: 137 pgs: 137 active+clean; 583 KiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 413 KiB/s rd, 275 KiB/s wr, 825 op/s 2024-04-07T16:36:17.441 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:36:17 smithi100 bash[74942]: audit 2024-04-07T16:36:16.749944+0000 mon.a (mon.0) 1984 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:36:17.441 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:36:17 smithi100 bash[76202]: audit 2024-04-07T16:36:16.749944+0000 mon.a (mon.0) 1984 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:36:17.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:36:17 smithi144 bash[66654]: audit 2024-04-07T16:36:16.749944+0000 mon.a (mon.0) 1984 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:36:18.441 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:36:18 smithi100 bash[74942]: cluster 2024-04-07T16:36:17.012953+0000 mgr.y (mgr.25175) 628 : cluster [DBG] pgmap v1252: 137 pgs: 137 active+clean; 583 KiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 260 KiB/s rd, 173 KiB/s wr, 520 op/s 2024-04-07T16:36:18.441 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:36:18 smithi100 bash[76202]: cluster 2024-04-07T16:36:17.012953+0000 mgr.y (mgr.25175) 628 : cluster [DBG] pgmap v1252: 137 pgs: 137 active+clean; 583 KiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 260 KiB/s rd, 173 KiB/s wr, 520 op/s 2024-04-07T16:36:18.441 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:36:18 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:36:18] "GET /metrics HTTP/1.1" 200 36377 "" "Prometheus/2.43.0" 2024-04-07T16:36:18.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:36:18 smithi144 bash[66654]: cluster 2024-04-07T16:36:17.012953+0000 mgr.y (mgr.25175) 628 : cluster [DBG] pgmap v1252: 137 pgs: 137 active+clean; 583 KiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 260 KiB/s rd, 173 KiB/s wr, 520 op/s 2024-04-07T16:36:20.441 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:36:20 smithi100 bash[74942]: cluster 2024-04-07T16:36:19.013938+0000 mgr.y (mgr.25175) 629 : cluster [DBG] pgmap v1253: 137 pgs: 137 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 377 KiB/s rd, 251 KiB/s wr, 753 op/s 2024-04-07T16:36:20.441 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:36:20 smithi100 bash[76202]: cluster 2024-04-07T16:36:19.013938+0000 mgr.y (mgr.25175) 629 : cluster [DBG] pgmap v1253: 137 pgs: 137 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 377 KiB/s rd, 251 KiB/s wr, 753 op/s 2024-04-07T16:36:20.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:36:20 smithi144 bash[66654]: cluster 2024-04-07T16:36:19.013938+0000 mgr.y (mgr.25175) 629 : cluster [DBG] pgmap v1253: 137 pgs: 137 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 377 KiB/s rd, 251 KiB/s wr, 753 op/s 2024-04-07T16:36:22.441 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:36:22 smithi100 bash[74942]: cluster 2024-04-07T16:36:21.014979+0000 mgr.y (mgr.25175) 630 : cluster [DBG] pgmap v1254: 137 pgs: 137 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 241 KiB/s rd, 161 KiB/s wr, 481 op/s 2024-04-07T16:36:22.441 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:36:22 smithi100 bash[76202]: cluster 2024-04-07T16:36:21.014979+0000 mgr.y (mgr.25175) 630 : cluster [DBG] pgmap v1254: 137 pgs: 137 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 241 KiB/s rd, 161 KiB/s wr, 481 op/s 2024-04-07T16:36:22.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:36:22 smithi144 bash[66654]: cluster 2024-04-07T16:36:21.014979+0000 mgr.y (mgr.25175) 630 : cluster [DBG] pgmap v1254: 137 pgs: 137 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 241 KiB/s rd, 161 KiB/s wr, 481 op/s 2024-04-07T16:36:22.607 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:36:22 smithi144 bash[95233]: ts=2024-04-07T16:36:22.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:36:24.441 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:36:24 smithi100 bash[74942]: cluster 2024-04-07T16:36:23.016060+0000 mgr.y (mgr.25175) 631 : cluster [DBG] pgmap v1255: 137 pgs: 137 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 241 KiB/s rd, 161 KiB/s wr, 481 op/s 2024-04-07T16:36:24.441 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:36:24 smithi100 bash[76202]: cluster 2024-04-07T16:36:23.016060+0000 mgr.y (mgr.25175) 631 : cluster [DBG] pgmap v1255: 137 pgs: 137 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 241 KiB/s rd, 161 KiB/s wr, 481 op/s 2024-04-07T16:36:24.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:36:24 smithi144 bash[66654]: cluster 2024-04-07T16:36:23.016060+0000 mgr.y (mgr.25175) 631 : cluster [DBG] pgmap v1255: 137 pgs: 137 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 241 KiB/s rd, 161 KiB/s wr, 481 op/s 2024-04-07T16:36:26.441 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:36:26 smithi100 bash[76202]: cluster 2024-04-07T16:36:25.017145+0000 mgr.y (mgr.25175) 632 : cluster [DBG] pgmap v1256: 137 pgs: 137 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 358 KiB/s rd, 239 KiB/s wr, 715 op/s 2024-04-07T16:36:26.441 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:36:26 smithi100 bash[74942]: cluster 2024-04-07T16:36:25.017145+0000 mgr.y (mgr.25175) 632 : cluster [DBG] pgmap v1256: 137 pgs: 137 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 358 KiB/s rd, 239 KiB/s wr, 715 op/s 2024-04-07T16:36:26.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:36:26 smithi144 bash[66654]: cluster 2024-04-07T16:36:25.017145+0000 mgr.y (mgr.25175) 632 : cluster [DBG] pgmap v1256: 137 pgs: 137 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 358 KiB/s rd, 239 KiB/s wr, 715 op/s 2024-04-07T16:36:28.441 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:36:28 smithi100 bash[74942]: cluster 2024-04-07T16:36:27.018051+0000 mgr.y (mgr.25175) 633 : cluster [DBG] pgmap v1257: 137 pgs: 137 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 233 KiB/s rd, 156 KiB/s wr, 466 op/s 2024-04-07T16:36:28.441 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:36:28 smithi100 bash[76202]: cluster 2024-04-07T16:36:27.018051+0000 mgr.y (mgr.25175) 633 : cluster [DBG] pgmap v1257: 137 pgs: 137 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 233 KiB/s rd, 156 KiB/s wr, 466 op/s 2024-04-07T16:36:28.441 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:36:28 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:36:28] "GET /metrics HTTP/1.1" 200 36377 "" "Prometheus/2.43.0" 2024-04-07T16:36:28.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:36:28 smithi144 bash[66654]: cluster 2024-04-07T16:36:27.018051+0000 mgr.y (mgr.25175) 633 : cluster [DBG] pgmap v1257: 137 pgs: 137 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 233 KiB/s rd, 156 KiB/s wr, 466 op/s 2024-04-07T16:36:30.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:36:30 smithi144 bash[66654]: cluster 2024-04-07T16:36:29.019205+0000 mgr.y (mgr.25175) 634 : cluster [DBG] pgmap v1258: 137 pgs: 137 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 348 KiB/s rd, 232 KiB/s wr, 695 op/s 2024-04-07T16:36:30.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:36:30 smithi100 bash[74942]: cluster 2024-04-07T16:36:29.019205+0000 mgr.y (mgr.25175) 634 : cluster [DBG] pgmap v1258: 137 pgs: 137 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 348 KiB/s rd, 232 KiB/s wr, 695 op/s 2024-04-07T16:36:30.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:36:30 smithi100 bash[76202]: cluster 2024-04-07T16:36:29.019205+0000 mgr.y (mgr.25175) 634 : cluster [DBG] pgmap v1258: 137 pgs: 137 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 348 KiB/s rd, 232 KiB/s wr, 695 op/s 2024-04-07T16:36:32.606 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:36:32 smithi144 bash[95233]: ts=2024-04-07T16:36:32.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:36:32.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:36:32 smithi144 bash[66654]: cluster 2024-04-07T16:36:31.020003+0000 mgr.y (mgr.25175) 635 : cluster [DBG] pgmap v1259: 137 pgs: 137 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 231 KiB/s rd, 154 KiB/s wr, 462 op/s 2024-04-07T16:36:32.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:36:32 smithi144 bash[66654]: audit 2024-04-07T16:36:31.750927+0000 mon.a (mon.0) 1985 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:36:32.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:36:32 smithi100 bash[74942]: cluster 2024-04-07T16:36:31.020003+0000 mgr.y (mgr.25175) 635 : cluster [DBG] pgmap v1259: 137 pgs: 137 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 231 KiB/s rd, 154 KiB/s wr, 462 op/s 2024-04-07T16:36:32.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:36:32 smithi100 bash[74942]: audit 2024-04-07T16:36:31.750927+0000 mon.a (mon.0) 1985 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:36:32.692 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:36:32 smithi100 bash[76202]: cluster 2024-04-07T16:36:31.020003+0000 mgr.y (mgr.25175) 635 : cluster [DBG] pgmap v1259: 137 pgs: 137 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 231 KiB/s rd, 154 KiB/s wr, 462 op/s 2024-04-07T16:36:32.692 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:36:32 smithi100 bash[76202]: audit 2024-04-07T16:36:31.750927+0000 mon.a (mon.0) 1985 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:36:34.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:36:34 smithi144 bash[66654]: cluster 2024-04-07T16:36:33.021294+0000 mgr.y (mgr.25175) 636 : cluster [DBG] pgmap v1260: 137 pgs: 137 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 231 KiB/s rd, 154 KiB/s wr, 462 op/s 2024-04-07T16:36:34.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:36:34 smithi100 bash[74942]: cluster 2024-04-07T16:36:33.021294+0000 mgr.y (mgr.25175) 636 : cluster [DBG] pgmap v1260: 137 pgs: 137 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 231 KiB/s rd, 154 KiB/s wr, 462 op/s 2024-04-07T16:36:34.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:36:34 smithi100 bash[76202]: cluster 2024-04-07T16:36:33.021294+0000 mgr.y (mgr.25175) 636 : cluster [DBG] pgmap v1260: 137 pgs: 137 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 231 KiB/s rd, 154 KiB/s wr, 462 op/s 2024-04-07T16:36:36.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:36:36 smithi144 bash[66654]: cluster 2024-04-07T16:36:35.022305+0000 mgr.y (mgr.25175) 637 : cluster [DBG] pgmap v1261: 137 pgs: 137 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 343 KiB/s rd, 229 KiB/s wr, 686 op/s 2024-04-07T16:36:36.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:36:36 smithi100 bash[74942]: cluster 2024-04-07T16:36:35.022305+0000 mgr.y (mgr.25175) 637 : cluster [DBG] pgmap v1261: 137 pgs: 137 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 343 KiB/s rd, 229 KiB/s wr, 686 op/s 2024-04-07T16:36:36.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:36:36 smithi100 bash[76202]: cluster 2024-04-07T16:36:35.022305+0000 mgr.y (mgr.25175) 637 : cluster [DBG] pgmap v1261: 137 pgs: 137 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 343 KiB/s rd, 229 KiB/s wr, 686 op/s 2024-04-07T16:36:38.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:36:38 smithi144 bash[66654]: cluster 2024-04-07T16:36:37.022941+0000 mgr.y (mgr.25175) 638 : cluster [DBG] pgmap v1262: 137 pgs: 137 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 226 KiB/s rd, 151 KiB/s wr, 452 op/s 2024-04-07T16:36:38.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:36:38 smithi100 bash[74942]: cluster 2024-04-07T16:36:37.022941+0000 mgr.y (mgr.25175) 638 : cluster [DBG] pgmap v1262: 137 pgs: 137 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 226 KiB/s rd, 151 KiB/s wr, 452 op/s 2024-04-07T16:36:38.691 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:36:38 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:36:38] "GET /metrics HTTP/1.1" 200 36381 "" "Prometheus/2.43.0" 2024-04-07T16:36:38.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:36:38 smithi100 bash[76202]: cluster 2024-04-07T16:36:37.022941+0000 mgr.y (mgr.25175) 638 : cluster [DBG] pgmap v1262: 137 pgs: 137 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 226 KiB/s rd, 151 KiB/s wr, 452 op/s 2024-04-07T16:36:40.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:36:40 smithi144 bash[66654]: cluster 2024-04-07T16:36:39.024309+0000 mgr.y (mgr.25175) 639 : cluster [DBG] pgmap v1263: 137 pgs: 137 active+clean; 583 KiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 342 KiB/s rd, 228 KiB/s wr, 683 op/s 2024-04-07T16:36:40.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:36:40 smithi144 bash[66654]: audit 2024-04-07T16:36:39.962369+0000 mon.a (mon.0) 1986 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:36:40.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:36:40 smithi100 bash[74942]: cluster 2024-04-07T16:36:39.024309+0000 mgr.y (mgr.25175) 639 : cluster [DBG] pgmap v1263: 137 pgs: 137 active+clean; 583 KiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 342 KiB/s rd, 228 KiB/s wr, 683 op/s 2024-04-07T16:36:40.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:36:40 smithi100 bash[74942]: audit 2024-04-07T16:36:39.962369+0000 mon.a (mon.0) 1986 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:36:40.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:36:40 smithi100 bash[76202]: cluster 2024-04-07T16:36:39.024309+0000 mgr.y (mgr.25175) 639 : cluster [DBG] pgmap v1263: 137 pgs: 137 active+clean; 583 KiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 342 KiB/s rd, 228 KiB/s wr, 683 op/s 2024-04-07T16:36:40.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:36:40 smithi100 bash[76202]: audit 2024-04-07T16:36:39.962369+0000 mon.a (mon.0) 1986 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:36:41.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:36:41 smithi144 bash[66654]: audit 2024-04-07T16:36:40.700856+0000 mon.a (mon.0) 1987 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:36:41.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:36:41 smithi144 bash[66654]: audit 2024-04-07T16:36:40.701793+0000 mon.a (mon.0) 1988 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:36:41.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:36:41 smithi144 bash[66654]: audit 2024-04-07T16:36:40.707501+0000 mon.a (mon.0) 1989 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:36:41.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:36:41 smithi100 bash[74942]: audit 2024-04-07T16:36:40.700856+0000 mon.a (mon.0) 1987 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:36:41.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:36:41 smithi100 bash[74942]: audit 2024-04-07T16:36:40.701793+0000 mon.a (mon.0) 1988 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:36:41.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:36:41 smithi100 bash[74942]: audit 2024-04-07T16:36:40.707501+0000 mon.a (mon.0) 1989 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:36:41.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:36:41 smithi100 bash[76202]: audit 2024-04-07T16:36:40.700856+0000 mon.a (mon.0) 1987 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:36:41.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:36:41 smithi100 bash[76202]: audit 2024-04-07T16:36:40.701793+0000 mon.a (mon.0) 1988 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:36:41.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:36:41 smithi100 bash[76202]: audit 2024-04-07T16:36:40.707501+0000 mon.a (mon.0) 1989 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:36:42.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:36:42 smithi144 bash[66654]: cluster 2024-04-07T16:36:41.025150+0000 mgr.y (mgr.25175) 640 : cluster [DBG] pgmap v1264: 137 pgs: 137 active+clean; 583 KiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 227 KiB/s rd, 152 KiB/s wr, 454 op/s 2024-04-07T16:36:42.607 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:36:42 smithi144 bash[95233]: ts=2024-04-07T16:36:42.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:36:42.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:36:42 smithi100 bash[74942]: cluster 2024-04-07T16:36:41.025150+0000 mgr.y (mgr.25175) 640 : cluster [DBG] pgmap v1264: 137 pgs: 137 active+clean; 583 KiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 227 KiB/s rd, 152 KiB/s wr, 454 op/s 2024-04-07T16:36:42.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:36:42 smithi100 bash[76202]: cluster 2024-04-07T16:36:41.025150+0000 mgr.y (mgr.25175) 640 : cluster [DBG] pgmap v1264: 137 pgs: 137 active+clean; 583 KiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 227 KiB/s rd, 152 KiB/s wr, 454 op/s 2024-04-07T16:36:44.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:36:44 smithi144 bash[66654]: cluster 2024-04-07T16:36:43.026213+0000 mgr.y (mgr.25175) 641 : cluster [DBG] pgmap v1265: 137 pgs: 137 active+clean; 583 KiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 227 KiB/s rd, 152 KiB/s wr, 454 op/s 2024-04-07T16:36:44.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:36:44 smithi100 bash[74942]: cluster 2024-04-07T16:36:43.026213+0000 mgr.y (mgr.25175) 641 : cluster [DBG] pgmap v1265: 137 pgs: 137 active+clean; 583 KiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 227 KiB/s rd, 152 KiB/s wr, 454 op/s 2024-04-07T16:36:44.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:36:44 smithi100 bash[76202]: cluster 2024-04-07T16:36:43.026213+0000 mgr.y (mgr.25175) 641 : cluster [DBG] pgmap v1265: 137 pgs: 137 active+clean; 583 KiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 227 KiB/s rd, 152 KiB/s wr, 454 op/s 2024-04-07T16:36:46.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:36:46 smithi144 bash[66654]: cluster 2024-04-07T16:36:45.027185+0000 mgr.y (mgr.25175) 642 : cluster [DBG] pgmap v1266: 137 pgs: 137 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 344 KiB/s rd, 230 KiB/s wr, 688 op/s 2024-04-07T16:36:46.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:36:46 smithi100 bash[74942]: cluster 2024-04-07T16:36:45.027185+0000 mgr.y (mgr.25175) 642 : cluster [DBG] pgmap v1266: 137 pgs: 137 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 344 KiB/s rd, 230 KiB/s wr, 688 op/s 2024-04-07T16:36:46.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:36:46 smithi100 bash[76202]: cluster 2024-04-07T16:36:45.027185+0000 mgr.y (mgr.25175) 642 : cluster [DBG] pgmap v1266: 137 pgs: 137 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 344 KiB/s rd, 230 KiB/s wr, 688 op/s 2024-04-07T16:36:47.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:36:47 smithi144 bash[66654]: audit 2024-04-07T16:36:46.750786+0000 mon.a (mon.0) 1990 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:36:47.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:36:47 smithi100 bash[74942]: audit 2024-04-07T16:36:46.750786+0000 mon.a (mon.0) 1990 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:36:47.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:36:47 smithi100 bash[76202]: audit 2024-04-07T16:36:46.750786+0000 mon.a (mon.0) 1990 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:36:48.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:36:48 smithi144 bash[66654]: cluster 2024-04-07T16:36:47.027899+0000 mgr.y (mgr.25175) 643 : cluster [DBG] pgmap v1267: 137 pgs: 137 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 233 KiB/s rd, 155 KiB/s wr, 465 op/s 2024-04-07T16:36:48.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:36:48 smithi100 bash[74942]: cluster 2024-04-07T16:36:47.027899+0000 mgr.y (mgr.25175) 643 : cluster [DBG] pgmap v1267: 137 pgs: 137 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 233 KiB/s rd, 155 KiB/s wr, 465 op/s 2024-04-07T16:36:48.691 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:36:48 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:36:48] "GET /metrics HTTP/1.1" 200 36378 "" "Prometheus/2.43.0" 2024-04-07T16:36:48.692 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:36:48 smithi100 bash[76202]: cluster 2024-04-07T16:36:47.027899+0000 mgr.y (mgr.25175) 643 : cluster [DBG] pgmap v1267: 137 pgs: 137 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 233 KiB/s rd, 155 KiB/s wr, 465 op/s 2024-04-07T16:36:50.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:36:50 smithi144 bash[66654]: cluster 2024-04-07T16:36:49.029274+0000 mgr.y (mgr.25175) 644 : cluster [DBG] pgmap v1268: 137 pgs: 137 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 345 KiB/s rd, 230 KiB/s wr, 690 op/s 2024-04-07T16:36:50.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:36:50 smithi100 bash[74942]: cluster 2024-04-07T16:36:49.029274+0000 mgr.y (mgr.25175) 644 : cluster [DBG] pgmap v1268: 137 pgs: 137 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 345 KiB/s rd, 230 KiB/s wr, 690 op/s 2024-04-07T16:36:50.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:36:50 smithi100 bash[76202]: cluster 2024-04-07T16:36:49.029274+0000 mgr.y (mgr.25175) 644 : cluster [DBG] pgmap v1268: 137 pgs: 137 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 345 KiB/s rd, 230 KiB/s wr, 690 op/s 2024-04-07T16:36:52.607 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:36:52 smithi144 bash[95233]: ts=2024-04-07T16:36:52.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:36:52.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:36:52 smithi144 bash[66654]: cluster 2024-04-07T16:36:51.030308+0000 mgr.y (mgr.25175) 645 : cluster [DBG] pgmap v1269: 137 pgs: 137 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 230 KiB/s rd, 153 KiB/s wr, 459 op/s 2024-04-07T16:36:52.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:36:52 smithi100 bash[74942]: cluster 2024-04-07T16:36:51.030308+0000 mgr.y (mgr.25175) 645 : cluster [DBG] pgmap v1269: 137 pgs: 137 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 230 KiB/s rd, 153 KiB/s wr, 459 op/s 2024-04-07T16:36:52.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:36:52 smithi100 bash[76202]: cluster 2024-04-07T16:36:51.030308+0000 mgr.y (mgr.25175) 645 : cluster [DBG] pgmap v1269: 137 pgs: 137 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 230 KiB/s rd, 153 KiB/s wr, 459 op/s 2024-04-07T16:36:54.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:36:54 smithi144 bash[66654]: cluster 2024-04-07T16:36:53.031310+0000 mgr.y (mgr.25175) 646 : cluster [DBG] pgmap v1270: 137 pgs: 137 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 230 KiB/s rd, 153 KiB/s wr, 459 op/s 2024-04-07T16:36:54.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:36:54 smithi100 bash[74942]: cluster 2024-04-07T16:36:53.031310+0000 mgr.y (mgr.25175) 646 : cluster [DBG] pgmap v1270: 137 pgs: 137 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 230 KiB/s rd, 153 KiB/s wr, 459 op/s 2024-04-07T16:36:54.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:36:54 smithi100 bash[76202]: cluster 2024-04-07T16:36:53.031310+0000 mgr.y (mgr.25175) 646 : cluster [DBG] pgmap v1270: 137 pgs: 137 active+clean; 583 KiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 230 KiB/s rd, 153 KiB/s wr, 459 op/s 2024-04-07T16:36:56.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:36:56 smithi144 bash[66654]: cluster 2024-04-07T16:36:55.032318+0000 mgr.y (mgr.25175) 647 : cluster [DBG] pgmap v1271: 137 pgs: 137 active+clean; 583 KiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 345 KiB/s rd, 230 KiB/s wr, 690 op/s 2024-04-07T16:36:56.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:36:56 smithi100 bash[74942]: cluster 2024-04-07T16:36:55.032318+0000 mgr.y (mgr.25175) 647 : cluster [DBG] pgmap v1271: 137 pgs: 137 active+clean; 583 KiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 345 KiB/s rd, 230 KiB/s wr, 690 op/s 2024-04-07T16:36:56.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:36:56 smithi100 bash[76202]: cluster 2024-04-07T16:36:55.032318+0000 mgr.y (mgr.25175) 647 : cluster [DBG] pgmap v1271: 137 pgs: 137 active+clean; 583 KiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 345 KiB/s rd, 230 KiB/s wr, 690 op/s 2024-04-07T16:36:58.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:36:58 smithi144 bash[66654]: cluster 2024-04-07T16:36:57.033027+0000 mgr.y (mgr.25175) 648 : cluster [DBG] pgmap v1272: 137 pgs: 137 active+clean; 583 KiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 228 KiB/s rd, 152 KiB/s wr, 456 op/s 2024-04-07T16:36:58.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:36:58 smithi100 bash[74942]: cluster 2024-04-07T16:36:57.033027+0000 mgr.y (mgr.25175) 648 : cluster [DBG] pgmap v1272: 137 pgs: 137 active+clean; 583 KiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 228 KiB/s rd, 152 KiB/s wr, 456 op/s 2024-04-07T16:36:58.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:36:58 smithi100 bash[76202]: cluster 2024-04-07T16:36:57.033027+0000 mgr.y (mgr.25175) 648 : cluster [DBG] pgmap v1272: 137 pgs: 137 active+clean; 583 KiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 228 KiB/s rd, 152 KiB/s wr, 456 op/s 2024-04-07T16:36:58.691 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:36:58 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:36:58] "GET /metrics HTTP/1.1" 200 36378 "" "Prometheus/2.43.0" 2024-04-07T16:37:00.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:37:00 smithi144 bash[66654]: cluster 2024-04-07T16:36:59.034665+0000 mgr.y (mgr.25175) 649 : cluster [DBG] pgmap v1273: 137 pgs: 137 active+clean; 583 KiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 340 KiB/s rd, 226 KiB/s wr, 679 op/s 2024-04-07T16:37:00.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:37:00 smithi100 bash[74942]: cluster 2024-04-07T16:36:59.034665+0000 mgr.y (mgr.25175) 649 : cluster [DBG] pgmap v1273: 137 pgs: 137 active+clean; 583 KiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 340 KiB/s rd, 226 KiB/s wr, 679 op/s 2024-04-07T16:37:00.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:37:00 smithi100 bash[76202]: cluster 2024-04-07T16:36:59.034665+0000 mgr.y (mgr.25175) 649 : cluster [DBG] pgmap v1273: 137 pgs: 137 active+clean; 583 KiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 340 KiB/s rd, 226 KiB/s wr, 679 op/s 2024-04-07T16:37:02.606 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:37:02 smithi144 bash[95233]: ts=2024-04-07T16:37:02.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:37:02.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:37:02 smithi144 bash[66654]: cluster 2024-04-07T16:37:01.035847+0000 mgr.y (mgr.25175) 650 : cluster [DBG] pgmap v1274: 137 pgs: 137 active+clean; 583 KiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 227 KiB/s rd, 151 KiB/s wr, 454 op/s 2024-04-07T16:37:02.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:37:02 smithi144 bash[66654]: audit 2024-04-07T16:37:01.751698+0000 mon.a (mon.0) 1991 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:37:02.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:37:02 smithi100 bash[74942]: cluster 2024-04-07T16:37:01.035847+0000 mgr.y (mgr.25175) 650 : cluster [DBG] pgmap v1274: 137 pgs: 137 active+clean; 583 KiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 227 KiB/s rd, 151 KiB/s wr, 454 op/s 2024-04-07T16:37:02.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:37:02 smithi100 bash[74942]: audit 2024-04-07T16:37:01.751698+0000 mon.a (mon.0) 1991 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:37:02.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:37:02 smithi100 bash[76202]: cluster 2024-04-07T16:37:01.035847+0000 mgr.y (mgr.25175) 650 : cluster [DBG] pgmap v1274: 137 pgs: 137 active+clean; 583 KiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 227 KiB/s rd, 151 KiB/s wr, 454 op/s 2024-04-07T16:37:02.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:37:02 smithi100 bash[76202]: audit 2024-04-07T16:37:01.751698+0000 mon.a (mon.0) 1991 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:37:04.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:37:04 smithi144 bash[66654]: cluster 2024-04-07T16:37:03.037034+0000 mgr.y (mgr.25175) 651 : cluster [DBG] pgmap v1275: 137 pgs: 137 active+clean; 583 KiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 227 KiB/s rd, 151 KiB/s wr, 454 op/s 2024-04-07T16:37:04.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:37:04 smithi100 bash[74942]: cluster 2024-04-07T16:37:03.037034+0000 mgr.y (mgr.25175) 651 : cluster [DBG] pgmap v1275: 137 pgs: 137 active+clean; 583 KiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 227 KiB/s rd, 151 KiB/s wr, 454 op/s 2024-04-07T16:37:04.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:37:04 smithi100 bash[76202]: cluster 2024-04-07T16:37:03.037034+0000 mgr.y (mgr.25175) 651 : cluster [DBG] pgmap v1275: 137 pgs: 137 active+clean; 583 KiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 227 KiB/s rd, 151 KiB/s wr, 454 op/s 2024-04-07T16:37:06.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:37:06 smithi144 bash[66654]: cluster 2024-04-07T16:37:05.038180+0000 mgr.y (mgr.25175) 652 : cluster [DBG] pgmap v1276: 137 pgs: 137 active+clean; 583 KiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 341 KiB/s rd, 227 KiB/s wr, 682 op/s 2024-04-07T16:37:06.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:37:06 smithi100 bash[74942]: cluster 2024-04-07T16:37:05.038180+0000 mgr.y (mgr.25175) 652 : cluster [DBG] pgmap v1276: 137 pgs: 137 active+clean; 583 KiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 341 KiB/s rd, 227 KiB/s wr, 682 op/s 2024-04-07T16:37:06.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:37:06 smithi100 bash[76202]: cluster 2024-04-07T16:37:05.038180+0000 mgr.y (mgr.25175) 652 : cluster [DBG] pgmap v1276: 137 pgs: 137 active+clean; 583 KiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 341 KiB/s rd, 227 KiB/s wr, 682 op/s 2024-04-07T16:37:08.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:37:08 smithi144 bash[66654]: cluster 2024-04-07T16:37:07.038936+0000 mgr.y (mgr.25175) 653 : cluster [DBG] pgmap v1277: 137 pgs: 137 active+clean; 583 KiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 225 KiB/s rd, 150 KiB/s wr, 450 op/s 2024-04-07T16:37:08.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:37:08 smithi100 bash[74942]: cluster 2024-04-07T16:37:07.038936+0000 mgr.y (mgr.25175) 653 : cluster [DBG] pgmap v1277: 137 pgs: 137 active+clean; 583 KiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 225 KiB/s rd, 150 KiB/s wr, 450 op/s 2024-04-07T16:37:08.692 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:37:08 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:37:08] "GET /metrics HTTP/1.1" 200 36378 "" "Prometheus/2.43.0" 2024-04-07T16:37:08.692 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:37:08 smithi100 bash[76202]: cluster 2024-04-07T16:37:07.038936+0000 mgr.y (mgr.25175) 653 : cluster [DBG] pgmap v1277: 137 pgs: 137 active+clean; 583 KiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 225 KiB/s rd, 150 KiB/s wr, 450 op/s 2024-04-07T16:37:10.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:37:10 smithi144 bash[66654]: cluster 2024-04-07T16:37:09.040366+0000 mgr.y (mgr.25175) 654 : cluster [DBG] pgmap v1278: 137 pgs: 137 active+clean; 583 KiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 341 KiB/s rd, 227 KiB/s wr, 681 op/s 2024-04-07T16:37:10.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:37:10 smithi100 bash[74942]: cluster 2024-04-07T16:37:09.040366+0000 mgr.y (mgr.25175) 654 : cluster [DBG] pgmap v1278: 137 pgs: 137 active+clean; 583 KiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 341 KiB/s rd, 227 KiB/s wr, 681 op/s 2024-04-07T16:37:10.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:37:10 smithi100 bash[76202]: cluster 2024-04-07T16:37:09.040366+0000 mgr.y (mgr.25175) 654 : cluster [DBG] pgmap v1278: 137 pgs: 137 active+clean; 583 KiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 341 KiB/s rd, 227 KiB/s wr, 681 op/s 2024-04-07T16:37:12.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:37:12 smithi144 bash[66654]: cluster 2024-04-07T16:37:11.041188+0000 mgr.y (mgr.25175) 655 : cluster [DBG] pgmap v1279: 137 pgs: 137 active+clean; 583 KiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 229 KiB/s rd, 153 KiB/s wr, 458 op/s 2024-04-07T16:37:12.607 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:37:12 smithi144 bash[95233]: ts=2024-04-07T16:37:12.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:37:12.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:37:12 smithi100 bash[76202]: cluster 2024-04-07T16:37:11.041188+0000 mgr.y (mgr.25175) 655 : cluster [DBG] pgmap v1279: 137 pgs: 137 active+clean; 583 KiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 229 KiB/s rd, 153 KiB/s wr, 458 op/s 2024-04-07T16:37:12.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:37:12 smithi100 bash[74942]: cluster 2024-04-07T16:37:11.041188+0000 mgr.y (mgr.25175) 655 : cluster [DBG] pgmap v1279: 137 pgs: 137 active+clean; 583 KiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 229 KiB/s rd, 153 KiB/s wr, 458 op/s 2024-04-07T16:37:14.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:37:14 smithi144 bash[66654]: cluster 2024-04-07T16:37:13.042533+0000 mgr.y (mgr.25175) 656 : cluster [DBG] pgmap v1280: 137 pgs: 137 active+clean; 583 KiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 229 KiB/s rd, 153 KiB/s wr, 458 op/s 2024-04-07T16:37:14.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:37:14 smithi100 bash[76202]: cluster 2024-04-07T16:37:13.042533+0000 mgr.y (mgr.25175) 656 : cluster [DBG] pgmap v1280: 137 pgs: 137 active+clean; 583 KiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 229 KiB/s rd, 153 KiB/s wr, 458 op/s 2024-04-07T16:37:14.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:37:14 smithi100 bash[74942]: cluster 2024-04-07T16:37:13.042533+0000 mgr.y (mgr.25175) 656 : cluster [DBG] pgmap v1280: 137 pgs: 137 active+clean; 583 KiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 229 KiB/s rd, 153 KiB/s wr, 458 op/s 2024-04-07T16:37:16.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:37:16 smithi144 bash[66654]: cluster 2024-04-07T16:37:15.043315+0000 mgr.y (mgr.25175) 657 : cluster [DBG] pgmap v1281: 137 pgs: 137 active+clean; 583 KiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 342 KiB/s rd, 228 KiB/s wr, 684 op/s 2024-04-07T16:37:16.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:37:16 smithi100 bash[74942]: cluster 2024-04-07T16:37:15.043315+0000 mgr.y (mgr.25175) 657 : cluster [DBG] pgmap v1281: 137 pgs: 137 active+clean; 583 KiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 342 KiB/s rd, 228 KiB/s wr, 684 op/s 2024-04-07T16:37:16.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:37:16 smithi100 bash[76202]: cluster 2024-04-07T16:37:15.043315+0000 mgr.y (mgr.25175) 657 : cluster [DBG] pgmap v1281: 137 pgs: 137 active+clean; 583 KiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 342 KiB/s rd, 228 KiB/s wr, 684 op/s 2024-04-07T16:37:17.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:37:17 smithi144 bash[66654]: audit 2024-04-07T16:37:16.752010+0000 mon.a (mon.0) 1992 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:37:17.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:37:17 smithi100 bash[74942]: audit 2024-04-07T16:37:16.752010+0000 mon.a (mon.0) 1992 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:37:17.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:37:17 smithi100 bash[76202]: audit 2024-04-07T16:37:16.752010+0000 mon.a (mon.0) 1992 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:37:18.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:37:18 smithi144 bash[66654]: cluster 2024-04-07T16:37:17.043961+0000 mgr.y (mgr.25175) 658 : cluster [DBG] pgmap v1282: 137 pgs: 137 active+clean; 583 KiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 228 KiB/s rd, 152 KiB/s wr, 456 op/s 2024-04-07T16:37:18.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:37:18 smithi100 bash[74942]: cluster 2024-04-07T16:37:17.043961+0000 mgr.y (mgr.25175) 658 : cluster [DBG] pgmap v1282: 137 pgs: 137 active+clean; 583 KiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 228 KiB/s rd, 152 KiB/s wr, 456 op/s 2024-04-07T16:37:18.691 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:37:18 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:37:18] "GET /metrics HTTP/1.1" 200 36376 "" "Prometheus/2.43.0" 2024-04-07T16:37:18.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:37:18 smithi100 bash[76202]: cluster 2024-04-07T16:37:17.043961+0000 mgr.y (mgr.25175) 658 : cluster [DBG] pgmap v1282: 137 pgs: 137 active+clean; 583 KiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 228 KiB/s rd, 152 KiB/s wr, 456 op/s 2024-04-07T16:37:20.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:37:20 smithi144 bash[66654]: cluster 2024-04-07T16:37:19.044759+0000 mgr.y (mgr.25175) 659 : cluster [DBG] pgmap v1283: 137 pgs: 137 active+clean; 583 KiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 341 KiB/s rd, 227 KiB/s wr, 682 op/s 2024-04-07T16:37:20.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:37:20 smithi100 bash[74942]: cluster 2024-04-07T16:37:19.044759+0000 mgr.y (mgr.25175) 659 : cluster [DBG] pgmap v1283: 137 pgs: 137 active+clean; 583 KiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 341 KiB/s rd, 227 KiB/s wr, 682 op/s 2024-04-07T16:37:20.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:37:20 smithi100 bash[76202]: cluster 2024-04-07T16:37:19.044759+0000 mgr.y (mgr.25175) 659 : cluster [DBG] pgmap v1283: 137 pgs: 137 active+clean; 583 KiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 341 KiB/s rd, 227 KiB/s wr, 682 op/s 2024-04-07T16:37:22.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:37:22 smithi144 bash[66654]: cluster 2024-04-07T16:37:21.045593+0000 mgr.y (mgr.25175) 660 : cluster [DBG] pgmap v1284: 137 pgs: 137 active+clean; 583 KiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 226 KiB/s rd, 151 KiB/s wr, 451 op/s 2024-04-07T16:37:22.607 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:37:22 smithi144 bash[95233]: ts=2024-04-07T16:37:22.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:37:22.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:37:22 smithi100 bash[74942]: cluster 2024-04-07T16:37:21.045593+0000 mgr.y (mgr.25175) 660 : cluster [DBG] pgmap v1284: 137 pgs: 137 active+clean; 583 KiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 226 KiB/s rd, 151 KiB/s wr, 451 op/s 2024-04-07T16:37:22.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:37:22 smithi100 bash[76202]: cluster 2024-04-07T16:37:21.045593+0000 mgr.y (mgr.25175) 660 : cluster [DBG] pgmap v1284: 137 pgs: 137 active+clean; 583 KiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 226 KiB/s rd, 151 KiB/s wr, 451 op/s 2024-04-07T16:37:24.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:37:24 smithi144 bash[66654]: cluster 2024-04-07T16:37:23.046678+0000 mgr.y (mgr.25175) 661 : cluster [DBG] pgmap v1285: 137 pgs: 137 active+clean; 583 KiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 226 KiB/s rd, 151 KiB/s wr, 451 op/s 2024-04-07T16:37:24.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:37:24 smithi100 bash[74942]: cluster 2024-04-07T16:37:23.046678+0000 mgr.y (mgr.25175) 661 : cluster [DBG] pgmap v1285: 137 pgs: 137 active+clean; 583 KiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 226 KiB/s rd, 151 KiB/s wr, 451 op/s 2024-04-07T16:37:24.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:37:24 smithi100 bash[76202]: cluster 2024-04-07T16:37:23.046678+0000 mgr.y (mgr.25175) 661 : cluster [DBG] pgmap v1285: 137 pgs: 137 active+clean; 583 KiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 226 KiB/s rd, 151 KiB/s wr, 451 op/s 2024-04-07T16:37:26.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:37:26 smithi144 bash[66654]: cluster 2024-04-07T16:37:25.047410+0000 mgr.y (mgr.25175) 662 : cluster [DBG] pgmap v1286: 137 pgs: 137 active+clean; 583 KiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 338 KiB/s rd, 225 KiB/s wr, 675 op/s 2024-04-07T16:37:26.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:37:26 smithi100 bash[74942]: cluster 2024-04-07T16:37:25.047410+0000 mgr.y (mgr.25175) 662 : cluster [DBG] pgmap v1286: 137 pgs: 137 active+clean; 583 KiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 338 KiB/s rd, 225 KiB/s wr, 675 op/s 2024-04-07T16:37:26.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:37:26 smithi100 bash[76202]: cluster 2024-04-07T16:37:25.047410+0000 mgr.y (mgr.25175) 662 : cluster [DBG] pgmap v1286: 137 pgs: 137 active+clean; 583 KiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 338 KiB/s rd, 225 KiB/s wr, 675 op/s 2024-04-07T16:37:27.897 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] cls_rgw.index_list_delimited (99332 ms) 2024-04-07T16:37:27.897 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] cls_rgw.bi_list 2024-04-07T16:37:28.056 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] cls_rgw.bi_list (159 ms) 2024-04-07T16:37:28.057 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] cls_rgw.gc_set 2024-04-07T16:37:28.118 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] cls_rgw.gc_set (62 ms) 2024-04-07T16:37:28.118 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] cls_rgw.gc_list 2024-04-07T16:37:28.178 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] cls_rgw.gc_list (60 ms) 2024-04-07T16:37:28.178 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] cls_rgw.gc_defer 2024-04-07T16:37:28.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:37:28 smithi144 bash[66654]: cluster 2024-04-07T16:37:27.048311+0000 mgr.y (mgr.25175) 663 : cluster [DBG] pgmap v1287: 137 pgs: 137 active+clean; 583 KiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 225 KiB/s rd, 150 KiB/s wr, 449 op/s 2024-04-07T16:37:28.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:37:28 smithi100 bash[74942]: cluster 2024-04-07T16:37:27.048311+0000 mgr.y (mgr.25175) 663 : cluster [DBG] pgmap v1287: 137 pgs: 137 active+clean; 583 KiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 225 KiB/s rd, 150 KiB/s wr, 449 op/s 2024-04-07T16:37:28.691 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:37:28 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:37:28] "GET /metrics HTTP/1.1" 200 36376 "" "Prometheus/2.43.0" 2024-04-07T16:37:28.692 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:37:28 smithi100 bash[76202]: cluster 2024-04-07T16:37:27.048311+0000 mgr.y (mgr.25175) 663 : cluster [DBG] pgmap v1287: 137 pgs: 137 active+clean; 583 KiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 225 KiB/s rd, 150 KiB/s wr, 449 op/s 2024-04-07T16:37:29.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:37:29 smithi144 bash[66654]: cluster 2024-04-07T16:37:28.343211+0000 mon.a (mon.0) 1993 : cluster [DBG] osdmap e1050: 8 total, 8 up, 8 in 2024-04-07T16:37:29.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:37:29 smithi144 bash[66654]: audit 2024-04-07T16:37:28.345838+0000 mon.c (mon.2) 55 : audit [INF] from='client.? 172.21.15.100:0/2390264952' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111390-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:37:29.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:37:29 smithi144 bash[66654]: audit 2024-04-07T16:37:28.348433+0000 mon.a (mon.0) 1994 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111390-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:37:29.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:37:29 smithi144 bash[66654]: audit 2024-04-07T16:37:29.332865+0000 mon.a (mon.0) 1995 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111390-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:37:29.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:37:29 smithi144 bash[66654]: cluster 2024-04-07T16:37:29.340836+0000 mon.a (mon.0) 1996 : cluster [DBG] osdmap e1051: 8 total, 8 up, 8 in 2024-04-07T16:37:29.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:37:29 smithi100 bash[74942]: cluster 2024-04-07T16:37:28.343211+0000 mon.a (mon.0) 1993 : cluster [DBG] osdmap e1050: 8 total, 8 up, 8 in 2024-04-07T16:37:29.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:37:29 smithi100 bash[74942]: audit 2024-04-07T16:37:28.345838+0000 mon.c (mon.2) 55 : audit [INF] from='client.? 172.21.15.100:0/2390264952' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111390-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:37:29.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:37:29 smithi100 bash[74942]: audit 2024-04-07T16:37:28.348433+0000 mon.a (mon.0) 1994 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111390-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:37:29.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:37:29 smithi100 bash[74942]: audit 2024-04-07T16:37:29.332865+0000 mon.a (mon.0) 1995 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111390-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:37:29.692 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:37:29 smithi100 bash[74942]: cluster 2024-04-07T16:37:29.340836+0000 mon.a (mon.0) 1996 : cluster [DBG] osdmap e1051: 8 total, 8 up, 8 in 2024-04-07T16:37:29.692 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:37:29 smithi100 bash[76202]: cluster 2024-04-07T16:37:28.343211+0000 mon.a (mon.0) 1993 : cluster [DBG] osdmap e1050: 8 total, 8 up, 8 in 2024-04-07T16:37:29.692 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:37:29 smithi100 bash[76202]: audit 2024-04-07T16:37:28.345838+0000 mon.c (mon.2) 55 : audit [INF] from='client.? 172.21.15.100:0/2390264952' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111390-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:37:29.692 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:37:29 smithi100 bash[76202]: audit 2024-04-07T16:37:28.348433+0000 mon.a (mon.0) 1994 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111390-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:37:29.692 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:37:29 smithi100 bash[76202]: audit 2024-04-07T16:37:29.332865+0000 mon.a (mon.0) 1995 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111390-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:37:29.692 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:37:29 smithi100 bash[76202]: cluster 2024-04-07T16:37:29.340836+0000 mon.a (mon.0) 1996 : cluster [DBG] osdmap e1051: 8 total, 8 up, 8 in 2024-04-07T16:37:30.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:37:30 smithi144 bash[66654]: cluster 2024-04-07T16:37:29.049165+0000 mgr.y (mgr.25175) 664 : cluster [DBG] pgmap v1289: 169 pgs: 3 creating+peering, 29 unknown, 137 active+clean; 583 KiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 401 KiB/s rd, 172 KiB/s wr, 512 op/s 2024-04-07T16:37:30.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:37:30 smithi144 bash[66654]: cluster 2024-04-07T16:37:29.351988+0000 mon.a (mon.0) 1997 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T16:37:30.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:37:30 smithi100 bash[74942]: cluster 2024-04-07T16:37:29.049165+0000 mgr.y (mgr.25175) 664 : cluster [DBG] pgmap v1289: 169 pgs: 3 creating+peering, 29 unknown, 137 active+clean; 583 KiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 401 KiB/s rd, 172 KiB/s wr, 512 op/s 2024-04-07T16:37:30.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:37:30 smithi100 bash[74942]: cluster 2024-04-07T16:37:29.351988+0000 mon.a (mon.0) 1997 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T16:37:30.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:37:30 smithi100 bash[76202]: cluster 2024-04-07T16:37:29.049165+0000 mgr.y (mgr.25175) 664 : cluster [DBG] pgmap v1289: 169 pgs: 3 creating+peering, 29 unknown, 137 active+clean; 583 KiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 401 KiB/s rd, 172 KiB/s wr, 512 op/s 2024-04-07T16:37:30.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:37:30 smithi100 bash[76202]: cluster 2024-04-07T16:37:29.351988+0000 mon.a (mon.0) 1997 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T16:37:31.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:37:31 smithi100 bash[74942]: cluster 2024-04-07T16:37:30.375846+0000 mon.a (mon.0) 1998 : cluster [DBG] osdmap e1052: 8 total, 8 up, 8 in 2024-04-07T16:37:31.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:37:31 smithi100 bash[76202]: cluster 2024-04-07T16:37:30.375846+0000 mon.a (mon.0) 1998 : cluster [DBG] osdmap e1052: 8 total, 8 up, 8 in 2024-04-07T16:37:31.856 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:37:31 smithi144 bash[66654]: cluster 2024-04-07T16:37:30.375846+0000 mon.a (mon.0) 1998 : cluster [DBG] osdmap e1052: 8 total, 8 up, 8 in 2024-04-07T16:37:32.606 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:37:32 smithi144 bash[95233]: ts=2024-04-07T16:37:32.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:37:32.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:37:32 smithi144 bash[66654]: cluster 2024-04-07T16:37:31.049896+0000 mgr.y (mgr.25175) 665 : cluster [DBG] pgmap v1292: 169 pgs: 6 creating+peering, 22 unknown, 141 active+clean; 583 KiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 445 KiB/s rd, 138 KiB/s wr, 407 op/s 2024-04-07T16:37:32.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:37:32 smithi144 bash[66654]: cluster 2024-04-07T16:37:31.367645+0000 mon.a (mon.0) 1999 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T16:37:32.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:37:32 smithi144 bash[66654]: cluster 2024-04-07T16:37:31.367705+0000 mon.a (mon.0) 2000 : cluster [INF] Cluster is now healthy 2024-04-07T16:37:32.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:37:32 smithi144 bash[66654]: audit 2024-04-07T16:37:31.752284+0000 mon.a (mon.0) 2001 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:37:32.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:37:32 smithi100 bash[74942]: cluster 2024-04-07T16:37:31.049896+0000 mgr.y (mgr.25175) 665 : cluster [DBG] pgmap v1292: 169 pgs: 6 creating+peering, 22 unknown, 141 active+clean; 583 KiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 445 KiB/s rd, 138 KiB/s wr, 407 op/s 2024-04-07T16:37:32.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:37:32 smithi100 bash[74942]: cluster 2024-04-07T16:37:31.367645+0000 mon.a (mon.0) 1999 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T16:37:32.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:37:32 smithi100 bash[74942]: cluster 2024-04-07T16:37:31.367705+0000 mon.a (mon.0) 2000 : cluster [INF] Cluster is now healthy 2024-04-07T16:37:32.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:37:32 smithi100 bash[74942]: audit 2024-04-07T16:37:31.752284+0000 mon.a (mon.0) 2001 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:37:32.692 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:37:32 smithi100 bash[76202]: cluster 2024-04-07T16:37:31.049896+0000 mgr.y (mgr.25175) 665 : cluster [DBG] pgmap v1292: 169 pgs: 6 creating+peering, 22 unknown, 141 active+clean; 583 KiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 445 KiB/s rd, 138 KiB/s wr, 407 op/s 2024-04-07T16:37:32.692 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:37:32 smithi100 bash[76202]: cluster 2024-04-07T16:37:31.367645+0000 mon.a (mon.0) 1999 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T16:37:32.692 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:37:32 smithi100 bash[76202]: cluster 2024-04-07T16:37:31.367705+0000 mon.a (mon.0) 2000 : cluster [INF] Cluster is now healthy 2024-04-07T16:37:32.692 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:37:32 smithi100 bash[76202]: audit 2024-04-07T16:37:31.752284+0000 mon.a (mon.0) 2001 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:37:34.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:37:34 smithi100 bash[74942]: cluster 2024-04-07T16:37:33.051304+0000 mgr.y (mgr.25175) 666 : cluster [DBG] pgmap v1293: 169 pgs: 6 creating+peering, 163 active+clean; 583 KiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 445 KiB/s rd, 138 KiB/s wr, 407 op/s 2024-04-07T16:37:34.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:37:34 smithi100 bash[76202]: cluster 2024-04-07T16:37:33.051304+0000 mgr.y (mgr.25175) 666 : cluster [DBG] pgmap v1293: 169 pgs: 6 creating+peering, 163 active+clean; 583 KiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 445 KiB/s rd, 138 KiB/s wr, 407 op/s 2024-04-07T16:37:34.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:37:34 smithi144 bash[66654]: cluster 2024-04-07T16:37:33.051304+0000 mgr.y (mgr.25175) 666 : cluster [DBG] pgmap v1293: 169 pgs: 6 creating+peering, 163 active+clean; 583 KiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 445 KiB/s rd, 138 KiB/s wr, 407 op/s 2024-04-07T16:37:35.399 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] cls_rgw.gc_defer (7221 ms) 2024-04-07T16:37:35.400 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] cls_rgw.usage_basic 2024-04-07T16:37:35.596 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] cls_rgw.usage_basic (197 ms) 2024-04-07T16:37:35.596 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] cls_rgw.usage_clear_no_obj 2024-04-07T16:37:35.600 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] cls_rgw.usage_clear_no_obj (4 ms) 2024-04-07T16:37:35.600 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] cls_rgw.usage_clear 2024-04-07T16:37:35.765 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] cls_rgw.usage_clear (165 ms) 2024-04-07T16:37:35.765 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] cls_rgw.bi_log_trim 2024-04-07T16:37:35.967 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] cls_rgw.bi_log_trim (202 ms) 2024-04-07T16:37:35.967 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] cls_rgw.index_racing_removes 2024-04-07T16:37:35.998 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] cls_rgw.index_racing_removes (31 ms) 2024-04-07T16:37:36.407 INFO:tasks.workunit.client.0.smithi100.stdout:[----------] 16 tests from cls_rgw (109977 ms total) 2024-04-07T16:37:36.407 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T16:37:36.407 INFO:tasks.workunit.client.0.smithi100.stdout:[----------] Global test environment tear-down 2024-04-07T16:37:36.407 INFO:tasks.workunit.client.0.smithi100.stdout:[==========] 16 tests from 1 test suite ran. (112268 ms total) 2024-04-07T16:37:36.407 INFO:tasks.workunit.client.0.smithi100.stdout:[ PASSED ] 16 tests. 2024-04-07T16:37:36.416 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-07T16:37:36.416 DEBUG:teuthology.orchestra.run.smithi100:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-07T16:37:36.431 INFO:tasks.workunit:Running workunit cls/test_cls_rgw_gc.sh... 2024-04-07T16:37:36.431 DEBUG:teuthology.orchestra.run.smithi100:workunit test cls/test_cls_rgw_gc.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=reef TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_rgw_gc.sh 2024-04-07T16:37:36.495 INFO:tasks.workunit.client.0.smithi100.stdout:Running main() from gmock_main.cc 2024-04-07T16:37:36.496 INFO:tasks.workunit.client.0.smithi100.stdout:[==========] Running 9 tests from 1 test suite. 2024-04-07T16:37:36.496 INFO:tasks.workunit.client.0.smithi100.stdout:[----------] Global test environment set-up. 2024-04-07T16:37:36.496 INFO:tasks.workunit.client.0.smithi100.stdout:[----------] 9 tests from cls_rgw_gc 2024-04-07T16:37:36.496 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] cls_rgw_gc.init 2024-04-07T16:37:36.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:37:36 smithi100 bash[74942]: cluster 2024-04-07T16:37:35.052375+0000 mgr.y (mgr.25175) 667 : cluster [DBG] pgmap v1294: 169 pgs: 169 active+clean; 583 KiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 398 KiB/s rd, 124 KiB/s wr, 365 op/s 2024-04-07T16:37:36.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:37:36 smithi100 bash[74942]: cluster 2024-04-07T16:37:35.402798+0000 mon.a (mon.0) 2002 : cluster [DBG] osdmap e1053: 8 total, 8 up, 8 in 2024-04-07T16:37:36.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:37:36 smithi100 bash[76202]: cluster 2024-04-07T16:37:35.052375+0000 mgr.y (mgr.25175) 667 : cluster [DBG] pgmap v1294: 169 pgs: 169 active+clean; 583 KiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 398 KiB/s rd, 124 KiB/s wr, 365 op/s 2024-04-07T16:37:36.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:37:36 smithi100 bash[76202]: cluster 2024-04-07T16:37:35.402798+0000 mon.a (mon.0) 2002 : cluster [DBG] osdmap e1053: 8 total, 8 up, 8 in 2024-04-07T16:37:36.856 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:37:36 smithi144 bash[66654]: cluster 2024-04-07T16:37:35.052375+0000 mgr.y (mgr.25175) 667 : cluster [DBG] pgmap v1294: 169 pgs: 169 active+clean; 583 KiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 398 KiB/s rd, 124 KiB/s wr, 365 op/s 2024-04-07T16:37:36.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:37:36 smithi144 bash[66654]: cluster 2024-04-07T16:37:35.402798+0000 mon.a (mon.0) 2002 : cluster [DBG] osdmap e1053: 8 total, 8 up, 8 in 2024-04-07T16:37:37.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:37:37 smithi100 bash[74942]: cluster 2024-04-07T16:37:36.412156+0000 mon.a (mon.0) 2003 : cluster [DBG] osdmap e1054: 8 total, 8 up, 8 in 2024-04-07T16:37:37.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:37:37 smithi100 bash[76202]: cluster 2024-04-07T16:37:36.412156+0000 mon.a (mon.0) 2003 : cluster [DBG] osdmap e1054: 8 total, 8 up, 8 in 2024-04-07T16:37:37.856 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:37:37 smithi144 bash[66654]: cluster 2024-04-07T16:37:36.412156+0000 mon.a (mon.0) 2003 : cluster [DBG] osdmap e1054: 8 total, 8 up, 8 in 2024-04-07T16:37:38.429 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] cls_rgw_gc.init (1934 ms) 2024-04-07T16:37:38.430 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] cls_rgw_gc.gc_queue_ops1 2024-04-07T16:37:38.479 INFO:tasks.workunit.client.0.smithi100.stderr:[ ] list info tag = chain-0 2024-04-07T16:37:38.479 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] cls_rgw_gc.gc_queue_ops1 (50 ms) 2024-04-07T16:37:38.480 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] cls_rgw_gc.gc_queue_ops2 2024-04-07T16:37:38.487 INFO:tasks.workunit.client.0.smithi100.stderr:[ ] next_marker is: = 0/4263 2024-04-07T16:37:38.488 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] cls_rgw_gc.gc_queue_ops2 (9 ms) 2024-04-07T16:37:38.488 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] cls_rgw_gc.gc_queue_ops5 2024-04-07T16:37:38.504 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] cls_rgw_gc.gc_queue_ops5 (16 ms) 2024-04-07T16:37:38.504 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] cls_rgw_gc.gc_queue_ops6 2024-04-07T16:37:38.518 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] cls_rgw_gc.gc_queue_ops6 (14 ms) 2024-04-07T16:37:38.518 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] cls_rgw_gc.gc_queue_ops7 2024-04-07T16:37:38.532 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] cls_rgw_gc.gc_queue_ops7 (15 ms) 2024-04-07T16:37:38.533 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] cls_rgw_gc.gc_queue_ops8 2024-04-07T16:37:38.547 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] cls_rgw_gc.gc_queue_ops8 (14 ms) 2024-04-07T16:37:38.547 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] cls_rgw_gc.gc_queue_ops9 2024-04-07T16:37:38.559 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] cls_rgw_gc.gc_queue_ops9 (12 ms) 2024-04-07T16:37:38.559 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] cls_rgw_gc.finalize 2024-04-07T16:37:38.691 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:37:38 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:37:38] "GET /metrics HTTP/1.1" 200 38747 "" "Prometheus/2.43.0" 2024-04-07T16:37:38.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:37:38 smithi100 bash[74942]: cluster 2024-04-07T16:37:37.053000+0000 mgr.y (mgr.25175) 668 : cluster [DBG] pgmap v1297: 105 pgs: 105 active+clean; 583 KiB data, 2.8 GiB used, 712 GiB / 715 GiB avail 2024-04-07T16:37:38.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:37:38 smithi100 bash[74942]: cluster 2024-04-07T16:37:37.427229+0000 mon.a (mon.0) 2004 : cluster [DBG] osdmap e1055: 8 total, 8 up, 8 in 2024-04-07T16:37:38.692 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:37:38 smithi100 bash[74942]: audit 2024-04-07T16:37:37.432690+0000 mon.c (mon.2) 56 : audit [INF] from='client.? 172.21.15.100:0/4046904843' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111446-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:37:38.692 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:37:38 smithi100 bash[74942]: audit 2024-04-07T16:37:37.435838+0000 mon.a (mon.0) 2005 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111446-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:37:38.692 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:37:38 smithi100 bash[76202]: cluster 2024-04-07T16:37:37.053000+0000 mgr.y (mgr.25175) 668 : cluster [DBG] pgmap v1297: 105 pgs: 105 active+clean; 583 KiB data, 2.8 GiB used, 712 GiB / 715 GiB avail 2024-04-07T16:37:38.692 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:37:38 smithi100 bash[76202]: cluster 2024-04-07T16:37:37.427229+0000 mon.a (mon.0) 2004 : cluster [DBG] osdmap e1055: 8 total, 8 up, 8 in 2024-04-07T16:37:38.692 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:37:38 smithi100 bash[76202]: audit 2024-04-07T16:37:37.432690+0000 mon.c (mon.2) 56 : audit [INF] from='client.? 172.21.15.100:0/4046904843' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111446-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:37:38.692 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:37:38 smithi100 bash[76202]: audit 2024-04-07T16:37:37.435838+0000 mon.a (mon.0) 2005 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111446-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:37:38.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:37:38 smithi144 bash[66654]: cluster 2024-04-07T16:37:37.053000+0000 mgr.y (mgr.25175) 668 : cluster [DBG] pgmap v1297: 105 pgs: 105 active+clean; 583 KiB data, 2.8 GiB used, 712 GiB / 715 GiB avail 2024-04-07T16:37:38.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:37:38 smithi144 bash[66654]: cluster 2024-04-07T16:37:37.427229+0000 mon.a (mon.0) 2004 : cluster [DBG] osdmap e1055: 8 total, 8 up, 8 in 2024-04-07T16:37:38.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:37:38 smithi144 bash[66654]: audit 2024-04-07T16:37:37.432690+0000 mon.c (mon.2) 56 : audit [INF] from='client.? 172.21.15.100:0/4046904843' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111446-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:37:38.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:37:38 smithi144 bash[66654]: audit 2024-04-07T16:37:37.435838+0000 mon.a (mon.0) 2005 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111446-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:37:39.445 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] cls_rgw_gc.finalize (886 ms) 2024-04-07T16:37:39.445 INFO:tasks.workunit.client.0.smithi100.stdout:[----------] 9 tests from cls_rgw_gc (2950 ms total) 2024-04-07T16:37:39.445 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T16:37:39.445 INFO:tasks.workunit.client.0.smithi100.stdout:[----------] Global test environment tear-down 2024-04-07T16:37:39.445 INFO:tasks.workunit.client.0.smithi100.stdout:[==========] 9 tests from 1 test suite ran. (2950 ms total) 2024-04-07T16:37:39.446 INFO:tasks.workunit.client.0.smithi100.stdout:[ PASSED ] 9 tests. 2024-04-07T16:37:39.451 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-07T16:37:39.451 DEBUG:teuthology.orchestra.run.smithi100:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-07T16:37:39.507 INFO:tasks.workunit:Running workunit cls/test_cls_rgw_stats.sh... 2024-04-07T16:37:39.507 DEBUG:teuthology.orchestra.run.smithi100:workunit test cls/test_cls_rgw_stats.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=reef TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_rgw_stats.sh 2024-04-07T16:37:39.581 INFO:tasks.workunit.client.0.smithi100.stdout:[==========] Running 1 test from 1 test suite. 2024-04-07T16:37:39.581 INFO:tasks.workunit.client.0.smithi100.stdout:[----------] Global test environment set-up. 2024-04-07T16:37:39.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:37:39 smithi100 bash[74942]: audit 2024-04-07T16:37:38.431275+0000 mon.a (mon.0) 2006 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111446-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:37:39.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:37:39 smithi100 bash[74942]: cluster 2024-04-07T16:37:38.438996+0000 mon.a (mon.0) 2007 : cluster [DBG] osdmap e1056: 8 total, 8 up, 8 in 2024-04-07T16:37:39.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:37:39 smithi100 bash[76202]: audit 2024-04-07T16:37:38.431275+0000 mon.a (mon.0) 2006 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111446-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:37:39.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:37:39 smithi100 bash[76202]: cluster 2024-04-07T16:37:38.438996+0000 mon.a (mon.0) 2007 : cluster [DBG] osdmap e1056: 8 total, 8 up, 8 in 2024-04-07T16:37:39.856 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:37:39 smithi144 bash[66654]: audit 2024-04-07T16:37:38.431275+0000 mon.a (mon.0) 2006 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111446-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:37:39.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:37:39 smithi144 bash[66654]: cluster 2024-04-07T16:37:38.438996+0000 mon.a (mon.0) 2007 : cluster [DBG] osdmap e1056: 8 total, 8 up, 8 in 2024-04-07T16:37:40.856 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:37:40 smithi144 bash[66654]: cluster 2024-04-07T16:37:39.053803+0000 mgr.y (mgr.25175) 669 : cluster [DBG] pgmap v1300: 137 pgs: 4 creating+peering, 26 unknown, 107 active+clean; 587 KiB data, 2.8 GiB used, 712 GiB / 715 GiB avail 2024-04-07T16:37:40.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:37:40 smithi144 bash[66654]: cluster 2024-04-07T16:37:39.449188+0000 mon.a (mon.0) 2008 : cluster [DBG] osdmap e1057: 8 total, 8 up, 8 in 2024-04-07T16:37:40.941 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:37:40 smithi100 bash[74942]: cluster 2024-04-07T16:37:39.053803+0000 mgr.y (mgr.25175) 669 : cluster [DBG] pgmap v1300: 137 pgs: 4 creating+peering, 26 unknown, 107 active+clean; 587 KiB data, 2.8 GiB used, 712 GiB / 715 GiB avail 2024-04-07T16:37:40.941 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:37:40 smithi100 bash[74942]: cluster 2024-04-07T16:37:39.449188+0000 mon.a (mon.0) 2008 : cluster [DBG] osdmap e1057: 8 total, 8 up, 8 in 2024-04-07T16:37:40.941 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:37:40 smithi100 bash[76202]: cluster 2024-04-07T16:37:39.053803+0000 mgr.y (mgr.25175) 669 : cluster [DBG] pgmap v1300: 137 pgs: 4 creating+peering, 26 unknown, 107 active+clean; 587 KiB data, 2.8 GiB used, 712 GiB / 715 GiB avail 2024-04-07T16:37:40.941 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:37:40 smithi100 bash[76202]: cluster 2024-04-07T16:37:39.449188+0000 mon.a (mon.0) 2008 : cluster [DBG] osdmap e1057: 8 total, 8 up, 8 in 2024-04-07T16:37:41.446 INFO:tasks.workunit.client.0.smithi100.stdout:[----------] 1 test from cls_rgw_stats 2024-04-07T16:37:41.446 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] cls_rgw_stats.simulate 2024-04-07T16:37:41.489 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.497+0000 7f346a3868c0 1 > prepared operation key=GZXHYZO5SMUP tag=XM396UAWDVIE type=del size=2159732 2024-04-07T16:37:41.491 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.501+0000 7f346a3868c0 1 > prepared operation key=7G5C6HBK6QWI tag=AC16H2Y146YR type=del size=505111 2024-04-07T16:37:41.493 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.501+0000 7f346a3868c0 1 > prepared operation key=1VBSUWNI3V1T tag=L2B554WGIAG3 type=del size=1068578 2024-04-07T16:37:41.496 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.505+0000 7f346a3868c0 1 > prepared operation key=IX3F0IUOKBX1 tag=FUQMR2VLMBR5 type=del size=1163858 2024-04-07T16:37:41.498 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.505+0000 7f346a3868c0 1 > prepared operation key=YOK8BAEOMPUR tag=100UZ48UFWNZ type=del size=2345887 2024-04-07T16:37:41.500 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.509+0000 7f346a3868c0 1 > prepared operation key=6XDS0A7L0IY8 tag=HYGLMING1URS type=del size=1618013 2024-04-07T16:37:41.502 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.509+0000 7f346a3868c0 1 > prepared operation key=N14BUXLJONHZ tag=LJL1E363J9I3 type=write size=407116 2024-04-07T16:37:41.506 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.513+0000 7f346a3868c0 1 > prepared part key=_multipart_U3EID5EK64HD.2~T4B29TOO25KE.0 size=1048576 2024-04-07T16:37:41.508 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.517+0000 7f346a3868c0 1 > prepared part key=_multipart_U3EID5EK64HD.2~T4B29TOO25KE.1 size=319272 2024-04-07T16:37:41.508 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.517+0000 7f346a3868c0 1 > prepared multipart upload key=U3EID5EK64HD upload=T4B29TOO25KE tag=EHR1A0REIPD2 type=write size=2416424 2024-04-07T16:37:41.510 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.517+0000 7f346a3868c0 1 > prepared operation key=GZXHYZO5SMUP tag=CY5MCUNH9EHZ type=del size=487196 2024-04-07T16:37:41.513 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.521+0000 7f346a3868c0 1 > prepared operation key=E84U0SX44NQ8 tag=M91MZ5F1GBAB type=del size=2687155 2024-04-07T16:37:41.515 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.525+0000 7f346a3868c0 1 > prepared operation key=C1BTVJNHIM0X tag=981QMTS4U3OV type=del size=453566 2024-04-07T16:37:41.517 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.525+0000 7f346a3868c0 1 > prepared operation key=QVOKSQAGHPA9 tag=DIK0WP8OQ4SS type=del size=3977943 2024-04-07T16:37:41.520 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.529+0000 7f346a3868c0 1 > prepared operation key=N14BUXLJONHZ tag=ELY5WKYH0MHG type=del size=727125 2024-04-07T16:37:41.524 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.533+0000 7f346a3868c0 1 > prepared part key=_multipart_1WGUPH7X5UA0.2~3CFF8EMZJEOP.0 size=360905 2024-04-07T16:37:41.525 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.533+0000 7f346a3868c0 1 > prepared multipart upload key=1WGUPH7X5UA0 upload=3CFF8EMZJEOP tag=Q069BPPD5PH1 type=write size=1409481 2024-04-07T16:37:41.529 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.537+0000 7f346a3868c0 1 > prepared part key=_multipart_N14BUXLJONHZ.2~M5TDK6F4MF4K.0 size=1048576 2024-04-07T16:37:41.530 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.537+0000 7f346a3868c0 1 > prepared part key=_multipart_N14BUXLJONHZ.2~M5TDK6F4MF4K.1 size=1048576 2024-04-07T16:37:41.532 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.541+0000 7f346a3868c0 1 > prepared part key=_multipart_N14BUXLJONHZ.2~M5TDK6F4MF4K.2 size=725494 2024-04-07T16:37:41.532 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.541+0000 7f346a3868c0 1 > prepared multipart upload key=N14BUXLJONHZ upload=M5TDK6F4MF4K tag=MNZ2B90OXOAN type=write size=3871222 2024-04-07T16:37:41.536 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.545+0000 7f346a3868c0 1 > prepared part key=_multipart_1VBSUWNI3V1T.2~16KZSH5CG8AY.0 size=1048576 2024-04-07T16:37:41.538 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.545+0000 7f346a3868c0 1 > prepared part key=_multipart_1VBSUWNI3V1T.2~16KZSH5CG8AY.1 size=1048576 2024-04-07T16:37:41.540 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.549+0000 7f346a3868c0 1 > prepared part key=_multipart_1VBSUWNI3V1T.2~16KZSH5CG8AY.2 size=137963 2024-04-07T16:37:41.540 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.549+0000 7f346a3868c0 1 > prepared multipart upload key=1VBSUWNI3V1T upload=16KZSH5CG8AY tag=KAJSXFI0608L type=write size=3283691 2024-04-07T16:37:41.542 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.549+0000 7f346a3868c0 1 < completed delete operation key=GZXHYZO5SMUP tag=XM396UAWDVIE type=del 2024-04-07T16:37:41.544 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.553+0000 7f346a3868c0 1 > prepared operation key=C1BTVJNHIM0X tag=D4UED1EN5VIP type=write size=88942 2024-04-07T16:37:41.546 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.553+0000 7f346a3868c0 1 < completed delete operation key=7G5C6HBK6QWI tag=AC16H2Y146YR type=del 2024-04-07T16:37:41.548 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.557+0000 7f346a3868c0 1 > prepared operation key=YXMDZB23YTB3 tag=U24AHIG6JD5N type=write size=15067 2024-04-07T16:37:41.553 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.561+0000 7f346a3868c0 1 < suggested remove operation key=1VBSUWNI3V1T tag=L2B554WGIAG3 type=del 2024-04-07T16:37:41.558 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.565+0000 7f346a3868c0 1 > prepared part key=_multipart_QVOKSQAGHPA9.2~DL4KG82OSG2J.0 size=1048576 2024-04-07T16:37:41.560 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.569+0000 7f346a3868c0 1 > prepared part key=_multipart_QVOKSQAGHPA9.2~DL4KG82OSG2J.1 size=40818 2024-04-07T16:37:41.560 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.569+0000 7f346a3868c0 1 > prepared multipart upload key=QVOKSQAGHPA9 upload=DL4KG82OSG2J tag=GFO0DACY57XB type=write size=2137970 2024-04-07T16:37:41.562 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.569+0000 7f346a3868c0 1 < completed delete operation key=IX3F0IUOKBX1 tag=FUQMR2VLMBR5 type=del 2024-04-07T16:37:41.564 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.573+0000 7f346a3868c0 1 > prepared operation key=YXMDZB23YTB3 tag=73C0AGA492IV type=write size=632302 2024-04-07T16:37:41.566 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.573+0000 7f346a3868c0 1 < completed delete operation key=YOK8BAEOMPUR tag=100UZ48UFWNZ type=del 2024-04-07T16:37:41.570 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.577+0000 7f346a3868c0 1 > prepared part key=_multipart_N14BUXLJONHZ.2~HRA9RS052UAC.0 size=24260 2024-04-07T16:37:41.570 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.577+0000 7f346a3868c0 1 > prepared multipart upload key=N14BUXLJONHZ upload=HRA9RS052UAC tag=VY2BSUWSEHKY type=write size=1072836 2024-04-07T16:37:41.572 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.581+0000 7f346a3868c0 1 < completed delete operation key=6XDS0A7L0IY8 tag=HYGLMING1URS type=del 2024-04-07T16:37:41.576 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.585+0000 7f346a3868c0 1 > prepared part key=_multipart_X5Q46GO8YTOK.2~8GO1JR8VCULC.0 size=1048576 2024-04-07T16:37:41.577 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.585+0000 7f346a3868c0 1 > prepared part key=_multipart_X5Q46GO8YTOK.2~8GO1JR8VCULC.1 size=1048576 2024-04-07T16:37:41.579 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.589+0000 7f346a3868c0 1 > prepared part key=_multipart_X5Q46GO8YTOK.2~8GO1JR8VCULC.2 size=978611 2024-04-07T16:37:41.579 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.589+0000 7f346a3868c0 1 > prepared multipart upload key=X5Q46GO8YTOK upload=8GO1JR8VCULC tag=3XUI7K4JU7CH type=write size=4124339 2024-04-07T16:37:41.581 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.589+0000 7f346a3868c0 1 < completed write operation key=N14BUXLJONHZ tag=LJL1E363J9I3 type=write size=407116 2024-04-07T16:37:41.584 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.593+0000 7f346a3868c0 1 > prepared operation key=XJLQZ7IS9U1T tag=MG5Z105ACXXM type=del size=826479 2024-04-07T16:37:41.586 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.593+0000 7f346a3868c0 1 < completed part key=_multipart_U3EID5EK64HD.2~T4B29TOO25KE.0 size=1048576 2024-04-07T16:37:41.588 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.597+0000 7f346a3868c0 1 < completed part key=_multipart_U3EID5EK64HD.2~T4B29TOO25KE.1 size=319272 2024-04-07T16:37:41.591 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.601+0000 7f346a3868c0 1 < completed multipart upload key=U3EID5EK64HD upload=T4B29TOO25KE tag=EHR1A0REIPD2 type=write size=2416424 2024-04-07T16:37:41.593 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.601+0000 7f346a3868c0 1 > prepared operation key=UIW3DN4NUYPP tag=LCY5L4DJRGJX type=del size=510767 2024-04-07T16:37:41.595 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.605+0000 7f346a3868c0 1 < completed delete operation key=GZXHYZO5SMUP tag=CY5MCUNH9EHZ type=del 2024-04-07T16:37:41.598 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.605+0000 7f346a3868c0 1 > prepared part key=_multipart_QVOKSQAGHPA9.2~1LPT4I10WNE6.0 size=1048576 2024-04-07T16:37:41.600 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.609+0000 7f346a3868c0 1 > prepared part key=_multipart_QVOKSQAGHPA9.2~1LPT4I10WNE6.1 size=1048576 2024-04-07T16:37:41.601 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.609+0000 7f346a3868c0 1 > prepared part key=_multipart_QVOKSQAGHPA9.2~1LPT4I10WNE6.2 size=49691 2024-04-07T16:37:41.601 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.609+0000 7f346a3868c0 1 > prepared multipart upload key=QVOKSQAGHPA9 upload=1LPT4I10WNE6 tag=220PZAF0HAMI type=write size=3195419 2024-04-07T16:37:41.603 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.613+0000 7f346a3868c0 1 < completed delete operation key=E84U0SX44NQ8 tag=M91MZ5F1GBAB type=del 2024-04-07T16:37:41.607 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.617+0000 7f346a3868c0 1 > prepared part key=_multipart_GZXHYZO5SMUP.2~BPITDHWXOVA4.0 size=1048576 2024-04-07T16:37:41.609 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.617+0000 7f346a3868c0 1 > prepared part key=_multipart_GZXHYZO5SMUP.2~BPITDHWXOVA4.1 size=248023 2024-04-07T16:37:41.609 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.617+0000 7f346a3868c0 1 > prepared multipart upload key=GZXHYZO5SMUP upload=BPITDHWXOVA4 tag=GAD3SUBT3GVQ type=write size=2345175 2024-04-07T16:37:41.611 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.617+0000 7f346a3868c0 1 < completed delete operation key=C1BTVJNHIM0X tag=981QMTS4U3OV type=del 2024-04-07T16:37:41.614 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.621+0000 7f346a3868c0 1 > prepared part key=_multipart_IP8YW2A75TCW.2~CUX7IAHJHN1O.0 size=609587 2024-04-07T16:37:41.614 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.621+0000 7f346a3868c0 1 > prepared multipart upload key=IP8YW2A75TCW upload=CUX7IAHJHN1O tag=UDJ8LKARVMVQ type=write size=1658163 2024-04-07T16:37:41.616 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.625+0000 7f346a3868c0 1 < completed delete operation key=QVOKSQAGHPA9 tag=DIK0WP8OQ4SS type=del 2024-04-07T16:37:41.620 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.629+0000 7f346a3868c0 1 > prepared part key=_multipart_DXTO3AJ429AC.2~OWUHIL6SN8E2.0 size=890691 2024-04-07T16:37:41.620 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.629+0000 7f346a3868c0 1 > prepared multipart upload key=DXTO3AJ429AC upload=OWUHIL6SN8E2 tag=UF3CPK0O8ZKT type=write size=1939267 2024-04-07T16:37:41.622 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.629+0000 7f346a3868c0 1 < completed delete operation key=N14BUXLJONHZ tag=ELY5WKYH0MHG type=del 2024-04-07T16:37:41.624 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.633+0000 7f346a3868c0 1 > prepared operation key=95YVKJ14JD8W tag=XAX9T3EI4ZLT type=del size=2704680 2024-04-07T16:37:41.626 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.633+0000 7f346a3868c0 1 < completed part key=_multipart_1WGUPH7X5UA0.2~3CFF8EMZJEOP.0 size=360905 2024-04-07T16:37:41.628 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.637+0000 7f346a3868c0 1 < completed multipart upload key=1WGUPH7X5UA0 upload=3CFF8EMZJEOP tag=Q069BPPD5PH1 type=write size=1409481 2024-04-07T16:37:41.630 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.637+0000 7f346a3868c0 1 > prepared operation key=1VKVQXSQMTU1 tag=2HYWUKRFCI13 type=del size=2431513 2024-04-07T16:37:41.632 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.641+0000 7f346a3868c0 1 < completed part key=_multipart_N14BUXLJONHZ.2~M5TDK6F4MF4K.0 size=1048576 2024-04-07T16:37:41.633 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.641+0000 7f346a3868c0 1 < completed part key=_multipart_N14BUXLJONHZ.2~M5TDK6F4MF4K.1 size=1048576 2024-04-07T16:37:41.633 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.641+0000 7f346a3868c0 1 < canceled part key=_multipart_N14BUXLJONHZ.2~M5TDK6F4MF4K.2 size=725494 2024-04-07T16:37:41.636 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.645+0000 7f346a3868c0 1 < canceled multipart upload key=N14BUXLJONHZ upload=M5TDK6F4MF4K tag=MNZ2B90OXOAN type=write size=3871222 2024-04-07T16:37:41.639 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.645+0000 7f346a3868c0 1 > prepared part key=_multipart_QVOKSQAGHPA9.2~2VR2O49HAGE7.0 size=1048576 2024-04-07T16:37:41.640 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.649+0000 7f346a3868c0 1 > prepared part key=_multipart_QVOKSQAGHPA9.2~2VR2O49HAGE7.1 size=230127 2024-04-07T16:37:41.640 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.649+0000 7f346a3868c0 1 > prepared multipart upload key=QVOKSQAGHPA9 upload=2VR2O49HAGE7 tag=WPEO2YPT2UM9 type=write size=2327279 2024-04-07T16:37:41.642 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.649+0000 7f346a3868c0 1 < completed part key=_multipart_1VBSUWNI3V1T.2~16KZSH5CG8AY.0 size=1048576 2024-04-07T16:37:41.642 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.649+0000 7f346a3868c0 1 < canceled part key=_multipart_1VBSUWNI3V1T.2~16KZSH5CG8AY.1 size=1048576 2024-04-07T16:37:41.643 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.653+0000 7f346a3868c0 1 < completed part key=_multipart_1VBSUWNI3V1T.2~16KZSH5CG8AY.2 size=137963 2024-04-07T16:37:41.646 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.653+0000 7f346a3868c0 1 < canceled multipart upload key=1VBSUWNI3V1T upload=16KZSH5CG8AY tag=KAJSXFI0608L type=write size=3283691 2024-04-07T16:37:41.650 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.657+0000 7f346a3868c0 1 > prepared part key=_multipart_1WGUPH7X5UA0.2~3EKFWKIQHL5E.0 size=1048576 2024-04-07T16:37:41.651 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.661+0000 7f346a3868c0 1 > prepared part key=_multipart_1WGUPH7X5UA0.2~3EKFWKIQHL5E.1 size=941781 2024-04-07T16:37:41.651 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.661+0000 7f346a3868c0 1 > prepared multipart upload key=1WGUPH7X5UA0 upload=3EKFWKIQHL5E tag=2FYW82DOUDSL type=write size=3038933 2024-04-07T16:37:41.653 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.661+0000 7f346a3868c0 1 < completed write operation key=C1BTVJNHIM0X tag=D4UED1EN5VIP type=write size=88942 2024-04-07T16:37:41.655 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.665+0000 7f346a3868c0 1 > prepared operation key=YXMDZB23YTB3 tag=1J8QEG6YDMKE type=del size=3610069 2024-04-07T16:37:41.657 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.665+0000 7f346a3868c0 1 < completed write operation key=YXMDZB23YTB3 tag=U24AHIG6JD5N type=write size=15067 2024-04-07T16:37:41.658 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.665+0000 7f346a3868c0 1 > prepared operation key=UIW3DN4NUYPP tag=3HPZEN0UWZX3 type=del size=4131083 2024-04-07T16:37:41.661 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.669+0000 7f346a3868c0 1 < completed part key=_multipart_QVOKSQAGHPA9.2~DL4KG82OSG2J.0 size=1048576 2024-04-07T16:37:41.662 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.669+0000 7f346a3868c0 1 < completed part key=_multipart_QVOKSQAGHPA9.2~DL4KG82OSG2J.1 size=40818 2024-04-07T16:37:41.666 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.673+0000 7f346a3868c0 1 < completed multipart upload key=QVOKSQAGHPA9 upload=DL4KG82OSG2J tag=GFO0DACY57XB type=write size=2137970 2024-04-07T16:37:41.668 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.677+0000 7f346a3868c0 1 > prepared operation key=LXOZ0T2KY6OR tag=5ODQ71OHBTHJ type=del size=2409782 2024-04-07T16:37:41.671 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.677+0000 7f346a3868c0 1 < completed write operation key=YXMDZB23YTB3 tag=73C0AGA492IV type=write size=632302 2024-04-07T16:37:41.675 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.685+0000 7f346a3868c0 1 > prepared part key=_multipart_N14BUXLJONHZ.2~WQ0QJIHEFG2M.0 size=823192 2024-04-07T16:37:41.675 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.685+0000 7f346a3868c0 1 > prepared multipart upload key=N14BUXLJONHZ upload=WQ0QJIHEFG2M tag=D07I0F43IMO0 type=write size=1871768 2024-04-07T16:37:41.677 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.685+0000 7f346a3868c0 1 < completed part key=_multipart_N14BUXLJONHZ.2~HRA9RS052UAC.0 size=24260 2024-04-07T16:37:41.681 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.689+0000 7f346a3868c0 1 < completed multipart upload key=N14BUXLJONHZ upload=HRA9RS052UAC tag=VY2BSUWSEHKY type=write size=1072836 2024-04-07T16:37:41.683 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.693+0000 7f346a3868c0 1 > prepared operation key=1VKVQXSQMTU1 tag=38M8YWZQB02H type=del size=725216 2024-04-07T16:37:41.685 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.693+0000 7f346a3868c0 1 < completed part key=_multipart_X5Q46GO8YTOK.2~8GO1JR8VCULC.0 size=1048576 2024-04-07T16:37:41.687 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.693+0000 7f346a3868c0 1 < completed part key=_multipart_X5Q46GO8YTOK.2~8GO1JR8VCULC.1 size=1048576 2024-04-07T16:37:41.688 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.697+0000 7f346a3868c0 1 < completed part key=_multipart_X5Q46GO8YTOK.2~8GO1JR8VCULC.2 size=978611 2024-04-07T16:37:41.692 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.701+0000 7f346a3868c0 1 < completed multipart upload key=X5Q46GO8YTOK upload=8GO1JR8VCULC tag=3XUI7K4JU7CH type=write size=4124339 2024-04-07T16:37:41.694 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.701+0000 7f346a3868c0 1 > prepared operation key=C1BTVJNHIM0X tag=5QQTL9XLO0T9 type=del size=1363793 2024-04-07T16:37:41.698 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.705+0000 7f346a3868c0 1 < suggested update operation key=XJLQZ7IS9U1T tag=MG5Z105ACXXM type=del size=826479 2024-04-07T16:37:41.700 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.709+0000 7f346a3868c0 1 > prepared operation key=GBQSME7SY1AH tag=PSBVC7G6C7KV type=del size=1000032 2024-04-07T16:37:41.702 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.709+0000 7f346a3868c0 1 < completed delete operation key=UIW3DN4NUYPP tag=LCY5L4DJRGJX type=del 2024-04-07T16:37:41.706 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.713+0000 7f346a3868c0 1 > prepared part key=_multipart_EQR0FY86AT5X.2~CSL7FMKOCP83.0 size=11452 2024-04-07T16:37:41.707 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.713+0000 7f346a3868c0 1 > prepared multipart upload key=EQR0FY86AT5X upload=CSL7FMKOCP83 tag=DEQT2AYGW4MR type=write size=1060028 2024-04-07T16:37:41.709 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.717+0000 7f346a3868c0 1 < completed part key=_multipart_QVOKSQAGHPA9.2~1LPT4I10WNE6.0 size=1048576 2024-04-07T16:37:41.711 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.717+0000 7f346a3868c0 1 < completed part key=_multipart_QVOKSQAGHPA9.2~1LPT4I10WNE6.1 size=1048576 2024-04-07T16:37:41.711 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.717+0000 7f346a3868c0 1 < canceled part key=_multipart_QVOKSQAGHPA9.2~1LPT4I10WNE6.2 size=49691 2024-04-07T16:37:41.714 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.721+0000 7f346a3868c0 1 < canceled multipart upload key=QVOKSQAGHPA9 upload=1LPT4I10WNE6 tag=220PZAF0HAMI type=write size=3195419 2024-04-07T16:37:41.716 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.725+0000 7f346a3868c0 1 > prepared operation key=6XDS0A7L0IY8 tag=WI9953MEH0T8 type=del size=2064980 2024-04-07T16:37:41.719 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.725+0000 7f346a3868c0 1 < completed part key=_multipart_GZXHYZO5SMUP.2~BPITDHWXOVA4.0 size=1048576 2024-04-07T16:37:41.721 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.729+0000 7f346a3868c0 1 < completed part key=_multipart_GZXHYZO5SMUP.2~BPITDHWXOVA4.1 size=248023 2024-04-07T16:37:41.724 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.733+0000 7f346a3868c0 1 < completed multipart upload key=GZXHYZO5SMUP upload=BPITDHWXOVA4 tag=GAD3SUBT3GVQ type=write size=2345175 2024-04-07T16:37:41.726 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.733+0000 7f346a3868c0 1 > prepared operation key=8F81G0P0YBFW tag=N3MEY5011IPV type=del size=1804262 2024-04-07T16:37:41.728 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.737+0000 7f346a3868c0 1 < completed part key=_multipart_IP8YW2A75TCW.2~CUX7IAHJHN1O.0 size=609587 2024-04-07T16:37:41.732 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.741+0000 7f346a3868c0 1 < completed multipart upload key=IP8YW2A75TCW upload=CUX7IAHJHN1O tag=UDJ8LKARVMVQ type=write size=1658163 2024-04-07T16:37:41.733 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.741+0000 7f346a3868c0 1 > prepared operation key=QS4HYZFFAJNF tag=66DDBNM7A8LE type=del size=3977665 2024-04-07T16:37:41.735 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.745+0000 7f346a3868c0 1 < completed part key=_multipart_DXTO3AJ429AC.2~OWUHIL6SN8E2.0 size=890691 2024-04-07T16:37:41.739 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.745+0000 7f346a3868c0 1 < completed multipart upload key=DXTO3AJ429AC upload=OWUHIL6SN8E2 tag=UF3CPK0O8ZKT type=write size=1939267 2024-04-07T16:37:41.741 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.749+0000 7f346a3868c0 1 > prepared operation key=C1BTVJNHIM0X tag=XDLKK6HHFANC type=del size=1304730 2024-04-07T16:37:41.743 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.753+0000 7f346a3868c0 1 < completed delete operation key=95YVKJ14JD8W tag=XAX9T3EI4ZLT type=del 2024-04-07T16:37:41.746 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.753+0000 7f346a3868c0 1 > prepared part key=_multipart_YXMDZB23YTB3.2~EN4TA8HN5QF2.0 size=1048576 2024-04-07T16:37:41.748 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.757+0000 7f346a3868c0 1 > prepared part key=_multipart_YXMDZB23YTB3.2~EN4TA8HN5QF2.1 size=1048576 2024-04-07T16:37:41.750 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.757+0000 7f346a3868c0 1 > prepared part key=_multipart_YXMDZB23YTB3.2~EN4TA8HN5QF2.2 size=131598 2024-04-07T16:37:41.750 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.757+0000 7f346a3868c0 1 > prepared multipart upload key=YXMDZB23YTB3 upload=EN4TA8HN5QF2 tag=JA9ALE0G1VR3 type=write size=3277326 2024-04-07T16:37:41.752 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.761+0000 7f346a3868c0 1 < completed delete operation key=1VKVQXSQMTU1 tag=2HYWUKRFCI13 type=del 2024-04-07T16:37:41.754 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.761+0000 7f346a3868c0 1 > prepared operation key=QS4HYZFFAJNF tag=A1C9WV2SPXQM type=del size=414938 2024-04-07T16:37:41.756 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.765+0000 7f346a3868c0 1 < completed part key=_multipart_QVOKSQAGHPA9.2~2VR2O49HAGE7.0 size=1048576 2024-04-07T16:37:41.757 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.765+0000 7f346a3868c0 1 < canceled part key=_multipart_QVOKSQAGHPA9.2~2VR2O49HAGE7.1 size=230127 2024-04-07T16:37:41.760 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.769+0000 7f346a3868c0 1 < canceled multipart upload key=QVOKSQAGHPA9 upload=2VR2O49HAGE7 tag=WPEO2YPT2UM9 type=write size=2327279 2024-04-07T16:37:41.763 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.769+0000 7f346a3868c0 1 > prepared operation key=3N0QJRYVO3FV tag=U032D8OUIB14 type=del size=4038211 2024-04-07T16:37:41.765 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.773+0000 7f346a3868c0 1 < completed part key=_multipart_1WGUPH7X5UA0.2~3EKFWKIQHL5E.0 size=1048576 2024-04-07T16:37:41.767 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.777+0000 7f346a3868c0 1 < completed part key=_multipart_1WGUPH7X5UA0.2~3EKFWKIQHL5E.1 size=941781 2024-04-07T16:37:41.771 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.781+0000 7f346a3868c0 1 < completed multipart upload key=1WGUPH7X5UA0 upload=3EKFWKIQHL5E tag=2FYW82DOUDSL type=write size=3038933 2024-04-07T16:37:41.774 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.781+0000 7f346a3868c0 1 > prepared operation key=C1BTVJNHIM0X tag=BBW0EAAD4W28 type=del size=809981 2024-04-07T16:37:41.776 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.785+0000 7f346a3868c0 1 < completed delete operation key=YXMDZB23YTB3 tag=1J8QEG6YDMKE type=del 2024-04-07T16:37:41.780 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.789+0000 7f346a3868c0 1 > prepared part key=_multipart_IP8YW2A75TCW.2~2X6HIS0T6NIK.0 size=1048576 2024-04-07T16:37:41.782 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.789+0000 7f346a3868c0 1 > prepared part key=_multipart_IP8YW2A75TCW.2~2X6HIS0T6NIK.1 size=820878 2024-04-07T16:37:41.782 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.789+0000 7f346a3868c0 1 > prepared multipart upload key=IP8YW2A75TCW upload=2X6HIS0T6NIK tag=QT7XHK9OE19R type=write size=2918030 2024-04-07T16:37:41.784 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.793+0000 7f346a3868c0 1 < canceled operation key=UIW3DN4NUYPP tag=3HPZEN0UWZX3 type=del size=4131083 2024-04-07T16:37:41.789 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.797+0000 7f346a3868c0 1 > prepared part key=_multipart_LXOZ0T2KY6OR.2~4YD8ZMSMK1JF.0 size=1048576 2024-04-07T16:37:41.790 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.797+0000 7f346a3868c0 1 > prepared part key=_multipart_LXOZ0T2KY6OR.2~4YD8ZMSMK1JF.1 size=1048576 2024-04-07T16:37:41.792 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.801+0000 7f346a3868c0 1 > prepared part key=_multipart_LXOZ0T2KY6OR.2~4YD8ZMSMK1JF.2 size=320654 2024-04-07T16:37:41.792 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.801+0000 7f346a3868c0 1 > prepared multipart upload key=LXOZ0T2KY6OR upload=4YD8ZMSMK1JF tag=QUO5740EHRI6 type=write size=3466382 2024-04-07T16:37:41.795 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.801+0000 7f346a3868c0 1 < canceled operation key=LXOZ0T2KY6OR tag=5ODQ71OHBTHJ type=del size=2409782 2024-04-07T16:37:41.799 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.809+0000 7f346a3868c0 1 > prepared part key=_multipart_6XDS0A7L0IY8.2~DISV7ES35PQH.0 size=1048576 2024-04-07T16:37:41.801 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.809+0000 7f346a3868c0 1 > prepared part key=_multipart_6XDS0A7L0IY8.2~DISV7ES35PQH.1 size=1048576 2024-04-07T16:37:41.803 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.809+0000 7f346a3868c0 1 > prepared part key=_multipart_6XDS0A7L0IY8.2~DISV7ES35PQH.2 size=1018296 2024-04-07T16:37:41.803 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.809+0000 7f346a3868c0 1 > prepared multipart upload key=6XDS0A7L0IY8 upload=DISV7ES35PQH tag=YXVXQ0ETUIDX type=write size=4164024 2024-04-07T16:37:41.803 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.813+0000 7f346a3868c0 1 < canceled part key=_multipart_N14BUXLJONHZ.2~WQ0QJIHEFG2M.0 size=823192 2024-04-07T16:37:41.807 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.817+0000 7f346a3868c0 1 < canceled multipart upload key=N14BUXLJONHZ upload=WQ0QJIHEFG2M tag=D07I0F43IMO0 type=write size=1871768 2024-04-07T16:37:41.812 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.821+0000 7f346a3868c0 1 > prepared part key=_multipart_3N0QJRYVO3FV.2~Z9FP3O9OYKV9.0 size=1048576 2024-04-07T16:37:41.814 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.821+0000 7f346a3868c0 1 > prepared part key=_multipart_3N0QJRYVO3FV.2~Z9FP3O9OYKV9.1 size=319627 2024-04-07T16:37:41.814 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.821+0000 7f346a3868c0 1 > prepared multipart upload key=3N0QJRYVO3FV upload=Z9FP3O9OYKV9 tag=M91H8MZU6FV5 type=write size=2416779 2024-04-07T16:37:41.816 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.825+0000 7f346a3868c0 1 < completed delete operation key=1VKVQXSQMTU1 tag=38M8YWZQB02H type=del 2024-04-07T16:37:41.820 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.829+0000 7f346a3868c0 1 > prepared part key=_multipart_EQR0FY86AT5X.2~39K3TPWWLH6P.0 size=462214 2024-04-07T16:37:41.820 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.829+0000 7f346a3868c0 1 > prepared multipart upload key=EQR0FY86AT5X upload=39K3TPWWLH6P tag=2DSE0ZB0ZQHT type=write size=1510790 2024-04-07T16:37:41.822 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.829+0000 7f346a3868c0 1 < canceled operation key=C1BTVJNHIM0X tag=5QQTL9XLO0T9 type=del size=1363793 2024-04-07T16:37:41.825 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.833+0000 7f346a3868c0 1 > prepared operation key=E84U0SX44NQ8 tag=2A81AI2FTK5L type=write size=561272 2024-04-07T16:37:41.827 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.833+0000 7f346a3868c0 1 < completed delete operation key=GBQSME7SY1AH tag=PSBVC7G6C7KV type=del 2024-04-07T16:37:41.829 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.837+0000 7f346a3868c0 1 > prepared operation key=QS4HYZFFAJNF tag=C7P47ODHB4JZ type=del size=3463531 2024-04-07T16:37:41.831 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.841+0000 7f346a3868c0 1 < completed part key=_multipart_EQR0FY86AT5X.2~CSL7FMKOCP83.0 size=11452 2024-04-07T16:37:41.835 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.845+0000 7f346a3868c0 1 < completed multipart upload key=EQR0FY86AT5X upload=CSL7FMKOCP83 tag=DEQT2AYGW4MR type=write size=1060028 2024-04-07T16:37:41.838 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.845+0000 7f346a3868c0 1 > prepared operation key=1WGUPH7X5UA0 tag=CZ1A768SIVO5 type=del size=583522 2024-04-07T16:37:41.841 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.849+0000 7f346a3868c0 1 < completed delete operation key=6XDS0A7L0IY8 tag=WI9953MEH0T8 type=del 2024-04-07T16:37:41.845 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.853+0000 7f346a3868c0 1 > prepared part key=_multipart_N14BUXLJONHZ.2~AUOE7N4EON1S.0 size=1048576 2024-04-07T16:37:41.846 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.853+0000 7f346a3868c0 1 > prepared part key=_multipart_N14BUXLJONHZ.2~AUOE7N4EON1S.1 size=956764 2024-04-07T16:37:41.846 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.853+0000 7f346a3868c0 1 > prepared multipart upload key=N14BUXLJONHZ upload=AUOE7N4EON1S tag=PSP6YF33G38T type=write size=3053916 2024-04-07T16:37:41.849 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.857+0000 7f346a3868c0 1 < completed delete operation key=8F81G0P0YBFW tag=N3MEY5011IPV type=del 2024-04-07T16:37:41.851 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.861+0000 7f346a3868c0 1 > prepared operation key=1WGUPH7X5UA0 tag=EQIA4HUG7ARF type=write size=129748 2024-04-07T16:37:41.854 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.861+0000 7f346a3868c0 1 < completed delete operation key=QS4HYZFFAJNF tag=66DDBNM7A8LE type=del 2024-04-07T16:37:41.856 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.865+0000 7f346a3868c0 1 > prepared operation key=HBLEXONKXJTL tag=C264E8DQHNXG type=write size=278733 2024-04-07T16:37:41.856 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:37:41 smithi144 bash[66654]: cluster 2024-04-07T16:37:40.452839+0000 mon.a (mon.0) 2009 : cluster [DBG] osdmap e1058: 8 total, 8 up, 8 in 2024-04-07T16:37:41.856 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:37:41 smithi144 bash[66654]: audit 2024-04-07T16:37:40.460847+0000 mon.c (mon.2) 57 : audit [INF] from='client.? 172.21.15.100:0/1163776616' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111470-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:37:41.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:37:41 smithi144 bash[66654]: audit 2024-04-07T16:37:40.474117+0000 mon.a (mon.0) 2010 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111470-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:37:41.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:37:41 smithi144 bash[66654]: audit 2024-04-07T16:37:40.767893+0000 mon.a (mon.0) 2011 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:37:41.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:37:41 smithi144 bash[66654]: audit 2024-04-07T16:37:41.447243+0000 mon.a (mon.0) 2012 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111470-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:37:41.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:37:41 smithi144 bash[66654]: cluster 2024-04-07T16:37:41.455555+0000 mon.a (mon.0) 2013 : cluster [DBG] osdmap e1059: 8 total, 8 up, 8 in 2024-04-07T16:37:41.858 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.865+0000 7f346a3868c0 1 < completed delete operation key=C1BTVJNHIM0X tag=XDLKK6HHFANC type=del 2024-04-07T16:37:41.861 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.869+0000 7f346a3868c0 1 > prepared operation key=N14BUXLJONHZ tag=5FUHCERT2AES type=del size=1315435 2024-04-07T16:37:41.864 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.873+0000 7f346a3868c0 1 < completed part key=_multipart_YXMDZB23YTB3.2~EN4TA8HN5QF2.0 size=1048576 2024-04-07T16:37:41.866 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.873+0000 7f346a3868c0 1 < completed part key=_multipart_YXMDZB23YTB3.2~EN4TA8HN5QF2.1 size=1048576 2024-04-07T16:37:41.868 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.877+0000 7f346a3868c0 1 < completed part key=_multipart_YXMDZB23YTB3.2~EN4TA8HN5QF2.2 size=131598 2024-04-07T16:37:41.872 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.881+0000 7f346a3868c0 1 < completed multipart upload key=YXMDZB23YTB3 upload=EN4TA8HN5QF2 tag=JA9ALE0G1VR3 type=write size=3277326 2024-04-07T16:37:41.875 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.881+0000 7f346a3868c0 1 > prepared operation key=GBQSME7SY1AH tag=7CFX9B3WZILK type=del size=1574887 2024-04-07T16:37:41.877 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.885+0000 7f346a3868c0 1 < completed delete operation key=QS4HYZFFAJNF tag=A1C9WV2SPXQM type=del 2024-04-07T16:37:41.880 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.889+0000 7f346a3868c0 1 > prepared operation key=ROFH30W3UCAX tag=H307O6PW48F1 type=del size=2224935 2024-04-07T16:37:41.885 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.893+0000 7f346a3868c0 1 < suggested update operation key=3N0QJRYVO3FV tag=U032D8OUIB14 type=del size=4038211 2024-04-07T16:37:41.889 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.897+0000 7f346a3868c0 1 > prepared part key=_multipart_95YVKJ14JD8W.2~THLMEQ74780D.0 size=788619 2024-04-07T16:37:41.889 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.897+0000 7f346a3868c0 1 > prepared multipart upload key=95YVKJ14JD8W upload=THLMEQ74780D tag=KUYPW7BDW4TO type=write size=1837195 2024-04-07T16:37:41.891 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.901+0000 7f346a3868c0 1 < completed delete operation key=C1BTVJNHIM0X tag=BBW0EAAD4W28 type=del 2024-04-07T16:37:41.895 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.901+0000 7f346a3868c0 1 > prepared operation key=CL4PP1RKZU0E tag=8BE3SR2MZ1P1 type=write size=755185 2024-04-07T16:37:41.897 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.905+0000 7f346a3868c0 1 < completed part key=_multipart_IP8YW2A75TCW.2~2X6HIS0T6NIK.0 size=1048576 2024-04-07T16:37:41.900 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.909+0000 7f346a3868c0 1 < completed part key=_multipart_IP8YW2A75TCW.2~2X6HIS0T6NIK.1 size=820878 2024-04-07T16:37:41.905 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.913+0000 7f346a3868c0 1 < completed multipart upload key=IP8YW2A75TCW upload=2X6HIS0T6NIK tag=QT7XHK9OE19R type=write size=2918030 2024-04-07T16:37:41.910 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.917+0000 7f346a3868c0 1 > prepared part key=_multipart_EOYFVEI50SQ1.2~55K28HII14P6.0 size=1048576 2024-04-07T16:37:41.912 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.921+0000 7f346a3868c0 1 > prepared part key=_multipart_EOYFVEI50SQ1.2~55K28HII14P6.1 size=1002335 2024-04-07T16:37:41.912 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.921+0000 7f346a3868c0 1 > prepared multipart upload key=EOYFVEI50SQ1 upload=55K28HII14P6 tag=LK8WOAX9YQA2 type=write size=3099487 2024-04-07T16:37:41.915 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.921+0000 7f346a3868c0 1 < completed part key=_multipart_LXOZ0T2KY6OR.2~4YD8ZMSMK1JF.0 size=1048576 2024-04-07T16:37:41.917 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.925+0000 7f346a3868c0 1 < completed part key=_multipart_LXOZ0T2KY6OR.2~4YD8ZMSMK1JF.1 size=1048576 2024-04-07T16:37:41.919 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.925+0000 7f346a3868c0 1 < completed part key=_multipart_LXOZ0T2KY6OR.2~4YD8ZMSMK1JF.2 size=320654 2024-04-07T16:37:41.923 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.933+0000 7f346a3868c0 1 < completed multipart upload key=LXOZ0T2KY6OR upload=4YD8ZMSMK1JF tag=QUO5740EHRI6 type=write size=3466382 2024-04-07T16:37:41.926 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.933+0000 7f346a3868c0 1 > prepared operation key=6XDS0A7L0IY8 tag=MN2A306Y7ATB type=del size=3746751 2024-04-07T16:37:41.929 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.937+0000 7f346a3868c0 1 < completed part key=_multipart_6XDS0A7L0IY8.2~DISV7ES35PQH.0 size=1048576 2024-04-07T16:37:41.930 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.937+0000 7f346a3868c0 1 < completed part key=_multipart_6XDS0A7L0IY8.2~DISV7ES35PQH.1 size=1048576 2024-04-07T16:37:41.933 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.941+0000 7f346a3868c0 1 < completed part key=_multipart_6XDS0A7L0IY8.2~DISV7ES35PQH.2 size=1018296 2024-04-07T16:37:41.937 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.945+0000 7f346a3868c0 1 < completed multipart upload key=6XDS0A7L0IY8 upload=DISV7ES35PQH tag=YXVXQ0ETUIDX type=write size=4164024 2024-04-07T16:37:41.940 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.949+0000 7f346a3868c0 1 > prepared operation key=1WGUPH7X5UA0 tag=LK3FUE7R3A8U type=write size=533457 2024-04-07T16:37:41.941 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:37:41 smithi100 bash[74942]: cluster 2024-04-07T16:37:40.452839+0000 mon.a (mon.0) 2009 : cluster [DBG] osdmap e1058: 8 total, 8 up, 8 in 2024-04-07T16:37:41.941 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:37:41 smithi100 bash[74942]: audit 2024-04-07T16:37:40.460847+0000 mon.c (mon.2) 57 : audit [INF] from='client.? 172.21.15.100:0/1163776616' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111470-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:37:41.941 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:37:41 smithi100 bash[74942]: audit 2024-04-07T16:37:40.474117+0000 mon.a (mon.0) 2010 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111470-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:37:41.941 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:37:41 smithi100 bash[74942]: audit 2024-04-07T16:37:40.767893+0000 mon.a (mon.0) 2011 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:37:41.941 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:37:41 smithi100 bash[74942]: audit 2024-04-07T16:37:41.447243+0000 mon.a (mon.0) 2012 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111470-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:37:41.941 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:37:41 smithi100 bash[74942]: cluster 2024-04-07T16:37:41.455555+0000 mon.a (mon.0) 2013 : cluster [DBG] osdmap e1059: 8 total, 8 up, 8 in 2024-04-07T16:37:41.941 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:37:41 smithi100 bash[76202]: cluster 2024-04-07T16:37:40.452839+0000 mon.a (mon.0) 2009 : cluster [DBG] osdmap e1058: 8 total, 8 up, 8 in 2024-04-07T16:37:41.942 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:37:41 smithi100 bash[76202]: audit 2024-04-07T16:37:40.460847+0000 mon.c (mon.2) 57 : audit [INF] from='client.? 172.21.15.100:0/1163776616' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111470-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:37:41.942 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:37:41 smithi100 bash[76202]: audit 2024-04-07T16:37:40.474117+0000 mon.a (mon.0) 2010 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111470-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:37:41.942 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:37:41 smithi100 bash[76202]: audit 2024-04-07T16:37:40.767893+0000 mon.a (mon.0) 2011 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:37:41.942 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:37:41 smithi100 bash[76202]: audit 2024-04-07T16:37:41.447243+0000 mon.a (mon.0) 2012 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111470-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:37:41.942 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:37:41 smithi100 bash[76202]: cluster 2024-04-07T16:37:41.455555+0000 mon.a (mon.0) 2013 : cluster [DBG] osdmap e1059: 8 total, 8 up, 8 in 2024-04-07T16:37:41.943 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.953+0000 7f346a3868c0 1 < completed part key=_multipart_3N0QJRYVO3FV.2~Z9FP3O9OYKV9.0 size=1048576 2024-04-07T16:37:41.945 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.953+0000 7f346a3868c0 1 < completed part key=_multipart_3N0QJRYVO3FV.2~Z9FP3O9OYKV9.1 size=319627 2024-04-07T16:37:41.950 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.957+0000 7f346a3868c0 1 < completed multipart upload key=3N0QJRYVO3FV upload=Z9FP3O9OYKV9 tag=M91H8MZU6FV5 type=write size=2416779 2024-04-07T16:37:41.953 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.961+0000 7f346a3868c0 1 > prepared operation key=1WGUPH7X5UA0 tag=61XO6TIGF1AT type=del size=316798 2024-04-07T16:37:41.956 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.965+0000 7f346a3868c0 1 < completed part key=_multipart_EQR0FY86AT5X.2~39K3TPWWLH6P.0 size=462214 2024-04-07T16:37:41.962 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.969+0000 7f346a3868c0 1 < completed multipart upload key=EQR0FY86AT5X upload=39K3TPWWLH6P tag=2DSE0ZB0ZQHT type=write size=1510790 2024-04-07T16:37:41.965 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.973+0000 7f346a3868c0 1 > prepared operation key=QTCIIPTXIRW2 tag=GBF0XIMERCRE type=write size=469649 2024-04-07T16:37:41.972 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.981+0000 7f346a3868c0 1 < suggested remove operation key=E84U0SX44NQ8 tag=2A81AI2FTK5L type=write 2024-04-07T16:37:41.978 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.985+0000 7f346a3868c0 1 > prepared part key=_multipart_U3EID5EK64HD.2~RBFGM0R9LVVY.0 size=515861 2024-04-07T16:37:41.978 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.985+0000 7f346a3868c0 1 > prepared multipart upload key=U3EID5EK64HD upload=RBFGM0R9LVVY tag=S9CTN816OJNN type=write size=1564437 2024-04-07T16:37:41.981 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.989+0000 7f346a3868c0 1 < completed delete operation key=QS4HYZFFAJNF tag=C7P47ODHB4JZ type=del 2024-04-07T16:37:41.984 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:41.993+0000 7f346a3868c0 1 > prepared operation key=EOYFVEI50SQ1 tag=W04H9EXEMNGG type=del size=85324 2024-04-07T16:37:41.991 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.001+0000 7f346a3868c0 1 < suggested remove operation key=1WGUPH7X5UA0 tag=CZ1A768SIVO5 type=del 2024-04-07T16:37:41.995 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.001+0000 7f346a3868c0 1 > prepared operation key=YOK8BAEOMPUR tag=YXGB56RKXHDC type=write size=65709 2024-04-07T16:37:41.998 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.005+0000 7f346a3868c0 1 < completed part key=_multipart_N14BUXLJONHZ.2~AUOE7N4EON1S.0 size=1048576 2024-04-07T16:37:42.001 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.009+0000 7f346a3868c0 1 < completed part key=_multipart_N14BUXLJONHZ.2~AUOE7N4EON1S.1 size=956764 2024-04-07T16:37:42.005 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.013+0000 7f346a3868c0 1 < completed multipart upload key=N14BUXLJONHZ upload=AUOE7N4EON1S tag=PSP6YF33G38T type=write size=3053916 2024-04-07T16:37:42.008 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.017+0000 7f346a3868c0 1 > prepared operation key=EQR0FY86AT5X tag=1STOKICP3XY8 type=write size=720090 2024-04-07T16:37:42.011 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.021+0000 7f346a3868c0 -1 < failed to complete operation key=1WGUPH7X5UA0 tag=EQIA4HUG7ARF type=write size=129748: (22) Invalid argument 2024-04-07T16:37:42.014 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.021+0000 7f346a3868c0 1 > prepared operation key=1VKVQXSQMTU1 tag=HRB771BX5VDD type=del size=2140906 2024-04-07T16:37:42.017 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.025+0000 7f346a3868c0 1 < canceled operation key=HBLEXONKXJTL tag=C264E8DQHNXG type=write size=278733 2024-04-07T16:37:42.022 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.029+0000 7f346a3868c0 1 > prepared part key=_multipart_X5Q46GO8YTOK.2~5NA35KU9USME.0 size=1048576 2024-04-07T16:37:42.024 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.033+0000 7f346a3868c0 1 > prepared part key=_multipart_X5Q46GO8YTOK.2~5NA35KU9USME.1 size=411370 2024-04-07T16:37:42.024 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.033+0000 7f346a3868c0 1 > prepared multipart upload key=X5Q46GO8YTOK upload=5NA35KU9USME tag=KKTV403EDLY9 type=write size=2508522 2024-04-07T16:37:42.030 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.037+0000 7f346a3868c0 1 < suggested remove operation key=N14BUXLJONHZ tag=5FUHCERT2AES type=del 2024-04-07T16:37:42.035 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.045+0000 7f346a3868c0 1 > prepared part key=_multipart_EQR0FY86AT5X.2~03ATV57B4ROZ.0 size=1048576 2024-04-07T16:37:42.037 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.045+0000 7f346a3868c0 1 > prepared part key=_multipart_EQR0FY86AT5X.2~03ATV57B4ROZ.1 size=199126 2024-04-07T16:37:42.037 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.045+0000 7f346a3868c0 1 > prepared multipart upload key=EQR0FY86AT5X upload=03ATV57B4ROZ tag=67ROYDYZCPE8 type=write size=2296278 2024-04-07T16:37:42.040 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.049+0000 7f346a3868c0 1 < completed delete operation key=GBQSME7SY1AH tag=7CFX9B3WZILK type=del 2024-04-07T16:37:42.043 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.053+0000 7f346a3868c0 1 > prepared operation key=1VKVQXSQMTU1 tag=6PJ2IL7MQ7Y4 type=del size=2123864 2024-04-07T16:37:42.046 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.053+0000 7f346a3868c0 1 < completed delete operation key=ROFH30W3UCAX tag=H307O6PW48F1 type=del 2024-04-07T16:37:42.051 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.061+0000 7f346a3868c0 1 > prepared part key=_multipart_DXTO3AJ429AC.2~GZDYHHYTSO0F.0 size=1048576 2024-04-07T16:37:42.054 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.061+0000 7f346a3868c0 1 > prepared part key=_multipart_DXTO3AJ429AC.2~GZDYHHYTSO0F.1 size=1048576 2024-04-07T16:37:42.056 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.065+0000 7f346a3868c0 1 > prepared part key=_multipart_DXTO3AJ429AC.2~GZDYHHYTSO0F.2 size=42257 2024-04-07T16:37:42.056 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.065+0000 7f346a3868c0 1 > prepared multipart upload key=DXTO3AJ429AC upload=GZDYHHYTSO0F tag=2Z67Q5E124VZ type=write size=3187985 2024-04-07T16:37:42.058 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.065+0000 7f346a3868c0 1 < completed part key=_multipart_95YVKJ14JD8W.2~THLMEQ74780D.0 size=788619 2024-04-07T16:37:42.063 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.073+0000 7f346a3868c0 1 < completed multipart upload key=95YVKJ14JD8W upload=THLMEQ74780D tag=KUYPW7BDW4TO type=write size=1837195 2024-04-07T16:37:42.066 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.073+0000 7f346a3868c0 1 > prepared operation key=3N0QJRYVO3FV tag=OVMUEMKQ494O type=del size=1903872 2024-04-07T16:37:42.073 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.081+0000 7f346a3868c0 1 < suggested update operation key=CL4PP1RKZU0E tag=8BE3SR2MZ1P1 type=write size=755185 2024-04-07T16:37:42.076 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.085+0000 7f346a3868c0 1 > prepared operation key=1WGUPH7X5UA0 tag=11YDE12RNTIT type=del size=1141597 2024-04-07T16:37:42.079 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.089+0000 7f346a3868c0 1 < completed part key=_multipart_EOYFVEI50SQ1.2~55K28HII14P6.0 size=1048576 2024-04-07T16:37:42.082 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.089+0000 7f346a3868c0 1 < completed part key=_multipart_EOYFVEI50SQ1.2~55K28HII14P6.1 size=1002335 2024-04-07T16:37:42.087 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.093+0000 7f346a3868c0 1 < completed multipart upload key=EOYFVEI50SQ1 upload=55K28HII14P6 tag=LK8WOAX9YQA2 type=write size=3099487 2024-04-07T16:37:42.090 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.097+0000 7f346a3868c0 1 > prepared operation key=1WGUPH7X5UA0 tag=NZ5GRZKNZ8RT type=del size=1787059 2024-04-07T16:37:42.094 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.101+0000 7f346a3868c0 1 < completed delete operation key=6XDS0A7L0IY8 tag=MN2A306Y7ATB type=del 2024-04-07T16:37:42.097 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.105+0000 7f346a3868c0 1 > prepared operation key=CL4PP1RKZU0E tag=F1XTS0RFW4N3 type=del size=910227 2024-04-07T16:37:42.104 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.113+0000 7f346a3868c0 1 < suggested remove operation key=1WGUPH7X5UA0 tag=LK3FUE7R3A8U type=write 2024-04-07T16:37:42.110 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.117+0000 7f346a3868c0 1 > prepared part key=_multipart_X5Q46GO8YTOK.2~PAXD5VM4S2IP.0 size=1048576 2024-04-07T16:37:42.112 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.121+0000 7f346a3868c0 1 > prepared part key=_multipart_X5Q46GO8YTOK.2~PAXD5VM4S2IP.1 size=669713 2024-04-07T16:37:42.112 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.121+0000 7f346a3868c0 1 > prepared multipart upload key=X5Q46GO8YTOK upload=PAXD5VM4S2IP tag=0RML1CF9HQVU type=write size=2766865 2024-04-07T16:37:42.116 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.125+0000 7f346a3868c0 -1 < failed to complete operation key=1WGUPH7X5UA0 tag=61XO6TIGF1AT type=del size=316798: (22) Invalid argument 2024-04-07T16:37:42.119 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.129+0000 7f346a3868c0 1 > prepared operation key=IX3F0IUOKBX1 tag=SV1ZYYYHQG0S type=del size=1453756 2024-04-07T16:37:42.124 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.133+0000 7f346a3868c0 1 < completed write operation key=QTCIIPTXIRW2 tag=GBF0XIMERCRE type=write size=469649 2024-04-07T16:37:42.128 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.137+0000 7f346a3868c0 1 > prepared operation key=1VBSUWNI3V1T tag=KX901YKLBYK8 type=write size=771476 2024-04-07T16:37:42.132 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.141+0000 7f346a3868c0 1 < completed part key=_multipart_U3EID5EK64HD.2~RBFGM0R9LVVY.0 size=515861 2024-04-07T16:37:42.137 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.145+0000 7f346a3868c0 1 < completed multipart upload key=U3EID5EK64HD upload=RBFGM0R9LVVY tag=S9CTN816OJNN type=write size=1564437 2024-04-07T16:37:42.143 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.153+0000 7f346a3868c0 1 > prepared part key=_multipart_YXMDZB23YTB3.2~95O552BIB1O7.0 size=1048576 2024-04-07T16:37:42.146 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.153+0000 7f346a3868c0 1 > prepared part key=_multipart_YXMDZB23YTB3.2~95O552BIB1O7.1 size=13973 2024-04-07T16:37:42.146 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.153+0000 7f346a3868c0 1 > prepared multipart upload key=YXMDZB23YTB3 upload=95O552BIB1O7 tag=QT3RIESRZOFA type=write size=2111125 2024-04-07T16:37:42.150 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.157+0000 7f346a3868c0 1 < completed delete operation key=EOYFVEI50SQ1 tag=W04H9EXEMNGG type=del 2024-04-07T16:37:42.153 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.161+0000 7f346a3868c0 1 > prepared operation key=1VBSUWNI3V1T tag=W7ALLZJXQFZU type=del size=3829217 2024-04-07T16:37:42.157 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.165+0000 7f346a3868c0 1 < canceled operation key=YOK8BAEOMPUR tag=YXGB56RKXHDC type=write size=65709 2024-04-07T16:37:42.163 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.173+0000 7f346a3868c0 1 > prepared part key=_multipart_GBQSME7SY1AH.2~R0LRRTHCIHRG.0 size=1048576 2024-04-07T16:37:42.166 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.173+0000 7f346a3868c0 1 > prepared part key=_multipart_GBQSME7SY1AH.2~R0LRRTHCIHRG.1 size=362784 2024-04-07T16:37:42.166 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.173+0000 7f346a3868c0 1 > prepared multipart upload key=GBQSME7SY1AH upload=R0LRRTHCIHRG tag=17EX2S48KBBS type=write size=2459936 2024-04-07T16:37:42.169 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.177+0000 7f346a3868c0 1 < completed write operation key=EQR0FY86AT5X tag=1STOKICP3XY8 type=write size=720090 2024-04-07T16:37:42.173 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.181+0000 7f346a3868c0 1 > prepared operation key=N14BUXLJONHZ tag=SED91Q785PMQ type=del size=1632564 2024-04-07T16:37:42.176 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.185+0000 7f346a3868c0 1 < canceled operation key=1VKVQXSQMTU1 tag=HRB771BX5VDD type=del size=2140906 2024-04-07T16:37:42.180 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.189+0000 7f346a3868c0 1 > prepared operation key=CL4PP1RKZU0E tag=XNLCZPS2ZK43 type=del size=1910371 2024-04-07T16:37:42.181 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.189+0000 7f346a3868c0 1 < canceled part key=_multipart_X5Q46GO8YTOK.2~5NA35KU9USME.0 size=1048576 2024-04-07T16:37:42.181 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.189+0000 7f346a3868c0 1 < canceled part key=_multipart_X5Q46GO8YTOK.2~5NA35KU9USME.1 size=411370 2024-04-07T16:37:42.186 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.193+0000 7f346a3868c0 1 < canceled multipart upload key=X5Q46GO8YTOK upload=5NA35KU9USME tag=KKTV403EDLY9 type=write size=2508522 2024-04-07T16:37:42.194 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.201+0000 7f346a3868c0 1 > prepared part key=_multipart_U3EID5EK64HD.2~0CT82NYMM5SK.0 size=1048576 2024-04-07T16:37:42.197 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.205+0000 7f346a3868c0 1 > prepared part key=_multipart_U3EID5EK64HD.2~0CT82NYMM5SK.1 size=224172 2024-04-07T16:37:42.197 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.205+0000 7f346a3868c0 1 > prepared multipart upload key=U3EID5EK64HD upload=0CT82NYMM5SK tag=T42A3KDKATQY type=write size=2321324 2024-04-07T16:37:42.201 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.209+0000 7f346a3868c0 1 < completed part key=_multipart_EQR0FY86AT5X.2~03ATV57B4ROZ.0 size=1048576 2024-04-07T16:37:42.203 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.213+0000 7f346a3868c0 1 < completed part key=_multipart_EQR0FY86AT5X.2~03ATV57B4ROZ.1 size=199126 2024-04-07T16:37:42.209 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.217+0000 7f346a3868c0 1 < completed multipart upload key=EQR0FY86AT5X upload=03ATV57B4ROZ tag=67ROYDYZCPE8 type=write size=2296278 2024-04-07T16:37:42.213 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.221+0000 7f346a3868c0 1 > prepared operation key=ROFH30W3UCAX tag=GJ24IYS04817 type=del size=333418 2024-04-07T16:37:42.216 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.225+0000 7f346a3868c0 1 < completed delete operation key=1VKVQXSQMTU1 tag=6PJ2IL7MQ7Y4 type=del 2024-04-07T16:37:42.220 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.229+0000 7f346a3868c0 1 > prepared operation key=EOYFVEI50SQ1 tag=KSH83WSBTA3M type=del size=1858148 2024-04-07T16:37:42.225 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.233+0000 7f346a3868c0 1 < completed part key=_multipart_DXTO3AJ429AC.2~GZDYHHYTSO0F.0 size=1048576 2024-04-07T16:37:42.226 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.233+0000 7f346a3868c0 1 < completed part key=_multipart_DXTO3AJ429AC.2~GZDYHHYTSO0F.1 size=1048576 2024-04-07T16:37:42.229 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.237+0000 7f346a3868c0 1 < completed part key=_multipart_DXTO3AJ429AC.2~GZDYHHYTSO0F.2 size=42257 2024-04-07T16:37:42.235 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.245+0000 7f346a3868c0 1 < completed multipart upload key=DXTO3AJ429AC upload=GZDYHHYTSO0F tag=2Z67Q5E124VZ type=write size=3187985 2024-04-07T16:37:42.239 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.249+0000 7f346a3868c0 1 > prepared operation key=1VBSUWNI3V1T tag=3LHLGWE6UIL4 type=del size=607161 2024-04-07T16:37:42.247 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.253+0000 7f346a3868c0 1 < suggested remove operation key=3N0QJRYVO3FV tag=OVMUEMKQ494O type=del 2024-04-07T16:37:42.250 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.257+0000 7f346a3868c0 1 > prepared operation key=QVOKSQAGHPA9 tag=5CL8VH9OH9JQ type=write size=639281 2024-04-07T16:37:42.254 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.261+0000 7f346a3868c0 -1 < failed to complete operation key=1WGUPH7X5UA0 tag=11YDE12RNTIT type=del size=1141597: (22) Invalid argument 2024-04-07T16:37:42.258 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.265+0000 7f346a3868c0 1 > prepared operation key=GBQSME7SY1AH tag=US4M9F0HP42A type=del size=3328634 2024-04-07T16:37:42.260 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.269+0000 7f346a3868c0 -1 < no bi entry to suggest for operation key=1WGUPH7X5UA0 tag=NZ5GRZKNZ8RT type=del size=1787059: (2) No such file or directory 2024-04-07T16:37:42.263 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.273+0000 7f346a3868c0 1 > prepared operation key=EQR0FY86AT5X tag=JAMGI4PYXKUS type=del size=1783154 2024-04-07T16:37:42.267 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.277+0000 7f346a3868c0 1 < completed delete operation key=CL4PP1RKZU0E tag=F1XTS0RFW4N3 type=del 2024-04-07T16:37:42.271 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.281+0000 7f346a3868c0 1 > prepared operation key=3N0QJRYVO3FV tag=P6IESFD7POX5 type=del size=279664 2024-04-07T16:37:42.275 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.285+0000 7f346a3868c0 1 < completed part key=_multipart_X5Q46GO8YTOK.2~PAXD5VM4S2IP.0 size=1048576 2024-04-07T16:37:42.278 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.285+0000 7f346a3868c0 1 < completed part key=_multipart_X5Q46GO8YTOK.2~PAXD5VM4S2IP.1 size=669713 2024-04-07T16:37:42.284 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.293+0000 7f346a3868c0 1 < completed multipart upload key=X5Q46GO8YTOK upload=PAXD5VM4S2IP tag=0RML1CF9HQVU type=write size=2766865 2024-04-07T16:37:42.290 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.297+0000 7f346a3868c0 1 > prepared part key=_multipart_GZXHYZO5SMUP.2~Y0MWRK0SRI2R.0 size=1048576 2024-04-07T16:37:42.293 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.301+0000 7f346a3868c0 1 > prepared part key=_multipart_GZXHYZO5SMUP.2~Y0MWRK0SRI2R.1 size=702967 2024-04-07T16:37:42.293 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.301+0000 7f346a3868c0 1 > prepared multipart upload key=GZXHYZO5SMUP upload=Y0MWRK0SRI2R tag=174OPCXA1UI2 type=write size=2800119 2024-04-07T16:37:42.297 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.305+0000 7f346a3868c0 1 < completed delete operation key=IX3F0IUOKBX1 tag=SV1ZYYYHQG0S type=del 2024-04-07T16:37:42.300 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.309+0000 7f346a3868c0 1 > prepared operation key=95YVKJ14JD8W tag=GGL3GHXG2LMM type=write size=532042 2024-04-07T16:37:42.304 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.313+0000 7f346a3868c0 1 < completed write operation key=1VBSUWNI3V1T tag=KX901YKLBYK8 type=write size=771476 2024-04-07T16:37:42.311 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.317+0000 7f346a3868c0 1 > prepared part key=_multipart_1WGUPH7X5UA0.2~SM4IMSBHWX80.0 size=570736 2024-04-07T16:37:42.311 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.317+0000 7f346a3868c0 1 > prepared multipart upload key=1WGUPH7X5UA0 upload=SM4IMSBHWX80 tag=2OT6IF6MDMMA type=write size=1619312 2024-04-07T16:37:42.312 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.321+0000 7f346a3868c0 1 < canceled part key=_multipart_YXMDZB23YTB3.2~95O552BIB1O7.0 size=1048576 2024-04-07T16:37:42.315 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.321+0000 7f346a3868c0 1 < completed part key=_multipart_YXMDZB23YTB3.2~95O552BIB1O7.1 size=13973 2024-04-07T16:37:42.321 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.329+0000 7f346a3868c0 1 < canceled multipart upload key=YXMDZB23YTB3 upload=95O552BIB1O7 tag=QT3RIESRZOFA type=write size=2111125 2024-04-07T16:37:42.327 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.337+0000 7f346a3868c0 1 > prepared part key=_multipart_7G5C6HBK6QWI.2~A6FF0B31XRHK.0 size=1048576 2024-04-07T16:37:42.330 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.337+0000 7f346a3868c0 1 > prepared part key=_multipart_7G5C6HBK6QWI.2~A6FF0B31XRHK.1 size=1048576 2024-04-07T16:37:42.332 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.341+0000 7f346a3868c0 1 > prepared part key=_multipart_7G5C6HBK6QWI.2~A6FF0B31XRHK.2 size=293873 2024-04-07T16:37:42.332 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.341+0000 7f346a3868c0 1 > prepared multipart upload key=7G5C6HBK6QWI upload=A6FF0B31XRHK tag=38WXCE5Z1X7Y type=write size=3439601 2024-04-07T16:37:42.335 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.345+0000 7f346a3868c0 1 < completed delete operation key=1VBSUWNI3V1T tag=W7ALLZJXQFZU type=del 2024-04-07T16:37:42.339 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.345+0000 7f346a3868c0 1 > prepared operation key=QTCIIPTXIRW2 tag=919S1ZOE5PYS type=del size=2313538 2024-04-07T16:37:42.342 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.349+0000 7f346a3868c0 1 < completed part key=_multipart_GBQSME7SY1AH.2~R0LRRTHCIHRG.0 size=1048576 2024-04-07T16:37:42.345 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.353+0000 7f346a3868c0 1 < completed part key=_multipart_GBQSME7SY1AH.2~R0LRRTHCIHRG.1 size=362784 2024-04-07T16:37:42.351 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.361+0000 7f346a3868c0 1 < completed multipart upload key=GBQSME7SY1AH upload=R0LRRTHCIHRG tag=17EX2S48KBBS type=write size=2459936 2024-04-07T16:37:42.355 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.365+0000 7f346a3868c0 1 > prepared operation key=3N0QJRYVO3FV tag=076SE3VDBSSW type=del size=2800766 2024-04-07T16:37:42.358 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.365+0000 7f346a3868c0 1 < canceled operation key=N14BUXLJONHZ tag=SED91Q785PMQ type=del size=1632564 2024-04-07T16:37:42.365 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.373+0000 7f346a3868c0 1 > prepared part key=_multipart_1WGUPH7X5UA0.2~0G2NBJYTANBT.0 size=259506 2024-04-07T16:37:42.365 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.373+0000 7f346a3868c0 1 > prepared multipart upload key=1WGUPH7X5UA0 upload=0G2NBJYTANBT tag=UCEHOS99JOT8 type=write size=1308082 2024-04-07T16:37:42.368 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.377+0000 7f346a3868c0 1 < completed delete operation key=CL4PP1RKZU0E tag=XNLCZPS2ZK43 type=del 2024-04-07T16:37:42.375 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.385+0000 7f346a3868c0 1 > prepared part key=_multipart_8F81G0P0YBFW.2~150KSM7J33JG.0 size=829225 2024-04-07T16:37:42.375 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.385+0000 7f346a3868c0 1 > prepared multipart upload key=8F81G0P0YBFW upload=150KSM7J33JG tag=HHS565LHFO7K type=write size=1877801 2024-04-07T16:37:42.378 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.385+0000 7f346a3868c0 1 < completed part key=_multipart_U3EID5EK64HD.2~0CT82NYMM5SK.0 size=1048576 2024-04-07T16:37:42.381 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.389+0000 7f346a3868c0 1 < completed part key=_multipart_U3EID5EK64HD.2~0CT82NYMM5SK.1 size=224172 2024-04-07T16:37:42.387 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.393+0000 7f346a3868c0 1 < completed multipart upload key=U3EID5EK64HD upload=0CT82NYMM5SK tag=T42A3KDKATQY type=write size=2321324 2024-04-07T16:37:42.393 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.401+0000 7f346a3868c0 1 > prepared part key=_multipart_DXTO3AJ429AC.2~IWCZV5HBMK0J.0 size=1048576 2024-04-07T16:37:42.396 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.405+0000 7f346a3868c0 1 > prepared part key=_multipart_DXTO3AJ429AC.2~IWCZV5HBMK0J.1 size=1048576 2024-04-07T16:37:42.399 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.405+0000 7f346a3868c0 1 > prepared part key=_multipart_DXTO3AJ429AC.2~IWCZV5HBMK0J.2 size=425083 2024-04-07T16:37:42.399 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.405+0000 7f346a3868c0 1 > prepared multipart upload key=DXTO3AJ429AC upload=IWCZV5HBMK0J tag=VV7Q16C4E0PP type=write size=3570811 2024-04-07T16:37:42.402 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.409+0000 7f346a3868c0 1 < completed delete operation key=ROFH30W3UCAX tag=GJ24IYS04817 type=del 2024-04-07T16:37:42.406 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.413+0000 7f346a3868c0 1 > prepared operation key=DXTO3AJ429AC tag=W8TRZZOX5HV6 type=write size=422156 2024-04-07T16:37:42.409 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.417+0000 7f346a3868c0 1 < completed delete operation key=EOYFVEI50SQ1 tag=KSH83WSBTA3M type=del 2024-04-07T16:37:42.416 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.425+0000 7f346a3868c0 1 > prepared part key=_multipart_U3EID5EK64HD.2~3X7J6HH77MNB.0 size=1048576 2024-04-07T16:37:42.418 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.425+0000 7f346a3868c0 1 > prepared part key=_multipart_U3EID5EK64HD.2~3X7J6HH77MNB.1 size=1048576 2024-04-07T16:37:42.421 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.429+0000 7f346a3868c0 1 > prepared part key=_multipart_U3EID5EK64HD.2~3X7J6HH77MNB.2 size=1012009 2024-04-07T16:37:42.421 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.429+0000 7f346a3868c0 1 > prepared multipart upload key=U3EID5EK64HD upload=3X7J6HH77MNB tag=H503K422V4DV type=write size=4157737 2024-04-07T16:37:42.424 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.433+0000 7f346a3868c0 1 < completed delete operation key=1VBSUWNI3V1T tag=3LHLGWE6UIL4 type=del 2024-04-07T16:37:42.429 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.437+0000 7f346a3868c0 1 > prepared operation key=U3EID5EK64HD tag=3YIIDZW5S3P1 type=del size=1464728 2024-04-07T16:37:42.433 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.441+0000 7f346a3868c0 1 < completed write operation key=QVOKSQAGHPA9 tag=5CL8VH9OH9JQ type=write size=639281 2024-04-07T16:37:42.436 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.445+0000 7f346a3868c0 1 > prepared operation key=8F81G0P0YBFW tag=46ERQ2EY7JHW type=write size=650851 2024-04-07T16:37:42.440 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.449+0000 7f346a3868c0 1 < completed delete operation key=GBQSME7SY1AH tag=US4M9F0HP42A type=del 2024-04-07T16:37:42.446 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.453+0000 7f346a3868c0 1 > prepared part key=_multipart_1WGUPH7X5UA0.2~HGTTVV3UVT4V.0 size=11336 2024-04-07T16:37:42.447 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.453+0000 7f346a3868c0 1 > prepared multipart upload key=1WGUPH7X5UA0 upload=HGTTVV3UVT4V tag=Z93D281JEDBW type=write size=1059912 2024-04-07T16:37:42.450 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.457+0000 7f346a3868c0 1 < completed delete operation key=EQR0FY86AT5X tag=JAMGI4PYXKUS type=del 2024-04-07T16:37:42.454 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.461+0000 7f346a3868c0 1 > prepared operation key=YOK8BAEOMPUR tag=ZGVJAEP8P64Y type=write size=430381 2024-04-07T16:37:42.468 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.477+0000 7f346a3868c0 1 < suggested remove operation key=3N0QJRYVO3FV tag=P6IESFD7POX5 type=del 2024-04-07T16:37:42.478 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.485+0000 7f346a3868c0 1 > prepared operation key=YXMDZB23YTB3 tag=RBG5W22SKG9S type=del size=2235046 2024-04-07T16:37:42.482 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.489+0000 7f346a3868c0 1 < completed part key=_multipart_GZXHYZO5SMUP.2~Y0MWRK0SRI2R.0 size=1048576 2024-04-07T16:37:42.485 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.493+0000 7f346a3868c0 1 < completed part key=_multipart_GZXHYZO5SMUP.2~Y0MWRK0SRI2R.1 size=702967 2024-04-07T16:37:42.491 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.497+0000 7f346a3868c0 1 < completed multipart upload key=GZXHYZO5SMUP upload=Y0MWRK0SRI2R tag=174OPCXA1UI2 type=write size=2800119 2024-04-07T16:37:42.494 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.501+0000 7f346a3868c0 1 > prepared operation key=NYET3212USVE tag=WUJJRLY3ZY9I type=write size=651683 2024-04-07T16:37:42.500 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.509+0000 7f346a3868c0 1 < suggested remove operation key=95YVKJ14JD8W tag=GGL3GHXG2LMM type=write 2024-04-07T16:37:42.506 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.513+0000 7f346a3868c0 1 > prepared part key=_multipart_C1BTVJNHIM0X.2~WYQQ8U8B8HOD.0 size=1048576 2024-04-07T16:37:42.509 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.517+0000 7f346a3868c0 1 > prepared part key=_multipart_C1BTVJNHIM0X.2~WYQQ8U8B8HOD.1 size=1048576 2024-04-07T16:37:42.511 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.521+0000 7f346a3868c0 1 > prepared part key=_multipart_C1BTVJNHIM0X.2~WYQQ8U8B8HOD.2 size=1037956 2024-04-07T16:37:42.511 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.521+0000 7f346a3868c0 1 > prepared multipart upload key=C1BTVJNHIM0X upload=WYQQ8U8B8HOD tag=WE620ETZ1HNM type=write size=4183684 2024-04-07T16:37:42.514 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.521+0000 7f346a3868c0 1 < completed part key=_multipart_1WGUPH7X5UA0.2~SM4IMSBHWX80.0 size=570736 2024-04-07T16:37:42.519 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.529+0000 7f346a3868c0 1 < completed multipart upload key=1WGUPH7X5UA0 upload=SM4IMSBHWX80 tag=2OT6IF6MDMMA type=write size=1619312 2024-04-07T16:37:42.522 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.529+0000 7f346a3868c0 1 > prepared operation key=1VBSUWNI3V1T tag=UX4RF2GMATUS type=del size=1154458 2024-04-07T16:37:42.525 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.533+0000 7f346a3868c0 1 < completed part key=_multipart_7G5C6HBK6QWI.2~A6FF0B31XRHK.0 size=1048576 2024-04-07T16:37:42.528 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.537+0000 7f346a3868c0 1 < completed part key=_multipart_7G5C6HBK6QWI.2~A6FF0B31XRHK.1 size=1048576 2024-04-07T16:37:42.528 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.537+0000 7f346a3868c0 1 < canceled part key=_multipart_7G5C6HBK6QWI.2~A6FF0B31XRHK.2 size=293873 2024-04-07T16:37:42.533 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.541+0000 7f346a3868c0 1 < canceled multipart upload key=7G5C6HBK6QWI upload=A6FF0B31XRHK tag=38WXCE5Z1X7Y type=write size=3439601 2024-04-07T16:37:42.541 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.549+0000 7f346a3868c0 1 > prepared part key=_multipart_8F81G0P0YBFW.2~BEOXI2HTLCG3.0 size=562159 2024-04-07T16:37:42.541 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.549+0000 7f346a3868c0 1 > prepared multipart upload key=8F81G0P0YBFW upload=BEOXI2HTLCG3 tag=R1U7V6YWYJH4 type=write size=1610735 2024-04-07T16:37:42.544 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.553+0000 7f346a3868c0 1 < completed delete operation key=QTCIIPTXIRW2 tag=919S1ZOE5PYS type=del 2024-04-07T16:37:42.548 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.557+0000 7f346a3868c0 1 > prepared operation key=IX3F0IUOKBX1 tag=EQ1DZH0D5YC1 type=del size=2155413 2024-04-07T16:37:42.551 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.561+0000 7f346a3868c0 -1 < failed to complete operation key=3N0QJRYVO3FV tag=076SE3VDBSSW type=del size=2800766: (22) Invalid argument 2024-04-07T16:37:42.558 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.565+0000 7f346a3868c0 1 > prepared part key=_multipart_7G5C6HBK6QWI.2~NY591A62LNZL.0 size=478032 2024-04-07T16:37:42.558 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.565+0000 7f346a3868c0 1 > prepared multipart upload key=7G5C6HBK6QWI upload=NY591A62LNZL tag=1XMOB45DNKU7 type=write size=1526608 2024-04-07T16:37:42.561 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.569+0000 7f346a3868c0 1 < completed part key=_multipart_1WGUPH7X5UA0.2~0G2NBJYTANBT.0 size=259506 2024-04-07T16:37:42.566 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.573+0000 7f346a3868c0 1 < completed multipart upload key=1WGUPH7X5UA0 upload=0G2NBJYTANBT tag=UCEHOS99JOT8 type=write size=1308082 2024-04-07T16:37:42.569 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.577+0000 7f346a3868c0 1 > prepared operation key=E84U0SX44NQ8 tag=UTK77U2BBBWZ type=del size=2329815 2024-04-07T16:37:42.572 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.581+0000 7f346a3868c0 1 < completed part key=_multipart_8F81G0P0YBFW.2~150KSM7J33JG.0 size=829225 2024-04-07T16:37:42.577 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.585+0000 7f346a3868c0 1 < completed multipart upload key=8F81G0P0YBFW upload=150KSM7J33JG tag=HHS565LHFO7K type=write size=1877801 2024-04-07T16:37:42.583 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.593+0000 7f346a3868c0 1 > prepared part key=_multipart_1VBSUWNI3V1T.2~K0KYWCCLNDT5.0 size=557546 2024-04-07T16:37:42.583 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.593+0000 7f346a3868c0 1 > prepared multipart upload key=1VBSUWNI3V1T upload=K0KYWCCLNDT5 tag=NXPCIX48WX0E type=write size=1606122 2024-04-07T16:37:42.587 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.593+0000 7f346a3868c0 1 < completed part key=_multipart_DXTO3AJ429AC.2~IWCZV5HBMK0J.0 size=1048576 2024-04-07T16:37:42.589 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.597+0000 7f346a3868c0 1 < completed part key=_multipart_DXTO3AJ429AC.2~IWCZV5HBMK0J.1 size=1048576 2024-04-07T16:37:42.592 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.601+0000 7f346a3868c0 1 < completed part key=_multipart_DXTO3AJ429AC.2~IWCZV5HBMK0J.2 size=425083 2024-04-07T16:37:42.597 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.605+0000 7f346a3868c0 1 < completed multipart upload key=DXTO3AJ429AC upload=IWCZV5HBMK0J tag=VV7Q16C4E0PP type=write size=3570811 2024-04-07T16:37:42.600 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.609+0000 7f346a3868c0 1 > prepared operation key=XJLQZ7IS9U1T tag=1XHQ2SL4I01N type=del size=1231541 2024-04-07T16:37:42.606 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:37:42 smithi144 bash[95233]: ts=2024-04-07T16:37:42.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:37:42.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:37:42 smithi144 bash[66654]: cluster 2024-04-07T16:37:41.054329+0000 mgr.y (mgr.25175) 670 : cluster [DBG] pgmap v1303: 137 pgs: 32 unknown, 105 active+clean; 583 KiB data, 2.8 GiB used, 712 GiB / 715 GiB avail 2024-04-07T16:37:42.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:37:42 smithi144 bash[66654]: cluster 2024-04-07T16:37:41.460935+0000 mon.a (mon.0) 2014 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T16:37:42.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:37:42 smithi144 bash[66654]: audit 2024-04-07T16:37:41.560558+0000 mon.a (mon.0) 2015 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:37:42.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:37:42 smithi144 bash[66654]: audit 2024-04-07T16:37:41.561711+0000 mon.a (mon.0) 2016 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:37:42.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:37:42 smithi144 bash[66654]: audit 2024-04-07T16:37:41.567674+0000 mon.a (mon.0) 2017 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:37:42.607 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.613+0000 7f346a3868c0 1 < suggested remove operation key=DXTO3AJ429AC tag=W8TRZZOX5HV6 type=write 2024-04-07T16:37:42.610 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.617+0000 7f346a3868c0 1 > prepared operation key=EOYFVEI50SQ1 tag=MMERWN8963CU type=write size=367061 2024-04-07T16:37:42.613 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.621+0000 7f346a3868c0 1 < completed part key=_multipart_U3EID5EK64HD.2~3X7J6HH77MNB.0 size=1048576 2024-04-07T16:37:42.616 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.625+0000 7f346a3868c0 1 < completed part key=_multipart_U3EID5EK64HD.2~3X7J6HH77MNB.1 size=1048576 2024-04-07T16:37:42.618 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.625+0000 7f346a3868c0 1 < completed part key=_multipart_U3EID5EK64HD.2~3X7J6HH77MNB.2 size=1012009 2024-04-07T16:37:42.624 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.633+0000 7f346a3868c0 1 < completed multipart upload key=U3EID5EK64HD upload=3X7J6HH77MNB tag=H503K422V4DV type=write size=4157737 2024-04-07T16:37:42.629 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.637+0000 7f346a3868c0 1 > prepared part key=_multipart_N14BUXLJONHZ.2~U8SX1SDWGY04.0 size=524038 2024-04-07T16:37:42.629 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.637+0000 7f346a3868c0 1 > prepared multipart upload key=N14BUXLJONHZ upload=U8SX1SDWGY04 tag=K63EYGDHUVFJ type=write size=1572614 2024-04-07T16:37:42.633 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.641+0000 7f346a3868c0 1 < completed delete operation key=U3EID5EK64HD tag=3YIIDZW5S3P1 type=del 2024-04-07T16:37:42.636 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.645+0000 7f346a3868c0 1 > prepared operation key=NYET3212USVE tag=QB4QMG0NX570 type=write size=581470 2024-04-07T16:37:42.640 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.649+0000 7f346a3868c0 1 < completed write operation key=8F81G0P0YBFW tag=46ERQ2EY7JHW type=write size=650851 2024-04-07T16:37:42.646 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.653+0000 7f346a3868c0 1 > prepared part key=_multipart_HBLEXONKXJTL.2~HT1ZG3AH8UI6.0 size=1048576 2024-04-07T16:37:42.649 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.657+0000 7f346a3868c0 1 > prepared part key=_multipart_HBLEXONKXJTL.2~HT1ZG3AH8UI6.1 size=1048576 2024-04-07T16:37:42.652 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.661+0000 7f346a3868c0 1 > prepared part key=_multipart_HBLEXONKXJTL.2~HT1ZG3AH8UI6.2 size=406288 2024-04-07T16:37:42.652 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.661+0000 7f346a3868c0 1 > prepared multipart upload key=HBLEXONKXJTL upload=HT1ZG3AH8UI6 tag=TCTUQ8GL5JXT type=write size=3552016 2024-04-07T16:37:42.656 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.665+0000 7f346a3868c0 1 < completed part key=_multipart_1WGUPH7X5UA0.2~HGTTVV3UVT4V.0 size=11336 2024-04-07T16:37:42.661 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.669+0000 7f346a3868c0 1 < completed multipart upload key=1WGUPH7X5UA0 upload=HGTTVV3UVT4V tag=Z93D281JEDBW type=write size=1059912 2024-04-07T16:37:42.664 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.673+0000 7f346a3868c0 1 > prepared operation key=EQR0FY86AT5X tag=7LGT9GNDH4IK type=del size=1388642 2024-04-07T16:37:42.668 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.677+0000 7f346a3868c0 1 < completed write operation key=YOK8BAEOMPUR tag=ZGVJAEP8P64Y type=write size=430381 2024-04-07T16:37:42.674 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.681+0000 7f346a3868c0 1 > prepared part key=_multipart_GZXHYZO5SMUP.2~TDAEGJY5BUHI.0 size=1048576 2024-04-07T16:37:42.677 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.685+0000 7f346a3868c0 1 > prepared part key=_multipart_GZXHYZO5SMUP.2~TDAEGJY5BUHI.1 size=1048576 2024-04-07T16:37:42.680 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.689+0000 7f346a3868c0 1 > prepared part key=_multipart_GZXHYZO5SMUP.2~TDAEGJY5BUHI.2 size=629408 2024-04-07T16:37:42.680 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.689+0000 7f346a3868c0 1 > prepared multipart upload key=GZXHYZO5SMUP upload=TDAEGJY5BUHI tag=DI243387M3VB type=write size=3775136 2024-04-07T16:37:42.684 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.693+0000 7f346a3868c0 1 < completed delete operation key=YXMDZB23YTB3 tag=RBG5W22SKG9S type=del 2024-04-07T16:37:42.688 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.697+0000 7f346a3868c0 1 > prepared operation key=GZXHYZO5SMUP tag=446QDFPKTYDD type=del size=1006895 2024-04-07T16:37:42.692 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.701+0000 7f346a3868c0 1 < completed write operation key=NYET3212USVE tag=WUJJRLY3ZY9I type=write size=651683 2024-04-07T16:37:42.697 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.705+0000 7f346a3868c0 1 > prepared part key=_multipart_1VKVQXSQMTU1.2~LNQIP930KAP8.0 size=1048576 2024-04-07T16:37:42.700 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.709+0000 7f346a3868c0 1 > prepared part key=_multipart_1VKVQXSQMTU1.2~LNQIP930KAP8.1 size=1048576 2024-04-07T16:37:42.702 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.709+0000 7f346a3868c0 1 > prepared part key=_multipart_1VKVQXSQMTU1.2~LNQIP930KAP8.2 size=905655 2024-04-07T16:37:42.702 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.709+0000 7f346a3868c0 1 > prepared multipart upload key=1VKVQXSQMTU1 upload=LNQIP930KAP8 tag=D3YDIBKYPNB0 type=write size=4051383 2024-04-07T16:37:42.706 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.713+0000 7f346a3868c0 1 < completed part key=_multipart_C1BTVJNHIM0X.2~WYQQ8U8B8HOD.0 size=1048576 2024-04-07T16:37:42.708 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.717+0000 7f346a3868c0 1 < completed part key=_multipart_C1BTVJNHIM0X.2~WYQQ8U8B8HOD.1 size=1048576 2024-04-07T16:37:42.711 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.721+0000 7f346a3868c0 1 < completed part key=_multipart_C1BTVJNHIM0X.2~WYQQ8U8B8HOD.2 size=1037956 2024-04-07T16:37:42.716 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.725+0000 7f346a3868c0 1 < completed multipart upload key=C1BTVJNHIM0X upload=WYQQ8U8B8HOD tag=WE620ETZ1HNM type=write size=4183684 2024-04-07T16:37:42.719 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.729+0000 7f346a3868c0 1 > prepared operation key=1VKVQXSQMTU1 tag=B26ZV5XBEHHF type=del size=1560554 2024-04-07T16:37:42.723 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.733+0000 7f346a3868c0 1 < canceled operation key=1VBSUWNI3V1T tag=UX4RF2GMATUS type=del size=1154458 2024-04-07T16:37:42.729 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.737+0000 7f346a3868c0 1 > prepared part key=_multipart_LXOZ0T2KY6OR.2~E8K94UT63N8G.0 size=472659 2024-04-07T16:37:42.729 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.737+0000 7f346a3868c0 1 > prepared multipart upload key=LXOZ0T2KY6OR upload=E8K94UT63N8G tag=W7ES64JKKP32 type=write size=1521235 2024-04-07T16:37:42.733 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.741+0000 7f346a3868c0 1 < completed part key=_multipart_8F81G0P0YBFW.2~BEOXI2HTLCG3.0 size=562159 2024-04-07T16:37:42.739 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.745+0000 7f346a3868c0 1 < completed multipart upload key=8F81G0P0YBFW upload=BEOXI2HTLCG3 tag=R1U7V6YWYJH4 type=write size=1610735 2024-04-07T16:37:42.741 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.749+0000 7f346a3868c0 1 > prepared operation key=GZXHYZO5SMUP tag=HGE7QTGYZ909 type=write size=696255 2024-04-07T16:37:42.745 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.753+0000 7f346a3868c0 1 < completed delete operation key=IX3F0IUOKBX1 tag=EQ1DZH0D5YC1 type=del 2024-04-07T16:37:42.749 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.757+0000 7f346a3868c0 1 > prepared operation key=IX3F0IUOKBX1 tag=IZ0EWB0RWQ6T type=del size=2634557 2024-04-07T16:37:42.752 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.761+0000 7f346a3868c0 1 < completed part key=_multipart_7G5C6HBK6QWI.2~NY591A62LNZL.0 size=478032 2024-04-07T16:37:42.757 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.765+0000 7f346a3868c0 1 < completed multipart upload key=7G5C6HBK6QWI upload=NY591A62LNZL tag=1XMOB45DNKU7 type=write size=1526608 2024-04-07T16:37:42.763 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.773+0000 7f346a3868c0 1 > prepared part key=_multipart_XJLQZ7IS9U1T.2~CL1OS8A8FSMZ.0 size=1048576 2024-04-07T16:37:42.766 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.773+0000 7f346a3868c0 1 > prepared part key=_multipart_XJLQZ7IS9U1T.2~CL1OS8A8FSMZ.1 size=1048576 2024-04-07T16:37:42.768 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.777+0000 7f346a3868c0 1 > prepared part key=_multipart_XJLQZ7IS9U1T.2~CL1OS8A8FSMZ.2 size=979767 2024-04-07T16:37:42.768 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.777+0000 7f346a3868c0 1 > prepared multipart upload key=XJLQZ7IS9U1T upload=CL1OS8A8FSMZ tag=4G43IVH23LJ6 type=write size=4125495 2024-04-07T16:37:42.772 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.781+0000 7f346a3868c0 1 < completed delete operation key=E84U0SX44NQ8 tag=UTK77U2BBBWZ type=del 2024-04-07T16:37:42.778 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.785+0000 7f346a3868c0 1 > prepared part key=_multipart_XJLQZ7IS9U1T.2~IXN6U05CWGDX.0 size=1048576 2024-04-07T16:37:42.781 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.789+0000 7f346a3868c0 1 > prepared part key=_multipart_XJLQZ7IS9U1T.2~IXN6U05CWGDX.1 size=1048576 2024-04-07T16:37:42.783 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.793+0000 7f346a3868c0 1 > prepared part key=_multipart_XJLQZ7IS9U1T.2~IXN6U05CWGDX.2 size=558407 2024-04-07T16:37:42.783 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.793+0000 7f346a3868c0 1 > prepared multipart upload key=XJLQZ7IS9U1T upload=IXN6U05CWGDX tag=4KO0ZEMULG9C type=write size=3704135 2024-04-07T16:37:42.790 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.797+0000 7f346a3868c0 1 < completed part key=_multipart_1VBSUWNI3V1T.2~K0KYWCCLNDT5.0 size=557546 2024-04-07T16:37:42.794 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.801+0000 7f346a3868c0 1 < completed multipart upload key=1VBSUWNI3V1T upload=K0KYWCCLNDT5 tag=NXPCIX48WX0E type=write size=1606122 2024-04-07T16:37:42.798 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.805+0000 7f346a3868c0 1 > prepared operation key=8F81G0P0YBFW tag=T14CY2U76R64 type=write size=749284 2024-04-07T16:37:42.802 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.809+0000 7f346a3868c0 1 < completed delete operation key=XJLQZ7IS9U1T tag=1XHQ2SL4I01N type=del 2024-04-07T16:37:42.808 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.817+0000 7f346a3868c0 1 > prepared part key=_multipart_8F81G0P0YBFW.2~O8JRB2MGDD3R.0 size=1048576 2024-04-07T16:37:42.811 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.821+0000 7f346a3868c0 1 > prepared part key=_multipart_8F81G0P0YBFW.2~O8JRB2MGDD3R.1 size=550088 2024-04-07T16:37:42.811 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.821+0000 7f346a3868c0 1 > prepared multipart upload key=8F81G0P0YBFW upload=O8JRB2MGDD3R tag=1O6CHEPRK59M type=write size=2647240 2024-04-07T16:37:42.815 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.821+0000 7f346a3868c0 1 < canceled operation key=EOYFVEI50SQ1 tag=MMERWN8963CU type=write size=367061 2024-04-07T16:37:42.821 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.829+0000 7f346a3868c0 1 > prepared part key=_multipart_GBQSME7SY1AH.2~9GX7GUG6O638.0 size=1048576 2024-04-07T16:37:42.824 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.833+0000 7f346a3868c0 1 > prepared part key=_multipart_GBQSME7SY1AH.2~9GX7GUG6O638.1 size=828995 2024-04-07T16:37:42.824 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.833+0000 7f346a3868c0 1 > prepared multipart upload key=GBQSME7SY1AH upload=9GX7GUG6O638 tag=P2LTJ1CDON44 type=write size=2926147 2024-04-07T16:37:42.828 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.837+0000 7f346a3868c0 1 < completed part key=_multipart_N14BUXLJONHZ.2~U8SX1SDWGY04.0 size=524038 2024-04-07T16:37:42.835 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.841+0000 7f346a3868c0 1 < completed multipart upload key=N14BUXLJONHZ upload=U8SX1SDWGY04 tag=K63EYGDHUVFJ type=write size=1572614 2024-04-07T16:37:42.842 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.849+0000 7f346a3868c0 1 > prepared part key=_multipart_IX3F0IUOKBX1.2~674RSWC6QOBW.0 size=151122 2024-04-07T16:37:42.842 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.849+0000 7f346a3868c0 1 > prepared multipart upload key=IX3F0IUOKBX1 upload=674RSWC6QOBW tag=WC0Y0ODTZY29 type=write size=1199698 2024-04-07T16:37:42.847 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.853+0000 7f346a3868c0 1 < completed write operation key=NYET3212USVE tag=QB4QMG0NX570 type=write size=581470 2024-04-07T16:37:42.851 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.857+0000 7f346a3868c0 1 > prepared operation key=HBLEXONKXJTL tag=V1KB9ZY02EGZ type=del size=659065 2024-04-07T16:37:42.855 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.865+0000 7f346a3868c0 1 < completed part key=_multipart_HBLEXONKXJTL.2~HT1ZG3AH8UI6.0 size=1048576 2024-04-07T16:37:42.858 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.865+0000 7f346a3868c0 1 < completed part key=_multipart_HBLEXONKXJTL.2~HT1ZG3AH8UI6.1 size=1048576 2024-04-07T16:37:42.858 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.865+0000 7f346a3868c0 1 < canceled part key=_multipart_HBLEXONKXJTL.2~HT1ZG3AH8UI6.2 size=406288 2024-04-07T16:37:42.864 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.873+0000 7f346a3868c0 1 < canceled multipart upload key=HBLEXONKXJTL upload=HT1ZG3AH8UI6 tag=TCTUQ8GL5JXT type=write size=3552016 2024-04-07T16:37:42.867 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.877+0000 7f346a3868c0 1 > prepared operation key=IP8YW2A75TCW tag=LEENHRZV1EZB type=write size=273282 2024-04-07T16:37:42.871 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.877+0000 7f346a3868c0 1 < completed delete operation key=EQR0FY86AT5X tag=7LGT9GNDH4IK type=del 2024-04-07T16:37:42.874 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.881+0000 7f346a3868c0 1 > prepared operation key=QVOKSQAGHPA9 tag=Z6R3Q0HKYOZ4 type=del size=2569846 2024-04-07T16:37:42.878 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.885+0000 7f346a3868c0 1 < completed part key=_multipart_GZXHYZO5SMUP.2~TDAEGJY5BUHI.0 size=1048576 2024-04-07T16:37:42.881 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.889+0000 7f346a3868c0 1 < completed part key=_multipart_GZXHYZO5SMUP.2~TDAEGJY5BUHI.1 size=1048576 2024-04-07T16:37:42.884 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.893+0000 7f346a3868c0 1 < completed part key=_multipart_GZXHYZO5SMUP.2~TDAEGJY5BUHI.2 size=629408 2024-04-07T16:37:42.889 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.897+0000 7f346a3868c0 1 < completed multipart upload key=GZXHYZO5SMUP upload=TDAEGJY5BUHI tag=DI243387M3VB type=write size=3775136 2024-04-07T16:37:42.896 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.905+0000 7f346a3868c0 1 > prepared part key=_multipart_IP8YW2A75TCW.2~09IFBRPHDWPU.0 size=1048576 2024-04-07T16:37:42.899 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.905+0000 7f346a3868c0 1 > prepared part key=_multipart_IP8YW2A75TCW.2~09IFBRPHDWPU.1 size=367512 2024-04-07T16:37:42.899 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.905+0000 7f346a3868c0 1 > prepared multipart upload key=IP8YW2A75TCW upload=09IFBRPHDWPU tag=7RZTJYR7P1ZL type=write size=2464664 2024-04-07T16:37:42.902 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.909+0000 7f346a3868c0 1 < completed delete operation key=GZXHYZO5SMUP tag=446QDFPKTYDD type=del 2024-04-07T16:37:42.906 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.913+0000 7f346a3868c0 1 > prepared operation key=IP8YW2A75TCW tag=QNMHWJ8M7392 type=del size=2002425 2024-04-07T16:37:42.909 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.917+0000 7f346a3868c0 1 < completed part key=_multipart_1VKVQXSQMTU1.2~LNQIP930KAP8.0 size=1048576 2024-04-07T16:37:42.912 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.921+0000 7f346a3868c0 1 < completed part key=_multipart_1VKVQXSQMTU1.2~LNQIP930KAP8.1 size=1048576 2024-04-07T16:37:42.915 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.925+0000 7f346a3868c0 1 < completed part key=_multipart_1VKVQXSQMTU1.2~LNQIP930KAP8.2 size=905655 2024-04-07T16:37:42.920 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.929+0000 7f346a3868c0 1 < completed multipart upload key=1VKVQXSQMTU1 upload=LNQIP930KAP8 tag=D3YDIBKYPNB0 type=write size=4051383 2024-04-07T16:37:42.924 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.933+0000 7f346a3868c0 1 > prepared operation key=QVOKSQAGHPA9 tag=OWZB2K15NDJH type=del size=254831 2024-04-07T16:37:42.931 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.941+0000 7f346a3868c0 1 < suggested update operation key=1VKVQXSQMTU1 tag=B26ZV5XBEHHF type=del size=1560554 2024-04-07T16:37:42.937 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.945+0000 7f346a3868c0 1 > prepared part key=_multipart_3N0QJRYVO3FV.2~M30WLYE3HD85.0 size=215728 2024-04-07T16:37:42.938 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.945+0000 7f346a3868c0 1 > prepared multipart upload key=3N0QJRYVO3FV upload=M30WLYE3HD85 tag=BACBBXHWW3ST type=write size=1264304 2024-04-07T16:37:42.941 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:37:42 smithi100 bash[74942]: cluster 2024-04-07T16:37:41.054329+0000 mgr.y (mgr.25175) 670 : cluster [DBG] pgmap v1303: 137 pgs: 32 unknown, 105 active+clean; 583 KiB data, 2.8 GiB used, 712 GiB / 715 GiB avail 2024-04-07T16:37:42.941 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:37:42 smithi100 bash[74942]: cluster 2024-04-07T16:37:41.460935+0000 mon.a (mon.0) 2014 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T16:37:42.941 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:37:42 smithi100 bash[74942]: audit 2024-04-07T16:37:41.560558+0000 mon.a (mon.0) 2015 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:37:42.941 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:37:42 smithi100 bash[74942]: audit 2024-04-07T16:37:41.561711+0000 mon.a (mon.0) 2016 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:37:42.941 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:37:42 smithi100 bash[74942]: audit 2024-04-07T16:37:41.567674+0000 mon.a (mon.0) 2017 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:37:42.941 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:37:42 smithi100 bash[76202]: cluster 2024-04-07T16:37:41.054329+0000 mgr.y (mgr.25175) 670 : cluster [DBG] pgmap v1303: 137 pgs: 32 unknown, 105 active+clean; 583 KiB data, 2.8 GiB used, 712 GiB / 715 GiB avail 2024-04-07T16:37:42.942 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:37:42 smithi100 bash[76202]: cluster 2024-04-07T16:37:41.460935+0000 mon.a (mon.0) 2014 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T16:37:42.942 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:37:42 smithi100 bash[76202]: audit 2024-04-07T16:37:41.560558+0000 mon.a (mon.0) 2015 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:37:42.942 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:37:42 smithi100 bash[76202]: audit 2024-04-07T16:37:41.561711+0000 mon.a (mon.0) 2016 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:37:42.942 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:37:42 smithi100 bash[76202]: audit 2024-04-07T16:37:41.567674+0000 mon.a (mon.0) 2017 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:37:42.942 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.949+0000 7f346a3868c0 1 < completed part key=_multipart_LXOZ0T2KY6OR.2~E8K94UT63N8G.0 size=472659 2024-04-07T16:37:42.947 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.957+0000 7f346a3868c0 1 < completed multipart upload key=LXOZ0T2KY6OR upload=E8K94UT63N8G tag=W7ES64JKKP32 type=write size=1521235 2024-04-07T16:37:42.954 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.961+0000 7f346a3868c0 1 > prepared part key=_multipart_X5Q46GO8YTOK.2~KTDO5K307DO1.0 size=1048576 2024-04-07T16:37:42.957 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.965+0000 7f346a3868c0 1 > prepared part key=_multipart_X5Q46GO8YTOK.2~KTDO5K307DO1.1 size=1048576 2024-04-07T16:37:42.960 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.969+0000 7f346a3868c0 1 > prepared part key=_multipart_X5Q46GO8YTOK.2~KTDO5K307DO1.2 size=146014 2024-04-07T16:37:42.960 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.969+0000 7f346a3868c0 1 > prepared multipart upload key=X5Q46GO8YTOK upload=KTDO5K307DO1 tag=I9FTBFX1TJF3 type=write size=3291742 2024-04-07T16:37:42.964 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.973+0000 7f346a3868c0 1 < completed write operation key=GZXHYZO5SMUP tag=HGE7QTGYZ909 type=write size=696255 2024-04-07T16:37:42.968 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.977+0000 7f346a3868c0 1 > prepared operation key=CL4PP1RKZU0E tag=K571UPRPFKCT type=del size=2414466 2024-04-07T16:37:42.975 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.985+0000 7f346a3868c0 1 < suggested update operation key=IX3F0IUOKBX1 tag=IZ0EWB0RWQ6T type=del size=2634557 2024-04-07T16:37:42.979 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.989+0000 7f346a3868c0 1 > prepared operation key=XJLQZ7IS9U1T tag=C7MS9BF4KQFB type=write size=385675 2024-04-07T16:37:42.983 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.989+0000 7f346a3868c0 1 < completed part key=_multipart_XJLQZ7IS9U1T.2~CL1OS8A8FSMZ.0 size=1048576 2024-04-07T16:37:42.986 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.993+0000 7f346a3868c0 1 < completed part key=_multipart_XJLQZ7IS9U1T.2~CL1OS8A8FSMZ.1 size=1048576 2024-04-07T16:37:42.989 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:42.997+0000 7f346a3868c0 1 < completed part key=_multipart_XJLQZ7IS9U1T.2~CL1OS8A8FSMZ.2 size=979767 2024-04-07T16:37:42.995 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.005+0000 7f346a3868c0 1 < completed multipart upload key=XJLQZ7IS9U1T upload=CL1OS8A8FSMZ tag=4G43IVH23LJ6 type=write size=4125495 2024-04-07T16:37:42.999 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.009+0000 7f346a3868c0 1 > prepared operation key=ROFH30W3UCAX tag=2B11O3VCT6GH type=del size=1489869 2024-04-07T16:37:43.003 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.013+0000 7f346a3868c0 1 < completed part key=_multipart_XJLQZ7IS9U1T.2~IXN6U05CWGDX.0 size=1048576 2024-04-07T16:37:43.006 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.013+0000 7f346a3868c0 1 < completed part key=_multipart_XJLQZ7IS9U1T.2~IXN6U05CWGDX.1 size=1048576 2024-04-07T16:37:43.006 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.013+0000 7f346a3868c0 1 < canceled part key=_multipart_XJLQZ7IS9U1T.2~IXN6U05CWGDX.2 size=558407 2024-04-07T16:37:43.014 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.021+0000 7f346a3868c0 1 < canceled multipart upload key=XJLQZ7IS9U1T upload=IXN6U05CWGDX tag=4KO0ZEMULG9C type=write size=3704135 2024-04-07T16:37:43.018 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.025+0000 7f346a3868c0 1 > prepared operation key=GBQSME7SY1AH tag=G8ZY5KP6CJI6 type=del size=2227087 2024-04-07T16:37:43.021 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.029+0000 7f346a3868c0 1 < completed write operation key=8F81G0P0YBFW tag=T14CY2U76R64 type=write size=749284 2024-04-07T16:37:43.025 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.033+0000 7f346a3868c0 1 > prepared operation key=U3EID5EK64HD tag=HCSMOSETOXQ8 type=del size=3140174 2024-04-07T16:37:43.030 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.037+0000 7f346a3868c0 1 < completed part key=_multipart_8F81G0P0YBFW.2~O8JRB2MGDD3R.0 size=1048576 2024-04-07T16:37:43.033 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.041+0000 7f346a3868c0 1 < completed part key=_multipart_8F81G0P0YBFW.2~O8JRB2MGDD3R.1 size=550088 2024-04-07T16:37:43.041 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.049+0000 7f346a3868c0 1 < completed multipart upload key=8F81G0P0YBFW upload=O8JRB2MGDD3R tag=1O6CHEPRK59M type=write size=2647240 2024-04-07T16:37:43.044 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.053+0000 7f346a3868c0 1 > prepared operation key=GZXHYZO5SMUP tag=ETKRHV6LW7FW type=write size=722745 2024-04-07T16:37:43.048 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.057+0000 7f346a3868c0 1 < completed part key=_multipart_GBQSME7SY1AH.2~9GX7GUG6O638.0 size=1048576 2024-04-07T16:37:43.051 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.057+0000 7f346a3868c0 1 < completed part key=_multipart_GBQSME7SY1AH.2~9GX7GUG6O638.1 size=828995 2024-04-07T16:37:43.056 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.065+0000 7f346a3868c0 1 < completed multipart upload key=GBQSME7SY1AH upload=9GX7GUG6O638 tag=P2LTJ1CDON44 type=write size=2926147 2024-04-07T16:37:43.060 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.069+0000 7f346a3868c0 1 > prepared operation key=DXTO3AJ429AC tag=6WNC2V9OT59V type=del size=1194820 2024-04-07T16:37:43.063 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.073+0000 7f346a3868c0 1 < completed part key=_multipart_IX3F0IUOKBX1.2~674RSWC6QOBW.0 size=151122 2024-04-07T16:37:43.069 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.077+0000 7f346a3868c0 1 < completed multipart upload key=IX3F0IUOKBX1 upload=674RSWC6QOBW tag=WC0Y0ODTZY29 type=write size=1199698 2024-04-07T16:37:43.075 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.085+0000 7f346a3868c0 1 > prepared part key=_multipart_NYET3212USVE.2~3D7GTR2YG8QS.0 size=1048576 2024-04-07T16:37:43.078 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.085+0000 7f346a3868c0 1 > prepared part key=_multipart_NYET3212USVE.2~3D7GTR2YG8QS.1 size=667495 2024-04-07T16:37:43.078 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.085+0000 7f346a3868c0 1 > prepared multipart upload key=NYET3212USVE upload=3D7GTR2YG8QS tag=JMIM4HKSV55A type=write size=2764647 2024-04-07T16:37:43.081 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.089+0000 7f346a3868c0 1 < completed delete operation key=HBLEXONKXJTL tag=V1KB9ZY02EGZ type=del 2024-04-07T16:37:43.085 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.093+0000 7f346a3868c0 1 > prepared operation key=YOK8BAEOMPUR tag=3MGP75DUWTKG type=del size=2589871 2024-04-07T16:37:43.089 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.097+0000 7f346a3868c0 1 < completed write operation key=IP8YW2A75TCW tag=LEENHRZV1EZB type=write size=273282 2024-04-07T16:37:43.092 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.101+0000 7f346a3868c0 1 > prepared operation key=UIW3DN4NUYPP tag=ITLE1I2RHM77 type=del size=3801357 2024-04-07T16:37:43.096 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.105+0000 7f346a3868c0 1 < completed delete operation key=QVOKSQAGHPA9 tag=Z6R3Q0HKYOZ4 type=del 2024-04-07T16:37:43.100 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.109+0000 7f346a3868c0 1 > prepared operation key=CL4PP1RKZU0E tag=0OS4M5CEP8V5 type=del size=2799548 2024-04-07T16:37:43.104 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.113+0000 7f346a3868c0 1 < completed part key=_multipart_IP8YW2A75TCW.2~09IFBRPHDWPU.0 size=1048576 2024-04-07T16:37:43.107 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.113+0000 7f346a3868c0 1 < completed part key=_multipart_IP8YW2A75TCW.2~09IFBRPHDWPU.1 size=367512 2024-04-07T16:37:43.113 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.121+0000 7f346a3868c0 1 < completed multipart upload key=IP8YW2A75TCW upload=09IFBRPHDWPU tag=7RZTJYR7P1ZL type=write size=2464664 2024-04-07T16:37:43.116 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.125+0000 7f346a3868c0 1 > prepared operation key=C1BTVJNHIM0X tag=LVB3EURKKC49 type=del size=2031334 2024-04-07T16:37:43.120 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.129+0000 7f346a3868c0 1 < completed delete operation key=IP8YW2A75TCW tag=QNMHWJ8M7392 type=del 2024-04-07T16:37:43.128 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.137+0000 7f346a3868c0 1 > prepared part key=_multipart_QS4HYZFFAJNF.2~KWGGD1X713MC.0 size=1048576 2024-04-07T16:37:43.131 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.141+0000 7f346a3868c0 1 > prepared part key=_multipart_QS4HYZFFAJNF.2~KWGGD1X713MC.1 size=1048576 2024-04-07T16:37:43.134 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.141+0000 7f346a3868c0 1 > prepared part key=_multipart_QS4HYZFFAJNF.2~KWGGD1X713MC.2 size=897133 2024-04-07T16:37:43.134 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.141+0000 7f346a3868c0 1 > prepared multipart upload key=QS4HYZFFAJNF upload=KWGGD1X713MC tag=8ND7P7JA2D4K type=write size=4042861 2024-04-07T16:37:43.138 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.145+0000 7f346a3868c0 1 < completed delete operation key=QVOKSQAGHPA9 tag=OWZB2K15NDJH type=del 2024-04-07T16:37:43.142 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.149+0000 7f346a3868c0 1 > prepared operation key=EQR0FY86AT5X tag=C266LR2N4RJN type=del size=2804957 2024-04-07T16:37:43.146 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.153+0000 7f346a3868c0 1 < completed part key=_multipart_3N0QJRYVO3FV.2~M30WLYE3HD85.0 size=215728 2024-04-07T16:37:43.153 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.161+0000 7f346a3868c0 1 < completed multipart upload key=3N0QJRYVO3FV upload=M30WLYE3HD85 tag=BACBBXHWW3ST type=write size=1264304 2024-04-07T16:37:43.157 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.165+0000 7f346a3868c0 1 > prepared operation key=HBLEXONKXJTL tag=YTLEEDY9O0A6 type=del size=1854956 2024-04-07T16:37:43.162 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.169+0000 7f346a3868c0 1 < completed part key=_multipart_X5Q46GO8YTOK.2~KTDO5K307DO1.0 size=1048576 2024-04-07T16:37:43.165 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.173+0000 7f346a3868c0 1 < completed part key=_multipart_X5Q46GO8YTOK.2~KTDO5K307DO1.1 size=1048576 2024-04-07T16:37:43.169 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.177+0000 7f346a3868c0 1 < completed part key=_multipart_X5Q46GO8YTOK.2~KTDO5K307DO1.2 size=146014 2024-04-07T16:37:43.176 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.185+0000 7f346a3868c0 1 < completed multipart upload key=X5Q46GO8YTOK upload=KTDO5K307DO1 tag=I9FTBFX1TJF3 type=write size=3291742 2024-04-07T16:37:43.183 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.193+0000 7f346a3868c0 1 > prepared part key=_multipart_E84U0SX44NQ8.2~DWWEWFPUME9U.0 size=1048576 2024-04-07T16:37:43.186 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.193+0000 7f346a3868c0 1 > prepared part key=_multipart_E84U0SX44NQ8.2~DWWEWFPUME9U.1 size=1048576 2024-04-07T16:37:43.189 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.197+0000 7f346a3868c0 1 > prepared part key=_multipart_E84U0SX44NQ8.2~DWWEWFPUME9U.2 size=1028925 2024-04-07T16:37:43.189 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.197+0000 7f346a3868c0 1 > prepared multipart upload key=E84U0SX44NQ8 upload=DWWEWFPUME9U tag=JQ8BJT1VOO0H type=write size=4174653 2024-04-07T16:37:43.193 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.201+0000 7f346a3868c0 1 < completed delete operation key=CL4PP1RKZU0E tag=K571UPRPFKCT type=del 2024-04-07T16:37:43.197 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.205+0000 7f346a3868c0 1 > prepared operation key=XJLQZ7IS9U1T tag=LGKTHXEF38IH type=write size=471352 2024-04-07T16:37:43.201 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.209+0000 7f346a3868c0 1 < canceled operation key=XJLQZ7IS9U1T tag=C7MS9BF4KQFB type=write size=385675 2024-04-07T16:37:43.204 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.213+0000 7f346a3868c0 1 > prepared operation key=GZXHYZO5SMUP tag=93NH340P73EB type=del size=3928292 2024-04-07T16:37:43.208 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.217+0000 7f346a3868c0 1 < completed delete operation key=ROFH30W3UCAX tag=2B11O3VCT6GH type=del 2024-04-07T16:37:43.213 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.221+0000 7f346a3868c0 1 > prepared operation key=1WGUPH7X5UA0 tag=37PPVISH7KV9 type=del size=3583569 2024-04-07T16:37:43.217 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.225+0000 7f346a3868c0 1 < completed delete operation key=GBQSME7SY1AH tag=G8ZY5KP6CJI6 type=del 2024-04-07T16:37:43.221 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.229+0000 7f346a3868c0 1 > prepared operation key=EOYFVEI50SQ1 tag=OIRCNSCOC993 type=del size=681392 2024-04-07T16:37:43.226 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.233+0000 7f346a3868c0 1 < canceled operation key=U3EID5EK64HD tag=HCSMOSETOXQ8 type=del size=3140174 2024-04-07T16:37:43.234 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.241+0000 7f346a3868c0 1 > prepared part key=_multipart_GZXHYZO5SMUP.2~501KXW26H223.0 size=1048576 2024-04-07T16:37:43.237 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.245+0000 7f346a3868c0 1 > prepared part key=_multipart_GZXHYZO5SMUP.2~501KXW26H223.1 size=153734 2024-04-07T16:37:43.237 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.245+0000 7f346a3868c0 1 > prepared multipart upload key=GZXHYZO5SMUP upload=501KXW26H223 tag=BPXOALFVP8JY type=write size=2250886 2024-04-07T16:37:43.241 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.249+0000 7f346a3868c0 1 < completed write operation key=GZXHYZO5SMUP tag=ETKRHV6LW7FW type=write size=722745 2024-04-07T16:37:43.244 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.253+0000 7f346a3868c0 1 > prepared operation key=EOYFVEI50SQ1 tag=GSJLICA8BE59 type=del size=72319 2024-04-07T16:37:43.252 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.261+0000 7f346a3868c0 1 < suggested remove operation key=DXTO3AJ429AC tag=6WNC2V9OT59V type=del 2024-04-07T16:37:43.256 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.265+0000 7f346a3868c0 1 > prepared operation key=X5Q46GO8YTOK tag=UG2FR3A5MA6O type=write size=252725 2024-04-07T16:37:43.260 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.269+0000 7f346a3868c0 1 < completed part key=_multipart_NYET3212USVE.2~3D7GTR2YG8QS.0 size=1048576 2024-04-07T16:37:43.263 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.269+0000 7f346a3868c0 1 < completed part key=_multipart_NYET3212USVE.2~3D7GTR2YG8QS.1 size=667495 2024-04-07T16:37:43.269 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.277+0000 7f346a3868c0 1 < completed multipart upload key=NYET3212USVE upload=3D7GTR2YG8QS tag=JMIM4HKSV55A type=write size=2764647 2024-04-07T16:37:43.273 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.281+0000 7f346a3868c0 1 > prepared operation key=GZXHYZO5SMUP tag=4029D6JT1RLD type=del size=3172070 2024-04-07T16:37:43.278 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.285+0000 7f346a3868c0 1 < completed delete operation key=YOK8BAEOMPUR tag=3MGP75DUWTKG type=del 2024-04-07T16:37:43.282 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.289+0000 7f346a3868c0 1 > prepared operation key=NYET3212USVE tag=ZJBTKHB5R7DS type=del size=1387926 2024-04-07T16:37:43.286 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.293+0000 7f346a3868c0 1 < completed delete operation key=UIW3DN4NUYPP tag=ITLE1I2RHM77 type=del 2024-04-07T16:37:43.297 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.305+0000 7f346a3868c0 1 > prepared part key=_multipart_1VKVQXSQMTU1.2~LVY0UCJMFBJK.0 size=259381 2024-04-07T16:37:43.297 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.305+0000 7f346a3868c0 1 > prepared multipart upload key=1VKVQXSQMTU1 upload=LVY0UCJMFBJK tag=TBY1NQ4IPG2R type=write size=1307957 2024-04-07T16:37:43.301 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.309+0000 7f346a3868c0 1 < completed delete operation key=CL4PP1RKZU0E tag=0OS4M5CEP8V5 type=del 2024-04-07T16:37:43.307 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.317+0000 7f346a3868c0 1 > prepared part key=_multipart_U3EID5EK64HD.2~VRP4SZZ2YHH2.0 size=1024586 2024-04-07T16:37:43.307 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.317+0000 7f346a3868c0 1 > prepared multipart upload key=U3EID5EK64HD upload=VRP4SZZ2YHH2 tag=UOA67N6ZXR3L type=write size=2073162 2024-04-07T16:37:43.314 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.321+0000 7f346a3868c0 1 < suggested update operation key=C1BTVJNHIM0X tag=LVB3EURKKC49 type=del size=2031334 2024-04-07T16:37:43.321 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.329+0000 7f346a3868c0 1 > prepared part key=_multipart_7G5C6HBK6QWI.2~CS2VJ57LVM8H.0 size=130238 2024-04-07T16:37:43.321 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.329+0000 7f346a3868c0 1 > prepared multipart upload key=7G5C6HBK6QWI upload=CS2VJ57LVM8H tag=QH3MR8LBSUAK type=write size=1178814 2024-04-07T16:37:43.325 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.333+0000 7f346a3868c0 1 < completed part key=_multipart_QS4HYZFFAJNF.2~KWGGD1X713MC.0 size=1048576 2024-04-07T16:37:43.328 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.337+0000 7f346a3868c0 1 < completed part key=_multipart_QS4HYZFFAJNF.2~KWGGD1X713MC.1 size=1048576 2024-04-07T16:37:43.330 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.337+0000 7f346a3868c0 1 < completed part key=_multipart_QS4HYZFFAJNF.2~KWGGD1X713MC.2 size=897133 2024-04-07T16:37:43.336 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.345+0000 7f346a3868c0 1 < completed multipart upload key=QS4HYZFFAJNF upload=KWGGD1X713MC tag=8ND7P7JA2D4K type=write size=4042861 2024-04-07T16:37:43.343 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.353+0000 7f346a3868c0 1 > prepared part key=_multipart_NYET3212USVE.2~DG3FEXFB3FJP.0 size=1048576 2024-04-07T16:37:43.346 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.353+0000 7f346a3868c0 1 > prepared part key=_multipart_NYET3212USVE.2~DG3FEXFB3FJP.1 size=57163 2024-04-07T16:37:43.346 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.353+0000 7f346a3868c0 1 > prepared multipart upload key=NYET3212USVE upload=DG3FEXFB3FJP tag=KCN6NOXCEOLX type=write size=2154315 2024-04-07T16:37:43.350 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.357+0000 7f346a3868c0 1 < completed delete operation key=EQR0FY86AT5X tag=C266LR2N4RJN type=del 2024-04-07T16:37:43.356 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.365+0000 7f346a3868c0 1 > prepared part key=_multipart_QVOKSQAGHPA9.2~PW0R7I3H4ZAZ.0 size=1048576 2024-04-07T16:37:43.359 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.369+0000 7f346a3868c0 1 > prepared part key=_multipart_QVOKSQAGHPA9.2~PW0R7I3H4ZAZ.1 size=1048576 2024-04-07T16:37:43.362 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.369+0000 7f346a3868c0 1 > prepared part key=_multipart_QVOKSQAGHPA9.2~PW0R7I3H4ZAZ.2 size=994826 2024-04-07T16:37:43.362 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.369+0000 7f346a3868c0 1 > prepared multipart upload key=QVOKSQAGHPA9 upload=PW0R7I3H4ZAZ tag=ZD0H19E5RY54 type=write size=4140554 2024-04-07T16:37:43.370 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.377+0000 7f346a3868c0 1 < suggested remove operation key=HBLEXONKXJTL tag=YTLEEDY9O0A6 type=del 2024-04-07T16:37:43.373 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.381+0000 7f346a3868c0 1 > prepared operation key=1VBSUWNI3V1T tag=SDIH3OUNKDA3 type=del size=2533324 2024-04-07T16:37:43.376 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.385+0000 7f346a3868c0 1 < completed part key=_multipart_E84U0SX44NQ8.2~DWWEWFPUME9U.0 size=1048576 2024-04-07T16:37:43.379 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.389+0000 7f346a3868c0 1 < completed part key=_multipart_E84U0SX44NQ8.2~DWWEWFPUME9U.1 size=1048576 2024-04-07T16:37:43.382 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.389+0000 7f346a3868c0 1 < completed part key=_multipart_E84U0SX44NQ8.2~DWWEWFPUME9U.2 size=1028925 2024-04-07T16:37:43.387 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.397+0000 7f346a3868c0 1 < completed multipart upload key=E84U0SX44NQ8 upload=DWWEWFPUME9U tag=JQ8BJT1VOO0H type=write size=4174653 2024-04-07T16:37:43.394 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.401+0000 7f346a3868c0 1 > prepared part key=_multipart_YOK8BAEOMPUR.2~AKONIYCN1CYM.0 size=499032 2024-04-07T16:37:43.394 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.401+0000 7f346a3868c0 1 > prepared multipart upload key=YOK8BAEOMPUR upload=AKONIYCN1CYM tag=UB5QRYK9V0X4 type=write size=1547608 2024-04-07T16:37:43.397 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.405+0000 7f346a3868c0 1 < completed write operation key=XJLQZ7IS9U1T tag=LGKTHXEF38IH type=write size=471352 2024-04-07T16:37:43.401 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.409+0000 7f346a3868c0 1 > prepared operation key=QTCIIPTXIRW2 tag=GZKXJTPNEAHJ type=del size=3860057 2024-04-07T16:37:43.404 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.413+0000 7f346a3868c0 1 < completed delete operation key=GZXHYZO5SMUP tag=93NH340P73EB type=del 2024-04-07T16:37:43.408 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.417+0000 7f346a3868c0 1 > prepared operation key=ROFH30W3UCAX tag=K38RYTA7CR8L type=write size=279961 2024-04-07T16:37:43.411 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.421+0000 7f346a3868c0 1 < completed delete operation key=1WGUPH7X5UA0 tag=37PPVISH7KV9 type=del 2024-04-07T16:37:43.417 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.425+0000 7f346a3868c0 1 > prepared part key=_multipart_CL4PP1RKZU0E.2~LYVLZPAQ35VU.0 size=1048576 2024-04-07T16:37:43.420 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.429+0000 7f346a3868c0 1 > prepared part key=_multipart_CL4PP1RKZU0E.2~LYVLZPAQ35VU.1 size=1048576 2024-04-07T16:37:43.423 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.433+0000 7f346a3868c0 1 > prepared part key=_multipart_CL4PP1RKZU0E.2~LYVLZPAQ35VU.2 size=774623 2024-04-07T16:37:43.423 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.433+0000 7f346a3868c0 1 > prepared multipart upload key=CL4PP1RKZU0E upload=LYVLZPAQ35VU tag=5S126H4UUEL7 type=write size=3920351 2024-04-07T16:37:43.427 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.433+0000 7f346a3868c0 1 < completed delete operation key=EOYFVEI50SQ1 tag=OIRCNSCOC993 type=del 2024-04-07T16:37:43.431 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.437+0000 7f346a3868c0 1 > prepared operation key=1WGUPH7X5UA0 tag=TBLPSJL68ZTM type=write size=669010 2024-04-07T16:37:43.435 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.445+0000 7f346a3868c0 1 < completed part key=_multipart_GZXHYZO5SMUP.2~501KXW26H223.0 size=1048576 2024-04-07T16:37:43.438 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.445+0000 7f346a3868c0 1 < completed part key=_multipart_GZXHYZO5SMUP.2~501KXW26H223.1 size=153734 2024-04-07T16:37:43.444 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.453+0000 7f346a3868c0 1 < completed multipart upload key=GZXHYZO5SMUP upload=501KXW26H223 tag=BPXOALFVP8JY type=write size=2250886 2024-04-07T16:37:43.449 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.457+0000 7f346a3868c0 1 > prepared operation key=6XDS0A7L0IY8 tag=WYNT4EQOB1NU type=del size=3880598 2024-04-07T16:37:43.451 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.461+0000 7f346a3868c0 1 < completed delete operation key=EOYFVEI50SQ1 tag=GSJLICA8BE59 type=del 2024-04-07T16:37:43.455 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.461+0000 7f346a3868c0 1 > prepared operation key=IX3F0IUOKBX1 tag=BK3B340PFHIG type=write size=928906 2024-04-07T16:37:43.458 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.465+0000 7f346a3868c0 1 < completed write operation key=X5Q46GO8YTOK tag=UG2FR3A5MA6O type=write size=252725 2024-04-07T16:37:43.462 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.469+0000 7f346a3868c0 1 > prepared operation key=95YVKJ14JD8W tag=0YPBUQD6SHAJ type=del size=2571819 2024-04-07T16:37:43.465 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.473+0000 7f346a3868c0 1 < completed delete operation key=GZXHYZO5SMUP tag=4029D6JT1RLD type=del 2024-04-07T16:37:43.473 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.481+0000 7f346a3868c0 1 > prepared part key=_multipart_GZXHYZO5SMUP.2~8P7DNADB1ZQY.0 size=1048576 2024-04-07T16:37:43.475 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.485+0000 7f346a3868c0 1 > prepared part key=_multipart_GZXHYZO5SMUP.2~8P7DNADB1ZQY.1 size=1011894 2024-04-07T16:37:43.475 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.485+0000 7f346a3868c0 1 > prepared multipart upload key=GZXHYZO5SMUP upload=8P7DNADB1ZQY tag=D5OSUNZS8DGP type=write size=3109046 2024-04-07T16:37:43.479 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.485+0000 7f346a3868c0 1 < completed delete operation key=NYET3212USVE tag=ZJBTKHB5R7DS type=del 2024-04-07T16:37:43.482 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.489+0000 7f346a3868c0 1 > prepared operation key=HBLEXONKXJTL tag=LBQ2TRMJTO5Q type=write size=870547 2024-04-07T16:37:43.486 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.493+0000 7f346a3868c0 1 < completed part key=_multipart_1VKVQXSQMTU1.2~LVY0UCJMFBJK.0 size=259381 2024-04-07T16:37:43.491 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.497+0000 7f346a3868c0 1 < completed multipart upload key=1VKVQXSQMTU1 upload=LVY0UCJMFBJK tag=TBY1NQ4IPG2R type=write size=1307957 2024-04-07T16:37:43.496 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.505+0000 7f346a3868c0 1 > prepared part key=_multipart_X5Q46GO8YTOK.2~D4ASQG2JJQ37.0 size=1048576 2024-04-07T16:37:43.499 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.509+0000 7f346a3868c0 1 > prepared part key=_multipart_X5Q46GO8YTOK.2~D4ASQG2JJQ37.1 size=1048576 2024-04-07T16:37:43.502 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.509+0000 7f346a3868c0 1 > prepared part key=_multipart_X5Q46GO8YTOK.2~D4ASQG2JJQ37.2 size=435126 2024-04-07T16:37:43.502 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.509+0000 7f346a3868c0 1 > prepared multipart upload key=X5Q46GO8YTOK upload=D4ASQG2JJQ37 tag=P9VG72RQN40D type=write size=3580854 2024-04-07T16:37:43.506 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.513+0000 7f346a3868c0 1 < completed part key=_multipart_U3EID5EK64HD.2~VRP4SZZ2YHH2.0 size=1024586 2024-04-07T16:37:43.512 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.521+0000 7f346a3868c0 1 < completed multipart upload key=U3EID5EK64HD upload=VRP4SZZ2YHH2 tag=UOA67N6ZXR3L type=write size=2073162 2024-04-07T16:37:43.518 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.525+0000 7f346a3868c0 1 > prepared part key=_multipart_GZXHYZO5SMUP.2~XQ12BYMWK9Q6.0 size=1048576 2024-04-07T16:37:43.521 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.529+0000 7f346a3868c0 1 > prepared part key=_multipart_GZXHYZO5SMUP.2~XQ12BYMWK9Q6.1 size=872924 2024-04-07T16:37:43.521 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.529+0000 7f346a3868c0 1 > prepared multipart upload key=GZXHYZO5SMUP upload=XQ12BYMWK9Q6 tag=2LNO9L8JVF8J type=write size=2970076 2024-04-07T16:37:43.524 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.533+0000 7f346a3868c0 1 < completed part key=_multipart_7G5C6HBK6QWI.2~CS2VJ57LVM8H.0 size=130238 2024-04-07T16:37:43.530 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.537+0000 7f346a3868c0 1 < completed multipart upload key=7G5C6HBK6QWI upload=CS2VJ57LVM8H tag=QH3MR8LBSUAK type=write size=1178814 2024-04-07T16:37:43.536 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.545+0000 7f346a3868c0 1 > prepared part key=_multipart_NYET3212USVE.2~PA4O3FBPE6U9.0 size=1048576 2024-04-07T16:37:43.539 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.549+0000 7f346a3868c0 1 > prepared part key=_multipart_NYET3212USVE.2~PA4O3FBPE6U9.1 size=815686 2024-04-07T16:37:43.539 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.549+0000 7f346a3868c0 1 > prepared multipart upload key=NYET3212USVE upload=PA4O3FBPE6U9 tag=BVW1NG4K0LVB type=write size=2912838 2024-04-07T16:37:43.543 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.553+0000 7f346a3868c0 1 < completed part key=_multipart_NYET3212USVE.2~DG3FEXFB3FJP.0 size=1048576 2024-04-07T16:37:43.546 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.553+0000 7f346a3868c0 1 < completed part key=_multipart_NYET3212USVE.2~DG3FEXFB3FJP.1 size=57163 2024-04-07T16:37:43.552 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.561+0000 7f346a3868c0 1 < completed multipart upload key=NYET3212USVE upload=DG3FEXFB3FJP tag=KCN6NOXCEOLX type=write size=2154315 2024-04-07T16:37:43.556 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.565+0000 7f346a3868c0 1 > prepared operation key=EOYFVEI50SQ1 tag=776XHJUCYWSJ type=write size=150785 2024-04-07T16:37:43.557 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.565+0000 7f346a3868c0 1 < canceled part key=_multipart_QVOKSQAGHPA9.2~PW0R7I3H4ZAZ.0 size=1048576 2024-04-07T16:37:43.560 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.569+0000 7f346a3868c0 1 < completed part key=_multipart_QVOKSQAGHPA9.2~PW0R7I3H4ZAZ.1 size=1048576 2024-04-07T16:37:43.562 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.569+0000 7f346a3868c0 1 < completed part key=_multipart_QVOKSQAGHPA9.2~PW0R7I3H4ZAZ.2 size=994826 2024-04-07T16:37:43.568 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.577+0000 7f346a3868c0 1 < canceled multipart upload key=QVOKSQAGHPA9 upload=PW0R7I3H4ZAZ tag=ZD0H19E5RY54 type=write size=4140554 2024-04-07T16:37:43.574 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.581+0000 7f346a3868c0 1 > prepared part key=_multipart_IP8YW2A75TCW.2~QTPX9NFMBMXJ.0 size=1048576 2024-04-07T16:37:43.576 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.585+0000 7f346a3868c0 1 > prepared part key=_multipart_IP8YW2A75TCW.2~QTPX9NFMBMXJ.1 size=1048576 2024-04-07T16:37:43.579 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.589+0000 7f346a3868c0 1 > prepared part key=_multipart_IP8YW2A75TCW.2~QTPX9NFMBMXJ.2 size=844218 2024-04-07T16:37:43.579 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.589+0000 7f346a3868c0 1 > prepared multipart upload key=IP8YW2A75TCW upload=QTPX9NFMBMXJ tag=IAHPKSL0C4AS type=write size=3989946 2024-04-07T16:37:43.583 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.589+0000 7f346a3868c0 1 < completed delete operation key=1VBSUWNI3V1T tag=SDIH3OUNKDA3 type=del 2024-04-07T16:37:43.589 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.597+0000 7f346a3868c0 1 > prepared part key=_multipart_CL4PP1RKZU0E.2~KTJ30ZIWV0QY.0 size=1048576 2024-04-07T16:37:43.593 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.601+0000 7f346a3868c0 1 > prepared part key=_multipart_CL4PP1RKZU0E.2~KTJ30ZIWV0QY.1 size=1048576 2024-04-07T16:37:43.596 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.605+0000 7f346a3868c0 1 > prepared part key=_multipart_CL4PP1RKZU0E.2~KTJ30ZIWV0QY.2 size=89812 2024-04-07T16:37:43.596 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.605+0000 7f346a3868c0 1 > prepared multipart upload key=CL4PP1RKZU0E upload=KTJ30ZIWV0QY tag=IYWURGDRW9VM type=write size=3235540 2024-04-07T16:37:43.599 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.609+0000 7f346a3868c0 1 < completed part key=_multipart_YOK8BAEOMPUR.2~AKONIYCN1CYM.0 size=499032 2024-04-07T16:37:43.605 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.613+0000 7f346a3868c0 1 < completed multipart upload key=YOK8BAEOMPUR upload=AKONIYCN1CYM tag=UB5QRYK9V0X4 type=write size=1547608 2024-04-07T16:37:43.609 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.617+0000 7f346a3868c0 1 > prepared operation key=QVOKSQAGHPA9 tag=CB1FBT3W3AY9 type=del size=912777 2024-04-07T16:37:43.612 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.621+0000 7f346a3868c0 1 < completed delete operation key=QTCIIPTXIRW2 tag=GZKXJTPNEAHJ type=del 2024-04-07T16:37:43.618 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.625+0000 7f346a3868c0 1 > prepared part key=_multipart_1WGUPH7X5UA0.2~UJ8N81X812WV.0 size=1048576 2024-04-07T16:37:43.622 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.629+0000 7f346a3868c0 1 > prepared part key=_multipart_1WGUPH7X5UA0.2~UJ8N81X812WV.1 size=1048576 2024-04-07T16:37:43.624 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.633+0000 7f346a3868c0 1 > prepared part key=_multipart_1WGUPH7X5UA0.2~UJ8N81X812WV.2 size=864383 2024-04-07T16:37:43.624 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.633+0000 7f346a3868c0 1 > prepared multipart upload key=1WGUPH7X5UA0 upload=UJ8N81X812WV tag=AGP8PBOWBY2K type=write size=4010111 2024-04-07T16:37:43.628 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.637+0000 7f346a3868c0 1 < completed write operation key=ROFH30W3UCAX tag=K38RYTA7CR8L type=write size=279961 2024-04-07T16:37:43.632 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.641+0000 7f346a3868c0 1 > prepared operation key=1WGUPH7X5UA0 tag=AA2FSO1AOAUC type=del size=955290 2024-04-07T16:37:43.635 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.645+0000 7f346a3868c0 1 < completed part key=_multipart_CL4PP1RKZU0E.2~LYVLZPAQ35VU.0 size=1048576 2024-04-07T16:37:43.639 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.645+0000 7f346a3868c0 1 < completed part key=_multipart_CL4PP1RKZU0E.2~LYVLZPAQ35VU.1 size=1048576 2024-04-07T16:37:43.642 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.649+0000 7f346a3868c0 1 < completed part key=_multipart_CL4PP1RKZU0E.2~LYVLZPAQ35VU.2 size=774623 2024-04-07T16:37:43.647 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.657+0000 7f346a3868c0 1 < completed multipart upload key=CL4PP1RKZU0E upload=LYVLZPAQ35VU tag=5S126H4UUEL7 type=write size=3920351 2024-04-07T16:37:43.650 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.657+0000 7f346a3868c0 1 > prepared operation key=YXMDZB23YTB3 tag=22UGF0KDPDTS type=del size=2636964 2024-04-07T16:37:43.654 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.661+0000 7f346a3868c0 1 < completed write operation key=1WGUPH7X5UA0 tag=TBLPSJL68ZTM type=write size=669010 2024-04-07T16:37:43.657 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.665+0000 7f346a3868c0 1 > prepared operation key=7G5C6HBK6QWI tag=NJ64VGW9QPFL type=del size=15752 2024-04-07T16:37:43.661 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.669+0000 7f346a3868c0 1 < completed delete operation key=6XDS0A7L0IY8 tag=WYNT4EQOB1NU type=del 2024-04-07T16:37:43.665 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.673+0000 7f346a3868c0 1 > prepared operation key=HBLEXONKXJTL tag=4BZCEEQJZXOH type=del size=1497064 2024-04-07T16:37:43.673 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.681+0000 7f346a3868c0 1 < suggested update operation key=IX3F0IUOKBX1 tag=BK3B340PFHIG type=write size=928906 2024-04-07T16:37:43.679 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.689+0000 7f346a3868c0 1 > prepared part key=_multipart_HBLEXONKXJTL.2~2VF73K8YZM8U.0 size=1048576 2024-04-07T16:37:43.682 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.689+0000 7f346a3868c0 1 > prepared part key=_multipart_HBLEXONKXJTL.2~2VF73K8YZM8U.1 size=1048576 2024-04-07T16:37:43.685 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.693+0000 7f346a3868c0 1 > prepared part key=_multipart_HBLEXONKXJTL.2~2VF73K8YZM8U.2 size=121499 2024-04-07T16:37:43.685 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.693+0000 7f346a3868c0 1 > prepared multipart upload key=HBLEXONKXJTL upload=2VF73K8YZM8U tag=ZCPYK7F4RHMZ type=write size=3267227 2024-04-07T16:37:43.689 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.697+0000 7f346a3868c0 1 < completed delete operation key=95YVKJ14JD8W tag=0YPBUQD6SHAJ type=del 2024-04-07T16:37:43.693 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.701+0000 7f346a3868c0 1 > prepared operation key=6XDS0A7L0IY8 tag=K7HORVS3EF5A type=write size=550363 2024-04-07T16:37:43.693 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.701+0000 7f346a3868c0 1 < canceled part key=_multipart_GZXHYZO5SMUP.2~8P7DNADB1ZQY.0 size=1048576 2024-04-07T16:37:43.697 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.705+0000 7f346a3868c0 1 < completed part key=_multipart_GZXHYZO5SMUP.2~8P7DNADB1ZQY.1 size=1011894 2024-04-07T16:37:43.702 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.709+0000 7f346a3868c0 1 < canceled multipart upload key=GZXHYZO5SMUP upload=8P7DNADB1ZQY tag=D5OSUNZS8DGP type=write size=3109046 2024-04-07T16:37:43.706 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.713+0000 7f346a3868c0 1 > prepared operation key=E84U0SX44NQ8 tag=YR1AVRYLA9AE type=del size=1572382 2024-04-07T16:37:43.709 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.717+0000 7f346a3868c0 1 < completed write operation key=HBLEXONKXJTL tag=LBQ2TRMJTO5Q type=write size=870547 2024-04-07T16:37:43.713 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.721+0000 7f346a3868c0 1 > prepared operation key=IX3F0IUOKBX1 tag=N13H2V4M4KNJ type=del size=159021 2024-04-07T16:37:43.716 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.725+0000 7f346a3868c0 1 < completed part key=_multipart_X5Q46GO8YTOK.2~D4ASQG2JJQ37.0 size=1048576 2024-04-07T16:37:43.719 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.729+0000 7f346a3868c0 1 < completed part key=_multipart_X5Q46GO8YTOK.2~D4ASQG2JJQ37.1 size=1048576 2024-04-07T16:37:43.722 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.729+0000 7f346a3868c0 1 < completed part key=_multipart_X5Q46GO8YTOK.2~D4ASQG2JJQ37.2 size=435126 2024-04-07T16:37:43.728 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.737+0000 7f346a3868c0 1 < completed multipart upload key=X5Q46GO8YTOK upload=D4ASQG2JJQ37 tag=P9VG72RQN40D type=write size=3580854 2024-04-07T16:37:43.736 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.745+0000 7f346a3868c0 1 > prepared part key=_multipart_95YVKJ14JD8W.2~34UH6UDUD62M.0 size=1048576 2024-04-07T16:37:43.739 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.749+0000 7f346a3868c0 1 > prepared part key=_multipart_95YVKJ14JD8W.2~34UH6UDUD62M.1 size=197499 2024-04-07T16:37:43.739 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.749+0000 7f346a3868c0 1 > prepared multipart upload key=95YVKJ14JD8W upload=34UH6UDUD62M tag=ICY2R9OCKIFA type=write size=2294651 2024-04-07T16:37:43.743 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.749+0000 7f346a3868c0 1 < completed part key=_multipart_GZXHYZO5SMUP.2~XQ12BYMWK9Q6.0 size=1048576 2024-04-07T16:37:43.747 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.753+0000 7f346a3868c0 1 < completed part key=_multipart_GZXHYZO5SMUP.2~XQ12BYMWK9Q6.1 size=872924 2024-04-07T16:37:43.753 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.761+0000 7f346a3868c0 1 < completed multipart upload key=GZXHYZO5SMUP upload=XQ12BYMWK9Q6 tag=2LNO9L8JVF8J type=write size=2970076 2024-04-07T16:37:43.756 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.765+0000 7f346a3868c0 1 > prepared operation key=X5Q46GO8YTOK tag=CS07HW6FYZLB type=del size=3962125 2024-04-07T16:37:43.760 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.769+0000 7f346a3868c0 1 < completed part key=_multipart_NYET3212USVE.2~PA4O3FBPE6U9.0 size=1048576 2024-04-07T16:37:43.763 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.773+0000 7f346a3868c0 1 < completed part key=_multipart_NYET3212USVE.2~PA4O3FBPE6U9.1 size=815686 2024-04-07T16:37:43.770 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.777+0000 7f346a3868c0 1 < completed multipart upload key=NYET3212USVE upload=PA4O3FBPE6U9 tag=BVW1NG4K0LVB type=write size=2912838 2024-04-07T16:37:43.777 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.785+0000 7f346a3868c0 1 > prepared part key=_multipart_LXOZ0T2KY6OR.2~TJ0M5IBOHDZR.0 size=551628 2024-04-07T16:37:43.777 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.785+0000 7f346a3868c0 1 > prepared multipart upload key=LXOZ0T2KY6OR upload=TJ0M5IBOHDZR tag=AXED98ORSPWT type=write size=1600204 2024-04-07T16:37:43.781 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.789+0000 7f346a3868c0 1 < completed write operation key=EOYFVEI50SQ1 tag=776XHJUCYWSJ type=write size=150785 2024-04-07T16:37:43.785 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.793+0000 7f346a3868c0 1 > prepared operation key=GZXHYZO5SMUP tag=S3XFUMUBEAG4 type=del size=385428 2024-04-07T16:37:43.789 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.797+0000 7f346a3868c0 1 < completed part key=_multipart_IP8YW2A75TCW.2~QTPX9NFMBMXJ.0 size=1048576 2024-04-07T16:37:43.792 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.801+0000 7f346a3868c0 1 < completed part key=_multipart_IP8YW2A75TCW.2~QTPX9NFMBMXJ.1 size=1048576 2024-04-07T16:37:43.795 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.805+0000 7f346a3868c0 1 < completed part key=_multipart_IP8YW2A75TCW.2~QTPX9NFMBMXJ.2 size=844218 2024-04-07T16:37:43.801 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.809+0000 7f346a3868c0 1 < completed multipart upload key=IP8YW2A75TCW upload=QTPX9NFMBMXJ tag=IAHPKSL0C4AS type=write size=3989946 2024-04-07T16:37:43.806 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:43.813+0000 7f346a3868c0 1 > prepared operation key=1VKVQXSQMTU1 tag=55C4I0U68QKX type=del size=731240 2024-04-07T16:37:43.856 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:37:43 smithi144 bash[66654]: cluster 2024-04-07T16:37:42.477763+0000 mon.a (mon.0) 2018 : cluster [DBG] osdmap e1060: 8 total, 8 up, 8 in 2024-04-07T16:37:43.941 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:37:43 smithi100 bash[74942]: cluster 2024-04-07T16:37:42.477763+0000 mon.a (mon.0) 2018 : cluster [DBG] osdmap e1060: 8 total, 8 up, 8 in 2024-04-07T16:37:43.941 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:37:43 smithi100 bash[76202]: cluster 2024-04-07T16:37:42.477763+0000 mon.a (mon.0) 2018 : cluster [DBG] osdmap e1060: 8 total, 8 up, 8 in 2024-04-07T16:37:44.018 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.025+0000 7f346a3868c0 1 < completed part key=_multipart_CL4PP1RKZU0E.2~KTJ30ZIWV0QY.0 size=1048576 2024-04-07T16:37:44.021 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.029+0000 7f346a3868c0 1 < completed part key=_multipart_CL4PP1RKZU0E.2~KTJ30ZIWV0QY.1 size=1048576 2024-04-07T16:37:44.024 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.033+0000 7f346a3868c0 1 < completed part key=_multipart_CL4PP1RKZU0E.2~KTJ30ZIWV0QY.2 size=89812 2024-04-07T16:37:44.031 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.041+0000 7f346a3868c0 1 < completed multipart upload key=CL4PP1RKZU0E upload=KTJ30ZIWV0QY tag=IYWURGDRW9VM type=write size=3235540 2024-04-07T16:37:44.035 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.045+0000 7f346a3868c0 1 > prepared operation key=DXTO3AJ429AC tag=3VESRG0TS31X type=del size=2056214 2024-04-07T16:37:44.039 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.049+0000 7f346a3868c0 1 < completed delete operation key=QVOKSQAGHPA9 tag=CB1FBT3W3AY9 type=del 2024-04-07T16:37:44.043 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.053+0000 7f346a3868c0 1 > prepared operation key=EQR0FY86AT5X tag=QK8ZBL8L27PY type=del size=4047744 2024-04-07T16:37:44.044 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.053+0000 7f346a3868c0 1 < canceled part key=_multipart_1WGUPH7X5UA0.2~UJ8N81X812WV.0 size=1048576 2024-04-07T16:37:44.047 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.057+0000 7f346a3868c0 1 < completed part key=_multipart_1WGUPH7X5UA0.2~UJ8N81X812WV.1 size=1048576 2024-04-07T16:37:44.050 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.057+0000 7f346a3868c0 1 < completed part key=_multipart_1WGUPH7X5UA0.2~UJ8N81X812WV.2 size=864383 2024-04-07T16:37:44.056 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.065+0000 7f346a3868c0 1 < canceled multipart upload key=1WGUPH7X5UA0 upload=UJ8N81X812WV tag=AGP8PBOWBY2K type=write size=4010111 2024-04-07T16:37:44.060 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.069+0000 7f346a3868c0 1 > prepared operation key=8F81G0P0YBFW tag=1YIMBL4YXULF type=write size=350067 2024-04-07T16:37:44.064 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.073+0000 7f346a3868c0 1 < canceled operation key=1WGUPH7X5UA0 tag=AA2FSO1AOAUC type=del size=955290 2024-04-07T16:37:44.069 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.077+0000 7f346a3868c0 1 > prepared operation key=E84U0SX44NQ8 tag=SGJGORP7FJDI type=del size=1783059 2024-04-07T16:37:44.073 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.081+0000 7f346a3868c0 1 < completed delete operation key=YXMDZB23YTB3 tag=22UGF0KDPDTS type=del 2024-04-07T16:37:44.077 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.085+0000 7f346a3868c0 1 > prepared operation key=EOYFVEI50SQ1 tag=JGS1K0670KSM type=write size=255616 2024-04-07T16:37:44.085 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.093+0000 7f346a3868c0 1 < suggested remove operation key=7G5C6HBK6QWI tag=NJ64VGW9QPFL type=del 2024-04-07T16:37:44.089 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.097+0000 7f346a3868c0 1 > prepared operation key=1VBSUWNI3V1T tag=3W3PSHKNWAVG type=del size=3305390 2024-04-07T16:37:44.097 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.105+0000 7f346a3868c0 1 < suggested update operation key=HBLEXONKXJTL tag=4BZCEEQJZXOH type=del size=1497064 2024-04-07T16:37:44.101 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.109+0000 7f346a3868c0 1 > prepared operation key=UIW3DN4NUYPP tag=5QFUF363B464 type=del size=285570 2024-04-07T16:37:44.102 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.109+0000 7f346a3868c0 1 < canceled part key=_multipart_HBLEXONKXJTL.2~2VF73K8YZM8U.0 size=1048576 2024-04-07T16:37:44.105 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.113+0000 7f346a3868c0 1 < completed part key=_multipart_HBLEXONKXJTL.2~2VF73K8YZM8U.1 size=1048576 2024-04-07T16:37:44.108 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.117+0000 7f346a3868c0 1 < completed part key=_multipart_HBLEXONKXJTL.2~2VF73K8YZM8U.2 size=121499 2024-04-07T16:37:44.114 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.121+0000 7f346a3868c0 1 < canceled multipart upload key=HBLEXONKXJTL upload=2VF73K8YZM8U tag=ZCPYK7F4RHMZ type=write size=3267227 2024-04-07T16:37:44.121 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.129+0000 7f346a3868c0 1 > prepared part key=_multipart_95YVKJ14JD8W.2~6PCDM3F3WES9.0 size=252621 2024-04-07T16:37:44.122 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.129+0000 7f346a3868c0 1 > prepared multipart upload key=95YVKJ14JD8W upload=6PCDM3F3WES9 tag=YWXDJ6FD9V23 type=write size=1301197 2024-04-07T16:37:44.137 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.145+0000 7f346a3868c0 1 < completed write operation key=6XDS0A7L0IY8 tag=K7HORVS3EF5A type=write size=550363 2024-04-07T16:37:44.145 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.153+0000 7f346a3868c0 1 > prepared part key=_multipart_7G5C6HBK6QWI.2~MXAN35ZJ09SA.0 size=419277 2024-04-07T16:37:44.145 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.153+0000 7f346a3868c0 1 > prepared multipart upload key=7G5C6HBK6QWI upload=MXAN35ZJ09SA tag=U1G63K1ND1SL type=write size=1467853 2024-04-07T16:37:44.149 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.157+0000 7f346a3868c0 1 < completed delete operation key=E84U0SX44NQ8 tag=YR1AVRYLA9AE type=del 2024-04-07T16:37:44.154 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.161+0000 7f346a3868c0 1 > prepared operation key=X5Q46GO8YTOK tag=RIVSA5HT04LF type=del size=2537320 2024-04-07T16:37:44.157 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.165+0000 7f346a3868c0 1 < completed delete operation key=IX3F0IUOKBX1 tag=N13H2V4M4KNJ type=del 2024-04-07T16:37:44.161 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.169+0000 7f346a3868c0 1 > prepared operation key=C1BTVJNHIM0X tag=V63COZ7AKJFQ type=del size=1545888 2024-04-07T16:37:44.164 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.173+0000 7f346a3868c0 1 < completed part key=_multipart_95YVKJ14JD8W.2~34UH6UDUD62M.0 size=1048576 2024-04-07T16:37:44.167 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.177+0000 7f346a3868c0 1 < completed part key=_multipart_95YVKJ14JD8W.2~34UH6UDUD62M.1 size=197499 2024-04-07T16:37:44.173 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.181+0000 7f346a3868c0 1 < completed multipart upload key=95YVKJ14JD8W upload=34UH6UDUD62M tag=ICY2R9OCKIFA type=write size=2294651 2024-04-07T16:37:44.179 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.189+0000 7f346a3868c0 1 > prepared part key=_multipart_CL4PP1RKZU0E.2~ZC0K9R5W5678.0 size=518778 2024-04-07T16:37:44.179 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.189+0000 7f346a3868c0 1 > prepared multipart upload key=CL4PP1RKZU0E upload=ZC0K9R5W5678 tag=ETQLZA8262F3 type=write size=1567354 2024-04-07T16:37:44.186 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.193+0000 7f346a3868c0 1 < suggested remove operation key=X5Q46GO8YTOK tag=CS07HW6FYZLB type=del 2024-04-07T16:37:44.190 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.197+0000 7f346a3868c0 1 > prepared operation key=UIW3DN4NUYPP tag=CE8H3R8L67U3 type=del size=3214189 2024-04-07T16:37:44.194 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.201+0000 7f346a3868c0 1 < completed part key=_multipart_LXOZ0T2KY6OR.2~TJ0M5IBOHDZR.0 size=551628 2024-04-07T16:37:44.200 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.209+0000 7f346a3868c0 1 < completed multipart upload key=LXOZ0T2KY6OR upload=TJ0M5IBOHDZR tag=AXED98ORSPWT type=write size=1600204 2024-04-07T16:37:44.203 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.213+0000 7f346a3868c0 1 > prepared operation key=QVOKSQAGHPA9 tag=JKU6JA2JX1EK type=del size=2711987 2024-04-07T16:37:44.207 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.217+0000 7f346a3868c0 1 < completed delete operation key=GZXHYZO5SMUP tag=S3XFUMUBEAG4 type=del 2024-04-07T16:37:44.211 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.217+0000 7f346a3868c0 1 > prepared operation key=E84U0SX44NQ8 tag=L6MYKV32P4P7 type=del size=247992 2024-04-07T16:37:44.215 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.221+0000 7f346a3868c0 1 < completed delete operation key=1VKVQXSQMTU1 tag=55C4I0U68QKX type=del 2024-04-07T16:37:44.219 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.225+0000 7f346a3868c0 1 > prepared operation key=CL4PP1RKZU0E tag=U9GKMLKAYFSC type=del size=2250671 2024-04-07T16:37:44.223 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.229+0000 7f346a3868c0 1 < canceled operation key=DXTO3AJ429AC tag=3VESRG0TS31X type=del size=2056214 2024-04-07T16:37:44.227 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.237+0000 7f346a3868c0 1 > prepared operation key=95YVKJ14JD8W tag=Z89PIJEJ1ZNB type=write size=572466 2024-04-07T16:37:44.231 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.241+0000 7f346a3868c0 1 < completed delete operation key=EQR0FY86AT5X tag=QK8ZBL8L27PY type=del 2024-04-07T16:37:44.238 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.245+0000 7f346a3868c0 1 > prepared part key=_multipart_ROFH30W3UCAX.2~7U162GXP5CJH.0 size=436884 2024-04-07T16:37:44.238 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.245+0000 7f346a3868c0 1 > prepared multipart upload key=ROFH30W3UCAX upload=7U162GXP5CJH tag=2R52Z47PACBE type=write size=1485460 2024-04-07T16:37:44.242 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.249+0000 7f346a3868c0 1 < completed write operation key=8F81G0P0YBFW tag=1YIMBL4YXULF type=write size=350067 2024-04-07T16:37:44.247 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.253+0000 7f346a3868c0 1 > prepared operation key=95YVKJ14JD8W tag=QMJU474COTM0 type=del size=1137979 2024-04-07T16:37:44.250 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.257+0000 7f346a3868c0 1 < canceled operation key=E84U0SX44NQ8 tag=SGJGORP7FJDI type=del size=1783059 2024-04-07T16:37:44.254 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.261+0000 7f346a3868c0 1 > prepared operation key=U3EID5EK64HD tag=NHS5OLRLGXOF type=del size=3737689 2024-04-07T16:37:44.258 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.265+0000 7f346a3868c0 1 < canceled operation key=EOYFVEI50SQ1 tag=JGS1K0670KSM type=write size=255616 2024-04-07T16:37:44.266 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.273+0000 7f346a3868c0 1 > prepared part key=_multipart_QTCIIPTXIRW2.2~H63988GMYJAD.0 size=856701 2024-04-07T16:37:44.266 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.273+0000 7f346a3868c0 1 > prepared multipart upload key=QTCIIPTXIRW2 upload=H63988GMYJAD tag=ICHDN41QZYG3 type=write size=1905277 2024-04-07T16:37:44.269 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.277+0000 7f346a3868c0 1 < completed delete operation key=1VBSUWNI3V1T tag=3W3PSHKNWAVG type=del 2024-04-07T16:37:44.276 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.285+0000 7f346a3868c0 1 > prepared part key=_multipart_C1BTVJNHIM0X.2~I6D5U0QPR76Z.0 size=1048576 2024-04-07T16:37:44.279 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.285+0000 7f346a3868c0 1 > prepared part key=_multipart_C1BTVJNHIM0X.2~I6D5U0QPR76Z.1 size=1048576 2024-04-07T16:37:44.282 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.289+0000 7f346a3868c0 1 > prepared part key=_multipart_C1BTVJNHIM0X.2~I6D5U0QPR76Z.2 size=546708 2024-04-07T16:37:44.282 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.289+0000 7f346a3868c0 1 > prepared multipart upload key=C1BTVJNHIM0X upload=I6D5U0QPR76Z tag=95VVFKLF058P type=write size=3692436 2024-04-07T16:37:44.286 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.293+0000 7f346a3868c0 1 < canceled operation key=UIW3DN4NUYPP tag=5QFUF363B464 type=del size=285570 2024-04-07T16:37:44.293 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.301+0000 7f346a3868c0 1 > prepared part key=_multipart_7G5C6HBK6QWI.2~WR5IW42YV3FV.0 size=1048576 2024-04-07T16:37:44.296 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.305+0000 7f346a3868c0 1 > prepared part key=_multipart_7G5C6HBK6QWI.2~WR5IW42YV3FV.1 size=579434 2024-04-07T16:37:44.296 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.305+0000 7f346a3868c0 1 > prepared multipart upload key=7G5C6HBK6QWI upload=WR5IW42YV3FV tag=TW9GKQPHQY3T type=write size=2676586 2024-04-07T16:37:44.300 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.309+0000 7f346a3868c0 1 < completed part key=_multipart_95YVKJ14JD8W.2~6PCDM3F3WES9.0 size=252621 2024-04-07T16:37:44.306 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.313+0000 7f346a3868c0 1 < completed multipart upload key=95YVKJ14JD8W upload=6PCDM3F3WES9 tag=YWXDJ6FD9V23 type=write size=1301197 2024-04-07T16:37:44.310 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.317+0000 7f346a3868c0 1 > prepared operation key=ROFH30W3UCAX tag=EQXJYC7QFKHH type=del size=2488789 2024-04-07T16:37:44.314 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.321+0000 7f346a3868c0 1 < completed part key=_multipart_7G5C6HBK6QWI.2~MXAN35ZJ09SA.0 size=419277 2024-04-07T16:37:44.320 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.329+0000 7f346a3868c0 1 < completed multipart upload key=7G5C6HBK6QWI upload=MXAN35ZJ09SA tag=U1G63K1ND1SL type=write size=1467853 2024-04-07T16:37:44.324 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.333+0000 7f346a3868c0 1 > prepared operation key=YXMDZB23YTB3 tag=9X4WIEZPR414 type=del size=1867974 2024-04-07T16:37:44.328 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.337+0000 7f346a3868c0 -1 < failed to complete operation key=X5Q46GO8YTOK tag=RIVSA5HT04LF type=del size=2537320: (22) Invalid argument 2024-04-07T16:37:44.332 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.341+0000 7f346a3868c0 1 > prepared operation key=YXMDZB23YTB3 tag=S2CPQTWGLIE3 type=del size=4161358 2024-04-07T16:37:44.336 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.345+0000 7f346a3868c0 1 < completed delete operation key=C1BTVJNHIM0X tag=V63COZ7AKJFQ type=del 2024-04-07T16:37:44.340 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.349+0000 7f346a3868c0 1 > prepared operation key=E84U0SX44NQ8 tag=EYQIA44GI7NN type=del size=3872456 2024-04-07T16:37:44.341 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.349+0000 7f346a3868c0 1 < canceled part key=_multipart_CL4PP1RKZU0E.2~ZC0K9R5W5678.0 size=518778 2024-04-07T16:37:44.347 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.353+0000 7f346a3868c0 1 < canceled multipart upload key=CL4PP1RKZU0E upload=ZC0K9R5W5678 tag=ETQLZA8262F3 type=write size=1567354 2024-04-07T16:37:44.354 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.361+0000 7f346a3868c0 1 > prepared part key=_multipart_HBLEXONKXJTL.2~IEE2QHV1IXSW.0 size=883795 2024-04-07T16:37:44.354 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.361+0000 7f346a3868c0 1 > prepared multipart upload key=HBLEXONKXJTL upload=IEE2QHV1IXSW tag=Q728XQKTLU44 type=write size=1932371 2024-04-07T16:37:44.358 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.365+0000 7f346a3868c0 1 < canceled operation key=UIW3DN4NUYPP tag=CE8H3R8L67U3 type=del size=3214189 2024-04-07T16:37:44.362 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.369+0000 7f346a3868c0 1 > prepared operation key=IP8YW2A75TCW tag=HWLVLZRQE51Y type=del size=2024423 2024-04-07T16:37:44.370 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.377+0000 7f346a3868c0 1 < suggested remove operation key=QVOKSQAGHPA9 tag=JKU6JA2JX1EK type=del 2024-04-07T16:37:44.373 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.381+0000 7f346a3868c0 1 > prepared operation key=1VKVQXSQMTU1 tag=4CKVFKDU43C0 type=del size=2801308 2024-04-07T16:37:44.381 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.389+0000 7f346a3868c0 1 < suggested update operation key=E84U0SX44NQ8 tag=L6MYKV32P4P7 type=del size=247992 2024-04-07T16:37:44.388 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.397+0000 7f346a3868c0 1 > prepared part key=_multipart_8F81G0P0YBFW.2~7ALZNTY69S69.0 size=1048576 2024-04-07T16:37:44.392 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.401+0000 7f346a3868c0 1 > prepared part key=_multipart_8F81G0P0YBFW.2~7ALZNTY69S69.1 size=1048576 2024-04-07T16:37:44.395 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.401+0000 7f346a3868c0 1 > prepared part key=_multipart_8F81G0P0YBFW.2~7ALZNTY69S69.2 size=613296 2024-04-07T16:37:44.395 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.401+0000 7f346a3868c0 1 > prepared multipart upload key=8F81G0P0YBFW upload=7ALZNTY69S69 tag=61FLQMU25WNR type=write size=3759024 2024-04-07T16:37:44.398 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.405+0000 7f346a3868c0 1 < completed delete operation key=CL4PP1RKZU0E tag=U9GKMLKAYFSC type=del 2024-04-07T16:37:44.402 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.409+0000 7f346a3868c0 1 > prepared operation key=NYET3212USVE tag=VVVWYHLEGEIN type=write size=487624 2024-04-07T16:37:44.410 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.417+0000 7f346a3868c0 1 < suggested update operation key=95YVKJ14JD8W tag=Z89PIJEJ1ZNB type=write size=572466 2024-04-07T16:37:44.414 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.421+0000 7f346a3868c0 1 > prepared operation key=LXOZ0T2KY6OR tag=5YO4RBS694O1 type=del size=1611242 2024-04-07T16:37:44.418 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.425+0000 7f346a3868c0 1 < completed part key=_multipart_ROFH30W3UCAX.2~7U162GXP5CJH.0 size=436884 2024-04-07T16:37:44.425 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.433+0000 7f346a3868c0 1 < completed multipart upload key=ROFH30W3UCAX upload=7U162GXP5CJH tag=2R52Z47PACBE type=write size=1485460 2024-04-07T16:37:44.429 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.437+0000 7f346a3868c0 1 > prepared operation key=LXOZ0T2KY6OR tag=NOZF21QP6MYO type=del size=1460938 2024-04-07T16:37:44.434 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.441+0000 7f346a3868c0 -1 < failed to complete operation key=95YVKJ14JD8W tag=QMJU474COTM0 type=del size=1137979: (22) Invalid argument 2024-04-07T16:37:44.437 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.445+0000 7f346a3868c0 1 > prepared operation key=X5Q46GO8YTOK tag=2HESVNC4LKN2 type=del size=1773602 2024-04-07T16:37:44.445 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.453+0000 7f346a3868c0 1 < suggested update operation key=U3EID5EK64HD tag=NHS5OLRLGXOF type=del size=3737689 2024-04-07T16:37:44.450 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.457+0000 7f346a3868c0 1 > prepared operation key=1VBSUWNI3V1T tag=LDXJN5S63XVF type=del size=1537270 2024-04-07T16:37:44.454 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.461+0000 7f346a3868c0 1 < completed part key=_multipart_QTCIIPTXIRW2.2~H63988GMYJAD.0 size=856701 2024-04-07T16:37:44.460 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.469+0000 7f346a3868c0 1 < completed multipart upload key=QTCIIPTXIRW2 upload=H63988GMYJAD tag=ICHDN41QZYG3 type=write size=1905277 2024-04-07T16:37:44.463 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.473+0000 7f346a3868c0 1 > prepared operation key=N14BUXLJONHZ tag=3XWQ3298UEA9 type=write size=416154 2024-04-07T16:37:44.467 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.477+0000 7f346a3868c0 1 < completed part key=_multipart_C1BTVJNHIM0X.2~I6D5U0QPR76Z.0 size=1048576 2024-04-07T16:37:44.470 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.477+0000 7f346a3868c0 1 < completed part key=_multipart_C1BTVJNHIM0X.2~I6D5U0QPR76Z.1 size=1048576 2024-04-07T16:37:44.474 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.481+0000 7f346a3868c0 1 < completed part key=_multipart_C1BTVJNHIM0X.2~I6D5U0QPR76Z.2 size=546708 2024-04-07T16:37:44.481 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.489+0000 7f346a3868c0 1 < completed multipart upload key=C1BTVJNHIM0X upload=I6D5U0QPR76Z tag=95VVFKLF058P type=write size=3692436 2024-04-07T16:37:44.487 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.497+0000 7f346a3868c0 1 > prepared part key=_multipart_EQR0FY86AT5X.2~RC7Q566UYB66.0 size=165979 2024-04-07T16:37:44.488 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.497+0000 7f346a3868c0 1 > prepared multipart upload key=EQR0FY86AT5X upload=RC7Q566UYB66 tag=BOOSORWWO1O5 type=write size=1214555 2024-04-07T16:37:44.492 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.501+0000 7f346a3868c0 1 < completed part key=_multipart_7G5C6HBK6QWI.2~WR5IW42YV3FV.0 size=1048576 2024-04-07T16:37:44.495 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.501+0000 7f346a3868c0 1 < completed part key=_multipart_7G5C6HBK6QWI.2~WR5IW42YV3FV.1 size=579434 2024-04-07T16:37:44.501 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.509+0000 7f346a3868c0 1 < completed multipart upload key=7G5C6HBK6QWI upload=WR5IW42YV3FV tag=TW9GKQPHQY3T type=write size=2676586 2024-04-07T16:37:44.508 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.517+0000 7f346a3868c0 1 > prepared part key=_multipart_IX3F0IUOKBX1.2~3KHDC43JKSOU.0 size=1048576 2024-04-07T16:37:44.511 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.521+0000 7f346a3868c0 1 > prepared part key=_multipart_IX3F0IUOKBX1.2~3KHDC43JKSOU.1 size=161982 2024-04-07T16:37:44.511 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.521+0000 7f346a3868c0 1 > prepared multipart upload key=IX3F0IUOKBX1 upload=3KHDC43JKSOU tag=E9YD16BH8P8P type=write size=2259134 2024-04-07T16:37:44.516 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.525+0000 7f346a3868c0 1 < completed delete operation key=ROFH30W3UCAX tag=EQXJYC7QFKHH type=del 2024-04-07T16:37:44.520 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.529+0000 7f346a3868c0 1 > prepared operation key=ROFH30W3UCAX tag=VO4LFPSVVJ52 type=del size=1132897 2024-04-07T16:37:44.524 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.533+0000 7f346a3868c0 1 < completed delete operation key=YXMDZB23YTB3 tag=9X4WIEZPR414 type=del 2024-04-07T16:37:44.528 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.537+0000 7f346a3868c0 1 > prepared operation key=1VKVQXSQMTU1 tag=9TZFYO6WMPBA type=del size=2115784 2024-04-07T16:37:44.532 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.541+0000 7f346a3868c0 1 < completed delete operation key=YXMDZB23YTB3 tag=S2CPQTWGLIE3 type=del 2024-04-07T16:37:44.537 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.545+0000 7f346a3868c0 1 > prepared operation key=QTCIIPTXIRW2 tag=JKB3FI4XRIHR type=del size=2094522 2024-04-07T16:37:44.541 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.549+0000 7f346a3868c0 -1 < failed to complete operation key=E84U0SX44NQ8 tag=EYQIA44GI7NN type=del size=3872456: (22) Invalid argument 2024-04-07T16:37:44.544 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.553+0000 7f346a3868c0 1 > prepared operation key=EOYFVEI50SQ1 tag=QZPRKHFDJKXG type=del size=3620899 2024-04-07T16:37:44.549 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.557+0000 7f346a3868c0 1 < completed part key=_multipart_HBLEXONKXJTL.2~IEE2QHV1IXSW.0 size=883795 2024-04-07T16:37:44.555 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.565+0000 7f346a3868c0 1 < completed multipart upload key=HBLEXONKXJTL upload=IEE2QHV1IXSW tag=Q728XQKTLU44 type=write size=1932371 2024-04-07T16:37:44.559 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.569+0000 7f346a3868c0 1 > prepared operation key=CL4PP1RKZU0E tag=9OYJNIFBQFL9 type=write size=199044 2024-04-07T16:37:44.563 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.573+0000 7f346a3868c0 1 < canceled operation key=IP8YW2A75TCW tag=HWLVLZRQE51Y type=del size=2024423 2024-04-07T16:37:44.567 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.577+0000 7f346a3868c0 1 > prepared operation key=QTCIIPTXIRW2 tag=VFNU3O79DW3C type=write size=527656 2024-04-07T16:37:44.575 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.585+0000 7f346a3868c0 1 < suggested remove operation key=1VKVQXSQMTU1 tag=4CKVFKDU43C0 type=del 2024-04-07T16:37:44.579 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.589+0000 7f346a3868c0 1 > prepared operation key=DXTO3AJ429AC tag=WK9ME6O2KFNW type=del size=2950374 2024-04-07T16:37:44.583 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.593+0000 7f346a3868c0 1 < completed part key=_multipart_8F81G0P0YBFW.2~7ALZNTY69S69.0 size=1048576 2024-04-07T16:37:44.586 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.593+0000 7f346a3868c0 1 < completed part key=_multipart_8F81G0P0YBFW.2~7ALZNTY69S69.1 size=1048576 2024-04-07T16:37:44.589 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.597+0000 7f346a3868c0 1 < completed part key=_multipart_8F81G0P0YBFW.2~7ALZNTY69S69.2 size=613296 2024-04-07T16:37:44.596 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.605+0000 7f346a3868c0 1 < completed multipart upload key=8F81G0P0YBFW upload=7ALZNTY69S69 tag=61FLQMU25WNR type=write size=3759024 2024-04-07T16:37:44.603 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.609+0000 7f346a3868c0 1 > prepared part key=_multipart_ROFH30W3UCAX.2~VS02OM4JK0O0.0 size=1048576 2024-04-07T16:37:44.605 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.613+0000 7f346a3868c0 1 > prepared part key=_multipart_ROFH30W3UCAX.2~VS02OM4JK0O0.1 size=1048576 2024-04-07T16:37:44.608 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.617+0000 7f346a3868c0 1 > prepared part key=_multipart_ROFH30W3UCAX.2~VS02OM4JK0O0.2 size=26872 2024-04-07T16:37:44.608 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.617+0000 7f346a3868c0 1 > prepared multipart upload key=ROFH30W3UCAX upload=VS02OM4JK0O0 tag=VFAW3AYKMTEH type=write size=3172600 2024-04-07T16:37:44.612 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.621+0000 7f346a3868c0 1 < canceled operation key=NYET3212USVE tag=VVVWYHLEGEIN type=write size=487624 2024-04-07T16:37:44.620 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.629+0000 7f346a3868c0 1 > prepared part key=_multipart_U3EID5EK64HD.2~NW7G5I1DJNZQ.0 size=1048576 2024-04-07T16:37:44.624 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.633+0000 7f346a3868c0 1 > prepared part key=_multipart_U3EID5EK64HD.2~NW7G5I1DJNZQ.1 size=538892 2024-04-07T16:37:44.624 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.633+0000 7f346a3868c0 1 > prepared multipart upload key=U3EID5EK64HD upload=NW7G5I1DJNZQ tag=M2V6ADRNKDF8 type=write size=2636044 2024-04-07T16:37:44.633 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.641+0000 7f346a3868c0 1 < suggested remove operation key=LXOZ0T2KY6OR tag=5YO4RBS694O1 type=del 2024-04-07T16:37:44.638 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.645+0000 7f346a3868c0 1 > prepared operation key=UIW3DN4NUYPP tag=26RD79VSZKS8 type=del size=1289152 2024-04-07T16:37:44.642 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.649+0000 7f346a3868c0 -1 < failed to complete operation key=LXOZ0T2KY6OR tag=NOZF21QP6MYO type=del size=1460938: (22) Invalid argument 2024-04-07T16:37:44.651 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.661+0000 7f346a3868c0 1 > prepared part key=_multipart_IX3F0IUOKBX1.2~MFT2CH5X2OAE.0 size=1048576 2024-04-07T16:37:44.654 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.661+0000 7f346a3868c0 1 > prepared part key=_multipart_IX3F0IUOKBX1.2~MFT2CH5X2OAE.1 size=1048576 2024-04-07T16:37:44.657 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.665+0000 7f346a3868c0 1 > prepared part key=_multipart_IX3F0IUOKBX1.2~MFT2CH5X2OAE.2 size=324599 2024-04-07T16:37:44.657 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.665+0000 7f346a3868c0 1 > prepared multipart upload key=IX3F0IUOKBX1 upload=MFT2CH5X2OAE tag=HF4BFE7MDQW5 type=write size=3470327 2024-04-07T16:37:44.661 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.669+0000 7f346a3868c0 1 < completed delete operation key=X5Q46GO8YTOK tag=2HESVNC4LKN2 type=del 2024-04-07T16:37:44.665 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.673+0000 7f346a3868c0 1 > prepared operation key=YXMDZB23YTB3 tag=QBG1R6WBED3S type=del size=101749 2024-04-07T16:37:44.669 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.677+0000 7f346a3868c0 1 < completed delete operation key=1VBSUWNI3V1T tag=LDXJN5S63XVF type=del 2024-04-07T16:37:44.674 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.681+0000 7f346a3868c0 1 > prepared operation key=IP8YW2A75TCW tag=795MEBLUKK6V type=del size=42497 2024-04-07T16:37:44.678 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.685+0000 7f346a3868c0 1 < completed write operation key=N14BUXLJONHZ tag=3XWQ3298UEA9 type=write size=416154 2024-04-07T16:37:44.686 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.693+0000 7f346a3868c0 1 > prepared part key=_multipart_GBQSME7SY1AH.2~252KG39PR4K0.0 size=1048576 2024-04-07T16:37:44.689 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.697+0000 7f346a3868c0 1 > prepared part key=_multipart_GBQSME7SY1AH.2~252KG39PR4K0.1 size=815555 2024-04-07T16:37:44.689 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.697+0000 7f346a3868c0 1 > prepared multipart upload key=GBQSME7SY1AH upload=252KG39PR4K0 tag=YXUP02IVF3IC type=write size=2912707 2024-04-07T16:37:44.693 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.701+0000 7f346a3868c0 1 < completed part key=_multipart_EQR0FY86AT5X.2~RC7Q566UYB66.0 size=165979 2024-04-07T16:37:44.700 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.709+0000 7f346a3868c0 1 < completed multipart upload key=EQR0FY86AT5X upload=RC7Q566UYB66 tag=BOOSORWWO1O5 type=write size=1214555 2024-04-07T16:37:44.707 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.717+0000 7f346a3868c0 1 > prepared part key=_multipart_N14BUXLJONHZ.2~7YIU02P3KLWH.0 size=1048576 2024-04-07T16:37:44.711 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.717+0000 7f346a3868c0 1 > prepared part key=_multipart_N14BUXLJONHZ.2~7YIU02P3KLWH.1 size=841935 2024-04-07T16:37:44.711 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.717+0000 7f346a3868c0 1 > prepared multipart upload key=N14BUXLJONHZ upload=7YIU02P3KLWH tag=Y655CWDW62SC type=write size=2939087 2024-04-07T16:37:44.715 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.725+0000 7f346a3868c0 1 < completed part key=_multipart_IX3F0IUOKBX1.2~3KHDC43JKSOU.0 size=1048576 2024-04-07T16:37:44.718 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.725+0000 7f346a3868c0 1 < completed part key=_multipart_IX3F0IUOKBX1.2~3KHDC43JKSOU.1 size=161982 2024-04-07T16:37:44.724 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.733+0000 7f346a3868c0 1 < completed multipart upload key=IX3F0IUOKBX1 upload=3KHDC43JKSOU tag=E9YD16BH8P8P type=write size=2259134 2024-04-07T16:37:44.728 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.737+0000 7f346a3868c0 1 > prepared operation key=C1BTVJNHIM0X tag=2IW5Y77CHD1B type=del size=2162060 2024-04-07T16:37:44.732 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.741+0000 7f346a3868c0 1 < completed delete operation key=ROFH30W3UCAX tag=VO4LFPSVVJ52 type=del 2024-04-07T16:37:44.740 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.749+0000 7f346a3868c0 1 > prepared part key=_multipart_QS4HYZFFAJNF.2~CM6X38CUCN8P.0 size=66897 2024-04-07T16:37:44.740 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.749+0000 7f346a3868c0 1 > prepared multipart upload key=QS4HYZFFAJNF upload=CM6X38CUCN8P tag=V6KKDRX53EHP type=write size=1115473 2024-04-07T16:37:44.744 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.753+0000 7f346a3868c0 -1 < failed to complete operation key=1VKVQXSQMTU1 tag=9TZFYO6WMPBA type=del size=2115784: (22) Invalid argument 2024-04-07T16:37:44.748 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.757+0000 7f346a3868c0 1 > prepared operation key=8F81G0P0YBFW tag=JAS7E8WQ7HAI type=del size=3194574 2024-04-07T16:37:44.752 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.761+0000 7f346a3868c0 1 < completed delete operation key=QTCIIPTXIRW2 tag=JKB3FI4XRIHR type=del 2024-04-07T16:37:44.755 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.765+0000 7f346a3868c0 1 > prepared operation key=6XDS0A7L0IY8 tag=7DRALEOQRXQE type=del size=3397436 2024-04-07T16:37:44.763 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.773+0000 7f346a3868c0 1 < suggested remove operation key=EOYFVEI50SQ1 tag=QZPRKHFDJKXG type=del 2024-04-07T16:37:44.770 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.777+0000 7f346a3868c0 1 > prepared part key=_multipart_6XDS0A7L0IY8.2~YBVUFW79PQEC.0 size=1048576 2024-04-07T16:37:44.773 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.781+0000 7f346a3868c0 1 > prepared part key=_multipart_6XDS0A7L0IY8.2~YBVUFW79PQEC.1 size=60574 2024-04-07T16:37:44.773 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.781+0000 7f346a3868c0 1 > prepared multipart upload key=6XDS0A7L0IY8 upload=YBVUFW79PQEC tag=EVI4009JJPYB type=write size=2157726 2024-04-07T16:37:44.778 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.785+0000 7f346a3868c0 1 < completed write operation key=CL4PP1RKZU0E tag=9OYJNIFBQFL9 type=write size=199044 2024-04-07T16:37:44.785 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.793+0000 7f346a3868c0 1 > prepared part key=_multipart_C1BTVJNHIM0X.2~WA7H0SQ5RMCQ.0 size=117104 2024-04-07T16:37:44.785 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.793+0000 7f346a3868c0 1 > prepared multipart upload key=C1BTVJNHIM0X upload=WA7H0SQ5RMCQ tag=SFIB87S6W6AG type=write size=1165680 2024-04-07T16:37:44.789 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.797+0000 7f346a3868c0 1 < completed write operation key=QTCIIPTXIRW2 tag=VFNU3O79DW3C type=write size=527656 2024-04-07T16:37:44.794 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.801+0000 7f346a3868c0 1 > prepared operation key=YXMDZB23YTB3 tag=1BEXYDGMCPVV type=write size=166344 2024-04-07T16:37:44.801 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.809+0000 7f346a3868c0 1 < suggested remove operation key=DXTO3AJ429AC tag=WK9ME6O2KFNW type=del 2024-04-07T16:37:44.805 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.813+0000 7f346a3868c0 1 > prepared operation key=1VKVQXSQMTU1 tag=8L89JP0D3UFU type=del size=4080659 2024-04-07T16:37:44.809 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.817+0000 7f346a3868c0 1 < completed part key=_multipart_ROFH30W3UCAX.2~VS02OM4JK0O0.0 size=1048576 2024-04-07T16:37:44.812 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.821+0000 7f346a3868c0 1 < completed part key=_multipart_ROFH30W3UCAX.2~VS02OM4JK0O0.1 size=1048576 2024-04-07T16:37:44.815 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.825+0000 7f346a3868c0 1 < completed part key=_multipart_ROFH30W3UCAX.2~VS02OM4JK0O0.2 size=26872 2024-04-07T16:37:44.822 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.829+0000 7f346a3868c0 1 < completed multipart upload key=ROFH30W3UCAX upload=VS02OM4JK0O0 tag=VFAW3AYKMTEH type=write size=3172600 2024-04-07T16:37:44.826 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.833+0000 7f346a3868c0 1 > prepared operation key=EQR0FY86AT5X tag=265U4ICPHDU7 type=del size=2239896 2024-04-07T16:37:44.830 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.837+0000 7f346a3868c0 1 < completed part key=_multipart_U3EID5EK64HD.2~NW7G5I1DJNZQ.0 size=1048576 2024-04-07T16:37:44.833 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.841+0000 7f346a3868c0 1 < completed part key=_multipart_U3EID5EK64HD.2~NW7G5I1DJNZQ.1 size=538892 2024-04-07T16:37:44.840 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.849+0000 7f346a3868c0 1 < completed multipart upload key=U3EID5EK64HD upload=NW7G5I1DJNZQ tag=M2V6ADRNKDF8 type=write size=2636044 2024-04-07T16:37:44.843 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.853+0000 7f346a3868c0 1 > prepared operation key=X5Q46GO8YTOK tag=CZ2J03NT0T6M type=del size=2623730 2024-04-07T16:37:44.847 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.857+0000 7f346a3868c0 1 < canceled operation key=UIW3DN4NUYPP tag=26RD79VSZKS8 type=del size=1289152 2024-04-07T16:37:44.854 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.861+0000 7f346a3868c0 1 > prepared part key=_multipart_6XDS0A7L0IY8.2~CPBW58WM4NW9.0 size=1048576 2024-04-07T16:37:44.856 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:37:44 smithi144 bash[66654]: cluster 2024-04-07T16:37:43.055135+0000 mgr.y (mgr.25175) 671 : cluster [DBG] pgmap v1306: 137 pgs: 11 unknown, 126 active+clean; 583 KiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 151 KiB/s rd, 52 KiB/s wr, 268 op/s 2024-04-07T16:37:44.856 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:37:44 smithi144 bash[66654]: cluster 2024-04-07T16:37:43.481188+0000 mon.a (mon.0) 2019 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T16:37:44.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:37:44 smithi144 bash[66654]: cluster 2024-04-07T16:37:43.481250+0000 mon.a (mon.0) 2020 : cluster [INF] Cluster is now healthy 2024-04-07T16:37:44.858 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.865+0000 7f346a3868c0 1 > prepared part key=_multipart_6XDS0A7L0IY8.2~CPBW58WM4NW9.1 size=197582 2024-04-07T16:37:44.858 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.865+0000 7f346a3868c0 1 > prepared multipart upload key=6XDS0A7L0IY8 upload=CPBW58WM4NW9 tag=X1VMLDJWJXEK type=write size=2294734 2024-04-07T16:37:44.862 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.869+0000 7f346a3868c0 1 < completed part key=_multipart_IX3F0IUOKBX1.2~MFT2CH5X2OAE.0 size=1048576 2024-04-07T16:37:44.865 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.873+0000 7f346a3868c0 1 < completed part key=_multipart_IX3F0IUOKBX1.2~MFT2CH5X2OAE.1 size=1048576 2024-04-07T16:37:44.868 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.877+0000 7f346a3868c0 1 < completed part key=_multipart_IX3F0IUOKBX1.2~MFT2CH5X2OAE.2 size=324599 2024-04-07T16:37:44.874 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.881+0000 7f346a3868c0 1 < completed multipart upload key=IX3F0IUOKBX1 upload=MFT2CH5X2OAE tag=HF4BFE7MDQW5 type=write size=3470327 2024-04-07T16:37:44.878 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.885+0000 7f346a3868c0 1 > prepared operation key=U3EID5EK64HD tag=LIDOSFHJ4FPU type=del size=1061502 2024-04-07T16:37:44.886 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.893+0000 7f346a3868c0 1 < suggested remove operation key=YXMDZB23YTB3 tag=QBG1R6WBED3S type=del 2024-04-07T16:37:44.893 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.901+0000 7f346a3868c0 1 > prepared part key=_multipart_NYET3212USVE.2~4C6KOD3XGJ7J.0 size=1048576 2024-04-07T16:37:44.896 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.905+0000 7f346a3868c0 1 > prepared part key=_multipart_NYET3212USVE.2~4C6KOD3XGJ7J.1 size=1048576 2024-04-07T16:37:44.899 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.909+0000 7f346a3868c0 1 > prepared part key=_multipart_NYET3212USVE.2~4C6KOD3XGJ7J.2 size=62063 2024-04-07T16:37:44.899 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.909+0000 7f346a3868c0 1 > prepared multipart upload key=NYET3212USVE upload=4C6KOD3XGJ7J tag=ZU3VO04B1Z62 type=write size=3207791 2024-04-07T16:37:44.903 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.913+0000 7f346a3868c0 1 < completed delete operation key=IP8YW2A75TCW tag=795MEBLUKK6V type=del 2024-04-07T16:37:44.910 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.917+0000 7f346a3868c0 1 > prepared part key=_multipart_QVOKSQAGHPA9.2~81NREBZW118Z.0 size=1048576 2024-04-07T16:37:44.913 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.921+0000 7f346a3868c0 1 > prepared part key=_multipart_QVOKSQAGHPA9.2~81NREBZW118Z.1 size=1048576 2024-04-07T16:37:44.916 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.925+0000 7f346a3868c0 1 > prepared part key=_multipart_QVOKSQAGHPA9.2~81NREBZW118Z.2 size=683393 2024-04-07T16:37:44.916 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.925+0000 7f346a3868c0 1 > prepared multipart upload key=QVOKSQAGHPA9 upload=81NREBZW118Z tag=POH8A6NL1RU8 type=write size=3829121 2024-04-07T16:37:44.920 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.929+0000 7f346a3868c0 1 < completed part key=_multipart_GBQSME7SY1AH.2~252KG39PR4K0.0 size=1048576 2024-04-07T16:37:44.923 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.933+0000 7f346a3868c0 1 < completed part key=_multipart_GBQSME7SY1AH.2~252KG39PR4K0.1 size=815555 2024-04-07T16:37:44.930 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.937+0000 7f346a3868c0 1 < completed multipart upload key=GBQSME7SY1AH upload=252KG39PR4K0 tag=YXUP02IVF3IC type=write size=2912707 2024-04-07T16:37:44.934 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.941+0000 7f346a3868c0 1 > prepared operation key=NYET3212USVE tag=8EEVYLLPZGG1 type=del size=317161 2024-04-07T16:37:44.938 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.945+0000 7f346a3868c0 1 < completed part key=_multipart_N14BUXLJONHZ.2~7YIU02P3KLWH.0 size=1048576 2024-04-07T16:37:44.941 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:37:44 smithi100 bash[74942]: cluster 2024-04-07T16:37:43.055135+0000 mgr.y (mgr.25175) 671 : cluster [DBG] pgmap v1306: 137 pgs: 11 unknown, 126 active+clean; 583 KiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 151 KiB/s rd, 52 KiB/s wr, 268 op/s 2024-04-07T16:37:44.941 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:37:44 smithi100 bash[74942]: cluster 2024-04-07T16:37:43.481188+0000 mon.a (mon.0) 2019 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T16:37:44.941 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:37:44 smithi100 bash[74942]: cluster 2024-04-07T16:37:43.481250+0000 mon.a (mon.0) 2020 : cluster [INF] Cluster is now healthy 2024-04-07T16:37:44.941 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:37:44 smithi100 bash[76202]: cluster 2024-04-07T16:37:43.055135+0000 mgr.y (mgr.25175) 671 : cluster [DBG] pgmap v1306: 137 pgs: 11 unknown, 126 active+clean; 583 KiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 151 KiB/s rd, 52 KiB/s wr, 268 op/s 2024-04-07T16:37:44.941 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:37:44 smithi100 bash[76202]: cluster 2024-04-07T16:37:43.481188+0000 mon.a (mon.0) 2019 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T16:37:44.941 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:37:44 smithi100 bash[76202]: cluster 2024-04-07T16:37:43.481250+0000 mon.a (mon.0) 2020 : cluster [INF] Cluster is now healthy 2024-04-07T16:37:44.942 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.949+0000 7f346a3868c0 1 < completed part key=_multipart_N14BUXLJONHZ.2~7YIU02P3KLWH.1 size=841935 2024-04-07T16:37:44.948 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.957+0000 7f346a3868c0 1 < completed multipart upload key=N14BUXLJONHZ upload=7YIU02P3KLWH tag=Y655CWDW62SC type=write size=2939087 2024-04-07T16:37:44.955 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.965+0000 7f346a3868c0 1 > prepared part key=_multipart_NYET3212USVE.2~T5KYFA2NJD3H.0 size=1048576 2024-04-07T16:37:44.958 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.965+0000 7f346a3868c0 1 > prepared part key=_multipart_NYET3212USVE.2~T5KYFA2NJD3H.1 size=1048576 2024-04-07T16:37:44.961 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.969+0000 7f346a3868c0 1 > prepared part key=_multipart_NYET3212USVE.2~T5KYFA2NJD3H.2 size=81320 2024-04-07T16:37:44.961 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.969+0000 7f346a3868c0 1 > prepared multipart upload key=NYET3212USVE upload=T5KYFA2NJD3H tag=7JHYH2KN220F type=write size=3227048 2024-04-07T16:37:44.965 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.973+0000 7f346a3868c0 1 < completed delete operation key=C1BTVJNHIM0X tag=2IW5Y77CHD1B type=del 2024-04-07T16:37:44.974 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.981+0000 7f346a3868c0 1 > prepared part key=_multipart_YOK8BAEOMPUR.2~8KND6H61905M.0 size=1048576 2024-04-07T16:37:44.977 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.985+0000 7f346a3868c0 1 > prepared part key=_multipart_YOK8BAEOMPUR.2~8KND6H61905M.1 size=1048576 2024-04-07T16:37:44.981 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.989+0000 7f346a3868c0 1 > prepared part key=_multipart_YOK8BAEOMPUR.2~8KND6H61905M.2 size=709779 2024-04-07T16:37:44.981 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.989+0000 7f346a3868c0 1 > prepared multipart upload key=YOK8BAEOMPUR upload=8KND6H61905M tag=XZEEKBB4K6EZ type=write size=3855507 2024-04-07T16:37:44.986 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:44.993+0000 7f346a3868c0 1 < completed part key=_multipart_QS4HYZFFAJNF.2~CM6X38CUCN8P.0 size=66897 2024-04-07T16:37:44.993 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.001+0000 7f346a3868c0 1 < completed multipart upload key=QS4HYZFFAJNF upload=CM6X38CUCN8P tag=V6KKDRX53EHP type=write size=1115473 2024-04-07T16:37:44.997 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.005+0000 7f346a3868c0 1 > prepared operation key=NYET3212USVE tag=TGRS221J7L8P type=del size=185372 2024-04-07T16:37:45.001 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.009+0000 7f346a3868c0 1 < completed delete operation key=8F81G0P0YBFW tag=JAS7E8WQ7HAI type=del 2024-04-07T16:37:45.005 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.013+0000 7f346a3868c0 1 > prepared operation key=XJLQZ7IS9U1T tag=WBQ9N8HVKRBP type=del size=3292793 2024-04-07T16:37:45.010 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.017+0000 7f346a3868c0 1 < completed delete operation key=6XDS0A7L0IY8 tag=7DRALEOQRXQE type=del 2024-04-07T16:37:45.015 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.025+0000 7f346a3868c0 1 > prepared operation key=GZXHYZO5SMUP tag=H58UN7C2NZ33 type=write size=803113 2024-04-07T16:37:45.019 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.029+0000 7f346a3868c0 1 < completed part key=_multipart_6XDS0A7L0IY8.2~YBVUFW79PQEC.0 size=1048576 2024-04-07T16:37:45.023 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.029+0000 7f346a3868c0 1 < completed part key=_multipart_6XDS0A7L0IY8.2~YBVUFW79PQEC.1 size=60574 2024-04-07T16:37:45.029 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.037+0000 7f346a3868c0 1 < completed multipart upload key=6XDS0A7L0IY8 upload=YBVUFW79PQEC tag=EVI4009JJPYB type=write size=2157726 2024-04-07T16:37:45.033 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.041+0000 7f346a3868c0 1 > prepared operation key=LXOZ0T2KY6OR tag=8VFMPU9T7WLY type=write size=376349 2024-04-07T16:37:45.037 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.045+0000 7f346a3868c0 1 < completed part key=_multipart_C1BTVJNHIM0X.2~WA7H0SQ5RMCQ.0 size=117104 2024-04-07T16:37:45.043 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.053+0000 7f346a3868c0 1 < completed multipart upload key=C1BTVJNHIM0X upload=WA7H0SQ5RMCQ tag=SFIB87S6W6AG type=write size=1165680 2024-04-07T16:37:45.052 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.061+0000 7f346a3868c0 1 > prepared part key=_multipart_YOK8BAEOMPUR.2~V162J1TZ33Z4.0 size=711885 2024-04-07T16:37:45.052 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.061+0000 7f346a3868c0 1 > prepared multipart upload key=YOK8BAEOMPUR upload=V162J1TZ33Z4 tag=S1EKYN3J0APF type=write size=1760461 2024-04-07T16:37:45.057 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.065+0000 7f346a3868c0 -1 < failed to complete operation key=YXMDZB23YTB3 tag=1BEXYDGMCPVV type=write size=166344: (22) Invalid argument 2024-04-07T16:37:45.064 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.073+0000 7f346a3868c0 1 > prepared part key=_multipart_IX3F0IUOKBX1.2~HPWUHV910IRK.0 size=1048576 2024-04-07T16:37:45.068 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.077+0000 7f346a3868c0 1 > prepared part key=_multipart_IX3F0IUOKBX1.2~HPWUHV910IRK.1 size=1048576 2024-04-07T16:37:45.071 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.081+0000 7f346a3868c0 1 > prepared part key=_multipart_IX3F0IUOKBX1.2~HPWUHV910IRK.2 size=962495 2024-04-07T16:37:45.071 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.081+0000 7f346a3868c0 1 > prepared multipart upload key=IX3F0IUOKBX1 upload=HPWUHV910IRK tag=B0OC7TPL9PJW type=write size=4108223 2024-04-07T16:37:45.075 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.085+0000 7f346a3868c0 1 < completed delete operation key=1VKVQXSQMTU1 tag=8L89JP0D3UFU type=del 2024-04-07T16:37:45.079 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.089+0000 7f346a3868c0 1 > prepared operation key=3N0QJRYVO3FV tag=MDLL9UFJAAV5 type=write size=685256 2024-04-07T16:37:45.083 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.093+0000 7f346a3868c0 1 < completed delete operation key=EQR0FY86AT5X tag=265U4ICPHDU7 type=del 2024-04-07T16:37:45.090 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.097+0000 7f346a3868c0 1 > prepared part key=_multipart_8F81G0P0YBFW.2~H9A1GFIF03XI.0 size=387856 2024-04-07T16:37:45.090 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.097+0000 7f346a3868c0 1 > prepared multipart upload key=8F81G0P0YBFW upload=H9A1GFIF03XI tag=TPZYYPZ3IOFW type=write size=1436432 2024-04-07T16:37:45.094 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.101+0000 7f346a3868c0 1 < completed delete operation key=X5Q46GO8YTOK tag=CZ2J03NT0T6M type=del 2024-04-07T16:37:45.101 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.109+0000 7f346a3868c0 1 > prepared part key=_multipart_XJLQZ7IS9U1T.2~G42STNCVHEO6.0 size=1048576 2024-04-07T16:37:45.104 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.113+0000 7f346a3868c0 1 > prepared part key=_multipart_XJLQZ7IS9U1T.2~G42STNCVHEO6.1 size=32267 2024-04-07T16:37:45.104 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.113+0000 7f346a3868c0 1 > prepared multipart upload key=XJLQZ7IS9U1T upload=G42STNCVHEO6 tag=FN61NI0M1X2M type=write size=2129419 2024-04-07T16:37:45.108 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.117+0000 7f346a3868c0 1 < completed part key=_multipart_6XDS0A7L0IY8.2~CPBW58WM4NW9.0 size=1048576 2024-04-07T16:37:45.109 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.117+0000 7f346a3868c0 1 < canceled part key=_multipart_6XDS0A7L0IY8.2~CPBW58WM4NW9.1 size=197582 2024-04-07T16:37:45.115 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.125+0000 7f346a3868c0 1 < canceled multipart upload key=6XDS0A7L0IY8 upload=CPBW58WM4NW9 tag=X1VMLDJWJXEK type=write size=2294734 2024-04-07T16:37:45.119 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.129+0000 7f346a3868c0 1 > prepared operation key=QTCIIPTXIRW2 tag=VELRJ432A00H type=del size=3554558 2024-04-07T16:37:45.123 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.133+0000 7f346a3868c0 1 < completed delete operation key=U3EID5EK64HD tag=LIDOSFHJ4FPU type=del 2024-04-07T16:37:45.131 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.141+0000 7f346a3868c0 1 > prepared part key=_multipart_CL4PP1RKZU0E.2~V5527QHNCU0L.0 size=671475 2024-04-07T16:37:45.131 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.141+0000 7f346a3868c0 1 > prepared multipart upload key=CL4PP1RKZU0E upload=V5527QHNCU0L tag=VMKZKJE90QW3 type=write size=1720051 2024-04-07T16:37:45.135 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.145+0000 7f346a3868c0 1 < completed part key=_multipart_NYET3212USVE.2~4C6KOD3XGJ7J.0 size=1048576 2024-04-07T16:37:45.138 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.145+0000 7f346a3868c0 1 < completed part key=_multipart_NYET3212USVE.2~4C6KOD3XGJ7J.1 size=1048576 2024-04-07T16:37:45.141 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.149+0000 7f346a3868c0 1 < completed part key=_multipart_NYET3212USVE.2~4C6KOD3XGJ7J.2 size=62063 2024-04-07T16:37:45.149 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.157+0000 7f346a3868c0 1 < completed multipart upload key=NYET3212USVE upload=4C6KOD3XGJ7J tag=ZU3VO04B1Z62 type=write size=3207791 2024-04-07T16:37:45.156 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.165+0000 7f346a3868c0 1 > prepared part key=_multipart_X5Q46GO8YTOK.2~0OVO93KYGA9U.0 size=1048576 2024-04-07T16:37:45.159 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.169+0000 7f346a3868c0 1 > prepared part key=_multipart_X5Q46GO8YTOK.2~0OVO93KYGA9U.1 size=369992 2024-04-07T16:37:45.159 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.169+0000 7f346a3868c0 1 > prepared multipart upload key=X5Q46GO8YTOK upload=0OVO93KYGA9U tag=4C195OJMZMO8 type=write size=2467144 2024-04-07T16:37:45.163 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.173+0000 7f346a3868c0 1 < completed part key=_multipart_QVOKSQAGHPA9.2~81NREBZW118Z.0 size=1048576 2024-04-07T16:37:45.167 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.173+0000 7f346a3868c0 1 < completed part key=_multipart_QVOKSQAGHPA9.2~81NREBZW118Z.1 size=1048576 2024-04-07T16:37:45.170 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.177+0000 7f346a3868c0 1 < completed part key=_multipart_QVOKSQAGHPA9.2~81NREBZW118Z.2 size=683393 2024-04-07T16:37:45.177 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.185+0000 7f346a3868c0 1 < completed multipart upload key=QVOKSQAGHPA9 upload=81NREBZW118Z tag=POH8A6NL1RU8 type=write size=3829121 2024-04-07T16:37:45.181 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.189+0000 7f346a3868c0 1 > prepared operation key=1VKVQXSQMTU1 tag=PEKO7ALBFWWM type=del size=1308416 2024-04-07T16:37:45.185 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.193+0000 7f346a3868c0 1 < completed delete operation key=NYET3212USVE tag=8EEVYLLPZGG1 type=del 2024-04-07T16:37:45.192 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.201+0000 7f346a3868c0 1 > prepared part key=_multipart_CL4PP1RKZU0E.2~L1WTV2OLXLLM.0 size=1048576 2024-04-07T16:37:45.195 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.201+0000 7f346a3868c0 1 > prepared part key=_multipart_CL4PP1RKZU0E.2~L1WTV2OLXLLM.1 size=785542 2024-04-07T16:37:45.195 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.201+0000 7f346a3868c0 1 > prepared multipart upload key=CL4PP1RKZU0E upload=L1WTV2OLXLLM tag=AWEUZMLFUPFX type=write size=2882694 2024-04-07T16:37:45.199 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.205+0000 7f346a3868c0 1 < completed part key=_multipart_NYET3212USVE.2~T5KYFA2NJD3H.0 size=1048576 2024-04-07T16:37:45.199 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.205+0000 7f346a3868c0 1 < canceled part key=_multipart_NYET3212USVE.2~T5KYFA2NJD3H.1 size=1048576 2024-04-07T16:37:45.199 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.205+0000 7f346a3868c0 1 < canceled part key=_multipart_NYET3212USVE.2~T5KYFA2NJD3H.2 size=81320 2024-04-07T16:37:45.205 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.213+0000 7f346a3868c0 1 < canceled multipart upload key=NYET3212USVE upload=T5KYFA2NJD3H tag=7JHYH2KN220F type=write size=3227048 2024-04-07T16:37:45.209 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.217+0000 7f346a3868c0 1 > prepared operation key=IX3F0IUOKBX1 tag=3DGWJHHPBRSE type=del size=3247592 2024-04-07T16:37:45.210 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.217+0000 7f346a3868c0 1 < canceled part key=_multipart_YOK8BAEOMPUR.2~8KND6H61905M.0 size=1048576 2024-04-07T16:37:45.213 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.221+0000 7f346a3868c0 1 < completed part key=_multipart_YOK8BAEOMPUR.2~8KND6H61905M.1 size=1048576 2024-04-07T16:37:45.216 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.225+0000 7f346a3868c0 1 < completed part key=_multipart_YOK8BAEOMPUR.2~8KND6H61905M.2 size=709779 2024-04-07T16:37:45.223 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.233+0000 7f346a3868c0 1 < canceled multipart upload key=YOK8BAEOMPUR upload=8KND6H61905M tag=XZEEKBB4K6EZ type=write size=3855507 2024-04-07T16:37:45.230 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.237+0000 7f346a3868c0 1 > prepared part key=_multipart_CL4PP1RKZU0E.2~C0D6CXN9EPV6.0 size=557328 2024-04-07T16:37:45.230 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.237+0000 7f346a3868c0 1 > prepared multipart upload key=CL4PP1RKZU0E upload=C0D6CXN9EPV6 tag=1V359CZHI7JW type=write size=1605904 2024-04-07T16:37:45.234 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.241+0000 7f346a3868c0 1 < completed delete operation key=NYET3212USVE tag=TGRS221J7L8P type=del 2024-04-07T16:37:45.243 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.253+0000 7f346a3868c0 1 > prepared part key=_multipart_N14BUXLJONHZ.2~VBMBLTJ3GSME.0 size=1048576 2024-04-07T16:37:45.247 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.253+0000 7f346a3868c0 1 > prepared part key=_multipart_N14BUXLJONHZ.2~VBMBLTJ3GSME.1 size=1048576 2024-04-07T16:37:45.250 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.257+0000 7f346a3868c0 1 > prepared part key=_multipart_N14BUXLJONHZ.2~VBMBLTJ3GSME.2 size=108744 2024-04-07T16:37:45.250 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.257+0000 7f346a3868c0 1 > prepared multipart upload key=N14BUXLJONHZ upload=VBMBLTJ3GSME tag=WMOOXQ34A9I1 type=write size=3254472 2024-04-07T16:37:45.255 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.265+0000 7f346a3868c0 1 < completed delete operation key=XJLQZ7IS9U1T tag=WBQ9N8HVKRBP type=del 2024-04-07T16:37:45.260 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.269+0000 7f346a3868c0 1 > prepared operation key=GZXHYZO5SMUP tag=QQ9QK2MUAZM4 type=del size=188861 2024-04-07T16:37:45.265 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.273+0000 7f346a3868c0 1 < completed write operation key=GZXHYZO5SMUP tag=H58UN7C2NZ33 type=write size=803113 2024-04-07T16:37:45.269 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.277+0000 7f346a3868c0 1 > prepared operation key=CL4PP1RKZU0E tag=APE9KWOHRNOL type=del size=3783608 2024-04-07T16:37:45.273 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.281+0000 7f346a3868c0 1 < completed write operation key=LXOZ0T2KY6OR tag=8VFMPU9T7WLY type=write size=376349 2024-04-07T16:37:45.278 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.285+0000 7f346a3868c0 1 > prepared operation key=N14BUXLJONHZ tag=U4RVSSRJ5JT6 type=del size=1456172 2024-04-07T16:37:45.285 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.289+0000 7f346a3868c0 1 < completed part key=_multipart_YOK8BAEOMPUR.2~V162J1TZ33Z4.0 size=711885 2024-04-07T16:37:45.289 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.297+0000 7f346a3868c0 1 < completed multipart upload key=YOK8BAEOMPUR upload=V162J1TZ33Z4 tag=S1EKYN3J0APF type=write size=1760461 2024-04-07T16:37:45.294 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.301+0000 7f346a3868c0 1 > prepared operation key=X5Q46GO8YTOK tag=SK3T29ML3CRG type=del size=869111 2024-04-07T16:37:45.297 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.305+0000 7f346a3868c0 1 < completed part key=_multipart_IX3F0IUOKBX1.2~HPWUHV910IRK.0 size=1048576 2024-04-07T16:37:45.300 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.309+0000 7f346a3868c0 1 < completed part key=_multipart_IX3F0IUOKBX1.2~HPWUHV910IRK.1 size=1048576 2024-04-07T16:37:45.303 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.313+0000 7f346a3868c0 1 < completed part key=_multipart_IX3F0IUOKBX1.2~HPWUHV910IRK.2 size=962495 2024-04-07T16:37:45.310 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.317+0000 7f346a3868c0 1 < completed multipart upload key=IX3F0IUOKBX1 upload=HPWUHV910IRK tag=B0OC7TPL9PJW type=write size=4108223 2024-04-07T16:37:45.314 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.321+0000 7f346a3868c0 1 > prepared operation key=3N0QJRYVO3FV tag=YB52NV7XIVO5 type=del size=3228042 2024-04-07T16:37:45.318 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.325+0000 7f346a3868c0 1 < completed write operation key=3N0QJRYVO3FV tag=MDLL9UFJAAV5 type=write size=685256 2024-04-07T16:37:45.325 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.333+0000 7f346a3868c0 1 > prepared part key=_multipart_HBLEXONKXJTL.2~VU1MZ46RZIK2.0 size=265823 2024-04-07T16:37:45.326 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.333+0000 7f346a3868c0 1 > prepared multipart upload key=HBLEXONKXJTL upload=VU1MZ46RZIK2 tag=0JUVOEHFGLL0 type=write size=1314399 2024-04-07T16:37:45.329 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.337+0000 7f346a3868c0 1 < completed part key=_multipart_8F81G0P0YBFW.2~H9A1GFIF03XI.0 size=387856 2024-04-07T16:37:45.336 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.345+0000 7f346a3868c0 1 < completed multipart upload key=8F81G0P0YBFW upload=H9A1GFIF03XI tag=TPZYYPZ3IOFW type=write size=1436432 2024-04-07T16:37:45.339 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.349+0000 7f346a3868c0 1 > prepared operation key=E84U0SX44NQ8 tag=4MA0JM4BHYQV type=del size=3021895 2024-04-07T16:37:45.344 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.353+0000 7f346a3868c0 1 < completed part key=_multipart_XJLQZ7IS9U1T.2~G42STNCVHEO6.0 size=1048576 2024-04-07T16:37:45.348 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.357+0000 7f346a3868c0 1 < completed part key=_multipart_XJLQZ7IS9U1T.2~G42STNCVHEO6.1 size=32267 2024-04-07T16:37:45.354 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.361+0000 7f346a3868c0 1 < completed multipart upload key=XJLQZ7IS9U1T upload=G42STNCVHEO6 tag=FN61NI0M1X2M type=write size=2129419 2024-04-07T16:37:45.358 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.365+0000 7f346a3868c0 1 > prepared operation key=DXTO3AJ429AC tag=KBGBXH780R80 type=write size=726007 2024-04-07T16:37:45.363 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.373+0000 7f346a3868c0 1 < completed delete operation key=QTCIIPTXIRW2 tag=VELRJ432A00H type=del 2024-04-07T16:37:45.367 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.377+0000 7f346a3868c0 1 > prepared operation key=3N0QJRYVO3FV tag=IXDD3CXUDITD type=write size=235012 2024-04-07T16:37:45.371 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.381+0000 7f346a3868c0 1 < completed part key=_multipart_CL4PP1RKZU0E.2~V5527QHNCU0L.0 size=671475 2024-04-07T16:37:45.379 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.385+0000 7f346a3868c0 1 < completed multipart upload key=CL4PP1RKZU0E upload=V5527QHNCU0L tag=VMKZKJE90QW3 type=write size=1720051 2024-04-07T16:37:45.383 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.393+0000 7f346a3868c0 1 > prepared operation key=N14BUXLJONHZ tag=43R9O0ED3BHV type=write size=1040485 2024-04-07T16:37:45.387 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.397+0000 7f346a3868c0 1 < completed part key=_multipart_X5Q46GO8YTOK.2~0OVO93KYGA9U.0 size=1048576 2024-04-07T16:37:45.390 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.397+0000 7f346a3868c0 1 < completed part key=_multipart_X5Q46GO8YTOK.2~0OVO93KYGA9U.1 size=369992 2024-04-07T16:37:45.397 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.405+0000 7f346a3868c0 1 < completed multipart upload key=X5Q46GO8YTOK upload=0OVO93KYGA9U tag=4C195OJMZMO8 type=write size=2467144 2024-04-07T16:37:45.401 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.409+0000 7f346a3868c0 1 > prepared operation key=DXTO3AJ429AC tag=KKN95KDDI6L4 type=write size=873350 2024-04-07T16:37:45.406 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.413+0000 7f346a3868c0 1 < completed delete operation key=1VKVQXSQMTU1 tag=PEKO7ALBFWWM type=del 2024-04-07T16:37:45.411 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.417+0000 7f346a3868c0 1 > prepared operation key=N14BUXLJONHZ tag=P7LBSF0O0351 type=del size=1263167 2024-04-07T16:37:45.416 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.425+0000 7f346a3868c0 1 < completed part key=_multipart_CL4PP1RKZU0E.2~L1WTV2OLXLLM.0 size=1048576 2024-04-07T16:37:45.419 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.429+0000 7f346a3868c0 1 < completed part key=_multipart_CL4PP1RKZU0E.2~L1WTV2OLXLLM.1 size=785542 2024-04-07T16:37:45.426 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.433+0000 7f346a3868c0 1 < completed multipart upload key=CL4PP1RKZU0E upload=L1WTV2OLXLLM tag=AWEUZMLFUPFX type=write size=2882694 2024-04-07T16:37:45.434 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.441+0000 7f346a3868c0 1 > prepared part key=_multipart_IX3F0IUOKBX1.2~1NG96D4QAHF2.0 size=1048576 2024-04-07T16:37:45.438 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.445+0000 7f346a3868c0 1 > prepared part key=_multipart_IX3F0IUOKBX1.2~1NG96D4QAHF2.1 size=1048576 2024-04-07T16:37:45.441 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.449+0000 7f346a3868c0 1 > prepared part key=_multipart_IX3F0IUOKBX1.2~1NG96D4QAHF2.2 size=935469 2024-04-07T16:37:45.442 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.449+0000 7f346a3868c0 1 > prepared multipart upload key=IX3F0IUOKBX1 upload=1NG96D4QAHF2 tag=SYQ8MXGPDM2G type=write size=4081197 2024-04-07T16:37:45.446 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.453+0000 7f346a3868c0 1 < completed delete operation key=IX3F0IUOKBX1 tag=3DGWJHHPBRSE type=del 2024-04-07T16:37:45.451 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.461+0000 7f346a3868c0 1 > prepared operation key=YOK8BAEOMPUR tag=6T5YW6GH5LBQ type=del size=2019104 2024-04-07T16:37:45.456 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.465+0000 7f346a3868c0 1 < completed part key=_multipart_CL4PP1RKZU0E.2~C0D6CXN9EPV6.0 size=557328 2024-04-07T16:37:45.462 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.469+0000 7f346a3868c0 1 < completed multipart upload key=CL4PP1RKZU0E upload=C0D6CXN9EPV6 tag=1V359CZHI7JW type=write size=1605904 2024-04-07T16:37:45.466 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.473+0000 7f346a3868c0 1 > prepared operation key=DXTO3AJ429AC tag=AUUZIWXC03UL type=write size=633588 2024-04-07T16:37:45.470 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.477+0000 7f346a3868c0 1 < completed part key=_multipart_N14BUXLJONHZ.2~VBMBLTJ3GSME.0 size=1048576 2024-04-07T16:37:45.474 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.481+0000 7f346a3868c0 1 < completed part key=_multipart_N14BUXLJONHZ.2~VBMBLTJ3GSME.1 size=1048576 2024-04-07T16:37:45.477 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.485+0000 7f346a3868c0 1 < completed part key=_multipart_N14BUXLJONHZ.2~VBMBLTJ3GSME.2 size=108744 2024-04-07T16:37:45.484 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.493+0000 7f346a3868c0 1 < completed multipart upload key=N14BUXLJONHZ upload=VBMBLTJ3GSME tag=WMOOXQ34A9I1 type=write size=3254472 2024-04-07T16:37:45.491 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.497+0000 7f346a3868c0 1 > prepared part key=_multipart_QS4HYZFFAJNF.2~72TP232086KZ.0 size=1048576 2024-04-07T16:37:45.494 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.501+0000 7f346a3868c0 1 > prepared part key=_multipart_QS4HYZFFAJNF.2~72TP232086KZ.1 size=1048576 2024-04-07T16:37:45.497 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.505+0000 7f346a3868c0 1 > prepared part key=_multipart_QS4HYZFFAJNF.2~72TP232086KZ.2 size=42944 2024-04-07T16:37:45.497 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.505+0000 7f346a3868c0 1 > prepared multipart upload key=QS4HYZFFAJNF upload=72TP232086KZ tag=FMZ0U4OS10TH type=write size=3188672 2024-04-07T16:37:45.501 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.509+0000 7f346a3868c0 1 < completed delete operation key=GZXHYZO5SMUP tag=QQ9QK2MUAZM4 type=del 2024-04-07T16:37:45.505 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.513+0000 7f346a3868c0 1 > prepared operation key=QTCIIPTXIRW2 tag=8X8P4SAZOGMB type=write size=202128 2024-04-07T16:37:45.511 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.517+0000 7f346a3868c0 1 < completed delete operation key=CL4PP1RKZU0E tag=APE9KWOHRNOL type=del 2024-04-07T16:37:45.515 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.525+0000 7f346a3868c0 1 > prepared operation key=1VBSUWNI3V1T tag=YI42G4UDPRRU type=del size=2950672 2024-04-07T16:37:45.519 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.529+0000 7f346a3868c0 1 < completed delete operation key=N14BUXLJONHZ tag=U4RVSSRJ5JT6 type=del 2024-04-07T16:37:45.524 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.533+0000 7f346a3868c0 1 > prepared operation key=UIW3DN4NUYPP tag=UKCNTPN3BVHK type=del size=3138466 2024-04-07T16:37:45.528 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.537+0000 7f346a3868c0 1 < completed delete operation key=X5Q46GO8YTOK tag=SK3T29ML3CRG type=del 2024-04-07T16:37:45.535 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.545+0000 7f346a3868c0 1 > prepared part key=_multipart_DXTO3AJ429AC.2~Z3FDLI80UFE7.0 size=507953 2024-04-07T16:37:45.535 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.545+0000 7f346a3868c0 1 > prepared multipart upload key=DXTO3AJ429AC upload=Z3FDLI80UFE7 tag=IO4TO6UH6RMM type=write size=1556529 2024-04-07T16:37:45.539 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.549+0000 7f346a3868c0 1 < completed delete operation key=3N0QJRYVO3FV tag=YB52NV7XIVO5 type=del 2024-04-07T16:37:45.547 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.557+0000 7f346a3868c0 1 > prepared part key=_multipart_N14BUXLJONHZ.2~BYKN2LOVTPXS.0 size=1048576 2024-04-07T16:37:45.551 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.561+0000 7f346a3868c0 1 > prepared part key=_multipart_N14BUXLJONHZ.2~BYKN2LOVTPXS.1 size=1048576 2024-04-07T16:37:45.554 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.561+0000 7f346a3868c0 1 > prepared part key=_multipart_N14BUXLJONHZ.2~BYKN2LOVTPXS.2 size=694105 2024-04-07T16:37:45.554 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.561+0000 7f346a3868c0 1 > prepared multipart upload key=N14BUXLJONHZ upload=BYKN2LOVTPXS tag=RK38QC3K0588 type=write size=3839833 2024-04-07T16:37:45.559 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.565+0000 7f346a3868c0 1 < completed part key=_multipart_HBLEXONKXJTL.2~VU1MZ46RZIK2.0 size=265823 2024-04-07T16:37:45.566 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.573+0000 7f346a3868c0 1 < completed multipart upload key=HBLEXONKXJTL upload=VU1MZ46RZIK2 tag=0JUVOEHFGLL0 type=write size=1314399 2024-04-07T16:37:45.569 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.577+0000 7f346a3868c0 1 > prepared operation key=U3EID5EK64HD tag=XZGHWGV3BG5A type=del size=3829792 2024-04-07T16:37:45.573 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.581+0000 7f346a3868c0 1 < completed delete operation key=E84U0SX44NQ8 tag=4MA0JM4BHYQV type=del 2024-04-07T16:37:45.578 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.585+0000 7f346a3868c0 1 > prepared operation key=EQR0FY86AT5X tag=HLP7BL2567I8 type=del size=2687777 2024-04-07T16:37:45.586 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.593+0000 7f346a3868c0 1 < suggested update operation key=DXTO3AJ429AC tag=KBGBXH780R80 type=write size=726007 2024-04-07T16:37:45.593 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.601+0000 7f346a3868c0 1 > prepared part key=_multipart_6XDS0A7L0IY8.2~3SAD5DBDUH4F.0 size=1048576 2024-04-07T16:37:45.596 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.605+0000 7f346a3868c0 1 > prepared part key=_multipart_6XDS0A7L0IY8.2~3SAD5DBDUH4F.1 size=1048576 2024-04-07T16:37:45.600 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.609+0000 7f346a3868c0 1 > prepared part key=_multipart_6XDS0A7L0IY8.2~3SAD5DBDUH4F.2 size=287442 2024-04-07T16:37:45.600 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.609+0000 7f346a3868c0 1 > prepared multipart upload key=6XDS0A7L0IY8 upload=3SAD5DBDUH4F tag=BC2QMXGXBBR7 type=write size=3433170 2024-04-07T16:37:45.607 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.617+0000 7f346a3868c0 1 < suggested update operation key=3N0QJRYVO3FV tag=IXDD3CXUDITD type=write size=235012 2024-04-07T16:37:45.615 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.625+0000 7f346a3868c0 1 > prepared part key=_multipart_YOK8BAEOMPUR.2~F7EV7728ST25.0 size=1048576 2024-04-07T16:37:45.619 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.629+0000 7f346a3868c0 1 > prepared part key=_multipart_YOK8BAEOMPUR.2~F7EV7728ST25.1 size=328913 2024-04-07T16:37:45.619 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.629+0000 7f346a3868c0 1 > prepared multipart upload key=YOK8BAEOMPUR upload=F7EV7728ST25 tag=I1XON5M6H3J4 type=write size=2426065 2024-04-07T16:37:45.623 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.633+0000 7f346a3868c0 1 < completed write operation key=N14BUXLJONHZ tag=43R9O0ED3BHV type=write size=1040485 2024-04-07T16:37:45.628 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.637+0000 7f346a3868c0 1 > prepared operation key=N14BUXLJONHZ tag=M8XAIHB08VJY type=del size=1810519 2024-04-07T16:37:45.632 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.641+0000 7f346a3868c0 -1 < failed to complete operation key=DXTO3AJ429AC tag=KKN95KDDI6L4 type=write size=873350: (22) Invalid argument 2024-04-07T16:37:45.636 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.645+0000 7f346a3868c0 1 > prepared operation key=QS4HYZFFAJNF tag=MCCWGRZULJH7 type=del size=1603859 2024-04-07T16:37:45.643 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.653+0000 7f346a3868c0 1 < suggested remove operation key=N14BUXLJONHZ tag=P7LBSF0O0351 type=del 2024-04-07T16:37:45.647 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.657+0000 7f346a3868c0 1 > prepared operation key=YXMDZB23YTB3 tag=GP6SF1OHT8VO type=write size=897440 2024-04-07T16:37:45.651 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.657+0000 7f346a3868c0 1 < completed part key=_multipart_IX3F0IUOKBX1.2~1NG96D4QAHF2.0 size=1048576 2024-04-07T16:37:45.654 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.661+0000 7f346a3868c0 1 < completed part key=_multipart_IX3F0IUOKBX1.2~1NG96D4QAHF2.1 size=1048576 2024-04-07T16:37:45.657 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.665+0000 7f346a3868c0 1 < completed part key=_multipart_IX3F0IUOKBX1.2~1NG96D4QAHF2.2 size=935469 2024-04-07T16:37:45.663 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.673+0000 7f346a3868c0 1 < completed multipart upload key=IX3F0IUOKBX1 upload=1NG96D4QAHF2 tag=SYQ8MXGPDM2G type=write size=4081197 2024-04-07T16:37:45.666 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.673+0000 7f346a3868c0 1 > prepared operation key=6XDS0A7L0IY8 tag=0PKV2NCZ31R6 type=del size=1980751 2024-04-07T16:37:45.670 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.677+0000 7f346a3868c0 1 < completed delete operation key=YOK8BAEOMPUR tag=6T5YW6GH5LBQ type=del 2024-04-07T16:37:45.677 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.685+0000 7f346a3868c0 1 > prepared part key=_multipart_IP8YW2A75TCW.2~DM2ZWW55FWN9.0 size=397015 2024-04-07T16:37:45.677 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.685+0000 7f346a3868c0 1 > prepared multipart upload key=IP8YW2A75TCW upload=DM2ZWW55FWN9 tag=P23SG7TO38QR type=write size=1445591 2024-04-07T16:37:45.682 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.689+0000 7f346a3868c0 -1 < failed to complete operation key=DXTO3AJ429AC tag=AUUZIWXC03UL type=write size=633588: (22) Invalid argument 2024-04-07T16:37:45.688 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.697+0000 7f346a3868c0 1 > prepared part key=_multipart_NYET3212USVE.2~7ERJ8E7HGAEV.0 size=1048576 2024-04-07T16:37:45.691 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.701+0000 7f346a3868c0 1 > prepared part key=_multipart_NYET3212USVE.2~7ERJ8E7HGAEV.1 size=1048576 2024-04-07T16:37:45.694 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.701+0000 7f346a3868c0 1 > prepared part key=_multipart_NYET3212USVE.2~7ERJ8E7HGAEV.2 size=569607 2024-04-07T16:37:45.694 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.701+0000 7f346a3868c0 1 > prepared multipart upload key=NYET3212USVE upload=7ERJ8E7HGAEV tag=DRVIXXDZBK2H type=write size=3715335 2024-04-07T16:37:45.698 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.705+0000 7f346a3868c0 1 < completed part key=_multipart_QS4HYZFFAJNF.2~72TP232086KZ.0 size=1048576 2024-04-07T16:37:45.701 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.709+0000 7f346a3868c0 1 < completed part key=_multipart_QS4HYZFFAJNF.2~72TP232086KZ.1 size=1048576 2024-04-07T16:37:45.704 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.713+0000 7f346a3868c0 1 < completed part key=_multipart_QS4HYZFFAJNF.2~72TP232086KZ.2 size=42944 2024-04-07T16:37:45.711 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.717+0000 7f346a3868c0 1 < completed multipart upload key=QS4HYZFFAJNF upload=72TP232086KZ tag=FMZ0U4OS10TH type=write size=3188672 2024-04-07T16:37:45.715 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.721+0000 7f346a3868c0 1 > prepared operation key=3N0QJRYVO3FV tag=ANMCAMC2VPO1 type=del size=3198950 2024-04-07T16:37:45.719 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.725+0000 7f346a3868c0 1 < completed write operation key=QTCIIPTXIRW2 tag=8X8P4SAZOGMB type=write size=202128 2024-04-07T16:37:45.723 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.729+0000 7f346a3868c0 1 > prepared operation key=C1BTVJNHIM0X tag=C1FBXJNGDPP6 type=del size=283963 2024-04-07T16:37:45.726 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.733+0000 7f346a3868c0 1 < completed delete operation key=1VBSUWNI3V1T tag=YI42G4UDPRRU type=del 2024-04-07T16:37:45.730 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.737+0000 7f346a3868c0 1 > prepared operation key=GBQSME7SY1AH tag=RVQ3Q5HKCXOA type=write size=923500 2024-04-07T16:37:45.734 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.741+0000 7f346a3868c0 1 < completed delete operation key=UIW3DN4NUYPP tag=UKCNTPN3BVHK type=del 2024-04-07T16:37:45.737 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.745+0000 7f346a3868c0 1 > prepared operation key=7G5C6HBK6QWI tag=T80E1C2T25CB type=write size=688721 2024-04-07T16:37:45.742 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.749+0000 7f346a3868c0 1 < completed part key=_multipart_DXTO3AJ429AC.2~Z3FDLI80UFE7.0 size=507953 2024-04-07T16:37:45.748 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.757+0000 7f346a3868c0 1 < completed multipart upload key=DXTO3AJ429AC upload=Z3FDLI80UFE7 tag=IO4TO6UH6RMM type=write size=1556529 2024-04-07T16:37:45.756 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.765+0000 7f346a3868c0 1 > prepared part key=_multipart_1WGUPH7X5UA0.2~U3S3QXLXXMFM.0 size=130620 2024-04-07T16:37:45.756 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.765+0000 7f346a3868c0 1 > prepared multipart upload key=1WGUPH7X5UA0 upload=U3S3QXLXXMFM tag=JUOQFJ2BZD8U type=write size=1179196 2024-04-07T16:37:45.760 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.769+0000 7f346a3868c0 1 < completed part key=_multipart_N14BUXLJONHZ.2~BYKN2LOVTPXS.0 size=1048576 2024-04-07T16:37:45.763 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.773+0000 7f346a3868c0 1 < completed part key=_multipart_N14BUXLJONHZ.2~BYKN2LOVTPXS.1 size=1048576 2024-04-07T16:37:45.767 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.773+0000 7f346a3868c0 1 < completed part key=_multipart_N14BUXLJONHZ.2~BYKN2LOVTPXS.2 size=694105 2024-04-07T16:37:45.773 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.781+0000 7f346a3868c0 1 < completed multipart upload key=N14BUXLJONHZ upload=BYKN2LOVTPXS tag=RK38QC3K0588 type=write size=3839833 2024-04-07T16:37:45.777 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.785+0000 7f346a3868c0 1 > prepared operation key=GZXHYZO5SMUP tag=TLF37MLPPYH0 type=del size=656453 2024-04-07T16:37:45.782 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.789+0000 7f346a3868c0 1 < completed delete operation key=U3EID5EK64HD tag=XZGHWGV3BG5A type=del 2024-04-07T16:37:45.786 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.793+0000 7f346a3868c0 1 > prepared operation key=IX3F0IUOKBX1 tag=PJ5EKZYARV37 type=del size=2906523 2024-04-07T16:37:45.790 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.797+0000 7f346a3868c0 1 < completed delete operation key=EQR0FY86AT5X tag=HLP7BL2567I8 type=del 2024-04-07T16:37:45.794 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.801+0000 7f346a3868c0 1 > prepared operation key=GZXHYZO5SMUP tag=M1RF9LXMWOB3 type=del size=3080718 2024-04-07T16:37:45.798 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.805+0000 7f346a3868c0 1 < completed part key=_multipart_6XDS0A7L0IY8.2~3SAD5DBDUH4F.0 size=1048576 2024-04-07T16:37:45.802 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.809+0000 7f346a3868c0 1 < completed part key=_multipart_6XDS0A7L0IY8.2~3SAD5DBDUH4F.1 size=1048576 2024-04-07T16:37:45.806 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.813+0000 7f346a3868c0 1 < completed part key=_multipart_6XDS0A7L0IY8.2~3SAD5DBDUH4F.2 size=287442 2024-04-07T16:37:45.812 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.821+0000 7f346a3868c0 1 < completed multipart upload key=6XDS0A7L0IY8 upload=3SAD5DBDUH4F tag=BC2QMXGXBBR7 type=write size=3433170 2024-04-07T16:37:45.816 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.825+0000 7f346a3868c0 1 > prepared operation key=U3EID5EK64HD tag=W9BBIKW9AMYQ type=del size=1280082 2024-04-07T16:37:45.821 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.829+0000 7f346a3868c0 1 < completed part key=_multipart_YOK8BAEOMPUR.2~F7EV7728ST25.0 size=1048576 2024-04-07T16:37:45.824 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.833+0000 7f346a3868c0 1 < completed part key=_multipart_YOK8BAEOMPUR.2~F7EV7728ST25.1 size=328913 2024-04-07T16:37:45.830 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.837+0000 7f346a3868c0 1 < completed multipart upload key=YOK8BAEOMPUR upload=F7EV7728ST25 tag=I1XON5M6H3J4 type=write size=2426065 2024-04-07T16:37:45.835 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.841+0000 7f346a3868c0 1 > prepared operation key=QTCIIPTXIRW2 tag=G7PMOYTC2TEV type=del size=3112174 2024-04-07T16:37:45.839 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.849+0000 7f346a3868c0 -1 < failed to complete operation key=N14BUXLJONHZ tag=M8XAIHB08VJY type=del size=1810519: (22) Invalid argument 2024-04-07T16:37:45.843 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.853+0000 7f346a3868c0 1 > prepared operation key=EOYFVEI50SQ1 tag=LOIFLWHT4HSG type=del size=2329330 2024-04-07T16:37:45.848 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.857+0000 7f346a3868c0 1 < completed delete operation key=QS4HYZFFAJNF tag=MCCWGRZULJH7 type=del 2024-04-07T16:37:45.855 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.865+0000 7f346a3868c0 1 > prepared part key=_multipart_EOYFVEI50SQ1.2~ZZS0XS04SJ2Q.0 size=1048576 2024-04-07T16:37:45.859 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.869+0000 7f346a3868c0 1 > prepared part key=_multipart_EOYFVEI50SQ1.2~ZZS0XS04SJ2Q.1 size=1048576 2024-04-07T16:37:45.862 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.869+0000 7f346a3868c0 1 > prepared part key=_multipart_EOYFVEI50SQ1.2~ZZS0XS04SJ2Q.2 size=44999 2024-04-07T16:37:45.862 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.869+0000 7f346a3868c0 1 > prepared multipart upload key=EOYFVEI50SQ1 upload=ZZS0XS04SJ2Q tag=0623DCSJ4OPX type=write size=3190727 2024-04-07T16:37:45.867 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.877+0000 7f346a3868c0 1 < completed write operation key=YXMDZB23YTB3 tag=GP6SF1OHT8VO type=write size=897440 2024-04-07T16:37:45.876 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.885+0000 7f346a3868c0 1 > prepared part key=_multipart_QTCIIPTXIRW2.2~T1WCLFASNN6L.0 size=99205 2024-04-07T16:37:45.876 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.885+0000 7f346a3868c0 1 > prepared multipart upload key=QTCIIPTXIRW2 upload=T1WCLFASNN6L tag=GGF46KDN2MQ5 type=write size=1147781 2024-04-07T16:37:45.885 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.893+0000 7f346a3868c0 1 < suggested remove operation key=6XDS0A7L0IY8 tag=0PKV2NCZ31R6 type=del 2024-04-07T16:37:45.893 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.901+0000 7f346a3868c0 1 > prepared part key=_multipart_E84U0SX44NQ8.2~Y5LG3GIK61RC.0 size=815153 2024-04-07T16:37:45.893 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.901+0000 7f346a3868c0 1 > prepared multipart upload key=E84U0SX44NQ8 upload=Y5LG3GIK61RC tag=J0LY7CUCRAM9 type=write size=1863729 2024-04-07T16:37:45.898 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.905+0000 7f346a3868c0 1 < completed part key=_multipart_IP8YW2A75TCW.2~DM2ZWW55FWN9.0 size=397015 2024-04-07T16:37:45.905 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.913+0000 7f346a3868c0 1 < completed multipart upload key=IP8YW2A75TCW upload=DM2ZWW55FWN9 tag=P23SG7TO38QR type=write size=1445591 2024-04-07T16:37:45.908 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.917+0000 7f346a3868c0 1 > prepared operation key=X5Q46GO8YTOK tag=89JMLO4LIT2E type=del size=553228 2024-04-07T16:37:45.913 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.921+0000 7f346a3868c0 1 < completed part key=_multipart_NYET3212USVE.2~7ERJ8E7HGAEV.0 size=1048576 2024-04-07T16:37:45.916 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.925+0000 7f346a3868c0 1 < completed part key=_multipart_NYET3212USVE.2~7ERJ8E7HGAEV.1 size=1048576 2024-04-07T16:37:45.920 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.929+0000 7f346a3868c0 1 < completed part key=_multipart_NYET3212USVE.2~7ERJ8E7HGAEV.2 size=569607 2024-04-07T16:37:45.927 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.933+0000 7f346a3868c0 1 < completed multipart upload key=NYET3212USVE upload=7ERJ8E7HGAEV tag=DRVIXXDZBK2H type=write size=3715335 2024-04-07T16:37:45.931 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.941+0000 7f346a3868c0 1 > prepared operation key=8F81G0P0YBFW tag=2OY1HL9MGJAG type=del size=245932 2024-04-07T16:37:45.935 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.945+0000 7f346a3868c0 1 < completed delete operation key=3N0QJRYVO3FV tag=ANMCAMC2VPO1 type=del 2024-04-07T16:37:45.939 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.949+0000 7f346a3868c0 1 > prepared operation key=QVOKSQAGHPA9 tag=B2RC5P8RHY09 type=del size=1671278 2024-04-07T16:37:45.943 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.953+0000 7f346a3868c0 1 < completed delete operation key=C1BTVJNHIM0X tag=C1FBXJNGDPP6 type=del 2024-04-07T16:37:45.951 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.957+0000 7f346a3868c0 1 > prepared part key=_multipart_IX3F0IUOKBX1.2~UM9HY7ZU54ZW.0 size=1048576 2024-04-07T16:37:45.954 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.961+0000 7f346a3868c0 1 > prepared part key=_multipart_IX3F0IUOKBX1.2~UM9HY7ZU54ZW.1 size=1048576 2024-04-07T16:37:45.957 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.965+0000 7f346a3868c0 1 > prepared part key=_multipart_IX3F0IUOKBX1.2~UM9HY7ZU54ZW.2 size=953692 2024-04-07T16:37:45.957 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.965+0000 7f346a3868c0 1 > prepared multipart upload key=IX3F0IUOKBX1 upload=UM9HY7ZU54ZW tag=5U9ZPQI44I5J type=write size=4099420 2024-04-07T16:37:45.961 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.969+0000 7f346a3868c0 1 < completed write operation key=GBQSME7SY1AH tag=RVQ3Q5HKCXOA type=write size=923500 2024-04-07T16:37:45.965 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.973+0000 7f346a3868c0 1 > prepared operation key=1VBSUWNI3V1T tag=1VRFW5TFRJ28 type=del size=4064273 2024-04-07T16:37:45.973 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.981+0000 7f346a3868c0 1 < suggested update operation key=7G5C6HBK6QWI tag=T80E1C2T25CB type=write size=688721 2024-04-07T16:37:45.980 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.989+0000 7f346a3868c0 1 > prepared part key=_multipart_IP8YW2A75TCW.2~G8ZCBIWKWBKQ.0 size=1048576 2024-04-07T16:37:45.983 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.993+0000 7f346a3868c0 1 > prepared part key=_multipart_IP8YW2A75TCW.2~G8ZCBIWKWBKQ.1 size=1048576 2024-04-07T16:37:45.987 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.993+0000 7f346a3868c0 1 > prepared part key=_multipart_IP8YW2A75TCW.2~G8ZCBIWKWBKQ.2 size=149458 2024-04-07T16:37:45.987 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.993+0000 7f346a3868c0 1 > prepared multipart upload key=IP8YW2A75TCW upload=G8ZCBIWKWBKQ tag=L7IGCJCM0KO8 type=write size=3295186 2024-04-07T16:37:45.991 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:45.997+0000 7f346a3868c0 1 < completed part key=_multipart_1WGUPH7X5UA0.2~U3S3QXLXXMFM.0 size=130620 2024-04-07T16:37:45.997 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.005+0000 7f346a3868c0 1 < completed multipart upload key=1WGUPH7X5UA0 upload=U3S3QXLXXMFM tag=JUOQFJ2BZD8U type=write size=1179196 2024-04-07T16:37:46.004 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.013+0000 7f346a3868c0 1 > prepared part key=_multipart_QS4HYZFFAJNF.2~4E5TLZYCMZ2W.0 size=951465 2024-04-07T16:37:46.004 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.013+0000 7f346a3868c0 1 > prepared multipart upload key=QS4HYZFFAJNF upload=4E5TLZYCMZ2W tag=MO9C30BGSB0I type=write size=2000041 2024-04-07T16:37:46.008 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.017+0000 7f346a3868c0 1 < completed delete operation key=GZXHYZO5SMUP tag=TLF37MLPPYH0 type=del 2024-04-07T16:37:46.015 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.021+0000 7f346a3868c0 1 > prepared part key=_multipart_1VKVQXSQMTU1.2~3FSR3RDGNJ03.0 size=734879 2024-04-07T16:37:46.015 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.021+0000 7f346a3868c0 1 > prepared multipart upload key=1VKVQXSQMTU1 upload=3FSR3RDGNJ03 tag=EX1J7K6AORSI type=write size=1783455 2024-04-07T16:37:46.018 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.025+0000 7f346a3868c0 1 < completed delete operation key=IX3F0IUOKBX1 tag=PJ5EKZYARV37 type=del 2024-04-07T16:37:46.022 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.029+0000 7f346a3868c0 1 > prepared operation key=IX3F0IUOKBX1 tag=8YL6V64GGX1B type=write size=719425 2024-04-07T16:37:46.030 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.036+0000 7f346a3868c0 1 < suggested update operation key=GZXHYZO5SMUP tag=M1RF9LXMWOB3 type=del size=3080718 2024-04-07T16:37:46.037 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.044+0000 7f346a3868c0 1 > prepared part key=_multipart_IX3F0IUOKBX1.2~0TB2NDIRYLTJ.0 size=1048576 2024-04-07T16:37:46.040 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.048+0000 7f346a3868c0 1 > prepared part key=_multipart_IX3F0IUOKBX1.2~0TB2NDIRYLTJ.1 size=300623 2024-04-07T16:37:46.040 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.048+0000 7f346a3868c0 1 > prepared multipart upload key=IX3F0IUOKBX1 upload=0TB2NDIRYLTJ tag=LCN2Z5CTKATB type=write size=2397775 2024-04-07T16:37:46.044 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.052+0000 7f346a3868c0 1 < completed delete operation key=U3EID5EK64HD tag=W9BBIKW9AMYQ type=del 2024-04-07T16:37:46.048 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.056+0000 7f346a3868c0 1 > prepared operation key=1WGUPH7X5UA0 tag=TBAGXRG66QB3 type=del size=1692404 2024-04-07T16:37:46.052 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.060+0000 7f346a3868c0 1 < completed delete operation key=QTCIIPTXIRW2 tag=G7PMOYTC2TEV type=del 2024-04-07T16:37:46.060 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.068+0000 7f346a3868c0 1 > prepared part key=_multipart_X5Q46GO8YTOK.2~2DP4IXSD81Q1.0 size=722474 2024-04-07T16:37:46.060 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.068+0000 7f346a3868c0 1 > prepared multipart upload key=X5Q46GO8YTOK upload=2DP4IXSD81Q1 tag=RFRQ08EAJS2R type=write size=1771050 2024-04-07T16:37:46.064 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.072+0000 7f346a3868c0 1 < completed delete operation key=EOYFVEI50SQ1 tag=LOIFLWHT4HSG type=del 2024-04-07T16:37:46.069 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.076+0000 7f346a3868c0 1 > prepared operation key=ROFH30W3UCAX tag=PSSJLRJUJPJ2 type=del size=4071331 2024-04-07T16:37:46.073 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.080+0000 7f346a3868c0 1 < completed part key=_multipart_EOYFVEI50SQ1.2~ZZS0XS04SJ2Q.0 size=1048576 2024-04-07T16:37:46.076 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.084+0000 7f346a3868c0 1 < completed part key=_multipart_EOYFVEI50SQ1.2~ZZS0XS04SJ2Q.1 size=1048576 2024-04-07T16:37:46.080 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.088+0000 7f346a3868c0 1 < completed part key=_multipart_EOYFVEI50SQ1.2~ZZS0XS04SJ2Q.2 size=44999 2024-04-07T16:37:46.086 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.092+0000 7f346a3868c0 1 < completed multipart upload key=EOYFVEI50SQ1 upload=ZZS0XS04SJ2Q tag=0623DCSJ4OPX type=write size=3190727 2024-04-07T16:37:46.090 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.096+0000 7f346a3868c0 1 > prepared operation key=GZXHYZO5SMUP tag=YM991K7AEOBE type=write size=874764 2024-04-07T16:37:46.095 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.100+0000 7f346a3868c0 1 < completed part key=_multipart_QTCIIPTXIRW2.2~T1WCLFASNN6L.0 size=99205 2024-04-07T16:37:46.101 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.108+0000 7f346a3868c0 1 < completed multipart upload key=QTCIIPTXIRW2 upload=T1WCLFASNN6L tag=GGF46KDN2MQ5 type=write size=1147781 2024-04-07T16:37:46.106 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.112+0000 7f346a3868c0 1 > prepared operation key=QVOKSQAGHPA9 tag=PG0WSEE5K2PC type=write size=690952 2024-04-07T16:37:46.110 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.116+0000 7f346a3868c0 1 < completed part key=_multipart_E84U0SX44NQ8.2~Y5LG3GIK61RC.0 size=815153 2024-04-07T16:37:46.117 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.124+0000 7f346a3868c0 1 < completed multipart upload key=E84U0SX44NQ8 upload=Y5LG3GIK61RC tag=J0LY7CUCRAM9 type=write size=1863729 2024-04-07T16:37:46.124 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.132+0000 7f346a3868c0 1 > prepared part key=_multipart_YOK8BAEOMPUR.2~HBV8PLCVOEL7.0 size=1048576 2024-04-07T16:37:46.127 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.136+0000 7f346a3868c0 1 > prepared part key=_multipart_YOK8BAEOMPUR.2~HBV8PLCVOEL7.1 size=1048576 2024-04-07T16:37:46.130 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.136+0000 7f346a3868c0 1 > prepared part key=_multipart_YOK8BAEOMPUR.2~HBV8PLCVOEL7.2 size=974442 2024-04-07T16:37:46.131 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.136+0000 7f346a3868c0 1 > prepared multipart upload key=YOK8BAEOMPUR upload=HBV8PLCVOEL7 tag=7P2S1EJ36YK7 type=write size=4120170 2024-04-07T16:37:46.135 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.144+0000 7f346a3868c0 1 < completed delete operation key=X5Q46GO8YTOK tag=89JMLO4LIT2E type=del 2024-04-07T16:37:46.139 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.148+0000 7f346a3868c0 1 > prepared operation key=NYET3212USVE tag=MHZ0W63TGTZP type=del size=2689695 2024-04-07T16:37:46.143 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.152+0000 7f346a3868c0 1 < completed delete operation key=8F81G0P0YBFW tag=2OY1HL9MGJAG type=del 2024-04-07T16:37:46.147 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.152+0000 7f346a3868c0 1 > prepared operation key=8F81G0P0YBFW tag=LQYNNWEY7P2D type=del size=972452 2024-04-07T16:37:46.151 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.160+0000 7f346a3868c0 1 < canceled operation key=QVOKSQAGHPA9 tag=B2RC5P8RHY09 type=del size=1671278 2024-04-07T16:37:46.155 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.164+0000 7f346a3868c0 1 > prepared operation key=LXOZ0T2KY6OR tag=DDPIDXHNW4MS type=del size=3414050 2024-04-07T16:37:46.159 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.168+0000 7f346a3868c0 1 < completed part key=_multipart_IX3F0IUOKBX1.2~UM9HY7ZU54ZW.0 size=1048576 2024-04-07T16:37:46.162 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.168+0000 7f346a3868c0 1 < completed part key=_multipart_IX3F0IUOKBX1.2~UM9HY7ZU54ZW.1 size=1048576 2024-04-07T16:37:46.166 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.172+0000 7f346a3868c0 1 < completed part key=_multipart_IX3F0IUOKBX1.2~UM9HY7ZU54ZW.2 size=953692 2024-04-07T16:37:46.172 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.180+0000 7f346a3868c0 1 < completed multipart upload key=IX3F0IUOKBX1 upload=UM9HY7ZU54ZW tag=5U9ZPQI44I5J type=write size=4099420 2024-04-07T16:37:46.176 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.184+0000 7f346a3868c0 1 > prepared operation key=3N0QJRYVO3FV tag=CCUB9T14RHQR type=write size=889701 2024-04-07T16:37:46.185 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.192+0000 7f346a3868c0 1 < suggested update operation key=1VBSUWNI3V1T tag=1VRFW5TFRJ28 type=del size=4064273 2024-04-07T16:37:46.189 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.196+0000 7f346a3868c0 1 > prepared operation key=NYET3212USVE tag=KZHUV9DQPE6S type=del size=1343110 2024-04-07T16:37:46.193 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.200+0000 7f346a3868c0 1 < completed part key=_multipart_IP8YW2A75TCW.2~G8ZCBIWKWBKQ.0 size=1048576 2024-04-07T16:37:46.197 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.204+0000 7f346a3868c0 1 < completed part key=_multipart_IP8YW2A75TCW.2~G8ZCBIWKWBKQ.1 size=1048576 2024-04-07T16:37:46.200 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.208+0000 7f346a3868c0 1 < completed part key=_multipart_IP8YW2A75TCW.2~G8ZCBIWKWBKQ.2 size=149458 2024-04-07T16:37:46.207 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.216+0000 7f346a3868c0 1 < completed multipart upload key=IP8YW2A75TCW upload=G8ZCBIWKWBKQ tag=L7IGCJCM0KO8 type=write size=3295186 2024-04-07T16:37:46.212 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.220+0000 7f346a3868c0 1 > prepared operation key=GBQSME7SY1AH tag=V7YEQV4Q2WS8 type=del size=743863 2024-04-07T16:37:46.216 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.224+0000 7f346a3868c0 1 < completed part key=_multipart_QS4HYZFFAJNF.2~4E5TLZYCMZ2W.0 size=951465 2024-04-07T16:37:46.223 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.232+0000 7f346a3868c0 1 < completed multipart upload key=QS4HYZFFAJNF upload=4E5TLZYCMZ2W tag=MO9C30BGSB0I type=write size=2000041 2024-04-07T16:37:46.232 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.240+0000 7f346a3868c0 1 > prepared part key=_multipart_YOK8BAEOMPUR.2~OBQQ5GBWB18V.0 size=1048576 2024-04-07T16:37:46.236 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.244+0000 7f346a3868c0 1 > prepared part key=_multipart_YOK8BAEOMPUR.2~OBQQ5GBWB18V.1 size=384653 2024-04-07T16:37:46.236 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.244+0000 7f346a3868c0 1 > prepared multipart upload key=YOK8BAEOMPUR upload=OBQQ5GBWB18V tag=XW07NYZYVSA0 type=write size=2481805 2024-04-07T16:37:46.240 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.248+0000 7f346a3868c0 1 < completed part key=_multipart_1VKVQXSQMTU1.2~3FSR3RDGNJ03.0 size=734879 2024-04-07T16:37:46.247 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.256+0000 7f346a3868c0 1 < completed multipart upload key=1VKVQXSQMTU1 upload=3FSR3RDGNJ03 tag=EX1J7K6AORSI type=write size=1783455 2024-04-07T16:37:46.255 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.260+0000 7f346a3868c0 1 > prepared part key=_multipart_IX3F0IUOKBX1.2~9V3Z0XTWN3FH.0 size=108169 2024-04-07T16:37:46.255 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.264+0000 7f346a3868c0 1 > prepared multipart upload key=IX3F0IUOKBX1 upload=9V3Z0XTWN3FH tag=CJ2EF61PDK3V type=write size=1156745 2024-04-07T16:37:46.263 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.272+0000 7f346a3868c0 1 < suggested update operation key=IX3F0IUOKBX1 tag=8YL6V64GGX1B type=write size=719425 2024-04-07T16:37:46.271 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.276+0000 7f346a3868c0 1 > prepared part key=_multipart_N14BUXLJONHZ.2~VBPPO4VXWS35.0 size=555779 2024-04-07T16:37:46.271 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.276+0000 7f346a3868c0 1 > prepared multipart upload key=N14BUXLJONHZ upload=VBPPO4VXWS35 tag=KJRFLXFRPU5W type=write size=1604355 2024-04-07T16:37:46.275 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.284+0000 7f346a3868c0 1 < completed part key=_multipart_IX3F0IUOKBX1.2~0TB2NDIRYLTJ.0 size=1048576 2024-04-07T16:37:46.278 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.284+0000 7f346a3868c0 1 < completed part key=_multipart_IX3F0IUOKBX1.2~0TB2NDIRYLTJ.1 size=300623 2024-04-07T16:37:46.285 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.292+0000 7f346a3868c0 1 < completed multipart upload key=IX3F0IUOKBX1 upload=0TB2NDIRYLTJ tag=LCN2Z5CTKATB type=write size=2397775 2024-04-07T16:37:46.289 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.296+0000 7f346a3868c0 1 > prepared operation key=XJLQZ7IS9U1T tag=8BZ0G2XC9OOE type=write size=153426 2024-04-07T16:37:46.293 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.300+0000 7f346a3868c0 1 < completed delete operation key=1WGUPH7X5UA0 tag=TBAGXRG66QB3 type=del 2024-04-07T16:37:46.301 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.308+0000 7f346a3868c0 1 > prepared part key=_multipart_EOYFVEI50SQ1.2~UG60WGK2R24G.0 size=1048576 2024-04-07T16:37:46.304 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.312+0000 7f346a3868c0 1 > prepared part key=_multipart_EOYFVEI50SQ1.2~UG60WGK2R24G.1 size=1048576 2024-04-07T16:37:46.308 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.316+0000 7f346a3868c0 1 > prepared part key=_multipart_EOYFVEI50SQ1.2~UG60WGK2R24G.2 size=290204 2024-04-07T16:37:46.308 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.316+0000 7f346a3868c0 1 > prepared multipart upload key=EOYFVEI50SQ1 upload=UG60WGK2R24G tag=M40IVID9G6ZU type=write size=3435932 2024-04-07T16:37:46.312 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.320+0000 7f346a3868c0 1 < completed part key=_multipart_X5Q46GO8YTOK.2~2DP4IXSD81Q1.0 size=722474 2024-04-07T16:37:46.320 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.328+0000 7f346a3868c0 1 < completed multipart upload key=X5Q46GO8YTOK upload=2DP4IXSD81Q1 tag=RFRQ08EAJS2R type=write size=1771050 2024-04-07T16:37:46.325 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.332+0000 7f346a3868c0 1 > prepared operation key=U3EID5EK64HD tag=CWBYXCT1NKLT type=del size=2710041 2024-04-07T16:37:46.335 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.344+0000 7f346a3868c0 1 < suggested update operation key=ROFH30W3UCAX tag=PSSJLRJUJPJ2 type=del size=4071331 2024-04-07T16:37:46.339 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.348+0000 7f346a3868c0 1 > prepared operation key=EQR0FY86AT5X tag=XOGXUA3DIS2I type=del size=782865 2024-04-07T16:37:46.344 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.352+0000 7f346a3868c0 1 < completed write operation key=GZXHYZO5SMUP tag=YM991K7AEOBE type=write size=874764 2024-04-07T16:37:46.349 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.356+0000 7f346a3868c0 1 > prepared operation key=7G5C6HBK6QWI tag=NK7E1Q5Q2C2X type=write size=54791 2024-04-07T16:37:46.358 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.364+0000 7f346a3868c0 1 < suggested update operation key=QVOKSQAGHPA9 tag=PG0WSEE5K2PC type=write size=690952 2024-04-07T16:37:46.363 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.368+0000 7f346a3868c0 1 > prepared operation key=CL4PP1RKZU0E tag=5RIX7WBFHA3M type=del size=2003590 2024-04-07T16:37:46.368 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.376+0000 7f346a3868c0 1 < completed part key=_multipart_YOK8BAEOMPUR.2~HBV8PLCVOEL7.0 size=1048576 2024-04-07T16:37:46.371 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.380+0000 7f346a3868c0 1 < completed part key=_multipart_YOK8BAEOMPUR.2~HBV8PLCVOEL7.1 size=1048576 2024-04-07T16:37:46.375 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.384+0000 7f346a3868c0 1 < completed part key=_multipart_YOK8BAEOMPUR.2~HBV8PLCVOEL7.2 size=974442 2024-04-07T16:37:46.382 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.388+0000 7f346a3868c0 1 < completed multipart upload key=YOK8BAEOMPUR upload=HBV8PLCVOEL7 tag=7P2S1EJ36YK7 type=write size=4120170 2024-04-07T16:37:46.386 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.392+0000 7f346a3868c0 1 > prepared operation key=YXMDZB23YTB3 tag=OHNRKIL0MWR6 type=del size=610712 2024-04-07T16:37:46.392 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.400+0000 7f346a3868c0 1 < completed delete operation key=NYET3212USVE tag=MHZ0W63TGTZP type=del 2024-04-07T16:37:46.396 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.404+0000 7f346a3868c0 1 > prepared operation key=X5Q46GO8YTOK tag=3TH7BUHE7OQY type=del size=2015081 2024-04-07T16:37:46.399 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.408+0000 7f346a3868c0 1 < completed delete operation key=8F81G0P0YBFW tag=LQYNNWEY7P2D type=del 2024-04-07T16:37:46.407 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.412+0000 7f346a3868c0 1 > prepared part key=_multipart_IP8YW2A75TCW.2~BE63ZKRWMR32.0 size=1048576 2024-04-07T16:37:46.410 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.416+0000 7f346a3868c0 1 > prepared part key=_multipart_IP8YW2A75TCW.2~BE63ZKRWMR32.1 size=1048576 2024-04-07T16:37:46.413 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.420+0000 7f346a3868c0 1 > prepared part key=_multipart_IP8YW2A75TCW.2~BE63ZKRWMR32.2 size=325556 2024-04-07T16:37:46.413 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.420+0000 7f346a3868c0 1 > prepared multipart upload key=IP8YW2A75TCW upload=BE63ZKRWMR32 tag=6I00LY0V8WJO type=write size=3471284 2024-04-07T16:37:46.417 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.424+0000 7f346a3868c0 1 < completed delete operation key=LXOZ0T2KY6OR tag=DDPIDXHNW4MS type=del 2024-04-07T16:37:46.425 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.432+0000 7f346a3868c0 1 > prepared part key=_multipart_UIW3DN4NUYPP.2~PLLAO7EG7J8P.0 size=497506 2024-04-07T16:37:46.425 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.432+0000 7f346a3868c0 1 > prepared multipart upload key=UIW3DN4NUYPP upload=PLLAO7EG7J8P tag=9RBYSKJB6VC9 type=write size=1546082 2024-04-07T16:37:46.433 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.440+0000 7f346a3868c0 1 < suggested remove operation key=3N0QJRYVO3FV tag=CCUB9T14RHQR type=write 2024-04-07T16:37:46.441 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.448+0000 7f346a3868c0 1 > prepared part key=_multipart_8F81G0P0YBFW.2~3B6230S0R2QR.0 size=1048576 2024-04-07T16:37:46.444 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.452+0000 7f346a3868c0 1 > prepared part key=_multipart_8F81G0P0YBFW.2~3B6230S0R2QR.1 size=1048576 2024-04-07T16:37:46.447 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.456+0000 7f346a3868c0 1 > prepared part key=_multipart_8F81G0P0YBFW.2~3B6230S0R2QR.2 size=815255 2024-04-07T16:37:46.447 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.456+0000 7f346a3868c0 1 > prepared multipart upload key=8F81G0P0YBFW upload=3B6230S0R2QR tag=JP16UDG3S43K type=write size=3960983 2024-04-07T16:37:46.451 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.460+0000 7f346a3868c0 1 < canceled operation key=NYET3212USVE tag=KZHUV9DQPE6S type=del size=1343110 2024-04-07T16:37:46.456 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.464+0000 7f346a3868c0 1 > prepared operation key=XJLQZ7IS9U1T tag=VLTPJMJ9YR3G type=write size=501057 2024-04-07T16:37:46.460 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.468+0000 7f346a3868c0 1 < completed delete operation key=GBQSME7SY1AH tag=V7YEQV4Q2WS8 type=del 2024-04-07T16:37:46.468 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.476+0000 7f346a3868c0 1 > prepared part key=_multipart_YOK8BAEOMPUR.2~LGCXLXOB5H77.0 size=1048576 2024-04-07T16:37:46.471 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.480+0000 7f346a3868c0 1 > prepared part key=_multipart_YOK8BAEOMPUR.2~LGCXLXOB5H77.1 size=729619 2024-04-07T16:37:46.471 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.480+0000 7f346a3868c0 1 > prepared multipart upload key=YOK8BAEOMPUR upload=LGCXLXOB5H77 tag=O43UJKZY5UOI type=write size=2826771 2024-04-07T16:37:46.472 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.480+0000 7f346a3868c0 1 < canceled part key=_multipart_YOK8BAEOMPUR.2~OBQQ5GBWB18V.0 size=1048576 2024-04-07T16:37:46.476 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.484+0000 7f346a3868c0 1 < completed part key=_multipart_YOK8BAEOMPUR.2~OBQQ5GBWB18V.1 size=384653 2024-04-07T16:37:46.483 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.492+0000 7f346a3868c0 1 < canceled multipart upload key=YOK8BAEOMPUR upload=OBQQ5GBWB18V tag=XW07NYZYVSA0 type=write size=2481805 2024-04-07T16:37:46.488 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.496+0000 7f346a3868c0 1 > prepared operation key=EQR0FY86AT5X tag=C1396HSLCUVX type=del size=2294907 2024-04-07T16:37:46.493 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.500+0000 7f346a3868c0 1 < completed part key=_multipart_IX3F0IUOKBX1.2~9V3Z0XTWN3FH.0 size=108169 2024-04-07T16:37:46.501 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.508+0000 7f346a3868c0 1 < completed multipart upload key=IX3F0IUOKBX1 upload=9V3Z0XTWN3FH tag=CJ2EF61PDK3V type=write size=1156745 2024-04-07T16:37:46.505 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.512+0000 7f346a3868c0 1 > prepared operation key=1VBSUWNI3V1T tag=AJJGA38DGVOP type=del size=2393024 2024-04-07T16:37:46.509 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.516+0000 7f346a3868c0 1 < completed part key=_multipart_N14BUXLJONHZ.2~VBPPO4VXWS35.0 size=555779 2024-04-07T16:37:46.517 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.524+0000 7f346a3868c0 1 < completed multipart upload key=N14BUXLJONHZ upload=VBPPO4VXWS35 tag=KJRFLXFRPU5W type=write size=1604355 2024-04-07T16:37:46.521 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.528+0000 7f346a3868c0 1 > prepared operation key=IP8YW2A75TCW tag=00CN9VOWLO4F type=del size=2443964 2024-04-07T16:37:46.526 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.532+0000 7f346a3868c0 1 < canceled operation key=XJLQZ7IS9U1T tag=8BZ0G2XC9OOE type=write size=153426 2024-04-07T16:37:46.530 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.536+0000 7f346a3868c0 1 > prepared operation key=7G5C6HBK6QWI tag=3C45VVPHLV3P type=del size=848039 2024-04-07T16:37:46.534 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.540+0000 7f346a3868c0 1 < completed part key=_multipart_EOYFVEI50SQ1.2~UG60WGK2R24G.0 size=1048576 2024-04-07T16:37:46.538 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.544+0000 7f346a3868c0 1 < completed part key=_multipart_EOYFVEI50SQ1.2~UG60WGK2R24G.1 size=1048576 2024-04-07T16:37:46.541 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.548+0000 7f346a3868c0 1 < completed part key=_multipart_EOYFVEI50SQ1.2~UG60WGK2R24G.2 size=290204 2024-04-07T16:37:46.548 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.556+0000 7f346a3868c0 1 < completed multipart upload key=EOYFVEI50SQ1 upload=UG60WGK2R24G tag=M40IVID9G6ZU type=write size=3435932 2024-04-07T16:37:46.556 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.564+0000 7f346a3868c0 1 > prepared part key=_multipart_GZXHYZO5SMUP.2~B25F710PM2RF.0 size=730573 2024-04-07T16:37:46.556 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.564+0000 7f346a3868c0 1 > prepared multipart upload key=GZXHYZO5SMUP upload=B25F710PM2RF tag=6OIR0DGFR6NI type=write size=1779149 2024-04-07T16:37:46.561 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.568+0000 7f346a3868c0 1 < completed delete operation key=U3EID5EK64HD tag=CWBYXCT1NKLT type=del 2024-04-07T16:37:46.565 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.572+0000 7f346a3868c0 1 > prepared operation key=IX3F0IUOKBX1 tag=KPS114JKU5O5 type=del size=3314876 2024-04-07T16:37:46.569 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.576+0000 7f346a3868c0 1 < canceled operation key=EQR0FY86AT5X tag=XOGXUA3DIS2I type=del size=782865 2024-04-07T16:37:46.574 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.580+0000 7f346a3868c0 1 > prepared operation key=NYET3212USVE tag=SN802AGX6ZN5 type=write size=430313 2024-04-07T16:37:46.578 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.584+0000 7f346a3868c0 1 < completed write operation key=7G5C6HBK6QWI tag=NK7E1Q5Q2C2X type=write size=54791 2024-04-07T16:37:46.586 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.592+0000 7f346a3868c0 1 > prepared part key=_multipart_XJLQZ7IS9U1T.2~RD41XCCJBPI5.0 size=1048576 2024-04-07T16:37:46.590 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.596+0000 7f346a3868c0 1 > prepared part key=_multipart_XJLQZ7IS9U1T.2~RD41XCCJBPI5.1 size=539894 2024-04-07T16:37:46.590 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.596+0000 7f346a3868c0 1 > prepared multipart upload key=XJLQZ7IS9U1T upload=RD41XCCJBPI5 tag=64F1WQN15EDY type=write size=2637046 2024-04-07T16:37:46.599 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.608+0000 7f346a3868c0 1 < suggested remove operation key=CL4PP1RKZU0E tag=5RIX7WBFHA3M type=del 2024-04-07T16:37:46.608 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.616+0000 7f346a3868c0 1 > prepared part key=_multipart_6XDS0A7L0IY8.2~6WP4J5JD8M7G.0 size=1048576 2024-04-07T16:37:46.611 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.620+0000 7f346a3868c0 1 > prepared part key=_multipart_6XDS0A7L0IY8.2~6WP4J5JD8M7G.1 size=181374 2024-04-07T16:37:46.611 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.620+0000 7f346a3868c0 1 > prepared multipart upload key=6XDS0A7L0IY8 upload=6WP4J5JD8M7G tag=G7HYPB2MJLOU type=write size=2278526 2024-04-07T16:37:46.616 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.624+0000 7f346a3868c0 1 < completed delete operation key=YXMDZB23YTB3 tag=OHNRKIL0MWR6 type=del 2024-04-07T16:37:46.620 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.628+0000 7f346a3868c0 1 > prepared operation key=QVOKSQAGHPA9 tag=UB7RQ5XD6QAT type=write size=65747 2024-04-07T16:37:46.624 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.632+0000 7f346a3868c0 1 < completed delete operation key=X5Q46GO8YTOK tag=3TH7BUHE7OQY type=del 2024-04-07T16:37:46.628 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.636+0000 7f346a3868c0 1 > prepared operation key=1VBSUWNI3V1T tag=2O3T5SLQ6BEE type=del size=910218 2024-04-07T16:37:46.632 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.640+0000 7f346a3868c0 1 < completed part key=_multipart_IP8YW2A75TCW.2~BE63ZKRWMR32.0 size=1048576 2024-04-07T16:37:46.635 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.644+0000 7f346a3868c0 1 < completed part key=_multipart_IP8YW2A75TCW.2~BE63ZKRWMR32.1 size=1048576 2024-04-07T16:37:46.639 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.648+0000 7f346a3868c0 1 < completed part key=_multipart_IP8YW2A75TCW.2~BE63ZKRWMR32.2 size=325556 2024-04-07T16:37:46.646 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.652+0000 7f346a3868c0 1 < completed multipart upload key=IP8YW2A75TCW upload=BE63ZKRWMR32 tag=6I00LY0V8WJO type=write size=3471284 2024-04-07T16:37:46.653 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.660+0000 7f346a3868c0 1 > prepared part key=_multipart_QVOKSQAGHPA9.2~QH81OY1XSH4K.0 size=620017 2024-04-07T16:37:46.653 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.660+0000 7f346a3868c0 1 > prepared multipart upload key=QVOKSQAGHPA9 upload=QH81OY1XSH4K tag=ZLUMPJJJ8U58 type=write size=1668593 2024-04-07T16:37:46.658 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.664+0000 7f346a3868c0 1 < completed part key=_multipart_UIW3DN4NUYPP.2~PLLAO7EG7J8P.0 size=497506 2024-04-07T16:37:46.664 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.672+0000 7f346a3868c0 1 < completed multipart upload key=UIW3DN4NUYPP upload=PLLAO7EG7J8P tag=9RBYSKJB6VC9 type=write size=1546082 2024-04-07T16:37:46.672 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.680+0000 7f346a3868c0 1 > prepared part key=_multipart_95YVKJ14JD8W.2~8IZLQ9DF8G3M.0 size=1048576 2024-04-07T16:37:46.675 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.684+0000 7f346a3868c0 1 > prepared part key=_multipart_95YVKJ14JD8W.2~8IZLQ9DF8G3M.1 size=1048576 2024-04-07T16:37:46.678 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.684+0000 7f346a3868c0 1 > prepared part key=_multipart_95YVKJ14JD8W.2~8IZLQ9DF8G3M.2 size=26787 2024-04-07T16:37:46.678 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.684+0000 7f346a3868c0 1 > prepared multipart upload key=95YVKJ14JD8W upload=8IZLQ9DF8G3M tag=XPK81F2W52VI type=write size=3172515 2024-04-07T16:37:46.682 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.688+0000 7f346a3868c0 1 < completed part key=_multipart_8F81G0P0YBFW.2~3B6230S0R2QR.0 size=1048576 2024-04-07T16:37:46.685 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.692+0000 7f346a3868c0 1 < completed part key=_multipart_8F81G0P0YBFW.2~3B6230S0R2QR.1 size=1048576 2024-04-07T16:37:46.689 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.696+0000 7f346a3868c0 1 < completed part key=_multipart_8F81G0P0YBFW.2~3B6230S0R2QR.2 size=815255 2024-04-07T16:37:46.696 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.704+0000 7f346a3868c0 1 < completed multipart upload key=8F81G0P0YBFW upload=3B6230S0R2QR tag=JP16UDG3S43K type=write size=3960983 2024-04-07T16:37:46.700 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.708+0000 7f346a3868c0 1 > prepared operation key=3N0QJRYVO3FV tag=GW47L8XGBQD2 type=del size=1666083 2024-04-07T16:37:46.708 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.716+0000 7f346a3868c0 1 < suggested update operation key=XJLQZ7IS9U1T tag=VLTPJMJ9YR3G type=write size=501057 2024-04-07T16:37:46.712 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.720+0000 7f346a3868c0 1 > prepared operation key=YXMDZB23YTB3 tag=WK92MPY6IDBB type=del size=398268 2024-04-07T16:37:46.716 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.724+0000 7f346a3868c0 1 < completed part key=_multipart_YOK8BAEOMPUR.2~LGCXLXOB5H77.0 size=1048576 2024-04-07T16:37:46.719 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.728+0000 7f346a3868c0 1 < completed part key=_multipart_YOK8BAEOMPUR.2~LGCXLXOB5H77.1 size=729619 2024-04-07T16:37:46.726 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.732+0000 7f346a3868c0 1 < completed multipart upload key=YOK8BAEOMPUR upload=LGCXLXOB5H77 tag=O43UJKZY5UOI type=write size=2826771 2024-04-07T16:37:46.733 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.740+0000 7f346a3868c0 1 > prepared part key=_multipart_E84U0SX44NQ8.2~91VLLPJBXHPA.0 size=130553 2024-04-07T16:37:46.734 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.740+0000 7f346a3868c0 1 > prepared multipart upload key=E84U0SX44NQ8 upload=91VLLPJBXHPA tag=KXJ81TYWHJNP type=write size=1179129 2024-04-07T16:37:46.738 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.744+0000 7f346a3868c0 1 < completed delete operation key=EQR0FY86AT5X tag=C1396HSLCUVX type=del 2024-04-07T16:37:46.742 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.748+0000 7f346a3868c0 1 > prepared operation key=7G5C6HBK6QWI tag=GCDLMYCQ6XF7 type=del size=2831941 2024-04-07T16:37:46.748 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.756+0000 7f346a3868c0 1 < completed delete operation key=1VBSUWNI3V1T tag=AJJGA38DGVOP type=del 2024-04-07T16:37:46.757 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.764+0000 7f346a3868c0 1 > prepared part key=_multipart_C1BTVJNHIM0X.2~HEHNCWCKUH1R.0 size=1048576 2024-04-07T16:37:46.761 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.768+0000 7f346a3868c0 1 > prepared part key=_multipart_C1BTVJNHIM0X.2~HEHNCWCKUH1R.1 size=801953 2024-04-07T16:37:46.761 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.768+0000 7f346a3868c0 1 > prepared multipart upload key=C1BTVJNHIM0X upload=HEHNCWCKUH1R tag=608TCQU25U6M type=write size=2899105 2024-04-07T16:37:46.766 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.772+0000 7f346a3868c0 1 < completed delete operation key=IP8YW2A75TCW tag=00CN9VOWLO4F type=del 2024-04-07T16:37:46.774 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.780+0000 7f346a3868c0 1 > prepared part key=_multipart_IP8YW2A75TCW.2~J55UK9EJCSHQ.0 size=1048576 2024-04-07T16:37:46.777 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.784+0000 7f346a3868c0 1 > prepared part key=_multipart_IP8YW2A75TCW.2~J55UK9EJCSHQ.1 size=1048576 2024-04-07T16:37:46.782 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.788+0000 7f346a3868c0 1 > prepared part key=_multipart_IP8YW2A75TCW.2~J55UK9EJCSHQ.2 size=144945 2024-04-07T16:37:46.782 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.788+0000 7f346a3868c0 1 > prepared multipart upload key=IP8YW2A75TCW upload=J55UK9EJCSHQ tag=O2NSMH11OE12 type=write size=3290673 2024-04-07T16:37:46.787 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.796+0000 7f346a3868c0 1 < completed delete operation key=7G5C6HBK6QWI tag=3C45VVPHLV3P type=del 2024-04-07T16:37:46.797 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.804+0000 7f346a3868c0 1 > prepared part key=_multipart_YOK8BAEOMPUR.2~L2UR6THJILOX.0 size=1048576 2024-04-07T16:37:46.800 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.808+0000 7f346a3868c0 1 > prepared part key=_multipart_YOK8BAEOMPUR.2~L2UR6THJILOX.1 size=1048576 2024-04-07T16:37:46.805 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.812+0000 7f346a3868c0 1 > prepared part key=_multipart_YOK8BAEOMPUR.2~L2UR6THJILOX.2 size=772538 2024-04-07T16:37:46.805 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.812+0000 7f346a3868c0 1 > prepared multipart upload key=YOK8BAEOMPUR upload=L2UR6THJILOX tag=IUS3SQMBD6O0 type=write size=3918266 2024-04-07T16:37:46.809 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.816+0000 7f346a3868c0 1 < completed part key=_multipart_GZXHYZO5SMUP.2~B25F710PM2RF.0 size=730573 2024-04-07T16:37:46.818 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.824+0000 7f346a3868c0 1 < completed multipart upload key=GZXHYZO5SMUP upload=B25F710PM2RF tag=6OIR0DGFR6NI type=write size=1779149 2024-04-07T16:37:46.822 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.828+0000 7f346a3868c0 1 > prepared operation key=NYET3212USVE tag=KAGY2NE00LXR type=del size=3981586 2024-04-07T16:37:46.825 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.832+0000 7f346a3868c0 1 < completed delete operation key=IX3F0IUOKBX1 tag=KPS114JKU5O5 type=del 2024-04-07T16:37:46.833 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.840+0000 7f346a3868c0 1 > prepared part key=_multipart_QTCIIPTXIRW2.2~50EPF1904S8L.0 size=1048576 2024-04-07T16:37:46.836 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.844+0000 7f346a3868c0 1 > prepared part key=_multipart_QTCIIPTXIRW2.2~50EPF1904S8L.1 size=179700 2024-04-07T16:37:46.836 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.844+0000 7f346a3868c0 1 > prepared multipart upload key=QTCIIPTXIRW2 upload=50EPF1904S8L tag=2OYSLNBX6KH2 type=write size=2276852 2024-04-07T16:37:46.840 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.848+0000 7f346a3868c0 1 < completed write operation key=NYET3212USVE tag=SN802AGX6ZN5 type=write size=430313 2024-04-07T16:37:46.845 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.852+0000 7f346a3868c0 1 > prepared operation key=3N0QJRYVO3FV tag=SD7T2ZPQMZAV type=del size=2308323 2024-04-07T16:37:46.850 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.856+0000 7f346a3868c0 1 < completed part key=_multipart_XJLQZ7IS9U1T.2~RD41XCCJBPI5.0 size=1048576 2024-04-07T16:37:46.854 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.860+0000 7f346a3868c0 1 < completed part key=_multipart_XJLQZ7IS9U1T.2~RD41XCCJBPI5.1 size=539894 2024-04-07T16:37:46.856 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:37:46 smithi144 bash[66654]: cluster 2024-04-07T16:37:45.056211+0000 mgr.y (mgr.25175) 672 : cluster [DBG] pgmap v1307: 137 pgs: 137 active+clean; 583 KiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 111 KiB/s rd, 39 KiB/s wr, 195 op/s 2024-04-07T16:37:46.860 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.868+0000 7f346a3868c0 1 < completed multipart upload key=XJLQZ7IS9U1T upload=RD41XCCJBPI5 tag=64F1WQN15EDY type=write size=2637046 2024-04-07T16:37:46.869 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.876+0000 7f346a3868c0 1 > prepared part key=_multipart_N14BUXLJONHZ.2~V3ZQZ948VYYP.0 size=1048576 2024-04-07T16:37:46.872 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.880+0000 7f346a3868c0 1 > prepared part key=_multipart_N14BUXLJONHZ.2~V3ZQZ948VYYP.1 size=1048576 2024-04-07T16:37:46.875 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.884+0000 7f346a3868c0 1 > prepared part key=_multipart_N14BUXLJONHZ.2~V3ZQZ948VYYP.2 size=1044303 2024-04-07T16:37:46.876 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.884+0000 7f346a3868c0 1 > prepared multipart upload key=N14BUXLJONHZ upload=V3ZQZ948VYYP tag=4COORHCRU0TP type=write size=4190031 2024-04-07T16:37:46.876 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.884+0000 7f346a3868c0 1 < canceled part key=_multipart_6XDS0A7L0IY8.2~6WP4J5JD8M7G.0 size=1048576 2024-04-07T16:37:46.879 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.888+0000 7f346a3868c0 1 < completed part key=_multipart_6XDS0A7L0IY8.2~6WP4J5JD8M7G.1 size=181374 2024-04-07T16:37:46.886 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.892+0000 7f346a3868c0 1 < canceled multipart upload key=6XDS0A7L0IY8 upload=6WP4J5JD8M7G tag=G7HYPB2MJLOU type=write size=2278526 2024-04-07T16:37:46.893 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.900+0000 7f346a3868c0 1 > prepared part key=_multipart_EQR0FY86AT5X.2~YPL7JEVQS5C2.0 size=1048576 2024-04-07T16:37:46.897 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.904+0000 7f346a3868c0 1 > prepared part key=_multipart_EQR0FY86AT5X.2~YPL7JEVQS5C2.1 size=1048576 2024-04-07T16:37:46.900 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.908+0000 7f346a3868c0 1 > prepared part key=_multipart_EQR0FY86AT5X.2~YPL7JEVQS5C2.2 size=405397 2024-04-07T16:37:46.900 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.908+0000 7f346a3868c0 1 > prepared multipart upload key=EQR0FY86AT5X upload=YPL7JEVQS5C2 tag=LV0VFXP00PVE type=write size=3551125 2024-04-07T16:37:46.904 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.912+0000 7f346a3868c0 1 < completed write operation key=QVOKSQAGHPA9 tag=UB7RQ5XD6QAT type=write size=65747 2024-04-07T16:37:46.908 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.916+0000 7f346a3868c0 1 > prepared operation key=E84U0SX44NQ8 tag=8YWOK0JNGS0A type=del size=1108491 2024-04-07T16:37:46.915 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.920+0000 7f346a3868c0 1 < suggested update operation key=1VBSUWNI3V1T tag=2O3T5SLQ6BEE type=del size=910218 2024-04-07T16:37:46.918 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.924+0000 7f346a3868c0 1 > prepared operation key=7G5C6HBK6QWI tag=9L8J89EYI3U4 type=del size=2750754 2024-04-07T16:37:46.919 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.928+0000 7f346a3868c0 1 < canceled part key=_multipart_QVOKSQAGHPA9.2~QH81OY1XSH4K.0 size=620017 2024-04-07T16:37:46.926 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.932+0000 7f346a3868c0 1 < canceled multipart upload key=QVOKSQAGHPA9 upload=QH81OY1XSH4K tag=ZLUMPJJJ8U58 type=write size=1668593 2024-04-07T16:37:46.934 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.940+0000 7f346a3868c0 1 > prepared part key=_multipart_CL4PP1RKZU0E.2~243HNI1MNE88.0 size=111541 2024-04-07T16:37:46.934 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.940+0000 7f346a3868c0 1 > prepared multipart upload key=CL4PP1RKZU0E upload=243HNI1MNE88 tag=KV6B9XZIS4NI type=write size=1160117 2024-04-07T16:37:46.938 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.944+0000 7f346a3868c0 1 < completed part key=_multipart_95YVKJ14JD8W.2~8IZLQ9DF8G3M.0 size=1048576 2024-04-07T16:37:46.941 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:37:46 smithi100 bash[74942]: cluster 2024-04-07T16:37:45.056211+0000 mgr.y (mgr.25175) 672 : cluster [DBG] pgmap v1307: 137 pgs: 137 active+clean; 583 KiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 111 KiB/s rd, 39 KiB/s wr, 195 op/s 2024-04-07T16:37:46.941 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:37:46 smithi100 bash[76202]: cluster 2024-04-07T16:37:45.056211+0000 mgr.y (mgr.25175) 672 : cluster [DBG] pgmap v1307: 137 pgs: 137 active+clean; 583 KiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 111 KiB/s rd, 39 KiB/s wr, 195 op/s 2024-04-07T16:37:46.942 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.948+0000 7f346a3868c0 1 < completed part key=_multipart_95YVKJ14JD8W.2~8IZLQ9DF8G3M.1 size=1048576 2024-04-07T16:37:46.945 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.952+0000 7f346a3868c0 1 < completed part key=_multipart_95YVKJ14JD8W.2~8IZLQ9DF8G3M.2 size=26787 2024-04-07T16:37:46.952 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.960+0000 7f346a3868c0 1 < completed multipart upload key=95YVKJ14JD8W upload=8IZLQ9DF8G3M tag=XPK81F2W52VI type=write size=3172515 2024-04-07T16:37:46.956 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.964+0000 7f346a3868c0 1 > prepared operation key=GZXHYZO5SMUP tag=2GS760HXSYA4 type=write size=146558 2024-04-07T16:37:46.964 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.972+0000 7f346a3868c0 1 < suggested update operation key=3N0QJRYVO3FV tag=GW47L8XGBQD2 type=del size=1666083 2024-04-07T16:37:46.972 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.980+0000 7f346a3868c0 1 > prepared part key=_multipart_EQR0FY86AT5X.2~G00WZM6QAIJB.0 size=1048576 2024-04-07T16:37:46.976 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.984+0000 7f346a3868c0 1 > prepared part key=_multipart_EQR0FY86AT5X.2~G00WZM6QAIJB.1 size=1028526 2024-04-07T16:37:46.976 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.984+0000 7f346a3868c0 1 > prepared multipart upload key=EQR0FY86AT5X upload=G00WZM6QAIJB tag=E5J63MJPKO5N type=write size=3125678 2024-04-07T16:37:46.980 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.988+0000 7f346a3868c0 1 < completed delete operation key=YXMDZB23YTB3 tag=WK92MPY6IDBB type=del 2024-04-07T16:37:46.985 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.992+0000 7f346a3868c0 1 > prepared operation key=HBLEXONKXJTL tag=YXDLL5CXO6I8 type=del size=506015 2024-04-07T16:37:46.990 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:46.996+0000 7f346a3868c0 1 < completed part key=_multipart_E84U0SX44NQ8.2~91VLLPJBXHPA.0 size=130553 2024-04-07T16:37:46.998 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.004+0000 7f346a3868c0 1 < completed multipart upload key=E84U0SX44NQ8 upload=91VLLPJBXHPA tag=KXJ81TYWHJNP type=write size=1179129 2024-04-07T16:37:47.005 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.012+0000 7f346a3868c0 1 > prepared part key=_multipart_GBQSME7SY1AH.2~X100XJGEVLP3.0 size=241910 2024-04-07T16:37:47.005 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.012+0000 7f346a3868c0 1 > prepared multipart upload key=GBQSME7SY1AH upload=X100XJGEVLP3 tag=DLEFXZGE50EX type=write size=1290486 2024-04-07T16:37:47.010 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.016+0000 7f346a3868c0 1 < canceled operation key=7G5C6HBK6QWI tag=GCDLMYCQ6XF7 type=del size=2831941 2024-04-07T16:37:47.017 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.024+0000 7f346a3868c0 1 > prepared part key=_multipart_1WGUPH7X5UA0.2~PEES8WJ25CM1.0 size=1048576 2024-04-07T16:37:47.020 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.028+0000 7f346a3868c0 1 > prepared part key=_multipart_1WGUPH7X5UA0.2~PEES8WJ25CM1.1 size=1048576 2024-04-07T16:37:47.024 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.032+0000 7f346a3868c0 1 > prepared part key=_multipart_1WGUPH7X5UA0.2~PEES8WJ25CM1.2 size=187880 2024-04-07T16:37:47.024 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.032+0000 7f346a3868c0 1 > prepared multipart upload key=1WGUPH7X5UA0 upload=PEES8WJ25CM1 tag=6VWOKPBP0AY3 type=write size=3333608 2024-04-07T16:37:47.025 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.032+0000 7f346a3868c0 1 < canceled part key=_multipart_C1BTVJNHIM0X.2~HEHNCWCKUH1R.0 size=1048576 2024-04-07T16:37:47.028 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.036+0000 7f346a3868c0 1 < completed part key=_multipart_C1BTVJNHIM0X.2~HEHNCWCKUH1R.1 size=801953 2024-04-07T16:37:47.034 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.040+0000 7f346a3868c0 1 < canceled multipart upload key=C1BTVJNHIM0X upload=HEHNCWCKUH1R tag=608TCQU25U6M type=write size=2899105 2024-04-07T16:37:47.042 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.048+0000 7f346a3868c0 1 > prepared part key=_multipart_QS4HYZFFAJNF.2~PELVZARMTKFG.0 size=1048576 2024-04-07T16:37:47.045 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.052+0000 7f346a3868c0 1 > prepared part key=_multipart_QS4HYZFFAJNF.2~PELVZARMTKFG.1 size=86972 2024-04-07T16:37:47.045 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.052+0000 7f346a3868c0 1 > prepared multipart upload key=QS4HYZFFAJNF upload=PELVZARMTKFG tag=OPOVPCOXR84R type=write size=2184124 2024-04-07T16:37:47.049 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.056+0000 7f346a3868c0 1 < completed part key=_multipart_IP8YW2A75TCW.2~J55UK9EJCSHQ.0 size=1048576 2024-04-07T16:37:47.052 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.060+0000 7f346a3868c0 1 < completed part key=_multipart_IP8YW2A75TCW.2~J55UK9EJCSHQ.1 size=1048576 2024-04-07T16:37:47.055 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.064+0000 7f346a3868c0 1 < completed part key=_multipart_IP8YW2A75TCW.2~J55UK9EJCSHQ.2 size=144945 2024-04-07T16:37:47.062 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.068+0000 7f346a3868c0 1 < completed multipart upload key=IP8YW2A75TCW upload=J55UK9EJCSHQ tag=O2NSMH11OE12 type=write size=3290673 2024-04-07T16:37:47.066 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.072+0000 7f346a3868c0 1 > prepared operation key=QTCIIPTXIRW2 tag=BG4FPMNB1C24 type=write size=629229 2024-04-07T16:37:47.070 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.076+0000 7f346a3868c0 1 < completed part key=_multipart_YOK8BAEOMPUR.2~L2UR6THJILOX.0 size=1048576 2024-04-07T16:37:47.073 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.080+0000 7f346a3868c0 1 < completed part key=_multipart_YOK8BAEOMPUR.2~L2UR6THJILOX.1 size=1048576 2024-04-07T16:37:47.077 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.084+0000 7f346a3868c0 1 < completed part key=_multipart_YOK8BAEOMPUR.2~L2UR6THJILOX.2 size=772538 2024-04-07T16:37:47.083 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.092+0000 7f346a3868c0 1 < completed multipart upload key=YOK8BAEOMPUR upload=L2UR6THJILOX tag=IUS3SQMBD6O0 type=write size=3918266 2024-04-07T16:37:47.088 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.096+0000 7f346a3868c0 1 > prepared operation key=IX3F0IUOKBX1 tag=2QCQKL4Q5UI4 type=write size=961796 2024-04-07T16:37:47.091 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.100+0000 7f346a3868c0 1 < completed delete operation key=NYET3212USVE tag=KAGY2NE00LXR type=del 2024-04-07T16:37:47.099 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.108+0000 7f346a3868c0 1 > prepared part key=_multipart_3N0QJRYVO3FV.2~ZCO6B57Q9PX5.0 size=1048576 2024-04-07T16:37:47.102 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.108+0000 7f346a3868c0 1 > prepared part key=_multipart_3N0QJRYVO3FV.2~ZCO6B57Q9PX5.1 size=1048576 2024-04-07T16:37:47.105 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.112+0000 7f346a3868c0 1 > prepared part key=_multipart_3N0QJRYVO3FV.2~ZCO6B57Q9PX5.2 size=497301 2024-04-07T16:37:47.105 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.112+0000 7f346a3868c0 1 > prepared multipart upload key=3N0QJRYVO3FV upload=ZCO6B57Q9PX5 tag=JZLIXONNCXG4 type=write size=3643029 2024-04-07T16:37:47.109 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.116+0000 7f346a3868c0 1 < completed part key=_multipart_QTCIIPTXIRW2.2~50EPF1904S8L.0 size=1048576 2024-04-07T16:37:47.113 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.120+0000 7f346a3868c0 1 < completed part key=_multipart_QTCIIPTXIRW2.2~50EPF1904S8L.1 size=179700 2024-04-07T16:37:47.120 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.128+0000 7f346a3868c0 1 < completed multipart upload key=QTCIIPTXIRW2 upload=50EPF1904S8L tag=2OYSLNBX6KH2 type=write size=2276852 2024-04-07T16:37:47.128 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.136+0000 7f346a3868c0 1 > prepared part key=_multipart_UIW3DN4NUYPP.2~XHQ0801WTBX6.0 size=943959 2024-04-07T16:37:47.128 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.136+0000 7f346a3868c0 1 > prepared multipart upload key=UIW3DN4NUYPP upload=XHQ0801WTBX6 tag=UW0KGSA3HQP6 type=write size=1992535 2024-04-07T16:37:47.133 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.140+0000 7f346a3868c0 -1 < failed to complete operation key=3N0QJRYVO3FV tag=SD7T2ZPQMZAV type=del size=2308323: (22) Invalid argument 2024-04-07T16:37:47.137 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.144+0000 7f346a3868c0 1 > prepared operation key=GZXHYZO5SMUP tag=MMY5PZCY1YUP type=write size=730050 2024-04-07T16:37:47.141 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.148+0000 7f346a3868c0 1 < completed part key=_multipart_N14BUXLJONHZ.2~V3ZQZ948VYYP.0 size=1048576 2024-04-07T16:37:47.144 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.152+0000 7f346a3868c0 1 < completed part key=_multipart_N14BUXLJONHZ.2~V3ZQZ948VYYP.1 size=1048576 2024-04-07T16:37:47.144 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.152+0000 7f346a3868c0 1 < canceled part key=_multipart_N14BUXLJONHZ.2~V3ZQZ948VYYP.2 size=1044303 2024-04-07T16:37:47.151 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.160+0000 7f346a3868c0 1 < canceled multipart upload key=N14BUXLJONHZ upload=V3ZQZ948VYYP tag=4COORHCRU0TP type=write size=4190031 2024-04-07T16:37:47.156 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.164+0000 7f346a3868c0 1 > prepared operation key=6XDS0A7L0IY8 tag=B1JG24NTWMSW type=write size=636868 2024-04-07T16:37:47.156 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.164+0000 7f346a3868c0 1 < canceled part key=_multipart_EQR0FY86AT5X.2~YPL7JEVQS5C2.0 size=1048576 2024-04-07T16:37:47.160 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.168+0000 7f346a3868c0 1 < completed part key=_multipart_EQR0FY86AT5X.2~YPL7JEVQS5C2.1 size=1048576 2024-04-07T16:37:47.164 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.172+0000 7f346a3868c0 1 < completed part key=_multipart_EQR0FY86AT5X.2~YPL7JEVQS5C2.2 size=405397 2024-04-07T16:37:47.170 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.176+0000 7f346a3868c0 1 < canceled multipart upload key=EQR0FY86AT5X upload=YPL7JEVQS5C2 tag=LV0VFXP00PVE type=write size=3551125 2024-04-07T16:37:47.174 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.180+0000 7f346a3868c0 1 > prepared operation key=XJLQZ7IS9U1T tag=D2U1YY0XE5PD type=del size=2620048 2024-04-07T16:37:47.182 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.188+0000 7f346a3868c0 1 < suggested remove operation key=E84U0SX44NQ8 tag=8YWOK0JNGS0A type=del 2024-04-07T16:37:47.187 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.196+0000 7f346a3868c0 1 > prepared operation key=QTCIIPTXIRW2 tag=1OEEZ804OMXU type=del size=2499102 2024-04-07T16:37:47.191 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.200+0000 7f346a3868c0 1 < completed delete operation key=7G5C6HBK6QWI tag=9L8J89EYI3U4 type=del 2024-04-07T16:37:47.195 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.204+0000 7f346a3868c0 1 > prepared operation key=X5Q46GO8YTOK tag=9OA6ZFQQ7ADU type=del size=2528697 2024-04-07T16:37:47.200 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.208+0000 7f346a3868c0 1 < completed part key=_multipart_CL4PP1RKZU0E.2~243HNI1MNE88.0 size=111541 2024-04-07T16:37:47.207 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.212+0000 7f346a3868c0 1 < completed multipart upload key=CL4PP1RKZU0E upload=243HNI1MNE88 tag=KV6B9XZIS4NI type=write size=1160117 2024-04-07T16:37:47.215 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.220+0000 7f346a3868c0 1 > prepared part key=_multipart_N14BUXLJONHZ.2~DM1BPGEV2ZVQ.0 size=1048576 2024-04-07T16:37:47.218 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.224+0000 7f346a3868c0 1 > prepared part key=_multipart_N14BUXLJONHZ.2~DM1BPGEV2ZVQ.1 size=358533 2024-04-07T16:37:47.218 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.224+0000 7f346a3868c0 1 > prepared multipart upload key=N14BUXLJONHZ upload=DM1BPGEV2ZVQ tag=6QIK4XN618LV type=write size=2455685 2024-04-07T16:37:47.222 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.228+0000 7f346a3868c0 1 < completed write operation key=GZXHYZO5SMUP tag=2GS760HXSYA4 type=write size=146558 2024-04-07T16:37:47.226 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.232+0000 7f346a3868c0 1 > prepared operation key=6XDS0A7L0IY8 tag=SGCAIR40JIF3 type=del size=3665833 2024-04-07T16:37:47.230 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.236+0000 7f346a3868c0 1 < completed part key=_multipart_EQR0FY86AT5X.2~G00WZM6QAIJB.0 size=1048576 2024-04-07T16:37:47.234 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.240+0000 7f346a3868c0 1 < completed part key=_multipart_EQR0FY86AT5X.2~G00WZM6QAIJB.1 size=1028526 2024-04-07T16:37:47.240 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.248+0000 7f346a3868c0 1 < completed multipart upload key=EQR0FY86AT5X upload=G00WZM6QAIJB tag=E5J63MJPKO5N type=write size=3125678 2024-04-07T16:37:47.244 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.252+0000 7f346a3868c0 1 > prepared operation key=HBLEXONKXJTL tag=88KPHLNRFMPX type=del size=3348085 2024-04-07T16:37:47.249 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.256+0000 7f346a3868c0 1 < completed delete operation key=HBLEXONKXJTL tag=YXDLL5CXO6I8 type=del 2024-04-07T16:37:47.253 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.260+0000 7f346a3868c0 1 > prepared operation key=NYET3212USVE tag=8D6KD6QFF5BO type=write size=870528 2024-04-07T16:37:47.257 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.264+0000 7f346a3868c0 1 < completed part key=_multipart_GBQSME7SY1AH.2~X100XJGEVLP3.0 size=241910 2024-04-07T16:37:47.264 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.272+0000 7f346a3868c0 1 < completed multipart upload key=GBQSME7SY1AH upload=X100XJGEVLP3 tag=DLEFXZGE50EX type=write size=1290486 2024-04-07T16:37:47.268 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.276+0000 7f346a3868c0 1 > prepared operation key=GZXHYZO5SMUP tag=NVI5D2SZ8YIX type=write size=720951 2024-04-07T16:37:47.272 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.280+0000 7f346a3868c0 1 < completed part key=_multipart_1WGUPH7X5UA0.2~PEES8WJ25CM1.0 size=1048576 2024-04-07T16:37:47.272 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.280+0000 7f346a3868c0 1 < canceled part key=_multipart_1WGUPH7X5UA0.2~PEES8WJ25CM1.1 size=1048576 2024-04-07T16:37:47.275 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.284+0000 7f346a3868c0 1 < completed part key=_multipart_1WGUPH7X5UA0.2~PEES8WJ25CM1.2 size=187880 2024-04-07T16:37:47.282 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.288+0000 7f346a3868c0 1 < canceled multipart upload key=1WGUPH7X5UA0 upload=PEES8WJ25CM1 tag=6VWOKPBP0AY3 type=write size=3333608 2024-04-07T16:37:47.290 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.296+0000 7f346a3868c0 1 > prepared part key=_multipart_NYET3212USVE.2~KWHI0SWDIX68.0 size=1048576 2024-04-07T16:37:47.293 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.300+0000 7f346a3868c0 1 > prepared part key=_multipart_NYET3212USVE.2~KWHI0SWDIX68.1 size=1048576 2024-04-07T16:37:47.296 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.304+0000 7f346a3868c0 1 > prepared part key=_multipart_NYET3212USVE.2~KWHI0SWDIX68.2 size=740558 2024-04-07T16:37:47.296 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.304+0000 7f346a3868c0 1 > prepared multipart upload key=NYET3212USVE upload=KWHI0SWDIX68 tag=2JEE5TLEA70B type=write size=3886286 2024-04-07T16:37:47.301 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.308+0000 7f346a3868c0 1 < completed part key=_multipart_QS4HYZFFAJNF.2~PELVZARMTKFG.0 size=1048576 2024-04-07T16:37:47.304 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.312+0000 7f346a3868c0 1 < completed part key=_multipart_QS4HYZFFAJNF.2~PELVZARMTKFG.1 size=86972 2024-04-07T16:37:47.312 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.320+0000 7f346a3868c0 1 < completed multipart upload key=QS4HYZFFAJNF upload=PELVZARMTKFG tag=OPOVPCOXR84R type=write size=2184124 2024-04-07T16:37:47.320 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.328+0000 7f346a3868c0 1 > prepared part key=_multipart_YOK8BAEOMPUR.2~YLPR2KYMOU1H.0 size=493738 2024-04-07T16:37:47.320 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.328+0000 7f346a3868c0 1 > prepared multipart upload key=YOK8BAEOMPUR upload=YLPR2KYMOU1H tag=2N3GKDFIUNFA type=write size=1542314 2024-04-07T16:37:47.329 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.336+0000 7f346a3868c0 1 < suggested update operation key=QTCIIPTXIRW2 tag=BG4FPMNB1C24 type=write size=629229 2024-04-07T16:37:47.334 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.340+0000 7f346a3868c0 1 > prepared operation key=QS4HYZFFAJNF tag=504J5SQNCGBX type=del size=1267973 2024-04-07T16:37:47.340 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.348+0000 7f346a3868c0 1 < completed write operation key=IX3F0IUOKBX1 tag=2QCQKL4Q5UI4 type=write size=961796 2024-04-07T16:37:47.348 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.356+0000 7f346a3868c0 1 > prepared part key=_multipart_UIW3DN4NUYPP.2~1U4U2JJBB5SN.0 size=84837 2024-04-07T16:37:47.348 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.356+0000 7f346a3868c0 1 > prepared multipart upload key=UIW3DN4NUYPP upload=1U4U2JJBB5SN tag=AZ56UWM8IBSL type=write size=1133413 2024-04-07T16:37:47.353 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.360+0000 7f346a3868c0 1 < completed part key=_multipart_3N0QJRYVO3FV.2~ZCO6B57Q9PX5.0 size=1048576 2024-04-07T16:37:47.357 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.364+0000 7f346a3868c0 1 < completed part key=_multipart_3N0QJRYVO3FV.2~ZCO6B57Q9PX5.1 size=1048576 2024-04-07T16:37:47.360 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.368+0000 7f346a3868c0 1 < completed part key=_multipart_3N0QJRYVO3FV.2~ZCO6B57Q9PX5.2 size=497301 2024-04-07T16:37:47.367 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.376+0000 7f346a3868c0 1 < completed multipart upload key=3N0QJRYVO3FV upload=ZCO6B57Q9PX5 tag=JZLIXONNCXG4 type=write size=3643029 2024-04-07T16:37:47.371 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.380+0000 7f346a3868c0 1 > prepared operation key=XJLQZ7IS9U1T tag=TZKRE31Q7FHL type=del size=2580157 2024-04-07T16:37:47.375 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.384+0000 7f346a3868c0 1 < completed part key=_multipart_UIW3DN4NUYPP.2~XHQ0801WTBX6.0 size=943959 2024-04-07T16:37:47.382 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.388+0000 7f346a3868c0 1 < completed multipart upload key=UIW3DN4NUYPP upload=XHQ0801WTBX6 tag=UW0KGSA3HQP6 type=write size=1992535 2024-04-07T16:37:47.389 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.396+0000 7f346a3868c0 1 > prepared part key=_multipart_U3EID5EK64HD.2~AWK1VHTUXYLG.0 size=1048576 2024-04-07T16:37:47.392 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.400+0000 7f346a3868c0 1 > prepared part key=_multipart_U3EID5EK64HD.2~AWK1VHTUXYLG.1 size=543157 2024-04-07T16:37:47.392 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.400+0000 7f346a3868c0 1 > prepared multipart upload key=U3EID5EK64HD upload=AWK1VHTUXYLG tag=FDZFSN6Y8Z48 type=write size=2640309 2024-04-07T16:37:47.396 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.404+0000 7f346a3868c0 1 < completed write operation key=GZXHYZO5SMUP tag=MMY5PZCY1YUP type=write size=730050 2024-04-07T16:37:47.400 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.408+0000 7f346a3868c0 1 > prepared operation key=HBLEXONKXJTL tag=3PDK3CJ38NC2 type=del size=842136 2024-04-07T16:37:47.404 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.412+0000 7f346a3868c0 1 < completed write operation key=6XDS0A7L0IY8 tag=B1JG24NTWMSW type=write size=636868 2024-04-07T16:37:47.408 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.416+0000 7f346a3868c0 1 > prepared operation key=C1BTVJNHIM0X tag=42090A4XAH2R type=del size=2321479 2024-04-07T16:37:47.416 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.424+0000 7f346a3868c0 1 < suggested remove operation key=XJLQZ7IS9U1T tag=D2U1YY0XE5PD type=del 2024-04-07T16:37:47.420 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.428+0000 7f346a3868c0 1 > prepared operation key=ROFH30W3UCAX tag=LH761BIRFT8M type=del size=3071279 2024-04-07T16:37:47.424 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.432+0000 7f346a3868c0 -1 < failed to complete operation key=QTCIIPTXIRW2 tag=1OEEZ804OMXU type=del size=2499102: (22) Invalid argument 2024-04-07T16:37:47.429 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.436+0000 7f346a3868c0 1 > prepared operation key=CL4PP1RKZU0E tag=8HT8I4JQ18SD type=del size=3009498 2024-04-07T16:37:47.433 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.440+0000 7f346a3868c0 1 < completed delete operation key=X5Q46GO8YTOK tag=9OA6ZFQQ7ADU type=del 2024-04-07T16:37:47.438 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.444+0000 7f346a3868c0 1 > prepared operation key=95YVKJ14JD8W tag=QNC4XZ925OU5 type=del size=91690 2024-04-07T16:37:47.443 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.448+0000 7f346a3868c0 1 < completed part key=_multipart_N14BUXLJONHZ.2~DM1BPGEV2ZVQ.0 size=1048576 2024-04-07T16:37:47.446 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.452+0000 7f346a3868c0 1 < completed part key=_multipart_N14BUXLJONHZ.2~DM1BPGEV2ZVQ.1 size=358533 2024-04-07T16:37:47.453 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.460+0000 7f346a3868c0 1 < completed multipart upload key=N14BUXLJONHZ upload=DM1BPGEV2ZVQ tag=6QIK4XN618LV type=write size=2455685 2024-04-07T16:37:47.461 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.468+0000 7f346a3868c0 1 > prepared part key=_multipart_HBLEXONKXJTL.2~G615N9W6JX33.0 size=1048576 2024-04-07T16:37:47.464 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.472+0000 7f346a3868c0 1 > prepared part key=_multipart_HBLEXONKXJTL.2~G615N9W6JX33.1 size=599214 2024-04-07T16:37:47.464 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.472+0000 7f346a3868c0 1 > prepared multipart upload key=HBLEXONKXJTL upload=G615N9W6JX33 tag=I2AUHW75P062 type=write size=2696366 2024-04-07T16:37:47.468 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.476+0000 7f346a3868c0 1 < completed delete operation key=6XDS0A7L0IY8 tag=SGCAIR40JIF3 type=del 2024-04-07T16:37:47.473 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.480+0000 7f346a3868c0 1 > prepared operation key=1VBSUWNI3V1T tag=V9F4UMD579SM type=del size=3903922 2024-04-07T16:37:47.477 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.484+0000 7f346a3868c0 1 < completed delete operation key=HBLEXONKXJTL tag=88KPHLNRFMPX type=del 2024-04-07T16:37:47.482 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.488+0000 7f346a3868c0 1 > prepared operation key=DXTO3AJ429AC tag=JY3DGC3RXR19 type=del size=4072345 2024-04-07T16:37:47.486 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.492+0000 7f346a3868c0 1 < completed write operation key=NYET3212USVE tag=8D6KD6QFF5BO type=write size=870528 2024-04-07T16:37:47.491 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.500+0000 7f346a3868c0 1 > prepared operation key=E84U0SX44NQ8 tag=3GZV2CKLLLCA type=del size=3059467 2024-04-07T16:37:47.496 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.504+0000 7f346a3868c0 1 < completed write operation key=GZXHYZO5SMUP tag=NVI5D2SZ8YIX type=write size=720951 2024-04-07T16:37:47.500 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.508+0000 7f346a3868c0 1 > prepared operation key=QVOKSQAGHPA9 tag=80JB04MOJF5K type=del size=3542822 2024-04-07T16:37:47.504 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.512+0000 7f346a3868c0 1 < completed part key=_multipart_NYET3212USVE.2~KWHI0SWDIX68.0 size=1048576 2024-04-07T16:37:47.507 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.512+0000 7f346a3868c0 1 < completed part key=_multipart_NYET3212USVE.2~KWHI0SWDIX68.1 size=1048576 2024-04-07T16:37:47.510 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.516+0000 7f346a3868c0 1 < completed part key=_multipart_NYET3212USVE.2~KWHI0SWDIX68.2 size=740558 2024-04-07T16:37:47.516 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.524+0000 7f346a3868c0 1 < completed multipart upload key=NYET3212USVE upload=KWHI0SWDIX68 tag=2JEE5TLEA70B type=write size=3886286 2024-04-07T16:37:47.520 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.528+0000 7f346a3868c0 1 > prepared operation key=8F81G0P0YBFW tag=O7O9SKQMQA4J type=del size=428739 2024-04-07T16:37:47.523 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.532+0000 7f346a3868c0 1 < completed part key=_multipart_YOK8BAEOMPUR.2~YLPR2KYMOU1H.0 size=493738 2024-04-07T16:37:47.529 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.536+0000 7f346a3868c0 1 < completed multipart upload key=YOK8BAEOMPUR upload=YLPR2KYMOU1H tag=2N3GKDFIUNFA type=write size=1542314 2024-04-07T16:37:47.533 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.540+0000 7f346a3868c0 1 > prepared operation key=EQR0FY86AT5X tag=2FL321V4ZB83 type=write size=586588 2024-04-07T16:37:47.538 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.544+0000 7f346a3868c0 1 < completed delete operation key=QS4HYZFFAJNF tag=504J5SQNCGBX type=del 2024-04-07T16:37:47.542 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.548+0000 7f346a3868c0 1 > prepared operation key=E84U0SX44NQ8 tag=VUFVYJXUGG4G type=del size=1490320 2024-04-07T16:37:47.546 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.552+0000 7f346a3868c0 1 < completed part key=_multipart_UIW3DN4NUYPP.2~1U4U2JJBB5SN.0 size=84837 2024-04-07T16:37:47.553 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.560+0000 7f346a3868c0 1 < completed multipart upload key=UIW3DN4NUYPP upload=1U4U2JJBB5SN tag=AZ56UWM8IBSL type=write size=1133413 2024-04-07T16:37:47.559 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.568+0000 7f346a3868c0 1 > prepared part key=_multipart_6XDS0A7L0IY8.2~Q17Q5NO3V8VE.0 size=440834 2024-04-07T16:37:47.560 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.568+0000 7f346a3868c0 1 > prepared multipart upload key=6XDS0A7L0IY8 upload=Q17Q5NO3V8VE tag=0DTCTYVZQJUD type=write size=1489410 2024-04-07T16:37:47.564 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.572+0000 7f346a3868c0 -1 < failed to complete operation key=XJLQZ7IS9U1T tag=TZKRE31Q7FHL type=del size=2580157: (22) Invalid argument 2024-04-07T16:37:47.571 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.576+0000 7f346a3868c0 1 > prepared part key=_multipart_U3EID5EK64HD.2~LTP42EWTB902.0 size=1048576 2024-04-07T16:37:47.574 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.580+0000 7f346a3868c0 1 > prepared part key=_multipart_U3EID5EK64HD.2~LTP42EWTB902.1 size=768346 2024-04-07T16:37:47.574 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.580+0000 7f346a3868c0 1 > prepared multipart upload key=U3EID5EK64HD upload=LTP42EWTB902 tag=B7YMZ3GYNOP0 type=write size=2865498 2024-04-07T16:37:47.578 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.584+0000 7f346a3868c0 1 < completed part key=_multipart_U3EID5EK64HD.2~AWK1VHTUXYLG.0 size=1048576 2024-04-07T16:37:47.581 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.588+0000 7f346a3868c0 1 < completed part key=_multipart_U3EID5EK64HD.2~AWK1VHTUXYLG.1 size=543157 2024-04-07T16:37:47.588 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.596+0000 7f346a3868c0 1 < completed multipart upload key=U3EID5EK64HD upload=AWK1VHTUXYLG tag=FDZFSN6Y8Z48 type=write size=2640309 2024-04-07T16:37:47.592 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.600+0000 7f346a3868c0 1 > prepared operation key=E84U0SX44NQ8 tag=L33YNSCONNZN type=del size=1273088 2024-04-07T16:37:47.597 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.604+0000 7f346a3868c0 1 < completed delete operation key=HBLEXONKXJTL tag=3PDK3CJ38NC2 type=del 2024-04-07T16:37:47.601 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.608+0000 7f346a3868c0 1 > prepared operation key=ROFH30W3UCAX tag=YXYF0BEE5JDX type=del size=1227311 2024-04-07T16:37:47.606 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.612+0000 7f346a3868c0 1 < completed delete operation key=C1BTVJNHIM0X tag=42090A4XAH2R type=del 2024-04-07T16:37:47.614 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.620+0000 7f346a3868c0 1 > prepared part key=_multipart_6XDS0A7L0IY8.2~U3ITPRNTD64W.0 size=1048576 2024-04-07T16:37:47.617 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.624+0000 7f346a3868c0 1 > prepared part key=_multipart_6XDS0A7L0IY8.2~U3ITPRNTD64W.1 size=1048576 2024-04-07T16:37:47.621 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.628+0000 7f346a3868c0 1 > prepared part key=_multipart_6XDS0A7L0IY8.2~U3ITPRNTD64W.2 size=78862 2024-04-07T16:37:47.621 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.628+0000 7f346a3868c0 1 > prepared multipart upload key=6XDS0A7L0IY8 upload=U3ITPRNTD64W tag=TEO6HM3O63T4 type=write size=3224590 2024-04-07T16:37:47.626 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.632+0000 7f346a3868c0 1 < canceled operation key=ROFH30W3UCAX tag=LH761BIRFT8M type=del size=3071279 2024-04-07T16:37:47.630 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.636+0000 7f346a3868c0 1 > prepared operation key=6XDS0A7L0IY8 tag=EPUIKZ84ZJPI type=del size=211857 2024-04-07T16:37:47.638 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.644+0000 7f346a3868c0 1 < suggested remove operation key=CL4PP1RKZU0E tag=8HT8I4JQ18SD type=del 2024-04-07T16:37:47.642 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.648+0000 7f346a3868c0 1 > prepared operation key=8F81G0P0YBFW tag=5VV8XU0JBFNC type=del size=4133755 2024-04-07T16:37:47.647 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.652+0000 7f346a3868c0 1 < canceled operation key=95YVKJ14JD8W tag=QNC4XZ925OU5 type=del size=91690 2024-04-07T16:37:47.654 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.660+0000 7f346a3868c0 1 > prepared part key=_multipart_1VBSUWNI3V1T.2~7OJHCJLFQONL.0 size=1048576 2024-04-07T16:37:47.657 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.664+0000 7f346a3868c0 1 > prepared part key=_multipart_1VBSUWNI3V1T.2~7OJHCJLFQONL.1 size=738535 2024-04-07T16:37:47.658 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.664+0000 7f346a3868c0 1 > prepared multipart upload key=1VBSUWNI3V1T upload=7OJHCJLFQONL tag=1194KTRDMBRH type=write size=2835687 2024-04-07T16:37:47.662 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.668+0000 7f346a3868c0 1 < completed part key=_multipart_HBLEXONKXJTL.2~G615N9W6JX33.0 size=1048576 2024-04-07T16:37:47.666 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.672+0000 7f346a3868c0 1 < completed part key=_multipart_HBLEXONKXJTL.2~G615N9W6JX33.1 size=599214 2024-04-07T16:37:47.673 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.680+0000 7f346a3868c0 1 < completed multipart upload key=HBLEXONKXJTL upload=G615N9W6JX33 tag=I2AUHW75P062 type=write size=2696366 2024-04-07T16:37:47.678 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.684+0000 7f346a3868c0 1 > prepared operation key=QVOKSQAGHPA9 tag=5NM3AUW3D8DH type=del size=2155875 2024-04-07T16:37:47.682 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.688+0000 7f346a3868c0 1 < completed delete operation key=1VBSUWNI3V1T tag=V9F4UMD579SM type=del 2024-04-07T16:37:47.687 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.696+0000 7f346a3868c0 1 > prepared operation key=QTCIIPTXIRW2 tag=S6S8JHQSKDQM type=write size=306391 2024-04-07T16:37:47.691 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.700+0000 7f346a3868c0 1 < completed delete operation key=DXTO3AJ429AC tag=JY3DGC3RXR19 type=del 2024-04-07T16:37:47.696 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.704+0000 7f346a3868c0 1 > prepared operation key=EQR0FY86AT5X tag=ZVE0RJYJ5XDU type=del size=3048468 2024-04-07T16:37:47.700 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.708+0000 7f346a3868c0 1 < completed delete operation key=E84U0SX44NQ8 tag=3GZV2CKLLLCA type=del 2024-04-07T16:37:47.708 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.716+0000 7f346a3868c0 1 > prepared part key=_multipart_UIW3DN4NUYPP.2~DCM0QMUJSNA8.0 size=604957 2024-04-07T16:37:47.708 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.716+0000 7f346a3868c0 1 > prepared multipart upload key=UIW3DN4NUYPP upload=DCM0QMUJSNA8 tag=SSMKZ3U9AVWO type=write size=1653533 2024-04-07T16:37:47.713 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.720+0000 7f346a3868c0 1 < completed delete operation key=QVOKSQAGHPA9 tag=80JB04MOJF5K type=del 2024-04-07T16:37:47.717 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.724+0000 7f346a3868c0 1 > prepared operation key=1VBSUWNI3V1T tag=GBJCGDIJ3WGW type=del size=2881375 2024-04-07T16:37:47.722 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.728+0000 7f346a3868c0 1 < completed delete operation key=8F81G0P0YBFW tag=O7O9SKQMQA4J type=del 2024-04-07T16:37:47.727 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.736+0000 7f346a3868c0 1 > prepared operation key=XJLQZ7IS9U1T tag=JG1343RRDVU6 type=del size=3417014 2024-04-07T16:37:47.733 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.740+0000 7f346a3868c0 1 < completed write operation key=EQR0FY86AT5X tag=2FL321V4ZB83 type=write size=586588 2024-04-07T16:37:47.740 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.748+0000 7f346a3868c0 1 > prepared part key=_multipart_YXMDZB23YTB3.2~NZDOFOOOBS9P.0 size=1048576 2024-04-07T16:37:47.744 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.752+0000 7f346a3868c0 1 > prepared part key=_multipart_YXMDZB23YTB3.2~NZDOFOOOBS9P.1 size=618040 2024-04-07T16:37:47.744 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.752+0000 7f346a3868c0 1 > prepared multipart upload key=YXMDZB23YTB3 upload=NZDOFOOOBS9P tag=GYA1UUKXEAMR type=write size=2715192 2024-04-07T16:37:47.748 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.756+0000 7f346a3868c0 1 < completed delete operation key=E84U0SX44NQ8 tag=VUFVYJXUGG4G type=del 2024-04-07T16:37:47.752 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.760+0000 7f346a3868c0 1 > prepared operation key=3N0QJRYVO3FV tag=EX651O2PQID8 type=write size=428729 2024-04-07T16:37:47.757 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.764+0000 7f346a3868c0 1 < completed part key=_multipart_6XDS0A7L0IY8.2~Q17Q5NO3V8VE.0 size=440834 2024-04-07T16:37:47.764 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.772+0000 7f346a3868c0 1 < completed multipart upload key=6XDS0A7L0IY8 upload=Q17Q5NO3V8VE tag=0DTCTYVZQJUD type=write size=1489410 2024-04-07T16:37:47.768 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.776+0000 7f346a3868c0 1 > prepared operation key=GBQSME7SY1AH tag=MJY7BVW2W24D type=del size=1990108 2024-04-07T16:37:47.772 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.780+0000 7f346a3868c0 1 < completed part key=_multipart_U3EID5EK64HD.2~LTP42EWTB902.0 size=1048576 2024-04-07T16:37:47.776 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.784+0000 7f346a3868c0 1 < completed part key=_multipart_U3EID5EK64HD.2~LTP42EWTB902.1 size=768346 2024-04-07T16:37:47.783 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.792+0000 7f346a3868c0 1 < completed multipart upload key=U3EID5EK64HD upload=LTP42EWTB902 tag=B7YMZ3GYNOP0 type=write size=2865498 2024-04-07T16:37:47.787 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.796+0000 7f346a3868c0 1 > prepared operation key=1WGUPH7X5UA0 tag=2Q1DYC0FB846 type=del size=520455 2024-04-07T16:37:47.792 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.800+0000 7f346a3868c0 1 < completed delete operation key=E84U0SX44NQ8 tag=L33YNSCONNZN type=del 2024-04-07T16:37:47.800 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.808+0000 7f346a3868c0 1 > prepared part key=_multipart_YOK8BAEOMPUR.2~X1D1NH5RVJ24.0 size=1048576 2024-04-07T16:37:47.804 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.812+0000 7f346a3868c0 1 > prepared part key=_multipart_YOK8BAEOMPUR.2~X1D1NH5RVJ24.1 size=1048576 2024-04-07T16:37:47.807 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.812+0000 7f346a3868c0 1 > prepared part key=_multipart_YOK8BAEOMPUR.2~X1D1NH5RVJ24.2 size=922325 2024-04-07T16:37:47.807 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.812+0000 7f346a3868c0 1 > prepared multipart upload key=YOK8BAEOMPUR upload=X1D1NH5RVJ24 tag=Z51SNRTZZAP7 type=write size=4068053 2024-04-07T16:37:47.815 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.824+0000 7f346a3868c0 1 < suggested remove operation key=ROFH30W3UCAX tag=YXYF0BEE5JDX type=del 2024-04-07T16:37:47.819 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.828+0000 7f346a3868c0 1 > prepared operation key=1VBSUWNI3V1T tag=CS2CKGZPZAF9 type=del size=3457969 2024-04-07T16:37:47.824 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.832+0000 7f346a3868c0 1 < completed part key=_multipart_6XDS0A7L0IY8.2~U3ITPRNTD64W.0 size=1048576 2024-04-07T16:37:47.828 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.836+0000 7f346a3868c0 1 < completed part key=_multipart_6XDS0A7L0IY8.2~U3ITPRNTD64W.1 size=1048576 2024-04-07T16:37:47.831 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.840+0000 7f346a3868c0 1 < completed part key=_multipart_6XDS0A7L0IY8.2~U3ITPRNTD64W.2 size=78862 2024-04-07T16:37:47.838 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.844+0000 7f346a3868c0 1 < completed multipart upload key=6XDS0A7L0IY8 upload=U3ITPRNTD64W tag=TEO6HM3O63T4 type=write size=3224590 2024-04-07T16:37:47.846 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.852+0000 7f346a3868c0 1 > prepared part key=_multipart_GZXHYZO5SMUP.2~6W5A1P05SPCE.0 size=1048576 2024-04-07T16:37:47.849 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.856+0000 7f346a3868c0 1 > prepared part key=_multipart_GZXHYZO5SMUP.2~6W5A1P05SPCE.1 size=1048576 2024-04-07T16:37:47.853 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.860+0000 7f346a3868c0 1 > prepared part key=_multipart_GZXHYZO5SMUP.2~6W5A1P05SPCE.2 size=509849 2024-04-07T16:37:47.853 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.860+0000 7f346a3868c0 1 > prepared multipart upload key=GZXHYZO5SMUP upload=6W5A1P05SPCE tag=U9JWI1TZRP3S type=write size=3655577 2024-04-07T16:37:47.856 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:37:47 smithi144 bash[66654]: audit 2024-04-07T16:37:46.752779+0000 mon.a (mon.0) 2021 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:37:47.858 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.864+0000 7f346a3868c0 1 < completed delete operation key=6XDS0A7L0IY8 tag=EPUIKZ84ZJPI type=del 2024-04-07T16:37:47.863 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.868+0000 7f346a3868c0 1 > prepared operation key=XJLQZ7IS9U1T tag=1ROHFAOTO3VM type=del size=16099 2024-04-07T16:37:47.867 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.876+0000 7f346a3868c0 1 < completed delete operation key=8F81G0P0YBFW tag=5VV8XU0JBFNC type=del 2024-04-07T16:37:47.871 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.876+0000 7f346a3868c0 1 > prepared operation key=95YVKJ14JD8W tag=1Y3ONTWBG350 type=write size=101505 2024-04-07T16:37:47.875 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.884+0000 7f346a3868c0 1 < completed part key=_multipart_1VBSUWNI3V1T.2~7OJHCJLFQONL.0 size=1048576 2024-04-07T16:37:47.879 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.888+0000 7f346a3868c0 1 < completed part key=_multipart_1VBSUWNI3V1T.2~7OJHCJLFQONL.1 size=738535 2024-04-07T16:37:47.886 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.892+0000 7f346a3868c0 1 < completed multipart upload key=1VBSUWNI3V1T upload=7OJHCJLFQONL tag=1194KTRDMBRH type=write size=2835687 2024-04-07T16:37:47.894 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.900+0000 7f346a3868c0 1 > prepared part key=_multipart_U3EID5EK64HD.2~SBAHEIM0XOAY.0 size=812400 2024-04-07T16:37:47.894 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.900+0000 7f346a3868c0 1 > prepared multipart upload key=U3EID5EK64HD upload=SBAHEIM0XOAY tag=SIY1JJKV12MX type=write size=1860976 2024-04-07T16:37:47.898 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.904+0000 7f346a3868c0 1 < completed delete operation key=QVOKSQAGHPA9 tag=5NM3AUW3D8DH type=del 2024-04-07T16:37:47.903 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.912+0000 7f346a3868c0 1 > prepared operation key=GBQSME7SY1AH tag=NCAMEEDHOW3D type=del size=2898658 2024-04-07T16:37:47.908 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.916+0000 7f346a3868c0 1 < completed write operation key=QTCIIPTXIRW2 tag=S6S8JHQSKDQM type=write size=306391 2024-04-07T16:37:47.916 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.924+0000 7f346a3868c0 1 > prepared part key=_multipart_NYET3212USVE.2~3E54S8S97ME2.0 size=1048576 2024-04-07T16:37:47.919 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.928+0000 7f346a3868c0 1 > prepared part key=_multipart_NYET3212USVE.2~3E54S8S97ME2.1 size=1048576 2024-04-07T16:37:47.922 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.928+0000 7f346a3868c0 1 > prepared part key=_multipart_NYET3212USVE.2~3E54S8S97ME2.2 size=624269 2024-04-07T16:37:47.923 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.928+0000 7f346a3868c0 1 > prepared multipart upload key=NYET3212USVE upload=3E54S8S97ME2 tag=WZ52PX5LGHI5 type=write size=3769997 2024-04-07T16:37:47.931 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.940+0000 7f346a3868c0 1 < suggested remove operation key=EQR0FY86AT5X tag=ZVE0RJYJ5XDU type=del 2024-04-07T16:37:47.939 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.944+0000 7f346a3868c0 1 > prepared part key=_multipart_QTCIIPTXIRW2.2~H2332UO3Z1XW.0 size=1048576 2024-04-07T16:37:47.941 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:37:47 smithi100 bash[74942]: audit 2024-04-07T16:37:46.752779+0000 mon.a (mon.0) 2021 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:37:47.941 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:37:47 smithi100 bash[76202]: audit 2024-04-07T16:37:46.752779+0000 mon.a (mon.0) 2021 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:37:47.942 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.948+0000 7f346a3868c0 1 > prepared part key=_multipart_QTCIIPTXIRW2.2~H2332UO3Z1XW.1 size=1048576 2024-04-07T16:37:47.945 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.952+0000 7f346a3868c0 1 > prepared part key=_multipart_QTCIIPTXIRW2.2~H2332UO3Z1XW.2 size=113220 2024-04-07T16:37:47.946 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.952+0000 7f346a3868c0 1 > prepared multipart upload key=QTCIIPTXIRW2 upload=H2332UO3Z1XW tag=QD1PVM7IEHPV type=write size=3258948 2024-04-07T16:37:47.951 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.956+0000 7f346a3868c0 1 < completed part key=_multipart_UIW3DN4NUYPP.2~DCM0QMUJSNA8.0 size=604957 2024-04-07T16:37:47.959 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.968+0000 7f346a3868c0 1 < completed multipart upload key=UIW3DN4NUYPP upload=DCM0QMUJSNA8 tag=SSMKZ3U9AVWO type=write size=1653533 2024-04-07T16:37:47.964 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.972+0000 7f346a3868c0 1 > prepared operation key=ROFH30W3UCAX tag=BQ9TCQ10TORY type=del size=1940482 2024-04-07T16:37:47.969 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.976+0000 7f346a3868c0 1 < completed delete operation key=1VBSUWNI3V1T tag=GBJCGDIJ3WGW type=del 2024-04-07T16:37:47.974 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.980+0000 7f346a3868c0 1 > prepared operation key=1VKVQXSQMTU1 tag=JW9TJZZ41BDB type=del size=1198590 2024-04-07T16:37:47.979 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.988+0000 7f346a3868c0 1 < completed delete operation key=XJLQZ7IS9U1T tag=JG1343RRDVU6 type=del 2024-04-07T16:37:47.987 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:47.996+0000 7f346a3868c0 1 > prepared part key=_multipart_IX3F0IUOKBX1.2~5YA37AG4DXNJ.0 size=1048576 2024-04-07T16:37:47.991 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.000+0000 7f346a3868c0 1 > prepared part key=_multipart_IX3F0IUOKBX1.2~5YA37AG4DXNJ.1 size=561933 2024-04-07T16:37:47.991 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.000+0000 7f346a3868c0 1 > prepared multipart upload key=IX3F0IUOKBX1 upload=5YA37AG4DXNJ tag=XIYDQ0GUS4SS type=write size=2659085 2024-04-07T16:37:47.996 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.004+0000 7f346a3868c0 1 < completed part key=_multipart_YXMDZB23YTB3.2~NZDOFOOOBS9P.0 size=1048576 2024-04-07T16:37:47.999 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.008+0000 7f346a3868c0 1 < completed part key=_multipart_YXMDZB23YTB3.2~NZDOFOOOBS9P.1 size=618040 2024-04-07T16:37:48.007 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.012+0000 7f346a3868c0 1 < completed multipart upload key=YXMDZB23YTB3 upload=NZDOFOOOBS9P tag=GYA1UUKXEAMR type=write size=2715192 2024-04-07T16:37:48.015 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.024+0000 7f346a3868c0 1 > prepared part key=_multipart_HBLEXONKXJTL.2~9G3CJ3T9AH04.0 size=1048576 2024-04-07T16:37:48.019 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.024+0000 7f346a3868c0 1 > prepared part key=_multipart_HBLEXONKXJTL.2~9G3CJ3T9AH04.1 size=558348 2024-04-07T16:37:48.019 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.024+0000 7f346a3868c0 1 > prepared multipart upload key=HBLEXONKXJTL upload=9G3CJ3T9AH04 tag=M9GCJ1WW0ROM type=write size=2655500 2024-04-07T16:37:48.023 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.032+0000 7f346a3868c0 1 < canceled operation key=3N0QJRYVO3FV tag=EX651O2PQID8 type=write size=428729 2024-04-07T16:37:48.032 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.040+0000 7f346a3868c0 1 > prepared part key=_multipart_QVOKSQAGHPA9.2~NBJ3A55KBAYP.0 size=1048576 2024-04-07T16:37:48.036 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.044+0000 7f346a3868c0 1 > prepared part key=_multipart_QVOKSQAGHPA9.2~NBJ3A55KBAYP.1 size=1048576 2024-04-07T16:37:48.039 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.048+0000 7f346a3868c0 1 > prepared part key=_multipart_QVOKSQAGHPA9.2~NBJ3A55KBAYP.2 size=715574 2024-04-07T16:37:48.039 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.048+0000 7f346a3868c0 1 > prepared multipart upload key=QVOKSQAGHPA9 upload=NBJ3A55KBAYP tag=5D5WB4R7FMLG type=write size=3861302 2024-04-07T16:37:48.044 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.052+0000 7f346a3868c0 1 < completed delete operation key=GBQSME7SY1AH tag=MJY7BVW2W24D type=del 2024-04-07T16:37:48.048 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.056+0000 7f346a3868c0 1 > prepared operation key=N14BUXLJONHZ tag=4359Z1JAA3Y4 type=write size=545850 2024-04-07T16:37:48.053 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.060+0000 7f346a3868c0 1 < canceled operation key=1WGUPH7X5UA0 tag=2Q1DYC0FB846 type=del size=520455 2024-04-07T16:37:48.062 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.068+0000 7f346a3868c0 1 > prepared part key=_multipart_DXTO3AJ429AC.2~SB111F2MUDQ3.0 size=1048576 2024-04-07T16:37:48.065 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.072+0000 7f346a3868c0 1 > prepared part key=_multipart_DXTO3AJ429AC.2~SB111F2MUDQ3.1 size=750842 2024-04-07T16:37:48.065 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.072+0000 7f346a3868c0 1 > prepared multipart upload key=DXTO3AJ429AC upload=SB111F2MUDQ3 tag=Z672C24X62E1 type=write size=2847994 2024-04-07T16:37:48.069 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.076+0000 7f346a3868c0 1 < completed part key=_multipart_YOK8BAEOMPUR.2~X1D1NH5RVJ24.0 size=1048576 2024-04-07T16:37:48.073 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.080+0000 7f346a3868c0 1 < completed part key=_multipart_YOK8BAEOMPUR.2~X1D1NH5RVJ24.1 size=1048576 2024-04-07T16:37:48.077 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.084+0000 7f346a3868c0 1 < completed part key=_multipart_YOK8BAEOMPUR.2~X1D1NH5RVJ24.2 size=922325 2024-04-07T16:37:48.084 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.092+0000 7f346a3868c0 1 < completed multipart upload key=YOK8BAEOMPUR upload=X1D1NH5RVJ24 tag=Z51SNRTZZAP7 type=write size=4068053 2024-04-07T16:37:48.091 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.100+0000 7f346a3868c0 1 > prepared part key=_multipart_QS4HYZFFAJNF.2~I2ORNKOQ5ES8.0 size=1048576 2024-04-07T16:37:48.094 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.100+0000 7f346a3868c0 1 > prepared part key=_multipart_QS4HYZFFAJNF.2~I2ORNKOQ5ES8.1 size=14178 2024-04-07T16:37:48.094 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.100+0000 7f346a3868c0 1 > prepared multipart upload key=QS4HYZFFAJNF upload=I2ORNKOQ5ES8 tag=YM1OID7NPBSF type=write size=2111330 2024-04-07T16:37:48.099 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.108+0000 7f346a3868c0 1 < completed delete operation key=1VBSUWNI3V1T tag=CS2CKGZPZAF9 type=del 2024-04-07T16:37:48.103 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.112+0000 7f346a3868c0 1 > prepared operation key=GBQSME7SY1AH tag=0Z83PU7SZXYU type=del size=2253543 2024-04-07T16:37:48.107 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.116+0000 7f346a3868c0 1 < completed part key=_multipart_GZXHYZO5SMUP.2~6W5A1P05SPCE.0 size=1048576 2024-04-07T16:37:48.107 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.116+0000 7f346a3868c0 1 < canceled part key=_multipart_GZXHYZO5SMUP.2~6W5A1P05SPCE.1 size=1048576 2024-04-07T16:37:48.110 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.116+0000 7f346a3868c0 1 < completed part key=_multipart_GZXHYZO5SMUP.2~6W5A1P05SPCE.2 size=509849 2024-04-07T16:37:48.117 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.124+0000 7f346a3868c0 1 < canceled multipart upload key=GZXHYZO5SMUP upload=6W5A1P05SPCE tag=U9JWI1TZRP3S type=write size=3655577 2024-04-07T16:37:48.122 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.128+0000 7f346a3868c0 1 > prepared operation key=GBQSME7SY1AH tag=6G4VMIH1SYQS type=del size=3722037 2024-04-07T16:37:48.126 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.132+0000 7f346a3868c0 1 < completed delete operation key=XJLQZ7IS9U1T tag=1ROHFAOTO3VM type=del 2024-04-07T16:37:48.130 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.136+0000 7f346a3868c0 1 > prepared operation key=6XDS0A7L0IY8 tag=VMD3WP8IY8P2 type=del size=1512233 2024-04-07T16:37:48.135 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.144+0000 7f346a3868c0 1 < completed write operation key=95YVKJ14JD8W tag=1Y3ONTWBG350 type=write size=101505 2024-04-07T16:37:48.141 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.148+0000 7f346a3868c0 1 > prepared operation key=YOK8BAEOMPUR tag=BU0FORB6WG6E type=del size=47395 2024-04-07T16:37:48.144 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.152+0000 7f346a3868c0 1 < completed part key=_multipart_U3EID5EK64HD.2~SBAHEIM0XOAY.0 size=812400 2024-04-07T16:37:48.152 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.160+0000 7f346a3868c0 1 < completed multipart upload key=U3EID5EK64HD upload=SBAHEIM0XOAY tag=SIY1JJKV12MX type=write size=1860976 2024-04-07T16:37:48.156 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.164+0000 7f346a3868c0 1 > prepared operation key=X5Q46GO8YTOK tag=GX4UZUC8N3F7 type=write size=766706 2024-04-07T16:37:48.164 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.172+0000 7f346a3868c0 1 < suggested remove operation key=GBQSME7SY1AH tag=NCAMEEDHOW3D type=del 2024-04-07T16:37:48.169 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.176+0000 7f346a3868c0 1 > prepared operation key=N14BUXLJONHZ tag=SY6FST49UV85 type=del size=4079449 2024-04-07T16:37:48.174 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.180+0000 7f346a3868c0 1 < completed part key=_multipart_NYET3212USVE.2~3E54S8S97ME2.0 size=1048576 2024-04-07T16:37:48.174 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.180+0000 7f346a3868c0 1 < canceled part key=_multipart_NYET3212USVE.2~3E54S8S97ME2.1 size=1048576 2024-04-07T16:37:48.174 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.180+0000 7f346a3868c0 1 < canceled part key=_multipart_NYET3212USVE.2~3E54S8S97ME2.2 size=624269 2024-04-07T16:37:48.181 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.188+0000 7f346a3868c0 1 < canceled multipart upload key=NYET3212USVE upload=3E54S8S97ME2 tag=WZ52PX5LGHI5 type=write size=3769997 2024-04-07T16:37:48.185 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.192+0000 7f346a3868c0 1 > prepared operation key=U3EID5EK64HD tag=CNLCJLGDLLUW type=write size=454909 2024-04-07T16:37:48.190 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.196+0000 7f346a3868c0 1 < completed part key=_multipart_QTCIIPTXIRW2.2~H2332UO3Z1XW.0 size=1048576 2024-04-07T16:37:48.190 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.196+0000 7f346a3868c0 1 < canceled part key=_multipart_QTCIIPTXIRW2.2~H2332UO3Z1XW.1 size=1048576 2024-04-07T16:37:48.190 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.196+0000 7f346a3868c0 1 < canceled part key=_multipart_QTCIIPTXIRW2.2~H2332UO3Z1XW.2 size=113220 2024-04-07T16:37:48.198 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.204+0000 7f346a3868c0 1 < canceled multipart upload key=QTCIIPTXIRW2 upload=H2332UO3Z1XW tag=QD1PVM7IEHPV type=write size=3258948 2024-04-07T16:37:48.202 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.208+0000 7f346a3868c0 1 > prepared operation key=95YVKJ14JD8W tag=JSZO2FXMFZ7O type=del size=2514331 2024-04-07T16:37:48.206 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.212+0000 7f346a3868c0 1 < completed delete operation key=ROFH30W3UCAX tag=BQ9TCQ10TORY type=del 2024-04-07T16:37:48.213 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.220+0000 7f346a3868c0 1 > prepared part key=_multipart_8F81G0P0YBFW.2~JZAFUHXPXHPB.0 size=393896 2024-04-07T16:37:48.213 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.220+0000 7f346a3868c0 1 > prepared multipart upload key=8F81G0P0YBFW upload=JZAFUHXPXHPB tag=KWKKGH43X6ZZ type=write size=1442472 2024-04-07T16:37:48.217 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.224+0000 7f346a3868c0 1 < completed delete operation key=1VKVQXSQMTU1 tag=JW9TJZZ41BDB type=del 2024-04-07T16:37:48.221 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.228+0000 7f346a3868c0 1 > prepared operation key=HBLEXONKXJTL tag=DXYX5YY4IOQF type=del size=3737636 2024-04-07T16:37:48.226 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.232+0000 7f346a3868c0 1 < completed part key=_multipart_IX3F0IUOKBX1.2~5YA37AG4DXNJ.0 size=1048576 2024-04-07T16:37:48.229 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.236+0000 7f346a3868c0 1 < completed part key=_multipart_IX3F0IUOKBX1.2~5YA37AG4DXNJ.1 size=561933 2024-04-07T16:37:48.237 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.244+0000 7f346a3868c0 1 < completed multipart upload key=IX3F0IUOKBX1 upload=5YA37AG4DXNJ tag=XIYDQ0GUS4SS type=write size=2659085 2024-04-07T16:37:48.241 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.248+0000 7f346a3868c0 1 > prepared operation key=8F81G0P0YBFW tag=HMYQJEM07WET type=write size=1004168 2024-04-07T16:37:48.246 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.252+0000 7f346a3868c0 1 < completed part key=_multipart_HBLEXONKXJTL.2~9G3CJ3T9AH04.0 size=1048576 2024-04-07T16:37:48.249 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.256+0000 7f346a3868c0 1 < completed part key=_multipart_HBLEXONKXJTL.2~9G3CJ3T9AH04.1 size=558348 2024-04-07T16:37:48.257 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.264+0000 7f346a3868c0 1 < completed multipart upload key=HBLEXONKXJTL upload=9G3CJ3T9AH04 tag=M9GCJ1WW0ROM type=write size=2655500 2024-04-07T16:37:48.261 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.268+0000 7f346a3868c0 1 > prepared operation key=8F81G0P0YBFW tag=2KX98XD1A7J6 type=write size=787345 2024-04-07T16:37:48.265 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.272+0000 7f346a3868c0 1 < completed part key=_multipart_QVOKSQAGHPA9.2~NBJ3A55KBAYP.0 size=1048576 2024-04-07T16:37:48.269 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.276+0000 7f346a3868c0 1 < completed part key=_multipart_QVOKSQAGHPA9.2~NBJ3A55KBAYP.1 size=1048576 2024-04-07T16:37:48.273 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.280+0000 7f346a3868c0 1 < completed part key=_multipart_QVOKSQAGHPA9.2~NBJ3A55KBAYP.2 size=715574 2024-04-07T16:37:48.280 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.288+0000 7f346a3868c0 1 < completed multipart upload key=QVOKSQAGHPA9 upload=NBJ3A55KBAYP tag=5D5WB4R7FMLG type=write size=3861302 2024-04-07T16:37:48.285 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.292+0000 7f346a3868c0 1 > prepared operation key=N14BUXLJONHZ tag=VXHL4O855V6S type=write size=349582 2024-04-07T16:37:48.290 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.296+0000 7f346a3868c0 1 < canceled operation key=N14BUXLJONHZ tag=4359Z1JAA3Y4 type=write size=545850 2024-04-07T16:37:48.295 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.304+0000 7f346a3868c0 1 > prepared operation key=EOYFVEI50SQ1 tag=9S17D6NVJMVJ type=del size=397541 2024-04-07T16:37:48.301 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.308+0000 7f346a3868c0 1 < completed part key=_multipart_DXTO3AJ429AC.2~SB111F2MUDQ3.0 size=1048576 2024-04-07T16:37:48.305 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.312+0000 7f346a3868c0 1 < completed part key=_multipart_DXTO3AJ429AC.2~SB111F2MUDQ3.1 size=750842 2024-04-07T16:37:48.313 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.320+0000 7f346a3868c0 1 < completed multipart upload key=DXTO3AJ429AC upload=SB111F2MUDQ3 tag=Z672C24X62E1 type=write size=2847994 2024-04-07T16:37:48.318 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.324+0000 7f346a3868c0 1 > prepared operation key=UIW3DN4NUYPP tag=HI41NGRA58C9 type=del size=895634 2024-04-07T16:37:48.322 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.328+0000 7f346a3868c0 1 < completed part key=_multipart_QS4HYZFFAJNF.2~I2ORNKOQ5ES8.0 size=1048576 2024-04-07T16:37:48.326 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.332+0000 7f346a3868c0 1 < completed part key=_multipart_QS4HYZFFAJNF.2~I2ORNKOQ5ES8.1 size=14178 2024-04-07T16:37:48.333 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.340+0000 7f346a3868c0 1 < completed multipart upload key=QS4HYZFFAJNF upload=I2ORNKOQ5ES8 tag=YM1OID7NPBSF type=write size=2111330 2024-04-07T16:37:48.338 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.344+0000 7f346a3868c0 1 > prepared operation key=QS4HYZFFAJNF tag=DGA29BURT1DN type=del size=1871897 2024-04-07T16:37:48.340 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.348+0000 7f346a3868c0 -1 < no bi entry to suggest for operation key=GBQSME7SY1AH tag=0Z83PU7SZXYU type=del size=2253543: (2) No such file or directory 2024-04-07T16:37:48.344 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.352+0000 7f346a3868c0 1 > prepared operation key=GZXHYZO5SMUP tag=SYPI6JMQ8N8O type=del size=1198376 2024-04-07T16:37:48.349 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.356+0000 7f346a3868c0 -1 < failed to complete operation key=GBQSME7SY1AH tag=6G4VMIH1SYQS type=del size=3722037: (22) Invalid argument 2024-04-07T16:37:48.353 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.360+0000 7f346a3868c0 1 > prepared operation key=E84U0SX44NQ8 tag=4NLAB1DRYXFG type=del size=3223137 2024-04-07T16:37:48.364 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.372+0000 7f346a3868c0 1 < suggested remove operation key=6XDS0A7L0IY8 tag=VMD3WP8IY8P2 type=del 2024-04-07T16:37:48.368 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.376+0000 7f346a3868c0 1 > prepared operation key=QS4HYZFFAJNF tag=EA7PZM4F11F4 type=del size=3936785 2024-04-07T16:37:48.377 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.384+0000 7f346a3868c0 1 < suggested remove operation key=YOK8BAEOMPUR tag=BU0FORB6WG6E type=del 2024-04-07T16:37:48.385 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.392+0000 7f346a3868c0 1 > prepared part key=_multipart_95YVKJ14JD8W.2~U49CQHR0MCYV.0 size=1048576 2024-04-07T16:37:48.389 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.396+0000 7f346a3868c0 1 > prepared part key=_multipart_95YVKJ14JD8W.2~U49CQHR0MCYV.1 size=417090 2024-04-07T16:37:48.389 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.396+0000 7f346a3868c0 1 > prepared multipart upload key=95YVKJ14JD8W upload=U49CQHR0MCYV tag=V19Z06JCSCGX type=write size=2514242 2024-04-07T16:37:48.398 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.404+0000 7f346a3868c0 1 < suggested update operation key=X5Q46GO8YTOK tag=GX4UZUC8N3F7 type=write size=766706 2024-04-07T16:37:48.405 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.412+0000 7f346a3868c0 1 > prepared part key=_multipart_CL4PP1RKZU0E.2~CBD6KVS2WO88.0 size=1048576 2024-04-07T16:37:48.409 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.416+0000 7f346a3868c0 1 > prepared part key=_multipart_CL4PP1RKZU0E.2~CBD6KVS2WO88.1 size=469078 2024-04-07T16:37:48.409 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.416+0000 7f346a3868c0 1 > prepared multipart upload key=CL4PP1RKZU0E upload=CBD6KVS2WO88 tag=XF0V8LMGJI7O type=write size=2566230 2024-04-07T16:37:48.413 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.420+0000 7f346a3868c0 1 < canceled operation key=N14BUXLJONHZ tag=SY6FST49UV85 type=del size=4079449 2024-04-07T16:37:48.422 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.428+0000 7f346a3868c0 1 > prepared part key=_multipart_6XDS0A7L0IY8.2~QX1E158LSLSQ.0 size=1048576 2024-04-07T16:37:48.425 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.432+0000 7f346a3868c0 1 > prepared part key=_multipart_6XDS0A7L0IY8.2~QX1E158LSLSQ.1 size=1048576 2024-04-07T16:37:48.429 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.436+0000 7f346a3868c0 1 > prepared part key=_multipart_6XDS0A7L0IY8.2~QX1E158LSLSQ.2 size=967301 2024-04-07T16:37:48.429 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.436+0000 7f346a3868c0 1 > prepared multipart upload key=6XDS0A7L0IY8 upload=QX1E158LSLSQ tag=IO8GYCUI7SGW type=write size=4113029 2024-04-07T16:37:48.439 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.444+0000 7f346a3868c0 1 < suggested update operation key=U3EID5EK64HD tag=CNLCJLGDLLUW type=write size=454909 2024-04-07T16:37:48.447 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.456+0000 7f346a3868c0 1 > prepared part key=_multipart_C1BTVJNHIM0X.2~TA207573GBCZ.0 size=843760 2024-04-07T16:37:48.447 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.456+0000 7f346a3868c0 1 > prepared multipart upload key=C1BTVJNHIM0X upload=TA207573GBCZ tag=U902FAZZB3W2 type=write size=1892336 2024-04-07T16:37:48.453 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.460+0000 7f346a3868c0 1 < completed delete operation key=95YVKJ14JD8W tag=JSZO2FXMFZ7O type=del 2024-04-07T16:37:48.457 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.464+0000 7f346a3868c0 1 > prepared operation key=QVOKSQAGHPA9 tag=8ZO1V02Y27DN type=del size=2974589 2024-04-07T16:37:48.461 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.468+0000 7f346a3868c0 1 < completed part key=_multipart_8F81G0P0YBFW.2~JZAFUHXPXHPB.0 size=393896 2024-04-07T16:37:48.468 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.476+0000 7f346a3868c0 1 < completed multipart upload key=8F81G0P0YBFW upload=JZAFUHXPXHPB tag=KWKKGH43X6ZZ type=write size=1442472 2024-04-07T16:37:48.477 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.484+0000 7f346a3868c0 1 > prepared part key=_multipart_CL4PP1RKZU0E.2~F5PL8AWZQA7I.0 size=1048576 2024-04-07T16:37:48.480 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.488+0000 7f346a3868c0 1 > prepared part key=_multipart_CL4PP1RKZU0E.2~F5PL8AWZQA7I.1 size=1048576 2024-04-07T16:37:48.483 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.492+0000 7f346a3868c0 1 > prepared part key=_multipart_CL4PP1RKZU0E.2~F5PL8AWZQA7I.2 size=343651 2024-04-07T16:37:48.483 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.492+0000 7f346a3868c0 1 > prepared multipart upload key=CL4PP1RKZU0E upload=F5PL8AWZQA7I tag=JB8P8AH8JAYC type=write size=3489379 2024-04-07T16:37:48.488 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.496+0000 7f346a3868c0 1 < completed delete operation key=HBLEXONKXJTL tag=DXYX5YY4IOQF type=del 2024-04-07T16:37:48.497 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.504+0000 7f346a3868c0 1 > prepared part key=_multipart_E84U0SX44NQ8.2~G0VUPDDCA625.0 size=469414 2024-04-07T16:37:48.497 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.504+0000 7f346a3868c0 1 > prepared multipart upload key=E84U0SX44NQ8 upload=G0VUPDDCA625 tag=2IPHW69X2USW type=write size=1517990 2024-04-07T16:37:48.501 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.508+0000 7f346a3868c0 1 < completed write operation key=8F81G0P0YBFW tag=HMYQJEM07WET type=write size=1004168 2024-04-07T16:37:48.508 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.516+0000 7f346a3868c0 1 > prepared part key=_multipart_N14BUXLJONHZ.2~ABEBGC1IBA21.0 size=1048576 2024-04-07T16:37:48.511 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.520+0000 7f346a3868c0 1 > prepared part key=_multipart_N14BUXLJONHZ.2~ABEBGC1IBA21.1 size=1048576 2024-04-07T16:37:48.514 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.520+0000 7f346a3868c0 1 > prepared part key=_multipart_N14BUXLJONHZ.2~ABEBGC1IBA21.2 size=518575 2024-04-07T16:37:48.514 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.520+0000 7f346a3868c0 1 > prepared multipart upload key=N14BUXLJONHZ upload=ABEBGC1IBA21 tag=92Q5ZA6XDMUH type=write size=3664303 2024-04-07T16:37:48.518 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.524+0000 7f346a3868c0 1 < completed write operation key=8F81G0P0YBFW tag=2KX98XD1A7J6 type=write size=787345 2024-04-07T16:37:48.526 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.532+0000 7f346a3868c0 1 > prepared part key=_multipart_6XDS0A7L0IY8.2~M0T6ZKMXGBG4.0 size=917164 2024-04-07T16:37:48.526 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.532+0000 7f346a3868c0 1 > prepared multipart upload key=6XDS0A7L0IY8 upload=M0T6ZKMXGBG4 tag=EMXX1VVRQ8AD type=write size=1965740 2024-04-07T16:37:48.530 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.536+0000 7f346a3868c0 1 < completed write operation key=N14BUXLJONHZ tag=VXHL4O855V6S type=write size=349582 2024-04-07T16:37:48.534 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.540+0000 7f346a3868c0 1 > prepared operation key=XJLQZ7IS9U1T tag=1RRXAK8HJVY1 type=write size=1022432 2024-04-07T16:37:48.537 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.544+0000 7f346a3868c0 1 < completed delete operation key=EOYFVEI50SQ1 tag=9S17D6NVJMVJ type=del 2024-04-07T16:37:48.545 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.552+0000 7f346a3868c0 1 > prepared part key=_multipart_LXOZ0T2KY6OR.2~SIKQ3KIWTSEQ.0 size=94553 2024-04-07T16:37:48.545 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.552+0000 7f346a3868c0 1 > prepared multipart upload key=LXOZ0T2KY6OR upload=SIKQ3KIWTSEQ tag=MJFTQ682R9WP type=write size=1143129 2024-04-07T16:37:48.550 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.556+0000 7f346a3868c0 1 < completed delete operation key=UIW3DN4NUYPP tag=HI41NGRA58C9 type=del 2024-04-07T16:37:48.553 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.560+0000 7f346a3868c0 1 > prepared operation key=LXOZ0T2KY6OR tag=4SJT1RGBCT1Y type=del size=1378411 2024-04-07T16:37:48.557 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.564+0000 7f346a3868c0 1 < completed delete operation key=QS4HYZFFAJNF tag=DGA29BURT1DN type=del 2024-04-07T16:37:48.562 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.568+0000 7f346a3868c0 1 > prepared operation key=DXTO3AJ429AC tag=RAAU8IH5G162 type=del size=2632697 2024-04-07T16:37:48.566 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.572+0000 7f346a3868c0 1 < completed delete operation key=GZXHYZO5SMUP tag=SYPI6JMQ8N8O type=del 2024-04-07T16:37:48.570 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.576+0000 7f346a3868c0 1 > prepared operation key=QS4HYZFFAJNF tag=DG2PIHO1EXMK type=del size=157514 2024-04-07T16:37:48.574 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.580+0000 7f346a3868c0 1 < completed delete operation key=E84U0SX44NQ8 tag=4NLAB1DRYXFG type=del 2024-04-07T16:37:48.578 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.584+0000 7f346a3868c0 1 > prepared operation key=GZXHYZO5SMUP tag=QOV1NWBWEUG6 type=del size=1654771 2024-04-07T16:37:48.582 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.588+0000 7f346a3868c0 1 < completed delete operation key=QS4HYZFFAJNF tag=EA7PZM4F11F4 type=del 2024-04-07T16:37:48.586 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.592+0000 7f346a3868c0 1 > prepared operation key=XJLQZ7IS9U1T tag=A0VDGJ4KQAVL type=del size=569458 2024-04-07T16:37:48.590 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.596+0000 7f346a3868c0 1 < completed part key=_multipart_95YVKJ14JD8W.2~U49CQHR0MCYV.0 size=1048576 2024-04-07T16:37:48.593 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.600+0000 7f346a3868c0 1 < completed part key=_multipart_95YVKJ14JD8W.2~U49CQHR0MCYV.1 size=417090 2024-04-07T16:37:48.600 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.608+0000 7f346a3868c0 1 < completed multipart upload key=95YVKJ14JD8W upload=U49CQHR0MCYV tag=V19Z06JCSCGX type=write size=2514242 2024-04-07T16:37:48.604 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.612+0000 7f346a3868c0 1 > prepared operation key=1VKVQXSQMTU1 tag=W0CNM0ORLE8J type=del size=3642436 2024-04-07T16:37:48.608 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.616+0000 7f346a3868c0 1 < completed part key=_multipart_CL4PP1RKZU0E.2~CBD6KVS2WO88.0 size=1048576 2024-04-07T16:37:48.608 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.616+0000 7f346a3868c0 1 < canceled part key=_multipart_CL4PP1RKZU0E.2~CBD6KVS2WO88.1 size=469078 2024-04-07T16:37:48.614 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.620+0000 7f346a3868c0 1 < canceled multipart upload key=CL4PP1RKZU0E upload=CBD6KVS2WO88 tag=XF0V8LMGJI7O type=write size=2566230 2024-04-07T16:37:48.621 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.628+0000 7f346a3868c0 1 > prepared part key=_multipart_1VBSUWNI3V1T.2~AH20RDFHU6ML.0 size=1048576 2024-04-07T16:37:48.625 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.632+0000 7f346a3868c0 1 > prepared part key=_multipart_1VBSUWNI3V1T.2~AH20RDFHU6ML.1 size=146825 2024-04-07T16:37:48.625 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.632+0000 7f346a3868c0 1 > prepared multipart upload key=1VBSUWNI3V1T upload=AH20RDFHU6ML tag=NQE6JKM5XC2E type=write size=2243977 2024-04-07T16:37:48.629 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.636+0000 7f346a3868c0 1 < completed part key=_multipart_6XDS0A7L0IY8.2~QX1E158LSLSQ.0 size=1048576 2024-04-07T16:37:48.632 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.640+0000 7f346a3868c0 1 < completed part key=_multipart_6XDS0A7L0IY8.2~QX1E158LSLSQ.1 size=1048576 2024-04-07T16:37:48.635 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.644+0000 7f346a3868c0 1 < completed part key=_multipart_6XDS0A7L0IY8.2~QX1E158LSLSQ.2 size=967301 2024-04-07T16:37:48.642 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.648+0000 7f346a3868c0 1 < completed multipart upload key=6XDS0A7L0IY8 upload=QX1E158LSLSQ tag=IO8GYCUI7SGW type=write size=4113029 2024-04-07T16:37:48.646 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.652+0000 7f346a3868c0 1 > prepared operation key=NYET3212USVE tag=310Y5FM5ILK7 type=del size=4049332 2024-04-07T16:37:48.651 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.656+0000 7f346a3868c0 1 < completed part key=_multipart_C1BTVJNHIM0X.2~TA207573GBCZ.0 size=843760 2024-04-07T16:37:48.658 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.664+0000 7f346a3868c0 1 < completed multipart upload key=C1BTVJNHIM0X upload=TA207573GBCZ tag=U902FAZZB3W2 type=write size=1892336 2024-04-07T16:37:48.663 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.668+0000 7f346a3868c0 1 > prepared operation key=U3EID5EK64HD tag=LTP0A5WAC95A type=del size=1385096 2024-04-07T16:37:48.668 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.676+0000 7f346a3868c0 1 < completed delete operation key=QVOKSQAGHPA9 tag=8ZO1V02Y27DN type=del 2024-04-07T16:37:48.672 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.680+0000 7f346a3868c0 1 > prepared operation key=CL4PP1RKZU0E tag=P3T4SX35PUFN type=write size=864592 2024-04-07T16:37:48.677 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.684+0000 7f346a3868c0 1 < completed part key=_multipart_CL4PP1RKZU0E.2~F5PL8AWZQA7I.0 size=1048576 2024-04-07T16:37:48.680 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.688+0000 7f346a3868c0 1 < completed part key=_multipart_CL4PP1RKZU0E.2~F5PL8AWZQA7I.1 size=1048576 2024-04-07T16:37:48.684 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.692+0000 7f346a3868c0 1 < completed part key=_multipart_CL4PP1RKZU0E.2~F5PL8AWZQA7I.2 size=343651 2024-04-07T16:37:48.690 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:37:48 smithi100 bash[74942]: cluster 2024-04-07T16:37:47.056916+0000 mgr.y (mgr.25175) 673 : cluster [DBG] pgmap v1308: 137 pgs: 137 active+clean; 583 KiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 94 KiB/s rd, 33 KiB/s wr, 165 op/s 2024-04-07T16:37:48.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:37:48 smithi100 bash[76202]: cluster 2024-04-07T16:37:47.056916+0000 mgr.y (mgr.25175) 673 : cluster [DBG] pgmap v1308: 137 pgs: 137 active+clean; 583 KiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 94 KiB/s rd, 33 KiB/s wr, 165 op/s 2024-04-07T16:37:48.691 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:37:48 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:37:48] "GET /metrics HTTP/1.1" 200 36370 "" "Prometheus/2.43.0" 2024-04-07T16:37:48.692 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.700+0000 7f346a3868c0 1 < completed multipart upload key=CL4PP1RKZU0E upload=F5PL8AWZQA7I tag=JB8P8AH8JAYC type=write size=3489379 2024-04-07T16:37:48.701 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.708+0000 7f346a3868c0 1 > prepared part key=_multipart_6XDS0A7L0IY8.2~WQR7RFCZ9NGC.0 size=1048576 2024-04-07T16:37:48.705 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.712+0000 7f346a3868c0 1 > prepared part key=_multipart_6XDS0A7L0IY8.2~WQR7RFCZ9NGC.1 size=1048576 2024-04-07T16:37:48.708 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.716+0000 7f346a3868c0 1 > prepared part key=_multipart_6XDS0A7L0IY8.2~WQR7RFCZ9NGC.2 size=523113 2024-04-07T16:37:48.708 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.716+0000 7f346a3868c0 1 > prepared multipart upload key=6XDS0A7L0IY8 upload=WQR7RFCZ9NGC tag=XRX0ZTJO2DRB type=write size=3668841 2024-04-07T16:37:48.713 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.720+0000 7f346a3868c0 1 < completed part key=_multipart_E84U0SX44NQ8.2~G0VUPDDCA625.0 size=469414 2024-04-07T16:37:48.721 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.728+0000 7f346a3868c0 1 < completed multipart upload key=E84U0SX44NQ8 upload=G0VUPDDCA625 tag=2IPHW69X2USW type=write size=1517990 2024-04-07T16:37:48.726 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.732+0000 7f346a3868c0 1 > prepared operation key=7G5C6HBK6QWI tag=UOYF7NYVD474 type=write size=893388 2024-04-07T16:37:48.731 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.736+0000 7f346a3868c0 1 < completed part key=_multipart_N14BUXLJONHZ.2~ABEBGC1IBA21.0 size=1048576 2024-04-07T16:37:48.734 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.740+0000 7f346a3868c0 1 < completed part key=_multipart_N14BUXLJONHZ.2~ABEBGC1IBA21.1 size=1048576 2024-04-07T16:37:48.738 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.744+0000 7f346a3868c0 1 < completed part key=_multipart_N14BUXLJONHZ.2~ABEBGC1IBA21.2 size=518575 2024-04-07T16:37:48.747 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.752+0000 7f346a3868c0 1 < completed multipart upload key=N14BUXLJONHZ upload=ABEBGC1IBA21 tag=92Q5ZA6XDMUH type=write size=3664303 2024-04-07T16:37:48.755 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.764+0000 7f346a3868c0 1 > prepared part key=_multipart_YOK8BAEOMPUR.2~3TWUL0NO3ZYU.0 size=1048576 2024-04-07T16:37:48.758 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.764+0000 7f346a3868c0 1 > prepared part key=_multipart_YOK8BAEOMPUR.2~3TWUL0NO3ZYU.1 size=1014132 2024-04-07T16:37:48.758 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.764+0000 7f346a3868c0 1 > prepared multipart upload key=YOK8BAEOMPUR upload=3TWUL0NO3ZYU tag=CDPXAWG7SF29 type=write size=3111284 2024-04-07T16:37:48.762 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.768+0000 7f346a3868c0 1 < completed part key=_multipart_6XDS0A7L0IY8.2~M0T6ZKMXGBG4.0 size=917164 2024-04-07T16:37:48.770 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.776+0000 7f346a3868c0 1 < completed multipart upload key=6XDS0A7L0IY8 upload=M0T6ZKMXGBG4 tag=EMXX1VVRQ8AD type=write size=1965740 2024-04-07T16:37:48.778 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.784+0000 7f346a3868c0 1 > prepared part key=_multipart_EOYFVEI50SQ1.2~81B03EH910WF.0 size=1048576 2024-04-07T16:37:48.782 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.788+0000 7f346a3868c0 1 > prepared part key=_multipart_EOYFVEI50SQ1.2~81B03EH910WF.1 size=1048576 2024-04-07T16:37:48.785 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.792+0000 7f346a3868c0 1 > prepared part key=_multipart_EOYFVEI50SQ1.2~81B03EH910WF.2 size=1026579 2024-04-07T16:37:48.785 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.792+0000 7f346a3868c0 1 > prepared multipart upload key=EOYFVEI50SQ1 upload=81B03EH910WF tag=YVT7BKUFHEOP type=write size=4172307 2024-04-07T16:37:48.794 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.800+0000 7f346a3868c0 1 < suggested update operation key=XJLQZ7IS9U1T tag=1RRXAK8HJVY1 type=write size=1022432 2024-04-07T16:37:48.802 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.808+0000 7f346a3868c0 1 > prepared part key=_multipart_IP8YW2A75TCW.2~Q1466ONL3ANG.0 size=1048576 2024-04-07T16:37:48.806 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.812+0000 7f346a3868c0 1 > prepared part key=_multipart_IP8YW2A75TCW.2~Q1466ONL3ANG.1 size=125501 2024-04-07T16:37:48.806 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.812+0000 7f346a3868c0 1 > prepared multipart upload key=IP8YW2A75TCW upload=Q1466ONL3ANG tag=XE8S266JMPYU type=write size=2222653 2024-04-07T16:37:48.810 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.816+0000 7f346a3868c0 1 < completed part key=_multipart_LXOZ0T2KY6OR.2~SIKQ3KIWTSEQ.0 size=94553 2024-04-07T16:37:48.818 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.824+0000 7f346a3868c0 1 < completed multipart upload key=LXOZ0T2KY6OR upload=SIKQ3KIWTSEQ tag=MJFTQ682R9WP type=write size=1143129 2024-04-07T16:37:48.826 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.832+0000 7f346a3868c0 1 > prepared part key=_multipart_YOK8BAEOMPUR.2~93ZHW5X5GMJ7.0 size=1048576 2024-04-07T16:37:48.829 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.836+0000 7f346a3868c0 1 > prepared part key=_multipart_YOK8BAEOMPUR.2~93ZHW5X5GMJ7.1 size=1048576 2024-04-07T16:37:48.833 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.840+0000 7f346a3868c0 1 > prepared part key=_multipart_YOK8BAEOMPUR.2~93ZHW5X5GMJ7.2 size=173022 2024-04-07T16:37:48.833 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.840+0000 7f346a3868c0 1 > prepared multipart upload key=YOK8BAEOMPUR upload=93ZHW5X5GMJ7 tag=7CP0N1YU8U7C type=write size=3318750 2024-04-07T16:37:48.837 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.844+0000 7f346a3868c0 1 < completed delete operation key=LXOZ0T2KY6OR tag=4SJT1RGBCT1Y type=del 2024-04-07T16:37:48.842 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.848+0000 7f346a3868c0 1 > prepared operation key=EQR0FY86AT5X tag=N2B79G8GXY01 type=del size=535555 2024-04-07T16:37:48.847 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.856+0000 7f346a3868c0 1 < completed delete operation key=DXTO3AJ429AC tag=RAAU8IH5G162 type=del 2024-04-07T16:37:48.854 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.860+0000 7f346a3868c0 1 > prepared operation key=GZXHYZO5SMUP tag=ZRTN2Z9SEVBN type=del size=1691660 2024-04-07T16:37:48.856 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:37:48 smithi144 bash[66654]: cluster 2024-04-07T16:37:47.056916+0000 mgr.y (mgr.25175) 673 : cluster [DBG] pgmap v1308: 137 pgs: 137 active+clean; 583 KiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 94 KiB/s rd, 33 KiB/s wr, 165 op/s 2024-04-07T16:37:48.856 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.864+0000 7f346a3868c0 1 < completed delete operation key=QS4HYZFFAJNF tag=DG2PIHO1EXMK type=del 2024-04-07T16:37:48.861 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.868+0000 7f346a3868c0 1 > prepared operation key=NYET3212USVE tag=GTW7WDPXZLOW type=del size=1810099 2024-04-07T16:37:48.865 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.872+0000 7f346a3868c0 1 < canceled operation key=GZXHYZO5SMUP tag=QOV1NWBWEUG6 type=del size=1654771 2024-04-07T16:37:48.873 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.880+0000 7f346a3868c0 1 > prepared part key=_multipart_IP8YW2A75TCW.2~BHRPPPQKWBI4.0 size=1048576 2024-04-07T16:37:48.877 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.884+0000 7f346a3868c0 1 > prepared part key=_multipart_IP8YW2A75TCW.2~BHRPPPQKWBI4.1 size=1048576 2024-04-07T16:37:48.880 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.888+0000 7f346a3868c0 1 > prepared part key=_multipart_IP8YW2A75TCW.2~BHRPPPQKWBI4.2 size=432201 2024-04-07T16:37:48.880 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.888+0000 7f346a3868c0 1 > prepared multipart upload key=IP8YW2A75TCW upload=BHRPPPQKWBI4 tag=1SSH1D2Y2AQF type=write size=3577929 2024-04-07T16:37:48.889 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.896+0000 7f346a3868c0 1 < suggested update operation key=XJLQZ7IS9U1T tag=A0VDGJ4KQAVL type=del size=569458 2024-04-07T16:37:48.894 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.900+0000 7f346a3868c0 1 > prepared operation key=UIW3DN4NUYPP tag=V1VOXM3NLKHT type=del size=950749 2024-04-07T16:37:48.898 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.904+0000 7f346a3868c0 1 < completed delete operation key=1VKVQXSQMTU1 tag=W0CNM0ORLE8J type=del 2024-04-07T16:37:48.903 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.908+0000 7f346a3868c0 1 > prepared operation key=X5Q46GO8YTOK tag=CPPQQL4E2A3R type=del size=849014 2024-04-07T16:37:48.907 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.916+0000 7f346a3868c0 1 < completed part key=_multipart_1VBSUWNI3V1T.2~AH20RDFHU6ML.0 size=1048576 2024-04-07T16:37:48.911 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.920+0000 7f346a3868c0 1 < completed part key=_multipart_1VBSUWNI3V1T.2~AH20RDFHU6ML.1 size=146825 2024-04-07T16:37:48.918 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.924+0000 7f346a3868c0 1 < completed multipart upload key=1VBSUWNI3V1T upload=AH20RDFHU6ML tag=NQE6JKM5XC2E type=write size=2243977 2024-04-07T16:37:48.923 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.928+0000 7f346a3868c0 1 > prepared operation key=1WGUPH7X5UA0 tag=X2UGT5M0YUB2 type=del size=571401 2024-04-07T16:37:48.932 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.940+0000 7f346a3868c0 1 < suggested update operation key=NYET3212USVE tag=310Y5FM5ILK7 type=del size=4049332 2024-04-07T16:37:48.937 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.944+0000 7f346a3868c0 1 > prepared operation key=X5Q46GO8YTOK tag=4BPUI9A2W1PT type=del size=463992 2024-04-07T16:37:48.942 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.948+0000 7f346a3868c0 1 < completed delete operation key=U3EID5EK64HD tag=LTP0A5WAC95A type=del 2024-04-07T16:37:48.951 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.956+0000 7f346a3868c0 1 > prepared part key=_multipart_QTCIIPTXIRW2.2~IR8A220HUDHA.0 size=1048576 2024-04-07T16:37:48.954 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.960+0000 7f346a3868c0 1 > prepared part key=_multipart_QTCIIPTXIRW2.2~IR8A220HUDHA.1 size=683517 2024-04-07T16:37:48.954 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.960+0000 7f346a3868c0 1 > prepared multipart upload key=QTCIIPTXIRW2 upload=IR8A220HUDHA tag=AQXCOWQ80GPO type=write size=2780669 2024-04-07T16:37:48.958 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.964+0000 7f346a3868c0 1 < completed write operation key=CL4PP1RKZU0E tag=P3T4SX35PUFN type=write size=864592 2024-04-07T16:37:48.962 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.968+0000 7f346a3868c0 1 > prepared operation key=UIW3DN4NUYPP tag=SBKBSFPVHB03 type=write size=507711 2024-04-07T16:37:48.966 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.972+0000 7f346a3868c0 1 < completed part key=_multipart_6XDS0A7L0IY8.2~WQR7RFCZ9NGC.0 size=1048576 2024-04-07T16:37:48.970 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.976+0000 7f346a3868c0 1 < completed part key=_multipart_6XDS0A7L0IY8.2~WQR7RFCZ9NGC.1 size=1048576 2024-04-07T16:37:48.973 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.980+0000 7f346a3868c0 1 < completed part key=_multipart_6XDS0A7L0IY8.2~WQR7RFCZ9NGC.2 size=523113 2024-04-07T16:37:48.981 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.988+0000 7f346a3868c0 1 < completed multipart upload key=6XDS0A7L0IY8 upload=WQR7RFCZ9NGC tag=XRX0ZTJO2DRB type=write size=3668841 2024-04-07T16:37:48.985 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.992+0000 7f346a3868c0 1 > prepared operation key=C1BTVJNHIM0X tag=71FD2H3Q1GFA type=del size=602333 2024-04-07T16:37:48.990 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:48.996+0000 7f346a3868c0 1 < completed write operation key=7G5C6HBK6QWI tag=UOYF7NYVD474 type=write size=893388 2024-04-07T16:37:48.994 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.000+0000 7f346a3868c0 1 > prepared operation key=CL4PP1RKZU0E tag=BWRBDCWIR8X2 type=del size=465624 2024-04-07T16:37:48.999 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.004+0000 7f346a3868c0 1 < completed part key=_multipart_YOK8BAEOMPUR.2~3TWUL0NO3ZYU.0 size=1048576 2024-04-07T16:37:49.002 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.008+0000 7f346a3868c0 1 < completed part key=_multipart_YOK8BAEOMPUR.2~3TWUL0NO3ZYU.1 size=1014132 2024-04-07T16:37:49.010 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.016+0000 7f346a3868c0 1 < completed multipart upload key=YOK8BAEOMPUR upload=3TWUL0NO3ZYU tag=CDPXAWG7SF29 type=write size=3111284 2024-04-07T16:37:49.014 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.020+0000 7f346a3868c0 1 > prepared operation key=XJLQZ7IS9U1T tag=3CHPOWYDEQ1A type=del size=1075577 2024-04-07T16:37:49.018 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.024+0000 7f346a3868c0 1 < completed part key=_multipart_EOYFVEI50SQ1.2~81B03EH910WF.0 size=1048576 2024-04-07T16:37:49.021 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.028+0000 7f346a3868c0 1 < completed part key=_multipart_EOYFVEI50SQ1.2~81B03EH910WF.1 size=1048576 2024-04-07T16:37:49.024 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.032+0000 7f346a3868c0 1 < completed part key=_multipart_EOYFVEI50SQ1.2~81B03EH910WF.2 size=1026579 2024-04-07T16:37:49.032 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.040+0000 7f346a3868c0 1 < completed multipart upload key=EOYFVEI50SQ1 upload=81B03EH910WF tag=YVT7BKUFHEOP type=write size=4172307 2024-04-07T16:37:49.036 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.044+0000 7f346a3868c0 1 > prepared operation key=95YVKJ14JD8W tag=BSVBQUPV697H type=del size=359405 2024-04-07T16:37:49.040 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.048+0000 7f346a3868c0 1 < completed part key=_multipart_IP8YW2A75TCW.2~Q1466ONL3ANG.0 size=1048576 2024-04-07T16:37:49.043 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.052+0000 7f346a3868c0 1 < completed part key=_multipart_IP8YW2A75TCW.2~Q1466ONL3ANG.1 size=125501 2024-04-07T16:37:49.049 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.056+0000 7f346a3868c0 1 < completed multipart upload key=IP8YW2A75TCW upload=Q1466ONL3ANG tag=XE8S266JMPYU type=write size=2222653 2024-04-07T16:37:49.057 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.064+0000 7f346a3868c0 1 > prepared part key=_multipart_NYET3212USVE.2~NT4V8ZM993GB.0 size=1048576 2024-04-07T16:37:49.060 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.068+0000 7f346a3868c0 1 > prepared part key=_multipart_NYET3212USVE.2~NT4V8ZM993GB.1 size=1048576 2024-04-07T16:37:49.063 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.072+0000 7f346a3868c0 1 > prepared part key=_multipart_NYET3212USVE.2~NT4V8ZM993GB.2 size=451903 2024-04-07T16:37:49.063 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.072+0000 7f346a3868c0 1 > prepared multipart upload key=NYET3212USVE upload=NT4V8ZM993GB tag=R473SGBPBPBI type=write size=3597631 2024-04-07T16:37:49.067 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.076+0000 7f346a3868c0 1 < completed part key=_multipart_YOK8BAEOMPUR.2~93ZHW5X5GMJ7.0 size=1048576 2024-04-07T16:37:49.071 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.076+0000 7f346a3868c0 1 < completed part key=_multipart_YOK8BAEOMPUR.2~93ZHW5X5GMJ7.1 size=1048576 2024-04-07T16:37:49.075 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.080+0000 7f346a3868c0 1 < completed part key=_multipart_YOK8BAEOMPUR.2~93ZHW5X5GMJ7.2 size=173022 2024-04-07T16:37:49.082 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.088+0000 7f346a3868c0 1 < completed multipart upload key=YOK8BAEOMPUR upload=93ZHW5X5GMJ7 tag=7CP0N1YU8U7C type=write size=3318750 2024-04-07T16:37:49.090 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.096+0000 7f346a3868c0 1 > prepared part key=_multipart_E84U0SX44NQ8.2~6PL5Q9KOWYJ6.0 size=1048576 2024-04-07T16:37:49.093 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.100+0000 7f346a3868c0 1 > prepared part key=_multipart_E84U0SX44NQ8.2~6PL5Q9KOWYJ6.1 size=59173 2024-04-07T16:37:49.093 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.100+0000 7f346a3868c0 1 > prepared multipart upload key=E84U0SX44NQ8 upload=6PL5Q9KOWYJ6 tag=VJCXDNIOXWRY type=write size=2156325 2024-04-07T16:37:49.097 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.104+0000 7f346a3868c0 1 < completed delete operation key=EQR0FY86AT5X tag=N2B79G8GXY01 type=del 2024-04-07T16:37:49.105 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.112+0000 7f346a3868c0 1 > prepared part key=_multipart_NYET3212USVE.2~1QZ6FDMZKLR4.0 size=727587 2024-04-07T16:37:49.105 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.112+0000 7f346a3868c0 1 > prepared multipart upload key=NYET3212USVE upload=1QZ6FDMZKLR4 tag=BOP3OCS5GHC2 type=write size=1776163 2024-04-07T16:37:49.109 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.116+0000 7f346a3868c0 1 < completed delete operation key=GZXHYZO5SMUP tag=ZRTN2Z9SEVBN type=del 2024-04-07T16:37:49.115 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.120+0000 7f346a3868c0 1 > prepared operation key=EQR0FY86AT5X tag=CTVLMUP6TRDP type=del size=3110800 2024-04-07T16:37:49.120 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.128+0000 7f346a3868c0 -1 < failed to complete operation key=NYET3212USVE tag=GTW7WDPXZLOW type=del size=1810099: (22) Invalid argument 2024-04-07T16:37:49.125 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.132+0000 7f346a3868c0 1 > prepared operation key=1WGUPH7X5UA0 tag=4E8ZS2TVSIS9 type=del size=1280582 2024-04-07T16:37:49.131 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.140+0000 7f346a3868c0 1 < completed part key=_multipart_IP8YW2A75TCW.2~BHRPPPQKWBI4.0 size=1048576 2024-04-07T16:37:49.135 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.140+0000 7f346a3868c0 1 < completed part key=_multipart_IP8YW2A75TCW.2~BHRPPPQKWBI4.1 size=1048576 2024-04-07T16:37:49.139 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.144+0000 7f346a3868c0 1 < completed part key=_multipart_IP8YW2A75TCW.2~BHRPPPQKWBI4.2 size=432201 2024-04-07T16:37:49.147 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.152+0000 7f346a3868c0 1 < completed multipart upload key=IP8YW2A75TCW upload=BHRPPPQKWBI4 tag=1SSH1D2Y2AQF type=write size=3577929 2024-04-07T16:37:49.151 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.160+0000 7f346a3868c0 1 > prepared operation key=E84U0SX44NQ8 tag=AAP3HY9RHVAF type=write size=8152 2024-04-07T16:37:49.156 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.164+0000 7f346a3868c0 1 < completed delete operation key=UIW3DN4NUYPP tag=V1VOXM3NLKHT type=del 2024-04-07T16:37:49.160 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.168+0000 7f346a3868c0 1 > prepared operation key=YOK8BAEOMPUR tag=UT8VYZVBR5Q1 type=del size=3205878 2024-04-07T16:37:49.169 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.176+0000 7f346a3868c0 1 < suggested update operation key=X5Q46GO8YTOK tag=CPPQQL4E2A3R type=del size=849014 2024-04-07T16:37:49.174 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.180+0000 7f346a3868c0 1 > prepared operation key=GBQSME7SY1AH tag=6MIEC820O8HO type=del size=3988237 2024-04-07T16:37:49.178 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.184+0000 7f346a3868c0 1 < canceled operation key=1WGUPH7X5UA0 tag=X2UGT5M0YUB2 type=del size=571401 2024-04-07T16:37:49.184 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.192+0000 7f346a3868c0 1 > prepared operation key=1VKVQXSQMTU1 tag=A8TUV5EJT0S8 type=write size=448958 2024-04-07T16:37:49.190 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.196+0000 7f346a3868c0 -1 < failed to complete operation key=X5Q46GO8YTOK tag=4BPUI9A2W1PT type=del size=463992: (22) Invalid argument 2024-04-07T16:37:49.195 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.200+0000 7f346a3868c0 1 > prepared operation key=EQR0FY86AT5X tag=X93KP19ZENE1 type=del size=384929 2024-04-07T16:37:49.199 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.208+0000 7f346a3868c0 1 < completed part key=_multipart_QTCIIPTXIRW2.2~IR8A220HUDHA.0 size=1048576 2024-04-07T16:37:49.199 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.208+0000 7f346a3868c0 1 < canceled part key=_multipart_QTCIIPTXIRW2.2~IR8A220HUDHA.1 size=683517 2024-04-07T16:37:49.207 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.212+0000 7f346a3868c0 1 < canceled multipart upload key=QTCIIPTXIRW2 upload=IR8A220HUDHA tag=AQXCOWQ80GPO type=write size=2780669 2024-04-07T16:37:49.212 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.220+0000 7f346a3868c0 1 > prepared operation key=U3EID5EK64HD tag=KC870UNISTHR type=del size=2362668 2024-04-07T16:37:49.216 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.224+0000 7f346a3868c0 1 < completed write operation key=UIW3DN4NUYPP tag=SBKBSFPVHB03 type=write size=507711 2024-04-07T16:37:49.220 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.228+0000 7f346a3868c0 1 > prepared operation key=NYET3212USVE tag=6IIPX6AZG5YV type=del size=754480 2024-04-07T16:37:49.225 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.232+0000 7f346a3868c0 1 < completed delete operation key=C1BTVJNHIM0X tag=71FD2H3Q1GFA type=del 2024-04-07T16:37:49.229 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.236+0000 7f346a3868c0 1 > prepared operation key=E84U0SX44NQ8 tag=KRUHZWFIGU2L type=del size=1618408 2024-04-07T16:37:49.234 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.240+0000 7f346a3868c0 1 < completed delete operation key=CL4PP1RKZU0E tag=BWRBDCWIR8X2 type=del 2024-04-07T16:37:49.242 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.248+0000 7f346a3868c0 1 > prepared part key=_multipart_CL4PP1RKZU0E.2~0K8TPDSMHFKU.0 size=1048576 2024-04-07T16:37:49.245 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.252+0000 7f346a3868c0 1 > prepared part key=_multipart_CL4PP1RKZU0E.2~0K8TPDSMHFKU.1 size=1048576 2024-04-07T16:37:49.249 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.256+0000 7f346a3868c0 1 > prepared part key=_multipart_CL4PP1RKZU0E.2~0K8TPDSMHFKU.2 size=817227 2024-04-07T16:37:49.249 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.256+0000 7f346a3868c0 1 > prepared multipart upload key=CL4PP1RKZU0E upload=0K8TPDSMHFKU tag=OLHQN5H5IR88 type=write size=3962955 2024-04-07T16:37:49.253 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.260+0000 7f346a3868c0 1 < completed delete operation key=XJLQZ7IS9U1T tag=3CHPOWYDEQ1A type=del 2024-04-07T16:37:49.257 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.264+0000 7f346a3868c0 1 > prepared operation key=E84U0SX44NQ8 tag=ZDPOZ1JHRIN7 type=write size=594286 2024-04-07T16:37:49.262 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.268+0000 7f346a3868c0 1 < completed delete operation key=95YVKJ14JD8W tag=BSVBQUPV697H type=del 2024-04-07T16:37:49.270 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.276+0000 7f346a3868c0 1 > prepared part key=_multipart_ROFH30W3UCAX.2~JMIC0004LX95.0 size=1048576 2024-04-07T16:37:49.273 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.280+0000 7f346a3868c0 1 > prepared part key=_multipart_ROFH30W3UCAX.2~JMIC0004LX95.1 size=1048576 2024-04-07T16:37:49.277 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.284+0000 7f346a3868c0 1 > prepared part key=_multipart_ROFH30W3UCAX.2~JMIC0004LX95.2 size=562463 2024-04-07T16:37:49.277 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.284+0000 7f346a3868c0 1 > prepared multipart upload key=ROFH30W3UCAX upload=JMIC0004LX95 tag=0U3O61P08VCX type=write size=3708191 2024-04-07T16:37:49.277 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.284+0000 7f346a3868c0 1 < canceled part key=_multipart_NYET3212USVE.2~NT4V8ZM993GB.0 size=1048576 2024-04-07T16:37:49.281 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.288+0000 7f346a3868c0 1 < completed part key=_multipart_NYET3212USVE.2~NT4V8ZM993GB.1 size=1048576 2024-04-07T16:37:49.285 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.292+0000 7f346a3868c0 1 < completed part key=_multipart_NYET3212USVE.2~NT4V8ZM993GB.2 size=451903 2024-04-07T16:37:49.293 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.300+0000 7f346a3868c0 1 < canceled multipart upload key=NYET3212USVE upload=NT4V8ZM993GB tag=R473SGBPBPBI type=write size=3597631 2024-04-07T16:37:49.298 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.304+0000 7f346a3868c0 1 > prepared operation key=X5Q46GO8YTOK tag=AOLM5MXTPO4N type=del size=3475387 2024-04-07T16:37:49.300 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.308+0000 7f346a3868c0 1 < canceled part key=_multipart_E84U0SX44NQ8.2~6PL5Q9KOWYJ6.0 size=1048576 2024-04-07T16:37:49.300 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.308+0000 7f346a3868c0 1 < canceled part key=_multipart_E84U0SX44NQ8.2~6PL5Q9KOWYJ6.1 size=59173 2024-04-07T16:37:49.307 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.316+0000 7f346a3868c0 1 < canceled multipart upload key=E84U0SX44NQ8 upload=6PL5Q9KOWYJ6 tag=VJCXDNIOXWRY type=write size=2156325 2024-04-07T16:37:49.312 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.320+0000 7f346a3868c0 1 > prepared operation key=YXMDZB23YTB3 tag=IX7H9T7K0VH3 type=del size=371532 2024-04-07T16:37:49.316 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.324+0000 7f346a3868c0 1 < completed part key=_multipart_NYET3212USVE.2~1QZ6FDMZKLR4.0 size=727587 2024-04-07T16:37:49.323 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.332+0000 7f346a3868c0 1 < completed multipart upload key=NYET3212USVE upload=1QZ6FDMZKLR4 tag=BOP3OCS5GHC2 type=write size=1776163 2024-04-07T16:37:49.328 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.336+0000 7f346a3868c0 1 > prepared operation key=1VKVQXSQMTU1 tag=0GFWIGUGOBAN type=del size=3972380 2024-04-07T16:37:49.332 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.340+0000 7f346a3868c0 1 < completed delete operation key=EQR0FY86AT5X tag=CTVLMUP6TRDP type=del 2024-04-07T16:37:49.341 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.348+0000 7f346a3868c0 1 > prepared part key=_multipart_N14BUXLJONHZ.2~5YJDXSLVLJO3.0 size=303302 2024-04-07T16:37:49.341 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.348+0000 7f346a3868c0 1 > prepared multipart upload key=N14BUXLJONHZ upload=5YJDXSLVLJO3 tag=N8L8DMRSYBXP type=write size=1351878 2024-04-07T16:37:49.345 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.352+0000 7f346a3868c0 1 < completed delete operation key=1WGUPH7X5UA0 tag=4E8ZS2TVSIS9 type=del 2024-04-07T16:37:49.350 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.356+0000 7f346a3868c0 1 > prepared operation key=95YVKJ14JD8W tag=XXYN8Q145V7G type=write size=883078 2024-04-07T16:37:49.355 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.360+0000 7f346a3868c0 1 < completed write operation key=E84U0SX44NQ8 tag=AAP3HY9RHVAF type=write size=8152 2024-04-07T16:37:49.363 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.372+0000 7f346a3868c0 1 > prepared part key=_multipart_1VKVQXSQMTU1.2~ZHV3FM65JWHS.0 size=1048576 2024-04-07T16:37:49.367 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.372+0000 7f346a3868c0 1 > prepared part key=_multipart_1VKVQXSQMTU1.2~ZHV3FM65JWHS.1 size=1048576 2024-04-07T16:37:49.370 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.376+0000 7f346a3868c0 1 > prepared part key=_multipart_1VKVQXSQMTU1.2~ZHV3FM65JWHS.2 size=307280 2024-04-07T16:37:49.370 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.376+0000 7f346a3868c0 1 > prepared multipart upload key=1VKVQXSQMTU1 upload=ZHV3FM65JWHS tag=VBFSSNDDO1FT type=write size=3453008 2024-04-07T16:37:49.375 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.384+0000 7f346a3868c0 1 < completed delete operation key=YOK8BAEOMPUR tag=UT8VYZVBR5Q1 type=del 2024-04-07T16:37:49.380 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.388+0000 7f346a3868c0 1 > prepared operation key=EQR0FY86AT5X tag=NMV9ERBTM1C7 type=del size=933671 2024-04-07T16:37:49.385 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.392+0000 7f346a3868c0 1 < canceled operation key=GBQSME7SY1AH tag=6MIEC820O8HO type=del size=3988237 2024-04-07T16:37:49.390 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.396+0000 7f346a3868c0 1 > prepared operation key=95YVKJ14JD8W tag=0JGD1CEPC8WN type=del size=1854597 2024-04-07T16:37:49.396 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.404+0000 7f346a3868c0 1 < canceled operation key=1VKVQXSQMTU1 tag=A8TUV5EJT0S8 type=write size=448958 2024-04-07T16:37:49.405 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.412+0000 7f346a3868c0 1 > prepared part key=_multipart_XJLQZ7IS9U1T.2~22D9T4FI62OU.0 size=1048576 2024-04-07T16:37:49.409 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.416+0000 7f346a3868c0 1 > prepared part key=_multipart_XJLQZ7IS9U1T.2~22D9T4FI62OU.1 size=1048576 2024-04-07T16:37:49.413 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.420+0000 7f346a3868c0 1 > prepared part key=_multipart_XJLQZ7IS9U1T.2~22D9T4FI62OU.2 size=318405 2024-04-07T16:37:49.413 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.420+0000 7f346a3868c0 1 > prepared multipart upload key=XJLQZ7IS9U1T upload=22D9T4FI62OU tag=AXNDE2L05FN4 type=write size=3464133 2024-04-07T16:37:49.419 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.428+0000 7f346a3868c0 1 < canceled operation key=EQR0FY86AT5X tag=X93KP19ZENE1 type=del size=384929 2024-04-07T16:37:49.428 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.436+0000 7f346a3868c0 1 > prepared part key=_multipart_XJLQZ7IS9U1T.2~D49SDQE8VVLZ.0 size=668122 2024-04-07T16:37:49.429 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.436+0000 7f346a3868c0 1 > prepared multipart upload key=XJLQZ7IS9U1T upload=D49SDQE8VVLZ tag=9OL4S2LUSNTP type=write size=1716698 2024-04-07T16:37:49.433 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.440+0000 7f346a3868c0 1 < completed delete operation key=U3EID5EK64HD tag=KC870UNISTHR type=del 2024-04-07T16:37:49.437 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.444+0000 7f346a3868c0 1 > prepared operation key=1WGUPH7X5UA0 tag=MY8FQL44N0BX type=del size=1527506 2024-04-07T16:37:49.442 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.448+0000 7f346a3868c0 1 < completed delete operation key=NYET3212USVE tag=6IIPX6AZG5YV type=del 2024-04-07T16:37:49.446 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.452+0000 7f346a3868c0 1 > prepared operation key=U3EID5EK64HD tag=43ERK8Y8ANIQ type=del size=3741539 2024-04-07T16:37:49.451 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.460+0000 7f346a3868c0 1 < completed delete operation key=E84U0SX44NQ8 tag=KRUHZWFIGU2L type=del 2024-04-07T16:37:49.456 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.464+0000 7f346a3868c0 1 > prepared operation key=U3EID5EK64HD tag=SYTALYBFM97I type=del size=952768 2024-04-07T16:37:49.461 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.468+0000 7f346a3868c0 1 < completed part key=_multipart_CL4PP1RKZU0E.2~0K8TPDSMHFKU.0 size=1048576 2024-04-07T16:37:49.466 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.472+0000 7f346a3868c0 1 < completed part key=_multipart_CL4PP1RKZU0E.2~0K8TPDSMHFKU.1 size=1048576 2024-04-07T16:37:49.471 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.476+0000 7f346a3868c0 1 < completed part key=_multipart_CL4PP1RKZU0E.2~0K8TPDSMHFKU.2 size=817227 2024-04-07T16:37:49.479 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.488+0000 7f346a3868c0 1 < completed multipart upload key=CL4PP1RKZU0E upload=0K8TPDSMHFKU tag=OLHQN5H5IR88 type=write size=3962955 2024-04-07T16:37:49.484 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.492+0000 7f346a3868c0 1 > prepared operation key=EQR0FY86AT5X tag=BJD64B3J29MU type=del size=1633712 2024-04-07T16:37:49.488 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.496+0000 7f346a3868c0 1 < completed write operation key=E84U0SX44NQ8 tag=ZDPOZ1JHRIN7 type=write size=594286 2024-04-07T16:37:49.493 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.500+0000 7f346a3868c0 1 > prepared operation key=X5Q46GO8YTOK tag=USW5U18PZ42S type=del size=3273025 2024-04-07T16:37:49.497 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.504+0000 7f346a3868c0 1 < completed part key=_multipart_ROFH30W3UCAX.2~JMIC0004LX95.0 size=1048576 2024-04-07T16:37:49.500 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.508+0000 7f346a3868c0 1 < completed part key=_multipart_ROFH30W3UCAX.2~JMIC0004LX95.1 size=1048576 2024-04-07T16:37:49.504 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.512+0000 7f346a3868c0 1 < completed part key=_multipart_ROFH30W3UCAX.2~JMIC0004LX95.2 size=562463 2024-04-07T16:37:49.511 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.516+0000 7f346a3868c0 1 < completed multipart upload key=ROFH30W3UCAX upload=JMIC0004LX95 tag=0U3O61P08VCX type=write size=3708191 2024-04-07T16:37:49.518 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.524+0000 7f346a3868c0 1 > prepared part key=_multipart_XJLQZ7IS9U1T.2~I0TCSTLU5YYV.0 size=1048576 2024-04-07T16:37:49.522 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.528+0000 7f346a3868c0 1 > prepared part key=_multipart_XJLQZ7IS9U1T.2~I0TCSTLU5YYV.1 size=696219 2024-04-07T16:37:49.522 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.528+0000 7f346a3868c0 1 > prepared multipart upload key=XJLQZ7IS9U1T upload=I0TCSTLU5YYV tag=EMA18U9ZNKI1 type=write size=2793371 2024-04-07T16:37:49.526 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.532+0000 7f346a3868c0 1 < completed delete operation key=X5Q46GO8YTOK tag=AOLM5MXTPO4N type=del 2024-04-07T16:37:49.531 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.536+0000 7f346a3868c0 1 > prepared operation key=QTCIIPTXIRW2 tag=89J5BPF4Z4AS type=del size=3846025 2024-04-07T16:37:49.535 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.544+0000 7f346a3868c0 1 < completed delete operation key=YXMDZB23YTB3 tag=IX7H9T7K0VH3 type=del 2024-04-07T16:37:49.543 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.552+0000 7f346a3868c0 1 > prepared part key=_multipart_E84U0SX44NQ8.2~FIE51YFIWPET.0 size=1048576 2024-04-07T16:37:49.546 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.552+0000 7f346a3868c0 1 > prepared part key=_multipart_E84U0SX44NQ8.2~FIE51YFIWPET.1 size=1048576 2024-04-07T16:37:49.549 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.556+0000 7f346a3868c0 1 > prepared part key=_multipart_E84U0SX44NQ8.2~FIE51YFIWPET.2 size=783270 2024-04-07T16:37:49.549 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.556+0000 7f346a3868c0 1 > prepared multipart upload key=E84U0SX44NQ8 upload=FIE51YFIWPET tag=SZDFUI5PR83U type=write size=3928998 2024-04-07T16:37:49.557 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.564+0000 7f346a3868c0 1 < suggested update operation key=1VKVQXSQMTU1 tag=0GFWIGUGOBAN type=del size=3972380 2024-04-07T16:37:49.561 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.568+0000 7f346a3868c0 1 > prepared operation key=CL4PP1RKZU0E tag=BL2Y4OBTKNUZ type=write size=123835 2024-04-07T16:37:49.566 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.572+0000 7f346a3868c0 1 < completed part key=_multipart_N14BUXLJONHZ.2~5YJDXSLVLJO3.0 size=303302 2024-04-07T16:37:49.573 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.580+0000 7f346a3868c0 1 < completed multipart upload key=N14BUXLJONHZ upload=5YJDXSLVLJO3 tag=N8L8DMRSYBXP type=write size=1351878 2024-04-07T16:37:49.578 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.584+0000 7f346a3868c0 1 > prepared operation key=X5Q46GO8YTOK tag=MRGRDB7YEBMV type=del size=1836566 2024-04-07T16:37:49.582 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.588+0000 7f346a3868c0 1 < completed write operation key=95YVKJ14JD8W tag=XXYN8Q145V7G type=write size=883078 2024-04-07T16:37:49.587 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.592+0000 7f346a3868c0 1 > prepared operation key=XJLQZ7IS9U1T tag=NN241H8NKS7K type=del size=1476885 2024-04-07T16:37:49.591 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.600+0000 7f346a3868c0 1 < completed part key=_multipart_1VKVQXSQMTU1.2~ZHV3FM65JWHS.0 size=1048576 2024-04-07T16:37:49.592 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.600+0000 7f346a3868c0 1 < canceled part key=_multipart_1VKVQXSQMTU1.2~ZHV3FM65JWHS.1 size=1048576 2024-04-07T16:37:49.595 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.604+0000 7f346a3868c0 1 < completed part key=_multipart_1VKVQXSQMTU1.2~ZHV3FM65JWHS.2 size=307280 2024-04-07T16:37:49.603 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.608+0000 7f346a3868c0 1 < canceled multipart upload key=1VKVQXSQMTU1 upload=ZHV3FM65JWHS tag=VBFSSNDDO1FT type=write size=3453008 2024-04-07T16:37:49.607 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.616+0000 7f346a3868c0 1 > prepared operation key=ROFH30W3UCAX tag=4S2TVB9WQCV8 type=del size=506370 2024-04-07T16:37:49.615 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.624+0000 7f346a3868c0 1 < suggested update operation key=EQR0FY86AT5X tag=NMV9ERBTM1C7 type=del size=933671 2024-04-07T16:37:49.623 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.632+0000 7f346a3868c0 1 > prepared part key=_multipart_IX3F0IUOKBX1.2~VL6CLQXBV8EW.0 size=301377 2024-04-07T16:37:49.623 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.632+0000 7f346a3868c0 1 > prepared multipart upload key=IX3F0IUOKBX1 upload=VL6CLQXBV8EW tag=F5QZUFQE676G type=write size=1349953 2024-04-07T16:37:49.628 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.636+0000 7f346a3868c0 1 < completed delete operation key=95YVKJ14JD8W tag=0JGD1CEPC8WN type=del 2024-04-07T16:37:49.637 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.644+0000 7f346a3868c0 1 > prepared part key=_multipart_YOK8BAEOMPUR.2~5HC3O855Q4BM.0 size=316651 2024-04-07T16:37:49.637 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.644+0000 7f346a3868c0 1 > prepared multipart upload key=YOK8BAEOMPUR upload=5HC3O855Q4BM tag=9M9RSVC2O7QK type=write size=1365227 2024-04-07T16:37:49.642 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.648+0000 7f346a3868c0 1 < completed part key=_multipart_XJLQZ7IS9U1T.2~22D9T4FI62OU.0 size=1048576 2024-04-07T16:37:49.646 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.652+0000 7f346a3868c0 1 < completed part key=_multipart_XJLQZ7IS9U1T.2~22D9T4FI62OU.1 size=1048576 2024-04-07T16:37:49.649 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.656+0000 7f346a3868c0 1 < completed part key=_multipart_XJLQZ7IS9U1T.2~22D9T4FI62OU.2 size=318405 2024-04-07T16:37:49.657 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.664+0000 7f346a3868c0 1 < completed multipart upload key=XJLQZ7IS9U1T upload=22D9T4FI62OU tag=AXNDE2L05FN4 type=write size=3464133 2024-04-07T16:37:49.662 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.668+0000 7f346a3868c0 1 > prepared operation key=IP8YW2A75TCW tag=QSJYL28BSD78 type=del size=1709678 2024-04-07T16:37:49.666 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.672+0000 7f346a3868c0 1 < completed part key=_multipart_XJLQZ7IS9U1T.2~D49SDQE8VVLZ.0 size=668122 2024-04-07T16:37:49.674 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.680+0000 7f346a3868c0 1 < completed multipart upload key=XJLQZ7IS9U1T upload=D49SDQE8VVLZ tag=9OL4S2LUSNTP type=write size=1716698 2024-04-07T16:37:49.679 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.688+0000 7f346a3868c0 1 > prepared operation key=HBLEXONKXJTL tag=CRM3IQ2JQSDI type=del size=2532583 2024-04-07T16:37:49.684 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.692+0000 7f346a3868c0 1 < completed delete operation key=1WGUPH7X5UA0 tag=MY8FQL44N0BX type=del 2024-04-07T16:37:49.689 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.696+0000 7f346a3868c0 1 > prepared operation key=IX3F0IUOKBX1 tag=LIFNU3CH1NP0 type=write size=358083 2024-04-07T16:37:49.693 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.700+0000 7f346a3868c0 1 < canceled operation key=U3EID5EK64HD tag=43ERK8Y8ANIQ type=del size=3741539 2024-04-07T16:37:49.702 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.708+0000 7f346a3868c0 1 > prepared part key=_multipart_IP8YW2A75TCW.2~OH5BYVOZGJ9S.0 size=425829 2024-04-07T16:37:49.702 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.708+0000 7f346a3868c0 1 > prepared multipart upload key=IP8YW2A75TCW upload=OH5BYVOZGJ9S tag=8FPHNR4V7OSN type=write size=1474405 2024-04-07T16:37:49.706 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.712+0000 7f346a3868c0 1 < completed delete operation key=U3EID5EK64HD tag=SYTALYBFM97I type=del 2024-04-07T16:37:49.711 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.720+0000 7f346a3868c0 1 > prepared operation key=1VKVQXSQMTU1 tag=JS3UKKN1LW0C type=del size=935617 2024-04-07T16:37:49.717 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.724+0000 7f346a3868c0 -1 < failed to complete operation key=EQR0FY86AT5X tag=BJD64B3J29MU type=del size=1633712: (22) Invalid argument 2024-04-07T16:37:49.721 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.728+0000 7f346a3868c0 1 > prepared operation key=YXMDZB23YTB3 tag=34I3K9S00XSV type=del size=2360400 2024-04-07T16:37:49.726 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.732+0000 7f346a3868c0 1 < completed delete operation key=X5Q46GO8YTOK tag=USW5U18PZ42S type=del 2024-04-07T16:37:49.731 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.740+0000 7f346a3868c0 1 > prepared operation key=CL4PP1RKZU0E tag=I0OUUGCSUQKL type=del size=4137881 2024-04-07T16:37:49.736 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.744+0000 7f346a3868c0 1 < completed part key=_multipart_XJLQZ7IS9U1T.2~I0TCSTLU5YYV.0 size=1048576 2024-04-07T16:37:49.740 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.748+0000 7f346a3868c0 1 < completed part key=_multipart_XJLQZ7IS9U1T.2~I0TCSTLU5YYV.1 size=696219 2024-04-07T16:37:49.748 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.756+0000 7f346a3868c0 1 < completed multipart upload key=XJLQZ7IS9U1T upload=I0TCSTLU5YYV tag=EMA18U9ZNKI1 type=write size=2793371 2024-04-07T16:37:49.753 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.760+0000 7f346a3868c0 1 > prepared operation key=HBLEXONKXJTL tag=UDJXJV5XNYG2 type=del size=2437292 2024-04-07T16:37:49.758 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.764+0000 7f346a3868c0 1 < completed delete operation key=QTCIIPTXIRW2 tag=89J5BPF4Z4AS type=del 2024-04-07T16:37:49.762 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.768+0000 7f346a3868c0 1 > prepared operation key=N14BUXLJONHZ tag=LY5YLTEHJKII type=del size=2642623 2024-04-07T16:37:49.767 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.776+0000 7f346a3868c0 1 < completed part key=_multipart_E84U0SX44NQ8.2~FIE51YFIWPET.0 size=1048576 2024-04-07T16:37:49.770 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.776+0000 7f346a3868c0 1 < completed part key=_multipart_E84U0SX44NQ8.2~FIE51YFIWPET.1 size=1048576 2024-04-07T16:37:49.774 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.780+0000 7f346a3868c0 1 < completed part key=_multipart_E84U0SX44NQ8.2~FIE51YFIWPET.2 size=783270 2024-04-07T16:37:49.782 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.788+0000 7f346a3868c0 1 < completed multipart upload key=E84U0SX44NQ8 upload=FIE51YFIWPET tag=SZDFUI5PR83U type=write size=3928998 2024-04-07T16:37:49.786 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.792+0000 7f346a3868c0 1 > prepared operation key=N14BUXLJONHZ tag=FV8OHSXT4DPG type=write size=338657 2024-04-07T16:37:49.791 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.796+0000 7f346a3868c0 1 < completed write operation key=CL4PP1RKZU0E tag=BL2Y4OBTKNUZ type=write size=123835 2024-04-07T16:37:49.795 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.804+0000 7f346a3868c0 1 > prepared operation key=E84U0SX44NQ8 tag=8MG0DZ0OP2Q7 type=del size=805973 2024-04-07T16:37:49.800 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.808+0000 7f346a3868c0 1 < completed delete operation key=X5Q46GO8YTOK tag=MRGRDB7YEBMV type=del 2024-04-07T16:37:49.808 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.816+0000 7f346a3868c0 1 > prepared part key=_multipart_8F81G0P0YBFW.2~SGJI1BEC6LP3.0 size=456874 2024-04-07T16:37:49.809 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.816+0000 7f346a3868c0 1 > prepared multipart upload key=8F81G0P0YBFW upload=SGJI1BEC6LP3 tag=F6GI7HDV96K9 type=write size=1505450 2024-04-07T16:37:49.813 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.820+0000 7f346a3868c0 1 < completed delete operation key=XJLQZ7IS9U1T tag=NN241H8NKS7K type=del 2024-04-07T16:37:49.817 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.824+0000 7f346a3868c0 1 > prepared operation key=UIW3DN4NUYPP tag=2JQEFC8TYO31 type=del size=1765909 2024-04-07T16:37:49.822 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.828+0000 7f346a3868c0 1 < completed delete operation key=ROFH30W3UCAX tag=4S2TVB9WQCV8 type=del 2024-04-07T16:37:49.827 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.836+0000 7f346a3868c0 1 > prepared operation key=C1BTVJNHIM0X tag=6JQ8TACYX68V type=del size=649096 2024-04-07T16:37:49.832 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.840+0000 7f346a3868c0 1 < completed part key=_multipart_IX3F0IUOKBX1.2~VL6CLQXBV8EW.0 size=301377 2024-04-07T16:37:49.839 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.848+0000 7f346a3868c0 1 < completed multipart upload key=IX3F0IUOKBX1 upload=VL6CLQXBV8EW tag=F5QZUFQE676G type=write size=1349953 2024-04-07T16:37:49.848 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.856+0000 7f346a3868c0 1 > prepared part key=_multipart_U3EID5EK64HD.2~HFQK3FALLFY0.0 size=1048576 2024-04-07T16:37:49.852 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.860+0000 7f346a3868c0 1 > prepared part key=_multipart_U3EID5EK64HD.2~HFQK3FALLFY0.1 size=881834 2024-04-07T16:37:49.852 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.860+0000 7f346a3868c0 1 > prepared multipart upload key=U3EID5EK64HD upload=HFQK3FALLFY0 tag=Z5YZYJF7DGEZ type=write size=2978986 2024-04-07T16:37:49.858 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.864+0000 7f346a3868c0 1 < completed part key=_multipart_YOK8BAEOMPUR.2~5HC3O855Q4BM.0 size=316651 2024-04-07T16:37:49.868 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.872+0000 7f346a3868c0 1 < completed multipart upload key=YOK8BAEOMPUR upload=5HC3O855Q4BM tag=9M9RSVC2O7QK type=write size=1365227 2024-04-07T16:37:49.870 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.876+0000 7f346a3868c0 1 > prepared operation key=QS4HYZFFAJNF tag=B6MP43W3D21P type=del size=4172254 2024-04-07T16:37:49.875 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.880+0000 7f346a3868c0 1 < completed delete operation key=IP8YW2A75TCW tag=QSJYL28BSD78 type=del 2024-04-07T16:37:49.883 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.892+0000 7f346a3868c0 1 > prepared part key=_multipart_YOK8BAEOMPUR.2~UW1CNWYSK031.0 size=1048576 2024-04-07T16:37:49.887 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.892+0000 7f346a3868c0 1 > prepared part key=_multipart_YOK8BAEOMPUR.2~UW1CNWYSK031.1 size=645393 2024-04-07T16:37:49.887 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.892+0000 7f346a3868c0 1 > prepared multipart upload key=YOK8BAEOMPUR upload=UW1CNWYSK031 tag=PBAEXHLPR574 type=write size=2742545 2024-04-07T16:37:49.895 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.904+0000 7f346a3868c0 1 < suggested update operation key=HBLEXONKXJTL tag=CRM3IQ2JQSDI type=del size=2532583 2024-04-07T16:37:49.904 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.912+0000 7f346a3868c0 1 > prepared part key=_multipart_QS4HYZFFAJNF.2~507BYP0TY6BA.0 size=863689 2024-04-07T16:37:49.904 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.912+0000 7f346a3868c0 1 > prepared multipart upload key=QS4HYZFFAJNF upload=507BYP0TY6BA tag=J2E9UIZ16OB2 type=write size=1912265 2024-04-07T16:37:49.909 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.916+0000 7f346a3868c0 1 < completed write operation key=IX3F0IUOKBX1 tag=LIFNU3CH1NP0 type=write size=358083 2024-04-07T16:37:49.915 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.920+0000 7f346a3868c0 1 > prepared operation key=EOYFVEI50SQ1 tag=P86STZ30HTH5 type=del size=2475668 2024-04-07T16:37:49.920 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.928+0000 7f346a3868c0 1 < completed part key=_multipart_IP8YW2A75TCW.2~OH5BYVOZGJ9S.0 size=425829 2024-04-07T16:37:49.927 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.936+0000 7f346a3868c0 1 < completed multipart upload key=IP8YW2A75TCW upload=OH5BYVOZGJ9S tag=8FPHNR4V7OSN type=write size=1474405 2024-04-07T16:37:49.932 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.940+0000 7f346a3868c0 1 > prepared operation key=ROFH30W3UCAX tag=NXXWFAVD8EOV type=del size=3773214 2024-04-07T16:37:49.941 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.948+0000 7f346a3868c0 1 < suggested update operation key=1VKVQXSQMTU1 tag=JS3UKKN1LW0C type=del size=935617 2024-04-07T16:37:49.950 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.956+0000 7f346a3868c0 1 > prepared part key=_multipart_HBLEXONKXJTL.2~G2DVH6BTWTGC.0 size=1048576 2024-04-07T16:37:49.954 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.960+0000 7f346a3868c0 1 > prepared part key=_multipart_HBLEXONKXJTL.2~G2DVH6BTWTGC.1 size=925607 2024-04-07T16:37:49.954 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.960+0000 7f346a3868c0 1 > prepared multipart upload key=HBLEXONKXJTL upload=G2DVH6BTWTGC tag=28VPF6C97WHI type=write size=3022759 2024-04-07T16:37:49.959 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.968+0000 7f346a3868c0 1 < completed delete operation key=YXMDZB23YTB3 tag=34I3K9S00XSV type=del 2024-04-07T16:37:49.967 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.972+0000 7f346a3868c0 1 > prepared part key=_multipart_6XDS0A7L0IY8.2~K7TCBF0Z3KQZ.0 size=1048576 2024-04-07T16:37:49.970 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.976+0000 7f346a3868c0 1 > prepared part key=_multipart_6XDS0A7L0IY8.2~K7TCBF0Z3KQZ.1 size=374104 2024-04-07T16:37:49.970 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.976+0000 7f346a3868c0 1 > prepared multipart upload key=6XDS0A7L0IY8 upload=K7TCBF0Z3KQZ tag=AB8J68VC8SAM type=write size=2471256 2024-04-07T16:37:49.980 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.988+0000 7f346a3868c0 1 < suggested remove operation key=CL4PP1RKZU0E tag=I0OUUGCSUQKL type=del 2024-04-07T16:37:49.985 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.992+0000 7f346a3868c0 1 > prepared operation key=95YVKJ14JD8W tag=OLR4E0O83D6A type=del size=1306477 2024-04-07T16:37:49.989 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:49.996+0000 7f346a3868c0 -1 < failed to complete operation key=HBLEXONKXJTL tag=UDJXJV5XNYG2 type=del size=2437292: (22) Invalid argument 2024-04-07T16:37:49.997 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.004+0000 7f346a3868c0 1 > prepared part key=_multipart_ROFH30W3UCAX.2~3VW8WZ11Z2YL.0 size=1048576 2024-04-07T16:37:50.000 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.008+0000 7f346a3868c0 1 > prepared part key=_multipart_ROFH30W3UCAX.2~3VW8WZ11Z2YL.1 size=498447 2024-04-07T16:37:50.000 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.008+0000 7f346a3868c0 1 > prepared multipart upload key=ROFH30W3UCAX upload=3VW8WZ11Z2YL tag=XP3OP94D306M type=write size=2595599 2024-04-07T16:37:50.004 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.012+0000 7f346a3868c0 1 < completed delete operation key=N14BUXLJONHZ tag=LY5YLTEHJKII type=del 2024-04-07T16:37:50.012 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.020+0000 7f346a3868c0 1 > prepared part key=_multipart_IX3F0IUOKBX1.2~CA6230X9TRK3.0 size=1048576 2024-04-07T16:37:50.016 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.024+0000 7f346a3868c0 1 > prepared part key=_multipart_IX3F0IUOKBX1.2~CA6230X9TRK3.1 size=79563 2024-04-07T16:37:50.016 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.024+0000 7f346a3868c0 1 > prepared multipart upload key=IX3F0IUOKBX1 upload=CA6230X9TRK3 tag=SGBT5OHS0KN0 type=write size=2176715 2024-04-07T16:37:50.026 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.032+0000 7f346a3868c0 1 < suggested remove operation key=N14BUXLJONHZ tag=FV8OHSXT4DPG type=write 2024-04-07T16:37:50.034 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.040+0000 7f346a3868c0 1 > prepared part key=_multipart_QVOKSQAGHPA9.2~DH2GUBQ01GZ5.0 size=1048576 2024-04-07T16:37:50.038 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.044+0000 7f346a3868c0 1 > prepared part key=_multipart_QVOKSQAGHPA9.2~DH2GUBQ01GZ5.1 size=866091 2024-04-07T16:37:50.038 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.044+0000 7f346a3868c0 1 > prepared multipart upload key=QVOKSQAGHPA9 upload=DH2GUBQ01GZ5 tag=QOY5PV4YQGQC type=write size=2963243 2024-04-07T16:37:50.042 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.048+0000 7f346a3868c0 1 < completed delete operation key=E84U0SX44NQ8 tag=8MG0DZ0OP2Q7 type=del 2024-04-07T16:37:50.047 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.052+0000 7f346a3868c0 1 > prepared operation key=N14BUXLJONHZ tag=RVF3H01DHDG3 type=del size=675406 2024-04-07T16:37:50.052 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.060+0000 7f346a3868c0 1 < completed part key=_multipart_8F81G0P0YBFW.2~SGJI1BEC6LP3.0 size=456874 2024-04-07T16:37:50.059 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.068+0000 7f346a3868c0 1 < completed multipart upload key=8F81G0P0YBFW upload=SGJI1BEC6LP3 tag=F6GI7HDV96K9 type=write size=1505450 2024-04-07T16:37:50.067 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.072+0000 7f346a3868c0 1 > prepared part key=_multipart_6XDS0A7L0IY8.2~J7IFM9DA2V1Q.0 size=1048576 2024-04-07T16:37:50.070 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.076+0000 7f346a3868c0 1 > prepared part key=_multipart_6XDS0A7L0IY8.2~J7IFM9DA2V1Q.1 size=1048576 2024-04-07T16:37:50.073 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.080+0000 7f346a3868c0 1 > prepared part key=_multipart_6XDS0A7L0IY8.2~J7IFM9DA2V1Q.2 size=841839 2024-04-07T16:37:50.073 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.080+0000 7f346a3868c0 1 > prepared multipart upload key=6XDS0A7L0IY8 upload=J7IFM9DA2V1Q tag=LJYKG66TA51V type=write size=3987567 2024-04-07T16:37:50.078 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.084+0000 7f346a3868c0 1 < completed delete operation key=UIW3DN4NUYPP tag=2JQEFC8TYO31 type=del 2024-04-07T16:37:50.082 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.088+0000 7f346a3868c0 1 > prepared operation key=7G5C6HBK6QWI tag=6CVGERE1R35S type=del size=3519456 2024-04-07T16:37:50.092 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.100+0000 7f346a3868c0 1 < suggested remove operation key=C1BTVJNHIM0X tag=6JQ8TACYX68V type=del 2024-04-07T16:37:50.096 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.104+0000 7f346a3868c0 1 > prepared operation key=C1BTVJNHIM0X tag=WDQGB23IHE8Z type=del size=4007355 2024-04-07T16:37:50.101 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.108+0000 7f346a3868c0 1 < completed part key=_multipart_U3EID5EK64HD.2~HFQK3FALLFY0.0 size=1048576 2024-04-07T16:37:50.104 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.112+0000 7f346a3868c0 1 < completed part key=_multipart_U3EID5EK64HD.2~HFQK3FALLFY0.1 size=881834 2024-04-07T16:37:50.111 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.120+0000 7f346a3868c0 1 < completed multipart upload key=U3EID5EK64HD upload=HFQK3FALLFY0 tag=Z5YZYJF7DGEZ type=write size=2978986 2024-04-07T16:37:50.116 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.124+0000 7f346a3868c0 1 > prepared operation key=U3EID5EK64HD tag=ZJUG10SSW8OM type=del size=1122706 2024-04-07T16:37:50.121 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.128+0000 7f346a3868c0 1 < canceled operation key=QS4HYZFFAJNF tag=B6MP43W3D21P type=del size=4172254 2024-04-07T16:37:50.125 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.132+0000 7f346a3868c0 1 > prepared operation key=1WGUPH7X5UA0 tag=7RVOOK71VFW7 type=del size=1016385 2024-04-07T16:37:50.129 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.136+0000 7f346a3868c0 1 < completed part key=_multipart_YOK8BAEOMPUR.2~UW1CNWYSK031.0 size=1048576 2024-04-07T16:37:50.133 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.140+0000 7f346a3868c0 1 < completed part key=_multipart_YOK8BAEOMPUR.2~UW1CNWYSK031.1 size=645393 2024-04-07T16:37:50.139 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.148+0000 7f346a3868c0 1 < completed multipart upload key=YOK8BAEOMPUR upload=UW1CNWYSK031 tag=PBAEXHLPR574 type=write size=2742545 2024-04-07T16:37:50.143 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.148+0000 7f346a3868c0 1 > prepared operation key=EOYFVEI50SQ1 tag=4RALVA3GC5BC type=del size=3731316 2024-04-07T16:37:50.154 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.160+0000 7f346a3868c0 1 < completed part key=_multipart_QS4HYZFFAJNF.2~507BYP0TY6BA.0 size=863689 2024-04-07T16:37:50.161 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.168+0000 7f346a3868c0 1 < completed multipart upload key=QS4HYZFFAJNF upload=507BYP0TY6BA tag=J2E9UIZ16OB2 type=write size=1912265 2024-04-07T16:37:50.168 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.176+0000 7f346a3868c0 1 > prepared part key=_multipart_XJLQZ7IS9U1T.2~487VJ3E3NEAH.0 size=1048576 2024-04-07T16:37:50.172 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.180+0000 7f346a3868c0 1 > prepared part key=_multipart_XJLQZ7IS9U1T.2~487VJ3E3NEAH.1 size=37328 2024-04-07T16:37:50.172 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.180+0000 7f346a3868c0 1 > prepared multipart upload key=XJLQZ7IS9U1T upload=487VJ3E3NEAH tag=ZELFCK0H2XIT type=write size=2134480 2024-04-07T16:37:50.175 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.184+0000 7f346a3868c0 1 < completed delete operation key=EOYFVEI50SQ1 tag=P86STZ30HTH5 type=del 2024-04-07T16:37:50.182 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.188+0000 7f346a3868c0 1 > prepared part key=_multipart_CL4PP1RKZU0E.2~N1SE339OCDBB.0 size=1048576 2024-04-07T16:37:50.185 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.192+0000 7f346a3868c0 1 > prepared part key=_multipart_CL4PP1RKZU0E.2~N1SE339OCDBB.1 size=1048576 2024-04-07T16:37:50.189 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.196+0000 7f346a3868c0 1 > prepared part key=_multipart_CL4PP1RKZU0E.2~N1SE339OCDBB.2 size=271651 2024-04-07T16:37:50.189 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.196+0000 7f346a3868c0 1 > prepared multipart upload key=CL4PP1RKZU0E upload=N1SE339OCDBB tag=1YUJ73QPL74J type=write size=3417379 2024-04-07T16:37:50.193 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.200+0000 7f346a3868c0 1 < canceled operation key=ROFH30W3UCAX tag=NXXWFAVD8EOV type=del size=3773214 2024-04-07T16:37:50.200 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.208+0000 7f346a3868c0 1 > prepared part key=_multipart_IX3F0IUOKBX1.2~YLCFGWBVY0ZE.0 size=1048576 2024-04-07T16:37:50.203 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.212+0000 7f346a3868c0 1 > prepared part key=_multipart_IX3F0IUOKBX1.2~YLCFGWBVY0ZE.1 size=466566 2024-04-07T16:37:50.203 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.212+0000 7f346a3868c0 1 > prepared multipart upload key=IX3F0IUOKBX1 upload=YLCFGWBVY0ZE tag=HC3IOMTPF04B type=write size=2563718 2024-04-07T16:37:50.204 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.212+0000 7f346a3868c0 1 < canceled part key=_multipart_HBLEXONKXJTL.2~G2DVH6BTWTGC.0 size=1048576 2024-04-07T16:37:50.207 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.216+0000 7f346a3868c0 1 < completed part key=_multipart_HBLEXONKXJTL.2~G2DVH6BTWTGC.1 size=925607 2024-04-07T16:37:50.214 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.220+0000 7f346a3868c0 1 < canceled multipart upload key=HBLEXONKXJTL upload=G2DVH6BTWTGC tag=28VPF6C97WHI type=write size=3022759 2024-04-07T16:37:50.222 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.228+0000 7f346a3868c0 1 > prepared part key=_multipart_N14BUXLJONHZ.2~J96JNNZ1X8FJ.0 size=161921 2024-04-07T16:37:50.223 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.228+0000 7f346a3868c0 1 > prepared multipart upload key=N14BUXLJONHZ upload=J96JNNZ1X8FJ tag=V817GAJ91NWV type=write size=1210497 2024-04-07T16:37:50.227 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.236+0000 7f346a3868c0 1 < completed part key=_multipart_6XDS0A7L0IY8.2~K7TCBF0Z3KQZ.0 size=1048576 2024-04-07T16:37:50.231 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.240+0000 7f346a3868c0 1 < completed part key=_multipart_6XDS0A7L0IY8.2~K7TCBF0Z3KQZ.1 size=374104 2024-04-07T16:37:50.239 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.248+0000 7f346a3868c0 1 < completed multipart upload key=6XDS0A7L0IY8 upload=K7TCBF0Z3KQZ tag=AB8J68VC8SAM type=write size=2471256 2024-04-07T16:37:50.243 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.252+0000 7f346a3868c0 1 > prepared operation key=EQR0FY86AT5X tag=S7FIY2L03WT0 type=del size=2930682 2024-04-07T16:37:50.249 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.256+0000 7f346a3868c0 1 < completed delete operation key=95YVKJ14JD8W tag=OLR4E0O83D6A type=del 2024-04-07T16:37:50.257 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.264+0000 7f346a3868c0 1 > prepared part key=_multipart_6XDS0A7L0IY8.2~07T35BAZYF63.0 size=197619 2024-04-07T16:37:50.257 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.264+0000 7f346a3868c0 1 > prepared multipart upload key=6XDS0A7L0IY8 upload=07T35BAZYF63 tag=2IR9QYFUUC5K type=write size=1246195 2024-04-07T16:37:50.261 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.268+0000 7f346a3868c0 1 < completed part key=_multipart_ROFH30W3UCAX.2~3VW8WZ11Z2YL.0 size=1048576 2024-04-07T16:37:50.265 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.272+0000 7f346a3868c0 1 < completed part key=_multipart_ROFH30W3UCAX.2~3VW8WZ11Z2YL.1 size=498447 2024-04-07T16:37:50.273 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.280+0000 7f346a3868c0 1 < completed multipart upload key=ROFH30W3UCAX upload=3VW8WZ11Z2YL tag=XP3OP94D306M type=write size=2595599 2024-04-07T16:37:50.281 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.288+0000 7f346a3868c0 1 > prepared part key=_multipart_NYET3212USVE.2~6SHVXLQSQ4GS.0 size=711684 2024-04-07T16:37:50.281 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.288+0000 7f346a3868c0 1 > prepared multipart upload key=NYET3212USVE upload=6SHVXLQSQ4GS tag=258TD3HEPACM type=write size=1760260 2024-04-07T16:37:50.285 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.292+0000 7f346a3868c0 1 < completed part key=_multipart_IX3F0IUOKBX1.2~CA6230X9TRK3.0 size=1048576 2024-04-07T16:37:50.285 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.292+0000 7f346a3868c0 1 < canceled part key=_multipart_IX3F0IUOKBX1.2~CA6230X9TRK3.1 size=79563 2024-04-07T16:37:50.293 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.300+0000 7f346a3868c0 1 < canceled multipart upload key=IX3F0IUOKBX1 upload=CA6230X9TRK3 tag=SGBT5OHS0KN0 type=write size=2176715 2024-04-07T16:37:50.298 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.304+0000 7f346a3868c0 1 > prepared operation key=95YVKJ14JD8W tag=61XNHB3PM7CT type=write size=237476 2024-04-07T16:37:50.302 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.308+0000 7f346a3868c0 1 < completed part key=_multipart_QVOKSQAGHPA9.2~DH2GUBQ01GZ5.0 size=1048576 2024-04-07T16:37:50.306 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.312+0000 7f346a3868c0 1 < completed part key=_multipart_QVOKSQAGHPA9.2~DH2GUBQ01GZ5.1 size=866091 2024-04-07T16:37:50.314 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.320+0000 7f346a3868c0 1 < completed multipart upload key=QVOKSQAGHPA9 upload=DH2GUBQ01GZ5 tag=QOY5PV4YQGQC type=write size=2963243 2024-04-07T16:37:50.319 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.324+0000 7f346a3868c0 1 > prepared operation key=EOYFVEI50SQ1 tag=XF2GBJWOOKFU type=write size=959846 2024-04-07T16:37:50.323 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.332+0000 7f346a3868c0 1 < completed delete operation key=N14BUXLJONHZ tag=RVF3H01DHDG3 type=del 2024-04-07T16:37:50.332 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.340+0000 7f346a3868c0 1 > prepared part key=_multipart_YXMDZB23YTB3.2~ECN0IGKM7S25.0 size=1048576 2024-04-07T16:37:50.335 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.344+0000 7f346a3868c0 1 > prepared part key=_multipart_YXMDZB23YTB3.2~ECN0IGKM7S25.1 size=978649 2024-04-07T16:37:50.336 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.344+0000 7f346a3868c0 1 > prepared multipart upload key=YXMDZB23YTB3 upload=ECN0IGKM7S25 tag=AX2RPE0R3400 type=write size=3075801 2024-04-07T16:37:50.340 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.348+0000 7f346a3868c0 1 < completed part key=_multipart_6XDS0A7L0IY8.2~J7IFM9DA2V1Q.0 size=1048576 2024-04-07T16:37:50.344 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.352+0000 7f346a3868c0 1 < completed part key=_multipart_6XDS0A7L0IY8.2~J7IFM9DA2V1Q.1 size=1048576 2024-04-07T16:37:50.348 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.356+0000 7f346a3868c0 1 < completed part key=_multipart_6XDS0A7L0IY8.2~J7IFM9DA2V1Q.2 size=841839 2024-04-07T16:37:50.355 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.364+0000 7f346a3868c0 1 < completed multipart upload key=6XDS0A7L0IY8 upload=J7IFM9DA2V1Q tag=LJYKG66TA51V type=write size=3987567 2024-04-07T16:37:50.360 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.368+0000 7f346a3868c0 1 > prepared operation key=CL4PP1RKZU0E tag=XL2IKHNMYUGW type=write size=411980 2024-04-07T16:37:50.364 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.372+0000 7f346a3868c0 1 < completed delete operation key=7G5C6HBK6QWI tag=6CVGERE1R35S type=del 2024-04-07T16:37:50.373 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.380+0000 7f346a3868c0 1 > prepared part key=_multipart_QTCIIPTXIRW2.2~UDXHYL9IT0L5.0 size=831360 2024-04-07T16:37:50.373 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.380+0000 7f346a3868c0 1 > prepared multipart upload key=QTCIIPTXIRW2 upload=UDXHYL9IT0L5 tag=Z5FNHIY38APC type=write size=1879936 2024-04-07T16:37:50.378 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.384+0000 7f346a3868c0 1 < completed delete operation key=C1BTVJNHIM0X tag=WDQGB23IHE8Z type=del 2024-04-07T16:37:50.383 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.388+0000 7f346a3868c0 1 > prepared operation key=GBQSME7SY1AH tag=7NXCS2LCODYB type=write size=395559 2024-04-07T16:37:50.387 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.396+0000 7f346a3868c0 1 < completed delete operation key=U3EID5EK64HD tag=ZJUG10SSW8OM type=del 2024-04-07T16:37:50.395 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.404+0000 7f346a3868c0 1 > prepared part key=_multipart_1VKVQXSQMTU1.2~RSF5TDGRNU56.0 size=1048576 2024-04-07T16:37:50.399 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.408+0000 7f346a3868c0 1 > prepared part key=_multipart_1VKVQXSQMTU1.2~RSF5TDGRNU56.1 size=1048576 2024-04-07T16:37:50.402 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.408+0000 7f346a3868c0 1 > prepared part key=_multipart_1VKVQXSQMTU1.2~RSF5TDGRNU56.2 size=181009 2024-04-07T16:37:50.403 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.408+0000 7f346a3868c0 1 > prepared multipart upload key=1VKVQXSQMTU1 upload=RSF5TDGRNU56 tag=NJQNWZ6RBGE3 type=write size=3326737 2024-04-07T16:37:50.407 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.416+0000 7f346a3868c0 1 < completed delete operation key=1WGUPH7X5UA0 tag=7RVOOK71VFW7 type=del 2024-04-07T16:37:50.416 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.424+0000 7f346a3868c0 1 > prepared part key=_multipart_U3EID5EK64HD.2~9ASYV15U50SG.0 size=380398 2024-04-07T16:37:50.416 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.424+0000 7f346a3868c0 1 > prepared multipart upload key=U3EID5EK64HD upload=9ASYV15U50SG tag=SN196UUDP3K1 type=write size=1428974 2024-04-07T16:37:50.421 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.428+0000 7f346a3868c0 1 < completed delete operation key=EOYFVEI50SQ1 tag=4RALVA3GC5BC type=del 2024-04-07T16:37:50.426 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.432+0000 7f346a3868c0 1 > prepared operation key=3N0QJRYVO3FV tag=11BFGFRH1QP2 type=del size=241042 2024-04-07T16:37:50.431 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.436+0000 7f346a3868c0 1 < completed part key=_multipart_XJLQZ7IS9U1T.2~487VJ3E3NEAH.0 size=1048576 2024-04-07T16:37:50.435 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.444+0000 7f346a3868c0 1 < completed part key=_multipart_XJLQZ7IS9U1T.2~487VJ3E3NEAH.1 size=37328 2024-04-07T16:37:50.444 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.452+0000 7f346a3868c0 1 < completed multipart upload key=XJLQZ7IS9U1T upload=487VJ3E3NEAH tag=ZELFCK0H2XIT type=write size=2134480 2024-04-07T16:37:50.448 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.456+0000 7f346a3868c0 1 > prepared operation key=YOK8BAEOMPUR tag=RAY7MGG4HFNO type=write size=174645 2024-04-07T16:37:50.453 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.460+0000 7f346a3868c0 1 < completed part key=_multipart_CL4PP1RKZU0E.2~N1SE339OCDBB.0 size=1048576 2024-04-07T16:37:50.456 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.464+0000 7f346a3868c0 1 < completed part key=_multipart_CL4PP1RKZU0E.2~N1SE339OCDBB.1 size=1048576 2024-04-07T16:37:50.460 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.468+0000 7f346a3868c0 1 < completed part key=_multipart_CL4PP1RKZU0E.2~N1SE339OCDBB.2 size=271651 2024-04-07T16:37:50.468 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.476+0000 7f346a3868c0 1 < completed multipart upload key=CL4PP1RKZU0E upload=N1SE339OCDBB tag=1YUJ73QPL74J type=write size=3417379 2024-04-07T16:37:50.473 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.480+0000 7f346a3868c0 1 > prepared operation key=8F81G0P0YBFW tag=744XGH1B5OKU type=del size=1636017 2024-04-07T16:37:50.478 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.484+0000 7f346a3868c0 1 < completed part key=_multipart_IX3F0IUOKBX1.2~YLCFGWBVY0ZE.0 size=1048576 2024-04-07T16:37:50.482 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.488+0000 7f346a3868c0 1 < completed part key=_multipart_IX3F0IUOKBX1.2~YLCFGWBVY0ZE.1 size=466566 2024-04-07T16:37:50.489 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.496+0000 7f346a3868c0 1 < completed multipart upload key=IX3F0IUOKBX1 upload=YLCFGWBVY0ZE tag=HC3IOMTPF04B type=write size=2563718 2024-04-07T16:37:50.500 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.508+0000 7f346a3868c0 1 > prepared part key=_multipart_IX3F0IUOKBX1.2~AQOUMZPPY275.0 size=1048576 2024-04-07T16:37:50.504 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.512+0000 7f346a3868c0 1 > prepared part key=_multipart_IX3F0IUOKBX1.2~AQOUMZPPY275.1 size=1048576 2024-04-07T16:37:50.508 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.516+0000 7f346a3868c0 1 > prepared part key=_multipart_IX3F0IUOKBX1.2~AQOUMZPPY275.2 size=101127 2024-04-07T16:37:50.508 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.516+0000 7f346a3868c0 1 > prepared multipart upload key=IX3F0IUOKBX1 upload=AQOUMZPPY275 tag=2B28U4EKPP6M type=write size=3246855 2024-04-07T16:37:50.509 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.516+0000 7f346a3868c0 1 < canceled part key=_multipart_N14BUXLJONHZ.2~J96JNNZ1X8FJ.0 size=161921 2024-04-07T16:37:50.518 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.524+0000 7f346a3868c0 1 < canceled multipart upload key=N14BUXLJONHZ upload=J96JNNZ1X8FJ tag=V817GAJ91NWV type=write size=1210497 2024-04-07T16:37:50.526 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.532+0000 7f346a3868c0 1 > prepared part key=_multipart_QS4HYZFFAJNF.2~QPE9TELZIH1V.0 size=1048576 2024-04-07T16:37:50.530 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.536+0000 7f346a3868c0 1 > prepared part key=_multipart_QS4HYZFFAJNF.2~QPE9TELZIH1V.1 size=1048576 2024-04-07T16:37:50.535 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.544+0000 7f346a3868c0 1 > prepared part key=_multipart_QS4HYZFFAJNF.2~QPE9TELZIH1V.2 size=907119 2024-04-07T16:37:50.535 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.544+0000 7f346a3868c0 1 > prepared multipart upload key=QS4HYZFFAJNF upload=QPE9TELZIH1V tag=KZ8M5T376X1I type=write size=4052847 2024-04-07T16:37:50.541 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.548+0000 7f346a3868c0 1 < completed delete operation key=EQR0FY86AT5X tag=S7FIY2L03WT0 type=del 2024-04-07T16:37:50.546 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.552+0000 7f346a3868c0 1 > prepared operation key=YXMDZB23YTB3 tag=0SM2DVOK722L type=del size=3208931 2024-04-07T16:37:50.551 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.560+0000 7f346a3868c0 1 < completed part key=_multipart_6XDS0A7L0IY8.2~07T35BAZYF63.0 size=197619 2024-04-07T16:37:50.560 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.568+0000 7f346a3868c0 1 < completed multipart upload key=6XDS0A7L0IY8 upload=07T35BAZYF63 tag=2IR9QYFUUC5K type=write size=1246195 2024-04-07T16:37:50.565 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.572+0000 7f346a3868c0 1 > prepared operation key=C1BTVJNHIM0X tag=9XW6OPAJTSSB type=del size=3269251 2024-04-07T16:37:50.571 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.576+0000 7f346a3868c0 1 < completed part key=_multipart_NYET3212USVE.2~6SHVXLQSQ4GS.0 size=711684 2024-04-07T16:37:50.579 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.584+0000 7f346a3868c0 1 < completed multipart upload key=NYET3212USVE upload=6SHVXLQSQ4GS tag=258TD3HEPACM type=write size=1760260 2024-04-07T16:37:50.587 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.596+0000 7f346a3868c0 1 > prepared part key=_multipart_GBQSME7SY1AH.2~CSS9QME04YU6.0 size=844810 2024-04-07T16:37:50.587 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.596+0000 7f346a3868c0 1 > prepared multipart upload key=GBQSME7SY1AH upload=CSS9QME04YU6 tag=M4D30VOXYVNK type=write size=1893386 2024-04-07T16:37:50.595 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.604+0000 7f346a3868c0 1 < suggested update operation key=95YVKJ14JD8W tag=61XNHB3PM7CT type=write size=237476 2024-04-07T16:37:50.600 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.608+0000 7f346a3868c0 1 > prepared operation key=1WGUPH7X5UA0 tag=R28XDHIZD8IB type=del size=3820155 2024-04-07T16:37:50.604 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.612+0000 7f346a3868c0 1 < completed write operation key=EOYFVEI50SQ1 tag=XF2GBJWOOKFU type=write size=959846 2024-04-07T16:37:50.608 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.616+0000 7f346a3868c0 1 > prepared operation key=LXOZ0T2KY6OR tag=ELA18SL3O1PU type=del size=1599056 2024-04-07T16:37:50.608 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.616+0000 7f346a3868c0 1 < canceled part key=_multipart_YXMDZB23YTB3.2~ECN0IGKM7S25.0 size=1048576 2024-04-07T16:37:50.608 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.616+0000 7f346a3868c0 1 < canceled part key=_multipart_YXMDZB23YTB3.2~ECN0IGKM7S25.1 size=978649 2024-04-07T16:37:50.616 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.624+0000 7f346a3868c0 1 < canceled multipart upload key=YXMDZB23YTB3 upload=ECN0IGKM7S25 tag=AX2RPE0R3400 type=write size=3075801 2024-04-07T16:37:50.621 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.628+0000 7f346a3868c0 1 > prepared operation key=DXTO3AJ429AC tag=IG4852Q6PI4K type=del size=4185525 2024-04-07T16:37:50.625 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.632+0000 7f346a3868c0 1 < canceled operation key=CL4PP1RKZU0E tag=XL2IKHNMYUGW type=write size=411980 2024-04-07T16:37:50.629 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.636+0000 7f346a3868c0 1 > prepared operation key=IX3F0IUOKBX1 tag=B32IV52KU36H type=del size=484990 2024-04-07T16:37:50.633 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.640+0000 7f346a3868c0 1 < completed part key=_multipart_QTCIIPTXIRW2.2~UDXHYL9IT0L5.0 size=831360 2024-04-07T16:37:50.640 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.648+0000 7f346a3868c0 1 < completed multipart upload key=QTCIIPTXIRW2 upload=UDXHYL9IT0L5 tag=Z5FNHIY38APC type=write size=1879936 2024-04-07T16:37:50.645 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.652+0000 7f346a3868c0 1 > prepared operation key=UIW3DN4NUYPP tag=2P368S88168G type=write size=150690 2024-04-07T16:37:50.649 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.656+0000 7f346a3868c0 1 < completed write operation key=GBQSME7SY1AH tag=7NXCS2LCODYB type=write size=395559 2024-04-07T16:37:50.654 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.660+0000 7f346a3868c0 1 > prepared operation key=NYET3212USVE tag=YR220P7WCJU2 type=write size=889558 2024-04-07T16:37:50.658 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.664+0000 7f346a3868c0 1 < completed part key=_multipart_1VKVQXSQMTU1.2~RSF5TDGRNU56.0 size=1048576 2024-04-07T16:37:50.662 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.668+0000 7f346a3868c0 1 < completed part key=_multipart_1VKVQXSQMTU1.2~RSF5TDGRNU56.1 size=1048576 2024-04-07T16:37:50.665 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.672+0000 7f346a3868c0 1 < completed part key=_multipart_1VKVQXSQMTU1.2~RSF5TDGRNU56.2 size=181009 2024-04-07T16:37:50.673 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.680+0000 7f346a3868c0 1 < completed multipart upload key=1VKVQXSQMTU1 upload=RSF5TDGRNU56 tag=NJQNWZ6RBGE3 type=write size=3326737 2024-04-07T16:37:50.678 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.684+0000 7f346a3868c0 1 > prepared operation key=8F81G0P0YBFW tag=KAO58ODQ39RK type=del size=3882527 2024-04-07T16:37:50.682 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.688+0000 7f346a3868c0 1 < completed part key=_multipart_U3EID5EK64HD.2~9ASYV15U50SG.0 size=380398 2024-04-07T16:37:50.690 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.696+0000 7f346a3868c0 1 < completed multipart upload key=U3EID5EK64HD upload=9ASYV15U50SG tag=SN196UUDP3K1 type=write size=1428974 2024-04-07T16:37:50.698 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.704+0000 7f346a3868c0 1 > prepared part key=_multipart_8F81G0P0YBFW.2~V43BTUNA06PN.0 size=1048576 2024-04-07T16:37:50.701 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.708+0000 7f346a3868c0 1 > prepared part key=_multipart_8F81G0P0YBFW.2~V43BTUNA06PN.1 size=661967 2024-04-07T16:37:50.702 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.708+0000 7f346a3868c0 1 > prepared multipart upload key=8F81G0P0YBFW upload=V43BTUNA06PN tag=692AA7X4NOU7 type=write size=2759119 2024-04-07T16:37:50.706 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.712+0000 7f346a3868c0 1 < completed delete operation key=3N0QJRYVO3FV tag=11BFGFRH1QP2 type=del 2024-04-07T16:37:50.711 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.716+0000 7f346a3868c0 1 > prepared operation key=6XDS0A7L0IY8 tag=U0VPLF8TC9VC type=del size=3582206 2024-04-07T16:37:50.720 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.728+0000 7f346a3868c0 1 < suggested remove operation key=YOK8BAEOMPUR tag=RAY7MGG4HFNO type=write 2024-04-07T16:37:50.729 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.736+0000 7f346a3868c0 1 > prepared part key=_multipart_NYET3212USVE.2~B3UP0AHMT21A.0 size=1048576 2024-04-07T16:37:50.733 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.740+0000 7f346a3868c0 1 > prepared part key=_multipart_NYET3212USVE.2~B3UP0AHMT21A.1 size=172757 2024-04-07T16:37:50.733 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.740+0000 7f346a3868c0 1 > prepared multipart upload key=NYET3212USVE upload=B3UP0AHMT21A tag=DX4FFNH5U4QB type=write size=2269909 2024-04-07T16:37:50.738 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.744+0000 7f346a3868c0 1 < completed delete operation key=8F81G0P0YBFW tag=744XGH1B5OKU type=del 2024-04-07T16:37:50.742 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.748+0000 7f346a3868c0 1 > prepared operation key=QS4HYZFFAJNF tag=GN8PTP0Z63XD type=del size=579297 2024-04-07T16:37:50.747 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.756+0000 7f346a3868c0 1 < completed part key=_multipart_IX3F0IUOKBX1.2~AQOUMZPPY275.0 size=1048576 2024-04-07T16:37:50.751 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.760+0000 7f346a3868c0 1 < completed part key=_multipart_IX3F0IUOKBX1.2~AQOUMZPPY275.1 size=1048576 2024-04-07T16:37:50.755 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.764+0000 7f346a3868c0 1 < completed part key=_multipart_IX3F0IUOKBX1.2~AQOUMZPPY275.2 size=101127 2024-04-07T16:37:50.764 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.772+0000 7f346a3868c0 1 < completed multipart upload key=IX3F0IUOKBX1 upload=AQOUMZPPY275 tag=2B28U4EKPP6M type=write size=3246855 2024-04-07T16:37:50.768 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.776+0000 7f346a3868c0 1 > prepared operation key=QTCIIPTXIRW2 tag=3O8TRSOK3YI1 type=write size=597037 2024-04-07T16:37:50.773 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.780+0000 7f346a3868c0 1 < completed part key=_multipart_QS4HYZFFAJNF.2~QPE9TELZIH1V.0 size=1048576 2024-04-07T16:37:50.777 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.784+0000 7f346a3868c0 1 < completed part key=_multipart_QS4HYZFFAJNF.2~QPE9TELZIH1V.1 size=1048576 2024-04-07T16:37:50.781 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.788+0000 7f346a3868c0 1 < completed part key=_multipart_QS4HYZFFAJNF.2~QPE9TELZIH1V.2 size=907119 2024-04-07T16:37:50.789 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.796+0000 7f346a3868c0 1 < completed multipart upload key=QS4HYZFFAJNF upload=QPE9TELZIH1V tag=KZ8M5T376X1I type=write size=4052847 2024-04-07T16:37:50.798 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.804+0000 7f346a3868c0 1 > prepared part key=_multipart_6XDS0A7L0IY8.2~LQTM0O0AKCM1.0 size=1048576 2024-04-07T16:37:50.802 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.808+0000 7f346a3868c0 1 > prepared part key=_multipart_6XDS0A7L0IY8.2~LQTM0O0AKCM1.1 size=363327 2024-04-07T16:37:50.802 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.808+0000 7f346a3868c0 1 > prepared multipart upload key=6XDS0A7L0IY8 upload=LQTM0O0AKCM1 tag=FCO13CC143S3 type=write size=2460479 2024-04-07T16:37:50.811 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.816+0000 7f346a3868c0 1 < suggested update operation key=YXMDZB23YTB3 tag=0SM2DVOK722L type=del size=3208931 2024-04-07T16:37:50.816 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.824+0000 7f346a3868c0 1 > prepared operation key=LXOZ0T2KY6OR tag=JWEPZOE5MS96 type=write size=302270 2024-04-07T16:37:50.820 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.828+0000 7f346a3868c0 1 < completed delete operation key=C1BTVJNHIM0X tag=9XW6OPAJTSSB type=del 2024-04-07T16:37:50.830 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.836+0000 7f346a3868c0 1 > prepared part key=_multipart_GBQSME7SY1AH.2~NXFEMU9JMLHN.0 size=1048576 2024-04-07T16:37:50.834 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.840+0000 7f346a3868c0 1 > prepared part key=_multipart_GBQSME7SY1AH.2~NXFEMU9JMLHN.1 size=1048576 2024-04-07T16:37:50.838 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.844+0000 7f346a3868c0 1 > prepared part key=_multipart_GBQSME7SY1AH.2~NXFEMU9JMLHN.2 size=201973 2024-04-07T16:37:50.838 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.844+0000 7f346a3868c0 1 > prepared multipart upload key=GBQSME7SY1AH upload=NXFEMU9JMLHN tag=PZLQ75H6F086 type=write size=3347701 2024-04-07T16:37:50.843 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.848+0000 7f346a3868c0 1 < completed part key=_multipart_GBQSME7SY1AH.2~CSS9QME04YU6.0 size=844810 2024-04-07T16:37:50.851 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.856+0000 7f346a3868c0 1 < completed multipart upload key=GBQSME7SY1AH upload=CSS9QME04YU6 tag=M4D30VOXYVNK type=write size=1893386 2024-04-07T16:37:50.856 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:37:50 smithi144 bash[66654]: cluster 2024-04-07T16:37:49.058249+0000 mgr.y (mgr.25175) 674 : cluster [DBG] pgmap v1309: 137 pgs: 137 active+clean; 583 KiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 458 KiB/s rd, 159 KiB/s wr, 808 op/s 2024-04-07T16:37:50.859 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.868+0000 7f346a3868c0 1 > prepared part key=_multipart_95YVKJ14JD8W.2~0MKN1FS3HGCS.0 size=1048576 2024-04-07T16:37:50.863 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.868+0000 7f346a3868c0 1 > prepared part key=_multipart_95YVKJ14JD8W.2~0MKN1FS3HGCS.1 size=1048576 2024-04-07T16:37:50.866 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.872+0000 7f346a3868c0 1 > prepared part key=_multipart_95YVKJ14JD8W.2~0MKN1FS3HGCS.2 size=235465 2024-04-07T16:37:50.866 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.872+0000 7f346a3868c0 1 > prepared multipart upload key=95YVKJ14JD8W upload=0MKN1FS3HGCS tag=2046GHAURYU9 type=write size=3381193 2024-04-07T16:37:50.871 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.880+0000 7f346a3868c0 1 < completed delete operation key=1WGUPH7X5UA0 tag=R28XDHIZD8IB type=del 2024-04-07T16:37:50.879 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.888+0000 7f346a3868c0 1 > prepared part key=_multipart_8F81G0P0YBFW.2~61NB0SAOWP61.0 size=1048576 2024-04-07T16:37:50.884 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.892+0000 7f346a3868c0 1 > prepared part key=_multipart_8F81G0P0YBFW.2~61NB0SAOWP61.1 size=1048576 2024-04-07T16:37:50.888 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.896+0000 7f346a3868c0 1 > prepared part key=_multipart_8F81G0P0YBFW.2~61NB0SAOWP61.2 size=444533 2024-04-07T16:37:50.888 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.896+0000 7f346a3868c0 1 > prepared multipart upload key=8F81G0P0YBFW upload=61NB0SAOWP61 tag=AGJOZQ91DLY1 type=write size=3590261 2024-04-07T16:37:50.892 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.900+0000 7f346a3868c0 1 < completed delete operation key=LXOZ0T2KY6OR tag=ELA18SL3O1PU type=del 2024-04-07T16:37:50.897 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.904+0000 7f346a3868c0 1 > prepared operation key=N14BUXLJONHZ tag=50YGT01KTG7G type=del size=3899826 2024-04-07T16:37:50.901 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.908+0000 7f346a3868c0 1 < completed delete operation key=DXTO3AJ429AC tag=IG4852Q6PI4K type=del 2024-04-07T16:37:50.906 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.912+0000 7f346a3868c0 1 > prepared operation key=N14BUXLJONHZ tag=CYN9ZU7O5F02 type=del size=1146679 2024-04-07T16:37:50.911 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.920+0000 7f346a3868c0 1 < completed delete operation key=IX3F0IUOKBX1 tag=B32IV52KU36H type=del 2024-04-07T16:37:50.920 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.928+0000 7f346a3868c0 1 > prepared part key=_multipart_UIW3DN4NUYPP.2~E60VWYZ71LX1.0 size=1048576 2024-04-07T16:37:50.924 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.932+0000 7f346a3868c0 1 > prepared part key=_multipart_UIW3DN4NUYPP.2~E60VWYZ71LX1.1 size=1048576 2024-04-07T16:37:50.927 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.936+0000 7f346a3868c0 1 > prepared part key=_multipart_UIW3DN4NUYPP.2~E60VWYZ71LX1.2 size=38133 2024-04-07T16:37:50.927 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.936+0000 7f346a3868c0 1 > prepared multipart upload key=UIW3DN4NUYPP upload=E60VWYZ71LX1 tag=MYOJVHJB2444 type=write size=3183861 2024-04-07T16:37:50.937 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.944+0000 7f346a3868c0 1 < suggested update operation key=UIW3DN4NUYPP tag=2P368S88168G type=write size=150690 2024-04-07T16:37:50.941 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:37:50 smithi100 bash[74942]: cluster 2024-04-07T16:37:49.058249+0000 mgr.y (mgr.25175) 674 : cluster [DBG] pgmap v1309: 137 pgs: 137 active+clean; 583 KiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 458 KiB/s rd, 159 KiB/s wr, 808 op/s 2024-04-07T16:37:50.941 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:37:50 smithi100 bash[76202]: cluster 2024-04-07T16:37:49.058249+0000 mgr.y (mgr.25175) 674 : cluster [DBG] pgmap v1309: 137 pgs: 137 active+clean; 583 KiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 458 KiB/s rd, 159 KiB/s wr, 808 op/s 2024-04-07T16:37:50.942 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.948+0000 7f346a3868c0 1 > prepared operation key=GBQSME7SY1AH tag=K0C95KK6SOP2 type=del size=663496 2024-04-07T16:37:50.947 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.952+0000 7f346a3868c0 1 < completed write operation key=NYET3212USVE tag=YR220P7WCJU2 type=write size=889558 2024-04-07T16:37:50.955 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.964+0000 7f346a3868c0 1 > prepared part key=_multipart_YXMDZB23YTB3.2~DI8E8CCC9LYV.0 size=1048576 2024-04-07T16:37:50.960 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.968+0000 7f346a3868c0 1 > prepared part key=_multipart_YXMDZB23YTB3.2~DI8E8CCC9LYV.1 size=1048576 2024-04-07T16:37:50.964 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.972+0000 7f346a3868c0 1 > prepared part key=_multipart_YXMDZB23YTB3.2~DI8E8CCC9LYV.2 size=63194 2024-04-07T16:37:50.964 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.972+0000 7f346a3868c0 1 > prepared multipart upload key=YXMDZB23YTB3 upload=DI8E8CCC9LYV tag=DR3WDU1L23GJ type=write size=3208922 2024-04-07T16:37:50.969 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.976+0000 7f346a3868c0 1 < completed delete operation key=8F81G0P0YBFW tag=KAO58ODQ39RK type=del 2024-04-07T16:37:50.974 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.980+0000 7f346a3868c0 1 > prepared operation key=3N0QJRYVO3FV tag=SF5O5WIRGPWY type=del size=831559 2024-04-07T16:37:50.978 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.984+0000 7f346a3868c0 1 < completed part key=_multipart_8F81G0P0YBFW.2~V43BTUNA06PN.0 size=1048576 2024-04-07T16:37:50.983 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.988+0000 7f346a3868c0 1 < completed part key=_multipart_8F81G0P0YBFW.2~V43BTUNA06PN.1 size=661967 2024-04-07T16:37:50.990 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:50.996+0000 7f346a3868c0 1 < completed multipart upload key=8F81G0P0YBFW upload=V43BTUNA06PN tag=692AA7X4NOU7 type=write size=2759119 2024-04-07T16:37:50.995 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.004+0000 7f346a3868c0 1 > prepared operation key=EOYFVEI50SQ1 tag=EHYPTLV2FVHI type=write size=671854 2024-04-07T16:37:51.000 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.008+0000 7f346a3868c0 1 < completed delete operation key=6XDS0A7L0IY8 tag=U0VPLF8TC9VC type=del 2024-04-07T16:37:51.005 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.012+0000 7f346a3868c0 1 > prepared operation key=YXMDZB23YTB3 tag=XMMOEWGKVLKR type=write size=932044 2024-04-07T16:37:51.009 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.016+0000 7f346a3868c0 1 < completed part key=_multipart_NYET3212USVE.2~B3UP0AHMT21A.0 size=1048576 2024-04-07T16:37:51.013 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.020+0000 7f346a3868c0 1 < completed part key=_multipart_NYET3212USVE.2~B3UP0AHMT21A.1 size=172757 2024-04-07T16:37:51.021 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.028+0000 7f346a3868c0 1 < completed multipart upload key=NYET3212USVE upload=B3UP0AHMT21A tag=DX4FFNH5U4QB type=write size=2269909 2024-04-07T16:37:51.030 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.036+0000 7f346a3868c0 1 > prepared part key=_multipart_HBLEXONKXJTL.2~E8YMJL5DBBCI.0 size=1048576 2024-04-07T16:37:51.034 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.040+0000 7f346a3868c0 1 > prepared part key=_multipart_HBLEXONKXJTL.2~E8YMJL5DBBCI.1 size=838203 2024-04-07T16:37:51.034 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.040+0000 7f346a3868c0 1 > prepared multipart upload key=HBLEXONKXJTL upload=E8YMJL5DBBCI tag=SQV126TPVMLZ type=write size=2935355 2024-04-07T16:37:51.044 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.052+0000 7f346a3868c0 1 < suggested remove operation key=QS4HYZFFAJNF tag=GN8PTP0Z63XD type=del 2024-04-07T16:37:51.053 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.060+0000 7f346a3868c0 1 > prepared part key=_multipart_EQR0FY86AT5X.2~30J29GK9SX3X.0 size=1048576 2024-04-07T16:37:51.056 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.064+0000 7f346a3868c0 1 > prepared part key=_multipart_EQR0FY86AT5X.2~30J29GK9SX3X.1 size=903989 2024-04-07T16:37:51.057 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.064+0000 7f346a3868c0 1 > prepared multipart upload key=EQR0FY86AT5X upload=30J29GK9SX3X tag=RR46CJGQYFKQ type=write size=3001141 2024-04-07T16:37:51.061 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.068+0000 7f346a3868c0 1 < completed write operation key=QTCIIPTXIRW2 tag=3O8TRSOK3YI1 type=write size=597037 2024-04-07T16:37:51.066 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.072+0000 7f346a3868c0 1 > prepared operation key=1VBSUWNI3V1T tag=VODH5D5DM304 type=del size=1779157 2024-04-07T16:37:51.071 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.080+0000 7f346a3868c0 1 < completed part key=_multipart_6XDS0A7L0IY8.2~LQTM0O0AKCM1.0 size=1048576 2024-04-07T16:37:51.075 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.084+0000 7f346a3868c0 1 < completed part key=_multipart_6XDS0A7L0IY8.2~LQTM0O0AKCM1.1 size=363327 2024-04-07T16:37:51.083 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.092+0000 7f346a3868c0 1 < completed multipart upload key=6XDS0A7L0IY8 upload=LQTM0O0AKCM1 tag=FCO13CC143S3 type=write size=2460479 2024-04-07T16:37:51.092 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.100+0000 7f346a3868c0 1 > prepared part key=_multipart_7G5C6HBK6QWI.2~IP7KO4OIDJ0G.0 size=1048576 2024-04-07T16:37:51.095 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.104+0000 7f346a3868c0 1 > prepared part key=_multipart_7G5C6HBK6QWI.2~IP7KO4OIDJ0G.1 size=1048576 2024-04-07T16:37:51.099 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.108+0000 7f346a3868c0 1 > prepared part key=_multipart_7G5C6HBK6QWI.2~IP7KO4OIDJ0G.2 size=770299 2024-04-07T16:37:51.099 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.108+0000 7f346a3868c0 1 > prepared multipart upload key=7G5C6HBK6QWI upload=IP7KO4OIDJ0G tag=2KKGC3QJN1IO type=write size=3916027 2024-04-07T16:37:51.110 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.116+0000 7f346a3868c0 1 < suggested remove operation key=LXOZ0T2KY6OR tag=JWEPZOE5MS96 type=write 2024-04-07T16:37:51.115 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.124+0000 7f346a3868c0 1 > prepared operation key=XJLQZ7IS9U1T tag=3SDEIZWFVYGE type=del size=504345 2024-04-07T16:37:51.116 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.124+0000 7f346a3868c0 1 < canceled part key=_multipart_GBQSME7SY1AH.2~NXFEMU9JMLHN.0 size=1048576 2024-04-07T16:37:51.120 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.128+0000 7f346a3868c0 1 < completed part key=_multipart_GBQSME7SY1AH.2~NXFEMU9JMLHN.1 size=1048576 2024-04-07T16:37:51.124 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.132+0000 7f346a3868c0 1 < completed part key=_multipart_GBQSME7SY1AH.2~NXFEMU9JMLHN.2 size=201973 2024-04-07T16:37:51.132 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.140+0000 7f346a3868c0 1 < canceled multipart upload key=GBQSME7SY1AH upload=NXFEMU9JMLHN tag=PZLQ75H6F086 type=write size=3347701 2024-04-07T16:37:51.137 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.144+0000 7f346a3868c0 1 > prepared operation key=1VBSUWNI3V1T tag=JVQH65GPZ5M6 type=write size=823717 2024-04-07T16:37:51.142 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.148+0000 7f346a3868c0 1 < completed part key=_multipart_95YVKJ14JD8W.2~0MKN1FS3HGCS.0 size=1048576 2024-04-07T16:37:51.146 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.152+0000 7f346a3868c0 1 < completed part key=_multipart_95YVKJ14JD8W.2~0MKN1FS3HGCS.1 size=1048576 2024-04-07T16:37:51.150 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.156+0000 7f346a3868c0 1 < completed part key=_multipart_95YVKJ14JD8W.2~0MKN1FS3HGCS.2 size=235465 2024-04-07T16:37:51.158 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.164+0000 7f346a3868c0 1 < completed multipart upload key=95YVKJ14JD8W upload=0MKN1FS3HGCS tag=2046GHAURYU9 type=write size=3381193 2024-04-07T16:37:51.163 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.172+0000 7f346a3868c0 1 > prepared operation key=IP8YW2A75TCW tag=CM47RENIWFCP type=del size=3169266 2024-04-07T16:37:51.168 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.176+0000 7f346a3868c0 1 < completed part key=_multipart_8F81G0P0YBFW.2~61NB0SAOWP61.0 size=1048576 2024-04-07T16:37:51.172 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.180+0000 7f346a3868c0 1 < completed part key=_multipart_8F81G0P0YBFW.2~61NB0SAOWP61.1 size=1048576 2024-04-07T16:37:51.176 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.184+0000 7f346a3868c0 1 < completed part key=_multipart_8F81G0P0YBFW.2~61NB0SAOWP61.2 size=444533 2024-04-07T16:37:51.184 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.192+0000 7f346a3868c0 1 < completed multipart upload key=8F81G0P0YBFW upload=61NB0SAOWP61 tag=AGJOZQ91DLY1 type=write size=3590261 2024-04-07T16:37:51.189 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.196+0000 7f346a3868c0 1 > prepared operation key=EOYFVEI50SQ1 tag=82WEF0K3CLYG type=del size=2924269 2024-04-07T16:37:51.194 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.200+0000 7f346a3868c0 1 < completed delete operation key=N14BUXLJONHZ tag=50YGT01KTG7G type=del 2024-04-07T16:37:51.199 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.204+0000 7f346a3868c0 1 > prepared operation key=NYET3212USVE tag=URE8KVVB4FZD type=del size=831156 2024-04-07T16:37:51.204 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.212+0000 7f346a3868c0 1 < canceled operation key=N14BUXLJONHZ tag=CYN9ZU7O5F02 type=del size=1146679 2024-04-07T16:37:51.208 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.216+0000 7f346a3868c0 1 > prepared operation key=QTCIIPTXIRW2 tag=EGBK20ITY0QW type=write size=295914 2024-04-07T16:37:51.213 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.220+0000 7f346a3868c0 1 < completed part key=_multipart_UIW3DN4NUYPP.2~E60VWYZ71LX1.0 size=1048576 2024-04-07T16:37:51.217 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.224+0000 7f346a3868c0 1 < completed part key=_multipart_UIW3DN4NUYPP.2~E60VWYZ71LX1.1 size=1048576 2024-04-07T16:37:51.217 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.224+0000 7f346a3868c0 1 < canceled part key=_multipart_UIW3DN4NUYPP.2~E60VWYZ71LX1.2 size=38133 2024-04-07T16:37:51.226 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.232+0000 7f346a3868c0 1 < canceled multipart upload key=UIW3DN4NUYPP upload=E60VWYZ71LX1 tag=MYOJVHJB2444 type=write size=3183861 2024-04-07T16:37:51.231 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.236+0000 7f346a3868c0 1 > prepared operation key=U3EID5EK64HD tag=003CJ1U4JAI1 type=write size=772239 2024-04-07T16:37:51.236 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.244+0000 7f346a3868c0 1 < completed delete operation key=GBQSME7SY1AH tag=K0C95KK6SOP2 type=del 2024-04-07T16:37:51.242 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.248+0000 7f346a3868c0 1 > prepared operation key=ROFH30W3UCAX tag=4FTR4DQD6XOI type=del size=1362621 2024-04-07T16:37:51.247 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.256+0000 7f346a3868c0 1 < completed part key=_multipart_YXMDZB23YTB3.2~DI8E8CCC9LYV.0 size=1048576 2024-04-07T16:37:51.251 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.260+0000 7f346a3868c0 1 < completed part key=_multipart_YXMDZB23YTB3.2~DI8E8CCC9LYV.1 size=1048576 2024-04-07T16:37:51.255 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.264+0000 7f346a3868c0 1 < completed part key=_multipart_YXMDZB23YTB3.2~DI8E8CCC9LYV.2 size=63194 2024-04-07T16:37:51.263 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.272+0000 7f346a3868c0 1 < completed multipart upload key=YXMDZB23YTB3 upload=DI8E8CCC9LYV tag=DR3WDU1L23GJ type=write size=3208922 2024-04-07T16:37:51.272 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.280+0000 7f346a3868c0 1 > prepared part key=_multipart_C1BTVJNHIM0X.2~SC29XW9L100G.0 size=1048576 2024-04-07T16:37:51.276 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.284+0000 7f346a3868c0 1 > prepared part key=_multipart_C1BTVJNHIM0X.2~SC29XW9L100G.1 size=844700 2024-04-07T16:37:51.276 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.284+0000 7f346a3868c0 1 > prepared multipart upload key=C1BTVJNHIM0X upload=SC29XW9L100G tag=V1N0KS1GJWQK type=write size=2941852 2024-04-07T16:37:51.285 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.292+0000 7f346a3868c0 1 < suggested remove operation key=3N0QJRYVO3FV tag=SF5O5WIRGPWY type=del 2024-04-07T16:37:51.290 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.296+0000 7f346a3868c0 1 > prepared operation key=U3EID5EK64HD tag=4U28TBSQV02S type=del size=2880382 2024-04-07T16:37:51.295 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.304+0000 7f346a3868c0 1 < completed write operation key=EOYFVEI50SQ1 tag=EHYPTLV2FVHI type=write size=671854 2024-04-07T16:37:51.300 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.308+0000 7f346a3868c0 1 > prepared operation key=1WGUPH7X5UA0 tag=9VF8KRARM2O8 type=write size=95536 2024-04-07T16:37:51.305 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.312+0000 7f346a3868c0 1 < canceled operation key=YXMDZB23YTB3 tag=XMMOEWGKVLKR type=write size=932044 2024-04-07T16:37:51.310 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.316+0000 7f346a3868c0 1 > prepared operation key=HBLEXONKXJTL tag=PN5R93XNSNIF type=del size=1547928 2024-04-07T16:37:51.314 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.320+0000 7f346a3868c0 1 < completed part key=_multipart_HBLEXONKXJTL.2~E8YMJL5DBBCI.0 size=1048576 2024-04-07T16:37:51.318 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.324+0000 7f346a3868c0 1 < completed part key=_multipart_HBLEXONKXJTL.2~E8YMJL5DBBCI.1 size=838203 2024-04-07T16:37:51.326 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.332+0000 7f346a3868c0 1 < completed multipart upload key=HBLEXONKXJTL upload=E8YMJL5DBBCI tag=SQV126TPVMLZ type=write size=2935355 2024-04-07T16:37:51.331 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.340+0000 7f346a3868c0 1 > prepared operation key=GZXHYZO5SMUP tag=FUO00CYBCBMI type=del size=140969 2024-04-07T16:37:51.336 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.344+0000 7f346a3868c0 1 < completed part key=_multipart_EQR0FY86AT5X.2~30J29GK9SX3X.0 size=1048576 2024-04-07T16:37:51.340 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.348+0000 7f346a3868c0 1 < completed part key=_multipart_EQR0FY86AT5X.2~30J29GK9SX3X.1 size=903989 2024-04-07T16:37:51.349 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.356+0000 7f346a3868c0 1 < completed multipart upload key=EQR0FY86AT5X upload=30J29GK9SX3X tag=RR46CJGQYFKQ type=write size=3001141 2024-04-07T16:37:51.353 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.360+0000 7f346a3868c0 1 > prepared operation key=E84U0SX44NQ8 tag=1O2R4VT5KZC1 type=del size=771788 2024-04-07T16:37:51.358 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.364+0000 7f346a3868c0 1 < completed delete operation key=1VBSUWNI3V1T tag=VODH5D5DM304 type=del 2024-04-07T16:37:51.362 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.368+0000 7f346a3868c0 1 > prepared operation key=E84U0SX44NQ8 tag=EU0G1S8U2K39 type=del size=2509530 2024-04-07T16:37:51.367 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.376+0000 7f346a3868c0 1 < completed part key=_multipart_7G5C6HBK6QWI.2~IP7KO4OIDJ0G.0 size=1048576 2024-04-07T16:37:51.372 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.380+0000 7f346a3868c0 1 < completed part key=_multipart_7G5C6HBK6QWI.2~IP7KO4OIDJ0G.1 size=1048576 2024-04-07T16:37:51.375 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.384+0000 7f346a3868c0 1 < completed part key=_multipart_7G5C6HBK6QWI.2~IP7KO4OIDJ0G.2 size=770299 2024-04-07T16:37:51.382 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.388+0000 7f346a3868c0 1 < completed multipart upload key=7G5C6HBK6QWI upload=IP7KO4OIDJ0G tag=2KKGC3QJN1IO type=write size=3916027 2024-04-07T16:37:51.387 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.392+0000 7f346a3868c0 1 > prepared operation key=UIW3DN4NUYPP tag=WJ62ZPOUV9QW type=del size=3000170 2024-04-07T16:37:51.392 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.400+0000 7f346a3868c0 1 < completed delete operation key=XJLQZ7IS9U1T tag=3SDEIZWFVYGE type=del 2024-04-07T16:37:51.402 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.408+0000 7f346a3868c0 1 > prepared part key=_multipart_HBLEXONKXJTL.2~LXLTBLEYV78V.0 size=1048576 2024-04-07T16:37:51.406 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.412+0000 7f346a3868c0 1 > prepared part key=_multipart_HBLEXONKXJTL.2~LXLTBLEYV78V.1 size=941930 2024-04-07T16:37:51.406 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.412+0000 7f346a3868c0 1 > prepared multipart upload key=HBLEXONKXJTL upload=LXLTBLEYV78V tag=HZ55W7AK2793 type=write size=3039082 2024-04-07T16:37:51.415 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.420+0000 7f346a3868c0 1 < suggested update operation key=1VBSUWNI3V1T tag=JVQH65GPZ5M6 type=write size=823717 2024-04-07T16:37:51.419 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.428+0000 7f346a3868c0 1 > prepared operation key=LXOZ0T2KY6OR tag=X9KI6DMQD32D type=del size=2874324 2024-04-07T16:37:51.429 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.436+0000 7f346a3868c0 1 < suggested remove operation key=IP8YW2A75TCW tag=CM47RENIWFCP type=del 2024-04-07T16:37:51.437 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.444+0000 7f346a3868c0 1 > prepared part key=_multipart_QS4HYZFFAJNF.2~M7KL3D693NWB.0 size=1048576 2024-04-07T16:37:51.441 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.448+0000 7f346a3868c0 1 > prepared part key=_multipart_QS4HYZFFAJNF.2~M7KL3D693NWB.1 size=1048576 2024-04-07T16:37:51.445 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.452+0000 7f346a3868c0 1 > prepared part key=_multipart_QS4HYZFFAJNF.2~M7KL3D693NWB.2 size=131644 2024-04-07T16:37:51.445 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.452+0000 7f346a3868c0 1 > prepared multipart upload key=QS4HYZFFAJNF upload=M7KL3D693NWB tag=CAPAFQ7EEAL9 type=write size=3277372 2024-04-07T16:37:51.454 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.460+0000 7f346a3868c0 1 < suggested update operation key=EOYFVEI50SQ1 tag=82WEF0K3CLYG type=del size=2924269 2024-04-07T16:37:51.459 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.464+0000 7f346a3868c0 1 > prepared operation key=1VKVQXSQMTU1 tag=35M0ULYMB7D3 type=del size=1140254 2024-04-07T16:37:51.469 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.476+0000 7f346a3868c0 1 < suggested update operation key=NYET3212USVE tag=URE8KVVB4FZD type=del size=831156 2024-04-07T16:37:51.474 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.480+0000 7f346a3868c0 1 > prepared operation key=LXOZ0T2KY6OR tag=EO2XL5OPWHZI type=del size=2388903 2024-04-07T16:37:51.479 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.488+0000 7f346a3868c0 1 < completed write operation key=QTCIIPTXIRW2 tag=EGBK20ITY0QW type=write size=295914 2024-04-07T16:37:51.488 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.496+0000 7f346a3868c0 1 > prepared part key=_multipart_XJLQZ7IS9U1T.2~ADA6MVCJL3RF.0 size=1048576 2024-04-07T16:37:51.491 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.500+0000 7f346a3868c0 1 > prepared part key=_multipart_XJLQZ7IS9U1T.2~ADA6MVCJL3RF.1 size=390427 2024-04-07T16:37:51.491 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.500+0000 7f346a3868c0 1 > prepared multipart upload key=XJLQZ7IS9U1T upload=ADA6MVCJL3RF tag=4ZOVD59HR0MC type=write size=2487579 2024-04-07T16:37:51.498 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.504+0000 7f346a3868c0 1 < completed write operation key=U3EID5EK64HD tag=003CJ1U4JAI1 type=write size=772239 2024-04-07T16:37:51.503 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.512+0000 7f346a3868c0 1 > prepared operation key=ROFH30W3UCAX tag=4A07UY41XS1A type=del size=1138332 2024-04-07T16:37:51.508 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.516+0000 7f346a3868c0 1 < completed delete operation key=ROFH30W3UCAX tag=4FTR4DQD6XOI type=del 2024-04-07T16:37:51.518 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.524+0000 7f346a3868c0 1 > prepared part key=_multipart_1VBSUWNI3V1T.2~X2SRP65XG7SE.0 size=611700 2024-04-07T16:37:51.518 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.524+0000 7f346a3868c0 1 > prepared multipart upload key=1VBSUWNI3V1T upload=X2SRP65XG7SE tag=442RB1QPC0PV type=write size=1660276 2024-04-07T16:37:51.523 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.532+0000 7f346a3868c0 1 < completed part key=_multipart_C1BTVJNHIM0X.2~SC29XW9L100G.0 size=1048576 2024-04-07T16:37:51.527 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.536+0000 7f346a3868c0 1 < completed part key=_multipart_C1BTVJNHIM0X.2~SC29XW9L100G.1 size=844700 2024-04-07T16:37:51.534 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.540+0000 7f346a3868c0 1 < completed multipart upload key=C1BTVJNHIM0X upload=SC29XW9L100G tag=V1N0KS1GJWQK type=write size=2941852 2024-04-07T16:37:51.539 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.548+0000 7f346a3868c0 1 > prepared operation key=UIW3DN4NUYPP tag=I8CEWELZKRK2 type=del size=49993 2024-04-07T16:37:51.544 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.552+0000 7f346a3868c0 1 < completed delete operation key=U3EID5EK64HD tag=4U28TBSQV02S type=del 2024-04-07T16:37:51.548 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.556+0000 7f346a3868c0 1 > prepared operation key=QS4HYZFFAJNF tag=IXRP0FJC1468 type=del size=28623 2024-04-07T16:37:51.554 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.560+0000 7f346a3868c0 1 < completed write operation key=1WGUPH7X5UA0 tag=9VF8KRARM2O8 type=write size=95536 2024-04-07T16:37:51.558 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.564+0000 7f346a3868c0 1 > prepared operation key=U3EID5EK64HD tag=AQ7P270DYQTR type=del size=2289529 2024-04-07T16:37:51.562 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.568+0000 7f346a3868c0 1 < completed delete operation key=HBLEXONKXJTL tag=PN5R93XNSNIF type=del 2024-04-07T16:37:51.567 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.576+0000 7f346a3868c0 1 > prepared operation key=HBLEXONKXJTL tag=X6ID9JGHR7D2 type=del size=3468587 2024-04-07T16:37:51.571 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.580+0000 7f346a3868c0 1 < completed delete operation key=GZXHYZO5SMUP tag=FUO00CYBCBMI type=del 2024-04-07T16:37:51.576 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.584+0000 7f346a3868c0 1 > prepared operation key=C1BTVJNHIM0X tag=PERZM2FT8LOY type=del size=3746674 2024-04-07T16:37:51.581 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.588+0000 7f346a3868c0 1 < completed delete operation key=E84U0SX44NQ8 tag=1O2R4VT5KZC1 type=del 2024-04-07T16:37:51.590 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.596+0000 7f346a3868c0 1 > prepared part key=_multipart_IX3F0IUOKBX1.2~VWFDTU4ZENIN.0 size=1048576 2024-04-07T16:37:51.593 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.600+0000 7f346a3868c0 1 > prepared part key=_multipart_IX3F0IUOKBX1.2~VWFDTU4ZENIN.1 size=1048576 2024-04-07T16:37:51.598 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.604+0000 7f346a3868c0 1 > prepared part key=_multipart_IX3F0IUOKBX1.2~VWFDTU4ZENIN.2 size=1024321 2024-04-07T16:37:51.598 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.604+0000 7f346a3868c0 1 > prepared multipart upload key=IX3F0IUOKBX1 upload=VWFDTU4ZENIN tag=S1D8WNB4BXQG type=write size=4170049 2024-04-07T16:37:51.608 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.616+0000 7f346a3868c0 1 < suggested update operation key=E84U0SX44NQ8 tag=EU0G1S8U2K39 type=del size=2509530 2024-04-07T16:37:51.613 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.620+0000 7f346a3868c0 1 > prepared operation key=8F81G0P0YBFW tag=9O4RMZ7J6XRG type=del size=2588681 2024-04-07T16:37:51.618 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.624+0000 7f346a3868c0 1 < completed delete operation key=UIW3DN4NUYPP tag=WJ62ZPOUV9QW type=del 2024-04-07T16:37:51.627 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.632+0000 7f346a3868c0 1 > prepared part key=_multipart_E84U0SX44NQ8.2~FTIN6Y9IPYXP.0 size=406713 2024-04-07T16:37:51.627 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.632+0000 7f346a3868c0 1 > prepared multipart upload key=E84U0SX44NQ8 upload=FTIN6Y9IPYXP tag=H3TFXSGH967F type=write size=1455289 2024-04-07T16:37:51.627 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.636+0000 7f346a3868c0 1 < canceled part key=_multipart_HBLEXONKXJTL.2~LXLTBLEYV78V.0 size=1048576 2024-04-07T16:37:51.631 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.640+0000 7f346a3868c0 1 < completed part key=_multipart_HBLEXONKXJTL.2~LXLTBLEYV78V.1 size=941930 2024-04-07T16:37:51.639 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.648+0000 7f346a3868c0 1 < canceled multipart upload key=HBLEXONKXJTL upload=LXLTBLEYV78V tag=HZ55W7AK2793 type=write size=3039082 2024-04-07T16:37:51.644 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.652+0000 7f346a3868c0 1 > prepared operation key=1VKVQXSQMTU1 tag=2GBX0K1LFZRT type=write size=643486 2024-04-07T16:37:51.649 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.656+0000 7f346a3868c0 1 < completed delete operation key=LXOZ0T2KY6OR tag=X9KI6DMQD32D type=del 2024-04-07T16:37:51.654 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.660+0000 7f346a3868c0 1 > prepared operation key=QVOKSQAGHPA9 tag=NLOOFAMQBJW0 type=del size=1195343 2024-04-07T16:37:51.660 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.668+0000 7f346a3868c0 1 < completed part key=_multipart_QS4HYZFFAJNF.2~M7KL3D693NWB.0 size=1048576 2024-04-07T16:37:51.664 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.672+0000 7f346a3868c0 1 < completed part key=_multipart_QS4HYZFFAJNF.2~M7KL3D693NWB.1 size=1048576 2024-04-07T16:37:51.668 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.676+0000 7f346a3868c0 1 < completed part key=_multipart_QS4HYZFFAJNF.2~M7KL3D693NWB.2 size=131644 2024-04-07T16:37:51.676 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.684+0000 7f346a3868c0 1 < completed multipart upload key=QS4HYZFFAJNF upload=M7KL3D693NWB tag=CAPAFQ7EEAL9 type=write size=3277372 2024-04-07T16:37:51.680 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.688+0000 7f346a3868c0 1 > prepared operation key=1WGUPH7X5UA0 tag=QZUAG32ZMGSK type=write size=1035818 2024-04-07T16:37:51.685 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.692+0000 7f346a3868c0 1 < completed delete operation key=1VKVQXSQMTU1 tag=35M0ULYMB7D3 type=del 2024-04-07T16:37:51.690 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.696+0000 7f346a3868c0 1 > prepared operation key=QTCIIPTXIRW2 tag=6GJWSOHR3O1H type=del size=4118284 2024-04-07T16:37:51.700 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.708+0000 7f346a3868c0 1 < suggested update operation key=LXOZ0T2KY6OR tag=EO2XL5OPWHZI type=del size=2388903 2024-04-07T16:37:51.708 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.716+0000 7f346a3868c0 1 > prepared part key=_multipart_7G5C6HBK6QWI.2~FAI5XYN9A1LB.0 size=644382 2024-04-07T16:37:51.708 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.716+0000 7f346a3868c0 1 > prepared multipart upload key=7G5C6HBK6QWI upload=FAI5XYN9A1LB tag=9UO4RQQ27MEO type=write size=1692958 2024-04-07T16:37:51.713 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.720+0000 7f346a3868c0 1 < completed part key=_multipart_XJLQZ7IS9U1T.2~ADA6MVCJL3RF.0 size=1048576 2024-04-07T16:37:51.716 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.724+0000 7f346a3868c0 1 < completed part key=_multipart_XJLQZ7IS9U1T.2~ADA6MVCJL3RF.1 size=390427 2024-04-07T16:37:51.724 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.732+0000 7f346a3868c0 1 < completed multipart upload key=XJLQZ7IS9U1T upload=ADA6MVCJL3RF tag=4ZOVD59HR0MC type=write size=2487579 2024-04-07T16:37:51.733 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.740+0000 7f346a3868c0 1 > prepared part key=_multipart_95YVKJ14JD8W.2~CGWM7Q2AFR2H.0 size=1048576 2024-04-07T16:37:51.737 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.744+0000 7f346a3868c0 1 > prepared part key=_multipart_95YVKJ14JD8W.2~CGWM7Q2AFR2H.1 size=1048576 2024-04-07T16:37:51.741 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.748+0000 7f346a3868c0 1 > prepared part key=_multipart_95YVKJ14JD8W.2~CGWM7Q2AFR2H.2 size=840403 2024-04-07T16:37:51.741 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.748+0000 7f346a3868c0 1 > prepared multipart upload key=95YVKJ14JD8W upload=CGWM7Q2AFR2H tag=NQHNFE846KWK type=write size=3986131 2024-04-07T16:37:51.745 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.752+0000 7f346a3868c0 1 < completed delete operation key=ROFH30W3UCAX tag=4A07UY41XS1A type=del 2024-04-07T16:37:51.750 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.756+0000 7f346a3868c0 1 > prepared operation key=QS4HYZFFAJNF tag=ZFQPYKR0JE9Q type=del size=3226811 2024-04-07T16:37:51.755 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.764+0000 7f346a3868c0 1 < completed part key=_multipart_1VBSUWNI3V1T.2~X2SRP65XG7SE.0 size=611700 2024-04-07T16:37:51.765 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.772+0000 7f346a3868c0 1 < completed multipart upload key=1VBSUWNI3V1T upload=X2SRP65XG7SE tag=442RB1QPC0PV type=write size=1660276 2024-04-07T16:37:51.771 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.776+0000 7f346a3868c0 1 > prepared operation key=C1BTVJNHIM0X tag=23FPK2RTTHRN type=del size=485577 2024-04-07T16:37:51.776 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.784+0000 7f346a3868c0 1 < completed delete operation key=UIW3DN4NUYPP tag=I8CEWELZKRK2 type=del 2024-04-07T16:37:51.786 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.792+0000 7f346a3868c0 1 > prepared part key=_multipart_X5Q46GO8YTOK.2~LIUKF6NZSBL3.0 size=1048576 2024-04-07T16:37:51.790 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.796+0000 7f346a3868c0 1 > prepared part key=_multipart_X5Q46GO8YTOK.2~LIUKF6NZSBL3.1 size=936329 2024-04-07T16:37:51.790 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.796+0000 7f346a3868c0 1 > prepared multipart upload key=X5Q46GO8YTOK upload=LIUKF6NZSBL3 tag=M4KHE7L0JHFL type=write size=3033481 2024-04-07T16:37:51.794 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.800+0000 7f346a3868c0 1 < completed delete operation key=QS4HYZFFAJNF tag=IXRP0FJC1468 type=del 2024-04-07T16:37:51.800 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.808+0000 7f346a3868c0 1 > prepared operation key=XJLQZ7IS9U1T tag=4OFEQ9158VLV type=write size=753895 2024-04-07T16:37:51.806 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.812+0000 7f346a3868c0 1 < canceled operation key=U3EID5EK64HD tag=AQ7P270DYQTR type=del size=2289529 2024-04-07T16:37:51.811 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.816+0000 7f346a3868c0 1 > prepared operation key=1WGUPH7X5UA0 tag=O3BRVADKHJ9K type=del size=2258755 2024-04-07T16:37:51.816 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.824+0000 7f346a3868c0 1 < completed delete operation key=HBLEXONKXJTL tag=X6ID9JGHR7D2 type=del 2024-04-07T16:37:51.820 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.828+0000 7f346a3868c0 1 > prepared operation key=7G5C6HBK6QWI tag=RFZSFYBU5EZ3 type=write size=11749 2024-04-07T16:37:51.825 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.832+0000 7f346a3868c0 1 < completed delete operation key=C1BTVJNHIM0X tag=PERZM2FT8LOY type=del 2024-04-07T16:37:51.830 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.836+0000 7f346a3868c0 1 > prepared operation key=95YVKJ14JD8W tag=PIEM7AQN3L2P type=del size=878850 2024-04-07T16:37:51.834 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.840+0000 7f346a3868c0 1 < completed part key=_multipart_IX3F0IUOKBX1.2~VWFDTU4ZENIN.0 size=1048576 2024-04-07T16:37:51.838 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.844+0000 7f346a3868c0 1 < completed part key=_multipart_IX3F0IUOKBX1.2~VWFDTU4ZENIN.1 size=1048576 2024-04-07T16:37:51.843 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.848+0000 7f346a3868c0 1 < completed part key=_multipart_IX3F0IUOKBX1.2~VWFDTU4ZENIN.2 size=1024321 2024-04-07T16:37:51.851 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.860+0000 7f346a3868c0 1 < completed multipart upload key=IX3F0IUOKBX1 upload=VWFDTU4ZENIN tag=S1D8WNB4BXQG type=write size=4170049 2024-04-07T16:37:51.856 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.864+0000 7f346a3868c0 1 > prepared operation key=HBLEXONKXJTL tag=02HIASYOL9M6 type=del size=1890986 2024-04-07T16:37:51.861 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.868+0000 7f346a3868c0 1 < completed delete operation key=8F81G0P0YBFW tag=9O4RMZ7J6XRG type=del 2024-04-07T16:37:51.866 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.872+0000 7f346a3868c0 1 > prepared operation key=X5Q46GO8YTOK tag=PXUBWGQAW9LA type=del size=1849024 2024-04-07T16:37:51.871 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.880+0000 7f346a3868c0 1 < completed part key=_multipart_E84U0SX44NQ8.2~FTIN6Y9IPYXP.0 size=406713 2024-04-07T16:37:51.879 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.888+0000 7f346a3868c0 1 < completed multipart upload key=E84U0SX44NQ8 upload=FTIN6Y9IPYXP tag=H3TFXSGH967F type=write size=1455289 2024-04-07T16:37:51.888 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.896+0000 7f346a3868c0 1 > prepared part key=_multipart_IP8YW2A75TCW.2~LJE9V62QYM7C.0 size=850950 2024-04-07T16:37:51.888 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.896+0000 7f346a3868c0 1 > prepared multipart upload key=IP8YW2A75TCW upload=LJE9V62QYM7C tag=S0S8WZKSJW8E type=write size=1899526 2024-04-07T16:37:51.893 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.900+0000 7f346a3868c0 1 < completed write operation key=1VKVQXSQMTU1 tag=2GBX0K1LFZRT type=write size=643486 2024-04-07T16:37:51.898 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.904+0000 7f346a3868c0 1 > prepared operation key=IP8YW2A75TCW tag=SB1R3D5KD6UK type=del size=1862383 2024-04-07T16:37:51.907 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.912+0000 7f346a3868c0 1 < suggested update operation key=QVOKSQAGHPA9 tag=NLOOFAMQBJW0 type=del size=1195343 2024-04-07T16:37:51.915 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.924+0000 7f346a3868c0 1 > prepared part key=_multipart_NYET3212USVE.2~SAGI2PMSKM2L.0 size=1048576 2024-04-07T16:37:51.920 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.928+0000 7f346a3868c0 1 > prepared part key=_multipart_NYET3212USVE.2~SAGI2PMSKM2L.1 size=1048576 2024-04-07T16:37:51.924 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.932+0000 7f346a3868c0 1 > prepared part key=_multipart_NYET3212USVE.2~SAGI2PMSKM2L.2 size=166373 2024-04-07T16:37:51.924 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.932+0000 7f346a3868c0 1 > prepared multipart upload key=NYET3212USVE upload=SAGI2PMSKM2L tag=2TQMGCJCI0QJ type=write size=3312101 2024-04-07T16:37:51.928 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.936+0000 7f346a3868c0 1 < canceled operation key=1WGUPH7X5UA0 tag=QZUAG32ZMGSK type=write size=1035818 2024-04-07T16:37:51.933 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.940+0000 7f346a3868c0 1 > prepared operation key=6XDS0A7L0IY8 tag=3V4RHTYROKZT type=write size=777104 2024-04-07T16:37:51.938 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.944+0000 7f346a3868c0 1 < completed delete operation key=QTCIIPTXIRW2 tag=6GJWSOHR3O1H type=del 2024-04-07T16:37:51.943 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.948+0000 7f346a3868c0 1 > prepared operation key=X5Q46GO8YTOK tag=8JYPIOGB1SL6 type=del size=1210158 2024-04-07T16:37:51.948 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.956+0000 7f346a3868c0 1 < completed part key=_multipart_7G5C6HBK6QWI.2~FAI5XYN9A1LB.0 size=644382 2024-04-07T16:37:51.956 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.964+0000 7f346a3868c0 1 < completed multipart upload key=7G5C6HBK6QWI upload=FAI5XYN9A1LB tag=9UO4RQQ27MEO type=write size=1692958 2024-04-07T16:37:51.965 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.972+0000 7f346a3868c0 1 > prepared part key=_multipart_YOK8BAEOMPUR.2~GS0K66YSYXIJ.0 size=55079 2024-04-07T16:37:51.965 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.972+0000 7f346a3868c0 1 > prepared multipart upload key=YOK8BAEOMPUR upload=GS0K66YSYXIJ tag=9Y95U3K7N951 type=write size=1103655 2024-04-07T16:37:51.970 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.976+0000 7f346a3868c0 1 < completed part key=_multipart_95YVKJ14JD8W.2~CGWM7Q2AFR2H.0 size=1048576 2024-04-07T16:37:51.974 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.980+0000 7f346a3868c0 1 < completed part key=_multipart_95YVKJ14JD8W.2~CGWM7Q2AFR2H.1 size=1048576 2024-04-07T16:37:51.978 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.984+0000 7f346a3868c0 1 < completed part key=_multipart_95YVKJ14JD8W.2~CGWM7Q2AFR2H.2 size=840403 2024-04-07T16:37:51.986 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.992+0000 7f346a3868c0 1 < completed multipart upload key=95YVKJ14JD8W upload=CGWM7Q2AFR2H tag=NQHNFE846KWK type=write size=3986131 2024-04-07T16:37:51.991 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:51.996+0000 7f346a3868c0 1 > prepared operation key=LXOZ0T2KY6OR tag=5G8KDM6HGBGZ type=del size=1277381 2024-04-07T16:37:51.996 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.004+0000 7f346a3868c0 1 < completed delete operation key=QS4HYZFFAJNF tag=ZFQPYKR0JE9Q type=del 2024-04-07T16:37:52.005 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.012+0000 7f346a3868c0 1 > prepared part key=_multipart_3N0QJRYVO3FV.2~533K7VS1K5GY.0 size=391258 2024-04-07T16:37:52.005 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.012+0000 7f346a3868c0 1 > prepared multipart upload key=3N0QJRYVO3FV upload=533K7VS1K5GY tag=NSOETOO959PL type=write size=1439834 2024-04-07T16:37:52.013 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.020+0000 7f346a3868c0 1 < suggested remove operation key=C1BTVJNHIM0X tag=23FPK2RTTHRN type=del 2024-04-07T16:37:52.022 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.028+0000 7f346a3868c0 1 > prepared part key=_multipart_C1BTVJNHIM0X.2~Q5NL8WJIRSP4.0 size=1048576 2024-04-07T16:37:52.027 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.032+0000 7f346a3868c0 1 > prepared part key=_multipart_C1BTVJNHIM0X.2~Q5NL8WJIRSP4.1 size=1048576 2024-04-07T16:37:52.031 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.036+0000 7f346a3868c0 1 > prepared part key=_multipart_C1BTVJNHIM0X.2~Q5NL8WJIRSP4.2 size=238416 2024-04-07T16:37:52.031 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.036+0000 7f346a3868c0 1 > prepared multipart upload key=C1BTVJNHIM0X upload=Q5NL8WJIRSP4 tag=8SL5H2TGIUFR type=write size=3384144 2024-04-07T16:37:52.036 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.044+0000 7f346a3868c0 1 < completed part key=_multipart_X5Q46GO8YTOK.2~LIUKF6NZSBL3.0 size=1048576 2024-04-07T16:37:52.040 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.048+0000 7f346a3868c0 1 < completed part key=_multipart_X5Q46GO8YTOK.2~LIUKF6NZSBL3.1 size=936329 2024-04-07T16:37:52.049 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.056+0000 7f346a3868c0 1 < completed multipart upload key=X5Q46GO8YTOK upload=LIUKF6NZSBL3 tag=M4KHE7L0JHFL type=write size=3033481 2024-04-07T16:37:52.058 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.064+0000 7f346a3868c0 1 > prepared part key=_multipart_ROFH30W3UCAX.2~1955HKFK6VQU.0 size=744976 2024-04-07T16:37:52.058 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.064+0000 7f346a3868c0 1 > prepared multipart upload key=ROFH30W3UCAX upload=1955HKFK6VQU tag=848GXLMYWZQD type=write size=1793552 2024-04-07T16:37:52.064 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.072+0000 7f346a3868c0 1 < completed write operation key=XJLQZ7IS9U1T tag=4OFEQ9158VLV type=write size=753895 2024-04-07T16:37:52.074 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.080+0000 7f346a3868c0 1 > prepared part key=_multipart_6XDS0A7L0IY8.2~V3KO48V329MZ.0 size=1048576 2024-04-07T16:37:52.078 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.084+0000 7f346a3868c0 1 > prepared part key=_multipart_6XDS0A7L0IY8.2~V3KO48V329MZ.1 size=1048576 2024-04-07T16:37:52.082 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.088+0000 7f346a3868c0 1 > prepared part key=_multipart_6XDS0A7L0IY8.2~V3KO48V329MZ.2 size=79046 2024-04-07T16:37:52.082 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.088+0000 7f346a3868c0 1 > prepared multipart upload key=6XDS0A7L0IY8 upload=V3KO48V329MZ tag=IA7YY1FUY19O type=write size=3224774 2024-04-07T16:37:52.086 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.092+0000 7f346a3868c0 1 < completed delete operation key=1WGUPH7X5UA0 tag=O3BRVADKHJ9K type=del 2024-04-07T16:37:52.091 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.100+0000 7f346a3868c0 1 > prepared operation key=N14BUXLJONHZ tag=VFW0XBID1NY7 type=del size=2989239 2024-04-07T16:37:52.097 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.104+0000 7f346a3868c0 1 < completed write operation key=7G5C6HBK6QWI tag=RFZSFYBU5EZ3 type=write size=11749 2024-04-07T16:37:52.102 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.108+0000 7f346a3868c0 1 > prepared operation key=QVOKSQAGHPA9 tag=U5WAX4Q0UOCR type=del size=2487491 2024-04-07T16:37:52.112 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.120+0000 7f346a3868c0 1 < suggested update operation key=95YVKJ14JD8W tag=PIEM7AQN3L2P type=del size=878850 2024-04-07T16:37:52.122 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.128+0000 7f346a3868c0 1 > prepared part key=_multipart_IP8YW2A75TCW.2~4YCKUBQQNWEC.0 size=1048576 2024-04-07T16:37:52.126 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.132+0000 7f346a3868c0 1 > prepared part key=_multipart_IP8YW2A75TCW.2~4YCKUBQQNWEC.1 size=751871 2024-04-07T16:37:52.127 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.132+0000 7f346a3868c0 1 > prepared multipart upload key=IP8YW2A75TCW upload=4YCKUBQQNWEC tag=2C6UCTEC8KBL type=write size=2849023 2024-04-07T16:37:52.132 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.140+0000 7f346a3868c0 1 < canceled operation key=HBLEXONKXJTL tag=02HIASYOL9M6 type=del size=1890986 2024-04-07T16:37:52.141 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.148+0000 7f346a3868c0 1 > prepared part key=_multipart_XJLQZ7IS9U1T.2~ZQMSIXXVJT8X.0 size=1002229 2024-04-07T16:37:52.141 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.148+0000 7f346a3868c0 1 > prepared multipart upload key=XJLQZ7IS9U1T upload=ZQMSIXXVJT8X tag=DVMSYZBEAOHK type=write size=2050805 2024-04-07T16:37:52.145 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.152+0000 7f346a3868c0 1 < completed delete operation key=X5Q46GO8YTOK tag=PXUBWGQAW9LA type=del 2024-04-07T16:37:52.150 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.156+0000 7f346a3868c0 1 > prepared operation key=1VKVQXSQMTU1 tag=70A5USVMSL1Y type=write size=623071 2024-04-07T16:37:52.156 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.164+0000 7f346a3868c0 1 < completed part key=_multipart_IP8YW2A75TCW.2~LJE9V62QYM7C.0 size=850950 2024-04-07T16:37:52.164 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.172+0000 7f346a3868c0 1 < completed multipart upload key=IP8YW2A75TCW upload=LJE9V62QYM7C tag=S0S8WZKSJW8E type=write size=1899526 2024-04-07T16:37:52.170 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.176+0000 7f346a3868c0 1 > prepared operation key=E84U0SX44NQ8 tag=OU8Z9B0R9IVI type=del size=806230 2024-04-07T16:37:52.175 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.184+0000 7f346a3868c0 1 < completed delete operation key=IP8YW2A75TCW tag=SB1R3D5KD6UK type=del 2024-04-07T16:37:52.184 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.192+0000 7f346a3868c0 1 > prepared part key=_multipart_EOYFVEI50SQ1.2~3Q11HB6YYG14.0 size=1009941 2024-04-07T16:37:52.184 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.192+0000 7f346a3868c0 1 > prepared multipart upload key=EOYFVEI50SQ1 upload=3Q11HB6YYG14 tag=9D78KRX4YP61 type=write size=2058517 2024-04-07T16:37:52.189 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.196+0000 7f346a3868c0 1 < completed part key=_multipart_NYET3212USVE.2~SAGI2PMSKM2L.0 size=1048576 2024-04-07T16:37:52.192 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.200+0000 7f346a3868c0 1 < completed part key=_multipart_NYET3212USVE.2~SAGI2PMSKM2L.1 size=1048576 2024-04-07T16:37:52.193 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.200+0000 7f346a3868c0 1 < canceled part key=_multipart_NYET3212USVE.2~SAGI2PMSKM2L.2 size=166373 2024-04-07T16:37:52.200 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.208+0000 7f346a3868c0 1 < canceled multipart upload key=NYET3212USVE upload=SAGI2PMSKM2L tag=2TQMGCJCI0QJ type=write size=3312101 2024-04-07T16:37:52.210 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.216+0000 7f346a3868c0 1 > prepared part key=_multipart_C1BTVJNHIM0X.2~X9WEQVQJX2W4.0 size=1048576 2024-04-07T16:37:52.213 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.220+0000 7f346a3868c0 1 > prepared part key=_multipart_C1BTVJNHIM0X.2~X9WEQVQJX2W4.1 size=1048576 2024-04-07T16:37:52.217 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.224+0000 7f346a3868c0 1 > prepared part key=_multipart_C1BTVJNHIM0X.2~X9WEQVQJX2W4.2 size=426292 2024-04-07T16:37:52.217 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.224+0000 7f346a3868c0 1 > prepared multipart upload key=C1BTVJNHIM0X upload=X9WEQVQJX2W4 tag=14B5V0VKI0KM type=write size=3572020 2024-04-07T16:37:52.222 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.228+0000 7f346a3868c0 1 < completed write operation key=6XDS0A7L0IY8 tag=3V4RHTYROKZT type=write size=777104 2024-04-07T16:37:52.231 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.236+0000 7f346a3868c0 1 > prepared part key=_multipart_XJLQZ7IS9U1T.2~U7FP6AJ8CYN9.0 size=1048576 2024-04-07T16:37:52.237 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.240+0000 7f346a3868c0 1 > prepared part key=_multipart_XJLQZ7IS9U1T.2~U7FP6AJ8CYN9.1 size=1048576 2024-04-07T16:37:52.239 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.244+0000 7f346a3868c0 1 > prepared part key=_multipart_XJLQZ7IS9U1T.2~U7FP6AJ8CYN9.2 size=874441 2024-04-07T16:37:52.239 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.244+0000 7f346a3868c0 1 > prepared multipart upload key=XJLQZ7IS9U1T upload=U7FP6AJ8CYN9 tag=Z6YRDG14UBHB type=write size=4020169 2024-04-07T16:37:52.243 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.252+0000 7f346a3868c0 1 < completed delete operation key=X5Q46GO8YTOK tag=8JYPIOGB1SL6 type=del 2024-04-07T16:37:52.252 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.260+0000 7f346a3868c0 1 > prepared part key=_multipart_GBQSME7SY1AH.2~07EU0YHBU711.0 size=821039 2024-04-07T16:37:52.252 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.260+0000 7f346a3868c0 1 > prepared multipart upload key=GBQSME7SY1AH upload=07EU0YHBU711 tag=0LPAGNOA93YD type=write size=1869615 2024-04-07T16:37:52.257 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.264+0000 7f346a3868c0 1 < completed part key=_multipart_YOK8BAEOMPUR.2~GS0K66YSYXIJ.0 size=55079 2024-04-07T16:37:52.264 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.272+0000 7f346a3868c0 1 < completed multipart upload key=YOK8BAEOMPUR upload=GS0K66YSYXIJ tag=9Y95U3K7N951 type=write size=1103655 2024-04-07T16:37:52.269 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.276+0000 7f346a3868c0 1 > prepared operation key=XJLQZ7IS9U1T tag=TE1D26BJP8B2 type=write size=333137 2024-04-07T16:37:52.274 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.280+0000 7f346a3868c0 1 < completed delete operation key=LXOZ0T2KY6OR tag=5G8KDM6HGBGZ type=del 2024-04-07T16:37:52.283 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.288+0000 7f346a3868c0 1 > prepared part key=_multipart_QVOKSQAGHPA9.2~SY55DGTJ7IP7.0 size=1048576 2024-04-07T16:37:52.286 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.292+0000 7f346a3868c0 1 > prepared part key=_multipart_QVOKSQAGHPA9.2~SY55DGTJ7IP7.1 size=642980 2024-04-07T16:37:52.286 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.292+0000 7f346a3868c0 1 > prepared multipart upload key=QVOKSQAGHPA9 upload=SY55DGTJ7IP7 tag=S78UMFXS1VQJ type=write size=2740132 2024-04-07T16:37:52.291 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.300+0000 7f346a3868c0 1 < completed part key=_multipart_3N0QJRYVO3FV.2~533K7VS1K5GY.0 size=391258 2024-04-07T16:37:52.299 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.308+0000 7f346a3868c0 1 < completed multipart upload key=3N0QJRYVO3FV upload=533K7VS1K5GY tag=NSOETOO959PL type=write size=1439834 2024-04-07T16:37:52.304 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.312+0000 7f346a3868c0 1 > prepared operation key=CL4PP1RKZU0E tag=F55F22F62XM7 type=write size=534054 2024-04-07T16:37:52.309 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.316+0000 7f346a3868c0 1 < completed part key=_multipart_C1BTVJNHIM0X.2~Q5NL8WJIRSP4.0 size=1048576 2024-04-07T16:37:52.313 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.320+0000 7f346a3868c0 1 < completed part key=_multipart_C1BTVJNHIM0X.2~Q5NL8WJIRSP4.1 size=1048576 2024-04-07T16:37:52.317 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.324+0000 7f346a3868c0 1 < completed part key=_multipart_C1BTVJNHIM0X.2~Q5NL8WJIRSP4.2 size=238416 2024-04-07T16:37:52.325 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.332+0000 7f346a3868c0 1 < completed multipart upload key=C1BTVJNHIM0X upload=Q5NL8WJIRSP4 tag=8SL5H2TGIUFR type=write size=3384144 2024-04-07T16:37:52.331 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.336+0000 7f346a3868c0 1 > prepared operation key=QVOKSQAGHPA9 tag=GQDMSLX3RHA8 type=del size=1971497 2024-04-07T16:37:52.335 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.344+0000 7f346a3868c0 1 < completed part key=_multipart_ROFH30W3UCAX.2~1955HKFK6VQU.0 size=744976 2024-04-07T16:37:52.343 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.352+0000 7f346a3868c0 1 < completed multipart upload key=ROFH30W3UCAX upload=1955HKFK6VQU tag=848GXLMYWZQD type=write size=1793552 2024-04-07T16:37:52.348 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.356+0000 7f346a3868c0 1 > prepared operation key=EQR0FY86AT5X tag=XP0I2TKPLGF3 type=write size=818292 2024-04-07T16:37:52.352 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.360+0000 7f346a3868c0 1 < completed part key=_multipart_6XDS0A7L0IY8.2~V3KO48V329MZ.0 size=1048576 2024-04-07T16:37:52.356 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.364+0000 7f346a3868c0 1 < completed part key=_multipart_6XDS0A7L0IY8.2~V3KO48V329MZ.1 size=1048576 2024-04-07T16:37:52.360 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.368+0000 7f346a3868c0 1 < completed part key=_multipart_6XDS0A7L0IY8.2~V3KO48V329MZ.2 size=79046 2024-04-07T16:37:52.367 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.376+0000 7f346a3868c0 1 < completed multipart upload key=6XDS0A7L0IY8 upload=V3KO48V329MZ tag=IA7YY1FUY19O type=write size=3224774 2024-04-07T16:37:52.372 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.380+0000 7f346a3868c0 1 > prepared operation key=QS4HYZFFAJNF tag=WMPNF7MO2FU1 type=write size=248556 2024-04-07T16:37:52.377 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.384+0000 7f346a3868c0 1 < canceled operation key=N14BUXLJONHZ tag=VFW0XBID1NY7 type=del size=2989239 2024-04-07T16:37:52.381 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.388+0000 7f346a3868c0 1 > prepared operation key=GBQSME7SY1AH tag=4WP3HQ33RK4P type=del size=3943972 2024-04-07T16:37:52.386 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.392+0000 7f346a3868c0 1 < canceled operation key=QVOKSQAGHPA9 tag=U5WAX4Q0UOCR type=del size=2487491 2024-04-07T16:37:52.391 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.396+0000 7f346a3868c0 1 > prepared operation key=6XDS0A7L0IY8 tag=A1EIR0PMK4TH type=del size=715317 2024-04-07T16:37:52.396 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.404+0000 7f346a3868c0 1 < completed part key=_multipart_IP8YW2A75TCW.2~4YCKUBQQNWEC.0 size=1048576 2024-04-07T16:37:52.399 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.408+0000 7f346a3868c0 1 < completed part key=_multipart_IP8YW2A75TCW.2~4YCKUBQQNWEC.1 size=751871 2024-04-07T16:37:52.407 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.416+0000 7f346a3868c0 1 < completed multipart upload key=IP8YW2A75TCW upload=4YCKUBQQNWEC tag=2C6UCTEC8KBL type=write size=2849023 2024-04-07T16:37:52.416 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.424+0000 7f346a3868c0 1 > prepared part key=_multipart_ROFH30W3UCAX.2~9P0IJ5CRVEQO.0 size=1048576 2024-04-07T16:37:52.420 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.428+0000 7f346a3868c0 1 > prepared part key=_multipart_ROFH30W3UCAX.2~9P0IJ5CRVEQO.1 size=1048576 2024-04-07T16:37:52.424 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.432+0000 7f346a3868c0 1 > prepared part key=_multipart_ROFH30W3UCAX.2~9P0IJ5CRVEQO.2 size=803398 2024-04-07T16:37:52.424 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.432+0000 7f346a3868c0 1 > prepared multipart upload key=ROFH30W3UCAX upload=9P0IJ5CRVEQO tag=4C84N15SFQ0V type=write size=3949126 2024-04-07T16:37:52.430 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.436+0000 7f346a3868c0 1 < completed part key=_multipart_XJLQZ7IS9U1T.2~ZQMSIXXVJT8X.0 size=1002229 2024-04-07T16:37:52.437 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.444+0000 7f346a3868c0 1 < completed multipart upload key=XJLQZ7IS9U1T upload=ZQMSIXXVJT8X tag=DVMSYZBEAOHK type=write size=2050805 2024-04-07T16:37:52.443 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.452+0000 7f346a3868c0 1 > prepared operation key=E84U0SX44NQ8 tag=2E9BXJEC636R type=write size=696424 2024-04-07T16:37:52.453 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.460+0000 7f346a3868c0 1 < suggested update operation key=1VKVQXSQMTU1 tag=70A5USVMSL1Y type=write size=623071 2024-04-07T16:37:52.462 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.468+0000 7f346a3868c0 1 > prepared part key=_multipart_1WGUPH7X5UA0.2~A484FHU0C4VW.0 size=1048576 2024-04-07T16:37:52.468 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.476+0000 7f346a3868c0 1 > prepared part key=_multipart_1WGUPH7X5UA0.2~A484FHU0C4VW.1 size=1048576 2024-04-07T16:37:52.472 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.480+0000 7f346a3868c0 1 > prepared part key=_multipart_1WGUPH7X5UA0.2~A484FHU0C4VW.2 size=252790 2024-04-07T16:37:52.472 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.480+0000 7f346a3868c0 1 > prepared multipart upload key=1WGUPH7X5UA0 upload=A484FHU0C4VW tag=BWHSHXLJI5CF type=write size=3398518 2024-04-07T16:37:52.478 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.484+0000 7f346a3868c0 1 < completed delete operation key=E84U0SX44NQ8 tag=OU8Z9B0R9IVI type=del 2024-04-07T16:37:52.483 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.488+0000 7f346a3868c0 1 > prepared operation key=95YVKJ14JD8W tag=CQK4ADIU1XSK type=del size=3628399 2024-04-07T16:37:52.488 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.496+0000 7f346a3868c0 1 < completed part key=_multipart_EOYFVEI50SQ1.2~3Q11HB6YYG14.0 size=1009941 2024-04-07T16:37:52.496 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.504+0000 7f346a3868c0 1 < completed multipart upload key=EOYFVEI50SQ1 upload=3Q11HB6YYG14 tag=9D78KRX4YP61 type=write size=2058517 2024-04-07T16:37:52.502 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.508+0000 7f346a3868c0 1 > prepared operation key=UIW3DN4NUYPP tag=6C7UQK4I5XPU type=del size=3733977 2024-04-07T16:37:52.507 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.516+0000 7f346a3868c0 1 < completed part key=_multipart_C1BTVJNHIM0X.2~X9WEQVQJX2W4.0 size=1048576 2024-04-07T16:37:52.511 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.520+0000 7f346a3868c0 1 < completed part key=_multipart_C1BTVJNHIM0X.2~X9WEQVQJX2W4.1 size=1048576 2024-04-07T16:37:52.515 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.524+0000 7f346a3868c0 1 < completed part key=_multipart_C1BTVJNHIM0X.2~X9WEQVQJX2W4.2 size=426292 2024-04-07T16:37:52.524 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.532+0000 7f346a3868c0 1 < completed multipart upload key=C1BTVJNHIM0X upload=X9WEQVQJX2W4 tag=14B5V0VKI0KM type=write size=3572020 2024-04-07T16:37:52.533 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.540+0000 7f346a3868c0 1 > prepared part key=_multipart_N14BUXLJONHZ.2~Z5OLQJ37Y2KN.0 size=638251 2024-04-07T16:37:52.533 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.540+0000 7f346a3868c0 1 > prepared multipart upload key=N14BUXLJONHZ upload=Z5OLQJ37Y2KN tag=JW3LRVTH6V3Y type=write size=1686827 2024-04-07T16:37:52.534 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.540+0000 7f346a3868c0 1 < canceled part key=_multipart_XJLQZ7IS9U1T.2~U7FP6AJ8CYN9.0 size=1048576 2024-04-07T16:37:52.539 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.544+0000 7f346a3868c0 1 < completed part key=_multipart_XJLQZ7IS9U1T.2~U7FP6AJ8CYN9.1 size=1048576 2024-04-07T16:37:52.543 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.552+0000 7f346a3868c0 1 < completed part key=_multipart_XJLQZ7IS9U1T.2~U7FP6AJ8CYN9.2 size=874441 2024-04-07T16:37:52.553 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.560+0000 7f346a3868c0 1 < canceled multipart upload key=XJLQZ7IS9U1T upload=U7FP6AJ8CYN9 tag=Z6YRDG14UBHB type=write size=4020169 2024-04-07T16:37:52.558 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.564+0000 7f346a3868c0 1 > prepared operation key=IX3F0IUOKBX1 tag=I6FTSC7MI8ST type=del size=3763714 2024-04-07T16:37:52.564 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.572+0000 7f346a3868c0 1 < completed part key=_multipart_GBQSME7SY1AH.2~07EU0YHBU711.0 size=821039 2024-04-07T16:37:52.573 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.580+0000 7f346a3868c0 1 < completed multipart upload key=GBQSME7SY1AH upload=07EU0YHBU711 tag=0LPAGNOA93YD type=write size=1869615 2024-04-07T16:37:52.582 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.588+0000 7f346a3868c0 1 > prepared part key=_multipart_GBQSME7SY1AH.2~B2IN3ZKOAFCB.0 size=1048576 2024-04-07T16:37:52.586 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.592+0000 7f346a3868c0 1 > prepared part key=_multipart_GBQSME7SY1AH.2~B2IN3ZKOAFCB.1 size=19481 2024-04-07T16:37:52.586 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.592+0000 7f346a3868c0 1 > prepared multipart upload key=GBQSME7SY1AH upload=B2IN3ZKOAFCB tag=2K0KID74TJQC type=write size=2116633 2024-04-07T16:37:52.591 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.600+0000 7f346a3868c0 1 < completed write operation key=XJLQZ7IS9U1T tag=TE1D26BJP8B2 type=write size=333137 2024-04-07T16:37:52.596 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.604+0000 7f346a3868c0 1 > prepared operation key=8F81G0P0YBFW tag=EY33A1RVPYPT type=write size=508381 2024-04-07T16:37:52.602 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.608+0000 7f346a3868c0 1 < completed part key=_multipart_QVOKSQAGHPA9.2~SY55DGTJ7IP7.0 size=1048576 2024-04-07T16:37:52.606 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.612+0000 7f346a3868c0 1 < completed part key=_multipart_QVOKSQAGHPA9.2~SY55DGTJ7IP7.1 size=642980 2024-04-07T16:37:52.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:37:52 smithi144 bash[66654]: cluster 2024-04-07T16:37:51.059260+0000 mgr.y (mgr.25175) 675 : cluster [DBG] pgmap v1310: 137 pgs: 137 active+clean; 583 KiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 381 KiB/s rd, 132 KiB/s wr, 673 op/s 2024-04-07T16:37:52.606 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:37:52 smithi144 bash[95233]: ts=2024-04-07T16:37:52.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:37:52.615 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.620+0000 7f346a3868c0 1 < completed multipart upload key=QVOKSQAGHPA9 upload=SY55DGTJ7IP7 tag=S78UMFXS1VQJ type=write size=2740132 2024-04-07T16:37:52.620 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.628+0000 7f346a3868c0 1 > prepared operation key=XJLQZ7IS9U1T tag=RKU1WZJYX1AP type=del size=3773357 2024-04-07T16:37:52.625 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.632+0000 7f346a3868c0 1 < completed write operation key=CL4PP1RKZU0E tag=F55F22F62XM7 type=write size=534054 2024-04-07T16:37:52.630 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.636+0000 7f346a3868c0 1 > prepared operation key=UIW3DN4NUYPP tag=2U03PDRDJOAH type=del size=3058831 2024-04-07T16:37:52.640 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.648+0000 7f346a3868c0 1 < suggested update operation key=QVOKSQAGHPA9 tag=GQDMSLX3RHA8 type=del size=1971497 2024-04-07T16:37:52.649 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.656+0000 7f346a3868c0 1 > prepared part key=_multipart_N14BUXLJONHZ.2~WET2HGLBKL9T.0 size=322546 2024-04-07T16:37:52.649 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.656+0000 7f346a3868c0 1 > prepared multipart upload key=N14BUXLJONHZ upload=WET2HGLBKL9T tag=XTW081OLWIIX type=write size=1371122 2024-04-07T16:37:52.654 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.660+0000 7f346a3868c0 1 < completed write operation key=EQR0FY86AT5X tag=XP0I2TKPLGF3 type=write size=818292 2024-04-07T16:37:52.663 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.672+0000 7f346a3868c0 1 > prepared part key=_multipart_C1BTVJNHIM0X.2~CWOBB4LV7GKH.0 size=1048576 2024-04-07T16:37:52.667 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.676+0000 7f346a3868c0 1 > prepared part key=_multipart_C1BTVJNHIM0X.2~CWOBB4LV7GKH.1 size=645482 2024-04-07T16:37:52.667 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.676+0000 7f346a3868c0 1 > prepared multipart upload key=C1BTVJNHIM0X upload=CWOBB4LV7GKH tag=QH1CN5EBBF8V type=write size=2742634 2024-04-07T16:37:52.672 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.680+0000 7f346a3868c0 1 < completed write operation key=QS4HYZFFAJNF tag=WMPNF7MO2FU1 type=write size=248556 2024-04-07T16:37:52.682 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.688+0000 7f346a3868c0 1 > prepared part key=_multipart_XJLQZ7IS9U1T.2~AJG9U9QB0R2J.0 size=1048576 2024-04-07T16:37:52.686 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.692+0000 7f346a3868c0 1 > prepared part key=_multipart_XJLQZ7IS9U1T.2~AJG9U9QB0R2J.1 size=1048576 2024-04-07T16:37:52.690 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.696+0000 7f346a3868c0 1 > prepared part key=_multipart_XJLQZ7IS9U1T.2~AJG9U9QB0R2J.2 size=698933 2024-04-07T16:37:52.690 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.696+0000 7f346a3868c0 1 > prepared multipart upload key=XJLQZ7IS9U1T upload=AJG9U9QB0R2J tag=4WKF5QLHK9PB type=write size=3844661 2024-04-07T16:37:52.695 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.704+0000 7f346a3868c0 1 < completed delete operation key=GBQSME7SY1AH tag=4WP3HQ33RK4P type=del 2024-04-07T16:37:52.704 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.712+0000 7f346a3868c0 1 > prepared part key=_multipart_1VKVQXSQMTU1.2~INKAY7XF1QE7.0 size=1048576 2024-04-07T16:37:52.707 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.716+0000 7f346a3868c0 1 > prepared part key=_multipart_1VKVQXSQMTU1.2~INKAY7XF1QE7.1 size=166599 2024-04-07T16:37:52.708 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.716+0000 7f346a3868c0 1 > prepared multipart upload key=1VKVQXSQMTU1 upload=INKAY7XF1QE7 tag=UKI9GKZA3KBO type=write size=2263751 2024-04-07T16:37:52.712 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.720+0000 7f346a3868c0 1 < completed delete operation key=6XDS0A7L0IY8 tag=A1EIR0PMK4TH type=del 2024-04-07T16:37:52.721 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.728+0000 7f346a3868c0 1 > prepared part key=_multipart_GBQSME7SY1AH.2~88ZNMECIVC0O.0 size=1048576 2024-04-07T16:37:52.725 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.732+0000 7f346a3868c0 1 > prepared part key=_multipart_GBQSME7SY1AH.2~88ZNMECIVC0O.1 size=1048576 2024-04-07T16:37:52.729 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.736+0000 7f346a3868c0 1 > prepared part key=_multipart_GBQSME7SY1AH.2~88ZNMECIVC0O.2 size=927628 2024-04-07T16:37:52.729 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.736+0000 7f346a3868c0 1 > prepared multipart upload key=GBQSME7SY1AH upload=88ZNMECIVC0O tag=T2Q3PCR8F0DT type=write size=4073356 2024-04-07T16:37:52.734 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.740+0000 7f346a3868c0 1 < completed part key=_multipart_ROFH30W3UCAX.2~9P0IJ5CRVEQO.0 size=1048576 2024-04-07T16:37:52.738 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.744+0000 7f346a3868c0 1 < completed part key=_multipart_ROFH30W3UCAX.2~9P0IJ5CRVEQO.1 size=1048576 2024-04-07T16:37:52.738 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.744+0000 7f346a3868c0 1 < canceled part key=_multipart_ROFH30W3UCAX.2~9P0IJ5CRVEQO.2 size=803398 2024-04-07T16:37:52.746 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.752+0000 7f346a3868c0 1 < canceled multipart upload key=ROFH30W3UCAX upload=9P0IJ5CRVEQO tag=4C84N15SFQ0V type=write size=3949126 2024-04-07T16:37:52.751 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.760+0000 7f346a3868c0 1 > prepared operation key=GBQSME7SY1AH tag=T0DUE3IA4L2E type=del size=731930 2024-04-07T16:37:52.761 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.768+0000 7f346a3868c0 1 < suggested update operation key=E84U0SX44NQ8 tag=2E9BXJEC636R type=write size=696424 2024-04-07T16:37:52.771 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.780+0000 7f346a3868c0 1 > prepared part key=_multipart_QTCIIPTXIRW2.2~1EMOX6SQ8QQ4.0 size=519523 2024-04-07T16:37:52.771 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.780+0000 7f346a3868c0 1 > prepared multipart upload key=QTCIIPTXIRW2 upload=1EMOX6SQ8QQ4 tag=S2WH1LZ1EJ29 type=write size=1568099 2024-04-07T16:37:52.776 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.784+0000 7f346a3868c0 1 < completed part key=_multipart_1WGUPH7X5UA0.2~A484FHU0C4VW.0 size=1048576 2024-04-07T16:37:52.781 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.788+0000 7f346a3868c0 1 < completed part key=_multipart_1WGUPH7X5UA0.2~A484FHU0C4VW.1 size=1048576 2024-04-07T16:37:52.784 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.792+0000 7f346a3868c0 1 < completed part key=_multipart_1WGUPH7X5UA0.2~A484FHU0C4VW.2 size=252790 2024-04-07T16:37:52.794 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.800+0000 7f346a3868c0 1 < completed multipart upload key=1WGUPH7X5UA0 upload=A484FHU0C4VW tag=BWHSHXLJI5CF type=write size=3398518 2024-04-07T16:37:52.799 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.804+0000 7f346a3868c0 1 > prepared operation key=ROFH30W3UCAX tag=4QRNGE13YELX type=del size=3348375 2024-04-07T16:37:52.803 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.812+0000 7f346a3868c0 1 < canceled operation key=95YVKJ14JD8W tag=CQK4ADIU1XSK type=del size=3628399 2024-04-07T16:37:52.808 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.816+0000 7f346a3868c0 1 > prepared operation key=UIW3DN4NUYPP tag=8Q7KE49WR2ZX type=write size=912170 2024-04-07T16:37:52.813 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.820+0000 7f346a3868c0 1 < canceled operation key=UIW3DN4NUYPP tag=6C7UQK4I5XPU type=del size=3733977 2024-04-07T16:37:52.822 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.828+0000 7f346a3868c0 1 > prepared part key=_multipart_X5Q46GO8YTOK.2~8TKECZDS7YYZ.0 size=1048576 2024-04-07T16:37:52.826 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.832+0000 7f346a3868c0 1 > prepared part key=_multipart_X5Q46GO8YTOK.2~8TKECZDS7YYZ.1 size=1048576 2024-04-07T16:37:52.830 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.836+0000 7f346a3868c0 1 > prepared part key=_multipart_X5Q46GO8YTOK.2~8TKECZDS7YYZ.2 size=954287 2024-04-07T16:37:52.830 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.836+0000 7f346a3868c0 1 > prepared multipart upload key=X5Q46GO8YTOK upload=8TKECZDS7YYZ tag=VUKRFXYAZDO5 type=write size=4100015 2024-04-07T16:37:52.830 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.836+0000 7f346a3868c0 1 < canceled part key=_multipart_N14BUXLJONHZ.2~Z5OLQJ37Y2KN.0 size=638251 2024-04-07T16:37:52.838 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.844+0000 7f346a3868c0 1 < canceled multipart upload key=N14BUXLJONHZ upload=Z5OLQJ37Y2KN tag=JW3LRVTH6V3Y type=write size=1686827 2024-04-07T16:37:52.843 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.852+0000 7f346a3868c0 1 > prepared operation key=1VKVQXSQMTU1 tag=3V7NE3BC5A3Z type=del size=876015 2024-04-07T16:37:52.848 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.856+0000 7f346a3868c0 1 < completed delete operation key=IX3F0IUOKBX1 tag=I6FTSC7MI8ST type=del 2024-04-07T16:37:52.857 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.864+0000 7f346a3868c0 1 > prepared part key=_multipart_IX3F0IUOKBX1.2~D6XA0PQAXYKD.0 size=1048576 2024-04-07T16:37:52.861 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.868+0000 7f346a3868c0 1 > prepared part key=_multipart_IX3F0IUOKBX1.2~D6XA0PQAXYKD.1 size=1026043 2024-04-07T16:37:52.861 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.868+0000 7f346a3868c0 1 > prepared multipart upload key=IX3F0IUOKBX1 upload=D6XA0PQAXYKD tag=RYA1FAP53XAJ type=write size=3123195 2024-04-07T16:37:52.867 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.872+0000 7f346a3868c0 1 < completed part key=_multipart_GBQSME7SY1AH.2~B2IN3ZKOAFCB.0 size=1048576 2024-04-07T16:37:52.871 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.880+0000 7f346a3868c0 1 < completed part key=_multipart_GBQSME7SY1AH.2~B2IN3ZKOAFCB.1 size=19481 2024-04-07T16:37:52.880 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.888+0000 7f346a3868c0 1 < completed multipart upload key=GBQSME7SY1AH upload=B2IN3ZKOAFCB tag=2K0KID74TJQC type=write size=2116633 2024-04-07T16:37:52.890 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.896+0000 7f346a3868c0 1 > prepared part key=_multipart_ROFH30W3UCAX.2~FPVRB566FGRG.0 size=1048576 2024-04-07T16:37:52.894 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.900+0000 7f346a3868c0 1 > prepared part key=_multipart_ROFH30W3UCAX.2~FPVRB566FGRG.1 size=1048576 2024-04-07T16:37:52.898 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.904+0000 7f346a3868c0 1 > prepared part key=_multipart_ROFH30W3UCAX.2~FPVRB566FGRG.2 size=439677 2024-04-07T16:37:52.898 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.904+0000 7f346a3868c0 1 > prepared multipart upload key=ROFH30W3UCAX upload=FPVRB566FGRG tag=PR2QDZTA21H2 type=write size=3585405 2024-04-07T16:37:52.904 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.912+0000 7f346a3868c0 1 < canceled operation key=8F81G0P0YBFW tag=EY33A1RVPYPT type=write size=508381 2024-04-07T16:37:52.909 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.916+0000 7f346a3868c0 1 > prepared operation key=IP8YW2A75TCW tag=0FE0ZTYPW3DH type=del size=2533263 2024-04-07T16:37:52.919 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.928+0000 7f346a3868c0 1 < suggested remove operation key=XJLQZ7IS9U1T tag=RKU1WZJYX1AP type=del 2024-04-07T16:37:52.928 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.936+0000 7f346a3868c0 1 > prepared part key=_multipart_N14BUXLJONHZ.2~0CZECAHQTV3M.0 size=1048576 2024-04-07T16:37:52.932 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.940+0000 7f346a3868c0 1 > prepared part key=_multipart_N14BUXLJONHZ.2~0CZECAHQTV3M.1 size=423041 2024-04-07T16:37:52.932 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.940+0000 7f346a3868c0 1 > prepared multipart upload key=N14BUXLJONHZ upload=0CZECAHQTV3M tag=V8VRSC9P5GF5 type=write size=2520193 2024-04-07T16:37:52.938 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.944+0000 7f346a3868c0 1 < completed delete operation key=UIW3DN4NUYPP tag=2U03PDRDJOAH type=del 2024-04-07T16:37:52.941 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:37:52 smithi100 bash[74942]: cluster 2024-04-07T16:37:51.059260+0000 mgr.y (mgr.25175) 675 : cluster [DBG] pgmap v1310: 137 pgs: 137 active+clean; 583 KiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 381 KiB/s rd, 132 KiB/s wr, 673 op/s 2024-04-07T16:37:52.941 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:37:52 smithi100 bash[76202]: cluster 2024-04-07T16:37:51.059260+0000 mgr.y (mgr.25175) 675 : cluster [DBG] pgmap v1310: 137 pgs: 137 active+clean; 583 KiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 381 KiB/s rd, 132 KiB/s wr, 673 op/s 2024-04-07T16:37:52.946 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.952+0000 7f346a3868c0 1 > prepared part key=_multipart_EOYFVEI50SQ1.2~5BI794UWXBE7.0 size=1048576 2024-04-07T16:37:52.950 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.956+0000 7f346a3868c0 1 > prepared part key=_multipart_EOYFVEI50SQ1.2~5BI794UWXBE7.1 size=560280 2024-04-07T16:37:52.950 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.956+0000 7f346a3868c0 1 > prepared multipart upload key=EOYFVEI50SQ1 upload=5BI794UWXBE7 tag=809AMV0EM7B4 type=write size=2657432 2024-04-07T16:37:52.956 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.964+0000 7f346a3868c0 1 < completed part key=_multipart_N14BUXLJONHZ.2~WET2HGLBKL9T.0 size=322546 2024-04-07T16:37:52.964 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.972+0000 7f346a3868c0 1 < completed multipart upload key=N14BUXLJONHZ upload=WET2HGLBKL9T tag=XTW081OLWIIX type=write size=1371122 2024-04-07T16:37:52.969 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.976+0000 7f346a3868c0 1 > prepared operation key=QVOKSQAGHPA9 tag=SM38CW39LNKE type=del size=4159250 2024-04-07T16:37:52.974 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.980+0000 7f346a3868c0 1 < completed part key=_multipart_C1BTVJNHIM0X.2~CWOBB4LV7GKH.0 size=1048576 2024-04-07T16:37:52.978 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.984+0000 7f346a3868c0 1 < completed part key=_multipart_C1BTVJNHIM0X.2~CWOBB4LV7GKH.1 size=645482 2024-04-07T16:37:52.987 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:52.992+0000 7f346a3868c0 1 < completed multipart upload key=C1BTVJNHIM0X upload=CWOBB4LV7GKH tag=QH1CN5EBBF8V type=write size=2742634 2024-04-07T16:37:52.992 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.000+0000 7f346a3868c0 1 > prepared operation key=HBLEXONKXJTL tag=YTQD4KUUV92P type=del size=3582045 2024-04-07T16:37:52.997 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.004+0000 7f346a3868c0 1 < completed part key=_multipart_XJLQZ7IS9U1T.2~AJG9U9QB0R2J.0 size=1048576 2024-04-07T16:37:53.001 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.008+0000 7f346a3868c0 1 < completed part key=_multipart_XJLQZ7IS9U1T.2~AJG9U9QB0R2J.1 size=1048576 2024-04-07T16:37:53.001 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.008+0000 7f346a3868c0 1 < canceled part key=_multipart_XJLQZ7IS9U1T.2~AJG9U9QB0R2J.2 size=698933 2024-04-07T16:37:53.009 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.016+0000 7f346a3868c0 1 < canceled multipart upload key=XJLQZ7IS9U1T upload=AJG9U9QB0R2J tag=4WKF5QLHK9PB type=write size=3844661 2024-04-07T16:37:53.015 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.020+0000 7f346a3868c0 1 > prepared operation key=YOK8BAEOMPUR tag=LPO1WGA0OCBN type=del size=2195008 2024-04-07T16:37:53.020 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.028+0000 7f346a3868c0 1 < completed part key=_multipart_1VKVQXSQMTU1.2~INKAY7XF1QE7.0 size=1048576 2024-04-07T16:37:53.024 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.032+0000 7f346a3868c0 1 < completed part key=_multipart_1VKVQXSQMTU1.2~INKAY7XF1QE7.1 size=166599 2024-04-07T16:37:53.034 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.040+0000 7f346a3868c0 1 < completed multipart upload key=1VKVQXSQMTU1 upload=INKAY7XF1QE7 tag=UKI9GKZA3KBO type=write size=2263751 2024-04-07T16:37:53.039 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.048+0000 7f346a3868c0 1 > prepared operation key=1WGUPH7X5UA0 tag=DGJTTY6GR5VS type=del size=3849004 2024-04-07T16:37:53.044 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.052+0000 7f346a3868c0 1 < completed part key=_multipart_GBQSME7SY1AH.2~88ZNMECIVC0O.0 size=1048576 2024-04-07T16:37:53.048 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.056+0000 7f346a3868c0 1 < completed part key=_multipart_GBQSME7SY1AH.2~88ZNMECIVC0O.1 size=1048576 2024-04-07T16:37:53.052 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.060+0000 7f346a3868c0 1 < completed part key=_multipart_GBQSME7SY1AH.2~88ZNMECIVC0O.2 size=927628 2024-04-07T16:37:53.060 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.068+0000 7f346a3868c0 1 < completed multipart upload key=GBQSME7SY1AH upload=88ZNMECIVC0O tag=T2Q3PCR8F0DT type=write size=4073356 2024-04-07T16:37:53.069 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.076+0000 7f346a3868c0 1 > prepared part key=_multipart_95YVKJ14JD8W.2~19L0Q3OU78OS.0 size=1048576 2024-04-07T16:37:53.073 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.080+0000 7f346a3868c0 1 > prepared part key=_multipart_95YVKJ14JD8W.2~19L0Q3OU78OS.1 size=491734 2024-04-07T16:37:53.073 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.080+0000 7f346a3868c0 1 > prepared multipart upload key=95YVKJ14JD8W upload=19L0Q3OU78OS tag=BAYVBH3MENMD type=write size=2588886 2024-04-07T16:37:53.078 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.084+0000 7f346a3868c0 1 < completed delete operation key=GBQSME7SY1AH tag=T0DUE3IA4L2E type=del 2024-04-07T16:37:53.083 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.088+0000 7f346a3868c0 1 > prepared operation key=YXMDZB23YTB3 tag=7JXAGGNJ5VG6 type=del size=1829225 2024-04-07T16:37:53.088 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.096+0000 7f346a3868c0 1 < completed part key=_multipart_QTCIIPTXIRW2.2~1EMOX6SQ8QQ4.0 size=519523 2024-04-07T16:37:53.096 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.104+0000 7f346a3868c0 1 < completed multipart upload key=QTCIIPTXIRW2 upload=1EMOX6SQ8QQ4 tag=S2WH1LZ1EJ29 type=write size=1568099 2024-04-07T16:37:53.101 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.108+0000 7f346a3868c0 1 > prepared operation key=DXTO3AJ429AC tag=PQ1KAAJ5U8UD type=write size=720457 2024-04-07T16:37:53.106 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.112+0000 7f346a3868c0 1 < completed delete operation key=ROFH30W3UCAX tag=4QRNGE13YELX type=del 2024-04-07T16:37:53.111 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.116+0000 7f346a3868c0 1 > prepared operation key=YXMDZB23YTB3 tag=6CDD9M54PBM6 type=del size=3177565 2024-04-07T16:37:53.116 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.124+0000 7f346a3868c0 1 < completed write operation key=UIW3DN4NUYPP tag=8Q7KE49WR2ZX type=write size=912170 2024-04-07T16:37:53.125 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.132+0000 7f346a3868c0 1 > prepared part key=_multipart_EQR0FY86AT5X.2~GHOMJ5AHCUOI.0 size=901080 2024-04-07T16:37:53.125 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.132+0000 7f346a3868c0 1 > prepared multipart upload key=EQR0FY86AT5X upload=GHOMJ5AHCUOI tag=1XV5UKNHRFM9 type=write size=1949656 2024-04-07T16:37:53.132 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.140+0000 7f346a3868c0 1 < completed part key=_multipart_X5Q46GO8YTOK.2~8TKECZDS7YYZ.0 size=1048576 2024-04-07T16:37:53.136 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.144+0000 7f346a3868c0 1 < completed part key=_multipart_X5Q46GO8YTOK.2~8TKECZDS7YYZ.1 size=1048576 2024-04-07T16:37:53.140 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.148+0000 7f346a3868c0 1 < completed part key=_multipart_X5Q46GO8YTOK.2~8TKECZDS7YYZ.2 size=954287 2024-04-07T16:37:53.148 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.156+0000 7f346a3868c0 1 < completed multipart upload key=X5Q46GO8YTOK upload=8TKECZDS7YYZ tag=VUKRFXYAZDO5 type=write size=4100015 2024-04-07T16:37:53.155 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.164+0000 7f346a3868c0 1 > prepared operation key=NYET3212USVE tag=45I9XY5OVUT6 type=del size=2308272 2024-04-07T16:37:53.160 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.168+0000 7f346a3868c0 1 < completed delete operation key=1VKVQXSQMTU1 tag=3V7NE3BC5A3Z type=del 2024-04-07T16:37:53.167 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.176+0000 7f346a3868c0 1 > prepared operation key=IP8YW2A75TCW tag=9SG6PWE8AQSF type=del size=405312 2024-04-07T16:37:53.172 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.180+0000 7f346a3868c0 1 < completed part key=_multipart_IX3F0IUOKBX1.2~D6XA0PQAXYKD.0 size=1048576 2024-04-07T16:37:53.176 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.184+0000 7f346a3868c0 1 < completed part key=_multipart_IX3F0IUOKBX1.2~D6XA0PQAXYKD.1 size=1026043 2024-04-07T16:37:53.184 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.192+0000 7f346a3868c0 1 < completed multipart upload key=IX3F0IUOKBX1 upload=D6XA0PQAXYKD tag=RYA1FAP53XAJ type=write size=3123195 2024-04-07T16:37:53.189 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.196+0000 7f346a3868c0 1 > prepared operation key=1WGUPH7X5UA0 tag=BSGQ9C4W5MDX type=del size=1102453 2024-04-07T16:37:53.190 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.196+0000 7f346a3868c0 1 < canceled part key=_multipart_ROFH30W3UCAX.2~FPVRB566FGRG.0 size=1048576 2024-04-07T16:37:53.194 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.200+0000 7f346a3868c0 1 < completed part key=_multipart_ROFH30W3UCAX.2~FPVRB566FGRG.1 size=1048576 2024-04-07T16:37:53.198 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.204+0000 7f346a3868c0 1 < completed part key=_multipart_ROFH30W3UCAX.2~FPVRB566FGRG.2 size=439677 2024-04-07T16:37:53.207 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.212+0000 7f346a3868c0 1 < canceled multipart upload key=ROFH30W3UCAX upload=FPVRB566FGRG tag=PR2QDZTA21H2 type=write size=3585405 2024-04-07T16:37:53.216 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.224+0000 7f346a3868c0 1 > prepared part key=_multipart_YXMDZB23YTB3.2~WD782VE54GG6.0 size=1048576 2024-04-07T16:37:53.220 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.228+0000 7f346a3868c0 1 > prepared part key=_multipart_YXMDZB23YTB3.2~WD782VE54GG6.1 size=989719 2024-04-07T16:37:53.220 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.228+0000 7f346a3868c0 1 > prepared multipart upload key=YXMDZB23YTB3 upload=WD782VE54GG6 tag=YKMPSZD0B41M type=write size=3086871 2024-04-07T16:37:53.231 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.236+0000 7f346a3868c0 1 < suggested remove operation key=IP8YW2A75TCW tag=0FE0ZTYPW3DH type=del 2024-04-07T16:37:53.240 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.248+0000 7f346a3868c0 1 > prepared part key=_multipart_1VKVQXSQMTU1.2~XEKG3SLUMUY5.0 size=1048576 2024-04-07T16:37:53.244 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.252+0000 7f346a3868c0 1 > prepared part key=_multipart_1VKVQXSQMTU1.2~XEKG3SLUMUY5.1 size=727626 2024-04-07T16:37:53.244 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.252+0000 7f346a3868c0 1 > prepared multipart upload key=1VKVQXSQMTU1 upload=XEKG3SLUMUY5 tag=7NYMT07B5L0C type=write size=2824778 2024-04-07T16:37:53.245 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.252+0000 7f346a3868c0 1 < canceled part key=_multipart_N14BUXLJONHZ.2~0CZECAHQTV3M.0 size=1048576 2024-04-07T16:37:53.249 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.256+0000 7f346a3868c0 1 < completed part key=_multipart_N14BUXLJONHZ.2~0CZECAHQTV3M.1 size=423041 2024-04-07T16:37:53.258 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.264+0000 7f346a3868c0 1 < canceled multipart upload key=N14BUXLJONHZ upload=0CZECAHQTV3M tag=V8VRSC9P5GF5 type=write size=2520193 2024-04-07T16:37:53.261 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.268+0000 7f346a3868c0 1 > prepared operation key=CL4PP1RKZU0E tag=Y9DGHOLXAQ98 type=del size=2828363 2024-04-07T16:37:53.266 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.272+0000 7f346a3868c0 1 < completed part key=_multipart_EOYFVEI50SQ1.2~5BI794UWXBE7.0 size=1048576 2024-04-07T16:37:53.266 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.272+0000 7f346a3868c0 1 < canceled part key=_multipart_EOYFVEI50SQ1.2~5BI794UWXBE7.1 size=560280 2024-04-07T16:37:53.275 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.280+0000 7f346a3868c0 1 < canceled multipart upload key=EOYFVEI50SQ1 upload=5BI794UWXBE7 tag=809AMV0EM7B4 type=write size=2657432 2024-04-07T16:37:53.284 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.292+0000 7f346a3868c0 1 > prepared part key=_multipart_NYET3212USVE.2~HCSPL58B7UB6.0 size=1048576 2024-04-07T16:37:53.288 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.296+0000 7f346a3868c0 1 > prepared part key=_multipart_NYET3212USVE.2~HCSPL58B7UB6.1 size=73408 2024-04-07T16:37:53.289 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.296+0000 7f346a3868c0 1 > prepared multipart upload key=NYET3212USVE upload=HCSPL58B7UB6 tag=3OILNTHW0WZO type=write size=2170560 2024-04-07T16:37:53.293 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.300+0000 7f346a3868c0 1 < completed delete operation key=QVOKSQAGHPA9 tag=SM38CW39LNKE type=del 2024-04-07T16:37:53.299 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.304+0000 7f346a3868c0 1 > prepared operation key=HBLEXONKXJTL tag=9711ZEMM865U type=del size=3642498 2024-04-07T16:37:53.303 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.312+0000 7f346a3868c0 1 < completed delete operation key=HBLEXONKXJTL tag=YTQD4KUUV92P type=del 2024-04-07T16:37:53.313 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.320+0000 7f346a3868c0 1 > prepared part key=_multipart_XJLQZ7IS9U1T.2~C91QW81GRT6R.0 size=1048576 2024-04-07T16:37:53.317 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.324+0000 7f346a3868c0 1 > prepared part key=_multipart_XJLQZ7IS9U1T.2~C91QW81GRT6R.1 size=416312 2024-04-07T16:37:53.317 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.324+0000 7f346a3868c0 1 > prepared multipart upload key=XJLQZ7IS9U1T upload=C91QW81GRT6R tag=T7FF3CZ0G81T type=write size=2513464 2024-04-07T16:37:53.323 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.328+0000 7f346a3868c0 1 < canceled operation key=YOK8BAEOMPUR tag=LPO1WGA0OCBN type=del size=2195008 2024-04-07T16:37:53.328 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.336+0000 7f346a3868c0 1 > prepared operation key=GZXHYZO5SMUP tag=QWZMB02XZ9K1 type=write size=773840 2024-04-07T16:37:53.333 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.340+0000 7f346a3868c0 1 < completed delete operation key=1WGUPH7X5UA0 tag=DGJTTY6GR5VS type=del 2024-04-07T16:37:53.344 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.352+0000 7f346a3868c0 1 > prepared part key=_multipart_QS4HYZFFAJNF.2~0J9RTET005QI.0 size=1048576 2024-04-07T16:37:53.348 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.356+0000 7f346a3868c0 1 > prepared part key=_multipart_QS4HYZFFAJNF.2~0J9RTET005QI.1 size=1048576 2024-04-07T16:37:53.353 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.360+0000 7f346a3868c0 1 > prepared part key=_multipart_QS4HYZFFAJNF.2~0J9RTET005QI.2 size=349782 2024-04-07T16:37:53.353 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.360+0000 7f346a3868c0 1 > prepared multipart upload key=QS4HYZFFAJNF upload=0J9RTET005QI tag=1M8JL7MRMP16 type=write size=3495510 2024-04-07T16:37:53.358 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.364+0000 7f346a3868c0 1 < completed part key=_multipart_95YVKJ14JD8W.2~19L0Q3OU78OS.0 size=1048576 2024-04-07T16:37:53.362 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.368+0000 7f346a3868c0 1 < completed part key=_multipart_95YVKJ14JD8W.2~19L0Q3OU78OS.1 size=491734 2024-04-07T16:37:53.371 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.380+0000 7f346a3868c0 1 < completed multipart upload key=95YVKJ14JD8W upload=19L0Q3OU78OS tag=BAYVBH3MENMD type=write size=2588886 2024-04-07T16:37:53.376 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.384+0000 7f346a3868c0 1 > prepared operation key=IP8YW2A75TCW tag=BW0HKUOK1IPT type=del size=1186890 2024-04-07T16:37:53.382 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.388+0000 7f346a3868c0 1 < completed delete operation key=YXMDZB23YTB3 tag=7JXAGGNJ5VG6 type=del 2024-04-07T16:37:53.386 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.392+0000 7f346a3868c0 1 > prepared operation key=8F81G0P0YBFW tag=WH0EOSDGH03S type=del size=3531958 2024-04-07T16:37:53.397 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.404+0000 7f346a3868c0 1 < suggested remove operation key=DXTO3AJ429AC tag=PQ1KAAJ5U8UD type=write 2024-04-07T16:37:53.402 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.408+0000 7f346a3868c0 1 > prepared operation key=95YVKJ14JD8W tag=M3AHCLWHU97P type=del size=1510259 2024-04-07T16:37:53.408 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.416+0000 7f346a3868c0 1 < completed delete operation key=YXMDZB23YTB3 tag=6CDD9M54PBM6 type=del 2024-04-07T16:37:53.419 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.424+0000 7f346a3868c0 1 > prepared part key=_multipart_GZXHYZO5SMUP.2~MB4JUAPKF97L.0 size=1048576 2024-04-07T16:37:53.423 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.432+0000 7f346a3868c0 1 > prepared part key=_multipart_GZXHYZO5SMUP.2~MB4JUAPKF97L.1 size=426024 2024-04-07T16:37:53.423 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.432+0000 7f346a3868c0 1 > prepared multipart upload key=GZXHYZO5SMUP upload=MB4JUAPKF97L tag=0JNN7HU6G7KH type=write size=2523176 2024-04-07T16:37:53.428 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.436+0000 7f346a3868c0 1 < completed part key=_multipart_EQR0FY86AT5X.2~GHOMJ5AHCUOI.0 size=901080 2024-04-07T16:37:53.436 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.444+0000 7f346a3868c0 1 < completed multipart upload key=EQR0FY86AT5X upload=GHOMJ5AHCUOI tag=1XV5UKNHRFM9 type=write size=1949656 2024-04-07T16:37:53.446 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.452+0000 7f346a3868c0 1 > prepared part key=_multipart_UIW3DN4NUYPP.2~LEBB7JU0FHKC.0 size=902546 2024-04-07T16:37:53.446 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.452+0000 7f346a3868c0 1 > prepared multipart upload key=UIW3DN4NUYPP upload=LEBB7JU0FHKC tag=NWFLAZQVPXY8 type=write size=1951122 2024-04-07T16:37:53.452 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.460+0000 7f346a3868c0 1 < canceled operation key=NYET3212USVE tag=45I9XY5OVUT6 type=del size=2308272 2024-04-07T16:37:53.460 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.468+0000 7f346a3868c0 1 > prepared part key=_multipart_N14BUXLJONHZ.2~H2N15A0LM714.0 size=120702 2024-04-07T16:37:53.460 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.468+0000 7f346a3868c0 1 > prepared multipart upload key=N14BUXLJONHZ upload=H2N15A0LM714 tag=5BORYOQV2JNI type=write size=1169278 2024-04-07T16:37:53.469 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.476+0000 7f346a3868c0 1 < suggested remove operation key=IP8YW2A75TCW tag=9SG6PWE8AQSF type=del 2024-04-07T16:37:53.478 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.484+0000 7f346a3868c0 1 > prepared part key=_multipart_C1BTVJNHIM0X.2~UZ1SBEIVF04T.0 size=783339 2024-04-07T16:37:53.478 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.484+0000 7f346a3868c0 1 > prepared multipart upload key=C1BTVJNHIM0X upload=UZ1SBEIVF04T tag=VWXZVLJ7PSYE type=write size=1831915 2024-04-07T16:37:53.483 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.488+0000 7f346a3868c0 1 < completed delete operation key=1WGUPH7X5UA0 tag=BSGQ9C4W5MDX type=del 2024-04-07T16:37:53.491 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.500+0000 7f346a3868c0 1 > prepared part key=_multipart_IP8YW2A75TCW.2~753PL3FP720Q.0 size=803741 2024-04-07T16:37:53.492 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.500+0000 7f346a3868c0 1 > prepared multipart upload key=IP8YW2A75TCW upload=753PL3FP720Q tag=FJ8ORY4UIV1T type=write size=1852317 2024-04-07T16:37:53.496 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.504+0000 7f346a3868c0 1 < completed part key=_multipart_YXMDZB23YTB3.2~WD782VE54GG6.0 size=1048576 2024-04-07T16:37:53.501 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.508+0000 7f346a3868c0 1 < completed part key=_multipart_YXMDZB23YTB3.2~WD782VE54GG6.1 size=989719 2024-04-07T16:37:53.510 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.516+0000 7f346a3868c0 1 < completed multipart upload key=YXMDZB23YTB3 upload=WD782VE54GG6 tag=YKMPSZD0B41M type=write size=3086871 2024-04-07T16:37:53.514 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.520+0000 7f346a3868c0 1 > prepared operation key=XJLQZ7IS9U1T tag=VQCRMV3TDAYR type=write size=139742 2024-04-07T16:37:53.519 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.524+0000 7f346a3868c0 1 < completed part key=_multipart_1VKVQXSQMTU1.2~XEKG3SLUMUY5.0 size=1048576 2024-04-07T16:37:53.523 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.528+0000 7f346a3868c0 1 < completed part key=_multipart_1VKVQXSQMTU1.2~XEKG3SLUMUY5.1 size=727626 2024-04-07T16:37:53.531 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.540+0000 7f346a3868c0 1 < completed multipart upload key=1VKVQXSQMTU1 upload=XEKG3SLUMUY5 tag=7NYMT07B5L0C type=write size=2824778 2024-04-07T16:37:53.536 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.544+0000 7f346a3868c0 1 > prepared operation key=ROFH30W3UCAX tag=3P8EZ9ORV4FD type=write size=49194 2024-04-07T16:37:53.541 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.548+0000 7f346a3868c0 1 < completed delete operation key=CL4PP1RKZU0E tag=Y9DGHOLXAQ98 type=del 2024-04-07T16:37:53.545 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.552+0000 7f346a3868c0 1 > prepared operation key=8F81G0P0YBFW tag=8OZ7PTMQFHQ2 type=del size=2751017 2024-04-07T16:37:53.546 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.552+0000 7f346a3868c0 1 < canceled part key=_multipart_NYET3212USVE.2~HCSPL58B7UB6.0 size=1048576 2024-04-07T16:37:53.551 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.560+0000 7f346a3868c0 1 < completed part key=_multipart_NYET3212USVE.2~HCSPL58B7UB6.1 size=73408 2024-04-07T16:37:53.560 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.568+0000 7f346a3868c0 1 < canceled multipart upload key=NYET3212USVE upload=HCSPL58B7UB6 tag=3OILNTHW0WZO type=write size=2170560 2024-04-07T16:37:53.568 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.576+0000 7f346a3868c0 1 > prepared part key=_multipart_U3EID5EK64HD.2~73MAKVR90MC7.0 size=1048576 2024-04-07T16:37:53.572 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.580+0000 7f346a3868c0 1 > prepared part key=_multipart_U3EID5EK64HD.2~73MAKVR90MC7.1 size=1048576 2024-04-07T16:37:53.575 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.584+0000 7f346a3868c0 1 > prepared part key=_multipart_U3EID5EK64HD.2~73MAKVR90MC7.2 size=743340 2024-04-07T16:37:53.576 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.584+0000 7f346a3868c0 1 > prepared multipart upload key=U3EID5EK64HD upload=73MAKVR90MC7 tag=WTSM96DY3I3C type=write size=3889068 2024-04-07T16:37:53.580 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.588+0000 7f346a3868c0 1 < completed delete operation key=HBLEXONKXJTL tag=9711ZEMM865U type=del 2024-04-07T16:37:53.585 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.592+0000 7f346a3868c0 1 > prepared operation key=EQR0FY86AT5X tag=08ITSM1C5POG type=del size=954942 2024-04-07T16:37:53.586 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.592+0000 7f346a3868c0 1 < canceled part key=_multipart_XJLQZ7IS9U1T.2~C91QW81GRT6R.0 size=1048576 2024-04-07T16:37:53.590 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.596+0000 7f346a3868c0 1 < completed part key=_multipart_XJLQZ7IS9U1T.2~C91QW81GRT6R.1 size=416312 2024-04-07T16:37:53.597 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.604+0000 7f346a3868c0 1 < canceled multipart upload key=XJLQZ7IS9U1T upload=C91QW81GRT6R tag=T7FF3CZ0G81T type=write size=2513464 2024-04-07T16:37:53.602 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.608+0000 7f346a3868c0 1 > prepared operation key=IP8YW2A75TCW tag=S29NHYKHJRRA type=del size=1276205 2024-04-07T16:37:53.607 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.612+0000 7f346a3868c0 1 < completed write operation key=GZXHYZO5SMUP tag=QWZMB02XZ9K1 type=write size=773840 2024-04-07T16:37:53.615 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.624+0000 7f346a3868c0 1 > prepared part key=_multipart_IP8YW2A75TCW.2~LJFF76ZF7QT4.0 size=1048576 2024-04-07T16:37:53.619 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.628+0000 7f346a3868c0 1 > prepared part key=_multipart_IP8YW2A75TCW.2~LJFF76ZF7QT4.1 size=704028 2024-04-07T16:37:53.619 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.628+0000 7f346a3868c0 1 > prepared multipart upload key=IP8YW2A75TCW upload=LJFF76ZF7QT4 tag=GZ5V68E7O3DA type=write size=2801180 2024-04-07T16:37:53.624 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.632+0000 7f346a3868c0 1 < completed part key=_multipart_QS4HYZFFAJNF.2~0J9RTET005QI.0 size=1048576 2024-04-07T16:37:53.627 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.636+0000 7f346a3868c0 1 < completed part key=_multipart_QS4HYZFFAJNF.2~0J9RTET005QI.1 size=1048576 2024-04-07T16:37:53.631 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.640+0000 7f346a3868c0 1 < completed part key=_multipart_QS4HYZFFAJNF.2~0J9RTET005QI.2 size=349782 2024-04-07T16:37:53.639 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.648+0000 7f346a3868c0 1 < completed multipart upload key=QS4HYZFFAJNF upload=0J9RTET005QI tag=1M8JL7MRMP16 type=write size=3495510 2024-04-07T16:37:53.647 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.656+0000 7f346a3868c0 1 > prepared part key=_multipart_U3EID5EK64HD.2~F53FFNQT5Z3W.0 size=456672 2024-04-07T16:37:53.647 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.656+0000 7f346a3868c0 1 > prepared multipart upload key=U3EID5EK64HD upload=F53FFNQT5Z3W tag=XXUQYE122JZM type=write size=1505248 2024-04-07T16:37:53.653 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.660+0000 7f346a3868c0 -1 < failed to complete operation key=IP8YW2A75TCW tag=BW0HKUOK1IPT type=del size=1186890: (22) Invalid argument 2024-04-07T16:37:53.658 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.664+0000 7f346a3868c0 1 > prepared operation key=ROFH30W3UCAX tag=G3IWF5BY62RL type=del size=3680756 2024-04-07T16:37:53.662 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.668+0000 7f346a3868c0 1 < completed delete operation key=8F81G0P0YBFW tag=WH0EOSDGH03S type=del 2024-04-07T16:37:53.667 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.676+0000 7f346a3868c0 1 > prepared operation key=HBLEXONKXJTL tag=WG8DC33VZY6L type=del size=1835210 2024-04-07T16:37:53.672 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.680+0000 7f346a3868c0 1 < completed delete operation key=95YVKJ14JD8W tag=M3AHCLWHU97P type=del 2024-04-07T16:37:53.676 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.684+0000 7f346a3868c0 1 > prepared operation key=X5Q46GO8YTOK tag=O1QVF117R0M3 type=del size=3214311 2024-04-07T16:37:53.681 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.688+0000 7f346a3868c0 1 < completed part key=_multipart_GZXHYZO5SMUP.2~MB4JUAPKF97L.0 size=1048576 2024-04-07T16:37:53.685 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.692+0000 7f346a3868c0 1 < completed part key=_multipart_GZXHYZO5SMUP.2~MB4JUAPKF97L.1 size=426024 2024-04-07T16:37:53.693 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.700+0000 7f346a3868c0 1 < completed multipart upload key=GZXHYZO5SMUP upload=MB4JUAPKF97L tag=0JNN7HU6G7KH type=write size=2523176 2024-04-07T16:37:53.702 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.708+0000 7f346a3868c0 1 > prepared part key=_multipart_HBLEXONKXJTL.2~VK6AHRM23XMV.0 size=1048576 2024-04-07T16:37:53.705 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.712+0000 7f346a3868c0 1 > prepared part key=_multipart_HBLEXONKXJTL.2~VK6AHRM23XMV.1 size=1048576 2024-04-07T16:37:53.709 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.716+0000 7f346a3868c0 1 > prepared part key=_multipart_HBLEXONKXJTL.2~VK6AHRM23XMV.2 size=832159 2024-04-07T16:37:53.709 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.716+0000 7f346a3868c0 1 > prepared multipart upload key=HBLEXONKXJTL upload=VK6AHRM23XMV tag=VYOCHL9VCQR0 type=write size=3977887 2024-04-07T16:37:53.714 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.720+0000 7f346a3868c0 1 < completed part key=_multipart_UIW3DN4NUYPP.2~LEBB7JU0FHKC.0 size=902546 2024-04-07T16:37:53.722 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.728+0000 7f346a3868c0 1 < completed multipart upload key=UIW3DN4NUYPP upload=LEBB7JU0FHKC tag=NWFLAZQVPXY8 type=write size=1951122 2024-04-07T16:37:53.731 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.740+0000 7f346a3868c0 1 > prepared part key=_multipart_GZXHYZO5SMUP.2~5GRMXV7WYM4P.0 size=1048576 2024-04-07T16:37:53.735 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.744+0000 7f346a3868c0 1 > prepared part key=_multipart_GZXHYZO5SMUP.2~5GRMXV7WYM4P.1 size=1048576 2024-04-07T16:37:53.739 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.748+0000 7f346a3868c0 1 > prepared part key=_multipart_GZXHYZO5SMUP.2~5GRMXV7WYM4P.2 size=534057 2024-04-07T16:37:53.739 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.748+0000 7f346a3868c0 1 > prepared multipart upload key=GZXHYZO5SMUP upload=5GRMXV7WYM4P tag=19R8A53QZ9D2 type=write size=3679785 2024-04-07T16:37:53.745 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.752+0000 7f346a3868c0 1 < completed part key=_multipart_N14BUXLJONHZ.2~H2N15A0LM714.0 size=120702 2024-04-07T16:37:53.753 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.760+0000 7f346a3868c0 1 < completed multipart upload key=N14BUXLJONHZ upload=H2N15A0LM714 tag=5BORYOQV2JNI type=write size=1169278 2024-04-07T16:37:53.758 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.764+0000 7f346a3868c0 1 > prepared operation key=DXTO3AJ429AC tag=VSKFQM5C3SJ0 type=del size=696226 2024-04-07T16:37:53.762 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.768+0000 7f346a3868c0 1 < completed part key=_multipart_C1BTVJNHIM0X.2~UZ1SBEIVF04T.0 size=783339 2024-04-07T16:37:53.771 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.780+0000 7f346a3868c0 1 < completed multipart upload key=C1BTVJNHIM0X upload=UZ1SBEIVF04T tag=VWXZVLJ7PSYE type=write size=1831915 2024-04-07T16:37:53.780 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.788+0000 7f346a3868c0 1 > prepared part key=_multipart_X5Q46GO8YTOK.2~ISFOZ6YM0EOM.0 size=858608 2024-04-07T16:37:53.780 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.788+0000 7f346a3868c0 1 > prepared multipart upload key=X5Q46GO8YTOK upload=ISFOZ6YM0EOM tag=2JAQMV7NR1QE type=write size=1907184 2024-04-07T16:37:53.784 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.792+0000 7f346a3868c0 1 < completed part key=_multipart_IP8YW2A75TCW.2~753PL3FP720Q.0 size=803741 2024-04-07T16:37:53.792 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.800+0000 7f346a3868c0 1 < completed multipart upload key=IP8YW2A75TCW upload=753PL3FP720Q tag=FJ8ORY4UIV1T type=write size=1852317 2024-04-07T16:37:53.796 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.804+0000 7f346a3868c0 1 > prepared operation key=DXTO3AJ429AC tag=V9Q1MAWZCTEJ type=del size=2129894 2024-04-07T16:37:53.801 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.808+0000 7f346a3868c0 1 < canceled operation key=XJLQZ7IS9U1T tag=VQCRMV3TDAYR type=write size=139742 2024-04-07T16:37:53.805 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.812+0000 7f346a3868c0 1 > prepared operation key=LXOZ0T2KY6OR tag=EVRTSZVF9NFM type=del size=1529712 2024-04-07T16:37:53.810 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.816+0000 7f346a3868c0 1 < completed write operation key=ROFH30W3UCAX tag=3P8EZ9ORV4FD type=write size=49194 2024-04-07T16:37:53.819 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.824+0000 7f346a3868c0 1 > prepared part key=_multipart_QS4HYZFFAJNF.2~RVBHH6QRU1GZ.0 size=42361 2024-04-07T16:37:53.819 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.824+0000 7f346a3868c0 1 > prepared multipart upload key=QS4HYZFFAJNF upload=RVBHH6QRU1GZ tag=PZFSPJ01HYNC type=write size=1090937 2024-04-07T16:37:53.824 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.832+0000 7f346a3868c0 1 < completed delete operation key=8F81G0P0YBFW tag=8OZ7PTMQFHQ2 type=del 2024-04-07T16:37:53.829 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.836+0000 7f346a3868c0 1 > prepared operation key=LXOZ0T2KY6OR tag=0KTOGZ3TU3GH type=write size=254522 2024-04-07T16:37:53.833 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.840+0000 7f346a3868c0 1 < completed part key=_multipart_U3EID5EK64HD.2~73MAKVR90MC7.0 size=1048576 2024-04-07T16:37:53.837 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.844+0000 7f346a3868c0 1 < completed part key=_multipart_U3EID5EK64HD.2~73MAKVR90MC7.1 size=1048576 2024-04-07T16:37:53.841 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.848+0000 7f346a3868c0 1 < completed part key=_multipart_U3EID5EK64HD.2~73MAKVR90MC7.2 size=743340 2024-04-07T16:37:53.849 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.856+0000 7f346a3868c0 1 < completed multipart upload key=U3EID5EK64HD upload=73MAKVR90MC7 tag=WTSM96DY3I3C type=write size=3889068 2024-04-07T16:37:53.853 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.860+0000 7f346a3868c0 1 > prepared operation key=GBQSME7SY1AH tag=QB0SHOQ73L8X type=write size=958845 2024-04-07T16:37:53.856 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:37:53 smithi144 bash[66654]: cluster 2024-04-07T16:37:53.060604+0000 mgr.y (mgr.25175) 676 : cluster [DBG] pgmap v1311: 137 pgs: 137 active+clean; 583 KiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 545 KiB/s rd, 189 KiB/s wr, 963 op/s 2024-04-07T16:37:53.858 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.864+0000 7f346a3868c0 1 < completed delete operation key=EQR0FY86AT5X tag=08ITSM1C5POG type=del 2024-04-07T16:37:53.867 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.872+0000 7f346a3868c0 1 > prepared part key=_multipart_7G5C6HBK6QWI.2~QML3LSR4O0LO.0 size=1048576 2024-04-07T16:37:53.871 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.876+0000 7f346a3868c0 1 > prepared part key=_multipart_7G5C6HBK6QWI.2~QML3LSR4O0LO.1 size=1048576 2024-04-07T16:37:53.874 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.880+0000 7f346a3868c0 1 > prepared part key=_multipart_7G5C6HBK6QWI.2~QML3LSR4O0LO.2 size=81161 2024-04-07T16:37:53.875 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.880+0000 7f346a3868c0 1 > prepared multipart upload key=7G5C6HBK6QWI upload=QML3LSR4O0LO tag=QLLN13CZUXYT type=write size=3226889 2024-04-07T16:37:53.885 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.892+0000 7f346a3868c0 1 < suggested remove operation key=IP8YW2A75TCW tag=S29NHYKHJRRA type=del 2024-04-07T16:37:53.890 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.896+0000 7f346a3868c0 1 > prepared operation key=3N0QJRYVO3FV tag=290YGIU6V6TW type=write size=567729 2024-04-07T16:37:53.895 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.904+0000 7f346a3868c0 1 < completed part key=_multipart_IP8YW2A75TCW.2~LJFF76ZF7QT4.0 size=1048576 2024-04-07T16:37:53.899 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.908+0000 7f346a3868c0 1 < completed part key=_multipart_IP8YW2A75TCW.2~LJFF76ZF7QT4.1 size=704028 2024-04-07T16:37:53.907 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.916+0000 7f346a3868c0 1 < completed multipart upload key=IP8YW2A75TCW upload=LJFF76ZF7QT4 tag=GZ5V68E7O3DA type=write size=2801180 2024-04-07T16:37:53.916 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.924+0000 7f346a3868c0 1 > prepared part key=_multipart_QVOKSQAGHPA9.2~BQ01PSPE4GFE.0 size=1023233 2024-04-07T16:37:53.916 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.924+0000 7f346a3868c0 1 > prepared multipart upload key=QVOKSQAGHPA9 upload=BQ01PSPE4GFE tag=W097XM31UAII type=write size=2071809 2024-04-07T16:37:53.921 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.928+0000 7f346a3868c0 1 < completed part key=_multipart_U3EID5EK64HD.2~F53FFNQT5Z3W.0 size=456672 2024-04-07T16:37:53.930 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.936+0000 7f346a3868c0 1 < completed multipart upload key=U3EID5EK64HD upload=F53FFNQT5Z3W tag=XXUQYE122JZM type=write size=1505248 2024-04-07T16:37:53.935 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.944+0000 7f346a3868c0 1 > prepared operation key=N14BUXLJONHZ tag=SZGC6N0YE0QZ type=del size=2084964 2024-04-07T16:37:53.941 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.948+0000 7f346a3868c0 1 < completed delete operation key=ROFH30W3UCAX tag=G3IWF5BY62RL type=del 2024-04-07T16:37:53.942 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:37:53 smithi100 bash[74942]: cluster 2024-04-07T16:37:53.060604+0000 mgr.y (mgr.25175) 676 : cluster [DBG] pgmap v1311: 137 pgs: 137 active+clean; 583 KiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 545 KiB/s rd, 189 KiB/s wr, 963 op/s 2024-04-07T16:37:53.942 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:37:53 smithi100 bash[76202]: cluster 2024-04-07T16:37:53.060604+0000 mgr.y (mgr.25175) 676 : cluster [DBG] pgmap v1311: 137 pgs: 137 active+clean; 583 KiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 545 KiB/s rd, 189 KiB/s wr, 963 op/s 2024-04-07T16:37:53.946 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.952+0000 7f346a3868c0 1 > prepared operation key=EQR0FY86AT5X tag=TADTIKNK7H20 type=write size=185433 2024-04-07T16:37:53.950 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.956+0000 7f346a3868c0 1 < completed delete operation key=HBLEXONKXJTL tag=WG8DC33VZY6L type=del 2024-04-07T16:37:53.959 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.968+0000 7f346a3868c0 1 > prepared part key=_multipart_IP8YW2A75TCW.2~N8JQLA5ZTJ1M.0 size=678691 2024-04-07T16:37:53.960 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.968+0000 7f346a3868c0 1 > prepared multipart upload key=IP8YW2A75TCW upload=N8JQLA5ZTJ1M tag=8QZZKBGCS0I7 type=write size=1727267 2024-04-07T16:37:53.965 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.972+0000 7f346a3868c0 1 < completed delete operation key=X5Q46GO8YTOK tag=O1QVF117R0M3 type=del 2024-04-07T16:37:53.970 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.976+0000 7f346a3868c0 1 > prepared operation key=LXOZ0T2KY6OR tag=A0MUQD9ED2WA type=del size=4087740 2024-04-07T16:37:53.974 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.980+0000 7f346a3868c0 1 < completed part key=_multipart_HBLEXONKXJTL.2~VK6AHRM23XMV.0 size=1048576 2024-04-07T16:37:53.974 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.980+0000 7f346a3868c0 1 < canceled part key=_multipart_HBLEXONKXJTL.2~VK6AHRM23XMV.1 size=1048576 2024-04-07T16:37:53.978 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.984+0000 7f346a3868c0 1 < completed part key=_multipart_HBLEXONKXJTL.2~VK6AHRM23XMV.2 size=832159 2024-04-07T16:37:53.987 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:53.992+0000 7f346a3868c0 1 < canceled multipart upload key=HBLEXONKXJTL upload=VK6AHRM23XMV tag=VYOCHL9VCQR0 type=write size=3977887 2024-04-07T16:37:53.996 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.004+0000 7f346a3868c0 1 > prepared part key=_multipart_LXOZ0T2KY6OR.2~RZJVNVBZQVCT.0 size=1048576 2024-04-07T16:37:54.000 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.008+0000 7f346a3868c0 1 > prepared part key=_multipart_LXOZ0T2KY6OR.2~RZJVNVBZQVCT.1 size=1048576 2024-04-07T16:37:54.004 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.012+0000 7f346a3868c0 1 > prepared part key=_multipart_LXOZ0T2KY6OR.2~RZJVNVBZQVCT.2 size=861598 2024-04-07T16:37:54.004 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.012+0000 7f346a3868c0 1 > prepared multipart upload key=LXOZ0T2KY6OR upload=RZJVNVBZQVCT tag=0XU9XIF135XK type=write size=4007326 2024-04-07T16:37:54.009 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.016+0000 7f346a3868c0 1 < completed part key=_multipart_GZXHYZO5SMUP.2~5GRMXV7WYM4P.0 size=1048576 2024-04-07T16:37:54.013 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.020+0000 7f346a3868c0 1 < completed part key=_multipart_GZXHYZO5SMUP.2~5GRMXV7WYM4P.1 size=1048576 2024-04-07T16:37:54.017 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.024+0000 7f346a3868c0 1 < completed part key=_multipart_GZXHYZO5SMUP.2~5GRMXV7WYM4P.2 size=534057 2024-04-07T16:37:54.025 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.032+0000 7f346a3868c0 1 < completed multipart upload key=GZXHYZO5SMUP upload=5GRMXV7WYM4P tag=19R8A53QZ9D2 type=write size=3679785 2024-04-07T16:37:54.034 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.040+0000 7f346a3868c0 1 > prepared part key=_multipart_IX3F0IUOKBX1.2~8V4MIUNXXOAE.0 size=1048576 2024-04-07T16:37:54.038 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.044+0000 7f346a3868c0 1 > prepared part key=_multipart_IX3F0IUOKBX1.2~8V4MIUNXXOAE.1 size=779864 2024-04-07T16:37:54.038 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.044+0000 7f346a3868c0 1 > prepared multipart upload key=IX3F0IUOKBX1 upload=8V4MIUNXXOAE tag=YIQUDNJ2USC3 type=write size=2877016 2024-04-07T16:37:54.043 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.052+0000 7f346a3868c0 1 < completed delete operation key=DXTO3AJ429AC tag=VSKFQM5C3SJ0 type=del 2024-04-07T16:37:54.053 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.060+0000 7f346a3868c0 1 > prepared part key=_multipart_U3EID5EK64HD.2~QRWHFAXY19JD.0 size=1027616 2024-04-07T16:37:54.053 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.060+0000 7f346a3868c0 1 > prepared multipart upload key=U3EID5EK64HD upload=QRWHFAXY19JD tag=C5ALKWMXP9DF type=write size=2076192 2024-04-07T16:37:54.114 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.120+0000 7f346a3868c0 1 < completed part key=_multipart_X5Q46GO8YTOK.2~ISFOZ6YM0EOM.0 size=858608 2024-04-07T16:37:54.124 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.132+0000 7f346a3868c0 1 < completed multipart upload key=X5Q46GO8YTOK upload=ISFOZ6YM0EOM tag=2JAQMV7NR1QE type=write size=1907184 2024-04-07T16:37:54.135 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.144+0000 7f346a3868c0 1 > prepared part key=_multipart_NYET3212USVE.2~364E3VGLY3MP.0 size=1048576 2024-04-07T16:37:54.139 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.148+0000 7f346a3868c0 1 > prepared part key=_multipart_NYET3212USVE.2~364E3VGLY3MP.1 size=1048576 2024-04-07T16:37:54.143 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.152+0000 7f346a3868c0 1 > prepared part key=_multipart_NYET3212USVE.2~364E3VGLY3MP.2 size=175219 2024-04-07T16:37:54.144 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.152+0000 7f346a3868c0 1 > prepared multipart upload key=NYET3212USVE upload=364E3VGLY3MP tag=FS01V8KFAEK5 type=write size=3320947 2024-04-07T16:37:54.149 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.156+0000 7f346a3868c0 1 < completed delete operation key=DXTO3AJ429AC tag=V9Q1MAWZCTEJ type=del 2024-04-07T16:37:54.155 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.164+0000 7f346a3868c0 1 > prepared operation key=1VKVQXSQMTU1 tag=0NGDCB4ZVG23 type=write size=600485 2024-04-07T16:37:54.161 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.168+0000 7f346a3868c0 1 < completed delete operation key=LXOZ0T2KY6OR tag=EVRTSZVF9NFM type=del 2024-04-07T16:37:54.166 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.172+0000 7f346a3868c0 1 > prepared operation key=IX3F0IUOKBX1 tag=E0BH68Z5Z66P type=del size=3518797 2024-04-07T16:37:54.171 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.180+0000 7f346a3868c0 1 < completed part key=_multipart_QS4HYZFFAJNF.2~RVBHH6QRU1GZ.0 size=42361 2024-04-07T16:37:54.180 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.188+0000 7f346a3868c0 1 < completed multipart upload key=QS4HYZFFAJNF upload=RVBHH6QRU1GZ tag=PZFSPJ01HYNC type=write size=1090937 2024-04-07T16:37:54.185 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.192+0000 7f346a3868c0 1 > prepared operation key=NYET3212USVE tag=ZMQS2WXNA6PI type=del size=232749 2024-04-07T16:37:54.190 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.196+0000 7f346a3868c0 1 < completed write operation key=LXOZ0T2KY6OR tag=0KTOGZ3TU3GH type=write size=254522 2024-04-07T16:37:54.196 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.204+0000 7f346a3868c0 1 > prepared operation key=QVOKSQAGHPA9 tag=A9G2H8EG2V4U type=del size=957729 2024-04-07T16:37:54.205 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.212+0000 7f346a3868c0 1 < suggested remove operation key=GBQSME7SY1AH tag=QB0SHOQ73L8X type=write 2024-04-07T16:37:54.210 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.216+0000 7f346a3868c0 1 > prepared operation key=QS4HYZFFAJNF tag=4H40G8HGD4UU type=write size=809619 2024-04-07T16:37:54.216 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.224+0000 7f346a3868c0 1 < completed part key=_multipart_7G5C6HBK6QWI.2~QML3LSR4O0LO.0 size=1048576 2024-04-07T16:37:54.220 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.228+0000 7f346a3868c0 1 < completed part key=_multipart_7G5C6HBK6QWI.2~QML3LSR4O0LO.1 size=1048576 2024-04-07T16:37:54.224 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.232+0000 7f346a3868c0 1 < completed part key=_multipart_7G5C6HBK6QWI.2~QML3LSR4O0LO.2 size=81161 2024-04-07T16:37:54.233 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.240+0000 7f346a3868c0 1 < completed multipart upload key=7G5C6HBK6QWI upload=QML3LSR4O0LO tag=QLLN13CZUXYT type=write size=3226889 2024-04-07T16:37:54.239 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.248+0000 7f346a3868c0 1 > prepared operation key=CL4PP1RKZU0E tag=SFHD6181B9IT type=del size=1391940 2024-04-07T16:37:54.245 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.252+0000 7f346a3868c0 1 < completed write operation key=3N0QJRYVO3FV tag=290YGIU6V6TW type=write size=567729 2024-04-07T16:37:54.255 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.264+0000 7f346a3868c0 1 > prepared part key=_multipart_1WGUPH7X5UA0.2~1J732LK1AAR5.0 size=1048576 2024-04-07T16:37:54.259 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.268+0000 7f346a3868c0 1 > prepared part key=_multipart_1WGUPH7X5UA0.2~1J732LK1AAR5.1 size=540767 2024-04-07T16:37:54.259 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.268+0000 7f346a3868c0 1 > prepared multipart upload key=1WGUPH7X5UA0 upload=1J732LK1AAR5 tag=8BDPQDQM1A2O type=write size=2637919 2024-04-07T16:37:54.265 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.272+0000 7f346a3868c0 1 < completed part key=_multipart_QVOKSQAGHPA9.2~BQ01PSPE4GFE.0 size=1023233 2024-04-07T16:37:54.275 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.280+0000 7f346a3868c0 1 < completed multipart upload key=QVOKSQAGHPA9 upload=BQ01PSPE4GFE tag=W097XM31UAII type=write size=2071809 2024-04-07T16:37:54.284 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.292+0000 7f346a3868c0 1 > prepared part key=_multipart_EQR0FY86AT5X.2~HF4FM47WS0J0.0 size=523600 2024-04-07T16:37:54.284 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.292+0000 7f346a3868c0 1 > prepared multipart upload key=EQR0FY86AT5X upload=HF4FM47WS0J0 tag=XZT14HUS1Y8L type=write size=1572176 2024-04-07T16:37:54.289 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.296+0000 7f346a3868c0 1 < completed delete operation key=N14BUXLJONHZ tag=SZGC6N0YE0QZ type=del 2024-04-07T16:37:54.293 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.300+0000 7f346a3868c0 1 > prepared operation key=1WGUPH7X5UA0 tag=WI0S5C8VUDR9 type=write size=620291 2024-04-07T16:37:54.298 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.304+0000 7f346a3868c0 1 < completed write operation key=EQR0FY86AT5X tag=TADTIKNK7H20 type=write size=185433 2024-04-07T16:37:54.307 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.312+0000 7f346a3868c0 1 > prepared part key=_multipart_DXTO3AJ429AC.2~2EPZI56TACOT.0 size=668314 2024-04-07T16:37:54.307 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.312+0000 7f346a3868c0 1 > prepared multipart upload key=DXTO3AJ429AC upload=2EPZI56TACOT tag=M5VRSO5XI6UF type=write size=1716890 2024-04-07T16:37:54.312 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.320+0000 7f346a3868c0 1 < completed part key=_multipart_IP8YW2A75TCW.2~N8JQLA5ZTJ1M.0 size=678691 2024-04-07T16:37:54.320 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.328+0000 7f346a3868c0 1 < completed multipart upload key=IP8YW2A75TCW upload=N8JQLA5ZTJ1M tag=8QZZKBGCS0I7 type=write size=1727267 2024-04-07T16:37:54.329 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.336+0000 7f346a3868c0 1 > prepared part key=_multipart_EOYFVEI50SQ1.2~8ZJPCTYJDYVW.0 size=440993 2024-04-07T16:37:54.330 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.336+0000 7f346a3868c0 1 > prepared multipart upload key=EOYFVEI50SQ1 upload=8ZJPCTYJDYVW tag=CJNG5LGEVOR0 type=write size=1489569 2024-04-07T16:37:54.340 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.348+0000 7f346a3868c0 1 < suggested update operation key=LXOZ0T2KY6OR tag=A0MUQD9ED2WA type=del size=4087740 2024-04-07T16:37:54.351 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.360+0000 7f346a3868c0 1 > prepared part key=_multipart_XJLQZ7IS9U1T.2~B6KJPDATAOWG.0 size=806331 2024-04-07T16:37:54.351 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.360+0000 7f346a3868c0 1 > prepared multipart upload key=XJLQZ7IS9U1T upload=B6KJPDATAOWG tag=IDLPR1779NZS type=write size=1854907 2024-04-07T16:37:54.357 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.364+0000 7f346a3868c0 1 < completed part key=_multipart_LXOZ0T2KY6OR.2~RZJVNVBZQVCT.0 size=1048576 2024-04-07T16:37:54.361 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.368+0000 7f346a3868c0 1 < completed part key=_multipart_LXOZ0T2KY6OR.2~RZJVNVBZQVCT.1 size=1048576 2024-04-07T16:37:54.366 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.372+0000 7f346a3868c0 1 < completed part key=_multipart_LXOZ0T2KY6OR.2~RZJVNVBZQVCT.2 size=861598 2024-04-07T16:37:54.374 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.380+0000 7f346a3868c0 1 < completed multipart upload key=LXOZ0T2KY6OR upload=RZJVNVBZQVCT tag=0XU9XIF135XK type=write size=4007326 2024-04-07T16:37:54.380 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.388+0000 7f346a3868c0 1 > prepared operation key=1VKVQXSQMTU1 tag=Z2ADIUN5U68P type=write size=735514 2024-04-07T16:37:54.386 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.392+0000 7f346a3868c0 1 < completed part key=_multipart_IX3F0IUOKBX1.2~8V4MIUNXXOAE.0 size=1048576 2024-04-07T16:37:54.390 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.396+0000 7f346a3868c0 1 < completed part key=_multipart_IX3F0IUOKBX1.2~8V4MIUNXXOAE.1 size=779864 2024-04-07T16:37:54.399 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.408+0000 7f346a3868c0 1 < completed multipart upload key=IX3F0IUOKBX1 upload=8V4MIUNXXOAE tag=YIQUDNJ2USC3 type=write size=2877016 2024-04-07T16:37:54.409 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.416+0000 7f346a3868c0 1 > prepared part key=_multipart_UIW3DN4NUYPP.2~BEBSQF7YYDXD.0 size=1048576 2024-04-07T16:37:54.414 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.420+0000 7f346a3868c0 1 > prepared part key=_multipart_UIW3DN4NUYPP.2~BEBSQF7YYDXD.1 size=1048576 2024-04-07T16:37:54.418 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.424+0000 7f346a3868c0 1 > prepared part key=_multipart_UIW3DN4NUYPP.2~BEBSQF7YYDXD.2 size=663540 2024-04-07T16:37:54.418 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.424+0000 7f346a3868c0 1 > prepared multipart upload key=UIW3DN4NUYPP upload=BEBSQF7YYDXD tag=GNH53MK56701 type=write size=3809268 2024-04-07T16:37:54.423 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.428+0000 7f346a3868c0 1 < completed part key=_multipart_U3EID5EK64HD.2~QRWHFAXY19JD.0 size=1027616 2024-04-07T16:37:54.431 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.440+0000 7f346a3868c0 1 < completed multipart upload key=U3EID5EK64HD upload=QRWHFAXY19JD tag=C5ALKWMXP9DF type=write size=2076192 2024-04-07T16:37:54.437 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.444+0000 7f346a3868c0 1 > prepared operation key=YXMDZB23YTB3 tag=E4KE2T0U6YW0 type=del size=458803 2024-04-07T16:37:54.442 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.448+0000 7f346a3868c0 1 < completed part key=_multipart_NYET3212USVE.2~364E3VGLY3MP.0 size=1048576 2024-04-07T16:37:54.446 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.452+0000 7f346a3868c0 1 < completed part key=_multipart_NYET3212USVE.2~364E3VGLY3MP.1 size=1048576 2024-04-07T16:37:54.450 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.456+0000 7f346a3868c0 1 < completed part key=_multipart_NYET3212USVE.2~364E3VGLY3MP.2 size=175219 2024-04-07T16:37:54.458 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.464+0000 7f346a3868c0 1 < completed multipart upload key=NYET3212USVE upload=364E3VGLY3MP tag=FS01V8KFAEK5 type=write size=3320947 2024-04-07T16:37:54.463 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.472+0000 7f346a3868c0 1 > prepared operation key=1VKVQXSQMTU1 tag=2NFHFK1V4VGF type=del size=2125799 2024-04-07T16:37:54.469 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.476+0000 7f346a3868c0 1 < completed write operation key=1VKVQXSQMTU1 tag=0NGDCB4ZVG23 type=write size=600485 2024-04-07T16:37:54.475 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.480+0000 7f346a3868c0 1 > prepared operation key=QVOKSQAGHPA9 tag=4J2JH4Y4I6RC type=del size=1741791 2024-04-07T16:37:54.481 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.488+0000 7f346a3868c0 1 < completed delete operation key=IX3F0IUOKBX1 tag=E0BH68Z5Z66P type=del 2024-04-07T16:37:54.485 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.492+0000 7f346a3868c0 1 > prepared operation key=QS4HYZFFAJNF tag=GIESF9YI7X45 type=del size=2162880 2024-04-07T16:37:54.490 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.496+0000 7f346a3868c0 1 < completed delete operation key=NYET3212USVE tag=ZMQS2WXNA6PI type=del 2024-04-07T16:37:54.495 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.504+0000 7f346a3868c0 1 > prepared operation key=CL4PP1RKZU0E tag=SCH6RK4X0XYR type=del size=1839915 2024-04-07T16:37:54.500 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.508+0000 7f346a3868c0 1 < completed delete operation key=QVOKSQAGHPA9 tag=A9G2H8EG2V4U type=del 2024-04-07T16:37:54.505 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.512+0000 7f346a3868c0 1 > prepared operation key=YOK8BAEOMPUR tag=VZASX0E919S6 type=del size=3089774 2024-04-07T16:37:54.510 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.516+0000 7f346a3868c0 1 < completed write operation key=QS4HYZFFAJNF tag=4H40G8HGD4UU type=write size=809619 2024-04-07T16:37:54.515 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.524+0000 7f346a3868c0 1 > prepared operation key=95YVKJ14JD8W tag=6SS226GKKH08 type=del size=1331399 2024-04-07T16:37:54.520 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.528+0000 7f346a3868c0 1 < completed delete operation key=CL4PP1RKZU0E tag=SFHD6181B9IT type=del 2024-04-07T16:37:54.529 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.536+0000 7f346a3868c0 1 > prepared part key=_multipart_E84U0SX44NQ8.2~VYZG6B8OQCLF.0 size=1045347 2024-04-07T16:37:54.529 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.536+0000 7f346a3868c0 1 > prepared multipart upload key=E84U0SX44NQ8 upload=VYZG6B8OQCLF tag=3FX368REI3X1 type=write size=2093923 2024-04-07T16:37:54.534 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.540+0000 7f346a3868c0 1 < completed part key=_multipart_1WGUPH7X5UA0.2~1J732LK1AAR5.0 size=1048576 2024-04-07T16:37:54.538 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.544+0000 7f346a3868c0 1 < completed part key=_multipart_1WGUPH7X5UA0.2~1J732LK1AAR5.1 size=540767 2024-04-07T16:37:54.546 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.552+0000 7f346a3868c0 1 < completed multipart upload key=1WGUPH7X5UA0 upload=1J732LK1AAR5 tag=8BDPQDQM1A2O type=write size=2637919 2024-04-07T16:37:54.551 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.560+0000 7f346a3868c0 1 > prepared operation key=1VBSUWNI3V1T tag=ZS1U9087JVTH type=del size=160062 2024-04-07T16:37:54.556 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.564+0000 7f346a3868c0 1 < completed part key=_multipart_EQR0FY86AT5X.2~HF4FM47WS0J0.0 size=523600 2024-04-07T16:37:54.564 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.572+0000 7f346a3868c0 1 < completed multipart upload key=EQR0FY86AT5X upload=HF4FM47WS0J0 tag=XZT14HUS1Y8L type=write size=1572176 2024-04-07T16:37:54.573 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.580+0000 7f346a3868c0 1 > prepared part key=_multipart_LXOZ0T2KY6OR.2~9TIQ34BCPQZ8.0 size=1048576 2024-04-07T16:37:54.577 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.584+0000 7f346a3868c0 1 > prepared part key=_multipart_LXOZ0T2KY6OR.2~9TIQ34BCPQZ8.1 size=1048576 2024-04-07T16:37:54.580 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.588+0000 7f346a3868c0 1 > prepared part key=_multipart_LXOZ0T2KY6OR.2~9TIQ34BCPQZ8.2 size=527520 2024-04-07T16:37:54.581 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.588+0000 7f346a3868c0 1 > prepared multipart upload key=LXOZ0T2KY6OR upload=9TIQ34BCPQZ8 tag=TX87B1OC0H2N type=write size=3673248 2024-04-07T16:37:54.585 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.592+0000 7f346a3868c0 1 < completed write operation key=1WGUPH7X5UA0 tag=WI0S5C8VUDR9 type=write size=620291 2024-04-07T16:37:54.594 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.600+0000 7f346a3868c0 1 > prepared part key=_multipart_GBQSME7SY1AH.2~LFXNYCEQG9U0.0 size=1048576 2024-04-07T16:37:54.597 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.604+0000 7f346a3868c0 1 > prepared part key=_multipart_GBQSME7SY1AH.2~LFXNYCEQG9U0.1 size=1048576 2024-04-07T16:37:54.601 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.608+0000 7f346a3868c0 1 > prepared part key=_multipart_GBQSME7SY1AH.2~LFXNYCEQG9U0.2 size=500517 2024-04-07T16:37:54.601 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.608+0000 7f346a3868c0 1 > prepared multipart upload key=GBQSME7SY1AH upload=LFXNYCEQG9U0 tag=8ZEJ6QPE5VAO type=write size=3646245 2024-04-07T16:37:54.606 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.612+0000 7f346a3868c0 1 < completed part key=_multipart_DXTO3AJ429AC.2~2EPZI56TACOT.0 size=668314 2024-04-07T16:37:54.615 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.620+0000 7f346a3868c0 1 < completed multipart upload key=DXTO3AJ429AC upload=2EPZI56TACOT tag=M5VRSO5XI6UF type=write size=1716890 2024-04-07T16:37:54.620 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.628+0000 7f346a3868c0 1 > prepared operation key=DXTO3AJ429AC tag=6H58FIYM0Y9G type=del size=2965970 2024-04-07T16:37:54.625 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.632+0000 7f346a3868c0 1 < completed part key=_multipart_EOYFVEI50SQ1.2~8ZJPCTYJDYVW.0 size=440993 2024-04-07T16:37:54.633 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.640+0000 7f346a3868c0 1 < completed multipart upload key=EOYFVEI50SQ1 upload=8ZJPCTYJDYVW tag=CJNG5LGEVOR0 type=write size=1489569 2024-04-07T16:37:54.638 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.644+0000 7f346a3868c0 1 > prepared operation key=EOYFVEI50SQ1 tag=LIGULPKX6OI3 type=del size=1464421 2024-04-07T16:37:54.644 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.652+0000 7f346a3868c0 1 < completed part key=_multipart_XJLQZ7IS9U1T.2~B6KJPDATAOWG.0 size=806331 2024-04-07T16:37:54.652 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.660+0000 7f346a3868c0 1 < completed multipart upload key=XJLQZ7IS9U1T upload=B6KJPDATAOWG tag=IDLPR1779NZS type=write size=1854907 2024-04-07T16:37:54.657 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.664+0000 7f346a3868c0 1 > prepared operation key=QS4HYZFFAJNF tag=5EGTVXCC9RBO type=del size=1937549 2024-04-07T16:37:54.663 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.668+0000 7f346a3868c0 1 < completed write operation key=1VKVQXSQMTU1 tag=Z2ADIUN5U68P type=write size=735514 2024-04-07T16:37:54.668 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.676+0000 7f346a3868c0 1 > prepared operation key=YXMDZB23YTB3 tag=EH63TVMHJDSQ type=write size=446599 2024-04-07T16:37:54.669 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.676+0000 7f346a3868c0 1 < canceled part key=_multipart_UIW3DN4NUYPP.2~BEBSQF7YYDXD.0 size=1048576 2024-04-07T16:37:54.672 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.680+0000 7f346a3868c0 1 < completed part key=_multipart_UIW3DN4NUYPP.2~BEBSQF7YYDXD.1 size=1048576 2024-04-07T16:37:54.676 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.684+0000 7f346a3868c0 1 < completed part key=_multipart_UIW3DN4NUYPP.2~BEBSQF7YYDXD.2 size=663540 2024-04-07T16:37:54.684 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.692+0000 7f346a3868c0 1 < canceled multipart upload key=UIW3DN4NUYPP upload=BEBSQF7YYDXD tag=GNH53MK56701 type=write size=3809268 2024-04-07T16:37:54.694 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.700+0000 7f346a3868c0 1 > prepared part key=_multipart_HBLEXONKXJTL.2~7ZZN28PHHNZH.0 size=1048576 2024-04-07T16:37:54.697 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.704+0000 7f346a3868c0 1 > prepared part key=_multipart_HBLEXONKXJTL.2~7ZZN28PHHNZH.1 size=1032861 2024-04-07T16:37:54.697 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.704+0000 7f346a3868c0 1 > prepared multipart upload key=HBLEXONKXJTL upload=7ZZN28PHHNZH tag=DJPH8R4QL0MS type=write size=3130013 2024-04-07T16:37:54.702 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.708+0000 7f346a3868c0 1 < completed delete operation key=YXMDZB23YTB3 tag=E4KE2T0U6YW0 type=del 2024-04-07T16:37:54.706 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.712+0000 7f346a3868c0 1 > prepared operation key=1VBSUWNI3V1T tag=V73T1VPMK4HH type=del size=1480250 2024-04-07T16:37:54.711 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.720+0000 7f346a3868c0 1 < completed delete operation key=1VKVQXSQMTU1 tag=2NFHFK1V4VGF type=del 2024-04-07T16:37:54.719 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.728+0000 7f346a3868c0 1 > prepared part key=_multipart_GZXHYZO5SMUP.2~6X2OJ3326VF1.0 size=1048576 2024-04-07T16:37:54.723 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.732+0000 7f346a3868c0 1 > prepared part key=_multipart_GZXHYZO5SMUP.2~6X2OJ3326VF1.1 size=780564 2024-04-07T16:37:54.723 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.732+0000 7f346a3868c0 1 > prepared multipart upload key=GZXHYZO5SMUP upload=6X2OJ3326VF1 tag=2JVP5PT0G349 type=write size=2877716 2024-04-07T16:37:54.728 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.736+0000 7f346a3868c0 1 < completed delete operation key=QVOKSQAGHPA9 tag=4J2JH4Y4I6RC type=del 2024-04-07T16:37:54.736 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.744+0000 7f346a3868c0 1 > prepared part key=_multipart_YXMDZB23YTB3.2~TWT1YM8GSSP0.0 size=1048576 2024-04-07T16:37:54.740 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.748+0000 7f346a3868c0 1 > prepared part key=_multipart_YXMDZB23YTB3.2~TWT1YM8GSSP0.1 size=528653 2024-04-07T16:37:54.740 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.748+0000 7f346a3868c0 1 > prepared multipart upload key=YXMDZB23YTB3 upload=TWT1YM8GSSP0 tag=W5NJJZL4Q276 type=write size=2625805 2024-04-07T16:37:54.745 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.752+0000 7f346a3868c0 1 < completed delete operation key=QS4HYZFFAJNF tag=GIESF9YI7X45 type=del 2024-04-07T16:37:54.753 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.760+0000 7f346a3868c0 1 > prepared part key=_multipart_7G5C6HBK6QWI.2~CARXB93K5OLN.0 size=1048576 2024-04-07T16:37:54.757 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.764+0000 7f346a3868c0 1 > prepared part key=_multipart_7G5C6HBK6QWI.2~CARXB93K5OLN.1 size=795081 2024-04-07T16:37:54.757 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.764+0000 7f346a3868c0 1 > prepared multipart upload key=7G5C6HBK6QWI upload=CARXB93K5OLN tag=HSDP8N498E9B type=write size=2892233 2024-04-07T16:37:54.762 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.768+0000 7f346a3868c0 1 < completed delete operation key=CL4PP1RKZU0E tag=SCH6RK4X0XYR type=del 2024-04-07T16:37:54.767 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.776+0000 7f346a3868c0 1 > prepared operation key=1WGUPH7X5UA0 tag=VWVTBIUR634M type=write size=959558 2024-04-07T16:37:54.772 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.780+0000 7f346a3868c0 1 < completed delete operation key=YOK8BAEOMPUR tag=VZASX0E919S6 type=del 2024-04-07T16:37:54.782 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.788+0000 7f346a3868c0 1 > prepared part key=_multipart_LXOZ0T2KY6OR.2~2Y3Y4E9U6H4L.0 size=1048576 2024-04-07T16:37:54.786 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.792+0000 7f346a3868c0 1 > prepared part key=_multipart_LXOZ0T2KY6OR.2~2Y3Y4E9U6H4L.1 size=1048576 2024-04-07T16:37:54.790 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.796+0000 7f346a3868c0 1 > prepared part key=_multipart_LXOZ0T2KY6OR.2~2Y3Y4E9U6H4L.2 size=251697 2024-04-07T16:37:54.790 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.796+0000 7f346a3868c0 1 > prepared multipart upload key=LXOZ0T2KY6OR upload=2Y3Y4E9U6H4L tag=MR83GVQI68EW type=write size=3397425 2024-04-07T16:37:54.800 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.808+0000 7f346a3868c0 1 < suggested remove operation key=95YVKJ14JD8W tag=6SS226GKKH08 type=del 2024-04-07T16:37:54.804 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.812+0000 7f346a3868c0 1 > prepared operation key=ROFH30W3UCAX tag=53K91AO2LGL7 type=del size=1601810 2024-04-07T16:37:54.806 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.812+0000 7f346a3868c0 1 < canceled part key=_multipart_E84U0SX44NQ8.2~VYZG6B8OQCLF.0 size=1045347 2024-04-07T16:37:54.815 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.820+0000 7f346a3868c0 1 < canceled multipart upload key=E84U0SX44NQ8 upload=VYZG6B8OQCLF tag=3FX368REI3X1 type=write size=2093923 2024-04-07T16:37:54.824 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.832+0000 7f346a3868c0 1 > prepared part key=_multipart_GBQSME7SY1AH.2~U4P7TP3XSFSJ.0 size=1048576 2024-04-07T16:37:54.828 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.836+0000 7f346a3868c0 1 > prepared part key=_multipart_GBQSME7SY1AH.2~U4P7TP3XSFSJ.1 size=486520 2024-04-07T16:37:54.828 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.836+0000 7f346a3868c0 1 > prepared multipart upload key=GBQSME7SY1AH upload=U4P7TP3XSFSJ tag=ULLOE9B9ZFZK type=write size=2583672 2024-04-07T16:37:54.834 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.840+0000 7f346a3868c0 1 < completed delete operation key=1VBSUWNI3V1T tag=ZS1U9087JVTH type=del 2024-04-07T16:37:54.839 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.848+0000 7f346a3868c0 1 > prepared operation key=IX3F0IUOKBX1 tag=6SN86K6Y1TJR type=del size=3517361 2024-04-07T16:37:54.844 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.852+0000 7f346a3868c0 1 < completed part key=_multipart_LXOZ0T2KY6OR.2~9TIQ34BCPQZ8.0 size=1048576 2024-04-07T16:37:54.850 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.856+0000 7f346a3868c0 1 < completed part key=_multipart_LXOZ0T2KY6OR.2~9TIQ34BCPQZ8.1 size=1048576 2024-04-07T16:37:54.854 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.860+0000 7f346a3868c0 1 < completed part key=_multipart_LXOZ0T2KY6OR.2~9TIQ34BCPQZ8.2 size=527520 2024-04-07T16:37:54.863 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.868+0000 7f346a3868c0 1 < completed multipart upload key=LXOZ0T2KY6OR upload=9TIQ34BCPQZ8 tag=TX87B1OC0H2N type=write size=3673248 2024-04-07T16:37:54.868 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.876+0000 7f346a3868c0 1 > prepared operation key=UIW3DN4NUYPP tag=EC8V4GWIIE9A type=del size=3818677 2024-04-07T16:37:54.873 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.880+0000 7f346a3868c0 1 < completed part key=_multipart_GBQSME7SY1AH.2~LFXNYCEQG9U0.0 size=1048576 2024-04-07T16:37:54.877 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.884+0000 7f346a3868c0 1 < completed part key=_multipart_GBQSME7SY1AH.2~LFXNYCEQG9U0.1 size=1048576 2024-04-07T16:37:54.882 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.888+0000 7f346a3868c0 1 < completed part key=_multipart_GBQSME7SY1AH.2~LFXNYCEQG9U0.2 size=500517 2024-04-07T16:37:54.892 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.900+0000 7f346a3868c0 1 < completed multipart upload key=GBQSME7SY1AH upload=LFXNYCEQG9U0 tag=8ZEJ6QPE5VAO type=write size=3646245 2024-04-07T16:37:54.897 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.904+0000 7f346a3868c0 1 > prepared operation key=EOYFVEI50SQ1 tag=0JT6MCAKZIT8 type=del size=2519692 2024-04-07T16:37:54.902 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.908+0000 7f346a3868c0 1 < completed delete operation key=DXTO3AJ429AC tag=6H58FIYM0Y9G type=del 2024-04-07T16:37:54.912 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.920+0000 7f346a3868c0 1 > prepared part key=_multipart_N14BUXLJONHZ.2~UWO4TAYHM2FA.0 size=1048576 2024-04-07T16:37:54.916 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.924+0000 7f346a3868c0 1 > prepared part key=_multipart_N14BUXLJONHZ.2~UWO4TAYHM2FA.1 size=948471 2024-04-07T16:37:54.917 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.924+0000 7f346a3868c0 1 > prepared multipart upload key=N14BUXLJONHZ upload=UWO4TAYHM2FA tag=F74P0P57HC5M type=write size=3045623 2024-04-07T16:37:54.926 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.932+0000 7f346a3868c0 1 < suggested remove operation key=EOYFVEI50SQ1 tag=LIGULPKX6OI3 type=del 2024-04-07T16:37:54.932 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.940+0000 7f346a3868c0 1 > prepared operation key=X5Q46GO8YTOK tag=CCY7ZW8AZO32 type=del size=1503727 2024-04-07T16:37:54.937 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.944+0000 7f346a3868c0 1 < completed delete operation key=QS4HYZFFAJNF tag=5EGTVXCC9RBO type=del 2024-04-07T16:37:54.946 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.952+0000 7f346a3868c0 1 > prepared part key=_multipart_EQR0FY86AT5X.2~HIS6S9BLSBSW.0 size=1048576 2024-04-07T16:37:54.950 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.956+0000 7f346a3868c0 1 > prepared part key=_multipart_EQR0FY86AT5X.2~HIS6S9BLSBSW.1 size=1048576 2024-04-07T16:37:54.954 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.960+0000 7f346a3868c0 1 > prepared part key=_multipart_EQR0FY86AT5X.2~HIS6S9BLSBSW.2 size=619207 2024-04-07T16:37:54.955 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.960+0000 7f346a3868c0 1 > prepared multipart upload key=EQR0FY86AT5X upload=HIS6S9BLSBSW tag=NMR1AJ4LIX4D type=write size=3764935 2024-04-07T16:37:54.959 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.968+0000 7f346a3868c0 1 < canceled operation key=YXMDZB23YTB3 tag=EH63TVMHJDSQ type=write size=446599 2024-04-07T16:37:54.964 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.972+0000 7f346a3868c0 1 > prepared operation key=E84U0SX44NQ8 tag=1ATT98ER278Q type=del size=3530000 2024-04-07T16:37:54.969 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.976+0000 7f346a3868c0 1 < completed part key=_multipart_HBLEXONKXJTL.2~7ZZN28PHHNZH.0 size=1048576 2024-04-07T16:37:54.975 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.980+0000 7f346a3868c0 1 < completed part key=_multipart_HBLEXONKXJTL.2~7ZZN28PHHNZH.1 size=1032861 2024-04-07T16:37:54.983 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.992+0000 7f346a3868c0 1 < completed multipart upload key=HBLEXONKXJTL upload=7ZZN28PHHNZH tag=DJPH8R4QL0MS type=write size=3130013 2024-04-07T16:37:54.988 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:54.996+0000 7f346a3868c0 1 > prepared operation key=DXTO3AJ429AC tag=HK43UOORLC4Q type=del size=3940344 2024-04-07T16:37:54.994 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.000+0000 7f346a3868c0 1 < completed delete operation key=1VBSUWNI3V1T tag=V73T1VPMK4HH type=del 2024-04-07T16:37:55.001 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.008+0000 7f346a3868c0 1 > prepared operation key=1VKVQXSQMTU1 tag=ST6F8MXREXXO type=del size=148103 2024-04-07T16:37:55.006 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.012+0000 7f346a3868c0 1 < completed part key=_multipart_GZXHYZO5SMUP.2~6X2OJ3326VF1.0 size=1048576 2024-04-07T16:37:55.011 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.020+0000 7f346a3868c0 1 < completed part key=_multipart_GZXHYZO5SMUP.2~6X2OJ3326VF1.1 size=780564 2024-04-07T16:37:55.019 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.024+0000 7f346a3868c0 1 < completed multipart upload key=GZXHYZO5SMUP upload=6X2OJ3326VF1 tag=2JVP5PT0G349 type=write size=2877716 2024-04-07T16:37:55.028 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.036+0000 7f346a3868c0 1 > prepared part key=_multipart_LXOZ0T2KY6OR.2~CDKRNRVBNZV5.0 size=907760 2024-04-07T16:37:55.028 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.036+0000 7f346a3868c0 1 > prepared multipart upload key=LXOZ0T2KY6OR upload=CDKRNRVBNZV5 tag=X7LW6RZJXLGX type=write size=1956336 2024-04-07T16:37:55.032 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.040+0000 7f346a3868c0 1 < completed part key=_multipart_YXMDZB23YTB3.2~TWT1YM8GSSP0.0 size=1048576 2024-04-07T16:37:55.036 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.044+0000 7f346a3868c0 1 < completed part key=_multipart_YXMDZB23YTB3.2~TWT1YM8GSSP0.1 size=528653 2024-04-07T16:37:55.043 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.052+0000 7f346a3868c0 1 < completed multipart upload key=YXMDZB23YTB3 upload=TWT1YM8GSSP0 tag=W5NJJZL4Q276 type=write size=2625805 2024-04-07T16:37:55.052 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.060+0000 7f346a3868c0 1 > prepared part key=_multipart_8F81G0P0YBFW.2~PPDD9GVID6X0.0 size=1048576 2024-04-07T16:37:55.055 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.064+0000 7f346a3868c0 1 > prepared part key=_multipart_8F81G0P0YBFW.2~PPDD9GVID6X0.1 size=172580 2024-04-07T16:37:55.055 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.064+0000 7f346a3868c0 1 > prepared multipart upload key=8F81G0P0YBFW upload=PPDD9GVID6X0 tag=6FADFSZJ97Z4 type=write size=2269732 2024-04-07T16:37:55.060 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.068+0000 7f346a3868c0 1 < completed part key=_multipart_7G5C6HBK6QWI.2~CARXB93K5OLN.0 size=1048576 2024-04-07T16:37:55.063 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.072+0000 7f346a3868c0 1 < completed part key=_multipart_7G5C6HBK6QWI.2~CARXB93K5OLN.1 size=795081 2024-04-07T16:37:55.070 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.076+0000 7f346a3868c0 1 < completed multipart upload key=7G5C6HBK6QWI upload=CARXB93K5OLN tag=HSDP8N498E9B type=write size=2892233 2024-04-07T16:37:55.080 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.088+0000 7f346a3868c0 1 > prepared part key=_multipart_EOYFVEI50SQ1.2~U5RP74Z1LB7Q.0 size=1048576 2024-04-07T16:37:55.084 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.092+0000 7f346a3868c0 1 > prepared part key=_multipart_EOYFVEI50SQ1.2~U5RP74Z1LB7Q.1 size=743582 2024-04-07T16:37:55.085 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.092+0000 7f346a3868c0 1 > prepared multipart upload key=EOYFVEI50SQ1 upload=U5RP74Z1LB7Q tag=VHVOGE0CN70B type=write size=2840734 2024-04-07T16:37:55.090 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.096+0000 7f346a3868c0 1 < completed write operation key=1WGUPH7X5UA0 tag=VWVTBIUR634M type=write size=959558 2024-04-07T16:37:55.100 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.108+0000 7f346a3868c0 1 > prepared part key=_multipart_UIW3DN4NUYPP.2~IUJSUNGOKY1P.0 size=987471 2024-04-07T16:37:55.100 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.108+0000 7f346a3868c0 1 > prepared multipart upload key=UIW3DN4NUYPP upload=IUJSUNGOKY1P tag=KVC0LQEZQ1ZS type=write size=2036047 2024-04-07T16:37:55.106 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.112+0000 7f346a3868c0 1 < completed part key=_multipart_LXOZ0T2KY6OR.2~2Y3Y4E9U6H4L.0 size=1048576 2024-04-07T16:37:55.111 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.116+0000 7f346a3868c0 1 < completed part key=_multipart_LXOZ0T2KY6OR.2~2Y3Y4E9U6H4L.1 size=1048576 2024-04-07T16:37:55.116 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.124+0000 7f346a3868c0 1 < completed part key=_multipart_LXOZ0T2KY6OR.2~2Y3Y4E9U6H4L.2 size=251697 2024-04-07T16:37:55.124 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.132+0000 7f346a3868c0 1 < completed multipart upload key=LXOZ0T2KY6OR upload=2Y3Y4E9U6H4L tag=MR83GVQI68EW type=write size=3397425 2024-04-07T16:37:55.133 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.140+0000 7f346a3868c0 1 > prepared part key=_multipart_EOYFVEI50SQ1.2~RRDYPOO5O464.0 size=437319 2024-04-07T16:37:55.133 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.140+0000 7f346a3868c0 1 > prepared multipart upload key=EOYFVEI50SQ1 upload=RRDYPOO5O464 tag=LW3XNAY8OWN0 type=write size=1485895 2024-04-07T16:37:55.138 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.144+0000 7f346a3868c0 1 < completed delete operation key=ROFH30W3UCAX tag=53K91AO2LGL7 type=del 2024-04-07T16:37:55.143 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.152+0000 7f346a3868c0 1 > prepared operation key=QS4HYZFFAJNF tag=THCP3X590B68 type=del size=3186918 2024-04-07T16:37:55.148 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.156+0000 7f346a3868c0 1 < completed part key=_multipart_GBQSME7SY1AH.2~U4P7TP3XSFSJ.0 size=1048576 2024-04-07T16:37:55.148 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.156+0000 7f346a3868c0 1 < canceled part key=_multipart_GBQSME7SY1AH.2~U4P7TP3XSFSJ.1 size=486520 2024-04-07T16:37:55.156 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.164+0000 7f346a3868c0 1 < canceled multipart upload key=GBQSME7SY1AH upload=U4P7TP3XSFSJ tag=ULLOE9B9ZFZK type=write size=2583672 2024-04-07T16:37:55.161 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.168+0000 7f346a3868c0 1 > prepared operation key=C1BTVJNHIM0X tag=KOHR1D1AQQON type=write size=722160 2024-04-07T16:37:55.171 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.180+0000 7f346a3868c0 1 < suggested update operation key=IX3F0IUOKBX1 tag=6SN86K6Y1TJR type=del size=3517361 2024-04-07T16:37:55.180 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.188+0000 7f346a3868c0 1 > prepared part key=_multipart_UIW3DN4NUYPP.2~DGD0OYLKF03N.0 size=136465 2024-04-07T16:37:55.180 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.188+0000 7f346a3868c0 1 > prepared multipart upload key=UIW3DN4NUYPP upload=DGD0OYLKF03N tag=SOXIMMXNABFE type=write size=1185041 2024-04-07T16:37:55.185 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.192+0000 7f346a3868c0 1 < completed delete operation key=UIW3DN4NUYPP tag=EC8V4GWIIE9A type=del 2024-04-07T16:37:55.190 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.196+0000 7f346a3868c0 1 > prepared operation key=IX3F0IUOKBX1 tag=G001BQ8MWRQ9 type=del size=2940077 2024-04-07T16:37:55.192 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.200+0000 7f346a3868c0 -1 < no bi entry to suggest for operation key=EOYFVEI50SQ1 tag=0JT6MCAKZIT8 type=del size=2519692: (2) No such file or directory 2024-04-07T16:37:55.198 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.204+0000 7f346a3868c0 1 > prepared operation key=95YVKJ14JD8W tag=E9W00UUTO6ER type=del size=509555 2024-04-07T16:37:55.203 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.212+0000 7f346a3868c0 1 < completed part key=_multipart_N14BUXLJONHZ.2~UWO4TAYHM2FA.0 size=1048576 2024-04-07T16:37:55.207 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.216+0000 7f346a3868c0 1 < completed part key=_multipart_N14BUXLJONHZ.2~UWO4TAYHM2FA.1 size=948471 2024-04-07T16:37:55.216 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.224+0000 7f346a3868c0 1 < completed multipart upload key=N14BUXLJONHZ upload=UWO4TAYHM2FA tag=F74P0P57HC5M type=write size=3045623 2024-04-07T16:37:55.226 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.232+0000 7f346a3868c0 1 > prepared part key=_multipart_ROFH30W3UCAX.2~U4AD4YTZQM0R.0 size=1048576 2024-04-07T16:37:55.230 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.236+0000 7f346a3868c0 1 > prepared part key=_multipart_ROFH30W3UCAX.2~U4AD4YTZQM0R.1 size=1048576 2024-04-07T16:37:55.236 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.244+0000 7f346a3868c0 1 > prepared part key=_multipart_ROFH30W3UCAX.2~U4AD4YTZQM0R.2 size=148975 2024-04-07T16:37:55.236 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.244+0000 7f346a3868c0 1 > prepared multipart upload key=ROFH30W3UCAX upload=U4AD4YTZQM0R tag=82E9JUQT9E3B type=write size=3294703 2024-04-07T16:37:55.241 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.248+0000 7f346a3868c0 1 < completed delete operation key=X5Q46GO8YTOK tag=CCY7ZW8AZO32 type=del 2024-04-07T16:37:55.246 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.252+0000 7f346a3868c0 1 > prepared operation key=GBQSME7SY1AH tag=IB8L49GJIHAQ type=write size=786074 2024-04-07T16:37:55.251 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.260+0000 7f346a3868c0 1 < completed part key=_multipart_EQR0FY86AT5X.2~HIS6S9BLSBSW.0 size=1048576 2024-04-07T16:37:55.256 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.264+0000 7f346a3868c0 1 < completed part key=_multipart_EQR0FY86AT5X.2~HIS6S9BLSBSW.1 size=1048576 2024-04-07T16:37:55.260 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.268+0000 7f346a3868c0 1 < completed part key=_multipart_EQR0FY86AT5X.2~HIS6S9BLSBSW.2 size=619207 2024-04-07T16:37:55.269 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.276+0000 7f346a3868c0 1 < completed multipart upload key=EQR0FY86AT5X upload=HIS6S9BLSBSW tag=NMR1AJ4LIX4D type=write size=3764935 2024-04-07T16:37:55.278 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.284+0000 7f346a3868c0 1 > prepared part key=_multipart_3N0QJRYVO3FV.2~7KIFGYPNY155.0 size=566268 2024-04-07T16:37:55.278 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.284+0000 7f346a3868c0 1 > prepared multipart upload key=3N0QJRYVO3FV upload=7KIFGYPNY155 tag=D65JYK9V4IAE type=write size=1614844 2024-04-07T16:37:55.288 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.296+0000 7f346a3868c0 1 < suggested remove operation key=E84U0SX44NQ8 tag=1ATT98ER278Q type=del 2024-04-07T16:37:55.293 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.300+0000 7f346a3868c0 1 > prepared operation key=EOYFVEI50SQ1 tag=EC4O0Z0Y3MSG type=write size=22168 2024-04-07T16:37:55.299 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.304+0000 7f346a3868c0 1 < completed delete operation key=DXTO3AJ429AC tag=HK43UOORLC4Q type=del 2024-04-07T16:37:55.308 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.316+0000 7f346a3868c0 1 > prepared part key=_multipart_UIW3DN4NUYPP.2~9OIGUS31T6LM.0 size=1048576 2024-04-07T16:37:55.312 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.320+0000 7f346a3868c0 1 > prepared part key=_multipart_UIW3DN4NUYPP.2~9OIGUS31T6LM.1 size=337978 2024-04-07T16:37:55.312 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.320+0000 7f346a3868c0 1 > prepared multipart upload key=UIW3DN4NUYPP upload=9OIGUS31T6LM tag=AES9XGDJ98SQ type=write size=2435130 2024-04-07T16:37:55.317 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.324+0000 7f346a3868c0 1 < completed delete operation key=1VKVQXSQMTU1 tag=ST6F8MXREXXO type=del 2024-04-07T16:37:55.322 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.328+0000 7f346a3868c0 1 > prepared operation key=95YVKJ14JD8W tag=427WCBLQOH7Z type=del size=2283526 2024-04-07T16:37:55.328 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.336+0000 7f346a3868c0 1 < completed part key=_multipart_LXOZ0T2KY6OR.2~CDKRNRVBNZV5.0 size=907760 2024-04-07T16:37:55.337 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.344+0000 7f346a3868c0 1 < completed multipart upload key=LXOZ0T2KY6OR upload=CDKRNRVBNZV5 tag=X7LW6RZJXLGX type=write size=1956336 2024-04-07T16:37:55.348 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.356+0000 7f346a3868c0 1 > prepared part key=_multipart_EOYFVEI50SQ1.2~QARU46RDFPIX.0 size=1048576 2024-04-07T16:37:55.354 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.360+0000 7f346a3868c0 1 > prepared part key=_multipart_EOYFVEI50SQ1.2~QARU46RDFPIX.1 size=38300 2024-04-07T16:37:55.354 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.360+0000 7f346a3868c0 1 > prepared multipart upload key=EOYFVEI50SQ1 upload=QARU46RDFPIX tag=QTKSBPPTVHWA type=write size=2135452 2024-04-07T16:37:55.360 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.368+0000 7f346a3868c0 1 < completed part key=_multipart_8F81G0P0YBFW.2~PPDD9GVID6X0.0 size=1048576 2024-04-07T16:37:55.364 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.372+0000 7f346a3868c0 1 < completed part key=_multipart_8F81G0P0YBFW.2~PPDD9GVID6X0.1 size=172580 2024-04-07T16:37:55.373 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.380+0000 7f346a3868c0 1 < completed multipart upload key=8F81G0P0YBFW upload=PPDD9GVID6X0 tag=6FADFSZJ97Z4 type=write size=2269732 2024-04-07T16:37:55.378 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.384+0000 7f346a3868c0 1 > prepared operation key=QVOKSQAGHPA9 tag=18TFBL5IZNI9 type=write size=350123 2024-04-07T16:37:55.383 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.392+0000 7f346a3868c0 1 < completed part key=_multipart_EOYFVEI50SQ1.2~U5RP74Z1LB7Q.0 size=1048576 2024-04-07T16:37:55.389 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.396+0000 7f346a3868c0 1 < completed part key=_multipart_EOYFVEI50SQ1.2~U5RP74Z1LB7Q.1 size=743582 2024-04-07T16:37:55.398 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.404+0000 7f346a3868c0 1 < completed multipart upload key=EOYFVEI50SQ1 upload=U5RP74Z1LB7Q tag=VHVOGE0CN70B type=write size=2840734 2024-04-07T16:37:55.403 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.412+0000 7f346a3868c0 1 > prepared operation key=EOYFVEI50SQ1 tag=MP5GQ1QT9FDX type=del size=1537707 2024-04-07T16:37:55.408 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.416+0000 7f346a3868c0 1 < completed part key=_multipart_UIW3DN4NUYPP.2~IUJSUNGOKY1P.0 size=987471 2024-04-07T16:37:55.417 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.424+0000 7f346a3868c0 1 < completed multipart upload key=UIW3DN4NUYPP upload=IUJSUNGOKY1P tag=KVC0LQEZQ1ZS type=write size=2036047 2024-04-07T16:37:55.422 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.428+0000 7f346a3868c0 1 > prepared operation key=U3EID5EK64HD tag=HTNIH2PAK10K type=del size=1408184 2024-04-07T16:37:55.428 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.436+0000 7f346a3868c0 1 < completed part key=_multipart_EOYFVEI50SQ1.2~RRDYPOO5O464.0 size=437319 2024-04-07T16:37:55.437 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.444+0000 7f346a3868c0 1 < completed multipart upload key=EOYFVEI50SQ1 upload=RRDYPOO5O464 tag=LW3XNAY8OWN0 type=write size=1485895 2024-04-07T16:37:55.442 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.448+0000 7f346a3868c0 1 > prepared operation key=X5Q46GO8YTOK tag=8EP14EOYP1X9 type=write size=456210 2024-04-07T16:37:55.447 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.456+0000 7f346a3868c0 1 < completed delete operation key=QS4HYZFFAJNF tag=THCP3X590B68 type=del 2024-04-07T16:37:55.457 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.464+0000 7f346a3868c0 1 > prepared part key=_multipart_YXMDZB23YTB3.2~5L5D4BYP0INL.0 size=205852 2024-04-07T16:37:55.457 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.464+0000 7f346a3868c0 1 > prepared multipart upload key=YXMDZB23YTB3 upload=5L5D4BYP0INL tag=70URVKXD9A9Q type=write size=1254428 2024-04-07T16:37:55.462 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.468+0000 7f346a3868c0 1 < completed write operation key=C1BTVJNHIM0X tag=KOHR1D1AQQON type=write size=722160 2024-04-07T16:37:55.468 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.476+0000 7f346a3868c0 1 > prepared operation key=6XDS0A7L0IY8 tag=BP3YV2MQ4ZJC type=del size=1983139 2024-04-07T16:37:55.474 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.480+0000 7f346a3868c0 1 < completed part key=_multipart_UIW3DN4NUYPP.2~DGD0OYLKF03N.0 size=136465 2024-04-07T16:37:55.483 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.492+0000 7f346a3868c0 1 < completed multipart upload key=UIW3DN4NUYPP upload=DGD0OYLKF03N tag=SOXIMMXNABFE type=write size=1185041 2024-04-07T16:37:55.488 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.496+0000 7f346a3868c0 1 > prepared operation key=XJLQZ7IS9U1T tag=KBGJMUHLI205 type=del size=2377590 2024-04-07T16:37:55.493 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.500+0000 7f346a3868c0 1 < completed delete operation key=IX3F0IUOKBX1 tag=G001BQ8MWRQ9 type=del 2024-04-07T16:37:55.498 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.504+0000 7f346a3868c0 1 > prepared operation key=LXOZ0T2KY6OR tag=LID7ANMR450G type=del size=759309 2024-04-07T16:37:55.506 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.512+0000 7f346a3868c0 1 < completed delete operation key=95YVKJ14JD8W tag=E9W00UUTO6ER type=del 2024-04-07T16:37:55.510 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.516+0000 7f346a3868c0 1 > prepared operation key=3N0QJRYVO3FV tag=VX33JLTIYCRV type=del size=881165 2024-04-07T16:37:55.511 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.516+0000 7f346a3868c0 1 < canceled part key=_multipart_ROFH30W3UCAX.2~U4AD4YTZQM0R.0 size=1048576 2024-04-07T16:37:55.516 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.524+0000 7f346a3868c0 1 < completed part key=_multipart_ROFH30W3UCAX.2~U4AD4YTZQM0R.1 size=1048576 2024-04-07T16:37:55.520 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.528+0000 7f346a3868c0 1 < completed part key=_multipart_ROFH30W3UCAX.2~U4AD4YTZQM0R.2 size=148975 2024-04-07T16:37:55.530 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.536+0000 7f346a3868c0 1 < canceled multipart upload key=ROFH30W3UCAX upload=U4AD4YTZQM0R tag=82E9JUQT9E3B type=write size=3294703 2024-04-07T16:37:55.536 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.544+0000 7f346a3868c0 1 > prepared operation key=95YVKJ14JD8W tag=OR8EIPHF91J3 type=del size=1398593 2024-04-07T16:37:55.541 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.548+0000 7f346a3868c0 1 < completed write operation key=GBQSME7SY1AH tag=IB8L49GJIHAQ type=write size=786074 2024-04-07T16:37:55.551 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.556+0000 7f346a3868c0 1 > prepared part key=_multipart_EQR0FY86AT5X.2~Y8YFCSN34GZF.0 size=1048576 2024-04-07T16:37:55.555 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.564+0000 7f346a3868c0 1 > prepared part key=_multipart_EQR0FY86AT5X.2~Y8YFCSN34GZF.1 size=1048576 2024-04-07T16:37:55.559 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.568+0000 7f346a3868c0 1 > prepared part key=_multipart_EQR0FY86AT5X.2~Y8YFCSN34GZF.2 size=170853 2024-04-07T16:37:55.560 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.568+0000 7f346a3868c0 1 > prepared multipart upload key=EQR0FY86AT5X upload=Y8YFCSN34GZF tag=YKAMFMUIYYMR type=write size=3316581 2024-04-07T16:37:55.565 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.572+0000 7f346a3868c0 1 < completed part key=_multipart_3N0QJRYVO3FV.2~7KIFGYPNY155.0 size=566268 2024-04-07T16:37:55.574 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.580+0000 7f346a3868c0 1 < completed multipart upload key=3N0QJRYVO3FV upload=7KIFGYPNY155 tag=D65JYK9V4IAE type=write size=1614844 2024-04-07T16:37:55.584 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.592+0000 7f346a3868c0 1 > prepared part key=_multipart_3N0QJRYVO3FV.2~RF5K02VK44VQ.0 size=812564 2024-04-07T16:37:55.584 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.592+0000 7f346a3868c0 1 > prepared multipart upload key=3N0QJRYVO3FV upload=RF5K02VK44VQ tag=IPYYDR92NU4S type=write size=1861140 2024-04-07T16:37:55.594 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.600+0000 7f346a3868c0 1 < suggested remove operation key=EOYFVEI50SQ1 tag=EC4O0Z0Y3MSG type=write 2024-04-07T16:37:55.604 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.612+0000 7f346a3868c0 1 > prepared part key=_multipart_C1BTVJNHIM0X.2~UGKS068620A3.0 size=777036 2024-04-07T16:37:55.604 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.612+0000 7f346a3868c0 1 > prepared multipart upload key=C1BTVJNHIM0X upload=UGKS068620A3 tag=4EMC90Y4JU6W type=write size=1825612 2024-04-07T16:37:55.609 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.616+0000 7f346a3868c0 1 < completed part key=_multipart_UIW3DN4NUYPP.2~9OIGUS31T6LM.0 size=1048576 2024-04-07T16:37:55.613 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.620+0000 7f346a3868c0 1 < completed part key=_multipart_UIW3DN4NUYPP.2~9OIGUS31T6LM.1 size=337978 2024-04-07T16:37:55.623 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.628+0000 7f346a3868c0 1 < completed multipart upload key=UIW3DN4NUYPP upload=9OIGUS31T6LM tag=AES9XGDJ98SQ type=write size=2435130 2024-04-07T16:37:55.627 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.636+0000 7f346a3868c0 1 > prepared operation key=3N0QJRYVO3FV tag=LNHN5I16V0PB type=del size=758371 2024-04-07T16:37:55.633 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.640+0000 7f346a3868c0 1 < completed delete operation key=95YVKJ14JD8W tag=427WCBLQOH7Z type=del 2024-04-07T16:37:55.638 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.644+0000 7f346a3868c0 1 > prepared operation key=8F81G0P0YBFW tag=GGG1L2BL0NEF type=del size=3628912 2024-04-07T16:37:55.639 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.648+0000 7f346a3868c0 1 < canceled part key=_multipart_EOYFVEI50SQ1.2~QARU46RDFPIX.0 size=1048576 2024-04-07T16:37:55.643 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.652+0000 7f346a3868c0 1 < completed part key=_multipart_EOYFVEI50SQ1.2~QARU46RDFPIX.1 size=38300 2024-04-07T16:37:55.652 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.660+0000 7f346a3868c0 1 < canceled multipart upload key=EOYFVEI50SQ1 upload=QARU46RDFPIX tag=QTKSBPPTVHWA type=write size=2135452 2024-04-07T16:37:55.661 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.668+0000 7f346a3868c0 1 > prepared part key=_multipart_GZXHYZO5SMUP.2~UH6HXKE4QOL9.0 size=1036149 2024-04-07T16:37:55.661 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.668+0000 7f346a3868c0 1 > prepared multipart upload key=GZXHYZO5SMUP upload=UH6HXKE4QOL9 tag=3K0UX0AW3WWK type=write size=2084725 2024-04-07T16:37:55.666 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.672+0000 7f346a3868c0 1 < completed write operation key=QVOKSQAGHPA9 tag=18TFBL5IZNI9 type=write size=350123 2024-04-07T16:37:55.671 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.680+0000 7f346a3868c0 1 > prepared operation key=C1BTVJNHIM0X tag=YW9CH4Q0XF6K type=del size=3125423 2024-04-07T16:37:55.679 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.684+0000 7f346a3868c0 -1 < failed to complete operation key=EOYFVEI50SQ1 tag=MP5GQ1QT9FDX type=del size=1537707: (22) Invalid argument 2024-04-07T16:37:55.689 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.696+0000 7f346a3868c0 1 > prepared part key=_multipart_EOYFVEI50SQ1.2~I5GQOX5EG19O.0 size=1048576 2024-04-07T16:37:55.693 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.700+0000 7f346a3868c0 1 > prepared part key=_multipart_EOYFVEI50SQ1.2~I5GQOX5EG19O.1 size=1048576 2024-04-07T16:37:55.698 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.704+0000 7f346a3868c0 1 > prepared part key=_multipart_EOYFVEI50SQ1.2~I5GQOX5EG19O.2 size=174525 2024-04-07T16:37:55.698 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.704+0000 7f346a3868c0 1 > prepared multipart upload key=EOYFVEI50SQ1 upload=I5GQOX5EG19O tag=MTVQKOE3LI2I type=write size=3320253 2024-04-07T16:37:55.704 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.712+0000 7f346a3868c0 1 < completed delete operation key=U3EID5EK64HD tag=HTNIH2PAK10K type=del 2024-04-07T16:37:55.710 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.716+0000 7f346a3868c0 1 > prepared operation key=GZXHYZO5SMUP tag=TZSYSK392YTZ type=del size=797051 2024-04-07T16:37:55.716 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.724+0000 7f346a3868c0 1 < completed write operation key=X5Q46GO8YTOK tag=8EP14EOYP1X9 type=write size=456210 2024-04-07T16:37:55.725 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.732+0000 7f346a3868c0 1 > prepared part key=_multipart_3N0QJRYVO3FV.2~UQ4CMWU8RYTC.0 size=1048576 2024-04-07T16:37:55.730 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.736+0000 7f346a3868c0 1 > prepared part key=_multipart_3N0QJRYVO3FV.2~UQ4CMWU8RYTC.1 size=1004528 2024-04-07T16:37:55.730 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.736+0000 7f346a3868c0 1 > prepared multipart upload key=3N0QJRYVO3FV upload=UQ4CMWU8RYTC tag=SQ04D0I7JSBM type=write size=3101680 2024-04-07T16:37:55.735 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.744+0000 7f346a3868c0 1 < completed part key=_multipart_YXMDZB23YTB3.2~5L5D4BYP0INL.0 size=205852 2024-04-07T16:37:55.744 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.752+0000 7f346a3868c0 1 < completed multipart upload key=YXMDZB23YTB3 upload=5L5D4BYP0INL tag=70URVKXD9A9Q type=write size=1254428 2024-04-07T16:37:55.750 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.756+0000 7f346a3868c0 1 > prepared operation key=U3EID5EK64HD tag=F2J7YLCTOAKO type=write size=143264 2024-04-07T16:37:55.756 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.764+0000 7f346a3868c0 1 < completed delete operation key=6XDS0A7L0IY8 tag=BP3YV2MQ4ZJC type=del 2024-04-07T16:37:55.762 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.768+0000 7f346a3868c0 1 > prepared operation key=95YVKJ14JD8W tag=HQOTVBVLHNG1 type=del size=3535334 2024-04-07T16:37:55.768 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.776+0000 7f346a3868c0 1 < completed delete operation key=XJLQZ7IS9U1T tag=KBGJMUHLI205 type=del 2024-04-07T16:37:55.779 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.784+0000 7f346a3868c0 1 > prepared part key=_multipart_CL4PP1RKZU0E.2~3FGJY1EJG8P5.0 size=1048576 2024-04-07T16:37:55.783 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.792+0000 7f346a3868c0 1 > prepared part key=_multipart_CL4PP1RKZU0E.2~3FGJY1EJG8P5.1 size=1012571 2024-04-07T16:37:55.783 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.792+0000 7f346a3868c0 1 > prepared multipart upload key=CL4PP1RKZU0E upload=3FGJY1EJG8P5 tag=DZXA5GB5SNGB type=write size=3109723 2024-04-07T16:37:55.789 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.796+0000 7f346a3868c0 1 < completed delete operation key=LXOZ0T2KY6OR tag=LID7ANMR450G type=del 2024-04-07T16:37:55.794 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.800+0000 7f346a3868c0 1 > prepared operation key=1VKVQXSQMTU1 tag=YGD4JQB4X99A type=del size=2467852 2024-04-07T16:37:55.800 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.808+0000 7f346a3868c0 1 < completed delete operation key=3N0QJRYVO3FV tag=VX33JLTIYCRV type=del 2024-04-07T16:37:55.810 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.816+0000 7f346a3868c0 1 > prepared part key=_multipart_95YVKJ14JD8W.2~O3ISVV4WG59U.0 size=1048576 2024-04-07T16:37:55.815 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.820+0000 7f346a3868c0 1 > prepared part key=_multipart_95YVKJ14JD8W.2~O3ISVV4WG59U.1 size=1048576 2024-04-07T16:37:55.820 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.828+0000 7f346a3868c0 1 > prepared part key=_multipart_95YVKJ14JD8W.2~O3ISVV4WG59U.2 size=963044 2024-04-07T16:37:55.820 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.828+0000 7f346a3868c0 1 > prepared multipart upload key=95YVKJ14JD8W upload=O3ISVV4WG59U tag=6OODFHQL7W2I type=write size=4108772 2024-04-07T16:37:55.831 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.840+0000 7f346a3868c0 1 < suggested update operation key=95YVKJ14JD8W tag=OR8EIPHF91J3 type=del size=1398593 2024-04-07T16:37:55.837 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.844+0000 7f346a3868c0 1 > prepared operation key=IP8YW2A75TCW tag=DE060OKEIXS4 type=del size=2713893 2024-04-07T16:37:55.842 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.848+0000 7f346a3868c0 1 < completed part key=_multipart_EQR0FY86AT5X.2~Y8YFCSN34GZF.0 size=1048576 2024-04-07T16:37:55.847 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.852+0000 7f346a3868c0 1 < completed part key=_multipart_EQR0FY86AT5X.2~Y8YFCSN34GZF.1 size=1048576 2024-04-07T16:37:55.851 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.860+0000 7f346a3868c0 1 < completed part key=_multipart_EQR0FY86AT5X.2~Y8YFCSN34GZF.2 size=170853 2024-04-07T16:37:55.860 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.868+0000 7f346a3868c0 1 < completed multipart upload key=EQR0FY86AT5X upload=Y8YFCSN34GZF tag=YKAMFMUIYYMR type=write size=3316581 2024-04-07T16:37:55.871 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.880+0000 7f346a3868c0 1 > prepared part key=_multipart_EOYFVEI50SQ1.2~F5XLWVYIFHAK.0 size=684392 2024-04-07T16:37:55.871 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.880+0000 7f346a3868c0 1 > prepared multipart upload key=EOYFVEI50SQ1 upload=F5XLWVYIFHAK tag=G574I1BWITRB type=write size=1732968 2024-04-07T16:37:55.877 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.884+0000 7f346a3868c0 1 < completed part key=_multipart_3N0QJRYVO3FV.2~RF5K02VK44VQ.0 size=812564 2024-04-07T16:37:55.887 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.896+0000 7f346a3868c0 1 < completed multipart upload key=3N0QJRYVO3FV upload=RF5K02VK44VQ tag=IPYYDR92NU4S type=write size=1861140 2024-04-07T16:37:55.892 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.900+0000 7f346a3868c0 1 > prepared operation key=1VBSUWNI3V1T tag=ZH7K2HTPDMH3 type=del size=2546323 2024-04-07T16:37:55.898 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.904+0000 7f346a3868c0 1 < completed part key=_multipart_C1BTVJNHIM0X.2~UGKS068620A3.0 size=777036 2024-04-07T16:37:55.909 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.916+0000 7f346a3868c0 1 < completed multipart upload key=C1BTVJNHIM0X upload=UGKS068620A3 tag=4EMC90Y4JU6W type=write size=1825612 2024-04-07T16:37:55.920 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.928+0000 7f346a3868c0 1 > prepared part key=_multipart_IP8YW2A75TCW.2~7AOLNMWIWO6Z.0 size=1048576 2024-04-07T16:37:55.925 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.932+0000 7f346a3868c0 1 > prepared part key=_multipart_IP8YW2A75TCW.2~7AOLNMWIWO6Z.1 size=1048576 2024-04-07T16:37:55.929 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.936+0000 7f346a3868c0 1 > prepared part key=_multipart_IP8YW2A75TCW.2~7AOLNMWIWO6Z.2 size=648605 2024-04-07T16:37:55.929 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.936+0000 7f346a3868c0 1 > prepared multipart upload key=IP8YW2A75TCW upload=7AOLNMWIWO6Z tag=YJQ14N198JPB type=write size=3794333 2024-04-07T16:37:55.934 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.940+0000 7f346a3868c0 1 < completed delete operation key=3N0QJRYVO3FV tag=LNHN5I16V0PB type=del 2024-04-07T16:37:55.944 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.952+0000 7f346a3868c0 1 > prepared part key=_multipart_3N0QJRYVO3FV.2~XEOWTWZKXC4H.0 size=1048576 2024-04-07T16:37:55.949 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.956+0000 7f346a3868c0 1 > prepared part key=_multipart_3N0QJRYVO3FV.2~XEOWTWZKXC4H.1 size=1012894 2024-04-07T16:37:55.949 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.956+0000 7f346a3868c0 1 > prepared multipart upload key=3N0QJRYVO3FV upload=XEOWTWZKXC4H tag=KU23S38DOQ5S type=write size=3110046 2024-04-07T16:37:55.954 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.960+0000 7f346a3868c0 1 < completed delete operation key=8F81G0P0YBFW tag=GGG1L2BL0NEF type=del 2024-04-07T16:37:55.960 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.968+0000 7f346a3868c0 1 > prepared operation key=GBQSME7SY1AH tag=0CBA2B9YV3CJ type=write size=643765 2024-04-07T16:37:55.965 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.972+0000 7f346a3868c0 1 < completed part key=_multipart_GZXHYZO5SMUP.2~UH6HXKE4QOL9.0 size=1036149 2024-04-07T16:37:55.974 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.980+0000 7f346a3868c0 1 < completed multipart upload key=GZXHYZO5SMUP upload=UH6HXKE4QOL9 tag=3K0UX0AW3WWK type=write size=2084725 2024-04-07T16:37:55.979 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.988+0000 7f346a3868c0 1 > prepared operation key=ROFH30W3UCAX tag=DFH6086OGIBX type=del size=2640505 2024-04-07T16:37:55.990 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:55.996+0000 7f346a3868c0 1 < suggested update operation key=C1BTVJNHIM0X tag=YW9CH4Q0XF6K type=del size=3125423 2024-04-07T16:37:55.996 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.004+0000 7f346a3868c0 1 > prepared operation key=EOYFVEI50SQ1 tag=JUABEYEF4KA4 type=write size=458102 2024-04-07T16:37:55.997 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.004+0000 7f346a3868c0 1 < canceled part key=_multipart_EOYFVEI50SQ1.2~I5GQOX5EG19O.0 size=1048576 2024-04-07T16:37:56.001 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.008+0000 7f346a3868c0 1 < completed part key=_multipart_EOYFVEI50SQ1.2~I5GQOX5EG19O.1 size=1048576 2024-04-07T16:37:56.005 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.012+0000 7f346a3868c0 1 < completed part key=_multipart_EOYFVEI50SQ1.2~I5GQOX5EG19O.2 size=174525 2024-04-07T16:37:56.014 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.020+0000 7f346a3868c0 1 < canceled multipart upload key=EOYFVEI50SQ1 upload=I5GQOX5EG19O tag=MTVQKOE3LI2I type=write size=3320253 2024-04-07T16:37:56.025 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.032+0000 7f346a3868c0 1 > prepared part key=_multipart_1VBSUWNI3V1T.2~HCVGJI9G65WX.0 size=1048576 2024-04-07T16:37:56.029 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.036+0000 7f346a3868c0 1 > prepared part key=_multipart_1VBSUWNI3V1T.2~HCVGJI9G65WX.1 size=841657 2024-04-07T16:37:56.029 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.036+0000 7f346a3868c0 1 > prepared multipart upload key=1VBSUWNI3V1T upload=HCVGJI9G65WX tag=Z29980O9LXWS type=write size=2938809 2024-04-07T16:37:56.041 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.048+0000 7f346a3868c0 1 < suggested update operation key=GZXHYZO5SMUP tag=TZSYSK392YTZ type=del size=797051 2024-04-07T16:37:56.046 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.052+0000 7f346a3868c0 1 > prepared operation key=U3EID5EK64HD tag=E3NJWBKJMYKL type=write size=919718 2024-04-07T16:37:56.053 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.060+0000 7f346a3868c0 1 < completed part key=_multipart_3N0QJRYVO3FV.2~UQ4CMWU8RYTC.0 size=1048576 2024-04-07T16:37:56.058 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.064+0000 7f346a3868c0 1 < completed part key=_multipart_3N0QJRYVO3FV.2~UQ4CMWU8RYTC.1 size=1004528 2024-04-07T16:37:56.067 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.076+0000 7f346a3868c0 1 < completed multipart upload key=3N0QJRYVO3FV upload=UQ4CMWU8RYTC tag=SQ04D0I7JSBM type=write size=3101680 2024-04-07T16:37:56.073 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.080+0000 7f346a3868c0 1 > prepared operation key=LXOZ0T2KY6OR tag=0RU0EQIZQIW0 type=del size=2973626 2024-04-07T16:37:56.079 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.088+0000 7f346a3868c0 1 < completed write operation key=U3EID5EK64HD tag=F2J7YLCTOAKO type=write size=143264 2024-04-07T16:37:56.090 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.096+0000 7f346a3868c0 1 > prepared part key=_multipart_GBQSME7SY1AH.2~S57A2BMENWDO.0 size=1048576 2024-04-07T16:37:56.095 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.100+0000 7f346a3868c0 1 > prepared part key=_multipart_GBQSME7SY1AH.2~S57A2BMENWDO.1 size=258109 2024-04-07T16:37:56.095 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.100+0000 7f346a3868c0 1 > prepared multipart upload key=GBQSME7SY1AH upload=S57A2BMENWDO tag=4JC9M1JFIGYM type=write size=2355261 2024-04-07T16:37:56.102 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.108+0000 7f346a3868c0 -1 < failed to complete operation key=95YVKJ14JD8W tag=HQOTVBVLHNG1 type=del size=3535334: (22) Invalid argument 2024-04-07T16:37:56.108 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.116+0000 7f346a3868c0 1 > prepared operation key=U3EID5EK64HD tag=X37O977ZWMIR type=del size=2437146 2024-04-07T16:37:56.113 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.120+0000 7f346a3868c0 1 < completed part key=_multipart_CL4PP1RKZU0E.2~3FGJY1EJG8P5.0 size=1048576 2024-04-07T16:37:56.118 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.124+0000 7f346a3868c0 1 < completed part key=_multipart_CL4PP1RKZU0E.2~3FGJY1EJG8P5.1 size=1012571 2024-04-07T16:37:56.126 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.132+0000 7f346a3868c0 1 < completed multipart upload key=CL4PP1RKZU0E upload=3FGJY1EJG8P5 tag=DZXA5GB5SNGB type=write size=3109723 2024-04-07T16:37:56.131 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.140+0000 7f346a3868c0 1 > prepared operation key=7G5C6HBK6QWI tag=YJE3FEW0AKPX type=del size=3749108 2024-04-07T16:37:56.136 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.144+0000 7f346a3868c0 1 < completed delete operation key=1VKVQXSQMTU1 tag=YGD4JQB4X99A type=del 2024-04-07T16:37:56.141 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.148+0000 7f346a3868c0 1 > prepared operation key=U3EID5EK64HD tag=4R6U60QAT6SV type=del size=1590934 2024-04-07T16:37:56.147 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.152+0000 7f346a3868c0 1 < completed part key=_multipart_95YVKJ14JD8W.2~O3ISVV4WG59U.0 size=1048576 2024-04-07T16:37:56.151 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.160+0000 7f346a3868c0 1 < completed part key=_multipart_95YVKJ14JD8W.2~O3ISVV4WG59U.1 size=1048576 2024-04-07T16:37:56.155 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.164+0000 7f346a3868c0 1 < completed part key=_multipart_95YVKJ14JD8W.2~O3ISVV4WG59U.2 size=963044 2024-04-07T16:37:56.164 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.172+0000 7f346a3868c0 1 < completed multipart upload key=95YVKJ14JD8W upload=O3ISVV4WG59U tag=6OODFHQL7W2I type=write size=4108772 2024-04-07T16:37:56.173 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.180+0000 7f346a3868c0 1 > prepared part key=_multipart_QTCIIPTXIRW2.2~OS2T7EB6HOB0.0 size=309083 2024-04-07T16:37:56.174 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.180+0000 7f346a3868c0 1 > prepared multipart upload key=QTCIIPTXIRW2 upload=OS2T7EB6HOB0 tag=3EMKQ09ULG2G type=write size=1357659 2024-04-07T16:37:56.179 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.184+0000 7f346a3868c0 1 < completed delete operation key=IP8YW2A75TCW tag=DE060OKEIXS4 type=del 2024-04-07T16:37:56.188 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.196+0000 7f346a3868c0 1 > prepared part key=_multipart_E84U0SX44NQ8.2~9DLUJZ22AXR8.0 size=1048576 2024-04-07T16:37:56.192 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.200+0000 7f346a3868c0 1 > prepared part key=_multipart_E84U0SX44NQ8.2~9DLUJZ22AXR8.1 size=781964 2024-04-07T16:37:56.192 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.200+0000 7f346a3868c0 1 > prepared multipart upload key=E84U0SX44NQ8 upload=9DLUJZ22AXR8 tag=LJQ1LO1AKILF type=write size=2879116 2024-04-07T16:37:56.198 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.204+0000 7f346a3868c0 1 < completed part key=_multipart_EOYFVEI50SQ1.2~F5XLWVYIFHAK.0 size=684392 2024-04-07T16:37:56.206 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.212+0000 7f346a3868c0 1 < completed multipart upload key=EOYFVEI50SQ1 upload=F5XLWVYIFHAK tag=G574I1BWITRB type=write size=1732968 2024-04-07T16:37:56.211 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.220+0000 7f346a3868c0 1 > prepared operation key=XJLQZ7IS9U1T tag=IB9DYBLXZI85 type=del size=3777148 2024-04-07T16:37:56.217 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.224+0000 7f346a3868c0 1 < completed delete operation key=1VBSUWNI3V1T tag=ZH7K2HTPDMH3 type=del 2024-04-07T16:37:56.226 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.232+0000 7f346a3868c0 1 > prepared part key=_multipart_UIW3DN4NUYPP.2~ZOS313SS7KDC.0 size=361997 2024-04-07T16:37:56.226 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.232+0000 7f346a3868c0 1 > prepared multipart upload key=UIW3DN4NUYPP upload=ZOS313SS7KDC tag=L2SLV79XRHRZ type=write size=1410573 2024-04-07T16:37:56.231 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.240+0000 7f346a3868c0 1 < completed part key=_multipart_IP8YW2A75TCW.2~7AOLNMWIWO6Z.0 size=1048576 2024-04-07T16:37:56.236 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.244+0000 7f346a3868c0 1 < completed part key=_multipart_IP8YW2A75TCW.2~7AOLNMWIWO6Z.1 size=1048576 2024-04-07T16:37:56.241 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.248+0000 7f346a3868c0 1 < completed part key=_multipart_IP8YW2A75TCW.2~7AOLNMWIWO6Z.2 size=648605 2024-04-07T16:37:56.250 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.256+0000 7f346a3868c0 1 < completed multipart upload key=IP8YW2A75TCW upload=7AOLNMWIWO6Z tag=YJQ14N198JPB type=write size=3794333 2024-04-07T16:37:56.263 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.272+0000 7f346a3868c0 1 > prepared part key=_multipart_6XDS0A7L0IY8.2~9ILYE6EM0DJ5.0 size=1048576 2024-04-07T16:37:56.268 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.276+0000 7f346a3868c0 1 > prepared part key=_multipart_6XDS0A7L0IY8.2~9ILYE6EM0DJ5.1 size=1048576 2024-04-07T16:37:56.272 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.280+0000 7f346a3868c0 1 > prepared part key=_multipart_6XDS0A7L0IY8.2~9ILYE6EM0DJ5.2 size=476876 2024-04-07T16:37:56.272 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.280+0000 7f346a3868c0 1 > prepared multipart upload key=6XDS0A7L0IY8 upload=9ILYE6EM0DJ5 tag=MCTMIT0MTIG3 type=write size=3622604 2024-04-07T16:37:56.277 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.284+0000 7f346a3868c0 1 < completed part key=_multipart_3N0QJRYVO3FV.2~XEOWTWZKXC4H.0 size=1048576 2024-04-07T16:37:56.282 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.288+0000 7f346a3868c0 1 < completed part key=_multipart_3N0QJRYVO3FV.2~XEOWTWZKXC4H.1 size=1012894 2024-04-07T16:37:56.290 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.296+0000 7f346a3868c0 1 < completed multipart upload key=3N0QJRYVO3FV upload=XEOWTWZKXC4H tag=KU23S38DOQ5S type=write size=3110046 2024-04-07T16:37:56.296 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.304+0000 7f346a3868c0 1 > prepared operation key=1VBSUWNI3V1T tag=4S6IT5KFX3HY type=del size=1970583 2024-04-07T16:37:56.301 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.308+0000 7f346a3868c0 1 < completed write operation key=GBQSME7SY1AH tag=0CBA2B9YV3CJ type=write size=643765 2024-04-07T16:37:56.311 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.320+0000 7f346a3868c0 1 > prepared part key=_multipart_HBLEXONKXJTL.2~Y68RA8YRZJ3V.0 size=1048576 2024-04-07T16:37:56.315 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.324+0000 7f346a3868c0 1 > prepared part key=_multipart_HBLEXONKXJTL.2~Y68RA8YRZJ3V.1 size=479206 2024-04-07T16:37:56.316 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.324+0000 7f346a3868c0 1 > prepared multipart upload key=HBLEXONKXJTL upload=Y68RA8YRZJ3V tag=QQAQ7G5VBN14 type=write size=2576358 2024-04-07T16:37:56.321 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.328+0000 7f346a3868c0 1 < completed delete operation key=ROFH30W3UCAX tag=DFH6086OGIBX type=del 2024-04-07T16:37:56.328 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.336+0000 7f346a3868c0 1 > prepared operation key=7G5C6HBK6QWI tag=HDVMMRS8Z4PR type=del size=3532563 2024-04-07T16:37:56.333 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.340+0000 7f346a3868c0 1 < completed write operation key=EOYFVEI50SQ1 tag=JUABEYEF4KA4 type=write size=458102 2024-04-07T16:37:56.342 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.348+0000 7f346a3868c0 1 > prepared part key=_multipart_N14BUXLJONHZ.2~992I04904KAV.0 size=825522 2024-04-07T16:37:56.343 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.348+0000 7f346a3868c0 1 > prepared multipart upload key=N14BUXLJONHZ upload=992I04904KAV tag=5JI8051VN8MW type=write size=1874098 2024-04-07T16:37:56.343 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.352+0000 7f346a3868c0 1 < canceled part key=_multipart_1VBSUWNI3V1T.2~HCVGJI9G65WX.0 size=1048576 2024-04-07T16:37:56.348 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.356+0000 7f346a3868c0 1 < completed part key=_multipart_1VBSUWNI3V1T.2~HCVGJI9G65WX.1 size=841657 2024-04-07T16:37:56.356 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.364+0000 7f346a3868c0 1 < canceled multipart upload key=1VBSUWNI3V1T upload=HCVGJI9G65WX tag=Z29980O9LXWS type=write size=2938809 2024-04-07T16:37:56.368 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.376+0000 7f346a3868c0 1 > prepared part key=_multipart_HBLEXONKXJTL.2~Z04Q3N7HMU3K.0 size=1048576 2024-04-07T16:37:56.373 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.380+0000 7f346a3868c0 1 > prepared part key=_multipart_HBLEXONKXJTL.2~Z04Q3N7HMU3K.1 size=1048576 2024-04-07T16:37:56.377 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.384+0000 7f346a3868c0 1 > prepared part key=_multipart_HBLEXONKXJTL.2~Z04Q3N7HMU3K.2 size=872209 2024-04-07T16:37:56.378 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.384+0000 7f346a3868c0 1 > prepared multipart upload key=HBLEXONKXJTL upload=Z04Q3N7HMU3K tag=NB3KO4ZO5A2H type=write size=4017937 2024-04-07T16:37:56.384 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.392+0000 7f346a3868c0 1 < completed write operation key=U3EID5EK64HD tag=E3NJWBKJMYKL type=write size=919718 2024-04-07T16:37:56.390 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.396+0000 7f346a3868c0 1 > prepared operation key=HBLEXONKXJTL tag=DIPLWV4QTXCQ type=del size=3412830 2024-04-07T16:37:56.395 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.400+0000 7f346a3868c0 1 < completed delete operation key=LXOZ0T2KY6OR tag=0RU0EQIZQIW0 type=del 2024-04-07T16:37:56.400 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.408+0000 7f346a3868c0 1 > prepared operation key=NYET3212USVE tag=8FAK4F87OB0X type=write size=701408 2024-04-07T16:37:56.406 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.412+0000 7f346a3868c0 1 < completed part key=_multipart_GBQSME7SY1AH.2~S57A2BMENWDO.0 size=1048576 2024-04-07T16:37:56.411 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.416+0000 7f346a3868c0 1 < completed part key=_multipart_GBQSME7SY1AH.2~S57A2BMENWDO.1 size=258109 2024-04-07T16:37:56.420 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.428+0000 7f346a3868c0 1 < completed multipart upload key=GBQSME7SY1AH upload=S57A2BMENWDO tag=4JC9M1JFIGYM type=write size=2355261 2024-04-07T16:37:56.425 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.432+0000 7f346a3868c0 1 > prepared operation key=6XDS0A7L0IY8 tag=XTX5JHLYLEW8 type=del size=1448838 2024-04-07T16:37:56.435 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.440+0000 7f346a3868c0 1 < suggested remove operation key=U3EID5EK64HD tag=X37O977ZWMIR type=del 2024-04-07T16:37:56.440 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.448+0000 7f346a3868c0 1 > prepared operation key=QVOKSQAGHPA9 tag=GL9U23DNTMUI type=write size=41855 2024-04-07T16:37:56.441 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:37:56 smithi100 bash[74942]: cluster 2024-04-07T16:37:55.061718+0000 mgr.y (mgr.25175) 677 : cluster [DBG] pgmap v1312: 137 pgs: 137 active+clean; 583 KiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 481 KiB/s rd, 167 KiB/s wr, 850 op/s 2024-04-07T16:37:56.441 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:37:56 smithi100 bash[76202]: cluster 2024-04-07T16:37:55.061718+0000 mgr.y (mgr.25175) 677 : cluster [DBG] pgmap v1312: 137 pgs: 137 active+clean; 583 KiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 481 KiB/s rd, 167 KiB/s wr, 850 op/s 2024-04-07T16:37:56.445 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.452+0000 7f346a3868c0 1 < completed delete operation key=7G5C6HBK6QWI tag=YJE3FEW0AKPX type=del 2024-04-07T16:37:56.454 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.460+0000 7f346a3868c0 1 > prepared part key=_multipart_E84U0SX44NQ8.2~EP7XTYUU8H9Y.0 size=80820 2024-04-07T16:37:56.455 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.460+0000 7f346a3868c0 1 > prepared multipart upload key=E84U0SX44NQ8 upload=EP7XTYUU8H9Y tag=ZNWITCQHZAMB type=write size=1129396 2024-04-07T16:37:56.461 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.468+0000 7f346a3868c0 -1 < failed to complete operation key=U3EID5EK64HD tag=4R6U60QAT6SV type=del size=1590934: (22) Invalid argument 2024-04-07T16:37:56.471 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.480+0000 7f346a3868c0 1 > prepared part key=_multipart_3N0QJRYVO3FV.2~7QFZBHZF7BTY.0 size=1048576 2024-04-07T16:37:56.475 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.484+0000 7f346a3868c0 1 > prepared part key=_multipart_3N0QJRYVO3FV.2~7QFZBHZF7BTY.1 size=1048576 2024-04-07T16:37:56.479 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.488+0000 7f346a3868c0 1 > prepared part key=_multipart_3N0QJRYVO3FV.2~7QFZBHZF7BTY.2 size=691908 2024-04-07T16:37:56.480 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.488+0000 7f346a3868c0 1 > prepared multipart upload key=3N0QJRYVO3FV upload=7QFZBHZF7BTY tag=ZFAS6DCT7W5L type=write size=3837636 2024-04-07T16:37:56.485 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.492+0000 7f346a3868c0 1 < completed part key=_multipart_QTCIIPTXIRW2.2~OS2T7EB6HOB0.0 size=309083 2024-04-07T16:37:56.494 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.500+0000 7f346a3868c0 1 < completed multipart upload key=QTCIIPTXIRW2 upload=OS2T7EB6HOB0 tag=3EMKQ09ULG2G type=write size=1357659 2024-04-07T16:37:56.498 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.504+0000 7f346a3868c0 1 > prepared operation key=E84U0SX44NQ8 tag=Q48MAEVPFF3D type=write size=355265 2024-04-07T16:37:56.504 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.512+0000 7f346a3868c0 1 < completed part key=_multipart_E84U0SX44NQ8.2~9DLUJZ22AXR8.0 size=1048576 2024-04-07T16:37:56.508 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.516+0000 7f346a3868c0 1 < completed part key=_multipart_E84U0SX44NQ8.2~9DLUJZ22AXR8.1 size=781964 2024-04-07T16:37:56.517 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.524+0000 7f346a3868c0 1 < completed multipart upload key=E84U0SX44NQ8 upload=9DLUJZ22AXR8 tag=LJQ1LO1AKILF type=write size=2879116 2024-04-07T16:37:56.522 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.528+0000 7f346a3868c0 1 > prepared operation key=QVOKSQAGHPA9 tag=41NOK99GO3KA type=del size=1992040 2024-04-07T16:37:56.527 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.536+0000 7f346a3868c0 1 < completed delete operation key=XJLQZ7IS9U1T tag=IB9DYBLXZI85 type=del 2024-04-07T16:37:56.533 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.540+0000 7f346a3868c0 1 > prepared operation key=X5Q46GO8YTOK tag=WG6F1JLNKFAQ type=del size=1956348 2024-04-07T16:37:56.538 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.544+0000 7f346a3868c0 1 < completed part key=_multipart_UIW3DN4NUYPP.2~ZOS313SS7KDC.0 size=361997 2024-04-07T16:37:56.547 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.556+0000 7f346a3868c0 1 < completed multipart upload key=UIW3DN4NUYPP upload=ZOS313SS7KDC tag=L2SLV79XRHRZ type=write size=1410573 2024-04-07T16:37:56.552 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.560+0000 7f346a3868c0 1 > prepared operation key=CL4PP1RKZU0E tag=R7FOHDNWHCOB type=del size=1247319 2024-04-07T16:37:56.557 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.564+0000 7f346a3868c0 1 < completed part key=_multipart_6XDS0A7L0IY8.2~9ILYE6EM0DJ5.0 size=1048576 2024-04-07T16:37:56.557 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.564+0000 7f346a3868c0 1 < canceled part key=_multipart_6XDS0A7L0IY8.2~9ILYE6EM0DJ5.1 size=1048576 2024-04-07T16:37:56.562 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.568+0000 7f346a3868c0 1 < completed part key=_multipart_6XDS0A7L0IY8.2~9ILYE6EM0DJ5.2 size=476876 2024-04-07T16:37:56.571 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.580+0000 7f346a3868c0 1 < canceled multipart upload key=6XDS0A7L0IY8 upload=9ILYE6EM0DJ5 tag=MCTMIT0MTIG3 type=write size=3622604 2024-04-07T16:37:56.576 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.584+0000 7f346a3868c0 1 > prepared operation key=1VBSUWNI3V1T tag=13GIBWYWKDM7 type=del size=1324829 2024-04-07T16:37:56.582 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.588+0000 7f346a3868c0 1 < completed delete operation key=1VBSUWNI3V1T tag=4S6IT5KFX3HY type=del 2024-04-07T16:37:56.588 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.596+0000 7f346a3868c0 1 > prepared operation key=CL4PP1RKZU0E tag=82B82FLUTPVC type=write size=559939 2024-04-07T16:37:56.594 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.600+0000 7f346a3868c0 1 < completed part key=_multipart_HBLEXONKXJTL.2~Y68RA8YRZJ3V.0 size=1048576 2024-04-07T16:37:56.598 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.604+0000 7f346a3868c0 1 < completed part key=_multipart_HBLEXONKXJTL.2~Y68RA8YRZJ3V.1 size=479206 2024-04-07T16:37:56.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:37:56 smithi144 bash[66654]: cluster 2024-04-07T16:37:55.061718+0000 mgr.y (mgr.25175) 677 : cluster [DBG] pgmap v1312: 137 pgs: 137 active+clean; 583 KiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 481 KiB/s rd, 167 KiB/s wr, 850 op/s 2024-04-07T16:37:56.607 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.616+0000 7f346a3868c0 1 < completed multipart upload key=HBLEXONKXJTL upload=Y68RA8YRZJ3V tag=QQAQ7G5VBN14 type=write size=2576358 2024-04-07T16:37:56.612 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.620+0000 7f346a3868c0 1 > prepared operation key=UIW3DN4NUYPP tag=VLTYDKT80ELW type=write size=493213 2024-04-07T16:37:56.618 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.624+0000 7f346a3868c0 1 < completed delete operation key=7G5C6HBK6QWI tag=HDVMMRS8Z4PR type=del 2024-04-07T16:37:56.629 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.636+0000 7f346a3868c0 1 > prepared part key=_multipart_YXMDZB23YTB3.2~2EN1VMA18HY9.0 size=1048576 2024-04-07T16:37:56.633 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.640+0000 7f346a3868c0 1 > prepared part key=_multipart_YXMDZB23YTB3.2~2EN1VMA18HY9.1 size=785137 2024-04-07T16:37:56.633 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.640+0000 7f346a3868c0 1 > prepared multipart upload key=YXMDZB23YTB3 upload=2EN1VMA18HY9 tag=A8SMY711DJYS type=write size=2882289 2024-04-07T16:37:56.634 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.640+0000 7f346a3868c0 1 < canceled part key=_multipart_N14BUXLJONHZ.2~992I04904KAV.0 size=825522 2024-04-07T16:37:56.643 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.648+0000 7f346a3868c0 1 < canceled multipart upload key=N14BUXLJONHZ upload=992I04904KAV tag=5JI8051VN8MW type=write size=1874098 2024-04-07T16:37:56.648 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.656+0000 7f346a3868c0 1 > prepared operation key=N14BUXLJONHZ tag=UZACAJNBJM2R type=del size=1788723 2024-04-07T16:37:56.653 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.660+0000 7f346a3868c0 1 < completed part key=_multipart_HBLEXONKXJTL.2~Z04Q3N7HMU3K.0 size=1048576 2024-04-07T16:37:56.657 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.664+0000 7f346a3868c0 1 < completed part key=_multipart_HBLEXONKXJTL.2~Z04Q3N7HMU3K.1 size=1048576 2024-04-07T16:37:56.662 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.668+0000 7f346a3868c0 1 < completed part key=_multipart_HBLEXONKXJTL.2~Z04Q3N7HMU3K.2 size=872209 2024-04-07T16:37:56.671 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.676+0000 7f346a3868c0 1 < completed multipart upload key=HBLEXONKXJTL upload=Z04Q3N7HMU3K tag=NB3KO4ZO5A2H type=write size=4017937 2024-04-07T16:37:56.676 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.684+0000 7f346a3868c0 1 > prepared operation key=NYET3212USVE tag=924HZEFPGHW9 type=del size=3049968 2024-04-07T16:37:56.682 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.688+0000 7f346a3868c0 1 < completed delete operation key=HBLEXONKXJTL tag=DIPLWV4QTXCQ type=del 2024-04-07T16:37:56.693 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.700+0000 7f346a3868c0 1 > prepared part key=_multipart_1VKVQXSQMTU1.2~GGUBWAY1R6VL.0 size=1028695 2024-04-07T16:37:56.693 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.700+0000 7f346a3868c0 1 > prepared multipart upload key=1VKVQXSQMTU1 upload=GGUBWAY1R6VL tag=BRQE77F54EJ9 type=write size=2077271 2024-04-07T16:37:56.698 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.704+0000 7f346a3868c0 1 < completed write operation key=NYET3212USVE tag=8FAK4F87OB0X type=write size=701408 2024-04-07T16:37:56.709 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.716+0000 7f346a3868c0 1 > prepared part key=_multipart_7G5C6HBK6QWI.2~XBKARZUY4QPK.0 size=811283 2024-04-07T16:37:56.709 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.716+0000 7f346a3868c0 1 > prepared multipart upload key=7G5C6HBK6QWI upload=XBKARZUY4QPK tag=5Z4KNNYD8XYU type=write size=1859859 2024-04-07T16:37:56.715 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.720+0000 7f346a3868c0 1 < canceled operation key=6XDS0A7L0IY8 tag=XTX5JHLYLEW8 type=del size=1448838 2024-04-07T16:37:56.725 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.732+0000 7f346a3868c0 1 > prepared part key=_multipart_XJLQZ7IS9U1T.2~2MY0Z6GEDVYM.0 size=1048576 2024-04-07T16:37:56.729 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.736+0000 7f346a3868c0 1 > prepared part key=_multipart_XJLQZ7IS9U1T.2~2MY0Z6GEDVYM.1 size=1048576 2024-04-07T16:37:56.734 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.740+0000 7f346a3868c0 1 > prepared part key=_multipart_XJLQZ7IS9U1T.2~2MY0Z6GEDVYM.2 size=959099 2024-04-07T16:37:56.734 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.740+0000 7f346a3868c0 1 > prepared multipart upload key=XJLQZ7IS9U1T upload=2MY0Z6GEDVYM tag=HVHQRSZ7ZOAH type=write size=4104827 2024-04-07T16:37:56.739 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.748+0000 7f346a3868c0 1 < completed write operation key=QVOKSQAGHPA9 tag=GL9U23DNTMUI type=write size=41855 2024-04-07T16:37:56.745 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.752+0000 7f346a3868c0 1 > prepared operation key=EQR0FY86AT5X tag=5Z07TU6MIT86 type=del size=676161 2024-04-07T16:37:56.751 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.760+0000 7f346a3868c0 1 < completed part key=_multipart_E84U0SX44NQ8.2~EP7XTYUU8H9Y.0 size=80820 2024-04-07T16:37:56.760 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.768+0000 7f346a3868c0 1 < completed multipart upload key=E84U0SX44NQ8 upload=EP7XTYUU8H9Y tag=ZNWITCQHZAMB type=write size=1129396 2024-04-07T16:37:56.770 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.776+0000 7f346a3868c0 1 > prepared part key=_multipart_XJLQZ7IS9U1T.2~H5VOY4KOIVC0.0 size=1048576 2024-04-07T16:37:56.774 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.780+0000 7f346a3868c0 1 > prepared part key=_multipart_XJLQZ7IS9U1T.2~H5VOY4KOIVC0.1 size=1048576 2024-04-07T16:37:56.778 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.784+0000 7f346a3868c0 1 > prepared part key=_multipart_XJLQZ7IS9U1T.2~H5VOY4KOIVC0.2 size=597743 2024-04-07T16:37:56.779 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.784+0000 7f346a3868c0 1 > prepared multipart upload key=XJLQZ7IS9U1T upload=H5VOY4KOIVC0 tag=2W6XLFOFZHOK type=write size=3743471 2024-04-07T16:37:56.785 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.792+0000 7f346a3868c0 1 < completed part key=_multipart_3N0QJRYVO3FV.2~7QFZBHZF7BTY.0 size=1048576 2024-04-07T16:37:56.790 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.796+0000 7f346a3868c0 1 < completed part key=_multipart_3N0QJRYVO3FV.2~7QFZBHZF7BTY.1 size=1048576 2024-04-07T16:37:56.795 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.800+0000 7f346a3868c0 1 < completed part key=_multipart_3N0QJRYVO3FV.2~7QFZBHZF7BTY.2 size=691908 2024-04-07T16:37:56.804 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.812+0000 7f346a3868c0 1 < completed multipart upload key=3N0QJRYVO3FV upload=7QFZBHZF7BTY tag=ZFAS6DCT7W5L type=write size=3837636 2024-04-07T16:37:56.809 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.816+0000 7f346a3868c0 1 > prepared operation key=95YVKJ14JD8W tag=XN0TL9O953HD type=del size=1472330 2024-04-07T16:37:56.820 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.828+0000 7f346a3868c0 1 < suggested remove operation key=E84U0SX44NQ8 tag=Q48MAEVPFF3D type=write 2024-04-07T16:37:56.829 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.836+0000 7f346a3868c0 1 > prepared part key=_multipart_IX3F0IUOKBX1.2~583FF3HW9VK0.0 size=1048576 2024-04-07T16:37:56.834 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.840+0000 7f346a3868c0 1 > prepared part key=_multipart_IX3F0IUOKBX1.2~583FF3HW9VK0.1 size=937271 2024-04-07T16:37:56.834 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.840+0000 7f346a3868c0 1 > prepared multipart upload key=IX3F0IUOKBX1 upload=583FF3HW9VK0 tag=WW9WVVL6KS2N type=write size=3034423 2024-04-07T16:37:56.839 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.848+0000 7f346a3868c0 1 < completed delete operation key=QVOKSQAGHPA9 tag=41NOK99GO3KA type=del 2024-04-07T16:37:56.845 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.852+0000 7f346a3868c0 1 > prepared operation key=IP8YW2A75TCW tag=9LAZQQ56L7XF type=del size=2146205 2024-04-07T16:37:56.855 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.860+0000 7f346a3868c0 1 < suggested remove operation key=X5Q46GO8YTOK tag=WG6F1JLNKFAQ type=del 2024-04-07T16:37:56.859 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.868+0000 7f346a3868c0 1 > prepared operation key=6XDS0A7L0IY8 tag=DR8390LJK4LF type=del size=16666 2024-04-07T16:37:56.864 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.872+0000 7f346a3868c0 1 < completed delete operation key=CL4PP1RKZU0E tag=R7FOHDNWHCOB type=del 2024-04-07T16:37:56.869 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.876+0000 7f346a3868c0 1 > prepared operation key=DXTO3AJ429AC tag=NA1H33B96U9N type=del size=2103991 2024-04-07T16:37:56.875 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.880+0000 7f346a3868c0 1 < completed delete operation key=1VBSUWNI3V1T tag=13GIBWYWKDM7 type=del 2024-04-07T16:37:56.880 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.888+0000 7f346a3868c0 1 > prepared operation key=C1BTVJNHIM0X tag=VKTRQQ8VY1YX type=del size=212610 2024-04-07T16:37:56.891 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.900+0000 7f346a3868c0 1 < suggested remove operation key=CL4PP1RKZU0E tag=82B82FLUTPVC type=write 2024-04-07T16:37:56.902 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.908+0000 7f346a3868c0 1 > prepared part key=_multipart_IP8YW2A75TCW.2~SS3HNEOUX52D.0 size=744014 2024-04-07T16:37:56.902 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.908+0000 7f346a3868c0 1 > prepared multipart upload key=IP8YW2A75TCW upload=SS3HNEOUX52D tag=G8A10021SKMZ type=write size=1792590 2024-04-07T16:37:56.907 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.916+0000 7f346a3868c0 1 < canceled operation key=UIW3DN4NUYPP tag=VLTYDKT80ELW type=write size=493213 2024-04-07T16:37:56.919 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.924+0000 7f346a3868c0 1 > prepared part key=_multipart_95YVKJ14JD8W.2~L4UK2PTSANO3.0 size=1048576 2024-04-07T16:37:56.923 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.932+0000 7f346a3868c0 1 > prepared part key=_multipart_95YVKJ14JD8W.2~L4UK2PTSANO3.1 size=878346 2024-04-07T16:37:56.923 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.932+0000 7f346a3868c0 1 > prepared multipart upload key=95YVKJ14JD8W upload=L4UK2PTSANO3 tag=X7S1WDEIS2YM type=write size=2975498 2024-04-07T16:37:56.929 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.936+0000 7f346a3868c0 1 < completed part key=_multipart_YXMDZB23YTB3.2~2EN1VMA18HY9.0 size=1048576 2024-04-07T16:37:56.933 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.940+0000 7f346a3868c0 1 < completed part key=_multipart_YXMDZB23YTB3.2~2EN1VMA18HY9.1 size=785137 2024-04-07T16:37:56.942 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.948+0000 7f346a3868c0 1 < completed multipart upload key=YXMDZB23YTB3 upload=2EN1VMA18HY9 tag=A8SMY711DJYS type=write size=2882289 2024-04-07T16:37:56.947 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.952+0000 7f346a3868c0 1 > prepared operation key=1VBSUWNI3V1T tag=AOZDP7MJSB5I type=del size=2099124 2024-04-07T16:37:56.952 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.960+0000 7f346a3868c0 1 < completed delete operation key=N14BUXLJONHZ tag=UZACAJNBJM2R type=del 2024-04-07T16:37:56.962 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.968+0000 7f346a3868c0 1 > prepared part key=_multipart_C1BTVJNHIM0X.2~2Y2VUNPJT25M.0 size=277548 2024-04-07T16:37:56.962 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.968+0000 7f346a3868c0 1 > prepared multipart upload key=C1BTVJNHIM0X upload=2Y2VUNPJT25M tag=YNIMQ1PO2N2N type=write size=1326124 2024-04-07T16:37:56.967 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.972+0000 7f346a3868c0 1 < completed delete operation key=NYET3212USVE tag=924HZEFPGHW9 type=del 2024-04-07T16:37:56.972 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.980+0000 7f346a3868c0 1 > prepared operation key=IP8YW2A75TCW tag=RLCRNGW6SOCA type=del size=2399590 2024-04-07T16:37:56.977 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.984+0000 7f346a3868c0 1 < completed part key=_multipart_1VKVQXSQMTU1.2~GGUBWAY1R6VL.0 size=1028695 2024-04-07T16:37:56.986 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:56.992+0000 7f346a3868c0 1 < completed multipart upload key=1VKVQXSQMTU1 upload=GGUBWAY1R6VL tag=BRQE77F54EJ9 type=write size=2077271 2024-04-07T16:37:56.993 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.000+0000 7f346a3868c0 1 > prepared operation key=X5Q46GO8YTOK tag=5UVUBPD4TU05 type=del size=3351203 2024-04-07T16:37:56.998 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.004+0000 7f346a3868c0 1 < completed part key=_multipart_7G5C6HBK6QWI.2~XBKARZUY4QPK.0 size=811283 2024-04-07T16:37:57.009 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.016+0000 7f346a3868c0 1 < completed multipart upload key=7G5C6HBK6QWI upload=XBKARZUY4QPK tag=5Z4KNNYD8XYU type=write size=1859859 2024-04-07T16:37:57.017 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.024+0000 7f346a3868c0 1 > prepared part key=_multipart_DXTO3AJ429AC.2~O1GJK3Z1K31O.0 size=1028110 2024-04-07T16:37:57.018 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.024+0000 7f346a3868c0 1 > prepared multipart upload key=DXTO3AJ429AC upload=O1GJK3Z1K31O tag=P7YZEYJCDVU0 type=write size=2076686 2024-04-07T16:37:57.022 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.028+0000 7f346a3868c0 1 < completed part key=_multipart_XJLQZ7IS9U1T.2~2MY0Z6GEDVYM.0 size=1048576 2024-04-07T16:37:57.022 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.028+0000 7f346a3868c0 1 < canceled part key=_multipart_XJLQZ7IS9U1T.2~2MY0Z6GEDVYM.1 size=1048576 2024-04-07T16:37:57.026 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.032+0000 7f346a3868c0 1 < completed part key=_multipart_XJLQZ7IS9U1T.2~2MY0Z6GEDVYM.2 size=959099 2024-04-07T16:37:57.036 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.044+0000 7f346a3868c0 1 < canceled multipart upload key=XJLQZ7IS9U1T upload=2MY0Z6GEDVYM tag=HVHQRSZ7ZOAH type=write size=4104827 2024-04-07T16:37:57.046 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.052+0000 7f346a3868c0 1 > prepared part key=_multipart_7G5C6HBK6QWI.2~QHCMJ50MX3E7.0 size=1048576 2024-04-07T16:37:57.050 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.056+0000 7f346a3868c0 1 > prepared part key=_multipart_7G5C6HBK6QWI.2~QHCMJ50MX3E7.1 size=1048576 2024-04-07T16:37:57.053 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.060+0000 7f346a3868c0 1 > prepared part key=_multipart_7G5C6HBK6QWI.2~QHCMJ50MX3E7.2 size=810752 2024-04-07T16:37:57.054 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.060+0000 7f346a3868c0 1 > prepared multipart upload key=7G5C6HBK6QWI upload=QHCMJ50MX3E7 tag=RIX5BC3JWUR2 type=write size=3956480 2024-04-07T16:37:57.058 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.064+0000 7f346a3868c0 1 < canceled operation key=EQR0FY86AT5X tag=5Z07TU6MIT86 type=del size=676161 2024-04-07T16:37:57.063 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.072+0000 7f346a3868c0 1 > prepared operation key=YXMDZB23YTB3 tag=HZ7YY25RMST0 type=del size=3284898 2024-04-07T16:37:57.069 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.076+0000 7f346a3868c0 1 < completed part key=_multipart_XJLQZ7IS9U1T.2~H5VOY4KOIVC0.0 size=1048576 2024-04-07T16:37:57.073 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.080+0000 7f346a3868c0 1 < completed part key=_multipart_XJLQZ7IS9U1T.2~H5VOY4KOIVC0.1 size=1048576 2024-04-07T16:37:57.078 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.084+0000 7f346a3868c0 1 < completed part key=_multipart_XJLQZ7IS9U1T.2~H5VOY4KOIVC0.2 size=597743 2024-04-07T16:37:57.087 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.092+0000 7f346a3868c0 1 < completed multipart upload key=XJLQZ7IS9U1T upload=H5VOY4KOIVC0 tag=2W6XLFOFZHOK type=write size=3743471 2024-04-07T16:37:57.096 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.104+0000 7f346a3868c0 1 > prepared part key=_multipart_DXTO3AJ429AC.2~QO3WT7L0WBU8.0 size=1048576 2024-04-07T16:37:57.100 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.108+0000 7f346a3868c0 1 > prepared part key=_multipart_DXTO3AJ429AC.2~QO3WT7L0WBU8.1 size=1048576 2024-04-07T16:37:57.104 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.112+0000 7f346a3868c0 1 > prepared part key=_multipart_DXTO3AJ429AC.2~QO3WT7L0WBU8.2 size=877967 2024-04-07T16:37:57.104 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.112+0000 7f346a3868c0 1 > prepared multipart upload key=DXTO3AJ429AC upload=QO3WT7L0WBU8 tag=92K0NTLGG2OU type=write size=4023695 2024-04-07T16:37:57.109 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.116+0000 7f346a3868c0 1 < completed delete operation key=95YVKJ14JD8W tag=XN0TL9O953HD type=del 2024-04-07T16:37:57.115 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.124+0000 7f346a3868c0 1 > prepared operation key=1VKVQXSQMTU1 tag=WLKOMWFGF1D2 type=write size=18830 2024-04-07T16:37:57.120 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.128+0000 7f346a3868c0 1 < completed part key=_multipart_IX3F0IUOKBX1.2~583FF3HW9VK0.0 size=1048576 2024-04-07T16:37:57.125 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.132+0000 7f346a3868c0 1 < completed part key=_multipart_IX3F0IUOKBX1.2~583FF3HW9VK0.1 size=937271 2024-04-07T16:37:57.134 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.140+0000 7f346a3868c0 1 < completed multipart upload key=IX3F0IUOKBX1 upload=583FF3HW9VK0 tag=WW9WVVL6KS2N type=write size=3034423 2024-04-07T16:37:57.139 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.148+0000 7f346a3868c0 1 > prepared operation key=HBLEXONKXJTL tag=22SOMIWCMX81 type=del size=2714790 2024-04-07T16:37:57.150 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.156+0000 7f346a3868c0 1 < suggested update operation key=IP8YW2A75TCW tag=9LAZQQ56L7XF type=del size=2146205 2024-04-07T16:37:57.159 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.168+0000 7f346a3868c0 1 > prepared part key=_multipart_IX3F0IUOKBX1.2~LBQFSGM9OKU4.0 size=1048576 2024-04-07T16:37:57.164 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.172+0000 7f346a3868c0 1 > prepared part key=_multipart_IX3F0IUOKBX1.2~LBQFSGM9OKU4.1 size=1048576 2024-04-07T16:37:57.168 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.176+0000 7f346a3868c0 1 > prepared part key=_multipart_IX3F0IUOKBX1.2~LBQFSGM9OKU4.2 size=565061 2024-04-07T16:37:57.168 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.176+0000 7f346a3868c0 1 > prepared multipart upload key=IX3F0IUOKBX1 upload=LBQFSGM9OKU4 tag=CZ6R1KRXHL27 type=write size=3710789 2024-04-07T16:37:57.173 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.180+0000 7f346a3868c0 1 < completed delete operation key=6XDS0A7L0IY8 tag=DR8390LJK4LF type=del 2024-04-07T16:37:57.183 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.192+0000 7f346a3868c0 1 > prepared part key=_multipart_QS4HYZFFAJNF.2~9TDZWFF7HSGZ.0 size=1048576 2024-04-07T16:37:57.188 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.196+0000 7f346a3868c0 1 > prepared part key=_multipart_QS4HYZFFAJNF.2~9TDZWFF7HSGZ.1 size=1048576 2024-04-07T16:37:57.193 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.200+0000 7f346a3868c0 1 > prepared part key=_multipart_QS4HYZFFAJNF.2~9TDZWFF7HSGZ.2 size=277535 2024-04-07T16:37:57.193 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.200+0000 7f346a3868c0 1 > prepared multipart upload key=QS4HYZFFAJNF upload=9TDZWFF7HSGZ tag=0RWIKRZARPAR type=write size=3423263 2024-04-07T16:37:57.199 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.204+0000 7f346a3868c0 1 < completed delete operation key=DXTO3AJ429AC tag=NA1H33B96U9N type=del 2024-04-07T16:37:57.204 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.212+0000 7f346a3868c0 1 > prepared operation key=8F81G0P0YBFW tag=7Z072AE857QN type=del size=2273829 2024-04-07T16:37:57.210 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.216+0000 7f346a3868c0 1 < completed delete operation key=C1BTVJNHIM0X tag=VKTRQQ8VY1YX type=del 2024-04-07T16:37:57.215 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.220+0000 7f346a3868c0 1 > prepared operation key=ROFH30W3UCAX tag=SCK8I2NPBLHK type=del size=2518364 2024-04-07T16:37:57.220 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.228+0000 7f346a3868c0 1 < completed part key=_multipart_IP8YW2A75TCW.2~SS3HNEOUX52D.0 size=744014 2024-04-07T16:37:57.228 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.236+0000 7f346a3868c0 1 < completed multipart upload key=IP8YW2A75TCW upload=SS3HNEOUX52D tag=G8A10021SKMZ type=write size=1792590 2024-04-07T16:37:57.234 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.240+0000 7f346a3868c0 1 > prepared operation key=HBLEXONKXJTL tag=USCSB6XM6CBF type=del size=2152941 2024-04-07T16:37:57.239 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.248+0000 7f346a3868c0 1 < completed part key=_multipart_95YVKJ14JD8W.2~L4UK2PTSANO3.0 size=1048576 2024-04-07T16:37:57.244 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.252+0000 7f346a3868c0 1 < completed part key=_multipart_95YVKJ14JD8W.2~L4UK2PTSANO3.1 size=878346 2024-04-07T16:37:57.252 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.260+0000 7f346a3868c0 1 < completed multipart upload key=95YVKJ14JD8W upload=L4UK2PTSANO3 tag=X7S1WDEIS2YM type=write size=2975498 2024-04-07T16:37:57.262 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.268+0000 7f346a3868c0 1 > prepared part key=_multipart_NYET3212USVE.2~ZHUIC3JYU01W.0 size=1048576 2024-04-07T16:37:57.266 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.272+0000 7f346a3868c0 1 > prepared part key=_multipart_NYET3212USVE.2~ZHUIC3JYU01W.1 size=1048576 2024-04-07T16:37:57.270 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.276+0000 7f346a3868c0 1 > prepared part key=_multipart_NYET3212USVE.2~ZHUIC3JYU01W.2 size=870588 2024-04-07T16:37:57.270 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.276+0000 7f346a3868c0 1 > prepared multipart upload key=NYET3212USVE upload=ZHUIC3JYU01W tag=60P5HVYN8N1D type=write size=4016316 2024-04-07T16:37:57.275 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.284+0000 7f346a3868c0 1 < completed delete operation key=1VBSUWNI3V1T tag=AOZDP7MJSB5I type=del 2024-04-07T16:37:57.280 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.288+0000 7f346a3868c0 1 > prepared operation key=DXTO3AJ429AC tag=T9R2V3FPFBWS type=del size=3339771 2024-04-07T16:37:57.286 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.292+0000 7f346a3868c0 1 < completed part key=_multipart_C1BTVJNHIM0X.2~2Y2VUNPJT25M.0 size=277548 2024-04-07T16:37:57.294 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.300+0000 7f346a3868c0 1 < completed multipart upload key=C1BTVJNHIM0X upload=2Y2VUNPJT25M tag=YNIMQ1PO2N2N type=write size=1326124 2024-04-07T16:37:57.299 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.308+0000 7f346a3868c0 1 > prepared operation key=YOK8BAEOMPUR tag=PWQMNNUUT8IU type=del size=2978054 2024-04-07T16:37:57.306 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.312+0000 7f346a3868c0 -1 < failed to complete operation key=IP8YW2A75TCW tag=RLCRNGW6SOCA type=del size=2399590: (22) Invalid argument 2024-04-07T16:37:57.316 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.324+0000 7f346a3868c0 1 > prepared part key=_multipart_U3EID5EK64HD.2~N6UHWYT2HU63.0 size=1048576 2024-04-07T16:37:57.322 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.328+0000 7f346a3868c0 1 > prepared part key=_multipart_U3EID5EK64HD.2~N6UHWYT2HU63.1 size=2389 2024-04-07T16:37:57.322 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.328+0000 7f346a3868c0 1 > prepared multipart upload key=U3EID5EK64HD upload=N6UHWYT2HU63 tag=UJSHHKZ2G455 type=write size=2099541 2024-04-07T16:37:57.327 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.332+0000 7f346a3868c0 1 < completed delete operation key=X5Q46GO8YTOK tag=5UVUBPD4TU05 type=del 2024-04-07T16:37:57.332 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.340+0000 7f346a3868c0 1 > prepared operation key=1WGUPH7X5UA0 tag=5YUK0A3DQJOR type=del size=3682890 2024-04-07T16:37:57.337 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.344+0000 7f346a3868c0 1 < completed part key=_multipart_DXTO3AJ429AC.2~O1GJK3Z1K31O.0 size=1028110 2024-04-07T16:37:57.345 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.352+0000 7f346a3868c0 1 < completed multipart upload key=DXTO3AJ429AC upload=O1GJK3Z1K31O tag=P7YZEYJCDVU0 type=write size=2076686 2024-04-07T16:37:57.356 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.364+0000 7f346a3868c0 1 > prepared part key=_multipart_YXMDZB23YTB3.2~FRT46I5ISNT3.0 size=623826 2024-04-07T16:37:57.356 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.364+0000 7f346a3868c0 1 > prepared multipart upload key=YXMDZB23YTB3 upload=FRT46I5ISNT3 tag=8CRM3CFOS39D type=write size=1672402 2024-04-07T16:37:57.363 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.372+0000 7f346a3868c0 1 < completed part key=_multipart_7G5C6HBK6QWI.2~QHCMJ50MX3E7.0 size=1048576 2024-04-07T16:37:57.368 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.376+0000 7f346a3868c0 1 < completed part key=_multipart_7G5C6HBK6QWI.2~QHCMJ50MX3E7.1 size=1048576 2024-04-07T16:37:57.372 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.380+0000 7f346a3868c0 1 < completed part key=_multipart_7G5C6HBK6QWI.2~QHCMJ50MX3E7.2 size=810752 2024-04-07T16:37:57.381 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.388+0000 7f346a3868c0 1 < completed multipart upload key=7G5C6HBK6QWI upload=QHCMJ50MX3E7 tag=RIX5BC3JWUR2 type=write size=3956480 2024-04-07T16:37:57.386 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.392+0000 7f346a3868c0 1 > prepared operation key=HBLEXONKXJTL tag=8JCJHKJ294K7 type=write size=511158 2024-04-07T16:37:57.397 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.404+0000 7f346a3868c0 1 < suggested remove operation key=YXMDZB23YTB3 tag=HZ7YY25RMST0 type=del 2024-04-07T16:37:57.402 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.408+0000 7f346a3868c0 1 > prepared operation key=YOK8BAEOMPUR tag=1K8DVQIKA5PQ type=del size=3020694 2024-04-07T16:37:57.408 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.416+0000 7f346a3868c0 1 < completed part key=_multipart_DXTO3AJ429AC.2~QO3WT7L0WBU8.0 size=1048576 2024-04-07T16:37:57.412 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.420+0000 7f346a3868c0 1 < completed part key=_multipart_DXTO3AJ429AC.2~QO3WT7L0WBU8.1 size=1048576 2024-04-07T16:37:57.417 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.424+0000 7f346a3868c0 1 < completed part key=_multipart_DXTO3AJ429AC.2~QO3WT7L0WBU8.2 size=877967 2024-04-07T16:37:57.425 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.432+0000 7f346a3868c0 1 < completed multipart upload key=DXTO3AJ429AC upload=QO3WT7L0WBU8 tag=92K0NTLGG2OU type=write size=4023695 2024-04-07T16:37:57.431 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.436+0000 7f346a3868c0 1 > prepared operation key=QTCIIPTXIRW2 tag=ROY4UKMTHLHQ type=del size=1621195 2024-04-07T16:37:57.436 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.444+0000 7f346a3868c0 1 < completed write operation key=1VKVQXSQMTU1 tag=WLKOMWFGF1D2 type=write size=18830 2024-04-07T16:37:57.441 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.448+0000 7f346a3868c0 1 > prepared operation key=IX3F0IUOKBX1 tag=KIC7RPQXJIE2 type=del size=3668701 2024-04-07T16:37:57.451 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.460+0000 7f346a3868c0 1 < suggested update operation key=HBLEXONKXJTL tag=22SOMIWCMX81 type=del size=2714790 2024-04-07T16:37:57.456 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.464+0000 7f346a3868c0 1 > prepared operation key=IX3F0IUOKBX1 tag=DP0P1ZYLH6DN type=del size=2586675 2024-04-07T16:37:57.462 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.468+0000 7f346a3868c0 1 < completed part key=_multipart_IX3F0IUOKBX1.2~LBQFSGM9OKU4.0 size=1048576 2024-04-07T16:37:57.467 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.476+0000 7f346a3868c0 1 < completed part key=_multipart_IX3F0IUOKBX1.2~LBQFSGM9OKU4.1 size=1048576 2024-04-07T16:37:57.472 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.480+0000 7f346a3868c0 1 < completed part key=_multipart_IX3F0IUOKBX1.2~LBQFSGM9OKU4.2 size=565061 2024-04-07T16:37:57.482 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.488+0000 7f346a3868c0 1 < completed multipart upload key=IX3F0IUOKBX1 upload=LBQFSGM9OKU4 tag=CZ6R1KRXHL27 type=write size=3710789 2024-04-07T16:37:57.491 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.500+0000 7f346a3868c0 1 > prepared part key=_multipart_CL4PP1RKZU0E.2~EDN8I3XDULNL.0 size=1048576 2024-04-07T16:37:57.495 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.500+0000 7f346a3868c0 1 > prepared part key=_multipart_CL4PP1RKZU0E.2~EDN8I3XDULNL.1 size=639953 2024-04-07T16:37:57.495 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.500+0000 7f346a3868c0 1 > prepared multipart upload key=CL4PP1RKZU0E upload=EDN8I3XDULNL tag=YZAUOM4VUSWL type=write size=2737105 2024-04-07T16:37:57.499 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.508+0000 7f346a3868c0 1 < completed part key=_multipart_QS4HYZFFAJNF.2~9TDZWFF7HSGZ.0 size=1048576 2024-04-07T16:37:57.499 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.508+0000 7f346a3868c0 1 < canceled part key=_multipart_QS4HYZFFAJNF.2~9TDZWFF7HSGZ.1 size=1048576 2024-04-07T16:37:57.503 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.512+0000 7f346a3868c0 1 < completed part key=_multipart_QS4HYZFFAJNF.2~9TDZWFF7HSGZ.2 size=277535 2024-04-07T16:37:57.511 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.520+0000 7f346a3868c0 1 < canceled multipart upload key=QS4HYZFFAJNF upload=9TDZWFF7HSGZ tag=0RWIKRZARPAR type=write size=3423263 2024-04-07T16:37:57.516 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.524+0000 7f346a3868c0 1 > prepared operation key=E84U0SX44NQ8 tag=70XXRS0MLRSW type=del size=1991371 2024-04-07T16:37:57.521 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.528+0000 7f346a3868c0 1 < canceled operation key=8F81G0P0YBFW tag=7Z072AE857QN type=del size=2273829 2024-04-07T16:37:57.526 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.532+0000 7f346a3868c0 1 > prepared operation key=E84U0SX44NQ8 tag=ZGQYGBKVVMYK type=del size=4045357 2024-04-07T16:37:57.531 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.540+0000 7f346a3868c0 1 < completed delete operation key=ROFH30W3UCAX tag=SCK8I2NPBLHK type=del 2024-04-07T16:37:57.536 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.544+0000 7f346a3868c0 1 > prepared operation key=DXTO3AJ429AC tag=2EQ3TFJVXTQO type=del size=2403191 2024-04-07T16:37:57.544 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.552+0000 7f346a3868c0 -1 < failed to complete operation key=HBLEXONKXJTL tag=USCSB6XM6CBF type=del size=2152941: (22) Invalid argument 2024-04-07T16:37:57.549 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.556+0000 7f346a3868c0 1 > prepared operation key=YXMDZB23YTB3 tag=JHTIJRCVSY1Z type=del size=995638 2024-04-07T16:37:57.555 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.560+0000 7f346a3868c0 1 < completed part key=_multipart_NYET3212USVE.2~ZHUIC3JYU01W.0 size=1048576 2024-04-07T16:37:57.559 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.564+0000 7f346a3868c0 1 < completed part key=_multipart_NYET3212USVE.2~ZHUIC3JYU01W.1 size=1048576 2024-04-07T16:37:57.563 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.572+0000 7f346a3868c0 1 < completed part key=_multipart_NYET3212USVE.2~ZHUIC3JYU01W.2 size=870588 2024-04-07T16:37:57.571 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.580+0000 7f346a3868c0 1 < completed multipart upload key=NYET3212USVE upload=ZHUIC3JYU01W tag=60P5HVYN8N1D type=write size=4016316 2024-04-07T16:37:57.576 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.584+0000 7f346a3868c0 1 > prepared operation key=95YVKJ14JD8W tag=1A4UBBGZ48IB type=del size=2391667 2024-04-07T16:37:57.581 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.588+0000 7f346a3868c0 1 < completed delete operation key=DXTO3AJ429AC tag=T9R2V3FPFBWS type=del 2024-04-07T16:37:57.587 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.592+0000 7f346a3868c0 1 > prepared operation key=HBLEXONKXJTL tag=RODAB6CMW21S type=write size=6901 2024-04-07T16:37:57.592 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.600+0000 7f346a3868c0 1 < completed delete operation key=YOK8BAEOMPUR tag=PWQMNNUUT8IU type=del 2024-04-07T16:37:57.597 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.604+0000 7f346a3868c0 1 > prepared operation key=YXMDZB23YTB3 tag=C1F8B42AP68A type=del size=841470 2024-04-07T16:37:57.603 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.608+0000 7f346a3868c0 1 < completed part key=_multipart_U3EID5EK64HD.2~N6UHWYT2HU63.0 size=1048576 2024-04-07T16:37:57.603 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.608+0000 7f346a3868c0 1 < canceled part key=_multipart_U3EID5EK64HD.2~N6UHWYT2HU63.1 size=2389 2024-04-07T16:37:57.611 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.620+0000 7f346a3868c0 1 < canceled multipart upload key=U3EID5EK64HD upload=N6UHWYT2HU63 tag=UJSHHKZ2G455 type=write size=2099541 2024-04-07T16:37:57.618 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.624+0000 7f346a3868c0 1 > prepared operation key=XJLQZ7IS9U1T tag=QZX0B9E94R54 type=del size=800947 2024-04-07T16:37:57.629 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.636+0000 7f346a3868c0 1 < suggested remove operation key=1WGUPH7X5UA0 tag=5YUK0A3DQJOR type=del 2024-04-07T16:37:57.639 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.648+0000 7f346a3868c0 1 > prepared part key=_multipart_8F81G0P0YBFW.2~VQHRX4PLMUUV.0 size=1048576 2024-04-07T16:37:57.644 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.652+0000 7f346a3868c0 1 > prepared part key=_multipart_8F81G0P0YBFW.2~VQHRX4PLMUUV.1 size=645595 2024-04-07T16:37:57.644 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.652+0000 7f346a3868c0 1 > prepared multipart upload key=8F81G0P0YBFW upload=VQHRX4PLMUUV tag=IC277WGI96VI type=write size=2742747 2024-04-07T16:37:57.650 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.656+0000 7f346a3868c0 1 < completed part key=_multipart_YXMDZB23YTB3.2~FRT46I5ISNT3.0 size=623826 2024-04-07T16:37:57.659 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.668+0000 7f346a3868c0 1 < completed multipart upload key=YXMDZB23YTB3 upload=FRT46I5ISNT3 tag=8CRM3CFOS39D type=write size=1672402 2024-04-07T16:37:57.670 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.676+0000 7f346a3868c0 1 > prepared part key=_multipart_NYET3212USVE.2~CXLD0CT54ICR.0 size=1048576 2024-04-07T16:37:57.675 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.680+0000 7f346a3868c0 1 > prepared part key=_multipart_NYET3212USVE.2~CXLD0CT54ICR.1 size=742578 2024-04-07T16:37:57.675 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.680+0000 7f346a3868c0 1 > prepared multipart upload key=NYET3212USVE upload=CXLD0CT54ICR tag=NM062RR2L283 type=write size=2839730 2024-04-07T16:37:57.682 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.688+0000 7f346a3868c0 -1 < failed to complete operation key=HBLEXONKXJTL tag=8JCJHKJ294K7 type=write size=511158: (22) Invalid argument 2024-04-07T16:37:57.688 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.696+0000 7f346a3868c0 1 > prepared operation key=GBQSME7SY1AH tag=G6CSIMMVJ5SP type=del size=3116233 2024-04-07T16:37:57.694 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.700+0000 7f346a3868c0 1 < completed delete operation key=YOK8BAEOMPUR tag=1K8DVQIKA5PQ type=del 2024-04-07T16:37:57.700 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.708+0000 7f346a3868c0 1 > prepared operation key=DXTO3AJ429AC tag=Y2BIL35GA8EH type=del size=1600305 2024-04-07T16:37:57.707 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.712+0000 7f346a3868c0 1 < canceled operation key=QTCIIPTXIRW2 tag=ROY4UKMTHLHQ type=del size=1621195 2024-04-07T16:37:57.712 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.720+0000 7f346a3868c0 1 > prepared operation key=QS4HYZFFAJNF tag=YXHLWNZ9HV7S type=del size=3043327 2024-04-07T16:37:57.717 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.724+0000 7f346a3868c0 1 < canceled operation key=IX3F0IUOKBX1 tag=KIC7RPQXJIE2 type=del size=3668701 2024-04-07T16:37:57.723 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.732+0000 7f346a3868c0 1 > prepared operation key=GZXHYZO5SMUP tag=V0LBL0RRJKOG type=del size=889756 2024-04-07T16:37:57.729 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.736+0000 7f346a3868c0 1 < completed delete operation key=IX3F0IUOKBX1 tag=DP0P1ZYLH6DN type=del 2024-04-07T16:37:57.734 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.740+0000 7f346a3868c0 1 > prepared operation key=YOK8BAEOMPUR tag=NY3ZD10SKUGY type=del size=1100450 2024-04-07T16:37:57.736 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.744+0000 7f346a3868c0 1 < canceled part key=_multipart_CL4PP1RKZU0E.2~EDN8I3XDULNL.0 size=1048576 2024-04-07T16:37:57.741 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.748+0000 7f346a3868c0 1 < completed part key=_multipart_CL4PP1RKZU0E.2~EDN8I3XDULNL.1 size=639953 2024-04-07T16:37:57.753 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.760+0000 7f346a3868c0 1 < canceled multipart upload key=CL4PP1RKZU0E upload=EDN8I3XDULNL tag=YZAUOM4VUSWL type=write size=2737105 2024-04-07T16:37:57.758 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.764+0000 7f346a3868c0 1 > prepared operation key=ROFH30W3UCAX tag=8G8VXR3G764C type=del size=3860345 2024-04-07T16:37:57.763 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.772+0000 7f346a3868c0 1 < completed delete operation key=E84U0SX44NQ8 tag=70XXRS0MLRSW type=del 2024-04-07T16:37:57.769 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.776+0000 7f346a3868c0 1 > prepared operation key=N14BUXLJONHZ tag=EKYUBCU9JB8W type=del size=1602179 2024-04-07T16:37:57.775 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.784+0000 7f346a3868c0 1 < completed delete operation key=E84U0SX44NQ8 tag=ZGQYGBKVVMYK type=del 2024-04-07T16:37:57.781 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.788+0000 7f346a3868c0 1 > prepared operation key=IX3F0IUOKBX1 tag=DTQS3HSS6SOT type=del size=1700708 2024-04-07T16:37:57.787 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.796+0000 7f346a3868c0 1 < completed delete operation key=DXTO3AJ429AC tag=2EQ3TFJVXTQO type=del 2024-04-07T16:37:57.793 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.800+0000 7f346a3868c0 1 > prepared operation key=X5Q46GO8YTOK tag=OM2STWI0LMLF type=write size=761273 2024-04-07T16:37:57.799 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.808+0000 7f346a3868c0 1 < completed delete operation key=YXMDZB23YTB3 tag=JHTIJRCVSY1Z type=del 2024-04-07T16:37:57.806 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.812+0000 7f346a3868c0 1 > prepared operation key=ROFH30W3UCAX tag=5WOULSLETHY9 type=del size=190367 2024-04-07T16:37:57.812 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.820+0000 7f346a3868c0 1 < completed delete operation key=95YVKJ14JD8W tag=1A4UBBGZ48IB type=del 2024-04-07T16:37:57.817 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.824+0000 7f346a3868c0 1 > prepared operation key=GBQSME7SY1AH tag=RUQ61LX3EHUT type=del size=3557445 2024-04-07T16:37:57.829 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.836+0000 7f346a3868c0 1 < suggested update operation key=HBLEXONKXJTL tag=RODAB6CMW21S type=write size=6901 2024-04-07T16:37:57.839 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.848+0000 7f346a3868c0 1 > prepared part key=_multipart_8F81G0P0YBFW.2~P9QD6VRIWV2M.0 size=910028 2024-04-07T16:37:57.839 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.848+0000 7f346a3868c0 1 > prepared multipart upload key=8F81G0P0YBFW upload=P9QD6VRIWV2M tag=LQ5RJWC19EXW type=write size=1958604 2024-04-07T16:37:57.845 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.852+0000 7f346a3868c0 1 < completed delete operation key=YXMDZB23YTB3 tag=C1F8B42AP68A type=del 2024-04-07T16:37:57.852 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.860+0000 7f346a3868c0 1 > prepared operation key=95YVKJ14JD8W tag=TFJ8B5XNMZ8P type=del size=1440715 2024-04-07T16:37:57.858 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.864+0000 7f346a3868c0 1 < completed delete operation key=XJLQZ7IS9U1T tag=QZX0B9E94R54 type=del 2024-04-07T16:37:57.868 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.876+0000 7f346a3868c0 1 > prepared part key=_multipart_6XDS0A7L0IY8.2~PCJRJRB3JIQC.0 size=1025022 2024-04-07T16:37:57.868 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.876+0000 7f346a3868c0 1 > prepared multipart upload key=6XDS0A7L0IY8 upload=PCJRJRB3JIQC tag=WUDEBK275WTD type=write size=2073598 2024-04-07T16:37:57.874 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.880+0000 7f346a3868c0 1 < completed part key=_multipart_8F81G0P0YBFW.2~VQHRX4PLMUUV.0 size=1048576 2024-04-07T16:37:57.880 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.888+0000 7f346a3868c0 1 < completed part key=_multipart_8F81G0P0YBFW.2~VQHRX4PLMUUV.1 size=645595 2024-04-07T16:37:57.890 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.896+0000 7f346a3868c0 1 < completed multipart upload key=8F81G0P0YBFW upload=VQHRX4PLMUUV tag=IC277WGI96VI type=write size=2742747 2024-04-07T16:37:57.895 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.900+0000 7f346a3868c0 1 > prepared operation key=CL4PP1RKZU0E tag=RRJQWC0UA2N2 type=del size=2588407 2024-04-07T16:37:57.901 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.908+0000 7f346a3868c0 1 < completed part key=_multipart_NYET3212USVE.2~CXLD0CT54ICR.0 size=1048576 2024-04-07T16:37:57.905 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.912+0000 7f346a3868c0 1 < completed part key=_multipart_NYET3212USVE.2~CXLD0CT54ICR.1 size=742578 2024-04-07T16:37:57.915 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.924+0000 7f346a3868c0 1 < completed multipart upload key=NYET3212USVE upload=CXLD0CT54ICR tag=NM062RR2L283 type=write size=2839730 2024-04-07T16:37:57.922 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.928+0000 7f346a3868c0 1 > prepared operation key=EQR0FY86AT5X tag=UQMFU2V2MU11 type=del size=2985376 2024-04-07T16:37:57.928 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.936+0000 7f346a3868c0 1 < completed delete operation key=GBQSME7SY1AH tag=G6CSIMMVJ5SP type=del 2024-04-07T16:37:57.933 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.940+0000 7f346a3868c0 1 > prepared operation key=DXTO3AJ429AC tag=OMT65AP9UBZA type=del size=369669 2024-04-07T16:37:57.939 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.948+0000 7f346a3868c0 1 < completed delete operation key=DXTO3AJ429AC tag=Y2BIL35GA8EH type=del 2024-04-07T16:37:57.952 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.960+0000 7f346a3868c0 1 > prepared part key=_multipart_7G5C6HBK6QWI.2~YOQCZES0J1YR.0 size=1048576 2024-04-07T16:37:57.957 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.964+0000 7f346a3868c0 1 > prepared part key=_multipart_7G5C6HBK6QWI.2~YOQCZES0J1YR.1 size=1023378 2024-04-07T16:37:57.957 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.964+0000 7f346a3868c0 1 > prepared multipart upload key=7G5C6HBK6QWI upload=YOQCZES0J1YR tag=40OC6GUAPPIG type=write size=3120530 2024-04-07T16:37:57.964 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.972+0000 7f346a3868c0 1 < completed delete operation key=QS4HYZFFAJNF tag=YXHLWNZ9HV7S type=del 2024-04-07T16:37:57.974 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.980+0000 7f346a3868c0 1 > prepared part key=_multipart_IP8YW2A75TCW.2~1PZC48ZIYBRO.0 size=1048576 2024-04-07T16:37:57.979 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.988+0000 7f346a3868c0 1 > prepared part key=_multipart_IP8YW2A75TCW.2~1PZC48ZIYBRO.1 size=1048576 2024-04-07T16:37:57.983 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.992+0000 7f346a3868c0 1 > prepared part key=_multipart_IP8YW2A75TCW.2~1PZC48ZIYBRO.2 size=554256 2024-04-07T16:37:57.984 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:57.992+0000 7f346a3868c0 1 > prepared multipart upload key=IP8YW2A75TCW upload=1PZC48ZIYBRO tag=R4307G5SAKGL type=write size=3699984 2024-04-07T16:37:57.995 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.004+0000 7f346a3868c0 1 < suggested remove operation key=GZXHYZO5SMUP tag=V0LBL0RRJKOG type=del 2024-04-07T16:37:58.000 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.008+0000 7f346a3868c0 1 > prepared operation key=QS4HYZFFAJNF tag=6QHI82FVLYPM type=del size=1359854 2024-04-07T16:37:58.006 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.012+0000 7f346a3868c0 1 < completed delete operation key=YOK8BAEOMPUR tag=NY3ZD10SKUGY type=del 2024-04-07T16:37:58.017 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.024+0000 7f346a3868c0 1 > prepared part key=_multipart_UIW3DN4NUYPP.2~8KHY6NQKDQDX.0 size=1048576 2024-04-07T16:37:58.021 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.028+0000 7f346a3868c0 1 > prepared part key=_multipart_UIW3DN4NUYPP.2~8KHY6NQKDQDX.1 size=1048576 2024-04-07T16:37:58.025 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.032+0000 7f346a3868c0 1 > prepared part key=_multipart_UIW3DN4NUYPP.2~8KHY6NQKDQDX.2 size=699394 2024-04-07T16:37:58.025 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.032+0000 7f346a3868c0 1 > prepared multipart upload key=UIW3DN4NUYPP upload=8KHY6NQKDQDX tag=Z00R3E7G3WHP type=write size=3845122 2024-04-07T16:37:58.031 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.036+0000 7f346a3868c0 1 < completed delete operation key=ROFH30W3UCAX tag=8G8VXR3G764C type=del 2024-04-07T16:37:58.038 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.044+0000 7f346a3868c0 1 > prepared operation key=U3EID5EK64HD tag=7QUR4R3K95SQ type=del size=278641 2024-04-07T16:37:58.043 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.052+0000 7f346a3868c0 1 < completed delete operation key=N14BUXLJONHZ tag=EKYUBCU9JB8W type=del 2024-04-07T16:37:58.049 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.056+0000 7f346a3868c0 1 > prepared operation key=ROFH30W3UCAX tag=DVEYILPYT0I2 type=del size=1694464 2024-04-07T16:37:58.054 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.060+0000 7f346a3868c0 1 < completed delete operation key=IX3F0IUOKBX1 tag=DTQS3HSS6SOT type=del 2024-04-07T16:37:58.059 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.068+0000 7f346a3868c0 1 > prepared operation key=IP8YW2A75TCW tag=4XFB3OSMJ524 type=write size=139319 2024-04-07T16:37:58.070 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.076+0000 7f346a3868c0 1 < suggested update operation key=X5Q46GO8YTOK tag=OM2STWI0LMLF type=write size=761273 2024-04-07T16:37:58.081 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.088+0000 7f346a3868c0 1 > prepared part key=_multipart_1WGUPH7X5UA0.2~KBRVWIH3U28S.0 size=606823 2024-04-07T16:37:58.081 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.088+0000 7f346a3868c0 1 > prepared multipart upload key=1WGUPH7X5UA0 upload=KBRVWIH3U28S tag=FVEM0V7FYQ0I type=write size=1655399 2024-04-07T16:37:58.086 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.092+0000 7f346a3868c0 1 < completed delete operation key=ROFH30W3UCAX tag=5WOULSLETHY9 type=del 2024-04-07T16:37:58.097 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.104+0000 7f346a3868c0 1 > prepared part key=_multipart_E84U0SX44NQ8.2~X15H3NET6BFF.0 size=1048576 2024-04-07T16:37:58.101 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.108+0000 7f346a3868c0 1 > prepared part key=_multipart_E84U0SX44NQ8.2~X15H3NET6BFF.1 size=1048576 2024-04-07T16:37:58.105 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.112+0000 7f346a3868c0 1 > prepared part key=_multipart_E84U0SX44NQ8.2~X15H3NET6BFF.2 size=235506 2024-04-07T16:37:58.105 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.112+0000 7f346a3868c0 1 > prepared multipart upload key=E84U0SX44NQ8 upload=X15H3NET6BFF tag=1EQMNP5295VL type=write size=3381234 2024-04-07T16:37:58.111 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.120+0000 7f346a3868c0 1 < completed delete operation key=GBQSME7SY1AH tag=RUQ61LX3EHUT type=del 2024-04-07T16:37:58.122 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.128+0000 7f346a3868c0 1 > prepared part key=_multipart_QTCIIPTXIRW2.2~6QU9S05C0EIU.0 size=487142 2024-04-07T16:37:58.122 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.128+0000 7f346a3868c0 1 > prepared multipart upload key=QTCIIPTXIRW2 upload=6QU9S05C0EIU tag=TP3RAX1FSGH2 type=write size=1535718 2024-04-07T16:37:58.128 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.136+0000 7f346a3868c0 1 < completed part key=_multipart_8F81G0P0YBFW.2~P9QD6VRIWV2M.0 size=910028 2024-04-07T16:37:58.138 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.144+0000 7f346a3868c0 1 < completed multipart upload key=8F81G0P0YBFW upload=P9QD6VRIWV2M tag=LQ5RJWC19EXW type=write size=1958604 2024-04-07T16:37:58.143 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.152+0000 7f346a3868c0 1 > prepared operation key=QVOKSQAGHPA9 tag=O3EL969TLTAM type=del size=1087452 2024-04-07T16:37:58.149 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.156+0000 7f346a3868c0 1 < canceled operation key=95YVKJ14JD8W tag=TFJ8B5XNMZ8P type=del size=1440715 2024-04-07T16:37:58.154 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.160+0000 7f346a3868c0 1 > prepared operation key=GBQSME7SY1AH tag=AD5I9XE65CEG type=del size=2761337 2024-04-07T16:37:58.161 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.168+0000 7f346a3868c0 1 < completed part key=_multipart_6XDS0A7L0IY8.2~PCJRJRB3JIQC.0 size=1025022 2024-04-07T16:37:58.169 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.176+0000 7f346a3868c0 1 < completed multipart upload key=6XDS0A7L0IY8 upload=PCJRJRB3JIQC tag=WUDEBK275WTD type=write size=2073598 2024-04-07T16:37:58.178 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.184+0000 7f346a3868c0 1 > prepared part key=_multipart_YOK8BAEOMPUR.2~G259BT5NNWXB.0 size=1048576 2024-04-07T16:37:58.183 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.192+0000 7f346a3868c0 1 > prepared part key=_multipart_YOK8BAEOMPUR.2~G259BT5NNWXB.1 size=504771 2024-04-07T16:37:58.183 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.192+0000 7f346a3868c0 1 > prepared multipart upload key=YOK8BAEOMPUR upload=G259BT5NNWXB tag=4SNQL0L6HHDW type=write size=2601923 2024-04-07T16:37:58.190 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.196+0000 7f346a3868c0 1 < completed delete operation key=CL4PP1RKZU0E tag=RRJQWC0UA2N2 type=del 2024-04-07T16:37:58.200 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.208+0000 7f346a3868c0 1 > prepared part key=_multipart_E84U0SX44NQ8.2~UWF8DSZB8K2Z.0 size=1048576 2024-04-07T16:37:58.205 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.212+0000 7f346a3868c0 1 > prepared part key=_multipart_E84U0SX44NQ8.2~UWF8DSZB8K2Z.1 size=865257 2024-04-07T16:37:58.205 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.212+0000 7f346a3868c0 1 > prepared multipart upload key=E84U0SX44NQ8 upload=UWF8DSZB8K2Z tag=WXXSA9BB02KD type=write size=2962409 2024-04-07T16:37:58.210 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.216+0000 7f346a3868c0 1 < completed delete operation key=EQR0FY86AT5X tag=UQMFU2V2MU11 type=del 2024-04-07T16:37:58.216 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.224+0000 7f346a3868c0 1 > prepared operation key=7G5C6HBK6QWI tag=5BLR8ZH1X450 type=del size=585169 2024-04-07T16:37:58.226 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.232+0000 7f346a3868c0 1 < suggested remove operation key=DXTO3AJ429AC tag=OMT65AP9UBZA type=del 2024-04-07T16:37:58.231 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.240+0000 7f346a3868c0 1 > prepared operation key=CL4PP1RKZU0E tag=A31KUMHS9AI4 type=write size=573076 2024-04-07T16:37:58.237 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.244+0000 7f346a3868c0 1 < completed part key=_multipart_7G5C6HBK6QWI.2~YOQCZES0J1YR.0 size=1048576 2024-04-07T16:37:58.241 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.248+0000 7f346a3868c0 1 < completed part key=_multipart_7G5C6HBK6QWI.2~YOQCZES0J1YR.1 size=1023378 2024-04-07T16:37:58.252 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.260+0000 7f346a3868c0 1 < completed multipart upload key=7G5C6HBK6QWI upload=YOQCZES0J1YR tag=40OC6GUAPPIG type=write size=3120530 2024-04-07T16:37:58.263 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.268+0000 7f346a3868c0 1 > prepared part key=_multipart_1VBSUWNI3V1T.2~IY7O7XM35WLH.0 size=765482 2024-04-07T16:37:58.263 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.268+0000 7f346a3868c0 1 > prepared multipart upload key=1VBSUWNI3V1T upload=IY7O7XM35WLH tag=DGSXF3MOOBKW type=write size=1814058 2024-04-07T16:37:58.268 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.276+0000 7f346a3868c0 1 < completed part key=_multipart_IP8YW2A75TCW.2~1PZC48ZIYBRO.0 size=1048576 2024-04-07T16:37:58.274 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.280+0000 7f346a3868c0 1 < completed part key=_multipart_IP8YW2A75TCW.2~1PZC48ZIYBRO.1 size=1048576 2024-04-07T16:37:58.280 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.288+0000 7f346a3868c0 1 < completed part key=_multipart_IP8YW2A75TCW.2~1PZC48ZIYBRO.2 size=554256 2024-04-07T16:37:58.289 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.296+0000 7f346a3868c0 1 < completed multipart upload key=IP8YW2A75TCW upload=1PZC48ZIYBRO tag=R4307G5SAKGL type=write size=3699984 2024-04-07T16:37:58.294 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.300+0000 7f346a3868c0 1 > prepared operation key=1WGUPH7X5UA0 tag=Y00ORT5GOYKW type=del size=413707 2024-04-07T16:37:58.299 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.308+0000 7f346a3868c0 1 < completed delete operation key=QS4HYZFFAJNF tag=6QHI82FVLYPM type=del 2024-04-07T16:37:58.311 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.320+0000 7f346a3868c0 1 > prepared part key=_multipart_XJLQZ7IS9U1T.2~IMHN2VN0S82T.0 size=248064 2024-04-07T16:37:58.311 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.320+0000 7f346a3868c0 1 > prepared multipart upload key=XJLQZ7IS9U1T upload=IMHN2VN0S82T tag=JOJRYHX1YBZ5 type=write size=1296640 2024-04-07T16:37:58.317 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.324+0000 7f346a3868c0 1 < completed part key=_multipart_UIW3DN4NUYPP.2~8KHY6NQKDQDX.0 size=1048576 2024-04-07T16:37:58.322 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.328+0000 7f346a3868c0 1 < completed part key=_multipart_UIW3DN4NUYPP.2~8KHY6NQKDQDX.1 size=1048576 2024-04-07T16:37:58.327 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.336+0000 7f346a3868c0 1 < completed part key=_multipart_UIW3DN4NUYPP.2~8KHY6NQKDQDX.2 size=699394 2024-04-07T16:37:58.336 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.344+0000 7f346a3868c0 1 < completed multipart upload key=UIW3DN4NUYPP upload=8KHY6NQKDQDX tag=Z00R3E7G3WHP type=write size=3845122 2024-04-07T16:37:58.346 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.352+0000 7f346a3868c0 1 > prepared part key=_multipart_UIW3DN4NUYPP.2~CHDDUSDVTEPC.0 size=1048576 2024-04-07T16:37:58.351 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.356+0000 7f346a3868c0 1 > prepared part key=_multipart_UIW3DN4NUYPP.2~CHDDUSDVTEPC.1 size=1048576 2024-04-07T16:37:58.357 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.364+0000 7f346a3868c0 1 > prepared part key=_multipart_UIW3DN4NUYPP.2~CHDDUSDVTEPC.2 size=413747 2024-04-07T16:37:58.357 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.364+0000 7f346a3868c0 1 > prepared multipart upload key=UIW3DN4NUYPP upload=CHDDUSDVTEPC tag=SY1V55E4XRJL type=write size=3559475 2024-04-07T16:37:58.362 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.368+0000 7f346a3868c0 1 < completed delete operation key=U3EID5EK64HD tag=7QUR4R3K95SQ type=del 2024-04-07T16:37:58.372 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.380+0000 7f346a3868c0 1 > prepared part key=_multipart_8F81G0P0YBFW.2~SMPVSZZS150L.0 size=1048576 2024-04-07T16:37:58.376 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.384+0000 7f346a3868c0 1 > prepared part key=_multipart_8F81G0P0YBFW.2~SMPVSZZS150L.1 size=1048576 2024-04-07T16:37:58.380 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.388+0000 7f346a3868c0 1 > prepared part key=_multipart_8F81G0P0YBFW.2~SMPVSZZS150L.2 size=774269 2024-04-07T16:37:58.380 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.388+0000 7f346a3868c0 1 > prepared multipart upload key=8F81G0P0YBFW upload=SMPVSZZS150L tag=4YTGBZBDO8JG type=write size=3919997 2024-04-07T16:37:58.391 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.400+0000 7f346a3868c0 1 < suggested update operation key=ROFH30W3UCAX tag=DVEYILPYT0I2 type=del size=1694464 2024-04-07T16:37:58.397 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.404+0000 7f346a3868c0 1 > prepared operation key=7G5C6HBK6QWI tag=XOSVP086UVLC type=del size=80105 2024-04-07T16:37:58.403 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.408+0000 7f346a3868c0 1 < completed write operation key=IP8YW2A75TCW tag=4XFB3OSMJ524 type=write size=139319 2024-04-07T16:37:58.408 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.416+0000 7f346a3868c0 1 > prepared operation key=HBLEXONKXJTL tag=M25G4JFF8H5E type=del size=1117276 2024-04-07T16:37:58.415 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.424+0000 7f346a3868c0 1 < completed part key=_multipart_1WGUPH7X5UA0.2~KBRVWIH3U28S.0 size=606823 2024-04-07T16:37:58.418 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:37:58 smithi100 bash[74942]: cluster 2024-04-07T16:37:57.062460+0000 mgr.y (mgr.25175) 678 : cluster [DBG] pgmap v1313: 137 pgs: 137 active+clean; 583 KiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 481 KiB/s rd, 167 KiB/s wr, 850 op/s 2024-04-07T16:37:58.419 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:37:58 smithi100 bash[76202]: cluster 2024-04-07T16:37:57.062460+0000 mgr.y (mgr.25175) 678 : cluster [DBG] pgmap v1313: 137 pgs: 137 active+clean; 583 KiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 481 KiB/s rd, 167 KiB/s wr, 850 op/s 2024-04-07T16:37:58.425 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.432+0000 7f346a3868c0 1 < completed multipart upload key=1WGUPH7X5UA0 upload=KBRVWIH3U28S tag=FVEM0V7FYQ0I type=write size=1655399 2024-04-07T16:37:58.430 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.436+0000 7f346a3868c0 1 > prepared operation key=QS4HYZFFAJNF tag=DHJVWEMT2X62 type=write size=470516 2024-04-07T16:37:58.436 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.444+0000 7f346a3868c0 1 < completed part key=_multipart_E84U0SX44NQ8.2~X15H3NET6BFF.0 size=1048576 2024-04-07T16:37:58.441 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.448+0000 7f346a3868c0 1 < completed part key=_multipart_E84U0SX44NQ8.2~X15H3NET6BFF.1 size=1048576 2024-04-07T16:37:58.445 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.452+0000 7f346a3868c0 1 < completed part key=_multipart_E84U0SX44NQ8.2~X15H3NET6BFF.2 size=235506 2024-04-07T16:37:58.455 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.460+0000 7f346a3868c0 1 < completed multipart upload key=E84U0SX44NQ8 upload=X15H3NET6BFF tag=1EQMNP5295VL type=write size=3381234 2024-04-07T16:37:58.460 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.468+0000 7f346a3868c0 1 > prepared operation key=YOK8BAEOMPUR tag=8VGXMYUR72NI type=del size=3550926 2024-04-07T16:37:58.465 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.472+0000 7f346a3868c0 1 < completed part key=_multipart_QTCIIPTXIRW2.2~6QU9S05C0EIU.0 size=487142 2024-04-07T16:37:58.475 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.484+0000 7f346a3868c0 1 < completed multipart upload key=QTCIIPTXIRW2 upload=6QU9S05C0EIU tag=TP3RAX1FSGH2 type=write size=1535718 2024-04-07T16:37:58.485 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.492+0000 7f346a3868c0 1 > prepared part key=_multipart_GZXHYZO5SMUP.2~3ERWFP8MIYAA.0 size=1048576 2024-04-07T16:37:58.490 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.496+0000 7f346a3868c0 1 > prepared part key=_multipart_GZXHYZO5SMUP.2~3ERWFP8MIYAA.1 size=1048576 2024-04-07T16:37:58.497 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.504+0000 7f346a3868c0 1 > prepared part key=_multipart_GZXHYZO5SMUP.2~3ERWFP8MIYAA.2 size=630200 2024-04-07T16:37:58.497 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.504+0000 7f346a3868c0 1 > prepared multipart upload key=GZXHYZO5SMUP upload=3ERWFP8MIYAA tag=A4YTRUF63G41 type=write size=3775928 2024-04-07T16:37:58.503 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.508+0000 7f346a3868c0 1 < completed delete operation key=QVOKSQAGHPA9 tag=O3EL969TLTAM type=del 2024-04-07T16:37:58.509 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.516+0000 7f346a3868c0 1 > prepared operation key=1VBSUWNI3V1T tag=OIDMAMALYNGX type=del size=1139633 2024-04-07T16:37:58.515 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.524+0000 7f346a3868c0 1 < completed delete operation key=GBQSME7SY1AH tag=AD5I9XE65CEG type=del 2024-04-07T16:37:58.521 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.528+0000 7f346a3868c0 1 > prepared operation key=EQR0FY86AT5X tag=L4WEKW9UGC8T type=del size=3325381 2024-04-07T16:37:58.527 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.536+0000 7f346a3868c0 1 < completed part key=_multipart_YOK8BAEOMPUR.2~G259BT5NNWXB.0 size=1048576 2024-04-07T16:37:58.532 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.540+0000 7f346a3868c0 1 < completed part key=_multipart_YOK8BAEOMPUR.2~G259BT5NNWXB.1 size=504771 2024-04-07T16:37:58.541 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.548+0000 7f346a3868c0 1 < completed multipart upload key=YOK8BAEOMPUR upload=G259BT5NNWXB tag=4SNQL0L6HHDW type=write size=2601923 2024-04-07T16:37:58.547 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.552+0000 7f346a3868c0 1 > prepared operation key=QTCIIPTXIRW2 tag=N37PQ1D1W2A0 type=del size=3423271 2024-04-07T16:37:58.552 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.560+0000 7f346a3868c0 1 < completed part key=_multipart_E84U0SX44NQ8.2~UWF8DSZB8K2Z.0 size=1048576 2024-04-07T16:37:58.556 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.564+0000 7f346a3868c0 1 < completed part key=_multipart_E84U0SX44NQ8.2~UWF8DSZB8K2Z.1 size=865257 2024-04-07T16:37:58.565 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.572+0000 7f346a3868c0 1 < completed multipart upload key=E84U0SX44NQ8 upload=UWF8DSZB8K2Z tag=WXXSA9BB02KD type=write size=2962409 2024-04-07T16:37:58.570 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.576+0000 7f346a3868c0 1 > prepared operation key=EOYFVEI50SQ1 tag=ZH6IYUC0YW2N type=del size=2402155 2024-04-07T16:37:58.576 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.584+0000 7f346a3868c0 1 < completed delete operation key=7G5C6HBK6QWI tag=5BLR8ZH1X450 type=del 2024-04-07T16:37:58.581 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.588+0000 7f346a3868c0 1 > prepared operation key=8F81G0P0YBFW tag=2ZP1IZCCHHL2 type=write size=443960 2024-04-07T16:37:58.587 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.596+0000 7f346a3868c0 1 < completed write operation key=CL4PP1RKZU0E tag=A31KUMHS9AI4 type=write size=573076 2024-04-07T16:37:58.592 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.600+0000 7f346a3868c0 1 > prepared operation key=E84U0SX44NQ8 tag=3IGGD4F1TIG3 type=del size=3536283 2024-04-07T16:37:58.599 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.604+0000 7f346a3868c0 1 < completed part key=_multipart_1VBSUWNI3V1T.2~IY7O7XM35WLH.0 size=765482 2024-04-07T16:37:58.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:37:58 smithi144 bash[66654]: cluster 2024-04-07T16:37:57.062460+0000 mgr.y (mgr.25175) 678 : cluster [DBG] pgmap v1313: 137 pgs: 137 active+clean; 583 KiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 481 KiB/s rd, 167 KiB/s wr, 850 op/s 2024-04-07T16:37:58.608 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.616+0000 7f346a3868c0 1 < completed multipart upload key=1VBSUWNI3V1T upload=IY7O7XM35WLH tag=DGSXF3MOOBKW type=write size=1814058 2024-04-07T16:37:58.613 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.620+0000 7f346a3868c0 1 > prepared operation key=8F81G0P0YBFW tag=4BD1P7JYVZ0A type=del size=1584962 2024-04-07T16:37:58.618 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.624+0000 7f346a3868c0 1 < completed delete operation key=1WGUPH7X5UA0 tag=Y00ORT5GOYKW type=del 2024-04-07T16:37:58.623 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.632+0000 7f346a3868c0 1 > prepared operation key=QVOKSQAGHPA9 tag=2WY3WJVUVR5G type=del size=1055851 2024-04-07T16:37:58.629 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.636+0000 7f346a3868c0 1 < completed part key=_multipart_XJLQZ7IS9U1T.2~IMHN2VN0S82T.0 size=248064 2024-04-07T16:37:58.638 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.644+0000 7f346a3868c0 1 < completed multipart upload key=XJLQZ7IS9U1T upload=IMHN2VN0S82T tag=JOJRYHX1YBZ5 type=write size=1296640 2024-04-07T16:37:58.643 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.652+0000 7f346a3868c0 1 > prepared operation key=XJLQZ7IS9U1T tag=IPV8ZMXYGRT2 type=del size=1247404 2024-04-07T16:37:58.650 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.656+0000 7f346a3868c0 1 < completed part key=_multipart_UIW3DN4NUYPP.2~CHDDUSDVTEPC.0 size=1048576 2024-04-07T16:37:58.656 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.664+0000 7f346a3868c0 1 < completed part key=_multipart_UIW3DN4NUYPP.2~CHDDUSDVTEPC.1 size=1048576 2024-04-07T16:37:58.661 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.668+0000 7f346a3868c0 1 < completed part key=_multipart_UIW3DN4NUYPP.2~CHDDUSDVTEPC.2 size=413747 2024-04-07T16:37:58.670 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.676+0000 7f346a3868c0 1 < completed multipart upload key=UIW3DN4NUYPP upload=CHDDUSDVTEPC tag=SY1V55E4XRJL type=write size=3559475 2024-04-07T16:37:58.676 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.684+0000 7f346a3868c0 1 > prepared operation key=LXOZ0T2KY6OR tag=TYQWIWEUP4DE type=del size=475116 2024-04-07T16:37:58.683 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.688+0000 7f346a3868c0 1 < completed part key=_multipart_8F81G0P0YBFW.2~SMPVSZZS150L.0 size=1048576 2024-04-07T16:37:58.688 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.696+0000 7f346a3868c0 1 < completed part key=_multipart_8F81G0P0YBFW.2~SMPVSZZS150L.1 size=1048576 2024-04-07T16:37:58.690 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:37:58 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:37:58] "GET /metrics HTTP/1.1" 200 36370 "" "Prometheus/2.43.0" 2024-04-07T16:37:58.692 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.700+0000 7f346a3868c0 1 < completed part key=_multipart_8F81G0P0YBFW.2~SMPVSZZS150L.2 size=774269 2024-04-07T16:37:58.701 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.708+0000 7f346a3868c0 1 < completed multipart upload key=8F81G0P0YBFW upload=SMPVSZZS150L tag=4YTGBZBDO8JG type=write size=3919997 2024-04-07T16:37:58.707 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.712+0000 7f346a3868c0 1 > prepared operation key=ROFH30W3UCAX tag=158NI2T7W1BU type=write size=67970 2024-04-07T16:37:58.712 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.720+0000 7f346a3868c0 1 < completed delete operation key=7G5C6HBK6QWI tag=XOSVP086UVLC type=del 2024-04-07T16:37:58.718 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.724+0000 7f346a3868c0 1 > prepared operation key=ROFH30W3UCAX tag=WH2UA4PAJYV2 type=write size=664326 2024-04-07T16:37:58.723 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.732+0000 7f346a3868c0 1 < completed delete operation key=HBLEXONKXJTL tag=M25G4JFF8H5E type=del 2024-04-07T16:37:58.728 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.736+0000 7f346a3868c0 1 > prepared operation key=N14BUXLJONHZ tag=PDII63WR5IXG type=write size=692324 2024-04-07T16:37:58.734 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.740+0000 7f346a3868c0 1 < completed write operation key=QS4HYZFFAJNF tag=DHJVWEMT2X62 type=write size=470516 2024-04-07T16:37:58.739 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.748+0000 7f346a3868c0 1 > prepared operation key=3N0QJRYVO3FV tag=1HHG7UNI1N40 type=write size=824843 2024-04-07T16:37:58.745 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.752+0000 7f346a3868c0 1 < completed delete operation key=YOK8BAEOMPUR tag=8VGXMYUR72NI type=del 2024-04-07T16:37:58.755 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.764+0000 7f346a3868c0 1 > prepared part key=_multipart_NYET3212USVE.2~WKU3A99PKVSI.0 size=1048576 2024-04-07T16:37:58.759 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.768+0000 7f346a3868c0 1 > prepared part key=_multipart_NYET3212USVE.2~WKU3A99PKVSI.1 size=1048576 2024-04-07T16:37:58.764 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.772+0000 7f346a3868c0 1 > prepared part key=_multipart_NYET3212USVE.2~WKU3A99PKVSI.2 size=140255 2024-04-07T16:37:58.764 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.772+0000 7f346a3868c0 1 > prepared multipart upload key=NYET3212USVE upload=WKU3A99PKVSI tag=I5EHH6NU0BGD type=write size=3285983 2024-04-07T16:37:58.771 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.776+0000 7f346a3868c0 1 < completed part key=_multipart_GZXHYZO5SMUP.2~3ERWFP8MIYAA.0 size=1048576 2024-04-07T16:37:58.771 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.776+0000 7f346a3868c0 1 < canceled part key=_multipart_GZXHYZO5SMUP.2~3ERWFP8MIYAA.1 size=1048576 2024-04-07T16:37:58.777 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.784+0000 7f346a3868c0 1 < completed part key=_multipart_GZXHYZO5SMUP.2~3ERWFP8MIYAA.2 size=630200 2024-04-07T16:37:58.786 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.792+0000 7f346a3868c0 1 < canceled multipart upload key=GZXHYZO5SMUP upload=3ERWFP8MIYAA tag=A4YTRUF63G41 type=write size=3775928 2024-04-07T16:37:58.796 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.804+0000 7f346a3868c0 1 > prepared part key=_multipart_EQR0FY86AT5X.2~0WSDYX1ST6YI.0 size=784932 2024-04-07T16:37:58.796 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.804+0000 7f346a3868c0 1 > prepared multipart upload key=EQR0FY86AT5X upload=0WSDYX1ST6YI tag=UM4UEQMN80O6 type=write size=1833508 2024-04-07T16:37:58.802 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.808+0000 7f346a3868c0 1 < completed delete operation key=1VBSUWNI3V1T tag=OIDMAMALYNGX type=del 2024-04-07T16:37:58.812 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.820+0000 7f346a3868c0 1 > prepared part key=_multipart_QTCIIPTXIRW2.2~BFCCUY78TOT2.0 size=1048576 2024-04-07T16:37:58.818 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.824+0000 7f346a3868c0 1 > prepared part key=_multipart_QTCIIPTXIRW2.2~BFCCUY78TOT2.1 size=626706 2024-04-07T16:37:58.818 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.824+0000 7f346a3868c0 1 > prepared multipart upload key=QTCIIPTXIRW2 upload=BFCCUY78TOT2 tag=E8YFTY5BZWJP type=write size=2723858 2024-04-07T16:37:58.824 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.832+0000 7f346a3868c0 1 < completed delete operation key=EQR0FY86AT5X tag=L4WEKW9UGC8T type=del 2024-04-07T16:37:58.830 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.836+0000 7f346a3868c0 1 > prepared operation key=DXTO3AJ429AC tag=5352AUOLDOT5 type=write size=815975 2024-04-07T16:37:58.835 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.844+0000 7f346a3868c0 1 < completed delete operation key=QTCIIPTXIRW2 tag=N37PQ1D1W2A0 type=del 2024-04-07T16:37:58.840 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.848+0000 7f346a3868c0 1 > prepared operation key=N14BUXLJONHZ tag=FEHL21X4IKUJ type=del size=549132 2024-04-07T16:37:58.850 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.856+0000 7f346a3868c0 1 < suggested update operation key=EOYFVEI50SQ1 tag=ZH6IYUC0YW2N type=del size=2402155 2024-04-07T16:37:58.856 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.864+0000 7f346a3868c0 1 > prepared operation key=1VKVQXSQMTU1 tag=FB3UTDFK8SQ8 type=del size=3386664 2024-04-07T16:37:58.866 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.872+0000 7f346a3868c0 1 < suggested update operation key=8F81G0P0YBFW tag=2ZP1IZCCHHL2 type=write size=443960 2024-04-07T16:37:58.872 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.880+0000 7f346a3868c0 1 > prepared operation key=1WGUPH7X5UA0 tag=QXNBXG919TS2 type=write size=913103 2024-04-07T16:37:58.877 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.884+0000 7f346a3868c0 1 < completed delete operation key=E84U0SX44NQ8 tag=3IGGD4F1TIG3 type=del 2024-04-07T16:37:58.884 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.892+0000 7f346a3868c0 1 > prepared operation key=ROFH30W3UCAX tag=GU0OL1SOPYMW type=del size=3184408 2024-04-07T16:37:58.891 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.900+0000 7f346a3868c0 -1 < failed to complete operation key=8F81G0P0YBFW tag=4BD1P7JYVZ0A type=del size=1584962: (22) Invalid argument 2024-04-07T16:37:58.897 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.904+0000 7f346a3868c0 1 > prepared operation key=1WGUPH7X5UA0 tag=ZM6T3RAPO96H type=del size=459395 2024-04-07T16:37:58.909 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.916+0000 7f346a3868c0 1 < suggested remove operation key=QVOKSQAGHPA9 tag=2WY3WJVUVR5G type=del 2024-04-07T16:37:58.920 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.928+0000 7f346a3868c0 1 > prepared part key=_multipart_IX3F0IUOKBX1.2~YYEAOMGJIHHS.0 size=1048576 2024-04-07T16:37:58.924 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.932+0000 7f346a3868c0 1 > prepared part key=_multipart_IX3F0IUOKBX1.2~YYEAOMGJIHHS.1 size=37643 2024-04-07T16:37:58.924 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.932+0000 7f346a3868c0 1 > prepared multipart upload key=IX3F0IUOKBX1 upload=YYEAOMGJIHHS tag=07FSGEH7EWI0 type=write size=2134795 2024-04-07T16:37:58.929 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.936+0000 7f346a3868c0 1 < completed delete operation key=XJLQZ7IS9U1T tag=IPV8ZMXYGRT2 type=del 2024-04-07T16:37:58.934 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.940+0000 7f346a3868c0 1 > prepared operation key=3N0QJRYVO3FV tag=UTGQYTXAJPM0 type=del size=1372619 2024-04-07T16:37:58.941 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.948+0000 7f346a3868c0 1 < completed delete operation key=LXOZ0T2KY6OR tag=TYQWIWEUP4DE type=del 2024-04-07T16:37:58.946 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.952+0000 7f346a3868c0 1 > prepared operation key=GBQSME7SY1AH tag=CWT8X78FX2O7 type=del size=3520327 2024-04-07T16:37:58.952 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.960+0000 7f346a3868c0 1 < completed write operation key=ROFH30W3UCAX tag=158NI2T7W1BU type=write size=67970 2024-04-07T16:37:58.963 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.968+0000 7f346a3868c0 1 > prepared part key=_multipart_LXOZ0T2KY6OR.2~MEVJB4NKXBN4.0 size=1048576 2024-04-07T16:37:58.967 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.976+0000 7f346a3868c0 1 > prepared part key=_multipart_LXOZ0T2KY6OR.2~MEVJB4NKXBN4.1 size=1048576 2024-04-07T16:37:58.972 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.980+0000 7f346a3868c0 1 > prepared part key=_multipart_LXOZ0T2KY6OR.2~MEVJB4NKXBN4.2 size=425164 2024-04-07T16:37:58.972 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.980+0000 7f346a3868c0 1 > prepared multipart upload key=LXOZ0T2KY6OR upload=MEVJB4NKXBN4 tag=RDHL2SFB64VX type=write size=3570892 2024-04-07T16:37:58.979 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.984+0000 7f346a3868c0 1 < completed write operation key=ROFH30W3UCAX tag=WH2UA4PAJYV2 type=write size=664326 2024-04-07T16:37:58.985 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.992+0000 7f346a3868c0 1 > prepared operation key=U3EID5EK64HD tag=ZLFK1G524KLR type=write size=246209 2024-04-07T16:37:58.990 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:58.996+0000 7f346a3868c0 1 < completed write operation key=N14BUXLJONHZ tag=PDII63WR5IXG type=write size=692324 2024-04-07T16:37:59.001 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.008+0000 7f346a3868c0 1 > prepared part key=_multipart_DXTO3AJ429AC.2~4KS0Q8S9KUTA.0 size=152684 2024-04-07T16:37:59.001 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.008+0000 7f346a3868c0 1 > prepared multipart upload key=DXTO3AJ429AC upload=4KS0Q8S9KUTA tag=WKIGGJI15YFN type=write size=1201260 2024-04-07T16:37:59.012 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.020+0000 7f346a3868c0 1 < suggested update operation key=3N0QJRYVO3FV tag=1HHG7UNI1N40 type=write size=824843 2024-04-07T16:37:59.017 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.024+0000 7f346a3868c0 1 > prepared operation key=QVOKSQAGHPA9 tag=SR5JL13ZDIIE type=del size=3414019 2024-04-07T16:37:59.023 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.028+0000 7f346a3868c0 1 < completed part key=_multipart_NYET3212USVE.2~WKU3A99PKVSI.0 size=1048576 2024-04-07T16:37:59.027 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.036+0000 7f346a3868c0 1 < completed part key=_multipart_NYET3212USVE.2~WKU3A99PKVSI.1 size=1048576 2024-04-07T16:37:59.032 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.040+0000 7f346a3868c0 1 < completed part key=_multipart_NYET3212USVE.2~WKU3A99PKVSI.2 size=140255 2024-04-07T16:37:59.041 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.048+0000 7f346a3868c0 1 < completed multipart upload key=NYET3212USVE upload=WKU3A99PKVSI tag=I5EHH6NU0BGD type=write size=3285983 2024-04-07T16:37:59.047 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.056+0000 7f346a3868c0 1 > prepared operation key=EOYFVEI50SQ1 tag=CKLC0CS2E0QG type=del size=3841831 2024-04-07T16:37:59.053 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.060+0000 7f346a3868c0 1 < completed part key=_multipart_EQR0FY86AT5X.2~0WSDYX1ST6YI.0 size=784932 2024-04-07T16:37:59.063 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.072+0000 7f346a3868c0 1 < completed multipart upload key=EQR0FY86AT5X upload=0WSDYX1ST6YI tag=UM4UEQMN80O6 type=write size=1833508 2024-04-07T16:37:59.073 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.080+0000 7f346a3868c0 1 > prepared part key=_multipart_LXOZ0T2KY6OR.2~M2ILFDTO60C4.0 size=1048576 2024-04-07T16:37:59.078 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.084+0000 7f346a3868c0 1 > prepared part key=_multipart_LXOZ0T2KY6OR.2~M2ILFDTO60C4.1 size=220717 2024-04-07T16:37:59.078 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.084+0000 7f346a3868c0 1 > prepared multipart upload key=LXOZ0T2KY6OR upload=M2ILFDTO60C4 tag=T9JXSS232MU7 type=write size=2317869 2024-04-07T16:37:59.083 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.092+0000 7f346a3868c0 1 < completed part key=_multipart_QTCIIPTXIRW2.2~BFCCUY78TOT2.0 size=1048576 2024-04-07T16:37:59.087 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.096+0000 7f346a3868c0 1 < completed part key=_multipart_QTCIIPTXIRW2.2~BFCCUY78TOT2.1 size=626706 2024-04-07T16:37:59.097 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.104+0000 7f346a3868c0 1 < completed multipart upload key=QTCIIPTXIRW2 upload=BFCCUY78TOT2 tag=E8YFTY5BZWJP type=write size=2723858 2024-04-07T16:37:59.102 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.108+0000 7f346a3868c0 1 > prepared operation key=XJLQZ7IS9U1T tag=MVUT3DN2DSD2 type=del size=121787 2024-04-07T16:37:59.108 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.116+0000 7f346a3868c0 1 < canceled operation key=DXTO3AJ429AC tag=5352AUOLDOT5 type=write size=815975 2024-04-07T16:37:59.119 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.128+0000 7f346a3868c0 1 > prepared part key=_multipart_QVOKSQAGHPA9.2~6NPTNQ76OC9U.0 size=1048576 2024-04-07T16:37:59.124 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.132+0000 7f346a3868c0 1 > prepared part key=_multipart_QVOKSQAGHPA9.2~6NPTNQ76OC9U.1 size=1048576 2024-04-07T16:37:59.129 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.136+0000 7f346a3868c0 1 > prepared part key=_multipart_QVOKSQAGHPA9.2~6NPTNQ76OC9U.2 size=574680 2024-04-07T16:37:59.129 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.136+0000 7f346a3868c0 1 > prepared multipart upload key=QVOKSQAGHPA9 upload=6NPTNQ76OC9U tag=M1QPQ621NIQE type=write size=3720408 2024-04-07T16:37:59.145 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.152+0000 7f346a3868c0 1 < completed delete operation key=N14BUXLJONHZ tag=FEHL21X4IKUJ type=del 2024-04-07T16:37:59.154 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.160+0000 7f346a3868c0 1 > prepared operation key=GZXHYZO5SMUP tag=6CQVLB325L52 type=del size=4090802 2024-04-07T16:37:59.165 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.172+0000 7f346a3868c0 1 < suggested remove operation key=1VKVQXSQMTU1 tag=FB3UTDFK8SQ8 type=del 2024-04-07T16:37:59.171 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.176+0000 7f346a3868c0 1 > prepared operation key=1VBSUWNI3V1T tag=DY1S91LDQN8Z type=del size=2388826 2024-04-07T16:37:59.176 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.184+0000 7f346a3868c0 1 < completed write operation key=1WGUPH7X5UA0 tag=QXNBXG919TS2 type=write size=913103 2024-04-07T16:37:59.182 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.188+0000 7f346a3868c0 1 > prepared operation key=HBLEXONKXJTL tag=A3FF564V95ZO type=del size=3425939 2024-04-07T16:37:59.193 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.200+0000 7f346a3868c0 1 < suggested remove operation key=ROFH30W3UCAX tag=GU0OL1SOPYMW type=del 2024-04-07T16:37:59.203 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.212+0000 7f346a3868c0 1 > prepared part key=_multipart_95YVKJ14JD8W.2~YUP61GCA3W02.0 size=1048576 2024-04-07T16:37:59.208 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.216+0000 7f346a3868c0 1 > prepared part key=_multipart_95YVKJ14JD8W.2~YUP61GCA3W02.1 size=1048576 2024-04-07T16:37:59.212 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.220+0000 7f346a3868c0 1 > prepared part key=_multipart_95YVKJ14JD8W.2~YUP61GCA3W02.2 size=22118 2024-04-07T16:37:59.212 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.220+0000 7f346a3868c0 1 > prepared multipart upload key=95YVKJ14JD8W upload=YUP61GCA3W02 tag=8MJUUWYN7CQV type=write size=3167846 2024-04-07T16:37:59.218 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.224+0000 7f346a3868c0 1 < canceled operation key=1WGUPH7X5UA0 tag=ZM6T3RAPO96H type=del size=459395 2024-04-07T16:37:59.228 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.236+0000 7f346a3868c0 1 > prepared part key=_multipart_X5Q46GO8YTOK.2~17S4EOZFZIAL.0 size=1048576 2024-04-07T16:37:59.232 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.240+0000 7f346a3868c0 1 > prepared part key=_multipart_X5Q46GO8YTOK.2~17S4EOZFZIAL.1 size=1048576 2024-04-07T16:37:59.237 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.244+0000 7f346a3868c0 1 > prepared part key=_multipart_X5Q46GO8YTOK.2~17S4EOZFZIAL.2 size=226158 2024-04-07T16:37:59.237 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.244+0000 7f346a3868c0 1 > prepared multipart upload key=X5Q46GO8YTOK upload=17S4EOZFZIAL tag=ZM5XT9KYLCGO type=write size=3371886 2024-04-07T16:37:59.238 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.244+0000 7f346a3868c0 1 < canceled part key=_multipart_IX3F0IUOKBX1.2~YYEAOMGJIHHS.0 size=1048576 2024-04-07T16:37:59.242 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.248+0000 7f346a3868c0 1 < completed part key=_multipart_IX3F0IUOKBX1.2~YYEAOMGJIHHS.1 size=37643 2024-04-07T16:37:59.252 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.260+0000 7f346a3868c0 1 < canceled multipart upload key=IX3F0IUOKBX1 upload=YYEAOMGJIHHS tag=07FSGEH7EWI0 type=write size=2134795 2024-04-07T16:37:59.258 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.264+0000 7f346a3868c0 1 > prepared operation key=8F81G0P0YBFW tag=HQOV5HGJAXAX type=del size=2214545 2024-04-07T16:37:59.266 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.272+0000 7f346a3868c0 -1 < failed to complete operation key=3N0QJRYVO3FV tag=UTGQYTXAJPM0 type=del size=1372619: (22) Invalid argument 2024-04-07T16:37:59.276 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.284+0000 7f346a3868c0 1 > prepared part key=_multipart_CL4PP1RKZU0E.2~OMAOPXC13DCU.0 size=1048576 2024-04-07T16:37:59.280 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.288+0000 7f346a3868c0 1 > prepared part key=_multipart_CL4PP1RKZU0E.2~OMAOPXC13DCU.1 size=774714 2024-04-07T16:37:59.281 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.288+0000 7f346a3868c0 1 > prepared multipart upload key=CL4PP1RKZU0E upload=OMAOPXC13DCU tag=T73LC1I70C6Q type=write size=2871866 2024-04-07T16:37:59.286 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.292+0000 7f346a3868c0 1 < completed delete operation key=GBQSME7SY1AH tag=CWT8X78FX2O7 type=del 2024-04-07T16:37:59.294 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.300+0000 7f346a3868c0 1 > prepared operation key=3N0QJRYVO3FV tag=G9XYXVAFWIOP type=del size=2729689 2024-04-07T16:37:59.299 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.308+0000 7f346a3868c0 1 < completed part key=_multipart_LXOZ0T2KY6OR.2~MEVJB4NKXBN4.0 size=1048576 2024-04-07T16:37:59.304 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.312+0000 7f346a3868c0 1 < completed part key=_multipart_LXOZ0T2KY6OR.2~MEVJB4NKXBN4.1 size=1048576 2024-04-07T16:37:59.309 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.316+0000 7f346a3868c0 1 < completed part key=_multipart_LXOZ0T2KY6OR.2~MEVJB4NKXBN4.2 size=425164 2024-04-07T16:37:59.318 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.324+0000 7f346a3868c0 1 < completed multipart upload key=LXOZ0T2KY6OR upload=MEVJB4NKXBN4 tag=RDHL2SFB64VX type=write size=3570892 2024-04-07T16:37:59.329 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.336+0000 7f346a3868c0 1 > prepared part key=_multipart_1VKVQXSQMTU1.2~RSGLYSVC6O5H.0 size=438923 2024-04-07T16:37:59.329 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.336+0000 7f346a3868c0 1 > prepared multipart upload key=1VKVQXSQMTU1 upload=RSGLYSVC6O5H tag=Z5UGZWICBTA3 type=write size=1487499 2024-04-07T16:37:59.335 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.344+0000 7f346a3868c0 1 < completed write operation key=U3EID5EK64HD tag=ZLFK1G524KLR type=write size=246209 2024-04-07T16:37:59.342 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.348+0000 7f346a3868c0 1 > prepared operation key=X5Q46GO8YTOK tag=76P43TBMI7YZ type=del size=605542 2024-04-07T16:37:59.348 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.356+0000 7f346a3868c0 1 < completed part key=_multipart_DXTO3AJ429AC.2~4KS0Q8S9KUTA.0 size=152684 2024-04-07T16:37:59.358 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.364+0000 7f346a3868c0 1 < completed multipart upload key=DXTO3AJ429AC upload=4KS0Q8S9KUTA tag=WKIGGJI15YFN type=write size=1201260 2024-04-07T16:37:59.364 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.372+0000 7f346a3868c0 1 > prepared operation key=1VBSUWNI3V1T tag=DJNUTX6QYA5N type=del size=3046720 2024-04-07T16:37:59.368 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.376+0000 7f346a3868c0 1 < completed delete operation key=QVOKSQAGHPA9 tag=SR5JL13ZDIIE type=del 2024-04-07T16:37:59.373 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.380+0000 7f346a3868c0 1 > prepared operation key=QS4HYZFFAJNF tag=GUPD0BV3G4M4 type=del size=1825986 2024-04-07T16:37:59.379 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.384+0000 7f346a3868c0 1 < completed delete operation key=EOYFVEI50SQ1 tag=CKLC0CS2E0QG type=del 2024-04-07T16:37:59.384 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.392+0000 7f346a3868c0 1 > prepared operation key=7G5C6HBK6QWI tag=G7W3KFNXEIHO type=del size=3661933 2024-04-07T16:37:59.385 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.392+0000 7f346a3868c0 1 < canceled part key=_multipart_LXOZ0T2KY6OR.2~M2ILFDTO60C4.0 size=1048576 2024-04-07T16:37:59.389 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.396+0000 7f346a3868c0 1 < completed part key=_multipart_LXOZ0T2KY6OR.2~M2ILFDTO60C4.1 size=220717 2024-04-07T16:37:59.399 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.408+0000 7f346a3868c0 1 < canceled multipart upload key=LXOZ0T2KY6OR upload=M2ILFDTO60C4 tag=T9JXSS232MU7 type=write size=2317869 2024-04-07T16:37:59.405 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.412+0000 7f346a3868c0 1 > prepared operation key=GZXHYZO5SMUP tag=3JZSGSZLKMQ2 type=del size=600009 2024-04-07T16:37:59.410 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.416+0000 7f346a3868c0 1 < completed delete operation key=XJLQZ7IS9U1T tag=MVUT3DN2DSD2 type=del 2024-04-07T16:37:59.415 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.424+0000 7f346a3868c0 1 > prepared operation key=3N0QJRYVO3FV tag=WA1RXG1KV1XK type=write size=714041 2024-04-07T16:37:59.421 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.428+0000 7f346a3868c0 1 < completed part key=_multipart_QVOKSQAGHPA9.2~6NPTNQ76OC9U.0 size=1048576 2024-04-07T16:37:59.426 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.432+0000 7f346a3868c0 1 < completed part key=_multipart_QVOKSQAGHPA9.2~6NPTNQ76OC9U.1 size=1048576 2024-04-07T16:37:59.431 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.440+0000 7f346a3868c0 1 < completed part key=_multipart_QVOKSQAGHPA9.2~6NPTNQ76OC9U.2 size=574680 2024-04-07T16:37:59.442 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.448+0000 7f346a3868c0 1 < completed multipart upload key=QVOKSQAGHPA9 upload=6NPTNQ76OC9U tag=M1QPQ621NIQE type=write size=3720408 2024-04-07T16:37:59.452 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.460+0000 7f346a3868c0 1 > prepared part key=_multipart_DXTO3AJ429AC.2~DHSRRN9CCOID.0 size=1048576 2024-04-07T16:37:59.457 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.464+0000 7f346a3868c0 1 > prepared part key=_multipart_DXTO3AJ429AC.2~DHSRRN9CCOID.1 size=378605 2024-04-07T16:37:59.457 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.464+0000 7f346a3868c0 1 > prepared multipart upload key=DXTO3AJ429AC upload=DHSRRN9CCOID tag=PJRLALWJTIU1 type=write size=2475757 2024-04-07T16:37:59.468 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.476+0000 7f346a3868c0 1 < suggested remove operation key=GZXHYZO5SMUP tag=6CQVLB325L52 type=del 2024-04-07T16:37:59.478 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.484+0000 7f346a3868c0 1 > prepared part key=_multipart_QS4HYZFFAJNF.2~CUKXBD0UR5PJ.0 size=1048576 2024-04-07T16:37:59.483 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.492+0000 7f346a3868c0 1 > prepared part key=_multipart_QS4HYZFFAJNF.2~CUKXBD0UR5PJ.1 size=642265 2024-04-07T16:37:59.484 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.492+0000 7f346a3868c0 1 > prepared multipart upload key=QS4HYZFFAJNF upload=CUKXBD0UR5PJ tag=KAY2Q4CL5NGX type=write size=2739417 2024-04-07T16:37:59.489 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.496+0000 7f346a3868c0 1 < completed delete operation key=1VBSUWNI3V1T tag=DY1S91LDQN8Z type=del 2024-04-07T16:37:59.500 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.508+0000 7f346a3868c0 1 > prepared part key=_multipart_EQR0FY86AT5X.2~8NPYUU1XUDYW.0 size=1048576 2024-04-07T16:37:59.505 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.512+0000 7f346a3868c0 1 > prepared part key=_multipart_EQR0FY86AT5X.2~8NPYUU1XUDYW.1 size=597441 2024-04-07T16:37:59.505 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.512+0000 7f346a3868c0 1 > prepared multipart upload key=EQR0FY86AT5X upload=8NPYUU1XUDYW tag=NWA9MPTYQ6O3 type=write size=2694593 2024-04-07T16:37:59.510 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.516+0000 7f346a3868c0 1 < completed delete operation key=HBLEXONKXJTL tag=A3FF564V95ZO type=del 2024-04-07T16:37:59.521 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.528+0000 7f346a3868c0 1 > prepared part key=_multipart_NYET3212USVE.2~Q7D9E727XMUY.0 size=721939 2024-04-07T16:37:59.521 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.528+0000 7f346a3868c0 1 > prepared multipart upload key=NYET3212USVE upload=Q7D9E727XMUY tag=E252IWY3EL6K type=write size=1770515 2024-04-07T16:37:59.527 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.536+0000 7f346a3868c0 1 < completed part key=_multipart_95YVKJ14JD8W.2~YUP61GCA3W02.0 size=1048576 2024-04-07T16:37:59.532 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.540+0000 7f346a3868c0 1 < completed part key=_multipart_95YVKJ14JD8W.2~YUP61GCA3W02.1 size=1048576 2024-04-07T16:37:59.537 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.544+0000 7f346a3868c0 1 < completed part key=_multipart_95YVKJ14JD8W.2~YUP61GCA3W02.2 size=22118 2024-04-07T16:37:59.547 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.556+0000 7f346a3868c0 1 < completed multipart upload key=95YVKJ14JD8W upload=YUP61GCA3W02 tag=8MJUUWYN7CQV type=write size=3167846 2024-04-07T16:37:59.554 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.560+0000 7f346a3868c0 1 > prepared operation key=YXMDZB23YTB3 tag=G56UWNJVE6I5 type=del size=2612112 2024-04-07T16:37:59.560 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.568+0000 7f346a3868c0 1 < completed part key=_multipart_X5Q46GO8YTOK.2~17S4EOZFZIAL.0 size=1048576 2024-04-07T16:37:59.565 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.572+0000 7f346a3868c0 1 < completed part key=_multipart_X5Q46GO8YTOK.2~17S4EOZFZIAL.1 size=1048576 2024-04-07T16:37:59.570 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.576+0000 7f346a3868c0 1 < completed part key=_multipart_X5Q46GO8YTOK.2~17S4EOZFZIAL.2 size=226158 2024-04-07T16:37:59.580 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.588+0000 7f346a3868c0 1 < completed multipart upload key=X5Q46GO8YTOK upload=17S4EOZFZIAL tag=ZM5XT9KYLCGO type=write size=3371886 2024-04-07T16:37:59.590 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.596+0000 7f346a3868c0 1 > prepared part key=_multipart_HBLEXONKXJTL.2~357DWL3HBE3V.0 size=1048576 2024-04-07T16:37:59.595 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.604+0000 7f346a3868c0 1 > prepared part key=_multipart_HBLEXONKXJTL.2~357DWL3HBE3V.1 size=1048576 2024-04-07T16:37:59.602 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.608+0000 7f346a3868c0 1 > prepared part key=_multipart_HBLEXONKXJTL.2~357DWL3HBE3V.2 size=1002974 2024-04-07T16:37:59.602 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.608+0000 7f346a3868c0 1 > prepared multipart upload key=HBLEXONKXJTL upload=357DWL3HBE3V tag=S0UF6BDPASS2 type=write size=4148702 2024-04-07T16:37:59.605 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.612+0000 7f346a3868c0 1 < completed delete operation key=8F81G0P0YBFW tag=HQOV5HGJAXAX type=del 2024-04-07T16:37:59.611 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.616+0000 7f346a3868c0 1 > prepared operation key=1VKVQXSQMTU1 tag=WEG5JZWRCRUQ type=write size=404602 2024-04-07T16:37:59.617 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.624+0000 7f346a3868c0 1 < completed part key=_multipart_CL4PP1RKZU0E.2~OMAOPXC13DCU.0 size=1048576 2024-04-07T16:37:59.617 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.624+0000 7f346a3868c0 1 < canceled part key=_multipart_CL4PP1RKZU0E.2~OMAOPXC13DCU.1 size=774714 2024-04-07T16:37:59.626 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.632+0000 7f346a3868c0 1 < canceled multipart upload key=CL4PP1RKZU0E upload=OMAOPXC13DCU tag=T73LC1I70C6Q type=write size=2871866 2024-04-07T16:37:59.632 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.640+0000 7f346a3868c0 1 > prepared operation key=DXTO3AJ429AC tag=SYM2SE1NAEI9 type=write size=88196 2024-04-07T16:37:59.637 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.644+0000 7f346a3868c0 1 < completed delete operation key=3N0QJRYVO3FV tag=G9XYXVAFWIOP type=del 2024-04-07T16:37:59.648 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.656+0000 7f346a3868c0 1 > prepared part key=_multipart_YXMDZB23YTB3.2~T4O5HQDK198W.0 size=1048576 2024-04-07T16:37:59.653 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.660+0000 7f346a3868c0 1 > prepared part key=_multipart_YXMDZB23YTB3.2~T4O5HQDK198W.1 size=661643 2024-04-07T16:37:59.653 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.660+0000 7f346a3868c0 1 > prepared multipart upload key=YXMDZB23YTB3 upload=T4O5HQDK198W tag=GPO57RFM33HX type=write size=2758795 2024-04-07T16:37:59.658 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.664+0000 7f346a3868c0 1 < completed part key=_multipart_1VKVQXSQMTU1.2~RSGLYSVC6O5H.0 size=438923 2024-04-07T16:37:59.668 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.676+0000 7f346a3868c0 1 < completed multipart upload key=1VKVQXSQMTU1 upload=RSGLYSVC6O5H tag=Z5UGZWICBTA3 type=write size=1487499 2024-04-07T16:37:59.673 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.680+0000 7f346a3868c0 1 > prepared operation key=7G5C6HBK6QWI tag=Z7HU7Y8NLIHW type=write size=882292 2024-04-07T16:37:59.679 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.688+0000 7f346a3868c0 1 < completed delete operation key=X5Q46GO8YTOK tag=76P43TBMI7YZ type=del 2024-04-07T16:37:59.689 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.696+0000 7f346a3868c0 1 > prepared part key=_multipart_XJLQZ7IS9U1T.2~U2RTN7PD587R.0 size=1048576 2024-04-07T16:37:59.694 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.700+0000 7f346a3868c0 1 > prepared part key=_multipart_XJLQZ7IS9U1T.2~U2RTN7PD587R.1 size=731015 2024-04-07T16:37:59.694 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.700+0000 7f346a3868c0 1 > prepared multipart upload key=XJLQZ7IS9U1T upload=U2RTN7PD587R tag=DIEQ918B0X2G type=write size=2828167 2024-04-07T16:37:59.700 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.708+0000 7f346a3868c0 1 < completed delete operation key=1VBSUWNI3V1T tag=DJNUTX6QYA5N type=del 2024-04-07T16:37:59.706 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.712+0000 7f346a3868c0 1 > prepared operation key=3N0QJRYVO3FV tag=2PQPVCLU0RH0 type=del size=3918046 2024-04-07T16:37:59.711 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.720+0000 7f346a3868c0 1 < completed delete operation key=QS4HYZFFAJNF tag=GUPD0BV3G4M4 type=del 2024-04-07T16:37:59.722 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.728+0000 7f346a3868c0 1 > prepared part key=_multipart_GBQSME7SY1AH.2~NUV90MWI7EPU.0 size=955138 2024-04-07T16:37:59.722 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.728+0000 7f346a3868c0 1 > prepared multipart upload key=GBQSME7SY1AH upload=NUV90MWI7EPU tag=8FD43BYQXP9N type=write size=2003714 2024-04-07T16:37:59.727 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.736+0000 7f346a3868c0 1 < completed delete operation key=7G5C6HBK6QWI tag=G7W3KFNXEIHO type=del 2024-04-07T16:37:59.737 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.744+0000 7f346a3868c0 1 > prepared part key=_multipart_1WGUPH7X5UA0.2~9OB4FBPYVSXB.0 size=1048576 2024-04-07T16:37:59.744 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.752+0000 7f346a3868c0 1 > prepared part key=_multipart_1WGUPH7X5UA0.2~9OB4FBPYVSXB.1 size=811212 2024-04-07T16:37:59.744 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.752+0000 7f346a3868c0 1 > prepared multipart upload key=1WGUPH7X5UA0 upload=9OB4FBPYVSXB tag=02E3FZ3QRMNY type=write size=2908364 2024-04-07T16:37:59.751 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.760+0000 7f346a3868c0 -1 < failed to complete operation key=GZXHYZO5SMUP tag=3JZSGSZLKMQ2 type=del size=600009: (22) Invalid argument 2024-04-07T16:37:59.758 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.764+0000 7f346a3868c0 1 > prepared operation key=NYET3212USVE tag=KBRR511WT6L5 type=del size=1524443 2024-04-07T16:37:59.764 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.772+0000 7f346a3868c0 1 < completed write operation key=3N0QJRYVO3FV tag=WA1RXG1KV1XK type=write size=714041 2024-04-07T16:37:59.770 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.776+0000 7f346a3868c0 1 > prepared operation key=1VKVQXSQMTU1 tag=KGSQO54EVTFM type=del size=1342009 2024-04-07T16:37:59.771 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.776+0000 7f346a3868c0 1 < canceled part key=_multipart_DXTO3AJ429AC.2~DHSRRN9CCOID.0 size=1048576 2024-04-07T16:37:59.771 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.776+0000 7f346a3868c0 1 < canceled part key=_multipart_DXTO3AJ429AC.2~DHSRRN9CCOID.1 size=378605 2024-04-07T16:37:59.781 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.788+0000 7f346a3868c0 1 < canceled multipart upload key=DXTO3AJ429AC upload=DHSRRN9CCOID tag=PJRLALWJTIU1 type=write size=2475757 2024-04-07T16:37:59.793 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.800+0000 7f346a3868c0 1 > prepared part key=_multipart_1WGUPH7X5UA0.2~FA1BYZ3C7KV1.0 size=1048576 2024-04-07T16:37:59.798 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.804+0000 7f346a3868c0 1 > prepared part key=_multipart_1WGUPH7X5UA0.2~FA1BYZ3C7KV1.1 size=473167 2024-04-07T16:37:59.798 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.804+0000 7f346a3868c0 1 > prepared multipart upload key=1WGUPH7X5UA0 upload=FA1BYZ3C7KV1 tag=N95XR1R4WGCF type=write size=2570319 2024-04-07T16:37:59.804 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.812+0000 7f346a3868c0 1 < completed part key=_multipart_QS4HYZFFAJNF.2~CUKXBD0UR5PJ.0 size=1048576 2024-04-07T16:37:59.808 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.816+0000 7f346a3868c0 1 < completed part key=_multipart_QS4HYZFFAJNF.2~CUKXBD0UR5PJ.1 size=642265 2024-04-07T16:37:59.818 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.824+0000 7f346a3868c0 1 < completed multipart upload key=QS4HYZFFAJNF upload=CUKXBD0UR5PJ tag=KAY2Q4CL5NGX type=write size=2739417 2024-04-07T16:37:59.823 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.832+0000 7f346a3868c0 1 > prepared operation key=XJLQZ7IS9U1T tag=0YIU72NP08HL type=del size=4167717 2024-04-07T16:37:59.829 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.836+0000 7f346a3868c0 1 < completed part key=_multipart_EQR0FY86AT5X.2~8NPYUU1XUDYW.0 size=1048576 2024-04-07T16:37:59.834 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.840+0000 7f346a3868c0 1 < completed part key=_multipart_EQR0FY86AT5X.2~8NPYUU1XUDYW.1 size=597441 2024-04-07T16:37:59.844 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.852+0000 7f346a3868c0 1 < completed multipart upload key=EQR0FY86AT5X upload=8NPYUU1XUDYW tag=NWA9MPTYQ6O3 type=write size=2694593 2024-04-07T16:37:59.850 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.856+0000 7f346a3868c0 1 > prepared operation key=QVOKSQAGHPA9 tag=RR9N0318UKNC type=write size=869172 2024-04-07T16:37:59.851 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.856+0000 7f346a3868c0 1 < canceled part key=_multipart_NYET3212USVE.2~Q7D9E727XMUY.0 size=721939 2024-04-07T16:37:59.860 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.868+0000 7f346a3868c0 1 < canceled multipart upload key=NYET3212USVE upload=Q7D9E727XMUY tag=E252IWY3EL6K type=write size=1770515 2024-04-07T16:37:59.866 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.872+0000 7f346a3868c0 1 > prepared operation key=DXTO3AJ429AC tag=UCBN1WW2OM8Y type=del size=1257940 2024-04-07T16:37:59.872 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.880+0000 7f346a3868c0 1 < completed delete operation key=YXMDZB23YTB3 tag=G56UWNJVE6I5 type=del 2024-04-07T16:37:59.877 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.884+0000 7f346a3868c0 1 > prepared operation key=GBQSME7SY1AH tag=N6E06IOP8ZSQ type=del size=1003022 2024-04-07T16:37:59.883 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.888+0000 7f346a3868c0 1 < completed part key=_multipart_HBLEXONKXJTL.2~357DWL3HBE3V.0 size=1048576 2024-04-07T16:37:59.887 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.896+0000 7f346a3868c0 1 < completed part key=_multipart_HBLEXONKXJTL.2~357DWL3HBE3V.1 size=1048576 2024-04-07T16:37:59.891 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.900+0000 7f346a3868c0 1 < completed part key=_multipart_HBLEXONKXJTL.2~357DWL3HBE3V.2 size=1002974 2024-04-07T16:37:59.901 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.908+0000 7f346a3868c0 1 < completed multipart upload key=HBLEXONKXJTL upload=357DWL3HBE3V tag=S0UF6BDPASS2 type=write size=4148702 2024-04-07T16:37:59.912 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.920+0000 7f346a3868c0 1 > prepared part key=_multipart_IX3F0IUOKBX1.2~EFCI6WKW5QDA.0 size=1048576 2024-04-07T16:37:59.917 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.924+0000 7f346a3868c0 1 > prepared part key=_multipart_IX3F0IUOKBX1.2~EFCI6WKW5QDA.1 size=1048576 2024-04-07T16:37:59.921 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.928+0000 7f346a3868c0 1 > prepared part key=_multipart_IX3F0IUOKBX1.2~EFCI6WKW5QDA.2 size=275363 2024-04-07T16:37:59.921 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.928+0000 7f346a3868c0 1 > prepared multipart upload key=IX3F0IUOKBX1 upload=EFCI6WKW5QDA tag=BH6SWVXSK9JN type=write size=3421091 2024-04-07T16:37:59.927 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.932+0000 7f346a3868c0 1 < completed write operation key=1VKVQXSQMTU1 tag=WEG5JZWRCRUQ type=write size=404602 2024-04-07T16:37:59.933 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.940+0000 7f346a3868c0 1 > prepared operation key=QTCIIPTXIRW2 tag=UJPUQYY9MP7B type=del size=581 2024-04-07T16:37:59.938 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.944+0000 7f346a3868c0 1 < completed write operation key=DXTO3AJ429AC tag=SYM2SE1NAEI9 type=write size=88196 2024-04-07T16:37:59.952 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.960+0000 7f346a3868c0 1 > prepared part key=_multipart_6XDS0A7L0IY8.2~118E72WNIM31.0 size=145183 2024-04-07T16:37:59.952 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.960+0000 7f346a3868c0 1 > prepared multipart upload key=6XDS0A7L0IY8 upload=118E72WNIM31 tag=KTZMZ531ZHPW type=write size=1193759 2024-04-07T16:37:59.958 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.964+0000 7f346a3868c0 1 < completed part key=_multipart_YXMDZB23YTB3.2~T4O5HQDK198W.0 size=1048576 2024-04-07T16:37:59.963 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.968+0000 7f346a3868c0 1 < completed part key=_multipart_YXMDZB23YTB3.2~T4O5HQDK198W.1 size=661643 2024-04-07T16:37:59.973 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.980+0000 7f346a3868c0 1 < completed multipart upload key=YXMDZB23YTB3 upload=T4O5HQDK198W tag=GPO57RFM33HX type=write size=2758795 2024-04-07T16:37:59.978 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:37:59.984+0000 7f346a3868c0 1 > prepared operation key=1VBSUWNI3V1T tag=SPH0TK8EP4UF type=del size=3508925 2024-04-07T16:37:59.991 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.000+0000 7f346a3868c0 1 < suggested update operation key=7G5C6HBK6QWI tag=Z7HU7Y8NLIHW type=write size=882292 2024-04-07T16:37:59.997 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.004+0000 7f346a3868c0 1 > prepared operation key=IP8YW2A75TCW tag=JIB558V7MV24 type=write size=503567 2024-04-07T16:38:00.003 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.012+0000 7f346a3868c0 1 < completed part key=_multipart_XJLQZ7IS9U1T.2~U2RTN7PD587R.0 size=1048576 2024-04-07T16:38:00.008 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.016+0000 7f346a3868c0 1 < completed part key=_multipart_XJLQZ7IS9U1T.2~U2RTN7PD587R.1 size=731015 2024-04-07T16:38:00.018 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.024+0000 7f346a3868c0 1 < completed multipart upload key=XJLQZ7IS9U1T upload=U2RTN7PD587R tag=DIEQ918B0X2G type=write size=2828167 2024-04-07T16:38:00.026 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.032+0000 7f346a3868c0 1 > prepared operation key=QS4HYZFFAJNF tag=VRR61JARABJO type=del size=91701 2024-04-07T16:38:00.033 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.040+0000 7f346a3868c0 1 < completed delete operation key=3N0QJRYVO3FV tag=2PQPVCLU0RH0 type=del 2024-04-07T16:38:00.039 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.044+0000 7f346a3868c0 1 > prepared operation key=3N0QJRYVO3FV tag=27R1MLIRAK7A type=del size=4190406 2024-04-07T16:38:00.045 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.052+0000 7f346a3868c0 1 < completed part key=_multipart_GBQSME7SY1AH.2~NUV90MWI7EPU.0 size=955138 2024-04-07T16:38:00.055 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.064+0000 7f346a3868c0 1 < completed multipart upload key=GBQSME7SY1AH upload=NUV90MWI7EPU tag=8FD43BYQXP9N type=write size=2003714 2024-04-07T16:38:00.061 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.068+0000 7f346a3868c0 1 > prepared operation key=UIW3DN4NUYPP tag=IKLSPZGQGQFD type=write size=678016 2024-04-07T16:38:00.068 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.076+0000 7f346a3868c0 1 < completed part key=_multipart_1WGUPH7X5UA0.2~9OB4FBPYVSXB.0 size=1048576 2024-04-07T16:38:00.073 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.080+0000 7f346a3868c0 1 < completed part key=_multipart_1WGUPH7X5UA0.2~9OB4FBPYVSXB.1 size=811212 2024-04-07T16:38:00.082 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.088+0000 7f346a3868c0 1 < completed multipart upload key=1WGUPH7X5UA0 upload=9OB4FBPYVSXB tag=02E3FZ3QRMNY type=write size=2908364 2024-04-07T16:38:00.088 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.096+0000 7f346a3868c0 1 > prepared operation key=E84U0SX44NQ8 tag=HHIFKZTPNJUO type=del size=3743728 2024-04-07T16:38:00.099 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.108+0000 7f346a3868c0 1 < suggested update operation key=NYET3212USVE tag=KBRR511WT6L5 type=del size=1524443 2024-04-07T16:38:00.105 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.112+0000 7f346a3868c0 1 > prepared operation key=8F81G0P0YBFW tag=7I4P80OWSE1V type=del size=1014798 2024-04-07T16:38:00.111 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.116+0000 7f346a3868c0 1 < completed delete operation key=1VKVQXSQMTU1 tag=KGSQO54EVTFM type=del 2024-04-07T16:38:00.123 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.132+0000 7f346a3868c0 1 > prepared part key=_multipart_YOK8BAEOMPUR.2~EE5O5OW86DD9.0 size=1048576 2024-04-07T16:38:00.128 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.136+0000 7f346a3868c0 1 > prepared part key=_multipart_YOK8BAEOMPUR.2~EE5O5OW86DD9.1 size=689108 2024-04-07T16:38:00.128 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.136+0000 7f346a3868c0 1 > prepared multipart upload key=YOK8BAEOMPUR upload=EE5O5OW86DD9 tag=I2UJUFNE1PZ1 type=write size=2786260 2024-04-07T16:38:00.128 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.136+0000 7f346a3868c0 1 < canceled part key=_multipart_1WGUPH7X5UA0.2~FA1BYZ3C7KV1.0 size=1048576 2024-04-07T16:38:00.133 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.140+0000 7f346a3868c0 1 < completed part key=_multipart_1WGUPH7X5UA0.2~FA1BYZ3C7KV1.1 size=473167 2024-04-07T16:38:00.142 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.148+0000 7f346a3868c0 1 < canceled multipart upload key=1WGUPH7X5UA0 upload=FA1BYZ3C7KV1 tag=N95XR1R4WGCF type=write size=2570319 2024-04-07T16:38:00.152 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.160+0000 7f346a3868c0 1 > prepared part key=_multipart_EQR0FY86AT5X.2~9U1U4ULGYMTM.0 size=1048576 2024-04-07T16:38:00.156 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.164+0000 7f346a3868c0 1 > prepared part key=_multipart_EQR0FY86AT5X.2~9U1U4ULGYMTM.1 size=420468 2024-04-07T16:38:00.156 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.164+0000 7f346a3868c0 1 > prepared multipart upload key=EQR0FY86AT5X upload=9U1U4ULGYMTM tag=ANPY1L59C6RM type=write size=2517620 2024-04-07T16:38:00.162 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.168+0000 7f346a3868c0 1 < completed delete operation key=XJLQZ7IS9U1T tag=0YIU72NP08HL type=del 2024-04-07T16:38:00.167 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.176+0000 7f346a3868c0 1 > prepared operation key=NYET3212USVE tag=SKRVGYIH6PQF type=del size=328412 2024-04-07T16:38:00.173 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.180+0000 7f346a3868c0 1 < completed write operation key=QVOKSQAGHPA9 tag=RR9N0318UKNC type=write size=869172 2024-04-07T16:38:00.179 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.184+0000 7f346a3868c0 1 > prepared operation key=YXMDZB23YTB3 tag=J79PRFRPICY7 type=del size=877818 2024-04-07T16:38:00.186 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.192+0000 7f346a3868c0 1 < completed delete operation key=DXTO3AJ429AC tag=UCBN1WW2OM8Y type=del 2024-04-07T16:38:00.196 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.204+0000 7f346a3868c0 1 > prepared part key=_multipart_EQR0FY86AT5X.2~WGPSU41F208V.0 size=1048576 2024-04-07T16:38:00.201 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.208+0000 7f346a3868c0 1 > prepared part key=_multipart_EQR0FY86AT5X.2~WGPSU41F208V.1 size=702858 2024-04-07T16:38:00.201 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.208+0000 7f346a3868c0 1 > prepared multipart upload key=EQR0FY86AT5X upload=WGPSU41F208V tag=HNXT0VSJAC1T type=write size=2800010 2024-04-07T16:38:00.206 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.212+0000 7f346a3868c0 1 < canceled operation key=GBQSME7SY1AH tag=N6E06IOP8ZSQ type=del size=1003022 2024-04-07T16:38:00.213 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.220+0000 7f346a3868c0 1 > prepared operation key=UIW3DN4NUYPP tag=U9DRFN8DR325 type=del size=281941 2024-04-07T16:38:00.219 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.228+0000 7f346a3868c0 1 < completed part key=_multipart_IX3F0IUOKBX1.2~EFCI6WKW5QDA.0 size=1048576 2024-04-07T16:38:00.224 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.232+0000 7f346a3868c0 1 < completed part key=_multipart_IX3F0IUOKBX1.2~EFCI6WKW5QDA.1 size=1048576 2024-04-07T16:38:00.229 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.236+0000 7f346a3868c0 1 < completed part key=_multipart_IX3F0IUOKBX1.2~EFCI6WKW5QDA.2 size=275363 2024-04-07T16:38:00.238 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.244+0000 7f346a3868c0 1 < completed multipart upload key=IX3F0IUOKBX1 upload=EFCI6WKW5QDA tag=BH6SWVXSK9JN type=write size=3421091 2024-04-07T16:38:00.245 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.252+0000 7f346a3868c0 1 > prepared operation key=YXMDZB23YTB3 tag=XPT7O96FY286 type=write size=518997 2024-04-07T16:38:00.252 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.260+0000 7f346a3868c0 1 < completed delete operation key=QTCIIPTXIRW2 tag=UJPUQYY9MP7B type=del 2024-04-07T16:38:00.263 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.268+0000 7f346a3868c0 1 > prepared part key=_multipart_N14BUXLJONHZ.2~EUE3PP1Y1AW1.0 size=1048576 2024-04-07T16:38:00.268 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.276+0000 7f346a3868c0 1 > prepared part key=_multipart_N14BUXLJONHZ.2~EUE3PP1Y1AW1.1 size=605060 2024-04-07T16:38:00.268 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.276+0000 7f346a3868c0 1 > prepared multipart upload key=N14BUXLJONHZ upload=EUE3PP1Y1AW1 tag=03TAKMS22CZN type=write size=2702212 2024-04-07T16:38:00.273 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.280+0000 7f346a3868c0 1 < completed part key=_multipart_6XDS0A7L0IY8.2~118E72WNIM31.0 size=145183 2024-04-07T16:38:00.283 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.288+0000 7f346a3868c0 1 < completed multipart upload key=6XDS0A7L0IY8 upload=118E72WNIM31 tag=KTZMZ531ZHPW type=write size=1193759 2024-04-07T16:38:00.294 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.300+0000 7f346a3868c0 1 > prepared part key=_multipart_IP8YW2A75TCW.2~HUPRE3XIM4KK.0 size=1048576 2024-04-07T16:38:00.299 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.308+0000 7f346a3868c0 1 > prepared part key=_multipart_IP8YW2A75TCW.2~HUPRE3XIM4KK.1 size=1048576 2024-04-07T16:38:00.305 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.312+0000 7f346a3868c0 1 > prepared part key=_multipart_IP8YW2A75TCW.2~HUPRE3XIM4KK.2 size=115564 2024-04-07T16:38:00.305 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.312+0000 7f346a3868c0 1 > prepared multipart upload key=IP8YW2A75TCW upload=HUPRE3XIM4KK tag=O20WA8SOBN2K type=write size=3261292 2024-04-07T16:38:00.310 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.316+0000 7f346a3868c0 1 < completed delete operation key=1VBSUWNI3V1T tag=SPH0TK8EP4UF type=del 2024-04-07T16:38:00.315 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.324+0000 7f346a3868c0 1 > prepared operation key=95YVKJ14JD8W tag=CNLAPC55AN3U type=write size=960483 2024-04-07T16:38:00.326 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.332+0000 7f346a3868c0 1 < suggested remove operation key=IP8YW2A75TCW tag=JIB558V7MV24 type=write 2024-04-07T16:38:00.331 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.340+0000 7f346a3868c0 1 > prepared operation key=UIW3DN4NUYPP tag=5ZXFEWSF7X8S type=del size=2954273 2024-04-07T16:38:00.337 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.344+0000 7f346a3868c0 1 < completed delete operation key=QS4HYZFFAJNF tag=VRR61JARABJO type=del 2024-04-07T16:38:00.348 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.356+0000 7f346a3868c0 1 > prepared part key=_multipart_E84U0SX44NQ8.2~1RZCCV5YGLI1.0 size=1048576 2024-04-07T16:38:00.352 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.360+0000 7f346a3868c0 1 > prepared part key=_multipart_E84U0SX44NQ8.2~1RZCCV5YGLI1.1 size=1048576 2024-04-07T16:38:00.358 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.364+0000 7f346a3868c0 1 > prepared part key=_multipart_E84U0SX44NQ8.2~1RZCCV5YGLI1.2 size=445568 2024-04-07T16:38:00.358 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.364+0000 7f346a3868c0 1 > prepared multipart upload key=E84U0SX44NQ8 upload=1RZCCV5YGLI1 tag=R5V3F4CSRIFD type=write size=3591296 2024-04-07T16:38:00.364 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.372+0000 7f346a3868c0 1 < completed delete operation key=3N0QJRYVO3FV tag=27R1MLIRAK7A type=del 2024-04-07T16:38:00.376 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.384+0000 7f346a3868c0 1 > prepared part key=_multipart_QTCIIPTXIRW2.2~EE4TCCNZD5HU.0 size=363006 2024-04-07T16:38:00.376 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.384+0000 7f346a3868c0 1 > prepared multipart upload key=QTCIIPTXIRW2 upload=EE4TCCNZD5HU tag=409O3E9BJW1V type=write size=1411582 2024-04-07T16:38:00.382 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.388+0000 7f346a3868c0 1 < completed write operation key=UIW3DN4NUYPP tag=IKLSPZGQGQFD type=write size=678016 2024-04-07T16:38:00.393 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.400+0000 7f346a3868c0 1 > prepared part key=_multipart_LXOZ0T2KY6OR.2~PMYR16MUDJT4.0 size=1048576 2024-04-07T16:38:00.398 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.404+0000 7f346a3868c0 1 > prepared part key=_multipart_LXOZ0T2KY6OR.2~PMYR16MUDJT4.1 size=1048576 2024-04-07T16:38:00.403 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.412+0000 7f346a3868c0 1 > prepared part key=_multipart_LXOZ0T2KY6OR.2~PMYR16MUDJT4.2 size=972489 2024-04-07T16:38:00.403 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.412+0000 7f346a3868c0 1 > prepared multipart upload key=LXOZ0T2KY6OR upload=PMYR16MUDJT4 tag=DRKAZOPQ3WL0 type=write size=4118217 2024-04-07T16:38:00.409 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.416+0000 7f346a3868c0 1 < completed delete operation key=E84U0SX44NQ8 tag=HHIFKZTPNJUO type=del 2024-04-07T16:38:00.417 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.424+0000 7f346a3868c0 1 > prepared operation key=DXTO3AJ429AC tag=VLLO1Y2T2XB4 type=write size=996273 2024-04-07T16:38:00.422 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.428+0000 7f346a3868c0 1 < completed delete operation key=8F81G0P0YBFW tag=7I4P80OWSE1V type=del 2024-04-07T16:38:00.428 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.436+0000 7f346a3868c0 1 > prepared operation key=QVOKSQAGHPA9 tag=1PIGSPD5HHDH type=del size=171556 2024-04-07T16:38:00.429 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.436+0000 7f346a3868c0 1 < canceled part key=_multipart_YOK8BAEOMPUR.2~EE5O5OW86DD9.0 size=1048576 2024-04-07T16:38:00.434 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.440+0000 7f346a3868c0 1 < completed part key=_multipart_YOK8BAEOMPUR.2~EE5O5OW86DD9.1 size=689108 2024-04-07T16:38:00.441 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:38:00 smithi100 bash[74942]: cluster 2024-04-07T16:37:59.063836+0000 mgr.y (mgr.25175) 679 : cluster [DBG] pgmap v1314: 137 pgs: 137 active+clean; 583 KiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 682 KiB/s rd, 236 KiB/s wr, 1.21k op/s 2024-04-07T16:38:00.441 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:38:00 smithi100 bash[76202]: cluster 2024-04-07T16:37:59.063836+0000 mgr.y (mgr.25175) 679 : cluster [DBG] pgmap v1314: 137 pgs: 137 active+clean; 583 KiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 682 KiB/s rd, 236 KiB/s wr, 1.21k op/s 2024-04-07T16:38:00.444 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.452+0000 7f346a3868c0 1 < canceled multipart upload key=YOK8BAEOMPUR upload=EE5O5OW86DD9 tag=I2UJUFNE1PZ1 type=write size=2786260 2024-04-07T16:38:00.454 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.460+0000 7f346a3868c0 1 > prepared part key=_multipart_1VBSUWNI3V1T.2~Q9SLC9NF0ZWU.0 size=1048576 2024-04-07T16:38:00.458 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.464+0000 7f346a3868c0 1 > prepared part key=_multipart_1VBSUWNI3V1T.2~Q9SLC9NF0ZWU.1 size=1048576 2024-04-07T16:38:00.462 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.468+0000 7f346a3868c0 1 > prepared part key=_multipart_1VBSUWNI3V1T.2~Q9SLC9NF0ZWU.2 size=562248 2024-04-07T16:38:00.462 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.468+0000 7f346a3868c0 1 > prepared multipart upload key=1VBSUWNI3V1T upload=Q9SLC9NF0ZWU tag=UA4N6XQTEU9T type=write size=3707976 2024-04-07T16:38:00.468 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.476+0000 7f346a3868c0 1 < completed part key=_multipart_EQR0FY86AT5X.2~9U1U4ULGYMTM.0 size=1048576 2024-04-07T16:38:00.473 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.480+0000 7f346a3868c0 1 < completed part key=_multipart_EQR0FY86AT5X.2~9U1U4ULGYMTM.1 size=420468 2024-04-07T16:38:00.483 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.492+0000 7f346a3868c0 1 < completed multipart upload key=EQR0FY86AT5X upload=9U1U4ULGYMTM tag=ANPY1L59C6RM type=write size=2517620 2024-04-07T16:38:00.613 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:38:00 smithi144 bash[66654]: cluster 2024-04-07T16:37:59.063836+0000 mgr.y (mgr.25175) 679 : cluster [DBG] pgmap v1314: 137 pgs: 137 active+clean; 583 KiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 682 KiB/s rd, 236 KiB/s wr, 1.21k op/s 2024-04-07T16:38:00.617 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.496+0000 7f346a3868c0 1 > prepared operation key=8F81G0P0YBFW tag=9JDBC1SPM7BL type=del size=3002271 2024-04-07T16:38:00.617 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.504+0000 7f346a3868c0 1 < completed delete operation key=NYET3212USVE tag=SKRVGYIH6PQF type=del 2024-04-07T16:38:00.617 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.508+0000 7f346a3868c0 1 > prepared operation key=NYET3212USVE tag=MF267259P7QU type=del size=1050730 2024-04-07T16:38:00.617 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.520+0000 7f346a3868c0 1 < suggested update operation key=YXMDZB23YTB3 tag=J79PRFRPICY7 type=del size=877818 2024-04-07T16:38:00.617 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.524+0000 7f346a3868c0 1 > prepared operation key=U3EID5EK64HD tag=9QA58VBYG4HA type=del size=850729 2024-04-07T16:38:00.618 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.528+0000 7f346a3868c0 1 < completed part key=_multipart_EQR0FY86AT5X.2~WGPSU41F208V.0 size=1048576 2024-04-07T16:38:00.618 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.536+0000 7f346a3868c0 1 < completed part key=_multipart_EQR0FY86AT5X.2~WGPSU41F208V.1 size=702858 2024-04-07T16:38:00.618 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.544+0000 7f346a3868c0 1 < completed multipart upload key=EQR0FY86AT5X upload=WGPSU41F208V tag=HNXT0VSJAC1T type=write size=2800010 2024-04-07T16:38:00.618 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.556+0000 7f346a3868c0 1 > prepared part key=_multipart_7G5C6HBK6QWI.2~YPMKYXJRUGG8.0 size=1048576 2024-04-07T16:38:00.618 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.560+0000 7f346a3868c0 1 > prepared part key=_multipart_7G5C6HBK6QWI.2~YPMKYXJRUGG8.1 size=348270 2024-04-07T16:38:00.618 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.560+0000 7f346a3868c0 1 > prepared multipart upload key=7G5C6HBK6QWI upload=YPMKYXJRUGG8 tag=UWJG39WNBAJN type=write size=2445422 2024-04-07T16:38:00.618 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.568+0000 7f346a3868c0 1 < completed delete operation key=UIW3DN4NUYPP tag=U9DRFN8DR325 type=del 2024-04-07T16:38:00.618 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.576+0000 7f346a3868c0 1 > prepared operation key=LXOZ0T2KY6OR tag=6F837LOK7AUX type=del size=2559462 2024-04-07T16:38:00.619 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.584+0000 7f346a3868c0 -1 < failed to complete operation key=YXMDZB23YTB3 tag=XPT7O96FY286 type=write size=518997: (22) Invalid argument 2024-04-07T16:38:00.619 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.588+0000 7f346a3868c0 1 > prepared operation key=N14BUXLJONHZ tag=OV2CYLJ7SPZG type=del size=1489338 2024-04-07T16:38:00.619 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.592+0000 7f346a3868c0 1 < completed part key=_multipart_N14BUXLJONHZ.2~EUE3PP1Y1AW1.0 size=1048576 2024-04-07T16:38:00.619 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.600+0000 7f346a3868c0 1 < completed part key=_multipart_N14BUXLJONHZ.2~EUE3PP1Y1AW1.1 size=605060 2024-04-07T16:38:00.619 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.608+0000 7f346a3868c0 1 < completed multipart upload key=N14BUXLJONHZ upload=EUE3PP1Y1AW1 tag=03TAKMS22CZN type=write size=2702212 2024-04-07T16:38:00.619 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.620+0000 7f346a3868c0 1 > prepared part key=_multipart_GZXHYZO5SMUP.2~YPRF3W4OQFU9.0 size=1048576 2024-04-07T16:38:00.619 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.624+0000 7f346a3868c0 1 > prepared part key=_multipart_GZXHYZO5SMUP.2~YPRF3W4OQFU9.1 size=555644 2024-04-07T16:38:00.619 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.624+0000 7f346a3868c0 1 > prepared multipart upload key=GZXHYZO5SMUP upload=YPRF3W4OQFU9 tag=3QYLHGMH17HQ type=write size=2652796 2024-04-07T16:38:00.622 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.628+0000 7f346a3868c0 1 < completed part key=_multipart_IP8YW2A75TCW.2~HUPRE3XIM4KK.0 size=1048576 2024-04-07T16:38:00.627 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.636+0000 7f346a3868c0 1 < completed part key=_multipart_IP8YW2A75TCW.2~HUPRE3XIM4KK.1 size=1048576 2024-04-07T16:38:00.633 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.640+0000 7f346a3868c0 1 < completed part key=_multipart_IP8YW2A75TCW.2~HUPRE3XIM4KK.2 size=115564 2024-04-07T16:38:00.644 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.652+0000 7f346a3868c0 1 < completed multipart upload key=IP8YW2A75TCW upload=HUPRE3XIM4KK tag=O20WA8SOBN2K type=write size=3261292 2024-04-07T16:38:00.650 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.656+0000 7f346a3868c0 1 > prepared operation key=U3EID5EK64HD tag=FLS2YOP2U5MH type=del size=990060 2024-04-07T16:38:00.658 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.664+0000 7f346a3868c0 1 < completed write operation key=95YVKJ14JD8W tag=CNLAPC55AN3U type=write size=960483 2024-04-07T16:38:00.664 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.672+0000 7f346a3868c0 1 > prepared operation key=YOK8BAEOMPUR tag=UF1UEU02VG91 type=del size=2587888 2024-04-07T16:38:00.670 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.676+0000 7f346a3868c0 1 < completed delete operation key=UIW3DN4NUYPP tag=5ZXFEWSF7X8S type=del 2024-04-07T16:38:00.675 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.684+0000 7f346a3868c0 1 > prepared operation key=6XDS0A7L0IY8 tag=XYCH1SBXID80 type=write size=110326 2024-04-07T16:38:00.681 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.688+0000 7f346a3868c0 1 < completed part key=_multipart_E84U0SX44NQ8.2~1RZCCV5YGLI1.0 size=1048576 2024-04-07T16:38:00.686 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.692+0000 7f346a3868c0 1 < completed part key=_multipart_E84U0SX44NQ8.2~1RZCCV5YGLI1.1 size=1048576 2024-04-07T16:38:00.691 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.696+0000 7f346a3868c0 1 < completed part key=_multipart_E84U0SX44NQ8.2~1RZCCV5YGLI1.2 size=445568 2024-04-07T16:38:00.701 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.708+0000 7f346a3868c0 1 < completed multipart upload key=E84U0SX44NQ8 upload=1RZCCV5YGLI1 tag=R5V3F4CSRIFD type=write size=3591296 2024-04-07T16:38:00.706 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.712+0000 7f346a3868c0 1 > prepared operation key=HBLEXONKXJTL tag=EBHYERALUJAD type=write size=181069 2024-04-07T16:38:00.713 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.720+0000 7f346a3868c0 1 < completed part key=_multipart_QTCIIPTXIRW2.2~EE4TCCNZD5HU.0 size=363006 2024-04-07T16:38:00.723 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.732+0000 7f346a3868c0 1 < completed multipart upload key=QTCIIPTXIRW2 upload=EE4TCCNZD5HU tag=409O3E9BJW1V type=write size=1411582 2024-04-07T16:38:00.729 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.736+0000 7f346a3868c0 1 > prepared operation key=GZXHYZO5SMUP tag=D95ZROP0QGW1 type=del size=1455656 2024-04-07T16:38:00.735 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.740+0000 7f346a3868c0 1 < completed part key=_multipart_LXOZ0T2KY6OR.2~PMYR16MUDJT4.0 size=1048576 2024-04-07T16:38:00.739 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.748+0000 7f346a3868c0 1 < completed part key=_multipart_LXOZ0T2KY6OR.2~PMYR16MUDJT4.1 size=1048576 2024-04-07T16:38:00.744 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.752+0000 7f346a3868c0 1 < completed part key=_multipart_LXOZ0T2KY6OR.2~PMYR16MUDJT4.2 size=972489 2024-04-07T16:38:00.754 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.760+0000 7f346a3868c0 1 < completed multipart upload key=LXOZ0T2KY6OR upload=PMYR16MUDJT4 tag=DRKAZOPQ3WL0 type=write size=4118217 2024-04-07T16:38:00.759 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.768+0000 7f346a3868c0 1 > prepared operation key=QTCIIPTXIRW2 tag=4GATFIZDUJIA type=del size=2247836 2024-04-07T16:38:00.765 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.772+0000 7f346a3868c0 1 < completed write operation key=DXTO3AJ429AC tag=VLLO1Y2T2XB4 type=write size=996273 2024-04-07T16:38:00.776 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.784+0000 7f346a3868c0 1 > prepared part key=_multipart_YOK8BAEOMPUR.2~5IZEOMB1ZZNL.0 size=898910 2024-04-07T16:38:00.776 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.784+0000 7f346a3868c0 1 > prepared multipart upload key=YOK8BAEOMPUR upload=5IZEOMB1ZZNL tag=Q06JJ2AA84FT type=write size=1947486 2024-04-07T16:38:00.781 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.788+0000 7f346a3868c0 1 < completed delete operation key=QVOKSQAGHPA9 tag=1PIGSPD5HHDH type=del 2024-04-07T16:38:00.787 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.792+0000 7f346a3868c0 1 > prepared operation key=UIW3DN4NUYPP tag=22PXMWWNEKTP type=del size=515095 2024-04-07T16:38:00.788 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.796+0000 7f346a3868c0 1 < canceled part key=_multipart_1VBSUWNI3V1T.2~Q9SLC9NF0ZWU.0 size=1048576 2024-04-07T16:38:00.793 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.800+0000 7f346a3868c0 1 < completed part key=_multipart_1VBSUWNI3V1T.2~Q9SLC9NF0ZWU.1 size=1048576 2024-04-07T16:38:00.798 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.804+0000 7f346a3868c0 1 < completed part key=_multipart_1VBSUWNI3V1T.2~Q9SLC9NF0ZWU.2 size=562248 2024-04-07T16:38:00.807 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.816+0000 7f346a3868c0 1 < canceled multipart upload key=1VBSUWNI3V1T upload=Q9SLC9NF0ZWU tag=UA4N6XQTEU9T type=write size=3707976 2024-04-07T16:38:00.813 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.820+0000 7f346a3868c0 1 > prepared operation key=NYET3212USVE tag=5ZS1YP18NQ97 type=del size=1891290 2024-04-07T16:38:00.819 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.828+0000 7f346a3868c0 1 < completed delete operation key=8F81G0P0YBFW tag=9JDBC1SPM7BL type=del 2024-04-07T16:38:00.829 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.836+0000 7f346a3868c0 1 > prepared part key=_multipart_U3EID5EK64HD.2~974MV2WJLRBE.0 size=283087 2024-04-07T16:38:00.830 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.836+0000 7f346a3868c0 1 > prepared multipart upload key=U3EID5EK64HD upload=974MV2WJLRBE tag=CCCL0HNFCYS8 type=write size=1331663 2024-04-07T16:38:00.836 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.844+0000 7f346a3868c0 1 < completed delete operation key=NYET3212USVE tag=MF267259P7QU type=del 2024-04-07T16:38:00.842 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.848+0000 7f346a3868c0 1 > prepared operation key=X5Q46GO8YTOK tag=DCDZGDE3M8C7 type=del size=2444422 2024-04-07T16:38:00.848 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.856+0000 7f346a3868c0 1 < completed delete operation key=U3EID5EK64HD tag=9QA58VBYG4HA type=del 2024-04-07T16:38:00.860 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.868+0000 7f346a3868c0 1 > prepared part key=_multipart_8F81G0P0YBFW.2~A3SABWLFF05B.0 size=1048576 2024-04-07T16:38:00.865 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.872+0000 7f346a3868c0 1 > prepared part key=_multipart_8F81G0P0YBFW.2~A3SABWLFF05B.1 size=23986 2024-04-07T16:38:00.865 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.872+0000 7f346a3868c0 1 > prepared multipart upload key=8F81G0P0YBFW upload=A3SABWLFF05B tag=2QYQK1XJHA7M type=write size=2121138 2024-04-07T16:38:00.866 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.872+0000 7f346a3868c0 1 < canceled part key=_multipart_7G5C6HBK6QWI.2~YPMKYXJRUGG8.0 size=1048576 2024-04-07T16:38:00.870 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.876+0000 7f346a3868c0 1 < completed part key=_multipart_7G5C6HBK6QWI.2~YPMKYXJRUGG8.1 size=348270 2024-04-07T16:38:00.879 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.888+0000 7f346a3868c0 1 < canceled multipart upload key=7G5C6HBK6QWI upload=YPMKYXJRUGG8 tag=UWJG39WNBAJN type=write size=2445422 2024-04-07T16:38:00.890 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.896+0000 7f346a3868c0 1 > prepared part key=_multipart_95YVKJ14JD8W.2~X892TC9CCU6K.0 size=1048576 2024-04-07T16:38:00.895 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.904+0000 7f346a3868c0 1 > prepared part key=_multipart_95YVKJ14JD8W.2~X892TC9CCU6K.1 size=683759 2024-04-07T16:38:00.895 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.904+0000 7f346a3868c0 1 > prepared multipart upload key=95YVKJ14JD8W upload=X892TC9CCU6K tag=EWW7OM2S6BB2 type=write size=2780911 2024-04-07T16:38:00.901 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.908+0000 7f346a3868c0 1 < completed delete operation key=LXOZ0T2KY6OR tag=6F837LOK7AUX type=del 2024-04-07T16:38:00.911 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.920+0000 7f346a3868c0 1 > prepared part key=_multipart_HBLEXONKXJTL.2~UJFWR2H71Q3Z.0 size=843057 2024-04-07T16:38:00.911 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.920+0000 7f346a3868c0 1 > prepared multipart upload key=HBLEXONKXJTL upload=UJFWR2H71Q3Z tag=MBJBOQMW3FXO type=write size=1891633 2024-04-07T16:38:00.922 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.928+0000 7f346a3868c0 1 < suggested update operation key=N14BUXLJONHZ tag=OV2CYLJ7SPZG type=del size=1489338 2024-04-07T16:38:00.929 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.936+0000 7f346a3868c0 1 > prepared operation key=1WGUPH7X5UA0 tag=L1H5AAD6JEFU type=del size=3921358 2024-04-07T16:38:00.934 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.940+0000 7f346a3868c0 1 < completed part key=_multipart_GZXHYZO5SMUP.2~YPRF3W4OQFU9.0 size=1048576 2024-04-07T16:38:00.939 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.948+0000 7f346a3868c0 1 < completed part key=_multipart_GZXHYZO5SMUP.2~YPRF3W4OQFU9.1 size=555644 2024-04-07T16:38:00.949 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.956+0000 7f346a3868c0 1 < completed multipart upload key=GZXHYZO5SMUP upload=YPRF3W4OQFU9 tag=3QYLHGMH17HQ type=write size=2652796 2024-04-07T16:38:00.959 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.968+0000 7f346a3868c0 1 > prepared part key=_multipart_E84U0SX44NQ8.2~Y04GDYCGBP0B.0 size=1036281 2024-04-07T16:38:00.960 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.968+0000 7f346a3868c0 1 > prepared multipart upload key=E84U0SX44NQ8 upload=Y04GDYCGBP0B tag=IJXCFVS62Y2O type=write size=2084857 2024-04-07T16:38:00.965 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.972+0000 7f346a3868c0 1 < completed delete operation key=U3EID5EK64HD tag=FLS2YOP2U5MH type=del 2024-04-07T16:38:00.975 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.984+0000 7f346a3868c0 1 > prepared part key=_multipart_1VBSUWNI3V1T.2~EA32OKVJZX13.0 size=9019 2024-04-07T16:38:00.976 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.984+0000 7f346a3868c0 1 > prepared multipart upload key=1VBSUWNI3V1T upload=EA32OKVJZX13 tag=9DC82PQL763C type=write size=1057595 2024-04-07T16:38:00.981 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.988+0000 7f346a3868c0 1 < completed delete operation key=YOK8BAEOMPUR tag=UF1UEU02VG91 type=del 2024-04-07T16:38:00.987 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:00.992+0000 7f346a3868c0 1 > prepared operation key=GBQSME7SY1AH tag=2Q9D8A3FML86 type=del size=512111 2024-04-07T16:38:00.992 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.000+0000 7f346a3868c0 1 < canceled operation key=6XDS0A7L0IY8 tag=XYCH1SBXID80 type=write size=110326 2024-04-07T16:38:00.998 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.004+0000 7f346a3868c0 1 > prepared operation key=QVOKSQAGHPA9 tag=NE8K5POJRCKY type=write size=994004 2024-04-07T16:38:01.009 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.016+0000 7f346a3868c0 1 < suggested remove operation key=HBLEXONKXJTL tag=EBHYERALUJAD type=write 2024-04-07T16:38:01.014 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.020+0000 7f346a3868c0 1 > prepared operation key=NYET3212USVE tag=8WVU4WZ23CF4 type=del size=3832705 2024-04-07T16:38:01.020 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.028+0000 7f346a3868c0 1 < canceled operation key=GZXHYZO5SMUP tag=D95ZROP0QGW1 type=del size=1455656 2024-04-07T16:38:01.031 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.036+0000 7f346a3868c0 1 > prepared part key=_multipart_XJLQZ7IS9U1T.2~LAF1NDJVYWX5.0 size=1048576 2024-04-07T16:38:01.036 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.044+0000 7f346a3868c0 1 > prepared part key=_multipart_XJLQZ7IS9U1T.2~LAF1NDJVYWX5.1 size=779870 2024-04-07T16:38:01.036 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.044+0000 7f346a3868c0 1 > prepared multipart upload key=XJLQZ7IS9U1T upload=LAF1NDJVYWX5 tag=VILQT1B12YV6 type=write size=2877022 2024-04-07T16:38:01.041 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.048+0000 7f346a3868c0 1 < canceled operation key=QTCIIPTXIRW2 tag=4GATFIZDUJIA type=del size=2247836 2024-04-07T16:38:01.047 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.052+0000 7f346a3868c0 1 > prepared operation key=1VKVQXSQMTU1 tag=OB8387G6IF64 type=del size=2157266 2024-04-07T16:38:01.053 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.060+0000 7f346a3868c0 1 < completed part key=_multipart_YOK8BAEOMPUR.2~5IZEOMB1ZZNL.0 size=898910 2024-04-07T16:38:01.063 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.068+0000 7f346a3868c0 1 < completed multipart upload key=YOK8BAEOMPUR upload=5IZEOMB1ZZNL tag=Q06JJ2AA84FT type=write size=1947486 2024-04-07T16:38:01.068 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.076+0000 7f346a3868c0 1 > prepared operation key=7G5C6HBK6QWI tag=LA7HCDPTKNLX type=del size=1295071 2024-04-07T16:38:01.073 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.080+0000 7f346a3868c0 1 < completed delete operation key=UIW3DN4NUYPP tag=22PXMWWNEKTP type=del 2024-04-07T16:38:01.079 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.084+0000 7f346a3868c0 1 > prepared operation key=1WGUPH7X5UA0 tag=FYSNFAEHMJ2G type=del size=2808611 2024-04-07T16:38:01.084 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.092+0000 7f346a3868c0 1 < completed delete operation key=NYET3212USVE tag=5ZS1YP18NQ97 type=del 2024-04-07T16:38:01.089 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.096+0000 7f346a3868c0 1 > prepared operation key=DXTO3AJ429AC tag=IU2Q9CXQXS48 type=del size=3375690 2024-04-07T16:38:01.096 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.104+0000 7f346a3868c0 1 < completed part key=_multipart_U3EID5EK64HD.2~974MV2WJLRBE.0 size=283087 2024-04-07T16:38:01.106 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.112+0000 7f346a3868c0 1 < completed multipart upload key=U3EID5EK64HD upload=974MV2WJLRBE tag=CCCL0HNFCYS8 type=write size=1331663 2024-04-07T16:38:01.112 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.120+0000 7f346a3868c0 1 > prepared operation key=6XDS0A7L0IY8 tag=CMSDQ558PKES type=del size=3824440 2024-04-07T16:38:01.119 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.128+0000 7f346a3868c0 1 < completed delete operation key=X5Q46GO8YTOK tag=DCDZGDE3M8C7 type=del 2024-04-07T16:38:01.130 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.136+0000 7f346a3868c0 1 > prepared part key=_multipart_QVOKSQAGHPA9.2~FTWDD0UCJ3DE.0 size=42881 2024-04-07T16:38:01.131 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.136+0000 7f346a3868c0 1 > prepared multipart upload key=QVOKSQAGHPA9 upload=FTWDD0UCJ3DE tag=ARLFQTQ6AT6D type=write size=1091457 2024-04-07T16:38:01.132 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.140+0000 7f346a3868c0 1 < canceled part key=_multipart_8F81G0P0YBFW.2~A3SABWLFF05B.0 size=1048576 2024-04-07T16:38:01.137 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.144+0000 7f346a3868c0 1 < completed part key=_multipart_8F81G0P0YBFW.2~A3SABWLFF05B.1 size=23986 2024-04-07T16:38:01.147 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.152+0000 7f346a3868c0 1 < canceled multipart upload key=8F81G0P0YBFW upload=A3SABWLFF05B tag=2QYQK1XJHA7M type=write size=2121138 2024-04-07T16:38:01.153 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.160+0000 7f346a3868c0 1 > prepared operation key=1VBSUWNI3V1T tag=Z0I9MYF5MEE2 type=write size=291070 2024-04-07T16:38:01.154 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.160+0000 7f346a3868c0 1 < canceled part key=_multipart_95YVKJ14JD8W.2~X892TC9CCU6K.0 size=1048576 2024-04-07T16:38:01.159 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.168+0000 7f346a3868c0 1 < completed part key=_multipart_95YVKJ14JD8W.2~X892TC9CCU6K.1 size=683759 2024-04-07T16:38:01.170 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.176+0000 7f346a3868c0 1 < canceled multipart upload key=95YVKJ14JD8W upload=X892TC9CCU6K tag=EWW7OM2S6BB2 type=write size=2780911 2024-04-07T16:38:01.176 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.184+0000 7f346a3868c0 1 > prepared operation key=8F81G0P0YBFW tag=2YN2N3R58MIR type=del size=1218193 2024-04-07T16:38:01.182 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.188+0000 7f346a3868c0 1 < completed part key=_multipart_HBLEXONKXJTL.2~UJFWR2H71Q3Z.0 size=843057 2024-04-07T16:38:01.192 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.200+0000 7f346a3868c0 1 < completed multipart upload key=HBLEXONKXJTL upload=UJFWR2H71Q3Z tag=MBJBOQMW3FXO type=write size=1891633 2024-04-07T16:38:01.197 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.204+0000 7f346a3868c0 1 > prepared operation key=CL4PP1RKZU0E tag=IXL15NRMWJ19 type=del size=3557082 2024-04-07T16:38:01.203 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.208+0000 7f346a3868c0 1 < completed delete operation key=1WGUPH7X5UA0 tag=L1H5AAD6JEFU type=del 2024-04-07T16:38:01.208 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.216+0000 7f346a3868c0 1 > prepared operation key=EOYFVEI50SQ1 tag=ZRKHMIU2OOD9 type=del size=3929400 2024-04-07T16:38:01.214 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.220+0000 7f346a3868c0 1 < completed part key=_multipart_E84U0SX44NQ8.2~Y04GDYCGBP0B.0 size=1036281 2024-04-07T16:38:01.225 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.232+0000 7f346a3868c0 1 < completed multipart upload key=E84U0SX44NQ8 upload=Y04GDYCGBP0B tag=IJXCFVS62Y2O type=write size=2084857 2024-04-07T16:38:01.231 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.240+0000 7f346a3868c0 1 > prepared operation key=1WGUPH7X5UA0 tag=39OUXRFHG07D type=del size=2406451 2024-04-07T16:38:01.232 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.240+0000 7f346a3868c0 1 < canceled part key=_multipart_1VBSUWNI3V1T.2~EA32OKVJZX13.0 size=9019 2024-04-07T16:38:01.242 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.248+0000 7f346a3868c0 1 < canceled multipart upload key=1VBSUWNI3V1T upload=EA32OKVJZX13 tag=9DC82PQL763C type=write size=1057595 2024-04-07T16:38:01.248 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.256+0000 7f346a3868c0 1 > prepared operation key=ROFH30W3UCAX tag=C0TDAKBNQDZI type=del size=3993800 2024-04-07T16:38:01.253 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.260+0000 7f346a3868c0 1 < completed delete operation key=GBQSME7SY1AH tag=2Q9D8A3FML86 type=del 2024-04-07T16:38:01.259 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.268+0000 7f346a3868c0 1 > prepared operation key=GZXHYZO5SMUP tag=PPIYPF0FL49R type=del size=73357 2024-04-07T16:38:01.265 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.272+0000 7f346a3868c0 1 < completed write operation key=QVOKSQAGHPA9 tag=NE8K5POJRCKY type=write size=994004 2024-04-07T16:38:01.277 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.284+0000 7f346a3868c0 1 > prepared part key=_multipart_IP8YW2A75TCW.2~VXFQE6BVH5TK.0 size=1048576 2024-04-07T16:38:01.282 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.288+0000 7f346a3868c0 1 > prepared part key=_multipart_IP8YW2A75TCW.2~VXFQE6BVH5TK.1 size=812180 2024-04-07T16:38:01.282 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.288+0000 7f346a3868c0 1 > prepared multipart upload key=IP8YW2A75TCW upload=VXFQE6BVH5TK tag=XSULWR7MWPB9 type=write size=2909332 2024-04-07T16:38:01.296 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.304+0000 7f346a3868c0 1 < suggested remove operation key=NYET3212USVE tag=8WVU4WZ23CF4 type=del 2024-04-07T16:38:01.308 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.316+0000 7f346a3868c0 1 > prepared part key=_multipart_1VKVQXSQMTU1.2~XEZ61C4PBC3M.0 size=394765 2024-04-07T16:38:01.309 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.316+0000 7f346a3868c0 1 > prepared multipart upload key=1VKVQXSQMTU1 upload=XEZ61C4PBC3M tag=IGXZ8TNY6YYJ type=write size=1443341 2024-04-07T16:38:01.315 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.320+0000 7f346a3868c0 1 < completed part key=_multipart_XJLQZ7IS9U1T.2~LAF1NDJVYWX5.0 size=1048576 2024-04-07T16:38:01.320 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.328+0000 7f346a3868c0 1 < completed part key=_multipart_XJLQZ7IS9U1T.2~LAF1NDJVYWX5.1 size=779870 2024-04-07T16:38:01.329 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.336+0000 7f346a3868c0 1 < completed multipart upload key=XJLQZ7IS9U1T upload=LAF1NDJVYWX5 tag=VILQT1B12YV6 type=write size=2877022 2024-04-07T16:38:01.338 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.344+0000 7f346a3868c0 1 > prepared operation key=QS4HYZFFAJNF tag=XHE28LYUI5OA type=del size=269886 2024-04-07T16:38:01.344 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.352+0000 7f346a3868c0 1 < completed delete operation key=1VKVQXSQMTU1 tag=OB8387G6IF64 type=del 2024-04-07T16:38:01.350 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.356+0000 7f346a3868c0 1 > prepared operation key=1WGUPH7X5UA0 tag=CG8FI5AMW81D type=write size=563469 2024-04-07T16:38:01.355 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.364+0000 7f346a3868c0 1 < completed delete operation key=7G5C6HBK6QWI tag=LA7HCDPTKNLX type=del 2024-04-07T16:38:01.365 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.372+0000 7f346a3868c0 1 > prepared part key=_multipart_NYET3212USVE.2~7AN2ZPZRI8C7.0 size=1048576 2024-04-07T16:38:01.370 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.376+0000 7f346a3868c0 1 > prepared part key=_multipart_NYET3212USVE.2~7AN2ZPZRI8C7.1 size=1048576 2024-04-07T16:38:01.375 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.380+0000 7f346a3868c0 1 > prepared part key=_multipart_NYET3212USVE.2~7AN2ZPZRI8C7.2 size=191018 2024-04-07T16:38:01.375 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.380+0000 7f346a3868c0 1 > prepared multipart upload key=NYET3212USVE upload=7AN2ZPZRI8C7 tag=X67HDZR1HWQE type=write size=3336746 2024-04-07T16:38:01.380 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.388+0000 7f346a3868c0 1 < completed delete operation key=1WGUPH7X5UA0 tag=FYSNFAEHMJ2G type=del 2024-04-07T16:38:01.391 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.400+0000 7f346a3868c0 1 > prepared part key=_multipart_NYET3212USVE.2~VWRJQQMJUWQA.0 size=1048576 2024-04-07T16:38:01.397 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.404+0000 7f346a3868c0 1 > prepared part key=_multipart_NYET3212USVE.2~VWRJQQMJUWQA.1 size=1048576 2024-04-07T16:38:01.401 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.408+0000 7f346a3868c0 1 > prepared part key=_multipart_NYET3212USVE.2~VWRJQQMJUWQA.2 size=288162 2024-04-07T16:38:01.401 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.408+0000 7f346a3868c0 1 > prepared multipart upload key=NYET3212USVE upload=VWRJQQMJUWQA tag=IVE73BZQBDT5 type=write size=3433890 2024-04-07T16:38:01.407 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.416+0000 7f346a3868c0 1 < canceled operation key=DXTO3AJ429AC tag=IU2Q9CXQXS48 type=del size=3375690 2024-04-07T16:38:01.418 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.424+0000 7f346a3868c0 1 > prepared part key=_multipart_LXOZ0T2KY6OR.2~47HAYZNVKNS2.0 size=1048576 2024-04-07T16:38:01.422 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.428+0000 7f346a3868c0 1 > prepared part key=_multipart_LXOZ0T2KY6OR.2~47HAYZNVKNS2.1 size=236906 2024-04-07T16:38:01.422 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.428+0000 7f346a3868c0 1 > prepared multipart upload key=LXOZ0T2KY6OR upload=47HAYZNVKNS2 tag=XPSIFPMF9OMC type=write size=2334058 2024-04-07T16:38:01.428 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.436+0000 7f346a3868c0 1 < completed delete operation key=6XDS0A7L0IY8 tag=CMSDQ558PKES type=del 2024-04-07T16:38:01.434 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.440+0000 7f346a3868c0 1 > prepared operation key=IX3F0IUOKBX1 tag=E70JHY9BMD9B type=write size=548688 2024-04-07T16:38:01.439 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.448+0000 7f346a3868c0 1 < completed part key=_multipart_QVOKSQAGHPA9.2~FTWDD0UCJ3DE.0 size=42881 2024-04-07T16:38:01.449 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.456+0000 7f346a3868c0 1 < completed multipart upload key=QVOKSQAGHPA9 upload=FTWDD0UCJ3DE tag=ARLFQTQ6AT6D type=write size=1091457 2024-04-07T16:38:01.461 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.468+0000 7f346a3868c0 1 > prepared part key=_multipart_E84U0SX44NQ8.2~MUOTQ445NACA.0 size=663784 2024-04-07T16:38:01.461 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.468+0000 7f346a3868c0 1 > prepared multipart upload key=E84U0SX44NQ8 upload=MUOTQ445NACA tag=G8Y0U56FQJWO type=write size=1712360 2024-04-07T16:38:01.473 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.480+0000 7f346a3868c0 1 < suggested update operation key=1VBSUWNI3V1T tag=Z0I9MYF5MEE2 type=write size=291070 2024-04-07T16:38:01.478 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.484+0000 7f346a3868c0 1 > prepared operation key=GBQSME7SY1AH tag=SZHZKX4LBOPP type=del size=1574298 2024-04-07T16:38:01.484 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.492+0000 7f346a3868c0 1 < canceled operation key=8F81G0P0YBFW tag=2YN2N3R58MIR type=del size=1218193 2024-04-07T16:38:01.490 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.496+0000 7f346a3868c0 1 > prepared operation key=C1BTVJNHIM0X tag=F2QSUBTBBVS0 type=del size=1094145 2024-04-07T16:38:01.496 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.504+0000 7f346a3868c0 1 < completed delete operation key=CL4PP1RKZU0E tag=IXL15NRMWJ19 type=del 2024-04-07T16:38:01.507 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.512+0000 7f346a3868c0 1 > prepared part key=_multipart_6XDS0A7L0IY8.2~GBKKXK1ADN2Y.0 size=516806 2024-04-07T16:38:01.507 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.512+0000 7f346a3868c0 1 > prepared multipart upload key=6XDS0A7L0IY8 upload=GBKKXK1ADN2Y tag=KTCRC64EGCUE type=write size=1565382 2024-04-07T16:38:01.512 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.520+0000 7f346a3868c0 1 < completed delete operation key=EOYFVEI50SQ1 tag=ZRKHMIU2OOD9 type=del 2024-04-07T16:38:01.518 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.524+0000 7f346a3868c0 1 > prepared operation key=HBLEXONKXJTL tag=282A0KNX9LKA type=del size=172966 2024-04-07T16:38:01.524 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.532+0000 7f346a3868c0 1 < completed delete operation key=1WGUPH7X5UA0 tag=39OUXRFHG07D type=del 2024-04-07T16:38:01.529 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.536+0000 7f346a3868c0 1 > prepared operation key=EOYFVEI50SQ1 tag=J2YUOMPCPVXU type=del size=2847743 2024-04-07T16:38:01.535 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.544+0000 7f346a3868c0 1 < completed delete operation key=ROFH30W3UCAX tag=C0TDAKBNQDZI type=del 2024-04-07T16:38:01.545 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.552+0000 7f346a3868c0 1 > prepared part key=_multipart_X5Q46GO8YTOK.2~OHUYW3F12L1V.0 size=1048576 2024-04-07T16:38:01.550 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.556+0000 7f346a3868c0 1 > prepared part key=_multipart_X5Q46GO8YTOK.2~OHUYW3F12L1V.1 size=1048576 2024-04-07T16:38:01.555 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.560+0000 7f346a3868c0 1 > prepared part key=_multipart_X5Q46GO8YTOK.2~OHUYW3F12L1V.2 size=838710 2024-04-07T16:38:01.555 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.560+0000 7f346a3868c0 1 > prepared multipart upload key=X5Q46GO8YTOK upload=OHUYW3F12L1V tag=782XWBZOZ11R type=write size=3984438 2024-04-07T16:38:01.560 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.568+0000 7f346a3868c0 1 < completed delete operation key=GZXHYZO5SMUP tag=PPIYPF0FL49R type=del 2024-04-07T16:38:01.566 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.572+0000 7f346a3868c0 1 > prepared operation key=IP8YW2A75TCW tag=KNOBADQG1RR8 type=del size=740011 2024-04-07T16:38:01.571 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.580+0000 7f346a3868c0 1 < completed part key=_multipart_IP8YW2A75TCW.2~VXFQE6BVH5TK.0 size=1048576 2024-04-07T16:38:01.577 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.584+0000 7f346a3868c0 1 < completed part key=_multipart_IP8YW2A75TCW.2~VXFQE6BVH5TK.1 size=812180 2024-04-07T16:38:01.588 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.596+0000 7f346a3868c0 1 < completed multipart upload key=IP8YW2A75TCW upload=VXFQE6BVH5TK tag=XSULWR7MWPB9 type=write size=2909332 2024-04-07T16:38:01.593 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.600+0000 7f346a3868c0 1 > prepared operation key=95YVKJ14JD8W tag=W7T7Y9H8FG9E type=del size=3431780 2024-04-07T16:38:01.598 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.604+0000 7f346a3868c0 1 < completed part key=_multipart_1VKVQXSQMTU1.2~XEZ61C4PBC3M.0 size=394765 2024-04-07T16:38:01.608 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.616+0000 7f346a3868c0 1 < completed multipart upload key=1VKVQXSQMTU1 upload=XEZ61C4PBC3M tag=IGXZ8TNY6YYJ type=write size=1443341 2024-04-07T16:38:01.613 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.620+0000 7f346a3868c0 1 > prepared operation key=1WGUPH7X5UA0 tag=R1525ZW20MD7 type=del size=3876105 2024-04-07T16:38:01.619 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.628+0000 7f346a3868c0 1 < completed delete operation key=QS4HYZFFAJNF tag=XHE28LYUI5OA type=del 2024-04-07T16:38:01.625 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.632+0000 7f346a3868c0 1 > prepared operation key=1VKVQXSQMTU1 tag=GF2S4QPRAJ96 type=del size=3575534 2024-04-07T16:38:01.631 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.640+0000 7f346a3868c0 1 < completed write operation key=1WGUPH7X5UA0 tag=CG8FI5AMW81D type=write size=563469 2024-04-07T16:38:01.637 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.644+0000 7f346a3868c0 1 > prepared operation key=1VBSUWNI3V1T tag=1V5BXP4GIWKP type=del size=3980425 2024-04-07T16:38:01.643 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.648+0000 7f346a3868c0 1 < completed part key=_multipart_NYET3212USVE.2~7AN2ZPZRI8C7.0 size=1048576 2024-04-07T16:38:01.648 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.656+0000 7f346a3868c0 1 < completed part key=_multipart_NYET3212USVE.2~7AN2ZPZRI8C7.1 size=1048576 2024-04-07T16:38:01.653 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.660+0000 7f346a3868c0 1 < completed part key=_multipart_NYET3212USVE.2~7AN2ZPZRI8C7.2 size=191018 2024-04-07T16:38:01.664 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.672+0000 7f346a3868c0 1 < completed multipart upload key=NYET3212USVE upload=7AN2ZPZRI8C7 tag=X67HDZR1HWQE type=write size=3336746 2024-04-07T16:38:01.670 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.676+0000 7f346a3868c0 1 > prepared operation key=QVOKSQAGHPA9 tag=TP5FBPDRMF3S type=del size=68877 2024-04-07T16:38:01.676 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.684+0000 7f346a3868c0 1 < completed part key=_multipart_NYET3212USVE.2~VWRJQQMJUWQA.0 size=1048576 2024-04-07T16:38:01.683 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.692+0000 7f346a3868c0 1 < completed part key=_multipart_NYET3212USVE.2~VWRJQQMJUWQA.1 size=1048576 2024-04-07T16:38:01.688 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.696+0000 7f346a3868c0 1 < completed part key=_multipart_NYET3212USVE.2~VWRJQQMJUWQA.2 size=288162 2024-04-07T16:38:01.697 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.704+0000 7f346a3868c0 1 < completed multipart upload key=NYET3212USVE upload=VWRJQQMJUWQA tag=IVE73BZQBDT5 type=write size=3433890 2024-04-07T16:38:01.703 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.712+0000 7f346a3868c0 1 > prepared operation key=X5Q46GO8YTOK tag=LLMUAVCRKTWY type=del size=420523 2024-04-07T16:38:01.709 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.716+0000 7f346a3868c0 1 < completed part key=_multipart_LXOZ0T2KY6OR.2~47HAYZNVKNS2.0 size=1048576 2024-04-07T16:38:01.714 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.720+0000 7f346a3868c0 1 < completed part key=_multipart_LXOZ0T2KY6OR.2~47HAYZNVKNS2.1 size=236906 2024-04-07T16:38:01.723 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.732+0000 7f346a3868c0 1 < completed multipart upload key=LXOZ0T2KY6OR upload=47HAYZNVKNS2 tag=XPSIFPMF9OMC type=write size=2334058 2024-04-07T16:38:01.734 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.740+0000 7f346a3868c0 1 > prepared part key=_multipart_7G5C6HBK6QWI.2~JEGFNBU4KINW.0 size=450365 2024-04-07T16:38:01.734 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.740+0000 7f346a3868c0 1 > prepared multipart upload key=7G5C6HBK6QWI upload=JEGFNBU4KINW tag=AVO2VMAH7KJ0 type=write size=1498941 2024-04-07T16:38:01.740 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.748+0000 7f346a3868c0 1 < completed write operation key=IX3F0IUOKBX1 tag=E70JHY9BMD9B type=write size=548688 2024-04-07T16:38:01.750 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.756+0000 7f346a3868c0 1 > prepared part key=_multipart_1WGUPH7X5UA0.2~3CTVMURHPXM9.0 size=1048576 2024-04-07T16:38:01.755 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.764+0000 7f346a3868c0 1 > prepared part key=_multipart_1WGUPH7X5UA0.2~3CTVMURHPXM9.1 size=833699 2024-04-07T16:38:01.755 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.764+0000 7f346a3868c0 1 > prepared multipart upload key=1WGUPH7X5UA0 upload=3CTVMURHPXM9 tag=OU2V1SU4ZV5F type=write size=2930851 2024-04-07T16:38:01.761 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.768+0000 7f346a3868c0 1 < completed part key=_multipart_E84U0SX44NQ8.2~MUOTQ445NACA.0 size=663784 2024-04-07T16:38:01.773 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.780+0000 7f346a3868c0 1 < completed multipart upload key=E84U0SX44NQ8 upload=MUOTQ445NACA tag=G8Y0U56FQJWO type=write size=1712360 2024-04-07T16:38:01.779 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.788+0000 7f346a3868c0 1 > prepared operation key=IP8YW2A75TCW tag=68T4CU32VHXP type=del size=2049967 2024-04-07T16:38:01.785 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.792+0000 7f346a3868c0 1 < completed delete operation key=GBQSME7SY1AH tag=SZHZKX4LBOPP type=del 2024-04-07T16:38:01.791 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.800+0000 7f346a3868c0 1 > prepared operation key=NYET3212USVE tag=LPXHFBYJD4HE type=del size=3146592 2024-04-07T16:38:01.798 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.804+0000 7f346a3868c0 1 < canceled operation key=C1BTVJNHIM0X tag=F2QSUBTBBVS0 type=del size=1094145 2024-04-07T16:38:01.804 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.812+0000 7f346a3868c0 1 > prepared operation key=E84U0SX44NQ8 tag=0KGKSW727IJ1 type=del size=1671492 2024-04-07T16:38:01.810 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.816+0000 7f346a3868c0 1 < completed part key=_multipart_6XDS0A7L0IY8.2~GBKKXK1ADN2Y.0 size=516806 2024-04-07T16:38:01.819 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.828+0000 7f346a3868c0 1 < completed multipart upload key=6XDS0A7L0IY8 upload=GBKKXK1ADN2Y tag=KTCRC64EGCUE type=write size=1565382 2024-04-07T16:38:01.825 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.832+0000 7f346a3868c0 1 > prepared operation key=U3EID5EK64HD tag=81ZCGQNBGO3S type=del size=2840275 2024-04-07T16:38:01.831 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.840+0000 7f346a3868c0 1 < canceled operation key=HBLEXONKXJTL tag=282A0KNX9LKA type=del size=172966 2024-04-07T16:38:01.842 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.848+0000 7f346a3868c0 1 > prepared part key=_multipart_N14BUXLJONHZ.2~IOLX57NNIGT3.0 size=1048576 2024-04-07T16:38:01.849 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.856+0000 7f346a3868c0 1 > prepared part key=_multipart_N14BUXLJONHZ.2~IOLX57NNIGT3.1 size=1048576 2024-04-07T16:38:01.852 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.860+0000 7f346a3868c0 1 > prepared part key=_multipart_N14BUXLJONHZ.2~IOLX57NNIGT3.2 size=231936 2024-04-07T16:38:01.852 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.860+0000 7f346a3868c0 1 > prepared multipart upload key=N14BUXLJONHZ upload=IOLX57NNIGT3 tag=ILHW3ITTSBP6 type=write size=3377664 2024-04-07T16:38:01.864 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.872+0000 7f346a3868c0 1 < suggested remove operation key=EOYFVEI50SQ1 tag=J2YUOMPCPVXU type=del 2024-04-07T16:38:01.870 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.876+0000 7f346a3868c0 1 > prepared operation key=YOK8BAEOMPUR tag=QH8FSOVZERQ9 type=del size=2073740 2024-04-07T16:38:01.876 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.884+0000 7f346a3868c0 1 < completed part key=_multipart_X5Q46GO8YTOK.2~OHUYW3F12L1V.0 size=1048576 2024-04-07T16:38:01.876 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.884+0000 7f346a3868c0 1 < canceled part key=_multipart_X5Q46GO8YTOK.2~OHUYW3F12L1V.1 size=1048576 2024-04-07T16:38:01.881 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.888+0000 7f346a3868c0 1 < completed part key=_multipart_X5Q46GO8YTOK.2~OHUYW3F12L1V.2 size=838710 2024-04-07T16:38:01.891 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.900+0000 7f346a3868c0 1 < canceled multipart upload key=X5Q46GO8YTOK upload=OHUYW3F12L1V tag=782XWBZOZ11R type=write size=3984438 2024-04-07T16:38:01.905 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.912+0000 7f346a3868c0 1 > prepared part key=_multipart_YXMDZB23YTB3.2~CL5DOS3LOLHA.0 size=1048576 2024-04-07T16:38:01.910 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.916+0000 7f346a3868c0 1 > prepared part key=_multipart_YXMDZB23YTB3.2~CL5DOS3LOLHA.1 size=1048576 2024-04-07T16:38:01.915 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.924+0000 7f346a3868c0 1 > prepared part key=_multipart_YXMDZB23YTB3.2~CL5DOS3LOLHA.2 size=607674 2024-04-07T16:38:01.915 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.924+0000 7f346a3868c0 1 > prepared multipart upload key=YXMDZB23YTB3 upload=CL5DOS3LOLHA tag=APX3XTTKVP3Y type=write size=3753402 2024-04-07T16:38:01.921 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.928+0000 7f346a3868c0 1 < completed delete operation key=IP8YW2A75TCW tag=KNOBADQG1RR8 type=del 2024-04-07T16:38:01.927 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.936+0000 7f346a3868c0 1 > prepared operation key=U3EID5EK64HD tag=VIA50B6DWBBQ type=del size=2256685 2024-04-07T16:38:01.939 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.948+0000 7f346a3868c0 1 < suggested update operation key=95YVKJ14JD8W tag=W7T7Y9H8FG9E type=del size=3431780 2024-04-07T16:38:01.945 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.952+0000 7f346a3868c0 1 > prepared operation key=1VKVQXSQMTU1 tag=YPV34XPQDQJG type=write size=565986 2024-04-07T16:38:01.951 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.956+0000 7f346a3868c0 1 < completed delete operation key=1WGUPH7X5UA0 tag=R1525ZW20MD7 type=del 2024-04-07T16:38:01.956 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.964+0000 7f346a3868c0 1 > prepared operation key=YXMDZB23YTB3 tag=3H5NA0W28MCN type=del size=1360331 2024-04-07T16:38:01.962 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.968+0000 7f346a3868c0 1 < completed delete operation key=1VKVQXSQMTU1 tag=GF2S4QPRAJ96 type=del 2024-04-07T16:38:01.969 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.976+0000 7f346a3868c0 1 > prepared operation key=QTCIIPTXIRW2 tag=QJR1GB25K4VA type=del size=1634288 2024-04-07T16:38:01.975 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.984+0000 7f346a3868c0 1 < completed delete operation key=1VBSUWNI3V1T tag=1V5BXP4GIWKP type=del 2024-04-07T16:38:01.987 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:01.992+0000 7f346a3868c0 1 > prepared part key=_multipart_CL4PP1RKZU0E.2~JBR903L7ME3K.0 size=1048576 2024-04-07T16:38:01.992 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.000+0000 7f346a3868c0 1 > prepared part key=_multipart_CL4PP1RKZU0E.2~JBR903L7ME3K.1 size=1048576 2024-04-07T16:38:01.997 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.004+0000 7f346a3868c0 1 > prepared part key=_multipart_CL4PP1RKZU0E.2~JBR903L7ME3K.2 size=117203 2024-04-07T16:38:01.997 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.004+0000 7f346a3868c0 1 > prepared multipart upload key=CL4PP1RKZU0E upload=JBR903L7ME3K tag=P8UYV01U9PH7 type=write size=3262931 2024-04-07T16:38:02.003 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.012+0000 7f346a3868c0 1 < completed delete operation key=QVOKSQAGHPA9 tag=TP5FBPDRMF3S type=del 2024-04-07T16:38:02.015 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.024+0000 7f346a3868c0 1 > prepared part key=_multipart_GBQSME7SY1AH.2~KDZ0PT5AUPR7.0 size=881504 2024-04-07T16:38:02.015 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.024+0000 7f346a3868c0 1 > prepared multipart upload key=GBQSME7SY1AH upload=KDZ0PT5AUPR7 tag=7S7YDHGQA8LP type=write size=1930080 2024-04-07T16:38:02.021 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.028+0000 7f346a3868c0 1 < completed delete operation key=X5Q46GO8YTOK tag=LLMUAVCRKTWY type=del 2024-04-07T16:38:02.033 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.040+0000 7f346a3868c0 1 > prepared part key=_multipart_7G5C6HBK6QWI.2~28BVQQZMF97Q.0 size=51662 2024-04-07T16:38:02.033 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.040+0000 7f346a3868c0 1 > prepared multipart upload key=7G5C6HBK6QWI upload=28BVQQZMF97Q tag=B3HADX3ONFL8 type=write size=1100238 2024-04-07T16:38:02.040 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.048+0000 7f346a3868c0 1 < completed part key=_multipart_7G5C6HBK6QWI.2~JEGFNBU4KINW.0 size=450365 2024-04-07T16:38:02.050 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.056+0000 7f346a3868c0 1 < completed multipart upload key=7G5C6HBK6QWI upload=JEGFNBU4KINW tag=AVO2VMAH7KJ0 type=write size=1498941 2024-04-07T16:38:02.061 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.068+0000 7f346a3868c0 1 > prepared part key=_multipart_IX3F0IUOKBX1.2~VXRHWB2R3Y4Y.0 size=1048576 2024-04-07T16:38:02.066 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.072+0000 7f346a3868c0 1 > prepared part key=_multipart_IX3F0IUOKBX1.2~VXRHWB2R3Y4Y.1 size=1048576 2024-04-07T16:38:02.071 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.080+0000 7f346a3868c0 1 > prepared part key=_multipart_IX3F0IUOKBX1.2~VXRHWB2R3Y4Y.2 size=558478 2024-04-07T16:38:02.072 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.080+0000 7f346a3868c0 1 > prepared multipart upload key=IX3F0IUOKBX1 upload=VXRHWB2R3Y4Y tag=IU0KX8GOU9YT type=write size=3704206 2024-04-07T16:38:02.079 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.084+0000 7f346a3868c0 1 < completed part key=_multipart_1WGUPH7X5UA0.2~3CTVMURHPXM9.0 size=1048576 2024-04-07T16:38:02.083 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.092+0000 7f346a3868c0 1 < completed part key=_multipart_1WGUPH7X5UA0.2~3CTVMURHPXM9.1 size=833699 2024-04-07T16:38:02.093 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.100+0000 7f346a3868c0 1 < completed multipart upload key=1WGUPH7X5UA0 upload=3CTVMURHPXM9 tag=OU2V1SU4ZV5F type=write size=2930851 2024-04-07T16:38:02.100 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.108+0000 7f346a3868c0 1 > prepared operation key=HBLEXONKXJTL tag=UNUW45ZEQBVJ type=del size=1501129 2024-04-07T16:38:02.105 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.112+0000 7f346a3868c0 1 < completed delete operation key=IP8YW2A75TCW tag=68T4CU32VHXP type=del 2024-04-07T16:38:02.111 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.120+0000 7f346a3868c0 1 > prepared operation key=QTCIIPTXIRW2 tag=FZC6HHVGT8I1 type=del size=3867899 2024-04-07T16:38:02.117 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.124+0000 7f346a3868c0 1 < completed delete operation key=NYET3212USVE tag=LPXHFBYJD4HE type=del 2024-04-07T16:38:02.128 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.136+0000 7f346a3868c0 1 > prepared operation key=LXOZ0T2KY6OR tag=8NE8BL84KBAY type=del size=4155617 2024-04-07T16:38:02.135 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.140+0000 7f346a3868c0 1 < completed delete operation key=E84U0SX44NQ8 tag=0KGKSW727IJ1 type=del 2024-04-07T16:38:02.150 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.156+0000 7f346a3868c0 1 > prepared part key=_multipart_ROFH30W3UCAX.2~YA1HECLBJ331.0 size=1048576 2024-04-07T16:38:02.155 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.164+0000 7f346a3868c0 1 > prepared part key=_multipart_ROFH30W3UCAX.2~YA1HECLBJ331.1 size=964733 2024-04-07T16:38:02.156 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.164+0000 7f346a3868c0 1 > prepared multipart upload key=ROFH30W3UCAX upload=YA1HECLBJ331 tag=1T3XPWAKCVOP type=write size=3061885 2024-04-07T16:38:02.163 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.168+0000 7f346a3868c0 1 < completed delete operation key=U3EID5EK64HD tag=81ZCGQNBGO3S type=del 2024-04-07T16:38:02.169 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.176+0000 7f346a3868c0 1 > prepared operation key=IX3F0IUOKBX1 tag=Z2EAW9F4I6GD type=del size=1482898 2024-04-07T16:38:02.175 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.184+0000 7f346a3868c0 1 < completed part key=_multipart_N14BUXLJONHZ.2~IOLX57NNIGT3.0 size=1048576 2024-04-07T16:38:02.180 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.188+0000 7f346a3868c0 1 < completed part key=_multipart_N14BUXLJONHZ.2~IOLX57NNIGT3.1 size=1048576 2024-04-07T16:38:02.185 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.192+0000 7f346a3868c0 1 < completed part key=_multipart_N14BUXLJONHZ.2~IOLX57NNIGT3.2 size=231936 2024-04-07T16:38:02.194 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.200+0000 7f346a3868c0 1 < completed multipart upload key=N14BUXLJONHZ upload=IOLX57NNIGT3 tag=ILHW3ITTSBP6 type=write size=3377664 2024-04-07T16:38:02.205 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.212+0000 7f346a3868c0 1 > prepared part key=_multipart_95YVKJ14JD8W.2~1YMOMO3C9K11.0 size=1023246 2024-04-07T16:38:02.205 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.212+0000 7f346a3868c0 1 > prepared multipart upload key=95YVKJ14JD8W upload=1YMOMO3C9K11 tag=164X2EBZ95NV type=write size=2071822 2024-04-07T16:38:02.211 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.216+0000 7f346a3868c0 1 < completed delete operation key=YOK8BAEOMPUR tag=QH8FSOVZERQ9 type=del 2024-04-07T16:38:02.216 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.224+0000 7f346a3868c0 1 > prepared operation key=8F81G0P0YBFW tag=66M7PFJRLN6T type=del size=1933482 2024-04-07T16:38:02.222 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.228+0000 7f346a3868c0 1 < completed part key=_multipart_YXMDZB23YTB3.2~CL5DOS3LOLHA.0 size=1048576 2024-04-07T16:38:02.228 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.236+0000 7f346a3868c0 1 < completed part key=_multipart_YXMDZB23YTB3.2~CL5DOS3LOLHA.1 size=1048576 2024-04-07T16:38:02.232 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.240+0000 7f346a3868c0 1 < completed part key=_multipart_YXMDZB23YTB3.2~CL5DOS3LOLHA.2 size=607674 2024-04-07T16:38:02.242 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.248+0000 7f346a3868c0 1 < completed multipart upload key=YXMDZB23YTB3 upload=CL5DOS3LOLHA tag=APX3XTTKVP3Y type=write size=3753402 2024-04-07T16:38:02.248 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.256+0000 7f346a3868c0 1 > prepared operation key=EOYFVEI50SQ1 tag=YOW9DVPOYI3F type=del size=2474398 2024-04-07T16:38:02.258 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.260+0000 7f346a3868c0 1 < completed delete operation key=U3EID5EK64HD tag=VIA50B6DWBBQ type=del 2024-04-07T16:38:02.261 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.268+0000 7f346a3868c0 1 > prepared operation key=NYET3212USVE tag=1IGH4693LNNM type=del size=1513130 2024-04-07T16:38:02.269 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.276+0000 7f346a3868c0 1 < completed write operation key=1VKVQXSQMTU1 tag=YPV34XPQDQJG type=write size=565986 2024-04-07T16:38:02.280 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.288+0000 7f346a3868c0 1 > prepared part key=_multipart_QTCIIPTXIRW2.2~9JL740EKSCZO.0 size=1048576 2024-04-07T16:38:02.284 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.292+0000 7f346a3868c0 1 > prepared part key=_multipart_QTCIIPTXIRW2.2~9JL740EKSCZO.1 size=1048576 2024-04-07T16:38:02.289 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.296+0000 7f346a3868c0 1 > prepared part key=_multipart_QTCIIPTXIRW2.2~9JL740EKSCZO.2 size=446856 2024-04-07T16:38:02.290 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.296+0000 7f346a3868c0 1 > prepared multipart upload key=QTCIIPTXIRW2 upload=9JL740EKSCZO tag=47LKRSF1Y6SR type=write size=3592584 2024-04-07T16:38:02.296 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.304+0000 7f346a3868c0 1 < completed delete operation key=YXMDZB23YTB3 tag=3H5NA0W28MCN type=del 2024-04-07T16:38:02.304 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.312+0000 7f346a3868c0 1 > prepared operation key=95YVKJ14JD8W tag=KSPHR1CQLYXN type=del size=647370 2024-04-07T16:38:02.311 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.316+0000 7f346a3868c0 1 < canceled operation key=QTCIIPTXIRW2 tag=QJR1GB25K4VA type=del size=1634288 2024-04-07T16:38:02.316 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.324+0000 7f346a3868c0 1 > prepared operation key=1VKVQXSQMTU1 tag=QMMTMBTE8BKF type=del size=2994999 2024-04-07T16:38:02.323 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.328+0000 7f346a3868c0 1 < completed part key=_multipart_CL4PP1RKZU0E.2~JBR903L7ME3K.0 size=1048576 2024-04-07T16:38:02.327 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.336+0000 7f346a3868c0 1 < completed part key=_multipart_CL4PP1RKZU0E.2~JBR903L7ME3K.1 size=1048576 2024-04-07T16:38:02.332 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.340+0000 7f346a3868c0 1 < completed part key=_multipart_CL4PP1RKZU0E.2~JBR903L7ME3K.2 size=117203 2024-04-07T16:38:02.341 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.348+0000 7f346a3868c0 1 < completed multipart upload key=CL4PP1RKZU0E upload=JBR903L7ME3K tag=P8UYV01U9PH7 type=write size=3262931 2024-04-07T16:38:02.351 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.360+0000 7f346a3868c0 1 > prepared part key=_multipart_CL4PP1RKZU0E.2~Z3QFXF6XW1JG.0 size=1048576 2024-04-07T16:38:02.356 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.364+0000 7f346a3868c0 1 > prepared part key=_multipart_CL4PP1RKZU0E.2~Z3QFXF6XW1JG.1 size=1048576 2024-04-07T16:38:02.361 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.368+0000 7f346a3868c0 1 > prepared part key=_multipart_CL4PP1RKZU0E.2~Z3QFXF6XW1JG.2 size=577672 2024-04-07T16:38:02.361 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.368+0000 7f346a3868c0 1 > prepared multipart upload key=CL4PP1RKZU0E upload=Z3QFXF6XW1JG tag=QJ4O1J2ML6BN type=write size=3723400 2024-04-07T16:38:02.367 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.372+0000 7f346a3868c0 1 < completed part key=_multipart_GBQSME7SY1AH.2~KDZ0PT5AUPR7.0 size=881504 2024-04-07T16:38:02.377 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.384+0000 7f346a3868c0 1 < completed multipart upload key=GBQSME7SY1AH upload=KDZ0PT5AUPR7 tag=7S7YDHGQA8LP type=write size=1930080 2024-04-07T16:38:02.388 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.396+0000 7f346a3868c0 1 > prepared part key=_multipart_3N0QJRYVO3FV.2~VQTJSLK3EPJB.0 size=1020395 2024-04-07T16:38:02.388 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.396+0000 7f346a3868c0 1 > prepared multipart upload key=3N0QJRYVO3FV upload=VQTJSLK3EPJB tag=AYMGU5RFSBAE type=write size=2068971 2024-04-07T16:38:02.394 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.400+0000 7f346a3868c0 1 < completed part key=_multipart_7G5C6HBK6QWI.2~28BVQQZMF97Q.0 size=51662 2024-04-07T16:38:02.403 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.412+0000 7f346a3868c0 1 < completed multipart upload key=7G5C6HBK6QWI upload=28BVQQZMF97Q tag=B3HADX3ONFL8 type=write size=1100238 2024-04-07T16:38:02.409 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.416+0000 7f346a3868c0 1 > prepared operation key=IP8YW2A75TCW tag=YY08RWWGC2EZ type=del size=1135319 2024-04-07T16:38:02.414 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.420+0000 7f346a3868c0 1 < completed part key=_multipart_IX3F0IUOKBX1.2~VXRHWB2R3Y4Y.0 size=1048576 2024-04-07T16:38:02.419 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.424+0000 7f346a3868c0 1 < completed part key=_multipart_IX3F0IUOKBX1.2~VXRHWB2R3Y4Y.1 size=1048576 2024-04-07T16:38:02.419 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.424+0000 7f346a3868c0 1 < canceled part key=_multipart_IX3F0IUOKBX1.2~VXRHWB2R3Y4Y.2 size=558478 2024-04-07T16:38:02.429 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.436+0000 7f346a3868c0 1 < canceled multipart upload key=IX3F0IUOKBX1 upload=VXRHWB2R3Y4Y tag=IU0KX8GOU9YT type=write size=3704206 2024-04-07T16:38:02.440 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.448+0000 7f346a3868c0 1 > prepared part key=_multipart_95YVKJ14JD8W.2~0778RAFO5Y5O.0 size=798430 2024-04-07T16:38:02.440 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.448+0000 7f346a3868c0 1 > prepared multipart upload key=95YVKJ14JD8W upload=0778RAFO5Y5O tag=XBWMRWRFRREC type=write size=1847006 2024-04-07T16:38:02.441 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:38:02 smithi100 bash[74942]: cluster 2024-04-07T16:38:01.064956+0000 mgr.y (mgr.25175) 680 : cluster [DBG] pgmap v1315: 137 pgs: 137 active+clean; 583 KiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 429 KiB/s rd, 148 KiB/s wr, 759 op/s 2024-04-07T16:38:02.441 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:38:02 smithi100 bash[74942]: audit 2024-04-07T16:38:01.753435+0000 mon.a (mon.0) 2022 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:38:02.441 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:38:02 smithi100 bash[76202]: cluster 2024-04-07T16:38:01.064956+0000 mgr.y (mgr.25175) 680 : cluster [DBG] pgmap v1315: 137 pgs: 137 active+clean; 583 KiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 429 KiB/s rd, 148 KiB/s wr, 759 op/s 2024-04-07T16:38:02.441 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:38:02 smithi100 bash[76202]: audit 2024-04-07T16:38:01.753435+0000 mon.a (mon.0) 2022 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:38:02.447 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.452+0000 7f346a3868c0 1 < completed delete operation key=HBLEXONKXJTL tag=UNUW45ZEQBVJ type=del 2024-04-07T16:38:02.452 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.460+0000 7f346a3868c0 1 > prepared operation key=EQR0FY86AT5X tag=DI80WCLITGQL type=del size=1710205 2024-04-07T16:38:02.458 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.464+0000 7f346a3868c0 1 < completed delete operation key=QTCIIPTXIRW2 tag=FZC6HHVGT8I1 type=del 2024-04-07T16:38:02.463 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.472+0000 7f346a3868c0 1 > prepared operation key=NYET3212USVE tag=78A9AEX3DTCP type=del size=3543281 2024-04-07T16:38:02.470 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.476+0000 7f346a3868c0 1 < completed delete operation key=LXOZ0T2KY6OR tag=8NE8BL84KBAY type=del 2024-04-07T16:38:02.476 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.484+0000 7f346a3868c0 1 > prepared operation key=QVOKSQAGHPA9 tag=9IOYPIUV47DD type=del size=35337 2024-04-07T16:38:02.483 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.492+0000 7f346a3868c0 1 < completed part key=_multipart_ROFH30W3UCAX.2~YA1HECLBJ331.0 size=1048576 2024-04-07T16:38:02.488 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.496+0000 7f346a3868c0 1 < completed part key=_multipart_ROFH30W3UCAX.2~YA1HECLBJ331.1 size=964733 2024-04-07T16:38:02.499 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.508+0000 7f346a3868c0 1 < completed multipart upload key=ROFH30W3UCAX upload=YA1HECLBJ331 tag=1T3XPWAKCVOP type=write size=3061885 2024-04-07T16:38:02.506 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.512+0000 7f346a3868c0 1 > prepared operation key=XJLQZ7IS9U1T tag=NAKIJMA3OZDH type=del size=4043632 2024-04-07T16:38:02.512 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.520+0000 7f346a3868c0 1 < completed delete operation key=IX3F0IUOKBX1 tag=Z2EAW9F4I6GD type=del 2024-04-07T16:38:02.518 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.524+0000 7f346a3868c0 1 > prepared operation key=X5Q46GO8YTOK tag=CXLWR24NYV3E type=del size=3002207 2024-04-07T16:38:02.523 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.532+0000 7f346a3868c0 1 < completed part key=_multipart_95YVKJ14JD8W.2~1YMOMO3C9K11.0 size=1023246 2024-04-07T16:38:02.533 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.540+0000 7f346a3868c0 1 < completed multipart upload key=95YVKJ14JD8W upload=1YMOMO3C9K11 tag=164X2EBZ95NV type=write size=2071822 2024-04-07T16:38:02.543 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.552+0000 7f346a3868c0 1 > prepared part key=_multipart_U3EID5EK64HD.2~VPRLA4UT9CT0.0 size=1048576 2024-04-07T16:38:02.548 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.556+0000 7f346a3868c0 1 > prepared part key=_multipart_U3EID5EK64HD.2~VPRLA4UT9CT0.1 size=1048576 2024-04-07T16:38:02.553 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.560+0000 7f346a3868c0 1 > prepared part key=_multipart_U3EID5EK64HD.2~VPRLA4UT9CT0.2 size=671479 2024-04-07T16:38:02.553 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.560+0000 7f346a3868c0 1 > prepared multipart upload key=U3EID5EK64HD upload=VPRLA4UT9CT0 tag=XHH5BSAW5MWZ type=write size=3817207 2024-04-07T16:38:02.558 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.564+0000 7f346a3868c0 1 < canceled operation key=8F81G0P0YBFW tag=66M7PFJRLN6T type=del size=1933482 2024-04-07T16:38:02.563 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.572+0000 7f346a3868c0 1 > prepared operation key=1WGUPH7X5UA0 tag=4C4Z1Z2TG646 type=write size=802902 2024-04-07T16:38:02.569 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.576+0000 7f346a3868c0 1 < canceled operation key=EOYFVEI50SQ1 tag=YOW9DVPOYI3F type=del size=2474398 2024-04-07T16:38:02.575 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.584+0000 7f346a3868c0 1 > prepared operation key=YOK8BAEOMPUR tag=OVVC00V34F4V type=del size=61591 2024-04-07T16:38:02.581 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.588+0000 7f346a3868c0 1 < completed delete operation key=NYET3212USVE tag=1IGH4693LNNM type=del 2024-04-07T16:38:02.587 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.596+0000 7f346a3868c0 1 > prepared operation key=C1BTVJNHIM0X tag=XYNYY4537QQR type=del size=2120297 2024-04-07T16:38:02.593 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.600+0000 7f346a3868c0 1 < completed part key=_multipart_QTCIIPTXIRW2.2~9JL740EKSCZO.0 size=1048576 2024-04-07T16:38:02.598 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.604+0000 7f346a3868c0 1 < completed part key=_multipart_QTCIIPTXIRW2.2~9JL740EKSCZO.1 size=1048576 2024-04-07T16:38:02.603 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.612+0000 7f346a3868c0 1 < completed part key=_multipart_QTCIIPTXIRW2.2~9JL740EKSCZO.2 size=446856 2024-04-07T16:38:02.606 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:38:02 smithi144 bash[95233]: ts=2024-04-07T16:38:02.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:38:02.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:38:02 smithi144 bash[66654]: cluster 2024-04-07T16:38:01.064956+0000 mgr.y (mgr.25175) 680 : cluster [DBG] pgmap v1315: 137 pgs: 137 active+clean; 583 KiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 429 KiB/s rd, 148 KiB/s wr, 759 op/s 2024-04-07T16:38:02.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:38:02 smithi144 bash[66654]: audit 2024-04-07T16:38:01.753435+0000 mon.a (mon.0) 2022 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:38:02.614 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.620+0000 7f346a3868c0 1 < completed multipart upload key=QTCIIPTXIRW2 upload=9JL740EKSCZO tag=47LKRSF1Y6SR type=write size=3592584 2024-04-07T16:38:02.620 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.628+0000 7f346a3868c0 1 > prepared operation key=NYET3212USVE tag=IWHHRUNRDNMI type=del size=2672552 2024-04-07T16:38:02.632 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.640+0000 7f346a3868c0 1 < suggested remove operation key=95YVKJ14JD8W tag=KSPHR1CQLYXN type=del 2024-04-07T16:38:02.643 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.652+0000 7f346a3868c0 1 > prepared part key=_multipart_95YVKJ14JD8W.2~BAK5KH9FRSR1.0 size=1048576 2024-04-07T16:38:02.648 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.656+0000 7f346a3868c0 1 > prepared part key=_multipart_95YVKJ14JD8W.2~BAK5KH9FRSR1.1 size=1048576 2024-04-07T16:38:02.653 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.660+0000 7f346a3868c0 1 > prepared part key=_multipart_95YVKJ14JD8W.2~BAK5KH9FRSR1.2 size=336407 2024-04-07T16:38:02.653 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.660+0000 7f346a3868c0 1 > prepared multipart upload key=95YVKJ14JD8W upload=BAK5KH9FRSR1 tag=G87FSMUEUK82 type=write size=3482135 2024-04-07T16:38:02.659 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.668+0000 7f346a3868c0 1 < canceled operation key=1VKVQXSQMTU1 tag=QMMTMBTE8BKF type=del size=2994999 2024-04-07T16:38:02.665 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.672+0000 7f346a3868c0 1 > prepared operation key=IP8YW2A75TCW tag=JIULHMPQIYW7 type=del size=2285409 2024-04-07T16:38:02.671 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.676+0000 7f346a3868c0 1 < completed part key=_multipart_CL4PP1RKZU0E.2~Z3QFXF6XW1JG.0 size=1048576 2024-04-07T16:38:02.675 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.684+0000 7f346a3868c0 1 < completed part key=_multipart_CL4PP1RKZU0E.2~Z3QFXF6XW1JG.1 size=1048576 2024-04-07T16:38:02.683 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.688+0000 7f346a3868c0 1 < completed part key=_multipart_CL4PP1RKZU0E.2~Z3QFXF6XW1JG.2 size=577672 2024-04-07T16:38:02.693 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.700+0000 7f346a3868c0 1 < completed multipart upload key=CL4PP1RKZU0E upload=Z3QFXF6XW1JG tag=QJ4O1J2ML6BN type=write size=3723400 2024-04-07T16:38:02.703 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.712+0000 7f346a3868c0 1 > prepared part key=_multipart_IP8YW2A75TCW.2~D6Q337KUML0I.0 size=1048576 2024-04-07T16:38:02.708 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.716+0000 7f346a3868c0 1 > prepared part key=_multipart_IP8YW2A75TCW.2~D6Q337KUML0I.1 size=1048576 2024-04-07T16:38:02.713 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.720+0000 7f346a3868c0 1 > prepared part key=_multipart_IP8YW2A75TCW.2~D6Q337KUML0I.2 size=750422 2024-04-07T16:38:02.713 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.720+0000 7f346a3868c0 1 > prepared multipart upload key=IP8YW2A75TCW upload=D6Q337KUML0I tag=P8OUC0BM2M2Y type=write size=3896150 2024-04-07T16:38:02.719 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.724+0000 7f346a3868c0 1 < completed part key=_multipart_3N0QJRYVO3FV.2~VQTJSLK3EPJB.0 size=1020395 2024-04-07T16:38:02.729 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.736+0000 7f346a3868c0 1 < completed multipart upload key=3N0QJRYVO3FV upload=VQTJSLK3EPJB tag=AYMGU5RFSBAE type=write size=2068971 2024-04-07T16:38:02.735 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.740+0000 7f346a3868c0 1 > prepared operation key=HBLEXONKXJTL tag=UBP7ZLROJJOL type=del size=1270394 2024-04-07T16:38:02.741 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.748+0000 7f346a3868c0 1 < completed delete operation key=IP8YW2A75TCW tag=YY08RWWGC2EZ type=del 2024-04-07T16:38:02.752 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.760+0000 7f346a3868c0 1 > prepared part key=_multipart_X5Q46GO8YTOK.2~MYVTHXO5HIIU.0 size=1048576 2024-04-07T16:38:02.757 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.764+0000 7f346a3868c0 1 > prepared part key=_multipart_X5Q46GO8YTOK.2~MYVTHXO5HIIU.1 size=1048576 2024-04-07T16:38:02.762 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.768+0000 7f346a3868c0 1 > prepared part key=_multipart_X5Q46GO8YTOK.2~MYVTHXO5HIIU.2 size=809133 2024-04-07T16:38:02.762 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.768+0000 7f346a3868c0 1 > prepared multipart upload key=X5Q46GO8YTOK upload=MYVTHXO5HIIU tag=9WT7EU1V7R87 type=write size=3954861 2024-04-07T16:38:02.768 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.776+0000 7f346a3868c0 1 < completed part key=_multipart_95YVKJ14JD8W.2~0778RAFO5Y5O.0 size=798430 2024-04-07T16:38:02.778 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.784+0000 7f346a3868c0 1 < completed multipart upload key=95YVKJ14JD8W upload=0778RAFO5Y5O tag=XBWMRWRFRREC type=write size=1847006 2024-04-07T16:38:02.790 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.796+0000 7f346a3868c0 1 > prepared part key=_multipart_IX3F0IUOKBX1.2~FRWFADBV14TA.0 size=1048576 2024-04-07T16:38:02.795 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.800+0000 7f346a3868c0 1 > prepared part key=_multipart_IX3F0IUOKBX1.2~FRWFADBV14TA.1 size=1048576 2024-04-07T16:38:02.800 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.808+0000 7f346a3868c0 1 > prepared part key=_multipart_IX3F0IUOKBX1.2~FRWFADBV14TA.2 size=271580 2024-04-07T16:38:02.800 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.808+0000 7f346a3868c0 1 > prepared multipart upload key=IX3F0IUOKBX1 upload=FRWFADBV14TA tag=JD6FS49Z2QOY type=write size=3417308 2024-04-07T16:38:02.806 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.812+0000 7f346a3868c0 1 < completed delete operation key=EQR0FY86AT5X tag=DI80WCLITGQL type=del 2024-04-07T16:38:02.816 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.824+0000 7f346a3868c0 1 > prepared part key=_multipart_7G5C6HBK6QWI.2~PVHWV8PCMS9V.0 size=1048576 2024-04-07T16:38:02.821 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.828+0000 7f346a3868c0 1 > prepared part key=_multipart_7G5C6HBK6QWI.2~PVHWV8PCMS9V.1 size=685226 2024-04-07T16:38:02.821 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.828+0000 7f346a3868c0 1 > prepared multipart upload key=7G5C6HBK6QWI upload=PVHWV8PCMS9V tag=QJYIY4XSXWD0 type=write size=2782378 2024-04-07T16:38:02.826 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.832+0000 7f346a3868c0 1 < completed delete operation key=NYET3212USVE tag=78A9AEX3DTCP type=del 2024-04-07T16:38:02.837 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.844+0000 7f346a3868c0 1 > prepared part key=_multipart_XJLQZ7IS9U1T.2~CQIS2RDNZTNK.0 size=1048576 2024-04-07T16:38:02.842 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.848+0000 7f346a3868c0 1 > prepared part key=_multipart_XJLQZ7IS9U1T.2~CQIS2RDNZTNK.1 size=689832 2024-04-07T16:38:02.842 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.848+0000 7f346a3868c0 1 > prepared multipart upload key=XJLQZ7IS9U1T upload=CQIS2RDNZTNK tag=YD4ANFDDV3RS type=write size=2786984 2024-04-07T16:38:02.847 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.856+0000 7f346a3868c0 1 < completed delete operation key=QVOKSQAGHPA9 tag=9IOYPIUV47DD type=del 2024-04-07T16:38:02.862 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.868+0000 7f346a3868c0 1 > prepared part key=_multipart_6XDS0A7L0IY8.2~BUJ38YY76ZUP.0 size=1048576 2024-04-07T16:38:02.869 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.876+0000 7f346a3868c0 1 > prepared part key=_multipart_6XDS0A7L0IY8.2~BUJ38YY76ZUP.1 size=829333 2024-04-07T16:38:02.870 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.876+0000 7f346a3868c0 1 > prepared multipart upload key=6XDS0A7L0IY8 upload=BUJ38YY76ZUP tag=BE01DPMNWME4 type=write size=2926485 2024-04-07T16:38:02.873 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.880+0000 7f346a3868c0 1 < completed delete operation key=XJLQZ7IS9U1T tag=NAKIJMA3OZDH type=del 2024-04-07T16:38:02.880 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.888+0000 7f346a3868c0 1 > prepared operation key=UIW3DN4NUYPP tag=OB5KTF7MN5MK type=del size=1413126 2024-04-07T16:38:02.886 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.892+0000 7f346a3868c0 1 < completed delete operation key=X5Q46GO8YTOK tag=CXLWR24NYV3E type=del 2024-04-07T16:38:02.897 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.904+0000 7f346a3868c0 1 > prepared part key=_multipart_1VBSUWNI3V1T.2~WIMEY23UQJD8.0 size=1048576 2024-04-07T16:38:02.902 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.908+0000 7f346a3868c0 1 > prepared part key=_multipart_1VBSUWNI3V1T.2~WIMEY23UQJD8.1 size=721573 2024-04-07T16:38:02.902 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.908+0000 7f346a3868c0 1 > prepared multipart upload key=1VBSUWNI3V1T upload=WIMEY23UQJD8 tag=T8TE70MI5L2Z type=write size=2818725 2024-04-07T16:38:02.907 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.916+0000 7f346a3868c0 1 < completed part key=_multipart_U3EID5EK64HD.2~VPRLA4UT9CT0.0 size=1048576 2024-04-07T16:38:02.912 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.920+0000 7f346a3868c0 1 < completed part key=_multipart_U3EID5EK64HD.2~VPRLA4UT9CT0.1 size=1048576 2024-04-07T16:38:02.917 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.924+0000 7f346a3868c0 1 < completed part key=_multipart_U3EID5EK64HD.2~VPRLA4UT9CT0.2 size=671479 2024-04-07T16:38:02.928 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.936+0000 7f346a3868c0 1 < completed multipart upload key=U3EID5EK64HD upload=VPRLA4UT9CT0 tag=XHH5BSAW5MWZ type=write size=3817207 2024-04-07T16:38:02.933 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.940+0000 7f346a3868c0 1 > prepared operation key=1WGUPH7X5UA0 tag=JH6R07L5F20E type=del size=1512730 2024-04-07T16:38:02.939 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.948+0000 7f346a3868c0 1 < completed write operation key=1WGUPH7X5UA0 tag=4C4Z1Z2TG646 type=write size=802902 2024-04-07T16:38:02.945 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.952+0000 7f346a3868c0 1 > prepared operation key=UIW3DN4NUYPP tag=VHX97MIMS7JV type=del size=1017953 2024-04-07T16:38:02.951 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.960+0000 7f346a3868c0 1 < canceled operation key=YOK8BAEOMPUR tag=OVVC00V34F4V type=del size=61591 2024-04-07T16:38:02.957 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.964+0000 7f346a3868c0 1 > prepared operation key=ROFH30W3UCAX tag=AIJPDVE5RB75 type=del size=3190942 2024-04-07T16:38:02.963 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.968+0000 7f346a3868c0 1 < completed delete operation key=C1BTVJNHIM0X tag=XYNYY4537QQR type=del 2024-04-07T16:38:02.969 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.976+0000 7f346a3868c0 1 > prepared operation key=YOK8BAEOMPUR tag=181TOGX6WO8R type=del size=2269347 2024-04-07T16:38:02.974 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.980+0000 7f346a3868c0 1 < completed delete operation key=NYET3212USVE tag=IWHHRUNRDNMI type=del 2024-04-07T16:38:02.980 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.988+0000 7f346a3868c0 1 > prepared operation key=7G5C6HBK6QWI tag=6WV8Y8E8MW66 type=del size=3885353 2024-04-07T16:38:02.986 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.992+0000 7f346a3868c0 1 < completed part key=_multipart_95YVKJ14JD8W.2~BAK5KH9FRSR1.0 size=1048576 2024-04-07T16:38:02.990 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:02.996+0000 7f346a3868c0 1 < completed part key=_multipart_95YVKJ14JD8W.2~BAK5KH9FRSR1.1 size=1048576 2024-04-07T16:38:02.997 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:03.004+0000 7f346a3868c0 1 < completed part key=_multipart_95YVKJ14JD8W.2~BAK5KH9FRSR1.2 size=336407 2024-04-07T16:38:03.009 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:03.016+0000 7f346a3868c0 1 < completed multipart upload key=95YVKJ14JD8W upload=BAK5KH9FRSR1 tag=G87FSMUEUK82 type=write size=3482135 2024-04-07T16:38:03.015 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:03.024+0000 7f346a3868c0 1 > prepared operation key=QTCIIPTXIRW2 tag=HB8IBUX65MIY type=write size=599046 2024-04-07T16:38:03.021 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:03.028+0000 7f346a3868c0 1 < completed delete operation key=IP8YW2A75TCW tag=JIULHMPQIYW7 type=del 2024-04-07T16:38:03.032 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:03.040+0000 7f346a3868c0 1 > prepared part key=_multipart_3N0QJRYVO3FV.2~HMY6L2SL43P1.0 size=1048576 2024-04-07T16:38:03.037 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:03.044+0000 7f346a3868c0 1 > prepared part key=_multipart_3N0QJRYVO3FV.2~HMY6L2SL43P1.1 size=1048576 2024-04-07T16:38:03.044 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:03.052+0000 7f346a3868c0 1 > prepared part key=_multipart_3N0QJRYVO3FV.2~HMY6L2SL43P1.2 size=758621 2024-04-07T16:38:03.044 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:03.052+0000 7f346a3868c0 1 > prepared multipart upload key=3N0QJRYVO3FV upload=HMY6L2SL43P1 tag=T3CNXDMXK1AZ type=write size=3904349 2024-04-07T16:38:03.049 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:03.056+0000 7f346a3868c0 1 < completed part key=_multipart_IP8YW2A75TCW.2~D6Q337KUML0I.0 size=1048576 2024-04-07T16:38:03.053 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:03.060+0000 7f346a3868c0 1 < completed part key=_multipart_IP8YW2A75TCW.2~D6Q337KUML0I.1 size=1048576 2024-04-07T16:38:03.058 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:03.064+0000 7f346a3868c0 1 < completed part key=_multipart_IP8YW2A75TCW.2~D6Q337KUML0I.2 size=750422 2024-04-07T16:38:03.068 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:03.076+0000 7f346a3868c0 1 < completed multipart upload key=IP8YW2A75TCW upload=D6Q337KUML0I tag=P8OUC0BM2M2Y type=write size=3896150 2024-04-07T16:38:03.079 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:03.084+0000 7f346a3868c0 1 > prepared part key=_multipart_8F81G0P0YBFW.2~PF9CGCPKRLS1.0 size=1048576 2024-04-07T16:38:03.084 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:03.092+0000 7f346a3868c0 1 > prepared part key=_multipart_8F81G0P0YBFW.2~PF9CGCPKRLS1.1 size=1048576 2024-04-07T16:38:03.088 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:03.096+0000 7f346a3868c0 1 > prepared part key=_multipart_8F81G0P0YBFW.2~PF9CGCPKRLS1.2 size=553458 2024-04-07T16:38:03.088 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:03.096+0000 7f346a3868c0 1 > prepared multipart upload key=8F81G0P0YBFW upload=PF9CGCPKRLS1 tag=571OS35Q5T8Q type=write size=3699186 2024-04-07T16:38:03.094 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:03.100+0000 7f346a3868c0 1 < canceled operation key=HBLEXONKXJTL tag=UBP7ZLROJJOL type=del size=1270394 2024-04-07T16:38:03.099 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:03.104+0000 7f346a3868c0 1 > prepared operation key=IP8YW2A75TCW tag=YYMBMR58JJSI type=del size=293153 2024-04-07T16:38:03.104 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:03.112+0000 7f346a3868c0 1 < completed part key=_multipart_X5Q46GO8YTOK.2~MYVTHXO5HIIU.0 size=1048576 2024-04-07T16:38:03.109 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:03.116+0000 7f346a3868c0 1 < completed part key=_multipart_X5Q46GO8YTOK.2~MYVTHXO5HIIU.1 size=1048576 2024-04-07T16:38:03.114 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:03.120+0000 7f346a3868c0 1 < completed part key=_multipart_X5Q46GO8YTOK.2~MYVTHXO5HIIU.2 size=809133 2024-04-07T16:38:03.125 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:03.132+0000 7f346a3868c0 1 < completed multipart upload key=X5Q46GO8YTOK upload=MYVTHXO5HIIU tag=9WT7EU1V7R87 type=write size=3954861 2024-04-07T16:38:03.131 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:03.140+0000 7f346a3868c0 1 > prepared operation key=GBQSME7SY1AH tag=OZF9TSBKKURB type=del size=2009534 2024-04-07T16:38:03.137 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:03.144+0000 7f346a3868c0 1 < completed part key=_multipart_IX3F0IUOKBX1.2~FRWFADBV14TA.0 size=1048576 2024-04-07T16:38:03.143 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:03.148+0000 7f346a3868c0 1 < completed part key=_multipart_IX3F0IUOKBX1.2~FRWFADBV14TA.1 size=1048576 2024-04-07T16:38:03.148 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:03.156+0000 7f346a3868c0 1 < completed part key=_multipart_IX3F0IUOKBX1.2~FRWFADBV14TA.2 size=271580 2024-04-07T16:38:03.160 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:03.168+0000 7f346a3868c0 1 < completed multipart upload key=IX3F0IUOKBX1 upload=FRWFADBV14TA tag=JD6FS49Z2QOY type=write size=3417308 2024-04-07T16:38:03.165 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:03.172+0000 7f346a3868c0 1 > prepared operation key=XJLQZ7IS9U1T tag=QN0T7T237EUA type=del size=3043972 2024-04-07T16:38:03.171 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:03.180+0000 7f346a3868c0 1 < completed part key=_multipart_7G5C6HBK6QWI.2~PVHWV8PCMS9V.0 size=1048576 2024-04-07T16:38:03.176 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:03.184+0000 7f346a3868c0 1 < completed part key=_multipart_7G5C6HBK6QWI.2~PVHWV8PCMS9V.1 size=685226 2024-04-07T16:38:03.186 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:03.192+0000 7f346a3868c0 1 < completed multipart upload key=7G5C6HBK6QWI upload=PVHWV8PCMS9V tag=QJYIY4XSXWD0 type=write size=2782378 2024-04-07T16:38:03.198 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:03.204+0000 7f346a3868c0 1 > prepared part key=_multipart_95YVKJ14JD8W.2~3R1Q2L79GX6S.0 size=1048576 2024-04-07T16:38:03.203 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:03.208+0000 7f346a3868c0 1 > prepared part key=_multipart_95YVKJ14JD8W.2~3R1Q2L79GX6S.1 size=1048576 2024-04-07T16:38:03.207 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:03.216+0000 7f346a3868c0 1 > prepared part key=_multipart_95YVKJ14JD8W.2~3R1Q2L79GX6S.2 size=615885 2024-04-07T16:38:03.207 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:03.216+0000 7f346a3868c0 1 > prepared multipart upload key=95YVKJ14JD8W upload=3R1Q2L79GX6S tag=044BC1F4Y07U type=write size=3761613 2024-04-07T16:38:03.214 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:03.220+0000 7f346a3868c0 1 < completed part key=_multipart_XJLQZ7IS9U1T.2~CQIS2RDNZTNK.0 size=1048576 2024-04-07T16:38:03.219 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:03.228+0000 7f346a3868c0 1 < completed part key=_multipart_XJLQZ7IS9U1T.2~CQIS2RDNZTNK.1 size=689832 2024-04-07T16:38:03.230 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:03.236+0000 7f346a3868c0 1 < completed multipart upload key=XJLQZ7IS9U1T upload=CQIS2RDNZTNK tag=YD4ANFDDV3RS type=write size=2786984 2024-04-07T16:38:03.241 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:03.248+0000 7f346a3868c0 1 > prepared part key=_multipart_LXOZ0T2KY6OR.2~QJ01TYNPL1SP.0 size=995377 2024-04-07T16:38:03.241 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:03.248+0000 7f346a3868c0 1 > prepared multipart upload key=LXOZ0T2KY6OR upload=QJ01TYNPL1SP tag=A953RQX2Q7HX type=write size=2043953 2024-04-07T16:38:03.246 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:03.252+0000 7f346a3868c0 1 < completed part key=_multipart_6XDS0A7L0IY8.2~BUJ38YY76ZUP.0 size=1048576 2024-04-07T16:38:03.252 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:03.260+0000 7f346a3868c0 1 < completed part key=_multipart_6XDS0A7L0IY8.2~BUJ38YY76ZUP.1 size=829333 2024-04-07T16:38:03.263 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:03.268+0000 7f346a3868c0 1 < completed multipart upload key=6XDS0A7L0IY8 upload=BUJ38YY76ZUP tag=BE01DPMNWME4 type=write size=2926485 2024-04-07T16:38:03.274 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:03.280+0000 7f346a3868c0 1 > prepared part key=_multipart_UIW3DN4NUYPP.2~BNTGWUDEB2OL.0 size=929675 2024-04-07T16:38:03.277 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:03.280+0000 7f346a3868c0 1 > prepared multipart upload key=UIW3DN4NUYPP upload=BNTGWUDEB2OL tag=MVINQDP1O7TI type=write size=1978251 2024-04-07T16:38:03.282 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:03.288+0000 7f346a3868c0 1 < completed delete operation key=UIW3DN4NUYPP tag=OB5KTF7MN5MK type=del 2024-04-07T16:38:03.290 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:03.296+0000 7f346a3868c0 1 > prepared operation key=QVOKSQAGHPA9 tag=9KBL7MZRG3JH type=write size=771314 2024-04-07T16:38:03.295 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:03.304+0000 7f346a3868c0 1 < completed part key=_multipart_1VBSUWNI3V1T.2~WIMEY23UQJD8.0 size=1048576 2024-04-07T16:38:03.301 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:03.308+0000 7f346a3868c0 1 < completed part key=_multipart_1VBSUWNI3V1T.2~WIMEY23UQJD8.1 size=721573 2024-04-07T16:38:03.311 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:03.320+0000 7f346a3868c0 1 < completed multipart upload key=1VBSUWNI3V1T upload=WIMEY23UQJD8 tag=T8TE70MI5L2Z type=write size=2818725 2024-04-07T16:38:03.322 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:03.328+0000 7f346a3868c0 1 > prepared part key=_multipart_95YVKJ14JD8W.2~TLO182HRV5DL.0 size=966160 2024-04-07T16:38:03.322 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:03.328+0000 7f346a3868c0 1 > prepared multipart upload key=95YVKJ14JD8W upload=TLO182HRV5DL tag=AL47KDG7P1I5 type=write size=2014736 2024-04-07T16:38:03.328 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:03.336+0000 7f346a3868c0 1 < completed delete operation key=1WGUPH7X5UA0 tag=JH6R07L5F20E type=del 2024-04-07T16:38:03.341 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:03.348+0000 7f346a3868c0 1 > prepared part key=_multipart_1VBSUWNI3V1T.2~AFKOI3P5DXXE.0 size=794065 2024-04-07T16:38:03.341 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:03.348+0000 7f346a3868c0 1 > prepared multipart upload key=1VBSUWNI3V1T upload=AFKOI3P5DXXE tag=O5F981GF75B5 type=write size=1842641 2024-04-07T16:38:03.354 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:03.360+0000 7f346a3868c0 1 < suggested remove operation key=UIW3DN4NUYPP tag=VHX97MIMS7JV type=del 2024-04-07T16:38:03.359 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:03.368+0000 7f346a3868c0 1 > prepared operation key=YOK8BAEOMPUR tag=9XI990I4942S type=del size=825605 2024-04-07T16:38:03.365 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:03.372+0000 7f346a3868c0 1 < completed delete operation key=ROFH30W3UCAX tag=AIJPDVE5RB75 type=del 2024-04-07T16:38:03.376 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:03.384+0000 7f346a3868c0 1 > prepared part key=_multipart_3N0QJRYVO3FV.2~5S580A7CM1PI.0 size=1048576 2024-04-07T16:38:03.381 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:03.388+0000 7f346a3868c0 1 > prepared part key=_multipart_3N0QJRYVO3FV.2~5S580A7CM1PI.1 size=336610 2024-04-07T16:38:03.381 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:03.388+0000 7f346a3868c0 1 > prepared multipart upload key=3N0QJRYVO3FV upload=5S580A7CM1PI tag=OZ0ADHMKW849 type=write size=2433762 2024-04-07T16:38:03.392 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:03.400+0000 7f346a3868c0 1 < suggested remove operation key=YOK8BAEOMPUR tag=181TOGX6WO8R type=del 2024-04-07T16:38:03.399 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:03.404+0000 7f346a3868c0 1 > prepared operation key=CL4PP1RKZU0E tag=IQFYELKZL6RK type=write size=834601 2024-04-07T16:38:03.404 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:03.412+0000 7f346a3868c0 1 < completed delete operation key=7G5C6HBK6QWI tag=6WV8Y8E8MW66 type=del 2024-04-07T16:38:03.410 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:03.416+0000 7f346a3868c0 1 > prepared operation key=EQR0FY86AT5X tag=6Q5S1Q7T934S type=write size=41250 2024-04-07T16:38:03.416 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:03.424+0000 7f346a3868c0 1 < completed write operation key=QTCIIPTXIRW2 tag=HB8IBUX65MIY type=write size=599046 2024-04-07T16:38:03.422 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:03.428+0000 7f346a3868c0 1 > prepared operation key=YOK8BAEOMPUR tag=MD8AX57WG2IB type=del size=4053239 2024-04-07T16:38:03.423 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:03.428+0000 7f346a3868c0 1 < canceled part key=_multipart_3N0QJRYVO3FV.2~HMY6L2SL43P1.0 size=1048576 2024-04-07T16:38:03.428 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:03.436+0000 7f346a3868c0 1 < completed part key=_multipart_3N0QJRYVO3FV.2~HMY6L2SL43P1.1 size=1048576 2024-04-07T16:38:03.432 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:03.440+0000 7f346a3868c0 1 < completed part key=_multipart_3N0QJRYVO3FV.2~HMY6L2SL43P1.2 size=758621 2024-04-07T16:38:03.443 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:03.448+0000 7f346a3868c0 1 < canceled multipart upload key=3N0QJRYVO3FV upload=HMY6L2SL43P1 tag=T3CNXDMXK1AZ type=write size=3904349 2024-04-07T16:38:03.449 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:03.456+0000 7f346a3868c0 1 > prepared operation key=3N0QJRYVO3FV tag=FSW2J9KAF3KU type=del size=2790460 2024-04-07T16:38:03.454 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:03.460+0000 7f346a3868c0 1 < completed part key=_multipart_8F81G0P0YBFW.2~PF9CGCPKRLS1.0 size=1048576 2024-04-07T16:38:03.459 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:03.468+0000 7f346a3868c0 1 < completed part key=_multipart_8F81G0P0YBFW.2~PF9CGCPKRLS1.1 size=1048576 2024-04-07T16:38:03.464 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:03.472+0000 7f346a3868c0 1 < completed part key=_multipart_8F81G0P0YBFW.2~PF9CGCPKRLS1.2 size=553458 2024-04-07T16:38:03.476 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:03.484+0000 7f346a3868c0 1 < completed multipart upload key=8F81G0P0YBFW upload=PF9CGCPKRLS1 tag=571OS35Q5T8Q type=write size=3699186 2024-04-07T16:38:03.482 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:03.488+0000 7f346a3868c0 1 > prepared operation key=HBLEXONKXJTL tag=Z1EGIG3M30F4 type=del size=2495709 2024-04-07T16:38:03.488 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:03.496+0000 7f346a3868c0 1 < canceled operation key=IP8YW2A75TCW tag=YYMBMR58JJSI type=del size=293153 2024-04-07T16:38:03.495 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:03.500+0000 7f346a3868c0 1 > prepared operation key=NYET3212USVE tag=LDGDJSDL378S type=write size=50378 2024-04-07T16:38:03.502 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:03.508+0000 7f346a3868c0 1 < completed delete operation key=GBQSME7SY1AH tag=OZF9TSBKKURB type=del 2024-04-07T16:38:03.508 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:03.516+0000 7f346a3868c0 1 > prepared operation key=1VBSUWNI3V1T tag=GD1EHU85K329 type=write size=411873 2024-04-07T16:38:03.519 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:03.528+0000 7f346a3868c0 1 < suggested update operation key=XJLQZ7IS9U1T tag=QN0T7T237EUA type=del size=3043972 2024-04-07T16:38:03.525 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:03.532+0000 7f346a3868c0 1 > prepared operation key=X5Q46GO8YTOK tag=9HEGN0GZYBVF type=del size=2015229 2024-04-07T16:38:03.532 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:03.540+0000 7f346a3868c0 1 < completed part key=_multipart_95YVKJ14JD8W.2~3R1Q2L79GX6S.0 size=1048576 2024-04-07T16:38:03.537 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:03.544+0000 7f346a3868c0 1 < completed part key=_multipart_95YVKJ14JD8W.2~3R1Q2L79GX6S.1 size=1048576 2024-04-07T16:38:03.542 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:03.548+0000 7f346a3868c0 1 < completed part key=_multipart_95YVKJ14JD8W.2~3R1Q2L79GX6S.2 size=615885 2024-04-07T16:38:03.553 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:03.560+0000 7f346a3868c0 1 < completed multipart upload key=95YVKJ14JD8W upload=3R1Q2L79GX6S tag=044BC1F4Y07U type=write size=3761613 2024-04-07T16:38:03.559 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:03.568+0000 7f346a3868c0 1 > prepared operation key=QS4HYZFFAJNF tag=13AQ2M3TSAMG type=del size=204836 2024-04-07T16:38:03.565 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:03.572+0000 7f346a3868c0 1 < completed part key=_multipart_LXOZ0T2KY6OR.2~QJ01TYNPL1SP.0 size=995377 2024-04-07T16:38:03.575 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:03.584+0000 7f346a3868c0 1 < completed multipart upload key=LXOZ0T2KY6OR upload=QJ01TYNPL1SP tag=A953RQX2Q7HX type=write size=2043953 2024-04-07T16:38:03.581 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:03.588+0000 7f346a3868c0 1 > prepared operation key=YXMDZB23YTB3 tag=YM502SK2O72M type=del size=1907196 2024-04-07T16:38:03.588 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:03.596+0000 7f346a3868c0 1 < completed part key=_multipart_UIW3DN4NUYPP.2~BNTGWUDEB2OL.0 size=929675 2024-04-07T16:38:03.598 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:03.604+0000 7f346a3868c0 1 < completed multipart upload key=UIW3DN4NUYPP upload=BNTGWUDEB2OL tag=MVINQDP1O7TI type=write size=1978251 2024-04-07T16:38:03.604 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:03.612+0000 7f346a3868c0 1 > prepared operation key=GBQSME7SY1AH tag=JJOOGQGED1RC type=del size=230855 2024-04-07T16:38:03.617 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:03.624+0000 7f346a3868c0 1 < suggested remove operation key=QVOKSQAGHPA9 tag=9KBL7MZRG3JH type=write 2024-04-07T16:38:03.632 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:03.640+0000 7f346a3868c0 1 > prepared part key=_multipart_YXMDZB23YTB3.2~U0U2526MP558.0 size=1048576 2024-04-07T16:38:03.637 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:03.644+0000 7f346a3868c0 1 > prepared part key=_multipart_YXMDZB23YTB3.2~U0U2526MP558.1 size=773053 2024-04-07T16:38:03.637 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:03.644+0000 7f346a3868c0 1 > prepared multipart upload key=YXMDZB23YTB3 upload=U0U2526MP558 tag=PI44VL89ISVJ type=write size=2870205 2024-04-07T16:38:03.644 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:03.652+0000 7f346a3868c0 1 < completed part key=_multipart_95YVKJ14JD8W.2~TLO182HRV5DL.0 size=966160 2024-04-07T16:38:03.655 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:03.664+0000 7f346a3868c0 1 < completed multipart upload key=95YVKJ14JD8W upload=TLO182HRV5DL tag=AL47KDG7P1I5 type=write size=2014736 2024-04-07T16:38:03.661 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:03.668+0000 7f346a3868c0 1 > prepared operation key=XJLQZ7IS9U1T tag=6J1UUNMWJZJO type=del size=2857419 2024-04-07T16:38:03.667 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:03.676+0000 7f346a3868c0 1 < completed part key=_multipart_1VBSUWNI3V1T.2~AFKOI3P5DXXE.0 size=794065 2024-04-07T16:38:03.677 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:03.684+0000 7f346a3868c0 1 < completed multipart upload key=1VBSUWNI3V1T upload=AFKOI3P5DXXE tag=O5F981GF75B5 type=write size=1842641 2024-04-07T16:38:03.684 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:03.692+0000 7f346a3868c0 1 > prepared operation key=1VKVQXSQMTU1 tag=7J2DVKH9W0HU type=del size=1097542 2024-04-07T16:38:03.692 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:03.700+0000 7f346a3868c0 -1 < failed to complete operation key=YOK8BAEOMPUR tag=9XI990I4942S type=del size=825605: (22) Invalid argument 2024-04-07T16:38:03.703 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:03.712+0000 7f346a3868c0 1 > prepared part key=_multipart_X5Q46GO8YTOK.2~KXBDTUB3F2YW.0 size=1048576 2024-04-07T16:38:03.708 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:03.716+0000 7f346a3868c0 1 > prepared part key=_multipart_X5Q46GO8YTOK.2~KXBDTUB3F2YW.1 size=753456 2024-04-07T16:38:03.708 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:03.716+0000 7f346a3868c0 1 > prepared multipart upload key=X5Q46GO8YTOK upload=KXBDTUB3F2YW tag=7GYLM3PQZS76 type=write size=2850608 2024-04-07T16:38:03.714 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:03.720+0000 7f346a3868c0 1 < completed part key=_multipart_3N0QJRYVO3FV.2~5S580A7CM1PI.0 size=1048576 2024-04-07T16:38:03.719 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:03.724+0000 7f346a3868c0 1 < completed part key=_multipart_3N0QJRYVO3FV.2~5S580A7CM1PI.1 size=336610 2024-04-07T16:38:03.730 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:03.736+0000 7f346a3868c0 1 < completed multipart upload key=3N0QJRYVO3FV upload=5S580A7CM1PI tag=OZ0ADHMKW849 type=write size=2433762 2024-04-07T16:38:03.736 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:03.744+0000 7f346a3868c0 1 > prepared operation key=3N0QJRYVO3FV tag=B372WFTHTQ1Z type=del size=139221 2024-04-07T16:38:03.742 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:03.748+0000 7f346a3868c0 1 < completed write operation key=CL4PP1RKZU0E tag=IQFYELKZL6RK type=write size=834601 2024-04-07T16:38:03.753 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:03.760+0000 7f346a3868c0 1 > prepared part key=_multipart_YOK8BAEOMPUR.2~Z5FZC5OKVS2D.0 size=371412 2024-04-07T16:38:03.754 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:03.760+0000 7f346a3868c0 1 > prepared multipart upload key=YOK8BAEOMPUR upload=Z5FZC5OKVS2D tag=5ISBJRJ3USSZ type=write size=1419988 2024-04-07T16:38:03.760 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:03.768+0000 7f346a3868c0 1 < completed write operation key=EQR0FY86AT5X tag=6Q5S1Q7T934S type=write size=41250 2024-04-07T16:38:03.771 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:03.780+0000 7f346a3868c0 1 > prepared part key=_multipart_LXOZ0T2KY6OR.2~QGNWRM4AGCS4.0 size=1048576 2024-04-07T16:38:03.776 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:03.784+0000 7f346a3868c0 1 > prepared part key=_multipart_LXOZ0T2KY6OR.2~QGNWRM4AGCS4.1 size=701029 2024-04-07T16:38:03.776 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:03.784+0000 7f346a3868c0 1 > prepared multipart upload key=LXOZ0T2KY6OR upload=QGNWRM4AGCS4 tag=GT32GBHTXALA type=write size=2798181 2024-04-07T16:38:03.782 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:03.788+0000 7f346a3868c0 1 < completed delete operation key=YOK8BAEOMPUR tag=MD8AX57WG2IB type=del 2024-04-07T16:38:03.794 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:03.800+0000 7f346a3868c0 1 > prepared part key=_multipart_C1BTVJNHIM0X.2~RWLKSKP2X3UM.0 size=1048576 2024-04-07T16:38:03.801 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:03.808+0000 7f346a3868c0 1 > prepared part key=_multipart_C1BTVJNHIM0X.2~RWLKSKP2X3UM.1 size=79208 2024-04-07T16:38:03.801 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:03.808+0000 7f346a3868c0 1 > prepared multipart upload key=C1BTVJNHIM0X upload=RWLKSKP2X3UM tag=LYTKN1FIVOKK type=write size=2176360 2024-04-07T16:38:03.806 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:03.812+0000 7f346a3868c0 1 < completed delete operation key=3N0QJRYVO3FV tag=FSW2J9KAF3KU type=del 2024-04-07T16:38:03.817 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:03.824+0000 7f346a3868c0 1 > prepared part key=_multipart_QVOKSQAGHPA9.2~HKH4FQDUEKQS.0 size=1048576 2024-04-07T16:38:03.822 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:03.828+0000 7f346a3868c0 1 > prepared part key=_multipart_QVOKSQAGHPA9.2~HKH4FQDUEKQS.1 size=491030 2024-04-07T16:38:03.822 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:03.828+0000 7f346a3868c0 1 > prepared multipart upload key=QVOKSQAGHPA9 upload=HKH4FQDUEKQS tag=5QZX7PPGP98G type=write size=2588182 2024-04-07T16:38:03.828 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:03.836+0000 7f346a3868c0 1 < completed delete operation key=HBLEXONKXJTL tag=Z1EGIG3M30F4 type=del 2024-04-07T16:38:03.833 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:03.840+0000 7f346a3868c0 1 > prepared operation key=1WGUPH7X5UA0 tag=DD9752O02TOA type=del size=495070 2024-04-07T16:38:03.839 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:03.848+0000 7f346a3868c0 1 < completed write operation key=NYET3212USVE tag=LDGDJSDL378S type=write size=50378 2024-04-07T16:38:03.846 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:03.852+0000 7f346a3868c0 1 > prepared operation key=QS4HYZFFAJNF tag=LBTDY9DAQSGX type=del size=2716107 2024-04-07T16:38:03.852 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:03.860+0000 7f346a3868c0 1 < completed write operation key=1VBSUWNI3V1T tag=GD1EHU85K329 type=write size=411873 2024-04-07T16:38:03.858 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:03.864+0000 7f346a3868c0 1 > prepared operation key=XJLQZ7IS9U1T tag=MC42C57I98RK type=del size=3132724 2024-04-07T16:38:03.864 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:03.872+0000 7f346a3868c0 1 < completed delete operation key=X5Q46GO8YTOK tag=9HEGN0GZYBVF type=del 2024-04-07T16:38:03.870 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:03.876+0000 7f346a3868c0 1 > prepared operation key=HBLEXONKXJTL tag=AFFZRE7M1Q20 type=del size=1405170 2024-04-07T16:38:03.876 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:03.884+0000 7f346a3868c0 1 < completed delete operation key=QS4HYZFFAJNF tag=13AQ2M3TSAMG type=del 2024-04-07T16:38:03.881 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:03.888+0000 7f346a3868c0 1 > prepared operation key=95YVKJ14JD8W tag=3N00C6211RZM type=write size=926736 2024-04-07T16:38:03.893 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:03.900+0000 7f346a3868c0 1 < suggested remove operation key=YXMDZB23YTB3 tag=YM502SK2O72M type=del 2024-04-07T16:38:03.898 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:03.904+0000 7f346a3868c0 1 > prepared operation key=UIW3DN4NUYPP tag=YUTGLTT1M586 type=del size=1382989 2024-04-07T16:38:03.904 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:03.912+0000 7f346a3868c0 1 < completed delete operation key=GBQSME7SY1AH tag=JJOOGQGED1RC type=del 2024-04-07T16:38:03.910 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:03.916+0000 7f346a3868c0 1 > prepared operation key=GBQSME7SY1AH tag=DEZTQHJV7W2J type=del size=374926 2024-04-07T16:38:03.916 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:03.924+0000 7f346a3868c0 1 < completed part key=_multipart_YXMDZB23YTB3.2~U0U2526MP558.0 size=1048576 2024-04-07T16:38:03.921 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:03.928+0000 7f346a3868c0 1 < completed part key=_multipart_YXMDZB23YTB3.2~U0U2526MP558.1 size=773053 2024-04-07T16:38:03.932 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:03.940+0000 7f346a3868c0 1 < completed multipart upload key=YXMDZB23YTB3 upload=U0U2526MP558 tag=PI44VL89ISVJ type=write size=2870205 2024-04-07T16:38:03.943 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:03.952+0000 7f346a3868c0 1 > prepared part key=_multipart_6XDS0A7L0IY8.2~GGP7MF9AQSER.0 size=1048576 2024-04-07T16:38:03.948 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:03.956+0000 7f346a3868c0 1 > prepared part key=_multipart_6XDS0A7L0IY8.2~GGP7MF9AQSER.1 size=995318 2024-04-07T16:38:03.948 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:03.956+0000 7f346a3868c0 1 > prepared multipart upload key=6XDS0A7L0IY8 upload=GGP7MF9AQSER tag=RS3BMQARGWCG type=write size=3092470 2024-04-07T16:38:03.954 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:03.960+0000 7f346a3868c0 1 < completed delete operation key=XJLQZ7IS9U1T tag=6J1UUNMWJZJO type=del 2024-04-07T16:38:03.960 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:03.968+0000 7f346a3868c0 1 > prepared operation key=1WGUPH7X5UA0 tag=1V8XNWCA280W type=del size=1587726 2024-04-07T16:38:03.965 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:03.972+0000 7f346a3868c0 1 < completed delete operation key=1VKVQXSQMTU1 tag=7J2DVKH9W0HU type=del 2024-04-07T16:38:03.971 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:03.976+0000 7f346a3868c0 1 > prepared operation key=HBLEXONKXJTL tag=D46XYSG4YS4D type=write size=340929 2024-04-07T16:38:03.976 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:03.984+0000 7f346a3868c0 1 < completed part key=_multipart_X5Q46GO8YTOK.2~KXBDTUB3F2YW.0 size=1048576 2024-04-07T16:38:03.981 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:03.988+0000 7f346a3868c0 1 < completed part key=_multipart_X5Q46GO8YTOK.2~KXBDTUB3F2YW.1 size=753456 2024-04-07T16:38:03.991 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:04.000+0000 7f346a3868c0 1 < completed multipart upload key=X5Q46GO8YTOK upload=KXBDTUB3F2YW tag=7GYLM3PQZS76 type=write size=2850608 2024-04-07T16:38:04.004 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:04.012+0000 7f346a3868c0 1 > prepared part key=_multipart_QS4HYZFFAJNF.2~HAW0PTYRVI2X.0 size=179954 2024-04-07T16:38:04.004 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:04.012+0000 7f346a3868c0 1 > prepared multipart upload key=QS4HYZFFAJNF upload=HAW0PTYRVI2X tag=3KS0X22DRRH3 type=write size=1228530 2024-04-07T16:38:04.015 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:04.020+0000 7f346a3868c0 1 < suggested remove operation key=3N0QJRYVO3FV tag=B372WFTHTQ1Z type=del 2024-04-07T16:38:04.021 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:04.028+0000 7f346a3868c0 1 > prepared operation key=HBLEXONKXJTL tag=RQHEL8D041H4 type=del size=1963780 2024-04-07T16:38:04.026 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:04.032+0000 7f346a3868c0 1 < completed part key=_multipart_YOK8BAEOMPUR.2~Z5FZC5OKVS2D.0 size=371412 2024-04-07T16:38:04.036 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:04.044+0000 7f346a3868c0 1 < completed multipart upload key=YOK8BAEOMPUR upload=Z5FZC5OKVS2D tag=5ISBJRJ3USSZ type=write size=1419988 2024-04-07T16:38:04.046 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:04.052+0000 7f346a3868c0 1 > prepared part key=_multipart_XJLQZ7IS9U1T.2~ZFI9BMSC3GZM.0 size=350510 2024-04-07T16:38:04.046 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:04.052+0000 7f346a3868c0 1 > prepared multipart upload key=XJLQZ7IS9U1T upload=ZFI9BMSC3GZM tag=67NBUP8H7W9B type=write size=1399086 2024-04-07T16:38:04.051 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:04.060+0000 7f346a3868c0 1 < completed part key=_multipart_LXOZ0T2KY6OR.2~QGNWRM4AGCS4.0 size=1048576 2024-04-07T16:38:04.052 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:04.060+0000 7f346a3868c0 1 < canceled part key=_multipart_LXOZ0T2KY6OR.2~QGNWRM4AGCS4.1 size=701029 2024-04-07T16:38:04.061 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:04.068+0000 7f346a3868c0 1 < canceled multipart upload key=LXOZ0T2KY6OR upload=QGNWRM4AGCS4 tag=GT32GBHTXALA type=write size=2798181 2024-04-07T16:38:04.071 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:04.080+0000 7f346a3868c0 1 > prepared part key=_multipart_QS4HYZFFAJNF.2~IL4YB2O20B1C.0 size=1048576 2024-04-07T16:38:04.076 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:04.084+0000 7f346a3868c0 1 > prepared part key=_multipart_QS4HYZFFAJNF.2~IL4YB2O20B1C.1 size=101244 2024-04-07T16:38:04.076 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:04.084+0000 7f346a3868c0 1 > prepared multipart upload key=QS4HYZFFAJNF upload=IL4YB2O20B1C tag=RE8S8LL9PYX3 type=write size=2198396 2024-04-07T16:38:04.081 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:04.088+0000 7f346a3868c0 1 < completed part key=_multipart_C1BTVJNHIM0X.2~RWLKSKP2X3UM.0 size=1048576 2024-04-07T16:38:04.087 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:04.092+0000 7f346a3868c0 1 < completed part key=_multipart_C1BTVJNHIM0X.2~RWLKSKP2X3UM.1 size=79208 2024-04-07T16:38:04.098 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:04.104+0000 7f346a3868c0 1 < completed multipart upload key=C1BTVJNHIM0X upload=RWLKSKP2X3UM tag=LYTKN1FIVOKK type=write size=2176360 2024-04-07T16:38:04.106 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:04.112+0000 7f346a3868c0 1 > prepared part key=_multipart_QS4HYZFFAJNF.2~VQAND043XCC6.0 size=1048576 2024-04-07T16:38:04.111 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:04.120+0000 7f346a3868c0 1 > prepared part key=_multipart_QS4HYZFFAJNF.2~VQAND043XCC6.1 size=1035407 2024-04-07T16:38:04.111 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:04.120+0000 7f346a3868c0 1 > prepared multipart upload key=QS4HYZFFAJNF upload=VQAND043XCC6 tag=VR6WBMMHYBIL type=write size=3132559 2024-04-07T16:38:04.117 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:04.124+0000 7f346a3868c0 1 < completed part key=_multipart_QVOKSQAGHPA9.2~HKH4FQDUEKQS.0 size=1048576 2024-04-07T16:38:04.123 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:04.128+0000 7f346a3868c0 1 < completed part key=_multipart_QVOKSQAGHPA9.2~HKH4FQDUEKQS.1 size=491030 2024-04-07T16:38:04.135 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:04.144+0000 7f346a3868c0 1 < completed multipart upload key=QVOKSQAGHPA9 upload=HKH4FQDUEKQS tag=5QZX7PPGP98G type=write size=2588182 2024-04-07T16:38:04.141 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:04.148+0000 7f346a3868c0 1 > prepared operation key=NYET3212USVE tag=GH601JU8X4C9 type=del size=1205586 2024-04-07T16:38:04.146 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:04.152+0000 7f346a3868c0 1 < completed delete operation key=1WGUPH7X5UA0 tag=DD9752O02TOA type=del 2024-04-07T16:38:04.152 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:04.160+0000 7f346a3868c0 1 > prepared operation key=ROFH30W3UCAX tag=FIUB7EO5QOAM type=del size=2795593 2024-04-07T16:38:04.157 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:04.164+0000 7f346a3868c0 1 < completed delete operation key=QS4HYZFFAJNF tag=LBTDY9DAQSGX type=del 2024-04-07T16:38:04.168 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:04.176+0000 7f346a3868c0 1 > prepared part key=_multipart_C1BTVJNHIM0X.2~KM8PX8JBHYB9.0 size=1048576 2024-04-07T16:38:04.173 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:04.180+0000 7f346a3868c0 1 > prepared part key=_multipart_C1BTVJNHIM0X.2~KM8PX8JBHYB9.1 size=760216 2024-04-07T16:38:04.173 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:04.180+0000 7f346a3868c0 1 > prepared multipart upload key=C1BTVJNHIM0X upload=KM8PX8JBHYB9 tag=8WAE9VF02635 type=write size=2857368 2024-04-07T16:38:04.179 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:04.188+0000 7f346a3868c0 1 < completed delete operation key=XJLQZ7IS9U1T tag=MC42C57I98RK type=del 2024-04-07T16:38:04.186 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:04.192+0000 7f346a3868c0 1 > prepared operation key=1WGUPH7X5UA0 tag=WLZJ13P1FZ4A type=del size=1155832 2024-04-07T16:38:04.193 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:04.200+0000 7f346a3868c0 1 < completed delete operation key=HBLEXONKXJTL tag=AFFZRE7M1Q20 type=del 2024-04-07T16:38:04.204 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:04.212+0000 7f346a3868c0 1 > prepared part key=_multipart_QTCIIPTXIRW2.2~FY4LE2SQS94V.0 size=1048576 2024-04-07T16:38:04.209 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:04.216+0000 7f346a3868c0 1 > prepared part key=_multipart_QTCIIPTXIRW2.2~FY4LE2SQS94V.1 size=228524 2024-04-07T16:38:04.209 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:04.216+0000 7f346a3868c0 1 > prepared multipart upload key=QTCIIPTXIRW2 upload=FY4LE2SQS94V tag=4CGDTM5ORA7H type=write size=2325676 2024-04-07T16:38:04.215 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:04.220+0000 7f346a3868c0 1 < canceled operation key=95YVKJ14JD8W tag=3N00C6211RZM type=write size=926736 2024-04-07T16:38:04.220 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:04.228+0000 7f346a3868c0 1 > prepared operation key=HBLEXONKXJTL tag=W6NTXIR9YFI3 type=write size=531496 2024-04-07T16:38:04.226 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:04.232+0000 7f346a3868c0 1 < completed delete operation key=UIW3DN4NUYPP tag=YUTGLTT1M586 type=del 2024-04-07T16:38:04.232 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:04.240+0000 7f346a3868c0 1 > prepared operation key=1WGUPH7X5UA0 tag=RM018KWZBAKE type=del size=4098443 2024-04-07T16:38:04.238 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:04.244+0000 7f346a3868c0 1 < completed delete operation key=GBQSME7SY1AH tag=DEZTQHJV7W2J type=del 2024-04-07T16:38:04.249 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:04.256+0000 7f346a3868c0 1 > prepared part key=_multipart_QS4HYZFFAJNF.2~R9HTIVHOTW5G.0 size=1048576 2024-04-07T16:38:04.254 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:04.260+0000 7f346a3868c0 1 > prepared part key=_multipart_QS4HYZFFAJNF.2~R9HTIVHOTW5G.1 size=717075 2024-04-07T16:38:04.254 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:04.260+0000 7f346a3868c0 1 > prepared multipart upload key=QS4HYZFFAJNF upload=R9HTIVHOTW5G tag=RIMY24G8NE16 type=write size=2814227 2024-04-07T16:38:04.261 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:04.268+0000 7f346a3868c0 1 < completed part key=_multipart_6XDS0A7L0IY8.2~GGP7MF9AQSER.0 size=1048576 2024-04-07T16:38:04.266 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:04.272+0000 7f346a3868c0 1 < completed part key=_multipart_6XDS0A7L0IY8.2~GGP7MF9AQSER.1 size=995318 2024-04-07T16:38:04.277 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:04.284+0000 7f346a3868c0 1 < completed multipart upload key=6XDS0A7L0IY8 upload=GGP7MF9AQSER tag=RS3BMQARGWCG type=write size=3092470 2024-04-07T16:38:04.283 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:04.292+0000 7f346a3868c0 1 > prepared operation key=HBLEXONKXJTL tag=M2A0PBK2ZFAO type=del size=3660606 2024-04-07T16:38:04.290 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:04.296+0000 7f346a3868c0 1 < completed delete operation key=1WGUPH7X5UA0 tag=1V8XNWCA280W type=del 2024-04-07T16:38:04.296 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:04.304+0000 7f346a3868c0 1 > prepared operation key=EOYFVEI50SQ1 tag=B2NL12KKGP2Y type=del size=1952428 2024-04-07T16:38:04.309 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:04.316+0000 7f346a3868c0 1 < suggested remove operation key=HBLEXONKXJTL tag=D46XYSG4YS4D type=write 2024-04-07T16:38:04.321 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:04.328+0000 7f346a3868c0 1 > prepared part key=_multipart_HBLEXONKXJTL.2~DSISTBFFYP1B.0 size=1048576 2024-04-07T16:38:04.326 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:04.332+0000 7f346a3868c0 1 > prepared part key=_multipart_HBLEXONKXJTL.2~DSISTBFFYP1B.1 size=1048576 2024-04-07T16:38:04.331 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:04.340+0000 7f346a3868c0 1 > prepared part key=_multipart_HBLEXONKXJTL.2~DSISTBFFYP1B.2 size=488916 2024-04-07T16:38:04.331 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:04.340+0000 7f346a3868c0 1 > prepared multipart upload key=HBLEXONKXJTL upload=DSISTBFFYP1B tag=UPH7O35N616R type=write size=3634644 2024-04-07T16:38:04.338 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:04.344+0000 7f346a3868c0 1 < completed part key=_multipart_QS4HYZFFAJNF.2~HAW0PTYRVI2X.0 size=179954 2024-04-07T16:38:04.348 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:04.356+0000 7f346a3868c0 1 < completed multipart upload key=QS4HYZFFAJNF upload=HAW0PTYRVI2X tag=3KS0X22DRRH3 type=write size=1228530 2024-04-07T16:38:04.358 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:04.364+0000 7f346a3868c0 1 > prepared part key=_multipart_QTCIIPTXIRW2.2~PCVO24UMJB7S.0 size=249234 2024-04-07T16:38:04.358 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:04.364+0000 7f346a3868c0 1 > prepared multipart upload key=QTCIIPTXIRW2 upload=PCVO24UMJB7S tag=URT0S4U3W3GN type=write size=1297810 2024-04-07T16:38:04.360 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:04.368+0000 7f346a3868c0 -1 < no bi entry to suggest for operation key=HBLEXONKXJTL tag=RQHEL8D041H4 type=del size=1963780: (2) No such file or directory 2024-04-07T16:38:04.366 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:04.372+0000 7f346a3868c0 1 > prepared operation key=QS4HYZFFAJNF tag=1KVLQ0YW4AOB type=del size=960244 2024-04-07T16:38:04.371 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:04.380+0000 7f346a3868c0 1 < completed part key=_multipart_XJLQZ7IS9U1T.2~ZFI9BMSC3GZM.0 size=350510 2024-04-07T16:38:04.381 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:04.388+0000 7f346a3868c0 1 < completed multipart upload key=XJLQZ7IS9U1T upload=ZFI9BMSC3GZM tag=67NBUP8H7W9B type=write size=1399086 2024-04-07T16:38:04.387 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:04.396+0000 7f346a3868c0 1 > prepared operation key=UIW3DN4NUYPP tag=SF2VLM3WWJXX type=write size=649111 2024-04-07T16:38:04.393 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:04.400+0000 7f346a3868c0 1 < completed part key=_multipart_QS4HYZFFAJNF.2~IL4YB2O20B1C.0 size=1048576 2024-04-07T16:38:04.397 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:04.404+0000 7f346a3868c0 1 < completed part key=_multipart_QS4HYZFFAJNF.2~IL4YB2O20B1C.1 size=101244 2024-04-07T16:38:04.407 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:04.416+0000 7f346a3868c0 1 < completed multipart upload key=QS4HYZFFAJNF upload=IL4YB2O20B1C tag=RE8S8LL9PYX3 type=write size=2198396 2024-04-07T16:38:04.419 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:04.428+0000 7f346a3868c0 1 > prepared part key=_multipart_C1BTVJNHIM0X.2~M7GU0DZMBGSZ.0 size=819862 2024-04-07T16:38:04.420 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:04.428+0000 7f346a3868c0 1 > prepared multipart upload key=C1BTVJNHIM0X upload=M7GU0DZMBGSZ tag=LV9A6SUGQ9BK type=write size=1868438 2024-04-07T16:38:04.425 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:04.432+0000 7f346a3868c0 1 < completed part key=_multipart_QS4HYZFFAJNF.2~VQAND043XCC6.0 size=1048576 2024-04-07T16:38:04.430 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:04.436+0000 7f346a3868c0 1 < completed part key=_multipart_QS4HYZFFAJNF.2~VQAND043XCC6.1 size=1035407 2024-04-07T16:38:04.440 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:04.448+0000 7f346a3868c0 1 < completed multipart upload key=QS4HYZFFAJNF upload=VQAND043XCC6 tag=VR6WBMMHYBIL type=write size=3132559 2024-04-07T16:38:04.441 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:38:04 smithi100 bash[74942]: cluster 2024-04-07T16:38:03.066340+0000 mgr.y (mgr.25175) 681 : cluster [DBG] pgmap v1316: 137 pgs: 137 active+clean; 583 KiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 607 KiB/s rd, 209 KiB/s wr, 1.07k op/s 2024-04-07T16:38:04.441 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:38:04 smithi100 bash[76202]: cluster 2024-04-07T16:38:03.066340+0000 mgr.y (mgr.25175) 681 : cluster [DBG] pgmap v1316: 137 pgs: 137 active+clean; 583 KiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 607 KiB/s rd, 209 KiB/s wr, 1.07k op/s 2024-04-07T16:38:04.446 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:04.452+0000 7f346a3868c0 1 > prepared operation key=C1BTVJNHIM0X tag=A4Q5Z6NKQ27C type=del size=3998008 2024-04-07T16:38:04.457 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:04.464+0000 7f346a3868c0 1 < suggested update operation key=NYET3212USVE tag=GH601JU8X4C9 type=del size=1205586 2024-04-07T16:38:04.468 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:04.476+0000 7f346a3868c0 1 > prepared part key=_multipart_95YVKJ14JD8W.2~U0GQRLKBTKAW.0 size=1048576 2024-04-07T16:38:04.473 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:04.480+0000 7f346a3868c0 1 > prepared part key=_multipart_95YVKJ14JD8W.2~U0GQRLKBTKAW.1 size=354612 2024-04-07T16:38:04.473 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:04.480+0000 7f346a3868c0 1 > prepared multipart upload key=95YVKJ14JD8W upload=U0GQRLKBTKAW tag=XDUCOG2OORIN type=write size=2451764 2024-04-07T16:38:04.487 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:04.492+0000 7f346a3868c0 1 < suggested remove operation key=ROFH30W3UCAX tag=FIUB7EO5QOAM type=del 2024-04-07T16:38:04.492 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:04.500+0000 7f346a3868c0 1 > prepared operation key=1WGUPH7X5UA0 tag=ZNE8X2GVMEDE type=write size=326754 2024-04-07T16:38:04.499 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:04.504+0000 7f346a3868c0 1 < completed part key=_multipart_C1BTVJNHIM0X.2~KM8PX8JBHYB9.0 size=1048576 2024-04-07T16:38:04.503 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:04.512+0000 7f346a3868c0 1 < completed part key=_multipart_C1BTVJNHIM0X.2~KM8PX8JBHYB9.1 size=760216 2024-04-07T16:38:04.514 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:04.520+0000 7f346a3868c0 1 < completed multipart upload key=C1BTVJNHIM0X upload=KM8PX8JBHYB9 tag=8WAE9VF02635 type=write size=2857368 2024-04-07T16:38:04.520 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:04.528+0000 7f346a3868c0 1 > prepared operation key=1WGUPH7X5UA0 tag=PU72UKCEKW30 type=del size=2381477 2024-04-07T16:38:04.532 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:04.540+0000 7f346a3868c0 1 < suggested update operation key=1WGUPH7X5UA0 tag=WLZJ13P1FZ4A type=del size=1155832 2024-04-07T16:38:04.543 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:04.552+0000 7f346a3868c0 1 > prepared part key=_multipart_XJLQZ7IS9U1T.2~JC7QNIFIT590.0 size=1048576 2024-04-07T16:38:04.548 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:04.556+0000 7f346a3868c0 1 > prepared part key=_multipart_XJLQZ7IS9U1T.2~JC7QNIFIT590.1 size=832937 2024-04-07T16:38:04.548 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:04.556+0000 7f346a3868c0 1 > prepared multipart upload key=XJLQZ7IS9U1T upload=JC7QNIFIT590 tag=T8OC6HABDLFD type=write size=2930089 2024-04-07T16:38:04.553 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:04.560+0000 7f346a3868c0 1 < completed part key=_multipart_QTCIIPTXIRW2.2~FY4LE2SQS94V.0 size=1048576 2024-04-07T16:38:04.558 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:04.564+0000 7f346a3868c0 1 < completed part key=_multipart_QTCIIPTXIRW2.2~FY4LE2SQS94V.1 size=228524 2024-04-07T16:38:04.568 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:04.576+0000 7f346a3868c0 1 < completed multipart upload key=QTCIIPTXIRW2 upload=FY4LE2SQS94V tag=4CGDTM5ORA7H type=write size=2325676 2024-04-07T16:38:04.574 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:04.580+0000 7f346a3868c0 1 > prepared operation key=QVOKSQAGHPA9 tag=TDKNVHMLB5PY type=del size=3563125 2024-04-07T16:38:04.582 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:04.588+0000 7f346a3868c0 -1 < failed to complete operation key=HBLEXONKXJTL tag=W6NTXIR9YFI3 type=write size=531496: (22) Invalid argument 2024-04-07T16:38:04.588 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:04.596+0000 7f346a3868c0 1 > prepared operation key=95YVKJ14JD8W tag=50RTF8YAQICC type=del size=763377 2024-04-07T16:38:04.596 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:04.604+0000 7f346a3868c0 -1 < failed to complete operation key=1WGUPH7X5UA0 tag=RM018KWZBAKE type=del size=4098443: (22) Invalid argument 2024-04-07T16:38:04.602 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:04.608+0000 7f346a3868c0 1 > prepared operation key=QS4HYZFFAJNF tag=3SQRX6Z1H0NZ type=del size=1543262 2024-04-07T16:38:04.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:38:04 smithi144 bash[66654]: cluster 2024-04-07T16:38:03.066340+0000 mgr.y (mgr.25175) 681 : cluster [DBG] pgmap v1316: 137 pgs: 137 active+clean; 583 KiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 607 KiB/s rd, 209 KiB/s wr, 1.07k op/s 2024-04-07T16:38:04.608 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:04.616+0000 7f346a3868c0 1 < completed part key=_multipart_QS4HYZFFAJNF.2~R9HTIVHOTW5G.0 size=1048576 2024-04-07T16:38:04.614 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:04.620+0000 7f346a3868c0 1 < completed part key=_multipart_QS4HYZFFAJNF.2~R9HTIVHOTW5G.1 size=717075 2024-04-07T16:38:04.624 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:04.632+0000 7f346a3868c0 1 < completed multipart upload key=QS4HYZFFAJNF upload=R9HTIVHOTW5G tag=RIMY24G8NE16 type=write size=2814227 2024-04-07T16:38:04.635 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:04.644+0000 7f346a3868c0 1 > prepared part key=_multipart_YXMDZB23YTB3.2~SI5ZR289F6E6.0 size=1048576 2024-04-07T16:38:04.640 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:04.648+0000 7f346a3868c0 1 > prepared part key=_multipart_YXMDZB23YTB3.2~SI5ZR289F6E6.1 size=779916 2024-04-07T16:38:04.640 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:04.648+0000 7f346a3868c0 1 > prepared multipart upload key=YXMDZB23YTB3 upload=SI5ZR289F6E6 tag=U2K4XAWZ9S2W type=write size=2877068 2024-04-07T16:38:04.642 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:04.648+0000 7f346a3868c0 -1 < no bi entry to suggest for operation key=HBLEXONKXJTL tag=M2A0PBK2ZFAO type=del size=3660606: (2) No such file or directory 2024-04-07T16:38:04.652 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:04.660+0000 7f346a3868c0 1 > prepared part key=_multipart_YOK8BAEOMPUR.2~LLNN1DY2QKI6.0 size=1048576 2024-04-07T16:38:04.658 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:04.664+0000 7f346a3868c0 1 > prepared part key=_multipart_YOK8BAEOMPUR.2~LLNN1DY2QKI6.1 size=1048576 2024-04-07T16:38:04.663 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:04.668+0000 7f346a3868c0 1 > prepared part key=_multipart_YOK8BAEOMPUR.2~LLNN1DY2QKI6.2 size=523874 2024-04-07T16:38:04.663 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:04.672+0000 7f346a3868c0 1 > prepared multipart upload key=YOK8BAEOMPUR upload=LLNN1DY2QKI6 tag=WCEQX2MBA02A type=write size=3669602 2024-04-07T16:38:04.670 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:04.676+0000 7f346a3868c0 1 < completed delete operation key=EOYFVEI50SQ1 tag=B2NL12KKGP2Y type=del 2024-04-07T16:38:04.676 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:04.684+0000 7f346a3868c0 1 > prepared operation key=ROFH30W3UCAX tag=ADIAH1Y46NRK type=del size=2913468 2024-04-07T16:38:04.683 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:04.692+0000 7f346a3868c0 1 < completed part key=_multipart_HBLEXONKXJTL.2~DSISTBFFYP1B.0 size=1048576 2024-04-07T16:38:04.688 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:04.696+0000 7f346a3868c0 1 < completed part key=_multipart_HBLEXONKXJTL.2~DSISTBFFYP1B.1 size=1048576 2024-04-07T16:38:04.693 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:04.700+0000 7f346a3868c0 1 < completed part key=_multipart_HBLEXONKXJTL.2~DSISTBFFYP1B.2 size=488916 2024-04-07T16:38:04.703 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:04.712+0000 7f346a3868c0 1 < completed multipart upload key=HBLEXONKXJTL upload=DSISTBFFYP1B tag=UPH7O35N616R type=write size=3634644 2024-04-07T16:38:04.714 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:04.720+0000 7f346a3868c0 1 > prepared part key=_multipart_UIW3DN4NUYPP.2~UCXC4WHVB0FB.0 size=1048576 2024-04-07T16:38:04.719 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:04.724+0000 7f346a3868c0 1 > prepared part key=_multipart_UIW3DN4NUYPP.2~UCXC4WHVB0FB.1 size=26439 2024-04-07T16:38:04.719 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:04.724+0000 7f346a3868c0 1 > prepared multipart upload key=UIW3DN4NUYPP upload=UCXC4WHVB0FB tag=7U3SS0F5FNB5 type=write size=2123591 2024-04-07T16:38:04.726 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:04.732+0000 7f346a3868c0 1 < completed part key=_multipart_QTCIIPTXIRW2.2~PCVO24UMJB7S.0 size=249234 2024-04-07T16:38:04.737 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:04.744+0000 7f346a3868c0 1 < completed multipart upload key=QTCIIPTXIRW2 upload=PCVO24UMJB7S tag=URT0S4U3W3GN type=write size=1297810 2024-04-07T16:38:04.747 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:04.756+0000 7f346a3868c0 1 > prepared part key=_multipart_QS4HYZFFAJNF.2~FKJBTM19WZEA.0 size=1048576 2024-04-07T16:38:04.754 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:04.760+0000 7f346a3868c0 1 > prepared part key=_multipart_QS4HYZFFAJNF.2~FKJBTM19WZEA.1 size=667634 2024-04-07T16:38:04.755 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:04.760+0000 7f346a3868c0 1 > prepared multipart upload key=QS4HYZFFAJNF upload=FKJBTM19WZEA tag=KK65F85XSNDD type=write size=2764786 2024-04-07T16:38:04.767 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:04.776+0000 7f346a3868c0 1 < suggested update operation key=QS4HYZFFAJNF tag=1KVLQ0YW4AOB type=del size=960244 2024-04-07T16:38:04.773 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:04.780+0000 7f346a3868c0 1 > prepared operation key=QS4HYZFFAJNF tag=Q7FNUEH57XIV type=write size=165609 2024-04-07T16:38:04.782 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:04.788+0000 7f346a3868c0 1 < completed write operation key=UIW3DN4NUYPP tag=SF2VLM3WWJXX type=write size=649111 2024-04-07T16:38:04.788 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:04.796+0000 7f346a3868c0 1 > prepared operation key=U3EID5EK64HD tag=QO3WUTD4WLN1 type=write size=423475 2024-04-07T16:38:04.794 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:04.800+0000 7f346a3868c0 1 < completed part key=_multipart_C1BTVJNHIM0X.2~M7GU0DZMBGSZ.0 size=819862 2024-04-07T16:38:04.804 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:04.812+0000 7f346a3868c0 1 < completed multipart upload key=C1BTVJNHIM0X upload=M7GU0DZMBGSZ tag=LV9A6SUGQ9BK type=write size=1868438 2024-04-07T16:38:04.810 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:04.816+0000 7f346a3868c0 1 > prepared operation key=EOYFVEI50SQ1 tag=GLE0K67FXO1B type=del size=531376 2024-04-07T16:38:04.821 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:04.828+0000 7f346a3868c0 1 < suggested update operation key=C1BTVJNHIM0X tag=A4Q5Z6NKQ27C type=del size=3998008 2024-04-07T16:38:04.827 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:04.836+0000 7f346a3868c0 1 > prepared operation key=CL4PP1RKZU0E tag=XT114UBV1MGL type=del size=4141387 2024-04-07T16:38:04.833 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:04.840+0000 7f346a3868c0 1 < completed part key=_multipart_95YVKJ14JD8W.2~U0GQRLKBTKAW.0 size=1048576 2024-04-07T16:38:04.838 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:04.844+0000 7f346a3868c0 1 < completed part key=_multipart_95YVKJ14JD8W.2~U0GQRLKBTKAW.1 size=354612 2024-04-07T16:38:04.848 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:04.856+0000 7f346a3868c0 1 < completed multipart upload key=95YVKJ14JD8W upload=U0GQRLKBTKAW tag=XDUCOG2OORIN type=write size=2451764 2024-04-07T16:38:04.854 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:04.860+0000 7f346a3868c0 1 > prepared operation key=E84U0SX44NQ8 tag=3DQAPCIN8HLV type=del size=2211541 2024-04-07T16:38:04.862 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:04.868+0000 7f346a3868c0 -1 < failed to complete operation key=1WGUPH7X5UA0 tag=ZNE8X2GVMEDE type=write size=326754: (22) Invalid argument 2024-04-07T16:38:04.867 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:04.876+0000 7f346a3868c0 1 > prepared operation key=N14BUXLJONHZ tag=WON4HGXNWUK1 type=del size=2504191 2024-04-07T16:38:04.875 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:04.884+0000 7f346a3868c0 -1 < failed to complete operation key=1WGUPH7X5UA0 tag=PU72UKCEKW30 type=del size=2381477: (22) Invalid argument 2024-04-07T16:38:04.881 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:04.888+0000 7f346a3868c0 1 > prepared operation key=NYET3212USVE tag=UB4J8O0OSMEM type=write size=822079 2024-04-07T16:38:04.886 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:04.892+0000 7f346a3868c0 1 < completed part key=_multipart_XJLQZ7IS9U1T.2~JC7QNIFIT590.0 size=1048576 2024-04-07T16:38:04.891 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:04.900+0000 7f346a3868c0 1 < completed part key=_multipart_XJLQZ7IS9U1T.2~JC7QNIFIT590.1 size=832937 2024-04-07T16:38:04.901 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:04.908+0000 7f346a3868c0 1 < completed multipart upload key=XJLQZ7IS9U1T upload=JC7QNIFIT590 tag=T8OC6HABDLFD type=write size=2930089 2024-04-07T16:38:04.907 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:04.912+0000 7f346a3868c0 1 > prepared operation key=C1BTVJNHIM0X tag=ARO9E3N3BIUK type=del size=2577373 2024-04-07T16:38:04.913 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:04.920+0000 7f346a3868c0 1 < completed delete operation key=QVOKSQAGHPA9 tag=TDKNVHMLB5PY type=del 2024-04-07T16:38:04.919 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:04.928+0000 7f346a3868c0 1 > prepared operation key=X5Q46GO8YTOK tag=6QTP20PL0P00 type=del size=2918329 2024-04-07T16:38:04.932 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:04.940+0000 7f346a3868c0 1 < suggested update operation key=95YVKJ14JD8W tag=50RTF8YAQICC type=del size=763377 2024-04-07T16:38:04.937 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:04.944+0000 7f346a3868c0 1 > prepared operation key=NYET3212USVE tag=LPC9VF6Z2QBN type=del size=3133391 2024-04-07T16:38:04.948 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:04.956+0000 7f346a3868c0 1 < suggested remove operation key=QS4HYZFFAJNF tag=3SQRX6Z1H0NZ type=del 2024-04-07T16:38:04.955 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:04.960+0000 7f346a3868c0 1 > prepared operation key=E84U0SX44NQ8 tag=SU7QMW1PF3OQ type=del size=1637411 2024-04-07T16:38:04.961 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:04.968+0000 7f346a3868c0 1 < completed part key=_multipart_YXMDZB23YTB3.2~SI5ZR289F6E6.0 size=1048576 2024-04-07T16:38:04.966 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:04.972+0000 7f346a3868c0 1 < completed part key=_multipart_YXMDZB23YTB3.2~SI5ZR289F6E6.1 size=779916 2024-04-07T16:38:04.975 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:04.984+0000 7f346a3868c0 1 < completed multipart upload key=YXMDZB23YTB3 upload=SI5ZR289F6E6 tag=U2K4XAWZ9S2W type=write size=2877068 2024-04-07T16:38:04.987 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:04.996+0000 7f346a3868c0 1 > prepared part key=_multipart_QTCIIPTXIRW2.2~NK3FAUDBRRRH.0 size=749908 2024-04-07T16:38:04.987 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:04.996+0000 7f346a3868c0 1 > prepared multipart upload key=QTCIIPTXIRW2 upload=NK3FAUDBRRRH tag=U8FCSV6PB5UY type=write size=1798484 2024-04-07T16:38:04.996 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.004+0000 7f346a3868c0 1 < completed part key=_multipart_YOK8BAEOMPUR.2~LLNN1DY2QKI6.0 size=1048576 2024-04-07T16:38:05.001 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.008+0000 7f346a3868c0 1 < completed part key=_multipart_YOK8BAEOMPUR.2~LLNN1DY2QKI6.1 size=1048576 2024-04-07T16:38:05.008 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.016+0000 7f346a3868c0 1 < completed part key=_multipart_YOK8BAEOMPUR.2~LLNN1DY2QKI6.2 size=523874 2024-04-07T16:38:05.018 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.024+0000 7f346a3868c0 1 < completed multipart upload key=YOK8BAEOMPUR upload=LLNN1DY2QKI6 tag=WCEQX2MBA02A type=write size=3669602 2024-04-07T16:38:05.028 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.036+0000 7f346a3868c0 1 > prepared part key=_multipart_CL4PP1RKZU0E.2~VX0SK4IRC62Y.0 size=1048576 2024-04-07T16:38:05.032 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.040+0000 7f346a3868c0 1 > prepared part key=_multipart_CL4PP1RKZU0E.2~VX0SK4IRC62Y.1 size=1048576 2024-04-07T16:38:05.037 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.044+0000 7f346a3868c0 1 > prepared part key=_multipart_CL4PP1RKZU0E.2~VX0SK4IRC62Y.2 size=575339 2024-04-07T16:38:05.037 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.044+0000 7f346a3868c0 1 > prepared multipart upload key=CL4PP1RKZU0E upload=VX0SK4IRC62Y tag=TX21HDMGUK78 type=write size=3721067 2024-04-07T16:38:05.042 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.048+0000 7f346a3868c0 1 < completed delete operation key=ROFH30W3UCAX tag=ADIAH1Y46NRK type=del 2024-04-07T16:38:05.050 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.056+0000 7f346a3868c0 1 > prepared operation key=N14BUXLJONHZ tag=JU5HFHSMRS02 type=del size=1329857 2024-04-07T16:38:05.057 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.064+0000 7f346a3868c0 1 < completed part key=_multipart_UIW3DN4NUYPP.2~UCXC4WHVB0FB.0 size=1048576 2024-04-07T16:38:05.061 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.068+0000 7f346a3868c0 1 < completed part key=_multipart_UIW3DN4NUYPP.2~UCXC4WHVB0FB.1 size=26439 2024-04-07T16:38:05.071 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.076+0000 7f346a3868c0 1 < completed multipart upload key=UIW3DN4NUYPP upload=UCXC4WHVB0FB tag=7U3SS0F5FNB5 type=write size=2123591 2024-04-07T16:38:05.083 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.092+0000 7f346a3868c0 1 > prepared part key=_multipart_N14BUXLJONHZ.2~ANUVVIX0I9JK.0 size=1048576 2024-04-07T16:38:05.088 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.096+0000 7f346a3868c0 1 > prepared part key=_multipart_N14BUXLJONHZ.2~ANUVVIX0I9JK.1 size=965321 2024-04-07T16:38:05.088 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.096+0000 7f346a3868c0 1 > prepared multipart upload key=N14BUXLJONHZ upload=ANUVVIX0I9JK tag=C1K0JHGW6CIU type=write size=3062473 2024-04-07T16:38:05.094 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.100+0000 7f346a3868c0 1 < completed part key=_multipart_QS4HYZFFAJNF.2~FKJBTM19WZEA.0 size=1048576 2024-04-07T16:38:05.101 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.108+0000 7f346a3868c0 1 < completed part key=_multipart_QS4HYZFFAJNF.2~FKJBTM19WZEA.1 size=667634 2024-04-07T16:38:05.111 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.116+0000 7f346a3868c0 1 < completed multipart upload key=QS4HYZFFAJNF upload=FKJBTM19WZEA tag=KK65F85XSNDD type=write size=2764786 2024-04-07T16:38:05.122 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.128+0000 7f346a3868c0 1 > prepared part key=_multipart_GBQSME7SY1AH.2~PSYNFC5U1R1Z.0 size=1048576 2024-04-07T16:38:05.128 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.136+0000 7f346a3868c0 1 > prepared part key=_multipart_GBQSME7SY1AH.2~PSYNFC5U1R1Z.1 size=1048576 2024-04-07T16:38:05.133 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.140+0000 7f346a3868c0 1 > prepared part key=_multipart_GBQSME7SY1AH.2~PSYNFC5U1R1Z.2 size=274805 2024-04-07T16:38:05.133 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.140+0000 7f346a3868c0 1 > prepared multipart upload key=GBQSME7SY1AH upload=PSYNFC5U1R1Z tag=KBAZXLOG1FOG type=write size=3420533 2024-04-07T16:38:05.141 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.148+0000 7f346a3868c0 -1 < failed to complete operation key=QS4HYZFFAJNF tag=Q7FNUEH57XIV type=write size=165609: (22) Invalid argument 2024-04-07T16:38:05.147 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.156+0000 7f346a3868c0 1 > prepared operation key=1WGUPH7X5UA0 tag=GXH665KHNR9I type=del size=3809502 2024-04-07T16:38:05.169 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.176+0000 7f346a3868c0 1 < suggested remove operation key=U3EID5EK64HD tag=QO3WUTD4WLN1 type=write 2024-04-07T16:38:05.177 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.184+0000 7f346a3868c0 1 > prepared operation key=QTCIIPTXIRW2 tag=EQLIIBQQPZUD type=del size=585307 2024-04-07T16:38:05.183 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.188+0000 7f346a3868c0 1 < completed delete operation key=EOYFVEI50SQ1 tag=GLE0K67FXO1B type=del 2024-04-07T16:38:05.192 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.200+0000 7f346a3868c0 1 > prepared part key=_multipart_NYET3212USVE.2~9CDR3ARN5YD2.0 size=1007319 2024-04-07T16:38:05.192 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.200+0000 7f346a3868c0 1 > prepared multipart upload key=NYET3212USVE upload=9CDR3ARN5YD2 tag=NY6WA9PRTWWF type=write size=2055895 2024-04-07T16:38:05.198 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.204+0000 7f346a3868c0 1 < canceled operation key=CL4PP1RKZU0E tag=XT114UBV1MGL type=del size=4141387 2024-04-07T16:38:05.208 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.216+0000 7f346a3868c0 1 > prepared part key=_multipart_N14BUXLJONHZ.2~Y5AOUN2G1SHT.0 size=1048576 2024-04-07T16:38:05.213 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.220+0000 7f346a3868c0 1 > prepared part key=_multipart_N14BUXLJONHZ.2~Y5AOUN2G1SHT.1 size=680162 2024-04-07T16:38:05.213 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.220+0000 7f346a3868c0 1 > prepared multipart upload key=N14BUXLJONHZ upload=Y5AOUN2G1SHT tag=6OUH2EVETH6O type=write size=2777314 2024-04-07T16:38:05.224 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.232+0000 7f346a3868c0 1 < suggested remove operation key=E84U0SX44NQ8 tag=3DQAPCIN8HLV type=del 2024-04-07T16:38:05.230 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.236+0000 7f346a3868c0 1 > prepared operation key=3N0QJRYVO3FV tag=BSVDMBH7GEVX type=del size=1434358 2024-04-07T16:38:05.236 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.244+0000 7f346a3868c0 1 < completed delete operation key=N14BUXLJONHZ tag=WON4HGXNWUK1 type=del 2024-04-07T16:38:05.247 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.256+0000 7f346a3868c0 1 > prepared part key=_multipart_95YVKJ14JD8W.2~320FQ3Z1ZVO4.0 size=1048576 2024-04-07T16:38:05.252 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.260+0000 7f346a3868c0 1 > prepared part key=_multipart_95YVKJ14JD8W.2~320FQ3Z1ZVO4.1 size=932602 2024-04-07T16:38:05.252 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.260+0000 7f346a3868c0 1 > prepared multipart upload key=95YVKJ14JD8W upload=320FQ3Z1ZVO4 tag=9VCEL4S5DUGH type=write size=3029754 2024-04-07T16:38:05.258 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.264+0000 7f346a3868c0 1 < completed write operation key=NYET3212USVE tag=UB4J8O0OSMEM type=write size=822079 2024-04-07T16:38:05.264 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.272+0000 7f346a3868c0 1 > prepared operation key=HBLEXONKXJTL tag=QS26NDLXE4W7 type=write size=480707 2024-04-07T16:38:05.271 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.280+0000 7f346a3868c0 1 < completed delete operation key=C1BTVJNHIM0X tag=ARO9E3N3BIUK type=del 2024-04-07T16:38:05.277 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.284+0000 7f346a3868c0 1 > prepared operation key=DXTO3AJ429AC tag=1JVXZ4CF23LA type=del size=1434971 2024-04-07T16:38:05.283 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.292+0000 7f346a3868c0 1 < completed delete operation key=X5Q46GO8YTOK tag=6QTP20PL0P00 type=del 2024-04-07T16:38:05.290 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.296+0000 7f346a3868c0 1 > prepared operation key=DXTO3AJ429AC tag=WFU5PEQELY62 type=del size=74894 2024-04-07T16:38:05.295 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.304+0000 7f346a3868c0 1 < completed delete operation key=NYET3212USVE tag=LPC9VF6Z2QBN type=del 2024-04-07T16:38:05.301 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.308+0000 7f346a3868c0 1 > prepared operation key=N14BUXLJONHZ tag=GLMXO9PYNJ14 type=del size=765838 2024-04-07T16:38:05.309 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.316+0000 7f346a3868c0 -1 < failed to complete operation key=E84U0SX44NQ8 tag=SU7QMW1PF3OQ type=del size=1637411: (22) Invalid argument 2024-04-07T16:38:05.315 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.320+0000 7f346a3868c0 1 > prepared operation key=QTCIIPTXIRW2 tag=MM2CB830UG66 type=write size=532623 2024-04-07T16:38:05.321 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.328+0000 7f346a3868c0 1 < completed part key=_multipart_QTCIIPTXIRW2.2~NK3FAUDBRRRH.0 size=749908 2024-04-07T16:38:05.331 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.340+0000 7f346a3868c0 1 < completed multipart upload key=QTCIIPTXIRW2 upload=NK3FAUDBRRRH tag=U8FCSV6PB5UY type=write size=1798484 2024-04-07T16:38:05.337 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.344+0000 7f346a3868c0 1 > prepared operation key=EOYFVEI50SQ1 tag=NL0KRJ4I86ZV type=del size=3346437 2024-04-07T16:38:05.345 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.352+0000 7f346a3868c0 1 < completed part key=_multipart_CL4PP1RKZU0E.2~VX0SK4IRC62Y.0 size=1048576 2024-04-07T16:38:05.350 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.356+0000 7f346a3868c0 1 < completed part key=_multipart_CL4PP1RKZU0E.2~VX0SK4IRC62Y.1 size=1048576 2024-04-07T16:38:05.355 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.360+0000 7f346a3868c0 1 < completed part key=_multipart_CL4PP1RKZU0E.2~VX0SK4IRC62Y.2 size=575339 2024-04-07T16:38:05.365 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.372+0000 7f346a3868c0 1 < completed multipart upload key=CL4PP1RKZU0E upload=VX0SK4IRC62Y tag=TX21HDMGUK78 type=write size=3721067 2024-04-07T16:38:05.376 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.384+0000 7f346a3868c0 1 > prepared part key=_multipart_DXTO3AJ429AC.2~58A80VGJVEFD.0 size=1048576 2024-04-07T16:38:05.381 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.388+0000 7f346a3868c0 1 > prepared part key=_multipart_DXTO3AJ429AC.2~58A80VGJVEFD.1 size=1048576 2024-04-07T16:38:05.386 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.392+0000 7f346a3868c0 1 > prepared part key=_multipart_DXTO3AJ429AC.2~58A80VGJVEFD.2 size=398661 2024-04-07T16:38:05.386 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.392+0000 7f346a3868c0 1 > prepared multipart upload key=DXTO3AJ429AC upload=58A80VGJVEFD tag=RPVO832AKPOT type=write size=3544389 2024-04-07T16:38:05.392 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.400+0000 7f346a3868c0 1 < completed delete operation key=N14BUXLJONHZ tag=JU5HFHSMRS02 type=del 2024-04-07T16:38:05.402 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.408+0000 7f346a3868c0 1 > prepared part key=_multipart_IP8YW2A75TCW.2~LOC8LV7AD0PI.0 size=999502 2024-04-07T16:38:05.402 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.408+0000 7f346a3868c0 1 > prepared multipart upload key=IP8YW2A75TCW upload=LOC8LV7AD0PI tag=F2EMSXGLPI6Y type=write size=2048078 2024-04-07T16:38:05.408 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.416+0000 7f346a3868c0 1 < completed part key=_multipart_N14BUXLJONHZ.2~ANUVVIX0I9JK.0 size=1048576 2024-04-07T16:38:05.413 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.420+0000 7f346a3868c0 1 < completed part key=_multipart_N14BUXLJONHZ.2~ANUVVIX0I9JK.1 size=965321 2024-04-07T16:38:05.423 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.432+0000 7f346a3868c0 1 < completed multipart upload key=N14BUXLJONHZ upload=ANUVVIX0I9JK tag=C1K0JHGW6CIU type=write size=3062473 2024-04-07T16:38:05.430 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.436+0000 7f346a3868c0 1 > prepared operation key=6XDS0A7L0IY8 tag=DZVDWLNVB1L9 type=del size=949249 2024-04-07T16:38:05.437 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.444+0000 7f346a3868c0 1 < completed part key=_multipart_GBQSME7SY1AH.2~PSYNFC5U1R1Z.0 size=1048576 2024-04-07T16:38:05.448 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.456+0000 7f346a3868c0 1 < completed part key=_multipart_GBQSME7SY1AH.2~PSYNFC5U1R1Z.1 size=1048576 2024-04-07T16:38:05.453 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.460+0000 7f346a3868c0 1 < completed part key=_multipart_GBQSME7SY1AH.2~PSYNFC5U1R1Z.2 size=274805 2024-04-07T16:38:05.464 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.472+0000 7f346a3868c0 1 < completed multipart upload key=GBQSME7SY1AH upload=PSYNFC5U1R1Z tag=KBAZXLOG1FOG type=write size=3420533 2024-04-07T16:38:05.471 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.476+0000 7f346a3868c0 1 > prepared operation key=YOK8BAEOMPUR tag=FVIJ83KNUH6R type=write size=857453 2024-04-07T16:38:05.483 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.492+0000 7f346a3868c0 1 < suggested update operation key=1WGUPH7X5UA0 tag=GXH665KHNR9I type=del size=3809502 2024-04-07T16:38:05.494 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.500+0000 7f346a3868c0 1 > prepared part key=_multipart_XJLQZ7IS9U1T.2~CU48YZQPKYSK.0 size=1048576 2024-04-07T16:38:05.499 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.504+0000 7f346a3868c0 1 > prepared part key=_multipart_XJLQZ7IS9U1T.2~CU48YZQPKYSK.1 size=1048576 2024-04-07T16:38:05.504 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.512+0000 7f346a3868c0 1 > prepared part key=_multipart_XJLQZ7IS9U1T.2~CU48YZQPKYSK.2 size=393562 2024-04-07T16:38:05.504 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.512+0000 7f346a3868c0 1 > prepared multipart upload key=XJLQZ7IS9U1T upload=CU48YZQPKYSK tag=RUXZPIBFMXBE type=write size=3539290 2024-04-07T16:38:05.509 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.516+0000 7f346a3868c0 1 < completed delete operation key=QTCIIPTXIRW2 tag=EQLIIBQQPZUD type=del 2024-04-07T16:38:05.515 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.524+0000 7f346a3868c0 1 > prepared operation key=QS4HYZFFAJNF tag=C2OFT7TIU79N type=del size=3212534 2024-04-07T16:38:05.521 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.528+0000 7f346a3868c0 1 < completed part key=_multipart_NYET3212USVE.2~9CDR3ARN5YD2.0 size=1007319 2024-04-07T16:38:05.530 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.536+0000 7f346a3868c0 1 < completed multipart upload key=NYET3212USVE upload=9CDR3ARN5YD2 tag=NY6WA9PRTWWF type=write size=2055895 2024-04-07T16:38:05.537 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.544+0000 7f346a3868c0 1 > prepared operation key=U3EID5EK64HD tag=CB5HBCDODKG0 type=write size=611845 2024-04-07T16:38:05.541 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.548+0000 7f346a3868c0 1 < completed part key=_multipart_N14BUXLJONHZ.2~Y5AOUN2G1SHT.0 size=1048576 2024-04-07T16:38:05.546 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.552+0000 7f346a3868c0 1 < completed part key=_multipart_N14BUXLJONHZ.2~Y5AOUN2G1SHT.1 size=680162 2024-04-07T16:38:05.556 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.564+0000 7f346a3868c0 1 < completed multipart upload key=N14BUXLJONHZ upload=Y5AOUN2G1SHT tag=6OUH2EVETH6O type=write size=2777314 2024-04-07T16:38:05.567 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.576+0000 7f346a3868c0 1 > prepared part key=_multipart_NYET3212USVE.2~GEC03923V0TM.0 size=1048576 2024-04-07T16:38:05.573 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.580+0000 7f346a3868c0 1 > prepared part key=_multipart_NYET3212USVE.2~GEC03923V0TM.1 size=1048576 2024-04-07T16:38:05.578 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.584+0000 7f346a3868c0 1 > prepared part key=_multipart_NYET3212USVE.2~GEC03923V0TM.2 size=341410 2024-04-07T16:38:05.578 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.584+0000 7f346a3868c0 1 > prepared multipart upload key=NYET3212USVE upload=GEC03923V0TM tag=BE470EPDZKLF type=write size=3487138 2024-04-07T16:38:05.584 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.592+0000 7f346a3868c0 1 < canceled operation key=3N0QJRYVO3FV tag=BSVDMBH7GEVX type=del size=1434358 2024-04-07T16:38:05.594 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.600+0000 7f346a3868c0 1 > prepared part key=_multipart_6XDS0A7L0IY8.2~NGDP7AZLFH2D.0 size=1048576 2024-04-07T16:38:05.599 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.604+0000 7f346a3868c0 1 > prepared part key=_multipart_6XDS0A7L0IY8.2~NGDP7AZLFH2D.1 size=1048576 2024-04-07T16:38:05.604 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.612+0000 7f346a3868c0 1 > prepared part key=_multipart_6XDS0A7L0IY8.2~NGDP7AZLFH2D.2 size=224241 2024-04-07T16:38:05.604 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.612+0000 7f346a3868c0 1 > prepared multipart upload key=6XDS0A7L0IY8 upload=NGDP7AZLFH2D tag=M47R2YJ8AJNC type=write size=3369969 2024-04-07T16:38:05.609 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.616+0000 7f346a3868c0 1 < completed part key=_multipart_95YVKJ14JD8W.2~320FQ3Z1ZVO4.0 size=1048576 2024-04-07T16:38:05.614 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.620+0000 7f346a3868c0 1 < completed part key=_multipart_95YVKJ14JD8W.2~320FQ3Z1ZVO4.1 size=932602 2024-04-07T16:38:05.625 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.632+0000 7f346a3868c0 1 < completed multipart upload key=95YVKJ14JD8W upload=320FQ3Z1ZVO4 tag=9VCEL4S5DUGH type=write size=3029754 2024-04-07T16:38:05.635 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.644+0000 7f346a3868c0 1 > prepared part key=_multipart_LXOZ0T2KY6OR.2~KQQPA3J242C5.0 size=1048576 2024-04-07T16:38:05.640 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.648+0000 7f346a3868c0 1 > prepared part key=_multipart_LXOZ0T2KY6OR.2~KQQPA3J242C5.1 size=1048576 2024-04-07T16:38:05.644 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.652+0000 7f346a3868c0 1 > prepared part key=_multipart_LXOZ0T2KY6OR.2~KQQPA3J242C5.2 size=392741 2024-04-07T16:38:05.644 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.652+0000 7f346a3868c0 1 > prepared multipart upload key=LXOZ0T2KY6OR upload=KQQPA3J242C5 tag=2S8LQOGHJD32 type=write size=3538469 2024-04-07T16:38:05.650 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.656+0000 7f346a3868c0 1 < completed write operation key=HBLEXONKXJTL tag=QS26NDLXE4W7 type=write size=480707 2024-04-07T16:38:05.660 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.668+0000 7f346a3868c0 1 > prepared part key=_multipart_GBQSME7SY1AH.2~P2WUUWR772E3.0 size=1048576 2024-04-07T16:38:05.665 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.672+0000 7f346a3868c0 1 > prepared part key=_multipart_GBQSME7SY1AH.2~P2WUUWR772E3.1 size=1048576 2024-04-07T16:38:05.670 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.676+0000 7f346a3868c0 1 > prepared part key=_multipart_GBQSME7SY1AH.2~P2WUUWR772E3.2 size=880523 2024-04-07T16:38:05.670 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.676+0000 7f346a3868c0 1 > prepared multipart upload key=GBQSME7SY1AH upload=P2WUUWR772E3 tag=3Y32LF5QHOVH type=write size=4026251 2024-04-07T16:38:05.676 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.684+0000 7f346a3868c0 1 < canceled operation key=DXTO3AJ429AC tag=1JVXZ4CF23LA type=del size=1434971 2024-04-07T16:38:05.681 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.688+0000 7f346a3868c0 1 > prepared operation key=CL4PP1RKZU0E tag=3HWN2HD93XF9 type=del size=1972374 2024-04-07T16:38:05.687 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.696+0000 7f346a3868c0 1 < canceled operation key=DXTO3AJ429AC tag=WFU5PEQELY62 type=del size=74894 2024-04-07T16:38:05.692 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.700+0000 7f346a3868c0 1 > prepared operation key=GZXHYZO5SMUP tag=5TK0LSO2T74H type=del size=1717421 2024-04-07T16:38:05.698 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.704+0000 7f346a3868c0 1 < completed delete operation key=N14BUXLJONHZ tag=GLMXO9PYNJ14 type=del 2024-04-07T16:38:05.705 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.712+0000 7f346a3868c0 1 > prepared operation key=UIW3DN4NUYPP tag=UVXZDQAE16RB type=del size=2971179 2024-04-07T16:38:05.712 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.720+0000 7f346a3868c0 1 < completed write operation key=QTCIIPTXIRW2 tag=MM2CB830UG66 type=write size=532623 2024-04-07T16:38:05.718 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.724+0000 7f346a3868c0 1 > prepared operation key=ROFH30W3UCAX tag=RZ2OWX6PVT12 type=del size=3913442 2024-04-07T16:38:05.724 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.732+0000 7f346a3868c0 1 < completed delete operation key=EOYFVEI50SQ1 tag=NL0KRJ4I86ZV type=del 2024-04-07T16:38:05.736 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.744+0000 7f346a3868c0 1 > prepared part key=_multipart_8F81G0P0YBFW.2~Q44L97QP88JG.0 size=1048576 2024-04-07T16:38:05.741 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.748+0000 7f346a3868c0 1 > prepared part key=_multipart_8F81G0P0YBFW.2~Q44L97QP88JG.1 size=855407 2024-04-07T16:38:05.742 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.748+0000 7f346a3868c0 1 > prepared multipart upload key=8F81G0P0YBFW upload=Q44L97QP88JG tag=KKSPQFV0R5Z1 type=write size=2952559 2024-04-07T16:38:05.742 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.748+0000 7f346a3868c0 1 < canceled part key=_multipart_DXTO3AJ429AC.2~58A80VGJVEFD.0 size=1048576 2024-04-07T16:38:05.748 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.756+0000 7f346a3868c0 1 < completed part key=_multipart_DXTO3AJ429AC.2~58A80VGJVEFD.1 size=1048576 2024-04-07T16:38:05.753 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.760+0000 7f346a3868c0 1 < completed part key=_multipart_DXTO3AJ429AC.2~58A80VGJVEFD.2 size=398661 2024-04-07T16:38:05.764 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.772+0000 7f346a3868c0 1 < canceled multipart upload key=DXTO3AJ429AC upload=58A80VGJVEFD tag=RPVO832AKPOT type=write size=3544389 2024-04-07T16:38:05.774 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.780+0000 7f346a3868c0 1 > prepared part key=_multipart_LXOZ0T2KY6OR.2~IQ4HDYUXJZX7.0 size=1048576 2024-04-07T16:38:05.779 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.784+0000 7f346a3868c0 1 > prepared part key=_multipart_LXOZ0T2KY6OR.2~IQ4HDYUXJZX7.1 size=967403 2024-04-07T16:38:05.779 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.784+0000 7f346a3868c0 1 > prepared multipart upload key=LXOZ0T2KY6OR upload=IQ4HDYUXJZX7 tag=PFZNES0TTXDL type=write size=3064555 2024-04-07T16:38:05.784 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.792+0000 7f346a3868c0 1 < completed part key=_multipart_IP8YW2A75TCW.2~LOC8LV7AD0PI.0 size=999502 2024-04-07T16:38:05.793 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.800+0000 7f346a3868c0 1 < completed multipart upload key=IP8YW2A75TCW upload=LOC8LV7AD0PI tag=F2EMSXGLPI6Y type=write size=2048078 2024-04-07T16:38:05.798 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.804+0000 7f346a3868c0 1 > prepared operation key=GZXHYZO5SMUP tag=571435A8VU3J type=del size=2747105 2024-04-07T16:38:05.803 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.812+0000 7f346a3868c0 1 < completed delete operation key=6XDS0A7L0IY8 tag=DZVDWLNVB1L9 type=del 2024-04-07T16:38:05.813 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.820+0000 7f346a3868c0 1 > prepared part key=_multipart_1VKVQXSQMTU1.2~SYRMNDPHFYET.0 size=1048576 2024-04-07T16:38:05.818 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.824+0000 7f346a3868c0 1 > prepared part key=_multipart_1VKVQXSQMTU1.2~SYRMNDPHFYET.1 size=1048576 2024-04-07T16:38:05.823 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.832+0000 7f346a3868c0 1 > prepared part key=_multipart_1VKVQXSQMTU1.2~SYRMNDPHFYET.2 size=929143 2024-04-07T16:38:05.824 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.832+0000 7f346a3868c0 1 > prepared multipart upload key=1VKVQXSQMTU1 upload=SYRMNDPHFYET tag=7E1KDIMW6AWJ type=write size=4074871 2024-04-07T16:38:05.829 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.836+0000 7f346a3868c0 1 < completed write operation key=YOK8BAEOMPUR tag=FVIJ83KNUH6R type=write size=857453 2024-04-07T16:38:05.835 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.844+0000 7f346a3868c0 1 > prepared operation key=GZXHYZO5SMUP tag=F0KZU2X783YO type=del size=1151822 2024-04-07T16:38:05.841 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.848+0000 7f346a3868c0 1 < completed part key=_multipart_XJLQZ7IS9U1T.2~CU48YZQPKYSK.0 size=1048576 2024-04-07T16:38:05.846 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.852+0000 7f346a3868c0 1 < completed part key=_multipart_XJLQZ7IS9U1T.2~CU48YZQPKYSK.1 size=1048576 2024-04-07T16:38:05.851 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.860+0000 7f346a3868c0 1 < completed part key=_multipart_XJLQZ7IS9U1T.2~CU48YZQPKYSK.2 size=393562 2024-04-07T16:38:05.861 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.868+0000 7f346a3868c0 1 < completed multipart upload key=XJLQZ7IS9U1T upload=CU48YZQPKYSK tag=RUXZPIBFMXBE type=write size=3539290 2024-04-07T16:38:05.867 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.876+0000 7f346a3868c0 1 > prepared operation key=3N0QJRYVO3FV tag=XK738MFRQ3CW type=del size=3794053 2024-04-07T16:38:05.873 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.880+0000 7f346a3868c0 1 < completed delete operation key=QS4HYZFFAJNF tag=C2OFT7TIU79N type=del 2024-04-07T16:38:05.879 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.884+0000 7f346a3868c0 1 > prepared operation key=1WGUPH7X5UA0 tag=RLCOPLW2FK3H type=del size=2513845 2024-04-07T16:38:05.885 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.892+0000 7f346a3868c0 1 < completed write operation key=U3EID5EK64HD tag=CB5HBCDODKG0 type=write size=611845 2024-04-07T16:38:05.893 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.900+0000 7f346a3868c0 1 > prepared operation key=IX3F0IUOKBX1 tag=7D7PTUM933TA type=del size=3872107 2024-04-07T16:38:05.898 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.904+0000 7f346a3868c0 1 < completed part key=_multipart_NYET3212USVE.2~GEC03923V0TM.0 size=1048576 2024-04-07T16:38:05.903 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.912+0000 7f346a3868c0 1 < completed part key=_multipart_NYET3212USVE.2~GEC03923V0TM.1 size=1048576 2024-04-07T16:38:05.908 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.916+0000 7f346a3868c0 1 < completed part key=_multipart_NYET3212USVE.2~GEC03923V0TM.2 size=341410 2024-04-07T16:38:05.918 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.924+0000 7f346a3868c0 1 < completed multipart upload key=NYET3212USVE upload=GEC03923V0TM tag=BE470EPDZKLF type=write size=3487138 2024-04-07T16:38:05.929 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.936+0000 7f346a3868c0 1 > prepared part key=_multipart_1VKVQXSQMTU1.2~S9RJP86J2V7N.0 size=1048576 2024-04-07T16:38:05.934 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.940+0000 7f346a3868c0 1 > prepared part key=_multipart_1VKVQXSQMTU1.2~S9RJP86J2V7N.1 size=10046 2024-04-07T16:38:05.934 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.940+0000 7f346a3868c0 1 > prepared multipart upload key=1VKVQXSQMTU1 upload=S9RJP86J2V7N tag=QGBT0CKDJVNZ type=write size=2107198 2024-04-07T16:38:05.939 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.948+0000 7f346a3868c0 1 < completed part key=_multipart_6XDS0A7L0IY8.2~NGDP7AZLFH2D.0 size=1048576 2024-04-07T16:38:05.944 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.952+0000 7f346a3868c0 1 < completed part key=_multipart_6XDS0A7L0IY8.2~NGDP7AZLFH2D.1 size=1048576 2024-04-07T16:38:05.949 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.956+0000 7f346a3868c0 1 < completed part key=_multipart_6XDS0A7L0IY8.2~NGDP7AZLFH2D.2 size=224241 2024-04-07T16:38:05.959 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.968+0000 7f346a3868c0 1 < completed multipart upload key=6XDS0A7L0IY8 upload=NGDP7AZLFH2D tag=M47R2YJ8AJNC type=write size=3369969 2024-04-07T16:38:05.970 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.976+0000 7f346a3868c0 1 > prepared part key=_multipart_3N0QJRYVO3FV.2~YIV2IP5F4XM3.0 size=232199 2024-04-07T16:38:05.971 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.976+0000 7f346a3868c0 1 > prepared multipart upload key=3N0QJRYVO3FV upload=YIV2IP5F4XM3 tag=WDG7KL0M7NGQ type=write size=1280775 2024-04-07T16:38:05.977 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.984+0000 7f346a3868c0 1 < completed part key=_multipart_LXOZ0T2KY6OR.2~KQQPA3J242C5.0 size=1048576 2024-04-07T16:38:05.982 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.988+0000 7f346a3868c0 1 < completed part key=_multipart_LXOZ0T2KY6OR.2~KQQPA3J242C5.1 size=1048576 2024-04-07T16:38:05.987 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:05.996+0000 7f346a3868c0 1 < completed part key=_multipart_LXOZ0T2KY6OR.2~KQQPA3J242C5.2 size=392741 2024-04-07T16:38:05.997 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.004+0000 7f346a3868c0 1 < completed multipart upload key=LXOZ0T2KY6OR upload=KQQPA3J242C5 tag=2S8LQOGHJD32 type=write size=3538469 2024-04-07T16:38:06.002 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.008+0000 7f346a3868c0 1 > prepared operation key=QTCIIPTXIRW2 tag=PLX1WG1SU6JX type=del size=3635506 2024-04-07T16:38:06.010 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.016+0000 7f346a3868c0 1 < completed part key=_multipart_GBQSME7SY1AH.2~P2WUUWR772E3.0 size=1048576 2024-04-07T16:38:06.010 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.016+0000 7f346a3868c0 1 < canceled part key=_multipart_GBQSME7SY1AH.2~P2WUUWR772E3.1 size=1048576 2024-04-07T16:38:06.010 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.016+0000 7f346a3868c0 1 < canceled part key=_multipart_GBQSME7SY1AH.2~P2WUUWR772E3.2 size=880523 2024-04-07T16:38:06.020 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.028+0000 7f346a3868c0 1 < canceled multipart upload key=GBQSME7SY1AH upload=P2WUUWR772E3 tag=3Y32LF5QHOVH type=write size=4026251 2024-04-07T16:38:06.025 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.032+0000 7f346a3868c0 1 > prepared operation key=C1BTVJNHIM0X tag=Z57VM1NO4NU4 type=del size=3290620 2024-04-07T16:38:06.031 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.036+0000 7f346a3868c0 1 < canceled operation key=CL4PP1RKZU0E tag=3HWN2HD93XF9 type=del size=1972374 2024-04-07T16:38:06.041 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.048+0000 7f346a3868c0 1 > prepared part key=_multipart_6XDS0A7L0IY8.2~ZWF3JYJ728UG.0 size=99936 2024-04-07T16:38:06.042 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.048+0000 7f346a3868c0 1 > prepared multipart upload key=6XDS0A7L0IY8 upload=ZWF3JYJ728UG tag=2L78O4MTF7EX type=write size=1148512 2024-04-07T16:38:06.052 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.060+0000 7f346a3868c0 1 < suggested remove operation key=GZXHYZO5SMUP tag=5TK0LSO2T74H type=del 2024-04-07T16:38:06.058 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.064+0000 7f346a3868c0 1 > prepared operation key=6XDS0A7L0IY8 tag=D9B5XNTBNKIU type=write size=334286 2024-04-07T16:38:06.064 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.072+0000 7f346a3868c0 1 < completed delete operation key=UIW3DN4NUYPP tag=UVXZDQAE16RB type=del 2024-04-07T16:38:06.071 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.080+0000 7f346a3868c0 1 > prepared operation key=EOYFVEI50SQ1 tag=MPHB0P1TUGWQ type=del size=3952848 2024-04-07T16:38:06.077 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.084+0000 7f346a3868c0 1 < completed delete operation key=ROFH30W3UCAX tag=RZ2OWX6PVT12 type=del 2024-04-07T16:38:06.088 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.096+0000 7f346a3868c0 1 > prepared part key=_multipart_X5Q46GO8YTOK.2~6GQUON3PQR4K.0 size=1048576 2024-04-07T16:38:06.092 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.100+0000 7f346a3868c0 1 > prepared part key=_multipart_X5Q46GO8YTOK.2~6GQUON3PQR4K.1 size=277430 2024-04-07T16:38:06.093 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.100+0000 7f346a3868c0 1 > prepared multipart upload key=X5Q46GO8YTOK upload=6GQUON3PQR4K tag=RI7VJB9Y9EZY type=write size=2374582 2024-04-07T16:38:06.098 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.104+0000 7f346a3868c0 1 < completed part key=_multipart_8F81G0P0YBFW.2~Q44L97QP88JG.0 size=1048576 2024-04-07T16:38:06.103 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.108+0000 7f346a3868c0 1 < completed part key=_multipart_8F81G0P0YBFW.2~Q44L97QP88JG.1 size=855407 2024-04-07T16:38:06.112 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.120+0000 7f346a3868c0 1 < completed multipart upload key=8F81G0P0YBFW upload=Q44L97QP88JG tag=KKSPQFV0R5Z1 type=write size=2952559 2024-04-07T16:38:06.118 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.124+0000 7f346a3868c0 1 > prepared operation key=6XDS0A7L0IY8 tag=7RZO6INQSPE2 type=del size=2738784 2024-04-07T16:38:06.125 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.132+0000 7f346a3868c0 1 < completed part key=_multipart_LXOZ0T2KY6OR.2~IQ4HDYUXJZX7.0 size=1048576 2024-04-07T16:38:06.135 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.144+0000 7f346a3868c0 1 < completed part key=_multipart_LXOZ0T2KY6OR.2~IQ4HDYUXJZX7.1 size=967403 2024-04-07T16:38:06.147 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.152+0000 7f346a3868c0 1 < completed multipart upload key=LXOZ0T2KY6OR upload=IQ4HDYUXJZX7 tag=PFZNES0TTXDL type=write size=3064555 2024-04-07T16:38:06.153 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.160+0000 7f346a3868c0 1 > prepared operation key=YOK8BAEOMPUR tag=X6R1E3A8G48C type=del size=3710781 2024-04-07T16:38:06.162 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.168+0000 7f346a3868c0 -1 < failed to complete operation key=GZXHYZO5SMUP tag=571435A8VU3J type=del size=2747105: (22) Invalid argument 2024-04-07T16:38:06.168 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.176+0000 7f346a3868c0 1 > prepared operation key=8F81G0P0YBFW tag=CTVGL8NWZ9TX type=write size=144470 2024-04-07T16:38:06.175 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.180+0000 7f346a3868c0 1 < completed part key=_multipart_1VKVQXSQMTU1.2~SYRMNDPHFYET.0 size=1048576 2024-04-07T16:38:06.180 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.188+0000 7f346a3868c0 1 < completed part key=_multipart_1VKVQXSQMTU1.2~SYRMNDPHFYET.1 size=1048576 2024-04-07T16:38:06.185 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.192+0000 7f346a3868c0 1 < completed part key=_multipart_1VKVQXSQMTU1.2~SYRMNDPHFYET.2 size=929143 2024-04-07T16:38:06.196 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.204+0000 7f346a3868c0 1 < completed multipart upload key=1VKVQXSQMTU1 upload=SYRMNDPHFYET tag=7E1KDIMW6AWJ type=write size=4074871 2024-04-07T16:38:06.208 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.216+0000 7f346a3868c0 1 > prepared part key=_multipart_QS4HYZFFAJNF.2~OVMIVBXVVUM8.0 size=1048576 2024-04-07T16:38:06.213 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.220+0000 7f346a3868c0 1 > prepared part key=_multipart_QS4HYZFFAJNF.2~OVMIVBXVVUM8.1 size=1048576 2024-04-07T16:38:06.217 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.224+0000 7f346a3868c0 1 > prepared part key=_multipart_QS4HYZFFAJNF.2~OVMIVBXVVUM8.2 size=785402 2024-04-07T16:38:06.217 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.224+0000 7f346a3868c0 1 > prepared multipart upload key=QS4HYZFFAJNF upload=OVMIVBXVVUM8 tag=MQ4XZ3VR5HY4 type=write size=3931130 2024-04-07T16:38:06.225 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.232+0000 7f346a3868c0 -1 < failed to complete operation key=GZXHYZO5SMUP tag=F0KZU2X783YO type=del size=1151822: (22) Invalid argument 2024-04-07T16:38:06.236 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.244+0000 7f346a3868c0 1 > prepared part key=_multipart_CL4PP1RKZU0E.2~1JQLBE7IBNMA.0 size=1048576 2024-04-07T16:38:06.242 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.248+0000 7f346a3868c0 1 > prepared part key=_multipart_CL4PP1RKZU0E.2~1JQLBE7IBNMA.1 size=1048576 2024-04-07T16:38:06.248 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.256+0000 7f346a3868c0 1 > prepared part key=_multipart_CL4PP1RKZU0E.2~1JQLBE7IBNMA.2 size=434809 2024-04-07T16:38:06.248 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.256+0000 7f346a3868c0 1 > prepared multipart upload key=CL4PP1RKZU0E upload=1JQLBE7IBNMA tag=JMJ2ZO3SH924 type=write size=3580537 2024-04-07T16:38:06.254 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.260+0000 7f346a3868c0 1 < completed delete operation key=3N0QJRYVO3FV tag=XK738MFRQ3CW type=del 2024-04-07T16:38:06.260 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.268+0000 7f346a3868c0 1 > prepared operation key=LXOZ0T2KY6OR tag=3C37BFXTWHPO type=del size=2909907 2024-04-07T16:38:06.267 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.276+0000 7f346a3868c0 1 < completed delete operation key=1WGUPH7X5UA0 tag=RLCOPLW2FK3H type=del 2024-04-07T16:38:06.273 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.280+0000 7f346a3868c0 1 > prepared operation key=N14BUXLJONHZ tag=WP8XD0OWB555 type=del size=3033346 2024-04-07T16:38:06.286 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.292+0000 7f346a3868c0 1 < suggested update operation key=IX3F0IUOKBX1 tag=7D7PTUM933TA type=del size=3872107 2024-04-07T16:38:06.292 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.300+0000 7f346a3868c0 1 > prepared operation key=QS4HYZFFAJNF tag=LWWNTMWYTFPL type=del size=3872737 2024-04-07T16:38:06.299 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.308+0000 7f346a3868c0 1 < completed part key=_multipart_1VKVQXSQMTU1.2~S9RJP86J2V7N.0 size=1048576 2024-04-07T16:38:06.305 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.312+0000 7f346a3868c0 1 < completed part key=_multipart_1VKVQXSQMTU1.2~S9RJP86J2V7N.1 size=10046 2024-04-07T16:38:06.316 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.324+0000 7f346a3868c0 1 < completed multipart upload key=1VKVQXSQMTU1 upload=S9RJP86J2V7N tag=QGBT0CKDJVNZ type=write size=2107198 2024-04-07T16:38:06.321 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.328+0000 7f346a3868c0 1 > prepared operation key=1VBSUWNI3V1T tag=KI6CKNSSF6UR type=del size=4118856 2024-04-07T16:38:06.327 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.332+0000 7f346a3868c0 1 < completed part key=_multipart_3N0QJRYVO3FV.2~YIV2IP5F4XM3.0 size=232199 2024-04-07T16:38:06.339 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.344+0000 7f346a3868c0 1 < completed multipart upload key=3N0QJRYVO3FV upload=YIV2IP5F4XM3 tag=WDG7KL0M7NGQ type=write size=1280775 2024-04-07T16:38:06.345 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.352+0000 7f346a3868c0 1 > prepared operation key=GZXHYZO5SMUP tag=7PSFX16ZYE8P type=write size=221428 2024-04-07T16:38:06.351 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.356+0000 7f346a3868c0 1 < completed delete operation key=QTCIIPTXIRW2 tag=PLX1WG1SU6JX type=del 2024-04-07T16:38:06.356 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.364+0000 7f346a3868c0 1 > prepared operation key=XJLQZ7IS9U1T tag=QP122OSC42A2 type=write size=494830 2024-04-07T16:38:06.363 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.368+0000 7f346a3868c0 1 < completed delete operation key=C1BTVJNHIM0X tag=Z57VM1NO4NU4 type=del 2024-04-07T16:38:06.368 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.376+0000 7f346a3868c0 1 > prepared operation key=1VBSUWNI3V1T tag=NBWEL7UHMUEZ type=write size=654775 2024-04-07T16:38:06.374 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.380+0000 7f346a3868c0 1 < completed part key=_multipart_6XDS0A7L0IY8.2~ZWF3JYJ728UG.0 size=99936 2024-04-07T16:38:06.385 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.392+0000 7f346a3868c0 1 < completed multipart upload key=6XDS0A7L0IY8 upload=ZWF3JYJ728UG tag=2L78O4MTF7EX type=write size=1148512 2024-04-07T16:38:06.393 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.400+0000 7f346a3868c0 1 > prepared operation key=EOYFVEI50SQ1 tag=SGJHBDWR58GW type=del size=3385823 2024-04-07T16:38:06.399 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.404+0000 7f346a3868c0 1 < completed write operation key=6XDS0A7L0IY8 tag=D9B5XNTBNKIU type=write size=334286 2024-04-07T16:38:06.405 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.412+0000 7f346a3868c0 1 > prepared operation key=YXMDZB23YTB3 tag=V304REZALYP6 type=del size=3788615 2024-04-07T16:38:06.411 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.416+0000 7f346a3868c0 1 < completed delete operation key=EOYFVEI50SQ1 tag=MPHB0P1TUGWQ type=del 2024-04-07T16:38:06.417 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.424+0000 7f346a3868c0 1 > prepared operation key=DXTO3AJ429AC tag=PJ436GN19ZKC type=del size=3582858 2024-04-07T16:38:06.423 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.432+0000 7f346a3868c0 1 < completed part key=_multipart_X5Q46GO8YTOK.2~6GQUON3PQR4K.0 size=1048576 2024-04-07T16:38:06.428 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.436+0000 7f346a3868c0 1 < completed part key=_multipart_X5Q46GO8YTOK.2~6GQUON3PQR4K.1 size=277430 2024-04-07T16:38:06.438 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.444+0000 7f346a3868c0 1 < completed multipart upload key=X5Q46GO8YTOK upload=6GQUON3PQR4K tag=RI7VJB9Y9EZY type=write size=2374582 2024-04-07T16:38:06.441 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:38:06 smithi100 bash[74942]: cluster 2024-04-07T16:38:05.067520+0000 mgr.y (mgr.25175) 682 : cluster [DBG] pgmap v1317: 137 pgs: 137 active+clean; 583 KiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 379 KiB/s rd, 129 KiB/s wr, 670 op/s 2024-04-07T16:38:06.441 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:38:06 smithi100 bash[76202]: cluster 2024-04-07T16:38:05.067520+0000 mgr.y (mgr.25175) 682 : cluster [DBG] pgmap v1317: 137 pgs: 137 active+clean; 583 KiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 379 KiB/s rd, 129 KiB/s wr, 670 op/s 2024-04-07T16:38:06.445 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.452+0000 7f346a3868c0 1 > prepared operation key=YXMDZB23YTB3 tag=DZHZX4FDVKYX type=del size=3083833 2024-04-07T16:38:06.451 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.456+0000 7f346a3868c0 1 < canceled operation key=6XDS0A7L0IY8 tag=7RZO6INQSPE2 type=del size=2738784 2024-04-07T16:38:06.457 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.464+0000 7f346a3868c0 1 > prepared operation key=NYET3212USVE tag=H0N699DBYL1L type=del size=2201670 2024-04-07T16:38:06.463 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.468+0000 7f346a3868c0 1 < canceled operation key=YOK8BAEOMPUR tag=X6R1E3A8G48C type=del size=3710781 2024-04-07T16:38:06.475 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.480+0000 7f346a3868c0 1 > prepared part key=_multipart_YOK8BAEOMPUR.2~1ZAXN4T7H1J8.0 size=1048576 2024-04-07T16:38:06.479 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.488+0000 7f346a3868c0 1 > prepared part key=_multipart_YOK8BAEOMPUR.2~1ZAXN4T7H1J8.1 size=48634 2024-04-07T16:38:06.480 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.488+0000 7f346a3868c0 1 > prepared multipart upload key=YOK8BAEOMPUR upload=1ZAXN4T7H1J8 tag=6LFC7J5E6OX6 type=write size=2145786 2024-04-07T16:38:06.485 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.492+0000 7f346a3868c0 1 < completed write operation key=8F81G0P0YBFW tag=CTVGL8NWZ9TX type=write size=144470 2024-04-07T16:38:06.491 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.500+0000 7f346a3868c0 1 > prepared operation key=E84U0SX44NQ8 tag=VN2GDERMI0KY type=del size=608408 2024-04-07T16:38:06.497 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.504+0000 7f346a3868c0 1 < completed part key=_multipart_QS4HYZFFAJNF.2~OVMIVBXVVUM8.0 size=1048576 2024-04-07T16:38:06.497 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.504+0000 7f346a3868c0 1 < canceled part key=_multipart_QS4HYZFFAJNF.2~OVMIVBXVVUM8.1 size=1048576 2024-04-07T16:38:06.498 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.504+0000 7f346a3868c0 1 < canceled part key=_multipart_QS4HYZFFAJNF.2~OVMIVBXVVUM8.2 size=785402 2024-04-07T16:38:06.507 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.516+0000 7f346a3868c0 1 < canceled multipart upload key=QS4HYZFFAJNF upload=OVMIVBXVVUM8 tag=MQ4XZ3VR5HY4 type=write size=3931130 2024-04-07T16:38:06.518 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.524+0000 7f346a3868c0 1 > prepared part key=_multipart_GBQSME7SY1AH.2~2Y5D3VI1CCJH.0 size=1048576 2024-04-07T16:38:06.524 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.532+0000 7f346a3868c0 1 > prepared part key=_multipart_GBQSME7SY1AH.2~2Y5D3VI1CCJH.1 size=1048576 2024-04-07T16:38:06.528 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.536+0000 7f346a3868c0 1 > prepared part key=_multipart_GBQSME7SY1AH.2~2Y5D3VI1CCJH.2 size=296499 2024-04-07T16:38:06.529 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.536+0000 7f346a3868c0 1 > prepared multipart upload key=GBQSME7SY1AH upload=2Y5D3VI1CCJH tag=VVPU4V0VLOQM type=write size=3442227 2024-04-07T16:38:06.534 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.540+0000 7f346a3868c0 1 < completed part key=_multipart_CL4PP1RKZU0E.2~1JQLBE7IBNMA.0 size=1048576 2024-04-07T16:38:06.540 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.548+0000 7f346a3868c0 1 < completed part key=_multipart_CL4PP1RKZU0E.2~1JQLBE7IBNMA.1 size=1048576 2024-04-07T16:38:06.546 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.552+0000 7f346a3868c0 1 < completed part key=_multipart_CL4PP1RKZU0E.2~1JQLBE7IBNMA.2 size=434809 2024-04-07T16:38:06.556 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.564+0000 7f346a3868c0 1 < completed multipart upload key=CL4PP1RKZU0E upload=1JQLBE7IBNMA tag=JMJ2ZO3SH924 type=write size=3580537 2024-04-07T16:38:06.566 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.572+0000 7f346a3868c0 1 > prepared part key=_multipart_8F81G0P0YBFW.2~070THO8SHNG8.0 size=1048576 2024-04-07T16:38:06.571 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.576+0000 7f346a3868c0 1 > prepared part key=_multipart_8F81G0P0YBFW.2~070THO8SHNG8.1 size=1033250 2024-04-07T16:38:06.571 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.576+0000 7f346a3868c0 1 > prepared multipart upload key=8F81G0P0YBFW upload=070THO8SHNG8 tag=AZ2U694FU2MI type=write size=3130402 2024-04-07T16:38:06.576 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.584+0000 7f346a3868c0 1 < completed delete operation key=LXOZ0T2KY6OR tag=3C37BFXTWHPO type=del 2024-04-07T16:38:06.582 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.588+0000 7f346a3868c0 1 > prepared operation key=95YVKJ14JD8W tag=8T4L5A22F7VM type=del size=1347816 2024-04-07T16:38:06.593 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.600+0000 7f346a3868c0 1 < suggested remove operation key=N14BUXLJONHZ tag=WP8XD0OWB555 type=del 2024-04-07T16:38:06.598 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.604+0000 7f346a3868c0 1 > prepared operation key=QS4HYZFFAJNF tag=00C0E1DW01VW type=del size=2894643 2024-04-07T16:38:06.603 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.612+0000 7f346a3868c0 1 < completed delete operation key=QS4HYZFFAJNF tag=LWWNTMWYTFPL type=del 2024-04-07T16:38:06.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:38:06 smithi144 bash[66654]: cluster 2024-04-07T16:38:05.067520+0000 mgr.y (mgr.25175) 682 : cluster [DBG] pgmap v1317: 137 pgs: 137 active+clean; 583 KiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 379 KiB/s rd, 129 KiB/s wr, 670 op/s 2024-04-07T16:38:06.614 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.620+0000 7f346a3868c0 1 > prepared part key=_multipart_C1BTVJNHIM0X.2~UJO9AOB1CUS6.0 size=1048576 2024-04-07T16:38:06.618 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.624+0000 7f346a3868c0 1 > prepared part key=_multipart_C1BTVJNHIM0X.2~UJO9AOB1CUS6.1 size=1048576 2024-04-07T16:38:06.623 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.632+0000 7f346a3868c0 1 > prepared part key=_multipart_C1BTVJNHIM0X.2~UJO9AOB1CUS6.2 size=273042 2024-04-07T16:38:06.623 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.632+0000 7f346a3868c0 1 > prepared multipart upload key=C1BTVJNHIM0X upload=UJO9AOB1CUS6 tag=XQFHL9R3QVW8 type=write size=3418770 2024-04-07T16:38:06.628 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.636+0000 7f346a3868c0 1 < completed delete operation key=1VBSUWNI3V1T tag=KI6CKNSSF6UR type=del 2024-04-07T16:38:06.634 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.640+0000 7f346a3868c0 1 > prepared operation key=1WGUPH7X5UA0 tag=JI58ZB28X13K type=del size=991762 2024-04-07T16:38:06.639 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.648+0000 7f346a3868c0 1 < completed write operation key=GZXHYZO5SMUP tag=7PSFX16ZYE8P type=write size=221428 2024-04-07T16:38:06.650 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.656+0000 7f346a3868c0 1 > prepared part key=_multipart_1VBSUWNI3V1T.2~G03SODY8TLX3.0 size=1048576 2024-04-07T16:38:06.655 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.660+0000 7f346a3868c0 1 > prepared part key=_multipart_1VBSUWNI3V1T.2~G03SODY8TLX3.1 size=198098 2024-04-07T16:38:06.655 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.660+0000 7f346a3868c0 1 > prepared multipart upload key=1VBSUWNI3V1T upload=G03SODY8TLX3 tag=JRU5588I5ZAF type=write size=2295250 2024-04-07T16:38:06.660 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.668+0000 7f346a3868c0 1 < completed write operation key=XJLQZ7IS9U1T tag=QP122OSC42A2 type=write size=494830 2024-04-07T16:38:06.667 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.676+0000 7f346a3868c0 1 > prepared operation key=NYET3212USVE tag=BHLXQ5CS5D6O type=del size=1317503 2024-04-07T16:38:06.672 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.680+0000 7f346a3868c0 1 < canceled operation key=1VBSUWNI3V1T tag=NBWEL7UHMUEZ type=write size=654775 2024-04-07T16:38:06.677 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.684+0000 7f346a3868c0 1 > prepared operation key=95YVKJ14JD8W tag=4Y26OHADF3Z6 type=del size=2534155 2024-04-07T16:38:06.683 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.688+0000 7f346a3868c0 1 < completed delete operation key=EOYFVEI50SQ1 tag=SGJHBDWR58GW type=del 2024-04-07T16:38:06.688 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.696+0000 7f346a3868c0 1 > prepared operation key=6XDS0A7L0IY8 tag=6M9RB5O0LWKA type=del size=1891408 2024-04-07T16:38:06.703 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.712+0000 7f346a3868c0 1 < suggested remove operation key=YXMDZB23YTB3 tag=V304REZALYP6 type=del 2024-04-07T16:38:06.709 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.716+0000 7f346a3868c0 1 > prepared operation key=DXTO3AJ429AC tag=ULPPT9ZQYBRY type=del size=4149230 2024-04-07T16:38:06.717 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.724+0000 7f346a3868c0 1 < completed delete operation key=DXTO3AJ429AC tag=PJ436GN19ZKC type=del 2024-04-07T16:38:06.722 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.728+0000 7f346a3868c0 1 > prepared operation key=GZXHYZO5SMUP tag=6YFTQXT13VGH type=write size=760018 2024-04-07T16:38:06.729 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.736+0000 7f346a3868c0 -1 < failed to complete operation key=YXMDZB23YTB3 tag=DZHZX4FDVKYX type=del size=3083833: (22) Invalid argument 2024-04-07T16:38:06.740 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.748+0000 7f346a3868c0 1 > prepared part key=_multipart_GBQSME7SY1AH.2~MIQ7OVD86U2J.0 size=385103 2024-04-07T16:38:06.740 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.748+0000 7f346a3868c0 1 > prepared multipart upload key=GBQSME7SY1AH upload=MIQ7OVD86U2J tag=X875OS1MECJR type=write size=1433679 2024-04-07T16:38:06.746 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.752+0000 7f346a3868c0 1 < completed delete operation key=NYET3212USVE tag=H0N699DBYL1L type=del 2024-04-07T16:38:06.751 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.760+0000 7f346a3868c0 1 > prepared operation key=LXOZ0T2KY6OR tag=KD2QBX69CJ26 type=del size=3084663 2024-04-07T16:38:06.757 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.764+0000 7f346a3868c0 1 < completed part key=_multipart_YOK8BAEOMPUR.2~1ZAXN4T7H1J8.0 size=1048576 2024-04-07T16:38:06.762 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.768+0000 7f346a3868c0 1 < completed part key=_multipart_YOK8BAEOMPUR.2~1ZAXN4T7H1J8.1 size=48634 2024-04-07T16:38:06.773 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.780+0000 7f346a3868c0 1 < completed multipart upload key=YOK8BAEOMPUR upload=1ZAXN4T7H1J8 tag=6LFC7J5E6OX6 type=write size=2145786 2024-04-07T16:38:06.785 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.792+0000 7f346a3868c0 1 > prepared part key=_multipart_CL4PP1RKZU0E.2~GK4GV53A79B0.0 size=1048576 2024-04-07T16:38:06.790 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.796+0000 7f346a3868c0 1 > prepared part key=_multipart_CL4PP1RKZU0E.2~GK4GV53A79B0.1 size=1048576 2024-04-07T16:38:06.795 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.800+0000 7f346a3868c0 1 > prepared part key=_multipart_CL4PP1RKZU0E.2~GK4GV53A79B0.2 size=973073 2024-04-07T16:38:06.795 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.800+0000 7f346a3868c0 1 > prepared multipart upload key=CL4PP1RKZU0E upload=GK4GV53A79B0 tag=HW31WNP3ZSCG type=write size=4118801 2024-04-07T16:38:06.806 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.812+0000 7f346a3868c0 1 < suggested remove operation key=E84U0SX44NQ8 tag=VN2GDERMI0KY type=del 2024-04-07T16:38:06.818 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.824+0000 7f346a3868c0 1 > prepared part key=_multipart_6XDS0A7L0IY8.2~EEQU7YIUSU54.0 size=1003307 2024-04-07T16:38:06.818 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.824+0000 7f346a3868c0 1 > prepared multipart upload key=6XDS0A7L0IY8 upload=EEQU7YIUSU54 tag=HRRNOKHPC08B type=write size=2051883 2024-04-07T16:38:06.819 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.828+0000 7f346a3868c0 1 < canceled part key=_multipart_GBQSME7SY1AH.2~2Y5D3VI1CCJH.0 size=1048576 2024-04-07T16:38:06.824 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.832+0000 7f346a3868c0 1 < completed part key=_multipart_GBQSME7SY1AH.2~2Y5D3VI1CCJH.1 size=1048576 2024-04-07T16:38:06.824 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.832+0000 7f346a3868c0 1 < canceled part key=_multipart_GBQSME7SY1AH.2~2Y5D3VI1CCJH.2 size=296499 2024-04-07T16:38:06.836 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.844+0000 7f346a3868c0 1 < canceled multipart upload key=GBQSME7SY1AH upload=2Y5D3VI1CCJH tag=VVPU4V0VLOQM type=write size=3442227 2024-04-07T16:38:06.847 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.856+0000 7f346a3868c0 1 > prepared part key=_multipart_HBLEXONKXJTL.2~OWYQOEUAZ0KJ.0 size=1048576 2024-04-07T16:38:06.852 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.860+0000 7f346a3868c0 1 > prepared part key=_multipart_HBLEXONKXJTL.2~OWYQOEUAZ0KJ.1 size=1034296 2024-04-07T16:38:06.852 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.860+0000 7f346a3868c0 1 > prepared multipart upload key=HBLEXONKXJTL upload=OWYQOEUAZ0KJ tag=HT76Z8EIR2AY type=write size=3131448 2024-04-07T16:38:06.858 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.864+0000 7f346a3868c0 1 < completed part key=_multipart_8F81G0P0YBFW.2~070THO8SHNG8.0 size=1048576 2024-04-07T16:38:06.863 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.872+0000 7f346a3868c0 1 < completed part key=_multipart_8F81G0P0YBFW.2~070THO8SHNG8.1 size=1033250 2024-04-07T16:38:06.875 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.880+0000 7f346a3868c0 1 < completed multipart upload key=8F81G0P0YBFW upload=070THO8SHNG8 tag=AZ2U694FU2MI type=write size=3130402 2024-04-07T16:38:06.881 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.888+0000 7f346a3868c0 1 > prepared operation key=1WGUPH7X5UA0 tag=XC70YCMNYBSZ type=del size=885920 2024-04-07T16:38:06.887 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.896+0000 7f346a3868c0 1 < canceled operation key=95YVKJ14JD8W tag=8T4L5A22F7VM type=del size=1347816 2024-04-07T16:38:06.899 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.908+0000 7f346a3868c0 1 > prepared part key=_multipart_8F81G0P0YBFW.2~378I0XA07V0Q.0 size=1048576 2024-04-07T16:38:06.904 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.912+0000 7f346a3868c0 1 > prepared part key=_multipart_8F81G0P0YBFW.2~378I0XA07V0Q.1 size=283612 2024-04-07T16:38:06.904 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.912+0000 7f346a3868c0 1 > prepared multipart upload key=8F81G0P0YBFW upload=378I0XA07V0Q tag=QZIAVUFLGLA2 type=write size=2380764 2024-04-07T16:38:06.909 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.916+0000 7f346a3868c0 1 < completed delete operation key=QS4HYZFFAJNF tag=00C0E1DW01VW type=del 2024-04-07T16:38:06.915 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.924+0000 7f346a3868c0 1 > prepared operation key=N14BUXLJONHZ tag=71HUZGPMJUDR type=del size=3073183 2024-04-07T16:38:06.923 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.928+0000 7f346a3868c0 1 < completed part key=_multipart_C1BTVJNHIM0X.2~UJO9AOB1CUS6.0 size=1048576 2024-04-07T16:38:06.928 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.936+0000 7f346a3868c0 1 < completed part key=_multipart_C1BTVJNHIM0X.2~UJO9AOB1CUS6.1 size=1048576 2024-04-07T16:38:06.928 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.936+0000 7f346a3868c0 1 < canceled part key=_multipart_C1BTVJNHIM0X.2~UJO9AOB1CUS6.2 size=273042 2024-04-07T16:38:06.940 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.948+0000 7f346a3868c0 1 < canceled multipart upload key=C1BTVJNHIM0X upload=UJO9AOB1CUS6 tag=XQFHL9R3QVW8 type=write size=3418770 2024-04-07T16:38:06.953 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.960+0000 7f346a3868c0 1 > prepared part key=_multipart_IP8YW2A75TCW.2~ORGU7I7P8REX.0 size=934112 2024-04-07T16:38:06.953 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.960+0000 7f346a3868c0 1 > prepared multipart upload key=IP8YW2A75TCW upload=ORGU7I7P8REX tag=SD01AZ4SEFNB type=write size=1982688 2024-04-07T16:38:06.959 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.968+0000 7f346a3868c0 1 < completed delete operation key=1WGUPH7X5UA0 tag=JI58ZB28X13K type=del 2024-04-07T16:38:06.966 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.972+0000 7f346a3868c0 1 > prepared operation key=NYET3212USVE tag=SK3WHWMPRSPS type=del size=2733549 2024-04-07T16:38:06.972 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.980+0000 7f346a3868c0 1 < completed part key=_multipart_1VBSUWNI3V1T.2~G03SODY8TLX3.0 size=1048576 2024-04-07T16:38:06.979 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.988+0000 7f346a3868c0 1 < completed part key=_multipart_1VBSUWNI3V1T.2~G03SODY8TLX3.1 size=198098 2024-04-07T16:38:06.989 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:06.996+0000 7f346a3868c0 1 < completed multipart upload key=1VBSUWNI3V1T upload=G03SODY8TLX3 tag=JRU5588I5ZAF type=write size=2295250 2024-04-07T16:38:06.995 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:07.004+0000 7f346a3868c0 1 > prepared operation key=YXMDZB23YTB3 tag=1IUGX3COML4A type=write size=545094 2024-04-07T16:38:07.002 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:07.008+0000 7f346a3868c0 1 < completed delete operation key=NYET3212USVE tag=BHLXQ5CS5D6O type=del 2024-04-07T16:38:07.013 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:07.020+0000 7f346a3868c0 1 > prepared part key=_multipart_QVOKSQAGHPA9.2~XSXKE95L7502.0 size=1048576 2024-04-07T16:38:07.018 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:07.024+0000 7f346a3868c0 1 > prepared part key=_multipart_QVOKSQAGHPA9.2~XSXKE95L7502.1 size=1048576 2024-04-07T16:38:07.023 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:07.028+0000 7f346a3868c0 1 > prepared part key=_multipart_QVOKSQAGHPA9.2~XSXKE95L7502.2 size=1045048 2024-04-07T16:38:07.023 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:07.028+0000 7f346a3868c0 1 > prepared multipart upload key=QVOKSQAGHPA9 upload=XSXKE95L7502 tag=TKXTPMAFFB49 type=write size=4190776 2024-04-07T16:38:07.029 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:07.036+0000 7f346a3868c0 1 < canceled operation key=95YVKJ14JD8W tag=4Y26OHADF3Z6 type=del size=2534155 2024-04-07T16:38:07.036 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:07.044+0000 7f346a3868c0 1 > prepared operation key=1WGUPH7X5UA0 tag=6MCXE6LYRL12 type=del size=2175763 2024-04-07T16:38:07.042 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:07.048+0000 7f346a3868c0 1 < completed delete operation key=6XDS0A7L0IY8 tag=6M9RB5O0LWKA type=del 2024-04-07T16:38:07.052 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:07.060+0000 7f346a3868c0 1 > prepared part key=_multipart_E84U0SX44NQ8.2~F7FPQ5SD4VL0.0 size=1048576 2024-04-07T16:38:07.057 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:07.064+0000 7f346a3868c0 1 > prepared part key=_multipart_E84U0SX44NQ8.2~F7FPQ5SD4VL0.1 size=373922 2024-04-07T16:38:07.057 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:07.064+0000 7f346a3868c0 1 > prepared multipart upload key=E84U0SX44NQ8 upload=F7FPQ5SD4VL0 tag=Q4QKJ4N51K9K type=write size=2471074 2024-04-07T16:38:07.062 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:07.068+0000 7f346a3868c0 1 < completed delete operation key=DXTO3AJ429AC tag=ULPPT9ZQYBRY type=del 2024-04-07T16:38:07.068 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:07.076+0000 7f346a3868c0 1 > prepared operation key=IP8YW2A75TCW tag=Q8A7YDR2QR9B type=del size=1317327 2024-04-07T16:38:07.073 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:07.080+0000 7f346a3868c0 1 < completed write operation key=GZXHYZO5SMUP tag=6YFTQXT13VGH type=write size=760018 2024-04-07T16:38:07.079 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:07.084+0000 7f346a3868c0 1 > prepared operation key=GBQSME7SY1AH tag=MJNIC61EJT90 type=del size=1275704 2024-04-07T16:38:07.084 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:07.092+0000 7f346a3868c0 1 < completed part key=_multipart_GBQSME7SY1AH.2~MIQ7OVD86U2J.0 size=385103 2024-04-07T16:38:07.094 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:07.100+0000 7f346a3868c0 1 < completed multipart upload key=GBQSME7SY1AH upload=MIQ7OVD86U2J tag=X875OS1MECJR type=write size=1433679 2024-04-07T16:38:07.105 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:07.112+0000 7f346a3868c0 1 > prepared part key=_multipart_IX3F0IUOKBX1.2~LGFUZYPFRLBO.0 size=1048576 2024-04-07T16:38:07.110 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:07.116+0000 7f346a3868c0 1 > prepared part key=_multipart_IX3F0IUOKBX1.2~LGFUZYPFRLBO.1 size=145436 2024-04-07T16:38:07.111 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:07.116+0000 7f346a3868c0 1 > prepared multipart upload key=IX3F0IUOKBX1 upload=LGFUZYPFRLBO tag=4NA22SE32ZU6 type=write size=2242588 2024-04-07T16:38:07.116 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:07.124+0000 7f346a3868c0 1 < completed delete operation key=LXOZ0T2KY6OR tag=KD2QBX69CJ26 type=del 2024-04-07T16:38:07.127 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:07.132+0000 7f346a3868c0 1 > prepared part key=_multipart_XJLQZ7IS9U1T.2~W40E118DU7F1.0 size=965520 2024-04-07T16:38:07.127 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:07.132+0000 7f346a3868c0 1 > prepared multipart upload key=XJLQZ7IS9U1T upload=W40E118DU7F1 tag=PLVHWW89OY3H type=write size=2014096 2024-04-07T16:38:07.137 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:07.144+0000 7f346a3868c0 1 < completed part key=_multipart_CL4PP1RKZU0E.2~GK4GV53A79B0.0 size=1048576 2024-04-07T16:38:07.137 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:07.144+0000 7f346a3868c0 1 < canceled part key=_multipart_CL4PP1RKZU0E.2~GK4GV53A79B0.1 size=1048576 2024-04-07T16:38:07.144 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:07.152+0000 7f346a3868c0 1 < completed part key=_multipart_CL4PP1RKZU0E.2~GK4GV53A79B0.2 size=973073 2024-04-07T16:38:07.156 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:07.164+0000 7f346a3868c0 1 < canceled multipart upload key=CL4PP1RKZU0E upload=GK4GV53A79B0 tag=HW31WNP3ZSCG type=write size=4118801 2024-04-07T16:38:07.161 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:07.168+0000 7f346a3868c0 1 > prepared operation key=C1BTVJNHIM0X tag=YNWPK0N5LL5A type=del size=1757250 2024-04-07T16:38:07.168 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:07.176+0000 7f346a3868c0 1 < completed part key=_multipart_6XDS0A7L0IY8.2~EEQU7YIUSU54.0 size=1003307 2024-04-07T16:38:07.182 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:07.188+0000 7f346a3868c0 1 < completed multipart upload key=6XDS0A7L0IY8 upload=EEQU7YIUSU54 tag=HRRNOKHPC08B type=write size=2051883 2024-04-07T16:38:07.184 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:07.192+0000 7f346a3868c0 1 > prepared operation key=YOK8BAEOMPUR tag=8P8BDZ7LBTVZ type=del size=3884260 2024-04-07T16:38:07.191 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:07.196+0000 7f346a3868c0 1 < completed part key=_multipart_HBLEXONKXJTL.2~OWYQOEUAZ0KJ.0 size=1048576 2024-04-07T16:38:07.196 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:07.204+0000 7f346a3868c0 1 < completed part key=_multipart_HBLEXONKXJTL.2~OWYQOEUAZ0KJ.1 size=1034296 2024-04-07T16:38:07.205 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:07.212+0000 7f346a3868c0 1 < completed multipart upload key=HBLEXONKXJTL upload=OWYQOEUAZ0KJ tag=HT76Z8EIR2AY type=write size=3131448 2024-04-07T16:38:07.211 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:07.216+0000 7f346a3868c0 1 > prepared operation key=EQR0FY86AT5X tag=D1JBF2I76D08 type=del size=4078038 2024-04-07T16:38:07.223 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:07.228+0000 7f346a3868c0 1 < suggested update operation key=1WGUPH7X5UA0 tag=XC70YCMNYBSZ type=del size=885920 2024-04-07T16:38:07.234 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:07.240+0000 7f346a3868c0 1 > prepared part key=_multipart_YXMDZB23YTB3.2~7ZJOOFLTRT8E.0 size=1048576 2024-04-07T16:38:07.239 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:07.248+0000 7f346a3868c0 1 > prepared part key=_multipart_YXMDZB23YTB3.2~7ZJOOFLTRT8E.1 size=430650 2024-04-07T16:38:07.239 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:07.248+0000 7f346a3868c0 1 > prepared multipart upload key=YXMDZB23YTB3 upload=7ZJOOFLTRT8E tag=X7244FJI8HD9 type=write size=2527802 2024-04-07T16:38:07.245 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:07.252+0000 7f346a3868c0 1 < completed part key=_multipart_8F81G0P0YBFW.2~378I0XA07V0Q.0 size=1048576 2024-04-07T16:38:07.250 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:07.256+0000 7f346a3868c0 1 < completed part key=_multipart_8F81G0P0YBFW.2~378I0XA07V0Q.1 size=283612 2024-04-07T16:38:07.260 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:07.268+0000 7f346a3868c0 1 < completed multipart upload key=8F81G0P0YBFW upload=378I0XA07V0Q tag=QZIAVUFLGLA2 type=write size=2380764 2024-04-07T16:38:07.272 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:07.280+0000 7f346a3868c0 1 > prepared part key=_multipart_6XDS0A7L0IY8.2~BI4Y31JNRTGI.0 size=1048576 2024-04-07T16:38:07.276 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:07.284+0000 7f346a3868c0 1 > prepared part key=_multipart_6XDS0A7L0IY8.2~BI4Y31JNRTGI.1 size=1048576 2024-04-07T16:38:07.282 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:07.288+0000 7f346a3868c0 1 > prepared part key=_multipart_6XDS0A7L0IY8.2~BI4Y31JNRTGI.2 size=526932 2024-04-07T16:38:07.282 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:07.288+0000 7f346a3868c0 1 > prepared multipart upload key=6XDS0A7L0IY8 upload=BI4Y31JNRTGI tag=NIGOBDJLAYH3 type=write size=3672660 2024-04-07T16:38:07.288 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:07.296+0000 7f346a3868c0 1 < completed delete operation key=N14BUXLJONHZ tag=71HUZGPMJUDR type=del 2024-04-07T16:38:07.293 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:07.300+0000 7f346a3868c0 1 > prepared operation key=1WGUPH7X5UA0 tag=T3L15IO7UL5F type=del size=4009359 2024-04-07T16:38:07.299 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:07.308+0000 7f346a3868c0 1 < completed part key=_multipart_IP8YW2A75TCW.2~ORGU7I7P8REX.0 size=934112 2024-04-07T16:38:07.309 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:07.316+0000 7f346a3868c0 1 < completed multipart upload key=IP8YW2A75TCW upload=ORGU7I7P8REX tag=SD01AZ4SEFNB type=write size=1982688 2024-04-07T16:38:07.314 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:07.320+0000 7f346a3868c0 1 > prepared operation key=NYET3212USVE tag=6CDW83CZ93I0 type=del size=1027320 2024-04-07T16:38:07.321 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:07.328+0000 7f346a3868c0 1 < completed delete operation key=NYET3212USVE tag=SK3WHWMPRSPS type=del 2024-04-07T16:38:07.332 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:07.340+0000 7f346a3868c0 1 > prepared part key=_multipart_LXOZ0T2KY6OR.2~B5UOEHI96LQY.0 size=1048576 2024-04-07T16:38:07.336 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:07.344+0000 7f346a3868c0 1 > prepared part key=_multipart_LXOZ0T2KY6OR.2~B5UOEHI96LQY.1 size=1048576 2024-04-07T16:38:07.341 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:07.348+0000 7f346a3868c0 1 > prepared part key=_multipart_LXOZ0T2KY6OR.2~B5UOEHI96LQY.2 size=140974 2024-04-07T16:38:07.342 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:07.348+0000 7f346a3868c0 1 > prepared multipart upload key=LXOZ0T2KY6OR upload=B5UOEHI96LQY tag=3GSEDWV48V3L type=write size=3286702 2024-04-07T16:38:07.347 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:07.356+0000 7f346a3868c0 1 < completed write operation key=YXMDZB23YTB3 tag=1IUGX3COML4A type=write size=545094 2024-04-07T16:38:07.358 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:07.364+0000 7f346a3868c0 1 > prepared part key=_multipart_1VBSUWNI3V1T.2~I67ELAOT01K2.0 size=505697 2024-04-07T16:38:07.358 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:07.364+0000 7f346a3868c0 1 > prepared multipart upload key=1VBSUWNI3V1T upload=I67ELAOT01K2 tag=CL5U6T6QBP5G type=write size=1554273 2024-04-07T16:38:07.364 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:07.372+0000 7f346a3868c0 1 < completed part key=_multipart_QVOKSQAGHPA9.2~XSXKE95L7502.0 size=1048576 2024-04-07T16:38:07.369 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:07.376+0000 7f346a3868c0 1 < completed part key=_multipart_QVOKSQAGHPA9.2~XSXKE95L7502.1 size=1048576 2024-04-07T16:38:07.374 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:07.380+0000 7f346a3868c0 1 < completed part key=_multipart_QVOKSQAGHPA9.2~XSXKE95L7502.2 size=1045048 2024-04-07T16:38:07.386 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:07.392+0000 7f346a3868c0 1 < completed multipart upload key=QVOKSQAGHPA9 upload=XSXKE95L7502 tag=TKXTPMAFFB49 type=write size=4190776 2024-04-07T16:38:07.391 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:07.396+0000 7f346a3868c0 1 > prepared operation key=EQR0FY86AT5X tag=5WN2PUHIXYHC type=del size=1020097 2024-04-07T16:38:07.399 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:07.404+0000 7f346a3868c0 -1 < failed to complete operation key=1WGUPH7X5UA0 tag=6MCXE6LYRL12 type=del size=2175763: (22) Invalid argument 2024-04-07T16:38:07.405 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:07.412+0000 7f346a3868c0 1 > prepared operation key=N14BUXLJONHZ tag=MJJ7ZP748KGW type=del size=2217957 2024-04-07T16:38:07.412 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:07.420+0000 7f346a3868c0 1 < completed part key=_multipart_E84U0SX44NQ8.2~F7FPQ5SD4VL0.0 size=1048576 2024-04-07T16:38:07.418 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:07.424+0000 7f346a3868c0 1 < completed part key=_multipart_E84U0SX44NQ8.2~F7FPQ5SD4VL0.1 size=373922 2024-04-07T16:38:07.429 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:07.436+0000 7f346a3868c0 1 < completed multipart upload key=E84U0SX44NQ8 upload=F7FPQ5SD4VL0 tag=Q4QKJ4N51K9K type=write size=2471074 2024-04-07T16:38:07.440 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:07.448+0000 7f346a3868c0 1 > prepared part key=_multipart_YXMDZB23YTB3.2~AE9BOM812B8P.0 size=1048576 2024-04-07T16:38:07.447 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:07.452+0000 7f346a3868c0 1 > prepared part key=_multipart_YXMDZB23YTB3.2~AE9BOM812B8P.1 size=323269 2024-04-07T16:38:07.447 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:07.452+0000 7f346a3868c0 1 > prepared multipart upload key=YXMDZB23YTB3 upload=AE9BOM812B8P tag=EMRNVNRGN07Z type=write size=2420421 2024-04-07T16:38:07.454 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:07.460+0000 7f346a3868c0 1 < completed delete operation key=IP8YW2A75TCW tag=Q8A7YDR2QR9B type=del 2024-04-07T16:38:07.462 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:07.468+0000 7f346a3868c0 1 > prepared operation key=1WGUPH7X5UA0 tag=6BKE21R05L8I type=del size=1007763 2024-04-07T16:38:07.469 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:07.476+0000 7f346a3868c0 1 < completed delete operation key=GBQSME7SY1AH tag=MJNIC61EJT90 type=del 2024-04-07T16:38:07.475 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:07.484+0000 7f346a3868c0 1 > prepared operation key=1WGUPH7X5UA0 tag=T9Q8SDRMVNZF type=del size=3303770 2024-04-07T16:38:07.482 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:07.488+0000 7f346a3868c0 1 < completed part key=_multipart_IX3F0IUOKBX1.2~LGFUZYPFRLBO.0 size=1048576 2024-04-07T16:38:07.487 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:07.496+0000 7f346a3868c0 1 < completed part key=_multipart_IX3F0IUOKBX1.2~LGFUZYPFRLBO.1 size=145436 2024-04-07T16:38:07.498 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:07.504+0000 7f346a3868c0 1 < completed multipart upload key=IX3F0IUOKBX1 upload=LGFUZYPFRLBO tag=4NA22SE32ZU6 type=write size=2242588 2024-04-07T16:38:07.504 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:07.512+0000 7f346a3868c0 1 > prepared operation key=UIW3DN4NUYPP tag=EVXSZSAHCEGB type=del size=971682 2024-04-07T16:38:07.505 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:07.512+0000 7f346a3868c0 1 < canceled part key=_multipart_XJLQZ7IS9U1T.2~W40E118DU7F1.0 size=965520 2024-04-07T16:38:07.515 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:07.524+0000 7f346a3868c0 1 < canceled multipart upload key=XJLQZ7IS9U1T upload=W40E118DU7F1 tag=PLVHWW89OY3H type=write size=2014096 2024-04-07T16:38:07.521 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:07.528+0000 7f346a3868c0 1 > prepared operation key=HBLEXONKXJTL tag=0EN6IOSGNXTD type=del size=1902308 2024-04-07T16:38:07.527 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:07.536+0000 7f346a3868c0 1 < completed delete operation key=C1BTVJNHIM0X tag=YNWPK0N5LL5A type=del 2024-04-07T16:38:07.533 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:07.540+0000 7f346a3868c0 1 > prepared operation key=QTCIIPTXIRW2 tag=AB90RYX33KDS type=del size=3394375 2024-04-07T16:38:07.539 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:07.544+0000 7f346a3868c0 1 < canceled operation key=YOK8BAEOMPUR tag=8P8BDZ7LBTVZ type=del size=3884260 2024-04-07T16:38:07.545 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:07.552+0000 7f346a3868c0 1 > prepared operation key=IX3F0IUOKBX1 tag=FREMUJHZB31W type=del size=3632809 2024-04-07T16:38:07.551 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:07.556+0000 7f346a3868c0 1 < completed delete operation key=EQR0FY86AT5X tag=D1JBF2I76D08 type=del 2024-04-07T16:38:07.557 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:07.564+0000 7f346a3868c0 1 > prepared operation key=YXMDZB23YTB3 tag=Y8MLADT7ZIN7 type=del size=2365911 2024-04-07T16:38:07.563 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:07.572+0000 7f346a3868c0 1 < completed part key=_multipart_YXMDZB23YTB3.2~7ZJOOFLTRT8E.0 size=1048576 2024-04-07T16:38:07.568 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:07.576+0000 7f346a3868c0 1 < completed part key=_multipart_YXMDZB23YTB3.2~7ZJOOFLTRT8E.1 size=430650 2024-04-07T16:38:07.578 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:07.584+0000 7f346a3868c0 1 < completed multipart upload key=YXMDZB23YTB3 upload=7ZJOOFLTRT8E tag=X7244FJI8HD9 type=write size=2527802 2024-04-07T16:38:07.589 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:07.596+0000 7f346a3868c0 1 > prepared part key=_multipart_YXMDZB23YTB3.2~PMZCF7YE8DWU.0 size=1048576 2024-04-07T16:38:07.594 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:07.600+0000 7f346a3868c0 1 > prepared part key=_multipart_YXMDZB23YTB3.2~PMZCF7YE8DWU.1 size=706699 2024-04-07T16:38:07.594 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:07.600+0000 7f346a3868c0 1 > prepared multipart upload key=YXMDZB23YTB3 upload=PMZCF7YE8DWU tag=LPYK0922P5UB type=write size=2803851 2024-04-07T16:38:07.600 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:07.608+0000 7f346a3868c0 1 < completed part key=_multipart_6XDS0A7L0IY8.2~BI4Y31JNRTGI.0 size=1048576 2024-04-07T16:38:07.607 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:07.616+0000 7f346a3868c0 1 < completed part key=_multipart_6XDS0A7L0IY8.2~BI4Y31JNRTGI.1 size=1048576 2024-04-07T16:38:07.612 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:07.620+0000 7f346a3868c0 1 < completed part key=_multipart_6XDS0A7L0IY8.2~BI4Y31JNRTGI.2 size=526932 2024-04-07T16:38:07.624 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:07.632+0000 7f346a3868c0 1 < completed multipart upload key=6XDS0A7L0IY8 upload=BI4Y31JNRTGI tag=NIGOBDJLAYH3 type=write size=3672660 2024-04-07T16:38:07.635 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:07.644+0000 7f346a3868c0 1 > prepared part key=_multipart_1WGUPH7X5UA0.2~LDN4RFCPD5FD.0 size=1048576 2024-04-07T16:38:07.641 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:07.648+0000 7f346a3868c0 1 > prepared part key=_multipart_1WGUPH7X5UA0.2~LDN4RFCPD5FD.1 size=1048576 2024-04-07T16:38:07.646 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:07.652+0000 7f346a3868c0 1 > prepared part key=_multipart_1WGUPH7X5UA0.2~LDN4RFCPD5FD.2 size=787849 2024-04-07T16:38:07.646 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:07.652+0000 7f346a3868c0 1 > prepared multipart upload key=1WGUPH7X5UA0 upload=LDN4RFCPD5FD tag=N4CCVW7WDEJZ type=write size=3933577 2024-04-07T16:38:07.653 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:07.660+0000 7f346a3868c0 1 < completed delete operation key=1WGUPH7X5UA0 tag=T3L15IO7UL5F type=del 2024-04-07T16:38:07.663 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:07.672+0000 7f346a3868c0 1 > prepared part key=_multipart_HBLEXONKXJTL.2~FZQZ4ZLGXIKN.0 size=1048576 2024-04-07T16:38:07.668 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:07.676+0000 7f346a3868c0 1 > prepared part key=_multipart_HBLEXONKXJTL.2~FZQZ4ZLGXIKN.1 size=686166 2024-04-07T16:38:07.668 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:07.676+0000 7f346a3868c0 1 > prepared multipart upload key=HBLEXONKXJTL upload=FZQZ4ZLGXIKN tag=WG7NT6S2BNHP type=write size=2783318 2024-04-07T16:38:07.679 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:07.688+0000 7f346a3868c0 1 < suggested remove operation key=NYET3212USVE tag=6CDW83CZ93I0 type=del 2024-04-07T16:38:07.690 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:07.696+0000 7f346a3868c0 1 > prepared part key=_multipart_IP8YW2A75TCW.2~EJ1794XG0VNY.0 size=1048576 2024-04-07T16:38:07.695 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:07.700+0000 7f346a3868c0 1 > prepared part key=_multipart_IP8YW2A75TCW.2~EJ1794XG0VNY.1 size=965773 2024-04-07T16:38:07.695 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:07.700+0000 7f346a3868c0 1 > prepared multipart upload key=IP8YW2A75TCW upload=EJ1794XG0VNY tag=I6WMIPHX9BHP type=write size=3062925 2024-04-07T16:38:07.701 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:07.708+0000 7f346a3868c0 1 < completed part key=_multipart_LXOZ0T2KY6OR.2~B5UOEHI96LQY.0 size=1048576 2024-04-07T16:38:07.706 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:07.712+0000 7f346a3868c0 1 < completed part key=_multipart_LXOZ0T2KY6OR.2~B5UOEHI96LQY.1 size=1048576 2024-04-07T16:38:07.710 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:07.716+0000 7f346a3868c0 1 < completed part key=_multipart_LXOZ0T2KY6OR.2~B5UOEHI96LQY.2 size=140974 2024-04-07T16:38:07.721 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:07.728+0000 7f346a3868c0 1 < completed multipart upload key=LXOZ0T2KY6OR upload=B5UOEHI96LQY tag=3GSEDWV48V3L type=write size=3286702 2024-04-07T16:38:07.727 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:07.732+0000 7f346a3868c0 1 > prepared operation key=ROFH30W3UCAX tag=P1NF5CXSV5QV type=del size=3800845 2024-04-07T16:38:07.733 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:07.740+0000 7f346a3868c0 1 < completed part key=_multipart_1VBSUWNI3V1T.2~I67ELAOT01K2.0 size=505697 2024-04-07T16:38:07.743 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:07.752+0000 7f346a3868c0 1 < completed multipart upload key=1VBSUWNI3V1T upload=I67ELAOT01K2 tag=CL5U6T6QBP5G type=write size=1554273 2024-04-07T16:38:07.754 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:07.760+0000 7f346a3868c0 1 > prepared part key=_multipart_95YVKJ14JD8W.2~XK841ONVXXR4.0 size=1048576 2024-04-07T16:38:07.759 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:07.768+0000 7f346a3868c0 1 > prepared part key=_multipart_95YVKJ14JD8W.2~XK841ONVXXR4.1 size=750478 2024-04-07T16:38:07.759 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:07.768+0000 7f346a3868c0 1 > prepared multipart upload key=95YVKJ14JD8W upload=XK841ONVXXR4 tag=X81C6P6QTSF3 type=write size=2847630 2024-04-07T16:38:07.765 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:07.772+0000 7f346a3868c0 1 < completed delete operation key=EQR0FY86AT5X tag=5WN2PUHIXYHC type=del 2024-04-07T16:38:07.771 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:07.780+0000 7f346a3868c0 1 > prepared operation key=EOYFVEI50SQ1 tag=SN7151QV1N5O type=del size=726882 2024-04-07T16:38:07.777 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:07.784+0000 7f346a3868c0 1 < completed delete operation key=N14BUXLJONHZ tag=MJJ7ZP748KGW type=del 2024-04-07T16:38:07.783 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:07.792+0000 7f346a3868c0 1 > prepared operation key=1VKVQXSQMTU1 tag=IDWZ7KVE8C79 type=del size=4058556 2024-04-07T16:38:07.789 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:07.796+0000 7f346a3868c0 1 < completed part key=_multipart_YXMDZB23YTB3.2~AE9BOM812B8P.0 size=1048576 2024-04-07T16:38:07.789 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:07.796+0000 7f346a3868c0 1 < canceled part key=_multipart_YXMDZB23YTB3.2~AE9BOM812B8P.1 size=323269 2024-04-07T16:38:07.799 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:07.808+0000 7f346a3868c0 1 < canceled multipart upload key=YXMDZB23YTB3 upload=AE9BOM812B8P tag=EMRNVNRGN07Z type=write size=2420421 2024-04-07T16:38:07.811 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:07.816+0000 7f346a3868c0 1 > prepared part key=_multipart_C1BTVJNHIM0X.2~P2A5L21U1ZXE.0 size=526504 2024-04-07T16:38:07.811 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:07.816+0000 7f346a3868c0 1 > prepared multipart upload key=C1BTVJNHIM0X upload=P2A5L21U1ZXE tag=1KVSOFNMLE2F type=write size=1575080 2024-04-07T16:38:07.822 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:07.828+0000 7f346a3868c0 1 < suggested update operation key=1WGUPH7X5UA0 tag=6BKE21R05L8I type=del size=1007763 2024-04-07T16:38:07.829 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:07.836+0000 7f346a3868c0 1 > prepared operation key=N14BUXLJONHZ tag=3XW766FIEGNW type=del size=3427247 2024-04-07T16:38:07.836 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:07.844+0000 7f346a3868c0 -1 < failed to complete operation key=1WGUPH7X5UA0 tag=T9Q8SDRMVNZF type=del size=3303770: (22) Invalid argument 2024-04-07T16:38:07.849 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:07.856+0000 7f346a3868c0 1 > prepared part key=_multipart_XJLQZ7IS9U1T.2~0JNQJ1NEUJSG.0 size=1048576 2024-04-07T16:38:07.854 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:07.860+0000 7f346a3868c0 1 > prepared part key=_multipart_XJLQZ7IS9U1T.2~0JNQJ1NEUJSG.1 size=166275 2024-04-07T16:38:07.854 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:07.860+0000 7f346a3868c0 1 > prepared multipart upload key=XJLQZ7IS9U1T upload=0JNQJ1NEUJSG tag=N4Y2MNA92MOU type=write size=2263427 2024-04-07T16:38:07.862 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:07.868+0000 7f346a3868c0 1 < canceled operation key=UIW3DN4NUYPP tag=EVXSZSAHCEGB type=del size=971682 2024-04-07T16:38:07.868 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:07.876+0000 7f346a3868c0 1 > prepared operation key=8F81G0P0YBFW tag=NZ8LDAZIP1ZA type=del size=2420012 2024-04-07T16:38:07.880 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:07.888+0000 7f346a3868c0 1 < suggested remove operation key=HBLEXONKXJTL tag=0EN6IOSGNXTD type=del 2024-04-07T16:38:07.886 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:07.892+0000 7f346a3868c0 1 > prepared operation key=HBLEXONKXJTL tag=2HSXVA70LUB4 type=del size=3198062 2024-04-07T16:38:07.892 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:07.900+0000 7f346a3868c0 1 < completed delete operation key=QTCIIPTXIRW2 tag=AB90RYX33KDS type=del 2024-04-07T16:38:07.903 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:07.912+0000 7f346a3868c0 1 > prepared part key=_multipart_QVOKSQAGHPA9.2~L9MPRPWE6C61.0 size=1048576 2024-04-07T16:38:07.908 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:07.916+0000 7f346a3868c0 1 > prepared part key=_multipart_QVOKSQAGHPA9.2~L9MPRPWE6C61.1 size=1048576 2024-04-07T16:38:07.913 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:07.920+0000 7f346a3868c0 1 > prepared part key=_multipart_QVOKSQAGHPA9.2~L9MPRPWE6C61.2 size=952436 2024-04-07T16:38:07.913 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:07.920+0000 7f346a3868c0 1 > prepared multipart upload key=QVOKSQAGHPA9 upload=L9MPRPWE6C61 tag=FX4B8S6BE8K8 type=write size=4098164 2024-04-07T16:38:07.919 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:07.928+0000 7f346a3868c0 1 < completed delete operation key=IX3F0IUOKBX1 tag=FREMUJHZB31W type=del 2024-04-07T16:38:07.925 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:07.932+0000 7f346a3868c0 1 > prepared operation key=1VKVQXSQMTU1 tag=2YYXTK7RXX2J type=del size=2920979 2024-04-07T16:38:07.931 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:07.936+0000 7f346a3868c0 1 < completed delete operation key=YXMDZB23YTB3 tag=Y8MLADT7ZIN7 type=del 2024-04-07T16:38:07.937 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:07.944+0000 7f346a3868c0 1 > prepared operation key=NYET3212USVE tag=HA8W05FSPJV5 type=del size=673014 2024-04-07T16:38:07.943 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:07.948+0000 7f346a3868c0 1 < completed part key=_multipart_YXMDZB23YTB3.2~PMZCF7YE8DWU.0 size=1048576 2024-04-07T16:38:07.948 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:07.956+0000 7f346a3868c0 1 < completed part key=_multipart_YXMDZB23YTB3.2~PMZCF7YE8DWU.1 size=706699 2024-04-07T16:38:07.959 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:07.964+0000 7f346a3868c0 1 < completed multipart upload key=YXMDZB23YTB3 upload=PMZCF7YE8DWU tag=LPYK0922P5UB type=write size=2803851 2024-04-07T16:38:07.965 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:07.972+0000 7f346a3868c0 1 > prepared operation key=EOYFVEI50SQ1 tag=26IR8OE2R5GU type=del size=2313431 2024-04-07T16:38:07.965 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:07.972+0000 7f346a3868c0 1 < canceled part key=_multipart_1WGUPH7X5UA0.2~LDN4RFCPD5FD.0 size=1048576 2024-04-07T16:38:07.970 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:07.976+0000 7f346a3868c0 1 < completed part key=_multipart_1WGUPH7X5UA0.2~LDN4RFCPD5FD.1 size=1048576 2024-04-07T16:38:07.975 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:07.984+0000 7f346a3868c0 1 < completed part key=_multipart_1WGUPH7X5UA0.2~LDN4RFCPD5FD.2 size=787849 2024-04-07T16:38:07.986 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:07.992+0000 7f346a3868c0 1 < canceled multipart upload key=1WGUPH7X5UA0 upload=LDN4RFCPD5FD tag=N4CCVW7WDEJZ type=write size=3933577 2024-04-07T16:38:07.997 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:08.004+0000 7f346a3868c0 1 > prepared part key=_multipart_HBLEXONKXJTL.2~80XMX18F0HW1.0 size=1009760 2024-04-07T16:38:07.997 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:08.004+0000 7f346a3868c0 1 > prepared multipart upload key=HBLEXONKXJTL upload=80XMX18F0HW1 tag=YLJUDCOQNITP type=write size=2058336 2024-04-07T16:38:08.003 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:08.012+0000 7f346a3868c0 1 < completed part key=_multipart_HBLEXONKXJTL.2~FZQZ4ZLGXIKN.0 size=1048576 2024-04-07T16:38:08.008 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:08.016+0000 7f346a3868c0 1 < completed part key=_multipart_HBLEXONKXJTL.2~FZQZ4ZLGXIKN.1 size=686166 2024-04-07T16:38:08.019 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:08.028+0000 7f346a3868c0 1 < completed multipart upload key=HBLEXONKXJTL upload=FZQZ4ZLGXIKN tag=WG7NT6S2BNHP type=write size=2783318 2024-04-07T16:38:08.026 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:08.032+0000 7f346a3868c0 1 > prepared operation key=C1BTVJNHIM0X tag=JE6R35ARA58D type=write size=226596 2024-04-07T16:38:08.032 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:08.040+0000 7f346a3868c0 1 < completed part key=_multipart_IP8YW2A75TCW.2~EJ1794XG0VNY.0 size=1048576 2024-04-07T16:38:08.037 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:08.044+0000 7f346a3868c0 1 < completed part key=_multipart_IP8YW2A75TCW.2~EJ1794XG0VNY.1 size=965773 2024-04-07T16:38:08.047 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:08.056+0000 7f346a3868c0 1 < completed multipart upload key=IP8YW2A75TCW upload=EJ1794XG0VNY tag=I6WMIPHX9BHP type=write size=3062925 2024-04-07T16:38:08.054 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:08.060+0000 7f346a3868c0 1 > prepared operation key=HBLEXONKXJTL tag=K62M5IWHZ0LB type=del size=2412668 2024-04-07T16:38:08.061 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:08.068+0000 7f346a3868c0 1 < completed delete operation key=ROFH30W3UCAX tag=P1NF5CXSV5QV type=del 2024-04-07T16:38:08.068 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:08.076+0000 7f346a3868c0 1 > prepared operation key=QVOKSQAGHPA9 tag=AUG5QZWRZHZ6 type=del size=3592435 2024-04-07T16:38:08.077 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:08.084+0000 7f346a3868c0 1 < completed part key=_multipart_95YVKJ14JD8W.2~XK841ONVXXR4.0 size=1048576 2024-04-07T16:38:08.082 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:08.088+0000 7f346a3868c0 1 < completed part key=_multipart_95YVKJ14JD8W.2~XK841ONVXXR4.1 size=750478 2024-04-07T16:38:08.094 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:08.100+0000 7f346a3868c0 1 < completed multipart upload key=95YVKJ14JD8W upload=XK841ONVXXR4 tag=X81C6P6QTSF3 type=write size=2847630 2024-04-07T16:38:08.100 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:08.108+0000 7f346a3868c0 1 > prepared operation key=QS4HYZFFAJNF tag=5EX21U9TS0KD type=del size=2010128 2024-04-07T16:38:08.107 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:08.112+0000 7f346a3868c0 1 < canceled operation key=EOYFVEI50SQ1 tag=SN7151QV1N5O type=del size=726882 2024-04-07T16:38:08.113 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:08.120+0000 7f346a3868c0 1 > prepared operation key=LXOZ0T2KY6OR tag=AN3JFFIHPS9S type=del size=2542276 2024-04-07T16:38:08.125 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:08.132+0000 7f346a3868c0 1 < suggested remove operation key=1VKVQXSQMTU1 tag=IDWZ7KVE8C79 type=del 2024-04-07T16:38:08.133 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:08.140+0000 7f346a3868c0 1 > prepared operation key=GBQSME7SY1AH tag=GJOH4VD7V368 type=del size=4172303 2024-04-07T16:38:08.140 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:08.148+0000 7f346a3868c0 1 < completed part key=_multipart_C1BTVJNHIM0X.2~P2A5L21U1ZXE.0 size=526504 2024-04-07T16:38:08.155 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:08.160+0000 7f346a3868c0 1 < completed multipart upload key=C1BTVJNHIM0X upload=P2A5L21U1ZXE tag=1KVSOFNMLE2F type=write size=1575080 2024-04-07T16:38:08.165 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:08.172+0000 7f346a3868c0 1 > prepared operation key=1VBSUWNI3V1T tag=89ZGEGSFEMGL type=del size=3521427 2024-04-07T16:38:08.178 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:08.184+0000 7f346a3868c0 1 < suggested update operation key=N14BUXLJONHZ tag=3XW766FIEGNW type=del size=3427247 2024-04-07T16:38:08.190 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:08.196+0000 7f346a3868c0 1 > prepared part key=_multipart_95YVKJ14JD8W.2~FZS3TSGECU1U.0 size=1048576 2024-04-07T16:38:08.197 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:08.204+0000 7f346a3868c0 1 > prepared part key=_multipart_95YVKJ14JD8W.2~FZS3TSGECU1U.1 size=958827 2024-04-07T16:38:08.197 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:08.204+0000 7f346a3868c0 1 > prepared multipart upload key=95YVKJ14JD8W upload=FZS3TSGECU1U tag=S2C0NT8ZMHU7 type=write size=3055979 2024-04-07T16:38:08.205 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:08.212+0000 7f346a3868c0 1 < completed part key=_multipart_XJLQZ7IS9U1T.2~0JNQJ1NEUJSG.0 size=1048576 2024-04-07T16:38:08.208 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:08.216+0000 7f346a3868c0 1 < completed part key=_multipart_XJLQZ7IS9U1T.2~0JNQJ1NEUJSG.1 size=166275 2024-04-07T16:38:08.218 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:08.224+0000 7f346a3868c0 1 < completed multipart upload key=XJLQZ7IS9U1T upload=0JNQJ1NEUJSG tag=N4Y2MNA92MOU type=write size=2263427 2024-04-07T16:38:08.224 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:08.232+0000 7f346a3868c0 1 > prepared operation key=IP8YW2A75TCW tag=M3SD9HO9MDPK type=del size=162938 2024-04-07T16:38:08.229 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:08.236+0000 7f346a3868c0 1 < completed delete operation key=8F81G0P0YBFW tag=NZ8LDAZIP1ZA type=del 2024-04-07T16:38:08.234 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:08.240+0000 7f346a3868c0 1 > prepared operation key=EOYFVEI50SQ1 tag=W926XMT8DQXF type=write size=609430 2024-04-07T16:38:08.245 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:08.252+0000 7f346a3868c0 1 < suggested remove operation key=HBLEXONKXJTL tag=2HSXVA70LUB4 type=del 2024-04-07T16:38:08.256 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:08.264+0000 7f346a3868c0 1 > prepared part key=_multipart_YOK8BAEOMPUR.2~8KRZVGGZW8V7.0 size=385723 2024-04-07T16:38:08.256 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:08.264+0000 7f346a3868c0 1 > prepared multipart upload key=YOK8BAEOMPUR upload=8KRZVGGZW8V7 tag=SPUZPAC3RZZO type=write size=1434299 2024-04-07T16:38:08.262 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:08.268+0000 7f346a3868c0 1 < completed part key=_multipart_QVOKSQAGHPA9.2~L9MPRPWE6C61.0 size=1048576 2024-04-07T16:38:08.267 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:08.272+0000 7f346a3868c0 1 < completed part key=_multipart_QVOKSQAGHPA9.2~L9MPRPWE6C61.1 size=1048576 2024-04-07T16:38:08.272 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:08.280+0000 7f346a3868c0 1 < completed part key=_multipart_QVOKSQAGHPA9.2~L9MPRPWE6C61.2 size=952436 2024-04-07T16:38:08.286 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:08.292+0000 7f346a3868c0 1 < completed multipart upload key=QVOKSQAGHPA9 upload=L9MPRPWE6C61 tag=FX4B8S6BE8K8 type=write size=4098164 2024-04-07T16:38:08.293 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:08.300+0000 7f346a3868c0 1 > prepared operation key=U3EID5EK64HD tag=8ZYIVAOJ6FMK type=write size=593110 2024-04-07T16:38:08.301 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:08.308+0000 7f346a3868c0 -1 < failed to complete operation key=1VKVQXSQMTU1 tag=2YYXTK7RXX2J type=del size=2920979: (22) Invalid argument 2024-04-07T16:38:08.307 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:08.316+0000 7f346a3868c0 1 > prepared operation key=E84U0SX44NQ8 tag=UTSJ8JXR420L type=del size=3931040 2024-04-07T16:38:08.314 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:08.320+0000 7f346a3868c0 1 < completed delete operation key=NYET3212USVE tag=HA8W05FSPJV5 type=del 2024-04-07T16:38:08.319 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:08.328+0000 7f346a3868c0 1 > prepared operation key=NYET3212USVE tag=D8QTXR6UX8D2 type=write size=785425 2024-04-07T16:38:08.332 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:08.340+0000 7f346a3868c0 1 < suggested update operation key=EOYFVEI50SQ1 tag=26IR8OE2R5GU type=del size=2313431 2024-04-07T16:38:08.343 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:08.348+0000 7f346a3868c0 1 > prepared part key=_multipart_UIW3DN4NUYPP.2~TAZ5FKBZK618.0 size=1048576 2024-04-07T16:38:08.347 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:08.356+0000 7f346a3868c0 1 > prepared part key=_multipart_UIW3DN4NUYPP.2~TAZ5FKBZK618.1 size=1048576 2024-04-07T16:38:08.355 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:08.364+0000 7f346a3868c0 1 > prepared part key=_multipart_UIW3DN4NUYPP.2~TAZ5FKBZK618.2 size=378124 2024-04-07T16:38:08.355 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:08.364+0000 7f346a3868c0 1 > prepared multipart upload key=UIW3DN4NUYPP upload=TAZ5FKBZK618 tag=1ACMUEY7TTZZ type=write size=3523852 2024-04-07T16:38:08.363 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:08.368+0000 7f346a3868c0 1 < completed part key=_multipart_HBLEXONKXJTL.2~80XMX18F0HW1.0 size=1009760 2024-04-07T16:38:08.373 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:08.380+0000 7f346a3868c0 1 < completed multipart upload key=HBLEXONKXJTL upload=80XMX18F0HW1 tag=YLJUDCOQNITP type=write size=2058336 2024-04-07T16:38:08.380 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:08.388+0000 7f346a3868c0 1 > prepared operation key=LXOZ0T2KY6OR tag=KXKD27PIR54S type=write size=288144 2024-04-07T16:38:08.385 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:08.392+0000 7f346a3868c0 1 < completed write operation key=C1BTVJNHIM0X tag=JE6R35ARA58D type=write size=226596 2024-04-07T16:38:08.396 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:08.404+0000 7f346a3868c0 1 > prepared part key=_multipart_NYET3212USVE.2~IBZZJEHJF5BQ.0 size=93480 2024-04-07T16:38:08.396 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:08.404+0000 7f346a3868c0 1 > prepared multipart upload key=NYET3212USVE upload=IBZZJEHJF5BQ tag=SVU8ZAAGUCYT type=write size=1142056 2024-04-07T16:38:08.404 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:08.412+0000 7f346a3868c0 -1 < failed to complete operation key=HBLEXONKXJTL tag=K62M5IWHZ0LB type=del size=2412668: (22) Invalid argument 2024-04-07T16:38:08.410 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:08.416+0000 7f346a3868c0 1 > prepared operation key=1VBSUWNI3V1T tag=3YFU8A5UO6YC type=write size=415983 2024-04-07T16:38:08.416 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:08.424+0000 7f346a3868c0 1 < completed delete operation key=QVOKSQAGHPA9 tag=AUG5QZWRZHZ6 type=del 2024-04-07T16:38:08.418 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:38:08 smithi100 bash[74942]: cluster 2024-04-07T16:38:07.068334+0000 mgr.y (mgr.25175) 683 : cluster [DBG] pgmap v1318: 137 pgs: 137 active+clean; 583 KiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 379 KiB/s rd, 129 KiB/s wr, 670 op/s 2024-04-07T16:38:08.418 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:38:08 smithi100 bash[76202]: cluster 2024-04-07T16:38:07.068334+0000 mgr.y (mgr.25175) 683 : cluster [DBG] pgmap v1318: 137 pgs: 137 active+clean; 583 KiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 379 KiB/s rd, 129 KiB/s wr, 670 op/s 2024-04-07T16:38:08.422 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:08.428+0000 7f346a3868c0 1 > prepared operation key=X5Q46GO8YTOK tag=T1O9H5WUTSU7 type=del size=307189 2024-04-07T16:38:08.428 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:08.436+0000 7f346a3868c0 1 < completed delete operation key=QS4HYZFFAJNF tag=5EX21U9TS0KD type=del 2024-04-07T16:38:08.433 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:08.440+0000 7f346a3868c0 1 > prepared operation key=GBQSME7SY1AH tag=NQ3RKW29X4O9 type=del size=1049190 2024-04-07T16:38:08.439 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:08.448+0000 7f346a3868c0 1 < completed delete operation key=LXOZ0T2KY6OR tag=AN3JFFIHPS9S type=del 2024-04-07T16:38:08.450 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:08.456+0000 7f346a3868c0 1 > prepared part key=_multipart_E84U0SX44NQ8.2~QOJUQAW51TP7.0 size=1048576 2024-04-07T16:38:08.454 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:08.460+0000 7f346a3868c0 1 > prepared part key=_multipart_E84U0SX44NQ8.2~QOJUQAW51TP7.1 size=903480 2024-04-07T16:38:08.454 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:08.460+0000 7f346a3868c0 1 > prepared multipart upload key=E84U0SX44NQ8 upload=QOJUQAW51TP7 tag=W4YR6WIIONJL type=write size=3000632 2024-04-07T16:38:08.460 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:08.468+0000 7f346a3868c0 1 < completed delete operation key=GBQSME7SY1AH tag=GJOH4VD7V368 type=del 2024-04-07T16:38:08.470 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:08.476+0000 7f346a3868c0 1 > prepared part key=_multipart_QTCIIPTXIRW2.2~LPRKOX72I8NK.0 size=1048576 2024-04-07T16:38:08.475 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:08.484+0000 7f346a3868c0 1 > prepared part key=_multipart_QTCIIPTXIRW2.2~LPRKOX72I8NK.1 size=1048576 2024-04-07T16:38:08.480 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:08.488+0000 7f346a3868c0 1 > prepared part key=_multipart_QTCIIPTXIRW2.2~LPRKOX72I8NK.2 size=67253 2024-04-07T16:38:08.480 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:08.488+0000 7f346a3868c0 1 > prepared multipart upload key=QTCIIPTXIRW2 upload=LPRKOX72I8NK tag=68WE6NW8AM8W type=write size=3212981 2024-04-07T16:38:08.486 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:08.492+0000 7f346a3868c0 1 < completed delete operation key=1VBSUWNI3V1T tag=89ZGEGSFEMGL type=del 2024-04-07T16:38:08.492 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:08.500+0000 7f346a3868c0 1 > prepared operation key=8F81G0P0YBFW tag=0ZE1I49FD8LG type=del size=8642 2024-04-07T16:38:08.498 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:08.504+0000 7f346a3868c0 1 < completed part key=_multipart_95YVKJ14JD8W.2~FZS3TSGECU1U.0 size=1048576 2024-04-07T16:38:08.503 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:08.512+0000 7f346a3868c0 1 < completed part key=_multipart_95YVKJ14JD8W.2~FZS3TSGECU1U.1 size=958827 2024-04-07T16:38:08.513 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:08.520+0000 7f346a3868c0 1 < completed multipart upload key=95YVKJ14JD8W upload=FZS3TSGECU1U tag=S2C0NT8ZMHU7 type=write size=3055979 2024-04-07T16:38:08.519 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:08.524+0000 7f346a3868c0 1 > prepared operation key=7G5C6HBK6QWI tag=QUU22UNEKM73 type=del size=439272 2024-04-07T16:38:08.525 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:08.532+0000 7f346a3868c0 1 < completed delete operation key=IP8YW2A75TCW tag=M3SD9HO9MDPK type=del 2024-04-07T16:38:08.531 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:08.540+0000 7f346a3868c0 1 > prepared operation key=U3EID5EK64HD tag=ABNPF35PQFMY type=del size=3191900 2024-04-07T16:38:08.544 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:08.552+0000 7f346a3868c0 1 < suggested remove operation key=EOYFVEI50SQ1 tag=W926XMT8DQXF type=write 2024-04-07T16:38:08.550 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:08.556+0000 7f346a3868c0 1 > prepared operation key=95YVKJ14JD8W tag=4M50PE422VEK type=del size=523820 2024-04-07T16:38:08.556 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:08.564+0000 7f346a3868c0 1 < completed part key=_multipart_YOK8BAEOMPUR.2~8KRZVGGZW8V7.0 size=385723 2024-04-07T16:38:08.566 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:08.572+0000 7f346a3868c0 1 < completed multipart upload key=YOK8BAEOMPUR upload=8KRZVGGZW8V7 tag=SPUZPAC3RZZO type=write size=1434299 2024-04-07T16:38:08.577 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:08.584+0000 7f346a3868c0 1 > prepared part key=_multipart_IP8YW2A75TCW.2~XP1MFMWHW9L3.0 size=1048576 2024-04-07T16:38:08.582 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:08.588+0000 7f346a3868c0 1 > prepared part key=_multipart_IP8YW2A75TCW.2~XP1MFMWHW9L3.1 size=1048576 2024-04-07T16:38:08.587 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:08.592+0000 7f346a3868c0 1 > prepared part key=_multipart_IP8YW2A75TCW.2~XP1MFMWHW9L3.2 size=132282 2024-04-07T16:38:08.587 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:08.592+0000 7f346a3868c0 1 > prepared multipart upload key=IP8YW2A75TCW upload=XP1MFMWHW9L3 tag=622RBDWAY6C9 type=write size=3278010 2024-04-07T16:38:08.593 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:08.600+0000 7f346a3868c0 1 < completed write operation key=U3EID5EK64HD tag=8ZYIVAOJ6FMK type=write size=593110 2024-04-07T16:38:08.599 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:08.608+0000 7f346a3868c0 1 > prepared operation key=1VKVQXSQMTU1 tag=PWDL8V3AJBCZ type=write size=241529 2024-04-07T16:38:08.606 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:08.612+0000 7f346a3868c0 1 < completed delete operation key=E84U0SX44NQ8 tag=UTSJ8JXR420L type=del 2024-04-07T16:38:08.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:38:08 smithi144 bash[66654]: cluster 2024-04-07T16:38:07.068334+0000 mgr.y (mgr.25175) 683 : cluster [DBG] pgmap v1318: 137 pgs: 137 active+clean; 583 KiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 379 KiB/s rd, 129 KiB/s wr, 670 op/s 2024-04-07T16:38:08.613 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:08.620+0000 7f346a3868c0 1 > prepared operation key=IP8YW2A75TCW tag=C0R07TPRC4G7 type=write size=220095 2024-04-07T16:38:08.622 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:08.628+0000 7f346a3868c0 1 < suggested remove operation key=NYET3212USVE tag=D8QTXR6UX8D2 type=write 2024-04-07T16:38:08.628 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:08.636+0000 7f346a3868c0 1 > prepared operation key=HBLEXONKXJTL tag=5GYD8E7WEFJP type=del size=158158 2024-04-07T16:38:08.634 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:08.640+0000 7f346a3868c0 1 < completed part key=_multipart_UIW3DN4NUYPP.2~TAZ5FKBZK618.0 size=1048576 2024-04-07T16:38:08.639 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:08.648+0000 7f346a3868c0 1 < completed part key=_multipart_UIW3DN4NUYPP.2~TAZ5FKBZK618.1 size=1048576 2024-04-07T16:38:08.644 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:08.652+0000 7f346a3868c0 1 < completed part key=_multipart_UIW3DN4NUYPP.2~TAZ5FKBZK618.2 size=378124 2024-04-07T16:38:08.654 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:08.660+0000 7f346a3868c0 1 < completed multipart upload key=UIW3DN4NUYPP upload=TAZ5FKBZK618 tag=1ACMUEY7TTZZ type=write size=3523852 2024-04-07T16:38:08.659 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:08.668+0000 7f346a3868c0 1 > prepared operation key=YXMDZB23YTB3 tag=DO23VXX5EZVV type=del size=3928627 2024-04-07T16:38:08.665 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:08.672+0000 7f346a3868c0 1 < completed write operation key=LXOZ0T2KY6OR tag=KXKD27PIR54S type=write size=288144 2024-04-07T16:38:08.678 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:08.684+0000 7f346a3868c0 1 > prepared part key=_multipart_95YVKJ14JD8W.2~B5UQVIE305FR.0 size=1048576 2024-04-07T16:38:08.683 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:08.688+0000 7f346a3868c0 1 > prepared part key=_multipart_95YVKJ14JD8W.2~B5UQVIE305FR.1 size=949221 2024-04-07T16:38:08.683 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:08.692+0000 7f346a3868c0 1 > prepared multipart upload key=95YVKJ14JD8W upload=B5UQVIE305FR tag=XK94L2S155BC type=write size=3046373 2024-04-07T16:38:08.689 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:08.696+0000 7f346a3868c0 1 < completed part key=_multipart_NYET3212USVE.2~IBZZJEHJF5BQ.0 size=93480 2024-04-07T16:38:08.691 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:38:08 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:38:08] "GET /metrics HTTP/1.1" 200 36370 "" "Prometheus/2.43.0" 2024-04-07T16:38:08.699 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:08.704+0000 7f346a3868c0 1 < completed multipart upload key=NYET3212USVE upload=IBZZJEHJF5BQ tag=SVU8ZAAGUCYT type=write size=1142056 2024-04-07T16:38:08.710 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:08.716+0000 7f346a3868c0 1 > prepared part key=_multipart_QS4HYZFFAJNF.2~W2L4M6MQM870.0 size=1048576 2024-04-07T16:38:08.715 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:08.724+0000 7f346a3868c0 1 > prepared part key=_multipart_QS4HYZFFAJNF.2~W2L4M6MQM870.1 size=553180 2024-04-07T16:38:08.715 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:08.724+0000 7f346a3868c0 1 > prepared multipart upload key=QS4HYZFFAJNF upload=W2L4M6MQM870 tag=RIICT07KIZSP type=write size=2650332 2024-04-07T16:38:08.721 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:08.728+0000 7f346a3868c0 1 < completed write operation key=1VBSUWNI3V1T tag=3YFU8A5UO6YC type=write size=415983 2024-04-07T16:38:08.732 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:08.740+0000 7f346a3868c0 1 > prepared part key=_multipart_C1BTVJNHIM0X.2~FWT2T7K4PDQU.0 size=1048576 2024-04-07T16:38:08.737 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:08.744+0000 7f346a3868c0 1 > prepared part key=_multipart_C1BTVJNHIM0X.2~FWT2T7K4PDQU.1 size=1048576 2024-04-07T16:38:08.742 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:08.748+0000 7f346a3868c0 1 > prepared part key=_multipart_C1BTVJNHIM0X.2~FWT2T7K4PDQU.2 size=25198 2024-04-07T16:38:08.742 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:08.748+0000 7f346a3868c0 1 > prepared multipart upload key=C1BTVJNHIM0X upload=FWT2T7K4PDQU tag=TKQ6IBVDO81Y type=write size=3170926 2024-04-07T16:38:08.749 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:08.756+0000 7f346a3868c0 1 < completed delete operation key=X5Q46GO8YTOK tag=T1O9H5WUTSU7 type=del 2024-04-07T16:38:08.760 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:08.768+0000 7f346a3868c0 1 > prepared part key=_multipart_E84U0SX44NQ8.2~LJSNIKDQ8M00.0 size=1048576 2024-04-07T16:38:08.766 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:08.772+0000 7f346a3868c0 1 > prepared part key=_multipart_E84U0SX44NQ8.2~LJSNIKDQ8M00.1 size=1048576 2024-04-07T16:38:08.771 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:08.776+0000 7f346a3868c0 1 > prepared part key=_multipart_E84U0SX44NQ8.2~LJSNIKDQ8M00.2 size=380553 2024-04-07T16:38:08.771 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:08.776+0000 7f346a3868c0 1 > prepared multipart upload key=E84U0SX44NQ8 upload=LJSNIKDQ8M00 tag=A4TILXJR77S8 type=write size=3526281 2024-04-07T16:38:08.777 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:08.784+0000 7f346a3868c0 1 < completed delete operation key=GBQSME7SY1AH tag=NQ3RKW29X4O9 type=del 2024-04-07T16:38:08.783 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:08.788+0000 7f346a3868c0 1 > prepared operation key=6XDS0A7L0IY8 tag=0BJL261RZDY8 type=del size=144336 2024-04-07T16:38:08.789 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:08.796+0000 7f346a3868c0 1 < completed part key=_multipart_E84U0SX44NQ8.2~QOJUQAW51TP7.0 size=1048576 2024-04-07T16:38:08.793 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:08.800+0000 7f346a3868c0 1 < completed part key=_multipart_E84U0SX44NQ8.2~QOJUQAW51TP7.1 size=903480 2024-04-07T16:38:08.803 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:08.812+0000 7f346a3868c0 1 < completed multipart upload key=E84U0SX44NQ8 upload=QOJUQAW51TP7 tag=W4YR6WIIONJL type=write size=3000632 2024-04-07T16:38:08.815 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:08.824+0000 7f346a3868c0 1 > prepared part key=_multipart_ROFH30W3UCAX.2~J51IYC9X3TLA.0 size=188799 2024-04-07T16:38:08.815 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:08.824+0000 7f346a3868c0 1 > prepared multipart upload key=ROFH30W3UCAX upload=J51IYC9X3TLA tag=G8U7AS8PKAK7 type=write size=1237375 2024-04-07T16:38:08.822 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:08.828+0000 7f346a3868c0 1 < completed part key=_multipart_QTCIIPTXIRW2.2~LPRKOX72I8NK.0 size=1048576 2024-04-07T16:38:08.827 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:08.836+0000 7f346a3868c0 1 < completed part key=_multipart_QTCIIPTXIRW2.2~LPRKOX72I8NK.1 size=1048576 2024-04-07T16:38:08.833 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:08.840+0000 7f346a3868c0 1 < completed part key=_multipart_QTCIIPTXIRW2.2~LPRKOX72I8NK.2 size=67253 2024-04-07T16:38:08.842 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:08.848+0000 7f346a3868c0 1 < completed multipart upload key=QTCIIPTXIRW2 upload=LPRKOX72I8NK tag=68WE6NW8AM8W type=write size=3212981 2024-04-07T16:38:08.848 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:08.856+0000 7f346a3868c0 1 > prepared operation key=EQR0FY86AT5X tag=DV5D6V5LX0PT type=write size=1016502 2024-04-07T16:38:08.859 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:08.868+0000 7f346a3868c0 1 < suggested remove operation key=8F81G0P0YBFW tag=0ZE1I49FD8LG type=del 2024-04-07T16:38:08.870 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:08.876+0000 7f346a3868c0 1 > prepared part key=_multipart_1VBSUWNI3V1T.2~D67N2GTDWFF2.0 size=1048576 2024-04-07T16:38:08.875 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:08.880+0000 7f346a3868c0 1 > prepared part key=_multipart_1VBSUWNI3V1T.2~D67N2GTDWFF2.1 size=917533 2024-04-07T16:38:08.875 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:08.880+0000 7f346a3868c0 1 > prepared multipart upload key=1VBSUWNI3V1T upload=D67N2GTDWFF2 tag=OF9ISKL0SJW2 type=write size=3014685 2024-04-07T16:38:08.880 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:08.888+0000 7f346a3868c0 1 < completed delete operation key=7G5C6HBK6QWI tag=QUU22UNEKM73 type=del 2024-04-07T16:38:08.891 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:08.900+0000 7f346a3868c0 1 > prepared part key=_multipart_EQR0FY86AT5X.2~5B5MYC9DMZF1.0 size=1048576 2024-04-07T16:38:08.897 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:08.904+0000 7f346a3868c0 1 > prepared part key=_multipart_EQR0FY86AT5X.2~5B5MYC9DMZF1.1 size=1048576 2024-04-07T16:38:08.902 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:08.908+0000 7f346a3868c0 1 > prepared part key=_multipart_EQR0FY86AT5X.2~5B5MYC9DMZF1.2 size=268645 2024-04-07T16:38:08.902 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:08.908+0000 7f346a3868c0 1 > prepared multipart upload key=EQR0FY86AT5X upload=5B5MYC9DMZF1 tag=IJK4KIZIM2QB type=write size=3414373 2024-04-07T16:38:08.908 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:08.916+0000 7f346a3868c0 1 < canceled operation key=U3EID5EK64HD tag=ABNPF35PQFMY type=del size=3191900 2024-04-07T16:38:08.914 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:08.920+0000 7f346a3868c0 1 > prepared operation key=95YVKJ14JD8W tag=WDTMK5XU3GHL type=del size=2651751 2024-04-07T16:38:08.920 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:08.928+0000 7f346a3868c0 1 < canceled operation key=95YVKJ14JD8W tag=4M50PE422VEK type=del size=523820 2024-04-07T16:38:08.927 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:08.936+0000 7f346a3868c0 1 > prepared operation key=EQR0FY86AT5X tag=JAFJLDZ64MKS type=del size=3107470 2024-04-07T16:38:08.933 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:08.940+0000 7f346a3868c0 1 < completed part key=_multipart_IP8YW2A75TCW.2~XP1MFMWHW9L3.0 size=1048576 2024-04-07T16:38:08.938 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:08.944+0000 7f346a3868c0 1 < completed part key=_multipart_IP8YW2A75TCW.2~XP1MFMWHW9L3.1 size=1048576 2024-04-07T16:38:08.943 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:08.952+0000 7f346a3868c0 1 < completed part key=_multipart_IP8YW2A75TCW.2~XP1MFMWHW9L3.2 size=132282 2024-04-07T16:38:08.953 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:08.960+0000 7f346a3868c0 1 < completed multipart upload key=IP8YW2A75TCW upload=XP1MFMWHW9L3 tag=622RBDWAY6C9 type=write size=3278010 2024-04-07T16:38:08.959 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:08.968+0000 7f346a3868c0 1 > prepared operation key=LXOZ0T2KY6OR tag=IGWLCWYYAX7A type=write size=982493 2024-04-07T16:38:08.970 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:08.976+0000 7f346a3868c0 1 < suggested remove operation key=1VKVQXSQMTU1 tag=PWDL8V3AJBCZ type=write 2024-04-07T16:38:08.976 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:08.984+0000 7f346a3868c0 1 > prepared operation key=YXMDZB23YTB3 tag=EVMJM0VXS4IK type=del size=3511459 2024-04-07T16:38:08.989 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:08.996+0000 7f346a3868c0 1 < suggested remove operation key=IP8YW2A75TCW tag=C0R07TPRC4G7 type=write 2024-04-07T16:38:08.995 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:09.004+0000 7f346a3868c0 1 > prepared operation key=1VKVQXSQMTU1 tag=6YLM10SU1DDQ type=del size=550399 2024-04-07T16:38:09.008 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:09.016+0000 7f346a3868c0 1 < suggested update operation key=HBLEXONKXJTL tag=5GYD8E7WEFJP type=del size=158158 2024-04-07T16:38:09.013 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:09.020+0000 7f346a3868c0 1 > prepared operation key=XJLQZ7IS9U1T tag=9RL3F0RADKBP type=del size=1388705 2024-04-07T16:38:09.019 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:09.028+0000 7f346a3868c0 1 < completed delete operation key=YXMDZB23YTB3 tag=DO23VXX5EZVV type=del 2024-04-07T16:38:09.030 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:09.036+0000 7f346a3868c0 1 > prepared part key=_multipart_95YVKJ14JD8W.2~6BZQSOJVL4K2.0 size=299769 2024-04-07T16:38:09.030 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:09.036+0000 7f346a3868c0 1 > prepared multipart upload key=95YVKJ14JD8W upload=6BZQSOJVL4K2 tag=OEEZDKOW5AX7 type=write size=1348345 2024-04-07T16:38:09.036 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:09.044+0000 7f346a3868c0 1 < completed part key=_multipart_95YVKJ14JD8W.2~B5UQVIE305FR.0 size=1048576 2024-04-07T16:38:09.041 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:09.048+0000 7f346a3868c0 1 < completed part key=_multipart_95YVKJ14JD8W.2~B5UQVIE305FR.1 size=949221 2024-04-07T16:38:09.052 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:09.060+0000 7f346a3868c0 1 < completed multipart upload key=95YVKJ14JD8W upload=B5UQVIE305FR tag=XK94L2S155BC type=write size=3046373 2024-04-07T16:38:09.057 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:09.064+0000 7f346a3868c0 1 > prepared operation key=UIW3DN4NUYPP tag=RFIBR7NMONV5 type=del size=1592762 2024-04-07T16:38:09.064 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:09.072+0000 7f346a3868c0 1 < completed part key=_multipart_QS4HYZFFAJNF.2~W2L4M6MQM870.0 size=1048576 2024-04-07T16:38:09.069 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:09.076+0000 7f346a3868c0 1 < completed part key=_multipart_QS4HYZFFAJNF.2~W2L4M6MQM870.1 size=553180 2024-04-07T16:38:09.079 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:09.088+0000 7f346a3868c0 1 < completed multipart upload key=QS4HYZFFAJNF upload=W2L4M6MQM870 tag=RIICT07KIZSP type=write size=2650332 2024-04-07T16:38:09.090 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:09.096+0000 7f346a3868c0 1 > prepared part key=_multipart_YOK8BAEOMPUR.2~XE8EZSWIPVOI.0 size=1048576 2024-04-07T16:38:09.095 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:09.100+0000 7f346a3868c0 1 > prepared part key=_multipart_YOK8BAEOMPUR.2~XE8EZSWIPVOI.1 size=686330 2024-04-07T16:38:09.095 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:09.100+0000 7f346a3868c0 1 > prepared multipart upload key=YOK8BAEOMPUR upload=XE8EZSWIPVOI tag=DPCZGAWMWHNZ type=write size=2783482 2024-04-07T16:38:09.100 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:09.108+0000 7f346a3868c0 1 < completed part key=_multipart_C1BTVJNHIM0X.2~FWT2T7K4PDQU.0 size=1048576 2024-04-07T16:38:09.106 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:09.112+0000 7f346a3868c0 1 < completed part key=_multipart_C1BTVJNHIM0X.2~FWT2T7K4PDQU.1 size=1048576 2024-04-07T16:38:09.110 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:09.116+0000 7f346a3868c0 1 < completed part key=_multipart_C1BTVJNHIM0X.2~FWT2T7K4PDQU.2 size=25198 2024-04-07T16:38:09.121 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:09.128+0000 7f346a3868c0 1 < completed multipart upload key=C1BTVJNHIM0X upload=FWT2T7K4PDQU tag=TKQ6IBVDO81Y type=write size=3170926 2024-04-07T16:38:09.127 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:09.132+0000 7f346a3868c0 1 > prepared operation key=95YVKJ14JD8W tag=Y57PDU7SDS3Y type=write size=75419 2024-04-07T16:38:09.132 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:09.140+0000 7f346a3868c0 1 < completed part key=_multipart_E84U0SX44NQ8.2~LJSNIKDQ8M00.0 size=1048576 2024-04-07T16:38:09.140 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:09.148+0000 7f346a3868c0 1 < completed part key=_multipart_E84U0SX44NQ8.2~LJSNIKDQ8M00.1 size=1048576 2024-04-07T16:38:09.145 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:09.152+0000 7f346a3868c0 1 < completed part key=_multipart_E84U0SX44NQ8.2~LJSNIKDQ8M00.2 size=380553 2024-04-07T16:38:09.156 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:09.164+0000 7f346a3868c0 1 < completed multipart upload key=E84U0SX44NQ8 upload=LJSNIKDQ8M00 tag=A4TILXJR77S8 type=write size=3526281 2024-04-07T16:38:09.161 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:09.168+0000 7f346a3868c0 1 > prepared operation key=QS4HYZFFAJNF tag=M6HDTU47VNJ6 type=del size=4141960 2024-04-07T16:38:09.166 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:09.172+0000 7f346a3868c0 1 < completed delete operation key=6XDS0A7L0IY8 tag=0BJL261RZDY8 type=del 2024-04-07T16:38:09.172 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:09.180+0000 7f346a3868c0 1 > prepared operation key=ROFH30W3UCAX tag=MAS36FGI3T2G type=del size=3476205 2024-04-07T16:38:09.177 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:09.184+0000 7f346a3868c0 1 < completed part key=_multipart_ROFH30W3UCAX.2~J51IYC9X3TLA.0 size=188799 2024-04-07T16:38:09.186 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:09.192+0000 7f346a3868c0 1 < completed multipart upload key=ROFH30W3UCAX upload=J51IYC9X3TLA tag=G8U7AS8PKAK7 type=write size=1237375 2024-04-07T16:38:09.196 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:09.204+0000 7f346a3868c0 1 > prepared part key=_multipart_EOYFVEI50SQ1.2~05IM5576JEHK.0 size=1048576 2024-04-07T16:38:09.201 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:09.208+0000 7f346a3868c0 1 > prepared part key=_multipart_EOYFVEI50SQ1.2~05IM5576JEHK.1 size=1048576 2024-04-07T16:38:09.206 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:09.212+0000 7f346a3868c0 1 > prepared part key=_multipart_EOYFVEI50SQ1.2~05IM5576JEHK.2 size=282226 2024-04-07T16:38:09.206 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:09.212+0000 7f346a3868c0 1 > prepared multipart upload key=EOYFVEI50SQ1 upload=05IM5576JEHK tag=6GWWWOXWRMWW type=write size=3427954 2024-04-07T16:38:09.212 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:09.220+0000 7f346a3868c0 1 < completed write operation key=EQR0FY86AT5X tag=DV5D6V5LX0PT type=write size=1016502 2024-04-07T16:38:09.217 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:09.224+0000 7f346a3868c0 1 > prepared operation key=EOYFVEI50SQ1 tag=4BC1PGJ6HBB1 type=write size=17884 2024-04-07T16:38:09.223 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:09.232+0000 7f346a3868c0 1 < completed part key=_multipart_1VBSUWNI3V1T.2~D67N2GTDWFF2.0 size=1048576 2024-04-07T16:38:09.229 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:09.236+0000 7f346a3868c0 1 < completed part key=_multipart_1VBSUWNI3V1T.2~D67N2GTDWFF2.1 size=917533 2024-04-07T16:38:09.237 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:09.244+0000 7f346a3868c0 1 < completed multipart upload key=1VBSUWNI3V1T upload=D67N2GTDWFF2 tag=OF9ISKL0SJW2 type=write size=3014685 2024-04-07T16:38:09.243 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:09.252+0000 7f346a3868c0 1 > prepared operation key=QTCIIPTXIRW2 tag=2ULJSV1R2GIJ type=del size=2804435 2024-04-07T16:38:09.248 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:09.256+0000 7f346a3868c0 1 < completed part key=_multipart_EQR0FY86AT5X.2~5B5MYC9DMZF1.0 size=1048576 2024-04-07T16:38:09.253 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:09.260+0000 7f346a3868c0 1 < completed part key=_multipart_EQR0FY86AT5X.2~5B5MYC9DMZF1.1 size=1048576 2024-04-07T16:38:09.258 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:09.264+0000 7f346a3868c0 1 < completed part key=_multipart_EQR0FY86AT5X.2~5B5MYC9DMZF1.2 size=268645 2024-04-07T16:38:09.269 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:09.276+0000 7f346a3868c0 1 < completed multipart upload key=EQR0FY86AT5X upload=5B5MYC9DMZF1 tag=IJK4KIZIM2QB type=write size=3414373 2024-04-07T16:38:09.274 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:09.280+0000 7f346a3868c0 1 > prepared operation key=DXTO3AJ429AC tag=5F0FS36PX5QT type=del size=3521201 2024-04-07T16:38:09.286 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:09.292+0000 7f346a3868c0 1 < suggested remove operation key=95YVKJ14JD8W tag=WDTMK5XU3GHL type=del 2024-04-07T16:38:09.299 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:09.304+0000 7f346a3868c0 1 > prepared part key=_multipart_IP8YW2A75TCW.2~LNLOWRAXWZNR.0 size=211936 2024-04-07T16:38:09.299 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:09.304+0000 7f346a3868c0 1 > prepared multipart upload key=IP8YW2A75TCW upload=LNLOWRAXWZNR tag=Q2YK9C7XW8U6 type=write size=1260512 2024-04-07T16:38:09.310 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:09.316+0000 7f346a3868c0 1 < suggested update operation key=EQR0FY86AT5X tag=JAFJLDZ64MKS type=del size=3107470 2024-04-07T16:38:09.318 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:09.324+0000 7f346a3868c0 1 > prepared operation key=6XDS0A7L0IY8 tag=3JB9BODZCVAK type=del size=72382 2024-04-07T16:38:09.324 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:09.332+0000 7f346a3868c0 1 < completed write operation key=LXOZ0T2KY6OR tag=IGWLCWYYAX7A type=write size=982493 2024-04-07T16:38:09.335 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:09.340+0000 7f346a3868c0 1 > prepared part key=_multipart_YOK8BAEOMPUR.2~JU7E1LCVAPMH.0 size=1048576 2024-04-07T16:38:09.339 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:09.348+0000 7f346a3868c0 1 > prepared part key=_multipart_YOK8BAEOMPUR.2~JU7E1LCVAPMH.1 size=1048576 2024-04-07T16:38:09.344 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:09.352+0000 7f346a3868c0 1 > prepared part key=_multipart_YOK8BAEOMPUR.2~JU7E1LCVAPMH.2 size=145096 2024-04-07T16:38:09.344 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:09.352+0000 7f346a3868c0 1 > prepared multipart upload key=YOK8BAEOMPUR upload=JU7E1LCVAPMH tag=03WBHRBUQA67 type=write size=3290824 2024-04-07T16:38:09.350 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:09.356+0000 7f346a3868c0 1 < completed delete operation key=YXMDZB23YTB3 tag=EVMJM0VXS4IK type=del 2024-04-07T16:38:09.362 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:09.368+0000 7f346a3868c0 1 > prepared part key=_multipart_HBLEXONKXJTL.2~6PH3TGHSW0BM.0 size=1048576 2024-04-07T16:38:09.367 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:09.376+0000 7f346a3868c0 1 > prepared part key=_multipart_HBLEXONKXJTL.2~6PH3TGHSW0BM.1 size=971314 2024-04-07T16:38:09.367 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:09.376+0000 7f346a3868c0 1 > prepared multipart upload key=HBLEXONKXJTL upload=6PH3TGHSW0BM tag=XMKJ5FUQQ5WU type=write size=3068466 2024-04-07T16:38:09.373 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:09.380+0000 7f346a3868c0 1 < completed delete operation key=1VKVQXSQMTU1 tag=6YLM10SU1DDQ type=del 2024-04-07T16:38:09.379 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:09.388+0000 7f346a3868c0 1 > prepared operation key=GZXHYZO5SMUP tag=NUL6XWR2BJGG type=del size=2506065 2024-04-07T16:38:09.385 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:09.392+0000 7f346a3868c0 1 < completed delete operation key=XJLQZ7IS9U1T tag=9RL3F0RADKBP type=del 2024-04-07T16:38:09.391 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:09.400+0000 7f346a3868c0 1 > prepared operation key=8F81G0P0YBFW tag=XK3NKOBXT4YS type=del size=3999531 2024-04-07T16:38:09.397 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:09.404+0000 7f346a3868c0 1 < completed part key=_multipart_95YVKJ14JD8W.2~6BZQSOJVL4K2.0 size=299769 2024-04-07T16:38:09.407 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:09.416+0000 7f346a3868c0 1 < completed multipart upload key=95YVKJ14JD8W upload=6BZQSOJVL4K2 tag=OEEZDKOW5AX7 type=write size=1348345 2024-04-07T16:38:09.413 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:09.420+0000 7f346a3868c0 1 > prepared operation key=LXOZ0T2KY6OR tag=VOGFSP7XAQWW type=write size=491704 2024-04-07T16:38:09.418 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:09.424+0000 7f346a3868c0 1 < completed delete operation key=UIW3DN4NUYPP tag=RFIBR7NMONV5 type=del 2024-04-07T16:38:09.424 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:09.432+0000 7f346a3868c0 1 > prepared operation key=YXMDZB23YTB3 tag=2AWUVJDPRZX1 type=del size=749870 2024-04-07T16:38:09.430 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:09.436+0000 7f346a3868c0 1 < completed part key=_multipart_YOK8BAEOMPUR.2~XE8EZSWIPVOI.0 size=1048576 2024-04-07T16:38:09.435 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:09.444+0000 7f346a3868c0 1 < completed part key=_multipart_YOK8BAEOMPUR.2~XE8EZSWIPVOI.1 size=686330 2024-04-07T16:38:09.444 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:09.452+0000 7f346a3868c0 1 < completed multipart upload key=YOK8BAEOMPUR upload=XE8EZSWIPVOI tag=DPCZGAWMWHNZ type=write size=2783482 2024-04-07T16:38:09.452 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:09.460+0000 7f346a3868c0 1 > prepared operation key=LXOZ0T2KY6OR tag=5FOHO8FYX1DG type=del size=3512486 2024-04-07T16:38:09.460 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:09.468+0000 7f346a3868c0 -1 < failed to complete operation key=95YVKJ14JD8W tag=Y57PDU7SDS3Y type=write size=75419: (22) Invalid argument 2024-04-07T16:38:09.468 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:09.476+0000 7f346a3868c0 1 > prepared operation key=6XDS0A7L0IY8 tag=MK8KWB1B9220 type=del size=2831015 2024-04-07T16:38:09.474 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:09.480+0000 7f346a3868c0 1 < canceled operation key=QS4HYZFFAJNF tag=M6HDTU47VNJ6 type=del size=4141960 2024-04-07T16:38:09.482 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:09.488+0000 7f346a3868c0 1 > prepared operation key=XJLQZ7IS9U1T tag=GGDAXHQO5ZJC type=write size=837666 2024-04-07T16:38:09.488 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:09.496+0000 7f346a3868c0 1 < canceled operation key=ROFH30W3UCAX tag=MAS36FGI3T2G type=del size=3476205 2024-04-07T16:38:09.494 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:09.500+0000 7f346a3868c0 1 > prepared operation key=ROFH30W3UCAX tag=H8N0C1WJ0PQT type=del size=303385 2024-04-07T16:38:09.500 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:09.508+0000 7f346a3868c0 1 < completed part key=_multipart_EOYFVEI50SQ1.2~05IM5576JEHK.0 size=1048576 2024-04-07T16:38:09.505 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:09.512+0000 7f346a3868c0 1 < completed part key=_multipart_EOYFVEI50SQ1.2~05IM5576JEHK.1 size=1048576 2024-04-07T16:38:09.510 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:09.516+0000 7f346a3868c0 1 < completed part key=_multipart_EOYFVEI50SQ1.2~05IM5576JEHK.2 size=282226 2024-04-07T16:38:09.519 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:09.528+0000 7f346a3868c0 1 < completed multipart upload key=EOYFVEI50SQ1 upload=05IM5576JEHK tag=6GWWWOXWRMWW type=write size=3427954 2024-04-07T16:38:09.533 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:09.540+0000 7f346a3868c0 1 > prepared part key=_multipart_CL4PP1RKZU0E.2~JESH3ZGN1GV3.0 size=1048576 2024-04-07T16:38:09.538 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:09.544+0000 7f346a3868c0 1 > prepared part key=_multipart_CL4PP1RKZU0E.2~JESH3ZGN1GV3.1 size=1025461 2024-04-07T16:38:09.538 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:09.544+0000 7f346a3868c0 1 > prepared multipart upload key=CL4PP1RKZU0E upload=JESH3ZGN1GV3 tag=LUKGX4BIMLPU type=write size=3122613 2024-04-07T16:38:09.551 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:09.560+0000 7f346a3868c0 1 < suggested update operation key=EOYFVEI50SQ1 tag=4BC1PGJ6HBB1 type=write size=17884 2024-04-07T16:38:09.557 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:09.564+0000 7f346a3868c0 1 > prepared operation key=1WGUPH7X5UA0 tag=IO2C2ZLJ2K65 type=del size=1701901 2024-04-07T16:38:09.565 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:09.572+0000 7f346a3868c0 1 < completed delete operation key=QTCIIPTXIRW2 tag=2ULJSV1R2GIJ type=del 2024-04-07T16:38:09.571 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:09.576+0000 7f346a3868c0 1 > prepared operation key=ROFH30W3UCAX tag=ZHA4GUJYFFGF type=del size=3288716 2024-04-07T16:38:09.577 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:09.584+0000 7f346a3868c0 1 < completed delete operation key=DXTO3AJ429AC tag=5F0FS36PX5QT type=del 2024-04-07T16:38:09.583 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:09.592+0000 7f346a3868c0 1 > prepared operation key=YOK8BAEOMPUR tag=77ZF3XOKTEDM type=del size=1028852 2024-04-07T16:38:09.589 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:09.596+0000 7f346a3868c0 1 < completed part key=_multipart_IP8YW2A75TCW.2~LNLOWRAXWZNR.0 size=211936 2024-04-07T16:38:09.598 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:09.604+0000 7f346a3868c0 1 < completed multipart upload key=IP8YW2A75TCW upload=LNLOWRAXWZNR tag=Q2YK9C7XW8U6 type=write size=1260512 2024-04-07T16:38:09.603 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:09.612+0000 7f346a3868c0 1 > prepared operation key=6XDS0A7L0IY8 tag=OQCIT0XEB2IY type=del size=3790391 2024-04-07T16:38:09.609 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:09.616+0000 7f346a3868c0 1 < completed delete operation key=6XDS0A7L0IY8 tag=3JB9BODZCVAK type=del 2024-04-07T16:38:09.619 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:09.628+0000 7f346a3868c0 1 > prepared part key=_multipart_GZXHYZO5SMUP.2~7MPC0T8AU6CW.0 size=1048576 2024-04-07T16:38:09.623 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:09.632+0000 7f346a3868c0 1 > prepared part key=_multipart_GZXHYZO5SMUP.2~7MPC0T8AU6CW.1 size=1048576 2024-04-07T16:38:09.628 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:09.636+0000 7f346a3868c0 1 > prepared part key=_multipart_GZXHYZO5SMUP.2~7MPC0T8AU6CW.2 size=728572 2024-04-07T16:38:09.628 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:09.636+0000 7f346a3868c0 1 > prepared multipart upload key=GZXHYZO5SMUP upload=7MPC0T8AU6CW tag=YG00OD3X6NGR type=write size=3874300 2024-04-07T16:38:09.634 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:09.640+0000 7f346a3868c0 1 < completed part key=_multipart_YOK8BAEOMPUR.2~JU7E1LCVAPMH.0 size=1048576 2024-04-07T16:38:09.639 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:09.648+0000 7f346a3868c0 1 < completed part key=_multipart_YOK8BAEOMPUR.2~JU7E1LCVAPMH.1 size=1048576 2024-04-07T16:38:09.644 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:09.652+0000 7f346a3868c0 1 < completed part key=_multipart_YOK8BAEOMPUR.2~JU7E1LCVAPMH.2 size=145096 2024-04-07T16:38:09.654 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:09.660+0000 7f346a3868c0 1 < completed multipart upload key=YOK8BAEOMPUR upload=JU7E1LCVAPMH tag=03WBHRBUQA67 type=write size=3290824 2024-04-07T16:38:09.664 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:09.672+0000 7f346a3868c0 1 > prepared part key=_multipart_7G5C6HBK6QWI.2~IX92VLQ5PUD3.0 size=130191 2024-04-07T16:38:09.665 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:09.672+0000 7f346a3868c0 1 > prepared multipart upload key=7G5C6HBK6QWI upload=IX92VLQ5PUD3 tag=4W4H4D499HAB type=write size=1178767 2024-04-07T16:38:09.670 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:09.676+0000 7f346a3868c0 1 < completed part key=_multipart_HBLEXONKXJTL.2~6PH3TGHSW0BM.0 size=1048576 2024-04-07T16:38:09.674 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:09.680+0000 7f346a3868c0 1 < completed part key=_multipart_HBLEXONKXJTL.2~6PH3TGHSW0BM.1 size=971314 2024-04-07T16:38:09.685 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:09.692+0000 7f346a3868c0 1 < completed multipart upload key=HBLEXONKXJTL upload=6PH3TGHSW0BM tag=XMKJ5FUQQ5WU type=write size=3068466 2024-04-07T16:38:09.690 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:09.696+0000 7f346a3868c0 1 > prepared operation key=HBLEXONKXJTL tag=0D80EN8MAXTJ type=del size=2147328 2024-04-07T16:38:09.696 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:09.704+0000 7f346a3868c0 1 < completed delete operation key=GZXHYZO5SMUP tag=NUL6XWR2BJGG type=del 2024-04-07T16:38:09.704 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:09.712+0000 7f346a3868c0 1 > prepared operation key=8F81G0P0YBFW tag=LMMH8KZLCN3N type=write size=717301 2024-04-07T16:38:09.709 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:09.716+0000 7f346a3868c0 1 < completed delete operation key=8F81G0P0YBFW tag=XK3NKOBXT4YS type=del 2024-04-07T16:38:09.719 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:09.728+0000 7f346a3868c0 1 > prepared part key=_multipart_LXOZ0T2KY6OR.2~AEX7XAPJ035P.0 size=1048576 2024-04-07T16:38:09.724 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:09.732+0000 7f346a3868c0 1 > prepared part key=_multipart_LXOZ0T2KY6OR.2~AEX7XAPJ035P.1 size=795680 2024-04-07T16:38:09.724 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:09.732+0000 7f346a3868c0 1 > prepared multipart upload key=LXOZ0T2KY6OR upload=AEX7XAPJ035P tag=XIOBB3DTY43Y type=write size=2892832 2024-04-07T16:38:09.734 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:09.740+0000 7f346a3868c0 1 < suggested remove operation key=LXOZ0T2KY6OR tag=VOGFSP7XAQWW type=write 2024-04-07T16:38:09.740 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:09.748+0000 7f346a3868c0 1 > prepared operation key=EOYFVEI50SQ1 tag=80KFICICGYVU type=del size=1407979 2024-04-07T16:38:09.745 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:09.752+0000 7f346a3868c0 1 < completed delete operation key=YXMDZB23YTB3 tag=2AWUVJDPRZX1 type=del 2024-04-07T16:38:09.755 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:09.764+0000 7f346a3868c0 1 > prepared part key=_multipart_U3EID5EK64HD.2~H1WWV16G1SSM.0 size=104361 2024-04-07T16:38:09.756 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:09.764+0000 7f346a3868c0 1 > prepared multipart upload key=U3EID5EK64HD upload=H1WWV16G1SSM tag=206T616K0KJ2 type=write size=1152937 2024-04-07T16:38:09.762 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:09.768+0000 7f346a3868c0 -1 < failed to complete operation key=LXOZ0T2KY6OR tag=5FOHO8FYX1DG type=del size=3512486: (22) Invalid argument 2024-04-07T16:38:09.769 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:09.776+0000 7f346a3868c0 1 > prepared operation key=GBQSME7SY1AH tag=9DLSJ6DN3DS3 type=del size=3690216 2024-04-07T16:38:09.781 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:09.788+0000 7f346a3868c0 1 < suggested update operation key=6XDS0A7L0IY8 tag=MK8KWB1B9220 type=del size=2831015 2024-04-07T16:38:09.786 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:09.792+0000 7f346a3868c0 1 > prepared operation key=CL4PP1RKZU0E tag=30HZQH8Y0UHR type=del size=2754341 2024-04-07T16:38:09.792 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:09.800+0000 7f346a3868c0 1 < completed write operation key=XJLQZ7IS9U1T tag=GGDAXHQO5ZJC type=write size=837666 2024-04-07T16:38:09.797 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:09.804+0000 7f346a3868c0 1 > prepared operation key=ROFH30W3UCAX tag=NR5TX1Z1WEF0 type=del size=312302 2024-04-07T16:38:09.802 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:09.808+0000 7f346a3868c0 1 < completed delete operation key=ROFH30W3UCAX tag=H8N0C1WJ0PQT type=del 2024-04-07T16:38:09.808 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:09.816+0000 7f346a3868c0 1 > prepared operation key=DXTO3AJ429AC tag=KRUWAH874TU6 type=write size=276444 2024-04-07T16:38:09.813 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:09.820+0000 7f346a3868c0 1 < completed part key=_multipart_CL4PP1RKZU0E.2~JESH3ZGN1GV3.0 size=1048576 2024-04-07T16:38:09.818 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:09.824+0000 7f346a3868c0 1 < completed part key=_multipart_CL4PP1RKZU0E.2~JESH3ZGN1GV3.1 size=1025461 2024-04-07T16:38:09.827 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:09.836+0000 7f346a3868c0 1 < completed multipart upload key=CL4PP1RKZU0E upload=JESH3ZGN1GV3 tag=LUKGX4BIMLPU type=write size=3122613 2024-04-07T16:38:09.841 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:09.848+0000 7f346a3868c0 1 > prepared part key=_multipart_LXOZ0T2KY6OR.2~IYMPQ8RNPGO2.0 size=1048576 2024-04-07T16:38:09.844 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:09.852+0000 7f346a3868c0 1 > prepared part key=_multipart_LXOZ0T2KY6OR.2~IYMPQ8RNPGO2.1 size=967664 2024-04-07T16:38:09.844 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:09.852+0000 7f346a3868c0 1 > prepared multipart upload key=LXOZ0T2KY6OR upload=IYMPQ8RNPGO2 tag=QDEAIOO1IFJE type=write size=3064816 2024-04-07T16:38:09.850 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:09.856+0000 7f346a3868c0 1 < canceled operation key=1WGUPH7X5UA0 tag=IO2C2ZLJ2K65 type=del size=1701901 2024-04-07T16:38:09.856 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:09.864+0000 7f346a3868c0 1 > prepared operation key=YOK8BAEOMPUR tag=GOD2AVIK9T8L type=del size=1332363 2024-04-07T16:38:09.863 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:09.868+0000 7f346a3868c0 1 < completed delete operation key=ROFH30W3UCAX tag=ZHA4GUJYFFGF type=del 2024-04-07T16:38:09.875 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:09.880+0000 7f346a3868c0 1 > prepared part key=_multipart_IP8YW2A75TCW.2~3QBSZZPD2AGR.0 size=769115 2024-04-07T16:38:09.875 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:09.880+0000 7f346a3868c0 1 > prepared multipart upload key=IP8YW2A75TCW upload=3QBSZZPD2AGR tag=I18OMQG3BI2E type=write size=1817691 2024-04-07T16:38:09.883 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:09.892+0000 7f346a3868c0 1 < completed delete operation key=YOK8BAEOMPUR tag=77ZF3XOKTEDM type=del 2024-04-07T16:38:09.888 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:09.896+0000 7f346a3868c0 1 > prepared operation key=1VBSUWNI3V1T tag=NG0MBNLHHV3Y type=del size=1840157 2024-04-07T16:38:09.901 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:09.908+0000 7f346a3868c0 1 < suggested remove operation key=6XDS0A7L0IY8 tag=OQCIT0XEB2IY type=del 2024-04-07T16:38:09.911 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:09.920+0000 7f346a3868c0 1 > prepared part key=_multipart_IX3F0IUOKBX1.2~9BB17EBTTANC.0 size=582808 2024-04-07T16:38:09.912 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:09.920+0000 7f346a3868c0 1 > prepared multipart upload key=IX3F0IUOKBX1 upload=9BB17EBTTANC tag=R9MTPTLUTNSY type=write size=1631384 2024-04-07T16:38:09.918 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:09.924+0000 7f346a3868c0 1 < completed part key=_multipart_GZXHYZO5SMUP.2~7MPC0T8AU6CW.0 size=1048576 2024-04-07T16:38:09.923 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:09.932+0000 7f346a3868c0 1 < completed part key=_multipart_GZXHYZO5SMUP.2~7MPC0T8AU6CW.1 size=1048576 2024-04-07T16:38:09.923 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:09.932+0000 7f346a3868c0 1 < canceled part key=_multipart_GZXHYZO5SMUP.2~7MPC0T8AU6CW.2 size=728572 2024-04-07T16:38:09.932 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:09.940+0000 7f346a3868c0 1 < canceled multipart upload key=GZXHYZO5SMUP upload=7MPC0T8AU6CW tag=YG00OD3X6NGR type=write size=3874300 2024-04-07T16:38:09.938 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:09.944+0000 7f346a3868c0 1 > prepared operation key=QTCIIPTXIRW2 tag=HUPJ1YJ42P0D type=del size=2319830 2024-04-07T16:38:09.943 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:09.952+0000 7f346a3868c0 1 < completed part key=_multipart_7G5C6HBK6QWI.2~IX92VLQ5PUD3.0 size=130191 2024-04-07T16:38:09.953 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:09.960+0000 7f346a3868c0 1 < completed multipart upload key=7G5C6HBK6QWI upload=IX92VLQ5PUD3 tag=4W4H4D499HAB type=write size=1178767 2024-04-07T16:38:09.963 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:09.972+0000 7f346a3868c0 1 > prepared part key=_multipart_ROFH30W3UCAX.2~J5IME70G8VQ2.0 size=1048576 2024-04-07T16:38:09.968 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:09.976+0000 7f346a3868c0 1 > prepared part key=_multipart_ROFH30W3UCAX.2~J5IME70G8VQ2.1 size=1048576 2024-04-07T16:38:09.972 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:09.980+0000 7f346a3868c0 1 > prepared part key=_multipart_ROFH30W3UCAX.2~J5IME70G8VQ2.2 size=746831 2024-04-07T16:38:09.972 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:09.980+0000 7f346a3868c0 1 > prepared multipart upload key=ROFH30W3UCAX upload=J5IME70G8VQ2 tag=8SIZGF0AXO9U type=write size=3892559 2024-04-07T16:38:09.981 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:09.988+0000 7f346a3868c0 1 < completed delete operation key=HBLEXONKXJTL tag=0D80EN8MAXTJ type=del 2024-04-07T16:38:09.992 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:10.000+0000 7f346a3868c0 1 > prepared part key=_multipart_6XDS0A7L0IY8.2~HM5OWQIGI3K3.0 size=1048576 2024-04-07T16:38:09.997 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:10.004+0000 7f346a3868c0 1 > prepared part key=_multipart_6XDS0A7L0IY8.2~HM5OWQIGI3K3.1 size=245238 2024-04-07T16:38:09.997 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:10.004+0000 7f346a3868c0 1 > prepared multipart upload key=6XDS0A7L0IY8 upload=HM5OWQIGI3K3 tag=D436UKNVV9TE type=write size=2342390 2024-04-07T16:38:10.003 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:10.012+0000 7f346a3868c0 1 < completed write operation key=8F81G0P0YBFW tag=LMMH8KZLCN3N type=write size=717301 2024-04-07T16:38:10.009 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:10.016+0000 7f346a3868c0 1 > prepared operation key=GZXHYZO5SMUP tag=Q3EO4EJFWHCL type=del size=2245883 2024-04-07T16:38:10.015 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:10.020+0000 7f346a3868c0 1 < completed part key=_multipart_LXOZ0T2KY6OR.2~AEX7XAPJ035P.0 size=1048576 2024-04-07T16:38:10.020 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:10.028+0000 7f346a3868c0 1 < completed part key=_multipart_LXOZ0T2KY6OR.2~AEX7XAPJ035P.1 size=795680 2024-04-07T16:38:10.030 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:10.036+0000 7f346a3868c0 1 < completed multipart upload key=LXOZ0T2KY6OR upload=AEX7XAPJ035P tag=XIOBB3DTY43Y type=write size=2892832 2024-04-07T16:38:10.041 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:10.048+0000 7f346a3868c0 1 > prepared part key=_multipart_EQR0FY86AT5X.2~KG7MZ0J0LZ6O.0 size=284696 2024-04-07T16:38:10.041 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:10.048+0000 7f346a3868c0 1 > prepared multipart upload key=EQR0FY86AT5X upload=KG7MZ0J0LZ6O tag=73KA9A8UKL1I type=write size=1333272 2024-04-07T16:38:10.053 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:10.060+0000 7f346a3868c0 1 < suggested update operation key=EOYFVEI50SQ1 tag=80KFICICGYVU type=del size=1407979 2024-04-07T16:38:10.060 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:10.068+0000 7f346a3868c0 1 > prepared operation key=IX3F0IUOKBX1 tag=R8Q1U1E0BOEZ type=del size=3439749 2024-04-07T16:38:10.066 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:10.072+0000 7f346a3868c0 1 < completed part key=_multipart_U3EID5EK64HD.2~H1WWV16G1SSM.0 size=104361 2024-04-07T16:38:10.077 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:10.084+0000 7f346a3868c0 1 < completed multipart upload key=U3EID5EK64HD upload=H1WWV16G1SSM tag=206T616K0KJ2 type=write size=1152937 2024-04-07T16:38:10.083 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:10.088+0000 7f346a3868c0 1 > prepared operation key=LXOZ0T2KY6OR tag=FAVQYZMBHTWK type=del size=2486729 2024-04-07T16:38:10.094 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:10.100+0000 7f346a3868c0 1 < suggested update operation key=GBQSME7SY1AH tag=9DLSJ6DN3DS3 type=del size=3690216 2024-04-07T16:38:10.099 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:10.108+0000 7f346a3868c0 1 > prepared operation key=QTCIIPTXIRW2 tag=Z3GNTG2EBRLZ type=del size=1739844 2024-04-07T16:38:10.111 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:10.120+0000 7f346a3868c0 1 < suggested update operation key=CL4PP1RKZU0E tag=30HZQH8Y0UHR type=del size=2754341 2024-04-07T16:38:10.121 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:10.128+0000 7f346a3868c0 1 > prepared part key=_multipart_EOYFVEI50SQ1.2~GK94N9EOTER2.0 size=1048576 2024-04-07T16:38:10.127 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:10.132+0000 7f346a3868c0 1 > prepared part key=_multipart_EOYFVEI50SQ1.2~GK94N9EOTER2.1 size=29386 2024-04-07T16:38:10.127 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:10.132+0000 7f346a3868c0 1 > prepared multipart upload key=EOYFVEI50SQ1 upload=GK94N9EOTER2 tag=P1F86BTHGY41 type=write size=2126538 2024-04-07T16:38:10.133 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:10.140+0000 7f346a3868c0 1 < completed delete operation key=ROFH30W3UCAX tag=NR5TX1Z1WEF0 type=del 2024-04-07T16:38:10.140 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:10.148+0000 7f346a3868c0 1 > prepared operation key=1WGUPH7X5UA0 tag=Y7RE8I8WD72X type=del size=940329 2024-04-07T16:38:10.146 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:10.152+0000 7f346a3868c0 1 < completed write operation key=DXTO3AJ429AC tag=KRUWAH874TU6 type=write size=276444 2024-04-07T16:38:10.152 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:10.160+0000 7f346a3868c0 1 > prepared operation key=NYET3212USVE tag=HBYVK2GEDBID type=del size=3583244 2024-04-07T16:38:10.158 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:10.164+0000 7f346a3868c0 1 < completed part key=_multipart_LXOZ0T2KY6OR.2~IYMPQ8RNPGO2.0 size=1048576 2024-04-07T16:38:10.163 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:10.168+0000 7f346a3868c0 1 < completed part key=_multipart_LXOZ0T2KY6OR.2~IYMPQ8RNPGO2.1 size=967664 2024-04-07T16:38:10.174 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:10.180+0000 7f346a3868c0 1 < completed multipart upload key=LXOZ0T2KY6OR upload=IYMPQ8RNPGO2 tag=QDEAIOO1IFJE type=write size=3064816 2024-04-07T16:38:10.181 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:10.188+0000 7f346a3868c0 1 > prepared operation key=QVOKSQAGHPA9 tag=D8IBBWC4PKRR type=write size=896581 2024-04-07T16:38:10.187 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:10.196+0000 7f346a3868c0 1 < completed delete operation key=YOK8BAEOMPUR tag=GOD2AVIK9T8L type=del 2024-04-07T16:38:10.193 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:10.200+0000 7f346a3868c0 1 > prepared operation key=7G5C6HBK6QWI tag=816BXR2M8U5D type=del size=1653786 2024-04-07T16:38:10.199 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:10.208+0000 7f346a3868c0 1 < completed part key=_multipart_IP8YW2A75TCW.2~3QBSZZPD2AGR.0 size=769115 2024-04-07T16:38:10.209 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:10.216+0000 7f346a3868c0 1 < completed multipart upload key=IP8YW2A75TCW upload=3QBSZZPD2AGR tag=I18OMQG3BI2E type=write size=1817691 2024-04-07T16:38:10.214 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:10.220+0000 7f346a3868c0 1 > prepared operation key=7G5C6HBK6QWI tag=DEGO4RZZQMW2 type=del size=3810525 2024-04-07T16:38:10.221 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:10.228+0000 7f346a3868c0 1 < completed delete operation key=1VBSUWNI3V1T tag=NG0MBNLHHV3Y type=del 2024-04-07T16:38:10.226 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:10.232+0000 7f346a3868c0 1 > prepared operation key=X5Q46GO8YTOK tag=ILCE1UAH61CQ type=del size=411484 2024-04-07T16:38:10.232 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:10.240+0000 7f346a3868c0 1 < completed part key=_multipart_IX3F0IUOKBX1.2~9BB17EBTTANC.0 size=582808 2024-04-07T16:38:10.243 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:10.248+0000 7f346a3868c0 1 < completed multipart upload key=IX3F0IUOKBX1 upload=9BB17EBTTANC tag=R9MTPTLUTNSY type=write size=1631384 2024-04-07T16:38:10.249 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:10.256+0000 7f346a3868c0 1 > prepared operation key=XJLQZ7IS9U1T tag=KOS3GYUWWTB8 type=del size=2870778 2024-04-07T16:38:10.254 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:10.260+0000 7f346a3868c0 1 < canceled operation key=QTCIIPTXIRW2 tag=HUPJ1YJ42P0D type=del size=2319830 2024-04-07T16:38:10.264 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:10.272+0000 7f346a3868c0 1 > prepared part key=_multipart_QTCIIPTXIRW2.2~V3E8OF2TEHHV.0 size=1048576 2024-04-07T16:38:10.269 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:10.276+0000 7f346a3868c0 1 > prepared part key=_multipart_QTCIIPTXIRW2.2~V3E8OF2TEHHV.1 size=1048576 2024-04-07T16:38:10.274 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:10.280+0000 7f346a3868c0 1 > prepared part key=_multipart_QTCIIPTXIRW2.2~V3E8OF2TEHHV.2 size=123387 2024-04-07T16:38:10.274 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:10.280+0000 7f346a3868c0 1 > prepared multipart upload key=QTCIIPTXIRW2 upload=V3E8OF2TEHHV tag=RHVCDTG674Z8 type=write size=3269115 2024-04-07T16:38:10.275 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:10.284+0000 7f346a3868c0 1 < canceled part key=_multipart_ROFH30W3UCAX.2~J5IME70G8VQ2.0 size=1048576 2024-04-07T16:38:10.280 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:10.288+0000 7f346a3868c0 1 < completed part key=_multipart_ROFH30W3UCAX.2~J5IME70G8VQ2.1 size=1048576 2024-04-07T16:38:10.285 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:10.292+0000 7f346a3868c0 1 < completed part key=_multipart_ROFH30W3UCAX.2~J5IME70G8VQ2.2 size=746831 2024-04-07T16:38:10.295 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:10.304+0000 7f346a3868c0 1 < canceled multipart upload key=ROFH30W3UCAX upload=J5IME70G8VQ2 tag=8SIZGF0AXO9U type=write size=3892559 2024-04-07T16:38:10.301 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:10.308+0000 7f346a3868c0 1 > prepared operation key=LXOZ0T2KY6OR tag=WRGEBN5VT6Y7 type=del size=1327993 2024-04-07T16:38:10.302 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:10.308+0000 7f346a3868c0 1 < canceled part key=_multipart_6XDS0A7L0IY8.2~HM5OWQIGI3K3.0 size=1048576 2024-04-07T16:38:10.307 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:10.312+0000 7f346a3868c0 1 < completed part key=_multipart_6XDS0A7L0IY8.2~HM5OWQIGI3K3.1 size=245238 2024-04-07T16:38:10.318 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:10.324+0000 7f346a3868c0 1 < canceled multipart upload key=6XDS0A7L0IY8 upload=HM5OWQIGI3K3 tag=D436UKNVV9TE type=write size=2342390 2024-04-07T16:38:10.325 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:10.332+0000 7f346a3868c0 1 > prepared operation key=7G5C6HBK6QWI tag=5XYZLZ0XYA6C type=write size=660497 2024-04-07T16:38:10.332 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:10.340+0000 7f346a3868c0 1 < completed delete operation key=GZXHYZO5SMUP tag=Q3EO4EJFWHCL type=del 2024-04-07T16:38:10.339 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:10.344+0000 7f346a3868c0 1 > prepared operation key=6XDS0A7L0IY8 tag=UMRCQBIVILY2 type=del size=3381958 2024-04-07T16:38:10.345 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:10.352+0000 7f346a3868c0 1 < completed part key=_multipart_EQR0FY86AT5X.2~KG7MZ0J0LZ6O.0 size=284696 2024-04-07T16:38:10.356 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:10.364+0000 7f346a3868c0 1 < completed multipart upload key=EQR0FY86AT5X upload=KG7MZ0J0LZ6O tag=73KA9A8UKL1I type=write size=1333272 2024-04-07T16:38:10.361 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:10.368+0000 7f346a3868c0 1 > prepared operation key=GZXHYZO5SMUP tag=ZVJTD0XXNHE5 type=del size=3206988 2024-04-07T16:38:10.367 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:10.376+0000 7f346a3868c0 1 < completed delete operation key=IX3F0IUOKBX1 tag=R8Q1U1E0BOEZ type=del 2024-04-07T16:38:10.377 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:10.384+0000 7f346a3868c0 1 > prepared part key=_multipart_GZXHYZO5SMUP.2~0ISPUICIEN50.0 size=1048576 2024-04-07T16:38:10.385 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:10.392+0000 7f346a3868c0 1 > prepared part key=_multipart_GZXHYZO5SMUP.2~0ISPUICIEN50.1 size=1048576 2024-04-07T16:38:10.390 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:10.396+0000 7f346a3868c0 1 > prepared part key=_multipart_GZXHYZO5SMUP.2~0ISPUICIEN50.2 size=749808 2024-04-07T16:38:10.390 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:10.396+0000 7f346a3868c0 1 > prepared multipart upload key=GZXHYZO5SMUP upload=0ISPUICIEN50 tag=RFNHB2VZ3BWN type=write size=3895536 2024-04-07T16:38:10.396 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:10.404+0000 7f346a3868c0 1 < completed delete operation key=LXOZ0T2KY6OR tag=FAVQYZMBHTWK type=del 2024-04-07T16:38:10.409 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:10.416+0000 7f346a3868c0 1 > prepared part key=_multipart_HBLEXONKXJTL.2~P64BOIW9O2TK.0 size=1048576 2024-04-07T16:38:10.415 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:10.424+0000 7f346a3868c0 1 > prepared part key=_multipart_HBLEXONKXJTL.2~P64BOIW9O2TK.1 size=254699 2024-04-07T16:38:10.416 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:10.424+0000 7f346a3868c0 1 > prepared multipart upload key=HBLEXONKXJTL upload=P64BOIW9O2TK tag=PSQG5PODM5IB type=write size=2351851 2024-04-07T16:38:10.422 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:10.428+0000 7f346a3868c0 1 < completed delete operation key=QTCIIPTXIRW2 tag=Z3GNTG2EBRLZ type=del 2024-04-07T16:38:10.430 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:10.436+0000 7f346a3868c0 1 > prepared operation key=8F81G0P0YBFW tag=1CN37KMMM7FD type=del size=1407118 2024-04-07T16:38:10.437 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:10.444+0000 7f346a3868c0 1 < completed part key=_multipart_EOYFVEI50SQ1.2~GK94N9EOTER2.0 size=1048576 2024-04-07T16:38:10.437 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:10.444+0000 7f346a3868c0 1 < canceled part key=_multipart_EOYFVEI50SQ1.2~GK94N9EOTER2.1 size=29386 2024-04-07T16:38:10.441 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:38:10 smithi100 bash[74942]: cluster 2024-04-07T16:38:09.069786+0000 mgr.y (mgr.25175) 684 : cluster [DBG] pgmap v1319: 137 pgs: 137 active+clean; 583 KiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 548 KiB/s rd, 188 KiB/s wr, 969 op/s 2024-04-07T16:38:10.441 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:38:10 smithi100 bash[76202]: cluster 2024-04-07T16:38:09.069786+0000 mgr.y (mgr.25175) 684 : cluster [DBG] pgmap v1319: 137 pgs: 137 active+clean; 583 KiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 548 KiB/s rd, 188 KiB/s wr, 969 op/s 2024-04-07T16:38:10.446 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:10.452+0000 7f346a3868c0 1 < canceled multipart upload key=EOYFVEI50SQ1 upload=GK94N9EOTER2 tag=P1F86BTHGY41 type=write size=2126538 2024-04-07T16:38:10.460 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:10.464+0000 7f346a3868c0 1 > prepared part key=_multipart_ROFH30W3UCAX.2~VNQHR5ZCKHXO.0 size=1048576 2024-04-07T16:38:10.464 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:10.472+0000 7f346a3868c0 1 > prepared part key=_multipart_ROFH30W3UCAX.2~VNQHR5ZCKHXO.1 size=1048576 2024-04-07T16:38:10.469 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:10.476+0000 7f346a3868c0 1 > prepared part key=_multipart_ROFH30W3UCAX.2~VNQHR5ZCKHXO.2 size=862073 2024-04-07T16:38:10.469 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:10.476+0000 7f346a3868c0 1 > prepared multipart upload key=ROFH30W3UCAX upload=VNQHR5ZCKHXO tag=JNQ79C5AF79M type=write size=4007801 2024-04-07T16:38:10.481 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:10.488+0000 7f346a3868c0 1 < suggested remove operation key=1WGUPH7X5UA0 tag=Y7RE8I8WD72X type=del 2024-04-07T16:38:10.487 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:10.496+0000 7f346a3868c0 1 > prepared operation key=YXMDZB23YTB3 tag=5LJ63JAG4FEP type=del size=3868480 2024-04-07T16:38:10.500 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:10.508+0000 7f346a3868c0 1 < suggested update operation key=NYET3212USVE tag=HBYVK2GEDBID type=del size=3583244 2024-04-07T16:38:10.511 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:10.520+0000 7f346a3868c0 1 > prepared part key=_multipart_IX3F0IUOKBX1.2~TB263GWD4SBQ.0 size=1048576 2024-04-07T16:38:10.516 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:10.524+0000 7f346a3868c0 1 > prepared part key=_multipart_IX3F0IUOKBX1.2~TB263GWD4SBQ.1 size=535755 2024-04-07T16:38:10.516 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:10.524+0000 7f346a3868c0 1 > prepared multipart upload key=IX3F0IUOKBX1 upload=TB263GWD4SBQ tag=I3CVF9Q66BAD type=write size=2632907 2024-04-07T16:38:10.522 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:10.528+0000 7f346a3868c0 1 < completed write operation key=QVOKSQAGHPA9 tag=D8IBBWC4PKRR type=write size=896581 2024-04-07T16:38:10.533 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:10.540+0000 7f346a3868c0 1 > prepared part key=_multipart_1WGUPH7X5UA0.2~02XGWS9446WP.0 size=1048576 2024-04-07T16:38:10.538 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:10.544+0000 7f346a3868c0 1 > prepared part key=_multipart_1WGUPH7X5UA0.2~02XGWS9446WP.1 size=1048576 2024-04-07T16:38:10.542 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:10.548+0000 7f346a3868c0 1 > prepared part key=_multipart_1WGUPH7X5UA0.2~02XGWS9446WP.2 size=528846 2024-04-07T16:38:10.542 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:10.548+0000 7f346a3868c0 1 > prepared multipart upload key=1WGUPH7X5UA0 upload=02XGWS9446WP tag=PY7PHG1JV0ZD type=write size=3674574 2024-04-07T16:38:10.549 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:10.556+0000 7f346a3868c0 1 < completed delete operation key=7G5C6HBK6QWI tag=816BXR2M8U5D type=del 2024-04-07T16:38:10.556 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:10.564+0000 7f346a3868c0 1 > prepared operation key=3N0QJRYVO3FV tag=6MVGGZNA8YWL type=del size=3696375 2024-04-07T16:38:10.563 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:10.568+0000 7f346a3868c0 1 < completed delete operation key=7G5C6HBK6QWI tag=DEGO4RZZQMW2 type=del 2024-04-07T16:38:10.569 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:10.576+0000 7f346a3868c0 1 > prepared operation key=8F81G0P0YBFW tag=REFXQW0H4JB3 type=del size=1253635 2024-04-07T16:38:10.575 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:10.580+0000 7f346a3868c0 1 < completed delete operation key=X5Q46GO8YTOK tag=ILCE1UAH61CQ type=del 2024-04-07T16:38:10.586 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:10.592+0000 7f346a3868c0 1 > prepared part key=_multipart_EOYFVEI50SQ1.2~1GO5QY78KFMG.0 size=1048576 2024-04-07T16:38:10.591 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:10.600+0000 7f346a3868c0 1 > prepared part key=_multipart_EOYFVEI50SQ1.2~1GO5QY78KFMG.1 size=592386 2024-04-07T16:38:10.591 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:10.600+0000 7f346a3868c0 1 > prepared multipart upload key=EOYFVEI50SQ1 upload=1GO5QY78KFMG tag=CSOBI0UR7EUV type=write size=2689538 2024-04-07T16:38:10.603 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:10.608+0000 7f346a3868c0 1 < suggested update operation key=XJLQZ7IS9U1T tag=KOS3GYUWWTB8 type=del size=2870778 2024-04-07T16:38:10.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:38:10 smithi144 bash[66654]: cluster 2024-04-07T16:38:09.069786+0000 mgr.y (mgr.25175) 684 : cluster [DBG] pgmap v1319: 137 pgs: 137 active+clean; 583 KiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 548 KiB/s rd, 188 KiB/s wr, 969 op/s 2024-04-07T16:38:10.609 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:10.616+0000 7f346a3868c0 1 > prepared operation key=E84U0SX44NQ8 tag=DS72RF40D9S8 type=del size=734420 2024-04-07T16:38:10.615 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:10.624+0000 7f346a3868c0 1 < completed part key=_multipart_QTCIIPTXIRW2.2~V3E8OF2TEHHV.0 size=1048576 2024-04-07T16:38:10.620 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:10.628+0000 7f346a3868c0 1 < completed part key=_multipart_QTCIIPTXIRW2.2~V3E8OF2TEHHV.1 size=1048576 2024-04-07T16:38:10.625 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:10.632+0000 7f346a3868c0 1 < completed part key=_multipart_QTCIIPTXIRW2.2~V3E8OF2TEHHV.2 size=123387 2024-04-07T16:38:10.635 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:10.644+0000 7f346a3868c0 1 < completed multipart upload key=QTCIIPTXIRW2 upload=V3E8OF2TEHHV tag=RHVCDTG674Z8 type=write size=3269115 2024-04-07T16:38:10.641 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:10.648+0000 7f346a3868c0 1 > prepared operation key=YOK8BAEOMPUR tag=32TUGKVWWDW3 type=del size=2204873 2024-04-07T16:38:10.647 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:10.656+0000 7f346a3868c0 1 < completed delete operation key=LXOZ0T2KY6OR tag=WRGEBN5VT6Y7 type=del 2024-04-07T16:38:10.658 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:10.664+0000 7f346a3868c0 1 > prepared part key=_multipart_DXTO3AJ429AC.2~KWE32X4E7UY9.0 size=1048576 2024-04-07T16:38:10.663 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:10.668+0000 7f346a3868c0 1 > prepared part key=_multipart_DXTO3AJ429AC.2~KWE32X4E7UY9.1 size=1048576 2024-04-07T16:38:10.668 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:10.676+0000 7f346a3868c0 1 > prepared part key=_multipart_DXTO3AJ429AC.2~KWE32X4E7UY9.2 size=495287 2024-04-07T16:38:10.668 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:10.676+0000 7f346a3868c0 1 > prepared multipart upload key=DXTO3AJ429AC upload=KWE32X4E7UY9 tag=8H3YFIO6YK40 type=write size=3641015 2024-04-07T16:38:10.674 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:10.680+0000 7f346a3868c0 1 < completed write operation key=7G5C6HBK6QWI tag=5XYZLZ0XYA6C type=write size=660497 2024-04-07T16:38:10.680 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:10.688+0000 7f346a3868c0 1 > prepared operation key=DXTO3AJ429AC tag=3AF3DLII6ZIA type=del size=1396505 2024-04-07T16:38:10.685 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:10.692+0000 7f346a3868c0 1 < canceled operation key=6XDS0A7L0IY8 tag=UMRCQBIVILY2 type=del size=3381958 2024-04-07T16:38:10.692 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:10.700+0000 7f346a3868c0 1 > prepared operation key=IX3F0IUOKBX1 tag=2X6NZHNPVQEP type=write size=193907 2024-04-07T16:38:10.697 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:10.704+0000 7f346a3868c0 1 < completed delete operation key=GZXHYZO5SMUP tag=ZVJTD0XXNHE5 type=del 2024-04-07T16:38:10.709 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:10.716+0000 7f346a3868c0 1 > prepared part key=_multipart_3N0QJRYVO3FV.2~9YZFHIOZFUUX.0 size=290796 2024-04-07T16:38:10.709 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:10.716+0000 7f346a3868c0 1 > prepared multipart upload key=3N0QJRYVO3FV upload=9YZFHIOZFUUX tag=QCMLSF8VUUL1 type=write size=1339372 2024-04-07T16:38:10.718 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:10.724+0000 7f346a3868c0 1 < completed part key=_multipart_GZXHYZO5SMUP.2~0ISPUICIEN50.0 size=1048576 2024-04-07T16:38:10.723 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:10.732+0000 7f346a3868c0 1 < completed part key=_multipart_GZXHYZO5SMUP.2~0ISPUICIEN50.1 size=1048576 2024-04-07T16:38:10.728 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:10.736+0000 7f346a3868c0 1 < completed part key=_multipart_GZXHYZO5SMUP.2~0ISPUICIEN50.2 size=749808 2024-04-07T16:38:10.740 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:10.748+0000 7f346a3868c0 1 < completed multipart upload key=GZXHYZO5SMUP upload=0ISPUICIEN50 tag=RFNHB2VZ3BWN type=write size=3895536 2024-04-07T16:38:10.747 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:10.752+0000 7f346a3868c0 1 > prepared operation key=1VBSUWNI3V1T tag=IS4ZW3I6IG90 type=del size=1274282 2024-04-07T16:38:10.752 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:10.760+0000 7f346a3868c0 1 < completed part key=_multipart_HBLEXONKXJTL.2~P64BOIW9O2TK.0 size=1048576 2024-04-07T16:38:10.757 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:10.764+0000 7f346a3868c0 1 < completed part key=_multipart_HBLEXONKXJTL.2~P64BOIW9O2TK.1 size=254699 2024-04-07T16:38:10.767 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:10.776+0000 7f346a3868c0 1 < completed multipart upload key=HBLEXONKXJTL upload=P64BOIW9O2TK tag=PSQG5PODM5IB type=write size=2351851 2024-04-07T16:38:10.774 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:10.780+0000 7f346a3868c0 1 > prepared operation key=IX3F0IUOKBX1 tag=KYBGXNSX1GKP type=del size=3905320 2024-04-07T16:38:10.780 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:10.788+0000 7f346a3868c0 1 < completed delete operation key=8F81G0P0YBFW tag=1CN37KMMM7FD type=del 2024-04-07T16:38:10.788 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:10.796+0000 7f346a3868c0 1 > prepared operation key=CL4PP1RKZU0E tag=HHZXNFZ3SKRF type=del size=2083979 2024-04-07T16:38:10.794 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:10.800+0000 7f346a3868c0 1 < completed part key=_multipart_ROFH30W3UCAX.2~VNQHR5ZCKHXO.0 size=1048576 2024-04-07T16:38:10.795 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:10.800+0000 7f346a3868c0 1 < canceled part key=_multipart_ROFH30W3UCAX.2~VNQHR5ZCKHXO.1 size=1048576 2024-04-07T16:38:10.801 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:10.808+0000 7f346a3868c0 1 < completed part key=_multipart_ROFH30W3UCAX.2~VNQHR5ZCKHXO.2 size=862073 2024-04-07T16:38:10.814 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:10.820+0000 7f346a3868c0 1 < canceled multipart upload key=ROFH30W3UCAX upload=VNQHR5ZCKHXO tag=JNQ79C5AF79M type=write size=4007801 2024-04-07T16:38:10.828 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:10.836+0000 7f346a3868c0 1 > prepared part key=_multipart_8F81G0P0YBFW.2~5M5LU0TFJXDJ.0 size=800534 2024-04-07T16:38:10.828 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:10.836+0000 7f346a3868c0 1 > prepared multipart upload key=8F81G0P0YBFW upload=5M5LU0TFJXDJ tag=Z3RRG5ZZ5BCR type=write size=1849110 2024-04-07T16:38:10.834 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:10.840+0000 7f346a3868c0 1 < canceled operation key=YXMDZB23YTB3 tag=5LJ63JAG4FEP type=del size=3868480 2024-04-07T16:38:10.845 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:10.852+0000 7f346a3868c0 1 > prepared part key=_multipart_LXOZ0T2KY6OR.2~ULAO5VE6GIC4.0 size=1048576 2024-04-07T16:38:10.852 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:10.860+0000 7f346a3868c0 1 > prepared part key=_multipart_LXOZ0T2KY6OR.2~ULAO5VE6GIC4.1 size=292983 2024-04-07T16:38:10.852 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:10.860+0000 7f346a3868c0 1 > prepared multipart upload key=LXOZ0T2KY6OR upload=ULAO5VE6GIC4 tag=Y9APEMT0ZYZ2 type=write size=2390135 2024-04-07T16:38:10.858 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:10.864+0000 7f346a3868c0 1 < completed part key=_multipart_IX3F0IUOKBX1.2~TB263GWD4SBQ.0 size=1048576 2024-04-07T16:38:10.863 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:10.872+0000 7f346a3868c0 1 < completed part key=_multipart_IX3F0IUOKBX1.2~TB263GWD4SBQ.1 size=535755 2024-04-07T16:38:10.874 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:10.880+0000 7f346a3868c0 1 < completed multipart upload key=IX3F0IUOKBX1 upload=TB263GWD4SBQ tag=I3CVF9Q66BAD type=write size=2632907 2024-04-07T16:38:10.883 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:10.888+0000 7f346a3868c0 1 > prepared operation key=ROFH30W3UCAX tag=PCSEBDP2OSXL type=del size=1384216 2024-04-07T16:38:10.890 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:10.896+0000 7f346a3868c0 1 < completed part key=_multipart_1WGUPH7X5UA0.2~02XGWS9446WP.0 size=1048576 2024-04-07T16:38:10.895 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:10.904+0000 7f346a3868c0 1 < completed part key=_multipart_1WGUPH7X5UA0.2~02XGWS9446WP.1 size=1048576 2024-04-07T16:38:10.900 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:10.908+0000 7f346a3868c0 1 < completed part key=_multipart_1WGUPH7X5UA0.2~02XGWS9446WP.2 size=528846 2024-04-07T16:38:10.910 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:10.916+0000 7f346a3868c0 1 < completed multipart upload key=1WGUPH7X5UA0 upload=02XGWS9446WP tag=PY7PHG1JV0ZD type=write size=3674574 2024-04-07T16:38:10.917 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:10.924+0000 7f346a3868c0 1 > prepared operation key=8F81G0P0YBFW tag=8B1OPG3248F3 type=del size=1349648 2024-04-07T16:38:10.923 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:10.932+0000 7f346a3868c0 1 < completed delete operation key=3N0QJRYVO3FV tag=6MVGGZNA8YWL type=del 2024-04-07T16:38:10.929 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:10.936+0000 7f346a3868c0 1 > prepared operation key=DXTO3AJ429AC tag=YFP84SU8VXLI type=del size=3328820 2024-04-07T16:38:10.936 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:10.944+0000 7f346a3868c0 1 < completed delete operation key=8F81G0P0YBFW tag=REFXQW0H4JB3 type=del 2024-04-07T16:38:10.942 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:10.948+0000 7f346a3868c0 1 > prepared operation key=X5Q46GO8YTOK tag=0UV311EV079Y type=del size=1102754 2024-04-07T16:38:10.949 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:10.956+0000 7f346a3868c0 1 < completed part key=_multipart_EOYFVEI50SQ1.2~1GO5QY78KFMG.0 size=1048576 2024-04-07T16:38:10.949 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:10.956+0000 7f346a3868c0 1 < canceled part key=_multipart_EOYFVEI50SQ1.2~1GO5QY78KFMG.1 size=592386 2024-04-07T16:38:10.959 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:10.964+0000 7f346a3868c0 1 < canceled multipart upload key=EOYFVEI50SQ1 upload=1GO5QY78KFMG tag=CSOBI0UR7EUV type=write size=2689538 2024-04-07T16:38:10.965 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:10.972+0000 7f346a3868c0 1 > prepared operation key=GBQSME7SY1AH tag=R2YZ32M6O2CS type=del size=1237550 2024-04-07T16:38:10.977 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:10.984+0000 7f346a3868c0 1 < suggested remove operation key=E84U0SX44NQ8 tag=DS72RF40D9S8 type=del 2024-04-07T16:38:10.989 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:10.996+0000 7f346a3868c0 1 > prepared part key=_multipart_C1BTVJNHIM0X.2~1LHANGBQJU5N.0 size=159517 2024-04-07T16:38:10.989 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:10.996+0000 7f346a3868c0 1 > prepared multipart upload key=C1BTVJNHIM0X upload=1LHANGBQJU5N tag=1I1KWQYPH6PD type=write size=1208093 2024-04-07T16:38:10.996 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:11.004+0000 7f346a3868c0 1 < completed delete operation key=YOK8BAEOMPUR tag=32TUGKVWWDW3 type=del 2024-04-07T16:38:11.003 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:11.008+0000 7f346a3868c0 1 > prepared operation key=U3EID5EK64HD tag=JYKP9KRC0M3S type=write size=550834 2024-04-07T16:38:11.010 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:11.016+0000 7f346a3868c0 1 < completed part key=_multipart_DXTO3AJ429AC.2~KWE32X4E7UY9.0 size=1048576 2024-04-07T16:38:11.015 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:11.024+0000 7f346a3868c0 1 < completed part key=_multipart_DXTO3AJ429AC.2~KWE32X4E7UY9.1 size=1048576 2024-04-07T16:38:11.021 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:11.028+0000 7f346a3868c0 1 < completed part key=_multipart_DXTO3AJ429AC.2~KWE32X4E7UY9.2 size=495287 2024-04-07T16:38:11.031 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:11.040+0000 7f346a3868c0 1 < completed multipart upload key=DXTO3AJ429AC upload=KWE32X4E7UY9 tag=8H3YFIO6YK40 type=write size=3641015 2024-04-07T16:38:11.037 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:11.044+0000 7f346a3868c0 1 > prepared operation key=CL4PP1RKZU0E tag=L7LUXGADXN5D type=del size=348410 2024-04-07T16:38:11.045 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:11.052+0000 7f346a3868c0 1 < completed delete operation key=DXTO3AJ429AC tag=3AF3DLII6ZIA type=del 2024-04-07T16:38:11.055 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:11.060+0000 7f346a3868c0 1 > prepared part key=_multipart_EQR0FY86AT5X.2~T8QRAFTCSPIX.0 size=677844 2024-04-07T16:38:11.055 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:11.060+0000 7f346a3868c0 1 > prepared multipart upload key=EQR0FY86AT5X upload=T8QRAFTCSPIX tag=20QG0LU2KKKW type=write size=1726420 2024-04-07T16:38:11.061 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:11.068+0000 7f346a3868c0 1 < completed write operation key=IX3F0IUOKBX1 tag=2X6NZHNPVQEP type=write size=193907 2024-04-07T16:38:11.066 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:11.072+0000 7f346a3868c0 1 > prepared operation key=NYET3212USVE tag=68TU0ZYB3SC2 type=del size=1489053 2024-04-07T16:38:11.071 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:11.080+0000 7f346a3868c0 1 < completed part key=_multipart_3N0QJRYVO3FV.2~9YZFHIOZFUUX.0 size=290796 2024-04-07T16:38:11.080 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:11.088+0000 7f346a3868c0 1 < completed multipart upload key=3N0QJRYVO3FV upload=9YZFHIOZFUUX tag=QCMLSF8VUUL1 type=write size=1339372 2024-04-07T16:38:11.086 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:11.092+0000 7f346a3868c0 1 > prepared operation key=X5Q46GO8YTOK tag=M2H6C6FYL9KA type=del size=866758 2024-04-07T16:38:11.092 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:11.100+0000 7f346a3868c0 1 < completed delete operation key=1VBSUWNI3V1T tag=IS4ZW3I6IG90 type=del 2024-04-07T16:38:11.097 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:11.104+0000 7f346a3868c0 1 > prepared operation key=DXTO3AJ429AC tag=19CNL4Y2F2B3 type=del size=3798600 2024-04-07T16:38:11.103 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:11.112+0000 7f346a3868c0 1 < completed delete operation key=IX3F0IUOKBX1 tag=KYBGXNSX1GKP type=del 2024-04-07T16:38:11.110 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:11.116+0000 7f346a3868c0 1 > prepared operation key=1WGUPH7X5UA0 tag=YWX6UGX2Z1V9 type=del size=2141452 2024-04-07T16:38:11.121 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:11.128+0000 7f346a3868c0 1 < suggested remove operation key=CL4PP1RKZU0E tag=HHZXNFZ3SKRF type=del 2024-04-07T16:38:11.128 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:11.136+0000 7f346a3868c0 1 > prepared operation key=CL4PP1RKZU0E tag=ADQCOEY52KKA type=write size=344986 2024-04-07T16:38:11.134 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:11.140+0000 7f346a3868c0 1 < completed part key=_multipart_8F81G0P0YBFW.2~5M5LU0TFJXDJ.0 size=800534 2024-04-07T16:38:11.146 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:11.152+0000 7f346a3868c0 1 < completed multipart upload key=8F81G0P0YBFW upload=5M5LU0TFJXDJ tag=Z3RRG5ZZ5BCR type=write size=1849110 2024-04-07T16:38:11.157 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:11.164+0000 7f346a3868c0 1 > prepared part key=_multipart_YXMDZB23YTB3.2~5AWJ05L30A6Y.0 size=292852 2024-04-07T16:38:11.157 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:11.164+0000 7f346a3868c0 1 > prepared multipart upload key=YXMDZB23YTB3 upload=5AWJ05L30A6Y tag=1UFX9EHRP9GO type=write size=1341428 2024-04-07T16:38:11.163 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:11.168+0000 7f346a3868c0 1 < completed part key=_multipart_LXOZ0T2KY6OR.2~ULAO5VE6GIC4.0 size=1048576 2024-04-07T16:38:11.167 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:11.176+0000 7f346a3868c0 1 < completed part key=_multipart_LXOZ0T2KY6OR.2~ULAO5VE6GIC4.1 size=292983 2024-04-07T16:38:11.176 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:11.184+0000 7f346a3868c0 1 < completed multipart upload key=LXOZ0T2KY6OR upload=ULAO5VE6GIC4 tag=Y9APEMT0ZYZ2 type=write size=2390135 2024-04-07T16:38:11.181 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:11.188+0000 7f346a3868c0 1 > prepared operation key=UIW3DN4NUYPP tag=UMWVK9Y2W083 type=del size=4159039 2024-04-07T16:38:11.192 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:11.200+0000 7f346a3868c0 1 < suggested update operation key=ROFH30W3UCAX tag=PCSEBDP2OSXL type=del size=1384216 2024-04-07T16:38:11.197 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:11.204+0000 7f346a3868c0 1 > prepared operation key=IX3F0IUOKBX1 tag=BVQNNTFACT1H type=del size=1849258 2024-04-07T16:38:11.202 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:11.208+0000 7f346a3868c0 1 < completed delete operation key=8F81G0P0YBFW tag=8B1OPG3248F3 type=del 2024-04-07T16:38:11.212 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:11.220+0000 7f346a3868c0 1 > prepared part key=_multipart_3N0QJRYVO3FV.2~SJWU5U95D2WM.0 size=719870 2024-04-07T16:38:11.212 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:11.220+0000 7f346a3868c0 1 > prepared multipart upload key=3N0QJRYVO3FV upload=SJWU5U95D2WM tag=INYUWX2L3C4Y type=write size=1768446 2024-04-07T16:38:11.217 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:11.224+0000 7f346a3868c0 1 < completed delete operation key=DXTO3AJ429AC tag=YFP84SU8VXLI type=del 2024-04-07T16:38:11.223 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:11.228+0000 7f346a3868c0 1 > prepared operation key=95YVKJ14JD8W tag=8D58RD8LUXPB type=del size=2461169 2024-04-07T16:38:11.228 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:11.236+0000 7f346a3868c0 1 < completed delete operation key=X5Q46GO8YTOK tag=0UV311EV079Y type=del 2024-04-07T16:38:11.239 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:11.248+0000 7f346a3868c0 1 > prepared part key=_multipart_X5Q46GO8YTOK.2~PBAG5ZXBGMYK.0 size=1048576 2024-04-07T16:38:11.244 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:11.252+0000 7f346a3868c0 1 > prepared part key=_multipart_X5Q46GO8YTOK.2~PBAG5ZXBGMYK.1 size=885296 2024-04-07T16:38:11.244 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:11.252+0000 7f346a3868c0 1 > prepared multipart upload key=X5Q46GO8YTOK upload=PBAG5ZXBGMYK tag=6IYZOHUKBD69 type=write size=2982448 2024-04-07T16:38:11.249 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:11.256+0000 7f346a3868c0 1 < completed delete operation key=GBQSME7SY1AH tag=R2YZ32M6O2CS type=del 2024-04-07T16:38:11.255 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:11.264+0000 7f346a3868c0 1 > prepared operation key=1VKVQXSQMTU1 tag=6OCO5BZDHOU9 type=del size=2441278 2024-04-07T16:38:11.261 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:11.268+0000 7f346a3868c0 1 < completed part key=_multipart_C1BTVJNHIM0X.2~1LHANGBQJU5N.0 size=159517 2024-04-07T16:38:11.270 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:11.276+0000 7f346a3868c0 1 < completed multipart upload key=C1BTVJNHIM0X upload=1LHANGBQJU5N tag=1I1KWQYPH6PD type=write size=1208093 2024-04-07T16:38:11.276 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:11.284+0000 7f346a3868c0 1 > prepared operation key=CL4PP1RKZU0E tag=UUEQ6ZM785MN type=write size=256891 2024-04-07T16:38:11.282 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:11.288+0000 7f346a3868c0 1 < completed write operation key=U3EID5EK64HD tag=JYKP9KRC0M3S type=write size=550834 2024-04-07T16:38:11.289 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:11.296+0000 7f346a3868c0 1 > prepared operation key=IP8YW2A75TCW tag=QW3GQPJCFQ04 type=del size=1743911 2024-04-07T16:38:11.297 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:11.304+0000 7f346a3868c0 -1 < failed to complete operation key=CL4PP1RKZU0E tag=L7LUXGADXN5D type=del size=348410: (22) Invalid argument 2024-04-07T16:38:11.308 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:11.316+0000 7f346a3868c0 1 > prepared part key=_multipart_C1BTVJNHIM0X.2~1GJIAKXW7XJ1.0 size=1001225 2024-04-07T16:38:11.308 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:11.316+0000 7f346a3868c0 1 > prepared multipart upload key=C1BTVJNHIM0X upload=1GJIAKXW7XJ1 tag=TT73DF0I0RPD type=write size=2049801 2024-04-07T16:38:11.313 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:11.320+0000 7f346a3868c0 1 < completed part key=_multipart_EQR0FY86AT5X.2~T8QRAFTCSPIX.0 size=677844 2024-04-07T16:38:11.324 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:11.332+0000 7f346a3868c0 1 < completed multipart upload key=EQR0FY86AT5X upload=T8QRAFTCSPIX tag=20QG0LU2KKKW type=write size=1726420 2024-04-07T16:38:11.331 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:11.336+0000 7f346a3868c0 1 > prepared operation key=E84U0SX44NQ8 tag=FM1U2EVOEM8B type=del size=1673270 2024-04-07T16:38:11.337 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:11.344+0000 7f346a3868c0 1 < completed delete operation key=NYET3212USVE tag=68TU0ZYB3SC2 type=del 2024-04-07T16:38:11.348 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:11.356+0000 7f346a3868c0 1 > prepared part key=_multipart_LXOZ0T2KY6OR.2~1TLHWBQ5JNRT.0 size=1048576 2024-04-07T16:38:11.353 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:11.360+0000 7f346a3868c0 1 > prepared part key=_multipart_LXOZ0T2KY6OR.2~1TLHWBQ5JNRT.1 size=171337 2024-04-07T16:38:11.353 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:11.360+0000 7f346a3868c0 1 > prepared multipart upload key=LXOZ0T2KY6OR upload=1TLHWBQ5JNRT tag=N1ERP0O6003X type=write size=2268489 2024-04-07T16:38:11.359 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:11.368+0000 7f346a3868c0 1 < completed delete operation key=X5Q46GO8YTOK tag=M2H6C6FYL9KA type=del 2024-04-07T16:38:11.365 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:11.372+0000 7f346a3868c0 1 > prepared operation key=DXTO3AJ429AC tag=M11EDHG0HYED type=write size=91702 2024-04-07T16:38:11.371 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:11.376+0000 7f346a3868c0 1 < completed delete operation key=DXTO3AJ429AC tag=19CNL4Y2F2B3 type=del 2024-04-07T16:38:11.382 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:11.388+0000 7f346a3868c0 1 > prepared part key=_multipart_C1BTVJNHIM0X.2~IUQ36PAD9GZR.0 size=1048576 2024-04-07T16:38:11.387 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:11.392+0000 7f346a3868c0 1 > prepared part key=_multipart_C1BTVJNHIM0X.2~IUQ36PAD9GZR.1 size=147573 2024-04-07T16:38:11.387 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:11.392+0000 7f346a3868c0 1 > prepared multipart upload key=C1BTVJNHIM0X upload=IUQ36PAD9GZR tag=S95C142XDLTE type=write size=2244725 2024-04-07T16:38:11.398 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:11.404+0000 7f346a3868c0 1 < suggested remove operation key=1WGUPH7X5UA0 tag=YWX6UGX2Z1V9 type=del 2024-04-07T16:38:11.404 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:11.412+0000 7f346a3868c0 1 > prepared operation key=GBQSME7SY1AH tag=0UUJC6RKJK3L type=del size=894166 2024-04-07T16:38:11.410 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:11.416+0000 7f346a3868c0 1 < canceled operation key=CL4PP1RKZU0E tag=ADQCOEY52KKA type=write size=344986 2024-04-07T16:38:11.416 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:11.424+0000 7f346a3868c0 1 > prepared operation key=1WGUPH7X5UA0 tag=MJ7OE035WL06 type=del size=801082 2024-04-07T16:38:11.422 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:11.428+0000 7f346a3868c0 1 < completed part key=_multipart_YXMDZB23YTB3.2~5AWJ05L30A6Y.0 size=292852 2024-04-07T16:38:11.434 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:11.440+0000 7f346a3868c0 1 < completed multipart upload key=YXMDZB23YTB3 upload=5AWJ05L30A6Y tag=1UFX9EHRP9GO type=write size=1341428 2024-04-07T16:38:11.446 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:11.452+0000 7f346a3868c0 1 > prepared part key=_multipart_3N0QJRYVO3FV.2~PFNBYFM07HX1.0 size=1048576 2024-04-07T16:38:11.451 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:11.460+0000 7f346a3868c0 1 > prepared part key=_multipart_3N0QJRYVO3FV.2~PFNBYFM07HX1.1 size=466889 2024-04-07T16:38:11.451 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:11.460+0000 7f346a3868c0 1 > prepared multipart upload key=3N0QJRYVO3FV upload=PFNBYFM07HX1 tag=NN12HAA861I3 type=write size=2564041 2024-04-07T16:38:11.457 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:11.464+0000 7f346a3868c0 1 < completed delete operation key=UIW3DN4NUYPP tag=UMWVK9Y2W083 type=del 2024-04-07T16:38:11.463 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:11.472+0000 7f346a3868c0 1 > prepared operation key=E84U0SX44NQ8 tag=N1S72OR5P8WU type=del size=3605631 2024-04-07T16:38:11.469 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:11.476+0000 7f346a3868c0 1 < completed delete operation key=IX3F0IUOKBX1 tag=BVQNNTFACT1H type=del 2024-04-07T16:38:11.480 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:11.488+0000 7f346a3868c0 1 > prepared part key=_multipart_CL4PP1RKZU0E.2~Z9JGA36I6J1G.0 size=828771 2024-04-07T16:38:11.481 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:11.488+0000 7f346a3868c0 1 > prepared multipart upload key=CL4PP1RKZU0E upload=Z9JGA36I6J1G tag=ER5LWFH5UGX8 type=write size=1877347 2024-04-07T16:38:11.482 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:11.488+0000 7f346a3868c0 1 < canceled part key=_multipart_3N0QJRYVO3FV.2~SJWU5U95D2WM.0 size=719870 2024-04-07T16:38:11.493 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:11.500+0000 7f346a3868c0 1 < canceled multipart upload key=3N0QJRYVO3FV upload=SJWU5U95D2WM tag=INYUWX2L3C4Y type=write size=1768446 2024-04-07T16:38:11.500 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:11.508+0000 7f346a3868c0 1 > prepared operation key=1WGUPH7X5UA0 tag=QWSCWAUDQC7P type=del size=2145265 2024-04-07T16:38:11.505 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:11.512+0000 7f346a3868c0 1 < completed delete operation key=95YVKJ14JD8W tag=8D58RD8LUXPB type=del 2024-04-07T16:38:11.516 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:11.524+0000 7f346a3868c0 1 > prepared part key=_multipart_7G5C6HBK6QWI.2~023FP1B69C8A.0 size=1048576 2024-04-07T16:38:11.521 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:11.528+0000 7f346a3868c0 1 > prepared part key=_multipart_7G5C6HBK6QWI.2~023FP1B69C8A.1 size=1048576 2024-04-07T16:38:11.526 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:11.532+0000 7f346a3868c0 1 > prepared part key=_multipart_7G5C6HBK6QWI.2~023FP1B69C8A.2 size=230225 2024-04-07T16:38:11.526 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:11.532+0000 7f346a3868c0 1 > prepared multipart upload key=7G5C6HBK6QWI upload=023FP1B69C8A tag=30MKQ99XL9HT type=write size=3375953 2024-04-07T16:38:11.532 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:11.540+0000 7f346a3868c0 1 < completed part key=_multipart_X5Q46GO8YTOK.2~PBAG5ZXBGMYK.0 size=1048576 2024-04-07T16:38:11.537 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:11.544+0000 7f346a3868c0 1 < completed part key=_multipart_X5Q46GO8YTOK.2~PBAG5ZXBGMYK.1 size=885296 2024-04-07T16:38:11.547 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:11.552+0000 7f346a3868c0 1 < completed multipart upload key=X5Q46GO8YTOK upload=PBAG5ZXBGMYK tag=6IYZOHUKBD69 type=write size=2982448 2024-04-07T16:38:11.553 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:11.560+0000 7f346a3868c0 1 > prepared operation key=3N0QJRYVO3FV tag=0OF26Q6U4S5C type=del size=1501516 2024-04-07T16:38:11.558 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:11.564+0000 7f346a3868c0 1 < completed delete operation key=1VKVQXSQMTU1 tag=6OCO5BZDHOU9 type=del 2024-04-07T16:38:11.565 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:11.572+0000 7f346a3868c0 1 > prepared operation key=N14BUXLJONHZ tag=RF4D4PP88YM4 type=del size=1770136 2024-04-07T16:38:11.576 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:11.584+0000 7f346a3868c0 1 < suggested remove operation key=CL4PP1RKZU0E tag=UUEQ6ZM785MN type=write 2024-04-07T16:38:11.582 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:11.588+0000 7f346a3868c0 1 > prepared operation key=LXOZ0T2KY6OR tag=H57ZJXUNRM3B type=write size=648149 2024-04-07T16:38:11.593 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:11.600+0000 7f346a3868c0 1 < suggested update operation key=IP8YW2A75TCW tag=QW3GQPJCFQ04 type=del size=1743911 2024-04-07T16:38:11.599 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:11.608+0000 7f346a3868c0 1 > prepared operation key=LXOZ0T2KY6OR tag=6APZY0VM7OII type=del size=2877590 2024-04-07T16:38:11.607 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:11.616+0000 7f346a3868c0 1 < completed part key=_multipart_C1BTVJNHIM0X.2~1GJIAKXW7XJ1.0 size=1001225 2024-04-07T16:38:11.617 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:11.624+0000 7f346a3868c0 1 < completed multipart upload key=C1BTVJNHIM0X upload=1GJIAKXW7XJ1 tag=TT73DF0I0RPD type=write size=2049801 2024-04-07T16:38:11.629 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:11.636+0000 7f346a3868c0 1 > prepared part key=_multipart_3N0QJRYVO3FV.2~703X6HQZUDE7.0 size=244770 2024-04-07T16:38:11.629 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:11.636+0000 7f346a3868c0 1 > prepared multipart upload key=3N0QJRYVO3FV upload=703X6HQZUDE7 tag=XMFRKKU8BTUB type=write size=1293346 2024-04-07T16:38:11.634 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:11.640+0000 7f346a3868c0 1 < completed delete operation key=E84U0SX44NQ8 tag=FM1U2EVOEM8B type=del 2024-04-07T16:38:11.643 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:11.652+0000 7f346a3868c0 1 > prepared operation key=YOK8BAEOMPUR tag=RZZ3XCCS1TW1 type=del size=2456914 2024-04-07T16:38:11.649 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:11.656+0000 7f346a3868c0 1 < completed part key=_multipart_LXOZ0T2KY6OR.2~1TLHWBQ5JNRT.0 size=1048576 2024-04-07T16:38:11.649 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:11.656+0000 7f346a3868c0 1 < canceled part key=_multipart_LXOZ0T2KY6OR.2~1TLHWBQ5JNRT.1 size=171337 2024-04-07T16:38:11.660 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:11.668+0000 7f346a3868c0 1 < canceled multipart upload key=LXOZ0T2KY6OR upload=1TLHWBQ5JNRT tag=N1ERP0O6003X type=write size=2268489 2024-04-07T16:38:11.666 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:11.672+0000 7f346a3868c0 1 > prepared operation key=GBQSME7SY1AH tag=UBACFAYLPII2 type=del size=3498750 2024-04-07T16:38:11.671 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:11.680+0000 7f346a3868c0 1 < completed write operation key=DXTO3AJ429AC tag=M11EDHG0HYED type=write size=91702 2024-04-07T16:38:11.677 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:11.684+0000 7f346a3868c0 1 > prepared operation key=UIW3DN4NUYPP tag=WNYNDFEG682L type=del size=4102107 2024-04-07T16:38:11.683 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:11.692+0000 7f346a3868c0 1 < completed part key=_multipart_C1BTVJNHIM0X.2~IUQ36PAD9GZR.0 size=1048576 2024-04-07T16:38:11.683 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:11.692+0000 7f346a3868c0 1 < canceled part key=_multipart_C1BTVJNHIM0X.2~IUQ36PAD9GZR.1 size=147573 2024-04-07T16:38:11.693 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:11.700+0000 7f346a3868c0 1 < canceled multipart upload key=C1BTVJNHIM0X upload=IUQ36PAD9GZR tag=S95C142XDLTE type=write size=2244725 2024-04-07T16:38:11.699 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:11.704+0000 7f346a3868c0 1 > prepared operation key=YXMDZB23YTB3 tag=6R33JHNDXMWT type=del size=3046895 2024-04-07T16:38:11.705 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:11.712+0000 7f346a3868c0 1 < completed delete operation key=GBQSME7SY1AH tag=0UUJC6RKJK3L type=del 2024-04-07T16:38:11.716 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:11.724+0000 7f346a3868c0 1 > prepared part key=_multipart_DXTO3AJ429AC.2~JV0Y765NTJ80.0 size=645674 2024-04-07T16:38:11.716 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:11.724+0000 7f346a3868c0 1 > prepared multipart upload key=DXTO3AJ429AC upload=JV0Y765NTJ80 tag=YPHIV5ZKP0RM type=write size=1694250 2024-04-07T16:38:11.722 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:11.728+0000 7f346a3868c0 1 < completed delete operation key=1WGUPH7X5UA0 tag=MJ7OE035WL06 type=del 2024-04-07T16:38:11.728 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:11.736+0000 7f346a3868c0 1 > prepared operation key=E84U0SX44NQ8 tag=7NF1MVM47WOA type=del size=919178 2024-04-07T16:38:11.729 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:11.736+0000 7f346a3868c0 1 < canceled part key=_multipart_3N0QJRYVO3FV.2~PFNBYFM07HX1.0 size=1048576 2024-04-07T16:38:11.734 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:11.740+0000 7f346a3868c0 1 < completed part key=_multipart_3N0QJRYVO3FV.2~PFNBYFM07HX1.1 size=466889 2024-04-07T16:38:11.744 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:11.752+0000 7f346a3868c0 1 < canceled multipart upload key=3N0QJRYVO3FV upload=PFNBYFM07HX1 tag=NN12HAA861I3 type=write size=2564041 2024-04-07T16:38:11.755 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:11.764+0000 7f346a3868c0 1 > prepared part key=_multipart_3N0QJRYVO3FV.2~S02PNTSHMWJC.0 size=1048576 2024-04-07T16:38:11.760 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:11.768+0000 7f346a3868c0 1 > prepared part key=_multipart_3N0QJRYVO3FV.2~S02PNTSHMWJC.1 size=1048576 2024-04-07T16:38:11.765 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:11.772+0000 7f346a3868c0 1 > prepared part key=_multipart_3N0QJRYVO3FV.2~S02PNTSHMWJC.2 size=137801 2024-04-07T16:38:11.765 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:11.772+0000 7f346a3868c0 1 > prepared multipart upload key=3N0QJRYVO3FV upload=S02PNTSHMWJC tag=3375ZB1Z9ZJM type=write size=3283529 2024-04-07T16:38:11.770 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:11.776+0000 7f346a3868c0 1 < completed delete operation key=E84U0SX44NQ8 tag=N1S72OR5P8WU type=del 2024-04-07T16:38:11.776 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:11.784+0000 7f346a3868c0 1 > prepared operation key=C1BTVJNHIM0X tag=WTLAJWK20L48 type=write size=984027 2024-04-07T16:38:11.782 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:11.788+0000 7f346a3868c0 1 < completed part key=_multipart_CL4PP1RKZU0E.2~Z9JGA36I6J1G.0 size=828771 2024-04-07T16:38:11.793 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:11.800+0000 7f346a3868c0 1 < completed multipart upload key=CL4PP1RKZU0E upload=Z9JGA36I6J1G tag=ER5LWFH5UGX8 type=write size=1877347 2024-04-07T16:38:11.798 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:11.804+0000 7f346a3868c0 1 > prepared operation key=YXMDZB23YTB3 tag=JGMJRTWBJDZL type=del size=1628029 2024-04-07T16:38:11.804 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:11.812+0000 7f346a3868c0 1 < completed delete operation key=1WGUPH7X5UA0 tag=QWSCWAUDQC7P type=del 2024-04-07T16:38:11.815 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:11.824+0000 7f346a3868c0 1 > prepared part key=_multipart_UIW3DN4NUYPP.2~4S64UY9EUJQW.0 size=1048576 2024-04-07T16:38:11.820 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:11.828+0000 7f346a3868c0 1 > prepared part key=_multipart_UIW3DN4NUYPP.2~4S64UY9EUJQW.1 size=16447 2024-04-07T16:38:11.820 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:11.828+0000 7f346a3868c0 1 > prepared multipart upload key=UIW3DN4NUYPP upload=4S64UY9EUJQW tag=77TPF2W192DF type=write size=2113599 2024-04-07T16:38:11.826 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:11.832+0000 7f346a3868c0 1 < completed part key=_multipart_7G5C6HBK6QWI.2~023FP1B69C8A.0 size=1048576 2024-04-07T16:38:11.831 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:11.840+0000 7f346a3868c0 1 < completed part key=_multipart_7G5C6HBK6QWI.2~023FP1B69C8A.1 size=1048576 2024-04-07T16:38:11.836 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:11.844+0000 7f346a3868c0 1 < completed part key=_multipart_7G5C6HBK6QWI.2~023FP1B69C8A.2 size=230225 2024-04-07T16:38:11.846 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:11.852+0000 7f346a3868c0 1 < completed multipart upload key=7G5C6HBK6QWI upload=023FP1B69C8A tag=30MKQ99XL9HT type=write size=3375953 2024-04-07T16:38:11.853 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:11.860+0000 7f346a3868c0 1 > prepared operation key=3N0QJRYVO3FV tag=KBHHCDUQQEY5 type=write size=569732 2024-04-07T16:38:11.860 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:11.868+0000 7f346a3868c0 1 < completed delete operation key=3N0QJRYVO3FV tag=0OF26Q6U4S5C type=del 2024-04-07T16:38:11.866 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:11.872+0000 7f346a3868c0 1 > prepared operation key=HBLEXONKXJTL tag=KYB3JOJC6CXG type=del size=3914351 2024-04-07T16:38:11.873 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:11.880+0000 7f346a3868c0 1 < completed delete operation key=N14BUXLJONHZ tag=RF4D4PP88YM4 type=del 2024-04-07T16:38:11.878 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:11.884+0000 7f346a3868c0 1 > prepared operation key=8F81G0P0YBFW tag=0DUV3D1SPNJ3 type=del size=1628587 2024-04-07T16:38:11.891 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:11.900+0000 7f346a3868c0 1 < suggested update operation key=LXOZ0T2KY6OR tag=H57ZJXUNRM3B type=write size=648149 2024-04-07T16:38:11.902 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:11.908+0000 7f346a3868c0 1 > prepared part key=_multipart_GZXHYZO5SMUP.2~X56Z03NPGWGG.0 size=1048576 2024-04-07T16:38:11.908 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:11.916+0000 7f346a3868c0 1 > prepared part key=_multipart_GZXHYZO5SMUP.2~X56Z03NPGWGG.1 size=1048576 2024-04-07T16:38:11.912 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:11.920+0000 7f346a3868c0 1 > prepared part key=_multipart_GZXHYZO5SMUP.2~X56Z03NPGWGG.2 size=971386 2024-04-07T16:38:11.913 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:11.920+0000 7f346a3868c0 1 > prepared multipart upload key=GZXHYZO5SMUP upload=X56Z03NPGWGG tag=Y1V1I8LTUZ9Y type=write size=4117114 2024-04-07T16:38:11.920 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:11.928+0000 7f346a3868c0 -1 < failed to complete operation key=LXOZ0T2KY6OR tag=6APZY0VM7OII type=del size=2877590: (22) Invalid argument 2024-04-07T16:38:11.926 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:11.932+0000 7f346a3868c0 1 > prepared operation key=U3EID5EK64HD tag=PYZMQL8QNWQ3 type=del size=2999709 2024-04-07T16:38:11.932 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:11.940+0000 7f346a3868c0 1 < completed part key=_multipart_3N0QJRYVO3FV.2~703X6HQZUDE7.0 size=244770 2024-04-07T16:38:11.942 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:11.948+0000 7f346a3868c0 1 < completed multipart upload key=3N0QJRYVO3FV upload=703X6HQZUDE7 tag=XMFRKKU8BTUB type=write size=1293346 2024-04-07T16:38:11.948 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:11.956+0000 7f346a3868c0 1 > prepared operation key=QS4HYZFFAJNF tag=21RETFA26MB1 type=del size=2595295 2024-04-07T16:38:11.959 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:11.968+0000 7f346a3868c0 1 < suggested remove operation key=YOK8BAEOMPUR tag=RZZ3XCCS1TW1 type=del 2024-04-07T16:38:11.965 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:11.972+0000 7f346a3868c0 1 > prepared operation key=3N0QJRYVO3FV tag=65UE8VJ1N415 type=del size=1395817 2024-04-07T16:38:11.977 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:11.984+0000 7f346a3868c0 1 < suggested remove operation key=GBQSME7SY1AH tag=UBACFAYLPII2 type=del 2024-04-07T16:38:11.983 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:11.992+0000 7f346a3868c0 1 > prepared operation key=GZXHYZO5SMUP tag=X1GTZYLTOW9B type=del size=1288556 2024-04-07T16:38:11.995 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:12.004+0000 7f346a3868c0 1 < suggested remove operation key=UIW3DN4NUYPP tag=WNYNDFEG682L type=del 2024-04-07T16:38:12.001 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:12.008+0000 7f346a3868c0 1 > prepared operation key=YOK8BAEOMPUR tag=O76JW0ANOED1 type=del size=1369691 2024-04-07T16:38:12.007 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:12.016+0000 7f346a3868c0 1 < completed delete operation key=YXMDZB23YTB3 tag=6R33JHNDXMWT type=del 2024-04-07T16:38:12.013 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:12.020+0000 7f346a3868c0 1 > prepared operation key=E84U0SX44NQ8 tag=UOQYUFSR1E0Q type=write size=468699 2024-04-07T16:38:12.020 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:12.028+0000 7f346a3868c0 1 < completed part key=_multipart_DXTO3AJ429AC.2~JV0Y765NTJ80.0 size=645674 2024-04-07T16:38:12.030 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:12.036+0000 7f346a3868c0 1 < completed multipart upload key=DXTO3AJ429AC upload=JV0Y765NTJ80 tag=YPHIV5ZKP0RM type=write size=1694250 2024-04-07T16:38:12.037 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:12.044+0000 7f346a3868c0 1 > prepared operation key=QTCIIPTXIRW2 tag=M3REJU0N4X4S type=del size=2292353 2024-04-07T16:38:12.042 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:12.048+0000 7f346a3868c0 1 < completed delete operation key=E84U0SX44NQ8 tag=7NF1MVM47WOA type=del 2024-04-07T16:38:12.054 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:12.060+0000 7f346a3868c0 1 > prepared part key=_multipart_YOK8BAEOMPUR.2~USR4OGX1FVGS.0 size=323418 2024-04-07T16:38:12.054 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:12.060+0000 7f346a3868c0 1 > prepared multipart upload key=YOK8BAEOMPUR upload=USR4OGX1FVGS tag=Q4XP8FZD7QGH type=write size=1371994 2024-04-07T16:38:12.064 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:12.072+0000 7f346a3868c0 1 < completed part key=_multipart_3N0QJRYVO3FV.2~S02PNTSHMWJC.0 size=1048576 2024-04-07T16:38:12.069 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:12.076+0000 7f346a3868c0 1 < completed part key=_multipart_3N0QJRYVO3FV.2~S02PNTSHMWJC.1 size=1048576 2024-04-07T16:38:12.074 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:12.080+0000 7f346a3868c0 1 < completed part key=_multipart_3N0QJRYVO3FV.2~S02PNTSHMWJC.2 size=137801 2024-04-07T16:38:12.086 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:12.092+0000 7f346a3868c0 1 < completed multipart upload key=3N0QJRYVO3FV upload=S02PNTSHMWJC tag=3375ZB1Z9ZJM type=write size=3283529 2024-04-07T16:38:12.101 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:12.108+0000 7f346a3868c0 1 > prepared part key=_multipart_QTCIIPTXIRW2.2~8YZSURHIM2K3.0 size=335263 2024-04-07T16:38:12.101 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:12.108+0000 7f346a3868c0 1 > prepared multipart upload key=QTCIIPTXIRW2 upload=8YZSURHIM2K3 tag=VMKCVPH20Y05 type=write size=1383839 2024-04-07T16:38:12.107 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:12.116+0000 7f346a3868c0 1 < completed write operation key=C1BTVJNHIM0X tag=WTLAJWK20L48 type=write size=984027 2024-04-07T16:38:12.114 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:12.120+0000 7f346a3868c0 1 > prepared operation key=HBLEXONKXJTL tag=TZLB01BCA8CQ type=del size=3636239 2024-04-07T16:38:12.120 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:12.128+0000 7f346a3868c0 1 < completed delete operation key=YXMDZB23YTB3 tag=JGMJRTWBJDZL type=del 2024-04-07T16:38:12.131 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:12.140+0000 7f346a3868c0 1 > prepared part key=_multipart_U3EID5EK64HD.2~GQ9H9ZZ5N8HW.0 size=1048576 2024-04-07T16:38:12.136 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:12.144+0000 7f346a3868c0 1 > prepared part key=_multipart_U3EID5EK64HD.2~GQ9H9ZZ5N8HW.1 size=245715 2024-04-07T16:38:12.136 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:12.144+0000 7f346a3868c0 1 > prepared multipart upload key=U3EID5EK64HD upload=GQ9H9ZZ5N8HW tag=GO8W0C2JHA1D type=write size=2342867 2024-04-07T16:38:12.144 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:12.152+0000 7f346a3868c0 1 < completed part key=_multipart_UIW3DN4NUYPP.2~4S64UY9EUJQW.0 size=1048576 2024-04-07T16:38:12.150 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:12.156+0000 7f346a3868c0 1 < completed part key=_multipart_UIW3DN4NUYPP.2~4S64UY9EUJQW.1 size=16447 2024-04-07T16:38:12.163 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:12.168+0000 7f346a3868c0 1 < completed multipart upload key=UIW3DN4NUYPP upload=4S64UY9EUJQW tag=77TPF2W192DF type=write size=2113599 2024-04-07T16:38:12.169 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:12.176+0000 7f346a3868c0 1 > prepared operation key=YOK8BAEOMPUR tag=A8D6XVRG1BKF type=del size=2075652 2024-04-07T16:38:12.183 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:12.192+0000 7f346a3868c0 1 < suggested update operation key=3N0QJRYVO3FV tag=KBHHCDUQQEY5 type=write size=569732 2024-04-07T16:38:12.190 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:12.196+0000 7f346a3868c0 1 > prepared operation key=N14BUXLJONHZ tag=BHHRV4O5N3LS type=del size=450180 2024-04-07T16:38:12.200 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:12.208+0000 7f346a3868c0 1 < completed delete operation key=HBLEXONKXJTL tag=KYB3JOJC6CXG type=del 2024-04-07T16:38:12.206 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:12.212+0000 7f346a3868c0 1 > prepared operation key=X5Q46GO8YTOK tag=KJ38MY5RKEHL type=del size=1770274 2024-04-07T16:38:12.213 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:12.220+0000 7f346a3868c0 1 < completed delete operation key=8F81G0P0YBFW tag=0DUV3D1SPNJ3 type=del 2024-04-07T16:38:12.219 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:12.228+0000 7f346a3868c0 1 > prepared operation key=8F81G0P0YBFW tag=O0C2MDBB20DC type=del size=1186150 2024-04-07T16:38:12.226 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:12.232+0000 7f346a3868c0 1 < completed part key=_multipart_GZXHYZO5SMUP.2~X56Z03NPGWGG.0 size=1048576 2024-04-07T16:38:12.231 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:12.240+0000 7f346a3868c0 1 < completed part key=_multipart_GZXHYZO5SMUP.2~X56Z03NPGWGG.1 size=1048576 2024-04-07T16:38:12.237 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:12.244+0000 7f346a3868c0 1 < completed part key=_multipart_GZXHYZO5SMUP.2~X56Z03NPGWGG.2 size=971386 2024-04-07T16:38:12.248 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:12.256+0000 7f346a3868c0 1 < completed multipart upload key=GZXHYZO5SMUP upload=X56Z03NPGWGG tag=Y1V1I8LTUZ9Y type=write size=4117114 2024-04-07T16:38:12.259 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:12.264+0000 7f346a3868c0 1 > prepared part key=_multipart_DXTO3AJ429AC.2~ZNF3PAXLI8AD.0 size=432015 2024-04-07T16:38:12.259 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:12.264+0000 7f346a3868c0 1 > prepared multipart upload key=DXTO3AJ429AC upload=ZNF3PAXLI8AD tag=7NWS1S42LA4K type=write size=1480591 2024-04-07T16:38:12.271 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:12.280+0000 7f346a3868c0 1 < suggested remove operation key=U3EID5EK64HD tag=PYZMQL8QNWQ3 type=del 2024-04-07T16:38:12.282 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:12.288+0000 7f346a3868c0 1 > prepared part key=_multipart_6XDS0A7L0IY8.2~HSLYA5LX30FE.0 size=1048576 2024-04-07T16:38:12.287 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:12.296+0000 7f346a3868c0 1 > prepared part key=_multipart_6XDS0A7L0IY8.2~HSLYA5LX30FE.1 size=765881 2024-04-07T16:38:12.287 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:12.296+0000 7f346a3868c0 1 > prepared multipart upload key=6XDS0A7L0IY8 upload=HSLYA5LX30FE tag=KSHDK6MOL454 type=write size=2863033 2024-04-07T16:38:12.293 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:12.300+0000 7f346a3868c0 1 < canceled operation key=QS4HYZFFAJNF tag=21RETFA26MB1 type=del size=2595295 2024-04-07T16:38:12.299 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:12.304+0000 7f346a3868c0 1 > prepared operation key=IX3F0IUOKBX1 tag=0MVUVWM8FR4N type=del size=3541846 2024-04-07T16:38:12.306 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:12.312+0000 7f346a3868c0 -1 < failed to complete operation key=3N0QJRYVO3FV tag=65UE8VJ1N415 type=del size=1395817: (22) Invalid argument 2024-04-07T16:38:12.317 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:12.324+0000 7f346a3868c0 1 > prepared part key=_multipart_HBLEXONKXJTL.2~FQLK369U72MN.0 size=1048576 2024-04-07T16:38:12.323 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:12.328+0000 7f346a3868c0 1 > prepared part key=_multipart_HBLEXONKXJTL.2~FQLK369U72MN.1 size=883859 2024-04-07T16:38:12.323 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:12.328+0000 7f346a3868c0 1 > prepared multipart upload key=HBLEXONKXJTL upload=FQLK369U72MN tag=M4NL0AG5RHJ1 type=write size=2981011 2024-04-07T16:38:12.329 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:12.336+0000 7f346a3868c0 1 < completed delete operation key=GZXHYZO5SMUP tag=X1GTZYLTOW9B type=del 2024-04-07T16:38:12.335 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:12.344+0000 7f346a3868c0 1 > prepared operation key=EOYFVEI50SQ1 tag=X7UD77U2FGYW type=del size=2710811 2024-04-07T16:38:12.347 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:12.352+0000 7f346a3868c0 1 < suggested remove operation key=YOK8BAEOMPUR tag=O76JW0ANOED1 type=del 2024-04-07T16:38:12.358 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:12.364+0000 7f346a3868c0 1 > prepared part key=_multipart_XJLQZ7IS9U1T.2~ZF7THL289U6J.0 size=742652 2024-04-07T16:38:12.358 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:12.364+0000 7f346a3868c0 1 > prepared multipart upload key=XJLQZ7IS9U1T upload=ZF7THL289U6J tag=W2QEHYEWDW4Y type=write size=1791228 2024-04-07T16:38:12.364 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:12.372+0000 7f346a3868c0 1 < completed write operation key=E84U0SX44NQ8 tag=UOQYUFSR1E0Q type=write size=468699 2024-04-07T16:38:12.370 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:12.376+0000 7f346a3868c0 1 > prepared operation key=EQR0FY86AT5X tag=EMMF1Q73L47G type=write size=846684 2024-04-07T16:38:12.382 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:12.388+0000 7f346a3868c0 1 < suggested update operation key=QTCIIPTXIRW2 tag=M3REJU0N4X4S type=del size=2292353 2024-04-07T16:38:12.393 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:12.400+0000 7f346a3868c0 1 > prepared part key=_multipart_ROFH30W3UCAX.2~XU73SNM6BNKE.0 size=1048576 2024-04-07T16:38:12.398 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:12.404+0000 7f346a3868c0 1 > prepared part key=_multipart_ROFH30W3UCAX.2~XU73SNM6BNKE.1 size=183247 2024-04-07T16:38:12.398 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:12.404+0000 7f346a3868c0 1 > prepared multipart upload key=ROFH30W3UCAX upload=XU73SNM6BNKE tag=NBYZIZYMFGU2 type=write size=2280399 2024-04-07T16:38:12.407 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:12.412+0000 7f346a3868c0 1 < completed part key=_multipart_YOK8BAEOMPUR.2~USR4OGX1FVGS.0 size=323418 2024-04-07T16:38:12.417 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:12.424+0000 7f346a3868c0 1 < completed multipart upload key=YOK8BAEOMPUR upload=USR4OGX1FVGS tag=Q4XP8FZD7QGH type=write size=1371994 2024-04-07T16:38:12.423 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:12.432+0000 7f346a3868c0 1 > prepared operation key=U3EID5EK64HD tag=XEFW21SI70XH type=del size=2324439 2024-04-07T16:38:12.424 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:12.432+0000 7f346a3868c0 1 < canceled part key=_multipart_QTCIIPTXIRW2.2~8YZSURHIM2K3.0 size=335263 2024-04-07T16:38:12.434 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:12.440+0000 7f346a3868c0 1 < canceled multipart upload key=QTCIIPTXIRW2 upload=8YZSURHIM2K3 tag=VMKCVPH20Y05 type=write size=1383839 2024-04-07T16:38:12.440 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:12.448+0000 7f346a3868c0 1 > prepared operation key=YXMDZB23YTB3 tag=NK9A5KIF5G2L type=del size=2062286 2024-04-07T16:38:12.441 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:38:12 smithi100 bash[76202]: cluster 2024-04-07T16:38:11.070841+0000 mgr.y (mgr.25175) 685 : cluster [DBG] pgmap v1320: 137 pgs: 137 active+clean; 583 KiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 348 KiB/s rd, 120 KiB/s wr, 612 op/s 2024-04-07T16:38:12.441 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:38:12 smithi100 bash[74942]: cluster 2024-04-07T16:38:11.070841+0000 mgr.y (mgr.25175) 685 : cluster [DBG] pgmap v1320: 137 pgs: 137 active+clean; 583 KiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 348 KiB/s rd, 120 KiB/s wr, 612 op/s 2024-04-07T16:38:12.445 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:12.452+0000 7f346a3868c0 1 < completed delete operation key=HBLEXONKXJTL tag=TZLB01BCA8CQ type=del 2024-04-07T16:38:12.451 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:12.460+0000 7f346a3868c0 1 > prepared operation key=DXTO3AJ429AC tag=U565Z7E2N804 type=del size=470597 2024-04-07T16:38:12.452 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:12.460+0000 7f346a3868c0 1 < canceled part key=_multipart_U3EID5EK64HD.2~GQ9H9ZZ5N8HW.0 size=1048576 2024-04-07T16:38:12.457 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:12.464+0000 7f346a3868c0 1 < completed part key=_multipart_U3EID5EK64HD.2~GQ9H9ZZ5N8HW.1 size=245715 2024-04-07T16:38:12.467 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:12.476+0000 7f346a3868c0 1 < canceled multipart upload key=U3EID5EK64HD upload=GQ9H9ZZ5N8HW tag=GO8W0C2JHA1D type=write size=2342867 2024-04-07T16:38:12.478 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:12.484+0000 7f346a3868c0 1 > prepared part key=_multipart_X5Q46GO8YTOK.2~92702EY30V22.0 size=54774 2024-04-07T16:38:12.478 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:12.484+0000 7f346a3868c0 1 > prepared multipart upload key=X5Q46GO8YTOK upload=92702EY30V22 tag=92MLXO42C5V3 type=write size=1103350 2024-04-07T16:38:12.486 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:12.492+0000 7f346a3868c0 -1 < failed to complete operation key=YOK8BAEOMPUR tag=A8D6XVRG1BKF type=del size=2075652: (22) Invalid argument 2024-04-07T16:38:12.494 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:12.500+0000 7f346a3868c0 1 > prepared operation key=YXMDZB23YTB3 tag=ZAQGREPM1R46 type=del size=716741 2024-04-07T16:38:12.502 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:12.508+0000 7f346a3868c0 1 < completed delete operation key=N14BUXLJONHZ tag=BHHRV4O5N3LS type=del 2024-04-07T16:38:12.514 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:12.520+0000 7f346a3868c0 1 > prepared part key=_multipart_EOYFVEI50SQ1.2~WDKLHF5RDHLD.0 size=204521 2024-04-07T16:38:12.514 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:12.520+0000 7f346a3868c0 1 > prepared multipart upload key=EOYFVEI50SQ1 upload=WDKLHF5RDHLD tag=MYEOFSBZI9K4 type=write size=1253097 2024-04-07T16:38:12.519 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:12.528+0000 7f346a3868c0 1 < completed delete operation key=X5Q46GO8YTOK tag=KJ38MY5RKEHL type=del 2024-04-07T16:38:12.530 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:12.536+0000 7f346a3868c0 1 > prepared part key=_multipart_7G5C6HBK6QWI.2~84UPXW9VMDZ8.0 size=1048576 2024-04-07T16:38:12.535 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:12.540+0000 7f346a3868c0 1 > prepared part key=_multipart_7G5C6HBK6QWI.2~84UPXW9VMDZ8.1 size=1048576 2024-04-07T16:38:12.540 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:12.548+0000 7f346a3868c0 1 > prepared part key=_multipart_7G5C6HBK6QWI.2~84UPXW9VMDZ8.2 size=102288 2024-04-07T16:38:12.540 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:12.548+0000 7f346a3868c0 1 > prepared multipart upload key=7G5C6HBK6QWI upload=84UPXW9VMDZ8 tag=5CEV9VFJ6BLH type=write size=3248016 2024-04-07T16:38:12.545 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:12.552+0000 7f346a3868c0 1 < completed delete operation key=8F81G0P0YBFW tag=O0C2MDBB20DC type=del 2024-04-07T16:38:12.550 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:12.556+0000 7f346a3868c0 1 > prepared operation key=IX3F0IUOKBX1 tag=3X94F5MXOWIH type=del size=3746024 2024-04-07T16:38:12.556 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:12.564+0000 7f346a3868c0 1 < completed part key=_multipart_DXTO3AJ429AC.2~ZNF3PAXLI8AD.0 size=432015 2024-04-07T16:38:12.566 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:12.572+0000 7f346a3868c0 1 < completed multipart upload key=DXTO3AJ429AC upload=ZNF3PAXLI8AD tag=7NWS1S42LA4K type=write size=1480591 2024-04-07T16:38:12.572 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:12.580+0000 7f346a3868c0 1 > prepared operation key=EQR0FY86AT5X tag=Y1OJO8GWXKAU type=del size=1914375 2024-04-07T16:38:12.577 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:12.584+0000 7f346a3868c0 1 < completed part key=_multipart_6XDS0A7L0IY8.2~HSLYA5LX30FE.0 size=1048576 2024-04-07T16:38:12.582 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:12.588+0000 7f346a3868c0 1 < completed part key=_multipart_6XDS0A7L0IY8.2~HSLYA5LX30FE.1 size=765881 2024-04-07T16:38:12.593 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:12.600+0000 7f346a3868c0 1 < completed multipart upload key=6XDS0A7L0IY8 upload=HSLYA5LX30FE tag=KSHDK6MOL454 type=write size=2863033 2024-04-07T16:38:12.599 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:12.608+0000 7f346a3868c0 1 > prepared operation key=EQR0FY86AT5X tag=VGB33QRT2N91 type=del size=1692051 2024-04-07T16:38:12.605 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:12.612+0000 7f346a3868c0 1 < completed delete operation key=IX3F0IUOKBX1 tag=0MVUVWM8FR4N type=del 2024-04-07T16:38:12.606 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:38:12 smithi144 bash[95233]: ts=2024-04-07T16:38:12.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:38:12.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:38:12 smithi144 bash[66654]: cluster 2024-04-07T16:38:11.070841+0000 mgr.y (mgr.25175) 685 : cluster [DBG] pgmap v1320: 137 pgs: 137 active+clean; 583 KiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 348 KiB/s rd, 120 KiB/s wr, 612 op/s 2024-04-07T16:38:12.611 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:12.616+0000 7f346a3868c0 1 > prepared operation key=8F81G0P0YBFW tag=V69MYXMEVA6D type=write size=374962 2024-04-07T16:38:12.616 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:12.624+0000 7f346a3868c0 1 < completed part key=_multipart_HBLEXONKXJTL.2~FQLK369U72MN.0 size=1048576 2024-04-07T16:38:12.621 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:12.628+0000 7f346a3868c0 1 < completed part key=_multipart_HBLEXONKXJTL.2~FQLK369U72MN.1 size=883859 2024-04-07T16:38:12.631 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:12.640+0000 7f346a3868c0 1 < completed multipart upload key=HBLEXONKXJTL upload=FQLK369U72MN tag=M4NL0AG5RHJ1 type=write size=2981011 2024-04-07T16:38:12.637 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:12.644+0000 7f346a3868c0 1 > prepared operation key=7G5C6HBK6QWI tag=SK01A0SGQY2D type=write size=51572 2024-04-07T16:38:12.649 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:12.656+0000 7f346a3868c0 1 < suggested remove operation key=EOYFVEI50SQ1 tag=X7UD77U2FGYW type=del 2024-04-07T16:38:12.660 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:12.668+0000 7f346a3868c0 1 > prepared part key=_multipart_EOYFVEI50SQ1.2~AT3HPPDF2KKS.0 size=384533 2024-04-07T16:38:12.660 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:12.668+0000 7f346a3868c0 1 > prepared multipart upload key=EOYFVEI50SQ1 upload=AT3HPPDF2KKS tag=6B4C37B893DV type=write size=1433109 2024-04-07T16:38:12.666 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:12.672+0000 7f346a3868c0 1 < completed part key=_multipart_XJLQZ7IS9U1T.2~ZF7THL289U6J.0 size=742652 2024-04-07T16:38:12.676 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:12.684+0000 7f346a3868c0 1 < completed multipart upload key=XJLQZ7IS9U1T upload=ZF7THL289U6J tag=W2QEHYEWDW4Y type=write size=1791228 2024-04-07T16:38:12.682 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:12.688+0000 7f346a3868c0 1 > prepared operation key=EQR0FY86AT5X tag=YDZT9B6R3SKO type=del size=2828284 2024-04-07T16:38:12.687 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:12.696+0000 7f346a3868c0 1 < canceled operation key=EQR0FY86AT5X tag=EMMF1Q73L47G type=write size=846684 2024-04-07T16:38:12.698 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:12.704+0000 7f346a3868c0 1 > prepared part key=_multipart_U3EID5EK64HD.2~XY3U0992SBQ7.0 size=1048576 2024-04-07T16:38:12.703 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:12.712+0000 7f346a3868c0 1 > prepared part key=_multipart_U3EID5EK64HD.2~XY3U0992SBQ7.1 size=523947 2024-04-07T16:38:12.703 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:12.712+0000 7f346a3868c0 1 > prepared multipart upload key=U3EID5EK64HD upload=XY3U0992SBQ7 tag=92S7SG1QTSJ3 type=write size=2621099 2024-04-07T16:38:12.709 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:12.716+0000 7f346a3868c0 1 < completed part key=_multipart_ROFH30W3UCAX.2~XU73SNM6BNKE.0 size=1048576 2024-04-07T16:38:12.714 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:12.720+0000 7f346a3868c0 1 < completed part key=_multipart_ROFH30W3UCAX.2~XU73SNM6BNKE.1 size=183247 2024-04-07T16:38:12.724 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:12.732+0000 7f346a3868c0 1 < completed multipart upload key=ROFH30W3UCAX upload=XU73SNM6BNKE tag=NBYZIZYMFGU2 type=write size=2280399 2024-04-07T16:38:12.730 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:12.736+0000 7f346a3868c0 1 > prepared operation key=QTCIIPTXIRW2 tag=ZVOWGSFJGTF9 type=del size=2239225 2024-04-07T16:38:12.741 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:12.748+0000 7f346a3868c0 1 < suggested remove operation key=U3EID5EK64HD tag=XEFW21SI70XH type=del 2024-04-07T16:38:12.747 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:12.752+0000 7f346a3868c0 1 > prepared operation key=QS4HYZFFAJNF tag=T6B3F85GOWCZ type=del size=3199356 2024-04-07T16:38:12.758 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:12.764+0000 7f346a3868c0 1 < suggested update operation key=YXMDZB23YTB3 tag=NK9A5KIF5G2L type=del size=2062286 2024-04-07T16:38:12.771 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:12.780+0000 7f346a3868c0 1 > prepared part key=_multipart_CL4PP1RKZU0E.2~EYFSZ3S38QXO.0 size=1048576 2024-04-07T16:38:12.778 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:12.784+0000 7f346a3868c0 1 > prepared part key=_multipart_CL4PP1RKZU0E.2~EYFSZ3S38QXO.1 size=778612 2024-04-07T16:38:12.778 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:12.784+0000 7f346a3868c0 1 > prepared multipart upload key=CL4PP1RKZU0E upload=EYFSZ3S38QXO tag=QBXWO0SD7EAV type=write size=2875764 2024-04-07T16:38:12.785 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:12.792+0000 7f346a3868c0 1 < completed delete operation key=DXTO3AJ429AC tag=U565Z7E2N804 type=del 2024-04-07T16:38:12.792 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:12.800+0000 7f346a3868c0 1 > prepared operation key=LXOZ0T2KY6OR tag=9PPV9OA7ERO5 type=del size=2298477 2024-04-07T16:38:12.798 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:12.804+0000 7f346a3868c0 1 < completed part key=_multipart_X5Q46GO8YTOK.2~92702EY30V22.0 size=54774 2024-04-07T16:38:12.807 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:12.816+0000 7f346a3868c0 1 < completed multipart upload key=X5Q46GO8YTOK upload=92702EY30V22 tag=92MLXO42C5V3 type=write size=1103350 2024-04-07T16:38:12.818 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:12.824+0000 7f346a3868c0 1 > prepared part key=_multipart_N14BUXLJONHZ.2~KKDT3TRVDB2J.0 size=1048576 2024-04-07T16:38:12.823 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:12.832+0000 7f346a3868c0 1 > prepared part key=_multipart_N14BUXLJONHZ.2~KKDT3TRVDB2J.1 size=1048576 2024-04-07T16:38:12.828 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:12.836+0000 7f346a3868c0 1 > prepared part key=_multipart_N14BUXLJONHZ.2~KKDT3TRVDB2J.2 size=762601 2024-04-07T16:38:12.828 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:12.836+0000 7f346a3868c0 1 > prepared multipart upload key=N14BUXLJONHZ upload=KKDT3TRVDB2J tag=0K8D7PNN160J type=write size=3908329 2024-04-07T16:38:12.839 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:12.848+0000 7f346a3868c0 -1 < failed to complete operation key=YXMDZB23YTB3 tag=ZAQGREPM1R46 type=del size=716741: (22) Invalid argument 2024-04-07T16:38:12.845 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:12.852+0000 7f346a3868c0 1 > prepared operation key=DXTO3AJ429AC tag=GJ9AM28BDR1M type=del size=567398 2024-04-07T16:38:12.851 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:12.860+0000 7f346a3868c0 1 < completed part key=_multipart_EOYFVEI50SQ1.2~WDKLHF5RDHLD.0 size=204521 2024-04-07T16:38:12.863 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:12.872+0000 7f346a3868c0 1 < completed multipart upload key=EOYFVEI50SQ1 upload=WDKLHF5RDHLD tag=MYEOFSBZI9K4 type=write size=1253097 2024-04-07T16:38:12.870 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:12.876+0000 7f346a3868c0 1 > prepared operation key=QVOKSQAGHPA9 tag=D2FXGKCNFHH9 type=del size=4178064 2024-04-07T16:38:12.876 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:12.884+0000 7f346a3868c0 1 < completed part key=_multipart_7G5C6HBK6QWI.2~84UPXW9VMDZ8.0 size=1048576 2024-04-07T16:38:12.881 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:12.888+0000 7f346a3868c0 1 < completed part key=_multipart_7G5C6HBK6QWI.2~84UPXW9VMDZ8.1 size=1048576 2024-04-07T16:38:12.886 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:12.892+0000 7f346a3868c0 1 < completed part key=_multipart_7G5C6HBK6QWI.2~84UPXW9VMDZ8.2 size=102288 2024-04-07T16:38:12.896 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:12.904+0000 7f346a3868c0 1 < completed multipart upload key=7G5C6HBK6QWI upload=84UPXW9VMDZ8 tag=5CEV9VFJ6BLH type=write size=3248016 2024-04-07T16:38:12.904 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:12.912+0000 7f346a3868c0 1 > prepared operation key=QS4HYZFFAJNF tag=TRPLAG9GXYTU type=del size=1784435 2024-04-07T16:38:12.910 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:12.916+0000 7f346a3868c0 1 < completed delete operation key=IX3F0IUOKBX1 tag=3X94F5MXOWIH type=del 2024-04-07T16:38:12.916 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:12.924+0000 7f346a3868c0 1 > prepared operation key=YOK8BAEOMPUR tag=U4UMSIRJY9WE type=write size=437205 2024-04-07T16:38:12.922 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:12.928+0000 7f346a3868c0 1 < completed delete operation key=EQR0FY86AT5X tag=Y1OJO8GWXKAU type=del 2024-04-07T16:38:12.933 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:12.940+0000 7f346a3868c0 1 > prepared part key=_multipart_3N0QJRYVO3FV.2~4A6A8W96PTX4.0 size=1048576 2024-04-07T16:38:12.938 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:12.944+0000 7f346a3868c0 1 > prepared part key=_multipart_3N0QJRYVO3FV.2~4A6A8W96PTX4.1 size=1048576 2024-04-07T16:38:12.943 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:12.952+0000 7f346a3868c0 1 > prepared part key=_multipart_3N0QJRYVO3FV.2~4A6A8W96PTX4.2 size=936982 2024-04-07T16:38:12.943 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:12.952+0000 7f346a3868c0 1 > prepared multipart upload key=3N0QJRYVO3FV upload=4A6A8W96PTX4 tag=UGNU1OZJLQFT type=write size=4082710 2024-04-07T16:38:12.949 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:12.956+0000 7f346a3868c0 1 < completed delete operation key=EQR0FY86AT5X tag=VGB33QRT2N91 type=del 2024-04-07T16:38:12.955 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:12.960+0000 7f346a3868c0 1 > prepared operation key=QTCIIPTXIRW2 tag=1GOKPZWQ7QU4 type=del size=3613713 2024-04-07T16:38:12.967 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:12.976+0000 7f346a3868c0 1 < suggested remove operation key=8F81G0P0YBFW tag=V69MYXMEVA6D type=write 2024-04-07T16:38:12.978 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:12.984+0000 7f346a3868c0 1 > prepared part key=_multipart_IX3F0IUOKBX1.2~ZSYAXJBUDD69.0 size=164337 2024-04-07T16:38:12.978 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:12.984+0000 7f346a3868c0 1 > prepared multipart upload key=IX3F0IUOKBX1 upload=ZSYAXJBUDD69 tag=RO1RPFF2RPQV type=write size=1212913 2024-04-07T16:38:12.985 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:12.992+0000 7f346a3868c0 1 < completed write operation key=7G5C6HBK6QWI tag=SK01A0SGQY2D type=write size=51572 2024-04-07T16:38:12.996 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:13.004+0000 7f346a3868c0 1 > prepared part key=_multipart_U3EID5EK64HD.2~DV2CVIUTWDYO.0 size=345979 2024-04-07T16:38:12.996 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:13.004+0000 7f346a3868c0 1 > prepared multipart upload key=U3EID5EK64HD upload=DV2CVIUTWDYO tag=1052VJ169ADW type=write size=1394555 2024-04-07T16:38:13.002 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:13.008+0000 7f346a3868c0 1 < completed part key=_multipart_EOYFVEI50SQ1.2~AT3HPPDF2KKS.0 size=384533 2024-04-07T16:38:13.014 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:13.020+0000 7f346a3868c0 1 < completed multipart upload key=EOYFVEI50SQ1 upload=AT3HPPDF2KKS tag=6B4C37B893DV type=write size=1433109 2024-04-07T16:38:13.025 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:13.032+0000 7f346a3868c0 1 > prepared part key=_multipart_QVOKSQAGHPA9.2~NC1GYYSST97G.0 size=2207 2024-04-07T16:38:13.025 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:13.032+0000 7f346a3868c0 1 > prepared multipart upload key=QVOKSQAGHPA9 upload=NC1GYYSST97G tag=42QF57S3U8J0 type=write size=1050783 2024-04-07T16:38:13.030 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:13.036+0000 7f346a3868c0 1 < completed delete operation key=EQR0FY86AT5X tag=YDZT9B6R3SKO type=del 2024-04-07T16:38:13.036 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:13.044+0000 7f346a3868c0 1 > prepared operation key=95YVKJ14JD8W tag=GQIVKT9BCMXF type=del size=2863450 2024-04-07T16:38:13.042 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:13.048+0000 7f346a3868c0 1 < completed part key=_multipart_U3EID5EK64HD.2~XY3U0992SBQ7.0 size=1048576 2024-04-07T16:38:13.046 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:13.052+0000 7f346a3868c0 1 < completed part key=_multipart_U3EID5EK64HD.2~XY3U0992SBQ7.1 size=523947 2024-04-07T16:38:13.056 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:13.064+0000 7f346a3868c0 1 < completed multipart upload key=U3EID5EK64HD upload=XY3U0992SBQ7 tag=92S7SG1QTSJ3 type=write size=2621099 2024-04-07T16:38:13.066 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:13.072+0000 7f346a3868c0 1 > prepared part key=_multipart_NYET3212USVE.2~UY7WE74LXL3P.0 size=1048576 2024-04-07T16:38:13.073 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:13.080+0000 7f346a3868c0 1 > prepared part key=_multipart_NYET3212USVE.2~UY7WE74LXL3P.1 size=394859 2024-04-07T16:38:13.073 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:13.080+0000 7f346a3868c0 1 > prepared multipart upload key=NYET3212USVE upload=UY7WE74LXL3P tag=3LZIESQ66QN4 type=write size=2492011 2024-04-07T16:38:13.085 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:13.092+0000 7f346a3868c0 1 < suggested remove operation key=QTCIIPTXIRW2 tag=ZVOWGSFJGTF9 type=del 2024-04-07T16:38:13.091 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:13.100+0000 7f346a3868c0 1 > prepared operation key=6XDS0A7L0IY8 tag=L4IX8XK3W3MR type=del size=3348841 2024-04-07T16:38:13.097 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:13.104+0000 7f346a3868c0 1 < completed delete operation key=QS4HYZFFAJNF tag=T6B3F85GOWCZ type=del 2024-04-07T16:38:13.103 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:13.108+0000 7f346a3868c0 1 > prepared operation key=8F81G0P0YBFW tag=FJRP4QTPZZK5 type=write size=422414 2024-04-07T16:38:13.109 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:13.116+0000 7f346a3868c0 1 < completed part key=_multipart_CL4PP1RKZU0E.2~EYFSZ3S38QXO.0 size=1048576 2024-04-07T16:38:13.109 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:13.116+0000 7f346a3868c0 1 < canceled part key=_multipart_CL4PP1RKZU0E.2~EYFSZ3S38QXO.1 size=778612 2024-04-07T16:38:13.120 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:13.128+0000 7f346a3868c0 1 < canceled multipart upload key=CL4PP1RKZU0E upload=EYFSZ3S38QXO tag=QBXWO0SD7EAV type=write size=2875764 2024-04-07T16:38:13.125 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:13.132+0000 7f346a3868c0 1 > prepared operation key=DXTO3AJ429AC tag=ACOUG0HD5OX8 type=del size=2905346 2024-04-07T16:38:13.137 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:13.144+0000 7f346a3868c0 1 < suggested remove operation key=LXOZ0T2KY6OR tag=9PPV9OA7ERO5 type=del 2024-04-07T16:38:13.144 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:13.152+0000 7f346a3868c0 1 > prepared operation key=YOK8BAEOMPUR tag=YQELFIZF1L6B type=del size=163857 2024-04-07T16:38:13.150 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:13.156+0000 7f346a3868c0 1 < completed part key=_multipart_N14BUXLJONHZ.2~KKDT3TRVDB2J.0 size=1048576 2024-04-07T16:38:13.150 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:13.156+0000 7f346a3868c0 1 < canceled part key=_multipart_N14BUXLJONHZ.2~KKDT3TRVDB2J.1 size=1048576 2024-04-07T16:38:13.155 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:13.164+0000 7f346a3868c0 1 < completed part key=_multipart_N14BUXLJONHZ.2~KKDT3TRVDB2J.2 size=762601 2024-04-07T16:38:13.165 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:13.172+0000 7f346a3868c0 1 < canceled multipart upload key=N14BUXLJONHZ upload=KKDT3TRVDB2J tag=0K8D7PNN160J type=write size=3908329 2024-04-07T16:38:13.171 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:13.176+0000 7f346a3868c0 1 > prepared operation key=ROFH30W3UCAX tag=0UD4CN63P31R type=write size=640275 2024-04-07T16:38:13.177 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:13.184+0000 7f346a3868c0 1 < completed delete operation key=DXTO3AJ429AC tag=GJ9AM28BDR1M type=del 2024-04-07T16:38:13.183 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:13.192+0000 7f346a3868c0 1 > prepared operation key=QVOKSQAGHPA9 tag=LGIYHYBBTS4Z type=del size=2267674 2024-04-07T16:38:13.189 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:13.196+0000 7f346a3868c0 1 < canceled operation key=QVOKSQAGHPA9 tag=D2FXGKCNFHH9 type=del size=4178064 2024-04-07T16:38:13.200 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:13.208+0000 7f346a3868c0 1 > prepared part key=_multipart_EQR0FY86AT5X.2~GI8GJ1PHPUSC.0 size=1048576 2024-04-07T16:38:13.205 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:13.212+0000 7f346a3868c0 1 > prepared part key=_multipart_EQR0FY86AT5X.2~GI8GJ1PHPUSC.1 size=758814 2024-04-07T16:38:13.205 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:13.212+0000 7f346a3868c0 1 > prepared multipart upload key=EQR0FY86AT5X upload=GI8GJ1PHPUSC tag=NTAJIASPXCAT type=write size=2855966 2024-04-07T16:38:13.211 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:13.216+0000 7f346a3868c0 1 < completed delete operation key=QS4HYZFFAJNF tag=TRPLAG9GXYTU type=del 2024-04-07T16:38:13.217 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:13.224+0000 7f346a3868c0 1 > prepared operation key=QTCIIPTXIRW2 tag=DJ196T8JWW7X type=del size=1412874 2024-04-07T16:38:13.222 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:13.228+0000 7f346a3868c0 1 < completed write operation key=YOK8BAEOMPUR tag=U4UMSIRJY9WE type=write size=437205 2024-04-07T16:38:13.233 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:13.240+0000 7f346a3868c0 1 > prepared part key=_multipart_E84U0SX44NQ8.2~Y7S5MS5PSN6Y.0 size=1048576 2024-04-07T16:38:13.238 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:13.244+0000 7f346a3868c0 1 > prepared part key=_multipart_E84U0SX44NQ8.2~Y7S5MS5PSN6Y.1 size=1032671 2024-04-07T16:38:13.238 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:13.244+0000 7f346a3868c0 1 > prepared multipart upload key=E84U0SX44NQ8 upload=Y7S5MS5PSN6Y tag=QZ6SLFAB5PJA type=write size=3129823 2024-04-07T16:38:13.239 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:13.248+0000 7f346a3868c0 1 < canceled part key=_multipart_3N0QJRYVO3FV.2~4A6A8W96PTX4.0 size=1048576 2024-04-07T16:38:13.244 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:13.252+0000 7f346a3868c0 1 < completed part key=_multipart_3N0QJRYVO3FV.2~4A6A8W96PTX4.1 size=1048576 2024-04-07T16:38:13.249 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:13.256+0000 7f346a3868c0 1 < completed part key=_multipart_3N0QJRYVO3FV.2~4A6A8W96PTX4.2 size=936982 2024-04-07T16:38:13.259 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:13.268+0000 7f346a3868c0 1 < canceled multipart upload key=3N0QJRYVO3FV upload=4A6A8W96PTX4 tag=UGNU1OZJLQFT type=write size=4082710 2024-04-07T16:38:13.270 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:13.276+0000 7f346a3868c0 1 > prepared part key=_multipart_YXMDZB23YTB3.2~2O4V1696A7EC.0 size=1048576 2024-04-07T16:38:13.275 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:13.280+0000 7f346a3868c0 1 > prepared part key=_multipart_YXMDZB23YTB3.2~2O4V1696A7EC.1 size=1048576 2024-04-07T16:38:13.280 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:13.288+0000 7f346a3868c0 1 > prepared part key=_multipart_YXMDZB23YTB3.2~2O4V1696A7EC.2 size=98472 2024-04-07T16:38:13.280 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:13.288+0000 7f346a3868c0 1 > prepared multipart upload key=YXMDZB23YTB3 upload=2O4V1696A7EC tag=M20DR5UYS1JP type=write size=3244200 2024-04-07T16:38:13.288 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:13.296+0000 7f346a3868c0 -1 < failed to complete operation key=QTCIIPTXIRW2 tag=1GOKPZWQ7QU4 type=del size=3613713: (22) Invalid argument 2024-04-07T16:38:13.293 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:13.300+0000 7f346a3868c0 1 > prepared operation key=X5Q46GO8YTOK tag=61WYTG5OC96X type=del size=817323 2024-04-07T16:38:13.300 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:13.308+0000 7f346a3868c0 1 < completed part key=_multipart_IX3F0IUOKBX1.2~ZSYAXJBUDD69.0 size=164337 2024-04-07T16:38:13.310 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:13.316+0000 7f346a3868c0 1 < completed multipart upload key=IX3F0IUOKBX1 upload=ZSYAXJBUDD69 tag=RO1RPFF2RPQV type=write size=1212913 2024-04-07T16:38:13.320 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:13.328+0000 7f346a3868c0 1 > prepared part key=_multipart_1WGUPH7X5UA0.2~XZHJW6K7NU5R.0 size=256960 2024-04-07T16:38:13.320 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:13.328+0000 7f346a3868c0 1 > prepared multipart upload key=1WGUPH7X5UA0 upload=XZHJW6K7NU5R tag=T1VR0GRD3YDR type=write size=1305536 2024-04-07T16:38:13.326 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:13.332+0000 7f346a3868c0 1 < completed part key=_multipart_U3EID5EK64HD.2~DV2CVIUTWDYO.0 size=345979 2024-04-07T16:38:13.336 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:13.344+0000 7f346a3868c0 1 < completed multipart upload key=U3EID5EK64HD upload=DV2CVIUTWDYO tag=1052VJ169ADW type=write size=1394555 2024-04-07T16:38:13.352 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:13.360+0000 7f346a3868c0 1 > prepared part key=_multipart_X5Q46GO8YTOK.2~ZUA0ORWWF8P2.0 size=476202 2024-04-07T16:38:13.353 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:13.360+0000 7f346a3868c0 1 > prepared multipart upload key=X5Q46GO8YTOK upload=ZUA0ORWWF8P2 tag=22M581A3ZT3C type=write size=1524778 2024-04-07T16:38:13.354 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:13.360+0000 7f346a3868c0 1 < canceled part key=_multipart_QVOKSQAGHPA9.2~NC1GYYSST97G.0 size=2207 2024-04-07T16:38:13.363 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:13.372+0000 7f346a3868c0 1 < canceled multipart upload key=QVOKSQAGHPA9 upload=NC1GYYSST97G tag=42QF57S3U8J0 type=write size=1050783 2024-04-07T16:38:13.370 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:13.376+0000 7f346a3868c0 1 > prepared operation key=IP8YW2A75TCW tag=519O9NNHZJLX type=del size=1126302 2024-04-07T16:38:13.375 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:13.384+0000 7f346a3868c0 1 < completed delete operation key=95YVKJ14JD8W tag=GQIVKT9BCMXF type=del 2024-04-07T16:38:13.386 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:13.392+0000 7f346a3868c0 1 > prepared part key=_multipart_YOK8BAEOMPUR.2~9DL8Y2QSA4KN.0 size=1048576 2024-04-07T16:38:13.391 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:13.400+0000 7f346a3868c0 1 > prepared part key=_multipart_YOK8BAEOMPUR.2~9DL8Y2QSA4KN.1 size=1048576 2024-04-07T16:38:13.396 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:13.404+0000 7f346a3868c0 1 > prepared part key=_multipart_YOK8BAEOMPUR.2~9DL8Y2QSA4KN.2 size=844826 2024-04-07T16:38:13.396 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:13.404+0000 7f346a3868c0 1 > prepared multipart upload key=YOK8BAEOMPUR upload=9DL8Y2QSA4KN tag=LSZ3E7ECUCS8 type=write size=3990554 2024-04-07T16:38:13.402 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:13.408+0000 7f346a3868c0 1 < completed part key=_multipart_NYET3212USVE.2~UY7WE74LXL3P.0 size=1048576 2024-04-07T16:38:13.407 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:13.412+0000 7f346a3868c0 1 < completed part key=_multipart_NYET3212USVE.2~UY7WE74LXL3P.1 size=394859 2024-04-07T16:38:13.418 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:13.424+0000 7f346a3868c0 1 < completed multipart upload key=NYET3212USVE upload=UY7WE74LXL3P tag=3LZIESQ66QN4 type=write size=2492011 2024-04-07T16:38:13.423 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:13.432+0000 7f346a3868c0 1 > prepared operation key=IX3F0IUOKBX1 tag=T3W8U2GNPLTY type=del size=4110574 2024-04-07T16:38:13.429 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:13.436+0000 7f346a3868c0 1 < completed delete operation key=6XDS0A7L0IY8 tag=L4IX8XK3W3MR type=del 2024-04-07T16:38:13.441 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:13.448+0000 7f346a3868c0 1 > prepared part key=_multipart_LXOZ0T2KY6OR.2~OJM3J0WDXQJK.0 size=1048576 2024-04-07T16:38:13.446 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:13.452+0000 7f346a3868c0 1 > prepared part key=_multipart_LXOZ0T2KY6OR.2~OJM3J0WDXQJK.1 size=477411 2024-04-07T16:38:13.446 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:13.452+0000 7f346a3868c0 1 > prepared multipart upload key=LXOZ0T2KY6OR upload=OJM3J0WDXQJK tag=47GAP27LCN84 type=write size=2574563 2024-04-07T16:38:13.452 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:13.460+0000 7f346a3868c0 1 < completed write operation key=8F81G0P0YBFW tag=FJRP4QTPZZK5 type=write size=422414 2024-04-07T16:38:13.458 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:13.464+0000 7f346a3868c0 1 > prepared operation key=U3EID5EK64HD tag=84SLY4HG7N1Q type=del size=2402592 2024-04-07T16:38:13.464 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:13.472+0000 7f346a3868c0 1 < completed delete operation key=DXTO3AJ429AC tag=ACOUG0HD5OX8 type=del 2024-04-07T16:38:13.478 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:13.484+0000 7f346a3868c0 1 > prepared part key=_multipart_CL4PP1RKZU0E.2~2BNRHKW81CC4.0 size=97877 2024-04-07T16:38:13.478 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:13.484+0000 7f346a3868c0 1 > prepared multipart upload key=CL4PP1RKZU0E upload=2BNRHKW81CC4 tag=24ZP8KPNZIC8 type=write size=1146453 2024-04-07T16:38:13.484 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:13.492+0000 7f346a3868c0 1 < completed delete operation key=YOK8BAEOMPUR tag=YQELFIZF1L6B type=del 2024-04-07T16:38:13.495 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:13.504+0000 7f346a3868c0 1 > prepared part key=_multipart_GBQSME7SY1AH.2~A8LX3Z7NMVDL.0 size=1048576 2024-04-07T16:38:13.500 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:13.508+0000 7f346a3868c0 1 > prepared part key=_multipart_GBQSME7SY1AH.2~A8LX3Z7NMVDL.1 size=202143 2024-04-07T16:38:13.500 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:13.508+0000 7f346a3868c0 1 > prepared multipart upload key=GBQSME7SY1AH upload=A8LX3Z7NMVDL tag=NUOR1D8NUR5U type=write size=2299295 2024-04-07T16:38:13.506 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:13.512+0000 7f346a3868c0 1 < completed write operation key=ROFH30W3UCAX tag=0UD4CN63P31R type=write size=640275 2024-04-07T16:38:13.511 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:13.520+0000 7f346a3868c0 1 > prepared operation key=95YVKJ14JD8W tag=Q0YQDSN79O7A type=del size=1958594 2024-04-07T16:38:13.518 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:13.524+0000 7f346a3868c0 1 < completed delete operation key=QVOKSQAGHPA9 tag=LGIYHYBBTS4Z type=del 2024-04-07T16:38:13.526 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:13.532+0000 7f346a3868c0 1 > prepared part key=_multipart_YXMDZB23YTB3.2~55TSL5E4D9O6.0 size=1048576 2024-04-07T16:38:13.531 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:13.540+0000 7f346a3868c0 1 > prepared part key=_multipart_YXMDZB23YTB3.2~55TSL5E4D9O6.1 size=1048576 2024-04-07T16:38:13.536 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:13.544+0000 7f346a3868c0 1 > prepared part key=_multipart_YXMDZB23YTB3.2~55TSL5E4D9O6.2 size=904838 2024-04-07T16:38:13.536 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:13.544+0000 7f346a3868c0 1 > prepared multipart upload key=YXMDZB23YTB3 upload=55TSL5E4D9O6 tag=L1RQW9ZYRSM6 type=write size=4050566 2024-04-07T16:38:13.541 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:13.548+0000 7f346a3868c0 1 < completed part key=_multipart_EQR0FY86AT5X.2~GI8GJ1PHPUSC.0 size=1048576 2024-04-07T16:38:13.546 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:13.552+0000 7f346a3868c0 1 < completed part key=_multipart_EQR0FY86AT5X.2~GI8GJ1PHPUSC.1 size=758814 2024-04-07T16:38:13.555 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:13.564+0000 7f346a3868c0 1 < completed multipart upload key=EQR0FY86AT5X upload=GI8GJ1PHPUSC tag=NTAJIASPXCAT type=write size=2855966 2024-04-07T16:38:13.561 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:13.568+0000 7f346a3868c0 1 > prepared operation key=EQR0FY86AT5X tag=RTTHUBZBQ15K type=del size=3964824 2024-04-07T16:38:13.568 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:13.576+0000 7f346a3868c0 1 < completed delete operation key=QTCIIPTXIRW2 tag=DJ196T8JWW7X type=del 2024-04-07T16:38:13.580 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:13.588+0000 7f346a3868c0 1 > prepared part key=_multipart_YXMDZB23YTB3.2~JF7ZYFAOC2UM.0 size=1048576 2024-04-07T16:38:13.585 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:13.592+0000 7f346a3868c0 1 > prepared part key=_multipart_YXMDZB23YTB3.2~JF7ZYFAOC2UM.1 size=1048576 2024-04-07T16:38:13.591 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:13.596+0000 7f346a3868c0 1 > prepared part key=_multipart_YXMDZB23YTB3.2~JF7ZYFAOC2UM.2 size=183121 2024-04-07T16:38:13.591 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:13.596+0000 7f346a3868c0 1 > prepared multipart upload key=YXMDZB23YTB3 upload=JF7ZYFAOC2UM tag=HXXMTH2NSW1V type=write size=3328849 2024-04-07T16:38:13.597 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:13.604+0000 7f346a3868c0 1 < completed part key=_multipart_E84U0SX44NQ8.2~Y7S5MS5PSN6Y.0 size=1048576 2024-04-07T16:38:13.601 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:13.608+0000 7f346a3868c0 1 < completed part key=_multipart_E84U0SX44NQ8.2~Y7S5MS5PSN6Y.1 size=1032671 2024-04-07T16:38:13.611 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:13.620+0000 7f346a3868c0 1 < completed multipart upload key=E84U0SX44NQ8 upload=Y7S5MS5PSN6Y tag=QZ6SLFAB5PJA type=write size=3129823 2024-04-07T16:38:13.617 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:13.624+0000 7f346a3868c0 1 > prepared operation key=XJLQZ7IS9U1T tag=NS05AU1WUQO1 type=del size=530800 2024-04-07T16:38:13.623 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:13.632+0000 7f346a3868c0 1 < completed part key=_multipart_YXMDZB23YTB3.2~2O4V1696A7EC.0 size=1048576 2024-04-07T16:38:13.628 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:13.636+0000 7f346a3868c0 1 < completed part key=_multipart_YXMDZB23YTB3.2~2O4V1696A7EC.1 size=1048576 2024-04-07T16:38:13.633 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:13.640+0000 7f346a3868c0 1 < completed part key=_multipart_YXMDZB23YTB3.2~2O4V1696A7EC.2 size=98472 2024-04-07T16:38:13.644 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:13.652+0000 7f346a3868c0 1 < completed multipart upload key=YXMDZB23YTB3 upload=2O4V1696A7EC tag=M20DR5UYS1JP type=write size=3244200 2024-04-07T16:38:13.650 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:13.656+0000 7f346a3868c0 1 > prepared operation key=1VKVQXSQMTU1 tag=SX71METANRXU type=del size=232317 2024-04-07T16:38:13.657 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:13.664+0000 7f346a3868c0 1 < canceled operation key=X5Q46GO8YTOK tag=61WYTG5OC96X type=del size=817323 2024-04-07T16:38:13.665 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:13.672+0000 7f346a3868c0 1 > prepared operation key=YXMDZB23YTB3 tag=VWO8LGAS31YC type=del size=3955819 2024-04-07T16:38:13.673 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:13.680+0000 7f346a3868c0 1 < completed part key=_multipart_1WGUPH7X5UA0.2~XZHJW6K7NU5R.0 size=256960 2024-04-07T16:38:13.684 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:13.692+0000 7f346a3868c0 1 < completed multipart upload key=1WGUPH7X5UA0 upload=XZHJW6K7NU5R tag=T1VR0GRD3YDR type=write size=1305536 2024-04-07T16:38:13.697 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:13.704+0000 7f346a3868c0 1 > prepared part key=_multipart_7G5C6HBK6QWI.2~QOXCJ8UMQUFA.0 size=1048576 2024-04-07T16:38:13.702 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:13.708+0000 7f346a3868c0 1 > prepared part key=_multipart_7G5C6HBK6QWI.2~QOXCJ8UMQUFA.1 size=337556 2024-04-07T16:38:13.703 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:13.708+0000 7f346a3868c0 1 > prepared multipart upload key=7G5C6HBK6QWI upload=QOXCJ8UMQUFA tag=QOVJHCE4NPC7 type=write size=2434708 2024-04-07T16:38:13.703 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:13.712+0000 7f346a3868c0 1 < canceled part key=_multipart_X5Q46GO8YTOK.2~ZUA0ORWWF8P2.0 size=476202 2024-04-07T16:38:13.714 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:13.720+0000 7f346a3868c0 1 < canceled multipart upload key=X5Q46GO8YTOK upload=ZUA0ORWWF8P2 tag=22M581A3ZT3C type=write size=1524778 2024-04-07T16:38:13.727 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:13.732+0000 7f346a3868c0 1 > prepared part key=_multipart_YXMDZB23YTB3.2~FJJUB0Q5ON6E.0 size=1048576 2024-04-07T16:38:13.731 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:13.740+0000 7f346a3868c0 1 > prepared part key=_multipart_YXMDZB23YTB3.2~FJJUB0Q5ON6E.1 size=1048576 2024-04-07T16:38:13.737 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:13.744+0000 7f346a3868c0 1 > prepared part key=_multipart_YXMDZB23YTB3.2~FJJUB0Q5ON6E.2 size=912294 2024-04-07T16:38:13.737 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:13.744+0000 7f346a3868c0 1 > prepared multipart upload key=YXMDZB23YTB3 upload=FJJUB0Q5ON6E tag=CSDOP348MACJ type=write size=4058022 2024-04-07T16:38:13.750 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:13.756+0000 7f346a3868c0 1 < suggested update operation key=IP8YW2A75TCW tag=519O9NNHZJLX type=del size=1126302 2024-04-07T16:38:13.756 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:13.764+0000 7f346a3868c0 1 > prepared operation key=HBLEXONKXJTL tag=9SS9Z9T7Z3EV type=del size=2556276 2024-04-07T16:38:13.762 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:13.768+0000 7f346a3868c0 1 < completed part key=_multipart_YOK8BAEOMPUR.2~9DL8Y2QSA4KN.0 size=1048576 2024-04-07T16:38:13.767 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:13.776+0000 7f346a3868c0 1 < completed part key=_multipart_YOK8BAEOMPUR.2~9DL8Y2QSA4KN.1 size=1048576 2024-04-07T16:38:13.773 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:13.780+0000 7f346a3868c0 1 < completed part key=_multipart_YOK8BAEOMPUR.2~9DL8Y2QSA4KN.2 size=844826 2024-04-07T16:38:13.783 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:13.792+0000 7f346a3868c0 1 < completed multipart upload key=YOK8BAEOMPUR upload=9DL8Y2QSA4KN tag=LSZ3E7ECUCS8 type=write size=3990554 2024-04-07T16:38:13.789 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:13.796+0000 7f346a3868c0 1 > prepared operation key=E84U0SX44NQ8 tag=C8INXQNW23ZW type=del size=2741688 2024-04-07T16:38:13.796 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:13.804+0000 7f346a3868c0 1 < completed delete operation key=IX3F0IUOKBX1 tag=T3W8U2GNPLTY type=del 2024-04-07T16:38:13.802 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:13.808+0000 7f346a3868c0 1 > prepared operation key=CL4PP1RKZU0E tag=4TA2RUXYUSQP type=del size=23969 2024-04-07T16:38:13.808 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:13.816+0000 7f346a3868c0 1 < completed part key=_multipart_LXOZ0T2KY6OR.2~OJM3J0WDXQJK.0 size=1048576 2024-04-07T16:38:13.814 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:13.820+0000 7f346a3868c0 1 < completed part key=_multipart_LXOZ0T2KY6OR.2~OJM3J0WDXQJK.1 size=477411 2024-04-07T16:38:13.825 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:13.832+0000 7f346a3868c0 1 < completed multipart upload key=LXOZ0T2KY6OR upload=OJM3J0WDXQJK tag=47GAP27LCN84 type=write size=2574563 2024-04-07T16:38:13.832 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:13.840+0000 7f346a3868c0 1 > prepared operation key=X5Q46GO8YTOK tag=IYYR2K24GDYS type=del size=219352 2024-04-07T16:38:13.845 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:13.852+0000 7f346a3868c0 1 < suggested remove operation key=U3EID5EK64HD tag=84SLY4HG7N1Q type=del 2024-04-07T16:38:13.857 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:13.864+0000 7f346a3868c0 1 > prepared part key=_multipart_DXTO3AJ429AC.2~HWZD2W27QPIC.0 size=360857 2024-04-07T16:38:13.857 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:13.864+0000 7f346a3868c0 1 > prepared multipart upload key=DXTO3AJ429AC upload=HWZD2W27QPIC tag=TBNFDDTBE4XR type=write size=1409433 2024-04-07T16:38:13.865 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:13.872+0000 7f346a3868c0 1 < completed part key=_multipart_CL4PP1RKZU0E.2~2BNRHKW81CC4.0 size=97877 2024-04-07T16:38:13.875 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:13.884+0000 7f346a3868c0 1 < completed multipart upload key=CL4PP1RKZU0E upload=2BNRHKW81CC4 tag=24ZP8KPNZIC8 type=write size=1146453 2024-04-07T16:38:13.886 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:13.892+0000 7f346a3868c0 1 > prepared part key=_multipart_X5Q46GO8YTOK.2~RY5ALBIHQ0BN.0 size=1048576 2024-04-07T16:38:13.892 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:13.900+0000 7f346a3868c0 1 > prepared part key=_multipart_X5Q46GO8YTOK.2~RY5ALBIHQ0BN.1 size=1048576 2024-04-07T16:38:13.897 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:13.904+0000 7f346a3868c0 1 > prepared part key=_multipart_X5Q46GO8YTOK.2~RY5ALBIHQ0BN.2 size=989383 2024-04-07T16:38:13.897 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:13.904+0000 7f346a3868c0 1 > prepared multipart upload key=X5Q46GO8YTOK upload=RY5ALBIHQ0BN tag=SC6OXRNZB7JO type=write size=4135111 2024-04-07T16:38:13.903 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:13.908+0000 7f346a3868c0 1 < completed part key=_multipart_GBQSME7SY1AH.2~A8LX3Z7NMVDL.0 size=1048576 2024-04-07T16:38:13.908 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:13.916+0000 7f346a3868c0 1 < completed part key=_multipart_GBQSME7SY1AH.2~A8LX3Z7NMVDL.1 size=202143 2024-04-07T16:38:13.921 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:13.928+0000 7f346a3868c0 1 < completed multipart upload key=GBQSME7SY1AH upload=A8LX3Z7NMVDL tag=NUOR1D8NUR5U type=write size=2299295 2024-04-07T16:38:13.928 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:13.936+0000 7f346a3868c0 1 > prepared operation key=CL4PP1RKZU0E tag=FOTGSCVO2DAP type=del size=2292941 2024-04-07T16:38:13.934 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:13.940+0000 7f346a3868c0 1 < canceled operation key=95YVKJ14JD8W tag=Q0YQDSN79O7A type=del size=1958594 2024-04-07T16:38:13.940 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:13.948+0000 7f346a3868c0 1 > prepared operation key=HBLEXONKXJTL tag=4NJQQQVNDPI6 type=del size=1356427 2024-04-07T16:38:13.946 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:13.952+0000 7f346a3868c0 1 < completed part key=_multipart_YXMDZB23YTB3.2~55TSL5E4D9O6.0 size=1048576 2024-04-07T16:38:13.951 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:13.960+0000 7f346a3868c0 1 < completed part key=_multipart_YXMDZB23YTB3.2~55TSL5E4D9O6.1 size=1048576 2024-04-07T16:38:13.957 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:13.964+0000 7f346a3868c0 1 < completed part key=_multipart_YXMDZB23YTB3.2~55TSL5E4D9O6.2 size=904838 2024-04-07T16:38:13.968 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:13.976+0000 7f346a3868c0 1 < completed multipart upload key=YXMDZB23YTB3 upload=55TSL5E4D9O6 tag=L1RQW9ZYRSM6 type=write size=4050566 2024-04-07T16:38:13.974 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:13.980+0000 7f346a3868c0 1 > prepared operation key=CL4PP1RKZU0E tag=Q0A4A7V48C3G type=del size=727327 2024-04-07T16:38:13.980 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:13.988+0000 7f346a3868c0 1 < completed delete operation key=EQR0FY86AT5X tag=RTTHUBZBQ15K type=del 2024-04-07T16:38:13.991 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:13.996+0000 7f346a3868c0 1 > prepared part key=_multipart_1WGUPH7X5UA0.2~8CVANJZUGQBB.0 size=1048576 2024-04-07T16:38:13.996 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:14.004+0000 7f346a3868c0 1 > prepared part key=_multipart_1WGUPH7X5UA0.2~8CVANJZUGQBB.1 size=48152 2024-04-07T16:38:13.996 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:14.004+0000 7f346a3868c0 1 > prepared multipart upload key=1WGUPH7X5UA0 upload=8CVANJZUGQBB tag=85NJW1QXL3ZT type=write size=2145304 2024-04-07T16:38:14.001 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:14.008+0000 7f346a3868c0 1 < completed part key=_multipart_YXMDZB23YTB3.2~JF7ZYFAOC2UM.0 size=1048576 2024-04-07T16:38:14.009 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:14.016+0000 7f346a3868c0 1 < completed part key=_multipart_YXMDZB23YTB3.2~JF7ZYFAOC2UM.1 size=1048576 2024-04-07T16:38:14.014 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:14.020+0000 7f346a3868c0 1 < completed part key=_multipart_YXMDZB23YTB3.2~JF7ZYFAOC2UM.2 size=183121 2024-04-07T16:38:14.025 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:14.032+0000 7f346a3868c0 1 < completed multipart upload key=YXMDZB23YTB3 upload=JF7ZYFAOC2UM tag=HXXMTH2NSW1V type=write size=3328849 2024-04-07T16:38:14.030 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:14.036+0000 7f346a3868c0 1 > prepared operation key=EOYFVEI50SQ1 tag=SOBVC830I1XZ type=write size=769859 2024-04-07T16:38:14.037 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:14.044+0000 7f346a3868c0 1 < completed delete operation key=XJLQZ7IS9U1T tag=NS05AU1WUQO1 type=del 2024-04-07T16:38:14.048 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:14.056+0000 7f346a3868c0 1 > prepared part key=_multipart_ROFH30W3UCAX.2~EVQ3YN9ZYZH2.0 size=1048576 2024-04-07T16:38:14.052 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:14.060+0000 7f346a3868c0 1 > prepared part key=_multipart_ROFH30W3UCAX.2~EVQ3YN9ZYZH2.1 size=1048576 2024-04-07T16:38:14.057 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:14.064+0000 7f346a3868c0 1 > prepared part key=_multipart_ROFH30W3UCAX.2~EVQ3YN9ZYZH2.2 size=248093 2024-04-07T16:38:14.057 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:14.064+0000 7f346a3868c0 1 > prepared multipart upload key=ROFH30W3UCAX upload=EVQ3YN9ZYZH2 tag=SJWUPINURWIC type=write size=3393821 2024-04-07T16:38:14.063 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:14.072+0000 7f346a3868c0 1 < completed delete operation key=1VKVQXSQMTU1 tag=SX71METANRXU type=del 2024-04-07T16:38:14.074 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:14.080+0000 7f346a3868c0 1 > prepared part key=_multipart_QTCIIPTXIRW2.2~BTTBPJV9L3Q9.0 size=1048576 2024-04-07T16:38:14.079 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:14.084+0000 7f346a3868c0 1 > prepared part key=_multipart_QTCIIPTXIRW2.2~BTTBPJV9L3Q9.1 size=1048576 2024-04-07T16:38:14.083 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:14.092+0000 7f346a3868c0 1 > prepared part key=_multipart_QTCIIPTXIRW2.2~BTTBPJV9L3Q9.2 size=96905 2024-04-07T16:38:14.083 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:14.092+0000 7f346a3868c0 1 > prepared multipart upload key=QTCIIPTXIRW2 upload=BTTBPJV9L3Q9 tag=ILMPQVD5SRCK type=write size=3242633 2024-04-07T16:38:14.095 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:14.104+0000 7f346a3868c0 1 < suggested update operation key=YXMDZB23YTB3 tag=VWO8LGAS31YC type=del size=3955819 2024-04-07T16:38:14.101 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:14.108+0000 7f346a3868c0 1 > prepared operation key=EQR0FY86AT5X tag=HLBE9IL0SJN2 type=del size=1376623 2024-04-07T16:38:14.107 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:14.112+0000 7f346a3868c0 1 < completed part key=_multipart_7G5C6HBK6QWI.2~QOXCJ8UMQUFA.0 size=1048576 2024-04-07T16:38:14.111 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:14.120+0000 7f346a3868c0 1 < completed part key=_multipart_7G5C6HBK6QWI.2~QOXCJ8UMQUFA.1 size=337556 2024-04-07T16:38:14.121 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:14.128+0000 7f346a3868c0 1 < completed multipart upload key=7G5C6HBK6QWI upload=QOXCJ8UMQUFA tag=QOVJHCE4NPC7 type=write size=2434708 2024-04-07T16:38:14.132 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:14.140+0000 7f346a3868c0 1 > prepared part key=_multipart_7G5C6HBK6QWI.2~J2M6BKKO9K3Z.0 size=240169 2024-04-07T16:38:14.132 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:14.140+0000 7f346a3868c0 1 > prepared multipart upload key=7G5C6HBK6QWI upload=J2M6BKKO9K3Z tag=JUK6FLCTNTYP type=write size=1288745 2024-04-07T16:38:14.133 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:14.140+0000 7f346a3868c0 1 < canceled part key=_multipart_YXMDZB23YTB3.2~FJJUB0Q5ON6E.0 size=1048576 2024-04-07T16:38:14.150 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:14.156+0000 7f346a3868c0 1 < completed part key=_multipart_YXMDZB23YTB3.2~FJJUB0Q5ON6E.1 size=1048576 2024-04-07T16:38:14.158 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:14.164+0000 7f346a3868c0 1 < completed part key=_multipart_YXMDZB23YTB3.2~FJJUB0Q5ON6E.2 size=912294 2024-04-07T16:38:14.170 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:14.176+0000 7f346a3868c0 1 < canceled multipart upload key=YXMDZB23YTB3 upload=FJJUB0Q5ON6E tag=CSDOP348MACJ type=write size=4058022 2024-04-07T16:38:14.181 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:14.188+0000 7f346a3868c0 1 > prepared part key=_multipart_UIW3DN4NUYPP.2~AJR24MAKOYGP.0 size=917136 2024-04-07T16:38:14.181 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:14.188+0000 7f346a3868c0 1 > prepared multipart upload key=UIW3DN4NUYPP upload=AJR24MAKOYGP tag=83V73N0C9O8M type=write size=1965712 2024-04-07T16:38:14.187 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:14.196+0000 7f346a3868c0 1 < completed delete operation key=HBLEXONKXJTL tag=9SS9Z9T7Z3EV type=del 2024-04-07T16:38:14.193 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:14.200+0000 7f346a3868c0 1 > prepared operation key=1VKVQXSQMTU1 tag=Y0B4RVAFMMT4 type=del size=184933 2024-04-07T16:38:14.199 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:14.204+0000 7f346a3868c0 1 < completed delete operation key=E84U0SX44NQ8 tag=C8INXQNW23ZW type=del 2024-04-07T16:38:14.204 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:14.212+0000 7f346a3868c0 1 > prepared operation key=QVOKSQAGHPA9 tag=26DVSZIWGMCV type=del size=954832 2024-04-07T16:38:14.210 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:14.216+0000 7f346a3868c0 1 < completed delete operation key=CL4PP1RKZU0E tag=4TA2RUXYUSQP type=del 2024-04-07T16:38:14.221 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:14.228+0000 7f346a3868c0 1 > prepared part key=_multipart_QVOKSQAGHPA9.2~XLRB9NBK7NDZ.0 size=1048576 2024-04-07T16:38:14.226 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:14.232+0000 7f346a3868c0 1 > prepared part key=_multipart_QVOKSQAGHPA9.2~XLRB9NBK7NDZ.1 size=794229 2024-04-07T16:38:14.226 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:14.232+0000 7f346a3868c0 1 > prepared multipart upload key=QVOKSQAGHPA9 upload=XLRB9NBK7NDZ tag=OYPZOLMDO3JQ type=write size=2891381 2024-04-07T16:38:14.232 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:14.240+0000 7f346a3868c0 1 < completed delete operation key=X5Q46GO8YTOK tag=IYYR2K24GDYS type=del 2024-04-07T16:38:14.238 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:14.244+0000 7f346a3868c0 1 > prepared operation key=DXTO3AJ429AC tag=KJQBNTBNJ9DB type=del size=3814486 2024-04-07T16:38:14.245 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:14.252+0000 7f346a3868c0 1 < completed part key=_multipart_DXTO3AJ429AC.2~HWZD2W27QPIC.0 size=360857 2024-04-07T16:38:14.256 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:14.264+0000 7f346a3868c0 1 < completed multipart upload key=DXTO3AJ429AC upload=HWZD2W27QPIC tag=TBNFDDTBE4XR type=write size=1409433 2024-04-07T16:38:14.261 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:14.268+0000 7f346a3868c0 1 > prepared operation key=C1BTVJNHIM0X tag=CI6AM2NWDQZO type=write size=258161 2024-04-07T16:38:14.268 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:14.276+0000 7f346a3868c0 1 < completed part key=_multipart_X5Q46GO8YTOK.2~RY5ALBIHQ0BN.0 size=1048576 2024-04-07T16:38:14.273 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:14.280+0000 7f346a3868c0 1 < completed part key=_multipart_X5Q46GO8YTOK.2~RY5ALBIHQ0BN.1 size=1048576 2024-04-07T16:38:14.280 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:14.288+0000 7f346a3868c0 1 < completed part key=_multipart_X5Q46GO8YTOK.2~RY5ALBIHQ0BN.2 size=989383 2024-04-07T16:38:14.293 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:14.300+0000 7f346a3868c0 1 < completed multipart upload key=X5Q46GO8YTOK upload=RY5ALBIHQ0BN tag=SC6OXRNZB7JO type=write size=4135111 2024-04-07T16:38:14.305 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:14.312+0000 7f346a3868c0 1 > prepared part key=_multipart_CL4PP1RKZU0E.2~8ODMYPQ93CMG.0 size=1048576 2024-04-07T16:38:14.310 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:14.316+0000 7f346a3868c0 1 > prepared part key=_multipart_CL4PP1RKZU0E.2~8ODMYPQ93CMG.1 size=436080 2024-04-07T16:38:14.311 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:14.316+0000 7f346a3868c0 1 > prepared multipart upload key=CL4PP1RKZU0E upload=8ODMYPQ93CMG tag=4Q3SW9OGITZ6 type=write size=2533232 2024-04-07T16:38:14.316 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:14.324+0000 7f346a3868c0 1 < canceled operation key=CL4PP1RKZU0E tag=FOTGSCVO2DAP type=del size=2292941 2024-04-07T16:38:14.322 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:14.328+0000 7f346a3868c0 1 > prepared operation key=U3EID5EK64HD tag=STNXA27YDD0I type=del size=1902945 2024-04-07T16:38:14.328 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:14.336+0000 7f346a3868c0 1 < completed delete operation key=HBLEXONKXJTL tag=4NJQQQVNDPI6 type=del 2024-04-07T16:38:14.341 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:14.348+0000 7f346a3868c0 1 > prepared part key=_multipart_EQR0FY86AT5X.2~CXVLYXPW4KF4.0 size=1048576 2024-04-07T16:38:14.348 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:14.356+0000 7f346a3868c0 1 > prepared part key=_multipart_EQR0FY86AT5X.2~CXVLYXPW4KF4.1 size=959887 2024-04-07T16:38:14.348 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:14.356+0000 7f346a3868c0 1 > prepared multipart upload key=EQR0FY86AT5X upload=CXVLYXPW4KF4 tag=AT1EDX6M6VYN type=write size=3057039 2024-04-07T16:38:14.354 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:14.360+0000 7f346a3868c0 1 < completed delete operation key=CL4PP1RKZU0E tag=Q0A4A7V48C3G type=del 2024-04-07T16:38:14.360 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:14.368+0000 7f346a3868c0 1 > prepared operation key=7G5C6HBK6QWI tag=3BZNVX4V2NKQ type=del size=2499947 2024-04-07T16:38:14.366 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:14.372+0000 7f346a3868c0 1 < completed part key=_multipart_1WGUPH7X5UA0.2~8CVANJZUGQBB.0 size=1048576 2024-04-07T16:38:14.371 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:14.376+0000 7f346a3868c0 1 < completed part key=_multipart_1WGUPH7X5UA0.2~8CVANJZUGQBB.1 size=48152 2024-04-07T16:38:14.384 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:14.392+0000 7f346a3868c0 1 < completed multipart upload key=1WGUPH7X5UA0 upload=8CVANJZUGQBB tag=85NJW1QXL3ZT type=write size=2145304 2024-04-07T16:38:14.390 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:14.396+0000 7f346a3868c0 1 > prepared operation key=QVOKSQAGHPA9 tag=T8DCFI4EKUM7 type=del size=1176845 2024-04-07T16:38:14.396 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:14.404+0000 7f346a3868c0 1 < completed write operation key=EOYFVEI50SQ1 tag=SOBVC830I1XZ type=write size=769859 2024-04-07T16:38:14.402 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:14.408+0000 7f346a3868c0 1 > prepared operation key=6XDS0A7L0IY8 tag=P47Z8K1A1ORO type=del size=3281195 2024-04-07T16:38:14.408 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:14.416+0000 7f346a3868c0 1 < completed part key=_multipart_ROFH30W3UCAX.2~EVQ3YN9ZYZH2.0 size=1048576 2024-04-07T16:38:14.413 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:14.420+0000 7f346a3868c0 1 < completed part key=_multipart_ROFH30W3UCAX.2~EVQ3YN9ZYZH2.1 size=1048576 2024-04-07T16:38:14.418 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:14.424+0000 7f346a3868c0 1 < completed part key=_multipart_ROFH30W3UCAX.2~EVQ3YN9ZYZH2.2 size=248093 2024-04-07T16:38:14.429 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:14.436+0000 7f346a3868c0 1 < completed multipart upload key=ROFH30W3UCAX upload=EVQ3YN9ZYZH2 tag=SJWUPINURWIC type=write size=3393821 2024-04-07T16:38:14.435 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:14.440+0000 7f346a3868c0 1 > prepared operation key=6XDS0A7L0IY8 tag=3I4X5WIS5OVE type=del size=2443196 2024-04-07T16:38:14.436 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:14.444+0000 7f346a3868c0 1 < canceled part key=_multipart_QTCIIPTXIRW2.2~BTTBPJV9L3Q9.0 size=1048576 2024-04-07T16:38:14.441 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:38:14 smithi100 bash[74942]: cluster 2024-04-07T16:38:13.072138+0000 mgr.y (mgr.25175) 686 : cluster [DBG] pgmap v1321: 137 pgs: 137 active+clean; 583 KiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 515 KiB/s rd, 177 KiB/s wr, 907 op/s 2024-04-07T16:38:14.441 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:38:14 smithi100 bash[76202]: cluster 2024-04-07T16:38:13.072138+0000 mgr.y (mgr.25175) 686 : cluster [DBG] pgmap v1321: 137 pgs: 137 active+clean; 583 KiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 515 KiB/s rd, 177 KiB/s wr, 907 op/s 2024-04-07T16:38:14.441 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:14.448+0000 7f346a3868c0 1 < completed part key=_multipart_QTCIIPTXIRW2.2~BTTBPJV9L3Q9.1 size=1048576 2024-04-07T16:38:14.446 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:14.452+0000 7f346a3868c0 1 < completed part key=_multipart_QTCIIPTXIRW2.2~BTTBPJV9L3Q9.2 size=96905 2024-04-07T16:38:14.456 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:14.464+0000 7f346a3868c0 1 < canceled multipart upload key=QTCIIPTXIRW2 upload=BTTBPJV9L3Q9 tag=ILMPQVD5SRCK type=write size=3242633 2024-04-07T16:38:14.462 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:14.468+0000 7f346a3868c0 1 > prepared operation key=7G5C6HBK6QWI tag=3GV6FNIWO8H3 type=del size=3727108 2024-04-07T16:38:14.468 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:14.476+0000 7f346a3868c0 1 < completed delete operation key=EQR0FY86AT5X tag=HLBE9IL0SJN2 type=del 2024-04-07T16:38:14.474 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:14.480+0000 7f346a3868c0 1 > prepared operation key=1VKVQXSQMTU1 tag=G96YQB8NLDXD type=del size=3071778 2024-04-07T16:38:14.480 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:14.488+0000 7f346a3868c0 1 < completed part key=_multipart_7G5C6HBK6QWI.2~J2M6BKKO9K3Z.0 size=240169 2024-04-07T16:38:14.489 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:14.496+0000 7f346a3868c0 1 < completed multipart upload key=7G5C6HBK6QWI upload=J2M6BKKO9K3Z tag=JUK6FLCTNTYP type=write size=1288745 2024-04-07T16:38:14.495 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:14.504+0000 7f346a3868c0 1 > prepared operation key=EOYFVEI50SQ1 tag=BFJWVNVLV1I6 type=del size=1624889 2024-04-07T16:38:14.501 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:14.508+0000 7f346a3868c0 1 < completed part key=_multipart_UIW3DN4NUYPP.2~AJR24MAKOYGP.0 size=917136 2024-04-07T16:38:14.513 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:14.520+0000 7f346a3868c0 1 < completed multipart upload key=UIW3DN4NUYPP upload=AJR24MAKOYGP tag=83V73N0C9O8M type=write size=1965712 2024-04-07T16:38:14.519 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:14.524+0000 7f346a3868c0 1 > prepared operation key=X5Q46GO8YTOK tag=C0EVCW2IKGY7 type=del size=2989398 2024-04-07T16:38:14.524 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:14.532+0000 7f346a3868c0 1 < completed delete operation key=1VKVQXSQMTU1 tag=Y0B4RVAFMMT4 type=del 2024-04-07T16:38:14.537 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:14.544+0000 7f346a3868c0 1 > prepared part key=_multipart_U3EID5EK64HD.2~8RJ2K21Z0Y9G.0 size=1048576 2024-04-07T16:38:14.540 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:14.548+0000 7f346a3868c0 1 > prepared part key=_multipart_U3EID5EK64HD.2~8RJ2K21Z0Y9G.1 size=706030 2024-04-07T16:38:14.540 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:14.548+0000 7f346a3868c0 1 > prepared multipart upload key=U3EID5EK64HD upload=8RJ2K21Z0Y9G tag=E8GFAPN3V4EC type=write size=2803182 2024-04-07T16:38:14.546 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:14.552+0000 7f346a3868c0 1 < completed delete operation key=QVOKSQAGHPA9 tag=26DVSZIWGMCV type=del 2024-04-07T16:38:14.552 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:14.560+0000 7f346a3868c0 1 > prepared operation key=QTCIIPTXIRW2 tag=1YKZG9V9HRR0 type=del size=2140622 2024-04-07T16:38:14.558 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:14.564+0000 7f346a3868c0 1 < completed part key=_multipart_QVOKSQAGHPA9.2~XLRB9NBK7NDZ.0 size=1048576 2024-04-07T16:38:14.564 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:14.572+0000 7f346a3868c0 1 < completed part key=_multipart_QVOKSQAGHPA9.2~XLRB9NBK7NDZ.1 size=794229 2024-04-07T16:38:14.574 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:14.580+0000 7f346a3868c0 1 < completed multipart upload key=QVOKSQAGHPA9 upload=XLRB9NBK7NDZ tag=OYPZOLMDO3JQ type=write size=2891381 2024-04-07T16:38:14.585 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:14.592+0000 7f346a3868c0 1 > prepared part key=_multipart_LXOZ0T2KY6OR.2~DABN0BRJX0BC.0 size=1048576 2024-04-07T16:38:14.590 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:14.596+0000 7f346a3868c0 1 > prepared part key=_multipart_LXOZ0T2KY6OR.2~DABN0BRJX0BC.1 size=629570 2024-04-07T16:38:14.591 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:14.596+0000 7f346a3868c0 1 > prepared multipart upload key=LXOZ0T2KY6OR upload=DABN0BRJX0BC tag=Y0Q4W4FQ1GAZ type=write size=2726722 2024-04-07T16:38:14.597 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:14.604+0000 7f346a3868c0 1 < completed delete operation key=DXTO3AJ429AC tag=KJQBNTBNJ9DB type=del 2024-04-07T16:38:14.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:38:14 smithi144 bash[66654]: cluster 2024-04-07T16:38:13.072138+0000 mgr.y (mgr.25175) 686 : cluster [DBG] pgmap v1321: 137 pgs: 137 active+clean; 583 KiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 515 KiB/s rd, 177 KiB/s wr, 907 op/s 2024-04-07T16:38:14.609 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:14.616+0000 7f346a3868c0 1 > prepared part key=_multipart_GBQSME7SY1AH.2~53YBM0DYIHG8.0 size=1048576 2024-04-07T16:38:14.614 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:14.620+0000 7f346a3868c0 1 > prepared part key=_multipart_GBQSME7SY1AH.2~53YBM0DYIHG8.1 size=1035405 2024-04-07T16:38:14.614 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:14.620+0000 7f346a3868c0 1 > prepared multipart upload key=GBQSME7SY1AH upload=53YBM0DYIHG8 tag=7NE1ZQ7HVD2V type=write size=3132557 2024-04-07T16:38:14.620 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:14.628+0000 7f346a3868c0 1 < completed write operation key=C1BTVJNHIM0X tag=CI6AM2NWDQZO type=write size=258161 2024-04-07T16:38:14.626 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:14.632+0000 7f346a3868c0 1 > prepared operation key=DXTO3AJ429AC tag=ZXRHMW4ZSD1Y type=del size=2416790 2024-04-07T16:38:14.632 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:14.640+0000 7f346a3868c0 1 < completed part key=_multipart_CL4PP1RKZU0E.2~8ODMYPQ93CMG.0 size=1048576 2024-04-07T16:38:14.637 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:14.644+0000 7f346a3868c0 1 < completed part key=_multipart_CL4PP1RKZU0E.2~8ODMYPQ93CMG.1 size=436080 2024-04-07T16:38:14.649 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:14.656+0000 7f346a3868c0 1 < completed multipart upload key=CL4PP1RKZU0E upload=8ODMYPQ93CMG tag=4Q3SW9OGITZ6 type=write size=2533232 2024-04-07T16:38:14.656 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:14.664+0000 7f346a3868c0 1 > prepared operation key=NYET3212USVE tag=W61VZCTA3DHN type=del size=3528838 2024-04-07T16:38:14.663 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:14.672+0000 7f346a3868c0 1 < completed delete operation key=U3EID5EK64HD tag=STNXA27YDD0I type=del 2024-04-07T16:38:14.670 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:14.676+0000 7f346a3868c0 1 > prepared operation key=3N0QJRYVO3FV tag=S0TDSWYV9U06 type=del size=3332234 2024-04-07T16:38:14.676 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:14.684+0000 7f346a3868c0 1 < completed part key=_multipart_EQR0FY86AT5X.2~CXVLYXPW4KF4.0 size=1048576 2024-04-07T16:38:14.681 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:14.688+0000 7f346a3868c0 1 < completed part key=_multipart_EQR0FY86AT5X.2~CXVLYXPW4KF4.1 size=959887 2024-04-07T16:38:14.692 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:14.700+0000 7f346a3868c0 1 < completed multipart upload key=EQR0FY86AT5X upload=CXVLYXPW4KF4 tag=AT1EDX6M6VYN type=write size=3057039 2024-04-07T16:38:14.699 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:14.704+0000 7f346a3868c0 1 > prepared operation key=QTCIIPTXIRW2 tag=FAQNM0SCBVY3 type=del size=3474411 2024-04-07T16:38:14.710 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:14.716+0000 7f346a3868c0 1 < suggested update operation key=7G5C6HBK6QWI tag=3BZNVX4V2NKQ type=del size=2499947 2024-04-07T16:38:14.721 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:14.728+0000 7f346a3868c0 1 > prepared part key=_multipart_QS4HYZFFAJNF.2~82JBOONQT603.0 size=1048576 2024-04-07T16:38:14.726 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:14.732+0000 7f346a3868c0 1 > prepared part key=_multipart_QS4HYZFFAJNF.2~82JBOONQT603.1 size=1048576 2024-04-07T16:38:14.731 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:14.740+0000 7f346a3868c0 1 > prepared part key=_multipart_QS4HYZFFAJNF.2~82JBOONQT603.2 size=358586 2024-04-07T16:38:14.731 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:14.740+0000 7f346a3868c0 1 > prepared multipart upload key=QS4HYZFFAJNF upload=82JBOONQT603 tag=MWR6TQI54P21 type=write size=3504314 2024-04-07T16:38:14.738 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:14.744+0000 7f346a3868c0 1 < completed delete operation key=QVOKSQAGHPA9 tag=T8DCFI4EKUM7 type=del 2024-04-07T16:38:14.750 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:14.756+0000 7f346a3868c0 1 > prepared part key=_multipart_YOK8BAEOMPUR.2~ZLB00C0XUA80.0 size=1048576 2024-04-07T16:38:14.755 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:14.764+0000 7f346a3868c0 1 > prepared part key=_multipart_YOK8BAEOMPUR.2~ZLB00C0XUA80.1 size=1048576 2024-04-07T16:38:14.760 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:14.768+0000 7f346a3868c0 1 > prepared part key=_multipart_YOK8BAEOMPUR.2~ZLB00C0XUA80.2 size=215033 2024-04-07T16:38:14.760 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:14.768+0000 7f346a3868c0 1 > prepared multipart upload key=YOK8BAEOMPUR upload=ZLB00C0XUA80 tag=NXPMOBTPGG93 type=write size=3360761 2024-04-07T16:38:14.765 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:14.772+0000 7f346a3868c0 1 < completed delete operation key=6XDS0A7L0IY8 tag=P47Z8K1A1ORO type=del 2024-04-07T16:38:14.771 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:14.780+0000 7f346a3868c0 1 > prepared operation key=EOYFVEI50SQ1 tag=58MRCFLWG3TZ type=del size=2719991 2024-04-07T16:38:14.777 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:14.784+0000 7f346a3868c0 1 < completed delete operation key=6XDS0A7L0IY8 tag=3I4X5WIS5OVE type=del 2024-04-07T16:38:14.782 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:14.788+0000 7f346a3868c0 1 > prepared operation key=X5Q46GO8YTOK tag=MPKALJGUOQ79 type=write size=534223 2024-04-07T16:38:14.791 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:14.796+0000 7f346a3868c0 -1 < failed to complete operation key=7G5C6HBK6QWI tag=3GV6FNIWO8H3 type=del size=3727108: (22) Invalid argument 2024-04-07T16:38:14.797 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:14.804+0000 7f346a3868c0 1 > prepared operation key=8F81G0P0YBFW tag=R2T7HHT49JDO type=del size=1019230 2024-04-07T16:38:14.809 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:14.816+0000 7f346a3868c0 1 < suggested update operation key=1VKVQXSQMTU1 tag=G96YQB8NLDXD type=del size=3071778 2024-04-07T16:38:14.820 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:14.828+0000 7f346a3868c0 1 > prepared part key=_multipart_EOYFVEI50SQ1.2~WF30L7AKEQYU.0 size=1048576 2024-04-07T16:38:14.825 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:14.832+0000 7f346a3868c0 1 > prepared part key=_multipart_EOYFVEI50SQ1.2~WF30L7AKEQYU.1 size=560683 2024-04-07T16:38:14.825 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:14.832+0000 7f346a3868c0 1 > prepared multipart upload key=EOYFVEI50SQ1 upload=WF30L7AKEQYU tag=H0V6EX6MUZ1U type=write size=2657835 2024-04-07T16:38:14.831 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:14.840+0000 7f346a3868c0 1 < completed delete operation key=EOYFVEI50SQ1 tag=BFJWVNVLV1I6 type=del 2024-04-07T16:38:14.837 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:14.844+0000 7f346a3868c0 1 > prepared operation key=1VBSUWNI3V1T tag=I6VCQ65F0C8G type=del size=819180 2024-04-07T16:38:14.849 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:14.856+0000 7f346a3868c0 1 < suggested update operation key=X5Q46GO8YTOK tag=C0EVCW2IKGY7 type=del size=2989398 2024-04-07T16:38:14.855 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:14.864+0000 7f346a3868c0 1 > prepared operation key=6XDS0A7L0IY8 tag=JFWSQUOCQRII type=del size=3694096 2024-04-07T16:38:14.861 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:14.868+0000 7f346a3868c0 1 < completed part key=_multipart_U3EID5EK64HD.2~8RJ2K21Z0Y9G.0 size=1048576 2024-04-07T16:38:14.866 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:14.872+0000 7f346a3868c0 1 < completed part key=_multipart_U3EID5EK64HD.2~8RJ2K21Z0Y9G.1 size=706030 2024-04-07T16:38:14.876 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:14.884+0000 7f346a3868c0 1 < completed multipart upload key=U3EID5EK64HD upload=8RJ2K21Z0Y9G tag=E8GFAPN3V4EC type=write size=2803182 2024-04-07T16:38:14.887 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:14.896+0000 7f346a3868c0 1 > prepared part key=_multipart_EQR0FY86AT5X.2~03U0AW1D5X4B.0 size=1048576 2024-04-07T16:38:14.892 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:14.900+0000 7f346a3868c0 1 > prepared part key=_multipart_EQR0FY86AT5X.2~03U0AW1D5X4B.1 size=832157 2024-04-07T16:38:14.892 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:14.900+0000 7f346a3868c0 1 > prepared multipart upload key=EQR0FY86AT5X upload=03U0AW1D5X4B tag=5WVLEPTWUHOR type=write size=2929309 2024-04-07T16:38:14.903 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:14.912+0000 7f346a3868c0 1 < suggested update operation key=QTCIIPTXIRW2 tag=1YKZG9V9HRR0 type=del size=2140622 2024-04-07T16:38:14.909 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:14.916+0000 7f346a3868c0 1 > prepared operation key=DXTO3AJ429AC tag=2946Q0SP06OZ type=del size=3015666 2024-04-07T16:38:14.915 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:14.920+0000 7f346a3868c0 1 < completed part key=_multipart_LXOZ0T2KY6OR.2~DABN0BRJX0BC.0 size=1048576 2024-04-07T16:38:14.920 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:14.928+0000 7f346a3868c0 1 < completed part key=_multipart_LXOZ0T2KY6OR.2~DABN0BRJX0BC.1 size=629570 2024-04-07T16:38:14.930 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:14.936+0000 7f346a3868c0 1 < completed multipart upload key=LXOZ0T2KY6OR upload=DABN0BRJX0BC tag=Y0Q4W4FQ1GAZ type=write size=2726722 2024-04-07T16:38:14.942 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:14.948+0000 7f346a3868c0 1 > prepared part key=_multipart_IX3F0IUOKBX1.2~LV9WHZLUR5E5.0 size=1048576 2024-04-07T16:38:14.947 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:14.952+0000 7f346a3868c0 1 > prepared part key=_multipart_IX3F0IUOKBX1.2~LV9WHZLUR5E5.1 size=153268 2024-04-07T16:38:14.947 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:14.952+0000 7f346a3868c0 1 > prepared multipart upload key=IX3F0IUOKBX1 upload=LV9WHZLUR5E5 tag=1KK0R4493AFY type=write size=2250420 2024-04-07T16:38:14.948 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:14.956+0000 7f346a3868c0 1 < canceled part key=_multipart_GBQSME7SY1AH.2~53YBM0DYIHG8.0 size=1048576 2024-04-07T16:38:14.953 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:14.960+0000 7f346a3868c0 1 < completed part key=_multipart_GBQSME7SY1AH.2~53YBM0DYIHG8.1 size=1035405 2024-04-07T16:38:14.963 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:14.968+0000 7f346a3868c0 1 < canceled multipart upload key=GBQSME7SY1AH upload=53YBM0DYIHG8 tag=7NE1ZQ7HVD2V type=write size=3132557 2024-04-07T16:38:14.975 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:14.980+0000 7f346a3868c0 1 > prepared part key=_multipart_LXOZ0T2KY6OR.2~UJGQO5MAVXJD.0 size=953404 2024-04-07T16:38:14.975 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:14.980+0000 7f346a3868c0 1 > prepared multipart upload key=LXOZ0T2KY6OR upload=UJGQO5MAVXJD tag=B1SLLZFGZMX8 type=write size=2001980 2024-04-07T16:38:14.981 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:14.988+0000 7f346a3868c0 1 < completed delete operation key=DXTO3AJ429AC tag=ZXRHMW4ZSD1Y type=del 2024-04-07T16:38:14.987 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:14.996+0000 7f346a3868c0 1 > prepared operation key=C1BTVJNHIM0X tag=AIT9ZA4JN6UF type=del size=2176232 2024-04-07T16:38:14.993 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:15.000+0000 7f346a3868c0 1 < completed delete operation key=NYET3212USVE tag=W61VZCTA3DHN type=del 2024-04-07T16:38:15.001 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:15.008+0000 7f346a3868c0 1 > prepared operation key=HBLEXONKXJTL tag=7FJAA8JE3R4V type=del size=2188751 2024-04-07T16:38:15.007 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:15.016+0000 7f346a3868c0 1 < completed delete operation key=3N0QJRYVO3FV tag=S0TDSWYV9U06 type=del 2024-04-07T16:38:15.013 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:15.020+0000 7f346a3868c0 1 > prepared operation key=YXMDZB23YTB3 tag=7XVRABDRUP41 type=del size=166918 2024-04-07T16:38:15.021 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:15.028+0000 7f346a3868c0 -1 < failed to complete operation key=QTCIIPTXIRW2 tag=FAQNM0SCBVY3 type=del size=3474411: (22) Invalid argument 2024-04-07T16:38:15.028 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:15.036+0000 7f346a3868c0 1 > prepared operation key=XJLQZ7IS9U1T tag=B3P1JHY2654O type=del size=735212 2024-04-07T16:38:15.035 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:15.044+0000 7f346a3868c0 1 < completed part key=_multipart_QS4HYZFFAJNF.2~82JBOONQT603.0 size=1048576 2024-04-07T16:38:15.035 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:15.044+0000 7f346a3868c0 1 < canceled part key=_multipart_QS4HYZFFAJNF.2~82JBOONQT603.1 size=1048576 2024-04-07T16:38:15.040 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:15.048+0000 7f346a3868c0 1 < completed part key=_multipart_QS4HYZFFAJNF.2~82JBOONQT603.2 size=358586 2024-04-07T16:38:15.051 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:15.056+0000 7f346a3868c0 1 < canceled multipart upload key=QS4HYZFFAJNF upload=82JBOONQT603 tag=MWR6TQI54P21 type=write size=3504314 2024-04-07T16:38:15.062 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:15.068+0000 7f346a3868c0 1 > prepared part key=_multipart_DXTO3AJ429AC.2~SE02I2HG6HVO.0 size=1048576 2024-04-07T16:38:15.068 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:15.076+0000 7f346a3868c0 1 > prepared part key=_multipart_DXTO3AJ429AC.2~SE02I2HG6HVO.1 size=1048576 2024-04-07T16:38:15.073 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:15.080+0000 7f346a3868c0 1 > prepared part key=_multipart_DXTO3AJ429AC.2~SE02I2HG6HVO.2 size=800329 2024-04-07T16:38:15.073 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:15.080+0000 7f346a3868c0 1 > prepared multipart upload key=DXTO3AJ429AC upload=SE02I2HG6HVO tag=KL5ZHK0688IQ type=write size=3946057 2024-04-07T16:38:15.079 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:15.088+0000 7f346a3868c0 1 < completed part key=_multipart_YOK8BAEOMPUR.2~ZLB00C0XUA80.0 size=1048576 2024-04-07T16:38:15.086 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:15.092+0000 7f346a3868c0 1 < completed part key=_multipart_YOK8BAEOMPUR.2~ZLB00C0XUA80.1 size=1048576 2024-04-07T16:38:15.092 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:15.100+0000 7f346a3868c0 1 < completed part key=_multipart_YOK8BAEOMPUR.2~ZLB00C0XUA80.2 size=215033 2024-04-07T16:38:15.102 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:15.108+0000 7f346a3868c0 1 < completed multipart upload key=YOK8BAEOMPUR upload=ZLB00C0XUA80 tag=NXPMOBTPGG93 type=write size=3360761 2024-04-07T16:38:15.108 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:15.116+0000 7f346a3868c0 1 > prepared operation key=YXMDZB23YTB3 tag=TNSR0M7XCHQY type=write size=564822 2024-04-07T16:38:15.113 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:15.120+0000 7f346a3868c0 1 < completed delete operation key=EOYFVEI50SQ1 tag=58MRCFLWG3TZ type=del 2024-04-07T16:38:15.119 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:15.128+0000 7f346a3868c0 1 > prepared operation key=QVOKSQAGHPA9 tag=BD8X4QU1QZVT type=del size=403327 2024-04-07T16:38:15.127 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:15.136+0000 7f346a3868c0 -1 < failed to complete operation key=X5Q46GO8YTOK tag=MPKALJGUOQ79 type=write size=534223: (22) Invalid argument 2024-04-07T16:38:15.138 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:15.144+0000 7f346a3868c0 1 > prepared part key=_multipart_N14BUXLJONHZ.2~MHHZ1Y9XMURV.0 size=1048576 2024-04-07T16:38:15.143 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:15.152+0000 7f346a3868c0 1 > prepared part key=_multipart_N14BUXLJONHZ.2~MHHZ1Y9XMURV.1 size=754193 2024-04-07T16:38:15.143 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:15.152+0000 7f346a3868c0 1 > prepared multipart upload key=N14BUXLJONHZ upload=MHHZ1Y9XMURV tag=TI1E4771Q6PR type=write size=2851345 2024-04-07T16:38:15.151 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:15.160+0000 7f346a3868c0 1 < completed delete operation key=8F81G0P0YBFW tag=R2T7HHT49JDO type=del 2024-04-07T16:38:15.157 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:15.164+0000 7f346a3868c0 1 > prepared operation key=IX3F0IUOKBX1 tag=DNPERAICN2H9 type=del size=4187543 2024-04-07T16:38:15.165 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:15.172+0000 7f346a3868c0 1 < completed part key=_multipart_EOYFVEI50SQ1.2~WF30L7AKEQYU.0 size=1048576 2024-04-07T16:38:15.170 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:15.176+0000 7f346a3868c0 1 < completed part key=_multipart_EOYFVEI50SQ1.2~WF30L7AKEQYU.1 size=560683 2024-04-07T16:38:15.180 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:15.188+0000 7f346a3868c0 1 < completed multipart upload key=EOYFVEI50SQ1 upload=WF30L7AKEQYU tag=H0V6EX6MUZ1U type=write size=2657835 2024-04-07T16:38:15.186 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:15.192+0000 7f346a3868c0 1 > prepared operation key=7G5C6HBK6QWI tag=90V07KU8Y09T type=del size=1838167 2024-04-07T16:38:15.194 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:15.200+0000 7f346a3868c0 1 < completed delete operation key=1VBSUWNI3V1T tag=I6VCQ65F0C8G type=del 2024-04-07T16:38:15.201 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:15.208+0000 7f346a3868c0 1 > prepared operation key=U3EID5EK64HD tag=0MQYBJOUOKRE type=write size=671603 2024-04-07T16:38:15.207 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:15.216+0000 7f346a3868c0 1 < completed delete operation key=6XDS0A7L0IY8 tag=JFWSQUOCQRII type=del 2024-04-07T16:38:15.213 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:15.220+0000 7f346a3868c0 1 > prepared operation key=QS4HYZFFAJNF tag=8F76ECCHO3Y4 type=del size=3496043 2024-04-07T16:38:15.218 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:15.224+0000 7f346a3868c0 1 < completed part key=_multipart_EQR0FY86AT5X.2~03U0AW1D5X4B.0 size=1048576 2024-04-07T16:38:15.223 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:15.232+0000 7f346a3868c0 1 < completed part key=_multipart_EQR0FY86AT5X.2~03U0AW1D5X4B.1 size=832157 2024-04-07T16:38:15.234 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:15.240+0000 7f346a3868c0 1 < completed multipart upload key=EQR0FY86AT5X upload=03U0AW1D5X4B tag=5WVLEPTWUHOR type=write size=2929309 2024-04-07T16:38:15.240 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:15.248+0000 7f346a3868c0 1 > prepared operation key=6XDS0A7L0IY8 tag=IVB506UT42SN type=write size=725358 2024-04-07T16:38:15.247 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:15.256+0000 7f346a3868c0 1 < completed delete operation key=DXTO3AJ429AC tag=2946Q0SP06OZ type=del 2024-04-07T16:38:15.255 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:15.264+0000 7f346a3868c0 1 > prepared operation key=IX3F0IUOKBX1 tag=IWMZB40TI9JU type=del size=2229521 2024-04-07T16:38:15.261 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:15.268+0000 7f346a3868c0 1 < completed part key=_multipart_IX3F0IUOKBX1.2~LV9WHZLUR5E5.0 size=1048576 2024-04-07T16:38:15.266 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:15.272+0000 7f346a3868c0 1 < completed part key=_multipart_IX3F0IUOKBX1.2~LV9WHZLUR5E5.1 size=153268 2024-04-07T16:38:15.276 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:15.284+0000 7f346a3868c0 1 < completed multipart upload key=IX3F0IUOKBX1 upload=LV9WHZLUR5E5 tag=1KK0R4493AFY type=write size=2250420 2024-04-07T16:38:15.282 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:15.288+0000 7f346a3868c0 1 > prepared operation key=HBLEXONKXJTL tag=NTSK6TC87KG3 type=del size=3416581 2024-04-07T16:38:15.288 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:15.296+0000 7f346a3868c0 1 < completed part key=_multipart_LXOZ0T2KY6OR.2~UJGQO5MAVXJD.0 size=953404 2024-04-07T16:38:15.298 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:15.304+0000 7f346a3868c0 1 < completed multipart upload key=LXOZ0T2KY6OR upload=UJGQO5MAVXJD tag=B1SLLZFGZMX8 type=write size=2001980 2024-04-07T16:38:15.304 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:15.312+0000 7f346a3868c0 1 > prepared operation key=ROFH30W3UCAX tag=3PXIE17GX9DY type=del size=1215339 2024-04-07T16:38:15.310 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:15.316+0000 7f346a3868c0 1 < completed delete operation key=C1BTVJNHIM0X tag=AIT9ZA4JN6UF type=del 2024-04-07T16:38:15.322 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:15.328+0000 7f346a3868c0 1 > prepared part key=_multipart_LXOZ0T2KY6OR.2~36SW2OTMTZYD.0 size=1048576 2024-04-07T16:38:15.326 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:15.332+0000 7f346a3868c0 1 > prepared part key=_multipart_LXOZ0T2KY6OR.2~36SW2OTMTZYD.1 size=835797 2024-04-07T16:38:15.326 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:15.332+0000 7f346a3868c0 1 > prepared multipart upload key=LXOZ0T2KY6OR upload=36SW2OTMTZYD tag=QA5HQMI9RC4D type=write size=2932949 2024-04-07T16:38:15.332 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:15.340+0000 7f346a3868c0 1 < completed delete operation key=HBLEXONKXJTL tag=7FJAA8JE3R4V type=del 2024-04-07T16:38:15.338 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:15.344+0000 7f346a3868c0 1 > prepared operation key=HBLEXONKXJTL tag=Q15JHY1ZVTZB type=del size=2812514 2024-04-07T16:38:15.344 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:15.352+0000 7f346a3868c0 1 < canceled operation key=YXMDZB23YTB3 tag=7XVRABDRUP41 type=del size=166918 2024-04-07T16:38:15.349 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:15.356+0000 7f346a3868c0 1 > prepared operation key=95YVKJ14JD8W tag=1H4ZYA4H6D33 type=write size=295331 2024-04-07T16:38:15.363 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:15.372+0000 7f346a3868c0 1 < suggested update operation key=XJLQZ7IS9U1T tag=B3P1JHY2654O type=del size=735212 2024-04-07T16:38:15.370 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:15.376+0000 7f346a3868c0 1 > prepared operation key=8F81G0P0YBFW tag=NJS3U40EHI7V type=write size=351854 2024-04-07T16:38:15.376 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:15.384+0000 7f346a3868c0 1 < completed part key=_multipart_DXTO3AJ429AC.2~SE02I2HG6HVO.0 size=1048576 2024-04-07T16:38:15.381 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:15.388+0000 7f346a3868c0 1 < completed part key=_multipart_DXTO3AJ429AC.2~SE02I2HG6HVO.1 size=1048576 2024-04-07T16:38:15.386 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:15.392+0000 7f346a3868c0 1 < completed part key=_multipart_DXTO3AJ429AC.2~SE02I2HG6HVO.2 size=800329 2024-04-07T16:38:15.395 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:15.404+0000 7f346a3868c0 1 < completed multipart upload key=DXTO3AJ429AC upload=SE02I2HG6HVO tag=KL5ZHK0688IQ type=write size=3946057 2024-04-07T16:38:15.401 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:15.408+0000 7f346a3868c0 1 > prepared operation key=GZXHYZO5SMUP tag=M5FGVLJRQ6F8 type=del size=343949 2024-04-07T16:38:15.406 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:15.412+0000 7f346a3868c0 1 < completed write operation key=YXMDZB23YTB3 tag=TNSR0M7XCHQY type=write size=564822 2024-04-07T16:38:15.419 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:15.428+0000 7f346a3868c0 1 > prepared part key=_multipart_QS4HYZFFAJNF.2~IBKY7Y61UF9I.0 size=1048576 2024-04-07T16:38:15.424 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:15.432+0000 7f346a3868c0 1 > prepared part key=_multipart_QS4HYZFFAJNF.2~IBKY7Y61UF9I.1 size=1048576 2024-04-07T16:38:15.429 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:15.436+0000 7f346a3868c0 1 > prepared part key=_multipart_QS4HYZFFAJNF.2~IBKY7Y61UF9I.2 size=1019700 2024-04-07T16:38:15.429 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:15.436+0000 7f346a3868c0 1 > prepared multipart upload key=QS4HYZFFAJNF upload=IBKY7Y61UF9I tag=RIAV9I8L3ORF type=write size=4165428 2024-04-07T16:38:15.437 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:15.444+0000 7f346a3868c0 1 < completed delete operation key=QVOKSQAGHPA9 tag=BD8X4QU1QZVT type=del 2024-04-07T16:38:15.449 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:15.456+0000 7f346a3868c0 1 > prepared part key=_multipart_XJLQZ7IS9U1T.2~ZEBOAC52LNVI.0 size=1048576 2024-04-07T16:38:15.454 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:15.460+0000 7f346a3868c0 1 > prepared part key=_multipart_XJLQZ7IS9U1T.2~ZEBOAC52LNVI.1 size=1048576 2024-04-07T16:38:15.459 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:15.464+0000 7f346a3868c0 1 > prepared part key=_multipart_XJLQZ7IS9U1T.2~ZEBOAC52LNVI.2 size=425809 2024-04-07T16:38:15.459 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:15.464+0000 7f346a3868c0 1 > prepared multipart upload key=XJLQZ7IS9U1T upload=ZEBOAC52LNVI tag=XADXOMM5O1SV type=write size=3571537 2024-04-07T16:38:15.465 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:15.472+0000 7f346a3868c0 1 < completed part key=_multipart_N14BUXLJONHZ.2~MHHZ1Y9XMURV.0 size=1048576 2024-04-07T16:38:15.469 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:15.476+0000 7f346a3868c0 1 < completed part key=_multipart_N14BUXLJONHZ.2~MHHZ1Y9XMURV.1 size=754193 2024-04-07T16:38:15.479 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:15.488+0000 7f346a3868c0 1 < completed multipart upload key=N14BUXLJONHZ upload=MHHZ1Y9XMURV tag=TI1E4771Q6PR type=write size=2851345 2024-04-07T16:38:15.485 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:15.492+0000 7f346a3868c0 1 > prepared operation key=1WGUPH7X5UA0 tag=CVZFHEAYJPG8 type=del size=1818887 2024-04-07T16:38:15.500 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:15.508+0000 7f346a3868c0 1 < suggested remove operation key=IX3F0IUOKBX1 tag=DNPERAICN2H9 type=del 2024-04-07T16:38:15.506 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:15.512+0000 7f346a3868c0 1 > prepared operation key=YOK8BAEOMPUR tag=QCXMD94EA0K1 type=del size=3788013 2024-04-07T16:38:15.512 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:15.520+0000 7f346a3868c0 1 < completed delete operation key=7G5C6HBK6QWI tag=90V07KU8Y09T type=del 2024-04-07T16:38:15.518 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:15.524+0000 7f346a3868c0 1 > prepared operation key=QS4HYZFFAJNF tag=5R9L9TXCQSXQ type=del size=300498 2024-04-07T16:38:15.524 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:15.532+0000 7f346a3868c0 1 < completed write operation key=U3EID5EK64HD tag=0MQYBJOUOKRE type=write size=671603 2024-04-07T16:38:15.535 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:15.544+0000 7f346a3868c0 1 > prepared part key=_multipart_7G5C6HBK6QWI.2~OAQGHZZJ7IKN.0 size=1024504 2024-04-07T16:38:15.536 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:15.544+0000 7f346a3868c0 1 > prepared multipart upload key=7G5C6HBK6QWI upload=OAQGHZZJ7IKN tag=42LL8NU0UV60 type=write size=2073080 2024-04-07T16:38:15.542 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:15.548+0000 7f346a3868c0 1 < completed delete operation key=QS4HYZFFAJNF tag=8F76ECCHO3Y4 type=del 2024-04-07T16:38:15.548 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:15.556+0000 7f346a3868c0 1 > prepared operation key=CL4PP1RKZU0E tag=NH4RA5C5EZEL type=del size=1788341 2024-04-07T16:38:15.554 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:15.560+0000 7f346a3868c0 1 < canceled operation key=6XDS0A7L0IY8 tag=IVB506UT42SN type=write size=725358 2024-04-07T16:38:15.559 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:15.568+0000 7f346a3868c0 1 > prepared operation key=GBQSME7SY1AH tag=1XTOE0516NY9 type=del size=3444549 2024-04-07T16:38:15.567 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:15.576+0000 7f346a3868c0 -1 < failed to complete operation key=IX3F0IUOKBX1 tag=IWMZB40TI9JU type=del size=2229521: (22) Invalid argument 2024-04-07T16:38:15.573 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:15.580+0000 7f346a3868c0 1 > prepared operation key=95YVKJ14JD8W tag=BLVFQIECL5WU type=del size=1755052 2024-04-07T16:38:15.579 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:15.588+0000 7f346a3868c0 1 < completed delete operation key=HBLEXONKXJTL tag=NTSK6TC87KG3 type=del 2024-04-07T16:38:15.585 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:15.592+0000 7f346a3868c0 1 > prepared operation key=YOK8BAEOMPUR tag=73DA78QIT0XR type=write size=150184 2024-04-07T16:38:15.591 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:15.600+0000 7f346a3868c0 1 < completed delete operation key=ROFH30W3UCAX tag=3PXIE17GX9DY type=del 2024-04-07T16:38:15.603 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:15.612+0000 7f346a3868c0 1 > prepared part key=_multipart_1VBSUWNI3V1T.2~Q88ZFNLSUY8J.0 size=1048576 2024-04-07T16:38:15.608 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:15.616+0000 7f346a3868c0 1 > prepared part key=_multipart_1VBSUWNI3V1T.2~Q88ZFNLSUY8J.1 size=1048576 2024-04-07T16:38:15.613 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:15.620+0000 7f346a3868c0 1 > prepared part key=_multipart_1VBSUWNI3V1T.2~Q88ZFNLSUY8J.2 size=905049 2024-04-07T16:38:15.613 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:15.620+0000 7f346a3868c0 1 > prepared multipart upload key=1VBSUWNI3V1T upload=Q88ZFNLSUY8J tag=E9BGFK52HWWC type=write size=4050777 2024-04-07T16:38:15.618 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:15.624+0000 7f346a3868c0 1 < completed part key=_multipart_LXOZ0T2KY6OR.2~36SW2OTMTZYD.0 size=1048576 2024-04-07T16:38:15.624 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:15.632+0000 7f346a3868c0 1 < completed part key=_multipart_LXOZ0T2KY6OR.2~36SW2OTMTZYD.1 size=835797 2024-04-07T16:38:15.635 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:15.640+0000 7f346a3868c0 1 < completed multipart upload key=LXOZ0T2KY6OR upload=36SW2OTMTZYD tag=QA5HQMI9RC4D type=write size=2932949 2024-04-07T16:38:15.641 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:15.648+0000 7f346a3868c0 1 > prepared operation key=GBQSME7SY1AH tag=OHXPLH0BEXNS type=del size=1251462 2024-04-07T16:38:15.647 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:15.656+0000 7f346a3868c0 1 < completed delete operation key=HBLEXONKXJTL tag=Q15JHY1ZVTZB type=del 2024-04-07T16:38:15.658 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:15.664+0000 7f346a3868c0 1 > prepared part key=_multipart_QVOKSQAGHPA9.2~WNBGSPUIS3JI.0 size=1048576 2024-04-07T16:38:15.663 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:15.672+0000 7f346a3868c0 1 > prepared part key=_multipart_QVOKSQAGHPA9.2~WNBGSPUIS3JI.1 size=240727 2024-04-07T16:38:15.663 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:15.672+0000 7f346a3868c0 1 > prepared multipart upload key=QVOKSQAGHPA9 upload=WNBGSPUIS3JI tag=ELW14CBMO6UK type=write size=2337879 2024-04-07T16:38:15.670 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:15.676+0000 7f346a3868c0 1 < completed write operation key=95YVKJ14JD8W tag=1H4ZYA4H6D33 type=write size=295331 2024-04-07T16:38:15.676 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:15.684+0000 7f346a3868c0 1 > prepared operation key=N14BUXLJONHZ tag=S1AUZBNC8TS9 type=del size=2821593 2024-04-07T16:38:15.681 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:15.688+0000 7f346a3868c0 1 < completed write operation key=8F81G0P0YBFW tag=NJS3U40EHI7V type=write size=351854 2024-04-07T16:38:15.687 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:15.696+0000 7f346a3868c0 1 > prepared operation key=ROFH30W3UCAX tag=40WD7BDC90VI type=del size=201475 2024-04-07T16:38:15.693 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:15.700+0000 7f346a3868c0 1 < completed delete operation key=GZXHYZO5SMUP tag=M5FGVLJRQ6F8 type=del 2024-04-07T16:38:15.699 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:15.708+0000 7f346a3868c0 1 > prepared operation key=QTCIIPTXIRW2 tag=9E58R71WLEMU type=del size=1596074 2024-04-07T16:38:15.707 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:15.712+0000 7f346a3868c0 1 < completed part key=_multipart_QS4HYZFFAJNF.2~IBKY7Y61UF9I.0 size=1048576 2024-04-07T16:38:15.713 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:15.720+0000 7f346a3868c0 1 < completed part key=_multipart_QS4HYZFFAJNF.2~IBKY7Y61UF9I.1 size=1048576 2024-04-07T16:38:15.720 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:15.728+0000 7f346a3868c0 1 < completed part key=_multipart_QS4HYZFFAJNF.2~IBKY7Y61UF9I.2 size=1019700 2024-04-07T16:38:15.730 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:15.736+0000 7f346a3868c0 1 < completed multipart upload key=QS4HYZFFAJNF upload=IBKY7Y61UF9I tag=RIAV9I8L3ORF type=write size=4165428 2024-04-07T16:38:15.741 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:15.748+0000 7f346a3868c0 1 > prepared part key=_multipart_3N0QJRYVO3FV.2~ITIMVXZE67TW.0 size=464886 2024-04-07T16:38:15.741 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:15.748+0000 7f346a3868c0 1 > prepared multipart upload key=3N0QJRYVO3FV upload=ITIMVXZE67TW tag=K1KIQ0MIOGX4 type=write size=1513462 2024-04-07T16:38:15.747 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:15.756+0000 7f346a3868c0 1 < completed part key=_multipart_XJLQZ7IS9U1T.2~ZEBOAC52LNVI.0 size=1048576 2024-04-07T16:38:15.752 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:15.760+0000 7f346a3868c0 1 < completed part key=_multipart_XJLQZ7IS9U1T.2~ZEBOAC52LNVI.1 size=1048576 2024-04-07T16:38:15.761 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:15.768+0000 7f346a3868c0 1 < completed part key=_multipart_XJLQZ7IS9U1T.2~ZEBOAC52LNVI.2 size=425809 2024-04-07T16:38:15.770 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:15.776+0000 7f346a3868c0 1 < completed multipart upload key=XJLQZ7IS9U1T upload=ZEBOAC52LNVI tag=XADXOMM5O1SV type=write size=3571537 2024-04-07T16:38:15.780 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:15.788+0000 7f346a3868c0 1 > prepared part key=_multipart_YXMDZB23YTB3.2~2ZEJZ57LQD5E.0 size=1048576 2024-04-07T16:38:15.785 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:15.792+0000 7f346a3868c0 1 > prepared part key=_multipart_YXMDZB23YTB3.2~2ZEJZ57LQD5E.1 size=1048576 2024-04-07T16:38:15.790 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:15.796+0000 7f346a3868c0 1 > prepared part key=_multipart_YXMDZB23YTB3.2~2ZEJZ57LQD5E.2 size=1027138 2024-04-07T16:38:15.790 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:15.796+0000 7f346a3868c0 1 > prepared multipart upload key=YXMDZB23YTB3 upload=2ZEJZ57LQD5E tag=NN2NT0R81MAF type=write size=4172866 2024-04-07T16:38:15.796 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:15.804+0000 7f346a3868c0 1 < completed delete operation key=1WGUPH7X5UA0 tag=CVZFHEAYJPG8 type=del 2024-04-07T16:38:15.804 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:15.812+0000 7f346a3868c0 1 > prepared operation key=XJLQZ7IS9U1T tag=NW0PO3BJ3OD7 type=del size=474919 2024-04-07T16:38:15.810 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:15.816+0000 7f346a3868c0 1 < completed delete operation key=YOK8BAEOMPUR tag=QCXMD94EA0K1 type=del 2024-04-07T16:38:15.822 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:15.828+0000 7f346a3868c0 1 > prepared part key=_multipart_N14BUXLJONHZ.2~YMSR5T37QVU6.0 size=1048576 2024-04-07T16:38:15.829 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:15.836+0000 7f346a3868c0 1 > prepared part key=_multipart_N14BUXLJONHZ.2~YMSR5T37QVU6.1 size=640316 2024-04-07T16:38:15.829 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:15.836+0000 7f346a3868c0 1 > prepared multipart upload key=N14BUXLJONHZ upload=YMSR5T37QVU6 tag=3GW008X5NCB3 type=write size=2737468 2024-04-07T16:38:15.837 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:15.844+0000 7f346a3868c0 1 < completed delete operation key=QS4HYZFFAJNF tag=5R9L9TXCQSXQ type=del 2024-04-07T16:38:15.852 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:15.860+0000 7f346a3868c0 1 > prepared part key=_multipart_6XDS0A7L0IY8.2~D1VI5N1XLFVS.0 size=1048576 2024-04-07T16:38:15.856 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:15.864+0000 7f346a3868c0 1 > prepared part key=_multipart_6XDS0A7L0IY8.2~D1VI5N1XLFVS.1 size=1048576 2024-04-07T16:38:15.864 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:15.872+0000 7f346a3868c0 1 > prepared part key=_multipart_6XDS0A7L0IY8.2~D1VI5N1XLFVS.2 size=918944 2024-04-07T16:38:15.864 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:15.872+0000 7f346a3868c0 1 > prepared multipart upload key=6XDS0A7L0IY8 upload=D1VI5N1XLFVS tag=QD92ICZCKIJL type=write size=4064672 2024-04-07T16:38:15.870 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:15.876+0000 7f346a3868c0 1 < completed part key=_multipart_7G5C6HBK6QWI.2~OAQGHZZJ7IKN.0 size=1024504 2024-04-07T16:38:15.880 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:15.888+0000 7f346a3868c0 1 < completed multipart upload key=7G5C6HBK6QWI upload=OAQGHZZJ7IKN tag=42LL8NU0UV60 type=write size=2073080 2024-04-07T16:38:15.891 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:15.900+0000 7f346a3868c0 1 > prepared part key=_multipart_7G5C6HBK6QWI.2~TV1H0N18YQ60.0 size=1048576 2024-04-07T16:38:15.897 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:15.904+0000 7f346a3868c0 1 > prepared part key=_multipart_7G5C6HBK6QWI.2~TV1H0N18YQ60.1 size=1048576 2024-04-07T16:38:15.901 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:15.908+0000 7f346a3868c0 1 > prepared part key=_multipart_7G5C6HBK6QWI.2~TV1H0N18YQ60.2 size=35644 2024-04-07T16:38:15.901 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:15.908+0000 7f346a3868c0 1 > prepared multipart upload key=7G5C6HBK6QWI upload=TV1H0N18YQ60 tag=1U2TJQETE2XQ type=write size=3181372 2024-04-07T16:38:15.915 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:15.920+0000 7f346a3868c0 1 < suggested update operation key=CL4PP1RKZU0E tag=NH4RA5C5EZEL type=del size=1788341 2024-04-07T16:38:15.920 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:15.928+0000 7f346a3868c0 1 > prepared operation key=CL4PP1RKZU0E tag=9KZQD875JPFE type=write size=137613 2024-04-07T16:38:15.926 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:15.932+0000 7f346a3868c0 1 < completed delete operation key=GBQSME7SY1AH tag=1XTOE0516NY9 type=del 2024-04-07T16:38:15.932 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:15.940+0000 7f346a3868c0 1 > prepared operation key=E84U0SX44NQ8 tag=1J7VLJW86ESP type=del size=592282 2024-04-07T16:38:15.938 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:15.944+0000 7f346a3868c0 1 < completed delete operation key=95YVKJ14JD8W tag=BLVFQIECL5WU type=del 2024-04-07T16:38:15.944 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:15.952+0000 7f346a3868c0 1 > prepared operation key=X5Q46GO8YTOK tag=BG41MYGHAJ99 type=del size=834776 2024-04-07T16:38:15.950 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:15.956+0000 7f346a3868c0 1 < completed write operation key=YOK8BAEOMPUR tag=73DA78QIT0XR type=write size=150184 2024-04-07T16:38:15.956 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:15.964+0000 7f346a3868c0 1 > prepared operation key=LXOZ0T2KY6OR tag=8B1DQ2SQ2BOS type=write size=255096 2024-04-07T16:38:15.965 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:15.972+0000 7f346a3868c0 1 < completed part key=_multipart_1VBSUWNI3V1T.2~Q88ZFNLSUY8J.0 size=1048576 2024-04-07T16:38:15.966 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:15.972+0000 7f346a3868c0 1 < canceled part key=_multipart_1VBSUWNI3V1T.2~Q88ZFNLSUY8J.1 size=1048576 2024-04-07T16:38:15.969 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:15.976+0000 7f346a3868c0 1 < completed part key=_multipart_1VBSUWNI3V1T.2~Q88ZFNLSUY8J.2 size=905049 2024-04-07T16:38:15.982 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:15.988+0000 7f346a3868c0 1 < canceled multipart upload key=1VBSUWNI3V1T upload=Q88ZFNLSUY8J tag=E9BGFK52HWWC type=write size=4050777 2024-04-07T16:38:15.988 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:15.996+0000 7f346a3868c0 1 > prepared operation key=C1BTVJNHIM0X tag=ODXHVF3BT87S type=write size=741613 2024-04-07T16:38:15.995 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.004+0000 7f346a3868c0 1 < completed delete operation key=GBQSME7SY1AH tag=OHXPLH0BEXNS type=del 2024-04-07T16:38:16.001 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.008+0000 7f346a3868c0 1 > prepared operation key=NYET3212USVE tag=8YBXGUR4PO3F type=write size=541788 2024-04-07T16:38:16.007 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.016+0000 7f346a3868c0 1 < completed part key=_multipart_QVOKSQAGHPA9.2~WNBGSPUIS3JI.0 size=1048576 2024-04-07T16:38:16.012 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.020+0000 7f346a3868c0 1 < completed part key=_multipart_QVOKSQAGHPA9.2~WNBGSPUIS3JI.1 size=240727 2024-04-07T16:38:16.023 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.028+0000 7f346a3868c0 1 < completed multipart upload key=QVOKSQAGHPA9 upload=WNBGSPUIS3JI tag=ELW14CBMO6UK type=write size=2337879 2024-04-07T16:38:16.034 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.040+0000 7f346a3868c0 1 > prepared part key=_multipart_YXMDZB23YTB3.2~WWFZW9X65SQ5.0 size=1048576 2024-04-07T16:38:16.038 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.044+0000 7f346a3868c0 1 > prepared part key=_multipart_YXMDZB23YTB3.2~WWFZW9X65SQ5.1 size=1048576 2024-04-07T16:38:16.043 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.052+0000 7f346a3868c0 1 > prepared part key=_multipart_YXMDZB23YTB3.2~WWFZW9X65SQ5.2 size=434391 2024-04-07T16:38:16.043 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.052+0000 7f346a3868c0 1 > prepared multipart upload key=YXMDZB23YTB3 upload=WWFZW9X65SQ5 tag=K8JRHPPX1UHO type=write size=3580119 2024-04-07T16:38:16.049 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.056+0000 7f346a3868c0 1 < completed delete operation key=N14BUXLJONHZ tag=S1AUZBNC8TS9 type=del 2024-04-07T16:38:16.062 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.068+0000 7f346a3868c0 1 > prepared part key=_multipart_1VKVQXSQMTU1.2~2W2D9OUE7ZZQ.0 size=1048576 2024-04-07T16:38:16.067 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.076+0000 7f346a3868c0 1 > prepared part key=_multipart_1VKVQXSQMTU1.2~2W2D9OUE7ZZQ.1 size=789028 2024-04-07T16:38:16.068 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.076+0000 7f346a3868c0 1 > prepared multipart upload key=1VKVQXSQMTU1 upload=2W2D9OUE7ZZQ tag=NAQM2MN2QBF6 type=write size=2886180 2024-04-07T16:38:16.073 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.080+0000 7f346a3868c0 1 < completed delete operation key=ROFH30W3UCAX tag=40WD7BDC90VI type=del 2024-04-07T16:38:16.079 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.088+0000 7f346a3868c0 1 > prepared operation key=GZXHYZO5SMUP tag=UBQ84Y139EA9 type=del size=3927036 2024-04-07T16:38:16.091 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.100+0000 7f346a3868c0 1 < suggested remove operation key=QTCIIPTXIRW2 tag=9E58R71WLEMU type=del 2024-04-07T16:38:16.098 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.104+0000 7f346a3868c0 1 > prepared operation key=1VBSUWNI3V1T tag=VL80TMRHGK4M type=del size=1843004 2024-04-07T16:38:16.104 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.112+0000 7f346a3868c0 1 < completed part key=_multipart_3N0QJRYVO3FV.2~ITIMVXZE67TW.0 size=464886 2024-04-07T16:38:16.114 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.120+0000 7f346a3868c0 1 < completed multipart upload key=3N0QJRYVO3FV upload=ITIMVXZE67TW tag=K1KIQ0MIOGX4 type=write size=1513462 2024-04-07T16:38:16.125 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.132+0000 7f346a3868c0 1 > prepared part key=_multipart_E84U0SX44NQ8.2~B8UUNE4IJ7RM.0 size=720310 2024-04-07T16:38:16.125 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.132+0000 7f346a3868c0 1 > prepared multipart upload key=E84U0SX44NQ8 upload=B8UUNE4IJ7RM tag=M2AVYW21O5E0 type=write size=1768886 2024-04-07T16:38:16.131 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.140+0000 7f346a3868c0 1 < completed part key=_multipart_YXMDZB23YTB3.2~2ZEJZ57LQD5E.0 size=1048576 2024-04-07T16:38:16.131 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.140+0000 7f346a3868c0 1 < canceled part key=_multipart_YXMDZB23YTB3.2~2ZEJZ57LQD5E.1 size=1048576 2024-04-07T16:38:16.136 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.144+0000 7f346a3868c0 1 < completed part key=_multipart_YXMDZB23YTB3.2~2ZEJZ57LQD5E.2 size=1027138 2024-04-07T16:38:16.148 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.156+0000 7f346a3868c0 1 < canceled multipart upload key=YXMDZB23YTB3 upload=2ZEJZ57LQD5E tag=NN2NT0R81MAF type=write size=4172866 2024-04-07T16:38:16.160 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.168+0000 7f346a3868c0 1 > prepared part key=_multipart_3N0QJRYVO3FV.2~FAYS8GXC4UQF.0 size=1048576 2024-04-07T16:38:16.167 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.172+0000 7f346a3868c0 1 > prepared part key=_multipart_3N0QJRYVO3FV.2~FAYS8GXC4UQF.1 size=394951 2024-04-07T16:38:16.167 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.172+0000 7f346a3868c0 1 > prepared multipart upload key=3N0QJRYVO3FV upload=FAYS8GXC4UQF tag=5NOGD3UHG529 type=write size=2492103 2024-04-07T16:38:16.174 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.180+0000 7f346a3868c0 1 < completed delete operation key=XJLQZ7IS9U1T tag=NW0PO3BJ3OD7 type=del 2024-04-07T16:38:16.184 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.192+0000 7f346a3868c0 1 > prepared part key=_multipart_IX3F0IUOKBX1.2~B8GGBP9GH2I5.0 size=1048576 2024-04-07T16:38:16.189 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.196+0000 7f346a3868c0 1 > prepared part key=_multipart_IX3F0IUOKBX1.2~B8GGBP9GH2I5.1 size=868730 2024-04-07T16:38:16.189 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.196+0000 7f346a3868c0 1 > prepared multipart upload key=IX3F0IUOKBX1 upload=B8GGBP9GH2I5 tag=AUFUFBYA1DC6 type=write size=2965882 2024-04-07T16:38:16.195 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.204+0000 7f346a3868c0 1 < completed part key=_multipart_N14BUXLJONHZ.2~YMSR5T37QVU6.0 size=1048576 2024-04-07T16:38:16.200 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.208+0000 7f346a3868c0 1 < completed part key=_multipart_N14BUXLJONHZ.2~YMSR5T37QVU6.1 size=640316 2024-04-07T16:38:16.210 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.216+0000 7f346a3868c0 1 < completed multipart upload key=N14BUXLJONHZ upload=YMSR5T37QVU6 tag=3GW008X5NCB3 type=write size=2737468 2024-04-07T16:38:16.216 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.224+0000 7f346a3868c0 1 > prepared operation key=1VKVQXSQMTU1 tag=9E5Z33QPYC2A type=del size=249991 2024-04-07T16:38:16.221 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.228+0000 7f346a3868c0 1 < completed part key=_multipart_6XDS0A7L0IY8.2~D1VI5N1XLFVS.0 size=1048576 2024-04-07T16:38:16.227 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.232+0000 7f346a3868c0 1 < completed part key=_multipart_6XDS0A7L0IY8.2~D1VI5N1XLFVS.1 size=1048576 2024-04-07T16:38:16.232 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.240+0000 7f346a3868c0 1 < completed part key=_multipart_6XDS0A7L0IY8.2~D1VI5N1XLFVS.2 size=918944 2024-04-07T16:38:16.242 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.248+0000 7f346a3868c0 1 < completed multipart upload key=6XDS0A7L0IY8 upload=D1VI5N1XLFVS tag=QD92ICZCKIJL type=write size=4064672 2024-04-07T16:38:16.254 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.260+0000 7f346a3868c0 1 > prepared part key=_multipart_ROFH30W3UCAX.2~ZT0XYOR4FZP9.0 size=897411 2024-04-07T16:38:16.254 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.260+0000 7f346a3868c0 1 > prepared multipart upload key=ROFH30W3UCAX upload=ZT0XYOR4FZP9 tag=BDHO92S2DERP type=write size=1945987 2024-04-07T16:38:16.260 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.268+0000 7f346a3868c0 1 < completed part key=_multipart_7G5C6HBK6QWI.2~TV1H0N18YQ60.0 size=1048576 2024-04-07T16:38:16.265 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.272+0000 7f346a3868c0 1 < completed part key=_multipart_7G5C6HBK6QWI.2~TV1H0N18YQ60.1 size=1048576 2024-04-07T16:38:16.269 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.276+0000 7f346a3868c0 1 < completed part key=_multipart_7G5C6HBK6QWI.2~TV1H0N18YQ60.2 size=35644 2024-04-07T16:38:16.279 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.288+0000 7f346a3868c0 1 < completed multipart upload key=7G5C6HBK6QWI upload=TV1H0N18YQ60 tag=1U2TJQETE2XQ type=write size=3181372 2024-04-07T16:38:16.285 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.292+0000 7f346a3868c0 1 > prepared operation key=GBQSME7SY1AH tag=0AKBXEZRML7Q type=del size=3053501 2024-04-07T16:38:16.290 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.296+0000 7f346a3868c0 1 < completed write operation key=CL4PP1RKZU0E tag=9KZQD875JPFE type=write size=137613 2024-04-07T16:38:16.301 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.308+0000 7f346a3868c0 1 > prepared part key=_multipart_1VBSUWNI3V1T.2~KT2GJXEJ6A38.0 size=1048576 2024-04-07T16:38:16.306 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.312+0000 7f346a3868c0 1 > prepared part key=_multipart_1VBSUWNI3V1T.2~KT2GJXEJ6A38.1 size=1048576 2024-04-07T16:38:16.311 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.320+0000 7f346a3868c0 1 > prepared part key=_multipart_1VBSUWNI3V1T.2~KT2GJXEJ6A38.2 size=327599 2024-04-07T16:38:16.311 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.320+0000 7f346a3868c0 1 > prepared multipart upload key=1VBSUWNI3V1T upload=KT2GJXEJ6A38 tag=W1HJY01FF016 type=write size=3473327 2024-04-07T16:38:16.317 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.324+0000 7f346a3868c0 1 < completed delete operation key=E84U0SX44NQ8 tag=1J7VLJW86ESP type=del 2024-04-07T16:38:16.328 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.336+0000 7f346a3868c0 1 > prepared part key=_multipart_DXTO3AJ429AC.2~CLOW1TMWM7NW.0 size=719621 2024-04-07T16:38:16.328 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.336+0000 7f346a3868c0 1 > prepared multipart upload key=DXTO3AJ429AC upload=CLOW1TMWM7NW tag=4XOTMS4Y85PZ type=write size=1768197 2024-04-07T16:38:16.334 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.340+0000 7f346a3868c0 1 < completed delete operation key=X5Q46GO8YTOK tag=BG41MYGHAJ99 type=del 2024-04-07T16:38:16.344 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.352+0000 7f346a3868c0 1 > prepared part key=_multipart_8F81G0P0YBFW.2~7WAZE6DI8IUR.0 size=638760 2024-04-07T16:38:16.344 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.352+0000 7f346a3868c0 1 > prepared multipart upload key=8F81G0P0YBFW upload=7WAZE6DI8IUR tag=BQKS1KVD3I2Y type=write size=1687336 2024-04-07T16:38:16.356 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.364+0000 7f346a3868c0 1 < suggested update operation key=LXOZ0T2KY6OR tag=8B1DQ2SQ2BOS type=write size=255096 2024-04-07T16:38:16.367 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.372+0000 7f346a3868c0 1 > prepared part key=_multipart_EQR0FY86AT5X.2~48IQ6VL82Y44.0 size=1048576 2024-04-07T16:38:16.373 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.380+0000 7f346a3868c0 1 > prepared part key=_multipart_EQR0FY86AT5X.2~48IQ6VL82Y44.1 size=600746 2024-04-07T16:38:16.374 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.380+0000 7f346a3868c0 1 > prepared multipart upload key=EQR0FY86AT5X upload=48IQ6VL82Y44 tag=0N8BXQD9OQ7I type=write size=2697898 2024-04-07T16:38:16.377 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.384+0000 7f346a3868c0 1 < completed write operation key=C1BTVJNHIM0X tag=ODXHVF3BT87S type=write size=741613 2024-04-07T16:38:16.389 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.396+0000 7f346a3868c0 1 > prepared part key=_multipart_LXOZ0T2KY6OR.2~Q1DZLNF4M2VZ.0 size=1048576 2024-04-07T16:38:16.394 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.400+0000 7f346a3868c0 1 > prepared part key=_multipart_LXOZ0T2KY6OR.2~Q1DZLNF4M2VZ.1 size=261061 2024-04-07T16:38:16.394 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.400+0000 7f346a3868c0 1 > prepared multipart upload key=LXOZ0T2KY6OR upload=Q1DZLNF4M2VZ tag=HOOJ58652AEP type=write size=2358213 2024-04-07T16:38:16.406 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.412+0000 7f346a3868c0 1 < suggested update operation key=NYET3212USVE tag=8YBXGUR4PO3F type=write size=541788 2024-04-07T16:38:16.412 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.420+0000 7f346a3868c0 1 > prepared operation key=DXTO3AJ429AC tag=G03AT02VMX12 type=del size=2261910 2024-04-07T16:38:16.417 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.424+0000 7f346a3868c0 1 < completed part key=_multipart_YXMDZB23YTB3.2~WWFZW9X65SQ5.0 size=1048576 2024-04-07T16:38:16.423 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.428+0000 7f346a3868c0 1 < completed part key=_multipart_YXMDZB23YTB3.2~WWFZW9X65SQ5.1 size=1048576 2024-04-07T16:38:16.428 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.436+0000 7f346a3868c0 1 < completed part key=_multipart_YXMDZB23YTB3.2~WWFZW9X65SQ5.2 size=434391 2024-04-07T16:38:16.439 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.444+0000 7f346a3868c0 1 < completed multipart upload key=YXMDZB23YTB3 upload=WWFZW9X65SQ5 tag=K8JRHPPX1UHO type=write size=3580119 2024-04-07T16:38:16.441 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:38:16 smithi100 bash[74942]: cluster 2024-04-07T16:38:15.073186+0000 mgr.y (mgr.25175) 687 : cluster [DBG] pgmap v1322: 137 pgs: 137 active+clean; 583 KiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 336 KiB/s rd, 117 KiB/s wr, 593 op/s 2024-04-07T16:38:16.441 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:38:16 smithi100 bash[76202]: cluster 2024-04-07T16:38:15.073186+0000 mgr.y (mgr.25175) 687 : cluster [DBG] pgmap v1322: 137 pgs: 137 active+clean; 583 KiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 336 KiB/s rd, 117 KiB/s wr, 593 op/s 2024-04-07T16:38:16.444 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.452+0000 7f346a3868c0 1 > prepared operation key=EQR0FY86AT5X tag=CGYPN94D1W0H type=del size=1730407 2024-04-07T16:38:16.451 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.456+0000 7f346a3868c0 1 < completed part key=_multipart_1VKVQXSQMTU1.2~2W2D9OUE7ZZQ.0 size=1048576 2024-04-07T16:38:16.455 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.464+0000 7f346a3868c0 1 < completed part key=_multipart_1VKVQXSQMTU1.2~2W2D9OUE7ZZQ.1 size=789028 2024-04-07T16:38:16.466 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.472+0000 7f346a3868c0 1 < completed multipart upload key=1VKVQXSQMTU1 upload=2W2D9OUE7ZZQ tag=NAQM2MN2QBF6 type=write size=2886180 2024-04-07T16:38:16.472 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.480+0000 7f346a3868c0 1 > prepared operation key=LXOZ0T2KY6OR tag=0AD8BKCGU5WS type=del size=1138308 2024-04-07T16:38:16.478 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.484+0000 7f346a3868c0 1 < completed delete operation key=GZXHYZO5SMUP tag=UBQ84Y139EA9 type=del 2024-04-07T16:38:16.484 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.492+0000 7f346a3868c0 1 > prepared operation key=3N0QJRYVO3FV tag=C644A3IPJNY6 type=del size=464891 2024-04-07T16:38:16.497 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.504+0000 7f346a3868c0 1 < suggested remove operation key=1VBSUWNI3V1T tag=VL80TMRHGK4M type=del 2024-04-07T16:38:16.503 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.508+0000 7f346a3868c0 1 > prepared operation key=NYET3212USVE tag=MF953I5BV91D type=del size=621070 2024-04-07T16:38:16.510 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.516+0000 7f346a3868c0 1 < completed part key=_multipart_E84U0SX44NQ8.2~B8UUNE4IJ7RM.0 size=720310 2024-04-07T16:38:16.521 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.528+0000 7f346a3868c0 1 < completed multipart upload key=E84U0SX44NQ8 upload=B8UUNE4IJ7RM tag=M2AVYW21O5E0 type=write size=1768886 2024-04-07T16:38:16.526 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.532+0000 7f346a3868c0 1 > prepared operation key=HBLEXONKXJTL tag=RIEYBWE3D6IZ type=del size=2792059 2024-04-07T16:38:16.532 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.540+0000 7f346a3868c0 1 < completed part key=_multipart_3N0QJRYVO3FV.2~FAYS8GXC4UQF.0 size=1048576 2024-04-07T16:38:16.537 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.544+0000 7f346a3868c0 1 < completed part key=_multipart_3N0QJRYVO3FV.2~FAYS8GXC4UQF.1 size=394951 2024-04-07T16:38:16.547 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.556+0000 7f346a3868c0 1 < completed multipart upload key=3N0QJRYVO3FV upload=FAYS8GXC4UQF tag=5NOGD3UHG529 type=write size=2492103 2024-04-07T16:38:16.558 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.564+0000 7f346a3868c0 1 > prepared part key=_multipart_IP8YW2A75TCW.2~TB14ZDKQEA6J.0 size=1048576 2024-04-07T16:38:16.563 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.568+0000 7f346a3868c0 1 > prepared part key=_multipart_IP8YW2A75TCW.2~TB14ZDKQEA6J.1 size=1048576 2024-04-07T16:38:16.568 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.576+0000 7f346a3868c0 1 > prepared part key=_multipart_IP8YW2A75TCW.2~TB14ZDKQEA6J.2 size=929372 2024-04-07T16:38:16.568 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.576+0000 7f346a3868c0 1 > prepared multipart upload key=IP8YW2A75TCW upload=TB14ZDKQEA6J tag=0V80BT6GOOLK type=write size=4075100 2024-04-07T16:38:16.573 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.580+0000 7f346a3868c0 1 < completed part key=_multipart_IX3F0IUOKBX1.2~B8GGBP9GH2I5.0 size=1048576 2024-04-07T16:38:16.578 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.584+0000 7f346a3868c0 1 < completed part key=_multipart_IX3F0IUOKBX1.2~B8GGBP9GH2I5.1 size=868730 2024-04-07T16:38:16.588 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.596+0000 7f346a3868c0 1 < completed multipart upload key=IX3F0IUOKBX1 upload=B8GGBP9GH2I5 tag=AUFUFBYA1DC6 type=write size=2965882 2024-04-07T16:38:16.593 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.600+0000 7f346a3868c0 1 > prepared operation key=LXOZ0T2KY6OR tag=O5QWRLTNSLP3 type=del size=1689299 2024-04-07T16:38:16.599 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.608+0000 7f346a3868c0 1 < completed delete operation key=1VKVQXSQMTU1 tag=9E5Z33QPYC2A type=del 2024-04-07T16:38:16.605 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.612+0000 7f346a3868c0 1 > prepared operation key=YXMDZB23YTB3 tag=Y0DXECBT78Z0 type=write size=511232 2024-04-07T16:38:16.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:38:16 smithi144 bash[66654]: cluster 2024-04-07T16:38:15.073186+0000 mgr.y (mgr.25175) 687 : cluster [DBG] pgmap v1322: 137 pgs: 137 active+clean; 583 KiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 336 KiB/s rd, 117 KiB/s wr, 593 op/s 2024-04-07T16:38:16.611 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.616+0000 7f346a3868c0 1 < completed part key=_multipart_ROFH30W3UCAX.2~ZT0XYOR4FZP9.0 size=897411 2024-04-07T16:38:16.622 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.628+0000 7f346a3868c0 1 < completed multipart upload key=ROFH30W3UCAX upload=ZT0XYOR4FZP9 tag=BDHO92S2DERP type=write size=1945987 2024-04-07T16:38:16.628 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.636+0000 7f346a3868c0 1 > prepared operation key=E84U0SX44NQ8 tag=53ELAUOPEY4E type=del size=3866288 2024-04-07T16:38:16.634 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.640+0000 7f346a3868c0 1 < completed delete operation key=GBQSME7SY1AH tag=0AKBXEZRML7Q type=del 2024-04-07T16:38:16.648 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.656+0000 7f346a3868c0 1 > prepared part key=_multipart_HBLEXONKXJTL.2~VJ2V71AADKW0.0 size=1048576 2024-04-07T16:38:16.653 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.660+0000 7f346a3868c0 1 > prepared part key=_multipart_HBLEXONKXJTL.2~VJ2V71AADKW0.1 size=1048576 2024-04-07T16:38:16.657 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.664+0000 7f346a3868c0 1 > prepared part key=_multipart_HBLEXONKXJTL.2~VJ2V71AADKW0.2 size=504186 2024-04-07T16:38:16.657 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.664+0000 7f346a3868c0 1 > prepared multipart upload key=HBLEXONKXJTL upload=VJ2V71AADKW0 tag=ZQ1ASJH0YCQ5 type=write size=3649914 2024-04-07T16:38:16.658 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.664+0000 7f346a3868c0 1 < canceled part key=_multipart_1VBSUWNI3V1T.2~KT2GJXEJ6A38.0 size=1048576 2024-04-07T16:38:16.663 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.672+0000 7f346a3868c0 1 < completed part key=_multipart_1VBSUWNI3V1T.2~KT2GJXEJ6A38.1 size=1048576 2024-04-07T16:38:16.663 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.672+0000 7f346a3868c0 1 < canceled part key=_multipart_1VBSUWNI3V1T.2~KT2GJXEJ6A38.2 size=327599 2024-04-07T16:38:16.673 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.680+0000 7f346a3868c0 1 < canceled multipart upload key=1VBSUWNI3V1T upload=KT2GJXEJ6A38 tag=W1HJY01FF016 type=write size=3473327 2024-04-07T16:38:16.678 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.684+0000 7f346a3868c0 1 > prepared operation key=QTCIIPTXIRW2 tag=QYEBH2XX126X type=write size=833412 2024-04-07T16:38:16.679 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.688+0000 7f346a3868c0 1 < canceled part key=_multipart_DXTO3AJ429AC.2~CLOW1TMWM7NW.0 size=719621 2024-04-07T16:38:16.690 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.696+0000 7f346a3868c0 1 < canceled multipart upload key=DXTO3AJ429AC upload=CLOW1TMWM7NW tag=4XOTMS4Y85PZ type=write size=1768197 2024-04-07T16:38:16.697 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.704+0000 7f346a3868c0 1 > prepared operation key=QS4HYZFFAJNF tag=9L0X923YEX9Z type=write size=227796 2024-04-07T16:38:16.698 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.704+0000 7f346a3868c0 1 < canceled part key=_multipart_8F81G0P0YBFW.2~7WAZE6DI8IUR.0 size=638760 2024-04-07T16:38:16.708 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.716+0000 7f346a3868c0 1 < canceled multipart upload key=8F81G0P0YBFW upload=7WAZE6DI8IUR tag=BQKS1KVD3I2Y type=write size=1687336 2024-04-07T16:38:16.714 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.720+0000 7f346a3868c0 1 > prepared operation key=UIW3DN4NUYPP tag=0J20PFI3VNVQ type=del size=1512800 2024-04-07T16:38:16.715 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.724+0000 7f346a3868c0 1 < canceled part key=_multipart_EQR0FY86AT5X.2~48IQ6VL82Y44.0 size=1048576 2024-04-07T16:38:16.715 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.724+0000 7f346a3868c0 1 < canceled part key=_multipart_EQR0FY86AT5X.2~48IQ6VL82Y44.1 size=600746 2024-04-07T16:38:16.727 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.732+0000 7f346a3868c0 1 < canceled multipart upload key=EQR0FY86AT5X upload=48IQ6VL82Y44 tag=0N8BXQD9OQ7I type=write size=2697898 2024-04-07T16:38:16.733 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.740+0000 7f346a3868c0 1 > prepared operation key=95YVKJ14JD8W tag=UO0S6SYDZNW4 type=del size=4189137 2024-04-07T16:38:16.740 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.748+0000 7f346a3868c0 1 < completed part key=_multipart_LXOZ0T2KY6OR.2~Q1DZLNF4M2VZ.0 size=1048576 2024-04-07T16:38:16.746 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.752+0000 7f346a3868c0 1 < completed part key=_multipart_LXOZ0T2KY6OR.2~Q1DZLNF4M2VZ.1 size=261061 2024-04-07T16:38:16.758 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.764+0000 7f346a3868c0 1 < completed multipart upload key=LXOZ0T2KY6OR upload=Q1DZLNF4M2VZ tag=HOOJ58652AEP type=write size=2358213 2024-04-07T16:38:16.770 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.776+0000 7f346a3868c0 1 > prepared part key=_multipart_LXOZ0T2KY6OR.2~CHIVQ3OXAN35.0 size=1048576 2024-04-07T16:38:16.775 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.780+0000 7f346a3868c0 1 > prepared part key=_multipart_LXOZ0T2KY6OR.2~CHIVQ3OXAN35.1 size=1048576 2024-04-07T16:38:16.780 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.788+0000 7f346a3868c0 1 > prepared part key=_multipart_LXOZ0T2KY6OR.2~CHIVQ3OXAN35.2 size=165572 2024-04-07T16:38:16.780 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.788+0000 7f346a3868c0 1 > prepared multipart upload key=LXOZ0T2KY6OR upload=CHIVQ3OXAN35 tag=2TIFXDDANAEC type=write size=3311300 2024-04-07T16:38:16.786 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.792+0000 7f346a3868c0 1 < completed delete operation key=DXTO3AJ429AC tag=G03AT02VMX12 type=del 2024-04-07T16:38:16.792 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.800+0000 7f346a3868c0 1 > prepared operation key=NYET3212USVE tag=F69QN8N8Z9MX type=del size=1204820 2024-04-07T16:38:16.799 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.808+0000 7f346a3868c0 1 < completed delete operation key=EQR0FY86AT5X tag=CGYPN94D1W0H type=del 2024-04-07T16:38:16.805 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.812+0000 7f346a3868c0 1 > prepared operation key=1VBSUWNI3V1T tag=2FKDSPN60YYO type=del size=3174936 2024-04-07T16:38:16.811 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.820+0000 7f346a3868c0 1 < completed delete operation key=LXOZ0T2KY6OR tag=0AD8BKCGU5WS type=del 2024-04-07T16:38:16.818 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.824+0000 7f346a3868c0 1 > prepared operation key=7G5C6HBK6QWI tag=YD5Y9S8EE9EE type=write size=432544 2024-04-07T16:38:16.825 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.832+0000 7f346a3868c0 1 < completed delete operation key=3N0QJRYVO3FV tag=C644A3IPJNY6 type=del 2024-04-07T16:38:16.837 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.844+0000 7f346a3868c0 1 > prepared part key=_multipart_QS4HYZFFAJNF.2~SEG78Y3FF9C4.0 size=1048576 2024-04-07T16:38:16.842 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.848+0000 7f346a3868c0 1 > prepared part key=_multipart_QS4HYZFFAJNF.2~SEG78Y3FF9C4.1 size=1048576 2024-04-07T16:38:16.847 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.856+0000 7f346a3868c0 1 > prepared part key=_multipart_QS4HYZFFAJNF.2~SEG78Y3FF9C4.2 size=315060 2024-04-07T16:38:16.847 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.856+0000 7f346a3868c0 1 > prepared multipart upload key=QS4HYZFFAJNF upload=SEG78Y3FF9C4 tag=24G5JYB17QAU type=write size=3460788 2024-04-07T16:38:16.853 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.860+0000 7f346a3868c0 1 < completed delete operation key=NYET3212USVE tag=MF953I5BV91D type=del 2024-04-07T16:38:16.862 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.868+0000 7f346a3868c0 1 > prepared operation key=6XDS0A7L0IY8 tag=BSTJOJN1RY16 type=del size=319101 2024-04-07T16:38:16.875 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.884+0000 7f346a3868c0 1 < suggested update operation key=HBLEXONKXJTL tag=RIEYBWE3D6IZ type=del size=2792059 2024-04-07T16:38:16.882 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.888+0000 7f346a3868c0 1 > prepared operation key=E84U0SX44NQ8 tag=O4RCSV6NMBDC type=del size=1999386 2024-04-07T16:38:16.883 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.888+0000 7f346a3868c0 1 < canceled part key=_multipart_IP8YW2A75TCW.2~TB14ZDKQEA6J.0 size=1048576 2024-04-07T16:38:16.887 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.896+0000 7f346a3868c0 1 < completed part key=_multipart_IP8YW2A75TCW.2~TB14ZDKQEA6J.1 size=1048576 2024-04-07T16:38:16.892 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.900+0000 7f346a3868c0 1 < completed part key=_multipart_IP8YW2A75TCW.2~TB14ZDKQEA6J.2 size=929372 2024-04-07T16:38:16.902 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.908+0000 7f346a3868c0 1 < canceled multipart upload key=IP8YW2A75TCW upload=TB14ZDKQEA6J tag=0V80BT6GOOLK type=write size=4075100 2024-04-07T16:38:16.908 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.916+0000 7f346a3868c0 1 > prepared operation key=1VKVQXSQMTU1 tag=H4SNV5Y8X4X3 type=write size=82957 2024-04-07T16:38:16.915 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.920+0000 7f346a3868c0 1 < completed delete operation key=LXOZ0T2KY6OR tag=O5QWRLTNSLP3 type=del 2024-04-07T16:38:16.920 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.928+0000 7f346a3868c0 1 > prepared operation key=1VKVQXSQMTU1 tag=29W8MHS81TYS type=write size=774978 2024-04-07T16:38:16.926 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.932+0000 7f346a3868c0 1 < completed write operation key=YXMDZB23YTB3 tag=Y0DXECBT78Z0 type=write size=511232 2024-04-07T16:38:16.938 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.944+0000 7f346a3868c0 1 > prepared part key=_multipart_ROFH30W3UCAX.2~4QRNJW4S2K4H.0 size=796753 2024-04-07T16:38:16.938 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.944+0000 7f346a3868c0 1 > prepared multipart upload key=ROFH30W3UCAX upload=4QRNJW4S2K4H tag=W8BHB89645IM type=write size=1845329 2024-04-07T16:38:16.944 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.952+0000 7f346a3868c0 1 < completed delete operation key=E84U0SX44NQ8 tag=53ELAUOPEY4E type=del 2024-04-07T16:38:16.950 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.956+0000 7f346a3868c0 1 > prepared operation key=C1BTVJNHIM0X tag=6UPAQ8714SDI type=del size=3230180 2024-04-07T16:38:16.957 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.964+0000 7f346a3868c0 1 < completed part key=_multipart_HBLEXONKXJTL.2~VJ2V71AADKW0.0 size=1048576 2024-04-07T16:38:16.963 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.968+0000 7f346a3868c0 1 < completed part key=_multipart_HBLEXONKXJTL.2~VJ2V71AADKW0.1 size=1048576 2024-04-07T16:38:16.968 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.976+0000 7f346a3868c0 1 < completed part key=_multipart_HBLEXONKXJTL.2~VJ2V71AADKW0.2 size=504186 2024-04-07T16:38:16.978 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.984+0000 7f346a3868c0 1 < completed multipart upload key=HBLEXONKXJTL upload=VJ2V71AADKW0 tag=ZQ1ASJH0YCQ5 type=write size=3649914 2024-04-07T16:38:16.989 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.996+0000 7f346a3868c0 1 > prepared part key=_multipart_X5Q46GO8YTOK.2~XWM8VI2KKY9G.0 size=541175 2024-04-07T16:38:16.989 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:16.996+0000 7f346a3868c0 1 > prepared multipart upload key=X5Q46GO8YTOK upload=XWM8VI2KKY9G tag=1OA6785EBCSZ type=write size=1589751 2024-04-07T16:38:16.995 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:17.004+0000 7f346a3868c0 1 < completed write operation key=QTCIIPTXIRW2 tag=QYEBH2XX126X type=write size=833412 2024-04-07T16:38:17.001 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:17.008+0000 7f346a3868c0 1 > prepared operation key=EQR0FY86AT5X tag=1NUYG6LMCGI8 type=del size=3147867 2024-04-07T16:38:17.013 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:17.020+0000 7f346a3868c0 1 < suggested update operation key=QS4HYZFFAJNF tag=9L0X923YEX9Z type=write size=227796 2024-04-07T16:38:17.024 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:17.032+0000 7f346a3868c0 1 > prepared part key=_multipart_EQR0FY86AT5X.2~Y9ZLPQK31WMP.0 size=1048576 2024-04-07T16:38:17.029 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:17.036+0000 7f346a3868c0 1 > prepared part key=_multipart_EQR0FY86AT5X.2~Y9ZLPQK31WMP.1 size=1048576 2024-04-07T16:38:17.035 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:17.040+0000 7f346a3868c0 1 > prepared part key=_multipart_EQR0FY86AT5X.2~Y9ZLPQK31WMP.2 size=218049 2024-04-07T16:38:17.035 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:17.040+0000 7f346a3868c0 1 > prepared multipart upload key=EQR0FY86AT5X upload=Y9ZLPQK31WMP tag=ZFKF0T3J7J66 type=write size=3363777 2024-04-07T16:38:17.041 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:17.048+0000 7f346a3868c0 1 < completed delete operation key=UIW3DN4NUYPP tag=0J20PFI3VNVQ type=del 2024-04-07T16:38:17.053 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:17.060+0000 7f346a3868c0 1 > prepared part key=_multipart_1VBSUWNI3V1T.2~ZIBD6PW90XOE.0 size=1048576 2024-04-07T16:38:17.058 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:17.064+0000 7f346a3868c0 1 > prepared part key=_multipart_1VBSUWNI3V1T.2~ZIBD6PW90XOE.1 size=1048576 2024-04-07T16:38:17.064 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:17.072+0000 7f346a3868c0 1 > prepared part key=_multipart_1VBSUWNI3V1T.2~ZIBD6PW90XOE.2 size=40181 2024-04-07T16:38:17.064 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:17.072+0000 7f346a3868c0 1 > prepared multipart upload key=1VBSUWNI3V1T upload=ZIBD6PW90XOE tag=SOG4TZA3P8HQ type=write size=3185909 2024-04-07T16:38:17.069 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:17.076+0000 7f346a3868c0 1 < completed delete operation key=95YVKJ14JD8W tag=UO0S6SYDZNW4 type=del 2024-04-07T16:38:17.075 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:17.084+0000 7f346a3868c0 1 > prepared operation key=U3EID5EK64HD tag=XETRMPYZDZAI type=del size=131994 2024-04-07T16:38:17.081 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:17.088+0000 7f346a3868c0 1 < completed part key=_multipart_LXOZ0T2KY6OR.2~CHIVQ3OXAN35.0 size=1048576 2024-04-07T16:38:17.086 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:17.092+0000 7f346a3868c0 1 < completed part key=_multipart_LXOZ0T2KY6OR.2~CHIVQ3OXAN35.1 size=1048576 2024-04-07T16:38:17.091 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:17.096+0000 7f346a3868c0 1 < completed part key=_multipart_LXOZ0T2KY6OR.2~CHIVQ3OXAN35.2 size=165572 2024-04-07T16:38:17.100 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:17.108+0000 7f346a3868c0 1 < completed multipart upload key=LXOZ0T2KY6OR upload=CHIVQ3OXAN35 tag=2TIFXDDANAEC type=write size=3311300 2024-04-07T16:38:17.106 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:17.112+0000 7f346a3868c0 1 > prepared operation key=IX3F0IUOKBX1 tag=6WR0PMVCVO7R type=del size=2730301 2024-04-07T16:38:17.117 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:17.124+0000 7f346a3868c0 1 < suggested remove operation key=NYET3212USVE tag=F69QN8N8Z9MX type=del 2024-04-07T16:38:17.123 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:17.128+0000 7f346a3868c0 1 > prepared operation key=GZXHYZO5SMUP tag=ONP00L160P6R type=del size=857737 2024-04-07T16:38:17.129 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:17.136+0000 7f346a3868c0 1 < completed delete operation key=1VBSUWNI3V1T tag=2FKDSPN60YYO type=del 2024-04-07T16:38:17.134 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:17.140+0000 7f346a3868c0 1 > prepared operation key=8F81G0P0YBFW tag=2KV14YHHJG5G type=del size=1294628 2024-04-07T16:38:17.140 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:17.148+0000 7f346a3868c0 1 < completed write operation key=7G5C6HBK6QWI tag=YD5Y9S8EE9EE type=write size=432544 2024-04-07T16:38:17.149 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:17.156+0000 7f346a3868c0 1 > prepared operation key=YXMDZB23YTB3 tag=0ND11815HKAK type=write size=764632 2024-04-07T16:38:17.156 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:17.164+0000 7f346a3868c0 1 < completed part key=_multipart_QS4HYZFFAJNF.2~SEG78Y3FF9C4.0 size=1048576 2024-04-07T16:38:17.162 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:17.168+0000 7f346a3868c0 1 < completed part key=_multipart_QS4HYZFFAJNF.2~SEG78Y3FF9C4.1 size=1048576 2024-04-07T16:38:17.167 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:17.176+0000 7f346a3868c0 1 < completed part key=_multipart_QS4HYZFFAJNF.2~SEG78Y3FF9C4.2 size=315060 2024-04-07T16:38:17.177 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:17.184+0000 7f346a3868c0 1 < completed multipart upload key=QS4HYZFFAJNF upload=SEG78Y3FF9C4 tag=24G5JYB17QAU type=write size=3460788 2024-04-07T16:38:17.189 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:17.196+0000 7f346a3868c0 1 > prepared part key=_multipart_EQR0FY86AT5X.2~E8C3ZGCB0W3J.0 size=164177 2024-04-07T16:38:17.189 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:17.196+0000 7f346a3868c0 1 > prepared multipart upload key=EQR0FY86AT5X upload=E8C3ZGCB0W3J tag=DP940RFK8JZX type=write size=1212753 2024-04-07T16:38:17.197 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:17.204+0000 7f346a3868c0 1 < suggested update operation key=6XDS0A7L0IY8 tag=BSTJOJN1RY16 type=del size=319101 2024-04-07T16:38:17.203 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:17.212+0000 7f346a3868c0 1 > prepared operation key=IP8YW2A75TCW tag=RJPWROVMBHHN type=del size=58352 2024-04-07T16:38:17.209 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:17.216+0000 7f346a3868c0 1 < completed delete operation key=E84U0SX44NQ8 tag=O4RCSV6NMBDC type=del 2024-04-07T16:38:17.214 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:17.220+0000 7f346a3868c0 1 > prepared operation key=GBQSME7SY1AH tag=QZN24ZGREYH3 type=del size=1426786 2024-04-07T16:38:17.225 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:17.232+0000 7f346a3868c0 1 < suggested update operation key=1VKVQXSQMTU1 tag=H4SNV5Y8X4X3 type=write size=82957 2024-04-07T16:38:17.231 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:17.240+0000 7f346a3868c0 1 > prepared operation key=C1BTVJNHIM0X tag=C9UC2AJZ7IF8 type=del size=652404 2024-04-07T16:38:17.245 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:17.252+0000 7f346a3868c0 1 < suggested update operation key=1VKVQXSQMTU1 tag=29W8MHS81TYS type=write size=774978 2024-04-07T16:38:17.256 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:17.264+0000 7f346a3868c0 1 > prepared part key=_multipart_EQR0FY86AT5X.2~6HX5BB8R1R17.0 size=1048576 2024-04-07T16:38:17.262 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:17.268+0000 7f346a3868c0 1 > prepared part key=_multipart_EQR0FY86AT5X.2~6HX5BB8R1R17.1 size=510446 2024-04-07T16:38:17.262 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:17.268+0000 7f346a3868c0 1 > prepared multipart upload key=EQR0FY86AT5X upload=6HX5BB8R1R17 tag=Y3QCM9EILSH2 type=write size=2607598 2024-04-07T16:38:17.269 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:17.276+0000 7f346a3868c0 1 < completed part key=_multipart_ROFH30W3UCAX.2~4QRNJW4S2K4H.0 size=796753 2024-04-07T16:38:17.279 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:17.288+0000 7f346a3868c0 1 < completed multipart upload key=ROFH30W3UCAX upload=4QRNJW4S2K4H tag=W8BHB89645IM type=write size=1845329 2024-04-07T16:38:17.285 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:17.292+0000 7f346a3868c0 1 > prepared operation key=6XDS0A7L0IY8 tag=1OR7G45TXC7X type=del size=3429736 2024-04-07T16:38:17.291 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:17.300+0000 7f346a3868c0 1 < completed delete operation key=C1BTVJNHIM0X tag=6UPAQ8714SDI type=del 2024-04-07T16:38:17.297 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:17.304+0000 7f346a3868c0 1 > prepared operation key=E84U0SX44NQ8 tag=RRKDYA01HLNV type=del size=3042964 2024-04-07T16:38:17.302 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:17.308+0000 7f346a3868c0 1 < completed part key=_multipart_X5Q46GO8YTOK.2~XWM8VI2KKY9G.0 size=541175 2024-04-07T16:38:17.312 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:17.320+0000 7f346a3868c0 1 < completed multipart upload key=X5Q46GO8YTOK upload=XWM8VI2KKY9G tag=1OA6785EBCSZ type=write size=1589751 2024-04-07T16:38:17.318 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:17.324+0000 7f346a3868c0 1 > prepared operation key=95YVKJ14JD8W tag=XFH27SOY56FZ type=del size=1056950 2024-04-07T16:38:17.323 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:17.332+0000 7f346a3868c0 1 < canceled operation key=EQR0FY86AT5X tag=1NUYG6LMCGI8 type=del size=3147867 2024-04-07T16:38:17.334 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:17.340+0000 7f346a3868c0 1 > prepared part key=_multipart_EQR0FY86AT5X.2~RJBAUIKF8WN5.0 size=656755 2024-04-07T16:38:17.334 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:17.340+0000 7f346a3868c0 1 > prepared multipart upload key=EQR0FY86AT5X upload=RJBAUIKF8WN5 tag=EV51ZX4BZ289 type=write size=1705331 2024-04-07T16:38:17.335 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:17.344+0000 7f346a3868c0 1 < canceled part key=_multipart_EQR0FY86AT5X.2~Y9ZLPQK31WMP.0 size=1048576 2024-04-07T16:38:17.341 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:17.348+0000 7f346a3868c0 1 < completed part key=_multipart_EQR0FY86AT5X.2~Y9ZLPQK31WMP.1 size=1048576 2024-04-07T16:38:17.345 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:17.352+0000 7f346a3868c0 1 < completed part key=_multipart_EQR0FY86AT5X.2~Y9ZLPQK31WMP.2 size=218049 2024-04-07T16:38:17.355 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:17.364+0000 7f346a3868c0 1 < canceled multipart upload key=EQR0FY86AT5X upload=Y9ZLPQK31WMP tag=ZFKF0T3J7J66 type=write size=3363777 2024-04-07T16:38:17.362 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:17.368+0000 7f346a3868c0 1 > prepared operation key=YOK8BAEOMPUR tag=2TN1WQMXZ1Z0 type=write size=28795 2024-04-07T16:38:17.367 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:17.376+0000 7f346a3868c0 1 < completed part key=_multipart_1VBSUWNI3V1T.2~ZIBD6PW90XOE.0 size=1048576 2024-04-07T16:38:17.372 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:17.380+0000 7f346a3868c0 1 < completed part key=_multipart_1VBSUWNI3V1T.2~ZIBD6PW90XOE.1 size=1048576 2024-04-07T16:38:17.377 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:17.384+0000 7f346a3868c0 1 < completed part key=_multipart_1VBSUWNI3V1T.2~ZIBD6PW90XOE.2 size=40181 2024-04-07T16:38:17.389 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:17.396+0000 7f346a3868c0 1 < completed multipart upload key=1VBSUWNI3V1T upload=ZIBD6PW90XOE tag=SOG4TZA3P8HQ type=write size=3185909 2024-04-07T16:38:17.401 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:17.408+0000 7f346a3868c0 1 > prepared part key=_multipart_N14BUXLJONHZ.2~QOG310NTDGL7.0 size=955674 2024-04-07T16:38:17.401 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:17.408+0000 7f346a3868c0 1 > prepared multipart upload key=N14BUXLJONHZ upload=QOG310NTDGL7 tag=R1WBVZ0CWQM0 type=write size=2004250 2024-04-07T16:38:17.406 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:17.412+0000 7f346a3868c0 1 < completed delete operation key=U3EID5EK64HD tag=XETRMPYZDZAI type=del 2024-04-07T16:38:17.412 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:17.420+0000 7f346a3868c0 1 > prepared operation key=U3EID5EK64HD tag=PNJ926TNBSD5 type=del size=538423 2024-04-07T16:38:17.418 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:17.424+0000 7f346a3868c0 1 < completed delete operation key=IX3F0IUOKBX1 tag=6WR0PMVCVO7R type=del 2024-04-07T16:38:17.423 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:17.432+0000 7f346a3868c0 1 > prepared operation key=GZXHYZO5SMUP tag=ME5T75O3BI8X type=del size=1698780 2024-04-07T16:38:17.429 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:17.436+0000 7f346a3868c0 1 < completed delete operation key=GZXHYZO5SMUP tag=ONP00L160P6R type=del 2024-04-07T16:38:17.435 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:17.440+0000 7f346a3868c0 1 > prepared operation key=EOYFVEI50SQ1 tag=KVNYE0XJIM9Q type=del size=1275888 2024-04-07T16:38:17.440 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:17.448+0000 7f346a3868c0 1 < completed delete operation key=8F81G0P0YBFW tag=2KV14YHHJG5G type=del 2024-04-07T16:38:17.441 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:38:17 smithi100 bash[74942]: audit 2024-04-07T16:38:16.753828+0000 mon.a (mon.0) 2023 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:38:17.441 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:38:17 smithi100 bash[76202]: audit 2024-04-07T16:38:16.753828+0000 mon.a (mon.0) 2023 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:38:17.446 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:17.452+0000 7f346a3868c0 1 > prepared operation key=C1BTVJNHIM0X tag=08WVEBQ5P05Q type=del size=2431311 2024-04-07T16:38:17.452 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:17.460+0000 7f346a3868c0 1 < completed write operation key=YXMDZB23YTB3 tag=0ND11815HKAK type=write size=764632 2024-04-07T16:38:17.458 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:17.464+0000 7f346a3868c0 1 > prepared operation key=N14BUXLJONHZ tag=SJDRQAYVMZ2P type=del size=2966475 2024-04-07T16:38:17.463 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:17.472+0000 7f346a3868c0 1 < completed part key=_multipart_EQR0FY86AT5X.2~E8C3ZGCB0W3J.0 size=164177 2024-04-07T16:38:17.473 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:17.480+0000 7f346a3868c0 1 < completed multipart upload key=EQR0FY86AT5X upload=E8C3ZGCB0W3J tag=DP940RFK8JZX type=write size=1212753 2024-04-07T16:38:17.479 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:17.488+0000 7f346a3868c0 1 > prepared operation key=GZXHYZO5SMUP tag=4Q8B99OU0HAS type=del size=3284875 2024-04-07T16:38:17.493 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:17.500+0000 7f346a3868c0 1 < suggested update operation key=IP8YW2A75TCW tag=RJPWROVMBHHN type=del size=58352 2024-04-07T16:38:17.498 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:17.504+0000 7f346a3868c0 1 > prepared operation key=QVOKSQAGHPA9 tag=MKDS578WZLFK type=del size=3660054 2024-04-07T16:38:17.504 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:17.512+0000 7f346a3868c0 1 < completed delete operation key=GBQSME7SY1AH tag=QZN24ZGREYH3 type=del 2024-04-07T16:38:17.515 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:17.520+0000 7f346a3868c0 1 > prepared part key=_multipart_GZXHYZO5SMUP.2~7DLNNBRFDKCW.0 size=914788 2024-04-07T16:38:17.515 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:17.520+0000 7f346a3868c0 1 > prepared multipart upload key=GZXHYZO5SMUP upload=7DLNNBRFDKCW tag=Z6PY2Q4G4FNA type=write size=1963364 2024-04-07T16:38:17.520 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:17.528+0000 7f346a3868c0 1 < canceled operation key=C1BTVJNHIM0X tag=C9UC2AJZ7IF8 type=del size=652404 2024-04-07T16:38:17.531 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:17.540+0000 7f346a3868c0 1 > prepared part key=_multipart_6XDS0A7L0IY8.2~GJ1270ULPUNA.0 size=1048576 2024-04-07T16:38:17.536 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:17.544+0000 7f346a3868c0 1 > prepared part key=_multipart_6XDS0A7L0IY8.2~GJ1270ULPUNA.1 size=1027730 2024-04-07T16:38:17.536 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:17.544+0000 7f346a3868c0 1 > prepared multipart upload key=6XDS0A7L0IY8 upload=GJ1270ULPUNA tag=6YL8B8BXOBII type=write size=3124882 2024-04-07T16:38:17.542 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:17.548+0000 7f346a3868c0 1 < completed part key=_multipart_EQR0FY86AT5X.2~6HX5BB8R1R17.0 size=1048576 2024-04-07T16:38:17.547 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:17.556+0000 7f346a3868c0 1 < completed part key=_multipart_EQR0FY86AT5X.2~6HX5BB8R1R17.1 size=510446 2024-04-07T16:38:17.557 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:17.564+0000 7f346a3868c0 1 < completed multipart upload key=EQR0FY86AT5X upload=6HX5BB8R1R17 tag=Y3QCM9EILSH2 type=write size=2607598 2024-04-07T16:38:17.568 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:17.576+0000 7f346a3868c0 1 > prepared part key=_multipart_N14BUXLJONHZ.2~EUADTNWO040K.0 size=1048576 2024-04-07T16:38:17.572 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:17.580+0000 7f346a3868c0 1 > prepared part key=_multipart_N14BUXLJONHZ.2~EUADTNWO040K.1 size=1048576 2024-04-07T16:38:17.577 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:17.584+0000 7f346a3868c0 1 > prepared part key=_multipart_N14BUXLJONHZ.2~EUADTNWO040K.2 size=722673 2024-04-07T16:38:17.577 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:17.584+0000 7f346a3868c0 1 > prepared multipart upload key=N14BUXLJONHZ upload=EUADTNWO040K tag=VQ54TPT0B684 type=write size=3868401 2024-04-07T16:38:17.583 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:17.592+0000 7f346a3868c0 1 < completed delete operation key=6XDS0A7L0IY8 tag=1OR7G45TXC7X type=del 2024-04-07T16:38:17.589 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:17.596+0000 7f346a3868c0 1 > prepared operation key=6XDS0A7L0IY8 tag=4KGC7FSDJ82V type=del size=2856526 2024-04-07T16:38:17.598 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:17.600+0000 7f346a3868c0 1 < canceled operation key=E84U0SX44NQ8 tag=RRKDYA01HLNV type=del size=3042964 2024-04-07T16:38:17.601 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:17.608+0000 7f346a3868c0 1 > prepared operation key=NYET3212USVE tag=6WNEMOOF0LR7 type=write size=767756 2024-04-07T16:38:17.606 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:17.612+0000 7f346a3868c0 1 < completed delete operation key=95YVKJ14JD8W tag=XFH27SOY56FZ type=del 2024-04-07T16:38:17.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:38:17 smithi144 bash[66654]: audit 2024-04-07T16:38:16.753828+0000 mon.a (mon.0) 2023 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:38:17.617 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:17.624+0000 7f346a3868c0 1 > prepared part key=_multipart_1VKVQXSQMTU1.2~342H38FJPLLJ.0 size=1048576 2024-04-07T16:38:17.622 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:17.628+0000 7f346a3868c0 1 > prepared part key=_multipart_1VKVQXSQMTU1.2~342H38FJPLLJ.1 size=175300 2024-04-07T16:38:17.622 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:17.628+0000 7f346a3868c0 1 > prepared multipart upload key=1VKVQXSQMTU1 upload=342H38FJPLLJ tag=1U62R0XVNKUY type=write size=2272452 2024-04-07T16:38:17.628 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:17.636+0000 7f346a3868c0 1 < completed part key=_multipart_EQR0FY86AT5X.2~RJBAUIKF8WN5.0 size=656755 2024-04-07T16:38:17.638 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:17.644+0000 7f346a3868c0 1 < completed multipart upload key=EQR0FY86AT5X upload=RJBAUIKF8WN5 tag=EV51ZX4BZ289 type=write size=1705331 2024-04-07T16:38:17.644 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:17.652+0000 7f346a3868c0 1 > prepared operation key=95YVKJ14JD8W tag=XRUQVXT0M070 type=del size=3119866 2024-04-07T16:38:17.658 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:17.664+0000 7f346a3868c0 1 < suggested update operation key=YOK8BAEOMPUR tag=2TN1WQMXZ1Z0 type=write size=28795 2024-04-07T16:38:17.668 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:17.676+0000 7f346a3868c0 1 > prepared part key=_multipart_HBLEXONKXJTL.2~Y841LYJEUEK5.0 size=355501 2024-04-07T16:38:17.668 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:17.676+0000 7f346a3868c0 1 > prepared multipart upload key=HBLEXONKXJTL upload=Y841LYJEUEK5 tag=WHMEE96K0429 type=write size=1404077 2024-04-07T16:38:17.674 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:17.680+0000 7f346a3868c0 1 < completed part key=_multipart_N14BUXLJONHZ.2~QOG310NTDGL7.0 size=955674 2024-04-07T16:38:17.685 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:17.692+0000 7f346a3868c0 1 < completed multipart upload key=N14BUXLJONHZ upload=QOG310NTDGL7 tag=R1WBVZ0CWQM0 type=write size=2004250 2024-04-07T16:38:17.696 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:17.704+0000 7f346a3868c0 1 > prepared part key=_multipart_C1BTVJNHIM0X.2~8R9W9H98QEKH.0 size=1048576 2024-04-07T16:38:17.701 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:17.708+0000 7f346a3868c0 1 > prepared part key=_multipart_C1BTVJNHIM0X.2~8R9W9H98QEKH.1 size=502128 2024-04-07T16:38:17.701 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:17.708+0000 7f346a3868c0 1 > prepared multipart upload key=C1BTVJNHIM0X upload=8R9W9H98QEKH tag=NPQ5UQTGU4NW type=write size=2599280 2024-04-07T16:38:17.712 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:17.720+0000 7f346a3868c0 1 < suggested remove operation key=U3EID5EK64HD tag=PNJ926TNBSD5 type=del 2024-04-07T16:38:17.718 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:17.724+0000 7f346a3868c0 1 > prepared operation key=3N0QJRYVO3FV tag=FUR31J3YWDLV type=del size=2216178 2024-04-07T16:38:17.729 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:17.736+0000 7f346a3868c0 1 < suggested update operation key=GZXHYZO5SMUP tag=ME5T75O3BI8X type=del size=1698780 2024-04-07T16:38:17.740 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:17.748+0000 7f346a3868c0 1 > prepared part key=_multipart_CL4PP1RKZU0E.2~3T1INH887ZRH.0 size=1048576 2024-04-07T16:38:17.745 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:17.752+0000 7f346a3868c0 1 > prepared part key=_multipart_CL4PP1RKZU0E.2~3T1INH887ZRH.1 size=1048576 2024-04-07T16:38:17.750 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:17.756+0000 7f346a3868c0 1 > prepared part key=_multipart_CL4PP1RKZU0E.2~3T1INH887ZRH.2 size=614662 2024-04-07T16:38:17.750 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:17.756+0000 7f346a3868c0 1 > prepared multipart upload key=CL4PP1RKZU0E upload=3T1INH887ZRH tag=XB10PE9D2HTB type=write size=3760390 2024-04-07T16:38:17.756 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:17.764+0000 7f346a3868c0 1 < completed delete operation key=EOYFVEI50SQ1 tag=KVNYE0XJIM9Q type=del 2024-04-07T16:38:17.761 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:17.768+0000 7f346a3868c0 1 > prepared operation key=6XDS0A7L0IY8 tag=XRP4KKB5OCN2 type=del size=2901530 2024-04-07T16:38:17.766 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:17.772+0000 7f346a3868c0 1 < canceled operation key=C1BTVJNHIM0X tag=08WVEBQ5P05Q type=del size=2431311 2024-04-07T16:38:17.772 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:17.780+0000 7f346a3868c0 1 > prepared operation key=QTCIIPTXIRW2 tag=E80QD3B4VSZR type=del size=840397 2024-04-07T16:38:17.778 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:17.784+0000 7f346a3868c0 1 < completed delete operation key=N14BUXLJONHZ tag=SJDRQAYVMZ2P type=del 2024-04-07T16:38:17.789 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:17.796+0000 7f346a3868c0 1 > prepared part key=_multipart_LXOZ0T2KY6OR.2~DC8Z91YDG1CF.0 size=1048576 2024-04-07T16:38:17.794 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:17.800+0000 7f346a3868c0 1 > prepared part key=_multipart_LXOZ0T2KY6OR.2~DC8Z91YDG1CF.1 size=1048576 2024-04-07T16:38:17.802 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:17.808+0000 7f346a3868c0 1 > prepared part key=_multipart_LXOZ0T2KY6OR.2~DC8Z91YDG1CF.2 size=871066 2024-04-07T16:38:17.802 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:17.808+0000 7f346a3868c0 1 > prepared multipart upload key=LXOZ0T2KY6OR upload=DC8Z91YDG1CF tag=8RXFCUJVX85N type=write size=4016794 2024-04-07T16:38:17.813 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:17.820+0000 7f346a3868c0 1 < suggested remove operation key=GZXHYZO5SMUP tag=4Q8B99OU0HAS type=del 2024-04-07T16:38:17.819 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:17.824+0000 7f346a3868c0 1 > prepared operation key=3N0QJRYVO3FV tag=1LEOOLFGZRYU type=write size=783354 2024-04-07T16:38:17.824 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:17.832+0000 7f346a3868c0 1 < completed delete operation key=QVOKSQAGHPA9 tag=MKDS578WZLFK type=del 2024-04-07T16:38:17.830 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:17.836+0000 7f346a3868c0 1 > prepared operation key=ROFH30W3UCAX tag=2K696VO9C6HZ type=write size=275240 2024-04-07T16:38:17.835 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:17.844+0000 7f346a3868c0 1 < completed part key=_multipart_GZXHYZO5SMUP.2~7DLNNBRFDKCW.0 size=914788 2024-04-07T16:38:17.846 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:17.852+0000 7f346a3868c0 1 < completed multipart upload key=GZXHYZO5SMUP upload=7DLNNBRFDKCW tag=Z6PY2Q4G4FNA type=write size=1963364 2024-04-07T16:38:17.858 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:17.864+0000 7f346a3868c0 1 > prepared part key=_multipart_YXMDZB23YTB3.2~L994KPYQSGKP.0 size=1048576 2024-04-07T16:38:17.863 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:17.868+0000 7f346a3868c0 1 > prepared part key=_multipart_YXMDZB23YTB3.2~L994KPYQSGKP.1 size=747392 2024-04-07T16:38:17.863 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:17.868+0000 7f346a3868c0 1 > prepared multipart upload key=YXMDZB23YTB3 upload=L994KPYQSGKP tag=D5UG2S09DUC2 type=write size=2844544 2024-04-07T16:38:17.868 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:17.876+0000 7f346a3868c0 1 < completed part key=_multipart_6XDS0A7L0IY8.2~GJ1270ULPUNA.0 size=1048576 2024-04-07T16:38:17.875 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:17.884+0000 7f346a3868c0 1 < completed part key=_multipart_6XDS0A7L0IY8.2~GJ1270ULPUNA.1 size=1027730 2024-04-07T16:38:17.886 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:17.892+0000 7f346a3868c0 1 < completed multipart upload key=6XDS0A7L0IY8 upload=GJ1270ULPUNA tag=6YL8B8BXOBII type=write size=3124882 2024-04-07T16:38:17.891 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:17.900+0000 7f346a3868c0 1 > prepared operation key=8F81G0P0YBFW tag=XQY19H02J3XZ type=del size=1980571 2024-04-07T16:38:17.900 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:17.908+0000 7f346a3868c0 1 < completed part key=_multipart_N14BUXLJONHZ.2~EUADTNWO040K.0 size=1048576 2024-04-07T16:38:17.905 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:17.912+0000 7f346a3868c0 1 < completed part key=_multipart_N14BUXLJONHZ.2~EUADTNWO040K.1 size=1048576 2024-04-07T16:38:17.910 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:17.916+0000 7f346a3868c0 1 < completed part key=_multipart_N14BUXLJONHZ.2~EUADTNWO040K.2 size=722673 2024-04-07T16:38:17.920 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:17.928+0000 7f346a3868c0 1 < completed multipart upload key=N14BUXLJONHZ upload=EUADTNWO040K tag=VQ54TPT0B684 type=write size=3868401 2024-04-07T16:38:17.926 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:17.932+0000 7f346a3868c0 1 > prepared operation key=EQR0FY86AT5X tag=IQE6WL9H8S65 type=del size=3250056 2024-04-07T16:38:17.932 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:17.940+0000 7f346a3868c0 1 < completed delete operation key=6XDS0A7L0IY8 tag=4KGC7FSDJ82V type=del 2024-04-07T16:38:17.938 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:17.944+0000 7f346a3868c0 1 > prepared operation key=HBLEXONKXJTL tag=TNCS1UWXS31O type=del size=1893199 2024-04-07T16:38:17.944 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:17.952+0000 7f346a3868c0 1 < completed write operation key=NYET3212USVE tag=6WNEMOOF0LR7 type=write size=767756 2024-04-07T16:38:17.953 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:17.960+0000 7f346a3868c0 1 > prepared operation key=NYET3212USVE tag=VY9PEJ6KRPD8 type=write size=733587 2024-04-07T16:38:17.960 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:17.968+0000 7f346a3868c0 1 < completed part key=_multipart_1VKVQXSQMTU1.2~342H38FJPLLJ.0 size=1048576 2024-04-07T16:38:17.964 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:17.972+0000 7f346a3868c0 1 < completed part key=_multipart_1VKVQXSQMTU1.2~342H38FJPLLJ.1 size=175300 2024-04-07T16:38:17.976 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:17.984+0000 7f346a3868c0 1 < completed multipart upload key=1VKVQXSQMTU1 upload=342H38FJPLLJ tag=1U62R0XVNKUY type=write size=2272452 2024-04-07T16:38:17.981 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:17.988+0000 7f346a3868c0 1 > prepared operation key=QVOKSQAGHPA9 tag=IWJBTW6HMX5P type=del size=1613641 2024-04-07T16:38:17.987 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:17.996+0000 7f346a3868c0 1 < completed delete operation key=95YVKJ14JD8W tag=XRUQVXT0M070 type=del 2024-04-07T16:38:17.999 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:18.008+0000 7f346a3868c0 1 > prepared part key=_multipart_E84U0SX44NQ8.2~960MLZCD70XO.0 size=921542 2024-04-07T16:38:17.999 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:18.008+0000 7f346a3868c0 1 > prepared multipart upload key=E84U0SX44NQ8 upload=960MLZCD70XO tag=XHSZ0UGDPDU9 type=write size=1970118 2024-04-07T16:38:18.000 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:18.008+0000 7f346a3868c0 1 < canceled part key=_multipart_HBLEXONKXJTL.2~Y841LYJEUEK5.0 size=355501 2024-04-07T16:38:18.010 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:18.016+0000 7f346a3868c0 1 < canceled multipart upload key=HBLEXONKXJTL upload=Y841LYJEUEK5 tag=WHMEE96K0429 type=write size=1404077 2024-04-07T16:38:18.015 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:18.024+0000 7f346a3868c0 1 > prepared operation key=LXOZ0T2KY6OR tag=DXWB1JLK89AO type=del size=167756 2024-04-07T16:38:18.022 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:18.028+0000 7f346a3868c0 1 < completed part key=_multipart_C1BTVJNHIM0X.2~8R9W9H98QEKH.0 size=1048576 2024-04-07T16:38:18.022 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:18.028+0000 7f346a3868c0 1 < canceled part key=_multipart_C1BTVJNHIM0X.2~8R9W9H98QEKH.1 size=502128 2024-04-07T16:38:18.032 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:18.040+0000 7f346a3868c0 1 < canceled multipart upload key=C1BTVJNHIM0X upload=8R9W9H98QEKH tag=NPQ5UQTGU4NW type=write size=2599280 2024-04-07T16:38:18.043 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:18.052+0000 7f346a3868c0 1 > prepared part key=_multipart_GBQSME7SY1AH.2~ZPVAP2X791JV.0 size=917813 2024-04-07T16:38:18.043 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:18.052+0000 7f346a3868c0 1 > prepared multipart upload key=GBQSME7SY1AH upload=ZPVAP2X791JV tag=H0222FHSFYTT type=write size=1966389 2024-04-07T16:38:18.055 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:18.064+0000 7f346a3868c0 1 < suggested update operation key=3N0QJRYVO3FV tag=FUR31J3YWDLV type=del size=2216178 2024-04-07T16:38:18.061 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:18.068+0000 7f346a3868c0 1 > prepared operation key=QTCIIPTXIRW2 tag=CDQ4HILZIR12 type=del size=2225642 2024-04-07T16:38:18.068 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:18.076+0000 7f346a3868c0 1 < completed part key=_multipart_CL4PP1RKZU0E.2~3T1INH887ZRH.0 size=1048576 2024-04-07T16:38:18.073 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:18.080+0000 7f346a3868c0 1 < completed part key=_multipart_CL4PP1RKZU0E.2~3T1INH887ZRH.1 size=1048576 2024-04-07T16:38:18.078 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:18.084+0000 7f346a3868c0 1 < completed part key=_multipart_CL4PP1RKZU0E.2~3T1INH887ZRH.2 size=614662 2024-04-07T16:38:18.088 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:18.096+0000 7f346a3868c0 1 < completed multipart upload key=CL4PP1RKZU0E upload=3T1INH887ZRH tag=XB10PE9D2HTB type=write size=3760390 2024-04-07T16:38:18.099 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:18.104+0000 7f346a3868c0 1 > prepared part key=_multipart_8F81G0P0YBFW.2~QNOCNX10NQ5H.0 size=1048576 2024-04-07T16:38:18.104 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:18.112+0000 7f346a3868c0 1 > prepared part key=_multipart_8F81G0P0YBFW.2~QNOCNX10NQ5H.1 size=1048576 2024-04-07T16:38:18.109 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:18.116+0000 7f346a3868c0 1 > prepared part key=_multipart_8F81G0P0YBFW.2~QNOCNX10NQ5H.2 size=152239 2024-04-07T16:38:18.109 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:18.116+0000 7f346a3868c0 1 > prepared multipart upload key=8F81G0P0YBFW upload=QNOCNX10NQ5H tag=2WMGQMMIVJ59 type=write size=3297967 2024-04-07T16:38:18.115 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:18.120+0000 7f346a3868c0 1 < completed delete operation key=6XDS0A7L0IY8 tag=XRP4KKB5OCN2 type=del 2024-04-07T16:38:18.126 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:18.132+0000 7f346a3868c0 1 > prepared part key=_multipart_1WGUPH7X5UA0.2~2J33CMNF4I5Z.0 size=871568 2024-04-07T16:38:18.126 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:18.132+0000 7f346a3868c0 1 > prepared multipart upload key=1WGUPH7X5UA0 upload=2J33CMNF4I5Z tag=MROH6OT2DMHO type=write size=1920144 2024-04-07T16:38:18.138 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:18.144+0000 7f346a3868c0 1 < suggested update operation key=QTCIIPTXIRW2 tag=E80QD3B4VSZR type=del size=840397 2024-04-07T16:38:18.144 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:18.152+0000 7f346a3868c0 1 > prepared operation key=LXOZ0T2KY6OR tag=696GC968YE0F type=write size=720077 2024-04-07T16:38:18.150 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:18.156+0000 7f346a3868c0 1 < completed part key=_multipart_LXOZ0T2KY6OR.2~DC8Z91YDG1CF.0 size=1048576 2024-04-07T16:38:18.155 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:18.164+0000 7f346a3868c0 1 < completed part key=_multipart_LXOZ0T2KY6OR.2~DC8Z91YDG1CF.1 size=1048576 2024-04-07T16:38:18.162 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:18.168+0000 7f346a3868c0 1 < completed part key=_multipart_LXOZ0T2KY6OR.2~DC8Z91YDG1CF.2 size=871066 2024-04-07T16:38:18.174 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:18.180+0000 7f346a3868c0 1 < completed multipart upload key=LXOZ0T2KY6OR upload=DC8Z91YDG1CF tag=8RXFCUJVX85N type=write size=4016794 2024-04-07T16:38:18.180 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:18.188+0000 7f346a3868c0 1 > prepared operation key=3N0QJRYVO3FV tag=9CVL38DLKSWK type=del size=350424 2024-04-07T16:38:18.189 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:18.196+0000 7f346a3868c0 -1 < failed to complete operation key=3N0QJRYVO3FV tag=1LEOOLFGZRYU type=write size=783354: (22) Invalid argument 2024-04-07T16:38:18.203 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:18.212+0000 7f346a3868c0 1 > prepared part key=_multipart_ROFH30W3UCAX.2~D0BIEH5WOV20.0 size=1048576 2024-04-07T16:38:18.208 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:18.216+0000 7f346a3868c0 1 > prepared part key=_multipart_ROFH30W3UCAX.2~D0BIEH5WOV20.1 size=232520 2024-04-07T16:38:18.209 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:18.216+0000 7f346a3868c0 1 > prepared multipart upload key=ROFH30W3UCAX upload=D0BIEH5WOV20 tag=K5CUZ32YI3RS type=write size=2329672 2024-04-07T16:38:18.214 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:18.220+0000 7f346a3868c0 1 < completed write operation key=ROFH30W3UCAX tag=2K696VO9C6HZ type=write size=275240 2024-04-07T16:38:18.220 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:18.228+0000 7f346a3868c0 1 > prepared operation key=QVOKSQAGHPA9 tag=52JYM0QBV2FX type=write size=587385 2024-04-07T16:38:18.227 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:18.232+0000 7f346a3868c0 1 < completed part key=_multipart_YXMDZB23YTB3.2~L994KPYQSGKP.0 size=1048576 2024-04-07T16:38:18.234 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:18.240+0000 7f346a3868c0 1 < completed part key=_multipart_YXMDZB23YTB3.2~L994KPYQSGKP.1 size=747392 2024-04-07T16:38:18.245 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:18.252+0000 7f346a3868c0 1 < completed multipart upload key=YXMDZB23YTB3 upload=L994KPYQSGKP tag=D5UG2S09DUC2 type=write size=2844544 2024-04-07T16:38:18.256 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:18.264+0000 7f346a3868c0 1 > prepared part key=_multipart_6XDS0A7L0IY8.2~FBFJZNQ5N5TR.0 size=953552 2024-04-07T16:38:18.256 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:18.264+0000 7f346a3868c0 1 > prepared multipart upload key=6XDS0A7L0IY8 upload=FBFJZNQ5N5TR tag=VRVI099FBXTO type=write size=2002128 2024-04-07T16:38:18.262 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:18.268+0000 7f346a3868c0 1 < canceled operation key=8F81G0P0YBFW tag=XQY19H02J3XZ type=del size=1980571 2024-04-07T16:38:18.270 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:18.276+0000 7f346a3868c0 1 > prepared operation key=EQR0FY86AT5X tag=BHOKY3YG5J2D type=del size=2375481 2024-04-07T16:38:18.282 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:18.288+0000 7f346a3868c0 1 < suggested update operation key=EQR0FY86AT5X tag=IQE6WL9H8S65 type=del size=3250056 2024-04-07T16:38:18.293 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:18.300+0000 7f346a3868c0 1 > prepared part key=_multipart_1VKVQXSQMTU1.2~5UZCNP22QX8W.0 size=1048576 2024-04-07T16:38:18.302 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:18.308+0000 7f346a3868c0 1 > prepared part key=_multipart_1VKVQXSQMTU1.2~5UZCNP22QX8W.1 size=559611 2024-04-07T16:38:18.302 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:18.308+0000 7f346a3868c0 1 > prepared multipart upload key=1VKVQXSQMTU1 upload=5UZCNP22QX8W tag=WKLFR1KZKEV8 type=write size=2656763 2024-04-07T16:38:18.317 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:18.324+0000 7f346a3868c0 1 < suggested update operation key=HBLEXONKXJTL tag=TNCS1UWXS31O type=del size=1893199 2024-04-07T16:38:18.329 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:18.336+0000 7f346a3868c0 1 > prepared part key=_multipart_UIW3DN4NUYPP.2~XJHYMWDX980H.0 size=1048576 2024-04-07T16:38:18.336 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:18.344+0000 7f346a3868c0 1 > prepared part key=_multipart_UIW3DN4NUYPP.2~XJHYMWDX980H.1 size=963212 2024-04-07T16:38:18.336 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:18.344+0000 7f346a3868c0 1 > prepared multipart upload key=UIW3DN4NUYPP upload=XJHYMWDX980H tag=UQVTHKQ50CV7 type=write size=3060364 2024-04-07T16:38:18.348 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:18.356+0000 7f346a3868c0 1 < suggested remove operation key=NYET3212USVE tag=VY9PEJ6KRPD8 type=write 2024-04-07T16:38:18.354 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:18.360+0000 7f346a3868c0 1 > prepared operation key=E84U0SX44NQ8 tag=DDN8N4JWSDNU type=del size=1788779 2024-04-07T16:38:18.363 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:18.372+0000 7f346a3868c0 1 < completed delete operation key=QVOKSQAGHPA9 tag=IWJBTW6HMX5P type=del 2024-04-07T16:38:18.375 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:18.380+0000 7f346a3868c0 1 > prepared part key=_multipart_7G5C6HBK6QWI.2~WFWKP4LJ6ID3.0 size=1048576 2024-04-07T16:38:18.380 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:18.388+0000 7f346a3868c0 1 > prepared part key=_multipart_7G5C6HBK6QWI.2~WFWKP4LJ6ID3.1 size=1048576 2024-04-07T16:38:18.385 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:18.392+0000 7f346a3868c0 1 > prepared part key=_multipart_7G5C6HBK6QWI.2~WFWKP4LJ6ID3.2 size=96309 2024-04-07T16:38:18.385 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:18.392+0000 7f346a3868c0 1 > prepared multipart upload key=7G5C6HBK6QWI upload=WFWKP4LJ6ID3 tag=FN4DQ5HU9FFB type=write size=3242037 2024-04-07T16:38:18.392 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:18.400+0000 7f346a3868c0 1 < completed part key=_multipart_E84U0SX44NQ8.2~960MLZCD70XO.0 size=921542 2024-04-07T16:38:18.402 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:18.408+0000 7f346a3868c0 1 < completed multipart upload key=E84U0SX44NQ8 upload=960MLZCD70XO tag=XHSZ0UGDPDU9 type=write size=1970118 2024-04-07T16:38:18.408 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:18.416+0000 7f346a3868c0 1 > prepared operation key=ROFH30W3UCAX tag=RNC08KJ8LF2N type=del size=3414594 2024-04-07T16:38:18.415 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:18.420+0000 7f346a3868c0 1 < completed delete operation key=LXOZ0T2KY6OR tag=DXWB1JLK89AO type=del 2024-04-07T16:38:18.418 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:38:18 smithi100 bash[74942]: cluster 2024-04-07T16:38:17.073849+0000 mgr.y (mgr.25175) 688 : cluster [DBG] pgmap v1323: 137 pgs: 137 active+clean; 583 KiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 336 KiB/s rd, 117 KiB/s wr, 593 op/s 2024-04-07T16:38:18.418 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:38:18 smithi100 bash[76202]: cluster 2024-04-07T16:38:17.073849+0000 mgr.y (mgr.25175) 688 : cluster [DBG] pgmap v1323: 137 pgs: 137 active+clean; 583 KiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 336 KiB/s rd, 117 KiB/s wr, 593 op/s 2024-04-07T16:38:18.422 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:18.428+0000 7f346a3868c0 1 > prepared operation key=CL4PP1RKZU0E tag=9CHNSMELN2U9 type=del size=2172703 2024-04-07T16:38:18.428 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:18.436+0000 7f346a3868c0 1 < completed part key=_multipart_GBQSME7SY1AH.2~ZPVAP2X791JV.0 size=917813 2024-04-07T16:38:18.438 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:18.444+0000 7f346a3868c0 1 < completed multipart upload key=GBQSME7SY1AH upload=ZPVAP2X791JV tag=H0222FHSFYTT type=write size=1966389 2024-04-07T16:38:18.443 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:18.452+0000 7f346a3868c0 1 > prepared operation key=QVOKSQAGHPA9 tag=51HLUSY4MZXU type=del size=4019887 2024-04-07T16:38:18.451 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:18.460+0000 7f346a3868c0 -1 < failed to complete operation key=QTCIIPTXIRW2 tag=CDQ4HILZIR12 type=del size=2225642: (22) Invalid argument 2024-04-07T16:38:18.457 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:18.464+0000 7f346a3868c0 1 > prepared operation key=1VBSUWNI3V1T tag=I5LARGV7REJK type=del size=2645917 2024-04-07T16:38:18.463 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:18.468+0000 7f346a3868c0 1 < completed part key=_multipart_8F81G0P0YBFW.2~QNOCNX10NQ5H.0 size=1048576 2024-04-07T16:38:18.468 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:18.476+0000 7f346a3868c0 1 < completed part key=_multipart_8F81G0P0YBFW.2~QNOCNX10NQ5H.1 size=1048576 2024-04-07T16:38:18.473 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:18.480+0000 7f346a3868c0 1 < completed part key=_multipart_8F81G0P0YBFW.2~QNOCNX10NQ5H.2 size=152239 2024-04-07T16:38:18.483 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:18.492+0000 7f346a3868c0 1 < completed multipart upload key=8F81G0P0YBFW upload=QNOCNX10NQ5H tag=2WMGQMMIVJ59 type=write size=3297967 2024-04-07T16:38:18.489 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:18.496+0000 7f346a3868c0 1 > prepared operation key=6XDS0A7L0IY8 tag=DAGKT0M74C9D type=del size=3523296 2024-04-07T16:38:18.495 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:18.504+0000 7f346a3868c0 1 < completed part key=_multipart_1WGUPH7X5UA0.2~2J33CMNF4I5Z.0 size=871568 2024-04-07T16:38:18.506 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:18.512+0000 7f346a3868c0 1 < completed multipart upload key=1WGUPH7X5UA0 upload=2J33CMNF4I5Z tag=MROH6OT2DMHO type=write size=1920144 2024-04-07T16:38:18.512 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:18.520+0000 7f346a3868c0 1 > prepared operation key=6XDS0A7L0IY8 tag=BMYGF7FT7AZL type=del size=261748 2024-04-07T16:38:18.519 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:18.524+0000 7f346a3868c0 1 < completed write operation key=LXOZ0T2KY6OR tag=696GC968YE0F type=write size=720077 2024-04-07T16:38:18.530 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:18.536+0000 7f346a3868c0 1 > prepared part key=_multipart_GBQSME7SY1AH.2~OWXX32ALX6VD.0 size=1048576 2024-04-07T16:38:18.535 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:18.544+0000 7f346a3868c0 1 > prepared part key=_multipart_GBQSME7SY1AH.2~OWXX32ALX6VD.1 size=673634 2024-04-07T16:38:18.535 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:18.544+0000 7f346a3868c0 1 > prepared multipart upload key=GBQSME7SY1AH upload=OWXX32ALX6VD tag=4WVNHYI1XJCZ type=write size=2770786 2024-04-07T16:38:18.551 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:18.560+0000 7f346a3868c0 1 < suggested update operation key=3N0QJRYVO3FV tag=9CVL38DLKSWK type=del size=350424 2024-04-07T16:38:18.560 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:18.568+0000 7f346a3868c0 1 > prepared operation key=95YVKJ14JD8W tag=11C11G1M5I75 type=del size=2835365 2024-04-07T16:38:18.567 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:18.576+0000 7f346a3868c0 1 < completed part key=_multipart_ROFH30W3UCAX.2~D0BIEH5WOV20.0 size=1048576 2024-04-07T16:38:18.573 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:18.580+0000 7f346a3868c0 1 < completed part key=_multipart_ROFH30W3UCAX.2~D0BIEH5WOV20.1 size=232520 2024-04-07T16:38:18.583 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:18.592+0000 7f346a3868c0 1 < completed multipart upload key=ROFH30W3UCAX upload=D0BIEH5WOV20 tag=K5CUZ32YI3RS type=write size=2329672 2024-04-07T16:38:18.589 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:18.596+0000 7f346a3868c0 1 > prepared operation key=95YVKJ14JD8W tag=INQXEQ0KJNCZ type=del size=3938203 2024-04-07T16:38:18.593 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:38:18 smithi144 bash[66654]: cluster 2024-04-07T16:38:17.073849+0000 mgr.y (mgr.25175) 688 : cluster [DBG] pgmap v1323: 137 pgs: 137 active+clean; 583 KiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 336 KiB/s rd, 117 KiB/s wr, 593 op/s 2024-04-07T16:38:18.596 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:18.604+0000 7f346a3868c0 1 < completed write operation key=QVOKSQAGHPA9 tag=52JYM0QBV2FX type=write size=587385 2024-04-07T16:38:18.602 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:18.608+0000 7f346a3868c0 1 > prepared operation key=8F81G0P0YBFW tag=4ZP9A80YHPFP type=del size=924930 2024-04-07T16:38:18.603 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:18.612+0000 7f346a3868c0 1 < canceled part key=_multipart_6XDS0A7L0IY8.2~FBFJZNQ5N5TR.0 size=953552 2024-04-07T16:38:18.613 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:18.620+0000 7f346a3868c0 1 < canceled multipart upload key=6XDS0A7L0IY8 upload=FBFJZNQ5N5TR tag=VRVI099FBXTO type=write size=2002128 2024-04-07T16:38:18.619 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:18.628+0000 7f346a3868c0 1 > prepared operation key=XJLQZ7IS9U1T tag=35QTTDTKLVZ2 type=del size=769729 2024-04-07T16:38:18.627 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:18.636+0000 7f346a3868c0 -1 < failed to complete operation key=EQR0FY86AT5X tag=BHOKY3YG5J2D type=del size=2375481: (22) Invalid argument 2024-04-07T16:38:18.633 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:18.640+0000 7f346a3868c0 1 > prepared operation key=UIW3DN4NUYPP tag=MH7KE7U0Y7NU type=del size=773441 2024-04-07T16:38:18.639 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:18.648+0000 7f346a3868c0 1 < completed part key=_multipart_1VKVQXSQMTU1.2~5UZCNP22QX8W.0 size=1048576 2024-04-07T16:38:18.644 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:18.652+0000 7f346a3868c0 1 < completed part key=_multipart_1VKVQXSQMTU1.2~5UZCNP22QX8W.1 size=559611 2024-04-07T16:38:18.656 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:18.664+0000 7f346a3868c0 1 < completed multipart upload key=1VKVQXSQMTU1 upload=5UZCNP22QX8W tag=WKLFR1KZKEV8 type=write size=2656763 2024-04-07T16:38:18.662 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:18.668+0000 7f346a3868c0 1 > prepared operation key=IP8YW2A75TCW tag=TTANI0P5S1JG type=del size=1526696 2024-04-07T16:38:18.668 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:18.676+0000 7f346a3868c0 1 < completed part key=_multipart_UIW3DN4NUYPP.2~XJHYMWDX980H.0 size=1048576 2024-04-07T16:38:18.673 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:18.680+0000 7f346a3868c0 1 < completed part key=_multipart_UIW3DN4NUYPP.2~XJHYMWDX980H.1 size=963212 2024-04-07T16:38:18.683 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:18.692+0000 7f346a3868c0 1 < completed multipart upload key=UIW3DN4NUYPP upload=XJHYMWDX980H tag=UQVTHKQ50CV7 type=write size=3060364 2024-04-07T16:38:18.691 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:38:18 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:38:18] "GET /metrics HTTP/1.1" 200 36375 "" "Prometheus/2.43.0" 2024-04-07T16:38:18.694 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:18.700+0000 7f346a3868c0 1 > prepared part key=_multipart_N14BUXLJONHZ.2~MMO65BEMBKME.0 size=1048576 2024-04-07T16:38:18.700 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:18.708+0000 7f346a3868c0 1 > prepared part key=_multipart_N14BUXLJONHZ.2~MMO65BEMBKME.1 size=301471 2024-04-07T16:38:18.700 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:18.708+0000 7f346a3868c0 1 > prepared multipart upload key=N14BUXLJONHZ upload=MMO65BEMBKME tag=2WSIY58ID0UC type=write size=2398623 2024-04-07T16:38:18.706 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:18.712+0000 7f346a3868c0 1 < completed delete operation key=E84U0SX44NQ8 tag=DDN8N4JWSDNU type=del 2024-04-07T16:38:18.720 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:18.728+0000 7f346a3868c0 1 > prepared part key=_multipart_LXOZ0T2KY6OR.2~8OSD46ESX9LU.0 size=1048576 2024-04-07T16:38:18.725 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:18.732+0000 7f346a3868c0 1 > prepared part key=_multipart_LXOZ0T2KY6OR.2~8OSD46ESX9LU.1 size=1048576 2024-04-07T16:38:18.730 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:18.736+0000 7f346a3868c0 1 > prepared part key=_multipart_LXOZ0T2KY6OR.2~8OSD46ESX9LU.2 size=178416 2024-04-07T16:38:18.730 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:18.736+0000 7f346a3868c0 1 > prepared multipart upload key=LXOZ0T2KY6OR upload=8OSD46ESX9LU tag=D4JJGKQZQAN8 type=write size=3324144 2024-04-07T16:38:18.736 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:18.744+0000 7f346a3868c0 1 < completed part key=_multipart_7G5C6HBK6QWI.2~WFWKP4LJ6ID3.0 size=1048576 2024-04-07T16:38:18.741 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:18.748+0000 7f346a3868c0 1 < completed part key=_multipart_7G5C6HBK6QWI.2~WFWKP4LJ6ID3.1 size=1048576 2024-04-07T16:38:18.746 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:18.752+0000 7f346a3868c0 1 < completed part key=_multipart_7G5C6HBK6QWI.2~WFWKP4LJ6ID3.2 size=96309 2024-04-07T16:38:18.759 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:18.764+0000 7f346a3868c0 1 < completed multipart upload key=7G5C6HBK6QWI upload=WFWKP4LJ6ID3 tag=FN4DQ5HU9FFB type=write size=3242037 2024-04-07T16:38:18.764 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:18.772+0000 7f346a3868c0 1 > prepared operation key=N14BUXLJONHZ tag=QI894DLY98A3 type=write size=785159 2024-04-07T16:38:18.770 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:18.776+0000 7f346a3868c0 1 < canceled operation key=ROFH30W3UCAX tag=RNC08KJ8LF2N type=del size=3414594 2024-04-07T16:38:18.781 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:18.788+0000 7f346a3868c0 1 > prepared part key=_multipart_LXOZ0T2KY6OR.2~0OTX60MFPE93.0 size=1048576 2024-04-07T16:38:18.786 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:18.792+0000 7f346a3868c0 1 > prepared part key=_multipart_LXOZ0T2KY6OR.2~0OTX60MFPE93.1 size=201697 2024-04-07T16:38:18.786 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:18.792+0000 7f346a3868c0 1 > prepared multipart upload key=LXOZ0T2KY6OR upload=0OTX60MFPE93 tag=XUCJP3R9LF5Y type=write size=2298849 2024-04-07T16:38:18.792 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:18.800+0000 7f346a3868c0 1 < completed delete operation key=CL4PP1RKZU0E tag=9CHNSMELN2U9 type=del 2024-04-07T16:38:18.798 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:18.804+0000 7f346a3868c0 1 > prepared operation key=HBLEXONKXJTL tag=VPJ1ZWNY5HOQ type=del size=2983074 2024-04-07T16:38:18.813 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:18.820+0000 7f346a3868c0 1 < suggested update operation key=QVOKSQAGHPA9 tag=51HLUSY4MZXU type=del size=4019887 2024-04-07T16:38:18.819 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:18.828+0000 7f346a3868c0 1 > prepared operation key=LXOZ0T2KY6OR tag=JBS9WQLBY30Y type=write size=305813 2024-04-07T16:38:18.825 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:18.832+0000 7f346a3868c0 1 < canceled operation key=1VBSUWNI3V1T tag=I5LARGV7REJK type=del size=2645917 2024-04-07T16:38:18.831 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:18.840+0000 7f346a3868c0 1 > prepared operation key=XJLQZ7IS9U1T tag=CW96L6F22FQI type=del size=1217112 2024-04-07T16:38:18.837 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:18.844+0000 7f346a3868c0 1 < completed delete operation key=6XDS0A7L0IY8 tag=DAGKT0M74C9D type=del 2024-04-07T16:38:18.843 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:18.852+0000 7f346a3868c0 1 > prepared operation key=C1BTVJNHIM0X tag=JNMFN6T4SE9L type=del size=1934506 2024-04-07T16:38:18.849 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:18.856+0000 7f346a3868c0 1 < completed delete operation key=6XDS0A7L0IY8 tag=BMYGF7FT7AZL type=del 2024-04-07T16:38:18.855 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:18.864+0000 7f346a3868c0 1 > prepared operation key=U3EID5EK64HD tag=V7XZZFCKCPL0 type=write size=67836 2024-04-07T16:38:18.856 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:38:18 smithi144 bash[98866]: logger=cleanup t=2024-04-07T16:38:18.602358779Z level=info msg="Completed cleanup jobs" duration=182.925083ms 2024-04-07T16:38:18.862 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:18.868+0000 7f346a3868c0 1 < completed part key=_multipart_GBQSME7SY1AH.2~OWXX32ALX6VD.0 size=1048576 2024-04-07T16:38:18.867 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:18.876+0000 7f346a3868c0 1 < completed part key=_multipart_GBQSME7SY1AH.2~OWXX32ALX6VD.1 size=673634 2024-04-07T16:38:18.877 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:18.884+0000 7f346a3868c0 1 < completed multipart upload key=GBQSME7SY1AH upload=OWXX32ALX6VD tag=4WVNHYI1XJCZ type=write size=2770786 2024-04-07T16:38:18.883 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:18.892+0000 7f346a3868c0 1 > prepared operation key=1WGUPH7X5UA0 tag=0K65D2VO9RKP type=del size=834092 2024-04-07T16:38:18.889 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:18.896+0000 7f346a3868c0 1 < completed delete operation key=95YVKJ14JD8W tag=11C11G1M5I75 type=del 2024-04-07T16:38:18.896 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:18.904+0000 7f346a3868c0 1 > prepared operation key=IP8YW2A75TCW tag=AHQEHC43I180 type=del size=2645525 2024-04-07T16:38:18.908 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:18.916+0000 7f346a3868c0 1 < suggested update operation key=95YVKJ14JD8W tag=INQXEQ0KJNCZ type=del size=3938203 2024-04-07T16:38:18.919 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:18.924+0000 7f346a3868c0 1 > prepared part key=_multipart_E84U0SX44NQ8.2~DVGKI0D9KP7E.0 size=1048576 2024-04-07T16:38:18.926 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:18.932+0000 7f346a3868c0 1 > prepared part key=_multipart_E84U0SX44NQ8.2~DVGKI0D9KP7E.1 size=1048576 2024-04-07T16:38:18.931 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:18.940+0000 7f346a3868c0 1 > prepared part key=_multipart_E84U0SX44NQ8.2~DVGKI0D9KP7E.2 size=987466 2024-04-07T16:38:18.931 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:18.940+0000 7f346a3868c0 1 > prepared multipart upload key=E84U0SX44NQ8 upload=DVGKI0D9KP7E tag=87PEGI8SUGFC type=write size=4133194 2024-04-07T16:38:18.938 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:18.944+0000 7f346a3868c0 1 < completed delete operation key=8F81G0P0YBFW tag=4ZP9A80YHPFP type=del 2024-04-07T16:38:18.949 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:18.956+0000 7f346a3868c0 1 > prepared part key=_multipart_XJLQZ7IS9U1T.2~WTWX4JVTA1IA.0 size=930467 2024-04-07T16:38:18.949 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:18.956+0000 7f346a3868c0 1 > prepared multipart upload key=XJLQZ7IS9U1T upload=WTWX4JVTA1IA tag=UOZCVWZ5TE3O type=write size=1979043 2024-04-07T16:38:18.955 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:18.964+0000 7f346a3868c0 1 < completed delete operation key=XJLQZ7IS9U1T tag=35QTTDTKLVZ2 type=del 2024-04-07T16:38:18.961 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:18.968+0000 7f346a3868c0 1 > prepared operation key=IX3F0IUOKBX1 tag=103JMEUSQUPL type=del size=1603074 2024-04-07T16:38:18.975 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:18.980+0000 7f346a3868c0 1 < suggested update operation key=UIW3DN4NUYPP tag=MH7KE7U0Y7NU type=del size=773441 2024-04-07T16:38:18.986 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:18.992+0000 7f346a3868c0 1 > prepared part key=_multipart_95YVKJ14JD8W.2~NX3Y7GYQYGK7.0 size=1048576 2024-04-07T16:38:18.991 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:19.000+0000 7f346a3868c0 1 > prepared part key=_multipart_95YVKJ14JD8W.2~NX3Y7GYQYGK7.1 size=1048576 2024-04-07T16:38:18.997 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:19.004+0000 7f346a3868c0 1 > prepared part key=_multipart_95YVKJ14JD8W.2~NX3Y7GYQYGK7.2 size=797981 2024-04-07T16:38:18.997 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:19.004+0000 7f346a3868c0 1 > prepared multipart upload key=95YVKJ14JD8W upload=NX3Y7GYQYGK7 tag=9X6Y31NF5U25 type=write size=3943709 2024-04-07T16:38:19.006 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:19.012+0000 7f346a3868c0 1 < completed delete operation key=IP8YW2A75TCW tag=TTANI0P5S1JG type=del 2024-04-07T16:38:19.013 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:19.020+0000 7f346a3868c0 1 > prepared operation key=C1BTVJNHIM0X tag=JVDYL08WSF9H type=del size=1285543 2024-04-07T16:38:19.021 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:19.028+0000 7f346a3868c0 1 < completed part key=_multipart_N14BUXLJONHZ.2~MMO65BEMBKME.0 size=1048576 2024-04-07T16:38:19.026 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:19.032+0000 7f346a3868c0 1 < completed part key=_multipart_N14BUXLJONHZ.2~MMO65BEMBKME.1 size=301471 2024-04-07T16:38:19.039 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:19.048+0000 7f346a3868c0 1 < completed multipart upload key=N14BUXLJONHZ upload=MMO65BEMBKME tag=2WSIY58ID0UC type=write size=2398623 2024-04-07T16:38:19.051 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:19.056+0000 7f346a3868c0 1 > prepared part key=_multipart_QS4HYZFFAJNF.2~5MAV8HQQSYZM.0 size=1048576 2024-04-07T16:38:19.056 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:19.064+0000 7f346a3868c0 1 > prepared part key=_multipart_QS4HYZFFAJNF.2~5MAV8HQQSYZM.1 size=914156 2024-04-07T16:38:19.056 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:19.064+0000 7f346a3868c0 1 > prepared multipart upload key=QS4HYZFFAJNF upload=5MAV8HQQSYZM tag=89TUJXY69EXC type=write size=3011308 2024-04-07T16:38:19.057 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:19.064+0000 7f346a3868c0 1 < canceled part key=_multipart_LXOZ0T2KY6OR.2~8OSD46ESX9LU.0 size=1048576 2024-04-07T16:38:19.057 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:19.064+0000 7f346a3868c0 1 < canceled part key=_multipart_LXOZ0T2KY6OR.2~8OSD46ESX9LU.1 size=1048576 2024-04-07T16:38:19.062 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:19.068+0000 7f346a3868c0 1 < completed part key=_multipart_LXOZ0T2KY6OR.2~8OSD46ESX9LU.2 size=178416 2024-04-07T16:38:19.073 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:19.080+0000 7f346a3868c0 1 < canceled multipart upload key=LXOZ0T2KY6OR upload=8OSD46ESX9LU tag=D4JJGKQZQAN8 type=write size=3324144 2024-04-07T16:38:19.084 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:19.092+0000 7f346a3868c0 1 > prepared part key=_multipart_U3EID5EK64HD.2~00RLM2IXSWSN.0 size=1048576 2024-04-07T16:38:19.089 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:19.096+0000 7f346a3868c0 1 > prepared part key=_multipart_U3EID5EK64HD.2~00RLM2IXSWSN.1 size=208205 2024-04-07T16:38:19.089 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:19.096+0000 7f346a3868c0 1 > prepared multipart upload key=U3EID5EK64HD upload=00RLM2IXSWSN tag=0D5C0J6V7IYZ type=write size=2305357 2024-04-07T16:38:19.096 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:19.104+0000 7f346a3868c0 1 < completed write operation key=N14BUXLJONHZ tag=QI894DLY98A3 type=write size=785159 2024-04-07T16:38:19.108 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:19.116+0000 7f346a3868c0 1 > prepared part key=_multipart_IX3F0IUOKBX1.2~7KJ7K7IFFU08.0 size=1048576 2024-04-07T16:38:19.113 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:19.120+0000 7f346a3868c0 1 > prepared part key=_multipart_IX3F0IUOKBX1.2~7KJ7K7IFFU08.1 size=1048576 2024-04-07T16:38:19.118 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:19.124+0000 7f346a3868c0 1 > prepared part key=_multipart_IX3F0IUOKBX1.2~7KJ7K7IFFU08.2 size=342302 2024-04-07T16:38:19.118 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:19.124+0000 7f346a3868c0 1 > prepared multipart upload key=IX3F0IUOKBX1 upload=7KJ7K7IFFU08 tag=EM93YHPHMW84 type=write size=3488030 2024-04-07T16:38:19.126 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:19.132+0000 7f346a3868c0 1 < completed part key=_multipart_LXOZ0T2KY6OR.2~0OTX60MFPE93.0 size=1048576 2024-04-07T16:38:19.132 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:19.140+0000 7f346a3868c0 1 < completed part key=_multipart_LXOZ0T2KY6OR.2~0OTX60MFPE93.1 size=201697 2024-04-07T16:38:19.144 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:19.152+0000 7f346a3868c0 1 < completed multipart upload key=LXOZ0T2KY6OR upload=0OTX60MFPE93 tag=XUCJP3R9LF5Y type=write size=2298849 2024-04-07T16:38:19.151 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:19.160+0000 7f346a3868c0 1 > prepared operation key=1VBSUWNI3V1T tag=QSBI1DU9CYPY type=del size=1975873 2024-04-07T16:38:19.158 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:19.164+0000 7f346a3868c0 1 < canceled operation key=HBLEXONKXJTL tag=VPJ1ZWNY5HOQ type=del size=2983074 2024-04-07T16:38:19.164 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:19.172+0000 7f346a3868c0 1 > prepared operation key=6XDS0A7L0IY8 tag=PA0S1ST5311X type=del size=103729 2024-04-07T16:38:19.171 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:19.180+0000 7f346a3868c0 1 < completed write operation key=LXOZ0T2KY6OR tag=JBS9WQLBY30Y type=write size=305813 2024-04-07T16:38:19.182 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:19.188+0000 7f346a3868c0 1 > prepared part key=_multipart_YXMDZB23YTB3.2~J8SYAWMTUD4D.0 size=640000 2024-04-07T16:38:19.182 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:19.188+0000 7f346a3868c0 1 > prepared multipart upload key=YXMDZB23YTB3 upload=J8SYAWMTUD4D tag=Q2VLHTXTT0QZ type=write size=1688576 2024-04-07T16:38:19.187 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:19.196+0000 7f346a3868c0 1 < canceled operation key=XJLQZ7IS9U1T tag=CW96L6F22FQI type=del size=1217112 2024-04-07T16:38:19.193 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:19.200+0000 7f346a3868c0 1 > prepared operation key=N14BUXLJONHZ tag=IORHZ1FRSP95 type=write size=741388 2024-04-07T16:38:19.199 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:19.208+0000 7f346a3868c0 1 < canceled operation key=C1BTVJNHIM0X tag=JNMFN6T4SE9L type=del size=1934506 2024-04-07T16:38:19.205 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:19.212+0000 7f346a3868c0 1 > prepared operation key=UIW3DN4NUYPP tag=OPJGQ2HUNPFX type=del size=2122000 2024-04-07T16:38:19.211 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:19.220+0000 7f346a3868c0 1 < completed write operation key=U3EID5EK64HD tag=V7XZZFCKCPL0 type=write size=67836 2024-04-07T16:38:19.217 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:19.224+0000 7f346a3868c0 1 > prepared operation key=IX3F0IUOKBX1 tag=AFRFMI6MVY5O type=del size=1119978 2024-04-07T16:38:19.223 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:19.232+0000 7f346a3868c0 1 < canceled operation key=1WGUPH7X5UA0 tag=0K65D2VO9RKP type=del size=834092 2024-04-07T16:38:19.229 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:19.236+0000 7f346a3868c0 1 > prepared operation key=ROFH30W3UCAX tag=I1XEBZ5UK0EY type=del size=2513697 2024-04-07T16:38:19.235 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:19.244+0000 7f346a3868c0 1 < completed delete operation key=IP8YW2A75TCW tag=AHQEHC43I180 type=del 2024-04-07T16:38:19.246 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:19.252+0000 7f346a3868c0 1 > prepared part key=_multipart_E84U0SX44NQ8.2~3QBABDH3UT1I.0 size=1048576 2024-04-07T16:38:19.251 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:19.260+0000 7f346a3868c0 1 > prepared part key=_multipart_E84U0SX44NQ8.2~3QBABDH3UT1I.1 size=403292 2024-04-07T16:38:19.251 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:19.260+0000 7f346a3868c0 1 > prepared multipart upload key=E84U0SX44NQ8 upload=3QBABDH3UT1I tag=ZU93L8NR5J0J type=write size=2500444 2024-04-07T16:38:19.257 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:19.264+0000 7f346a3868c0 1 < completed part key=_multipart_E84U0SX44NQ8.2~DVGKI0D9KP7E.0 size=1048576 2024-04-07T16:38:19.262 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:19.268+0000 7f346a3868c0 1 < completed part key=_multipart_E84U0SX44NQ8.2~DVGKI0D9KP7E.1 size=1048576 2024-04-07T16:38:19.267 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:19.272+0000 7f346a3868c0 1 < completed part key=_multipart_E84U0SX44NQ8.2~DVGKI0D9KP7E.2 size=987466 2024-04-07T16:38:19.277 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:19.284+0000 7f346a3868c0 1 < completed multipart upload key=E84U0SX44NQ8 upload=DVGKI0D9KP7E tag=87PEGI8SUGFC type=write size=4133194 2024-04-07T16:38:19.288 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:19.296+0000 7f346a3868c0 1 > prepared part key=_multipart_ROFH30W3UCAX.2~70CEKXQX1TO5.0 size=1048576 2024-04-07T16:38:19.293 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:19.300+0000 7f346a3868c0 1 > prepared part key=_multipart_ROFH30W3UCAX.2~70CEKXQX1TO5.1 size=425393 2024-04-07T16:38:19.293 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:19.300+0000 7f346a3868c0 1 > prepared multipart upload key=ROFH30W3UCAX upload=70CEKXQX1TO5 tag=PEOKHBC6S6S1 type=write size=2522545 2024-04-07T16:38:19.298 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:19.304+0000 7f346a3868c0 1 < completed part key=_multipart_XJLQZ7IS9U1T.2~WTWX4JVTA1IA.0 size=930467 2024-04-07T16:38:19.308 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:19.316+0000 7f346a3868c0 1 < completed multipart upload key=XJLQZ7IS9U1T upload=WTWX4JVTA1IA tag=UOZCVWZ5TE3O type=write size=1979043 2024-04-07T16:38:19.314 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:19.320+0000 7f346a3868c0 1 > prepared operation key=8F81G0P0YBFW tag=ITZN24ZLKMAB type=write size=665716 2024-04-07T16:38:19.319 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:19.328+0000 7f346a3868c0 1 < completed delete operation key=IX3F0IUOKBX1 tag=103JMEUSQUPL type=del 2024-04-07T16:38:19.325 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:19.332+0000 7f346a3868c0 1 > prepared operation key=QVOKSQAGHPA9 tag=7Y9MMVI1XD2Y type=del size=1832907 2024-04-07T16:38:19.331 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:19.340+0000 7f346a3868c0 1 < completed part key=_multipart_95YVKJ14JD8W.2~NX3Y7GYQYGK7.0 size=1048576 2024-04-07T16:38:19.336 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:19.344+0000 7f346a3868c0 1 < completed part key=_multipart_95YVKJ14JD8W.2~NX3Y7GYQYGK7.1 size=1048576 2024-04-07T16:38:19.341 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:19.348+0000 7f346a3868c0 1 < completed part key=_multipart_95YVKJ14JD8W.2~NX3Y7GYQYGK7.2 size=797981 2024-04-07T16:38:19.350 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:19.356+0000 7f346a3868c0 1 < completed multipart upload key=95YVKJ14JD8W upload=NX3Y7GYQYGK7 tag=9X6Y31NF5U25 type=write size=3943709 2024-04-07T16:38:19.355 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:19.364+0000 7f346a3868c0 1 > prepared operation key=HBLEXONKXJTL tag=60KNVKEKRP3V type=del size=2220345 2024-04-07T16:38:19.361 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:19.368+0000 7f346a3868c0 1 < completed delete operation key=C1BTVJNHIM0X tag=JVDYL08WSF9H type=del 2024-04-07T16:38:19.367 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:19.372+0000 7f346a3868c0 1 > prepared operation key=ROFH30W3UCAX tag=LEBWASODP21P type=del size=2422681 2024-04-07T16:38:19.372 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:19.380+0000 7f346a3868c0 1 < completed part key=_multipart_QS4HYZFFAJNF.2~5MAV8HQQSYZM.0 size=1048576 2024-04-07T16:38:19.378 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:19.384+0000 7f346a3868c0 1 < completed part key=_multipart_QS4HYZFFAJNF.2~5MAV8HQQSYZM.1 size=914156 2024-04-07T16:38:19.389 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:19.396+0000 7f346a3868c0 1 < completed multipart upload key=QS4HYZFFAJNF upload=5MAV8HQQSYZM tag=89TUJXY69EXC type=write size=3011308 2024-04-07T16:38:19.395 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:19.404+0000 7f346a3868c0 1 > prepared operation key=1VBSUWNI3V1T tag=0TS4KXASK724 type=write size=300320 2024-04-07T16:38:19.396 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:19.404+0000 7f346a3868c0 1 < canceled part key=_multipart_U3EID5EK64HD.2~00RLM2IXSWSN.0 size=1048576 2024-04-07T16:38:19.402 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:19.408+0000 7f346a3868c0 1 < completed part key=_multipart_U3EID5EK64HD.2~00RLM2IXSWSN.1 size=208205 2024-04-07T16:38:19.412 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:19.420+0000 7f346a3868c0 1 < canceled multipart upload key=U3EID5EK64HD upload=00RLM2IXSWSN tag=0D5C0J6V7IYZ type=write size=2305357 2024-04-07T16:38:19.423 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:19.432+0000 7f346a3868c0 1 > prepared part key=_multipart_QVOKSQAGHPA9.2~QGBPN9BUAONG.0 size=1048576 2024-04-07T16:38:19.428 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:19.436+0000 7f346a3868c0 1 > prepared part key=_multipart_QVOKSQAGHPA9.2~QGBPN9BUAONG.1 size=915 2024-04-07T16:38:19.429 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:19.436+0000 7f346a3868c0 1 > prepared multipart upload key=QVOKSQAGHPA9 upload=QGBPN9BUAONG tag=BHKODDL8GLV4 type=write size=2098067 2024-04-07T16:38:19.434 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:19.440+0000 7f346a3868c0 1 < completed part key=_multipart_IX3F0IUOKBX1.2~7KJ7K7IFFU08.0 size=1048576 2024-04-07T16:38:19.439 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:19.448+0000 7f346a3868c0 1 < completed part key=_multipart_IX3F0IUOKBX1.2~7KJ7K7IFFU08.1 size=1048576 2024-04-07T16:38:19.444 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:19.452+0000 7f346a3868c0 1 < completed part key=_multipart_IX3F0IUOKBX1.2~7KJ7K7IFFU08.2 size=342302 2024-04-07T16:38:19.455 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:19.460+0000 7f346a3868c0 1 < completed multipart upload key=IX3F0IUOKBX1 upload=7KJ7K7IFFU08 tag=EM93YHPHMW84 type=write size=3488030 2024-04-07T16:38:19.466 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:19.472+0000 7f346a3868c0 1 > prepared part key=_multipart_ROFH30W3UCAX.2~FDFT3ZY2D8HH.0 size=664033 2024-04-07T16:38:19.466 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:19.472+0000 7f346a3868c0 1 > prepared multipart upload key=ROFH30W3UCAX upload=FDFT3ZY2D8HH tag=3K9KU6NSMOMD type=write size=1712609 2024-04-07T16:38:19.478 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:19.484+0000 7f346a3868c0 1 < suggested update operation key=1VBSUWNI3V1T tag=QSBI1DU9CYPY type=del size=1975873 2024-04-07T16:38:19.489 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:19.496+0000 7f346a3868c0 1 > prepared part key=_multipart_HBLEXONKXJTL.2~FO57D8TB4QEO.0 size=1048576 2024-04-07T16:38:19.494 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:19.500+0000 7f346a3868c0 1 > prepared part key=_multipart_HBLEXONKXJTL.2~FO57D8TB4QEO.1 size=1048576 2024-04-07T16:38:19.499 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:19.508+0000 7f346a3868c0 1 > prepared part key=_multipart_HBLEXONKXJTL.2~FO57D8TB4QEO.2 size=27270 2024-04-07T16:38:19.499 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:19.508+0000 7f346a3868c0 1 > prepared multipart upload key=HBLEXONKXJTL upload=FO57D8TB4QEO tag=FCL7P937X44G type=write size=3172998 2024-04-07T16:38:19.505 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:19.512+0000 7f346a3868c0 1 < completed delete operation key=6XDS0A7L0IY8 tag=PA0S1ST5311X type=del 2024-04-07T16:38:19.511 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:19.520+0000 7f346a3868c0 1 > prepared operation key=1VBSUWNI3V1T tag=PNW1HEBX1TIQ type=write size=305211 2024-04-07T16:38:19.517 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:19.524+0000 7f346a3868c0 1 < completed part key=_multipart_YXMDZB23YTB3.2~J8SYAWMTUD4D.0 size=640000 2024-04-07T16:38:19.527 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:19.532+0000 7f346a3868c0 1 < completed multipart upload key=YXMDZB23YTB3 upload=J8SYAWMTUD4D tag=Q2VLHTXTT0QZ type=write size=1688576 2024-04-07T16:38:19.537 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:19.544+0000 7f346a3868c0 1 > prepared part key=_multipart_IX3F0IUOKBX1.2~NS11B1HLZTEA.0 size=48012 2024-04-07T16:38:19.538 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:19.544+0000 7f346a3868c0 1 > prepared multipart upload key=IX3F0IUOKBX1 upload=NS11B1HLZTEA tag=A9OS783WK70I type=write size=1096588 2024-04-07T16:38:19.544 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:19.552+0000 7f346a3868c0 1 < completed write operation key=N14BUXLJONHZ tag=IORHZ1FRSP95 type=write size=741388 2024-04-07T16:38:19.550 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:19.556+0000 7f346a3868c0 1 > prepared operation key=QTCIIPTXIRW2 tag=YXPMFYRQ06ME type=del size=3137599 2024-04-07T16:38:19.561 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:19.568+0000 7f346a3868c0 1 < suggested update operation key=UIW3DN4NUYPP tag=OPJGQ2HUNPFX type=del size=2122000 2024-04-07T16:38:19.572 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:19.580+0000 7f346a3868c0 1 > prepared part key=_multipart_CL4PP1RKZU0E.2~U95HA2OZJ8AW.0 size=1048576 2024-04-07T16:38:19.576 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:19.584+0000 7f346a3868c0 1 > prepared part key=_multipart_CL4PP1RKZU0E.2~U95HA2OZJ8AW.1 size=662183 2024-04-07T16:38:19.576 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:19.584+0000 7f346a3868c0 1 > prepared multipart upload key=CL4PP1RKZU0E upload=U95HA2OZJ8AW tag=JH9FB1S4DHAA type=write size=2759335 2024-04-07T16:38:19.582 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:19.588+0000 7f346a3868c0 1 < completed delete operation key=IX3F0IUOKBX1 tag=AFRFMI6MVY5O type=del 2024-04-07T16:38:19.588 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:19.596+0000 7f346a3868c0 1 > prepared operation key=E84U0SX44NQ8 tag=YVPNS5NUPULI type=del size=443250 2024-04-07T16:38:19.594 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:19.600+0000 7f346a3868c0 1 < completed delete operation key=ROFH30W3UCAX tag=I1XEBZ5UK0EY type=del 2024-04-07T16:38:19.604 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:19.612+0000 7f346a3868c0 1 > prepared part key=_multipart_8F81G0P0YBFW.2~TIC6LK3U4NS1.0 size=223517 2024-04-07T16:38:19.605 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:19.612+0000 7f346a3868c0 1 > prepared multipart upload key=8F81G0P0YBFW upload=TIC6LK3U4NS1 tag=L07PQ9HNMQNL type=write size=1272093 2024-04-07T16:38:19.610 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:19.616+0000 7f346a3868c0 1 < completed part key=_multipart_E84U0SX44NQ8.2~3QBABDH3UT1I.0 size=1048576 2024-04-07T16:38:19.610 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:19.616+0000 7f346a3868c0 1 < canceled part key=_multipart_E84U0SX44NQ8.2~3QBABDH3UT1I.1 size=403292 2024-04-07T16:38:19.620 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:19.628+0000 7f346a3868c0 1 < canceled multipart upload key=E84U0SX44NQ8 upload=3QBABDH3UT1I tag=ZU93L8NR5J0J type=write size=2500444 2024-04-07T16:38:19.627 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:19.632+0000 7f346a3868c0 1 > prepared operation key=UIW3DN4NUYPP tag=648B7RVGFQV4 type=write size=525257 2024-04-07T16:38:19.633 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:19.640+0000 7f346a3868c0 1 < completed part key=_multipart_ROFH30W3UCAX.2~70CEKXQX1TO5.0 size=1048576 2024-04-07T16:38:19.638 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:19.644+0000 7f346a3868c0 1 < completed part key=_multipart_ROFH30W3UCAX.2~70CEKXQX1TO5.1 size=425393 2024-04-07T16:38:19.648 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:19.656+0000 7f346a3868c0 1 < completed multipart upload key=ROFH30W3UCAX upload=70CEKXQX1TO5 tag=PEOKHBC6S6S1 type=write size=2522545 2024-04-07T16:38:19.654 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:19.660+0000 7f346a3868c0 1 > prepared operation key=1VKVQXSQMTU1 tag=765XBBPLZCQF type=del size=3003372 2024-04-07T16:38:19.661 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:19.668+0000 7f346a3868c0 1 < completed write operation key=8F81G0P0YBFW tag=ITZN24ZLKMAB type=write size=665716 2024-04-07T16:38:19.673 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:19.680+0000 7f346a3868c0 1 > prepared part key=_multipart_GBQSME7SY1AH.2~SMKVA05D8AET.0 size=347991 2024-04-07T16:38:19.673 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:19.680+0000 7f346a3868c0 1 > prepared multipart upload key=GBQSME7SY1AH upload=SMKVA05D8AET tag=EGQ4IMLSNP54 type=write size=1396567 2024-04-07T16:38:19.685 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:19.692+0000 7f346a3868c0 1 < suggested remove operation key=QVOKSQAGHPA9 tag=7Y9MMVI1XD2Y type=del 2024-04-07T16:38:19.697 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:19.704+0000 7f346a3868c0 1 > prepared part key=_multipart_3N0QJRYVO3FV.2~ZA47IG406CSM.0 size=1048576 2024-04-07T16:38:19.702 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:19.708+0000 7f346a3868c0 1 > prepared part key=_multipart_3N0QJRYVO3FV.2~ZA47IG406CSM.1 size=617721 2024-04-07T16:38:19.702 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:19.708+0000 7f346a3868c0 1 > prepared multipart upload key=3N0QJRYVO3FV upload=ZA47IG406CSM tag=7YLVZO6NBT6D type=write size=2714873 2024-04-07T16:38:19.712 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:19.720+0000 7f346a3868c0 1 < completed delete operation key=HBLEXONKXJTL tag=60KNVKEKRP3V type=del 2024-04-07T16:38:19.719 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:19.724+0000 7f346a3868c0 1 > prepared operation key=YOK8BAEOMPUR tag=CD5K1J3HTDFZ type=del size=2787633 2024-04-07T16:38:19.726 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:19.732+0000 7f346a3868c0 1 < completed delete operation key=ROFH30W3UCAX tag=LEBWASODP21P type=del 2024-04-07T16:38:19.731 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:19.740+0000 7f346a3868c0 1 > prepared operation key=NYET3212USVE tag=X30RV2T7VOZC type=del size=4097742 2024-04-07T16:38:19.739 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:19.748+0000 7f346a3868c0 -1 < failed to complete operation key=1VBSUWNI3V1T tag=0TS4KXASK724 type=write size=300320: (22) Invalid argument 2024-04-07T16:38:19.747 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:19.756+0000 7f346a3868c0 1 > prepared operation key=IP8YW2A75TCW tag=AKCD4IQKPN3T type=del size=2709982 2024-04-07T16:38:19.753 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:19.760+0000 7f346a3868c0 1 < completed part key=_multipart_QVOKSQAGHPA9.2~QGBPN9BUAONG.0 size=1048576 2024-04-07T16:38:19.758 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:19.764+0000 7f346a3868c0 1 < completed part key=_multipart_QVOKSQAGHPA9.2~QGBPN9BUAONG.1 size=915 2024-04-07T16:38:19.769 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:19.776+0000 7f346a3868c0 1 < completed multipart upload key=QVOKSQAGHPA9 upload=QGBPN9BUAONG tag=BHKODDL8GLV4 type=write size=2098067 2024-04-07T16:38:19.774 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:19.780+0000 7f346a3868c0 1 > prepared operation key=XJLQZ7IS9U1T tag=7U55WSLQDDNR type=del size=1319293 2024-04-07T16:38:19.780 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:19.788+0000 7f346a3868c0 1 < completed part key=_multipart_ROFH30W3UCAX.2~FDFT3ZY2D8HH.0 size=664033 2024-04-07T16:38:19.790 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:19.796+0000 7f346a3868c0 1 < completed multipart upload key=ROFH30W3UCAX upload=FDFT3ZY2D8HH tag=3K9KU6NSMOMD type=write size=1712609 2024-04-07T16:38:19.796 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:19.804+0000 7f346a3868c0 1 > prepared operation key=XJLQZ7IS9U1T tag=S6ZPZEFU1W3W type=del size=506733 2024-04-07T16:38:19.802 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:19.808+0000 7f346a3868c0 1 < completed part key=_multipart_HBLEXONKXJTL.2~FO57D8TB4QEO.0 size=1048576 2024-04-07T16:38:19.807 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:19.812+0000 7f346a3868c0 1 < completed part key=_multipart_HBLEXONKXJTL.2~FO57D8TB4QEO.1 size=1048576 2024-04-07T16:38:19.812 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:19.820+0000 7f346a3868c0 1 < completed part key=_multipart_HBLEXONKXJTL.2~FO57D8TB4QEO.2 size=27270 2024-04-07T16:38:19.822 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:19.828+0000 7f346a3868c0 1 < completed multipart upload key=HBLEXONKXJTL upload=FO57D8TB4QEO tag=FCL7P937X44G type=write size=3172998 2024-04-07T16:38:19.833 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:19.840+0000 7f346a3868c0 1 > prepared part key=_multipart_3N0QJRYVO3FV.2~OXZ1A10G8PCK.0 size=1048576 2024-04-07T16:38:19.837 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:19.844+0000 7f346a3868c0 1 > prepared part key=_multipart_3N0QJRYVO3FV.2~OXZ1A10G8PCK.1 size=1048576 2024-04-07T16:38:19.842 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:19.848+0000 7f346a3868c0 1 > prepared part key=_multipart_3N0QJRYVO3FV.2~OXZ1A10G8PCK.2 size=346049 2024-04-07T16:38:19.842 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:19.848+0000 7f346a3868c0 1 > prepared multipart upload key=3N0QJRYVO3FV upload=OXZ1A10G8PCK tag=OBVQDOIGKEHL type=write size=3491777 2024-04-07T16:38:19.848 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:19.856+0000 7f346a3868c0 1 < completed write operation key=1VBSUWNI3V1T tag=PNW1HEBX1TIQ type=write size=305211 2024-04-07T16:38:19.854 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:19.860+0000 7f346a3868c0 1 > prepared operation key=QS4HYZFFAJNF tag=82ZKK7K619BL type=del size=3383440 2024-04-07T16:38:19.860 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:19.868+0000 7f346a3868c0 1 < completed part key=_multipart_IX3F0IUOKBX1.2~NS11B1HLZTEA.0 size=48012 2024-04-07T16:38:19.870 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:19.876+0000 7f346a3868c0 1 < completed multipart upload key=IX3F0IUOKBX1 upload=NS11B1HLZTEA tag=A9OS783WK70I type=write size=1096588 2024-04-07T16:38:19.876 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:19.884+0000 7f346a3868c0 1 > prepared operation key=E84U0SX44NQ8 tag=019HR8YC4RAE type=del size=2005176 2024-04-07T16:38:19.888 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:19.896+0000 7f346a3868c0 1 < suggested remove operation key=QTCIIPTXIRW2 tag=YXPMFYRQ06ME type=del 2024-04-07T16:38:19.900 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:19.908+0000 7f346a3868c0 1 > prepared part key=_multipart_95YVKJ14JD8W.2~3FX6KX4B24KN.0 size=1048576 2024-04-07T16:38:19.905 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:19.912+0000 7f346a3868c0 1 > prepared part key=_multipart_95YVKJ14JD8W.2~3FX6KX4B24KN.1 size=916126 2024-04-07T16:38:19.905 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:19.912+0000 7f346a3868c0 1 > prepared multipart upload key=95YVKJ14JD8W upload=3FX6KX4B24KN tag=EJP9XFJJUCWK type=write size=3013278 2024-04-07T16:38:19.911 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:19.920+0000 7f346a3868c0 1 < completed part key=_multipart_CL4PP1RKZU0E.2~U95HA2OZJ8AW.0 size=1048576 2024-04-07T16:38:19.916 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:19.924+0000 7f346a3868c0 1 < completed part key=_multipart_CL4PP1RKZU0E.2~U95HA2OZJ8AW.1 size=662183 2024-04-07T16:38:19.927 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:19.936+0000 7f346a3868c0 1 < completed multipart upload key=CL4PP1RKZU0E upload=U95HA2OZJ8AW tag=JH9FB1S4DHAA type=write size=2759335 2024-04-07T16:38:19.933 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:19.940+0000 7f346a3868c0 1 > prepared operation key=3N0QJRYVO3FV tag=JGCO12I60ZQX type=del size=1941954 2024-04-07T16:38:19.939 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:19.948+0000 7f346a3868c0 1 < completed delete operation key=E84U0SX44NQ8 tag=YVPNS5NUPULI type=del 2024-04-07T16:38:19.945 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:19.952+0000 7f346a3868c0 1 > prepared operation key=GBQSME7SY1AH tag=TQ2DCOKZ8INX type=del size=778934 2024-04-07T16:38:19.951 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:19.960+0000 7f346a3868c0 1 < completed part key=_multipart_8F81G0P0YBFW.2~TIC6LK3U4NS1.0 size=223517 2024-04-07T16:38:19.962 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:19.968+0000 7f346a3868c0 1 < completed multipart upload key=8F81G0P0YBFW upload=TIC6LK3U4NS1 tag=L07PQ9HNMQNL type=write size=1272093 2024-04-07T16:38:19.973 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:19.980+0000 7f346a3868c0 1 > prepared part key=_multipart_1VBSUWNI3V1T.2~OXYE1NS4EM8B.0 size=959539 2024-04-07T16:38:19.973 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:19.980+0000 7f346a3868c0 1 > prepared multipart upload key=1VBSUWNI3V1T upload=OXYE1NS4EM8B tag=F3PWOLA5LZFU type=write size=2008115 2024-04-07T16:38:19.978 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:19.984+0000 7f346a3868c0 1 < canceled operation key=UIW3DN4NUYPP tag=648B7RVGFQV4 type=write size=525257 2024-04-07T16:38:19.989 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:19.996+0000 7f346a3868c0 1 > prepared part key=_multipart_IP8YW2A75TCW.2~3X35550JU1IN.0 size=357082 2024-04-07T16:38:19.989 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:19.996+0000 7f346a3868c0 1 > prepared multipart upload key=IP8YW2A75TCW upload=3X35550JU1IN tag=41MPYWNFYU5J type=write size=1405658 2024-04-07T16:38:19.995 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:20.004+0000 7f346a3868c0 1 < completed delete operation key=1VKVQXSQMTU1 tag=765XBBPLZCQF type=del 2024-04-07T16:38:20.006 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:20.012+0000 7f346a3868c0 1 > prepared part key=_multipart_EQR0FY86AT5X.2~C50T8XA34A9V.0 size=1048576 2024-04-07T16:38:20.010 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:20.016+0000 7f346a3868c0 1 > prepared part key=_multipart_EQR0FY86AT5X.2~C50T8XA34A9V.1 size=742059 2024-04-07T16:38:20.010 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:20.016+0000 7f346a3868c0 1 > prepared multipart upload key=EQR0FY86AT5X upload=C50T8XA34A9V tag=SB3MMV7O9KW7 type=write size=2839211 2024-04-07T16:38:20.015 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:20.024+0000 7f346a3868c0 1 < completed part key=_multipart_GBQSME7SY1AH.2~SMKVA05D8AET.0 size=347991 2024-04-07T16:38:20.024 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:20.032+0000 7f346a3868c0 1 < completed multipart upload key=GBQSME7SY1AH upload=SMKVA05D8AET tag=EGQ4IMLSNP54 type=write size=1396567 2024-04-07T16:38:20.034 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:20.040+0000 7f346a3868c0 1 > prepared part key=_multipart_EQR0FY86AT5X.2~8GE3VAWWY5FB.0 size=736407 2024-04-07T16:38:20.034 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:20.040+0000 7f346a3868c0 1 > prepared multipart upload key=EQR0FY86AT5X upload=8GE3VAWWY5FB tag=EQGVZB0FSF32 type=write size=1784983 2024-04-07T16:38:20.039 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:20.048+0000 7f346a3868c0 1 < completed part key=_multipart_3N0QJRYVO3FV.2~ZA47IG406CSM.0 size=1048576 2024-04-07T16:38:20.046 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:20.052+0000 7f346a3868c0 1 < completed part key=_multipart_3N0QJRYVO3FV.2~ZA47IG406CSM.1 size=617721 2024-04-07T16:38:20.056 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:20.064+0000 7f346a3868c0 1 < completed multipart upload key=3N0QJRYVO3FV upload=ZA47IG406CSM tag=7YLVZO6NBT6D type=write size=2714873 2024-04-07T16:38:20.067 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:20.076+0000 7f346a3868c0 1 > prepared part key=_multipart_HBLEXONKXJTL.2~JZH42EK282SU.0 size=43001 2024-04-07T16:38:20.067 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:20.076+0000 7f346a3868c0 1 > prepared multipart upload key=HBLEXONKXJTL upload=JZH42EK282SU tag=FXH8LAAIZ5GJ type=write size=1091577 2024-04-07T16:38:20.073 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:20.080+0000 7f346a3868c0 1 < completed delete operation key=YOK8BAEOMPUR tag=CD5K1J3HTDFZ type=del 2024-04-07T16:38:20.085 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:20.092+0000 7f346a3868c0 1 > prepared part key=_multipart_GZXHYZO5SMUP.2~JPW6QZ2BME2O.0 size=1048576 2024-04-07T16:38:20.093 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:20.100+0000 7f346a3868c0 1 > prepared part key=_multipart_GZXHYZO5SMUP.2~JPW6QZ2BME2O.1 size=1048576 2024-04-07T16:38:20.098 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:20.104+0000 7f346a3868c0 1 > prepared part key=_multipart_GZXHYZO5SMUP.2~JPW6QZ2BME2O.2 size=865810 2024-04-07T16:38:20.098 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:20.104+0000 7f346a3868c0 1 > prepared multipart upload key=GZXHYZO5SMUP upload=JPW6QZ2BME2O tag=MI6W4SIV8HTP type=write size=4011538 2024-04-07T16:38:20.112 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:20.120+0000 7f346a3868c0 1 < suggested remove operation key=NYET3212USVE tag=X30RV2T7VOZC type=del 2024-04-07T16:38:20.123 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:20.128+0000 7f346a3868c0 1 > prepared part key=_multipart_QVOKSQAGHPA9.2~RSOGM9GK1DFF.0 size=1048576 2024-04-07T16:38:20.130 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:20.136+0000 7f346a3868c0 1 > prepared part key=_multipart_QVOKSQAGHPA9.2~RSOGM9GK1DFF.1 size=307719 2024-04-07T16:38:20.130 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:20.136+0000 7f346a3868c0 1 > prepared multipart upload key=QVOKSQAGHPA9 upload=RSOGM9GK1DFF tag=6QR7UGT5ATFQ type=write size=2404871 2024-04-07T16:38:20.139 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:20.148+0000 7f346a3868c0 1 < completed delete operation key=IP8YW2A75TCW tag=AKCD4IQKPN3T type=del 2024-04-07T16:38:20.148 INFO:tasks.workunit.client.0.smithi100.stderr:2024-04-07T16:38:20.156+0000 7f346a3868c0 1 > prepared operation key=QVOKSQAGHPA9 tag=JBLUIKHWQCEH type=del size=2365648 2024-04-07T16:38:20.150 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] cls_rgw_stats.simulate (38704 ms) 2024-04-07T16:38:20.150 INFO:tasks.workunit.client.0.smithi100.stdout:[----------] 1 test from cls_rgw_stats (38704 ms total) 2024-04-07T16:38:20.150 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T16:38:20.150 INFO:tasks.workunit.client.0.smithi100.stdout:[----------] Global test environment tear-down 2024-04-07T16:38:20.191 INFO:tasks.workunit.client.0.smithi100.stdout:[==========] 1 test from 1 test suite ran. (40611 ms total) 2024-04-07T16:38:20.192 INFO:tasks.workunit.client.0.smithi100.stdout:[ PASSED ] 1 test. 2024-04-07T16:38:20.200 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-07T16:38:20.200 DEBUG:teuthology.orchestra.run.smithi100:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-07T16:38:20.254 INFO:tasks.workunit:Running workunit cls/test_cls_sdk.sh... 2024-04-07T16:38:20.254 DEBUG:teuthology.orchestra.run.smithi100:workunit test cls/test_cls_sdk.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=reef TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_sdk.sh 2024-04-07T16:38:20.314 INFO:tasks.workunit.client.0.smithi100.stdout:Running main() from gmock_main.cc 2024-04-07T16:38:20.314 INFO:tasks.workunit.client.0.smithi100.stdout:[==========] Running 2 tests from 1 test suite. 2024-04-07T16:38:20.314 INFO:tasks.workunit.client.0.smithi100.stdout:[----------] Global test environment set-up. 2024-04-07T16:38:20.314 INFO:tasks.workunit.client.0.smithi100.stdout:[----------] 2 tests from ClsSDK 2024-04-07T16:38:20.314 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] ClsSDK.TestSDKCoverageWrite 2024-04-07T16:38:20.441 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:38:20 smithi100 bash[74942]: cluster 2024-04-07T16:38:19.075353+0000 mgr.y (mgr.25175) 689 : cluster [DBG] pgmap v1324: 137 pgs: 137 active+clean; 583 KiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 502 KiB/s rd, 174 KiB/s wr, 885 op/s 2024-04-07T16:38:20.441 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:38:20 smithi100 bash[76202]: cluster 2024-04-07T16:38:19.075353+0000 mgr.y (mgr.25175) 689 : cluster [DBG] pgmap v1324: 137 pgs: 137 active+clean; 583 KiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 502 KiB/s rd, 174 KiB/s wr, 885 op/s 2024-04-07T16:38:20.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:38:20 smithi144 bash[66654]: cluster 2024-04-07T16:38:19.075353+0000 mgr.y (mgr.25175) 689 : cluster [DBG] pgmap v1324: 137 pgs: 137 active+clean; 583 KiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 502 KiB/s rd, 174 KiB/s wr, 885 op/s 2024-04-07T16:38:21.441 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:38:21 smithi100 bash[74942]: cluster 2024-04-07T16:38:20.194326+0000 mon.a (mon.0) 2024 : cluster [DBG] osdmap e1061: 8 total, 8 up, 8 in 2024-04-07T16:38:21.441 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:38:21 smithi100 bash[76202]: cluster 2024-04-07T16:38:20.194326+0000 mon.a (mon.0) 2024 : cluster [DBG] osdmap e1061: 8 total, 8 up, 8 in 2024-04-07T16:38:21.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:38:21 smithi144 bash[66654]: cluster 2024-04-07T16:38:20.194326+0000 mon.a (mon.0) 2024 : cluster [DBG] osdmap e1061: 8 total, 8 up, 8 in 2024-04-07T16:38:22.607 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:38:22 smithi144 bash[95233]: ts=2024-04-07T16:38:22.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:38:22.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:38:22 smithi144 bash[66654]: cluster 2024-04-07T16:38:21.076030+0000 mgr.y (mgr.25175) 690 : cluster [DBG] pgmap v1326: 105 pgs: 105 active+clean; 583 KiB data, 2.8 GiB used, 712 GiB / 715 GiB avail 2024-04-07T16:38:22.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:38:22 smithi144 bash[66654]: cluster 2024-04-07T16:38:21.204263+0000 mon.a (mon.0) 2025 : cluster [DBG] osdmap e1062: 8 total, 8 up, 8 in 2024-04-07T16:38:22.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:38:22 smithi144 bash[66654]: audit 2024-04-07T16:38:21.210476+0000 mon.c (mon.2) 58 : audit [INF] from='client.? 172.21.15.100:0/404578159' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111509-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:38:22.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:38:22 smithi144 bash[66654]: audit 2024-04-07T16:38:21.217203+0000 mon.a (mon.0) 2026 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111509-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:38:22.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:38:22 smithi100 bash[74942]: cluster 2024-04-07T16:38:21.076030+0000 mgr.y (mgr.25175) 690 : cluster [DBG] pgmap v1326: 105 pgs: 105 active+clean; 583 KiB data, 2.8 GiB used, 712 GiB / 715 GiB avail 2024-04-07T16:38:22.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:38:22 smithi100 bash[74942]: cluster 2024-04-07T16:38:21.204263+0000 mon.a (mon.0) 2025 : cluster [DBG] osdmap e1062: 8 total, 8 up, 8 in 2024-04-07T16:38:22.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:38:22 smithi100 bash[74942]: audit 2024-04-07T16:38:21.210476+0000 mon.c (mon.2) 58 : audit [INF] from='client.? 172.21.15.100:0/404578159' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111509-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:38:22.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:38:22 smithi100 bash[74942]: audit 2024-04-07T16:38:21.217203+0000 mon.a (mon.0) 2026 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111509-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:38:22.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:38:22 smithi100 bash[76202]: cluster 2024-04-07T16:38:21.076030+0000 mgr.y (mgr.25175) 690 : cluster [DBG] pgmap v1326: 105 pgs: 105 active+clean; 583 KiB data, 2.8 GiB used, 712 GiB / 715 GiB avail 2024-04-07T16:38:22.692 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:38:22 smithi100 bash[76202]: cluster 2024-04-07T16:38:21.204263+0000 mon.a (mon.0) 2025 : cluster [DBG] osdmap e1062: 8 total, 8 up, 8 in 2024-04-07T16:38:22.692 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:38:22 smithi100 bash[76202]: audit 2024-04-07T16:38:21.210476+0000 mon.c (mon.2) 58 : audit [INF] from='client.? 172.21.15.100:0/404578159' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111509-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:38:22.692 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:38:22 smithi100 bash[76202]: audit 2024-04-07T16:38:21.217203+0000 mon.a (mon.0) 2026 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111509-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:38:23.219 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] ClsSDK.TestSDKCoverageWrite (2905 ms) 2024-04-07T16:38:23.219 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] ClsSDK.TestSDKCoverageReplay 2024-04-07T16:38:23.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:38:23 smithi144 bash[66654]: audit 2024-04-07T16:38:22.197518+0000 mon.a (mon.0) 2027 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111509-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:38:23.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:38:23 smithi144 bash[66654]: cluster 2024-04-07T16:38:22.206655+0000 mon.a (mon.0) 2028 : cluster [DBG] osdmap e1063: 8 total, 8 up, 8 in 2024-04-07T16:38:23.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:38:23 smithi100 bash[74942]: audit 2024-04-07T16:38:22.197518+0000 mon.a (mon.0) 2027 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111509-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:38:23.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:38:23 smithi100 bash[74942]: cluster 2024-04-07T16:38:22.206655+0000 mon.a (mon.0) 2028 : cluster [DBG] osdmap e1063: 8 total, 8 up, 8 in 2024-04-07T16:38:23.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:38:23 smithi100 bash[76202]: audit 2024-04-07T16:38:22.197518+0000 mon.a (mon.0) 2027 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111509-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:38:23.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:38:23 smithi100 bash[76202]: cluster 2024-04-07T16:38:22.206655+0000 mon.a (mon.0) 2028 : cluster [DBG] osdmap e1063: 8 total, 8 up, 8 in 2024-04-07T16:38:24.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:38:24 smithi144 bash[66654]: cluster 2024-04-07T16:38:23.076936+0000 mgr.y (mgr.25175) 691 : cluster [DBG] pgmap v1329: 137 pgs: 19 creating+peering, 9 unknown, 109 active+clean; 583 KiB data, 2.9 GiB used, 712 GiB / 715 GiB avail 2024-04-07T16:38:24.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:38:24 smithi144 bash[66654]: cluster 2024-04-07T16:38:23.226810+0000 mon.a (mon.0) 2029 : cluster [DBG] osdmap e1064: 8 total, 8 up, 8 in 2024-04-07T16:38:24.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:38:24 smithi100 bash[74942]: cluster 2024-04-07T16:38:23.076936+0000 mgr.y (mgr.25175) 691 : cluster [DBG] pgmap v1329: 137 pgs: 19 creating+peering, 9 unknown, 109 active+clean; 583 KiB data, 2.9 GiB used, 712 GiB / 715 GiB avail 2024-04-07T16:38:24.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:38:24 smithi100 bash[74942]: cluster 2024-04-07T16:38:23.226810+0000 mon.a (mon.0) 2029 : cluster [DBG] osdmap e1064: 8 total, 8 up, 8 in 2024-04-07T16:38:24.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:38:24 smithi100 bash[76202]: cluster 2024-04-07T16:38:23.076936+0000 mgr.y (mgr.25175) 691 : cluster [DBG] pgmap v1329: 137 pgs: 19 creating+peering, 9 unknown, 109 active+clean; 583 KiB data, 2.9 GiB used, 712 GiB / 715 GiB avail 2024-04-07T16:38:24.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:38:24 smithi100 bash[76202]: cluster 2024-04-07T16:38:23.226810+0000 mon.a (mon.0) 2029 : cluster [DBG] osdmap e1064: 8 total, 8 up, 8 in 2024-04-07T16:38:25.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:38:25 smithi144 bash[66654]: cluster 2024-04-07T16:38:24.233301+0000 mon.a (mon.0) 2030 : cluster [DBG] osdmap e1065: 8 total, 8 up, 8 in 2024-04-07T16:38:25.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:38:25 smithi144 bash[66654]: audit 2024-04-07T16:38:24.235451+0000 mon.c (mon.2) 59 : audit [INF] from='client.? 172.21.15.100:0/3454415801' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111509-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:38:25.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:38:25 smithi144 bash[66654]: audit 2024-04-07T16:38:24.251912+0000 mon.a (mon.0) 2031 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111509-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:38:25.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:38:25 smithi144 bash[66654]: audit 2024-04-07T16:38:25.226762+0000 mon.a (mon.0) 2032 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111509-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:38:25.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:38:25 smithi144 bash[66654]: cluster 2024-04-07T16:38:25.235271+0000 mon.a (mon.0) 2033 : cluster [DBG] osdmap e1066: 8 total, 8 up, 8 in 2024-04-07T16:38:25.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:38:25 smithi100 bash[74942]: cluster 2024-04-07T16:38:24.233301+0000 mon.a (mon.0) 2030 : cluster [DBG] osdmap e1065: 8 total, 8 up, 8 in 2024-04-07T16:38:25.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:38:25 smithi100 bash[74942]: audit 2024-04-07T16:38:24.235451+0000 mon.c (mon.2) 59 : audit [INF] from='client.? 172.21.15.100:0/3454415801' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111509-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:38:25.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:38:25 smithi100 bash[74942]: audit 2024-04-07T16:38:24.251912+0000 mon.a (mon.0) 2031 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111509-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:38:25.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:38:25 smithi100 bash[74942]: audit 2024-04-07T16:38:25.226762+0000 mon.a (mon.0) 2032 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111509-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:38:25.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:38:25 smithi100 bash[74942]: cluster 2024-04-07T16:38:25.235271+0000 mon.a (mon.0) 2033 : cluster [DBG] osdmap e1066: 8 total, 8 up, 8 in 2024-04-07T16:38:25.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:38:25 smithi100 bash[76202]: cluster 2024-04-07T16:38:24.233301+0000 mon.a (mon.0) 2030 : cluster [DBG] osdmap e1065: 8 total, 8 up, 8 in 2024-04-07T16:38:25.692 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:38:25 smithi100 bash[76202]: audit 2024-04-07T16:38:24.235451+0000 mon.c (mon.2) 59 : audit [INF] from='client.? 172.21.15.100:0/3454415801' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111509-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:38:25.692 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:38:25 smithi100 bash[76202]: audit 2024-04-07T16:38:24.251912+0000 mon.a (mon.0) 2031 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111509-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T16:38:25.692 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:38:25 smithi100 bash[76202]: audit 2024-04-07T16:38:25.226762+0000 mon.a (mon.0) 2032 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-111509-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T16:38:25.692 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:38:25 smithi100 bash[76202]: cluster 2024-04-07T16:38:25.235271+0000 mon.a (mon.0) 2033 : cluster [DBG] osdmap e1066: 8 total, 8 up, 8 in 2024-04-07T16:38:26.256 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] ClsSDK.TestSDKCoverageReplay (3037 ms) 2024-04-07T16:38:26.256 INFO:tasks.workunit.client.0.smithi100.stdout:[----------] 2 tests from ClsSDK (5942 ms total) 2024-04-07T16:38:26.257 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T16:38:26.257 INFO:tasks.workunit.client.0.smithi100.stdout:[----------] Global test environment tear-down 2024-04-07T16:38:26.257 INFO:tasks.workunit.client.0.smithi100.stdout:[==========] 2 tests from 1 test suite ran. (5942 ms total) 2024-04-07T16:38:26.257 INFO:tasks.workunit.client.0.smithi100.stdout:[ PASSED ] 2 tests. 2024-04-07T16:38:26.261 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-07T16:38:26.261 DEBUG:teuthology.orchestra.run.smithi100:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-07T16:38:26.317 INFO:tasks.workunit:Stopping ['cls'] on client.0... 2024-04-07T16:38:26.318 DEBUG:teuthology.orchestra.run.smithi100:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-04-07T16:38:26.580 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:38:26 smithi100 bash[74942]: cluster 2024-04-07T16:38:25.077912+0000 mgr.y (mgr.25175) 692 : cluster [DBG] pgmap v1332: 137 pgs: 5 creating+peering, 27 unknown, 105 active+clean; 583 KiB data, 2.9 GiB used, 712 GiB / 715 GiB avail 2024-04-07T16:38:26.580 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:38:26 smithi100 bash[74942]: cluster 2024-04-07T16:38:25.242534+0000 mon.a (mon.0) 2034 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T16:38:26.580 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:38:26 smithi100 bash[76202]: cluster 2024-04-07T16:38:25.077912+0000 mgr.y (mgr.25175) 692 : cluster [DBG] pgmap v1332: 137 pgs: 5 creating+peering, 27 unknown, 105 active+clean; 583 KiB data, 2.9 GiB used, 712 GiB / 715 GiB avail 2024-04-07T16:38:26.580 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:38:26 smithi100 bash[76202]: cluster 2024-04-07T16:38:25.242534+0000 mon.a (mon.0) 2034 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T16:38:26.581 DEBUG:teuthology.parallel:result is None 2024-04-07T16:38:26.582 DEBUG:teuthology.orchestra.run.smithi100:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-04-07T16:38:26.594 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-04-07T16:38:26.594 DEBUG:teuthology.orchestra.run.smithi100:> rmdir -- /home/ubuntu/cephtest/mnt.0 2024-04-07T16:38:26.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:38:26 smithi144 bash[66654]: cluster 2024-04-07T16:38:25.077912+0000 mgr.y (mgr.25175) 692 : cluster [DBG] pgmap v1332: 137 pgs: 5 creating+peering, 27 unknown, 105 active+clean; 583 KiB data, 2.9 GiB used, 712 GiB / 715 GiB avail 2024-04-07T16:38:26.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:38:26 smithi144 bash[66654]: cluster 2024-04-07T16:38:25.242534+0000 mon.a (mon.0) 2034 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T16:38:26.639 INFO:tasks.workunit:Deleted artificial mount point /home/ubuntu/cephtest/mnt.0/client.0 2024-04-07T16:38:26.639 INFO:teuthology.task.full_sequential:In full_sequential, running task print... 2024-04-07T16:38:26.639 INFO:teuthology.task.print:**** done end rados_api.yaml 2024-04-07T16:38:26.639 INFO:teuthology.task.full_sequential:In full_sequential, running task print... 2024-04-07T16:38:26.640 INFO:teuthology.task.print:**** done start rados_loadgenbig.yaml 2024-04-07T16:38:26.640 INFO:teuthology.task.full_sequential:In full_sequential, running task workunit... 2024-04-07T16:38:26.640 INFO:tasks.workunit:Pulling workunits from ref reef 2024-04-07T16:38:26.640 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-04-07T16:38:26.640 DEBUG:teuthology.orchestra.run.smithi100:> stat -- /home/ubuntu/cephtest/mnt.0 2024-04-07T16:38:26.687 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-07T16:38:26.687 INFO:teuthology.orchestra.run.smithi100.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-04-07T16:38:26.688 DEBUG:teuthology.orchestra.run.smithi100:> mkdir -- /home/ubuntu/cephtest/mnt.0 2024-04-07T16:38:26.735 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2024-04-07T16:38:26.736 DEBUG:teuthology.orchestra.run.smithi100:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2024-04-07T16:38:26.786 INFO:tasks.workunit:timeout=3h 2024-04-07T16:38:26.786 INFO:tasks.workunit:cleanup=True 2024-04-07T16:38:26.786 DEBUG:teuthology.orchestra.run.smithi100:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone --depth 1 --branch reef https://github.com/ceph/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 2024-04-07T16:38:26.834 INFO:tasks.workunit.client.0.smithi100.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-04-07T16:38:27.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:38:27 smithi144 bash[66654]: cluster 2024-04-07T16:38:26.261797+0000 mon.a (mon.0) 2035 : cluster [DBG] osdmap e1067: 8 total, 8 up, 8 in 2024-04-07T16:38:27.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:38:27 smithi100 bash[74942]: cluster 2024-04-07T16:38:26.261797+0000 mon.a (mon.0) 2035 : cluster [DBG] osdmap e1067: 8 total, 8 up, 8 in 2024-04-07T16:38:27.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:38:27 smithi100 bash[76202]: cluster 2024-04-07T16:38:26.261797+0000 mon.a (mon.0) 2035 : cluster [DBG] osdmap e1067: 8 total, 8 up, 8 in 2024-04-07T16:38:28.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:38:28 smithi144 bash[66654]: cluster 2024-04-07T16:38:27.078734+0000 mgr.y (mgr.25175) 693 : cluster [DBG] pgmap v1335: 105 pgs: 105 active+clean; 583 KiB data, 2.9 GiB used, 712 GiB / 715 GiB avail 2024-04-07T16:38:28.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:38:28 smithi144 bash[66654]: cluster 2024-04-07T16:38:27.254972+0000 mon.a (mon.0) 2036 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T16:38:28.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:38:28 smithi144 bash[66654]: cluster 2024-04-07T16:38:27.255019+0000 mon.a (mon.0) 2037 : cluster [INF] Cluster is now healthy 2024-04-07T16:38:28.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:38:28 smithi100 bash[74942]: cluster 2024-04-07T16:38:27.078734+0000 mgr.y (mgr.25175) 693 : cluster [DBG] pgmap v1335: 105 pgs: 105 active+clean; 583 KiB data, 2.9 GiB used, 712 GiB / 715 GiB avail 2024-04-07T16:38:28.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:38:28 smithi100 bash[74942]: cluster 2024-04-07T16:38:27.254972+0000 mon.a (mon.0) 2036 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T16:38:28.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:38:28 smithi100 bash[74942]: cluster 2024-04-07T16:38:27.255019+0000 mon.a (mon.0) 2037 : cluster [INF] Cluster is now healthy 2024-04-07T16:38:28.692 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:38:28 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:38:28] "GET /metrics HTTP/1.1" 200 36375 "" "Prometheus/2.43.0" 2024-04-07T16:38:28.692 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:38:28 smithi100 bash[76202]: cluster 2024-04-07T16:38:27.078734+0000 mgr.y (mgr.25175) 693 : cluster [DBG] pgmap v1335: 105 pgs: 105 active+clean; 583 KiB data, 2.9 GiB used, 712 GiB / 715 GiB avail 2024-04-07T16:38:28.692 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:38:28 smithi100 bash[76202]: cluster 2024-04-07T16:38:27.254972+0000 mon.a (mon.0) 2036 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T16:38:28.692 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:38:28 smithi100 bash[76202]: cluster 2024-04-07T16:38:27.255019+0000 mon.a (mon.0) 2037 : cluster [INF] Cluster is now healthy 2024-04-07T16:38:30.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:38:30 smithi144 bash[66654]: cluster 2024-04-07T16:38:29.080067+0000 mgr.y (mgr.25175) 694 : cluster [DBG] pgmap v1336: 105 pgs: 105 active+clean; 583 KiB data, 2.9 GiB used, 712 GiB / 715 GiB avail 2024-04-07T16:38:30.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:38:30 smithi100 bash[74942]: cluster 2024-04-07T16:38:29.080067+0000 mgr.y (mgr.25175) 694 : cluster [DBG] pgmap v1336: 105 pgs: 105 active+clean; 583 KiB data, 2.9 GiB used, 712 GiB / 715 GiB avail 2024-04-07T16:38:30.692 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:38:30 smithi100 bash[76202]: cluster 2024-04-07T16:38:29.080067+0000 mgr.y (mgr.25175) 694 : cluster [DBG] pgmap v1336: 105 pgs: 105 active+clean; 583 KiB data, 2.9 GiB used, 712 GiB / 715 GiB avail 2024-04-07T16:38:32.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:38:32 smithi144 bash[66654]: cluster 2024-04-07T16:38:31.081116+0000 mgr.y (mgr.25175) 695 : cluster [DBG] pgmap v1337: 105 pgs: 105 active+clean; 583 KiB data, 2.9 GiB used, 712 GiB / 715 GiB avail 2024-04-07T16:38:32.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:38:32 smithi144 bash[66654]: audit 2024-04-07T16:38:31.753890+0000 mon.a (mon.0) 2038 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:38:32.607 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:38:32 smithi144 bash[95233]: ts=2024-04-07T16:38:32.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:38:32.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:38:32 smithi100 bash[74942]: cluster 2024-04-07T16:38:31.081116+0000 mgr.y (mgr.25175) 695 : cluster [DBG] pgmap v1337: 105 pgs: 105 active+clean; 583 KiB data, 2.9 GiB used, 712 GiB / 715 GiB avail 2024-04-07T16:38:32.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:38:32 smithi100 bash[74942]: audit 2024-04-07T16:38:31.753890+0000 mon.a (mon.0) 2038 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:38:32.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:38:32 smithi100 bash[76202]: cluster 2024-04-07T16:38:31.081116+0000 mgr.y (mgr.25175) 695 : cluster [DBG] pgmap v1337: 105 pgs: 105 active+clean; 583 KiB data, 2.9 GiB used, 712 GiB / 715 GiB avail 2024-04-07T16:38:32.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:38:32 smithi100 bash[76202]: audit 2024-04-07T16:38:31.753890+0000 mon.a (mon.0) 2038 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:38:33.700 DEBUG:teuthology.orchestra.run.smithi100:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-04-07T16:38:33.748 INFO:tasks.workunit.client.0.smithi100.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-04-07T16:38:33.751 INFO:tasks.workunit.client.0.smithi100.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-07T16:38:33.751 INFO:tasks.workunit.client.0.smithi100.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-04-07T16:38:33.807 INFO:tasks.workunit.client.0.smithi100.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-04-07T16:38:33.863 INFO:tasks.workunit.client.0.smithi100.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-04-07T16:38:33.904 INFO:tasks.workunit.client.0.smithi100.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-07T16:38:33.906 INFO:tasks.workunit.client.0.smithi100.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-07T16:38:33.906 INFO:tasks.workunit.client.0.smithi100.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-04-07T16:38:33.951 INFO:tasks.workunit.client.0.smithi100.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-07T16:38:33.954 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-07T16:38:33.954 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-04-07T16:38:33.999 INFO:tasks.workunit:Running workunits matching rados/load-gen-big.sh on client.0... 2024-04-07T16:38:34.000 INFO:tasks.workunit:Running workunit rados/load-gen-big.sh... 2024-04-07T16:38:34.000 DEBUG:teuthology.orchestra.run.smithi100:workunit test rados/load-gen-big.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=reef TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/load-gen-big.sh 2024-04-07T16:38:34.093 INFO:tasks.workunit.client.0.smithi100.stdout:run length 1200 seconds 2024-04-07T16:38:34.093 INFO:tasks.workunit.client.0.smithi100.stdout:preparing 10240 objects 2024-04-07T16:38:34.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:38:34 smithi144 bash[66654]: cluster 2024-04-07T16:38:33.082287+0000 mgr.y (mgr.25175) 696 : cluster [DBG] pgmap v1338: 105 pgs: 105 active+clean; 583 KiB data, 2.9 GiB used, 712 GiB / 715 GiB avail 2024-04-07T16:38:34.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:38:34 smithi100 bash[74942]: cluster 2024-04-07T16:38:33.082287+0000 mgr.y (mgr.25175) 696 : cluster [DBG] pgmap v1338: 105 pgs: 105 active+clean; 583 KiB data, 2.9 GiB used, 712 GiB / 715 GiB avail 2024-04-07T16:38:34.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:38:34 smithi100 bash[76202]: cluster 2024-04-07T16:38:33.082287+0000 mgr.y (mgr.25175) 696 : cluster [DBG] pgmap v1338: 105 pgs: 105 active+clean; 583 KiB data, 2.9 GiB used, 712 GiB / 715 GiB avail 2024-04-07T16:38:36.464 INFO:tasks.workunit.client.0.smithi100.stdout:load-gen will run 1200 seconds 2024-04-07T16:38:36.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:38:36 smithi144 bash[66654]: cluster 2024-04-07T16:38:35.082945+0000 mgr.y (mgr.25175) 697 : cluster [DBG] pgmap v1339: 105 pgs: 105 active+clean; 16 GiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 130 KiB/s wr, 130 op/s 2024-04-07T16:38:36.663 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:38:36 smithi100 bash[74942]: cluster 2024-04-07T16:38:35.082945+0000 mgr.y (mgr.25175) 697 : cluster [DBG] pgmap v1339: 105 pgs: 105 active+clean; 16 GiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 130 KiB/s wr, 130 op/s 2024-04-07T16:38:36.941 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:38:36 smithi100 bash[76202]: cluster 2024-04-07T16:38:35.082945+0000 mgr.y (mgr.25175) 697 : cluster [DBG] pgmap v1339: 105 pgs: 105 active+clean; 16 GiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 130 KiB/s wr, 130 op/s 2024-04-07T16:38:37.464 INFO:tasks.workunit.client.0.smithi100.stdout: 1: throughput=0MB/sec pending data=0 2024-04-07T16:38:37.464 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-X-sfNyKW_rtiMPw off=1057193 len=1526835 2024-04-07T16:38:37.470 INFO:tasks.workunit.client.0.smithi100.stdout:op 0 completed, throughput=1.45MB/sec 2024-04-07T16:38:37.470 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Nz_DJZQcKV5vqfl off=826406 len=176231 2024-04-07T16:38:37.476 INFO:tasks.workunit.client.0.smithi100.stdout:op 1 completed, throughput=1.61MB/sec 2024-04-07T16:38:37.476 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-NZGz0rMY-fVeA1k off=13578211 len=1191138 2024-04-07T16:38:37.487 INFO:tasks.workunit.client.0.smithi100.stdout:op 2 completed, throughput=2.7MB/sec 2024-04-07T16:38:37.487 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-AWDSZNMdNrO_8RN off=8498689 len=815915 2024-04-07T16:38:37.489 INFO:tasks.workunit.client.0.smithi100.stdout:op 3 completed, throughput=3.45MB/sec 2024-04-07T16:38:37.489 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-lyE5yXrph8NzhIU off=17214737 len=427509 2024-04-07T16:38:37.490 INFO:tasks.workunit.client.0.smithi100.stdout:op 4 completed, throughput=3.85MB/sec 2024-04-07T16:38:37.490 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-kdzdhGL_3lYlk-e off=9750334 len=741546 2024-04-07T16:38:37.493 INFO:tasks.workunit.client.0.smithi100.stdout:op 5 completed, throughput=4.52MB/sec 2024-04-07T16:38:37.493 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-2sxNKXDLJlTQzrU off=792020 len=782125 2024-04-07T16:38:37.496 INFO:tasks.workunit.client.0.smithi100.stdout:op 6 completed, throughput=5.23MB/sec 2024-04-07T16:38:38.497 INFO:tasks.workunit.client.0.smithi100.stdout: 2: throughput=2.66MB/sec pending data=0 2024-04-07T16:38:38.497 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-oTUWwUx--LDZdv8 off=3746535 len=1444481 2024-04-07T16:38:38.503 INFO:tasks.workunit.client.0.smithi100.stdout:op 7 completed, throughput=3.32MB/sec 2024-04-07T16:38:38.503 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-CrqJNDPPqGdIef_ off=1799902 len=1860523 2024-04-07T16:38:38.521 INFO:tasks.workunit.client.0.smithi100.stdout:op 8 completed, throughput=4.16MB/sec 2024-04-07T16:38:38.521 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-l4_xNwy4axBcLM3 off=3737302 len=1053453 2024-04-07T16:38:38.524 INFO:tasks.workunit.client.0.smithi100.stdout:op 9 completed, throughput=4.64MB/sec 2024-04-07T16:38:38.524 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-PSpDOSZdAo5Wt4q off=2295664 len=813721 2024-04-07T16:38:38.527 INFO:tasks.workunit.client.0.smithi100.stdout:op 10 completed, throughput=5.01MB/sec 2024-04-07T16:38:38.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:38:38 smithi144 bash[66654]: cluster 2024-04-07T16:38:37.083604+0000 mgr.y (mgr.25175) 698 : cluster [DBG] pgmap v1340: 105 pgs: 105 active+clean; 16 GiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 118 KiB/s wr, 118 op/s 2024-04-07T16:38:38.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:38:38 smithi100 bash[74942]: cluster 2024-04-07T16:38:37.083604+0000 mgr.y (mgr.25175) 698 : cluster [DBG] pgmap v1340: 105 pgs: 105 active+clean; 16 GiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 118 KiB/s wr, 118 op/s 2024-04-07T16:38:38.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:38:38 smithi100 bash[76202]: cluster 2024-04-07T16:38:37.083604+0000 mgr.y (mgr.25175) 698 : cluster [DBG] pgmap v1340: 105 pgs: 105 active+clean; 16 GiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 118 KiB/s wr, 118 op/s 2024-04-07T16:38:38.691 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:38:38 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:38:38] "GET /metrics HTTP/1.1" 200 33986 "" "Prometheus/2.43.0" 2024-04-07T16:38:39.527 INFO:tasks.workunit.client.0.smithi100.stdout: 3: throughput=3.37MB/sec pending data=0 2024-04-07T16:38:39.527 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-ozQt6dIYc_JABPO off=15290932 len=1212949 2024-04-07T16:38:39.537 INFO:tasks.workunit.client.0.smithi100.stdout:op 11 completed, throughput=3.74MB/sec 2024-04-07T16:38:39.538 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-zGdmrfxgR-kBeB1 off=18447332 len=238328 2024-04-07T16:38:39.539 INFO:tasks.workunit.client.0.smithi100.stdout:op 12 completed, throughput=3.81MB/sec 2024-04-07T16:38:39.539 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-XkxTAQz4YxLlFq4 off=13939240 len=799187 2024-04-07T16:38:39.542 INFO:tasks.workunit.client.0.smithi100.stdout:op 13 completed, throughput=4.05MB/sec 2024-04-07T16:38:39.542 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-qc2XkKgBvBXS1ZB off=19500297 len=997864 2024-04-07T16:38:39.550 INFO:tasks.workunit.client.0.smithi100.stdout:op 14 completed, throughput=4.35MB/sec 2024-04-07T16:38:39.550 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-WCfGqvm8ZwjqzsB off=9129711 len=1660267 2024-04-07T16:38:39.562 INFO:tasks.workunit.client.0.smithi100.stdout:op 15 completed, throughput=4.85MB/sec 2024-04-07T16:38:39.562 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-3Vacwvuqmepdd0o off=18906481 len=64018 2024-04-07T16:38:39.563 INFO:tasks.workunit.client.0.smithi100.stdout:op 16 completed, throughput=4.86MB/sec 2024-04-07T16:38:39.563 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-nXVZjLx1VWa2ajK off=10014568 len=1213301 2024-04-07T16:38:39.566 INFO:tasks.workunit.client.0.smithi100.stdout:op 17 completed, throughput=5.23MB/sec 2024-04-07T16:38:40.566 INFO:tasks.workunit.client.0.smithi100.stdout: 4: throughput=3.96MB/sec pending data=0 2024-04-07T16:38:40.566 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-7TTnn2ScNBmW4lh off=5068105 len=934032 2024-04-07T16:38:40.569 INFO:tasks.workunit.client.0.smithi100.stdout:op 18 completed, throughput=4.17MB/sec 2024-04-07T16:38:40.571 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-ZzNmTTd8zg-EvT9 off=4390994 len=1916174 2024-04-07T16:38:40.574 INFO:tasks.workunit.client.0.smithi100.stdout:op 19 completed, throughput=4.61MB/sec 2024-04-07T16:38:40.574 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-JsrEjGTQiEhXOGX off=231979 len=532088 2024-04-07T16:38:40.575 INFO:tasks.workunit.client.0.smithi100.stdout:op 20 completed, throughput=4.73MB/sec 2024-04-07T16:38:40.576 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Zy1np7f4kmYhYSM off=1896582 len=936582 2024-04-07T16:38:40.585 INFO:tasks.workunit.client.0.smithi100.stdout:op 21 completed, throughput=4.94MB/sec 2024-04-07T16:38:40.585 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-TxmFTn_mW746EkC off=2267566 len=620710 2024-04-07T16:38:40.587 INFO:tasks.workunit.client.0.smithi100.stdout:op 22 completed, throughput=5.08MB/sec 2024-04-07T16:38:40.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:38:40 smithi144 bash[66654]: cluster 2024-04-07T16:38:39.084709+0000 mgr.y (mgr.25175) 699 : cluster [DBG] pgmap v1341: 105 pgs: 105 active+clean; 96 GiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 191 KiB/s rd, 650 KiB/s wr, 650 op/s 2024-04-07T16:38:40.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:38:40 smithi100 bash[74942]: cluster 2024-04-07T16:38:39.084709+0000 mgr.y (mgr.25175) 699 : cluster [DBG] pgmap v1341: 105 pgs: 105 active+clean; 96 GiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 191 KiB/s rd, 650 KiB/s wr, 650 op/s 2024-04-07T16:38:40.693 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:38:40 smithi100 bash[76202]: cluster 2024-04-07T16:38:39.084709+0000 mgr.y (mgr.25175) 699 : cluster [DBG] pgmap v1341: 105 pgs: 105 active+clean; 96 GiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 191 KiB/s rd, 650 KiB/s wr, 650 op/s 2024-04-07T16:38:41.587 INFO:tasks.workunit.client.0.smithi100.stdout: 5: throughput=4.09MB/sec pending data=0 2024-04-07T16:38:41.588 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-mH9XYnk7C1ogi5X off=20389077 len=1665546 2024-04-07T16:38:41.592 INFO:tasks.workunit.client.0.smithi100.stdout:op 23 completed, throughput=4.39MB/sec 2024-04-07T16:38:41.592 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-v5liglQAtYozywU off=0 len=1685391 2024-04-07T16:38:41.606 INFO:tasks.workunit.client.0.smithi100.stdout:op 24 completed, throughput=4.69MB/sec 2024-04-07T16:38:41.606 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-KolQg1K0goKt-fn off=1153451 len=1277604 2024-04-07T16:38:41.610 INFO:tasks.workunit.client.0.smithi100.stdout:op 25 completed, throughput=4.93MB/sec 2024-04-07T16:38:41.610 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-PIqjBi_WbfQcG0a off=4700334 len=782699 2024-04-07T16:38:41.618 INFO:tasks.workunit.client.0.smithi100.stdout:op 26 completed, throughput=5.06MB/sec 2024-04-07T16:38:42.607 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:38:42 smithi144 bash[95233]: ts=2024-04-07T16:38:42.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:38:42.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:38:42 smithi144 bash[66654]: cluster 2024-04-07T16:38:41.085581+0000 mgr.y (mgr.25175) 700 : cluster [DBG] pgmap v1342: 105 pgs: 105 active+clean; 126 GiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 434 KiB/s rd, 1.1 MiB/s wr, 853 op/s 2024-04-07T16:38:42.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:38:42 smithi144 bash[66654]: audit 2024-04-07T16:38:41.617027+0000 mon.a (mon.0) 2039 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:38:42.619 INFO:tasks.workunit.client.0.smithi100.stdout: 6: throughput=4.24MB/sec pending data=0 2024-04-07T16:38:42.619 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-GydkTbhwZad1SNi off=3887748 len=1376767 2024-04-07T16:38:42.622 INFO:tasks.workunit.client.0.smithi100.stdout:op 27 completed, throughput=4.45MB/sec 2024-04-07T16:38:42.622 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-w6QlFTcBdctgepL off=12762832 len=1035643 2024-04-07T16:38:42.632 INFO:tasks.workunit.client.0.smithi100.stdout:op 28 completed, throughput=4.61MB/sec 2024-04-07T16:38:42.632 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-U-Ngz62ZsStyh4v off=1086372 len=1159518 2024-04-07T16:38:42.642 INFO:tasks.workunit.client.0.smithi100.stdout:op 29 completed, throughput=4.78MB/sec 2024-04-07T16:38:42.642 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-ZFePGTyeF6BmDn1 off=13639664 len=1420205 2024-04-07T16:38:42.647 INFO:tasks.workunit.client.0.smithi100.stdout:op 30 completed, throughput=4.99MB/sec 2024-04-07T16:38:42.647 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Bb63ffHfpsl7yNy off=4588583 len=1261120 2024-04-07T16:38:42.649 INFO:tasks.workunit.client.0.smithi100.stdout:op 31 completed, throughput=5.18MB/sec 2024-04-07T16:38:42.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:38:42 smithi100 bash[74942]: cluster 2024-04-07T16:38:41.085581+0000 mgr.y (mgr.25175) 700 : cluster [DBG] pgmap v1342: 105 pgs: 105 active+clean; 126 GiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 434 KiB/s rd, 1.1 MiB/s wr, 853 op/s 2024-04-07T16:38:42.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:38:42 smithi100 bash[74942]: audit 2024-04-07T16:38:41.617027+0000 mon.a (mon.0) 2039 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:38:42.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:38:42 smithi100 bash[76202]: cluster 2024-04-07T16:38:41.085581+0000 mgr.y (mgr.25175) 700 : cluster [DBG] pgmap v1342: 105 pgs: 105 active+clean; 126 GiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 434 KiB/s rd, 1.1 MiB/s wr, 853 op/s 2024-04-07T16:38:42.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:38:42 smithi100 bash[76202]: audit 2024-04-07T16:38:41.617027+0000 mon.a (mon.0) 2039 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:38:43.650 INFO:tasks.workunit.client.0.smithi100.stdout: 7: throughput=4.46MB/sec pending data=0 2024-04-07T16:38:43.650 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-KWbhzReoc73OcCH off=10891301 len=451408 2024-04-07T16:38:43.652 INFO:tasks.workunit.client.0.smithi100.stdout:op 32 completed, throughput=4.52MB/sec 2024-04-07T16:38:43.652 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-k11LxgJmBmpUEKV off=5152291 len=334585 2024-04-07T16:38:43.654 INFO:tasks.workunit.client.0.smithi100.stdout:op 33 completed, throughput=4.56MB/sec 2024-04-07T16:38:43.654 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-svUliIk4BaKGXH8 off=7780919 len=859077 2024-04-07T16:38:43.656 INFO:tasks.workunit.client.0.smithi100.stdout:op 34 completed, throughput=4.68MB/sec 2024-04-07T16:38:43.656 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-TQv4QNpWQvNAgDB off=631502 len=324579 2024-04-07T16:38:43.658 INFO:tasks.workunit.client.0.smithi100.stdout:op 35 completed, throughput=4.72MB/sec 2024-04-07T16:38:43.658 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-gnpWYPoZ3eMgPfc off=1410584 len=75149 2024-04-07T16:38:43.659 INFO:tasks.workunit.client.0.smithi100.stdout:op 36 completed, throughput=4.73MB/sec 2024-04-07T16:38:43.659 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-n3kjQovswcCNcMQ off=6371028 len=613794 2024-04-07T16:38:43.666 INFO:tasks.workunit.client.0.smithi100.stdout:op 37 completed, throughput=4.8MB/sec 2024-04-07T16:38:43.666 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-pzeHigby-2M72jR off=4799439 len=732664 2024-04-07T16:38:43.674 INFO:tasks.workunit.client.0.smithi100.stdout:op 38 completed, throughput=4.9MB/sec 2024-04-07T16:38:43.674 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Mcq8jD9PjKFjYKC off=375832 len=538481 2024-04-07T16:38:43.680 INFO:tasks.workunit.client.0.smithi100.stdout:op 39 completed, throughput=4.96MB/sec 2024-04-07T16:38:43.680 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-3QaikphGvFJS6ji off=2411135 len=474605 2024-04-07T16:38:43.682 INFO:tasks.workunit.client.0.smithi100.stdout:op 40 completed, throughput=5.02MB/sec 2024-04-07T16:38:44.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:38:44 smithi144 bash[66654]: cluster 2024-04-07T16:38:43.086728+0000 mgr.y (mgr.25175) 701 : cluster [DBG] pgmap v1343: 105 pgs: 105 active+clean; 126 GiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 855 op/s 2024-04-07T16:38:44.683 INFO:tasks.workunit.client.0.smithi100.stdout: 8: throughput=4.41MB/sec pending data=0 2024-04-07T16:38:44.683 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-b1TVf8OKL7sWqtH off=5888846 len=1879297 2024-04-07T16:38:44.687 INFO:tasks.workunit.client.0.smithi100.stdout:op 41 completed, throughput=4.63MB/sec 2024-04-07T16:38:44.687 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-B4RseVi1HmreXjM off=15138727 len=969915 2024-04-07T16:38:44.689 INFO:tasks.workunit.client.0.smithi100.stdout:op 42 completed, throughput=4.74MB/sec 2024-04-07T16:38:44.689 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-6gMpYpD1sBmbOZt off=7976903 len=1520676 2024-04-07T16:38:44.690 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:38:44 smithi100 bash[74942]: cluster 2024-04-07T16:38:43.086728+0000 mgr.y (mgr.25175) 701 : cluster [DBG] pgmap v1343: 105 pgs: 105 active+clean; 126 GiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 855 op/s 2024-04-07T16:38:44.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:38:44 smithi100 bash[76202]: cluster 2024-04-07T16:38:43.086728+0000 mgr.y (mgr.25175) 701 : cluster [DBG] pgmap v1343: 105 pgs: 105 active+clean; 126 GiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 855 op/s 2024-04-07T16:38:44.691 INFO:tasks.workunit.client.0.smithi100.stdout:op 43 completed, throughput=4.91MB/sec 2024-04-07T16:38:44.691 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-sZ6uFvOwnBxyEGW off=825323 len=1854864 2024-04-07T16:38:44.694 INFO:tasks.workunit.client.0.smithi100.stdout:op 44 completed, throughput=5.13MB/sec 2024-04-07T16:38:45.694 INFO:tasks.workunit.client.0.smithi100.stdout: 9: throughput=4.57MB/sec pending data=0 2024-04-07T16:38:45.694 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-iqrDjQkvPIJcqAY off=240128 len=1729663 2024-04-07T16:38:45.699 INFO:tasks.workunit.client.0.smithi100.stdout:op 45 completed, throughput=4.75MB/sec 2024-04-07T16:38:45.699 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-8md_NsBFPWe3_nL off=868111 len=1078474 2024-04-07T16:38:45.702 INFO:tasks.workunit.client.0.smithi100.stdout:op 46 completed, throughput=4.86MB/sec 2024-04-07T16:38:45.702 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-LHwPQGeybaykfNI off=2951289 len=402231 2024-04-07T16:38:45.704 INFO:tasks.workunit.client.0.smithi100.stdout:op 47 completed, throughput=4.9MB/sec 2024-04-07T16:38:45.704 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-uGVe_t3PHIkdIlx off=593224 len=1809204 2024-04-07T16:38:45.708 INFO:tasks.workunit.client.0.smithi100.stdout:op 48 completed, throughput=5.08MB/sec 2024-04-07T16:38:46.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:38:46 smithi144 bash[66654]: cluster 2024-04-07T16:38:45.087693+0000 mgr.y (mgr.25175) 702 : cluster [DBG] pgmap v1344: 105 pgs: 105 active+clean; 126 GiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.9 MiB/s rd, 1.8 MiB/s wr, 855 op/s 2024-04-07T16:38:46.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:38:46 smithi100 bash[74942]: cluster 2024-04-07T16:38:45.087693+0000 mgr.y (mgr.25175) 702 : cluster [DBG] pgmap v1344: 105 pgs: 105 active+clean; 126 GiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.9 MiB/s rd, 1.8 MiB/s wr, 855 op/s 2024-04-07T16:38:46.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:38:46 smithi100 bash[76202]: cluster 2024-04-07T16:38:45.087693+0000 mgr.y (mgr.25175) 702 : cluster [DBG] pgmap v1344: 105 pgs: 105 active+clean; 126 GiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.9 MiB/s rd, 1.8 MiB/s wr, 855 op/s 2024-04-07T16:38:46.708 INFO:tasks.workunit.client.0.smithi100.stdout: 10: throughput=4.59MB/sec pending data=0 2024-04-07T16:38:46.708 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-5XIcFagxJcf8XPj off=11518516 len=696504 2024-04-07T16:38:46.712 INFO:tasks.workunit.client.0.smithi100.stdout:op 49 completed, throughput=4.65MB/sec 2024-04-07T16:38:46.712 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-eaDxhyanV7ryAs_ off=15343758 len=945576 2024-04-07T16:38:46.714 INFO:tasks.workunit.client.0.smithi100.stdout:op 50 completed, throughput=4.74MB/sec 2024-04-07T16:38:46.714 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-MYTkXwEay4QbL0y off=6538812 len=1839079 2024-04-07T16:38:46.716 INFO:tasks.workunit.client.0.smithi100.stdout:op 51 completed, throughput=4.91MB/sec 2024-04-07T16:38:46.716 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-VKKSNCcxmyv1PkT off=3951821 len=631322 2024-04-07T16:38:46.720 INFO:tasks.workunit.client.0.smithi100.stdout:op 52 completed, throughput=4.96MB/sec 2024-04-07T16:38:46.721 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-bPHyPsM_YVQloKu off=4294698 len=1733968 2024-04-07T16:38:46.724 INFO:tasks.workunit.client.0.smithi100.stdout:op 53 completed, throughput=5.12MB/sec 2024-04-07T16:38:47.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:38:47 smithi144 bash[66654]: audit 2024-04-07T16:38:46.753617+0000 mon.a (mon.0) 2040 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:38:47.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:38:47 smithi144 bash[66654]: audit 2024-04-07T16:38:47.261252+0000 mon.a (mon.0) 2041 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:38:47.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:38:47 smithi144 bash[66654]: audit 2024-04-07T16:38:47.272154+0000 mon.a (mon.0) 2042 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:38:47.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:38:47 smithi100 bash[74942]: audit 2024-04-07T16:38:46.753617+0000 mon.a (mon.0) 2040 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:38:47.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:38:47 smithi100 bash[74942]: audit 2024-04-07T16:38:47.261252+0000 mon.a (mon.0) 2041 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:38:47.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:38:47 smithi100 bash[74942]: audit 2024-04-07T16:38:47.272154+0000 mon.a (mon.0) 2042 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:38:47.692 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:38:47 smithi100 bash[76202]: audit 2024-04-07T16:38:46.753617+0000 mon.a (mon.0) 2040 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:38:47.692 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:38:47 smithi100 bash[76202]: audit 2024-04-07T16:38:47.261252+0000 mon.a (mon.0) 2041 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:38:47.692 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:38:47 smithi100 bash[76202]: audit 2024-04-07T16:38:47.272154+0000 mon.a (mon.0) 2042 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:38:47.724 INFO:tasks.workunit.client.0.smithi100.stdout: 11: throughput=4.67MB/sec pending data=0 2024-04-07T16:38:47.724 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-vW6tI2QtVYgmAeV off=612997 len=545462 2024-04-07T16:38:47.728 INFO:tasks.workunit.client.0.smithi100.stdout:op 54 completed, throughput=4.71MB/sec 2024-04-07T16:38:47.728 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Lhq76ZCOvdbD-nY off=1309338 len=1772145 2024-04-07T16:38:47.737 INFO:tasks.workunit.client.0.smithi100.stdout:op 55 completed, throughput=4.86MB/sec 2024-04-07T16:38:47.737 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-aswYn0V9xeglwcO off=1801461 len=655767 2024-04-07T16:38:47.737 INFO:tasks.workunit.client.0.smithi100.stdout:op 56 completed, throughput=4.91MB/sec 2024-04-07T16:38:47.737 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-N3UkWHLDABMUYF1 off=10670901 len=987645 2024-04-07T16:38:47.745 INFO:tasks.workunit.client.0.smithi100.stdout:op 57 completed, throughput=4.99MB/sec 2024-04-07T16:38:47.745 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-OtPGZQvs8MFP_uX off=3116041 len=970745 2024-04-07T16:38:47.747 INFO:tasks.workunit.client.0.smithi100.stdout:op 58 completed, throughput=5.08MB/sec 2024-04-07T16:38:48.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:38:48 smithi144 bash[66654]: cluster 2024-04-07T16:38:47.088179+0000 mgr.y (mgr.25175) 703 : cluster [DBG] pgmap v1345: 105 pgs: 105 active+clean; 126 GiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 2.0 MiB/s rd, 1.8 MiB/s wr, 749 op/s 2024-04-07T16:38:48.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:38:48 smithi144 bash[66654]: audit 2024-04-07T16:38:47.566295+0000 mon.a (mon.0) 2043 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:38:48.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:38:48 smithi144 bash[66654]: audit 2024-04-07T16:38:47.577698+0000 mon.a (mon.0) 2044 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:38:48.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:38:48 smithi144 bash[66654]: audit 2024-04-07T16:38:48.206549+0000 mon.a (mon.0) 2045 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:38:48.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:38:48 smithi144 bash[66654]: audit 2024-04-07T16:38:48.208583+0000 mon.a (mon.0) 2046 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:38:48.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:38:48 smithi144 bash[66654]: audit 2024-04-07T16:38:48.217342+0000 mon.a (mon.0) 2047 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:38:48.691 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:38:48 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:38:48] "GET /metrics HTTP/1.1" 200 34046 "" "Prometheus/2.43.0" 2024-04-07T16:38:48.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:38:48 smithi100 bash[74942]: cluster 2024-04-07T16:38:47.088179+0000 mgr.y (mgr.25175) 703 : cluster [DBG] pgmap v1345: 105 pgs: 105 active+clean; 126 GiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 2.0 MiB/s rd, 1.8 MiB/s wr, 749 op/s 2024-04-07T16:38:48.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:38:48 smithi100 bash[74942]: audit 2024-04-07T16:38:47.566295+0000 mon.a (mon.0) 2043 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:38:48.692 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:38:48 smithi100 bash[74942]: audit 2024-04-07T16:38:47.577698+0000 mon.a (mon.0) 2044 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:38:48.692 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:38:48 smithi100 bash[74942]: audit 2024-04-07T16:38:48.206549+0000 mon.a (mon.0) 2045 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:38:48.692 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:38:48 smithi100 bash[74942]: audit 2024-04-07T16:38:48.208583+0000 mon.a (mon.0) 2046 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:38:48.692 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:38:48 smithi100 bash[74942]: audit 2024-04-07T16:38:48.217342+0000 mon.a (mon.0) 2047 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:38:48.692 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:38:48 smithi100 bash[76202]: cluster 2024-04-07T16:38:47.088179+0000 mgr.y (mgr.25175) 703 : cluster [DBG] pgmap v1345: 105 pgs: 105 active+clean; 126 GiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 2.0 MiB/s rd, 1.8 MiB/s wr, 749 op/s 2024-04-07T16:38:48.692 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:38:48 smithi100 bash[76202]: audit 2024-04-07T16:38:47.566295+0000 mon.a (mon.0) 2043 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:38:48.692 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:38:48 smithi100 bash[76202]: audit 2024-04-07T16:38:47.577698+0000 mon.a (mon.0) 2044 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:38:48.692 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:38:48 smithi100 bash[76202]: audit 2024-04-07T16:38:48.206549+0000 mon.a (mon.0) 2045 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:38:48.692 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:38:48 smithi100 bash[76202]: audit 2024-04-07T16:38:48.208583+0000 mon.a (mon.0) 2046 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:38:48.692 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:38:48 smithi100 bash[76202]: audit 2024-04-07T16:38:48.217342+0000 mon.a (mon.0) 2047 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:38:48.747 INFO:tasks.workunit.client.0.smithi100.stdout: 12: throughput=4.66MB/sec pending data=0 2024-04-07T16:38:48.747 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-S_XcJXt4yv_SRi_ off=1094604 len=70445 2024-04-07T16:38:48.752 INFO:tasks.workunit.client.0.smithi100.stdout:op 59 completed, throughput=4.67MB/sec 2024-04-07T16:38:48.752 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Ugo61Ox_w1AHLc2 off=2113191 len=991293 2024-04-07T16:38:48.755 INFO:tasks.workunit.client.0.smithi100.stdout:op 60 completed, throughput=4.74MB/sec 2024-04-07T16:38:48.755 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-_jN2-P08or1pQ5d off=19465 len=385217 2024-04-07T16:38:48.762 INFO:tasks.workunit.client.0.smithi100.stdout:op 61 completed, throughput=4.77MB/sec 2024-04-07T16:38:48.762 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-U6_H7zI7PWJOtuT off=10059228 len=1292960 2024-04-07T16:38:48.765 INFO:tasks.workunit.client.0.smithi100.stdout:op 62 completed, throughput=4.87MB/sec 2024-04-07T16:38:48.765 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-nIpIRbQ98vwxjzd off=712890 len=1709512 2024-04-07T16:38:48.769 INFO:tasks.workunit.client.0.smithi100.stdout:op 63 completed, throughput=5MB/sec 2024-04-07T16:38:48.769 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-xXkOTaEgB83gvX4 off=929675 len=864516 2024-04-07T16:38:48.772 INFO:tasks.workunit.client.0.smithi100.stdout:op 64 completed, throughput=5.06MB/sec 2024-04-07T16:38:49.772 INFO:tasks.workunit.client.0.smithi100.stdout: 13: throughput=4.68MB/sec pending data=0 2024-04-07T16:38:49.772 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-CsWnu77SsC8IMR5 off=2848346 len=1382214 2024-04-07T16:38:49.783 INFO:tasks.workunit.client.0.smithi100.stdout:op 65 completed, throughput=4.78MB/sec 2024-04-07T16:38:49.783 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-RjUKM5o-YvhQoDl off=9762872 len=937091 2024-04-07T16:38:49.792 INFO:tasks.workunit.client.0.smithi100.stdout:op 66 completed, throughput=4.84MB/sec 2024-04-07T16:38:49.792 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-QEjbJeIbtuwadze off=1636440 len=1089358 2024-04-07T16:38:49.802 INFO:tasks.workunit.client.0.smithi100.stdout:op 67 completed, throughput=4.92MB/sec 2024-04-07T16:38:49.802 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-OdalvbhzqH1HdDm off=568121 len=75652 2024-04-07T16:38:49.803 INFO:tasks.workunit.client.0.smithi100.stdout:op 68 completed, throughput=4.92MB/sec 2024-04-07T16:38:49.803 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-VMFYdUiz_ucslkB off=3268268 len=547946 2024-04-07T16:38:49.805 INFO:tasks.workunit.client.0.smithi100.stdout:op 69 completed, throughput=4.96MB/sec 2024-04-07T16:38:49.805 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-x1aY4CtDWa1-6-H off=1897733 len=1397942 2024-04-07T16:38:49.807 INFO:tasks.workunit.client.0.smithi100.stdout:op 70 completed, throughput=5.06MB/sec 2024-04-07T16:38:50.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:38:50 smithi144 bash[66654]: cluster 2024-04-07T16:38:49.089219+0000 mgr.y (mgr.25175) 704 : cluster [DBG] pgmap v1346: 105 pgs: 105 active+clean; 126 GiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 2.9 MiB/s rd, 2.0 MiB/s wr, 750 op/s 2024-04-07T16:38:50.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:38:50 smithi100 bash[74942]: cluster 2024-04-07T16:38:49.089219+0000 mgr.y (mgr.25175) 704 : cluster [DBG] pgmap v1346: 105 pgs: 105 active+clean; 126 GiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 2.9 MiB/s rd, 2.0 MiB/s wr, 750 op/s 2024-04-07T16:38:50.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:38:50 smithi100 bash[76202]: cluster 2024-04-07T16:38:49.089219+0000 mgr.y (mgr.25175) 704 : cluster [DBG] pgmap v1346: 105 pgs: 105 active+clean; 126 GiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 2.9 MiB/s rd, 2.0 MiB/s wr, 750 op/s 2024-04-07T16:38:50.807 INFO:tasks.workunit.client.0.smithi100.stdout: 14: throughput=4.71MB/sec pending data=0 2024-04-07T16:38:50.807 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-ik2HfTILWVLKDZw off=13898240 len=1596161 2024-04-07T16:38:50.811 INFO:tasks.workunit.client.0.smithi100.stdout:op 71 completed, throughput=4.81MB/sec 2024-04-07T16:38:50.811 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-1ihMEgkwKAJMSOl off=4967635 len=1534261 2024-04-07T16:38:50.824 INFO:tasks.workunit.client.0.smithi100.stdout:op 72 completed, throughput=4.91MB/sec 2024-04-07T16:38:50.824 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-V60jaIL1rk3iDx- off=23440386 len=131412 2024-04-07T16:38:50.829 INFO:tasks.workunit.client.0.smithi100.stdout:op 73 completed, throughput=4.92MB/sec 2024-04-07T16:38:50.829 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-_godbPE8Qu3HVWW off=1919314 len=1305755 2024-04-07T16:38:50.833 INFO:tasks.workunit.client.0.smithi100.stdout:op 74 completed, throughput=5MB/sec 2024-04-07T16:38:51.834 INFO:tasks.workunit.client.0.smithi100.stdout: 15: throughput=4.68MB/sec pending data=0 2024-04-07T16:38:51.834 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-u07t5w9OTX6Lldx off=1806640 len=2009355 2024-04-07T16:38:51.839 INFO:tasks.workunit.client.0.smithi100.stdout:op 75 completed, throughput=4.8MB/sec 2024-04-07T16:38:51.839 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-jR7lF6ITECWFxLg off=8838328 len=1285444 2024-04-07T16:38:51.843 INFO:tasks.workunit.client.0.smithi100.stdout:op 76 completed, throughput=4.88MB/sec 2024-04-07T16:38:51.843 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-b7EGHmx1m51g9Lx off=16562993 len=80105 2024-04-07T16:38:51.847 INFO:tasks.workunit.client.0.smithi100.stdout:op 77 completed, throughput=4.88MB/sec 2024-04-07T16:38:51.847 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-GnGA1WXD4-1SWqO off=15534396 len=1094949 2024-04-07T16:38:51.851 INFO:tasks.workunit.client.0.smithi100.stdout:op 78 completed, throughput=4.95MB/sec 2024-04-07T16:38:51.851 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-ly9VFIqftSH2Mqq off=2445256 len=982805 2024-04-07T16:38:51.855 INFO:tasks.workunit.client.0.smithi100.stdout:op 79 completed, throughput=5.01MB/sec 2024-04-07T16:38:52.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:38:52 smithi144 bash[66654]: cluster 2024-04-07T16:38:51.090149+0000 mgr.y (mgr.25175) 705 : cluster [DBG] pgmap v1347: 105 pgs: 105 active+clean; 126 GiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 3.2 MiB/s rd, 1.8 MiB/s wr, 207 op/s 2024-04-07T16:38:52.607 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:38:52 smithi144 bash[95233]: ts=2024-04-07T16:38:52.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:38:52.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:38:52 smithi100 bash[74942]: cluster 2024-04-07T16:38:51.090149+0000 mgr.y (mgr.25175) 705 : cluster [DBG] pgmap v1347: 105 pgs: 105 active+clean; 126 GiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 3.2 MiB/s rd, 1.8 MiB/s wr, 207 op/s 2024-04-07T16:38:52.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:38:52 smithi100 bash[76202]: cluster 2024-04-07T16:38:51.090149+0000 mgr.y (mgr.25175) 705 : cluster [DBG] pgmap v1347: 105 pgs: 105 active+clean; 126 GiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 3.2 MiB/s rd, 1.8 MiB/s wr, 207 op/s 2024-04-07T16:38:52.855 INFO:tasks.workunit.client.0.smithi100.stdout: 16: throughput=4.7MB/sec pending data=0 2024-04-07T16:38:52.855 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-wijL9yiTT29AGAm off=13591340 len=1457918 2024-04-07T16:38:52.868 INFO:tasks.workunit.client.0.smithi100.stdout:op 80 completed, throughput=4.78MB/sec 2024-04-07T16:38:52.868 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-0tuF1jMbiXvpcwW off=14900474 len=1860948 2024-04-07T16:38:52.873 INFO:tasks.workunit.client.0.smithi100.stdout:op 81 completed, throughput=4.89MB/sec 2024-04-07T16:38:52.873 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-AXAf6OF9yvzygo5 off=11156076 len=306568 2024-04-07T16:38:52.878 INFO:tasks.workunit.client.0.smithi100.stdout:op 82 completed, throughput=4.91MB/sec 2024-04-07T16:38:52.878 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-GvGasBcimI4TAE2 off=975774 len=862639 2024-04-07T16:38:52.886 INFO:tasks.workunit.client.0.smithi100.stdout:op 83 completed, throughput=4.95MB/sec 2024-04-07T16:38:52.886 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-EF7Q4_ugeP3dmJt off=7101455 len=225243 2024-04-07T16:38:52.890 INFO:tasks.workunit.client.0.smithi100.stdout:op 84 completed, throughput=4.97MB/sec 2024-04-07T16:38:52.891 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-z7s1WYxKX6JqT8S off=5250078 len=1294044 2024-04-07T16:38:52.893 INFO:tasks.workunit.client.0.smithi100.stdout:op 85 completed, throughput=5.04MB/sec 2024-04-07T16:38:53.893 INFO:tasks.workunit.client.0.smithi100.stdout: 17: throughput=4.75MB/sec pending data=0 2024-04-07T16:38:53.893 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-mp-9I0RR29nfOV1 off=11576173 len=1220211 2024-04-07T16:38:53.902 INFO:tasks.workunit.client.0.smithi100.stdout:op 86 completed, throughput=4.81MB/sec 2024-04-07T16:38:53.903 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-_9dqjn1JtYEuPHt off=137185 len=136286 2024-04-07T16:38:53.904 INFO:tasks.workunit.client.0.smithi100.stdout:op 87 completed, throughput=4.82MB/sec 2024-04-07T16:38:53.904 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-NxQaM5K2qMIDOeg off=1381082 len=85650 2024-04-07T16:38:53.905 INFO:tasks.workunit.client.0.smithi100.stdout:op 88 completed, throughput=4.83MB/sec 2024-04-07T16:38:53.906 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-EB6xkSaATkosx2P off=11553424 len=1501433 2024-04-07T16:38:53.918 INFO:tasks.workunit.client.0.smithi100.stdout:op 89 completed, throughput=4.9MB/sec 2024-04-07T16:38:53.918 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-fW0KpLfQw9UaLDE off=2154543 len=839823 2024-04-07T16:38:53.921 INFO:tasks.workunit.client.0.smithi100.stdout:op 90 completed, throughput=4.95MB/sec 2024-04-07T16:38:53.921 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-KmzEdqrUYOSA89j off=3074822 len=1234539 2024-04-07T16:38:53.925 INFO:tasks.workunit.client.0.smithi100.stdout:op 91 completed, throughput=5.02MB/sec 2024-04-07T16:38:54.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:38:54 smithi144 bash[66654]: cluster 2024-04-07T16:38:53.091266+0000 mgr.y (mgr.25175) 706 : cluster [DBG] pgmap v1348: 105 pgs: 105 active+clean; 126 GiB data, 3.1 GiB used, 712 GiB / 715 GiB avail; 3.8 MiB/s rd, 1.8 MiB/s wr, 5 op/s 2024-04-07T16:38:54.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:38:54 smithi100 bash[74942]: cluster 2024-04-07T16:38:53.091266+0000 mgr.y (mgr.25175) 706 : cluster [DBG] pgmap v1348: 105 pgs: 105 active+clean; 126 GiB data, 3.1 GiB used, 712 GiB / 715 GiB avail; 3.8 MiB/s rd, 1.8 MiB/s wr, 5 op/s 2024-04-07T16:38:54.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:38:54 smithi100 bash[76202]: cluster 2024-04-07T16:38:53.091266+0000 mgr.y (mgr.25175) 706 : cluster [DBG] pgmap v1348: 105 pgs: 105 active+clean; 126 GiB data, 3.1 GiB used, 712 GiB / 715 GiB avail; 3.8 MiB/s rd, 1.8 MiB/s wr, 5 op/s 2024-04-07T16:38:54.925 INFO:tasks.workunit.client.0.smithi100.stdout: 18: throughput=4.74MB/sec pending data=0 2024-04-07T16:38:54.926 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-fQy40uBEqQUr3Hk off=2576756 len=1049936 2024-04-07T16:38:54.928 INFO:tasks.workunit.client.0.smithi100.stdout:op 92 completed, throughput=4.8MB/sec 2024-04-07T16:38:54.928 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-WLHYDH4BbOvpJ-c off=567115 len=120944 2024-04-07T16:38:54.930 INFO:tasks.workunit.client.0.smithi100.stdout:op 93 completed, throughput=4.8MB/sec 2024-04-07T16:38:54.930 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-H8SzEM0d71BEYzj off=1625952 len=1737700 2024-04-07T16:38:54.944 INFO:tasks.workunit.client.0.smithi100.stdout:op 94 completed, throughput=4.89MB/sec 2024-04-07T16:38:54.944 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-9621VIoyPnodFof off=1741791 len=369136 2024-04-07T16:38:54.950 INFO:tasks.workunit.client.0.smithi100.stdout:op 95 completed, throughput=4.91MB/sec 2024-04-07T16:38:54.950 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-6XQ6uWajcITSac7 off=10486780 len=1550124 2024-04-07T16:38:54.954 INFO:tasks.workunit.client.0.smithi100.stdout:op 96 completed, throughput=4.99MB/sec 2024-04-07T16:38:54.954 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-C1__mGFgtnoyZgb off=4836972 len=555209 2024-04-07T16:38:54.956 INFO:tasks.workunit.client.0.smithi100.stdout:op 97 completed, throughput=5.01MB/sec 2024-04-07T16:38:55.957 INFO:tasks.workunit.client.0.smithi100.stdout: 19: throughput=4.76MB/sec pending data=0 2024-04-07T16:38:55.957 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-DH1TmwBn0LZx5YF off=3387127 len=107612 2024-04-07T16:38:55.961 INFO:tasks.workunit.client.0.smithi100.stdout:op 98 completed, throughput=4.76MB/sec 2024-04-07T16:38:55.962 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-04ybS_7irkc-S_U off=10432370 len=226074 2024-04-07T16:38:55.963 INFO:tasks.workunit.client.0.smithi100.stdout:op 99 completed, throughput=4.77MB/sec 2024-04-07T16:38:55.963 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-gPrB108zKAtCzsz off=269131 len=1118432 2024-04-07T16:38:55.972 INFO:tasks.workunit.client.0.smithi100.stdout:op 100 completed, throughput=4.82MB/sec 2024-04-07T16:38:55.972 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-VMTBzOR2uiEBmWd off=1962729 len=631889 2024-04-07T16:38:55.979 INFO:tasks.workunit.client.0.smithi100.stdout:op 101 completed, throughput=4.85MB/sec 2024-04-07T16:38:55.979 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-SQb-4KONggjUnIT off=4392858 len=1302250 2024-04-07T16:38:55.990 INFO:tasks.workunit.client.0.smithi100.stdout:op 102 completed, throughput=4.91MB/sec 2024-04-07T16:38:55.990 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-cAz01jhseMox803 off=569739 len=1504416 2024-04-07T16:38:55.993 INFO:tasks.workunit.client.0.smithi100.stdout:op 103 completed, throughput=4.99MB/sec 2024-04-07T16:38:55.994 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-RP67jXwRDGnMYnX off=7848220 len=1649153 2024-04-07T16:38:56.003 INFO:tasks.workunit.client.0.smithi100.stdout:op 104 completed, throughput=5.06MB/sec 2024-04-07T16:38:56.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:38:56 smithi144 bash[66654]: cluster 2024-04-07T16:38:55.092314+0000 mgr.y (mgr.25175) 707 : cluster [DBG] pgmap v1349: 105 pgs: 105 active+clean; 126 GiB data, 3.1 GiB used, 712 GiB / 715 GiB avail; 3.2 MiB/s rd, 1.6 MiB/s wr, 4 op/s 2024-04-07T16:38:56.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:38:56 smithi100 bash[74942]: cluster 2024-04-07T16:38:55.092314+0000 mgr.y (mgr.25175) 707 : cluster [DBG] pgmap v1349: 105 pgs: 105 active+clean; 126 GiB data, 3.1 GiB used, 712 GiB / 715 GiB avail; 3.2 MiB/s rd, 1.6 MiB/s wr, 4 op/s 2024-04-07T16:38:56.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:38:56 smithi100 bash[76202]: cluster 2024-04-07T16:38:55.092314+0000 mgr.y (mgr.25175) 707 : cluster [DBG] pgmap v1349: 105 pgs: 105 active+clean; 126 GiB data, 3.1 GiB used, 712 GiB / 715 GiB avail; 3.2 MiB/s rd, 1.6 MiB/s wr, 4 op/s 2024-04-07T16:38:57.004 INFO:tasks.workunit.client.0.smithi100.stdout: 20: throughput=4.82MB/sec pending data=0 2024-04-07T16:38:57.004 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Js9ereOcOmF_z_o off=2876116 len=1342203 2024-04-07T16:38:57.007 INFO:tasks.workunit.client.0.smithi100.stdout:op 105 completed, throughput=4.88MB/sec 2024-04-07T16:38:57.007 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-dGRq0Hwz5k5a1vp off=1810810 len=983212 2024-04-07T16:38:57.010 INFO:tasks.workunit.client.0.smithi100.stdout:op 106 completed, throughput=4.92MB/sec 2024-04-07T16:38:57.010 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-ppAPH7R9tHPwe20 off=3729996 len=1141898 2024-04-07T16:38:57.019 INFO:tasks.workunit.client.0.smithi100.stdout:op 107 completed, throughput=4.98MB/sec 2024-04-07T16:38:57.019 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-e_yovDypQXmEu6B off=606098 len=534225 2024-04-07T16:38:57.020 INFO:tasks.workunit.client.0.smithi100.stdout:op 108 completed, throughput=5MB/sec 2024-04-07T16:38:57.020 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-cTMsAR81F7mpsVM off=947379 len=888858 2024-04-07T16:38:57.027 INFO:tasks.workunit.client.0.smithi100.stdout:op 109 completed, throughput=5.04MB/sec 2024-04-07T16:38:58.027 INFO:tasks.workunit.client.0.smithi100.stdout: 21: throughput=4.81MB/sec pending data=0 2024-04-07T16:38:58.028 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-o_Cg9yG5gX9eRrv off=2734827 len=1798005 2024-04-07T16:38:58.041 INFO:tasks.workunit.client.0.smithi100.stdout:op 110 completed, throughput=4.88MB/sec 2024-04-07T16:38:58.041 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Eer5saPQmvgzFMW off=1974190 len=245974 2024-04-07T16:38:58.046 INFO:tasks.workunit.client.0.smithi100.stdout:op 111 completed, throughput=4.89MB/sec 2024-04-07T16:38:58.046 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-sUHreuPnVr8ILmg off=20072441 len=1861207 2024-04-07T16:38:58.061 INFO:tasks.workunit.client.0.smithi100.stdout:op 112 completed, throughput=4.97MB/sec 2024-04-07T16:38:58.061 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-nF96FCcjE8ptP4O off=65117 len=38963 2024-04-07T16:38:58.062 INFO:tasks.workunit.client.0.smithi100.stdout:op 113 completed, throughput=4.97MB/sec 2024-04-07T16:38:58.062 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-mrrV6IlScP7UaUa off=4008103 len=1946095 2024-04-07T16:38:58.069 INFO:tasks.workunit.client.0.smithi100.stdout:op 114 completed, throughput=5.06MB/sec 2024-04-07T16:38:58.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:38:58 smithi144 bash[66654]: cluster 2024-04-07T16:38:57.093118+0000 mgr.y (mgr.25175) 708 : cluster [DBG] pgmap v1350: 105 pgs: 105 active+clean; 126 GiB data, 3.1 GiB used, 712 GiB / 715 GiB avail; 2.7 MiB/s rd, 1.6 MiB/s wr, 4 op/s 2024-04-07T16:38:58.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:38:58 smithi100 bash[74942]: cluster 2024-04-07T16:38:57.093118+0000 mgr.y (mgr.25175) 708 : cluster [DBG] pgmap v1350: 105 pgs: 105 active+clean; 126 GiB data, 3.1 GiB used, 712 GiB / 715 GiB avail; 2.7 MiB/s rd, 1.6 MiB/s wr, 4 op/s 2024-04-07T16:38:58.692 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:38:58 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:38:58] "GET /metrics HTTP/1.1" 200 34046 "" "Prometheus/2.43.0" 2024-04-07T16:38:58.692 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:38:58 smithi100 bash[76202]: cluster 2024-04-07T16:38:57.093118+0000 mgr.y (mgr.25175) 708 : cluster [DBG] pgmap v1350: 105 pgs: 105 active+clean; 126 GiB data, 3.1 GiB used, 712 GiB / 715 GiB avail; 2.7 MiB/s rd, 1.6 MiB/s wr, 4 op/s 2024-04-07T16:38:59.069 INFO:tasks.workunit.client.0.smithi100.stdout: 22: throughput=4.83MB/sec pending data=0 2024-04-07T16:38:59.069 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-5GzcihHixrItT6T off=1080021 len=1688042 2024-04-07T16:38:59.073 INFO:tasks.workunit.client.0.smithi100.stdout:op 115 completed, throughput=4.9MB/sec 2024-04-07T16:38:59.073 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-4IU17cVHADaw1V3 off=4442794 len=766583 2024-04-07T16:38:59.077 INFO:tasks.workunit.client.0.smithi100.stdout:op 116 completed, throughput=4.93MB/sec 2024-04-07T16:38:59.077 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-C1qzYVzVjGoRVQf off=15928858 len=468172 2024-04-07T16:38:59.083 INFO:tasks.workunit.client.0.smithi100.stdout:op 117 completed, throughput=4.95MB/sec 2024-04-07T16:38:59.083 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-39eC1TPtgLxtkvY off=8375004 len=1125252 2024-04-07T16:38:59.093 INFO:tasks.workunit.client.0.smithi100.stdout:op 118 completed, throughput=5MB/sec 2024-04-07T16:38:59.093 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-SaWb0IELbp92ezM off=21618520 len=99580 2024-04-07T16:38:59.098 INFO:tasks.workunit.client.0.smithi100.stdout:op 119 completed, throughput=5MB/sec 2024-04-07T16:39:00.098 INFO:tasks.workunit.client.0.smithi100.stdout: 23: throughput=4.79MB/sec pending data=0 2024-04-07T16:39:00.098 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-7TTnn2ScNBmW4lh off=983761 len=1180937 2024-04-07T16:39:00.108 INFO:tasks.workunit.client.0.smithi100.stdout:op 120 completed, throughput=4.84MB/sec 2024-04-07T16:39:00.108 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Lacyr5nOHfLD_g5 off=227209 len=132182 2024-04-07T16:39:00.112 INFO:tasks.workunit.client.0.smithi100.stdout:op 121 completed, throughput=4.84MB/sec 2024-04-07T16:39:00.112 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-vmsHpSCUvAvxc4D off=719861 len=356284 2024-04-07T16:39:00.113 INFO:tasks.workunit.client.0.smithi100.stdout:op 122 completed, throughput=4.85MB/sec 2024-04-07T16:39:00.113 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-vOIiX0l1_9wRtI7 off=1956483 len=1516513 2024-04-07T16:39:00.119 INFO:tasks.workunit.client.0.smithi100.stdout:op 123 completed, throughput=4.91MB/sec 2024-04-07T16:39:00.119 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-ZQzUCurdI5z8IK0 off=9706737 len=1312630 2024-04-07T16:39:00.128 INFO:tasks.workunit.client.0.smithi100.stdout:op 124 completed, throughput=4.96MB/sec 2024-04-07T16:39:00.129 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-qTZr91TlS5VuECW off=2658927 len=1258907 2024-04-07T16:39:00.141 INFO:tasks.workunit.client.0.smithi100.stdout:op 125 completed, throughput=5.01MB/sec 2024-04-07T16:39:00.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:39:00 smithi144 bash[66654]: cluster 2024-04-07T16:38:59.094177+0000 mgr.y (mgr.25175) 709 : cluster [DBG] pgmap v1351: 105 pgs: 105 active+clean; 126 GiB data, 3.1 GiB used, 712 GiB / 715 GiB avail; 3.4 MiB/s rd, 1.9 MiB/s wr, 5 op/s 2024-04-07T16:39:00.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:39:00 smithi100 bash[76202]: cluster 2024-04-07T16:38:59.094177+0000 mgr.y (mgr.25175) 709 : cluster [DBG] pgmap v1351: 105 pgs: 105 active+clean; 126 GiB data, 3.1 GiB used, 712 GiB / 715 GiB avail; 3.4 MiB/s rd, 1.9 MiB/s wr, 5 op/s 2024-04-07T16:39:00.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:39:00 smithi100 bash[74942]: cluster 2024-04-07T16:38:59.094177+0000 mgr.y (mgr.25175) 709 : cluster [DBG] pgmap v1351: 105 pgs: 105 active+clean; 126 GiB data, 3.1 GiB used, 712 GiB / 715 GiB avail; 3.4 MiB/s rd, 1.9 MiB/s wr, 5 op/s 2024-04-07T16:39:01.141 INFO:tasks.workunit.client.0.smithi100.stdout: 24: throughput=4.81MB/sec pending data=0 2024-04-07T16:39:01.141 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-tNuvhj6mpmC8Cy_ off=4344491 len=1132704 2024-04-07T16:39:01.144 INFO:tasks.workunit.client.0.smithi100.stdout:op 126 completed, throughput=4.85MB/sec 2024-04-07T16:39:01.144 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-AXotXCESeIkG7sE off=3544076 len=3589 2024-04-07T16:39:01.148 INFO:tasks.workunit.client.0.smithi100.stdout:op 127 completed, throughput=4.85MB/sec 2024-04-07T16:39:01.148 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-aQb9D0mrt5ct88E off=18539616 len=99078 2024-04-07T16:39:01.149 INFO:tasks.workunit.client.0.smithi100.stdout:op 128 completed, throughput=4.86MB/sec 2024-04-07T16:39:01.149 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-L0t4Y7zesU1TC-c off=8358198 len=651660 2024-04-07T16:39:01.156 INFO:tasks.workunit.client.0.smithi100.stdout:op 129 completed, throughput=4.88MB/sec 2024-04-07T16:39:01.157 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-8192HVTNRyXGZRU off=3908748 len=1678265 2024-04-07T16:39:01.160 INFO:tasks.workunit.client.0.smithi100.stdout:op 130 completed, throughput=4.94MB/sec 2024-04-07T16:39:01.161 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-xusAYN4eZERAKfc off=5340640 len=1811875 2024-04-07T16:39:01.175 INFO:tasks.workunit.client.0.smithi100.stdout:op 131 completed, throughput=5.01MB/sec 2024-04-07T16:39:02.178 INFO:tasks.workunit.client.0.smithi100.stdout: 25: throughput=4.82MB/sec pending data=0 2024-04-07T16:39:02.178 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-6wV_BTmQBUSIcxz off=16782850 len=173336 2024-04-07T16:39:02.178 INFO:tasks.workunit.client.0.smithi100.stdout:op 132 completed, throughput=4.82MB/sec 2024-04-07T16:39:02.178 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-vWSc9I0rUnDfLoT off=10384137 len=607000 2024-04-07T16:39:02.180 INFO:tasks.workunit.client.0.smithi100.stdout:op 133 completed, throughput=4.84MB/sec 2024-04-07T16:39:02.180 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-ny3-Q03teQi-f7E off=13490988 len=232073 2024-04-07T16:39:02.181 INFO:tasks.workunit.client.0.smithi100.stdout:op 134 completed, throughput=4.85MB/sec 2024-04-07T16:39:02.181 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-_2IfeULT2gWifG2 off=1491850 len=1292776 2024-04-07T16:39:02.193 INFO:tasks.workunit.client.0.smithi100.stdout:op 135 completed, throughput=4.9MB/sec 2024-04-07T16:39:02.193 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-j7mJk8FZvEDQqKD off=1456615 len=512443 2024-04-07T16:39:02.194 INFO:tasks.workunit.client.0.smithi100.stdout:op 136 completed, throughput=4.92MB/sec 2024-04-07T16:39:02.195 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Nr3Qzd1TsiOa3vB off=4768818 len=1448867 2024-04-07T16:39:02.207 INFO:tasks.workunit.client.0.smithi100.stdout:op 137 completed, throughput=4.97MB/sec 2024-04-07T16:39:02.207 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-6wUTfwVhWGF6qd4 off=4919291 len=1267702 2024-04-07T16:39:02.211 INFO:tasks.workunit.client.0.smithi100.stdout:op 138 completed, throughput=5.01MB/sec 2024-04-07T16:39:02.607 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:39:02 smithi144 bash[95233]: ts=2024-04-07T16:39:02.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:39:02.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:39:02 smithi144 bash[66654]: cluster 2024-04-07T16:39:01.095129+0000 mgr.y (mgr.25175) 710 : cluster [DBG] pgmap v1352: 105 pgs: 105 active+clean; 126 GiB data, 3.1 GiB used, 712 GiB / 715 GiB avail; 2.6 MiB/s rd, 1.9 MiB/s wr, 5 op/s 2024-04-07T16:39:02.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:39:02 smithi144 bash[66654]: audit 2024-04-07T16:39:01.754883+0000 mon.a (mon.0) 2048 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:39:02.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:39:02 smithi100 bash[74942]: cluster 2024-04-07T16:39:01.095129+0000 mgr.y (mgr.25175) 710 : cluster [DBG] pgmap v1352: 105 pgs: 105 active+clean; 126 GiB data, 3.1 GiB used, 712 GiB / 715 GiB avail; 2.6 MiB/s rd, 1.9 MiB/s wr, 5 op/s 2024-04-07T16:39:02.692 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:39:02 smithi100 bash[74942]: audit 2024-04-07T16:39:01.754883+0000 mon.a (mon.0) 2048 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:39:02.692 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:39:02 smithi100 bash[76202]: cluster 2024-04-07T16:39:01.095129+0000 mgr.y (mgr.25175) 710 : cluster [DBG] pgmap v1352: 105 pgs: 105 active+clean; 126 GiB data, 3.1 GiB used, 712 GiB / 715 GiB avail; 2.6 MiB/s rd, 1.9 MiB/s wr, 5 op/s 2024-04-07T16:39:02.692 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:39:02 smithi100 bash[76202]: audit 2024-04-07T16:39:01.754883+0000 mon.a (mon.0) 2048 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:39:03.211 INFO:tasks.workunit.client.0.smithi100.stdout: 26: throughput=4.83MB/sec pending data=0 2024-04-07T16:39:03.211 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-zsoPsoURjEzf8Ok off=21049336 len=1305446 2024-04-07T16:39:03.215 INFO:tasks.workunit.client.0.smithi100.stdout:op 139 completed, throughput=4.87MB/sec 2024-04-07T16:39:03.215 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Cv2H80K9nwfCqXU off=142005 len=1059279 2024-04-07T16:39:03.225 INFO:tasks.workunit.client.0.smithi100.stdout:op 140 completed, throughput=4.91MB/sec 2024-04-07T16:39:03.225 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-k2xS6oEJI3-5A-a off=21631383 len=1588813 2024-04-07T16:39:03.228 INFO:tasks.workunit.client.0.smithi100.stdout:op 141 completed, throughput=4.96MB/sec 2024-04-07T16:39:03.228 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-haf_qwYtuOaMisI off=9619606 len=712215 2024-04-07T16:39:03.231 INFO:tasks.workunit.client.0.smithi100.stdout:op 142 completed, throughput=4.99MB/sec 2024-04-07T16:39:03.231 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-9SXGDQoErsXFw_6 off=8078265 len=1986439 2024-04-07T16:39:03.236 INFO:tasks.workunit.client.0.smithi100.stdout:op 143 completed, throughput=5.06MB/sec 2024-04-07T16:39:04.236 INFO:tasks.workunit.client.0.smithi100.stdout: 27: throughput=4.88MB/sec pending data=0 2024-04-07T16:39:04.236 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-o_ZkWu2HKemOqvC off=1843541 len=1252477 2024-04-07T16:39:04.247 INFO:tasks.workunit.client.0.smithi100.stdout:op 144 completed, throughput=4.92MB/sec 2024-04-07T16:39:04.247 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-GspiDLC9Slb1xrt off=11109260 len=939418 2024-04-07T16:39:04.255 INFO:tasks.workunit.client.0.smithi100.stdout:op 145 completed, throughput=4.95MB/sec 2024-04-07T16:39:04.255 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-vdOhoszAsZ1keUN off=486192 len=65959 2024-04-07T16:39:04.256 INFO:tasks.workunit.client.0.smithi100.stdout:op 146 completed, throughput=4.95MB/sec 2024-04-07T16:39:04.256 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-D2YuI9ZZTFm_vsA off=16714 len=616930 2024-04-07T16:39:04.258 INFO:tasks.workunit.client.0.smithi100.stdout:op 147 completed, throughput=4.97MB/sec 2024-04-07T16:39:04.258 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-6M-mjlVhWZ1_IKR off=655837 len=581002 2024-04-07T16:39:04.264 INFO:tasks.workunit.client.0.smithi100.stdout:op 148 completed, throughput=4.99MB/sec 2024-04-07T16:39:04.265 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-gX2JACmIeJaztJj off=3781951 len=1256155 2024-04-07T16:39:04.275 INFO:tasks.workunit.client.0.smithi100.stdout:op 149 completed, throughput=5.03MB/sec 2024-04-07T16:39:04.606 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:39:04 smithi144 bash[66654]: cluster 2024-04-07T16:39:03.096227+0000 mgr.y (mgr.25175) 711 : cluster [DBG] pgmap v1353: 105 pgs: 105 active+clean; 126 GiB data, 3.1 GiB used, 712 GiB / 715 GiB avail; 3.0 MiB/s rd, 2.5 MiB/s wr, 5 op/s 2024-04-07T16:39:04.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:39:04 smithi100 bash[74942]: cluster 2024-04-07T16:39:03.096227+0000 mgr.y (mgr.25175) 711 : cluster [DBG] pgmap v1353: 105 pgs: 105 active+clean; 126 GiB data, 3.1 GiB used, 712 GiB / 715 GiB avail; 3.0 MiB/s rd, 2.5 MiB/s wr, 5 op/s 2024-04-07T16:39:04.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:39:04 smithi100 bash[76202]: cluster 2024-04-07T16:39:03.096227+0000 mgr.y (mgr.25175) 711 : cluster [DBG] pgmap v1353: 105 pgs: 105 active+clean; 126 GiB data, 3.1 GiB used, 712 GiB / 715 GiB avail; 3.0 MiB/s rd, 2.5 MiB/s wr, 5 op/s 2024-04-07T16:39:05.275 INFO:tasks.workunit.client.0.smithi100.stdout: 28: throughput=4.86MB/sec pending data=0 2024-04-07T16:39:05.275 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-o5pvSrMoXs4gWvD off=16758537 len=1094807 2024-04-07T16:39:05.278 INFO:tasks.workunit.client.0.smithi100.stdout:op 150 completed, throughput=4.89MB/sec 2024-04-07T16:39:05.278 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-k8sJFTBnBCFdShw off=12116225 len=1190881 2024-04-07T16:39:05.288 INFO:tasks.workunit.client.0.smithi100.stdout:op 151 completed, throughput=4.93MB/sec 2024-04-07T16:39:05.288 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-8ix1Nog73vn1mY0 off=7328257 len=115521 2024-04-07T16:39:05.293 INFO:tasks.workunit.client.0.smithi100.stdout:op 152 completed, throughput=4.93MB/sec 2024-04-07T16:39:05.294 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-HJeTya1A71C073u off=1245781 len=1270194 2024-04-07T16:39:05.297 INFO:tasks.workunit.client.0.smithi100.stdout:op 153 completed, throughput=4.97MB/sec 2024-04-07T16:39:05.297 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-jf1quTQblLS_yHW off=18376513 len=1036927 2024-04-07T16:39:05.298 INFO:tasks.workunit.client.0.smithi100.stdout:op 154 completed, throughput=5.01MB/sec 2024-04-07T16:39:06.299 INFO:tasks.workunit.client.0.smithi100.stdout: 29: throughput=4.84MB/sec pending data=0 2024-04-07T16:39:06.299 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-0PAhn7A4Vx3hR_E off=21773025 len=1852228 2024-04-07T16:39:06.305 INFO:tasks.workunit.client.0.smithi100.stdout:op 155 completed, throughput=4.9MB/sec 2024-04-07T16:39:06.305 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-KWQw_M7Soj8MXY_ off=1254082 len=579868 2024-04-07T16:39:06.307 INFO:tasks.workunit.client.0.smithi100.stdout:op 156 completed, throughput=4.92MB/sec 2024-04-07T16:39:06.307 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Cjjb_UgiS3m9-69 off=8842998 len=1332146 2024-04-07T16:39:06.319 INFO:tasks.workunit.client.0.smithi100.stdout:op 157 completed, throughput=4.96MB/sec 2024-04-07T16:39:06.319 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-yhQ-LNI-LJHOodz off=14755188 len=1803401 2024-04-07T16:39:06.324 INFO:tasks.workunit.client.0.smithi100.stdout:op 158 completed, throughput=5.01MB/sec 2024-04-07T16:39:06.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:39:06 smithi144 bash[66654]: cluster 2024-04-07T16:39:05.097280+0000 mgr.y (mgr.25175) 712 : cluster [DBG] pgmap v1354: 105 pgs: 105 active+clean; 126 GiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 2.4 MiB/s rd, 2.2 MiB/s wr, 5 op/s 2024-04-07T16:39:06.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:39:06 smithi100 bash[74942]: cluster 2024-04-07T16:39:05.097280+0000 mgr.y (mgr.25175) 712 : cluster [DBG] pgmap v1354: 105 pgs: 105 active+clean; 126 GiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 2.4 MiB/s rd, 2.2 MiB/s wr, 5 op/s 2024-04-07T16:39:06.692 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:39:06 smithi100 bash[76202]: cluster 2024-04-07T16:39:05.097280+0000 mgr.y (mgr.25175) 712 : cluster [DBG] pgmap v1354: 105 pgs: 105 active+clean; 126 GiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 2.4 MiB/s rd, 2.2 MiB/s wr, 5 op/s 2024-04-07T16:39:07.325 INFO:tasks.workunit.client.0.smithi100.stdout: 30: throughput=4.85MB/sec pending data=0 2024-04-07T16:39:07.325 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-muEz-DJ8favHmEh off=2567874 len=438889 2024-04-07T16:39:07.331 INFO:tasks.workunit.client.0.smithi100.stdout:op 159 completed, throughput=4.86MB/sec 2024-04-07T16:39:07.332 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-sUZXPGsWYcDvlwM off=5498568 len=548350 2024-04-07T16:39:07.333 INFO:tasks.workunit.client.0.smithi100.stdout:op 160 completed, throughput=4.88MB/sec 2024-04-07T16:39:07.333 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-EGn0x_TJcpcviAx off=2045496 len=627874 2024-04-07T16:39:07.340 INFO:tasks.workunit.client.0.smithi100.stdout:op 161 completed, throughput=4.9MB/sec 2024-04-07T16:39:07.340 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-UHFyViDLwJnjIjt off=5330048 len=603907 2024-04-07T16:39:07.341 INFO:tasks.workunit.client.0.smithi100.stdout:op 162 completed, throughput=4.92MB/sec 2024-04-07T16:39:07.342 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-p1QuEtUkkbQminD off=2323791 len=944518 2024-04-07T16:39:07.350 INFO:tasks.workunit.client.0.smithi100.stdout:op 163 completed, throughput=4.95MB/sec 2024-04-07T16:39:07.350 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-zJjwmyif8tbv3eM off=1524801 len=964558 2024-04-07T16:39:07.353 INFO:tasks.workunit.client.0.smithi100.stdout:op 164 completed, throughput=4.97MB/sec 2024-04-07T16:39:07.353 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-3rzV2Yy-27-0sI3 off=7871547 len=2009754 2024-04-07T16:39:07.369 INFO:tasks.workunit.client.0.smithi100.stdout:op 165 completed, throughput=5.03MB/sec 2024-04-07T16:39:08.370 INFO:tasks.workunit.client.0.smithi100.stdout: 31: throughput=4.88MB/sec pending data=0 2024-04-07T16:39:08.370 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-y4dFTRDOAzpaVTd off=2257173 len=739120 2024-04-07T16:39:08.377 INFO:tasks.workunit.client.0.smithi100.stdout:op 166 completed, throughput=4.9MB/sec 2024-04-07T16:39:08.377 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-A90r8H6hfHKqcZg off=7680358 len=1823 2024-04-07T16:39:08.381 INFO:tasks.workunit.client.0.smithi100.stdout:op 167 completed, throughput=4.9MB/sec 2024-04-07T16:39:08.382 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-UZS13PpE5mFTqkQ off=1865836 len=127315 2024-04-07T16:39:08.382 INFO:tasks.workunit.client.0.smithi100.stdout:op 168 completed, throughput=4.9MB/sec 2024-04-07T16:39:08.382 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-l6gNruN2aoBa5S9 off=1666554 len=1817736 2024-04-07T16:39:08.387 INFO:tasks.workunit.client.0.smithi100.stdout:op 169 completed, throughput=4.95MB/sec 2024-04-07T16:39:08.387 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-FAZ_rvWCNaGFqaT off=5764385 len=456631 2024-04-07T16:39:08.389 INFO:tasks.workunit.client.0.smithi100.stdout:op 170 completed, throughput=4.97MB/sec 2024-04-07T16:39:08.389 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-fH_dEKWiSLlAEfL off=9048035 len=1324491 2024-04-07T16:39:08.393 INFO:tasks.workunit.client.0.smithi100.stdout:op 171 completed, throughput=5.01MB/sec 2024-04-07T16:39:08.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:39:08 smithi144 bash[66654]: cluster 2024-04-07T16:39:07.098207+0000 mgr.y (mgr.25175) 713 : cluster [DBG] pgmap v1355: 105 pgs: 105 active+clean; 126 GiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 2.1 MiB/s rd, 2.0 MiB/s wr, 4 op/s 2024-04-07T16:39:08.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:39:08 smithi100 bash[74942]: cluster 2024-04-07T16:39:07.098207+0000 mgr.y (mgr.25175) 713 : cluster [DBG] pgmap v1355: 105 pgs: 105 active+clean; 126 GiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 2.1 MiB/s rd, 2.0 MiB/s wr, 4 op/s 2024-04-07T16:39:08.692 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:39:08 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:39:08] "GET /metrics HTTP/1.1" 200 34047 "" "Prometheus/2.43.0" 2024-04-07T16:39:08.692 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:39:08 smithi100 bash[76202]: cluster 2024-04-07T16:39:07.098207+0000 mgr.y (mgr.25175) 713 : cluster [DBG] pgmap v1355: 105 pgs: 105 active+clean; 126 GiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 2.1 MiB/s rd, 2.0 MiB/s wr, 4 op/s 2024-04-07T16:39:09.393 INFO:tasks.workunit.client.0.smithi100.stdout: 32: throughput=4.85MB/sec pending data=0 2024-04-07T16:39:09.393 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-xQqoZhTfiYVAlZ_ off=12631600 len=1174160 2024-04-07T16:39:09.397 INFO:tasks.workunit.client.0.smithi100.stdout:op 172 completed, throughput=4.89MB/sec 2024-04-07T16:39:09.397 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Ppu9JF4cH5GbOMc off=3218048 len=425807 2024-04-07T16:39:09.399 INFO:tasks.workunit.client.0.smithi100.stdout:op 173 completed, throughput=4.9MB/sec 2024-04-07T16:39:09.399 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-KhKNVjYVmDljK6Y off=5743336 len=1109671 2024-04-07T16:39:09.402 INFO:tasks.workunit.client.0.smithi100.stdout:op 174 completed, throughput=4.93MB/sec 2024-04-07T16:39:09.402 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-0ktyecprt5m1639 off=1706630 len=1624907 2024-04-07T16:39:09.416 INFO:tasks.workunit.client.0.smithi100.stdout:op 175 completed, throughput=4.98MB/sec 2024-04-07T16:39:09.416 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-HKWeO7Twz8OLH3P off=4103978 len=1099801 2024-04-07T16:39:09.425 INFO:tasks.workunit.client.0.smithi100.stdout:op 176 completed, throughput=5.01MB/sec 2024-04-07T16:39:10.425 INFO:tasks.workunit.client.0.smithi100.stdout: 33: throughput=4.86MB/sec pending data=0 2024-04-07T16:39:10.426 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-DF-fnjT0yEDBPgc off=532406 len=317715 2024-04-07T16:39:10.427 INFO:tasks.workunit.client.0.smithi100.stdout:op 177 completed, throughput=4.87MB/sec 2024-04-07T16:39:10.428 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-b1l3BwmSfC4Xib5 off=279925 len=671727 2024-04-07T16:39:10.435 INFO:tasks.workunit.client.0.smithi100.stdout:op 178 completed, throughput=4.89MB/sec 2024-04-07T16:39:10.435 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-0qA8i5QOsmcyqIp off=5300177 len=247155 2024-04-07T16:39:10.437 INFO:tasks.workunit.client.0.smithi100.stdout:op 179 completed, throughput=4.89MB/sec 2024-04-07T16:39:10.437 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-JbJM_RUybl6vaWc off=5999641 len=1180265 2024-04-07T16:39:10.441 INFO:tasks.workunit.client.0.smithi100.stdout:op 180 completed, throughput=4.92MB/sec 2024-04-07T16:39:10.441 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-KqdtBHzI0IUUGOr off=4001622 len=420487 2024-04-07T16:39:10.447 INFO:tasks.workunit.client.0.smithi100.stdout:op 181 completed, throughput=4.94MB/sec 2024-04-07T16:39:10.448 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-OZkGczhF52_waps off=15018158 len=1578183 2024-04-07T16:39:10.462 INFO:tasks.workunit.client.0.smithi100.stdout:op 182 completed, throughput=4.98MB/sec 2024-04-07T16:39:10.462 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-o5AbREU7-gVAHGP off=7898679 len=1918038 2024-04-07T16:39:10.466 INFO:tasks.workunit.client.0.smithi100.stdout:op 183 completed, throughput=5.03MB/sec 2024-04-07T16:39:10.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:39:10 smithi144 bash[66654]: cluster 2024-04-07T16:39:09.099464+0000 mgr.y (mgr.25175) 714 : cluster [DBG] pgmap v1356: 105 pgs: 105 active+clean; 126 GiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 2.9 MiB/s rd, 2.4 MiB/s wr, 6 op/s 2024-04-07T16:39:10.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:39:10 smithi100 bash[74942]: cluster 2024-04-07T16:39:09.099464+0000 mgr.y (mgr.25175) 714 : cluster [DBG] pgmap v1356: 105 pgs: 105 active+clean; 126 GiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 2.9 MiB/s rd, 2.4 MiB/s wr, 6 op/s 2024-04-07T16:39:10.692 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:39:10 smithi100 bash[76202]: cluster 2024-04-07T16:39:09.099464+0000 mgr.y (mgr.25175) 714 : cluster [DBG] pgmap v1356: 105 pgs: 105 active+clean; 126 GiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 2.9 MiB/s rd, 2.4 MiB/s wr, 6 op/s 2024-04-07T16:39:11.467 INFO:tasks.workunit.client.0.smithi100.stdout: 34: throughput=4.89MB/sec pending data=0 2024-04-07T16:39:11.467 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Qi-T627t4f2luLc off=12581076 len=1198266 2024-04-07T16:39:11.477 INFO:tasks.workunit.client.0.smithi100.stdout:op 184 completed, throughput=4.92MB/sec 2024-04-07T16:39:11.478 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-bmVMeLkidnNyG0B off=17373745 len=1658571 2024-04-07T16:39:11.482 INFO:tasks.workunit.client.0.smithi100.stdout:op 185 completed, throughput=4.96MB/sec 2024-04-07T16:39:11.482 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Ljjkb9H2cfjGKQF off=4613453 len=1762186 2024-04-07T16:39:11.486 INFO:tasks.workunit.client.0.smithi100.stdout:op 186 completed, throughput=5.01MB/sec 2024-04-07T16:39:12.486 INFO:tasks.workunit.client.0.smithi100.stdout: 35: throughput=4.87MB/sec pending data=0 2024-04-07T16:39:12.487 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-JCNNgDv40oL9wO0 off=2108175 len=1328228 2024-04-07T16:39:12.490 INFO:tasks.workunit.client.0.smithi100.stdout:op 187 completed, throughput=4.91MB/sec 2024-04-07T16:39:12.490 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-cj0nrFHyBr-yCOO off=4922742 len=811864 2024-04-07T16:39:12.493 INFO:tasks.workunit.client.0.smithi100.stdout:op 188 completed, throughput=4.93MB/sec 2024-04-07T16:39:12.493 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-UwOWdapwdQFXR5S off=11183450 len=2057111 2024-04-07T16:39:12.498 INFO:tasks.workunit.client.0.smithi100.stdout:op 189 completed, throughput=4.98MB/sec 2024-04-07T16:39:12.498 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-l9k9L6qT97OBVUh off=12695254 len=183766 2024-04-07T16:39:12.500 INFO:tasks.workunit.client.0.smithi100.stdout:op 190 completed, throughput=4.99MB/sec 2024-04-07T16:39:12.500 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-1q0r7VEH1J5t8Z3 off=11595167 len=1008093 2024-04-07T16:39:12.509 INFO:tasks.workunit.client.0.smithi100.stdout:op 191 completed, throughput=5.01MB/sec 2024-04-07T16:39:12.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:39:12 smithi144 bash[66654]: cluster 2024-04-07T16:39:11.100467+0000 mgr.y (mgr.25175) 715 : cluster [DBG] pgmap v1357: 105 pgs: 105 active+clean; 126 GiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.3 MiB/s wr, 5 op/s 2024-04-07T16:39:12.607 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:39:12 smithi144 bash[95233]: ts=2024-04-07T16:39:12.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:39:12.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:39:12 smithi100 bash[74942]: cluster 2024-04-07T16:39:11.100467+0000 mgr.y (mgr.25175) 715 : cluster [DBG] pgmap v1357: 105 pgs: 105 active+clean; 126 GiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.3 MiB/s wr, 5 op/s 2024-04-07T16:39:12.692 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:39:12 smithi100 bash[76202]: cluster 2024-04-07T16:39:11.100467+0000 mgr.y (mgr.25175) 715 : cluster [DBG] pgmap v1357: 105 pgs: 105 active+clean; 126 GiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.3 MiB/s wr, 5 op/s 2024-04-07T16:39:13.510 INFO:tasks.workunit.client.0.smithi100.stdout: 36: throughput=4.88MB/sec pending data=0 2024-04-07T16:39:13.510 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-HsUzjY_FglBWzq9 off=4393641 len=1113621 2024-04-07T16:39:13.520 INFO:tasks.workunit.client.0.smithi100.stdout:op 192 completed, throughput=4.9MB/sec 2024-04-07T16:39:13.520 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-RMzMiibhvV2p6zI off=413257 len=944975 2024-04-07T16:39:13.523 INFO:tasks.workunit.client.0.smithi100.stdout:op 193 completed, throughput=4.93MB/sec 2024-04-07T16:39:13.523 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-2VeEYm7CMRja8SQ off=12627637 len=710077 2024-04-07T16:39:13.525 INFO:tasks.workunit.client.0.smithi100.stdout:op 194 completed, throughput=4.94MB/sec 2024-04-07T16:39:13.525 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-6gMpYpD1sBmbOZt off=962201 len=1269783 2024-04-07T16:39:13.529 INFO:tasks.workunit.client.0.smithi100.stdout:op 195 completed, throughput=4.98MB/sec 2024-04-07T16:39:13.529 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-ee2yYBacRdTQt2S off=2235033 len=571900 2024-04-07T16:39:13.531 INFO:tasks.workunit.client.0.smithi100.stdout:op 196 completed, throughput=4.99MB/sec 2024-04-07T16:39:13.531 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-1CJ3VpajJsKUBVw off=4436931 len=1760708 2024-04-07T16:39:13.535 INFO:tasks.workunit.client.0.smithi100.stdout:op 197 completed, throughput=5.04MB/sec 2024-04-07T16:39:14.536 INFO:tasks.workunit.client.0.smithi100.stdout: 37: throughput=4.9MB/sec pending data=0 2024-04-07T16:39:14.536 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-TutMSvDMlks9Vyf off=11914041 len=656711 2024-04-07T16:39:14.544 INFO:tasks.workunit.client.0.smithi100.stdout:op 198 completed, throughput=4.92MB/sec 2024-04-07T16:39:14.544 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-wBxtAiKLl3z4E69 off=4189574 len=1620187 2024-04-07T16:39:14.558 INFO:tasks.workunit.client.0.smithi100.stdout:op 199 completed, throughput=4.96MB/sec 2024-04-07T16:39:14.558 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-z9XxZ0cTPj1ONpB off=1673272 len=1274030 2024-04-07T16:39:14.568 INFO:tasks.workunit.client.0.smithi100.stdout:op 200 completed, throughput=4.99MB/sec 2024-04-07T16:39:14.568 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-5YaW8Gs5WvXXogV off=5424205 len=224671 2024-04-07T16:39:14.573 INFO:tasks.workunit.client.0.smithi100.stdout:op 201 completed, throughput=4.99MB/sec 2024-04-07T16:39:14.573 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-ENuFKKyDhIcu_yB off=2418006 len=2035370 2024-04-07T16:39:14.578 INFO:tasks.workunit.client.0.smithi100.stdout:op 202 completed, throughput=5.04MB/sec 2024-04-07T16:39:14.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:39:14 smithi100 bash[74942]: cluster 2024-04-07T16:39:13.101581+0000 mgr.y (mgr.25175) 716 : cluster [DBG] pgmap v1358: 105 pgs: 105 active+clean; 126 GiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 3.1 MiB/s rd, 2.6 MiB/s wr, 6 op/s 2024-04-07T16:39:14.692 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:39:14 smithi100 bash[76202]: cluster 2024-04-07T16:39:13.101581+0000 mgr.y (mgr.25175) 716 : cluster [DBG] pgmap v1358: 105 pgs: 105 active+clean; 126 GiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 3.1 MiB/s rd, 2.6 MiB/s wr, 6 op/s 2024-04-07T16:39:14.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:39:14 smithi144 bash[66654]: cluster 2024-04-07T16:39:13.101581+0000 mgr.y (mgr.25175) 716 : cluster [DBG] pgmap v1358: 105 pgs: 105 active+clean; 126 GiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 3.1 MiB/s rd, 2.6 MiB/s wr, 6 op/s 2024-04-07T16:39:15.578 INFO:tasks.workunit.client.0.smithi100.stdout: 38: throughput=4.91MB/sec pending data=0 2024-04-07T16:39:15.578 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj--bPZgoo18yTDxLm off=9440468 len=895147 2024-04-07T16:39:15.586 INFO:tasks.workunit.client.0.smithi100.stdout:op 203 completed, throughput=4.94MB/sec 2024-04-07T16:39:15.586 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-RP67jXwRDGnMYnX off=12024002 len=1920202 2024-04-07T16:39:15.601 INFO:tasks.workunit.client.0.smithi100.stdout:op 204 completed, throughput=4.98MB/sec 2024-04-07T16:39:15.601 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-xBFvOP4icfsHfvE off=1913318 len=1027489 2024-04-07T16:39:15.605 INFO:tasks.workunit.client.0.smithi100.stdout:op 205 completed, throughput=5MB/sec 2024-04-07T16:39:16.606 INFO:tasks.workunit.client.0.smithi100.stdout: 39: throughput=4.88MB/sec pending data=0 2024-04-07T16:39:16.606 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-mVUbjcZpsSiQZRH off=5778338 len=1321237 2024-04-07T16:39:16.611 INFO:tasks.workunit.client.0.smithi100.stdout:op 206 completed, throughput=4.91MB/sec 2024-04-07T16:39:16.611 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-WiroigkPhqaG5qA off=6189334 len=358731 2024-04-07T16:39:16.617 INFO:tasks.workunit.client.0.smithi100.stdout:op 207 completed, throughput=4.92MB/sec 2024-04-07T16:39:16.617 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-k7PlgoHm7FfFGXz off=951232 len=1772964 2024-04-07T16:39:16.633 INFO:tasks.workunit.client.0.smithi100.stdout:op 208 completed, throughput=4.96MB/sec 2024-04-07T16:39:16.633 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Ds-3N6okSre6RAO off=3047586 len=737796 2024-04-07T16:39:16.640 INFO:tasks.workunit.client.0.smithi100.stdout:op 209 completed, throughput=4.98MB/sec 2024-04-07T16:39:16.640 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-3bTopyq5oHJM-OW off=10646400 len=367599 2024-04-07T16:39:16.646 INFO:tasks.workunit.client.0.smithi100.stdout:op 210 completed, throughput=4.98MB/sec 2024-04-07T16:39:16.646 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-m1px--0wwlysfC4 off=4022861 len=932908 2024-04-07T16:39:16.648 INFO:tasks.workunit.client.0.smithi100.stdout:op 211 completed, throughput=5.01MB/sec 2024-04-07T16:39:16.694 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:39:16 smithi100 bash[74942]: cluster 2024-04-07T16:39:15.102676+0000 mgr.y (mgr.25175) 717 : cluster [DBG] pgmap v1359: 105 pgs: 105 active+clean; 126 GiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 2.9 MiB/s rd, 1.8 MiB/s wr, 4 op/s 2024-04-07T16:39:16.694 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:39:16 smithi100 bash[76202]: cluster 2024-04-07T16:39:15.102676+0000 mgr.y (mgr.25175) 717 : cluster [DBG] pgmap v1359: 105 pgs: 105 active+clean; 126 GiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 2.9 MiB/s rd, 1.8 MiB/s wr, 4 op/s 2024-04-07T16:39:16.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:39:16 smithi144 bash[66654]: cluster 2024-04-07T16:39:15.102676+0000 mgr.y (mgr.25175) 717 : cluster [DBG] pgmap v1359: 105 pgs: 105 active+clean; 126 GiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 2.9 MiB/s rd, 1.8 MiB/s wr, 4 op/s 2024-04-07T16:39:17.649 INFO:tasks.workunit.client.0.smithi100.stdout: 40: throughput=4.88MB/sec pending data=0 2024-04-07T16:39:17.649 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-dVK2VF8CzhG2SL7 off=457094 len=1320935 2024-04-07T16:39:17.652 INFO:tasks.workunit.client.0.smithi100.stdout:op 212 completed, throughput=4.91MB/sec 2024-04-07T16:39:17.652 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Z0e66pB4UoBy3jZ off=13770772 len=595915 2024-04-07T16:39:17.659 INFO:tasks.workunit.client.0.smithi100.stdout:op 213 completed, throughput=4.93MB/sec 2024-04-07T16:39:17.659 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-EKF73lebc5ruY_7 off=4123340 len=1318264 2024-04-07T16:39:17.671 INFO:tasks.workunit.client.0.smithi100.stdout:op 214 completed, throughput=4.96MB/sec 2024-04-07T16:39:17.671 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-pV7VpTBwO8b8W9a off=766021 len=1048183 2024-04-07T16:39:17.674 INFO:tasks.workunit.client.0.smithi100.stdout:op 215 completed, throughput=4.98MB/sec 2024-04-07T16:39:17.674 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-h2_fFkG-96qg6Up off=16695463 len=1944779 2024-04-07T16:39:17.691 INFO:tasks.workunit.client.0.smithi100.stdout:op 216 completed, throughput=5.02MB/sec 2024-04-07T16:39:17.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:39:17 smithi100 bash[74942]: audit 2024-04-07T16:39:16.755262+0000 mon.a (mon.0) 2049 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:39:17.691 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:39:17 smithi100 bash[76202]: audit 2024-04-07T16:39:16.755262+0000 mon.a (mon.0) 2049 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:39:17.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:39:17 smithi144 bash[66654]: audit 2024-04-07T16:39:16.755262+0000 mon.a (mon.0) 2049 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:39:18.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:39:18 smithi100 bash[74942]: cluster 2024-04-07T16:39:17.103583+0000 mgr.y (mgr.25175) 718 : cluster [DBG] pgmap v1360: 105 pgs: 105 active+clean; 126 GiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 3.0 MiB/s rd, 1.8 MiB/s wr, 4 op/s 2024-04-07T16:39:18.692 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:39:18 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:39:18] "GET /metrics HTTP/1.1" 200 34054 "" "Prometheus/2.43.0" 2024-04-07T16:39:18.692 INFO:tasks.workunit.client.0.smithi100.stdout: 41: throughput=4.9MB/sec pending data=0 2024-04-07T16:39:18.692 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-BsLt8NJ2DNAe-Dc off=13554485 len=1420873 2024-04-07T16:39:18.692 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:39:18 smithi100 bash[76202]: cluster 2024-04-07T16:39:17.103583+0000 mgr.y (mgr.25175) 718 : cluster [DBG] pgmap v1360: 105 pgs: 105 active+clean; 126 GiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 3.0 MiB/s rd, 1.8 MiB/s wr, 4 op/s 2024-04-07T16:39:18.695 INFO:tasks.workunit.client.0.smithi100.stdout:op 217 completed, throughput=4.94MB/sec 2024-04-07T16:39:18.695 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-L8t6xiShJHWRpO4 off=7684876 len=207713 2024-04-07T16:39:18.697 INFO:tasks.workunit.client.0.smithi100.stdout:op 218 completed, throughput=4.94MB/sec 2024-04-07T16:39:18.697 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-5SBSsNhi-aqA7YJ off=1323117 len=522366 2024-04-07T16:39:18.699 INFO:tasks.workunit.client.0.smithi100.stdout:op 219 completed, throughput=4.95MB/sec 2024-04-07T16:39:18.699 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-zmpUK2Dm4GdwBc5 off=9014232 len=1569453 2024-04-07T16:39:18.713 INFO:tasks.workunit.client.0.smithi100.stdout:op 220 completed, throughput=4.99MB/sec 2024-04-07T16:39:18.713 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-mxFQVxMJeH_i-vq off=9682213 len=1594555 2024-04-07T16:39:18.726 INFO:tasks.workunit.client.0.smithi100.stdout:op 221 completed, throughput=5.02MB/sec 2024-04-07T16:39:18.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:39:18 smithi144 bash[66654]: cluster 2024-04-07T16:39:17.103583+0000 mgr.y (mgr.25175) 718 : cluster [DBG] pgmap v1360: 105 pgs: 105 active+clean; 126 GiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 3.0 MiB/s rd, 1.8 MiB/s wr, 4 op/s 2024-04-07T16:39:19.727 INFO:tasks.workunit.client.0.smithi100.stdout: 42: throughput=4.9MB/sec pending data=0 2024-04-07T16:39:19.727 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-gq84WXkoF4azY1J off=17496150 len=683402 2024-04-07T16:39:19.734 INFO:tasks.workunit.client.0.smithi100.stdout:op 222 completed, throughput=4.92MB/sec 2024-04-07T16:39:19.734 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-0CzGjGKh15WAQJ_ off=3561252 len=491671 2024-04-07T16:39:19.740 INFO:tasks.workunit.client.0.smithi100.stdout:op 223 completed, throughput=4.93MB/sec 2024-04-07T16:39:19.740 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-c3xCoTxTC5BurEa off=8839856 len=1050467 2024-04-07T16:39:19.743 INFO:tasks.workunit.client.0.smithi100.stdout:op 224 completed, throughput=4.95MB/sec 2024-04-07T16:39:19.743 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-IRyAwYAY9TAI3vp off=4713668 len=613755 2024-04-07T16:39:19.746 INFO:tasks.workunit.client.0.smithi100.stdout:op 225 completed, throughput=4.96MB/sec 2024-04-07T16:39:19.746 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-WMEv_zM5meoGQ3Z off=18467537 len=1795071 2024-04-07T16:39:19.753 INFO:tasks.workunit.client.0.smithi100.stdout:op 226 completed, throughput=5MB/sec 2024-04-07T16:39:20.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:39:20 smithi100 bash[74942]: cluster 2024-04-07T16:39:19.104546+0000 mgr.y (mgr.25175) 719 : cluster [DBG] pgmap v1361: 105 pgs: 105 active+clean; 126 GiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 3.5 MiB/s rd, 2.4 MiB/s wr, 5 op/s 2024-04-07T16:39:20.692 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:39:20 smithi100 bash[76202]: cluster 2024-04-07T16:39:19.104546+0000 mgr.y (mgr.25175) 719 : cluster [DBG] pgmap v1361: 105 pgs: 105 active+clean; 126 GiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 3.5 MiB/s rd, 2.4 MiB/s wr, 5 op/s 2024-04-07T16:39:20.753 INFO:tasks.workunit.client.0.smithi100.stdout: 43: throughput=4.89MB/sec pending data=0 2024-04-07T16:39:20.753 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-pRaNb5d6gURvKSp off=4613778 len=830955 2024-04-07T16:39:20.755 INFO:tasks.workunit.client.0.smithi100.stdout:op 227 completed, throughput=4.91MB/sec 2024-04-07T16:39:20.756 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-fs6ghndg9db_ug8 off=4351435 len=739861 2024-04-07T16:39:20.758 INFO:tasks.workunit.client.0.smithi100.stdout:op 228 completed, throughput=4.92MB/sec 2024-04-07T16:39:20.758 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-2T-9Si85ts-ezgb off=6859318 len=855047 2024-04-07T16:39:20.767 INFO:tasks.workunit.client.0.smithi100.stdout:op 229 completed, throughput=4.94MB/sec 2024-04-07T16:39:20.767 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-H2NFNxmPwXwM8tf off=82167 len=145394 2024-04-07T16:39:20.768 INFO:tasks.workunit.client.0.smithi100.stdout:op 230 completed, throughput=4.94MB/sec 2024-04-07T16:39:20.768 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-4xL3NuHMbtVNig2 off=1475847 len=1067647 2024-04-07T16:39:20.771 INFO:tasks.workunit.client.0.smithi100.stdout:op 231 completed, throughput=4.97MB/sec 2024-04-07T16:39:20.771 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-3q3BOWqNsQYhKTZ off=4230992 len=1338572 2024-04-07T16:39:20.776 INFO:tasks.workunit.client.0.smithi100.stdout:op 232 completed, throughput=4.99MB/sec 2024-04-07T16:39:20.776 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-7-nKhCRtBkeP5EU off=7425612 len=1498727 2024-04-07T16:39:20.790 INFO:tasks.workunit.client.0.smithi100.stdout:op 233 completed, throughput=5.02MB/sec 2024-04-07T16:39:20.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:39:20 smithi144 bash[66654]: cluster 2024-04-07T16:39:19.104546+0000 mgr.y (mgr.25175) 719 : cluster [DBG] pgmap v1361: 105 pgs: 105 active+clean; 126 GiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 3.5 MiB/s rd, 2.4 MiB/s wr, 5 op/s 2024-04-07T16:39:21.790 INFO:tasks.workunit.client.0.smithi100.stdout: 44: throughput=4.91MB/sec pending data=0 2024-04-07T16:39:21.790 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-6hLk5THpV0EB27Y off=3326766 len=1293513 2024-04-07T16:39:21.793 INFO:tasks.workunit.client.0.smithi100.stdout:op 234 completed, throughput=4.94MB/sec 2024-04-07T16:39:21.794 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-F8v75dTmaCZ1xpS off=10521086 len=812027 2024-04-07T16:39:21.797 INFO:tasks.workunit.client.0.smithi100.stdout:op 235 completed, throughput=4.96MB/sec 2024-04-07T16:39:21.797 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-m54rOB0L5zTcS8Q off=3598792 len=1984813 2024-04-07T16:39:21.803 INFO:tasks.workunit.client.0.smithi100.stdout:op 236 completed, throughput=5MB/sec 2024-04-07T16:39:21.803 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-qhN-SqJJEOmafeq off=0 len=1084669 2024-04-07T16:39:21.808 INFO:tasks.workunit.client.0.smithi100.stdout:op 237 completed, throughput=5.02MB/sec 2024-04-07T16:39:22.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:39:22 smithi144 bash[66654]: cluster 2024-04-07T16:39:21.105478+0000 mgr.y (mgr.25175) 720 : cluster [DBG] pgmap v1362: 105 pgs: 105 active+clean; 126 GiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 2.8 MiB/s rd, 2.0 MiB/s wr, 4 op/s 2024-04-07T16:39:22.607 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:39:22 smithi144 bash[95233]: ts=2024-04-07T16:39:22.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:39:22.692 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:39:22 smithi100 bash[74942]: cluster 2024-04-07T16:39:21.105478+0000 mgr.y (mgr.25175) 720 : cluster [DBG] pgmap v1362: 105 pgs: 105 active+clean; 126 GiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 2.8 MiB/s rd, 2.0 MiB/s wr, 4 op/s 2024-04-07T16:39:22.692 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:39:22 smithi100 bash[76202]: cluster 2024-04-07T16:39:21.105478+0000 mgr.y (mgr.25175) 720 : cluster [DBG] pgmap v1362: 105 pgs: 105 active+clean; 126 GiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 2.8 MiB/s rd, 2.0 MiB/s wr, 4 op/s 2024-04-07T16:39:22.808 INFO:tasks.workunit.client.0.smithi100.stdout: 45: throughput=4.91MB/sec pending data=0 2024-04-07T16:39:22.808 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-sqUJqrU5aWTSunI off=1086434 len=1922992 2024-04-07T16:39:22.815 INFO:tasks.workunit.client.0.smithi100.stdout:op 238 completed, throughput=4.95MB/sec 2024-04-07T16:39:22.815 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-JTpL3rG0BWd8HYJ off=1483416 len=1528071 2024-04-07T16:39:22.828 INFO:tasks.workunit.client.0.smithi100.stdout:op 239 completed, throughput=4.98MB/sec 2024-04-07T16:39:22.828 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-IUfYdHgvFHpBxKv off=2156976 len=2020201 2024-04-07T16:39:22.833 INFO:tasks.workunit.client.0.smithi100.stdout:op 240 completed, throughput=5.02MB/sec 2024-04-07T16:39:23.833 INFO:tasks.workunit.client.0.smithi100.stdout: 46: throughput=4.92MB/sec pending data=0 2024-04-07T16:39:23.833 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-FVIObqHKHtyrX_K off=371090 len=892844 2024-04-07T16:39:23.841 INFO:tasks.workunit.client.0.smithi100.stdout:op 241 completed, throughput=4.93MB/sec 2024-04-07T16:39:23.841 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-yCc1eqVh6qcobXa off=6530224 len=583345 2024-04-07T16:39:23.844 INFO:tasks.workunit.client.0.smithi100.stdout:op 242 completed, throughput=4.95MB/sec 2024-04-07T16:39:23.844 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-VusNmfwhcPK5lCf off=11546553 len=165941 2024-04-07T16:39:23.849 INFO:tasks.workunit.client.0.smithi100.stdout:op 243 completed, throughput=4.95MB/sec 2024-04-07T16:39:23.849 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-7EbFtxoDSnEVszi off=1711726 len=2036419 2024-04-07T16:39:23.854 INFO:tasks.workunit.client.0.smithi100.stdout:op 244 completed, throughput=4.99MB/sec 2024-04-07T16:39:23.854 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-4iLJHb6zFX7z4P3 off=675208 len=951360 2024-04-07T16:39:23.857 INFO:tasks.workunit.client.0.smithi100.stdout:op 245 completed, throughput=5.01MB/sec 2024-04-07T16:39:24.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:39:24 smithi100 bash[74942]: cluster 2024-04-07T16:39:23.106589+0000 mgr.y (mgr.25175) 721 : cluster [DBG] pgmap v1363: 105 pgs: 105 active+clean; 126 GiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 3.6 MiB/s rd, 2.4 MiB/s wr, 5 op/s 2024-04-07T16:39:24.692 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:39:24 smithi100 bash[76202]: cluster 2024-04-07T16:39:23.106589+0000 mgr.y (mgr.25175) 721 : cluster [DBG] pgmap v1363: 105 pgs: 105 active+clean; 126 GiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 3.6 MiB/s rd, 2.4 MiB/s wr, 5 op/s 2024-04-07T16:39:24.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:39:24 smithi144 bash[66654]: cluster 2024-04-07T16:39:23.106589+0000 mgr.y (mgr.25175) 721 : cluster [DBG] pgmap v1363: 105 pgs: 105 active+clean; 126 GiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 3.6 MiB/s rd, 2.4 MiB/s wr, 5 op/s 2024-04-07T16:39:24.857 INFO:tasks.workunit.client.0.smithi100.stdout: 47: throughput=4.9MB/sec pending data=0 2024-04-07T16:39:24.857 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-WmGMRaluCeCNig_ off=6199474 len=1257841 2024-04-07T16:39:24.862 INFO:tasks.workunit.client.0.smithi100.stdout:op 246 completed, throughput=4.93MB/sec 2024-04-07T16:39:24.862 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-qIV_bGAs9GgIND1 off=7959652 len=310967 2024-04-07T16:39:24.868 INFO:tasks.workunit.client.0.smithi100.stdout:op 247 completed, throughput=4.93MB/sec 2024-04-07T16:39:24.868 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-S4Y4A_5B8Tl1bLF off=22019085 len=771398 2024-04-07T16:39:24.876 INFO:tasks.workunit.client.0.smithi100.stdout:op 248 completed, throughput=4.95MB/sec 2024-04-07T16:39:24.876 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-VMFYdUiz_ucslkB off=8932033 len=1871005 2024-04-07T16:39:24.891 INFO:tasks.workunit.client.0.smithi100.stdout:op 249 completed, throughput=4.98MB/sec 2024-04-07T16:39:24.892 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-q60-evvjIX5C6eV off=2623724 len=70461 2024-04-07T16:39:24.896 INFO:tasks.workunit.client.0.smithi100.stdout:op 250 completed, throughput=4.98MB/sec 2024-04-07T16:39:24.896 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-7IHFVvV2S-kyKRC off=7189998 len=1770801 2024-04-07T16:39:24.909 INFO:tasks.workunit.client.0.smithi100.stdout:op 251 completed, throughput=5.02MB/sec 2024-04-07T16:39:25.910 INFO:tasks.workunit.client.0.smithi100.stdout: 48: throughput=4.92MB/sec pending data=0 2024-04-07T16:39:25.910 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-UAgT_WXuwyvSN6v off=3666473 len=1778684 2024-04-07T16:39:25.914 INFO:tasks.workunit.client.0.smithi100.stdout:op 252 completed, throughput=4.95MB/sec 2024-04-07T16:39:25.914 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-kSIYalJ4ytAdnku off=6614963 len=869296 2024-04-07T16:39:25.917 INFO:tasks.workunit.client.0.smithi100.stdout:op 253 completed, throughput=4.97MB/sec 2024-04-07T16:39:25.917 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-MKCgV_RMGitR_Au off=1443661 len=350295 2024-04-07T16:39:25.923 INFO:tasks.workunit.client.0.smithi100.stdout:op 254 completed, throughput=4.97MB/sec 2024-04-07T16:39:25.923 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-y6vJbnWgUHbBLhC off=4751021 len=988134 2024-04-07T16:39:25.932 INFO:tasks.workunit.client.0.smithi100.stdout:op 255 completed, throughput=4.99MB/sec 2024-04-07T16:39:25.932 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-fzkeuQssuphPHDc off=9488620 len=673402 2024-04-07T16:39:25.933 INFO:tasks.workunit.client.0.smithi100.stdout:op 256 completed, throughput=5MB/sec 2024-04-07T16:39:26.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:39:26 smithi100 bash[74942]: cluster 2024-04-07T16:39:25.107621+0000 mgr.y (mgr.25175) 722 : cluster [DBG] pgmap v1364: 105 pgs: 105 active+clean; 126 GiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 3.0 MiB/s rd, 2.0 MiB/s wr, 4 op/s 2024-04-07T16:39:26.692 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:39:26 smithi100 bash[76202]: cluster 2024-04-07T16:39:25.107621+0000 mgr.y (mgr.25175) 722 : cluster [DBG] pgmap v1364: 105 pgs: 105 active+clean; 126 GiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 3.0 MiB/s rd, 2.0 MiB/s wr, 4 op/s 2024-04-07T16:39:26.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:39:26 smithi144 bash[66654]: cluster 2024-04-07T16:39:25.107621+0000 mgr.y (mgr.25175) 722 : cluster [DBG] pgmap v1364: 105 pgs: 105 active+clean; 126 GiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 3.0 MiB/s rd, 2.0 MiB/s wr, 4 op/s 2024-04-07T16:39:26.934 INFO:tasks.workunit.client.0.smithi100.stdout: 49: throughput=4.9MB/sec pending data=0 2024-04-07T16:39:26.934 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-kq6TiS3r44MJDA5 off=6891590 len=784369 2024-04-07T16:39:26.936 INFO:tasks.workunit.client.0.smithi100.stdout:op 257 completed, throughput=4.92MB/sec 2024-04-07T16:39:26.937 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-f7vqVu_Vm2Yi9Z4 off=298256 len=1084374 2024-04-07T16:39:26.945 INFO:tasks.workunit.client.0.smithi100.stdout:op 258 completed, throughput=4.94MB/sec 2024-04-07T16:39:26.945 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-JdieS7DhG5i-pEw off=3613735 len=572955 2024-04-07T16:39:26.947 INFO:tasks.workunit.client.0.smithi100.stdout:op 259 completed, throughput=4.95MB/sec 2024-04-07T16:39:26.947 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-9PlJSZtInIkEu5O off=622500 len=1274021 2024-04-07T16:39:26.957 INFO:tasks.workunit.client.0.smithi100.stdout:op 260 completed, throughput=4.97MB/sec 2024-04-07T16:39:26.957 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-y0tTKM7--KacKYr off=6973525 len=79252 2024-04-07T16:39:26.958 INFO:tasks.workunit.client.0.smithi100.stdout:op 261 completed, throughput=4.97MB/sec 2024-04-07T16:39:26.959 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-8b9lnN5W_FjwGsA off=4743307 len=300484 2024-04-07T16:39:26.960 INFO:tasks.workunit.client.0.smithi100.stdout:op 262 completed, throughput=4.98MB/sec 2024-04-07T16:39:26.960 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-ThBAY9_eJ3piftp off=1027269 len=1211280 2024-04-07T16:39:26.971 INFO:tasks.workunit.client.0.smithi100.stdout:op 263 completed, throughput=5MB/sec 2024-04-07T16:39:27.971 INFO:tasks.workunit.client.0.smithi100.stdout: 50: throughput=4.9MB/sec pending data=0 2024-04-07T16:39:27.971 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-ssjrDWC9ZSh09d3 off=1097428 len=405123 2024-04-07T16:39:27.973 INFO:tasks.workunit.client.0.smithi100.stdout:op 264 completed, throughput=4.91MB/sec 2024-04-07T16:39:27.973 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-ttyJsc_IRzeYy6P off=11853650 len=1756780 2024-04-07T16:39:27.977 INFO:tasks.workunit.client.0.smithi100.stdout:op 265 completed, throughput=4.94MB/sec 2024-04-07T16:39:27.977 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-zVLUaCiCcwi1qx9 off=11394671 len=1799422 2024-04-07T16:39:27.991 INFO:tasks.workunit.client.0.smithi100.stdout:op 266 completed, throughput=4.98MB/sec 2024-04-07T16:39:27.992 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-S2DeamDdMZPBiTZ off=6724952 len=1318915 2024-04-07T16:39:27.996 INFO:tasks.workunit.client.0.smithi100.stdout:op 267 completed, throughput=5MB/sec 2024-04-07T16:39:27.996 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-A4mKyZZuCUY6HkI off=10742480 len=912091 2024-04-07T16:39:27.998 INFO:tasks.workunit.client.0.smithi100.stdout:op 268 completed, throughput=5.02MB/sec 2024-04-07T16:39:28.692 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:39:28 smithi100 bash[74942]: cluster 2024-04-07T16:39:27.108494+0000 mgr.y (mgr.25175) 723 : cluster [DBG] pgmap v1365: 105 pgs: 105 active+clean; 126 GiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 2.3 MiB/s rd, 1.9 MiB/s wr, 4 op/s 2024-04-07T16:39:28.692 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:39:28 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:39:28] "GET /metrics HTTP/1.1" 200 34054 "" "Prometheus/2.43.0" 2024-04-07T16:39:28.692 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:39:28 smithi100 bash[76202]: cluster 2024-04-07T16:39:27.108494+0000 mgr.y (mgr.25175) 723 : cluster [DBG] pgmap v1365: 105 pgs: 105 active+clean; 126 GiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 2.3 MiB/s rd, 1.9 MiB/s wr, 4 op/s 2024-04-07T16:39:28.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:39:28 smithi144 bash[66654]: cluster 2024-04-07T16:39:27.108494+0000 mgr.y (mgr.25175) 723 : cluster [DBG] pgmap v1365: 105 pgs: 105 active+clean; 126 GiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 2.3 MiB/s rd, 1.9 MiB/s wr, 4 op/s 2024-04-07T16:39:28.999 INFO:tasks.workunit.client.0.smithi100.stdout: 51: throughput=4.92MB/sec pending data=0 2024-04-07T16:39:28.999 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-ARkO-CnTiu8aK5_ off=1120975 len=143681 2024-04-07T16:39:29.004 INFO:tasks.workunit.client.0.smithi100.stdout:op 269 completed, throughput=4.92MB/sec 2024-04-07T16:39:29.004 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-PvSaHVt91pB9cFy off=3312726 len=1681390 2024-04-07T16:39:29.008 INFO:tasks.workunit.client.0.smithi100.stdout:op 270 completed, throughput=4.95MB/sec 2024-04-07T16:39:29.008 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-1v9Sg_IM6Y9mWW7 off=9972830 len=1862899 2024-04-07T16:39:29.024 INFO:tasks.workunit.client.0.smithi100.stdout:op 271 completed, throughput=4.98MB/sec 2024-04-07T16:39:29.024 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-O1BF9jnrVRlfQGp off=2291300 len=1761173 2024-04-07T16:39:29.039 INFO:tasks.workunit.client.0.smithi100.stdout:op 272 completed, throughput=5.02MB/sec 2024-04-07T16:39:30.040 INFO:tasks.workunit.client.0.smithi100.stdout: 52: throughput=4.92MB/sec pending data=0 2024-04-07T16:39:30.040 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-WxGXHrHoK6C6mSo off=4201939 len=1863497 2024-04-07T16:39:30.054 INFO:tasks.workunit.client.0.smithi100.stdout:op 273 completed, throughput=4.95MB/sec 2024-04-07T16:39:30.054 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-pKURkn-d3NC0Ef3 off=3281357 len=1891779 2024-04-07T16:39:30.069 INFO:tasks.workunit.client.0.smithi100.stdout:op 274 completed, throughput=4.99MB/sec 2024-04-07T16:39:30.069 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-vaHLD23yt1CQbPe off=531871 len=1139530 2024-04-07T16:39:30.079 INFO:tasks.workunit.client.0.smithi100.stdout:op 275 completed, throughput=5.01MB/sec 2024-04-07T16:39:30.692 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:39:30 smithi100 bash[74942]: cluster 2024-04-07T16:39:29.109555+0000 mgr.y (mgr.25175) 724 : cluster [DBG] pgmap v1366: 105 pgs: 105 active+clean; 126 GiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.1 MiB/s wr, 4 op/s 2024-04-07T16:39:30.692 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:39:30 smithi100 bash[76202]: cluster 2024-04-07T16:39:29.109555+0000 mgr.y (mgr.25175) 724 : cluster [DBG] pgmap v1366: 105 pgs: 105 active+clean; 126 GiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.1 MiB/s wr, 4 op/s 2024-04-07T16:39:30.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:39:30 smithi144 bash[66654]: cluster 2024-04-07T16:39:29.109555+0000 mgr.y (mgr.25175) 724 : cluster [DBG] pgmap v1366: 105 pgs: 105 active+clean; 126 GiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.1 MiB/s wr, 4 op/s 2024-04-07T16:39:31.079 INFO:tasks.workunit.client.0.smithi100.stdout: 53: throughput=4.91MB/sec pending data=0 2024-04-07T16:39:31.079 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-rFjdUG0TQmPf3Pt off=2407676 len=1553678 2024-04-07T16:39:31.092 INFO:tasks.workunit.client.0.smithi100.stdout:op 276 completed, throughput=4.94MB/sec 2024-04-07T16:39:31.092 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-oIbF1z5oTFG0kUN off=3266450 len=563613 2024-04-07T16:39:31.095 INFO:tasks.workunit.client.0.smithi100.stdout:op 277 completed, throughput=4.95MB/sec 2024-04-07T16:39:31.095 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-RRF2uNIQZp8tRSk off=2421410 len=1911148 2024-04-07T16:39:31.112 INFO:tasks.workunit.client.0.smithi100.stdout:op 278 completed, throughput=4.98MB/sec 2024-04-07T16:39:31.112 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-c9uM1SmtG3DNjgr off=2066147 len=1759171 2024-04-07T16:39:31.116 INFO:tasks.workunit.client.0.smithi100.stdout:op 279 completed, throughput=5.01MB/sec 2024-04-07T16:39:32.116 INFO:tasks.workunit.client.0.smithi100.stdout: 54: throughput=4.92MB/sec pending data=0 2024-04-07T16:39:32.116 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-vIbg1mDOoo6qsGC off=10836990 len=65246 2024-04-07T16:39:32.121 INFO:tasks.workunit.client.0.smithi100.stdout:op 280 completed, throughput=4.92MB/sec 2024-04-07T16:39:32.121 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-NJ5WKTEPnZmoIhd off=1465758 len=672175 2024-04-07T16:39:32.124 INFO:tasks.workunit.client.0.smithi100.stdout:op 281 completed, throughput=4.93MB/sec 2024-04-07T16:39:32.124 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-heZSPYHBCtYeNkv off=1137508 len=272868 2024-04-07T16:39:32.125 INFO:tasks.workunit.client.0.smithi100.stdout:op 282 completed, throughput=4.94MB/sec 2024-04-07T16:39:32.125 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-ax7ylhHjEPof7_1 off=2123309 len=1765781 2024-04-07T16:39:32.140 INFO:tasks.workunit.client.0.smithi100.stdout:op 283 completed, throughput=4.97MB/sec 2024-04-07T16:39:32.141 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-CQ2QXL8jgH5-SfJ off=20847307 len=925751 2024-04-07T16:39:32.150 INFO:tasks.workunit.client.0.smithi100.stdout:op 284 completed, throughput=4.98MB/sec 2024-04-07T16:39:32.150 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-HRHvyN5kyzMhvuI off=3944682 len=1811705 2024-04-07T16:39:32.166 INFO:tasks.workunit.client.0.smithi100.stdout:op 285 completed, throughput=5.01MB/sec 2024-04-07T16:39:32.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:39:32 smithi144 bash[66654]: cluster 2024-04-07T16:39:31.110434+0000 mgr.y (mgr.25175) 725 : cluster [DBG] pgmap v1367: 105 pgs: 105 active+clean; 126 GiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 2.5 MiB/s rd, 1.9 MiB/s wr, 4 op/s 2024-04-07T16:39:32.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:39:32 smithi144 bash[66654]: audit 2024-04-07T16:39:31.755721+0000 mon.a (mon.0) 2050 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:39:32.607 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:39:32 smithi144 bash[95233]: ts=2024-04-07T16:39:32.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:39:32.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:39:32 smithi100 bash[74942]: cluster 2024-04-07T16:39:31.110434+0000 mgr.y (mgr.25175) 725 : cluster [DBG] pgmap v1367: 105 pgs: 105 active+clean; 126 GiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 2.5 MiB/s rd, 1.9 MiB/s wr, 4 op/s 2024-04-07T16:39:32.696 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:39:32 smithi100 bash[74942]: audit 2024-04-07T16:39:31.755721+0000 mon.a (mon.0) 2050 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:39:32.697 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:39:32 smithi100 bash[76202]: cluster 2024-04-07T16:39:31.110434+0000 mgr.y (mgr.25175) 725 : cluster [DBG] pgmap v1367: 105 pgs: 105 active+clean; 126 GiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 2.5 MiB/s rd, 1.9 MiB/s wr, 4 op/s 2024-04-07T16:39:32.697 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:39:32 smithi100 bash[76202]: audit 2024-04-07T16:39:31.755721+0000 mon.a (mon.0) 2050 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:39:33.166 INFO:tasks.workunit.client.0.smithi100.stdout: 55: throughput=4.92MB/sec pending data=0 2024-04-07T16:39:33.166 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-dLk6d3891PjXtWI off=12003432 len=793693 2024-04-07T16:39:33.174 INFO:tasks.workunit.client.0.smithi100.stdout:op 286 completed, throughput=4.94MB/sec 2024-04-07T16:39:33.174 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-MDURDsMBy5Enobc off=12074506 len=1861798 2024-04-07T16:39:33.178 INFO:tasks.workunit.client.0.smithi100.stdout:op 287 completed, throughput=4.97MB/sec 2024-04-07T16:39:33.178 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-n_BHjIT7Efcs97p off=11753430 len=1094426 2024-04-07T16:39:33.181 INFO:tasks.workunit.client.0.smithi100.stdout:op 288 completed, throughput=4.98MB/sec 2024-04-07T16:39:33.181 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-xkgflNorcmAmTJb off=11650651 len=1988053 2024-04-07T16:39:33.186 INFO:tasks.workunit.client.0.smithi100.stdout:op 289 completed, throughput=5.02MB/sec 2024-04-07T16:39:34.186 INFO:tasks.workunit.client.0.smithi100.stdout: 56: throughput=4.93MB/sec pending data=0 2024-04-07T16:39:34.186 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-uz3aSEc7RY2wLGa off=3750803 len=379012 2024-04-07T16:39:34.188 INFO:tasks.workunit.client.0.smithi100.stdout:op 290 completed, throughput=4.94MB/sec 2024-04-07T16:39:34.188 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-1oZ-rsBq8ICnUcI off=4818486 len=55853 2024-04-07T16:39:34.193 INFO:tasks.workunit.client.0.smithi100.stdout:op 291 completed, throughput=4.94MB/sec 2024-04-07T16:39:34.193 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-ES-bSEHarHdaSvk off=9513463 len=7975 2024-04-07T16:39:34.193 INFO:tasks.workunit.client.0.smithi100.stdout:op 292 completed, throughput=4.94MB/sec 2024-04-07T16:39:34.194 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Nrmxdtp830CRwh2 off=3363512 len=2039807 2024-04-07T16:39:34.200 INFO:tasks.workunit.client.0.smithi100.stdout:op 293 completed, throughput=4.97MB/sec 2024-04-07T16:39:34.200 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-8uKjV_kCIXWpegd off=632009 len=145282 2024-04-07T16:39:34.202 INFO:tasks.workunit.client.0.smithi100.stdout:op 294 completed, throughput=4.97MB/sec 2024-04-07T16:39:34.202 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-uxKy9D_h0ZwpiBv off=6151549 len=2083362 2024-04-07T16:39:34.217 INFO:tasks.workunit.client.0.smithi100.stdout:op 295 completed, throughput=5.01MB/sec 2024-04-07T16:39:34.692 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:39:34 smithi100 bash[74942]: cluster 2024-04-07T16:39:33.111560+0000 mgr.y (mgr.25175) 726 : cluster [DBG] pgmap v1368: 105 pgs: 105 active+clean; 126 GiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 3.0 MiB/s rd, 2.4 MiB/s wr, 5 op/s 2024-04-07T16:39:34.692 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:39:34 smithi100 bash[76202]: cluster 2024-04-07T16:39:33.111560+0000 mgr.y (mgr.25175) 726 : cluster [DBG] pgmap v1368: 105 pgs: 105 active+clean; 126 GiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 3.0 MiB/s rd, 2.4 MiB/s wr, 5 op/s 2024-04-07T16:39:34.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:39:34 smithi144 bash[66654]: cluster 2024-04-07T16:39:33.111560+0000 mgr.y (mgr.25175) 726 : cluster [DBG] pgmap v1368: 105 pgs: 105 active+clean; 126 GiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 3.0 MiB/s rd, 2.4 MiB/s wr, 5 op/s 2024-04-07T16:39:35.217 INFO:tasks.workunit.client.0.smithi100.stdout: 57: throughput=4.92MB/sec pending data=0 2024-04-07T16:39:35.217 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-5b17040NpY80Jh4 off=6442627 len=2075498 2024-04-07T16:39:35.233 INFO:tasks.workunit.client.0.smithi100.stdout:op 296 completed, throughput=4.95MB/sec 2024-04-07T16:39:35.233 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Li8Brok9cEvlYr1 off=8636987 len=2018499 2024-04-07T16:39:35.238 INFO:tasks.workunit.client.0.smithi100.stdout:op 297 completed, throughput=4.99MB/sec 2024-04-07T16:39:35.238 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-R_fIn6XI088aUWN off=12126036 len=559390 2024-04-07T16:39:35.240 INFO:tasks.workunit.client.0.smithi100.stdout:op 298 completed, throughput=4.99MB/sec 2024-04-07T16:39:35.240 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-zNRmgZEp9mFJgj_ off=10365038 len=1675393 2024-04-07T16:39:35.242 INFO:tasks.workunit.client.0.smithi100.stdout:op 299 completed, throughput=5.02MB/sec 2024-04-07T16:39:36.243 INFO:tasks.workunit.client.0.smithi100.stdout: 58: throughput=4.94MB/sec pending data=0 2024-04-07T16:39:36.243 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-mn_O6CaB18A-hFL off=11797382 len=862100 2024-04-07T16:39:36.251 INFO:tasks.workunit.client.0.smithi100.stdout:op 300 completed, throughput=4.95MB/sec 2024-04-07T16:39:36.251 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-L0RCsy5XLor5WAK off=17539708 len=1828650 2024-04-07T16:39:36.265 INFO:tasks.workunit.client.0.smithi100.stdout:op 301 completed, throughput=4.98MB/sec 2024-04-07T16:39:36.265 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-9qIZ5oTBO0WhOR8 off=401030 len=1439107 2024-04-07T16:39:36.267 INFO:tasks.workunit.client.0.smithi100.stdout:op 302 completed, throughput=5MB/sec 2024-04-07T16:39:36.692 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:39:36 smithi100 bash[74942]: cluster 2024-04-07T16:39:35.112491+0000 mgr.y (mgr.25175) 727 : cluster [DBG] pgmap v1369: 105 pgs: 105 active+clean; 126 GiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 1.9 MiB/s rd, 2.4 MiB/s wr, 4 op/s 2024-04-07T16:39:36.692 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:39:36 smithi100 bash[76202]: cluster 2024-04-07T16:39:35.112491+0000 mgr.y (mgr.25175) 727 : cluster [DBG] pgmap v1369: 105 pgs: 105 active+clean; 126 GiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 1.9 MiB/s rd, 2.4 MiB/s wr, 4 op/s 2024-04-07T16:39:36.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:39:36 smithi144 bash[66654]: cluster 2024-04-07T16:39:35.112491+0000 mgr.y (mgr.25175) 727 : cluster [DBG] pgmap v1369: 105 pgs: 105 active+clean; 126 GiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 1.9 MiB/s rd, 2.4 MiB/s wr, 4 op/s 2024-04-07T16:39:37.268 INFO:tasks.workunit.client.0.smithi100.stdout: 59: throughput=4.92MB/sec pending data=0 2024-04-07T16:39:37.268 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-4I3NS55mYgRevgN off=1650167 len=1653997 2024-04-07T16:39:37.281 INFO:tasks.workunit.client.0.smithi100.stdout:op 303 completed, throughput=4.94MB/sec 2024-04-07T16:39:37.281 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-SW3Bo0oVb3pzeMS off=6968346 len=1246277 2024-04-07T16:39:37.284 INFO:tasks.workunit.client.0.smithi100.stdout:op 304 completed, throughput=4.96MB/sec 2024-04-07T16:39:37.284 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-9dEjHE1m9yEuO7L off=1667751 len=1202845 2024-04-07T16:39:37.294 INFO:tasks.workunit.client.0.smithi100.stdout:op 305 completed, throughput=4.98MB/sec 2024-04-07T16:39:37.294 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-paabQjjoz8Duvci off=7239567 len=1359534 2024-04-07T16:39:37.299 INFO:tasks.workunit.client.0.smithi100.stdout:op 306 completed, throughput=5MB/sec 2024-04-07T16:39:38.299 INFO:tasks.workunit.client.0.smithi100.stdout: 60: throughput=4.92MB/sec pending data=0 2024-04-07T16:39:38.299 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-HDLXCco6ILra8-v off=268262 len=1415022 2024-04-07T16:39:38.303 INFO:tasks.workunit.client.0.smithi100.stdout:op 307 completed, throughput=4.94MB/sec 2024-04-07T16:39:38.303 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-0qR0if3azSq4PRB off=8141697 len=168448 2024-04-07T16:39:38.308 INFO:tasks.workunit.client.0.smithi100.stdout:op 308 completed, throughput=4.94MB/sec 2024-04-07T16:39:38.308 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-KyyJHvAQYR0-k0e off=987370 len=750270 2024-04-07T16:39:38.310 INFO:tasks.workunit.client.0.smithi100.stdout:op 309 completed, throughput=4.96MB/sec 2024-04-07T16:39:38.310 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-K82qOOVsdDqfYHx off=1982043 len=401600 2024-04-07T16:39:38.316 INFO:tasks.workunit.client.0.smithi100.stdout:op 310 completed, throughput=4.96MB/sec 2024-04-07T16:39:38.316 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-PB60wvQyRKwwIV5 off=11492814 len=1097644 2024-04-07T16:39:38.325 INFO:tasks.workunit.client.0.smithi100.stdout:op 311 completed, throughput=4.98MB/sec 2024-04-07T16:39:38.326 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-UYi5jEO6QapMmfU off=5295152 len=2073397 2024-04-07T16:39:38.330 INFO:tasks.workunit.client.0.smithi100.stdout:op 312 completed, throughput=5.01MB/sec 2024-04-07T16:39:38.692 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:39:38 smithi100 bash[74942]: cluster 2024-04-07T16:39:37.113342+0000 mgr.y (mgr.25175) 728 : cluster [DBG] pgmap v1370: 105 pgs: 105 active+clean; 126 GiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 2.2 MiB/s rd, 2.3 MiB/s wr, 4 op/s 2024-04-07T16:39:38.692 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:39:38 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:39:38] "GET /metrics HTTP/1.1" 200 34052 "" "Prometheus/2.43.0" 2024-04-07T16:39:38.692 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:39:38 smithi100 bash[76202]: cluster 2024-04-07T16:39:37.113342+0000 mgr.y (mgr.25175) 728 : cluster [DBG] pgmap v1370: 105 pgs: 105 active+clean; 126 GiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 2.2 MiB/s rd, 2.3 MiB/s wr, 4 op/s 2024-04-07T16:39:38.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:39:38 smithi144 bash[66654]: cluster 2024-04-07T16:39:37.113342+0000 mgr.y (mgr.25175) 728 : cluster [DBG] pgmap v1370: 105 pgs: 105 active+clean; 126 GiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 2.2 MiB/s rd, 2.3 MiB/s wr, 4 op/s 2024-04-07T16:39:39.331 INFO:tasks.workunit.client.0.smithi100.stdout: 61: throughput=4.93MB/sec pending data=0 2024-04-07T16:39:39.331 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-ZZulThY5RKlyal1 off=11327489 len=313534 2024-04-07T16:39:39.336 INFO:tasks.workunit.client.0.smithi100.stdout:op 313 completed, throughput=4.93MB/sec 2024-04-07T16:39:39.336 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-g_z35p8GSOwpxU2 off=6057585 len=421755 2024-04-07T16:39:39.342 INFO:tasks.workunit.client.0.smithi100.stdout:op 314 completed, throughput=4.94MB/sec 2024-04-07T16:39:39.342 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-55r5O44x-oOB7lr off=1662629 len=99992 2024-04-07T16:39:39.343 INFO:tasks.workunit.client.0.smithi100.stdout:op 315 completed, throughput=4.94MB/sec 2024-04-07T16:39:39.343 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-C6ZP2HwG2CsRwpp off=12804487 len=1164416 2024-04-07T16:39:39.352 INFO:tasks.workunit.client.0.smithi100.stdout:op 316 completed, throughput=4.96MB/sec 2024-04-07T16:39:39.352 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-SJZJP_q4mmvsA8v off=12648745 len=1653428 2024-04-07T16:39:39.356 INFO:tasks.workunit.client.0.smithi100.stdout:op 317 completed, throughput=4.98MB/sec 2024-04-07T16:39:39.356 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-lp14Lf1jWizfMmm off=230443 len=1590426 2024-04-07T16:39:39.366 INFO:tasks.workunit.client.0.smithi100.stdout:op 318 completed, throughput=5.01MB/sec 2024-04-07T16:39:40.366 INFO:tasks.workunit.client.0.smithi100.stdout: 62: throughput=4.93MB/sec pending data=0 2024-04-07T16:39:40.366 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-EiGVnJ-ZgA3jVg6 off=1672656 len=714021 2024-04-07T16:39:40.374 INFO:tasks.workunit.client.0.smithi100.stdout:op 319 completed, throughput=4.94MB/sec 2024-04-07T16:39:40.374 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-_W-ixq7Iey9Iwsy off=3376750 len=1505627 2024-04-07T16:39:40.386 INFO:tasks.workunit.client.0.smithi100.stdout:op 320 completed, throughput=4.96MB/sec 2024-04-07T16:39:40.386 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-bPHyPsM_YVQloKu off=4928485 len=382495 2024-04-07T16:39:40.388 INFO:tasks.workunit.client.0.smithi100.stdout:op 321 completed, throughput=4.97MB/sec 2024-04-07T16:39:40.388 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-ZX4H9YnYXjB-dvP off=934130 len=1973896 2024-04-07T16:39:40.401 INFO:tasks.workunit.client.0.smithi100.stdout:op 322 completed, throughput=4.99MB/sec 2024-04-07T16:39:40.401 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj--50W1L6FIWNerJE off=946942 len=1340782 2024-04-07T16:39:40.406 INFO:tasks.workunit.client.0.smithi100.stdout:op 323 completed, throughput=5.01MB/sec 2024-04-07T16:39:40.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:39:40 smithi100 bash[74942]: cluster 2024-04-07T16:39:39.114368+0000 mgr.y (mgr.25175) 729 : cluster [DBG] pgmap v1371: 105 pgs: 105 active+clean; 126 GiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 2.6 MiB/s rd, 2.5 MiB/s wr, 4 op/s 2024-04-07T16:39:40.692 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:39:40 smithi100 bash[76202]: cluster 2024-04-07T16:39:39.114368+0000 mgr.y (mgr.25175) 729 : cluster [DBG] pgmap v1371: 105 pgs: 105 active+clean; 126 GiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 2.6 MiB/s rd, 2.5 MiB/s wr, 4 op/s 2024-04-07T16:39:40.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:39:40 smithi144 bash[66654]: cluster 2024-04-07T16:39:39.114368+0000 mgr.y (mgr.25175) 729 : cluster [DBG] pgmap v1371: 105 pgs: 105 active+clean; 126 GiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 2.6 MiB/s rd, 2.5 MiB/s wr, 4 op/s 2024-04-07T16:39:41.406 INFO:tasks.workunit.client.0.smithi100.stdout: 63: throughput=4.94MB/sec pending data=0 2024-04-07T16:39:41.407 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-kgiL_oB6hptD2k2 off=6910773 len=1531547 2024-04-07T16:39:41.410 INFO:tasks.workunit.client.0.smithi100.stdout:op 324 completed, throughput=4.96MB/sec 2024-04-07T16:39:41.410 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Vu1kciuLxwpwPsh off=15708348 len=272133 2024-04-07T16:39:41.416 INFO:tasks.workunit.client.0.smithi100.stdout:op 325 completed, throughput=4.96MB/sec 2024-04-07T16:39:41.416 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-jXuCpeITSOu7OWu off=1079627 len=1437972 2024-04-07T16:39:41.420 INFO:tasks.workunit.client.0.smithi100.stdout:op 326 completed, throughput=4.98MB/sec 2024-04-07T16:39:41.420 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-QiXONvOaGAH-Fho off=0 len=1320505 2024-04-07T16:39:41.430 INFO:tasks.workunit.client.0.smithi100.stdout:op 327 completed, throughput=5MB/sec 2024-04-07T16:39:42.431 INFO:tasks.workunit.client.0.smithi100.stdout: 64: throughput=4.93MB/sec pending data=0 2024-04-07T16:39:42.431 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-axmnCYg_HPfzRaD off=13771085 len=1495232 2024-04-07T16:39:42.435 INFO:tasks.workunit.client.0.smithi100.stdout:op 328 completed, throughput=4.95MB/sec 2024-04-07T16:39:42.435 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-La_LWlq_TK-yA3A off=11792064 len=858191 2024-04-07T16:39:42.437 INFO:tasks.workunit.client.0.smithi100.stdout:op 329 completed, throughput=4.96MB/sec 2024-04-07T16:39:42.437 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-rGJcMliULM_M51o off=14697397 len=168100 2024-04-07T16:39:42.445 INFO:tasks.workunit.client.0.smithi100.stdout:op 330 completed, throughput=4.96MB/sec 2024-04-07T16:39:42.445 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-SJZJP_q4mmvsA8v off=106913 len=832673 2024-04-07T16:39:42.448 INFO:tasks.workunit.client.0.smithi100.stdout:op 331 completed, throughput=4.97MB/sec 2024-04-07T16:39:42.448 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-IEfeeS14Egieper off=4968141 len=981731 2024-04-07T16:39:42.451 INFO:tasks.workunit.client.0.smithi100.stdout:op 332 completed, throughput=4.99MB/sec 2024-04-07T16:39:42.451 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-npE7Kns2609jlXz off=5126622 len=386011 2024-04-07T16:39:42.456 INFO:tasks.workunit.client.0.smithi100.stdout:op 333 completed, throughput=4.99MB/sec 2024-04-07T16:39:42.456 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-UmYX9a2sa-JIA8_ off=11067192 len=838881 2024-04-07T16:39:42.458 INFO:tasks.workunit.client.0.smithi100.stdout:op 334 completed, throughput=5MB/sec 2024-04-07T16:39:42.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:39:42 smithi144 bash[66654]: cluster 2024-04-07T16:39:41.115286+0000 mgr.y (mgr.25175) 730 : cluster [DBG] pgmap v1372: 105 pgs: 105 active+clean; 126 GiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.8 MiB/s wr, 4 op/s 2024-04-07T16:39:42.607 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:39:42 smithi144 bash[95233]: ts=2024-04-07T16:39:42.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:39:42.942 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:39:42 smithi100 bash[74942]: cluster 2024-04-07T16:39:41.115286+0000 mgr.y (mgr.25175) 730 : cluster [DBG] pgmap v1372: 105 pgs: 105 active+clean; 126 GiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.8 MiB/s wr, 4 op/s 2024-04-07T16:39:42.942 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:39:42 smithi100 bash[76202]: cluster 2024-04-07T16:39:41.115286+0000 mgr.y (mgr.25175) 730 : cluster [DBG] pgmap v1372: 105 pgs: 105 active+clean; 126 GiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.8 MiB/s wr, 4 op/s 2024-04-07T16:39:43.459 INFO:tasks.workunit.client.0.smithi100.stdout: 65: throughput=4.93MB/sec pending data=0 2024-04-07T16:39:43.459 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-jTJaSfqTOtitYiJ off=10289055 len=957433 2024-04-07T16:39:43.463 INFO:tasks.workunit.client.0.smithi100.stdout:op 335 completed, throughput=4.94MB/sec 2024-04-07T16:39:43.463 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-OgnXIfOk_fkD3UG off=768425 len=1201673 2024-04-07T16:39:43.474 INFO:tasks.workunit.client.0.smithi100.stdout:op 336 completed, throughput=4.96MB/sec 2024-04-07T16:39:43.474 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-pwinG92FBPZBKOT off=743115 len=1048385 2024-04-07T16:39:43.476 INFO:tasks.workunit.client.0.smithi100.stdout:op 337 completed, throughput=4.97MB/sec 2024-04-07T16:39:43.476 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-rtU2G65dn5FrYk1 off=3592511 len=1577652 2024-04-07T16:39:43.487 INFO:tasks.workunit.client.0.smithi100.stdout:op 338 completed, throughput=4.99MB/sec 2024-04-07T16:39:43.487 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-_VyCqsjHaQC8L2b off=2496197 len=653727 2024-04-07T16:39:43.495 INFO:tasks.workunit.client.0.smithi100.stdout:op 339 completed, throughput=5MB/sec 2024-04-07T16:39:44.496 INFO:tasks.workunit.client.0.smithi100.stdout: 66: throughput=4.93MB/sec pending data=0 2024-04-07T16:39:44.496 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-sJHK5g5q6P20r_7 off=1913337 len=187040 2024-04-07T16:39:44.501 INFO:tasks.workunit.client.0.smithi100.stdout:op 340 completed, throughput=4.93MB/sec 2024-04-07T16:39:44.501 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-YDH115OH5PDjHtI off=3998428 len=1176035 2024-04-07T16:39:44.511 INFO:tasks.workunit.client.0.smithi100.stdout:op 341 completed, throughput=4.95MB/sec 2024-04-07T16:39:44.511 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Cb4cdmmjUhR1wdO off=6502702 len=1678705 2024-04-07T16:39:44.515 INFO:tasks.workunit.client.0.smithi100.stdout:op 342 completed, throughput=4.97MB/sec 2024-04-07T16:39:44.515 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-tEhVnmgXBpARhu_ off=15758133 len=1644333 2024-04-07T16:39:44.528 INFO:tasks.workunit.client.0.smithi100.stdout:op 343 completed, throughput=4.99MB/sec 2024-04-07T16:39:44.528 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-oplAW_1Grjc2RAf off=0 len=1069444 2024-04-07T16:39:44.537 INFO:tasks.workunit.client.0.smithi100.stdout:op 344 completed, throughput=5.01MB/sec 2024-04-07T16:39:44.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:39:44 smithi144 bash[66654]: cluster 2024-04-07T16:39:43.116402+0000 mgr.y (mgr.25175) 731 : cluster [DBG] pgmap v1373: 105 pgs: 105 active+clean; 126 GiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 2.9 MiB/s rd, 2.6 MiB/s wr, 4 op/s 2024-04-07T16:39:44.942 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:39:44 smithi100 bash[74942]: cluster 2024-04-07T16:39:43.116402+0000 mgr.y (mgr.25175) 731 : cluster [DBG] pgmap v1373: 105 pgs: 105 active+clean; 126 GiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 2.9 MiB/s rd, 2.6 MiB/s wr, 4 op/s 2024-04-07T16:39:44.942 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:39:44 smithi100 bash[76202]: cluster 2024-04-07T16:39:43.116402+0000 mgr.y (mgr.25175) 731 : cluster [DBG] pgmap v1373: 105 pgs: 105 active+clean; 126 GiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 2.9 MiB/s rd, 2.6 MiB/s wr, 4 op/s 2024-04-07T16:39:45.537 INFO:tasks.workunit.client.0.smithi100.stdout: 67: throughput=4.94MB/sec pending data=0 2024-04-07T16:39:45.538 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-fHJ59I8eqGnFjuE off=8377076 len=593775 2024-04-07T16:39:45.540 INFO:tasks.workunit.client.0.smithi100.stdout:op 345 completed, throughput=4.94MB/sec 2024-04-07T16:39:45.540 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-uSVrIga34xNf_IR off=11046805 len=757393 2024-04-07T16:39:45.542 INFO:tasks.workunit.client.0.smithi100.stdout:op 346 completed, throughput=4.95MB/sec 2024-04-07T16:39:45.542 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj--mo1yHRmyc0KNAt off=967222 len=1290643 2024-04-07T16:39:45.554 INFO:tasks.workunit.client.0.smithi100.stdout:op 347 completed, throughput=4.97MB/sec 2024-04-07T16:39:45.554 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj---_K_e6KZ6IuHOX off=5761106 len=1672137 2024-04-07T16:39:45.560 INFO:tasks.workunit.client.0.smithi100.stdout:op 348 completed, throughput=4.99MB/sec 2024-04-07T16:39:45.560 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-48wfLHbSMU4TyX1 off=4760716 len=1672593 2024-04-07T16:39:45.573 INFO:tasks.workunit.client.0.smithi100.stdout:op 349 completed, throughput=5.02MB/sec 2024-04-07T16:39:46.574 INFO:tasks.workunit.client.0.smithi100.stdout: 68: throughput=4.94MB/sec pending data=0 2024-04-07T16:39:46.574 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-_LloczacYMvieyI off=1407892 len=888006 2024-04-07T16:39:46.582 INFO:tasks.workunit.client.0.smithi100.stdout:op 350 completed, throughput=4.96MB/sec 2024-04-07T16:39:46.582 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-x3xcjTvz5Cn-OdF off=15536454 len=532054 2024-04-07T16:39:46.589 INFO:tasks.workunit.client.0.smithi100.stdout:op 351 completed, throughput=4.96MB/sec 2024-04-07T16:39:46.589 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-GCi-SeyAvErMajI off=6926382 len=978720 2024-04-07T16:39:46.593 INFO:tasks.workunit.client.0.smithi100.stdout:op 352 completed, throughput=4.97MB/sec 2024-04-07T16:39:46.593 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-r-rLTHvNUicrniW off=15806528 len=2060106 2024-04-07T16:39:46.611 INFO:tasks.workunit.client.0.smithi100.stdout:op 353 completed, throughput=5MB/sec 2024-04-07T16:39:46.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:39:46 smithi144 bash[66654]: cluster 2024-04-07T16:39:45.117381+0000 mgr.y (mgr.25175) 732 : cluster [DBG] pgmap v1374: 105 pgs: 105 active+clean; 126 GiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.4 MiB/s wr, 4 op/s 2024-04-07T16:39:46.942 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:39:46 smithi100 bash[74942]: cluster 2024-04-07T16:39:45.117381+0000 mgr.y (mgr.25175) 732 : cluster [DBG] pgmap v1374: 105 pgs: 105 active+clean; 126 GiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.4 MiB/s wr, 4 op/s 2024-04-07T16:39:46.942 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:39:46 smithi100 bash[76202]: cluster 2024-04-07T16:39:45.117381+0000 mgr.y (mgr.25175) 732 : cluster [DBG] pgmap v1374: 105 pgs: 105 active+clean; 126 GiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.4 MiB/s wr, 4 op/s 2024-04-07T16:39:47.611 INFO:tasks.workunit.client.0.smithi100.stdout: 69: throughput=4.93MB/sec pending data=0 2024-04-07T16:39:47.611 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-IuFO4i3dP_ZVxvq off=8763204 len=1542767 2024-04-07T16:39:47.625 INFO:tasks.workunit.client.0.smithi100.stdout:op 354 completed, throughput=4.95MB/sec 2024-04-07T16:39:47.625 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-JJz9QD0hYaDnGIT off=7269167 len=1559297 2024-04-07T16:39:47.643 INFO:tasks.workunit.client.0.smithi100.stdout:op 355 completed, throughput=4.97MB/sec 2024-04-07T16:39:47.643 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-sAvtoh6ezVB61UB off=2402100 len=722659 2024-04-07T16:39:47.651 INFO:tasks.workunit.client.0.smithi100.stdout:op 356 completed, throughput=4.98MB/sec 2024-04-07T16:39:47.651 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-8VbVz1mLcbYVPOi off=6009921 len=1404989 2024-04-07T16:39:47.661 INFO:tasks.workunit.client.0.smithi100.stdout:op 357 completed, throughput=5MB/sec 2024-04-07T16:39:47.662 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj--tMbzhquJM-RVI6 off=8013793 len=1045414 2024-04-07T16:39:47.664 INFO:tasks.workunit.client.0.smithi100.stdout:op 358 completed, throughput=5.01MB/sec 2024-04-07T16:39:47.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:39:47 smithi144 bash[66654]: audit 2024-04-07T16:39:46.756134+0000 mon.a (mon.0) 2051 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:39:47.942 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:39:47 smithi100 bash[74942]: audit 2024-04-07T16:39:46.756134+0000 mon.a (mon.0) 2051 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:39:47.942 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:39:47 smithi100 bash[76202]: audit 2024-04-07T16:39:46.756134+0000 mon.a (mon.0) 2051 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:39:48.664 INFO:tasks.workunit.client.0.smithi100.stdout: 70: throughput=4.94MB/sec pending data=0 2024-04-07T16:39:48.665 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-A9KHLud2TFLyUH1 off=5620865 len=29132 2024-04-07T16:39:48.665 INFO:tasks.workunit.client.0.smithi100.stdout:op 359 completed, throughput=4.94MB/sec 2024-04-07T16:39:48.665 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-jyBPoEwVcLfeQqz off=1703447 len=2007512 2024-04-07T16:39:48.680 INFO:tasks.workunit.client.0.smithi100.stdout:op 360 completed, throughput=4.97MB/sec 2024-04-07T16:39:48.681 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-NJ5KnJaGiFhPs63 off=7178295 len=671397 2024-04-07T16:39:48.688 INFO:tasks.workunit.client.0.smithi100.stdout:op 361 completed, throughput=4.98MB/sec 2024-04-07T16:39:48.688 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-HVn2e8o6ILeX9ZT off=4582126 len=1906059 2024-04-07T16:39:48.692 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:39:48 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:39:48] "GET /metrics HTTP/1.1" 200 34058 "" "Prometheus/2.43.0" 2024-04-07T16:39:48.692 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:39:48 smithi100 bash[74942]: cluster 2024-04-07T16:39:47.118188+0000 mgr.y (mgr.25175) 733 : cluster [DBG] pgmap v1375: 105 pgs: 105 active+clean; 126 GiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 3.0 MiB/s rd, 2.1 MiB/s wr, 4 op/s 2024-04-07T16:39:48.692 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:39:48 smithi100 bash[74942]: audit 2024-04-07T16:39:48.305794+0000 mon.a (mon.0) 2052 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:39:48.693 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:39:48 smithi100 bash[76202]: cluster 2024-04-07T16:39:47.118188+0000 mgr.y (mgr.25175) 733 : cluster [DBG] pgmap v1375: 105 pgs: 105 active+clean; 126 GiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 3.0 MiB/s rd, 2.1 MiB/s wr, 4 op/s 2024-04-07T16:39:48.693 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:39:48 smithi100 bash[76202]: audit 2024-04-07T16:39:48.305794+0000 mon.a (mon.0) 2052 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:39:48.693 INFO:tasks.workunit.client.0.smithi100.stdout:op 362 completed, throughput=5MB/sec 2024-04-07T16:39:48.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:39:48 smithi144 bash[66654]: cluster 2024-04-07T16:39:47.118188+0000 mgr.y (mgr.25175) 733 : cluster [DBG] pgmap v1375: 105 pgs: 105 active+clean; 126 GiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 3.0 MiB/s rd, 2.1 MiB/s wr, 4 op/s 2024-04-07T16:39:48.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:39:48 smithi144 bash[66654]: audit 2024-04-07T16:39:48.305794+0000 mon.a (mon.0) 2052 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:39:49.692 INFO:tasks.workunit.client.0.smithi100.stdout: 71: throughput=4.93MB/sec pending data=0 2024-04-07T16:39:49.692 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-PbCOJhnsVUV6eo4 off=7970894 len=991751 2024-04-07T16:39:49.701 INFO:tasks.workunit.client.0.smithi100.stdout:op 363 completed, throughput=4.95MB/sec 2024-04-07T16:39:49.701 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-RAl35_wwRrrRVNv off=5360371 len=1700466 2024-04-07T16:39:49.705 INFO:tasks.workunit.client.0.smithi100.stdout:op 364 completed, throughput=4.97MB/sec 2024-04-07T16:39:49.705 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-mrrV6IlScP7UaUa off=9162940 len=1353055 2024-04-07T16:39:49.717 INFO:tasks.workunit.client.0.smithi100.stdout:op 365 completed, throughput=4.98MB/sec 2024-04-07T16:39:49.717 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-t2Qo3Keii21iDgx off=6688723 len=267019 2024-04-07T16:39:49.718 INFO:tasks.workunit.client.0.smithi100.stdout:op 366 completed, throughput=4.99MB/sec 2024-04-07T16:39:49.718 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-s-Dd-9yzHO802x0 off=11861789 len=1609486 2024-04-07T16:39:49.729 INFO:tasks.workunit.client.0.smithi100.stdout:op 367 completed, throughput=5.01MB/sec 2024-04-07T16:39:49.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:39:49 smithi144 bash[66654]: audit 2024-04-07T16:39:49.043989+0000 mon.a (mon.0) 2053 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:39:49.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:39:49 smithi144 bash[66654]: audit 2024-04-07T16:39:49.045102+0000 mon.a (mon.0) 2054 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:39:49.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:39:49 smithi144 bash[66654]: audit 2024-04-07T16:39:49.053420+0000 mon.a (mon.0) 2055 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:39:49.942 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:39:49 smithi100 bash[74942]: audit 2024-04-07T16:39:49.043989+0000 mon.a (mon.0) 2053 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:39:49.942 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:39:49 smithi100 bash[74942]: audit 2024-04-07T16:39:49.045102+0000 mon.a (mon.0) 2054 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:39:49.942 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:39:49 smithi100 bash[74942]: audit 2024-04-07T16:39:49.053420+0000 mon.a (mon.0) 2055 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:39:49.942 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:39:49 smithi100 bash[76202]: audit 2024-04-07T16:39:49.043989+0000 mon.a (mon.0) 2053 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:39:49.943 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:39:49 smithi100 bash[76202]: audit 2024-04-07T16:39:49.045102+0000 mon.a (mon.0) 2054 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:39:49.943 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:39:49 smithi100 bash[76202]: audit 2024-04-07T16:39:49.053420+0000 mon.a (mon.0) 2055 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:39:50.730 INFO:tasks.workunit.client.0.smithi100.stdout: 72: throughput=4.94MB/sec pending data=0 2024-04-07T16:39:50.730 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-0e12GcfkOuvDalB off=240968 len=1506856 2024-04-07T16:39:50.734 INFO:tasks.workunit.client.0.smithi100.stdout:op 368 completed, throughput=4.96MB/sec 2024-04-07T16:39:50.734 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-il0vnnTkpT7ePKa off=1689302 len=707302 2024-04-07T16:39:50.736 INFO:tasks.workunit.client.0.smithi100.stdout:op 369 completed, throughput=4.97MB/sec 2024-04-07T16:39:50.737 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-xyEGJ5sC6TlTnRU off=5319014 len=1844800 2024-04-07T16:39:50.755 INFO:tasks.workunit.client.0.smithi100.stdout:op 370 completed, throughput=4.99MB/sec 2024-04-07T16:39:50.755 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-x4Taih1_dLowjF6 off=3666743 len=300160 2024-04-07T16:39:50.756 INFO:tasks.workunit.client.0.smithi100.stdout:op 371 completed, throughput=4.99MB/sec 2024-04-07T16:39:50.757 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-IyuCxmQ_UjUhcbM off=20545146 len=1384310 2024-04-07T16:39:50.761 INFO:tasks.workunit.client.0.smithi100.stdout:op 372 completed, throughput=5.01MB/sec 2024-04-07T16:39:50.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:39:50 smithi144 bash[66654]: cluster 2024-04-07T16:39:49.119236+0000 mgr.y (mgr.25175) 734 : cluster [DBG] pgmap v1376: 105 pgs: 105 active+clean; 126 GiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.5 MiB/s wr, 4 op/s 2024-04-07T16:39:50.942 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:39:50 smithi100 bash[74942]: cluster 2024-04-07T16:39:49.119236+0000 mgr.y (mgr.25175) 734 : cluster [DBG] pgmap v1376: 105 pgs: 105 active+clean; 126 GiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.5 MiB/s wr, 4 op/s 2024-04-07T16:39:50.942 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:39:50 smithi100 bash[76202]: cluster 2024-04-07T16:39:49.119236+0000 mgr.y (mgr.25175) 734 : cluster [DBG] pgmap v1376: 105 pgs: 105 active+clean; 126 GiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.5 MiB/s wr, 4 op/s 2024-04-07T16:39:51.762 INFO:tasks.workunit.client.0.smithi100.stdout: 73: throughput=4.95MB/sec pending data=0 2024-04-07T16:39:51.762 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-DE2BacrjyQqLZh5 off=16149834 len=1310053 2024-04-07T16:39:51.765 INFO:tasks.workunit.client.0.smithi100.stdout:op 373 completed, throughput=4.96MB/sec 2024-04-07T16:39:51.766 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-LlaWtBQNUZLpg-0 off=3596965 len=742663 2024-04-07T16:39:51.773 INFO:tasks.workunit.client.0.smithi100.stdout:op 374 completed, throughput=4.97MB/sec 2024-04-07T16:39:51.773 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-8RpxVHRDO_rDE_8 off=9026860 len=1083024 2024-04-07T16:39:51.783 INFO:tasks.workunit.client.0.smithi100.stdout:op 375 completed, throughput=4.98MB/sec 2024-04-07T16:39:51.784 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-zTDBpmKEq8QSBdN off=10116078 len=868226 2024-04-07T16:39:51.792 INFO:tasks.workunit.client.0.smithi100.stdout:op 376 completed, throughput=4.99MB/sec 2024-04-07T16:39:51.792 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Bcg7Hzw3CgqFrdk off=10062787 len=2001060 2024-04-07T16:39:51.809 INFO:tasks.workunit.client.0.smithi100.stdout:op 377 completed, throughput=5.02MB/sec 2024-04-07T16:39:52.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:39:52 smithi144 bash[66654]: cluster 2024-04-07T16:39:51.120196+0000 mgr.y (mgr.25175) 735 : cluster [DBG] pgmap v1377: 105 pgs: 105 active+clean; 126 GiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 2.4 MiB/s rd, 2.9 MiB/s wr, 5 op/s 2024-04-07T16:39:52.607 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:39:52 smithi144 bash[95233]: ts=2024-04-07T16:39:52.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:39:52.809 INFO:tasks.workunit.client.0.smithi100.stdout: 74: throughput=4.95MB/sec pending data=0 2024-04-07T16:39:52.809 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-eaU4jMi_rZlKkse off=4463421 len=1863504 2024-04-07T16:39:52.813 INFO:tasks.workunit.client.0.smithi100.stdout:op 378 completed, throughput=4.98MB/sec 2024-04-07T16:39:52.813 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-aynOBGMkXig5_hg off=3404988 len=2069345 2024-04-07T16:39:52.820 INFO:tasks.workunit.client.0.smithi100.stdout:op 379 completed, throughput=5MB/sec 2024-04-07T16:39:52.942 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:39:52 smithi100 bash[74942]: cluster 2024-04-07T16:39:51.120196+0000 mgr.y (mgr.25175) 735 : cluster [DBG] pgmap v1377: 105 pgs: 105 active+clean; 126 GiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 2.4 MiB/s rd, 2.9 MiB/s wr, 5 op/s 2024-04-07T16:39:52.942 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:39:52 smithi100 bash[76202]: cluster 2024-04-07T16:39:51.120196+0000 mgr.y (mgr.25175) 735 : cluster [DBG] pgmap v1377: 105 pgs: 105 active+clean; 126 GiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 2.4 MiB/s rd, 2.9 MiB/s wr, 5 op/s 2024-04-07T16:39:53.821 INFO:tasks.workunit.client.0.smithi100.stdout: 75: throughput=4.94MB/sec pending data=0 2024-04-07T16:39:53.821 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-ztau8eTUSb1hUxA off=942507 len=1310239 2024-04-07T16:39:53.825 INFO:tasks.workunit.client.0.smithi100.stdout:op 380 completed, throughput=4.95MB/sec 2024-04-07T16:39:53.825 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-qEd32INLkG8go9w off=1101864 len=1405190 2024-04-07T16:39:53.836 INFO:tasks.workunit.client.0.smithi100.stdout:op 381 completed, throughput=4.97MB/sec 2024-04-07T16:39:53.836 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-8wfONl6v4d7WQKL off=11238769 len=1945318 2024-04-07T16:39:53.840 INFO:tasks.workunit.client.0.smithi100.stdout:op 382 completed, throughput=4.99MB/sec 2024-04-07T16:39:53.840 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-te4TDtIL5t91gp- off=1872911 len=285031 2024-04-07T16:39:53.846 INFO:tasks.workunit.client.0.smithi100.stdout:op 383 completed, throughput=5MB/sec 2024-04-07T16:39:53.846 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-v8K8OFVz7xkJkOj off=4076293 len=1328313 2024-04-07T16:39:53.848 INFO:tasks.workunit.client.0.smithi100.stdout:op 384 completed, throughput=5.01MB/sec 2024-04-07T16:39:54.849 INFO:tasks.workunit.client.0.smithi100.stdout: 76: throughput=4.95MB/sec pending data=0 2024-04-07T16:39:54.849 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-i1GAL4JtVlrgIaY off=6624093 len=243005 2024-04-07T16:39:54.850 INFO:tasks.workunit.client.0.smithi100.stdout:op 385 completed, throughput=4.95MB/sec 2024-04-07T16:39:54.850 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-veAafhujq5hKIi_ off=8766649 len=1367018 2024-04-07T16:39:54.854 INFO:tasks.workunit.client.0.smithi100.stdout:op 386 completed, throughput=4.97MB/sec 2024-04-07T16:39:54.854 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-MX1kJBvdbJQHXSh off=683879 len=1182459 2024-04-07T16:39:54.856 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:39:54 smithi144 bash[66654]: cluster 2024-04-07T16:39:53.121303+0000 mgr.y (mgr.25175) 736 : cluster [DBG] pgmap v1378: 105 pgs: 105 active+clean; 126 GiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 2.6 MiB/s rd, 3.1 MiB/s wr, 5 op/s 2024-04-07T16:39:54.857 INFO:tasks.workunit.client.0.smithi100.stdout:op 387 completed, throughput=4.98MB/sec 2024-04-07T16:39:54.857 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-L-hGuir3Cln4x1j off=5670313 len=166133 2024-04-07T16:39:54.862 INFO:tasks.workunit.client.0.smithi100.stdout:op 388 completed, throughput=4.98MB/sec 2024-04-07T16:39:54.863 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-CfpWc4i-pJSqaSO off=6364831 len=206321 2024-04-07T16:39:54.867 INFO:tasks.workunit.client.0.smithi100.stdout:op 389 completed, throughput=4.98MB/sec 2024-04-07T16:39:54.867 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-WipZd3sCd1TI6D4 off=4286024 len=1459848 2024-04-07T16:39:54.871 INFO:tasks.workunit.client.0.smithi100.stdout:op 390 completed, throughput=5MB/sec 2024-04-07T16:39:54.942 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:39:54 smithi100 bash[74942]: cluster 2024-04-07T16:39:53.121303+0000 mgr.y (mgr.25175) 736 : cluster [DBG] pgmap v1378: 105 pgs: 105 active+clean; 126 GiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 2.6 MiB/s rd, 3.1 MiB/s wr, 5 op/s 2024-04-07T16:39:54.942 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:39:54 smithi100 bash[76202]: cluster 2024-04-07T16:39:53.121303+0000 mgr.y (mgr.25175) 736 : cluster [DBG] pgmap v1378: 105 pgs: 105 active+clean; 126 GiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 2.6 MiB/s rd, 3.1 MiB/s wr, 5 op/s 2024-04-07T16:39:55.871 INFO:tasks.workunit.client.0.smithi100.stdout: 77: throughput=4.94MB/sec pending data=0 2024-04-07T16:39:55.871 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-q0QMYwvAH72HLlo off=1403896 len=1169211 2024-04-07T16:39:55.875 INFO:tasks.workunit.client.0.smithi100.stdout:op 391 completed, throughput=4.95MB/sec 2024-04-07T16:39:55.875 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-155kWxOl9LGYjvO off=1977079 len=577760 2024-04-07T16:39:55.882 INFO:tasks.workunit.client.0.smithi100.stdout:op 392 completed, throughput=4.96MB/sec 2024-04-07T16:39:55.882 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-1FocwxMN0TkT59Y off=7969833 len=2009172 2024-04-07T16:39:55.889 INFO:tasks.workunit.client.0.smithi100.stdout:op 393 completed, throughput=4.98MB/sec 2024-04-07T16:39:55.889 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-UYRlbDrG0PI77Xe off=1873797 len=425181 2024-04-07T16:39:55.895 INFO:tasks.workunit.client.0.smithi100.stdout:op 394 completed, throughput=4.99MB/sec 2024-04-07T16:39:55.895 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-57nk2Ghex9kKLmd off=11790360 len=89483 2024-04-07T16:39:55.900 INFO:tasks.workunit.client.0.smithi100.stdout:op 395 completed, throughput=4.99MB/sec 2024-04-07T16:39:55.900 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-sDxHISCqkR_shyW off=555048 len=876851 2024-04-07T16:39:55.909 INFO:tasks.workunit.client.0.smithi100.stdout:op 396 completed, throughput=5MB/sec 2024-04-07T16:39:55.909 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-JXBZweG3F1N0yyM off=1009447 len=1076745 2024-04-07T16:39:55.917 INFO:tasks.workunit.client.0.smithi100.stdout:op 397 completed, throughput=5.01MB/sec 2024-04-07T16:39:56.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:39:56 smithi144 bash[66654]: cluster 2024-04-07T16:39:55.122297+0000 mgr.y (mgr.25175) 737 : cluster [DBG] pgmap v1379: 105 pgs: 105 active+clean; 126 GiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 2.1 MiB/s rd, 3.1 MiB/s wr, 4 op/s 2024-04-07T16:39:56.918 INFO:tasks.workunit.client.0.smithi100.stdout: 78: throughput=4.95MB/sec pending data=0 2024-04-07T16:39:56.918 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-cj0nrFHyBr-yCOO off=5204918 len=313250 2024-04-07T16:39:56.924 INFO:tasks.workunit.client.0.smithi100.stdout:op 398 completed, throughput=4.95MB/sec 2024-04-07T16:39:56.924 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-yBOlndcJWRHcrJw off=1027133 len=94375 2024-04-07T16:39:56.925 INFO:tasks.workunit.client.0.smithi100.stdout:op 399 completed, throughput=4.95MB/sec 2024-04-07T16:39:56.925 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-_033XXkBl0mpgm2 off=11718326 len=1556324 2024-04-07T16:39:56.937 INFO:tasks.workunit.client.0.smithi100.stdout:op 400 completed, throughput=4.97MB/sec 2024-04-07T16:39:56.937 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-1beE2yR2gksNAH5 off=8480587 len=1225673 2024-04-07T16:39:56.942 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:39:56 smithi100 bash[74942]: cluster 2024-04-07T16:39:55.122297+0000 mgr.y (mgr.25175) 737 : cluster [DBG] pgmap v1379: 105 pgs: 105 active+clean; 126 GiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 2.1 MiB/s rd, 3.1 MiB/s wr, 4 op/s 2024-04-07T16:39:56.942 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:39:56 smithi100 bash[76202]: cluster 2024-04-07T16:39:55.122297+0000 mgr.y (mgr.25175) 737 : cluster [DBG] pgmap v1379: 105 pgs: 105 active+clean; 126 GiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 2.1 MiB/s rd, 3.1 MiB/s wr, 4 op/s 2024-04-07T16:39:56.945 INFO:tasks.workunit.client.0.smithi100.stdout:op 401 completed, throughput=4.99MB/sec 2024-04-07T16:39:56.945 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-nkwqPR2vS3B5CQE off=4391943 len=1434194 2024-04-07T16:39:56.955 INFO:tasks.workunit.client.0.smithi100.stdout:op 402 completed, throughput=5MB/sec 2024-04-07T16:39:57.956 INFO:tasks.workunit.client.0.smithi100.stdout: 79: throughput=4.94MB/sec pending data=0 2024-04-07T16:39:57.956 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-syO-cS07cLhVIck off=7924453 len=1350946 2024-04-07T16:39:57.966 INFO:tasks.workunit.client.0.smithi100.stdout:op 403 completed, throughput=4.96MB/sec 2024-04-07T16:39:57.966 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-0tuF1jMbiXvpcwW off=17011051 len=1330904 2024-04-07T16:39:57.976 INFO:tasks.workunit.client.0.smithi100.stdout:op 404 completed, throughput=4.97MB/sec 2024-04-07T16:39:57.976 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-7IC3R9tjmOdvnXl off=15505335 len=2015401 2024-04-07T16:39:57.980 INFO:tasks.workunit.client.0.smithi100.stdout:op 405 completed, throughput=4.99MB/sec 2024-04-07T16:39:57.980 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-dmdZWJOEFx7wMNT off=2384627 len=447131 2024-04-07T16:39:57.986 INFO:tasks.workunit.client.0.smithi100.stdout:op 406 completed, throughput=5MB/sec 2024-04-07T16:39:57.986 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-i_dCf8F--7TCRyw off=7325119 len=1913687 2024-04-07T16:39:57.990 INFO:tasks.workunit.client.0.smithi100.stdout:op 407 completed, throughput=5.02MB/sec 2024-04-07T16:39:58.692 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:39:58 smithi100 bash[74942]: cluster 2024-04-07T16:39:57.123187+0000 mgr.y (mgr.25175) 738 : cluster [DBG] pgmap v1380: 105 pgs: 105 active+clean; 126 GiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 2.0 MiB/s rd, 2.7 MiB/s wr, 4 op/s 2024-04-07T16:39:58.692 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:39:58 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:39:58] "GET /metrics HTTP/1.1" 200 34058 "" "Prometheus/2.43.0" 2024-04-07T16:39:58.692 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:39:58 smithi100 bash[76202]: cluster 2024-04-07T16:39:57.123187+0000 mgr.y (mgr.25175) 738 : cluster [DBG] pgmap v1380: 105 pgs: 105 active+clean; 126 GiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 2.0 MiB/s rd, 2.7 MiB/s wr, 4 op/s 2024-04-07T16:39:58.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:39:58 smithi144 bash[66654]: cluster 2024-04-07T16:39:57.123187+0000 mgr.y (mgr.25175) 738 : cluster [DBG] pgmap v1380: 105 pgs: 105 active+clean; 126 GiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 2.0 MiB/s rd, 2.7 MiB/s wr, 4 op/s 2024-04-07T16:39:58.991 INFO:tasks.workunit.client.0.smithi100.stdout: 80: throughput=4.96MB/sec pending data=0 2024-04-07T16:39:58.991 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-ZnEarGDDFNJgdkz off=23530222 len=1088084 2024-04-07T16:39:58.994 INFO:tasks.workunit.client.0.smithi100.stdout:op 408 completed, throughput=4.97MB/sec 2024-04-07T16:39:58.994 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-LkYlhZ2RSsl7YVi off=647201 len=607718 2024-04-07T16:39:58.997 INFO:tasks.workunit.client.0.smithi100.stdout:op 409 completed, throughput=4.98MB/sec 2024-04-07T16:39:58.997 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-0P0NXLX63N6-u99 off=4100343 len=1995281 2024-04-07T16:39:59.003 INFO:tasks.workunit.client.0.smithi100.stdout:op 410 completed, throughput=5MB/sec 2024-04-07T16:40:00.004 INFO:tasks.workunit.client.0.smithi100.stdout: 81: throughput=4.94MB/sec pending data=0 2024-04-07T16:40:00.004 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-9LFgGUURE0ah9Er off=633360 len=1588825 2024-04-07T16:40:00.016 INFO:tasks.workunit.client.0.smithi100.stdout:op 411 completed, throughput=4.96MB/sec 2024-04-07T16:40:00.017 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-8TauTa8tz6muhYU off=18182801 len=123651 2024-04-07T16:40:00.017 INFO:tasks.workunit.client.0.smithi100.stdout:op 412 completed, throughput=4.96MB/sec 2024-04-07T16:40:00.017 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-9p4Uh4dX-xNkzXb off=19150821 len=2035385 2024-04-07T16:40:00.034 INFO:tasks.workunit.client.0.smithi100.stdout:op 413 completed, throughput=4.98MB/sec 2024-04-07T16:40:00.034 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-NxQaM5K2qMIDOeg off=1086079 len=1499259 2024-04-07T16:40:00.048 INFO:tasks.workunit.client.0.smithi100.stdout:op 414 completed, throughput=5MB/sec 2024-04-07T16:40:00.048 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-dPL6yqdIOf_BK-q off=16851591 len=754546 2024-04-07T16:40:00.051 INFO:tasks.workunit.client.0.smithi100.stdout:op 415 completed, throughput=5.01MB/sec 2024-04-07T16:40:00.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:40:00 smithi144 bash[66654]: cluster 2024-04-07T16:39:59.124354+0000 mgr.y (mgr.25175) 739 : cluster [DBG] pgmap v1381: 105 pgs: 105 active+clean; 126 GiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 2.1 MiB/s rd, 2.6 MiB/s wr, 4 op/s 2024-04-07T16:40:00.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:40:00 smithi144 bash[66654]: cluster 2024-04-07T16:40:00.000127+0000 mon.a (mon.0) 2056 : cluster [INF] overall HEALTH_OK 2024-04-07T16:40:00.942 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:40:00 smithi100 bash[74942]: cluster 2024-04-07T16:39:59.124354+0000 mgr.y (mgr.25175) 739 : cluster [DBG] pgmap v1381: 105 pgs: 105 active+clean; 126 GiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 2.1 MiB/s rd, 2.6 MiB/s wr, 4 op/s 2024-04-07T16:40:00.942 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:40:00 smithi100 bash[74942]: cluster 2024-04-07T16:40:00.000127+0000 mon.a (mon.0) 2056 : cluster [INF] overall HEALTH_OK 2024-04-07T16:40:00.942 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:40:00 smithi100 bash[76202]: cluster 2024-04-07T16:39:59.124354+0000 mgr.y (mgr.25175) 739 : cluster [DBG] pgmap v1381: 105 pgs: 105 active+clean; 126 GiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 2.1 MiB/s rd, 2.6 MiB/s wr, 4 op/s 2024-04-07T16:40:00.942 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:40:00 smithi100 bash[76202]: cluster 2024-04-07T16:40:00.000127+0000 mon.a (mon.0) 2056 : cluster [INF] overall HEALTH_OK 2024-04-07T16:40:01.051 INFO:tasks.workunit.client.0.smithi100.stdout: 82: throughput=4.95MB/sec pending data=0 2024-04-07T16:40:01.051 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-OhVjStb9LCiEn1W off=13148912 len=339747 2024-04-07T16:40:01.053 INFO:tasks.workunit.client.0.smithi100.stdout:op 416 completed, throughput=4.95MB/sec 2024-04-07T16:40:01.053 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-azHGzooPo5BZccH off=2267177 len=295780 2024-04-07T16:40:01.059 INFO:tasks.workunit.client.0.smithi100.stdout:op 417 completed, throughput=4.95MB/sec 2024-04-07T16:40:01.059 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-bnL6i48A9oCqZ7J off=4115383 len=1224005 2024-04-07T16:40:01.069 INFO:tasks.workunit.client.0.smithi100.stdout:op 418 completed, throughput=4.97MB/sec 2024-04-07T16:40:01.069 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-jI1E9LSny5TwfQ- off=7712055 len=1829121 2024-04-07T16:40:01.086 INFO:tasks.workunit.client.0.smithi100.stdout:op 419 completed, throughput=4.99MB/sec 2024-04-07T16:40:01.086 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-p6QJEJzrqIfi9al off=16886049 len=1351360 2024-04-07T16:40:01.088 INFO:tasks.workunit.client.0.smithi100.stdout:op 420 completed, throughput=5MB/sec 2024-04-07T16:40:02.089 INFO:tasks.workunit.client.0.smithi100.stdout: 83: throughput=4.94MB/sec pending data=0 2024-04-07T16:40:02.089 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj--dzzH4LAGvWQIit off=9046470 len=1997800 2024-04-07T16:40:02.104 INFO:tasks.workunit.client.0.smithi100.stdout:op 421 completed, throughput=4.97MB/sec 2024-04-07T16:40:02.104 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-p3rqPeAZBJGifb2 off=8395743 len=59018 2024-04-07T16:40:02.109 INFO:tasks.workunit.client.0.smithi100.stdout:op 422 completed, throughput=4.97MB/sec 2024-04-07T16:40:02.109 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-KNFR7I4KaGs6vcH off=12500455 len=1104530 2024-04-07T16:40:02.113 INFO:tasks.workunit.client.0.smithi100.stdout:op 423 completed, throughput=4.98MB/sec 2024-04-07T16:40:02.113 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-0IILJOkewFRNAqL off=6581310 len=1613578 2024-04-07T16:40:02.116 INFO:tasks.workunit.client.0.smithi100.stdout:op 424 completed, throughput=5MB/sec 2024-04-07T16:40:02.117 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-JTc0O_P7d3Qlo8h off=2081037 len=939710 2024-04-07T16:40:02.120 INFO:tasks.workunit.client.0.smithi100.stdout:op 425 completed, throughput=5.01MB/sec 2024-04-07T16:40:02.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:40:02 smithi144 bash[66654]: cluster 2024-04-07T16:40:01.125393+0000 mgr.y (mgr.25175) 740 : cluster [DBG] pgmap v1382: 105 pgs: 105 active+clean; 126 GiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 2.2 MiB/s rd, 2.7 MiB/s wr, 4 op/s 2024-04-07T16:40:02.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:40:02 smithi144 bash[66654]: audit 2024-04-07T16:40:01.756659+0000 mon.a (mon.0) 2057 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:40:02.607 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:40:02 smithi144 bash[95233]: ts=2024-04-07T16:40:02.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:40:02.942 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:40:02 smithi100 bash[74942]: cluster 2024-04-07T16:40:01.125393+0000 mgr.y (mgr.25175) 740 : cluster [DBG] pgmap v1382: 105 pgs: 105 active+clean; 126 GiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 2.2 MiB/s rd, 2.7 MiB/s wr, 4 op/s 2024-04-07T16:40:02.942 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:40:02 smithi100 bash[74942]: audit 2024-04-07T16:40:01.756659+0000 mon.a (mon.0) 2057 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:40:02.942 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:40:02 smithi100 bash[76202]: cluster 2024-04-07T16:40:01.125393+0000 mgr.y (mgr.25175) 740 : cluster [DBG] pgmap v1382: 105 pgs: 105 active+clean; 126 GiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 2.2 MiB/s rd, 2.7 MiB/s wr, 4 op/s 2024-04-07T16:40:02.942 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:40:02 smithi100 bash[76202]: audit 2024-04-07T16:40:01.756659+0000 mon.a (mon.0) 2057 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:40:03.120 INFO:tasks.workunit.client.0.smithi100.stdout: 84: throughput=4.95MB/sec pending data=0 2024-04-07T16:40:03.120 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Z2KOyyVBqyQK5ua off=12526176 len=471056 2024-04-07T16:40:03.127 INFO:tasks.workunit.client.0.smithi100.stdout:op 426 completed, throughput=4.95MB/sec 2024-04-07T16:40:03.127 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-I6QBJqq6yAe2yAR off=15901964 len=209626 2024-04-07T16:40:03.129 INFO:tasks.workunit.client.0.smithi100.stdout:op 427 completed, throughput=4.96MB/sec 2024-04-07T16:40:03.129 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-KyyJHvAQYR0-k0e off=0 len=1737640 2024-04-07T16:40:03.133 INFO:tasks.workunit.client.0.smithi100.stdout:op 428 completed, throughput=4.97MB/sec 2024-04-07T16:40:03.133 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-gX2JACmIeJaztJj off=2689542 len=1623709 2024-04-07T16:40:03.136 INFO:tasks.workunit.client.0.smithi100.stdout:op 429 completed, throughput=4.99MB/sec 2024-04-07T16:40:03.136 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-6ZObfKsaT9guGaw off=387230 len=653387 2024-04-07T16:40:03.143 INFO:tasks.workunit.client.0.smithi100.stdout:op 430 completed, throughput=5MB/sec 2024-04-07T16:40:03.143 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-G05JKgJzsnIcu1g off=1260110 len=1534210 2024-04-07T16:40:03.156 INFO:tasks.workunit.client.0.smithi100.stdout:op 431 completed, throughput=5.01MB/sec 2024-04-07T16:40:04.156 INFO:tasks.workunit.client.0.smithi100.stdout: 85: throughput=4.96MB/sec pending data=0 2024-04-07T16:40:04.156 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-xEHXCIlldmQDMwP off=6837024 len=1453422 2024-04-07T16:40:04.167 INFO:tasks.workunit.client.0.smithi100.stdout:op 432 completed, throughput=4.97MB/sec 2024-04-07T16:40:04.167 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-fdAx9m0TzXF-IQt off=9620931 len=681538 2024-04-07T16:40:04.174 INFO:tasks.workunit.client.0.smithi100.stdout:op 433 completed, throughput=4.98MB/sec 2024-04-07T16:40:04.174 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-aHCKNPf68jWoVlF off=5921417 len=1416327 2024-04-07T16:40:04.185 INFO:tasks.workunit.client.0.smithi100.stdout:op 434 completed, throughput=4.99MB/sec 2024-04-07T16:40:04.185 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-U8jfGXfoT1z-nHm off=14906525 len=1980880 2024-04-07T16:40:04.200 INFO:tasks.workunit.client.0.smithi100.stdout:op 435 completed, throughput=5.02MB/sec 2024-04-07T16:40:04.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:40:04 smithi144 bash[66654]: cluster 2024-04-07T16:40:03.126561+0000 mgr.y (mgr.25175) 741 : cluster [DBG] pgmap v1383: 105 pgs: 105 active+clean; 126 GiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.7 MiB/s wr, 5 op/s 2024-04-07T16:40:04.942 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:40:04 smithi100 bash[74942]: cluster 2024-04-07T16:40:03.126561+0000 mgr.y (mgr.25175) 741 : cluster [DBG] pgmap v1383: 105 pgs: 105 active+clean; 126 GiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.7 MiB/s wr, 5 op/s 2024-04-07T16:40:04.942 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:40:04 smithi100 bash[76202]: cluster 2024-04-07T16:40:03.126561+0000 mgr.y (mgr.25175) 741 : cluster [DBG] pgmap v1383: 105 pgs: 105 active+clean; 126 GiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.7 MiB/s wr, 5 op/s 2024-04-07T16:40:05.201 INFO:tasks.workunit.client.0.smithi100.stdout: 86: throughput=4.96MB/sec pending data=0 2024-04-07T16:40:05.201 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-RhRXZ6DDq1NBt-V off=6547607 len=348336 2024-04-07T16:40:05.206 INFO:tasks.workunit.client.0.smithi100.stdout:op 436 completed, throughput=4.96MB/sec 2024-04-07T16:40:05.206 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-mnk5rm2gPmGYUQi off=10087059 len=15848 2024-04-07T16:40:05.210 INFO:tasks.workunit.client.0.smithi100.stdout:op 437 completed, throughput=4.96MB/sec 2024-04-07T16:40:05.210 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-IcVgINB1teqaaib off=17251584 len=352359 2024-04-07T16:40:05.212 INFO:tasks.workunit.client.0.smithi100.stdout:op 438 completed, throughput=4.97MB/sec 2024-04-07T16:40:05.212 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-pXiDkLdRtrJ4782 off=592668 len=834840 2024-04-07T16:40:05.214 INFO:tasks.workunit.client.0.smithi100.stdout:op 439 completed, throughput=4.97MB/sec 2024-04-07T16:40:05.214 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-cucSYPB0MytUy5a off=387633 len=772668 2024-04-07T16:40:05.222 INFO:tasks.workunit.client.0.smithi100.stdout:op 440 completed, throughput=4.98MB/sec 2024-04-07T16:40:05.222 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-enbrTbHurAIYE1y off=4701898 len=768374 2024-04-07T16:40:05.224 INFO:tasks.workunit.client.0.smithi100.stdout:op 441 completed, throughput=4.99MB/sec 2024-04-07T16:40:05.224 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-1rFbBAy1QwfXjK0 off=17996535 len=18105 2024-04-07T16:40:05.228 INFO:tasks.workunit.client.0.smithi100.stdout:op 442 completed, throughput=4.99MB/sec 2024-04-07T16:40:05.228 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-FZjeLIg_6MMlc0V off=11812561 len=1091896 2024-04-07T16:40:05.238 INFO:tasks.workunit.client.0.smithi100.stdout:op 443 completed, throughput=5MB/sec 2024-04-07T16:40:06.238 INFO:tasks.workunit.client.0.smithi100.stdout: 87: throughput=4.95MB/sec pending data=0 2024-04-07T16:40:06.239 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-3or1dLM6J4TTY0G off=6341183 len=547958 2024-04-07T16:40:06.241 INFO:tasks.workunit.client.0.smithi100.stdout:op 444 completed, throughput=4.95MB/sec 2024-04-07T16:40:06.241 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-sBVN-sHLWm2AuQW off=5571600 len=1723402 2024-04-07T16:40:06.255 INFO:tasks.workunit.client.0.smithi100.stdout:op 445 completed, throughput=4.97MB/sec 2024-04-07T16:40:06.255 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-ECszPhJcCvxXmzk off=13342284 len=1226946 2024-04-07T16:40:06.265 INFO:tasks.workunit.client.0.smithi100.stdout:op 446 completed, throughput=4.98MB/sec 2024-04-07T16:40:06.265 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-48wfLHbSMU4TyX1 off=7036807 len=2035716 2024-04-07T16:40:06.282 INFO:tasks.workunit.client.0.smithi100.stdout:op 447 completed, throughput=5MB/sec 2024-04-07T16:40:06.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:40:06 smithi144 bash[66654]: cluster 2024-04-07T16:40:05.127554+0000 mgr.y (mgr.25175) 742 : cluster [DBG] pgmap v1384: 105 pgs: 105 active+clean; 126 GiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.4 MiB/s wr, 4 op/s 2024-04-07T16:40:06.942 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:40:06 smithi100 bash[74942]: cluster 2024-04-07T16:40:05.127554+0000 mgr.y (mgr.25175) 742 : cluster [DBG] pgmap v1384: 105 pgs: 105 active+clean; 126 GiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.4 MiB/s wr, 4 op/s 2024-04-07T16:40:06.942 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:40:06 smithi100 bash[76202]: cluster 2024-04-07T16:40:05.127554+0000 mgr.y (mgr.25175) 742 : cluster [DBG] pgmap v1384: 105 pgs: 105 active+clean; 126 GiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.4 MiB/s wr, 4 op/s 2024-04-07T16:40:07.283 INFO:tasks.workunit.client.0.smithi100.stdout: 88: throughput=4.95MB/sec pending data=0 2024-04-07T16:40:07.283 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-6K1vYgspaoRGNQg off=3738071 len=767127 2024-04-07T16:40:07.291 INFO:tasks.workunit.client.0.smithi100.stdout:op 448 completed, throughput=4.95MB/sec 2024-04-07T16:40:07.291 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-dV_BOv7TtefFCrv off=13932022 len=15490 2024-04-07T16:40:07.292 INFO:tasks.workunit.client.0.smithi100.stdout:op 449 completed, throughput=4.96MB/sec 2024-04-07T16:40:07.292 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-cb9hGJwTIaFZ3sf off=7659686 len=1560296 2024-04-07T16:40:07.295 INFO:tasks.workunit.client.0.smithi100.stdout:op 450 completed, throughput=4.97MB/sec 2024-04-07T16:40:07.295 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-l1qjBFO9iieAj4U off=1589024 len=1030349 2024-04-07T16:40:07.299 INFO:tasks.workunit.client.0.smithi100.stdout:op 451 completed, throughput=4.98MB/sec 2024-04-07T16:40:07.299 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-28JGnNMtIuZQ9Fz off=50836 len=409674 2024-04-07T16:40:07.301 INFO:tasks.workunit.client.0.smithi100.stdout:op 452 completed, throughput=4.99MB/sec 2024-04-07T16:40:07.301 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-6VKZNgO5Km5Ptk0 off=1358508 len=712646 2024-04-07T16:40:07.304 INFO:tasks.workunit.client.0.smithi100.stdout:op 453 completed, throughput=4.99MB/sec 2024-04-07T16:40:07.304 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-7CMcgmvxP2USOua off=5463758 len=1917555 2024-04-07T16:40:07.319 INFO:tasks.workunit.client.0.smithi100.stdout:op 454 completed, throughput=5.01MB/sec 2024-04-07T16:40:08.319 INFO:tasks.workunit.client.0.smithi100.stdout: 89: throughput=4.96MB/sec pending data=0 2024-04-07T16:40:08.320 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Gk57KvEeCKgMDIY off=14996593 len=377451 2024-04-07T16:40:08.325 INFO:tasks.workunit.client.0.smithi100.stdout:op 455 completed, throughput=4.96MB/sec 2024-04-07T16:40:08.325 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-IgYCFkuq_8jydg8 off=16524091 len=2084922 2024-04-07T16:40:08.331 INFO:tasks.workunit.client.0.smithi100.stdout:op 456 completed, throughput=4.98MB/sec 2024-04-07T16:40:08.331 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Yli5QWNbWrJKKDA off=857714 len=1053182 2024-04-07T16:40:08.335 INFO:tasks.workunit.client.0.smithi100.stdout:op 457 completed, throughput=4.99MB/sec 2024-04-07T16:40:08.335 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-55PPMCNhl38X8YX off=10741509 len=323390 2024-04-07T16:40:08.340 INFO:tasks.workunit.client.0.smithi100.stdout:op 458 completed, throughput=5MB/sec 2024-04-07T16:40:08.340 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-i-xbF5P9rFrj7zB off=7061309 len=1584297 2024-04-07T16:40:08.345 INFO:tasks.workunit.client.0.smithi100.stdout:op 459 completed, throughput=5.01MB/sec 2024-04-07T16:40:08.692 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:40:08 smithi100 bash[74942]: cluster 2024-04-07T16:40:07.128414+0000 mgr.y (mgr.25175) 743 : cluster [DBG] pgmap v1385: 105 pgs: 105 active+clean; 126 GiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 2.3 MiB/s rd, 2.6 MiB/s wr, 4 op/s 2024-04-07T16:40:08.693 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:40:08 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:40:08] "GET /metrics HTTP/1.1" 200 34051 "" "Prometheus/2.43.0" 2024-04-07T16:40:08.693 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:40:08 smithi100 bash[76202]: cluster 2024-04-07T16:40:07.128414+0000 mgr.y (mgr.25175) 743 : cluster [DBG] pgmap v1385: 105 pgs: 105 active+clean; 126 GiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 2.3 MiB/s rd, 2.6 MiB/s wr, 4 op/s 2024-04-07T16:40:08.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:40:08 smithi144 bash[66654]: cluster 2024-04-07T16:40:07.128414+0000 mgr.y (mgr.25175) 743 : cluster [DBG] pgmap v1385: 105 pgs: 105 active+clean; 126 GiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 2.3 MiB/s rd, 2.6 MiB/s wr, 4 op/s 2024-04-07T16:40:09.345 INFO:tasks.workunit.client.0.smithi100.stdout: 90: throughput=4.96MB/sec pending data=0 2024-04-07T16:40:09.345 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-9Q3ESLjeT-0zm3a off=7477666 len=902720 2024-04-07T16:40:09.349 INFO:tasks.workunit.client.0.smithi100.stdout:op 460 completed, throughput=4.97MB/sec 2024-04-07T16:40:09.349 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-YRM47ovR6w_pcdV off=10769101 len=384797 2024-04-07T16:40:09.351 INFO:tasks.workunit.client.0.smithi100.stdout:op 461 completed, throughput=4.97MB/sec 2024-04-07T16:40:09.351 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-0eksKOu9ZiVxtB8 off=9328605 len=746293 2024-04-07T16:40:09.359 INFO:tasks.workunit.client.0.smithi100.stdout:op 462 completed, throughput=4.98MB/sec 2024-04-07T16:40:09.359 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-htMUpRalchBqLpr off=4052052 len=241745 2024-04-07T16:40:09.364 INFO:tasks.workunit.client.0.smithi100.stdout:op 463 completed, throughput=4.98MB/sec 2024-04-07T16:40:09.364 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-pP_pz9OinpKzHl_ off=4831162 len=972220 2024-04-07T16:40:09.373 INFO:tasks.workunit.client.0.smithi100.stdout:op 464 completed, throughput=4.99MB/sec 2024-04-07T16:40:09.373 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-4Oa5VmzWBLIT_4a off=13238555 len=1940311 2024-04-07T16:40:09.388 INFO:tasks.workunit.client.0.smithi100.stdout:op 465 completed, throughput=5.01MB/sec 2024-04-07T16:40:10.389 INFO:tasks.workunit.client.0.smithi100.stdout: 91: throughput=4.96MB/sec pending data=0 2024-04-07T16:40:10.389 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-ypDsKjRSgyOoUXM off=894845 len=2011218 2024-04-07T16:40:10.404 INFO:tasks.workunit.client.0.smithi100.stdout:op 466 completed, throughput=4.98MB/sec 2024-04-07T16:40:10.404 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-tz6yeDi2QUuLs0j off=2757589 len=2028405 2024-04-07T16:40:10.410 INFO:tasks.workunit.client.0.smithi100.stdout:op 467 completed, throughput=5MB/sec 2024-04-07T16:40:10.410 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj--8VFUBTYdGGdtdi off=0 len=1463599 2024-04-07T16:40:10.414 INFO:tasks.workunit.client.0.smithi100.stdout:op 468 completed, throughput=5.01MB/sec 2024-04-07T16:40:10.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:40:10 smithi144 bash[66654]: cluster 2024-04-07T16:40:09.129369+0000 mgr.y (mgr.25175) 744 : cluster [DBG] pgmap v1386: 105 pgs: 105 active+clean; 126 GiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 2.4 MiB/s rd, 2.9 MiB/s wr, 5 op/s 2024-04-07T16:40:10.942 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:40:10 smithi100 bash[74942]: cluster 2024-04-07T16:40:09.129369+0000 mgr.y (mgr.25175) 744 : cluster [DBG] pgmap v1386: 105 pgs: 105 active+clean; 126 GiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 2.4 MiB/s rd, 2.9 MiB/s wr, 5 op/s 2024-04-07T16:40:10.942 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:40:10 smithi100 bash[76202]: cluster 2024-04-07T16:40:09.129369+0000 mgr.y (mgr.25175) 744 : cluster [DBG] pgmap v1386: 105 pgs: 105 active+clean; 126 GiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 2.4 MiB/s rd, 2.9 MiB/s wr, 5 op/s 2024-04-07T16:40:11.414 INFO:tasks.workunit.client.0.smithi100.stdout: 92: throughput=4.96MB/sec pending data=0 2024-04-07T16:40:11.414 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-7eszuOtbhHh2Kbq off=1913238 len=312984 2024-04-07T16:40:11.419 INFO:tasks.workunit.client.0.smithi100.stdout:op 469 completed, throughput=4.96MB/sec 2024-04-07T16:40:11.420 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-dLpAsIzKas_7fRZ off=6133820 len=1093535 2024-04-07T16:40:11.422 INFO:tasks.workunit.client.0.smithi100.stdout:op 470 completed, throughput=4.97MB/sec 2024-04-07T16:40:11.423 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-rktkaM4pt14EcWy off=6313077 len=1047841 2024-04-07T16:40:11.430 INFO:tasks.workunit.client.0.smithi100.stdout:op 471 completed, throughput=4.98MB/sec 2024-04-07T16:40:11.430 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-QupPEj4-sYwTADV off=14814571 len=1399398 2024-04-07T16:40:11.433 INFO:tasks.workunit.client.0.smithi100.stdout:op 472 completed, throughput=5MB/sec 2024-04-07T16:40:11.433 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-2WCqJlKc6D-4cBX off=8204372 len=874930 2024-04-07T16:40:11.435 INFO:tasks.workunit.client.0.smithi100.stdout:op 473 completed, throughput=5MB/sec 2024-04-07T16:40:12.435 INFO:tasks.workunit.client.0.smithi100.stdout: 93: throughput=4.95MB/sec pending data=0 2024-04-07T16:40:12.435 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-KmN9hh0i90TJweJ off=699585 len=1221146 2024-04-07T16:40:12.445 INFO:tasks.workunit.client.0.smithi100.stdout:op 474 completed, throughput=4.96MB/sec 2024-04-07T16:40:12.445 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-hQBh5oRknMnQYZg off=1375370 len=1189585 2024-04-07T16:40:12.454 INFO:tasks.workunit.client.0.smithi100.stdout:op 475 completed, throughput=4.98MB/sec 2024-04-07T16:40:12.455 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-akMTSJODG0z9sRk off=2914681 len=1227373 2024-04-07T16:40:12.463 INFO:tasks.workunit.client.0.smithi100.stdout:op 476 completed, throughput=4.99MB/sec 2024-04-07T16:40:12.464 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-N_7dabGHaYmNthE off=10369055 len=119430 2024-04-07T16:40:12.465 INFO:tasks.workunit.client.0.smithi100.stdout:op 477 completed, throughput=4.99MB/sec 2024-04-07T16:40:12.465 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-RtULqPS87_f5KpE off=5999667 len=494568 2024-04-07T16:40:12.466 INFO:tasks.workunit.client.0.smithi100.stdout:op 478 completed, throughput=4.99MB/sec 2024-04-07T16:40:12.467 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-NcjWCp0az61do7m off=7750976 len=1701070 2024-04-07T16:40:12.480 INFO:tasks.workunit.client.0.smithi100.stdout:op 479 completed, throughput=5.01MB/sec 2024-04-07T16:40:12.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:40:12 smithi144 bash[66654]: cluster 2024-04-07T16:40:11.130265+0000 mgr.y (mgr.25175) 745 : cluster [DBG] pgmap v1387: 105 pgs: 105 active+clean; 126 GiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 1.9 MiB/s rd, 2.9 MiB/s wr, 4 op/s 2024-04-07T16:40:12.607 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:40:12 smithi144 bash[95233]: ts=2024-04-07T16:40:12.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:40:12.942 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:40:12 smithi100 bash[74942]: cluster 2024-04-07T16:40:11.130265+0000 mgr.y (mgr.25175) 745 : cluster [DBG] pgmap v1387: 105 pgs: 105 active+clean; 126 GiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 1.9 MiB/s rd, 2.9 MiB/s wr, 4 op/s 2024-04-07T16:40:12.942 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:40:12 smithi100 bash[76202]: cluster 2024-04-07T16:40:11.130265+0000 mgr.y (mgr.25175) 745 : cluster [DBG] pgmap v1387: 105 pgs: 105 active+clean; 126 GiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 1.9 MiB/s rd, 2.9 MiB/s wr, 4 op/s 2024-04-07T16:40:13.480 INFO:tasks.workunit.client.0.smithi100.stdout: 94: throughput=4.96MB/sec pending data=0 2024-04-07T16:40:13.480 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-0VfNeHtNyIkAVO1 off=238093 len=1972283 2024-04-07T16:40:13.497 INFO:tasks.workunit.client.0.smithi100.stdout:op 480 completed, throughput=4.98MB/sec 2024-04-07T16:40:13.497 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Humzm8RmyQIXEgP off=380377 len=852526 2024-04-07T16:40:13.506 INFO:tasks.workunit.client.0.smithi100.stdout:op 481 completed, throughput=4.98MB/sec 2024-04-07T16:40:13.506 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-PX1IkCqmVLo_F61 off=12009099 len=411624 2024-04-07T16:40:13.512 INFO:tasks.workunit.client.0.smithi100.stdout:op 482 completed, throughput=4.99MB/sec 2024-04-07T16:40:13.512 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-8LhNbJsbU7A7SJQ off=20763848 len=1589309 2024-04-07T16:40:13.526 INFO:tasks.workunit.client.0.smithi100.stdout:op 483 completed, throughput=5MB/sec 2024-04-07T16:40:14.526 INFO:tasks.workunit.client.0.smithi100.stdout: 95: throughput=4.95MB/sec pending data=0 2024-04-07T16:40:14.526 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Y0tTiam7dP4Su2H off=0 len=1297546 2024-04-07T16:40:14.530 INFO:tasks.workunit.client.0.smithi100.stdout:op 484 completed, throughput=4.96MB/sec 2024-04-07T16:40:14.530 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-nJDDOANMtnoS5Gg off=12236171 len=1823250 2024-04-07T16:40:14.535 INFO:tasks.workunit.client.0.smithi100.stdout:op 485 completed, throughput=4.98MB/sec 2024-04-07T16:40:14.535 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-ZXKT41HZlCd3OmW off=5247504 len=962636 2024-04-07T16:40:14.545 INFO:tasks.workunit.client.0.smithi100.stdout:op 486 completed, throughput=4.99MB/sec 2024-04-07T16:40:14.545 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-mLtudjjrM0Lm7Bo off=6402978 len=732833 2024-04-07T16:40:14.552 INFO:tasks.workunit.client.0.smithi100.stdout:op 487 completed, throughput=5MB/sec 2024-04-07T16:40:14.552 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-3__Hs7Wpm6f_xeL off=21516630 len=982773 2024-04-07T16:40:14.554 INFO:tasks.workunit.client.0.smithi100.stdout:op 488 completed, throughput=5.01MB/sec 2024-04-07T16:40:14.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:40:14 smithi144 bash[66654]: cluster 2024-04-07T16:40:13.131458+0000 mgr.y (mgr.25175) 746 : cluster [DBG] pgmap v1388: 105 pgs: 105 active+clean; 126 GiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 2.4 MiB/s rd, 2.8 MiB/s wr, 5 op/s 2024-04-07T16:40:14.942 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:40:14 smithi100 bash[74942]: cluster 2024-04-07T16:40:13.131458+0000 mgr.y (mgr.25175) 746 : cluster [DBG] pgmap v1388: 105 pgs: 105 active+clean; 126 GiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 2.4 MiB/s rd, 2.8 MiB/s wr, 5 op/s 2024-04-07T16:40:14.942 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:40:14 smithi100 bash[76202]: cluster 2024-04-07T16:40:13.131458+0000 mgr.y (mgr.25175) 746 : cluster [DBG] pgmap v1388: 105 pgs: 105 active+clean; 126 GiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 2.4 MiB/s rd, 2.8 MiB/s wr, 5 op/s 2024-04-07T16:40:15.554 INFO:tasks.workunit.client.0.smithi100.stdout: 96: throughput=4.96MB/sec pending data=0 2024-04-07T16:40:15.555 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Tju17bp7AcgSreg off=16441 len=1583446 2024-04-07T16:40:15.566 INFO:tasks.workunit.client.0.smithi100.stdout:op 489 completed, throughput=4.97MB/sec 2024-04-07T16:40:15.566 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-YMOROv3EHMnXBne off=5150515 len=1100767 2024-04-07T16:40:15.574 INFO:tasks.workunit.client.0.smithi100.stdout:op 490 completed, throughput=4.98MB/sec 2024-04-07T16:40:15.574 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-iI4Cmss2TtqE6da off=17038029 len=1256528 2024-04-07T16:40:15.586 INFO:tasks.workunit.client.0.smithi100.stdout:op 491 completed, throughput=4.99MB/sec 2024-04-07T16:40:15.586 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-RAB_2gnbxHjKHYX off=1328819 len=157118 2024-04-07T16:40:15.591 INFO:tasks.workunit.client.0.smithi100.stdout:op 492 completed, throughput=4.99MB/sec 2024-04-07T16:40:15.591 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-OjZhSJqUefC6GnP off=2137937 len=125813 2024-04-07T16:40:15.593 INFO:tasks.workunit.client.0.smithi100.stdout:op 493 completed, throughput=4.99MB/sec 2024-04-07T16:40:15.593 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-8wpmw5qAJyd1FE2 off=1116974 len=412633 2024-04-07T16:40:15.598 INFO:tasks.workunit.client.0.smithi100.stdout:op 494 completed, throughput=5MB/sec 2024-04-07T16:40:15.598 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-afKNe2OxQnKBb-y off=19001923 len=1309673 2024-04-07T16:40:15.601 INFO:tasks.workunit.client.0.smithi100.stdout:op 495 completed, throughput=5.01MB/sec 2024-04-07T16:40:16.602 INFO:tasks.workunit.client.0.smithi100.stdout: 97: throughput=4.96MB/sec pending data=0 2024-04-07T16:40:16.602 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-GBxEJZ98LZnfzL3 off=18096168 len=425390 2024-04-07T16:40:16.610 INFO:tasks.workunit.client.0.smithi100.stdout:op 496 completed, throughput=4.96MB/sec 2024-04-07T16:40:16.610 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Xnk5gBYtjI-5FmA off=12333788 len=123151 2024-04-07T16:40:16.612 INFO:tasks.workunit.client.0.smithi100.stdout:op 497 completed, throughput=4.97MB/sec 2024-04-07T16:40:16.612 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-3uVLxCTkPX8_DKZ off=5639186 len=487077 2024-04-07T16:40:16.619 INFO:tasks.workunit.client.0.smithi100.stdout:op 498 completed, throughput=4.97MB/sec 2024-04-07T16:40:16.619 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-B8jeWwNPDwZZ3yC off=8104194 len=2006283 2024-04-07T16:40:16.624 INFO:tasks.workunit.client.0.smithi100.stdout:op 499 completed, throughput=4.99MB/sec 2024-04-07T16:40:16.624 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-IFTVWmL5ZMUK3Pn off=14512797 len=207988 2024-04-07T16:40:16.629 INFO:tasks.workunit.client.0.smithi100.stdout:op 500 completed, throughput=4.99MB/sec 2024-04-07T16:40:16.629 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-ShhXXEFfpV72MPx off=6805289 len=1637237 2024-04-07T16:40:16.641 INFO:tasks.workunit.client.0.smithi100.stdout:op 501 completed, throughput=5.01MB/sec 2024-04-07T16:40:16.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:40:16 smithi144 bash[66654]: cluster 2024-04-07T16:40:15.132345+0000 mgr.y (mgr.25175) 747 : cluster [DBG] pgmap v1389: 105 pgs: 105 active+clean; 126 GiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 2.1 MiB/s rd, 2.7 MiB/s wr, 4 op/s 2024-04-07T16:40:16.942 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:40:16 smithi100 bash[74942]: cluster 2024-04-07T16:40:15.132345+0000 mgr.y (mgr.25175) 747 : cluster [DBG] pgmap v1389: 105 pgs: 105 active+clean; 126 GiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 2.1 MiB/s rd, 2.7 MiB/s wr, 4 op/s 2024-04-07T16:40:16.942 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:40:16 smithi100 bash[76202]: cluster 2024-04-07T16:40:15.132345+0000 mgr.y (mgr.25175) 747 : cluster [DBG] pgmap v1389: 105 pgs: 105 active+clean; 126 GiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 2.1 MiB/s rd, 2.7 MiB/s wr, 4 op/s 2024-04-07T16:40:17.641 INFO:tasks.workunit.client.0.smithi100.stdout: 98: throughput=4.96MB/sec pending data=0 2024-04-07T16:40:17.642 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-UJmVkR3KXElmLKs off=12613785 len=392298 2024-04-07T16:40:17.643 INFO:tasks.workunit.client.0.smithi100.stdout:op 502 completed, throughput=4.96MB/sec 2024-04-07T16:40:17.643 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-91Ef2tXNa_W3kUA off=148034 len=1502484 2024-04-07T16:40:17.647 INFO:tasks.workunit.client.0.smithi100.stdout:op 503 completed, throughput=4.97MB/sec 2024-04-07T16:40:17.647 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Ugo61Ox_w1AHLc2 off=7639406 len=340646 2024-04-07T16:40:17.649 INFO:tasks.workunit.client.0.smithi100.stdout:op 504 completed, throughput=4.98MB/sec 2024-04-07T16:40:17.649 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj--KaSIxnY2TSkTVm off=73380 len=471566 2024-04-07T16:40:17.655 INFO:tasks.workunit.client.0.smithi100.stdout:op 505 completed, throughput=4.98MB/sec 2024-04-07T16:40:17.655 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Aoz0N_2jpEuJfgQ off=2233291 len=1988962 2024-04-07T16:40:17.660 INFO:tasks.workunit.client.0.smithi100.stdout:op 506 completed, throughput=5MB/sec 2024-04-07T16:40:17.660 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-LO6fV9V73RYUcU6 off=11413489 len=1440551 2024-04-07T16:40:17.671 INFO:tasks.workunit.client.0.smithi100.stdout:op 507 completed, throughput=5.01MB/sec 2024-04-07T16:40:17.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:40:17 smithi144 bash[66654]: audit 2024-04-07T16:40:16.757142+0000 mon.a (mon.0) 2058 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:40:17.942 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:40:17 smithi100 bash[74942]: audit 2024-04-07T16:40:16.757142+0000 mon.a (mon.0) 2058 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:40:17.942 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:40:17 smithi100 bash[76202]: audit 2024-04-07T16:40:16.757142+0000 mon.a (mon.0) 2058 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:40:18.672 INFO:tasks.workunit.client.0.smithi100.stdout: 99: throughput=4.96MB/sec pending data=0 2024-04-07T16:40:18.672 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-IUbawJeQeNRk1te off=407314 len=1485517 2024-04-07T16:40:18.685 INFO:tasks.workunit.client.0.smithi100.stdout:op 508 completed, throughput=4.98MB/sec 2024-04-07T16:40:18.685 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-9UowMnaALyZnQHw off=1561870 len=1426255 2024-04-07T16:40:18.692 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:40:18 smithi100 bash[74942]: cluster 2024-04-07T16:40:17.133321+0000 mgr.y (mgr.25175) 748 : cluster [DBG] pgmap v1390: 105 pgs: 105 active+clean; 126 GiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 2.0 MiB/s rd, 2.7 MiB/s wr, 4 op/s 2024-04-07T16:40:18.692 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:40:18 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:40:18] "GET /metrics HTTP/1.1" 200 34053 "" "Prometheus/2.43.0" 2024-04-07T16:40:18.692 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:40:18 smithi100 bash[76202]: cluster 2024-04-07T16:40:17.133321+0000 mgr.y (mgr.25175) 748 : cluster [DBG] pgmap v1390: 105 pgs: 105 active+clean; 126 GiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 2.0 MiB/s rd, 2.7 MiB/s wr, 4 op/s 2024-04-07T16:40:18.696 INFO:tasks.workunit.client.0.smithi100.stdout:op 509 completed, throughput=4.99MB/sec 2024-04-07T16:40:18.696 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-FiAKiezcxz5sQWB off=790490 len=1465838 2024-04-07T16:40:18.709 INFO:tasks.workunit.client.0.smithi100.stdout:op 510 completed, throughput=5MB/sec 2024-04-07T16:40:18.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:40:18 smithi144 bash[66654]: cluster 2024-04-07T16:40:17.133321+0000 mgr.y (mgr.25175) 748 : cluster [DBG] pgmap v1390: 105 pgs: 105 active+clean; 126 GiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 2.0 MiB/s rd, 2.7 MiB/s wr, 4 op/s 2024-04-07T16:40:19.709 INFO:tasks.workunit.client.0.smithi100.stdout: 100: throughput=4.95MB/sec pending data=0 2024-04-07T16:40:19.709 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-eWlqDHxzTKzhJsW off=2155880 len=991832 2024-04-07T16:40:19.717 INFO:tasks.workunit.client.0.smithi100.stdout:op 511 completed, throughput=4.96MB/sec 2024-04-07T16:40:19.718 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-7zsFoSHlMN7RpZJ off=10228322 len=1833957 2024-04-07T16:40:19.724 INFO:tasks.workunit.client.0.smithi100.stdout:op 512 completed, throughput=4.98MB/sec 2024-04-07T16:40:19.724 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-7LUjKTxDuUhTEyy off=10761646 len=426085 2024-04-07T16:40:19.730 INFO:tasks.workunit.client.0.smithi100.stdout:op 513 completed, throughput=4.98MB/sec 2024-04-07T16:40:19.730 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-FDvSwFl9Ow8C2Cy off=7773205 len=1155714 2024-04-07T16:40:19.739 INFO:tasks.workunit.client.0.smithi100.stdout:op 514 completed, throughput=4.99MB/sec 2024-04-07T16:40:19.739 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-R9v1YRh1PiOPUZQ off=1273792 len=548969 2024-04-07T16:40:19.741 INFO:tasks.workunit.client.0.smithi100.stdout:op 515 completed, throughput=5MB/sec 2024-04-07T16:40:19.741 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-_uX4yOmRpfZkAjs off=0 len=1180438 2024-04-07T16:40:19.744 INFO:tasks.workunit.client.0.smithi100.stdout:op 516 completed, throughput=5.01MB/sec 2024-04-07T16:40:19.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:40:19 smithi144 bash[66654]: cluster 2024-04-07T16:40:19.134149+0000 mgr.y (mgr.25175) 749 : cluster [DBG] pgmap v1391: 105 pgs: 105 active+clean; 126 GiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 2.1 MiB/s rd, 2.5 MiB/s wr, 5 op/s 2024-04-07T16:40:19.942 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:40:19 smithi100 bash[74942]: cluster 2024-04-07T16:40:19.134149+0000 mgr.y (mgr.25175) 749 : cluster [DBG] pgmap v1391: 105 pgs: 105 active+clean; 126 GiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 2.1 MiB/s rd, 2.5 MiB/s wr, 5 op/s 2024-04-07T16:40:19.942 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:40:19 smithi100 bash[76202]: cluster 2024-04-07T16:40:19.134149+0000 mgr.y (mgr.25175) 749 : cluster [DBG] pgmap v1391: 105 pgs: 105 active+clean; 126 GiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 2.1 MiB/s rd, 2.5 MiB/s wr, 5 op/s 2024-04-07T16:40:20.744 INFO:tasks.workunit.client.0.smithi100.stdout: 101: throughput=4.96MB/sec pending data=0 2024-04-07T16:40:20.744 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-w4lX5goVHJ0ixyn off=2586503 len=437387 2024-04-07T16:40:20.750 INFO:tasks.workunit.client.0.smithi100.stdout:op 517 completed, throughput=4.96MB/sec 2024-04-07T16:40:20.750 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-4Cfd2xUtoVJngSt off=9450988 len=210838 2024-04-07T16:40:20.752 INFO:tasks.workunit.client.0.smithi100.stdout:op 518 completed, throughput=4.97MB/sec 2024-04-07T16:40:20.752 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-s-ZBLQUm1ILSRf7 off=13702934 len=1468528 2024-04-07T16:40:20.755 INFO:tasks.workunit.client.0.smithi100.stdout:op 519 completed, throughput=4.98MB/sec 2024-04-07T16:40:20.755 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-m54rOB0L5zTcS8Q off=2803915 len=306903 2024-04-07T16:40:20.761 INFO:tasks.workunit.client.0.smithi100.stdout:op 520 completed, throughput=4.98MB/sec 2024-04-07T16:40:20.762 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-U60gXNhjSMhA7H5 off=2237156 len=1610193 2024-04-07T16:40:20.775 INFO:tasks.workunit.client.0.smithi100.stdout:op 521 completed, throughput=5MB/sec 2024-04-07T16:40:20.775 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-_Fe9s6-8_Ym_d2m off=2851626 len=934041 2024-04-07T16:40:20.784 INFO:tasks.workunit.client.0.smithi100.stdout:op 522 completed, throughput=5MB/sec 2024-04-07T16:40:21.784 INFO:tasks.workunit.client.0.smithi100.stdout: 102: throughput=4.96MB/sec pending data=0 2024-04-07T16:40:21.784 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-D-hOYTeUkZNbAux off=17276846 len=296345 2024-04-07T16:40:21.786 INFO:tasks.workunit.client.0.smithi100.stdout:op 523 completed, throughput=4.96MB/sec 2024-04-07T16:40:21.786 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-AvxfRzO-uiVmUY9 off=1186967 len=758693 2024-04-07T16:40:21.794 INFO:tasks.workunit.client.0.smithi100.stdout:op 524 completed, throughput=4.97MB/sec 2024-04-07T16:40:21.794 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-wvUXcWs-mIV33EB off=0 len=1410968 2024-04-07T16:40:21.805 INFO:tasks.workunit.client.0.smithi100.stdout:op 525 completed, throughput=4.98MB/sec 2024-04-07T16:40:21.805 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-xJjm00dZn_HIjin off=1496335 len=1335769 2024-04-07T16:40:21.807 INFO:tasks.workunit.client.0.smithi100.stdout:op 526 completed, throughput=4.99MB/sec 2024-04-07T16:40:21.807 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-2NXqEBx1fMoeiIU off=15598462 len=974916 2024-04-07T16:40:21.816 INFO:tasks.workunit.client.0.smithi100.stdout:op 527 completed, throughput=5MB/sec 2024-04-07T16:40:21.816 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-rXuH7ivyEVPrC4I off=2250841 len=842003 2024-04-07T16:40:21.823 INFO:tasks.workunit.client.0.smithi100.stdout:op 528 completed, throughput=5.01MB/sec 2024-04-07T16:40:22.442 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:40:22 smithi100 bash[74942]: cluster 2024-04-07T16:40:21.135053+0000 mgr.y (mgr.25175) 750 : cluster [DBG] pgmap v1392: 105 pgs: 105 active+clean; 126 GiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 2.3 MiB/s rd, 2.7 MiB/s wr, 5 op/s 2024-04-07T16:40:22.442 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:40:22 smithi100 bash[76202]: cluster 2024-04-07T16:40:21.135053+0000 mgr.y (mgr.25175) 750 : cluster [DBG] pgmap v1392: 105 pgs: 105 active+clean; 126 GiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 2.3 MiB/s rd, 2.7 MiB/s wr, 5 op/s 2024-04-07T16:40:22.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:40:22 smithi144 bash[66654]: cluster 2024-04-07T16:40:21.135053+0000 mgr.y (mgr.25175) 750 : cluster [DBG] pgmap v1392: 105 pgs: 105 active+clean; 126 GiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 2.3 MiB/s rd, 2.7 MiB/s wr, 5 op/s 2024-04-07T16:40:22.607 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:40:22 smithi144 bash[95233]: ts=2024-04-07T16:40:22.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:40:22.824 INFO:tasks.workunit.client.0.smithi100.stdout: 103: throughput=4.96MB/sec pending data=0 2024-04-07T16:40:22.824 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-XH2Owj0iKmI-l3G off=2896594 len=1704667 2024-04-07T16:40:22.829 INFO:tasks.workunit.client.0.smithi100.stdout:op 529 completed, throughput=4.97MB/sec 2024-04-07T16:40:22.829 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-q0cnP43WgPi5K9x off=945879 len=1708152 2024-04-07T16:40:22.844 INFO:tasks.workunit.client.0.smithi100.stdout:op 530 completed, throughput=4.99MB/sec 2024-04-07T16:40:22.844 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-vorvUipWSuKO4uT off=3277981 len=848054 2024-04-07T16:40:22.852 INFO:tasks.workunit.client.0.smithi100.stdout:op 531 completed, throughput=5MB/sec 2024-04-07T16:40:22.852 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-HZ_AE_OZcspy4FO off=932764 len=478001 2024-04-07T16:40:22.854 INFO:tasks.workunit.client.0.smithi100.stdout:op 532 completed, throughput=5MB/sec 2024-04-07T16:40:22.854 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-oZKnr7YdCkSLur6 off=5683673 len=444197 2024-04-07T16:40:22.860 INFO:tasks.workunit.client.0.smithi100.stdout:op 533 completed, throughput=5MB/sec 2024-04-07T16:40:23.860 INFO:tasks.workunit.client.0.smithi100.stdout: 104: throughput=4.96MB/sec pending data=0 2024-04-07T16:40:23.941 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-berinTfHw7oMqAS off=1348708 len=758190 2024-04-07T16:40:23.943 INFO:tasks.workunit.client.0.smithi100.stdout:op 534 completed, throughput=4.96MB/sec 2024-04-07T16:40:23.943 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-CZQ2YOJMga4CF1r off=19407310 len=332931 2024-04-07T16:40:23.943 INFO:tasks.workunit.client.0.smithi100.stdout:op 535 completed, throughput=4.97MB/sec 2024-04-07T16:40:23.943 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-EaAgwQ-oTDorU_Z off=1634059 len=891355 2024-04-07T16:40:23.943 INFO:tasks.workunit.client.0.smithi100.stdout:op 536 completed, throughput=4.97MB/sec 2024-04-07T16:40:23.943 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-USszGY3_PEKgRjV off=3078980 len=996649 2024-04-07T16:40:23.943 INFO:tasks.workunit.client.0.smithi100.stdout:op 537 completed, throughput=4.98MB/sec 2024-04-07T16:40:23.944 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-2_IL0MvQWMW09al off=697620 len=198370 2024-04-07T16:40:23.944 INFO:tasks.workunit.client.0.smithi100.stdout:op 538 completed, throughput=4.98MB/sec 2024-04-07T16:40:23.944 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Qt1Wx4OQ5T54WyN off=6620485 len=189790 2024-04-07T16:40:23.944 INFO:tasks.workunit.client.0.smithi100.stdout:op 539 completed, throughput=4.99MB/sec 2024-04-07T16:40:23.944 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-82Sz_mWAocFaCAB off=9301996 len=1171256 2024-04-07T16:40:23.944 INFO:tasks.workunit.client.0.smithi100.stdout:op 540 completed, throughput=5MB/sec 2024-04-07T16:40:23.944 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-KPA3d6J1PiCfRLi off=4971600 len=985109 2024-04-07T16:40:23.944 INFO:tasks.workunit.client.0.smithi100.stdout:op 541 completed, throughput=5MB/sec 2024-04-07T16:40:24.442 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:40:24 smithi100 bash[74942]: cluster 2024-04-07T16:40:23.136093+0000 mgr.y (mgr.25175) 751 : cluster [DBG] pgmap v1393: 105 pgs: 105 active+clean; 126 GiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 2.7 MiB/s rd, 3.2 MiB/s wr, 5 op/s 2024-04-07T16:40:24.442 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:40:24 smithi100 bash[76202]: cluster 2024-04-07T16:40:23.136093+0000 mgr.y (mgr.25175) 751 : cluster [DBG] pgmap v1393: 105 pgs: 105 active+clean; 126 GiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 2.7 MiB/s rd, 3.2 MiB/s wr, 5 op/s 2024-04-07T16:40:24.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:40:24 smithi144 bash[66654]: cluster 2024-04-07T16:40:23.136093+0000 mgr.y (mgr.25175) 751 : cluster [DBG] pgmap v1393: 105 pgs: 105 active+clean; 126 GiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 2.7 MiB/s rd, 3.2 MiB/s wr, 5 op/s 2024-04-07T16:40:24.895 INFO:tasks.workunit.client.0.smithi100.stdout: 105: throughput=4.96MB/sec pending data=0 2024-04-07T16:40:24.895 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-pfnwdt8Tecjhw7X off=4652711 len=1967339 2024-04-07T16:40:24.900 INFO:tasks.workunit.client.0.smithi100.stdout:op 542 completed, throughput=4.98MB/sec 2024-04-07T16:40:24.900 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-mbBwDnqJpbrJ3CW off=7424417 len=2064436 2024-04-07T16:40:24.904 INFO:tasks.workunit.client.0.smithi100.stdout:op 543 completed, throughput=4.99MB/sec 2024-04-07T16:40:24.904 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-AxBnjXt3eS_d51D off=4698274 len=1238006 2024-04-07T16:40:24.908 INFO:tasks.workunit.client.0.smithi100.stdout:op 544 completed, throughput=5MB/sec 2024-04-07T16:40:25.908 INFO:tasks.workunit.client.0.smithi100.stdout: 106: throughput=4.96MB/sec pending data=0 2024-04-07T16:40:25.908 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-tw_4UfkVhKdLvgi off=11310705 len=948684 2024-04-07T16:40:25.916 INFO:tasks.workunit.client.0.smithi100.stdout:op 545 completed, throughput=4.97MB/sec 2024-04-07T16:40:25.916 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-LfR8NWhRSyypz6d off=10740788 len=734055 2024-04-07T16:40:25.918 INFO:tasks.workunit.client.0.smithi100.stdout:op 546 completed, throughput=4.97MB/sec 2024-04-07T16:40:25.918 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-k2xS6oEJI3-5A-a off=15361373 len=1615484 2024-04-07T16:40:25.923 INFO:tasks.workunit.client.0.smithi100.stdout:op 547 completed, throughput=4.99MB/sec 2024-04-07T16:40:25.924 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-r3OJREcWzjZ5Qah off=8754658 len=181134 2024-04-07T16:40:25.929 INFO:tasks.workunit.client.0.smithi100.stdout:op 548 completed, throughput=4.99MB/sec 2024-04-07T16:40:25.929 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Bd-ZeJ4v1xg5gcB off=1676117 len=1685832 2024-04-07T16:40:25.933 INFO:tasks.workunit.client.0.smithi100.stdout:op 549 completed, throughput=5MB/sec 2024-04-07T16:40:26.442 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:40:26 smithi100 bash[74942]: cluster 2024-04-07T16:40:25.137080+0000 mgr.y (mgr.25175) 752 : cluster [DBG] pgmap v1394: 105 pgs: 105 active+clean; 126 GiB data, 3.7 GiB used, 711 GiB / 715 GiB avail; 2.1 MiB/s rd, 3.0 MiB/s wr, 5 op/s 2024-04-07T16:40:26.443 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:40:26 smithi100 bash[76202]: cluster 2024-04-07T16:40:25.137080+0000 mgr.y (mgr.25175) 752 : cluster [DBG] pgmap v1394: 105 pgs: 105 active+clean; 126 GiB data, 3.7 GiB used, 711 GiB / 715 GiB avail; 2.1 MiB/s rd, 3.0 MiB/s wr, 5 op/s 2024-04-07T16:40:26.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:40:26 smithi144 bash[66654]: cluster 2024-04-07T16:40:25.137080+0000 mgr.y (mgr.25175) 752 : cluster [DBG] pgmap v1394: 105 pgs: 105 active+clean; 126 GiB data, 3.7 GiB used, 711 GiB / 715 GiB avail; 2.1 MiB/s rd, 3.0 MiB/s wr, 5 op/s 2024-04-07T16:40:26.933 INFO:tasks.workunit.client.0.smithi100.stdout: 107: throughput=4.96MB/sec pending data=0 2024-04-07T16:40:26.933 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-ed2VaU6W2GcRUma off=7792454 len=1920813 2024-04-07T16:40:26.948 INFO:tasks.workunit.client.0.smithi100.stdout:op 550 completed, throughput=4.97MB/sec 2024-04-07T16:40:26.949 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Pyp71AvoqZEUjJy off=15989774 len=137780 2024-04-07T16:40:26.953 INFO:tasks.workunit.client.0.smithi100.stdout:op 551 completed, throughput=4.97MB/sec 2024-04-07T16:40:26.953 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-5GzcihHixrItT6T off=4227532 len=935666 2024-04-07T16:40:26.956 INFO:tasks.workunit.client.0.smithi100.stdout:op 552 completed, throughput=4.98MB/sec 2024-04-07T16:40:26.956 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-0cbt9fyOjxVXTXN off=5228866 len=661384 2024-04-07T16:40:26.970 INFO:tasks.workunit.client.0.smithi100.stdout:op 553 completed, throughput=4.99MB/sec 2024-04-07T16:40:26.970 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-grajuHr7RyxHEqc off=580152 len=578810 2024-04-07T16:40:26.973 INFO:tasks.workunit.client.0.smithi100.stdout:op 554 completed, throughput=4.99MB/sec 2024-04-07T16:40:26.973 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-QiXONvOaGAH-Fho off=8104 len=178459 2024-04-07T16:40:26.974 INFO:tasks.workunit.client.0.smithi100.stdout:op 555 completed, throughput=4.99MB/sec 2024-04-07T16:40:26.974 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-CRm6CrPCnsmPFSr off=0 len=1118167 2024-04-07T16:40:26.983 INFO:tasks.workunit.client.0.smithi100.stdout:op 556 completed, throughput=5MB/sec 2024-04-07T16:40:27.984 INFO:tasks.workunit.client.0.smithi100.stdout: 108: throughput=4.96MB/sec pending data=0 2024-04-07T16:40:27.984 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-qiwJo02ga2jGJ1C off=5968971 len=12678 2024-04-07T16:40:27.985 INFO:tasks.workunit.client.0.smithi100.stdout:op 557 completed, throughput=4.96MB/sec 2024-04-07T16:40:27.985 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-VmiT5FdN4FDRp1- off=11779517 len=1620613 2024-04-07T16:40:27.989 INFO:tasks.workunit.client.0.smithi100.stdout:op 558 completed, throughput=4.97MB/sec 2024-04-07T16:40:27.989 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-ZGWVqAp3A47TBZe off=4210652 len=1592518 2024-04-07T16:40:27.993 INFO:tasks.workunit.client.0.smithi100.stdout:op 559 completed, throughput=4.98MB/sec 2024-04-07T16:40:27.993 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-JNMsJKWdDWHnpBa off=4666291 len=1841090 2024-04-07T16:40:27.997 INFO:tasks.workunit.client.0.smithi100.stdout:op 560 completed, throughput=5MB/sec 2024-04-07T16:40:28.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:40:28 smithi144 bash[66654]: cluster 2024-04-07T16:40:27.138220+0000 mgr.y (mgr.25175) 753 : cluster [DBG] pgmap v1395: 105 pgs: 105 active+clean; 126 GiB data, 3.7 GiB used, 711 GiB / 715 GiB avail; 2.2 MiB/s rd, 2.7 MiB/s wr, 5 op/s 2024-04-07T16:40:28.692 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:40:28 smithi100 bash[74942]: cluster 2024-04-07T16:40:27.138220+0000 mgr.y (mgr.25175) 753 : cluster [DBG] pgmap v1395: 105 pgs: 105 active+clean; 126 GiB data, 3.7 GiB used, 711 GiB / 715 GiB avail; 2.2 MiB/s rd, 2.7 MiB/s wr, 5 op/s 2024-04-07T16:40:28.692 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:40:28 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:40:28] "GET /metrics HTTP/1.1" 200 34053 "" "Prometheus/2.43.0" 2024-04-07T16:40:28.693 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:40:28 smithi100 bash[76202]: cluster 2024-04-07T16:40:27.138220+0000 mgr.y (mgr.25175) 753 : cluster [DBG] pgmap v1395: 105 pgs: 105 active+clean; 126 GiB data, 3.7 GiB used, 711 GiB / 715 GiB avail; 2.2 MiB/s rd, 2.7 MiB/s wr, 5 op/s 2024-04-07T16:40:28.997 INFO:tasks.workunit.client.0.smithi100.stdout: 109: throughput=4.96MB/sec pending data=0 2024-04-07T16:40:28.997 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-fvbeeCXZvTqajYK off=3954733 len=993964 2024-04-07T16:40:29.007 INFO:tasks.workunit.client.0.smithi100.stdout:op 561 completed, throughput=4.96MB/sec 2024-04-07T16:40:29.007 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-B-JU2hdiWhWDB4E off=20327889 len=708371 2024-04-07T16:40:29.009 INFO:tasks.workunit.client.0.smithi100.stdout:op 562 completed, throughput=4.97MB/sec 2024-04-07T16:40:29.009 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-PeZX4kSOwSrF1wB off=3353717 len=1947517 2024-04-07T16:40:29.016 INFO:tasks.workunit.client.0.smithi100.stdout:op 563 completed, throughput=4.99MB/sec 2024-04-07T16:40:29.016 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-MWejPoX299wpTDd off=7850757 len=2048571 2024-04-07T16:40:29.032 INFO:tasks.workunit.client.0.smithi100.stdout:op 564 completed, throughput=5MB/sec 2024-04-07T16:40:30.033 INFO:tasks.workunit.client.0.smithi100.stdout: 110: throughput=4.96MB/sec pending data=0 2024-04-07T16:40:30.033 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-gBmO4DGVq-1e2_X off=1805813 len=979798 2024-04-07T16:40:30.041 INFO:tasks.workunit.client.0.smithi100.stdout:op 565 completed, throughput=4.97MB/sec 2024-04-07T16:40:30.041 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-mAPuyvkSRePTkPZ off=19241055 len=742179 2024-04-07T16:40:30.049 INFO:tasks.workunit.client.0.smithi100.stdout:op 566 completed, throughput=4.97MB/sec 2024-04-07T16:40:30.049 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-8_aX7K6cMXiSSeB off=8938250 len=1053833 2024-04-07T16:40:30.052 INFO:tasks.workunit.client.0.smithi100.stdout:op 567 completed, throughput=4.98MB/sec 2024-04-07T16:40:30.052 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-R2ZbBhbg3mYAaot off=9300393 len=397525 2024-04-07T16:40:30.058 INFO:tasks.workunit.client.0.smithi100.stdout:op 568 completed, throughput=4.98MB/sec 2024-04-07T16:40:30.058 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-o_ZkWu2HKemOqvC off=1324460 len=1194191 2024-04-07T16:40:30.068 INFO:tasks.workunit.client.0.smithi100.stdout:op 569 completed, throughput=4.99MB/sec 2024-04-07T16:40:30.068 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-uTvTuG6_dw0Eyhc off=504784 len=1802953 2024-04-07T16:40:30.086 INFO:tasks.workunit.client.0.smithi100.stdout:op 570 completed, throughput=5.01MB/sec 2024-04-07T16:40:30.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:40:30 smithi144 bash[66654]: cluster 2024-04-07T16:40:29.139088+0000 mgr.y (mgr.25175) 754 : cluster [DBG] pgmap v1396: 105 pgs: 105 active+clean; 126 GiB data, 3.7 GiB used, 711 GiB / 715 GiB avail; 2.3 MiB/s rd, 2.4 MiB/s wr, 4 op/s 2024-04-07T16:40:30.692 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:40:30 smithi100 bash[74942]: cluster 2024-04-07T16:40:29.139088+0000 mgr.y (mgr.25175) 754 : cluster [DBG] pgmap v1396: 105 pgs: 105 active+clean; 126 GiB data, 3.7 GiB used, 711 GiB / 715 GiB avail; 2.3 MiB/s rd, 2.4 MiB/s wr, 4 op/s 2024-04-07T16:40:30.692 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:40:30 smithi100 bash[76202]: cluster 2024-04-07T16:40:29.139088+0000 mgr.y (mgr.25175) 754 : cluster [DBG] pgmap v1396: 105 pgs: 105 active+clean; 126 GiB data, 3.7 GiB used, 711 GiB / 715 GiB avail; 2.3 MiB/s rd, 2.4 MiB/s wr, 4 op/s 2024-04-07T16:40:31.086 INFO:tasks.workunit.client.0.smithi100.stdout: 111: throughput=4.96MB/sec pending data=0 2024-04-07T16:40:31.087 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Gz51HCfhSweRAqy off=16948095 len=1236434 2024-04-07T16:40:31.091 INFO:tasks.workunit.client.0.smithi100.stdout:op 571 completed, throughput=4.97MB/sec 2024-04-07T16:40:31.091 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-02oyuQrGsJDPjm9 off=10368027 len=988598 2024-04-07T16:40:31.095 INFO:tasks.workunit.client.0.smithi100.stdout:op 572 completed, throughput=4.98MB/sec 2024-04-07T16:40:31.095 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-iam2-e-JZw0BZ4X off=3980899 len=1668981 2024-04-07T16:40:31.099 INFO:tasks.workunit.client.0.smithi100.stdout:op 573 completed, throughput=5MB/sec 2024-04-07T16:40:31.099 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-IIm9qhSSdx5OyEp off=3387739 len=1136028 2024-04-07T16:40:31.113 INFO:tasks.workunit.client.0.smithi100.stdout:op 574 completed, throughput=5MB/sec 2024-04-07T16:40:32.112 INFO:tasks.workunit.client.0.smithi100.stdout: 112: throughput=4.96MB/sec pending data=0 2024-04-07T16:40:32.112 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-1mK5Y79ZKrKdtX9 off=11214123 len=137400 2024-04-07T16:40:32.113 INFO:tasks.workunit.client.0.smithi100.stdout:op 575 completed, throughput=4.96MB/sec 2024-04-07T16:40:32.113 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-YaIkxwQ3PDIp2qU off=443092 len=1761530 2024-04-07T16:40:32.129 INFO:tasks.workunit.client.0.smithi100.stdout:op 576 completed, throughput=4.98MB/sec 2024-04-07T16:40:32.129 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-FcE2zD26TukZzqn off=4346787 len=1029415 2024-04-07T16:40:32.132 INFO:tasks.workunit.client.0.smithi100.stdout:op 577 completed, throughput=4.98MB/sec 2024-04-07T16:40:32.132 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-F5Pq7gCz1b6G3QU off=1369040 len=490431 2024-04-07T16:40:32.134 INFO:tasks.workunit.client.0.smithi100.stdout:op 578 completed, throughput=4.99MB/sec 2024-04-07T16:40:32.134 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-fZwjjdMYMd18EjV off=2235666 len=333222 2024-04-07T16:40:32.140 INFO:tasks.workunit.client.0.smithi100.stdout:op 579 completed, throughput=4.99MB/sec 2024-04-07T16:40:32.140 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-kUew7oN-c-OBaFB off=1377403 len=1727224 2024-04-07T16:40:32.157 INFO:tasks.workunit.client.0.smithi100.stdout:op 580 completed, throughput=5MB/sec 2024-04-07T16:40:32.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:40:32 smithi144 bash[66654]: cluster 2024-04-07T16:40:31.140043+0000 mgr.y (mgr.25175) 755 : cluster [DBG] pgmap v1397: 105 pgs: 105 active+clean; 126 GiB data, 3.7 GiB used, 711 GiB / 715 GiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 4 op/s 2024-04-07T16:40:32.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:40:32 smithi144 bash[66654]: audit 2024-04-07T16:40:31.757422+0000 mon.a (mon.0) 2059 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:40:32.607 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:40:32 smithi144 bash[95233]: ts=2024-04-07T16:40:32.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:40:32.692 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:40:32 smithi100 bash[74942]: cluster 2024-04-07T16:40:31.140043+0000 mgr.y (mgr.25175) 755 : cluster [DBG] pgmap v1397: 105 pgs: 105 active+clean; 126 GiB data, 3.7 GiB used, 711 GiB / 715 GiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 4 op/s 2024-04-07T16:40:32.692 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:40:32 smithi100 bash[74942]: audit 2024-04-07T16:40:31.757422+0000 mon.a (mon.0) 2059 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:40:32.693 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:40:32 smithi100 bash[76202]: cluster 2024-04-07T16:40:31.140043+0000 mgr.y (mgr.25175) 755 : cluster [DBG] pgmap v1397: 105 pgs: 105 active+clean; 126 GiB data, 3.7 GiB used, 711 GiB / 715 GiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 4 op/s 2024-04-07T16:40:32.693 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:40:32 smithi100 bash[76202]: audit 2024-04-07T16:40:31.757422+0000 mon.a (mon.0) 2059 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:40:33.158 INFO:tasks.workunit.client.0.smithi100.stdout: 113: throughput=4.96MB/sec pending data=0 2024-04-07T16:40:33.158 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-oSWJ0XgvioWG7W3 off=2025984 len=1790955 2024-04-07T16:40:33.173 INFO:tasks.workunit.client.0.smithi100.stdout:op 581 completed, throughput=4.98MB/sec 2024-04-07T16:40:33.173 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-4XmKuyBlQvXHlXJ off=3059182 len=945642 2024-04-07T16:40:33.177 INFO:tasks.workunit.client.0.smithi100.stdout:op 582 completed, throughput=4.98MB/sec 2024-04-07T16:40:33.177 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-mL3uouejCuwxdsG off=2232828 len=46727 2024-04-07T16:40:33.181 INFO:tasks.workunit.client.0.smithi100.stdout:op 583 completed, throughput=4.98MB/sec 2024-04-07T16:40:33.181 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-PO09ZGs0Nn1CVLp off=18307549 len=53928 2024-04-07T16:40:33.186 INFO:tasks.workunit.client.0.smithi100.stdout:op 584 completed, throughput=4.98MB/sec 2024-04-07T16:40:33.186 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-uBiMeWTGza66UrT off=1023387 len=1925002 2024-04-07T16:40:33.190 INFO:tasks.workunit.client.0.smithi100.stdout:op 585 completed, throughput=5MB/sec 2024-04-07T16:40:33.190 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-pUUETi6tPEvqutB off=5283588 len=985771 2024-04-07T16:40:33.194 INFO:tasks.workunit.client.0.smithi100.stdout:op 586 completed, throughput=5.01MB/sec 2024-04-07T16:40:34.194 INFO:tasks.workunit.client.0.smithi100.stdout: 114: throughput=4.96MB/sec pending data=0 2024-04-07T16:40:34.194 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-6Cj-kp2NdkOcn9d off=17823849 len=1978371 2024-04-07T16:40:34.213 INFO:tasks.workunit.client.0.smithi100.stdout:op 587 completed, throughput=4.98MB/sec 2024-04-07T16:40:34.214 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-xr_AU4wCuWEZMPm off=5035815 len=1808177 2024-04-07T16:40:34.229 INFO:tasks.workunit.client.0.smithi100.stdout:op 588 completed, throughput=4.99MB/sec 2024-04-07T16:40:34.229 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-srItPO81fa730yF off=1007022 len=1217626 2024-04-07T16:40:34.232 INFO:tasks.workunit.client.0.smithi100.stdout:op 589 completed, throughput=5MB/sec 2024-04-07T16:40:34.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:40:34 smithi144 bash[66654]: cluster 2024-04-07T16:40:33.141078+0000 mgr.y (mgr.25175) 756 : cluster [DBG] pgmap v1398: 105 pgs: 105 active+clean; 126 GiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 2.6 MiB/s rd, 2.3 MiB/s wr, 5 op/s 2024-04-07T16:40:34.692 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:40:34 smithi100 bash[74942]: cluster 2024-04-07T16:40:33.141078+0000 mgr.y (mgr.25175) 756 : cluster [DBG] pgmap v1398: 105 pgs: 105 active+clean; 126 GiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 2.6 MiB/s rd, 2.3 MiB/s wr, 5 op/s 2024-04-07T16:40:34.692 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:40:34 smithi100 bash[76202]: cluster 2024-04-07T16:40:33.141078+0000 mgr.y (mgr.25175) 756 : cluster [DBG] pgmap v1398: 105 pgs: 105 active+clean; 126 GiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 2.6 MiB/s rd, 2.3 MiB/s wr, 5 op/s 2024-04-07T16:40:35.232 INFO:tasks.workunit.client.0.smithi100.stdout: 115: throughput=4.96MB/sec pending data=0 2024-04-07T16:40:35.232 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-D5qTeiqJcvee0bc off=8595525 len=741399 2024-04-07T16:40:35.235 INFO:tasks.workunit.client.0.smithi100.stdout:op 590 completed, throughput=4.97MB/sec 2024-04-07T16:40:35.235 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-KVdbHHwKm-tfE28 off=3628963 len=19050 2024-04-07T16:40:35.239 INFO:tasks.workunit.client.0.smithi100.stdout:op 591 completed, throughput=4.97MB/sec 2024-04-07T16:40:35.239 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-cD7n1Jo9hjHv4AK off=2913627 len=1713396 2024-04-07T16:40:35.252 INFO:tasks.workunit.client.0.smithi100.stdout:op 592 completed, throughput=4.98MB/sec 2024-04-07T16:40:35.252 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-oBk1kwO1XkBAPzE off=3781848 len=1170605 2024-04-07T16:40:35.262 INFO:tasks.workunit.client.0.smithi100.stdout:op 593 completed, throughput=4.99MB/sec 2024-04-07T16:40:35.262 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-X5QOsfbTXPRvyAL off=2534096 len=1764160 2024-04-07T16:40:35.266 INFO:tasks.workunit.client.0.smithi100.stdout:op 594 completed, throughput=5MB/sec 2024-04-07T16:40:36.266 INFO:tasks.workunit.client.0.smithi100.stdout: 116: throughput=4.96MB/sec pending data=0 2024-04-07T16:40:36.266 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-I2Z4I5tr-7SYOqu off=5299053 len=1301769 2024-04-07T16:40:36.277 INFO:tasks.workunit.client.0.smithi100.stdout:op 595 completed, throughput=4.97MB/sec 2024-04-07T16:40:36.277 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-v8K8OFVz7xkJkOj off=17873882 len=125269 2024-04-07T16:40:36.278 INFO:tasks.workunit.client.0.smithi100.stdout:op 596 completed, throughput=4.97MB/sec 2024-04-07T16:40:36.279 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-HOZfzBRP6LEi54u off=4305509 len=156615 2024-04-07T16:40:36.280 INFO:tasks.workunit.client.0.smithi100.stdout:op 597 completed, throughput=4.97MB/sec 2024-04-07T16:40:36.280 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-NNIF9DzTBSetCdu off=16988234 len=1770472 2024-04-07T16:40:36.284 INFO:tasks.workunit.client.0.smithi100.stdout:op 598 completed, throughput=4.99MB/sec 2024-04-07T16:40:36.284 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-09mKhBOhMrWCgMl off=8445708 len=1113027 2024-04-07T16:40:36.292 INFO:tasks.workunit.client.0.smithi100.stdout:op 599 completed, throughput=5MB/sec 2024-04-07T16:40:36.292 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-qLmxJhJd9I3C7me off=2978408 len=841823 2024-04-07T16:40:36.300 INFO:tasks.workunit.client.0.smithi100.stdout:op 600 completed, throughput=5MB/sec 2024-04-07T16:40:36.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:40:36 smithi144 bash[66654]: cluster 2024-04-07T16:40:35.142062+0000 mgr.y (mgr.25175) 757 : cluster [DBG] pgmap v1399: 105 pgs: 105 active+clean; 126 GiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 2.4 MiB/s rd, 1.7 MiB/s wr, 4 op/s 2024-04-07T16:40:36.692 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:40:36 smithi100 bash[74942]: cluster 2024-04-07T16:40:35.142062+0000 mgr.y (mgr.25175) 757 : cluster [DBG] pgmap v1399: 105 pgs: 105 active+clean; 126 GiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 2.4 MiB/s rd, 1.7 MiB/s wr, 4 op/s 2024-04-07T16:40:36.692 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:40:36 smithi100 bash[76202]: cluster 2024-04-07T16:40:35.142062+0000 mgr.y (mgr.25175) 757 : cluster [DBG] pgmap v1399: 105 pgs: 105 active+clean; 126 GiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 2.4 MiB/s rd, 1.7 MiB/s wr, 4 op/s 2024-04-07T16:40:37.300 INFO:tasks.workunit.client.0.smithi100.stdout: 117: throughput=4.96MB/sec pending data=0 2024-04-07T16:40:37.300 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-B4QdmhtlBuT6W5G off=5557245 len=1627413 2024-04-07T16:40:37.306 INFO:tasks.workunit.client.0.smithi100.stdout:op 601 completed, throughput=4.97MB/sec 2024-04-07T16:40:37.306 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj--Yhj2mVFJiYk4uE off=4609790 len=153655 2024-04-07T16:40:37.308 INFO:tasks.workunit.client.0.smithi100.stdout:op 602 completed, throughput=4.98MB/sec 2024-04-07T16:40:37.308 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-30xRcz9agwqeZGn off=2011103 len=316338 2024-04-07T16:40:37.310 INFO:tasks.workunit.client.0.smithi100.stdout:op 603 completed, throughput=4.98MB/sec 2024-04-07T16:40:37.310 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-bLkJRu-6IlMMXyZ off=1337331 len=1236498 2024-04-07T16:40:37.320 INFO:tasks.workunit.client.0.smithi100.stdout:op 604 completed, throughput=4.99MB/sec 2024-04-07T16:40:37.320 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-ip8-wRmZwnG3Pvb off=11247226 len=1444023 2024-04-07T16:40:37.324 INFO:tasks.workunit.client.0.smithi100.stdout:op 605 completed, throughput=5MB/sec 2024-04-07T16:40:37.324 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-I4aZM9hu3nqedW6 off=49819 len=1776237 2024-04-07T16:40:37.339 INFO:tasks.workunit.client.0.smithi100.stdout:op 606 completed, throughput=5.01MB/sec 2024-04-07T16:40:38.340 INFO:tasks.workunit.client.0.smithi100.stdout: 118: throughput=4.97MB/sec pending data=0 2024-04-07T16:40:38.340 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Vme37VFfH2_w7Xy off=11093382 len=1156212 2024-04-07T16:40:38.345 INFO:tasks.workunit.client.0.smithi100.stdout:op 607 completed, throughput=4.98MB/sec 2024-04-07T16:40:38.345 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Jyn_NDvxegippUV off=7047215 len=739011 2024-04-07T16:40:38.353 INFO:tasks.workunit.client.0.smithi100.stdout:op 608 completed, throughput=4.98MB/sec 2024-04-07T16:40:38.353 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-OE3fs8BP9kVy45O off=323302 len=2034820 2024-04-07T16:40:38.372 INFO:tasks.workunit.client.0.smithi100.stdout:op 609 completed, throughput=5MB/sec 2024-04-07T16:40:38.372 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-ZX2Lm3TBMQpSmuo off=13201557 len=1284820 2024-04-07T16:40:38.377 INFO:tasks.workunit.client.0.smithi100.stdout:op 610 completed, throughput=5.01MB/sec 2024-04-07T16:40:38.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:40:38 smithi144 bash[66654]: cluster 2024-04-07T16:40:37.143089+0000 mgr.y (mgr.25175) 758 : cluster [DBG] pgmap v1400: 105 pgs: 105 active+clean; 126 GiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.1 MiB/s wr, 4 op/s 2024-04-07T16:40:38.692 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:40:38 smithi100 bash[74942]: cluster 2024-04-07T16:40:37.143089+0000 mgr.y (mgr.25175) 758 : cluster [DBG] pgmap v1400: 105 pgs: 105 active+clean; 126 GiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.1 MiB/s wr, 4 op/s 2024-04-07T16:40:38.692 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:40:38 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:40:38] "GET /metrics HTTP/1.1" 200 34053 "" "Prometheus/2.43.0" 2024-04-07T16:40:38.693 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:40:38 smithi100 bash[76202]: cluster 2024-04-07T16:40:37.143089+0000 mgr.y (mgr.25175) 758 : cluster [DBG] pgmap v1400: 105 pgs: 105 active+clean; 126 GiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.1 MiB/s wr, 4 op/s 2024-04-07T16:40:39.377 INFO:tasks.workunit.client.0.smithi100.stdout: 119: throughput=4.97MB/sec pending data=0 2024-04-07T16:40:39.377 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-JNMsJKWdDWHnpBa off=1510667 len=1138905 2024-04-07T16:40:39.387 INFO:tasks.workunit.client.0.smithi100.stdout:op 611 completed, throughput=4.98MB/sec 2024-04-07T16:40:39.387 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-9aApfBCaJ7AkzLl off=1961841 len=1312834 2024-04-07T16:40:39.397 INFO:tasks.workunit.client.0.smithi100.stdout:op 612 completed, throughput=4.99MB/sec 2024-04-07T16:40:39.397 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-o5s_rZQr0Mt0vwy off=18095066 len=1423847 2024-04-07T16:40:39.401 INFO:tasks.workunit.client.0.smithi100.stdout:op 613 completed, throughput=5MB/sec 2024-04-07T16:40:39.401 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-TaaX9BYdADoskG7 off=2977173 len=1193723 2024-04-07T16:40:39.404 INFO:tasks.workunit.client.0.smithi100.stdout:op 614 completed, throughput=5.01MB/sec 2024-04-07T16:40:40.405 INFO:tasks.workunit.client.0.smithi100.stdout: 120: throughput=4.97MB/sec pending data=0 2024-04-07T16:40:40.405 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-FBBo_7pjjE_0SsT off=1792037 len=1811699 2024-04-07T16:40:40.420 INFO:tasks.workunit.client.0.smithi100.stdout:op 615 completed, throughput=4.98MB/sec 2024-04-07T16:40:40.420 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-_Fe9s6-8_Ym_d2m off=4332075 len=1221056 2024-04-07T16:40:40.424 INFO:tasks.workunit.client.0.smithi100.stdout:op 616 completed, throughput=4.99MB/sec 2024-04-07T16:40:40.424 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-2T_6W2K2MQf5QXr off=2852672 len=1346348 2024-04-07T16:40:40.427 INFO:tasks.workunit.client.0.smithi100.stdout:op 617 completed, throughput=5MB/sec 2024-04-07T16:40:40.427 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-AfKS7dCycabIr97 off=1584352 len=457225 2024-04-07T16:40:40.434 INFO:tasks.workunit.client.0.smithi100.stdout:op 618 completed, throughput=5MB/sec 2024-04-07T16:40:40.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:40:40 smithi144 bash[66654]: cluster 2024-04-07T16:40:39.144032+0000 mgr.y (mgr.25175) 759 : cluster [DBG] pgmap v1401: 105 pgs: 105 active+clean; 126 GiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 2.3 MiB/s rd, 2.2 MiB/s wr, 4 op/s 2024-04-07T16:40:40.692 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:40:40 smithi100 bash[74942]: cluster 2024-04-07T16:40:39.144032+0000 mgr.y (mgr.25175) 759 : cluster [DBG] pgmap v1401: 105 pgs: 105 active+clean; 126 GiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 2.3 MiB/s rd, 2.2 MiB/s wr, 4 op/s 2024-04-07T16:40:40.692 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:40:40 smithi100 bash[76202]: cluster 2024-04-07T16:40:39.144032+0000 mgr.y (mgr.25175) 759 : cluster [DBG] pgmap v1401: 105 pgs: 105 active+clean; 126 GiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 2.3 MiB/s rd, 2.2 MiB/s wr, 4 op/s 2024-04-07T16:40:41.434 INFO:tasks.workunit.client.0.smithi100.stdout: 121: throughput=4.96MB/sec pending data=0 2024-04-07T16:40:41.434 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-VR6lggY4CqnidXo off=85283 len=1941081 2024-04-07T16:40:41.449 INFO:tasks.workunit.client.0.smithi100.stdout:op 619 completed, throughput=4.98MB/sec 2024-04-07T16:40:41.449 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-igunnfxfJBnNd_l off=11715011 len=54995 2024-04-07T16:40:41.451 INFO:tasks.workunit.client.0.smithi100.stdout:op 620 completed, throughput=4.98MB/sec 2024-04-07T16:40:41.451 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-iesU1UiRUEt8te6 off=7506888 len=88314 2024-04-07T16:40:41.452 INFO:tasks.workunit.client.0.smithi100.stdout:op 621 completed, throughput=4.98MB/sec 2024-04-07T16:40:41.452 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-LFeoib_SC8X9m51 off=3189950 len=675099 2024-04-07T16:40:41.460 INFO:tasks.workunit.client.0.smithi100.stdout:op 622 completed, throughput=4.98MB/sec 2024-04-07T16:40:41.460 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-nCNHggDeEgDB7c9 off=927628 len=825443 2024-04-07T16:40:41.462 INFO:tasks.workunit.client.0.smithi100.stdout:op 623 completed, throughput=4.99MB/sec 2024-04-07T16:40:41.462 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-1gsci6oxalFHYcn off=7057495 len=692082 2024-04-07T16:40:41.465 INFO:tasks.workunit.client.0.smithi100.stdout:op 624 completed, throughput=4.99MB/sec 2024-04-07T16:40:41.465 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-bd7RQ1dDiGF6bKE off=8547874 len=744532 2024-04-07T16:40:41.468 INFO:tasks.workunit.client.0.smithi100.stdout:op 625 completed, throughput=5MB/sec 2024-04-07T16:40:41.468 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-u0ZPa5qkBUW-vfz off=7083842 len=412974 2024-04-07T16:40:41.470 INFO:tasks.workunit.client.0.smithi100.stdout:op 626 completed, throughput=5MB/sec 2024-04-07T16:40:42.471 INFO:tasks.workunit.client.0.smithi100.stdout: 122: throughput=4.96MB/sec pending data=0 2024-04-07T16:40:42.471 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-AEJ68r3XtCHaZft off=1490161 len=665063 2024-04-07T16:40:42.473 INFO:tasks.workunit.client.0.smithi100.stdout:op 627 completed, throughput=4.97MB/sec 2024-04-07T16:40:42.473 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-t82TAlGqhbBQ0h2 off=7589806 len=504830 2024-04-07T16:40:42.480 INFO:tasks.workunit.client.0.smithi100.stdout:op 628 completed, throughput=4.97MB/sec 2024-04-07T16:40:42.480 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-BD2rkfGQBtfVB1I off=2308227 len=252237 2024-04-07T16:40:42.481 INFO:tasks.workunit.client.0.smithi100.stdout:op 629 completed, throughput=4.97MB/sec 2024-04-07T16:40:42.481 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-ARNFXo6zJV3L98b off=13503870 len=324026 2024-04-07T16:40:42.486 INFO:tasks.workunit.client.0.smithi100.stdout:op 630 completed, throughput=4.98MB/sec 2024-04-07T16:40:42.487 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Ip6x5uc5vKcVepb off=13611280 len=301707 2024-04-07T16:40:42.488 INFO:tasks.workunit.client.0.smithi100.stdout:op 631 completed, throughput=4.98MB/sec 2024-04-07T16:40:42.488 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-uXqs0FAXm7M02AB off=16324356 len=1059488 2024-04-07T16:40:42.499 INFO:tasks.workunit.client.0.smithi100.stdout:op 632 completed, throughput=4.99MB/sec 2024-04-07T16:40:42.499 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-PadHIv04adz8-22 off=6686958 len=1350436 2024-04-07T16:40:42.511 INFO:tasks.workunit.client.0.smithi100.stdout:op 633 completed, throughput=5MB/sec 2024-04-07T16:40:42.511 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-MdZ5vVGfCWX9ON7 off=3803805 len=1862145 2024-04-07T16:40:42.525 INFO:tasks.workunit.client.0.smithi100.stdout:op 634 completed, throughput=5.01MB/sec 2024-04-07T16:40:42.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:40:42 smithi144 bash[66654]: cluster 2024-04-07T16:40:41.144985+0000 mgr.y (mgr.25175) 760 : cluster [DBG] pgmap v1402: 105 pgs: 105 active+clean; 126 GiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 4 op/s 2024-04-07T16:40:42.607 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:40:42 smithi144 bash[95233]: ts=2024-04-07T16:40:42.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:40:42.692 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:40:42 smithi100 bash[74942]: cluster 2024-04-07T16:40:41.144985+0000 mgr.y (mgr.25175) 760 : cluster [DBG] pgmap v1402: 105 pgs: 105 active+clean; 126 GiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 4 op/s 2024-04-07T16:40:42.692 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:40:42 smithi100 bash[76202]: cluster 2024-04-07T16:40:41.144985+0000 mgr.y (mgr.25175) 760 : cluster [DBG] pgmap v1402: 105 pgs: 105 active+clean; 126 GiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 4 op/s 2024-04-07T16:40:43.525 INFO:tasks.workunit.client.0.smithi100.stdout: 123: throughput=4.97MB/sec pending data=0 2024-04-07T16:40:43.525 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-ZuonLwMozEIbTAl off=2429115 len=809898 2024-04-07T16:40:43.534 INFO:tasks.workunit.client.0.smithi100.stdout:op 635 completed, throughput=4.97MB/sec 2024-04-07T16:40:43.534 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-vkcVJtjeklMsCQm off=14906733 len=2053594 2024-04-07T16:40:43.550 INFO:tasks.workunit.client.0.smithi100.stdout:op 636 completed, throughput=4.99MB/sec 2024-04-07T16:40:43.550 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-q8GvI4LJ5jRFbFL off=2095084 len=46122 2024-04-07T16:40:43.555 INFO:tasks.workunit.client.0.smithi100.stdout:op 637 completed, throughput=4.99MB/sec 2024-04-07T16:40:43.555 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-dLScp00Wmtbx7eI off=9289011 len=231226 2024-04-07T16:40:43.557 INFO:tasks.workunit.client.0.smithi100.stdout:op 638 completed, throughput=4.99MB/sec 2024-04-07T16:40:43.557 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-mb39QNIEMyO9nF1 off=0 len=1308469 2024-04-07T16:40:43.559 INFO:tasks.workunit.client.0.smithi100.stdout:op 639 completed, throughput=5MB/sec 2024-04-07T16:40:44.560 INFO:tasks.workunit.client.0.smithi100.stdout: 124: throughput=4.96MB/sec pending data=0 2024-04-07T16:40:44.560 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-NrT03CD60tX7SLJ off=2412352 len=852407 2024-04-07T16:40:44.568 INFO:tasks.workunit.client.0.smithi100.stdout:op 640 completed, throughput=4.97MB/sec 2024-04-07T16:40:44.568 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-px1FmXlXJ0ZA30S off=837595 len=1268659 2024-04-07T16:40:44.579 INFO:tasks.workunit.client.0.smithi100.stdout:op 641 completed, throughput=4.98MB/sec 2024-04-07T16:40:44.579 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-uhHadAkxUrtczpJ off=3961211 len=393448 2024-04-07T16:40:44.586 INFO:tasks.workunit.client.0.smithi100.stdout:op 642 completed, throughput=4.98MB/sec 2024-04-07T16:40:44.586 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Ikc5F0qvupLdX6g off=1752296 len=1527229 2024-04-07T16:40:44.597 INFO:tasks.workunit.client.0.smithi100.stdout:op 643 completed, throughput=4.99MB/sec 2024-04-07T16:40:44.598 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-vYnQvNDwa5hKAFq off=0 len=1355225 2024-04-07T16:40:44.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:40:44 smithi144 bash[66654]: cluster 2024-04-07T16:40:43.146095+0000 mgr.y (mgr.25175) 761 : cluster [DBG] pgmap v1403: 105 pgs: 105 active+clean; 126 GiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 2.8 MiB/s rd, 3.1 MiB/s wr, 5 op/s 2024-04-07T16:40:44.609 INFO:tasks.workunit.client.0.smithi100.stdout:op 644 completed, throughput=5MB/sec 2024-04-07T16:40:44.692 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:40:44 smithi100 bash[74942]: cluster 2024-04-07T16:40:43.146095+0000 mgr.y (mgr.25175) 761 : cluster [DBG] pgmap v1403: 105 pgs: 105 active+clean; 126 GiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 2.8 MiB/s rd, 3.1 MiB/s wr, 5 op/s 2024-04-07T16:40:44.693 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:40:44 smithi100 bash[76202]: cluster 2024-04-07T16:40:43.146095+0000 mgr.y (mgr.25175) 761 : cluster [DBG] pgmap v1403: 105 pgs: 105 active+clean; 126 GiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 2.8 MiB/s rd, 3.1 MiB/s wr, 5 op/s 2024-04-07T16:40:45.609 INFO:tasks.workunit.client.0.smithi100.stdout: 125: throughput=4.96MB/sec pending data=0 2024-04-07T16:40:45.609 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-IX1nAIWuuN2Po1j off=7348217 len=1930146 2024-04-07T16:40:45.614 INFO:tasks.workunit.client.0.smithi100.stdout:op 645 completed, throughput=4.98MB/sec 2024-04-07T16:40:45.614 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-0o_5yyyi22LxLj- off=0 len=1089546 2024-04-07T16:40:45.626 INFO:tasks.workunit.client.0.smithi100.stdout:op 646 completed, throughput=4.98MB/sec 2024-04-07T16:40:45.626 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-ZuonLwMozEIbTAl off=4411290 len=631389 2024-04-07T16:40:45.631 INFO:tasks.workunit.client.0.smithi100.stdout:op 647 completed, throughput=4.99MB/sec 2024-04-07T16:40:45.631 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-S-xZtfo_15OWTWY off=5542317 len=1264071 2024-04-07T16:40:45.641 INFO:tasks.workunit.client.0.smithi100.stdout:op 648 completed, throughput=5MB/sec 2024-04-07T16:40:45.641 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-HsK8sKK3yUj2vCE off=8935759 len=2085162 2024-04-07T16:40:45.656 INFO:tasks.workunit.client.0.smithi100.stdout:op 649 completed, throughput=5.01MB/sec 2024-04-07T16:40:46.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:40:46 smithi144 bash[66654]: cluster 2024-04-07T16:40:45.147000+0000 mgr.y (mgr.25175) 762 : cluster [DBG] pgmap v1404: 105 pgs: 105 active+clean; 126 GiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 2.2 MiB/s rd, 3.0 MiB/s wr, 5 op/s 2024-04-07T16:40:46.656 INFO:tasks.workunit.client.0.smithi100.stdout: 126: throughput=4.97MB/sec pending data=0 2024-04-07T16:40:46.656 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-OfHsO4kYKoog_UN off=19492513 len=1113908 2024-04-07T16:40:46.665 INFO:tasks.workunit.client.0.smithi100.stdout:op 650 completed, throughput=4.98MB/sec 2024-04-07T16:40:46.665 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-XW0mbUKO1FpXyoK off=10643447 len=493479 2024-04-07T16:40:46.667 INFO:tasks.workunit.client.0.smithi100.stdout:op 651 completed, throughput=4.98MB/sec 2024-04-07T16:40:46.667 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-E8wrgLz4sViJ0_E off=20760891 len=1738542 2024-04-07T16:40:46.671 INFO:tasks.workunit.client.0.smithi100.stdout:op 652 completed, throughput=5MB/sec 2024-04-07T16:40:46.671 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-cyF7BXaelRwCn0R off=1026137 len=1455093 2024-04-07T16:40:46.683 INFO:tasks.workunit.client.0.smithi100.stdout:op 653 completed, throughput=5.01MB/sec 2024-04-07T16:40:46.692 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:40:46 smithi100 bash[74942]: cluster 2024-04-07T16:40:45.147000+0000 mgr.y (mgr.25175) 762 : cluster [DBG] pgmap v1404: 105 pgs: 105 active+clean; 126 GiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 2.2 MiB/s rd, 3.0 MiB/s wr, 5 op/s 2024-04-07T16:40:46.692 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:40:46 smithi100 bash[76202]: cluster 2024-04-07T16:40:45.147000+0000 mgr.y (mgr.25175) 762 : cluster [DBG] pgmap v1404: 105 pgs: 105 active+clean; 126 GiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 2.2 MiB/s rd, 3.0 MiB/s wr, 5 op/s 2024-04-07T16:40:47.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:40:47 smithi144 bash[66654]: audit 2024-04-07T16:40:46.758050+0000 mon.a (mon.0) 2060 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:40:47.684 INFO:tasks.workunit.client.0.smithi100.stdout: 127: throughput=4.97MB/sec pending data=0 2024-04-07T16:40:47.684 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-9jrqpzP3W_GWDo- off=1175695 len=1632854 2024-04-07T16:40:47.692 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:40:47 smithi100 bash[74942]: audit 2024-04-07T16:40:46.758050+0000 mon.a (mon.0) 2060 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:40:47.692 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:40:47 smithi100 bash[76202]: audit 2024-04-07T16:40:46.758050+0000 mon.a (mon.0) 2060 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:40:47.695 INFO:tasks.workunit.client.0.smithi100.stdout:op 654 completed, throughput=4.98MB/sec 2024-04-07T16:40:47.696 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-3qCvGRftgrQhNMY off=185937 len=607085 2024-04-07T16:40:47.698 INFO:tasks.workunit.client.0.smithi100.stdout:op 655 completed, throughput=4.98MB/sec 2024-04-07T16:40:47.698 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Mltz6fnwrk2JANE off=3673862 len=726883 2024-04-07T16:40:47.700 INFO:tasks.workunit.client.0.smithi100.stdout:op 656 completed, throughput=4.99MB/sec 2024-04-07T16:40:47.700 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-EaAgwQ-oTDorU_Z off=884979 len=263078 2024-04-07T16:40:47.701 INFO:tasks.workunit.client.0.smithi100.stdout:op 657 completed, throughput=4.99MB/sec 2024-04-07T16:40:47.701 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-EY5G11Bq_wzKsOB off=15458884 len=1422201 2024-04-07T16:40:47.713 INFO:tasks.workunit.client.0.smithi100.stdout:op 658 completed, throughput=5MB/sec 2024-04-07T16:40:48.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:40:48 smithi144 bash[66654]: cluster 2024-04-07T16:40:47.148033+0000 mgr.y (mgr.25175) 763 : cluster [DBG] pgmap v1405: 105 pgs: 105 active+clean; 126 GiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 2.1 MiB/s rd, 3.7 MiB/s wr, 5 op/s 2024-04-07T16:40:48.692 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:40:48 smithi100 bash[74942]: cluster 2024-04-07T16:40:47.148033+0000 mgr.y (mgr.25175) 763 : cluster [DBG] pgmap v1405: 105 pgs: 105 active+clean; 126 GiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 2.1 MiB/s rd, 3.7 MiB/s wr, 5 op/s 2024-04-07T16:40:48.693 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:40:48 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:40:48] "GET /metrics HTTP/1.1" 200 34055 "" "Prometheus/2.43.0" 2024-04-07T16:40:48.693 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:40:48 smithi100 bash[76202]: cluster 2024-04-07T16:40:47.148033+0000 mgr.y (mgr.25175) 763 : cluster [DBG] pgmap v1405: 105 pgs: 105 active+clean; 126 GiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 2.1 MiB/s rd, 3.7 MiB/s wr, 5 op/s 2024-04-07T16:40:48.713 INFO:tasks.workunit.client.0.smithi100.stdout: 128: throughput=4.96MB/sec pending data=0 2024-04-07T16:40:48.713 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-W0EsZS6XYsy6NhQ off=8326954 len=644814 2024-04-07T16:40:48.716 INFO:tasks.workunit.client.0.smithi100.stdout:op 659 completed, throughput=4.97MB/sec 2024-04-07T16:40:48.716 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-nWeNhkF7RQOAs0P off=6759326 len=1191214 2024-04-07T16:40:48.726 INFO:tasks.workunit.client.0.smithi100.stdout:op 660 completed, throughput=4.98MB/sec 2024-04-07T16:40:48.726 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-SBIFf1yc1fshROo off=6717023 len=1323775 2024-04-07T16:40:48.730 INFO:tasks.workunit.client.0.smithi100.stdout:op 661 completed, throughput=4.99MB/sec 2024-04-07T16:40:48.730 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-ZzqW8KY2NmPg9xR off=16451082 len=1099160 2024-04-07T16:40:48.739 INFO:tasks.workunit.client.0.smithi100.stdout:op 662 completed, throughput=4.99MB/sec 2024-04-07T16:40:48.739 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-mx0IkEXWi43ngzA off=1304555 len=1081498 2024-04-07T16:40:48.749 INFO:tasks.workunit.client.0.smithi100.stdout:op 663 completed, throughput=5MB/sec 2024-04-07T16:40:49.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:40:49 smithi144 bash[66654]: audit 2024-04-07T16:40:49.136129+0000 mon.a (mon.0) 2061 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:40:49.692 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:40:49 smithi100 bash[76202]: audit 2024-04-07T16:40:49.136129+0000 mon.a (mon.0) 2061 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:40:49.692 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:40:49 smithi100 bash[74942]: audit 2024-04-07T16:40:49.136129+0000 mon.a (mon.0) 2061 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:40:49.749 INFO:tasks.workunit.client.0.smithi100.stdout: 129: throughput=4.96MB/sec pending data=0 2024-04-07T16:40:49.749 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-986x-pEw70bQj52 off=6641656 len=1981848 2024-04-07T16:40:49.753 INFO:tasks.workunit.client.0.smithi100.stdout:op 664 completed, throughput=4.98MB/sec 2024-04-07T16:40:49.753 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-MrWjj8g5py-ez_5 off=10353094 len=319150 2024-04-07T16:40:49.759 INFO:tasks.workunit.client.0.smithi100.stdout:op 665 completed, throughput=4.98MB/sec 2024-04-07T16:40:49.759 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-c4oY100jDEgGjqS off=1731909 len=1427380 2024-04-07T16:40:49.770 INFO:tasks.workunit.client.0.smithi100.stdout:op 666 completed, throughput=4.99MB/sec 2024-04-07T16:40:49.770 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-M8_oO9y-4HUwDRE off=15198394 len=1989197 2024-04-07T16:40:49.774 INFO:tasks.workunit.client.0.smithi100.stdout:op 667 completed, throughput=5MB/sec 2024-04-07T16:40:50.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:40:50 smithi144 bash[66654]: cluster 2024-04-07T16:40:49.148805+0000 mgr.y (mgr.25175) 764 : cluster [DBG] pgmap v1406: 105 pgs: 105 active+clean; 126 GiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 1.9 MiB/s rd, 3.3 MiB/s wr, 5 op/s 2024-04-07T16:40:50.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:40:50 smithi144 bash[66654]: audit 2024-04-07T16:40:49.927813+0000 mon.a (mon.0) 2062 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:40:50.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:40:50 smithi144 bash[66654]: audit 2024-04-07T16:40:49.929510+0000 mon.a (mon.0) 2063 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:40:50.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:40:50 smithi144 bash[66654]: audit 2024-04-07T16:40:49.940868+0000 mon.a (mon.0) 2064 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:40:50.692 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:40:50 smithi100 bash[74942]: cluster 2024-04-07T16:40:49.148805+0000 mgr.y (mgr.25175) 764 : cluster [DBG] pgmap v1406: 105 pgs: 105 active+clean; 126 GiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 1.9 MiB/s rd, 3.3 MiB/s wr, 5 op/s 2024-04-07T16:40:50.692 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:40:50 smithi100 bash[74942]: audit 2024-04-07T16:40:49.927813+0000 mon.a (mon.0) 2062 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:40:50.693 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:40:50 smithi100 bash[74942]: audit 2024-04-07T16:40:49.929510+0000 mon.a (mon.0) 2063 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:40:50.693 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:40:50 smithi100 bash[74942]: audit 2024-04-07T16:40:49.940868+0000 mon.a (mon.0) 2064 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:40:50.693 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:40:50 smithi100 bash[76202]: cluster 2024-04-07T16:40:49.148805+0000 mgr.y (mgr.25175) 764 : cluster [DBG] pgmap v1406: 105 pgs: 105 active+clean; 126 GiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 1.9 MiB/s rd, 3.3 MiB/s wr, 5 op/s 2024-04-07T16:40:50.693 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:40:50 smithi100 bash[76202]: audit 2024-04-07T16:40:49.927813+0000 mon.a (mon.0) 2062 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:40:50.693 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:40:50 smithi100 bash[76202]: audit 2024-04-07T16:40:49.929510+0000 mon.a (mon.0) 2063 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:40:50.693 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:40:50 smithi100 bash[76202]: audit 2024-04-07T16:40:49.940868+0000 mon.a (mon.0) 2064 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:40:50.774 INFO:tasks.workunit.client.0.smithi100.stdout: 130: throughput=4.97MB/sec pending data=0 2024-04-07T16:40:50.774 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-9LaRBfRtXpCt2Wb off=2396218 len=609852 2024-04-07T16:40:50.781 INFO:tasks.workunit.client.0.smithi100.stdout:op 668 completed, throughput=4.97MB/sec 2024-04-07T16:40:50.781 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-L1kfKVQIq9Y1m37 off=742925 len=45201 2024-04-07T16:40:50.783 INFO:tasks.workunit.client.0.smithi100.stdout:op 669 completed, throughput=4.97MB/sec 2024-04-07T16:40:50.783 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-u-ayfWCsWuHLX-p off=3816112 len=76170 2024-04-07T16:40:50.787 INFO:tasks.workunit.client.0.smithi100.stdout:op 670 completed, throughput=4.97MB/sec 2024-04-07T16:40:50.787 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-pC_PPHj554xbYCx off=4200266 len=1084250 2024-04-07T16:40:50.790 INFO:tasks.workunit.client.0.smithi100.stdout:op 671 completed, throughput=4.98MB/sec 2024-04-07T16:40:50.790 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-7ise0ZBLJ2Immbi off=8109980 len=1782531 2024-04-07T16:40:50.805 INFO:tasks.workunit.client.0.smithi100.stdout:op 672 completed, throughput=4.99MB/sec 2024-04-07T16:40:50.805 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-ziIjdd86iNhHuwy off=8733363 len=1620583 2024-04-07T16:40:50.809 INFO:tasks.workunit.client.0.smithi100.stdout:op 673 completed, throughput=5MB/sec 2024-04-07T16:40:51.810 INFO:tasks.workunit.client.0.smithi100.stdout: 131: throughput=4.96MB/sec pending data=0 2024-04-07T16:40:51.810 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-sEfjRAjzE0Djoag off=4492989 len=1777647 2024-04-07T16:40:51.814 INFO:tasks.workunit.client.0.smithi100.stdout:op 674 completed, throughput=4.98MB/sec 2024-04-07T16:40:51.814 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-GElaxxtQR_rIUtw off=4304997 len=1807252 2024-04-07T16:40:51.830 INFO:tasks.workunit.client.0.smithi100.stdout:op 675 completed, throughput=4.99MB/sec 2024-04-07T16:40:51.830 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-hldBVmdKgcPczz4 off=7226915 len=657339 2024-04-07T16:40:51.833 INFO:tasks.workunit.client.0.smithi100.stdout:op 676 completed, throughput=4.99MB/sec 2024-04-07T16:40:51.833 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-rrPcErwdtZdLNVT off=369551 len=1135149 2024-04-07T16:40:51.836 INFO:tasks.workunit.client.0.smithi100.stdout:op 677 completed, throughput=5MB/sec 2024-04-07T16:40:52.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:40:52 smithi144 bash[66654]: cluster 2024-04-07T16:40:51.149734+0000 mgr.y (mgr.25175) 765 : cluster [DBG] pgmap v1407: 105 pgs: 105 active+clean; 126 GiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 2.1 MiB/s rd, 3.0 MiB/s wr, 5 op/s 2024-04-07T16:40:52.607 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:40:52 smithi144 bash[95233]: ts=2024-04-07T16:40:52.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:40:52.692 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:40:52 smithi100 bash[74942]: cluster 2024-04-07T16:40:51.149734+0000 mgr.y (mgr.25175) 765 : cluster [DBG] pgmap v1407: 105 pgs: 105 active+clean; 126 GiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 2.1 MiB/s rd, 3.0 MiB/s wr, 5 op/s 2024-04-07T16:40:52.693 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:40:52 smithi100 bash[76202]: cluster 2024-04-07T16:40:51.149734+0000 mgr.y (mgr.25175) 765 : cluster [DBG] pgmap v1407: 105 pgs: 105 active+clean; 126 GiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 2.1 MiB/s rd, 3.0 MiB/s wr, 5 op/s 2024-04-07T16:40:52.836 INFO:tasks.workunit.client.0.smithi100.stdout: 132: throughput=4.97MB/sec pending data=0 2024-04-07T16:40:52.837 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-5ujZLIMuRLgyVxh off=914651 len=784521 2024-04-07T16:40:52.840 INFO:tasks.workunit.client.0.smithi100.stdout:op 678 completed, throughput=4.97MB/sec 2024-04-07T16:40:52.840 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-nLDTc3NCWt-iFgL off=788707 len=1100205 2024-04-07T16:40:52.849 INFO:tasks.workunit.client.0.smithi100.stdout:op 679 completed, throughput=4.98MB/sec 2024-04-07T16:40:52.849 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-VZJ80RGZYtwSQcj off=1219828 len=517351 2024-04-07T16:40:52.855 INFO:tasks.workunit.client.0.smithi100.stdout:op 680 completed, throughput=4.98MB/sec 2024-04-07T16:40:52.855 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-LboZM71fG2tGbd1 off=4099411 len=1757590 2024-04-07T16:40:52.870 INFO:tasks.workunit.client.0.smithi100.stdout:op 681 completed, throughput=4.99MB/sec 2024-04-07T16:40:52.870 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-lkbSnX-f0fUasvh off=3447287 len=31621 2024-04-07T16:40:52.871 INFO:tasks.workunit.client.0.smithi100.stdout:op 682 completed, throughput=4.99MB/sec 2024-04-07T16:40:52.871 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-32IKDRbNvcRr3CD off=6696424 len=190368 2024-04-07T16:40:52.876 INFO:tasks.workunit.client.0.smithi100.stdout:op 683 completed, throughput=4.99MB/sec 2024-04-07T16:40:52.876 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-FaVuaGZ4jGe7eig off=5837103 len=1516617 2024-04-07T16:40:52.881 INFO:tasks.workunit.client.0.smithi100.stdout:op 684 completed, throughput=5MB/sec 2024-04-07T16:40:53.882 INFO:tasks.workunit.client.0.smithi100.stdout: 133: throughput=4.97MB/sec pending data=0 2024-04-07T16:40:53.882 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-a-z1kz5UzrnJgGF off=18304929 len=129411 2024-04-07T16:40:53.883 INFO:tasks.workunit.client.0.smithi100.stdout:op 685 completed, throughput=4.97MB/sec 2024-04-07T16:40:53.883 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-43jWURAgQ4nuRMH off=395905 len=1522970 2024-04-07T16:40:53.888 INFO:tasks.workunit.client.0.smithi100.stdout:op 686 completed, throughput=4.98MB/sec 2024-04-07T16:40:53.888 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-O77jgCgtl5nKYA4 off=15546928 len=702130 2024-04-07T16:40:53.896 INFO:tasks.workunit.client.0.smithi100.stdout:op 687 completed, throughput=4.98MB/sec 2024-04-07T16:40:53.896 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-McrQdvNzxRmWfti off=4649955 len=1478431 2024-04-07T16:40:53.907 INFO:tasks.workunit.client.0.smithi100.stdout:op 688 completed, throughput=4.99MB/sec 2024-04-07T16:40:53.907 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-mb39QNIEMyO9nF1 off=0 len=1308469 2024-04-07T16:40:53.918 INFO:tasks.workunit.client.0.smithi100.stdout:op 689 completed, throughput=5MB/sec 2024-04-07T16:40:54.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:40:54 smithi144 bash[66654]: cluster 2024-04-07T16:40:53.150934+0000 mgr.y (mgr.25175) 766 : cluster [DBG] pgmap v1408: 105 pgs: 105 active+clean; 126 GiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 2.4 MiB/s rd, 3.7 MiB/s wr, 6 op/s 2024-04-07T16:40:54.694 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:40:54 smithi100 bash[74942]: cluster 2024-04-07T16:40:53.150934+0000 mgr.y (mgr.25175) 766 : cluster [DBG] pgmap v1408: 105 pgs: 105 active+clean; 126 GiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 2.4 MiB/s rd, 3.7 MiB/s wr, 6 op/s 2024-04-07T16:40:54.695 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:40:54 smithi100 bash[76202]: cluster 2024-04-07T16:40:53.150934+0000 mgr.y (mgr.25175) 766 : cluster [DBG] pgmap v1408: 105 pgs: 105 active+clean; 126 GiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 2.4 MiB/s rd, 3.7 MiB/s wr, 6 op/s 2024-04-07T16:40:54.918 INFO:tasks.workunit.client.0.smithi100.stdout: 134: throughput=4.97MB/sec pending data=0 2024-04-07T16:40:54.918 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-jXZ4YVhqzwwxUy1 off=393766 len=1874203 2024-04-07T16:40:54.923 INFO:tasks.workunit.client.0.smithi100.stdout:op 690 completed, throughput=4.98MB/sec 2024-04-07T16:40:54.923 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-TuFlldoIz5CEBse off=3862407 len=571962 2024-04-07T16:40:54.930 INFO:tasks.workunit.client.0.smithi100.stdout:op 691 completed, throughput=4.98MB/sec 2024-04-07T16:40:54.930 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-EDZxK7UM30GUFyy off=5571315 len=1724056 2024-04-07T16:40:54.944 INFO:tasks.workunit.client.0.smithi100.stdout:op 692 completed, throughput=4.99MB/sec 2024-04-07T16:40:54.944 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-W6u9i8K9IG6j778 off=5838452 len=1123211 2024-04-07T16:40:54.955 INFO:tasks.workunit.client.0.smithi100.stdout:op 693 completed, throughput=5MB/sec 2024-04-07T16:40:55.953 INFO:tasks.workunit.client.0.smithi100.stdout: 135: throughput=4.97MB/sec pending data=0 2024-04-07T16:40:55.953 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-jyBPoEwVcLfeQqz off=2097035 len=1361296 2024-04-07T16:40:55.966 INFO:tasks.workunit.client.0.smithi100.stdout:op 694 completed, throughput=4.97MB/sec 2024-04-07T16:40:55.966 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-ADd3Mafky4kEcOv off=4310490 len=437205 2024-04-07T16:40:55.968 INFO:tasks.workunit.client.0.smithi100.stdout:op 695 completed, throughput=4.98MB/sec 2024-04-07T16:40:55.968 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-rDDR8q6aj01mF2S off=12685344 len=1364803 2024-04-07T16:40:55.980 INFO:tasks.workunit.client.0.smithi100.stdout:op 696 completed, throughput=4.99MB/sec 2024-04-07T16:40:55.980 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-gUjAxBndbjss1jt off=12589301 len=1479115 2024-04-07T16:40:55.984 INFO:tasks.workunit.client.0.smithi100.stdout:op 697 completed, throughput=5MB/sec 2024-04-07T16:40:55.984 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-7grBMpGyFMbmcpX off=14177398 len=665417 2024-04-07T16:40:55.986 INFO:tasks.workunit.client.0.smithi100.stdout:op 698 completed, throughput=5MB/sec 2024-04-07T16:40:56.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:40:56 smithi144 bash[66654]: cluster 2024-04-07T16:40:55.151982+0000 mgr.y (mgr.25175) 767 : cluster [DBG] pgmap v1409: 105 pgs: 105 active+clean; 126 GiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 1.8 MiB/s rd, 2.9 MiB/s wr, 4 op/s 2024-04-07T16:40:56.692 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:40:56 smithi100 bash[74942]: cluster 2024-04-07T16:40:55.151982+0000 mgr.y (mgr.25175) 767 : cluster [DBG] pgmap v1409: 105 pgs: 105 active+clean; 126 GiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 1.8 MiB/s rd, 2.9 MiB/s wr, 4 op/s 2024-04-07T16:40:56.693 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:40:56 smithi100 bash[76202]: cluster 2024-04-07T16:40:55.151982+0000 mgr.y (mgr.25175) 767 : cluster [DBG] pgmap v1409: 105 pgs: 105 active+clean; 126 GiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 1.8 MiB/s rd, 2.9 MiB/s wr, 4 op/s 2024-04-07T16:40:56.987 INFO:tasks.workunit.client.0.smithi100.stdout: 136: throughput=4.97MB/sec pending data=0 2024-04-07T16:40:56.987 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-4-TAchmPztNbqT_ off=1264757 len=812438 2024-04-07T16:40:56.990 INFO:tasks.workunit.client.0.smithi100.stdout:op 699 completed, throughput=4.97MB/sec 2024-04-07T16:40:56.990 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-TfpREWaYFqkN2aG off=12747898 len=1771069 2024-04-07T16:40:56.996 INFO:tasks.workunit.client.0.smithi100.stdout:op 700 completed, throughput=4.98MB/sec 2024-04-07T16:40:56.996 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-fklpTDu3WoAcKSZ off=4484790 len=85532 2024-04-07T16:40:57.000 INFO:tasks.workunit.client.0.smithi100.stdout:op 701 completed, throughput=4.98MB/sec 2024-04-07T16:40:57.001 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-rCTh8JZNPaGaH36 off=5484801 len=879249 2024-04-07T16:40:57.004 INFO:tasks.workunit.client.0.smithi100.stdout:op 702 completed, throughput=4.99MB/sec 2024-04-07T16:40:57.004 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-uXt2vHxcbcuAlX2 off=19009888 len=192043 2024-04-07T16:40:57.005 INFO:tasks.workunit.client.0.smithi100.stdout:op 703 completed, throughput=4.99MB/sec 2024-04-07T16:40:57.005 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-odsilPeYYAqVs_x off=5451674 len=974473 2024-04-07T16:40:57.008 INFO:tasks.workunit.client.0.smithi100.stdout:op 704 completed, throughput=5MB/sec 2024-04-07T16:40:57.008 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-bsWAHwf3futkGme off=13806538 len=1161408 2024-04-07T16:40:57.018 INFO:tasks.workunit.client.0.smithi100.stdout:op 705 completed, throughput=5MB/sec 2024-04-07T16:40:58.018 INFO:tasks.workunit.client.0.smithi100.stdout: 137: throughput=4.97MB/sec pending data=0 2024-04-07T16:40:58.019 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-OX3UqcjWcXEt21N off=11603496 len=1097396 2024-04-07T16:40:58.028 INFO:tasks.workunit.client.0.smithi100.stdout:op 706 completed, throughput=4.98MB/sec 2024-04-07T16:40:58.028 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj--ZvIylmKNEGaytQ off=1887487 len=2033565 2024-04-07T16:40:58.035 INFO:tasks.workunit.client.0.smithi100.stdout:op 707 completed, throughput=4.99MB/sec 2024-04-07T16:40:58.035 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-eP3tWtq6odirrFR off=1324090 len=402078 2024-04-07T16:40:58.037 INFO:tasks.workunit.client.0.smithi100.stdout:op 708 completed, throughput=4.99MB/sec 2024-04-07T16:40:58.037 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-kWk0auv9YmtsdSL off=5499286 len=1079770 2024-04-07T16:40:58.046 INFO:tasks.workunit.client.0.smithi100.stdout:op 709 completed, throughput=5MB/sec 2024-04-07T16:40:58.046 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-IUPd00xnzXc5Wmr off=1820311 len=714642 2024-04-07T16:40:58.053 INFO:tasks.workunit.client.0.smithi100.stdout:op 710 completed, throughput=5MB/sec 2024-04-07T16:40:58.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:40:58 smithi144 bash[66654]: cluster 2024-04-07T16:40:57.153091+0000 mgr.y (mgr.25175) 768 : cluster [DBG] pgmap v1410: 105 pgs: 105 active+clean; 126 GiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 1.9 MiB/s rd, 2.7 MiB/s wr, 4 op/s 2024-04-07T16:40:58.692 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:40:58 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:40:58] "GET /metrics HTTP/1.1" 200 34055 "" "Prometheus/2.43.0" 2024-04-07T16:40:58.693 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:40:58 smithi100 bash[74942]: cluster 2024-04-07T16:40:57.153091+0000 mgr.y (mgr.25175) 768 : cluster [DBG] pgmap v1410: 105 pgs: 105 active+clean; 126 GiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 1.9 MiB/s rd, 2.7 MiB/s wr, 4 op/s 2024-04-07T16:40:58.693 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:40:58 smithi100 bash[76202]: cluster 2024-04-07T16:40:57.153091+0000 mgr.y (mgr.25175) 768 : cluster [DBG] pgmap v1410: 105 pgs: 105 active+clean; 126 GiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 1.9 MiB/s rd, 2.7 MiB/s wr, 4 op/s 2024-04-07T16:40:59.053 INFO:tasks.workunit.client.0.smithi100.stdout: 138: throughput=4.97MB/sec pending data=0 2024-04-07T16:40:59.053 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-WHGQNd0HDykmAdF off=412212 len=1216621 2024-04-07T16:40:59.063 INFO:tasks.workunit.client.0.smithi100.stdout:op 711 completed, throughput=4.98MB/sec 2024-04-07T16:40:59.064 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-F9-0_Wz9A5o-UwM off=8678956 len=1308512 2024-04-07T16:40:59.066 INFO:tasks.workunit.client.0.smithi100.stdout:op 712 completed, throughput=4.98MB/sec 2024-04-07T16:40:59.066 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-39bv69_P5IGXuv6 off=176576 len=1518335 2024-04-07T16:40:59.078 INFO:tasks.workunit.client.0.smithi100.stdout:op 713 completed, throughput=4.99MB/sec 2024-04-07T16:40:59.079 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-9rOtSFJPbGl7-aF off=7857402 len=1681968 2024-04-07T16:40:59.084 INFO:tasks.workunit.client.0.smithi100.stdout:op 714 completed, throughput=5.01MB/sec 2024-04-07T16:41:00.084 INFO:tasks.workunit.client.0.smithi100.stdout: 139: throughput=4.97MB/sec pending data=0 2024-04-07T16:41:00.084 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-YMemfuf2oy9_Yhx off=8828329 len=92223 2024-04-07T16:41:00.085 INFO:tasks.workunit.client.0.smithi100.stdout:op 715 completed, throughput=4.97MB/sec 2024-04-07T16:41:00.085 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-d-zhX87Nt8fsq2l off=306395 len=570401 2024-04-07T16:41:00.092 INFO:tasks.workunit.client.0.smithi100.stdout:op 716 completed, throughput=4.97MB/sec 2024-04-07T16:41:00.092 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-OJ3zf0snZ7JAjzv off=9324491 len=2020748 2024-04-07T16:41:00.096 INFO:tasks.workunit.client.0.smithi100.stdout:op 717 completed, throughput=4.99MB/sec 2024-04-07T16:41:00.096 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-iEs5dkIlg81HZWh off=4339210 len=396972 2024-04-07T16:41:00.102 INFO:tasks.workunit.client.0.smithi100.stdout:op 718 completed, throughput=4.99MB/sec 2024-04-07T16:41:00.102 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-dE0R7nfR81eu9Gm off=9734537 len=804777 2024-04-07T16:41:00.104 INFO:tasks.workunit.client.0.smithi100.stdout:op 719 completed, throughput=5MB/sec 2024-04-07T16:41:00.104 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-82Sz_mWAocFaCAB off=3161621 len=815598 2024-04-07T16:41:00.112 INFO:tasks.workunit.client.0.smithi100.stdout:op 720 completed, throughput=5MB/sec 2024-04-07T16:41:00.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:41:00 smithi144 bash[66654]: cluster 2024-04-07T16:40:59.154061+0000 mgr.y (mgr.25175) 769 : cluster [DBG] pgmap v1411: 105 pgs: 105 active+clean; 126 GiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 2.0 MiB/s rd, 2.2 MiB/s wr, 4 op/s 2024-04-07T16:41:00.692 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:41:00 smithi100 bash[74942]: cluster 2024-04-07T16:40:59.154061+0000 mgr.y (mgr.25175) 769 : cluster [DBG] pgmap v1411: 105 pgs: 105 active+clean; 126 GiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 2.0 MiB/s rd, 2.2 MiB/s wr, 4 op/s 2024-04-07T16:41:00.693 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:41:00 smithi100 bash[76202]: cluster 2024-04-07T16:40:59.154061+0000 mgr.y (mgr.25175) 769 : cluster [DBG] pgmap v1411: 105 pgs: 105 active+clean; 126 GiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 2.0 MiB/s rd, 2.2 MiB/s wr, 4 op/s 2024-04-07T16:41:01.112 INFO:tasks.workunit.client.0.smithi100.stdout: 140: throughput=4.97MB/sec pending data=0 2024-04-07T16:41:01.112 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-N3ZXKIg7AbcfxJj off=190528 len=429277 2024-04-07T16:41:01.118 INFO:tasks.workunit.client.0.smithi100.stdout:op 721 completed, throughput=4.97MB/sec 2024-04-07T16:41:01.119 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-7wzyKs9PeeAF0FG off=182141 len=8274 2024-04-07T16:41:01.119 INFO:tasks.workunit.client.0.smithi100.stdout:op 722 completed, throughput=4.97MB/sec 2024-04-07T16:41:01.119 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-75ENXRF8W6LwVQp off=12117851 len=1534032 2024-04-07T16:41:01.132 INFO:tasks.workunit.client.0.smithi100.stdout:op 723 completed, throughput=4.98MB/sec 2024-04-07T16:41:01.132 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-JKTNFQx6-EXska4 off=479290 len=470968 2024-04-07T16:41:01.134 INFO:tasks.workunit.client.0.smithi100.stdout:op 724 completed, throughput=4.98MB/sec 2024-04-07T16:41:01.134 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-wBzVE6_rbmzl_wK off=1818364 len=805729 2024-04-07T16:41:01.136 INFO:tasks.workunit.client.0.smithi100.stdout:op 725 completed, throughput=4.99MB/sec 2024-04-07T16:41:01.136 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-sSuZGfVhkLwupsJ off=13856195 len=87609 2024-04-07T16:41:01.137 INFO:tasks.workunit.client.0.smithi100.stdout:op 726 completed, throughput=4.99MB/sec 2024-04-07T16:41:01.137 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Z2KOyyVBqyQK5ua off=9897203 len=2026021 2024-04-07T16:41:01.155 INFO:tasks.workunit.client.0.smithi100.stdout:op 727 completed, throughput=5MB/sec 2024-04-07T16:41:02.155 INFO:tasks.workunit.client.0.smithi100.stdout: 141: throughput=4.97MB/sec pending data=0 2024-04-07T16:41:02.156 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-mVjCsMUrbbOYU8L off=9571033 len=836367 2024-04-07T16:41:02.158 INFO:tasks.workunit.client.0.smithi100.stdout:op 728 completed, throughput=4.97MB/sec 2024-04-07T16:41:02.158 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj--pMWN_6B0dEuVQe off=14243457 len=600781 2024-04-07T16:41:02.160 INFO:tasks.workunit.client.0.smithi100.stdout:op 729 completed, throughput=4.98MB/sec 2024-04-07T16:41:02.160 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-JMdkPj5lXshmxB6 off=396322 len=1530570 2024-04-07T16:41:02.164 INFO:tasks.workunit.client.0.smithi100.stdout:op 730 completed, throughput=4.99MB/sec 2024-04-07T16:41:02.164 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-NxEuc9fSyL5fpj7 off=1063592 len=1662106 2024-04-07T16:41:02.180 INFO:tasks.workunit.client.0.smithi100.stdout:op 731 completed, throughput=5MB/sec 2024-04-07T16:41:02.180 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-04ybS_7irkc-S_U off=7086536 len=135211 2024-04-07T16:41:02.181 INFO:tasks.workunit.client.0.smithi100.stdout:op 732 completed, throughput=5MB/sec 2024-04-07T16:41:02.181 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-g1tOmYSvZvo_gLE off=20750131 len=280774 2024-04-07T16:41:02.185 INFO:tasks.workunit.client.0.smithi100.stdout:op 733 completed, throughput=5MB/sec 2024-04-07T16:41:02.185 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-m6r9mg4xqztB1Yx off=7107093 len=1502411 2024-04-07T16:41:02.193 INFO:tasks.workunit.client.0.smithi100.stdout:op 734 completed, throughput=5.01MB/sec 2024-04-07T16:41:02.607 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:41:02 smithi144 bash[95233]: ts=2024-04-07T16:41:02.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:41:02.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:41:02 smithi144 bash[66654]: cluster 2024-04-07T16:41:01.154976+0000 mgr.y (mgr.25175) 770 : cluster [DBG] pgmap v1412: 105 pgs: 105 active+clean; 126 GiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 2.0 MiB/s rd, 2.2 MiB/s wr, 4 op/s 2024-04-07T16:41:02.608 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:41:02 smithi144 bash[66654]: audit 2024-04-07T16:41:01.758423+0000 mon.a (mon.0) 2065 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:41:02.692 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:41:02 smithi100 bash[74942]: cluster 2024-04-07T16:41:01.154976+0000 mgr.y (mgr.25175) 770 : cluster [DBG] pgmap v1412: 105 pgs: 105 active+clean; 126 GiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 2.0 MiB/s rd, 2.2 MiB/s wr, 4 op/s 2024-04-07T16:41:02.692 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:41:02 smithi100 bash[74942]: audit 2024-04-07T16:41:01.758423+0000 mon.a (mon.0) 2065 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:41:02.693 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:41:02 smithi100 bash[76202]: cluster 2024-04-07T16:41:01.154976+0000 mgr.y (mgr.25175) 770 : cluster [DBG] pgmap v1412: 105 pgs: 105 active+clean; 126 GiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 2.0 MiB/s rd, 2.2 MiB/s wr, 4 op/s 2024-04-07T16:41:02.693 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:41:02 smithi100 bash[76202]: audit 2024-04-07T16:41:01.758423+0000 mon.a (mon.0) 2065 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:41:03.194 INFO:tasks.workunit.client.0.smithi100.stdout: 142: throughput=4.97MB/sec pending data=0 2024-04-07T16:41:03.194 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-0iAka496GfV-q0Q off=21086426 len=832161 2024-04-07T16:41:03.203 INFO:tasks.workunit.client.0.smithi100.stdout:op 735 completed, throughput=4.98MB/sec 2024-04-07T16:41:03.203 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-4i-VtQDvM1ffj6D off=4583825 len=491512 2024-04-07T16:41:03.209 INFO:tasks.workunit.client.0.smithi100.stdout:op 736 completed, throughput=4.98MB/sec 2024-04-07T16:41:03.209 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj--5MNbk23Z8oBHzD off=12070061 len=1224785 2024-04-07T16:41:03.212 INFO:tasks.workunit.client.0.smithi100.stdout:op 737 completed, throughput=4.99MB/sec 2024-04-07T16:41:03.213 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-3UCjHQjJ6w17wmw off=7968027 len=545482 2024-04-07T16:41:03.215 INFO:tasks.workunit.client.0.smithi100.stdout:op 738 completed, throughput=4.99MB/sec 2024-04-07T16:41:03.215 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-qSEg3d1XyMgmTcX off=6137348 len=1041010 2024-04-07T16:41:03.225 INFO:tasks.workunit.client.0.smithi100.stdout:op 739 completed, throughput=5MB/sec 2024-04-07T16:41:03.225 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-szF0I4Em29rfQmT off=4322170 len=1268203 2024-04-07T16:41:03.235 INFO:tasks.workunit.client.0.smithi100.stdout:op 740 completed, throughput=5.01MB/sec 2024-04-07T16:41:04.235 INFO:tasks.workunit.client.0.smithi100.stdout: 143: throughput=4.97MB/sec pending data=0 2024-04-07T16:41:04.235 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-EVVQijGzxc6_PQ3 off=15761659 len=2029260 2024-04-07T16:41:04.251 INFO:tasks.workunit.client.0.smithi100.stdout:op 741 completed, throughput=4.99MB/sec 2024-04-07T16:41:04.251 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Nn1-IK3RLN3rkRu off=4438215 len=1712471 2024-04-07T16:41:04.255 INFO:tasks.workunit.client.0.smithi100.stdout:op 742 completed, throughput=5MB/sec 2024-04-07T16:41:04.255 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-moRUHG9XQTGxkMU off=8016538 len=105961 2024-04-07T16:41:04.259 INFO:tasks.workunit.client.0.smithi100.stdout:op 743 completed, throughput=5MB/sec 2024-04-07T16:41:04.260 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-MdTQcAXcvQ6xSWC off=2463034 len=828193 2024-04-07T16:41:04.261 INFO:tasks.workunit.client.0.smithi100.stdout:op 744 completed, throughput=5MB/sec 2024-04-07T16:41:04.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:41:04 smithi144 bash[66654]: cluster 2024-04-07T16:41:03.156116+0000 mgr.y (mgr.25175) 771 : cluster [DBG] pgmap v1413: 105 pgs: 105 active+clean; 126 GiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 2.6 MiB/s rd, 2.9 MiB/s wr, 5 op/s 2024-04-07T16:41:04.692 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:41:04 smithi100 bash[74942]: cluster 2024-04-07T16:41:03.156116+0000 mgr.y (mgr.25175) 771 : cluster [DBG] pgmap v1413: 105 pgs: 105 active+clean; 126 GiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 2.6 MiB/s rd, 2.9 MiB/s wr, 5 op/s 2024-04-07T16:41:04.693 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:41:04 smithi100 bash[76202]: cluster 2024-04-07T16:41:03.156116+0000 mgr.y (mgr.25175) 771 : cluster [DBG] pgmap v1413: 105 pgs: 105 active+clean; 126 GiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 2.6 MiB/s rd, 2.9 MiB/s wr, 5 op/s 2024-04-07T16:41:05.262 INFO:tasks.workunit.client.0.smithi100.stdout: 144: throughput=4.97MB/sec pending data=0 2024-04-07T16:41:05.262 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-6L7kiuOaoRTOMIq off=87154 len=1191660 2024-04-07T16:41:05.266 INFO:tasks.workunit.client.0.smithi100.stdout:op 745 completed, throughput=4.98MB/sec 2024-04-07T16:41:05.266 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-OfHsO4kYKoog_UN off=14004981 len=1380653 2024-04-07T16:41:05.278 INFO:tasks.workunit.client.0.smithi100.stdout:op 746 completed, throughput=4.98MB/sec 2024-04-07T16:41:05.278 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-EtzaVYWThWHqi-7 off=2568827 len=1611559 2024-04-07T16:41:05.290 INFO:tasks.workunit.client.0.smithi100.stdout:op 747 completed, throughput=4.99MB/sec 2024-04-07T16:41:05.290 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-WWEzZmr6Os7_q1Z off=8416007 len=1581649 2024-04-07T16:41:05.302 INFO:tasks.workunit.client.0.smithi100.stdout:op 748 completed, throughput=5MB/sec 2024-04-07T16:41:06.303 INFO:tasks.workunit.client.0.smithi100.stdout: 145: throughput=4.97MB/sec pending data=0 2024-04-07T16:41:06.303 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-EqCrNklkF2yeroI off=1638073 len=1138014 2024-04-07T16:41:06.307 INFO:tasks.workunit.client.0.smithi100.stdout:op 749 completed, throughput=4.98MB/sec 2024-04-07T16:41:06.307 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-5ZI1hbgQUUioJ-M off=2852967 len=625781 2024-04-07T16:41:06.309 INFO:tasks.workunit.client.0.smithi100.stdout:op 750 completed, throughput=4.98MB/sec 2024-04-07T16:41:06.310 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-l3ZfXWU-7X-er3m off=8906967 len=569455 2024-04-07T16:41:06.311 INFO:tasks.workunit.client.0.smithi100.stdout:op 751 completed, throughput=4.99MB/sec 2024-04-07T16:41:06.311 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-RlW6gLLXB1UgPZB off=12251951 len=526195 2024-04-07T16:41:06.317 INFO:tasks.workunit.client.0.smithi100.stdout:op 752 completed, throughput=4.99MB/sec 2024-04-07T16:41:06.317 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-CYyDYrnpmExsWa4 off=3273989 len=1985988 2024-04-07T16:41:06.333 INFO:tasks.workunit.client.0.smithi100.stdout:op 753 completed, throughput=5MB/sec 2024-04-07T16:41:06.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:41:06 smithi144 bash[66654]: cluster 2024-04-07T16:41:05.157078+0000 mgr.y (mgr.25175) 772 : cluster [DBG] pgmap v1414: 105 pgs: 105 active+clean; 126 GiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.3 MiB/s wr, 5 op/s 2024-04-07T16:41:06.692 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:41:06 smithi100 bash[74942]: cluster 2024-04-07T16:41:05.157078+0000 mgr.y (mgr.25175) 772 : cluster [DBG] pgmap v1414: 105 pgs: 105 active+clean; 126 GiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.3 MiB/s wr, 5 op/s 2024-04-07T16:41:06.693 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:41:06 smithi100 bash[76202]: cluster 2024-04-07T16:41:05.157078+0000 mgr.y (mgr.25175) 772 : cluster [DBG] pgmap v1414: 105 pgs: 105 active+clean; 126 GiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.3 MiB/s wr, 5 op/s 2024-04-07T16:41:07.334 INFO:tasks.workunit.client.0.smithi100.stdout: 146: throughput=4.97MB/sec pending data=0 2024-04-07T16:41:07.334 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-T902c4i3e-Mnh3Z off=9838087 len=134661 2024-04-07T16:41:07.335 INFO:tasks.workunit.client.0.smithi100.stdout:op 754 completed, throughput=4.97MB/sec 2024-04-07T16:41:07.335 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-rfLycIec_tgXzQm off=11174817 len=1361074 2024-04-07T16:41:07.338 INFO:tasks.workunit.client.0.smithi100.stdout:op 755 completed, throughput=4.98MB/sec 2024-04-07T16:41:07.338 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-sFONtEcWk2uBA4T off=3299946 len=1752815 2024-04-07T16:41:07.342 INFO:tasks.workunit.client.0.smithi100.stdout:op 756 completed, throughput=4.99MB/sec 2024-04-07T16:41:07.342 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-THbfGivgaYawDOL off=7939922 len=186776 2024-04-07T16:41:07.348 INFO:tasks.workunit.client.0.smithi100.stdout:op 757 completed, throughput=4.99MB/sec 2024-04-07T16:41:07.348 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-_v0WRnVdXUaGEcm off=16994085 len=520324 2024-04-07T16:41:07.350 INFO:tasks.workunit.client.0.smithi100.stdout:op 758 completed, throughput=4.99MB/sec 2024-04-07T16:41:07.350 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-mikkkyyaC28rdAS off=0 len=1548892 2024-04-07T16:41:07.362 INFO:tasks.workunit.client.0.smithi100.stdout:op 759 completed, throughput=5MB/sec 2024-04-07T16:41:08.362 INFO:tasks.workunit.client.0.smithi100.stdout: 147: throughput=4.97MB/sec pending data=0 2024-04-07T16:41:08.362 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-TJUMEwLKcZSvl-B off=19297294 len=916993 2024-04-07T16:41:08.366 INFO:tasks.workunit.client.0.smithi100.stdout:op 760 completed, throughput=4.97MB/sec 2024-04-07T16:41:08.366 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-s2aBFXp4dAm1HCZ off=7210861 len=77358 2024-04-07T16:41:08.370 INFO:tasks.workunit.client.0.smithi100.stdout:op 761 completed, throughput=4.97MB/sec 2024-04-07T16:41:08.371 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-J8dLUdKPwS2THju off=243805 len=451353 2024-04-07T16:41:08.372 INFO:tasks.workunit.client.0.smithi100.stdout:op 762 completed, throughput=4.98MB/sec 2024-04-07T16:41:08.372 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-sW31kG_e4oNWzo_ off=16230066 len=405298 2024-04-07T16:41:08.378 INFO:tasks.workunit.client.0.smithi100.stdout:op 763 completed, throughput=4.98MB/sec 2024-04-07T16:41:08.378 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-OhwhQgtSvDDoVPZ off=1354845 len=1329210 2024-04-07T16:41:08.381 INFO:tasks.workunit.client.0.smithi100.stdout:op 764 completed, throughput=4.99MB/sec 2024-04-07T16:41:08.382 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-BxA-fiUZmL6o2vi off=6875875 len=28267 2024-04-07T16:41:08.386 INFO:tasks.workunit.client.0.smithi100.stdout:op 765 completed, throughput=4.99MB/sec 2024-04-07T16:41:08.386 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-oSzdTRPO1OK54rC off=20106154 len=1422854 2024-04-07T16:41:08.397 INFO:tasks.workunit.client.0.smithi100.stdout:op 766 completed, throughput=5MB/sec 2024-04-07T16:41:08.397 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-URwzNCeNuZXff9J off=9147641 len=1123456 2024-04-07T16:41:08.400 INFO:tasks.workunit.client.0.smithi100.stdout:op 767 completed, throughput=5MB/sec 2024-04-07T16:41:08.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:41:08 smithi144 bash[66654]: cluster 2024-04-07T16:41:07.158137+0000 mgr.y (mgr.25175) 773 : cluster [DBG] pgmap v1415: 105 pgs: 105 active+clean; 126 GiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.7 MiB/s wr, 5 op/s 2024-04-07T16:41:08.692 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:41:08 smithi100 bash[74942]: cluster 2024-04-07T16:41:07.158137+0000 mgr.y (mgr.25175) 773 : cluster [DBG] pgmap v1415: 105 pgs: 105 active+clean; 126 GiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.7 MiB/s wr, 5 op/s 2024-04-07T16:41:08.693 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:41:08 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:41:08] "GET /metrics HTTP/1.1" 200 34055 "" "Prometheus/2.43.0" 2024-04-07T16:41:08.693 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:41:08 smithi100 bash[76202]: cluster 2024-04-07T16:41:07.158137+0000 mgr.y (mgr.25175) 773 : cluster [DBG] pgmap v1415: 105 pgs: 105 active+clean; 126 GiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.7 MiB/s wr, 5 op/s 2024-04-07T16:41:09.401 INFO:tasks.workunit.client.0.smithi100.stdout: 148: throughput=4.97MB/sec pending data=0 2024-04-07T16:41:09.401 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-JVlUB4Tg4jJT0Xx off=15549136 len=911993 2024-04-07T16:41:09.409 INFO:tasks.workunit.client.0.smithi100.stdout:op 768 completed, throughput=4.98MB/sec 2024-04-07T16:41:09.409 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-_a3woyP1y0BF8YS off=2409347 len=429031 2024-04-07T16:41:09.411 INFO:tasks.workunit.client.0.smithi100.stdout:op 769 completed, throughput=4.98MB/sec 2024-04-07T16:41:09.411 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-iuMWMvApYivtW86 off=2844917 len=945989 2024-04-07T16:41:09.419 INFO:tasks.workunit.client.0.smithi100.stdout:op 770 completed, throughput=4.98MB/sec 2024-04-07T16:41:09.419 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Is5kLp-wpxV6dmn off=825513 len=160823 2024-04-07T16:41:09.424 INFO:tasks.workunit.client.0.smithi100.stdout:op 771 completed, throughput=4.99MB/sec 2024-04-07T16:41:09.424 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-mXQ6hUkgGi0X7wQ off=9640436 len=788104 2024-04-07T16:41:09.431 INFO:tasks.workunit.client.0.smithi100.stdout:op 772 completed, throughput=4.99MB/sec 2024-04-07T16:41:09.432 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-jXTugkeKi6LPS9c off=21108397 len=857738 2024-04-07T16:41:09.434 INFO:tasks.workunit.client.0.smithi100.stdout:op 773 completed, throughput=5MB/sec 2024-04-07T16:41:09.434 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-sQ1iH5IVBsYzPII off=2753517 len=783900 2024-04-07T16:41:09.435 INFO:tasks.workunit.client.0.smithi100.stdout:op 774 completed, throughput=5MB/sec 2024-04-07T16:41:10.436 INFO:tasks.workunit.client.0.smithi100.stdout: 149: throughput=4.97MB/sec pending data=0 2024-04-07T16:41:10.436 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-3WY3wFgNcS2GFFc off=13181 len=1469916 2024-04-07T16:41:10.447 INFO:tasks.workunit.client.0.smithi100.stdout:op 775 completed, throughput=4.98MB/sec 2024-04-07T16:41:10.447 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-fbFkp7M8Ru8Fi1y off=12615754 len=1170014 2024-04-07T16:41:10.450 INFO:tasks.workunit.client.0.smithi100.stdout:op 776 completed, throughput=4.98MB/sec 2024-04-07T16:41:10.451 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-8YYEhWSgdGuBNrT off=6900827 len=980743 2024-04-07T16:41:10.459 INFO:tasks.workunit.client.0.smithi100.stdout:op 777 completed, throughput=4.99MB/sec 2024-04-07T16:41:10.459 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-zmCM9iFXx7xcVha off=8774064 len=1817259 2024-04-07T16:41:10.464 INFO:tasks.workunit.client.0.smithi100.stdout:op 778 completed, throughput=5MB/sec 2024-04-07T16:41:10.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:41:10 smithi144 bash[66654]: cluster 2024-04-07T16:41:09.158968+0000 mgr.y (mgr.25175) 774 : cluster [DBG] pgmap v1416: 105 pgs: 105 active+clean; 126 GiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 2.6 MiB/s rd, 2.5 MiB/s wr, 5 op/s 2024-04-07T16:41:10.692 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:41:10 smithi100 bash[74942]: cluster 2024-04-07T16:41:09.158968+0000 mgr.y (mgr.25175) 774 : cluster [DBG] pgmap v1416: 105 pgs: 105 active+clean; 126 GiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 2.6 MiB/s rd, 2.5 MiB/s wr, 5 op/s 2024-04-07T16:41:10.692 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:41:10 smithi100 bash[76202]: cluster 2024-04-07T16:41:09.158968+0000 mgr.y (mgr.25175) 774 : cluster [DBG] pgmap v1416: 105 pgs: 105 active+clean; 126 GiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 2.6 MiB/s rd, 2.5 MiB/s wr, 5 op/s 2024-04-07T16:41:11.464 INFO:tasks.workunit.client.0.smithi100.stdout: 150: throughput=4.97MB/sec pending data=0 2024-04-07T16:41:11.464 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-921qTDKb0XhaPos off=1191120 len=1443227 2024-04-07T16:41:11.468 INFO:tasks.workunit.client.0.smithi100.stdout:op 779 completed, throughput=4.98MB/sec 2024-04-07T16:41:11.468 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-7kAk3LHsrKl6YyN off=3822375 len=2058118 2024-04-07T16:41:11.483 INFO:tasks.workunit.client.0.smithi100.stdout:op 780 completed, throughput=4.99MB/sec 2024-04-07T16:41:11.484 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-xRCTzPKR8RNUeWI off=9202041 len=472523 2024-04-07T16:41:11.490 INFO:tasks.workunit.client.0.smithi100.stdout:op 781 completed, throughput=4.99MB/sec 2024-04-07T16:41:11.490 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-nwuNxe_WZkYOMpd off=1499780 len=966330 2024-04-07T16:41:11.493 INFO:tasks.workunit.client.0.smithi100.stdout:op 782 completed, throughput=5MB/sec 2024-04-07T16:41:11.493 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-LEAlDPNPm4MWoo- off=1736454 len=314110 2024-04-07T16:41:11.498 INFO:tasks.workunit.client.0.smithi100.stdout:op 783 completed, throughput=5MB/sec 2024-04-07T16:41:11.498 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-5Dd_b7kgNaRac59 off=7659709 len=1457187 2024-04-07T16:41:11.509 INFO:tasks.workunit.client.0.smithi100.stdout:op 784 completed, throughput=5.01MB/sec 2024-04-07T16:41:12.510 INFO:tasks.workunit.client.0.smithi100.stdout: 151: throughput=4.98MB/sec pending data=0 2024-04-07T16:41:12.510 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-B8HWS8ewD91xN5i off=6985133 len=1512759 2024-04-07T16:41:12.515 INFO:tasks.workunit.client.0.smithi100.stdout:op 785 completed, throughput=4.98MB/sec 2024-04-07T16:41:12.515 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-MfnsMNoeL1a_Tua off=6665611 len=13542 2024-04-07T16:41:12.516 INFO:tasks.workunit.client.0.smithi100.stdout:op 786 completed, throughput=4.98MB/sec 2024-04-07T16:41:12.516 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-btSKtPjCrL6iIDQ off=2478994 len=1282423 2024-04-07T16:41:12.520 INFO:tasks.workunit.client.0.smithi100.stdout:op 787 completed, throughput=4.99MB/sec 2024-04-07T16:41:12.520 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-TXP7cmlPRTBMTyd off=8839981 len=298093 2024-04-07T16:41:12.526 INFO:tasks.workunit.client.0.smithi100.stdout:op 788 completed, throughput=4.99MB/sec 2024-04-07T16:41:12.526 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-HK_DlEg1sM0aabj off=11847411 len=1025659 2024-04-07T16:41:12.536 INFO:tasks.workunit.client.0.smithi100.stdout:op 789 completed, throughput=5MB/sec 2024-04-07T16:41:12.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:41:12 smithi144 bash[66654]: cluster 2024-04-07T16:41:11.159950+0000 mgr.y (mgr.25175) 775 : cluster [DBG] pgmap v1417: 105 pgs: 105 active+clean; 126 GiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.6 MiB/s wr, 5 op/s 2024-04-07T16:41:12.607 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:41:12 smithi144 bash[95233]: ts=2024-04-07T16:41:12.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:41:12.692 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:41:12 smithi100 bash[74942]: cluster 2024-04-07T16:41:11.159950+0000 mgr.y (mgr.25175) 775 : cluster [DBG] pgmap v1417: 105 pgs: 105 active+clean; 126 GiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.6 MiB/s wr, 5 op/s 2024-04-07T16:41:12.693 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:41:12 smithi100 bash[76202]: cluster 2024-04-07T16:41:11.159950+0000 mgr.y (mgr.25175) 775 : cluster [DBG] pgmap v1417: 105 pgs: 105 active+clean; 126 GiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.6 MiB/s wr, 5 op/s 2024-04-07T16:41:13.536 INFO:tasks.workunit.client.0.smithi100.stdout: 152: throughput=4.97MB/sec pending data=0 2024-04-07T16:41:13.536 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-qLWfbfSldZdiYBj off=4587339 len=628867 2024-04-07T16:41:13.543 INFO:tasks.workunit.client.0.smithi100.stdout:op 790 completed, throughput=4.97MB/sec 2024-04-07T16:41:13.543 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-I2Z4I5tr-7SYOqu off=3217631 len=908446 2024-04-07T16:41:13.546 INFO:tasks.workunit.client.0.smithi100.stdout:op 791 completed, throughput=4.98MB/sec 2024-04-07T16:41:13.546 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-qHgHDV-c-WYCQn8 off=118546 len=1731526 2024-04-07T16:41:13.559 INFO:tasks.workunit.client.0.smithi100.stdout:op 792 completed, throughput=4.99MB/sec 2024-04-07T16:41:13.559 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-MioZbj0i_tWFd4p off=21907330 len=871598 2024-04-07T16:41:13.567 INFO:tasks.workunit.client.0.smithi100.stdout:op 793 completed, throughput=4.99MB/sec 2024-04-07T16:41:13.567 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-ee2yYBacRdTQt2S off=4483308 len=1238730 2024-04-07T16:41:13.579 INFO:tasks.workunit.client.0.smithi100.stdout:op 794 completed, throughput=5MB/sec 2024-04-07T16:41:13.580 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj--Im6oeHe7-e9itC off=3749199 len=224170 2024-04-07T16:41:13.581 INFO:tasks.workunit.client.0.smithi100.stdout:op 795 completed, throughput=5MB/sec 2024-04-07T16:41:14.582 INFO:tasks.workunit.client.0.smithi100.stdout: 153: throughput=4.97MB/sec pending data=0 2024-04-07T16:41:14.582 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-SBYHsTg-qEMs3Mj off=6748548 len=1609608 2024-04-07T16:41:14.594 INFO:tasks.workunit.client.0.smithi100.stdout:op 796 completed, throughput=4.98MB/sec 2024-04-07T16:41:14.594 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-r9OeTJ4q7jh1nDb off=22377494 len=487896 2024-04-07T16:41:14.599 INFO:tasks.workunit.client.0.smithi100.stdout:op 797 completed, throughput=4.98MB/sec 2024-04-07T16:41:14.600 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-XsfrFO2hM6TpQQ7 off=1949596 len=135646 2024-04-07T16:41:14.601 INFO:tasks.workunit.client.0.smithi100.stdout:op 798 completed, throughput=4.98MB/sec 2024-04-07T16:41:14.601 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-JwDpcZGy3B4hBL5 off=2470277 len=1525262 2024-04-07T16:41:14.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:41:14 smithi144 bash[66654]: cluster 2024-04-07T16:41:13.161092+0000 mgr.y (mgr.25175) 776 : cluster [DBG] pgmap v1418: 105 pgs: 105 active+clean; 126 GiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 2.9 MiB/s rd, 3.0 MiB/s wr, 6 op/s 2024-04-07T16:41:14.616 INFO:tasks.workunit.client.0.smithi100.stdout:op 799 completed, throughput=4.99MB/sec 2024-04-07T16:41:14.616 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-3AXLpbb6Ts6Aiwx off=4000929 len=81721 2024-04-07T16:41:14.620 INFO:tasks.workunit.client.0.smithi100.stdout:op 800 completed, throughput=4.99MB/sec 2024-04-07T16:41:14.620 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-SDYJdINLFhGug0a off=925827 len=1360442 2024-04-07T16:41:14.629 INFO:tasks.workunit.client.0.smithi100.stdout:op 801 completed, throughput=5MB/sec 2024-04-07T16:41:14.629 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-uC9WqEYqy_lEOF8 off=1665574 len=1794848 2024-04-07T16:41:14.634 INFO:tasks.workunit.client.0.smithi100.stdout:op 802 completed, throughput=5.01MB/sec 2024-04-07T16:41:14.692 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:41:14 smithi100 bash[74942]: cluster 2024-04-07T16:41:13.161092+0000 mgr.y (mgr.25175) 776 : cluster [DBG] pgmap v1418: 105 pgs: 105 active+clean; 126 GiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 2.9 MiB/s rd, 3.0 MiB/s wr, 6 op/s 2024-04-07T16:41:14.693 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:41:14 smithi100 bash[76202]: cluster 2024-04-07T16:41:13.161092+0000 mgr.y (mgr.25175) 776 : cluster [DBG] pgmap v1418: 105 pgs: 105 active+clean; 126 GiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 2.9 MiB/s rd, 3.0 MiB/s wr, 6 op/s 2024-04-07T16:41:15.634 INFO:tasks.workunit.client.0.smithi100.stdout: 154: throughput=4.98MB/sec pending data=0 2024-04-07T16:41:15.635 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-lkjyTMH5NfYck-d off=9495411 len=691391 2024-04-07T16:41:15.637 INFO:tasks.workunit.client.0.smithi100.stdout:op 803 completed, throughput=4.98MB/sec 2024-04-07T16:41:15.637 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-06Iw4iRUoaDRMA_ off=2075279 len=683376 2024-04-07T16:41:15.640 INFO:tasks.workunit.client.0.smithi100.stdout:op 804 completed, throughput=4.99MB/sec 2024-04-07T16:41:15.640 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-vkN2bDEjzvOafc6 off=4945719 len=142001 2024-04-07T16:41:15.645 INFO:tasks.workunit.client.0.smithi100.stdout:op 805 completed, throughput=4.99MB/sec 2024-04-07T16:41:15.645 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Hu8GomMlX-prxl8 off=961470 len=1061044 2024-04-07T16:41:15.654 INFO:tasks.workunit.client.0.smithi100.stdout:op 806 completed, throughput=4.99MB/sec 2024-04-07T16:41:15.654 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-fniGFn6VX1tg0Yh off=10795064 len=1366295 2024-04-07T16:41:15.664 INFO:tasks.workunit.client.0.smithi100.stdout:op 807 completed, throughput=5MB/sec 2024-04-07T16:41:16.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:41:16 smithi144 bash[66654]: cluster 2024-04-07T16:41:15.162050+0000 mgr.y (mgr.25175) 777 : cluster [DBG] pgmap v1419: 105 pgs: 105 active+clean; 126 GiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 2.2 MiB/s rd, 2.4 MiB/s wr, 5 op/s 2024-04-07T16:41:16.665 INFO:tasks.workunit.client.0.smithi100.stdout: 155: throughput=4.97MB/sec pending data=0 2024-04-07T16:41:16.665 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-umpHh3_wz3sYMsG off=12438061 len=846246 2024-04-07T16:41:16.673 INFO:tasks.workunit.client.0.smithi100.stdout:op 808 completed, throughput=4.97MB/sec 2024-04-07T16:41:16.673 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-mjUtoBBKjsp0UGk off=721396 len=1448463 2024-04-07T16:41:16.677 INFO:tasks.workunit.client.0.smithi100.stdout:op 809 completed, throughput=4.98MB/sec 2024-04-07T16:41:16.677 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-GYqGpqnXm6oZkCP off=7513549 len=1941119 2024-04-07T16:41:16.682 INFO:tasks.workunit.client.0.smithi100.stdout:op 810 completed, throughput=4.99MB/sec 2024-04-07T16:41:16.682 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-yi1mHl25nv6hgHk off=18306057 len=105071 2024-04-07T16:41:16.684 INFO:tasks.workunit.client.0.smithi100.stdout:op 811 completed, throughput=5MB/sec 2024-04-07T16:41:16.684 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-_-iXG4Yjhpc-Z0e off=0 len=1797640 2024-04-07T16:41:16.690 INFO:tasks.workunit.client.0.smithi100.stdout:op 812 completed, throughput=5.01MB/sec 2024-04-07T16:41:16.692 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:41:16 smithi100 bash[74942]: cluster 2024-04-07T16:41:15.162050+0000 mgr.y (mgr.25175) 777 : cluster [DBG] pgmap v1419: 105 pgs: 105 active+clean; 126 GiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 2.2 MiB/s rd, 2.4 MiB/s wr, 5 op/s 2024-04-07T16:41:16.692 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:41:16 smithi100 bash[76202]: cluster 2024-04-07T16:41:15.162050+0000 mgr.y (mgr.25175) 777 : cluster [DBG] pgmap v1419: 105 pgs: 105 active+clean; 126 GiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 2.2 MiB/s rd, 2.4 MiB/s wr, 5 op/s 2024-04-07T16:41:17.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:41:17 smithi144 bash[66654]: audit 2024-04-07T16:41:16.758955+0000 mon.a (mon.0) 2066 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:41:17.691 INFO:tasks.workunit.client.0.smithi100.stdout: 156: throughput=4.97MB/sec pending data=0 2024-04-07T16:41:17.691 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-vuQKF1cJ6qZaBKB off=8000148 len=1328287 2024-04-07T16:41:17.692 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:41:17 smithi100 bash[74942]: audit 2024-04-07T16:41:16.758955+0000 mon.a (mon.0) 2066 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:41:17.692 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:41:17 smithi100 bash[76202]: audit 2024-04-07T16:41:16.758955+0000 mon.a (mon.0) 2066 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:41:17.703 INFO:tasks.workunit.client.0.smithi100.stdout:op 813 completed, throughput=4.98MB/sec 2024-04-07T16:41:17.703 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-6Q4ABEAYs-ENKyR off=4698617 len=1024706 2024-04-07T16:41:17.712 INFO:tasks.workunit.client.0.smithi100.stdout:op 814 completed, throughput=4.99MB/sec 2024-04-07T16:41:17.712 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Lxz0T1NUjhqNfYz off=1177220 len=1723847 2024-04-07T16:41:17.727 INFO:tasks.workunit.client.0.smithi100.stdout:op 815 completed, throughput=5MB/sec 2024-04-07T16:41:17.727 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-s_Z-O0PgM0MYQus off=3995349 len=154671 2024-04-07T16:41:17.728 INFO:tasks.workunit.client.0.smithi100.stdout:op 816 completed, throughput=5MB/sec 2024-04-07T16:41:17.728 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-s2aBFXp4dAm1HCZ off=4038740 len=621975 2024-04-07T16:41:17.730 INFO:tasks.workunit.client.0.smithi100.stdout:op 817 completed, throughput=5MB/sec 2024-04-07T16:41:18.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:41:18 smithi144 bash[66654]: cluster 2024-04-07T16:41:17.163094+0000 mgr.y (mgr.25175) 778 : cluster [DBG] pgmap v1420: 105 pgs: 105 active+clean; 126 GiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 2.0 MiB/s rd, 2.7 MiB/s wr, 5 op/s 2024-04-07T16:41:18.692 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:41:18 smithi100 bash[74942]: cluster 2024-04-07T16:41:17.163094+0000 mgr.y (mgr.25175) 778 : cluster [DBG] pgmap v1420: 105 pgs: 105 active+clean; 126 GiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 2.0 MiB/s rd, 2.7 MiB/s wr, 5 op/s 2024-04-07T16:41:18.693 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:41:18 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:41:18] "GET /metrics HTTP/1.1" 200 34058 "" "Prometheus/2.43.0" 2024-04-07T16:41:18.693 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:41:18 smithi100 bash[76202]: cluster 2024-04-07T16:41:17.163094+0000 mgr.y (mgr.25175) 778 : cluster [DBG] pgmap v1420: 105 pgs: 105 active+clean; 126 GiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 2.0 MiB/s rd, 2.7 MiB/s wr, 5 op/s 2024-04-07T16:41:18.730 INFO:tasks.workunit.client.0.smithi100.stdout: 157: throughput=4.97MB/sec pending data=0 2024-04-07T16:41:18.730 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-gAWKJNUxPcoQZE5 off=1786444 len=1606249 2024-04-07T16:41:18.744 INFO:tasks.workunit.client.0.smithi100.stdout:op 818 completed, throughput=4.98MB/sec 2024-04-07T16:41:18.744 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-S0zpLCzHkgKWy49 off=7734925 len=726165 2024-04-07T16:41:18.746 INFO:tasks.workunit.client.0.smithi100.stdout:op 819 completed, throughput=4.98MB/sec 2024-04-07T16:41:18.746 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-O-uyVSKh8Fm0YvU off=0 len=1347602 2024-04-07T16:41:18.750 INFO:tasks.workunit.client.0.smithi100.stdout:op 820 completed, throughput=4.99MB/sec 2024-04-07T16:41:18.750 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-oQsUwbqZ46MfI03 off=6945029 len=1808988 2024-04-07T16:41:18.754 INFO:tasks.workunit.client.0.smithi100.stdout:op 821 completed, throughput=5MB/sec 2024-04-07T16:41:19.754 INFO:tasks.workunit.client.0.smithi100.stdout: 158: throughput=4.97MB/sec pending data=0 2024-04-07T16:41:19.755 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-7s7LfC9oiLbkzRX off=4498904 len=279945 2024-04-07T16:41:19.760 INFO:tasks.workunit.client.0.smithi100.stdout:op 822 completed, throughput=4.97MB/sec 2024-04-07T16:41:19.760 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-d16DaHrU1t0V331 off=4114216 len=686959 2024-04-07T16:41:19.768 INFO:tasks.workunit.client.0.smithi100.stdout:op 823 completed, throughput=4.98MB/sec 2024-04-07T16:41:19.768 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-hldBVmdKgcPczz4 off=7105846 len=1282103 2024-04-07T16:41:19.773 INFO:tasks.workunit.client.0.smithi100.stdout:op 824 completed, throughput=4.98MB/sec 2024-04-07T16:41:19.773 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-gGCLWfSBl1bgj9- off=18266225 len=684035 2024-04-07T16:41:19.774 INFO:tasks.workunit.client.0.smithi100.stdout:op 825 completed, throughput=4.99MB/sec 2024-04-07T16:41:19.774 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-m5P-0I-GAnGldYN off=3022798 len=1660477 2024-04-07T16:41:19.778 INFO:tasks.workunit.client.0.smithi100.stdout:op 826 completed, throughput=5MB/sec 2024-04-07T16:41:19.778 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-3B9m53crH9ZyX1S off=822773 len=462461 2024-04-07T16:41:19.785 INFO:tasks.workunit.client.0.smithi100.stdout:op 827 completed, throughput=5MB/sec 2024-04-07T16:41:20.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:41:20 smithi144 bash[66654]: cluster 2024-04-07T16:41:19.163946+0000 mgr.y (mgr.25175) 779 : cluster [DBG] pgmap v1421: 105 pgs: 105 active+clean; 126 GiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 2.4 MiB/s rd, 2.4 MiB/s wr, 5 op/s 2024-04-07T16:41:20.692 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:41:20 smithi100 bash[74942]: cluster 2024-04-07T16:41:19.163946+0000 mgr.y (mgr.25175) 779 : cluster [DBG] pgmap v1421: 105 pgs: 105 active+clean; 126 GiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 2.4 MiB/s rd, 2.4 MiB/s wr, 5 op/s 2024-04-07T16:41:20.693 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:41:20 smithi100 bash[76202]: cluster 2024-04-07T16:41:19.163946+0000 mgr.y (mgr.25175) 779 : cluster [DBG] pgmap v1421: 105 pgs: 105 active+clean; 126 GiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 2.4 MiB/s rd, 2.4 MiB/s wr, 5 op/s 2024-04-07T16:41:20.785 INFO:tasks.workunit.client.0.smithi100.stdout: 159: throughput=4.97MB/sec pending data=0 2024-04-07T16:41:20.785 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-4zEn7sbHwW2hUIv off=12463667 len=1149233 2024-04-07T16:41:20.788 INFO:tasks.workunit.client.0.smithi100.stdout:op 828 completed, throughput=4.98MB/sec 2024-04-07T16:41:20.789 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-PiF_aJrd7ZK0dZu off=2629519 len=926316 2024-04-07T16:41:20.792 INFO:tasks.workunit.client.0.smithi100.stdout:op 829 completed, throughput=4.98MB/sec 2024-04-07T16:41:20.792 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-iqErQLD9hRYdgnP off=8792140 len=750288 2024-04-07T16:41:20.801 INFO:tasks.workunit.client.0.smithi100.stdout:op 830 completed, throughput=4.99MB/sec 2024-04-07T16:41:20.801 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-r42E-0GAJNVZege off=6640113 len=222913 2024-04-07T16:41:20.802 INFO:tasks.workunit.client.0.smithi100.stdout:op 831 completed, throughput=4.99MB/sec 2024-04-07T16:41:20.802 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-CgK-c78P_qSrjFk off=234423 len=2089567 2024-04-07T16:41:20.817 INFO:tasks.workunit.client.0.smithi100.stdout:op 832 completed, throughput=5MB/sec 2024-04-07T16:41:20.817 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-TFsQm_oGbf10bVH off=2534580 len=941190 2024-04-07T16:41:20.820 INFO:tasks.workunit.client.0.smithi100.stdout:op 833 completed, throughput=5MB/sec 2024-04-07T16:41:21.820 INFO:tasks.workunit.client.0.smithi100.stdout: 160: throughput=4.97MB/sec pending data=0 2024-04-07T16:41:21.821 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-fA-DD0E364nY0XS off=12119370 len=548172 2024-04-07T16:41:21.823 INFO:tasks.workunit.client.0.smithi100.stdout:op 834 completed, throughput=4.98MB/sec 2024-04-07T16:41:21.824 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-hIpoC9MkPqPbYWW off=12735427 len=816526 2024-04-07T16:41:21.832 INFO:tasks.workunit.client.0.smithi100.stdout:op 835 completed, throughput=4.98MB/sec 2024-04-07T16:41:21.832 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-hzwSxMzvLaVwgZs off=1884964 len=859529 2024-04-07T16:41:21.835 INFO:tasks.workunit.client.0.smithi100.stdout:op 836 completed, throughput=4.99MB/sec 2024-04-07T16:41:21.835 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-6hLk5THpV0EB27Y off=10543423 len=1172054 2024-04-07T16:41:21.845 INFO:tasks.workunit.client.0.smithi100.stdout:op 837 completed, throughput=4.99MB/sec 2024-04-07T16:41:21.846 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-1gsci6oxalFHYcn off=7969220 len=373028 2024-04-07T16:41:21.849 INFO:tasks.workunit.client.0.smithi100.stdout:op 838 completed, throughput=5MB/sec 2024-04-07T16:41:21.849 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-PiW-dB3vNJZ3Bz4 off=5316977 len=1872111 2024-04-07T16:41:21.861 INFO:tasks.workunit.client.0.smithi100.stdout:op 839 completed, throughput=5.01MB/sec 2024-04-07T16:41:22.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:41:22 smithi144 bash[66654]: cluster 2024-04-07T16:41:21.164872+0000 mgr.y (mgr.25175) 780 : cluster [DBG] pgmap v1422: 105 pgs: 105 active+clean; 126 GiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 2.4 MiB/s rd, 2.4 MiB/s wr, 5 op/s 2024-04-07T16:41:22.607 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:41:22 smithi144 bash[95233]: ts=2024-04-07T16:41:22.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:41:22.692 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:41:22 smithi100 bash[74942]: cluster 2024-04-07T16:41:21.164872+0000 mgr.y (mgr.25175) 780 : cluster [DBG] pgmap v1422: 105 pgs: 105 active+clean; 126 GiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 2.4 MiB/s rd, 2.4 MiB/s wr, 5 op/s 2024-04-07T16:41:22.692 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:41:22 smithi100 bash[76202]: cluster 2024-04-07T16:41:21.164872+0000 mgr.y (mgr.25175) 780 : cluster [DBG] pgmap v1422: 105 pgs: 105 active+clean; 126 GiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 2.4 MiB/s rd, 2.4 MiB/s wr, 5 op/s 2024-04-07T16:41:22.862 INFO:tasks.workunit.client.0.smithi100.stdout: 161: throughput=4.98MB/sec pending data=0 2024-04-07T16:41:22.862 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Yb4DxQUM2-fVJ_Q off=10555556 len=742577 2024-04-07T16:41:22.864 INFO:tasks.workunit.client.0.smithi100.stdout:op 840 completed, throughput=4.98MB/sec 2024-04-07T16:41:22.864 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-zzw-iZv2SJrMOFy off=17800217 len=381186 2024-04-07T16:41:22.866 INFO:tasks.workunit.client.0.smithi100.stdout:op 841 completed, throughput=4.98MB/sec 2024-04-07T16:41:22.866 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Wxoc0OrSBFCTSkN off=3315859 len=1610287 2024-04-07T16:41:22.879 INFO:tasks.workunit.client.0.smithi100.stdout:op 842 completed, throughput=4.99MB/sec 2024-04-07T16:41:22.879 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-WB9pd9reCLTA8Ad off=207865 len=1280140 2024-04-07T16:41:22.882 INFO:tasks.workunit.client.0.smithi100.stdout:op 843 completed, throughput=5MB/sec 2024-04-07T16:41:22.882 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-y0dA1PFrreIqNWm off=3805262 len=1082808 2024-04-07T16:41:22.892 INFO:tasks.workunit.client.0.smithi100.stdout:op 844 completed, throughput=5MB/sec 2024-04-07T16:41:23.894 INFO:tasks.workunit.client.0.smithi100.stdout: 162: throughput=4.97MB/sec pending data=0 2024-04-07T16:41:23.894 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-bmFeuolgxJCnsxQ off=2512048 len=1116455 2024-04-07T16:41:23.895 INFO:tasks.workunit.client.0.smithi100.stdout:op 845 completed, throughput=4.98MB/sec 2024-04-07T16:41:23.895 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-tnXq3RX0JWOdao1 off=1879662 len=1429124 2024-04-07T16:41:23.908 INFO:tasks.workunit.client.0.smithi100.stdout:op 846 completed, throughput=4.99MB/sec 2024-04-07T16:41:23.908 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-HaZ_PTUhrq_Hokx off=8833047 len=1641686 2024-04-07T16:41:23.914 INFO:tasks.workunit.client.0.smithi100.stdout:op 847 completed, throughput=5MB/sec 2024-04-07T16:41:23.914 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-FZKOTpqlDuyA1j4 off=10513456 len=1465999 2024-04-07T16:41:23.925 INFO:tasks.workunit.client.0.smithi100.stdout:op 848 completed, throughput=5MB/sec 2024-04-07T16:41:24.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:41:24 smithi144 bash[66654]: cluster 2024-04-07T16:41:23.165971+0000 mgr.y (mgr.25175) 781 : cluster [DBG] pgmap v1423: 105 pgs: 105 active+clean; 126 GiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 2.8 MiB/s rd, 3.0 MiB/s wr, 6 op/s 2024-04-07T16:41:24.692 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:41:24 smithi100 bash[74942]: cluster 2024-04-07T16:41:23.165971+0000 mgr.y (mgr.25175) 781 : cluster [DBG] pgmap v1423: 105 pgs: 105 active+clean; 126 GiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 2.8 MiB/s rd, 3.0 MiB/s wr, 6 op/s 2024-04-07T16:41:24.693 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:41:24 smithi100 bash[76202]: cluster 2024-04-07T16:41:23.165971+0000 mgr.y (mgr.25175) 781 : cluster [DBG] pgmap v1423: 105 pgs: 105 active+clean; 126 GiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 2.8 MiB/s rd, 3.0 MiB/s wr, 6 op/s 2024-04-07T16:41:24.926 INFO:tasks.workunit.client.0.smithi100.stdout: 163: throughput=4.98MB/sec pending data=0 2024-04-07T16:41:24.926 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj--ftzO-5__FZAswO off=2277673 len=1083419 2024-04-07T16:41:24.930 INFO:tasks.workunit.client.0.smithi100.stdout:op 849 completed, throughput=4.98MB/sec 2024-04-07T16:41:24.930 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-kuScHAwvGGoFiG- off=2903350 len=1615197 2024-04-07T16:41:24.943 INFO:tasks.workunit.client.0.smithi100.stdout:op 850 completed, throughput=4.99MB/sec 2024-04-07T16:41:24.943 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-a-z1kz5UzrnJgGF off=10205397 len=1939397 2024-04-07T16:41:24.947 INFO:tasks.workunit.client.0.smithi100.stdout:op 851 completed, throughput=5MB/sec 2024-04-07T16:41:25.948 INFO:tasks.workunit.client.0.smithi100.stdout: 164: throughput=4.97MB/sec pending data=0 2024-04-07T16:41:25.948 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-5ZI1hbgQUUioJ-M off=1204887 len=1465592 2024-04-07T16:41:25.951 INFO:tasks.workunit.client.0.smithi100.stdout:op 852 completed, throughput=4.98MB/sec 2024-04-07T16:41:25.952 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-u7ujExEUq7y1tTK off=11865746 len=219253 2024-04-07T16:41:25.954 INFO:tasks.workunit.client.0.smithi100.stdout:op 853 completed, throughput=4.98MB/sec 2024-04-07T16:41:25.954 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-WhOkiBU-28ZtLfI off=16581705 len=147732 2024-04-07T16:41:25.955 INFO:tasks.workunit.client.0.smithi100.stdout:op 854 completed, throughput=4.98MB/sec 2024-04-07T16:41:25.955 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-V52PIaSlvkDsdtf off=1545686 len=32902 2024-04-07T16:41:25.957 INFO:tasks.workunit.client.0.smithi100.stdout:op 855 completed, throughput=4.98MB/sec 2024-04-07T16:41:25.957 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-jQP6pMHHjIZT6Oa off=1963979 len=2085607 2024-04-07T16:41:25.975 INFO:tasks.workunit.client.0.smithi100.stdout:op 856 completed, throughput=4.99MB/sec 2024-04-07T16:41:25.975 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-mHZyXpJO9wp-aNS off=10638251 len=39672 2024-04-07T16:41:25.980 INFO:tasks.workunit.client.0.smithi100.stdout:op 857 completed, throughput=4.99MB/sec 2024-04-07T16:41:25.980 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-UJmVkR3KXElmLKs off=14619830 len=2061856 2024-04-07T16:41:25.985 INFO:tasks.workunit.client.0.smithi100.stdout:op 858 completed, throughput=5MB/sec 2024-04-07T16:41:26.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:41:26 smithi144 bash[66654]: cluster 2024-04-07T16:41:25.166950+0000 mgr.y (mgr.25175) 782 : cluster [DBG] pgmap v1424: 105 pgs: 105 active+clean; 126 GiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.7 MiB/s wr, 5 op/s 2024-04-07T16:41:26.692 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:41:26 smithi100 bash[74942]: cluster 2024-04-07T16:41:25.166950+0000 mgr.y (mgr.25175) 782 : cluster [DBG] pgmap v1424: 105 pgs: 105 active+clean; 126 GiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.7 MiB/s wr, 5 op/s 2024-04-07T16:41:26.693 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:41:26 smithi100 bash[76202]: cluster 2024-04-07T16:41:25.166950+0000 mgr.y (mgr.25175) 782 : cluster [DBG] pgmap v1424: 105 pgs: 105 active+clean; 126 GiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.7 MiB/s wr, 5 op/s 2024-04-07T16:41:26.985 INFO:tasks.workunit.client.0.smithi100.stdout: 165: throughput=4.97MB/sec pending data=0 2024-04-07T16:41:26.985 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-iv6j-5UZmCOZmMg off=16091860 len=134996 2024-04-07T16:41:26.990 INFO:tasks.workunit.client.0.smithi100.stdout:op 859 completed, throughput=4.98MB/sec 2024-04-07T16:41:26.991 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-QOdZ5rrygEtKAsn off=3352378 len=277805 2024-04-07T16:41:26.992 INFO:tasks.workunit.client.0.smithi100.stdout:op 860 completed, throughput=4.98MB/sec 2024-04-07T16:41:26.992 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-YSIJtyrS7JUc1l8 off=1789166 len=1920285 2024-04-07T16:41:26.996 INFO:tasks.workunit.client.0.smithi100.stdout:op 861 completed, throughput=4.99MB/sec 2024-04-07T16:41:26.996 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-6NgoewM7c0Dt5Cs off=628147 len=345311 2024-04-07T16:41:27.002 INFO:tasks.workunit.client.0.smithi100.stdout:op 862 completed, throughput=4.99MB/sec 2024-04-07T16:41:27.003 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-V6BPqst9R1n8_DI off=11282801 len=717859 2024-04-07T16:41:27.011 INFO:tasks.workunit.client.0.smithi100.stdout:op 863 completed, throughput=4.99MB/sec 2024-04-07T16:41:27.011 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-aZFDdclOwgmodos off=9685598 len=772839 2024-04-07T16:41:27.018 INFO:tasks.workunit.client.0.smithi100.stdout:op 864 completed, throughput=5MB/sec 2024-04-07T16:41:27.018 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-pTeirFUln4rgJZn off=3369341 len=135994 2024-04-07T16:41:27.019 INFO:tasks.workunit.client.0.smithi100.stdout:op 865 completed, throughput=5MB/sec 2024-04-07T16:41:27.020 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-jfP_tPk0SihO62H off=5993186 len=573729 2024-04-07T16:41:27.027 INFO:tasks.workunit.client.0.smithi100.stdout:op 866 completed, throughput=5MB/sec 2024-04-07T16:41:28.027 INFO:tasks.workunit.client.0.smithi100.stdout: 166: throughput=4.97MB/sec pending data=0 2024-04-07T16:41:28.027 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-X1lsnDGPCG8pq3v off=4741535 len=2040469 2024-04-07T16:41:28.032 INFO:tasks.workunit.client.0.smithi100.stdout:op 867 completed, throughput=4.98MB/sec 2024-04-07T16:41:28.032 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-IUfYdHgvFHpBxKv off=2170294 len=36598 2024-04-07T16:41:28.036 INFO:tasks.workunit.client.0.smithi100.stdout:op 868 completed, throughput=4.98MB/sec 2024-04-07T16:41:28.036 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-H6LiSIACKlI8B2t off=8051185 len=1764257 2024-04-07T16:41:28.040 INFO:tasks.workunit.client.0.smithi100.stdout:op 869 completed, throughput=4.99MB/sec 2024-04-07T16:41:28.041 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-OFFfV7ryz1Mb_p1 off=13688257 len=1910575 2024-04-07T16:41:28.057 INFO:tasks.workunit.client.0.smithi100.stdout:op 870 completed, throughput=5MB/sec 2024-04-07T16:41:28.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:41:28 smithi144 bash[66654]: cluster 2024-04-07T16:41:27.168040+0000 mgr.y (mgr.25175) 783 : cluster [DBG] pgmap v1425: 105 pgs: 105 active+clean; 126 GiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.7 MiB/s wr, 5 op/s 2024-04-07T16:41:28.692 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:41:28 smithi100 bash[74942]: cluster 2024-04-07T16:41:27.168040+0000 mgr.y (mgr.25175) 783 : cluster [DBG] pgmap v1425: 105 pgs: 105 active+clean; 126 GiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.7 MiB/s wr, 5 op/s 2024-04-07T16:41:28.693 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:41:28 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:41:28] "GET /metrics HTTP/1.1" 200 34058 "" "Prometheus/2.43.0" 2024-04-07T16:41:28.693 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:41:28 smithi100 bash[76202]: cluster 2024-04-07T16:41:27.168040+0000 mgr.y (mgr.25175) 783 : cluster [DBG] pgmap v1425: 105 pgs: 105 active+clean; 126 GiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.7 MiB/s wr, 5 op/s 2024-04-07T16:41:29.057 INFO:tasks.workunit.client.0.smithi100.stdout: 167: throughput=4.97MB/sec pending data=0 2024-04-07T16:41:29.057 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-bEOqVI362DnEe7Y off=11885277 len=1598496 2024-04-07T16:41:29.070 INFO:tasks.workunit.client.0.smithi100.stdout:op 871 completed, throughput=4.98MB/sec 2024-04-07T16:41:29.070 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-N4CpEpNT-K7kOez off=16234779 len=445074 2024-04-07T16:41:29.076 INFO:tasks.workunit.client.0.smithi100.stdout:op 872 completed, throughput=4.98MB/sec 2024-04-07T16:41:29.076 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-VUxM-4Hix9qmno4 off=2318327 len=390491 2024-04-07T16:41:29.078 INFO:tasks.workunit.client.0.smithi100.stdout:op 873 completed, throughput=4.99MB/sec 2024-04-07T16:41:29.078 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-9BCUZTCUvbmzRlZ off=3267267 len=210204 2024-04-07T16:41:29.084 INFO:tasks.workunit.client.0.smithi100.stdout:op 874 completed, throughput=4.99MB/sec 2024-04-07T16:41:29.084 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-HAnfO3ABQRCBTO_ off=7421063 len=1145063 2024-04-07T16:41:29.087 INFO:tasks.workunit.client.0.smithi100.stdout:op 875 completed, throughput=4.99MB/sec 2024-04-07T16:41:29.087 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-wijL9yiTT29AGAm off=9763975 len=1281951 2024-04-07T16:41:29.097 INFO:tasks.workunit.client.0.smithi100.stdout:op 876 completed, throughput=5MB/sec 2024-04-07T16:41:30.097 INFO:tasks.workunit.client.0.smithi100.stdout: 168: throughput=4.97MB/sec pending data=0 2024-04-07T16:41:30.097 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-yUWFBD35lYBf62k off=3139422 len=750870 2024-04-07T16:41:30.104 INFO:tasks.workunit.client.0.smithi100.stdout:op 877 completed, throughput=4.98MB/sec 2024-04-07T16:41:30.104 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-42LcZcWUn4Piagj off=9433060 len=312641 2024-04-07T16:41:30.106 INFO:tasks.workunit.client.0.smithi100.stdout:op 878 completed, throughput=4.98MB/sec 2024-04-07T16:41:30.106 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-YJUlP5TZX4kmsTI off=4415132 len=1189152 2024-04-07T16:41:30.111 INFO:tasks.workunit.client.0.smithi100.stdout:op 879 completed, throughput=4.98MB/sec 2024-04-07T16:41:30.111 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-JzLohjsiVFo3e3h off=354638 len=1017348 2024-04-07T16:41:30.114 INFO:tasks.workunit.client.0.smithi100.stdout:op 880 completed, throughput=4.99MB/sec 2024-04-07T16:41:30.114 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-YV00HxG-DCWHS28 off=6157554 len=388666 2024-04-07T16:41:30.116 INFO:tasks.workunit.client.0.smithi100.stdout:op 881 completed, throughput=4.99MB/sec 2024-04-07T16:41:30.116 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-dgGKgeIHMr0W1el off=10752682 len=227981 2024-04-07T16:41:30.117 INFO:tasks.workunit.client.0.smithi100.stdout:op 882 completed, throughput=4.99MB/sec 2024-04-07T16:41:30.117 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-2aFzA8SbINc9CYi off=8873597 len=1303809 2024-04-07T16:41:30.121 INFO:tasks.workunit.client.0.smithi100.stdout:op 883 completed, throughput=5MB/sec 2024-04-07T16:41:30.121 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-wwdN-YZNxTmZHWO off=419354 len=1444215 2024-04-07T16:41:30.123 INFO:tasks.workunit.client.0.smithi100.stdout:op 884 completed, throughput=5.01MB/sec 2024-04-07T16:41:30.692 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:41:30 smithi100 bash[74942]: cluster 2024-04-07T16:41:29.168917+0000 mgr.y (mgr.25175) 784 : cluster [DBG] pgmap v1426: 105 pgs: 105 active+clean; 126 GiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 2.4 MiB/s rd, 2.1 MiB/s wr, 4 op/s 2024-04-07T16:41:30.693 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:41:30 smithi100 bash[76202]: cluster 2024-04-07T16:41:29.168917+0000 mgr.y (mgr.25175) 784 : cluster [DBG] pgmap v1426: 105 pgs: 105 active+clean; 126 GiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 2.4 MiB/s rd, 2.1 MiB/s wr, 4 op/s 2024-04-07T16:41:30.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:41:30 smithi144 bash[66654]: cluster 2024-04-07T16:41:29.168917+0000 mgr.y (mgr.25175) 784 : cluster [DBG] pgmap v1426: 105 pgs: 105 active+clean; 126 GiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 2.4 MiB/s rd, 2.1 MiB/s wr, 4 op/s 2024-04-07T16:41:31.124 INFO:tasks.workunit.client.0.smithi100.stdout: 169: throughput=4.98MB/sec pending data=0 2024-04-07T16:41:31.124 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-rxRHQ58NiE907DY off=3340838 len=682806 2024-04-07T16:41:31.126 INFO:tasks.workunit.client.0.smithi100.stdout:op 885 completed, throughput=4.98MB/sec 2024-04-07T16:41:31.126 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Z41Y0F-zvZ-bnfD off=1957531 len=1669534 2024-04-07T16:41:31.139 INFO:tasks.workunit.client.0.smithi100.stdout:op 886 completed, throughput=4.99MB/sec 2024-04-07T16:41:31.139 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-RTbfuVDO82Qb6g4 off=16985143 len=295460 2024-04-07T16:41:31.141 INFO:tasks.workunit.client.0.smithi100.stdout:op 887 completed, throughput=4.99MB/sec 2024-04-07T16:41:31.141 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-NdSoxDf3aAnLqwb off=383391 len=1567995 2024-04-07T16:41:31.146 INFO:tasks.workunit.client.0.smithi100.stdout:op 888 completed, throughput=5MB/sec 2024-04-07T16:41:32.146 INFO:tasks.workunit.client.0.smithi100.stdout: 170: throughput=4.97MB/sec pending data=0 2024-04-07T16:41:32.146 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-zTGuabdP0J0z03g off=10864711 len=64344 2024-04-07T16:41:32.151 INFO:tasks.workunit.client.0.smithi100.stdout:op 889 completed, throughput=4.97MB/sec 2024-04-07T16:41:32.151 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-LsePYUZGwjDPcBI off=3258156 len=1386881 2024-04-07T16:41:32.155 INFO:tasks.workunit.client.0.smithi100.stdout:op 890 completed, throughput=4.98MB/sec 2024-04-07T16:41:32.155 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-iZNiSCEhf7Kss7U off=11135424 len=1320749 2024-04-07T16:41:32.167 INFO:tasks.workunit.client.0.smithi100.stdout:op 891 completed, throughput=4.99MB/sec 2024-04-07T16:41:32.167 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-T0KdmpKjv0j9PO5 off=6928789 len=359982 2024-04-07T16:41:32.173 INFO:tasks.workunit.client.0.smithi100.stdout:op 892 completed, throughput=4.99MB/sec 2024-04-07T16:41:32.173 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-ycKE1dp_wOh72v1 off=15153036 len=2025284 2024-04-07T16:41:32.177 INFO:tasks.workunit.client.0.smithi100.stdout:op 893 completed, throughput=5MB/sec 2024-04-07T16:41:32.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:41:32 smithi144 bash[66654]: cluster 2024-04-07T16:41:31.169888+0000 mgr.y (mgr.25175) 785 : cluster [DBG] pgmap v1427: 105 pgs: 105 active+clean; 126 GiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 2.3 MiB/s rd, 2.4 MiB/s wr, 4 op/s 2024-04-07T16:41:32.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:41:32 smithi144 bash[66654]: audit 2024-04-07T16:41:31.759558+0000 mon.a (mon.0) 2067 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:41:32.608 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:41:32 smithi144 bash[95233]: ts=2024-04-07T16:41:32.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:41:32.694 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:41:32 smithi100 bash[74942]: cluster 2024-04-07T16:41:31.169888+0000 mgr.y (mgr.25175) 785 : cluster [DBG] pgmap v1427: 105 pgs: 105 active+clean; 126 GiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 2.3 MiB/s rd, 2.4 MiB/s wr, 4 op/s 2024-04-07T16:41:32.694 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:41:32 smithi100 bash[74942]: audit 2024-04-07T16:41:31.759558+0000 mon.a (mon.0) 2067 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:41:32.694 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:41:32 smithi100 bash[76202]: cluster 2024-04-07T16:41:31.169888+0000 mgr.y (mgr.25175) 785 : cluster [DBG] pgmap v1427: 105 pgs: 105 active+clean; 126 GiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 2.3 MiB/s rd, 2.4 MiB/s wr, 4 op/s 2024-04-07T16:41:32.694 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:41:32 smithi100 bash[76202]: audit 2024-04-07T16:41:31.759558+0000 mon.a (mon.0) 2067 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:41:33.177 INFO:tasks.workunit.client.0.smithi100.stdout: 171: throughput=4.97MB/sec pending data=0 2024-04-07T16:41:33.178 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-zM_swd7LThHkxBN off=6307474 len=63336 2024-04-07T16:41:33.182 INFO:tasks.workunit.client.0.smithi100.stdout:op 894 completed, throughput=4.97MB/sec 2024-04-07T16:41:33.182 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-mJP8HGQ1WylQEQ1 off=197037 len=1454073 2024-04-07T16:41:33.186 INFO:tasks.workunit.client.0.smithi100.stdout:op 895 completed, throughput=4.98MB/sec 2024-04-07T16:41:33.186 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Kqq5NAVzImwuQkQ off=4426195 len=427691 2024-04-07T16:41:33.188 INFO:tasks.workunit.client.0.smithi100.stdout:op 896 completed, throughput=4.98MB/sec 2024-04-07T16:41:33.188 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-YOlwN_7QSaXG5SD off=1509581 len=764568 2024-04-07T16:41:33.190 INFO:tasks.workunit.client.0.smithi100.stdout:op 897 completed, throughput=4.99MB/sec 2024-04-07T16:41:33.190 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-vKYp7VABDqM3zOu off=5993169 len=345219 2024-04-07T16:41:33.196 INFO:tasks.workunit.client.0.smithi100.stdout:op 898 completed, throughput=4.99MB/sec 2024-04-07T16:41:33.196 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-0OVgRKZN3i3h8pl off=3974180 len=1581618 2024-04-07T16:41:33.199 INFO:tasks.workunit.client.0.smithi100.stdout:op 899 completed, throughput=5MB/sec 2024-04-07T16:41:33.200 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-9zRUPKmDFIQ6v-E off=21263684 len=1394044 2024-04-07T16:41:33.210 INFO:tasks.workunit.client.0.smithi100.stdout:op 900 completed, throughput=5MB/sec 2024-04-07T16:41:34.210 INFO:tasks.workunit.client.0.smithi100.stdout: 172: throughput=4.98MB/sec pending data=0 2024-04-07T16:41:34.210 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-x6hkY8g-0fXe102 off=944143 len=304004 2024-04-07T16:41:34.212 INFO:tasks.workunit.client.0.smithi100.stdout:op 901 completed, throughput=4.98MB/sec 2024-04-07T16:41:34.212 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-d703hG6JyKabnRq off=18783296 len=1467145 2024-04-07T16:41:34.215 INFO:tasks.workunit.client.0.smithi100.stdout:op 902 completed, throughput=4.98MB/sec 2024-04-07T16:41:34.215 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-0US5owbi9Mjc0s1 off=430324 len=2028585 2024-04-07T16:41:34.234 INFO:tasks.workunit.client.0.smithi100.stdout:op 903 completed, throughput=4.99MB/sec 2024-04-07T16:41:34.235 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-LwP-kDJXu-IS3vC off=10866548 len=845872 2024-04-07T16:41:34.238 INFO:tasks.workunit.client.0.smithi100.stdout:op 904 completed, throughput=5MB/sec 2024-04-07T16:41:34.238 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-sTGlDnxsg_PvSFs off=7117162 len=1115455 2024-04-07T16:41:34.249 INFO:tasks.workunit.client.0.smithi100.stdout:op 905 completed, throughput=5.01MB/sec 2024-04-07T16:41:34.692 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:41:34 smithi100 bash[74942]: cluster 2024-04-07T16:41:33.170982+0000 mgr.y (mgr.25175) 786 : cluster [DBG] pgmap v1428: 105 pgs: 105 active+clean; 126 GiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 2.4 MiB/s rd, 2.6 MiB/s wr, 5 op/s 2024-04-07T16:41:34.693 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:41:34 smithi100 bash[76202]: cluster 2024-04-07T16:41:33.170982+0000 mgr.y (mgr.25175) 786 : cluster [DBG] pgmap v1428: 105 pgs: 105 active+clean; 126 GiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 2.4 MiB/s rd, 2.6 MiB/s wr, 5 op/s 2024-04-07T16:41:34.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:41:34 smithi144 bash[66654]: cluster 2024-04-07T16:41:33.170982+0000 mgr.y (mgr.25175) 786 : cluster [DBG] pgmap v1428: 105 pgs: 105 active+clean; 126 GiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 2.4 MiB/s rd, 2.6 MiB/s wr, 5 op/s 2024-04-07T16:41:35.250 INFO:tasks.workunit.client.0.smithi100.stdout: 173: throughput=4.98MB/sec pending data=0 2024-04-07T16:41:35.250 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-MLk1Bpy8JSbntC9 off=1666895 len=961003 2024-04-07T16:41:35.259 INFO:tasks.workunit.client.0.smithi100.stdout:op 906 completed, throughput=4.98MB/sec 2024-04-07T16:41:35.259 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-CjnUqVIbb5xkI7c off=10238787 len=85733 2024-04-07T16:41:35.264 INFO:tasks.workunit.client.0.smithi100.stdout:op 907 completed, throughput=4.98MB/sec 2024-04-07T16:41:35.264 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-aNEMrYLSCWH1yLN off=6103408 len=242036 2024-04-07T16:41:35.269 INFO:tasks.workunit.client.0.smithi100.stdout:op 908 completed, throughput=4.98MB/sec 2024-04-07T16:41:35.269 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Nsz684pBKYHGRS9 off=7725500 len=79633 2024-04-07T16:41:35.274 INFO:tasks.workunit.client.0.smithi100.stdout:op 909 completed, throughput=4.98MB/sec 2024-04-07T16:41:35.274 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-GAELsC27EP0jyb_ off=4910375 len=1290388 2024-04-07T16:41:35.279 INFO:tasks.workunit.client.0.smithi100.stdout:op 910 completed, throughput=4.99MB/sec 2024-04-07T16:41:35.279 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj--zhYfx7gxx8rDIo off=12095558 len=415116 2024-04-07T16:41:35.281 INFO:tasks.workunit.client.0.smithi100.stdout:op 911 completed, throughput=4.99MB/sec 2024-04-07T16:41:35.281 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Vu1kciuLxwpwPsh off=6695286 len=406343 2024-04-07T16:41:35.286 INFO:tasks.workunit.client.0.smithi100.stdout:op 912 completed, throughput=4.99MB/sec 2024-04-07T16:41:35.286 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-ekJHp9zc356s9SH off=9891471 len=258745 2024-04-07T16:41:35.292 INFO:tasks.workunit.client.0.smithi100.stdout:op 913 completed, throughput=5MB/sec 2024-04-07T16:41:35.292 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-GUAZKEQuBkaBkKV off=12455123 len=1456411 2024-04-07T16:41:35.304 INFO:tasks.workunit.client.0.smithi100.stdout:op 914 completed, throughput=5MB/sec 2024-04-07T16:41:36.305 INFO:tasks.workunit.client.0.smithi100.stdout: 174: throughput=4.98MB/sec pending data=0 2024-04-07T16:41:36.305 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-x1aY4CtDWa1-6-H off=4218996 len=2011381 2024-04-07T16:41:36.309 INFO:tasks.workunit.client.0.smithi100.stdout:op 915 completed, throughput=4.99MB/sec 2024-04-07T16:41:36.309 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-lKnVZahMyuM_2Mj off=9121644 len=1964208 2024-04-07T16:41:36.324 INFO:tasks.workunit.client.0.smithi100.stdout:op 916 completed, throughput=5MB/sec 2024-04-07T16:41:36.324 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-6he7gNZYvh90x6x off=2684071 len=911759 2024-04-07T16:41:36.328 INFO:tasks.workunit.client.0.smithi100.stdout:op 917 completed, throughput=5MB/sec 2024-04-07T16:41:36.693 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:41:36 smithi100 bash[74942]: cluster 2024-04-07T16:41:35.171938+0000 mgr.y (mgr.25175) 787 : cluster [DBG] pgmap v1429: 105 pgs: 105 active+clean; 126 GiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 2.5 MiB/s rd, 1.8 MiB/s wr, 4 op/s 2024-04-07T16:41:36.693 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:41:36 smithi100 bash[76202]: cluster 2024-04-07T16:41:35.171938+0000 mgr.y (mgr.25175) 787 : cluster [DBG] pgmap v1429: 105 pgs: 105 active+clean; 126 GiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 2.5 MiB/s rd, 1.8 MiB/s wr, 4 op/s 2024-04-07T16:41:36.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:41:36 smithi144 bash[66654]: cluster 2024-04-07T16:41:35.171938+0000 mgr.y (mgr.25175) 787 : cluster [DBG] pgmap v1429: 105 pgs: 105 active+clean; 126 GiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 2.5 MiB/s rd, 1.8 MiB/s wr, 4 op/s 2024-04-07T16:41:37.329 INFO:tasks.workunit.client.0.smithi100.stdout: 175: throughput=4.97MB/sec pending data=0 2024-04-07T16:41:37.329 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-JdSo-hNpQZfmRiZ off=1563227 len=823076 2024-04-07T16:41:37.331 INFO:tasks.workunit.client.0.smithi100.stdout:op 918 completed, throughput=4.98MB/sec 2024-04-07T16:41:37.331 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-KAeoMu0xVnqqmJu off=2381366 len=1742445 2024-04-07T16:41:37.337 INFO:tasks.workunit.client.0.smithi100.stdout:op 919 completed, throughput=4.99MB/sec 2024-04-07T16:41:37.337 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-aEdbQWveNUZC1vL off=1369042 len=1041090 2024-04-07T16:41:37.340 INFO:tasks.workunit.client.0.smithi100.stdout:op 920 completed, throughput=4.99MB/sec 2024-04-07T16:41:37.340 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-xcGk5YTa5TKg4U9 off=14185013 len=210722 2024-04-07T16:41:37.342 INFO:tasks.workunit.client.0.smithi100.stdout:op 921 completed, throughput=4.99MB/sec 2024-04-07T16:41:37.342 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-7IC3R9tjmOdvnXl off=19000093 len=29694 2024-04-07T16:41:37.342 INFO:tasks.workunit.client.0.smithi100.stdout:op 922 completed, throughput=4.99MB/sec 2024-04-07T16:41:37.343 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-9ny5EgC_GGa2XrL off=15537950 len=1473775 2024-04-07T16:41:37.346 INFO:tasks.workunit.client.0.smithi100.stdout:op 923 completed, throughput=5MB/sec 2024-04-07T16:41:38.347 INFO:tasks.workunit.client.0.smithi100.stdout: 176: throughput=4.97MB/sec pending data=0 2024-04-07T16:41:38.347 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-U16NaFcyXoW7cel off=725743 len=702782 2024-04-07T16:41:38.349 INFO:tasks.workunit.client.0.smithi100.stdout:op 924 completed, throughput=4.98MB/sec 2024-04-07T16:41:38.349 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-sa2MgMYpwV-WJA7 off=12638379 len=1622229 2024-04-07T16:41:38.353 INFO:tasks.workunit.client.0.smithi100.stdout:op 925 completed, throughput=4.99MB/sec 2024-04-07T16:41:38.353 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Vz5cZJS22BUW10_ off=6570246 len=826489 2024-04-07T16:41:38.355 INFO:tasks.workunit.client.0.smithi100.stdout:op 926 completed, throughput=4.99MB/sec 2024-04-07T16:41:38.355 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-f4mNCDb35mqPVWj off=12804592 len=1428561 2024-04-07T16:41:38.359 INFO:tasks.workunit.client.0.smithi100.stdout:op 927 completed, throughput=5MB/sec 2024-04-07T16:41:38.359 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-F9-0_Wz9A5o-UwM off=14698288 len=484017 2024-04-07T16:41:38.361 INFO:tasks.workunit.client.0.smithi100.stdout:op 928 completed, throughput=5MB/sec 2024-04-07T16:41:38.361 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Kvi0c6mz6FaS4C3 off=9271089 len=1456670 2024-04-07T16:41:38.374 INFO:tasks.workunit.client.0.smithi100.stdout:op 929 completed, throughput=5.01MB/sec 2024-04-07T16:41:38.692 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:41:38 smithi100 bash[74942]: cluster 2024-04-07T16:41:37.172994+0000 mgr.y (mgr.25175) 788 : cluster [DBG] pgmap v1430: 105 pgs: 105 active+clean; 126 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 2.8 MiB/s rd, 2.3 MiB/s wr, 5 op/s 2024-04-07T16:41:38.693 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:41:38 smithi100 bash[76202]: cluster 2024-04-07T16:41:37.172994+0000 mgr.y (mgr.25175) 788 : cluster [DBG] pgmap v1430: 105 pgs: 105 active+clean; 126 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 2.8 MiB/s rd, 2.3 MiB/s wr, 5 op/s 2024-04-07T16:41:38.693 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:41:38 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:41:38] "GET /metrics HTTP/1.1" 200 34056 "" "Prometheus/2.43.0" 2024-04-07T16:41:38.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:41:38 smithi144 bash[66654]: cluster 2024-04-07T16:41:37.172994+0000 mgr.y (mgr.25175) 788 : cluster [DBG] pgmap v1430: 105 pgs: 105 active+clean; 126 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 2.8 MiB/s rd, 2.3 MiB/s wr, 5 op/s 2024-04-07T16:41:39.374 INFO:tasks.workunit.client.0.smithi100.stdout: 177: throughput=4.98MB/sec pending data=0 2024-04-07T16:41:39.375 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-HOkuBBD_nlOOaC3 off=1252349 len=55311 2024-04-07T16:41:39.379 INFO:tasks.workunit.client.0.smithi100.stdout:op 930 completed, throughput=4.98MB/sec 2024-04-07T16:41:39.379 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-b6rfdoPRTIGEwlU off=4753546 len=693693 2024-04-07T16:41:39.386 INFO:tasks.workunit.client.0.smithi100.stdout:op 931 completed, throughput=4.98MB/sec 2024-04-07T16:41:39.386 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-uz3aSEc7RY2wLGa off=5339297 len=1580893 2024-04-07T16:41:39.392 INFO:tasks.workunit.client.0.smithi100.stdout:op 932 completed, throughput=4.99MB/sec 2024-04-07T16:41:39.392 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-nUV2drDPKmDRN_0 off=2119570 len=2040795 2024-04-07T16:41:39.408 INFO:tasks.workunit.client.0.smithi100.stdout:op 933 completed, throughput=5MB/sec 2024-04-07T16:41:40.409 INFO:tasks.workunit.client.0.smithi100.stdout: 178: throughput=4.97MB/sec pending data=0 2024-04-07T16:41:40.409 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Whm-SggNVCj8M4j off=2842720 len=1765200 2024-04-07T16:41:40.413 INFO:tasks.workunit.client.0.smithi100.stdout:op 934 completed, throughput=4.98MB/sec 2024-04-07T16:41:40.413 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-yeViE1ZG1Q3udFE off=4336010 len=151112 2024-04-07T16:41:40.414 INFO:tasks.workunit.client.0.smithi100.stdout:op 935 completed, throughput=4.98MB/sec 2024-04-07T16:41:40.414 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-bKX3njfQUUI29Wx off=4948549 len=682043 2024-04-07T16:41:40.422 INFO:tasks.workunit.client.0.smithi100.stdout:op 936 completed, throughput=4.99MB/sec 2024-04-07T16:41:40.422 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-tCTh4fFNRy6ckRs off=4245962 len=500959 2024-04-07T16:41:40.428 INFO:tasks.workunit.client.0.smithi100.stdout:op 937 completed, throughput=4.99MB/sec 2024-04-07T16:41:40.428 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-5ZNeW6bGcfn6I89 off=16845347 len=529669 2024-04-07T16:41:40.429 INFO:tasks.workunit.client.0.smithi100.stdout:op 938 completed, throughput=4.99MB/sec 2024-04-07T16:41:40.430 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-08nMDQzD_4pBTgf off=4217652 len=728549 2024-04-07T16:41:40.433 INFO:tasks.workunit.client.0.smithi100.stdout:op 939 completed, throughput=5MB/sec 2024-04-07T16:41:40.433 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-zYpZLawezFpiDUY off=1434435 len=1912822 2024-04-07T16:41:40.438 INFO:tasks.workunit.client.0.smithi100.stdout:op 940 completed, throughput=5.01MB/sec 2024-04-07T16:41:40.692 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:41:40 smithi100 bash[74942]: cluster 2024-04-07T16:41:39.173855+0000 mgr.y (mgr.25175) 789 : cluster [DBG] pgmap v1431: 105 pgs: 105 active+clean; 126 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 2.9 MiB/s rd, 2.0 MiB/s wr, 5 op/s 2024-04-07T16:41:40.693 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:41:40 smithi100 bash[76202]: cluster 2024-04-07T16:41:39.173855+0000 mgr.y (mgr.25175) 789 : cluster [DBG] pgmap v1431: 105 pgs: 105 active+clean; 126 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 2.9 MiB/s rd, 2.0 MiB/s wr, 5 op/s 2024-04-07T16:41:40.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:41:40 smithi144 bash[66654]: cluster 2024-04-07T16:41:39.173855+0000 mgr.y (mgr.25175) 789 : cluster [DBG] pgmap v1431: 105 pgs: 105 active+clean; 126 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 2.9 MiB/s rd, 2.0 MiB/s wr, 5 op/s 2024-04-07T16:41:41.438 INFO:tasks.workunit.client.0.smithi100.stdout: 179: throughput=4.98MB/sec pending data=0 2024-04-07T16:41:41.438 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-A2-wv1xdnvvcxRD off=6945778 len=48864 2024-04-07T16:41:41.439 INFO:tasks.workunit.client.0.smithi100.stdout:op 941 completed, throughput=4.98MB/sec 2024-04-07T16:41:41.440 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-vVq1KFp0aBz9bcb off=422173 len=686937 2024-04-07T16:41:41.447 INFO:tasks.workunit.client.0.smithi100.stdout:op 942 completed, throughput=4.98MB/sec 2024-04-07T16:41:41.447 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-bwHfZBspmsxAIEU off=4503142 len=1305775 2024-04-07T16:41:41.457 INFO:tasks.workunit.client.0.smithi100.stdout:op 943 completed, throughput=4.99MB/sec 2024-04-07T16:41:41.457 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-exGd4mAQ8bbI0L0 off=3376575 len=878614 2024-04-07T16:41:41.465 INFO:tasks.workunit.client.0.smithi100.stdout:op 944 completed, throughput=4.99MB/sec 2024-04-07T16:41:41.465 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-z_d_-ZnTs0lD1Ww off=7071135 len=623152 2024-04-07T16:41:41.472 INFO:tasks.workunit.client.0.smithi100.stdout:op 945 completed, throughput=5MB/sec 2024-04-07T16:41:41.472 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-wyeYMEqku7drH00 off=1988565 len=1683336 2024-04-07T16:41:41.487 INFO:tasks.workunit.client.0.smithi100.stdout:op 946 completed, throughput=5MB/sec 2024-04-07T16:41:42.487 INFO:tasks.workunit.client.0.smithi100.stdout: 180: throughput=4.98MB/sec pending data=0 2024-04-07T16:41:42.488 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-tXs8HaGZAcqXmgV off=2420382 len=1269938 2024-04-07T16:41:42.491 INFO:tasks.workunit.client.0.smithi100.stdout:op 947 completed, throughput=4.98MB/sec 2024-04-07T16:41:42.491 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Mcq8jD9PjKFjYKC off=242295 len=1028051 2024-04-07T16:41:42.495 INFO:tasks.workunit.client.0.smithi100.stdout:op 948 completed, throughput=4.99MB/sec 2024-04-07T16:41:42.495 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-bY2AhN8NcEjymir off=2520917 len=1273950 2024-04-07T16:41:42.506 INFO:tasks.workunit.client.0.smithi100.stdout:op 949 completed, throughput=4.99MB/sec 2024-04-07T16:41:42.506 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-6BPBM0hqw-2is6t off=21403296 len=1990852 2024-04-07T16:41:42.510 INFO:tasks.workunit.client.0.smithi100.stdout:op 950 completed, throughput=5.01MB/sec 2024-04-07T16:41:42.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:41:42 smithi144 bash[66654]: cluster 2024-04-07T16:41:41.174780+0000 mgr.y (mgr.25175) 790 : cluster [DBG] pgmap v1432: 105 pgs: 105 active+clean; 126 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 3.1 MiB/s rd, 2.1 MiB/s wr, 5 op/s 2024-04-07T16:41:42.607 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:41:42 smithi144 bash[95233]: ts=2024-04-07T16:41:42.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:41:42.692 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:41:42 smithi100 bash[74942]: cluster 2024-04-07T16:41:41.174780+0000 mgr.y (mgr.25175) 790 : cluster [DBG] pgmap v1432: 105 pgs: 105 active+clean; 126 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 3.1 MiB/s rd, 2.1 MiB/s wr, 5 op/s 2024-04-07T16:41:42.693 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:41:42 smithi100 bash[76202]: cluster 2024-04-07T16:41:41.174780+0000 mgr.y (mgr.25175) 790 : cluster [DBG] pgmap v1432: 105 pgs: 105 active+clean; 126 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 3.1 MiB/s rd, 2.1 MiB/s wr, 5 op/s 2024-04-07T16:41:43.511 INFO:tasks.workunit.client.0.smithi100.stdout: 181: throughput=4.98MB/sec pending data=0 2024-04-07T16:41:43.511 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-puxIJNL26htvSWp off=1595705 len=1041940 2024-04-07T16:41:43.520 INFO:tasks.workunit.client.0.smithi100.stdout:op 951 completed, throughput=4.98MB/sec 2024-04-07T16:41:43.520 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-U6_H7zI7PWJOtuT off=5129396 len=1612591 2024-04-07T16:41:43.532 INFO:tasks.workunit.client.0.smithi100.stdout:op 952 completed, throughput=4.99MB/sec 2024-04-07T16:41:43.532 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-O96gW30fx9fMAu_ off=1655916 len=1375971 2024-04-07T16:41:43.537 INFO:tasks.workunit.client.0.smithi100.stdout:op 953 completed, throughput=5MB/sec 2024-04-07T16:41:43.537 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-F7GLeBXhgY6Nyjd off=668595 len=609805 2024-04-07T16:41:43.539 INFO:tasks.workunit.client.0.smithi100.stdout:op 954 completed, throughput=5MB/sec 2024-04-07T16:41:44.540 INFO:tasks.workunit.client.0.smithi100.stdout: 182: throughput=4.97MB/sec pending data=0 2024-04-07T16:41:44.540 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-whUGmqTllmfa4KN off=191391 len=1977766 2024-04-07T16:41:44.546 INFO:tasks.workunit.client.0.smithi100.stdout:op 955 completed, throughput=4.98MB/sec 2024-04-07T16:41:44.546 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-FsiVouXLu3yZREP off=1611694 len=1341534 2024-04-07T16:41:44.553 INFO:tasks.workunit.client.0.smithi100.stdout:op 956 completed, throughput=4.99MB/sec 2024-04-07T16:41:44.554 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-YlSgZtZlW_W2dFT off=5035623 len=431801 2024-04-07T16:41:44.554 INFO:tasks.workunit.client.0.smithi100.stdout:op 957 completed, throughput=4.99MB/sec 2024-04-07T16:41:44.554 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-tn4xBpaYhj1-PrY off=756670 len=16774 2024-04-07T16:41:44.554 INFO:tasks.workunit.client.0.smithi100.stdout:op 958 completed, throughput=4.99MB/sec 2024-04-07T16:41:44.554 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-h8Nwm15qzRQstzj off=18789418 len=1985864 2024-04-07T16:41:44.559 INFO:tasks.workunit.client.0.smithi100.stdout:op 959 completed, throughput=5MB/sec 2024-04-07T16:41:44.692 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:41:44 smithi100 bash[74942]: cluster 2024-04-07T16:41:43.175915+0000 mgr.y (mgr.25175) 791 : cluster [DBG] pgmap v1433: 105 pgs: 105 active+clean; 126 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 3.5 MiB/s rd, 2.3 MiB/s wr, 6 op/s 2024-04-07T16:41:44.692 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:41:44 smithi100 bash[76202]: cluster 2024-04-07T16:41:43.175915+0000 mgr.y (mgr.25175) 791 : cluster [DBG] pgmap v1433: 105 pgs: 105 active+clean; 126 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 3.5 MiB/s rd, 2.3 MiB/s wr, 6 op/s 2024-04-07T16:41:44.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:41:44 smithi144 bash[66654]: cluster 2024-04-07T16:41:43.175915+0000 mgr.y (mgr.25175) 791 : cluster [DBG] pgmap v1433: 105 pgs: 105 active+clean; 126 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 3.5 MiB/s rd, 2.3 MiB/s wr, 6 op/s 2024-04-07T16:41:45.559 INFO:tasks.workunit.client.0.smithi100.stdout: 183: throughput=4.98MB/sec pending data=0 2024-04-07T16:41:45.559 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-_AEmcFW_kMZBDrn off=21077035 len=565874 2024-04-07T16:41:45.561 INFO:tasks.workunit.client.0.smithi100.stdout:op 960 completed, throughput=4.98MB/sec 2024-04-07T16:41:45.561 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-SDHzCZWGwMIY-ZC off=6368178 len=1335562 2024-04-07T16:41:45.564 INFO:tasks.workunit.client.0.smithi100.stdout:op 961 completed, throughput=4.99MB/sec 2024-04-07T16:41:45.565 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-br_VijxPD3X_xN1 off=3184543 len=1399473 2024-04-07T16:41:45.568 INFO:tasks.workunit.client.0.smithi100.stdout:op 962 completed, throughput=4.99MB/sec 2024-04-07T16:41:45.568 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-wi8_MtYmOjYCwqj off=2316650 len=529604 2024-04-07T16:41:45.569 INFO:tasks.workunit.client.0.smithi100.stdout:op 963 completed, throughput=5MB/sec 2024-04-07T16:41:45.569 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-D9mKZftJFAFjamk off=6928593 len=851322 2024-04-07T16:41:45.579 INFO:tasks.workunit.client.0.smithi100.stdout:op 964 completed, throughput=5MB/sec 2024-04-07T16:41:45.579 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-cVbQWIolhjgbo-O off=226311 len=471318 2024-04-07T16:41:45.580 INFO:tasks.workunit.client.0.smithi100.stdout:op 965 completed, throughput=5MB/sec 2024-04-07T16:41:46.580 INFO:tasks.workunit.client.0.smithi100.stdout: 184: throughput=4.98MB/sec pending data=0 2024-04-07T16:41:46.581 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-zAl6W45rnuj85N6 off=6621253 len=559698 2024-04-07T16:41:46.588 INFO:tasks.workunit.client.0.smithi100.stdout:op 966 completed, throughput=4.98MB/sec 2024-04-07T16:41:46.588 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-cyEWa5IocXvRYRN off=716839 len=27158 2024-04-07T16:41:46.592 INFO:tasks.workunit.client.0.smithi100.stdout:op 967 completed, throughput=4.98MB/sec 2024-04-07T16:41:46.592 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj--lkx4ApR9D3lkDi off=10995298 len=352369 2024-04-07T16:41:46.598 INFO:tasks.workunit.client.0.smithi100.stdout:op 968 completed, throughput=4.98MB/sec 2024-04-07T16:41:46.598 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-T902c4i3e-Mnh3Z off=3381254 len=1149515 2024-04-07T16:41:46.601 INFO:tasks.workunit.client.0.smithi100.stdout:op 969 completed, throughput=4.99MB/sec 2024-04-07T16:41:46.601 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-S_Uje0AzHNYGmJ7 off=4830845 len=27892 2024-04-07T16:41:46.605 INFO:tasks.workunit.client.0.smithi100.stdout:op 970 completed, throughput=4.99MB/sec 2024-04-07T16:41:46.605 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-AES6KSd582qf7Ao off=10757925 len=1466642 2024-04-07T16:41:46.616 INFO:tasks.workunit.client.0.smithi100.stdout:op 971 completed, throughput=4.99MB/sec 2024-04-07T16:41:46.616 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-0SkfILSHkJldhOL off=5221454 len=755778 2024-04-07T16:41:46.619 INFO:tasks.workunit.client.0.smithi100.stdout:op 972 completed, throughput=5MB/sec 2024-04-07T16:41:46.619 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-T5qLORVR5Bej5xI off=2111442 len=241627 2024-04-07T16:41:46.620 INFO:tasks.workunit.client.0.smithi100.stdout:op 973 completed, throughput=5MB/sec 2024-04-07T16:41:46.620 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-wzok_oFonky9c6R off=1820387 len=1716660 2024-04-07T16:41:46.626 INFO:tasks.workunit.client.0.smithi100.stdout:op 974 completed, throughput=5.01MB/sec 2024-04-07T16:41:46.692 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:41:46 smithi100 bash[74942]: cluster 2024-04-07T16:41:45.176849+0000 mgr.y (mgr.25175) 792 : cluster [DBG] pgmap v1434: 105 pgs: 105 active+clean; 126 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 3.0 MiB/s rd, 1.9 MiB/s wr, 5 op/s 2024-04-07T16:41:46.693 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:41:46 smithi100 bash[76202]: cluster 2024-04-07T16:41:45.176849+0000 mgr.y (mgr.25175) 792 : cluster [DBG] pgmap v1434: 105 pgs: 105 active+clean; 126 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 3.0 MiB/s rd, 1.9 MiB/s wr, 5 op/s 2024-04-07T16:41:46.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:41:46 smithi144 bash[66654]: cluster 2024-04-07T16:41:45.176849+0000 mgr.y (mgr.25175) 792 : cluster [DBG] pgmap v1434: 105 pgs: 105 active+clean; 126 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 3.0 MiB/s rd, 1.9 MiB/s wr, 5 op/s 2024-04-07T16:41:47.626 INFO:tasks.workunit.client.0.smithi100.stdout: 185: throughput=4.98MB/sec pending data=0 2024-04-07T16:41:47.627 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-WJCu823zJAmG2Ox off=10566128 len=35025 2024-04-07T16:41:47.628 INFO:tasks.workunit.client.0.smithi100.stdout:op 975 completed, throughput=4.98MB/sec 2024-04-07T16:41:47.628 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-ZbC55OdO-FEHzuF off=1329316 len=1355750 2024-04-07T16:41:47.639 INFO:tasks.workunit.client.0.smithi100.stdout:op 976 completed, throughput=4.99MB/sec 2024-04-07T16:41:47.639 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-9Vy-NgxDIqXI9An off=989433 len=1219990 2024-04-07T16:41:47.641 INFO:tasks.workunit.client.0.smithi100.stdout:op 977 completed, throughput=4.99MB/sec 2024-04-07T16:41:47.642 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-xGi6XrGSI1mDEPj off=1982622 len=673276 2024-04-07T16:41:47.648 INFO:tasks.workunit.client.0.smithi100.stdout:op 978 completed, throughput=5MB/sec 2024-04-07T16:41:47.648 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-EN6bqR0B6b5dFx5 off=828264 len=697497 2024-04-07T16:41:47.655 INFO:tasks.workunit.client.0.smithi100.stdout:op 979 completed, throughput=5MB/sec 2024-04-07T16:41:47.655 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-FkkckJjkw_PWO66 off=1739861 len=1099489 2024-04-07T16:41:47.663 INFO:tasks.workunit.client.0.smithi100.stdout:op 980 completed, throughput=5MB/sec 2024-04-07T16:41:47.692 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:41:47 smithi100 bash[74942]: audit 2024-04-07T16:41:46.759838+0000 mon.a (mon.0) 2068 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:41:47.692 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:41:47 smithi100 bash[76202]: audit 2024-04-07T16:41:46.759838+0000 mon.a (mon.0) 2068 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:41:47.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:41:47 smithi144 bash[66654]: audit 2024-04-07T16:41:46.759838+0000 mon.a (mon.0) 2068 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:41:48.664 INFO:tasks.workunit.client.0.smithi100.stdout: 186: throughput=4.98MB/sec pending data=0 2024-04-07T16:41:48.664 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-zDIBt47d6CmoIH9 off=0 len=1783364 2024-04-07T16:41:48.680 INFO:tasks.workunit.client.0.smithi100.stdout:op 981 completed, throughput=4.99MB/sec 2024-04-07T16:41:48.680 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-djkToJ8hwLtMqRK off=6975892 len=709213 2024-04-07T16:41:48.682 INFO:tasks.workunit.client.0.smithi100.stdout:op 982 completed, throughput=4.99MB/sec 2024-04-07T16:41:48.682 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-0ryPCrbqiUNyOqw off=2429441 len=801642 2024-04-07T16:41:48.690 INFO:tasks.workunit.client.0.smithi100.stdout:op 983 completed, throughput=4.99MB/sec 2024-04-07T16:41:48.690 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-gb4zYBwN6Y2HO_D off=8442742 len=2060600 2024-04-07T16:41:48.692 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:41:48 smithi100 bash[74942]: cluster 2024-04-07T16:41:47.177982+0000 mgr.y (mgr.25175) 793 : cluster [DBG] pgmap v1435: 105 pgs: 105 active+clean; 126 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 3.2 MiB/s rd, 1.9 MiB/s wr, 5 op/s 2024-04-07T16:41:48.693 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:41:48 smithi100 bash[76202]: cluster 2024-04-07T16:41:47.177982+0000 mgr.y (mgr.25175) 793 : cluster [DBG] pgmap v1435: 105 pgs: 105 active+clean; 126 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 3.2 MiB/s rd, 1.9 MiB/s wr, 5 op/s 2024-04-07T16:41:48.693 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:41:48 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:41:48] "GET /metrics HTTP/1.1" 200 34054 "" "Prometheus/2.43.0" 2024-04-07T16:41:48.705 INFO:tasks.workunit.client.0.smithi100.stdout:op 984 completed, throughput=5MB/sec 2024-04-07T16:41:48.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:41:48 smithi144 bash[66654]: cluster 2024-04-07T16:41:47.177982+0000 mgr.y (mgr.25175) 793 : cluster [DBG] pgmap v1435: 105 pgs: 105 active+clean; 126 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 3.2 MiB/s rd, 1.9 MiB/s wr, 5 op/s 2024-04-07T16:41:49.706 INFO:tasks.workunit.client.0.smithi100.stdout: 187: throughput=4.98MB/sec pending data=0 2024-04-07T16:41:49.706 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-1x5Q6pTOo2PfnDB off=357502 len=82631 2024-04-07T16:41:49.707 INFO:tasks.workunit.client.0.smithi100.stdout:op 985 completed, throughput=4.98MB/sec 2024-04-07T16:41:49.707 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-2oFEvJ3k0TQIxBx off=509305 len=597855 2024-04-07T16:41:49.710 INFO:tasks.workunit.client.0.smithi100.stdout:op 986 completed, throughput=4.98MB/sec 2024-04-07T16:41:49.710 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Rb4LWfNRjvv-IdH off=5501073 len=193748 2024-04-07T16:41:49.711 INFO:tasks.workunit.client.0.smithi100.stdout:op 987 completed, throughput=4.98MB/sec 2024-04-07T16:41:49.711 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-1EhBE_lzkOmpTDT off=3696504 len=194206 2024-04-07T16:41:49.717 INFO:tasks.workunit.client.0.smithi100.stdout:op 988 completed, throughput=4.98MB/sec 2024-04-07T16:41:49.717 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-g4Y1XIdt-5tQ5si off=16851783 len=147010 2024-04-07T16:41:49.718 INFO:tasks.workunit.client.0.smithi100.stdout:op 989 completed, throughput=4.98MB/sec 2024-04-07T16:41:49.718 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-C1__mGFgtnoyZgb off=8549671 len=1096907 2024-04-07T16:41:49.721 INFO:tasks.workunit.client.0.smithi100.stdout:op 990 completed, throughput=4.99MB/sec 2024-04-07T16:41:49.721 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-W9tOcrEsZGbHatp off=11410894 len=1051129 2024-04-07T16:41:49.724 INFO:tasks.workunit.client.0.smithi100.stdout:op 991 completed, throughput=4.99MB/sec 2024-04-07T16:41:49.724 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-fHJ59I8eqGnFjuE off=3999419 len=1271669 2024-04-07T16:41:49.727 INFO:tasks.workunit.client.0.smithi100.stdout:op 992 completed, throughput=5MB/sec 2024-04-07T16:41:50.692 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:41:50 smithi100 bash[76202]: cluster 2024-04-07T16:41:49.178858+0000 mgr.y (mgr.25175) 794 : cluster [DBG] pgmap v1436: 105 pgs: 105 active+clean; 126 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 3.1 MiB/s rd, 1.2 MiB/s wr, 4 op/s 2024-04-07T16:41:50.692 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:41:50 smithi100 bash[76202]: audit 2024-04-07T16:41:50.030068+0000 mon.a (mon.0) 2069 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:41:50.693 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:41:50 smithi100 bash[74942]: cluster 2024-04-07T16:41:49.178858+0000 mgr.y (mgr.25175) 794 : cluster [DBG] pgmap v1436: 105 pgs: 105 active+clean; 126 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 3.1 MiB/s rd, 1.2 MiB/s wr, 4 op/s 2024-04-07T16:41:50.693 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:41:50 smithi100 bash[74942]: audit 2024-04-07T16:41:50.030068+0000 mon.a (mon.0) 2069 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:41:50.727 INFO:tasks.workunit.client.0.smithi100.stdout: 188: throughput=4.97MB/sec pending data=0 2024-04-07T16:41:50.727 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-YSIJtyrS7JUc1l8 off=780426 len=1496513 2024-04-07T16:41:50.731 INFO:tasks.workunit.client.0.smithi100.stdout:op 993 completed, throughput=4.98MB/sec 2024-04-07T16:41:50.731 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-UMNlVq1PL3TO22E off=16721602 len=1920111 2024-04-07T16:41:50.735 INFO:tasks.workunit.client.0.smithi100.stdout:op 994 completed, throughput=4.99MB/sec 2024-04-07T16:41:50.735 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-K7U_5RQO6qWFDlM off=9372083 len=687997 2024-04-07T16:41:50.737 INFO:tasks.workunit.client.0.smithi100.stdout:op 995 completed, throughput=4.99MB/sec 2024-04-07T16:41:50.737 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-qyRr2X4b2eAyBLN off=1731620 len=1278037 2024-04-07T16:41:50.746 INFO:tasks.workunit.client.0.smithi100.stdout:op 996 completed, throughput=5MB/sec 2024-04-07T16:41:50.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:41:50 smithi144 bash[66654]: cluster 2024-04-07T16:41:49.178858+0000 mgr.y (mgr.25175) 794 : cluster [DBG] pgmap v1436: 105 pgs: 105 active+clean; 126 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 3.1 MiB/s rd, 1.2 MiB/s wr, 4 op/s 2024-04-07T16:41:50.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:41:50 smithi144 bash[66654]: audit 2024-04-07T16:41:50.030068+0000 mon.a (mon.0) 2069 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:41:51.693 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:41:51 smithi100 bash[74942]: audit 2024-04-07T16:41:50.758203+0000 mon.a (mon.0) 2070 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:41:51.693 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:41:51 smithi100 bash[74942]: audit 2024-04-07T16:41:50.759736+0000 mon.a (mon.0) 2071 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:41:51.693 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:41:51 smithi100 bash[74942]: audit 2024-04-07T16:41:50.767002+0000 mon.a (mon.0) 2072 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:41:51.693 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:41:51 smithi100 bash[76202]: audit 2024-04-07T16:41:50.758203+0000 mon.a (mon.0) 2070 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:41:51.693 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:41:51 smithi100 bash[76202]: audit 2024-04-07T16:41:50.759736+0000 mon.a (mon.0) 2071 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:41:51.693 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:41:51 smithi100 bash[76202]: audit 2024-04-07T16:41:50.767002+0000 mon.a (mon.0) 2072 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:41:51.746 INFO:tasks.workunit.client.0.smithi100.stdout: 189: throughput=4.97MB/sec pending data=0 2024-04-07T16:41:51.747 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-mK-P26BPIwlJDMm off=6374507 len=899995 2024-04-07T16:41:51.755 INFO:tasks.workunit.client.0.smithi100.stdout:op 997 completed, throughput=4.98MB/sec 2024-04-07T16:41:51.756 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-fir-TcbrKIesfye off=8027 len=1339273 2024-04-07T16:41:51.759 INFO:tasks.workunit.client.0.smithi100.stdout:op 998 completed, throughput=4.99MB/sec 2024-04-07T16:41:51.759 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-lVMd5lDl_OUrVc8 off=425140 len=151260 2024-04-07T16:41:51.761 INFO:tasks.workunit.client.0.smithi100.stdout:op 999 completed, throughput=4.99MB/sec 2024-04-07T16:41:51.761 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-L8ihh-Kx5gpMIVw off=7139574 len=1711024 2024-04-07T16:41:51.764 INFO:tasks.workunit.client.0.smithi100.stdout:op 1000 completed, throughput=4.99MB/sec 2024-04-07T16:41:51.764 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-XUaa7u7DLrZDFX_ off=1037099 len=2042164 2024-04-07T16:41:51.778 INFO:tasks.workunit.client.0.smithi100.stdout:op 1001 completed, throughput=5MB/sec 2024-04-07T16:41:51.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:41:51 smithi144 bash[66654]: audit 2024-04-07T16:41:50.758203+0000 mon.a (mon.0) 2070 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:41:51.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:41:51 smithi144 bash[66654]: audit 2024-04-07T16:41:50.759736+0000 mon.a (mon.0) 2071 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:41:51.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:41:51 smithi144 bash[66654]: audit 2024-04-07T16:41:50.767002+0000 mon.a (mon.0) 2072 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:41:52.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:41:52 smithi144 bash[66654]: cluster 2024-04-07T16:41:51.179891+0000 mgr.y (mgr.25175) 795 : cluster [DBG] pgmap v1437: 105 pgs: 105 active+clean; 126 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 3.1 MiB/s rd, 1.8 MiB/s wr, 5 op/s 2024-04-07T16:41:52.607 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:41:52 smithi144 bash[95233]: ts=2024-04-07T16:41:52.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:41:52.692 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:41:52 smithi100 bash[74942]: cluster 2024-04-07T16:41:51.179891+0000 mgr.y (mgr.25175) 795 : cluster [DBG] pgmap v1437: 105 pgs: 105 active+clean; 126 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 3.1 MiB/s rd, 1.8 MiB/s wr, 5 op/s 2024-04-07T16:41:52.692 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:41:52 smithi100 bash[76202]: cluster 2024-04-07T16:41:51.179891+0000 mgr.y (mgr.25175) 795 : cluster [DBG] pgmap v1437: 105 pgs: 105 active+clean; 126 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 3.1 MiB/s rd, 1.8 MiB/s wr, 5 op/s 2024-04-07T16:41:52.779 INFO:tasks.workunit.client.0.smithi100.stdout: 190: throughput=4.98MB/sec pending data=0 2024-04-07T16:41:52.779 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-4TM5WaaTF8bR8rG off=13938274 len=1978839 2024-04-07T16:41:52.783 INFO:tasks.workunit.client.0.smithi100.stdout:op 1002 completed, throughput=4.99MB/sec 2024-04-07T16:41:52.783 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-h7bnsrLCe9F3hOB off=9707454 len=764968 2024-04-07T16:41:52.791 INFO:tasks.workunit.client.0.smithi100.stdout:op 1003 completed, throughput=4.99MB/sec 2024-04-07T16:41:52.791 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-UdYc76fZSY9JH49 off=1615014 len=287306 2024-04-07T16:41:52.793 INFO:tasks.workunit.client.0.smithi100.stdout:op 1004 completed, throughput=4.99MB/sec 2024-04-07T16:41:52.793 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-VusNmfwhcPK5lCf off=2732195 len=121954 2024-04-07T16:41:52.797 INFO:tasks.workunit.client.0.smithi100.stdout:op 1005 completed, throughput=4.99MB/sec 2024-04-07T16:41:52.797 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-aYRu7GsL0pZxjY6 off=2133812 len=324732 2024-04-07T16:41:52.800 INFO:tasks.workunit.client.0.smithi100.stdout:op 1006 completed, throughput=4.99MB/sec 2024-04-07T16:41:52.800 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-LSBKSE9rRgVHTx- off=1720569 len=1969805 2024-04-07T16:41:52.802 INFO:tasks.workunit.client.0.smithi100.stdout:op 1007 completed, throughput=5MB/sec 2024-04-07T16:41:53.803 INFO:tasks.workunit.client.0.smithi100.stdout: 191: throughput=4.98MB/sec pending data=0 2024-04-07T16:41:53.803 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-LAr_nkoW8W0UCap off=11378098 len=1284836 2024-04-07T16:41:53.814 INFO:tasks.workunit.client.0.smithi100.stdout:op 1008 completed, throughput=4.98MB/sec 2024-04-07T16:41:53.814 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-9XX18qLpvndmm5X off=1068529 len=981323 2024-04-07T16:41:53.816 INFO:tasks.workunit.client.0.smithi100.stdout:op 1009 completed, throughput=4.99MB/sec 2024-04-07T16:41:53.816 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-lvaZyNWX6LBDePr off=6839619 len=1059941 2024-04-07T16:41:53.825 INFO:tasks.workunit.client.0.smithi100.stdout:op 1010 completed, throughput=4.99MB/sec 2024-04-07T16:41:53.825 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Pzb6MTna1sW8xQv off=2180252 len=345817 2024-04-07T16:41:53.826 INFO:tasks.workunit.client.0.smithi100.stdout:op 1011 completed, throughput=5MB/sec 2024-04-07T16:41:53.826 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-ZIBpS_NoSyrwqBn off=4223287 len=325734 2024-04-07T16:41:53.832 INFO:tasks.workunit.client.0.smithi100.stdout:op 1012 completed, throughput=5MB/sec 2024-04-07T16:41:53.832 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-z6ESHkpxE_srxj9 off=235203 len=986417 2024-04-07T16:41:53.840 INFO:tasks.workunit.client.0.smithi100.stdout:op 1013 completed, throughput=5MB/sec 2024-04-07T16:41:54.693 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:41:54 smithi100 bash[74942]: cluster 2024-04-07T16:41:53.181057+0000 mgr.y (mgr.25175) 796 : cluster [DBG] pgmap v1438: 105 pgs: 105 active+clean; 126 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 3.3 MiB/s rd, 2.0 MiB/s wr, 5 op/s 2024-04-07T16:41:54.693 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:41:54 smithi100 bash[76202]: cluster 2024-04-07T16:41:53.181057+0000 mgr.y (mgr.25175) 796 : cluster [DBG] pgmap v1438: 105 pgs: 105 active+clean; 126 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 3.3 MiB/s rd, 2.0 MiB/s wr, 5 op/s 2024-04-07T16:41:54.840 INFO:tasks.workunit.client.0.smithi100.stdout: 192: throughput=4.98MB/sec pending data=0 2024-04-07T16:41:54.840 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-UJr7Gejpy7n0JrG off=8280290 len=790166 2024-04-07T16:41:54.843 INFO:tasks.workunit.client.0.smithi100.stdout:op 1014 completed, throughput=4.98MB/sec 2024-04-07T16:41:54.843 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-AwQDDqLxetk8bcG off=1663656 len=734631 2024-04-07T16:41:54.845 INFO:tasks.workunit.client.0.smithi100.stdout:op 1015 completed, throughput=4.98MB/sec 2024-04-07T16:41:54.845 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-DkvlKhW2XWqjrCS off=2531421 len=203955 2024-04-07T16:41:54.850 INFO:tasks.workunit.client.0.smithi100.stdout:op 1016 completed, throughput=4.99MB/sec 2024-04-07T16:41:54.851 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-fe1BKVkkvRfF0bl off=2539915 len=1276050 2024-04-07T16:41:54.854 INFO:tasks.workunit.client.0.smithi100.stdout:op 1017 completed, throughput=4.99MB/sec 2024-04-07T16:41:54.854 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-8wfONl6v4d7WQKL off=10982513 len=179560 2024-04-07T16:41:54.855 INFO:tasks.workunit.client.0.smithi100.stdout:op 1018 completed, throughput=4.99MB/sec 2024-04-07T16:41:54.855 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Vn7aY105HuqzXmj off=461810 len=1064157 2024-04-07T16:41:54.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:41:54 smithi144 bash[66654]: cluster 2024-04-07T16:41:53.181057+0000 mgr.y (mgr.25175) 796 : cluster [DBG] pgmap v1438: 105 pgs: 105 active+clean; 126 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 3.3 MiB/s rd, 2.0 MiB/s wr, 5 op/s 2024-04-07T16:41:54.865 INFO:tasks.workunit.client.0.smithi100.stdout:op 1019 completed, throughput=5MB/sec 2024-04-07T16:41:54.865 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-L8xFvRMXO3DSsCp off=1573896 len=98109 2024-04-07T16:41:54.870 INFO:tasks.workunit.client.0.smithi100.stdout:op 1020 completed, throughput=5MB/sec 2024-04-07T16:41:54.870 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-mGzUVwy_rojeFFe off=13005531 len=151720 2024-04-07T16:41:54.874 INFO:tasks.workunit.client.0.smithi100.stdout:op 1021 completed, throughput=5MB/sec 2024-04-07T16:41:54.874 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-r0_m5wU-iWM4eSu off=7269209 len=1256845 2024-04-07T16:41:54.884 INFO:tasks.workunit.client.0.smithi100.stdout:op 1022 completed, throughput=5MB/sec 2024-04-07T16:41:55.885 INFO:tasks.workunit.client.0.smithi100.stdout: 193: throughput=4.98MB/sec pending data=0 2024-04-07T16:41:55.885 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-K2u5IokUUuwLDsX off=6783630 len=961681 2024-04-07T16:41:55.894 INFO:tasks.workunit.client.0.smithi100.stdout:op 1023 completed, throughput=4.98MB/sec 2024-04-07T16:41:55.894 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-8oe3CKtFR5hRL3P off=18188270 len=664809 2024-04-07T16:41:55.897 INFO:tasks.workunit.client.0.smithi100.stdout:op 1024 completed, throughput=4.99MB/sec 2024-04-07T16:41:55.897 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-DIrzZpqdl5aXWG1 off=3491864 len=57081 2024-04-07T16:41:55.898 INFO:tasks.workunit.client.0.smithi100.stdout:op 1025 completed, throughput=4.99MB/sec 2024-04-07T16:41:55.898 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-XLlWaFVnlDYLEvp off=5210118 len=609987 2024-04-07T16:41:55.900 INFO:tasks.workunit.client.0.smithi100.stdout:op 1026 completed, throughput=4.99MB/sec 2024-04-07T16:41:55.900 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-6kudbTUfukRlFS0 off=5326689 len=609395 2024-04-07T16:41:55.902 INFO:tasks.workunit.client.0.smithi100.stdout:op 1027 completed, throughput=4.99MB/sec 2024-04-07T16:41:55.902 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-AWkFO1S5x5AveVD off=178172 len=287659 2024-04-07T16:41:55.903 INFO:tasks.workunit.client.0.smithi100.stdout:op 1028 completed, throughput=4.99MB/sec 2024-04-07T16:41:55.903 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-uB1NmyRrN1qRzTD off=14788529 len=529410 2024-04-07T16:41:55.905 INFO:tasks.workunit.client.0.smithi100.stdout:op 1029 completed, throughput=5MB/sec 2024-04-07T16:41:55.905 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-p657OD9fqF37dr1 off=8262062 len=965244 2024-04-07T16:41:55.907 INFO:tasks.workunit.client.0.smithi100.stdout:op 1030 completed, throughput=5MB/sec 2024-04-07T16:41:56.692 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:41:56 smithi100 bash[74942]: cluster 2024-04-07T16:41:55.182016+0000 mgr.y (mgr.25175) 797 : cluster [DBG] pgmap v1439: 105 pgs: 105 active+clean; 126 GiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 3.3 MiB/s rd, 1.8 MiB/s wr, 5 op/s 2024-04-07T16:41:56.693 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:41:56 smithi100 bash[76202]: cluster 2024-04-07T16:41:55.182016+0000 mgr.y (mgr.25175) 797 : cluster [DBG] pgmap v1439: 105 pgs: 105 active+clean; 126 GiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 3.3 MiB/s rd, 1.8 MiB/s wr, 5 op/s 2024-04-07T16:41:56.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:41:56 smithi144 bash[66654]: cluster 2024-04-07T16:41:55.182016+0000 mgr.y (mgr.25175) 797 : cluster [DBG] pgmap v1439: 105 pgs: 105 active+clean; 126 GiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 3.3 MiB/s rd, 1.8 MiB/s wr, 5 op/s 2024-04-07T16:41:56.907 INFO:tasks.workunit.client.0.smithi100.stdout: 194: throughput=4.98MB/sec pending data=0 2024-04-07T16:41:56.907 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-F-H5lk11iV4J6-1 off=2511165 len=207563 2024-04-07T16:41:56.909 INFO:tasks.workunit.client.0.smithi100.stdout:op 1031 completed, throughput=4.98MB/sec 2024-04-07T16:41:56.909 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-_BdepnrenIROTJz off=4191094 len=998612 2024-04-07T16:41:56.919 INFO:tasks.workunit.client.0.smithi100.stdout:op 1032 completed, throughput=4.98MB/sec 2024-04-07T16:41:56.919 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-UAywZNOBXJVKIsA off=1104016 len=1265812 2024-04-07T16:41:56.922 INFO:tasks.workunit.client.0.smithi100.stdout:op 1033 completed, throughput=4.99MB/sec 2024-04-07T16:41:56.922 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-yl-ZzDpBEpuc3Zb off=238188 len=1390060 2024-04-07T16:41:56.934 INFO:tasks.workunit.client.0.smithi100.stdout:op 1034 completed, throughput=4.99MB/sec 2024-04-07T16:41:56.934 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-U48PfzhOvQtG1d_ off=5313370 len=72315 2024-04-07T16:41:56.938 INFO:tasks.workunit.client.0.smithi100.stdout:op 1035 completed, throughput=4.99MB/sec 2024-04-07T16:41:56.938 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-I6QBJqq6yAe2yAR off=11761557 len=1832095 2024-04-07T16:41:56.944 INFO:tasks.workunit.client.0.smithi100.stdout:op 1036 completed, throughput=5MB/sec 2024-04-07T16:41:57.944 INFO:tasks.workunit.client.0.smithi100.stdout: 195: throughput=4.98MB/sec pending data=0 2024-04-07T16:41:57.945 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-l9e1K9MWIQF73i0 off=3885494 len=1556789 2024-04-07T16:41:57.957 INFO:tasks.workunit.client.0.smithi100.stdout:op 1037 completed, throughput=4.98MB/sec 2024-04-07T16:41:57.957 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-btOkJpq6s5xXX1x off=3499470 len=1767672 2024-04-07T16:41:57.970 INFO:tasks.workunit.client.0.smithi100.stdout:op 1038 completed, throughput=4.99MB/sec 2024-04-07T16:41:57.970 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-SQ7jZTIXSbGP69r off=3125248 len=2019105 2024-04-07T16:41:57.984 INFO:tasks.workunit.client.0.smithi100.stdout:op 1039 completed, throughput=5MB/sec 2024-04-07T16:41:58.692 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:41:58 smithi100 bash[74942]: cluster 2024-04-07T16:41:57.183089+0000 mgr.y (mgr.25175) 798 : cluster [DBG] pgmap v1440: 105 pgs: 105 active+clean; 126 GiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 3.3 MiB/s rd, 2.0 MiB/s wr, 5 op/s 2024-04-07T16:41:58.693 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:41:58 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:41:58] "GET /metrics HTTP/1.1" 200 34054 "" "Prometheus/2.43.0" 2024-04-07T16:41:58.693 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:41:58 smithi100 bash[76202]: cluster 2024-04-07T16:41:57.183089+0000 mgr.y (mgr.25175) 798 : cluster [DBG] pgmap v1440: 105 pgs: 105 active+clean; 126 GiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 3.3 MiB/s rd, 2.0 MiB/s wr, 5 op/s 2024-04-07T16:41:58.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:41:58 smithi144 bash[66654]: cluster 2024-04-07T16:41:57.183089+0000 mgr.y (mgr.25175) 798 : cluster [DBG] pgmap v1440: 105 pgs: 105 active+clean; 126 GiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 3.3 MiB/s rd, 2.0 MiB/s wr, 5 op/s 2024-04-07T16:41:58.985 INFO:tasks.workunit.client.0.smithi100.stdout: 196: throughput=4.98MB/sec pending data=0 2024-04-07T16:41:58.985 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-G6RDDgDLYa3LQwq off=686909 len=881289 2024-04-07T16:41:58.988 INFO:tasks.workunit.client.0.smithi100.stdout:op 1040 completed, throughput=4.98MB/sec 2024-04-07T16:41:58.988 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-d4f-9RvHWtP98cV off=11586535 len=1218819 2024-04-07T16:41:58.991 INFO:tasks.workunit.client.0.smithi100.stdout:op 1041 completed, throughput=4.99MB/sec 2024-04-07T16:41:58.991 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-MEQsLQOKb9I9iv9 off=7863453 len=696972 2024-04-07T16:41:58.998 INFO:tasks.workunit.client.0.smithi100.stdout:op 1042 completed, throughput=4.99MB/sec 2024-04-07T16:41:58.998 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-l12UaaGmxVnH0dY off=9560743 len=1170123 2024-04-07T16:41:59.008 INFO:tasks.workunit.client.0.smithi100.stdout:op 1043 completed, throughput=4.99MB/sec 2024-04-07T16:41:59.008 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-fZe4Alacze-Pw-p off=5074246 len=1135666 2024-04-07T16:41:59.010 INFO:tasks.workunit.client.0.smithi100.stdout:op 1044 completed, throughput=5MB/sec 2024-04-07T16:42:00.010 INFO:tasks.workunit.client.0.smithi100.stdout: 197: throughput=4.98MB/sec pending data=0 2024-04-07T16:42:00.010 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-uEyDLv3_Zi_gq_5 off=4914296 len=1312980 2024-04-07T16:42:00.015 INFO:tasks.workunit.client.0.smithi100.stdout:op 1045 completed, throughput=4.98MB/sec 2024-04-07T16:42:00.015 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-kQPIOhpjFd0XlY9 off=5384706 len=830360 2024-04-07T16:42:00.022 INFO:tasks.workunit.client.0.smithi100.stdout:op 1046 completed, throughput=4.99MB/sec 2024-04-07T16:42:00.023 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-mdnu2O1BqOa7yI- off=3554723 len=1256860 2024-04-07T16:42:00.032 INFO:tasks.workunit.client.0.smithi100.stdout:op 1047 completed, throughput=4.99MB/sec 2024-04-07T16:42:00.032 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-uhlu-6P2TdkdRZs off=11587989 len=740576 2024-04-07T16:42:00.034 INFO:tasks.workunit.client.0.smithi100.stdout:op 1048 completed, throughput=4.99MB/sec 2024-04-07T16:42:00.034 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-2kSsP-LusC46XPv off=4072586 len=1455508 2024-04-07T16:42:00.047 INFO:tasks.workunit.client.0.smithi100.stdout:op 1049 completed, throughput=5MB/sec 2024-04-07T16:42:00.693 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:42:00 smithi100 bash[74942]: cluster 2024-04-07T16:41:59.184016+0000 mgr.y (mgr.25175) 799 : cluster [DBG] pgmap v1441: 105 pgs: 105 active+clean; 126 GiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.1 MiB/s wr, 5 op/s 2024-04-07T16:42:00.693 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:42:00 smithi100 bash[76202]: cluster 2024-04-07T16:41:59.184016+0000 mgr.y (mgr.25175) 799 : cluster [DBG] pgmap v1441: 105 pgs: 105 active+clean; 126 GiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.1 MiB/s wr, 5 op/s 2024-04-07T16:42:00.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:42:00 smithi144 bash[66654]: cluster 2024-04-07T16:41:59.184016+0000 mgr.y (mgr.25175) 799 : cluster [DBG] pgmap v1441: 105 pgs: 105 active+clean; 126 GiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.1 MiB/s wr, 5 op/s 2024-04-07T16:42:01.047 INFO:tasks.workunit.client.0.smithi100.stdout: 198: throughput=4.98MB/sec pending data=0 2024-04-07T16:42:01.048 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-2uCxtNeMx3R7Kyo off=517612 len=1671756 2024-04-07T16:42:01.054 INFO:tasks.workunit.client.0.smithi100.stdout:op 1050 completed, throughput=4.98MB/sec 2024-04-07T16:42:01.054 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-GbMA6frw-6UMB5V off=3839779 len=1125226 2024-04-07T16:42:01.057 INFO:tasks.workunit.client.0.smithi100.stdout:op 1051 completed, throughput=4.99MB/sec 2024-04-07T16:42:01.057 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-TQv4QNpWQvNAgDB off=2757524 len=1770279 2024-04-07T16:42:01.062 INFO:tasks.workunit.client.0.smithi100.stdout:op 1052 completed, throughput=5MB/sec 2024-04-07T16:42:01.062 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-7AnZM2QZuB1LSwU off=7257281 len=2037202 2024-04-07T16:42:01.066 INFO:tasks.workunit.client.0.smithi100.stdout:op 1053 completed, throughput=5.01MB/sec 2024-04-07T16:42:02.067 INFO:tasks.workunit.client.0.smithi100.stdout: 199: throughput=4.98MB/sec pending data=0 2024-04-07T16:42:02.067 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-NmdzuwrfFIIE7Ab off=11351179 len=711010 2024-04-07T16:42:02.075 INFO:tasks.workunit.client.0.smithi100.stdout:op 1054 completed, throughput=4.99MB/sec 2024-04-07T16:42:02.075 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-4WflWzpp1Rq6vcq off=2155518 len=1056563 2024-04-07T16:42:02.085 INFO:tasks.workunit.client.0.smithi100.stdout:op 1055 completed, throughput=4.99MB/sec 2024-04-07T16:42:02.085 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-msy8SiH_5igOJmR off=6146734 len=1397887 2024-04-07T16:42:02.097 INFO:tasks.workunit.client.0.smithi100.stdout:op 1056 completed, throughput=5MB/sec 2024-04-07T16:42:02.097 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-sTGlDnxsg_PvSFs off=17423386 len=1657792 2024-04-07T16:42:02.101 INFO:tasks.workunit.client.0.smithi100.stdout:op 1057 completed, throughput=5MB/sec 2024-04-07T16:42:02.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:42:02 smithi144 bash[66654]: cluster 2024-04-07T16:42:01.184989+0000 mgr.y (mgr.25175) 800 : cluster [DBG] pgmap v1442: 105 pgs: 105 active+clean; 126 GiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 2.6 MiB/s rd, 2.3 MiB/s wr, 5 op/s 2024-04-07T16:42:02.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:42:02 smithi144 bash[66654]: audit 2024-04-07T16:42:01.760345+0000 mon.a (mon.0) 2073 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:42:02.607 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:42:02 smithi144 bash[95233]: ts=2024-04-07T16:42:02.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:42:02.692 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:42:02 smithi100 bash[74942]: cluster 2024-04-07T16:42:01.184989+0000 mgr.y (mgr.25175) 800 : cluster [DBG] pgmap v1442: 105 pgs: 105 active+clean; 126 GiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 2.6 MiB/s rd, 2.3 MiB/s wr, 5 op/s 2024-04-07T16:42:02.693 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:42:02 smithi100 bash[74942]: audit 2024-04-07T16:42:01.760345+0000 mon.a (mon.0) 2073 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:42:02.693 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:42:02 smithi100 bash[76202]: cluster 2024-04-07T16:42:01.184989+0000 mgr.y (mgr.25175) 800 : cluster [DBG] pgmap v1442: 105 pgs: 105 active+clean; 126 GiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 2.6 MiB/s rd, 2.3 MiB/s wr, 5 op/s 2024-04-07T16:42:02.693 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:42:02 smithi100 bash[76202]: audit 2024-04-07T16:42:01.760345+0000 mon.a (mon.0) 2073 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:42:03.101 INFO:tasks.workunit.client.0.smithi100.stdout: 200: throughput=4.98MB/sec pending data=0 2024-04-07T16:42:03.102 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-iF9HgR4MKsVYLLH off=6668666 len=1438788 2024-04-07T16:42:03.106 INFO:tasks.workunit.client.0.smithi100.stdout:op 1058 completed, throughput=4.99MB/sec 2024-04-07T16:42:03.106 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Km3Oj37e_6gw1kQ off=20797490 len=67149 2024-04-07T16:42:03.111 INFO:tasks.workunit.client.0.smithi100.stdout:op 1059 completed, throughput=4.99MB/sec 2024-04-07T16:42:03.111 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-0IILJOkewFRNAqL off=17574702 len=1475018 2024-04-07T16:42:03.124 INFO:tasks.workunit.client.0.smithi100.stdout:op 1060 completed, throughput=4.99MB/sec 2024-04-07T16:42:03.125 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-AYZlCQd32m-d36A off=8102338 len=1734492 2024-04-07T16:42:03.139 INFO:tasks.workunit.client.0.smithi100.stdout:op 1061 completed, throughput=5MB/sec 2024-04-07T16:42:04.139 INFO:tasks.workunit.client.0.smithi100.stdout: 201: throughput=4.98MB/sec pending data=0 2024-04-07T16:42:04.139 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-KVJouVwiGS_W4w5 off=2497554 len=1267433 2024-04-07T16:42:04.143 INFO:tasks.workunit.client.0.smithi100.stdout:op 1062 completed, throughput=4.98MB/sec 2024-04-07T16:42:04.143 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Rhil2D33D2XH9jV off=15555254 len=189804 2024-04-07T16:42:04.148 INFO:tasks.workunit.client.0.smithi100.stdout:op 1063 completed, throughput=4.98MB/sec 2024-04-07T16:42:04.148 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-34ZMEH_iluOC47a off=2226100 len=126894 2024-04-07T16:42:04.149 INFO:tasks.workunit.client.0.smithi100.stdout:op 1064 completed, throughput=4.98MB/sec 2024-04-07T16:42:04.149 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-WeIlCWFHI9jnTc7 off=5850914 len=750044 2024-04-07T16:42:04.158 INFO:tasks.workunit.client.0.smithi100.stdout:op 1065 completed, throughput=4.99MB/sec 2024-04-07T16:42:04.158 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-WWeyQgkYZuqbE8N off=17925531 len=2035642 2024-04-07T16:42:04.162 INFO:tasks.workunit.client.0.smithi100.stdout:op 1066 completed, throughput=5MB/sec 2024-04-07T16:42:04.162 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-T9Cl_mN7lUzmi2d off=15404409 len=1814032 2024-04-07T16:42:04.176 INFO:tasks.workunit.client.0.smithi100.stdout:op 1067 completed, throughput=5MB/sec 2024-04-07T16:42:04.693 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:42:04 smithi100 bash[74942]: cluster 2024-04-07T16:42:03.186242+0000 mgr.y (mgr.25175) 801 : cluster [DBG] pgmap v1443: 105 pgs: 105 active+clean; 126 GiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 3.0 MiB/s rd, 2.3 MiB/s wr, 5 op/s 2024-04-07T16:42:04.693 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:42:04 smithi100 bash[76202]: cluster 2024-04-07T16:42:03.186242+0000 mgr.y (mgr.25175) 801 : cluster [DBG] pgmap v1443: 105 pgs: 105 active+clean; 126 GiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 3.0 MiB/s rd, 2.3 MiB/s wr, 5 op/s 2024-04-07T16:42:04.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:42:04 smithi144 bash[66654]: cluster 2024-04-07T16:42:03.186242+0000 mgr.y (mgr.25175) 801 : cluster [DBG] pgmap v1443: 105 pgs: 105 active+clean; 126 GiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 3.0 MiB/s rd, 2.3 MiB/s wr, 5 op/s 2024-04-07T16:42:05.177 INFO:tasks.workunit.client.0.smithi100.stdout: 202: throughput=4.98MB/sec pending data=0 2024-04-07T16:42:05.177 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-fdhWrdET9IiWIdx off=2873885 len=136048 2024-04-07T16:42:05.182 INFO:tasks.workunit.client.0.smithi100.stdout:op 1068 completed, throughput=4.98MB/sec 2024-04-07T16:42:05.182 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-x5FAn4GNeW1tPI4 off=1635457 len=1237723 2024-04-07T16:42:05.185 INFO:tasks.workunit.client.0.smithi100.stdout:op 1069 completed, throughput=4.99MB/sec 2024-04-07T16:42:05.185 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-yhQ-LNI-LJHOodz off=20040789 len=362986 2024-04-07T16:42:05.187 INFO:tasks.workunit.client.0.smithi100.stdout:op 1070 completed, throughput=4.99MB/sec 2024-04-07T16:42:05.187 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-VHJri1q3Ufoir0b off=7501707 len=1729625 2024-04-07T16:42:05.192 INFO:tasks.workunit.client.0.smithi100.stdout:op 1071 completed, throughput=5MB/sec 2024-04-07T16:42:05.192 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Sv6zNIqRaUSoWRV off=5282721 len=1989275 2024-04-07T16:42:05.196 INFO:tasks.workunit.client.0.smithi100.stdout:op 1072 completed, throughput=5MB/sec 2024-04-07T16:42:06.197 INFO:tasks.workunit.client.0.smithi100.stdout: 203: throughput=4.98MB/sec pending data=0 2024-04-07T16:42:06.197 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-cqHIzs1zPeo-dwp off=234024 len=1501115 2024-04-07T16:42:06.211 INFO:tasks.workunit.client.0.smithi100.stdout:op 1073 completed, throughput=4.99MB/sec 2024-04-07T16:42:06.211 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-6bOEldPsBEjgx0P off=14472157 len=2072554 2024-04-07T16:42:06.226 INFO:tasks.workunit.client.0.smithi100.stdout:op 1074 completed, throughput=5MB/sec 2024-04-07T16:42:06.227 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-TULosKxhQYxGeVu off=1118039 len=664121 2024-04-07T16:42:06.234 INFO:tasks.workunit.client.0.smithi100.stdout:op 1075 completed, throughput=5MB/sec 2024-04-07T16:42:06.234 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-gq-N1Wes0WK-12F off=1843607 len=284640 2024-04-07T16:42:06.236 INFO:tasks.workunit.client.0.smithi100.stdout:op 1076 completed, throughput=5MB/sec 2024-04-07T16:42:06.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:42:06 smithi144 bash[66654]: cluster 2024-04-07T16:42:05.187199+0000 mgr.y (mgr.25175) 802 : cluster [DBG] pgmap v1444: 105 pgs: 105 active+clean; 126 GiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.0 MiB/s wr, 5 op/s 2024-04-07T16:42:06.942 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:42:06 smithi100 bash[74942]: cluster 2024-04-07T16:42:05.187199+0000 mgr.y (mgr.25175) 802 : cluster [DBG] pgmap v1444: 105 pgs: 105 active+clean; 126 GiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.0 MiB/s wr, 5 op/s 2024-04-07T16:42:06.943 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:42:06 smithi100 bash[76202]: cluster 2024-04-07T16:42:05.187199+0000 mgr.y (mgr.25175) 802 : cluster [DBG] pgmap v1444: 105 pgs: 105 active+clean; 126 GiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.0 MiB/s wr, 5 op/s 2024-04-07T16:42:07.236 INFO:tasks.workunit.client.0.smithi100.stdout: 204: throughput=4.98MB/sec pending data=0 2024-04-07T16:42:07.236 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-AvpzBGmCpFcLUwb off=2792567 len=714271 2024-04-07T16:42:07.238 INFO:tasks.workunit.client.0.smithi100.stdout:op 1077 completed, throughput=4.98MB/sec 2024-04-07T16:42:07.239 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-m1qiH8D3MCkF42O off=8117953 len=1854040 2024-04-07T16:42:07.253 INFO:tasks.workunit.client.0.smithi100.stdout:op 1078 completed, throughput=4.99MB/sec 2024-04-07T16:42:07.253 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-KWVwbyS8g_Z6xxG off=9200452 len=1610374 2024-04-07T16:42:07.265 INFO:tasks.workunit.client.0.smithi100.stdout:op 1079 completed, throughput=4.99MB/sec 2024-04-07T16:42:07.265 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-3OWtTJ-bTul2swu off=16262321 len=1735683 2024-04-07T16:42:07.268 INFO:tasks.workunit.client.0.smithi100.stdout:op 1080 completed, throughput=5MB/sec 2024-04-07T16:42:08.268 INFO:tasks.workunit.client.0.smithi100.stdout: 205: throughput=4.98MB/sec pending data=0 2024-04-07T16:42:08.268 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Z70G1uST9s07l_6 off=18574220 len=166715 2024-04-07T16:42:08.273 INFO:tasks.workunit.client.0.smithi100.stdout:op 1081 completed, throughput=4.98MB/sec 2024-04-07T16:42:08.273 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-MWejPoX299wpTDd off=7881289 len=338981 2024-04-07T16:42:08.280 INFO:tasks.workunit.client.0.smithi100.stdout:op 1082 completed, throughput=4.98MB/sec 2024-04-07T16:42:08.280 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-r4Ix0rl3Zih3I9a off=7535750 len=1467967 2024-04-07T16:42:08.291 INFO:tasks.workunit.client.0.smithi100.stdout:op 1083 completed, throughput=4.99MB/sec 2024-04-07T16:42:08.291 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-3bnPlMVFP4hjqkd off=730043 len=1216200 2024-04-07T16:42:08.295 INFO:tasks.workunit.client.0.smithi100.stdout:op 1084 completed, throughput=4.99MB/sec 2024-04-07T16:42:08.295 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-xn-51a6C6Jeo7o7 off=2189219 len=182923 2024-04-07T16:42:08.300 INFO:tasks.workunit.client.0.smithi100.stdout:op 1085 completed, throughput=4.99MB/sec 2024-04-07T16:42:08.300 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-BF-2M59SRLT9Zgy off=8805405 len=83804 2024-04-07T16:42:08.302 INFO:tasks.workunit.client.0.smithi100.stdout:op 1086 completed, throughput=4.99MB/sec 2024-04-07T16:42:08.302 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-2iRnAJv9pnbJb6C off=735784 len=1396178 2024-04-07T16:42:08.304 INFO:tasks.workunit.client.0.smithi100.stdout:op 1087 completed, throughput=5MB/sec 2024-04-07T16:42:08.304 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-BL1fFYWMvv0Uyur off=4073800 len=835716 2024-04-07T16:42:08.307 INFO:tasks.workunit.client.0.smithi100.stdout:op 1088 completed, throughput=5MB/sec 2024-04-07T16:42:08.692 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:42:08 smithi100 bash[74942]: cluster 2024-04-07T16:42:07.188286+0000 mgr.y (mgr.25175) 803 : cluster [DBG] pgmap v1445: 105 pgs: 105 active+clean; 126 GiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 2.2 MiB/s rd, 2.4 MiB/s wr, 4 op/s 2024-04-07T16:42:08.693 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:42:08 smithi100 bash[76202]: cluster 2024-04-07T16:42:07.188286+0000 mgr.y (mgr.25175) 803 : cluster [DBG] pgmap v1445: 105 pgs: 105 active+clean; 126 GiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 2.2 MiB/s rd, 2.4 MiB/s wr, 4 op/s 2024-04-07T16:42:08.693 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:42:08 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:42:08] "GET /metrics HTTP/1.1" 200 34057 "" "Prometheus/2.43.0" 2024-04-07T16:42:08.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:42:08 smithi144 bash[66654]: cluster 2024-04-07T16:42:07.188286+0000 mgr.y (mgr.25175) 803 : cluster [DBG] pgmap v1445: 105 pgs: 105 active+clean; 126 GiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 2.2 MiB/s rd, 2.4 MiB/s wr, 4 op/s 2024-04-07T16:42:09.307 INFO:tasks.workunit.client.0.smithi100.stdout: 206: throughput=4.98MB/sec pending data=0 2024-04-07T16:42:09.307 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-zDIBt47d6CmoIH9 off=75222 len=1221362 2024-04-07T16:42:09.319 INFO:tasks.workunit.client.0.smithi100.stdout:op 1089 completed, throughput=4.99MB/sec 2024-04-07T16:42:09.319 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-wjkLb6iH4nk7YFu off=1771205 len=1623069 2024-04-07T16:42:09.332 INFO:tasks.workunit.client.0.smithi100.stdout:op 1090 completed, throughput=4.99MB/sec 2024-04-07T16:42:09.332 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-iqZpiWGyFsUBHvy off=11118859 len=608878 2024-04-07T16:42:09.338 INFO:tasks.workunit.client.0.smithi100.stdout:op 1091 completed, throughput=4.99MB/sec 2024-04-07T16:42:09.338 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-0IILJOkewFRNAqL off=2287064 len=1826422 2024-04-07T16:42:09.345 INFO:tasks.workunit.client.0.smithi100.stdout:op 1092 completed, throughput=5MB/sec 2024-04-07T16:42:10.345 INFO:tasks.workunit.client.0.smithi100.stdout: 207: throughput=4.98MB/sec pending data=0 2024-04-07T16:42:10.345 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-z_H9XwQLuij3Uc1 off=4150158 len=783545 2024-04-07T16:42:10.348 INFO:tasks.workunit.client.0.smithi100.stdout:op 1093 completed, throughput=4.98MB/sec 2024-04-07T16:42:10.348 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-_bvY5WYePfc6LjT off=270882 len=713888 2024-04-07T16:42:10.356 INFO:tasks.workunit.client.0.smithi100.stdout:op 1094 completed, throughput=4.99MB/sec 2024-04-07T16:42:10.356 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-ICHCY7ibOZx5T1X off=2419605 len=122822 2024-04-07T16:42:10.358 INFO:tasks.workunit.client.0.smithi100.stdout:op 1095 completed, throughput=4.99MB/sec 2024-04-07T16:42:10.358 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-uJVBksCYVZ8E_rB off=8349223 len=773473 2024-04-07T16:42:10.360 INFO:tasks.workunit.client.0.smithi100.stdout:op 1096 completed, throughput=4.99MB/sec 2024-04-07T16:42:10.361 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-PcRwkRiqPaIPV4U off=4504949 len=1535874 2024-04-07T16:42:10.373 INFO:tasks.workunit.client.0.smithi100.stdout:op 1097 completed, throughput=5MB/sec 2024-04-07T16:42:10.373 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Oz_EVjrYjfnbqtv off=7377423 len=846821 2024-04-07T16:42:10.376 INFO:tasks.workunit.client.0.smithi100.stdout:op 1098 completed, throughput=5MB/sec 2024-04-07T16:42:10.376 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-kgFPMHwCssaAZ70 off=4087855 len=862408 2024-04-07T16:42:10.382 INFO:tasks.workunit.client.0.smithi100.stdout:op 1099 completed, throughput=5MB/sec 2024-04-07T16:42:10.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:42:10 smithi144 bash[66654]: cluster 2024-04-07T16:42:09.189092+0000 mgr.y (mgr.25175) 804 : cluster [DBG] pgmap v1446: 105 pgs: 105 active+clean; 126 GiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 4 op/s 2024-04-07T16:42:10.942 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:42:10 smithi100 bash[74942]: cluster 2024-04-07T16:42:09.189092+0000 mgr.y (mgr.25175) 804 : cluster [DBG] pgmap v1446: 105 pgs: 105 active+clean; 126 GiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 4 op/s 2024-04-07T16:42:10.943 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:42:10 smithi100 bash[76202]: cluster 2024-04-07T16:42:09.189092+0000 mgr.y (mgr.25175) 804 : cluster [DBG] pgmap v1446: 105 pgs: 105 active+clean; 126 GiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 4 op/s 2024-04-07T16:42:11.383 INFO:tasks.workunit.client.0.smithi100.stdout: 208: throughput=4.98MB/sec pending data=0 2024-04-07T16:42:11.383 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-nyD9XZwsnvTBYte off=2516781 len=384759 2024-04-07T16:42:11.385 INFO:tasks.workunit.client.0.smithi100.stdout:op 1100 completed, throughput=4.98MB/sec 2024-04-07T16:42:11.385 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-jiUZgjWuHLQOE41 off=17619396 len=800967 2024-04-07T16:42:11.387 INFO:tasks.workunit.client.0.smithi100.stdout:op 1101 completed, throughput=4.99MB/sec 2024-04-07T16:42:11.387 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-f7oxsOJmtHoDLsk off=357215 len=578029 2024-04-07T16:42:11.389 INFO:tasks.workunit.client.0.smithi100.stdout:op 1102 completed, throughput=4.99MB/sec 2024-04-07T16:42:11.389 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-cqDsQqiiBj4G9XE off=7625041 len=1756228 2024-04-07T16:42:11.402 INFO:tasks.workunit.client.0.smithi100.stdout:op 1103 completed, throughput=5MB/sec 2024-04-07T16:42:11.402 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-4c1V8qieQY50rXP off=3978379 len=1402887 2024-04-07T16:42:11.413 INFO:tasks.workunit.client.0.smithi100.stdout:op 1104 completed, throughput=5MB/sec 2024-04-07T16:42:12.414 INFO:tasks.workunit.client.0.smithi100.stdout: 209: throughput=4.98MB/sec pending data=0 2024-04-07T16:42:12.414 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-fcVeikA9MdmlNCc off=9082464 len=978138 2024-04-07T16:42:12.423 INFO:tasks.workunit.client.0.smithi100.stdout:op 1105 completed, throughput=4.98MB/sec 2024-04-07T16:42:12.423 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-jP9kIHnNnv3zL6X off=8838257 len=682499 2024-04-07T16:42:12.425 INFO:tasks.workunit.client.0.smithi100.stdout:op 1106 completed, throughput=4.99MB/sec 2024-04-07T16:42:12.425 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-yJwJIfBXfoLo87d off=209090 len=1970032 2024-04-07T16:42:12.430 INFO:tasks.workunit.client.0.smithi100.stdout:op 1107 completed, throughput=4.99MB/sec 2024-04-07T16:42:12.430 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-GlByFRWMZvoMRiA off=34563 len=300617 2024-04-07T16:42:12.432 INFO:tasks.workunit.client.0.smithi100.stdout:op 1108 completed, throughput=5MB/sec 2024-04-07T16:42:12.432 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-UfP4Fa51Vh6kgoV off=2042238 len=2040912 2024-04-07T16:42:12.447 INFO:tasks.workunit.client.0.smithi100.stdout:op 1109 completed, throughput=5MB/sec 2024-04-07T16:42:12.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:42:12 smithi144 bash[66654]: cluster 2024-04-07T16:42:11.190009+0000 mgr.y (mgr.25175) 805 : cluster [DBG] pgmap v1447: 105 pgs: 105 active+clean; 126 GiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.5 MiB/s wr, 4 op/s 2024-04-07T16:42:12.607 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:42:12 smithi144 bash[95233]: ts=2024-04-07T16:42:12.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:42:12.942 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:42:12 smithi100 bash[74942]: cluster 2024-04-07T16:42:11.190009+0000 mgr.y (mgr.25175) 805 : cluster [DBG] pgmap v1447: 105 pgs: 105 active+clean; 126 GiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.5 MiB/s wr, 4 op/s 2024-04-07T16:42:12.943 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:42:12 smithi100 bash[76202]: cluster 2024-04-07T16:42:11.190009+0000 mgr.y (mgr.25175) 805 : cluster [DBG] pgmap v1447: 105 pgs: 105 active+clean; 126 GiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.5 MiB/s wr, 4 op/s 2024-04-07T16:42:13.448 INFO:tasks.workunit.client.0.smithi100.stdout: 210: throughput=4.98MB/sec pending data=0 2024-04-07T16:42:13.448 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-BAozPKjmp0MMM1u off=326508 len=430796 2024-04-07T16:42:13.450 INFO:tasks.workunit.client.0.smithi100.stdout:op 1110 completed, throughput=4.98MB/sec 2024-04-07T16:42:13.450 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-bLIdFRalv2MH-oO off=14348724 len=311842 2024-04-07T16:42:13.456 INFO:tasks.workunit.client.0.smithi100.stdout:op 1111 completed, throughput=4.98MB/sec 2024-04-07T16:42:13.456 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-IUPd00xnzXc5Wmr off=20257832 len=830549 2024-04-07T16:42:13.465 INFO:tasks.workunit.client.0.smithi100.stdout:op 1112 completed, throughput=4.99MB/sec 2024-04-07T16:42:13.465 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-fdiWya0VyEIE4Yw off=3686107 len=525518 2024-04-07T16:42:13.467 INFO:tasks.workunit.client.0.smithi100.stdout:op 1113 completed, throughput=4.99MB/sec 2024-04-07T16:42:13.467 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-kJNdvZr9i3INMF8 off=10023895 len=1163559 2024-04-07T16:42:13.477 INFO:tasks.workunit.client.0.smithi100.stdout:op 1114 completed, throughput=4.99MB/sec 2024-04-07T16:42:13.477 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-JRgekhOnAHqzIP6 off=14130385 len=1963861 2024-04-07T16:42:13.482 INFO:tasks.workunit.client.0.smithi100.stdout:op 1115 completed, throughput=5MB/sec 2024-04-07T16:42:14.482 INFO:tasks.workunit.client.0.smithi100.stdout: 211: throughput=4.98MB/sec pending data=0 2024-04-07T16:42:14.483 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-sXW_g1HSoEdq3SH off=7216351 len=480724 2024-04-07T16:42:14.485 INFO:tasks.workunit.client.0.smithi100.stdout:op 1116 completed, throughput=4.98MB/sec 2024-04-07T16:42:14.485 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-15EjOVqPfKR5rhQ off=943063 len=1605037 2024-04-07T16:42:14.500 INFO:tasks.workunit.client.0.smithi100.stdout:op 1117 completed, throughput=4.99MB/sec 2024-04-07T16:42:14.500 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-PobUf3-u8YWsAIN off=5601193 len=387281 2024-04-07T16:42:14.507 INFO:tasks.workunit.client.0.smithi100.stdout:op 1118 completed, throughput=4.99MB/sec 2024-04-07T16:42:14.507 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-4jfrJRnL8UxEGFu off=3211354 len=496707 2024-04-07T16:42:14.513 INFO:tasks.workunit.client.0.smithi100.stdout:op 1119 completed, throughput=4.99MB/sec 2024-04-07T16:42:14.513 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-qhN-SqJJEOmafeq off=95327 len=81142 2024-04-07T16:42:14.515 INFO:tasks.workunit.client.0.smithi100.stdout:op 1120 completed, throughput=4.99MB/sec 2024-04-07T16:42:14.515 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-g8IJpY1t5qj1m9X off=470163 len=430018 2024-04-07T16:42:14.521 INFO:tasks.workunit.client.0.smithi100.stdout:op 1121 completed, throughput=4.99MB/sec 2024-04-07T16:42:14.521 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-qSlfamtnDYFmnxm off=1570322 len=1839301 2024-04-07T16:42:14.534 INFO:tasks.workunit.client.0.smithi100.stdout:op 1122 completed, throughput=5MB/sec 2024-04-07T16:42:14.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:42:14 smithi144 bash[66654]: cluster 2024-04-07T16:42:13.191134+0000 mgr.y (mgr.25175) 806 : cluster [DBG] pgmap v1448: 105 pgs: 105 active+clean; 126 GiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 2.9 MiB/s rd, 2.8 MiB/s wr, 5 op/s 2024-04-07T16:42:14.943 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:42:14 smithi100 bash[74942]: cluster 2024-04-07T16:42:13.191134+0000 mgr.y (mgr.25175) 806 : cluster [DBG] pgmap v1448: 105 pgs: 105 active+clean; 126 GiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 2.9 MiB/s rd, 2.8 MiB/s wr, 5 op/s 2024-04-07T16:42:14.943 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:42:14 smithi100 bash[76202]: cluster 2024-04-07T16:42:13.191134+0000 mgr.y (mgr.25175) 806 : cluster [DBG] pgmap v1448: 105 pgs: 105 active+clean; 126 GiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 2.9 MiB/s rd, 2.8 MiB/s wr, 5 op/s 2024-04-07T16:42:15.535 INFO:tasks.workunit.client.0.smithi100.stdout: 212: throughput=4.98MB/sec pending data=0 2024-04-07T16:42:15.535 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-ZyMu5OvveBEvfk9 off=17098962 len=1492589 2024-04-07T16:42:15.538 INFO:tasks.workunit.client.0.smithi100.stdout:op 1123 completed, throughput=4.99MB/sec 2024-04-07T16:42:15.539 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-iCwKPpbklFvQyIN off=6950021 len=1787432 2024-04-07T16:42:15.554 INFO:tasks.workunit.client.0.smithi100.stdout:op 1124 completed, throughput=4.99MB/sec 2024-04-07T16:42:15.554 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-zWhQm1-jm5A5oBp off=3138280 len=99087 2024-04-07T16:42:15.556 INFO:tasks.workunit.client.0.smithi100.stdout:op 1125 completed, throughput=4.99MB/sec 2024-04-07T16:42:15.556 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-tuCNmQjXNytSK6m off=9404161 len=1669152 2024-04-07T16:42:15.559 INFO:tasks.workunit.client.0.smithi100.stdout:op 1126 completed, throughput=5MB/sec 2024-04-07T16:42:16.560 INFO:tasks.workunit.client.0.smithi100.stdout: 213: throughput=4.98MB/sec pending data=0 2024-04-07T16:42:16.560 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-WRoxIQJ5VHyETbg off=1489820 len=595396 2024-04-07T16:42:16.567 INFO:tasks.workunit.client.0.smithi100.stdout:op 1127 completed, throughput=4.98MB/sec 2024-04-07T16:42:16.567 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Po7-Lpt6CRLPAdO off=5934862 len=1952342 2024-04-07T16:42:16.581 INFO:tasks.workunit.client.0.smithi100.stdout:op 1128 completed, throughput=4.99MB/sec 2024-04-07T16:42:16.582 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Yhp22FlHulqwvES off=15791106 len=1591163 2024-04-07T16:42:16.594 INFO:tasks.workunit.client.0.smithi100.stdout:op 1129 completed, throughput=5MB/sec 2024-04-07T16:42:16.594 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-g4RhHLvqVhwyVsc off=1969422 len=1516981 2024-04-07T16:42:16.604 INFO:tasks.workunit.client.0.smithi100.stdout:op 1130 completed, throughput=5MB/sec 2024-04-07T16:42:16.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:42:16 smithi144 bash[66654]: cluster 2024-04-07T16:42:15.192116+0000 mgr.y (mgr.25175) 807 : cluster [DBG] pgmap v1449: 105 pgs: 105 active+clean; 126 GiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 2.4 MiB/s rd, 2.6 MiB/s wr, 4 op/s 2024-04-07T16:42:16.942 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:42:16 smithi100 bash[74942]: cluster 2024-04-07T16:42:15.192116+0000 mgr.y (mgr.25175) 807 : cluster [DBG] pgmap v1449: 105 pgs: 105 active+clean; 126 GiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 2.4 MiB/s rd, 2.6 MiB/s wr, 4 op/s 2024-04-07T16:42:16.943 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:42:16 smithi100 bash[76202]: cluster 2024-04-07T16:42:15.192116+0000 mgr.y (mgr.25175) 807 : cluster [DBG] pgmap v1449: 105 pgs: 105 active+clean; 126 GiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 2.4 MiB/s rd, 2.6 MiB/s wr, 4 op/s 2024-04-07T16:42:17.604 INFO:tasks.workunit.client.0.smithi100.stdout: 214: throughput=4.98MB/sec pending data=0 2024-04-07T16:42:17.604 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-GyjTWMOkLqyLbZ8 off=13442059 len=668046 2024-04-07T16:42:17.607 INFO:tasks.workunit.client.0.smithi100.stdout:op 1131 completed, throughput=4.98MB/sec 2024-04-07T16:42:17.607 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-KANP8I34F-FOATr off=6536253 len=997222 2024-04-07T16:42:17.616 INFO:tasks.workunit.client.0.smithi100.stdout:op 1132 completed, throughput=4.99MB/sec 2024-04-07T16:42:17.616 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-usK7nO7VFW9Rzx3 off=774444 len=1876037 2024-04-07T16:42:17.632 INFO:tasks.workunit.client.0.smithi100.stdout:op 1133 completed, throughput=4.99MB/sec 2024-04-07T16:42:17.632 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-7TTnn2ScNBmW4lh off=699034 len=2029178 2024-04-07T16:42:17.637 INFO:tasks.workunit.client.0.smithi100.stdout:op 1134 completed, throughput=5MB/sec 2024-04-07T16:42:17.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:42:17 smithi144 bash[66654]: audit 2024-04-07T16:42:16.760894+0000 mon.a (mon.0) 2074 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:42:17.942 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:42:17 smithi100 bash[74942]: audit 2024-04-07T16:42:16.760894+0000 mon.a (mon.0) 2074 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:42:17.943 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:42:17 smithi100 bash[76202]: audit 2024-04-07T16:42:16.760894+0000 mon.a (mon.0) 2074 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:42:18.637 INFO:tasks.workunit.client.0.smithi100.stdout: 215: throughput=4.98MB/sec pending data=0 2024-04-07T16:42:18.638 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-wl00iqcA5ko8IZw off=452060 len=576406 2024-04-07T16:42:18.640 INFO:tasks.workunit.client.0.smithi100.stdout:op 1135 completed, throughput=4.98MB/sec 2024-04-07T16:42:18.640 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-sow_E8CaOYsbV_8 off=12902241 len=1315129 2024-04-07T16:42:18.643 INFO:tasks.workunit.client.0.smithi100.stdout:op 1136 completed, throughput=4.99MB/sec 2024-04-07T16:42:18.643 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-fLQA4H5zGxI_Guh off=0 len=1484109 2024-04-07T16:42:18.649 INFO:tasks.workunit.client.0.smithi100.stdout:op 1137 completed, throughput=4.99MB/sec 2024-04-07T16:42:18.649 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-0Dj1-Ku0NYoJL6w off=19234845 len=141518 2024-04-07T16:42:18.650 INFO:tasks.workunit.client.0.smithi100.stdout:op 1138 completed, throughput=4.99MB/sec 2024-04-07T16:42:18.650 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-mS6wTIb3UhCbx0t off=747368 len=269177 2024-04-07T16:42:18.651 INFO:tasks.workunit.client.0.smithi100.stdout:op 1139 completed, throughput=5MB/sec 2024-04-07T16:42:18.651 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Oz_EVjrYjfnbqtv off=17691265 len=1202265 2024-04-07T16:42:18.662 INFO:tasks.workunit.client.0.smithi100.stdout:op 1140 completed, throughput=5MB/sec 2024-04-07T16:42:18.692 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:42:18 smithi100 bash[74942]: cluster 2024-04-07T16:42:17.193112+0000 mgr.y (mgr.25175) 808 : cluster [DBG] pgmap v1450: 105 pgs: 105 active+clean; 126 GiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 2.3 MiB/s rd, 3.3 MiB/s wr, 5 op/s 2024-04-07T16:42:18.693 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:42:18 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:42:18] "GET /metrics HTTP/1.1" 200 34055 "" "Prometheus/2.43.0" 2024-04-07T16:42:18.693 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:42:18 smithi100 bash[76202]: cluster 2024-04-07T16:42:17.193112+0000 mgr.y (mgr.25175) 808 : cluster [DBG] pgmap v1450: 105 pgs: 105 active+clean; 126 GiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 2.3 MiB/s rd, 3.3 MiB/s wr, 5 op/s 2024-04-07T16:42:18.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:42:18 smithi144 bash[66654]: cluster 2024-04-07T16:42:17.193112+0000 mgr.y (mgr.25175) 808 : cluster [DBG] pgmap v1450: 105 pgs: 105 active+clean; 126 GiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 2.3 MiB/s rd, 3.3 MiB/s wr, 5 op/s 2024-04-07T16:42:19.662 INFO:tasks.workunit.client.0.smithi100.stdout: 216: throughput=4.98MB/sec pending data=0 2024-04-07T16:42:19.663 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-fUlOo_k3w3S2fZU off=395729 len=1573484 2024-04-07T16:42:19.674 INFO:tasks.workunit.client.0.smithi100.stdout:op 1141 completed, throughput=4.98MB/sec 2024-04-07T16:42:19.674 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-80ipgMd0g5_g1jz off=10169173 len=601398 2024-04-07T16:42:19.677 INFO:tasks.workunit.client.0.smithi100.stdout:op 1142 completed, throughput=4.99MB/sec 2024-04-07T16:42:19.677 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-p9x4AakkWOcMHHb off=0 len=1517722 2024-04-07T16:42:19.681 INFO:tasks.workunit.client.0.smithi100.stdout:op 1143 completed, throughput=4.99MB/sec 2024-04-07T16:42:19.681 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-lZicUy_9ZGNsow4 off=2601199 len=1921905 2024-04-07T16:42:19.686 INFO:tasks.workunit.client.0.smithi100.stdout:op 1144 completed, throughput=5MB/sec 2024-04-07T16:42:20.686 INFO:tasks.workunit.client.0.smithi100.stdout: 217: throughput=4.98MB/sec pending data=0 2024-04-07T16:42:20.686 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-IrAVBQUntQ_W_0k off=6290508 len=1003033 2024-04-07T16:42:20.689 INFO:tasks.workunit.client.0.smithi100.stdout:op 1145 completed, throughput=4.98MB/sec 2024-04-07T16:42:20.690 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-g5S6icEweOTZ7QO off=15575106 len=2048978 2024-04-07T16:42:20.708 INFO:tasks.workunit.client.0.smithi100.stdout:op 1146 completed, throughput=4.99MB/sec 2024-04-07T16:42:20.708 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-dDIxRB4uNPOzAzv off=12496264 len=1809133 2024-04-07T16:42:20.712 INFO:tasks.workunit.client.0.smithi100.stdout:op 1147 completed, throughput=5MB/sec 2024-04-07T16:42:20.712 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-gb4zYBwN6Y2HO_D off=7260685 len=1260624 2024-04-07T16:42:20.714 INFO:tasks.workunit.client.0.smithi100.stdout:op 1148 completed, throughput=5MB/sec 2024-04-07T16:42:20.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:42:20 smithi144 bash[66654]: cluster 2024-04-07T16:42:19.193959+0000 mgr.y (mgr.25175) 809 : cluster [DBG] pgmap v1451: 105 pgs: 105 active+clean; 126 GiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 2.1 MiB/s rd, 2.7 MiB/s wr, 4 op/s 2024-04-07T16:42:20.943 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:42:20 smithi100 bash[74942]: cluster 2024-04-07T16:42:19.193959+0000 mgr.y (mgr.25175) 809 : cluster [DBG] pgmap v1451: 105 pgs: 105 active+clean; 126 GiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 2.1 MiB/s rd, 2.7 MiB/s wr, 4 op/s 2024-04-07T16:42:20.943 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:42:20 smithi100 bash[76202]: cluster 2024-04-07T16:42:19.193959+0000 mgr.y (mgr.25175) 809 : cluster [DBG] pgmap v1451: 105 pgs: 105 active+clean; 126 GiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 2.1 MiB/s rd, 2.7 MiB/s wr, 4 op/s 2024-04-07T16:42:21.715 INFO:tasks.workunit.client.0.smithi100.stdout: 218: throughput=4.98MB/sec pending data=0 2024-04-07T16:42:21.715 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-F3rmx7zdgbe58JY off=382658 len=487537 2024-04-07T16:42:21.718 INFO:tasks.workunit.client.0.smithi100.stdout:op 1149 completed, throughput=4.98MB/sec 2024-04-07T16:42:21.718 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Qul01xM7hrk2tIs off=9109701 len=247300 2024-04-07T16:42:21.723 INFO:tasks.workunit.client.0.smithi100.stdout:op 1150 completed, throughput=4.99MB/sec 2024-04-07T16:42:21.723 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-qdidzmysWWqzvCc off=13487331 len=1628618 2024-04-07T16:42:21.727 INFO:tasks.workunit.client.0.smithi100.stdout:op 1151 completed, throughput=4.99MB/sec 2024-04-07T16:42:21.727 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-nkEHgyW3kjwx06a off=3388071 len=87821 2024-04-07T16:42:21.729 INFO:tasks.workunit.client.0.smithi100.stdout:op 1152 completed, throughput=4.99MB/sec 2024-04-07T16:42:21.729 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-zidiUA9Xt61Jtkj off=3408545 len=501469 2024-04-07T16:42:21.735 INFO:tasks.workunit.client.0.smithi100.stdout:op 1153 completed, throughput=4.99MB/sec 2024-04-07T16:42:21.735 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-EPIy_FxC21_LYpI off=7566815 len=1323100 2024-04-07T16:42:21.739 INFO:tasks.workunit.client.0.smithi100.stdout:op 1154 completed, throughput=5MB/sec 2024-04-07T16:42:21.739 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-RX8-L6dT4bMHq32 off=9409181 len=374105 2024-04-07T16:42:21.745 INFO:tasks.workunit.client.0.smithi100.stdout:op 1155 completed, throughput=5MB/sec 2024-04-07T16:42:22.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:42:22 smithi144 bash[66654]: cluster 2024-04-07T16:42:21.194847+0000 mgr.y (mgr.25175) 810 : cluster [DBG] pgmap v1452: 105 pgs: 105 active+clean; 126 GiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 2.1 MiB/s rd, 2.9 MiB/s wr, 5 op/s 2024-04-07T16:42:22.608 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:42:22 smithi144 bash[95233]: ts=2024-04-07T16:42:22.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:42:22.745 INFO:tasks.workunit.client.0.smithi100.stdout: 219: throughput=4.98MB/sec pending data=0 2024-04-07T16:42:22.746 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-niNVKkWvY2UaG_4 off=844668 len=112884 2024-04-07T16:42:22.747 INFO:tasks.workunit.client.0.smithi100.stdout:op 1156 completed, throughput=4.98MB/sec 2024-04-07T16:42:22.747 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-fW2vUpcr94ing32 off=11407840 len=78703 2024-04-07T16:42:22.752 INFO:tasks.workunit.client.0.smithi100.stdout:op 1157 completed, throughput=4.98MB/sec 2024-04-07T16:42:22.752 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-K9zmKILdh8-DudF off=13056635 len=293236 2024-04-07T16:42:22.753 INFO:tasks.workunit.client.0.smithi100.stdout:op 1158 completed, throughput=4.98MB/sec 2024-04-07T16:42:22.753 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-fV9hMcBjHHd8RRU off=3039419 len=1668454 2024-04-07T16:42:22.758 INFO:tasks.workunit.client.0.smithi100.stdout:op 1159 completed, throughput=4.99MB/sec 2024-04-07T16:42:22.758 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-YeNMJWHF7H2rJ8p off=1316292 len=1460077 2024-04-07T16:42:22.770 INFO:tasks.workunit.client.0.smithi100.stdout:op 1160 completed, throughput=4.99MB/sec 2024-04-07T16:42:22.770 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-TEP5_hFbA6Rgx8S off=1086874 len=956234 2024-04-07T16:42:22.778 INFO:tasks.workunit.client.0.smithi100.stdout:op 1161 completed, throughput=5MB/sec 2024-04-07T16:42:22.778 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-07GrybRFRSovC_c off=6047058 len=1188973 2024-04-07T16:42:22.788 INFO:tasks.workunit.client.0.smithi100.stdout:op 1162 completed, throughput=5MB/sec 2024-04-07T16:42:22.942 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:42:22 smithi100 bash[74942]: cluster 2024-04-07T16:42:21.194847+0000 mgr.y (mgr.25175) 810 : cluster [DBG] pgmap v1452: 105 pgs: 105 active+clean; 126 GiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 2.1 MiB/s rd, 2.9 MiB/s wr, 5 op/s 2024-04-07T16:42:22.943 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:42:22 smithi100 bash[76202]: cluster 2024-04-07T16:42:21.194847+0000 mgr.y (mgr.25175) 810 : cluster [DBG] pgmap v1452: 105 pgs: 105 active+clean; 126 GiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 2.1 MiB/s rd, 2.9 MiB/s wr, 5 op/s 2024-04-07T16:42:23.788 INFO:tasks.workunit.client.0.smithi100.stdout: 220: throughput=4.98MB/sec pending data=0 2024-04-07T16:42:23.789 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-LpQ4hWcoj25or61 off=23285822 len=1090463 2024-04-07T16:42:23.798 INFO:tasks.workunit.client.0.smithi100.stdout:op 1163 completed, throughput=4.98MB/sec 2024-04-07T16:42:23.798 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-zNRmgZEp9mFJgj_ off=10529194 len=213739 2024-04-07T16:42:23.803 INFO:tasks.workunit.client.0.smithi100.stdout:op 1164 completed, throughput=4.99MB/sec 2024-04-07T16:42:23.803 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-r78aJ9ywAdjK03c off=8463829 len=1312286 2024-04-07T16:42:23.806 INFO:tasks.workunit.client.0.smithi100.stdout:op 1165 completed, throughput=4.99MB/sec 2024-04-07T16:42:23.806 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-RlXWjm1Tuduttxm off=5512952 len=1152398 2024-04-07T16:42:23.816 INFO:tasks.workunit.client.0.smithi100.stdout:op 1166 completed, throughput=5MB/sec 2024-04-07T16:42:23.816 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-RbFuzRbd47JyJkW off=16644183 len=1747884 2024-04-07T16:42:23.822 INFO:tasks.workunit.client.0.smithi100.stdout:op 1167 completed, throughput=5MB/sec 2024-04-07T16:42:24.822 INFO:tasks.workunit.client.0.smithi100.stdout: 221: throughput=4.98MB/sec pending data=0 2024-04-07T16:42:24.823 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-0cVu2_oyCz7RYnW off=3923587 len=45486 2024-04-07T16:42:24.823 INFO:tasks.workunit.client.0.smithi100.stdout:op 1168 completed, throughput=4.98MB/sec 2024-04-07T16:42:24.824 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-m8r86ss4lwXrjYI off=7075472 len=490139 2024-04-07T16:42:24.830 INFO:tasks.workunit.client.0.smithi100.stdout:op 1169 completed, throughput=4.98MB/sec 2024-04-07T16:42:24.830 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-v3v9dsxCQtUWlqi off=3064781 len=648847 2024-04-07T16:42:24.832 INFO:tasks.workunit.client.0.smithi100.stdout:op 1170 completed, throughput=4.99MB/sec 2024-04-07T16:42:24.832 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-sow_E8CaOYsbV_8 off=19525123 len=1525414 2024-04-07T16:42:24.845 INFO:tasks.workunit.client.0.smithi100.stdout:op 1171 completed, throughput=4.99MB/sec 2024-04-07T16:42:24.845 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-rYyC_MuKqBqsOFK off=3422686 len=1241926 2024-04-07T16:42:24.855 INFO:tasks.workunit.client.0.smithi100.stdout:op 1172 completed, throughput=5MB/sec 2024-04-07T16:42:24.855 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-3pLTQMK_SgSlEil off=1647924 len=446762 2024-04-07T16:42:24.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:42:24 smithi144 bash[66654]: cluster 2024-04-07T16:42:23.195982+0000 mgr.y (mgr.25175) 811 : cluster [DBG] pgmap v1453: 105 pgs: 105 active+clean; 126 GiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 2.9 MiB/s rd, 3.0 MiB/s wr, 5 op/s 2024-04-07T16:42:24.860 INFO:tasks.workunit.client.0.smithi100.stdout:op 1173 completed, throughput=5MB/sec 2024-04-07T16:42:24.860 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-LyoC51hNZZSucZt off=2877914 len=957073 2024-04-07T16:42:24.864 INFO:tasks.workunit.client.0.smithi100.stdout:op 1174 completed, throughput=5MB/sec 2024-04-07T16:42:24.942 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:42:24 smithi100 bash[74942]: cluster 2024-04-07T16:42:23.195982+0000 mgr.y (mgr.25175) 811 : cluster [DBG] pgmap v1453: 105 pgs: 105 active+clean; 126 GiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 2.9 MiB/s rd, 3.0 MiB/s wr, 5 op/s 2024-04-07T16:42:24.943 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:42:24 smithi100 bash[76202]: cluster 2024-04-07T16:42:23.195982+0000 mgr.y (mgr.25175) 811 : cluster [DBG] pgmap v1453: 105 pgs: 105 active+clean; 126 GiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 2.9 MiB/s rd, 3.0 MiB/s wr, 5 op/s 2024-04-07T16:42:25.864 INFO:tasks.workunit.client.0.smithi100.stdout: 222: throughput=4.98MB/sec pending data=0 2024-04-07T16:42:25.864 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-2jMWWcZB7oi-s_E off=10770242 len=711340 2024-04-07T16:42:25.866 INFO:tasks.workunit.client.0.smithi100.stdout:op 1175 completed, throughput=4.98MB/sec 2024-04-07T16:42:25.866 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-tiu874C0ru-b_ZS off=7555866 len=1217607 2024-04-07T16:42:25.877 INFO:tasks.workunit.client.0.smithi100.stdout:op 1176 completed, throughput=4.99MB/sec 2024-04-07T16:42:25.878 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-2JdIth1XsxP8gOc off=6948619 len=790564 2024-04-07T16:42:25.880 INFO:tasks.workunit.client.0.smithi100.stdout:op 1177 completed, throughput=4.99MB/sec 2024-04-07T16:42:25.880 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-EWoj_ygQrw9O8-X off=330256 len=1291966 2024-04-07T16:42:25.884 INFO:tasks.workunit.client.0.smithi100.stdout:op 1178 completed, throughput=5MB/sec 2024-04-07T16:42:25.884 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-0qzUVG8W1_hGIr7 off=311597 len=973478 2024-04-07T16:42:25.886 INFO:tasks.workunit.client.0.smithi100.stdout:op 1179 completed, throughput=5MB/sec 2024-04-07T16:42:26.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:42:26 smithi144 bash[66654]: cluster 2024-04-07T16:42:25.196928+0000 mgr.y (mgr.25175) 812 : cluster [DBG] pgmap v1454: 105 pgs: 105 active+clean; 126 GiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 2.4 MiB/s rd, 2.8 MiB/s wr, 5 op/s 2024-04-07T16:42:26.886 INFO:tasks.workunit.client.0.smithi100.stdout: 223: throughput=4.98MB/sec pending data=0 2024-04-07T16:42:26.886 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj--IGBRWOeZKOehDR off=6280850 len=1769189 2024-04-07T16:42:26.893 INFO:tasks.workunit.client.0.smithi100.stdout:op 1180 completed, throughput=4.99MB/sec 2024-04-07T16:42:26.893 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-5g50IgLkAiKXtyf off=3175079 len=82273 2024-04-07T16:42:26.894 INFO:tasks.workunit.client.0.smithi100.stdout:op 1181 completed, throughput=4.99MB/sec 2024-04-07T16:42:26.894 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-XV6Y_9i97B_dLAa off=13286974 len=248809 2024-04-07T16:42:26.899 INFO:tasks.workunit.client.0.smithi100.stdout:op 1182 completed, throughput=4.99MB/sec 2024-04-07T16:42:26.900 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-KRcWeH5AiMhp6kA off=8871334 len=1176935 2024-04-07T16:42:26.910 INFO:tasks.workunit.client.0.smithi100.stdout:op 1183 completed, throughput=4.99MB/sec 2024-04-07T16:42:26.910 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-tAhgvPiBWbfYJSO off=2310238 len=1255188 2024-04-07T16:42:26.913 INFO:tasks.workunit.client.0.smithi100.stdout:op 1184 completed, throughput=5MB/sec 2024-04-07T16:42:26.913 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-hGFhljive4rJ1IU off=11445504 len=794051 2024-04-07T16:42:26.915 INFO:tasks.workunit.client.0.smithi100.stdout:op 1185 completed, throughput=5MB/sec 2024-04-07T16:42:26.942 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:42:26 smithi100 bash[74942]: cluster 2024-04-07T16:42:25.196928+0000 mgr.y (mgr.25175) 812 : cluster [DBG] pgmap v1454: 105 pgs: 105 active+clean; 126 GiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 2.4 MiB/s rd, 2.8 MiB/s wr, 5 op/s 2024-04-07T16:42:26.943 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:42:26 smithi100 bash[76202]: cluster 2024-04-07T16:42:25.196928+0000 mgr.y (mgr.25175) 812 : cluster [DBG] pgmap v1454: 105 pgs: 105 active+clean; 126 GiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 2.4 MiB/s rd, 2.8 MiB/s wr, 5 op/s 2024-04-07T16:42:27.916 INFO:tasks.workunit.client.0.smithi100.stdout: 224: throughput=4.98MB/sec pending data=0 2024-04-07T16:42:27.916 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-eJrI0kN7EX5PAgo off=15870501 len=402663 2024-04-07T16:42:27.918 INFO:tasks.workunit.client.0.smithi100.stdout:op 1186 completed, throughput=4.98MB/sec 2024-04-07T16:42:27.918 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-URkzPIMiLmYEqKM off=2025577 len=1888728 2024-04-07T16:42:27.923 INFO:tasks.workunit.client.0.smithi100.stdout:op 1187 completed, throughput=4.99MB/sec 2024-04-07T16:42:27.923 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-owvhWtFT6w7oXMM off=3771119 len=2086015 2024-04-07T16:42:27.940 INFO:tasks.workunit.client.0.smithi100.stdout:op 1188 completed, throughput=5MB/sec 2024-04-07T16:42:27.940 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-HMWQYY8sWlA_RB- off=3216598 len=1264433 2024-04-07T16:42:27.951 INFO:tasks.workunit.client.0.smithi100.stdout:op 1189 completed, throughput=5MB/sec 2024-04-07T16:42:28.692 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:42:28 smithi100 bash[74942]: cluster 2024-04-07T16:42:27.198008+0000 mgr.y (mgr.25175) 813 : cluster [DBG] pgmap v1455: 105 pgs: 105 active+clean; 126 GiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 2.6 MiB/s rd, 2.7 MiB/s wr, 5 op/s 2024-04-07T16:42:28.693 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:42:28 smithi100 bash[76202]: cluster 2024-04-07T16:42:27.198008+0000 mgr.y (mgr.25175) 813 : cluster [DBG] pgmap v1455: 105 pgs: 105 active+clean; 126 GiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 2.6 MiB/s rd, 2.7 MiB/s wr, 5 op/s 2024-04-07T16:42:28.693 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:42:28 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:42:28] "GET /metrics HTTP/1.1" 200 34055 "" "Prometheus/2.43.0" 2024-04-07T16:42:28.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:42:28 smithi144 bash[66654]: cluster 2024-04-07T16:42:27.198008+0000 mgr.y (mgr.25175) 813 : cluster [DBG] pgmap v1455: 105 pgs: 105 active+clean; 126 GiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 2.6 MiB/s rd, 2.7 MiB/s wr, 5 op/s 2024-04-07T16:42:28.951 INFO:tasks.workunit.client.0.smithi100.stdout: 225: throughput=4.98MB/sec pending data=0 2024-04-07T16:42:28.951 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-cOjLGt530UngB6w off=14473184 len=28428 2024-04-07T16:42:28.952 INFO:tasks.workunit.client.0.smithi100.stdout:op 1190 completed, throughput=4.98MB/sec 2024-04-07T16:42:28.952 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-sIyyb1NgGGHyTuI off=11945655 len=102924 2024-04-07T16:42:28.953 INFO:tasks.workunit.client.0.smithi100.stdout:op 1191 completed, throughput=4.98MB/sec 2024-04-07T16:42:28.953 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-02oyuQrGsJDPjm9 off=4229761 len=41483 2024-04-07T16:42:28.955 INFO:tasks.workunit.client.0.smithi100.stdout:op 1192 completed, throughput=4.98MB/sec 2024-04-07T16:42:28.955 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-ES-bSEHarHdaSvk off=3179837 len=407207 2024-04-07T16:42:28.961 INFO:tasks.workunit.client.0.smithi100.stdout:op 1193 completed, throughput=4.98MB/sec 2024-04-07T16:42:28.961 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-3nS7arLdei0Jmjz off=3190939 len=6163 2024-04-07T16:42:28.965 INFO:tasks.workunit.client.0.smithi100.stdout:op 1194 completed, throughput=4.98MB/sec 2024-04-07T16:42:28.965 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-YrwZ8Isk4LStWVn off=1390001 len=576824 2024-04-07T16:42:28.967 INFO:tasks.workunit.client.0.smithi100.stdout:op 1195 completed, throughput=4.98MB/sec 2024-04-07T16:42:28.967 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-lX_ouLQQkLgqJvI off=1357550 len=827333 2024-04-07T16:42:28.969 INFO:tasks.workunit.client.0.smithi100.stdout:op 1196 completed, throughput=4.99MB/sec 2024-04-07T16:42:28.969 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-LsePYUZGwjDPcBI off=3077991 len=57238 2024-04-07T16:42:28.973 INFO:tasks.workunit.client.0.smithi100.stdout:op 1197 completed, throughput=4.99MB/sec 2024-04-07T16:42:28.973 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-TXP7cmlPRTBMTyd off=5870527 len=1869271 2024-04-07T16:42:28.987 INFO:tasks.workunit.client.0.smithi100.stdout:op 1198 completed, throughput=5MB/sec 2024-04-07T16:42:28.988 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-gnpWYPoZ3eMgPfc off=21570438 len=902158 2024-04-07T16:42:28.991 INFO:tasks.workunit.client.0.smithi100.stdout:op 1199 completed, throughput=5MB/sec 2024-04-07T16:42:28.992 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-L_V-HLQDAmkrmGz off=4206869 len=1189600 2024-04-07T16:42:29.002 INFO:tasks.workunit.client.0.smithi100.stdout:op 1200 completed, throughput=5MB/sec 2024-04-07T16:42:30.002 INFO:tasks.workunit.client.0.smithi100.stdout: 226: throughput=4.98MB/sec pending data=0 2024-04-07T16:42:30.002 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-HJgUVVqrV6oiz-L off=3328598 len=666263 2024-04-07T16:42:30.004 INFO:tasks.workunit.client.0.smithi100.stdout:op 1201 completed, throughput=4.98MB/sec 2024-04-07T16:42:30.005 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-J46MSmocNJvYJvk off=12610121 len=361745 2024-04-07T16:42:30.011 INFO:tasks.workunit.client.0.smithi100.stdout:op 1202 completed, throughput=4.99MB/sec 2024-04-07T16:42:30.011 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-CgvHSfHnHYO634k off=15400165 len=1384274 2024-04-07T16:42:30.014 INFO:tasks.workunit.client.0.smithi100.stdout:op 1203 completed, throughput=4.99MB/sec 2024-04-07T16:42:30.014 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-kZ--8DKPS5oO1-5 off=5353408 len=684719 2024-04-07T16:42:30.021 INFO:tasks.workunit.client.0.smithi100.stdout:op 1204 completed, throughput=4.99MB/sec 2024-04-07T16:42:30.021 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-k9AsolqRHX70XbR off=21680577 len=2079332 2024-04-07T16:42:30.027 INFO:tasks.workunit.client.0.smithi100.stdout:op 1205 completed, throughput=5MB/sec 2024-04-07T16:42:30.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:42:30 smithi144 bash[66654]: cluster 2024-04-07T16:42:29.198904+0000 mgr.y (mgr.25175) 814 : cluster [DBG] pgmap v1456: 105 pgs: 105 active+clean; 126 GiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 2.6 MiB/s rd, 2.0 MiB/s wr, 4 op/s 2024-04-07T16:42:30.942 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:42:30 smithi100 bash[74942]: cluster 2024-04-07T16:42:29.198904+0000 mgr.y (mgr.25175) 814 : cluster [DBG] pgmap v1456: 105 pgs: 105 active+clean; 126 GiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 2.6 MiB/s rd, 2.0 MiB/s wr, 4 op/s 2024-04-07T16:42:30.943 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:42:30 smithi100 bash[76202]: cluster 2024-04-07T16:42:29.198904+0000 mgr.y (mgr.25175) 814 : cluster [DBG] pgmap v1456: 105 pgs: 105 active+clean; 126 GiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 2.6 MiB/s rd, 2.0 MiB/s wr, 4 op/s 2024-04-07T16:42:31.027 INFO:tasks.workunit.client.0.smithi100.stdout: 227: throughput=4.98MB/sec pending data=0 2024-04-07T16:42:31.027 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-TssmVivSSQGVIcH off=3858195 len=417094 2024-04-07T16:42:31.029 INFO:tasks.workunit.client.0.smithi100.stdout:op 1206 completed, throughput=4.98MB/sec 2024-04-07T16:42:31.029 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Ywj8NyCwsMFO428 off=6884195 len=1461926 2024-04-07T16:42:31.041 INFO:tasks.workunit.client.0.smithi100.stdout:op 1207 completed, throughput=4.99MB/sec 2024-04-07T16:42:31.042 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-MsWM604doA4Ktyd off=337205 len=957461 2024-04-07T16:42:31.050 INFO:tasks.workunit.client.0.smithi100.stdout:op 1208 completed, throughput=4.99MB/sec 2024-04-07T16:42:31.050 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-PhJ-JfXcFANUx1Y off=5148090 len=341952 2024-04-07T16:42:31.052 INFO:tasks.workunit.client.0.smithi100.stdout:op 1209 completed, throughput=4.99MB/sec 2024-04-07T16:42:31.052 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-ZN1YBd-9O0EVUz6 off=551451 len=1626074 2024-04-07T16:42:31.067 INFO:tasks.workunit.client.0.smithi100.stdout:op 1210 completed, throughput=5MB/sec 2024-04-07T16:42:32.067 INFO:tasks.workunit.client.0.smithi100.stdout: 228: throughput=4.98MB/sec pending data=0 2024-04-07T16:42:32.068 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-tiakL33_kV3bLlk off=7461349 len=1948690 2024-04-07T16:42:32.083 INFO:tasks.workunit.client.0.smithi100.stdout:op 1211 completed, throughput=4.99MB/sec 2024-04-07T16:42:32.083 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-JbjpqRgW3RtsdS6 off=12195132 len=1928948 2024-04-07T16:42:32.097 INFO:tasks.workunit.client.0.smithi100.stdout:op 1212 completed, throughput=4.99MB/sec 2024-04-07T16:42:32.098 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-dleZWmZ7JnI21Ob off=10249127 len=1712711 2024-04-07T16:42:32.100 INFO:tasks.workunit.client.0.smithi100.stdout:op 1213 completed, throughput=5MB/sec 2024-04-07T16:42:32.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:42:32 smithi144 bash[66654]: cluster 2024-04-07T16:42:31.199875+0000 mgr.y (mgr.25175) 815 : cluster [DBG] pgmap v1457: 105 pgs: 105 active+clean; 126 GiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 2.7 MiB/s rd, 1.9 MiB/s wr, 5 op/s 2024-04-07T16:42:32.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:42:32 smithi144 bash[66654]: audit 2024-04-07T16:42:31.761331+0000 mon.a (mon.0) 2075 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:42:32.608 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:42:32 smithi144 bash[95233]: ts=2024-04-07T16:42:32.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:42:32.943 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:42:32 smithi100 bash[74942]: cluster 2024-04-07T16:42:31.199875+0000 mgr.y (mgr.25175) 815 : cluster [DBG] pgmap v1457: 105 pgs: 105 active+clean; 126 GiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 2.7 MiB/s rd, 1.9 MiB/s wr, 5 op/s 2024-04-07T16:42:32.943 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:42:32 smithi100 bash[74942]: audit 2024-04-07T16:42:31.761331+0000 mon.a (mon.0) 2075 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:42:32.943 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:42:32 smithi100 bash[76202]: cluster 2024-04-07T16:42:31.199875+0000 mgr.y (mgr.25175) 815 : cluster [DBG] pgmap v1457: 105 pgs: 105 active+clean; 126 GiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 2.7 MiB/s rd, 1.9 MiB/s wr, 5 op/s 2024-04-07T16:42:32.943 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:42:32 smithi100 bash[76202]: audit 2024-04-07T16:42:31.761331+0000 mon.a (mon.0) 2075 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:42:33.100 INFO:tasks.workunit.client.0.smithi100.stdout: 229: throughput=4.98MB/sec pending data=0 2024-04-07T16:42:33.100 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-zdA_4bGtBQGtI5Z off=2542371 len=823630 2024-04-07T16:42:33.108 INFO:tasks.workunit.client.0.smithi100.stdout:op 1214 completed, throughput=4.98MB/sec 2024-04-07T16:42:33.108 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-HsNRfPYtIbpwrty off=3372614 len=30528 2024-04-07T16:42:33.109 INFO:tasks.workunit.client.0.smithi100.stdout:op 1215 completed, throughput=4.98MB/sec 2024-04-07T16:42:33.110 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Nzns7WhPkF5s366 off=404811 len=298997 2024-04-07T16:42:33.115 INFO:tasks.workunit.client.0.smithi100.stdout:op 1216 completed, throughput=4.98MB/sec 2024-04-07T16:42:33.115 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-1ZTIRw2LyYj36ID off=2778184 len=16567 2024-04-07T16:42:33.116 INFO:tasks.workunit.client.0.smithi100.stdout:op 1217 completed, throughput=4.98MB/sec 2024-04-07T16:42:33.116 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-KeL39IJnWE8jvYh off=3880516 len=357182 2024-04-07T16:42:33.117 INFO:tasks.workunit.client.0.smithi100.stdout:op 1218 completed, throughput=4.99MB/sec 2024-04-07T16:42:33.118 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-sHz9UA9WDel6Imh off=8572264 len=1768102 2024-04-07T16:42:33.122 INFO:tasks.workunit.client.0.smithi100.stdout:op 1219 completed, throughput=4.99MB/sec 2024-04-07T16:42:33.122 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-521BFbimL9smY6f off=931946 len=625806 2024-04-07T16:42:33.124 INFO:tasks.workunit.client.0.smithi100.stdout:op 1220 completed, throughput=4.99MB/sec 2024-04-07T16:42:33.124 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-6JCHpPPZV-g_c9S off=2975653 len=871290 2024-04-07T16:42:33.126 INFO:tasks.workunit.client.0.smithi100.stdout:op 1221 completed, throughput=5MB/sec 2024-04-07T16:42:33.127 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Z70G1uST9s07l_6 off=3381812 len=770927 2024-04-07T16:42:33.134 INFO:tasks.workunit.client.0.smithi100.stdout:op 1222 completed, throughput=5MB/sec 2024-04-07T16:42:34.135 INFO:tasks.workunit.client.0.smithi100.stdout: 230: throughput=4.98MB/sec pending data=0 2024-04-07T16:42:34.135 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-QKMOxHCE_mMf4pD off=1572086 len=398827 2024-04-07T16:42:34.141 INFO:tasks.workunit.client.0.smithi100.stdout:op 1223 completed, throughput=4.98MB/sec 2024-04-07T16:42:34.141 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Zy1np7f4kmYhYSM off=2039953 len=793211 2024-04-07T16:42:34.152 INFO:tasks.workunit.client.0.smithi100.stdout:op 1224 completed, throughput=4.98MB/sec 2024-04-07T16:42:34.152 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-a_1k2YHg2iu6Y4b off=573458 len=1766609 2024-04-07T16:42:34.156 INFO:tasks.workunit.client.0.smithi100.stdout:op 1225 completed, throughput=4.99MB/sec 2024-04-07T16:42:34.156 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-tYwywUvUJGt0aVT off=2323000 len=1844974 2024-04-07T16:42:34.171 INFO:tasks.workunit.client.0.smithi100.stdout:op 1226 completed, throughput=5MB/sec 2024-04-07T16:42:34.171 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-IUbawJeQeNRk1te off=7750201 len=810660 2024-04-07T16:42:34.179 INFO:tasks.workunit.client.0.smithi100.stdout:op 1227 completed, throughput=5MB/sec 2024-04-07T16:42:34.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:42:34 smithi144 bash[66654]: cluster 2024-04-07T16:42:33.201031+0000 mgr.y (mgr.25175) 816 : cluster [DBG] pgmap v1458: 105 pgs: 105 active+clean; 126 GiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 2.9 MiB/s rd, 2.4 MiB/s wr, 5 op/s 2024-04-07T16:42:34.942 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:42:34 smithi100 bash[74942]: cluster 2024-04-07T16:42:33.201031+0000 mgr.y (mgr.25175) 816 : cluster [DBG] pgmap v1458: 105 pgs: 105 active+clean; 126 GiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 2.9 MiB/s rd, 2.4 MiB/s wr, 5 op/s 2024-04-07T16:42:34.943 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:42:34 smithi100 bash[76202]: cluster 2024-04-07T16:42:33.201031+0000 mgr.y (mgr.25175) 816 : cluster [DBG] pgmap v1458: 105 pgs: 105 active+clean; 126 GiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 2.9 MiB/s rd, 2.4 MiB/s wr, 5 op/s 2024-04-07T16:42:35.179 INFO:tasks.workunit.client.0.smithi100.stdout: 231: throughput=4.98MB/sec pending data=0 2024-04-07T16:42:35.179 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-0ulS4ppIh-O_Kfi off=2051144 len=1865607 2024-04-07T16:42:35.184 INFO:tasks.workunit.client.0.smithi100.stdout:op 1228 completed, throughput=4.99MB/sec 2024-04-07T16:42:35.184 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-XsMwGGnfDpDjqli off=8937885 len=143964 2024-04-07T16:42:35.189 INFO:tasks.workunit.client.0.smithi100.stdout:op 1229 completed, throughput=4.99MB/sec 2024-04-07T16:42:35.189 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-D1n8L5c-Oojsc9s off=98075 len=1406782 2024-04-07T16:42:35.200 INFO:tasks.workunit.client.0.smithi100.stdout:op 1230 completed, throughput=4.99MB/sec 2024-04-07T16:42:35.200 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-UkcMPrj8wQ4y-gv off=14149290 len=472433 2024-04-07T16:42:35.202 INFO:tasks.workunit.client.0.smithi100.stdout:op 1231 completed, throughput=5MB/sec 2024-04-07T16:42:35.202 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-cqSaI3PHgebO2PW off=5812016 len=906727 2024-04-07T16:42:35.210 INFO:tasks.workunit.client.0.smithi100.stdout:op 1232 completed, throughput=5MB/sec 2024-04-07T16:42:35.210 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-4lWZSMFYCSoC0KU off=1256347 len=1482792 2024-04-07T16:42:35.215 INFO:tasks.workunit.client.0.smithi100.stdout:op 1233 completed, throughput=5.01MB/sec 2024-04-07T16:42:36.215 INFO:tasks.workunit.client.0.smithi100.stdout: 232: throughput=4.98MB/sec pending data=0 2024-04-07T16:42:36.215 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-9lzy_1QYMD3A7d2 off=4029744 len=1292292 2024-04-07T16:42:36.219 INFO:tasks.workunit.client.0.smithi100.stdout:op 1234 completed, throughput=4.99MB/sec 2024-04-07T16:42:36.219 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-8oj4O_P46-MltuT off=17801150 len=1433088 2024-04-07T16:42:36.232 INFO:tasks.workunit.client.0.smithi100.stdout:op 1235 completed, throughput=4.99MB/sec 2024-04-07T16:42:36.232 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-l93MiKphu1ocGeT off=15822465 len=17965 2024-04-07T16:42:36.232 INFO:tasks.workunit.client.0.smithi100.stdout:op 1236 completed, throughput=4.99MB/sec 2024-04-07T16:42:36.232 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-O7_wfFgPL5rbbU- off=7789743 len=1263903 2024-04-07T16:42:36.243 INFO:tasks.workunit.client.0.smithi100.stdout:op 1237 completed, throughput=5MB/sec 2024-04-07T16:42:36.243 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-UGJOUpBlOHtmTw2 off=2918907 len=442521 2024-04-07T16:42:36.248 INFO:tasks.workunit.client.0.smithi100.stdout:op 1238 completed, throughput=5MB/sec 2024-04-07T16:42:36.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:42:36 smithi144 bash[66654]: cluster 2024-04-07T16:42:35.201987+0000 mgr.y (mgr.25175) 817 : cluster [DBG] pgmap v1459: 105 pgs: 105 active+clean; 126 GiB data, 4.7 GiB used, 711 GiB / 715 GiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 5 op/s 2024-04-07T16:42:36.943 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:42:36 smithi100 bash[74942]: cluster 2024-04-07T16:42:35.201987+0000 mgr.y (mgr.25175) 817 : cluster [DBG] pgmap v1459: 105 pgs: 105 active+clean; 126 GiB data, 4.7 GiB used, 711 GiB / 715 GiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 5 op/s 2024-04-07T16:42:36.943 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:42:36 smithi100 bash[76202]: cluster 2024-04-07T16:42:35.201987+0000 mgr.y (mgr.25175) 817 : cluster [DBG] pgmap v1459: 105 pgs: 105 active+clean; 126 GiB data, 4.7 GiB used, 711 GiB / 715 GiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 5 op/s 2024-04-07T16:42:37.248 INFO:tasks.workunit.client.0.smithi100.stdout: 233: throughput=4.98MB/sec pending data=0 2024-04-07T16:42:37.248 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-PQQV20kFf8HSCu9 off=9274897 len=273971 2024-04-07T16:42:37.254 INFO:tasks.workunit.client.0.smithi100.stdout:op 1239 completed, throughput=4.98MB/sec 2024-04-07T16:42:37.254 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-IDMnwf7oow2Pnf_ off=8982936 len=411902 2024-04-07T16:42:37.260 INFO:tasks.workunit.client.0.smithi100.stdout:op 1240 completed, throughput=4.98MB/sec 2024-04-07T16:42:37.260 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-UMNlVq1PL3TO22E off=8481817 len=650427 2024-04-07T16:42:37.267 INFO:tasks.workunit.client.0.smithi100.stdout:op 1241 completed, throughput=4.99MB/sec 2024-04-07T16:42:37.268 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-155kWxOl9LGYjvO off=7073020 len=1090751 2024-04-07T16:42:37.271 INFO:tasks.workunit.client.0.smithi100.stdout:op 1242 completed, throughput=4.99MB/sec 2024-04-07T16:42:37.271 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-RdPxDsuHAuwbGuD off=2807117 len=1254208 2024-04-07T16:42:37.284 INFO:tasks.workunit.client.0.smithi100.stdout:op 1243 completed, throughput=4.99MB/sec 2024-04-07T16:42:37.284 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-4_HdbIKa0lZBKmq off=1202976 len=1283261 2024-04-07T16:42:37.287 INFO:tasks.workunit.client.0.smithi100.stdout:op 1244 completed, throughput=5MB/sec 2024-04-07T16:42:37.288 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-RsYkgYGbej7g25z off=3106915 len=1813117 2024-04-07T16:42:37.302 INFO:tasks.workunit.client.0.smithi100.stdout:op 1245 completed, throughput=5.01MB/sec 2024-04-07T16:42:38.302 INFO:tasks.workunit.client.0.smithi100.stdout: 234: throughput=4.99MB/sec pending data=0 2024-04-07T16:42:38.302 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-SlLqsmedD5fWHwp off=9216488 len=402893 2024-04-07T16:42:38.303 INFO:tasks.workunit.client.0.smithi100.stdout:op 1246 completed, throughput=4.99MB/sec 2024-04-07T16:42:38.304 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-0h8-euPUppJF5sq off=218700 len=78777 2024-04-07T16:42:38.305 INFO:tasks.workunit.client.0.smithi100.stdout:op 1247 completed, throughput=4.99MB/sec 2024-04-07T16:42:38.305 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-_LPLHY1MrXx_aza off=7499444 len=200346 2024-04-07T16:42:38.306 INFO:tasks.workunit.client.0.smithi100.stdout:op 1248 completed, throughput=4.99MB/sec 2024-04-07T16:42:38.306 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-UrSo5FZnrLO_hVt off=3464730 len=219991 2024-04-07T16:42:38.307 INFO:tasks.workunit.client.0.smithi100.stdout:op 1249 completed, throughput=4.99MB/sec 2024-04-07T16:42:38.307 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-ClaxeUDrnwDg19F off=75395 len=1489880 2024-04-07T16:42:38.319 INFO:tasks.workunit.client.0.smithi100.stdout:op 1250 completed, throughput=4.99MB/sec 2024-04-07T16:42:38.319 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-aA_edeAIbM5Div7 off=11731541 len=424859 2024-04-07T16:42:38.321 INFO:tasks.workunit.client.0.smithi100.stdout:op 1251 completed, throughput=5MB/sec 2024-04-07T16:42:38.321 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-7vBEljvFnQq7X2l off=12183582 len=686889 2024-04-07T16:42:38.324 INFO:tasks.workunit.client.0.smithi100.stdout:op 1252 completed, throughput=5MB/sec 2024-04-07T16:42:38.324 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-2oFEvJ3k0TQIxBx off=2754719 len=736680 2024-04-07T16:42:38.326 INFO:tasks.workunit.client.0.smithi100.stdout:op 1253 completed, throughput=5MB/sec 2024-04-07T16:42:38.692 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:42:38 smithi100 bash[74942]: cluster 2024-04-07T16:42:37.203002+0000 mgr.y (mgr.25175) 818 : cluster [DBG] pgmap v1460: 105 pgs: 105 active+clean; 126 GiB data, 4.7 GiB used, 711 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.3 MiB/s wr, 5 op/s 2024-04-07T16:42:38.693 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:42:38 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:42:38] "GET /metrics HTTP/1.1" 200 34056 "" "Prometheus/2.43.0" 2024-04-07T16:42:38.693 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:42:38 smithi100 bash[76202]: cluster 2024-04-07T16:42:37.203002+0000 mgr.y (mgr.25175) 818 : cluster [DBG] pgmap v1460: 105 pgs: 105 active+clean; 126 GiB data, 4.7 GiB used, 711 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.3 MiB/s wr, 5 op/s 2024-04-07T16:42:38.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:42:38 smithi144 bash[66654]: cluster 2024-04-07T16:42:37.203002+0000 mgr.y (mgr.25175) 818 : cluster [DBG] pgmap v1460: 105 pgs: 105 active+clean; 126 GiB data, 4.7 GiB used, 711 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.3 MiB/s wr, 5 op/s 2024-04-07T16:42:39.327 INFO:tasks.workunit.client.0.smithi100.stdout: 235: throughput=4.98MB/sec pending data=0 2024-04-07T16:42:39.327 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-YcQ7fNITnloRdto off=4215343 len=2033094 2024-04-07T16:42:39.331 INFO:tasks.workunit.client.0.smithi100.stdout:op 1254 completed, throughput=4.99MB/sec 2024-04-07T16:42:39.331 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-W_mEY1CW2Mhemvo off=5747950 len=1765681 2024-04-07T16:42:39.338 INFO:tasks.workunit.client.0.smithi100.stdout:op 1255 completed, throughput=5MB/sec 2024-04-07T16:42:39.338 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-eG4syYfm20Usdu3 off=10535011 len=922387 2024-04-07T16:42:39.342 INFO:tasks.workunit.client.0.smithi100.stdout:op 1256 completed, throughput=5MB/sec 2024-04-07T16:42:39.342 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-1Frv4l_L2Vj1rCA off=1801331 len=701046 2024-04-07T16:42:39.350 INFO:tasks.workunit.client.0.smithi100.stdout:op 1257 completed, throughput=5MB/sec 2024-04-07T16:42:40.350 INFO:tasks.workunit.client.0.smithi100.stdout: 236: throughput=4.98MB/sec pending data=0 2024-04-07T16:42:40.350 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-iUE9knDqrdrrzxb off=7033498 len=340105 2024-04-07T16:42:40.356 INFO:tasks.workunit.client.0.smithi100.stdout:op 1258 completed, throughput=4.98MB/sec 2024-04-07T16:42:40.356 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-1QNE3EuJY6om-ec off=14041450 len=427479 2024-04-07T16:42:40.362 INFO:tasks.workunit.client.0.smithi100.stdout:op 1259 completed, throughput=4.98MB/sec 2024-04-07T16:42:40.362 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-KojEf5AnWV4gnQq off=9015363 len=1573556 2024-04-07T16:42:40.370 INFO:tasks.workunit.client.0.smithi100.stdout:op 1260 completed, throughput=4.99MB/sec 2024-04-07T16:42:40.370 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-zkCaximuaysMgdO off=6807920 len=705988 2024-04-07T16:42:40.370 INFO:tasks.workunit.client.0.smithi100.stdout:op 1261 completed, throughput=4.99MB/sec 2024-04-07T16:42:40.370 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-rSoc8uz1D4mac4- off=6951151 len=120240 2024-04-07T16:42:40.374 INFO:tasks.workunit.client.0.smithi100.stdout:op 1262 completed, throughput=4.99MB/sec 2024-04-07T16:42:40.375 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-TxQIagaTzBkoee9 off=1672654 len=345569 2024-04-07T16:42:40.376 INFO:tasks.workunit.client.0.smithi100.stdout:op 1263 completed, throughput=4.99MB/sec 2024-04-07T16:42:40.376 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-xyZUGV5ZPBF5QG_ off=308288 len=1028453 2024-04-07T16:42:40.379 INFO:tasks.workunit.client.0.smithi100.stdout:op 1264 completed, throughput=5MB/sec 2024-04-07T16:42:40.379 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-xZ9iDPWawtVq70F off=4265925 len=522045 2024-04-07T16:42:40.386 INFO:tasks.workunit.client.0.smithi100.stdout:op 1265 completed, throughput=5MB/sec 2024-04-07T16:42:40.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:42:40 smithi144 bash[66654]: cluster 2024-04-07T16:42:39.203840+0000 mgr.y (mgr.25175) 819 : cluster [DBG] pgmap v1461: 105 pgs: 105 active+clean; 126 GiB data, 4.7 GiB used, 711 GiB / 715 GiB avail; 2.1 MiB/s rd, 2.2 MiB/s wr, 4 op/s 2024-04-07T16:42:40.943 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:42:40 smithi100 bash[74942]: cluster 2024-04-07T16:42:39.203840+0000 mgr.y (mgr.25175) 819 : cluster [DBG] pgmap v1461: 105 pgs: 105 active+clean; 126 GiB data, 4.7 GiB used, 711 GiB / 715 GiB avail; 2.1 MiB/s rd, 2.2 MiB/s wr, 4 op/s 2024-04-07T16:42:40.943 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:42:40 smithi100 bash[76202]: cluster 2024-04-07T16:42:39.203840+0000 mgr.y (mgr.25175) 819 : cluster [DBG] pgmap v1461: 105 pgs: 105 active+clean; 126 GiB data, 4.7 GiB used, 711 GiB / 715 GiB avail; 2.1 MiB/s rd, 2.2 MiB/s wr, 4 op/s 2024-04-07T16:42:41.389 INFO:tasks.workunit.client.0.smithi100.stdout: 237: throughput=4.98MB/sec pending data=0 2024-04-07T16:42:41.390 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Ef9hUcUFoC2qkTR off=11036259 len=32657 2024-04-07T16:42:41.391 INFO:tasks.workunit.client.0.smithi100.stdout:op 1266 completed, throughput=4.98MB/sec 2024-04-07T16:42:41.391 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-n89XCFN8SxERoCN off=2563069 len=2085252 2024-04-07T16:42:41.395 INFO:tasks.workunit.client.0.smithi100.stdout:op 1267 completed, throughput=4.99MB/sec 2024-04-07T16:42:41.395 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-FcL7m1z5u5nDndn off=10700257 len=608316 2024-04-07T16:42:41.399 INFO:tasks.workunit.client.0.smithi100.stdout:op 1268 completed, throughput=4.99MB/sec 2024-04-07T16:42:41.399 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-blKfHtRN6Ce1bxh off=2500786 len=1551868 2024-04-07T16:42:41.403 INFO:tasks.workunit.client.0.smithi100.stdout:op 1269 completed, throughput=5MB/sec 2024-04-07T16:42:41.403 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-vlWchf18u4tcGKp off=240866 len=1831403 2024-04-07T16:42:41.419 INFO:tasks.workunit.client.0.smithi100.stdout:op 1270 completed, throughput=5MB/sec 2024-04-07T16:42:42.420 INFO:tasks.workunit.client.0.smithi100.stdout: 238: throughput=4.98MB/sec pending data=0 2024-04-07T16:42:42.420 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-TfpREWaYFqkN2aG off=5028938 len=255846 2024-04-07T16:42:42.422 INFO:tasks.workunit.client.0.smithi100.stdout:op 1271 completed, throughput=4.98MB/sec 2024-04-07T16:42:42.422 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-3ahc-iwoFNjIvMz off=3924300 len=73319 2024-04-07T16:42:42.427 INFO:tasks.workunit.client.0.smithi100.stdout:op 1272 completed, throughput=4.98MB/sec 2024-04-07T16:42:42.428 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Q0qjG920sY89z7G off=5452330 len=280314 2024-04-07T16:42:42.433 INFO:tasks.workunit.client.0.smithi100.stdout:op 1273 completed, throughput=4.99MB/sec 2024-04-07T16:42:42.433 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-LVz5dezEyoIB4z6 off=5355575 len=68683 2024-04-07T16:42:42.438 INFO:tasks.workunit.client.0.smithi100.stdout:op 1274 completed, throughput=4.99MB/sec 2024-04-07T16:42:42.438 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-H5rlJP0P2etY0c6 off=18766408 len=620392 2024-04-07T16:42:42.441 INFO:tasks.workunit.client.0.smithi100.stdout:op 1275 completed, throughput=4.99MB/sec 2024-04-07T16:42:42.441 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-OjZhSJqUefC6GnP off=1680562 len=175348 2024-04-07T16:42:42.446 INFO:tasks.workunit.client.0.smithi100.stdout:op 1276 completed, throughput=4.99MB/sec 2024-04-07T16:42:42.446 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-OYdoex0WAoj5uxy off=8993583 len=185139 2024-04-07T16:42:42.451 INFO:tasks.workunit.client.0.smithi100.stdout:op 1277 completed, throughput=4.99MB/sec 2024-04-07T16:42:42.451 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-ckTc8E_M1kqHhij off=8767705 len=1554102 2024-04-07T16:42:42.455 INFO:tasks.workunit.client.0.smithi100.stdout:op 1278 completed, throughput=4.99MB/sec 2024-04-07T16:42:42.455 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-o14U7C8seupHPoz off=7650964 len=117978 2024-04-07T16:42:42.460 INFO:tasks.workunit.client.0.smithi100.stdout:op 1279 completed, throughput=5MB/sec 2024-04-07T16:42:42.460 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-j9b0m5lqI8R8pU_ off=10027495 len=1232212 2024-04-07T16:42:42.470 INFO:tasks.workunit.client.0.smithi100.stdout:op 1280 completed, throughput=5MB/sec 2024-04-07T16:42:42.470 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-DkvlKhW2XWqjrCS off=62905 len=429763 2024-04-07T16:42:42.471 INFO:tasks.workunit.client.0.smithi100.stdout:op 1281 completed, throughput=5MB/sec 2024-04-07T16:42:42.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:42:42 smithi144 bash[66654]: cluster 2024-04-07T16:42:41.204752+0000 mgr.y (mgr.25175) 820 : cluster [DBG] pgmap v1462: 105 pgs: 105 active+clean; 126 GiB data, 4.7 GiB used, 711 GiB / 715 GiB avail; 2.0 MiB/s rd, 2.2 MiB/s wr, 4 op/s 2024-04-07T16:42:42.608 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:42:42 smithi144 bash[95233]: ts=2024-04-07T16:42:42.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:42:42.943 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:42:42 smithi100 bash[74942]: cluster 2024-04-07T16:42:41.204752+0000 mgr.y (mgr.25175) 820 : cluster [DBG] pgmap v1462: 105 pgs: 105 active+clean; 126 GiB data, 4.7 GiB used, 711 GiB / 715 GiB avail; 2.0 MiB/s rd, 2.2 MiB/s wr, 4 op/s 2024-04-07T16:42:42.943 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:42:42 smithi100 bash[76202]: cluster 2024-04-07T16:42:41.204752+0000 mgr.y (mgr.25175) 820 : cluster [DBG] pgmap v1462: 105 pgs: 105 active+clean; 126 GiB data, 4.7 GiB used, 711 GiB / 715 GiB avail; 2.0 MiB/s rd, 2.2 MiB/s wr, 4 op/s 2024-04-07T16:42:43.472 INFO:tasks.workunit.client.0.smithi100.stdout: 239: throughput=4.98MB/sec pending data=0 2024-04-07T16:42:43.472 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-v3v9dsxCQtUWlqi off=2425178 len=1443566 2024-04-07T16:42:43.484 INFO:tasks.workunit.client.0.smithi100.stdout:op 1282 completed, throughput=4.99MB/sec 2024-04-07T16:42:43.484 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-YHG0wYoZIHjdiI7 off=10535637 len=1227328 2024-04-07T16:42:43.493 INFO:tasks.workunit.client.0.smithi100.stdout:op 1283 completed, throughput=4.99MB/sec 2024-04-07T16:42:43.493 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-TPrHDx8FbiRE8Mn off=990065 len=1811583 2024-04-07T16:42:43.497 INFO:tasks.workunit.client.0.smithi100.stdout:op 1284 completed, throughput=5MB/sec 2024-04-07T16:42:43.497 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Re3X4izVOpsUE1d off=8805959 len=1028150 2024-04-07T16:42:43.500 INFO:tasks.workunit.client.0.smithi100.stdout:op 1285 completed, throughput=5MB/sec 2024-04-07T16:42:44.500 INFO:tasks.workunit.client.0.smithi100.stdout: 240: throughput=4.98MB/sec pending data=0 2024-04-07T16:42:44.500 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-zFJFjWqRuJijz9w off=1533068 len=586777 2024-04-07T16:42:44.502 INFO:tasks.workunit.client.0.smithi100.stdout:op 1286 completed, throughput=4.98MB/sec 2024-04-07T16:42:44.502 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-UjVxoDEVxTM7MRZ off=2290707 len=934263 2024-04-07T16:42:44.511 INFO:tasks.workunit.client.0.smithi100.stdout:op 1287 completed, throughput=4.99MB/sec 2024-04-07T16:42:44.511 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Ppu9JF4cH5GbOMc off=1180587 len=552683 2024-04-07T16:42:44.512 INFO:tasks.workunit.client.0.smithi100.stdout:op 1288 completed, throughput=4.99MB/sec 2024-04-07T16:42:44.512 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-bDHSmgHLBfwypVH off=2965644 len=1156898 2024-04-07T16:42:44.523 INFO:tasks.workunit.client.0.smithi100.stdout:op 1289 completed, throughput=4.99MB/sec 2024-04-07T16:42:44.523 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-qKD2AWyVN-KWvkk off=6378897 len=2089540 2024-04-07T16:42:44.541 INFO:tasks.workunit.client.0.smithi100.stdout:op 1290 completed, throughput=5MB/sec 2024-04-07T16:42:44.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:42:44 smithi144 bash[66654]: cluster 2024-04-07T16:42:43.205806+0000 mgr.y (mgr.25175) 821 : cluster [DBG] pgmap v1463: 105 pgs: 105 active+clean; 126 GiB data, 4.7 GiB used, 711 GiB / 715 GiB avail; 2.8 MiB/s rd, 2.7 MiB/s wr, 6 op/s 2024-04-07T16:42:44.943 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:42:44 smithi100 bash[74942]: cluster 2024-04-07T16:42:43.205806+0000 mgr.y (mgr.25175) 821 : cluster [DBG] pgmap v1463: 105 pgs: 105 active+clean; 126 GiB data, 4.7 GiB used, 711 GiB / 715 GiB avail; 2.8 MiB/s rd, 2.7 MiB/s wr, 6 op/s 2024-04-07T16:42:44.943 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:42:44 smithi100 bash[76202]: cluster 2024-04-07T16:42:43.205806+0000 mgr.y (mgr.25175) 821 : cluster [DBG] pgmap v1463: 105 pgs: 105 active+clean; 126 GiB data, 4.7 GiB used, 711 GiB / 715 GiB avail; 2.8 MiB/s rd, 2.7 MiB/s wr, 6 op/s 2024-04-07T16:42:45.541 INFO:tasks.workunit.client.0.smithi100.stdout: 241: throughput=4.98MB/sec pending data=0 2024-04-07T16:42:45.541 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-0tk4HFR84lP6WWK off=12945759 len=1995193 2024-04-07T16:42:45.560 INFO:tasks.workunit.client.0.smithi100.stdout:op 1291 completed, throughput=4.99MB/sec 2024-04-07T16:42:45.560 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-n1gbSIKScqTi3uH off=14717739 len=767528 2024-04-07T16:42:45.568 INFO:tasks.workunit.client.0.smithi100.stdout:op 1292 completed, throughput=4.99MB/sec 2024-04-07T16:42:45.569 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-p3rqPeAZBJGifb2 off=9621382 len=107946 2024-04-07T16:42:45.570 INFO:tasks.workunit.client.0.smithi100.stdout:op 1293 completed, throughput=4.99MB/sec 2024-04-07T16:42:45.570 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-qj-fUW2uN23xl5n off=7310362 len=1480202 2024-04-07T16:42:45.582 INFO:tasks.workunit.client.0.smithi100.stdout:op 1294 completed, throughput=5MB/sec 2024-04-07T16:42:45.582 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-rrlUbYmpzGHcU9L off=14321763 len=1577057 2024-04-07T16:42:45.594 INFO:tasks.workunit.client.0.smithi100.stdout:op 1295 completed, throughput=5MB/sec 2024-04-07T16:42:46.594 INFO:tasks.workunit.client.0.smithi100.stdout: 242: throughput=4.98MB/sec pending data=0 2024-04-07T16:42:46.595 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-zyp4quOpK_rKciL off=13776627 len=1165566 2024-04-07T16:42:46.599 INFO:tasks.workunit.client.0.smithi100.stdout:op 1296 completed, throughput=4.99MB/sec 2024-04-07T16:42:46.599 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-tAP_mAlOiXfFxOV off=20491125 len=1931333 2024-04-07T16:42:46.614 INFO:tasks.workunit.client.0.smithi100.stdout:op 1297 completed, throughput=4.99MB/sec 2024-04-07T16:42:46.614 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-8WOu8bqi4RbC9R3 off=1216416 len=1312078 2024-04-07T16:42:46.627 INFO:tasks.workunit.client.0.smithi100.stdout:op 1298 completed, throughput=5MB/sec 2024-04-07T16:42:46.627 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj--luq0-Ui-BiCfnW off=5472793 len=685997 2024-04-07T16:42:46.629 INFO:tasks.workunit.client.0.smithi100.stdout:op 1299 completed, throughput=5MB/sec 2024-04-07T16:42:46.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:42:46 smithi144 bash[66654]: cluster 2024-04-07T16:42:45.206763+0000 mgr.y (mgr.25175) 822 : cluster [DBG] pgmap v1464: 105 pgs: 105 active+clean; 126 GiB data, 4.7 GiB used, 711 GiB / 715 GiB avail; 3.0 MiB/s rd, 2.2 MiB/s wr, 5 op/s 2024-04-07T16:42:46.942 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:42:46 smithi100 bash[74942]: cluster 2024-04-07T16:42:45.206763+0000 mgr.y (mgr.25175) 822 : cluster [DBG] pgmap v1464: 105 pgs: 105 active+clean; 126 GiB data, 4.7 GiB used, 711 GiB / 715 GiB avail; 3.0 MiB/s rd, 2.2 MiB/s wr, 5 op/s 2024-04-07T16:42:46.943 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:42:46 smithi100 bash[76202]: cluster 2024-04-07T16:42:45.206763+0000 mgr.y (mgr.25175) 822 : cluster [DBG] pgmap v1464: 105 pgs: 105 active+clean; 126 GiB data, 4.7 GiB used, 711 GiB / 715 GiB avail; 3.0 MiB/s rd, 2.2 MiB/s wr, 5 op/s 2024-04-07T16:42:47.630 INFO:tasks.workunit.client.0.smithi100.stdout: 243: throughput=4.98MB/sec pending data=0 2024-04-07T16:42:47.630 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Qatule9fNnKzoaI off=14874483 len=1821015 2024-04-07T16:42:47.643 INFO:tasks.workunit.client.0.smithi100.stdout:op 1300 completed, throughput=4.99MB/sec 2024-04-07T16:42:47.643 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-jYBDUPhrmfXR9R2 off=9886209 len=1434312 2024-04-07T16:42:47.645 INFO:tasks.workunit.client.0.smithi100.stdout:op 1301 completed, throughput=4.99MB/sec 2024-04-07T16:42:47.645 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-9ALiB0BqNFUdROH off=763849 len=1315207 2024-04-07T16:42:47.650 INFO:tasks.workunit.client.0.smithi100.stdout:op 1302 completed, throughput=5MB/sec 2024-04-07T16:42:47.650 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-6eW72LwLIetthaB off=5216062 len=243193 2024-04-07T16:42:47.651 INFO:tasks.workunit.client.0.smithi100.stdout:op 1303 completed, throughput=5MB/sec 2024-04-07T16:42:47.651 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-9_VXCrdN5BX7t6s off=5382322 len=1052464 2024-04-07T16:42:47.659 INFO:tasks.workunit.client.0.smithi100.stdout:op 1304 completed, throughput=5MB/sec 2024-04-07T16:42:47.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:42:47 smithi144 bash[66654]: audit 2024-04-07T16:42:46.761771+0000 mon.a (mon.0) 2076 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:42:47.943 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:42:47 smithi100 bash[74942]: audit 2024-04-07T16:42:46.761771+0000 mon.a (mon.0) 2076 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:42:47.943 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:42:47 smithi100 bash[76202]: audit 2024-04-07T16:42:46.761771+0000 mon.a (mon.0) 2076 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:42:48.659 INFO:tasks.workunit.client.0.smithi100.stdout: 244: throughput=4.98MB/sec pending data=0 2024-04-07T16:42:48.660 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-PBgmTYTxktDMRJV off=1717318 len=1308137 2024-04-07T16:42:48.670 INFO:tasks.workunit.client.0.smithi100.stdout:op 1305 completed, throughput=4.99MB/sec 2024-04-07T16:42:48.670 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-_0tkesUiwdETCwb off=8296328 len=1573166 2024-04-07T16:42:48.674 INFO:tasks.workunit.client.0.smithi100.stdout:op 1306 completed, throughput=4.99MB/sec 2024-04-07T16:42:48.674 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-2H8ZEkN-3A3rVcw off=5641464 len=994928 2024-04-07T16:42:48.677 INFO:tasks.workunit.client.0.smithi100.stdout:op 1307 completed, throughput=5MB/sec 2024-04-07T16:42:48.677 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Is5kLp-wpxV6dmn off=0 len=1233537 2024-04-07T16:42:48.687 INFO:tasks.workunit.client.0.smithi100.stdout:op 1308 completed, throughput=5MB/sec 2024-04-07T16:42:48.692 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:42:48 smithi100 bash[74942]: cluster 2024-04-07T16:42:47.207818+0000 mgr.y (mgr.25175) 823 : cluster [DBG] pgmap v1465: 105 pgs: 105 active+clean; 126 GiB data, 4.7 GiB used, 710 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.5 MiB/s wr, 6 op/s 2024-04-07T16:42:48.693 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:42:48 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:42:48] "GET /metrics HTTP/1.1" 200 34058 "" "Prometheus/2.43.0" 2024-04-07T16:42:48.693 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:42:48 smithi100 bash[76202]: cluster 2024-04-07T16:42:47.207818+0000 mgr.y (mgr.25175) 823 : cluster [DBG] pgmap v1465: 105 pgs: 105 active+clean; 126 GiB data, 4.7 GiB used, 710 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.5 MiB/s wr, 6 op/s 2024-04-07T16:42:48.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:42:48 smithi144 bash[66654]: cluster 2024-04-07T16:42:47.207818+0000 mgr.y (mgr.25175) 823 : cluster [DBG] pgmap v1465: 105 pgs: 105 active+clean; 126 GiB data, 4.7 GiB used, 710 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.5 MiB/s wr, 6 op/s 2024-04-07T16:42:49.687 INFO:tasks.workunit.client.0.smithi100.stdout: 245: throughput=4.98MB/sec pending data=0 2024-04-07T16:42:49.687 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-UNDqSmfl553N2il off=5119020 len=730132 2024-04-07T16:42:49.694 INFO:tasks.workunit.client.0.smithi100.stdout:op 1309 completed, throughput=4.99MB/sec 2024-04-07T16:42:49.694 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Kc-XETDV1Ew8wQi off=4275513 len=1433009 2024-04-07T16:42:49.708 INFO:tasks.workunit.client.0.smithi100.stdout:op 1310 completed, throughput=4.99MB/sec 2024-04-07T16:42:49.708 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-BOLDuq2x_AMKZrp off=4808641 len=732500 2024-04-07T16:42:49.710 INFO:tasks.workunit.client.0.smithi100.stdout:op 1311 completed, throughput=4.99MB/sec 2024-04-07T16:42:49.710 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-AqjEDplrfG5ei8l off=48127 len=896809 2024-04-07T16:42:49.713 INFO:tasks.workunit.client.0.smithi100.stdout:op 1312 completed, throughput=5MB/sec 2024-04-07T16:42:49.713 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-rQJYcGmKIN846Om off=2489270 len=1535126 2024-04-07T16:42:49.726 INFO:tasks.workunit.client.0.smithi100.stdout:op 1313 completed, throughput=5MB/sec 2024-04-07T16:42:50.726 INFO:tasks.workunit.client.0.smithi100.stdout: 246: throughput=4.98MB/sec pending data=0 2024-04-07T16:42:50.726 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-X5lKnPn6ytY-9eR off=1392284 len=1605069 2024-04-07T16:42:50.730 INFO:tasks.workunit.client.0.smithi100.stdout:op 1314 completed, throughput=4.99MB/sec 2024-04-07T16:42:50.730 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-MKgT7UBJklT8E2i off=1282875 len=1353133 2024-04-07T16:42:50.732 INFO:tasks.workunit.client.0.smithi100.stdout:op 1315 completed, throughput=4.99MB/sec 2024-04-07T16:42:50.732 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-ypDsKjRSgyOoUXM off=4781744 len=415732 2024-04-07T16:42:50.738 INFO:tasks.workunit.client.0.smithi100.stdout:op 1316 completed, throughput=4.99MB/sec 2024-04-07T16:42:50.738 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-h7uMrlyNwl8tTb6 off=2820293 len=2092602 2024-04-07T16:42:50.745 INFO:tasks.workunit.client.0.smithi100.stdout:op 1317 completed, throughput=5MB/sec 2024-04-07T16:42:50.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:42:50 smithi144 bash[66654]: cluster 2024-04-07T16:42:49.208650+0000 mgr.y (mgr.25175) 824 : cluster [DBG] pgmap v1466: 105 pgs: 105 active+clean; 126 GiB data, 4.7 GiB used, 710 GiB / 715 GiB avail; 2.4 MiB/s rd, 2.2 MiB/s wr, 5 op/s 2024-04-07T16:42:50.942 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:42:50 smithi100 bash[74942]: cluster 2024-04-07T16:42:49.208650+0000 mgr.y (mgr.25175) 824 : cluster [DBG] pgmap v1466: 105 pgs: 105 active+clean; 126 GiB data, 4.7 GiB used, 710 GiB / 715 GiB avail; 2.4 MiB/s rd, 2.2 MiB/s wr, 5 op/s 2024-04-07T16:42:50.943 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:42:50 smithi100 bash[76202]: cluster 2024-04-07T16:42:49.208650+0000 mgr.y (mgr.25175) 824 : cluster [DBG] pgmap v1466: 105 pgs: 105 active+clean; 126 GiB data, 4.7 GiB used, 710 GiB / 715 GiB avail; 2.4 MiB/s rd, 2.2 MiB/s wr, 5 op/s 2024-04-07T16:42:51.745 INFO:tasks.workunit.client.0.smithi100.stdout: 247: throughput=4.98MB/sec pending data=0 2024-04-07T16:42:51.746 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-NEoF7zTqjflOlQf off=6093231 len=1413179 2024-04-07T16:42:51.757 INFO:tasks.workunit.client.0.smithi100.stdout:op 1318 completed, throughput=4.99MB/sec 2024-04-07T16:42:51.757 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-GQl2hPWwFROzzYe off=2533241 len=1268874 2024-04-07T16:42:51.769 INFO:tasks.workunit.client.0.smithi100.stdout:op 1319 completed, throughput=4.99MB/sec 2024-04-07T16:42:51.769 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-g6qJK9VHVqagyob off=13662982 len=424385 2024-04-07T16:42:51.771 INFO:tasks.workunit.client.0.smithi100.stdout:op 1320 completed, throughput=4.99MB/sec 2024-04-07T16:42:51.771 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-zUPsCH5AuN4ZYDd off=8327635 len=2010034 2024-04-07T16:42:51.785 INFO:tasks.workunit.client.0.smithi100.stdout:op 1321 completed, throughput=5MB/sec 2024-04-07T16:42:51.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:42:51 smithi144 bash[66654]: audit 2024-04-07T16:42:50.826116+0000 mon.a (mon.0) 2077 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:42:51.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:42:51 smithi144 bash[66654]: audit 2024-04-07T16:42:51.560392+0000 mon.a (mon.0) 2078 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:42:51.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:42:51 smithi144 bash[66654]: audit 2024-04-07T16:42:51.561905+0000 mon.a (mon.0) 2079 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:42:51.943 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:42:51 smithi100 bash[74942]: audit 2024-04-07T16:42:50.826116+0000 mon.a (mon.0) 2077 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:42:51.943 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:42:51 smithi100 bash[74942]: audit 2024-04-07T16:42:51.560392+0000 mon.a (mon.0) 2078 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:42:51.943 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:42:51 smithi100 bash[74942]: audit 2024-04-07T16:42:51.561905+0000 mon.a (mon.0) 2079 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:42:51.943 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:42:51 smithi100 bash[76202]: audit 2024-04-07T16:42:50.826116+0000 mon.a (mon.0) 2077 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:42:51.943 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:42:51 smithi100 bash[76202]: audit 2024-04-07T16:42:51.560392+0000 mon.a (mon.0) 2078 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:42:51.943 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:42:51 smithi100 bash[76202]: audit 2024-04-07T16:42:51.561905+0000 mon.a (mon.0) 2079 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:42:52.567 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:42:52 smithi144 bash[95233]: ts=2024-04-07T16:42:52.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:42:52.786 INFO:tasks.workunit.client.0.smithi100.stdout: 248: throughput=4.98MB/sec pending data=0 2024-04-07T16:42:52.786 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-MwFRsZj8Kizi_E1 off=2571700 len=1590256 2024-04-07T16:42:52.791 INFO:tasks.workunit.client.0.smithi100.stdout:op 1322 completed, throughput=4.99MB/sec 2024-04-07T16:42:52.792 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-B1oI48FE4zTabHc off=11842482 len=917590 2024-04-07T16:42:52.794 INFO:tasks.workunit.client.0.smithi100.stdout:op 1323 completed, throughput=4.99MB/sec 2024-04-07T16:42:52.794 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-IxKBgF0f2HoQ_F2 off=395824 len=960578 2024-04-07T16:42:52.798 INFO:tasks.workunit.client.0.smithi100.stdout:op 1324 completed, throughput=4.99MB/sec 2024-04-07T16:42:52.798 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-bStduIwt2yprckH off=986552 len=346168 2024-04-07T16:42:52.803 INFO:tasks.workunit.client.0.smithi100.stdout:op 1325 completed, throughput=5MB/sec 2024-04-07T16:42:52.803 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-6D9uQVZrMb391Mf off=1774091 len=689534 2024-04-07T16:42:52.811 INFO:tasks.workunit.client.0.smithi100.stdout:op 1326 completed, throughput=5MB/sec 2024-04-07T16:42:52.811 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-sFYymCSMcfrTmqo off=3316806 len=1514534 2024-04-07T16:42:52.814 INFO:tasks.workunit.client.0.smithi100.stdout:op 1327 completed, throughput=5MB/sec 2024-04-07T16:42:52.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:42:52 smithi144 bash[66654]: cluster 2024-04-07T16:42:51.209522+0000 mgr.y (mgr.25175) 825 : cluster [DBG] pgmap v1467: 105 pgs: 105 active+clean; 126 GiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 2.6 MiB/s rd, 2.9 MiB/s wr, 6 op/s 2024-04-07T16:42:52.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:42:52 smithi144 bash[66654]: audit 2024-04-07T16:42:51.589808+0000 mon.a (mon.0) 2080 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:42:52.943 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:42:52 smithi100 bash[74942]: cluster 2024-04-07T16:42:51.209522+0000 mgr.y (mgr.25175) 825 : cluster [DBG] pgmap v1467: 105 pgs: 105 active+clean; 126 GiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 2.6 MiB/s rd, 2.9 MiB/s wr, 6 op/s 2024-04-07T16:42:52.943 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:42:52 smithi100 bash[74942]: audit 2024-04-07T16:42:51.589808+0000 mon.a (mon.0) 2080 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:42:52.943 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:42:52 smithi100 bash[76202]: cluster 2024-04-07T16:42:51.209522+0000 mgr.y (mgr.25175) 825 : cluster [DBG] pgmap v1467: 105 pgs: 105 active+clean; 126 GiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 2.6 MiB/s rd, 2.9 MiB/s wr, 6 op/s 2024-04-07T16:42:52.943 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:42:52 smithi100 bash[76202]: audit 2024-04-07T16:42:51.589808+0000 mon.a (mon.0) 2080 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:42:53.815 INFO:tasks.workunit.client.0.smithi100.stdout: 249: throughput=4.98MB/sec pending data=0 2024-04-07T16:42:53.815 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Hv2x2xwX6hI67zp off=3586850 len=1291767 2024-04-07T16:42:53.820 INFO:tasks.workunit.client.0.smithi100.stdout:op 1328 completed, throughput=4.99MB/sec 2024-04-07T16:42:53.820 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-1ihMEgkwKAJMSOl off=322550 len=810966 2024-04-07T16:42:53.828 INFO:tasks.workunit.client.0.smithi100.stdout:op 1329 completed, throughput=4.99MB/sec 2024-04-07T16:42:53.828 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-XjWjEYI-9J1nYAw off=16669376 len=721986 2024-04-07T16:42:53.835 INFO:tasks.workunit.client.0.smithi100.stdout:op 1330 completed, throughput=4.99MB/sec 2024-04-07T16:42:53.836 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-4HGPoT-SP4PkVqB off=18675958 len=269029 2024-04-07T16:42:53.837 INFO:tasks.workunit.client.0.smithi100.stdout:op 1331 completed, throughput=4.99MB/sec 2024-04-07T16:42:53.837 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-DUb_75HQrNiQAfB off=73314 len=1913507 2024-04-07T16:42:53.842 INFO:tasks.workunit.client.0.smithi100.stdout:op 1332 completed, throughput=5MB/sec 2024-04-07T16:42:53.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:42:53 smithi144 bash[66654]: cluster 2024-04-07T16:42:53.210611+0000 mgr.y (mgr.25175) 826 : cluster [DBG] pgmap v1468: 105 pgs: 105 active+clean; 126 GiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 2.8 MiB/s rd, 3.3 MiB/s wr, 6 op/s 2024-04-07T16:42:53.943 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:42:53 smithi100 bash[74942]: cluster 2024-04-07T16:42:53.210611+0000 mgr.y (mgr.25175) 826 : cluster [DBG] pgmap v1468: 105 pgs: 105 active+clean; 126 GiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 2.8 MiB/s rd, 3.3 MiB/s wr, 6 op/s 2024-04-07T16:42:53.943 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:42:53 smithi100 bash[76202]: cluster 2024-04-07T16:42:53.210611+0000 mgr.y (mgr.25175) 826 : cluster [DBG] pgmap v1468: 105 pgs: 105 active+clean; 126 GiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 2.8 MiB/s rd, 3.3 MiB/s wr, 6 op/s 2024-04-07T16:42:54.842 INFO:tasks.workunit.client.0.smithi100.stdout: 250: throughput=4.98MB/sec pending data=0 2024-04-07T16:42:54.842 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-MdivRxqyKQAyKfK off=5732601 len=361007 2024-04-07T16:42:54.845 INFO:tasks.workunit.client.0.smithi100.stdout:op 1333 completed, throughput=4.98MB/sec 2024-04-07T16:42:54.845 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-_OUz_b-iD8-a7ih off=594865 len=1471548 2024-04-07T16:42:54.849 INFO:tasks.workunit.client.0.smithi100.stdout:op 1334 completed, throughput=4.99MB/sec 2024-04-07T16:42:54.849 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-RAl35_wwRrrRVNv off=4073119 len=1932908 2024-04-07T16:42:54.864 INFO:tasks.workunit.client.0.smithi100.stdout:op 1335 completed, throughput=5MB/sec 2024-04-07T16:42:54.865 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-57jX2VkvKOriu-v off=921457 len=1457108 2024-04-07T16:42:54.875 INFO:tasks.workunit.client.0.smithi100.stdout:op 1336 completed, throughput=5MB/sec 2024-04-07T16:42:55.875 INFO:tasks.workunit.client.0.smithi100.stdout: 251: throughput=4.98MB/sec pending data=0 2024-04-07T16:42:55.875 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-_fILFGogFg7HIXi off=17528342 len=677431 2024-04-07T16:42:55.877 INFO:tasks.workunit.client.0.smithi100.stdout:op 1337 completed, throughput=4.98MB/sec 2024-04-07T16:42:55.878 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-wMPVhMs9yHtvSyz off=3041393 len=1884698 2024-04-07T16:42:55.894 INFO:tasks.workunit.client.0.smithi100.stdout:op 1338 completed, throughput=4.99MB/sec 2024-04-07T16:42:55.894 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-eIyRQFR2pm76pvU off=1655300 len=1124288 2024-04-07T16:42:55.898 INFO:tasks.workunit.client.0.smithi100.stdout:op 1339 completed, throughput=4.99MB/sec 2024-04-07T16:42:55.898 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-eJrI0kN7EX5PAgo off=6506641 len=1622437 2024-04-07T16:42:55.905 INFO:tasks.workunit.client.0.smithi100.stdout:op 1340 completed, throughput=5MB/sec 2024-04-07T16:42:56.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:42:56 smithi144 bash[66654]: cluster 2024-04-07T16:42:55.211716+0000 mgr.y (mgr.25175) 827 : cluster [DBG] pgmap v1469: 105 pgs: 105 active+clean; 126 GiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 2.1 MiB/s rd, 3.0 MiB/s wr, 5 op/s 2024-04-07T16:42:56.693 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:42:56 smithi100 bash[74942]: cluster 2024-04-07T16:42:55.211716+0000 mgr.y (mgr.25175) 827 : cluster [DBG] pgmap v1469: 105 pgs: 105 active+clean; 126 GiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 2.1 MiB/s rd, 3.0 MiB/s wr, 5 op/s 2024-04-07T16:42:56.693 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:42:56 smithi100 bash[76202]: cluster 2024-04-07T16:42:55.211716+0000 mgr.y (mgr.25175) 827 : cluster [DBG] pgmap v1469: 105 pgs: 105 active+clean; 126 GiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 2.1 MiB/s rd, 3.0 MiB/s wr, 5 op/s 2024-04-07T16:42:56.904 INFO:tasks.workunit.client.0.smithi100.stdout: 252: throughput=4.98MB/sec pending data=0 2024-04-07T16:42:56.904 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj--qCudJDgMtES7px off=3827001 len=619021 2024-04-07T16:42:56.907 INFO:tasks.workunit.client.0.smithi100.stdout:op 1341 completed, throughput=4.98MB/sec 2024-04-07T16:42:56.907 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-nnnTnKdJFjN4hHG off=5991983 len=1103142 2024-04-07T16:42:56.911 INFO:tasks.workunit.client.0.smithi100.stdout:op 1342 completed, throughput=4.99MB/sec 2024-04-07T16:42:56.911 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-V60jaIL1rk3iDx- off=3256437 len=1089233 2024-04-07T16:42:56.920 INFO:tasks.workunit.client.0.smithi100.stdout:op 1343 completed, throughput=4.99MB/sec 2024-04-07T16:42:56.920 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-mHyhHm6EeKllSLS off=2911803 len=109220 2024-04-07T16:42:56.921 INFO:tasks.workunit.client.0.smithi100.stdout:op 1344 completed, throughput=4.99MB/sec 2024-04-07T16:42:56.921 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-29JHTgOkTfmDfqB off=8227596 len=1926921 2024-04-07T16:42:56.926 INFO:tasks.workunit.client.0.smithi100.stdout:op 1345 completed, throughput=5MB/sec 2024-04-07T16:42:56.926 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-4NEq9JSG6Wch3gs off=18448088 len=676293 2024-04-07T16:42:56.930 INFO:tasks.workunit.client.0.smithi100.stdout:op 1346 completed, throughput=5MB/sec 2024-04-07T16:42:57.929 INFO:tasks.workunit.client.0.smithi100.stdout: 253: throughput=4.98MB/sec pending data=0 2024-04-07T16:42:57.929 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-I7T8zEwwtbvxOXK off=1001262 len=1116842 2024-04-07T16:42:57.938 INFO:tasks.workunit.client.0.smithi100.stdout:op 1347 completed, throughput=4.99MB/sec 2024-04-07T16:42:57.938 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-k3VqK6dBaO6TJNl off=18901502 len=662090 2024-04-07T16:42:57.945 INFO:tasks.workunit.client.0.smithi100.stdout:op 1348 completed, throughput=4.99MB/sec 2024-04-07T16:42:57.945 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-NMrm0IPtXXP6Or0 off=12834879 len=1987610 2024-04-07T16:42:57.962 INFO:tasks.workunit.client.0.smithi100.stdout:op 1349 completed, throughput=5MB/sec 2024-04-07T16:42:57.963 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-RWXpu3jT-lNdX_O off=44368 len=1434105 2024-04-07T16:42:57.973 INFO:tasks.workunit.client.0.smithi100.stdout:op 1350 completed, throughput=5MB/sec 2024-04-07T16:42:58.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:42:58 smithi144 bash[66654]: cluster 2024-04-07T16:42:57.212792+0000 mgr.y (mgr.25175) 828 : cluster [DBG] pgmap v1470: 105 pgs: 105 active+clean; 126 GiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 2.4 MiB/s rd, 2.7 MiB/s wr, 4 op/s 2024-04-07T16:42:58.693 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:42:58 smithi100 bash[74942]: cluster 2024-04-07T16:42:57.212792+0000 mgr.y (mgr.25175) 828 : cluster [DBG] pgmap v1470: 105 pgs: 105 active+clean; 126 GiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 2.4 MiB/s rd, 2.7 MiB/s wr, 4 op/s 2024-04-07T16:42:58.693 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:42:58 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:42:58] "GET /metrics HTTP/1.1" 200 34058 "" "Prometheus/2.43.0" 2024-04-07T16:42:58.693 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:42:58 smithi100 bash[76202]: cluster 2024-04-07T16:42:57.212792+0000 mgr.y (mgr.25175) 828 : cluster [DBG] pgmap v1470: 105 pgs: 105 active+clean; 126 GiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 2.4 MiB/s rd, 2.7 MiB/s wr, 4 op/s 2024-04-07T16:42:58.974 INFO:tasks.workunit.client.0.smithi100.stdout: 254: throughput=4.98MB/sec pending data=0 2024-04-07T16:42:58.974 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-VejP1rPMXjHdFg9 off=345645 len=944476 2024-04-07T16:42:58.976 INFO:tasks.workunit.client.0.smithi100.stdout:op 1351 completed, throughput=4.98MB/sec 2024-04-07T16:42:58.976 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-m5ctNgGugkd_vu8 off=14034816 len=1902860 2024-04-07T16:42:58.993 INFO:tasks.workunit.client.0.smithi100.stdout:op 1352 completed, throughput=4.99MB/sec 2024-04-07T16:42:58.993 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-SPJsDQn1BFz1gSV off=2578217 len=2013383 2024-04-07T16:42:58.998 INFO:tasks.workunit.client.0.smithi100.stdout:op 1353 completed, throughput=5MB/sec 2024-04-07T16:42:58.998 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-ncxzJfXKYGwvQgZ off=1058963 len=616603 2024-04-07T16:42:59.000 INFO:tasks.workunit.client.0.smithi100.stdout:op 1354 completed, throughput=5MB/sec 2024-04-07T16:43:00.000 INFO:tasks.workunit.client.0.smithi100.stdout: 255: throughput=4.98MB/sec pending data=0 2024-04-07T16:43:00.000 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-3ahc-iwoFNjIvMz off=5935568 len=381672 2024-04-07T16:43:00.002 INFO:tasks.workunit.client.0.smithi100.stdout:op 1355 completed, throughput=4.98MB/sec 2024-04-07T16:43:00.002 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-dWQ7UIsT5_8oJs6 off=16591203 len=486023 2024-04-07T16:43:00.009 INFO:tasks.workunit.client.0.smithi100.stdout:op 1356 completed, throughput=4.98MB/sec 2024-04-07T16:43:00.009 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-GvOUBZnDR4uOI-h off=542864 len=2043312 2024-04-07T16:43:00.025 INFO:tasks.workunit.client.0.smithi100.stdout:op 1357 completed, throughput=4.99MB/sec 2024-04-07T16:43:00.025 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-O77jgCgtl5nKYA4 off=4775038 len=1345917 2024-04-07T16:43:00.036 INFO:tasks.workunit.client.0.smithi100.stdout:op 1358 completed, throughput=5MB/sec 2024-04-07T16:43:00.036 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-wBDREaLw26Rocnn off=5028678 len=2028137 2024-04-07T16:43:00.053 INFO:tasks.workunit.client.0.smithi100.stdout:op 1359 completed, throughput=5MB/sec 2024-04-07T16:43:00.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:43:00 smithi144 bash[66654]: cluster 2024-04-07T16:42:59.213671+0000 mgr.y (mgr.25175) 829 : cluster [DBG] pgmap v1471: 105 pgs: 105 active+clean; 126 GiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 2.3 MiB/s rd, 2.2 MiB/s wr, 3 op/s 2024-04-07T16:43:00.693 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:43:00 smithi100 bash[74942]: cluster 2024-04-07T16:42:59.213671+0000 mgr.y (mgr.25175) 829 : cluster [DBG] pgmap v1471: 105 pgs: 105 active+clean; 126 GiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 2.3 MiB/s rd, 2.2 MiB/s wr, 3 op/s 2024-04-07T16:43:00.693 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:43:00 smithi100 bash[76202]: cluster 2024-04-07T16:42:59.213671+0000 mgr.y (mgr.25175) 829 : cluster [DBG] pgmap v1471: 105 pgs: 105 active+clean; 126 GiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 2.3 MiB/s rd, 2.2 MiB/s wr, 3 op/s 2024-04-07T16:43:01.054 INFO:tasks.workunit.client.0.smithi100.stdout: 256: throughput=4.98MB/sec pending data=0 2024-04-07T16:43:01.054 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-U4YvWeijjI0yQUM off=5875191 len=833164 2024-04-07T16:43:01.056 INFO:tasks.workunit.client.0.smithi100.stdout:op 1360 completed, throughput=4.99MB/sec 2024-04-07T16:43:01.056 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Sjt_meIZwEXcS8b off=1071074 len=1422761 2024-04-07T16:43:01.060 INFO:tasks.workunit.client.0.smithi100.stdout:op 1361 completed, throughput=4.99MB/sec 2024-04-07T16:43:01.060 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-RMphIXcAbOUz-da off=10968988 len=914722 2024-04-07T16:43:01.063 INFO:tasks.workunit.client.0.smithi100.stdout:op 1362 completed, throughput=5MB/sec 2024-04-07T16:43:01.063 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-M2cNcD24NHIkhmu off=8530450 len=612277 2024-04-07T16:43:01.070 INFO:tasks.workunit.client.0.smithi100.stdout:op 1363 completed, throughput=5MB/sec 2024-04-07T16:43:01.070 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-KETf2nHTi387KNK off=10692450 len=1228306 2024-04-07T16:43:01.081 INFO:tasks.workunit.client.0.smithi100.stdout:op 1364 completed, throughput=5MB/sec 2024-04-07T16:43:02.081 INFO:tasks.workunit.client.0.smithi100.stdout: 257: throughput=4.98MB/sec pending data=0 2024-04-07T16:43:02.082 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-rdFZcGm0LwdD9eP off=28660 len=1694268 2024-04-07T16:43:02.087 INFO:tasks.workunit.client.0.smithi100.stdout:op 1365 completed, throughput=4.99MB/sec 2024-04-07T16:43:02.087 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-dV_BOv7TtefFCrv off=5941783 len=1393975 2024-04-07T16:43:02.091 INFO:tasks.workunit.client.0.smithi100.stdout:op 1366 completed, throughput=4.99MB/sec 2024-04-07T16:43:02.091 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-zZ9ibcqAteztrU6 off=469362 len=557792 2024-04-07T16:43:02.093 INFO:tasks.workunit.client.0.smithi100.stdout:op 1367 completed, throughput=5MB/sec 2024-04-07T16:43:02.093 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-rCTh8JZNPaGaH36 off=5489039 len=1171355 2024-04-07T16:43:02.105 INFO:tasks.workunit.client.0.smithi100.stdout:op 1368 completed, throughput=5MB/sec 2024-04-07T16:43:02.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:43:02 smithi144 bash[66654]: cluster 2024-04-07T16:43:01.214628+0000 mgr.y (mgr.25175) 830 : cluster [DBG] pgmap v1472: 105 pgs: 105 active+clean; 126 GiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.3 MiB/s wr, 4 op/s 2024-04-07T16:43:02.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:43:02 smithi144 bash[66654]: audit 2024-04-07T16:43:01.762156+0000 mon.a (mon.0) 2081 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:43:02.608 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:43:02 smithi144 bash[95233]: ts=2024-04-07T16:43:02.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:43:02.693 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:43:02 smithi100 bash[74942]: cluster 2024-04-07T16:43:01.214628+0000 mgr.y (mgr.25175) 830 : cluster [DBG] pgmap v1472: 105 pgs: 105 active+clean; 126 GiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.3 MiB/s wr, 4 op/s 2024-04-07T16:43:02.693 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:43:02 smithi100 bash[74942]: audit 2024-04-07T16:43:01.762156+0000 mon.a (mon.0) 2081 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:43:02.693 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:43:02 smithi100 bash[76202]: cluster 2024-04-07T16:43:01.214628+0000 mgr.y (mgr.25175) 830 : cluster [DBG] pgmap v1472: 105 pgs: 105 active+clean; 126 GiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.3 MiB/s wr, 4 op/s 2024-04-07T16:43:02.693 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:43:02 smithi100 bash[76202]: audit 2024-04-07T16:43:01.762156+0000 mon.a (mon.0) 2081 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:43:03.105 INFO:tasks.workunit.client.0.smithi100.stdout: 258: throughput=4.98MB/sec pending data=0 2024-04-07T16:43:03.106 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Xv1_vCDWJLlpfHA off=14603770 len=1010558 2024-04-07T16:43:03.115 INFO:tasks.workunit.client.0.smithi100.stdout:op 1369 completed, throughput=4.98MB/sec 2024-04-07T16:43:03.115 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-P6dWsgLN2aZsHrK off=13839942 len=342123 2024-04-07T16:43:03.117 INFO:tasks.workunit.client.0.smithi100.stdout:op 1370 completed, throughput=4.99MB/sec 2024-04-07T16:43:03.117 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-cv2zogDyWKZbKpT off=1414329 len=647807 2024-04-07T16:43:03.119 INFO:tasks.workunit.client.0.smithi100.stdout:op 1371 completed, throughput=4.99MB/sec 2024-04-07T16:43:03.119 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-h6UGEzpLBZ5j4Iu off=14879830 len=416528 2024-04-07T16:43:03.121 INFO:tasks.workunit.client.0.smithi100.stdout:op 1372 completed, throughput=4.99MB/sec 2024-04-07T16:43:03.121 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-oJWDHc1Zuhbis_D off=7971598 len=1116004 2024-04-07T16:43:03.126 INFO:tasks.workunit.client.0.smithi100.stdout:op 1373 completed, throughput=4.99MB/sec 2024-04-07T16:43:03.126 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-YtI0UNQeHepK7GR off=13078268 len=297324 2024-04-07T16:43:03.131 INFO:tasks.workunit.client.0.smithi100.stdout:op 1374 completed, throughput=4.99MB/sec 2024-04-07T16:43:03.131 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-q55wl_vD2xewSr3 off=1830328 len=1194549 2024-04-07T16:43:03.134 INFO:tasks.workunit.client.0.smithi100.stdout:op 1375 completed, throughput=5MB/sec 2024-04-07T16:43:03.134 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-QsAGU7AW2t_vulg off=1281028 len=1699399 2024-04-07T16:43:03.149 INFO:tasks.workunit.client.0.smithi100.stdout:op 1376 completed, throughput=5MB/sec 2024-04-07T16:43:04.149 INFO:tasks.workunit.client.0.smithi100.stdout: 259: throughput=4.99MB/sec pending data=0 2024-04-07T16:43:04.149 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-AM4A6yVYw6ftf0R off=246980 len=1092591 2024-04-07T16:43:04.158 INFO:tasks.workunit.client.0.smithi100.stdout:op 1377 completed, throughput=4.99MB/sec 2024-04-07T16:43:04.158 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-lNyVQ6tm49yZsLl off=6248049 len=577691 2024-04-07T16:43:04.165 INFO:tasks.workunit.client.0.smithi100.stdout:op 1378 completed, throughput=4.99MB/sec 2024-04-07T16:43:04.165 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-n6QJY1a_sQAHyXC off=5519546 len=665262 2024-04-07T16:43:04.168 INFO:tasks.workunit.client.0.smithi100.stdout:op 1379 completed, throughput=4.99MB/sec 2024-04-07T16:43:04.168 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-nUV2drDPKmDRN_0 off=10057653 len=525798 2024-04-07T16:43:04.170 INFO:tasks.workunit.client.0.smithi100.stdout:op 1380 completed, throughput=5MB/sec 2024-04-07T16:43:04.170 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-JfUkLq3Qnhr_OZ1 off=10786093 len=1180396 2024-04-07T16:43:04.180 INFO:tasks.workunit.client.0.smithi100.stdout:op 1381 completed, throughput=5MB/sec 2024-04-07T16:43:04.180 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-GFelntI7qlkmz83 off=6308075 len=76073 2024-04-07T16:43:04.184 INFO:tasks.workunit.client.0.smithi100.stdout:op 1382 completed, throughput=5MB/sec 2024-04-07T16:43:04.184 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-RXSMRfruI0iVSzB off=152037 len=1751300 2024-04-07T16:43:04.198 INFO:tasks.workunit.client.0.smithi100.stdout:op 1383 completed, throughput=5.01MB/sec 2024-04-07T16:43:04.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:43:04 smithi144 bash[66654]: cluster 2024-04-07T16:43:03.215814+0000 mgr.y (mgr.25175) 831 : cluster [DBG] pgmap v1473: 105 pgs: 105 active+clean; 126 GiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 2.8 MiB/s rd, 2.1 MiB/s wr, 4 op/s 2024-04-07T16:43:04.693 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:43:04 smithi100 bash[74942]: cluster 2024-04-07T16:43:03.215814+0000 mgr.y (mgr.25175) 831 : cluster [DBG] pgmap v1473: 105 pgs: 105 active+clean; 126 GiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 2.8 MiB/s rd, 2.1 MiB/s wr, 4 op/s 2024-04-07T16:43:04.693 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:43:04 smithi100 bash[76202]: cluster 2024-04-07T16:43:03.215814+0000 mgr.y (mgr.25175) 831 : cluster [DBG] pgmap v1473: 105 pgs: 105 active+clean; 126 GiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 2.8 MiB/s rd, 2.1 MiB/s wr, 4 op/s 2024-04-07T16:43:05.199 INFO:tasks.workunit.client.0.smithi100.stdout: 260: throughput=4.99MB/sec pending data=0 2024-04-07T16:43:05.199 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-DhhHv8DDKx0cwph off=2955063 len=495724 2024-04-07T16:43:05.200 INFO:tasks.workunit.client.0.smithi100.stdout:op 1384 completed, throughput=4.99MB/sec 2024-04-07T16:43:05.201 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-nK-uADf3Ics6Ahh off=2044590 len=131269 2024-04-07T16:43:05.202 INFO:tasks.workunit.client.0.smithi100.stdout:op 1385 completed, throughput=4.99MB/sec 2024-04-07T16:43:05.202 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj--UBx91mxXfOh15i off=3913509 len=85645 2024-04-07T16:43:05.203 INFO:tasks.workunit.client.0.smithi100.stdout:op 1386 completed, throughput=4.99MB/sec 2024-04-07T16:43:05.203 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-84P6MFTpHsbPBsh off=296688 len=2083086 2024-04-07T16:43:05.208 INFO:tasks.workunit.client.0.smithi100.stdout:op 1387 completed, throughput=5MB/sec 2024-04-07T16:43:05.208 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-bgbaw-YqgtAXuXx off=19839467 len=2031160 2024-04-07T16:43:05.211 INFO:tasks.workunit.client.0.smithi100.stdout:op 1388 completed, throughput=5MB/sec 2024-04-07T16:43:06.212 INFO:tasks.workunit.client.0.smithi100.stdout: 261: throughput=4.99MB/sec pending data=0 2024-04-07T16:43:06.212 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-66VNkeNz2muF0dG off=2986775 len=807239 2024-04-07T16:43:06.220 INFO:tasks.workunit.client.0.smithi100.stdout:op 1389 completed, throughput=4.99MB/sec 2024-04-07T16:43:06.220 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-rVNlHNmsXuJ93ZF off=12459761 len=1771832 2024-04-07T16:43:06.233 INFO:tasks.workunit.client.0.smithi100.stdout:op 1390 completed, throughput=4.99MB/sec 2024-04-07T16:43:06.233 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-jQrpGGpmOtEIDni off=1724233 len=275322 2024-04-07T16:43:06.235 INFO:tasks.workunit.client.0.smithi100.stdout:op 1391 completed, throughput=5MB/sec 2024-04-07T16:43:06.235 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-17ckjppVOkLsnIH off=9774072 len=98468 2024-04-07T16:43:06.238 INFO:tasks.workunit.client.0.smithi100.stdout:op 1392 completed, throughput=5MB/sec 2024-04-07T16:43:06.239 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-eYWgNOorGOpFkzT off=1572003 len=889329 2024-04-07T16:43:06.242 INFO:tasks.workunit.client.0.smithi100.stdout:op 1393 completed, throughput=5MB/sec 2024-04-07T16:43:06.242 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-zWK0yupejYNdKjP off=3905992 len=1867748 2024-04-07T16:43:06.254 INFO:tasks.workunit.client.0.smithi100.stdout:op 1394 completed, throughput=5MB/sec 2024-04-07T16:43:06.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:43:06 smithi144 bash[66654]: cluster 2024-04-07T16:43:05.216784+0000 mgr.y (mgr.25175) 832 : cluster [DBG] pgmap v1474: 105 pgs: 105 active+clean; 126 GiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.0 MiB/s wr, 4 op/s 2024-04-07T16:43:06.693 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:43:06 smithi100 bash[74942]: cluster 2024-04-07T16:43:05.216784+0000 mgr.y (mgr.25175) 832 : cluster [DBG] pgmap v1474: 105 pgs: 105 active+clean; 126 GiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.0 MiB/s wr, 4 op/s 2024-04-07T16:43:06.693 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:43:06 smithi100 bash[76202]: cluster 2024-04-07T16:43:05.216784+0000 mgr.y (mgr.25175) 832 : cluster [DBG] pgmap v1474: 105 pgs: 105 active+clean; 126 GiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.0 MiB/s wr, 4 op/s 2024-04-07T16:43:07.255 INFO:tasks.workunit.client.0.smithi100.stdout: 262: throughput=4.99MB/sec pending data=0 2024-04-07T16:43:07.255 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-y7XfxqvyBw1DsYE off=17553063 len=1322768 2024-04-07T16:43:07.268 INFO:tasks.workunit.client.0.smithi100.stdout:op 1395 completed, throughput=4.99MB/sec 2024-04-07T16:43:07.268 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-2j2pXvR_skjTvUf off=8762259 len=1712404 2024-04-07T16:43:07.279 INFO:tasks.workunit.client.0.smithi100.stdout:op 1396 completed, throughput=5MB/sec 2024-04-07T16:43:07.279 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-V_OFyocHGmPLMdA off=8805747 len=418369 2024-04-07T16:43:07.286 INFO:tasks.workunit.client.0.smithi100.stdout:op 1397 completed, throughput=5MB/sec 2024-04-07T16:43:07.286 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-cqHIzs1zPeo-dwp off=1787906 len=652655 2024-04-07T16:43:07.289 INFO:tasks.workunit.client.0.smithi100.stdout:op 1398 completed, throughput=5MB/sec 2024-04-07T16:43:08.289 INFO:tasks.workunit.client.0.smithi100.stdout: 263: throughput=4.98MB/sec pending data=0 2024-04-07T16:43:08.290 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-m3rBK6kwUxlaXX5 off=7568075 len=811442 2024-04-07T16:43:08.292 INFO:tasks.workunit.client.0.smithi100.stdout:op 1399 completed, throughput=4.98MB/sec 2024-04-07T16:43:08.292 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-r2JcrTabnui01aD off=14788827 len=1065221 2024-04-07T16:43:08.296 INFO:tasks.workunit.client.0.smithi100.stdout:op 1400 completed, throughput=4.99MB/sec 2024-04-07T16:43:08.296 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-_UyPdQfgBv4kNh6 off=3391753 len=310995 2024-04-07T16:43:08.298 INFO:tasks.workunit.client.0.smithi100.stdout:op 1401 completed, throughput=4.99MB/sec 2024-04-07T16:43:08.298 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-zj4oQwU0ru__gAb off=6397073 len=1498588 2024-04-07T16:43:08.302 INFO:tasks.workunit.client.0.smithi100.stdout:op 1402 completed, throughput=4.99MB/sec 2024-04-07T16:43:08.302 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Z6PqueplXc76sRF off=7464747 len=1676728 2024-04-07T16:43:08.307 INFO:tasks.workunit.client.0.smithi100.stdout:op 1403 completed, throughput=5MB/sec 2024-04-07T16:43:08.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:43:08 smithi144 bash[66654]: cluster 2024-04-07T16:43:07.217856+0000 mgr.y (mgr.25175) 833 : cluster [DBG] pgmap v1475: 105 pgs: 105 active+clean; 126 GiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.1 MiB/s wr, 4 op/s 2024-04-07T16:43:08.693 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:43:08 smithi100 bash[74942]: cluster 2024-04-07T16:43:07.217856+0000 mgr.y (mgr.25175) 833 : cluster [DBG] pgmap v1475: 105 pgs: 105 active+clean; 126 GiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.1 MiB/s wr, 4 op/s 2024-04-07T16:43:08.693 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:43:08 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:43:08] "GET /metrics HTTP/1.1" 200 34055 "" "Prometheus/2.43.0" 2024-04-07T16:43:08.693 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:43:08 smithi100 bash[76202]: cluster 2024-04-07T16:43:07.217856+0000 mgr.y (mgr.25175) 833 : cluster [DBG] pgmap v1475: 105 pgs: 105 active+clean; 126 GiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.1 MiB/s wr, 4 op/s 2024-04-07T16:43:09.308 INFO:tasks.workunit.client.0.smithi100.stdout: 264: throughput=4.98MB/sec pending data=0 2024-04-07T16:43:09.308 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-zbVlD6Eai7tJNAA off=6483301 len=626523 2024-04-07T16:43:09.315 INFO:tasks.workunit.client.0.smithi100.stdout:op 1404 completed, throughput=4.98MB/sec 2024-04-07T16:43:09.315 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-u-cV7fDIfatIU_C off=16308898 len=1347856 2024-04-07T16:43:09.328 INFO:tasks.workunit.client.0.smithi100.stdout:op 1405 completed, throughput=4.99MB/sec 2024-04-07T16:43:09.328 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-PacjVPvDFd6m2oQ off=4302586 len=504996 2024-04-07T16:43:09.335 INFO:tasks.workunit.client.0.smithi100.stdout:op 1406 completed, throughput=4.99MB/sec 2024-04-07T16:43:09.335 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-7a0llmVn3FDbaSc off=1638321 len=1407703 2024-04-07T16:43:09.338 INFO:tasks.workunit.client.0.smithi100.stdout:op 1407 completed, throughput=5MB/sec 2024-04-07T16:43:09.338 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-nDUWfDddpuZEQM9 off=2558751 len=359366 2024-04-07T16:43:09.344 INFO:tasks.workunit.client.0.smithi100.stdout:op 1408 completed, throughput=5MB/sec 2024-04-07T16:43:09.344 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-TZ2939u-v16btG_ off=6374209 len=968000 2024-04-07T16:43:09.347 INFO:tasks.workunit.client.0.smithi100.stdout:op 1409 completed, throughput=5MB/sec 2024-04-07T16:43:09.347 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Xw1OhUcJNhG3r1X off=1527127 len=1234773 2024-04-07T16:43:09.358 INFO:tasks.workunit.client.0.smithi100.stdout:op 1410 completed, throughput=5MB/sec 2024-04-07T16:43:10.358 INFO:tasks.workunit.client.0.smithi100.stdout: 265: throughput=4.99MB/sec pending data=0 2024-04-07T16:43:10.358 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-F8v75dTmaCZ1xpS off=1068713 len=1428372 2024-04-07T16:43:10.370 INFO:tasks.workunit.client.0.smithi100.stdout:op 1411 completed, throughput=4.99MB/sec 2024-04-07T16:43:10.370 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-_z3P7vNn4WboAVu off=15912173 len=884436 2024-04-07T16:43:10.378 INFO:tasks.workunit.client.0.smithi100.stdout:op 1412 completed, throughput=4.99MB/sec 2024-04-07T16:43:10.379 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-QC1yTPrYfAgSiLs off=17748272 len=1050909 2024-04-07T16:43:10.382 INFO:tasks.workunit.client.0.smithi100.stdout:op 1413 completed, throughput=5MB/sec 2024-04-07T16:43:10.382 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-tsGTqxZcXQw2RbC off=1538199 len=687092 2024-04-07T16:43:10.390 INFO:tasks.workunit.client.0.smithi100.stdout:op 1414 completed, throughput=5MB/sec 2024-04-07T16:43:10.391 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-pl8sO4yX2A1sNTd off=2153629 len=1531576 2024-04-07T16:43:10.402 INFO:tasks.workunit.client.0.smithi100.stdout:op 1415 completed, throughput=5MB/sec 2024-04-07T16:43:10.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:43:10 smithi144 bash[66654]: cluster 2024-04-07T16:43:09.218699+0000 mgr.y (mgr.25175) 834 : cluster [DBG] pgmap v1476: 105 pgs: 105 active+clean; 126 GiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 1.8 MiB/s rd, 2.2 MiB/s wr, 4 op/s 2024-04-07T16:43:10.693 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:43:10 smithi100 bash[74942]: cluster 2024-04-07T16:43:09.218699+0000 mgr.y (mgr.25175) 834 : cluster [DBG] pgmap v1476: 105 pgs: 105 active+clean; 126 GiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 1.8 MiB/s rd, 2.2 MiB/s wr, 4 op/s 2024-04-07T16:43:10.693 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:43:10 smithi100 bash[76202]: cluster 2024-04-07T16:43:09.218699+0000 mgr.y (mgr.25175) 834 : cluster [DBG] pgmap v1476: 105 pgs: 105 active+clean; 126 GiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 1.8 MiB/s rd, 2.2 MiB/s wr, 4 op/s 2024-04-07T16:43:11.402 INFO:tasks.workunit.client.0.smithi100.stdout: 266: throughput=4.99MB/sec pending data=0 2024-04-07T16:43:11.403 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-wbGPJp5pjrpS22G off=15821780 len=831772 2024-04-07T16:43:11.405 INFO:tasks.workunit.client.0.smithi100.stdout:op 1416 completed, throughput=4.99MB/sec 2024-04-07T16:43:11.405 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-rAGT4ScxDS5RVMd off=1940752 len=1436158 2024-04-07T16:43:11.410 INFO:tasks.workunit.client.0.smithi100.stdout:op 1417 completed, throughput=4.99MB/sec 2024-04-07T16:43:11.410 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-viLdqn3wiu5JclM off=3276984 len=1858096 2024-04-07T16:43:11.417 INFO:tasks.workunit.client.0.smithi100.stdout:op 1418 completed, throughput=5MB/sec 2024-04-07T16:43:11.417 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-x1gotqZiR7IJVKU off=1236919 len=1497251 2024-04-07T16:43:11.420 INFO:tasks.workunit.client.0.smithi100.stdout:op 1419 completed, throughput=5MB/sec 2024-04-07T16:43:12.421 INFO:tasks.workunit.client.0.smithi100.stdout: 267: throughput=4.99MB/sec pending data=0 2024-04-07T16:43:12.421 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-EHIQf9qX4dhxel3 off=12941324 len=1708218 2024-04-07T16:43:12.435 INFO:tasks.workunit.client.0.smithi100.stdout:op 1420 completed, throughput=4.99MB/sec 2024-04-07T16:43:12.435 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-JUHqJYMEF_cioDc off=5028026 len=540805 2024-04-07T16:43:12.442 INFO:tasks.workunit.client.0.smithi100.stdout:op 1421 completed, throughput=4.99MB/sec 2024-04-07T16:43:12.442 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-wKpFg4Yc6pwUhq9 off=1579621 len=891928 2024-04-07T16:43:12.445 INFO:tasks.workunit.client.0.smithi100.stdout:op 1422 completed, throughput=5MB/sec 2024-04-07T16:43:12.445 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-VBoVPoRXk0ncuVB off=1883549 len=1998487 2024-04-07T16:43:12.450 INFO:tasks.workunit.client.0.smithi100.stdout:op 1423 completed, throughput=5MB/sec 2024-04-07T16:43:12.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:43:12 smithi144 bash[66654]: cluster 2024-04-07T16:43:11.219627+0000 mgr.y (mgr.25175) 835 : cluster [DBG] pgmap v1477: 105 pgs: 105 active+clean; 126 GiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 2.1 MiB/s rd, 2.5 MiB/s wr, 4 op/s 2024-04-07T16:43:12.608 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:43:12 smithi144 bash[95233]: ts=2024-04-07T16:43:12.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:43:12.693 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:43:12 smithi100 bash[74942]: cluster 2024-04-07T16:43:11.219627+0000 mgr.y (mgr.25175) 835 : cluster [DBG] pgmap v1477: 105 pgs: 105 active+clean; 126 GiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 2.1 MiB/s rd, 2.5 MiB/s wr, 4 op/s 2024-04-07T16:43:12.693 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:43:12 smithi100 bash[76202]: cluster 2024-04-07T16:43:11.219627+0000 mgr.y (mgr.25175) 835 : cluster [DBG] pgmap v1477: 105 pgs: 105 active+clean; 126 GiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 2.1 MiB/s rd, 2.5 MiB/s wr, 4 op/s 2024-04-07T16:43:13.450 INFO:tasks.workunit.client.0.smithi100.stdout: 268: throughput=4.99MB/sec pending data=0 2024-04-07T16:43:13.450 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-XvMxznyVx_bdNdD off=4200652 len=573287 2024-04-07T16:43:13.452 INFO:tasks.workunit.client.0.smithi100.stdout:op 1424 completed, throughput=4.99MB/sec 2024-04-07T16:43:13.452 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-_ZK3L3Luhiz0eYX off=2013452 len=1243468 2024-04-07T16:43:13.455 INFO:tasks.workunit.client.0.smithi100.stdout:op 1425 completed, throughput=4.99MB/sec 2024-04-07T16:43:13.455 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-jsZm17ymUk1_umw off=5369744 len=1188082 2024-04-07T16:43:13.459 INFO:tasks.workunit.client.0.smithi100.stdout:op 1426 completed, throughput=5MB/sec 2024-04-07T16:43:13.459 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-OAG4SD_QuMlfr6e off=1956898 len=480193 2024-04-07T16:43:13.465 INFO:tasks.workunit.client.0.smithi100.stdout:op 1427 completed, throughput=5MB/sec 2024-04-07T16:43:13.465 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-j9qDuvjSOlqUrrY off=12517746 len=610810 2024-04-07T16:43:13.474 INFO:tasks.workunit.client.0.smithi100.stdout:op 1428 completed, throughput=5MB/sec 2024-04-07T16:43:13.474 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-0Zk_zFiNiRM4lSW off=1303490 len=818470 2024-04-07T16:43:13.475 INFO:tasks.workunit.client.0.smithi100.stdout:op 1429 completed, throughput=5MB/sec 2024-04-07T16:43:14.476 INFO:tasks.workunit.client.0.smithi100.stdout: 269: throughput=4.98MB/sec pending data=0 2024-04-07T16:43:14.476 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-5lRXrAt0uUYiRSt off=0 len=1233092 2024-04-07T16:43:14.488 INFO:tasks.workunit.client.0.smithi100.stdout:op 1430 completed, throughput=4.99MB/sec 2024-04-07T16:43:14.488 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-ZzNmTTd8zg-EvT9 off=6759952 len=1397622 2024-04-07T16:43:14.492 INFO:tasks.workunit.client.0.smithi100.stdout:op 1431 completed, throughput=4.99MB/sec 2024-04-07T16:43:14.492 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-XIk__uKlVhj4FbD off=703391 len=1546429 2024-04-07T16:43:14.503 INFO:tasks.workunit.client.0.smithi100.stdout:op 1432 completed, throughput=5MB/sec 2024-04-07T16:43:14.504 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-h8uzPvrv3I32gP6 off=4027386 len=34158 2024-04-07T16:43:14.505 INFO:tasks.workunit.client.0.smithi100.stdout:op 1433 completed, throughput=5MB/sec 2024-04-07T16:43:14.505 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-KojEf5AnWV4gnQq off=9710840 len=1765097 2024-04-07T16:43:14.510 INFO:tasks.workunit.client.0.smithi100.stdout:op 1434 completed, throughput=5MB/sec 2024-04-07T16:43:14.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:43:14 smithi144 bash[66654]: cluster 2024-04-07T16:43:13.220793+0000 mgr.y (mgr.25175) 836 : cluster [DBG] pgmap v1478: 105 pgs: 105 active+clean; 126 GiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 2.8 MiB/s rd, 3.2 MiB/s wr, 5 op/s 2024-04-07T16:43:14.693 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:43:14 smithi100 bash[74942]: cluster 2024-04-07T16:43:13.220793+0000 mgr.y (mgr.25175) 836 : cluster [DBG] pgmap v1478: 105 pgs: 105 active+clean; 126 GiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 2.8 MiB/s rd, 3.2 MiB/s wr, 5 op/s 2024-04-07T16:43:14.693 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:43:14 smithi100 bash[76202]: cluster 2024-04-07T16:43:13.220793+0000 mgr.y (mgr.25175) 836 : cluster [DBG] pgmap v1478: 105 pgs: 105 active+clean; 126 GiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 2.8 MiB/s rd, 3.2 MiB/s wr, 5 op/s 2024-04-07T16:43:15.510 INFO:tasks.workunit.client.0.smithi100.stdout: 270: throughput=4.99MB/sec pending data=0 2024-04-07T16:43:15.511 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-PsFBYIr7BZL4y2a off=5290914 len=1531554 2024-04-07T16:43:15.514 INFO:tasks.workunit.client.0.smithi100.stdout:op 1435 completed, throughput=4.99MB/sec 2024-04-07T16:43:15.515 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-M87jskljhCdYjHz off=3331782 len=936150 2024-04-07T16:43:15.518 INFO:tasks.workunit.client.0.smithi100.stdout:op 1436 completed, throughput=4.99MB/sec 2024-04-07T16:43:15.518 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-ngB070iAJkP36hT off=15390569 len=1112265 2024-04-07T16:43:15.522 INFO:tasks.workunit.client.0.smithi100.stdout:op 1437 completed, throughput=5MB/sec 2024-04-07T16:43:15.522 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-_KJ_OM3Fxcjf_v5 off=20119840 len=1685516 2024-04-07T16:43:15.534 INFO:tasks.workunit.client.0.smithi100.stdout:op 1438 completed, throughput=5MB/sec 2024-04-07T16:43:16.535 INFO:tasks.workunit.client.0.smithi100.stdout: 271: throughput=4.99MB/sec pending data=0 2024-04-07T16:43:16.535 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-9Oe92DmpO7rQxUe off=3695154 len=650647 2024-04-07T16:43:16.538 INFO:tasks.workunit.client.0.smithi100.stdout:op 1439 completed, throughput=4.99MB/sec 2024-04-07T16:43:16.538 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-9uMXkvk5AHI8TUZ off=18983010 len=1642457 2024-04-07T16:43:16.551 INFO:tasks.workunit.client.0.smithi100.stdout:op 1440 completed, throughput=4.99MB/sec 2024-04-07T16:43:16.551 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-4DkMPObV33OxAoQ off=6034140 len=1482309 2024-04-07T16:43:16.559 INFO:tasks.workunit.client.0.smithi100.stdout:op 1441 completed, throughput=5MB/sec 2024-04-07T16:43:16.559 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-AQ6GUpBrdHMAXAP off=2049463 len=244326 2024-04-07T16:43:16.561 INFO:tasks.workunit.client.0.smithi100.stdout:op 1442 completed, throughput=5MB/sec 2024-04-07T16:43:16.561 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-rTqOtZ1rPRwWgPN off=552188 len=1325559 2024-04-07T16:43:16.565 INFO:tasks.workunit.client.0.smithi100.stdout:op 1443 completed, throughput=5MB/sec 2024-04-07T16:43:16.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:43:16 smithi144 bash[66654]: cluster 2024-04-07T16:43:15.221893+0000 mgr.y (mgr.25175) 837 : cluster [DBG] pgmap v1479: 105 pgs: 105 active+clean; 126 GiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 2.6 MiB/s rd, 2.7 MiB/s wr, 5 op/s 2024-04-07T16:43:16.693 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:43:16 smithi100 bash[74942]: cluster 2024-04-07T16:43:15.221893+0000 mgr.y (mgr.25175) 837 : cluster [DBG] pgmap v1479: 105 pgs: 105 active+clean; 126 GiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 2.6 MiB/s rd, 2.7 MiB/s wr, 5 op/s 2024-04-07T16:43:16.693 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:43:16 smithi100 bash[76202]: cluster 2024-04-07T16:43:15.221893+0000 mgr.y (mgr.25175) 837 : cluster [DBG] pgmap v1479: 105 pgs: 105 active+clean; 126 GiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 2.6 MiB/s rd, 2.7 MiB/s wr, 5 op/s 2024-04-07T16:43:17.566 INFO:tasks.workunit.client.0.smithi100.stdout: 272: throughput=4.99MB/sec pending data=0 2024-04-07T16:43:17.566 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj--q4viBA1cx8tMHV off=3518701 len=1713532 2024-04-07T16:43:17.571 INFO:tasks.workunit.client.0.smithi100.stdout:op 1444 completed, throughput=4.99MB/sec 2024-04-07T16:43:17.571 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-2NXqEBx1fMoeiIU off=7379776 len=720926 2024-04-07T16:43:17.579 INFO:tasks.workunit.client.0.smithi100.stdout:op 1445 completed, throughput=4.99MB/sec 2024-04-07T16:43:17.579 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-NN_sFaeraJgxMgO off=1544636 len=1174978 2024-04-07T16:43:17.588 INFO:tasks.workunit.client.0.smithi100.stdout:op 1446 completed, throughput=5MB/sec 2024-04-07T16:43:17.588 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-I2Z4I5tr-7SYOqu off=5548797 len=2016004 2024-04-07T16:43:17.604 INFO:tasks.workunit.client.0.smithi100.stdout:op 1447 completed, throughput=5MB/sec 2024-04-07T16:43:17.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:43:17 smithi144 bash[66654]: audit 2024-04-07T16:43:16.762854+0000 mon.a (mon.0) 2082 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:43:17.607 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:43:17 smithi144 bash[98866]: logger=sqlstore.transactions t=2024-04-07T16:43:17.282289497Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-04-07T16:43:17.693 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:43:17 smithi100 bash[74942]: audit 2024-04-07T16:43:16.762854+0000 mon.a (mon.0) 2082 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:43:17.693 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:43:17 smithi100 bash[76202]: audit 2024-04-07T16:43:16.762854+0000 mon.a (mon.0) 2082 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:43:18.605 INFO:tasks.workunit.client.0.smithi100.stdout: 273: throughput=4.99MB/sec pending data=0 2024-04-07T16:43:18.605 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-aVaM3eU9iZtSweu off=8784122 len=1448315 2024-04-07T16:43:18.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:43:18 smithi144 bash[66654]: cluster 2024-04-07T16:43:17.222968+0000 mgr.y (mgr.25175) 838 : cluster [DBG] pgmap v1480: 105 pgs: 105 active+clean; 126 GiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 3.0 MiB/s rd, 2.5 MiB/s wr, 5 op/s 2024-04-07T16:43:18.609 INFO:tasks.workunit.client.0.smithi100.stdout:op 1448 completed, throughput=4.99MB/sec 2024-04-07T16:43:18.609 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-rdbuzAm8mLDWRgh off=7939320 len=1252867 2024-04-07T16:43:18.612 INFO:tasks.workunit.client.0.smithi100.stdout:op 1449 completed, throughput=5MB/sec 2024-04-07T16:43:18.612 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-XLB8L4kxMX4mDIY off=16055800 len=413264 2024-04-07T16:43:18.614 INFO:tasks.workunit.client.0.smithi100.stdout:op 1450 completed, throughput=5MB/sec 2024-04-07T16:43:18.614 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-CY5gp9w9n94b3C4 off=2823426 len=1042449 2024-04-07T16:43:18.618 INFO:tasks.workunit.client.0.smithi100.stdout:op 1451 completed, throughput=5MB/sec 2024-04-07T16:43:18.693 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:43:18 smithi100 bash[74942]: cluster 2024-04-07T16:43:17.222968+0000 mgr.y (mgr.25175) 838 : cluster [DBG] pgmap v1480: 105 pgs: 105 active+clean; 126 GiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 3.0 MiB/s rd, 2.5 MiB/s wr, 5 op/s 2024-04-07T16:43:18.693 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:43:18 smithi100 bash[76202]: cluster 2024-04-07T16:43:17.222968+0000 mgr.y (mgr.25175) 838 : cluster [DBG] pgmap v1480: 105 pgs: 105 active+clean; 126 GiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 3.0 MiB/s rd, 2.5 MiB/s wr, 5 op/s 2024-04-07T16:43:18.693 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:43:18 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:43:18] "GET /metrics HTTP/1.1" 200 34057 "" "Prometheus/2.43.0" 2024-04-07T16:43:19.619 INFO:tasks.workunit.client.0.smithi100.stdout: 274: throughput=4.98MB/sec pending data=0 2024-04-07T16:43:19.619 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-WAU243GSC4NhPx_ off=22728488 len=599078 2024-04-07T16:43:19.626 INFO:tasks.workunit.client.0.smithi100.stdout:op 1452 completed, throughput=4.98MB/sec 2024-04-07T16:43:19.626 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-wGR7xkEXTXrka2M off=16570953 len=613034 2024-04-07T16:43:19.633 INFO:tasks.workunit.client.0.smithi100.stdout:op 1453 completed, throughput=4.99MB/sec 2024-04-07T16:43:19.633 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-_bpQcI1_fOHjO6u off=48400 len=451849 2024-04-07T16:43:19.637 INFO:tasks.workunit.client.0.smithi100.stdout:op 1454 completed, throughput=4.99MB/sec 2024-04-07T16:43:19.637 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-raUGtFianN0dwsP off=1809494 len=1360957 2024-04-07T16:43:19.640 INFO:tasks.workunit.client.0.smithi100.stdout:op 1455 completed, throughput=4.99MB/sec 2024-04-07T16:43:19.640 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-gufkrkibOWvnXcv off=2484000 len=758737 2024-04-07T16:43:19.642 INFO:tasks.workunit.client.0.smithi100.stdout:op 1456 completed, throughput=5MB/sec 2024-04-07T16:43:19.642 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-8Nv-Hgpoum8qxnF off=12341471 len=1957020 2024-04-07T16:43:19.647 INFO:tasks.workunit.client.0.smithi100.stdout:op 1457 completed, throughput=5MB/sec 2024-04-07T16:43:20.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:43:20 smithi144 bash[66654]: cluster 2024-04-07T16:43:19.223810+0000 mgr.y (mgr.25175) 839 : cluster [DBG] pgmap v1481: 105 pgs: 105 active+clean; 126 GiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 2.8 MiB/s rd, 2.2 MiB/s wr, 4 op/s 2024-04-07T16:43:20.648 INFO:tasks.workunit.client.0.smithi100.stdout: 275: throughput=4.98MB/sec pending data=0 2024-04-07T16:43:20.648 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-9i296EMDQX9digm off=4183311 len=1775077 2024-04-07T16:43:20.652 INFO:tasks.workunit.client.0.smithi100.stdout:op 1458 completed, throughput=4.99MB/sec 2024-04-07T16:43:20.652 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-TfpREWaYFqkN2aG off=4910844 len=1735718 2024-04-07T16:43:20.668 INFO:tasks.workunit.client.0.smithi100.stdout:op 1459 completed, throughput=5MB/sec 2024-04-07T16:43:20.668 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-f5WXm5xOuCk6tNH off=2625340 len=402505 2024-04-07T16:43:20.670 INFO:tasks.workunit.client.0.smithi100.stdout:op 1460 completed, throughput=5MB/sec 2024-04-07T16:43:20.670 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-k66um6M-YVcyFKJ off=2105286 len=996435 2024-04-07T16:43:20.679 INFO:tasks.workunit.client.0.smithi100.stdout:op 1461 completed, throughput=5MB/sec 2024-04-07T16:43:20.679 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-XkSaOOaCpGxWD3M off=129216 len=539070 2024-04-07T16:43:20.681 INFO:tasks.workunit.client.0.smithi100.stdout:op 1462 completed, throughput=5MB/sec 2024-04-07T16:43:20.692 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:43:20 smithi100 bash[74942]: cluster 2024-04-07T16:43:19.223810+0000 mgr.y (mgr.25175) 839 : cluster [DBG] pgmap v1481: 105 pgs: 105 active+clean; 126 GiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 2.8 MiB/s rd, 2.2 MiB/s wr, 4 op/s 2024-04-07T16:43:20.693 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:43:20 smithi100 bash[76202]: cluster 2024-04-07T16:43:19.223810+0000 mgr.y (mgr.25175) 839 : cluster [DBG] pgmap v1481: 105 pgs: 105 active+clean; 126 GiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 2.8 MiB/s rd, 2.2 MiB/s wr, 4 op/s 2024-04-07T16:43:21.682 INFO:tasks.workunit.client.0.smithi100.stdout: 276: throughput=4.98MB/sec pending data=0 2024-04-07T16:43:21.683 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-aoRGsRJ3wG6Ue7l off=4034938 len=81034 2024-04-07T16:43:21.683 INFO:tasks.workunit.client.0.smithi100.stdout:op 1463 completed, throughput=4.98MB/sec 2024-04-07T16:43:21.683 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-itdRHxoPH_3PXVn off=2423247 len=1790127 2024-04-07T16:43:21.697 INFO:tasks.workunit.client.0.smithi100.stdout:op 1464 completed, throughput=4.99MB/sec 2024-04-07T16:43:21.697 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-pru7WAXwqqiJJm8 off=4388041 len=1253460 2024-04-07T16:43:21.701 INFO:tasks.workunit.client.0.smithi100.stdout:op 1465 completed, throughput=4.99MB/sec 2024-04-07T16:43:21.701 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-CrXuT4gG9v3lYqj off=6526170 len=883408 2024-04-07T16:43:21.703 INFO:tasks.workunit.client.0.smithi100.stdout:op 1466 completed, throughput=5MB/sec 2024-04-07T16:43:21.703 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-a1F0v6cuqtXG9k7 off=2042342 len=968083 2024-04-07T16:43:21.712 INFO:tasks.workunit.client.0.smithi100.stdout:op 1467 completed, throughput=5MB/sec 2024-04-07T16:43:22.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:43:22 smithi144 bash[66654]: cluster 2024-04-07T16:43:21.224706+0000 mgr.y (mgr.25175) 840 : cluster [DBG] pgmap v1482: 105 pgs: 105 active+clean; 126 GiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 3.1 MiB/s rd, 2.5 MiB/s wr, 5 op/s 2024-04-07T16:43:22.608 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:43:22 smithi144 bash[95233]: ts=2024-04-07T16:43:22.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:43:22.693 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:43:22 smithi100 bash[74942]: cluster 2024-04-07T16:43:21.224706+0000 mgr.y (mgr.25175) 840 : cluster [DBG] pgmap v1482: 105 pgs: 105 active+clean; 126 GiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 3.1 MiB/s rd, 2.5 MiB/s wr, 5 op/s 2024-04-07T16:43:22.693 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:43:22 smithi100 bash[76202]: cluster 2024-04-07T16:43:21.224706+0000 mgr.y (mgr.25175) 840 : cluster [DBG] pgmap v1482: 105 pgs: 105 active+clean; 126 GiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 3.1 MiB/s rd, 2.5 MiB/s wr, 5 op/s 2024-04-07T16:43:22.712 INFO:tasks.workunit.client.0.smithi100.stdout: 277: throughput=4.98MB/sec pending data=0 2024-04-07T16:43:22.712 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-OCVyxMgGAeLNJrS off=2763670 len=1433749 2024-04-07T16:43:22.716 INFO:tasks.workunit.client.0.smithi100.stdout:op 1468 completed, throughput=4.99MB/sec 2024-04-07T16:43:22.716 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-8-13mIi3SXGCnlj off=1417419 len=669995 2024-04-07T16:43:22.723 INFO:tasks.workunit.client.0.smithi100.stdout:op 1469 completed, throughput=4.99MB/sec 2024-04-07T16:43:22.723 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-jL20xI80XMyB1Tj off=3668508 len=1523095 2024-04-07T16:43:22.735 INFO:tasks.workunit.client.0.smithi100.stdout:op 1470 completed, throughput=4.99MB/sec 2024-04-07T16:43:22.735 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-H7uUYc3o-u5a_PX off=5665830 len=495128 2024-04-07T16:43:22.741 INFO:tasks.workunit.client.0.smithi100.stdout:op 1471 completed, throughput=5MB/sec 2024-04-07T16:43:22.741 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-fHX3v2cyNaV_NL_ off=2774079 len=849670 2024-04-07T16:43:22.750 INFO:tasks.workunit.client.0.smithi100.stdout:op 1472 completed, throughput=5MB/sec 2024-04-07T16:43:22.750 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-TZ2939u-v16btG_ off=7550291 len=121640 2024-04-07T16:43:22.755 INFO:tasks.workunit.client.0.smithi100.stdout:op 1473 completed, throughput=5MB/sec 2024-04-07T16:43:22.755 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-uHLvvqkvf8LzosI off=11140379 len=380942 2024-04-07T16:43:22.761 INFO:tasks.workunit.client.0.smithi100.stdout:op 1474 completed, throughput=5MB/sec 2024-04-07T16:43:23.761 INFO:tasks.workunit.client.0.smithi100.stdout: 278: throughput=4.98MB/sec pending data=0 2024-04-07T16:43:23.761 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-PJH1uzU6XAkHPDF off=4367112 len=1724489 2024-04-07T16:43:23.766 INFO:tasks.workunit.client.0.smithi100.stdout:op 1475 completed, throughput=4.99MB/sec 2024-04-07T16:43:23.766 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-92nZF3OsYjT5dI2 off=8078747 len=1673240 2024-04-07T16:43:23.770 INFO:tasks.workunit.client.0.smithi100.stdout:op 1476 completed, throughput=4.99MB/sec 2024-04-07T16:43:23.770 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-s9zzKWGCJjdCdeA off=1685010 len=1640445 2024-04-07T16:43:23.773 INFO:tasks.workunit.client.0.smithi100.stdout:op 1477 completed, throughput=5MB/sec 2024-04-07T16:43:23.773 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-kWk0auv9YmtsdSL off=3407624 len=1428762 2024-04-07T16:43:23.785 INFO:tasks.workunit.client.0.smithi100.stdout:op 1478 completed, throughput=5MB/sec 2024-04-07T16:43:24.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:43:24 smithi144 bash[66654]: cluster 2024-04-07T16:43:23.225837+0000 mgr.y (mgr.25175) 841 : cluster [DBG] pgmap v1483: 105 pgs: 105 active+clean; 126 GiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 3.4 MiB/s rd, 2.5 MiB/s wr, 5 op/s 2024-04-07T16:43:24.693 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:43:24 smithi100 bash[74942]: cluster 2024-04-07T16:43:23.225837+0000 mgr.y (mgr.25175) 841 : cluster [DBG] pgmap v1483: 105 pgs: 105 active+clean; 126 GiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 3.4 MiB/s rd, 2.5 MiB/s wr, 5 op/s 2024-04-07T16:43:24.693 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:43:24 smithi100 bash[76202]: cluster 2024-04-07T16:43:23.225837+0000 mgr.y (mgr.25175) 841 : cluster [DBG] pgmap v1483: 105 pgs: 105 active+clean; 126 GiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 3.4 MiB/s rd, 2.5 MiB/s wr, 5 op/s 2024-04-07T16:43:24.785 INFO:tasks.workunit.client.0.smithi100.stdout: 279: throughput=4.99MB/sec pending data=0 2024-04-07T16:43:24.785 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-IkgabsD_qT9qYvD off=19441679 len=1196503 2024-04-07T16:43:24.789 INFO:tasks.workunit.client.0.smithi100.stdout:op 1479 completed, throughput=4.99MB/sec 2024-04-07T16:43:24.789 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-jAF0eXa_xHJnL22 off=3972968 len=963867 2024-04-07T16:43:24.792 INFO:tasks.workunit.client.0.smithi100.stdout:op 1480 completed, throughput=4.99MB/sec 2024-04-07T16:43:24.792 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-ZZB4Oank2-ZsAZq off=10899464 len=922960 2024-04-07T16:43:24.801 INFO:tasks.workunit.client.0.smithi100.stdout:op 1481 completed, throughput=5MB/sec 2024-04-07T16:43:24.801 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-cFb2Hq7jIXZho6- off=9202196 len=782976 2024-04-07T16:43:24.803 INFO:tasks.workunit.client.0.smithi100.stdout:op 1482 completed, throughput=5MB/sec 2024-04-07T16:43:24.803 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-L8t6xiShJHWRpO4 off=4771022 len=1690933 2024-04-07T16:43:24.816 INFO:tasks.workunit.client.0.smithi100.stdout:op 1483 completed, throughput=5MB/sec 2024-04-07T16:43:25.817 INFO:tasks.workunit.client.0.smithi100.stdout: 280: throughput=4.99MB/sec pending data=0 2024-04-07T16:43:25.817 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-oDupMjM83LZogrf off=1477174 len=189935 2024-04-07T16:43:25.822 INFO:tasks.workunit.client.0.smithi100.stdout:op 1484 completed, throughput=4.99MB/sec 2024-04-07T16:43:25.822 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-lqy_sJOgT29SCck off=9492936 len=1397360 2024-04-07T16:43:25.832 INFO:tasks.workunit.client.0.smithi100.stdout:op 1485 completed, throughput=4.99MB/sec 2024-04-07T16:43:25.832 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-vonEiGkOqw2IFjg off=5038355 len=1051872 2024-04-07T16:43:25.840 INFO:tasks.workunit.client.0.smithi100.stdout:op 1486 completed, throughput=5MB/sec 2024-04-07T16:43:25.840 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-31iw9-xjdY2DG7Q off=8753342 len=2061836 2024-04-07T16:43:25.844 INFO:tasks.workunit.client.0.smithi100.stdout:op 1487 completed, throughput=5MB/sec 2024-04-07T16:43:26.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:43:26 smithi144 bash[66654]: cluster 2024-04-07T16:43:25.226833+0000 mgr.y (mgr.25175) 842 : cluster [DBG] pgmap v1484: 105 pgs: 105 active+clean; 126 GiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 2.7 MiB/s rd, 1.8 MiB/s wr, 4 op/s 2024-04-07T16:43:26.693 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:43:26 smithi100 bash[76202]: cluster 2024-04-07T16:43:25.226833+0000 mgr.y (mgr.25175) 842 : cluster [DBG] pgmap v1484: 105 pgs: 105 active+clean; 126 GiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 2.7 MiB/s rd, 1.8 MiB/s wr, 4 op/s 2024-04-07T16:43:26.693 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:43:26 smithi100 bash[74942]: cluster 2024-04-07T16:43:25.226833+0000 mgr.y (mgr.25175) 842 : cluster [DBG] pgmap v1484: 105 pgs: 105 active+clean; 126 GiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 2.7 MiB/s rd, 1.8 MiB/s wr, 4 op/s 2024-04-07T16:43:26.845 INFO:tasks.workunit.client.0.smithi100.stdout: 281: throughput=4.98MB/sec pending data=0 2024-04-07T16:43:26.845 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-mIQlvYCdSwViTp6 off=5414418 len=359813 2024-04-07T16:43:26.847 INFO:tasks.workunit.client.0.smithi100.stdout:op 1488 completed, throughput=4.99MB/sec 2024-04-07T16:43:26.847 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-h2FzE-W7yHjqrjb off=16091402 len=1317771 2024-04-07T16:43:26.850 INFO:tasks.workunit.client.0.smithi100.stdout:op 1489 completed, throughput=4.99MB/sec 2024-04-07T16:43:26.850 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-yU0kxxPx0e3ll1T off=653410 len=1754726 2024-04-07T16:43:26.864 INFO:tasks.workunit.client.0.smithi100.stdout:op 1490 completed, throughput=5MB/sec 2024-04-07T16:43:26.864 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-ZqII9LtzSeRzwHo off=6755210 len=1681347 2024-04-07T16:43:26.867 INFO:tasks.workunit.client.0.smithi100.stdout:op 1491 completed, throughput=5MB/sec 2024-04-07T16:43:27.868 INFO:tasks.workunit.client.0.smithi100.stdout: 282: throughput=4.98MB/sec pending data=0 2024-04-07T16:43:27.868 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-5VcWXbDpY8fjXz7 off=15968243 len=1154689 2024-04-07T16:43:27.871 INFO:tasks.workunit.client.0.smithi100.stdout:op 1492 completed, throughput=4.99MB/sec 2024-04-07T16:43:27.871 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-_8UFkpX67JIjTLL off=9213500 len=1810424 2024-04-07T16:43:27.875 INFO:tasks.workunit.client.0.smithi100.stdout:op 1493 completed, throughput=4.99MB/sec 2024-04-07T16:43:27.876 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj--AEyoh4e7x-PhGF off=10440650 len=745390 2024-04-07T16:43:27.878 INFO:tasks.workunit.client.0.smithi100.stdout:op 1494 completed, throughput=5MB/sec 2024-04-07T16:43:27.878 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-vv4f-RqkVFf6Z97 off=276921 len=740341 2024-04-07T16:43:27.885 INFO:tasks.workunit.client.0.smithi100.stdout:op 1495 completed, throughput=5MB/sec 2024-04-07T16:43:27.885 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-pR1htuW5EpVp6XA off=459210 len=1166859 2024-04-07T16:43:27.895 INFO:tasks.workunit.client.0.smithi100.stdout:op 1496 completed, throughput=5MB/sec 2024-04-07T16:43:28.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:43:28 smithi144 bash[66654]: cluster 2024-04-07T16:43:27.227867+0000 mgr.y (mgr.25175) 843 : cluster [DBG] pgmap v1485: 105 pgs: 105 active+clean; 126 GiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 2.4 MiB/s rd, 1.8 MiB/s wr, 4 op/s 2024-04-07T16:43:28.693 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:43:28 smithi100 bash[74942]: cluster 2024-04-07T16:43:27.227867+0000 mgr.y (mgr.25175) 843 : cluster [DBG] pgmap v1485: 105 pgs: 105 active+clean; 126 GiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 2.4 MiB/s rd, 1.8 MiB/s wr, 4 op/s 2024-04-07T16:43:28.693 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:43:28 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:43:28] "GET /metrics HTTP/1.1" 200 34057 "" "Prometheus/2.43.0" 2024-04-07T16:43:28.693 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:43:28 smithi100 bash[76202]: cluster 2024-04-07T16:43:27.227867+0000 mgr.y (mgr.25175) 843 : cluster [DBG] pgmap v1485: 105 pgs: 105 active+clean; 126 GiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 2.4 MiB/s rd, 1.8 MiB/s wr, 4 op/s 2024-04-07T16:43:28.895 INFO:tasks.workunit.client.0.smithi100.stdout: 283: throughput=4.98MB/sec pending data=0 2024-04-07T16:43:28.896 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-hLFxiwQ0NKaMKCW off=9240762 len=955398 2024-04-07T16:43:28.898 INFO:tasks.workunit.client.0.smithi100.stdout:op 1497 completed, throughput=4.99MB/sec 2024-04-07T16:43:28.899 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-IU_DuvjfOUSfIEl off=323101 len=347561 2024-04-07T16:43:28.905 INFO:tasks.workunit.client.0.smithi100.stdout:op 1498 completed, throughput=4.99MB/sec 2024-04-07T16:43:28.905 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-9tWTs3BE4kA2LsA off=1160927 len=563285 2024-04-07T16:43:28.908 INFO:tasks.workunit.client.0.smithi100.stdout:op 1499 completed, throughput=4.99MB/sec 2024-04-07T16:43:28.908 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-gfL9jYZ4oYk8R6a off=18169955 len=556658 2024-04-07T16:43:28.910 INFO:tasks.workunit.client.0.smithi100.stdout:op 1500 completed, throughput=4.99MB/sec 2024-04-07T16:43:28.910 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-0F5FCRnknY5lJM7 off=269659 len=1471556 2024-04-07T16:43:28.914 INFO:tasks.workunit.client.0.smithi100.stdout:op 1501 completed, throughput=5MB/sec 2024-04-07T16:43:28.914 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-DGovuQu98srUpLO off=236685 len=434756 2024-04-07T16:43:28.920 INFO:tasks.workunit.client.0.smithi100.stdout:op 1502 completed, throughput=5MB/sec 2024-04-07T16:43:28.920 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-q6FrJfJqbfKbBU4 off=12793244 len=861590 2024-04-07T16:43:28.928 INFO:tasks.workunit.client.0.smithi100.stdout:op 1503 completed, throughput=5MB/sec 2024-04-07T16:43:29.928 INFO:tasks.workunit.client.0.smithi100.stdout: 284: throughput=4.98MB/sec pending data=0 2024-04-07T16:43:29.928 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-QWw_rErlaExd0Tb off=17720486 len=1981133 2024-04-07T16:43:29.933 INFO:tasks.workunit.client.0.smithi100.stdout:op 1504 completed, throughput=4.99MB/sec 2024-04-07T16:43:29.933 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-T_1CxfcVBjAML_y off=140609 len=506134 2024-04-07T16:43:29.940 INFO:tasks.workunit.client.0.smithi100.stdout:op 1505 completed, throughput=4.99MB/sec 2024-04-07T16:43:29.941 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-mjiBldhSMHusy-1 off=3603789 len=1468821 2024-04-07T16:43:29.944 INFO:tasks.workunit.client.0.smithi100.stdout:op 1506 completed, throughput=5MB/sec 2024-04-07T16:43:29.944 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-7_u2XrDgR_ykuQJ off=1187430 len=832440 2024-04-07T16:43:29.947 INFO:tasks.workunit.client.0.smithi100.stdout:op 1507 completed, throughput=5MB/sec 2024-04-07T16:43:29.947 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Px1dcUOA1fv4nVA off=2586751 len=469596 2024-04-07T16:43:29.953 INFO:tasks.workunit.client.0.smithi100.stdout:op 1508 completed, throughput=5MB/sec 2024-04-07T16:43:30.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:43:30 smithi144 bash[66654]: cluster 2024-04-07T16:43:29.228715+0000 mgr.y (mgr.25175) 844 : cluster [DBG] pgmap v1486: 105 pgs: 105 active+clean; 126 GiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 2.6 MiB/s rd, 1.9 MiB/s wr, 4 op/s 2024-04-07T16:43:30.693 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:43:30 smithi100 bash[74942]: cluster 2024-04-07T16:43:29.228715+0000 mgr.y (mgr.25175) 844 : cluster [DBG] pgmap v1486: 105 pgs: 105 active+clean; 126 GiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 2.6 MiB/s rd, 1.9 MiB/s wr, 4 op/s 2024-04-07T16:43:30.693 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:43:30 smithi100 bash[76202]: cluster 2024-04-07T16:43:29.228715+0000 mgr.y (mgr.25175) 844 : cluster [DBG] pgmap v1486: 105 pgs: 105 active+clean; 126 GiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 2.6 MiB/s rd, 1.9 MiB/s wr, 4 op/s 2024-04-07T16:43:30.953 INFO:tasks.workunit.client.0.smithi100.stdout: 285: throughput=4.98MB/sec pending data=0 2024-04-07T16:43:30.954 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-cy11M3jf6J3ZNad off=6619245 len=157911 2024-04-07T16:43:30.955 INFO:tasks.workunit.client.0.smithi100.stdout:op 1509 completed, throughput=4.98MB/sec 2024-04-07T16:43:30.955 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-vLGDF_L-dQQcwTY off=5606561 len=749388 2024-04-07T16:43:30.962 INFO:tasks.workunit.client.0.smithi100.stdout:op 1510 completed, throughput=4.99MB/sec 2024-04-07T16:43:30.962 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-tXnqL5b5vqeOCrN off=6037412 len=819864 2024-04-07T16:43:30.970 INFO:tasks.workunit.client.0.smithi100.stdout:op 1511 completed, throughput=4.99MB/sec 2024-04-07T16:43:30.970 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-g2nSNOL6kfbvp45 off=7641916 len=92484 2024-04-07T16:43:30.971 INFO:tasks.workunit.client.0.smithi100.stdout:op 1512 completed, throughput=4.99MB/sec 2024-04-07T16:43:30.971 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-bf5Xuc1P_dPrvGk off=12271275 len=1833516 2024-04-07T16:43:30.987 INFO:tasks.workunit.client.0.smithi100.stdout:op 1513 completed, throughput=5MB/sec 2024-04-07T16:43:30.987 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-THUKB6G9l6MTzZy off=12473807 len=1225800 2024-04-07T16:43:30.997 INFO:tasks.workunit.client.0.smithi100.stdout:op 1514 completed, throughput=5MB/sec 2024-04-07T16:43:30.997 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-AYZlCQd32m-d36A off=20311550 len=1304699 2024-04-07T16:43:31.007 INFO:tasks.workunit.client.0.smithi100.stdout:op 1515 completed, throughput=5MB/sec 2024-04-07T16:43:32.008 INFO:tasks.workunit.client.0.smithi100.stdout: 286: throughput=4.99MB/sec pending data=0 2024-04-07T16:43:32.008 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-cVcC-Xx3Vz447yp off=3846285 len=1177509 2024-04-07T16:43:32.012 INFO:tasks.workunit.client.0.smithi100.stdout:op 1516 completed, throughput=4.99MB/sec 2024-04-07T16:43:32.012 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-g9Fo29VV3k07-n3 off=498760 len=73028 2024-04-07T16:43:32.014 INFO:tasks.workunit.client.0.smithi100.stdout:op 1517 completed, throughput=4.99MB/sec 2024-04-07T16:43:32.014 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-h1dEQEsj7gW4stT off=1647449 len=842959 2024-04-07T16:43:32.016 INFO:tasks.workunit.client.0.smithi100.stdout:op 1518 completed, throughput=4.99MB/sec 2024-04-07T16:43:32.016 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-NnlzqGveVRLL1lV off=238665 len=1341920 2024-04-07T16:43:32.027 INFO:tasks.workunit.client.0.smithi100.stdout:op 1519 completed, throughput=5MB/sec 2024-04-07T16:43:32.028 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-HNaEd4Gua0eVNxv off=218565 len=1432234 2024-04-07T16:43:32.040 INFO:tasks.workunit.client.0.smithi100.stdout:op 1520 completed, throughput=5MB/sec 2024-04-07T16:43:32.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:43:32 smithi144 bash[66654]: cluster 2024-04-07T16:43:31.229826+0000 mgr.y (mgr.25175) 845 : cluster [DBG] pgmap v1487: 105 pgs: 105 active+clean; 126 GiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.1 MiB/s wr, 4 op/s 2024-04-07T16:43:32.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:43:32 smithi144 bash[66654]: audit 2024-04-07T16:43:31.763124+0000 mon.a (mon.0) 2083 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:43:32.608 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:43:32 smithi144 bash[95233]: ts=2024-04-07T16:43:32.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:43:32.693 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:43:32 smithi100 bash[74942]: cluster 2024-04-07T16:43:31.229826+0000 mgr.y (mgr.25175) 845 : cluster [DBG] pgmap v1487: 105 pgs: 105 active+clean; 126 GiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.1 MiB/s wr, 4 op/s 2024-04-07T16:43:32.693 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:43:32 smithi100 bash[74942]: audit 2024-04-07T16:43:31.763124+0000 mon.a (mon.0) 2083 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:43:32.693 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:43:32 smithi100 bash[76202]: cluster 2024-04-07T16:43:31.229826+0000 mgr.y (mgr.25175) 845 : cluster [DBG] pgmap v1487: 105 pgs: 105 active+clean; 126 GiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.1 MiB/s wr, 4 op/s 2024-04-07T16:43:32.693 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:43:32 smithi100 bash[76202]: audit 2024-04-07T16:43:31.763124+0000 mon.a (mon.0) 2083 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:43:33.040 INFO:tasks.workunit.client.0.smithi100.stdout: 287: throughput=4.98MB/sec pending data=0 2024-04-07T16:43:33.041 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-sUZXPGsWYcDvlwM off=9460083 len=291091 2024-04-07T16:43:33.046 INFO:tasks.workunit.client.0.smithi100.stdout:op 1521 completed, throughput=4.99MB/sec 2024-04-07T16:43:33.047 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-j806UM4GofJ_ygf off=4166550 len=1479990 2024-04-07T16:43:33.050 INFO:tasks.workunit.client.0.smithi100.stdout:op 1522 completed, throughput=4.99MB/sec 2024-04-07T16:43:33.050 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-TuFlldoIz5CEBse off=1615073 len=2088974 2024-04-07T16:43:33.055 INFO:tasks.workunit.client.0.smithi100.stdout:op 1523 completed, throughput=5MB/sec 2024-04-07T16:43:33.055 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Pzf6HGRAYzkqg_x off=11722660 len=1886486 2024-04-07T16:43:33.071 INFO:tasks.workunit.client.0.smithi100.stdout:op 1524 completed, throughput=5MB/sec 2024-04-07T16:43:34.072 INFO:tasks.workunit.client.0.smithi100.stdout: 288: throughput=4.99MB/sec pending data=0 2024-04-07T16:43:34.072 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-DxC0MVSxYEd8AB- off=1839001 len=1812969 2024-04-07T16:43:34.085 INFO:tasks.workunit.client.0.smithi100.stdout:op 1525 completed, throughput=4.99MB/sec 2024-04-07T16:43:34.086 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-dE0R7nfR81eu9Gm off=19686397 len=593833 2024-04-07T16:43:34.088 INFO:tasks.workunit.client.0.smithi100.stdout:op 1526 completed, throughput=4.99MB/sec 2024-04-07T16:43:34.088 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-e7gdPXzgcuSr7Cw off=2117678 len=1116939 2024-04-07T16:43:34.091 INFO:tasks.workunit.client.0.smithi100.stdout:op 1527 completed, throughput=5MB/sec 2024-04-07T16:43:34.091 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-8NUJX5H6IBd2fJU off=13472132 len=1314810 2024-04-07T16:43:34.096 INFO:tasks.workunit.client.0.smithi100.stdout:op 1528 completed, throughput=5MB/sec 2024-04-07T16:43:34.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:43:34 smithi144 bash[66654]: cluster 2024-04-07T16:43:33.231088+0000 mgr.y (mgr.25175) 846 : cluster [DBG] pgmap v1488: 105 pgs: 105 active+clean; 126 GiB data, 5.1 GiB used, 710 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.2 MiB/s wr, 5 op/s 2024-04-07T16:43:34.693 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:43:34 smithi100 bash[74942]: cluster 2024-04-07T16:43:33.231088+0000 mgr.y (mgr.25175) 846 : cluster [DBG] pgmap v1488: 105 pgs: 105 active+clean; 126 GiB data, 5.1 GiB used, 710 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.2 MiB/s wr, 5 op/s 2024-04-07T16:43:34.693 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:43:34 smithi100 bash[76202]: cluster 2024-04-07T16:43:33.231088+0000 mgr.y (mgr.25175) 846 : cluster [DBG] pgmap v1488: 105 pgs: 105 active+clean; 126 GiB data, 5.1 GiB used, 710 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.2 MiB/s wr, 5 op/s 2024-04-07T16:43:35.096 INFO:tasks.workunit.client.0.smithi100.stdout: 289: throughput=4.98MB/sec pending data=0 2024-04-07T16:43:35.097 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-RTV9rqdVFvhu2sC off=533504 len=781159 2024-04-07T16:43:35.099 INFO:tasks.workunit.client.0.smithi100.stdout:op 1529 completed, throughput=4.99MB/sec 2024-04-07T16:43:35.099 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-SprRH4wFCa1Sn8c off=10925831 len=1866153 2024-04-07T16:43:35.106 INFO:tasks.workunit.client.0.smithi100.stdout:op 1530 completed, throughput=4.99MB/sec 2024-04-07T16:43:35.106 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-fIvk5B7ZrJ9EFiw off=15679637 len=273145 2024-04-07T16:43:35.107 INFO:tasks.workunit.client.0.smithi100.stdout:op 1531 completed, throughput=4.99MB/sec 2024-04-07T16:43:35.107 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-_YWuBPWS4AuqrUB off=5988731 len=1529717 2024-04-07T16:43:35.111 INFO:tasks.workunit.client.0.smithi100.stdout:op 1532 completed, throughput=5MB/sec 2024-04-07T16:43:35.111 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-nhiPxYrsPbknqom off=5768480 len=217819 2024-04-07T16:43:35.116 INFO:tasks.workunit.client.0.smithi100.stdout:op 1533 completed, throughput=5MB/sec 2024-04-07T16:43:35.116 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-kCMyPrWZXpJqUZP off=3519714 len=684861 2024-04-07T16:43:35.119 INFO:tasks.workunit.client.0.smithi100.stdout:op 1534 completed, throughput=5MB/sec 2024-04-07T16:43:36.120 INFO:tasks.workunit.client.0.smithi100.stdout: 290: throughput=4.98MB/sec pending data=0 2024-04-07T16:43:36.120 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Af4n01HMkRh-WI3 off=1236771 len=1794473 2024-04-07T16:43:36.124 INFO:tasks.workunit.client.0.smithi100.stdout:op 1535 completed, throughput=4.99MB/sec 2024-04-07T16:43:36.124 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Z6ERHcCpxOhwFqI off=5955035 len=508028 2024-04-07T16:43:36.126 INFO:tasks.workunit.client.0.smithi100.stdout:op 1536 completed, throughput=4.99MB/sec 2024-04-07T16:43:36.126 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-vwPFpeGuxgb6zt- off=3721862 len=1878968 2024-04-07T16:43:36.131 INFO:tasks.workunit.client.0.smithi100.stdout:op 1537 completed, throughput=5MB/sec 2024-04-07T16:43:36.131 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-G45jnKZPn8kYe9Z off=1814705 len=1876206 2024-04-07T16:43:36.135 INFO:tasks.workunit.client.0.smithi100.stdout:op 1538 completed, throughput=5MB/sec 2024-04-07T16:43:36.715 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:43:36 smithi100 bash[74942]: cluster 2024-04-07T16:43:35.232226+0000 mgr.y (mgr.25175) 847 : cluster [DBG] pgmap v1489: 105 pgs: 105 active+clean; 126 GiB data, 5.1 GiB used, 710 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.0 MiB/s wr, 4 op/s 2024-04-07T16:43:36.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:43:36 smithi144 bash[66654]: cluster 2024-04-07T16:43:35.232226+0000 mgr.y (mgr.25175) 847 : cluster [DBG] pgmap v1489: 105 pgs: 105 active+clean; 126 GiB data, 5.1 GiB used, 710 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.0 MiB/s wr, 4 op/s 2024-04-07T16:43:37.136 INFO:tasks.workunit.client.0.smithi100.stdout: 291: throughput=4.99MB/sec pending data=0 2024-04-07T16:43:37.136 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-KaIMbTw-yd29Xfj off=23234509 len=713381 2024-04-07T16:43:37.138 INFO:tasks.workunit.client.0.smithi100.stdout:op 1539 completed, throughput=4.99MB/sec 2024-04-07T16:43:37.138 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-VK0QbxQTbeSWfSN off=6100027 len=415355 2024-04-07T16:43:37.141 INFO:tasks.workunit.client.0.smithi100.stdout:op 1540 completed, throughput=4.99MB/sec 2024-04-07T16:43:37.141 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-7lOvc5HGsw2u9Jn off=10649122 len=743293 2024-04-07T16:43:37.145 INFO:tasks.workunit.client.0.smithi100.stdout:op 1541 completed, throughput=4.99MB/sec 2024-04-07T16:43:37.145 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj--14AMEBgl2JrMrw off=4049031 len=618577 2024-04-07T16:43:37.146 INFO:tasks.workunit.client.0.smithi100.stdout:op 1542 completed, throughput=4.99MB/sec 2024-04-07T16:43:37.147 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-wxOjQ2zsOhKrXGO off=2337532 len=523744 2024-04-07T16:43:37.154 INFO:tasks.workunit.client.0.smithi100.stdout:op 1543 completed, throughput=5MB/sec 2024-04-07T16:43:37.154 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-F5j7VKClbjazzLc off=1857976 len=213058 2024-04-07T16:43:37.155 INFO:tasks.workunit.client.0.smithi100.stdout:op 1544 completed, throughput=5MB/sec 2024-04-07T16:43:37.155 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-JUHqJYMEF_cioDc off=590842 len=1294740 2024-04-07T16:43:37.158 INFO:tasks.workunit.client.0.smithi100.stdout:op 1545 completed, throughput=5MB/sec 2024-04-07T16:43:37.193 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:43:36 smithi100 bash[76202]: cluster 2024-04-07T16:43:35.232226+0000 mgr.y (mgr.25175) 847 : cluster [DBG] pgmap v1489: 105 pgs: 105 active+clean; 126 GiB data, 5.1 GiB used, 710 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.0 MiB/s wr, 4 op/s 2024-04-07T16:43:38.159 INFO:tasks.workunit.client.0.smithi100.stdout: 292: throughput=4.98MB/sec pending data=0 2024-04-07T16:43:38.159 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-FBTuGgotkUiUh_0 off=12988546 len=185294 2024-04-07T16:43:38.160 INFO:tasks.workunit.client.0.smithi100.stdout:op 1546 completed, throughput=4.98MB/sec 2024-04-07T16:43:38.161 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-c-RUMyrmGdnBcj_ off=2861090 len=908811 2024-04-07T16:43:38.163 INFO:tasks.workunit.client.0.smithi100.stdout:op 1547 completed, throughput=4.99MB/sec 2024-04-07T16:43:38.163 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-F3DPIUpOkVsWi44 off=7519322 len=1397832 2024-04-07T16:43:38.175 INFO:tasks.workunit.client.0.smithi100.stdout:op 1548 completed, throughput=4.99MB/sec 2024-04-07T16:43:38.175 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-8OiJi1Wzxo5HfF- off=0 len=1151532 2024-04-07T16:43:38.177 INFO:tasks.workunit.client.0.smithi100.stdout:op 1549 completed, throughput=4.99MB/sec 2024-04-07T16:43:38.177 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-QyZLjwW37aW24LN off=1924488 len=1645758 2024-04-07T16:43:38.192 INFO:tasks.workunit.client.0.smithi100.stdout:op 1550 completed, throughput=5MB/sec 2024-04-07T16:43:38.192 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-XlnmDqKGcbebsYJ off=12221939 len=1898110 2024-04-07T16:43:38.196 INFO:tasks.workunit.client.0.smithi100.stdout:op 1551 completed, throughput=5.01MB/sec 2024-04-07T16:43:38.693 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:43:38 smithi100 bash[74942]: cluster 2024-04-07T16:43:37.233408+0000 mgr.y (mgr.25175) 848 : cluster [DBG] pgmap v1490: 105 pgs: 105 active+clean; 126 GiB data, 5.1 GiB used, 710 GiB / 715 GiB avail; 3.1 MiB/s rd, 2.1 MiB/s wr, 4 op/s 2024-04-07T16:43:38.693 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:43:38 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:43:38] "GET /metrics HTTP/1.1" 200 34054 "" "Prometheus/2.43.0" 2024-04-07T16:43:38.693 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:43:38 smithi100 bash[76202]: cluster 2024-04-07T16:43:37.233408+0000 mgr.y (mgr.25175) 848 : cluster [DBG] pgmap v1490: 105 pgs: 105 active+clean; 126 GiB data, 5.1 GiB used, 710 GiB / 715 GiB avail; 3.1 MiB/s rd, 2.1 MiB/s wr, 4 op/s 2024-04-07T16:43:38.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:43:38 smithi144 bash[66654]: cluster 2024-04-07T16:43:37.233408+0000 mgr.y (mgr.25175) 848 : cluster [DBG] pgmap v1490: 105 pgs: 105 active+clean; 126 GiB data, 5.1 GiB used, 710 GiB / 715 GiB avail; 3.1 MiB/s rd, 2.1 MiB/s wr, 4 op/s 2024-04-07T16:43:39.196 INFO:tasks.workunit.client.0.smithi100.stdout: 293: throughput=4.99MB/sec pending data=0 2024-04-07T16:43:39.197 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Bx2gv5MF_sfnJ6i off=1926270 len=464968 2024-04-07T16:43:39.199 INFO:tasks.workunit.client.0.smithi100.stdout:op 1552 completed, throughput=4.99MB/sec 2024-04-07T16:43:39.199 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-bwSXbYe34odoxcu off=2911298 len=353456 2024-04-07T16:43:39.201 INFO:tasks.workunit.client.0.smithi100.stdout:op 1553 completed, throughput=4.99MB/sec 2024-04-07T16:43:39.201 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-vYxtV4_Jv-ga83B off=2459138 len=207357 2024-04-07T16:43:39.202 INFO:tasks.workunit.client.0.smithi100.stdout:op 1554 completed, throughput=4.99MB/sec 2024-04-07T16:43:39.203 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-l7UL4ineMrvBsm- off=1141773 len=833928 2024-04-07T16:43:39.211 INFO:tasks.workunit.client.0.smithi100.stdout:op 1555 completed, throughput=4.99MB/sec 2024-04-07T16:43:39.211 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-8n9wcgkRJtIUG9J off=163677 len=509873 2024-04-07T16:43:39.213 INFO:tasks.workunit.client.0.smithi100.stdout:op 1556 completed, throughput=5MB/sec 2024-04-07T16:43:39.213 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-bNAPPrVWoy8RPAF off=351083 len=1989382 2024-04-07T16:43:39.218 INFO:tasks.workunit.client.0.smithi100.stdout:op 1557 completed, throughput=5MB/sec 2024-04-07T16:43:40.219 INFO:tasks.workunit.client.0.smithi100.stdout: 294: throughput=4.99MB/sec pending data=0 2024-04-07T16:43:40.219 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-wv_hvmwvpk8lS9r off=2580301 len=21548 2024-04-07T16:43:40.223 INFO:tasks.workunit.client.0.smithi100.stdout:op 1558 completed, throughput=4.99MB/sec 2024-04-07T16:43:40.223 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-gy-lSrFHhViDf3u off=19379262 len=1900629 2024-04-07T16:43:40.237 INFO:tasks.workunit.client.0.smithi100.stdout:op 1559 completed, throughput=4.99MB/sec 2024-04-07T16:43:40.237 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-VBSNPv8gdddfo5W off=897208 len=1439039 2024-04-07T16:43:40.242 INFO:tasks.workunit.client.0.smithi100.stdout:op 1560 completed, throughput=5MB/sec 2024-04-07T16:43:40.242 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-vAXjvNJ9Fo668dV off=465667 len=1962483 2024-04-07T16:43:40.254 INFO:tasks.workunit.client.0.smithi100.stdout:op 1561 completed, throughput=5MB/sec 2024-04-07T16:43:40.692 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:43:40 smithi100 bash[74942]: cluster 2024-04-07T16:43:39.234436+0000 mgr.y (mgr.25175) 849 : cluster [DBG] pgmap v1491: 105 pgs: 105 active+clean; 126 GiB data, 5.1 GiB used, 710 GiB / 715 GiB avail; 3.2 MiB/s rd, 2.0 MiB/s wr, 5 op/s 2024-04-07T16:43:40.693 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:43:40 smithi100 bash[76202]: cluster 2024-04-07T16:43:39.234436+0000 mgr.y (mgr.25175) 849 : cluster [DBG] pgmap v1491: 105 pgs: 105 active+clean; 126 GiB data, 5.1 GiB used, 710 GiB / 715 GiB avail; 3.2 MiB/s rd, 2.0 MiB/s wr, 5 op/s 2024-04-07T16:43:40.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:43:40 smithi144 bash[66654]: cluster 2024-04-07T16:43:39.234436+0000 mgr.y (mgr.25175) 849 : cluster [DBG] pgmap v1491: 105 pgs: 105 active+clean; 126 GiB data, 5.1 GiB used, 710 GiB / 715 GiB avail; 3.2 MiB/s rd, 2.0 MiB/s wr, 5 op/s 2024-04-07T16:43:41.254 INFO:tasks.workunit.client.0.smithi100.stdout: 295: throughput=4.99MB/sec pending data=0 2024-04-07T16:43:41.255 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-XtgbTEL6i2i3MUQ off=563154 len=743521 2024-04-07T16:43:41.263 INFO:tasks.workunit.client.0.smithi100.stdout:op 1562 completed, throughput=4.99MB/sec 2024-04-07T16:43:41.263 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-qh5UwTIXljXhQJ5 off=9813348 len=131594 2024-04-07T16:43:41.264 INFO:tasks.workunit.client.0.smithi100.stdout:op 1563 completed, throughput=4.99MB/sec 2024-04-07T16:43:41.264 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-9QiKHQsWpPtWEYn off=0 len=1100978 2024-04-07T16:43:41.267 INFO:tasks.workunit.client.0.smithi100.stdout:op 1564 completed, throughput=4.99MB/sec 2024-04-07T16:43:41.267 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Gjju1j3ULrxbxuA off=22413329 len=1037057 2024-04-07T16:43:41.276 INFO:tasks.workunit.client.0.smithi100.stdout:op 1565 completed, throughput=4.99MB/sec 2024-04-07T16:43:41.276 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-xyEGJ5sC6TlTnRU off=12819829 len=101713 2024-04-07T16:43:41.281 INFO:tasks.workunit.client.0.smithi100.stdout:op 1566 completed, throughput=4.99MB/sec 2024-04-07T16:43:41.281 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-pnoXAwrQfOkIgje off=7197849 len=1530391 2024-04-07T16:43:41.284 INFO:tasks.workunit.client.0.smithi100.stdout:op 1567 completed, throughput=5MB/sec 2024-04-07T16:43:41.284 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Lmw8RAM0flK7T1F off=7090785 len=1473238 2024-04-07T16:43:41.287 INFO:tasks.workunit.client.0.smithi100.stdout:op 1568 completed, throughput=5MB/sec 2024-04-07T16:43:42.287 INFO:tasks.workunit.client.0.smithi100.stdout: 296: throughput=4.99MB/sec pending data=0 2024-04-07T16:43:42.287 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-lVMd5lDl_OUrVc8 off=3473424 len=574535 2024-04-07T16:43:42.294 INFO:tasks.workunit.client.0.smithi100.stdout:op 1569 completed, throughput=4.99MB/sec 2024-04-07T16:43:42.294 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-n4X1HG9-e8q3DqC off=1631555 len=1296860 2024-04-07T16:43:42.305 INFO:tasks.workunit.client.0.smithi100.stdout:op 1570 completed, throughput=4.99MB/sec 2024-04-07T16:43:42.305 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-D7pTa-sgZRknCQz off=9936926 len=185499 2024-04-07T16:43:42.306 INFO:tasks.workunit.client.0.smithi100.stdout:op 1571 completed, throughput=4.99MB/sec 2024-04-07T16:43:42.307 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-QElDCblRAnqUUUc off=4821692 len=1350699 2024-04-07T16:43:42.319 INFO:tasks.workunit.client.0.smithi100.stdout:op 1572 completed, throughput=5MB/sec 2024-04-07T16:43:42.319 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-EMVI1GyZwXOMR37 off=7891076 len=852456 2024-04-07T16:43:42.327 INFO:tasks.workunit.client.0.smithi100.stdout:op 1573 completed, throughput=5MB/sec 2024-04-07T16:43:42.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:43:42 smithi144 bash[66654]: cluster 2024-04-07T16:43:41.235373+0000 mgr.y (mgr.25175) 850 : cluster [DBG] pgmap v1492: 105 pgs: 105 active+clean; 126 GiB data, 5.1 GiB used, 710 GiB / 715 GiB avail; 3.1 MiB/s rd, 1.8 MiB/s wr, 5 op/s 2024-04-07T16:43:42.607 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:43:42 smithi144 bash[95233]: ts=2024-04-07T16:43:42.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:43:42.693 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:43:42 smithi100 bash[74942]: cluster 2024-04-07T16:43:41.235373+0000 mgr.y (mgr.25175) 850 : cluster [DBG] pgmap v1492: 105 pgs: 105 active+clean; 126 GiB data, 5.1 GiB used, 710 GiB / 715 GiB avail; 3.1 MiB/s rd, 1.8 MiB/s wr, 5 op/s 2024-04-07T16:43:42.693 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:43:42 smithi100 bash[76202]: cluster 2024-04-07T16:43:41.235373+0000 mgr.y (mgr.25175) 850 : cluster [DBG] pgmap v1492: 105 pgs: 105 active+clean; 126 GiB data, 5.1 GiB used, 710 GiB / 715 GiB avail; 3.1 MiB/s rd, 1.8 MiB/s wr, 5 op/s 2024-04-07T16:43:43.328 INFO:tasks.workunit.client.0.smithi100.stdout: 297: throughput=4.98MB/sec pending data=0 2024-04-07T16:43:43.328 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-y2Fon0in-u3Wc7v off=8370294 len=247900 2024-04-07T16:43:43.334 INFO:tasks.workunit.client.0.smithi100.stdout:op 1574 completed, throughput=4.98MB/sec 2024-04-07T16:43:43.334 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-1CJ3VpajJsKUBVw off=3803650 len=289811 2024-04-07T16:43:43.339 INFO:tasks.workunit.client.0.smithi100.stdout:op 1575 completed, throughput=4.99MB/sec 2024-04-07T16:43:43.339 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-L3W_FaUF5GPK4Xj off=6635331 len=1778555 2024-04-07T16:43:43.354 INFO:tasks.workunit.client.0.smithi100.stdout:op 1576 completed, throughput=4.99MB/sec 2024-04-07T16:43:43.355 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-_jditG32Vbi4ARW off=23278789 len=1564243 2024-04-07T16:43:43.368 INFO:tasks.workunit.client.0.smithi100.stdout:op 1577 completed, throughput=5MB/sec 2024-04-07T16:43:43.369 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-aj_7xnrmD-lStTw off=8426423 len=1796042 2024-04-07T16:43:43.384 INFO:tasks.workunit.client.0.smithi100.stdout:op 1578 completed, throughput=5MB/sec 2024-04-07T16:43:44.385 INFO:tasks.workunit.client.0.smithi100.stdout: 298: throughput=4.98MB/sec pending data=0 2024-04-07T16:43:44.385 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-m83XYkvQ7zYtX1_ off=8845680 len=531171 2024-04-07T16:43:44.387 INFO:tasks.workunit.client.0.smithi100.stdout:op 1579 completed, throughput=4.99MB/sec 2024-04-07T16:43:44.387 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj---JwM0TkGzoliXk off=3073602 len=1669 2024-04-07T16:43:44.391 INFO:tasks.workunit.client.0.smithi100.stdout:op 1580 completed, throughput=4.99MB/sec 2024-04-07T16:43:44.391 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-02oyuQrGsJDPjm9 off=7565208 len=110760 2024-04-07T16:43:44.396 INFO:tasks.workunit.client.0.smithi100.stdout:op 1581 completed, throughput=4.99MB/sec 2024-04-07T16:43:44.396 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-FmtAr2GUoBVnm8K off=1071224 len=572326 2024-04-07T16:43:44.398 INFO:tasks.workunit.client.0.smithi100.stdout:op 1582 completed, throughput=4.99MB/sec 2024-04-07T16:43:44.398 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Jymqt8Tnv7PaaYM off=205928 len=110584 2024-04-07T16:43:44.402 INFO:tasks.workunit.client.0.smithi100.stdout:op 1583 completed, throughput=4.99MB/sec 2024-04-07T16:43:44.402 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-_H9PA_WfRphE96O off=19626279 len=1499351 2024-04-07T16:43:44.406 INFO:tasks.workunit.client.0.smithi100.stdout:op 1584 completed, throughput=4.99MB/sec 2024-04-07T16:43:44.406 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-c2u0ew66b7fLMV1 off=2647752 len=180995 2024-04-07T16:43:44.411 INFO:tasks.workunit.client.0.smithi100.stdout:op 1585 completed, throughput=4.99MB/sec 2024-04-07T16:43:44.411 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-0RkLU8fuD72alEC off=8034146 len=1407201 2024-04-07T16:43:44.416 INFO:tasks.workunit.client.0.smithi100.stdout:op 1586 completed, throughput=5MB/sec 2024-04-07T16:43:44.416 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-yexrb-XXjMII_UU off=23630339 len=1769752 2024-04-07T16:43:44.420 INFO:tasks.workunit.client.0.smithi100.stdout:op 1587 completed, throughput=5MB/sec 2024-04-07T16:43:44.693 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:43:44 smithi100 bash[74942]: cluster 2024-04-07T16:43:43.236495+0000 mgr.y (mgr.25175) 851 : cluster [DBG] pgmap v1493: 105 pgs: 105 active+clean; 126 GiB data, 5.1 GiB used, 710 GiB / 715 GiB avail; 3.5 MiB/s rd, 1.7 MiB/s wr, 5 op/s 2024-04-07T16:43:44.693 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:43:44 smithi100 bash[76202]: cluster 2024-04-07T16:43:43.236495+0000 mgr.y (mgr.25175) 851 : cluster [DBG] pgmap v1493: 105 pgs: 105 active+clean; 126 GiB data, 5.1 GiB used, 710 GiB / 715 GiB avail; 3.5 MiB/s rd, 1.7 MiB/s wr, 5 op/s 2024-04-07T16:43:44.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:43:44 smithi144 bash[66654]: cluster 2024-04-07T16:43:43.236495+0000 mgr.y (mgr.25175) 851 : cluster [DBG] pgmap v1493: 105 pgs: 105 active+clean; 126 GiB data, 5.1 GiB used, 710 GiB / 715 GiB avail; 3.5 MiB/s rd, 1.7 MiB/s wr, 5 op/s 2024-04-07T16:43:45.422 INFO:tasks.workunit.client.0.smithi100.stdout: 299: throughput=4.99MB/sec pending data=0 2024-04-07T16:43:45.422 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-M_MJNMWPPmYrGaH off=15822280 len=1781092 2024-04-07T16:43:45.427 INFO:tasks.workunit.client.0.smithi100.stdout:op 1588 completed, throughput=4.99MB/sec 2024-04-07T16:43:45.427 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-SMaUl_SE5GKqERk off=4097276 len=1651908 2024-04-07T16:43:45.431 INFO:tasks.workunit.client.0.smithi100.stdout:op 1589 completed, throughput=5MB/sec 2024-04-07T16:43:45.431 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-YFrrb4waUE0rCeg off=162715 len=1704626 2024-04-07T16:43:45.435 INFO:tasks.workunit.client.0.smithi100.stdout:op 1590 completed, throughput=5MB/sec 2024-04-07T16:43:46.436 INFO:tasks.workunit.client.0.smithi100.stdout: 300: throughput=4.99MB/sec pending data=0 2024-04-07T16:43:46.436 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-QnVmQ9TDZ4ZSp81 off=6746693 len=1726291 2024-04-07T16:43:46.450 INFO:tasks.workunit.client.0.smithi100.stdout:op 1591 completed, throughput=4.99MB/sec 2024-04-07T16:43:46.450 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-VdY27zUphg4PMbV off=10970514 len=425363 2024-04-07T16:43:46.451 INFO:tasks.workunit.client.0.smithi100.stdout:op 1592 completed, throughput=4.99MB/sec 2024-04-07T16:43:46.452 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-wtLiFpn_5T9cRsl off=17711426 len=851134 2024-04-07T16:43:46.454 INFO:tasks.workunit.client.0.smithi100.stdout:op 1593 completed, throughput=5MB/sec 2024-04-07T16:43:46.454 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-norUOwNbUIwsNkQ off=729492 len=1656964 2024-04-07T16:43:46.458 INFO:tasks.workunit.client.0.smithi100.stdout:op 1594 completed, throughput=5MB/sec 2024-04-07T16:43:46.693 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:43:46 smithi100 bash[74942]: cluster 2024-04-07T16:43:45.237570+0000 mgr.y (mgr.25175) 852 : cluster [DBG] pgmap v1494: 105 pgs: 105 active+clean; 126 GiB data, 5.1 GiB used, 710 GiB / 715 GiB avail; 3.4 MiB/s rd, 1.7 MiB/s wr, 5 op/s 2024-04-07T16:43:46.693 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:43:46 smithi100 bash[76202]: cluster 2024-04-07T16:43:45.237570+0000 mgr.y (mgr.25175) 852 : cluster [DBG] pgmap v1494: 105 pgs: 105 active+clean; 126 GiB data, 5.1 GiB used, 710 GiB / 715 GiB avail; 3.4 MiB/s rd, 1.7 MiB/s wr, 5 op/s 2024-04-07T16:43:46.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:43:46 smithi144 bash[66654]: cluster 2024-04-07T16:43:45.237570+0000 mgr.y (mgr.25175) 852 : cluster [DBG] pgmap v1494: 105 pgs: 105 active+clean; 126 GiB data, 5.1 GiB used, 710 GiB / 715 GiB avail; 3.4 MiB/s rd, 1.7 MiB/s wr, 5 op/s 2024-04-07T16:43:47.459 INFO:tasks.workunit.client.0.smithi100.stdout: 301: throughput=4.98MB/sec pending data=0 2024-04-07T16:43:47.459 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-bY2AhN8NcEjymir off=3662509 len=1204476 2024-04-07T16:43:47.462 INFO:tasks.workunit.client.0.smithi100.stdout:op 1595 completed, throughput=4.99MB/sec 2024-04-07T16:43:47.463 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-EGmUNLf99TvKz1k off=21837293 len=1001118 2024-04-07T16:43:47.466 INFO:tasks.workunit.client.0.smithi100.stdout:op 1596 completed, throughput=4.99MB/sec 2024-04-07T16:43:47.466 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-oDNUeSr2JfwE1Un off=975242 len=619680 2024-04-07T16:43:47.468 INFO:tasks.workunit.client.0.smithi100.stdout:op 1597 completed, throughput=4.99MB/sec 2024-04-07T16:43:47.468 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-T9Cl_mN7lUzmi2d off=13215127 len=749657 2024-04-07T16:43:47.476 INFO:tasks.workunit.client.0.smithi100.stdout:op 1598 completed, throughput=5MB/sec 2024-04-07T16:43:47.476 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-WMO6wkck7mPkP_6 off=19419026 len=1909498 2024-04-07T16:43:47.481 INFO:tasks.workunit.client.0.smithi100.stdout:op 1599 completed, throughput=5MB/sec 2024-04-07T16:43:47.693 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:43:47 smithi100 bash[74942]: audit 2024-04-07T16:43:46.763671+0000 mon.a (mon.0) 2084 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:43:47.693 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:43:47 smithi100 bash[76202]: audit 2024-04-07T16:43:46.763671+0000 mon.a (mon.0) 2084 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:43:47.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:43:47 smithi144 bash[66654]: audit 2024-04-07T16:43:46.763671+0000 mon.a (mon.0) 2084 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:43:48.481 INFO:tasks.workunit.client.0.smithi100.stdout: 302: throughput=4.98MB/sec pending data=0 2024-04-07T16:43:48.481 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-essQ7GOtY0xZYjl off=12951463 len=1336375 2024-04-07T16:43:48.492 INFO:tasks.workunit.client.0.smithi100.stdout:op 1600 completed, throughput=4.99MB/sec 2024-04-07T16:43:48.492 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-VTfJ_psYJVGMqPo off=4985727 len=1754318 2024-04-07T16:43:48.496 INFO:tasks.workunit.client.0.smithi100.stdout:op 1601 completed, throughput=4.99MB/sec 2024-04-07T16:43:48.496 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-6CCSdH8n7XRKmW5 off=2093726 len=479394 2024-04-07T16:43:48.498 INFO:tasks.workunit.client.0.smithi100.stdout:op 1602 completed, throughput=5MB/sec 2024-04-07T16:43:48.498 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-kNZ-WdBi9tHXiRL off=15074688 len=1753255 2024-04-07T16:43:48.510 INFO:tasks.workunit.client.0.smithi100.stdout:op 1603 completed, throughput=5MB/sec 2024-04-07T16:43:48.693 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:43:48 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:43:48] "GET /metrics HTTP/1.1" 200 34055 "" "Prometheus/2.43.0" 2024-04-07T16:43:48.693 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:43:48 smithi100 bash[74942]: cluster 2024-04-07T16:43:47.238676+0000 mgr.y (mgr.25175) 853 : cluster [DBG] pgmap v1495: 105 pgs: 105 active+clean; 126 GiB data, 5.1 GiB used, 710 GiB / 715 GiB avail; 3.2 MiB/s rd, 1.8 MiB/s wr, 5 op/s 2024-04-07T16:43:48.693 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:43:48 smithi100 bash[76202]: cluster 2024-04-07T16:43:47.238676+0000 mgr.y (mgr.25175) 853 : cluster [DBG] pgmap v1495: 105 pgs: 105 active+clean; 126 GiB data, 5.1 GiB used, 710 GiB / 715 GiB avail; 3.2 MiB/s rd, 1.8 MiB/s wr, 5 op/s 2024-04-07T16:43:48.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:43:48 smithi144 bash[66654]: cluster 2024-04-07T16:43:47.238676+0000 mgr.y (mgr.25175) 853 : cluster [DBG] pgmap v1495: 105 pgs: 105 active+clean; 126 GiB data, 5.1 GiB used, 710 GiB / 715 GiB avail; 3.2 MiB/s rd, 1.8 MiB/s wr, 5 op/s 2024-04-07T16:43:49.510 INFO:tasks.workunit.client.0.smithi100.stdout: 303: throughput=4.98MB/sec pending data=0 2024-04-07T16:43:49.510 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Ls6XuYFOuj-3OG2 off=739734 len=1931909 2024-04-07T16:43:49.517 INFO:tasks.workunit.client.0.smithi100.stdout:op 1604 completed, throughput=4.99MB/sec 2024-04-07T16:43:49.517 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-WA_q2bmmEGIQDjG off=6926992 len=695837 2024-04-07T16:43:49.525 INFO:tasks.workunit.client.0.smithi100.stdout:op 1605 completed, throughput=4.99MB/sec 2024-04-07T16:43:49.525 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-NvoKn3lO40ZGy0T off=3409940 len=1905998 2024-04-07T16:43:49.538 INFO:tasks.workunit.client.0.smithi100.stdout:op 1606 completed, throughput=5MB/sec 2024-04-07T16:43:49.538 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-uayt-Zwu0GEKxhl off=3659231 len=597631 2024-04-07T16:43:49.541 INFO:tasks.workunit.client.0.smithi100.stdout:op 1607 completed, throughput=5MB/sec 2024-04-07T16:43:49.541 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-VgsR9YQhVle4cxn off=1844150 len=32356 2024-04-07T16:43:49.542 INFO:tasks.workunit.client.0.smithi100.stdout:op 1608 completed, throughput=5MB/sec 2024-04-07T16:43:49.542 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-yd5-NSZIuC9VwgQ off=3365488 len=81523 2024-04-07T16:43:49.546 INFO:tasks.workunit.client.0.smithi100.stdout:op 1609 completed, throughput=5MB/sec 2024-04-07T16:43:50.546 INFO:tasks.workunit.client.0.smithi100.stdout: 304: throughput=4.98MB/sec pending data=0 2024-04-07T16:43:50.546 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-CE1KbuRD6J1llTF off=1424868 len=1614461 2024-04-07T16:43:50.558 INFO:tasks.workunit.client.0.smithi100.stdout:op 1610 completed, throughput=4.99MB/sec 2024-04-07T16:43:50.558 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-TsRp6j0lIkCMK-4 off=1281629 len=1145261 2024-04-07T16:43:50.567 INFO:tasks.workunit.client.0.smithi100.stdout:op 1611 completed, throughput=4.99MB/sec 2024-04-07T16:43:50.567 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-0639gMcLNxP7ksb off=58020 len=1466548 2024-04-07T16:43:50.581 INFO:tasks.workunit.client.0.smithi100.stdout:op 1612 completed, throughput=5MB/sec 2024-04-07T16:43:50.582 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-AXxo0P_bfykxfKk off=1489824 len=1159350 2024-04-07T16:43:50.585 INFO:tasks.workunit.client.0.smithi100.stdout:op 1613 completed, throughput=5MB/sec 2024-04-07T16:43:50.585 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-e24MYICiUaRUOes off=5908261 len=352820 2024-04-07T16:43:50.587 INFO:tasks.workunit.client.0.smithi100.stdout:op 1614 completed, throughput=5MB/sec 2024-04-07T16:43:50.692 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:43:50 smithi100 bash[74942]: cluster 2024-04-07T16:43:49.239531+0000 mgr.y (mgr.25175) 854 : cluster [DBG] pgmap v1496: 105 pgs: 105 active+clean; 126 GiB data, 5.1 GiB used, 710 GiB / 715 GiB avail; 3.0 MiB/s rd, 1.9 MiB/s wr, 5 op/s 2024-04-07T16:43:50.693 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:43:50 smithi100 bash[76202]: cluster 2024-04-07T16:43:49.239531+0000 mgr.y (mgr.25175) 854 : cluster [DBG] pgmap v1496: 105 pgs: 105 active+clean; 126 GiB data, 5.1 GiB used, 710 GiB / 715 GiB avail; 3.0 MiB/s rd, 1.9 MiB/s wr, 5 op/s 2024-04-07T16:43:50.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:43:50 smithi144 bash[66654]: cluster 2024-04-07T16:43:49.239531+0000 mgr.y (mgr.25175) 854 : cluster [DBG] pgmap v1496: 105 pgs: 105 active+clean; 126 GiB data, 5.1 GiB used, 710 GiB / 715 GiB avail; 3.0 MiB/s rd, 1.9 MiB/s wr, 5 op/s 2024-04-07T16:43:51.587 INFO:tasks.workunit.client.0.smithi100.stdout: 305: throughput=4.99MB/sec pending data=0 2024-04-07T16:43:51.587 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-omjEmfKtj8TCs4h off=5543156 len=1462961 2024-04-07T16:43:51.600 INFO:tasks.workunit.client.0.smithi100.stdout:op 1615 completed, throughput=4.99MB/sec 2024-04-07T16:43:51.600 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-3sQ6QojZpZVvrRD off=13413789 len=382574 2024-04-07T16:43:51.605 INFO:tasks.workunit.client.0.smithi100.stdout:op 1616 completed, throughput=4.99MB/sec 2024-04-07T16:43:51.605 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-LfvCsO-s92WDNZk off=3239995 len=550661 2024-04-07T16:43:51.612 INFO:tasks.workunit.client.0.smithi100.stdout:op 1617 completed, throughput=4.99MB/sec 2024-04-07T16:43:51.612 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-A3N7mOsHll0iIKD off=1748165 len=506569 2024-04-07T16:43:51.617 INFO:tasks.workunit.client.0.smithi100.stdout:op 1618 completed, throughput=4.99MB/sec 2024-04-07T16:43:51.618 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-t8VFtKfLR6Jgav9 off=18605228 len=2057347 2024-04-07T16:43:51.632 INFO:tasks.workunit.client.0.smithi100.stdout:op 1619 completed, throughput=5MB/sec 2024-04-07T16:43:51.632 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-bdLBlKirBu4uKA_ off=2391460 len=985596 2024-04-07T16:43:51.635 INFO:tasks.workunit.client.0.smithi100.stdout:op 1620 completed, throughput=5MB/sec 2024-04-07T16:43:52.607 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:43:52 smithi144 bash[95233]: ts=2024-04-07T16:43:52.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:43:52.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:43:52 smithi144 bash[66654]: cluster 2024-04-07T16:43:51.240443+0000 mgr.y (mgr.25175) 855 : cluster [DBG] pgmap v1497: 105 pgs: 105 active+clean; 126 GiB data, 5.1 GiB used, 710 GiB / 715 GiB avail; 3.1 MiB/s rd, 2.0 MiB/s wr, 5 op/s 2024-04-07T16:43:52.608 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:43:52 smithi144 bash[66654]: audit 2024-04-07T16:43:51.679137+0000 mon.a (mon.0) 2085 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:43:52.608 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:43:52 smithi144 bash[66654]: audit 2024-04-07T16:43:52.262661+0000 mon.a (mon.0) 2086 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:43:52.608 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:43:52 smithi144 bash[66654]: audit 2024-04-07T16:43:52.271779+0000 mon.a (mon.0) 2087 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:43:52.635 INFO:tasks.workunit.client.0.smithi100.stdout: 306: throughput=4.99MB/sec pending data=0 2024-04-07T16:43:52.635 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-BqGjcUZe6f8VdXl off=2258197 len=120391 2024-04-07T16:43:52.641 INFO:tasks.workunit.client.0.smithi100.stdout:op 1621 completed, throughput=4.99MB/sec 2024-04-07T16:43:52.641 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-82LpxEodys0KXly off=3445208 len=127385 2024-04-07T16:43:52.642 INFO:tasks.workunit.client.0.smithi100.stdout:op 1622 completed, throughput=4.99MB/sec 2024-04-07T16:43:52.642 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-ntdqdmxqIozykx1 off=1996187 len=390625 2024-04-07T16:43:52.646 INFO:tasks.workunit.client.0.smithi100.stdout:op 1623 completed, throughput=4.99MB/sec 2024-04-07T16:43:52.646 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-0vvySpag_Vr3-j6 off=1402480 len=103727 2024-04-07T16:43:52.650 INFO:tasks.workunit.client.0.smithi100.stdout:op 1624 completed, throughput=4.99MB/sec 2024-04-07T16:43:52.650 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-LbPA1uxBf-3JfQJ off=5759163 len=130469 2024-04-07T16:43:52.654 INFO:tasks.workunit.client.0.smithi100.stdout:op 1625 completed, throughput=4.99MB/sec 2024-04-07T16:43:52.654 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-1vi8ik4_ugL8u-w off=18555294 len=1452557 2024-04-07T16:43:52.659 INFO:tasks.workunit.client.0.smithi100.stdout:op 1626 completed, throughput=4.99MB/sec 2024-04-07T16:43:52.659 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-4fSyG44nfyPZdah off=12579770 len=1217207 2024-04-07T16:43:52.670 INFO:tasks.workunit.client.0.smithi100.stdout:op 1627 completed, throughput=5MB/sec 2024-04-07T16:43:52.670 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-0YVUAVYClDOg_kP off=13847282 len=436848 2024-04-07T16:43:52.676 INFO:tasks.workunit.client.0.smithi100.stdout:op 1628 completed, throughput=5MB/sec 2024-04-07T16:43:52.676 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-6SISLknx0Y-ICXA off=1760924 len=1190172 2024-04-07T16:43:52.684 INFO:tasks.workunit.client.0.smithi100.stdout:op 1629 completed, throughput=5MB/sec 2024-04-07T16:43:52.693 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:43:52 smithi100 bash[74942]: cluster 2024-04-07T16:43:51.240443+0000 mgr.y (mgr.25175) 855 : cluster [DBG] pgmap v1497: 105 pgs: 105 active+clean; 126 GiB data, 5.1 GiB used, 710 GiB / 715 GiB avail; 3.1 MiB/s rd, 2.0 MiB/s wr, 5 op/s 2024-04-07T16:43:52.693 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:43:52 smithi100 bash[74942]: audit 2024-04-07T16:43:51.679137+0000 mon.a (mon.0) 2085 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:43:52.693 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:43:52 smithi100 bash[74942]: audit 2024-04-07T16:43:52.262661+0000 mon.a (mon.0) 2086 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:43:52.693 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:43:52 smithi100 bash[74942]: audit 2024-04-07T16:43:52.271779+0000 mon.a (mon.0) 2087 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:43:52.693 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:43:52 smithi100 bash[76202]: cluster 2024-04-07T16:43:51.240443+0000 mgr.y (mgr.25175) 855 : cluster [DBG] pgmap v1497: 105 pgs: 105 active+clean; 126 GiB data, 5.1 GiB used, 710 GiB / 715 GiB avail; 3.1 MiB/s rd, 2.0 MiB/s wr, 5 op/s 2024-04-07T16:43:52.693 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:43:52 smithi100 bash[76202]: audit 2024-04-07T16:43:51.679137+0000 mon.a (mon.0) 2085 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:43:52.693 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:43:52 smithi100 bash[76202]: audit 2024-04-07T16:43:52.262661+0000 mon.a (mon.0) 2086 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:43:52.694 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:43:52 smithi100 bash[76202]: audit 2024-04-07T16:43:52.271779+0000 mon.a (mon.0) 2087 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:43:53.685 INFO:tasks.workunit.client.0.smithi100.stdout: 307: throughput=4.99MB/sec pending data=0 2024-04-07T16:43:53.685 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-gUg5Pu5-nxP4Y7z off=639524 len=221029 2024-04-07T16:43:53.690 INFO:tasks.workunit.client.0.smithi100.stdout:op 1630 completed, throughput=4.99MB/sec 2024-04-07T16:43:53.690 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-GydkTbhwZad1SNi off=2869846 len=1817718 2024-04-07T16:43:53.692 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:43:53 smithi100 bash[74942]: audit 2024-04-07T16:43:53.030724+0000 mon.a (mon.0) 2088 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:43:53.693 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:43:53 smithi100 bash[74942]: audit 2024-04-07T16:43:53.032395+0000 mon.a (mon.0) 2089 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:43:53.693 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:43:53 smithi100 bash[74942]: audit 2024-04-07T16:43:53.044854+0000 mon.a (mon.0) 2090 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:43:53.693 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:43:53 smithi100 bash[76202]: audit 2024-04-07T16:43:53.030724+0000 mon.a (mon.0) 2088 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:43:53.693 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:43:53 smithi100 bash[76202]: audit 2024-04-07T16:43:53.032395+0000 mon.a (mon.0) 2089 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:43:53.693 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:43:53 smithi100 bash[76202]: audit 2024-04-07T16:43:53.044854+0000 mon.a (mon.0) 2090 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:43:53.694 INFO:tasks.workunit.client.0.smithi100.stdout:op 1631 completed, throughput=4.99MB/sec 2024-04-07T16:43:53.694 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-oJysHY6Kj3ALfyu off=5180453 len=181737 2024-04-07T16:43:53.699 INFO:tasks.workunit.client.0.smithi100.stdout:op 1632 completed, throughput=4.99MB/sec 2024-04-07T16:43:53.699 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-d52oSKLDFmDC4YB off=3517170 len=565819 2024-04-07T16:43:53.702 INFO:tasks.workunit.client.0.smithi100.stdout:op 1633 completed, throughput=4.99MB/sec 2024-04-07T16:43:53.702 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-xDz7Qpy902Punh_ off=11766870 len=686532 2024-04-07T16:43:53.705 INFO:tasks.workunit.client.0.smithi100.stdout:op 1634 completed, throughput=5MB/sec 2024-04-07T16:43:53.705 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-666Foq8RvXdAWv6 off=501599 len=987661 2024-04-07T16:43:53.708 INFO:tasks.workunit.client.0.smithi100.stdout:op 1635 completed, throughput=5MB/sec 2024-04-07T16:43:53.708 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-oG6ZlJ-5DBXetZ0 off=0 len=1734126 2024-04-07T16:43:53.712 INFO:tasks.workunit.client.0.smithi100.stdout:op 1636 completed, throughput=5MB/sec 2024-04-07T16:43:53.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:43:53 smithi144 bash[66654]: audit 2024-04-07T16:43:53.030724+0000 mon.a (mon.0) 2088 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:43:53.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:43:53 smithi144 bash[66654]: audit 2024-04-07T16:43:53.032395+0000 mon.a (mon.0) 2089 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:43:53.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:43:53 smithi144 bash[66654]: audit 2024-04-07T16:43:53.044854+0000 mon.a (mon.0) 2090 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:43:54.693 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:43:54 smithi100 bash[74942]: cluster 2024-04-07T16:43:53.241597+0000 mgr.y (mgr.25175) 856 : cluster [DBG] pgmap v1498: 105 pgs: 105 active+clean; 126 GiB data, 5.2 GiB used, 710 GiB / 715 GiB avail; 2.8 MiB/s rd, 2.7 MiB/s wr, 5 op/s 2024-04-07T16:43:54.693 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:43:54 smithi100 bash[76202]: cluster 2024-04-07T16:43:53.241597+0000 mgr.y (mgr.25175) 856 : cluster [DBG] pgmap v1498: 105 pgs: 105 active+clean; 126 GiB data, 5.2 GiB used, 710 GiB / 715 GiB avail; 2.8 MiB/s rd, 2.7 MiB/s wr, 5 op/s 2024-04-07T16:43:54.712 INFO:tasks.workunit.client.0.smithi100.stdout: 308: throughput=4.99MB/sec pending data=0 2024-04-07T16:43:54.712 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-mSRWZ8AZr1sv_wv off=13255679 len=1177182 2024-04-07T16:43:54.721 INFO:tasks.workunit.client.0.smithi100.stdout:op 1637 completed, throughput=4.99MB/sec 2024-04-07T16:43:54.722 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-ZqPRUuQXpUEJtlY off=28883 len=281404 2024-04-07T16:43:54.727 INFO:tasks.workunit.client.0.smithi100.stdout:op 1638 completed, throughput=4.99MB/sec 2024-04-07T16:43:54.727 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-60DCVDZeY5Aap5T off=5375070 len=1307068 2024-04-07T16:43:54.730 INFO:tasks.workunit.client.0.smithi100.stdout:op 1639 completed, throughput=5MB/sec 2024-04-07T16:43:54.730 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-kBmVHN1c-iLsn1u off=3176443 len=1324002 2024-04-07T16:43:54.734 INFO:tasks.workunit.client.0.smithi100.stdout:op 1640 completed, throughput=5MB/sec 2024-04-07T16:43:54.734 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-OqW5aQTLfWRmoxD off=5882898 len=291316 2024-04-07T16:43:54.736 INFO:tasks.workunit.client.0.smithi100.stdout:op 1641 completed, throughput=5MB/sec 2024-04-07T16:43:54.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:43:54 smithi144 bash[66654]: cluster 2024-04-07T16:43:53.241597+0000 mgr.y (mgr.25175) 856 : cluster [DBG] pgmap v1498: 105 pgs: 105 active+clean; 126 GiB data, 5.2 GiB used, 710 GiB / 715 GiB avail; 2.8 MiB/s rd, 2.7 MiB/s wr, 5 op/s 2024-04-07T16:43:55.737 INFO:tasks.workunit.client.0.smithi100.stdout: 309: throughput=4.99MB/sec pending data=0 2024-04-07T16:43:55.737 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-uHMF_m5djwxlXNM off=3812262 len=1111999 2024-04-07T16:43:55.740 INFO:tasks.workunit.client.0.smithi100.stdout:op 1642 completed, throughput=4.99MB/sec 2024-04-07T16:43:55.740 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-uBhkeAoL4M-mrRM off=3203494 len=1789538 2024-04-07T16:43:55.754 INFO:tasks.workunit.client.0.smithi100.stdout:op 1643 completed, throughput=4.99MB/sec 2024-04-07T16:43:55.754 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-B1iZERVzMJ7CcCq off=17587367 len=895398 2024-04-07T16:43:55.763 INFO:tasks.workunit.client.0.smithi100.stdout:op 1644 completed, throughput=5MB/sec 2024-04-07T16:43:55.763 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-NVBX-IdjUcdEddR off=8076810 len=959888 2024-04-07T16:43:55.765 INFO:tasks.workunit.client.0.smithi100.stdout:op 1645 completed, throughput=5MB/sec 2024-04-07T16:43:55.765 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-lARY5koUOmhktno off=12182228 len=234960 2024-04-07T16:43:55.771 INFO:tasks.workunit.client.0.smithi100.stdout:op 1646 completed, throughput=5MB/sec 2024-04-07T16:43:55.771 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-hFt3ywMcDdLj7Ks off=14099258 len=156606 2024-04-07T16:43:55.776 INFO:tasks.workunit.client.0.smithi100.stdout:op 1647 completed, throughput=5MB/sec 2024-04-07T16:43:55.776 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-YMOROv3EHMnXBne off=4914679 len=420097 2024-04-07T16:43:55.778 INFO:tasks.workunit.client.0.smithi100.stdout:op 1648 completed, throughput=5MB/sec 2024-04-07T16:43:56.693 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:43:56 smithi100 bash[74942]: cluster 2024-04-07T16:43:55.242565+0000 mgr.y (mgr.25175) 857 : cluster [DBG] pgmap v1499: 105 pgs: 105 active+clean; 126 GiB data, 5.2 GiB used, 710 GiB / 715 GiB avail; 2.4 MiB/s rd, 2.8 MiB/s wr, 5 op/s 2024-04-07T16:43:56.693 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:43:56 smithi100 bash[76202]: cluster 2024-04-07T16:43:55.242565+0000 mgr.y (mgr.25175) 857 : cluster [DBG] pgmap v1499: 105 pgs: 105 active+clean; 126 GiB data, 5.2 GiB used, 710 GiB / 715 GiB avail; 2.4 MiB/s rd, 2.8 MiB/s wr, 5 op/s 2024-04-07T16:43:56.778 INFO:tasks.workunit.client.0.smithi100.stdout: 310: throughput=4.99MB/sec pending data=0 2024-04-07T16:43:56.779 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-uNEnHn6vErivj-7 off=11339164 len=1000933 2024-04-07T16:43:56.788 INFO:tasks.workunit.client.0.smithi100.stdout:op 1649 completed, throughput=4.99MB/sec 2024-04-07T16:43:56.788 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-ojMZDgkW2RhZozh off=5013165 len=1495975 2024-04-07T16:43:56.799 INFO:tasks.workunit.client.0.smithi100.stdout:op 1650 completed, throughput=4.99MB/sec 2024-04-07T16:43:56.800 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Y_lkOWm7hOUe-h0 off=11053296 len=65625 2024-04-07T16:43:56.800 INFO:tasks.workunit.client.0.smithi100.stdout:op 1651 completed, throughput=4.99MB/sec 2024-04-07T16:43:56.801 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-VZP2108ocdRH4B9 off=1182286 len=1532815 2024-04-07T16:43:56.804 INFO:tasks.workunit.client.0.smithi100.stdout:op 1652 completed, throughput=5MB/sec 2024-04-07T16:43:56.804 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-D7oitiokZQXbqUa off=9845774 len=1046622 2024-04-07T16:43:56.813 INFO:tasks.workunit.client.0.smithi100.stdout:op 1653 completed, throughput=5MB/sec 2024-04-07T16:43:56.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:43:56 smithi144 bash[66654]: cluster 2024-04-07T16:43:55.242565+0000 mgr.y (mgr.25175) 857 : cluster [DBG] pgmap v1499: 105 pgs: 105 active+clean; 126 GiB data, 5.2 GiB used, 710 GiB / 715 GiB avail; 2.4 MiB/s rd, 2.8 MiB/s wr, 5 op/s 2024-04-07T16:43:57.814 INFO:tasks.workunit.client.0.smithi100.stdout: 311: throughput=4.98MB/sec pending data=0 2024-04-07T16:43:57.814 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-aqslNfBqpP-DzAW off=5831404 len=205157 2024-04-07T16:43:57.815 INFO:tasks.workunit.client.0.smithi100.stdout:op 1654 completed, throughput=4.99MB/sec 2024-04-07T16:43:57.815 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-RVY7FCuzaLKRJGG off=18239688 len=1332149 2024-04-07T16:43:57.819 INFO:tasks.workunit.client.0.smithi100.stdout:op 1655 completed, throughput=4.99MB/sec 2024-04-07T16:43:57.819 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-rkudWEI1Ac0BB5Z off=3998417 len=944799 2024-04-07T16:43:57.822 INFO:tasks.workunit.client.0.smithi100.stdout:op 1656 completed, throughput=4.99MB/sec 2024-04-07T16:43:57.822 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-jcTUuaecUhTTGRE off=3476839 len=1951099 2024-04-07T16:43:57.836 INFO:tasks.workunit.client.0.smithi100.stdout:op 1657 completed, throughput=5MB/sec 2024-04-07T16:43:57.836 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-gEGV2am66bVv5rn off=5260087 len=102931 2024-04-07T16:43:57.841 INFO:tasks.workunit.client.0.smithi100.stdout:op 1658 completed, throughput=5MB/sec 2024-04-07T16:43:57.841 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-HmrGi-QwstaHMo7 off=806055 len=286751 2024-04-07T16:43:57.843 INFO:tasks.workunit.client.0.smithi100.stdout:op 1659 completed, throughput=5MB/sec 2024-04-07T16:43:57.843 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-A4W_V0k0TZeX8ZX off=18644386 len=1263773 2024-04-07T16:43:57.846 INFO:tasks.workunit.client.0.smithi100.stdout:op 1660 completed, throughput=5MB/sec 2024-04-07T16:43:58.693 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:43:58 smithi100 bash[74942]: cluster 2024-04-07T16:43:57.243556+0000 mgr.y (mgr.25175) 858 : cluster [DBG] pgmap v1500: 105 pgs: 105 active+clean; 126 GiB data, 5.2 GiB used, 710 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.6 MiB/s wr, 5 op/s 2024-04-07T16:43:58.693 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:43:58 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:43:58] "GET /metrics HTTP/1.1" 200 34055 "" "Prometheus/2.43.0" 2024-04-07T16:43:58.693 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:43:58 smithi100 bash[76202]: cluster 2024-04-07T16:43:57.243556+0000 mgr.y (mgr.25175) 858 : cluster [DBG] pgmap v1500: 105 pgs: 105 active+clean; 126 GiB data, 5.2 GiB used, 710 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.6 MiB/s wr, 5 op/s 2024-04-07T16:43:58.846 INFO:tasks.workunit.client.0.smithi100.stdout: 312: throughput=4.99MB/sec pending data=0 2024-04-07T16:43:58.847 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-qj-fUW2uN23xl5n off=278832 len=263435 2024-04-07T16:43:58.852 INFO:tasks.workunit.client.0.smithi100.stdout:op 1661 completed, throughput=4.99MB/sec 2024-04-07T16:43:58.852 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-eMkNsVh9JunalKg off=2222523 len=1228137 2024-04-07T16:43:58.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:43:58 smithi144 bash[66654]: cluster 2024-04-07T16:43:57.243556+0000 mgr.y (mgr.25175) 858 : cluster [DBG] pgmap v1500: 105 pgs: 105 active+clean; 126 GiB data, 5.2 GiB used, 710 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.6 MiB/s wr, 5 op/s 2024-04-07T16:43:58.863 INFO:tasks.workunit.client.0.smithi100.stdout:op 1662 completed, throughput=4.99MB/sec 2024-04-07T16:43:58.863 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-aDqw9-s1hu-r-1E off=7489675 len=1249976 2024-04-07T16:43:58.872 INFO:tasks.workunit.client.0.smithi100.stdout:op 1663 completed, throughput=4.99MB/sec 2024-04-07T16:43:58.873 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-LT-ZhvClwiSNXNo off=9529402 len=285134 2024-04-07T16:43:58.874 INFO:tasks.workunit.client.0.smithi100.stdout:op 1664 completed, throughput=5MB/sec 2024-04-07T16:43:58.874 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-GZLKEP7ZDCUiHFm off=6970370 len=1336323 2024-04-07T16:43:58.886 INFO:tasks.workunit.client.0.smithi100.stdout:op 1665 completed, throughput=5MB/sec 2024-04-07T16:43:58.886 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-_z5Q33k_gksJR6O off=3364951 len=884743 2024-04-07T16:43:58.889 INFO:tasks.workunit.client.0.smithi100.stdout:op 1666 completed, throughput=5MB/sec 2024-04-07T16:43:59.889 INFO:tasks.workunit.client.0.smithi100.stdout: 313: throughput=4.99MB/sec pending data=0 2024-04-07T16:43:59.889 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-ZQzUCurdI5z8IK0 off=9009173 len=276964 2024-04-07T16:43:59.894 INFO:tasks.workunit.client.0.smithi100.stdout:op 1667 completed, throughput=4.99MB/sec 2024-04-07T16:43:59.895 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-l4I3zSGqh2eWQIW off=4599552 len=2071174 2024-04-07T16:43:59.899 INFO:tasks.workunit.client.0.smithi100.stdout:op 1668 completed, throughput=4.99MB/sec 2024-04-07T16:43:59.899 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-dnnVoD7nlKV1GUU off=8549734 len=1964836 2024-04-07T16:43:59.904 INFO:tasks.workunit.client.0.smithi100.stdout:op 1669 completed, throughput=5MB/sec 2024-04-07T16:43:59.904 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-2arR6LYVn-D5iNy off=615101 len=185329 2024-04-07T16:43:59.909 INFO:tasks.workunit.client.0.smithi100.stdout:op 1670 completed, throughput=5MB/sec 2024-04-07T16:43:59.909 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-gTZLOrRmCCVLF7H off=337493 len=1092844 2024-04-07T16:43:59.918 INFO:tasks.workunit.client.0.smithi100.stdout:op 1671 completed, throughput=5MB/sec 2024-04-07T16:44:00.693 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:44:00 smithi100 bash[74942]: cluster 2024-04-07T16:43:59.244391+0000 mgr.y (mgr.25175) 859 : cluster [DBG] pgmap v1501: 105 pgs: 105 active+clean; 126 GiB data, 5.2 GiB used, 710 GiB / 715 GiB avail; 2.6 MiB/s rd, 2.6 MiB/s wr, 5 op/s 2024-04-07T16:44:00.693 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:44:00 smithi100 bash[76202]: cluster 2024-04-07T16:43:59.244391+0000 mgr.y (mgr.25175) 859 : cluster [DBG] pgmap v1501: 105 pgs: 105 active+clean; 126 GiB data, 5.2 GiB used, 710 GiB / 715 GiB avail; 2.6 MiB/s rd, 2.6 MiB/s wr, 5 op/s 2024-04-07T16:44:00.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:44:00 smithi144 bash[66654]: cluster 2024-04-07T16:43:59.244391+0000 mgr.y (mgr.25175) 859 : cluster [DBG] pgmap v1501: 105 pgs: 105 active+clean; 126 GiB data, 5.2 GiB used, 710 GiB / 715 GiB avail; 2.6 MiB/s rd, 2.6 MiB/s wr, 5 op/s 2024-04-07T16:44:00.919 INFO:tasks.workunit.client.0.smithi100.stdout: 314: throughput=4.99MB/sec pending data=0 2024-04-07T16:44:00.919 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-0MRoNtRQMTj5Dpz off=6824252 len=1983376 2024-04-07T16:44:00.923 INFO:tasks.workunit.client.0.smithi100.stdout:op 1672 completed, throughput=4.99MB/sec 2024-04-07T16:44:00.924 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-j-OIkfGSUEj5Al1 off=13226894 len=1403976 2024-04-07T16:44:00.928 INFO:tasks.workunit.client.0.smithi100.stdout:op 1673 completed, throughput=5MB/sec 2024-04-07T16:44:00.928 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-fWPmJ3ASP41icOr off=4377990 len=638692 2024-04-07T16:44:00.930 INFO:tasks.workunit.client.0.smithi100.stdout:op 1674 completed, throughput=5MB/sec 2024-04-07T16:44:00.930 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-kNipAZ8Ir9B6x_C off=4524290 len=626530 2024-04-07T16:44:00.934 INFO:tasks.workunit.client.0.smithi100.stdout:op 1675 completed, throughput=5MB/sec 2024-04-07T16:44:01.934 INFO:tasks.workunit.client.0.smithi100.stdout: 315: throughput=4.98MB/sec pending data=0 2024-04-07T16:44:01.934 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-W8wiCtV9qmw4FAA off=17827160 len=1668927 2024-04-07T16:44:01.938 INFO:tasks.workunit.client.0.smithi100.stdout:op 1676 completed, throughput=4.99MB/sec 2024-04-07T16:44:01.938 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-5ZNeW6bGcfn6I89 off=18863417 len=2006932 2024-04-07T16:44:01.944 INFO:tasks.workunit.client.0.smithi100.stdout:op 1677 completed, throughput=5MB/sec 2024-04-07T16:44:01.944 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-U8JwbraGXdqA1XM off=18327658 len=1665929 2024-04-07T16:44:01.955 INFO:tasks.workunit.client.0.smithi100.stdout:op 1678 completed, throughput=5MB/sec 2024-04-07T16:44:02.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:44:02 smithi144 bash[66654]: cluster 2024-04-07T16:44:01.245296+0000 mgr.y (mgr.25175) 860 : cluster [DBG] pgmap v1502: 105 pgs: 105 active+clean; 126 GiB data, 5.2 GiB used, 710 GiB / 715 GiB avail; 2.6 MiB/s rd, 2.4 MiB/s wr, 5 op/s 2024-04-07T16:44:02.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:44:02 smithi144 bash[66654]: audit 2024-04-07T16:44:01.764119+0000 mon.a (mon.0) 2091 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:44:02.608 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:44:02 smithi144 bash[95233]: ts=2024-04-07T16:44:02.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:44:02.693 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:44:02 smithi100 bash[74942]: cluster 2024-04-07T16:44:01.245296+0000 mgr.y (mgr.25175) 860 : cluster [DBG] pgmap v1502: 105 pgs: 105 active+clean; 126 GiB data, 5.2 GiB used, 710 GiB / 715 GiB avail; 2.6 MiB/s rd, 2.4 MiB/s wr, 5 op/s 2024-04-07T16:44:02.693 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:44:02 smithi100 bash[74942]: audit 2024-04-07T16:44:01.764119+0000 mon.a (mon.0) 2091 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:44:02.693 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:44:02 smithi100 bash[76202]: cluster 2024-04-07T16:44:01.245296+0000 mgr.y (mgr.25175) 860 : cluster [DBG] pgmap v1502: 105 pgs: 105 active+clean; 126 GiB data, 5.2 GiB used, 710 GiB / 715 GiB avail; 2.6 MiB/s rd, 2.4 MiB/s wr, 5 op/s 2024-04-07T16:44:02.693 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:44:02 smithi100 bash[76202]: audit 2024-04-07T16:44:01.764119+0000 mon.a (mon.0) 2091 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:44:02.955 INFO:tasks.workunit.client.0.smithi100.stdout: 316: throughput=4.98MB/sec pending data=0 2024-04-07T16:44:02.955 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-APBr_nwigZKKgqw off=156911 len=514744 2024-04-07T16:44:02.958 INFO:tasks.workunit.client.0.smithi100.stdout:op 1679 completed, throughput=4.99MB/sec 2024-04-07T16:44:02.958 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-1V08a_8KApoisoB off=4595796 len=260274 2024-04-07T16:44:02.963 INFO:tasks.workunit.client.0.smithi100.stdout:op 1680 completed, throughput=4.99MB/sec 2024-04-07T16:44:02.963 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-FreECrGGl9fFEJd off=7448600 len=749982 2024-04-07T16:44:02.966 INFO:tasks.workunit.client.0.smithi100.stdout:op 1681 completed, throughput=4.99MB/sec 2024-04-07T16:44:02.966 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-lkLLTuEhv23B-xe off=7669131 len=25918 2024-04-07T16:44:02.966 INFO:tasks.workunit.client.0.smithi100.stdout:op 1682 completed, throughput=4.99MB/sec 2024-04-07T16:44:02.967 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-gYtbnzoIuzz8n64 off=1263784 len=813349 2024-04-07T16:44:02.969 INFO:tasks.workunit.client.0.smithi100.stdout:op 1683 completed, throughput=4.99MB/sec 2024-04-07T16:44:02.970 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-p6QJEJzrqIfi9al off=5796722 len=1747835 2024-04-07T16:44:02.983 INFO:tasks.workunit.client.0.smithi100.stdout:op 1684 completed, throughput=5MB/sec 2024-04-07T16:44:02.983 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-DfpZzp_0d1yHjA3 off=2488256 len=1002465 2024-04-07T16:44:02.987 INFO:tasks.workunit.client.0.smithi100.stdout:op 1685 completed, throughput=5MB/sec 2024-04-07T16:44:02.988 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-GlXApdflrNcCwbY off=3503516 len=235136 2024-04-07T16:44:02.993 INFO:tasks.workunit.client.0.smithi100.stdout:op 1686 completed, throughput=5MB/sec 2024-04-07T16:44:02.993 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-R_8tqeMq9nDFW05 off=6556439 len=682794 2024-04-07T16:44:03.000 INFO:tasks.workunit.client.0.smithi100.stdout:op 1687 completed, throughput=5MB/sec 2024-04-07T16:44:04.000 INFO:tasks.workunit.client.0.smithi100.stdout: 317: throughput=4.99MB/sec pending data=0 2024-04-07T16:44:04.001 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-OdcYAq9WEeeum6z off=6321719 len=113111 2024-04-07T16:44:04.002 INFO:tasks.workunit.client.0.smithi100.stdout:op 1688 completed, throughput=4.99MB/sec 2024-04-07T16:44:04.002 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-OZzgXD44weGpVB0 off=7032889 len=501822 2024-04-07T16:44:04.004 INFO:tasks.workunit.client.0.smithi100.stdout:op 1689 completed, throughput=4.99MB/sec 2024-04-07T16:44:04.004 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-TsEMOurzrLRVsjg off=7398634 len=486590 2024-04-07T16:44:04.005 INFO:tasks.workunit.client.0.smithi100.stdout:op 1690 completed, throughput=4.99MB/sec 2024-04-07T16:44:04.005 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-5o9LvyodnMPuXqH off=6556178 len=2056093 2024-04-07T16:44:04.021 INFO:tasks.workunit.client.0.smithi100.stdout:op 1691 completed, throughput=5MB/sec 2024-04-07T16:44:04.022 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-lXyy9aAp5-LdZ4y off=5196001 len=1506013 2024-04-07T16:44:04.034 INFO:tasks.workunit.client.0.smithi100.stdout:op 1692 completed, throughput=5MB/sec 2024-04-07T16:44:04.034 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-rILrXyd7IOC7FXo off=3829955 len=1659877 2024-04-07T16:44:04.048 INFO:tasks.workunit.client.0.smithi100.stdout:op 1693 completed, throughput=5MB/sec 2024-04-07T16:44:04.693 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:44:04 smithi100 bash[74942]: cluster 2024-04-07T16:44:03.246415+0000 mgr.y (mgr.25175) 861 : cluster [DBG] pgmap v1503: 105 pgs: 105 active+clean; 126 GiB data, 5.2 GiB used, 710 GiB / 715 GiB avail; 2.8 MiB/s rd, 2.7 MiB/s wr, 5 op/s 2024-04-07T16:44:04.693 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:44:04 smithi100 bash[76202]: cluster 2024-04-07T16:44:03.246415+0000 mgr.y (mgr.25175) 861 : cluster [DBG] pgmap v1503: 105 pgs: 105 active+clean; 126 GiB data, 5.2 GiB used, 710 GiB / 715 GiB avail; 2.8 MiB/s rd, 2.7 MiB/s wr, 5 op/s 2024-04-07T16:44:04.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:44:04 smithi144 bash[66654]: cluster 2024-04-07T16:44:03.246415+0000 mgr.y (mgr.25175) 861 : cluster [DBG] pgmap v1503: 105 pgs: 105 active+clean; 126 GiB data, 5.2 GiB used, 710 GiB / 715 GiB avail; 2.8 MiB/s rd, 2.7 MiB/s wr, 5 op/s 2024-04-07T16:44:05.048 INFO:tasks.workunit.client.0.smithi100.stdout: 318: throughput=4.99MB/sec pending data=0 2024-04-07T16:44:05.048 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-gJIFjuZOJe32LB4 off=4503062 len=1641818 2024-04-07T16:44:05.052 INFO:tasks.workunit.client.0.smithi100.stdout:op 1694 completed, throughput=4.99MB/sec 2024-04-07T16:44:05.052 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-P-tYc83i7c_6YeX off=0 len=1438566 2024-04-07T16:44:05.069 INFO:tasks.workunit.client.0.smithi100.stdout:op 1695 completed, throughput=5MB/sec 2024-04-07T16:44:05.069 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-DREvr_qjJwaWQaN off=13051383 len=576722 2024-04-07T16:44:05.072 INFO:tasks.workunit.client.0.smithi100.stdout:op 1696 completed, throughput=5MB/sec 2024-04-07T16:44:05.072 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-RFI1u6QLt0bIU08 off=1271224 len=8813 2024-04-07T16:44:05.073 INFO:tasks.workunit.client.0.smithi100.stdout:op 1697 completed, throughput=5MB/sec 2024-04-07T16:44:05.073 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-vwPFpeGuxgb6zt- off=5080881 len=2074632 2024-04-07T16:44:05.088 INFO:tasks.workunit.client.0.smithi100.stdout:op 1698 completed, throughput=5MB/sec 2024-04-07T16:44:06.088 INFO:tasks.workunit.client.0.smithi100.stdout: 319: throughput=4.99MB/sec pending data=0 2024-04-07T16:44:06.088 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-r2nD1Eoixp-xXfW off=2016122 len=1561657 2024-04-07T16:44:06.092 INFO:tasks.workunit.client.0.smithi100.stdout:op 1699 completed, throughput=4.99MB/sec 2024-04-07T16:44:06.092 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-CPQgUtDffF_XPc3 off=23392708 len=1033807 2024-04-07T16:44:06.101 INFO:tasks.workunit.client.0.smithi100.stdout:op 1700 completed, throughput=5MB/sec 2024-04-07T16:44:06.101 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-wTKz_-M24XwTkhf off=5684348 len=421064 2024-04-07T16:44:06.107 INFO:tasks.workunit.client.0.smithi100.stdout:op 1701 completed, throughput=5MB/sec 2024-04-07T16:44:06.107 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-QCNhiGBvmyebU3w off=3735476 len=746556 2024-04-07T16:44:06.108 INFO:tasks.workunit.client.0.smithi100.stdout:op 1702 completed, throughput=5MB/sec 2024-04-07T16:44:06.693 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:44:06 smithi100 bash[74942]: cluster 2024-04-07T16:44:05.247399+0000 mgr.y (mgr.25175) 862 : cluster [DBG] pgmap v1504: 105 pgs: 105 active+clean; 126 GiB data, 5.3 GiB used, 710 GiB / 715 GiB avail; 2.8 MiB/s rd, 2.2 MiB/s wr, 5 op/s 2024-04-07T16:44:06.693 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:44:06 smithi100 bash[76202]: cluster 2024-04-07T16:44:05.247399+0000 mgr.y (mgr.25175) 862 : cluster [DBG] pgmap v1504: 105 pgs: 105 active+clean; 126 GiB data, 5.3 GiB used, 710 GiB / 715 GiB avail; 2.8 MiB/s rd, 2.2 MiB/s wr, 5 op/s 2024-04-07T16:44:06.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:44:06 smithi144 bash[66654]: cluster 2024-04-07T16:44:05.247399+0000 mgr.y (mgr.25175) 862 : cluster [DBG] pgmap v1504: 105 pgs: 105 active+clean; 126 GiB data, 5.3 GiB used, 710 GiB / 715 GiB avail; 2.8 MiB/s rd, 2.2 MiB/s wr, 5 op/s 2024-04-07T16:44:07.108 INFO:tasks.workunit.client.0.smithi100.stdout: 320: throughput=4.99MB/sec pending data=0 2024-04-07T16:44:07.109 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-0cVu2_oyCz7RYnW off=2454129 len=1488658 2024-04-07T16:44:07.112 INFO:tasks.workunit.client.0.smithi100.stdout:op 1703 completed, throughput=4.99MB/sec 2024-04-07T16:44:07.112 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-gGCLWfSBl1bgj9- off=10108096 len=258247 2024-04-07T16:44:07.114 INFO:tasks.workunit.client.0.smithi100.stdout:op 1704 completed, throughput=4.99MB/sec 2024-04-07T16:44:07.114 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-7BKA2wghBEbFVRY off=4534155 len=1560499 2024-04-07T16:44:07.127 INFO:tasks.workunit.client.0.smithi100.stdout:op 1705 completed, throughput=4.99MB/sec 2024-04-07T16:44:07.127 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-PXyPqPvjJe2IjoA off=4496685 len=1915032 2024-04-07T16:44:07.143 INFO:tasks.workunit.client.0.smithi100.stdout:op 1706 completed, throughput=5MB/sec 2024-04-07T16:44:07.143 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-rxq817ZPU6-PAQm off=623254 len=1352638 2024-04-07T16:44:07.153 INFO:tasks.workunit.client.0.smithi100.stdout:op 1707 completed, throughput=5MB/sec 2024-04-07T16:44:08.154 INFO:tasks.workunit.client.0.smithi100.stdout: 321: throughput=4.99MB/sec pending data=0 2024-04-07T16:44:08.154 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-EirQjjOAfCq40Ei off=13549969 len=1015819 2024-04-07T16:44:08.162 INFO:tasks.workunit.client.0.smithi100.stdout:op 1708 completed, throughput=4.99MB/sec 2024-04-07T16:44:08.162 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-1ro4Btgc9w_YgCM off=4490086 len=1964226 2024-04-07T16:44:08.182 INFO:tasks.workunit.client.0.smithi100.stdout:op 1709 completed, throughput=5MB/sec 2024-04-07T16:44:08.182 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-NmA408xt_8_K3wc off=9519483 len=1771715 2024-04-07T16:44:08.186 INFO:tasks.workunit.client.0.smithi100.stdout:op 1710 completed, throughput=5MB/sec 2024-04-07T16:44:08.693 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:44:08 smithi100 bash[74942]: cluster 2024-04-07T16:44:07.248432+0000 mgr.y (mgr.25175) 863 : cluster [DBG] pgmap v1505: 105 pgs: 105 active+clean; 126 GiB data, 5.3 GiB used, 710 GiB / 715 GiB avail; 2.6 MiB/s rd, 2.1 MiB/s wr, 5 op/s 2024-04-07T16:44:08.693 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:44:08 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:44:08] "GET /metrics HTTP/1.1" 200 34055 "" "Prometheus/2.43.0" 2024-04-07T16:44:08.693 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:44:08 smithi100 bash[76202]: cluster 2024-04-07T16:44:07.248432+0000 mgr.y (mgr.25175) 863 : cluster [DBG] pgmap v1505: 105 pgs: 105 active+clean; 126 GiB data, 5.3 GiB used, 710 GiB / 715 GiB avail; 2.6 MiB/s rd, 2.1 MiB/s wr, 5 op/s 2024-04-07T16:44:08.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:44:08 smithi144 bash[66654]: cluster 2024-04-07T16:44:07.248432+0000 mgr.y (mgr.25175) 863 : cluster [DBG] pgmap v1505: 105 pgs: 105 active+clean; 126 GiB data, 5.3 GiB used, 710 GiB / 715 GiB avail; 2.6 MiB/s rd, 2.1 MiB/s wr, 5 op/s 2024-04-07T16:44:09.186 INFO:tasks.workunit.client.0.smithi100.stdout: 322: throughput=4.99MB/sec pending data=0 2024-04-07T16:44:09.186 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-ZZn6DYS7CUbWF46 off=18099856 len=295841 2024-04-07T16:44:09.188 INFO:tasks.workunit.client.0.smithi100.stdout:op 1711 completed, throughput=4.99MB/sec 2024-04-07T16:44:09.188 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-_ve-5KOV8ciBm4j off=7533164 len=1873245 2024-04-07T16:44:09.201 INFO:tasks.workunit.client.0.smithi100.stdout:op 1712 completed, throughput=4.99MB/sec 2024-04-07T16:44:09.201 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-o715vDhdve3GpLr off=807491 len=737486 2024-04-07T16:44:09.208 INFO:tasks.workunit.client.0.smithi100.stdout:op 1713 completed, throughput=4.99MB/sec 2024-04-07T16:44:09.208 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-HDiW-oc8HMYrzpt off=4688308 len=1481805 2024-04-07T16:44:09.217 INFO:tasks.workunit.client.0.smithi100.stdout:op 1714 completed, throughput=5MB/sec 2024-04-07T16:44:09.217 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-y55zNpOVGFipdF9 off=835737 len=1499441 2024-04-07T16:44:09.220 INFO:tasks.workunit.client.0.smithi100.stdout:op 1715 completed, throughput=5MB/sec 2024-04-07T16:44:10.221 INFO:tasks.workunit.client.0.smithi100.stdout: 323: throughput=4.99MB/sec pending data=0 2024-04-07T16:44:10.221 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-N4bIoCkbE-GCdNX off=1323103 len=1922923 2024-04-07T16:44:10.238 INFO:tasks.workunit.client.0.smithi100.stdout:op 1716 completed, throughput=4.99MB/sec 2024-04-07T16:44:10.238 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-fO0Asi1PrXscNHz off=16096840 len=1748008 2024-04-07T16:44:10.243 INFO:tasks.workunit.client.0.smithi100.stdout:op 1717 completed, throughput=5MB/sec 2024-04-07T16:44:10.243 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-GtNx4pLUVervKWv off=5794374 len=1381483 2024-04-07T16:44:10.255 INFO:tasks.workunit.client.0.smithi100.stdout:op 1718 completed, throughput=5MB/sec 2024-04-07T16:44:10.693 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:44:10 smithi100 bash[74942]: cluster 2024-04-07T16:44:09.249308+0000 mgr.y (mgr.25175) 864 : cluster [DBG] pgmap v1506: 105 pgs: 105 active+clean; 126 GiB data, 5.3 GiB used, 710 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.4 MiB/s wr, 5 op/s 2024-04-07T16:44:10.693 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:44:10 smithi100 bash[76202]: cluster 2024-04-07T16:44:09.249308+0000 mgr.y (mgr.25175) 864 : cluster [DBG] pgmap v1506: 105 pgs: 105 active+clean; 126 GiB data, 5.3 GiB used, 710 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.4 MiB/s wr, 5 op/s 2024-04-07T16:44:10.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:44:10 smithi144 bash[66654]: cluster 2024-04-07T16:44:09.249308+0000 mgr.y (mgr.25175) 864 : cluster [DBG] pgmap v1506: 105 pgs: 105 active+clean; 126 GiB data, 5.3 GiB used, 710 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.4 MiB/s wr, 5 op/s 2024-04-07T16:44:11.256 INFO:tasks.workunit.client.0.smithi100.stdout: 324: throughput=4.99MB/sec pending data=0 2024-04-07T16:44:11.256 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-HFyhcELJfRt4wCu off=9706911 len=1769088 2024-04-07T16:44:11.268 INFO:tasks.workunit.client.0.smithi100.stdout:op 1719 completed, throughput=4.99MB/sec 2024-04-07T16:44:11.269 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Td01pLsGUgDIw8w off=9432487 len=1493368 2024-04-07T16:44:11.281 INFO:tasks.workunit.client.0.smithi100.stdout:op 1720 completed, throughput=5MB/sec 2024-04-07T16:44:11.281 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-bBv61feCYMQvG7j off=14398408 len=1819641 2024-04-07T16:44:11.285 INFO:tasks.workunit.client.0.smithi100.stdout:op 1721 completed, throughput=5MB/sec 2024-04-07T16:44:12.286 INFO:tasks.workunit.client.0.smithi100.stdout: 325: throughput=4.99MB/sec pending data=0 2024-04-07T16:44:12.286 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-bmVMeLkidnNyG0B off=16203955 len=1162567 2024-04-07T16:44:12.295 INFO:tasks.workunit.client.0.smithi100.stdout:op 1722 completed, throughput=4.99MB/sec 2024-04-07T16:44:12.295 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-dDIxRB4uNPOzAzv off=8199890 len=1288408 2024-04-07T16:44:12.305 INFO:tasks.workunit.client.0.smithi100.stdout:op 1723 completed, throughput=4.99MB/sec 2024-04-07T16:44:12.305 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-L4J4hxzWhzDZ98t off=782799 len=620681 2024-04-07T16:44:12.312 INFO:tasks.workunit.client.0.smithi100.stdout:op 1724 completed, throughput=4.99MB/sec 2024-04-07T16:44:12.312 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-pfr7G33e4BcXHPC off=2659746 len=1288421 2024-04-07T16:44:12.324 INFO:tasks.workunit.client.0.smithi100.stdout:op 1725 completed, throughput=5MB/sec 2024-04-07T16:44:12.324 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-NTgW8ypVj4NdWiQ off=1264011 len=1775281 2024-04-07T16:44:12.336 INFO:tasks.workunit.client.0.smithi100.stdout:op 1726 completed, throughput=5MB/sec 2024-04-07T16:44:12.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:44:12 smithi144 bash[66654]: cluster 2024-04-07T16:44:11.250155+0000 mgr.y (mgr.25175) 865 : cluster [DBG] pgmap v1507: 105 pgs: 105 active+clean; 126 GiB data, 5.3 GiB used, 710 GiB / 715 GiB avail; 2.4 MiB/s rd, 2.5 MiB/s wr, 5 op/s 2024-04-07T16:44:12.607 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:44:12 smithi144 bash[95233]: ts=2024-04-07T16:44:12.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:44:12.693 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:44:12 smithi100 bash[74942]: cluster 2024-04-07T16:44:11.250155+0000 mgr.y (mgr.25175) 865 : cluster [DBG] pgmap v1507: 105 pgs: 105 active+clean; 126 GiB data, 5.3 GiB used, 710 GiB / 715 GiB avail; 2.4 MiB/s rd, 2.5 MiB/s wr, 5 op/s 2024-04-07T16:44:12.693 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:44:12 smithi100 bash[76202]: cluster 2024-04-07T16:44:11.250155+0000 mgr.y (mgr.25175) 865 : cluster [DBG] pgmap v1507: 105 pgs: 105 active+clean; 126 GiB data, 5.3 GiB used, 710 GiB / 715 GiB avail; 2.4 MiB/s rd, 2.5 MiB/s wr, 5 op/s 2024-04-07T16:44:13.336 INFO:tasks.workunit.client.0.smithi100.stdout: 326: throughput=4.99MB/sec pending data=0 2024-04-07T16:44:13.336 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-p6QJEJzrqIfi9al off=5043383 len=1387220 2024-04-07T16:44:13.340 INFO:tasks.workunit.client.0.smithi100.stdout:op 1727 completed, throughput=4.99MB/sec 2024-04-07T16:44:13.340 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-aDiUmojNFHCXA_B off=673219 len=665353 2024-04-07T16:44:13.348 INFO:tasks.workunit.client.0.smithi100.stdout:op 1728 completed, throughput=4.99MB/sec 2024-04-07T16:44:13.348 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-vzmczqlZurVZUZM off=2718421 len=63862 2024-04-07T16:44:13.352 INFO:tasks.workunit.client.0.smithi100.stdout:op 1729 completed, throughput=4.99MB/sec 2024-04-07T16:44:13.352 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-BNdZy9LPq6epXyk off=8982355 len=254030 2024-04-07T16:44:13.354 INFO:tasks.workunit.client.0.smithi100.stdout:op 1730 completed, throughput=4.99MB/sec 2024-04-07T16:44:13.354 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-lVMd5lDl_OUrVc8 off=11476918 len=2243 2024-04-07T16:44:13.355 INFO:tasks.workunit.client.0.smithi100.stdout:op 1731 completed, throughput=4.99MB/sec 2024-04-07T16:44:13.355 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-vtAXUCb2xmtxvQS off=6956214 len=189492 2024-04-07T16:44:13.356 INFO:tasks.workunit.client.0.smithi100.stdout:op 1732 completed, throughput=4.99MB/sec 2024-04-07T16:44:13.356 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-kmeINR924JKaaTy off=1334120 len=799799 2024-04-07T16:44:13.359 INFO:tasks.workunit.client.0.smithi100.stdout:op 1733 completed, throughput=5MB/sec 2024-04-07T16:44:13.359 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-gC9ponwySMi29Zy off=2024686 len=1141638 2024-04-07T16:44:13.368 INFO:tasks.workunit.client.0.smithi100.stdout:op 1734 completed, throughput=5MB/sec 2024-04-07T16:44:14.369 INFO:tasks.workunit.client.0.smithi100.stdout: 327: throughput=4.99MB/sec pending data=0 2024-04-07T16:44:14.369 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-4Xz4-DQFE_gU-3j off=620203 len=1614301 2024-04-07T16:44:14.382 INFO:tasks.workunit.client.0.smithi100.stdout:op 1735 completed, throughput=4.99MB/sec 2024-04-07T16:44:14.382 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-7vupFi0ZFluCH_O off=2073663 len=455607 2024-04-07T16:44:14.385 INFO:tasks.workunit.client.0.smithi100.stdout:op 1736 completed, throughput=4.99MB/sec 2024-04-07T16:44:14.385 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-lFO8juggfdPSE9p off=10522812 len=2078391 2024-04-07T16:44:14.389 INFO:tasks.workunit.client.0.smithi100.stdout:op 1737 completed, throughput=5MB/sec 2024-04-07T16:44:14.389 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-fKeE1TJxH2lg5Ro off=1441971 len=1641035 2024-04-07T16:44:14.401 INFO:tasks.workunit.client.0.smithi100.stdout:op 1738 completed, throughput=5MB/sec 2024-04-07T16:44:14.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:44:14 smithi144 bash[66654]: cluster 2024-04-07T16:44:13.251306+0000 mgr.y (mgr.25175) 866 : cluster [DBG] pgmap v1508: 105 pgs: 105 active+clean; 126 GiB data, 5.3 GiB used, 710 GiB / 715 GiB avail; 2.1 MiB/s rd, 3.2 MiB/s wr, 4 op/s 2024-04-07T16:44:14.942 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:44:14 smithi100 bash[74942]: cluster 2024-04-07T16:44:13.251306+0000 mgr.y (mgr.25175) 866 : cluster [DBG] pgmap v1508: 105 pgs: 105 active+clean; 126 GiB data, 5.3 GiB used, 710 GiB / 715 GiB avail; 2.1 MiB/s rd, 3.2 MiB/s wr, 4 op/s 2024-04-07T16:44:14.943 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:44:14 smithi100 bash[76202]: cluster 2024-04-07T16:44:13.251306+0000 mgr.y (mgr.25175) 866 : cluster [DBG] pgmap v1508: 105 pgs: 105 active+clean; 126 GiB data, 5.3 GiB used, 710 GiB / 715 GiB avail; 2.1 MiB/s rd, 3.2 MiB/s wr, 4 op/s 2024-04-07T16:44:15.402 INFO:tasks.workunit.client.0.smithi100.stdout: 328: throughput=4.99MB/sec pending data=0 2024-04-07T16:44:15.402 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-AVP6NrQqnQdrt9U off=11161394 len=1121780 2024-04-07T16:44:15.412 INFO:tasks.workunit.client.0.smithi100.stdout:op 1739 completed, throughput=4.99MB/sec 2024-04-07T16:44:15.412 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-e54nHAki-Q_7ly8 off=2107777 len=31110 2024-04-07T16:44:15.413 INFO:tasks.workunit.client.0.smithi100.stdout:op 1740 completed, throughput=4.99MB/sec 2024-04-07T16:44:15.413 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-6DXl5-DU65LMaQx off=2533400 len=1051552 2024-04-07T16:44:15.423 INFO:tasks.workunit.client.0.smithi100.stdout:op 1741 completed, throughput=4.99MB/sec 2024-04-07T16:44:15.423 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-2ejoYwdjmrL46a3 off=6719805 len=1784952 2024-04-07T16:44:15.437 INFO:tasks.workunit.client.0.smithi100.stdout:op 1742 completed, throughput=5MB/sec 2024-04-07T16:44:15.438 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-D5piWUofP-smMST off=1286982 len=422982 2024-04-07T16:44:15.439 INFO:tasks.workunit.client.0.smithi100.stdout:op 1743 completed, throughput=5MB/sec 2024-04-07T16:44:15.440 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-icrmpu6YYmrmUpc off=1611170 len=1161724 2024-04-07T16:44:15.443 INFO:tasks.workunit.client.0.smithi100.stdout:op 1744 completed, throughput=5MB/sec 2024-04-07T16:44:16.443 INFO:tasks.workunit.client.0.smithi100.stdout: 329: throughput=4.99MB/sec pending data=0 2024-04-07T16:44:16.443 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-7IqLtJmrsUGJm7O off=12702485 len=345233 2024-04-07T16:44:16.445 INFO:tasks.workunit.client.0.smithi100.stdout:op 1745 completed, throughput=4.99MB/sec 2024-04-07T16:44:16.445 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-2g1xijWeEU45YCP off=3989472 len=568166 2024-04-07T16:44:16.452 INFO:tasks.workunit.client.0.smithi100.stdout:op 1746 completed, throughput=4.99MB/sec 2024-04-07T16:44:16.452 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-nNON0bFOo6zsQa3 off=1321469 len=1780782 2024-04-07T16:44:16.457 INFO:tasks.workunit.client.0.smithi100.stdout:op 1747 completed, throughput=4.99MB/sec 2024-04-07T16:44:16.457 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj--t__fMGc3NE4038 off=7911900 len=56913 2024-04-07T16:44:16.461 INFO:tasks.workunit.client.0.smithi100.stdout:op 1748 completed, throughput=4.99MB/sec 2024-04-07T16:44:16.461 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-xPi7F_1Q1mMDESJ off=23063083 len=1165327 2024-04-07T16:44:16.463 INFO:tasks.workunit.client.0.smithi100.stdout:op 1749 completed, throughput=5MB/sec 2024-04-07T16:44:16.463 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-lkbSnX-f0fUasvh off=8022419 len=1594382 2024-04-07T16:44:16.475 INFO:tasks.workunit.client.0.smithi100.stdout:op 1750 completed, throughput=5MB/sec 2024-04-07T16:44:16.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:44:16 smithi144 bash[66654]: cluster 2024-04-07T16:44:15.252246+0000 mgr.y (mgr.25175) 867 : cluster [DBG] pgmap v1509: 105 pgs: 105 active+clean; 126 GiB data, 5.4 GiB used, 710 GiB / 715 GiB avail; 1.7 MiB/s rd, 3.1 MiB/s wr, 4 op/s 2024-04-07T16:44:16.943 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:44:16 smithi100 bash[74942]: cluster 2024-04-07T16:44:15.252246+0000 mgr.y (mgr.25175) 867 : cluster [DBG] pgmap v1509: 105 pgs: 105 active+clean; 126 GiB data, 5.4 GiB used, 710 GiB / 715 GiB avail; 1.7 MiB/s rd, 3.1 MiB/s wr, 4 op/s 2024-04-07T16:44:16.943 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:44:16 smithi100 bash[76202]: cluster 2024-04-07T16:44:15.252246+0000 mgr.y (mgr.25175) 867 : cluster [DBG] pgmap v1509: 105 pgs: 105 active+clean; 126 GiB data, 5.4 GiB used, 710 GiB / 715 GiB avail; 1.7 MiB/s rd, 3.1 MiB/s wr, 4 op/s 2024-04-07T16:44:17.475 INFO:tasks.workunit.client.0.smithi100.stdout: 330: throughput=4.99MB/sec pending data=0 2024-04-07T16:44:17.476 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Vxl6u2kTAcHBMwC off=3514565 len=952245 2024-04-07T16:44:17.478 INFO:tasks.workunit.client.0.smithi100.stdout:op 1751 completed, throughput=4.99MB/sec 2024-04-07T16:44:17.478 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-IBYF1Xb5EfshKcV off=975663 len=1618663 2024-04-07T16:44:17.482 INFO:tasks.workunit.client.0.smithi100.stdout:op 1752 completed, throughput=4.99MB/sec 2024-04-07T16:44:17.483 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-36vj164feVN_i-y off=16753256 len=1522186 2024-04-07T16:44:17.495 INFO:tasks.workunit.client.0.smithi100.stdout:op 1753 completed, throughput=5MB/sec 2024-04-07T16:44:17.495 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-FBRZvoJ1uDfziVk off=10490659 len=6628 2024-04-07T16:44:17.499 INFO:tasks.workunit.client.0.smithi100.stdout:op 1754 completed, throughput=5MB/sec 2024-04-07T16:44:17.499 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-po3L6a9eHdDsB1k off=4154697 len=2041672 2024-04-07T16:44:17.515 INFO:tasks.workunit.client.0.smithi100.stdout:op 1755 completed, throughput=5MB/sec 2024-04-07T16:44:17.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:44:17 smithi144 bash[66654]: audit 2024-04-07T16:44:16.764498+0000 mon.a (mon.0) 2092 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:44:17.943 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:44:17 smithi100 bash[74942]: audit 2024-04-07T16:44:16.764498+0000 mon.a (mon.0) 2092 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:44:17.943 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:44:17 smithi100 bash[76202]: audit 2024-04-07T16:44:16.764498+0000 mon.a (mon.0) 2092 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:44:18.516 INFO:tasks.workunit.client.0.smithi100.stdout: 331: throughput=4.99MB/sec pending data=0 2024-04-07T16:44:18.516 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-ph8wYEtxDsLYFtm off=15088368 len=157458 2024-04-07T16:44:18.522 INFO:tasks.workunit.client.0.smithi100.stdout:op 1756 completed, throughput=4.99MB/sec 2024-04-07T16:44:18.522 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Kvi0c6mz6FaS4C3 off=2790145 len=1397077 2024-04-07T16:44:18.536 INFO:tasks.workunit.client.0.smithi100.stdout:op 1757 completed, throughput=4.99MB/sec 2024-04-07T16:44:18.536 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-3sVNklbLqpkwwn2 off=16422131 len=587851 2024-04-07T16:44:18.543 INFO:tasks.workunit.client.0.smithi100.stdout:op 1758 completed, throughput=4.99MB/sec 2024-04-07T16:44:18.543 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-TAus-P8ktCOC4sL off=14210004 len=138131 2024-04-07T16:44:18.547 INFO:tasks.workunit.client.0.smithi100.stdout:op 1759 completed, throughput=4.99MB/sec 2024-04-07T16:44:18.547 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-qr-0qbQc_IbuKSg off=9578386 len=1223663 2024-04-07T16:44:18.551 INFO:tasks.workunit.client.0.smithi100.stdout:op 1760 completed, throughput=5MB/sec 2024-04-07T16:44:18.551 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-r_6cy1o-YAPDoTK off=3652872 len=544130 2024-04-07T16:44:18.554 INFO:tasks.workunit.client.0.smithi100.stdout:op 1761 completed, throughput=5MB/sec 2024-04-07T16:44:18.554 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-8Y0SQ_DEba8A12T off=9152161 len=1079458 2024-04-07T16:44:18.555 INFO:tasks.workunit.client.0.smithi100.stdout:op 1762 completed, throughput=5MB/sec 2024-04-07T16:44:18.693 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:44:18 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:44:18] "GET /metrics HTTP/1.1" 200 34055 "" "Prometheus/2.43.0" 2024-04-07T16:44:18.693 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:44:18 smithi100 bash[74942]: cluster 2024-04-07T16:44:17.253263+0000 mgr.y (mgr.25175) 868 : cluster [DBG] pgmap v1510: 105 pgs: 105 active+clean; 126 GiB data, 5.4 GiB used, 710 GiB / 715 GiB avail; 1.5 MiB/s rd, 3.3 MiB/s wr, 4 op/s 2024-04-07T16:44:18.693 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:44:18 smithi100 bash[76202]: cluster 2024-04-07T16:44:17.253263+0000 mgr.y (mgr.25175) 868 : cluster [DBG] pgmap v1510: 105 pgs: 105 active+clean; 126 GiB data, 5.4 GiB used, 710 GiB / 715 GiB avail; 1.5 MiB/s rd, 3.3 MiB/s wr, 4 op/s 2024-04-07T16:44:18.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:44:18 smithi144 bash[66654]: cluster 2024-04-07T16:44:17.253263+0000 mgr.y (mgr.25175) 868 : cluster [DBG] pgmap v1510: 105 pgs: 105 active+clean; 126 GiB data, 5.4 GiB used, 710 GiB / 715 GiB avail; 1.5 MiB/s rd, 3.3 MiB/s wr, 4 op/s 2024-04-07T16:44:19.556 INFO:tasks.workunit.client.0.smithi100.stdout: 332: throughput=4.99MB/sec pending data=0 2024-04-07T16:44:19.556 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-dNyUFB1nibuTkLr off=6349964 len=750728 2024-04-07T16:44:19.564 INFO:tasks.workunit.client.0.smithi100.stdout:op 1763 completed, throughput=4.99MB/sec 2024-04-07T16:44:19.564 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-uHMF_m5djwxlXNM off=3790360 len=725061 2024-04-07T16:44:19.572 INFO:tasks.workunit.client.0.smithi100.stdout:op 1764 completed, throughput=4.99MB/sec 2024-04-07T16:44:19.572 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-2W2DqFb573bMdTB off=8849366 len=1815497 2024-04-07T16:44:19.587 INFO:tasks.workunit.client.0.smithi100.stdout:op 1765 completed, throughput=5MB/sec 2024-04-07T16:44:19.587 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-pF79Mpn8o3L0Zio off=3196642 len=29761 2024-04-07T16:44:19.591 INFO:tasks.workunit.client.0.smithi100.stdout:op 1766 completed, throughput=5MB/sec 2024-04-07T16:44:19.592 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-wGR7xkEXTXrka2M off=10638030 len=943039 2024-04-07T16:44:19.600 INFO:tasks.workunit.client.0.smithi100.stdout:op 1767 completed, throughput=5MB/sec 2024-04-07T16:44:19.600 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-VLOb0weHjsPYZ_f off=2909077 len=1604967 2024-04-07T16:44:19.611 INFO:tasks.workunit.client.0.smithi100.stdout:op 1768 completed, throughput=5MB/sec 2024-04-07T16:44:20.612 INFO:tasks.workunit.client.0.smithi100.stdout: 333: throughput=4.99MB/sec pending data=0 2024-04-07T16:44:20.612 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-TrfI7YR4j_S4Kca off=4012971 len=1767537 2024-04-07T16:44:20.627 INFO:tasks.workunit.client.0.smithi100.stdout:op 1769 completed, throughput=4.99MB/sec 2024-04-07T16:44:20.627 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-t-I_Z6hd2iAz_wZ off=1681907 len=1711012 2024-04-07T16:44:20.640 INFO:tasks.workunit.client.0.smithi100.stdout:op 1770 completed, throughput=5MB/sec 2024-04-07T16:44:20.640 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-iZ6yYEO_lf91roU off=3273559 len=613301 2024-04-07T16:44:20.647 INFO:tasks.workunit.client.0.smithi100.stdout:op 1771 completed, throughput=5MB/sec 2024-04-07T16:44:20.648 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-h2lEHrWzqQB4QEk off=21079315 len=805329 2024-04-07T16:44:20.655 INFO:tasks.workunit.client.0.smithi100.stdout:op 1772 completed, throughput=5MB/sec 2024-04-07T16:44:20.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:44:20 smithi144 bash[66654]: cluster 2024-04-07T16:44:19.254090+0000 mgr.y (mgr.25175) 869 : cluster [DBG] pgmap v1511: 105 pgs: 105 active+clean; 126 GiB data, 5.4 GiB used, 710 GiB / 715 GiB avail; 1.9 MiB/s rd, 2.9 MiB/s wr, 4 op/s 2024-04-07T16:44:20.942 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:44:20 smithi100 bash[74942]: cluster 2024-04-07T16:44:19.254090+0000 mgr.y (mgr.25175) 869 : cluster [DBG] pgmap v1511: 105 pgs: 105 active+clean; 126 GiB data, 5.4 GiB used, 710 GiB / 715 GiB avail; 1.9 MiB/s rd, 2.9 MiB/s wr, 4 op/s 2024-04-07T16:44:20.943 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:44:20 smithi100 bash[76202]: cluster 2024-04-07T16:44:19.254090+0000 mgr.y (mgr.25175) 869 : cluster [DBG] pgmap v1511: 105 pgs: 105 active+clean; 126 GiB data, 5.4 GiB used, 710 GiB / 715 GiB avail; 1.9 MiB/s rd, 2.9 MiB/s wr, 4 op/s 2024-04-07T16:44:21.656 INFO:tasks.workunit.client.0.smithi100.stdout: 334: throughput=4.99MB/sec pending data=0 2024-04-07T16:44:21.656 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-6qOxJTk2eBmV8Yi off=5166828 len=1210171 2024-04-07T16:44:21.659 INFO:tasks.workunit.client.0.smithi100.stdout:op 1773 completed, throughput=4.99MB/sec 2024-04-07T16:44:21.659 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-RWQmqe_1XRaAdnh off=9291681 len=1655021 2024-04-07T16:44:21.672 INFO:tasks.workunit.client.0.smithi100.stdout:op 1774 completed, throughput=5MB/sec 2024-04-07T16:44:21.672 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-XmlhCbQJ-yfuGmm off=1466278 len=1989502 2024-04-07T16:44:21.679 INFO:tasks.workunit.client.0.smithi100.stdout:op 1775 completed, throughput=5MB/sec 2024-04-07T16:44:22.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:44:22 smithi144 bash[66654]: cluster 2024-04-07T16:44:21.255021+0000 mgr.y (mgr.25175) 870 : cluster [DBG] pgmap v1512: 105 pgs: 105 active+clean; 126 GiB data, 5.4 GiB used, 710 GiB / 715 GiB avail; 1.9 MiB/s rd, 3.0 MiB/s wr, 4 op/s 2024-04-07T16:44:22.608 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:44:22 smithi144 bash[95233]: ts=2024-04-07T16:44:22.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:44:22.679 INFO:tasks.workunit.client.0.smithi100.stdout: 335: throughput=4.99MB/sec pending data=0 2024-04-07T16:44:22.679 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj--hj1U3dSG0V2UN7 off=11751631 len=2014021 2024-04-07T16:44:22.684 INFO:tasks.workunit.client.0.smithi100.stdout:op 1776 completed, throughput=4.99MB/sec 2024-04-07T16:44:22.684 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-wKpFg4Yc6pwUhq9 off=2255659 len=215890 2024-04-07T16:44:22.690 INFO:tasks.workunit.client.0.smithi100.stdout:op 1777 completed, throughput=4.99MB/sec 2024-04-07T16:44:22.690 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-jXWjONxOGGYE9tq off=5540056 len=1684455 2024-04-07T16:44:22.705 INFO:tasks.workunit.client.0.smithi100.stdout:op 1778 completed, throughput=5MB/sec 2024-04-07T16:44:22.706 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-OI8OjvTFpQIwMO2 off=4321525 len=1538690 2024-04-07T16:44:22.709 INFO:tasks.workunit.client.0.smithi100.stdout:op 1779 completed, throughput=5MB/sec 2024-04-07T16:44:22.943 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:44:22 smithi100 bash[74942]: cluster 2024-04-07T16:44:21.255021+0000 mgr.y (mgr.25175) 870 : cluster [DBG] pgmap v1512: 105 pgs: 105 active+clean; 126 GiB data, 5.4 GiB used, 710 GiB / 715 GiB avail; 1.9 MiB/s rd, 3.0 MiB/s wr, 4 op/s 2024-04-07T16:44:22.943 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:44:22 smithi100 bash[76202]: cluster 2024-04-07T16:44:21.255021+0000 mgr.y (mgr.25175) 870 : cluster [DBG] pgmap v1512: 105 pgs: 105 active+clean; 126 GiB data, 5.4 GiB used, 710 GiB / 715 GiB avail; 1.9 MiB/s rd, 3.0 MiB/s wr, 4 op/s 2024-04-07T16:44:23.710 INFO:tasks.workunit.client.0.smithi100.stdout: 336: throughput=4.99MB/sec pending data=0 2024-04-07T16:44:23.710 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-GePpjs1jtXw-ESC off=865518 len=2015973 2024-04-07T16:44:23.715 INFO:tasks.workunit.client.0.smithi100.stdout:op 1780 completed, throughput=4.99MB/sec 2024-04-07T16:44:23.715 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Jn_v_stgoazXv_w off=799950 len=1246843 2024-04-07T16:44:23.727 INFO:tasks.workunit.client.0.smithi100.stdout:op 1781 completed, throughput=5MB/sec 2024-04-07T16:44:23.727 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-vvdTW9_t9HZMi95 off=6851636 len=837125 2024-04-07T16:44:23.730 INFO:tasks.workunit.client.0.smithi100.stdout:op 1782 completed, throughput=5MB/sec 2024-04-07T16:44:23.730 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-xsHcSrtO4Eq56hq off=5090663 len=1097238 2024-04-07T16:44:23.740 INFO:tasks.workunit.client.0.smithi100.stdout:op 1783 completed, throughput=5MB/sec 2024-04-07T16:44:24.740 INFO:tasks.workunit.client.0.smithi100.stdout: 337: throughput=4.99MB/sec pending data=0 2024-04-07T16:44:24.803 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-BzsfA4yTirhUrzc off=19501042 len=1662554 2024-04-07T16:44:24.804 INFO:tasks.workunit.client.0.smithi100.stdout:op 1784 completed, throughput=4.99MB/sec 2024-04-07T16:44:24.804 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-gTZLOrRmCCVLF7H off=3194448 len=1404890 2024-04-07T16:44:24.804 INFO:tasks.workunit.client.0.smithi100.stdout:op 1785 completed, throughput=4.99MB/sec 2024-04-07T16:44:24.804 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-qGIO4Taf6I1drkH off=73673 len=1921967 2024-04-07T16:44:24.804 INFO:tasks.workunit.client.0.smithi100.stdout:op 1786 completed, throughput=5MB/sec 2024-04-07T16:44:24.804 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-icrmpu6YYmrmUpc off=17509736 len=949456 2024-04-07T16:44:24.804 INFO:tasks.workunit.client.0.smithi100.stdout:op 1787 completed, throughput=5MB/sec 2024-04-07T16:44:24.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:44:24 smithi144 bash[66654]: cluster 2024-04-07T16:44:23.256148+0000 mgr.y (mgr.25175) 871 : cluster [DBG] pgmap v1513: 105 pgs: 105 active+clean; 126 GiB data, 5.4 GiB used, 710 GiB / 715 GiB avail; 1.9 MiB/s rd, 3.5 MiB/s wr, 5 op/s 2024-04-07T16:44:24.943 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:44:24 smithi100 bash[74942]: cluster 2024-04-07T16:44:23.256148+0000 mgr.y (mgr.25175) 871 : cluster [DBG] pgmap v1513: 105 pgs: 105 active+clean; 126 GiB data, 5.4 GiB used, 710 GiB / 715 GiB avail; 1.9 MiB/s rd, 3.5 MiB/s wr, 5 op/s 2024-04-07T16:44:24.943 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:44:24 smithi100 bash[76202]: cluster 2024-04-07T16:44:23.256148+0000 mgr.y (mgr.25175) 871 : cluster [DBG] pgmap v1513: 105 pgs: 105 active+clean; 126 GiB data, 5.4 GiB used, 710 GiB / 715 GiB avail; 1.9 MiB/s rd, 3.5 MiB/s wr, 5 op/s 2024-04-07T16:44:25.777 INFO:tasks.workunit.client.0.smithi100.stdout: 338: throughput=4.99MB/sec pending data=0 2024-04-07T16:44:25.777 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-raeWbOyRsqb6Drz off=8828443 len=1295437 2024-04-07T16:44:25.788 INFO:tasks.workunit.client.0.smithi100.stdout:op 1788 completed, throughput=4.99MB/sec 2024-04-07T16:44:25.788 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-TF-eSbqu2T7aS4T off=2652471 len=400226 2024-04-07T16:44:25.794 INFO:tasks.workunit.client.0.smithi100.stdout:op 1789 completed, throughput=4.99MB/sec 2024-04-07T16:44:25.794 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-ZT-FB_jm-2wxpaX off=3476576 len=82110 2024-04-07T16:44:25.798 INFO:tasks.workunit.client.0.smithi100.stdout:op 1790 completed, throughput=4.99MB/sec 2024-04-07T16:44:25.798 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-zpJoIqb3Jpyj8FI off=6357781 len=591191 2024-04-07T16:44:25.804 INFO:tasks.workunit.client.0.smithi100.stdout:op 1791 completed, throughput=4.99MB/sec 2024-04-07T16:44:25.805 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-c0JpUxM5pxD6N8p off=17433785 len=1623509 2024-04-07T16:44:25.817 INFO:tasks.workunit.client.0.smithi100.stdout:op 1792 completed, throughput=5MB/sec 2024-04-07T16:44:25.817 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-UELlwjf9ntnKN-t off=4621881 len=3701 2024-04-07T16:44:25.821 INFO:tasks.workunit.client.0.smithi100.stdout:op 1793 completed, throughput=5MB/sec 2024-04-07T16:44:25.821 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-PpZOZ7j8n_-o_Bf off=9485891 len=1779561 2024-04-07T16:44:25.823 INFO:tasks.workunit.client.0.smithi100.stdout:op 1794 completed, throughput=5MB/sec 2024-04-07T16:44:26.824 INFO:tasks.workunit.client.0.smithi100.stdout: 339: throughput=4.99MB/sec pending data=0 2024-04-07T16:44:26.824 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-TGevlwn6Bt1aDQB off=805886 len=127742 2024-04-07T16:44:26.826 INFO:tasks.workunit.client.0.smithi100.stdout:op 1795 completed, throughput=4.99MB/sec 2024-04-07T16:44:26.826 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-whUGmqTllmfa4KN off=5044985 len=1433407 2024-04-07T16:44:26.839 INFO:tasks.workunit.client.0.smithi100.stdout:op 1796 completed, throughput=4.99MB/sec 2024-04-07T16:44:26.839 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-TQv4QNpWQvNAgDB off=3146868 len=313993 2024-04-07T16:44:26.845 INFO:tasks.workunit.client.0.smithi100.stdout:op 1797 completed, throughput=4.99MB/sec 2024-04-07T16:44:26.845 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-z0HsKk_Bv1pVsPo off=1650527 len=840886 2024-04-07T16:44:26.847 INFO:tasks.workunit.client.0.smithi100.stdout:op 1798 completed, throughput=5MB/sec 2024-04-07T16:44:26.847 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-qlVEGatig1evdo2 off=16680328 len=307229 2024-04-07T16:44:26.849 INFO:tasks.workunit.client.0.smithi100.stdout:op 1799 completed, throughput=5MB/sec 2024-04-07T16:44:26.849 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-qdidzmysWWqzvCc off=3287408 len=790024 2024-04-07T16:44:26.852 INFO:tasks.workunit.client.0.smithi100.stdout:op 1800 completed, throughput=5MB/sec 2024-04-07T16:44:26.852 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-R0KseleFA21CHkw off=1828892 len=766331 2024-04-07T16:44:26.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:44:26 smithi144 bash[66654]: cluster 2024-04-07T16:44:25.257088+0000 mgr.y (mgr.25175) 872 : cluster [DBG] pgmap v1514: 105 pgs: 105 active+clean; 126 GiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 1.7 MiB/s rd, 2.9 MiB/s wr, 4 op/s 2024-04-07T16:44:26.860 INFO:tasks.workunit.client.0.smithi100.stdout:op 1801 completed, throughput=5MB/sec 2024-04-07T16:44:26.942 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:44:26 smithi100 bash[74942]: cluster 2024-04-07T16:44:25.257088+0000 mgr.y (mgr.25175) 872 : cluster [DBG] pgmap v1514: 105 pgs: 105 active+clean; 126 GiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 1.7 MiB/s rd, 2.9 MiB/s wr, 4 op/s 2024-04-07T16:44:26.943 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:44:26 smithi100 bash[76202]: cluster 2024-04-07T16:44:25.257088+0000 mgr.y (mgr.25175) 872 : cluster [DBG] pgmap v1514: 105 pgs: 105 active+clean; 126 GiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 1.7 MiB/s rd, 2.9 MiB/s wr, 4 op/s 2024-04-07T16:44:27.860 INFO:tasks.workunit.client.0.smithi100.stdout: 340: throughput=4.99MB/sec pending data=0 2024-04-07T16:44:27.860 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-U51dI5tgc7hLolr off=4621069 len=1987798 2024-04-07T16:44:27.875 INFO:tasks.workunit.client.0.smithi100.stdout:op 1802 completed, throughput=4.99MB/sec 2024-04-07T16:44:27.875 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-02a_VRtVQo9lc0G off=9608174 len=248459 2024-04-07T16:44:27.880 INFO:tasks.workunit.client.0.smithi100.stdout:op 1803 completed, throughput=4.99MB/sec 2024-04-07T16:44:27.881 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-GRr2pS5ON3vdTyx off=14315441 len=1316683 2024-04-07T16:44:27.884 INFO:tasks.workunit.client.0.smithi100.stdout:op 1804 completed, throughput=5MB/sec 2024-04-07T16:44:27.884 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-sxSPO58EddMX4NH off=2439722 len=402508 2024-04-07T16:44:27.886 INFO:tasks.workunit.client.0.smithi100.stdout:op 1805 completed, throughput=5MB/sec 2024-04-07T16:44:27.886 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-cAm3whg0GCTo8eU off=15646340 len=357141 2024-04-07T16:44:27.888 INFO:tasks.workunit.client.0.smithi100.stdout:op 1806 completed, throughput=5MB/sec 2024-04-07T16:44:27.888 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-ZhCOtiMgQTeACXk off=5533857 len=53897 2024-04-07T16:44:27.889 INFO:tasks.workunit.client.0.smithi100.stdout:op 1807 completed, throughput=5MB/sec 2024-04-07T16:44:27.889 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-nBEJ1Y8DwiBhYZh off=62572 len=1424497 2024-04-07T16:44:27.892 INFO:tasks.workunit.client.0.smithi100.stdout:op 1808 completed, throughput=5MB/sec 2024-04-07T16:44:28.693 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:44:28 smithi100 bash[74942]: cluster 2024-04-07T16:44:27.258087+0000 mgr.y (mgr.25175) 873 : cluster [DBG] pgmap v1515: 105 pgs: 105 active+clean; 126 GiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 2.1 MiB/s rd, 3.2 MiB/s wr, 5 op/s 2024-04-07T16:44:28.693 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:44:28 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:44:28] "GET /metrics HTTP/1.1" 200 34055 "" "Prometheus/2.43.0" 2024-04-07T16:44:28.693 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:44:28 smithi100 bash[76202]: cluster 2024-04-07T16:44:27.258087+0000 mgr.y (mgr.25175) 873 : cluster [DBG] pgmap v1515: 105 pgs: 105 active+clean; 126 GiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 2.1 MiB/s rd, 3.2 MiB/s wr, 5 op/s 2024-04-07T16:44:28.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:44:28 smithi144 bash[66654]: cluster 2024-04-07T16:44:27.258087+0000 mgr.y (mgr.25175) 873 : cluster [DBG] pgmap v1515: 105 pgs: 105 active+clean; 126 GiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 2.1 MiB/s rd, 3.2 MiB/s wr, 5 op/s 2024-04-07T16:44:28.893 INFO:tasks.workunit.client.0.smithi100.stdout: 341: throughput=4.99MB/sec pending data=0 2024-04-07T16:44:28.893 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-4OHOrsyJrIIw1Wu off=1528766 len=1815146 2024-04-07T16:44:28.897 INFO:tasks.workunit.client.0.smithi100.stdout:op 1809 completed, throughput=4.99MB/sec 2024-04-07T16:44:28.897 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-KhKNVjYVmDljK6Y off=3607664 len=1562692 2024-04-07T16:44:28.908 INFO:tasks.workunit.client.0.smithi100.stdout:op 1810 completed, throughput=5MB/sec 2024-04-07T16:44:28.908 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-sCWqw08NdlmLDUm off=8760458 len=870549 2024-04-07T16:44:28.916 INFO:tasks.workunit.client.0.smithi100.stdout:op 1811 completed, throughput=5MB/sec 2024-04-07T16:44:28.916 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-yqWnC9Jk0gdZjep off=18441677 len=94079 2024-04-07T16:44:28.918 INFO:tasks.workunit.client.0.smithi100.stdout:op 1812 completed, throughput=5MB/sec 2024-04-07T16:44:28.918 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-8fujlSwUGf22JYk off=12600657 len=1448813 2024-04-07T16:44:28.927 INFO:tasks.workunit.client.0.smithi100.stdout:op 1813 completed, throughput=5MB/sec 2024-04-07T16:44:29.928 INFO:tasks.workunit.client.0.smithi100.stdout: 342: throughput=4.99MB/sec pending data=0 2024-04-07T16:44:29.928 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj--Cwj-KawAoTknLD off=130165 len=1980136 2024-04-07T16:44:29.932 INFO:tasks.workunit.client.0.smithi100.stdout:op 1814 completed, throughput=4.99MB/sec 2024-04-07T16:44:29.932 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-sK6lZPjs2N2ZaOi off=500347 len=1494891 2024-04-07T16:44:29.938 INFO:tasks.workunit.client.0.smithi100.stdout:op 1815 completed, throughput=5MB/sec 2024-04-07T16:44:29.938 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-RlEL000ZZocFcfO off=3084072 len=698431 2024-04-07T16:44:29.945 INFO:tasks.workunit.client.0.smithi100.stdout:op 1816 completed, throughput=5MB/sec 2024-04-07T16:44:29.945 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-AO8_EK3atVJTthr off=15949993 len=1495121 2024-04-07T16:44:29.950 INFO:tasks.workunit.client.0.smithi100.stdout:op 1817 completed, throughput=5MB/sec 2024-04-07T16:44:30.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:44:30 smithi144 bash[66654]: cluster 2024-04-07T16:44:29.258912+0000 mgr.y (mgr.25175) 874 : cluster [DBG] pgmap v1516: 105 pgs: 105 active+clean; 126 GiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 2.1 MiB/s rd, 2.7 MiB/s wr, 4 op/s 2024-04-07T16:44:30.943 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:44:30 smithi100 bash[74942]: cluster 2024-04-07T16:44:29.258912+0000 mgr.y (mgr.25175) 874 : cluster [DBG] pgmap v1516: 105 pgs: 105 active+clean; 126 GiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 2.1 MiB/s rd, 2.7 MiB/s wr, 4 op/s 2024-04-07T16:44:30.943 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:44:30 smithi100 bash[76202]: cluster 2024-04-07T16:44:29.258912+0000 mgr.y (mgr.25175) 874 : cluster [DBG] pgmap v1516: 105 pgs: 105 active+clean; 126 GiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 2.1 MiB/s rd, 2.7 MiB/s wr, 4 op/s 2024-04-07T16:44:30.951 INFO:tasks.workunit.client.0.smithi100.stdout: 343: throughput=4.99MB/sec pending data=0 2024-04-07T16:44:30.951 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-YrwZ8Isk4LStWVn off=133407 len=652749 2024-04-07T16:44:30.958 INFO:tasks.workunit.client.0.smithi100.stdout:op 1818 completed, throughput=4.99MB/sec 2024-04-07T16:44:30.958 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-F-fKIhCxJkQLRW7 off=2245708 len=511640 2024-04-07T16:44:30.959 INFO:tasks.workunit.client.0.smithi100.stdout:op 1819 completed, throughput=4.99MB/sec 2024-04-07T16:44:30.959 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-qdCha99HMEpMA8D off=5857122 len=402925 2024-04-07T16:44:30.961 INFO:tasks.workunit.client.0.smithi100.stdout:op 1820 completed, throughput=4.99MB/sec 2024-04-07T16:44:30.961 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-CbzE0gH79srrL63 off=17238102 len=1829131 2024-04-07T16:44:30.976 INFO:tasks.workunit.client.0.smithi100.stdout:op 1821 completed, throughput=5MB/sec 2024-04-07T16:44:30.976 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-CMJj5b64_2IVawK off=2128000 len=1966048 2024-04-07T16:44:30.981 INFO:tasks.workunit.client.0.smithi100.stdout:op 1822 completed, throughput=5MB/sec 2024-04-07T16:44:31.981 INFO:tasks.workunit.client.0.smithi100.stdout: 344: throughput=4.99MB/sec pending data=0 2024-04-07T16:44:31.981 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-V38Cn4UnWPPHwTE off=6493053 len=614000 2024-04-07T16:44:31.988 INFO:tasks.workunit.client.0.smithi100.stdout:op 1823 completed, throughput=4.99MB/sec 2024-04-07T16:44:31.988 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-umpHh3_wz3sYMsG off=3325022 len=2029698 2024-04-07T16:44:31.993 INFO:tasks.workunit.client.0.smithi100.stdout:op 1824 completed, throughput=5MB/sec 2024-04-07T16:44:31.993 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Vx_iDLdwbjorHn3 off=7667685 len=149614 2024-04-07T16:44:31.994 INFO:tasks.workunit.client.0.smithi100.stdout:op 1825 completed, throughput=5MB/sec 2024-04-07T16:44:31.994 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-6rtRXasBtMaVvZN off=15579021 len=364865 2024-04-07T16:44:31.999 INFO:tasks.workunit.client.0.smithi100.stdout:op 1826 completed, throughput=5MB/sec 2024-04-07T16:44:32.000 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-2Kqq9SuCzCIlf48 off=3175242 len=84409 2024-04-07T16:44:32.001 INFO:tasks.workunit.client.0.smithi100.stdout:op 1827 completed, throughput=5MB/sec 2024-04-07T16:44:32.001 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-4XvNaDWNcD4pj3b off=11914945 len=1453153 2024-04-07T16:44:32.014 INFO:tasks.workunit.client.0.smithi100.stdout:op 1828 completed, throughput=5MB/sec 2024-04-07T16:44:32.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:44:32 smithi144 bash[66654]: cluster 2024-04-07T16:44:31.260007+0000 mgr.y (mgr.25175) 875 : cluster [DBG] pgmap v1517: 105 pgs: 105 active+clean; 126 GiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 2.0 MiB/s rd, 2.9 MiB/s wr, 5 op/s 2024-04-07T16:44:32.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:44:32 smithi144 bash[66654]: audit 2024-04-07T16:44:31.764959+0000 mon.a (mon.0) 2093 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:44:32.608 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:44:32 smithi144 bash[95233]: ts=2024-04-07T16:44:32.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:44:32.943 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:44:32 smithi100 bash[74942]: cluster 2024-04-07T16:44:31.260007+0000 mgr.y (mgr.25175) 875 : cluster [DBG] pgmap v1517: 105 pgs: 105 active+clean; 126 GiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 2.0 MiB/s rd, 2.9 MiB/s wr, 5 op/s 2024-04-07T16:44:32.943 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:44:32 smithi100 bash[74942]: audit 2024-04-07T16:44:31.764959+0000 mon.a (mon.0) 2093 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:44:32.943 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:44:32 smithi100 bash[76202]: cluster 2024-04-07T16:44:31.260007+0000 mgr.y (mgr.25175) 875 : cluster [DBG] pgmap v1517: 105 pgs: 105 active+clean; 126 GiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 2.0 MiB/s rd, 2.9 MiB/s wr, 5 op/s 2024-04-07T16:44:32.943 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:44:32 smithi100 bash[76202]: audit 2024-04-07T16:44:31.764959+0000 mon.a (mon.0) 2093 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:44:33.014 INFO:tasks.workunit.client.0.smithi100.stdout: 345: throughput=4.99MB/sec pending data=0 2024-04-07T16:44:33.015 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-DJOeDC7mMPWrAAK off=119842 len=1272268 2024-04-07T16:44:33.018 INFO:tasks.workunit.client.0.smithi100.stdout:op 1829 completed, throughput=4.99MB/sec 2024-04-07T16:44:33.018 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-bnL6i48A9oCqZ7J off=7523965 len=469741 2024-04-07T16:44:33.024 INFO:tasks.workunit.client.0.smithi100.stdout:op 1830 completed, throughput=4.99MB/sec 2024-04-07T16:44:33.024 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-333sPDf_Fb35cIA off=2237026 len=1161398 2024-04-07T16:44:33.028 INFO:tasks.workunit.client.0.smithi100.stdout:op 1831 completed, throughput=4.99MB/sec 2024-04-07T16:44:33.028 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-gYtbnzoIuzz8n64 off=532750 len=1212256 2024-04-07T16:44:33.038 INFO:tasks.workunit.client.0.smithi100.stdout:op 1832 completed, throughput=5MB/sec 2024-04-07T16:44:33.038 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-lcI39xuQd8HzYeU off=9576044 len=1971627 2024-04-07T16:44:33.042 INFO:tasks.workunit.client.0.smithi100.stdout:op 1833 completed, throughput=5MB/sec 2024-04-07T16:44:34.043 INFO:tasks.workunit.client.0.smithi100.stdout: 346: throughput=4.99MB/sec pending data=0 2024-04-07T16:44:34.043 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-B4AJfhQuziaEwCc off=53995 len=751193 2024-04-07T16:44:34.051 INFO:tasks.workunit.client.0.smithi100.stdout:op 1834 completed, throughput=4.99MB/sec 2024-04-07T16:44:34.051 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Bui_b8We9bDBR9q off=0 len=1739936 2024-04-07T16:44:34.064 INFO:tasks.workunit.client.0.smithi100.stdout:op 1835 completed, throughput=5MB/sec 2024-04-07T16:44:34.064 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-ZIDYhr8SpN5nIlP off=2452655 len=356433 2024-04-07T16:44:34.066 INFO:tasks.workunit.client.0.smithi100.stdout:op 1836 completed, throughput=5MB/sec 2024-04-07T16:44:34.066 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-IHOCxulDfw4iiCH off=3203553 len=414490 2024-04-07T16:44:34.067 INFO:tasks.workunit.client.0.smithi100.stdout:op 1837 completed, throughput=5MB/sec 2024-04-07T16:44:34.067 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-zsHYv2xY0cki2ro off=6662647 len=413754 2024-04-07T16:44:34.073 INFO:tasks.workunit.client.0.smithi100.stdout:op 1838 completed, throughput=5MB/sec 2024-04-07T16:44:34.073 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-ezowixSWDYS399e off=3642853 len=408044 2024-04-07T16:44:34.075 INFO:tasks.workunit.client.0.smithi100.stdout:op 1839 completed, throughput=5MB/sec 2024-04-07T16:44:34.076 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-RX-UzMbzXyV7qXu off=8912797 len=485743 2024-04-07T16:44:34.077 INFO:tasks.workunit.client.0.smithi100.stdout:op 1840 completed, throughput=5MB/sec 2024-04-07T16:44:34.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:44:34 smithi144 bash[66654]: cluster 2024-04-07T16:44:33.261253+0000 mgr.y (mgr.25175) 876 : cluster [DBG] pgmap v1518: 105 pgs: 105 active+clean; 126 GiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.9 MiB/s wr, 5 op/s 2024-04-07T16:44:34.943 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:44:34 smithi100 bash[74942]: cluster 2024-04-07T16:44:33.261253+0000 mgr.y (mgr.25175) 876 : cluster [DBG] pgmap v1518: 105 pgs: 105 active+clean; 126 GiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.9 MiB/s wr, 5 op/s 2024-04-07T16:44:34.943 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:44:34 smithi100 bash[76202]: cluster 2024-04-07T16:44:33.261253+0000 mgr.y (mgr.25175) 876 : cluster [DBG] pgmap v1518: 105 pgs: 105 active+clean; 126 GiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.9 MiB/s wr, 5 op/s 2024-04-07T16:44:35.077 INFO:tasks.workunit.client.0.smithi100.stdout: 347: throughput=4.99MB/sec pending data=0 2024-04-07T16:44:35.077 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-c-KbcQmm6s7Wi1b off=5860050 len=499420 2024-04-07T16:44:35.079 INFO:tasks.workunit.client.0.smithi100.stdout:op 1841 completed, throughput=4.99MB/sec 2024-04-07T16:44:35.079 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-3qT1mencGbA3eow off=19105379 len=1939943 2024-04-07T16:44:35.094 INFO:tasks.workunit.client.0.smithi100.stdout:op 1842 completed, throughput=4.99MB/sec 2024-04-07T16:44:35.094 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-CErGL3weYgYXmxR off=460444 len=1693385 2024-04-07T16:44:35.112 INFO:tasks.workunit.client.0.smithi100.stdout:op 1843 completed, throughput=5MB/sec 2024-04-07T16:44:35.112 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-coWkpSo1XwdvoE7 off=191432 len=1649849 2024-04-07T16:44:35.117 INFO:tasks.workunit.client.0.smithi100.stdout:op 1844 completed, throughput=5MB/sec 2024-04-07T16:44:36.117 INFO:tasks.workunit.client.0.smithi100.stdout: 348: throughput=4.99MB/sec pending data=0 2024-04-07T16:44:36.117 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-CT5IcTcxImikTZ_ off=7958808 len=2080635 2024-04-07T16:44:36.122 INFO:tasks.workunit.client.0.smithi100.stdout:op 1845 completed, throughput=4.99MB/sec 2024-04-07T16:44:36.122 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-_fifrkIu_7-hP00 off=1139674 len=1126380 2024-04-07T16:44:36.126 INFO:tasks.workunit.client.0.smithi100.stdout:op 1846 completed, throughput=5MB/sec 2024-04-07T16:44:36.126 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-bdLBlKirBu4uKA_ off=3346491 len=919497 2024-04-07T16:44:36.134 INFO:tasks.workunit.client.0.smithi100.stdout:op 1847 completed, throughput=5MB/sec 2024-04-07T16:44:36.134 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-VuffuGdJ9yPAeBu off=21615293 len=1828648 2024-04-07T16:44:36.138 INFO:tasks.workunit.client.0.smithi100.stdout:op 1848 completed, throughput=5MB/sec 2024-04-07T16:44:36.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:44:36 smithi144 bash[66654]: cluster 2024-04-07T16:44:35.262263+0000 mgr.y (mgr.25175) 877 : cluster [DBG] pgmap v1519: 105 pgs: 105 active+clean; 126 GiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 2.4 MiB/s rd, 2.2 MiB/s wr, 4 op/s 2024-04-07T16:44:36.943 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:44:36 smithi100 bash[74942]: cluster 2024-04-07T16:44:35.262263+0000 mgr.y (mgr.25175) 877 : cluster [DBG] pgmap v1519: 105 pgs: 105 active+clean; 126 GiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 2.4 MiB/s rd, 2.2 MiB/s wr, 4 op/s 2024-04-07T16:44:36.943 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:44:36 smithi100 bash[76202]: cluster 2024-04-07T16:44:35.262263+0000 mgr.y (mgr.25175) 877 : cluster [DBG] pgmap v1519: 105 pgs: 105 active+clean; 126 GiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 2.4 MiB/s rd, 2.2 MiB/s wr, 4 op/s 2024-04-07T16:44:37.139 INFO:tasks.workunit.client.0.smithi100.stdout: 349: throughput=4.99MB/sec pending data=0 2024-04-07T16:44:37.139 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-xvE7HrCSxrBHs4p off=4953059 len=574155 2024-04-07T16:44:37.142 INFO:tasks.workunit.client.0.smithi100.stdout:op 1849 completed, throughput=4.99MB/sec 2024-04-07T16:44:37.142 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-5MRzehSa56gbrE8 off=9587752 len=1728021 2024-04-07T16:44:37.146 INFO:tasks.workunit.client.0.smithi100.stdout:op 1850 completed, throughput=4.99MB/sec 2024-04-07T16:44:37.146 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-CrXuT4gG9v3lYqj off=7653490 len=212112 2024-04-07T16:44:37.151 INFO:tasks.workunit.client.0.smithi100.stdout:op 1851 completed, throughput=5MB/sec 2024-04-07T16:44:37.151 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj--O6Mu4Vaago3LL3 off=3589883 len=590574 2024-04-07T16:44:37.153 INFO:tasks.workunit.client.0.smithi100.stdout:op 1852 completed, throughput=5MB/sec 2024-04-07T16:44:37.153 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-6KA81TElx34zhyq off=4291702 len=966086 2024-04-07T16:44:37.161 INFO:tasks.workunit.client.0.smithi100.stdout:op 1853 completed, throughput=5MB/sec 2024-04-07T16:44:37.161 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-oqurYK1TcQu3lb_ off=6093871 len=1754944 2024-04-07T16:44:37.165 INFO:tasks.workunit.client.0.smithi100.stdout:op 1854 completed, throughput=5MB/sec 2024-04-07T16:44:38.166 INFO:tasks.workunit.client.0.smithi100.stdout: 350: throughput=4.99MB/sec pending data=0 2024-04-07T16:44:38.166 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-FOBMd5k2lQmJO8Q off=3794118 len=1502524 2024-04-07T16:44:38.179 INFO:tasks.workunit.client.0.smithi100.stdout:op 1855 completed, throughput=4.99MB/sec 2024-04-07T16:44:38.179 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Z_XUm9Hf3RbK0j6 off=8552167 len=125284 2024-04-07T16:44:38.183 INFO:tasks.workunit.client.0.smithi100.stdout:op 1856 completed, throughput=4.99MB/sec 2024-04-07T16:44:38.183 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-BD2rkfGQBtfVB1I off=7078134 len=376087 2024-04-07T16:44:38.185 INFO:tasks.workunit.client.0.smithi100.stdout:op 1857 completed, throughput=5MB/sec 2024-04-07T16:44:38.185 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-1uNXTsGKm-Rf7JH off=5711856 len=1435096 2024-04-07T16:44:38.195 INFO:tasks.workunit.client.0.smithi100.stdout:op 1858 completed, throughput=5MB/sec 2024-04-07T16:44:38.195 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-uE5P8bL3iCcnTaA off=6998623 len=1791401 2024-04-07T16:44:38.201 INFO:tasks.workunit.client.0.smithi100.stdout:op 1859 completed, throughput=5MB/sec 2024-04-07T16:44:38.692 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:44:38 smithi100 bash[74942]: cluster 2024-04-07T16:44:37.263258+0000 mgr.y (mgr.25175) 878 : cluster [DBG] pgmap v1520: 105 pgs: 105 active+clean; 126 GiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.3 MiB/s wr, 5 op/s 2024-04-07T16:44:38.693 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:44:38 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:44:38] "GET /metrics HTTP/1.1" 200 34052 "" "Prometheus/2.43.0" 2024-04-07T16:44:38.693 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:44:38 smithi100 bash[76202]: cluster 2024-04-07T16:44:37.263258+0000 mgr.y (mgr.25175) 878 : cluster [DBG] pgmap v1520: 105 pgs: 105 active+clean; 126 GiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.3 MiB/s wr, 5 op/s 2024-04-07T16:44:38.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:44:38 smithi144 bash[66654]: cluster 2024-04-07T16:44:37.263258+0000 mgr.y (mgr.25175) 878 : cluster [DBG] pgmap v1520: 105 pgs: 105 active+clean; 126 GiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.3 MiB/s wr, 5 op/s 2024-04-07T16:44:39.201 INFO:tasks.workunit.client.0.smithi100.stdout: 351: throughput=4.99MB/sec pending data=0 2024-04-07T16:44:39.202 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-ntiQ31WCHCBwrfD off=115404 len=1897658 2024-04-07T16:44:39.218 INFO:tasks.workunit.client.0.smithi100.stdout:op 1860 completed, throughput=4.99MB/sec 2024-04-07T16:44:39.218 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-utDkjYJVuQzq1mj off=3128610 len=1903044 2024-04-07T16:44:39.225 INFO:tasks.workunit.client.0.smithi100.stdout:op 1861 completed, throughput=5MB/sec 2024-04-07T16:44:39.225 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-taKG3tJu8mDfAzQ off=9858006 len=1410371 2024-04-07T16:44:39.236 INFO:tasks.workunit.client.0.smithi100.stdout:op 1862 completed, throughput=5MB/sec 2024-04-07T16:44:40.237 INFO:tasks.workunit.client.0.smithi100.stdout: 352: throughput=4.99MB/sec pending data=0 2024-04-07T16:44:40.237 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-aVaM3eU9iZtSweu off=8463183 len=1769254 2024-04-07T16:44:40.240 INFO:tasks.workunit.client.0.smithi100.stdout:op 1863 completed, throughput=4.99MB/sec 2024-04-07T16:44:40.241 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Y7r9oLt2eUl7-sN off=1091401 len=1250321 2024-04-07T16:44:40.244 INFO:tasks.workunit.client.0.smithi100.stdout:op 1864 completed, throughput=5MB/sec 2024-04-07T16:44:40.244 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-vD9GyEJCDkYaZXh off=0 len=1087327 2024-04-07T16:44:40.253 INFO:tasks.workunit.client.0.smithi100.stdout:op 1865 completed, throughput=5MB/sec 2024-04-07T16:44:40.253 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-c22JrkMu3MXSB65 off=5344395 len=269746 2024-04-07T16:44:40.258 INFO:tasks.workunit.client.0.smithi100.stdout:op 1866 completed, throughput=5MB/sec 2024-04-07T16:44:40.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:44:40 smithi144 bash[66654]: cluster 2024-04-07T16:44:39.264133+0000 mgr.y (mgr.25175) 879 : cluster [DBG] pgmap v1521: 105 pgs: 105 active+clean; 126 GiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 2.4 MiB/s rd, 1.8 MiB/s wr, 4 op/s 2024-04-07T16:44:40.943 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:44:40 smithi100 bash[74942]: cluster 2024-04-07T16:44:39.264133+0000 mgr.y (mgr.25175) 879 : cluster [DBG] pgmap v1521: 105 pgs: 105 active+clean; 126 GiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 2.4 MiB/s rd, 1.8 MiB/s wr, 4 op/s 2024-04-07T16:44:40.943 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:44:40 smithi100 bash[76202]: cluster 2024-04-07T16:44:39.264133+0000 mgr.y (mgr.25175) 879 : cluster [DBG] pgmap v1521: 105 pgs: 105 active+clean; 126 GiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 2.4 MiB/s rd, 1.8 MiB/s wr, 4 op/s 2024-04-07T16:44:41.258 INFO:tasks.workunit.client.0.smithi100.stdout: 353: throughput=4.99MB/sec pending data=0 2024-04-07T16:44:41.259 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-V_OFyocHGmPLMdA off=1042639 len=1491014 2024-04-07T16:44:41.263 INFO:tasks.workunit.client.0.smithi100.stdout:op 1867 completed, throughput=4.99MB/sec 2024-04-07T16:44:41.263 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-cBD2pTgeBM6OCQe off=19466847 len=521093 2024-04-07T16:44:41.270 INFO:tasks.workunit.client.0.smithi100.stdout:op 1868 completed, throughput=4.99MB/sec 2024-04-07T16:44:41.270 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-f4mNCDb35mqPVWj off=18817307 len=933099 2024-04-07T16:44:41.273 INFO:tasks.workunit.client.0.smithi100.stdout:op 1869 completed, throughput=4.99MB/sec 2024-04-07T16:44:41.273 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-iKa2GnkGcXA91tw off=2164571 len=1229354 2024-04-07T16:44:41.284 INFO:tasks.workunit.client.0.smithi100.stdout:op 1870 completed, throughput=5MB/sec 2024-04-07T16:44:41.284 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-QSA2z8i2WCKjD6H off=4259211 len=1623178 2024-04-07T16:44:41.296 INFO:tasks.workunit.client.0.smithi100.stdout:op 1871 completed, throughput=5MB/sec 2024-04-07T16:44:42.296 INFO:tasks.workunit.client.0.smithi100.stdout: 354: throughput=4.99MB/sec pending data=0 2024-04-07T16:44:42.296 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-sCWqw08NdlmLDUm off=7007376 len=1416566 2024-04-07T16:44:42.311 INFO:tasks.workunit.client.0.smithi100.stdout:op 1872 completed, throughput=4.99MB/sec 2024-04-07T16:44:42.311 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-AqBaoF0brMNJRe7 off=3366883 len=722094 2024-04-07T16:44:42.314 INFO:tasks.workunit.client.0.smithi100.stdout:op 1873 completed, throughput=4.99MB/sec 2024-04-07T16:44:42.314 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-J8iQlh7yqE-OGag off=9718651 len=219609 2024-04-07T16:44:42.319 INFO:tasks.workunit.client.0.smithi100.stdout:op 1874 completed, throughput=4.99MB/sec 2024-04-07T16:44:42.319 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-9OI2olpHIAWRFS0 off=3452661 len=1537638 2024-04-07T16:44:42.331 INFO:tasks.workunit.client.0.smithi100.stdout:op 1875 completed, throughput=5MB/sec 2024-04-07T16:44:42.331 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-DgXahwWOkm9kNnV off=3396341 len=144972 2024-04-07T16:44:42.332 INFO:tasks.workunit.client.0.smithi100.stdout:op 1876 completed, throughput=5MB/sec 2024-04-07T16:44:42.332 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-ezAaei4sJPOIyAe off=499129 len=846276 2024-04-07T16:44:42.334 INFO:tasks.workunit.client.0.smithi100.stdout:op 1877 completed, throughput=5MB/sec 2024-04-07T16:44:42.335 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-hJIW2qQ19WXLVGR off=967476 len=1813564 2024-04-07T16:44:42.339 INFO:tasks.workunit.client.0.smithi100.stdout:op 1878 completed, throughput=5MB/sec 2024-04-07T16:44:42.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:44:42 smithi144 bash[66654]: cluster 2024-04-07T16:44:41.265153+0000 mgr.y (mgr.25175) 880 : cluster [DBG] pgmap v1522: 105 pgs: 105 active+clean; 126 GiB data, 5.6 GiB used, 710 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.1 MiB/s wr, 4 op/s 2024-04-07T16:44:42.608 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:44:42 smithi144 bash[95233]: ts=2024-04-07T16:44:42.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:44:42.943 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:44:42 smithi100 bash[74942]: cluster 2024-04-07T16:44:41.265153+0000 mgr.y (mgr.25175) 880 : cluster [DBG] pgmap v1522: 105 pgs: 105 active+clean; 126 GiB data, 5.6 GiB used, 710 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.1 MiB/s wr, 4 op/s 2024-04-07T16:44:42.943 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:44:42 smithi100 bash[76202]: cluster 2024-04-07T16:44:41.265153+0000 mgr.y (mgr.25175) 880 : cluster [DBG] pgmap v1522: 105 pgs: 105 active+clean; 126 GiB data, 5.6 GiB used, 710 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.1 MiB/s wr, 4 op/s 2024-04-07T16:44:43.340 INFO:tasks.workunit.client.0.smithi100.stdout: 355: throughput=4.99MB/sec pending data=0 2024-04-07T16:44:43.340 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-LhFPcfgASEI698n off=4450143 len=1837181 2024-04-07T16:44:43.344 INFO:tasks.workunit.client.0.smithi100.stdout:op 1879 completed, throughput=5MB/sec 2024-04-07T16:44:43.344 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-xqtOMEgOSp6ryib off=7194139 len=1806382 2024-04-07T16:44:43.358 INFO:tasks.workunit.client.0.smithi100.stdout:op 1880 completed, throughput=5MB/sec 2024-04-07T16:44:44.358 INFO:tasks.workunit.client.0.smithi100.stdout: 356: throughput=4.99MB/sec pending data=0 2024-04-07T16:44:44.358 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-EoL5bf84-wrGqni off=2351946 len=108281 2024-04-07T16:44:44.360 INFO:tasks.workunit.client.0.smithi100.stdout:op 1881 completed, throughput=4.99MB/sec 2024-04-07T16:44:44.360 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-viK50k8PaPtWndI off=11604453 len=451465 2024-04-07T16:44:44.362 INFO:tasks.workunit.client.0.smithi100.stdout:op 1882 completed, throughput=4.99MB/sec 2024-04-07T16:44:44.362 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-nCNHggDeEgDB7c9 off=12145020 len=1096935 2024-04-07T16:44:44.371 INFO:tasks.workunit.client.0.smithi100.stdout:op 1883 completed, throughput=4.99MB/sec 2024-04-07T16:44:44.371 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-UbkKK_s7lPgHbmk off=86600 len=1184175 2024-04-07T16:44:44.382 INFO:tasks.workunit.client.0.smithi100.stdout:op 1884 completed, throughput=4.99MB/sec 2024-04-07T16:44:44.382 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-soxSWXz0Kx82roh off=2986319 len=1797692 2024-04-07T16:44:44.385 INFO:tasks.workunit.client.0.smithi100.stdout:op 1885 completed, throughput=5MB/sec 2024-04-07T16:44:44.385 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Do2KbklW1viLDyQ off=14017923 len=544300 2024-04-07T16:44:44.392 INFO:tasks.workunit.client.0.smithi100.stdout:op 1886 completed, throughput=5MB/sec 2024-04-07T16:44:44.392 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-yIx8CzHuRz_mvxF off=3050844 len=60475 2024-04-07T16:44:44.393 INFO:tasks.workunit.client.0.smithi100.stdout:op 1887 completed, throughput=5MB/sec 2024-04-07T16:44:44.393 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-ThBAY9_eJ3piftp off=2184447 len=924447 2024-04-07T16:44:44.402 INFO:tasks.workunit.client.0.smithi100.stdout:op 1888 completed, throughput=5MB/sec 2024-04-07T16:44:44.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:44:44 smithi144 bash[66654]: cluster 2024-04-07T16:44:43.266258+0000 mgr.y (mgr.25175) 881 : cluster [DBG] pgmap v1523: 105 pgs: 105 active+clean; 126 GiB data, 5.6 GiB used, 710 GiB / 715 GiB avail; 2.8 MiB/s rd, 2.6 MiB/s wr, 4 op/s 2024-04-07T16:44:44.943 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:44:44 smithi100 bash[74942]: cluster 2024-04-07T16:44:43.266258+0000 mgr.y (mgr.25175) 881 : cluster [DBG] pgmap v1523: 105 pgs: 105 active+clean; 126 GiB data, 5.6 GiB used, 710 GiB / 715 GiB avail; 2.8 MiB/s rd, 2.6 MiB/s wr, 4 op/s 2024-04-07T16:44:44.943 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:44:44 smithi100 bash[76202]: cluster 2024-04-07T16:44:43.266258+0000 mgr.y (mgr.25175) 881 : cluster [DBG] pgmap v1523: 105 pgs: 105 active+clean; 126 GiB data, 5.6 GiB used, 710 GiB / 715 GiB avail; 2.8 MiB/s rd, 2.6 MiB/s wr, 4 op/s 2024-04-07T16:44:45.402 INFO:tasks.workunit.client.0.smithi100.stdout: 357: throughput=4.99MB/sec pending data=0 2024-04-07T16:44:45.403 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-k5YlKAfw-daeiP- off=3566502 len=597082 2024-04-07T16:44:45.405 INFO:tasks.workunit.client.0.smithi100.stdout:op 1889 completed, throughput=4.99MB/sec 2024-04-07T16:44:45.405 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-_AEmcFW_kMZBDrn off=19883577 len=1497464 2024-04-07T16:44:45.416 INFO:tasks.workunit.client.0.smithi100.stdout:op 1890 completed, throughput=4.99MB/sec 2024-04-07T16:44:45.416 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-3SX3rvwa1EDetWB off=1615065 len=2039425 2024-04-07T16:44:45.421 INFO:tasks.workunit.client.0.smithi100.stdout:op 1891 completed, throughput=5MB/sec 2024-04-07T16:44:45.421 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-o8CzUJUC9y1J2MY off=10875131 len=1786815 2024-04-07T16:44:45.427 INFO:tasks.workunit.client.0.smithi100.stdout:op 1892 completed, throughput=5MB/sec 2024-04-07T16:44:46.427 INFO:tasks.workunit.client.0.smithi100.stdout: 358: throughput=4.99MB/sec pending data=0 2024-04-07T16:44:46.427 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Gy9BfaK9_gtreVu off=1602345 len=30650 2024-04-07T16:44:46.431 INFO:tasks.workunit.client.0.smithi100.stdout:op 1893 completed, throughput=4.99MB/sec 2024-04-07T16:44:46.431 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-3HlT72lZ3QPM82p off=6226928 len=364554 2024-04-07T16:44:46.437 INFO:tasks.workunit.client.0.smithi100.stdout:op 1894 completed, throughput=4.99MB/sec 2024-04-07T16:44:46.437 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-YJ17Pm4Vu8ws-fI off=14164981 len=652563 2024-04-07T16:44:46.444 INFO:tasks.workunit.client.0.smithi100.stdout:op 1895 completed, throughput=4.99MB/sec 2024-04-07T16:44:46.444 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-ib3NAmiolnTnZhv off=5618155 len=1789758 2024-04-07T16:44:46.460 INFO:tasks.workunit.client.0.smithi100.stdout:op 1896 completed, throughput=5MB/sec 2024-04-07T16:44:46.461 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-th-dW-8hc4u38Yi off=6143070 len=1510884 2024-04-07T16:44:46.471 INFO:tasks.workunit.client.0.smithi100.stdout:op 1897 completed, throughput=5MB/sec 2024-04-07T16:44:46.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:44:46 smithi144 bash[66654]: cluster 2024-04-07T16:44:45.267241+0000 mgr.y (mgr.25175) 882 : cluster [DBG] pgmap v1524: 105 pgs: 105 active+clean; 126 GiB data, 5.6 GiB used, 710 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.3 MiB/s wr, 4 op/s 2024-04-07T16:44:46.943 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:44:46 smithi100 bash[74942]: cluster 2024-04-07T16:44:45.267241+0000 mgr.y (mgr.25175) 882 : cluster [DBG] pgmap v1524: 105 pgs: 105 active+clean; 126 GiB data, 5.6 GiB used, 710 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.3 MiB/s wr, 4 op/s 2024-04-07T16:44:46.943 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:44:46 smithi100 bash[76202]: cluster 2024-04-07T16:44:45.267241+0000 mgr.y (mgr.25175) 882 : cluster [DBG] pgmap v1524: 105 pgs: 105 active+clean; 126 GiB data, 5.6 GiB used, 710 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.3 MiB/s wr, 4 op/s 2024-04-07T16:44:47.472 INFO:tasks.workunit.client.0.smithi100.stdout: 359: throughput=4.99MB/sec pending data=0 2024-04-07T16:44:47.472 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-h4S-YMgM2N0lzJp off=9093748 len=1546629 2024-04-07T16:44:47.476 INFO:tasks.workunit.client.0.smithi100.stdout:op 1898 completed, throughput=4.99MB/sec 2024-04-07T16:44:47.476 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-u-LtEgh7MchDpKe off=8058854 len=1007904 2024-04-07T16:44:47.484 INFO:tasks.workunit.client.0.smithi100.stdout:op 1899 completed, throughput=4.99MB/sec 2024-04-07T16:44:47.484 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-07QLH0HyO_30oA2 off=18155806 len=1454182 2024-04-07T16:44:47.486 INFO:tasks.workunit.client.0.smithi100.stdout:op 1900 completed, throughput=5MB/sec 2024-04-07T16:44:47.486 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-iUDhlcf8XBM6MWf off=755658 len=507792 2024-04-07T16:44:47.492 INFO:tasks.workunit.client.0.smithi100.stdout:op 1901 completed, throughput=5MB/sec 2024-04-07T16:44:47.492 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-SeCabn_KffKy8Rb off=268792 len=221011 2024-04-07T16:44:47.498 INFO:tasks.workunit.client.0.smithi100.stdout:op 1902 completed, throughput=5MB/sec 2024-04-07T16:44:47.498 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-pfr7G33e4BcXHPC off=3679939 len=2001307 2024-04-07T16:44:47.517 INFO:tasks.workunit.client.0.smithi100.stdout:op 1903 completed, throughput=5MB/sec 2024-04-07T16:44:47.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:44:47 smithi144 bash[66654]: audit 2024-04-07T16:44:46.765427+0000 mon.a (mon.0) 2094 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:44:47.943 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:44:47 smithi100 bash[74942]: audit 2024-04-07T16:44:46.765427+0000 mon.a (mon.0) 2094 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:44:47.943 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:44:47 smithi100 bash[76202]: audit 2024-04-07T16:44:46.765427+0000 mon.a (mon.0) 2094 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:44:48.517 INFO:tasks.workunit.client.0.smithi100.stdout: 360: throughput=4.99MB/sec pending data=0 2024-04-07T16:44:48.517 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-axWmIjCnBPvm783 off=11122313 len=123123 2024-04-07T16:44:48.522 INFO:tasks.workunit.client.0.smithi100.stdout:op 1904 completed, throughput=4.99MB/sec 2024-04-07T16:44:48.522 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-vv4f-RqkVFf6Z97 off=4604972 len=464664 2024-04-07T16:44:48.524 INFO:tasks.workunit.client.0.smithi100.stdout:op 1905 completed, throughput=4.99MB/sec 2024-04-07T16:44:48.525 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-gZxQz3qfSC_savB off=5356013 len=977499 2024-04-07T16:44:48.533 INFO:tasks.workunit.client.0.smithi100.stdout:op 1906 completed, throughput=4.99MB/sec 2024-04-07T16:44:48.533 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-viliziyVpD8nWGw off=2836019 len=1536793 2024-04-07T16:44:48.539 INFO:tasks.workunit.client.0.smithi100.stdout:op 1907 completed, throughput=5MB/sec 2024-04-07T16:44:48.539 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-zshFwjNojNkNeKz off=13681087 len=433996 2024-04-07T16:44:48.545 INFO:tasks.workunit.client.0.smithi100.stdout:op 1908 completed, throughput=5MB/sec 2024-04-07T16:44:48.545 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-ORE4FBgtBOqABk7 off=5796136 len=867785 2024-04-07T16:44:48.553 INFO:tasks.workunit.client.0.smithi100.stdout:op 1909 completed, throughput=5MB/sec 2024-04-07T16:44:48.693 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:44:48 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:44:48] "GET /metrics HTTP/1.1" 200 34051 "" "Prometheus/2.43.0" 2024-04-07T16:44:48.693 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:44:48 smithi100 bash[74942]: cluster 2024-04-07T16:44:47.268241+0000 mgr.y (mgr.25175) 883 : cluster [DBG] pgmap v1525: 105 pgs: 105 active+clean; 126 GiB data, 5.6 GiB used, 710 GiB / 715 GiB avail; 2.6 MiB/s rd, 2.3 MiB/s wr, 5 op/s 2024-04-07T16:44:48.693 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:44:48 smithi100 bash[76202]: cluster 2024-04-07T16:44:47.268241+0000 mgr.y (mgr.25175) 883 : cluster [DBG] pgmap v1525: 105 pgs: 105 active+clean; 126 GiB data, 5.6 GiB used, 710 GiB / 715 GiB avail; 2.6 MiB/s rd, 2.3 MiB/s wr, 5 op/s 2024-04-07T16:44:48.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:44:48 smithi144 bash[66654]: cluster 2024-04-07T16:44:47.268241+0000 mgr.y (mgr.25175) 883 : cluster [DBG] pgmap v1525: 105 pgs: 105 active+clean; 126 GiB data, 5.6 GiB used, 710 GiB / 715 GiB avail; 2.6 MiB/s rd, 2.3 MiB/s wr, 5 op/s 2024-04-07T16:44:49.553 INFO:tasks.workunit.client.0.smithi100.stdout: 361: throughput=4.99MB/sec pending data=0 2024-04-07T16:44:49.553 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Gges8w0wXNTxtHz off=7846573 len=1271760 2024-04-07T16:44:49.558 INFO:tasks.workunit.client.0.smithi100.stdout:op 1910 completed, throughput=4.99MB/sec 2024-04-07T16:44:49.558 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-lffbQglAgLJxpUR off=2777527 len=1603694 2024-04-07T16:44:49.563 INFO:tasks.workunit.client.0.smithi100.stdout:op 1911 completed, throughput=4.99MB/sec 2024-04-07T16:44:49.563 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-koT_S8DC491A4te off=418779 len=1191042 2024-04-07T16:44:49.566 INFO:tasks.workunit.client.0.smithi100.stdout:op 1912 completed, throughput=5MB/sec 2024-04-07T16:44:49.567 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-iKa2GnkGcXA91tw off=5492561 len=1377233 2024-04-07T16:44:49.579 INFO:tasks.workunit.client.0.smithi100.stdout:op 1913 completed, throughput=5MB/sec 2024-04-07T16:44:50.579 INFO:tasks.workunit.client.0.smithi100.stdout: 362: throughput=4.99MB/sec pending data=0 2024-04-07T16:44:50.663 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-OhapxvL12zZjNiI off=1714914 len=576772 2024-04-07T16:44:50.664 INFO:tasks.workunit.client.0.smithi100.stdout:op 1914 completed, throughput=4.99MB/sec 2024-04-07T16:44:50.664 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-cyUx_6_JJyH05XW off=13003112 len=1836582 2024-04-07T16:44:50.664 INFO:tasks.workunit.client.0.smithi100.stdout:op 1915 completed, throughput=4.99MB/sec 2024-04-07T16:44:50.664 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-5fnWWuKLJM8Nx3J off=17090607 len=581454 2024-04-07T16:44:50.664 INFO:tasks.workunit.client.0.smithi100.stdout:op 1916 completed, throughput=5MB/sec 2024-04-07T16:44:50.664 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-iV7QOPdKjXX2LR9 off=3986454 len=1952281 2024-04-07T16:44:50.664 INFO:tasks.workunit.client.0.smithi100.stdout:op 1917 completed, throughput=5MB/sec 2024-04-07T16:44:50.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:44:50 smithi144 bash[66654]: cluster 2024-04-07T16:44:49.269016+0000 mgr.y (mgr.25175) 884 : cluster [DBG] pgmap v1526: 105 pgs: 105 active+clean; 126 GiB data, 5.6 GiB used, 710 GiB / 715 GiB avail; 2.8 MiB/s rd, 2.3 MiB/s wr, 4 op/s 2024-04-07T16:44:50.943 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:44:50 smithi100 bash[74942]: cluster 2024-04-07T16:44:49.269016+0000 mgr.y (mgr.25175) 884 : cluster [DBG] pgmap v1526: 105 pgs: 105 active+clean; 126 GiB data, 5.6 GiB used, 710 GiB / 715 GiB avail; 2.8 MiB/s rd, 2.3 MiB/s wr, 4 op/s 2024-04-07T16:44:50.943 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:44:50 smithi100 bash[76202]: cluster 2024-04-07T16:44:49.269016+0000 mgr.y (mgr.25175) 884 : cluster [DBG] pgmap v1526: 105 pgs: 105 active+clean; 126 GiB data, 5.6 GiB used, 710 GiB / 715 GiB avail; 2.8 MiB/s rd, 2.3 MiB/s wr, 4 op/s 2024-04-07T16:44:51.599 INFO:tasks.workunit.client.0.smithi100.stdout: 363: throughput=4.99MB/sec pending data=0 2024-04-07T16:44:51.599 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-5Xw2hBmgwW0Q4Vx off=8561387 len=1504176 2024-04-07T16:44:51.614 INFO:tasks.workunit.client.0.smithi100.stdout:op 1918 completed, throughput=4.99MB/sec 2024-04-07T16:44:51.614 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-MvTPIddarlhuxvd off=6269696 len=660419 2024-04-07T16:44:51.616 INFO:tasks.workunit.client.0.smithi100.stdout:op 1919 completed, throughput=4.99MB/sec 2024-04-07T16:44:51.616 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-YwBjlFaUXlUvna- off=2605370 len=91360 2024-04-07T16:44:51.618 INFO:tasks.workunit.client.0.smithi100.stdout:op 1920 completed, throughput=4.99MB/sec 2024-04-07T16:44:51.618 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-pN5tcHKHrB_I7jg off=0 len=1509909 2024-04-07T16:44:51.629 INFO:tasks.workunit.client.0.smithi100.stdout:op 1921 completed, throughput=5MB/sec 2024-04-07T16:44:51.629 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-_eOHrWEIzTgc_vE off=9033570 len=162042 2024-04-07T16:44:51.630 INFO:tasks.workunit.client.0.smithi100.stdout:op 1922 completed, throughput=5MB/sec 2024-04-07T16:44:51.630 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-p8wkGn_4WWZg-qc off=13307645 len=1218947 2024-04-07T16:44:51.640 INFO:tasks.workunit.client.0.smithi100.stdout:op 1923 completed, throughput=5MB/sec 2024-04-07T16:44:51.640 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-AMIsq5RhmgFNVIF off=19945016 len=1503329 2024-04-07T16:44:51.641 INFO:tasks.workunit.client.0.smithi100.stdout:op 1924 completed, throughput=5MB/sec 2024-04-07T16:44:52.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:44:52 smithi144 bash[66654]: cluster 2024-04-07T16:44:51.269964+0000 mgr.y (mgr.25175) 885 : cluster [DBG] pgmap v1527: 105 pgs: 105 active+clean; 126 GiB data, 5.6 GiB used, 710 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.5 MiB/s wr, 4 op/s 2024-04-07T16:44:52.608 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:44:52 smithi144 bash[95233]: ts=2024-04-07T16:44:52.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:44:52.642 INFO:tasks.workunit.client.0.smithi100.stdout: 364: throughput=4.99MB/sec pending data=0 2024-04-07T16:44:52.642 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-xlBOBlNSUQWzZG7 off=2298209 len=634722 2024-04-07T16:44:52.649 INFO:tasks.workunit.client.0.smithi100.stdout:op 1925 completed, throughput=4.99MB/sec 2024-04-07T16:44:52.649 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-7xIvwMhogwukgBC off=5748306 len=49118 2024-04-07T16:44:52.650 INFO:tasks.workunit.client.0.smithi100.stdout:op 1926 completed, throughput=4.99MB/sec 2024-04-07T16:44:52.650 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-UGRxcSHe-0kCr_v off=1995810 len=1828422 2024-04-07T16:44:52.668 INFO:tasks.workunit.client.0.smithi100.stdout:op 1927 completed, throughput=5MB/sec 2024-04-07T16:44:52.668 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-cVKeW-oG5N2QDmo off=218443 len=44324 2024-04-07T16:44:52.669 INFO:tasks.workunit.client.0.smithi100.stdout:op 1928 completed, throughput=5MB/sec 2024-04-07T16:44:52.669 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Mh8kC1KNiG3V9go off=13025376 len=267493 2024-04-07T16:44:52.674 INFO:tasks.workunit.client.0.smithi100.stdout:op 1929 completed, throughput=5MB/sec 2024-04-07T16:44:52.674 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-QPvWFiW52-rsahi off=1835350 len=138914 2024-04-07T16:44:52.680 INFO:tasks.workunit.client.0.smithi100.stdout:op 1930 completed, throughput=5MB/sec 2024-04-07T16:44:52.680 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-hynUtBk-S2djY7i off=0 len=1064107 2024-04-07T16:44:52.683 INFO:tasks.workunit.client.0.smithi100.stdout:op 1931 completed, throughput=5MB/sec 2024-04-07T16:44:52.683 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-2lAKukAP8Wy2TOC off=14342495 len=725059 2024-04-07T16:44:52.685 INFO:tasks.workunit.client.0.smithi100.stdout:op 1932 completed, throughput=5MB/sec 2024-04-07T16:44:52.943 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:44:52 smithi100 bash[74942]: cluster 2024-04-07T16:44:51.269964+0000 mgr.y (mgr.25175) 885 : cluster [DBG] pgmap v1527: 105 pgs: 105 active+clean; 126 GiB data, 5.6 GiB used, 710 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.5 MiB/s wr, 4 op/s 2024-04-07T16:44:52.943 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:44:52 smithi100 bash[76202]: cluster 2024-04-07T16:44:51.269964+0000 mgr.y (mgr.25175) 885 : cluster [DBG] pgmap v1527: 105 pgs: 105 active+clean; 126 GiB data, 5.6 GiB used, 710 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.5 MiB/s wr, 4 op/s 2024-04-07T16:44:53.686 INFO:tasks.workunit.client.0.smithi100.stdout: 365: throughput=4.99MB/sec pending data=0 2024-04-07T16:44:53.686 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-bhoF4c4V3f9YN9y off=3504557 len=100301 2024-04-07T16:44:53.686 INFO:tasks.workunit.client.0.smithi100.stdout:op 1933 completed, throughput=4.99MB/sec 2024-04-07T16:44:53.686 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-DGovuQu98srUpLO off=3679249 len=880587 2024-04-07T16:44:53.687 INFO:tasks.workunit.client.0.smithi100.stdout:op 1934 completed, throughput=4.99MB/sec 2024-04-07T16:44:53.688 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-loEuuBNNT5QHTPD off=13848217 len=560723 2024-04-07T16:44:53.689 INFO:tasks.workunit.client.0.smithi100.stdout:op 1935 completed, throughput=4.99MB/sec 2024-04-07T16:44:53.689 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-EWZSxV0B0oXuUgt off=2439688 len=1148884 2024-04-07T16:44:53.690 INFO:tasks.workunit.client.0.smithi100.stdout:op 1936 completed, throughput=4.99MB/sec 2024-04-07T16:44:53.690 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-MfnsMNoeL1a_Tua off=16908139 len=1585937 2024-04-07T16:44:53.692 INFO:tasks.workunit.client.0.smithi100.stdout:op 1937 completed, throughput=5MB/sec 2024-04-07T16:44:53.693 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-d7wYBJbkFbmkMZR off=18592422 len=1487230 2024-04-07T16:44:53.700 INFO:tasks.workunit.client.0.smithi100.stdout:op 1938 completed, throughput=5MB/sec 2024-04-07T16:44:53.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:44:53 smithi144 bash[66654]: audit 2024-04-07T16:44:53.138962+0000 mon.a (mon.0) 2095 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:44:53.943 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:44:53 smithi100 bash[74942]: audit 2024-04-07T16:44:53.138962+0000 mon.a (mon.0) 2095 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:44:53.943 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:44:53 smithi100 bash[76202]: audit 2024-04-07T16:44:53.138962+0000 mon.a (mon.0) 2095 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:44:54.700 INFO:tasks.workunit.client.0.smithi100.stdout: 366: throughput=4.99MB/sec pending data=0 2024-04-07T16:44:54.701 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-kgrL2uo-u57e5ji off=4384072 len=1966691 2024-04-07T16:44:54.705 INFO:tasks.workunit.client.0.smithi100.stdout:op 1939 completed, throughput=4.99MB/sec 2024-04-07T16:44:54.705 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-0GDBmW7f9AV-QT5 off=8125122 len=1950053 2024-04-07T16:44:54.719 INFO:tasks.workunit.client.0.smithi100.stdout:op 1940 completed, throughput=5MB/sec 2024-04-07T16:44:54.719 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-BGyHVgjtn9xx3o8 off=8574542 len=1135624 2024-04-07T16:44:54.729 INFO:tasks.workunit.client.0.smithi100.stdout:op 1941 completed, throughput=5MB/sec 2024-04-07T16:44:54.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:44:54 smithi144 bash[66654]: cluster 2024-04-07T16:44:53.271059+0000 mgr.y (mgr.25175) 886 : cluster [DBG] pgmap v1528: 105 pgs: 105 active+clean; 126 GiB data, 5.6 GiB used, 710 GiB / 715 GiB avail; 2.8 MiB/s rd, 2.6 MiB/s wr, 5 op/s 2024-04-07T16:44:54.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:44:54 smithi144 bash[66654]: audit 2024-04-07T16:44:53.667064+0000 mon.a (mon.0) 2096 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:44:54.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:44:54 smithi144 bash[66654]: audit 2024-04-07T16:44:53.674655+0000 mon.a (mon.0) 2097 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:44:54.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:44:54 smithi144 bash[66654]: audit 2024-04-07T16:44:54.365960+0000 mon.a (mon.0) 2098 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:44:54.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:44:54 smithi144 bash[66654]: audit 2024-04-07T16:44:54.367723+0000 mon.a (mon.0) 2099 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:44:54.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:44:54 smithi144 bash[66654]: audit 2024-04-07T16:44:54.377851+0000 mon.a (mon.0) 2100 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:44:54.943 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:44:54 smithi100 bash[74942]: cluster 2024-04-07T16:44:53.271059+0000 mgr.y (mgr.25175) 886 : cluster [DBG] pgmap v1528: 105 pgs: 105 active+clean; 126 GiB data, 5.6 GiB used, 710 GiB / 715 GiB avail; 2.8 MiB/s rd, 2.6 MiB/s wr, 5 op/s 2024-04-07T16:44:54.943 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:44:54 smithi100 bash[74942]: audit 2024-04-07T16:44:53.667064+0000 mon.a (mon.0) 2096 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:44:54.943 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:44:54 smithi100 bash[74942]: audit 2024-04-07T16:44:53.674655+0000 mon.a (mon.0) 2097 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:44:54.943 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:44:54 smithi100 bash[74942]: audit 2024-04-07T16:44:54.365960+0000 mon.a (mon.0) 2098 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:44:54.943 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:44:54 smithi100 bash[74942]: audit 2024-04-07T16:44:54.367723+0000 mon.a (mon.0) 2099 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:44:54.944 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:44:54 smithi100 bash[74942]: audit 2024-04-07T16:44:54.377851+0000 mon.a (mon.0) 2100 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:44:54.944 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:44:54 smithi100 bash[76202]: cluster 2024-04-07T16:44:53.271059+0000 mgr.y (mgr.25175) 886 : cluster [DBG] pgmap v1528: 105 pgs: 105 active+clean; 126 GiB data, 5.6 GiB used, 710 GiB / 715 GiB avail; 2.8 MiB/s rd, 2.6 MiB/s wr, 5 op/s 2024-04-07T16:44:54.944 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:44:54 smithi100 bash[76202]: audit 2024-04-07T16:44:53.667064+0000 mon.a (mon.0) 2096 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:44:54.944 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:44:54 smithi100 bash[76202]: audit 2024-04-07T16:44:53.674655+0000 mon.a (mon.0) 2097 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:44:54.944 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:44:54 smithi100 bash[76202]: audit 2024-04-07T16:44:54.365960+0000 mon.a (mon.0) 2098 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:44:54.944 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:44:54 smithi100 bash[76202]: audit 2024-04-07T16:44:54.367723+0000 mon.a (mon.0) 2099 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:44:54.944 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:44:54 smithi100 bash[76202]: audit 2024-04-07T16:44:54.377851+0000 mon.a (mon.0) 2100 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:44:55.729 INFO:tasks.workunit.client.0.smithi100.stdout: 367: throughput=4.99MB/sec pending data=0 2024-04-07T16:44:55.729 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-wMlWwJ0VrTMexZE off=14037651 len=285613 2024-04-07T16:44:55.731 INFO:tasks.workunit.client.0.smithi100.stdout:op 1942 completed, throughput=4.99MB/sec 2024-04-07T16:44:55.731 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj--ZNdrjnvKMB2V_y off=0 len=1808494 2024-04-07T16:44:55.744 INFO:tasks.workunit.client.0.smithi100.stdout:op 1943 completed, throughput=4.99MB/sec 2024-04-07T16:44:55.744 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-UNGDFARDjx18MMP off=3248000 len=1254832 2024-04-07T16:44:55.755 INFO:tasks.workunit.client.0.smithi100.stdout:op 1944 completed, throughput=5MB/sec 2024-04-07T16:44:55.755 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-3VQOm3roKrvtZYx off=297111 len=78014 2024-04-07T16:44:55.756 INFO:tasks.workunit.client.0.smithi100.stdout:op 1945 completed, throughput=5MB/sec 2024-04-07T16:44:55.756 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-DQuvVNEICMS1zNI off=8679300 len=542880 2024-04-07T16:44:55.758 INFO:tasks.workunit.client.0.smithi100.stdout:op 1946 completed, throughput=5MB/sec 2024-04-07T16:44:55.758 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-aDiUmojNFHCXA_B off=219000 len=1134015 2024-04-07T16:44:55.761 INFO:tasks.workunit.client.0.smithi100.stdout:op 1947 completed, throughput=5MB/sec 2024-04-07T16:44:56.761 INFO:tasks.workunit.client.0.smithi100.stdout: 368: throughput=4.99MB/sec pending data=0 2024-04-07T16:44:56.762 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-PKLE4Ns3_anDhXX off=8764998 len=834505 2024-04-07T16:44:56.764 INFO:tasks.workunit.client.0.smithi100.stdout:op 1948 completed, throughput=4.99MB/sec 2024-04-07T16:44:56.764 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-p7kkbfT7UtWoSLG off=3870992 len=23153 2024-04-07T16:44:56.766 INFO:tasks.workunit.client.0.smithi100.stdout:op 1949 completed, throughput=4.99MB/sec 2024-04-07T16:44:56.766 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-evIgbSwo0e8NQh2 off=8724303 len=944544 2024-04-07T16:44:56.775 INFO:tasks.workunit.client.0.smithi100.stdout:op 1950 completed, throughput=4.99MB/sec 2024-04-07T16:44:56.775 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-k1VW07X5Cq7c19l off=3615465 len=670288 2024-04-07T16:44:56.778 INFO:tasks.workunit.client.0.smithi100.stdout:op 1951 completed, throughput=4.99MB/sec 2024-04-07T16:44:56.778 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-wl00iqcA5ko8IZw off=154832 len=1762602 2024-04-07T16:44:56.782 INFO:tasks.workunit.client.0.smithi100.stdout:op 1952 completed, throughput=5MB/sec 2024-04-07T16:44:56.782 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-f7vqVu_Vm2Yi9Z4 off=93451 len=1492472 2024-04-07T16:44:56.786 INFO:tasks.workunit.client.0.smithi100.stdout:op 1953 completed, throughput=5MB/sec 2024-04-07T16:44:56.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:44:56 smithi144 bash[66654]: cluster 2024-04-07T16:44:55.272057+0000 mgr.y (mgr.25175) 887 : cluster [DBG] pgmap v1529: 105 pgs: 105 active+clean; 126 GiB data, 5.7 GiB used, 710 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.4 MiB/s wr, 5 op/s 2024-04-07T16:44:56.943 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:44:56 smithi100 bash[74942]: cluster 2024-04-07T16:44:55.272057+0000 mgr.y (mgr.25175) 887 : cluster [DBG] pgmap v1529: 105 pgs: 105 active+clean; 126 GiB data, 5.7 GiB used, 710 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.4 MiB/s wr, 5 op/s 2024-04-07T16:44:56.943 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:44:56 smithi100 bash[76202]: cluster 2024-04-07T16:44:55.272057+0000 mgr.y (mgr.25175) 887 : cluster [DBG] pgmap v1529: 105 pgs: 105 active+clean; 126 GiB data, 5.7 GiB used, 710 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.4 MiB/s wr, 5 op/s 2024-04-07T16:44:57.787 INFO:tasks.workunit.client.0.smithi100.stdout: 369: throughput=4.99MB/sec pending data=0 2024-04-07T16:44:57.787 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-aj_7xnrmD-lStTw off=9182303 len=1530022 2024-04-07T16:44:57.799 INFO:tasks.workunit.client.0.smithi100.stdout:op 1954 completed, throughput=4.99MB/sec 2024-04-07T16:44:57.800 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-EpTq_67xn8Rv09R off=7313965 len=1874166 2024-04-07T16:44:57.814 INFO:tasks.workunit.client.0.smithi100.stdout:op 1955 completed, throughput=5MB/sec 2024-04-07T16:44:57.814 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj--5MNbk23Z8oBHzD off=3934508 len=471832 2024-04-07T16:44:57.820 INFO:tasks.workunit.client.0.smithi100.stdout:op 1956 completed, throughput=5MB/sec 2024-04-07T16:44:57.820 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-SgB5uqasgN5bIXF off=14261571 len=1743396 2024-04-07T16:44:57.824 INFO:tasks.workunit.client.0.smithi100.stdout:op 1957 completed, throughput=5MB/sec 2024-04-07T16:44:58.693 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:44:58 smithi100 bash[74942]: cluster 2024-04-07T16:44:57.273080+0000 mgr.y (mgr.25175) 888 : cluster [DBG] pgmap v1530: 105 pgs: 105 active+clean; 126 GiB data, 5.7 GiB used, 710 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.4 MiB/s wr, 5 op/s 2024-04-07T16:44:58.693 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:44:58 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:44:58] "GET /metrics HTTP/1.1" 200 34051 "" "Prometheus/2.43.0" 2024-04-07T16:44:58.694 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:44:58 smithi100 bash[76202]: cluster 2024-04-07T16:44:57.273080+0000 mgr.y (mgr.25175) 888 : cluster [DBG] pgmap v1530: 105 pgs: 105 active+clean; 126 GiB data, 5.7 GiB used, 710 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.4 MiB/s wr, 5 op/s 2024-04-07T16:44:58.825 INFO:tasks.workunit.client.0.smithi100.stdout: 370: throughput=4.99MB/sec pending data=0 2024-04-07T16:44:58.825 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-YFagGIsaN12G7ht off=922006 len=1076757 2024-04-07T16:44:58.835 INFO:tasks.workunit.client.0.smithi100.stdout:op 1958 completed, throughput=4.99MB/sec 2024-04-07T16:44:58.835 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-CbYdl9CbgRufup_ off=8300545 len=9735 2024-04-07T16:44:58.835 INFO:tasks.workunit.client.0.smithi100.stdout:op 1959 completed, throughput=4.99MB/sec 2024-04-07T16:44:58.836 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Wr62lHHfEaJjcn4 off=1855651 len=1593592 2024-04-07T16:44:58.848 INFO:tasks.workunit.client.0.smithi100.stdout:op 1960 completed, throughput=5MB/sec 2024-04-07T16:44:58.848 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-EkRBXaAap_igGYb off=5146002 len=173867 2024-04-07T16:44:58.849 INFO:tasks.workunit.client.0.smithi100.stdout:op 1961 completed, throughput=5MB/sec 2024-04-07T16:44:58.849 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-LYNlvFYVBtv72k- off=4042267 len=894487 2024-04-07T16:44:58.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:44:58 smithi144 bash[66654]: cluster 2024-04-07T16:44:57.273080+0000 mgr.y (mgr.25175) 888 : cluster [DBG] pgmap v1530: 105 pgs: 105 active+clean; 126 GiB data, 5.7 GiB used, 710 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.4 MiB/s wr, 5 op/s 2024-04-07T16:44:58.857 INFO:tasks.workunit.client.0.smithi100.stdout:op 1962 completed, throughput=5MB/sec 2024-04-07T16:44:58.858 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-incfB2uVqb7mV7v off=1896399 len=1908913 2024-04-07T16:44:58.870 INFO:tasks.workunit.client.0.smithi100.stdout:op 1963 completed, throughput=5MB/sec 2024-04-07T16:44:59.871 INFO:tasks.workunit.client.0.smithi100.stdout: 371: throughput=4.99MB/sec pending data=0 2024-04-07T16:44:59.871 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-4l6tJX86zKG8SOz off=0 len=1527338 2024-04-07T16:44:59.883 INFO:tasks.workunit.client.0.smithi100.stdout:op 1964 completed, throughput=4.99MB/sec 2024-04-07T16:44:59.883 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-5iRSCoY7Ly0kxiB off=1983866 len=1031337 2024-04-07T16:44:59.892 INFO:tasks.workunit.client.0.smithi100.stdout:op 1965 completed, throughput=5MB/sec 2024-04-07T16:44:59.892 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-pg27GVL7ng6k9z7 off=1473396 len=1591957 2024-04-07T16:44:59.898 INFO:tasks.workunit.client.0.smithi100.stdout:op 1966 completed, throughput=5MB/sec 2024-04-07T16:44:59.898 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-R1hGZYV3csRG6uC off=4873351 len=1724748 2024-04-07T16:44:59.902 INFO:tasks.workunit.client.0.smithi100.stdout:op 1967 completed, throughput=5MB/sec 2024-04-07T16:45:00.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:45:00 smithi144 bash[66654]: cluster 2024-04-07T16:44:59.273915+0000 mgr.y (mgr.25175) 889 : cluster [DBG] pgmap v1531: 105 pgs: 105 active+clean; 126 GiB data, 5.7 GiB used, 710 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.2 MiB/s wr, 4 op/s 2024-04-07T16:45:00.902 INFO:tasks.workunit.client.0.smithi100.stdout: 372: throughput=4.99MB/sec pending data=0 2024-04-07T16:45:00.903 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Bd-ZeJ4v1xg5gcB off=1244167 len=1978675 2024-04-07T16:45:00.917 INFO:tasks.workunit.client.0.smithi100.stdout:op 1968 completed, throughput=5MB/sec 2024-04-07T16:45:00.918 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-yl-ZzDpBEpuc3Zb off=3760132 len=1151263 2024-04-07T16:45:00.920 INFO:tasks.workunit.client.0.smithi100.stdout:op 1969 completed, throughput=5MB/sec 2024-04-07T16:45:00.920 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-0MzdMMLI7YJ6-h7 off=19739295 len=151354 2024-04-07T16:45:00.921 INFO:tasks.workunit.client.0.smithi100.stdout:op 1970 completed, throughput=5MB/sec 2024-04-07T16:45:00.921 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-6kudbTUfukRlFS0 off=2164886 len=1111209 2024-04-07T16:45:00.931 INFO:tasks.workunit.client.0.smithi100.stdout:op 1971 completed, throughput=5MB/sec 2024-04-07T16:45:00.943 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:45:00 smithi100 bash[74942]: cluster 2024-04-07T16:44:59.273915+0000 mgr.y (mgr.25175) 889 : cluster [DBG] pgmap v1531: 105 pgs: 105 active+clean; 126 GiB data, 5.7 GiB used, 710 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.2 MiB/s wr, 4 op/s 2024-04-07T16:45:00.943 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:45:00 smithi100 bash[76202]: cluster 2024-04-07T16:44:59.273915+0000 mgr.y (mgr.25175) 889 : cluster [DBG] pgmap v1531: 105 pgs: 105 active+clean; 126 GiB data, 5.7 GiB used, 710 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.2 MiB/s wr, 4 op/s 2024-04-07T16:45:01.931 INFO:tasks.workunit.client.0.smithi100.stdout: 373: throughput=4.99MB/sec pending data=0 2024-04-07T16:45:01.932 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-CD6mzNAiBeaXL_3 off=0 len=1582653 2024-04-07T16:45:01.944 INFO:tasks.workunit.client.0.smithi100.stdout:op 1972 completed, throughput=4.99MB/sec 2024-04-07T16:45:01.944 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-CAHz-N3nRUxfmRQ off=22731019 len=127412 2024-04-07T16:45:01.949 INFO:tasks.workunit.client.0.smithi100.stdout:op 1973 completed, throughput=4.99MB/sec 2024-04-07T16:45:01.949 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-J2yt0yWmOls94fv off=10285701 len=827864 2024-04-07T16:45:01.958 INFO:tasks.workunit.client.0.smithi100.stdout:op 1974 completed, throughput=4.99MB/sec 2024-04-07T16:45:01.958 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-xcbx8TuQVZHfpa7 off=9757805 len=1720173 2024-04-07T16:45:01.971 INFO:tasks.workunit.client.0.smithi100.stdout:op 1975 completed, throughput=5MB/sec 2024-04-07T16:45:01.971 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-C1zGKT1SDqj6KeT off=3855238 len=1451980 2024-04-07T16:45:01.973 INFO:tasks.workunit.client.0.smithi100.stdout:op 1976 completed, throughput=5MB/sec 2024-04-07T16:45:02.573 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:45:02 smithi144 bash[95233]: ts=2024-04-07T16:45:02.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:45:02.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:45:02 smithi144 bash[66654]: cluster 2024-04-07T16:45:01.274843+0000 mgr.y (mgr.25175) 890 : cluster [DBG] pgmap v1532: 105 pgs: 105 active+clean; 126 GiB data, 5.7 GiB used, 710 GiB / 715 GiB avail; 2.3 MiB/s rd, 2.5 MiB/s wr, 5 op/s 2024-04-07T16:45:02.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:45:02 smithi144 bash[66654]: audit 2024-04-07T16:45:01.765955+0000 mon.a (mon.0) 2101 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:45:02.943 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:45:02 smithi100 bash[74942]: cluster 2024-04-07T16:45:01.274843+0000 mgr.y (mgr.25175) 890 : cluster [DBG] pgmap v1532: 105 pgs: 105 active+clean; 126 GiB data, 5.7 GiB used, 710 GiB / 715 GiB avail; 2.3 MiB/s rd, 2.5 MiB/s wr, 5 op/s 2024-04-07T16:45:02.943 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:45:02 smithi100 bash[74942]: audit 2024-04-07T16:45:01.765955+0000 mon.a (mon.0) 2101 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:45:02.944 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:45:02 smithi100 bash[76202]: cluster 2024-04-07T16:45:01.274843+0000 mgr.y (mgr.25175) 890 : cluster [DBG] pgmap v1532: 105 pgs: 105 active+clean; 126 GiB data, 5.7 GiB used, 710 GiB / 715 GiB avail; 2.3 MiB/s rd, 2.5 MiB/s wr, 5 op/s 2024-04-07T16:45:02.944 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:45:02 smithi100 bash[76202]: audit 2024-04-07T16:45:01.765955+0000 mon.a (mon.0) 2101 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:45:02.974 INFO:tasks.workunit.client.0.smithi100.stdout: 374: throughput=4.99MB/sec pending data=0 2024-04-07T16:45:02.974 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Gn_HgLnwgkBsJiF off=6975919 len=650320 2024-04-07T16:45:02.976 INFO:tasks.workunit.client.0.smithi100.stdout:op 1977 completed, throughput=4.99MB/sec 2024-04-07T16:45:02.976 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-p1QuEtUkkbQminD off=1408486 len=895837 2024-04-07T16:45:02.984 INFO:tasks.workunit.client.0.smithi100.stdout:op 1978 completed, throughput=4.99MB/sec 2024-04-07T16:45:02.984 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-fcVeikA9MdmlNCc off=6013347 len=571077 2024-04-07T16:45:02.991 INFO:tasks.workunit.client.0.smithi100.stdout:op 1979 completed, throughput=4.99MB/sec 2024-04-07T16:45:02.991 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-fbB3rbh8M3xFe6L off=1573749 len=346420 2024-04-07T16:45:02.993 INFO:tasks.workunit.client.0.smithi100.stdout:op 1980 completed, throughput=4.99MB/sec 2024-04-07T16:45:02.993 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-dS408lnQrcc6wtM off=9430090 len=1487349 2024-04-07T16:45:02.998 INFO:tasks.workunit.client.0.smithi100.stdout:op 1981 completed, throughput=5MB/sec 2024-04-07T16:45:02.998 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-ipnSnwkoHUUjNOA off=13957942 len=779713 2024-04-07T16:45:03.000 INFO:tasks.workunit.client.0.smithi100.stdout:op 1982 completed, throughput=5MB/sec 2024-04-07T16:45:04.001 INFO:tasks.workunit.client.0.smithi100.stdout: 375: throughput=4.99MB/sec pending data=0 2024-04-07T16:45:04.001 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-q8nloFRFcn3W20T off=3173419 len=1672077 2024-04-07T16:45:04.006 INFO:tasks.workunit.client.0.smithi100.stdout:op 1983 completed, throughput=4.99MB/sec 2024-04-07T16:45:04.007 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-OxXTGFTFSIH5oqJ off=2214993 len=284724 2024-04-07T16:45:04.008 INFO:tasks.workunit.client.0.smithi100.stdout:op 1984 completed, throughput=4.99MB/sec 2024-04-07T16:45:04.008 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-5kjOasgIyDxnyeL off=8228063 len=762622 2024-04-07T16:45:04.010 INFO:tasks.workunit.client.0.smithi100.stdout:op 1985 completed, throughput=4.99MB/sec 2024-04-07T16:45:04.010 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-DsabGl7vL2FefaV off=17256263 len=411015 2024-04-07T16:45:04.012 INFO:tasks.workunit.client.0.smithi100.stdout:op 1986 completed, throughput=4.99MB/sec 2024-04-07T16:45:04.012 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-wBDREaLw26Rocnn off=1731007 len=322732 2024-04-07T16:45:04.018 INFO:tasks.workunit.client.0.smithi100.stdout:op 1987 completed, throughput=5MB/sec 2024-04-07T16:45:04.018 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-WzFA43WwSlarsxs off=1317439 len=1018599 2024-04-07T16:45:04.027 INFO:tasks.workunit.client.0.smithi100.stdout:op 1988 completed, throughput=5MB/sec 2024-04-07T16:45:04.027 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-4lWZSMFYCSoC0KU off=2210742 len=262228 2024-04-07T16:45:04.033 INFO:tasks.workunit.client.0.smithi100.stdout:op 1989 completed, throughput=5MB/sec 2024-04-07T16:45:04.033 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-V2YxaWFzRUaxukB off=7264863 len=1923203 2024-04-07T16:45:04.037 INFO:tasks.workunit.client.0.smithi100.stdout:op 1990 completed, throughput=5MB/sec 2024-04-07T16:45:04.857 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:45:04 smithi144 bash[66654]: cluster 2024-04-07T16:45:03.276027+0000 mgr.y (mgr.25175) 891 : cluster [DBG] pgmap v1533: 105 pgs: 105 active+clean; 126 GiB data, 5.7 GiB used, 709 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.8 MiB/s wr, 5 op/s 2024-04-07T16:45:04.943 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:45:04 smithi100 bash[74942]: cluster 2024-04-07T16:45:03.276027+0000 mgr.y (mgr.25175) 891 : cluster [DBG] pgmap v1533: 105 pgs: 105 active+clean; 126 GiB data, 5.7 GiB used, 709 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.8 MiB/s wr, 5 op/s 2024-04-07T16:45:04.943 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:45:04 smithi100 bash[76202]: cluster 2024-04-07T16:45:03.276027+0000 mgr.y (mgr.25175) 891 : cluster [DBG] pgmap v1533: 105 pgs: 105 active+clean; 126 GiB data, 5.7 GiB used, 709 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.8 MiB/s wr, 5 op/s 2024-04-07T16:45:05.038 INFO:tasks.workunit.client.0.smithi100.stdout: 376: throughput=4.99MB/sec pending data=0 2024-04-07T16:45:05.038 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-bdd4q1ibUFXfCWI off=1096731 len=598643 2024-04-07T16:45:05.040 INFO:tasks.workunit.client.0.smithi100.stdout:op 1991 completed, throughput=4.99MB/sec 2024-04-07T16:45:05.040 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-qnfsQIIBgoM3J-z off=6591508 len=295573 2024-04-07T16:45:05.046 INFO:tasks.workunit.client.0.smithi100.stdout:op 1992 completed, throughput=4.99MB/sec 2024-04-07T16:45:05.046 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-m3rBK6kwUxlaXX5 off=6520795 len=250049 2024-04-07T16:45:05.047 INFO:tasks.workunit.client.0.smithi100.stdout:op 1993 completed, throughput=4.99MB/sec 2024-04-07T16:45:05.047 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-oqJ4FGBuYVKWLOz off=3749396 len=1546332 2024-04-07T16:45:05.059 INFO:tasks.workunit.client.0.smithi100.stdout:op 1994 completed, throughput=5MB/sec 2024-04-07T16:45:05.059 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-soiNeMXm9F5vFNp off=19334429 len=1036053 2024-04-07T16:45:05.062 INFO:tasks.workunit.client.0.smithi100.stdout:op 1995 completed, throughput=5MB/sec 2024-04-07T16:45:05.062 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-PobUf3-u8YWsAIN off=2852822 len=788286 2024-04-07T16:45:05.070 INFO:tasks.workunit.client.0.smithi100.stdout:op 1996 completed, throughput=5MB/sec 2024-04-07T16:45:06.070 INFO:tasks.workunit.client.0.smithi100.stdout: 377: throughput=4.99MB/sec pending data=0 2024-04-07T16:45:06.071 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-f_KIPwt2Kk9KUyL off=1280825 len=1690290 2024-04-07T16:45:06.083 INFO:tasks.workunit.client.0.smithi100.stdout:op 1997 completed, throughput=4.99MB/sec 2024-04-07T16:45:06.083 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-vWg1EWlPPj5lkag off=9676753 len=1276073 2024-04-07T16:45:06.085 INFO:tasks.workunit.client.0.smithi100.stdout:op 1998 completed, throughput=5MB/sec 2024-04-07T16:45:06.085 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-LzOQ-6P0HCv37DF off=2644430 len=734811 2024-04-07T16:45:06.093 INFO:tasks.workunit.client.0.smithi100.stdout:op 1999 completed, throughput=5MB/sec 2024-04-07T16:45:06.093 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-s2HPKrgCe67wSi_ off=9310451 len=836342 2024-04-07T16:45:06.095 INFO:tasks.workunit.client.0.smithi100.stdout:op 2000 completed, throughput=5MB/sec 2024-04-07T16:45:06.095 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-zS7iN_fTYpsSERH off=15367843 len=1528929 2024-04-07T16:45:06.098 INFO:tasks.workunit.client.0.smithi100.stdout:op 2001 completed, throughput=5MB/sec 2024-04-07T16:45:06.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:45:06 smithi144 bash[66654]: cluster 2024-04-07T16:45:05.277038+0000 mgr.y (mgr.25175) 892 : cluster [DBG] pgmap v1534: 105 pgs: 105 active+clean; 126 GiB data, 5.7 GiB used, 709 GiB / 715 GiB avail; 2.4 MiB/s rd, 2.8 MiB/s wr, 5 op/s 2024-04-07T16:45:06.943 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:45:06 smithi100 bash[74942]: cluster 2024-04-07T16:45:05.277038+0000 mgr.y (mgr.25175) 892 : cluster [DBG] pgmap v1534: 105 pgs: 105 active+clean; 126 GiB data, 5.7 GiB used, 709 GiB / 715 GiB avail; 2.4 MiB/s rd, 2.8 MiB/s wr, 5 op/s 2024-04-07T16:45:06.943 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:45:06 smithi100 bash[76202]: cluster 2024-04-07T16:45:05.277038+0000 mgr.y (mgr.25175) 892 : cluster [DBG] pgmap v1534: 105 pgs: 105 active+clean; 126 GiB data, 5.7 GiB used, 709 GiB / 715 GiB avail; 2.4 MiB/s rd, 2.8 MiB/s wr, 5 op/s 2024-04-07T16:45:07.098 INFO:tasks.workunit.client.0.smithi100.stdout: 378: throughput=4.99MB/sec pending data=0 2024-04-07T16:45:07.098 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Px1dcUOA1fv4nVA off=6215039 len=842600 2024-04-07T16:45:07.101 INFO:tasks.workunit.client.0.smithi100.stdout:op 2002 completed, throughput=4.99MB/sec 2024-04-07T16:45:07.101 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-1SsRNrUtTNsQUnw off=9278901 len=1559306 2024-04-07T16:45:07.106 INFO:tasks.workunit.client.0.smithi100.stdout:op 2003 completed, throughput=5MB/sec 2024-04-07T16:45:07.106 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-tkbikMxiwmgmoBB off=7664454 len=99520 2024-04-07T16:45:07.107 INFO:tasks.workunit.client.0.smithi100.stdout:op 2004 completed, throughput=5MB/sec 2024-04-07T16:45:07.107 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj--_Vm7FqcngB3M9e off=342390 len=1907145 2024-04-07T16:45:07.120 INFO:tasks.workunit.client.0.smithi100.stdout:op 2005 completed, throughput=5MB/sec 2024-04-07T16:45:08.121 INFO:tasks.workunit.client.0.smithi100.stdout: 379: throughput=4.99MB/sec pending data=0 2024-04-07T16:45:08.121 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj--UrZX5dIqIKSQH7 off=22722772 len=1226839 2024-04-07T16:45:08.133 INFO:tasks.workunit.client.0.smithi100.stdout:op 2006 completed, throughput=4.99MB/sec 2024-04-07T16:45:08.133 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-ndglBuWrlDPZCv4 off=10458364 len=1385261 2024-04-07T16:45:08.144 INFO:tasks.workunit.client.0.smithi100.stdout:op 2007 completed, throughput=4.99MB/sec 2024-04-07T16:45:08.144 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-dxgErCEwqrl2PRC off=5468241 len=1202215 2024-04-07T16:45:08.153 INFO:tasks.workunit.client.0.smithi100.stdout:op 2008 completed, throughput=5MB/sec 2024-04-07T16:45:08.153 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-xpFT_47Stm-FsYz off=749710 len=1307601 2024-04-07T16:45:08.162 INFO:tasks.workunit.client.0.smithi100.stdout:op 2009 completed, throughput=5MB/sec 2024-04-07T16:45:08.162 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-XltyXuyZGAb4tda off=1097546 len=143056 2024-04-07T16:45:08.163 INFO:tasks.workunit.client.0.smithi100.stdout:op 2010 completed, throughput=5MB/sec 2024-04-07T16:45:08.163 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-q7FkhM6eahI-zsY off=2973484 len=794645 2024-04-07T16:45:08.171 INFO:tasks.workunit.client.0.smithi100.stdout:op 2011 completed, throughput=5MB/sec 2024-04-07T16:45:08.693 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:45:08 smithi100 bash[74942]: cluster 2024-04-07T16:45:07.278088+0000 mgr.y (mgr.25175) 893 : cluster [DBG] pgmap v1535: 105 pgs: 105 active+clean; 126 GiB data, 5.8 GiB used, 709 GiB / 715 GiB avail; 2.2 MiB/s rd, 2.5 MiB/s wr, 4 op/s 2024-04-07T16:45:08.693 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:45:08 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:45:08] "GET /metrics HTTP/1.1" 200 34070 "" "Prometheus/2.43.0" 2024-04-07T16:45:08.694 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:45:08 smithi100 bash[76202]: cluster 2024-04-07T16:45:07.278088+0000 mgr.y (mgr.25175) 893 : cluster [DBG] pgmap v1535: 105 pgs: 105 active+clean; 126 GiB data, 5.8 GiB used, 709 GiB / 715 GiB avail; 2.2 MiB/s rd, 2.5 MiB/s wr, 4 op/s 2024-04-07T16:45:09.108 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:45:08 smithi144 bash[66654]: cluster 2024-04-07T16:45:07.278088+0000 mgr.y (mgr.25175) 893 : cluster [DBG] pgmap v1535: 105 pgs: 105 active+clean; 126 GiB data, 5.8 GiB used, 709 GiB / 715 GiB avail; 2.2 MiB/s rd, 2.5 MiB/s wr, 4 op/s 2024-04-07T16:45:09.172 INFO:tasks.workunit.client.0.smithi100.stdout: 380: throughput=4.99MB/sec pending data=0 2024-04-07T16:45:09.172 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-TN2x0-YSbCfuhcB off=6304567 len=1274373 2024-04-07T16:45:09.183 INFO:tasks.workunit.client.0.smithi100.stdout:op 2012 completed, throughput=4.99MB/sec 2024-04-07T16:45:09.183 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-9A3WWgoX9JWPmdN off=342860 len=106970 2024-04-07T16:45:09.184 INFO:tasks.workunit.client.0.smithi100.stdout:op 2013 completed, throughput=4.99MB/sec 2024-04-07T16:45:09.184 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-X1TkE_QSiXyVOI1 off=625991 len=607434 2024-04-07T16:45:09.190 INFO:tasks.workunit.client.0.smithi100.stdout:op 2014 completed, throughput=4.99MB/sec 2024-04-07T16:45:09.191 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-EPlHnxfHYoFUdWV off=0 len=1325613 2024-04-07T16:45:09.194 INFO:tasks.workunit.client.0.smithi100.stdout:op 2015 completed, throughput=5MB/sec 2024-04-07T16:45:09.194 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-ZIDYhr8SpN5nIlP off=6673118 len=628644 2024-04-07T16:45:09.201 INFO:tasks.workunit.client.0.smithi100.stdout:op 2016 completed, throughput=5MB/sec 2024-04-07T16:45:09.201 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Km3Oj37e_6gw1kQ off=968896 len=298376 2024-04-07T16:45:09.206 INFO:tasks.workunit.client.0.smithi100.stdout:op 2017 completed, throughput=5MB/sec 2024-04-07T16:45:09.206 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-s8JPqa_lN2MxdZw off=12777692 len=688252 2024-04-07T16:45:09.208 INFO:tasks.workunit.client.0.smithi100.stdout:op 2018 completed, throughput=5MB/sec 2024-04-07T16:45:09.943 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:45:09 smithi100 bash[76202]: cluster 2024-04-07T16:45:09.278903+0000 mgr.y (mgr.25175) 894 : cluster [DBG] pgmap v1536: 105 pgs: 105 active+clean; 126 GiB data, 5.8 GiB used, 709 GiB / 715 GiB avail; 2.0 MiB/s rd, 2.5 MiB/s wr, 4 op/s 2024-04-07T16:45:09.943 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:45:09 smithi100 bash[74942]: cluster 2024-04-07T16:45:09.278903+0000 mgr.y (mgr.25175) 894 : cluster [DBG] pgmap v1536: 105 pgs: 105 active+clean; 126 GiB data, 5.8 GiB used, 709 GiB / 715 GiB avail; 2.0 MiB/s rd, 2.5 MiB/s wr, 4 op/s 2024-04-07T16:45:10.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:45:09 smithi144 bash[66654]: cluster 2024-04-07T16:45:09.278903+0000 mgr.y (mgr.25175) 894 : cluster [DBG] pgmap v1536: 105 pgs: 105 active+clean; 126 GiB data, 5.8 GiB used, 709 GiB / 715 GiB avail; 2.0 MiB/s rd, 2.5 MiB/s wr, 4 op/s 2024-04-07T16:45:10.209 INFO:tasks.workunit.client.0.smithi100.stdout: 381: throughput=4.99MB/sec pending data=0 2024-04-07T16:45:10.209 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-utDkjYJVuQzq1mj off=5717562 len=1583005 2024-04-07T16:45:10.222 INFO:tasks.workunit.client.0.smithi100.stdout:op 2019 completed, throughput=4.99MB/sec 2024-04-07T16:45:10.222 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-LpyS2DrEccetzYc off=4580880 len=512031 2024-04-07T16:45:10.224 INFO:tasks.workunit.client.0.smithi100.stdout:op 2020 completed, throughput=4.99MB/sec 2024-04-07T16:45:10.224 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-piMNDQe373BR7R- off=5154134 len=509668 2024-04-07T16:45:10.226 INFO:tasks.workunit.client.0.smithi100.stdout:op 2021 completed, throughput=4.99MB/sec 2024-04-07T16:45:10.226 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-7_2gXMG7Cy57dPa off=70351 len=1264406 2024-04-07T16:45:10.237 INFO:tasks.workunit.client.0.smithi100.stdout:op 2022 completed, throughput=5MB/sec 2024-04-07T16:45:10.237 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-BgUagh3dXFQnrKQ off=2698724 len=328838 2024-04-07T16:45:10.239 INFO:tasks.workunit.client.0.smithi100.stdout:op 2023 completed, throughput=5MB/sec 2024-04-07T16:45:10.239 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-KolQg1K0goKt-fn off=525718 len=1194557 2024-04-07T16:45:10.242 INFO:tasks.workunit.client.0.smithi100.stdout:op 2024 completed, throughput=5MB/sec 2024-04-07T16:45:11.242 INFO:tasks.workunit.client.0.smithi100.stdout: 382: throughput=4.99MB/sec pending data=0 2024-04-07T16:45:11.243 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-bQECONGChzvIcfb off=1183894 len=634960 2024-04-07T16:45:11.249 INFO:tasks.workunit.client.0.smithi100.stdout:op 2025 completed, throughput=4.99MB/sec 2024-04-07T16:45:11.249 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-p1ajTpksI_ooCJF off=8050322 len=1419808 2024-04-07T16:45:11.262 INFO:tasks.workunit.client.0.smithi100.stdout:op 2026 completed, throughput=4.99MB/sec 2024-04-07T16:45:11.263 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-u8GL934nJQdgOhw off=4863409 len=1832938 2024-04-07T16:45:11.267 INFO:tasks.workunit.client.0.smithi100.stdout:op 2027 completed, throughput=5MB/sec 2024-04-07T16:45:11.267 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-oPfSneiUp30k3a1 off=21439524 len=1608025 2024-04-07T16:45:11.278 INFO:tasks.workunit.client.0.smithi100.stdout:op 2028 completed, throughput=5MB/sec 2024-04-07T16:45:12.279 INFO:tasks.workunit.client.0.smithi100.stdout: 383: throughput=4.99MB/sec pending data=0 2024-04-07T16:45:12.279 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-QXIOhcwNp6Snlen off=0 len=1478359 2024-04-07T16:45:12.290 INFO:tasks.workunit.client.0.smithi100.stdout:op 2029 completed, throughput=4.99MB/sec 2024-04-07T16:45:12.290 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-JzLohjsiVFo3e3h off=1607501 len=1499006 2024-04-07T16:45:12.294 INFO:tasks.workunit.client.0.smithi100.stdout:op 2030 completed, throughput=4.99MB/sec 2024-04-07T16:45:12.294 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-pvO_GzV4dxnC7Ah off=723778 len=1620426 2024-04-07T16:45:12.305 INFO:tasks.workunit.client.0.smithi100.stdout:op 2031 completed, throughput=5MB/sec 2024-04-07T16:45:12.305 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-lwMinEgw_0afouK off=290476 len=1554064 2024-04-07T16:45:12.311 INFO:tasks.workunit.client.0.smithi100.stdout:op 2032 completed, throughput=5MB/sec 2024-04-07T16:45:12.608 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:45:12 smithi144 bash[66654]: cluster 2024-04-07T16:45:11.279877+0000 mgr.y (mgr.25175) 895 : cluster [DBG] pgmap v1537: 105 pgs: 105 active+clean; 126 GiB data, 5.8 GiB used, 709 GiB / 715 GiB avail; 2.0 MiB/s rd, 2.9 MiB/s wr, 5 op/s 2024-04-07T16:45:12.608 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:45:12 smithi144 bash[95233]: ts=2024-04-07T16:45:12.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:45:12.693 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:45:12 smithi100 bash[74942]: cluster 2024-04-07T16:45:11.279877+0000 mgr.y (mgr.25175) 895 : cluster [DBG] pgmap v1537: 105 pgs: 105 active+clean; 126 GiB data, 5.8 GiB used, 709 GiB / 715 GiB avail; 2.0 MiB/s rd, 2.9 MiB/s wr, 5 op/s 2024-04-07T16:45:12.693 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:45:12 smithi100 bash[76202]: cluster 2024-04-07T16:45:11.279877+0000 mgr.y (mgr.25175) 895 : cluster [DBG] pgmap v1537: 105 pgs: 105 active+clean; 126 GiB data, 5.8 GiB used, 709 GiB / 715 GiB avail; 2.0 MiB/s rd, 2.9 MiB/s wr, 5 op/s 2024-04-07T16:45:13.311 INFO:tasks.workunit.client.0.smithi100.stdout: 384: throughput=4.99MB/sec pending data=0 2024-04-07T16:45:13.311 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Ot0W7It997-URzJ off=1079334 len=274206 2024-04-07T16:45:13.313 INFO:tasks.workunit.client.0.smithi100.stdout:op 2033 completed, throughput=4.99MB/sec 2024-04-07T16:45:13.314 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-QrARshO0hwvmosE off=20972196 len=500130 2024-04-07T16:45:13.315 INFO:tasks.workunit.client.0.smithi100.stdout:op 2034 completed, throughput=4.99MB/sec 2024-04-07T16:45:13.316 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-I6QBJqq6yAe2yAR off=18416978 len=1822412 2024-04-07T16:45:13.333 INFO:tasks.workunit.client.0.smithi100.stdout:op 2035 completed, throughput=5MB/sec 2024-04-07T16:45:13.333 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-DWi3asFnBhGy0Ss off=38798 len=859129 2024-04-07T16:45:13.341 INFO:tasks.workunit.client.0.smithi100.stdout:op 2036 completed, throughput=5MB/sec 2024-04-07T16:45:13.341 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Im_LMPBAxY4L7UZ off=9220284 len=121661 2024-04-07T16:45:13.342 INFO:tasks.workunit.client.0.smithi100.stdout:op 2037 completed, throughput=5MB/sec 2024-04-07T16:45:13.342 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-wntJ3wBVqqmvT1P off=0 len=1221533 2024-04-07T16:45:13.345 INFO:tasks.workunit.client.0.smithi100.stdout:op 2038 completed, throughput=5MB/sec 2024-04-07T16:45:14.345 INFO:tasks.workunit.client.0.smithi100.stdout: 385: throughput=4.99MB/sec pending data=0 2024-04-07T16:45:14.346 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-D7pTa-sgZRknCQz off=2066636 len=927140 2024-04-07T16:45:14.355 INFO:tasks.workunit.client.0.smithi100.stdout:op 2039 completed, throughput=4.99MB/sec 2024-04-07T16:45:14.355 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-mIjNZba3ykBsi0E off=9108850 len=554599 2024-04-07T16:45:14.362 INFO:tasks.workunit.client.0.smithi100.stdout:op 2040 completed, throughput=4.99MB/sec 2024-04-07T16:45:14.362 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-qppm9mZyF0mxbl7 off=12676255 len=196874 2024-04-07T16:45:14.371 INFO:tasks.workunit.client.0.smithi100.stdout:op 2041 completed, throughput=4.99MB/sec 2024-04-07T16:45:14.371 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-jbsFGWs0T0iL6kJ off=9140419 len=248171 2024-04-07T16:45:14.371 INFO:tasks.workunit.client.0.smithi100.stdout:op 2042 completed, throughput=4.99MB/sec 2024-04-07T16:45:14.371 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-SMaUl_SE5GKqERk off=1942614 len=1455972 2024-04-07T16:45:14.380 INFO:tasks.workunit.client.0.smithi100.stdout:op 2043 completed, throughput=5MB/sec 2024-04-07T16:45:14.380 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-r_tYmSz7EAG_88y off=19716 len=774526 2024-04-07T16:45:14.382 INFO:tasks.workunit.client.0.smithi100.stdout:op 2044 completed, throughput=5MB/sec 2024-04-07T16:45:14.382 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-jdBnikA6OTS-_C1 off=10287887 len=1348477 2024-04-07T16:45:14.386 INFO:tasks.workunit.client.0.smithi100.stdout:op 2045 completed, throughput=5MB/sec 2024-04-07T16:45:14.608 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:45:14 smithi144 bash[66654]: cluster 2024-04-07T16:45:13.280916+0000 mgr.y (mgr.25175) 896 : cluster [DBG] pgmap v1538: 105 pgs: 105 active+clean; 126 GiB data, 5.8 GiB used, 709 GiB / 715 GiB avail; 2.1 MiB/s rd, 2.7 MiB/s wr, 5 op/s 2024-04-07T16:45:14.693 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:45:14 smithi100 bash[74942]: cluster 2024-04-07T16:45:13.280916+0000 mgr.y (mgr.25175) 896 : cluster [DBG] pgmap v1538: 105 pgs: 105 active+clean; 126 GiB data, 5.8 GiB used, 709 GiB / 715 GiB avail; 2.1 MiB/s rd, 2.7 MiB/s wr, 5 op/s 2024-04-07T16:45:14.693 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:45:14 smithi100 bash[76202]: cluster 2024-04-07T16:45:13.280916+0000 mgr.y (mgr.25175) 896 : cluster [DBG] pgmap v1538: 105 pgs: 105 active+clean; 126 GiB data, 5.8 GiB used, 709 GiB / 715 GiB avail; 2.1 MiB/s rd, 2.7 MiB/s wr, 5 op/s 2024-04-07T16:45:15.386 INFO:tasks.workunit.client.0.smithi100.stdout: 386: throughput=4.99MB/sec pending data=0 2024-04-07T16:45:15.386 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-P5xyO9BgP-YfqME off=2483007 len=426473 2024-04-07T16:45:15.389 INFO:tasks.workunit.client.0.smithi100.stdout:op 2046 completed, throughput=4.99MB/sec 2024-04-07T16:45:15.389 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-7oIklmiTEqg4nBU off=8436542 len=952639 2024-04-07T16:45:15.392 INFO:tasks.workunit.client.0.smithi100.stdout:op 2047 completed, throughput=4.99MB/sec 2024-04-07T16:45:15.392 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Z1CPKge4zT8YZjG off=1998582 len=853802 2024-04-07T16:45:15.401 INFO:tasks.workunit.client.0.smithi100.stdout:op 2048 completed, throughput=4.99MB/sec 2024-04-07T16:45:15.401 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-7TzB_YCecNUC0-I off=10261712 len=1576847 2024-04-07T16:45:15.414 INFO:tasks.workunit.client.0.smithi100.stdout:op 2049 completed, throughput=5MB/sec 2024-04-07T16:45:15.414 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-5tJQAOguQ3yKSdE off=17034845 len=1659320 2024-04-07T16:45:15.421 INFO:tasks.workunit.client.0.smithi100.stdout:op 2050 completed, throughput=5MB/sec 2024-04-07T16:45:16.421 INFO:tasks.workunit.client.0.smithi100.stdout: 387: throughput=4.99MB/sec pending data=0 2024-04-07T16:45:16.422 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-VVD_Ni_LI-5tuXP off=2069783 len=1521041 2024-04-07T16:45:16.425 INFO:tasks.workunit.client.0.smithi100.stdout:op 2051 completed, throughput=4.99MB/sec 2024-04-07T16:45:16.426 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Bk-UlzHEywt2UzZ off=19517064 len=1657812 2024-04-07T16:45:16.438 INFO:tasks.workunit.client.0.smithi100.stdout:op 2052 completed, throughput=5MB/sec 2024-04-07T16:45:16.438 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-rppoAvAt83zXJxk off=11377945 len=1805173 2024-04-07T16:45:16.444 INFO:tasks.workunit.client.0.smithi100.stdout:op 2053 completed, throughput=5MB/sec 2024-04-07T16:45:16.444 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-UyhvVYTsln4wQuq off=9024245 len=682908 2024-04-07T16:45:16.451 INFO:tasks.workunit.client.0.smithi100.stdout:op 2054 completed, throughput=5MB/sec 2024-04-07T16:45:16.608 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:45:16 smithi144 bash[66654]: cluster 2024-04-07T16:45:15.281937+0000 mgr.y (mgr.25175) 897 : cluster [DBG] pgmap v1539: 105 pgs: 105 active+clean; 126 GiB data, 5.8 GiB used, 709 GiB / 715 GiB avail; 2.3 MiB/s rd, 2.4 MiB/s wr, 5 op/s 2024-04-07T16:45:16.694 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:45:16 smithi100 bash[74942]: cluster 2024-04-07T16:45:15.281937+0000 mgr.y (mgr.25175) 897 : cluster [DBG] pgmap v1539: 105 pgs: 105 active+clean; 126 GiB data, 5.8 GiB used, 709 GiB / 715 GiB avail; 2.3 MiB/s rd, 2.4 MiB/s wr, 5 op/s 2024-04-07T16:45:16.694 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:45:16 smithi100 bash[76202]: cluster 2024-04-07T16:45:15.281937+0000 mgr.y (mgr.25175) 897 : cluster [DBG] pgmap v1539: 105 pgs: 105 active+clean; 126 GiB data, 5.8 GiB used, 709 GiB / 715 GiB avail; 2.3 MiB/s rd, 2.4 MiB/s wr, 5 op/s 2024-04-07T16:45:17.452 INFO:tasks.workunit.client.0.smithi100.stdout: 388: throughput=4.99MB/sec pending data=0 2024-04-07T16:45:17.452 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-lvaZyNWX6LBDePr off=13985266 len=533426 2024-04-07T16:45:17.453 INFO:tasks.workunit.client.0.smithi100.stdout:op 2055 completed, throughput=4.99MB/sec 2024-04-07T16:45:17.454 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-r9OeTJ4q7jh1nDb off=6265377 len=1100353 2024-04-07T16:45:17.456 INFO:tasks.workunit.client.0.smithi100.stdout:op 2056 completed, throughput=4.99MB/sec 2024-04-07T16:45:17.456 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-AhCJMNghXBVE7Kz off=640164 len=12413 2024-04-07T16:45:17.461 INFO:tasks.workunit.client.0.smithi100.stdout:op 2057 completed, throughput=4.99MB/sec 2024-04-07T16:45:17.461 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-6ikvyfobHtzEF-k off=1413156 len=1956690 2024-04-07T16:45:17.475 INFO:tasks.workunit.client.0.smithi100.stdout:op 2058 completed, throughput=5MB/sec 2024-04-07T16:45:17.475 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Qe8WVHwuVJu7nQ2 off=2631904 len=1207307 2024-04-07T16:45:17.484 INFO:tasks.workunit.client.0.smithi100.stdout:op 2059 completed, throughput=5MB/sec 2024-04-07T16:45:17.608 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:45:17 smithi144 bash[66654]: audit 2024-04-07T16:45:16.766400+0000 mon.a (mon.0) 2102 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:45:17.693 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:45:17 smithi100 bash[74942]: audit 2024-04-07T16:45:16.766400+0000 mon.a (mon.0) 2102 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:45:17.693 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:45:17 smithi100 bash[76202]: audit 2024-04-07T16:45:16.766400+0000 mon.a (mon.0) 2102 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:45:18.485 INFO:tasks.workunit.client.0.smithi100.stdout: 389: throughput=4.99MB/sec pending data=0 2024-04-07T16:45:18.485 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-dYhD8-hTwl4wY3Q off=1435202 len=1714893 2024-04-07T16:45:18.489 INFO:tasks.workunit.client.0.smithi100.stdout:op 2060 completed, throughput=4.99MB/sec 2024-04-07T16:45:18.489 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-YQP-ZiMbIkRCf2Q off=1652153 len=1907512 2024-04-07T16:45:18.502 INFO:tasks.workunit.client.0.smithi100.stdout:op 2061 completed, throughput=5MB/sec 2024-04-07T16:45:18.503 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-z-r9zV1oMYa05Il off=7726207 len=1604530 2024-04-07T16:45:18.505 INFO:tasks.workunit.client.0.smithi100.stdout:op 2062 completed, throughput=5MB/sec 2024-04-07T16:45:18.505 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-jXRGE0qs1s17qgr off=4342659 len=1795240 2024-04-07T16:45:18.521 INFO:tasks.workunit.client.0.smithi100.stdout:op 2063 completed, throughput=5MB/sec 2024-04-07T16:45:18.608 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:45:18 smithi144 bash[66654]: cluster 2024-04-07T16:45:17.283096+0000 mgr.y (mgr.25175) 898 : cluster [DBG] pgmap v1540: 105 pgs: 105 active+clean; 126 GiB data, 5.8 GiB used, 709 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.8 MiB/s wr, 5 op/s 2024-04-07T16:45:18.693 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:45:18 smithi100 bash[74942]: cluster 2024-04-07T16:45:17.283096+0000 mgr.y (mgr.25175) 898 : cluster [DBG] pgmap v1540: 105 pgs: 105 active+clean; 126 GiB data, 5.8 GiB used, 709 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.8 MiB/s wr, 5 op/s 2024-04-07T16:45:18.694 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:45:18 smithi100 bash[76202]: cluster 2024-04-07T16:45:17.283096+0000 mgr.y (mgr.25175) 898 : cluster [DBG] pgmap v1540: 105 pgs: 105 active+clean; 126 GiB data, 5.8 GiB used, 709 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.8 MiB/s wr, 5 op/s 2024-04-07T16:45:18.694 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:45:18 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:45:18] "GET /metrics HTTP/1.1" 200 34072 "" "Prometheus/2.43.0" 2024-04-07T16:45:19.521 INFO:tasks.workunit.client.0.smithi100.stdout: 390: throughput=4.99MB/sec pending data=0 2024-04-07T16:45:19.521 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-wz7FfDBEy6OWWiR off=6487233 len=198832 2024-04-07T16:45:19.523 INFO:tasks.workunit.client.0.smithi100.stdout:op 2064 completed, throughput=4.99MB/sec 2024-04-07T16:45:19.523 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-LIjXRXYjDm_Y_vp off=12555460 len=600564 2024-04-07T16:45:19.529 INFO:tasks.workunit.client.0.smithi100.stdout:op 2065 completed, throughput=4.99MB/sec 2024-04-07T16:45:19.529 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-TDeOtUdz-DY9o22 off=110759 len=16749 2024-04-07T16:45:19.534 INFO:tasks.workunit.client.0.smithi100.stdout:op 2066 completed, throughput=4.99MB/sec 2024-04-07T16:45:19.534 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-MaXZl-w6aakGsDq off=18533551 len=705244 2024-04-07T16:45:19.536 INFO:tasks.workunit.client.0.smithi100.stdout:op 2067 completed, throughput=4.99MB/sec 2024-04-07T16:45:19.536 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-C1qzYVzVjGoRVQf off=3249241 len=1871724 2024-04-07T16:45:19.553 INFO:tasks.workunit.client.0.smithi100.stdout:op 2068 completed, throughput=5MB/sec 2024-04-07T16:45:19.553 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-osd5ZHxCzieySNy off=13665346 len=683412 2024-04-07T16:45:19.560 INFO:tasks.workunit.client.0.smithi100.stdout:op 2069 completed, throughput=5MB/sec 2024-04-07T16:45:20.561 INFO:tasks.workunit.client.0.smithi100.stdout: 391: throughput=4.99MB/sec pending data=0 2024-04-07T16:45:20.561 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-yXOzQszhOSUCjrt off=10877291 len=1385524 2024-04-07T16:45:20.566 INFO:tasks.workunit.client.0.smithi100.stdout:op 2070 completed, throughput=4.99MB/sec 2024-04-07T16:45:20.566 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-yy_eKx82lGEsa8a off=781113 len=1674687 2024-04-07T16:45:20.570 INFO:tasks.workunit.client.0.smithi100.stdout:op 2071 completed, throughput=5MB/sec 2024-04-07T16:45:20.570 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-2CbkS6fU0eq0HKK off=6074284 len=2015799 2024-04-07T16:45:20.584 INFO:tasks.workunit.client.0.smithi100.stdout:op 2072 completed, throughput=5MB/sec 2024-04-07T16:45:20.585 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-0Vy34hmO0KU80AD off=2445131 len=1321405 2024-04-07T16:45:20.595 INFO:tasks.workunit.client.0.smithi100.stdout:op 2073 completed, throughput=5MB/sec 2024-04-07T16:45:20.608 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:45:20 smithi144 bash[66654]: cluster 2024-04-07T16:45:19.284001+0000 mgr.y (mgr.25175) 899 : cluster [DBG] pgmap v1541: 105 pgs: 105 active+clean; 126 GiB data, 5.8 GiB used, 709 GiB / 715 GiB avail; 2.3 MiB/s rd, 2.5 MiB/s wr, 5 op/s 2024-04-07T16:45:20.693 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:45:20 smithi100 bash[74942]: cluster 2024-04-07T16:45:19.284001+0000 mgr.y (mgr.25175) 899 : cluster [DBG] pgmap v1541: 105 pgs: 105 active+clean; 126 GiB data, 5.8 GiB used, 709 GiB / 715 GiB avail; 2.3 MiB/s rd, 2.5 MiB/s wr, 5 op/s 2024-04-07T16:45:20.693 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:45:20 smithi100 bash[76202]: cluster 2024-04-07T16:45:19.284001+0000 mgr.y (mgr.25175) 899 : cluster [DBG] pgmap v1541: 105 pgs: 105 active+clean; 126 GiB data, 5.8 GiB used, 709 GiB / 715 GiB avail; 2.3 MiB/s rd, 2.5 MiB/s wr, 5 op/s 2024-04-07T16:45:21.595 INFO:tasks.workunit.client.0.smithi100.stdout: 392: throughput=4.99MB/sec pending data=0 2024-04-07T16:45:21.595 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-6VKZNgO5Km5Ptk0 off=13821072 len=1167073 2024-04-07T16:45:21.606 INFO:tasks.workunit.client.0.smithi100.stdout:op 2074 completed, throughput=4.99MB/sec 2024-04-07T16:45:21.606 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-eKSKrOMI4ITEwTV off=20890789 len=1906499 2024-04-07T16:45:21.612 INFO:tasks.workunit.client.0.smithi100.stdout:op 2075 completed, throughput=5MB/sec 2024-04-07T16:45:21.612 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-_vx2y8JbRadrDCZ off=844025 len=1256685 2024-04-07T16:45:21.624 INFO:tasks.workunit.client.0.smithi100.stdout:op 2076 completed, throughput=5MB/sec 2024-04-07T16:45:22.608 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:45:22 smithi144 bash[66654]: cluster 2024-04-07T16:45:21.284907+0000 mgr.y (mgr.25175) 900 : cluster [DBG] pgmap v1542: 105 pgs: 105 active+clean; 126 GiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 2.4 MiB/s rd, 3.0 MiB/s wr, 5 op/s 2024-04-07T16:45:22.608 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:45:22 smithi144 bash[95233]: ts=2024-04-07T16:45:22.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:45:22.624 INFO:tasks.workunit.client.0.smithi100.stdout: 393: throughput=4.99MB/sec pending data=0 2024-04-07T16:45:22.624 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-yuDDiB935FGewra off=8168375 len=1817969 2024-04-07T16:45:22.628 INFO:tasks.workunit.client.0.smithi100.stdout:op 2077 completed, throughput=4.99MB/sec 2024-04-07T16:45:22.629 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-K4ti10hIzWiQnzN off=6006474 len=808336 2024-04-07T16:45:22.631 INFO:tasks.workunit.client.0.smithi100.stdout:op 2078 completed, throughput=4.99MB/sec 2024-04-07T16:45:22.631 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-KmzEdqrUYOSA89j off=935624 len=1098475 2024-04-07T16:45:22.636 INFO:tasks.workunit.client.0.smithi100.stdout:op 2079 completed, throughput=5MB/sec 2024-04-07T16:45:22.636 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-V17ktNfcVqtL1pz off=150733 len=1804076 2024-04-07T16:45:22.640 INFO:tasks.workunit.client.0.smithi100.stdout:op 2080 completed, throughput=5MB/sec 2024-04-07T16:45:22.693 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:45:22 smithi100 bash[74942]: cluster 2024-04-07T16:45:21.284907+0000 mgr.y (mgr.25175) 900 : cluster [DBG] pgmap v1542: 105 pgs: 105 active+clean; 126 GiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 2.4 MiB/s rd, 3.0 MiB/s wr, 5 op/s 2024-04-07T16:45:22.693 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:45:22 smithi100 bash[76202]: cluster 2024-04-07T16:45:21.284907+0000 mgr.y (mgr.25175) 900 : cluster [DBG] pgmap v1542: 105 pgs: 105 active+clean; 126 GiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 2.4 MiB/s rd, 3.0 MiB/s wr, 5 op/s 2024-04-07T16:45:23.640 INFO:tasks.workunit.client.0.smithi100.stdout: 394: throughput=4.99MB/sec pending data=0 2024-04-07T16:45:23.640 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-10ercpgJdT6Qr_i off=2818193 len=1448013 2024-04-07T16:45:23.653 INFO:tasks.workunit.client.0.smithi100.stdout:op 2081 completed, throughput=4.99MB/sec 2024-04-07T16:45:23.653 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-FWi-t8tsVGuFR-m off=2590004 len=263991 2024-04-07T16:45:23.659 INFO:tasks.workunit.client.0.smithi100.stdout:op 2082 completed, throughput=4.99MB/sec 2024-04-07T16:45:23.659 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-6UU1-AL9w5fOgZ8 off=5689393 len=729020 2024-04-07T16:45:23.661 INFO:tasks.workunit.client.0.smithi100.stdout:op 2083 completed, throughput=4.99MB/sec 2024-04-07T16:45:23.661 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Z4ISfVtYZtXwv3m off=11098316 len=1636677 2024-04-07T16:45:23.665 INFO:tasks.workunit.client.0.smithi100.stdout:op 2084 completed, throughput=5MB/sec 2024-04-07T16:45:23.665 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-FdW5jAKOxiAY9Ro off=1125122 len=1135025 2024-04-07T16:45:23.673 INFO:tasks.workunit.client.0.smithi100.stdout:op 2085 completed, throughput=5MB/sec 2024-04-07T16:45:24.674 INFO:tasks.workunit.client.0.smithi100.stdout: 395: throughput=4.99MB/sec pending data=0 2024-04-07T16:45:24.674 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-N21WD_-ElTzZfDS off=6011591 len=873769 2024-04-07T16:45:24.677 INFO:tasks.workunit.client.0.smithi100.stdout:op 2086 completed, throughput=4.99MB/sec 2024-04-07T16:45:24.677 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-1nyEuawzF9MN8ZF off=490721 len=781154 2024-04-07T16:45:24.678 INFO:tasks.workunit.client.0.smithi100.stdout:op 2087 completed, throughput=4.99MB/sec 2024-04-07T16:45:24.678 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-3wYpkPUg7hcouiG off=444946 len=2012389 2024-04-07T16:45:24.693 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:45:24 smithi100 bash[74942]: cluster 2024-04-07T16:45:23.286007+0000 mgr.y (mgr.25175) 901 : cluster [DBG] pgmap v1543: 105 pgs: 105 active+clean; 126 GiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 2.4 MiB/s rd, 3.0 MiB/s wr, 5 op/s 2024-04-07T16:45:24.693 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:45:24 smithi100 bash[76202]: cluster 2024-04-07T16:45:23.286007+0000 mgr.y (mgr.25175) 901 : cluster [DBG] pgmap v1543: 105 pgs: 105 active+clean; 126 GiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 2.4 MiB/s rd, 3.0 MiB/s wr, 5 op/s 2024-04-07T16:45:24.694 INFO:tasks.workunit.client.0.smithi100.stdout:op 2088 completed, throughput=5MB/sec 2024-04-07T16:45:24.694 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-xaqyaWCLL7orNp- off=6461718 len=731794 2024-04-07T16:45:24.700 INFO:tasks.workunit.client.0.smithi100.stdout:op 2089 completed, throughput=5MB/sec 2024-04-07T16:45:24.701 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-D5mQRdyMMUOpDPE off=8149190 len=1115636 2024-04-07T16:45:24.703 INFO:tasks.workunit.client.0.smithi100.stdout:op 2090 completed, throughput=5MB/sec 2024-04-07T16:45:24.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:45:24 smithi144 bash[66654]: cluster 2024-04-07T16:45:23.286007+0000 mgr.y (mgr.25175) 901 : cluster [DBG] pgmap v1543: 105 pgs: 105 active+clean; 126 GiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 2.4 MiB/s rd, 3.0 MiB/s wr, 5 op/s 2024-04-07T16:45:25.704 INFO:tasks.workunit.client.0.smithi100.stdout: 396: throughput=4.99MB/sec pending data=0 2024-04-07T16:45:25.704 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-O5ozYUeW3mvwK9W off=6603726 len=1995937 2024-04-07T16:45:25.709 INFO:tasks.workunit.client.0.smithi100.stdout:op 2091 completed, throughput=4.99MB/sec 2024-04-07T16:45:25.709 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-FaVuaGZ4jGe7eig off=9954046 len=1334465 2024-04-07T16:45:25.713 INFO:tasks.workunit.client.0.smithi100.stdout:op 2092 completed, throughput=5MB/sec 2024-04-07T16:45:25.713 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-mFzhL_zVra8af9e off=9456511 len=861734 2024-04-07T16:45:25.716 INFO:tasks.workunit.client.0.smithi100.stdout:op 2093 completed, throughput=5MB/sec 2024-04-07T16:45:25.716 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-_YWuBPWS4AuqrUB off=21453051 len=1876315 2024-04-07T16:45:25.723 INFO:tasks.workunit.client.0.smithi100.stdout:op 2094 completed, throughput=5MB/sec 2024-04-07T16:45:26.693 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:45:26 smithi100 bash[74942]: cluster 2024-04-07T16:45:25.286910+0000 mgr.y (mgr.25175) 902 : cluster [DBG] pgmap v1544: 105 pgs: 105 active+clean; 126 GiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.8 MiB/s wr, 5 op/s 2024-04-07T16:45:26.694 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:45:26 smithi100 bash[76202]: cluster 2024-04-07T16:45:25.286910+0000 mgr.y (mgr.25175) 902 : cluster [DBG] pgmap v1544: 105 pgs: 105 active+clean; 126 GiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.8 MiB/s wr, 5 op/s 2024-04-07T16:45:26.725 INFO:tasks.workunit.client.0.smithi100.stdout: 397: throughput=4.99MB/sec pending data=0 2024-04-07T16:45:26.725 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-S3bZcgzKa5CGQCE off=611456 len=590119 2024-04-07T16:45:26.726 INFO:tasks.workunit.client.0.smithi100.stdout:op 2095 completed, throughput=4.99MB/sec 2024-04-07T16:45:26.726 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-U3z0yas8l7SoHYA off=8784823 len=606300 2024-04-07T16:45:26.728 INFO:tasks.workunit.client.0.smithi100.stdout:op 2096 completed, throughput=4.99MB/sec 2024-04-07T16:45:26.728 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-IG0PgwqI2t2bbeS off=11972338 len=871820 2024-04-07T16:45:26.730 INFO:tasks.workunit.client.0.smithi100.stdout:op 2097 completed, throughput=4.99MB/sec 2024-04-07T16:45:26.731 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-I7T8zEwwtbvxOXK off=459312 len=280051 2024-04-07T16:45:26.736 INFO:tasks.workunit.client.0.smithi100.stdout:op 2098 completed, throughput=5MB/sec 2024-04-07T16:45:26.736 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-vRHH0goaFTvY3tf off=7280482 len=1073470 2024-04-07T16:45:26.739 INFO:tasks.workunit.client.0.smithi100.stdout:op 2099 completed, throughput=5MB/sec 2024-04-07T16:45:26.739 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-6EPKqv00L_cEWPD off=13568565 len=249080 2024-04-07T16:45:26.744 INFO:tasks.workunit.client.0.smithi100.stdout:op 2100 completed, throughput=5MB/sec 2024-04-07T16:45:26.744 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-wpY8_VNUqLbW2yD off=15226890 len=1075341 2024-04-07T16:45:26.748 INFO:tasks.workunit.client.0.smithi100.stdout:op 2101 completed, throughput=5MB/sec 2024-04-07T16:45:26.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:45:26 smithi144 bash[66654]: cluster 2024-04-07T16:45:25.286910+0000 mgr.y (mgr.25175) 902 : cluster [DBG] pgmap v1544: 105 pgs: 105 active+clean; 126 GiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.8 MiB/s wr, 5 op/s 2024-04-07T16:45:27.748 INFO:tasks.workunit.client.0.smithi100.stdout: 398: throughput=4.99MB/sec pending data=0 2024-04-07T16:45:27.749 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-JXiqgTi3JO3fbY2 off=3045760 len=24252 2024-04-07T16:45:27.753 INFO:tasks.workunit.client.0.smithi100.stdout:op 2102 completed, throughput=4.99MB/sec 2024-04-07T16:45:27.753 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-dU2LisKAc3CyW-T off=15225185 len=1374333 2024-04-07T16:45:27.757 INFO:tasks.workunit.client.0.smithi100.stdout:op 2103 completed, throughput=4.99MB/sec 2024-04-07T16:45:27.757 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-MOLA6sF1fQwSvZi off=6314980 len=242384 2024-04-07T16:45:27.758 INFO:tasks.workunit.client.0.smithi100.stdout:op 2104 completed, throughput=4.99MB/sec 2024-04-07T16:45:27.758 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-2MsD-e0gJX5yWhL off=8600142 len=1855207 2024-04-07T16:45:27.771 INFO:tasks.workunit.client.0.smithi100.stdout:op 2105 completed, throughput=5MB/sec 2024-04-07T16:45:27.772 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-wjgjJVUA7id8M9Q off=1401218 len=1986797 2024-04-07T16:45:27.785 INFO:tasks.workunit.client.0.smithi100.stdout:op 2106 completed, throughput=5MB/sec 2024-04-07T16:45:28.693 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:45:28 smithi100 bash[74942]: cluster 2024-04-07T16:45:27.288030+0000 mgr.y (mgr.25175) 903 : cluster [DBG] pgmap v1545: 105 pgs: 105 active+clean; 126 GiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 2.8 MiB/s rd, 2.7 MiB/s wr, 4 op/s 2024-04-07T16:45:28.694 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:45:28 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:45:28] "GET /metrics HTTP/1.1" 200 34072 "" "Prometheus/2.43.0" 2024-04-07T16:45:28.694 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:45:28 smithi100 bash[76202]: cluster 2024-04-07T16:45:27.288030+0000 mgr.y (mgr.25175) 903 : cluster [DBG] pgmap v1545: 105 pgs: 105 active+clean; 126 GiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 2.8 MiB/s rd, 2.7 MiB/s wr, 4 op/s 2024-04-07T16:45:28.786 INFO:tasks.workunit.client.0.smithi100.stdout: 399: throughput=4.99MB/sec pending data=0 2024-04-07T16:45:28.786 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-23wxvp1i1w-K2qd off=2506569 len=1147943 2024-04-07T16:45:28.795 INFO:tasks.workunit.client.0.smithi100.stdout:op 2107 completed, throughput=4.99MB/sec 2024-04-07T16:45:28.795 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-gQS-xMZHZlCbEUP off=714459 len=189483 2024-04-07T16:45:28.800 INFO:tasks.workunit.client.0.smithi100.stdout:op 2108 completed, throughput=4.99MB/sec 2024-04-07T16:45:28.800 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-XEM0ibpoHyijn01 off=14046466 len=1579446 2024-04-07T16:45:28.814 INFO:tasks.workunit.client.0.smithi100.stdout:op 2109 completed, throughput=5MB/sec 2024-04-07T16:45:28.814 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Ekfd53HdusO8WaL off=6349160 len=766381 2024-04-07T16:45:28.820 INFO:tasks.workunit.client.0.smithi100.stdout:op 2110 completed, throughput=5MB/sec 2024-04-07T16:45:28.821 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-quJyBIR49iRm7Iz off=855647 len=201422 2024-04-07T16:45:28.825 INFO:tasks.workunit.client.0.smithi100.stdout:op 2111 completed, throughput=5MB/sec 2024-04-07T16:45:28.826 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-czvfdHjP-MkFJaj off=1558410 len=730369 2024-04-07T16:45:28.828 INFO:tasks.workunit.client.0.smithi100.stdout:op 2112 completed, throughput=5MB/sec 2024-04-07T16:45:28.828 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-WoZT6bks-d1fVGS off=20542476 len=1566854 2024-04-07T16:45:28.831 INFO:tasks.workunit.client.0.smithi100.stdout:op 2113 completed, throughput=5MB/sec 2024-04-07T16:45:28.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:45:28 smithi144 bash[66654]: cluster 2024-04-07T16:45:27.288030+0000 mgr.y (mgr.25175) 903 : cluster [DBG] pgmap v1545: 105 pgs: 105 active+clean; 126 GiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 2.8 MiB/s rd, 2.7 MiB/s wr, 4 op/s 2024-04-07T16:45:29.832 INFO:tasks.workunit.client.0.smithi100.stdout: 400: throughput=4.99MB/sec pending data=0 2024-04-07T16:45:29.832 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj--1JRaqKzct5e8VE off=1340920 len=1131913 2024-04-07T16:45:29.842 INFO:tasks.workunit.client.0.smithi100.stdout:op 2114 completed, throughput=4.99MB/sec 2024-04-07T16:45:29.842 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-JUVXtfG6IxJXmwU off=578165 len=844012 2024-04-07T16:45:29.845 INFO:tasks.workunit.client.0.smithi100.stdout:op 2115 completed, throughput=4.99MB/sec 2024-04-07T16:45:29.845 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-aPQGhQvqJCOyeXA off=1041686 len=1927983 2024-04-07T16:45:29.861 INFO:tasks.workunit.client.0.smithi100.stdout:op 2116 completed, throughput=5MB/sec 2024-04-07T16:45:29.861 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Qi2ysdvCNiGcrL1 off=1979655 len=1080775 2024-04-07T16:45:29.864 INFO:tasks.workunit.client.0.smithi100.stdout:op 2117 completed, throughput=5MB/sec 2024-04-07T16:45:30.693 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:45:30 smithi100 bash[74942]: cluster 2024-04-07T16:45:29.288952+0000 mgr.y (mgr.25175) 904 : cluster [DBG] pgmap v1546: 105 pgs: 105 active+clean; 126 GiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.0 MiB/s wr, 3 op/s 2024-04-07T16:45:30.694 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:45:30 smithi100 bash[76202]: cluster 2024-04-07T16:45:29.288952+0000 mgr.y (mgr.25175) 904 : cluster [DBG] pgmap v1546: 105 pgs: 105 active+clean; 126 GiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.0 MiB/s wr, 3 op/s 2024-04-07T16:45:30.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:45:30 smithi144 bash[66654]: cluster 2024-04-07T16:45:29.288952+0000 mgr.y (mgr.25175) 904 : cluster [DBG] pgmap v1546: 105 pgs: 105 active+clean; 126 GiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.0 MiB/s wr, 3 op/s 2024-04-07T16:45:30.864 INFO:tasks.workunit.client.0.smithi100.stdout: 401: throughput=4.99MB/sec pending data=0 2024-04-07T16:45:30.864 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-TPrHDx8FbiRE8Mn off=4400883 len=747556 2024-04-07T16:45:30.871 INFO:tasks.workunit.client.0.smithi100.stdout:op 2118 completed, throughput=4.99MB/sec 2024-04-07T16:45:30.871 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-y8-ppP2weqBLPfL off=21535143 len=536437 2024-04-07T16:45:30.873 INFO:tasks.workunit.client.0.smithi100.stdout:op 2119 completed, throughput=4.99MB/sec 2024-04-07T16:45:30.873 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj--TfAazDmdMd5GFG off=1493397 len=1787611 2024-04-07T16:45:30.889 INFO:tasks.workunit.client.0.smithi100.stdout:op 2120 completed, throughput=5MB/sec 2024-04-07T16:45:30.889 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-PxLRw7GfGP174gp off=18577198 len=1308693 2024-04-07T16:45:30.900 INFO:tasks.workunit.client.0.smithi100.stdout:op 2121 completed, throughput=5MB/sec 2024-04-07T16:45:30.900 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-BxecSydJy0ZoSXS off=916344 len=2078841 2024-04-07T16:45:30.906 INFO:tasks.workunit.client.0.smithi100.stdout:op 2122 completed, throughput=5MB/sec 2024-04-07T16:45:31.907 INFO:tasks.workunit.client.0.smithi100.stdout: 402: throughput=4.99MB/sec pending data=0 2024-04-07T16:45:31.907 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-hVNLGrWz2476foF off=5791423 len=1616694 2024-04-07T16:45:31.911 INFO:tasks.workunit.client.0.smithi100.stdout:op 2123 completed, throughput=5MB/sec 2024-04-07T16:45:31.911 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-yLmoUcjgPlXQ0rO off=1232607 len=1445341 2024-04-07T16:45:31.921 INFO:tasks.workunit.client.0.smithi100.stdout:op 2124 completed, throughput=5MB/sec 2024-04-07T16:45:31.921 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-syKIh7dJPqyfD_8 off=299848 len=1143108 2024-04-07T16:45:31.930 INFO:tasks.workunit.client.0.smithi100.stdout:op 2125 completed, throughput=5MB/sec 2024-04-07T16:45:32.608 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:45:32 smithi144 bash[66654]: cluster 2024-04-07T16:45:31.289997+0000 mgr.y (mgr.25175) 905 : cluster [DBG] pgmap v1547: 105 pgs: 105 active+clean; 126 GiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.3 MiB/s wr, 4 op/s 2024-04-07T16:45:32.608 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:45:32 smithi144 bash[66654]: audit 2024-04-07T16:45:31.767054+0000 mon.a (mon.0) 2103 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:45:32.608 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:45:32 smithi144 bash[95233]: ts=2024-04-07T16:45:32.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:45:32.693 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:45:32 smithi100 bash[74942]: cluster 2024-04-07T16:45:31.289997+0000 mgr.y (mgr.25175) 905 : cluster [DBG] pgmap v1547: 105 pgs: 105 active+clean; 126 GiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.3 MiB/s wr, 4 op/s 2024-04-07T16:45:32.693 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:45:32 smithi100 bash[74942]: audit 2024-04-07T16:45:31.767054+0000 mon.a (mon.0) 2103 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:45:32.694 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:45:32 smithi100 bash[76202]: cluster 2024-04-07T16:45:31.289997+0000 mgr.y (mgr.25175) 905 : cluster [DBG] pgmap v1547: 105 pgs: 105 active+clean; 126 GiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.3 MiB/s wr, 4 op/s 2024-04-07T16:45:32.694 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:45:32 smithi100 bash[76202]: audit 2024-04-07T16:45:31.767054+0000 mon.a (mon.0) 2103 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:45:32.931 INFO:tasks.workunit.client.0.smithi100.stdout: 403: throughput=4.99MB/sec pending data=0 2024-04-07T16:45:32.931 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-xY_L3g3U3UJ2mrc off=10663898 len=132776 2024-04-07T16:45:32.932 INFO:tasks.workunit.client.0.smithi100.stdout:op 2126 completed, throughput=4.99MB/sec 2024-04-07T16:45:32.932 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-pnHUi6w8eZfGiDF off=3825389 len=751606 2024-04-07T16:45:32.935 INFO:tasks.workunit.client.0.smithi100.stdout:op 2127 completed, throughput=4.99MB/sec 2024-04-07T16:45:32.935 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-ND3Z-o5xc4pQx6X off=1731863 len=1402869 2024-04-07T16:45:32.948 INFO:tasks.workunit.client.0.smithi100.stdout:op 2128 completed, throughput=4.99MB/sec 2024-04-07T16:45:32.948 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-WrjJV6vU7Weic3b off=3335589 len=221953 2024-04-07T16:45:32.953 INFO:tasks.workunit.client.0.smithi100.stdout:op 2129 completed, throughput=4.99MB/sec 2024-04-07T16:45:32.953 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-7Vbg1V84VAE1SV5 off=3072168 len=658631 2024-04-07T16:45:32.956 INFO:tasks.workunit.client.0.smithi100.stdout:op 2130 completed, throughput=5MB/sec 2024-04-07T16:45:32.956 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj--xLescIGBXdFxT7 off=4774061 len=1006517 2024-04-07T16:45:32.959 INFO:tasks.workunit.client.0.smithi100.stdout:op 2131 completed, throughput=5MB/sec 2024-04-07T16:45:32.959 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-GtNx4pLUVervKWv off=16525895 len=1419230 2024-04-07T16:45:32.971 INFO:tasks.workunit.client.0.smithi100.stdout:op 2132 completed, throughput=5MB/sec 2024-04-07T16:45:33.971 INFO:tasks.workunit.client.0.smithi100.stdout: 404: throughput=4.99MB/sec pending data=0 2024-04-07T16:45:33.971 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-_YWOfHATk4_fLVD off=4382340 len=999164 2024-04-07T16:45:33.974 INFO:tasks.workunit.client.0.smithi100.stdout:op 2133 completed, throughput=4.99MB/sec 2024-04-07T16:45:33.974 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-vAL8NW_E8g74F52 off=1008921 len=1747955 2024-04-07T16:45:33.978 INFO:tasks.workunit.client.0.smithi100.stdout:op 2134 completed, throughput=5MB/sec 2024-04-07T16:45:33.978 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-m9ToSR0hYp1C8WW off=13421321 len=48522 2024-04-07T16:45:33.979 INFO:tasks.workunit.client.0.smithi100.stdout:op 2135 completed, throughput=5MB/sec 2024-04-07T16:45:33.980 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-czJ8-dqk_tF2t48 off=2554270 len=2030442 2024-04-07T16:45:33.996 INFO:tasks.workunit.client.0.smithi100.stdout:op 2136 completed, throughput=5MB/sec 2024-04-07T16:45:34.693 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:45:34 smithi100 bash[74942]: cluster 2024-04-07T16:45:33.291264+0000 mgr.y (mgr.25175) 906 : cluster [DBG] pgmap v1548: 105 pgs: 105 active+clean; 126 GiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 2.9 MiB/s rd, 2.1 MiB/s wr, 4 op/s 2024-04-07T16:45:34.694 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:45:34 smithi100 bash[76202]: cluster 2024-04-07T16:45:33.291264+0000 mgr.y (mgr.25175) 906 : cluster [DBG] pgmap v1548: 105 pgs: 105 active+clean; 126 GiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 2.9 MiB/s rd, 2.1 MiB/s wr, 4 op/s 2024-04-07T16:45:34.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:45:34 smithi144 bash[66654]: cluster 2024-04-07T16:45:33.291264+0000 mgr.y (mgr.25175) 906 : cluster [DBG] pgmap v1548: 105 pgs: 105 active+clean; 126 GiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 2.9 MiB/s rd, 2.1 MiB/s wr, 4 op/s 2024-04-07T16:45:34.996 INFO:tasks.workunit.client.0.smithi100.stdout: 405: throughput=4.99MB/sec pending data=0 2024-04-07T16:45:34.997 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-fAYuRdTrtr3ATQE off=2611350 len=561023 2024-04-07T16:45:35.004 INFO:tasks.workunit.client.0.smithi100.stdout:op 2137 completed, throughput=4.99MB/sec 2024-04-07T16:45:35.004 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-vhUA6M8urt_8uIx off=1250384 len=1597847 2024-04-07T16:45:35.018 INFO:tasks.workunit.client.0.smithi100.stdout:op 2138 completed, throughput=4.99MB/sec 2024-04-07T16:45:35.018 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-VlsXKqCNnP7Jdig off=2652795 len=1884733 2024-04-07T16:45:35.030 INFO:tasks.workunit.client.0.smithi100.stdout:op 2139 completed, throughput=5MB/sec 2024-04-07T16:45:35.030 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-blNWv9jPFQgmL7E off=1372589 len=1876458 2024-04-07T16:45:35.045 INFO:tasks.workunit.client.0.smithi100.stdout:op 2140 completed, throughput=5MB/sec 2024-04-07T16:45:36.045 INFO:tasks.workunit.client.0.smithi100.stdout: 406: throughput=4.99MB/sec pending data=0 2024-04-07T16:45:36.045 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-CFwzKobuSaNMZBE off=4828938 len=1799187 2024-04-07T16:45:36.060 INFO:tasks.workunit.client.0.smithi100.stdout:op 2141 completed, throughput=4.99MB/sec 2024-04-07T16:45:36.060 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-dLUQ2fwH95uo3aA off=519355 len=851930 2024-04-07T16:45:36.062 INFO:tasks.workunit.client.0.smithi100.stdout:op 2142 completed, throughput=5MB/sec 2024-04-07T16:45:36.063 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-3L9q94TshSpcilb off=1156084 len=12637 2024-04-07T16:45:36.063 INFO:tasks.workunit.client.0.smithi100.stdout:op 2143 completed, throughput=5MB/sec 2024-04-07T16:45:36.063 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-YrWM9fvovXjNP-V off=9884291 len=680815 2024-04-07T16:45:36.066 INFO:tasks.workunit.client.0.smithi100.stdout:op 2144 completed, throughput=5MB/sec 2024-04-07T16:45:36.066 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-MX1kJBvdbJQHXSh off=1876794 len=73362 2024-04-07T16:45:36.070 INFO:tasks.workunit.client.0.smithi100.stdout:op 2145 completed, throughput=5MB/sec 2024-04-07T16:45:36.071 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Yb7KeOl87aGSROp off=1258786 len=1815961 2024-04-07T16:45:36.085 INFO:tasks.workunit.client.0.smithi100.stdout:op 2146 completed, throughput=5MB/sec 2024-04-07T16:45:36.693 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:45:36 smithi100 bash[74942]: cluster 2024-04-07T16:45:35.292253+0000 mgr.y (mgr.25175) 907 : cluster [DBG] pgmap v1549: 105 pgs: 105 active+clean; 126 GiB data, 6.0 GiB used, 709 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.0 MiB/s wr, 4 op/s 2024-04-07T16:45:36.694 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:45:36 smithi100 bash[76202]: cluster 2024-04-07T16:45:35.292253+0000 mgr.y (mgr.25175) 907 : cluster [DBG] pgmap v1549: 105 pgs: 105 active+clean; 126 GiB data, 6.0 GiB used, 709 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.0 MiB/s wr, 4 op/s 2024-04-07T16:45:36.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:45:36 smithi144 bash[66654]: cluster 2024-04-07T16:45:35.292253+0000 mgr.y (mgr.25175) 907 : cluster [DBG] pgmap v1549: 105 pgs: 105 active+clean; 126 GiB data, 6.0 GiB used, 709 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.0 MiB/s wr, 4 op/s 2024-04-07T16:45:37.085 INFO:tasks.workunit.client.0.smithi100.stdout: 407: throughput=4.99MB/sec pending data=0 2024-04-07T16:45:37.086 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-v5liglQAtYozywU off=599527 len=677492 2024-04-07T16:45:37.093 INFO:tasks.workunit.client.0.smithi100.stdout:op 2147 completed, throughput=4.99MB/sec 2024-04-07T16:45:37.093 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-hbNUtsbIvnGO8hm off=875957 len=91880 2024-04-07T16:45:37.098 INFO:tasks.workunit.client.0.smithi100.stdout:op 2148 completed, throughput=4.99MB/sec 2024-04-07T16:45:37.098 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-J_mInNSaDJJXu_Q off=10789158 len=648112 2024-04-07T16:45:37.100 INFO:tasks.workunit.client.0.smithi100.stdout:op 2149 completed, throughput=4.99MB/sec 2024-04-07T16:45:37.101 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-VPJYiGwITK958Nl off=160740 len=164122 2024-04-07T16:45:37.102 INFO:tasks.workunit.client.0.smithi100.stdout:op 2150 completed, throughput=4.99MB/sec 2024-04-07T16:45:37.102 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-iyeSNHNrKaEqKNP off=250157 len=160618 2024-04-07T16:45:37.107 INFO:tasks.workunit.client.0.smithi100.stdout:op 2151 completed, throughput=4.99MB/sec 2024-04-07T16:45:37.107 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-sTI1nEwMrFByyVq off=387589 len=969308 2024-04-07T16:45:37.116 INFO:tasks.workunit.client.0.smithi100.stdout:op 2152 completed, throughput=4.99MB/sec 2024-04-07T16:45:37.116 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-py932k0SCQ68jw9 off=3545565 len=92432 2024-04-07T16:45:37.121 INFO:tasks.workunit.client.0.smithi100.stdout:op 2153 completed, throughput=4.99MB/sec 2024-04-07T16:45:37.121 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-VbUqVsEdNw4hwSy off=6120903 len=1047588 2024-04-07T16:45:37.124 INFO:tasks.workunit.client.0.smithi100.stdout:op 2154 completed, throughput=5MB/sec 2024-04-07T16:45:37.124 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-ie5GYglbWO53avd off=4370478 len=226023 2024-04-07T16:45:37.129 INFO:tasks.workunit.client.0.smithi100.stdout:op 2155 completed, throughput=5MB/sec 2024-04-07T16:45:37.129 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-_vx2y8JbRadrDCZ off=3850637 len=908484 2024-04-07T16:45:37.138 INFO:tasks.workunit.client.0.smithi100.stdout:op 2156 completed, throughput=5MB/sec 2024-04-07T16:45:37.138 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-R0OTNd5v6ssLuEn off=1867102 len=1974904 2024-04-07T16:45:37.140 INFO:tasks.workunit.client.0.smithi100.stdout:op 2157 completed, throughput=5MB/sec 2024-04-07T16:45:38.141 INFO:tasks.workunit.client.0.smithi100.stdout: 408: throughput=4.99MB/sec pending data=0 2024-04-07T16:45:38.141 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-MWyH1sYuX63dKKz off=5799014 len=781256 2024-04-07T16:45:38.149 INFO:tasks.workunit.client.0.smithi100.stdout:op 2158 completed, throughput=4.99MB/sec 2024-04-07T16:45:38.149 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-5NQvhp_b1vr7o7m off=8743948 len=1381618 2024-04-07T16:45:38.153 INFO:tasks.workunit.client.0.smithi100.stdout:op 2159 completed, throughput=5MB/sec 2024-04-07T16:45:38.153 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-jCu2HuS_bSGwBnp off=630319 len=1375345 2024-04-07T16:45:38.164 INFO:tasks.workunit.client.0.smithi100.stdout:op 2160 completed, throughput=5MB/sec 2024-04-07T16:45:38.164 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-nvn0l_gFxnB0QwM off=4751043 len=608392 2024-04-07T16:45:38.171 INFO:tasks.workunit.client.0.smithi100.stdout:op 2161 completed, throughput=5MB/sec 2024-04-07T16:45:38.693 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:45:38 smithi100 bash[74942]: cluster 2024-04-07T16:45:37.293308+0000 mgr.y (mgr.25175) 908 : cluster [DBG] pgmap v1550: 105 pgs: 105 active+clean; 126 GiB data, 6.0 GiB used, 709 GiB / 715 GiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 4 op/s 2024-04-07T16:45:38.694 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:45:38 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:45:38] "GET /metrics HTTP/1.1" 200 34075 "" "Prometheus/2.43.0" 2024-04-07T16:45:38.694 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:45:38 smithi100 bash[76202]: cluster 2024-04-07T16:45:37.293308+0000 mgr.y (mgr.25175) 908 : cluster [DBG] pgmap v1550: 105 pgs: 105 active+clean; 126 GiB data, 6.0 GiB used, 709 GiB / 715 GiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 4 op/s 2024-04-07T16:45:38.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:45:38 smithi144 bash[66654]: cluster 2024-04-07T16:45:37.293308+0000 mgr.y (mgr.25175) 908 : cluster [DBG] pgmap v1550: 105 pgs: 105 active+clean; 126 GiB data, 6.0 GiB used, 709 GiB / 715 GiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 4 op/s 2024-04-07T16:45:39.172 INFO:tasks.workunit.client.0.smithi100.stdout: 409: throughput=4.99MB/sec pending data=0 2024-04-07T16:45:39.172 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-RRfuIRAbu_CjjNO off=390320 len=866538 2024-04-07T16:45:39.180 INFO:tasks.workunit.client.0.smithi100.stdout:op 2162 completed, throughput=4.99MB/sec 2024-04-07T16:45:39.180 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-tF4dTg9ghG-yQqi off=8012453 len=913195 2024-04-07T16:45:39.182 INFO:tasks.workunit.client.0.smithi100.stdout:op 2163 completed, throughput=4.99MB/sec 2024-04-07T16:45:39.182 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-T_v-rEc_8xVsPeb off=953278 len=909762 2024-04-07T16:45:39.184 INFO:tasks.workunit.client.0.smithi100.stdout:op 2164 completed, throughput=5MB/sec 2024-04-07T16:45:39.184 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-tkV7586MsgSUULP off=788194 len=2068357 2024-04-07T16:45:39.188 INFO:tasks.workunit.client.0.smithi100.stdout:op 2165 completed, throughput=5MB/sec 2024-04-07T16:45:39.188 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-OYdoex0WAoj5uxy off=2554954 len=1231359 2024-04-07T16:45:39.193 INFO:tasks.workunit.client.0.smithi100.stdout:op 2166 completed, throughput=5MB/sec 2024-04-07T16:45:40.193 INFO:tasks.workunit.client.0.smithi100.stdout: 410: throughput=4.99MB/sec pending data=0 2024-04-07T16:45:40.193 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-wuOcZqUnhRQ-djZ off=2016593 len=1935813 2024-04-07T16:45:40.198 INFO:tasks.workunit.client.0.smithi100.stdout:op 2167 completed, throughput=4.99MB/sec 2024-04-07T16:45:40.198 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-xsKz7snD2mZxYAe off=7133934 len=937540 2024-04-07T16:45:40.201 INFO:tasks.workunit.client.0.smithi100.stdout:op 2168 completed, throughput=5MB/sec 2024-04-07T16:45:40.201 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-R7ghHxlF0tKNiw- off=6146676 len=1702094 2024-04-07T16:45:40.214 INFO:tasks.workunit.client.0.smithi100.stdout:op 2169 completed, throughput=5MB/sec 2024-04-07T16:45:40.693 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:45:40 smithi100 bash[74942]: cluster 2024-04-07T16:45:39.294126+0000 mgr.y (mgr.25175) 909 : cluster [DBG] pgmap v1551: 105 pgs: 105 active+clean; 126 GiB data, 6.0 GiB used, 709 GiB / 715 GiB avail; 2.0 MiB/s rd, 2.5 MiB/s wr, 4 op/s 2024-04-07T16:45:40.694 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:45:40 smithi100 bash[76202]: cluster 2024-04-07T16:45:39.294126+0000 mgr.y (mgr.25175) 909 : cluster [DBG] pgmap v1551: 105 pgs: 105 active+clean; 126 GiB data, 6.0 GiB used, 709 GiB / 715 GiB avail; 2.0 MiB/s rd, 2.5 MiB/s wr, 4 op/s 2024-04-07T16:45:40.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:45:40 smithi144 bash[66654]: cluster 2024-04-07T16:45:39.294126+0000 mgr.y (mgr.25175) 909 : cluster [DBG] pgmap v1551: 105 pgs: 105 active+clean; 126 GiB data, 6.0 GiB used, 709 GiB / 715 GiB avail; 2.0 MiB/s rd, 2.5 MiB/s wr, 4 op/s 2024-04-07T16:45:41.214 INFO:tasks.workunit.client.0.smithi100.stdout: 411: throughput=4.99MB/sec pending data=0 2024-04-07T16:45:41.215 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-dQKhP05uycb7dsn off=1803313 len=587447 2024-04-07T16:45:41.217 INFO:tasks.workunit.client.0.smithi100.stdout:op 2170 completed, throughput=4.99MB/sec 2024-04-07T16:45:41.217 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-hh6m-FtvxpCMevg off=4986868 len=1041898 2024-04-07T16:45:41.226 INFO:tasks.workunit.client.0.smithi100.stdout:op 2171 completed, throughput=4.99MB/sec 2024-04-07T16:45:41.226 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-HnJ24K6EWaR4UsG off=15565934 len=1680526 2024-04-07T16:45:41.237 INFO:tasks.workunit.client.0.smithi100.stdout:op 2172 completed, throughput=5MB/sec 2024-04-07T16:45:41.237 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-6tn2pa-EkUg6pKL off=2931923 len=1910157 2024-04-07T16:45:41.253 INFO:tasks.workunit.client.0.smithi100.stdout:op 2173 completed, throughput=5MB/sec 2024-04-07T16:45:42.253 INFO:tasks.workunit.client.0.smithi100.stdout: 412: throughput=4.99MB/sec pending data=0 2024-04-07T16:45:42.254 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Xa_egBgoeuauJ87 off=1801432 len=65924 2024-04-07T16:45:42.254 INFO:tasks.workunit.client.0.smithi100.stdout:op 2174 completed, throughput=4.99MB/sec 2024-04-07T16:45:42.255 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-rAGT4ScxDS5RVMd off=924557 len=1174467 2024-04-07T16:45:42.265 INFO:tasks.workunit.client.0.smithi100.stdout:op 2175 completed, throughput=4.99MB/sec 2024-04-07T16:45:42.265 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-ZqT7k925WE6WgoN off=1361385 len=933602 2024-04-07T16:45:42.268 INFO:tasks.workunit.client.0.smithi100.stdout:op 2176 completed, throughput=4.99MB/sec 2024-04-07T16:45:42.268 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-HiPZHoCbqKqvoVQ off=1394956 len=25271 2024-04-07T16:45:42.269 INFO:tasks.workunit.client.0.smithi100.stdout:op 2177 completed, throughput=4.99MB/sec 2024-04-07T16:45:42.269 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-FLU4i3Ze78knz7- off=8890950 len=1893861 2024-04-07T16:45:42.274 INFO:tasks.workunit.client.0.smithi100.stdout:op 2178 completed, throughput=5MB/sec 2024-04-07T16:45:42.274 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-rceVfqSkjxzneqd off=348273 len=948114 2024-04-07T16:45:42.276 INFO:tasks.workunit.client.0.smithi100.stdout:op 2179 completed, throughput=5MB/sec 2024-04-07T16:45:42.276 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-666Foq8RvXdAWv6 off=4553016 len=60666 2024-04-07T16:45:42.281 INFO:tasks.workunit.client.0.smithi100.stdout:op 2180 completed, throughput=5MB/sec 2024-04-07T16:45:42.281 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-iyByukcMxiQ5sOd off=12100503 len=1401003 2024-04-07T16:45:42.292 INFO:tasks.workunit.client.0.smithi100.stdout:op 2181 completed, throughput=5MB/sec 2024-04-07T16:45:42.608 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:45:42 smithi144 bash[66654]: cluster 2024-04-07T16:45:41.295089+0000 mgr.y (mgr.25175) 910 : cluster [DBG] pgmap v1552: 105 pgs: 105 active+clean; 126 GiB data, 6.0 GiB used, 709 GiB / 715 GiB avail; 1.9 MiB/s rd, 2.8 MiB/s wr, 4 op/s 2024-04-07T16:45:42.608 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:45:42 smithi144 bash[95233]: ts=2024-04-07T16:45:42.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:45:42.694 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:45:42 smithi100 bash[74942]: cluster 2024-04-07T16:45:41.295089+0000 mgr.y (mgr.25175) 910 : cluster [DBG] pgmap v1552: 105 pgs: 105 active+clean; 126 GiB data, 6.0 GiB used, 709 GiB / 715 GiB avail; 1.9 MiB/s rd, 2.8 MiB/s wr, 4 op/s 2024-04-07T16:45:42.694 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:45:42 smithi100 bash[76202]: cluster 2024-04-07T16:45:41.295089+0000 mgr.y (mgr.25175) 910 : cluster [DBG] pgmap v1552: 105 pgs: 105 active+clean; 126 GiB data, 6.0 GiB used, 709 GiB / 715 GiB avail; 1.9 MiB/s rd, 2.8 MiB/s wr, 4 op/s 2024-04-07T16:45:43.293 INFO:tasks.workunit.client.0.smithi100.stdout: 413: throughput=4.99MB/sec pending data=0 2024-04-07T16:45:43.293 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-u3jq0643rncw3hD off=10215057 len=1215549 2024-04-07T16:45:43.303 INFO:tasks.workunit.client.0.smithi100.stdout:op 2182 completed, throughput=4.99MB/sec 2024-04-07T16:45:43.303 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-XYsf9SJbuVt35GT off=4181582 len=88921 2024-04-07T16:45:43.305 INFO:tasks.workunit.client.0.smithi100.stdout:op 2183 completed, throughput=4.99MB/sec 2024-04-07T16:45:43.305 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-_5ZKHHqfApuet5q off=2656955 len=1629897 2024-04-07T16:45:43.309 INFO:tasks.workunit.client.0.smithi100.stdout:op 2184 completed, throughput=5MB/sec 2024-04-07T16:45:43.309 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-U7ZzL5yWgu5wvBm off=3800469 len=924806 2024-04-07T16:45:43.318 INFO:tasks.workunit.client.0.smithi100.stdout:op 2185 completed, throughput=5MB/sec 2024-04-07T16:45:43.318 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-_Fe9s6-8_Ym_d2m off=386368 len=1004748 2024-04-07T16:45:43.326 INFO:tasks.workunit.client.0.smithi100.stdout:op 2186 completed, throughput=5MB/sec 2024-04-07T16:45:44.326 INFO:tasks.workunit.client.0.smithi100.stdout: 414: throughput=4.99MB/sec pending data=0 2024-04-07T16:45:44.326 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-rFjdUG0TQmPf3Pt off=338122 len=634529 2024-04-07T16:45:44.328 INFO:tasks.workunit.client.0.smithi100.stdout:op 2187 completed, throughput=4.99MB/sec 2024-04-07T16:45:44.328 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-dVK2VF8CzhG2SL7 off=10477232 len=1616252 2024-04-07T16:45:44.341 INFO:tasks.workunit.client.0.smithi100.stdout:op 2188 completed, throughput=4.99MB/sec 2024-04-07T16:45:44.342 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-6SISLknx0Y-ICXA off=8966165 len=1328091 2024-04-07T16:45:44.345 INFO:tasks.workunit.client.0.smithi100.stdout:op 2189 completed, throughput=5MB/sec 2024-04-07T16:45:44.345 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj--2cCoAzqmI7wtXV off=2050749 len=1989129 2024-04-07T16:45:44.351 INFO:tasks.workunit.client.0.smithi100.stdout:op 2190 completed, throughput=5MB/sec 2024-04-07T16:45:44.693 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:45:44 smithi100 bash[74942]: cluster 2024-04-07T16:45:43.296251+0000 mgr.y (mgr.25175) 911 : cluster [DBG] pgmap v1553: 105 pgs: 105 active+clean; 126 GiB data, 6.0 GiB used, 709 GiB / 715 GiB avail; 2.2 MiB/s rd, 2.9 MiB/s wr, 5 op/s 2024-04-07T16:45:44.694 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:45:44 smithi100 bash[76202]: cluster 2024-04-07T16:45:43.296251+0000 mgr.y (mgr.25175) 911 : cluster [DBG] pgmap v1553: 105 pgs: 105 active+clean; 126 GiB data, 6.0 GiB used, 709 GiB / 715 GiB avail; 2.2 MiB/s rd, 2.9 MiB/s wr, 5 op/s 2024-04-07T16:45:44.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:45:44 smithi144 bash[66654]: cluster 2024-04-07T16:45:43.296251+0000 mgr.y (mgr.25175) 911 : cluster [DBG] pgmap v1553: 105 pgs: 105 active+clean; 126 GiB data, 6.0 GiB used, 709 GiB / 715 GiB avail; 2.2 MiB/s rd, 2.9 MiB/s wr, 5 op/s 2024-04-07T16:45:45.352 INFO:tasks.workunit.client.0.smithi100.stdout: 415: throughput=4.99MB/sec pending data=0 2024-04-07T16:45:45.352 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-JdSo-hNpQZfmRiZ off=9818467 len=2085521 2024-04-07T16:45:45.369 INFO:tasks.workunit.client.0.smithi100.stdout:op 2191 completed, throughput=4.99MB/sec 2024-04-07T16:45:45.369 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Y0GQfT54qskDfGb off=517400 len=1443811 2024-04-07T16:45:45.380 INFO:tasks.workunit.client.0.smithi100.stdout:op 2192 completed, throughput=5MB/sec 2024-04-07T16:45:45.381 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-WFyRAP-teQovKQ_ off=441253 len=1967292 2024-04-07T16:45:45.385 INFO:tasks.workunit.client.0.smithi100.stdout:op 2193 completed, throughput=5MB/sec 2024-04-07T16:45:46.385 INFO:tasks.workunit.client.0.smithi100.stdout: 416: throughput=4.99MB/sec pending data=0 2024-04-07T16:45:46.385 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-O8GLs2d8n_kDxCZ off=1436855 len=968899 2024-04-07T16:45:46.395 INFO:tasks.workunit.client.0.smithi100.stdout:op 2194 completed, throughput=4.99MB/sec 2024-04-07T16:45:46.395 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-8rFvjgxSqxdi7vY off=8324388 len=1701491 2024-04-07T16:45:46.399 INFO:tasks.workunit.client.0.smithi100.stdout:op 2195 completed, throughput=5MB/sec 2024-04-07T16:45:46.399 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-AjnnIOZ38HA8joZ off=1840620 len=988698 2024-04-07T16:45:46.402 INFO:tasks.workunit.client.0.smithi100.stdout:op 2196 completed, throughput=5MB/sec 2024-04-07T16:45:46.402 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-6589zTlLT0TVUkZ off=5941414 len=2000326 2024-04-07T16:45:46.409 INFO:tasks.workunit.client.0.smithi100.stdout:op 2197 completed, throughput=5MB/sec 2024-04-07T16:45:46.693 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:45:46 smithi100 bash[74942]: cluster 2024-04-07T16:45:45.297236+0000 mgr.y (mgr.25175) 912 : cluster [DBG] pgmap v1554: 105 pgs: 105 active+clean; 126 GiB data, 6.0 GiB used, 709 GiB / 715 GiB avail; 1.9 MiB/s rd, 2.8 MiB/s wr, 4 op/s 2024-04-07T16:45:46.694 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:45:46 smithi100 bash[76202]: cluster 2024-04-07T16:45:45.297236+0000 mgr.y (mgr.25175) 912 : cluster [DBG] pgmap v1554: 105 pgs: 105 active+clean; 126 GiB data, 6.0 GiB used, 709 GiB / 715 GiB avail; 1.9 MiB/s rd, 2.8 MiB/s wr, 4 op/s 2024-04-07T16:45:46.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:45:46 smithi144 bash[66654]: cluster 2024-04-07T16:45:45.297236+0000 mgr.y (mgr.25175) 912 : cluster [DBG] pgmap v1554: 105 pgs: 105 active+clean; 126 GiB data, 6.0 GiB used, 709 GiB / 715 GiB avail; 1.9 MiB/s rd, 2.8 MiB/s wr, 4 op/s 2024-04-07T16:45:47.409 INFO:tasks.workunit.client.0.smithi100.stdout: 417: throughput=4.99MB/sec pending data=0 2024-04-07T16:45:47.409 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-FYWMZLlnjNMiIsQ off=2849548 len=411188 2024-04-07T16:45:47.411 INFO:tasks.workunit.client.0.smithi100.stdout:op 2198 completed, throughput=4.99MB/sec 2024-04-07T16:45:47.411 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-GgH8XtAdH1Fs5HV off=4502981 len=1029732 2024-04-07T16:45:47.422 INFO:tasks.workunit.client.0.smithi100.stdout:op 2199 completed, throughput=4.99MB/sec 2024-04-07T16:45:47.422 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-MeyKJSmFM2eeKzX off=7909058 len=717052 2024-04-07T16:45:47.424 INFO:tasks.workunit.client.0.smithi100.stdout:op 2200 completed, throughput=5MB/sec 2024-04-07T16:45:47.424 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-T5qLORVR5Bej5xI off=5482252 len=1511545 2024-04-07T16:45:47.437 INFO:tasks.workunit.client.0.smithi100.stdout:op 2201 completed, throughput=5MB/sec 2024-04-07T16:45:47.437 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-5T8E1q8NkgtdhVw off=2176073 len=1150935 2024-04-07T16:45:47.441 INFO:tasks.workunit.client.0.smithi100.stdout:op 2202 completed, throughput=5MB/sec 2024-04-07T16:45:47.693 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:45:47 smithi100 bash[74942]: audit 2024-04-07T16:45:46.767369+0000 mon.a (mon.0) 2104 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:45:47.694 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:45:47 smithi100 bash[76202]: audit 2024-04-07T16:45:46.767369+0000 mon.a (mon.0) 2104 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:45:47.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:45:47 smithi144 bash[66654]: audit 2024-04-07T16:45:46.767369+0000 mon.a (mon.0) 2104 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:45:48.442 INFO:tasks.workunit.client.0.smithi100.stdout: 418: throughput=4.99MB/sec pending data=0 2024-04-07T16:45:48.442 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-6lG8Qj7o32JG4Oe off=3436645 len=123497 2024-04-07T16:45:48.443 INFO:tasks.workunit.client.0.smithi100.stdout:op 2203 completed, throughput=4.99MB/sec 2024-04-07T16:45:48.443 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-GWLQmWe1UNyEcId off=2283270 len=475831 2024-04-07T16:45:48.445 INFO:tasks.workunit.client.0.smithi100.stdout:op 2204 completed, throughput=4.99MB/sec 2024-04-07T16:45:48.445 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-1ue7cLCaXr9Dh5w off=8588216 len=443291 2024-04-07T16:45:48.452 INFO:tasks.workunit.client.0.smithi100.stdout:op 2205 completed, throughput=4.99MB/sec 2024-04-07T16:45:48.452 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-If_WpQY-FckATAK off=13413804 len=367064 2024-04-07T16:45:48.454 INFO:tasks.workunit.client.0.smithi100.stdout:op 2206 completed, throughput=4.99MB/sec 2024-04-07T16:45:48.454 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-iW-UNG8htdhYqLD off=14203039 len=824271 2024-04-07T16:45:48.456 INFO:tasks.workunit.client.0.smithi100.stdout:op 2207 completed, throughput=4.99MB/sec 2024-04-07T16:45:48.456 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-g5bEDbUz1Cq1TaT off=2086667 len=534501 2024-04-07T16:45:48.458 INFO:tasks.workunit.client.0.smithi100.stdout:op 2208 completed, throughput=5MB/sec 2024-04-07T16:45:48.458 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-b-j-Aj0gw6zLbm9 off=3599753 len=1312935 2024-04-07T16:45:48.461 INFO:tasks.workunit.client.0.smithi100.stdout:op 2209 completed, throughput=5MB/sec 2024-04-07T16:45:48.461 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-aHn6CCzc9M_anHn off=22830907 len=1245676 2024-04-07T16:45:48.465 INFO:tasks.workunit.client.0.smithi100.stdout:op 2210 completed, throughput=5MB/sec 2024-04-07T16:45:48.693 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:45:48 smithi100 bash[74942]: cluster 2024-04-07T16:45:47.298273+0000 mgr.y (mgr.25175) 913 : cluster [DBG] pgmap v1555: 105 pgs: 105 active+clean; 126 GiB data, 6.1 GiB used, 709 GiB / 715 GiB avail; 2.6 MiB/s rd, 2.6 MiB/s wr, 4 op/s 2024-04-07T16:45:48.694 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:45:48 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:45:48] "GET /metrics HTTP/1.1" 200 34070 "" "Prometheus/2.43.0" 2024-04-07T16:45:48.694 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:45:48 smithi100 bash[76202]: cluster 2024-04-07T16:45:47.298273+0000 mgr.y (mgr.25175) 913 : cluster [DBG] pgmap v1555: 105 pgs: 105 active+clean; 126 GiB data, 6.1 GiB used, 709 GiB / 715 GiB avail; 2.6 MiB/s rd, 2.6 MiB/s wr, 4 op/s 2024-04-07T16:45:48.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:45:48 smithi144 bash[66654]: cluster 2024-04-07T16:45:47.298273+0000 mgr.y (mgr.25175) 913 : cluster [DBG] pgmap v1555: 105 pgs: 105 active+clean; 126 GiB data, 6.1 GiB used, 709 GiB / 715 GiB avail; 2.6 MiB/s rd, 2.6 MiB/s wr, 4 op/s 2024-04-07T16:45:49.465 INFO:tasks.workunit.client.0.smithi100.stdout: 419: throughput=4.99MB/sec pending data=0 2024-04-07T16:45:49.465 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-EVBhUtNwMjW5LIo off=86927 len=997929 2024-04-07T16:45:49.474 INFO:tasks.workunit.client.0.smithi100.stdout:op 2211 completed, throughput=4.99MB/sec 2024-04-07T16:45:49.474 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-t2KI8e95moq05Jb off=1308843 len=654380 2024-04-07T16:45:49.481 INFO:tasks.workunit.client.0.smithi100.stdout:op 2212 completed, throughput=4.99MB/sec 2024-04-07T16:45:49.481 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-5qnWyQKiJ1UpT_v off=12963946 len=448336 2024-04-07T16:45:49.487 INFO:tasks.workunit.client.0.smithi100.stdout:op 2213 completed, throughput=4.99MB/sec 2024-04-07T16:45:49.488 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-z5vpfWTBFTql70K off=13272853 len=132400 2024-04-07T16:45:49.492 INFO:tasks.workunit.client.0.smithi100.stdout:op 2214 completed, throughput=4.99MB/sec 2024-04-07T16:45:49.492 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-46e33AQPH7XYFGC off=1615023 len=721809 2024-04-07T16:45:49.499 INFO:tasks.workunit.client.0.smithi100.stdout:op 2215 completed, throughput=5MB/sec 2024-04-07T16:45:49.500 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-bV8h7_T6FeIZVg- off=6870857 len=907082 2024-04-07T16:45:49.502 INFO:tasks.workunit.client.0.smithi100.stdout:op 2216 completed, throughput=5MB/sec 2024-04-07T16:45:49.502 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-lcI39xuQd8HzYeU off=1927287 len=206429 2024-04-07T16:45:49.503 INFO:tasks.workunit.client.0.smithi100.stdout:op 2217 completed, throughput=5MB/sec 2024-04-07T16:45:49.503 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-XqdyYbaN9hEG9kF off=10427971 len=1207065 2024-04-07T16:45:49.506 INFO:tasks.workunit.client.0.smithi100.stdout:op 2218 completed, throughput=5MB/sec 2024-04-07T16:45:50.506 INFO:tasks.workunit.client.0.smithi100.stdout: 420: throughput=4.99MB/sec pending data=0 2024-04-07T16:45:50.506 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-drWs3qk1MloYk6I off=225442 len=1527844 2024-04-07T16:45:50.512 INFO:tasks.workunit.client.0.smithi100.stdout:op 2219 completed, throughput=4.99MB/sec 2024-04-07T16:45:50.512 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-RemyBWipgyFlBhW off=450709 len=931315 2024-04-07T16:45:50.516 INFO:tasks.workunit.client.0.smithi100.stdout:op 2220 completed, throughput=4.99MB/sec 2024-04-07T16:45:50.516 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-ScgnY_zZIGFm32F off=2852038 len=406716 2024-04-07T16:45:50.522 INFO:tasks.workunit.client.0.smithi100.stdout:op 2221 completed, throughput=5MB/sec 2024-04-07T16:45:50.522 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Dc4N-qiF_piZbZ- off=8814788 len=1784942 2024-04-07T16:45:50.541 INFO:tasks.workunit.client.0.smithi100.stdout:op 2222 completed, throughput=5MB/sec 2024-04-07T16:45:50.541 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-LLO6o6UEhKzLdgE off=1865281 len=1929235 2024-04-07T16:45:50.545 INFO:tasks.workunit.client.0.smithi100.stdout:op 2223 completed, throughput=5MB/sec 2024-04-07T16:45:50.694 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:45:50 smithi100 bash[74942]: cluster 2024-04-07T16:45:49.299044+0000 mgr.y (mgr.25175) 914 : cluster [DBG] pgmap v1556: 105 pgs: 105 active+clean; 126 GiB data, 6.1 GiB used, 709 GiB / 715 GiB avail; 2.4 MiB/s rd, 2.6 MiB/s wr, 4 op/s 2024-04-07T16:45:50.694 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:45:50 smithi100 bash[76202]: cluster 2024-04-07T16:45:49.299044+0000 mgr.y (mgr.25175) 914 : cluster [DBG] pgmap v1556: 105 pgs: 105 active+clean; 126 GiB data, 6.1 GiB used, 709 GiB / 715 GiB avail; 2.4 MiB/s rd, 2.6 MiB/s wr, 4 op/s 2024-04-07T16:45:50.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:45:50 smithi144 bash[66654]: cluster 2024-04-07T16:45:49.299044+0000 mgr.y (mgr.25175) 914 : cluster [DBG] pgmap v1556: 105 pgs: 105 active+clean; 126 GiB data, 6.1 GiB used, 709 GiB / 715 GiB avail; 2.4 MiB/s rd, 2.6 MiB/s wr, 4 op/s 2024-04-07T16:45:51.545 INFO:tasks.workunit.client.0.smithi100.stdout: 421: throughput=4.99MB/sec pending data=0 2024-04-07T16:45:51.545 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-g4nseqNT0zh07SQ off=3644491 len=1716755 2024-04-07T16:45:51.551 INFO:tasks.workunit.client.0.smithi100.stdout:op 2224 completed, throughput=5MB/sec 2024-04-07T16:45:51.551 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-QYHcRTRiPYvzojK off=979813 len=733875 2024-04-07T16:45:51.559 INFO:tasks.workunit.client.0.smithi100.stdout:op 2225 completed, throughput=5MB/sec 2024-04-07T16:45:51.559 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-EpTq_67xn8Rv09R off=2860780 len=84038 2024-04-07T16:45:51.560 INFO:tasks.workunit.client.0.smithi100.stdout:op 2226 completed, throughput=5MB/sec 2024-04-07T16:45:51.560 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-EKF73lebc5ruY_7 off=28805 len=354272 2024-04-07T16:45:51.565 INFO:tasks.workunit.client.0.smithi100.stdout:op 2227 completed, throughput=5MB/sec 2024-04-07T16:45:51.565 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-5d1CvB3s5CuWHMn off=1532224 len=1572620 2024-04-07T16:45:51.578 INFO:tasks.workunit.client.0.smithi100.stdout:op 2228 completed, throughput=5MB/sec 2024-04-07T16:45:52.578 INFO:tasks.workunit.client.0.smithi100.stdout: 422: throughput=4.99MB/sec pending data=0 2024-04-07T16:45:52.579 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-RdCczqVqH3a4uuK off=1489709 len=1155955 2024-04-07T16:45:52.588 INFO:tasks.workunit.client.0.smithi100.stdout:op 2229 completed, throughput=4.99MB/sec 2024-04-07T16:45:52.588 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Zq09cKo-YxY5yPY off=1461480 len=1101048 2024-04-07T16:45:52.590 INFO:tasks.workunit.client.0.smithi100.stdout:op 2230 completed, throughput=4.99MB/sec 2024-04-07T16:45:52.590 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-EPIy_FxC21_LYpI off=18063495 len=529577 2024-04-07T16:45:52.593 INFO:tasks.workunit.client.0.smithi100.stdout:op 2231 completed, throughput=5MB/sec 2024-04-07T16:45:52.593 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-eok0ariF5PeBIRo off=9574649 len=1008227 2024-04-07T16:45:52.595 INFO:tasks.workunit.client.0.smithi100.stdout:op 2232 completed, throughput=5MB/sec 2024-04-07T16:45:52.596 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-4fwJBnrhWEf__C0 off=3274519 len=231723 2024-04-07T16:45:52.601 INFO:tasks.workunit.client.0.smithi100.stdout:op 2233 completed, throughput=5MB/sec 2024-04-07T16:45:52.601 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-pk5-l2j7SChZrT7 off=1431923 len=746461 2024-04-07T16:45:52.603 INFO:tasks.workunit.client.0.smithi100.stdout:op 2234 completed, throughput=5MB/sec 2024-04-07T16:45:52.603 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-C8nskuj2toXM7S8 off=15401545 len=168934 2024-04-07T16:45:52.604 INFO:tasks.workunit.client.0.smithi100.stdout:op 2235 completed, throughput=5MB/sec 2024-04-07T16:45:52.608 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:45:52 smithi144 bash[66654]: cluster 2024-04-07T16:45:51.299909+0000 mgr.y (mgr.25175) 915 : cluster [DBG] pgmap v1557: 105 pgs: 105 active+clean; 126 GiB data, 6.1 GiB used, 709 GiB / 715 GiB avail; 2.6 MiB/s rd, 2.3 MiB/s wr, 4 op/s 2024-04-07T16:45:52.608 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:45:52 smithi144 bash[95233]: ts=2024-04-07T16:45:52.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:45:52.694 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:45:52 smithi100 bash[74942]: cluster 2024-04-07T16:45:51.299909+0000 mgr.y (mgr.25175) 915 : cluster [DBG] pgmap v1557: 105 pgs: 105 active+clean; 126 GiB data, 6.1 GiB used, 709 GiB / 715 GiB avail; 2.6 MiB/s rd, 2.3 MiB/s wr, 4 op/s 2024-04-07T16:45:52.694 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:45:52 smithi100 bash[76202]: cluster 2024-04-07T16:45:51.299909+0000 mgr.y (mgr.25175) 915 : cluster [DBG] pgmap v1557: 105 pgs: 105 active+clean; 126 GiB data, 6.1 GiB used, 709 GiB / 715 GiB avail; 2.6 MiB/s rd, 2.3 MiB/s wr, 4 op/s 2024-04-07T16:45:53.605 INFO:tasks.workunit.client.0.smithi100.stdout: 423: throughput=4.99MB/sec pending data=0 2024-04-07T16:45:53.605 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-PRa8XbNubu4gaqO off=14268461 len=1260753 2024-04-07T16:45:53.609 INFO:tasks.workunit.client.0.smithi100.stdout:op 2236 completed, throughput=4.99MB/sec 2024-04-07T16:45:53.609 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-SwgFa4ZQql-ZG-2 off=6593155 len=137497 2024-04-07T16:45:53.614 INFO:tasks.workunit.client.0.smithi100.stdout:op 2237 completed, throughput=4.99MB/sec 2024-04-07T16:45:53.614 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Z6TBP0u2hVu-vb4 off=1139105 len=283246 2024-04-07T16:45:53.616 INFO:tasks.workunit.client.0.smithi100.stdout:op 2238 completed, throughput=4.99MB/sec 2024-04-07T16:45:53.616 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Yq5-enRI-h_BJ1W off=11709743 len=1577188 2024-04-07T16:45:53.620 INFO:tasks.workunit.client.0.smithi100.stdout:op 2239 completed, throughput=5MB/sec 2024-04-07T16:45:53.620 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-sQj0WRyYMFLQAIw off=4364473 len=719599 2024-04-07T16:45:53.627 INFO:tasks.workunit.client.0.smithi100.stdout:op 2240 completed, throughput=5MB/sec 2024-04-07T16:45:53.627 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-SaPCjtyep8E8TZT off=1140332 len=1075664 2024-04-07T16:45:53.629 INFO:tasks.workunit.client.0.smithi100.stdout:op 2241 completed, throughput=5MB/sec 2024-04-07T16:45:53.629 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-4g2XCShrFZk_TjG off=5631003 len=1378276 2024-04-07T16:45:53.631 INFO:tasks.workunit.client.0.smithi100.stdout:op 2242 completed, throughput=5MB/sec 2024-04-07T16:45:54.632 INFO:tasks.workunit.client.0.smithi100.stdout: 424: throughput=4.99MB/sec pending data=0 2024-04-07T16:45:54.632 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Hil8FrPVA67cZ-p off=2757156 len=1763165 2024-04-07T16:45:54.644 INFO:tasks.workunit.client.0.smithi100.stdout:op 2243 completed, throughput=4.99MB/sec 2024-04-07T16:45:54.644 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-0lTZvMsA7-iTa3l off=13644347 len=320677 2024-04-07T16:45:54.645 INFO:tasks.workunit.client.0.smithi100.stdout:op 2244 completed, throughput=5MB/sec 2024-04-07T16:45:54.645 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-RIqLyHVWR0uPlSU off=6768258 len=767081 2024-04-07T16:45:54.652 INFO:tasks.workunit.client.0.smithi100.stdout:op 2245 completed, throughput=5MB/sec 2024-04-07T16:45:54.652 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-3izrdhML1BuFnwg off=2510466 len=2035624 2024-04-07T16:45:54.667 INFO:tasks.workunit.client.0.smithi100.stdout:op 2246 completed, throughput=5MB/sec 2024-04-07T16:45:54.694 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:45:54 smithi100 bash[76202]: cluster 2024-04-07T16:45:53.300990+0000 mgr.y (mgr.25175) 916 : cluster [DBG] pgmap v1558: 105 pgs: 105 active+clean; 126 GiB data, 6.1 GiB used, 709 GiB / 715 GiB avail; 3.3 MiB/s rd, 2.5 MiB/s wr, 5 op/s 2024-04-07T16:45:54.694 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:45:54 smithi100 bash[74942]: cluster 2024-04-07T16:45:53.300990+0000 mgr.y (mgr.25175) 916 : cluster [DBG] pgmap v1558: 105 pgs: 105 active+clean; 126 GiB data, 6.1 GiB used, 709 GiB / 715 GiB avail; 3.3 MiB/s rd, 2.5 MiB/s wr, 5 op/s 2024-04-07T16:45:54.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:45:54 smithi144 bash[66654]: cluster 2024-04-07T16:45:53.300990+0000 mgr.y (mgr.25175) 916 : cluster [DBG] pgmap v1558: 105 pgs: 105 active+clean; 126 GiB data, 6.1 GiB used, 709 GiB / 715 GiB avail; 3.3 MiB/s rd, 2.5 MiB/s wr, 5 op/s 2024-04-07T16:45:55.667 INFO:tasks.workunit.client.0.smithi100.stdout: 425: throughput=4.99MB/sec pending data=0 2024-04-07T16:45:55.667 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-6B6KJZDpIJFCpXD off=3075683 len=258379 2024-04-07T16:45:55.669 INFO:tasks.workunit.client.0.smithi100.stdout:op 2247 completed, throughput=4.99MB/sec 2024-04-07T16:45:55.669 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-_AEmcFW_kMZBDrn off=5785134 len=354799 2024-04-07T16:45:55.671 INFO:tasks.workunit.client.0.smithi100.stdout:op 2248 completed, throughput=4.99MB/sec 2024-04-07T16:45:55.671 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-KMauUUey9adnPPk off=13010455 len=1087160 2024-04-07T16:45:55.674 INFO:tasks.workunit.client.0.smithi100.stdout:op 2249 completed, throughput=4.99MB/sec 2024-04-07T16:45:55.674 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-HNmLiW6d-G-fJ11 off=682341 len=1374064 2024-04-07T16:45:55.686 INFO:tasks.workunit.client.0.smithi100.stdout:op 2250 completed, throughput=5MB/sec 2024-04-07T16:45:55.686 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-39SF2lOm2XxuhY4 off=10646081 len=2041269 2024-04-07T16:45:55.694 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:45:55 smithi100 bash[74942]: audit 2024-04-07T16:45:54.470026+0000 mon.a (mon.0) 2105 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:45:55.694 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:45:55 smithi100 bash[74942]: audit 2024-04-07T16:45:55.202489+0000 mon.a (mon.0) 2106 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:45:55.694 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:45:55 smithi100 bash[74942]: audit 2024-04-07T16:45:55.204312+0000 mon.a (mon.0) 2107 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:45:55.694 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:45:55 smithi100 bash[74942]: audit 2024-04-07T16:45:55.214530+0000 mon.a (mon.0) 2108 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:45:55.694 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:45:55 smithi100 bash[76202]: audit 2024-04-07T16:45:54.470026+0000 mon.a (mon.0) 2105 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:45:55.694 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:45:55 smithi100 bash[76202]: audit 2024-04-07T16:45:55.202489+0000 mon.a (mon.0) 2106 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:45:55.694 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:45:55 smithi100 bash[76202]: audit 2024-04-07T16:45:55.204312+0000 mon.a (mon.0) 2107 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:45:55.694 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:45:55 smithi100 bash[76202]: audit 2024-04-07T16:45:55.214530+0000 mon.a (mon.0) 2108 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:45:55.702 INFO:tasks.workunit.client.0.smithi100.stdout:op 2251 completed, throughput=5MB/sec 2024-04-07T16:45:55.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:45:55 smithi144 bash[66654]: audit 2024-04-07T16:45:54.470026+0000 mon.a (mon.0) 2105 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:45:55.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:45:55 smithi144 bash[66654]: audit 2024-04-07T16:45:55.202489+0000 mon.a (mon.0) 2106 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:45:55.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:45:55 smithi144 bash[66654]: audit 2024-04-07T16:45:55.204312+0000 mon.a (mon.0) 2107 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:45:55.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:45:55 smithi144 bash[66654]: audit 2024-04-07T16:45:55.214530+0000 mon.a (mon.0) 2108 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:45:56.694 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:45:56 smithi100 bash[74942]: cluster 2024-04-07T16:45:55.301859+0000 mgr.y (mgr.25175) 917 : cluster [DBG] pgmap v1559: 105 pgs: 105 active+clean; 126 GiB data, 6.1 GiB used, 709 GiB / 715 GiB avail; 3.0 MiB/s rd, 2.4 MiB/s wr, 5 op/s 2024-04-07T16:45:56.694 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:45:56 smithi100 bash[76202]: cluster 2024-04-07T16:45:55.301859+0000 mgr.y (mgr.25175) 917 : cluster [DBG] pgmap v1559: 105 pgs: 105 active+clean; 126 GiB data, 6.1 GiB used, 709 GiB / 715 GiB avail; 3.0 MiB/s rd, 2.4 MiB/s wr, 5 op/s 2024-04-07T16:45:56.703 INFO:tasks.workunit.client.0.smithi100.stdout: 426: throughput=4.99MB/sec pending data=0 2024-04-07T16:45:56.703 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-zspRCeRd352_V-9 off=2154238 len=409185 2024-04-07T16:45:56.709 INFO:tasks.workunit.client.0.smithi100.stdout:op 2252 completed, throughput=4.99MB/sec 2024-04-07T16:45:56.710 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-wJlkXBX0k7fLtU5 off=436530 len=1657851 2024-04-07T16:45:56.714 INFO:tasks.workunit.client.0.smithi100.stdout:op 2253 completed, throughput=4.99MB/sec 2024-04-07T16:45:56.714 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-csV8_Feg8He91kV off=3500928 len=1767424 2024-04-07T16:45:56.727 INFO:tasks.workunit.client.0.smithi100.stdout:op 2254 completed, throughput=5MB/sec 2024-04-07T16:45:56.727 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-vkNIABSKspIzfab off=4699423 len=1566664 2024-04-07T16:45:56.738 INFO:tasks.workunit.client.0.smithi100.stdout:op 2255 completed, throughput=5MB/sec 2024-04-07T16:45:56.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:45:56 smithi144 bash[66654]: cluster 2024-04-07T16:45:55.301859+0000 mgr.y (mgr.25175) 917 : cluster [DBG] pgmap v1559: 105 pgs: 105 active+clean; 126 GiB data, 6.1 GiB used, 709 GiB / 715 GiB avail; 3.0 MiB/s rd, 2.4 MiB/s wr, 5 op/s 2024-04-07T16:45:57.739 INFO:tasks.workunit.client.0.smithi100.stdout: 427: throughput=4.99MB/sec pending data=0 2024-04-07T16:45:57.739 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-IT_IgcXbgUaaB9C off=191122 len=899435 2024-04-07T16:45:57.743 INFO:tasks.workunit.client.0.smithi100.stdout:op 2256 completed, throughput=4.99MB/sec 2024-04-07T16:45:57.743 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-9yZEDb_P1YAreQn off=11464844 len=1979466 2024-04-07T16:45:57.747 INFO:tasks.workunit.client.0.smithi100.stdout:op 2257 completed, throughput=5MB/sec 2024-04-07T16:45:57.748 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-xCxKQsRM2Iu2pgT off=3466278 len=2081971 2024-04-07T16:45:57.766 INFO:tasks.workunit.client.0.smithi100.stdout:op 2258 completed, throughput=5MB/sec 2024-04-07T16:45:57.766 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-mf1LUEvGcgq7J-x off=3986932 len=1370133 2024-04-07T16:45:57.777 INFO:tasks.workunit.client.0.smithi100.stdout:op 2259 completed, throughput=5MB/sec 2024-04-07T16:45:58.694 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:45:58 smithi100 bash[74942]: cluster 2024-04-07T16:45:57.302799+0000 mgr.y (mgr.25175) 918 : cluster [DBG] pgmap v1560: 105 pgs: 105 active+clean; 126 GiB data, 6.1 GiB used, 709 GiB / 715 GiB avail; 3.0 MiB/s rd, 2.2 MiB/s wr, 5 op/s 2024-04-07T16:45:58.694 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:45:58 smithi100 bash[76202]: cluster 2024-04-07T16:45:57.302799+0000 mgr.y (mgr.25175) 918 : cluster [DBG] pgmap v1560: 105 pgs: 105 active+clean; 126 GiB data, 6.1 GiB used, 709 GiB / 715 GiB avail; 3.0 MiB/s rd, 2.2 MiB/s wr, 5 op/s 2024-04-07T16:45:58.694 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:45:58 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:45:58] "GET /metrics HTTP/1.1" 200 34070 "" "Prometheus/2.43.0" 2024-04-07T16:45:58.777 INFO:tasks.workunit.client.0.smithi100.stdout: 428: throughput=4.99MB/sec pending data=0 2024-04-07T16:45:58.777 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-wyfdKmT0uVoJT3P off=1027006 len=588867 2024-04-07T16:45:58.785 INFO:tasks.workunit.client.0.smithi100.stdout:op 2260 completed, throughput=4.99MB/sec 2024-04-07T16:45:58.785 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Qmegr3RA6gYo2cH off=10424007 len=1024907 2024-04-07T16:45:58.789 INFO:tasks.workunit.client.0.smithi100.stdout:op 2261 completed, throughput=4.99MB/sec 2024-04-07T16:45:58.790 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-votMl9oqY2giC0G off=10294141 len=2077442 2024-04-07T16:45:58.806 INFO:tasks.workunit.client.0.smithi100.stdout:op 2262 completed, throughput=5MB/sec 2024-04-07T16:45:58.807 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-th-dW-8hc4u38Yi off=13993757 len=875563 2024-04-07T16:45:58.813 INFO:tasks.workunit.client.0.smithi100.stdout:op 2263 completed, throughput=5MB/sec 2024-04-07T16:45:58.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:45:58 smithi144 bash[66654]: cluster 2024-04-07T16:45:57.302799+0000 mgr.y (mgr.25175) 918 : cluster [DBG] pgmap v1560: 105 pgs: 105 active+clean; 126 GiB data, 6.1 GiB used, 709 GiB / 715 GiB avail; 3.0 MiB/s rd, 2.2 MiB/s wr, 5 op/s 2024-04-07T16:45:59.814 INFO:tasks.workunit.client.0.smithi100.stdout: 429: throughput=4.99MB/sec pending data=0 2024-04-07T16:45:59.814 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-biWASqEK-CKaE0_ off=4406640 len=1366274 2024-04-07T16:45:59.819 INFO:tasks.workunit.client.0.smithi100.stdout:op 2264 completed, throughput=4.99MB/sec 2024-04-07T16:45:59.819 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-ahkwDdzHYzfPgsQ off=10308257 len=31441 2024-04-07T16:45:59.820 INFO:tasks.workunit.client.0.smithi100.stdout:op 2265 completed, throughput=4.99MB/sec 2024-04-07T16:45:59.820 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-VyWLP74SP-R3-OZ off=740466 len=1304343 2024-04-07T16:45:59.831 INFO:tasks.workunit.client.0.smithi100.stdout:op 2266 completed, throughput=4.99MB/sec 2024-04-07T16:45:59.831 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-u1KK1ruFnNMOS5L off=2377250 len=339795 2024-04-07T16:45:59.837 INFO:tasks.workunit.client.0.smithi100.stdout:op 2267 completed, throughput=5MB/sec 2024-04-07T16:45:59.837 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-dZ8OwkPFGMolOHS off=6965981 len=1035367 2024-04-07T16:45:59.845 INFO:tasks.workunit.client.0.smithi100.stdout:op 2268 completed, throughput=5MB/sec 2024-04-07T16:45:59.845 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-fX8jNAQlKwTmXSm off=1670793 len=1695339 2024-04-07T16:45:59.858 INFO:tasks.workunit.client.0.smithi100.stdout:op 2269 completed, throughput=5MB/sec 2024-04-07T16:46:00.694 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:46:00 smithi100 bash[74942]: cluster 2024-04-07T16:45:59.303631+0000 mgr.y (mgr.25175) 919 : cluster [DBG] pgmap v1561: 105 pgs: 105 active+clean; 126 GiB data, 6.1 GiB used, 709 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.3 MiB/s wr, 5 op/s 2024-04-07T16:46:00.694 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:46:00 smithi100 bash[76202]: cluster 2024-04-07T16:45:59.303631+0000 mgr.y (mgr.25175) 919 : cluster [DBG] pgmap v1561: 105 pgs: 105 active+clean; 126 GiB data, 6.1 GiB used, 709 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.3 MiB/s wr, 5 op/s 2024-04-07T16:46:00.858 INFO:tasks.workunit.client.0.smithi100.stdout: 430: throughput=4.99MB/sec pending data=0 2024-04-07T16:46:00.858 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-xGjLGUSDGiwYgkm off=977701 len=86880 2024-04-07T16:46:00.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:46:00 smithi144 bash[66654]: cluster 2024-04-07T16:45:59.303631+0000 mgr.y (mgr.25175) 919 : cluster [DBG] pgmap v1561: 105 pgs: 105 active+clean; 126 GiB data, 6.1 GiB used, 709 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.3 MiB/s wr, 5 op/s 2024-04-07T16:46:00.863 INFO:tasks.workunit.client.0.smithi100.stdout:op 2270 completed, throughput=4.99MB/sec 2024-04-07T16:46:00.863 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-VV7JqXlfa6f6HSA off=812416 len=1252943 2024-04-07T16:46:00.868 INFO:tasks.workunit.client.0.smithi100.stdout:op 2271 completed, throughput=4.99MB/sec 2024-04-07T16:46:00.868 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-RJ9KOSSKVrNQYjL off=617983 len=881133 2024-04-07T16:46:00.876 INFO:tasks.workunit.client.0.smithi100.stdout:op 2272 completed, throughput=4.99MB/sec 2024-04-07T16:46:00.876 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-B-L6KI5stGO_GV2 off=5118043 len=526691 2024-04-07T16:46:00.882 INFO:tasks.workunit.client.0.smithi100.stdout:op 2273 completed, throughput=5MB/sec 2024-04-07T16:46:00.882 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-17k70n4JKZf6PyC off=12291610 len=1486474 2024-04-07T16:46:00.886 INFO:tasks.workunit.client.0.smithi100.stdout:op 2274 completed, throughput=5MB/sec 2024-04-07T16:46:00.886 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-M7Wq2hM8j0e7xlH off=9521385 len=685731 2024-04-07T16:46:00.893 INFO:tasks.workunit.client.0.smithi100.stdout:op 2275 completed, throughput=5MB/sec 2024-04-07T16:46:01.894 INFO:tasks.workunit.client.0.smithi100.stdout: 431: throughput=4.99MB/sec pending data=0 2024-04-07T16:46:01.894 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-gHY_xW7KfWEYZoc off=7883098 len=2068543 2024-04-07T16:46:01.899 INFO:tasks.workunit.client.0.smithi100.stdout:op 2276 completed, throughput=4.99MB/sec 2024-04-07T16:46:01.899 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-_rtiz9GXF41CQoJ off=18528017 len=913139 2024-04-07T16:46:01.909 INFO:tasks.workunit.client.0.smithi100.stdout:op 2277 completed, throughput=5MB/sec 2024-04-07T16:46:01.909 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-ds6JlQrAvLprMLh off=6085008 len=1039352 2024-04-07T16:46:01.918 INFO:tasks.workunit.client.0.smithi100.stdout:op 2278 completed, throughput=5MB/sec 2024-04-07T16:46:01.918 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-YYj5992epAEfXDH off=902469 len=1963821 2024-04-07T16:46:01.920 INFO:tasks.workunit.client.0.smithi100.stdout:op 2279 completed, throughput=5MB/sec 2024-04-07T16:46:02.608 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:46:02 smithi144 bash[66654]: cluster 2024-04-07T16:46:01.304489+0000 mgr.y (mgr.25175) 920 : cluster [DBG] pgmap v1562: 105 pgs: 105 active+clean; 126 GiB data, 6.1 GiB used, 709 GiB / 715 GiB avail; 2.6 MiB/s rd, 2.1 MiB/s wr, 5 op/s 2024-04-07T16:46:02.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:46:02 smithi144 bash[66654]: audit 2024-04-07T16:46:01.767807+0000 mon.a (mon.0) 2109 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:46:02.609 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:46:02 smithi144 bash[95233]: ts=2024-04-07T16:46:02.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:46:02.921 INFO:tasks.workunit.client.0.smithi100.stdout: 432: throughput=4.99MB/sec pending data=0 2024-04-07T16:46:02.921 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-2jjuQ1nOWkL2jB4 off=14646918 len=1883136 2024-04-07T16:46:02.925 INFO:tasks.workunit.client.0.smithi100.stdout:op 2280 completed, throughput=4.99MB/sec 2024-04-07T16:46:02.925 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-QYHcRTRiPYvzojK off=3021243 len=348742 2024-04-07T16:46:02.927 INFO:tasks.workunit.client.0.smithi100.stdout:op 2281 completed, throughput=4.99MB/sec 2024-04-07T16:46:02.928 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-koBXxT9P5p25CH- off=1934764 len=1377507 2024-04-07T16:46:02.930 INFO:tasks.workunit.client.0.smithi100.stdout:op 2282 completed, throughput=5MB/sec 2024-04-07T16:46:02.930 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-2pJc6sIfMhZxBF4 off=1359771 len=1004047 2024-04-07T16:46:02.938 INFO:tasks.workunit.client.0.smithi100.stdout:op 2283 completed, throughput=5MB/sec 2024-04-07T16:46:02.938 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-jnBP3VG85xrvWYV off=12484822 len=310456 2024-04-07T16:46:02.939 INFO:tasks.workunit.client.0.smithi100.stdout:op 2284 completed, throughput=5MB/sec 2024-04-07T16:46:02.943 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:46:02 smithi100 bash[74942]: cluster 2024-04-07T16:46:01.304489+0000 mgr.y (mgr.25175) 920 : cluster [DBG] pgmap v1562: 105 pgs: 105 active+clean; 126 GiB data, 6.1 GiB used, 709 GiB / 715 GiB avail; 2.6 MiB/s rd, 2.1 MiB/s wr, 5 op/s 2024-04-07T16:46:02.944 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:46:02 smithi100 bash[74942]: audit 2024-04-07T16:46:01.767807+0000 mon.a (mon.0) 2109 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:46:02.944 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:46:02 smithi100 bash[76202]: cluster 2024-04-07T16:46:01.304489+0000 mgr.y (mgr.25175) 920 : cluster [DBG] pgmap v1562: 105 pgs: 105 active+clean; 126 GiB data, 6.1 GiB used, 709 GiB / 715 GiB avail; 2.6 MiB/s rd, 2.1 MiB/s wr, 5 op/s 2024-04-07T16:46:02.944 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:46:02 smithi100 bash[76202]: audit 2024-04-07T16:46:01.767807+0000 mon.a (mon.0) 2109 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:46:03.940 INFO:tasks.workunit.client.0.smithi100.stdout: 433: throughput=4.99MB/sec pending data=0 2024-04-07T16:46:03.940 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-NpL_P_SKbbbK7ZU off=5248956 len=64812 2024-04-07T16:46:03.944 INFO:tasks.workunit.client.0.smithi100.stdout:op 2285 completed, throughput=4.99MB/sec 2024-04-07T16:46:03.944 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Kp-XB0qyddUL8j2 off=11930587 len=454306 2024-04-07T16:46:03.947 INFO:tasks.workunit.client.0.smithi100.stdout:op 2286 completed, throughput=4.99MB/sec 2024-04-07T16:46:03.947 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-0AtHj8HHQENye5A off=5518302 len=1984820 2024-04-07T16:46:03.952 INFO:tasks.workunit.client.0.smithi100.stdout:op 2287 completed, throughput=4.99MB/sec 2024-04-07T16:46:03.952 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-N4tgMbfveeiQMAl off=9551535 len=117129 2024-04-07T16:46:03.957 INFO:tasks.workunit.client.0.smithi100.stdout:op 2288 completed, throughput=4.99MB/sec 2024-04-07T16:46:03.958 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-L-0NUW03BLE1-RM off=1392675 len=1365241 2024-04-07T16:46:03.970 INFO:tasks.workunit.client.0.smithi100.stdout:op 2289 completed, throughput=5MB/sec 2024-04-07T16:46:03.970 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-rQT6pcUe-2gyt2K off=14295075 len=1031683 2024-04-07T16:46:03.973 INFO:tasks.workunit.client.0.smithi100.stdout:op 2290 completed, throughput=5MB/sec 2024-04-07T16:46:03.973 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-FYXKQfLLCG6Efsq off=147088 len=656625 2024-04-07T16:46:03.979 INFO:tasks.workunit.client.0.smithi100.stdout:op 2291 completed, throughput=5MB/sec 2024-04-07T16:46:04.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:46:04 smithi144 bash[66654]: cluster 2024-04-07T16:46:03.305668+0000 mgr.y (mgr.25175) 921 : cluster [DBG] pgmap v1563: 105 pgs: 105 active+clean; 126 GiB data, 6.2 GiB used, 709 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.8 MiB/s wr, 5 op/s 2024-04-07T16:46:04.944 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:46:04 smithi100 bash[74942]: cluster 2024-04-07T16:46:03.305668+0000 mgr.y (mgr.25175) 921 : cluster [DBG] pgmap v1563: 105 pgs: 105 active+clean; 126 GiB data, 6.2 GiB used, 709 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.8 MiB/s wr, 5 op/s 2024-04-07T16:46:04.944 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:46:04 smithi100 bash[76202]: cluster 2024-04-07T16:46:03.305668+0000 mgr.y (mgr.25175) 921 : cluster [DBG] pgmap v1563: 105 pgs: 105 active+clean; 126 GiB data, 6.2 GiB used, 709 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.8 MiB/s wr, 5 op/s 2024-04-07T16:46:04.980 INFO:tasks.workunit.client.0.smithi100.stdout: 434: throughput=4.99MB/sec pending data=0 2024-04-07T16:46:04.980 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-qvErZ9hmZSQNRF4 off=3127028 len=1505372 2024-04-07T16:46:04.994 INFO:tasks.workunit.client.0.smithi100.stdout:op 2292 completed, throughput=4.99MB/sec 2024-04-07T16:46:04.994 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-DH1TmwBn0LZx5YF off=4969671 len=1942368 2024-04-07T16:46:05.008 INFO:tasks.workunit.client.0.smithi100.stdout:op 2293 completed, throughput=5MB/sec 2024-04-07T16:46:05.008 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-B8HWS8ewD91xN5i off=2025676 len=1589591 2024-04-07T16:46:05.013 INFO:tasks.workunit.client.0.smithi100.stdout:op 2294 completed, throughput=5MB/sec 2024-04-07T16:46:06.014 INFO:tasks.workunit.client.0.smithi100.stdout: 435: throughput=4.99MB/sec pending data=0 2024-04-07T16:46:06.014 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-qF1g3pcYk52_00b off=3615123 len=1303736 2024-04-07T16:46:06.024 INFO:tasks.workunit.client.0.smithi100.stdout:op 2295 completed, throughput=4.99MB/sec 2024-04-07T16:46:06.024 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Td01pLsGUgDIw8w off=1962939 len=1380597 2024-04-07T16:46:06.028 INFO:tasks.workunit.client.0.smithi100.stdout:op 2296 completed, throughput=4.99MB/sec 2024-04-07T16:46:06.028 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-TXP7cmlPRTBMTyd off=2554833 len=2004370 2024-04-07T16:46:06.033 INFO:tasks.workunit.client.0.smithi100.stdout:op 2297 completed, throughput=5MB/sec 2024-04-07T16:46:06.033 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-yEv5YEiZwppFViP off=64402 len=774033 2024-04-07T16:46:06.037 INFO:tasks.workunit.client.0.smithi100.stdout:op 2298 completed, throughput=5MB/sec 2024-04-07T16:46:06.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:46:06 smithi144 bash[66654]: cluster 2024-04-07T16:46:05.306661+0000 mgr.y (mgr.25175) 922 : cluster [DBG] pgmap v1564: 105 pgs: 105 active+clean; 126 GiB data, 6.2 GiB used, 709 GiB / 715 GiB avail; 2.3 MiB/s rd, 2.6 MiB/s wr, 4 op/s 2024-04-07T16:46:06.944 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:46:06 smithi100 bash[74942]: cluster 2024-04-07T16:46:05.306661+0000 mgr.y (mgr.25175) 922 : cluster [DBG] pgmap v1564: 105 pgs: 105 active+clean; 126 GiB data, 6.2 GiB used, 709 GiB / 715 GiB avail; 2.3 MiB/s rd, 2.6 MiB/s wr, 4 op/s 2024-04-07T16:46:06.944 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:46:06 smithi100 bash[76202]: cluster 2024-04-07T16:46:05.306661+0000 mgr.y (mgr.25175) 922 : cluster [DBG] pgmap v1564: 105 pgs: 105 active+clean; 126 GiB data, 6.2 GiB used, 709 GiB / 715 GiB avail; 2.3 MiB/s rd, 2.6 MiB/s wr, 4 op/s 2024-04-07T16:46:07.037 INFO:tasks.workunit.client.0.smithi100.stdout: 436: throughput=4.99MB/sec pending data=0 2024-04-07T16:46:07.037 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-FYV-YW6W7OT50_q off=3127823 len=243897 2024-04-07T16:46:07.038 INFO:tasks.workunit.client.0.smithi100.stdout:op 2299 completed, throughput=4.99MB/sec 2024-04-07T16:46:07.038 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-uLYmsRJtfR-j8ca off=5051776 len=122252 2024-04-07T16:46:07.040 INFO:tasks.workunit.client.0.smithi100.stdout:op 2300 completed, throughput=4.99MB/sec 2024-04-07T16:46:07.040 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Z9pKIid_WRqorQw off=16027391 len=1327125 2024-04-07T16:46:07.050 INFO:tasks.workunit.client.0.smithi100.stdout:op 2301 completed, throughput=4.99MB/sec 2024-04-07T16:46:07.050 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-gcYq-jeQ_S_bKNv off=9356393 len=1157029 2024-04-07T16:46:07.053 INFO:tasks.workunit.client.0.smithi100.stdout:op 2302 completed, throughput=5MB/sec 2024-04-07T16:46:07.053 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-ltzecHnhYHYVXm5 off=3487649 len=1770152 2024-04-07T16:46:07.057 INFO:tasks.workunit.client.0.smithi100.stdout:op 2303 completed, throughput=5MB/sec 2024-04-07T16:46:07.057 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-flEVYSJRAfZZGot off=6617971 len=1250712 2024-04-07T16:46:07.066 INFO:tasks.workunit.client.0.smithi100.stdout:op 2304 completed, throughput=5MB/sec 2024-04-07T16:46:08.066 INFO:tasks.workunit.client.0.smithi100.stdout: 437: throughput=4.99MB/sec pending data=0 2024-04-07T16:46:08.067 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-YHy64xjywMIQqV1 off=1555146 len=90423 2024-04-07T16:46:08.071 INFO:tasks.workunit.client.0.smithi100.stdout:op 2305 completed, throughput=4.99MB/sec 2024-04-07T16:46:08.071 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-kKLgdEncp3mLHPl off=3710339 len=1714250 2024-04-07T16:46:08.075 INFO:tasks.workunit.client.0.smithi100.stdout:op 2306 completed, throughput=4.99MB/sec 2024-04-07T16:46:08.076 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-LgE32ou4aUIKcl6 off=731790 len=1958930 2024-04-07T16:46:08.080 INFO:tasks.workunit.client.0.smithi100.stdout:op 2307 completed, throughput=5MB/sec 2024-04-07T16:46:08.080 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-cJLoMtfGR9tFDJd off=6107568 len=691205 2024-04-07T16:46:08.082 INFO:tasks.workunit.client.0.smithi100.stdout:op 2308 completed, throughput=5MB/sec 2024-04-07T16:46:08.083 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-KwDBW6FBHVAvZsW off=6199096 len=1007735 2024-04-07T16:46:08.093 INFO:tasks.workunit.client.0.smithi100.stdout:op 2309 completed, throughput=5MB/sec 2024-04-07T16:46:08.608 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:46:08 smithi144 bash[66654]: cluster 2024-04-07T16:46:07.307727+0000 mgr.y (mgr.25175) 923 : cluster [DBG] pgmap v1565: 105 pgs: 105 active+clean; 126 GiB data, 6.2 GiB used, 709 GiB / 715 GiB avail; 2.3 MiB/s rd, 2.6 MiB/s wr, 4 op/s 2024-04-07T16:46:08.694 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:46:08 smithi100 bash[74942]: cluster 2024-04-07T16:46:07.307727+0000 mgr.y (mgr.25175) 923 : cluster [DBG] pgmap v1565: 105 pgs: 105 active+clean; 126 GiB data, 6.2 GiB used, 709 GiB / 715 GiB avail; 2.3 MiB/s rd, 2.6 MiB/s wr, 4 op/s 2024-04-07T16:46:08.694 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:46:08 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:46:08] "GET /metrics HTTP/1.1" 200 34069 "" "Prometheus/2.43.0" 2024-04-07T16:46:08.694 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:46:08 smithi100 bash[76202]: cluster 2024-04-07T16:46:07.307727+0000 mgr.y (mgr.25175) 923 : cluster [DBG] pgmap v1565: 105 pgs: 105 active+clean; 126 GiB data, 6.2 GiB used, 709 GiB / 715 GiB avail; 2.3 MiB/s rd, 2.6 MiB/s wr, 4 op/s 2024-04-07T16:46:09.093 INFO:tasks.workunit.client.0.smithi100.stdout: 438: throughput=4.99MB/sec pending data=0 2024-04-07T16:46:09.093 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-AEJ68r3XtCHaZft off=542339 len=167010 2024-04-07T16:46:09.098 INFO:tasks.workunit.client.0.smithi100.stdout:op 2310 completed, throughput=4.99MB/sec 2024-04-07T16:46:09.098 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-V1L2UJHyOtI6krG off=2331106 len=315992 2024-04-07T16:46:09.100 INFO:tasks.workunit.client.0.smithi100.stdout:op 2311 completed, throughput=4.99MB/sec 2024-04-07T16:46:09.100 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-d_S_j67Z4ioqLLP off=11945192 len=275100 2024-04-07T16:46:09.105 INFO:tasks.workunit.client.0.smithi100.stdout:op 2312 completed, throughput=4.99MB/sec 2024-04-07T16:46:09.105 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-QQUciSAiSCHj5ss off=3381321 len=1207773 2024-04-07T16:46:09.110 INFO:tasks.workunit.client.0.smithi100.stdout:op 2313 completed, throughput=4.99MB/sec 2024-04-07T16:46:09.110 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-VWZ-_KNLK8rQeqY off=9671762 len=724678 2024-04-07T16:46:09.112 INFO:tasks.workunit.client.0.smithi100.stdout:op 2314 completed, throughput=5MB/sec 2024-04-07T16:46:09.112 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-xM5FkbZ78I14qd1 off=7706293 len=960273 2024-04-07T16:46:09.121 INFO:tasks.workunit.client.0.smithi100.stdout:op 2315 completed, throughput=5MB/sec 2024-04-07T16:46:09.121 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-8n0fWmuZhN1ckab off=5126503 len=896626 2024-04-07T16:46:09.129 INFO:tasks.workunit.client.0.smithi100.stdout:op 2316 completed, throughput=5MB/sec 2024-04-07T16:46:09.129 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-zpdz_vcgM0ZWLVj off=9561877 len=1890862 2024-04-07T16:46:09.144 INFO:tasks.workunit.client.0.smithi100.stdout:op 2317 completed, throughput=5MB/sec 2024-04-07T16:46:10.144 INFO:tasks.workunit.client.0.smithi100.stdout: 439: throughput=4.99MB/sec pending data=0 2024-04-07T16:46:10.144 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-JSqSf9VACVfYUAp off=12444108 len=1769445 2024-04-07T16:46:10.158 INFO:tasks.workunit.client.0.smithi100.stdout:op 2318 completed, throughput=5MB/sec 2024-04-07T16:46:10.158 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-_Qr3I5vmFOIFSRn off=2051207 len=910484 2024-04-07T16:46:10.167 INFO:tasks.workunit.client.0.smithi100.stdout:op 2319 completed, throughput=5MB/sec 2024-04-07T16:46:10.167 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-9ekzklCKS1V6Eav off=11980678 len=1525635 2024-04-07T16:46:10.171 INFO:tasks.workunit.client.0.smithi100.stdout:op 2320 completed, throughput=5MB/sec 2024-04-07T16:46:10.608 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:46:10 smithi144 bash[66654]: cluster 2024-04-07T16:46:09.308626+0000 mgr.y (mgr.25175) 924 : cluster [DBG] pgmap v1566: 105 pgs: 105 active+clean; 126 GiB data, 6.2 GiB used, 709 GiB / 715 GiB avail; 2.3 MiB/s rd, 2.5 MiB/s wr, 4 op/s 2024-04-07T16:46:10.694 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:46:10 smithi100 bash[74942]: cluster 2024-04-07T16:46:09.308626+0000 mgr.y (mgr.25175) 924 : cluster [DBG] pgmap v1566: 105 pgs: 105 active+clean; 126 GiB data, 6.2 GiB used, 709 GiB / 715 GiB avail; 2.3 MiB/s rd, 2.5 MiB/s wr, 4 op/s 2024-04-07T16:46:10.694 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:46:10 smithi100 bash[76202]: cluster 2024-04-07T16:46:09.308626+0000 mgr.y (mgr.25175) 924 : cluster [DBG] pgmap v1566: 105 pgs: 105 active+clean; 126 GiB data, 6.2 GiB used, 709 GiB / 715 GiB avail; 2.3 MiB/s rd, 2.5 MiB/s wr, 4 op/s 2024-04-07T16:46:11.171 INFO:tasks.workunit.client.0.smithi100.stdout: 440: throughput=4.99MB/sec pending data=0 2024-04-07T16:46:11.171 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-BiE6JoZa8Me9oJn off=13552123 len=621290 2024-04-07T16:46:11.180 INFO:tasks.workunit.client.0.smithi100.stdout:op 2321 completed, throughput=4.99MB/sec 2024-04-07T16:46:11.180 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-N_6M9x8TRLbuVSK off=1912193 len=632261 2024-04-07T16:46:11.187 INFO:tasks.workunit.client.0.smithi100.stdout:op 2322 completed, throughput=4.99MB/sec 2024-04-07T16:46:11.187 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-UjVxoDEVxTM7MRZ off=6084340 len=2073150 2024-04-07T16:46:11.192 INFO:tasks.workunit.client.0.smithi100.stdout:op 2323 completed, throughput=5MB/sec 2024-04-07T16:46:11.192 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-AM9ETdwTmYCx9Bf off=8691396 len=942411 2024-04-07T16:46:11.200 INFO:tasks.workunit.client.0.smithi100.stdout:op 2324 completed, throughput=5MB/sec 2024-04-07T16:46:11.200 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-7ArNpK_YrIuQDX- off=7937820 len=524197 2024-04-07T16:46:11.201 INFO:tasks.workunit.client.0.smithi100.stdout:op 2325 completed, throughput=5MB/sec 2024-04-07T16:46:11.201 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-UYaMpeyxlmp74_q off=7348737 len=1784197 2024-04-07T16:46:11.204 INFO:tasks.workunit.client.0.smithi100.stdout:op 2326 completed, throughput=5MB/sec 2024-04-07T16:46:12.204 INFO:tasks.workunit.client.0.smithi100.stdout: 441: throughput=4.99MB/sec pending data=0 2024-04-07T16:46:12.204 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-2M2rVB6OfZdF_Kn off=13531458 len=704631 2024-04-07T16:46:12.212 INFO:tasks.workunit.client.0.smithi100.stdout:op 2327 completed, throughput=4.99MB/sec 2024-04-07T16:46:12.212 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-BgUagh3dXFQnrKQ off=6159875 len=629706 2024-04-07T16:46:12.213 INFO:tasks.workunit.client.0.smithi100.stdout:op 2328 completed, throughput=5MB/sec 2024-04-07T16:46:12.213 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-ZFePGTyeF6BmDn1 off=20854992 len=1522193 2024-04-07T16:46:12.218 INFO:tasks.workunit.client.0.smithi100.stdout:op 2329 completed, throughput=5MB/sec 2024-04-07T16:46:12.219 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-aHNOX_x73hzFhjR off=5380796 len=1017513 2024-04-07T16:46:12.228 INFO:tasks.workunit.client.0.smithi100.stdout:op 2330 completed, throughput=5MB/sec 2024-04-07T16:46:12.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:46:12 smithi144 bash[66654]: cluster 2024-04-07T16:46:11.309588+0000 mgr.y (mgr.25175) 925 : cluster [DBG] pgmap v1567: 105 pgs: 105 active+clean; 126 GiB data, 6.2 GiB used, 709 GiB / 715 GiB avail; 2.3 MiB/s rd, 2.4 MiB/s wr, 4 op/s 2024-04-07T16:46:12.609 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:46:12 smithi144 bash[95233]: ts=2024-04-07T16:46:12.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:46:12.694 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:46:12 smithi100 bash[74942]: cluster 2024-04-07T16:46:11.309588+0000 mgr.y (mgr.25175) 925 : cluster [DBG] pgmap v1567: 105 pgs: 105 active+clean; 126 GiB data, 6.2 GiB used, 709 GiB / 715 GiB avail; 2.3 MiB/s rd, 2.4 MiB/s wr, 4 op/s 2024-04-07T16:46:12.694 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:46:12 smithi100 bash[76202]: cluster 2024-04-07T16:46:11.309588+0000 mgr.y (mgr.25175) 925 : cluster [DBG] pgmap v1567: 105 pgs: 105 active+clean; 126 GiB data, 6.2 GiB used, 709 GiB / 715 GiB avail; 2.3 MiB/s rd, 2.4 MiB/s wr, 4 op/s 2024-04-07T16:46:13.228 INFO:tasks.workunit.client.0.smithi100.stdout: 442: throughput=4.99MB/sec pending data=0 2024-04-07T16:46:13.228 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-ukvLCDTz4uQqoEE off=3450547 len=215539 2024-04-07T16:46:13.233 INFO:tasks.workunit.client.0.smithi100.stdout:op 2331 completed, throughput=4.99MB/sec 2024-04-07T16:46:13.233 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-AwQDDqLxetk8bcG off=2683439 len=1496050 2024-04-07T16:46:13.245 INFO:tasks.workunit.client.0.smithi100.stdout:op 2332 completed, throughput=4.99MB/sec 2024-04-07T16:46:13.245 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-UDXeVJ2uu1lyqqB off=382070 len=1472545 2024-04-07T16:46:13.247 INFO:tasks.workunit.client.0.smithi100.stdout:op 2333 completed, throughput=5MB/sec 2024-04-07T16:46:13.248 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-gq84WXkoF4azY1J off=15986795 len=1778450 2024-04-07T16:46:13.253 INFO:tasks.workunit.client.0.smithi100.stdout:op 2334 completed, throughput=5MB/sec 2024-04-07T16:46:13.254 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-MQHDNJWqKJ7DM08 off=7869385 len=836748 2024-04-07T16:46:13.262 INFO:tasks.workunit.client.0.smithi100.stdout:op 2335 completed, throughput=5MB/sec 2024-04-07T16:46:14.262 INFO:tasks.workunit.client.0.smithi100.stdout: 443: throughput=4.99MB/sec pending data=0 2024-04-07T16:46:14.262 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-YqvsOR1tlCwcR3d off=1583273 len=1674589 2024-04-07T16:46:14.276 INFO:tasks.workunit.client.0.smithi100.stdout:op 2336 completed, throughput=4.99MB/sec 2024-04-07T16:46:14.276 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-fPXh8bGXUDKMJkL off=2700668 len=1339290 2024-04-07T16:46:14.281 INFO:tasks.workunit.client.0.smithi100.stdout:op 2337 completed, throughput=5MB/sec 2024-04-07T16:46:14.281 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-oVLwwbSwV-x9-Ue off=1749224 len=1259473 2024-04-07T16:46:14.286 INFO:tasks.workunit.client.0.smithi100.stdout:op 2338 completed, throughput=5MB/sec 2024-04-07T16:46:14.286 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-cr1pKX4aISt8teQ off=4714147 len=472854 2024-04-07T16:46:14.292 INFO:tasks.workunit.client.0.smithi100.stdout:op 2339 completed, throughput=5MB/sec 2024-04-07T16:46:14.293 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-WRoxIQJ5VHyETbg off=1633479 len=1919721 2024-04-07T16:46:14.298 INFO:tasks.workunit.client.0.smithi100.stdout:op 2340 completed, throughput=5MB/sec 2024-04-07T16:46:14.694 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:46:14 smithi100 bash[74942]: cluster 2024-04-07T16:46:13.310844+0000 mgr.y (mgr.25175) 926 : cluster [DBG] pgmap v1568: 105 pgs: 105 active+clean; 126 GiB data, 6.2 GiB used, 709 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.8 MiB/s wr, 5 op/s 2024-04-07T16:46:14.694 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:46:14 smithi100 bash[76202]: cluster 2024-04-07T16:46:13.310844+0000 mgr.y (mgr.25175) 926 : cluster [DBG] pgmap v1568: 105 pgs: 105 active+clean; 126 GiB data, 6.2 GiB used, 709 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.8 MiB/s wr, 5 op/s 2024-04-07T16:46:14.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:46:14 smithi144 bash[66654]: cluster 2024-04-07T16:46:13.310844+0000 mgr.y (mgr.25175) 926 : cluster [DBG] pgmap v1568: 105 pgs: 105 active+clean; 126 GiB data, 6.2 GiB used, 709 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.8 MiB/s wr, 5 op/s 2024-04-07T16:46:15.298 INFO:tasks.workunit.client.0.smithi100.stdout: 444: throughput=4.99MB/sec pending data=0 2024-04-07T16:46:15.298 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-N1hHcDKSPqee0_D off=8850107 len=110747 2024-04-07T16:46:15.299 INFO:tasks.workunit.client.0.smithi100.stdout:op 2341 completed, throughput=4.99MB/sec 2024-04-07T16:46:15.299 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-BjPFpR7r47-TLY- off=3757423 len=855974 2024-04-07T16:46:15.302 INFO:tasks.workunit.client.0.smithi100.stdout:op 2342 completed, throughput=4.99MB/sec 2024-04-07T16:46:15.302 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Im_LMPBAxY4L7UZ off=3720487 len=1864506 2024-04-07T16:46:15.306 INFO:tasks.workunit.client.0.smithi100.stdout:op 2343 completed, throughput=5MB/sec 2024-04-07T16:46:15.306 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-M74Vp85UsiYlUia off=2428978 len=1213518 2024-04-07T16:46:15.317 INFO:tasks.workunit.client.0.smithi100.stdout:op 2344 completed, throughput=5MB/sec 2024-04-07T16:46:16.317 INFO:tasks.workunit.client.0.smithi100.stdout: 445: throughput=4.99MB/sec pending data=0 2024-04-07T16:46:16.317 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-ekJHp9zc356s9SH off=8235169 len=1532849 2024-04-07T16:46:16.330 INFO:tasks.workunit.client.0.smithi100.stdout:op 2345 completed, throughput=4.99MB/sec 2024-04-07T16:46:16.330 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-uqmJjuuV100B13G off=15638345 len=587953 2024-04-07T16:46:16.332 INFO:tasks.workunit.client.0.smithi100.stdout:op 2346 completed, throughput=4.99MB/sec 2024-04-07T16:46:16.332 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-bTbjmV6AGDTqtrn off=11762936 len=465297 2024-04-07T16:46:16.339 INFO:tasks.workunit.client.0.smithi100.stdout:op 2347 completed, throughput=5MB/sec 2024-04-07T16:46:16.339 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Z-vWdSZIoRARWpd off=2717878 len=866527 2024-04-07T16:46:16.347 INFO:tasks.workunit.client.0.smithi100.stdout:op 2348 completed, throughput=5MB/sec 2024-04-07T16:46:16.347 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-HfYsntZ2qlWk8d8 off=1419254 len=435149 2024-04-07T16:46:16.348 INFO:tasks.workunit.client.0.smithi100.stdout:op 2349 completed, throughput=5MB/sec 2024-04-07T16:46:16.348 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-aIuiLLWPerETn4a off=2417136 len=1479297 2024-04-07T16:46:16.361 INFO:tasks.workunit.client.0.smithi100.stdout:op 2350 completed, throughput=5MB/sec 2024-04-07T16:46:16.694 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:46:16 smithi100 bash[74942]: cluster 2024-04-07T16:46:15.312005+0000 mgr.y (mgr.25175) 927 : cluster [DBG] pgmap v1569: 105 pgs: 105 active+clean; 126 GiB data, 6.3 GiB used, 709 GiB / 715 GiB avail; 2.6 MiB/s rd, 2.2 MiB/s wr, 4 op/s 2024-04-07T16:46:16.694 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:46:16 smithi100 bash[76202]: cluster 2024-04-07T16:46:15.312005+0000 mgr.y (mgr.25175) 927 : cluster [DBG] pgmap v1569: 105 pgs: 105 active+clean; 126 GiB data, 6.3 GiB used, 709 GiB / 715 GiB avail; 2.6 MiB/s rd, 2.2 MiB/s wr, 4 op/s 2024-04-07T16:46:16.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:46:16 smithi144 bash[66654]: cluster 2024-04-07T16:46:15.312005+0000 mgr.y (mgr.25175) 927 : cluster [DBG] pgmap v1569: 105 pgs: 105 active+clean; 126 GiB data, 6.3 GiB used, 709 GiB / 715 GiB avail; 2.6 MiB/s rd, 2.2 MiB/s wr, 4 op/s 2024-04-07T16:46:17.361 INFO:tasks.workunit.client.0.smithi100.stdout: 446: throughput=4.99MB/sec pending data=0 2024-04-07T16:46:17.361 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-7R7o02XEw7XBHY1 off=5976075 len=1392236 2024-04-07T16:46:17.373 INFO:tasks.workunit.client.0.smithi100.stdout:op 2351 completed, throughput=4.99MB/sec 2024-04-07T16:46:17.373 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-6hLLj2KlBJ8GRHn off=4234042 len=643529 2024-04-07T16:46:17.381 INFO:tasks.workunit.client.0.smithi100.stdout:op 2352 completed, throughput=4.99MB/sec 2024-04-07T16:46:17.381 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-rL3l4QFyaCoUqXr off=10800377 len=1523162 2024-04-07T16:46:17.392 INFO:tasks.workunit.client.0.smithi100.stdout:op 2353 completed, throughput=5MB/sec 2024-04-07T16:46:17.393 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-USszGY3_PEKgRjV off=20193832 len=1243190 2024-04-07T16:46:17.403 INFO:tasks.workunit.client.0.smithi100.stdout:op 2354 completed, throughput=5MB/sec 2024-04-07T16:46:17.403 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-oJkWTrgkBPEaeAK off=8245842 len=1659400 2024-04-07T16:46:17.407 INFO:tasks.workunit.client.0.smithi100.stdout:op 2355 completed, throughput=5MB/sec 2024-04-07T16:46:17.694 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:46:17 smithi100 bash[74942]: audit 2024-04-07T16:46:16.768277+0000 mon.a (mon.0) 2110 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:46:17.694 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:46:17 smithi100 bash[76202]: audit 2024-04-07T16:46:16.768277+0000 mon.a (mon.0) 2110 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:46:17.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:46:17 smithi144 bash[66654]: audit 2024-04-07T16:46:16.768277+0000 mon.a (mon.0) 2110 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:46:18.408 INFO:tasks.workunit.client.0.smithi100.stdout: 447: throughput=4.99MB/sec pending data=0 2024-04-07T16:46:18.408 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-z1n5lzJQORPVoHO off=7352026 len=1681520 2024-04-07T16:46:18.422 INFO:tasks.workunit.client.0.smithi100.stdout:op 2356 completed, throughput=5MB/sec 2024-04-07T16:46:18.422 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-ThAn-Aj5epCzqmL off=11301615 len=1025428 2024-04-07T16:46:18.426 INFO:tasks.workunit.client.0.smithi100.stdout:op 2357 completed, throughput=5MB/sec 2024-04-07T16:46:18.426 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-mliOy_8hKeZ0RTU off=7673203 len=1687442 2024-04-07T16:46:18.429 INFO:tasks.workunit.client.0.smithi100.stdout:op 2358 completed, throughput=5MB/sec 2024-04-07T16:46:18.694 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:46:18 smithi100 bash[74942]: cluster 2024-04-07T16:46:17.313019+0000 mgr.y (mgr.25175) 928 : cluster [DBG] pgmap v1570: 105 pgs: 105 active+clean; 126 GiB data, 6.3 GiB used, 709 GiB / 715 GiB avail; 2.8 MiB/s rd, 2.2 MiB/s wr, 4 op/s 2024-04-07T16:46:18.695 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:46:18 smithi100 bash[76202]: cluster 2024-04-07T16:46:17.313019+0000 mgr.y (mgr.25175) 928 : cluster [DBG] pgmap v1570: 105 pgs: 105 active+clean; 126 GiB data, 6.3 GiB used, 709 GiB / 715 GiB avail; 2.8 MiB/s rd, 2.2 MiB/s wr, 4 op/s 2024-04-07T16:46:18.695 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:46:18 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:46:18] "GET /metrics HTTP/1.1" 200 34071 "" "Prometheus/2.43.0" 2024-04-07T16:46:18.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:46:18 smithi144 bash[66654]: cluster 2024-04-07T16:46:17.313019+0000 mgr.y (mgr.25175) 928 : cluster [DBG] pgmap v1570: 105 pgs: 105 active+clean; 126 GiB data, 6.3 GiB used, 709 GiB / 715 GiB avail; 2.8 MiB/s rd, 2.2 MiB/s wr, 4 op/s 2024-04-07T16:46:19.430 INFO:tasks.workunit.client.0.smithi100.stdout: 448: throughput=4.99MB/sec pending data=0 2024-04-07T16:46:19.430 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-W66l64E_GPjDrvH off=7657639 len=2086504 2024-04-07T16:46:19.437 INFO:tasks.workunit.client.0.smithi100.stdout:op 2359 completed, throughput=4.99MB/sec 2024-04-07T16:46:19.437 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-JVlUB4Tg4jJT0Xx off=13393409 len=539517 2024-04-07T16:46:19.444 INFO:tasks.workunit.client.0.smithi100.stdout:op 2360 completed, throughput=5MB/sec 2024-04-07T16:46:19.444 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-2F9qLW2cVrwbnGW off=211040 len=664097 2024-04-07T16:46:19.446 INFO:tasks.workunit.client.0.smithi100.stdout:op 2361 completed, throughput=5MB/sec 2024-04-07T16:46:19.446 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-T71VlT6UcxQk5C6 off=15034541 len=1591289 2024-04-07T16:46:19.450 INFO:tasks.workunit.client.0.smithi100.stdout:op 2362 completed, throughput=5MB/sec 2024-04-07T16:46:19.450 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-V4y27vAojx2cAOi off=5328157 len=448692 2024-04-07T16:46:19.456 INFO:tasks.workunit.client.0.smithi100.stdout:op 2363 completed, throughput=5MB/sec 2024-04-07T16:46:20.457 INFO:tasks.workunit.client.0.smithi100.stdout: 449: throughput=4.99MB/sec pending data=0 2024-04-07T16:46:20.457 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-sn71oPm3ZPd3TiQ off=5839114 len=1135447 2024-04-07T16:46:20.460 INFO:tasks.workunit.client.0.smithi100.stdout:op 2364 completed, throughput=4.99MB/sec 2024-04-07T16:46:20.460 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-QTgk124w9FyKMsJ off=3625577 len=1146896 2024-04-07T16:46:20.471 INFO:tasks.workunit.client.0.smithi100.stdout:op 2365 completed, throughput=4.99MB/sec 2024-04-07T16:46:20.471 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-VMFYdUiz_ucslkB off=15933925 len=378071 2024-04-07T16:46:20.477 INFO:tasks.workunit.client.0.smithi100.stdout:op 2366 completed, throughput=5MB/sec 2024-04-07T16:46:20.477 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-fgZdDT9F95dzoln off=19700499 len=1102438 2024-04-07T16:46:20.480 INFO:tasks.workunit.client.0.smithi100.stdout:op 2367 completed, throughput=5MB/sec 2024-04-07T16:46:20.480 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-NQ3yhpJ2NYFGqXa off=12070549 len=90825 2024-04-07T16:46:20.481 INFO:tasks.workunit.client.0.smithi100.stdout:op 2368 completed, throughput=5MB/sec 2024-04-07T16:46:20.481 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-uLfo24p0TWmu88G off=19255187 len=503862 2024-04-07T16:46:20.483 INFO:tasks.workunit.client.0.smithi100.stdout:op 2369 completed, throughput=5MB/sec 2024-04-07T16:46:20.483 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-KrmqUPJRMBvrLvm off=4625507 len=1397829 2024-04-07T16:46:20.494 INFO:tasks.workunit.client.0.smithi100.stdout:op 2370 completed, throughput=5MB/sec 2024-04-07T16:46:20.694 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:46:20 smithi100 bash[74942]: cluster 2024-04-07T16:46:19.313849+0000 mgr.y (mgr.25175) 929 : cluster [DBG] pgmap v1571: 105 pgs: 105 active+clean; 126 GiB data, 6.3 GiB used, 709 GiB / 715 GiB avail; 2.6 MiB/s rd, 1.9 MiB/s wr, 4 op/s 2024-04-07T16:46:20.694 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:46:20 smithi100 bash[76202]: cluster 2024-04-07T16:46:19.313849+0000 mgr.y (mgr.25175) 929 : cluster [DBG] pgmap v1571: 105 pgs: 105 active+clean; 126 GiB data, 6.3 GiB used, 709 GiB / 715 GiB avail; 2.6 MiB/s rd, 1.9 MiB/s wr, 4 op/s 2024-04-07T16:46:20.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:46:20 smithi144 bash[66654]: cluster 2024-04-07T16:46:19.313849+0000 mgr.y (mgr.25175) 929 : cluster [DBG] pgmap v1571: 105 pgs: 105 active+clean; 126 GiB data, 6.3 GiB used, 709 GiB / 715 GiB avail; 2.6 MiB/s rd, 1.9 MiB/s wr, 4 op/s 2024-04-07T16:46:21.495 INFO:tasks.workunit.client.0.smithi100.stdout: 450: throughput=4.99MB/sec pending data=0 2024-04-07T16:46:21.495 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-FeMQIkvDJQxiQR9 off=2109449 len=1348043 2024-04-07T16:46:21.498 INFO:tasks.workunit.client.0.smithi100.stdout:op 2371 completed, throughput=4.99MB/sec 2024-04-07T16:46:21.498 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-9_PEo9PwsYZTqtP off=3367815 len=274346 2024-04-07T16:46:21.500 INFO:tasks.workunit.client.0.smithi100.stdout:op 2372 completed, throughput=4.99MB/sec 2024-04-07T16:46:21.500 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj--4qfqzaxiSpL0BU off=7218373 len=41113 2024-04-07T16:46:21.502 INFO:tasks.workunit.client.0.smithi100.stdout:op 2373 completed, throughput=4.99MB/sec 2024-04-07T16:46:21.502 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-7Q7NOKuddasgngj off=8354974 len=1362546 2024-04-07T16:46:21.505 INFO:tasks.workunit.client.0.smithi100.stdout:op 2374 completed, throughput=5MB/sec 2024-04-07T16:46:21.505 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-9OzXhY1iKDtiZel off=2253606 len=1495132 2024-04-07T16:46:21.517 INFO:tasks.workunit.client.0.smithi100.stdout:op 2375 completed, throughput=5MB/sec 2024-04-07T16:46:21.517 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-dDl2rQ-sCiRACr1 off=10712052 len=1292206 2024-04-07T16:46:21.527 INFO:tasks.workunit.client.0.smithi100.stdout:op 2376 completed, throughput=5MB/sec 2024-04-07T16:46:22.528 INFO:tasks.workunit.client.0.smithi100.stdout: 451: throughput=4.99MB/sec pending data=0 2024-04-07T16:46:22.528 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-WhOkiBU-28ZtLfI off=12158571 len=1002315 2024-04-07T16:46:22.532 INFO:tasks.workunit.client.0.smithi100.stdout:op 2377 completed, throughput=4.99MB/sec 2024-04-07T16:46:22.533 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-MoelWDLw2udX6_y off=9105513 len=711382 2024-04-07T16:46:22.539 INFO:tasks.workunit.client.0.smithi100.stdout:op 2378 completed, throughput=4.99MB/sec 2024-04-07T16:46:22.539 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-rBRH-t36xeP-Cid off=1704389 len=379154 2024-04-07T16:46:22.541 INFO:tasks.workunit.client.0.smithi100.stdout:op 2379 completed, throughput=5MB/sec 2024-04-07T16:46:22.541 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-s4HhJU3KRIwR69G off=2729114 len=2005767 2024-04-07T16:46:22.546 INFO:tasks.workunit.client.0.smithi100.stdout:op 2380 completed, throughput=5MB/sec 2024-04-07T16:46:22.546 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-hxah47FDJSUpF9R off=1053543 len=1987888 2024-04-07T16:46:22.550 INFO:tasks.workunit.client.0.smithi100.stdout:op 2381 completed, throughput=5MB/sec 2024-04-07T16:46:22.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:46:22 smithi144 bash[66654]: cluster 2024-04-07T16:46:21.314768+0000 mgr.y (mgr.25175) 930 : cluster [DBG] pgmap v1572: 105 pgs: 105 active+clean; 126 GiB data, 6.3 GiB used, 709 GiB / 715 GiB avail; 2.6 MiB/s rd, 2.1 MiB/s wr, 4 op/s 2024-04-07T16:46:22.609 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:46:22 smithi144 bash[95233]: ts=2024-04-07T16:46:22.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:46:22.694 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:46:22 smithi100 bash[76202]: cluster 2024-04-07T16:46:21.314768+0000 mgr.y (mgr.25175) 930 : cluster [DBG] pgmap v1572: 105 pgs: 105 active+clean; 126 GiB data, 6.3 GiB used, 709 GiB / 715 GiB avail; 2.6 MiB/s rd, 2.1 MiB/s wr, 4 op/s 2024-04-07T16:46:22.694 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:46:22 smithi100 bash[74942]: cluster 2024-04-07T16:46:21.314768+0000 mgr.y (mgr.25175) 930 : cluster [DBG] pgmap v1572: 105 pgs: 105 active+clean; 126 GiB data, 6.3 GiB used, 709 GiB / 715 GiB avail; 2.6 MiB/s rd, 2.1 MiB/s wr, 4 op/s 2024-04-07T16:46:23.550 INFO:tasks.workunit.client.0.smithi100.stdout: 452: throughput=4.99MB/sec pending data=0 2024-04-07T16:46:23.551 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-BWDqxu3bl_Du9US off=18997198 len=1827015 2024-04-07T16:46:23.566 INFO:tasks.workunit.client.0.smithi100.stdout:op 2382 completed, throughput=5MB/sec 2024-04-07T16:46:23.566 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-b89XAal0xvE5QOD off=5964501 len=465263 2024-04-07T16:46:23.568 INFO:tasks.workunit.client.0.smithi100.stdout:op 2383 completed, throughput=5MB/sec 2024-04-07T16:46:23.568 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-FM1U9aJyh4mriJm off=296393 len=562778 2024-04-07T16:46:23.570 INFO:tasks.workunit.client.0.smithi100.stdout:op 2384 completed, throughput=5MB/sec 2024-04-07T16:46:23.571 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-xCZt8_ycKAHZQ7o off=18061792 len=1883911 2024-04-07T16:46:23.584 INFO:tasks.workunit.client.0.smithi100.stdout:op 2385 completed, throughput=5MB/sec 2024-04-07T16:46:24.585 INFO:tasks.workunit.client.0.smithi100.stdout: 453: throughput=4.99MB/sec pending data=0 2024-04-07T16:46:24.585 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-wFucr3gjR0QPt6C off=1060870 len=1692152 2024-04-07T16:46:24.589 INFO:tasks.workunit.client.0.smithi100.stdout:op 2386 completed, throughput=4.99MB/sec 2024-04-07T16:46:24.589 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-DcWQhLvn929HKyw off=4808045 len=89433 2024-04-07T16:46:24.590 INFO:tasks.workunit.client.0.smithi100.stdout:op 2387 completed, throughput=5MB/sec 2024-04-07T16:46:24.590 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-yUFmGIXJBDI3fGc off=10205088 len=418647 2024-04-07T16:46:24.597 INFO:tasks.workunit.client.0.smithi100.stdout:op 2388 completed, throughput=5MB/sec 2024-04-07T16:46:24.597 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Aqy96yyPCH1qyBx off=9710977 len=1033542 2024-04-07T16:46:24.599 INFO:tasks.workunit.client.0.smithi100.stdout:op 2389 completed, throughput=5MB/sec 2024-04-07T16:46:24.599 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-kgGiCaYSgheT4md off=3112876 len=45499 2024-04-07T16:46:24.603 INFO:tasks.workunit.client.0.smithi100.stdout:op 2390 completed, throughput=5MB/sec 2024-04-07T16:46:24.604 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj--5pJjVv8QGffx6A off=1130796 len=1536065 2024-04-07T16:46:24.608 INFO:tasks.workunit.client.0.smithi100.stdout:op 2391 completed, throughput=5MB/sec 2024-04-07T16:46:24.694 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:46:24 smithi100 bash[74942]: cluster 2024-04-07T16:46:23.315940+0000 mgr.y (mgr.25175) 931 : cluster [DBG] pgmap v1573: 105 pgs: 105 active+clean; 126 GiB data, 6.3 GiB used, 709 GiB / 715 GiB avail; 3.2 MiB/s rd, 2.6 MiB/s wr, 5 op/s 2024-04-07T16:46:24.694 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:46:24 smithi100 bash[76202]: cluster 2024-04-07T16:46:23.315940+0000 mgr.y (mgr.25175) 931 : cluster [DBG] pgmap v1573: 105 pgs: 105 active+clean; 126 GiB data, 6.3 GiB used, 709 GiB / 715 GiB avail; 3.2 MiB/s rd, 2.6 MiB/s wr, 5 op/s 2024-04-07T16:46:24.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:46:24 smithi144 bash[66654]: cluster 2024-04-07T16:46:23.315940+0000 mgr.y (mgr.25175) 931 : cluster [DBG] pgmap v1573: 105 pgs: 105 active+clean; 126 GiB data, 6.3 GiB used, 709 GiB / 715 GiB avail; 3.2 MiB/s rd, 2.6 MiB/s wr, 5 op/s 2024-04-07T16:46:25.608 INFO:tasks.workunit.client.0.smithi100.stdout: 454: throughput=4.99MB/sec pending data=0 2024-04-07T16:46:25.608 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-NOtjBrD-86u8sJG off=4156971 len=365439 2024-04-07T16:46:25.614 INFO:tasks.workunit.client.0.smithi100.stdout:op 2392 completed, throughput=4.99MB/sec 2024-04-07T16:46:25.614 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-3xY4WWS8WUIvGNI off=13716039 len=384410 2024-04-07T16:46:25.616 INFO:tasks.workunit.client.0.smithi100.stdout:op 2393 completed, throughput=4.99MB/sec 2024-04-07T16:46:25.616 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-wb6ORPMiwxWyOMq off=1855705 len=675105 2024-04-07T16:46:25.618 INFO:tasks.workunit.client.0.smithi100.stdout:op 2394 completed, throughput=4.99MB/sec 2024-04-07T16:46:25.618 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-tdbvmVr1VmMLt9s off=1041608 len=959353 2024-04-07T16:46:25.627 INFO:tasks.workunit.client.0.smithi100.stdout:op 2395 completed, throughput=4.99MB/sec 2024-04-07T16:46:25.627 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-yF5caWWzMNt_JMG off=13077463 len=750395 2024-04-07T16:46:25.629 INFO:tasks.workunit.client.0.smithi100.stdout:op 2396 completed, throughput=5MB/sec 2024-04-07T16:46:25.630 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-sCWqw08NdlmLDUm off=7972495 len=762734 2024-04-07T16:46:25.632 INFO:tasks.workunit.client.0.smithi100.stdout:op 2397 completed, throughput=5MB/sec 2024-04-07T16:46:25.632 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-9rOtSFJPbGl7-aF off=6356633 len=60087 2024-04-07T16:46:25.634 INFO:tasks.workunit.client.0.smithi100.stdout:op 2398 completed, throughput=5MB/sec 2024-04-07T16:46:25.634 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Wa5Qdx8hTewQHZE off=1906098 len=1369639 2024-04-07T16:46:25.645 INFO:tasks.workunit.client.0.smithi100.stdout:op 2399 completed, throughput=5MB/sec 2024-04-07T16:46:26.645 INFO:tasks.workunit.client.0.smithi100.stdout: 455: throughput=4.99MB/sec pending data=0 2024-04-07T16:46:26.646 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-CQ2QXL8jgH5-SfJ off=11473464 len=833985 2024-04-07T16:46:26.654 INFO:tasks.workunit.client.0.smithi100.stdout:op 2400 completed, throughput=4.99MB/sec 2024-04-07T16:46:26.654 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-qxPazc9czXtJc5u off=7007022 len=340292 2024-04-07T16:46:26.661 INFO:tasks.workunit.client.0.smithi100.stdout:op 2401 completed, throughput=4.99MB/sec 2024-04-07T16:46:26.661 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-dNlfEIshGoGnEjs off=5328524 len=906509 2024-04-07T16:46:26.668 INFO:tasks.workunit.client.0.smithi100.stdout:op 2402 completed, throughput=4.99MB/sec 2024-04-07T16:46:26.668 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-57jlwlmu3igcUAq off=6529965 len=2014366 2024-04-07T16:46:26.687 INFO:tasks.workunit.client.0.smithi100.stdout:op 2403 completed, throughput=5MB/sec 2024-04-07T16:46:26.687 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-G3VaJc7xmFrTFTe off=17919058 len=263521 2024-04-07T16:46:26.687 INFO:tasks.workunit.client.0.smithi100.stdout:op 2404 completed, throughput=5MB/sec 2024-04-07T16:46:26.688 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-_SOjwutquHW_QRA off=14937366 len=1643946 2024-04-07T16:46:26.691 INFO:tasks.workunit.client.0.smithi100.stdout:op 2405 completed, throughput=5MB/sec 2024-04-07T16:46:26.694 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:46:26 smithi100 bash[74942]: cluster 2024-04-07T16:46:25.316896+0000 mgr.y (mgr.25175) 932 : cluster [DBG] pgmap v1574: 105 pgs: 105 active+clean; 126 GiB data, 6.3 GiB used, 709 GiB / 715 GiB avail; 3.1 MiB/s rd, 2.1 MiB/s wr, 4 op/s 2024-04-07T16:46:26.694 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:46:26 smithi100 bash[76202]: cluster 2024-04-07T16:46:25.316896+0000 mgr.y (mgr.25175) 932 : cluster [DBG] pgmap v1574: 105 pgs: 105 active+clean; 126 GiB data, 6.3 GiB used, 709 GiB / 715 GiB avail; 3.1 MiB/s rd, 2.1 MiB/s wr, 4 op/s 2024-04-07T16:46:26.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:46:26 smithi144 bash[66654]: cluster 2024-04-07T16:46:25.316896+0000 mgr.y (mgr.25175) 932 : cluster [DBG] pgmap v1574: 105 pgs: 105 active+clean; 126 GiB data, 6.3 GiB used, 709 GiB / 715 GiB avail; 3.1 MiB/s rd, 2.1 MiB/s wr, 4 op/s 2024-04-07T16:46:27.692 INFO:tasks.workunit.client.0.smithi100.stdout: 456: throughput=4.99MB/sec pending data=0 2024-04-07T16:46:27.692 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-QE0wR8gGaaNriTk off=4564226 len=548704 2024-04-07T16:46:27.694 INFO:tasks.workunit.client.0.smithi100.stdout:op 2406 completed, throughput=4.99MB/sec 2024-04-07T16:46:27.694 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-HyMJMSfz3uxf0PM off=1219998 len=365384 2024-04-07T16:46:27.696 INFO:tasks.workunit.client.0.smithi100.stdout:op 2407 completed, throughput=4.99MB/sec 2024-04-07T16:46:27.696 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-VIRPCT0owjaPTa2 off=2341807 len=1319351 2024-04-07T16:46:27.706 INFO:tasks.workunit.client.0.smithi100.stdout:op 2408 completed, throughput=5MB/sec 2024-04-07T16:46:27.706 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj--vtBiy2V7EEnslG off=589882 len=1867352 2024-04-07T16:46:27.711 INFO:tasks.workunit.client.0.smithi100.stdout:op 2409 completed, throughput=5MB/sec 2024-04-07T16:46:27.711 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-VKrJoxyK29YR-hw off=13955203 len=1834728 2024-04-07T16:46:27.726 INFO:tasks.workunit.client.0.smithi100.stdout:op 2410 completed, throughput=5MB/sec 2024-04-07T16:46:28.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:46:28 smithi144 bash[66654]: cluster 2024-04-07T16:46:27.317933+0000 mgr.y (mgr.25175) 933 : cluster [DBG] pgmap v1575: 105 pgs: 105 active+clean; 126 GiB data, 6.3 GiB used, 709 GiB / 715 GiB avail; 3.1 MiB/s rd, 2.0 MiB/s wr, 5 op/s 2024-04-07T16:46:28.694 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:46:28 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:46:28] "GET /metrics HTTP/1.1" 200 34071 "" "Prometheus/2.43.0" 2024-04-07T16:46:28.694 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:46:28 smithi100 bash[74942]: cluster 2024-04-07T16:46:27.317933+0000 mgr.y (mgr.25175) 933 : cluster [DBG] pgmap v1575: 105 pgs: 105 active+clean; 126 GiB data, 6.3 GiB used, 709 GiB / 715 GiB avail; 3.1 MiB/s rd, 2.0 MiB/s wr, 5 op/s 2024-04-07T16:46:28.695 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:46:28 smithi100 bash[76202]: cluster 2024-04-07T16:46:27.317933+0000 mgr.y (mgr.25175) 933 : cluster [DBG] pgmap v1575: 105 pgs: 105 active+clean; 126 GiB data, 6.3 GiB used, 709 GiB / 715 GiB avail; 3.1 MiB/s rd, 2.0 MiB/s wr, 5 op/s 2024-04-07T16:46:28.726 INFO:tasks.workunit.client.0.smithi100.stdout: 457: throughput=4.99MB/sec pending data=0 2024-04-07T16:46:28.726 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-k1DESWf-RP0yFA6 off=17085699 len=740447 2024-04-07T16:46:28.734 INFO:tasks.workunit.client.0.smithi100.stdout:op 2411 completed, throughput=4.99MB/sec 2024-04-07T16:46:28.734 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-oJkWTrgkBPEaeAK off=4189625 len=562562 2024-04-07T16:46:28.736 INFO:tasks.workunit.client.0.smithi100.stdout:op 2412 completed, throughput=4.99MB/sec 2024-04-07T16:46:28.736 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-0PHmK3fCThTsFuX off=2492474 len=1186226 2024-04-07T16:46:28.746 INFO:tasks.workunit.client.0.smithi100.stdout:op 2413 completed, throughput=5MB/sec 2024-04-07T16:46:28.747 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-ujjKckqemtEAQ54 off=1048191 len=310482 2024-04-07T16:46:28.753 INFO:tasks.workunit.client.0.smithi100.stdout:op 2414 completed, throughput=5MB/sec 2024-04-07T16:46:28.753 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-46oatdvZBl4MKHT off=7682683 len=1852729 2024-04-07T16:46:28.766 INFO:tasks.workunit.client.0.smithi100.stdout:op 2415 completed, throughput=5MB/sec 2024-04-07T16:46:29.767 INFO:tasks.workunit.client.0.smithi100.stdout: 458: throughput=4.99MB/sec pending data=0 2024-04-07T16:46:29.767 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-RdCczqVqH3a4uuK off=44677 len=199299 2024-04-07T16:46:29.772 INFO:tasks.workunit.client.0.smithi100.stdout:op 2416 completed, throughput=4.99MB/sec 2024-04-07T16:46:29.772 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-lddaQcIIW267pi6 off=1833528 len=1933672 2024-04-07T16:46:29.776 INFO:tasks.workunit.client.0.smithi100.stdout:op 2417 completed, throughput=4.99MB/sec 2024-04-07T16:46:29.777 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-hLFxiwQ0NKaMKCW off=15572252 len=1094450 2024-04-07T16:46:29.787 INFO:tasks.workunit.client.0.smithi100.stdout:op 2418 completed, throughput=5MB/sec 2024-04-07T16:46:29.787 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-VnX6g05CwYRaM3B off=3391998 len=1614315 2024-04-07T16:46:29.793 INFO:tasks.workunit.client.0.smithi100.stdout:op 2419 completed, throughput=5MB/sec 2024-04-07T16:46:30.694 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:46:30 smithi100 bash[74942]: cluster 2024-04-07T16:46:29.318782+0000 mgr.y (mgr.25175) 934 : cluster [DBG] pgmap v1576: 105 pgs: 105 active+clean; 126 GiB data, 6.3 GiB used, 709 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.1 MiB/s wr, 4 op/s 2024-04-07T16:46:30.694 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:46:30 smithi100 bash[76202]: cluster 2024-04-07T16:46:29.318782+0000 mgr.y (mgr.25175) 934 : cluster [DBG] pgmap v1576: 105 pgs: 105 active+clean; 126 GiB data, 6.3 GiB used, 709 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.1 MiB/s wr, 4 op/s 2024-04-07T16:46:30.793 INFO:tasks.workunit.client.0.smithi100.stdout: 459: throughput=4.99MB/sec pending data=0 2024-04-07T16:46:30.794 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-s4BQzGEEEBSm-Bo off=20351514 len=561503 2024-04-07T16:46:30.796 INFO:tasks.workunit.client.0.smithi100.stdout:op 2420 completed, throughput=4.99MB/sec 2024-04-07T16:46:30.796 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-kjHAPYOAKc8Nt6t off=11907859 len=427499 2024-04-07T16:46:30.798 INFO:tasks.workunit.client.0.smithi100.stdout:op 2421 completed, throughput=4.99MB/sec 2024-04-07T16:46:30.798 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-lBQkSdNPKot1zrJ off=7505049 len=1424542 2024-04-07T16:46:30.802 INFO:tasks.workunit.client.0.smithi100.stdout:op 2422 completed, throughput=4.99MB/sec 2024-04-07T16:46:30.802 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-hACe-NRJGT5xOKg off=1549774 len=311392 2024-04-07T16:46:30.808 INFO:tasks.workunit.client.0.smithi100.stdout:op 2423 completed, throughput=4.99MB/sec 2024-04-07T16:46:30.808 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-0AtHj8HHQENye5A off=8943425 len=286753 2024-04-07T16:46:30.813 INFO:tasks.workunit.client.0.smithi100.stdout:op 2424 completed, throughput=5MB/sec 2024-04-07T16:46:30.813 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-eo5kpC09dKoKjGc off=1589592 len=610752 2024-04-07T16:46:30.820 INFO:tasks.workunit.client.0.smithi100.stdout:op 2425 completed, throughput=5MB/sec 2024-04-07T16:46:30.821 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Y-Alibjn8NYS644 off=2225581 len=1558406 2024-04-07T16:46:30.832 INFO:tasks.workunit.client.0.smithi100.stdout:op 2426 completed, throughput=5MB/sec 2024-04-07T16:46:30.832 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-nz8Ra9R4azpsRwd off=5297427 len=868924 2024-04-07T16:46:30.834 INFO:tasks.workunit.client.0.smithi100.stdout:op 2427 completed, throughput=5MB/sec 2024-04-07T16:46:30.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:46:30 smithi144 bash[66654]: cluster 2024-04-07T16:46:29.318782+0000 mgr.y (mgr.25175) 934 : cluster [DBG] pgmap v1576: 105 pgs: 105 active+clean; 126 GiB data, 6.3 GiB used, 709 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.1 MiB/s wr, 4 op/s 2024-04-07T16:46:31.835 INFO:tasks.workunit.client.0.smithi100.stdout: 460: throughput=4.99MB/sec pending data=0 2024-04-07T16:46:31.835 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-NBQPwLrmS-DcoeI off=2649014 len=650908 2024-04-07T16:46:31.842 INFO:tasks.workunit.client.0.smithi100.stdout:op 2428 completed, throughput=4.99MB/sec 2024-04-07T16:46:31.843 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-JDJVD4n3dCQU-oA off=477866 len=1864968 2024-04-07T16:46:31.857 INFO:tasks.workunit.client.0.smithi100.stdout:op 2429 completed, throughput=5MB/sec 2024-04-07T16:46:31.857 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-bbO3FJdGtfSQ9tk off=10136043 len=994336 2024-04-07T16:46:31.860 INFO:tasks.workunit.client.0.smithi100.stdout:op 2430 completed, throughput=5MB/sec 2024-04-07T16:46:31.860 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Oz-AOOJwfox7xIX off=11062110 len=1640682 2024-04-07T16:46:31.874 INFO:tasks.workunit.client.0.smithi100.stdout:op 2431 completed, throughput=5MB/sec 2024-04-07T16:46:32.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:46:32 smithi144 bash[66654]: cluster 2024-04-07T16:46:31.319736+0000 mgr.y (mgr.25175) 935 : cluster [DBG] pgmap v1577: 105 pgs: 105 active+clean; 126 GiB data, 6.3 GiB used, 709 GiB / 715 GiB avail; 2.6 MiB/s rd, 2.1 MiB/s wr, 5 op/s 2024-04-07T16:46:32.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:46:32 smithi144 bash[66654]: audit 2024-04-07T16:46:31.768903+0000 mon.a (mon.0) 2111 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:46:32.609 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:46:32 smithi144 bash[95233]: ts=2024-04-07T16:46:32.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:46:32.694 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:46:32 smithi100 bash[74942]: cluster 2024-04-07T16:46:31.319736+0000 mgr.y (mgr.25175) 935 : cluster [DBG] pgmap v1577: 105 pgs: 105 active+clean; 126 GiB data, 6.3 GiB used, 709 GiB / 715 GiB avail; 2.6 MiB/s rd, 2.1 MiB/s wr, 5 op/s 2024-04-07T16:46:32.694 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:46:32 smithi100 bash[74942]: audit 2024-04-07T16:46:31.768903+0000 mon.a (mon.0) 2111 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:46:32.695 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:46:32 smithi100 bash[76202]: cluster 2024-04-07T16:46:31.319736+0000 mgr.y (mgr.25175) 935 : cluster [DBG] pgmap v1577: 105 pgs: 105 active+clean; 126 GiB data, 6.3 GiB used, 709 GiB / 715 GiB avail; 2.6 MiB/s rd, 2.1 MiB/s wr, 5 op/s 2024-04-07T16:46:32.695 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:46:32 smithi100 bash[76202]: audit 2024-04-07T16:46:31.768903+0000 mon.a (mon.0) 2111 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:46:32.874 INFO:tasks.workunit.client.0.smithi100.stdout: 461: throughput=4.99MB/sec pending data=0 2024-04-07T16:46:32.874 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-cnk1XofmM7CgiUQ off=2473293 len=348569 2024-04-07T16:46:32.877 INFO:tasks.workunit.client.0.smithi100.stdout:op 2432 completed, throughput=4.99MB/sec 2024-04-07T16:46:32.877 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-8QeU-Iphn7Qd27k off=4636751 len=555951 2024-04-07T16:46:32.879 INFO:tasks.workunit.client.0.smithi100.stdout:op 2433 completed, throughput=4.99MB/sec 2024-04-07T16:46:32.879 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-iYJUQwW-9r-kQSU off=20956052 len=1481138 2024-04-07T16:46:32.892 INFO:tasks.workunit.client.0.smithi100.stdout:op 2434 completed, throughput=4.99MB/sec 2024-04-07T16:46:32.892 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-_Cx5uzVT3IcVRiq off=4254340 len=90666 2024-04-07T16:46:32.896 INFO:tasks.workunit.client.0.smithi100.stdout:op 2435 completed, throughput=4.99MB/sec 2024-04-07T16:46:32.896 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Oh-sWUrgpjklNCO off=2477794 len=124027 2024-04-07T16:46:32.900 INFO:tasks.workunit.client.0.smithi100.stdout:op 2436 completed, throughput=5MB/sec 2024-04-07T16:46:32.900 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-cflKefypk4Khz7G off=22411216 len=1140115 2024-04-07T16:46:32.910 INFO:tasks.workunit.client.0.smithi100.stdout:op 2437 completed, throughput=5MB/sec 2024-04-07T16:46:32.910 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-ITO_1tkz-Ey12Nm off=852669 len=635417 2024-04-07T16:46:32.916 INFO:tasks.workunit.client.0.smithi100.stdout:op 2438 completed, throughput=5MB/sec 2024-04-07T16:46:32.917 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-T8AsAsKKMhFV7io off=14155519 len=401929 2024-04-07T16:46:32.923 INFO:tasks.workunit.client.0.smithi100.stdout:op 2439 completed, throughput=5MB/sec 2024-04-07T16:46:32.923 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-CcAvdWrnRZJDzir off=1187320 len=98625 2024-04-07T16:46:32.924 INFO:tasks.workunit.client.0.smithi100.stdout:op 2440 completed, throughput=5MB/sec 2024-04-07T16:46:32.924 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-SMXL1GXCaRNL0YG off=11955076 len=1015281 2024-04-07T16:46:32.927 INFO:tasks.workunit.client.0.smithi100.stdout:op 2441 completed, throughput=5MB/sec 2024-04-07T16:46:33.927 INFO:tasks.workunit.client.0.smithi100.stdout: 462: throughput=4.99MB/sec pending data=0 2024-04-07T16:46:33.927 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-7TzB_YCecNUC0-I off=11139823 len=602074 2024-04-07T16:46:33.930 INFO:tasks.workunit.client.0.smithi100.stdout:op 2442 completed, throughput=4.99MB/sec 2024-04-07T16:46:33.930 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-f3Z1YmokCw7Witq off=16087191 len=1562369 2024-04-07T16:46:33.935 INFO:tasks.workunit.client.0.smithi100.stdout:op 2443 completed, throughput=5MB/sec 2024-04-07T16:46:33.935 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-c3gLl6HzY6e2yuV off=6463223 len=653779 2024-04-07T16:46:33.937 INFO:tasks.workunit.client.0.smithi100.stdout:op 2444 completed, throughput=5MB/sec 2024-04-07T16:46:33.937 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-pH0bM27tauMEONs off=4054362 len=1590901 2024-04-07T16:46:33.951 INFO:tasks.workunit.client.0.smithi100.stdout:op 2445 completed, throughput=5MB/sec 2024-04-07T16:46:33.951 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-S-8835I47dNgPf7 off=10928641 len=626741 2024-04-07T16:46:33.953 INFO:tasks.workunit.client.0.smithi100.stdout:op 2446 completed, throughput=5MB/sec 2024-04-07T16:46:34.694 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:46:34 smithi100 bash[74942]: cluster 2024-04-07T16:46:33.320812+0000 mgr.y (mgr.25175) 936 : cluster [DBG] pgmap v1578: 105 pgs: 105 active+clean; 126 GiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 2.9 MiB/s rd, 2.9 MiB/s wr, 6 op/s 2024-04-07T16:46:34.694 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:46:34 smithi100 bash[76202]: cluster 2024-04-07T16:46:33.320812+0000 mgr.y (mgr.25175) 936 : cluster [DBG] pgmap v1578: 105 pgs: 105 active+clean; 126 GiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 2.9 MiB/s rd, 2.9 MiB/s wr, 6 op/s 2024-04-07T16:46:34.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:46:34 smithi144 bash[66654]: cluster 2024-04-07T16:46:33.320812+0000 mgr.y (mgr.25175) 936 : cluster [DBG] pgmap v1578: 105 pgs: 105 active+clean; 126 GiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 2.9 MiB/s rd, 2.9 MiB/s wr, 6 op/s 2024-04-07T16:46:34.953 INFO:tasks.workunit.client.0.smithi100.stdout: 463: throughput=4.99MB/sec pending data=0 2024-04-07T16:46:34.953 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-aGN93xLdCKRxGbw off=3701527 len=1748213 2024-04-07T16:46:34.959 INFO:tasks.workunit.client.0.smithi100.stdout:op 2447 completed, throughput=4.99MB/sec 2024-04-07T16:46:34.959 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-kCSAs-C1Gfo06X0 off=10881793 len=844586 2024-04-07T16:46:34.968 INFO:tasks.workunit.client.0.smithi100.stdout:op 2448 completed, throughput=5MB/sec 2024-04-07T16:46:34.969 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-0T7L3LAWOMLX9-1 off=6218569 len=677141 2024-04-07T16:46:34.971 INFO:tasks.workunit.client.0.smithi100.stdout:op 2449 completed, throughput=5MB/sec 2024-04-07T16:46:34.971 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-PZ9kMzdQx4MPfsq off=6704381 len=1934886 2024-04-07T16:46:34.984 INFO:tasks.workunit.client.0.smithi100.stdout:op 2450 completed, throughput=5MB/sec 2024-04-07T16:46:35.984 INFO:tasks.workunit.client.0.smithi100.stdout: 464: throughput=4.99MB/sec pending data=0 2024-04-07T16:46:35.985 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-iCwKPpbklFvQyIN off=9692671 len=1108294 2024-04-07T16:46:35.989 INFO:tasks.workunit.client.0.smithi100.stdout:op 2451 completed, throughput=4.99MB/sec 2024-04-07T16:46:35.989 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-ZM9B8dr4MD-xTIn off=8628041 len=1963507 2024-04-07T16:46:36.006 INFO:tasks.workunit.client.0.smithi100.stdout:op 2452 completed, throughput=5MB/sec 2024-04-07T16:46:36.006 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-kS6WDUpkJ3JYUGY off=1500862 len=271728 2024-04-07T16:46:36.012 INFO:tasks.workunit.client.0.smithi100.stdout:op 2453 completed, throughput=5MB/sec 2024-04-07T16:46:36.012 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-58RVWzeWY_NIP9E off=8083985 len=1393865 2024-04-07T16:46:36.015 INFO:tasks.workunit.client.0.smithi100.stdout:op 2454 completed, throughput=5MB/sec 2024-04-07T16:46:36.015 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-h2_fFkG-96qg6Up off=2726526 len=716142 2024-04-07T16:46:36.018 INFO:tasks.workunit.client.0.smithi100.stdout:op 2455 completed, throughput=5MB/sec 2024-04-07T16:46:36.694 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:46:36 smithi100 bash[74942]: cluster 2024-04-07T16:46:35.321779+0000 mgr.y (mgr.25175) 937 : cluster [DBG] pgmap v1579: 105 pgs: 105 active+clean; 126 GiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 2.3 MiB/s rd, 2.6 MiB/s wr, 5 op/s 2024-04-07T16:46:36.694 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:46:36 smithi100 bash[76202]: cluster 2024-04-07T16:46:35.321779+0000 mgr.y (mgr.25175) 937 : cluster [DBG] pgmap v1579: 105 pgs: 105 active+clean; 126 GiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 2.3 MiB/s rd, 2.6 MiB/s wr, 5 op/s 2024-04-07T16:46:36.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:46:36 smithi144 bash[66654]: cluster 2024-04-07T16:46:35.321779+0000 mgr.y (mgr.25175) 937 : cluster [DBG] pgmap v1579: 105 pgs: 105 active+clean; 126 GiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 2.3 MiB/s rd, 2.6 MiB/s wr, 5 op/s 2024-04-07T16:46:37.018 INFO:tasks.workunit.client.0.smithi100.stdout: 465: throughput=4.99MB/sec pending data=0 2024-04-07T16:46:37.019 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Nqo7sxKA8rD84g6 off=1266411 len=1234510 2024-04-07T16:46:37.029 INFO:tasks.workunit.client.0.smithi100.stdout:op 2456 completed, throughput=4.99MB/sec 2024-04-07T16:46:37.030 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-PiPI1CXt6aRidxZ off=7383975 len=1567949 2024-04-07T16:46:37.043 INFO:tasks.workunit.client.0.smithi100.stdout:op 2457 completed, throughput=5MB/sec 2024-04-07T16:46:37.043 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-j08f0EpWT8S32nw off=11691099 len=1282069 2024-04-07T16:46:37.045 INFO:tasks.workunit.client.0.smithi100.stdout:op 2458 completed, throughput=5MB/sec 2024-04-07T16:46:37.045 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-MR7g8M-0iYixe2q off=78426 len=606457 2024-04-07T16:46:37.052 INFO:tasks.workunit.client.0.smithi100.stdout:op 2459 completed, throughput=5MB/sec 2024-04-07T16:46:37.052 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-PNzWX9Gh-ENXe1A off=766271 len=939876 2024-04-07T16:46:37.060 INFO:tasks.workunit.client.0.smithi100.stdout:op 2460 completed, throughput=5MB/sec 2024-04-07T16:46:38.061 INFO:tasks.workunit.client.0.smithi100.stdout: 466: throughput=4.99MB/sec pending data=0 2024-04-07T16:46:38.061 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-QrARshO0hwvmosE off=15214818 len=195139 2024-04-07T16:46:38.062 INFO:tasks.workunit.client.0.smithi100.stdout:op 2461 completed, throughput=4.99MB/sec 2024-04-07T16:46:38.062 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-UkA2lJlT1LGBiVf off=2051208 len=537739 2024-04-07T16:46:38.064 INFO:tasks.workunit.client.0.smithi100.stdout:op 2462 completed, throughput=4.99MB/sec 2024-04-07T16:46:38.064 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-G8ssepZWqG5_w4a off=1067575 len=972060 2024-04-07T16:46:38.073 INFO:tasks.workunit.client.0.smithi100.stdout:op 2463 completed, throughput=4.99MB/sec 2024-04-07T16:46:38.074 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-B-dAeNcaMWbFAXS off=14595310 len=2086917 2024-04-07T16:46:38.078 INFO:tasks.workunit.client.0.smithi100.stdout:op 2464 completed, throughput=5MB/sec 2024-04-07T16:46:38.078 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-JzqJrsmNxJgurwa off=6672245 len=347345 2024-04-07T16:46:38.084 INFO:tasks.workunit.client.0.smithi100.stdout:op 2465 completed, throughput=5MB/sec 2024-04-07T16:46:38.084 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-TOq0pea1hpWZ-CW off=10350646 len=321141 2024-04-07T16:46:38.090 INFO:tasks.workunit.client.0.smithi100.stdout:op 2466 completed, throughput=5MB/sec 2024-04-07T16:46:38.090 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-bdh4ugN6xBte2Ez off=478838 len=153920 2024-04-07T16:46:38.091 INFO:tasks.workunit.client.0.smithi100.stdout:op 2467 completed, throughput=5MB/sec 2024-04-07T16:46:38.091 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-50YNUfwpeLXsUW2 off=4734033 len=1005900 2024-04-07T16:46:38.100 INFO:tasks.workunit.client.0.smithi100.stdout:op 2468 completed, throughput=5MB/sec 2024-04-07T16:46:38.694 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:46:38 smithi100 bash[74942]: cluster 2024-04-07T16:46:37.322809+0000 mgr.y (mgr.25175) 938 : cluster [DBG] pgmap v1580: 105 pgs: 105 active+clean; 126 GiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.5 MiB/s wr, 5 op/s 2024-04-07T16:46:38.695 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:46:38 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:46:38] "GET /metrics HTTP/1.1" 200 34069 "" "Prometheus/2.43.0" 2024-04-07T16:46:38.695 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:46:38 smithi100 bash[76202]: cluster 2024-04-07T16:46:37.322809+0000 mgr.y (mgr.25175) 938 : cluster [DBG] pgmap v1580: 105 pgs: 105 active+clean; 126 GiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.5 MiB/s wr, 5 op/s 2024-04-07T16:46:38.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:46:38 smithi144 bash[66654]: cluster 2024-04-07T16:46:37.322809+0000 mgr.y (mgr.25175) 938 : cluster [DBG] pgmap v1580: 105 pgs: 105 active+clean; 126 GiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.5 MiB/s wr, 5 op/s 2024-04-07T16:46:39.101 INFO:tasks.workunit.client.0.smithi100.stdout: 467: throughput=4.99MB/sec pending data=0 2024-04-07T16:46:39.101 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-aYHrbF88Ef1dXft off=9776034 len=1306879 2024-04-07T16:46:39.112 INFO:tasks.workunit.client.0.smithi100.stdout:op 2469 completed, throughput=4.99MB/sec 2024-04-07T16:46:39.112 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-xM5FkbZ78I14qd1 off=5819887 len=277092 2024-04-07T16:46:39.113 INFO:tasks.workunit.client.0.smithi100.stdout:op 2470 completed, throughput=4.99MB/sec 2024-04-07T16:46:39.113 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-5NQXWDf5RxD08SQ off=4879958 len=1661532 2024-04-07T16:46:39.119 INFO:tasks.workunit.client.0.smithi100.stdout:op 2471 completed, throughput=5MB/sec 2024-04-07T16:46:39.119 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-G9qVVEMwVogiAFj off=2641806 len=1274439 2024-04-07T16:46:39.127 INFO:tasks.workunit.client.0.smithi100.stdout:op 2472 completed, throughput=5MB/sec 2024-04-07T16:46:39.127 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-p2orzdsyK24PVds off=754635 len=1420484 2024-04-07T16:46:39.138 INFO:tasks.workunit.client.0.smithi100.stdout:op 2473 completed, throughput=5MB/sec 2024-04-07T16:46:40.138 INFO:tasks.workunit.client.0.smithi100.stdout: 468: throughput=4.99MB/sec pending data=0 2024-04-07T16:46:40.139 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-TutMSvDMlks9Vyf off=11581796 len=592323 2024-04-07T16:46:40.146 INFO:tasks.workunit.client.0.smithi100.stdout:op 2474 completed, throughput=4.99MB/sec 2024-04-07T16:46:40.146 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-SB-rOxa923MYU2n off=5838884 len=1523985 2024-04-07T16:46:40.156 INFO:tasks.workunit.client.0.smithi100.stdout:op 2475 completed, throughput=5MB/sec 2024-04-07T16:46:40.157 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-71CMBlkcMPAtY85 off=4509088 len=1526107 2024-04-07T16:46:40.162 INFO:tasks.workunit.client.0.smithi100.stdout:op 2476 completed, throughput=5MB/sec 2024-04-07T16:46:40.162 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-m3rBK6kwUxlaXX5 off=6449115 len=482510 2024-04-07T16:46:40.169 INFO:tasks.workunit.client.0.smithi100.stdout:op 2477 completed, throughput=5MB/sec 2024-04-07T16:46:40.169 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-XIk__uKlVhj4FbD off=1682961 len=108016 2024-04-07T16:46:40.171 INFO:tasks.workunit.client.0.smithi100.stdout:op 2478 completed, throughput=5MB/sec 2024-04-07T16:46:40.171 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-RQ8PzrOd2jWH7Wb off=8455670 len=1319867 2024-04-07T16:46:40.173 INFO:tasks.workunit.client.0.smithi100.stdout:op 2479 completed, throughput=5MB/sec 2024-04-07T16:46:40.694 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:46:40 smithi100 bash[74942]: cluster 2024-04-07T16:46:39.323726+0000 mgr.y (mgr.25175) 939 : cluster [DBG] pgmap v1581: 105 pgs: 105 active+clean; 126 GiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 2.0 MiB/s rd, 2.4 MiB/s wr, 4 op/s 2024-04-07T16:46:40.694 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:46:40 smithi100 bash[76202]: cluster 2024-04-07T16:46:39.323726+0000 mgr.y (mgr.25175) 939 : cluster [DBG] pgmap v1581: 105 pgs: 105 active+clean; 126 GiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 2.0 MiB/s rd, 2.4 MiB/s wr, 4 op/s 2024-04-07T16:46:40.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:46:40 smithi144 bash[66654]: cluster 2024-04-07T16:46:39.323726+0000 mgr.y (mgr.25175) 939 : cluster [DBG] pgmap v1581: 105 pgs: 105 active+clean; 126 GiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 2.0 MiB/s rd, 2.4 MiB/s wr, 4 op/s 2024-04-07T16:46:41.173 INFO:tasks.workunit.client.0.smithi100.stdout: 469: throughput=4.99MB/sec pending data=0 2024-04-07T16:46:41.173 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Hi8AaNnR0GwXSiy off=1353732 len=844968 2024-04-07T16:46:41.181 INFO:tasks.workunit.client.0.smithi100.stdout:op 2480 completed, throughput=4.99MB/sec 2024-04-07T16:46:41.181 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-J5nIT8wVmVqv_lf off=11885714 len=1099889 2024-04-07T16:46:41.184 INFO:tasks.workunit.client.0.smithi100.stdout:op 2481 completed, throughput=5MB/sec 2024-04-07T16:46:41.184 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-0Zk_zFiNiRM4lSW off=1820263 len=56457 2024-04-07T16:46:41.189 INFO:tasks.workunit.client.0.smithi100.stdout:op 2482 completed, throughput=5MB/sec 2024-04-07T16:46:41.189 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-zA4dx9B6RPAqw-L off=5170559 len=691964 2024-04-07T16:46:41.196 INFO:tasks.workunit.client.0.smithi100.stdout:op 2483 completed, throughput=5MB/sec 2024-04-07T16:46:41.196 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-9v1k1CeMRzNRy01 off=2198419 len=1492894 2024-04-07T16:46:41.208 INFO:tasks.workunit.client.0.smithi100.stdout:op 2484 completed, throughput=5MB/sec 2024-04-07T16:46:41.208 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-1OUYmNnIpyzO7V7 off=1909463 len=932058 2024-04-07T16:46:41.217 INFO:tasks.workunit.client.0.smithi100.stdout:op 2485 completed, throughput=5MB/sec 2024-04-07T16:46:42.217 INFO:tasks.workunit.client.0.smithi100.stdout: 470: throughput=4.99MB/sec pending data=0 2024-04-07T16:46:42.217 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-OL6EAV8Wd0dBZD_ off=19131261 len=700627 2024-04-07T16:46:42.225 INFO:tasks.workunit.client.0.smithi100.stdout:op 2486 completed, throughput=4.99MB/sec 2024-04-07T16:46:42.225 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-2HsxjixLweMaGRY off=1224760 len=1375191 2024-04-07T16:46:42.236 INFO:tasks.workunit.client.0.smithi100.stdout:op 2487 completed, throughput=5MB/sec 2024-04-07T16:46:42.236 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-0tuF1jMbiXvpcwW off=9691545 len=46509 2024-04-07T16:46:42.237 INFO:tasks.workunit.client.0.smithi100.stdout:op 2488 completed, throughput=5MB/sec 2024-04-07T16:46:42.237 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-qIt6Am-s2uYhUJ8 off=3430155 len=692214 2024-04-07T16:46:42.241 INFO:tasks.workunit.client.0.smithi100.stdout:op 2489 completed, throughput=5MB/sec 2024-04-07T16:46:42.241 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-vTmTXEJd6o4E3dW off=12445827 len=66253 2024-04-07T16:46:42.244 INFO:tasks.workunit.client.0.smithi100.stdout:op 2490 completed, throughput=5MB/sec 2024-04-07T16:46:42.244 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-7HKr3q5Ip0UwRLF off=12905770 len=882188 2024-04-07T16:46:42.252 INFO:tasks.workunit.client.0.smithi100.stdout:op 2491 completed, throughput=5MB/sec 2024-04-07T16:46:42.252 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-AqBaoF0brMNJRe7 off=2203795 len=1961761 2024-04-07T16:46:42.270 INFO:tasks.workunit.client.0.smithi100.stdout:op 2492 completed, throughput=5MB/sec 2024-04-07T16:46:42.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:46:42 smithi144 bash[66654]: cluster 2024-04-07T16:46:41.324689+0000 mgr.y (mgr.25175) 940 : cluster [DBG] pgmap v1582: 105 pgs: 105 active+clean; 126 GiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 2.1 MiB/s rd, 2.7 MiB/s wr, 5 op/s 2024-04-07T16:46:42.609 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:46:42 smithi144 bash[95233]: ts=2024-04-07T16:46:42.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:46:42.694 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:46:42 smithi100 bash[74942]: cluster 2024-04-07T16:46:41.324689+0000 mgr.y (mgr.25175) 940 : cluster [DBG] pgmap v1582: 105 pgs: 105 active+clean; 126 GiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 2.1 MiB/s rd, 2.7 MiB/s wr, 5 op/s 2024-04-07T16:46:42.695 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:46:42 smithi100 bash[76202]: cluster 2024-04-07T16:46:41.324689+0000 mgr.y (mgr.25175) 940 : cluster [DBG] pgmap v1582: 105 pgs: 105 active+clean; 126 GiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 2.1 MiB/s rd, 2.7 MiB/s wr, 5 op/s 2024-04-07T16:46:43.270 INFO:tasks.workunit.client.0.smithi100.stdout: 471: throughput=4.99MB/sec pending data=0 2024-04-07T16:46:43.271 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-8bc0BZc5kpCOALX off=3497606 len=447042 2024-04-07T16:46:43.277 INFO:tasks.workunit.client.0.smithi100.stdout:op 2493 completed, throughput=4.99MB/sec 2024-04-07T16:46:43.278 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-l18eW2r_Q8Pgp-7 off=14079795 len=1199102 2024-04-07T16:46:43.288 INFO:tasks.workunit.client.0.smithi100.stdout:op 2494 completed, throughput=4.99MB/sec 2024-04-07T16:46:43.288 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-A8T_4Gqy4RSbP9d off=11220285 len=242401 2024-04-07T16:46:43.290 INFO:tasks.workunit.client.0.smithi100.stdout:op 2495 completed, throughput=5MB/sec 2024-04-07T16:46:43.290 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-q6ieZpg2kve-EXZ off=15428783 len=1835247 2024-04-07T16:46:43.303 INFO:tasks.workunit.client.0.smithi100.stdout:op 2496 completed, throughput=5MB/sec 2024-04-07T16:46:43.303 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-yuDDiB935FGewra off=6993668 len=1525731 2024-04-07T16:46:43.316 INFO:tasks.workunit.client.0.smithi100.stdout:op 2497 completed, throughput=5MB/sec 2024-04-07T16:46:44.316 INFO:tasks.workunit.client.0.smithi100.stdout: 472: throughput=4.99MB/sec pending data=0 2024-04-07T16:46:44.316 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-ldMSRYNM7LmAsSo off=5157489 len=1537230 2024-04-07T16:46:44.320 INFO:tasks.workunit.client.0.smithi100.stdout:op 2498 completed, throughput=4.99MB/sec 2024-04-07T16:46:44.320 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-QQUciSAiSCHj5ss off=12451424 len=819883 2024-04-07T16:46:44.324 INFO:tasks.workunit.client.0.smithi100.stdout:op 2499 completed, throughput=5MB/sec 2024-04-07T16:46:44.324 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-blKfHtRN6Ce1bxh off=3093350 len=959304 2024-04-07T16:46:44.326 INFO:tasks.workunit.client.0.smithi100.stdout:op 2500 completed, throughput=5MB/sec 2024-04-07T16:46:44.326 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-kr0WiADsAJIde16 off=150821 len=1354623 2024-04-07T16:46:44.330 INFO:tasks.workunit.client.0.smithi100.stdout:op 2501 completed, throughput=5MB/sec 2024-04-07T16:46:44.694 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:46:44 smithi100 bash[74942]: cluster 2024-04-07T16:46:43.325819+0000 mgr.y (mgr.25175) 941 : cluster [DBG] pgmap v1583: 105 pgs: 105 active+clean; 126 GiB data, 6.5 GiB used, 709 GiB / 715 GiB avail; 2.4 MiB/s rd, 3.3 MiB/s wr, 6 op/s 2024-04-07T16:46:44.695 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:46:44 smithi100 bash[76202]: cluster 2024-04-07T16:46:43.325819+0000 mgr.y (mgr.25175) 941 : cluster [DBG] pgmap v1583: 105 pgs: 105 active+clean; 126 GiB data, 6.5 GiB used, 709 GiB / 715 GiB avail; 2.4 MiB/s rd, 3.3 MiB/s wr, 6 op/s 2024-04-07T16:46:44.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:46:44 smithi144 bash[66654]: cluster 2024-04-07T16:46:43.325819+0000 mgr.y (mgr.25175) 941 : cluster [DBG] pgmap v1583: 105 pgs: 105 active+clean; 126 GiB data, 6.5 GiB used, 709 GiB / 715 GiB avail; 2.4 MiB/s rd, 3.3 MiB/s wr, 6 op/s 2024-04-07T16:46:45.330 INFO:tasks.workunit.client.0.smithi100.stdout: 473: throughput=4.99MB/sec pending data=0 2024-04-07T16:46:45.330 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Nz_oyG60PYonhr3 off=3770027 len=468755 2024-04-07T16:46:45.333 INFO:tasks.workunit.client.0.smithi100.stdout:op 2502 completed, throughput=4.99MB/sec 2024-04-07T16:46:45.333 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-2JY6rPh8HD72j1o off=12034392 len=1317513 2024-04-07T16:46:45.336 INFO:tasks.workunit.client.0.smithi100.stdout:op 2503 completed, throughput=4.99MB/sec 2024-04-07T16:46:45.337 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-ww4ue8o_9dPBGDS off=18234447 len=834043 2024-04-07T16:46:45.345 INFO:tasks.workunit.client.0.smithi100.stdout:op 2504 completed, throughput=5MB/sec 2024-04-07T16:46:45.345 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Hz_UIsKd02pzw62 off=16744460 len=2038493 2024-04-07T16:46:45.349 INFO:tasks.workunit.client.0.smithi100.stdout:op 2505 completed, throughput=5MB/sec 2024-04-07T16:46:45.350 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-YlBEvMerPkWfM7p off=5825787 len=869644 2024-04-07T16:46:45.358 INFO:tasks.workunit.client.0.smithi100.stdout:op 2506 completed, throughput=5MB/sec 2024-04-07T16:46:46.358 INFO:tasks.workunit.client.0.smithi100.stdout: 474: throughput=4.99MB/sec pending data=0 2024-04-07T16:46:46.359 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-8aWgmwVqRnj7B2t off=2045349 len=1561654 2024-04-07T16:46:46.371 INFO:tasks.workunit.client.0.smithi100.stdout:op 2507 completed, throughput=4.99MB/sec 2024-04-07T16:46:46.371 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-iqZpiWGyFsUBHvy off=14447582 len=323403 2024-04-07T16:46:46.376 INFO:tasks.workunit.client.0.smithi100.stdout:op 2508 completed, throughput=4.99MB/sec 2024-04-07T16:46:46.376 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-NP8s4PrCKed6SNF off=701622 len=1506954 2024-04-07T16:46:46.388 INFO:tasks.workunit.client.0.smithi100.stdout:op 2509 completed, throughput=5MB/sec 2024-04-07T16:46:46.388 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-IDMnwf7oow2Pnf_ off=7096125 len=232573 2024-04-07T16:46:46.389 INFO:tasks.workunit.client.0.smithi100.stdout:op 2510 completed, throughput=5MB/sec 2024-04-07T16:46:46.389 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-z13H15_fLrz8VPo off=452251 len=619314 2024-04-07T16:46:46.396 INFO:tasks.workunit.client.0.smithi100.stdout:op 2511 completed, throughput=5MB/sec 2024-04-07T16:46:46.396 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-tOElpeO0SsMiWR6 off=18078501 len=1106416 2024-04-07T16:46:46.405 INFO:tasks.workunit.client.0.smithi100.stdout:op 2512 completed, throughput=5MB/sec 2024-04-07T16:46:46.694 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:46:46 smithi100 bash[74942]: cluster 2024-04-07T16:46:45.326836+0000 mgr.y (mgr.25175) 942 : cluster [DBG] pgmap v1584: 105 pgs: 105 active+clean; 126 GiB data, 6.5 GiB used, 709 GiB / 715 GiB avail; 1.8 MiB/s rd, 2.5 MiB/s wr, 4 op/s 2024-04-07T16:46:46.694 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:46:46 smithi100 bash[76202]: cluster 2024-04-07T16:46:45.326836+0000 mgr.y (mgr.25175) 942 : cluster [DBG] pgmap v1584: 105 pgs: 105 active+clean; 126 GiB data, 6.5 GiB used, 709 GiB / 715 GiB avail; 1.8 MiB/s rd, 2.5 MiB/s wr, 4 op/s 2024-04-07T16:46:46.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:46:46 smithi144 bash[66654]: cluster 2024-04-07T16:46:45.326836+0000 mgr.y (mgr.25175) 942 : cluster [DBG] pgmap v1584: 105 pgs: 105 active+clean; 126 GiB data, 6.5 GiB used, 709 GiB / 715 GiB avail; 1.8 MiB/s rd, 2.5 MiB/s wr, 4 op/s 2024-04-07T16:46:47.405 INFO:tasks.workunit.client.0.smithi100.stdout: 475: throughput=4.99MB/sec pending data=0 2024-04-07T16:46:47.405 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-V6SqJQVIrSGKwog off=5315762 len=987090 2024-04-07T16:46:47.409 INFO:tasks.workunit.client.0.smithi100.stdout:op 2513 completed, throughput=4.99MB/sec 2024-04-07T16:46:47.409 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-ecrkOpaih-6pUir off=6255 len=1329041 2024-04-07T16:46:47.422 INFO:tasks.workunit.client.0.smithi100.stdout:op 2514 completed, throughput=4.99MB/sec 2024-04-07T16:46:47.422 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-2VyuRDUpU8izVEW off=5547753 len=865714 2024-04-07T16:46:47.429 INFO:tasks.workunit.client.0.smithi100.stdout:op 2515 completed, throughput=5MB/sec 2024-04-07T16:46:47.429 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-uJL1H_QnVRh72bD off=11151735 len=2039834 2024-04-07T16:46:47.444 INFO:tasks.workunit.client.0.smithi100.stdout:op 2516 completed, throughput=5MB/sec 2024-04-07T16:46:47.444 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-aVW0oZ9TxXoirHc off=14052865 len=1737681 2024-04-07T16:46:47.448 INFO:tasks.workunit.client.0.smithi100.stdout:op 2517 completed, throughput=5MB/sec 2024-04-07T16:46:47.694 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:46:47 smithi100 bash[74942]: audit 2024-04-07T16:46:46.769154+0000 mon.a (mon.0) 2112 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:46:47.695 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:46:47 smithi100 bash[76202]: audit 2024-04-07T16:46:46.769154+0000 mon.a (mon.0) 2112 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:46:47.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:46:47 smithi144 bash[66654]: audit 2024-04-07T16:46:46.769154+0000 mon.a (mon.0) 2112 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:46:48.449 INFO:tasks.workunit.client.0.smithi100.stdout: 476: throughput=4.99MB/sec pending data=0 2024-04-07T16:46:48.449 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-xy6_lgtivXmWLes off=9625837 len=1120138 2024-04-07T16:46:48.460 INFO:tasks.workunit.client.0.smithi100.stdout:op 2518 completed, throughput=5MB/sec 2024-04-07T16:46:48.460 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-c93aMl4Y3fnLdu2 off=6641898 len=1878854 2024-04-07T16:46:48.464 INFO:tasks.workunit.client.0.smithi100.stdout:op 2519 completed, throughput=5MB/sec 2024-04-07T16:46:48.464 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-UELlwjf9ntnKN-t off=6640130 len=1181884 2024-04-07T16:46:48.469 INFO:tasks.workunit.client.0.smithi100.stdout:op 2520 completed, throughput=5MB/sec 2024-04-07T16:46:48.694 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:46:48 smithi100 bash[74942]: cluster 2024-04-07T16:46:47.328043+0000 mgr.y (mgr.25175) 943 : cluster [DBG] pgmap v1585: 105 pgs: 105 active+clean; 126 GiB data, 6.5 GiB used, 709 GiB / 715 GiB avail; 2.1 MiB/s rd, 3.0 MiB/s wr, 5 op/s 2024-04-07T16:46:48.695 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:46:48 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:46:48] "GET /metrics HTTP/1.1" 200 34068 "" "Prometheus/2.43.0" 2024-04-07T16:46:48.695 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:46:48 smithi100 bash[76202]: cluster 2024-04-07T16:46:47.328043+0000 mgr.y (mgr.25175) 943 : cluster [DBG] pgmap v1585: 105 pgs: 105 active+clean; 126 GiB data, 6.5 GiB used, 709 GiB / 715 GiB avail; 2.1 MiB/s rd, 3.0 MiB/s wr, 5 op/s 2024-04-07T16:46:48.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:46:48 smithi144 bash[66654]: cluster 2024-04-07T16:46:47.328043+0000 mgr.y (mgr.25175) 943 : cluster [DBG] pgmap v1585: 105 pgs: 105 active+clean; 126 GiB data, 6.5 GiB used, 709 GiB / 715 GiB avail; 2.1 MiB/s rd, 3.0 MiB/s wr, 5 op/s 2024-04-07T16:46:49.469 INFO:tasks.workunit.client.0.smithi100.stdout: 477: throughput=4.99MB/sec pending data=0 2024-04-07T16:46:49.469 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-TEG77i6D9k9UN4k off=210345 len=442370 2024-04-07T16:46:49.471 INFO:tasks.workunit.client.0.smithi100.stdout:op 2521 completed, throughput=4.99MB/sec 2024-04-07T16:46:49.471 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-cS4G9vFKMg3Bnyc off=708654 len=535087 2024-04-07T16:46:49.479 INFO:tasks.workunit.client.0.smithi100.stdout:op 2522 completed, throughput=4.99MB/sec 2024-04-07T16:46:49.479 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-yvh0HDIaUBhhVa6 off=18156369 len=1958264 2024-04-07T16:46:49.483 INFO:tasks.workunit.client.0.smithi100.stdout:op 2523 completed, throughput=5MB/sec 2024-04-07T16:46:49.483 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-euSiQrnXG-yzxoA off=1362490 len=1039577 2024-04-07T16:46:49.486 INFO:tasks.workunit.client.0.smithi100.stdout:op 2524 completed, throughput=5MB/sec 2024-04-07T16:46:49.486 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-dX-4IBuoTsQAN4D off=10238094 len=757608 2024-04-07T16:46:49.489 INFO:tasks.workunit.client.0.smithi100.stdout:op 2525 completed, throughput=5MB/sec 2024-04-07T16:46:49.489 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-1fGsUGtU_rbXMQY off=4131102 len=550080 2024-04-07T16:46:49.491 INFO:tasks.workunit.client.0.smithi100.stdout:op 2526 completed, throughput=5MB/sec 2024-04-07T16:46:50.491 INFO:tasks.workunit.client.0.smithi100.stdout: 478: throughput=4.99MB/sec pending data=0 2024-04-07T16:46:50.492 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-jwLbI_BX90wp43S off=10572538 len=1201087 2024-04-07T16:46:50.494 INFO:tasks.workunit.client.0.smithi100.stdout:op 2527 completed, throughput=4.99MB/sec 2024-04-07T16:46:50.495 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-q6FrJfJqbfKbBU4 off=13503847 len=1305058 2024-04-07T16:46:50.499 INFO:tasks.workunit.client.0.smithi100.stdout:op 2528 completed, throughput=5MB/sec 2024-04-07T16:46:50.499 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-uXt2vHxcbcuAlX2 off=20761987 len=1078858 2024-04-07T16:46:50.501 INFO:tasks.workunit.client.0.smithi100.stdout:op 2529 completed, throughput=5MB/sec 2024-04-07T16:46:50.501 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Ls6XuYFOuj-3OG2 off=20721 len=862495 2024-04-07T16:46:50.505 INFO:tasks.workunit.client.0.smithi100.stdout:op 2530 completed, throughput=5MB/sec 2024-04-07T16:46:50.505 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Mu0uHf7AmHSE_j1 off=6888138 len=1295843 2024-04-07T16:46:50.516 INFO:tasks.workunit.client.0.smithi100.stdout:op 2531 completed, throughput=5MB/sec 2024-04-07T16:46:50.694 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:46:50 smithi100 bash[74942]: cluster 2024-04-07T16:46:49.329005+0000 mgr.y (mgr.25175) 944 : cluster [DBG] pgmap v1586: 105 pgs: 105 active+clean; 126 GiB data, 6.5 GiB used, 709 GiB / 715 GiB avail; 1.6 MiB/s rd, 3.1 MiB/s wr, 4 op/s 2024-04-07T16:46:50.695 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:46:50 smithi100 bash[76202]: cluster 2024-04-07T16:46:49.329005+0000 mgr.y (mgr.25175) 944 : cluster [DBG] pgmap v1586: 105 pgs: 105 active+clean; 126 GiB data, 6.5 GiB used, 709 GiB / 715 GiB avail; 1.6 MiB/s rd, 3.1 MiB/s wr, 4 op/s 2024-04-07T16:46:50.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:46:50 smithi144 bash[66654]: cluster 2024-04-07T16:46:49.329005+0000 mgr.y (mgr.25175) 944 : cluster [DBG] pgmap v1586: 105 pgs: 105 active+clean; 126 GiB data, 6.5 GiB used, 709 GiB / 715 GiB avail; 1.6 MiB/s rd, 3.1 MiB/s wr, 4 op/s 2024-04-07T16:46:51.517 INFO:tasks.workunit.client.0.smithi100.stdout: 479: throughput=4.99MB/sec pending data=0 2024-04-07T16:46:51.517 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-PUF2ZnyeHwQZ_ZC off=10276910 len=863806 2024-04-07T16:46:51.525 INFO:tasks.workunit.client.0.smithi100.stdout:op 2532 completed, throughput=4.99MB/sec 2024-04-07T16:46:51.525 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Cb4cdmmjUhR1wdO off=3205215 len=1765907 2024-04-07T16:46:51.538 INFO:tasks.workunit.client.0.smithi100.stdout:op 2533 completed, throughput=5MB/sec 2024-04-07T16:46:51.539 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-SHYRUFpqFIi9ujf off=3236206 len=789392 2024-04-07T16:46:51.541 INFO:tasks.workunit.client.0.smithi100.stdout:op 2534 completed, throughput=5MB/sec 2024-04-07T16:46:51.541 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-ppAPH7R9tHPwe20 off=14885564 len=1589926 2024-04-07T16:46:51.545 INFO:tasks.workunit.client.0.smithi100.stdout:op 2535 completed, throughput=5MB/sec 2024-04-07T16:46:52.545 INFO:tasks.workunit.client.0.smithi100.stdout: 480: throughput=4.99MB/sec pending data=0 2024-04-07T16:46:52.545 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-yy_eKx82lGEsa8a off=1139567 len=423663 2024-04-07T16:46:52.547 INFO:tasks.workunit.client.0.smithi100.stdout:op 2536 completed, throughput=4.99MB/sec 2024-04-07T16:46:52.547 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-pwR6_Hx0cEyeYkO off=2404127 len=372938 2024-04-07T16:46:52.549 INFO:tasks.workunit.client.0.smithi100.stdout:op 2537 completed, throughput=4.99MB/sec 2024-04-07T16:46:52.549 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-77IIA_OTT0qcCt_ off=1657677 len=1506612 2024-04-07T16:46:52.561 INFO:tasks.workunit.client.0.smithi100.stdout:op 2538 completed, throughput=4.99MB/sec 2024-04-07T16:46:52.561 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-pGwcQ1uuL5kXv61 off=1780923 len=1010883 2024-04-07T16:46:52.570 INFO:tasks.workunit.client.0.smithi100.stdout:op 2539 completed, throughput=5MB/sec 2024-04-07T16:46:52.570 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-xurH2kx4YDJ0eLs off=5022543 len=4483 2024-04-07T16:46:52.571 INFO:tasks.workunit.client.0.smithi100.stdout:op 2540 completed, throughput=5MB/sec 2024-04-07T16:46:52.571 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-OTvV8Vpi4oLWYa3 off=7505955 len=1753616 2024-04-07T16:46:52.586 INFO:tasks.workunit.client.0.smithi100.stdout:op 2541 completed, throughput=5MB/sec 2024-04-07T16:46:52.586 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-h3-Hm-nxjDMix6A off=3515383 len=969753 2024-04-07T16:46:52.589 INFO:tasks.workunit.client.0.smithi100.stdout:op 2542 completed, throughput=5MB/sec 2024-04-07T16:46:52.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:46:52 smithi144 bash[66654]: cluster 2024-04-07T16:46:51.330063+0000 mgr.y (mgr.25175) 945 : cluster [DBG] pgmap v1587: 105 pgs: 105 active+clean; 126 GiB data, 6.5 GiB used, 709 GiB / 715 GiB avail; 2.2 MiB/s rd, 3.2 MiB/s wr, 5 op/s 2024-04-07T16:46:52.609 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:46:52 smithi144 bash[95233]: ts=2024-04-07T16:46:52.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:46:52.694 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:46:52 smithi100 bash[74942]: cluster 2024-04-07T16:46:51.330063+0000 mgr.y (mgr.25175) 945 : cluster [DBG] pgmap v1587: 105 pgs: 105 active+clean; 126 GiB data, 6.5 GiB used, 709 GiB / 715 GiB avail; 2.2 MiB/s rd, 3.2 MiB/s wr, 5 op/s 2024-04-07T16:46:52.695 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:46:52 smithi100 bash[76202]: cluster 2024-04-07T16:46:51.330063+0000 mgr.y (mgr.25175) 945 : cluster [DBG] pgmap v1587: 105 pgs: 105 active+clean; 126 GiB data, 6.5 GiB used, 709 GiB / 715 GiB avail; 2.2 MiB/s rd, 3.2 MiB/s wr, 5 op/s 2024-04-07T16:46:53.589 INFO:tasks.workunit.client.0.smithi100.stdout: 481: throughput=4.99MB/sec pending data=0 2024-04-07T16:46:53.590 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-H6LiSIACKlI8B2t off=7333408 len=2064204 2024-04-07T16:46:53.604 INFO:tasks.workunit.client.0.smithi100.stdout:op 2543 completed, throughput=5MB/sec 2024-04-07T16:46:53.604 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-6exQWJr2TBDMbuz off=622472 len=1049673 2024-04-07T16:46:53.614 INFO:tasks.workunit.client.0.smithi100.stdout:op 2544 completed, throughput=5MB/sec 2024-04-07T16:46:53.614 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Atea1Zt2Bk1qEwx off=3355864 len=1349941 2024-04-07T16:46:53.619 INFO:tasks.workunit.client.0.smithi100.stdout:op 2545 completed, throughput=5MB/sec 2024-04-07T16:46:54.620 INFO:tasks.workunit.client.0.smithi100.stdout: 482: throughput=4.99MB/sec pending data=0 2024-04-07T16:46:54.620 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-hODB3bcscdjc55L off=10959850 len=81992 2024-04-07T16:46:54.625 INFO:tasks.workunit.client.0.smithi100.stdout:op 2546 completed, throughput=4.99MB/sec 2024-04-07T16:46:54.625 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-FWi-t8tsVGuFR-m off=3149427 len=443435 2024-04-07T16:46:54.631 INFO:tasks.workunit.client.0.smithi100.stdout:op 2547 completed, throughput=4.99MB/sec 2024-04-07T16:46:54.631 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-QhTlVqVfJW3GZ6Q off=4326399 len=192583 2024-04-07T16:46:54.636 INFO:tasks.workunit.client.0.smithi100.stdout:op 2548 completed, throughput=4.99MB/sec 2024-04-07T16:46:54.636 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-mBsWotsBDjMD3GW off=11555693 len=1107367 2024-04-07T16:46:54.639 INFO:tasks.workunit.client.0.smithi100.stdout:op 2549 completed, throughput=4.99MB/sec 2024-04-07T16:46:54.640 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-WGThfl9jaSws4dS off=1836117 len=1017855 2024-04-07T16:46:54.642 INFO:tasks.workunit.client.0.smithi100.stdout:op 2550 completed, throughput=5MB/sec 2024-04-07T16:46:54.642 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-DQuvVNEICMS1zNI off=8268436 len=194745 2024-04-07T16:46:54.642 INFO:tasks.workunit.client.0.smithi100.stdout:op 2551 completed, throughput=5MB/sec 2024-04-07T16:46:54.643 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-yd6lgX0h58WezMF off=3849200 len=1565822 2024-04-07T16:46:54.648 INFO:tasks.workunit.client.0.smithi100.stdout:op 2552 completed, throughput=5MB/sec 2024-04-07T16:46:54.648 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-mVxC5QHsbIBCY5S off=1708533 len=214320 2024-04-07T16:46:54.649 INFO:tasks.workunit.client.0.smithi100.stdout:op 2553 completed, throughput=5MB/sec 2024-04-07T16:46:54.649 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-pru7WAXwqqiJJm8 off=10473103 len=985483 2024-04-07T16:46:54.652 INFO:tasks.workunit.client.0.smithi100.stdout:op 2554 completed, throughput=5MB/sec 2024-04-07T16:46:54.694 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:46:54 smithi100 bash[74942]: cluster 2024-04-07T16:46:53.331191+0000 mgr.y (mgr.25175) 946 : cluster [DBG] pgmap v1588: 105 pgs: 105 active+clean; 126 GiB data, 6.5 GiB used, 709 GiB / 715 GiB avail; 2.3 MiB/s rd, 3.0 MiB/s wr, 5 op/s 2024-04-07T16:46:54.695 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:46:54 smithi100 bash[76202]: cluster 2024-04-07T16:46:53.331191+0000 mgr.y (mgr.25175) 946 : cluster [DBG] pgmap v1588: 105 pgs: 105 active+clean; 126 GiB data, 6.5 GiB used, 709 GiB / 715 GiB avail; 2.3 MiB/s rd, 3.0 MiB/s wr, 5 op/s 2024-04-07T16:46:54.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:46:54 smithi144 bash[66654]: cluster 2024-04-07T16:46:53.331191+0000 mgr.y (mgr.25175) 946 : cluster [DBG] pgmap v1588: 105 pgs: 105 active+clean; 126 GiB data, 6.5 GiB used, 709 GiB / 715 GiB avail; 2.3 MiB/s rd, 3.0 MiB/s wr, 5 op/s 2024-04-07T16:46:55.653 INFO:tasks.workunit.client.0.smithi100.stdout: 483: throughput=4.99MB/sec pending data=0 2024-04-07T16:46:55.653 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-_2IfeULT2gWifG2 off=2114196 len=1951518 2024-04-07T16:46:55.656 INFO:tasks.workunit.client.0.smithi100.stdout:op 2555 completed, throughput=4.99MB/sec 2024-04-07T16:46:55.657 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-VGZirp4pCvAayrZ off=2088037 len=1235641 2024-04-07T16:46:55.658 INFO:tasks.workunit.client.0.smithi100.stdout:op 2556 completed, throughput=5MB/sec 2024-04-07T16:46:55.658 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-ctq8HrBLZd8fX2P off=4141618 len=1074704 2024-04-07T16:46:55.660 INFO:tasks.workunit.client.0.smithi100.stdout:op 2557 completed, throughput=5MB/sec 2024-04-07T16:46:55.660 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-7EbFtxoDSnEVszi off=13394113 len=971998 2024-04-07T16:46:55.668 INFO:tasks.workunit.client.0.smithi100.stdout:op 2558 completed, throughput=5MB/sec 2024-04-07T16:46:55.694 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:46:55 smithi100 bash[74942]: audit 2024-04-07T16:46:55.297773+0000 mon.a (mon.0) 2113 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:46:55.695 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:46:55 smithi100 bash[76202]: audit 2024-04-07T16:46:55.297773+0000 mon.a (mon.0) 2113 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:46:55.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:46:55 smithi144 bash[66654]: audit 2024-04-07T16:46:55.297773+0000 mon.a (mon.0) 2113 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:46:56.668 INFO:tasks.workunit.client.0.smithi100.stdout: 484: throughput=4.99MB/sec pending data=0 2024-04-07T16:46:56.668 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-E_jmoxcw3XQJZCU off=2041574 len=907570 2024-04-07T16:46:56.671 INFO:tasks.workunit.client.0.smithi100.stdout:op 2559 completed, throughput=4.99MB/sec 2024-04-07T16:46:56.671 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-srtroS6ja8sqvVD off=6350237 len=1823000 2024-04-07T16:46:56.675 INFO:tasks.workunit.client.0.smithi100.stdout:op 2560 completed, throughput=5MB/sec 2024-04-07T16:46:56.675 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-mikkkyyaC28rdAS off=744310 len=804582 2024-04-07T16:46:56.683 INFO:tasks.workunit.client.0.smithi100.stdout:op 2561 completed, throughput=5MB/sec 2024-04-07T16:46:56.683 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-aQSBPxOaO8FqSJU off=17647470 len=2071933 2024-04-07T16:46:56.694 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:46:56 smithi100 bash[74942]: cluster 2024-04-07T16:46:55.332399+0000 mgr.y (mgr.25175) 947 : cluster [DBG] pgmap v1589: 105 pgs: 105 active+clean; 126 GiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 2.2 MiB/s rd, 2.8 MiB/s wr, 4 op/s 2024-04-07T16:46:56.694 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:46:56 smithi100 bash[74942]: audit 2024-04-07T16:46:56.032401+0000 mon.a (mon.0) 2114 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:46:56.695 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:46:56 smithi100 bash[74942]: audit 2024-04-07T16:46:56.034407+0000 mon.a (mon.0) 2115 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:46:56.695 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:46:56 smithi100 bash[74942]: audit 2024-04-07T16:46:56.044992+0000 mon.a (mon.0) 2116 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:46:56.695 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:46:56 smithi100 bash[76202]: cluster 2024-04-07T16:46:55.332399+0000 mgr.y (mgr.25175) 947 : cluster [DBG] pgmap v1589: 105 pgs: 105 active+clean; 126 GiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 2.2 MiB/s rd, 2.8 MiB/s wr, 4 op/s 2024-04-07T16:46:56.695 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:46:56 smithi100 bash[76202]: audit 2024-04-07T16:46:56.032401+0000 mon.a (mon.0) 2114 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:46:56.695 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:46:56 smithi100 bash[76202]: audit 2024-04-07T16:46:56.034407+0000 mon.a (mon.0) 2115 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:46:56.695 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:46:56 smithi100 bash[76202]: audit 2024-04-07T16:46:56.044992+0000 mon.a (mon.0) 2116 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:46:56.697 INFO:tasks.workunit.client.0.smithi100.stdout:op 2562 completed, throughput=5MB/sec 2024-04-07T16:46:56.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:46:56 smithi144 bash[66654]: cluster 2024-04-07T16:46:55.332399+0000 mgr.y (mgr.25175) 947 : cluster [DBG] pgmap v1589: 105 pgs: 105 active+clean; 126 GiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 2.2 MiB/s rd, 2.8 MiB/s wr, 4 op/s 2024-04-07T16:46:56.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:46:56 smithi144 bash[66654]: audit 2024-04-07T16:46:56.032401+0000 mon.a (mon.0) 2114 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:46:56.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:46:56 smithi144 bash[66654]: audit 2024-04-07T16:46:56.034407+0000 mon.a (mon.0) 2115 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:46:56.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:46:56 smithi144 bash[66654]: audit 2024-04-07T16:46:56.044992+0000 mon.a (mon.0) 2116 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:46:57.698 INFO:tasks.workunit.client.0.smithi100.stdout: 485: throughput=4.99MB/sec pending data=0 2024-04-07T16:46:57.698 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-qsdq0obHBHJ7m29 off=15732292 len=192412 2024-04-07T16:46:57.699 INFO:tasks.workunit.client.0.smithi100.stdout:op 2563 completed, throughput=4.99MB/sec 2024-04-07T16:46:57.699 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-SfAFpLXQsKvNNp1 off=1618575 len=877480 2024-04-07T16:46:57.703 INFO:tasks.workunit.client.0.smithi100.stdout:op 2564 completed, throughput=4.99MB/sec 2024-04-07T16:46:57.703 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-xEtyYsDETS-TWgo off=14561908 len=952302 2024-04-07T16:46:57.705 INFO:tasks.workunit.client.0.smithi100.stdout:op 2565 completed, throughput=4.99MB/sec 2024-04-07T16:46:57.706 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-asX1gRRTjIWQa8- off=5894812 len=517153 2024-04-07T16:46:57.713 INFO:tasks.workunit.client.0.smithi100.stdout:op 2566 completed, throughput=5MB/sec 2024-04-07T16:46:57.713 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-RZ2fUypFAFqFdxe off=10648512 len=1128494 2024-04-07T16:46:57.716 INFO:tasks.workunit.client.0.smithi100.stdout:op 2567 completed, throughput=5MB/sec 2024-04-07T16:46:57.716 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-VdY27zUphg4PMbV off=11481695 len=421999 2024-04-07T16:46:57.722 INFO:tasks.workunit.client.0.smithi100.stdout:op 2568 completed, throughput=5MB/sec 2024-04-07T16:46:57.723 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-9kduKJevUpjGtr7 off=1054817 len=395458 2024-04-07T16:46:57.728 INFO:tasks.workunit.client.0.smithi100.stdout:op 2569 completed, throughput=5MB/sec 2024-04-07T16:46:57.729 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-CdjMa-fWpb7DjJA off=10646075 len=1120589 2024-04-07T16:46:57.738 INFO:tasks.workunit.client.0.smithi100.stdout:op 2570 completed, throughput=5MB/sec 2024-04-07T16:46:58.694 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:46:58 smithi100 bash[74942]: cluster 2024-04-07T16:46:57.333466+0000 mgr.y (mgr.25175) 948 : cluster [DBG] pgmap v1590: 105 pgs: 105 active+clean; 126 GiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 2.6 MiB/s rd, 2.6 MiB/s wr, 5 op/s 2024-04-07T16:46:58.695 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:46:58 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:46:58] "GET /metrics HTTP/1.1" 200 34068 "" "Prometheus/2.43.0" 2024-04-07T16:46:58.695 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:46:58 smithi100 bash[76202]: cluster 2024-04-07T16:46:57.333466+0000 mgr.y (mgr.25175) 948 : cluster [DBG] pgmap v1590: 105 pgs: 105 active+clean; 126 GiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 2.6 MiB/s rd, 2.6 MiB/s wr, 5 op/s 2024-04-07T16:46:58.738 INFO:tasks.workunit.client.0.smithi100.stdout: 486: throughput=4.99MB/sec pending data=0 2024-04-07T16:46:58.738 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-wi8_MtYmOjYCwqj off=9554278 len=474886 2024-04-07T16:46:58.740 INFO:tasks.workunit.client.0.smithi100.stdout:op 2571 completed, throughput=4.99MB/sec 2024-04-07T16:46:58.740 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Tfixe7yF7LYXMN9 off=3785870 len=1870128 2024-04-07T16:46:58.754 INFO:tasks.workunit.client.0.smithi100.stdout:op 2572 completed, throughput=5MB/sec 2024-04-07T16:46:58.754 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Ip6x5uc5vKcVepb off=4690663 len=1490785 2024-04-07T16:46:58.758 INFO:tasks.workunit.client.0.smithi100.stdout:op 2573 completed, throughput=5MB/sec 2024-04-07T16:46:58.758 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-nx7gbgL8yFjM-hk off=5781490 len=1873285 2024-04-07T16:46:58.772 INFO:tasks.workunit.client.0.smithi100.stdout:op 2574 completed, throughput=5MB/sec 2024-04-07T16:46:58.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:46:58 smithi144 bash[66654]: cluster 2024-04-07T16:46:57.333466+0000 mgr.y (mgr.25175) 948 : cluster [DBG] pgmap v1590: 105 pgs: 105 active+clean; 126 GiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 2.6 MiB/s rd, 2.6 MiB/s wr, 5 op/s 2024-04-07T16:46:59.773 INFO:tasks.workunit.client.0.smithi100.stdout: 487: throughput=4.99MB/sec pending data=0 2024-04-07T16:46:59.773 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-kQ2enFKpg80_W6J off=2457289 len=1322572 2024-04-07T16:46:59.776 INFO:tasks.workunit.client.0.smithi100.stdout:op 2575 completed, throughput=4.99MB/sec 2024-04-07T16:46:59.776 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-8BwQdXVcYsM1ZDb off=363801 len=1749800 2024-04-07T16:46:59.791 INFO:tasks.workunit.client.0.smithi100.stdout:op 2576 completed, throughput=5MB/sec 2024-04-07T16:46:59.791 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-k3VqK6dBaO6TJNl off=3911140 len=305382 2024-04-07T16:46:59.792 INFO:tasks.workunit.client.0.smithi100.stdout:op 2577 completed, throughput=5MB/sec 2024-04-07T16:46:59.792 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-vuQKF1cJ6qZaBKB off=3152746 len=1228137 2024-04-07T16:46:59.796 INFO:tasks.workunit.client.0.smithi100.stdout:op 2578 completed, throughput=5MB/sec 2024-04-07T16:47:00.694 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:47:00 smithi100 bash[74942]: cluster 2024-04-07T16:46:59.334293+0000 mgr.y (mgr.25175) 949 : cluster [DBG] pgmap v1591: 105 pgs: 105 active+clean; 126 GiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 2.4 MiB/s rd, 1.9 MiB/s wr, 4 op/s 2024-04-07T16:47:00.695 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:47:00 smithi100 bash[76202]: cluster 2024-04-07T16:46:59.334293+0000 mgr.y (mgr.25175) 949 : cluster [DBG] pgmap v1591: 105 pgs: 105 active+clean; 126 GiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 2.4 MiB/s rd, 1.9 MiB/s wr, 4 op/s 2024-04-07T16:47:00.797 INFO:tasks.workunit.client.0.smithi100.stdout: 488: throughput=4.99MB/sec pending data=0 2024-04-07T16:47:00.797 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-mHyhHm6EeKllSLS off=21642039 len=576107 2024-04-07T16:47:00.799 INFO:tasks.workunit.client.0.smithi100.stdout:op 2579 completed, throughput=4.99MB/sec 2024-04-07T16:47:00.799 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Cc3R7k8dEyQDgRe off=296736 len=32847 2024-04-07T16:47:00.800 INFO:tasks.workunit.client.0.smithi100.stdout:op 2580 completed, throughput=4.99MB/sec 2024-04-07T16:47:00.800 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-aIiH7rOy5sOIDn6 off=391218 len=1233214 2024-04-07T16:47:00.803 INFO:tasks.workunit.client.0.smithi100.stdout:op 2581 completed, throughput=4.99MB/sec 2024-04-07T16:47:00.803 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-_godbPE8Qu3HVWW off=4166000 len=362361 2024-04-07T16:47:00.805 INFO:tasks.workunit.client.0.smithi100.stdout:op 2582 completed, throughput=4.99MB/sec 2024-04-07T16:47:00.805 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-lXyy9aAp5-LdZ4y off=16056731 len=1742920 2024-04-07T16:47:00.811 INFO:tasks.workunit.client.0.smithi100.stdout:op 2583 completed, throughput=5MB/sec 2024-04-07T16:47:00.811 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-a4Af3bXqdWh2MeA off=17388903 len=1580824 2024-04-07T16:47:00.824 INFO:tasks.workunit.client.0.smithi100.stdout:op 2584 completed, throughput=5MB/sec 2024-04-07T16:47:00.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:47:00 smithi144 bash[66654]: cluster 2024-04-07T16:46:59.334293+0000 mgr.y (mgr.25175) 949 : cluster [DBG] pgmap v1591: 105 pgs: 105 active+clean; 126 GiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 2.4 MiB/s rd, 1.9 MiB/s wr, 4 op/s 2024-04-07T16:47:01.824 INFO:tasks.workunit.client.0.smithi100.stdout: 489: throughput=4.99MB/sec pending data=0 2024-04-07T16:47:01.824 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-zgMnYu-hN8XVTj9 off=374420 len=1566996 2024-04-07T16:47:01.839 INFO:tasks.workunit.client.0.smithi100.stdout:op 2585 completed, throughput=4.99MB/sec 2024-04-07T16:47:01.839 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-KyyJHvAQYR0-k0e off=0 len=1737640 2024-04-07T16:47:01.843 INFO:tasks.workunit.client.0.smithi100.stdout:op 2586 completed, throughput=5MB/sec 2024-04-07T16:47:01.843 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-aVW0oZ9TxXoirHc off=14020487 len=1353108 2024-04-07T16:47:01.845 INFO:tasks.workunit.client.0.smithi100.stdout:op 2587 completed, throughput=5MB/sec 2024-04-07T16:47:01.845 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-_godbPE8Qu3HVWW off=1534280 len=2090353 2024-04-07T16:47:01.852 INFO:tasks.workunit.client.0.smithi100.stdout:op 2588 completed, throughput=5MB/sec 2024-04-07T16:47:02.609 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:47:02 smithi144 bash[95233]: ts=2024-04-07T16:47:02.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:47:02.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:47:02 smithi144 bash[66654]: cluster 2024-04-07T16:47:01.335297+0000 mgr.y (mgr.25175) 950 : cluster [DBG] pgmap v1592: 105 pgs: 105 active+clean; 126 GiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 2.8 MiB/s rd, 2.1 MiB/s wr, 4 op/s 2024-04-07T16:47:02.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:47:02 smithi144 bash[66654]: audit 2024-04-07T16:47:01.769493+0000 mon.a (mon.0) 2117 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:47:02.853 INFO:tasks.workunit.client.0.smithi100.stdout: 490: throughput=4.99MB/sec pending data=0 2024-04-07T16:47:02.853 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-4tAE24FaCPUX40t off=48237 len=1178933 2024-04-07T16:47:02.856 INFO:tasks.workunit.client.0.smithi100.stdout:op 2589 completed, throughput=5MB/sec 2024-04-07T16:47:02.856 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-CfpWc4i-pJSqaSO off=4644524 len=585887 2024-04-07T16:47:02.863 INFO:tasks.workunit.client.0.smithi100.stdout:op 2590 completed, throughput=5MB/sec 2024-04-07T16:47:02.863 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Twp15MLmCEYmv2N off=361833 len=1550777 2024-04-07T16:47:02.867 INFO:tasks.workunit.client.0.smithi100.stdout:op 2591 completed, throughput=5MB/sec 2024-04-07T16:47:02.867 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-AbZiPvdSdf7kF8D off=7596498 len=1480312 2024-04-07T16:47:02.871 INFO:tasks.workunit.client.0.smithi100.stdout:op 2592 completed, throughput=5MB/sec 2024-04-07T16:47:02.944 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:47:02 smithi100 bash[74942]: cluster 2024-04-07T16:47:01.335297+0000 mgr.y (mgr.25175) 950 : cluster [DBG] pgmap v1592: 105 pgs: 105 active+clean; 126 GiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 2.8 MiB/s rd, 2.1 MiB/s wr, 4 op/s 2024-04-07T16:47:02.945 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:47:02 smithi100 bash[74942]: audit 2024-04-07T16:47:01.769493+0000 mon.a (mon.0) 2117 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:47:02.945 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:47:02 smithi100 bash[76202]: cluster 2024-04-07T16:47:01.335297+0000 mgr.y (mgr.25175) 950 : cluster [DBG] pgmap v1592: 105 pgs: 105 active+clean; 126 GiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 2.8 MiB/s rd, 2.1 MiB/s wr, 4 op/s 2024-04-07T16:47:02.945 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:47:02 smithi100 bash[76202]: audit 2024-04-07T16:47:01.769493+0000 mon.a (mon.0) 2117 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:47:03.871 INFO:tasks.workunit.client.0.smithi100.stdout: 491: throughput=4.99MB/sec pending data=0 2024-04-07T16:47:03.871 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-rIgDbnMFb9Hsj4U off=6606673 len=241719 2024-04-07T16:47:03.877 INFO:tasks.workunit.client.0.smithi100.stdout:op 2593 completed, throughput=4.99MB/sec 2024-04-07T16:47:03.877 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-eUSz4siUDZNjaxu off=4340827 len=1528731 2024-04-07T16:47:03.881 INFO:tasks.workunit.client.0.smithi100.stdout:op 2594 completed, throughput=5MB/sec 2024-04-07T16:47:03.881 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Cjjb_UgiS3m9-69 off=6106338 len=878291 2024-04-07T16:47:03.892 INFO:tasks.workunit.client.0.smithi100.stdout:op 2595 completed, throughput=5MB/sec 2024-04-07T16:47:03.892 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-ldMSRYNM7LmAsSo off=9298112 len=1636365 2024-04-07T16:47:03.896 INFO:tasks.workunit.client.0.smithi100.stdout:op 2596 completed, throughput=5MB/sec 2024-04-07T16:47:04.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:47:04 smithi144 bash[66654]: cluster 2024-04-07T16:47:03.336540+0000 mgr.y (mgr.25175) 951 : cluster [DBG] pgmap v1593: 105 pgs: 105 active+clean; 126 GiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 3.1 MiB/s rd, 2.3 MiB/s wr, 5 op/s 2024-04-07T16:47:04.896 INFO:tasks.workunit.client.0.smithi100.stdout: 492: throughput=4.99MB/sec pending data=0 2024-04-07T16:47:04.896 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-RHIgKbia2twUKjP off=280961 len=639767 2024-04-07T16:47:04.899 INFO:tasks.workunit.client.0.smithi100.stdout:op 2597 completed, throughput=4.99MB/sec 2024-04-07T16:47:04.899 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-kj-ogXfnI-ZhLR6 off=10125647 len=855057 2024-04-07T16:47:04.907 INFO:tasks.workunit.client.0.smithi100.stdout:op 2598 completed, throughput=4.99MB/sec 2024-04-07T16:47:04.908 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-FM1U9aJyh4mriJm off=18887992 len=1418986 2024-04-07T16:47:04.912 INFO:tasks.workunit.client.0.smithi100.stdout:op 2599 completed, throughput=5MB/sec 2024-04-07T16:47:04.912 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-ugLi_PK4GdRGOi5 off=16694833 len=2025429 2024-04-07T16:47:04.928 INFO:tasks.workunit.client.0.smithi100.stdout:op 2600 completed, throughput=5MB/sec 2024-04-07T16:47:04.928 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-dwphwGpwRbFwu8u off=4719666 len=1518559 2024-04-07T16:47:04.938 INFO:tasks.workunit.client.0.smithi100.stdout:op 2601 completed, throughput=5MB/sec 2024-04-07T16:47:04.944 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:47:04 smithi100 bash[74942]: cluster 2024-04-07T16:47:03.336540+0000 mgr.y (mgr.25175) 951 : cluster [DBG] pgmap v1593: 105 pgs: 105 active+clean; 126 GiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 3.1 MiB/s rd, 2.3 MiB/s wr, 5 op/s 2024-04-07T16:47:04.944 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:47:04 smithi100 bash[76202]: cluster 2024-04-07T16:47:03.336540+0000 mgr.y (mgr.25175) 951 : cluster [DBG] pgmap v1593: 105 pgs: 105 active+clean; 126 GiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 3.1 MiB/s rd, 2.3 MiB/s wr, 5 op/s 2024-04-07T16:47:05.939 INFO:tasks.workunit.client.0.smithi100.stdout: 493: throughput=4.99MB/sec pending data=0 2024-04-07T16:47:05.939 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-M9B_DVFQ4DCTwxE off=3023637 len=1337043 2024-04-07T16:47:05.952 INFO:tasks.workunit.client.0.smithi100.stdout:op 2602 completed, throughput=4.99MB/sec 2024-04-07T16:47:05.952 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-jrGtQiYQU4ntcol off=6443222 len=1857308 2024-04-07T16:47:05.965 INFO:tasks.workunit.client.0.smithi100.stdout:op 2603 completed, throughput=5MB/sec 2024-04-07T16:47:05.966 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-H6mNPZ_2QpX-JWo off=13684976 len=2076158 2024-04-07T16:47:05.984 INFO:tasks.workunit.client.0.smithi100.stdout:op 2604 completed, throughput=5MB/sec 2024-04-07T16:47:06.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:47:06 smithi144 bash[66654]: cluster 2024-04-07T16:47:05.337436+0000 mgr.y (mgr.25175) 952 : cluster [DBG] pgmap v1594: 105 pgs: 105 active+clean; 126 GiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 3.1 MiB/s rd, 1.9 MiB/s wr, 4 op/s 2024-04-07T16:47:06.944 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:47:06 smithi100 bash[74942]: cluster 2024-04-07T16:47:05.337436+0000 mgr.y (mgr.25175) 952 : cluster [DBG] pgmap v1594: 105 pgs: 105 active+clean; 126 GiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 3.1 MiB/s rd, 1.9 MiB/s wr, 4 op/s 2024-04-07T16:47:06.945 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:47:06 smithi100 bash[76202]: cluster 2024-04-07T16:47:05.337436+0000 mgr.y (mgr.25175) 952 : cluster [DBG] pgmap v1594: 105 pgs: 105 active+clean; 126 GiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 3.1 MiB/s rd, 1.9 MiB/s wr, 4 op/s 2024-04-07T16:47:06.984 INFO:tasks.workunit.client.0.smithi100.stdout: 494: throughput=4.99MB/sec pending data=0 2024-04-07T16:47:06.984 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-CPcg4_MkdZx7CF9 off=13990844 len=1953004 2024-04-07T16:47:06.989 INFO:tasks.workunit.client.0.smithi100.stdout:op 2605 completed, throughput=5MB/sec 2024-04-07T16:47:06.989 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-U7ZzL5yWgu5wvBm off=4290815 len=453898 2024-04-07T16:47:06.996 INFO:tasks.workunit.client.0.smithi100.stdout:op 2606 completed, throughput=5MB/sec 2024-04-07T16:47:06.996 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-ORE4FBgtBOqABk7 off=12717271 len=1791582 2024-04-07T16:47:07.000 INFO:tasks.workunit.client.0.smithi100.stdout:op 2607 completed, throughput=5MB/sec 2024-04-07T16:47:07.000 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Aoz0N_2jpEuJfgQ off=2944276 len=1277977 2024-04-07T16:47:07.002 INFO:tasks.workunit.client.0.smithi100.stdout:op 2608 completed, throughput=5MB/sec 2024-04-07T16:47:08.003 INFO:tasks.workunit.client.0.smithi100.stdout: 495: throughput=4.99MB/sec pending data=0 2024-04-07T16:47:08.003 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-n3C0JbrKLlg3oId off=15695445 len=1140733 2024-04-07T16:47:08.012 INFO:tasks.workunit.client.0.smithi100.stdout:op 2609 completed, throughput=4.99MB/sec 2024-04-07T16:47:08.013 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-J_mInNSaDJJXu_Q off=6269115 len=1439285 2024-04-07T16:47:08.017 INFO:tasks.workunit.client.0.smithi100.stdout:op 2610 completed, throughput=5MB/sec 2024-04-07T16:47:08.017 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-W2f82e585HWmZE4 off=21337925 len=1800190 2024-04-07T16:47:08.031 INFO:tasks.workunit.client.0.smithi100.stdout:op 2611 completed, throughput=5MB/sec 2024-04-07T16:47:08.031 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-rBr_AXOsFALe8Hh off=3014584 len=1300650 2024-04-07T16:47:08.034 INFO:tasks.workunit.client.0.smithi100.stdout:op 2612 completed, throughput=5MB/sec 2024-04-07T16:47:08.694 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:47:08 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:47:08] "GET /metrics HTTP/1.1" 200 34072 "" "Prometheus/2.43.0" 2024-04-07T16:47:08.695 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:47:08 smithi100 bash[74942]: cluster 2024-04-07T16:47:07.338493+0000 mgr.y (mgr.25175) 953 : cluster [DBG] pgmap v1595: 105 pgs: 105 active+clean; 126 GiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 3.0 MiB/s rd, 1.6 MiB/s wr, 4 op/s 2024-04-07T16:47:08.695 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:47:08 smithi100 bash[76202]: cluster 2024-04-07T16:47:07.338493+0000 mgr.y (mgr.25175) 953 : cluster [DBG] pgmap v1595: 105 pgs: 105 active+clean; 126 GiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 3.0 MiB/s rd, 1.6 MiB/s wr, 4 op/s 2024-04-07T16:47:08.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:47:08 smithi144 bash[66654]: cluster 2024-04-07T16:47:07.338493+0000 mgr.y (mgr.25175) 953 : cluster [DBG] pgmap v1595: 105 pgs: 105 active+clean; 126 GiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 3.0 MiB/s rd, 1.6 MiB/s wr, 4 op/s 2024-04-07T16:47:09.034 INFO:tasks.workunit.client.0.smithi100.stdout: 496: throughput=4.99MB/sec pending data=0 2024-04-07T16:47:09.034 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-g5tFYNB0FMt9QtH off=19918302 len=526207 2024-04-07T16:47:09.041 INFO:tasks.workunit.client.0.smithi100.stdout:op 2613 completed, throughput=4.99MB/sec 2024-04-07T16:47:09.041 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-cO1TvZxbDsnBx90 off=2899307 len=1885836 2024-04-07T16:47:09.047 INFO:tasks.workunit.client.0.smithi100.stdout:op 2614 completed, throughput=5MB/sec 2024-04-07T16:47:09.047 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-5ewa3RVlNKQLtx1 off=10171299 len=899376 2024-04-07T16:47:09.055 INFO:tasks.workunit.client.0.smithi100.stdout:op 2615 completed, throughput=5MB/sec 2024-04-07T16:47:09.055 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-SlmHtSt3WWK-Nnz off=3991433 len=39148 2024-04-07T16:47:09.056 INFO:tasks.workunit.client.0.smithi100.stdout:op 2616 completed, throughput=5MB/sec 2024-04-07T16:47:09.057 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-3L9q94TshSpcilb off=14398989 len=1577099 2024-04-07T16:47:09.060 INFO:tasks.workunit.client.0.smithi100.stdout:op 2617 completed, throughput=5MB/sec 2024-04-07T16:47:10.061 INFO:tasks.workunit.client.0.smithi100.stdout: 497: throughput=4.99MB/sec pending data=0 2024-04-07T16:47:10.061 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-_825c53Np9ZSucr off=6591261 len=1325764 2024-04-07T16:47:10.072 INFO:tasks.workunit.client.0.smithi100.stdout:op 2618 completed, throughput=4.99MB/sec 2024-04-07T16:47:10.072 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-KWbhzReoc73OcCH off=18238042 len=439968 2024-04-07T16:47:10.077 INFO:tasks.workunit.client.0.smithi100.stdout:op 2619 completed, throughput=4.99MB/sec 2024-04-07T16:47:10.077 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-QMq-d2VN2VZIU1P off=9092559 len=1721297 2024-04-07T16:47:10.094 INFO:tasks.workunit.client.0.smithi100.stdout:op 2620 completed, throughput=5MB/sec 2024-04-07T16:47:10.094 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-hg2pLgwlqFc8UQP off=384395 len=1971417 2024-04-07T16:47:10.098 INFO:tasks.workunit.client.0.smithi100.stdout:op 2621 completed, throughput=5MB/sec 2024-04-07T16:47:10.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:47:10 smithi144 bash[66654]: cluster 2024-04-07T16:47:09.339345+0000 mgr.y (mgr.25175) 954 : cluster [DBG] pgmap v1596: 105 pgs: 105 active+clean; 126 GiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 3.0 MiB/s rd, 2.1 MiB/s wr, 4 op/s 2024-04-07T16:47:10.944 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:47:10 smithi100 bash[74942]: cluster 2024-04-07T16:47:09.339345+0000 mgr.y (mgr.25175) 954 : cluster [DBG] pgmap v1596: 105 pgs: 105 active+clean; 126 GiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 3.0 MiB/s rd, 2.1 MiB/s wr, 4 op/s 2024-04-07T16:47:10.944 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:47:10 smithi100 bash[76202]: cluster 2024-04-07T16:47:09.339345+0000 mgr.y (mgr.25175) 954 : cluster [DBG] pgmap v1596: 105 pgs: 105 active+clean; 126 GiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 3.0 MiB/s rd, 2.1 MiB/s wr, 4 op/s 2024-04-07T16:47:11.099 INFO:tasks.workunit.client.0.smithi100.stdout: 498: throughput=4.99MB/sec pending data=0 2024-04-07T16:47:11.099 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-oZKnr7YdCkSLur6 off=6225679 len=2038275 2024-04-07T16:47:11.113 INFO:tasks.workunit.client.0.smithi100.stdout:op 2622 completed, throughput=5MB/sec 2024-04-07T16:47:11.113 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-LWag0cyDNe4Oc5g off=7139459 len=513201 2024-04-07T16:47:11.115 INFO:tasks.workunit.client.0.smithi100.stdout:op 2623 completed, throughput=5MB/sec 2024-04-07T16:47:11.115 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-lYzP6OY7SnEyZjg off=388471 len=639539 2024-04-07T16:47:11.117 INFO:tasks.workunit.client.0.smithi100.stdout:op 2624 completed, throughput=5MB/sec 2024-04-07T16:47:11.118 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-tXI31kvSa70MggK off=6053834 len=1197555 2024-04-07T16:47:11.129 INFO:tasks.workunit.client.0.smithi100.stdout:op 2625 completed, throughput=5MB/sec 2024-04-07T16:47:11.130 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-ibvVB3T2tcZGoZf off=10948759 len=640085 2024-04-07T16:47:11.131 INFO:tasks.workunit.client.0.smithi100.stdout:op 2626 completed, throughput=5MB/sec 2024-04-07T16:47:12.131 INFO:tasks.workunit.client.0.smithi100.stdout: 499: throughput=4.99MB/sec pending data=0 2024-04-07T16:47:12.131 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-YNmBY21Vux4GX3Q off=1166795 len=621958 2024-04-07T16:47:12.133 INFO:tasks.workunit.client.0.smithi100.stdout:op 2627 completed, throughput=4.99MB/sec 2024-04-07T16:47:12.133 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-0md2VV_UtF7AUhj off=8348519 len=1986081 2024-04-07T16:47:12.147 INFO:tasks.workunit.client.0.smithi100.stdout:op 2628 completed, throughput=5MB/sec 2024-04-07T16:47:12.147 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj--JFUUnIjDUXkY40 off=733695 len=579629 2024-04-07T16:47:12.150 INFO:tasks.workunit.client.0.smithi100.stdout:op 2629 completed, throughput=5MB/sec 2024-04-07T16:47:12.150 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-AdkJq35cS6PZXoB off=969354 len=1293247 2024-04-07T16:47:12.154 INFO:tasks.workunit.client.0.smithi100.stdout:op 2630 completed, throughput=5MB/sec 2024-04-07T16:47:12.154 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-4TqbzhNGdvdX_rw off=3521802 len=1689510 2024-04-07T16:47:12.167 INFO:tasks.workunit.client.0.smithi100.stdout:op 2631 completed, throughput=5MB/sec 2024-04-07T16:47:12.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:47:12 smithi144 bash[66654]: cluster 2024-04-07T16:47:11.340248+0000 mgr.y (mgr.25175) 955 : cluster [DBG] pgmap v1597: 105 pgs: 105 active+clean; 126 GiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 3.3 MiB/s rd, 2.1 MiB/s wr, 4 op/s 2024-04-07T16:47:12.610 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:47:12 smithi144 bash[95233]: ts=2024-04-07T16:47:12.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:47:12.944 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:47:12 smithi100 bash[74942]: cluster 2024-04-07T16:47:11.340248+0000 mgr.y (mgr.25175) 955 : cluster [DBG] pgmap v1597: 105 pgs: 105 active+clean; 126 GiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 3.3 MiB/s rd, 2.1 MiB/s wr, 4 op/s 2024-04-07T16:47:12.945 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:47:12 smithi100 bash[76202]: cluster 2024-04-07T16:47:11.340248+0000 mgr.y (mgr.25175) 955 : cluster [DBG] pgmap v1597: 105 pgs: 105 active+clean; 126 GiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 3.3 MiB/s rd, 2.1 MiB/s wr, 4 op/s 2024-04-07T16:47:13.168 INFO:tasks.workunit.client.0.smithi100.stdout: 500: throughput=4.99MB/sec pending data=0 2024-04-07T16:47:13.168 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Im_LMPBAxY4L7UZ off=625164 len=201300 2024-04-07T16:47:13.173 INFO:tasks.workunit.client.0.smithi100.stdout:op 2632 completed, throughput=4.99MB/sec 2024-04-07T16:47:13.173 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-3tr8cd-NfXSPpbc off=5070505 len=863272 2024-04-07T16:47:13.176 INFO:tasks.workunit.client.0.smithi100.stdout:op 2633 completed, throughput=4.99MB/sec 2024-04-07T16:47:13.176 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-aeot9X3OnTa69dD off=12886960 len=1360198 2024-04-07T16:47:13.180 INFO:tasks.workunit.client.0.smithi100.stdout:op 2634 completed, throughput=5MB/sec 2024-04-07T16:47:13.180 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-qr-0qbQc_IbuKSg off=16383321 len=363109 2024-04-07T16:47:13.186 INFO:tasks.workunit.client.0.smithi100.stdout:op 2635 completed, throughput=5MB/sec 2024-04-07T16:47:13.186 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-bLIdFRalv2MH-oO off=4707075 len=1552808 2024-04-07T16:47:13.197 INFO:tasks.workunit.client.0.smithi100.stdout:op 2636 completed, throughput=5MB/sec 2024-04-07T16:47:14.197 INFO:tasks.workunit.client.0.smithi100.stdout: 501: throughput=4.99MB/sec pending data=0 2024-04-07T16:47:14.197 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-F_8fQPQM7k1JBX_ off=19246711 len=681304 2024-04-07T16:47:14.205 INFO:tasks.workunit.client.0.smithi100.stdout:op 2637 completed, throughput=4.99MB/sec 2024-04-07T16:47:14.205 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-w6n2C4JQS4Nn40t off=4713322 len=29178 2024-04-07T16:47:14.209 INFO:tasks.workunit.client.0.smithi100.stdout:op 2638 completed, throughput=4.99MB/sec 2024-04-07T16:47:14.210 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-2_1DB8SqKxajFBs off=14975999 len=655107 2024-04-07T16:47:14.217 INFO:tasks.workunit.client.0.smithi100.stdout:op 2639 completed, throughput=4.99MB/sec 2024-04-07T16:47:14.217 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Qmegr3RA6gYo2cH off=6412236 len=1381120 2024-04-07T16:47:14.220 INFO:tasks.workunit.client.0.smithi100.stdout:op 2640 completed, throughput=5MB/sec 2024-04-07T16:47:14.220 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-IUAI0kRL-o2l3n8 off=7182462 len=1337889 2024-04-07T16:47:14.231 INFO:tasks.workunit.client.0.smithi100.stdout:op 2641 completed, throughput=5MB/sec 2024-04-07T16:47:14.231 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-uP0Oqz7nYmb-mPg off=5386866 len=393000 2024-04-07T16:47:14.237 INFO:tasks.workunit.client.0.smithi100.stdout:op 2642 completed, throughput=5MB/sec 2024-04-07T16:47:14.237 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-ETqYxWZDU9EOVPz off=15819453 len=193771 2024-04-07T16:47:14.238 INFO:tasks.workunit.client.0.smithi100.stdout:op 2643 completed, throughput=5MB/sec 2024-04-07T16:47:14.239 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-m-jBamS7uEGRXrm off=17381410 len=2034394 2024-04-07T16:47:14.244 INFO:tasks.workunit.client.0.smithi100.stdout:op 2644 completed, throughput=5MB/sec 2024-04-07T16:47:14.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:47:14 smithi144 bash[66654]: cluster 2024-04-07T16:47:13.341474+0000 mgr.y (mgr.25175) 956 : cluster [DBG] pgmap v1598: 105 pgs: 105 active+clean; 126 GiB data, 6.7 GiB used, 709 GiB / 715 GiB avail; 2.8 MiB/s rd, 2.2 MiB/s wr, 4 op/s 2024-04-07T16:47:14.944 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:47:14 smithi100 bash[74942]: cluster 2024-04-07T16:47:13.341474+0000 mgr.y (mgr.25175) 956 : cluster [DBG] pgmap v1598: 105 pgs: 105 active+clean; 126 GiB data, 6.7 GiB used, 709 GiB / 715 GiB avail; 2.8 MiB/s rd, 2.2 MiB/s wr, 4 op/s 2024-04-07T16:47:14.945 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:47:14 smithi100 bash[76202]: cluster 2024-04-07T16:47:13.341474+0000 mgr.y (mgr.25175) 956 : cluster [DBG] pgmap v1598: 105 pgs: 105 active+clean; 126 GiB data, 6.7 GiB used, 709 GiB / 715 GiB avail; 2.8 MiB/s rd, 2.2 MiB/s wr, 4 op/s 2024-04-07T16:47:15.244 INFO:tasks.workunit.client.0.smithi100.stdout: 502: throughput=4.99MB/sec pending data=0 2024-04-07T16:47:15.244 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-nhMDTSNsN8gTQe6 off=2143698 len=671221 2024-04-07T16:47:15.246 INFO:tasks.workunit.client.0.smithi100.stdout:op 2645 completed, throughput=4.99MB/sec 2024-04-07T16:47:15.246 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-LJVkdSvnWBkjNJP off=20131699 len=1094822 2024-04-07T16:47:15.255 INFO:tasks.workunit.client.0.smithi100.stdout:op 2646 completed, throughput=5MB/sec 2024-04-07T16:47:15.255 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-8XpEckpl4BVib_F off=2455827 len=1945619 2024-04-07T16:47:15.269 INFO:tasks.workunit.client.0.smithi100.stdout:op 2647 completed, throughput=5MB/sec 2024-04-07T16:47:15.269 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-MaYwW2jrEyyzJQ8 off=15162868 len=1731665 2024-04-07T16:47:15.285 INFO:tasks.workunit.client.0.smithi100.stdout:op 2648 completed, throughput=5MB/sec 2024-04-07T16:47:16.285 INFO:tasks.workunit.client.0.smithi100.stdout: 503: throughput=4.99MB/sec pending data=0 2024-04-07T16:47:16.285 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-z9rCVpGihdgrSlw off=505226 len=411601 2024-04-07T16:47:16.287 INFO:tasks.workunit.client.0.smithi100.stdout:op 2649 completed, throughput=4.99MB/sec 2024-04-07T16:47:16.287 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj--cHlabAvLtFe3LL off=4437114 len=172772 2024-04-07T16:47:16.288 INFO:tasks.workunit.client.0.smithi100.stdout:op 2650 completed, throughput=4.99MB/sec 2024-04-07T16:47:16.288 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-0n7g_mtzafMMJT9 off=15321279 len=687415 2024-04-07T16:47:16.295 INFO:tasks.workunit.client.0.smithi100.stdout:op 2651 completed, throughput=4.99MB/sec 2024-04-07T16:47:16.295 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-rrPcErwdtZdLNVT off=816797 len=1836171 2024-04-07T16:47:16.300 INFO:tasks.workunit.client.0.smithi100.stdout:op 2652 completed, throughput=5MB/sec 2024-04-07T16:47:16.300 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-9pju3O55Br6zFLj off=30227 len=267051 2024-04-07T16:47:16.301 INFO:tasks.workunit.client.0.smithi100.stdout:op 2653 completed, throughput=5MB/sec 2024-04-07T16:47:16.302 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-_H9PA_WfRphE96O off=4726409 len=448257 2024-04-07T16:47:16.303 INFO:tasks.workunit.client.0.smithi100.stdout:op 2654 completed, throughput=5MB/sec 2024-04-07T16:47:16.303 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Xwj5gNMmQK36lEf off=776825 len=1410169 2024-04-07T16:47:16.308 INFO:tasks.workunit.client.0.smithi100.stdout:op 2655 completed, throughput=5MB/sec 2024-04-07T16:47:16.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:47:16 smithi144 bash[66654]: cluster 2024-04-07T16:47:15.342469+0000 mgr.y (mgr.25175) 957 : cluster [DBG] pgmap v1599: 105 pgs: 105 active+clean; 126 GiB data, 6.7 GiB used, 709 GiB / 715 GiB avail; 2.4 MiB/s rd, 2.0 MiB/s wr, 3 op/s 2024-04-07T16:47:16.945 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:47:16 smithi100 bash[74942]: cluster 2024-04-07T16:47:15.342469+0000 mgr.y (mgr.25175) 957 : cluster [DBG] pgmap v1599: 105 pgs: 105 active+clean; 126 GiB data, 6.7 GiB used, 709 GiB / 715 GiB avail; 2.4 MiB/s rd, 2.0 MiB/s wr, 3 op/s 2024-04-07T16:47:16.945 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:47:16 smithi100 bash[76202]: cluster 2024-04-07T16:47:15.342469+0000 mgr.y (mgr.25175) 957 : cluster [DBG] pgmap v1599: 105 pgs: 105 active+clean; 126 GiB data, 6.7 GiB used, 709 GiB / 715 GiB avail; 2.4 MiB/s rd, 2.0 MiB/s wr, 3 op/s 2024-04-07T16:47:17.309 INFO:tasks.workunit.client.0.smithi100.stdout: 504: throughput=4.99MB/sec pending data=0 2024-04-07T16:47:17.309 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-_lpDh5SfvZKpJe3 off=11740074 len=1109762 2024-04-07T16:47:17.318 INFO:tasks.workunit.client.0.smithi100.stdout:op 2656 completed, throughput=4.99MB/sec 2024-04-07T16:47:17.318 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-oG-ceZDp49zpixS off=9703496 len=1726205 2024-04-07T16:47:17.332 INFO:tasks.workunit.client.0.smithi100.stdout:op 2657 completed, throughput=5MB/sec 2024-04-07T16:47:17.332 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-xVEGzEcgznSKN-M off=10979191 len=928481 2024-04-07T16:47:17.334 INFO:tasks.workunit.client.0.smithi100.stdout:op 2658 completed, throughput=5MB/sec 2024-04-07T16:47:17.334 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-VjfBnjV9Viq0gTP off=1454197 len=2067555 2024-04-07T16:47:17.339 INFO:tasks.workunit.client.0.smithi100.stdout:op 2659 completed, throughput=5MB/sec 2024-04-07T16:47:17.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:47:17 smithi144 bash[66654]: audit 2024-04-07T16:47:16.769992+0000 mon.a (mon.0) 2118 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:47:17.944 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:47:17 smithi100 bash[74942]: audit 2024-04-07T16:47:16.769992+0000 mon.a (mon.0) 2118 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:47:17.945 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:47:17 smithi100 bash[76202]: audit 2024-04-07T16:47:16.769992+0000 mon.a (mon.0) 2118 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:47:18.339 INFO:tasks.workunit.client.0.smithi100.stdout: 505: throughput=4.99MB/sec pending data=0 2024-04-07T16:47:18.339 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-i_p26ey3-wX_Dcj off=8626876 len=598025 2024-04-07T16:47:18.346 INFO:tasks.workunit.client.0.smithi100.stdout:op 2660 completed, throughput=4.99MB/sec 2024-04-07T16:47:18.346 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-OdcYAq9WEeeum6z off=8573302 len=470489 2024-04-07T16:47:18.352 INFO:tasks.workunit.client.0.smithi100.stdout:op 2661 completed, throughput=5MB/sec 2024-04-07T16:47:18.352 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-ZlgPrd4f6c90M18 off=1469593 len=1495510 2024-04-07T16:47:18.363 INFO:tasks.workunit.client.0.smithi100.stdout:op 2662 completed, throughput=5MB/sec 2024-04-07T16:47:18.363 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-nlFwfzyTO9dJdOU off=5175118 len=486466 2024-04-07T16:47:18.364 INFO:tasks.workunit.client.0.smithi100.stdout:op 2663 completed, throughput=5MB/sec 2024-04-07T16:47:18.365 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-AEA0r44zCutQ32O off=7779845 len=880527 2024-04-07T16:47:18.374 INFO:tasks.workunit.client.0.smithi100.stdout:op 2664 completed, throughput=5MB/sec 2024-04-07T16:47:18.694 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:47:18 smithi100 bash[74942]: cluster 2024-04-07T16:47:17.343707+0000 mgr.y (mgr.25175) 958 : cluster [DBG] pgmap v1600: 105 pgs: 105 active+clean; 126 GiB data, 6.7 GiB used, 709 GiB / 715 GiB avail; 2.4 MiB/s rd, 2.7 MiB/s wr, 4 op/s 2024-04-07T16:47:18.695 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:47:18 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:47:18] "GET /metrics HTTP/1.1" 200 34072 "" "Prometheus/2.43.0" 2024-04-07T16:47:18.695 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:47:18 smithi100 bash[76202]: cluster 2024-04-07T16:47:17.343707+0000 mgr.y (mgr.25175) 958 : cluster [DBG] pgmap v1600: 105 pgs: 105 active+clean; 126 GiB data, 6.7 GiB used, 709 GiB / 715 GiB avail; 2.4 MiB/s rd, 2.7 MiB/s wr, 4 op/s 2024-04-07T16:47:18.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:47:18 smithi144 bash[66654]: cluster 2024-04-07T16:47:17.343707+0000 mgr.y (mgr.25175) 958 : cluster [DBG] pgmap v1600: 105 pgs: 105 active+clean; 126 GiB data, 6.7 GiB used, 709 GiB / 715 GiB avail; 2.4 MiB/s rd, 2.7 MiB/s wr, 4 op/s 2024-04-07T16:47:19.374 INFO:tasks.workunit.client.0.smithi100.stdout: 506: throughput=4.99MB/sec pending data=0 2024-04-07T16:47:19.374 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-MjNOcRETZVjuRgL off=15159832 len=681180 2024-04-07T16:47:19.377 INFO:tasks.workunit.client.0.smithi100.stdout:op 2665 completed, throughput=4.99MB/sec 2024-04-07T16:47:19.377 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-B3TgiZSnrD-M1KG off=3939316 len=274934 2024-04-07T16:47:19.383 INFO:tasks.workunit.client.0.smithi100.stdout:op 2666 completed, throughput=4.99MB/sec 2024-04-07T16:47:19.383 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-vSO4iy2ONqI7Qbk off=81343 len=462954 2024-04-07T16:47:19.389 INFO:tasks.workunit.client.0.smithi100.stdout:op 2667 completed, throughput=4.99MB/sec 2024-04-07T16:47:19.389 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj--mI1U4MdsLdYD9b off=5834062 len=1644705 2024-04-07T16:47:19.401 INFO:tasks.workunit.client.0.smithi100.stdout:op 2668 completed, throughput=5MB/sec 2024-04-07T16:47:19.401 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-UdQIlYIDvV782NM off=289432 len=1942983 2024-04-07T16:47:19.415 INFO:tasks.workunit.client.0.smithi100.stdout:op 2669 completed, throughput=5MB/sec 2024-04-07T16:47:19.415 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-elM_FgDN688X2r0 off=1345051 len=160180 2024-04-07T16:47:19.421 INFO:tasks.workunit.client.0.smithi100.stdout:op 2670 completed, throughput=5MB/sec 2024-04-07T16:47:19.421 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-lTi_uBaeVk3V3ZQ off=5399899 len=29269 2024-04-07T16:47:19.424 INFO:tasks.workunit.client.0.smithi100.stdout:op 2671 completed, throughput=5MB/sec 2024-04-07T16:47:19.424 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-zkSmUV0OK97CKPm off=3602517 len=1908623 2024-04-07T16:47:19.430 INFO:tasks.workunit.client.0.smithi100.stdout:op 2672 completed, throughput=5MB/sec 2024-04-07T16:47:20.431 INFO:tasks.workunit.client.0.smithi100.stdout: 507: throughput=4.99MB/sec pending data=0 2024-04-07T16:47:20.431 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-H6mNPZ_2QpX-JWo off=8958288 len=1006159 2024-04-07T16:47:20.440 INFO:tasks.workunit.client.0.smithi100.stdout:op 2673 completed, throughput=5MB/sec 2024-04-07T16:47:20.440 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-O3SR6I7kVkplBWm off=2116251 len=1051297 2024-04-07T16:47:20.443 INFO:tasks.workunit.client.0.smithi100.stdout:op 2674 completed, throughput=5MB/sec 2024-04-07T16:47:20.443 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-ILbM8dnOyI1-Uso off=1381670 len=1647278 2024-04-07T16:47:20.447 INFO:tasks.workunit.client.0.smithi100.stdout:op 2675 completed, throughput=5MB/sec 2024-04-07T16:47:20.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:47:20 smithi144 bash[66654]: cluster 2024-04-07T16:47:19.344555+0000 mgr.y (mgr.25175) 959 : cluster [DBG] pgmap v1601: 105 pgs: 105 active+clean; 126 GiB data, 6.7 GiB used, 709 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.6 MiB/s wr, 4 op/s 2024-04-07T16:47:20.944 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:47:20 smithi100 bash[74942]: cluster 2024-04-07T16:47:19.344555+0000 mgr.y (mgr.25175) 959 : cluster [DBG] pgmap v1601: 105 pgs: 105 active+clean; 126 GiB data, 6.7 GiB used, 709 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.6 MiB/s wr, 4 op/s 2024-04-07T16:47:20.945 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:47:20 smithi100 bash[76202]: cluster 2024-04-07T16:47:19.344555+0000 mgr.y (mgr.25175) 959 : cluster [DBG] pgmap v1601: 105 pgs: 105 active+clean; 126 GiB data, 6.7 GiB used, 709 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.6 MiB/s wr, 4 op/s 2024-04-07T16:47:21.448 INFO:tasks.workunit.client.0.smithi100.stdout: 508: throughput=4.99MB/sec pending data=0 2024-04-07T16:47:21.448 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-S7jWd1KbS5CRMf_ off=668702 len=1313405 2024-04-07T16:47:21.458 INFO:tasks.workunit.client.0.smithi100.stdout:op 2676 completed, throughput=4.99MB/sec 2024-04-07T16:47:21.458 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-YWZm7H0isTGdIB7 off=818059 len=563061 2024-04-07T16:47:21.460 INFO:tasks.workunit.client.0.smithi100.stdout:op 2677 completed, throughput=4.99MB/sec 2024-04-07T16:47:21.460 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-W0WkmEchuClo-DL off=5222088 len=761020 2024-04-07T16:47:21.462 INFO:tasks.workunit.client.0.smithi100.stdout:op 2678 completed, throughput=5MB/sec 2024-04-07T16:47:21.462 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-41R3Nq4-ujVSflZ off=17888936 len=193520 2024-04-07T16:47:21.464 INFO:tasks.workunit.client.0.smithi100.stdout:op 2679 completed, throughput=5MB/sec 2024-04-07T16:47:21.464 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-ra7k9AsK-vSETcM off=8827656 len=1137711 2024-04-07T16:47:21.473 INFO:tasks.workunit.client.0.smithi100.stdout:op 2680 completed, throughput=5MB/sec 2024-04-07T16:47:21.473 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-P0Qq6qLvYWDmQgP off=11495740 len=793302 2024-04-07T16:47:21.476 INFO:tasks.workunit.client.0.smithi100.stdout:op 2681 completed, throughput=5MB/sec 2024-04-07T16:47:21.476 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-3ewIX5M4NUyZ35V off=3608109 len=313865 2024-04-07T16:47:21.478 INFO:tasks.workunit.client.0.smithi100.stdout:op 2682 completed, throughput=5MB/sec 2024-04-07T16:47:21.478 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-ojI_qgg5ewa5TSh off=10263610 len=85306 2024-04-07T16:47:21.479 INFO:tasks.workunit.client.0.smithi100.stdout:op 2683 completed, throughput=5MB/sec 2024-04-07T16:47:21.479 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-6wZC-Va7zl4AE-q off=1210692 len=308446 2024-04-07T16:47:21.481 INFO:tasks.workunit.client.0.smithi100.stdout:op 2684 completed, throughput=5MB/sec 2024-04-07T16:47:22.481 INFO:tasks.workunit.client.0.smithi100.stdout: 509: throughput=4.99MB/sec pending data=0 2024-04-07T16:47:22.481 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-c0JpUxM5pxD6N8p off=7937587 len=1132317 2024-04-07T16:47:22.491 INFO:tasks.workunit.client.0.smithi100.stdout:op 2685 completed, throughput=4.99MB/sec 2024-04-07T16:47:22.491 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-yLdzqBHIQkk3N_g off=9221561 len=1275959 2024-04-07T16:47:22.503 INFO:tasks.workunit.client.0.smithi100.stdout:op 2686 completed, throughput=4.99MB/sec 2024-04-07T16:47:22.503 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-r78aJ9ywAdjK03c off=5765302 len=1688963 2024-04-07T16:47:22.507 INFO:tasks.workunit.client.0.smithi100.stdout:op 2687 completed, throughput=5MB/sec 2024-04-07T16:47:22.507 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-HdZezRc2a9hi2gz off=5388469 len=1731892 2024-04-07T16:47:22.513 INFO:tasks.workunit.client.0.smithi100.stdout:op 2688 completed, throughput=5MB/sec 2024-04-07T16:47:22.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:47:22 smithi144 bash[66654]: cluster 2024-04-07T16:47:21.345487+0000 mgr.y (mgr.25175) 960 : cluster [DBG] pgmap v1602: 105 pgs: 105 active+clean; 126 GiB data, 6.7 GiB used, 709 GiB / 715 GiB avail; 2.4 MiB/s rd, 2.5 MiB/s wr, 4 op/s 2024-04-07T16:47:22.610 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:47:22 smithi144 bash[95233]: ts=2024-04-07T16:47:22.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:47:22.945 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:47:22 smithi100 bash[74942]: cluster 2024-04-07T16:47:21.345487+0000 mgr.y (mgr.25175) 960 : cluster [DBG] pgmap v1602: 105 pgs: 105 active+clean; 126 GiB data, 6.7 GiB used, 709 GiB / 715 GiB avail; 2.4 MiB/s rd, 2.5 MiB/s wr, 4 op/s 2024-04-07T16:47:22.945 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:47:22 smithi100 bash[76202]: cluster 2024-04-07T16:47:21.345487+0000 mgr.y (mgr.25175) 960 : cluster [DBG] pgmap v1602: 105 pgs: 105 active+clean; 126 GiB data, 6.7 GiB used, 709 GiB / 715 GiB avail; 2.4 MiB/s rd, 2.5 MiB/s wr, 4 op/s 2024-04-07T16:47:23.513 INFO:tasks.workunit.client.0.smithi100.stdout: 510: throughput=4.99MB/sec pending data=0 2024-04-07T16:47:23.513 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-OVL15tAWAdGMpvb off=797043 len=892617 2024-04-07T16:47:23.517 INFO:tasks.workunit.client.0.smithi100.stdout:op 2689 completed, throughput=4.99MB/sec 2024-04-07T16:47:23.517 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-j91A5gEWcUtHLmw off=2148798 len=1282516 2024-04-07T16:47:23.521 INFO:tasks.workunit.client.0.smithi100.stdout:op 2690 completed, throughput=5MB/sec 2024-04-07T16:47:23.521 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-l17DrZiWOPQLoYK off=3955462 len=1850385 2024-04-07T16:47:23.526 INFO:tasks.workunit.client.0.smithi100.stdout:op 2691 completed, throughput=5MB/sec 2024-04-07T16:47:23.526 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-2bmlNGMUKASdlXe off=829398 len=100501 2024-04-07T16:47:23.531 INFO:tasks.workunit.client.0.smithi100.stdout:op 2692 completed, throughput=5MB/sec 2024-04-07T16:47:23.531 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-3x6UPByTGRf5MKg off=827723 len=1131649 2024-04-07T16:47:23.534 INFO:tasks.workunit.client.0.smithi100.stdout:op 2693 completed, throughput=5MB/sec 2024-04-07T16:47:24.535 INFO:tasks.workunit.client.0.smithi100.stdout: 511: throughput=4.99MB/sec pending data=0 2024-04-07T16:47:24.535 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-TJBFfa--8UYN3Lz off=808471 len=552095 2024-04-07T16:47:24.537 INFO:tasks.workunit.client.0.smithi100.stdout:op 2694 completed, throughput=4.99MB/sec 2024-04-07T16:47:24.537 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-6mydHmFrCatNwlh off=757969 len=1920145 2024-04-07T16:47:24.541 INFO:tasks.workunit.client.0.smithi100.stdout:op 2695 completed, throughput=5MB/sec 2024-04-07T16:47:24.541 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-62Z7N94Tb_iozaa off=2092267 len=1340575 2024-04-07T16:47:24.545 INFO:tasks.workunit.client.0.smithi100.stdout:op 2696 completed, throughput=5MB/sec 2024-04-07T16:47:24.545 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-7MqtHAhvbl9eVjQ off=899202 len=1638186 2024-04-07T16:47:24.558 INFO:tasks.workunit.client.0.smithi100.stdout:op 2697 completed, throughput=5MB/sec 2024-04-07T16:47:24.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:47:24 smithi144 bash[66654]: cluster 2024-04-07T16:47:23.346598+0000 mgr.y (mgr.25175) 961 : cluster [DBG] pgmap v1603: 105 pgs: 105 active+clean; 126 GiB data, 6.7 GiB used, 708 GiB / 715 GiB avail; 2.5 MiB/s rd, 3.0 MiB/s wr, 5 op/s 2024-04-07T16:47:24.945 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:47:24 smithi100 bash[74942]: cluster 2024-04-07T16:47:23.346598+0000 mgr.y (mgr.25175) 961 : cluster [DBG] pgmap v1603: 105 pgs: 105 active+clean; 126 GiB data, 6.7 GiB used, 708 GiB / 715 GiB avail; 2.5 MiB/s rd, 3.0 MiB/s wr, 5 op/s 2024-04-07T16:47:24.945 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:47:24 smithi100 bash[76202]: cluster 2024-04-07T16:47:23.346598+0000 mgr.y (mgr.25175) 961 : cluster [DBG] pgmap v1603: 105 pgs: 105 active+clean; 126 GiB data, 6.7 GiB used, 708 GiB / 715 GiB avail; 2.5 MiB/s rd, 3.0 MiB/s wr, 5 op/s 2024-04-07T16:47:25.558 INFO:tasks.workunit.client.0.smithi100.stdout: 512: throughput=4.99MB/sec pending data=0 2024-04-07T16:47:25.558 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-BypAuTKF6pno13T off=2776688 len=869779 2024-04-07T16:47:25.561 INFO:tasks.workunit.client.0.smithi100.stdout:op 2698 completed, throughput=4.99MB/sec 2024-04-07T16:47:25.562 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-61fFZkPbTpwjbzt off=18454536 len=86593 2024-04-07T16:47:25.567 INFO:tasks.workunit.client.0.smithi100.stdout:op 2699 completed, throughput=4.99MB/sec 2024-04-07T16:47:25.567 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-aPQGhQvqJCOyeXA off=15916779 len=1451281 2024-04-07T16:47:25.582 INFO:tasks.workunit.client.0.smithi100.stdout:op 2700 completed, throughput=5MB/sec 2024-04-07T16:47:25.582 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-x9dTlA7LJpOxk02 off=17352620 len=1727446 2024-04-07T16:47:25.596 INFO:tasks.workunit.client.0.smithi100.stdout:op 2701 completed, throughput=5MB/sec 2024-04-07T16:47:25.596 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-xHHhcy5vqNjca_A off=1081139 len=1827985 2024-04-07T16:47:25.601 INFO:tasks.workunit.client.0.smithi100.stdout:op 2702 completed, throughput=5MB/sec 2024-04-07T16:47:26.601 INFO:tasks.workunit.client.0.smithi100.stdout: 513: throughput=4.99MB/sec pending data=0 2024-04-07T16:47:26.601 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-i6f80pBTi_r-Aex off=15480790 len=1464401 2024-04-07T16:47:26.614 INFO:tasks.workunit.client.0.smithi100.stdout:op 2703 completed, throughput=4.99MB/sec 2024-04-07T16:47:26.614 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-sUHreuPnVr8ILmg off=10990665 len=2075526 2024-04-07T16:47:26.633 INFO:tasks.workunit.client.0.smithi100.stdout:op 2704 completed, throughput=5MB/sec 2024-04-07T16:47:26.633 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-io7NFUqQebfQFdL off=7465381 len=338462 2024-04-07T16:47:26.638 INFO:tasks.workunit.client.0.smithi100.stdout:op 2705 completed, throughput=5MB/sec 2024-04-07T16:47:26.638 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-MVdVdBkePozIxXa off=244090 len=1929591 2024-04-07T16:47:26.643 INFO:tasks.workunit.client.0.smithi100.stdout:op 2706 completed, throughput=5MB/sec 2024-04-07T16:47:26.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:47:26 smithi144 bash[66654]: cluster 2024-04-07T16:47:25.347583+0000 mgr.y (mgr.25175) 962 : cluster [DBG] pgmap v1604: 105 pgs: 105 active+clean; 126 GiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 2.8 MiB/s rd, 2.6 MiB/s wr, 5 op/s 2024-04-07T16:47:26.945 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:47:26 smithi100 bash[74942]: cluster 2024-04-07T16:47:25.347583+0000 mgr.y (mgr.25175) 962 : cluster [DBG] pgmap v1604: 105 pgs: 105 active+clean; 126 GiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 2.8 MiB/s rd, 2.6 MiB/s wr, 5 op/s 2024-04-07T16:47:26.945 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:47:26 smithi100 bash[76202]: cluster 2024-04-07T16:47:25.347583+0000 mgr.y (mgr.25175) 962 : cluster [DBG] pgmap v1604: 105 pgs: 105 active+clean; 126 GiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 2.8 MiB/s rd, 2.6 MiB/s wr, 5 op/s 2024-04-07T16:47:27.643 INFO:tasks.workunit.client.0.smithi100.stdout: 514: throughput=4.99MB/sec pending data=0 2024-04-07T16:47:27.644 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-U3z0yas8l7SoHYA off=9860496 len=1018334 2024-04-07T16:47:27.647 INFO:tasks.workunit.client.0.smithi100.stdout:op 2707 completed, throughput=4.99MB/sec 2024-04-07T16:47:27.647 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-9jRqNQNKL1va8o9 off=750167 len=9320 2024-04-07T16:47:27.648 INFO:tasks.workunit.client.0.smithi100.stdout:op 2708 completed, throughput=4.99MB/sec 2024-04-07T16:47:27.648 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-hGFhljive4rJ1IU off=16783214 len=399530 2024-04-07T16:47:27.654 INFO:tasks.workunit.client.0.smithi100.stdout:op 2709 completed, throughput=5MB/sec 2024-04-07T16:47:27.654 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-3fC4hBi1uAYL_6I off=831788 len=138242 2024-04-07T16:47:27.655 INFO:tasks.workunit.client.0.smithi100.stdout:op 2710 completed, throughput=5MB/sec 2024-04-07T16:47:27.655 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-cb9hGJwTIaFZ3sf off=5328873 len=1369248 2024-04-07T16:47:27.667 INFO:tasks.workunit.client.0.smithi100.stdout:op 2711 completed, throughput=5MB/sec 2024-04-07T16:47:27.667 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-s0orXXYq1f3JxdD off=3679643 len=169167 2024-04-07T16:47:27.671 INFO:tasks.workunit.client.0.smithi100.stdout:op 2712 completed, throughput=5MB/sec 2024-04-07T16:47:27.671 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-WZQ8rN4V-sl4GXX off=3960254 len=388358 2024-04-07T16:47:27.672 INFO:tasks.workunit.client.0.smithi100.stdout:op 2713 completed, throughput=5MB/sec 2024-04-07T16:47:27.672 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-mQjlM8bazLBLT5r off=11622319 len=1894122 2024-04-07T16:47:27.677 INFO:tasks.workunit.client.0.smithi100.stdout:op 2714 completed, throughput=5MB/sec 2024-04-07T16:47:28.678 INFO:tasks.workunit.client.0.smithi100.stdout: 515: throughput=4.99MB/sec pending data=0 2024-04-07T16:47:28.678 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-01VF0fNE6qcNZAk off=3967660 len=1542520 2024-04-07T16:47:28.690 INFO:tasks.workunit.client.0.smithi100.stdout:op 2715 completed, throughput=5MB/sec 2024-04-07T16:47:28.690 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-obbziMLiIAjq8bd off=11097269 len=1348972 2024-04-07T16:47:28.694 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:47:28 smithi100 bash[74942]: cluster 2024-04-07T16:47:27.348574+0000 mgr.y (mgr.25175) 963 : cluster [DBG] pgmap v1605: 105 pgs: 105 active+clean; 126 GiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.7 MiB/s wr, 5 op/s 2024-04-07T16:47:28.695 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:47:28 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:47:28] "GET /metrics HTTP/1.1" 200 34072 "" "Prometheus/2.43.0" 2024-04-07T16:47:28.695 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:47:28 smithi100 bash[76202]: cluster 2024-04-07T16:47:27.348574+0000 mgr.y (mgr.25175) 963 : cluster [DBG] pgmap v1605: 105 pgs: 105 active+clean; 126 GiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.7 MiB/s wr, 5 op/s 2024-04-07T16:47:28.702 INFO:tasks.workunit.client.0.smithi100.stdout:op 2716 completed, throughput=5MB/sec 2024-04-07T16:47:28.702 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-coaPH66vkO8Mgc1 off=6764190 len=1282540 2024-04-07T16:47:28.713 INFO:tasks.workunit.client.0.smithi100.stdout:op 2717 completed, throughput=5MB/sec 2024-04-07T16:47:28.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:47:28 smithi144 bash[66654]: cluster 2024-04-07T16:47:27.348574+0000 mgr.y (mgr.25175) 963 : cluster [DBG] pgmap v1605: 105 pgs: 105 active+clean; 126 GiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.7 MiB/s wr, 5 op/s 2024-04-07T16:47:29.713 INFO:tasks.workunit.client.0.smithi100.stdout: 516: throughput=4.99MB/sec pending data=0 2024-04-07T16:47:29.713 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-K9zmKILdh8-DudF off=5343740 len=565565 2024-04-07T16:47:29.715 INFO:tasks.workunit.client.0.smithi100.stdout:op 2718 completed, throughput=4.99MB/sec 2024-04-07T16:47:29.715 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-cO1TvZxbDsnBx90 off=249696 len=422279 2024-04-07T16:47:29.722 INFO:tasks.workunit.client.0.smithi100.stdout:op 2719 completed, throughput=4.99MB/sec 2024-04-07T16:47:29.722 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-3Uo8WQrXb8VOnzH off=758116 len=1766896 2024-04-07T16:47:29.736 INFO:tasks.workunit.client.0.smithi100.stdout:op 2720 completed, throughput=5MB/sec 2024-04-07T16:47:29.736 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-OtPGZQvs8MFP_uX off=2825299 len=1882869 2024-04-07T16:47:29.751 INFO:tasks.workunit.client.0.smithi100.stdout:op 2721 completed, throughput=5MB/sec 2024-04-07T16:47:29.751 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-cjezeCZq2Hb6EPo off=12670393 len=1063834 2024-04-07T16:47:29.760 INFO:tasks.workunit.client.0.smithi100.stdout:op 2722 completed, throughput=5MB/sec 2024-04-07T16:47:30.760 INFO:tasks.workunit.client.0.smithi100.stdout: 517: throughput=4.99MB/sec pending data=0 2024-04-07T16:47:30.760 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-x-goM7b7JQ-cN3- off=8457117 len=24845 2024-04-07T16:47:30.765 INFO:tasks.workunit.client.0.smithi100.stdout:op 2723 completed, throughput=4.99MB/sec 2024-04-07T16:47:30.765 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-XsWXZsKyLQiVhLT off=1050373 len=882618 2024-04-07T16:47:30.773 INFO:tasks.workunit.client.0.smithi100.stdout:op 2724 completed, throughput=4.99MB/sec 2024-04-07T16:47:30.773 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-UAywZNOBXJVKIsA off=2617967 len=841488 2024-04-07T16:47:30.780 INFO:tasks.workunit.client.0.smithi100.stdout:op 2725 completed, throughput=4.99MB/sec 2024-04-07T16:47:30.780 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-dIxEtkYAr3HE_Eq off=1864735 len=1600583 2024-04-07T16:47:30.783 INFO:tasks.workunit.client.0.smithi100.stdout:op 2726 completed, throughput=5MB/sec 2024-04-07T16:47:30.784 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-KVcJOjhpMcPUJuR off=2361799 len=932896 2024-04-07T16:47:30.791 INFO:tasks.workunit.client.0.smithi100.stdout:op 2727 completed, throughput=5MB/sec 2024-04-07T16:47:30.792 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-PUm6DTo7t9BfNU1 off=1048281 len=2079010 2024-04-07T16:47:30.796 INFO:tasks.workunit.client.0.smithi100.stdout:op 2728 completed, throughput=5MB/sec 2024-04-07T16:47:30.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:47:30 smithi144 bash[66654]: cluster 2024-04-07T16:47:29.349387+0000 mgr.y (mgr.25175) 964 : cluster [DBG] pgmap v1606: 105 pgs: 105 active+clean; 126 GiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.4 MiB/s wr, 5 op/s 2024-04-07T16:47:30.945 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:47:30 smithi100 bash[74942]: cluster 2024-04-07T16:47:29.349387+0000 mgr.y (mgr.25175) 964 : cluster [DBG] pgmap v1606: 105 pgs: 105 active+clean; 126 GiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.4 MiB/s wr, 5 op/s 2024-04-07T16:47:30.945 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:47:30 smithi100 bash[76202]: cluster 2024-04-07T16:47:29.349387+0000 mgr.y (mgr.25175) 964 : cluster [DBG] pgmap v1606: 105 pgs: 105 active+clean; 126 GiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.4 MiB/s wr, 5 op/s 2024-04-07T16:47:31.797 INFO:tasks.workunit.client.0.smithi100.stdout: 518: throughput=4.99MB/sec pending data=0 2024-04-07T16:47:31.797 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-SuVC5iZy1Nk_J_x off=7005084 len=1566691 2024-04-07T16:47:31.809 INFO:tasks.workunit.client.0.smithi100.stdout:op 2729 completed, throughput=5MB/sec 2024-04-07T16:47:31.809 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-MVdVdBkePozIxXa off=294983 len=299011 2024-04-07T16:47:31.815 INFO:tasks.workunit.client.0.smithi100.stdout:op 2730 completed, throughput=5MB/sec 2024-04-07T16:47:31.815 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-xOL2loIDfg0DcEa off=8979417 len=1526859 2024-04-07T16:47:31.820 INFO:tasks.workunit.client.0.smithi100.stdout:op 2731 completed, throughput=5MB/sec 2024-04-07T16:47:31.820 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-qfimu-WswKoCh5S off=8026094 len=216082 2024-04-07T16:47:31.821 INFO:tasks.workunit.client.0.smithi100.stdout:op 2732 completed, throughput=5MB/sec 2024-04-07T16:47:31.821 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-6msTmbP0qD-AKR- off=21594702 len=2050120 2024-04-07T16:47:31.839 INFO:tasks.workunit.client.0.smithi100.stdout:op 2733 completed, throughput=5MB/sec 2024-04-07T16:47:32.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:47:32 smithi144 bash[66654]: cluster 2024-04-07T16:47:31.350318+0000 mgr.y (mgr.25175) 965 : cluster [DBG] pgmap v1607: 105 pgs: 105 active+clean; 126 GiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 2.6 MiB/s rd, 2.5 MiB/s wr, 5 op/s 2024-04-07T16:47:32.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:47:32 smithi144 bash[66654]: audit 2024-04-07T16:47:31.770477+0000 mon.a (mon.0) 2119 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:47:32.610 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:47:32 smithi144 bash[95233]: ts=2024-04-07T16:47:32.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:47:32.839 INFO:tasks.workunit.client.0.smithi100.stdout: 519: throughput=4.99MB/sec pending data=0 2024-04-07T16:47:32.840 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-BAozPKjmp0MMM1u off=0 len=1160190 2024-04-07T16:47:32.850 INFO:tasks.workunit.client.0.smithi100.stdout:op 2734 completed, throughput=5MB/sec 2024-04-07T16:47:32.850 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-fAB6jSaHt2BUYZQ off=1974495 len=2017709 2024-04-07T16:47:32.855 INFO:tasks.workunit.client.0.smithi100.stdout:op 2735 completed, throughput=5MB/sec 2024-04-07T16:47:32.855 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-TgDmNtxc3hafG6c off=17424130 len=731828 2024-04-07T16:47:32.863 INFO:tasks.workunit.client.0.smithi100.stdout:op 2736 completed, throughput=5MB/sec 2024-04-07T16:47:32.863 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-TFsQm_oGbf10bVH off=553443 len=597109 2024-04-07T16:47:32.870 INFO:tasks.workunit.client.0.smithi100.stdout:op 2737 completed, throughput=5MB/sec 2024-04-07T16:47:32.945 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:47:32 smithi100 bash[74942]: cluster 2024-04-07T16:47:31.350318+0000 mgr.y (mgr.25175) 965 : cluster [DBG] pgmap v1607: 105 pgs: 105 active+clean; 126 GiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 2.6 MiB/s rd, 2.5 MiB/s wr, 5 op/s 2024-04-07T16:47:32.945 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:47:32 smithi100 bash[74942]: audit 2024-04-07T16:47:31.770477+0000 mon.a (mon.0) 2119 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:47:32.945 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:47:32 smithi100 bash[76202]: cluster 2024-04-07T16:47:31.350318+0000 mgr.y (mgr.25175) 965 : cluster [DBG] pgmap v1607: 105 pgs: 105 active+clean; 126 GiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 2.6 MiB/s rd, 2.5 MiB/s wr, 5 op/s 2024-04-07T16:47:32.945 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:47:32 smithi100 bash[76202]: audit 2024-04-07T16:47:31.770477+0000 mon.a (mon.0) 2119 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:47:33.871 INFO:tasks.workunit.client.0.smithi100.stdout: 520: throughput=4.99MB/sec pending data=0 2024-04-07T16:47:33.871 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-B4p2m35uJwkE73l off=14199096 len=127502 2024-04-07T16:47:33.872 INFO:tasks.workunit.client.0.smithi100.stdout:op 2738 completed, throughput=4.99MB/sec 2024-04-07T16:47:33.872 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-DAhpI_M_-WCs8BR off=550368 len=1845147 2024-04-07T16:47:33.888 INFO:tasks.workunit.client.0.smithi100.stdout:op 2739 completed, throughput=4.99MB/sec 2024-04-07T16:47:33.888 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-wOJr9MrJ7Mcqxgh off=2065818 len=245725 2024-04-07T16:47:33.890 INFO:tasks.workunit.client.0.smithi100.stdout:op 2740 completed, throughput=5MB/sec 2024-04-07T16:47:33.890 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-k9pWdZqRyFJDCOX off=13923349 len=1704857 2024-04-07T16:47:33.902 INFO:tasks.workunit.client.0.smithi100.stdout:op 2741 completed, throughput=5MB/sec 2024-04-07T16:47:33.902 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Oc6Je097kYXeD5a off=12789132 len=1188018 2024-04-07T16:47:33.911 INFO:tasks.workunit.client.0.smithi100.stdout:op 2742 completed, throughput=5MB/sec 2024-04-07T16:47:34.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:47:34 smithi144 bash[66654]: cluster 2024-04-07T16:47:33.351473+0000 mgr.y (mgr.25175) 966 : cluster [DBG] pgmap v1608: 105 pgs: 105 active+clean; 126 GiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 3.0 MiB/s rd, 2.8 MiB/s wr, 5 op/s 2024-04-07T16:47:34.911 INFO:tasks.workunit.client.0.smithi100.stdout: 521: throughput=4.99MB/sec pending data=0 2024-04-07T16:47:34.911 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-3b74EAvMgaJsWZd off=4084636 len=1190970 2024-04-07T16:47:34.914 INFO:tasks.workunit.client.0.smithi100.stdout:op 2743 completed, throughput=4.99MB/sec 2024-04-07T16:47:34.914 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-KMauUUey9adnPPk off=23353861 len=1798828 2024-04-07T16:47:34.919 INFO:tasks.workunit.client.0.smithi100.stdout:op 2744 completed, throughput=5MB/sec 2024-04-07T16:47:34.919 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-ZOHoXqmQglpE971 off=141198 len=1490750 2024-04-07T16:47:34.930 INFO:tasks.workunit.client.0.smithi100.stdout:op 2745 completed, throughput=5MB/sec 2024-04-07T16:47:34.930 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-r3auoty-GjrFzMm off=8884372 len=10848 2024-04-07T16:47:34.934 INFO:tasks.workunit.client.0.smithi100.stdout:op 2746 completed, throughput=5MB/sec 2024-04-07T16:47:34.935 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-wYvXieafIEozuAt off=2712047 len=1651907 2024-04-07T16:47:34.945 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:47:34 smithi100 bash[74942]: cluster 2024-04-07T16:47:33.351473+0000 mgr.y (mgr.25175) 966 : cluster [DBG] pgmap v1608: 105 pgs: 105 active+clean; 126 GiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 3.0 MiB/s rd, 2.8 MiB/s wr, 5 op/s 2024-04-07T16:47:34.945 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:47:34 smithi100 bash[76202]: cluster 2024-04-07T16:47:33.351473+0000 mgr.y (mgr.25175) 966 : cluster [DBG] pgmap v1608: 105 pgs: 105 active+clean; 126 GiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 3.0 MiB/s rd, 2.8 MiB/s wr, 5 op/s 2024-04-07T16:47:34.947 INFO:tasks.workunit.client.0.smithi100.stdout:op 2747 completed, throughput=5MB/sec 2024-04-07T16:47:35.947 INFO:tasks.workunit.client.0.smithi100.stdout: 522: throughput=4.99MB/sec pending data=0 2024-04-07T16:47:35.947 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-SaWb0IELbp92ezM off=14180432 len=1254244 2024-04-07T16:47:35.952 INFO:tasks.workunit.client.0.smithi100.stdout:op 2748 completed, throughput=4.99MB/sec 2024-04-07T16:47:35.952 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-ZAW--jcizAJGdQA off=17546057 len=985746 2024-04-07T16:47:35.956 INFO:tasks.workunit.client.0.smithi100.stdout:op 2749 completed, throughput=5MB/sec 2024-04-07T16:47:35.956 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-KuiKQ581Sv0KSiZ off=6431974 len=1731235 2024-04-07T16:47:35.969 INFO:tasks.workunit.client.0.smithi100.stdout:op 2750 completed, throughput=5MB/sec 2024-04-07T16:47:35.969 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-6FsCdYwD6QqVRgl off=20075288 len=1356479 2024-04-07T16:47:35.972 INFO:tasks.workunit.client.0.smithi100.stdout:op 2751 completed, throughput=5MB/sec 2024-04-07T16:47:36.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:47:36 smithi144 bash[66654]: cluster 2024-04-07T16:47:35.352384+0000 mgr.y (mgr.25175) 967 : cluster [DBG] pgmap v1609: 105 pgs: 105 active+clean; 126 GiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.4 MiB/s wr, 4 op/s 2024-04-07T16:47:36.945 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:47:36 smithi100 bash[74942]: cluster 2024-04-07T16:47:35.352384+0000 mgr.y (mgr.25175) 967 : cluster [DBG] pgmap v1609: 105 pgs: 105 active+clean; 126 GiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.4 MiB/s wr, 4 op/s 2024-04-07T16:47:36.945 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:47:36 smithi100 bash[76202]: cluster 2024-04-07T16:47:35.352384+0000 mgr.y (mgr.25175) 967 : cluster [DBG] pgmap v1609: 105 pgs: 105 active+clean; 126 GiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.4 MiB/s wr, 4 op/s 2024-04-07T16:47:36.973 INFO:tasks.workunit.client.0.smithi100.stdout: 523: throughput=4.99MB/sec pending data=0 2024-04-07T16:47:36.973 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-h_v5r-Uy5m0O7BC off=6837737 len=654412 2024-04-07T16:47:36.976 INFO:tasks.workunit.client.0.smithi100.stdout:op 2752 completed, throughput=4.99MB/sec 2024-04-07T16:47:36.976 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-JJc8APJlexDe07W off=19013689 len=689542 2024-04-07T16:47:36.984 INFO:tasks.workunit.client.0.smithi100.stdout:op 2753 completed, throughput=4.99MB/sec 2024-04-07T16:47:36.984 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-A6qo8b6lm-6h9CK off=1744097 len=333321 2024-04-07T16:47:36.990 INFO:tasks.workunit.client.0.smithi100.stdout:op 2754 completed, throughput=4.99MB/sec 2024-04-07T16:47:36.990 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-OkH20roYjJ5TRKQ off=787017 len=1277264 2024-04-07T16:47:36.994 INFO:tasks.workunit.client.0.smithi100.stdout:op 2755 completed, throughput=5MB/sec 2024-04-07T16:47:36.994 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-CgKn3NeMjqKVrRX off=90010 len=342792 2024-04-07T16:47:37.000 INFO:tasks.workunit.client.0.smithi100.stdout:op 2756 completed, throughput=5MB/sec 2024-04-07T16:47:37.000 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-LfR8NWhRSyypz6d off=7579981 len=89675 2024-04-07T16:47:37.001 INFO:tasks.workunit.client.0.smithi100.stdout:op 2757 completed, throughput=5MB/sec 2024-04-07T16:47:37.001 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Oyl6PCAa7JD4kA9 off=4365314 len=1251597 2024-04-07T16:47:37.003 INFO:tasks.workunit.client.0.smithi100.stdout:op 2758 completed, throughput=5MB/sec 2024-04-07T16:47:38.004 INFO:tasks.workunit.client.0.smithi100.stdout: 524: throughput=4.99MB/sec pending data=0 2024-04-07T16:47:38.004 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-5d1CvB3s5CuWHMn off=15678050 len=1361348 2024-04-07T16:47:38.015 INFO:tasks.workunit.client.0.smithi100.stdout:op 2759 completed, throughput=4.99MB/sec 2024-04-07T16:47:38.015 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-QGXdblQBOlTFGNf off=7751225 len=82938 2024-04-07T16:47:38.016 INFO:tasks.workunit.client.0.smithi100.stdout:op 2760 completed, throughput=4.99MB/sec 2024-04-07T16:47:38.016 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-EDGjHBxQEXGFQlS off=13407773 len=757801 2024-04-07T16:47:38.019 INFO:tasks.workunit.client.0.smithi100.stdout:op 2761 completed, throughput=4.99MB/sec 2024-04-07T16:47:38.019 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-uTZzxVadKS8ct0H off=6039697 len=3682 2024-04-07T16:47:38.020 INFO:tasks.workunit.client.0.smithi100.stdout:op 2762 completed, throughput=4.99MB/sec 2024-04-07T16:47:38.020 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-J_U_5BNXDZfxJxI off=10983092 len=453278 2024-04-07T16:47:38.021 INFO:tasks.workunit.client.0.smithi100.stdout:op 2763 completed, throughput=5MB/sec 2024-04-07T16:47:38.022 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-GwTGv4feQvIDrGh off=3623576 len=1397637 2024-04-07T16:47:38.033 INFO:tasks.workunit.client.0.smithi100.stdout:op 2764 completed, throughput=5MB/sec 2024-04-07T16:47:38.033 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-iNXHoT-iqmC9gxJ off=18379658 len=1631220 2024-04-07T16:47:38.044 INFO:tasks.workunit.client.0.smithi100.stdout:op 2765 completed, throughput=5MB/sec 2024-04-07T16:47:38.695 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:47:38 smithi100 bash[74942]: cluster 2024-04-07T16:47:37.353408+0000 mgr.y (mgr.25175) 968 : cluster [DBG] pgmap v1610: 105 pgs: 105 active+clean; 126 GiB data, 6.9 GiB used, 708 GiB / 715 GiB avail; 2.1 MiB/s rd, 2.7 MiB/s wr, 4 op/s 2024-04-07T16:47:38.695 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:47:38 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:47:38] "GET /metrics HTTP/1.1" 200 34073 "" "Prometheus/2.43.0" 2024-04-07T16:47:38.695 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:47:38 smithi100 bash[76202]: cluster 2024-04-07T16:47:37.353408+0000 mgr.y (mgr.25175) 968 : cluster [DBG] pgmap v1610: 105 pgs: 105 active+clean; 126 GiB data, 6.9 GiB used, 708 GiB / 715 GiB avail; 2.1 MiB/s rd, 2.7 MiB/s wr, 4 op/s 2024-04-07T16:47:38.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:47:38 smithi144 bash[66654]: cluster 2024-04-07T16:47:37.353408+0000 mgr.y (mgr.25175) 968 : cluster [DBG] pgmap v1610: 105 pgs: 105 active+clean; 126 GiB data, 6.9 GiB used, 708 GiB / 715 GiB avail; 2.1 MiB/s rd, 2.7 MiB/s wr, 4 op/s 2024-04-07T16:47:39.045 INFO:tasks.workunit.client.0.smithi100.stdout: 525: throughput=4.99MB/sec pending data=0 2024-04-07T16:47:39.045 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-euSiQrnXG-yzxoA off=1293737 len=1725234 2024-04-07T16:47:39.058 INFO:tasks.workunit.client.0.smithi100.stdout:op 2766 completed, throughput=4.99MB/sec 2024-04-07T16:47:39.058 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-bfvqvePEtqmw5dV off=8834806 len=1494428 2024-04-07T16:47:39.069 INFO:tasks.workunit.client.0.smithi100.stdout:op 2767 completed, throughput=5MB/sec 2024-04-07T16:47:39.069 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-60dWq99Ts2Jpc2_ off=993659 len=1823793 2024-04-07T16:47:39.084 INFO:tasks.workunit.client.0.smithi100.stdout:op 2768 completed, throughput=5MB/sec 2024-04-07T16:47:39.085 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-6zuW91T4-6IbXI3 off=14105062 len=314692 2024-04-07T16:47:39.090 INFO:tasks.workunit.client.0.smithi100.stdout:op 2769 completed, throughput=5MB/sec 2024-04-07T16:47:40.090 INFO:tasks.workunit.client.0.smithi100.stdout: 526: throughput=4.99MB/sec pending data=0 2024-04-07T16:47:40.090 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-WWeyQgkYZuqbE8N off=8165638 len=1861893 2024-04-07T16:47:40.105 INFO:tasks.workunit.client.0.smithi100.stdout:op 2770 completed, throughput=4.99MB/sec 2024-04-07T16:47:40.105 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-VQMG6WK9G0ewEzl off=13535490 len=2078325 2024-04-07T16:47:40.121 INFO:tasks.workunit.client.0.smithi100.stdout:op 2771 completed, throughput=5MB/sec 2024-04-07T16:47:40.121 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-7QJb6a8GvUoai8e off=7858621 len=69459 2024-04-07T16:47:40.122 INFO:tasks.workunit.client.0.smithi100.stdout:op 2772 completed, throughput=5MB/sec 2024-04-07T16:47:40.123 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-bLkJRu-6IlMMXyZ off=703647 len=2078358 2024-04-07T16:47:40.127 INFO:tasks.workunit.client.0.smithi100.stdout:op 2773 completed, throughput=5MB/sec 2024-04-07T16:47:40.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:47:40 smithi144 bash[66654]: cluster 2024-04-07T16:47:39.354298+0000 mgr.y (mgr.25175) 969 : cluster [DBG] pgmap v1611: 105 pgs: 105 active+clean; 126 GiB data, 6.9 GiB used, 708 GiB / 715 GiB avail; 1.9 MiB/s rd, 2.4 MiB/s wr, 4 op/s 2024-04-07T16:47:40.945 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:47:40 smithi100 bash[74942]: cluster 2024-04-07T16:47:39.354298+0000 mgr.y (mgr.25175) 969 : cluster [DBG] pgmap v1611: 105 pgs: 105 active+clean; 126 GiB data, 6.9 GiB used, 708 GiB / 715 GiB avail; 1.9 MiB/s rd, 2.4 MiB/s wr, 4 op/s 2024-04-07T16:47:40.945 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:47:40 smithi100 bash[76202]: cluster 2024-04-07T16:47:39.354298+0000 mgr.y (mgr.25175) 969 : cluster [DBG] pgmap v1611: 105 pgs: 105 active+clean; 126 GiB data, 6.9 GiB used, 708 GiB / 715 GiB avail; 1.9 MiB/s rd, 2.4 MiB/s wr, 4 op/s 2024-04-07T16:47:41.127 INFO:tasks.workunit.client.0.smithi100.stdout: 527: throughput=4.99MB/sec pending data=0 2024-04-07T16:47:41.128 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-IDXbYwmpYu7ZG66 off=4232745 len=808883 2024-04-07T16:47:41.131 INFO:tasks.workunit.client.0.smithi100.stdout:op 2774 completed, throughput=4.99MB/sec 2024-04-07T16:47:41.131 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-4DkMPObV33OxAoQ off=3463706 len=667364 2024-04-07T16:47:41.133 INFO:tasks.workunit.client.0.smithi100.stdout:op 2775 completed, throughput=4.99MB/sec 2024-04-07T16:47:41.133 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj--bLl_Q-9rts-0S5 off=924056 len=893088 2024-04-07T16:47:41.141 INFO:tasks.workunit.client.0.smithi100.stdout:op 2776 completed, throughput=5MB/sec 2024-04-07T16:47:41.142 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-nr5kxwtkxe5afAb off=483605 len=1985156 2024-04-07T16:47:41.146 INFO:tasks.workunit.client.0.smithi100.stdout:op 2777 completed, throughput=5MB/sec 2024-04-07T16:47:41.146 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-6KE3wI7VFKdnpIK off=1755454 len=1385965 2024-04-07T16:47:41.158 INFO:tasks.workunit.client.0.smithi100.stdout:op 2778 completed, throughput=5MB/sec 2024-04-07T16:47:42.158 INFO:tasks.workunit.client.0.smithi100.stdout: 528: throughput=4.99MB/sec pending data=0 2024-04-07T16:47:42.159 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-nvn0l_gFxnB0QwM off=6909345 len=429492 2024-04-07T16:47:42.160 INFO:tasks.workunit.client.0.smithi100.stdout:op 2779 completed, throughput=4.99MB/sec 2024-04-07T16:47:42.160 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-M9B_DVFQ4DCTwxE off=16302474 len=2059579 2024-04-07T16:47:42.166 INFO:tasks.workunit.client.0.smithi100.stdout:op 2780 completed, throughput=5MB/sec 2024-04-07T16:47:42.167 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Kc-XETDV1Ew8wQi off=12662044 len=765374 2024-04-07T16:47:42.175 INFO:tasks.workunit.client.0.smithi100.stdout:op 2781 completed, throughput=5MB/sec 2024-04-07T16:47:42.175 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-mwuv76MzdYX8yGl off=14950121 len=2047590 2024-04-07T16:47:42.179 INFO:tasks.workunit.client.0.smithi100.stdout:op 2782 completed, throughput=5MB/sec 2024-04-07T16:47:42.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:47:42 smithi144 bash[66654]: cluster 2024-04-07T16:47:41.355229+0000 mgr.y (mgr.25175) 970 : cluster [DBG] pgmap v1612: 105 pgs: 105 active+clean; 126 GiB data, 6.9 GiB used, 708 GiB / 715 GiB avail; 1.7 MiB/s rd, 2.8 MiB/s wr, 4 op/s 2024-04-07T16:47:42.610 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:47:42 smithi144 bash[95233]: ts=2024-04-07T16:47:42.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:47:42.945 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:47:42 smithi100 bash[74942]: cluster 2024-04-07T16:47:41.355229+0000 mgr.y (mgr.25175) 970 : cluster [DBG] pgmap v1612: 105 pgs: 105 active+clean; 126 GiB data, 6.9 GiB used, 708 GiB / 715 GiB avail; 1.7 MiB/s rd, 2.8 MiB/s wr, 4 op/s 2024-04-07T16:47:42.945 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:47:42 smithi100 bash[76202]: cluster 2024-04-07T16:47:41.355229+0000 mgr.y (mgr.25175) 970 : cluster [DBG] pgmap v1612: 105 pgs: 105 active+clean; 126 GiB data, 6.9 GiB used, 708 GiB / 715 GiB avail; 1.7 MiB/s rd, 2.8 MiB/s wr, 4 op/s 2024-04-07T16:47:43.180 INFO:tasks.workunit.client.0.smithi100.stdout: 529: throughput=4.99MB/sec pending data=0 2024-04-07T16:47:43.180 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-1aU48SuwNg6Fm5_ off=2887870 len=1442277 2024-04-07T16:47:43.192 INFO:tasks.workunit.client.0.smithi100.stdout:op 2783 completed, throughput=5MB/sec 2024-04-07T16:47:43.192 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-GftUG86pKMN-R0A off=3986322 len=1163126 2024-04-07T16:47:43.194 INFO:tasks.workunit.client.0.smithi100.stdout:op 2784 completed, throughput=5MB/sec 2024-04-07T16:47:43.194 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-X__Vr6QC2sdSUTA off=27460 len=884705 2024-04-07T16:47:43.198 INFO:tasks.workunit.client.0.smithi100.stdout:op 2785 completed, throughput=5MB/sec 2024-04-07T16:47:43.198 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-amnUD6Hvt6AJ7H5 off=2260218 len=1191688 2024-04-07T16:47:43.207 INFO:tasks.workunit.client.0.smithi100.stdout:op 2786 completed, throughput=5MB/sec 2024-04-07T16:47:44.207 INFO:tasks.workunit.client.0.smithi100.stdout: 530: throughput=4.99MB/sec pending data=0 2024-04-07T16:47:44.207 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-IcVgINB1teqaaib off=17251004 len=1400124 2024-04-07T16:47:44.211 INFO:tasks.workunit.client.0.smithi100.stdout:op 2787 completed, throughput=4.99MB/sec 2024-04-07T16:47:44.211 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-HfqTbeF02i1fyGh off=10893 len=822066 2024-04-07T16:47:44.213 INFO:tasks.workunit.client.0.smithi100.stdout:op 2788 completed, throughput=5MB/sec 2024-04-07T16:47:44.214 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-B1OTtcKd9Wg86sq off=12233118 len=1831722 2024-04-07T16:47:44.218 INFO:tasks.workunit.client.0.smithi100.stdout:op 2789 completed, throughput=5MB/sec 2024-04-07T16:47:44.218 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-AryfA4BKZRF0C1n off=6581998 len=1606461 2024-04-07T16:47:44.222 INFO:tasks.workunit.client.0.smithi100.stdout:op 2790 completed, throughput=5MB/sec 2024-04-07T16:47:44.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:47:44 smithi144 bash[66654]: cluster 2024-04-07T16:47:43.356469+0000 mgr.y (mgr.25175) 971 : cluster [DBG] pgmap v1613: 105 pgs: 105 active+clean; 126 GiB data, 6.9 GiB used, 708 GiB / 715 GiB avail; 2.0 MiB/s rd, 2.9 MiB/s wr, 4 op/s 2024-04-07T16:47:44.945 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:47:44 smithi100 bash[74942]: cluster 2024-04-07T16:47:43.356469+0000 mgr.y (mgr.25175) 971 : cluster [DBG] pgmap v1613: 105 pgs: 105 active+clean; 126 GiB data, 6.9 GiB used, 708 GiB / 715 GiB avail; 2.0 MiB/s rd, 2.9 MiB/s wr, 4 op/s 2024-04-07T16:47:44.945 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:47:44 smithi100 bash[76202]: cluster 2024-04-07T16:47:43.356469+0000 mgr.y (mgr.25175) 971 : cluster [DBG] pgmap v1613: 105 pgs: 105 active+clean; 126 GiB data, 6.9 GiB used, 708 GiB / 715 GiB avail; 2.0 MiB/s rd, 2.9 MiB/s wr, 4 op/s 2024-04-07T16:47:45.222 INFO:tasks.workunit.client.0.smithi100.stdout: 531: throughput=4.99MB/sec pending data=0 2024-04-07T16:47:45.223 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-9PhXfcMHkY20lRv off=8607717 len=842243 2024-04-07T16:47:45.231 INFO:tasks.workunit.client.0.smithi100.stdout:op 2791 completed, throughput=4.99MB/sec 2024-04-07T16:47:45.231 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-ZtDzxtBCDj6yy6S off=19345619 len=288135 2024-04-07T16:47:45.236 INFO:tasks.workunit.client.0.smithi100.stdout:op 2792 completed, throughput=4.99MB/sec 2024-04-07T16:47:45.237 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-q1umw-nnt1kaMvQ off=3491399 len=1541355 2024-04-07T16:47:45.240 INFO:tasks.workunit.client.0.smithi100.stdout:op 2793 completed, throughput=5MB/sec 2024-04-07T16:47:45.240 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-0eghNu9YC_Ls6hz off=1597762 len=1021108 2024-04-07T16:47:45.250 INFO:tasks.workunit.client.0.smithi100.stdout:op 2794 completed, throughput=5MB/sec 2024-04-07T16:47:45.250 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-xe_f0McaW69ZDVk off=1911274 len=818834 2024-04-07T16:47:45.252 INFO:tasks.workunit.client.0.smithi100.stdout:op 2795 completed, throughput=5MB/sec 2024-04-07T16:47:45.252 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-5cZkNvo_gx16rG_ off=4252732 len=1239896 2024-04-07T16:47:45.261 INFO:tasks.workunit.client.0.smithi100.stdout:op 2796 completed, throughput=5MB/sec 2024-04-07T16:47:46.262 INFO:tasks.workunit.client.0.smithi100.stdout: 532: throughput=4.99MB/sec pending data=0 2024-04-07T16:47:46.262 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-NKPQfVzd806Sk1l off=1448342 len=1442012 2024-04-07T16:47:46.274 INFO:tasks.workunit.client.0.smithi100.stdout:op 2797 completed, throughput=5MB/sec 2024-04-07T16:47:46.275 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-xbfuAMlI4jAT3d9 off=15426150 len=1081691 2024-04-07T16:47:46.277 INFO:tasks.workunit.client.0.smithi100.stdout:op 2798 completed, throughput=5MB/sec 2024-04-07T16:47:46.278 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-vONdO2sTYH1G9ZY off=19789712 len=1592938 2024-04-07T16:47:46.291 INFO:tasks.workunit.client.0.smithi100.stdout:op 2799 completed, throughput=5MB/sec 2024-04-07T16:47:46.291 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-ltzecHnhYHYVXm5 off=17575515 len=304332 2024-04-07T16:47:46.292 INFO:tasks.workunit.client.0.smithi100.stdout:op 2800 completed, throughput=5MB/sec 2024-04-07T16:47:46.860 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:47:46 smithi144 bash[66654]: cluster 2024-04-07T16:47:45.357488+0000 mgr.y (mgr.25175) 972 : cluster [DBG] pgmap v1614: 105 pgs: 105 active+clean; 126 GiB data, 6.9 GiB used, 708 GiB / 715 GiB avail; 2.0 MiB/s rd, 2.7 MiB/s wr, 4 op/s 2024-04-07T16:47:46.945 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:47:46 smithi100 bash[74942]: cluster 2024-04-07T16:47:45.357488+0000 mgr.y (mgr.25175) 972 : cluster [DBG] pgmap v1614: 105 pgs: 105 active+clean; 126 GiB data, 6.9 GiB used, 708 GiB / 715 GiB avail; 2.0 MiB/s rd, 2.7 MiB/s wr, 4 op/s 2024-04-07T16:47:46.945 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:47:46 smithi100 bash[76202]: cluster 2024-04-07T16:47:45.357488+0000 mgr.y (mgr.25175) 972 : cluster [DBG] pgmap v1614: 105 pgs: 105 active+clean; 126 GiB data, 6.9 GiB used, 708 GiB / 715 GiB avail; 2.0 MiB/s rd, 2.7 MiB/s wr, 4 op/s 2024-04-07T16:47:47.293 INFO:tasks.workunit.client.0.smithi100.stdout: 533: throughput=4.99MB/sec pending data=0 2024-04-07T16:47:47.293 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-KELjeXISuVuWsgI off=10663153 len=461263 2024-04-07T16:47:47.294 INFO:tasks.workunit.client.0.smithi100.stdout:op 2801 completed, throughput=4.99MB/sec 2024-04-07T16:47:47.295 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-zgOsYnDGXT_IZdg off=3002025 len=922128 2024-04-07T16:47:47.303 INFO:tasks.workunit.client.0.smithi100.stdout:op 2802 completed, throughput=4.99MB/sec 2024-04-07T16:47:47.303 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-RHIgKbia2twUKjP off=9050110 len=2022257 2024-04-07T16:47:47.319 INFO:tasks.workunit.client.0.smithi100.stdout:op 2803 completed, throughput=5MB/sec 2024-04-07T16:47:47.319 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-2I5MzIEZkb_RHkj off=7783473 len=1939650 2024-04-07T16:47:47.324 INFO:tasks.workunit.client.0.smithi100.stdout:op 2804 completed, throughput=5MB/sec 2024-04-07T16:47:47.324 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-aYHrbF88Ef1dXft off=23688301 len=1032187 2024-04-07T16:47:47.326 INFO:tasks.workunit.client.0.smithi100.stdout:op 2805 completed, throughput=5MB/sec 2024-04-07T16:47:47.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:47:47 smithi144 bash[66654]: audit 2024-04-07T16:47:46.770991+0000 mon.a (mon.0) 2120 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:47:47.945 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:47:47 smithi100 bash[74942]: audit 2024-04-07T16:47:46.770991+0000 mon.a (mon.0) 2120 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:47:47.945 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:47:47 smithi100 bash[76202]: audit 2024-04-07T16:47:46.770991+0000 mon.a (mon.0) 2120 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:47:48.326 INFO:tasks.workunit.client.0.smithi100.stdout: 534: throughput=4.99MB/sec pending data=0 2024-04-07T16:47:48.326 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-ILfL4YPDr2RHnsT off=953812 len=16501 2024-04-07T16:47:48.327 INFO:tasks.workunit.client.0.smithi100.stdout:op 2806 completed, throughput=4.99MB/sec 2024-04-07T16:47:48.327 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj--O0tWYyKZj3jFtl off=627216 len=1734964 2024-04-07T16:47:48.331 INFO:tasks.workunit.client.0.smithi100.stdout:op 2807 completed, throughput=5MB/sec 2024-04-07T16:47:48.331 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-wBFlNHomHLuSLyx off=4193091 len=1062912 2024-04-07T16:47:48.334 INFO:tasks.workunit.client.0.smithi100.stdout:op 2808 completed, throughput=5MB/sec 2024-04-07T16:47:48.334 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-4iILh1_L7B_dV2q off=3606211 len=191888 2024-04-07T16:47:48.338 INFO:tasks.workunit.client.0.smithi100.stdout:op 2809 completed, throughput=5MB/sec 2024-04-07T16:47:48.339 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-KrCkz8a5WIWu1QQ off=18248804 len=1410518 2024-04-07T16:47:48.352 INFO:tasks.workunit.client.0.smithi100.stdout:op 2810 completed, throughput=5MB/sec 2024-04-07T16:47:48.695 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:47:48 smithi100 bash[74942]: cluster 2024-04-07T16:47:47.358548+0000 mgr.y (mgr.25175) 973 : cluster [DBG] pgmap v1615: 105 pgs: 105 active+clean; 126 GiB data, 6.9 GiB used, 708 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.6 MiB/s wr, 4 op/s 2024-04-07T16:47:48.695 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:47:48 smithi100 bash[76202]: cluster 2024-04-07T16:47:47.358548+0000 mgr.y (mgr.25175) 973 : cluster [DBG] pgmap v1615: 105 pgs: 105 active+clean; 126 GiB data, 6.9 GiB used, 708 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.6 MiB/s wr, 4 op/s 2024-04-07T16:47:48.696 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:47:48 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:47:48] "GET /metrics HTTP/1.1" 200 34071 "" "Prometheus/2.43.0" 2024-04-07T16:47:48.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:47:48 smithi144 bash[66654]: cluster 2024-04-07T16:47:47.358548+0000 mgr.y (mgr.25175) 973 : cluster [DBG] pgmap v1615: 105 pgs: 105 active+clean; 126 GiB data, 6.9 GiB used, 708 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.6 MiB/s wr, 4 op/s 2024-04-07T16:47:49.352 INFO:tasks.workunit.client.0.smithi100.stdout: 535: throughput=4.99MB/sec pending data=0 2024-04-07T16:47:49.352 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-6bOEldPsBEjgx0P off=20033633 len=1173806 2024-04-07T16:47:49.356 INFO:tasks.workunit.client.0.smithi100.stdout:op 2811 completed, throughput=4.99MB/sec 2024-04-07T16:47:49.356 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-o_ZkWu2HKemOqvC off=1966257 len=101274 2024-04-07T16:47:49.361 INFO:tasks.workunit.client.0.smithi100.stdout:op 2812 completed, throughput=4.99MB/sec 2024-04-07T16:47:49.361 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-T75cqobu_OihxB7 off=1061596 len=467950 2024-04-07T16:47:49.363 INFO:tasks.workunit.client.0.smithi100.stdout:op 2813 completed, throughput=4.99MB/sec 2024-04-07T16:47:49.363 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-SxJ8vay29OKJtwR off=100357 len=713386 2024-04-07T16:47:49.370 INFO:tasks.workunit.client.0.smithi100.stdout:op 2814 completed, throughput=5MB/sec 2024-04-07T16:47:49.370 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-m2niwCuwYEs4YD_ off=17655125 len=180569 2024-04-07T16:47:49.372 INFO:tasks.workunit.client.0.smithi100.stdout:op 2815 completed, throughput=5MB/sec 2024-04-07T16:47:49.372 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-TAus-P8ktCOC4sL off=6405172 len=148682 2024-04-07T16:47:49.377 INFO:tasks.workunit.client.0.smithi100.stdout:op 2816 completed, throughput=5MB/sec 2024-04-07T16:47:49.377 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-EHIQf9qX4dhxel3 off=5563793 len=883001 2024-04-07T16:47:49.379 INFO:tasks.workunit.client.0.smithi100.stdout:op 2817 completed, throughput=5MB/sec 2024-04-07T16:47:49.379 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-TCUwNS7GBIfdGQ3 off=1845520 len=1649857 2024-04-07T16:47:49.383 INFO:tasks.workunit.client.0.smithi100.stdout:op 2818 completed, throughput=5MB/sec 2024-04-07T16:47:49.383 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-zZBbJyYZkV7WX1c off=1644815 len=1595119 2024-04-07T16:47:49.396 INFO:tasks.workunit.client.0.smithi100.stdout:op 2819 completed, throughput=5MB/sec 2024-04-07T16:47:50.397 INFO:tasks.workunit.client.0.smithi100.stdout: 536: throughput=4.99MB/sec pending data=0 2024-04-07T16:47:50.397 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Y8FXr1NL5sJeKSy off=12871430 len=1420361 2024-04-07T16:47:50.400 INFO:tasks.workunit.client.0.smithi100.stdout:op 2820 completed, throughput=5MB/sec 2024-04-07T16:47:50.401 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-qC_WfmcDQIOBB5S off=352505 len=471087 2024-04-07T16:47:50.407 INFO:tasks.workunit.client.0.smithi100.stdout:op 2821 completed, throughput=5MB/sec 2024-04-07T16:47:50.408 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-gZxQz3qfSC_savB off=5089072 len=488863 2024-04-07T16:47:50.414 INFO:tasks.workunit.client.0.smithi100.stdout:op 2822 completed, throughput=5MB/sec 2024-04-07T16:47:50.414 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-dIxEtkYAr3HE_Eq off=3910982 len=2071149 2024-04-07T16:47:50.432 INFO:tasks.workunit.client.0.smithi100.stdout:op 2823 completed, throughput=5MB/sec 2024-04-07T16:47:50.860 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:47:50 smithi144 bash[66654]: cluster 2024-04-07T16:47:49.359405+0000 mgr.y (mgr.25175) 974 : cluster [DBG] pgmap v1616: 105 pgs: 105 active+clean; 126 GiB data, 6.9 GiB used, 708 GiB / 715 GiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 4 op/s 2024-04-07T16:47:50.945 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:47:50 smithi100 bash[74942]: cluster 2024-04-07T16:47:49.359405+0000 mgr.y (mgr.25175) 974 : cluster [DBG] pgmap v1616: 105 pgs: 105 active+clean; 126 GiB data, 6.9 GiB used, 708 GiB / 715 GiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 4 op/s 2024-04-07T16:47:50.945 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:47:50 smithi100 bash[76202]: cluster 2024-04-07T16:47:49.359405+0000 mgr.y (mgr.25175) 974 : cluster [DBG] pgmap v1616: 105 pgs: 105 active+clean; 126 GiB data, 6.9 GiB used, 708 GiB / 715 GiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 4 op/s 2024-04-07T16:47:51.432 INFO:tasks.workunit.client.0.smithi100.stdout: 537: throughput=4.99MB/sec pending data=0 2024-04-07T16:47:51.432 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj--UrZX5dIqIKSQH7 off=17163223 len=1286205 2024-04-07T16:47:51.442 INFO:tasks.workunit.client.0.smithi100.stdout:op 2824 completed, throughput=4.99MB/sec 2024-04-07T16:47:51.442 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-dLpAsIzKas_7fRZ off=5398282 len=1425312 2024-04-07T16:47:51.446 INFO:tasks.workunit.client.0.smithi100.stdout:op 2825 completed, throughput=5MB/sec 2024-04-07T16:47:51.446 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-XW7Q-Sl0qucLt5A off=7870711 len=1233882 2024-04-07T16:47:51.456 INFO:tasks.workunit.client.0.smithi100.stdout:op 2826 completed, throughput=5MB/sec 2024-04-07T16:47:51.456 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-WVlYmuNDklRgzsb off=5131527 len=472215 2024-04-07T16:47:51.462 INFO:tasks.workunit.client.0.smithi100.stdout:op 2827 completed, throughput=5MB/sec 2024-04-07T16:47:51.462 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-9_PEo9PwsYZTqtP off=877402 len=1290874 2024-04-07T16:47:51.466 INFO:tasks.workunit.client.0.smithi100.stdout:op 2828 completed, throughput=5MB/sec 2024-04-07T16:47:52.466 INFO:tasks.workunit.client.0.smithi100.stdout: 538: throughput=4.99MB/sec pending data=0 2024-04-07T16:47:52.466 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Sc3G8BlMf-j0uHX off=3731366 len=892728 2024-04-07T16:47:52.469 INFO:tasks.workunit.client.0.smithi100.stdout:op 2829 completed, throughput=4.99MB/sec 2024-04-07T16:47:52.469 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-jkhxWPMAQ2Hm_Zx off=10853449 len=405404 2024-04-07T16:47:52.471 INFO:tasks.workunit.client.0.smithi100.stdout:op 2830 completed, throughput=4.99MB/sec 2024-04-07T16:47:52.471 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-LdRStV4iAWbN6aW off=2642666 len=1511377 2024-04-07T16:47:52.483 INFO:tasks.workunit.client.0.smithi100.stdout:op 2831 completed, throughput=5MB/sec 2024-04-07T16:47:52.483 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-yL9XwjEI1Aam4wK off=3637339 len=1220805 2024-04-07T16:47:52.487 INFO:tasks.workunit.client.0.smithi100.stdout:op 2832 completed, throughput=5MB/sec 2024-04-07T16:47:52.487 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-GspiDLC9Slb1xrt off=3010162 len=113840 2024-04-07T16:47:52.492 INFO:tasks.workunit.client.0.smithi100.stdout:op 2833 completed, throughput=5MB/sec 2024-04-07T16:47:52.492 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-U1cRlk-SSJWAUCe off=18994635 len=1760370 2024-04-07T16:47:52.504 INFO:tasks.workunit.client.0.smithi100.stdout:op 2834 completed, throughput=5MB/sec 2024-04-07T16:47:52.579 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:47:52 smithi144 bash[95233]: ts=2024-04-07T16:47:52.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:47:52.860 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:47:52 smithi144 bash[66654]: cluster 2024-04-07T16:47:51.360313+0000 mgr.y (mgr.25175) 975 : cluster [DBG] pgmap v1617: 105 pgs: 105 active+clean; 126 GiB data, 7.0 GiB used, 708 GiB / 715 GiB avail; 2.6 MiB/s rd, 2.6 MiB/s wr, 5 op/s 2024-04-07T16:47:52.945 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:47:52 smithi100 bash[74942]: cluster 2024-04-07T16:47:51.360313+0000 mgr.y (mgr.25175) 975 : cluster [DBG] pgmap v1617: 105 pgs: 105 active+clean; 126 GiB data, 7.0 GiB used, 708 GiB / 715 GiB avail; 2.6 MiB/s rd, 2.6 MiB/s wr, 5 op/s 2024-04-07T16:47:52.945 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:47:52 smithi100 bash[76202]: cluster 2024-04-07T16:47:51.360313+0000 mgr.y (mgr.25175) 975 : cluster [DBG] pgmap v1617: 105 pgs: 105 active+clean; 126 GiB data, 7.0 GiB used, 708 GiB / 715 GiB avail; 2.6 MiB/s rd, 2.6 MiB/s wr, 5 op/s 2024-04-07T16:47:53.505 INFO:tasks.workunit.client.0.smithi100.stdout: 539: throughput=4.99MB/sec pending data=0 2024-04-07T16:47:53.505 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Ghrz0Z9QlQ-ws3a off=6520289 len=1436469 2024-04-07T16:47:53.509 INFO:tasks.workunit.client.0.smithi100.stdout:op 2835 completed, throughput=5MB/sec 2024-04-07T16:47:53.509 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-o5ENfaB6HEaDtWA off=1106162 len=1510230 2024-04-07T16:47:53.520 INFO:tasks.workunit.client.0.smithi100.stdout:op 2836 completed, throughput=5MB/sec 2024-04-07T16:47:53.520 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-NwnoGNk_OTrJzsr off=15865683 len=1646185 2024-04-07T16:47:53.524 INFO:tasks.workunit.client.0.smithi100.stdout:op 2837 completed, throughput=5MB/sec 2024-04-07T16:47:54.525 INFO:tasks.workunit.client.0.smithi100.stdout: 540: throughput=4.99MB/sec pending data=0 2024-04-07T16:47:54.525 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-RmIJJ9v47J5W-kO off=2033545 len=2050028 2024-04-07T16:47:54.540 INFO:tasks.workunit.client.0.smithi100.stdout:op 2838 completed, throughput=5MB/sec 2024-04-07T16:47:54.540 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-xaR8wpqRY_SUFGu off=508159 len=782790 2024-04-07T16:47:54.547 INFO:tasks.workunit.client.0.smithi100.stdout:op 2839 completed, throughput=5MB/sec 2024-04-07T16:47:54.547 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-eFKVqUQfwT2KMre off=17110896 len=602795 2024-04-07T16:47:54.550 INFO:tasks.workunit.client.0.smithi100.stdout:op 2840 completed, throughput=5MB/sec 2024-04-07T16:47:54.550 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-AM4A6yVYw6ftf0R off=4988633 len=461571 2024-04-07T16:47:54.557 INFO:tasks.workunit.client.0.smithi100.stdout:op 2841 completed, throughput=5MB/sec 2024-04-07T16:47:54.557 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-zmYO1XUP0cKtStX off=6587199 len=1624204 2024-04-07T16:47:54.561 INFO:tasks.workunit.client.0.smithi100.stdout:op 2842 completed, throughput=5MB/sec 2024-04-07T16:47:54.860 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:47:54 smithi144 bash[66654]: cluster 2024-04-07T16:47:53.361419+0000 mgr.y (mgr.25175) 976 : cluster [DBG] pgmap v1618: 105 pgs: 105 active+clean; 126 GiB data, 7.0 GiB used, 708 GiB / 715 GiB avail; 2.9 MiB/s rd, 2.7 MiB/s wr, 5 op/s 2024-04-07T16:47:54.945 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:47:54 smithi100 bash[74942]: cluster 2024-04-07T16:47:53.361419+0000 mgr.y (mgr.25175) 976 : cluster [DBG] pgmap v1618: 105 pgs: 105 active+clean; 126 GiB data, 7.0 GiB used, 708 GiB / 715 GiB avail; 2.9 MiB/s rd, 2.7 MiB/s wr, 5 op/s 2024-04-07T16:47:54.945 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:47:54 smithi100 bash[76202]: cluster 2024-04-07T16:47:53.361419+0000 mgr.y (mgr.25175) 976 : cluster [DBG] pgmap v1618: 105 pgs: 105 active+clean; 126 GiB data, 7.0 GiB used, 708 GiB / 715 GiB avail; 2.9 MiB/s rd, 2.7 MiB/s wr, 5 op/s 2024-04-07T16:47:55.561 INFO:tasks.workunit.client.0.smithi100.stdout: 541: throughput=4.99MB/sec pending data=0 2024-04-07T16:47:55.561 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-kmeINR924JKaaTy off=691770 len=1734902 2024-04-07T16:47:55.565 INFO:tasks.workunit.client.0.smithi100.stdout:op 2843 completed, throughput=4.99MB/sec 2024-04-07T16:47:55.566 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-8qMriI_-A6hmOnk off=3473655 len=1871063 2024-04-07T16:47:55.580 INFO:tasks.workunit.client.0.smithi100.stdout:op 2844 completed, throughput=5MB/sec 2024-04-07T16:47:55.581 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-UbkKK_s7lPgHbmk off=1903819 len=679243 2024-04-07T16:47:55.582 INFO:tasks.workunit.client.0.smithi100.stdout:op 2845 completed, throughput=5MB/sec 2024-04-07T16:47:55.583 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-2mKAZAZNmjodbTH off=17101529 len=1690125 2024-04-07T16:47:55.586 INFO:tasks.workunit.client.0.smithi100.stdout:op 2846 completed, throughput=5MB/sec 2024-04-07T16:47:56.586 INFO:tasks.workunit.client.0.smithi100.stdout: 542: throughput=4.99MB/sec pending data=0 2024-04-07T16:47:56.586 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-fkJVLO674vHT2pQ off=625640 len=807222 2024-04-07T16:47:56.592 INFO:tasks.workunit.client.0.smithi100.stdout:op 2847 completed, throughput=4.99MB/sec 2024-04-07T16:47:56.592 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-jp9-f5URB15sAZE off=9024185 len=442350 2024-04-07T16:47:56.596 INFO:tasks.workunit.client.0.smithi100.stdout:op 2848 completed, throughput=5MB/sec 2024-04-07T16:47:56.596 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-b5TE8Ukuou4tXr8 off=16943998 len=421861 2024-04-07T16:47:56.596 INFO:tasks.workunit.client.0.smithi100.stdout:op 2849 completed, throughput=5MB/sec 2024-04-07T16:47:56.596 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-VVFi_vTKD3Gy8YX off=1180596 len=1564879 2024-04-07T16:47:56.604 INFO:tasks.workunit.client.0.smithi100.stdout:op 2850 completed, throughput=5MB/sec 2024-04-07T16:47:56.605 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-xx8JWFvDwtn-o53 off=7624536 len=494565 2024-04-07T16:47:56.609 INFO:tasks.workunit.client.0.smithi100.stdout:op 2851 completed, throughput=5MB/sec 2024-04-07T16:47:56.609 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Nmqx1Azn1fM15lX off=6047514 len=352866 2024-04-07T16:47:56.610 INFO:tasks.workunit.client.0.smithi100.stdout:op 2852 completed, throughput=5MB/sec 2024-04-07T16:47:56.610 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-uVQ8UFE-_sW-Qo5 off=5878716 len=1568927 2024-04-07T16:47:56.618 INFO:tasks.workunit.client.0.smithi100.stdout:op 2853 completed, throughput=5MB/sec 2024-04-07T16:47:56.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:47:56 smithi144 bash[66654]: cluster 2024-04-07T16:47:55.362355+0000 mgr.y (mgr.25175) 977 : cluster [DBG] pgmap v1619: 105 pgs: 105 active+clean; 126 GiB data, 7.0 GiB used, 708 GiB / 715 GiB avail; 2.8 MiB/s rd, 2.7 MiB/s wr, 5 op/s 2024-04-07T16:47:56.860 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:47:56 smithi144 bash[66654]: audit 2024-04-07T16:47:56.121391+0000 mon.a (mon.0) 2121 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:47:56.945 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:47:56 smithi100 bash[74942]: cluster 2024-04-07T16:47:55.362355+0000 mgr.y (mgr.25175) 977 : cluster [DBG] pgmap v1619: 105 pgs: 105 active+clean; 126 GiB data, 7.0 GiB used, 708 GiB / 715 GiB avail; 2.8 MiB/s rd, 2.7 MiB/s wr, 5 op/s 2024-04-07T16:47:56.945 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:47:56 smithi100 bash[74942]: audit 2024-04-07T16:47:56.121391+0000 mon.a (mon.0) 2121 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:47:56.945 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:47:56 smithi100 bash[76202]: cluster 2024-04-07T16:47:55.362355+0000 mgr.y (mgr.25175) 977 : cluster [DBG] pgmap v1619: 105 pgs: 105 active+clean; 126 GiB data, 7.0 GiB used, 708 GiB / 715 GiB avail; 2.8 MiB/s rd, 2.7 MiB/s wr, 5 op/s 2024-04-07T16:47:56.945 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:47:56 smithi100 bash[76202]: audit 2024-04-07T16:47:56.121391+0000 mon.a (mon.0) 2121 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:47:57.618 INFO:tasks.workunit.client.0.smithi100.stdout: 543: throughput=4.99MB/sec pending data=0 2024-04-07T16:47:57.618 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-9Mh9AyIvCOnPLAV off=2912292 len=1094110 2024-04-07T16:47:57.628 INFO:tasks.workunit.client.0.smithi100.stdout:op 2854 completed, throughput=5MB/sec 2024-04-07T16:47:57.628 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-78DBto2WX6W5iv4 off=2777709 len=565300 2024-04-07T16:47:57.635 INFO:tasks.workunit.client.0.smithi100.stdout:op 2855 completed, throughput=5MB/sec 2024-04-07T16:47:57.635 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-OKsgQq8sZgK-xke off=4478090 len=2084008 2024-04-07T16:47:57.640 INFO:tasks.workunit.client.0.smithi100.stdout:op 2856 completed, throughput=5MB/sec 2024-04-07T16:47:57.640 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-lo1Ktl94V9QEpVI off=7761415 len=427830 2024-04-07T16:47:57.641 INFO:tasks.workunit.client.0.smithi100.stdout:op 2857 completed, throughput=5MB/sec 2024-04-07T16:47:57.860 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:47:57 smithi144 bash[66654]: audit 2024-04-07T16:47:56.854701+0000 mon.a (mon.0) 2122 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:47:57.860 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:47:57 smithi144 bash[66654]: audit 2024-04-07T16:47:56.856499+0000 mon.a (mon.0) 2123 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:47:57.860 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:47:57 smithi144 bash[66654]: audit 2024-04-07T16:47:56.865410+0000 mon.a (mon.0) 2124 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:47:57.945 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:47:57 smithi100 bash[74942]: audit 2024-04-07T16:47:56.854701+0000 mon.a (mon.0) 2122 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:47:57.945 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:47:57 smithi100 bash[74942]: audit 2024-04-07T16:47:56.856499+0000 mon.a (mon.0) 2123 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:47:57.945 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:47:57 smithi100 bash[74942]: audit 2024-04-07T16:47:56.865410+0000 mon.a (mon.0) 2124 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:47:57.946 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:47:57 smithi100 bash[76202]: audit 2024-04-07T16:47:56.854701+0000 mon.a (mon.0) 2122 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:47:57.946 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:47:57 smithi100 bash[76202]: audit 2024-04-07T16:47:56.856499+0000 mon.a (mon.0) 2123 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:47:57.946 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:47:57 smithi100 bash[76202]: audit 2024-04-07T16:47:56.865410+0000 mon.a (mon.0) 2124 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:47:58.642 INFO:tasks.workunit.client.0.smithi100.stdout: 544: throughput=4.99MB/sec pending data=0 2024-04-07T16:47:58.642 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-EAZADYwqCFVY1nl off=1915804 len=1735955 2024-04-07T16:47:58.646 INFO:tasks.workunit.client.0.smithi100.stdout:op 2858 completed, throughput=4.99MB/sec 2024-04-07T16:47:58.646 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-WHxQP4kZ7BGtLtV off=6532438 len=1083639 2024-04-07T16:47:58.649 INFO:tasks.workunit.client.0.smithi100.stdout:op 2859 completed, throughput=5MB/sec 2024-04-07T16:47:58.649 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-qnfsQIIBgoM3J-z off=2293446 len=1536947 2024-04-07T16:47:58.655 INFO:tasks.workunit.client.0.smithi100.stdout:op 2860 completed, throughput=5MB/sec 2024-04-07T16:47:58.655 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-qeXp6NoUzUIL_HE off=3401118 len=1332685 2024-04-07T16:47:58.658 INFO:tasks.workunit.client.0.smithi100.stdout:op 2861 completed, throughput=5MB/sec 2024-04-07T16:47:58.695 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:47:58 smithi100 bash[74942]: cluster 2024-04-07T16:47:57.363410+0000 mgr.y (mgr.25175) 978 : cluster [DBG] pgmap v1620: 105 pgs: 105 active+clean; 126 GiB data, 7.0 GiB used, 708 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.5 MiB/s wr, 4 op/s 2024-04-07T16:47:58.695 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:47:58 smithi100 bash[76202]: cluster 2024-04-07T16:47:57.363410+0000 mgr.y (mgr.25175) 978 : cluster [DBG] pgmap v1620: 105 pgs: 105 active+clean; 126 GiB data, 7.0 GiB used, 708 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.5 MiB/s wr, 4 op/s 2024-04-07T16:47:58.695 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:47:58 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:47:58] "GET /metrics HTTP/1.1" 200 34071 "" "Prometheus/2.43.0" 2024-04-07T16:47:58.860 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:47:58 smithi144 bash[66654]: cluster 2024-04-07T16:47:57.363410+0000 mgr.y (mgr.25175) 978 : cluster [DBG] pgmap v1620: 105 pgs: 105 active+clean; 126 GiB data, 7.0 GiB used, 708 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.5 MiB/s wr, 4 op/s 2024-04-07T16:47:59.659 INFO:tasks.workunit.client.0.smithi100.stdout: 545: throughput=4.99MB/sec pending data=0 2024-04-07T16:47:59.659 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-GXbH947gNVd-KG7 off=1049764 len=467137 2024-04-07T16:47:59.661 INFO:tasks.workunit.client.0.smithi100.stdout:op 2862 completed, throughput=4.99MB/sec 2024-04-07T16:47:59.661 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Di5xLuIvfzPiR_X off=8088072 len=2015116 2024-04-07T16:47:59.675 INFO:tasks.workunit.client.0.smithi100.stdout:op 2863 completed, throughput=5MB/sec 2024-04-07T16:47:59.675 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-3iQKL3It5LH2Z4H off=20033007 len=1356853 2024-04-07T16:47:59.679 INFO:tasks.workunit.client.0.smithi100.stdout:op 2864 completed, throughput=5MB/sec 2024-04-07T16:47:59.679 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-W6yOMLDBKEw2mtI off=6192919 len=183049 2024-04-07T16:47:59.683 INFO:tasks.workunit.client.0.smithi100.stdout:op 2865 completed, throughput=5MB/sec 2024-04-07T16:47:59.683 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-OMpPSjotj8ulWui off=3779387 len=978967 2024-04-07T16:47:59.686 INFO:tasks.workunit.client.0.smithi100.stdout:op 2866 completed, throughput=5MB/sec 2024-04-07T16:48:00.686 INFO:tasks.workunit.client.0.smithi100.stdout: 546: throughput=4.99MB/sec pending data=0 2024-04-07T16:48:00.687 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-pb0J-WMIR--AQe3 off=1459735 len=301820 2024-04-07T16:48:00.693 INFO:tasks.workunit.client.0.smithi100.stdout:op 2867 completed, throughput=4.99MB/sec 2024-04-07T16:48:00.693 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-gq-72e18vJMUYPh off=10687299 len=209106 2024-04-07T16:48:00.694 INFO:tasks.workunit.client.0.smithi100.stdout:op 2868 completed, throughput=4.99MB/sec 2024-04-07T16:48:00.694 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-RyHe2iZM6u9IIr- off=1990205 len=2000935 2024-04-07T16:48:00.700 INFO:tasks.workunit.client.0.smithi100.stdout:op 2869 completed, throughput=5MB/sec 2024-04-07T16:48:00.701 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-owvhWtFT6w7oXMM off=2689760 len=1665742 2024-04-07T16:48:00.714 INFO:tasks.workunit.client.0.smithi100.stdout:op 2870 completed, throughput=5MB/sec 2024-04-07T16:48:00.714 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-OJ3zf0snZ7JAjzv off=10761980 len=1184991 2024-04-07T16:48:00.724 INFO:tasks.workunit.client.0.smithi100.stdout:op 2871 completed, throughput=5MB/sec 2024-04-07T16:48:00.860 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:48:00 smithi144 bash[66654]: cluster 2024-04-07T16:47:59.364202+0000 mgr.y (mgr.25175) 979 : cluster [DBG] pgmap v1621: 105 pgs: 105 active+clean; 126 GiB data, 7.0 GiB used, 708 GiB / 715 GiB avail; 2.3 MiB/s rd, 2.5 MiB/s wr, 4 op/s 2024-04-07T16:48:00.945 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:48:00 smithi100 bash[74942]: cluster 2024-04-07T16:47:59.364202+0000 mgr.y (mgr.25175) 979 : cluster [DBG] pgmap v1621: 105 pgs: 105 active+clean; 126 GiB data, 7.0 GiB used, 708 GiB / 715 GiB avail; 2.3 MiB/s rd, 2.5 MiB/s wr, 4 op/s 2024-04-07T16:48:00.945 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:48:00 smithi100 bash[76202]: cluster 2024-04-07T16:47:59.364202+0000 mgr.y (mgr.25175) 979 : cluster [DBG] pgmap v1621: 105 pgs: 105 active+clean; 126 GiB data, 7.0 GiB used, 708 GiB / 715 GiB avail; 2.3 MiB/s rd, 2.5 MiB/s wr, 4 op/s 2024-04-07T16:48:01.724 INFO:tasks.workunit.client.0.smithi100.stdout: 547: throughput=4.99MB/sec pending data=0 2024-04-07T16:48:01.724 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-t2KI8e95moq05Jb off=8124916 len=1991849 2024-04-07T16:48:01.731 INFO:tasks.workunit.client.0.smithi100.stdout:op 2872 completed, throughput=4.99MB/sec 2024-04-07T16:48:01.731 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-v5hwSanaZCGyw6- off=8366037 len=734895 2024-04-07T16:48:01.738 INFO:tasks.workunit.client.0.smithi100.stdout:op 2873 completed, throughput=5MB/sec 2024-04-07T16:48:01.739 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-zyrA-n9v24q7YuW off=2515354 len=1192550 2024-04-07T16:48:01.748 INFO:tasks.workunit.client.0.smithi100.stdout:op 2874 completed, throughput=5MB/sec 2024-04-07T16:48:01.748 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-m8r86ss4lwXrjYI off=6012336 len=101313 2024-04-07T16:48:01.749 INFO:tasks.workunit.client.0.smithi100.stdout:op 2875 completed, throughput=5MB/sec 2024-04-07T16:48:01.749 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-zyKM3PRq0gRCdAp off=6166358 len=112282 2024-04-07T16:48:01.754 INFO:tasks.workunit.client.0.smithi100.stdout:op 2876 completed, throughput=5MB/sec 2024-04-07T16:48:01.754 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-jYBDUPhrmfXR9R2 off=13301412 len=410035 2024-04-07T16:48:01.759 INFO:tasks.workunit.client.0.smithi100.stdout:op 2877 completed, throughput=5MB/sec 2024-04-07T16:48:01.759 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-2MZm6KlCwCpxw38 off=13769313 len=48543 2024-04-07T16:48:01.760 INFO:tasks.workunit.client.0.smithi100.stdout:op 2878 completed, throughput=5MB/sec 2024-04-07T16:48:01.760 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-FdW5jAKOxiAY9Ro off=8215092 len=1434711 2024-04-07T16:48:01.764 INFO:tasks.workunit.client.0.smithi100.stdout:op 2879 completed, throughput=5MB/sec 2024-04-07T16:48:02.604 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:48:02 smithi144 bash[95233]: ts=2024-04-07T16:48:02.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:48:02.764 INFO:tasks.workunit.client.0.smithi100.stdout: 548: throughput=4.99MB/sec pending data=0 2024-04-07T16:48:02.764 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-IsNJe0bBMwmEuYY off=6907444 len=448021 2024-04-07T16:48:02.767 INFO:tasks.workunit.client.0.smithi100.stdout:op 2880 completed, throughput=4.99MB/sec 2024-04-07T16:48:02.767 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-ICObNPy6GhGFtje off=12153939 len=1353832 2024-04-07T16:48:02.777 INFO:tasks.workunit.client.0.smithi100.stdout:op 2881 completed, throughput=5MB/sec 2024-04-07T16:48:02.778 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-wV0Jhc7bIC92DeG off=4222298 len=1165674 2024-04-07T16:48:02.781 INFO:tasks.workunit.client.0.smithi100.stdout:op 2882 completed, throughput=5MB/sec 2024-04-07T16:48:02.781 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Wxoc0OrSBFCTSkN off=507482 len=1815191 2024-04-07T16:48:02.794 INFO:tasks.workunit.client.0.smithi100.stdout:op 2883 completed, throughput=5MB/sec 2024-04-07T16:48:02.860 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:48:02 smithi144 bash[66654]: cluster 2024-04-07T16:48:01.365125+0000 mgr.y (mgr.25175) 980 : cluster [DBG] pgmap v1622: 105 pgs: 105 active+clean; 126 GiB data, 7.0 GiB used, 708 GiB / 715 GiB avail; 2.6 MiB/s rd, 2.6 MiB/s wr, 5 op/s 2024-04-07T16:48:02.860 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:48:02 smithi144 bash[66654]: audit 2024-04-07T16:48:01.771525+0000 mon.a (mon.0) 2125 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:48:02.945 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:48:02 smithi100 bash[74942]: cluster 2024-04-07T16:48:01.365125+0000 mgr.y (mgr.25175) 980 : cluster [DBG] pgmap v1622: 105 pgs: 105 active+clean; 126 GiB data, 7.0 GiB used, 708 GiB / 715 GiB avail; 2.6 MiB/s rd, 2.6 MiB/s wr, 5 op/s 2024-04-07T16:48:02.945 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:48:02 smithi100 bash[74942]: audit 2024-04-07T16:48:01.771525+0000 mon.a (mon.0) 2125 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:48:02.945 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:48:02 smithi100 bash[76202]: cluster 2024-04-07T16:48:01.365125+0000 mgr.y (mgr.25175) 980 : cluster [DBG] pgmap v1622: 105 pgs: 105 active+clean; 126 GiB data, 7.0 GiB used, 708 GiB / 715 GiB avail; 2.6 MiB/s rd, 2.6 MiB/s wr, 5 op/s 2024-04-07T16:48:02.945 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:48:02 smithi100 bash[76202]: audit 2024-04-07T16:48:01.771525+0000 mon.a (mon.0) 2125 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:48:03.794 INFO:tasks.workunit.client.0.smithi100.stdout: 549: throughput=4.99MB/sec pending data=0 2024-04-07T16:48:03.795 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-ILxCq7wk_k4flQi off=9078876 len=856444 2024-04-07T16:48:03.802 INFO:tasks.workunit.client.0.smithi100.stdout:op 2884 completed, throughput=4.99MB/sec 2024-04-07T16:48:03.802 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-x5PyVGMokuVHnIz off=10270533 len=236959 2024-04-07T16:48:03.804 INFO:tasks.workunit.client.0.smithi100.stdout:op 2885 completed, throughput=4.99MB/sec 2024-04-07T16:48:03.804 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-ydDALtG5joUBkq6 off=6723752 len=772966 2024-04-07T16:48:03.810 INFO:tasks.workunit.client.0.smithi100.stdout:op 2886 completed, throughput=4.99MB/sec 2024-04-07T16:48:03.810 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-pkHHetBgTO2cDKh off=5129376 len=1132710 2024-04-07T16:48:03.813 INFO:tasks.workunit.client.0.smithi100.stdout:op 2887 completed, throughput=5MB/sec 2024-04-07T16:48:03.814 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-jXTugkeKi6LPS9c off=6234695 len=508156 2024-04-07T16:48:03.820 INFO:tasks.workunit.client.0.smithi100.stdout:op 2888 completed, throughput=5MB/sec 2024-04-07T16:48:03.820 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-c3xCoTxTC5BurEa off=10058596 len=2022499 2024-04-07T16:48:03.825 INFO:tasks.workunit.client.0.smithi100.stdout:op 2889 completed, throughput=5MB/sec 2024-04-07T16:48:04.825 INFO:tasks.workunit.client.0.smithi100.stdout: 550: throughput=4.99MB/sec pending data=0 2024-04-07T16:48:04.825 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-4d80HOz-PMgzP68 off=4461824 len=769157 2024-04-07T16:48:04.828 INFO:tasks.workunit.client.0.smithi100.stdout:op 2890 completed, throughput=4.99MB/sec 2024-04-07T16:48:04.828 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-hACe-NRJGT5xOKg off=774045 len=1734344 2024-04-07T16:48:04.840 INFO:tasks.workunit.client.0.smithi100.stdout:op 2891 completed, throughput=5MB/sec 2024-04-07T16:48:04.840 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-kdzdhGL_3lYlk-e off=9891465 len=1377668 2024-04-07T16:48:04.842 INFO:tasks.workunit.client.0.smithi100.stdout:op 2892 completed, throughput=5MB/sec 2024-04-07T16:48:04.842 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-uD6EpyVuCNYOJk3 off=2152195 len=1746438 2024-04-07T16:48:04.848 INFO:tasks.workunit.client.0.smithi100.stdout:op 2893 completed, throughput=5MB/sec 2024-04-07T16:48:04.860 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:48:04 smithi144 bash[66654]: cluster 2024-04-07T16:48:03.366225+0000 mgr.y (mgr.25175) 981 : cluster [DBG] pgmap v1623: 105 pgs: 105 active+clean; 126 GiB data, 7.0 GiB used, 708 GiB / 715 GiB avail; 3.0 MiB/s rd, 2.2 MiB/s wr, 5 op/s 2024-04-07T16:48:04.945 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:48:04 smithi100 bash[74942]: cluster 2024-04-07T16:48:03.366225+0000 mgr.y (mgr.25175) 981 : cluster [DBG] pgmap v1623: 105 pgs: 105 active+clean; 126 GiB data, 7.0 GiB used, 708 GiB / 715 GiB avail; 3.0 MiB/s rd, 2.2 MiB/s wr, 5 op/s 2024-04-07T16:48:04.979 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:48:04 smithi100 bash[76202]: cluster 2024-04-07T16:48:03.366225+0000 mgr.y (mgr.25175) 981 : cluster [DBG] pgmap v1623: 105 pgs: 105 active+clean; 126 GiB data, 7.0 GiB used, 708 GiB / 715 GiB avail; 3.0 MiB/s rd, 2.2 MiB/s wr, 5 op/s 2024-04-07T16:48:05.848 INFO:tasks.workunit.client.0.smithi100.stdout: 551: throughput=4.99MB/sec pending data=0 2024-04-07T16:48:05.848 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-VhKozujDzwedhWz off=995621 len=376614 2024-04-07T16:48:05.854 INFO:tasks.workunit.client.0.smithi100.stdout:op 2894 completed, throughput=4.99MB/sec 2024-04-07T16:48:05.854 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj--M-lzjpaFCGjJyj off=4928793 len=1281540 2024-04-07T16:48:05.865 INFO:tasks.workunit.client.0.smithi100.stdout:op 2895 completed, throughput=4.99MB/sec 2024-04-07T16:48:05.865 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-FAZ_rvWCNaGFqaT off=12871274 len=501228 2024-04-07T16:48:05.867 INFO:tasks.workunit.client.0.smithi100.stdout:op 2896 completed, throughput=5MB/sec 2024-04-07T16:48:05.867 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-n5CDz4pCNw9QF2U off=0 len=1637329 2024-04-07T16:48:05.872 INFO:tasks.workunit.client.0.smithi100.stdout:op 2897 completed, throughput=5MB/sec 2024-04-07T16:48:05.872 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-J-oMtRDrgGM7IVX off=5992735 len=982631 2024-04-07T16:48:05.881 INFO:tasks.workunit.client.0.smithi100.stdout:op 2898 completed, throughput=5MB/sec 2024-04-07T16:48:05.881 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-qDPHoVilzNDQRb7 off=11795206 len=1504 2024-04-07T16:48:05.885 INFO:tasks.workunit.client.0.smithi100.stdout:op 2899 completed, throughput=5MB/sec 2024-04-07T16:48:05.885 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-kQPIOhpjFd0XlY9 off=2595029 len=569917 2024-04-07T16:48:05.890 INFO:tasks.workunit.client.0.smithi100.stdout:op 2900 completed, throughput=5MB/sec 2024-04-07T16:48:06.891 INFO:tasks.workunit.client.0.smithi100.stdout: 552: throughput=4.99MB/sec pending data=0 2024-04-07T16:48:06.891 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-5qZSvg4LQj7XTgw off=1054656 len=1360620 2024-04-07T16:48:06.894 INFO:tasks.workunit.client.0.smithi100.stdout:op 2901 completed, throughput=4.99MB/sec 2024-04-07T16:48:06.895 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Xv5oRbBYdCmSOD3 off=241431 len=582518 2024-04-07T16:48:06.901 INFO:tasks.workunit.client.0.smithi100.stdout:op 2902 completed, throughput=4.99MB/sec 2024-04-07T16:48:06.901 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-4tGAh1C3nzTDJha off=4802326 len=1962783 2024-04-07T16:48:06.908 INFO:tasks.workunit.client.0.smithi100.stdout:op 2903 completed, throughput=5MB/sec 2024-04-07T16:48:06.908 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-VIoHMsTtdfx6P65 off=8722525 len=2094486 2024-04-07T16:48:06.913 INFO:tasks.workunit.client.0.smithi100.stdout:op 2904 completed, throughput=5MB/sec 2024-04-07T16:48:06.945 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:48:06 smithi100 bash[76202]: cluster 2024-04-07T16:48:05.367163+0000 mgr.y (mgr.25175) 982 : cluster [DBG] pgmap v1624: 105 pgs: 105 active+clean; 126 GiB data, 7.1 GiB used, 708 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.4 MiB/s wr, 4 op/s 2024-04-07T16:48:06.945 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:48:06 smithi100 bash[74942]: cluster 2024-04-07T16:48:05.367163+0000 mgr.y (mgr.25175) 982 : cluster [DBG] pgmap v1624: 105 pgs: 105 active+clean; 126 GiB data, 7.1 GiB used, 708 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.4 MiB/s wr, 4 op/s 2024-04-07T16:48:07.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:48:06 smithi144 bash[66654]: cluster 2024-04-07T16:48:05.367163+0000 mgr.y (mgr.25175) 982 : cluster [DBG] pgmap v1624: 105 pgs: 105 active+clean; 126 GiB data, 7.1 GiB used, 708 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.4 MiB/s wr, 4 op/s 2024-04-07T16:48:07.913 INFO:tasks.workunit.client.0.smithi100.stdout: 553: throughput=4.99MB/sec pending data=0 2024-04-07T16:48:07.913 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-uWipYCRVwS0w6zv off=21990541 len=1822650 2024-04-07T16:48:07.917 INFO:tasks.workunit.client.0.smithi100.stdout:op 2905 completed, throughput=5MB/sec 2024-04-07T16:48:07.917 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-_MvAR6AUZ0EdtGW off=20699523 len=1820153 2024-04-07T16:48:07.922 INFO:tasks.workunit.client.0.smithi100.stdout:op 2906 completed, throughput=5MB/sec 2024-04-07T16:48:07.922 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-NtjJfSfobP7UWjy off=17669849 len=1874055 2024-04-07T16:48:07.928 INFO:tasks.workunit.client.0.smithi100.stdout:op 2907 completed, throughput=5MB/sec 2024-04-07T16:48:08.695 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:48:08 smithi100 bash[74942]: cluster 2024-04-07T16:48:07.368174+0000 mgr.y (mgr.25175) 983 : cluster [DBG] pgmap v1625: 105 pgs: 105 active+clean; 126 GiB data, 7.1 GiB used, 708 GiB / 715 GiB avail; 2.8 MiB/s rd, 2.2 MiB/s wr, 5 op/s 2024-04-07T16:48:08.696 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:48:08 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:48:08] "GET /metrics HTTP/1.1" 200 34069 "" "Prometheus/2.43.0" 2024-04-07T16:48:08.696 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:48:08 smithi100 bash[76202]: cluster 2024-04-07T16:48:07.368174+0000 mgr.y (mgr.25175) 983 : cluster [DBG] pgmap v1625: 105 pgs: 105 active+clean; 126 GiB data, 7.1 GiB used, 708 GiB / 715 GiB avail; 2.8 MiB/s rd, 2.2 MiB/s wr, 5 op/s 2024-04-07T16:48:08.928 INFO:tasks.workunit.client.0.smithi100.stdout: 554: throughput=4.99MB/sec pending data=0 2024-04-07T16:48:08.928 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-HuktFe9q9wnGf-h off=13532819 len=1541265 2024-04-07T16:48:08.942 INFO:tasks.workunit.client.0.smithi100.stdout:op 2908 completed, throughput=5MB/sec 2024-04-07T16:48:08.942 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-xRCTzPKR8RNUeWI off=7850521 len=1667264 2024-04-07T16:48:08.948 INFO:tasks.workunit.client.0.smithi100.stdout:op 2909 completed, throughput=5MB/sec 2024-04-07T16:48:08.948 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-r7VofV3s3zHI_ph off=11631310 len=1370547 2024-04-07T16:48:08.959 INFO:tasks.workunit.client.0.smithi100.stdout:op 2910 completed, throughput=5MB/sec 2024-04-07T16:48:09.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:48:08 smithi144 bash[66654]: cluster 2024-04-07T16:48:07.368174+0000 mgr.y (mgr.25175) 983 : cluster [DBG] pgmap v1625: 105 pgs: 105 active+clean; 126 GiB data, 7.1 GiB used, 708 GiB / 715 GiB avail; 2.8 MiB/s rd, 2.2 MiB/s wr, 5 op/s 2024-04-07T16:48:09.960 INFO:tasks.workunit.client.0.smithi100.stdout: 555: throughput=4.99MB/sec pending data=0 2024-04-07T16:48:09.960 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-s_4zAKE77e2O7lT off=814572 len=1035475 2024-04-07T16:48:09.970 INFO:tasks.workunit.client.0.smithi100.stdout:op 2911 completed, throughput=4.99MB/sec 2024-04-07T16:48:09.970 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-zttApF9MP80x9aM off=11697300 len=680138 2024-04-07T16:48:09.973 INFO:tasks.workunit.client.0.smithi100.stdout:op 2912 completed, throughput=4.99MB/sec 2024-04-07T16:48:09.973 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-BIYXgg5akkX5t25 off=12529832 len=57263 2024-04-07T16:48:09.977 INFO:tasks.workunit.client.0.smithi100.stdout:op 2913 completed, throughput=4.99MB/sec 2024-04-07T16:48:09.978 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-VHHHPsPjdwsxvvo off=12799083 len=1353732 2024-04-07T16:48:09.981 INFO:tasks.workunit.client.0.smithi100.stdout:op 2914 completed, throughput=5MB/sec 2024-04-07T16:48:09.981 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-xGn5OlccYsJOvnV off=13075149 len=1660527 2024-04-07T16:48:09.985 INFO:tasks.workunit.client.0.smithi100.stdout:op 2915 completed, throughput=5MB/sec 2024-04-07T16:48:09.985 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Gamq4jvS8G1x7Ml off=5818339 len=1189768 2024-04-07T16:48:09.995 INFO:tasks.workunit.client.0.smithi100.stdout:op 2916 completed, throughput=5MB/sec 2024-04-07T16:48:10.945 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:48:10 smithi100 bash[74942]: cluster 2024-04-07T16:48:09.369015+0000 mgr.y (mgr.25175) 984 : cluster [DBG] pgmap v1626: 105 pgs: 105 active+clean; 126 GiB data, 7.1 GiB used, 708 GiB / 715 GiB avail; 2.8 MiB/s rd, 2.0 MiB/s wr, 4 op/s 2024-04-07T16:48:10.945 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:48:10 smithi100 bash[76202]: cluster 2024-04-07T16:48:09.369015+0000 mgr.y (mgr.25175) 984 : cluster [DBG] pgmap v1626: 105 pgs: 105 active+clean; 126 GiB data, 7.1 GiB used, 708 GiB / 715 GiB avail; 2.8 MiB/s rd, 2.0 MiB/s wr, 4 op/s 2024-04-07T16:48:10.995 INFO:tasks.workunit.client.0.smithi100.stdout: 556: throughput=4.99MB/sec pending data=0 2024-04-07T16:48:10.995 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-r7VofV3s3zHI_ph off=12317820 len=2028758 2024-04-07T16:48:11.013 INFO:tasks.workunit.client.0.smithi100.stdout:op 2917 completed, throughput=5MB/sec 2024-04-07T16:48:11.013 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-6giYRyFVr2rc0Ir off=6723421 len=684722 2024-04-07T16:48:11.015 INFO:tasks.workunit.client.0.smithi100.stdout:op 2918 completed, throughput=5MB/sec 2024-04-07T16:48:11.015 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-P0Qq6qLvYWDmQgP off=1566180 len=1861728 2024-04-07T16:48:11.020 INFO:tasks.workunit.client.0.smithi100.stdout:op 2919 completed, throughput=5MB/sec 2024-04-07T16:48:11.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:48:10 smithi144 bash[66654]: cluster 2024-04-07T16:48:09.369015+0000 mgr.y (mgr.25175) 984 : cluster [DBG] pgmap v1626: 105 pgs: 105 active+clean; 126 GiB data, 7.1 GiB used, 708 GiB / 715 GiB avail; 2.8 MiB/s rd, 2.0 MiB/s wr, 4 op/s 2024-04-07T16:48:12.021 INFO:tasks.workunit.client.0.smithi100.stdout: 557: throughput=4.99MB/sec pending data=0 2024-04-07T16:48:12.021 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-_I5KmPnbUQ06ceO off=10008555 len=347601 2024-04-07T16:48:12.027 INFO:tasks.workunit.client.0.smithi100.stdout:op 2920 completed, throughput=4.99MB/sec 2024-04-07T16:48:12.027 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Sfb6mKaXjI0H0a9 off=887655 len=1089494 2024-04-07T16:48:12.030 INFO:tasks.workunit.client.0.smithi100.stdout:op 2921 completed, throughput=4.99MB/sec 2024-04-07T16:48:12.030 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-SOhvZ20oK4-A9X9 off=5959671 len=1399323 2024-04-07T16:48:12.034 INFO:tasks.workunit.client.0.smithi100.stdout:op 2922 completed, throughput=5MB/sec 2024-04-07T16:48:12.034 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-7MvTn4JuJoCqwko off=8267148 len=1426560 2024-04-07T16:48:12.044 INFO:tasks.workunit.client.0.smithi100.stdout:op 2923 completed, throughput=5MB/sec 2024-04-07T16:48:12.044 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-itdRHxoPH_3PXVn off=7733759 len=577594 2024-04-07T16:48:12.050 INFO:tasks.workunit.client.0.smithi100.stdout:op 2924 completed, throughput=5MB/sec 2024-04-07T16:48:12.050 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-gDLkYJy7sAlcKoM off=7058303 len=1856535 2024-04-07T16:48:12.064 INFO:tasks.workunit.client.0.smithi100.stdout:op 2925 completed, throughput=5MB/sec 2024-04-07T16:48:12.610 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:48:12 smithi144 bash[95233]: ts=2024-04-07T16:48:12.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:48:12.945 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:48:12 smithi100 bash[74942]: cluster 2024-04-07T16:48:11.369930+0000 mgr.y (mgr.25175) 985 : cluster [DBG] pgmap v1627: 105 pgs: 105 active+clean; 126 GiB data, 7.1 GiB used, 708 GiB / 715 GiB avail; 2.9 MiB/s rd, 1.8 MiB/s wr, 4 op/s 2024-04-07T16:48:12.946 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:48:12 smithi100 bash[76202]: cluster 2024-04-07T16:48:11.369930+0000 mgr.y (mgr.25175) 985 : cluster [DBG] pgmap v1627: 105 pgs: 105 active+clean; 126 GiB data, 7.1 GiB used, 708 GiB / 715 GiB avail; 2.9 MiB/s rd, 1.8 MiB/s wr, 4 op/s 2024-04-07T16:48:13.064 INFO:tasks.workunit.client.0.smithi100.stdout: 558: throughput=4.99MB/sec pending data=0 2024-04-07T16:48:13.064 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-CShJgUzaMlWBMpT off=4892419 len=1509874 2024-04-07T16:48:13.076 INFO:tasks.workunit.client.0.smithi100.stdout:op 2926 completed, throughput=5MB/sec 2024-04-07T16:48:13.076 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-wu9WiFyyIgT_ARQ off=1843999 len=137800 2024-04-07T16:48:13.078 INFO:tasks.workunit.client.0.smithi100.stdout:op 2927 completed, throughput=5MB/sec 2024-04-07T16:48:13.078 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-4lg7Px6BynxByAY off=8875181 len=1786284 2024-04-07T16:48:13.091 INFO:tasks.workunit.client.0.smithi100.stdout:op 2928 completed, throughput=5MB/sec 2024-04-07T16:48:13.091 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-4JYwYh0eTvBV5S2 off=1702398 len=263464 2024-04-07T16:48:13.096 INFO:tasks.workunit.client.0.smithi100.stdout:op 2929 completed, throughput=5MB/sec 2024-04-07T16:48:13.096 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Cg_gEluCq31j3TQ off=10172806 len=631379 2024-04-07T16:48:13.103 INFO:tasks.workunit.client.0.smithi100.stdout:op 2930 completed, throughput=5MB/sec 2024-04-07T16:48:13.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:48:12 smithi144 bash[66654]: cluster 2024-04-07T16:48:11.369930+0000 mgr.y (mgr.25175) 985 : cluster [DBG] pgmap v1627: 105 pgs: 105 active+clean; 126 GiB data, 7.1 GiB used, 708 GiB / 715 GiB avail; 2.9 MiB/s rd, 1.8 MiB/s wr, 4 op/s 2024-04-07T16:48:13.945 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:48:13 smithi100 bash[74942]: cluster 2024-04-07T16:48:13.371034+0000 mgr.y (mgr.25175) 986 : cluster [DBG] pgmap v1628: 105 pgs: 105 active+clean; 126 GiB data, 7.1 GiB used, 708 GiB / 715 GiB avail; 3.3 MiB/s rd, 2.1 MiB/s wr, 4 op/s 2024-04-07T16:48:13.946 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:48:13 smithi100 bash[76202]: cluster 2024-04-07T16:48:13.371034+0000 mgr.y (mgr.25175) 986 : cluster [DBG] pgmap v1628: 105 pgs: 105 active+clean; 126 GiB data, 7.1 GiB used, 708 GiB / 715 GiB avail; 3.3 MiB/s rd, 2.1 MiB/s wr, 4 op/s 2024-04-07T16:48:14.104 INFO:tasks.workunit.client.0.smithi100.stdout: 559: throughput=4.99MB/sec pending data=0 2024-04-07T16:48:14.104 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-sqc5xnFR7WX4r55 off=5355888 len=1819507 2024-04-07T16:48:14.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:48:13 smithi144 bash[66654]: cluster 2024-04-07T16:48:13.371034+0000 mgr.y (mgr.25175) 986 : cluster [DBG] pgmap v1628: 105 pgs: 105 active+clean; 126 GiB data, 7.1 GiB used, 708 GiB / 715 GiB avail; 3.3 MiB/s rd, 2.1 MiB/s wr, 4 op/s 2024-04-07T16:48:14.126 INFO:tasks.workunit.client.0.smithi100.stdout:op 2931 completed, throughput=4.99MB/sec 2024-04-07T16:48:14.127 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-hur7L-jxRGaPkfV off=14564710 len=897149 2024-04-07T16:48:14.136 INFO:tasks.workunit.client.0.smithi100.stdout:op 2932 completed, throughput=5MB/sec 2024-04-07T16:48:14.136 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-zMXIp70a6ias24Y off=207254 len=526723 2024-04-07T16:48:14.142 INFO:tasks.workunit.client.0.smithi100.stdout:op 2933 completed, throughput=5MB/sec 2024-04-07T16:48:14.142 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-quBEHQmUD7Zy9Lt off=14489404 len=1964894 2024-04-07T16:48:14.158 INFO:tasks.workunit.client.0.smithi100.stdout:op 2934 completed, throughput=5MB/sec 2024-04-07T16:48:14.158 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-8hVknmL199xaehk off=6223189 len=495078 2024-04-07T16:48:14.164 INFO:tasks.workunit.client.0.smithi100.stdout:op 2935 completed, throughput=5MB/sec 2024-04-07T16:48:15.165 INFO:tasks.workunit.client.0.smithi100.stdout: 560: throughput=4.99MB/sec pending data=0 2024-04-07T16:48:15.165 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-PoLfU7aBQz2KUYD off=428729 len=160135 2024-04-07T16:48:15.169 INFO:tasks.workunit.client.0.smithi100.stdout:op 2936 completed, throughput=4.99MB/sec 2024-04-07T16:48:15.170 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-dX9DBn9gac-KY01 off=1564015 len=2009412 2024-04-07T16:48:15.174 INFO:tasks.workunit.client.0.smithi100.stdout:op 2937 completed, throughput=5MB/sec 2024-04-07T16:48:15.174 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-O77jgCgtl5nKYA4 off=17567356 len=1516449 2024-04-07T16:48:15.178 INFO:tasks.workunit.client.0.smithi100.stdout:op 2938 completed, throughput=5MB/sec 2024-04-07T16:48:15.178 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-MnWeiqG1vRnTESD off=1961577 len=878052 2024-04-07T16:48:15.187 INFO:tasks.workunit.client.0.smithi100.stdout:op 2939 completed, throughput=5MB/sec 2024-04-07T16:48:15.187 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-eNvh3g1N9SblQIw off=4973962 len=601757 2024-04-07T16:48:15.194 INFO:tasks.workunit.client.0.smithi100.stdout:op 2940 completed, throughput=5MB/sec 2024-04-07T16:48:16.194 INFO:tasks.workunit.client.0.smithi100.stdout: 561: throughput=4.99MB/sec pending data=0 2024-04-07T16:48:16.194 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-orkcdbbuMtd8HVE off=2603252 len=856417 2024-04-07T16:48:16.202 INFO:tasks.workunit.client.0.smithi100.stdout:op 2941 completed, throughput=4.99MB/sec 2024-04-07T16:48:16.203 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-gbhjBI6GJurV--D off=10473719 len=1422444 2024-04-07T16:48:16.214 INFO:tasks.workunit.client.0.smithi100.stdout:op 2942 completed, throughput=5MB/sec 2024-04-07T16:48:16.214 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-OevcOF5NKzLq5t3 off=911773 len=2036819 2024-04-07T16:48:16.219 INFO:tasks.workunit.client.0.smithi100.stdout:op 2943 completed, throughput=5MB/sec 2024-04-07T16:48:16.219 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-WIZGiTMNnCaB8ae off=9243545 len=544619 2024-04-07T16:48:16.221 INFO:tasks.workunit.client.0.smithi100.stdout:op 2944 completed, throughput=5MB/sec 2024-04-07T16:48:16.221 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-BysrNEvEj4_fVxt off=5626981 len=86418 2024-04-07T16:48:16.226 INFO:tasks.workunit.client.0.smithi100.stdout:op 2945 completed, throughput=5MB/sec 2024-04-07T16:48:16.226 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-RipY_gVAzXRZNYK off=2955987 len=1156167 2024-04-07T16:48:16.230 INFO:tasks.workunit.client.0.smithi100.stdout:op 2946 completed, throughput=5MB/sec 2024-04-07T16:48:16.695 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:48:16 smithi100 bash[74942]: cluster 2024-04-07T16:48:15.372039+0000 mgr.y (mgr.25175) 987 : cluster [DBG] pgmap v1629: 105 pgs: 105 active+clean; 126 GiB data, 7.1 GiB used, 708 GiB / 715 GiB avail; 2.6 MiB/s rd, 2.5 MiB/s wr, 4 op/s 2024-04-07T16:48:16.696 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:48:16 smithi100 bash[76202]: cluster 2024-04-07T16:48:15.372039+0000 mgr.y (mgr.25175) 987 : cluster [DBG] pgmap v1629: 105 pgs: 105 active+clean; 126 GiB data, 7.1 GiB used, 708 GiB / 715 GiB avail; 2.6 MiB/s rd, 2.5 MiB/s wr, 4 op/s 2024-04-07T16:48:16.860 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:48:16 smithi144 bash[66654]: cluster 2024-04-07T16:48:15.372039+0000 mgr.y (mgr.25175) 987 : cluster [DBG] pgmap v1629: 105 pgs: 105 active+clean; 126 GiB data, 7.1 GiB used, 708 GiB / 715 GiB avail; 2.6 MiB/s rd, 2.5 MiB/s wr, 4 op/s 2024-04-07T16:48:17.230 INFO:tasks.workunit.client.0.smithi100.stdout: 562: throughput=4.99MB/sec pending data=0 2024-04-07T16:48:17.231 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-rjhSaFaKlUxsTOQ off=903991 len=798378 2024-04-07T16:48:17.234 INFO:tasks.workunit.client.0.smithi100.stdout:op 2947 completed, throughput=4.99MB/sec 2024-04-07T16:48:17.234 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-dV3kMsq3lPTeTET off=2757758 len=613557 2024-04-07T16:48:17.237 INFO:tasks.workunit.client.0.smithi100.stdout:op 2948 completed, throughput=4.99MB/sec 2024-04-07T16:48:17.237 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-w3C3ue6VWPXu92h off=13213657 len=1593549 2024-04-07T16:48:17.241 INFO:tasks.workunit.client.0.smithi100.stdout:op 2949 completed, throughput=5MB/sec 2024-04-07T16:48:17.241 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-nIWxKdnopZFG7YG off=14029383 len=1299587 2024-04-07T16:48:17.244 INFO:tasks.workunit.client.0.smithi100.stdout:op 2950 completed, throughput=5MB/sec 2024-04-07T16:48:17.245 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Wb3vALUGyaR3vCH off=2860533 len=1646818 2024-04-07T16:48:17.249 INFO:tasks.workunit.client.0.smithi100.stdout:op 2951 completed, throughput=5MB/sec 2024-04-07T16:48:17.695 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:48:17 smithi100 bash[76202]: audit 2024-04-07T16:48:16.771906+0000 mon.a (mon.0) 2126 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:48:17.696 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:48:17 smithi100 bash[74942]: audit 2024-04-07T16:48:16.771906+0000 mon.a (mon.0) 2126 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:48:17.860 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:48:17 smithi144 bash[66654]: audit 2024-04-07T16:48:16.771906+0000 mon.a (mon.0) 2126 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:48:18.249 INFO:tasks.workunit.client.0.smithi100.stdout: 563: throughput=4.99MB/sec pending data=0 2024-04-07T16:48:18.249 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-zni_uyBxuK7iwcr off=5918009 len=239548 2024-04-07T16:48:18.251 INFO:tasks.workunit.client.0.smithi100.stdout:op 2952 completed, throughput=4.99MB/sec 2024-04-07T16:48:18.251 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-86qnoohk0Df__D- off=16219153 len=1840841 2024-04-07T16:48:18.255 INFO:tasks.workunit.client.0.smithi100.stdout:op 2953 completed, throughput=5MB/sec 2024-04-07T16:48:18.255 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-97FqjuSvnQcgxQT off=552590 len=740963 2024-04-07T16:48:18.263 INFO:tasks.workunit.client.0.smithi100.stdout:op 2954 completed, throughput=5MB/sec 2024-04-07T16:48:18.264 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-zuldoMszqspdmxE off=6679477 len=1761949 2024-04-07T16:48:18.266 INFO:tasks.workunit.client.0.smithi100.stdout:op 2955 completed, throughput=5MB/sec 2024-04-07T16:48:18.695 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:48:18 smithi100 bash[74942]: cluster 2024-04-07T16:48:17.373067+0000 mgr.y (mgr.25175) 988 : cluster [DBG] pgmap v1630: 105 pgs: 105 active+clean; 126 GiB data, 7.1 GiB used, 708 GiB / 715 GiB avail; 2.4 MiB/s rd, 2.1 MiB/s wr, 4 op/s 2024-04-07T16:48:18.696 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:48:18 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:48:18] "GET /metrics HTTP/1.1" 200 34066 "" "Prometheus/2.43.0" 2024-04-07T16:48:18.696 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:48:18 smithi100 bash[76202]: cluster 2024-04-07T16:48:17.373067+0000 mgr.y (mgr.25175) 988 : cluster [DBG] pgmap v1630: 105 pgs: 105 active+clean; 126 GiB data, 7.1 GiB used, 708 GiB / 715 GiB avail; 2.4 MiB/s rd, 2.1 MiB/s wr, 4 op/s 2024-04-07T16:48:18.860 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:48:18 smithi144 bash[66654]: cluster 2024-04-07T16:48:17.373067+0000 mgr.y (mgr.25175) 988 : cluster [DBG] pgmap v1630: 105 pgs: 105 active+clean; 126 GiB data, 7.1 GiB used, 708 GiB / 715 GiB avail; 2.4 MiB/s rd, 2.1 MiB/s wr, 4 op/s 2024-04-07T16:48:18.860 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:48:18 smithi144 bash[98866]: logger=cleanup t=2024-04-07T16:48:18.4750787Z level=info msg="Completed cleanup jobs" duration=56.030913ms 2024-04-07T16:48:19.266 INFO:tasks.workunit.client.0.smithi100.stdout: 564: throughput=4.99MB/sec pending data=0 2024-04-07T16:48:19.266 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-GXupI6hilNwMdaD off=10033321 len=1447542 2024-04-07T16:48:19.280 INFO:tasks.workunit.client.0.smithi100.stdout:op 2956 completed, throughput=4.99MB/sec 2024-04-07T16:48:19.280 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-xvfPms1NkqqooVj off=8798460 len=634826 2024-04-07T16:48:19.287 INFO:tasks.workunit.client.0.smithi100.stdout:op 2957 completed, throughput=5MB/sec 2024-04-07T16:48:19.287 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-qE4fw2YSNrraXoc off=206440 len=623303 2024-04-07T16:48:19.294 INFO:tasks.workunit.client.0.smithi100.stdout:op 2958 completed, throughput=5MB/sec 2024-04-07T16:48:19.294 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-FcwFDFEcguiSOy1 off=6748203 len=849180 2024-04-07T16:48:19.303 INFO:tasks.workunit.client.0.smithi100.stdout:op 2959 completed, throughput=5MB/sec 2024-04-07T16:48:19.303 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-7ise0ZBLJ2Immbi off=20713062 len=1296214 2024-04-07T16:48:19.306 INFO:tasks.workunit.client.0.smithi100.stdout:op 2960 completed, throughput=5MB/sec 2024-04-07T16:48:20.307 INFO:tasks.workunit.client.0.smithi100.stdout: 565: throughput=4.99MB/sec pending data=0 2024-04-07T16:48:20.307 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-QMqlN3NX4FTgATC off=15439823 len=1281955 2024-04-07T16:48:20.310 INFO:tasks.workunit.client.0.smithi100.stdout:op 2961 completed, throughput=4.99MB/sec 2024-04-07T16:48:20.310 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-9WkN50lZz_Fc3Vf off=9094019 len=1164926 2024-04-07T16:48:20.320 INFO:tasks.workunit.client.0.smithi100.stdout:op 2962 completed, throughput=5MB/sec 2024-04-07T16:48:20.320 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-XLMyx5NZaHr1n2F off=11106181 len=887915 2024-04-07T16:48:20.328 INFO:tasks.workunit.client.0.smithi100.stdout:op 2963 completed, throughput=5MB/sec 2024-04-07T16:48:20.328 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-9JteWnIOPlI8j0h off=499409 len=811475 2024-04-07T16:48:20.330 INFO:tasks.workunit.client.0.smithi100.stdout:op 2964 completed, throughput=5MB/sec 2024-04-07T16:48:20.330 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-PcDYCxNdahEvtPG off=33934 len=2089694 2024-04-07T16:48:20.345 INFO:tasks.workunit.client.0.smithi100.stdout:op 2965 completed, throughput=5MB/sec 2024-04-07T16:48:20.695 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:48:20 smithi100 bash[74942]: cluster 2024-04-07T16:48:19.373899+0000 mgr.y (mgr.25175) 989 : cluster [DBG] pgmap v1631: 105 pgs: 105 active+clean; 126 GiB data, 7.1 GiB used, 708 GiB / 715 GiB avail; 2.2 MiB/s rd, 2.0 MiB/s wr, 3 op/s 2024-04-07T16:48:20.696 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:48:20 smithi100 bash[76202]: cluster 2024-04-07T16:48:19.373899+0000 mgr.y (mgr.25175) 989 : cluster [DBG] pgmap v1631: 105 pgs: 105 active+clean; 126 GiB data, 7.1 GiB used, 708 GiB / 715 GiB avail; 2.2 MiB/s rd, 2.0 MiB/s wr, 3 op/s 2024-04-07T16:48:20.860 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:48:20 smithi144 bash[66654]: cluster 2024-04-07T16:48:19.373899+0000 mgr.y (mgr.25175) 989 : cluster [DBG] pgmap v1631: 105 pgs: 105 active+clean; 126 GiB data, 7.1 GiB used, 708 GiB / 715 GiB avail; 2.2 MiB/s rd, 2.0 MiB/s wr, 3 op/s 2024-04-07T16:48:21.345 INFO:tasks.workunit.client.0.smithi100.stdout: 566: throughput=4.99MB/sec pending data=0 2024-04-07T16:48:21.345 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-A2Qtqh2LfZf9RMQ off=13510105 len=350271 2024-04-07T16:48:21.351 INFO:tasks.workunit.client.0.smithi100.stdout:op 2966 completed, throughput=4.99MB/sec 2024-04-07T16:48:21.351 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-voFw7NRNkaP-NGw off=4484602 len=2050705 2024-04-07T16:48:21.355 INFO:tasks.workunit.client.0.smithi100.stdout:op 2967 completed, throughput=5MB/sec 2024-04-07T16:48:21.355 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-pRkxg2kbv7Fr3RS off=16269728 len=751054 2024-04-07T16:48:21.359 INFO:tasks.workunit.client.0.smithi100.stdout:op 2968 completed, throughput=5MB/sec 2024-04-07T16:48:21.359 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-C3y6Ef_qzzTFfOO off=1616067 len=1320331 2024-04-07T16:48:21.371 INFO:tasks.workunit.client.0.smithi100.stdout:op 2969 completed, throughput=5MB/sec 2024-04-07T16:48:21.371 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-ICsWLOrZvesOPSb off=12262271 len=741066 2024-04-07T16:48:21.373 INFO:tasks.workunit.client.0.smithi100.stdout:op 2970 completed, throughput=5MB/sec 2024-04-07T16:48:22.374 INFO:tasks.workunit.client.0.smithi100.stdout: 567: throughput=4.99MB/sec pending data=0 2024-04-07T16:48:22.374 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-q8o8DcMVSaa16wh off=11286136 len=236040 2024-04-07T16:48:22.379 INFO:tasks.workunit.client.0.smithi100.stdout:op 2971 completed, throughput=4.99MB/sec 2024-04-07T16:48:22.379 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-fax4q4KCOZ59bYA off=17571312 len=747175 2024-04-07T16:48:22.382 INFO:tasks.workunit.client.0.smithi100.stdout:op 2972 completed, throughput=4.99MB/sec 2024-04-07T16:48:22.383 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-bYUEGopRFu3buKK off=11645444 len=1368441 2024-04-07T16:48:22.393 INFO:tasks.workunit.client.0.smithi100.stdout:op 2973 completed, throughput=5MB/sec 2024-04-07T16:48:22.393 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Mtx3afyTGAXpesA off=8084385 len=902971 2024-04-07T16:48:22.396 INFO:tasks.workunit.client.0.smithi100.stdout:op 2974 completed, throughput=5MB/sec 2024-04-07T16:48:22.396 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-90lSIp4WVvHfD3f off=3354455 len=469990 2024-04-07T16:48:22.402 INFO:tasks.workunit.client.0.smithi100.stdout:op 2975 completed, throughput=5MB/sec 2024-04-07T16:48:22.402 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-u1KK1ruFnNMOS5L off=2136946 len=580099 2024-04-07T16:48:22.409 INFO:tasks.workunit.client.0.smithi100.stdout:op 2976 completed, throughput=5MB/sec 2024-04-07T16:48:22.409 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-0aRWyBxQtGzz3NP off=196510 len=1549427 2024-04-07T16:48:22.419 INFO:tasks.workunit.client.0.smithi100.stdout:op 2977 completed, throughput=5MB/sec 2024-04-07T16:48:22.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:48:22 smithi144 bash[66654]: cluster 2024-04-07T16:48:21.374823+0000 mgr.y (mgr.25175) 990 : cluster [DBG] pgmap v1632: 105 pgs: 105 active+clean; 126 GiB data, 7.1 GiB used, 708 GiB / 715 GiB avail; 2.3 MiB/s rd, 2.1 MiB/s wr, 3 op/s 2024-04-07T16:48:22.610 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:48:22 smithi144 bash[95233]: ts=2024-04-07T16:48:22.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:48:22.695 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:48:22 smithi100 bash[74942]: cluster 2024-04-07T16:48:21.374823+0000 mgr.y (mgr.25175) 990 : cluster [DBG] pgmap v1632: 105 pgs: 105 active+clean; 126 GiB data, 7.1 GiB used, 708 GiB / 715 GiB avail; 2.3 MiB/s rd, 2.1 MiB/s wr, 3 op/s 2024-04-07T16:48:22.695 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:48:22 smithi100 bash[76202]: cluster 2024-04-07T16:48:21.374823+0000 mgr.y (mgr.25175) 990 : cluster [DBG] pgmap v1632: 105 pgs: 105 active+clean; 126 GiB data, 7.1 GiB used, 708 GiB / 715 GiB avail; 2.3 MiB/s rd, 2.1 MiB/s wr, 3 op/s 2024-04-07T16:48:23.420 INFO:tasks.workunit.client.0.smithi100.stdout: 568: throughput=4.99MB/sec pending data=0 2024-04-07T16:48:23.420 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-1QoYyBu0eZIsYUl off=2499492 len=811763 2024-04-07T16:48:23.423 INFO:tasks.workunit.client.0.smithi100.stdout:op 2978 completed, throughput=4.99MB/sec 2024-04-07T16:48:23.423 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-G_jDr-7Szw72wJy off=4517587 len=430042 2024-04-07T16:48:23.424 INFO:tasks.workunit.client.0.smithi100.stdout:op 2979 completed, throughput=5MB/sec 2024-04-07T16:48:23.425 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-FwnZ1tqsFmqgU43 off=17509929 len=1878172 2024-04-07T16:48:23.429 INFO:tasks.workunit.client.0.smithi100.stdout:op 2980 completed, throughput=5MB/sec 2024-04-07T16:48:23.429 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-dbQ24ufW9ySPqHa off=10740285 len=1745479 2024-04-07T16:48:23.442 INFO:tasks.workunit.client.0.smithi100.stdout:op 2981 completed, throughput=5MB/sec 2024-04-07T16:48:24.442 INFO:tasks.workunit.client.0.smithi100.stdout: 569: throughput=4.99MB/sec pending data=0 2024-04-07T16:48:24.442 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Q9BZnH2JNjNIMrL off=11879534 len=2056446 2024-04-07T16:48:24.448 INFO:tasks.workunit.client.0.smithi100.stdout:op 2982 completed, throughput=5MB/sec 2024-04-07T16:48:24.448 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-CShJgUzaMlWBMpT off=17470157 len=745306 2024-04-07T16:48:24.450 INFO:tasks.workunit.client.0.smithi100.stdout:op 2983 completed, throughput=5MB/sec 2024-04-07T16:48:24.450 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-GrcvDl9MtYeULqr off=6510810 len=1890084 2024-04-07T16:48:24.464 INFO:tasks.workunit.client.0.smithi100.stdout:op 2984 completed, throughput=5MB/sec 2024-04-07T16:48:24.465 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-W2f82e585HWmZE4 off=2980301 len=938749 2024-04-07T16:48:24.472 INFO:tasks.workunit.client.0.smithi100.stdout:op 2985 completed, throughput=5MB/sec 2024-04-07T16:48:24.860 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:48:24 smithi144 bash[66654]: cluster 2024-04-07T16:48:23.375986+0000 mgr.y (mgr.25175) 991 : cluster [DBG] pgmap v1633: 105 pgs: 105 active+clean; 126 GiB data, 7.2 GiB used, 708 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.7 MiB/s wr, 4 op/s 2024-04-07T16:48:24.945 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:48:24 smithi100 bash[74942]: cluster 2024-04-07T16:48:23.375986+0000 mgr.y (mgr.25175) 991 : cluster [DBG] pgmap v1633: 105 pgs: 105 active+clean; 126 GiB data, 7.2 GiB used, 708 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.7 MiB/s wr, 4 op/s 2024-04-07T16:48:24.946 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:48:24 smithi100 bash[76202]: cluster 2024-04-07T16:48:23.375986+0000 mgr.y (mgr.25175) 991 : cluster [DBG] pgmap v1633: 105 pgs: 105 active+clean; 126 GiB data, 7.2 GiB used, 708 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.7 MiB/s wr, 4 op/s 2024-04-07T16:48:25.473 INFO:tasks.workunit.client.0.smithi100.stdout: 570: throughput=4.99MB/sec pending data=0 2024-04-07T16:48:25.473 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-d52oSKLDFmDC4YB off=3628647 len=695715 2024-04-07T16:48:25.476 INFO:tasks.workunit.client.0.smithi100.stdout:op 2986 completed, throughput=4.99MB/sec 2024-04-07T16:48:25.476 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-SQb-4KONggjUnIT off=8513501 len=794935 2024-04-07T16:48:25.484 INFO:tasks.workunit.client.0.smithi100.stdout:op 2987 completed, throughput=5MB/sec 2024-04-07T16:48:25.484 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-hGZ4cDD4GNT4fv- off=6850091 len=1432451 2024-04-07T16:48:25.488 INFO:tasks.workunit.client.0.smithi100.stdout:op 2988 completed, throughput=5MB/sec 2024-04-07T16:48:25.488 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-odsilPeYYAqVs_x off=6925122 len=321771 2024-04-07T16:48:25.489 INFO:tasks.workunit.client.0.smithi100.stdout:op 2989 completed, throughput=5MB/sec 2024-04-07T16:48:25.489 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-G6Ey-FT66Yn3thc off=921510 len=560565 2024-04-07T16:48:25.496 INFO:tasks.workunit.client.0.smithi100.stdout:op 2990 completed, throughput=5MB/sec 2024-04-07T16:48:25.496 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-2NXqEBx1fMoeiIU off=2232994 len=1431875 2024-04-07T16:48:25.498 INFO:tasks.workunit.client.0.smithi100.stdout:op 2991 completed, throughput=5MB/sec 2024-04-07T16:48:26.499 INFO:tasks.workunit.client.0.smithi100.stdout: 571: throughput=4.99MB/sec pending data=0 2024-04-07T16:48:26.499 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-9tWTs3BE4kA2LsA off=11159182 len=1899871 2024-04-07T16:48:26.505 INFO:tasks.workunit.client.0.smithi100.stdout:op 2992 completed, throughput=5MB/sec 2024-04-07T16:48:26.505 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-ogiS0LoyPhKuqgA off=14670268 len=764245 2024-04-07T16:48:26.507 INFO:tasks.workunit.client.0.smithi100.stdout:op 2993 completed, throughput=5MB/sec 2024-04-07T16:48:26.507 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Fk9JruTJ-lWgjBx off=6942690 len=1647402 2024-04-07T16:48:26.511 INFO:tasks.workunit.client.0.smithi100.stdout:op 2994 completed, throughput=5MB/sec 2024-04-07T16:48:26.511 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-gqzfu2xgbAdRE-a off=9415608 len=218385 2024-04-07T16:48:26.513 INFO:tasks.workunit.client.0.smithi100.stdout:op 2995 completed, throughput=5MB/sec 2024-04-07T16:48:26.513 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-vwyBJ0JeRyu6GMU off=339851 len=40956 2024-04-07T16:48:26.517 INFO:tasks.workunit.client.0.smithi100.stdout:op 2996 completed, throughput=5MB/sec 2024-04-07T16:48:26.517 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-s_dnz5wbtcQ3zXt off=15415674 len=130249 2024-04-07T16:48:26.522 INFO:tasks.workunit.client.0.smithi100.stdout:op 2997 completed, throughput=5MB/sec 2024-04-07T16:48:26.522 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-LqFBqiUGPhOMuaK off=10202888 len=1496956 2024-04-07T16:48:26.525 INFO:tasks.workunit.client.0.smithi100.stdout:op 2998 completed, throughput=5MB/sec 2024-04-07T16:48:26.860 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:48:26 smithi144 bash[66654]: cluster 2024-04-07T16:48:25.376926+0000 mgr.y (mgr.25175) 992 : cluster [DBG] pgmap v1634: 105 pgs: 105 active+clean; 126 GiB data, 7.2 GiB used, 708 GiB / 715 GiB avail; 2.3 MiB/s rd, 3.0 MiB/s wr, 5 op/s 2024-04-07T16:48:26.945 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:48:26 smithi100 bash[74942]: cluster 2024-04-07T16:48:25.376926+0000 mgr.y (mgr.25175) 992 : cluster [DBG] pgmap v1634: 105 pgs: 105 active+clean; 126 GiB data, 7.2 GiB used, 708 GiB / 715 GiB avail; 2.3 MiB/s rd, 3.0 MiB/s wr, 5 op/s 2024-04-07T16:48:26.945 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:48:26 smithi100 bash[76202]: cluster 2024-04-07T16:48:25.376926+0000 mgr.y (mgr.25175) 992 : cluster [DBG] pgmap v1634: 105 pgs: 105 active+clean; 126 GiB data, 7.2 GiB used, 708 GiB / 715 GiB avail; 2.3 MiB/s rd, 3.0 MiB/s wr, 5 op/s 2024-04-07T16:48:27.525 INFO:tasks.workunit.client.0.smithi100.stdout: 572: throughput=4.99MB/sec pending data=0 2024-04-07T16:48:27.525 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-L3CmBMQ71ISkUWN off=17949082 len=1480990 2024-04-07T16:48:27.536 INFO:tasks.workunit.client.0.smithi100.stdout:op 2999 completed, throughput=5MB/sec 2024-04-07T16:48:27.536 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-5BzEsd-_L0oLMzQ off=15251456 len=1232423 2024-04-07T16:48:27.547 INFO:tasks.workunit.client.0.smithi100.stdout:op 3000 completed, throughput=5MB/sec 2024-04-07T16:48:27.547 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-0jpeDBDnlN-7qgx off=24014927 len=808082 2024-04-07T16:48:27.555 INFO:tasks.workunit.client.0.smithi100.stdout:op 3001 completed, throughput=5MB/sec 2024-04-07T16:48:27.555 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-n8xiYLWG9iMyb0W off=6453370 len=1133255 2024-04-07T16:48:27.563 INFO:tasks.workunit.client.0.smithi100.stdout:op 3002 completed, throughput=5MB/sec 2024-04-07T16:48:28.564 INFO:tasks.workunit.client.0.smithi100.stdout: 573: throughput=4.99MB/sec pending data=0 2024-04-07T16:48:28.564 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-zwonKCRP5muYCiD off=6757745 len=1338193 2024-04-07T16:48:28.575 INFO:tasks.workunit.client.0.smithi100.stdout:op 3003 completed, throughput=4.99MB/sec 2024-04-07T16:48:28.575 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-ykQI8AF8S6WZA4l off=4660313 len=595155 2024-04-07T16:48:28.577 INFO:tasks.workunit.client.0.smithi100.stdout:op 3004 completed, throughput=5MB/sec 2024-04-07T16:48:28.577 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-3mB--r2j9l4ggHT off=1304077 len=1708377 2024-04-07T16:48:28.581 INFO:tasks.workunit.client.0.smithi100.stdout:op 3005 completed, throughput=5MB/sec 2024-04-07T16:48:28.581 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-u_BipLJfARu6Gra off=9320330 len=304668 2024-04-07T16:48:28.586 INFO:tasks.workunit.client.0.smithi100.stdout:op 3006 completed, throughput=5MB/sec 2024-04-07T16:48:28.587 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-HOkuBBD_nlOOaC3 off=9526823 len=1813746 2024-04-07T16:48:28.591 INFO:tasks.workunit.client.0.smithi100.stdout:op 3007 completed, throughput=5MB/sec 2024-04-07T16:48:28.695 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:48:28 smithi100 bash[74942]: cluster 2024-04-07T16:48:27.377971+0000 mgr.y (mgr.25175) 993 : cluster [DBG] pgmap v1635: 105 pgs: 105 active+clean; 126 GiB data, 7.2 GiB used, 708 GiB / 715 GiB avail; 2.6 MiB/s rd, 2.4 MiB/s wr, 5 op/s 2024-04-07T16:48:28.696 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:48:28 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:48:28] "GET /metrics HTTP/1.1" 200 34066 "" "Prometheus/2.43.0" 2024-04-07T16:48:28.696 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:48:28 smithi100 bash[76202]: cluster 2024-04-07T16:48:27.377971+0000 mgr.y (mgr.25175) 993 : cluster [DBG] pgmap v1635: 105 pgs: 105 active+clean; 126 GiB data, 7.2 GiB used, 708 GiB / 715 GiB avail; 2.6 MiB/s rd, 2.4 MiB/s wr, 5 op/s 2024-04-07T16:48:28.860 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:48:28 smithi144 bash[66654]: cluster 2024-04-07T16:48:27.377971+0000 mgr.y (mgr.25175) 993 : cluster [DBG] pgmap v1635: 105 pgs: 105 active+clean; 126 GiB data, 7.2 GiB used, 708 GiB / 715 GiB avail; 2.6 MiB/s rd, 2.4 MiB/s wr, 5 op/s 2024-04-07T16:48:29.591 INFO:tasks.workunit.client.0.smithi100.stdout: 574: throughput=4.99MB/sec pending data=0 2024-04-07T16:48:29.591 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Y3FlGCetSagpFwH off=948454 len=1286183 2024-04-07T16:48:29.603 INFO:tasks.workunit.client.0.smithi100.stdout:op 3008 completed, throughput=5MB/sec 2024-04-07T16:48:29.603 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-2J85xJSh-SHaGWY off=2002801 len=238869 2024-04-07T16:48:29.608 INFO:tasks.workunit.client.0.smithi100.stdout:op 3009 completed, throughput=5MB/sec 2024-04-07T16:48:29.608 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-zDnP3iAovqD56QH off=7806954 len=715714 2024-04-07T16:48:29.616 INFO:tasks.workunit.client.0.smithi100.stdout:op 3010 completed, throughput=5MB/sec 2024-04-07T16:48:29.616 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-XFzlplseXEn_-VE off=12684060 len=695990 2024-04-07T16:48:29.622 INFO:tasks.workunit.client.0.smithi100.stdout:op 3011 completed, throughput=5MB/sec 2024-04-07T16:48:29.623 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-szzbcDobtAYDw51 off=19487284 len=868222 2024-04-07T16:48:29.625 INFO:tasks.workunit.client.0.smithi100.stdout:op 3012 completed, throughput=5MB/sec 2024-04-07T16:48:29.625 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-8LYEtTGCZMw_cr0 off=8108874 len=2053330 2024-04-07T16:48:29.641 INFO:tasks.workunit.client.0.smithi100.stdout:op 3013 completed, throughput=5MB/sec 2024-04-07T16:48:30.642 INFO:tasks.workunit.client.0.smithi100.stdout: 575: throughput=4.99MB/sec pending data=0 2024-04-07T16:48:30.642 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-yL9XwjEI1Aam4wK off=711444 len=545571 2024-04-07T16:48:30.648 INFO:tasks.workunit.client.0.smithi100.stdout:op 3014 completed, throughput=4.99MB/sec 2024-04-07T16:48:30.649 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-P5MCg9DAYiznax2 off=2259909 len=21256 2024-04-07T16:48:30.652 INFO:tasks.workunit.client.0.smithi100.stdout:op 3015 completed, throughput=4.99MB/sec 2024-04-07T16:48:30.652 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-X327F7FKVA9oTzE off=6252167 len=1227408 2024-04-07T16:48:30.654 INFO:tasks.workunit.client.0.smithi100.stdout:op 3016 completed, throughput=5MB/sec 2024-04-07T16:48:30.654 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-4NCX_myQOwM_tED off=8284658 len=1616113 2024-04-07T16:48:30.666 INFO:tasks.workunit.client.0.smithi100.stdout:op 3017 completed, throughput=5MB/sec 2024-04-07T16:48:30.666 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-aA1J502We5wbdKU off=6181725 len=1330586 2024-04-07T16:48:30.676 INFO:tasks.workunit.client.0.smithi100.stdout:op 3018 completed, throughput=5MB/sec 2024-04-07T16:48:30.860 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:48:30 smithi144 bash[66654]: cluster 2024-04-07T16:48:29.378810+0000 mgr.y (mgr.25175) 994 : cluster [DBG] pgmap v1636: 105 pgs: 105 active+clean; 126 GiB data, 7.2 GiB used, 708 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.1 MiB/s wr, 4 op/s 2024-04-07T16:48:30.945 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:48:30 smithi100 bash[74942]: cluster 2024-04-07T16:48:29.378810+0000 mgr.y (mgr.25175) 994 : cluster [DBG] pgmap v1636: 105 pgs: 105 active+clean; 126 GiB data, 7.2 GiB used, 708 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.1 MiB/s wr, 4 op/s 2024-04-07T16:48:30.946 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:48:30 smithi100 bash[76202]: cluster 2024-04-07T16:48:29.378810+0000 mgr.y (mgr.25175) 994 : cluster [DBG] pgmap v1636: 105 pgs: 105 active+clean; 126 GiB data, 7.2 GiB used, 708 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.1 MiB/s wr, 4 op/s 2024-04-07T16:48:31.677 INFO:tasks.workunit.client.0.smithi100.stdout: 576: throughput=4.99MB/sec pending data=0 2024-04-07T16:48:31.677 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-DhYHldMRijVp97H off=16777693 len=863970 2024-04-07T16:48:31.686 INFO:tasks.workunit.client.0.smithi100.stdout:op 3019 completed, throughput=4.99MB/sec 2024-04-07T16:48:31.686 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-7PQ_b6OKZKX1VDt off=2943285 len=1408561 2024-04-07T16:48:31.690 INFO:tasks.workunit.client.0.smithi100.stdout:op 3020 completed, throughput=5MB/sec 2024-04-07T16:48:31.690 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-lNaCzHJisWMnv_m off=4220667 len=334963 2024-04-07T16:48:31.692 INFO:tasks.workunit.client.0.smithi100.stdout:op 3021 completed, throughput=5MB/sec 2024-04-07T16:48:31.692 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-F1zbZHlxFVaEQHq off=205000 len=2036913 2024-04-07T16:48:31.697 INFO:tasks.workunit.client.0.smithi100.stdout:op 3022 completed, throughput=5MB/sec 2024-04-07T16:48:31.697 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-M7BSTOeKAD5sEAJ off=2157754 len=984427 2024-04-07T16:48:31.699 INFO:tasks.workunit.client.0.smithi100.stdout:op 3023 completed, throughput=5MB/sec 2024-04-07T16:48:32.610 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:48:32 smithi144 bash[95233]: ts=2024-04-07T16:48:32.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:48:32.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:48:32 smithi144 bash[66654]: cluster 2024-04-07T16:48:31.379722+0000 mgr.y (mgr.25175) 995 : cluster [DBG] pgmap v1637: 105 pgs: 105 active+clean; 126 GiB data, 7.2 GiB used, 708 GiB / 715 GiB avail; 3.3 MiB/s rd, 2.1 MiB/s wr, 5 op/s 2024-04-07T16:48:32.611 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:48:32 smithi144 bash[66654]: audit 2024-04-07T16:48:31.772437+0000 mon.a (mon.0) 2127 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:48:32.700 INFO:tasks.workunit.client.0.smithi100.stdout: 577: throughput=4.99MB/sec pending data=0 2024-04-07T16:48:32.700 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-T-6w9gvtaXgU24x off=8478654 len=1247233 2024-04-07T16:48:32.711 INFO:tasks.workunit.client.0.smithi100.stdout:op 3024 completed, throughput=4.99MB/sec 2024-04-07T16:48:32.711 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-sFONtEcWk2uBA4T off=5554658 len=740505 2024-04-07T16:48:32.718 INFO:tasks.workunit.client.0.smithi100.stdout:op 3025 completed, throughput=5MB/sec 2024-04-07T16:48:32.718 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-t2KI8e95moq05Jb off=8849750 len=1267015 2024-04-07T16:48:32.723 INFO:tasks.workunit.client.0.smithi100.stdout:op 3026 completed, throughput=5MB/sec 2024-04-07T16:48:32.723 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-yfRND_XRbXaWSJw off=2034131 len=594934 2024-04-07T16:48:32.730 INFO:tasks.workunit.client.0.smithi100.stdout:op 3027 completed, throughput=5MB/sec 2024-04-07T16:48:32.730 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-X_YAeSPQVtVR-ea off=3600829 len=1008263 2024-04-07T16:48:32.734 INFO:tasks.workunit.client.0.smithi100.stdout:op 3028 completed, throughput=5MB/sec 2024-04-07T16:48:32.945 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:48:32 smithi100 bash[74942]: cluster 2024-04-07T16:48:31.379722+0000 mgr.y (mgr.25175) 995 : cluster [DBG] pgmap v1637: 105 pgs: 105 active+clean; 126 GiB data, 7.2 GiB used, 708 GiB / 715 GiB avail; 3.3 MiB/s rd, 2.1 MiB/s wr, 5 op/s 2024-04-07T16:48:32.946 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:48:32 smithi100 bash[74942]: audit 2024-04-07T16:48:31.772437+0000 mon.a (mon.0) 2127 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:48:32.946 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:48:32 smithi100 bash[76202]: cluster 2024-04-07T16:48:31.379722+0000 mgr.y (mgr.25175) 995 : cluster [DBG] pgmap v1637: 105 pgs: 105 active+clean; 126 GiB data, 7.2 GiB used, 708 GiB / 715 GiB avail; 3.3 MiB/s rd, 2.1 MiB/s wr, 5 op/s 2024-04-07T16:48:32.946 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:48:32 smithi100 bash[76202]: audit 2024-04-07T16:48:31.772437+0000 mon.a (mon.0) 2127 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:48:33.734 INFO:tasks.workunit.client.0.smithi100.stdout: 578: throughput=4.99MB/sec pending data=0 2024-04-07T16:48:33.735 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Li8Brok9cEvlYr1 off=16897550 len=2006334 2024-04-07T16:48:33.739 INFO:tasks.workunit.client.0.smithi100.stdout:op 3029 completed, throughput=5MB/sec 2024-04-07T16:48:33.739 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-mHZyXpJO9wp-aNS off=9824301 len=1582177 2024-04-07T16:48:33.743 INFO:tasks.workunit.client.0.smithi100.stdout:op 3030 completed, throughput=5MB/sec 2024-04-07T16:48:33.743 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-4s3qTTNrwGjw4ww off=1499265 len=377311 2024-04-07T16:48:33.746 INFO:tasks.workunit.client.0.smithi100.stdout:op 3031 completed, throughput=5MB/sec 2024-04-07T16:48:33.746 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-uqcDuKId1pzvMEh off=11702101 len=1260239 2024-04-07T16:48:33.756 INFO:tasks.workunit.client.0.smithi100.stdout:op 3032 completed, throughput=5MB/sec 2024-04-07T16:48:34.757 INFO:tasks.workunit.client.0.smithi100.stdout: 579: throughput=4.99MB/sec pending data=0 2024-04-07T16:48:34.757 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-StDS0X_9hpwSGvc off=6942120 len=672193 2024-04-07T16:48:34.765 INFO:tasks.workunit.client.0.smithi100.stdout:op 3033 completed, throughput=4.99MB/sec 2024-04-07T16:48:34.765 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-UAgT_WXuwyvSN6v off=1404828 len=1603329 2024-04-07T16:48:34.769 INFO:tasks.workunit.client.0.smithi100.stdout:op 3034 completed, throughput=5MB/sec 2024-04-07T16:48:34.769 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-NJ5WKTEPnZmoIhd off=169560 len=128183 2024-04-07T16:48:34.770 INFO:tasks.workunit.client.0.smithi100.stdout:op 3035 completed, throughput=5MB/sec 2024-04-07T16:48:34.770 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-DCa1d0i4PEDnCjy off=2832690 len=1729250 2024-04-07T16:48:34.784 INFO:tasks.workunit.client.0.smithi100.stdout:op 3036 completed, throughput=5MB/sec 2024-04-07T16:48:34.784 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-h7bnsrLCe9F3hOB off=3591225 len=1053299 2024-04-07T16:48:34.788 INFO:tasks.workunit.client.0.smithi100.stdout:op 3037 completed, throughput=5MB/sec 2024-04-07T16:48:34.788 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-R17-Becrz_nF5gB off=12940464 len=2077925 2024-04-07T16:48:34.804 INFO:tasks.workunit.client.0.smithi100.stdout:op 3038 completed, throughput=5MB/sec 2024-04-07T16:48:34.860 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:48:34 smithi144 bash[66654]: cluster 2024-04-07T16:48:33.380833+0000 mgr.y (mgr.25175) 996 : cluster [DBG] pgmap v1638: 105 pgs: 105 active+clean; 126 GiB data, 7.2 GiB used, 708 GiB / 715 GiB avail; 3.2 MiB/s rd, 2.8 MiB/s wr, 5 op/s 2024-04-07T16:48:34.945 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:48:34 smithi100 bash[74942]: cluster 2024-04-07T16:48:33.380833+0000 mgr.y (mgr.25175) 996 : cluster [DBG] pgmap v1638: 105 pgs: 105 active+clean; 126 GiB data, 7.2 GiB used, 708 GiB / 715 GiB avail; 3.2 MiB/s rd, 2.8 MiB/s wr, 5 op/s 2024-04-07T16:48:34.946 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:48:34 smithi100 bash[76202]: cluster 2024-04-07T16:48:33.380833+0000 mgr.y (mgr.25175) 996 : cluster [DBG] pgmap v1638: 105 pgs: 105 active+clean; 126 GiB data, 7.2 GiB used, 708 GiB / 715 GiB avail; 3.2 MiB/s rd, 2.8 MiB/s wr, 5 op/s 2024-04-07T16:48:35.804 INFO:tasks.workunit.client.0.smithi100.stdout: 580: throughput=4.99MB/sec pending data=0 2024-04-07T16:48:35.804 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-HDiW-oc8HMYrzpt off=6648629 len=1087270 2024-04-07T16:48:35.807 INFO:tasks.workunit.client.0.smithi100.stdout:op 3039 completed, throughput=5MB/sec 2024-04-07T16:48:35.807 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-d52oSKLDFmDC4YB off=1913569 len=1652368 2024-04-07T16:48:35.813 INFO:tasks.workunit.client.0.smithi100.stdout:op 3040 completed, throughput=5MB/sec 2024-04-07T16:48:35.813 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-HS4Mn74nJ-OZWLJ off=21117155 len=524822 2024-04-07T16:48:35.819 INFO:tasks.workunit.client.0.smithi100.stdout:op 3041 completed, throughput=5MB/sec 2024-04-07T16:48:35.819 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-1v9Sg_IM6Y9mWW7 off=5821028 len=321612 2024-04-07T16:48:35.820 INFO:tasks.workunit.client.0.smithi100.stdout:op 3042 completed, throughput=5MB/sec 2024-04-07T16:48:36.746 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:48:36 smithi100 bash[74942]: cluster 2024-04-07T16:48:35.381750+0000 mgr.y (mgr.25175) 997 : cluster [DBG] pgmap v1639: 105 pgs: 105 active+clean; 126 GiB data, 7.2 GiB used, 708 GiB / 715 GiB avail; 2.8 MiB/s rd, 2.5 MiB/s wr, 5 op/s 2024-04-07T16:48:36.821 INFO:tasks.workunit.client.0.smithi100.stdout: 581: throughput=4.99MB/sec pending data=0 2024-04-07T16:48:36.821 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-XUaTolfBnp6mozU off=10982291 len=1676874 2024-04-07T16:48:36.825 INFO:tasks.workunit.client.0.smithi100.stdout:op 3043 completed, throughput=4.99MB/sec 2024-04-07T16:48:36.825 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-ggLOZ-kz0j5Hdwg off=15472174 len=1846119 2024-04-07T16:48:36.842 INFO:tasks.workunit.client.0.smithi100.stdout:op 3044 completed, throughput=5MB/sec 2024-04-07T16:48:36.842 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-tD7r0cVMVruOPhY off=8600422 len=566815 2024-04-07T16:48:36.848 INFO:tasks.workunit.client.0.smithi100.stdout:op 3045 completed, throughput=5MB/sec 2024-04-07T16:48:36.848 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-IVZpLFdDiuimr0Z off=3743566 len=1235871 2024-04-07T16:48:36.857 INFO:tasks.workunit.client.0.smithi100.stdout:op 3046 completed, throughput=5MB/sec 2024-04-07T16:48:36.860 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:48:36 smithi144 bash[66654]: cluster 2024-04-07T16:48:35.381750+0000 mgr.y (mgr.25175) 997 : cluster [DBG] pgmap v1639: 105 pgs: 105 active+clean; 126 GiB data, 7.2 GiB used, 708 GiB / 715 GiB avail; 2.8 MiB/s rd, 2.5 MiB/s wr, 5 op/s 2024-04-07T16:48:37.196 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:48:36 smithi100 bash[76202]: cluster 2024-04-07T16:48:35.381750+0000 mgr.y (mgr.25175) 997 : cluster [DBG] pgmap v1639: 105 pgs: 105 active+clean; 126 GiB data, 7.2 GiB used, 708 GiB / 715 GiB avail; 2.8 MiB/s rd, 2.5 MiB/s wr, 5 op/s 2024-04-07T16:48:37.857 INFO:tasks.workunit.client.0.smithi100.stdout: 582: throughput=4.99MB/sec pending data=0 2024-04-07T16:48:37.858 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-IT_IgcXbgUaaB9C off=845341 len=635299 2024-04-07T16:48:37.860 INFO:tasks.workunit.client.0.smithi100.stdout:op 3047 completed, throughput=4.99MB/sec 2024-04-07T16:48:37.860 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-XkgLrBA41-vsn4B off=16245147 len=253192 2024-04-07T16:48:37.862 INFO:tasks.workunit.client.0.smithi100.stdout:op 3048 completed, throughput=4.99MB/sec 2024-04-07T16:48:37.862 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-DgXahwWOkm9kNnV off=653940 len=158759 2024-04-07T16:48:37.867 INFO:tasks.workunit.client.0.smithi100.stdout:op 3049 completed, throughput=4.99MB/sec 2024-04-07T16:48:37.867 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-z6ESHkpxE_srxj9 off=0 len=1603715 2024-04-07T16:48:37.880 INFO:tasks.workunit.client.0.smithi100.stdout:op 3050 completed, throughput=5MB/sec 2024-04-07T16:48:37.880 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-VdLUQlZKAySjNnC off=3726930 len=993538 2024-04-07T16:48:37.883 INFO:tasks.workunit.client.0.smithi100.stdout:op 3051 completed, throughput=5MB/sec 2024-04-07T16:48:37.883 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-U0cW6s0LMEYz9mv off=5380813 len=1860021 2024-04-07T16:48:37.897 INFO:tasks.workunit.client.0.smithi100.stdout:op 3052 completed, throughput=5MB/sec 2024-04-07T16:48:38.696 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:48:38 smithi100 bash[74942]: cluster 2024-04-07T16:48:37.382766+0000 mgr.y (mgr.25175) 998 : cluster [DBG] pgmap v1640: 105 pgs: 105 active+clean; 126 GiB data, 7.3 GiB used, 708 GiB / 715 GiB avail; 2.8 MiB/s rd, 2.1 MiB/s wr, 4 op/s 2024-04-07T16:48:38.696 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:48:38 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:48:38] "GET /metrics HTTP/1.1" 200 34071 "" "Prometheus/2.43.0" 2024-04-07T16:48:38.696 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:48:38 smithi100 bash[76202]: cluster 2024-04-07T16:48:37.382766+0000 mgr.y (mgr.25175) 998 : cluster [DBG] pgmap v1640: 105 pgs: 105 active+clean; 126 GiB data, 7.3 GiB used, 708 GiB / 715 GiB avail; 2.8 MiB/s rd, 2.1 MiB/s wr, 4 op/s 2024-04-07T16:48:38.860 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:48:38 smithi144 bash[66654]: cluster 2024-04-07T16:48:37.382766+0000 mgr.y (mgr.25175) 998 : cluster [DBG] pgmap v1640: 105 pgs: 105 active+clean; 126 GiB data, 7.3 GiB used, 708 GiB / 715 GiB avail; 2.8 MiB/s rd, 2.1 MiB/s wr, 4 op/s 2024-04-07T16:48:38.897 INFO:tasks.workunit.client.0.smithi100.stdout: 583: throughput=4.99MB/sec pending data=0 2024-04-07T16:48:38.897 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-AjeAjqMceICm_Ul off=22246023 len=1314563 2024-04-07T16:48:38.908 INFO:tasks.workunit.client.0.smithi100.stdout:op 3053 completed, throughput=4.99MB/sec 2024-04-07T16:48:38.908 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-nkwqPR2vS3B5CQE off=2912937 len=1551580 2024-04-07T16:48:38.912 INFO:tasks.workunit.client.0.smithi100.stdout:op 3054 completed, throughput=5MB/sec 2024-04-07T16:48:38.913 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-YlSgZtZlW_W2dFT off=8196487 len=2087638 2024-04-07T16:48:38.919 INFO:tasks.workunit.client.0.smithi100.stdout:op 3055 completed, throughput=5MB/sec 2024-04-07T16:48:38.919 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-6rz9hEruActtYs5 off=13100795 len=2018549 2024-04-07T16:48:38.924 INFO:tasks.workunit.client.0.smithi100.stdout:op 3056 completed, throughput=5MB/sec 2024-04-07T16:48:39.924 INFO:tasks.workunit.client.0.smithi100.stdout: 584: throughput=4.99MB/sec pending data=0 2024-04-07T16:48:39.924 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-6ErhQNh2JXlllft off=9346763 len=52793 2024-04-07T16:48:39.928 INFO:tasks.workunit.client.0.smithi100.stdout:op 3057 completed, throughput=4.99MB/sec 2024-04-07T16:48:39.929 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-bRY0Tkq2jymqwFd off=2880998 len=573788 2024-04-07T16:48:39.936 INFO:tasks.workunit.client.0.smithi100.stdout:op 3058 completed, throughput=5MB/sec 2024-04-07T16:48:39.936 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-biaceAFjv2Y9X4L off=1209240 len=1619434 2024-04-07T16:48:39.949 INFO:tasks.workunit.client.0.smithi100.stdout:op 3059 completed, throughput=5MB/sec 2024-04-07T16:48:39.949 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-dRuWow_f9KHCafw off=9140380 len=638456 2024-04-07T16:48:39.956 INFO:tasks.workunit.client.0.smithi100.stdout:op 3060 completed, throughput=5MB/sec 2024-04-07T16:48:39.956 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-AVLNu2Wf4FGjDhY off=1407999 len=1084611 2024-04-07T16:48:39.965 INFO:tasks.workunit.client.0.smithi100.stdout:op 3061 completed, throughput=5MB/sec 2024-04-07T16:48:40.860 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:48:40 smithi144 bash[66654]: cluster 2024-04-07T16:48:39.383654+0000 mgr.y (mgr.25175) 999 : cluster [DBG] pgmap v1641: 105 pgs: 105 active+clean; 126 GiB data, 7.3 GiB used, 708 GiB / 715 GiB avail; 2.6 MiB/s rd, 2.2 MiB/s wr, 4 op/s 2024-04-07T16:48:40.946 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:48:40 smithi100 bash[74942]: cluster 2024-04-07T16:48:39.383654+0000 mgr.y (mgr.25175) 999 : cluster [DBG] pgmap v1641: 105 pgs: 105 active+clean; 126 GiB data, 7.3 GiB used, 708 GiB / 715 GiB avail; 2.6 MiB/s rd, 2.2 MiB/s wr, 4 op/s 2024-04-07T16:48:40.946 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:48:40 smithi100 bash[76202]: cluster 2024-04-07T16:48:39.383654+0000 mgr.y (mgr.25175) 999 : cluster [DBG] pgmap v1641: 105 pgs: 105 active+clean; 126 GiB data, 7.3 GiB used, 708 GiB / 715 GiB avail; 2.6 MiB/s rd, 2.2 MiB/s wr, 4 op/s 2024-04-07T16:48:40.965 INFO:tasks.workunit.client.0.smithi100.stdout: 585: throughput=4.99MB/sec pending data=0 2024-04-07T16:48:40.965 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-pgZPpJjDY4yto-p off=2426300 len=31950 2024-04-07T16:48:40.969 INFO:tasks.workunit.client.0.smithi100.stdout:op 3062 completed, throughput=4.99MB/sec 2024-04-07T16:48:40.970 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-t036cf6JF77n-AL off=2223594 len=815055 2024-04-07T16:48:40.972 INFO:tasks.workunit.client.0.smithi100.stdout:op 3063 completed, throughput=4.99MB/sec 2024-04-07T16:48:40.972 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-5J0tmEghIldRA_x off=23688795 len=1847740 2024-04-07T16:48:40.989 INFO:tasks.workunit.client.0.smithi100.stdout:op 3064 completed, throughput=5MB/sec 2024-04-07T16:48:40.989 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-i4Nllu54OsU6NCw off=4190572 len=1324129 2024-04-07T16:48:40.993 INFO:tasks.workunit.client.0.smithi100.stdout:op 3065 completed, throughput=5MB/sec 2024-04-07T16:48:40.993 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-P7sebu3lCnQUaMU off=4118741 len=286439 2024-04-07T16:48:40.995 INFO:tasks.workunit.client.0.smithi100.stdout:op 3066 completed, throughput=5MB/sec 2024-04-07T16:48:40.995 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-_6lOCpFnB-PzH3T off=47978 len=1934717 2024-04-07T16:48:41.011 INFO:tasks.workunit.client.0.smithi100.stdout:op 3067 completed, throughput=5MB/sec 2024-04-07T16:48:42.012 INFO:tasks.workunit.client.0.smithi100.stdout: 586: throughput=4.99MB/sec pending data=0 2024-04-07T16:48:42.012 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-2ez2M6FCRl55IIB off=8780034 len=1539996 2024-04-07T16:48:42.026 INFO:tasks.workunit.client.0.smithi100.stdout:op 3068 completed, throughput=5MB/sec 2024-04-07T16:48:42.026 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-uW1Wa6y0xQNLiGR off=9650927 len=461279 2024-04-07T16:48:42.029 INFO:tasks.workunit.client.0.smithi100.stdout:op 3069 completed, throughput=5MB/sec 2024-04-07T16:48:42.029 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Ejv8NepDe8nb0cT off=2542724 len=100739 2024-04-07T16:48:42.030 INFO:tasks.workunit.client.0.smithi100.stdout:op 3070 completed, throughput=5MB/sec 2024-04-07T16:48:42.030 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-mf1LUEvGcgq7J-x off=2456690 len=591625 2024-04-07T16:48:42.032 INFO:tasks.workunit.client.0.smithi100.stdout:op 3071 completed, throughput=5MB/sec 2024-04-07T16:48:42.032 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-f5WXm5xOuCk6tNH off=2342166 len=699221 2024-04-07T16:48:42.035 INFO:tasks.workunit.client.0.smithi100.stdout:op 3072 completed, throughput=5MB/sec 2024-04-07T16:48:42.035 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-CfXFFXC5vSYOxtq off=15973229 len=1764240 2024-04-07T16:48:42.039 INFO:tasks.workunit.client.0.smithi100.stdout:op 3073 completed, throughput=5MB/sec 2024-04-07T16:48:42.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:48:42 smithi144 bash[66654]: cluster 2024-04-07T16:48:41.384600+0000 mgr.y (mgr.25175) 1000 : cluster [DBG] pgmap v1642: 105 pgs: 105 active+clean; 126 GiB data, 7.3 GiB used, 708 GiB / 715 GiB avail; 2.6 MiB/s rd, 2.3 MiB/s wr, 4 op/s 2024-04-07T16:48:42.614 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:48:42 smithi144 bash[95233]: ts=2024-04-07T16:48:42.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:48:42.946 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:48:42 smithi100 bash[74942]: cluster 2024-04-07T16:48:41.384600+0000 mgr.y (mgr.25175) 1000 : cluster [DBG] pgmap v1642: 105 pgs: 105 active+clean; 126 GiB data, 7.3 GiB used, 708 GiB / 715 GiB avail; 2.6 MiB/s rd, 2.3 MiB/s wr, 4 op/s 2024-04-07T16:48:42.946 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:48:42 smithi100 bash[76202]: cluster 2024-04-07T16:48:41.384600+0000 mgr.y (mgr.25175) 1000 : cluster [DBG] pgmap v1642: 105 pgs: 105 active+clean; 126 GiB data, 7.3 GiB used, 708 GiB / 715 GiB avail; 2.6 MiB/s rd, 2.3 MiB/s wr, 4 op/s 2024-04-07T16:48:43.039 INFO:tasks.workunit.client.0.smithi100.stdout: 587: throughput=4.99MB/sec pending data=0 2024-04-07T16:48:43.040 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-9wAnBt61G5n94Bn off=9641246 len=1211050 2024-04-07T16:48:43.045 INFO:tasks.workunit.client.0.smithi100.stdout:op 3074 completed, throughput=4.99MB/sec 2024-04-07T16:48:43.045 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-zUAm9CUUDJc-uTG off=566532 len=482695 2024-04-07T16:48:43.052 INFO:tasks.workunit.client.0.smithi100.stdout:op 3075 completed, throughput=5MB/sec 2024-04-07T16:48:43.052 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-yFQPnLaUgRPk9yy off=7717905 len=1871004 2024-04-07T16:48:43.057 INFO:tasks.workunit.client.0.smithi100.stdout:op 3076 completed, throughput=5MB/sec 2024-04-07T16:48:43.057 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-_Cx5uzVT3IcVRiq off=7773737 len=1906610 2024-04-07T16:48:43.073 INFO:tasks.workunit.client.0.smithi100.stdout:op 3077 completed, throughput=5MB/sec 2024-04-07T16:48:44.073 INFO:tasks.workunit.client.0.smithi100.stdout: 588: throughput=4.99MB/sec pending data=0 2024-04-07T16:48:44.074 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-usfiE3Yd3TJSshA off=1263996 len=893991 2024-04-07T16:48:44.083 INFO:tasks.workunit.client.0.smithi100.stdout:op 3078 completed, throughput=4.99MB/sec 2024-04-07T16:48:44.083 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-iTv_NCBzEDXXjId off=3748915 len=1883223 2024-04-07T16:48:44.088 INFO:tasks.workunit.client.0.smithi100.stdout:op 3079 completed, throughput=5MB/sec 2024-04-07T16:48:44.088 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-PpK5NBzoGkZPWyl off=5076727 len=647777 2024-04-07T16:48:44.095 INFO:tasks.workunit.client.0.smithi100.stdout:op 3080 completed, throughput=5MB/sec 2024-04-07T16:48:44.095 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-ee2yYBacRdTQt2S off=12718866 len=334514 2024-04-07T16:48:44.102 INFO:tasks.workunit.client.0.smithi100.stdout:op 3081 completed, throughput=5MB/sec 2024-04-07T16:48:44.102 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-hynUtBk-S2djY7i off=0 len=1064107 2024-04-07T16:48:44.110 INFO:tasks.workunit.client.0.smithi100.stdout:op 3082 completed, throughput=5MB/sec 2024-04-07T16:48:44.861 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:48:44 smithi144 bash[66654]: cluster 2024-04-07T16:48:43.385678+0000 mgr.y (mgr.25175) 1001 : cluster [DBG] pgmap v1643: 105 pgs: 105 active+clean; 126 GiB data, 7.3 GiB used, 708 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.9 MiB/s wr, 5 op/s 2024-04-07T16:48:44.946 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:48:44 smithi100 bash[74942]: cluster 2024-04-07T16:48:43.385678+0000 mgr.y (mgr.25175) 1001 : cluster [DBG] pgmap v1643: 105 pgs: 105 active+clean; 126 GiB data, 7.3 GiB used, 708 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.9 MiB/s wr, 5 op/s 2024-04-07T16:48:44.946 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:48:44 smithi100 bash[76202]: cluster 2024-04-07T16:48:43.385678+0000 mgr.y (mgr.25175) 1001 : cluster [DBG] pgmap v1643: 105 pgs: 105 active+clean; 126 GiB data, 7.3 GiB used, 708 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.9 MiB/s wr, 5 op/s 2024-04-07T16:48:45.111 INFO:tasks.workunit.client.0.smithi100.stdout: 589: throughput=4.99MB/sec pending data=0 2024-04-07T16:48:45.111 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-VzHh9Y3tbofTJ_Y off=1794339 len=815561 2024-04-07T16:48:45.113 INFO:tasks.workunit.client.0.smithi100.stdout:op 3083 completed, throughput=4.99MB/sec 2024-04-07T16:48:45.113 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-jfJrWL6PppZx_bD off=3726141 len=956161 2024-04-07T16:48:45.116 INFO:tasks.workunit.client.0.smithi100.stdout:op 3084 completed, throughput=4.99MB/sec 2024-04-07T16:48:45.116 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-RKezFw8TV9FLEIO off=4461367 len=1141300 2024-04-07T16:48:45.127 INFO:tasks.workunit.client.0.smithi100.stdout:op 3085 completed, throughput=5MB/sec 2024-04-07T16:48:45.127 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Js6H-PIqfWusfsR off=7727987 len=928823 2024-04-07T16:48:45.136 INFO:tasks.workunit.client.0.smithi100.stdout:op 3086 completed, throughput=5MB/sec 2024-04-07T16:48:45.136 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-vhkVnwijkXdtVXQ off=17966967 len=295496 2024-04-07T16:48:45.141 INFO:tasks.workunit.client.0.smithi100.stdout:op 3087 completed, throughput=5MB/sec 2024-04-07T16:48:45.141 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-zbMDom9Uo_bB9-X off=10684037 len=658430 2024-04-07T16:48:45.143 INFO:tasks.workunit.client.0.smithi100.stdout:op 3088 completed, throughput=5MB/sec 2024-04-07T16:48:45.143 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Ip2IcG-5JfxhQbB off=15240947 len=973782 2024-04-07T16:48:45.145 INFO:tasks.workunit.client.0.smithi100.stdout:op 3089 completed, throughput=5MB/sec 2024-04-07T16:48:46.145 INFO:tasks.workunit.client.0.smithi100.stdout: 590: throughput=4.99MB/sec pending data=0 2024-04-07T16:48:46.145 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-eIyRQFR2pm76pvU off=4250993 len=1606570 2024-04-07T16:48:46.159 INFO:tasks.workunit.client.0.smithi100.stdout:op 3090 completed, throughput=5MB/sec 2024-04-07T16:48:46.159 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-lOpxrW9I3dgSRRK off=5887868 len=1271173 2024-04-07T16:48:46.168 INFO:tasks.workunit.client.0.smithi100.stdout:op 3091 completed, throughput=5MB/sec 2024-04-07T16:48:46.168 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-jdPXazCryK1YBco off=19818797 len=1553489 2024-04-07T16:48:46.171 INFO:tasks.workunit.client.0.smithi100.stdout:op 3092 completed, throughput=5MB/sec 2024-04-07T16:48:46.171 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-XlnmDqKGcbebsYJ off=9003831 len=1282931 2024-04-07T16:48:46.174 INFO:tasks.workunit.client.0.smithi100.stdout:op 3093 completed, throughput=5MB/sec 2024-04-07T16:48:46.860 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:48:46 smithi144 bash[66654]: cluster 2024-04-07T16:48:45.386594+0000 mgr.y (mgr.25175) 1002 : cluster [DBG] pgmap v1644: 105 pgs: 105 active+clean; 126 GiB data, 7.3 GiB used, 708 GiB / 715 GiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 4 op/s 2024-04-07T16:48:46.946 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:48:46 smithi100 bash[74942]: cluster 2024-04-07T16:48:45.386594+0000 mgr.y (mgr.25175) 1002 : cluster [DBG] pgmap v1644: 105 pgs: 105 active+clean; 126 GiB data, 7.3 GiB used, 708 GiB / 715 GiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 4 op/s 2024-04-07T16:48:46.946 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:48:46 smithi100 bash[76202]: cluster 2024-04-07T16:48:45.386594+0000 mgr.y (mgr.25175) 1002 : cluster [DBG] pgmap v1644: 105 pgs: 105 active+clean; 126 GiB data, 7.3 GiB used, 708 GiB / 715 GiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 4 op/s 2024-04-07T16:48:47.174 INFO:tasks.workunit.client.0.smithi100.stdout: 591: throughput=4.99MB/sec pending data=0 2024-04-07T16:48:47.175 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-ZYl_KYUZj-Vv6iV off=5481721 len=1201435 2024-04-07T16:48:47.178 INFO:tasks.workunit.client.0.smithi100.stdout:op 3094 completed, throughput=5MB/sec 2024-04-07T16:48:47.178 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-FWsF8WfnbACS-6A off=1367160 len=964593 2024-04-07T16:48:47.187 INFO:tasks.workunit.client.0.smithi100.stdout:op 3095 completed, throughput=5MB/sec 2024-04-07T16:48:47.187 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-rXuH7ivyEVPrC4I off=1652650 len=1440194 2024-04-07T16:48:47.190 INFO:tasks.workunit.client.0.smithi100.stdout:op 3096 completed, throughput=5MB/sec 2024-04-07T16:48:47.191 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-rZRQKqdq4pVUxr4 off=12689383 len=99790 2024-04-07T16:48:47.192 INFO:tasks.workunit.client.0.smithi100.stdout:op 3097 completed, throughput=5MB/sec 2024-04-07T16:48:47.192 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-O5ozYUeW3mvwK9W off=942118 len=946481 2024-04-07T16:48:47.200 INFO:tasks.workunit.client.0.smithi100.stdout:op 3098 completed, throughput=5MB/sec 2024-04-07T16:48:47.860 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:48:47 smithi144 bash[66654]: audit 2024-04-07T16:48:46.772770+0000 mon.a (mon.0) 2128 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:48:47.946 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:48:47 smithi100 bash[74942]: audit 2024-04-07T16:48:46.772770+0000 mon.a (mon.0) 2128 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:48:47.946 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:48:47 smithi100 bash[76202]: audit 2024-04-07T16:48:46.772770+0000 mon.a (mon.0) 2128 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:48:48.200 INFO:tasks.workunit.client.0.smithi100.stdout: 592: throughput=4.99MB/sec pending data=0 2024-04-07T16:48:48.200 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-mx0IkEXWi43ngzA off=2626793 len=1609330 2024-04-07T16:48:48.204 INFO:tasks.workunit.client.0.smithi100.stdout:op 3099 completed, throughput=4.99MB/sec 2024-04-07T16:48:48.204 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-A4W_V0k0TZeX8ZX off=3274563 len=1967927 2024-04-07T16:48:48.209 INFO:tasks.workunit.client.0.smithi100.stdout:op 3100 completed, throughput=5MB/sec 2024-04-07T16:48:48.209 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-OVhfOoyeKSt3BT3 off=18967740 len=1899254 2024-04-07T16:48:48.224 INFO:tasks.workunit.client.0.smithi100.stdout:op 3101 completed, throughput=5MB/sec 2024-04-07T16:48:48.696 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:48:48 smithi100 bash[74942]: cluster 2024-04-07T16:48:47.387611+0000 mgr.y (mgr.25175) 1003 : cluster [DBG] pgmap v1645: 105 pgs: 105 active+clean; 126 GiB data, 7.3 GiB used, 708 GiB / 715 GiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 4 op/s 2024-04-07T16:48:48.696 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:48:48 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:48:48] "GET /metrics HTTP/1.1" 200 34070 "" "Prometheus/2.43.0" 2024-04-07T16:48:48.696 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:48:48 smithi100 bash[76202]: cluster 2024-04-07T16:48:47.387611+0000 mgr.y (mgr.25175) 1003 : cluster [DBG] pgmap v1645: 105 pgs: 105 active+clean; 126 GiB data, 7.3 GiB used, 708 GiB / 715 GiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 4 op/s 2024-04-07T16:48:48.860 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:48:48 smithi144 bash[66654]: cluster 2024-04-07T16:48:47.387611+0000 mgr.y (mgr.25175) 1003 : cluster [DBG] pgmap v1645: 105 pgs: 105 active+clean; 126 GiB data, 7.3 GiB used, 708 GiB / 715 GiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 4 op/s 2024-04-07T16:48:49.225 INFO:tasks.workunit.client.0.smithi100.stdout: 593: throughput=4.99MB/sec pending data=0 2024-04-07T16:48:49.225 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-VD6gqR2LDOlnVaz off=12595547 len=1593887 2024-04-07T16:48:49.239 INFO:tasks.workunit.client.0.smithi100.stdout:op 3102 completed, throughput=4.99MB/sec 2024-04-07T16:48:49.239 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-KojEf5AnWV4gnQq off=8596382 len=1407847 2024-04-07T16:48:49.249 INFO:tasks.workunit.client.0.smithi100.stdout:op 3103 completed, throughput=5MB/sec 2024-04-07T16:48:49.249 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-JTpL3rG0BWd8HYJ off=3880939 len=1650119 2024-04-07T16:48:49.253 INFO:tasks.workunit.client.0.smithi100.stdout:op 3104 completed, throughput=5MB/sec 2024-04-07T16:48:49.253 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-FrCz0_aJFjg-Rhr off=2990955 len=1301250 2024-04-07T16:48:49.257 INFO:tasks.workunit.client.0.smithi100.stdout:op 3105 completed, throughput=5MB/sec 2024-04-07T16:48:50.257 INFO:tasks.workunit.client.0.smithi100.stdout: 594: throughput=4.99MB/sec pending data=0 2024-04-07T16:48:50.257 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-tz6yeDi2QUuLs0j off=2887598 len=1413092 2024-04-07T16:48:50.269 INFO:tasks.workunit.client.0.smithi100.stdout:op 3106 completed, throughput=5MB/sec 2024-04-07T16:48:50.269 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-NOMalwWajOXVF-R off=9627987 len=705638 2024-04-07T16:48:50.277 INFO:tasks.workunit.client.0.smithi100.stdout:op 3107 completed, throughput=5MB/sec 2024-04-07T16:48:50.277 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj--5pJjVv8QGffx6A off=6913261 len=1564364 2024-04-07T16:48:50.288 INFO:tasks.workunit.client.0.smithi100.stdout:op 3108 completed, throughput=5MB/sec 2024-04-07T16:48:50.288 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-D-hOYTeUkZNbAux off=20724687 len=832038 2024-04-07T16:48:50.296 INFO:tasks.workunit.client.0.smithi100.stdout:op 3109 completed, throughput=5MB/sec 2024-04-07T16:48:50.296 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-wr45Rdvh1I_qyu9 off=7130801 len=1154779 2024-04-07T16:48:50.299 INFO:tasks.workunit.client.0.smithi100.stdout:op 3110 completed, throughput=5MB/sec 2024-04-07T16:48:50.860 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:48:50 smithi144 bash[66654]: cluster 2024-04-07T16:48:49.388420+0000 mgr.y (mgr.25175) 1004 : cluster [DBG] pgmap v1646: 105 pgs: 105 active+clean; 126 GiB data, 7.3 GiB used, 708 GiB / 715 GiB avail; 1.9 MiB/s rd, 2.1 MiB/s wr, 3 op/s 2024-04-07T16:48:50.946 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:48:50 smithi100 bash[74942]: cluster 2024-04-07T16:48:49.388420+0000 mgr.y (mgr.25175) 1004 : cluster [DBG] pgmap v1646: 105 pgs: 105 active+clean; 126 GiB data, 7.3 GiB used, 708 GiB / 715 GiB avail; 1.9 MiB/s rd, 2.1 MiB/s wr, 3 op/s 2024-04-07T16:48:50.946 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:48:50 smithi100 bash[76202]: cluster 2024-04-07T16:48:49.388420+0000 mgr.y (mgr.25175) 1004 : cluster [DBG] pgmap v1646: 105 pgs: 105 active+clean; 126 GiB data, 7.3 GiB used, 708 GiB / 715 GiB avail; 1.9 MiB/s rd, 2.1 MiB/s wr, 3 op/s 2024-04-07T16:48:51.300 INFO:tasks.workunit.client.0.smithi100.stdout: 595: throughput=4.99MB/sec pending data=0 2024-04-07T16:48:51.300 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Do7jHKquu4OeReZ off=6059265 len=2036341 2024-04-07T16:48:51.305 INFO:tasks.workunit.client.0.smithi100.stdout:op 3111 completed, throughput=5MB/sec 2024-04-07T16:48:51.305 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-JO3LISm1A-tZ7bS off=7728794 len=1570225 2024-04-07T16:48:51.310 INFO:tasks.workunit.client.0.smithi100.stdout:op 3112 completed, throughput=5MB/sec 2024-04-07T16:48:51.310 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-SM2tcd_CUAduEhH off=8553472 len=1401363 2024-04-07T16:48:51.314 INFO:tasks.workunit.client.0.smithi100.stdout:op 3113 completed, throughput=5MB/sec 2024-04-07T16:48:52.314 INFO:tasks.workunit.client.0.smithi100.stdout: 596: throughput=4.99MB/sec pending data=0 2024-04-07T16:48:52.314 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-w4ZSnjW-93e47Li off=14667742 len=1831481 2024-04-07T16:48:52.318 INFO:tasks.workunit.client.0.smithi100.stdout:op 3114 completed, throughput=5MB/sec 2024-04-07T16:48:52.319 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-yGayoxRl9I1iHxx off=6587534 len=1521945 2024-04-07T16:48:52.322 INFO:tasks.workunit.client.0.smithi100.stdout:op 3115 completed, throughput=5MB/sec 2024-04-07T16:48:52.323 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-icRDqd-kP94sAPx off=8371276 len=1500924 2024-04-07T16:48:52.328 INFO:tasks.workunit.client.0.smithi100.stdout:op 3116 completed, throughput=5MB/sec 2024-04-07T16:48:52.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:48:52 smithi144 bash[66654]: cluster 2024-04-07T16:48:51.389345+0000 mgr.y (mgr.25175) 1005 : cluster [DBG] pgmap v1647: 105 pgs: 105 active+clean; 126 GiB data, 7.4 GiB used, 708 GiB / 715 GiB avail; 2.6 MiB/s rd, 2.0 MiB/s wr, 4 op/s 2024-04-07T16:48:52.611 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:48:52 smithi144 bash[95233]: ts=2024-04-07T16:48:52.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:48:52.946 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:48:52 smithi100 bash[74942]: cluster 2024-04-07T16:48:51.389345+0000 mgr.y (mgr.25175) 1005 : cluster [DBG] pgmap v1647: 105 pgs: 105 active+clean; 126 GiB data, 7.4 GiB used, 708 GiB / 715 GiB avail; 2.6 MiB/s rd, 2.0 MiB/s wr, 4 op/s 2024-04-07T16:48:52.946 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:48:52 smithi100 bash[76202]: cluster 2024-04-07T16:48:51.389345+0000 mgr.y (mgr.25175) 1005 : cluster [DBG] pgmap v1647: 105 pgs: 105 active+clean; 126 GiB data, 7.4 GiB used, 708 GiB / 715 GiB avail; 2.6 MiB/s rd, 2.0 MiB/s wr, 4 op/s 2024-04-07T16:48:53.328 INFO:tasks.workunit.client.0.smithi100.stdout: 597: throughput=4.99MB/sec pending data=0 2024-04-07T16:48:53.328 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-sZ6uFvOwnBxyEGW off=11164580 len=1968872 2024-04-07T16:48:53.345 INFO:tasks.workunit.client.0.smithi100.stdout:op 3117 completed, throughput=5MB/sec 2024-04-07T16:48:53.345 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-iYvoMfh3ZuRkgKA off=1432096 len=330873 2024-04-07T16:48:53.351 INFO:tasks.workunit.client.0.smithi100.stdout:op 3118 completed, throughput=5MB/sec 2024-04-07T16:48:53.351 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-dpVBq-dqv_4LCTp off=8328396 len=351122 2024-04-07T16:48:53.357 INFO:tasks.workunit.client.0.smithi100.stdout:op 3119 completed, throughput=5MB/sec 2024-04-07T16:48:53.357 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-IYg1NPZ6VT-JWzT off=913477 len=297636 2024-04-07T16:48:53.363 INFO:tasks.workunit.client.0.smithi100.stdout:op 3120 completed, throughput=5MB/sec 2024-04-07T16:48:53.363 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-GmAS1Q-Y4xR-G0l off=5889536 len=1879186 2024-04-07T16:48:53.376 INFO:tasks.workunit.client.0.smithi100.stdout:op 3121 completed, throughput=5MB/sec 2024-04-07T16:48:53.377 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-bLkJRu-6IlMMXyZ off=124681 len=1581846 2024-04-07T16:48:53.388 INFO:tasks.workunit.client.0.smithi100.stdout:op 3122 completed, throughput=5MB/sec 2024-04-07T16:48:54.389 INFO:tasks.workunit.client.0.smithi100.stdout: 598: throughput=4.99MB/sec pending data=0 2024-04-07T16:48:54.389 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-ilT40qBOZDLJTvL off=9744086 len=1350579 2024-04-07T16:48:54.392 INFO:tasks.workunit.client.0.smithi100.stdout:op 3123 completed, throughput=5MB/sec 2024-04-07T16:48:54.392 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-eIl-MGeRNpdSifO off=11686414 len=1934939 2024-04-07T16:48:54.399 INFO:tasks.workunit.client.0.smithi100.stdout:op 3124 completed, throughput=5MB/sec 2024-04-07T16:48:54.399 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-6wUTfwVhWGF6qd4 off=4034917 len=1873118 2024-04-07T16:48:54.404 INFO:tasks.workunit.client.0.smithi100.stdout:op 3125 completed, throughput=5MB/sec 2024-04-07T16:48:54.860 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:48:54 smithi144 bash[66654]: cluster 2024-04-07T16:48:53.390813+0000 mgr.y (mgr.25175) 1006 : cluster [DBG] pgmap v1648: 105 pgs: 105 active+clean; 126 GiB data, 7.4 GiB used, 708 GiB / 715 GiB avail; 3.0 MiB/s rd, 2.6 MiB/s wr, 5 op/s 2024-04-07T16:48:54.946 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:48:54 smithi100 bash[74942]: cluster 2024-04-07T16:48:53.390813+0000 mgr.y (mgr.25175) 1006 : cluster [DBG] pgmap v1648: 105 pgs: 105 active+clean; 126 GiB data, 7.4 GiB used, 708 GiB / 715 GiB avail; 3.0 MiB/s rd, 2.6 MiB/s wr, 5 op/s 2024-04-07T16:48:54.946 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:48:54 smithi100 bash[76202]: cluster 2024-04-07T16:48:53.390813+0000 mgr.y (mgr.25175) 1006 : cluster [DBG] pgmap v1648: 105 pgs: 105 active+clean; 126 GiB data, 7.4 GiB used, 708 GiB / 715 GiB avail; 3.0 MiB/s rd, 2.6 MiB/s wr, 5 op/s 2024-04-07T16:48:55.404 INFO:tasks.workunit.client.0.smithi100.stdout: 599: throughput=4.99MB/sec pending data=0 2024-04-07T16:48:55.404 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-1j4KHBRqqZ4nxQL off=3752370 len=13796 2024-04-07T16:48:55.405 INFO:tasks.workunit.client.0.smithi100.stdout:op 3126 completed, throughput=4.99MB/sec 2024-04-07T16:48:55.405 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-VLG12Ptj3Nn5cjI off=15265516 len=613766 2024-04-07T16:48:55.412 INFO:tasks.workunit.client.0.smithi100.stdout:op 3127 completed, throughput=4.99MB/sec 2024-04-07T16:48:55.412 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-6AOP57_4qj5ajL1 off=6387278 len=1699098 2024-04-07T16:48:55.416 INFO:tasks.workunit.client.0.smithi100.stdout:op 3128 completed, throughput=5MB/sec 2024-04-07T16:48:55.416 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-7U2b2z2kznvwWyt off=24139402 len=1227966 2024-04-07T16:48:55.427 INFO:tasks.workunit.client.0.smithi100.stdout:op 3129 completed, throughput=5MB/sec 2024-04-07T16:48:55.427 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-ne49OuE0JS5DEpz off=13719764 len=1886278 2024-04-07T16:48:55.440 INFO:tasks.workunit.client.0.smithi100.stdout:op 3130 completed, throughput=5MB/sec 2024-04-07T16:48:56.440 INFO:tasks.workunit.client.0.smithi100.stdout: 600: throughput=4.99MB/sec pending data=0 2024-04-07T16:48:56.440 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-0MkEdQEwD-c2Wrd off=3180782 len=56728 2024-04-07T16:48:56.445 INFO:tasks.workunit.client.0.smithi100.stdout:op 3131 completed, throughput=4.99MB/sec 2024-04-07T16:48:56.445 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-iq6PpgHlNMaNl14 off=3297061 len=1078782 2024-04-07T16:48:56.455 INFO:tasks.workunit.client.0.smithi100.stdout:op 3132 completed, throughput=5MB/sec 2024-04-07T16:48:56.455 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-lKoWEKA6ohcqc6j off=3317270 len=977727 2024-04-07T16:48:56.464 INFO:tasks.workunit.client.0.smithi100.stdout:op 3133 completed, throughput=5MB/sec 2024-04-07T16:48:56.464 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-eRPb091UTVZpYWR off=1082046 len=1410661 2024-04-07T16:48:56.474 INFO:tasks.workunit.client.0.smithi100.stdout:op 3134 completed, throughput=5MB/sec 2024-04-07T16:48:56.474 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-lO_H2eMGFEapSSQ off=425067 len=1371218 2024-04-07T16:48:56.476 INFO:tasks.workunit.client.0.smithi100.stdout:op 3135 completed, throughput=5MB/sec 2024-04-07T16:48:56.860 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:48:56 smithi144 bash[66654]: cluster 2024-04-07T16:48:55.391791+0000 mgr.y (mgr.25175) 1007 : cluster [DBG] pgmap v1649: 105 pgs: 105 active+clean; 126 GiB data, 7.4 GiB used, 708 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.1 MiB/s wr, 4 op/s 2024-04-07T16:48:56.946 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:48:56 smithi100 bash[74942]: cluster 2024-04-07T16:48:55.391791+0000 mgr.y (mgr.25175) 1007 : cluster [DBG] pgmap v1649: 105 pgs: 105 active+clean; 126 GiB data, 7.4 GiB used, 708 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.1 MiB/s wr, 4 op/s 2024-04-07T16:48:56.946 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:48:56 smithi100 bash[76202]: cluster 2024-04-07T16:48:55.391791+0000 mgr.y (mgr.25175) 1007 : cluster [DBG] pgmap v1649: 105 pgs: 105 active+clean; 126 GiB data, 7.4 GiB used, 708 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.1 MiB/s wr, 4 op/s 2024-04-07T16:48:57.476 INFO:tasks.workunit.client.0.smithi100.stdout: 601: throughput=4.99MB/sec pending data=0 2024-04-07T16:48:57.476 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-PUm6DTo7t9BfNU1 off=2111351 len=63497 2024-04-07T16:48:57.480 INFO:tasks.workunit.client.0.smithi100.stdout:op 3136 completed, throughput=4.99MB/sec 2024-04-07T16:48:57.480 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-A2Qtqh2LfZf9RMQ off=9805874 len=1533638 2024-04-07T16:48:57.493 INFO:tasks.workunit.client.0.smithi100.stdout:op 3137 completed, throughput=4.99MB/sec 2024-04-07T16:48:57.493 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-_H9PA_WfRphE96O off=6531001 len=2019353 2024-04-07T16:48:57.497 INFO:tasks.workunit.client.0.smithi100.stdout:op 3138 completed, throughput=5MB/sec 2024-04-07T16:48:57.497 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-3uCmQmVUkDrZgXN off=2440695 len=1991654 2024-04-07T16:48:57.500 INFO:tasks.workunit.client.0.smithi100.stdout:op 3139 completed, throughput=5MB/sec 2024-04-07T16:48:57.860 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:48:57 smithi144 bash[66654]: audit 2024-04-07T16:48:56.924740+0000 mon.a (mon.0) 2129 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:48:57.946 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:48:57 smithi100 bash[74942]: audit 2024-04-07T16:48:56.924740+0000 mon.a (mon.0) 2129 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:48:57.946 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:48:57 smithi100 bash[76202]: audit 2024-04-07T16:48:56.924740+0000 mon.a (mon.0) 2129 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:48:58.500 INFO:tasks.workunit.client.0.smithi100.stdout: 602: throughput=4.99MB/sec pending data=0 2024-04-07T16:48:58.500 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-WckaJi9eMBJrgDs off=14212151 len=839013 2024-04-07T16:48:58.502 INFO:tasks.workunit.client.0.smithi100.stdout:op 3140 completed, throughput=4.99MB/sec 2024-04-07T16:48:58.503 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-G0BM_mIHV0-oVtr off=15158268 len=1159327 2024-04-07T16:48:58.506 INFO:tasks.workunit.client.0.smithi100.stdout:op 3141 completed, throughput=5MB/sec 2024-04-07T16:48:58.506 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-ulE_JG6sxaaOZh7 off=465217 len=909798 2024-04-07T16:48:58.508 INFO:tasks.workunit.client.0.smithi100.stdout:op 3142 completed, throughput=5MB/sec 2024-04-07T16:48:58.509 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-tp8DV0fkbNM316H off=12304223 len=299724 2024-04-07T16:48:58.510 INFO:tasks.workunit.client.0.smithi100.stdout:op 3143 completed, throughput=5MB/sec 2024-04-07T16:48:58.510 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-ed2VaU6W2GcRUma off=11695846 len=1657071 2024-04-07T16:48:58.514 INFO:tasks.workunit.client.0.smithi100.stdout:op 3144 completed, throughput=5MB/sec 2024-04-07T16:48:58.696 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:48:58 smithi100 bash[74942]: cluster 2024-04-07T16:48:57.392372+0000 mgr.y (mgr.25175) 1008 : cluster [DBG] pgmap v1650: 105 pgs: 105 active+clean; 126 GiB data, 7.4 GiB used, 708 GiB / 715 GiB avail; 2.9 MiB/s rd, 2.4 MiB/s wr, 4 op/s 2024-04-07T16:48:58.696 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:48:58 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:48:58] "GET /metrics HTTP/1.1" 200 34070 "" "Prometheus/2.43.0" 2024-04-07T16:48:58.696 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:48:58 smithi100 bash[76202]: cluster 2024-04-07T16:48:57.392372+0000 mgr.y (mgr.25175) 1008 : cluster [DBG] pgmap v1650: 105 pgs: 105 active+clean; 126 GiB data, 7.4 GiB used, 708 GiB / 715 GiB avail; 2.9 MiB/s rd, 2.4 MiB/s wr, 4 op/s 2024-04-07T16:48:58.860 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:48:58 smithi144 bash[66654]: cluster 2024-04-07T16:48:57.392372+0000 mgr.y (mgr.25175) 1008 : cluster [DBG] pgmap v1650: 105 pgs: 105 active+clean; 126 GiB data, 7.4 GiB used, 708 GiB / 715 GiB avail; 2.9 MiB/s rd, 2.4 MiB/s wr, 4 op/s 2024-04-07T16:48:59.515 INFO:tasks.workunit.client.0.smithi100.stdout: 603: throughput=4.99MB/sec pending data=0 2024-04-07T16:48:59.515 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-4NbuAbtZxU6cp4n off=4355899 len=60734 2024-04-07T16:48:59.520 INFO:tasks.workunit.client.0.smithi100.stdout:op 3145 completed, throughput=4.99MB/sec 2024-04-07T16:48:59.520 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-i-MEah46gglNrnq off=118039 len=1745281 2024-04-07T16:48:59.532 INFO:tasks.workunit.client.0.smithi100.stdout:op 3146 completed, throughput=4.99MB/sec 2024-04-07T16:48:59.533 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-ZYdbqadgO8V-3dk off=5632562 len=1118255 2024-04-07T16:48:59.543 INFO:tasks.workunit.client.0.smithi100.stdout:op 3147 completed, throughput=5MB/sec 2024-04-07T16:48:59.543 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-bBByoixXnaJnEfD off=18851439 len=1209104 2024-04-07T16:48:59.553 INFO:tasks.workunit.client.0.smithi100.stdout:op 3148 completed, throughput=5MB/sec 2024-04-07T16:48:59.553 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-sI6zeOF_TlwApPf off=3890164 len=1054810 2024-04-07T16:48:59.562 INFO:tasks.workunit.client.0.smithi100.stdout:op 3149 completed, throughput=5MB/sec 2024-04-07T16:48:59.563 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-LIXnbQ_evATKvTk off=862214 len=1056868 2024-04-07T16:48:59.571 INFO:tasks.workunit.client.0.smithi100.stdout:op 3150 completed, throughput=5MB/sec 2024-04-07T16:49:00.571 INFO:tasks.workunit.client.0.smithi100.stdout: 604: throughput=4.99MB/sec pending data=0 2024-04-07T16:49:00.572 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-UUWy2-7e6vX7Kkf off=14506235 len=655108 2024-04-07T16:49:00.574 INFO:tasks.workunit.client.0.smithi100.stdout:op 3151 completed, throughput=4.99MB/sec 2024-04-07T16:49:00.574 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-SQ_arVqFtFu_Hdh off=132592 len=871604 2024-04-07T16:49:00.583 INFO:tasks.workunit.client.0.smithi100.stdout:op 3152 completed, throughput=5MB/sec 2024-04-07T16:49:00.583 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-MvBtjqBnV7QPUgy off=919207 len=1866935 2024-04-07T16:49:00.587 INFO:tasks.workunit.client.0.smithi100.stdout:op 3153 completed, throughput=5MB/sec 2024-04-07T16:49:00.587 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-OJuFyTwfVCj9Dht off=667708 len=440501 2024-04-07T16:49:00.594 INFO:tasks.workunit.client.0.smithi100.stdout:op 3154 completed, throughput=5MB/sec 2024-04-07T16:49:00.594 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-ZxsfmiftBKcZWGc off=665327 len=465528 2024-04-07T16:49:00.601 INFO:tasks.workunit.client.0.smithi100.stdout:op 3155 completed, throughput=5MB/sec 2024-04-07T16:49:00.601 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Pzf6HGRAYzkqg_x off=3175730 len=338316 2024-04-07T16:49:00.603 INFO:tasks.workunit.client.0.smithi100.stdout:op 3156 completed, throughput=5MB/sec 2024-04-07T16:49:00.860 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:49:00 smithi144 bash[66654]: cluster 2024-04-07T16:48:59.393189+0000 mgr.y (mgr.25175) 1009 : cluster [DBG] pgmap v1651: 105 pgs: 105 active+clean; 126 GiB data, 7.4 GiB used, 708 GiB / 715 GiB avail; 2.9 MiB/s rd, 2.3 MiB/s wr, 4 op/s 2024-04-07T16:49:00.946 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:49:00 smithi100 bash[74942]: cluster 2024-04-07T16:48:59.393189+0000 mgr.y (mgr.25175) 1009 : cluster [DBG] pgmap v1651: 105 pgs: 105 active+clean; 126 GiB data, 7.4 GiB used, 708 GiB / 715 GiB avail; 2.9 MiB/s rd, 2.3 MiB/s wr, 4 op/s 2024-04-07T16:49:00.946 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:49:00 smithi100 bash[76202]: cluster 2024-04-07T16:48:59.393189+0000 mgr.y (mgr.25175) 1009 : cluster [DBG] pgmap v1651: 105 pgs: 105 active+clean; 126 GiB data, 7.4 GiB used, 708 GiB / 715 GiB avail; 2.9 MiB/s rd, 2.3 MiB/s wr, 4 op/s 2024-04-07T16:49:01.603 INFO:tasks.workunit.client.0.smithi100.stdout: 605: throughput=4.99MB/sec pending data=0 2024-04-07T16:49:01.603 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-DpMbTxJkDMZnV0M off=1295014 len=275301 2024-04-07T16:49:01.605 INFO:tasks.workunit.client.0.smithi100.stdout:op 3157 completed, throughput=4.99MB/sec 2024-04-07T16:49:01.605 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-l8Unfw5JCK_FNlp off=458159 len=270548 2024-04-07T16:49:01.610 INFO:tasks.workunit.client.0.smithi100.stdout:op 3158 completed, throughput=4.99MB/sec 2024-04-07T16:49:01.610 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-tFKDBYvSMvNQ45n off=17195373 len=1251294 2024-04-07T16:49:01.613 INFO:tasks.workunit.client.0.smithi100.stdout:op 3159 completed, throughput=4.99MB/sec 2024-04-07T16:49:01.613 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-kFqcytkA2BJ3eEM off=718170 len=3776 2024-04-07T16:49:01.617 INFO:tasks.workunit.client.0.smithi100.stdout:op 3160 completed, throughput=4.99MB/sec 2024-04-07T16:49:01.617 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Sv6zNIqRaUSoWRV off=7382596 len=1842922 2024-04-07T16:49:01.622 INFO:tasks.workunit.client.0.smithi100.stdout:op 3161 completed, throughput=5MB/sec 2024-04-07T16:49:01.622 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-gYFhjXLhvg1pn_K off=7799785 len=1598252 2024-04-07T16:49:01.627 INFO:tasks.workunit.client.0.smithi100.stdout:op 3162 completed, throughput=5MB/sec 2024-04-07T16:49:02.610 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:49:02 smithi144 bash[95233]: ts=2024-04-07T16:49:02.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:49:02.611 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:49:02 smithi144 bash[66654]: cluster 2024-04-07T16:49:01.394131+0000 mgr.y (mgr.25175) 1010 : cluster [DBG] pgmap v1652: 105 pgs: 105 active+clean; 126 GiB data, 7.4 GiB used, 708 GiB / 715 GiB avail; 3.0 MiB/s rd, 2.5 MiB/s wr, 4 op/s 2024-04-07T16:49:02.611 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:49:02 smithi144 bash[66654]: audit 2024-04-07T16:49:01.773303+0000 mon.a (mon.0) 2130 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:49:02.627 INFO:tasks.workunit.client.0.smithi100.stdout: 606: throughput=4.99MB/sec pending data=0 2024-04-07T16:49:02.627 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-aIuiLLWPerETn4a off=6597609 len=2018628 2024-04-07T16:49:02.630 INFO:tasks.workunit.client.0.smithi100.stdout:op 3163 completed, throughput=5MB/sec 2024-04-07T16:49:02.630 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-1uSU5S6lkjnhDsQ off=6219844 len=1391497 2024-04-07T16:49:02.639 INFO:tasks.workunit.client.0.smithi100.stdout:op 3164 completed, throughput=5MB/sec 2024-04-07T16:49:02.639 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-T902c4i3e-Mnh3Z off=9698335 len=1523408 2024-04-07T16:49:02.641 INFO:tasks.workunit.client.0.smithi100.stdout:op 3165 completed, throughput=5MB/sec 2024-04-07T16:49:02.641 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-pla1HZ9QxD7l4nA off=3645448 len=383239 2024-04-07T16:49:02.642 INFO:tasks.workunit.client.0.smithi100.stdout:op 3166 completed, throughput=5MB/sec 2024-04-07T16:49:02.946 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:49:02 smithi100 bash[74942]: cluster 2024-04-07T16:49:01.394131+0000 mgr.y (mgr.25175) 1010 : cluster [DBG] pgmap v1652: 105 pgs: 105 active+clean; 126 GiB data, 7.4 GiB used, 708 GiB / 715 GiB avail; 3.0 MiB/s rd, 2.5 MiB/s wr, 4 op/s 2024-04-07T16:49:02.946 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:49:02 smithi100 bash[74942]: audit 2024-04-07T16:49:01.773303+0000 mon.a (mon.0) 2130 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:49:02.946 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:49:02 smithi100 bash[76202]: cluster 2024-04-07T16:49:01.394131+0000 mgr.y (mgr.25175) 1010 : cluster [DBG] pgmap v1652: 105 pgs: 105 active+clean; 126 GiB data, 7.4 GiB used, 708 GiB / 715 GiB avail; 3.0 MiB/s rd, 2.5 MiB/s wr, 4 op/s 2024-04-07T16:49:02.947 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:49:02 smithi100 bash[76202]: audit 2024-04-07T16:49:01.773303+0000 mon.a (mon.0) 2130 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:49:03.643 INFO:tasks.workunit.client.0.smithi100.stdout: 607: throughput=4.99MB/sec pending data=0 2024-04-07T16:49:03.643 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-GEh6I4a8tyMFkaf off=5651012 len=695859 2024-04-07T16:49:03.650 INFO:tasks.workunit.client.0.smithi100.stdout:op 3167 completed, throughput=4.99MB/sec 2024-04-07T16:49:03.650 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-HNOIPW6jFt5W2Rb off=20252955 len=1710853 2024-04-07T16:49:03.653 INFO:tasks.workunit.client.0.smithi100.stdout:op 3168 completed, throughput=5MB/sec 2024-04-07T16:49:03.654 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-kp-N3E2edIjEEOv off=13675575 len=268297 2024-04-07T16:49:03.656 INFO:tasks.workunit.client.0.smithi100.stdout:op 3169 completed, throughput=5MB/sec 2024-04-07T16:49:03.656 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-NCW12DCPiRZz-HU off=1928431 len=1496635 2024-04-07T16:49:03.670 INFO:tasks.workunit.client.0.smithi100.stdout:op 3170 completed, throughput=5MB/sec 2024-04-07T16:49:03.670 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-d8xfU4StTFj-_SJ off=5794703 len=873183 2024-04-07T16:49:03.672 INFO:tasks.workunit.client.0.smithi100.stdout:op 3171 completed, throughput=5MB/sec 2024-04-07T16:49:03.673 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-1o7u46V4y4qldDj off=11792670 len=1312813 2024-04-07T16:49:03.676 INFO:tasks.workunit.client.0.smithi100.stdout:op 3172 completed, throughput=5MB/sec 2024-04-07T16:49:03.860 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:49:03 smithi144 bash[66654]: audit 2024-04-07T16:49:02.577585+0000 mon.a (mon.0) 2131 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:49:03.861 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:49:03 smithi144 bash[66654]: audit 2024-04-07T16:49:02.584941+0000 mon.a (mon.0) 2132 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:49:03.861 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:49:03 smithi144 bash[66654]: audit 2024-04-07T16:49:02.839779+0000 mon.a (mon.0) 2133 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:49:03.861 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:49:03 smithi144 bash[66654]: audit 2024-04-07T16:49:02.851940+0000 mon.a (mon.0) 2134 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:49:03.861 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:49:03 smithi144 bash[66654]: audit 2024-04-07T16:49:03.488349+0000 mon.a (mon.0) 2135 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:49:03.861 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:49:03 smithi144 bash[66654]: audit 2024-04-07T16:49:03.490354+0000 mon.a (mon.0) 2136 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:49:03.861 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:49:03 smithi144 bash[66654]: audit 2024-04-07T16:49:03.500943+0000 mon.a (mon.0) 2137 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:49:03.946 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:49:03 smithi100 bash[74942]: audit 2024-04-07T16:49:02.577585+0000 mon.a (mon.0) 2131 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:49:03.946 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:49:03 smithi100 bash[74942]: audit 2024-04-07T16:49:02.584941+0000 mon.a (mon.0) 2132 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:49:03.946 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:49:03 smithi100 bash[74942]: audit 2024-04-07T16:49:02.839779+0000 mon.a (mon.0) 2133 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:49:03.947 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:49:03 smithi100 bash[74942]: audit 2024-04-07T16:49:02.851940+0000 mon.a (mon.0) 2134 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:49:03.947 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:49:03 smithi100 bash[74942]: audit 2024-04-07T16:49:03.488349+0000 mon.a (mon.0) 2135 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:49:03.947 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:49:03 smithi100 bash[74942]: audit 2024-04-07T16:49:03.490354+0000 mon.a (mon.0) 2136 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:49:03.947 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:49:03 smithi100 bash[74942]: audit 2024-04-07T16:49:03.500943+0000 mon.a (mon.0) 2137 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:49:03.947 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:49:03 smithi100 bash[76202]: audit 2024-04-07T16:49:02.577585+0000 mon.a (mon.0) 2131 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:49:03.947 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:49:03 smithi100 bash[76202]: audit 2024-04-07T16:49:02.584941+0000 mon.a (mon.0) 2132 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:49:03.947 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:49:03 smithi100 bash[76202]: audit 2024-04-07T16:49:02.839779+0000 mon.a (mon.0) 2133 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:49:03.947 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:49:03 smithi100 bash[76202]: audit 2024-04-07T16:49:02.851940+0000 mon.a (mon.0) 2134 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:49:03.947 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:49:03 smithi100 bash[76202]: audit 2024-04-07T16:49:03.488349+0000 mon.a (mon.0) 2135 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:49:03.947 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:49:03 smithi100 bash[76202]: audit 2024-04-07T16:49:03.490354+0000 mon.a (mon.0) 2136 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:49:03.948 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:49:03 smithi100 bash[76202]: audit 2024-04-07T16:49:03.500943+0000 mon.a (mon.0) 2137 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:49:04.676 INFO:tasks.workunit.client.0.smithi100.stdout: 608: throughput=4.99MB/sec pending data=0 2024-04-07T16:49:04.677 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-1FE6D7WXEz94Qd3 off=869481 len=1792169 2024-04-07T16:49:04.690 INFO:tasks.workunit.client.0.smithi100.stdout:op 3173 completed, throughput=5MB/sec 2024-04-07T16:49:04.690 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-ISXkk1zWuoWQSNr off=3161370 len=1467765 2024-04-07T16:49:04.701 INFO:tasks.workunit.client.0.smithi100.stdout:op 3174 completed, throughput=5MB/sec 2024-04-07T16:49:04.701 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-DsUHl3N7Hzr0EhM off=12323166 len=2052854 2024-04-07T16:49:04.716 INFO:tasks.workunit.client.0.smithi100.stdout:op 3175 completed, throughput=5MB/sec 2024-04-07T16:49:04.860 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:49:04 smithi144 bash[66654]: cluster 2024-04-07T16:49:03.395251+0000 mgr.y (mgr.25175) 1011 : cluster [DBG] pgmap v1653: 105 pgs: 105 active+clean; 126 GiB data, 7.4 GiB used, 708 GiB / 715 GiB avail; 3.0 MiB/s rd, 2.7 MiB/s wr, 5 op/s 2024-04-07T16:49:04.946 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:49:04 smithi100 bash[74942]: cluster 2024-04-07T16:49:03.395251+0000 mgr.y (mgr.25175) 1011 : cluster [DBG] pgmap v1653: 105 pgs: 105 active+clean; 126 GiB data, 7.4 GiB used, 708 GiB / 715 GiB avail; 3.0 MiB/s rd, 2.7 MiB/s wr, 5 op/s 2024-04-07T16:49:04.946 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:49:04 smithi100 bash[76202]: cluster 2024-04-07T16:49:03.395251+0000 mgr.y (mgr.25175) 1011 : cluster [DBG] pgmap v1653: 105 pgs: 105 active+clean; 126 GiB data, 7.4 GiB used, 708 GiB / 715 GiB avail; 3.0 MiB/s rd, 2.7 MiB/s wr, 5 op/s 2024-04-07T16:49:05.717 INFO:tasks.workunit.client.0.smithi100.stdout: 609: throughput=4.99MB/sec pending data=0 2024-04-07T16:49:05.717 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-fwjUab9z5Md6SRk off=14631461 len=1330765 2024-04-07T16:49:05.721 INFO:tasks.workunit.client.0.smithi100.stdout:op 3176 completed, throughput=5MB/sec 2024-04-07T16:49:05.721 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Yi4BpsWvxksH03t off=3780520 len=604418 2024-04-07T16:49:05.724 INFO:tasks.workunit.client.0.smithi100.stdout:op 3177 completed, throughput=5MB/sec 2024-04-07T16:49:05.724 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-t5IojFagmVKaRW1 off=4988611 len=1440878 2024-04-07T16:49:05.727 INFO:tasks.workunit.client.0.smithi100.stdout:op 3178 completed, throughput=5MB/sec 2024-04-07T16:49:05.728 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-7tLo14hWqDsJ9_Y off=17467891 len=970264 2024-04-07T16:49:05.730 INFO:tasks.workunit.client.0.smithi100.stdout:op 3179 completed, throughput=5MB/sec 2024-04-07T16:49:05.731 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-7wzyKs9PeeAF0FG off=169374 len=1552830 2024-04-07T16:49:05.736 INFO:tasks.workunit.client.0.smithi100.stdout:op 3180 completed, throughput=5MB/sec 2024-04-07T16:49:06.737 INFO:tasks.workunit.client.0.smithi100.stdout: 610: throughput=4.99MB/sec pending data=0 2024-04-07T16:49:06.737 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-ZquaDJSG8Vf3Dmp off=22521589 len=1099189 2024-04-07T16:49:06.740 INFO:tasks.workunit.client.0.smithi100.stdout:op 3181 completed, throughput=5MB/sec 2024-04-07T16:49:06.740 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-VBoVPoRXk0ncuVB off=16616274 len=1411768 2024-04-07T16:49:06.752 INFO:tasks.workunit.client.0.smithi100.stdout:op 3182 completed, throughput=5MB/sec 2024-04-07T16:49:06.752 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-sxSPO58EddMX4NH off=1679035 len=212198 2024-04-07T16:49:06.758 INFO:tasks.workunit.client.0.smithi100.stdout:op 3183 completed, throughput=5MB/sec 2024-04-07T16:49:06.758 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-ib3NAmiolnTnZhv off=3394899 len=1539688 2024-04-07T16:49:06.761 INFO:tasks.workunit.client.0.smithi100.stdout:op 3184 completed, throughput=5MB/sec 2024-04-07T16:49:06.860 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:49:06 smithi144 bash[66654]: cluster 2024-04-07T16:49:05.396546+0000 mgr.y (mgr.25175) 1012 : cluster [DBG] pgmap v1654: 105 pgs: 105 active+clean; 126 GiB data, 7.4 GiB used, 708 GiB / 715 GiB avail; 2.9 MiB/s rd, 2.1 MiB/s wr, 4 op/s 2024-04-07T16:49:06.946 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:49:06 smithi100 bash[74942]: cluster 2024-04-07T16:49:05.396546+0000 mgr.y (mgr.25175) 1012 : cluster [DBG] pgmap v1654: 105 pgs: 105 active+clean; 126 GiB data, 7.4 GiB used, 708 GiB / 715 GiB avail; 2.9 MiB/s rd, 2.1 MiB/s wr, 4 op/s 2024-04-07T16:49:06.947 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:49:06 smithi100 bash[76202]: cluster 2024-04-07T16:49:05.396546+0000 mgr.y (mgr.25175) 1012 : cluster [DBG] pgmap v1654: 105 pgs: 105 active+clean; 126 GiB data, 7.4 GiB used, 708 GiB / 715 GiB avail; 2.9 MiB/s rd, 2.1 MiB/s wr, 4 op/s 2024-04-07T16:49:07.761 INFO:tasks.workunit.client.0.smithi100.stdout: 611: throughput=4.99MB/sec pending data=0 2024-04-07T16:49:07.762 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-BtjO_g6f0bJOU0s off=757666 len=1004150 2024-04-07T16:49:07.764 INFO:tasks.workunit.client.0.smithi100.stdout:op 3185 completed, throughput=4.99MB/sec 2024-04-07T16:49:07.764 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-rGJcMliULM_M51o off=12156301 len=2049863 2024-04-07T16:49:07.769 INFO:tasks.workunit.client.0.smithi100.stdout:op 3186 completed, throughput=5MB/sec 2024-04-07T16:49:07.769 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-ulkRXJD5TaR_NcO off=82855 len=1871903 2024-04-07T16:49:07.776 INFO:tasks.workunit.client.0.smithi100.stdout:op 3187 completed, throughput=5MB/sec 2024-04-07T16:49:07.776 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-3drUB4rsj64KNpa off=3152350 len=128749 2024-04-07T16:49:07.777 INFO:tasks.workunit.client.0.smithi100.stdout:op 3188 completed, throughput=5MB/sec 2024-04-07T16:49:07.777 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-FmTjatdovMU5_gm off=1844665 len=893024 2024-04-07T16:49:07.780 INFO:tasks.workunit.client.0.smithi100.stdout:op 3189 completed, throughput=5MB/sec 2024-04-07T16:49:08.696 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:49:08 smithi100 bash[74942]: cluster 2024-04-07T16:49:07.397686+0000 mgr.y (mgr.25175) 1013 : cluster [DBG] pgmap v1655: 105 pgs: 105 active+clean; 126 GiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 2.9 MiB/s rd, 2.2 MiB/s wr, 4 op/s 2024-04-07T16:49:08.696 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:49:08 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:49:08] "GET /metrics HTTP/1.1" 200 34072 "" "Prometheus/2.43.0" 2024-04-07T16:49:08.696 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:49:08 smithi100 bash[76202]: cluster 2024-04-07T16:49:07.397686+0000 mgr.y (mgr.25175) 1013 : cluster [DBG] pgmap v1655: 105 pgs: 105 active+clean; 126 GiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 2.9 MiB/s rd, 2.2 MiB/s wr, 4 op/s 2024-04-07T16:49:08.780 INFO:tasks.workunit.client.0.smithi100.stdout: 612: throughput=4.99MB/sec pending data=0 2024-04-07T16:49:08.780 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-u85Rfbck6DkURrs off=14305523 len=1901367 2024-04-07T16:49:08.795 INFO:tasks.workunit.client.0.smithi100.stdout:op 3190 completed, throughput=5MB/sec 2024-04-07T16:49:08.795 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-dFYOhedaSIIoiAU off=6022239 len=1652759 2024-04-07T16:49:08.800 INFO:tasks.workunit.client.0.smithi100.stdout:op 3191 completed, throughput=5MB/sec 2024-04-07T16:49:08.801 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-FmTjatdovMU5_gm off=997401 len=681361 2024-04-07T16:49:08.807 INFO:tasks.workunit.client.0.smithi100.stdout:op 3192 completed, throughput=5MB/sec 2024-04-07T16:49:08.807 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-aj_7xnrmD-lStTw off=546858 len=713763 2024-04-07T16:49:08.815 INFO:tasks.workunit.client.0.smithi100.stdout:op 3193 completed, throughput=5MB/sec 2024-04-07T16:49:08.860 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:49:08 smithi144 bash[66654]: cluster 2024-04-07T16:49:07.397686+0000 mgr.y (mgr.25175) 1013 : cluster [DBG] pgmap v1655: 105 pgs: 105 active+clean; 126 GiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 2.9 MiB/s rd, 2.2 MiB/s wr, 4 op/s 2024-04-07T16:49:09.815 INFO:tasks.workunit.client.0.smithi100.stdout: 613: throughput=4.99MB/sec pending data=0 2024-04-07T16:49:09.815 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-QgGCsNuaGT5WRDW off=68501 len=1219802 2024-04-07T16:49:09.819 INFO:tasks.workunit.client.0.smithi100.stdout:op 3194 completed, throughput=4.99MB/sec 2024-04-07T16:49:09.819 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-1gsci6oxalFHYcn off=12889654 len=410612 2024-04-07T16:49:09.821 INFO:tasks.workunit.client.0.smithi100.stdout:op 3195 completed, throughput=5MB/sec 2024-04-07T16:49:09.821 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-DFBHHtCofuso79u off=16784412 len=415009 2024-04-07T16:49:09.823 INFO:tasks.workunit.client.0.smithi100.stdout:op 3196 completed, throughput=5MB/sec 2024-04-07T16:49:09.823 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-a7dZDPrLz8eTLrB off=3006860 len=1431349 2024-04-07T16:49:09.834 INFO:tasks.workunit.client.0.smithi100.stdout:op 3197 completed, throughput=5MB/sec 2024-04-07T16:49:09.834 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-JJT--0e_dmFBey9 off=5287419 len=57829 2024-04-07T16:49:09.838 INFO:tasks.workunit.client.0.smithi100.stdout:op 3198 completed, throughput=5MB/sec 2024-04-07T16:49:09.839 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-r0upF2k71FL-7PL off=5116498 len=986051 2024-04-07T16:49:09.847 INFO:tasks.workunit.client.0.smithi100.stdout:op 3199 completed, throughput=5MB/sec 2024-04-07T16:49:09.847 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Bfsb6SdFWMjXYDR off=2472997 len=186047 2024-04-07T16:49:09.849 INFO:tasks.workunit.client.0.smithi100.stdout:op 3200 completed, throughput=5MB/sec 2024-04-07T16:49:09.849 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-FKPgPqJAgGCk7_7 off=10192625 len=1091784 2024-04-07T16:49:09.850 INFO:tasks.workunit.client.0.smithi100.stdout:op 3201 completed, throughput=5MB/sec 2024-04-07T16:49:10.851 INFO:tasks.workunit.client.0.smithi100.stdout: 614: throughput=4.99MB/sec pending data=0 2024-04-07T16:49:10.851 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-JM1Wrd6nxXAKvuO off=4508908 len=1377236 2024-04-07T16:49:10.856 INFO:tasks.workunit.client.0.smithi100.stdout:op 3202 completed, throughput=5MB/sec 2024-04-07T16:49:10.856 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-e6ku_Cj3JHyxRde off=8430676 len=220566 2024-04-07T16:49:10.860 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:49:10 smithi144 bash[66654]: cluster 2024-04-07T16:49:09.398528+0000 mgr.y (mgr.25175) 1014 : cluster [DBG] pgmap v1656: 105 pgs: 105 active+clean; 126 GiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 2.8 MiB/s rd, 2.0 MiB/s wr, 4 op/s 2024-04-07T16:49:10.861 INFO:tasks.workunit.client.0.smithi100.stdout:op 3203 completed, throughput=5MB/sec 2024-04-07T16:49:10.861 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Pw2ALW7QA6xzmiD off=2019893 len=1369172 2024-04-07T16:49:10.865 INFO:tasks.workunit.client.0.smithi100.stdout:op 3204 completed, throughput=5MB/sec 2024-04-07T16:49:10.865 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-iW-UNG8htdhYqLD off=946237 len=666098 2024-04-07T16:49:10.867 INFO:tasks.workunit.client.0.smithi100.stdout:op 3205 completed, throughput=5MB/sec 2024-04-07T16:49:10.867 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-KPA3d6J1PiCfRLi off=2131353 len=2048923 2024-04-07T16:49:10.884 INFO:tasks.workunit.client.0.smithi100.stdout:op 3206 completed, throughput=5MB/sec 2024-04-07T16:49:10.946 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:49:10 smithi100 bash[74942]: cluster 2024-04-07T16:49:09.398528+0000 mgr.y (mgr.25175) 1014 : cluster [DBG] pgmap v1656: 105 pgs: 105 active+clean; 126 GiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 2.8 MiB/s rd, 2.0 MiB/s wr, 4 op/s 2024-04-07T16:49:10.946 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:49:10 smithi100 bash[76202]: cluster 2024-04-07T16:49:09.398528+0000 mgr.y (mgr.25175) 1014 : cluster [DBG] pgmap v1656: 105 pgs: 105 active+clean; 126 GiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 2.8 MiB/s rd, 2.0 MiB/s wr, 4 op/s 2024-04-07T16:49:11.884 INFO:tasks.workunit.client.0.smithi100.stdout: 615: throughput=4.99MB/sec pending data=0 2024-04-07T16:49:11.885 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-3q3BOWqNsQYhKTZ off=3197130 len=517144 2024-04-07T16:49:11.887 INFO:tasks.workunit.client.0.smithi100.stdout:op 3207 completed, throughput=4.99MB/sec 2024-04-07T16:49:11.887 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-XPc7D5_FO4xQAKE off=3041347 len=1218328 2024-04-07T16:49:11.891 INFO:tasks.workunit.client.0.smithi100.stdout:op 3208 completed, throughput=5MB/sec 2024-04-07T16:49:11.891 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-H4TxiyRbsM3BQJC off=1761899 len=1277659 2024-04-07T16:49:11.904 INFO:tasks.workunit.client.0.smithi100.stdout:op 3209 completed, throughput=5MB/sec 2024-04-07T16:49:11.904 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-lxhoxx-QuiO-EHm off=2824808 len=2096516 2024-04-07T16:49:11.919 INFO:tasks.workunit.client.0.smithi100.stdout:op 3210 completed, throughput=5MB/sec 2024-04-07T16:49:12.591 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:49:12 smithi144 bash[95233]: ts=2024-04-07T16:49:12.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:49:12.860 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:49:12 smithi144 bash[66654]: cluster 2024-04-07T16:49:11.399474+0000 mgr.y (mgr.25175) 1015 : cluster [DBG] pgmap v1657: 105 pgs: 105 active+clean; 126 GiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 2.9 MiB/s rd, 2.1 MiB/s wr, 4 op/s 2024-04-07T16:49:12.919 INFO:tasks.workunit.client.0.smithi100.stdout: 616: throughput=4.99MB/sec pending data=0 2024-04-07T16:49:12.919 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-R3NL09FW_whjw8U off=20698789 len=1475261 2024-04-07T16:49:12.931 INFO:tasks.workunit.client.0.smithi100.stdout:op 3211 completed, throughput=5MB/sec 2024-04-07T16:49:12.931 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-CIagMSkomQk_7gu off=12023383 len=162175 2024-04-07T16:49:12.936 INFO:tasks.workunit.client.0.smithi100.stdout:op 3212 completed, throughput=5MB/sec 2024-04-07T16:49:12.936 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-_9NBRNgAc7bcwst off=639321 len=177599 2024-04-07T16:49:12.937 INFO:tasks.workunit.client.0.smithi100.stdout:op 3213 completed, throughput=5MB/sec 2024-04-07T16:49:12.937 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-QqsRSaX-KEL9Xkv off=1397283 len=1456791 2024-04-07T16:49:12.941 INFO:tasks.workunit.client.0.smithi100.stdout:op 3214 completed, throughput=5MB/sec 2024-04-07T16:49:12.941 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-xTl2Z4TWXLDkBwW off=6672131 len=806759 2024-04-07T16:49:12.946 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:49:12 smithi100 bash[74942]: cluster 2024-04-07T16:49:11.399474+0000 mgr.y (mgr.25175) 1015 : cluster [DBG] pgmap v1657: 105 pgs: 105 active+clean; 126 GiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 2.9 MiB/s rd, 2.1 MiB/s wr, 4 op/s 2024-04-07T16:49:12.946 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:49:12 smithi100 bash[76202]: cluster 2024-04-07T16:49:11.399474+0000 mgr.y (mgr.25175) 1015 : cluster [DBG] pgmap v1657: 105 pgs: 105 active+clean; 126 GiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 2.9 MiB/s rd, 2.1 MiB/s wr, 4 op/s 2024-04-07T16:49:12.948 INFO:tasks.workunit.client.0.smithi100.stdout:op 3215 completed, throughput=5MB/sec 2024-04-07T16:49:12.948 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-zi5jElbm2RQySIO off=1715642 len=1868706 2024-04-07T16:49:12.952 INFO:tasks.workunit.client.0.smithi100.stdout:op 3216 completed, throughput=5MB/sec 2024-04-07T16:49:13.953 INFO:tasks.workunit.client.0.smithi100.stdout: 617: throughput=4.99MB/sec pending data=0 2024-04-07T16:49:13.953 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-rZSAegLUEoyoVYn off=8129468 len=1305651 2024-04-07T16:49:13.964 INFO:tasks.workunit.client.0.smithi100.stdout:op 3217 completed, throughput=5MB/sec 2024-04-07T16:49:13.964 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-GbMmlydNGcsQyRJ off=6413243 len=24889 2024-04-07T16:49:13.964 INFO:tasks.workunit.client.0.smithi100.stdout:op 3218 completed, throughput=5MB/sec 2024-04-07T16:49:13.964 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-SN_NgPAcCn14aLc off=4166271 len=1312644 2024-04-07T16:49:13.976 INFO:tasks.workunit.client.0.smithi100.stdout:op 3219 completed, throughput=5MB/sec 2024-04-07T16:49:13.976 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-JkAecA1ajePt-Bo off=15117002 len=1487459 2024-04-07T16:49:13.979 INFO:tasks.workunit.client.0.smithi100.stdout:op 3220 completed, throughput=5MB/sec 2024-04-07T16:49:13.979 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-DpMbTxJkDMZnV0M off=6888044 len=1362934 2024-04-07T16:49:13.981 INFO:tasks.workunit.client.0.smithi100.stdout:op 3221 completed, throughput=5MB/sec 2024-04-07T16:49:14.860 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:49:14 smithi144 bash[66654]: cluster 2024-04-07T16:49:13.400619+0000 mgr.y (mgr.25175) 1016 : cluster [DBG] pgmap v1658: 105 pgs: 105 active+clean; 126 GiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 3.2 MiB/s rd, 2.0 MiB/s wr, 5 op/s 2024-04-07T16:49:14.946 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:49:14 smithi100 bash[74942]: cluster 2024-04-07T16:49:13.400619+0000 mgr.y (mgr.25175) 1016 : cluster [DBG] pgmap v1658: 105 pgs: 105 active+clean; 126 GiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 3.2 MiB/s rd, 2.0 MiB/s wr, 5 op/s 2024-04-07T16:49:14.946 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:49:14 smithi100 bash[76202]: cluster 2024-04-07T16:49:13.400619+0000 mgr.y (mgr.25175) 1016 : cluster [DBG] pgmap v1658: 105 pgs: 105 active+clean; 126 GiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 3.2 MiB/s rd, 2.0 MiB/s wr, 5 op/s 2024-04-07T16:49:14.982 INFO:tasks.workunit.client.0.smithi100.stdout: 618: throughput=4.99MB/sec pending data=0 2024-04-07T16:49:14.982 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-mYewQhNGkyMgD3M off=3787219 len=449914 2024-04-07T16:49:14.984 INFO:tasks.workunit.client.0.smithi100.stdout:op 3222 completed, throughput=4.99MB/sec 2024-04-07T16:49:14.984 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-dLScp00Wmtbx7eI off=8012519 len=236202 2024-04-07T16:49:14.985 INFO:tasks.workunit.client.0.smithi100.stdout:op 3223 completed, throughput=5MB/sec 2024-04-07T16:49:14.985 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Clx4ailo7Ah89U5 off=1058025 len=1711815 2024-04-07T16:49:15.001 INFO:tasks.workunit.client.0.smithi100.stdout:op 3224 completed, throughput=5MB/sec 2024-04-07T16:49:15.001 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-pgZPpJjDY4yto-p off=1810323 len=1035170 2024-04-07T16:49:15.009 INFO:tasks.workunit.client.0.smithi100.stdout:op 3225 completed, throughput=5MB/sec 2024-04-07T16:49:15.009 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-jXTugkeKi6LPS9c off=4277524 len=948489 2024-04-07T16:49:15.018 INFO:tasks.workunit.client.0.smithi100.stdout:op 3226 completed, throughput=5MB/sec 2024-04-07T16:49:16.018 INFO:tasks.workunit.client.0.smithi100.stdout: 619: throughput=4.99MB/sec pending data=0 2024-04-07T16:49:16.019 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-bB4LXcHzpSlwTLu off=6189165 len=1626598 2024-04-07T16:49:16.030 INFO:tasks.workunit.client.0.smithi100.stdout:op 3227 completed, throughput=4.99MB/sec 2024-04-07T16:49:16.030 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-MYWY-pT0z2jaV1A off=385194 len=1659006 2024-04-07T16:49:16.042 INFO:tasks.workunit.client.0.smithi100.stdout:op 3228 completed, throughput=5MB/sec 2024-04-07T16:49:16.043 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-KRop707aFQ4H0yh off=1141877 len=63200 2024-04-07T16:49:16.047 INFO:tasks.workunit.client.0.smithi100.stdout:op 3229 completed, throughput=5MB/sec 2024-04-07T16:49:16.047 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-c-OKB2UZnvipjnp off=5470016 len=1011556 2024-04-07T16:49:16.049 INFO:tasks.workunit.client.0.smithi100.stdout:op 3230 completed, throughput=5MB/sec 2024-04-07T16:49:16.049 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-zkLoh1wxe25k9Vc off=9053957 len=1491232 2024-04-07T16:49:16.054 INFO:tasks.workunit.client.0.smithi100.stdout:op 3231 completed, throughput=5MB/sec 2024-04-07T16:49:16.860 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:49:16 smithi144 bash[66654]: cluster 2024-04-07T16:49:15.401597+0000 mgr.y (mgr.25175) 1017 : cluster [DBG] pgmap v1659: 105 pgs: 105 active+clean; 126 GiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 2.8 MiB/s rd, 2.0 MiB/s wr, 4 op/s 2024-04-07T16:49:16.946 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:49:16 smithi100 bash[74942]: cluster 2024-04-07T16:49:15.401597+0000 mgr.y (mgr.25175) 1017 : cluster [DBG] pgmap v1659: 105 pgs: 105 active+clean; 126 GiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 2.8 MiB/s rd, 2.0 MiB/s wr, 4 op/s 2024-04-07T16:49:16.946 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:49:16 smithi100 bash[76202]: cluster 2024-04-07T16:49:15.401597+0000 mgr.y (mgr.25175) 1017 : cluster [DBG] pgmap v1659: 105 pgs: 105 active+clean; 126 GiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 2.8 MiB/s rd, 2.0 MiB/s wr, 4 op/s 2024-04-07T16:49:17.054 INFO:tasks.workunit.client.0.smithi100.stdout: 620: throughput=4.99MB/sec pending data=0 2024-04-07T16:49:17.055 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-3bdP2JDUVBpEZXk off=5116512 len=1809467 2024-04-07T16:49:17.060 INFO:tasks.workunit.client.0.smithi100.stdout:op 3232 completed, throughput=5MB/sec 2024-04-07T16:49:17.060 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-YwLInpnflv2ZQZF off=15728547 len=1166034 2024-04-07T16:49:17.064 INFO:tasks.workunit.client.0.smithi100.stdout:op 3233 completed, throughput=5MB/sec 2024-04-07T16:49:17.064 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-n-F2jcGchdvdc0y off=8106322 len=1932649 2024-04-07T16:49:17.068 INFO:tasks.workunit.client.0.smithi100.stdout:op 3234 completed, throughput=5MB/sec 2024-04-07T16:49:17.860 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:49:17 smithi144 bash[66654]: audit 2024-04-07T16:49:16.773710+0000 mon.a (mon.0) 2138 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:49:17.946 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:49:17 smithi100 bash[74942]: audit 2024-04-07T16:49:16.773710+0000 mon.a (mon.0) 2138 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:49:17.946 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:49:17 smithi100 bash[76202]: audit 2024-04-07T16:49:16.773710+0000 mon.a (mon.0) 2138 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:49:18.068 INFO:tasks.workunit.client.0.smithi100.stdout: 621: throughput=4.99MB/sec pending data=0 2024-04-07T16:49:18.069 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-3HNVPWEOQRSRf1b off=2531013 len=402001 2024-04-07T16:49:18.074 INFO:tasks.workunit.client.0.smithi100.stdout:op 3235 completed, throughput=4.99MB/sec 2024-04-07T16:49:18.074 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-apiMJii1OpMoFYQ off=1746701 len=268955 2024-04-07T16:49:18.079 INFO:tasks.workunit.client.0.smithi100.stdout:op 3236 completed, throughput=4.99MB/sec 2024-04-07T16:49:18.079 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-I0Wgar4MBcJtlgW off=1127497 len=960891 2024-04-07T16:49:18.082 INFO:tasks.workunit.client.0.smithi100.stdout:op 3237 completed, throughput=4.99MB/sec 2024-04-07T16:49:18.082 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-irKTvJ_ErGLNre2 off=14800541 len=1183134 2024-04-07T16:49:18.091 INFO:tasks.workunit.client.0.smithi100.stdout:op 3238 completed, throughput=5MB/sec 2024-04-07T16:49:18.091 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-BNomQioi5dDnZ4Z off=10667954 len=1066583 2024-04-07T16:49:18.094 INFO:tasks.workunit.client.0.smithi100.stdout:op 3239 completed, throughput=5MB/sec 2024-04-07T16:49:18.094 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-wQP6S1m6gjk_uo2 off=6296722 len=1636907 2024-04-07T16:49:18.106 INFO:tasks.workunit.client.0.smithi100.stdout:op 3240 completed, throughput=5MB/sec 2024-04-07T16:49:18.696 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:49:18 smithi100 bash[74942]: cluster 2024-04-07T16:49:17.402589+0000 mgr.y (mgr.25175) 1018 : cluster [DBG] pgmap v1660: 105 pgs: 105 active+clean; 126 GiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.1 MiB/s wr, 4 op/s 2024-04-07T16:49:18.696 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:49:18 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:49:18] "GET /metrics HTTP/1.1" 200 34068 "" "Prometheus/2.43.0" 2024-04-07T16:49:18.697 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:49:18 smithi100 bash[76202]: cluster 2024-04-07T16:49:17.402589+0000 mgr.y (mgr.25175) 1018 : cluster [DBG] pgmap v1660: 105 pgs: 105 active+clean; 126 GiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.1 MiB/s wr, 4 op/s 2024-04-07T16:49:19.107 INFO:tasks.workunit.client.0.smithi100.stdout: 622: throughput=4.99MB/sec pending data=0 2024-04-07T16:49:19.107 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-gtJip3GCBehPqeE off=18650046 len=1529919 2024-04-07T16:49:19.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:49:18 smithi144 bash[66654]: cluster 2024-04-07T16:49:17.402589+0000 mgr.y (mgr.25175) 1018 : cluster [DBG] pgmap v1660: 105 pgs: 105 active+clean; 126 GiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.1 MiB/s wr, 4 op/s 2024-04-07T16:49:19.111 INFO:tasks.workunit.client.0.smithi100.stdout:op 3241 completed, throughput=4.99MB/sec 2024-04-07T16:49:19.111 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-cj0nrFHyBr-yCOO off=19844145 len=861365 2024-04-07T16:49:19.114 INFO:tasks.workunit.client.0.smithi100.stdout:op 3242 completed, throughput=5MB/sec 2024-04-07T16:49:19.114 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-WJVKoV4CTLWonpq off=2852337 len=1692181 2024-04-07T16:49:19.118 INFO:tasks.workunit.client.0.smithi100.stdout:op 3243 completed, throughput=5MB/sec 2024-04-07T16:49:19.118 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-4qpYS35Fe6dTcxk off=4550945 len=1773323 2024-04-07T16:49:19.131 INFO:tasks.workunit.client.0.smithi100.stdout:op 3244 completed, throughput=5MB/sec 2024-04-07T16:49:20.131 INFO:tasks.workunit.client.0.smithi100.stdout: 623: throughput=4.99MB/sec pending data=0 2024-04-07T16:49:20.131 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-8uKjV_kCIXWpegd off=706589 len=1409664 2024-04-07T16:49:20.136 INFO:tasks.workunit.client.0.smithi100.stdout:op 3245 completed, throughput=5MB/sec 2024-04-07T16:49:20.136 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-LETG8N-IOyreiKE off=8576824 len=1504829 2024-04-07T16:49:20.141 INFO:tasks.workunit.client.0.smithi100.stdout:op 3246 completed, throughput=5MB/sec 2024-04-07T16:49:20.141 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-GBxEJZ98LZnfzL3 off=6183357 len=1233751 2024-04-07T16:49:20.152 INFO:tasks.workunit.client.0.smithi100.stdout:op 3247 completed, throughput=5MB/sec 2024-04-07T16:49:20.153 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-0gliZjf5hddKNXi off=17698483 len=792583 2024-04-07T16:49:20.162 INFO:tasks.workunit.client.0.smithi100.stdout:op 3248 completed, throughput=5MB/sec 2024-04-07T16:49:20.946 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:49:20 smithi100 bash[74942]: cluster 2024-04-07T16:49:19.403476+0000 mgr.y (mgr.25175) 1019 : cluster [DBG] pgmap v1661: 105 pgs: 105 active+clean; 126 GiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 2.6 MiB/s rd, 2.1 MiB/s wr, 4 op/s 2024-04-07T16:49:20.947 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:49:20 smithi100 bash[76202]: cluster 2024-04-07T16:49:19.403476+0000 mgr.y (mgr.25175) 1019 : cluster [DBG] pgmap v1661: 105 pgs: 105 active+clean; 126 GiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 2.6 MiB/s rd, 2.1 MiB/s wr, 4 op/s 2024-04-07T16:49:21.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:49:20 smithi144 bash[66654]: cluster 2024-04-07T16:49:19.403476+0000 mgr.y (mgr.25175) 1019 : cluster [DBG] pgmap v1661: 105 pgs: 105 active+clean; 126 GiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 2.6 MiB/s rd, 2.1 MiB/s wr, 4 op/s 2024-04-07T16:49:21.162 INFO:tasks.workunit.client.0.smithi100.stdout: 624: throughput=4.99MB/sec pending data=0 2024-04-07T16:49:21.162 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-HNP4upH43tAUaz4 off=3875518 len=544295 2024-04-07T16:49:21.169 INFO:tasks.workunit.client.0.smithi100.stdout:op 3249 completed, throughput=4.99MB/sec 2024-04-07T16:49:21.169 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-SBIFf1yc1fshROo off=2269927 len=1742590 2024-04-07T16:49:21.173 INFO:tasks.workunit.client.0.smithi100.stdout:op 3250 completed, throughput=5MB/sec 2024-04-07T16:49:21.173 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-6DXl5-DU65LMaQx off=13948163 len=1840233 2024-04-07T16:49:21.178 INFO:tasks.workunit.client.0.smithi100.stdout:op 3251 completed, throughput=5MB/sec 2024-04-07T16:49:21.178 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-QXQsCs1pSX6X83F off=2753569 len=802000 2024-04-07T16:49:21.180 INFO:tasks.workunit.client.0.smithi100.stdout:op 3252 completed, throughput=5MB/sec 2024-04-07T16:49:21.180 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Jn_v_stgoazXv_w off=1657953 len=1578289 2024-04-07T16:49:21.193 INFO:tasks.workunit.client.0.smithi100.stdout:op 3253 completed, throughput=5MB/sec 2024-04-07T16:49:22.194 INFO:tasks.workunit.client.0.smithi100.stdout: 625: throughput=4.99MB/sec pending data=0 2024-04-07T16:49:22.194 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-krWUn1ai3RpHRKn off=5401386 len=211318 2024-04-07T16:49:22.195 INFO:tasks.workunit.client.0.smithi100.stdout:op 3254 completed, throughput=4.99MB/sec 2024-04-07T16:49:22.195 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-NEXnwtBMmJGFZom off=1308241 len=492460 2024-04-07T16:49:22.197 INFO:tasks.workunit.client.0.smithi100.stdout:op 3255 completed, throughput=5MB/sec 2024-04-07T16:49:22.197 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-ixuMoynQu6OP2AP off=16886252 len=643757 2024-04-07T16:49:22.199 INFO:tasks.workunit.client.0.smithi100.stdout:op 3256 completed, throughput=5MB/sec 2024-04-07T16:49:22.199 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-sTI1nEwMrFByyVq off=4057081 len=1804737 2024-04-07T16:49:22.203 INFO:tasks.workunit.client.0.smithi100.stdout:op 3257 completed, throughput=5MB/sec 2024-04-07T16:49:22.203 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Fj1j15fEVuTQHLn off=3446468 len=1317501 2024-04-07T16:49:22.208 INFO:tasks.workunit.client.0.smithi100.stdout:op 3258 completed, throughput=5MB/sec 2024-04-07T16:49:22.611 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:49:22 smithi144 bash[95233]: ts=2024-04-07T16:49:22.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:49:22.946 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:49:22 smithi100 bash[74942]: cluster 2024-04-07T16:49:21.404377+0000 mgr.y (mgr.25175) 1020 : cluster [DBG] pgmap v1662: 105 pgs: 105 active+clean; 126 GiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.2 MiB/s wr, 4 op/s 2024-04-07T16:49:22.946 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:49:22 smithi100 bash[76202]: cluster 2024-04-07T16:49:21.404377+0000 mgr.y (mgr.25175) 1020 : cluster [DBG] pgmap v1662: 105 pgs: 105 active+clean; 126 GiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.2 MiB/s wr, 4 op/s 2024-04-07T16:49:23.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:49:22 smithi144 bash[66654]: cluster 2024-04-07T16:49:21.404377+0000 mgr.y (mgr.25175) 1020 : cluster [DBG] pgmap v1662: 105 pgs: 105 active+clean; 126 GiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.2 MiB/s wr, 4 op/s 2024-04-07T16:49:23.209 INFO:tasks.workunit.client.0.smithi100.stdout: 626: throughput=4.99MB/sec pending data=0 2024-04-07T16:49:23.209 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-OWdAOKychE4DQ-Z off=76676 len=1985841 2024-04-07T16:49:23.215 INFO:tasks.workunit.client.0.smithi100.stdout:op 3259 completed, throughput=5MB/sec 2024-04-07T16:49:23.215 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-odsilPeYYAqVs_x off=6860670 len=647443 2024-04-07T16:49:23.223 INFO:tasks.workunit.client.0.smithi100.stdout:op 3260 completed, throughput=5MB/sec 2024-04-07T16:49:23.223 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj--328JRcahPOC0HE off=1305856 len=349379 2024-04-07T16:49:23.224 INFO:tasks.workunit.client.0.smithi100.stdout:op 3261 completed, throughput=5MB/sec 2024-04-07T16:49:23.224 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-PsMQpBGqFZYVIqs off=20329125 len=1137480 2024-04-07T16:49:23.232 INFO:tasks.workunit.client.0.smithi100.stdout:op 3262 completed, throughput=5MB/sec 2024-04-07T16:49:23.232 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-kFIaOL4LA6gxvAV off=3676450 len=2066026 2024-04-07T16:49:23.238 INFO:tasks.workunit.client.0.smithi100.stdout:op 3263 completed, throughput=5MB/sec 2024-04-07T16:49:24.239 INFO:tasks.workunit.client.0.smithi100.stdout: 627: throughput=4.99MB/sec pending data=0 2024-04-07T16:49:24.239 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-9Q3ESLjeT-0zm3a off=14751778 len=1132982 2024-04-07T16:49:24.250 INFO:tasks.workunit.client.0.smithi100.stdout:op 3264 completed, throughput=5MB/sec 2024-04-07T16:49:24.250 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-jN-hw13Z51p7odw off=6079278 len=224046 2024-04-07T16:49:24.251 INFO:tasks.workunit.client.0.smithi100.stdout:op 3265 completed, throughput=5MB/sec 2024-04-07T16:49:24.252 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-0PHmK3fCThTsFuX off=164487 len=976171 2024-04-07T16:49:24.254 INFO:tasks.workunit.client.0.smithi100.stdout:op 3266 completed, throughput=5MB/sec 2024-04-07T16:49:24.254 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Sy3xFJGdc9HcNJc off=10148987 len=1662926 2024-04-07T16:49:24.266 INFO:tasks.workunit.client.0.smithi100.stdout:op 3267 completed, throughput=5MB/sec 2024-04-07T16:49:24.946 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:49:24 smithi100 bash[74942]: cluster 2024-04-07T16:49:23.405514+0000 mgr.y (mgr.25175) 1021 : cluster [DBG] pgmap v1663: 105 pgs: 105 active+clean; 126 GiB data, 7.6 GiB used, 708 GiB / 715 GiB avail; 3.3 MiB/s rd, 2.2 MiB/s wr, 4 op/s 2024-04-07T16:49:24.946 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:49:24 smithi100 bash[76202]: cluster 2024-04-07T16:49:23.405514+0000 mgr.y (mgr.25175) 1021 : cluster [DBG] pgmap v1663: 105 pgs: 105 active+clean; 126 GiB data, 7.6 GiB used, 708 GiB / 715 GiB avail; 3.3 MiB/s rd, 2.2 MiB/s wr, 4 op/s 2024-04-07T16:49:25.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:49:24 smithi144 bash[66654]: cluster 2024-04-07T16:49:23.405514+0000 mgr.y (mgr.25175) 1021 : cluster [DBG] pgmap v1663: 105 pgs: 105 active+clean; 126 GiB data, 7.6 GiB used, 708 GiB / 715 GiB avail; 3.3 MiB/s rd, 2.2 MiB/s wr, 4 op/s 2024-04-07T16:49:25.266 INFO:tasks.workunit.client.0.smithi100.stdout: 628: throughput=4.99MB/sec pending data=0 2024-04-07T16:49:25.266 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-LWwGz2Bkte3tyrP off=12330784 len=520195 2024-04-07T16:49:25.273 INFO:tasks.workunit.client.0.smithi100.stdout:op 3268 completed, throughput=4.99MB/sec 2024-04-07T16:49:25.273 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-_qU2kvGdD7PUGMu off=6948493 len=839980 2024-04-07T16:49:25.276 INFO:tasks.workunit.client.0.smithi100.stdout:op 3269 completed, throughput=4.99MB/sec 2024-04-07T16:49:25.276 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-zWhQm1-jm5A5oBp off=406932 len=1158109 2024-04-07T16:49:25.279 INFO:tasks.workunit.client.0.smithi100.stdout:op 3270 completed, throughput=5MB/sec 2024-04-07T16:49:25.279 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-grl0Np7EvJ-AsKz off=4450736 len=1124856 2024-04-07T16:49:25.288 INFO:tasks.workunit.client.0.smithi100.stdout:op 3271 completed, throughput=5MB/sec 2024-04-07T16:49:25.289 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-aiZxMndn4tWh-Fn off=11173355 len=1102893 2024-04-07T16:49:25.292 INFO:tasks.workunit.client.0.smithi100.stdout:op 3272 completed, throughput=5MB/sec 2024-04-07T16:49:25.292 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-28rKiB7pfge17vP off=20798768 len=645874 2024-04-07T16:49:25.294 INFO:tasks.workunit.client.0.smithi100.stdout:op 3273 completed, throughput=5MB/sec 2024-04-07T16:49:26.294 INFO:tasks.workunit.client.0.smithi100.stdout: 629: throughput=4.99MB/sec pending data=0 2024-04-07T16:49:26.294 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-m-jKoZz0yFakljz off=9750364 len=692971 2024-04-07T16:49:26.302 INFO:tasks.workunit.client.0.smithi100.stdout:op 3274 completed, throughput=4.99MB/sec 2024-04-07T16:49:26.302 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-HAnfO3ABQRCBTO_ off=10876490 len=358367 2024-04-07T16:49:26.304 INFO:tasks.workunit.client.0.smithi100.stdout:op 3275 completed, throughput=4.99MB/sec 2024-04-07T16:49:26.304 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-wzDNiqM6YWrMHpR off=10761257 len=1142581 2024-04-07T16:49:26.313 INFO:tasks.workunit.client.0.smithi100.stdout:op 3276 completed, throughput=5MB/sec 2024-04-07T16:49:26.313 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Y8sSGVrSDA4H_Gm off=12949836 len=4225 2024-04-07T16:49:26.314 INFO:tasks.workunit.client.0.smithi100.stdout:op 3277 completed, throughput=5MB/sec 2024-04-07T16:49:26.314 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Li8Brok9cEvlYr1 off=5571097 len=312632 2024-04-07T16:49:26.319 INFO:tasks.workunit.client.0.smithi100.stdout:op 3278 completed, throughput=5MB/sec 2024-04-07T16:49:26.320 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-U56bggLjrBGyzUY off=23177761 len=1480387 2024-04-07T16:49:26.332 INFO:tasks.workunit.client.0.smithi100.stdout:op 3279 completed, throughput=5MB/sec 2024-04-07T16:49:26.332 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-rrlUbYmpzGHcU9L off=7180374 len=2023474 2024-04-07T16:49:26.334 INFO:tasks.workunit.client.0.smithi100.stdout:op 3280 completed, throughput=5MB/sec 2024-04-07T16:49:26.947 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:49:26 smithi100 bash[76202]: cluster 2024-04-07T16:49:25.406431+0000 mgr.y (mgr.25175) 1022 : cluster [DBG] pgmap v1664: 105 pgs: 105 active+clean; 126 GiB data, 7.6 GiB used, 708 GiB / 715 GiB avail; 2.6 MiB/s rd, 1.8 MiB/s wr, 4 op/s 2024-04-07T16:49:26.947 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:49:26 smithi100 bash[74942]: cluster 2024-04-07T16:49:25.406431+0000 mgr.y (mgr.25175) 1022 : cluster [DBG] pgmap v1664: 105 pgs: 105 active+clean; 126 GiB data, 7.6 GiB used, 708 GiB / 715 GiB avail; 2.6 MiB/s rd, 1.8 MiB/s wr, 4 op/s 2024-04-07T16:49:27.111 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:49:26 smithi144 bash[66654]: cluster 2024-04-07T16:49:25.406431+0000 mgr.y (mgr.25175) 1022 : cluster [DBG] pgmap v1664: 105 pgs: 105 active+clean; 126 GiB data, 7.6 GiB used, 708 GiB / 715 GiB avail; 2.6 MiB/s rd, 1.8 MiB/s wr, 4 op/s 2024-04-07T16:49:27.335 INFO:tasks.workunit.client.0.smithi100.stdout: 630: throughput=4.99MB/sec pending data=0 2024-04-07T16:49:27.335 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-wBxtAiKLl3z4E69 off=5883433 len=1174618 2024-04-07T16:49:27.339 INFO:tasks.workunit.client.0.smithi100.stdout:op 3281 completed, throughput=4.99MB/sec 2024-04-07T16:49:27.339 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-MKgT7UBJklT8E2i off=1868777 len=1769024 2024-04-07T16:49:27.353 INFO:tasks.workunit.client.0.smithi100.stdout:op 3282 completed, throughput=5MB/sec 2024-04-07T16:49:27.353 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-GpsGf_LzJkKeGBd off=10930558 len=2070445 2024-04-07T16:49:27.367 INFO:tasks.workunit.client.0.smithi100.stdout:op 3283 completed, throughput=5MB/sec 2024-04-07T16:49:28.368 INFO:tasks.workunit.client.0.smithi100.stdout: 631: throughput=4.99MB/sec pending data=0 2024-04-07T16:49:28.368 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-FyBqpEHNxleIjt4 off=4731398 len=1445898 2024-04-07T16:49:28.382 INFO:tasks.workunit.client.0.smithi100.stdout:op 3284 completed, throughput=4.99MB/sec 2024-04-07T16:49:28.382 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-qE4fw2YSNrraXoc off=916423 len=1058562 2024-04-07T16:49:28.390 INFO:tasks.workunit.client.0.smithi100.stdout:op 3285 completed, throughput=5MB/sec 2024-04-07T16:49:28.390 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-bFLlZ0xagLL3Hqd off=21329532 len=1802060 2024-04-07T16:49:28.394 INFO:tasks.workunit.client.0.smithi100.stdout:op 3286 completed, throughput=5MB/sec 2024-04-07T16:49:28.395 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-07GrybRFRSovC_c off=478467 len=1844381 2024-04-07T16:49:28.399 INFO:tasks.workunit.client.0.smithi100.stdout:op 3287 completed, throughput=5MB/sec 2024-04-07T16:49:28.696 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:49:28 smithi100 bash[74942]: cluster 2024-04-07T16:49:27.407527+0000 mgr.y (mgr.25175) 1023 : cluster [DBG] pgmap v1665: 105 pgs: 105 active+clean; 126 GiB data, 7.6 GiB used, 708 GiB / 715 GiB avail; 2.9 MiB/s rd, 2.1 MiB/s wr, 4 op/s 2024-04-07T16:49:28.697 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:49:28 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:49:28] "GET /metrics HTTP/1.1" 200 34068 "" "Prometheus/2.43.0" 2024-04-07T16:49:28.697 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:49:28 smithi100 bash[76202]: cluster 2024-04-07T16:49:27.407527+0000 mgr.y (mgr.25175) 1023 : cluster [DBG] pgmap v1665: 105 pgs: 105 active+clean; 126 GiB data, 7.6 GiB used, 708 GiB / 715 GiB avail; 2.9 MiB/s rd, 2.1 MiB/s wr, 4 op/s 2024-04-07T16:49:29.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:49:28 smithi144 bash[66654]: cluster 2024-04-07T16:49:27.407527+0000 mgr.y (mgr.25175) 1023 : cluster [DBG] pgmap v1665: 105 pgs: 105 active+clean; 126 GiB data, 7.6 GiB used, 708 GiB / 715 GiB avail; 2.9 MiB/s rd, 2.1 MiB/s wr, 4 op/s 2024-04-07T16:49:29.399 INFO:tasks.workunit.client.0.smithi100.stdout: 632: throughput=4.99MB/sec pending data=0 2024-04-07T16:49:29.399 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-L8t6xiShJHWRpO4 off=7923100 len=2057930 2024-04-07T16:49:29.404 INFO:tasks.workunit.client.0.smithi100.stdout:op 3288 completed, throughput=5MB/sec 2024-04-07T16:49:29.404 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-GJ87aUxK5y_iili off=3654413 len=1606411 2024-04-07T16:49:29.419 INFO:tasks.workunit.client.0.smithi100.stdout:op 3289 completed, throughput=5MB/sec 2024-04-07T16:49:29.419 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-7Q7NOKuddasgngj off=4144829 len=1539200 2024-04-07T16:49:29.431 INFO:tasks.workunit.client.0.smithi100.stdout:op 3290 completed, throughput=5MB/sec 2024-04-07T16:49:30.431 INFO:tasks.workunit.client.0.smithi100.stdout: 633: throughput=4.99MB/sec pending data=0 2024-04-07T16:49:30.432 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-5Q53xylkAGp0mtB off=15594741 len=1859568 2024-04-07T16:49:30.436 INFO:tasks.workunit.client.0.smithi100.stdout:op 3291 completed, throughput=5MB/sec 2024-04-07T16:49:30.436 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-llDH-A2YoROBcS8 off=8947842 len=1852599 2024-04-07T16:49:30.443 INFO:tasks.workunit.client.0.smithi100.stdout:op 3292 completed, throughput=5MB/sec 2024-04-07T16:49:30.443 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Yli5QWNbWrJKKDA off=2689818 len=331268 2024-04-07T16:49:30.449 INFO:tasks.workunit.client.0.smithi100.stdout:op 3293 completed, throughput=5MB/sec 2024-04-07T16:49:30.449 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Pyp71AvoqZEUjJy off=13068049 len=1929451 2024-04-07T16:49:30.466 INFO:tasks.workunit.client.0.smithi100.stdout:op 3294 completed, throughput=5MB/sec 2024-04-07T16:49:30.946 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:49:30 smithi100 bash[74942]: cluster 2024-04-07T16:49:29.408355+0000 mgr.y (mgr.25175) 1024 : cluster [DBG] pgmap v1666: 105 pgs: 105 active+clean; 126 GiB data, 7.6 GiB used, 708 GiB / 715 GiB avail; 2.8 MiB/s rd, 1.9 MiB/s wr, 4 op/s 2024-04-07T16:49:30.946 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:49:30 smithi100 bash[76202]: cluster 2024-04-07T16:49:29.408355+0000 mgr.y (mgr.25175) 1024 : cluster [DBG] pgmap v1666: 105 pgs: 105 active+clean; 126 GiB data, 7.6 GiB used, 708 GiB / 715 GiB avail; 2.8 MiB/s rd, 1.9 MiB/s wr, 4 op/s 2024-04-07T16:49:31.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:49:30 smithi144 bash[66654]: cluster 2024-04-07T16:49:29.408355+0000 mgr.y (mgr.25175) 1024 : cluster [DBG] pgmap v1666: 105 pgs: 105 active+clean; 126 GiB data, 7.6 GiB used, 708 GiB / 715 GiB avail; 2.8 MiB/s rd, 1.9 MiB/s wr, 4 op/s 2024-04-07T16:49:31.466 INFO:tasks.workunit.client.0.smithi100.stdout: 634: throughput=4.99MB/sec pending data=0 2024-04-07T16:49:31.466 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-akZiOI_hB6rfDG5 off=624145 len=1221250 2024-04-07T16:49:31.477 INFO:tasks.workunit.client.0.smithi100.stdout:op 3295 completed, throughput=5MB/sec 2024-04-07T16:49:31.477 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Ljjkb9H2cfjGKQF off=11279553 len=525687 2024-04-07T16:49:31.478 INFO:tasks.workunit.client.0.smithi100.stdout:op 3296 completed, throughput=5MB/sec 2024-04-07T16:49:31.478 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Pk8J4ppBJlG__Pk off=823499 len=731857 2024-04-07T16:49:31.481 INFO:tasks.workunit.client.0.smithi100.stdout:op 3297 completed, throughput=5MB/sec 2024-04-07T16:49:31.481 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-4uMp8aEnwLzhQYg off=11320888 len=1368936 2024-04-07T16:49:31.485 INFO:tasks.workunit.client.0.smithi100.stdout:op 3298 completed, throughput=5MB/sec 2024-04-07T16:49:31.486 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-2o-UrgDUVIEuolH off=5977495 len=651713 2024-04-07T16:49:31.493 INFO:tasks.workunit.client.0.smithi100.stdout:op 3299 completed, throughput=5MB/sec 2024-04-07T16:49:32.493 INFO:tasks.workunit.client.0.smithi100.stdout: 635: throughput=4.99MB/sec pending data=0 2024-04-07T16:49:32.494 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-1RP1bRrc81_5iQI off=4611203 len=310125 2024-04-07T16:49:32.495 INFO:tasks.workunit.client.0.smithi100.stdout:op 3300 completed, throughput=4.99MB/sec 2024-04-07T16:49:32.495 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-dLg8Ds36ZNBt72B off=2059850 len=404203 2024-04-07T16:49:32.497 INFO:tasks.workunit.client.0.smithi100.stdout:op 3301 completed, throughput=4.99MB/sec 2024-04-07T16:49:32.497 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-KNMaT_IK2POvIgs off=1147637 len=2064129 2024-04-07T16:49:32.502 INFO:tasks.workunit.client.0.smithi100.stdout:op 3302 completed, throughput=5MB/sec 2024-04-07T16:49:32.502 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-jfP_tPk0SihO62H off=2726426 len=438042 2024-04-07T16:49:32.508 INFO:tasks.workunit.client.0.smithi100.stdout:op 3303 completed, throughput=5MB/sec 2024-04-07T16:49:32.508 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-WnlIjnWYSyrJ8Kw off=5288978 len=1229392 2024-04-07T16:49:32.512 INFO:tasks.workunit.client.0.smithi100.stdout:op 3304 completed, throughput=5MB/sec 2024-04-07T16:49:32.512 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Qd8BpYRlA24IHip off=15619362 len=574754 2024-04-07T16:49:32.514 INFO:tasks.workunit.client.0.smithi100.stdout:op 3305 completed, throughput=5MB/sec 2024-04-07T16:49:32.611 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:49:32 smithi144 bash[95233]: ts=2024-04-07T16:49:32.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:49:32.946 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:49:32 smithi100 bash[74942]: cluster 2024-04-07T16:49:31.409374+0000 mgr.y (mgr.25175) 1025 : cluster [DBG] pgmap v1667: 105 pgs: 105 active+clean; 126 GiB data, 7.6 GiB used, 708 GiB / 715 GiB avail; 2.8 MiB/s rd, 2.0 MiB/s wr, 4 op/s 2024-04-07T16:49:32.947 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:49:32 smithi100 bash[74942]: audit 2024-04-07T16:49:31.774160+0000 mon.a (mon.0) 2139 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:49:32.947 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:49:32 smithi100 bash[76202]: cluster 2024-04-07T16:49:31.409374+0000 mgr.y (mgr.25175) 1025 : cluster [DBG] pgmap v1667: 105 pgs: 105 active+clean; 126 GiB data, 7.6 GiB used, 708 GiB / 715 GiB avail; 2.8 MiB/s rd, 2.0 MiB/s wr, 4 op/s 2024-04-07T16:49:32.947 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:49:32 smithi100 bash[76202]: audit 2024-04-07T16:49:31.774160+0000 mon.a (mon.0) 2139 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:49:33.111 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:49:32 smithi144 bash[66654]: cluster 2024-04-07T16:49:31.409374+0000 mgr.y (mgr.25175) 1025 : cluster [DBG] pgmap v1667: 105 pgs: 105 active+clean; 126 GiB data, 7.6 GiB used, 708 GiB / 715 GiB avail; 2.8 MiB/s rd, 2.0 MiB/s wr, 4 op/s 2024-04-07T16:49:33.111 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:49:32 smithi144 bash[66654]: audit 2024-04-07T16:49:31.774160+0000 mon.a (mon.0) 2139 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:49:33.514 INFO:tasks.workunit.client.0.smithi100.stdout: 636: throughput=4.99MB/sec pending data=0 2024-04-07T16:49:33.515 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-O8CkjnblpmzZS6s off=1058003 len=525855 2024-04-07T16:49:33.517 INFO:tasks.workunit.client.0.smithi100.stdout:op 3306 completed, throughput=4.99MB/sec 2024-04-07T16:49:33.517 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-TNHw2_6DdWk7N_U off=6137063 len=1088580 2024-04-07T16:49:33.521 INFO:tasks.workunit.client.0.smithi100.stdout:op 3307 completed, throughput=4.99MB/sec 2024-04-07T16:49:33.521 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-9nrWM5_Y1jOWIbz off=10225775 len=850755 2024-04-07T16:49:33.528 INFO:tasks.workunit.client.0.smithi100.stdout:op 3308 completed, throughput=5MB/sec 2024-04-07T16:49:33.528 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-CaDDtla1LAPsDyF off=9678874 len=1993435 2024-04-07T16:49:33.533 INFO:tasks.workunit.client.0.smithi100.stdout:op 3309 completed, throughput=5MB/sec 2024-04-07T16:49:33.533 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-FKPgPqJAgGCk7_7 off=9337737 len=250647 2024-04-07T16:49:33.538 INFO:tasks.workunit.client.0.smithi100.stdout:op 3310 completed, throughput=5MB/sec 2024-04-07T16:49:33.538 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-ngBASU7RhMVLM4v off=97970 len=841659 2024-04-07T16:49:33.546 INFO:tasks.workunit.client.0.smithi100.stdout:op 3311 completed, throughput=5MB/sec 2024-04-07T16:49:33.946 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:49:33 smithi100 bash[74942]: cluster 2024-04-07T16:49:33.410486+0000 mgr.y (mgr.25175) 1026 : cluster [DBG] pgmap v1668: 105 pgs: 105 active+clean; 126 GiB data, 7.6 GiB used, 708 GiB / 715 GiB avail; 3.5 MiB/s rd, 2.1 MiB/s wr, 4 op/s 2024-04-07T16:49:33.947 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:49:33 smithi100 bash[76202]: cluster 2024-04-07T16:49:33.410486+0000 mgr.y (mgr.25175) 1026 : cluster [DBG] pgmap v1668: 105 pgs: 105 active+clean; 126 GiB data, 7.6 GiB used, 708 GiB / 715 GiB avail; 3.5 MiB/s rd, 2.1 MiB/s wr, 4 op/s 2024-04-07T16:49:34.111 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:49:33 smithi144 bash[66654]: cluster 2024-04-07T16:49:33.410486+0000 mgr.y (mgr.25175) 1026 : cluster [DBG] pgmap v1668: 105 pgs: 105 active+clean; 126 GiB data, 7.6 GiB used, 708 GiB / 715 GiB avail; 3.5 MiB/s rd, 2.1 MiB/s wr, 4 op/s 2024-04-07T16:49:34.547 INFO:tasks.workunit.client.0.smithi100.stdout: 637: throughput=4.99MB/sec pending data=0 2024-04-07T16:49:34.547 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-_jN2-P08or1pQ5d off=6562050 len=1096743 2024-04-07T16:49:34.550 INFO:tasks.workunit.client.0.smithi100.stdout:op 3312 completed, throughput=4.99MB/sec 2024-04-07T16:49:34.550 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-GrcvDl9MtYeULqr off=18233449 len=714740 2024-04-07T16:49:34.557 INFO:tasks.workunit.client.0.smithi100.stdout:op 3313 completed, throughput=5MB/sec 2024-04-07T16:49:34.557 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-tCAb7PRzNNuRE_8 off=7614154 len=532646 2024-04-07T16:49:34.559 INFO:tasks.workunit.client.0.smithi100.stdout:op 3314 completed, throughput=5MB/sec 2024-04-07T16:49:34.559 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Po3zwMXnl54gf68 off=3965987 len=1375278 2024-04-07T16:49:34.568 INFO:tasks.workunit.client.0.smithi100.stdout:op 3315 completed, throughput=5MB/sec 2024-04-07T16:49:34.569 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-YrWM9fvovXjNP-V off=2584579 len=1857633 2024-04-07T16:49:34.573 INFO:tasks.workunit.client.0.smithi100.stdout:op 3316 completed, throughput=5MB/sec 2024-04-07T16:49:35.574 INFO:tasks.workunit.client.0.smithi100.stdout: 638: throughput=4.99MB/sec pending data=0 2024-04-07T16:49:35.574 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-bvfbNNNcpUSDPte off=2396822 len=935128 2024-04-07T16:49:35.576 INFO:tasks.workunit.client.0.smithi100.stdout:op 3317 completed, throughput=4.99MB/sec 2024-04-07T16:49:35.577 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-k6i5Le67aNbheT5 off=10253275 len=877063 2024-04-07T16:49:35.584 INFO:tasks.workunit.client.0.smithi100.stdout:op 3318 completed, throughput=5MB/sec 2024-04-07T16:49:35.585 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-04ybS_7irkc-S_U off=3047976 len=1129043 2024-04-07T16:49:35.594 INFO:tasks.workunit.client.0.smithi100.stdout:op 3319 completed, throughput=5MB/sec 2024-04-07T16:49:35.594 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-XKYU4a_tP5HbAYN off=1379451 len=953470 2024-04-07T16:49:35.603 INFO:tasks.workunit.client.0.smithi100.stdout:op 3320 completed, throughput=5MB/sec 2024-04-07T16:49:35.603 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-NrbLOOMX8Uf5-8E off=2769279 len=112014 2024-04-07T16:49:35.608 INFO:tasks.workunit.client.0.smithi100.stdout:op 3321 completed, throughput=5MB/sec 2024-04-07T16:49:35.608 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-hthAKdNKVb1309H off=4083343 len=512353 2024-04-07T16:49:35.615 INFO:tasks.workunit.client.0.smithi100.stdout:op 3322 completed, throughput=5MB/sec 2024-04-07T16:49:35.615 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-le7BPtYcR1QNp5j off=3647328 len=483340 2024-04-07T16:49:35.621 INFO:tasks.workunit.client.0.smithi100.stdout:op 3323 completed, throughput=5MB/sec 2024-04-07T16:49:35.621 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-5tSeN_TU8bPf5hW off=5411128 len=716907 2024-04-07T16:49:35.628 INFO:tasks.workunit.client.0.smithi100.stdout:op 3324 completed, throughput=5MB/sec 2024-04-07T16:49:36.629 INFO:tasks.workunit.client.0.smithi100.stdout: 639: throughput=4.99MB/sec pending data=0 2024-04-07T16:49:36.629 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-lARY5koUOmhktno off=2975645 len=481071 2024-04-07T16:49:36.631 INFO:tasks.workunit.client.0.smithi100.stdout:op 3325 completed, throughput=4.99MB/sec 2024-04-07T16:49:36.631 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-cflKefypk4Khz7G off=5720622 len=393481 2024-04-07T16:49:36.633 INFO:tasks.workunit.client.0.smithi100.stdout:op 3326 completed, throughput=4.99MB/sec 2024-04-07T16:49:36.633 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-qij_vpnG_ijx3x_ off=4349372 len=2075313 2024-04-07T16:49:36.652 INFO:tasks.workunit.client.0.smithi100.stdout:op 3327 completed, throughput=5MB/sec 2024-04-07T16:49:36.652 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-3mB--r2j9l4ggHT off=4780349 len=1547312 2024-04-07T16:49:36.656 INFO:tasks.workunit.client.0.smithi100.stdout:op 3328 completed, throughput=5MB/sec 2024-04-07T16:49:36.656 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-ctq8HrBLZd8fX2P off=5220677 len=773015 2024-04-07T16:49:36.658 INFO:tasks.workunit.client.0.smithi100.stdout:op 3329 completed, throughput=5MB/sec 2024-04-07T16:49:36.862 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:49:36 smithi144 bash[66654]: cluster 2024-04-07T16:49:35.411449+0000 mgr.y (mgr.25175) 1027 : cluster [DBG] pgmap v1669: 105 pgs: 105 active+clean; 126 GiB data, 7.6 GiB used, 708 GiB / 715 GiB avail; 2.9 MiB/s rd, 2.2 MiB/s wr, 4 op/s 2024-04-07T16:49:36.946 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:49:36 smithi100 bash[74942]: cluster 2024-04-07T16:49:35.411449+0000 mgr.y (mgr.25175) 1027 : cluster [DBG] pgmap v1669: 105 pgs: 105 active+clean; 126 GiB data, 7.6 GiB used, 708 GiB / 715 GiB avail; 2.9 MiB/s rd, 2.2 MiB/s wr, 4 op/s 2024-04-07T16:49:36.947 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:49:36 smithi100 bash[76202]: cluster 2024-04-07T16:49:35.411449+0000 mgr.y (mgr.25175) 1027 : cluster [DBG] pgmap v1669: 105 pgs: 105 active+clean; 126 GiB data, 7.6 GiB used, 708 GiB / 715 GiB avail; 2.9 MiB/s rd, 2.2 MiB/s wr, 4 op/s 2024-04-07T16:49:37.659 INFO:tasks.workunit.client.0.smithi100.stdout: 640: throughput=4.99MB/sec pending data=0 2024-04-07T16:49:37.659 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-dtviW9Szr5qU-_L off=13477868 len=1402772 2024-04-07T16:49:37.662 INFO:tasks.workunit.client.0.smithi100.stdout:op 3330 completed, throughput=5MB/sec 2024-04-07T16:49:37.663 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-I4aZM9hu3nqedW6 off=1201582 len=598795 2024-04-07T16:49:37.665 INFO:tasks.workunit.client.0.smithi100.stdout:op 3331 completed, throughput=5MB/sec 2024-04-07T16:49:37.665 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-yGayoxRl9I1iHxx off=1757636 len=1877469 2024-04-07T16:49:37.681 INFO:tasks.workunit.client.0.smithi100.stdout:op 3332 completed, throughput=5MB/sec 2024-04-07T16:49:37.681 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Jn8a_2zndf_aqUm off=4139853 len=2069524 2024-04-07T16:49:37.686 INFO:tasks.workunit.client.0.smithi100.stdout:op 3333 completed, throughput=5MB/sec 2024-04-07T16:49:38.686 INFO:tasks.workunit.client.0.smithi100.stdout: 641: throughput=4.99MB/sec pending data=0 2024-04-07T16:49:38.686 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-nLDTc3NCWt-iFgL off=1520375 len=373877 2024-04-07T16:49:38.693 INFO:tasks.workunit.client.0.smithi100.stdout:op 3334 completed, throughput=4.99MB/sec 2024-04-07T16:49:38.693 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-NGJp7cLHD1mSew2 off=4246049 len=1464095 2024-04-07T16:49:38.696 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:49:38 smithi100 bash[74942]: cluster 2024-04-07T16:49:37.412584+0000 mgr.y (mgr.25175) 1028 : cluster [DBG] pgmap v1670: 105 pgs: 105 active+clean; 126 GiB data, 7.6 GiB used, 708 GiB / 715 GiB avail; 3.1 MiB/s rd, 2.4 MiB/s wr, 5 op/s 2024-04-07T16:49:38.696 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:49:38 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:49:38] "GET /metrics HTTP/1.1" 200 34068 "" "Prometheus/2.43.0" 2024-04-07T16:49:38.697 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:49:38 smithi100 bash[76202]: cluster 2024-04-07T16:49:37.412584+0000 mgr.y (mgr.25175) 1028 : cluster [DBG] pgmap v1670: 105 pgs: 105 active+clean; 126 GiB data, 7.6 GiB used, 708 GiB / 715 GiB avail; 3.1 MiB/s rd, 2.4 MiB/s wr, 5 op/s 2024-04-07T16:49:38.704 INFO:tasks.workunit.client.0.smithi100.stdout:op 3335 completed, throughput=5MB/sec 2024-04-07T16:49:38.704 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-LB1sTDJmiwjTgkB off=2178909 len=1959236 2024-04-07T16:49:38.721 INFO:tasks.workunit.client.0.smithi100.stdout:op 3336 completed, throughput=5MB/sec 2024-04-07T16:49:38.721 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-SGPvjz1MGFoms6i off=3506286 len=640924 2024-04-07T16:49:38.723 INFO:tasks.workunit.client.0.smithi100.stdout:op 3337 completed, throughput=5MB/sec 2024-04-07T16:49:38.723 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-lOpxrW9I3dgSRRK off=19736533 len=1503828 2024-04-07T16:49:38.735 INFO:tasks.workunit.client.0.smithi100.stdout:op 3338 completed, throughput=5MB/sec 2024-04-07T16:49:38.861 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:49:38 smithi144 bash[66654]: cluster 2024-04-07T16:49:37.412584+0000 mgr.y (mgr.25175) 1028 : cluster [DBG] pgmap v1670: 105 pgs: 105 active+clean; 126 GiB data, 7.6 GiB used, 708 GiB / 715 GiB avail; 3.1 MiB/s rd, 2.4 MiB/s wr, 5 op/s 2024-04-07T16:49:39.735 INFO:tasks.workunit.client.0.smithi100.stdout: 642: throughput=4.99MB/sec pending data=0 2024-04-07T16:49:39.736 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-ZX4H9YnYXjB-dvP off=1405727 len=1502299 2024-04-07T16:49:39.739 INFO:tasks.workunit.client.0.smithi100.stdout:op 3339 completed, throughput=5MB/sec 2024-04-07T16:49:39.739 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-4FZczbSzN5pfKNQ off=16937394 len=1338617 2024-04-07T16:49:39.743 INFO:tasks.workunit.client.0.smithi100.stdout:op 3340 completed, throughput=5MB/sec 2024-04-07T16:49:39.743 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-bV8h7_T6FeIZVg- off=8035099 len=1062564 2024-04-07T16:49:39.752 INFO:tasks.workunit.client.0.smithi100.stdout:op 3341 completed, throughput=5MB/sec 2024-04-07T16:49:40.752 INFO:tasks.workunit.client.0.smithi100.stdout: 643: throughput=4.99MB/sec pending data=0 2024-04-07T16:49:40.752 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-QSA2z8i2WCKjD6H off=5701183 len=5187 2024-04-07T16:49:40.758 INFO:tasks.workunit.client.0.smithi100.stdout:op 3342 completed, throughput=4.99MB/sec 2024-04-07T16:49:40.758 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-gPrB108zKAtCzsz off=852940 len=1154327 2024-04-07T16:49:40.761 INFO:tasks.workunit.client.0.smithi100.stdout:op 3343 completed, throughput=4.99MB/sec 2024-04-07T16:49:40.761 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-EaAgwQ-oTDorU_Z off=4146409 len=971519 2024-04-07T16:49:40.771 INFO:tasks.workunit.client.0.smithi100.stdout:op 3344 completed, throughput=5MB/sec 2024-04-07T16:49:40.771 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-HeES5xb6eVzXwD7 off=2938206 len=507932 2024-04-07T16:49:40.773 INFO:tasks.workunit.client.0.smithi100.stdout:op 3345 completed, throughput=5MB/sec 2024-04-07T16:49:40.773 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Tp2mb-jXauNu5yQ off=2510769 len=80894 2024-04-07T16:49:40.774 INFO:tasks.workunit.client.0.smithi100.stdout:op 3346 completed, throughput=5MB/sec 2024-04-07T16:49:40.774 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-HXHOEbCJ2n30LVt off=6238682 len=1237489 2024-04-07T16:49:40.777 INFO:tasks.workunit.client.0.smithi100.stdout:op 3347 completed, throughput=5MB/sec 2024-04-07T16:49:40.777 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-bx37VkQsXOEEyYV off=10264259 len=1429919 2024-04-07T16:49:40.791 INFO:tasks.workunit.client.0.smithi100.stdout:op 3348 completed, throughput=5MB/sec 2024-04-07T16:49:40.791 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-39eC1TPtgLxtkvY off=8994720 len=75123 2024-04-07T16:49:40.792 INFO:tasks.workunit.client.0.smithi100.stdout:op 3349 completed, throughput=5MB/sec 2024-04-07T16:49:40.860 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:49:40 smithi144 bash[66654]: cluster 2024-04-07T16:49:39.413456+0000 mgr.y (mgr.25175) 1029 : cluster [DBG] pgmap v1671: 105 pgs: 105 active+clean; 126 GiB data, 7.7 GiB used, 708 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.0 MiB/s wr, 4 op/s 2024-04-07T16:49:40.946 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:49:40 smithi100 bash[74942]: cluster 2024-04-07T16:49:39.413456+0000 mgr.y (mgr.25175) 1029 : cluster [DBG] pgmap v1671: 105 pgs: 105 active+clean; 126 GiB data, 7.7 GiB used, 708 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.0 MiB/s wr, 4 op/s 2024-04-07T16:49:40.947 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:49:40 smithi100 bash[76202]: cluster 2024-04-07T16:49:39.413456+0000 mgr.y (mgr.25175) 1029 : cluster [DBG] pgmap v1671: 105 pgs: 105 active+clean; 126 GiB data, 7.7 GiB used, 708 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.0 MiB/s wr, 4 op/s 2024-04-07T16:49:41.793 INFO:tasks.workunit.client.0.smithi100.stdout: 644: throughput=4.99MB/sec pending data=0 2024-04-07T16:49:41.793 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-V5DbP_iOxuXQ4Xe off=8170958 len=702740 2024-04-07T16:49:41.801 INFO:tasks.workunit.client.0.smithi100.stdout:op 3350 completed, throughput=4.99MB/sec 2024-04-07T16:49:41.801 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-poHv8d1z-NYlSKl off=608967 len=502133 2024-04-07T16:49:41.803 INFO:tasks.workunit.client.0.smithi100.stdout:op 3351 completed, throughput=4.99MB/sec 2024-04-07T16:49:41.804 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-lLY1mMmsL70TFsm off=4074441 len=9204 2024-04-07T16:49:41.804 INFO:tasks.workunit.client.0.smithi100.stdout:op 3352 completed, throughput=4.99MB/sec 2024-04-07T16:49:41.805 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-DIiyNwBJW0tpO6i off=4651426 len=275563 2024-04-07T16:49:41.810 INFO:tasks.workunit.client.0.smithi100.stdout:op 3353 completed, throughput=4.99MB/sec 2024-04-07T16:49:41.810 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-vOIiX0l1_9wRtI7 off=192363 len=2072851 2024-04-07T16:49:41.816 INFO:tasks.workunit.client.0.smithi100.stdout:op 3354 completed, throughput=5MB/sec 2024-04-07T16:49:41.816 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-8PukZyT0Fvaezzg off=6601556 len=637933 2024-04-07T16:49:41.818 INFO:tasks.workunit.client.0.smithi100.stdout:op 3355 completed, throughput=5MB/sec 2024-04-07T16:49:41.818 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-ayX3HECRD5u8cO1 off=5256722 len=65569 2024-04-07T16:49:41.819 INFO:tasks.workunit.client.0.smithi100.stdout:op 3356 completed, throughput=5MB/sec 2024-04-07T16:49:41.819 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-2Ki3HnsSB4lpzf7 off=4056662 len=1974442 2024-04-07T16:49:41.833 INFO:tasks.workunit.client.0.smithi100.stdout:op 3357 completed, throughput=5MB/sec 2024-04-07T16:49:42.611 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:49:42 smithi144 bash[66654]: cluster 2024-04-07T16:49:41.414497+0000 mgr.y (mgr.25175) 1030 : cluster [DBG] pgmap v1672: 105 pgs: 105 active+clean; 126 GiB data, 7.7 GiB used, 708 GiB / 715 GiB avail; 3.1 MiB/s rd, 2.3 MiB/s wr, 5 op/s 2024-04-07T16:49:42.611 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:49:42 smithi144 bash[95233]: ts=2024-04-07T16:49:42.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:49:42.833 INFO:tasks.workunit.client.0.smithi100.stdout: 645: throughput=4.99MB/sec pending data=0 2024-04-07T16:49:42.833 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-bYUEGopRFu3buKK off=7626402 len=589790 2024-04-07T16:49:42.835 INFO:tasks.workunit.client.0.smithi100.stdout:op 3358 completed, throughput=4.99MB/sec 2024-04-07T16:49:42.836 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-zpJoIqb3Jpyj8FI off=5930943 len=318420 2024-04-07T16:49:42.837 INFO:tasks.workunit.client.0.smithi100.stdout:op 3359 completed, throughput=4.99MB/sec 2024-04-07T16:49:42.837 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-g24Dm-uxvD2eqb5 off=458692 len=2062877 2024-04-07T16:49:42.853 INFO:tasks.workunit.client.0.smithi100.stdout:op 3360 completed, throughput=5MB/sec 2024-04-07T16:49:42.854 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-AdHZkCL6k2_bK_q off=2496779 len=1078899 2024-04-07T16:49:42.857 INFO:tasks.workunit.client.0.smithi100.stdout:op 3361 completed, throughput=5MB/sec 2024-04-07T16:49:42.857 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-67eBAzVZSD8vuJs off=1866391 len=2035585 2024-04-07T16:49:42.868 INFO:tasks.workunit.client.0.smithi100.stdout:op 3362 completed, throughput=5MB/sec 2024-04-07T16:49:42.946 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:49:42 smithi100 bash[74942]: cluster 2024-04-07T16:49:41.414497+0000 mgr.y (mgr.25175) 1030 : cluster [DBG] pgmap v1672: 105 pgs: 105 active+clean; 126 GiB data, 7.7 GiB used, 708 GiB / 715 GiB avail; 3.1 MiB/s rd, 2.3 MiB/s wr, 5 op/s 2024-04-07T16:49:42.946 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:49:42 smithi100 bash[76202]: cluster 2024-04-07T16:49:41.414497+0000 mgr.y (mgr.25175) 1030 : cluster [DBG] pgmap v1672: 105 pgs: 105 active+clean; 126 GiB data, 7.7 GiB used, 708 GiB / 715 GiB avail; 3.1 MiB/s rd, 2.3 MiB/s wr, 5 op/s 2024-04-07T16:49:43.869 INFO:tasks.workunit.client.0.smithi100.stdout: 646: throughput=4.99MB/sec pending data=0 2024-04-07T16:49:43.869 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-q7zf7GNyElZK5FS off=5006439 len=2096208 2024-04-07T16:49:43.874 INFO:tasks.workunit.client.0.smithi100.stdout:op 3363 completed, throughput=5MB/sec 2024-04-07T16:49:43.874 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Z-bDHdzuM-E3yzO off=18967501 len=1928345 2024-04-07T16:49:43.890 INFO:tasks.workunit.client.0.smithi100.stdout:op 3364 completed, throughput=5MB/sec 2024-04-07T16:49:44.861 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:49:44 smithi144 bash[66654]: cluster 2024-04-07T16:49:43.415767+0000 mgr.y (mgr.25175) 1031 : cluster [DBG] pgmap v1673: 105 pgs: 105 active+clean; 126 GiB data, 7.7 GiB used, 708 GiB / 715 GiB avail; 3.1 MiB/s rd, 2.7 MiB/s wr, 5 op/s 2024-04-07T16:49:44.890 INFO:tasks.workunit.client.0.smithi100.stdout: 647: throughput=4.99MB/sec pending data=0 2024-04-07T16:49:44.891 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-8NUJX5H6IBd2fJU off=6504200 len=2064166 2024-04-07T16:49:44.897 INFO:tasks.workunit.client.0.smithi100.stdout:op 3365 completed, throughput=5MB/sec 2024-04-07T16:49:44.897 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Qk35iBF03uEIKfA off=23016239 len=825181 2024-04-07T16:49:44.905 INFO:tasks.workunit.client.0.smithi100.stdout:op 3366 completed, throughput=5MB/sec 2024-04-07T16:49:44.905 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-NmFhdxbrbbFYseC off=3627064 len=1775356 2024-04-07T16:49:44.921 INFO:tasks.workunit.client.0.smithi100.stdout:op 3367 completed, throughput=5MB/sec 2024-04-07T16:49:44.921 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-BAozPKjmp0MMM1u off=0 len=1160190 2024-04-07T16:49:44.925 INFO:tasks.workunit.client.0.smithi100.stdout:op 3368 completed, throughput=5MB/sec 2024-04-07T16:49:44.946 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:49:44 smithi100 bash[74942]: cluster 2024-04-07T16:49:43.415767+0000 mgr.y (mgr.25175) 1031 : cluster [DBG] pgmap v1673: 105 pgs: 105 active+clean; 126 GiB data, 7.7 GiB used, 708 GiB / 715 GiB avail; 3.1 MiB/s rd, 2.7 MiB/s wr, 5 op/s 2024-04-07T16:49:44.946 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:49:44 smithi100 bash[76202]: cluster 2024-04-07T16:49:43.415767+0000 mgr.y (mgr.25175) 1031 : cluster [DBG] pgmap v1673: 105 pgs: 105 active+clean; 126 GiB data, 7.7 GiB used, 708 GiB / 715 GiB avail; 3.1 MiB/s rd, 2.7 MiB/s wr, 5 op/s 2024-04-07T16:49:45.925 INFO:tasks.workunit.client.0.smithi100.stdout: 648: throughput=4.99MB/sec pending data=0 2024-04-07T16:49:45.926 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-IPJFOJtyohlYAVF off=4022796 len=1521914 2024-04-07T16:49:45.930 INFO:tasks.workunit.client.0.smithi100.stdout:op 3369 completed, throughput=5MB/sec 2024-04-07T16:49:45.930 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-1QoYyBu0eZIsYUl off=4046005 len=1768977 2024-04-07T16:49:45.936 INFO:tasks.workunit.client.0.smithi100.stdout:op 3370 completed, throughput=5MB/sec 2024-04-07T16:49:45.936 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj--gkRsQ-9aW7gKEp off=13244917 len=1933826 2024-04-07T16:49:45.952 INFO:tasks.workunit.client.0.smithi100.stdout:op 3371 completed, throughput=5MB/sec 2024-04-07T16:49:46.861 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:49:46 smithi144 bash[66654]: cluster 2024-04-07T16:49:45.416707+0000 mgr.y (mgr.25175) 1032 : cluster [DBG] pgmap v1674: 105 pgs: 105 active+clean; 126 GiB data, 7.7 GiB used, 708 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.6 MiB/s wr, 5 op/s 2024-04-07T16:49:46.946 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:49:46 smithi100 bash[74942]: cluster 2024-04-07T16:49:45.416707+0000 mgr.y (mgr.25175) 1032 : cluster [DBG] pgmap v1674: 105 pgs: 105 active+clean; 126 GiB data, 7.7 GiB used, 708 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.6 MiB/s wr, 5 op/s 2024-04-07T16:49:46.947 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:49:46 smithi100 bash[76202]: cluster 2024-04-07T16:49:45.416707+0000 mgr.y (mgr.25175) 1032 : cluster [DBG] pgmap v1674: 105 pgs: 105 active+clean; 126 GiB data, 7.7 GiB used, 708 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.6 MiB/s wr, 5 op/s 2024-04-07T16:49:46.952 INFO:tasks.workunit.client.0.smithi100.stdout: 649: throughput=4.99MB/sec pending data=0 2024-04-07T16:49:46.952 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-YOM5CCTdK04N1w4 off=5686560 len=1514789 2024-04-07T16:49:46.965 INFO:tasks.workunit.client.0.smithi100.stdout:op 3372 completed, throughput=5MB/sec 2024-04-07T16:49:46.966 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-klTRcet4OIg7lMy off=4669407 len=1669596 2024-04-07T16:49:46.980 INFO:tasks.workunit.client.0.smithi100.stdout:op 3373 completed, throughput=5MB/sec 2024-04-07T16:49:46.980 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-pn60o1acAOHINVT off=4649683 len=1347247 2024-04-07T16:49:46.991 INFO:tasks.workunit.client.0.smithi100.stdout:op 3374 completed, throughput=5MB/sec 2024-04-07T16:49:46.991 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-a6qWhmvNuzmVKR6 off=4301307 len=1472212 2024-04-07T16:49:46.993 INFO:tasks.workunit.client.0.smithi100.stdout:op 3375 completed, throughput=5MB/sec 2024-04-07T16:49:47.861 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:49:47 smithi144 bash[66654]: audit 2024-04-07T16:49:46.774568+0000 mon.a (mon.0) 2140 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:49:47.946 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:49:47 smithi100 bash[74942]: audit 2024-04-07T16:49:46.774568+0000 mon.a (mon.0) 2140 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:49:47.947 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:49:47 smithi100 bash[76202]: audit 2024-04-07T16:49:46.774568+0000 mon.a (mon.0) 2140 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:49:47.993 INFO:tasks.workunit.client.0.smithi100.stdout: 650: throughput=4.99MB/sec pending data=0 2024-04-07T16:49:47.993 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-2MsD-e0gJX5yWhL off=5607573 len=151447 2024-04-07T16:49:47.995 INFO:tasks.workunit.client.0.smithi100.stdout:op 3376 completed, throughput=4.99MB/sec 2024-04-07T16:49:47.995 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-uExnuGqXey7f0JQ off=1519010 len=92902 2024-04-07T16:49:47.996 INFO:tasks.workunit.client.0.smithi100.stdout:op 3377 completed, throughput=4.99MB/sec 2024-04-07T16:49:47.996 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-1MukubLcFckEhX8 off=5676762 len=385821 2024-04-07T16:49:47.997 INFO:tasks.workunit.client.0.smithi100.stdout:op 3378 completed, throughput=4.99MB/sec 2024-04-07T16:49:47.997 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-C6ZP2HwG2CsRwpp off=11480215 len=1536372 2024-04-07T16:49:48.001 INFO:tasks.workunit.client.0.smithi100.stdout:op 3379 completed, throughput=5MB/sec 2024-04-07T16:49:48.001 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Mu0uHf7AmHSE_j1 off=5637305 len=191521 2024-04-07T16:49:48.002 INFO:tasks.workunit.client.0.smithi100.stdout:op 3380 completed, throughput=5MB/sec 2024-04-07T16:49:48.003 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-vCYdqev3-3W3ZeR off=1686758 len=490440 2024-04-07T16:49:48.004 INFO:tasks.workunit.client.0.smithi100.stdout:op 3381 completed, throughput=5MB/sec 2024-04-07T16:49:48.004 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-gQtxwiTruBPAVP7 off=2377006 len=1556037 2024-04-07T16:49:48.017 INFO:tasks.workunit.client.0.smithi100.stdout:op 3382 completed, throughput=5MB/sec 2024-04-07T16:49:48.017 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-0RkLU8fuD72alEC off=606190 len=1122037 2024-04-07T16:49:48.020 INFO:tasks.workunit.client.0.smithi100.stdout:op 3383 completed, throughput=5MB/sec 2024-04-07T16:49:48.696 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:49:48 smithi100 bash[74942]: cluster 2024-04-07T16:49:47.417732+0000 mgr.y (mgr.25175) 1033 : cluster [DBG] pgmap v1675: 105 pgs: 105 active+clean; 126 GiB data, 7.7 GiB used, 707 GiB / 715 GiB avail; 2.6 MiB/s rd, 2.7 MiB/s wr, 5 op/s 2024-04-07T16:49:48.697 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:49:48 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:49:48] "GET /metrics HTTP/1.1" 200 34069 "" "Prometheus/2.43.0" 2024-04-07T16:49:48.697 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:49:48 smithi100 bash[76202]: cluster 2024-04-07T16:49:47.417732+0000 mgr.y (mgr.25175) 1033 : cluster [DBG] pgmap v1675: 105 pgs: 105 active+clean; 126 GiB data, 7.7 GiB used, 707 GiB / 715 GiB avail; 2.6 MiB/s rd, 2.7 MiB/s wr, 5 op/s 2024-04-07T16:49:48.861 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:49:48 smithi144 bash[66654]: cluster 2024-04-07T16:49:47.417732+0000 mgr.y (mgr.25175) 1033 : cluster [DBG] pgmap v1675: 105 pgs: 105 active+clean; 126 GiB data, 7.7 GiB used, 707 GiB / 715 GiB avail; 2.6 MiB/s rd, 2.7 MiB/s wr, 5 op/s 2024-04-07T16:49:49.021 INFO:tasks.workunit.client.0.smithi100.stdout: 651: throughput=4.99MB/sec pending data=0 2024-04-07T16:49:49.021 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-m8-mhQJpag5px8J off=13544089 len=1707348 2024-04-07T16:49:49.025 INFO:tasks.workunit.client.0.smithi100.stdout:op 3384 completed, throughput=5MB/sec 2024-04-07T16:49:49.025 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-HYOGLI39_uFINvy off=6489745 len=1034312 2024-04-07T16:49:49.027 INFO:tasks.workunit.client.0.smithi100.stdout:op 3385 completed, throughput=5MB/sec 2024-04-07T16:49:49.028 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-UanjEstMjzdr5n- off=4246174 len=1113455 2024-04-07T16:49:49.031 INFO:tasks.workunit.client.0.smithi100.stdout:op 3386 completed, throughput=5MB/sec 2024-04-07T16:49:49.031 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-QIfETSjoi5PP7RX off=13576576 len=70636 2024-04-07T16:49:49.036 INFO:tasks.workunit.client.0.smithi100.stdout:op 3387 completed, throughput=5MB/sec 2024-04-07T16:49:49.037 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-waMwWWJdjsst32- off=14207179 len=1797791 2024-04-07T16:49:49.041 INFO:tasks.workunit.client.0.smithi100.stdout:op 3388 completed, throughput=5MB/sec 2024-04-07T16:49:50.042 INFO:tasks.workunit.client.0.smithi100.stdout: 652: throughput=4.99MB/sec pending data=0 2024-04-07T16:49:50.042 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-1EmpHFCcfAk8hAo off=1288194 len=380370 2024-04-07T16:49:50.043 INFO:tasks.workunit.client.0.smithi100.stdout:op 3389 completed, throughput=5MB/sec 2024-04-07T16:49:50.044 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-tNuvhj6mpmC8Cy_ off=5909620 len=668676 2024-04-07T16:49:50.051 INFO:tasks.workunit.client.0.smithi100.stdout:op 3390 completed, throughput=5MB/sec 2024-04-07T16:49:50.051 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-xkRO1sv8FC1lTGS off=531317 len=709151 2024-04-07T16:49:50.059 INFO:tasks.workunit.client.0.smithi100.stdout:op 3391 completed, throughput=5MB/sec 2024-04-07T16:49:50.059 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-NcNuUrY44-6oGcD off=15605933 len=502468 2024-04-07T16:49:50.066 INFO:tasks.workunit.client.0.smithi100.stdout:op 3392 completed, throughput=5MB/sec 2024-04-07T16:49:50.066 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-4cVn_1eKnB7evgz off=10067183 len=678013 2024-04-07T16:49:50.068 INFO:tasks.workunit.client.0.smithi100.stdout:op 3393 completed, throughput=5MB/sec 2024-04-07T16:49:50.068 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-ET8jkanMEJ1KZ8W off=4969316 len=617865 2024-04-07T16:49:50.070 INFO:tasks.workunit.client.0.smithi100.stdout:op 3394 completed, throughput=5MB/sec 2024-04-07T16:49:50.070 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-jP9kIHnNnv3zL6X off=1547338 len=555924 2024-04-07T16:49:50.076 INFO:tasks.workunit.client.0.smithi100.stdout:op 3395 completed, throughput=5MB/sec 2024-04-07T16:49:50.861 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:49:50 smithi144 bash[66654]: cluster 2024-04-07T16:49:49.418589+0000 mgr.y (mgr.25175) 1034 : cluster [DBG] pgmap v1676: 105 pgs: 105 active+clean; 126 GiB data, 7.7 GiB used, 707 GiB / 715 GiB avail; 2.2 MiB/s rd, 2.4 MiB/s wr, 4 op/s 2024-04-07T16:49:50.946 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:49:50 smithi100 bash[74942]: cluster 2024-04-07T16:49:49.418589+0000 mgr.y (mgr.25175) 1034 : cluster [DBG] pgmap v1676: 105 pgs: 105 active+clean; 126 GiB data, 7.7 GiB used, 707 GiB / 715 GiB avail; 2.2 MiB/s rd, 2.4 MiB/s wr, 4 op/s 2024-04-07T16:49:50.947 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:49:50 smithi100 bash[76202]: cluster 2024-04-07T16:49:49.418589+0000 mgr.y (mgr.25175) 1034 : cluster [DBG] pgmap v1676: 105 pgs: 105 active+clean; 126 GiB data, 7.7 GiB used, 707 GiB / 715 GiB avail; 2.2 MiB/s rd, 2.4 MiB/s wr, 4 op/s 2024-04-07T16:49:51.076 INFO:tasks.workunit.client.0.smithi100.stdout: 653: throughput=4.99MB/sec pending data=0 2024-04-07T16:49:51.077 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-uaM6v663l2X9_vX off=2671032 len=2034706 2024-04-07T16:49:51.096 INFO:tasks.workunit.client.0.smithi100.stdout:op 3396 completed, throughput=5MB/sec 2024-04-07T16:49:51.096 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-xM5FkbZ78I14qd1 off=1677754 len=1404673 2024-04-07T16:49:51.099 INFO:tasks.workunit.client.0.smithi100.stdout:op 3397 completed, throughput=5MB/sec 2024-04-07T16:49:51.099 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-OKs_OPumEmeOw9a off=1651445 len=1328052 2024-04-07T16:49:51.103 INFO:tasks.workunit.client.0.smithi100.stdout:op 3398 completed, throughput=5MB/sec 2024-04-07T16:49:51.103 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-tX-FNfOmYLHJYsn off=7390370 len=1113835 2024-04-07T16:49:51.106 INFO:tasks.workunit.client.0.smithi100.stdout:op 3399 completed, throughput=5MB/sec 2024-04-07T16:49:52.106 INFO:tasks.workunit.client.0.smithi100.stdout: 654: throughput=4.99MB/sec pending data=0 2024-04-07T16:49:52.107 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-r8iW4j7BVv2-kzJ off=3366527 len=128867 2024-04-07T16:49:52.107 INFO:tasks.workunit.client.0.smithi100.stdout:op 3400 completed, throughput=4.99MB/sec 2024-04-07T16:49:52.108 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-NJXj3mHBhmx1P2_ off=72178 len=995887 2024-04-07T16:49:52.111 INFO:tasks.workunit.client.0.smithi100.stdout:op 3401 completed, throughput=4.99MB/sec 2024-04-07T16:49:52.111 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-l0puCW7U2Z6bEQ6 off=7187674 len=1211574 2024-04-07T16:49:52.114 INFO:tasks.workunit.client.0.smithi100.stdout:op 3402 completed, throughput=5MB/sec 2024-04-07T16:49:52.114 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-upAxHUYz9mAyNHE off=1203270 len=232745 2024-04-07T16:49:52.120 INFO:tasks.workunit.client.0.smithi100.stdout:op 3403 completed, throughput=5MB/sec 2024-04-07T16:49:52.120 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-rNsSPW9vFtyu7rK off=4125503 len=1887587 2024-04-07T16:49:52.138 INFO:tasks.workunit.client.0.smithi100.stdout:op 3404 completed, throughput=5MB/sec 2024-04-07T16:49:52.138 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-V1L2UJHyOtI6krG off=6284673 len=347732 2024-04-07T16:49:52.145 INFO:tasks.workunit.client.0.smithi100.stdout:op 3405 completed, throughput=5MB/sec 2024-04-07T16:49:52.145 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-zZU_swndAqWaGyS off=971206 len=1617409 2024-04-07T16:49:52.157 INFO:tasks.workunit.client.0.smithi100.stdout:op 3406 completed, throughput=5MB/sec 2024-04-07T16:49:52.611 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:49:52 smithi144 bash[66654]: cluster 2024-04-07T16:49:51.419544+0000 mgr.y (mgr.25175) 1035 : cluster [DBG] pgmap v1677: 105 pgs: 105 active+clean; 126 GiB data, 7.7 GiB used, 707 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.6 MiB/s wr, 4 op/s 2024-04-07T16:49:52.611 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:49:52 smithi144 bash[95233]: ts=2024-04-07T16:49:52.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:49:52.946 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:49:52 smithi100 bash[74942]: cluster 2024-04-07T16:49:51.419544+0000 mgr.y (mgr.25175) 1035 : cluster [DBG] pgmap v1677: 105 pgs: 105 active+clean; 126 GiB data, 7.7 GiB used, 707 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.6 MiB/s wr, 4 op/s 2024-04-07T16:49:52.947 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:49:52 smithi100 bash[76202]: cluster 2024-04-07T16:49:51.419544+0000 mgr.y (mgr.25175) 1035 : cluster [DBG] pgmap v1677: 105 pgs: 105 active+clean; 126 GiB data, 7.7 GiB used, 707 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.6 MiB/s wr, 4 op/s 2024-04-07T16:49:53.157 INFO:tasks.workunit.client.0.smithi100.stdout: 655: throughput=4.99MB/sec pending data=0 2024-04-07T16:49:53.157 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-hGKm6e0naeMeBQ9 off=2993074 len=156915 2024-04-07T16:49:53.158 INFO:tasks.workunit.client.0.smithi100.stdout:op 3407 completed, throughput=4.99MB/sec 2024-04-07T16:49:53.158 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-aNCF0kLCpEisHUf off=7141037 len=771126 2024-04-07T16:49:53.166 INFO:tasks.workunit.client.0.smithi100.stdout:op 3408 completed, throughput=5MB/sec 2024-04-07T16:49:53.167 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-PjrQdttIkhzJ-lA off=2786520 len=484556 2024-04-07T16:49:53.169 INFO:tasks.workunit.client.0.smithi100.stdout:op 3409 completed, throughput=5MB/sec 2024-04-07T16:49:53.169 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-uaBbqDiiYFKGiDM off=13513526 len=588951 2024-04-07T16:49:53.176 INFO:tasks.workunit.client.0.smithi100.stdout:op 3410 completed, throughput=5MB/sec 2024-04-07T16:49:53.176 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-fsRKjncyCXB1IYO off=3080468 len=1406141 2024-04-07T16:49:53.180 INFO:tasks.workunit.client.0.smithi100.stdout:op 3411 completed, throughput=5MB/sec 2024-04-07T16:49:53.180 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-d3Wn-gt7SYaTIy0 off=380836 len=1829380 2024-04-07T16:49:53.193 INFO:tasks.workunit.client.0.smithi100.stdout:op 3412 completed, throughput=5MB/sec 2024-04-07T16:49:54.194 INFO:tasks.workunit.client.0.smithi100.stdout: 656: throughput=4.99MB/sec pending data=0 2024-04-07T16:49:54.194 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-lPyeN6Hla4Gp2fS off=2033495 len=890455 2024-04-07T16:49:54.203 INFO:tasks.workunit.client.0.smithi100.stdout:op 3413 completed, throughput=5MB/sec 2024-04-07T16:49:54.203 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-nLDTc3NCWt-iFgL off=3466846 len=793152 2024-04-07T16:49:54.211 INFO:tasks.workunit.client.0.smithi100.stdout:op 3414 completed, throughput=5MB/sec 2024-04-07T16:49:54.211 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-SZLLBaq9PAzN4ol off=3275429 len=2032117 2024-04-07T16:49:54.231 INFO:tasks.workunit.client.0.smithi100.stdout:op 3415 completed, throughput=5MB/sec 2024-04-07T16:49:54.231 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-sqc5xnFR7WX4r55 off=5611574 len=1722309 2024-04-07T16:49:54.236 INFO:tasks.workunit.client.0.smithi100.stdout:op 3416 completed, throughput=5MB/sec 2024-04-07T16:49:54.861 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:49:54 smithi144 bash[66654]: cluster 2024-04-07T16:49:53.420615+0000 mgr.y (mgr.25175) 1036 : cluster [DBG] pgmap v1678: 105 pgs: 105 active+clean; 126 GiB data, 7.8 GiB used, 707 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.4 MiB/s wr, 4 op/s 2024-04-07T16:49:54.946 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:49:54 smithi100 bash[74942]: cluster 2024-04-07T16:49:53.420615+0000 mgr.y (mgr.25175) 1036 : cluster [DBG] pgmap v1678: 105 pgs: 105 active+clean; 126 GiB data, 7.8 GiB used, 707 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.4 MiB/s wr, 4 op/s 2024-04-07T16:49:54.947 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:49:54 smithi100 bash[76202]: cluster 2024-04-07T16:49:53.420615+0000 mgr.y (mgr.25175) 1036 : cluster [DBG] pgmap v1678: 105 pgs: 105 active+clean; 126 GiB data, 7.8 GiB used, 707 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.4 MiB/s wr, 4 op/s 2024-04-07T16:49:55.236 INFO:tasks.workunit.client.0.smithi100.stdout: 657: throughput=4.99MB/sec pending data=0 2024-04-07T16:49:55.237 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-jfrCKtLH0hJOlAU off=433965 len=797367 2024-04-07T16:49:55.239 INFO:tasks.workunit.client.0.smithi100.stdout:op 3417 completed, throughput=5MB/sec 2024-04-07T16:49:55.239 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj--jFuhMkPSyCQGRr off=12620604 len=960172 2024-04-07T16:49:55.248 INFO:tasks.workunit.client.0.smithi100.stdout:op 3418 completed, throughput=5MB/sec 2024-04-07T16:49:55.248 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-MGiKEpP60gpYROM off=2703371 len=2013591 2024-04-07T16:49:55.252 INFO:tasks.workunit.client.0.smithi100.stdout:op 3419 completed, throughput=5MB/sec 2024-04-07T16:49:55.253 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-KpoQTLxfIyAxYzB off=4894148 len=1445817 2024-04-07T16:49:55.256 INFO:tasks.workunit.client.0.smithi100.stdout:op 3420 completed, throughput=5MB/sec 2024-04-07T16:49:56.257 INFO:tasks.workunit.client.0.smithi100.stdout: 658: throughput=4.99MB/sec pending data=0 2024-04-07T16:49:56.257 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-fax4q4KCOZ59bYA off=1833482 len=1519214 2024-04-07T16:49:56.268 INFO:tasks.workunit.client.0.smithi100.stdout:op 3421 completed, throughput=5MB/sec 2024-04-07T16:49:56.269 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-aU_BgVZ0MfbwZQO off=3014004 len=2008729 2024-04-07T16:49:56.285 INFO:tasks.workunit.client.0.smithi100.stdout:op 3422 completed, throughput=5MB/sec 2024-04-07T16:49:56.285 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-r_tYmSz7EAG_88y off=1525534 len=366809 2024-04-07T16:49:56.286 INFO:tasks.workunit.client.0.smithi100.stdout:op 3423 completed, throughput=5MB/sec 2024-04-07T16:49:56.286 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-BdDD5EBhzNfO-GU off=5849873 len=360834 2024-04-07T16:49:56.291 INFO:tasks.workunit.client.0.smithi100.stdout:op 3424 completed, throughput=5MB/sec 2024-04-07T16:49:56.292 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-g_Axl_xo6GfIO-X off=6824630 len=1680534 2024-04-07T16:49:56.303 INFO:tasks.workunit.client.0.smithi100.stdout:op 3425 completed, throughput=5MB/sec 2024-04-07T16:49:56.861 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:49:56 smithi144 bash[66654]: cluster 2024-04-07T16:49:55.421613+0000 mgr.y (mgr.25175) 1037 : cluster [DBG] pgmap v1679: 105 pgs: 105 active+clean; 126 GiB data, 7.8 GiB used, 707 GiB / 715 GiB avail; 2.6 MiB/s rd, 2.0 MiB/s wr, 4 op/s 2024-04-07T16:49:56.946 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:49:56 smithi100 bash[74942]: cluster 2024-04-07T16:49:55.421613+0000 mgr.y (mgr.25175) 1037 : cluster [DBG] pgmap v1679: 105 pgs: 105 active+clean; 126 GiB data, 7.8 GiB used, 707 GiB / 715 GiB avail; 2.6 MiB/s rd, 2.0 MiB/s wr, 4 op/s 2024-04-07T16:49:56.947 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:49:56 smithi100 bash[76202]: cluster 2024-04-07T16:49:55.421613+0000 mgr.y (mgr.25175) 1037 : cluster [DBG] pgmap v1679: 105 pgs: 105 active+clean; 126 GiB data, 7.8 GiB used, 707 GiB / 715 GiB avail; 2.6 MiB/s rd, 2.0 MiB/s wr, 4 op/s 2024-04-07T16:49:57.304 INFO:tasks.workunit.client.0.smithi100.stdout: 659: throughput=4.99MB/sec pending data=0 2024-04-07T16:49:57.304 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-8HytFhdO1Jn9IiW off=8011407 len=931326 2024-04-07T16:49:57.306 INFO:tasks.workunit.client.0.smithi100.stdout:op 3426 completed, throughput=5MB/sec 2024-04-07T16:49:57.306 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-vbS_QYGWFLgsc65 off=2094302 len=807979 2024-04-07T16:49:57.309 INFO:tasks.workunit.client.0.smithi100.stdout:op 3427 completed, throughput=5MB/sec 2024-04-07T16:49:57.309 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-jdBnikA6OTS-_C1 off=11382696 len=2061097 2024-04-07T16:49:57.316 INFO:tasks.workunit.client.0.smithi100.stdout:op 3428 completed, throughput=5MB/sec 2024-04-07T16:49:58.317 INFO:tasks.workunit.client.0.smithi100.stdout: 660: throughput=4.99MB/sec pending data=0 2024-04-07T16:49:58.317 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-9jk1WRitIeCLb-s off=11052234 len=334492 2024-04-07T16:49:58.322 INFO:tasks.workunit.client.0.smithi100.stdout:op 3429 completed, throughput=4.99MB/sec 2024-04-07T16:49:58.322 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-blTSM1kJkNiTShK off=7695985 len=1856536 2024-04-07T16:49:58.327 INFO:tasks.workunit.client.0.smithi100.stdout:op 3430 completed, throughput=5MB/sec 2024-04-07T16:49:58.327 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Q4pNwkjmjjqIFf_ off=19688907 len=410393 2024-04-07T16:49:58.333 INFO:tasks.workunit.client.0.smithi100.stdout:op 3431 completed, throughput=5MB/sec 2024-04-07T16:49:58.333 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-4xL3NuHMbtVNig2 off=9216934 len=1019444 2024-04-07T16:49:58.336 INFO:tasks.workunit.client.0.smithi100.stdout:op 3432 completed, throughput=5MB/sec 2024-04-07T16:49:58.336 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-AoTeM-qIEtnpl6Y off=5462862 len=1326823 2024-04-07T16:49:58.346 INFO:tasks.workunit.client.0.smithi100.stdout:op 3433 completed, throughput=5MB/sec 2024-04-07T16:49:58.346 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-UF0eDOMKJWb1Ajj off=3575454 len=1848518 2024-04-07T16:49:58.353 INFO:tasks.workunit.client.0.smithi100.stdout:op 3434 completed, throughput=5MB/sec 2024-04-07T16:49:58.696 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:49:58 smithi100 bash[74942]: cluster 2024-04-07T16:49:57.422645+0000 mgr.y (mgr.25175) 1038 : cluster [DBG] pgmap v1680: 105 pgs: 105 active+clean; 126 GiB data, 7.8 GiB used, 707 GiB / 715 GiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 4 op/s 2024-04-07T16:49:58.697 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:49:58 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:49:58] "GET /metrics HTTP/1.1" 200 34069 "" "Prometheus/2.43.0" 2024-04-07T16:49:58.697 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:49:58 smithi100 bash[76202]: cluster 2024-04-07T16:49:57.422645+0000 mgr.y (mgr.25175) 1038 : cluster [DBG] pgmap v1680: 105 pgs: 105 active+clean; 126 GiB data, 7.8 GiB used, 707 GiB / 715 GiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 4 op/s 2024-04-07T16:49:58.861 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:49:58 smithi144 bash[66654]: cluster 2024-04-07T16:49:57.422645+0000 mgr.y (mgr.25175) 1038 : cluster [DBG] pgmap v1680: 105 pgs: 105 active+clean; 126 GiB data, 7.8 GiB used, 707 GiB / 715 GiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 4 op/s 2024-04-07T16:49:59.353 INFO:tasks.workunit.client.0.smithi100.stdout: 661: throughput=4.99MB/sec pending data=0 2024-04-07T16:49:59.353 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-bBv61feCYMQvG7j off=6690748 len=286062 2024-04-07T16:49:59.359 INFO:tasks.workunit.client.0.smithi100.stdout:op 3435 completed, throughput=5MB/sec 2024-04-07T16:49:59.359 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-SXxaTRd2cOAH-nU off=1258479 len=946260 2024-04-07T16:49:59.362 INFO:tasks.workunit.client.0.smithi100.stdout:op 3436 completed, throughput=5MB/sec 2024-04-07T16:49:59.362 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-rWzWX1hc_9VKDsy off=4462870 len=743883 2024-04-07T16:49:59.364 INFO:tasks.workunit.client.0.smithi100.stdout:op 3437 completed, throughput=5MB/sec 2024-04-07T16:49:59.365 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-4d80HOz-PMgzP68 off=2867859 len=185015 2024-04-07T16:49:59.366 INFO:tasks.workunit.client.0.smithi100.stdout:op 3438 completed, throughput=5MB/sec 2024-04-07T16:49:59.366 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-pCKmI0Pc_KkDVBH off=5212260 len=1344316 2024-04-07T16:49:59.369 INFO:tasks.workunit.client.0.smithi100.stdout:op 3439 completed, throughput=5MB/sec 2024-04-07T16:49:59.369 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-O8h1ideNqbYqQrX off=17861576 len=1044045 2024-04-07T16:49:59.378 INFO:tasks.workunit.client.0.smithi100.stdout:op 3440 completed, throughput=5MB/sec 2024-04-07T16:50:00.379 INFO:tasks.workunit.client.0.smithi100.stdout: 662: throughput=4.99MB/sec pending data=0 2024-04-07T16:50:00.379 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-rZSAegLUEoyoVYn off=10142327 len=1627597 2024-04-07T16:50:00.391 INFO:tasks.workunit.client.0.smithi100.stdout:op 3441 completed, throughput=5MB/sec 2024-04-07T16:50:00.391 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-tiakL33_kV3bLlk off=735745 len=2040538 2024-04-07T16:50:00.407 INFO:tasks.workunit.client.0.smithi100.stdout:op 3442 completed, throughput=5MB/sec 2024-04-07T16:50:00.407 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-PWPKNXtcm3wypOd off=1090068 len=598223 2024-04-07T16:50:00.409 INFO:tasks.workunit.client.0.smithi100.stdout:op 3443 completed, throughput=5MB/sec 2024-04-07T16:50:00.409 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-tAGY0GovMcjR1fH off=5748959 len=334225 2024-04-07T16:50:00.411 INFO:tasks.workunit.client.0.smithi100.stdout:op 3444 completed, throughput=5MB/sec 2024-04-07T16:50:00.411 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-3y24N26ATUq11rU off=28894 len=995758 2024-04-07T16:50:00.420 INFO:tasks.workunit.client.0.smithi100.stdout:op 3445 completed, throughput=5MB/sec 2024-04-07T16:50:00.861 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:50:00 smithi144 bash[66654]: cluster 2024-04-07T16:49:59.423545+0000 mgr.y (mgr.25175) 1039 : cluster [DBG] pgmap v1681: 105 pgs: 105 active+clean; 126 GiB data, 7.8 GiB used, 707 GiB / 715 GiB avail; 2.1 MiB/s rd, 2.0 MiB/s wr, 4 op/s 2024-04-07T16:50:00.861 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:50:00 smithi144 bash[66654]: cluster 2024-04-07T16:50:00.000144+0000 mon.a (mon.0) 2141 : cluster [INF] overall HEALTH_OK 2024-04-07T16:50:00.946 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:50:00 smithi100 bash[74942]: cluster 2024-04-07T16:49:59.423545+0000 mgr.y (mgr.25175) 1039 : cluster [DBG] pgmap v1681: 105 pgs: 105 active+clean; 126 GiB data, 7.8 GiB used, 707 GiB / 715 GiB avail; 2.1 MiB/s rd, 2.0 MiB/s wr, 4 op/s 2024-04-07T16:50:00.947 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:50:00 smithi100 bash[74942]: cluster 2024-04-07T16:50:00.000144+0000 mon.a (mon.0) 2141 : cluster [INF] overall HEALTH_OK 2024-04-07T16:50:00.947 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:50:00 smithi100 bash[76202]: cluster 2024-04-07T16:49:59.423545+0000 mgr.y (mgr.25175) 1039 : cluster [DBG] pgmap v1681: 105 pgs: 105 active+clean; 126 GiB data, 7.8 GiB used, 707 GiB / 715 GiB avail; 2.1 MiB/s rd, 2.0 MiB/s wr, 4 op/s 2024-04-07T16:50:00.947 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:50:00 smithi100 bash[76202]: cluster 2024-04-07T16:50:00.000144+0000 mon.a (mon.0) 2141 : cluster [INF] overall HEALTH_OK 2024-04-07T16:50:01.421 INFO:tasks.workunit.client.0.smithi100.stdout: 663: throughput=4.99MB/sec pending data=0 2024-04-07T16:50:01.421 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-yFJ2IsrFeeQpzbr off=2508038 len=1607489 2024-04-07T16:50:01.434 INFO:tasks.workunit.client.0.smithi100.stdout:op 3446 completed, throughput=5MB/sec 2024-04-07T16:50:01.434 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-UJ7fNlEjZs3C3kn off=1964349 len=826458 2024-04-07T16:50:01.442 INFO:tasks.workunit.client.0.smithi100.stdout:op 3447 completed, throughput=5MB/sec 2024-04-07T16:50:01.443 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-iGf2gBQicUZSqd1 off=1843270 len=330970 2024-04-07T16:50:01.444 INFO:tasks.workunit.client.0.smithi100.stdout:op 3448 completed, throughput=5MB/sec 2024-04-07T16:50:01.444 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-tXaoq0c-CHf2csd off=4070473 len=215417 2024-04-07T16:50:01.450 INFO:tasks.workunit.client.0.smithi100.stdout:op 3449 completed, throughput=5MB/sec 2024-04-07T16:50:01.450 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Q9BZnH2JNjNIMrL off=6962213 len=1111594 2024-04-07T16:50:01.452 INFO:tasks.workunit.client.0.smithi100.stdout:op 3450 completed, throughput=5MB/sec 2024-04-07T16:50:01.452 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-YUpbSGJevl4b8e8 off=136845 len=1140501 2024-04-07T16:50:01.455 INFO:tasks.workunit.client.0.smithi100.stdout:op 3451 completed, throughput=5MB/sec 2024-04-07T16:50:02.456 INFO:tasks.workunit.client.0.smithi100.stdout: 664: throughput=4.99MB/sec pending data=0 2024-04-07T16:50:02.456 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-dUb2Jg8FEPqdfZZ off=7296660 len=1772235 2024-04-07T16:50:02.460 INFO:tasks.workunit.client.0.smithi100.stdout:op 3452 completed, throughput=5MB/sec 2024-04-07T16:50:02.460 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-o2OE4v669PwGKrf off=6653024 len=556901 2024-04-07T16:50:02.463 INFO:tasks.workunit.client.0.smithi100.stdout:op 3453 completed, throughput=5MB/sec 2024-04-07T16:50:02.463 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-d8v944Ln9c8KqLs off=2461705 len=577031 2024-04-07T16:50:02.465 INFO:tasks.workunit.client.0.smithi100.stdout:op 3454 completed, throughput=5MB/sec 2024-04-07T16:50:02.465 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Aqb0p3lL1yOH5dF off=1133425 len=2021577 2024-04-07T16:50:02.470 INFO:tasks.workunit.client.0.smithi100.stdout:op 3455 completed, throughput=5MB/sec 2024-04-07T16:50:02.611 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:50:02 smithi144 bash[66654]: cluster 2024-04-07T16:50:01.424482+0000 mgr.y (mgr.25175) 1040 : cluster [DBG] pgmap v1682: 105 pgs: 105 active+clean; 126 GiB data, 7.8 GiB used, 707 GiB / 715 GiB avail; 2.6 MiB/s rd, 2.2 MiB/s wr, 4 op/s 2024-04-07T16:50:02.611 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:50:02 smithi144 bash[66654]: audit 2024-04-07T16:50:01.775150+0000 mon.a (mon.0) 2142 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:50:02.612 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:50:02 smithi144 bash[95233]: ts=2024-04-07T16:50:02.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:50:02.946 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:50:02 smithi100 bash[74942]: cluster 2024-04-07T16:50:01.424482+0000 mgr.y (mgr.25175) 1040 : cluster [DBG] pgmap v1682: 105 pgs: 105 active+clean; 126 GiB data, 7.8 GiB used, 707 GiB / 715 GiB avail; 2.6 MiB/s rd, 2.2 MiB/s wr, 4 op/s 2024-04-07T16:50:02.947 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:50:02 smithi100 bash[74942]: audit 2024-04-07T16:50:01.775150+0000 mon.a (mon.0) 2142 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:50:02.947 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:50:02 smithi100 bash[76202]: cluster 2024-04-07T16:50:01.424482+0000 mgr.y (mgr.25175) 1040 : cluster [DBG] pgmap v1682: 105 pgs: 105 active+clean; 126 GiB data, 7.8 GiB used, 707 GiB / 715 GiB avail; 2.6 MiB/s rd, 2.2 MiB/s wr, 4 op/s 2024-04-07T16:50:02.947 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:50:02 smithi100 bash[76202]: audit 2024-04-07T16:50:01.775150+0000 mon.a (mon.0) 2142 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:50:03.470 INFO:tasks.workunit.client.0.smithi100.stdout: 665: throughput=4.99MB/sec pending data=0 2024-04-07T16:50:03.471 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-rrPcErwdtZdLNVT off=147858 len=825565 2024-04-07T16:50:03.473 INFO:tasks.workunit.client.0.smithi100.stdout:op 3456 completed, throughput=4.99MB/sec 2024-04-07T16:50:03.473 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-9H1SOv7tIDC9Z_f off=1425710 len=667852 2024-04-07T16:50:03.480 INFO:tasks.workunit.client.0.smithi100.stdout:op 3457 completed, throughput=4.99MB/sec 2024-04-07T16:50:03.480 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-pZi4V9Bf2-1xG0M off=8702101 len=1333394 2024-04-07T16:50:03.491 INFO:tasks.workunit.client.0.smithi100.stdout:op 3458 completed, throughput=5MB/sec 2024-04-07T16:50:03.491 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-KKHZfvTWeyRD3rL off=13253582 len=1212854 2024-04-07T16:50:03.494 INFO:tasks.workunit.client.0.smithi100.stdout:op 3459 completed, throughput=5MB/sec 2024-04-07T16:50:03.494 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-xZQ2672EYSCtGSA off=5867414 len=1296883 2024-04-07T16:50:03.506 INFO:tasks.workunit.client.0.smithi100.stdout:op 3460 completed, throughput=5MB/sec 2024-04-07T16:50:04.506 INFO:tasks.workunit.client.0.smithi100.stdout: 666: throughput=4.99MB/sec pending data=0 2024-04-07T16:50:04.506 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-AwQDDqLxetk8bcG off=1844310 len=827594 2024-04-07T16:50:04.509 INFO:tasks.workunit.client.0.smithi100.stdout:op 3461 completed, throughput=4.99MB/sec 2024-04-07T16:50:04.509 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-03jhRPi_V1gpkkK off=5326863 len=1448893 2024-04-07T16:50:04.513 INFO:tasks.workunit.client.0.smithi100.stdout:op 3462 completed, throughput=5MB/sec 2024-04-07T16:50:04.513 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-bQ5GY4QlKgzytba off=8877761 len=300544 2024-04-07T16:50:04.519 INFO:tasks.workunit.client.0.smithi100.stdout:op 3463 completed, throughput=5MB/sec 2024-04-07T16:50:04.519 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-9iZHb6Bk6IMQYPQ off=13316945 len=1412099 2024-04-07T16:50:04.522 INFO:tasks.workunit.client.0.smithi100.stdout:op 3464 completed, throughput=5MB/sec 2024-04-07T16:50:04.522 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-805eoO5RCmoaL7i off=11089133 len=1882280 2024-04-07T16:50:04.527 INFO:tasks.workunit.client.0.smithi100.stdout:op 3465 completed, throughput=5MB/sec 2024-04-07T16:50:04.861 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:50:04 smithi144 bash[66654]: cluster 2024-04-07T16:50:03.425572+0000 mgr.y (mgr.25175) 1041 : cluster [DBG] pgmap v1683: 105 pgs: 105 active+clean; 126 GiB data, 7.8 GiB used, 707 GiB / 715 GiB avail; 2.8 MiB/s rd, 2.4 MiB/s wr, 5 op/s 2024-04-07T16:50:04.861 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:50:04 smithi144 bash[66654]: audit 2024-04-07T16:50:03.589412+0000 mon.a (mon.0) 2143 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:50:04.862 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:50:04 smithi144 bash[66654]: audit 2024-04-07T16:50:04.323681+0000 mon.a (mon.0) 2144 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:50:04.862 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:50:04 smithi144 bash[66654]: audit 2024-04-07T16:50:04.325426+0000 mon.a (mon.0) 2145 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:50:04.862 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:50:04 smithi144 bash[66654]: audit 2024-04-07T16:50:04.337091+0000 mon.a (mon.0) 2146 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:50:04.946 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:50:04 smithi100 bash[74942]: cluster 2024-04-07T16:50:03.425572+0000 mgr.y (mgr.25175) 1041 : cluster [DBG] pgmap v1683: 105 pgs: 105 active+clean; 126 GiB data, 7.8 GiB used, 707 GiB / 715 GiB avail; 2.8 MiB/s rd, 2.4 MiB/s wr, 5 op/s 2024-04-07T16:50:04.947 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:50:04 smithi100 bash[74942]: audit 2024-04-07T16:50:03.589412+0000 mon.a (mon.0) 2143 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:50:04.947 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:50:04 smithi100 bash[74942]: audit 2024-04-07T16:50:04.323681+0000 mon.a (mon.0) 2144 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:50:04.947 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:50:04 smithi100 bash[74942]: audit 2024-04-07T16:50:04.325426+0000 mon.a (mon.0) 2145 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:50:04.947 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:50:04 smithi100 bash[74942]: audit 2024-04-07T16:50:04.337091+0000 mon.a (mon.0) 2146 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:50:04.947 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:50:04 smithi100 bash[76202]: cluster 2024-04-07T16:50:03.425572+0000 mgr.y (mgr.25175) 1041 : cluster [DBG] pgmap v1683: 105 pgs: 105 active+clean; 126 GiB data, 7.8 GiB used, 707 GiB / 715 GiB avail; 2.8 MiB/s rd, 2.4 MiB/s wr, 5 op/s 2024-04-07T16:50:04.947 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:50:04 smithi100 bash[76202]: audit 2024-04-07T16:50:03.589412+0000 mon.a (mon.0) 2143 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:50:04.947 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:50:04 smithi100 bash[76202]: audit 2024-04-07T16:50:04.323681+0000 mon.a (mon.0) 2144 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:50:04.947 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:50:04 smithi100 bash[76202]: audit 2024-04-07T16:50:04.325426+0000 mon.a (mon.0) 2145 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:50:04.947 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:50:04 smithi100 bash[76202]: audit 2024-04-07T16:50:04.337091+0000 mon.a (mon.0) 2146 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:50:05.527 INFO:tasks.workunit.client.0.smithi100.stdout: 667: throughput=4.99MB/sec pending data=0 2024-04-07T16:50:05.527 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-g3CIp1cluO7o9OD off=5402101 len=1441460 2024-04-07T16:50:05.540 INFO:tasks.workunit.client.0.smithi100.stdout:op 3466 completed, throughput=5MB/sec 2024-04-07T16:50:05.540 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-pkHHetBgTO2cDKh off=2167641 len=1597220 2024-04-07T16:50:05.553 INFO:tasks.workunit.client.0.smithi100.stdout:op 3467 completed, throughput=5MB/sec 2024-04-07T16:50:05.553 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-nZfmpCsXq0MpEzQ off=5751605 len=1718968 2024-04-07T16:50:05.568 INFO:tasks.workunit.client.0.smithi100.stdout:op 3468 completed, throughput=5MB/sec 2024-04-07T16:50:05.568 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-rZRQKqdq4pVUxr4 off=17740215 len=2081049 2024-04-07T16:50:05.581 INFO:tasks.workunit.client.0.smithi100.stdout:op 3469 completed, throughput=5MB/sec 2024-04-07T16:50:06.581 INFO:tasks.workunit.client.0.smithi100.stdout: 668: throughput=5MB/sec pending data=0 2024-04-07T16:50:06.582 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-HK_DlEg1sM0aabj off=378338 len=1220237 2024-04-07T16:50:06.586 INFO:tasks.workunit.client.0.smithi100.stdout:op 3470 completed, throughput=5MB/sec 2024-04-07T16:50:06.587 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-oUAFFkpBA651s8H off=2673991 len=535752 2024-04-07T16:50:06.589 INFO:tasks.workunit.client.0.smithi100.stdout:op 3471 completed, throughput=5MB/sec 2024-04-07T16:50:06.589 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-ryhUDEsuPKPtGRC off=542186 len=1031950 2024-04-07T16:50:06.598 INFO:tasks.workunit.client.0.smithi100.stdout:op 3472 completed, throughput=5MB/sec 2024-04-07T16:50:06.598 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-RGDoeO8WBOoA2-Z off=10782504 len=1929032 2024-04-07T16:50:06.612 INFO:tasks.workunit.client.0.smithi100.stdout:op 3473 completed, throughput=5MB/sec 2024-04-07T16:50:06.861 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:50:06 smithi144 bash[66654]: cluster 2024-04-07T16:50:05.426512+0000 mgr.y (mgr.25175) 1042 : cluster [DBG] pgmap v1684: 105 pgs: 105 active+clean; 126 GiB data, 7.8 GiB used, 707 GiB / 715 GiB avail; 3.0 MiB/s rd, 2.5 MiB/s wr, 5 op/s 2024-04-07T16:50:06.946 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:50:06 smithi100 bash[74942]: cluster 2024-04-07T16:50:05.426512+0000 mgr.y (mgr.25175) 1042 : cluster [DBG] pgmap v1684: 105 pgs: 105 active+clean; 126 GiB data, 7.8 GiB used, 707 GiB / 715 GiB avail; 3.0 MiB/s rd, 2.5 MiB/s wr, 5 op/s 2024-04-07T16:50:06.947 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:50:06 smithi100 bash[76202]: cluster 2024-04-07T16:50:05.426512+0000 mgr.y (mgr.25175) 1042 : cluster [DBG] pgmap v1684: 105 pgs: 105 active+clean; 126 GiB data, 7.8 GiB used, 707 GiB / 715 GiB avail; 3.0 MiB/s rd, 2.5 MiB/s wr, 5 op/s 2024-04-07T16:50:07.613 INFO:tasks.workunit.client.0.smithi100.stdout: 669: throughput=4.99MB/sec pending data=0 2024-04-07T16:50:07.613 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-HTc5Qhr67qAIvC8 off=2411926 len=1869734 2024-04-07T16:50:07.617 INFO:tasks.workunit.client.0.smithi100.stdout:op 3474 completed, throughput=5MB/sec 2024-04-07T16:50:07.617 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-0YVUAVYClDOg_kP off=15604076 len=1859964 2024-04-07T16:50:07.622 INFO:tasks.workunit.client.0.smithi100.stdout:op 3475 completed, throughput=5MB/sec 2024-04-07T16:50:07.622 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-NP8s4PrCKed6SNF off=175321 len=871114 2024-04-07T16:50:07.631 INFO:tasks.workunit.client.0.smithi100.stdout:op 3476 completed, throughput=5MB/sec 2024-04-07T16:50:08.631 INFO:tasks.workunit.client.0.smithi100.stdout: 670: throughput=4.99MB/sec pending data=0 2024-04-07T16:50:08.631 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-S4Y4A_5B8Tl1bLF off=3139004 len=922529 2024-04-07T16:50:08.640 INFO:tasks.workunit.client.0.smithi100.stdout:op 3477 completed, throughput=4.99MB/sec 2024-04-07T16:50:08.640 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-vm9n-QQR6ZlD8zu off=1615424 len=1285317 2024-04-07T16:50:08.652 INFO:tasks.workunit.client.0.smithi100.stdout:op 3478 completed, throughput=5MB/sec 2024-04-07T16:50:08.652 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-gvEIABDx1NPQomW off=2196012 len=2086463 2024-04-07T16:50:08.667 INFO:tasks.workunit.client.0.smithi100.stdout:op 3479 completed, throughput=5MB/sec 2024-04-07T16:50:08.667 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-RQ8PzrOd2jWH7Wb off=12273394 len=782847 2024-04-07T16:50:08.674 INFO:tasks.workunit.client.0.smithi100.stdout:op 3480 completed, throughput=5MB/sec 2024-04-07T16:50:08.696 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:50:08 smithi100 bash[74942]: cluster 2024-04-07T16:50:07.427662+0000 mgr.y (mgr.25175) 1043 : cluster [DBG] pgmap v1685: 105 pgs: 105 active+clean; 126 GiB data, 7.9 GiB used, 707 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.4 MiB/s wr, 4 op/s 2024-04-07T16:50:08.697 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:50:08 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:50:08] "GET /metrics HTTP/1.1" 200 34070 "" "Prometheus/2.43.0" 2024-04-07T16:50:08.697 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:50:08 smithi100 bash[76202]: cluster 2024-04-07T16:50:07.427662+0000 mgr.y (mgr.25175) 1043 : cluster [DBG] pgmap v1685: 105 pgs: 105 active+clean; 126 GiB data, 7.9 GiB used, 707 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.4 MiB/s wr, 4 op/s 2024-04-07T16:50:08.861 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:50:08 smithi144 bash[66654]: cluster 2024-04-07T16:50:07.427662+0000 mgr.y (mgr.25175) 1043 : cluster [DBG] pgmap v1685: 105 pgs: 105 active+clean; 126 GiB data, 7.9 GiB used, 707 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.4 MiB/s wr, 4 op/s 2024-04-07T16:50:09.675 INFO:tasks.workunit.client.0.smithi100.stdout: 671: throughput=4.99MB/sec pending data=0 2024-04-07T16:50:09.675 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-MvTPIddarlhuxvd off=4934226 len=1660255 2024-04-07T16:50:09.688 INFO:tasks.workunit.client.0.smithi100.stdout:op 3481 completed, throughput=5MB/sec 2024-04-07T16:50:09.688 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-2qbwdCaxFyihhuc off=3731051 len=1311486 2024-04-07T16:50:09.693 INFO:tasks.workunit.client.0.smithi100.stdout:op 3482 completed, throughput=5MB/sec 2024-04-07T16:50:09.693 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-IUbawJeQeNRk1te off=1203984 len=1408786 2024-04-07T16:50:09.698 INFO:tasks.workunit.client.0.smithi100.stdout:op 3483 completed, throughput=5MB/sec 2024-04-07T16:50:09.698 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-OjRFrsOGUw6F5yW off=3084741 len=2083010 2024-04-07T16:50:09.713 INFO:tasks.workunit.client.0.smithi100.stdout:op 3484 completed, throughput=5MB/sec 2024-04-07T16:50:10.714 INFO:tasks.workunit.client.0.smithi100.stdout: 672: throughput=4.99MB/sec pending data=0 2024-04-07T16:50:10.714 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Ywaa0E0tmXl7stX off=8700936 len=2072550 2024-04-07T16:50:10.719 INFO:tasks.workunit.client.0.smithi100.stdout:op 3485 completed, throughput=5MB/sec 2024-04-07T16:50:10.719 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-5bdO6ARdxdLPlTN off=2014928 len=1488871 2024-04-07T16:50:10.732 INFO:tasks.workunit.client.0.smithi100.stdout:op 3486 completed, throughput=5MB/sec 2024-04-07T16:50:10.732 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-meNqL9bZcp6gGjw off=9380786 len=526654 2024-04-07T16:50:10.739 INFO:tasks.workunit.client.0.smithi100.stdout:op 3487 completed, throughput=5MB/sec 2024-04-07T16:50:10.739 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-JY4BG0vnD456Z6A off=2912863 len=956810 2024-04-07T16:50:10.742 INFO:tasks.workunit.client.0.smithi100.stdout:op 3488 completed, throughput=5MB/sec 2024-04-07T16:50:10.861 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:50:10 smithi144 bash[66654]: cluster 2024-04-07T16:50:09.428552+0000 mgr.y (mgr.25175) 1044 : cluster [DBG] pgmap v1686: 105 pgs: 105 active+clean; 126 GiB data, 7.9 GiB used, 707 GiB / 715 GiB avail; 2.9 MiB/s rd, 2.2 MiB/s wr, 4 op/s 2024-04-07T16:50:10.946 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:50:10 smithi100 bash[74942]: cluster 2024-04-07T16:50:09.428552+0000 mgr.y (mgr.25175) 1044 : cluster [DBG] pgmap v1686: 105 pgs: 105 active+clean; 126 GiB data, 7.9 GiB used, 707 GiB / 715 GiB avail; 2.9 MiB/s rd, 2.2 MiB/s wr, 4 op/s 2024-04-07T16:50:10.947 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:50:10 smithi100 bash[76202]: cluster 2024-04-07T16:50:09.428552+0000 mgr.y (mgr.25175) 1044 : cluster [DBG] pgmap v1686: 105 pgs: 105 active+clean; 126 GiB data, 7.9 GiB used, 707 GiB / 715 GiB avail; 2.9 MiB/s rd, 2.2 MiB/s wr, 4 op/s 2024-04-07T16:50:11.742 INFO:tasks.workunit.client.0.smithi100.stdout: 673: throughput=4.99MB/sec pending data=0 2024-04-07T16:50:11.742 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-XkgLrBA41-vsn4B off=9311498 len=1511783 2024-04-07T16:50:11.756 INFO:tasks.workunit.client.0.smithi100.stdout:op 3489 completed, throughput=5MB/sec 2024-04-07T16:50:11.756 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-FbQfdq5Xo4Q52Gn off=3794456 len=974705 2024-04-07T16:50:11.759 INFO:tasks.workunit.client.0.smithi100.stdout:op 3490 completed, throughput=5MB/sec 2024-04-07T16:50:11.759 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-NP8s4PrCKed6SNF off=121187 len=2087389 2024-04-07T16:50:11.766 INFO:tasks.workunit.client.0.smithi100.stdout:op 3491 completed, throughput=5MB/sec 2024-04-07T16:50:11.766 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-P-H-QYa-0KcggId off=17427313 len=1307065 2024-04-07T16:50:11.776 INFO:tasks.workunit.client.0.smithi100.stdout:op 3492 completed, throughput=5MB/sec 2024-04-07T16:50:12.611 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:50:12 smithi144 bash[66654]: cluster 2024-04-07T16:50:11.429475+0000 mgr.y (mgr.25175) 1045 : cluster [DBG] pgmap v1687: 105 pgs: 105 active+clean; 126 GiB data, 7.9 GiB used, 707 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.4 MiB/s wr, 4 op/s 2024-04-07T16:50:12.611 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:50:12 smithi144 bash[95233]: ts=2024-04-07T16:50:12.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:50:12.777 INFO:tasks.workunit.client.0.smithi100.stdout: 674: throughput=4.99MB/sec pending data=0 2024-04-07T16:50:12.777 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-ME8q0dGosJjEpkK off=140083 len=406313 2024-04-07T16:50:12.784 INFO:tasks.workunit.client.0.smithi100.stdout:op 3493 completed, throughput=4.99MB/sec 2024-04-07T16:50:12.784 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-TQ6yq1lbb6CFW45 off=151564 len=1784491 2024-04-07T16:50:12.798 INFO:tasks.workunit.client.0.smithi100.stdout:op 3494 completed, throughput=5MB/sec 2024-04-07T16:50:12.798 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-CQoTmYd9SKUJram off=1115671 len=476574 2024-04-07T16:50:12.800 INFO:tasks.workunit.client.0.smithi100.stdout:op 3495 completed, throughput=5MB/sec 2024-04-07T16:50:12.800 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-e417HjgryTthQSK off=283042 len=361386 2024-04-07T16:50:12.802 INFO:tasks.workunit.client.0.smithi100.stdout:op 3496 completed, throughput=5MB/sec 2024-04-07T16:50:12.802 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-oxy-veSVpa-EO8d off=12011755 len=2048120 2024-04-07T16:50:12.807 INFO:tasks.workunit.client.0.smithi100.stdout:op 3497 completed, throughput=5MB/sec 2024-04-07T16:50:12.946 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:50:12 smithi100 bash[74942]: cluster 2024-04-07T16:50:11.429475+0000 mgr.y (mgr.25175) 1045 : cluster [DBG] pgmap v1687: 105 pgs: 105 active+clean; 126 GiB data, 7.9 GiB used, 707 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.4 MiB/s wr, 4 op/s 2024-04-07T16:50:12.947 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:50:12 smithi100 bash[76202]: cluster 2024-04-07T16:50:11.429475+0000 mgr.y (mgr.25175) 1045 : cluster [DBG] pgmap v1687: 105 pgs: 105 active+clean; 126 GiB data, 7.9 GiB used, 707 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.4 MiB/s wr, 4 op/s 2024-04-07T16:50:13.807 INFO:tasks.workunit.client.0.smithi100.stdout: 675: throughput=4.99MB/sec pending data=0 2024-04-07T16:50:13.807 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-eP3tWtq6odirrFR off=1236425 len=1862484 2024-04-07T16:50:13.821 INFO:tasks.workunit.client.0.smithi100.stdout:op 3498 completed, throughput=5MB/sec 2024-04-07T16:50:13.821 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-sI6zeOF_TlwApPf off=11770936 len=1698482 2024-04-07T16:50:13.824 INFO:tasks.workunit.client.0.smithi100.stdout:op 3499 completed, throughput=5MB/sec 2024-04-07T16:50:13.824 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-o_Cg9yG5gX9eRrv off=4895061 len=76973 2024-04-07T16:50:13.829 INFO:tasks.workunit.client.0.smithi100.stdout:op 3500 completed, throughput=5MB/sec 2024-04-07T16:50:13.829 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Ii_jSJI64LILh6E off=18151522 len=586212 2024-04-07T16:50:13.835 INFO:tasks.workunit.client.0.smithi100.stdout:op 3501 completed, throughput=5MB/sec 2024-04-07T16:50:13.835 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-7e9uFYmJYRD-OnE off=4898621 len=804450 2024-04-07T16:50:13.843 INFO:tasks.workunit.client.0.smithi100.stdout:op 3502 completed, throughput=5MB/sec 2024-04-07T16:50:14.843 INFO:tasks.workunit.client.0.smithi100.stdout: 676: throughput=4.99MB/sec pending data=0 2024-04-07T16:50:14.843 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-_LPLHY1MrXx_aza off=12748319 len=745788 2024-04-07T16:50:14.846 INFO:tasks.workunit.client.0.smithi100.stdout:op 3503 completed, throughput=4.99MB/sec 2024-04-07T16:50:14.846 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-VO76Xmoom6WlGS1 off=5141039 len=1588609 2024-04-07T16:50:14.861 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:50:14 smithi144 bash[66654]: cluster 2024-04-07T16:50:13.430591+0000 mgr.y (mgr.25175) 1046 : cluster [DBG] pgmap v1688: 105 pgs: 105 active+clean; 126 GiB data, 7.9 GiB used, 707 GiB / 715 GiB avail; 2.8 MiB/s rd, 2.9 MiB/s wr, 4 op/s 2024-04-07T16:50:14.861 INFO:tasks.workunit.client.0.smithi100.stdout:op 3504 completed, throughput=5MB/sec 2024-04-07T16:50:14.861 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-wOr4Exc9ff-ly5x off=2385829 len=731751 2024-04-07T16:50:14.864 INFO:tasks.workunit.client.0.smithi100.stdout:op 3505 completed, throughput=5MB/sec 2024-04-07T16:50:14.864 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-WaDfLy25oWgiTOf off=3044519 len=855846 2024-04-07T16:50:14.867 INFO:tasks.workunit.client.0.smithi100.stdout:op 3506 completed, throughput=5MB/sec 2024-04-07T16:50:14.867 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-XBYMiKBXMbZiwrP off=8734578 len=1573548 2024-04-07T16:50:14.871 INFO:tasks.workunit.client.0.smithi100.stdout:op 3507 completed, throughput=5MB/sec 2024-04-07T16:50:14.946 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:50:14 smithi100 bash[74942]: cluster 2024-04-07T16:50:13.430591+0000 mgr.y (mgr.25175) 1046 : cluster [DBG] pgmap v1688: 105 pgs: 105 active+clean; 126 GiB data, 7.9 GiB used, 707 GiB / 715 GiB avail; 2.8 MiB/s rd, 2.9 MiB/s wr, 4 op/s 2024-04-07T16:50:14.947 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:50:14 smithi100 bash[76202]: cluster 2024-04-07T16:50:13.430591+0000 mgr.y (mgr.25175) 1046 : cluster [DBG] pgmap v1688: 105 pgs: 105 active+clean; 126 GiB data, 7.9 GiB used, 707 GiB / 715 GiB avail; 2.8 MiB/s rd, 2.9 MiB/s wr, 4 op/s 2024-04-07T16:50:15.871 INFO:tasks.workunit.client.0.smithi100.stdout: 677: throughput=4.99MB/sec pending data=0 2024-04-07T16:50:15.871 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-diOFCDjoYnqG7jx off=3384531 len=662121 2024-04-07T16:50:15.874 INFO:tasks.workunit.client.0.smithi100.stdout:op 3508 completed, throughput=4.99MB/sec 2024-04-07T16:50:15.874 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj--s_XHdaMHseqYmn off=10928392 len=1511292 2024-04-07T16:50:15.886 INFO:tasks.workunit.client.0.smithi100.stdout:op 3509 completed, throughput=5MB/sec 2024-04-07T16:50:15.886 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-9YghhgEa66AtiEB off=4590756 len=1270343 2024-04-07T16:50:15.897 INFO:tasks.workunit.client.0.smithi100.stdout:op 3510 completed, throughput=5MB/sec 2024-04-07T16:50:15.897 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-HDiW-oc8HMYrzpt off=1623194 len=1563512 2024-04-07T16:50:15.900 INFO:tasks.workunit.client.0.smithi100.stdout:op 3511 completed, throughput=5MB/sec 2024-04-07T16:50:16.861 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:50:16 smithi144 bash[66654]: cluster 2024-04-07T16:50:15.431575+0000 mgr.y (mgr.25175) 1047 : cluster [DBG] pgmap v1689: 105 pgs: 105 active+clean; 126 GiB data, 7.9 GiB used, 707 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.5 MiB/s wr, 4 op/s 2024-04-07T16:50:16.901 INFO:tasks.workunit.client.0.smithi100.stdout: 678: throughput=4.99MB/sec pending data=0 2024-04-07T16:50:16.901 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Ou__mK9JqtsP-Lj off=7850079 len=1178749 2024-04-07T16:50:16.904 INFO:tasks.workunit.client.0.smithi100.stdout:op 3512 completed, throughput=4.99MB/sec 2024-04-07T16:50:16.904 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-7kAk3LHsrKl6YyN off=4381519 len=1498974 2024-04-07T16:50:16.908 INFO:tasks.workunit.client.0.smithi100.stdout:op 3513 completed, throughput=5MB/sec 2024-04-07T16:50:16.908 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-DAIuta9xDNAZXLI off=2905808 len=930799 2024-04-07T16:50:16.911 INFO:tasks.workunit.client.0.smithi100.stdout:op 3514 completed, throughput=5MB/sec 2024-04-07T16:50:16.911 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-SVMdBRakuAd2lS- off=6253838 len=1824783 2024-04-07T16:50:16.917 INFO:tasks.workunit.client.0.smithi100.stdout:op 3515 completed, throughput=5MB/sec 2024-04-07T16:50:16.946 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:50:16 smithi100 bash[74942]: cluster 2024-04-07T16:50:15.431575+0000 mgr.y (mgr.25175) 1047 : cluster [DBG] pgmap v1689: 105 pgs: 105 active+clean; 126 GiB data, 7.9 GiB used, 707 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.5 MiB/s wr, 4 op/s 2024-04-07T16:50:16.946 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:50:16 smithi100 bash[76202]: cluster 2024-04-07T16:50:15.431575+0000 mgr.y (mgr.25175) 1047 : cluster [DBG] pgmap v1689: 105 pgs: 105 active+clean; 126 GiB data, 7.9 GiB used, 707 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.5 MiB/s wr, 4 op/s 2024-04-07T16:50:17.861 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:50:17 smithi144 bash[66654]: audit 2024-04-07T16:50:16.775577+0000 mon.a (mon.0) 2147 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:50:17.918 INFO:tasks.workunit.client.0.smithi100.stdout: 679: throughput=4.99MB/sec pending data=0 2024-04-07T16:50:17.918 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-hIpoC9MkPqPbYWW off=7836487 len=1370139 2024-04-07T16:50:17.922 INFO:tasks.workunit.client.0.smithi100.stdout:op 3516 completed, throughput=5MB/sec 2024-04-07T16:50:17.922 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-eMkNsVh9JunalKg off=4894260 len=1360577 2024-04-07T16:50:17.934 INFO:tasks.workunit.client.0.smithi100.stdout:op 3517 completed, throughput=5MB/sec 2024-04-07T16:50:17.934 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-ZZNSqi5-Qpa26vY off=1669419 len=475315 2024-04-07T16:50:17.940 INFO:tasks.workunit.client.0.smithi100.stdout:op 3518 completed, throughput=5MB/sec 2024-04-07T16:50:17.941 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-1CQUALzeeXqrXsH off=1361954 len=1340076 2024-04-07T16:50:17.942 INFO:tasks.workunit.client.0.smithi100.stdout:op 3519 completed, throughput=5MB/sec 2024-04-07T16:50:17.943 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-pTeirFUln4rgJZn off=3357056 len=1596808 2024-04-07T16:50:17.945 INFO:tasks.workunit.client.0.smithi100.stdout:op 3520 completed, throughput=5MB/sec 2024-04-07T16:50:17.946 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:50:17 smithi100 bash[74942]: audit 2024-04-07T16:50:16.775577+0000 mon.a (mon.0) 2147 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:50:17.947 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:50:17 smithi100 bash[76202]: audit 2024-04-07T16:50:16.775577+0000 mon.a (mon.0) 2147 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:50:18.696 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:50:18 smithi100 bash[74942]: cluster 2024-04-07T16:50:17.432574+0000 mgr.y (mgr.25175) 1048 : cluster [DBG] pgmap v1690: 105 pgs: 105 active+clean; 126 GiB data, 7.9 GiB used, 707 GiB / 715 GiB avail; 2.2 MiB/s rd, 2.6 MiB/s wr, 3 op/s 2024-04-07T16:50:18.697 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:50:18 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:50:18] "GET /metrics HTTP/1.1" 200 34070 "" "Prometheus/2.43.0" 2024-04-07T16:50:18.697 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:50:18 smithi100 bash[76202]: cluster 2024-04-07T16:50:17.432574+0000 mgr.y (mgr.25175) 1048 : cluster [DBG] pgmap v1690: 105 pgs: 105 active+clean; 126 GiB data, 7.9 GiB used, 707 GiB / 715 GiB avail; 2.2 MiB/s rd, 2.6 MiB/s wr, 3 op/s 2024-04-07T16:50:18.861 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:50:18 smithi144 bash[66654]: cluster 2024-04-07T16:50:17.432574+0000 mgr.y (mgr.25175) 1048 : cluster [DBG] pgmap v1690: 105 pgs: 105 active+clean; 126 GiB data, 7.9 GiB used, 707 GiB / 715 GiB avail; 2.2 MiB/s rd, 2.6 MiB/s wr, 3 op/s 2024-04-07T16:50:18.946 INFO:tasks.workunit.client.0.smithi100.stdout: 680: throughput=4.99MB/sec pending data=0 2024-04-07T16:50:18.946 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-tkbikMxiwmgmoBB off=16525334 len=327170 2024-04-07T16:50:18.952 INFO:tasks.workunit.client.0.smithi100.stdout:op 3521 completed, throughput=4.99MB/sec 2024-04-07T16:50:18.952 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-icRDqd-kP94sAPx off=64973 len=819945 2024-04-07T16:50:18.955 INFO:tasks.workunit.client.0.smithi100.stdout:op 3522 completed, throughput=5MB/sec 2024-04-07T16:50:18.955 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-H-zsBkWWIwbAa_D off=178377 len=893704 2024-04-07T16:50:18.964 INFO:tasks.workunit.client.0.smithi100.stdout:op 3523 completed, throughput=5MB/sec 2024-04-07T16:50:18.964 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj--6BzmEVdjbnlWeC off=1016061 len=314352 2024-04-07T16:50:18.966 INFO:tasks.workunit.client.0.smithi100.stdout:op 3524 completed, throughput=5MB/sec 2024-04-07T16:50:18.966 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-qhMqOLAMyralrZN off=5327238 len=870775 2024-04-07T16:50:18.974 INFO:tasks.workunit.client.0.smithi100.stdout:op 3525 completed, throughput=5MB/sec 2024-04-07T16:50:18.974 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Er4iEzndUoZD3JS off=14570680 len=322916 2024-04-07T16:50:18.979 INFO:tasks.workunit.client.0.smithi100.stdout:op 3526 completed, throughput=5MB/sec 2024-04-07T16:50:18.979 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-63PSOAvEmtl8FKk off=3692098 len=845122 2024-04-07T16:50:18.987 INFO:tasks.workunit.client.0.smithi100.stdout:op 3527 completed, throughput=5MB/sec 2024-04-07T16:50:18.987 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-_9dqjn1JtYEuPHt off=4395034 len=608165 2024-04-07T16:50:18.993 INFO:tasks.workunit.client.0.smithi100.stdout:op 3528 completed, throughput=5MB/sec 2024-04-07T16:50:19.993 INFO:tasks.workunit.client.0.smithi100.stdout: 681: throughput=4.99MB/sec pending data=0 2024-04-07T16:50:19.993 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-iTIJHWovbi9Pcsd off=935620 len=1742085 2024-04-07T16:50:20.006 INFO:tasks.workunit.client.0.smithi100.stdout:op 3529 completed, throughput=5MB/sec 2024-04-07T16:50:20.006 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-I67txPmm2Gb-9mb off=4426030 len=1568552 2024-04-07T16:50:20.016 INFO:tasks.workunit.client.0.smithi100.stdout:op 3530 completed, throughput=5MB/sec 2024-04-07T16:50:20.016 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-vdOhoszAsZ1keUN off=0 len=1192261 2024-04-07T16:50:20.019 INFO:tasks.workunit.client.0.smithi100.stdout:op 3531 completed, throughput=5MB/sec 2024-04-07T16:50:20.019 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-J4FfLd0NhXyfH45 off=435815 len=761716 2024-04-07T16:50:20.021 INFO:tasks.workunit.client.0.smithi100.stdout:op 3532 completed, throughput=5MB/sec 2024-04-07T16:50:20.861 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:50:20 smithi144 bash[66654]: cluster 2024-04-07T16:50:19.433433+0000 mgr.y (mgr.25175) 1049 : cluster [DBG] pgmap v1691: 105 pgs: 105 active+clean; 126 GiB data, 7.9 GiB used, 707 GiB / 715 GiB avail; 2.3 MiB/s rd, 2.5 MiB/s wr, 3 op/s 2024-04-07T16:50:20.946 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:50:20 smithi100 bash[74942]: cluster 2024-04-07T16:50:19.433433+0000 mgr.y (mgr.25175) 1049 : cluster [DBG] pgmap v1691: 105 pgs: 105 active+clean; 126 GiB data, 7.9 GiB used, 707 GiB / 715 GiB avail; 2.3 MiB/s rd, 2.5 MiB/s wr, 3 op/s 2024-04-07T16:50:20.947 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:50:20 smithi100 bash[76202]: cluster 2024-04-07T16:50:19.433433+0000 mgr.y (mgr.25175) 1049 : cluster [DBG] pgmap v1691: 105 pgs: 105 active+clean; 126 GiB data, 7.9 GiB used, 707 GiB / 715 GiB avail; 2.3 MiB/s rd, 2.5 MiB/s wr, 3 op/s 2024-04-07T16:50:21.022 INFO:tasks.workunit.client.0.smithi100.stdout: 682: throughput=4.99MB/sec pending data=0 2024-04-07T16:50:21.022 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-z6ESHkpxE_srxj9 off=0 len=1603715 2024-04-07T16:50:21.026 INFO:tasks.workunit.client.0.smithi100.stdout:op 3533 completed, throughput=5MB/sec 2024-04-07T16:50:21.026 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-iONoKB_AkFCOwf8 off=40462 len=154501 2024-04-07T16:50:21.031 INFO:tasks.workunit.client.0.smithi100.stdout:op 3534 completed, throughput=5MB/sec 2024-04-07T16:50:21.032 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-4Oa5VmzWBLIT_4a off=1290891 len=680078 2024-04-07T16:50:21.034 INFO:tasks.workunit.client.0.smithi100.stdout:op 3535 completed, throughput=5MB/sec 2024-04-07T16:50:21.034 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-GElaxxtQR_rIUtw off=463747 len=614940 2024-04-07T16:50:21.042 INFO:tasks.workunit.client.0.smithi100.stdout:op 3536 completed, throughput=5MB/sec 2024-04-07T16:50:21.042 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-2OhMB91LK8h6uCc off=5311023 len=2034759 2024-04-07T16:50:21.061 INFO:tasks.workunit.client.0.smithi100.stdout:op 3537 completed, throughput=5MB/sec 2024-04-07T16:50:22.061 INFO:tasks.workunit.client.0.smithi100.stdout: 683: throughput=4.99MB/sec pending data=0 2024-04-07T16:50:22.061 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-jfrCKtLH0hJOlAU off=2154327 len=1537581 2024-04-07T16:50:22.065 INFO:tasks.workunit.client.0.smithi100.stdout:op 3538 completed, throughput=5MB/sec 2024-04-07T16:50:22.065 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-aH4weMar08I9vqb off=4042731 len=1525648 2024-04-07T16:50:22.071 INFO:tasks.workunit.client.0.smithi100.stdout:op 3539 completed, throughput=5MB/sec 2024-04-07T16:50:22.071 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-t_wCQwMEkCRMrp1 off=4012327 len=1150837 2024-04-07T16:50:22.081 INFO:tasks.workunit.client.0.smithi100.stdout:op 3540 completed, throughput=5MB/sec 2024-04-07T16:50:22.081 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-JdieS7DhG5i-pEw off=214268 len=829557 2024-04-07T16:50:22.083 INFO:tasks.workunit.client.0.smithi100.stdout:op 3541 completed, throughput=5MB/sec 2024-04-07T16:50:22.083 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-s97c7roxdoc22Ee off=5990899 len=1867493 2024-04-07T16:50:22.089 INFO:tasks.workunit.client.0.smithi100.stdout:op 3542 completed, throughput=5MB/sec 2024-04-07T16:50:22.576 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:50:22 smithi144 bash[95233]: ts=2024-04-07T16:50:22.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:50:22.861 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:50:22 smithi144 bash[66654]: cluster 2024-04-07T16:50:21.434376+0000 mgr.y (mgr.25175) 1050 : cluster [DBG] pgmap v1692: 105 pgs: 105 active+clean; 126 GiB data, 8.0 GiB used, 707 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.4 MiB/s wr, 4 op/s 2024-04-07T16:50:22.947 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:50:22 smithi100 bash[74942]: cluster 2024-04-07T16:50:21.434376+0000 mgr.y (mgr.25175) 1050 : cluster [DBG] pgmap v1692: 105 pgs: 105 active+clean; 126 GiB data, 8.0 GiB used, 707 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.4 MiB/s wr, 4 op/s 2024-04-07T16:50:22.947 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:50:22 smithi100 bash[76202]: cluster 2024-04-07T16:50:21.434376+0000 mgr.y (mgr.25175) 1050 : cluster [DBG] pgmap v1692: 105 pgs: 105 active+clean; 126 GiB data, 8.0 GiB used, 707 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.4 MiB/s wr, 4 op/s 2024-04-07T16:50:23.090 INFO:tasks.workunit.client.0.smithi100.stdout: 684: throughput=5MB/sec pending data=0 2024-04-07T16:50:23.090 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-rI-UlA81GMdQ_Mm off=6748926 len=330928 2024-04-07T16:50:23.096 INFO:tasks.workunit.client.0.smithi100.stdout:op 3543 completed, throughput=5MB/sec 2024-04-07T16:50:23.096 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-MjDr6dzJC0kUt6H off=6172553 len=955259 2024-04-07T16:50:23.105 INFO:tasks.workunit.client.0.smithi100.stdout:op 3544 completed, throughput=5MB/sec 2024-04-07T16:50:23.105 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-VVAwCLfM-aw9PJw off=1617282 len=685882 2024-04-07T16:50:23.108 INFO:tasks.workunit.client.0.smithi100.stdout:op 3545 completed, throughput=5MB/sec 2024-04-07T16:50:23.108 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-O9D0Ake6QS6ucn6 off=8130027 len=354214 2024-04-07T16:50:23.109 INFO:tasks.workunit.client.0.smithi100.stdout:op 3546 completed, throughput=5MB/sec 2024-04-07T16:50:23.109 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-MsVkgZJIpyMlilE off=20363205 len=1700505 2024-04-07T16:50:23.124 INFO:tasks.workunit.client.0.smithi100.stdout:op 3547 completed, throughput=5MB/sec 2024-04-07T16:50:24.125 INFO:tasks.workunit.client.0.smithi100.stdout: 685: throughput=4.99MB/sec pending data=0 2024-04-07T16:50:24.125 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-cE364bAU-BFqPj- off=545334 len=1241464 2024-04-07T16:50:24.129 INFO:tasks.workunit.client.0.smithi100.stdout:op 3548 completed, throughput=4.99MB/sec 2024-04-07T16:50:24.129 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Y78k6IvLqwypoRV off=18392220 len=1717590 2024-04-07T16:50:24.133 INFO:tasks.workunit.client.0.smithi100.stdout:op 3549 completed, throughput=5MB/sec 2024-04-07T16:50:24.133 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-pN5tcHKHrB_I7jg off=304700 len=222905 2024-04-07T16:50:24.138 INFO:tasks.workunit.client.0.smithi100.stdout:op 3550 completed, throughput=5MB/sec 2024-04-07T16:50:24.138 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-92nZF3OsYjT5dI2 off=7971523 len=575563 2024-04-07T16:50:24.142 INFO:tasks.workunit.client.0.smithi100.stdout:op 3551 completed, throughput=5MB/sec 2024-04-07T16:50:24.142 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-5n9k7VQ_tV9oQNr off=1800198 len=1348777 2024-04-07T16:50:24.144 INFO:tasks.workunit.client.0.smithi100.stdout:op 3552 completed, throughput=5MB/sec 2024-04-07T16:50:24.144 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-9QHYBJ3A8k39nUV off=3163203 len=914191 2024-04-07T16:50:24.147 INFO:tasks.workunit.client.0.smithi100.stdout:op 3553 completed, throughput=5MB/sec 2024-04-07T16:50:24.861 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:50:24 smithi144 bash[66654]: cluster 2024-04-07T16:50:23.435552+0000 mgr.y (mgr.25175) 1051 : cluster [DBG] pgmap v1693: 105 pgs: 105 active+clean; 126 GiB data, 8.0 GiB used, 707 GiB / 715 GiB avail; 2.9 MiB/s rd, 2.3 MiB/s wr, 4 op/s 2024-04-07T16:50:24.947 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:50:24 smithi100 bash[74942]: cluster 2024-04-07T16:50:23.435552+0000 mgr.y (mgr.25175) 1051 : cluster [DBG] pgmap v1693: 105 pgs: 105 active+clean; 126 GiB data, 8.0 GiB used, 707 GiB / 715 GiB avail; 2.9 MiB/s rd, 2.3 MiB/s wr, 4 op/s 2024-04-07T16:50:24.947 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:50:24 smithi100 bash[76202]: cluster 2024-04-07T16:50:23.435552+0000 mgr.y (mgr.25175) 1051 : cluster [DBG] pgmap v1693: 105 pgs: 105 active+clean; 126 GiB data, 8.0 GiB used, 707 GiB / 715 GiB avail; 2.9 MiB/s rd, 2.3 MiB/s wr, 4 op/s 2024-04-07T16:50:25.147 INFO:tasks.workunit.client.0.smithi100.stdout: 686: throughput=4.99MB/sec pending data=0 2024-04-07T16:50:25.147 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-OZta3mUWdy1qi6r off=485726 len=430521 2024-04-07T16:50:25.153 INFO:tasks.workunit.client.0.smithi100.stdout:op 3554 completed, throughput=4.99MB/sec 2024-04-07T16:50:25.154 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-rMDB1yqGpzMZtw4 off=1022916 len=212548 2024-04-07T16:50:25.155 INFO:tasks.workunit.client.0.smithi100.stdout:op 3555 completed, throughput=4.99MB/sec 2024-04-07T16:50:25.155 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-lKTr7T9VU9xyebA off=2501271 len=521176 2024-04-07T16:50:25.162 INFO:tasks.workunit.client.0.smithi100.stdout:op 3556 completed, throughput=5MB/sec 2024-04-07T16:50:25.162 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-041h1PyxN4bEAkX off=150964 len=1480903 2024-04-07T16:50:25.173 INFO:tasks.workunit.client.0.smithi100.stdout:op 3557 completed, throughput=5MB/sec 2024-04-07T16:50:25.173 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-IG0PgwqI2t2bbeS off=4540078 len=182736 2024-04-07T16:50:25.178 INFO:tasks.workunit.client.0.smithi100.stdout:op 3558 completed, throughput=5MB/sec 2024-04-07T16:50:25.178 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Qul01xM7hrk2tIs off=4168911 len=347324 2024-04-07T16:50:25.179 INFO:tasks.workunit.client.0.smithi100.stdout:op 3559 completed, throughput=5MB/sec 2024-04-07T16:50:25.179 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-HVn2e8o6ILeX9ZT off=1877827 len=402453 2024-04-07T16:50:25.181 INFO:tasks.workunit.client.0.smithi100.stdout:op 3560 completed, throughput=5MB/sec 2024-04-07T16:50:25.181 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-hafIPVqstdnSV__ off=2845044 len=1959811 2024-04-07T16:50:25.185 INFO:tasks.workunit.client.0.smithi100.stdout:op 3561 completed, throughput=5MB/sec 2024-04-07T16:50:26.186 INFO:tasks.workunit.client.0.smithi100.stdout: 687: throughput=4.99MB/sec pending data=0 2024-04-07T16:50:26.186 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-inZ5iJoT1GBpu-N off=14115010 len=433627 2024-04-07T16:50:26.193 INFO:tasks.workunit.client.0.smithi100.stdout:op 3562 completed, throughput=4.99MB/sec 2024-04-07T16:50:26.193 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-voFw7NRNkaP-NGw off=5477017 len=1772090 2024-04-07T16:50:26.207 INFO:tasks.workunit.client.0.smithi100.stdout:op 3563 completed, throughput=5MB/sec 2024-04-07T16:50:26.207 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-PAuS9qcXFlqPKzI off=0 len=1553665 2024-04-07T16:50:26.219 INFO:tasks.workunit.client.0.smithi100.stdout:op 3564 completed, throughput=5MB/sec 2024-04-07T16:50:26.219 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-JioYju-G9ZQbVpz off=840425 len=655899 2024-04-07T16:50:26.227 INFO:tasks.workunit.client.0.smithi100.stdout:op 3565 completed, throughput=5MB/sec 2024-04-07T16:50:26.227 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-GZ6Do_XECtcB8xI off=3352077 len=1366895 2024-04-07T16:50:26.237 INFO:tasks.workunit.client.0.smithi100.stdout:op 3566 completed, throughput=5MB/sec 2024-04-07T16:50:26.861 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:50:26 smithi144 bash[66654]: cluster 2024-04-07T16:50:25.436498+0000 mgr.y (mgr.25175) 1052 : cluster [DBG] pgmap v1694: 105 pgs: 105 active+clean; 126 GiB data, 8.0 GiB used, 707 GiB / 715 GiB avail; 2.5 MiB/s rd, 1.8 MiB/s wr, 4 op/s 2024-04-07T16:50:26.947 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:50:26 smithi100 bash[74942]: cluster 2024-04-07T16:50:25.436498+0000 mgr.y (mgr.25175) 1052 : cluster [DBG] pgmap v1694: 105 pgs: 105 active+clean; 126 GiB data, 8.0 GiB used, 707 GiB / 715 GiB avail; 2.5 MiB/s rd, 1.8 MiB/s wr, 4 op/s 2024-04-07T16:50:26.947 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:50:26 smithi100 bash[76202]: cluster 2024-04-07T16:50:25.436498+0000 mgr.y (mgr.25175) 1052 : cluster [DBG] pgmap v1694: 105 pgs: 105 active+clean; 126 GiB data, 8.0 GiB used, 707 GiB / 715 GiB avail; 2.5 MiB/s rd, 1.8 MiB/s wr, 4 op/s 2024-04-07T16:50:27.237 INFO:tasks.workunit.client.0.smithi100.stdout: 688: throughput=4.99MB/sec pending data=0 2024-04-07T16:50:27.237 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-fUIqtCknxNQG6oD off=9831990 len=255873 2024-04-07T16:50:27.239 INFO:tasks.workunit.client.0.smithi100.stdout:op 3567 completed, throughput=4.99MB/sec 2024-04-07T16:50:27.239 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-srItPO81fa730yF off=8328490 len=2028531 2024-04-07T16:50:27.254 INFO:tasks.workunit.client.0.smithi100.stdout:op 3568 completed, throughput=5MB/sec 2024-04-07T16:50:27.254 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-YHsPphHVxyIoNfy off=8546480 len=527765 2024-04-07T16:50:27.256 INFO:tasks.workunit.client.0.smithi100.stdout:op 3569 completed, throughput=5MB/sec 2024-04-07T16:50:27.256 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-dvPhSe8NXpqejCY off=20289250 len=686420 2024-04-07T16:50:27.264 INFO:tasks.workunit.client.0.smithi100.stdout:op 3570 completed, throughput=5MB/sec 2024-04-07T16:50:27.264 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-RI-cOghPI14KOoB off=4832811 len=1029494 2024-04-07T16:50:27.266 INFO:tasks.workunit.client.0.smithi100.stdout:op 3571 completed, throughput=5MB/sec 2024-04-07T16:50:28.266 INFO:tasks.workunit.client.0.smithi100.stdout: 689: throughput=4.99MB/sec pending data=0 2024-04-07T16:50:28.266 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Ca37sMZwx74O4K- off=4990391 len=764742 2024-04-07T16:50:28.274 INFO:tasks.workunit.client.0.smithi100.stdout:op 3572 completed, throughput=4.99MB/sec 2024-04-07T16:50:28.274 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-r3auoty-GjrFzMm off=9964931 len=321433 2024-04-07T16:50:28.276 INFO:tasks.workunit.client.0.smithi100.stdout:op 3573 completed, throughput=4.99MB/sec 2024-04-07T16:50:28.276 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj---_K_e6KZ6IuHOX off=7873608 len=717487 2024-04-07T16:50:28.280 INFO:tasks.workunit.client.0.smithi100.stdout:op 3574 completed, throughput=5MB/sec 2024-04-07T16:50:28.280 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-v_n_L5D6OHK2Qq8 off=14873758 len=1937680 2024-04-07T16:50:28.286 INFO:tasks.workunit.client.0.smithi100.stdout:op 3575 completed, throughput=5MB/sec 2024-04-07T16:50:28.286 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-sgv5fu8lqkEAXld off=9049725 len=1734942 2024-04-07T16:50:28.299 INFO:tasks.workunit.client.0.smithi100.stdout:op 3576 completed, throughput=5MB/sec 2024-04-07T16:50:28.611 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:50:28 smithi144 bash[66654]: cluster 2024-04-07T16:50:27.437544+0000 mgr.y (mgr.25175) 1053 : cluster [DBG] pgmap v1695: 105 pgs: 105 active+clean; 126 GiB data, 8.0 GiB used, 707 GiB / 715 GiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 4 op/s 2024-04-07T16:50:28.697 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:50:28 smithi100 bash[74942]: cluster 2024-04-07T16:50:27.437544+0000 mgr.y (mgr.25175) 1053 : cluster [DBG] pgmap v1695: 105 pgs: 105 active+clean; 126 GiB data, 8.0 GiB used, 707 GiB / 715 GiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 4 op/s 2024-04-07T16:50:28.697 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:50:28 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:50:28] "GET /metrics HTTP/1.1" 200 34070 "" "Prometheus/2.43.0" 2024-04-07T16:50:28.697 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:50:28 smithi100 bash[76202]: cluster 2024-04-07T16:50:27.437544+0000 mgr.y (mgr.25175) 1053 : cluster [DBG] pgmap v1695: 105 pgs: 105 active+clean; 126 GiB data, 8.0 GiB used, 707 GiB / 715 GiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 4 op/s 2024-04-07T16:50:29.300 INFO:tasks.workunit.client.0.smithi100.stdout: 690: throughput=4.99MB/sec pending data=0 2024-04-07T16:50:29.300 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-3kYVnZaiXlMFBHe off=7185656 len=2066349 2024-04-07T16:50:29.305 INFO:tasks.workunit.client.0.smithi100.stdout:op 3577 completed, throughput=5MB/sec 2024-04-07T16:50:29.305 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Y78k6IvLqwypoRV off=14130266 len=1539130 2024-04-07T16:50:29.307 INFO:tasks.workunit.client.0.smithi100.stdout:op 3578 completed, throughput=5MB/sec 2024-04-07T16:50:29.307 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-ZXKT41HZlCd3OmW off=4299944 len=688111 2024-04-07T16:50:29.315 INFO:tasks.workunit.client.0.smithi100.stdout:op 3579 completed, throughput=5MB/sec 2024-04-07T16:50:29.315 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-hbNUtsbIvnGO8hm off=3461237 len=596911 2024-04-07T16:50:29.322 INFO:tasks.workunit.client.0.smithi100.stdout:op 3580 completed, throughput=5MB/sec 2024-04-07T16:50:29.322 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-dtwSV6f_Y7Q1FZC off=4177507 len=1167985 2024-04-07T16:50:29.331 INFO:tasks.workunit.client.0.smithi100.stdout:op 3581 completed, throughput=5MB/sec 2024-04-07T16:50:30.332 INFO:tasks.workunit.client.0.smithi100.stdout: 691: throughput=4.99MB/sec pending data=0 2024-04-07T16:50:30.332 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-ukNObl9W7AAEfiA off=13511284 len=60119 2024-04-07T16:50:30.333 INFO:tasks.workunit.client.0.smithi100.stdout:op 3582 completed, throughput=4.99MB/sec 2024-04-07T16:50:30.333 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-YGDCzxNIOLz7Zrh off=6904628 len=1846098 2024-04-07T16:50:30.348 INFO:tasks.workunit.client.0.smithi100.stdout:op 3583 completed, throughput=5MB/sec 2024-04-07T16:50:30.348 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-d8v944Ln9c8KqLs off=8830301 len=1528300 2024-04-07T16:50:30.353 INFO:tasks.workunit.client.0.smithi100.stdout:op 3584 completed, throughput=5MB/sec 2024-04-07T16:50:30.353 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-29JHTgOkTfmDfqB off=8755874 len=183520 2024-04-07T16:50:30.357 INFO:tasks.workunit.client.0.smithi100.stdout:op 3585 completed, throughput=5MB/sec 2024-04-07T16:50:30.358 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Bj5l4p_4ZL4t1cb off=14740837 len=114818 2024-04-07T16:50:30.359 INFO:tasks.workunit.client.0.smithi100.stdout:op 3586 completed, throughput=5MB/sec 2024-04-07T16:50:30.359 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-9jWYNZ9Cn_GVoGW off=13009780 len=855610 2024-04-07T16:50:30.361 INFO:tasks.workunit.client.0.smithi100.stdout:op 3587 completed, throughput=5MB/sec 2024-04-07T16:50:30.861 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:50:30 smithi144 bash[66654]: cluster 2024-04-07T16:50:29.438407+0000 mgr.y (mgr.25175) 1054 : cluster [DBG] pgmap v1696: 105 pgs: 105 active+clean; 126 GiB data, 8.0 GiB used, 707 GiB / 715 GiB avail; 2.3 MiB/s rd, 1.9 MiB/s wr, 4 op/s 2024-04-07T16:50:30.947 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:50:30 smithi100 bash[74942]: cluster 2024-04-07T16:50:29.438407+0000 mgr.y (mgr.25175) 1054 : cluster [DBG] pgmap v1696: 105 pgs: 105 active+clean; 126 GiB data, 8.0 GiB used, 707 GiB / 715 GiB avail; 2.3 MiB/s rd, 1.9 MiB/s wr, 4 op/s 2024-04-07T16:50:30.947 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:50:30 smithi100 bash[76202]: cluster 2024-04-07T16:50:29.438407+0000 mgr.y (mgr.25175) 1054 : cluster [DBG] pgmap v1696: 105 pgs: 105 active+clean; 126 GiB data, 8.0 GiB used, 707 GiB / 715 GiB avail; 2.3 MiB/s rd, 1.9 MiB/s wr, 4 op/s 2024-04-07T16:50:31.362 INFO:tasks.workunit.client.0.smithi100.stdout: 692: throughput=4.99MB/sec pending data=0 2024-04-07T16:50:31.362 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-KUp9sewzlMXB6hY off=1420247 len=234491 2024-04-07T16:50:31.367 INFO:tasks.workunit.client.0.smithi100.stdout:op 3588 completed, throughput=4.99MB/sec 2024-04-07T16:50:31.367 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-quJyBIR49iRm7Iz off=19895709 len=483270 2024-04-07T16:50:31.369 INFO:tasks.workunit.client.0.smithi100.stdout:op 3589 completed, throughput=4.99MB/sec 2024-04-07T16:50:31.369 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-JMw7ErdUpkOj-QD off=320896 len=1591826 2024-04-07T16:50:31.372 INFO:tasks.workunit.client.0.smithi100.stdout:op 3590 completed, throughput=5MB/sec 2024-04-07T16:50:31.372 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-YV00HxG-DCWHS28 off=7657357 len=1856718 2024-04-07T16:50:31.377 INFO:tasks.workunit.client.0.smithi100.stdout:op 3591 completed, throughput=5MB/sec 2024-04-07T16:50:31.377 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-1Iq303OITk7oROu off=8493715 len=1974453 2024-04-07T16:50:31.383 INFO:tasks.workunit.client.0.smithi100.stdout:op 3592 completed, throughput=5MB/sec 2024-04-07T16:50:32.383 INFO:tasks.workunit.client.0.smithi100.stdout: 693: throughput=4.99MB/sec pending data=0 2024-04-07T16:50:32.383 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-BccTIaVJK_mlcWy off=4602287 len=1278063 2024-04-07T16:50:32.386 INFO:tasks.workunit.client.0.smithi100.stdout:op 3593 completed, throughput=5MB/sec 2024-04-07T16:50:32.387 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-HIW1DZDhelsBcf0 off=5057954 len=1159878 2024-04-07T16:50:32.390 INFO:tasks.workunit.client.0.smithi100.stdout:op 3594 completed, throughput=5MB/sec 2024-04-07T16:50:32.390 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-DxCml43gM6kljTq off=11619921 len=1281562 2024-04-07T16:50:32.401 INFO:tasks.workunit.client.0.smithi100.stdout:op 3595 completed, throughput=5MB/sec 2024-04-07T16:50:32.402 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-wCPTwjbkjCpJWxi off=9448645 len=863178 2024-04-07T16:50:32.404 INFO:tasks.workunit.client.0.smithi100.stdout:op 3596 completed, throughput=5MB/sec 2024-04-07T16:50:32.611 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:50:32 smithi144 bash[66654]: cluster 2024-04-07T16:50:31.439336+0000 mgr.y (mgr.25175) 1055 : cluster [DBG] pgmap v1697: 105 pgs: 105 active+clean; 126 GiB data, 8.0 GiB used, 707 GiB / 715 GiB avail; 2.2 MiB/s rd, 2.3 MiB/s wr, 4 op/s 2024-04-07T16:50:32.611 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:50:32 smithi144 bash[66654]: audit 2024-04-07T16:50:31.776217+0000 mon.a (mon.0) 2148 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:50:32.611 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:50:32 smithi144 bash[95233]: ts=2024-04-07T16:50:32.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:50:32.947 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:50:32 smithi100 bash[74942]: cluster 2024-04-07T16:50:31.439336+0000 mgr.y (mgr.25175) 1055 : cluster [DBG] pgmap v1697: 105 pgs: 105 active+clean; 126 GiB data, 8.0 GiB used, 707 GiB / 715 GiB avail; 2.2 MiB/s rd, 2.3 MiB/s wr, 4 op/s 2024-04-07T16:50:32.947 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:50:32 smithi100 bash[74942]: audit 2024-04-07T16:50:31.776217+0000 mon.a (mon.0) 2148 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:50:32.947 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:50:32 smithi100 bash[76202]: cluster 2024-04-07T16:50:31.439336+0000 mgr.y (mgr.25175) 1055 : cluster [DBG] pgmap v1697: 105 pgs: 105 active+clean; 126 GiB data, 8.0 GiB used, 707 GiB / 715 GiB avail; 2.2 MiB/s rd, 2.3 MiB/s wr, 4 op/s 2024-04-07T16:50:32.947 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:50:32 smithi100 bash[76202]: audit 2024-04-07T16:50:31.776217+0000 mon.a (mon.0) 2148 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:50:33.405 INFO:tasks.workunit.client.0.smithi100.stdout: 694: throughput=4.99MB/sec pending data=0 2024-04-07T16:50:33.405 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-npgvcn0qp2cLlYv off=2001613 len=1451734 2024-04-07T16:50:33.416 INFO:tasks.workunit.client.0.smithi100.stdout:op 3597 completed, throughput=5MB/sec 2024-04-07T16:50:33.416 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-v9Zi8DeN1X38ljX off=4114926 len=826597 2024-04-07T16:50:33.424 INFO:tasks.workunit.client.0.smithi100.stdout:op 3598 completed, throughput=5MB/sec 2024-04-07T16:50:33.424 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-CT5IcTcxImikTZ_ off=16317495 len=1837623 2024-04-07T16:50:33.438 INFO:tasks.workunit.client.0.smithi100.stdout:op 3599 completed, throughput=5MB/sec 2024-04-07T16:50:33.438 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-k7bSd33JbUmyit6 off=13365834 len=808206 2024-04-07T16:50:33.446 INFO:tasks.workunit.client.0.smithi100.stdout:op 3600 completed, throughput=5MB/sec 2024-04-07T16:50:33.446 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-e8OAk744A6JPogu off=10430293 len=1627108 2024-04-07T16:50:33.448 INFO:tasks.workunit.client.0.smithi100.stdout:op 3601 completed, throughput=5MB/sec 2024-04-07T16:50:34.448 INFO:tasks.workunit.client.0.smithi100.stdout: 695: throughput=4.99MB/sec pending data=0 2024-04-07T16:50:34.449 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-9mhTgenuy1eGrwu off=1221760 len=1256305 2024-04-07T16:50:34.460 INFO:tasks.workunit.client.0.smithi100.stdout:op 3602 completed, throughput=5MB/sec 2024-04-07T16:50:34.460 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-DQuvVNEICMS1zNI off=3410467 len=494923 2024-04-07T16:50:34.462 INFO:tasks.workunit.client.0.smithi100.stdout:op 3603 completed, throughput=5MB/sec 2024-04-07T16:50:34.462 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-AxeKUwIFg5GeWPg off=2136119 len=1104459 2024-04-07T16:50:34.472 INFO:tasks.workunit.client.0.smithi100.stdout:op 3604 completed, throughput=5MB/sec 2024-04-07T16:50:34.472 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-o5s_rZQr0Mt0vwy off=39796 len=1581593 2024-04-07T16:50:34.476 INFO:tasks.workunit.client.0.smithi100.stdout:op 3605 completed, throughput=5MB/sec 2024-04-07T16:50:34.861 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:50:34 smithi144 bash[66654]: cluster 2024-04-07T16:50:33.440655+0000 mgr.y (mgr.25175) 1056 : cluster [DBG] pgmap v1698: 105 pgs: 105 active+clean; 126 GiB data, 8.0 GiB used, 707 GiB / 715 GiB avail; 2.8 MiB/s rd, 2.5 MiB/s wr, 5 op/s 2024-04-07T16:50:34.947 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:50:34 smithi100 bash[74942]: cluster 2024-04-07T16:50:33.440655+0000 mgr.y (mgr.25175) 1056 : cluster [DBG] pgmap v1698: 105 pgs: 105 active+clean; 126 GiB data, 8.0 GiB used, 707 GiB / 715 GiB avail; 2.8 MiB/s rd, 2.5 MiB/s wr, 5 op/s 2024-04-07T16:50:34.947 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:50:34 smithi100 bash[76202]: cluster 2024-04-07T16:50:33.440655+0000 mgr.y (mgr.25175) 1056 : cluster [DBG] pgmap v1698: 105 pgs: 105 active+clean; 126 GiB data, 8.0 GiB used, 707 GiB / 715 GiB avail; 2.8 MiB/s rd, 2.5 MiB/s wr, 5 op/s 2024-04-07T16:50:35.476 INFO:tasks.workunit.client.0.smithi100.stdout: 696: throughput=4.99MB/sec pending data=0 2024-04-07T16:50:35.476 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj--O4BCbjtNSK8BEm off=9039648 len=2086939 2024-04-07T16:50:35.481 INFO:tasks.workunit.client.0.smithi100.stdout:op 3606 completed, throughput=5MB/sec 2024-04-07T16:50:35.481 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-0-x61CMGFpExpYv off=10177951 len=1219791 2024-04-07T16:50:35.485 INFO:tasks.workunit.client.0.smithi100.stdout:op 3607 completed, throughput=5MB/sec 2024-04-07T16:50:35.485 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-3c_PT2VTL-lvMFq off=14569069 len=2076572 2024-04-07T16:50:35.503 INFO:tasks.workunit.client.0.smithi100.stdout:op 3608 completed, throughput=5MB/sec 2024-04-07T16:50:36.504 INFO:tasks.workunit.client.0.smithi100.stdout: 697: throughput=4.99MB/sec pending data=0 2024-04-07T16:50:36.504 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-oXKnCf9JFZEGzNn off=2931990 len=123143 2024-04-07T16:50:36.505 INFO:tasks.workunit.client.0.smithi100.stdout:op 3609 completed, throughput=4.99MB/sec 2024-04-07T16:50:36.505 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-1gLQYG-giD7ripU off=9339732 len=1417092 2024-04-07T16:50:36.517 INFO:tasks.workunit.client.0.smithi100.stdout:op 3610 completed, throughput=5MB/sec 2024-04-07T16:50:36.518 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-0IILJOkewFRNAqL off=6434949 len=913445 2024-04-07T16:50:36.521 INFO:tasks.workunit.client.0.smithi100.stdout:op 3611 completed, throughput=5MB/sec 2024-04-07T16:50:36.521 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-UZS13PpE5mFTqkQ off=10032680 len=411130 2024-04-07T16:50:36.523 INFO:tasks.workunit.client.0.smithi100.stdout:op 3612 completed, throughput=5MB/sec 2024-04-07T16:50:36.523 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-pqZLV-latV037WS off=17372295 len=1002089 2024-04-07T16:50:36.531 INFO:tasks.workunit.client.0.smithi100.stdout:op 3613 completed, throughput=5MB/sec 2024-04-07T16:50:36.531 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-lj6d8_6d4llk2Dj off=256510 len=1281755 2024-04-07T16:50:36.541 INFO:tasks.workunit.client.0.smithi100.stdout:op 3614 completed, throughput=5MB/sec 2024-04-07T16:50:36.861 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:50:36 smithi144 bash[66654]: cluster 2024-04-07T16:50:35.441736+0000 mgr.y (mgr.25175) 1057 : cluster [DBG] pgmap v1699: 105 pgs: 105 active+clean; 126 GiB data, 8.0 GiB used, 707 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.1 MiB/s wr, 4 op/s 2024-04-07T16:50:36.947 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:50:36 smithi100 bash[74942]: cluster 2024-04-07T16:50:35.441736+0000 mgr.y (mgr.25175) 1057 : cluster [DBG] pgmap v1699: 105 pgs: 105 active+clean; 126 GiB data, 8.0 GiB used, 707 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.1 MiB/s wr, 4 op/s 2024-04-07T16:50:36.947 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:50:36 smithi100 bash[76202]: cluster 2024-04-07T16:50:35.441736+0000 mgr.y (mgr.25175) 1057 : cluster [DBG] pgmap v1699: 105 pgs: 105 active+clean; 126 GiB data, 8.0 GiB used, 707 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.1 MiB/s wr, 4 op/s 2024-04-07T16:50:37.541 INFO:tasks.workunit.client.0.smithi100.stdout: 698: throughput=4.99MB/sec pending data=0 2024-04-07T16:50:37.542 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-hph-b0CRyfaXEqv off=2242116 len=1435147 2024-04-07T16:50:37.552 INFO:tasks.workunit.client.0.smithi100.stdout:op 3615 completed, throughput=4.99MB/sec 2024-04-07T16:50:37.553 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-vYnQvNDwa5hKAFq off=1179405 len=157355 2024-04-07T16:50:37.556 INFO:tasks.workunit.client.0.smithi100.stdout:op 3616 completed, throughput=5MB/sec 2024-04-07T16:50:37.556 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-CbeOmZsiFq7A-hj off=17722697 len=777018 2024-04-07T16:50:37.564 INFO:tasks.workunit.client.0.smithi100.stdout:op 3617 completed, throughput=5MB/sec 2024-04-07T16:50:37.564 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-4X504_c-KDnAIHi off=3807594 len=937434 2024-04-07T16:50:37.566 INFO:tasks.workunit.client.0.smithi100.stdout:op 3618 completed, throughput=5MB/sec 2024-04-07T16:50:37.567 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-WmzZu8JyfG0VyEq off=2756525 len=494288 2024-04-07T16:50:37.574 INFO:tasks.workunit.client.0.smithi100.stdout:op 3619 completed, throughput=5MB/sec 2024-04-07T16:50:37.574 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Twp15MLmCEYmv2N off=5276754 len=1806890 2024-04-07T16:50:37.587 INFO:tasks.workunit.client.0.smithi100.stdout:op 3620 completed, throughput=5MB/sec 2024-04-07T16:50:38.588 INFO:tasks.workunit.client.0.smithi100.stdout: 699: throughput=4.99MB/sec pending data=0 2024-04-07T16:50:38.589 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-nzrVENG4Ujt-mzg off=19523437 len=1546889 2024-04-07T16:50:38.591 INFO:tasks.workunit.client.0.smithi100.stdout:op 3621 completed, throughput=5MB/sec 2024-04-07T16:50:38.591 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-9yABG-nKKCRZfzP off=6607841 len=244767 2024-04-07T16:50:38.597 INFO:tasks.workunit.client.0.smithi100.stdout:op 3622 completed, throughput=5MB/sec 2024-04-07T16:50:38.597 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-gnEHuBTUf3vu6ia off=791730 len=371296 2024-04-07T16:50:38.603 INFO:tasks.workunit.client.0.smithi100.stdout:op 3623 completed, throughput=5MB/sec 2024-04-07T16:50:38.603 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-iZNiSCEhf7Kss7U off=2088710 len=552682 2024-04-07T16:50:38.605 INFO:tasks.workunit.client.0.smithi100.stdout:op 3624 completed, throughput=5MB/sec 2024-04-07T16:50:38.605 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-HS4Mn74nJ-OZWLJ off=8962924 len=1669356 2024-04-07T16:50:38.608 INFO:tasks.workunit.client.0.smithi100.stdout:op 3625 completed, throughput=5MB/sec 2024-04-07T16:50:38.608 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-zA4dx9B6RPAqw-L off=13908872 len=1281618 2024-04-07T16:50:38.619 INFO:tasks.workunit.client.0.smithi100.stdout:op 3626 completed, throughput=5MB/sec 2024-04-07T16:50:38.697 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:50:38 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:50:38] "GET /metrics HTTP/1.1" 200 34072 "" "Prometheus/2.43.0" 2024-04-07T16:50:38.697 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:50:38 smithi100 bash[74942]: cluster 2024-04-07T16:50:37.442791+0000 mgr.y (mgr.25175) 1058 : cluster [DBG] pgmap v1700: 105 pgs: 105 active+clean; 126 GiB data, 8.1 GiB used, 707 GiB / 715 GiB avail; 2.9 MiB/s rd, 2.4 MiB/s wr, 5 op/s 2024-04-07T16:50:38.697 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:50:38 smithi100 bash[76202]: cluster 2024-04-07T16:50:37.442791+0000 mgr.y (mgr.25175) 1058 : cluster [DBG] pgmap v1700: 105 pgs: 105 active+clean; 126 GiB data, 8.1 GiB used, 707 GiB / 715 GiB avail; 2.9 MiB/s rd, 2.4 MiB/s wr, 5 op/s 2024-04-07T16:50:38.861 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:50:38 smithi144 bash[66654]: cluster 2024-04-07T16:50:37.442791+0000 mgr.y (mgr.25175) 1058 : cluster [DBG] pgmap v1700: 105 pgs: 105 active+clean; 126 GiB data, 8.1 GiB used, 707 GiB / 715 GiB avail; 2.9 MiB/s rd, 2.4 MiB/s wr, 5 op/s 2024-04-07T16:50:39.619 INFO:tasks.workunit.client.0.smithi100.stdout: 700: throughput=4.99MB/sec pending data=0 2024-04-07T16:50:39.619 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-QC1yTPrYfAgSiLs off=10328303 len=368814 2024-04-07T16:50:39.625 INFO:tasks.workunit.client.0.smithi100.stdout:op 3627 completed, throughput=4.99MB/sec 2024-04-07T16:50:39.625 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-QdyrvToSkU0bn8J off=9202966 len=1230564 2024-04-07T16:50:39.636 INFO:tasks.workunit.client.0.smithi100.stdout:op 3628 completed, throughput=5MB/sec 2024-04-07T16:50:39.636 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-sAsK5Hg3NxhM1Oa off=424632 len=1112074 2024-04-07T16:50:39.640 INFO:tasks.workunit.client.0.smithi100.stdout:op 3629 completed, throughput=5MB/sec 2024-04-07T16:50:39.640 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-tuEr4J8bm6HkOHc off=486637 len=291014 2024-04-07T16:50:39.645 INFO:tasks.workunit.client.0.smithi100.stdout:op 3630 completed, throughput=5MB/sec 2024-04-07T16:50:39.646 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-symwWLhu_1tbwrh off=9806090 len=123124 2024-04-07T16:50:39.647 INFO:tasks.workunit.client.0.smithi100.stdout:op 3631 completed, throughput=5MB/sec 2024-04-07T16:50:39.647 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-h2uBeqjHhGT73Nc off=1307430 len=8143 2024-04-07T16:50:39.652 INFO:tasks.workunit.client.0.smithi100.stdout:op 3632 completed, throughput=5MB/sec 2024-04-07T16:50:39.652 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-9UowMnaALyZnQHw off=14636258 len=1489626 2024-04-07T16:50:39.655 INFO:tasks.workunit.client.0.smithi100.stdout:op 3633 completed, throughput=5MB/sec 2024-04-07T16:50:39.655 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-3vhqBzlUVJ4fayW off=15838140 len=1495835 2024-04-07T16:50:39.667 INFO:tasks.workunit.client.0.smithi100.stdout:op 3634 completed, throughput=5MB/sec 2024-04-07T16:50:40.668 INFO:tasks.workunit.client.0.smithi100.stdout: 701: throughput=4.99MB/sec pending data=0 2024-04-07T16:50:40.668 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-XR5B3HzDz03rAza off=6254619 len=495694 2024-04-07T16:50:40.670 INFO:tasks.workunit.client.0.smithi100.stdout:op 3635 completed, throughput=5MB/sec 2024-04-07T16:50:40.670 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-90FySeUWUE8oo9X off=9755261 len=485911 2024-04-07T16:50:40.676 INFO:tasks.workunit.client.0.smithi100.stdout:op 3636 completed, throughput=5MB/sec 2024-04-07T16:50:40.676 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-7z2gOIKeuhcaTns off=2659050 len=1071529 2024-04-07T16:50:40.686 INFO:tasks.workunit.client.0.smithi100.stdout:op 3637 completed, throughput=5MB/sec 2024-04-07T16:50:40.686 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-GAELsC27EP0jyb_ off=11796407 len=973991 2024-04-07T16:50:40.695 INFO:tasks.workunit.client.0.smithi100.stdout:op 3638 completed, throughput=5MB/sec 2024-04-07T16:50:40.695 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-1mK5Y79ZKrKdtX9 off=9621759 len=654590 2024-04-07T16:50:40.697 INFO:tasks.workunit.client.0.smithi100.stdout:op 3639 completed, throughput=5MB/sec 2024-04-07T16:50:40.697 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-GiPyL5bLdna0agD off=5566320 len=286485 2024-04-07T16:50:40.699 INFO:tasks.workunit.client.0.smithi100.stdout:op 3640 completed, throughput=5MB/sec 2024-04-07T16:50:40.699 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Xa4pjKXTSeXRYE8 off=7740908 len=1131562 2024-04-07T16:50:40.709 INFO:tasks.workunit.client.0.smithi100.stdout:op 3641 completed, throughput=5MB/sec 2024-04-07T16:50:40.861 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:50:40 smithi144 bash[66654]: cluster 2024-04-07T16:50:39.443666+0000 mgr.y (mgr.25175) 1059 : cluster [DBG] pgmap v1701: 105 pgs: 105 active+clean; 126 GiB data, 8.1 GiB used, 707 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.4 MiB/s wr, 4 op/s 2024-04-07T16:50:40.947 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:50:40 smithi100 bash[74942]: cluster 2024-04-07T16:50:39.443666+0000 mgr.y (mgr.25175) 1059 : cluster [DBG] pgmap v1701: 105 pgs: 105 active+clean; 126 GiB data, 8.1 GiB used, 707 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.4 MiB/s wr, 4 op/s 2024-04-07T16:50:40.947 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:50:40 smithi100 bash[76202]: cluster 2024-04-07T16:50:39.443666+0000 mgr.y (mgr.25175) 1059 : cluster [DBG] pgmap v1701: 105 pgs: 105 active+clean; 126 GiB data, 8.1 GiB used, 707 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.4 MiB/s wr, 4 op/s 2024-04-07T16:50:41.709 INFO:tasks.workunit.client.0.smithi100.stdout: 702: throughput=4.99MB/sec pending data=0 2024-04-07T16:50:41.709 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-HaZ_PTUhrq_Hokx off=13158164 len=1505851 2024-04-07T16:50:41.723 INFO:tasks.workunit.client.0.smithi100.stdout:op 3642 completed, throughput=5MB/sec 2024-04-07T16:50:41.723 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-hTT_iSIkAEZlwWi off=4635990 len=1085037 2024-04-07T16:50:41.733 INFO:tasks.workunit.client.0.smithi100.stdout:op 3643 completed, throughput=5MB/sec 2024-04-07T16:50:41.733 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-fiU-4nGyHlJ0QZf off=34198 len=1383197 2024-04-07T16:50:41.743 INFO:tasks.workunit.client.0.smithi100.stdout:op 3644 completed, throughput=5MB/sec 2024-04-07T16:50:41.743 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-ctq8HrBLZd8fX2P off=1168951 len=1470065 2024-04-07T16:50:41.745 INFO:tasks.workunit.client.0.smithi100.stdout:op 3645 completed, throughput=5MB/sec 2024-04-07T16:50:42.611 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:50:42 smithi144 bash[66654]: cluster 2024-04-07T16:50:41.444557+0000 mgr.y (mgr.25175) 1060 : cluster [DBG] pgmap v1702: 105 pgs: 105 active+clean; 126 GiB data, 8.1 GiB used, 707 GiB / 715 GiB avail; 2.4 MiB/s rd, 2.6 MiB/s wr, 4 op/s 2024-04-07T16:50:42.611 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:50:42 smithi144 bash[95233]: ts=2024-04-07T16:50:42.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:50:42.745 INFO:tasks.workunit.client.0.smithi100.stdout: 703: throughput=4.99MB/sec pending data=0 2024-04-07T16:50:42.745 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-oNjlSN8bMmqjn05 off=20530323 len=1032792 2024-04-07T16:50:42.755 INFO:tasks.workunit.client.0.smithi100.stdout:op 3646 completed, throughput=5MB/sec 2024-04-07T16:50:42.755 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-YlRC-xkSaf9Ng0f off=843676 len=1226675 2024-04-07T16:50:42.766 INFO:tasks.workunit.client.0.smithi100.stdout:op 3647 completed, throughput=5MB/sec 2024-04-07T16:50:42.766 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-eZVW4Aj39w1EMp3 off=784934 len=476813 2024-04-07T16:50:42.768 INFO:tasks.workunit.client.0.smithi100.stdout:op 3648 completed, throughput=5MB/sec 2024-04-07T16:50:42.768 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-VVFi_vTKD3Gy8YX off=15089257 len=544770 2024-04-07T16:50:42.770 INFO:tasks.workunit.client.0.smithi100.stdout:op 3649 completed, throughput=5MB/sec 2024-04-07T16:50:42.770 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-bDciCkTVPPYiFYb off=7139675 len=2002179 2024-04-07T16:50:42.790 INFO:tasks.workunit.client.0.smithi100.stdout:op 3650 completed, throughput=5MB/sec 2024-04-07T16:50:42.947 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:50:42 smithi100 bash[74942]: cluster 2024-04-07T16:50:41.444557+0000 mgr.y (mgr.25175) 1060 : cluster [DBG] pgmap v1702: 105 pgs: 105 active+clean; 126 GiB data, 8.1 GiB used, 707 GiB / 715 GiB avail; 2.4 MiB/s rd, 2.6 MiB/s wr, 4 op/s 2024-04-07T16:50:42.949 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:50:42 smithi100 bash[76202]: cluster 2024-04-07T16:50:41.444557+0000 mgr.y (mgr.25175) 1060 : cluster [DBG] pgmap v1702: 105 pgs: 105 active+clean; 126 GiB data, 8.1 GiB used, 707 GiB / 715 GiB avail; 2.4 MiB/s rd, 2.6 MiB/s wr, 4 op/s 2024-04-07T16:50:43.790 INFO:tasks.workunit.client.0.smithi100.stdout: 704: throughput=4.99MB/sec pending data=0 2024-04-07T16:50:43.790 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-fhiAHJynl9VyiQF off=2865835 len=582394 2024-04-07T16:50:43.797 INFO:tasks.workunit.client.0.smithi100.stdout:op 3651 completed, throughput=4.99MB/sec 2024-04-07T16:50:43.798 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-XcQy71J0F7AQI3r off=1531979 len=1101214 2024-04-07T16:50:43.807 INFO:tasks.workunit.client.0.smithi100.stdout:op 3652 completed, throughput=5MB/sec 2024-04-07T16:50:43.807 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-MUHkoRtXNkfZsH7 off=7305589 len=32390 2024-04-07T16:50:43.812 INFO:tasks.workunit.client.0.smithi100.stdout:op 3653 completed, throughput=5MB/sec 2024-04-07T16:50:43.812 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-U9eYtSq41LvjHLT off=4917279 len=244709 2024-04-07T16:50:43.813 INFO:tasks.workunit.client.0.smithi100.stdout:op 3654 completed, throughput=5MB/sec 2024-04-07T16:50:43.813 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-1sCCLuK3GIWpkoj off=1035447 len=652748 2024-04-07T16:50:43.815 INFO:tasks.workunit.client.0.smithi100.stdout:op 3655 completed, throughput=5MB/sec 2024-04-07T16:50:43.815 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-MxDg61vRh6jE4W6 off=6856583 len=1043965 2024-04-07T16:50:43.823 INFO:tasks.workunit.client.0.smithi100.stdout:op 3656 completed, throughput=5MB/sec 2024-04-07T16:50:43.823 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-ARNFXo6zJV3L98b off=16782819 len=536168 2024-04-07T16:50:43.830 INFO:tasks.workunit.client.0.smithi100.stdout:op 3657 completed, throughput=5MB/sec 2024-04-07T16:50:43.830 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-JidKFOEJl_Cz2r9 off=7480455 len=1871043 2024-04-07T16:50:43.837 INFO:tasks.workunit.client.0.smithi100.stdout:op 3658 completed, throughput=5MB/sec 2024-04-07T16:50:44.837 INFO:tasks.workunit.client.0.smithi100.stdout: 705: throughput=4.99MB/sec pending data=0 2024-04-07T16:50:44.837 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-0W_Lf11nL8qkMji off=1044741 len=1408417 2024-04-07T16:50:44.842 INFO:tasks.workunit.client.0.smithi100.stdout:op 3659 completed, throughput=5MB/sec 2024-04-07T16:50:44.842 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-pceK9CZUGBjzgoq off=22085944 len=1962657 2024-04-07T16:50:44.861 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:50:44 smithi144 bash[66654]: cluster 2024-04-07T16:50:43.445680+0000 mgr.y (mgr.25175) 1061 : cluster [DBG] pgmap v1703: 105 pgs: 105 active+clean; 126 GiB data, 8.1 GiB used, 707 GiB / 715 GiB avail; 2.9 MiB/s rd, 2.8 MiB/s wr, 5 op/s 2024-04-07T16:50:44.861 INFO:tasks.workunit.client.0.smithi100.stdout:op 3660 completed, throughput=5MB/sec 2024-04-07T16:50:44.862 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-_AEmcFW_kMZBDrn off=7178519 len=1517242 2024-04-07T16:50:44.864 INFO:tasks.workunit.client.0.smithi100.stdout:op 3661 completed, throughput=5MB/sec 2024-04-07T16:50:44.947 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:50:44 smithi100 bash[74942]: cluster 2024-04-07T16:50:43.445680+0000 mgr.y (mgr.25175) 1061 : cluster [DBG] pgmap v1703: 105 pgs: 105 active+clean; 126 GiB data, 8.1 GiB used, 707 GiB / 715 GiB avail; 2.9 MiB/s rd, 2.8 MiB/s wr, 5 op/s 2024-04-07T16:50:44.947 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:50:44 smithi100 bash[76202]: cluster 2024-04-07T16:50:43.445680+0000 mgr.y (mgr.25175) 1061 : cluster [DBG] pgmap v1703: 105 pgs: 105 active+clean; 126 GiB data, 8.1 GiB used, 707 GiB / 715 GiB avail; 2.9 MiB/s rd, 2.8 MiB/s wr, 5 op/s 2024-04-07T16:50:45.865 INFO:tasks.workunit.client.0.smithi100.stdout: 706: throughput=4.99MB/sec pending data=0 2024-04-07T16:50:45.865 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-2V6eJOCAN1mR5sz off=155737 len=2006963 2024-04-07T16:50:45.881 INFO:tasks.workunit.client.0.smithi100.stdout:op 3662 completed, throughput=5MB/sec 2024-04-07T16:50:45.881 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-bDHSmgHLBfwypVH off=4129768 len=847645 2024-04-07T16:50:45.888 INFO:tasks.workunit.client.0.smithi100.stdout:op 3663 completed, throughput=5MB/sec 2024-04-07T16:50:45.888 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-aIP5Xiq_W6wDd0M off=7460836 len=189603 2024-04-07T16:50:45.894 INFO:tasks.workunit.client.0.smithi100.stdout:op 3664 completed, throughput=5MB/sec 2024-04-07T16:50:45.894 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-g5tFYNB0FMt9QtH off=17399990 len=1431835 2024-04-07T16:50:45.905 INFO:tasks.workunit.client.0.smithi100.stdout:op 3665 completed, throughput=5MB/sec 2024-04-07T16:50:45.906 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-lBQkSdNPKot1zrJ off=14499163 len=270259 2024-04-07T16:50:45.911 INFO:tasks.workunit.client.0.smithi100.stdout:op 3666 completed, throughput=5MB/sec 2024-04-07T16:50:45.911 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-FsiVouXLu3yZREP off=7827638 len=476007 2024-04-07T16:50:45.914 INFO:tasks.workunit.client.0.smithi100.stdout:op 3667 completed, throughput=5MB/sec 2024-04-07T16:50:46.861 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:50:46 smithi144 bash[66654]: cluster 2024-04-07T16:50:45.446639+0000 mgr.y (mgr.25175) 1062 : cluster [DBG] pgmap v1704: 105 pgs: 105 active+clean; 126 GiB data, 8.1 GiB used, 707 GiB / 715 GiB avail; 2.0 MiB/s rd, 2.8 MiB/s wr, 5 op/s 2024-04-07T16:50:46.914 INFO:tasks.workunit.client.0.smithi100.stdout: 707: throughput=4.99MB/sec pending data=0 2024-04-07T16:50:46.914 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-TSX003OjA6jz-Xl off=17950382 len=541906 2024-04-07T16:50:46.917 INFO:tasks.workunit.client.0.smithi100.stdout:op 3668 completed, throughput=4.99MB/sec 2024-04-07T16:50:46.917 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-wF-iT2kFP0oZKeA off=2895894 len=2078976 2024-04-07T16:50:46.932 INFO:tasks.workunit.client.0.smithi100.stdout:op 3669 completed, throughput=5MB/sec 2024-04-07T16:50:46.932 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-aVaM3eU9iZtSweu off=8063897 len=1720444 2024-04-07T16:50:46.936 INFO:tasks.workunit.client.0.smithi100.stdout:op 3670 completed, throughput=5MB/sec 2024-04-07T16:50:46.936 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj--QrS72ie3X4zEhx off=19026883 len=1841739 2024-04-07T16:50:46.940 INFO:tasks.workunit.client.0.smithi100.stdout:op 3671 completed, throughput=5MB/sec 2024-04-07T16:50:46.947 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:50:46 smithi100 bash[74942]: cluster 2024-04-07T16:50:45.446639+0000 mgr.y (mgr.25175) 1062 : cluster [DBG] pgmap v1704: 105 pgs: 105 active+clean; 126 GiB data, 8.1 GiB used, 707 GiB / 715 GiB avail; 2.0 MiB/s rd, 2.8 MiB/s wr, 5 op/s 2024-04-07T16:50:46.947 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:50:46 smithi100 bash[76202]: cluster 2024-04-07T16:50:45.446639+0000 mgr.y (mgr.25175) 1062 : cluster [DBG] pgmap v1704: 105 pgs: 105 active+clean; 126 GiB data, 8.1 GiB used, 707 GiB / 715 GiB avail; 2.0 MiB/s rd, 2.8 MiB/s wr, 5 op/s 2024-04-07T16:50:47.861 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:50:47 smithi144 bash[66654]: audit 2024-04-07T16:50:46.776539+0000 mon.a (mon.0) 2149 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:50:47.941 INFO:tasks.workunit.client.0.smithi100.stdout: 708: throughput=4.99MB/sec pending data=0 2024-04-07T16:50:47.941 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-fhiAHJynl9VyiQF off=11187150 len=1953431 2024-04-07T16:50:47.947 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:50:47 smithi100 bash[74942]: audit 2024-04-07T16:50:46.776539+0000 mon.a (mon.0) 2149 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:50:47.947 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:50:47 smithi100 bash[76202]: audit 2024-04-07T16:50:46.776539+0000 mon.a (mon.0) 2149 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:50:47.957 INFO:tasks.workunit.client.0.smithi100.stdout:op 3672 completed, throughput=5MB/sec 2024-04-07T16:50:47.957 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-6SISLknx0Y-ICXA off=9179859 len=1892715 2024-04-07T16:50:47.971 INFO:tasks.workunit.client.0.smithi100.stdout:op 3673 completed, throughput=5MB/sec 2024-04-07T16:50:47.971 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-9JSWC6ZfoDJGPHU off=13536299 len=1976462 2024-04-07T16:50:47.986 INFO:tasks.workunit.client.0.smithi100.stdout:op 3674 completed, throughput=5MB/sec 2024-04-07T16:50:48.697 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:50:48 smithi100 bash[74942]: cluster 2024-04-07T16:50:47.447294+0000 mgr.y (mgr.25175) 1063 : cluster [DBG] pgmap v1705: 105 pgs: 105 active+clean; 126 GiB data, 8.2 GiB used, 707 GiB / 715 GiB avail; 1.9 MiB/s rd, 3.3 MiB/s wr, 5 op/s 2024-04-07T16:50:48.697 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:50:48 smithi100 bash[76202]: cluster 2024-04-07T16:50:47.447294+0000 mgr.y (mgr.25175) 1063 : cluster [DBG] pgmap v1705: 105 pgs: 105 active+clean; 126 GiB data, 8.2 GiB used, 707 GiB / 715 GiB avail; 1.9 MiB/s rd, 3.3 MiB/s wr, 5 op/s 2024-04-07T16:50:48.698 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:50:48 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:50:48] "GET /metrics HTTP/1.1" 200 34070 "" "Prometheus/2.43.0" 2024-04-07T16:50:48.861 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:50:48 smithi144 bash[66654]: cluster 2024-04-07T16:50:47.447294+0000 mgr.y (mgr.25175) 1063 : cluster [DBG] pgmap v1705: 105 pgs: 105 active+clean; 126 GiB data, 8.2 GiB used, 707 GiB / 715 GiB avail; 1.9 MiB/s rd, 3.3 MiB/s wr, 5 op/s 2024-04-07T16:50:48.987 INFO:tasks.workunit.client.0.smithi100.stdout: 709: throughput=5MB/sec pending data=0 2024-04-07T16:50:48.987 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-nDEOpH8sHQzxySY off=3650974 len=439928 2024-04-07T16:50:48.994 INFO:tasks.workunit.client.0.smithi100.stdout:op 3675 completed, throughput=5MB/sec 2024-04-07T16:50:48.994 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-6aXRXYyxwwYqdv7 off=8595545 len=1389369 2024-04-07T16:50:49.005 INFO:tasks.workunit.client.0.smithi100.stdout:op 3676 completed, throughput=5MB/sec 2024-04-07T16:50:49.005 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-_kuM3jE3JbX6p6J off=0 len=1272282 2024-04-07T16:50:49.010 INFO:tasks.workunit.client.0.smithi100.stdout:op 3677 completed, throughput=5MB/sec 2024-04-07T16:50:49.010 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-bNAPPrVWoy8RPAF off=1402059 len=1745730 2024-04-07T16:50:49.015 INFO:tasks.workunit.client.0.smithi100.stdout:op 3678 completed, throughput=5MB/sec 2024-04-07T16:50:50.016 INFO:tasks.workunit.client.0.smithi100.stdout: 710: throughput=4.99MB/sec pending data=0 2024-04-07T16:50:50.016 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-ND4_XAuBTrOmS1c off=977911 len=169105 2024-04-07T16:50:50.022 INFO:tasks.workunit.client.0.smithi100.stdout:op 3679 completed, throughput=4.99MB/sec 2024-04-07T16:50:50.022 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Bx2gv5MF_sfnJ6i off=4020397 len=1575267 2024-04-07T16:50:50.035 INFO:tasks.workunit.client.0.smithi100.stdout:op 3680 completed, throughput=5MB/sec 2024-04-07T16:50:50.035 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-biUATuybCfLimT9 off=1909425 len=11795 2024-04-07T16:50:50.037 INFO:tasks.workunit.client.0.smithi100.stdout:op 3681 completed, throughput=5MB/sec 2024-04-07T16:50:50.037 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-r44RYtAjkwsKc5- off=852766 len=1516615 2024-04-07T16:50:50.049 INFO:tasks.workunit.client.0.smithi100.stdout:op 3682 completed, throughput=5MB/sec 2024-04-07T16:50:50.049 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-aU_BgVZ0MfbwZQO off=7185773 len=1625812 2024-04-07T16:50:50.052 INFO:tasks.workunit.client.0.smithi100.stdout:op 3683 completed, throughput=5MB/sec 2024-04-07T16:50:50.861 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:50:50 smithi144 bash[66654]: cluster 2024-04-07T16:50:49.448110+0000 mgr.y (mgr.25175) 1064 : cluster [DBG] pgmap v1706: 105 pgs: 105 active+clean; 126 GiB data, 8.2 GiB used, 707 GiB / 715 GiB avail; 1.8 MiB/s rd, 2.9 MiB/s wr, 5 op/s 2024-04-07T16:50:50.947 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:50:50 smithi100 bash[74942]: cluster 2024-04-07T16:50:49.448110+0000 mgr.y (mgr.25175) 1064 : cluster [DBG] pgmap v1706: 105 pgs: 105 active+clean; 126 GiB data, 8.2 GiB used, 707 GiB / 715 GiB avail; 1.8 MiB/s rd, 2.9 MiB/s wr, 5 op/s 2024-04-07T16:50:50.947 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:50:50 smithi100 bash[76202]: cluster 2024-04-07T16:50:49.448110+0000 mgr.y (mgr.25175) 1064 : cluster [DBG] pgmap v1706: 105 pgs: 105 active+clean; 126 GiB data, 8.2 GiB used, 707 GiB / 715 GiB avail; 1.8 MiB/s rd, 2.9 MiB/s wr, 5 op/s 2024-04-07T16:50:51.053 INFO:tasks.workunit.client.0.smithi100.stdout: 711: throughput=4.99MB/sec pending data=0 2024-04-07T16:50:51.053 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-ukhSb4MXgRFZtYf off=412638 len=1500007 2024-04-07T16:50:51.065 INFO:tasks.workunit.client.0.smithi100.stdout:op 3684 completed, throughput=5MB/sec 2024-04-07T16:50:51.065 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-3vBPvLicY5j61ey off=1454416 len=391981 2024-04-07T16:50:51.067 INFO:tasks.workunit.client.0.smithi100.stdout:op 3685 completed, throughput=5MB/sec 2024-04-07T16:50:51.068 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-OXbdKkld19V1T3G off=9411777 len=796532 2024-04-07T16:50:51.070 INFO:tasks.workunit.client.0.smithi100.stdout:op 3686 completed, throughput=5MB/sec 2024-04-07T16:50:51.071 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-2ufmOx8S6fwgPCO off=351495 len=1109310 2024-04-07T16:50:51.079 INFO:tasks.workunit.client.0.smithi100.stdout:op 3687 completed, throughput=5MB/sec 2024-04-07T16:50:51.079 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-dxgErCEwqrl2PRC off=6432737 len=917484 2024-04-07T16:50:51.082 INFO:tasks.workunit.client.0.smithi100.stdout:op 3688 completed, throughput=5MB/sec 2024-04-07T16:50:51.082 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-J1UB_RnBw6hh3L_ off=1177172 len=1906840 2024-04-07T16:50:51.088 INFO:tasks.workunit.client.0.smithi100.stdout:op 3689 completed, throughput=5MB/sec 2024-04-07T16:50:52.089 INFO:tasks.workunit.client.0.smithi100.stdout: 712: throughput=5MB/sec pending data=0 2024-04-07T16:50:52.089 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-h4S-YMgM2N0lzJp off=8078328 len=1875908 2024-04-07T16:50:52.094 INFO:tasks.workunit.client.0.smithi100.stdout:op 3690 completed, throughput=5MB/sec 2024-04-07T16:50:52.094 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-J1pgDlZB36y5lRI off=451292 len=1797098 2024-04-07T16:50:52.100 INFO:tasks.workunit.client.0.smithi100.stdout:op 3691 completed, throughput=5MB/sec 2024-04-07T16:50:52.100 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj--1JRaqKzct5e8VE off=5414277 len=1547492 2024-04-07T16:50:52.113 INFO:tasks.workunit.client.0.smithi100.stdout:op 3692 completed, throughput=5MB/sec 2024-04-07T16:50:52.611 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:50:52 smithi144 bash[66654]: cluster 2024-04-07T16:50:51.449048+0000 mgr.y (mgr.25175) 1065 : cluster [DBG] pgmap v1707: 105 pgs: 105 active+clean; 126 GiB data, 8.2 GiB used, 707 GiB / 715 GiB avail; 1.9 MiB/s rd, 2.8 MiB/s wr, 4 op/s 2024-04-07T16:50:52.611 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:50:52 smithi144 bash[95233]: ts=2024-04-07T16:50:52.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:50:52.947 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:50:52 smithi100 bash[74942]: cluster 2024-04-07T16:50:51.449048+0000 mgr.y (mgr.25175) 1065 : cluster [DBG] pgmap v1707: 105 pgs: 105 active+clean; 126 GiB data, 8.2 GiB used, 707 GiB / 715 GiB avail; 1.9 MiB/s rd, 2.8 MiB/s wr, 4 op/s 2024-04-07T16:50:52.947 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:50:52 smithi100 bash[76202]: cluster 2024-04-07T16:50:51.449048+0000 mgr.y (mgr.25175) 1065 : cluster [DBG] pgmap v1707: 105 pgs: 105 active+clean; 126 GiB data, 8.2 GiB used, 707 GiB / 715 GiB avail; 1.9 MiB/s rd, 2.8 MiB/s wr, 4 op/s 2024-04-07T16:50:53.114 INFO:tasks.workunit.client.0.smithi100.stdout: 713: throughput=5MB/sec pending data=0 2024-04-07T16:50:53.114 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-4q-DwkG71fMXb8O off=19207026 len=144510 2024-04-07T16:50:53.115 INFO:tasks.workunit.client.0.smithi100.stdout:op 3693 completed, throughput=5MB/sec 2024-04-07T16:50:53.115 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-tV3I3kmBFbw6BX8 off=5868491 len=1477087 2024-04-07T16:50:53.127 INFO:tasks.workunit.client.0.smithi100.stdout:op 3694 completed, throughput=5MB/sec 2024-04-07T16:50:53.127 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-ZSzJKeRCJnYKVUW off=15957940 len=1170037 2024-04-07T16:50:53.141 INFO:tasks.workunit.client.0.smithi100.stdout:op 3695 completed, throughput=5MB/sec 2024-04-07T16:50:53.141 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-VQ5LLh_tyWvk5ic off=19218663 len=1801175 2024-04-07T16:50:53.154 INFO:tasks.workunit.client.0.smithi100.stdout:op 3696 completed, throughput=5MB/sec 2024-04-07T16:50:54.155 INFO:tasks.workunit.client.0.smithi100.stdout: 714: throughput=4.99MB/sec pending data=0 2024-04-07T16:50:54.155 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-FKPgPqJAgGCk7_7 off=3678907 len=1398 2024-04-07T16:50:54.159 INFO:tasks.workunit.client.0.smithi100.stdout:op 3697 completed, throughput=4.99MB/sec 2024-04-07T16:50:54.159 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-XBYMiKBXMbZiwrP off=1486350 len=1907276 2024-04-07T16:50:54.174 INFO:tasks.workunit.client.0.smithi100.stdout:op 3698 completed, throughput=5MB/sec 2024-04-07T16:50:54.174 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-mgLeYWIFVBPL92b off=1210715 len=150916 2024-04-07T16:50:54.180 INFO:tasks.workunit.client.0.smithi100.stdout:op 3699 completed, throughput=5MB/sec 2024-04-07T16:50:54.180 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-MGGLlxCItXBTotn off=10108318 len=1791932 2024-04-07T16:50:54.192 INFO:tasks.workunit.client.0.smithi100.stdout:op 3700 completed, throughput=5MB/sec 2024-04-07T16:50:54.192 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-7vupFi0ZFluCH_O off=8723144 len=1308894 2024-04-07T16:50:54.203 INFO:tasks.workunit.client.0.smithi100.stdout:op 3701 completed, throughput=5MB/sec 2024-04-07T16:50:54.861 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:50:54 smithi144 bash[66654]: cluster 2024-04-07T16:50:53.450204+0000 mgr.y (mgr.25175) 1066 : cluster [DBG] pgmap v1708: 105 pgs: 105 active+clean; 126 GiB data, 8.2 GiB used, 707 GiB / 715 GiB avail; 2.5 MiB/s rd, 3.1 MiB/s wr, 5 op/s 2024-04-07T16:50:54.947 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:50:54 smithi100 bash[74942]: cluster 2024-04-07T16:50:53.450204+0000 mgr.y (mgr.25175) 1066 : cluster [DBG] pgmap v1708: 105 pgs: 105 active+clean; 126 GiB data, 8.2 GiB used, 707 GiB / 715 GiB avail; 2.5 MiB/s rd, 3.1 MiB/s wr, 5 op/s 2024-04-07T16:50:54.947 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:50:54 smithi100 bash[76202]: cluster 2024-04-07T16:50:53.450204+0000 mgr.y (mgr.25175) 1066 : cluster [DBG] pgmap v1708: 105 pgs: 105 active+clean; 126 GiB data, 8.2 GiB used, 707 GiB / 715 GiB avail; 2.5 MiB/s rd, 3.1 MiB/s wr, 5 op/s 2024-04-07T16:50:55.204 INFO:tasks.workunit.client.0.smithi100.stdout: 715: throughput=4.99MB/sec pending data=0 2024-04-07T16:50:55.204 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-F2w7vbQ3CBpX4Cy off=1648174 len=473887 2024-04-07T16:50:55.206 INFO:tasks.workunit.client.0.smithi100.stdout:op 3702 completed, throughput=4.99MB/sec 2024-04-07T16:50:55.206 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Mw1CYL_-8g--2PK off=16584545 len=638352 2024-04-07T16:50:55.208 INFO:tasks.workunit.client.0.smithi100.stdout:op 3703 completed, throughput=4.99MB/sec 2024-04-07T16:50:55.208 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-LkEt-2Kagxk15UC off=7982669 len=1732564 2024-04-07T16:50:55.212 INFO:tasks.workunit.client.0.smithi100.stdout:op 3704 completed, throughput=5MB/sec 2024-04-07T16:50:55.212 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-15f2eitP7XAyEJx off=15523633 len=2010670 2024-04-07T16:50:55.227 INFO:tasks.workunit.client.0.smithi100.stdout:op 3705 completed, throughput=5MB/sec 2024-04-07T16:50:55.227 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-_825c53Np9ZSucr off=671962 len=685443 2024-04-07T16:50:55.229 INFO:tasks.workunit.client.0.smithi100.stdout:op 3706 completed, throughput=5MB/sec 2024-04-07T16:50:56.229 INFO:tasks.workunit.client.0.smithi100.stdout: 716: throughput=4.99MB/sec pending data=0 2024-04-07T16:50:56.230 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-oEOO4MUrTxTEPk7 off=4826363 len=195736 2024-04-07T16:50:56.230 INFO:tasks.workunit.client.0.smithi100.stdout:op 3707 completed, throughput=4.99MB/sec 2024-04-07T16:50:56.231 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-5-E68aqM-tRpSwM off=21447804 len=1447946 2024-04-07T16:50:56.243 INFO:tasks.workunit.client.0.smithi100.stdout:op 3708 completed, throughput=5MB/sec 2024-04-07T16:50:56.243 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-x1aY4CtDWa1-6-H off=9131145 len=1161462 2024-04-07T16:50:56.251 INFO:tasks.workunit.client.0.smithi100.stdout:op 3709 completed, throughput=5MB/sec 2024-04-07T16:50:56.251 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-RTV9rqdVFvhu2sC off=1826115 len=1364727 2024-04-07T16:50:56.262 INFO:tasks.workunit.client.0.smithi100.stdout:op 3710 completed, throughput=5MB/sec 2024-04-07T16:50:56.262 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-z7WMQLJyNvRES6C off=8293404 len=899630 2024-04-07T16:50:56.265 INFO:tasks.workunit.client.0.smithi100.stdout:op 3711 completed, throughput=5MB/sec 2024-04-07T16:50:56.861 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:50:56 smithi144 bash[66654]: cluster 2024-04-07T16:50:55.451214+0000 mgr.y (mgr.25175) 1067 : cluster [DBG] pgmap v1709: 105 pgs: 105 active+clean; 126 GiB data, 8.2 GiB used, 707 GiB / 715 GiB avail; 1.9 MiB/s rd, 3.0 MiB/s wr, 4 op/s 2024-04-07T16:50:56.947 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:50:56 smithi100 bash[74942]: cluster 2024-04-07T16:50:55.451214+0000 mgr.y (mgr.25175) 1067 : cluster [DBG] pgmap v1709: 105 pgs: 105 active+clean; 126 GiB data, 8.2 GiB used, 707 GiB / 715 GiB avail; 1.9 MiB/s rd, 3.0 MiB/s wr, 4 op/s 2024-04-07T16:50:56.947 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:50:56 smithi100 bash[76202]: cluster 2024-04-07T16:50:55.451214+0000 mgr.y (mgr.25175) 1067 : cluster [DBG] pgmap v1709: 105 pgs: 105 active+clean; 126 GiB data, 8.2 GiB used, 707 GiB / 715 GiB avail; 1.9 MiB/s rd, 3.0 MiB/s wr, 4 op/s 2024-04-07T16:50:57.265 INFO:tasks.workunit.client.0.smithi100.stdout: 717: throughput=4.99MB/sec pending data=0 2024-04-07T16:50:57.265 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-PLJYpzLohzebAL8 off=6930767 len=1077843 2024-04-07T16:50:57.275 INFO:tasks.workunit.client.0.smithi100.stdout:op 3712 completed, throughput=4.99MB/sec 2024-04-07T16:50:57.275 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-rkQp4R6A8d9eqkc off=9480372 len=1472844 2024-04-07T16:50:57.287 INFO:tasks.workunit.client.0.smithi100.stdout:op 3713 completed, throughput=5MB/sec 2024-04-07T16:50:57.287 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Be_YZFJ75_-PpBd off=0 len=1397181 2024-04-07T16:50:57.289 INFO:tasks.workunit.client.0.smithi100.stdout:op 3714 completed, throughput=5MB/sec 2024-04-07T16:50:57.289 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-EirQjjOAfCq40Ei off=10740485 len=84702 2024-04-07T16:50:57.290 INFO:tasks.workunit.client.0.smithi100.stdout:op 3715 completed, throughput=5MB/sec 2024-04-07T16:50:57.290 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-JM1Wrd6nxXAKvuO off=318230 len=669963 2024-04-07T16:50:57.297 INFO:tasks.workunit.client.0.smithi100.stdout:op 3716 completed, throughput=5MB/sec 2024-04-07T16:50:57.297 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-2j66SvCu020iN4L off=1213432 len=208414 2024-04-07T16:50:57.302 INFO:tasks.workunit.client.0.smithi100.stdout:op 3717 completed, throughput=5MB/sec 2024-04-07T16:50:57.302 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-FaVuaGZ4jGe7eig off=17675083 len=864345 2024-04-07T16:50:57.309 INFO:tasks.workunit.client.0.smithi100.stdout:op 3718 completed, throughput=5MB/sec 2024-04-07T16:50:58.310 INFO:tasks.workunit.client.0.smithi100.stdout: 718: throughput=4.99MB/sec pending data=0 2024-04-07T16:50:58.310 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-_jN2-P08or1pQ5d off=5442157 len=752722 2024-04-07T16:50:58.319 INFO:tasks.workunit.client.0.smithi100.stdout:op 3719 completed, throughput=4.99MB/sec 2024-04-07T16:50:58.319 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-K1r5GdymASUHVPJ off=3726871 len=1693962 2024-04-07T16:50:58.332 INFO:tasks.workunit.client.0.smithi100.stdout:op 3720 completed, throughput=5MB/sec 2024-04-07T16:50:58.332 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-mlXypWhBO0bP3K- off=1560331 len=1973382 2024-04-07T16:50:58.338 INFO:tasks.workunit.client.0.smithi100.stdout:op 3721 completed, throughput=5MB/sec 2024-04-07T16:50:58.338 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-by6wHyheyEINGjg off=7318876 len=1168739 2024-04-07T16:50:58.342 INFO:tasks.workunit.client.0.smithi100.stdout:op 3722 completed, throughput=5MB/sec 2024-04-07T16:50:58.697 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:50:58 smithi100 bash[74942]: cluster 2024-04-07T16:50:57.452289+0000 mgr.y (mgr.25175) 1068 : cluster [DBG] pgmap v1710: 105 pgs: 105 active+clean; 126 GiB data, 8.2 GiB used, 707 GiB / 715 GiB avail; 1.9 MiB/s rd, 3.2 MiB/s wr, 4 op/s 2024-04-07T16:50:58.697 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:50:58 smithi100 bash[76202]: cluster 2024-04-07T16:50:57.452289+0000 mgr.y (mgr.25175) 1068 : cluster [DBG] pgmap v1710: 105 pgs: 105 active+clean; 126 GiB data, 8.2 GiB used, 707 GiB / 715 GiB avail; 1.9 MiB/s rd, 3.2 MiB/s wr, 4 op/s 2024-04-07T16:50:58.698 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:50:58 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:50:58] "GET /metrics HTTP/1.1" 200 34070 "" "Prometheus/2.43.0" 2024-04-07T16:50:58.861 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:50:58 smithi144 bash[66654]: cluster 2024-04-07T16:50:57.452289+0000 mgr.y (mgr.25175) 1068 : cluster [DBG] pgmap v1710: 105 pgs: 105 active+clean; 126 GiB data, 8.2 GiB used, 707 GiB / 715 GiB avail; 1.9 MiB/s rd, 3.2 MiB/s wr, 4 op/s 2024-04-07T16:50:59.342 INFO:tasks.workunit.client.0.smithi100.stdout: 719: throughput=4.99MB/sec pending data=0 2024-04-07T16:50:59.342 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-W79z6fGp3uba1Dz off=1774565 len=765355 2024-04-07T16:50:59.344 INFO:tasks.workunit.client.0.smithi100.stdout:op 3723 completed, throughput=4.99MB/sec 2024-04-07T16:50:59.344 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-TK8iRRLUS8Pj48V off=8880106 len=787048 2024-04-07T16:50:59.347 INFO:tasks.workunit.client.0.smithi100.stdout:op 3724 completed, throughput=5MB/sec 2024-04-07T16:50:59.347 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Yhp22FlHulqwvES off=13515578 len=88693 2024-04-07T16:50:59.352 INFO:tasks.workunit.client.0.smithi100.stdout:op 3725 completed, throughput=5MB/sec 2024-04-07T16:50:59.352 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-BQlxb-MZXBVfUMV off=1962864 len=1421183 2024-04-07T16:50:59.356 INFO:tasks.workunit.client.0.smithi100.stdout:op 3726 completed, throughput=5MB/sec 2024-04-07T16:50:59.356 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-gQS-xMZHZlCbEUP off=258815 len=1774741 2024-04-07T16:50:59.369 INFO:tasks.workunit.client.0.smithi100.stdout:op 3727 completed, throughput=5MB/sec 2024-04-07T16:50:59.370 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-jG6pHt-hiaBP6l9 off=24823907 len=160986 2024-04-07T16:50:59.374 INFO:tasks.workunit.client.0.smithi100.stdout:op 3728 completed, throughput=5MB/sec 2024-04-07T16:51:00.375 INFO:tasks.workunit.client.0.smithi100.stdout: 720: throughput=4.99MB/sec pending data=0 2024-04-07T16:51:00.375 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-WSgPNGvhEk0tzvT off=1015503 len=437878 2024-04-07T16:51:00.381 INFO:tasks.workunit.client.0.smithi100.stdout:op 3729 completed, throughput=4.99MB/sec 2024-04-07T16:51:00.381 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-R7ghHxlF0tKNiw- off=1557408 len=1853882 2024-04-07T16:51:00.396 INFO:tasks.workunit.client.0.smithi100.stdout:op 3730 completed, throughput=5MB/sec 2024-04-07T16:51:00.396 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-3v_Q08xiUH-ZxT7 off=14502865 len=653322 2024-04-07T16:51:00.398 INFO:tasks.workunit.client.0.smithi100.stdout:op 3731 completed, throughput=5MB/sec 2024-04-07T16:51:00.398 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-KaIMbTw-yd29Xfj off=19845650 len=1321158 2024-04-07T16:51:00.400 INFO:tasks.workunit.client.0.smithi100.stdout:op 3732 completed, throughput=5MB/sec 2024-04-07T16:51:00.400 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-tWs-mw1wUsGq2LT off=8963362 len=1192982 2024-04-07T16:51:00.402 INFO:tasks.workunit.client.0.smithi100.stdout:op 3733 completed, throughput=5MB/sec 2024-04-07T16:51:00.861 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:51:00 smithi144 bash[66654]: cluster 2024-04-07T16:50:59.453176+0000 mgr.y (mgr.25175) 1069 : cluster [DBG] pgmap v1711: 105 pgs: 105 active+clean; 126 GiB data, 8.3 GiB used, 707 GiB / 715 GiB avail; 1.7 MiB/s rd, 2.7 MiB/s wr, 3 op/s 2024-04-07T16:51:00.947 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:51:00 smithi100 bash[74942]: cluster 2024-04-07T16:50:59.453176+0000 mgr.y (mgr.25175) 1069 : cluster [DBG] pgmap v1711: 105 pgs: 105 active+clean; 126 GiB data, 8.3 GiB used, 707 GiB / 715 GiB avail; 1.7 MiB/s rd, 2.7 MiB/s wr, 3 op/s 2024-04-07T16:51:00.947 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:51:00 smithi100 bash[76202]: cluster 2024-04-07T16:50:59.453176+0000 mgr.y (mgr.25175) 1069 : cluster [DBG] pgmap v1711: 105 pgs: 105 active+clean; 126 GiB data, 8.3 GiB used, 707 GiB / 715 GiB avail; 1.7 MiB/s rd, 2.7 MiB/s wr, 3 op/s 2024-04-07T16:51:01.403 INFO:tasks.workunit.client.0.smithi100.stdout: 721: throughput=4.99MB/sec pending data=0 2024-04-07T16:51:01.403 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-wr45Rdvh1I_qyu9 off=6230004 len=1468715 2024-04-07T16:51:01.414 INFO:tasks.workunit.client.0.smithi100.stdout:op 3734 completed, throughput=5MB/sec 2024-04-07T16:51:01.414 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-ZoLJ-DGMyYBMWSN off=3449902 len=1974945 2024-04-07T16:51:01.420 INFO:tasks.workunit.client.0.smithi100.stdout:op 3735 completed, throughput=5MB/sec 2024-04-07T16:51:01.420 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Kvi0c6mz6FaS4C3 off=2786497 len=1911414 2024-04-07T16:51:01.427 INFO:tasks.workunit.client.0.smithi100.stdout:op 3736 completed, throughput=5MB/sec 2024-04-07T16:51:02.428 INFO:tasks.workunit.client.0.smithi100.stdout: 722: throughput=4.99MB/sec pending data=0 2024-04-07T16:51:02.428 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-u7ujExEUq7y1tTK off=6062419 len=1273450 2024-04-07T16:51:02.440 INFO:tasks.workunit.client.0.smithi100.stdout:op 3737 completed, throughput=5MB/sec 2024-04-07T16:51:02.440 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-ahkwDdzHYzfPgsQ off=3738770 len=371900 2024-04-07T16:51:02.446 INFO:tasks.workunit.client.0.smithi100.stdout:op 3738 completed, throughput=5MB/sec 2024-04-07T16:51:02.447 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-8ix1Nog73vn1mY0 off=10530413 len=1141981 2024-04-07T16:51:02.450 INFO:tasks.workunit.client.0.smithi100.stdout:op 3739 completed, throughput=5MB/sec 2024-04-07T16:51:02.450 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-WNZeKghf1eghKuh off=14991185 len=141999 2024-04-07T16:51:02.456 INFO:tasks.workunit.client.0.smithi100.stdout:op 3740 completed, throughput=5MB/sec 2024-04-07T16:51:02.456 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-DmV61-E51RhD5e1 off=1007908 len=571908 2024-04-07T16:51:02.462 INFO:tasks.workunit.client.0.smithi100.stdout:op 3741 completed, throughput=5MB/sec 2024-04-07T16:51:02.462 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-IEKMmI_l7rY30tC off=6624031 len=1446664 2024-04-07T16:51:02.474 INFO:tasks.workunit.client.0.smithi100.stdout:op 3742 completed, throughput=5MB/sec 2024-04-07T16:51:02.474 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-39eC1TPtgLxtkvY off=8360915 len=1085447 2024-04-07T16:51:02.479 INFO:tasks.workunit.client.0.smithi100.stdout:op 3743 completed, throughput=5MB/sec 2024-04-07T16:51:02.575 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:51:02 smithi144 bash[95233]: ts=2024-04-07T16:51:02.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:51:02.861 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:51:02 smithi144 bash[66654]: cluster 2024-04-07T16:51:01.454135+0000 mgr.y (mgr.25175) 1070 : cluster [DBG] pgmap v1712: 105 pgs: 105 active+clean; 126 GiB data, 8.3 GiB used, 707 GiB / 715 GiB avail; 1.9 MiB/s rd, 3.1 MiB/s wr, 4 op/s 2024-04-07T16:51:02.861 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:51:02 smithi144 bash[66654]: audit 2024-04-07T16:51:01.776999+0000 mon.a (mon.0) 2150 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:51:02.947 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:51:02 smithi100 bash[74942]: cluster 2024-04-07T16:51:01.454135+0000 mgr.y (mgr.25175) 1070 : cluster [DBG] pgmap v1712: 105 pgs: 105 active+clean; 126 GiB data, 8.3 GiB used, 707 GiB / 715 GiB avail; 1.9 MiB/s rd, 3.1 MiB/s wr, 4 op/s 2024-04-07T16:51:02.947 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:51:02 smithi100 bash[74942]: audit 2024-04-07T16:51:01.776999+0000 mon.a (mon.0) 2150 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:51:02.947 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:51:02 smithi100 bash[76202]: cluster 2024-04-07T16:51:01.454135+0000 mgr.y (mgr.25175) 1070 : cluster [DBG] pgmap v1712: 105 pgs: 105 active+clean; 126 GiB data, 8.3 GiB used, 707 GiB / 715 GiB avail; 1.9 MiB/s rd, 3.1 MiB/s wr, 4 op/s 2024-04-07T16:51:02.948 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:51:02 smithi100 bash[76202]: audit 2024-04-07T16:51:01.776999+0000 mon.a (mon.0) 2150 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:51:03.480 INFO:tasks.workunit.client.0.smithi100.stdout: 723: throughput=4.99MB/sec pending data=0 2024-04-07T16:51:03.480 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-2W2DqFb573bMdTB off=15123711 len=744822 2024-04-07T16:51:03.483 INFO:tasks.workunit.client.0.smithi100.stdout:op 3744 completed, throughput=5MB/sec 2024-04-07T16:51:03.483 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-ZcGA9UTSl2Tq24F off=7971002 len=18860 2024-04-07T16:51:03.487 INFO:tasks.workunit.client.0.smithi100.stdout:op 3745 completed, throughput=5MB/sec 2024-04-07T16:51:03.487 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-9QRyp-Al3aR5IIB off=4809829 len=1902608 2024-04-07T16:51:03.505 INFO:tasks.workunit.client.0.smithi100.stdout:op 3746 completed, throughput=5MB/sec 2024-04-07T16:51:03.505 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-IYg1NPZ6VT-JWzT off=696342 len=844911 2024-04-07T16:51:03.514 INFO:tasks.workunit.client.0.smithi100.stdout:op 3747 completed, throughput=5MB/sec 2024-04-07T16:51:03.514 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-vwPFpeGuxgb6zt- off=390231 len=1583467 2024-04-07T16:51:03.518 INFO:tasks.workunit.client.0.smithi100.stdout:op 3748 completed, throughput=5MB/sec 2024-04-07T16:51:04.518 INFO:tasks.workunit.client.0.smithi100.stdout: 724: throughput=4.99MB/sec pending data=0 2024-04-07T16:51:04.518 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-QMq-d2VN2VZIU1P off=12678525 len=1999220 2024-04-07T16:51:04.524 INFO:tasks.workunit.client.0.smithi100.stdout:op 3749 completed, throughput=5MB/sec 2024-04-07T16:51:04.524 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-O9SPbW2F45iHI7y off=11363695 len=603300 2024-04-07T16:51:04.530 INFO:tasks.workunit.client.0.smithi100.stdout:op 3750 completed, throughput=5MB/sec 2024-04-07T16:51:04.530 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-SDHzCZWGwMIY-ZC off=2412706 len=1044309 2024-04-07T16:51:04.537 INFO:tasks.workunit.client.0.smithi100.stdout:op 3751 completed, throughput=5MB/sec 2024-04-07T16:51:04.537 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-GKxeEb-kDkCVaMu off=470760 len=2057136 2024-04-07T16:51:04.543 INFO:tasks.workunit.client.0.smithi100.stdout:op 3752 completed, throughput=5MB/sec 2024-04-07T16:51:04.861 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:51:04 smithi144 bash[66654]: cluster 2024-04-07T16:51:03.455243+0000 mgr.y (mgr.25175) 1071 : cluster [DBG] pgmap v1713: 105 pgs: 105 active+clean; 126 GiB data, 8.3 GiB used, 707 GiB / 715 GiB avail; 2.4 MiB/s rd, 3.1 MiB/s wr, 5 op/s 2024-04-07T16:51:04.861 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:51:04 smithi144 bash[66654]: audit 2024-04-07T16:51:04.426590+0000 mon.a (mon.0) 2151 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:51:04.947 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:51:04 smithi100 bash[76202]: cluster 2024-04-07T16:51:03.455243+0000 mgr.y (mgr.25175) 1071 : cluster [DBG] pgmap v1713: 105 pgs: 105 active+clean; 126 GiB data, 8.3 GiB used, 707 GiB / 715 GiB avail; 2.4 MiB/s rd, 3.1 MiB/s wr, 5 op/s 2024-04-07T16:51:04.947 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:51:04 smithi100 bash[76202]: audit 2024-04-07T16:51:04.426590+0000 mon.a (mon.0) 2151 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:51:04.947 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:51:04 smithi100 bash[74942]: cluster 2024-04-07T16:51:03.455243+0000 mgr.y (mgr.25175) 1071 : cluster [DBG] pgmap v1713: 105 pgs: 105 active+clean; 126 GiB data, 8.3 GiB used, 707 GiB / 715 GiB avail; 2.4 MiB/s rd, 3.1 MiB/s wr, 5 op/s 2024-04-07T16:51:04.947 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:51:04 smithi100 bash[74942]: audit 2024-04-07T16:51:04.426590+0000 mon.a (mon.0) 2151 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:51:05.543 INFO:tasks.workunit.client.0.smithi100.stdout: 725: throughput=4.99MB/sec pending data=0 2024-04-07T16:51:05.543 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-UKQityXvcBhkDD6 off=8886021 len=1840201 2024-04-07T16:51:05.549 INFO:tasks.workunit.client.0.smithi100.stdout:op 3753 completed, throughput=5MB/sec 2024-04-07T16:51:05.549 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-BiE6JoZa8Me9oJn off=5257379 len=1805740 2024-04-07T16:51:05.564 INFO:tasks.workunit.client.0.smithi100.stdout:op 3754 completed, throughput=5MB/sec 2024-04-07T16:51:05.565 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-ItRsIGY7x44TayH off=1301649 len=941984 2024-04-07T16:51:05.573 INFO:tasks.workunit.client.0.smithi100.stdout:op 3755 completed, throughput=5MB/sec 2024-04-07T16:51:05.573 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-8Tb75v22997BMKb off=11347330 len=867555 2024-04-07T16:51:05.580 INFO:tasks.workunit.client.0.smithi100.stdout:op 3756 completed, throughput=5MB/sec 2024-04-07T16:51:05.861 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:51:05 smithi144 bash[66654]: audit 2024-04-07T16:51:05.171327+0000 mon.a (mon.0) 2152 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:51:05.861 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:51:05 smithi144 bash[66654]: audit 2024-04-07T16:51:05.172945+0000 mon.a (mon.0) 2153 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:51:05.862 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:51:05 smithi144 bash[66654]: audit 2024-04-07T16:51:05.182496+0000 mon.a (mon.0) 2154 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:51:05.947 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:51:05 smithi100 bash[74942]: audit 2024-04-07T16:51:05.171327+0000 mon.a (mon.0) 2152 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:51:05.947 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:51:05 smithi100 bash[74942]: audit 2024-04-07T16:51:05.172945+0000 mon.a (mon.0) 2153 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:51:05.947 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:51:05 smithi100 bash[74942]: audit 2024-04-07T16:51:05.182496+0000 mon.a (mon.0) 2154 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:51:05.948 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:51:05 smithi100 bash[76202]: audit 2024-04-07T16:51:05.171327+0000 mon.a (mon.0) 2152 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:51:05.948 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:51:05 smithi100 bash[76202]: audit 2024-04-07T16:51:05.172945+0000 mon.a (mon.0) 2153 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:51:05.948 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:51:05 smithi100 bash[76202]: audit 2024-04-07T16:51:05.182496+0000 mon.a (mon.0) 2154 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:51:06.581 INFO:tasks.workunit.client.0.smithi100.stdout: 726: throughput=4.99MB/sec pending data=0 2024-04-07T16:51:06.581 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-U8RxJZLWZVfypWP off=1606602 len=133334 2024-04-07T16:51:06.586 INFO:tasks.workunit.client.0.smithi100.stdout:op 3757 completed, throughput=4.99MB/sec 2024-04-07T16:51:06.586 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-6Cj-kp2NdkOcn9d off=7361286 len=1562920 2024-04-07T16:51:06.599 INFO:tasks.workunit.client.0.smithi100.stdout:op 3758 completed, throughput=5MB/sec 2024-04-07T16:51:06.599 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-lxhfJjgcJCNjjAz off=15763714 len=1522780 2024-04-07T16:51:06.611 INFO:tasks.workunit.client.0.smithi100.stdout:op 3759 completed, throughput=5MB/sec 2024-04-07T16:51:06.611 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-h6UGEzpLBZ5j4Iu off=10592341 len=2033499 2024-04-07T16:51:06.616 INFO:tasks.workunit.client.0.smithi100.stdout:op 3760 completed, throughput=5MB/sec 2024-04-07T16:51:06.861 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:51:06 smithi144 bash[66654]: cluster 2024-04-07T16:51:05.456225+0000 mgr.y (mgr.25175) 1072 : cluster [DBG] pgmap v1714: 105 pgs: 105 active+clean; 126 GiB data, 8.3 GiB used, 707 GiB / 715 GiB avail; 1.8 MiB/s rd, 3.0 MiB/s wr, 4 op/s 2024-04-07T16:51:06.947 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:51:06 smithi100 bash[74942]: cluster 2024-04-07T16:51:05.456225+0000 mgr.y (mgr.25175) 1072 : cluster [DBG] pgmap v1714: 105 pgs: 105 active+clean; 126 GiB data, 8.3 GiB used, 707 GiB / 715 GiB avail; 1.8 MiB/s rd, 3.0 MiB/s wr, 4 op/s 2024-04-07T16:51:06.947 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:51:06 smithi100 bash[76202]: cluster 2024-04-07T16:51:05.456225+0000 mgr.y (mgr.25175) 1072 : cluster [DBG] pgmap v1714: 105 pgs: 105 active+clean; 126 GiB data, 8.3 GiB used, 707 GiB / 715 GiB avail; 1.8 MiB/s rd, 3.0 MiB/s wr, 4 op/s 2024-04-07T16:51:07.616 INFO:tasks.workunit.client.0.smithi100.stdout: 727: throughput=4.99MB/sec pending data=0 2024-04-07T16:51:07.616 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-B2OLPkeANZl_ZVH off=11141484 len=1874549 2024-04-07T16:51:07.632 INFO:tasks.workunit.client.0.smithi100.stdout:op 3761 completed, throughput=5MB/sec 2024-04-07T16:51:07.632 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-1aVt1OrXvcZvDaF off=5902818 len=1355024 2024-04-07T16:51:07.643 INFO:tasks.workunit.client.0.smithi100.stdout:op 3762 completed, throughput=5MB/sec 2024-04-07T16:51:07.643 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-cAH5XTR3VU2Kif1 off=5591362 len=1777210 2024-04-07T16:51:07.647 INFO:tasks.workunit.client.0.smithi100.stdout:op 3763 completed, throughput=5MB/sec 2024-04-07T16:51:08.648 INFO:tasks.workunit.client.0.smithi100.stdout: 728: throughput=4.99MB/sec pending data=0 2024-04-07T16:51:08.648 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-PNzWX9Gh-ENXe1A off=3700119 len=1392417 2024-04-07T16:51:08.652 INFO:tasks.workunit.client.0.smithi100.stdout:op 3764 completed, throughput=5MB/sec 2024-04-07T16:51:08.652 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Pk8J4ppBJlG__Pk off=15102882 len=1156201 2024-04-07T16:51:08.654 INFO:tasks.workunit.client.0.smithi100.stdout:op 3765 completed, throughput=5MB/sec 2024-04-07T16:51:08.654 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-oKYZuCVsCqm9dTs off=13885029 len=1713196 2024-04-07T16:51:08.661 INFO:tasks.workunit.client.0.smithi100.stdout:op 3766 completed, throughput=5MB/sec 2024-04-07T16:51:08.661 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-y16vMf_UlJgBbbH off=2177537 len=1761686 2024-04-07T16:51:08.675 INFO:tasks.workunit.client.0.smithi100.stdout:op 3767 completed, throughput=5MB/sec 2024-04-07T16:51:08.697 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:51:08 smithi100 bash[74942]: cluster 2024-04-07T16:51:07.457251+0000 mgr.y (mgr.25175) 1073 : cluster [DBG] pgmap v1715: 105 pgs: 105 active+clean; 126 GiB data, 8.3 GiB used, 707 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.8 MiB/s wr, 5 op/s 2024-04-07T16:51:08.697 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:51:08 smithi100 bash[76202]: cluster 2024-04-07T16:51:07.457251+0000 mgr.y (mgr.25175) 1073 : cluster [DBG] pgmap v1715: 105 pgs: 105 active+clean; 126 GiB data, 8.3 GiB used, 707 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.8 MiB/s wr, 5 op/s 2024-04-07T16:51:08.697 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:51:08 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:51:08] "GET /metrics HTTP/1.1" 200 34072 "" "Prometheus/2.43.0" 2024-04-07T16:51:08.861 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:51:08 smithi144 bash[66654]: cluster 2024-04-07T16:51:07.457251+0000 mgr.y (mgr.25175) 1073 : cluster [DBG] pgmap v1715: 105 pgs: 105 active+clean; 126 GiB data, 8.3 GiB used, 707 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.8 MiB/s wr, 5 op/s 2024-04-07T16:51:09.675 INFO:tasks.workunit.client.0.smithi100.stdout: 729: throughput=4.99MB/sec pending data=0 2024-04-07T16:51:09.675 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-iTv_NCBzEDXXjId off=331643 len=1941832 2024-04-07T16:51:09.693 INFO:tasks.workunit.client.0.smithi100.stdout:op 3768 completed, throughput=5MB/sec 2024-04-07T16:51:09.693 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-BstntSwbiqDaBBd off=3513138 len=186468 2024-04-07T16:51:09.698 INFO:tasks.workunit.client.0.smithi100.stdout:op 3769 completed, throughput=5MB/sec 2024-04-07T16:51:09.698 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-eI8XGT_FY3bRKWD off=11511747 len=1398039 2024-04-07T16:51:09.702 INFO:tasks.workunit.client.0.smithi100.stdout:op 3770 completed, throughput=5MB/sec 2024-04-07T16:51:09.702 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-CAHz-N3nRUxfmRQ off=14389708 len=16068 2024-04-07T16:51:09.706 INFO:tasks.workunit.client.0.smithi100.stdout:op 3771 completed, throughput=5MB/sec 2024-04-07T16:51:09.706 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj--dAVb3ZlMufuCS7 off=16500378 len=1748169 2024-04-07T16:51:09.720 INFO:tasks.workunit.client.0.smithi100.stdout:op 3772 completed, throughput=5MB/sec 2024-04-07T16:51:10.721 INFO:tasks.workunit.client.0.smithi100.stdout: 730: throughput=4.99MB/sec pending data=0 2024-04-07T16:51:10.721 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-QhOwmcDz33Xo8Ut off=2094437 len=1538811 2024-04-07T16:51:10.734 INFO:tasks.workunit.client.0.smithi100.stdout:op 3773 completed, throughput=5MB/sec 2024-04-07T16:51:10.734 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-b89XAal0xvE5QOD off=3998848 len=1024000 2024-04-07T16:51:10.743 INFO:tasks.workunit.client.0.smithi100.stdout:op 3774 completed, throughput=5MB/sec 2024-04-07T16:51:10.743 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-4I1GA_PNuKXLY18 off=13414938 len=969772 2024-04-07T16:51:10.750 INFO:tasks.workunit.client.0.smithi100.stdout:op 3775 completed, throughput=5MB/sec 2024-04-07T16:51:10.750 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-KRNOH7QS4CaQm8r off=1596352 len=1608313 2024-04-07T16:51:10.762 INFO:tasks.workunit.client.0.smithi100.stdout:op 3776 completed, throughput=5MB/sec 2024-04-07T16:51:10.861 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:51:10 smithi144 bash[66654]: cluster 2024-04-07T16:51:09.458202+0000 mgr.y (mgr.25175) 1074 : cluster [DBG] pgmap v1716: 105 pgs: 105 active+clean; 126 GiB data, 8.3 GiB used, 707 GiB / 715 GiB avail; 2.4 MiB/s rd, 2.7 MiB/s wr, 4 op/s 2024-04-07T16:51:10.947 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:51:10 smithi100 bash[74942]: cluster 2024-04-07T16:51:09.458202+0000 mgr.y (mgr.25175) 1074 : cluster [DBG] pgmap v1716: 105 pgs: 105 active+clean; 126 GiB data, 8.3 GiB used, 707 GiB / 715 GiB avail; 2.4 MiB/s rd, 2.7 MiB/s wr, 4 op/s 2024-04-07T16:51:10.947 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:51:10 smithi100 bash[76202]: cluster 2024-04-07T16:51:09.458202+0000 mgr.y (mgr.25175) 1074 : cluster [DBG] pgmap v1716: 105 pgs: 105 active+clean; 126 GiB data, 8.3 GiB used, 707 GiB / 715 GiB avail; 2.4 MiB/s rd, 2.7 MiB/s wr, 4 op/s 2024-04-07T16:51:11.762 INFO:tasks.workunit.client.0.smithi100.stdout: 731: throughput=4.99MB/sec pending data=0 2024-04-07T16:51:11.763 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-K82qOOVsdDqfYHx off=3807507 len=1245760 2024-04-07T16:51:11.766 INFO:tasks.workunit.client.0.smithi100.stdout:op 3777 completed, throughput=5MB/sec 2024-04-07T16:51:11.766 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-1UaYI_n3yneNgru off=10660707 len=1131887 2024-04-07T16:51:11.769 INFO:tasks.workunit.client.0.smithi100.stdout:op 3778 completed, throughput=5MB/sec 2024-04-07T16:51:11.770 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-3hHJO8uwYamrUAw off=547475 len=1723345 2024-04-07T16:51:11.774 INFO:tasks.workunit.client.0.smithi100.stdout:op 3779 completed, throughput=5MB/sec 2024-04-07T16:51:11.774 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-0Y4al1WXhqCLc8W off=6029962 len=889228 2024-04-07T16:51:11.776 INFO:tasks.workunit.client.0.smithi100.stdout:op 3780 completed, throughput=5MB/sec 2024-04-07T16:51:11.776 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-o3NmGTNGOKsO0Kn off=0 len=1205532 2024-04-07T16:51:11.780 INFO:tasks.workunit.client.0.smithi100.stdout:op 3781 completed, throughput=5MB/sec 2024-04-07T16:51:12.596 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:51:12 smithi144 bash[95233]: ts=2024-04-07T16:51:12.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:51:12.780 INFO:tasks.workunit.client.0.smithi100.stdout: 732: throughput=4.99MB/sec pending data=0 2024-04-07T16:51:12.780 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-bOdSY0L67MvTCJa off=2146463 len=1574759 2024-04-07T16:51:12.784 INFO:tasks.workunit.client.0.smithi100.stdout:op 3782 completed, throughput=5MB/sec 2024-04-07T16:51:12.784 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-zj4oQwU0ru__gAb off=3201088 len=689226 2024-04-07T16:51:12.791 INFO:tasks.workunit.client.0.smithi100.stdout:op 3783 completed, throughput=5MB/sec 2024-04-07T16:51:12.791 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-YDH115OH5PDjHtI off=598362 len=2058386 2024-04-07T16:51:12.796 INFO:tasks.workunit.client.0.smithi100.stdout:op 3784 completed, throughput=5MB/sec 2024-04-07T16:51:12.861 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:51:12 smithi144 bash[66654]: cluster 2024-04-07T16:51:11.458957+0000 mgr.y (mgr.25175) 1075 : cluster [DBG] pgmap v1717: 105 pgs: 105 active+clean; 126 GiB data, 8.3 GiB used, 707 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.6 MiB/s wr, 4 op/s 2024-04-07T16:51:12.947 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:51:12 smithi100 bash[74942]: cluster 2024-04-07T16:51:11.458957+0000 mgr.y (mgr.25175) 1075 : cluster [DBG] pgmap v1717: 105 pgs: 105 active+clean; 126 GiB data, 8.3 GiB used, 707 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.6 MiB/s wr, 4 op/s 2024-04-07T16:51:12.947 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:51:12 smithi100 bash[76202]: cluster 2024-04-07T16:51:11.458957+0000 mgr.y (mgr.25175) 1075 : cluster [DBG] pgmap v1717: 105 pgs: 105 active+clean; 126 GiB data, 8.3 GiB used, 707 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.6 MiB/s wr, 4 op/s 2024-04-07T16:51:13.796 INFO:tasks.workunit.client.0.smithi100.stdout: 733: throughput=4.99MB/sec pending data=0 2024-04-07T16:51:13.797 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-A6qUvfPbmLhLT00 off=3711477 len=1749832 2024-04-07T16:51:13.801 INFO:tasks.workunit.client.0.smithi100.stdout:op 3785 completed, throughput=5MB/sec 2024-04-07T16:51:13.801 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-u_LOP3kvX03__Nb off=11319872 len=166850 2024-04-07T16:51:13.802 INFO:tasks.workunit.client.0.smithi100.stdout:op 3786 completed, throughput=5MB/sec 2024-04-07T16:51:13.803 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-0iM5Lax3t2CESAF off=5961267 len=1095496 2024-04-07T16:51:13.813 INFO:tasks.workunit.client.0.smithi100.stdout:op 3787 completed, throughput=5MB/sec 2024-04-07T16:51:13.813 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-JSqSf9VACVfYUAp off=7152915 len=1264316 2024-04-07T16:51:13.817 INFO:tasks.workunit.client.0.smithi100.stdout:op 3788 completed, throughput=5MB/sec 2024-04-07T16:51:13.817 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-k-VfS9u9R4ystRw off=16580521 len=2075455 2024-04-07T16:51:13.833 INFO:tasks.workunit.client.0.smithi100.stdout:op 3789 completed, throughput=5MB/sec 2024-04-07T16:51:14.833 INFO:tasks.workunit.client.0.smithi100.stdout: 734: throughput=4.99MB/sec pending data=0 2024-04-07T16:51:14.834 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-GlXApdflrNcCwbY off=6780899 len=1470718 2024-04-07T16:51:14.845 INFO:tasks.workunit.client.0.smithi100.stdout:op 3790 completed, throughput=5MB/sec 2024-04-07T16:51:14.846 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-4IU17cVHADaw1V3 off=11569968 len=1062757 2024-04-07T16:51:14.850 INFO:tasks.workunit.client.0.smithi100.stdout:op 3791 completed, throughput=5MB/sec 2024-04-07T16:51:14.851 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-UMNlVq1PL3TO22E off=17587975 len=16863 2024-04-07T16:51:14.855 INFO:tasks.workunit.client.0.smithi100.stdout:op 3792 completed, throughput=5MB/sec 2024-04-07T16:51:14.855 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-u-LtEgh7MchDpKe off=7585988 len=923177 2024-04-07T16:51:14.858 INFO:tasks.workunit.client.0.smithi100.stdout:op 3793 completed, throughput=5MB/sec 2024-04-07T16:51:14.858 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-fdIU_g7W0zrG6BB off=13976802 len=1713697 2024-04-07T16:51:14.861 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:51:14 smithi144 bash[66654]: cluster 2024-04-07T16:51:13.460082+0000 mgr.y (mgr.25175) 1076 : cluster [DBG] pgmap v1718: 105 pgs: 105 active+clean; 126 GiB data, 8.4 GiB used, 707 GiB / 715 GiB avail; 2.9 MiB/s rd, 2.6 MiB/s wr, 4 op/s 2024-04-07T16:51:14.862 INFO:tasks.workunit.client.0.smithi100.stdout:op 3794 completed, throughput=5MB/sec 2024-04-07T16:51:14.947 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:51:14 smithi100 bash[74942]: cluster 2024-04-07T16:51:13.460082+0000 mgr.y (mgr.25175) 1076 : cluster [DBG] pgmap v1718: 105 pgs: 105 active+clean; 126 GiB data, 8.4 GiB used, 707 GiB / 715 GiB avail; 2.9 MiB/s rd, 2.6 MiB/s wr, 4 op/s 2024-04-07T16:51:14.947 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:51:14 smithi100 bash[76202]: cluster 2024-04-07T16:51:13.460082+0000 mgr.y (mgr.25175) 1076 : cluster [DBG] pgmap v1718: 105 pgs: 105 active+clean; 126 GiB data, 8.4 GiB used, 707 GiB / 715 GiB avail; 2.9 MiB/s rd, 2.6 MiB/s wr, 4 op/s 2024-04-07T16:51:15.863 INFO:tasks.workunit.client.0.smithi100.stdout: 735: throughput=4.99MB/sec pending data=0 2024-04-07T16:51:15.863 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-lJltFcOYgOnd7nL off=21914932 len=951583 2024-04-07T16:51:15.866 INFO:tasks.workunit.client.0.smithi100.stdout:op 3795 completed, throughput=5MB/sec 2024-04-07T16:51:15.866 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Zp9KOgZ9hUDohhL off=12196037 len=1252919 2024-04-07T16:51:15.876 INFO:tasks.workunit.client.0.smithi100.stdout:op 3796 completed, throughput=5MB/sec 2024-04-07T16:51:15.876 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-hTT_iSIkAEZlwWi off=5197740 len=527284 2024-04-07T16:51:15.883 INFO:tasks.workunit.client.0.smithi100.stdout:op 3797 completed, throughput=5MB/sec 2024-04-07T16:51:15.883 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-pD6_QgzX47TghaW off=2251372 len=904972 2024-04-07T16:51:15.893 INFO:tasks.workunit.client.0.smithi100.stdout:op 3798 completed, throughput=5MB/sec 2024-04-07T16:51:15.893 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-q7IBOrklz5gLjXE off=491030 len=1237700 2024-04-07T16:51:15.896 INFO:tasks.workunit.client.0.smithi100.stdout:op 3799 completed, throughput=5MB/sec 2024-04-07T16:51:16.861 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:51:16 smithi144 bash[66654]: cluster 2024-04-07T16:51:15.460985+0000 mgr.y (mgr.25175) 1077 : cluster [DBG] pgmap v1719: 105 pgs: 105 active+clean; 126 GiB data, 8.4 GiB used, 707 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.5 MiB/s wr, 4 op/s 2024-04-07T16:51:16.896 INFO:tasks.workunit.client.0.smithi100.stdout: 736: throughput=4.99MB/sec pending data=0 2024-04-07T16:51:16.896 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-M5K3hXeWwlKk3cD off=7076223 len=1181223 2024-04-07T16:51:16.901 INFO:tasks.workunit.client.0.smithi100.stdout:op 3800 completed, throughput=5MB/sec 2024-04-07T16:51:16.902 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-4X_-zwrGsb6b1uA off=11872697 len=748276 2024-04-07T16:51:16.909 INFO:tasks.workunit.client.0.smithi100.stdout:op 3801 completed, throughput=5MB/sec 2024-04-07T16:51:16.909 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-4zYfEAzU5e2flD5 off=1519822 len=1372361 2024-04-07T16:51:16.920 INFO:tasks.workunit.client.0.smithi100.stdout:op 3802 completed, throughput=5MB/sec 2024-04-07T16:51:16.920 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-IEKMmI_l7rY30tC off=8839127 len=206767 2024-04-07T16:51:16.926 INFO:tasks.workunit.client.0.smithi100.stdout:op 3803 completed, throughput=5MB/sec 2024-04-07T16:51:16.926 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Xv0Sbkyk5ZTNXgW off=615317 len=1993116 2024-04-07T16:51:16.940 INFO:tasks.workunit.client.0.smithi100.stdout:op 3804 completed, throughput=5MB/sec 2024-04-07T16:51:16.947 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:51:16 smithi100 bash[74942]: cluster 2024-04-07T16:51:15.460985+0000 mgr.y (mgr.25175) 1077 : cluster [DBG] pgmap v1719: 105 pgs: 105 active+clean; 126 GiB data, 8.4 GiB used, 707 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.5 MiB/s wr, 4 op/s 2024-04-07T16:51:16.947 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:51:16 smithi100 bash[76202]: cluster 2024-04-07T16:51:15.460985+0000 mgr.y (mgr.25175) 1077 : cluster [DBG] pgmap v1719: 105 pgs: 105 active+clean; 126 GiB data, 8.4 GiB used, 707 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.5 MiB/s wr, 4 op/s 2024-04-07T16:51:17.941 INFO:tasks.workunit.client.0.smithi100.stdout: 737: throughput=4.99MB/sec pending data=0 2024-04-07T16:51:17.941 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-_4Q27KZdj8QKWZQ off=15765645 len=13497 2024-04-07T16:51:17.946 INFO:tasks.workunit.client.0.smithi100.stdout:op 3805 completed, throughput=4.99MB/sec 2024-04-07T16:51:17.946 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-M2cNcD24NHIkhmu off=7815858 len=690006 2024-04-07T16:51:17.947 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:51:17 smithi100 bash[74942]: audit 2024-04-07T16:51:16.777622+0000 mon.a (mon.0) 2155 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:51:17.947 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:51:17 smithi100 bash[76202]: audit 2024-04-07T16:51:16.777622+0000 mon.a (mon.0) 2155 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:51:17.948 INFO:tasks.workunit.client.0.smithi100.stdout:op 3806 completed, throughput=4.99MB/sec 2024-04-07T16:51:17.948 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-4d80HOz-PMgzP68 off=3954043 len=82972 2024-04-07T16:51:17.952 INFO:tasks.workunit.client.0.smithi100.stdout:op 3807 completed, throughput=4.99MB/sec 2024-04-07T16:51:17.953 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-b6rfdoPRTIGEwlU off=5498296 len=1458214 2024-04-07T16:51:17.965 INFO:tasks.workunit.client.0.smithi100.stdout:op 3808 completed, throughput=5MB/sec 2024-04-07T16:51:17.965 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Re3X4izVOpsUE1d off=7563577 len=108522 2024-04-07T16:51:17.970 INFO:tasks.workunit.client.0.smithi100.stdout:op 3809 completed, throughput=5MB/sec 2024-04-07T16:51:17.970 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-flwhYxWjxD_bLvI off=2812697 len=1257684 2024-04-07T16:51:17.973 INFO:tasks.workunit.client.0.smithi100.stdout:op 3810 completed, throughput=5MB/sec 2024-04-07T16:51:17.973 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-WfhzmJNWwVKnwIh off=18571034 len=435233 2024-04-07T16:51:17.974 INFO:tasks.workunit.client.0.smithi100.stdout:op 3811 completed, throughput=5MB/sec 2024-04-07T16:51:17.974 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-dIYCU-o3p1Vx4ja off=10072087 len=441169 2024-04-07T16:51:17.980 INFO:tasks.workunit.client.0.smithi100.stdout:op 3812 completed, throughput=5MB/sec 2024-04-07T16:51:17.980 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-OvEBlgzJUCTUHYX off=9990877 len=1723678 2024-04-07T16:51:17.985 INFO:tasks.workunit.client.0.smithi100.stdout:op 3813 completed, throughput=5MB/sec 2024-04-07T16:51:18.111 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:51:17 smithi144 bash[66654]: audit 2024-04-07T16:51:16.777622+0000 mon.a (mon.0) 2155 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:51:18.697 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:51:18 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:51:18] "GET /metrics HTTP/1.1" 200 34069 "" "Prometheus/2.43.0" 2024-04-07T16:51:18.698 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:51:18 smithi100 bash[74942]: cluster 2024-04-07T16:51:17.462027+0000 mgr.y (mgr.25175) 1078 : cluster [DBG] pgmap v1720: 105 pgs: 105 active+clean; 126 GiB data, 8.4 GiB used, 707 GiB / 715 GiB avail; 2.9 MiB/s rd, 2.6 MiB/s wr, 4 op/s 2024-04-07T16:51:18.698 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:51:18 smithi100 bash[76202]: cluster 2024-04-07T16:51:17.462027+0000 mgr.y (mgr.25175) 1078 : cluster [DBG] pgmap v1720: 105 pgs: 105 active+clean; 126 GiB data, 8.4 GiB used, 707 GiB / 715 GiB avail; 2.9 MiB/s rd, 2.6 MiB/s wr, 4 op/s 2024-04-07T16:51:18.986 INFO:tasks.workunit.client.0.smithi100.stdout: 738: throughput=4.99MB/sec pending data=0 2024-04-07T16:51:18.986 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Tzu9_vz_yIBUWDS off=15876986 len=1452562 2024-04-07T16:51:18.998 INFO:tasks.workunit.client.0.smithi100.stdout:op 3814 completed, throughput=5MB/sec 2024-04-07T16:51:18.998 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-jigEXTU70kl5MTF off=1361367 len=1054046 2024-04-07T16:51:19.008 INFO:tasks.workunit.client.0.smithi100.stdout:op 3815 completed, throughput=5MB/sec 2024-04-07T16:51:19.008 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-OJ_zL5zGBEp6OeN off=9162255 len=855311 2024-04-07T16:51:19.011 INFO:tasks.workunit.client.0.smithi100.stdout:op 3816 completed, throughput=5MB/sec 2024-04-07T16:51:19.011 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-LS0GNoeIzK0OgPa off=1721219 len=454237 2024-04-07T16:51:19.017 INFO:tasks.workunit.client.0.smithi100.stdout:op 3817 completed, throughput=5MB/sec 2024-04-07T16:51:19.017 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-wPv36K4Wm3l2Y_I off=4642216 len=198373 2024-04-07T16:51:19.022 INFO:tasks.workunit.client.0.smithi100.stdout:op 3818 completed, throughput=5MB/sec 2024-04-07T16:51:19.022 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-vW6tI2QtVYgmAeV off=1912457 len=131636 2024-04-07T16:51:19.023 INFO:tasks.workunit.client.0.smithi100.stdout:op 3819 completed, throughput=5MB/sec 2024-04-07T16:51:19.023 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-vrd5gPCDbu_mtqe off=10506221 len=949787 2024-04-07T16:51:19.031 INFO:tasks.workunit.client.0.smithi100.stdout:op 3820 completed, throughput=5MB/sec 2024-04-07T16:51:19.111 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:51:18 smithi144 bash[66654]: cluster 2024-04-07T16:51:17.462027+0000 mgr.y (mgr.25175) 1078 : cluster [DBG] pgmap v1720: 105 pgs: 105 active+clean; 126 GiB data, 8.4 GiB used, 707 GiB / 715 GiB avail; 2.9 MiB/s rd, 2.6 MiB/s wr, 4 op/s 2024-04-07T16:51:20.032 INFO:tasks.workunit.client.0.smithi100.stdout: 739: throughput=4.99MB/sec pending data=0 2024-04-07T16:51:20.032 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-u_LOP3kvX03__Nb off=4095094 len=1996650 2024-04-07T16:51:20.039 INFO:tasks.workunit.client.0.smithi100.stdout:op 3821 completed, throughput=5MB/sec 2024-04-07T16:51:20.039 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-hp8DMleCNq-pAiN off=15896449 len=580582 2024-04-07T16:51:20.046 INFO:tasks.workunit.client.0.smithi100.stdout:op 3822 completed, throughput=5MB/sec 2024-04-07T16:51:20.046 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-9uQV9FOZe4tEPuT off=267275 len=1140259 2024-04-07T16:51:20.056 INFO:tasks.workunit.client.0.smithi100.stdout:op 3823 completed, throughput=5MB/sec 2024-04-07T16:51:20.056 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-_033XXkBl0mpgm2 off=18141018 len=828650 2024-04-07T16:51:20.059 INFO:tasks.workunit.client.0.smithi100.stdout:op 3824 completed, throughput=5MB/sec 2024-04-07T16:51:20.059 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-qmUGgZE1nwntbxi off=768013 len=731856 2024-04-07T16:51:20.060 INFO:tasks.workunit.client.0.smithi100.stdout:op 3825 completed, throughput=5MB/sec 2024-04-07T16:51:20.947 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:51:20 smithi100 bash[74942]: cluster 2024-04-07T16:51:19.462990+0000 mgr.y (mgr.25175) 1079 : cluster [DBG] pgmap v1721: 105 pgs: 105 active+clean; 126 GiB data, 8.4 GiB used, 707 GiB / 715 GiB avail; 2.4 MiB/s rd, 2.2 MiB/s wr, 3 op/s 2024-04-07T16:51:20.948 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:51:20 smithi100 bash[76202]: cluster 2024-04-07T16:51:19.462990+0000 mgr.y (mgr.25175) 1079 : cluster [DBG] pgmap v1721: 105 pgs: 105 active+clean; 126 GiB data, 8.4 GiB used, 707 GiB / 715 GiB avail; 2.4 MiB/s rd, 2.2 MiB/s wr, 3 op/s 2024-04-07T16:51:21.060 INFO:tasks.workunit.client.0.smithi100.stdout: 740: throughput=4.99MB/sec pending data=0 2024-04-07T16:51:21.060 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-sQ-JQpbEp8Z98mN off=1157172 len=162999 2024-04-07T16:51:21.062 INFO:tasks.workunit.client.0.smithi100.stdout:op 3826 completed, throughput=4.99MB/sec 2024-04-07T16:51:21.062 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-MVRBDFNMy8-2uv2 off=0 len=1326782 2024-04-07T16:51:21.075 INFO:tasks.workunit.client.0.smithi100.stdout:op 3827 completed, throughput=5MB/sec 2024-04-07T16:51:21.075 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-2jjuQ1nOWkL2jB4 off=14738208 len=475385 2024-04-07T16:51:21.081 INFO:tasks.workunit.client.0.smithi100.stdout:op 3828 completed, throughput=5MB/sec 2024-04-07T16:51:21.081 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-f6TKkg2Sda7pMMO off=12146468 len=1510663 2024-04-07T16:51:21.085 INFO:tasks.workunit.client.0.smithi100.stdout:op 3829 completed, throughput=5MB/sec 2024-04-07T16:51:21.085 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-jcTUuaecUhTTGRE off=9443645 len=847496 2024-04-07T16:51:21.088 INFO:tasks.workunit.client.0.smithi100.stdout:op 3830 completed, throughput=5MB/sec 2024-04-07T16:51:21.088 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-PzPnk_CCrluOKDY off=1414268 len=1218510 2024-04-07T16:51:21.092 INFO:tasks.workunit.client.0.smithi100.stdout:op 3831 completed, throughput=5MB/sec 2024-04-07T16:51:21.111 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:51:20 smithi144 bash[66654]: cluster 2024-04-07T16:51:19.462990+0000 mgr.y (mgr.25175) 1079 : cluster [DBG] pgmap v1721: 105 pgs: 105 active+clean; 126 GiB data, 8.4 GiB used, 707 GiB / 715 GiB avail; 2.4 MiB/s rd, 2.2 MiB/s wr, 3 op/s 2024-04-07T16:51:22.092 INFO:tasks.workunit.client.0.smithi100.stdout: 741: throughput=4.99MB/sec pending data=0 2024-04-07T16:51:22.092 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-UhzcSyt6I1aZwti off=6638721 len=1395371 2024-04-07T16:51:22.103 INFO:tasks.workunit.client.0.smithi100.stdout:op 3832 completed, throughput=5MB/sec 2024-04-07T16:51:22.104 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-4pdsC1TCDyIP9vd off=3086133 len=542901 2024-04-07T16:51:22.105 INFO:tasks.workunit.client.0.smithi100.stdout:op 3833 completed, throughput=5MB/sec 2024-04-07T16:51:22.105 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-lCHQG9i6kr8zVbv off=261513 len=1011732 2024-04-07T16:51:22.108 INFO:tasks.workunit.client.0.smithi100.stdout:op 3834 completed, throughput=5MB/sec 2024-04-07T16:51:22.108 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-ihZ1JTRwa1C4E3x off=7615147 len=1429417 2024-04-07T16:51:22.112 INFO:tasks.workunit.client.0.smithi100.stdout:op 3835 completed, throughput=5MB/sec 2024-04-07T16:51:22.112 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-GnGA1WXD4-1SWqO off=7697028 len=970111 2024-04-07T16:51:22.121 INFO:tasks.workunit.client.0.smithi100.stdout:op 3836 completed, throughput=5MB/sec 2024-04-07T16:51:22.612 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:51:22 smithi144 bash[95233]: ts=2024-04-07T16:51:22.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:51:22.947 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:51:22 smithi100 bash[74942]: cluster 2024-04-07T16:51:21.463868+0000 mgr.y (mgr.25175) 1080 : cluster [DBG] pgmap v1722: 105 pgs: 105 active+clean; 126 GiB data, 8.4 GiB used, 707 GiB / 715 GiB avail; 2.4 MiB/s rd, 2.3 MiB/s wr, 4 op/s 2024-04-07T16:51:22.947 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:51:22 smithi100 bash[76202]: cluster 2024-04-07T16:51:21.463868+0000 mgr.y (mgr.25175) 1080 : cluster [DBG] pgmap v1722: 105 pgs: 105 active+clean; 126 GiB data, 8.4 GiB used, 707 GiB / 715 GiB avail; 2.4 MiB/s rd, 2.3 MiB/s wr, 4 op/s 2024-04-07T16:51:23.111 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:51:22 smithi144 bash[66654]: cluster 2024-04-07T16:51:21.463868+0000 mgr.y (mgr.25175) 1080 : cluster [DBG] pgmap v1722: 105 pgs: 105 active+clean; 126 GiB data, 8.4 GiB used, 707 GiB / 715 GiB avail; 2.4 MiB/s rd, 2.3 MiB/s wr, 4 op/s 2024-04-07T16:51:23.122 INFO:tasks.workunit.client.0.smithi100.stdout: 742: throughput=4.99MB/sec pending data=0 2024-04-07T16:51:23.122 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-gmlHG5QW2z5LE3e off=9806443 len=505477 2024-04-07T16:51:23.128 INFO:tasks.workunit.client.0.smithi100.stdout:op 3837 completed, throughput=4.99MB/sec 2024-04-07T16:51:23.128 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-lc3v61EvyfRS4nn off=10102225 len=2027373 2024-04-07T16:51:23.143 INFO:tasks.workunit.client.0.smithi100.stdout:op 3838 completed, throughput=5MB/sec 2024-04-07T16:51:23.143 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-bG5jYhv80Rzzzn_ off=2840475 len=1184779 2024-04-07T16:51:23.153 INFO:tasks.workunit.client.0.smithi100.stdout:op 3839 completed, throughput=5MB/sec 2024-04-07T16:51:23.153 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Gh_XLdovPuVJmKU off=992395 len=805239 2024-04-07T16:51:23.157 INFO:tasks.workunit.client.0.smithi100.stdout:op 3840 completed, throughput=5MB/sec 2024-04-07T16:51:23.157 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-rqVnD2X3EtjbZc6 off=1455182 len=27298 2024-04-07T16:51:23.161 INFO:tasks.workunit.client.0.smithi100.stdout:op 3841 completed, throughput=5MB/sec 2024-04-07T16:51:23.162 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-FkkckJjkw_PWO66 off=651047 len=339812 2024-04-07T16:51:23.163 INFO:tasks.workunit.client.0.smithi100.stdout:op 3842 completed, throughput=5MB/sec 2024-04-07T16:51:23.163 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-4tAE24FaCPUX40t off=2800668 len=1102976 2024-04-07T16:51:23.166 INFO:tasks.workunit.client.0.smithi100.stdout:op 3843 completed, throughput=5MB/sec 2024-04-07T16:51:24.167 INFO:tasks.workunit.client.0.smithi100.stdout: 743: throughput=4.99MB/sec pending data=0 2024-04-07T16:51:24.167 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-oAPz6nu22PexoAq off=15049070 len=764144 2024-04-07T16:51:24.174 INFO:tasks.workunit.client.0.smithi100.stdout:op 3844 completed, throughput=5MB/sec 2024-04-07T16:51:24.174 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-qKAyx8Mm5c2vdhu off=1082664 len=453067 2024-04-07T16:51:24.176 INFO:tasks.workunit.client.0.smithi100.stdout:op 3845 completed, throughput=5MB/sec 2024-04-07T16:51:24.176 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Qx_w6sv42j19YZ6 off=23293262 len=132282 2024-04-07T16:51:24.181 INFO:tasks.workunit.client.0.smithi100.stdout:op 3846 completed, throughput=5MB/sec 2024-04-07T16:51:24.181 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-13Hqvy9Fge2ua1p off=16045042 len=557128 2024-04-07T16:51:24.184 INFO:tasks.workunit.client.0.smithi100.stdout:op 3847 completed, throughput=5MB/sec 2024-04-07T16:51:24.184 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-kXOtLWXmoykR9jt off=6897828 len=353268 2024-04-07T16:51:24.185 INFO:tasks.workunit.client.0.smithi100.stdout:op 3848 completed, throughput=5MB/sec 2024-04-07T16:51:24.185 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-BohnjAetGmVNse2 off=6678999 len=160079 2024-04-07T16:51:24.190 INFO:tasks.workunit.client.0.smithi100.stdout:op 3849 completed, throughput=5MB/sec 2024-04-07T16:51:24.190 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-FeLGy3WY5wLjTm4 off=8980047 len=103669 2024-04-07T16:51:24.195 INFO:tasks.workunit.client.0.smithi100.stdout:op 3850 completed, throughput=5MB/sec 2024-04-07T16:51:24.195 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-gYFhjXLhvg1pn_K off=10259522 len=558368 2024-04-07T16:51:24.198 INFO:tasks.workunit.client.0.smithi100.stdout:op 3851 completed, throughput=5MB/sec 2024-04-07T16:51:24.198 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-puHtooEDNgxQ5TC off=17888735 len=899041 2024-04-07T16:51:24.200 INFO:tasks.workunit.client.0.smithi100.stdout:op 3852 completed, throughput=5MB/sec 2024-04-07T16:51:24.201 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-EQX-ra5wpmMr21m off=4655711 len=1009008 2024-04-07T16:51:24.203 INFO:tasks.workunit.client.0.smithi100.stdout:op 3853 completed, throughput=5MB/sec 2024-04-07T16:51:24.947 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:51:24 smithi100 bash[74942]: cluster 2024-04-07T16:51:23.465004+0000 mgr.y (mgr.25175) 1081 : cluster [DBG] pgmap v1723: 105 pgs: 105 active+clean; 126 GiB data, 8.4 GiB used, 707 GiB / 715 GiB avail; 2.6 MiB/s rd, 2.5 MiB/s wr, 5 op/s 2024-04-07T16:51:24.948 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:51:24 smithi100 bash[76202]: cluster 2024-04-07T16:51:23.465004+0000 mgr.y (mgr.25175) 1081 : cluster [DBG] pgmap v1723: 105 pgs: 105 active+clean; 126 GiB data, 8.4 GiB used, 707 GiB / 715 GiB avail; 2.6 MiB/s rd, 2.5 MiB/s wr, 5 op/s 2024-04-07T16:51:25.111 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:51:24 smithi144 bash[66654]: cluster 2024-04-07T16:51:23.465004+0000 mgr.y (mgr.25175) 1081 : cluster [DBG] pgmap v1723: 105 pgs: 105 active+clean; 126 GiB data, 8.4 GiB used, 707 GiB / 715 GiB avail; 2.6 MiB/s rd, 2.5 MiB/s wr, 5 op/s 2024-04-07T16:51:25.203 INFO:tasks.workunit.client.0.smithi100.stdout: 744: throughput=4.99MB/sec pending data=0 2024-04-07T16:51:25.204 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-vCaoLgJ6oPt-fRU off=3128256 len=534740 2024-04-07T16:51:25.210 INFO:tasks.workunit.client.0.smithi100.stdout:op 3854 completed, throughput=4.99MB/sec 2024-04-07T16:51:25.211 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-bwHfZBspmsxAIEU off=3485254 len=359065 2024-04-07T16:51:25.212 INFO:tasks.workunit.client.0.smithi100.stdout:op 3855 completed, throughput=5MB/sec 2024-04-07T16:51:25.212 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-fOIrgSN9vwyCnCX off=13474986 len=1411232 2024-04-07T16:51:25.217 INFO:tasks.workunit.client.0.smithi100.stdout:op 3856 completed, throughput=5MB/sec 2024-04-07T16:51:25.217 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-rgTL1oOY_ovZ2XK off=7545579 len=461618 2024-04-07T16:51:25.224 INFO:tasks.workunit.client.0.smithi100.stdout:op 3857 completed, throughput=5MB/sec 2024-04-07T16:51:25.224 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-82LpxEodys0KXly off=3280001 len=1377162 2024-04-07T16:51:25.234 INFO:tasks.workunit.client.0.smithi100.stdout:op 3858 completed, throughput=5MB/sec 2024-04-07T16:51:25.234 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-KoEEZjOTNmpSvUm off=13313079 len=1005231 2024-04-07T16:51:25.243 INFO:tasks.workunit.client.0.smithi100.stdout:op 3859 completed, throughput=5MB/sec 2024-04-07T16:51:26.243 INFO:tasks.workunit.client.0.smithi100.stdout: 745: throughput=4.99MB/sec pending data=0 2024-04-07T16:51:26.243 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-pru7WAXwqqiJJm8 off=12163354 len=701750 2024-04-07T16:51:26.246 INFO:tasks.workunit.client.0.smithi100.stdout:op 3860 completed, throughput=4.99MB/sec 2024-04-07T16:51:26.247 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-PJpxigOZyDdoE9l off=12605204 len=2062178 2024-04-07T16:51:26.262 INFO:tasks.workunit.client.0.smithi100.stdout:op 3861 completed, throughput=5MB/sec 2024-04-07T16:51:26.262 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-11LEXzsA6PxeAdQ off=8064470 len=1242000 2024-04-07T16:51:26.274 INFO:tasks.workunit.client.0.smithi100.stdout:op 3862 completed, throughput=5MB/sec 2024-04-07T16:51:26.274 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-tAjlEcdy0J44dbY off=0 len=1282063 2024-04-07T16:51:26.285 INFO:tasks.workunit.client.0.smithi100.stdout:op 3863 completed, throughput=5MB/sec 2024-04-07T16:51:26.947 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:51:26 smithi100 bash[74942]: cluster 2024-04-07T16:51:25.465982+0000 mgr.y (mgr.25175) 1082 : cluster [DBG] pgmap v1724: 105 pgs: 105 active+clean; 126 GiB data, 8.4 GiB used, 707 GiB / 715 GiB avail; 2.4 MiB/s rd, 2.2 MiB/s wr, 5 op/s 2024-04-07T16:51:26.947 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:51:26 smithi100 bash[76202]: cluster 2024-04-07T16:51:25.465982+0000 mgr.y (mgr.25175) 1082 : cluster [DBG] pgmap v1724: 105 pgs: 105 active+clean; 126 GiB data, 8.4 GiB used, 707 GiB / 715 GiB avail; 2.4 MiB/s rd, 2.2 MiB/s wr, 5 op/s 2024-04-07T16:51:27.111 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:51:26 smithi144 bash[66654]: cluster 2024-04-07T16:51:25.465982+0000 mgr.y (mgr.25175) 1082 : cluster [DBG] pgmap v1724: 105 pgs: 105 active+clean; 126 GiB data, 8.4 GiB used, 707 GiB / 715 GiB avail; 2.4 MiB/s rd, 2.2 MiB/s wr, 5 op/s 2024-04-07T16:51:27.285 INFO:tasks.workunit.client.0.smithi100.stdout: 746: throughput=4.99MB/sec pending data=0 2024-04-07T16:51:27.285 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-2X5G4FuDz_0vIc2 off=8864816 len=637707 2024-04-07T16:51:27.293 INFO:tasks.workunit.client.0.smithi100.stdout:op 3864 completed, throughput=4.99MB/sec 2024-04-07T16:51:27.293 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-4Xz4-DQFE_gU-3j off=39107 len=739713 2024-04-07T16:51:27.300 INFO:tasks.workunit.client.0.smithi100.stdout:op 3865 completed, throughput=5MB/sec 2024-04-07T16:51:27.300 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-epdqhSRtsIcr-RV off=8107779 len=382180 2024-04-07T16:51:27.302 INFO:tasks.workunit.client.0.smithi100.stdout:op 3866 completed, throughput=5MB/sec 2024-04-07T16:51:27.303 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-AZnfvv8Lj8hl8vl off=5730244 len=5815 2024-04-07T16:51:27.303 INFO:tasks.workunit.client.0.smithi100.stdout:op 3867 completed, throughput=5MB/sec 2024-04-07T16:51:27.303 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-_ZTqGFAxyXsaXFO off=1171144 len=2019711 2024-04-07T16:51:27.308 INFO:tasks.workunit.client.0.smithi100.stdout:op 3868 completed, throughput=5MB/sec 2024-04-07T16:51:27.308 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Dwt1F8ZXzZqai4V off=12175883 len=1402630 2024-04-07T16:51:27.311 INFO:tasks.workunit.client.0.smithi100.stdout:op 3869 completed, throughput=5MB/sec 2024-04-07T16:51:27.311 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-L8i5s7rHIvK3w9O off=15233720 len=1797678 2024-04-07T16:51:27.329 INFO:tasks.workunit.client.0.smithi100.stdout:op 3870 completed, throughput=5MB/sec 2024-04-07T16:51:28.328 INFO:tasks.workunit.client.0.smithi100.stdout: 747: throughput=5MB/sec pending data=0 2024-04-07T16:51:28.328 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-vT3wtHous9DQwel off=3124752 len=1368373 2024-04-07T16:51:28.331 INFO:tasks.workunit.client.0.smithi100.stdout:op 3871 completed, throughput=5MB/sec 2024-04-07T16:51:28.332 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-PcRwkRiqPaIPV4U off=885696 len=1907154 2024-04-07T16:51:28.338 INFO:tasks.workunit.client.0.smithi100.stdout:op 3872 completed, throughput=5MB/sec 2024-04-07T16:51:28.338 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Vn7aY105HuqzXmj off=1283929 len=275719 2024-04-07T16:51:28.347 INFO:tasks.workunit.client.0.smithi100.stdout:op 3873 completed, throughput=5MB/sec 2024-04-07T16:51:28.347 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-ujjKckqemtEAQ54 off=15227457 len=1625764 2024-04-07T16:51:28.351 INFO:tasks.workunit.client.0.smithi100.stdout:op 3874 completed, throughput=5MB/sec 2024-04-07T16:51:28.697 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:51:28 smithi100 bash[74942]: cluster 2024-04-07T16:51:27.467036+0000 mgr.y (mgr.25175) 1083 : cluster [DBG] pgmap v1725: 105 pgs: 105 active+clean; 126 GiB data, 8.5 GiB used, 707 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.3 MiB/s wr, 5 op/s 2024-04-07T16:51:28.698 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:51:28 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:51:28] "GET /metrics HTTP/1.1" 200 34069 "" "Prometheus/2.43.0" 2024-04-07T16:51:28.698 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:51:28 smithi100 bash[76202]: cluster 2024-04-07T16:51:27.467036+0000 mgr.y (mgr.25175) 1083 : cluster [DBG] pgmap v1725: 105 pgs: 105 active+clean; 126 GiB data, 8.5 GiB used, 707 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.3 MiB/s wr, 5 op/s 2024-04-07T16:51:29.111 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:51:28 smithi144 bash[66654]: cluster 2024-04-07T16:51:27.467036+0000 mgr.y (mgr.25175) 1083 : cluster [DBG] pgmap v1725: 105 pgs: 105 active+clean; 126 GiB data, 8.5 GiB used, 707 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.3 MiB/s wr, 5 op/s 2024-04-07T16:51:29.351 INFO:tasks.workunit.client.0.smithi100.stdout: 748: throughput=5MB/sec pending data=0 2024-04-07T16:51:29.351 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-9IXvirKm_SSTvJ2 off=3409386 len=944896 2024-04-07T16:51:29.360 INFO:tasks.workunit.client.0.smithi100.stdout:op 3875 completed, throughput=5MB/sec 2024-04-07T16:51:29.360 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-2EfMOx5WUgL9LuV off=236816 len=948223 2024-04-07T16:51:29.369 INFO:tasks.workunit.client.0.smithi100.stdout:op 3876 completed, throughput=5MB/sec 2024-04-07T16:51:29.369 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-yUWFBD35lYBf62k off=5402548 len=473650 2024-04-07T16:51:29.377 INFO:tasks.workunit.client.0.smithi100.stdout:op 3877 completed, throughput=5MB/sec 2024-04-07T16:51:29.377 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-axmnCYg_HPfzRaD off=3430324 len=2052409 2024-04-07T16:51:29.396 INFO:tasks.workunit.client.0.smithi100.stdout:op 3878 completed, throughput=5MB/sec 2024-04-07T16:51:30.396 INFO:tasks.workunit.client.0.smithi100.stdout: 749: throughput=4.99MB/sec pending data=0 2024-04-07T16:51:30.396 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-QUJwwqDkdYvHNbB off=6098687 len=2077019 2024-04-07T16:51:30.401 INFO:tasks.workunit.client.0.smithi100.stdout:op 3879 completed, throughput=5MB/sec 2024-04-07T16:51:30.401 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-AFG4XLVuOBwjGb6 off=4847996 len=871539 2024-04-07T16:51:30.404 INFO:tasks.workunit.client.0.smithi100.stdout:op 3880 completed, throughput=5MB/sec 2024-04-07T16:51:30.405 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-wxBU439OYfJifvh off=2868801 len=508528 2024-04-07T16:51:30.406 INFO:tasks.workunit.client.0.smithi100.stdout:op 3881 completed, throughput=5MB/sec 2024-04-07T16:51:30.407 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Z70G1uST9s07l_6 off=6314352 len=432046 2024-04-07T16:51:30.413 INFO:tasks.workunit.client.0.smithi100.stdout:op 3882 completed, throughput=5MB/sec 2024-04-07T16:51:30.414 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-O7reni1_HZUHxe1 off=3721366 len=304139 2024-04-07T16:51:30.415 INFO:tasks.workunit.client.0.smithi100.stdout:op 3883 completed, throughput=5MB/sec 2024-04-07T16:51:30.415 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-aPQGhQvqJCOyeXA off=3456278 len=1982524 2024-04-07T16:51:30.421 INFO:tasks.workunit.client.0.smithi100.stdout:op 3884 completed, throughput=5MB/sec 2024-04-07T16:51:30.947 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:51:30 smithi100 bash[74942]: cluster 2024-04-07T16:51:29.468033+0000 mgr.y (mgr.25175) 1084 : cluster [DBG] pgmap v1726: 105 pgs: 105 active+clean; 126 GiB data, 8.5 GiB used, 707 GiB / 715 GiB avail; 2.3 MiB/s rd, 2.0 MiB/s wr, 5 op/s 2024-04-07T16:51:30.948 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:51:30 smithi100 bash[76202]: cluster 2024-04-07T16:51:29.468033+0000 mgr.y (mgr.25175) 1084 : cluster [DBG] pgmap v1726: 105 pgs: 105 active+clean; 126 GiB data, 8.5 GiB used, 707 GiB / 715 GiB avail; 2.3 MiB/s rd, 2.0 MiB/s wr, 5 op/s 2024-04-07T16:51:31.111 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:51:30 smithi144 bash[66654]: cluster 2024-04-07T16:51:29.468033+0000 mgr.y (mgr.25175) 1084 : cluster [DBG] pgmap v1726: 105 pgs: 105 active+clean; 126 GiB data, 8.5 GiB used, 707 GiB / 715 GiB avail; 2.3 MiB/s rd, 2.0 MiB/s wr, 5 op/s 2024-04-07T16:51:31.421 INFO:tasks.workunit.client.0.smithi100.stdout: 750: throughput=5MB/sec pending data=0 2024-04-07T16:51:31.421 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-WxgsGtBLkXQBNZ1 off=20788925 len=1889290 2024-04-07T16:51:31.426 INFO:tasks.workunit.client.0.smithi100.stdout:op 3885 completed, throughput=5MB/sec 2024-04-07T16:51:31.426 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-dnH1-dRkHBIqBtv off=19304851 len=1380805 2024-04-07T16:51:31.429 INFO:tasks.workunit.client.0.smithi100.stdout:op 3886 completed, throughput=5MB/sec 2024-04-07T16:51:31.429 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-zR5jG3yG5RK3Kzx off=13779896 len=1161064 2024-04-07T16:51:31.439 INFO:tasks.workunit.client.0.smithi100.stdout:op 3887 completed, throughput=5MB/sec 2024-04-07T16:51:32.440 INFO:tasks.workunit.client.0.smithi100.stdout: 751: throughput=4.99MB/sec pending data=0 2024-04-07T16:51:32.440 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-1UjLyRJ4UlFxyiv off=3286083 len=171552 2024-04-07T16:51:32.445 INFO:tasks.workunit.client.0.smithi100.stdout:op 3888 completed, throughput=4.99MB/sec 2024-04-07T16:51:32.445 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-eVlJV2lwqWfoKPC off=151248 len=1950197 2024-04-07T16:51:32.449 INFO:tasks.workunit.client.0.smithi100.stdout:op 3889 completed, throughput=5MB/sec 2024-04-07T16:51:32.449 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-b1TVf8OKL7sWqtH off=286430 len=1685787 2024-04-07T16:51:32.464 INFO:tasks.workunit.client.0.smithi100.stdout:op 3890 completed, throughput=5MB/sec 2024-04-07T16:51:32.464 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-TILjCMDH8PjTZG4 off=8554504 len=286812 2024-04-07T16:51:32.466 INFO:tasks.workunit.client.0.smithi100.stdout:op 3891 completed, throughput=5MB/sec 2024-04-07T16:51:32.466 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-WnMVhMjkplbRZuX off=8554815 len=1720693 2024-04-07T16:51:32.480 INFO:tasks.workunit.client.0.smithi100.stdout:op 3892 completed, throughput=5MB/sec 2024-04-07T16:51:32.611 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:51:32 smithi144 bash[95233]: ts=2024-04-07T16:51:32.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:51:32.947 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:51:32 smithi100 bash[74942]: cluster 2024-04-07T16:51:31.468858+0000 mgr.y (mgr.25175) 1085 : cluster [DBG] pgmap v1727: 105 pgs: 105 active+clean; 126 GiB data, 8.5 GiB used, 707 GiB / 715 GiB avail; 2.1 MiB/s rd, 2.2 MiB/s wr, 5 op/s 2024-04-07T16:51:32.947 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:51:32 smithi100 bash[74942]: audit 2024-04-07T16:51:31.777867+0000 mon.a (mon.0) 2156 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:51:32.948 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:51:32 smithi100 bash[76202]: cluster 2024-04-07T16:51:31.468858+0000 mgr.y (mgr.25175) 1085 : cluster [DBG] pgmap v1727: 105 pgs: 105 active+clean; 126 GiB data, 8.5 GiB used, 707 GiB / 715 GiB avail; 2.1 MiB/s rd, 2.2 MiB/s wr, 5 op/s 2024-04-07T16:51:32.948 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:51:32 smithi100 bash[76202]: audit 2024-04-07T16:51:31.777867+0000 mon.a (mon.0) 2156 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:51:33.111 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:51:32 smithi144 bash[66654]: cluster 2024-04-07T16:51:31.468858+0000 mgr.y (mgr.25175) 1085 : cluster [DBG] pgmap v1727: 105 pgs: 105 active+clean; 126 GiB data, 8.5 GiB used, 707 GiB / 715 GiB avail; 2.1 MiB/s rd, 2.2 MiB/s wr, 5 op/s 2024-04-07T16:51:33.111 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:51:32 smithi144 bash[66654]: audit 2024-04-07T16:51:31.777867+0000 mon.a (mon.0) 2156 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:51:33.480 INFO:tasks.workunit.client.0.smithi100.stdout: 752: throughput=4.99MB/sec pending data=0 2024-04-07T16:51:33.480 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-anAo76tF8Vd_mB6 off=2342375 len=1684092 2024-04-07T16:51:33.493 INFO:tasks.workunit.client.0.smithi100.stdout:op 3893 completed, throughput=5MB/sec 2024-04-07T16:51:33.493 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-ZBCcXSWOh0I4RFd off=8925383 len=2041639 2024-04-07T16:51:33.506 INFO:tasks.workunit.client.0.smithi100.stdout:op 3894 completed, throughput=5MB/sec 2024-04-07T16:51:33.506 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-ednkE_Iw-ofOB3J off=21281733 len=743687 2024-04-07T16:51:33.509 INFO:tasks.workunit.client.0.smithi100.stdout:op 3895 completed, throughput=5MB/sec 2024-04-07T16:51:33.509 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-OXWbevsX_3vjCLt off=2960512 len=1854217 2024-04-07T16:51:33.523 INFO:tasks.workunit.client.0.smithi100.stdout:op 3896 completed, throughput=5MB/sec 2024-04-07T16:51:34.523 INFO:tasks.workunit.client.0.smithi100.stdout: 753: throughput=5MB/sec pending data=0 2024-04-07T16:51:34.524 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-tNuvhj6mpmC8Cy_ off=745988 len=482929 2024-04-07T16:51:34.530 INFO:tasks.workunit.client.0.smithi100.stdout:op 3897 completed, throughput=5MB/sec 2024-04-07T16:51:34.530 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-xirs2VuqS8rxmOF off=2093756 len=23747 2024-04-07T16:51:34.531 INFO:tasks.workunit.client.0.smithi100.stdout:op 3898 completed, throughput=5MB/sec 2024-04-07T16:51:34.531 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-nzrVENG4Ujt-mzg off=16744505 len=1346729 2024-04-07T16:51:34.541 INFO:tasks.workunit.client.0.smithi100.stdout:op 3899 completed, throughput=5MB/sec 2024-04-07T16:51:34.541 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-uf-FW8g-_cUXFq5 off=607958 len=604986 2024-04-07T16:51:34.543 INFO:tasks.workunit.client.0.smithi100.stdout:op 3900 completed, throughput=5MB/sec 2024-04-07T16:51:34.543 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-zfPVv-Hlm3nXhxk off=1148837 len=1881232 2024-04-07T16:51:34.557 INFO:tasks.workunit.client.0.smithi100.stdout:op 3901 completed, throughput=5MB/sec 2024-04-07T16:51:34.948 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:51:34 smithi100 bash[74942]: cluster 2024-04-07T16:51:33.469965+0000 mgr.y (mgr.25175) 1086 : cluster [DBG] pgmap v1728: 105 pgs: 105 active+clean; 126 GiB data, 8.5 GiB used, 707 GiB / 715 GiB avail; 2.9 MiB/s rd, 2.4 MiB/s wr, 5 op/s 2024-04-07T16:51:34.948 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:51:34 smithi100 bash[76202]: cluster 2024-04-07T16:51:33.469965+0000 mgr.y (mgr.25175) 1086 : cluster [DBG] pgmap v1728: 105 pgs: 105 active+clean; 126 GiB data, 8.5 GiB used, 707 GiB / 715 GiB avail; 2.9 MiB/s rd, 2.4 MiB/s wr, 5 op/s 2024-04-07T16:51:35.111 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:51:34 smithi144 bash[66654]: cluster 2024-04-07T16:51:33.469965+0000 mgr.y (mgr.25175) 1086 : cluster [DBG] pgmap v1728: 105 pgs: 105 active+clean; 126 GiB data, 8.5 GiB used, 707 GiB / 715 GiB avail; 2.9 MiB/s rd, 2.4 MiB/s wr, 5 op/s 2024-04-07T16:51:35.558 INFO:tasks.workunit.client.0.smithi100.stdout: 754: throughput=4.99MB/sec pending data=0 2024-04-07T16:51:35.558 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Dd4sQv_4Ny0_JQ6 off=15075624 len=1208614 2024-04-07T16:51:35.561 INFO:tasks.workunit.client.0.smithi100.stdout:op 3902 completed, throughput=5MB/sec 2024-04-07T16:51:35.561 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-vwV2nY7JibW_XrE off=1536048 len=1886438 2024-04-07T16:51:35.575 INFO:tasks.workunit.client.0.smithi100.stdout:op 3903 completed, throughput=5MB/sec 2024-04-07T16:51:35.575 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-yhbc8QeEYE0xA8X off=4166194 len=1688373 2024-04-07T16:51:35.581 INFO:tasks.workunit.client.0.smithi100.stdout:op 3904 completed, throughput=5MB/sec 2024-04-07T16:51:35.581 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-iJFzxTeajYLkYAL off=918183 len=949455 2024-04-07T16:51:35.583 INFO:tasks.workunit.client.0.smithi100.stdout:op 3905 completed, throughput=5MB/sec 2024-04-07T16:51:36.584 INFO:tasks.workunit.client.0.smithi100.stdout: 755: throughput=4.99MB/sec pending data=0 2024-04-07T16:51:36.584 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-3YBenVQuvVoSqva off=9663799 len=2014199 2024-04-07T16:51:36.602 INFO:tasks.workunit.client.0.smithi100.stdout:op 3906 completed, throughput=5MB/sec 2024-04-07T16:51:36.602 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-U56bggLjrBGyzUY off=1001325 len=676080 2024-04-07T16:51:36.604 INFO:tasks.workunit.client.0.smithi100.stdout:op 3907 completed, throughput=5MB/sec 2024-04-07T16:51:36.604 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-qSlfamtnDYFmnxm off=2150855 len=501566 2024-04-07T16:51:36.611 INFO:tasks.workunit.client.0.smithi100.stdout:op 3908 completed, throughput=5MB/sec 2024-04-07T16:51:36.611 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-zJTX-ma9zk_-1HT off=3520905 len=669054 2024-04-07T16:51:36.617 INFO:tasks.workunit.client.0.smithi100.stdout:op 3909 completed, throughput=5MB/sec 2024-04-07T16:51:36.618 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-fZ0qwXGLadUuZHG off=5828526 len=456583 2024-04-07T16:51:36.624 INFO:tasks.workunit.client.0.smithi100.stdout:op 3910 completed, throughput=5MB/sec 2024-04-07T16:51:36.624 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-qkOcNKR71EdeM89 off=12874377 len=2061181 2024-04-07T16:51:36.629 INFO:tasks.workunit.client.0.smithi100.stdout:op 3911 completed, throughput=5MB/sec 2024-04-07T16:51:36.947 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:51:36 smithi100 bash[74942]: cluster 2024-04-07T16:51:35.470905+0000 mgr.y (mgr.25175) 1087 : cluster [DBG] pgmap v1729: 105 pgs: 105 active+clean; 126 GiB data, 8.5 GiB used, 707 GiB / 715 GiB avail; 2.6 MiB/s rd, 2.3 MiB/s wr, 5 op/s 2024-04-07T16:51:36.948 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:51:36 smithi100 bash[76202]: cluster 2024-04-07T16:51:35.470905+0000 mgr.y (mgr.25175) 1087 : cluster [DBG] pgmap v1729: 105 pgs: 105 active+clean; 126 GiB data, 8.5 GiB used, 707 GiB / 715 GiB avail; 2.6 MiB/s rd, 2.3 MiB/s wr, 5 op/s 2024-04-07T16:51:37.111 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:51:36 smithi144 bash[66654]: cluster 2024-04-07T16:51:35.470905+0000 mgr.y (mgr.25175) 1087 : cluster [DBG] pgmap v1729: 105 pgs: 105 active+clean; 126 GiB data, 8.5 GiB used, 707 GiB / 715 GiB avail; 2.6 MiB/s rd, 2.3 MiB/s wr, 5 op/s 2024-04-07T16:51:37.629 INFO:tasks.workunit.client.0.smithi100.stdout: 756: throughput=5MB/sec pending data=0 2024-04-07T16:51:37.629 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-TKqyLOH8v8RB91J off=5736499 len=433619 2024-04-07T16:51:37.637 INFO:tasks.workunit.client.0.smithi100.stdout:op 3912 completed, throughput=5MB/sec 2024-04-07T16:51:37.637 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Z6PqueplXc76sRF off=12220815 len=1976480 2024-04-07T16:51:37.655 INFO:tasks.workunit.client.0.smithi100.stdout:op 3913 completed, throughput=5MB/sec 2024-04-07T16:51:37.655 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-QdyrvToSkU0bn8J off=8521723 len=475647 2024-04-07T16:51:37.663 INFO:tasks.workunit.client.0.smithi100.stdout:op 3914 completed, throughput=5MB/sec 2024-04-07T16:51:37.663 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-5lRXrAt0uUYiRSt off=0 len=1233092 2024-04-07T16:51:37.667 INFO:tasks.workunit.client.0.smithi100.stdout:op 3915 completed, throughput=5MB/sec 2024-04-07T16:51:38.668 INFO:tasks.workunit.client.0.smithi100.stdout: 757: throughput=4.99MB/sec pending data=0 2024-04-07T16:51:38.668 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj--dMoidRMsIwW8d0 off=6531181 len=337467 2024-04-07T16:51:38.669 INFO:tasks.workunit.client.0.smithi100.stdout:op 3916 completed, throughput=4.99MB/sec 2024-04-07T16:51:38.669 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-RGDoeO8WBOoA2-Z off=187167 len=1829653 2024-04-07T16:51:38.673 INFO:tasks.workunit.client.0.smithi100.stdout:op 3917 completed, throughput=5MB/sec 2024-04-07T16:51:38.673 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-NOtjBrD-86u8sJG off=2938446 len=1505148 2024-04-07T16:51:38.679 INFO:tasks.workunit.client.0.smithi100.stdout:op 3918 completed, throughput=5MB/sec 2024-04-07T16:51:38.679 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-K9dQJk_dE4f-PSc off=897569 len=15728 2024-04-07T16:51:38.683 INFO:tasks.workunit.client.0.smithi100.stdout:op 3919 completed, throughput=5MB/sec 2024-04-07T16:51:38.683 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-CPcg4_MkdZx7CF9 off=8958879 len=2065778 2024-04-07T16:51:38.689 INFO:tasks.workunit.client.0.smithi100.stdout:op 3920 completed, throughput=5MB/sec 2024-04-07T16:51:38.697 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:51:38 smithi100 bash[74942]: cluster 2024-04-07T16:51:37.471975+0000 mgr.y (mgr.25175) 1088 : cluster [DBG] pgmap v1730: 105 pgs: 105 active+clean; 126 GiB data, 8.5 GiB used, 707 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.6 MiB/s wr, 5 op/s 2024-04-07T16:51:38.697 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:51:38 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:51:38] "GET /metrics HTTP/1.1" 200 34072 "" "Prometheus/2.43.0" 2024-04-07T16:51:38.698 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:51:38 smithi100 bash[76202]: cluster 2024-04-07T16:51:37.471975+0000 mgr.y (mgr.25175) 1088 : cluster [DBG] pgmap v1730: 105 pgs: 105 active+clean; 126 GiB data, 8.5 GiB used, 707 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.6 MiB/s wr, 5 op/s 2024-04-07T16:51:39.111 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:51:38 smithi144 bash[66654]: cluster 2024-04-07T16:51:37.471975+0000 mgr.y (mgr.25175) 1088 : cluster [DBG] pgmap v1730: 105 pgs: 105 active+clean; 126 GiB data, 8.5 GiB used, 707 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.6 MiB/s wr, 5 op/s 2024-04-07T16:51:39.689 INFO:tasks.workunit.client.0.smithi100.stdout: 758: throughput=4.99MB/sec pending data=0 2024-04-07T16:51:39.689 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-AIzwdkxFGdjaGiJ off=4350677 len=1803811 2024-04-07T16:51:39.703 INFO:tasks.workunit.client.0.smithi100.stdout:op 3921 completed, throughput=5MB/sec 2024-04-07T16:51:39.703 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-dvQd_9lO-VwKMCw off=1192795 len=628930 2024-04-07T16:51:39.705 INFO:tasks.workunit.client.0.smithi100.stdout:op 3922 completed, throughput=5MB/sec 2024-04-07T16:51:39.705 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-4OJ6i0Ie5f-OFWr off=4372989 len=361298 2024-04-07T16:51:39.707 INFO:tasks.workunit.client.0.smithi100.stdout:op 3923 completed, throughput=5MB/sec 2024-04-07T16:51:39.707 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-oRK5xxHotDFjwmM off=15475611 len=23828 2024-04-07T16:51:39.708 INFO:tasks.workunit.client.0.smithi100.stdout:op 3924 completed, throughput=5MB/sec 2024-04-07T16:51:39.709 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-9YHFre32tyrfdET off=2454869 len=487452 2024-04-07T16:51:39.710 INFO:tasks.workunit.client.0.smithi100.stdout:op 3925 completed, throughput=5MB/sec 2024-04-07T16:51:39.710 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-NRjaXQ39qTNSLZJ off=0 len=1200705 2024-04-07T16:51:39.720 INFO:tasks.workunit.client.0.smithi100.stdout:op 3926 completed, throughput=5MB/sec 2024-04-07T16:51:39.720 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-1NkHyn9-f7jRbBI off=1730974 len=1996026 2024-04-07T16:51:39.724 INFO:tasks.workunit.client.0.smithi100.stdout:op 3927 completed, throughput=5MB/sec 2024-04-07T16:51:40.724 INFO:tasks.workunit.client.0.smithi100.stdout: 759: throughput=5MB/sec pending data=0 2024-04-07T16:51:40.725 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-cFI7cauCqjgpFCz off=374755 len=1185802 2024-04-07T16:51:40.729 INFO:tasks.workunit.client.0.smithi100.stdout:op 3928 completed, throughput=5MB/sec 2024-04-07T16:51:40.730 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-CAHz-N3nRUxfmRQ off=14311539 len=1591396 2024-04-07T16:51:40.746 INFO:tasks.workunit.client.0.smithi100.stdout:op 3929 completed, throughput=5MB/sec 2024-04-07T16:51:40.746 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-6UU1-AL9w5fOgZ8 off=12390889 len=1747009 2024-04-07T16:51:40.750 INFO:tasks.workunit.client.0.smithi100.stdout:op 3930 completed, throughput=5MB/sec 2024-04-07T16:51:40.947 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:51:40 smithi100 bash[74942]: cluster 2024-04-07T16:51:39.472901+0000 mgr.y (mgr.25175) 1089 : cluster [DBG] pgmap v1731: 105 pgs: 105 active+clean; 126 GiB data, 8.6 GiB used, 707 GiB / 715 GiB avail; 2.3 MiB/s rd, 2.7 MiB/s wr, 4 op/s 2024-04-07T16:51:40.948 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:51:40 smithi100 bash[76202]: cluster 2024-04-07T16:51:39.472901+0000 mgr.y (mgr.25175) 1089 : cluster [DBG] pgmap v1731: 105 pgs: 105 active+clean; 126 GiB data, 8.6 GiB used, 707 GiB / 715 GiB avail; 2.3 MiB/s rd, 2.7 MiB/s wr, 4 op/s 2024-04-07T16:51:41.111 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:51:40 smithi144 bash[66654]: cluster 2024-04-07T16:51:39.472901+0000 mgr.y (mgr.25175) 1089 : cluster [DBG] pgmap v1731: 105 pgs: 105 active+clean; 126 GiB data, 8.6 GiB used, 707 GiB / 715 GiB avail; 2.3 MiB/s rd, 2.7 MiB/s wr, 4 op/s 2024-04-07T16:51:41.750 INFO:tasks.workunit.client.0.smithi100.stdout: 760: throughput=4.99MB/sec pending data=0 2024-04-07T16:51:41.751 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-gMax11acdUhG3pz off=7948173 len=1264152 2024-04-07T16:51:41.754 INFO:tasks.workunit.client.0.smithi100.stdout:op 3931 completed, throughput=5MB/sec 2024-04-07T16:51:41.754 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Cy4aEMoEA8-kTyq off=3027564 len=1137078 2024-04-07T16:51:41.764 INFO:tasks.workunit.client.0.smithi100.stdout:op 3932 completed, throughput=5MB/sec 2024-04-07T16:51:41.764 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-hBYZMPH9mzvj32f off=14049756 len=797562 2024-04-07T16:51:41.767 INFO:tasks.workunit.client.0.smithi100.stdout:op 3933 completed, throughput=5MB/sec 2024-04-07T16:51:41.767 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-bbY8cVbtwy4fb8u off=6954651 len=148335 2024-04-07T16:51:41.772 INFO:tasks.workunit.client.0.smithi100.stdout:op 3934 completed, throughput=5MB/sec 2024-04-07T16:51:41.772 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-tXs8HaGZAcqXmgV off=1409830 len=1451598 2024-04-07T16:51:41.775 INFO:tasks.workunit.client.0.smithi100.stdout:op 3935 completed, throughput=5MB/sec 2024-04-07T16:51:42.611 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:51:42 smithi144 bash[95233]: ts=2024-04-07T16:51:42.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:51:42.776 INFO:tasks.workunit.client.0.smithi100.stdout: 761: throughput=4.99MB/sec pending data=0 2024-04-07T16:51:42.776 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-qxRQS9mEMTRKhTA off=2340157 len=331343 2024-04-07T16:51:42.782 INFO:tasks.workunit.client.0.smithi100.stdout:op 3936 completed, throughput=4.99MB/sec 2024-04-07T16:51:42.782 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-zj4oQwU0ru__gAb off=5493913 len=126011 2024-04-07T16:51:42.783 INFO:tasks.workunit.client.0.smithi100.stdout:op 3937 completed, throughput=4.99MB/sec 2024-04-07T16:51:42.783 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-CmY5R0vQz5DW-Su off=5380402 len=154692 2024-04-07T16:51:42.785 INFO:tasks.workunit.client.0.smithi100.stdout:op 3938 completed, throughput=4.99MB/sec 2024-04-07T16:51:42.785 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-LO6fV9V73RYUcU6 off=5424144 len=1632042 2024-04-07T16:51:42.789 INFO:tasks.workunit.client.0.smithi100.stdout:op 3939 completed, throughput=5MB/sec 2024-04-07T16:51:42.789 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-XwGrCpmn9zPlRY0 off=4363998 len=1056354 2024-04-07T16:51:42.798 INFO:tasks.workunit.client.0.smithi100.stdout:op 3940 completed, throughput=5MB/sec 2024-04-07T16:51:42.798 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-E0TWOB-Hgwd4HV3 off=2684324 len=65554 2024-04-07T16:51:42.799 INFO:tasks.workunit.client.0.smithi100.stdout:op 3941 completed, throughput=5MB/sec 2024-04-07T16:51:42.799 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-G8ssepZWqG5_w4a off=8466430 len=1198856 2024-04-07T16:51:42.811 INFO:tasks.workunit.client.0.smithi100.stdout:op 3942 completed, throughput=5MB/sec 2024-04-07T16:51:42.811 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-oG6ZlJ-5DBXetZ0 off=113434 len=1620692 2024-04-07T16:51:42.813 INFO:tasks.workunit.client.0.smithi100.stdout:op 3943 completed, throughput=5MB/sec 2024-04-07T16:51:42.947 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:51:42 smithi100 bash[74942]: cluster 2024-04-07T16:51:41.473710+0000 mgr.y (mgr.25175) 1090 : cluster [DBG] pgmap v1732: 105 pgs: 105 active+clean; 126 GiB data, 8.6 GiB used, 707 GiB / 715 GiB avail; 2.2 MiB/s rd, 2.7 MiB/s wr, 4 op/s 2024-04-07T16:51:42.948 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:51:42 smithi100 bash[76202]: cluster 2024-04-07T16:51:41.473710+0000 mgr.y (mgr.25175) 1090 : cluster [DBG] pgmap v1732: 105 pgs: 105 active+clean; 126 GiB data, 8.6 GiB used, 707 GiB / 715 GiB avail; 2.2 MiB/s rd, 2.7 MiB/s wr, 4 op/s 2024-04-07T16:51:43.111 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:51:42 smithi144 bash[66654]: cluster 2024-04-07T16:51:41.473710+0000 mgr.y (mgr.25175) 1090 : cluster [DBG] pgmap v1732: 105 pgs: 105 active+clean; 126 GiB data, 8.6 GiB used, 707 GiB / 715 GiB avail; 2.2 MiB/s rd, 2.7 MiB/s wr, 4 op/s 2024-04-07T16:51:43.814 INFO:tasks.workunit.client.0.smithi100.stdout: 762: throughput=5MB/sec pending data=0 2024-04-07T16:51:43.814 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-3tWNeTK_CJoIKpe off=504842 len=1433116 2024-04-07T16:51:43.826 INFO:tasks.workunit.client.0.smithi100.stdout:op 3944 completed, throughput=5MB/sec 2024-04-07T16:51:43.826 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-GHX1B2kzebsr_PP off=1383553 len=906566 2024-04-07T16:51:43.835 INFO:tasks.workunit.client.0.smithi100.stdout:op 3945 completed, throughput=5MB/sec 2024-04-07T16:51:43.835 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-rJ0CK5vEscHhWMR off=3463200 len=32682 2024-04-07T16:51:43.836 INFO:tasks.workunit.client.0.smithi100.stdout:op 3946 completed, throughput=5MB/sec 2024-04-07T16:51:43.836 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-iBSFzyvqNAAsiM- off=2126933 len=630919 2024-04-07T16:51:43.838 INFO:tasks.workunit.client.0.smithi100.stdout:op 3947 completed, throughput=5MB/sec 2024-04-07T16:51:43.838 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-11LEXzsA6PxeAdQ off=9055191 len=2007523 2024-04-07T16:51:43.855 INFO:tasks.workunit.client.0.smithi100.stdout:op 3948 completed, throughput=5MB/sec 2024-04-07T16:51:44.855 INFO:tasks.workunit.client.0.smithi100.stdout: 763: throughput=4.99MB/sec pending data=0 2024-04-07T16:51:44.855 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-AGWLXQ5dQ6Cb3QD off=9927713 len=862434 2024-04-07T16:51:44.859 INFO:tasks.workunit.client.0.smithi100.stdout:op 3949 completed, throughput=5MB/sec 2024-04-07T16:51:44.859 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-zUAm9CUUDJc-uTG off=1359471 len=771221 2024-04-07T16:51:44.867 INFO:tasks.workunit.client.0.smithi100.stdout:op 3950 completed, throughput=5MB/sec 2024-04-07T16:51:44.868 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-1aox6AJ3bE9dXg3 off=10760810 len=854444 2024-04-07T16:51:44.870 INFO:tasks.workunit.client.0.smithi100.stdout:op 3951 completed, throughput=5MB/sec 2024-04-07T16:51:44.870 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-M0SUn4nmkUJbtZZ off=1231368 len=546645 2024-04-07T16:51:44.872 INFO:tasks.workunit.client.0.smithi100.stdout:op 3952 completed, throughput=5MB/sec 2024-04-07T16:51:44.872 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Kg8dgs52bHG5Go4 off=1130352 len=2026074 2024-04-07T16:51:44.892 INFO:tasks.workunit.client.0.smithi100.stdout:op 3953 completed, throughput=5MB/sec 2024-04-07T16:51:44.947 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:51:44 smithi100 bash[74942]: cluster 2024-04-07T16:51:43.474862+0000 mgr.y (mgr.25175) 1091 : cluster [DBG] pgmap v1733: 105 pgs: 105 active+clean; 126 GiB data, 8.6 GiB used, 707 GiB / 715 GiB avail; 3.0 MiB/s rd, 2.9 MiB/s wr, 5 op/s 2024-04-07T16:51:44.948 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:51:44 smithi100 bash[76202]: cluster 2024-04-07T16:51:43.474862+0000 mgr.y (mgr.25175) 1091 : cluster [DBG] pgmap v1733: 105 pgs: 105 active+clean; 126 GiB data, 8.6 GiB used, 707 GiB / 715 GiB avail; 3.0 MiB/s rd, 2.9 MiB/s wr, 5 op/s 2024-04-07T16:51:45.111 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:51:44 smithi144 bash[66654]: cluster 2024-04-07T16:51:43.474862+0000 mgr.y (mgr.25175) 1091 : cluster [DBG] pgmap v1733: 105 pgs: 105 active+clean; 126 GiB data, 8.6 GiB used, 707 GiB / 715 GiB avail; 3.0 MiB/s rd, 2.9 MiB/s wr, 5 op/s 2024-04-07T16:51:45.892 INFO:tasks.workunit.client.0.smithi100.stdout: 764: throughput=4.99MB/sec pending data=0 2024-04-07T16:51:45.892 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-TTyXjVLa2d_FTf7 off=14237543 len=69234 2024-04-07T16:51:45.893 INFO:tasks.workunit.client.0.smithi100.stdout:op 3954 completed, throughput=4.99MB/sec 2024-04-07T16:51:45.893 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-vutyu30eBuFGnnG off=3162616 len=618301 2024-04-07T16:51:45.901 INFO:tasks.workunit.client.0.smithi100.stdout:op 3955 completed, throughput=4.99MB/sec 2024-04-07T16:51:45.901 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Girh011TW0CsAFm off=815978 len=1944641 2024-04-07T16:51:45.906 INFO:tasks.workunit.client.0.smithi100.stdout:op 3956 completed, throughput=5MB/sec 2024-04-07T16:51:45.906 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-FYQ4sGXaOyKzYhx off=966476 len=1149105 2024-04-07T16:51:45.909 INFO:tasks.workunit.client.0.smithi100.stdout:op 3957 completed, throughput=5MB/sec 2024-04-07T16:51:45.909 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Fufe9GT0kYBOZar off=1253214 len=104995 2024-04-07T16:51:45.910 INFO:tasks.workunit.client.0.smithi100.stdout:op 3958 completed, throughput=5MB/sec 2024-04-07T16:51:45.910 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-zvKiKFFLuXJk1vw off=12166769 len=1277833 2024-04-07T16:51:45.921 INFO:tasks.workunit.client.0.smithi100.stdout:op 3959 completed, throughput=5MB/sec 2024-04-07T16:51:46.111 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:51:45 smithi144 bash[66654]: cluster 2024-04-07T16:51:45.475778+0000 mgr.y (mgr.25175) 1092 : cluster [DBG] pgmap v1734: 105 pgs: 105 active+clean; 126 GiB data, 8.6 GiB used, 707 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.4 MiB/s wr, 4 op/s 2024-04-07T16:51:46.197 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:51:45 smithi100 bash[74942]: cluster 2024-04-07T16:51:45.475778+0000 mgr.y (mgr.25175) 1092 : cluster [DBG] pgmap v1734: 105 pgs: 105 active+clean; 126 GiB data, 8.6 GiB used, 707 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.4 MiB/s wr, 4 op/s 2024-04-07T16:51:46.198 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:51:45 smithi100 bash[76202]: cluster 2024-04-07T16:51:45.475778+0000 mgr.y (mgr.25175) 1092 : cluster [DBG] pgmap v1734: 105 pgs: 105 active+clean; 126 GiB data, 8.6 GiB used, 707 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.4 MiB/s wr, 4 op/s 2024-04-07T16:51:46.921 INFO:tasks.workunit.client.0.smithi100.stdout: 765: throughput=4.99MB/sec pending data=0 2024-04-07T16:51:46.921 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-vTmTXEJd6o4E3dW off=10749557 len=504961 2024-04-07T16:51:46.924 INFO:tasks.workunit.client.0.smithi100.stdout:op 3960 completed, throughput=4.99MB/sec 2024-04-07T16:51:46.924 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-rZWMPVk_sUYwCJF off=3356820 len=1023248 2024-04-07T16:51:46.927 INFO:tasks.workunit.client.0.smithi100.stdout:op 3961 completed, throughput=5MB/sec 2024-04-07T16:51:46.927 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-W0HpEBW7vt4FE2Z off=7787676 len=229448 2024-04-07T16:51:46.929 INFO:tasks.workunit.client.0.smithi100.stdout:op 3962 completed, throughput=5MB/sec 2024-04-07T16:51:46.929 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-1r1NFwvKtpFs3G- off=3157640 len=316112 2024-04-07T16:51:46.934 INFO:tasks.workunit.client.0.smithi100.stdout:op 3963 completed, throughput=5MB/sec 2024-04-07T16:51:46.934 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-6B8nNgqu0e9Lxah off=10764791 len=1999449 2024-04-07T16:51:46.939 INFO:tasks.workunit.client.0.smithi100.stdout:op 3964 completed, throughput=5MB/sec 2024-04-07T16:51:46.939 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-qEd32INLkG8go9w off=739500 len=1389956 2024-04-07T16:51:46.951 INFO:tasks.workunit.client.0.smithi100.stdout:op 3965 completed, throughput=5MB/sec 2024-04-07T16:51:47.111 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:51:46 smithi144 bash[66654]: audit 2024-04-07T16:51:46.778372+0000 mon.a (mon.0) 2157 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:51:47.198 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:51:46 smithi100 bash[74942]: audit 2024-04-07T16:51:46.778372+0000 mon.a (mon.0) 2157 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:51:47.198 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:51:46 smithi100 bash[76202]: audit 2024-04-07T16:51:46.778372+0000 mon.a (mon.0) 2157 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:51:47.951 INFO:tasks.workunit.client.0.smithi100.stdout: 766: throughput=4.99MB/sec pending data=0 2024-04-07T16:51:47.951 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-i-xbF5P9rFrj7zB off=8769112 len=1652198 2024-04-07T16:51:47.956 INFO:tasks.workunit.client.0.smithi100.stdout:op 3966 completed, throughput=5MB/sec 2024-04-07T16:51:47.956 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-po15eS32evUyJ9O off=665834 len=1024483 2024-04-07T16:51:47.960 INFO:tasks.workunit.client.0.smithi100.stdout:op 3967 completed, throughput=5MB/sec 2024-04-07T16:51:47.960 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Ko3BxbiLZhNMl1_ off=7645681 len=907473 2024-04-07T16:51:47.968 INFO:tasks.workunit.client.0.smithi100.stdout:op 3968 completed, throughput=5MB/sec 2024-04-07T16:51:47.968 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-FiAKiezcxz5sQWB off=1877624 len=124774 2024-04-07T16:51:47.974 INFO:tasks.workunit.client.0.smithi100.stdout:op 3969 completed, throughput=5MB/sec 2024-04-07T16:51:47.974 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-IdHbc6J1S2Wrkh1 off=701602 len=1384697 2024-04-07T16:51:47.985 INFO:tasks.workunit.client.0.smithi100.stdout:op 3970 completed, throughput=5MB/sec 2024-04-07T16:51:47.985 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-PsFBYIr7BZL4y2a off=3745508 len=1264724 2024-04-07T16:51:47.995 INFO:tasks.workunit.client.0.smithi100.stdout:op 3971 completed, throughput=5MB/sec 2024-04-07T16:51:48.111 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:51:47 smithi144 bash[66654]: cluster 2024-04-07T16:51:47.476752+0000 mgr.y (mgr.25175) 1093 : cluster [DBG] pgmap v1735: 105 pgs: 105 active+clean; 126 GiB data, 8.6 GiB used, 707 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.7 MiB/s wr, 5 op/s 2024-04-07T16:51:48.197 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:51:47 smithi100 bash[74942]: cluster 2024-04-07T16:51:47.476752+0000 mgr.y (mgr.25175) 1093 : cluster [DBG] pgmap v1735: 105 pgs: 105 active+clean; 126 GiB data, 8.6 GiB used, 707 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.7 MiB/s wr, 5 op/s 2024-04-07T16:51:48.198 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:51:47 smithi100 bash[76202]: cluster 2024-04-07T16:51:47.476752+0000 mgr.y (mgr.25175) 1093 : cluster [DBG] pgmap v1735: 105 pgs: 105 active+clean; 126 GiB data, 8.6 GiB used, 707 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.7 MiB/s wr, 5 op/s 2024-04-07T16:51:48.697 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:51:48 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:51:48] "GET /metrics HTTP/1.1" 200 34065 "" "Prometheus/2.43.0" 2024-04-07T16:51:48.995 INFO:tasks.workunit.client.0.smithi100.stdout: 767: throughput=4.99MB/sec pending data=0 2024-04-07T16:51:48.996 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-_U_Vvrsrqiypiwa off=7631784 len=1085566 2024-04-07T16:51:48.998 INFO:tasks.workunit.client.0.smithi100.stdout:op 3972 completed, throughput=5MB/sec 2024-04-07T16:51:48.998 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-IS9hI3n4hpIoT1T off=14877958 len=1988861 2024-04-07T16:51:49.013 INFO:tasks.workunit.client.0.smithi100.stdout:op 3973 completed, throughput=5MB/sec 2024-04-07T16:51:49.013 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-XsqCV1HSgtkmtpj off=4325040 len=1541939 2024-04-07T16:51:49.027 INFO:tasks.workunit.client.0.smithi100.stdout:op 3974 completed, throughput=5MB/sec 2024-04-07T16:51:50.027 INFO:tasks.workunit.client.0.smithi100.stdout: 768: throughput=4.99MB/sec pending data=0 2024-04-07T16:51:50.027 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-ncRoaUwEaPCCrLE off=4728471 len=429848 2024-04-07T16:51:50.029 INFO:tasks.workunit.client.0.smithi100.stdout:op 3975 completed, throughput=4.99MB/sec 2024-04-07T16:51:50.029 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-0l_d0osvksng5DG off=18891333 len=312711 2024-04-07T16:51:50.030 INFO:tasks.workunit.client.0.smithi100.stdout:op 3976 completed, throughput=4.99MB/sec 2024-04-07T16:51:50.030 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-sS-Fbg1iuEqt9kH off=3656456 len=1790510 2024-04-07T16:51:50.034 INFO:tasks.workunit.client.0.smithi100.stdout:op 3977 completed, throughput=5MB/sec 2024-04-07T16:51:50.035 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-CbYdl9CbgRufup_ off=8535348 len=1469647 2024-04-07T16:51:50.045 INFO:tasks.workunit.client.0.smithi100.stdout:op 3978 completed, throughput=5MB/sec 2024-04-07T16:51:50.045 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-JkEfnLBBEgggNM7 off=295195 len=713277 2024-04-07T16:51:50.047 INFO:tasks.workunit.client.0.smithi100.stdout:op 3979 completed, throughput=5MB/sec 2024-04-07T16:51:50.047 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-lX_ouLQQkLgqJvI off=1714085 len=292789 2024-04-07T16:51:50.052 INFO:tasks.workunit.client.0.smithi100.stdout:op 3980 completed, throughput=5MB/sec 2024-04-07T16:51:50.053 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-z-4Pg-3NCAvHViy off=6906717 len=277099 2024-04-07T16:51:50.059 INFO:tasks.workunit.client.0.smithi100.stdout:op 3981 completed, throughput=5MB/sec 2024-04-07T16:51:50.861 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:51:50 smithi144 bash[66654]: cluster 2024-04-07T16:51:49.477661+0000 mgr.y (mgr.25175) 1094 : cluster [DBG] pgmap v1736: 105 pgs: 105 active+clean; 126 GiB data, 8.6 GiB used, 707 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.5 MiB/s wr, 5 op/s 2024-04-07T16:51:50.948 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:51:50 smithi100 bash[74942]: cluster 2024-04-07T16:51:49.477661+0000 mgr.y (mgr.25175) 1094 : cluster [DBG] pgmap v1736: 105 pgs: 105 active+clean; 126 GiB data, 8.6 GiB used, 707 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.5 MiB/s wr, 5 op/s 2024-04-07T16:51:50.948 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:51:50 smithi100 bash[76202]: cluster 2024-04-07T16:51:49.477661+0000 mgr.y (mgr.25175) 1094 : cluster [DBG] pgmap v1736: 105 pgs: 105 active+clean; 126 GiB data, 8.6 GiB used, 707 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.5 MiB/s wr, 5 op/s 2024-04-07T16:51:51.059 INFO:tasks.workunit.client.0.smithi100.stdout: 769: throughput=4.99MB/sec pending data=0 2024-04-07T16:51:51.059 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-pYMViyCHrlZMF8g off=5023922 len=641006 2024-04-07T16:51:51.066 INFO:tasks.workunit.client.0.smithi100.stdout:op 3982 completed, throughput=4.99MB/sec 2024-04-07T16:51:51.067 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-tXaoq0c-CHf2csd off=6626493 len=1019130 2024-04-07T16:51:51.069 INFO:tasks.workunit.client.0.smithi100.stdout:op 3983 completed, throughput=5MB/sec 2024-04-07T16:51:51.069 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-x5bvu-gIMtfV68C off=4902155 len=722168 2024-04-07T16:51:51.071 INFO:tasks.workunit.client.0.smithi100.stdout:op 3984 completed, throughput=5MB/sec 2024-04-07T16:51:51.071 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Yli5QWNbWrJKKDA off=2335824 len=146586 2024-04-07T16:51:51.077 INFO:tasks.workunit.client.0.smithi100.stdout:op 3985 completed, throughput=5MB/sec 2024-04-07T16:51:51.077 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-lUZqrhkKQumYrMX off=745240 len=1030340 2024-04-07T16:51:51.082 INFO:tasks.workunit.client.0.smithi100.stdout:op 3986 completed, throughput=5MB/sec 2024-04-07T16:51:51.082 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Rxt0g56PpSVZ3fg off=6302134 len=1843065 2024-04-07T16:51:51.095 INFO:tasks.workunit.client.0.smithi100.stdout:op 3987 completed, throughput=5MB/sec 2024-04-07T16:51:52.095 INFO:tasks.workunit.client.0.smithi100.stdout: 770: throughput=4.99MB/sec pending data=0 2024-04-07T16:51:52.095 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-VQKy4ldKqfn0FiZ off=1337097 len=343432 2024-04-07T16:51:52.097 INFO:tasks.workunit.client.0.smithi100.stdout:op 3988 completed, throughput=4.99MB/sec 2024-04-07T16:51:52.097 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-D9mKZftJFAFjamk off=674394 len=726366 2024-04-07T16:51:52.101 INFO:tasks.workunit.client.0.smithi100.stdout:op 3989 completed, throughput=5MB/sec 2024-04-07T16:51:52.102 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-KH4iUM_V5b66Tbg off=1190260 len=1124051 2024-04-07T16:51:52.104 INFO:tasks.workunit.client.0.smithi100.stdout:op 3990 completed, throughput=5MB/sec 2024-04-07T16:51:52.104 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-pbxWZIJ51Ezn5gP off=149504 len=1096831 2024-04-07T16:51:52.113 INFO:tasks.workunit.client.0.smithi100.stdout:op 3991 completed, throughput=5MB/sec 2024-04-07T16:51:52.114 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-trGV57QDlR1FF1- off=3939743 len=1696468 2024-04-07T16:51:52.128 INFO:tasks.workunit.client.0.smithi100.stdout:op 3992 completed, throughput=5MB/sec 2024-04-07T16:51:52.128 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-qGp9aNslvqgAE7T off=16410765 len=1002467 2024-04-07T16:51:52.129 INFO:tasks.workunit.client.0.smithi100.stdout:op 3993 completed, throughput=5MB/sec 2024-04-07T16:51:52.611 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:51:52 smithi144 bash[66654]: cluster 2024-04-07T16:51:51.478433+0000 mgr.y (mgr.25175) 1095 : cluster [DBG] pgmap v1737: 105 pgs: 105 active+clean; 126 GiB data, 8.6 GiB used, 707 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.1 MiB/s wr, 5 op/s 2024-04-07T16:51:52.612 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:51:52 smithi144 bash[95233]: ts=2024-04-07T16:51:52.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:51:52.948 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:51:52 smithi100 bash[74942]: cluster 2024-04-07T16:51:51.478433+0000 mgr.y (mgr.25175) 1095 : cluster [DBG] pgmap v1737: 105 pgs: 105 active+clean; 126 GiB data, 8.6 GiB used, 707 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.1 MiB/s wr, 5 op/s 2024-04-07T16:51:52.948 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:51:52 smithi100 bash[76202]: cluster 2024-04-07T16:51:51.478433+0000 mgr.y (mgr.25175) 1095 : cluster [DBG] pgmap v1737: 105 pgs: 105 active+clean; 126 GiB data, 8.6 GiB used, 707 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.1 MiB/s wr, 5 op/s 2024-04-07T16:51:53.130 INFO:tasks.workunit.client.0.smithi100.stdout: 771: throughput=4.99MB/sec pending data=0 2024-04-07T16:51:53.130 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-B3TgiZSnrD-M1KG off=19250342 len=1390039 2024-04-07T16:51:53.141 INFO:tasks.workunit.client.0.smithi100.stdout:op 3994 completed, throughput=5MB/sec 2024-04-07T16:51:53.141 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-1Ovjrlk632mW6QB off=692792 len=895198 2024-04-07T16:51:53.144 INFO:tasks.workunit.client.0.smithi100.stdout:op 3995 completed, throughput=5MB/sec 2024-04-07T16:51:53.144 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-_LloczacYMvieyI off=2570430 len=381826 2024-04-07T16:51:53.146 INFO:tasks.workunit.client.0.smithi100.stdout:op 3996 completed, throughput=5MB/sec 2024-04-07T16:51:53.146 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-7Fn-09RPOFrNyZl off=462789 len=1493385 2024-04-07T16:51:53.157 INFO:tasks.workunit.client.0.smithi100.stdout:op 3997 completed, throughput=5MB/sec 2024-04-07T16:51:53.158 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-yeasVyfSevQGsOI off=1911602 len=2024871 2024-04-07T16:51:53.162 INFO:tasks.workunit.client.0.smithi100.stdout:op 3998 completed, throughput=5MB/sec 2024-04-07T16:51:54.163 INFO:tasks.workunit.client.0.smithi100.stdout: 772: throughput=5MB/sec pending data=0 2024-04-07T16:51:54.163 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Aoz0N_2jpEuJfgQ off=2970476 len=23197 2024-04-07T16:51:54.163 INFO:tasks.workunit.client.0.smithi100.stdout:op 3999 completed, throughput=5MB/sec 2024-04-07T16:51:54.164 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-pHyqduYlPfCemGC off=1201921 len=1024760 2024-04-07T16:51:54.166 INFO:tasks.workunit.client.0.smithi100.stdout:op 4000 completed, throughput=5MB/sec 2024-04-07T16:51:54.167 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-1pKUcalWWf3nol5 off=3720372 len=998409 2024-04-07T16:51:54.175 INFO:tasks.workunit.client.0.smithi100.stdout:op 4001 completed, throughput=5MB/sec 2024-04-07T16:51:54.175 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-QnMxLjxsg7cRhvw off=14778808 len=1169555 2024-04-07T16:51:54.186 INFO:tasks.workunit.client.0.smithi100.stdout:op 4002 completed, throughput=5MB/sec 2024-04-07T16:51:54.186 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-bsWAHwf3futkGme off=23605656 len=523839 2024-04-07T16:51:54.188 INFO:tasks.workunit.client.0.smithi100.stdout:op 4003 completed, throughput=5MB/sec 2024-04-07T16:51:54.188 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-ohw9BiznhQthNGz off=3333117 len=939876 2024-04-07T16:51:54.196 INFO:tasks.workunit.client.0.smithi100.stdout:op 4004 completed, throughput=5MB/sec 2024-04-07T16:51:54.861 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:51:54 smithi144 bash[66654]: cluster 2024-04-07T16:51:53.479614+0000 mgr.y (mgr.25175) 1096 : cluster [DBG] pgmap v1738: 105 pgs: 105 active+clean; 126 GiB data, 8.7 GiB used, 707 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.4 MiB/s wr, 5 op/s 2024-04-07T16:51:54.948 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:51:54 smithi100 bash[74942]: cluster 2024-04-07T16:51:53.479614+0000 mgr.y (mgr.25175) 1096 : cluster [DBG] pgmap v1738: 105 pgs: 105 active+clean; 126 GiB data, 8.7 GiB used, 707 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.4 MiB/s wr, 5 op/s 2024-04-07T16:51:54.948 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:51:54 smithi100 bash[76202]: cluster 2024-04-07T16:51:53.479614+0000 mgr.y (mgr.25175) 1096 : cluster [DBG] pgmap v1738: 105 pgs: 105 active+clean; 126 GiB data, 8.7 GiB used, 707 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.4 MiB/s wr, 5 op/s 2024-04-07T16:51:55.197 INFO:tasks.workunit.client.0.smithi100.stdout: 773: throughput=4.99MB/sec pending data=0 2024-04-07T16:51:55.197 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Sib5PLnxFbMnsfJ off=9944831 len=923706 2024-04-07T16:51:55.200 INFO:tasks.workunit.client.0.smithi100.stdout:op 4005 completed, throughput=5MB/sec 2024-04-07T16:51:55.201 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-5H6fHkZkiQMFGCc off=890483 len=583060 2024-04-07T16:51:55.208 INFO:tasks.workunit.client.0.smithi100.stdout:op 4006 completed, throughput=5MB/sec 2024-04-07T16:51:55.208 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-z6aCm1HYK-LIpsx off=13675361 len=1021946 2024-04-07T16:51:55.217 INFO:tasks.workunit.client.0.smithi100.stdout:op 4007 completed, throughput=5MB/sec 2024-04-07T16:51:55.217 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-bT6TgZTNoBv1x8F off=3474826 len=1118840 2024-04-07T16:51:55.220 INFO:tasks.workunit.client.0.smithi100.stdout:op 4008 completed, throughput=5MB/sec 2024-04-07T16:51:55.220 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-KTWh_roBYMB9OPe off=1248055 len=1068225 2024-04-07T16:51:55.224 INFO:tasks.workunit.client.0.smithi100.stdout:op 4009 completed, throughput=5MB/sec 2024-04-07T16:51:56.224 INFO:tasks.workunit.client.0.smithi100.stdout: 774: throughput=4.99MB/sec pending data=0 2024-04-07T16:51:56.225 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj--W5c5g_1t2xgttO off=19372221 len=1289231 2024-04-07T16:51:56.229 INFO:tasks.workunit.client.0.smithi100.stdout:op 4010 completed, throughput=5MB/sec 2024-04-07T16:51:56.229 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-cdWZxzm84aA2TWF off=13473503 len=301611 2024-04-07T16:51:56.231 INFO:tasks.workunit.client.0.smithi100.stdout:op 4011 completed, throughput=5MB/sec 2024-04-07T16:51:56.231 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-7qKU7kotv87CZyb off=12147603 len=177398 2024-04-07T16:51:56.232 INFO:tasks.workunit.client.0.smithi100.stdout:op 4012 completed, throughput=5MB/sec 2024-04-07T16:51:56.232 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-pbjpyrkrDAAz4jg off=2474374 len=1505742 2024-04-07T16:51:56.244 INFO:tasks.workunit.client.0.smithi100.stdout:op 4013 completed, throughput=5MB/sec 2024-04-07T16:51:56.244 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-vMIYk-FCMuAnNTU off=11956443 len=1081720 2024-04-07T16:51:56.254 INFO:tasks.workunit.client.0.smithi100.stdout:op 4014 completed, throughput=5MB/sec 2024-04-07T16:51:56.254 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-oXCirmg_4ELLBjk off=10836036 len=1081065 2024-04-07T16:51:56.256 INFO:tasks.workunit.client.0.smithi100.stdout:op 4015 completed, throughput=5MB/sec 2024-04-07T16:51:56.861 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:51:56 smithi144 bash[66654]: cluster 2024-04-07T16:51:55.480586+0000 mgr.y (mgr.25175) 1097 : cluster [DBG] pgmap v1739: 105 pgs: 105 active+clean; 126 GiB data, 8.7 GiB used, 707 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.4 MiB/s wr, 5 op/s 2024-04-07T16:51:56.948 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:51:56 smithi100 bash[74942]: cluster 2024-04-07T16:51:55.480586+0000 mgr.y (mgr.25175) 1097 : cluster [DBG] pgmap v1739: 105 pgs: 105 active+clean; 126 GiB data, 8.7 GiB used, 707 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.4 MiB/s wr, 5 op/s 2024-04-07T16:51:56.948 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:51:56 smithi100 bash[76202]: cluster 2024-04-07T16:51:55.480586+0000 mgr.y (mgr.25175) 1097 : cluster [DBG] pgmap v1739: 105 pgs: 105 active+clean; 126 GiB data, 8.7 GiB used, 707 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.4 MiB/s wr, 5 op/s 2024-04-07T16:51:57.256 INFO:tasks.workunit.client.0.smithi100.stdout: 775: throughput=4.99MB/sec pending data=0 2024-04-07T16:51:57.257 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-wY0OJqvOqe-xySl off=815116 len=2084840 2024-04-07T16:51:57.272 INFO:tasks.workunit.client.0.smithi100.stdout:op 4016 completed, throughput=5MB/sec 2024-04-07T16:51:57.272 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-YF3GTVmLyqQ9cde off=1124191 len=842183 2024-04-07T16:51:57.274 INFO:tasks.workunit.client.0.smithi100.stdout:op 4017 completed, throughput=5MB/sec 2024-04-07T16:51:57.274 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-QsAGU7AW2t_vulg off=3734930 len=795036 2024-04-07T16:51:57.283 INFO:tasks.workunit.client.0.smithi100.stdout:op 4018 completed, throughput=5MB/sec 2024-04-07T16:51:57.283 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-z_zryLMoslvU4tG off=1390679 len=659413 2024-04-07T16:51:57.285 INFO:tasks.workunit.client.0.smithi100.stdout:op 4019 completed, throughput=5MB/sec 2024-04-07T16:51:57.285 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-rZSAegLUEoyoVYn off=6863677 len=1642418 2024-04-07T16:51:57.289 INFO:tasks.workunit.client.0.smithi100.stdout:op 4020 completed, throughput=5MB/sec 2024-04-07T16:51:58.289 INFO:tasks.workunit.client.0.smithi100.stdout: 776: throughput=4.99MB/sec pending data=0 2024-04-07T16:51:58.289 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-RoLQaGdmwiG-e8q off=15872686 len=129645 2024-04-07T16:51:58.291 INFO:tasks.workunit.client.0.smithi100.stdout:op 4021 completed, throughput=4.99MB/sec 2024-04-07T16:51:58.291 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-A_XFzntqKPYhrpZ off=811616 len=1446844 2024-04-07T16:51:58.295 INFO:tasks.workunit.client.0.smithi100.stdout:op 4022 completed, throughput=5MB/sec 2024-04-07T16:51:58.295 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-xqMrzLl4jQKTlUH off=3253579 len=1092781 2024-04-07T16:51:58.298 INFO:tasks.workunit.client.0.smithi100.stdout:op 4023 completed, throughput=5MB/sec 2024-04-07T16:51:58.298 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-VkcUcoMRWfL7cOm off=6710690 len=1096198 2024-04-07T16:51:58.308 INFO:tasks.workunit.client.0.smithi100.stdout:op 4024 completed, throughput=5MB/sec 2024-04-07T16:51:58.308 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-2xx3Bfjat5i5BED off=12278347 len=1586171 2024-04-07T16:51:58.313 INFO:tasks.workunit.client.0.smithi100.stdout:op 4025 completed, throughput=5MB/sec 2024-04-07T16:51:58.698 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:51:58 smithi100 bash[74942]: cluster 2024-04-07T16:51:57.481633+0000 mgr.y (mgr.25175) 1098 : cluster [DBG] pgmap v1740: 105 pgs: 105 active+clean; 126 GiB data, 8.7 GiB used, 707 GiB / 715 GiB avail; 2.4 MiB/s rd, 2.6 MiB/s wr, 5 op/s 2024-04-07T16:51:58.698 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:51:58 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:51:58] "GET /metrics HTTP/1.1" 200 34065 "" "Prometheus/2.43.0" 2024-04-07T16:51:58.698 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:51:58 smithi100 bash[76202]: cluster 2024-04-07T16:51:57.481633+0000 mgr.y (mgr.25175) 1098 : cluster [DBG] pgmap v1740: 105 pgs: 105 active+clean; 126 GiB data, 8.7 GiB used, 707 GiB / 715 GiB avail; 2.4 MiB/s rd, 2.6 MiB/s wr, 5 op/s 2024-04-07T16:51:58.862 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:51:58 smithi144 bash[66654]: cluster 2024-04-07T16:51:57.481633+0000 mgr.y (mgr.25175) 1098 : cluster [DBG] pgmap v1740: 105 pgs: 105 active+clean; 126 GiB data, 8.7 GiB used, 707 GiB / 715 GiB avail; 2.4 MiB/s rd, 2.6 MiB/s wr, 5 op/s 2024-04-07T16:51:59.313 INFO:tasks.workunit.client.0.smithi100.stdout: 777: throughput=4.99MB/sec pending data=0 2024-04-07T16:51:59.313 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-jeTVEvahSDp_H9c off=181425 len=1012480 2024-04-07T16:51:59.323 INFO:tasks.workunit.client.0.smithi100.stdout:op 4026 completed, throughput=5MB/sec 2024-04-07T16:51:59.323 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-jLjG9h9TdkZChBe off=7660245 len=1688707 2024-04-07T16:51:59.327 INFO:tasks.workunit.client.0.smithi100.stdout:op 4027 completed, throughput=5MB/sec 2024-04-07T16:51:59.327 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Ge2w08YMY8UH1Vb off=675449 len=205043 2024-04-07T16:51:59.329 INFO:tasks.workunit.client.0.smithi100.stdout:op 4028 completed, throughput=5MB/sec 2024-04-07T16:51:59.329 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-qPbJdzJgIvkSG4g off=5195770 len=1683334 2024-04-07T16:51:59.333 INFO:tasks.workunit.client.0.smithi100.stdout:op 4029 completed, throughput=5MB/sec 2024-04-07T16:51:59.333 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-ih6H1uzbexZQpd5 off=3567930 len=1300123 2024-04-07T16:51:59.343 INFO:tasks.workunit.client.0.smithi100.stdout:op 4030 completed, throughput=5MB/sec 2024-04-07T16:52:00.344 INFO:tasks.workunit.client.0.smithi100.stdout: 778: throughput=5MB/sec pending data=0 2024-04-07T16:52:00.344 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-zR8FOu9EzzBNkAp off=6762575 len=348841 2024-04-07T16:52:00.350 INFO:tasks.workunit.client.0.smithi100.stdout:op 4031 completed, throughput=5MB/sec 2024-04-07T16:52:00.350 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-6AOP57_4qj5ajL1 off=7867322 len=1096078 2024-04-07T16:52:00.353 INFO:tasks.workunit.client.0.smithi100.stdout:op 4032 completed, throughput=5MB/sec 2024-04-07T16:52:00.353 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-WZPbItNs5KSJkIa off=8416820 len=2066666 2024-04-07T16:52:00.369 INFO:tasks.workunit.client.0.smithi100.stdout:op 4033 completed, throughput=5MB/sec 2024-04-07T16:52:00.369 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-KojEf5AnWV4gnQq off=10733926 len=606412 2024-04-07T16:52:00.372 INFO:tasks.workunit.client.0.smithi100.stdout:op 4034 completed, throughput=5MB/sec 2024-04-07T16:52:00.372 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-SUSjciicqaxDebi off=4732245 len=619504 2024-04-07T16:52:00.374 INFO:tasks.workunit.client.0.smithi100.stdout:op 4035 completed, throughput=5MB/sec 2024-04-07T16:52:00.861 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:52:00 smithi144 bash[66654]: cluster 2024-04-07T16:51:59.482676+0000 mgr.y (mgr.25175) 1099 : cluster [DBG] pgmap v1741: 105 pgs: 105 active+clean; 126 GiB data, 8.7 GiB used, 707 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.0 MiB/s wr, 4 op/s 2024-04-07T16:52:00.948 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:52:00 smithi100 bash[74942]: cluster 2024-04-07T16:51:59.482676+0000 mgr.y (mgr.25175) 1099 : cluster [DBG] pgmap v1741: 105 pgs: 105 active+clean; 126 GiB data, 8.7 GiB used, 707 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.0 MiB/s wr, 4 op/s 2024-04-07T16:52:00.948 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:52:00 smithi100 bash[76202]: cluster 2024-04-07T16:51:59.482676+0000 mgr.y (mgr.25175) 1099 : cluster [DBG] pgmap v1741: 105 pgs: 105 active+clean; 126 GiB data, 8.7 GiB used, 707 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.0 MiB/s wr, 4 op/s 2024-04-07T16:52:01.374 INFO:tasks.workunit.client.0.smithi100.stdout: 779: throughput=4.99MB/sec pending data=0 2024-04-07T16:52:01.375 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-NN1xtWjYbpKfYcS off=2971881 len=537466 2024-04-07T16:52:01.376 INFO:tasks.workunit.client.0.smithi100.stdout:op 4036 completed, throughput=4.99MB/sec 2024-04-07T16:52:01.376 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-7yAXXeeEP2XYovB off=1318700 len=979222 2024-04-07T16:52:01.379 INFO:tasks.workunit.client.0.smithi100.stdout:op 4037 completed, throughput=5MB/sec 2024-04-07T16:52:01.380 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-2kEl-2GGdEZqQ7f off=503558 len=1642488 2024-04-07T16:52:01.392 INFO:tasks.workunit.client.0.smithi100.stdout:op 4038 completed, throughput=5MB/sec 2024-04-07T16:52:01.392 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-CcEOCvfbdDJQTiO off=1113213 len=678009 2024-04-07T16:52:01.399 INFO:tasks.workunit.client.0.smithi100.stdout:op 4039 completed, throughput=5MB/sec 2024-04-07T16:52:01.399 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Rw_9fNdpcGcctNF off=11686293 len=1916063 2024-04-07T16:52:01.406 INFO:tasks.workunit.client.0.smithi100.stdout:op 4040 completed, throughput=5MB/sec 2024-04-07T16:52:02.406 INFO:tasks.workunit.client.0.smithi100.stdout: 780: throughput=4.99MB/sec pending data=0 2024-04-07T16:52:02.406 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-6UU1-AL9w5fOgZ8 off=382141 len=1505852 2024-04-07T16:52:02.419 INFO:tasks.workunit.client.0.smithi100.stdout:op 4041 completed, throughput=5MB/sec 2024-04-07T16:52:02.419 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-IrAVBQUntQ_W_0k off=2465004 len=1675067 2024-04-07T16:52:02.423 INFO:tasks.workunit.client.0.smithi100.stdout:op 4042 completed, throughput=5MB/sec 2024-04-07T16:52:02.423 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Fk3iT9xo1B4D4xy off=48101 len=275911 2024-04-07T16:52:02.428 INFO:tasks.workunit.client.0.smithi100.stdout:op 4043 completed, throughput=5MB/sec 2024-04-07T16:52:02.429 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-F3SJEowXRFG3GaT off=8803598 len=1371535 2024-04-07T16:52:02.439 INFO:tasks.workunit.client.0.smithi100.stdout:op 4044 completed, throughput=5MB/sec 2024-04-07T16:52:02.612 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:52:02 smithi144 bash[66654]: cluster 2024-04-07T16:52:01.483572+0000 mgr.y (mgr.25175) 1100 : cluster [DBG] pgmap v1742: 105 pgs: 105 active+clean; 126 GiB data, 8.7 GiB used, 707 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.0 MiB/s wr, 4 op/s 2024-04-07T16:52:02.612 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:52:02 smithi144 bash[66654]: audit 2024-04-07T16:52:01.778807+0000 mon.a (mon.0) 2158 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:52:02.612 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:52:02 smithi144 bash[95233]: ts=2024-04-07T16:52:02.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:52:02.948 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:52:02 smithi100 bash[74942]: cluster 2024-04-07T16:52:01.483572+0000 mgr.y (mgr.25175) 1100 : cluster [DBG] pgmap v1742: 105 pgs: 105 active+clean; 126 GiB data, 8.7 GiB used, 707 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.0 MiB/s wr, 4 op/s 2024-04-07T16:52:02.948 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:52:02 smithi100 bash[74942]: audit 2024-04-07T16:52:01.778807+0000 mon.a (mon.0) 2158 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:52:02.948 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:52:02 smithi100 bash[76202]: cluster 2024-04-07T16:52:01.483572+0000 mgr.y (mgr.25175) 1100 : cluster [DBG] pgmap v1742: 105 pgs: 105 active+clean; 126 GiB data, 8.7 GiB used, 707 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.0 MiB/s wr, 4 op/s 2024-04-07T16:52:02.948 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:52:02 smithi100 bash[76202]: audit 2024-04-07T16:52:01.778807+0000 mon.a (mon.0) 2158 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:52:03.439 INFO:tasks.workunit.client.0.smithi100.stdout: 781: throughput=4.99MB/sec pending data=0 2024-04-07T16:52:03.439 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-IHOCxulDfw4iiCH off=1294803 len=1000779 2024-04-07T16:52:03.442 INFO:tasks.workunit.client.0.smithi100.stdout:op 4045 completed, throughput=5MB/sec 2024-04-07T16:52:03.442 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-b5Y-pVST0TIyGOV off=3643915 len=931150 2024-04-07T16:52:03.446 INFO:tasks.workunit.client.0.smithi100.stdout:op 4046 completed, throughput=5MB/sec 2024-04-07T16:52:03.446 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-1OUYmNnIpyzO7V7 off=2391363 len=170176 2024-04-07T16:52:03.448 INFO:tasks.workunit.client.0.smithi100.stdout:op 4047 completed, throughput=5MB/sec 2024-04-07T16:52:03.448 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-o_ZkWu2HKemOqvC off=4430139 len=167599 2024-04-07T16:52:03.453 INFO:tasks.workunit.client.0.smithi100.stdout:op 4048 completed, throughput=5MB/sec 2024-04-07T16:52:03.453 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-pAaVqWGEsRtBEa1 off=2485292 len=284253 2024-04-07T16:52:03.455 INFO:tasks.workunit.client.0.smithi100.stdout:op 4049 completed, throughput=5MB/sec 2024-04-07T16:52:03.455 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-X__Vr6QC2sdSUTA off=1030583 len=790510 2024-04-07T16:52:03.463 INFO:tasks.workunit.client.0.smithi100.stdout:op 4050 completed, throughput=5MB/sec 2024-04-07T16:52:03.464 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-KX6H5ntPwpCE5nz off=3689280 len=35789 2024-04-07T16:52:03.469 INFO:tasks.workunit.client.0.smithi100.stdout:op 4051 completed, throughput=5MB/sec 2024-04-07T16:52:03.469 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-bVWIxZj-YOc79zx off=6761032 len=389144 2024-04-07T16:52:03.470 INFO:tasks.workunit.client.0.smithi100.stdout:op 4052 completed, throughput=5MB/sec 2024-04-07T16:52:03.470 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-BogfboiFG236tHh off=9229710 len=1010555 2024-04-07T16:52:03.473 INFO:tasks.workunit.client.0.smithi100.stdout:op 4053 completed, throughput=5MB/sec 2024-04-07T16:52:03.473 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-G8ssepZWqG5_w4a off=5109350 len=308528 2024-04-07T16:52:03.479 INFO:tasks.workunit.client.0.smithi100.stdout:op 4054 completed, throughput=5MB/sec 2024-04-07T16:52:03.479 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-wCPTwjbkjCpJWxi off=17954155 len=1707109 2024-04-07T16:52:03.484 INFO:tasks.workunit.client.0.smithi100.stdout:op 4055 completed, throughput=5MB/sec 2024-04-07T16:52:04.484 INFO:tasks.workunit.client.0.smithi100.stdout: 782: throughput=5MB/sec pending data=0 2024-04-07T16:52:04.484 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-WXOtBM3a6Y4EYNi off=6926450 len=523928 2024-04-07T16:52:04.491 INFO:tasks.workunit.client.0.smithi100.stdout:op 4056 completed, throughput=5MB/sec 2024-04-07T16:52:04.491 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-telGACn0c6owFK- off=20501078 len=1503836 2024-04-07T16:52:04.503 INFO:tasks.workunit.client.0.smithi100.stdout:op 4057 completed, throughput=5MB/sec 2024-04-07T16:52:04.503 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-OhJsvUoVdZ4Ktpu off=924922 len=1084741 2024-04-07T16:52:04.512 INFO:tasks.workunit.client.0.smithi100.stdout:op 4058 completed, throughput=5MB/sec 2024-04-07T16:52:04.512 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-LBX1BGrPXNQmZ43 off=2067759 len=1623534 2024-04-07T16:52:04.524 INFO:tasks.workunit.client.0.smithi100.stdout:op 4059 completed, throughput=5MB/sec 2024-04-07T16:52:04.861 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:52:04 smithi144 bash[66654]: cluster 2024-04-07T16:52:03.484659+0000 mgr.y (mgr.25175) 1101 : cluster [DBG] pgmap v1743: 105 pgs: 105 active+clean; 126 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail; 3.0 MiB/s rd, 2.6 MiB/s wr, 5 op/s 2024-04-07T16:52:04.947 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:52:04 smithi100 bash[74942]: cluster 2024-04-07T16:52:03.484659+0000 mgr.y (mgr.25175) 1101 : cluster [DBG] pgmap v1743: 105 pgs: 105 active+clean; 126 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail; 3.0 MiB/s rd, 2.6 MiB/s wr, 5 op/s 2024-04-07T16:52:04.948 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:52:04 smithi100 bash[76202]: cluster 2024-04-07T16:52:03.484659+0000 mgr.y (mgr.25175) 1101 : cluster [DBG] pgmap v1743: 105 pgs: 105 active+clean; 126 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail; 3.0 MiB/s rd, 2.6 MiB/s wr, 5 op/s 2024-04-07T16:52:05.524 INFO:tasks.workunit.client.0.smithi100.stdout: 783: throughput=4.99MB/sec pending data=0 2024-04-07T16:52:05.524 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj--I3uTGbA9PzYn84 off=5900897 len=1330194 2024-04-07T16:52:05.526 INFO:tasks.workunit.client.0.smithi100.stdout:op 4060 completed, throughput=5MB/sec 2024-04-07T16:52:05.527 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-oKYZuCVsCqm9dTs off=8629205 len=1103371 2024-04-07T16:52:05.535 INFO:tasks.workunit.client.0.smithi100.stdout:op 4061 completed, throughput=5MB/sec 2024-04-07T16:52:05.535 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj--mWjGryD5IAVNDI off=2799573 len=1268998 2024-04-07T16:52:05.546 INFO:tasks.workunit.client.0.smithi100.stdout:op 4062 completed, throughput=5MB/sec 2024-04-07T16:52:05.546 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-ZHKc7M5tt7BojI0 off=2637399 len=1946819 2024-04-07T16:52:05.550 INFO:tasks.workunit.client.0.smithi100.stdout:op 4063 completed, throughput=5MB/sec 2024-04-07T16:52:05.861 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:52:05 smithi144 bash[66654]: audit 2024-04-07T16:52:05.259063+0000 mon.a (mon.0) 2159 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:52:05.948 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:52:05 smithi100 bash[76202]: audit 2024-04-07T16:52:05.259063+0000 mon.a (mon.0) 2159 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:52:05.948 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:52:05 smithi100 bash[74942]: audit 2024-04-07T16:52:05.259063+0000 mon.a (mon.0) 2159 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:52:06.551 INFO:tasks.workunit.client.0.smithi100.stdout: 784: throughput=5MB/sec pending data=0 2024-04-07T16:52:06.551 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-SLM8c3u7rjm53J_ off=13685335 len=586312 2024-04-07T16:52:06.553 INFO:tasks.workunit.client.0.smithi100.stdout:op 4064 completed, throughput=5MB/sec 2024-04-07T16:52:06.553 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-bWddQWFVNl3EnAs off=2971431 len=673391 2024-04-07T16:52:06.561 INFO:tasks.workunit.client.0.smithi100.stdout:op 4065 completed, throughput=5MB/sec 2024-04-07T16:52:06.561 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-IJ3DBB4p8b7XCLB off=18739211 len=1090825 2024-04-07T16:52:06.564 INFO:tasks.workunit.client.0.smithi100.stdout:op 4066 completed, throughput=5MB/sec 2024-04-07T16:52:06.564 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-iGf2gBQicUZSqd1 off=1433284 len=1403434 2024-04-07T16:52:06.576 INFO:tasks.workunit.client.0.smithi100.stdout:op 4067 completed, throughput=5MB/sec 2024-04-07T16:52:06.576 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-IrpTHDHVTSqUlRH off=1620562 len=376535 2024-04-07T16:52:06.582 INFO:tasks.workunit.client.0.smithi100.stdout:op 4068 completed, throughput=5MB/sec 2024-04-07T16:52:06.583 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-lhySdP5oe8Ze0eS off=408968 len=242849 2024-04-07T16:52:06.584 INFO:tasks.workunit.client.0.smithi100.stdout:op 4069 completed, throughput=5MB/sec 2024-04-07T16:52:06.862 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:52:06 smithi144 bash[66654]: cluster 2024-04-07T16:52:05.485597+0000 mgr.y (mgr.25175) 1102 : cluster [DBG] pgmap v1744: 105 pgs: 105 active+clean; 126 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail; 3.1 MiB/s rd, 2.4 MiB/s wr, 5 op/s 2024-04-07T16:52:06.862 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:52:06 smithi144 bash[66654]: audit 2024-04-07T16:52:06.050435+0000 mon.a (mon.0) 2160 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:52:06.862 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:52:06 smithi144 bash[66654]: audit 2024-04-07T16:52:06.052440+0000 mon.a (mon.0) 2161 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:52:06.862 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:52:06 smithi144 bash[66654]: audit 2024-04-07T16:52:06.062936+0000 mon.a (mon.0) 2162 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:52:06.948 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:52:06 smithi100 bash[74942]: cluster 2024-04-07T16:52:05.485597+0000 mgr.y (mgr.25175) 1102 : cluster [DBG] pgmap v1744: 105 pgs: 105 active+clean; 126 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail; 3.1 MiB/s rd, 2.4 MiB/s wr, 5 op/s 2024-04-07T16:52:06.948 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:52:06 smithi100 bash[74942]: audit 2024-04-07T16:52:06.050435+0000 mon.a (mon.0) 2160 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:52:06.948 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:52:06 smithi100 bash[74942]: audit 2024-04-07T16:52:06.052440+0000 mon.a (mon.0) 2161 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:52:06.948 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:52:06 smithi100 bash[74942]: audit 2024-04-07T16:52:06.062936+0000 mon.a (mon.0) 2162 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:52:06.948 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:52:06 smithi100 bash[76202]: cluster 2024-04-07T16:52:05.485597+0000 mgr.y (mgr.25175) 1102 : cluster [DBG] pgmap v1744: 105 pgs: 105 active+clean; 126 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail; 3.1 MiB/s rd, 2.4 MiB/s wr, 5 op/s 2024-04-07T16:52:06.948 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:52:06 smithi100 bash[76202]: audit 2024-04-07T16:52:06.050435+0000 mon.a (mon.0) 2160 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:52:06.948 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:52:06 smithi100 bash[76202]: audit 2024-04-07T16:52:06.052440+0000 mon.a (mon.0) 2161 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:52:06.948 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:52:06 smithi100 bash[76202]: audit 2024-04-07T16:52:06.062936+0000 mon.a (mon.0) 2162 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:52:07.584 INFO:tasks.workunit.client.0.smithi100.stdout: 785: throughput=4.99MB/sec pending data=0 2024-04-07T16:52:07.585 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-cVcC-Xx3Vz447yp off=110629 len=1009735 2024-04-07T16:52:07.589 INFO:tasks.workunit.client.0.smithi100.stdout:op 4070 completed, throughput=5MB/sec 2024-04-07T16:52:07.589 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-8b9lnN5W_FjwGsA off=4346576 len=184805 2024-04-07T16:52:07.593 INFO:tasks.workunit.client.0.smithi100.stdout:op 4071 completed, throughput=5MB/sec 2024-04-07T16:52:07.593 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-4EHsUhRbJfCtUW9 off=232058 len=1329791 2024-04-07T16:52:07.597 INFO:tasks.workunit.client.0.smithi100.stdout:op 4072 completed, throughput=5MB/sec 2024-04-07T16:52:07.597 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-UEhhuF7j2ClOeuU off=18675648 len=1073971 2024-04-07T16:52:07.600 INFO:tasks.workunit.client.0.smithi100.stdout:op 4073 completed, throughput=5MB/sec 2024-04-07T16:52:07.600 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-tLNQYETV9X5AEzx off=5859437 len=406719 2024-04-07T16:52:07.602 INFO:tasks.workunit.client.0.smithi100.stdout:op 4074 completed, throughput=5MB/sec 2024-04-07T16:52:07.602 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-4-3SMGuMzFdNiQN off=4997826 len=1252361 2024-04-07T16:52:07.605 INFO:tasks.workunit.client.0.smithi100.stdout:op 4075 completed, throughput=5MB/sec 2024-04-07T16:52:07.605 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Kc4m4dIgm0RtQPr off=7161673 len=393760 2024-04-07T16:52:07.607 INFO:tasks.workunit.client.0.smithi100.stdout:op 4076 completed, throughput=5MB/sec 2024-04-07T16:52:08.607 INFO:tasks.workunit.client.0.smithi100.stdout: 786: throughput=4.99MB/sec pending data=0 2024-04-07T16:52:08.608 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-WZPbItNs5KSJkIa off=12078748 len=1364564 2024-04-07T16:52:08.619 INFO:tasks.workunit.client.0.smithi100.stdout:op 4077 completed, throughput=5MB/sec 2024-04-07T16:52:08.619 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-CfJfWaKhF5H55bQ off=5237274 len=220296 2024-04-07T16:52:08.624 INFO:tasks.workunit.client.0.smithi100.stdout:op 4078 completed, throughput=5MB/sec 2024-04-07T16:52:08.624 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-JNMsJKWdDWHnpBa off=1895000 len=69311 2024-04-07T16:52:08.625 INFO:tasks.workunit.client.0.smithi100.stdout:op 4079 completed, throughput=5MB/sec 2024-04-07T16:52:08.625 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-LpQ4hWcoj25or61 off=6573233 len=108367 2024-04-07T16:52:08.626 INFO:tasks.workunit.client.0.smithi100.stdout:op 4080 completed, throughput=5MB/sec 2024-04-07T16:52:08.626 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-aIPGBSe1gNt6S2e off=3335316 len=1488405 2024-04-07T16:52:08.639 INFO:tasks.workunit.client.0.smithi100.stdout:op 4081 completed, throughput=5MB/sec 2024-04-07T16:52:08.639 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-rxq817ZPU6-PAQm off=3459473 len=2067100 2024-04-07T16:52:08.644 INFO:tasks.workunit.client.0.smithi100.stdout:op 4082 completed, throughput=5MB/sec 2024-04-07T16:52:08.698 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:52:08 smithi100 bash[74942]: cluster 2024-04-07T16:52:07.486622+0000 mgr.y (mgr.25175) 1103 : cluster [DBG] pgmap v1745: 105 pgs: 105 active+clean; 126 GiB data, 8.8 GiB used, 706 GiB / 715 GiB avail; 2.8 MiB/s rd, 2.2 MiB/s wr, 4 op/s 2024-04-07T16:52:08.698 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:52:08 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:52:08] "GET /metrics HTTP/1.1" 200 34064 "" "Prometheus/2.43.0" 2024-04-07T16:52:08.698 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:52:08 smithi100 bash[76202]: cluster 2024-04-07T16:52:07.486622+0000 mgr.y (mgr.25175) 1103 : cluster [DBG] pgmap v1745: 105 pgs: 105 active+clean; 126 GiB data, 8.8 GiB used, 706 GiB / 715 GiB avail; 2.8 MiB/s rd, 2.2 MiB/s wr, 4 op/s 2024-04-07T16:52:08.861 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:52:08 smithi144 bash[66654]: cluster 2024-04-07T16:52:07.486622+0000 mgr.y (mgr.25175) 1103 : cluster [DBG] pgmap v1745: 105 pgs: 105 active+clean; 126 GiB data, 8.8 GiB used, 706 GiB / 715 GiB avail; 2.8 MiB/s rd, 2.2 MiB/s wr, 4 op/s 2024-04-07T16:52:09.644 INFO:tasks.workunit.client.0.smithi100.stdout: 787: throughput=4.99MB/sec pending data=0 2024-04-07T16:52:09.644 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-ga3J-o5JknckE6x off=1639485 len=1766273 2024-04-07T16:52:09.658 INFO:tasks.workunit.client.0.smithi100.stdout:op 4083 completed, throughput=5MB/sec 2024-04-07T16:52:09.658 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-0gliZjf5hddKNXi off=16821356 len=1487220 2024-04-07T16:52:09.671 INFO:tasks.workunit.client.0.smithi100.stdout:op 4084 completed, throughput=5MB/sec 2024-04-07T16:52:09.671 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-iam2-e-JZw0BZ4X off=6892751 len=1607123 2024-04-07T16:52:09.679 INFO:tasks.workunit.client.0.smithi100.stdout:op 4085 completed, throughput=5MB/sec 2024-04-07T16:52:09.679 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-7JF-OzcIIGrKuqq off=13872885 len=574109 2024-04-07T16:52:09.686 INFO:tasks.workunit.client.0.smithi100.stdout:op 4086 completed, throughput=5MB/sec 2024-04-07T16:52:10.686 INFO:tasks.workunit.client.0.smithi100.stdout: 788: throughput=4.99MB/sec pending data=0 2024-04-07T16:52:10.686 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-8GhjgOjI_Q4mmhi off=3212786 len=121008 2024-04-07T16:52:10.691 INFO:tasks.workunit.client.0.smithi100.stdout:op 4087 completed, throughput=4.99MB/sec 2024-04-07T16:52:10.691 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-MR7g8M-0iYixe2q off=0 len=1177158 2024-04-07T16:52:10.696 INFO:tasks.workunit.client.0.smithi100.stdout:op 4088 completed, throughput=5MB/sec 2024-04-07T16:52:10.696 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-dvPhSe8NXpqejCY off=2371653 len=332573 2024-04-07T16:52:10.702 INFO:tasks.workunit.client.0.smithi100.stdout:op 4089 completed, throughput=5MB/sec 2024-04-07T16:52:10.702 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-vTYoYpuDT5mVCQb off=18390541 len=616151 2024-04-07T16:52:10.710 INFO:tasks.workunit.client.0.smithi100.stdout:op 4090 completed, throughput=5MB/sec 2024-04-07T16:52:10.710 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-uqW8kkEMYT4BqTX off=703781 len=1413022 2024-04-07T16:52:10.714 INFO:tasks.workunit.client.0.smithi100.stdout:op 4091 completed, throughput=5MB/sec 2024-04-07T16:52:10.714 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-yLdzqBHIQkk3N_g off=1611562 len=914789 2024-04-07T16:52:10.721 INFO:tasks.workunit.client.0.smithi100.stdout:op 4092 completed, throughput=5MB/sec 2024-04-07T16:52:10.721 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-V8AX2Q4tgs7zEKm off=2713396 len=550086 2024-04-07T16:52:10.728 INFO:tasks.workunit.client.0.smithi100.stdout:op 4093 completed, throughput=5MB/sec 2024-04-07T16:52:10.728 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-oJdg-qykU3EDs95 off=5976508 len=1905095 2024-04-07T16:52:10.732 INFO:tasks.workunit.client.0.smithi100.stdout:op 4094 completed, throughput=5MB/sec 2024-04-07T16:52:10.861 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:52:10 smithi144 bash[66654]: cluster 2024-04-07T16:52:09.487564+0000 mgr.y (mgr.25175) 1104 : cluster [DBG] pgmap v1746: 105 pgs: 105 active+clean; 126 GiB data, 8.8 GiB used, 706 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.1 MiB/s wr, 4 op/s 2024-04-07T16:52:10.948 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:52:10 smithi100 bash[74942]: cluster 2024-04-07T16:52:09.487564+0000 mgr.y (mgr.25175) 1104 : cluster [DBG] pgmap v1746: 105 pgs: 105 active+clean; 126 GiB data, 8.8 GiB used, 706 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.1 MiB/s wr, 4 op/s 2024-04-07T16:52:10.948 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:52:10 smithi100 bash[76202]: cluster 2024-04-07T16:52:09.487564+0000 mgr.y (mgr.25175) 1104 : cluster [DBG] pgmap v1746: 105 pgs: 105 active+clean; 126 GiB data, 8.8 GiB used, 706 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.1 MiB/s wr, 4 op/s 2024-04-07T16:52:11.733 INFO:tasks.workunit.client.0.smithi100.stdout: 789: throughput=5MB/sec pending data=0 2024-04-07T16:52:11.733 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-g0A7eL6QjbYSlXz off=13507502 len=69608 2024-04-07T16:52:11.737 INFO:tasks.workunit.client.0.smithi100.stdout:op 4095 completed, throughput=5MB/sec 2024-04-07T16:52:11.738 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-fpAB9QLtOHVp6Me off=18119363 len=2032884 2024-04-07T16:52:11.744 INFO:tasks.workunit.client.0.smithi100.stdout:op 4096 completed, throughput=5MB/sec 2024-04-07T16:52:11.744 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-SqpO8aetW97i-kj off=14059446 len=216981 2024-04-07T16:52:11.750 INFO:tasks.workunit.client.0.smithi100.stdout:op 4097 completed, throughput=5MB/sec 2024-04-07T16:52:11.750 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-PI2FbNcvIDx0R3Z off=3677951 len=305130 2024-04-07T16:52:11.756 INFO:tasks.workunit.client.0.smithi100.stdout:op 4098 completed, throughput=5MB/sec 2024-04-07T16:52:11.756 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-uExnuGqXey7f0JQ off=1641121 len=993556 2024-04-07T16:52:11.766 INFO:tasks.workunit.client.0.smithi100.stdout:op 4099 completed, throughput=5MB/sec 2024-04-07T16:52:11.766 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-GXyRQqQCdku5XAH off=2546774 len=2065975 2024-04-07T16:52:11.786 INFO:tasks.workunit.client.0.smithi100.stdout:op 4100 completed, throughput=5MB/sec 2024-04-07T16:52:12.581 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:52:12 smithi144 bash[95233]: ts=2024-04-07T16:52:12.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:52:12.786 INFO:tasks.workunit.client.0.smithi100.stdout: 790: throughput=5MB/sec pending data=0 2024-04-07T16:52:12.786 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-RVY7FCuzaLKRJGG off=8348549 len=1476207 2024-04-07T16:52:12.790 INFO:tasks.workunit.client.0.smithi100.stdout:op 4101 completed, throughput=5MB/sec 2024-04-07T16:52:12.790 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-CFG-P-D8WITOpPN off=304079 len=1150410 2024-04-07T16:52:12.793 INFO:tasks.workunit.client.0.smithi100.stdout:op 4102 completed, throughput=5MB/sec 2024-04-07T16:52:12.793 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-4DMHrCI8S_MW3iP off=1686832 len=240984 2024-04-07T16:52:12.799 INFO:tasks.workunit.client.0.smithi100.stdout:op 4103 completed, throughput=5MB/sec 2024-04-07T16:52:12.799 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-LwV99TyOiUhNwSr off=3654275 len=905811 2024-04-07T16:52:12.808 INFO:tasks.workunit.client.0.smithi100.stdout:op 4104 completed, throughput=5MB/sec 2024-04-07T16:52:12.861 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:52:12 smithi144 bash[66654]: cluster 2024-04-07T16:52:11.488424+0000 mgr.y (mgr.25175) 1105 : cluster [DBG] pgmap v1747: 105 pgs: 105 active+clean; 126 GiB data, 8.8 GiB used, 706 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.5 MiB/s wr, 5 op/s 2024-04-07T16:52:12.948 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:52:12 smithi100 bash[74942]: cluster 2024-04-07T16:52:11.488424+0000 mgr.y (mgr.25175) 1105 : cluster [DBG] pgmap v1747: 105 pgs: 105 active+clean; 126 GiB data, 8.8 GiB used, 706 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.5 MiB/s wr, 5 op/s 2024-04-07T16:52:12.948 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:52:12 smithi100 bash[76202]: cluster 2024-04-07T16:52:11.488424+0000 mgr.y (mgr.25175) 1105 : cluster [DBG] pgmap v1747: 105 pgs: 105 active+clean; 126 GiB data, 8.8 GiB used, 706 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.5 MiB/s wr, 5 op/s 2024-04-07T16:52:13.809 INFO:tasks.workunit.client.0.smithi100.stdout: 791: throughput=4.99MB/sec pending data=0 2024-04-07T16:52:13.810 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-63RfCqnVNAiXTwy off=6595419 len=1708923 2024-04-07T16:52:13.813 INFO:tasks.workunit.client.0.smithi100.stdout:op 4105 completed, throughput=5MB/sec 2024-04-07T16:52:13.813 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-KOdwnvaDHOSOD6t off=7460898 len=1839227 2024-04-07T16:52:13.828 INFO:tasks.workunit.client.0.smithi100.stdout:op 4106 completed, throughput=5MB/sec 2024-04-07T16:52:13.828 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-z1RFM8kzWTq7X-q off=5022944 len=1955413 2024-04-07T16:52:13.841 INFO:tasks.workunit.client.0.smithi100.stdout:op 4107 completed, throughput=5MB/sec 2024-04-07T16:52:14.842 INFO:tasks.workunit.client.0.smithi100.stdout: 792: throughput=4.99MB/sec pending data=0 2024-04-07T16:52:14.842 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-DYhFZuGpEssgNwD off=14669441 len=1702838 2024-04-07T16:52:14.855 INFO:tasks.workunit.client.0.smithi100.stdout:op 4108 completed, throughput=5MB/sec 2024-04-07T16:52:14.855 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-p3rqPeAZBJGifb2 off=10219630 len=306101 2024-04-07T16:52:14.861 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:52:14 smithi144 bash[66654]: cluster 2024-04-07T16:52:13.489629+0000 mgr.y (mgr.25175) 1106 : cluster [DBG] pgmap v1748: 105 pgs: 105 active+clean; 126 GiB data, 8.8 GiB used, 706 GiB / 715 GiB avail; 3.0 MiB/s rd, 3.0 MiB/s wr, 6 op/s 2024-04-07T16:52:14.862 INFO:tasks.workunit.client.0.smithi100.stdout:op 4109 completed, throughput=5MB/sec 2024-04-07T16:52:14.862 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-9ULhrvpq_piOFVo off=2271367 len=1687521 2024-04-07T16:52:14.873 INFO:tasks.workunit.client.0.smithi100.stdout:op 4110 completed, throughput=5MB/sec 2024-04-07T16:52:14.874 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-lO-RXQ2BT8tEoXJ off=980539 len=1882209 2024-04-07T16:52:14.890 INFO:tasks.workunit.client.0.smithi100.stdout:op 4111 completed, throughput=5MB/sec 2024-04-07T16:52:14.948 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:52:14 smithi100 bash[74942]: cluster 2024-04-07T16:52:13.489629+0000 mgr.y (mgr.25175) 1106 : cluster [DBG] pgmap v1748: 105 pgs: 105 active+clean; 126 GiB data, 8.8 GiB used, 706 GiB / 715 GiB avail; 3.0 MiB/s rd, 3.0 MiB/s wr, 6 op/s 2024-04-07T16:52:14.948 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:52:14 smithi100 bash[76202]: cluster 2024-04-07T16:52:13.489629+0000 mgr.y (mgr.25175) 1106 : cluster [DBG] pgmap v1748: 105 pgs: 105 active+clean; 126 GiB data, 8.8 GiB used, 706 GiB / 715 GiB avail; 3.0 MiB/s rd, 3.0 MiB/s wr, 6 op/s 2024-04-07T16:52:15.890 INFO:tasks.workunit.client.0.smithi100.stdout: 793: throughput=4.99MB/sec pending data=0 2024-04-07T16:52:15.890 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-7jx6hf2XmhSFiCs off=4717045 len=1678579 2024-04-07T16:52:15.904 INFO:tasks.workunit.client.0.smithi100.stdout:op 4112 completed, throughput=5MB/sec 2024-04-07T16:52:15.904 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-KPvdZ4Pu8AeHvKq off=9873665 len=1483620 2024-04-07T16:52:15.908 INFO:tasks.workunit.client.0.smithi100.stdout:op 4113 completed, throughput=5MB/sec 2024-04-07T16:52:15.908 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Fsypad1wRU4Sjmw off=3086317 len=1958356 2024-04-07T16:52:15.924 INFO:tasks.workunit.client.0.smithi100.stdout:op 4114 completed, throughput=5MB/sec 2024-04-07T16:52:16.861 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:52:16 smithi144 bash[66654]: cluster 2024-04-07T16:52:15.490711+0000 mgr.y (mgr.25175) 1107 : cluster [DBG] pgmap v1749: 105 pgs: 105 active+clean; 126 GiB data, 8.8 GiB used, 706 GiB / 715 GiB avail; 2.3 MiB/s rd, 2.8 MiB/s wr, 5 op/s 2024-04-07T16:52:16.925 INFO:tasks.workunit.client.0.smithi100.stdout: 794: throughput=4.99MB/sec pending data=0 2024-04-07T16:52:16.925 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Fx_2yGeV2ium0Fa off=2119546 len=209669 2024-04-07T16:52:16.930 INFO:tasks.workunit.client.0.smithi100.stdout:op 4115 completed, throughput=4.99MB/sec 2024-04-07T16:52:16.930 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-uJVBksCYVZ8E_rB off=3776305 len=479331 2024-04-07T16:52:16.937 INFO:tasks.workunit.client.0.smithi100.stdout:op 4116 completed, throughput=4.99MB/sec 2024-04-07T16:52:16.938 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-e1xiBOVwhpxJ8TF off=7986374 len=584950 2024-04-07T16:52:16.945 INFO:tasks.workunit.client.0.smithi100.stdout:op 4117 completed, throughput=5MB/sec 2024-04-07T16:52:16.945 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-VvDGh8EDT0-ibdD off=6218629 len=991050 2024-04-07T16:52:16.947 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:52:16 smithi100 bash[74942]: cluster 2024-04-07T16:52:15.490711+0000 mgr.y (mgr.25175) 1107 : cluster [DBG] pgmap v1749: 105 pgs: 105 active+clean; 126 GiB data, 8.8 GiB used, 706 GiB / 715 GiB avail; 2.3 MiB/s rd, 2.8 MiB/s wr, 5 op/s 2024-04-07T16:52:16.948 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:52:16 smithi100 bash[76202]: cluster 2024-04-07T16:52:15.490711+0000 mgr.y (mgr.25175) 1107 : cluster [DBG] pgmap v1749: 105 pgs: 105 active+clean; 126 GiB data, 8.8 GiB used, 706 GiB / 715 GiB avail; 2.3 MiB/s rd, 2.8 MiB/s wr, 5 op/s 2024-04-07T16:52:16.953 INFO:tasks.workunit.client.0.smithi100.stdout:op 4118 completed, throughput=5MB/sec 2024-04-07T16:52:16.953 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-qj-fUW2uN23xl5n off=2061946 len=1292746 2024-04-07T16:52:16.964 INFO:tasks.workunit.client.0.smithi100.stdout:op 4119 completed, throughput=5MB/sec 2024-04-07T16:52:16.964 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj--56qTEomAEHrF7w off=9681109 len=598961 2024-04-07T16:52:16.966 INFO:tasks.workunit.client.0.smithi100.stdout:op 4120 completed, throughput=5MB/sec 2024-04-07T16:52:16.966 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-wOCUa4BejDa-qkH off=1847644 len=107368 2024-04-07T16:52:16.971 INFO:tasks.workunit.client.0.smithi100.stdout:op 4121 completed, throughput=5MB/sec 2024-04-07T16:52:16.971 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-L1kfKVQIq9Y1m37 off=6513565 len=840711 2024-04-07T16:52:16.978 INFO:tasks.workunit.client.0.smithi100.stdout:op 4122 completed, throughput=5MB/sec 2024-04-07T16:52:16.978 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-3bdP2JDUVBpEZXk off=16687057 len=416879 2024-04-07T16:52:16.981 INFO:tasks.workunit.client.0.smithi100.stdout:op 4123 completed, throughput=5MB/sec 2024-04-07T16:52:17.862 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:52:17 smithi144 bash[66654]: audit 2024-04-07T16:52:16.779600+0000 mon.a (mon.0) 2163 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:52:17.948 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:52:17 smithi100 bash[74942]: audit 2024-04-07T16:52:16.779600+0000 mon.a (mon.0) 2163 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:52:17.948 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:52:17 smithi100 bash[76202]: audit 2024-04-07T16:52:16.779600+0000 mon.a (mon.0) 2163 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:52:17.981 INFO:tasks.workunit.client.0.smithi100.stdout: 795: throughput=4.99MB/sec pending data=0 2024-04-07T16:52:17.981 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-p4rrwFADFxpo759 off=21495843 len=1374072 2024-04-07T16:52:17.993 INFO:tasks.workunit.client.0.smithi100.stdout:op 4124 completed, throughput=5MB/sec 2024-04-07T16:52:17.993 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Mw1CYL_-8g--2PK off=24151391 len=164359 2024-04-07T16:52:17.998 INFO:tasks.workunit.client.0.smithi100.stdout:op 4125 completed, throughput=5MB/sec 2024-04-07T16:52:17.998 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-1mWGyrAUPZLHVY4 off=1599142 len=1250025 2024-04-07T16:52:18.009 INFO:tasks.workunit.client.0.smithi100.stdout:op 4126 completed, throughput=5MB/sec 2024-04-07T16:52:18.009 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-VFWWarFRY07SysI off=17740002 len=1509935 2024-04-07T16:52:18.013 INFO:tasks.workunit.client.0.smithi100.stdout:op 4127 completed, throughput=5MB/sec 2024-04-07T16:52:18.013 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-VnDzzSigc_MxVzw off=4222304 len=1400071 2024-04-07T16:52:18.024 INFO:tasks.workunit.client.0.smithi100.stdout:op 4128 completed, throughput=5MB/sec 2024-04-07T16:52:18.698 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:52:18 smithi100 bash[76202]: cluster 2024-04-07T16:52:17.491797+0000 mgr.y (mgr.25175) 1108 : cluster [DBG] pgmap v1750: 105 pgs: 105 active+clean; 126 GiB data, 8.8 GiB used, 706 GiB / 715 GiB avail; 2.1 MiB/s rd, 3.2 MiB/s wr, 5 op/s 2024-04-07T16:52:18.698 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:52:18 smithi100 bash[74942]: cluster 2024-04-07T16:52:17.491797+0000 mgr.y (mgr.25175) 1108 : cluster [DBG] pgmap v1750: 105 pgs: 105 active+clean; 126 GiB data, 8.8 GiB used, 706 GiB / 715 GiB avail; 2.1 MiB/s rd, 3.2 MiB/s wr, 5 op/s 2024-04-07T16:52:18.698 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:52:18 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:52:18] "GET /metrics HTTP/1.1" 200 34073 "" "Prometheus/2.43.0" 2024-04-07T16:52:18.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:52:18 smithi144 bash[66654]: cluster 2024-04-07T16:52:17.491797+0000 mgr.y (mgr.25175) 1108 : cluster [DBG] pgmap v1750: 105 pgs: 105 active+clean; 126 GiB data, 8.8 GiB used, 706 GiB / 715 GiB avail; 2.1 MiB/s rd, 3.2 MiB/s wr, 5 op/s 2024-04-07T16:52:19.024 INFO:tasks.workunit.client.0.smithi100.stdout: 796: throughput=4.99MB/sec pending data=0 2024-04-07T16:52:19.025 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-KnrTa2EzdXJtOvf off=2634304 len=309421 2024-04-07T16:52:19.030 INFO:tasks.workunit.client.0.smithi100.stdout:op 4129 completed, throughput=4.99MB/sec 2024-04-07T16:52:19.031 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-oKRcGtOO7Oxm1ID off=2137895 len=1300564 2024-04-07T16:52:19.043 INFO:tasks.workunit.client.0.smithi100.stdout:op 4130 completed, throughput=5MB/sec 2024-04-07T16:52:19.043 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-qAq2yOlZVXFbrrF off=5695106 len=1702213 2024-04-07T16:52:19.058 INFO:tasks.workunit.client.0.smithi100.stdout:op 4131 completed, throughput=5MB/sec 2024-04-07T16:52:19.059 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-y0UfHcavQqbELVe off=11231217 len=651299 2024-04-07T16:52:19.066 INFO:tasks.workunit.client.0.smithi100.stdout:op 4132 completed, throughput=5MB/sec 2024-04-07T16:52:19.066 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-mAxj97cTxs1iL3o off=14053172 len=1867941 2024-04-07T16:52:19.073 INFO:tasks.workunit.client.0.smithi100.stdout:op 4133 completed, throughput=5MB/sec 2024-04-07T16:52:20.073 INFO:tasks.workunit.client.0.smithi100.stdout: 797: throughput=4.99MB/sec pending data=0 2024-04-07T16:52:20.073 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-J-oMtRDrgGM7IVX off=3231599 len=980713 2024-04-07T16:52:20.077 INFO:tasks.workunit.client.0.smithi100.stdout:op 4134 completed, throughput=5MB/sec 2024-04-07T16:52:20.077 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-9XqSWfRo8hZ2l0r off=2505537 len=836186 2024-04-07T16:52:20.080 INFO:tasks.workunit.client.0.smithi100.stdout:op 4135 completed, throughput=5MB/sec 2024-04-07T16:52:20.080 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-XDOCqtfx5taSWOO off=837713 len=114117 2024-04-07T16:52:20.085 INFO:tasks.workunit.client.0.smithi100.stdout:op 4136 completed, throughput=5MB/sec 2024-04-07T16:52:20.085 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-jQrpGGpmOtEIDni off=3949924 len=1029030 2024-04-07T16:52:20.088 INFO:tasks.workunit.client.0.smithi100.stdout:op 4137 completed, throughput=5MB/sec 2024-04-07T16:52:20.088 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-8YYEhWSgdGuBNrT off=7173648 len=2061577 2024-04-07T16:52:20.106 INFO:tasks.workunit.client.0.smithi100.stdout:op 4138 completed, throughput=5MB/sec 2024-04-07T16:52:20.862 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:52:20 smithi144 bash[66654]: cluster 2024-04-07T16:52:19.492737+0000 mgr.y (mgr.25175) 1109 : cluster [DBG] pgmap v1751: 105 pgs: 105 active+clean; 126 GiB data, 8.9 GiB used, 706 GiB / 715 GiB avail; 1.8 MiB/s rd, 3.1 MiB/s wr, 5 op/s 2024-04-07T16:52:20.948 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:52:20 smithi100 bash[74942]: cluster 2024-04-07T16:52:19.492737+0000 mgr.y (mgr.25175) 1109 : cluster [DBG] pgmap v1751: 105 pgs: 105 active+clean; 126 GiB data, 8.9 GiB used, 706 GiB / 715 GiB avail; 1.8 MiB/s rd, 3.1 MiB/s wr, 5 op/s 2024-04-07T16:52:20.948 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:52:20 smithi100 bash[76202]: cluster 2024-04-07T16:52:19.492737+0000 mgr.y (mgr.25175) 1109 : cluster [DBG] pgmap v1751: 105 pgs: 105 active+clean; 126 GiB data, 8.9 GiB used, 706 GiB / 715 GiB avail; 1.8 MiB/s rd, 3.1 MiB/s wr, 5 op/s 2024-04-07T16:52:21.106 INFO:tasks.workunit.client.0.smithi100.stdout: 798: throughput=4.99MB/sec pending data=0 2024-04-07T16:52:21.106 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-QsAGU7AW2t_vulg off=5778940 len=1372910 2024-04-07T16:52:21.111 INFO:tasks.workunit.client.0.smithi100.stdout:op 4139 completed, throughput=5MB/sec 2024-04-07T16:52:21.111 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-fJZPLqyqhzYY1gu off=12149489 len=1499677 2024-04-07T16:52:21.123 INFO:tasks.workunit.client.0.smithi100.stdout:op 4140 completed, throughput=5MB/sec 2024-04-07T16:52:21.123 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-SM2tcd_CUAduEhH off=8753895 len=1491829 2024-04-07T16:52:21.126 INFO:tasks.workunit.client.0.smithi100.stdout:op 4141 completed, throughput=5MB/sec 2024-04-07T16:52:21.126 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-J4FfLd0NhXyfH45 off=0 len=1256532 2024-04-07T16:52:21.136 INFO:tasks.workunit.client.0.smithi100.stdout:op 4142 completed, throughput=5MB/sec 2024-04-07T16:52:22.136 INFO:tasks.workunit.client.0.smithi100.stdout: 799: throughput=4.99MB/sec pending data=0 2024-04-07T16:52:22.136 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-RcgYiQJEmYdgLgd off=12240204 len=1566933 2024-04-07T16:52:22.140 INFO:tasks.workunit.client.0.smithi100.stdout:op 4143 completed, throughput=5MB/sec 2024-04-07T16:52:22.140 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-QDtEum2zBVevrD6 off=2976841 len=1954237 2024-04-07T16:52:22.158 INFO:tasks.workunit.client.0.smithi100.stdout:op 4144 completed, throughput=5MB/sec 2024-04-07T16:52:22.158 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-yhbc8QeEYE0xA8X off=4641115 len=177185 2024-04-07T16:52:22.159 INFO:tasks.workunit.client.0.smithi100.stdout:op 4145 completed, throughput=5MB/sec 2024-04-07T16:52:22.159 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-IPjWbmAzIXUYg2e off=579159 len=1627135 2024-04-07T16:52:22.172 INFO:tasks.workunit.client.0.smithi100.stdout:op 4146 completed, throughput=5MB/sec 2024-04-07T16:52:22.609 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:52:22 smithi144 bash[95233]: ts=2024-04-07T16:52:22.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:52:22.861 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:52:22 smithi144 bash[66654]: cluster 2024-04-07T16:52:21.493599+0000 mgr.y (mgr.25175) 1110 : cluster [DBG] pgmap v1752: 105 pgs: 105 active+clean; 126 GiB data, 8.9 GiB used, 706 GiB / 715 GiB avail; 1.7 MiB/s rd, 3.0 MiB/s wr, 4 op/s 2024-04-07T16:52:22.948 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:52:22 smithi100 bash[74942]: cluster 2024-04-07T16:52:21.493599+0000 mgr.y (mgr.25175) 1110 : cluster [DBG] pgmap v1752: 105 pgs: 105 active+clean; 126 GiB data, 8.9 GiB used, 706 GiB / 715 GiB avail; 1.7 MiB/s rd, 3.0 MiB/s wr, 4 op/s 2024-04-07T16:52:22.948 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:52:22 smithi100 bash[76202]: cluster 2024-04-07T16:52:21.493599+0000 mgr.y (mgr.25175) 1110 : cluster [DBG] pgmap v1752: 105 pgs: 105 active+clean; 126 GiB data, 8.9 GiB used, 706 GiB / 715 GiB avail; 1.7 MiB/s rd, 3.0 MiB/s wr, 4 op/s 2024-04-07T16:52:23.173 INFO:tasks.workunit.client.0.smithi100.stdout: 800: throughput=4.99MB/sec pending data=0 2024-04-07T16:52:23.173 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-OYl3bnXEyGtAueU off=9443423 len=1093517 2024-04-07T16:52:23.176 INFO:tasks.workunit.client.0.smithi100.stdout:op 4147 completed, throughput=5MB/sec 2024-04-07T16:52:23.176 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-MeUsXfBTCmEm2b8 off=899207 len=1292269 2024-04-07T16:52:23.188 INFO:tasks.workunit.client.0.smithi100.stdout:op 4148 completed, throughput=5MB/sec 2024-04-07T16:52:23.189 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-_M-L945URKhg9Nw off=0 len=1856413 2024-04-07T16:52:23.193 INFO:tasks.workunit.client.0.smithi100.stdout:op 4149 completed, throughput=5MB/sec 2024-04-07T16:52:23.193 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-rWWUdXXgBU0NFMv off=2937917 len=2067381 2024-04-07T16:52:23.208 INFO:tasks.workunit.client.0.smithi100.stdout:op 4150 completed, throughput=5MB/sec 2024-04-07T16:52:24.209 INFO:tasks.workunit.client.0.smithi100.stdout: 801: throughput=5MB/sec pending data=0 2024-04-07T16:52:24.209 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-5g50IgLkAiKXtyf off=17251861 len=465303 2024-04-07T16:52:24.211 INFO:tasks.workunit.client.0.smithi100.stdout:op 4151 completed, throughput=5MB/sec 2024-04-07T16:52:24.211 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-h4SjTzGaNLB40jB off=8531094 len=1207722 2024-04-07T16:52:24.214 INFO:tasks.workunit.client.0.smithi100.stdout:op 4152 completed, throughput=5MB/sec 2024-04-07T16:52:24.214 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-9bp-OHT6BnpL52T off=5477501 len=86177 2024-04-07T16:52:24.219 INFO:tasks.workunit.client.0.smithi100.stdout:op 4153 completed, throughput=5MB/sec 2024-04-07T16:52:24.220 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-g_MLfVoAPpTwnsa off=1486854 len=1618941 2024-04-07T16:52:24.232 INFO:tasks.workunit.client.0.smithi100.stdout:op 4154 completed, throughput=5MB/sec 2024-04-07T16:52:24.233 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-J-sLrR9XM-B836q off=15650275 len=1598158 2024-04-07T16:52:24.236 INFO:tasks.workunit.client.0.smithi100.stdout:op 4155 completed, throughput=5MB/sec 2024-04-07T16:52:24.948 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:52:24 smithi100 bash[74942]: cluster 2024-04-07T16:52:23.494701+0000 mgr.y (mgr.25175) 1111 : cluster [DBG] pgmap v1753: 105 pgs: 105 active+clean; 126 GiB data, 8.9 GiB used, 706 GiB / 715 GiB avail; 1.8 MiB/s rd, 3.4 MiB/s wr, 4 op/s 2024-04-07T16:52:24.948 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:52:24 smithi100 bash[76202]: cluster 2024-04-07T16:52:23.494701+0000 mgr.y (mgr.25175) 1111 : cluster [DBG] pgmap v1753: 105 pgs: 105 active+clean; 126 GiB data, 8.9 GiB used, 706 GiB / 715 GiB avail; 1.8 MiB/s rd, 3.4 MiB/s wr, 4 op/s 2024-04-07T16:52:25.111 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:52:24 smithi144 bash[66654]: cluster 2024-04-07T16:52:23.494701+0000 mgr.y (mgr.25175) 1111 : cluster [DBG] pgmap v1753: 105 pgs: 105 active+clean; 126 GiB data, 8.9 GiB used, 706 GiB / 715 GiB avail; 1.8 MiB/s rd, 3.4 MiB/s wr, 4 op/s 2024-04-07T16:52:25.236 INFO:tasks.workunit.client.0.smithi100.stdout: 802: throughput=4.99MB/sec pending data=0 2024-04-07T16:52:25.237 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-iIiYjeKLhFqRwBT off=218079 len=1043553 2024-04-07T16:52:25.239 INFO:tasks.workunit.client.0.smithi100.stdout:op 4156 completed, throughput=5MB/sec 2024-04-07T16:52:25.240 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-7hk2KwEP5rAdbue off=12556282 len=376313 2024-04-07T16:52:25.246 INFO:tasks.workunit.client.0.smithi100.stdout:op 4157 completed, throughput=5MB/sec 2024-04-07T16:52:25.246 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-1fNCeOyPzqJxiYv off=3144567 len=1378885 2024-04-07T16:52:25.249 INFO:tasks.workunit.client.0.smithi100.stdout:op 4158 completed, throughput=5MB/sec 2024-04-07T16:52:25.249 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-0vvySpag_Vr3-j6 off=9136204 len=1834591 2024-04-07T16:52:25.264 INFO:tasks.workunit.client.0.smithi100.stdout:op 4159 completed, throughput=5MB/sec 2024-04-07T16:52:26.264 INFO:tasks.workunit.client.0.smithi100.stdout: 803: throughput=4.99MB/sec pending data=0 2024-04-07T16:52:26.264 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-ELrltF2CDqkNS7w off=5242143 len=57775 2024-04-07T16:52:26.265 INFO:tasks.workunit.client.0.smithi100.stdout:op 4160 completed, throughput=4.99MB/sec 2024-04-07T16:52:26.265 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-X5QOsfbTXPRvyAL off=15670500 len=199399 2024-04-07T16:52:26.271 INFO:tasks.workunit.client.0.smithi100.stdout:op 4161 completed, throughput=4.99MB/sec 2024-04-07T16:52:26.271 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-ZX2Lm3TBMQpSmuo off=17878585 len=2079561 2024-04-07T16:52:26.289 INFO:tasks.workunit.client.0.smithi100.stdout:op 4162 completed, throughput=5MB/sec 2024-04-07T16:52:26.289 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-ZYl_KYUZj-Vv6iV off=5952841 len=833092 2024-04-07T16:52:26.297 INFO:tasks.workunit.client.0.smithi100.stdout:op 4163 completed, throughput=5MB/sec 2024-04-07T16:52:26.297 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-jWQE-rGUH3KLzqM off=2306854 len=204707 2024-04-07T16:52:26.298 INFO:tasks.workunit.client.0.smithi100.stdout:op 4164 completed, throughput=5MB/sec 2024-04-07T16:52:26.299 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-kq6TiS3r44MJDA5 off=3314170 len=1922665 2024-04-07T16:52:26.315 INFO:tasks.workunit.client.0.smithi100.stdout:op 4165 completed, throughput=5MB/sec 2024-04-07T16:52:26.315 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-uz3aSEc7RY2wLGa off=8974880 len=1097410 2024-04-07T16:52:26.319 INFO:tasks.workunit.client.0.smithi100.stdout:op 4166 completed, throughput=5MB/sec 2024-04-07T16:52:26.948 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:52:26 smithi100 bash[74942]: cluster 2024-04-07T16:52:25.495686+0000 mgr.y (mgr.25175) 1112 : cluster [DBG] pgmap v1754: 105 pgs: 105 active+clean; 126 GiB data, 8.9 GiB used, 706 GiB / 715 GiB avail; 1.6 MiB/s rd, 2.9 MiB/s wr, 4 op/s 2024-04-07T16:52:26.948 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:52:26 smithi100 bash[76202]: cluster 2024-04-07T16:52:25.495686+0000 mgr.y (mgr.25175) 1112 : cluster [DBG] pgmap v1754: 105 pgs: 105 active+clean; 126 GiB data, 8.9 GiB used, 706 GiB / 715 GiB avail; 1.6 MiB/s rd, 2.9 MiB/s wr, 4 op/s 2024-04-07T16:52:27.112 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:52:26 smithi144 bash[66654]: cluster 2024-04-07T16:52:25.495686+0000 mgr.y (mgr.25175) 1112 : cluster [DBG] pgmap v1754: 105 pgs: 105 active+clean; 126 GiB data, 8.9 GiB used, 706 GiB / 715 GiB avail; 1.6 MiB/s rd, 2.9 MiB/s wr, 4 op/s 2024-04-07T16:52:27.320 INFO:tasks.workunit.client.0.smithi100.stdout: 804: throughput=5MB/sec pending data=0 2024-04-07T16:52:27.320 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-boegcmkGWsUqVVM off=5345222 len=1083892 2024-04-07T16:52:27.324 INFO:tasks.workunit.client.0.smithi100.stdout:op 4167 completed, throughput=5MB/sec 2024-04-07T16:52:27.324 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-9N45bTf-ntJuWYg off=10206597 len=1147455 2024-04-07T16:52:27.327 INFO:tasks.workunit.client.0.smithi100.stdout:op 4168 completed, throughput=5MB/sec 2024-04-07T16:52:27.327 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-lp14Lf1jWizfMmm off=1273462 len=547407 2024-04-07T16:52:27.329 INFO:tasks.workunit.client.0.smithi100.stdout:op 4169 completed, throughput=5MB/sec 2024-04-07T16:52:27.329 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-wwgypcgsDho4oFe off=7135075 len=100807 2024-04-07T16:52:27.331 INFO:tasks.workunit.client.0.smithi100.stdout:op 4170 completed, throughput=5MB/sec 2024-04-07T16:52:27.331 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-2MsD-e0gJX5yWhL off=3517777 len=1656626 2024-04-07T16:52:27.344 INFO:tasks.workunit.client.0.smithi100.stdout:op 4171 completed, throughput=5MB/sec 2024-04-07T16:52:28.344 INFO:tasks.workunit.client.0.smithi100.stdout: 805: throughput=4.99MB/sec pending data=0 2024-04-07T16:52:28.344 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-SDrqWZGnwRLYVMh off=1859034 len=2089966 2024-04-07T16:52:28.364 INFO:tasks.workunit.client.0.smithi100.stdout:op 4172 completed, throughput=5MB/sec 2024-04-07T16:52:28.364 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-hGKm6e0naeMeBQ9 off=6808542 len=1552244 2024-04-07T16:52:28.376 INFO:tasks.workunit.client.0.smithi100.stdout:op 4173 completed, throughput=5MB/sec 2024-04-07T16:52:28.376 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-pW11VG0r2VyWIMe off=334321 len=504637 2024-04-07T16:52:28.382 INFO:tasks.workunit.client.0.smithi100.stdout:op 4174 completed, throughput=5MB/sec 2024-04-07T16:52:28.382 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-TILjCMDH8PjTZG4 off=15350640 len=1083684 2024-04-07T16:52:28.390 INFO:tasks.workunit.client.0.smithi100.stdout:op 4175 completed, throughput=5MB/sec 2024-04-07T16:52:28.390 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-_b_xrVZYcDjt6cJ off=10720479 len=424954 2024-04-07T16:52:28.396 INFO:tasks.workunit.client.0.smithi100.stdout:op 4176 completed, throughput=5MB/sec 2024-04-07T16:52:28.698 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:52:28 smithi100 bash[74942]: cluster 2024-04-07T16:52:27.496822+0000 mgr.y (mgr.25175) 1113 : cluster [DBG] pgmap v1755: 105 pgs: 105 active+clean; 126 GiB data, 8.9 GiB used, 706 GiB / 715 GiB avail; 1.6 MiB/s rd, 3.0 MiB/s wr, 4 op/s 2024-04-07T16:52:28.698 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:52:28 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:52:28] "GET /metrics HTTP/1.1" 200 34073 "" "Prometheus/2.43.0" 2024-04-07T16:52:28.698 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:52:28 smithi100 bash[76202]: cluster 2024-04-07T16:52:27.496822+0000 mgr.y (mgr.25175) 1113 : cluster [DBG] pgmap v1755: 105 pgs: 105 active+clean; 126 GiB data, 8.9 GiB used, 706 GiB / 715 GiB avail; 1.6 MiB/s rd, 3.0 MiB/s wr, 4 op/s 2024-04-07T16:52:29.112 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:52:28 smithi144 bash[66654]: cluster 2024-04-07T16:52:27.496822+0000 mgr.y (mgr.25175) 1113 : cluster [DBG] pgmap v1755: 105 pgs: 105 active+clean; 126 GiB data, 8.9 GiB used, 706 GiB / 715 GiB avail; 1.6 MiB/s rd, 3.0 MiB/s wr, 4 op/s 2024-04-07T16:52:29.397 INFO:tasks.workunit.client.0.smithi100.stdout: 806: throughput=4.99MB/sec pending data=0 2024-04-07T16:52:29.397 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-sluCvkJNa_dpYKC off=6347622 len=677039 2024-04-07T16:52:29.399 INFO:tasks.workunit.client.0.smithi100.stdout:op 4177 completed, throughput=5MB/sec 2024-04-07T16:52:29.399 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-I4Iyv1xbr25TXa_ off=10103208 len=360812 2024-04-07T16:52:29.401 INFO:tasks.workunit.client.0.smithi100.stdout:op 4178 completed, throughput=5MB/sec 2024-04-07T16:52:29.401 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-nlFwfzyTO9dJdOU off=7240665 len=688629 2024-04-07T16:52:29.403 INFO:tasks.workunit.client.0.smithi100.stdout:op 4179 completed, throughput=5MB/sec 2024-04-07T16:52:29.403 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-k7bSd33JbUmyit6 off=5314149 len=515281 2024-04-07T16:52:29.405 INFO:tasks.workunit.client.0.smithi100.stdout:op 4180 completed, throughput=5MB/sec 2024-04-07T16:52:29.405 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-1vu5qoF1V_hRfWB off=20090 len=1004493 2024-04-07T16:52:29.413 INFO:tasks.workunit.client.0.smithi100.stdout:op 4181 completed, throughput=5MB/sec 2024-04-07T16:52:29.414 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-dDl2rQ-sCiRACr1 off=8276854 len=411247 2024-04-07T16:52:29.415 INFO:tasks.workunit.client.0.smithi100.stdout:op 4182 completed, throughput=5MB/sec 2024-04-07T16:52:29.415 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Rhil2D33D2XH9jV off=3455099 len=2091837 2024-04-07T16:52:29.420 INFO:tasks.workunit.client.0.smithi100.stdout:op 4183 completed, throughput=5MB/sec 2024-04-07T16:52:30.420 INFO:tasks.workunit.client.0.smithi100.stdout: 807: throughput=4.99MB/sec pending data=0 2024-04-07T16:52:30.420 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-nVZawXTbps-u5YW off=16712265 len=1098046 2024-04-07T16:52:30.423 INFO:tasks.workunit.client.0.smithi100.stdout:op 4184 completed, throughput=5MB/sec 2024-04-07T16:52:30.423 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-yT_Vg9oB4arPL_A off=4035944 len=1849638 2024-04-07T16:52:30.438 INFO:tasks.workunit.client.0.smithi100.stdout:op 4185 completed, throughput=5MB/sec 2024-04-07T16:52:30.438 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj--lraeuoCqGkKQoJ off=458627 len=953439 2024-04-07T16:52:30.447 INFO:tasks.workunit.client.0.smithi100.stdout:op 4186 completed, throughput=5MB/sec 2024-04-07T16:52:30.447 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-5Nj4DbzxLpE08Aa off=4945956 len=505665 2024-04-07T16:52:30.453 INFO:tasks.workunit.client.0.smithi100.stdout:op 4187 completed, throughput=5MB/sec 2024-04-07T16:52:30.453 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-j3q-T7eIZT2ni7Q off=11694948 len=2079554 2024-04-07T16:52:30.458 INFO:tasks.workunit.client.0.smithi100.stdout:op 4188 completed, throughput=5MB/sec 2024-04-07T16:52:30.948 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:52:30 smithi100 bash[74942]: cluster 2024-04-07T16:52:29.497807+0000 mgr.y (mgr.25175) 1114 : cluster [DBG] pgmap v1756: 105 pgs: 105 active+clean; 126 GiB data, 8.9 GiB used, 706 GiB / 715 GiB avail; 1.9 MiB/s rd, 2.2 MiB/s wr, 3 op/s 2024-04-07T16:52:30.948 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:52:30 smithi100 bash[76202]: cluster 2024-04-07T16:52:29.497807+0000 mgr.y (mgr.25175) 1114 : cluster [DBG] pgmap v1756: 105 pgs: 105 active+clean; 126 GiB data, 8.9 GiB used, 706 GiB / 715 GiB avail; 1.9 MiB/s rd, 2.2 MiB/s wr, 3 op/s 2024-04-07T16:52:31.112 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:52:30 smithi144 bash[66654]: cluster 2024-04-07T16:52:29.497807+0000 mgr.y (mgr.25175) 1114 : cluster [DBG] pgmap v1756: 105 pgs: 105 active+clean; 126 GiB data, 8.9 GiB used, 706 GiB / 715 GiB avail; 1.9 MiB/s rd, 2.2 MiB/s wr, 3 op/s 2024-04-07T16:52:31.458 INFO:tasks.workunit.client.0.smithi100.stdout: 808: throughput=5MB/sec pending data=0 2024-04-07T16:52:31.458 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-hclHmplybmiLki6 off=137214 len=751196 2024-04-07T16:52:31.461 INFO:tasks.workunit.client.0.smithi100.stdout:op 4189 completed, throughput=5MB/sec 2024-04-07T16:52:31.461 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-vRHH0goaFTvY3tf off=783586 len=1620534 2024-04-07T16:52:31.465 INFO:tasks.workunit.client.0.smithi100.stdout:op 4190 completed, throughput=5MB/sec 2024-04-07T16:52:31.465 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-1JhWLJptdQp7HRs off=833307 len=1643246 2024-04-07T16:52:31.477 INFO:tasks.workunit.client.0.smithi100.stdout:op 4191 completed, throughput=5MB/sec 2024-04-07T16:52:32.477 INFO:tasks.workunit.client.0.smithi100.stdout: 809: throughput=4.99MB/sec pending data=0 2024-04-07T16:52:32.477 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-4lg7Px6BynxByAY off=872340 len=296255 2024-04-07T16:52:32.483 INFO:tasks.workunit.client.0.smithi100.stdout:op 4192 completed, throughput=4.99MB/sec 2024-04-07T16:52:32.483 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-BRoQVuOrGK0cWQl off=10892597 len=1924170 2024-04-07T16:52:32.499 INFO:tasks.workunit.client.0.smithi100.stdout:op 4193 completed, throughput=5MB/sec 2024-04-07T16:52:32.499 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-6p6uPXM15kT4bk9 off=13769414 len=1542359 2024-04-07T16:52:32.503 INFO:tasks.workunit.client.0.smithi100.stdout:op 4194 completed, throughput=5MB/sec 2024-04-07T16:52:32.503 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-sWOrWDnVdnvsUem off=1285882 len=714274 2024-04-07T16:52:32.506 INFO:tasks.workunit.client.0.smithi100.stdout:op 4195 completed, throughput=5MB/sec 2024-04-07T16:52:32.506 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-NcNuUrY44-6oGcD off=20044130 len=992497 2024-04-07T16:52:32.509 INFO:tasks.workunit.client.0.smithi100.stdout:op 4196 completed, throughput=5MB/sec 2024-04-07T16:52:32.612 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:52:32 smithi144 bash[95233]: ts=2024-04-07T16:52:32.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:52:32.948 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:52:32 smithi100 bash[74942]: cluster 2024-04-07T16:52:31.498632+0000 mgr.y (mgr.25175) 1115 : cluster [DBG] pgmap v1757: 105 pgs: 105 active+clean; 126 GiB data, 9.0 GiB used, 706 GiB / 715 GiB avail; 2.1 MiB/s rd, 2.8 MiB/s wr, 4 op/s 2024-04-07T16:52:32.948 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:52:32 smithi100 bash[74942]: audit 2024-04-07T16:52:31.779786+0000 mon.a (mon.0) 2164 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:52:32.948 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:52:32 smithi100 bash[76202]: cluster 2024-04-07T16:52:31.498632+0000 mgr.y (mgr.25175) 1115 : cluster [DBG] pgmap v1757: 105 pgs: 105 active+clean; 126 GiB data, 9.0 GiB used, 706 GiB / 715 GiB avail; 2.1 MiB/s rd, 2.8 MiB/s wr, 4 op/s 2024-04-07T16:52:32.949 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:52:32 smithi100 bash[76202]: audit 2024-04-07T16:52:31.779786+0000 mon.a (mon.0) 2164 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:52:33.112 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:52:32 smithi144 bash[66654]: cluster 2024-04-07T16:52:31.498632+0000 mgr.y (mgr.25175) 1115 : cluster [DBG] pgmap v1757: 105 pgs: 105 active+clean; 126 GiB data, 9.0 GiB used, 706 GiB / 715 GiB avail; 2.1 MiB/s rd, 2.8 MiB/s wr, 4 op/s 2024-04-07T16:52:33.112 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:52:32 smithi144 bash[66654]: audit 2024-04-07T16:52:31.779786+0000 mon.a (mon.0) 2164 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:52:33.510 INFO:tasks.workunit.client.0.smithi100.stdout: 810: throughput=4.99MB/sec pending data=0 2024-04-07T16:52:33.510 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-iAgC2eMGt3D8Ufn off=5800446 len=1304307 2024-04-07T16:52:33.520 INFO:tasks.workunit.client.0.smithi100.stdout:op 4197 completed, throughput=5MB/sec 2024-04-07T16:52:33.520 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-39eC1TPtgLxtkvY off=7211240 len=771990 2024-04-07T16:52:33.528 INFO:tasks.workunit.client.0.smithi100.stdout:op 4198 completed, throughput=5MB/sec 2024-04-07T16:52:33.528 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-3QnwvuOGwx-zbfg off=3591160 len=1187950 2024-04-07T16:52:33.531 INFO:tasks.workunit.client.0.smithi100.stdout:op 4199 completed, throughput=5MB/sec 2024-04-07T16:52:33.531 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-s-TcMxa7cTuP-3y off=2190663 len=1690796 2024-04-07T16:52:33.546 INFO:tasks.workunit.client.0.smithi100.stdout:op 4200 completed, throughput=5MB/sec 2024-04-07T16:52:33.546 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-4i-VtQDvM1ffj6D off=2207748 len=274840 2024-04-07T16:52:33.552 INFO:tasks.workunit.client.0.smithi100.stdout:op 4201 completed, throughput=5MB/sec 2024-04-07T16:52:34.552 INFO:tasks.workunit.client.0.smithi100.stdout: 811: throughput=4.99MB/sec pending data=0 2024-04-07T16:52:34.553 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Uy2fAVUj9QgWze8 off=352534 len=1660835 2024-04-07T16:52:34.556 INFO:tasks.workunit.client.0.smithi100.stdout:op 4202 completed, throughput=5MB/sec 2024-04-07T16:52:34.556 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-rQT6pcUe-2gyt2K off=609878 len=1274892 2024-04-07T16:52:34.568 INFO:tasks.workunit.client.0.smithi100.stdout:op 4203 completed, throughput=5MB/sec 2024-04-07T16:52:34.568 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-0gliZjf5hddKNXi off=16334304 len=1799414 2024-04-07T16:52:34.581 INFO:tasks.workunit.client.0.smithi100.stdout:op 4204 completed, throughput=5MB/sec 2024-04-07T16:52:34.581 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-9p4Uh4dX-xNkzXb off=10476647 len=1115598 2024-04-07T16:52:34.586 INFO:tasks.workunit.client.0.smithi100.stdout:op 4205 completed, throughput=5MB/sec 2024-04-07T16:52:34.948 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:52:34 smithi100 bash[74942]: cluster 2024-04-07T16:52:33.499783+0000 mgr.y (mgr.25175) 1116 : cluster [DBG] pgmap v1758: 105 pgs: 105 active+clean; 126 GiB data, 9.0 GiB used, 706 GiB / 715 GiB avail; 2.2 MiB/s rd, 3.0 MiB/s wr, 4 op/s 2024-04-07T16:52:34.948 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:52:34 smithi100 bash[76202]: cluster 2024-04-07T16:52:33.499783+0000 mgr.y (mgr.25175) 1116 : cluster [DBG] pgmap v1758: 105 pgs: 105 active+clean; 126 GiB data, 9.0 GiB used, 706 GiB / 715 GiB avail; 2.2 MiB/s rd, 3.0 MiB/s wr, 4 op/s 2024-04-07T16:52:35.112 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:52:34 smithi144 bash[66654]: cluster 2024-04-07T16:52:33.499783+0000 mgr.y (mgr.25175) 1116 : cluster [DBG] pgmap v1758: 105 pgs: 105 active+clean; 126 GiB data, 9.0 GiB used, 706 GiB / 715 GiB avail; 2.2 MiB/s rd, 3.0 MiB/s wr, 4 op/s 2024-04-07T16:52:35.587 INFO:tasks.workunit.client.0.smithi100.stdout: 812: throughput=4.99MB/sec pending data=0 2024-04-07T16:52:35.587 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-oQsUwbqZ46MfI03 off=10921575 len=357967 2024-04-07T16:52:35.588 INFO:tasks.workunit.client.0.smithi100.stdout:op 4206 completed, throughput=5MB/sec 2024-04-07T16:52:35.588 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-mx0IkEXWi43ngzA off=1572985 len=1781634 2024-04-07T16:52:35.602 INFO:tasks.workunit.client.0.smithi100.stdout:op 4207 completed, throughput=5MB/sec 2024-04-07T16:52:35.602 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-gdq37plOwjGqJI_ off=15275811 len=1947917 2024-04-07T16:52:35.608 INFO:tasks.workunit.client.0.smithi100.stdout:op 4208 completed, throughput=5MB/sec 2024-04-07T16:52:35.608 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-PacjVPvDFd6m2oQ off=1010102 len=334083 2024-04-07T16:52:35.614 INFO:tasks.workunit.client.0.smithi100.stdout:op 4209 completed, throughput=5MB/sec 2024-04-07T16:52:35.614 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-kZ--8DKPS5oO1-5 off=9095366 len=491212 2024-04-07T16:52:35.615 INFO:tasks.workunit.client.0.smithi100.stdout:op 4210 completed, throughput=5MB/sec 2024-04-07T16:52:36.616 INFO:tasks.workunit.client.0.smithi100.stdout: 813: throughput=4.99MB/sec pending data=0 2024-04-07T16:52:36.616 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-xusAYN4eZERAKfc off=3853279 len=833718 2024-04-07T16:52:36.624 INFO:tasks.workunit.client.0.smithi100.stdout:op 4211 completed, throughput=5MB/sec 2024-04-07T16:52:36.624 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-7TTnn2ScNBmW4lh off=14799433 len=1163076 2024-04-07T16:52:36.634 INFO:tasks.workunit.client.0.smithi100.stdout:op 4212 completed, throughput=5MB/sec 2024-04-07T16:52:36.634 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-jPdgFoDtcU1ay75 off=17319068 len=1427114 2024-04-07T16:52:36.648 INFO:tasks.workunit.client.0.smithi100.stdout:op 4213 completed, throughput=5MB/sec 2024-04-07T16:52:36.648 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-vAi6hSV8pyWxKp5 off=1224979 len=1751254 2024-04-07T16:52:36.654 INFO:tasks.workunit.client.0.smithi100.stdout:op 4214 completed, throughput=5MB/sec 2024-04-07T16:52:36.655 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-B4QdmhtlBuT6W5G off=5267506 len=1191186 2024-04-07T16:52:36.664 INFO:tasks.workunit.client.0.smithi100.stdout:op 4215 completed, throughput=5MB/sec 2024-04-07T16:52:36.948 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:52:36 smithi100 bash[74942]: cluster 2024-04-07T16:52:35.500725+0000 mgr.y (mgr.25175) 1117 : cluster [DBG] pgmap v1759: 105 pgs: 105 active+clean; 126 GiB data, 9.0 GiB used, 706 GiB / 715 GiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 4 op/s 2024-04-07T16:52:36.948 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:52:36 smithi100 bash[76202]: cluster 2024-04-07T16:52:35.500725+0000 mgr.y (mgr.25175) 1117 : cluster [DBG] pgmap v1759: 105 pgs: 105 active+clean; 126 GiB data, 9.0 GiB used, 706 GiB / 715 GiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 4 op/s 2024-04-07T16:52:37.112 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:52:36 smithi144 bash[66654]: cluster 2024-04-07T16:52:35.500725+0000 mgr.y (mgr.25175) 1117 : cluster [DBG] pgmap v1759: 105 pgs: 105 active+clean; 126 GiB data, 9.0 GiB used, 706 GiB / 715 GiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 4 op/s 2024-04-07T16:52:37.664 INFO:tasks.workunit.client.0.smithi100.stdout: 814: throughput=5MB/sec pending data=0 2024-04-07T16:52:37.665 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-jITvSFyp8zEMy-r off=12846666 len=389300 2024-04-07T16:52:37.670 INFO:tasks.workunit.client.0.smithi100.stdout:op 4216 completed, throughput=5MB/sec 2024-04-07T16:52:37.671 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-4heULiiNGCowUl0 off=7719513 len=1700722 2024-04-07T16:52:37.675 INFO:tasks.workunit.client.0.smithi100.stdout:op 4217 completed, throughput=5MB/sec 2024-04-07T16:52:37.675 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-6lx6a2_0PFpOkkB off=2371313 len=403994 2024-04-07T16:52:37.681 INFO:tasks.workunit.client.0.smithi100.stdout:op 4218 completed, throughput=5MB/sec 2024-04-07T16:52:37.681 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-J2yt0yWmOls94fv off=19500954 len=1845984 2024-04-07T16:52:37.696 INFO:tasks.workunit.client.0.smithi100.stdout:op 4219 completed, throughput=5MB/sec 2024-04-07T16:52:37.696 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-GQJ_zaQd_h7yTWm off=2879730 len=1897391 2024-04-07T16:52:37.708 INFO:tasks.workunit.client.0.smithi100.stdout:op 4220 completed, throughput=5MB/sec 2024-04-07T16:52:38.698 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:52:38 smithi100 bash[74942]: cluster 2024-04-07T16:52:37.501738+0000 mgr.y (mgr.25175) 1118 : cluster [DBG] pgmap v1760: 105 pgs: 105 active+clean; 126 GiB data, 9.0 GiB used, 706 GiB / 715 GiB avail; 2.4 MiB/s rd, 2.9 MiB/s wr, 4 op/s 2024-04-07T16:52:38.698 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:52:38 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:52:38] "GET /metrics HTTP/1.1" 200 34067 "" "Prometheus/2.43.0" 2024-04-07T16:52:38.699 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:52:38 smithi100 bash[76202]: cluster 2024-04-07T16:52:37.501738+0000 mgr.y (mgr.25175) 1118 : cluster [DBG] pgmap v1760: 105 pgs: 105 active+clean; 126 GiB data, 9.0 GiB used, 706 GiB / 715 GiB avail; 2.4 MiB/s rd, 2.9 MiB/s wr, 4 op/s 2024-04-07T16:52:38.709 INFO:tasks.workunit.client.0.smithi100.stdout: 815: throughput=5MB/sec pending data=0 2024-04-07T16:52:38.709 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-epjo-wpmdt8cRK4 off=1147259 len=267824 2024-04-07T16:52:38.710 INFO:tasks.workunit.client.0.smithi100.stdout:op 4221 completed, throughput=5MB/sec 2024-04-07T16:52:38.710 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-omkQEw4qH8I-7Fn off=1043648 len=1932487 2024-04-07T16:52:38.715 INFO:tasks.workunit.client.0.smithi100.stdout:op 4222 completed, throughput=5MB/sec 2024-04-07T16:52:38.715 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-fW2vUpcr94ing32 off=15648930 len=1037277 2024-04-07T16:52:38.719 INFO:tasks.workunit.client.0.smithi100.stdout:op 4223 completed, throughput=5MB/sec 2024-04-07T16:52:38.719 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-D2qqehgBkH_t2Qg off=12650160 len=723082 2024-04-07T16:52:38.727 INFO:tasks.workunit.client.0.smithi100.stdout:op 4224 completed, throughput=5MB/sec 2024-04-07T16:52:39.112 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:52:38 smithi144 bash[66654]: cluster 2024-04-07T16:52:37.501738+0000 mgr.y (mgr.25175) 1118 : cluster [DBG] pgmap v1760: 105 pgs: 105 active+clean; 126 GiB data, 9.0 GiB used, 706 GiB / 715 GiB avail; 2.4 MiB/s rd, 2.9 MiB/s wr, 4 op/s 2024-04-07T16:52:39.728 INFO:tasks.workunit.client.0.smithi100.stdout: 816: throughput=4.99MB/sec pending data=0 2024-04-07T16:52:39.728 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-vIcNpiBmgS30poT off=6211355 len=1559381 2024-04-07T16:52:39.742 INFO:tasks.workunit.client.0.smithi100.stdout:op 4225 completed, throughput=5MB/sec 2024-04-07T16:52:39.742 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-puGE82U-GgBkAIn off=6544715 len=1185579 2024-04-07T16:52:39.745 INFO:tasks.workunit.client.0.smithi100.stdout:op 4226 completed, throughput=5MB/sec 2024-04-07T16:52:39.746 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-omv7aePkWKWZR72 off=9357750 len=1244547 2024-04-07T16:52:39.748 INFO:tasks.workunit.client.0.smithi100.stdout:op 4227 completed, throughput=5MB/sec 2024-04-07T16:52:39.749 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Rcnn2RG6sfOCTQ8 off=1807113 len=120503 2024-04-07T16:52:39.750 INFO:tasks.workunit.client.0.smithi100.stdout:op 4228 completed, throughput=5MB/sec 2024-04-07T16:52:39.750 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-nIsleky_BL-GxEY off=1101050 len=437322 2024-04-07T16:52:39.751 INFO:tasks.workunit.client.0.smithi100.stdout:op 4229 completed, throughput=5MB/sec 2024-04-07T16:52:39.751 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-ztswCsI-8eXPnVx off=1226866 len=796996 2024-04-07T16:52:39.760 INFO:tasks.workunit.client.0.smithi100.stdout:op 4230 completed, throughput=5MB/sec 2024-04-07T16:52:40.760 INFO:tasks.workunit.client.0.smithi100.stdout: 817: throughput=4.99MB/sec pending data=0 2024-04-07T16:52:40.760 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-6R360L9Ix1ajayE off=8070007 len=1660391 2024-04-07T16:52:40.764 INFO:tasks.workunit.client.0.smithi100.stdout:op 4231 completed, throughput=5MB/sec 2024-04-07T16:52:40.764 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-MgvwpqqHd4tH30s off=4019192 len=2068001 2024-04-07T16:52:40.769 INFO:tasks.workunit.client.0.smithi100.stdout:op 4232 completed, throughput=5MB/sec 2024-04-07T16:52:40.769 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-oJkWTrgkBPEaeAK off=13477066 len=1205135 2024-04-07T16:52:40.773 INFO:tasks.workunit.client.0.smithi100.stdout:op 4233 completed, throughput=5MB/sec 2024-04-07T16:52:40.773 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-mV-CMtIxEK2GQwi off=15237947 len=845113 2024-04-07T16:52:40.782 INFO:tasks.workunit.client.0.smithi100.stdout:op 4234 completed, throughput=5MB/sec 2024-04-07T16:52:40.948 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:52:40 smithi100 bash[74942]: cluster 2024-04-07T16:52:39.502681+0000 mgr.y (mgr.25175) 1119 : cluster [DBG] pgmap v1761: 105 pgs: 105 active+clean; 126 GiB data, 9.0 GiB used, 706 GiB / 715 GiB avail; 2.4 MiB/s rd, 2.9 MiB/s wr, 4 op/s 2024-04-07T16:52:40.948 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:52:40 smithi100 bash[76202]: cluster 2024-04-07T16:52:39.502681+0000 mgr.y (mgr.25175) 1119 : cluster [DBG] pgmap v1761: 105 pgs: 105 active+clean; 126 GiB data, 9.0 GiB used, 706 GiB / 715 GiB avail; 2.4 MiB/s rd, 2.9 MiB/s wr, 4 op/s 2024-04-07T16:52:41.112 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:52:40 smithi144 bash[66654]: cluster 2024-04-07T16:52:39.502681+0000 mgr.y (mgr.25175) 1119 : cluster [DBG] pgmap v1761: 105 pgs: 105 active+clean; 126 GiB data, 9.0 GiB used, 706 GiB / 715 GiB avail; 2.4 MiB/s rd, 2.9 MiB/s wr, 4 op/s 2024-04-07T16:52:41.782 INFO:tasks.workunit.client.0.smithi100.stdout: 818: throughput=4.99MB/sec pending data=0 2024-04-07T16:52:41.782 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-buavWgtZVc-l3Bs off=1027041 len=1263194 2024-04-07T16:52:41.794 INFO:tasks.workunit.client.0.smithi100.stdout:op 4235 completed, throughput=5MB/sec 2024-04-07T16:52:41.794 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-rgTL1oOY_ovZ2XK off=11357769 len=370673 2024-04-07T16:52:41.800 INFO:tasks.workunit.client.0.smithi100.stdout:op 4236 completed, throughput=5MB/sec 2024-04-07T16:52:41.800 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-VKDSOIiukWGAk1R off=6942309 len=1244491 2024-04-07T16:52:41.803 INFO:tasks.workunit.client.0.smithi100.stdout:op 4237 completed, throughput=5MB/sec 2024-04-07T16:52:41.803 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-HxZfWNqtI5GRZse off=5714851 len=1044270 2024-04-07T16:52:41.811 INFO:tasks.workunit.client.0.smithi100.stdout:op 4238 completed, throughput=5MB/sec 2024-04-07T16:52:41.811 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-XeI_yuKKpkGPBeT off=1145568 len=2004118 2024-04-07T16:52:41.816 INFO:tasks.workunit.client.0.smithi100.stdout:op 4239 completed, throughput=5MB/sec 2024-04-07T16:52:42.612 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:52:42 smithi144 bash[95233]: ts=2024-04-07T16:52:42.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:52:42.816 INFO:tasks.workunit.client.0.smithi100.stdout: 819: throughput=5MB/sec pending data=0 2024-04-07T16:52:42.816 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-quJyBIR49iRm7Iz off=20630778 len=303219 2024-04-07T16:52:42.819 INFO:tasks.workunit.client.0.smithi100.stdout:op 4240 completed, throughput=5MB/sec 2024-04-07T16:52:42.819 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-l-t3u-ZWnEVwOw7 off=102294 len=634278 2024-04-07T16:52:42.825 INFO:tasks.workunit.client.0.smithi100.stdout:op 4241 completed, throughput=5MB/sec 2024-04-07T16:52:42.825 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-s8v0DBWmTUkwQj0 off=1302719 len=7927 2024-04-07T16:52:42.826 INFO:tasks.workunit.client.0.smithi100.stdout:op 4242 completed, throughput=5MB/sec 2024-04-07T16:52:42.826 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-f5YWWHaR2bUJZsU off=5083031 len=527740 2024-04-07T16:52:42.832 INFO:tasks.workunit.client.0.smithi100.stdout:op 4243 completed, throughput=5MB/sec 2024-04-07T16:52:42.832 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-c22JrkMu3MXSB65 off=66026 len=211151 2024-04-07T16:52:42.834 INFO:tasks.workunit.client.0.smithi100.stdout:op 4244 completed, throughput=5MB/sec 2024-04-07T16:52:42.834 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Cpv2zXrd-Jn7APd off=1343045 len=1534975 2024-04-07T16:52:42.838 INFO:tasks.workunit.client.0.smithi100.stdout:op 4245 completed, throughput=5MB/sec 2024-04-07T16:52:42.838 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-uOpJF0RPyT47nqE off=1876248 len=336190 2024-04-07T16:52:42.845 INFO:tasks.workunit.client.0.smithi100.stdout:op 4246 completed, throughput=5MB/sec 2024-04-07T16:52:42.845 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-xTL8zkKDtPZ0a7p off=4996056 len=1884607 2024-04-07T16:52:42.849 INFO:tasks.workunit.client.0.smithi100.stdout:op 4247 completed, throughput=5MB/sec 2024-04-07T16:52:42.948 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:52:42 smithi100 bash[74942]: cluster 2024-04-07T16:52:41.503518+0000 mgr.y (mgr.25175) 1120 : cluster [DBG] pgmap v1762: 105 pgs: 105 active+clean; 126 GiB data, 9.0 GiB used, 706 GiB / 715 GiB avail; 2.5 MiB/s rd, 3.1 MiB/s wr, 5 op/s 2024-04-07T16:52:42.948 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:52:42 smithi100 bash[76202]: cluster 2024-04-07T16:52:41.503518+0000 mgr.y (mgr.25175) 1120 : cluster [DBG] pgmap v1762: 105 pgs: 105 active+clean; 126 GiB data, 9.0 GiB used, 706 GiB / 715 GiB avail; 2.5 MiB/s rd, 3.1 MiB/s wr, 5 op/s 2024-04-07T16:52:43.112 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:52:42 smithi144 bash[66654]: cluster 2024-04-07T16:52:41.503518+0000 mgr.y (mgr.25175) 1120 : cluster [DBG] pgmap v1762: 105 pgs: 105 active+clean; 126 GiB data, 9.0 GiB used, 706 GiB / 715 GiB avail; 2.5 MiB/s rd, 3.1 MiB/s wr, 5 op/s 2024-04-07T16:52:43.849 INFO:tasks.workunit.client.0.smithi100.stdout: 820: throughput=5MB/sec pending data=0 2024-04-07T16:52:43.850 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-B4RseVi1HmreXjM off=10215927 len=1222431 2024-04-07T16:52:43.853 INFO:tasks.workunit.client.0.smithi100.stdout:op 4248 completed, throughput=5MB/sec 2024-04-07T16:52:43.853 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-PDb2plS0DUnkYVU off=12458836 len=732037 2024-04-07T16:52:43.855 INFO:tasks.workunit.client.0.smithi100.stdout:op 4249 completed, throughput=5MB/sec 2024-04-07T16:52:43.855 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-4yf7jC72thU0jGy off=2181683 len=1802863 2024-04-07T16:52:43.862 INFO:tasks.workunit.client.0.smithi100.stdout:op 4250 completed, throughput=5MB/sec 2024-04-07T16:52:43.862 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-MRICOXpBQhrWy-E off=4138987 len=475251 2024-04-07T16:52:43.868 INFO:tasks.workunit.client.0.smithi100.stdout:op 4251 completed, throughput=5MB/sec 2024-04-07T16:52:44.868 INFO:tasks.workunit.client.0.smithi100.stdout: 821: throughput=4.99MB/sec pending data=0 2024-04-07T16:52:44.869 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-H86ERA6WsOihPi7 off=2080852 len=309393 2024-04-07T16:52:44.874 INFO:tasks.workunit.client.0.smithi100.stdout:op 4252 completed, throughput=4.99MB/sec 2024-04-07T16:52:44.874 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-5ggaWmKr4ilpf_s off=5964753 len=447774 2024-04-07T16:52:44.881 INFO:tasks.workunit.client.0.smithi100.stdout:op 4253 completed, throughput=5MB/sec 2024-04-07T16:52:44.881 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-tsBmfDreAjcWGpX off=11415394 len=294244 2024-04-07T16:52:44.883 INFO:tasks.workunit.client.0.smithi100.stdout:op 4254 completed, throughput=5MB/sec 2024-04-07T16:52:44.883 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-EwSYG1i7IiSK7Z3 off=1534835 len=1011297 2024-04-07T16:52:44.886 INFO:tasks.workunit.client.0.smithi100.stdout:op 4255 completed, throughput=5MB/sec 2024-04-07T16:52:44.886 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-NEoF7zTqjflOlQf off=1570250 len=608580 2024-04-07T16:52:44.892 INFO:tasks.workunit.client.0.smithi100.stdout:op 4256 completed, throughput=5MB/sec 2024-04-07T16:52:44.893 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-WyVXXT4ACeXlOaD off=17186561 len=44686 2024-04-07T16:52:44.893 INFO:tasks.workunit.client.0.smithi100.stdout:op 4257 completed, throughput=5MB/sec 2024-04-07T16:52:44.893 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-x1slFnMPXBq0BGy off=21797255 len=2085183 2024-04-07T16:52:44.898 INFO:tasks.workunit.client.0.smithi100.stdout:op 4258 completed, throughput=5MB/sec 2024-04-07T16:52:44.898 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-WB9pd9reCLTA8Ad off=941767 len=922942 2024-04-07T16:52:44.901 INFO:tasks.workunit.client.0.smithi100.stdout:op 4259 completed, throughput=5MB/sec 2024-04-07T16:52:44.948 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:52:44 smithi100 bash[74942]: cluster 2024-04-07T16:52:43.504639+0000 mgr.y (mgr.25175) 1121 : cluster [DBG] pgmap v1763: 105 pgs: 105 active+clean; 126 GiB data, 9.0 GiB used, 706 GiB / 715 GiB avail; 2.6 MiB/s rd, 2.8 MiB/s wr, 5 op/s 2024-04-07T16:52:44.948 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:52:44 smithi100 bash[76202]: cluster 2024-04-07T16:52:43.504639+0000 mgr.y (mgr.25175) 1121 : cluster [DBG] pgmap v1763: 105 pgs: 105 active+clean; 126 GiB data, 9.0 GiB used, 706 GiB / 715 GiB avail; 2.6 MiB/s rd, 2.8 MiB/s wr, 5 op/s 2024-04-07T16:52:45.112 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:52:44 smithi144 bash[66654]: cluster 2024-04-07T16:52:43.504639+0000 mgr.y (mgr.25175) 1121 : cluster [DBG] pgmap v1763: 105 pgs: 105 active+clean; 126 GiB data, 9.0 GiB used, 706 GiB / 715 GiB avail; 2.6 MiB/s rd, 2.8 MiB/s wr, 5 op/s 2024-04-07T16:52:45.901 INFO:tasks.workunit.client.0.smithi100.stdout: 822: throughput=4.99MB/sec pending data=0 2024-04-07T16:52:45.901 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj--Yhj2mVFJiYk4uE off=9021072 len=1893593 2024-04-07T16:52:45.907 INFO:tasks.workunit.client.0.smithi100.stdout:op 4260 completed, throughput=5MB/sec 2024-04-07T16:52:45.907 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-tXnqL5b5vqeOCrN off=9711002 len=9829 2024-04-07T16:52:45.908 INFO:tasks.workunit.client.0.smithi100.stdout:op 4261 completed, throughput=5MB/sec 2024-04-07T16:52:45.908 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-8eb50278vNSY2NW off=3937636 len=802437 2024-04-07T16:52:45.911 INFO:tasks.workunit.client.0.smithi100.stdout:op 4262 completed, throughput=5MB/sec 2024-04-07T16:52:45.911 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-IUfYdHgvFHpBxKv off=1407835 len=1171828 2024-04-07T16:52:45.921 INFO:tasks.workunit.client.0.smithi100.stdout:op 4263 completed, throughput=5MB/sec 2024-04-07T16:52:45.921 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-R9v1YRh1PiOPUZQ off=1381692 len=1996028 2024-04-07T16:52:45.926 INFO:tasks.workunit.client.0.smithi100.stdout:op 4264 completed, throughput=5MB/sec 2024-04-07T16:52:46.926 INFO:tasks.workunit.client.0.smithi100.stdout: 823: throughput=5MB/sec pending data=0 2024-04-07T16:52:46.926 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-aYRu7GsL0pZxjY6 off=7238377 len=810710 2024-04-07T16:52:46.935 INFO:tasks.workunit.client.0.smithi100.stdout:op 4265 completed, throughput=5MB/sec 2024-04-07T16:52:46.935 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-sULbW-T3oHbxWUB off=2790364 len=788913 2024-04-07T16:52:46.937 INFO:tasks.workunit.client.0.smithi100.stdout:op 4266 completed, throughput=5MB/sec 2024-04-07T16:52:46.937 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-J-9lqQu5k_g8Igb off=3322372 len=336121 2024-04-07T16:52:46.944 INFO:tasks.workunit.client.0.smithi100.stdout:op 4267 completed, throughput=5MB/sec 2024-04-07T16:52:46.944 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-3nMpDWVRG1pWD1g off=14434402 len=1228293 2024-04-07T16:52:46.947 INFO:tasks.workunit.client.0.smithi100.stdout:op 4268 completed, throughput=5MB/sec 2024-04-07T16:52:46.947 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-iYpPG8zYbFHCGLZ off=4366606 len=1672019 2024-04-07T16:52:46.948 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:52:46 smithi100 bash[74942]: cluster 2024-04-07T16:52:45.505637+0000 mgr.y (mgr.25175) 1122 : cluster [DBG] pgmap v1764: 105 pgs: 105 active+clean; 126 GiB data, 9.1 GiB used, 706 GiB / 715 GiB avail; 2.8 MiB/s rd, 2.5 MiB/s wr, 4 op/s 2024-04-07T16:52:46.949 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:52:46 smithi100 bash[76202]: cluster 2024-04-07T16:52:45.505637+0000 mgr.y (mgr.25175) 1122 : cluster [DBG] pgmap v1764: 105 pgs: 105 active+clean; 126 GiB data, 9.1 GiB used, 706 GiB / 715 GiB avail; 2.8 MiB/s rd, 2.5 MiB/s wr, 4 op/s 2024-04-07T16:52:46.961 INFO:tasks.workunit.client.0.smithi100.stdout:op 4269 completed, throughput=5MB/sec 2024-04-07T16:52:47.112 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:52:46 smithi144 bash[66654]: cluster 2024-04-07T16:52:45.505637+0000 mgr.y (mgr.25175) 1122 : cluster [DBG] pgmap v1764: 105 pgs: 105 active+clean; 126 GiB data, 9.1 GiB used, 706 GiB / 715 GiB avail; 2.8 MiB/s rd, 2.5 MiB/s wr, 4 op/s 2024-04-07T16:52:47.948 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:52:47 smithi100 bash[74942]: audit 2024-04-07T16:52:46.780245+0000 mon.a (mon.0) 2165 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:52:47.948 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:52:47 smithi100 bash[76202]: audit 2024-04-07T16:52:46.780245+0000 mon.a (mon.0) 2165 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:52:47.961 INFO:tasks.workunit.client.0.smithi100.stdout: 824: throughput=4.99MB/sec pending data=0 2024-04-07T16:52:47.961 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-IAHmJTiVgYAlUot off=9018523 len=150369 2024-04-07T16:52:47.966 INFO:tasks.workunit.client.0.smithi100.stdout:op 4270 completed, throughput=4.99MB/sec 2024-04-07T16:52:47.966 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Vb7Isk3TJPTb9ri off=1189733 len=60151 2024-04-07T16:52:47.967 INFO:tasks.workunit.client.0.smithi100.stdout:op 4271 completed, throughput=4.99MB/sec 2024-04-07T16:52:47.968 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-gr_3X-rS1XQTFL_ off=8397212 len=649057 2024-04-07T16:52:47.970 INFO:tasks.workunit.client.0.smithi100.stdout:op 4272 completed, throughput=5MB/sec 2024-04-07T16:52:47.970 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-vvAIXjbsuSiSPxN off=563621 len=1973652 2024-04-07T16:52:47.974 INFO:tasks.workunit.client.0.smithi100.stdout:op 4273 completed, throughput=5MB/sec 2024-04-07T16:52:47.974 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-5kRCOWoJTHARWKx off=1113792 len=37583 2024-04-07T16:52:47.975 INFO:tasks.workunit.client.0.smithi100.stdout:op 4274 completed, throughput=5MB/sec 2024-04-07T16:52:47.976 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-AdF7PnhCZtbr7oK off=11148986 len=561593 2024-04-07T16:52:47.977 INFO:tasks.workunit.client.0.smithi100.stdout:op 4275 completed, throughput=5MB/sec 2024-04-07T16:52:47.977 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-m-99yljE0v9-P4H off=5709131 len=378428 2024-04-07T16:52:47.980 INFO:tasks.workunit.client.0.smithi100.stdout:op 4276 completed, throughput=5MB/sec 2024-04-07T16:52:47.980 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-MsWM604doA4Ktyd off=916783 len=1970805 2024-04-07T16:52:47.995 INFO:tasks.workunit.client.0.smithi100.stdout:op 4277 completed, throughput=5MB/sec 2024-04-07T16:52:48.112 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:52:47 smithi144 bash[66654]: audit 2024-04-07T16:52:46.780245+0000 mon.a (mon.0) 2165 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:52:48.698 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:52:48 smithi100 bash[74942]: cluster 2024-04-07T16:52:47.506646+0000 mgr.y (mgr.25175) 1123 : cluster [DBG] pgmap v1765: 105 pgs: 105 active+clean; 126 GiB data, 9.1 GiB used, 706 GiB / 715 GiB avail; 3.0 MiB/s rd, 2.4 MiB/s wr, 5 op/s 2024-04-07T16:52:48.698 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:52:48 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:52:48] "GET /metrics HTTP/1.1" 200 34070 "" "Prometheus/2.43.0" 2024-04-07T16:52:48.699 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:52:48 smithi100 bash[76202]: cluster 2024-04-07T16:52:47.506646+0000 mgr.y (mgr.25175) 1123 : cluster [DBG] pgmap v1765: 105 pgs: 105 active+clean; 126 GiB data, 9.1 GiB used, 706 GiB / 715 GiB avail; 3.0 MiB/s rd, 2.4 MiB/s wr, 5 op/s 2024-04-07T16:52:48.995 INFO:tasks.workunit.client.0.smithi100.stdout: 825: throughput=4.99MB/sec pending data=0 2024-04-07T16:52:48.995 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-s79MAGSI4k1JJ3r off=4350329 len=445211 2024-04-07T16:52:49.001 INFO:tasks.workunit.client.0.smithi100.stdout:op 4278 completed, throughput=5MB/sec 2024-04-07T16:52:49.001 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-n6QJY1a_sQAHyXC off=6901675 len=1122960 2024-04-07T16:52:49.010 INFO:tasks.workunit.client.0.smithi100.stdout:op 4279 completed, throughput=5MB/sec 2024-04-07T16:52:49.010 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-vFKY4xJm7_8N1Up off=6293401 len=1530243 2024-04-07T16:52:49.014 INFO:tasks.workunit.client.0.smithi100.stdout:op 4280 completed, throughput=5MB/sec 2024-04-07T16:52:49.014 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-UAywZNOBXJVKIsA off=257909 len=1431866 2024-04-07T16:52:49.025 INFO:tasks.workunit.client.0.smithi100.stdout:op 4281 completed, throughput=5MB/sec 2024-04-07T16:52:49.025 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-zn08O9UYy6TaVWJ off=3797201 len=840428 2024-04-07T16:52:49.028 INFO:tasks.workunit.client.0.smithi100.stdout:op 4282 completed, throughput=5MB/sec 2024-04-07T16:52:49.112 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:52:48 smithi144 bash[66654]: cluster 2024-04-07T16:52:47.506646+0000 mgr.y (mgr.25175) 1123 : cluster [DBG] pgmap v1765: 105 pgs: 105 active+clean; 126 GiB data, 9.1 GiB used, 706 GiB / 715 GiB avail; 3.0 MiB/s rd, 2.4 MiB/s wr, 5 op/s 2024-04-07T16:52:50.029 INFO:tasks.workunit.client.0.smithi100.stdout: 826: throughput=4.99MB/sec pending data=0 2024-04-07T16:52:50.029 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-qo6E1ZTvmf33e58 off=2012674 len=1544673 2024-04-07T16:52:50.033 INFO:tasks.workunit.client.0.smithi100.stdout:op 4283 completed, throughput=5MB/sec 2024-04-07T16:52:50.033 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-LVz5dezEyoIB4z6 off=20433276 len=1708862 2024-04-07T16:52:50.038 INFO:tasks.workunit.client.0.smithi100.stdout:op 4284 completed, throughput=5MB/sec 2024-04-07T16:52:50.038 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-wC4GesbBHtt-7D- off=5495860 len=2077023 2024-04-07T16:52:50.043 INFO:tasks.workunit.client.0.smithi100.stdout:op 4285 completed, throughput=5MB/sec 2024-04-07T16:52:50.948 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:52:50 smithi100 bash[74942]: cluster 2024-04-07T16:52:49.507653+0000 mgr.y (mgr.25175) 1124 : cluster [DBG] pgmap v1766: 105 pgs: 105 active+clean; 126 GiB data, 9.1 GiB used, 706 GiB / 715 GiB avail; 2.6 MiB/s rd, 1.9 MiB/s wr, 4 op/s 2024-04-07T16:52:50.948 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:52:50 smithi100 bash[76202]: cluster 2024-04-07T16:52:49.507653+0000 mgr.y (mgr.25175) 1124 : cluster [DBG] pgmap v1766: 105 pgs: 105 active+clean; 126 GiB data, 9.1 GiB used, 706 GiB / 715 GiB avail; 2.6 MiB/s rd, 1.9 MiB/s wr, 4 op/s 2024-04-07T16:52:51.043 INFO:tasks.workunit.client.0.smithi100.stdout: 827: throughput=4.99MB/sec pending data=0 2024-04-07T16:52:51.043 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-y5VZnHwU-BiRs4P off=2994843 len=237148 2024-04-07T16:52:51.045 INFO:tasks.workunit.client.0.smithi100.stdout:op 4286 completed, throughput=5MB/sec 2024-04-07T16:52:51.045 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-fMuum17pwiVRFSZ off=1771290 len=736341 2024-04-07T16:52:51.049 INFO:tasks.workunit.client.0.smithi100.stdout:op 4287 completed, throughput=5MB/sec 2024-04-07T16:52:51.049 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-4l6tJX86zKG8SOz off=269253 len=1258085 2024-04-07T16:52:51.053 INFO:tasks.workunit.client.0.smithi100.stdout:op 4288 completed, throughput=5MB/sec 2024-04-07T16:52:51.053 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-MhmXWRyvSVMwYn3 off=2270756 len=863514 2024-04-07T16:52:51.062 INFO:tasks.workunit.client.0.smithi100.stdout:op 4289 completed, throughput=5MB/sec 2024-04-07T16:52:51.062 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-j5tmkI_mZqSpLEn off=1680388 len=1277163 2024-04-07T16:52:51.065 INFO:tasks.workunit.client.0.smithi100.stdout:op 4290 completed, throughput=5MB/sec 2024-04-07T16:52:51.065 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-MeUsXfBTCmEm2b8 off=1013642 len=1994043 2024-04-07T16:52:51.072 INFO:tasks.workunit.client.0.smithi100.stdout:op 4291 completed, throughput=5MB/sec 2024-04-07T16:52:51.112 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:52:50 smithi144 bash[66654]: cluster 2024-04-07T16:52:49.507653+0000 mgr.y (mgr.25175) 1124 : cluster [DBG] pgmap v1766: 105 pgs: 105 active+clean; 126 GiB data, 9.1 GiB used, 706 GiB / 715 GiB avail; 2.6 MiB/s rd, 1.9 MiB/s wr, 4 op/s 2024-04-07T16:52:52.072 INFO:tasks.workunit.client.0.smithi100.stdout: 828: throughput=5MB/sec pending data=0 2024-04-07T16:52:52.072 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-E0UfjFmgqryTSE_ off=3764911 len=990058 2024-04-07T16:52:52.082 INFO:tasks.workunit.client.0.smithi100.stdout:op 4292 completed, throughput=5MB/sec 2024-04-07T16:52:52.082 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-X_dUGLB0U3aO0sU off=21929467 len=1388793 2024-04-07T16:52:52.086 INFO:tasks.workunit.client.0.smithi100.stdout:op 4293 completed, throughput=5MB/sec 2024-04-07T16:52:52.086 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-AOvzazVGK0NaUH0 off=555948 len=1692774 2024-04-07T16:52:52.098 INFO:tasks.workunit.client.0.smithi100.stdout:op 4294 completed, throughput=5MB/sec 2024-04-07T16:52:52.612 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:52:52 smithi144 bash[95233]: ts=2024-04-07T16:52:52.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:52:52.948 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:52:52 smithi100 bash[74942]: cluster 2024-04-07T16:52:51.508492+0000 mgr.y (mgr.25175) 1125 : cluster [DBG] pgmap v1767: 105 pgs: 105 active+clean; 126 GiB data, 9.1 GiB used, 706 GiB / 715 GiB avail; 3.1 MiB/s rd, 1.6 MiB/s wr, 5 op/s 2024-04-07T16:52:52.948 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:52:52 smithi100 bash[76202]: cluster 2024-04-07T16:52:51.508492+0000 mgr.y (mgr.25175) 1125 : cluster [DBG] pgmap v1767: 105 pgs: 105 active+clean; 126 GiB data, 9.1 GiB used, 706 GiB / 715 GiB avail; 3.1 MiB/s rd, 1.6 MiB/s wr, 5 op/s 2024-04-07T16:52:53.099 INFO:tasks.workunit.client.0.smithi100.stdout: 829: throughput=4.99MB/sec pending data=0 2024-04-07T16:52:53.099 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-9Aa8-Shuy5-Sc64 off=0 len=1126801 2024-04-07T16:52:53.102 INFO:tasks.workunit.client.0.smithi100.stdout:op 4295 completed, throughput=5MB/sec 2024-04-07T16:52:53.102 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-tFKDBYvSMvNQ45n off=14509992 len=1025302 2024-04-07T16:52:53.104 INFO:tasks.workunit.client.0.smithi100.stdout:op 4296 completed, throughput=5MB/sec 2024-04-07T16:52:53.104 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-K0BAuwlH7ai0Rmp off=4298168 len=691449 2024-04-07T16:52:53.106 INFO:tasks.workunit.client.0.smithi100.stdout:op 4297 completed, throughput=5MB/sec 2024-04-07T16:52:53.106 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-wpY8_VNUqLbW2yD off=10817379 len=1355939 2024-04-07T16:52:53.114 INFO:tasks.workunit.client.0.smithi100.stdout:op 4298 completed, throughput=5MB/sec 2024-04-07T16:52:53.114 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-LaPs7ZIJ_mY41bl off=9921338 len=419965 2024-04-07T16:52:53.114 INFO:tasks.workunit.client.0.smithi100.stdout:op 4299 completed, throughput=5MB/sec 2024-04-07T16:52:53.114 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj--bLl_Q-9rts-0S5 off=2486846 len=1901461 2024-04-07T16:52:53.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:52:52 smithi144 bash[66654]: cluster 2024-04-07T16:52:51.508492+0000 mgr.y (mgr.25175) 1125 : cluster [DBG] pgmap v1767: 105 pgs: 105 active+clean; 126 GiB data, 9.1 GiB used, 706 GiB / 715 GiB avail; 3.1 MiB/s rd, 1.6 MiB/s wr, 5 op/s 2024-04-07T16:52:53.127 INFO:tasks.workunit.client.0.smithi100.stdout:op 4300 completed, throughput=5MB/sec 2024-04-07T16:52:54.128 INFO:tasks.workunit.client.0.smithi100.stdout: 830: throughput=5MB/sec pending data=0 2024-04-07T16:52:54.128 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-ipnSnwkoHUUjNOA off=8673185 len=627332 2024-04-07T16:52:54.130 INFO:tasks.workunit.client.0.smithi100.stdout:op 4301 completed, throughput=5MB/sec 2024-04-07T16:52:54.130 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-S5LbfhIrhUzqMx5 off=3286022 len=1792021 2024-04-07T16:52:54.146 INFO:tasks.workunit.client.0.smithi100.stdout:op 4302 completed, throughput=5MB/sec 2024-04-07T16:52:54.146 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-eIyRQFR2pm76pvU off=11171466 len=603137 2024-04-07T16:52:54.148 INFO:tasks.workunit.client.0.smithi100.stdout:op 4303 completed, throughput=5MB/sec 2024-04-07T16:52:54.149 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Dybo13JXpdIHhU- off=270059 len=1292982 2024-04-07T16:52:54.161 INFO:tasks.workunit.client.0.smithi100.stdout:op 4304 completed, throughput=5MB/sec 2024-04-07T16:52:54.948 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:52:54 smithi100 bash[74942]: cluster 2024-04-07T16:52:53.509648+0000 mgr.y (mgr.25175) 1126 : cluster [DBG] pgmap v1768: 105 pgs: 105 active+clean; 126 GiB data, 9.1 GiB used, 706 GiB / 715 GiB avail; 3.1 MiB/s rd, 1.5 MiB/s wr, 5 op/s 2024-04-07T16:52:54.948 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:52:54 smithi100 bash[76202]: cluster 2024-04-07T16:52:53.509648+0000 mgr.y (mgr.25175) 1126 : cluster [DBG] pgmap v1768: 105 pgs: 105 active+clean; 126 GiB data, 9.1 GiB used, 706 GiB / 715 GiB avail; 3.1 MiB/s rd, 1.5 MiB/s wr, 5 op/s 2024-04-07T16:52:55.112 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:52:54 smithi144 bash[66654]: cluster 2024-04-07T16:52:53.509648+0000 mgr.y (mgr.25175) 1126 : cluster [DBG] pgmap v1768: 105 pgs: 105 active+clean; 126 GiB data, 9.1 GiB used, 706 GiB / 715 GiB avail; 3.1 MiB/s rd, 1.5 MiB/s wr, 5 op/s 2024-04-07T16:52:55.161 INFO:tasks.workunit.client.0.smithi100.stdout: 831: throughput=4.99MB/sec pending data=0 2024-04-07T16:52:55.161 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-KO0YN77zq6lOZRq off=19450402 len=32206 2024-04-07T16:52:55.166 INFO:tasks.workunit.client.0.smithi100.stdout:op 4305 completed, throughput=4.99MB/sec 2024-04-07T16:52:55.166 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-KS1tnr_nDjNSHpM off=0 len=1171587 2024-04-07T16:52:55.176 INFO:tasks.workunit.client.0.smithi100.stdout:op 4306 completed, throughput=5MB/sec 2024-04-07T16:52:55.176 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-R1Z-m700BUzX1On off=3686105 len=2001512 2024-04-07T16:52:55.196 INFO:tasks.workunit.client.0.smithi100.stdout:op 4307 completed, throughput=5MB/sec 2024-04-07T16:52:55.196 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-_W-ixq7Iey9Iwsy off=12486036 len=991604 2024-04-07T16:52:55.204 INFO:tasks.workunit.client.0.smithi100.stdout:op 4308 completed, throughput=5MB/sec 2024-04-07T16:52:55.204 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-sLtetHvzVAIy0Bf off=15333606 len=1488602 2024-04-07T16:52:55.216 INFO:tasks.workunit.client.0.smithi100.stdout:op 4309 completed, throughput=5MB/sec 2024-04-07T16:52:56.217 INFO:tasks.workunit.client.0.smithi100.stdout: 832: throughput=4.99MB/sec pending data=0 2024-04-07T16:52:56.217 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Qatule9fNnKzoaI off=14622803 len=2090286 2024-04-07T16:52:56.222 INFO:tasks.workunit.client.0.smithi100.stdout:op 4310 completed, throughput=5MB/sec 2024-04-07T16:52:56.222 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-CIagMSkomQk_7gu off=22525513 len=1114545 2024-04-07T16:52:56.232 INFO:tasks.workunit.client.0.smithi100.stdout:op 4311 completed, throughput=5MB/sec 2024-04-07T16:52:56.232 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-4d8f5mLsQrpg8At off=3920974 len=1895063 2024-04-07T16:52:56.238 INFO:tasks.workunit.client.0.smithi100.stdout:op 4312 completed, throughput=5MB/sec 2024-04-07T16:52:56.948 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:52:56 smithi100 bash[74942]: cluster 2024-04-07T16:52:55.510619+0000 mgr.y (mgr.25175) 1127 : cluster [DBG] pgmap v1769: 105 pgs: 105 active+clean; 126 GiB data, 9.1 GiB used, 706 GiB / 715 GiB avail; 3.2 MiB/s rd, 1.3 MiB/s wr, 4 op/s 2024-04-07T16:52:56.949 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:52:56 smithi100 bash[76202]: cluster 2024-04-07T16:52:55.510619+0000 mgr.y (mgr.25175) 1127 : cluster [DBG] pgmap v1769: 105 pgs: 105 active+clean; 126 GiB data, 9.1 GiB used, 706 GiB / 715 GiB avail; 3.2 MiB/s rd, 1.3 MiB/s wr, 4 op/s 2024-04-07T16:52:57.112 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:52:56 smithi144 bash[66654]: cluster 2024-04-07T16:52:55.510619+0000 mgr.y (mgr.25175) 1127 : cluster [DBG] pgmap v1769: 105 pgs: 105 active+clean; 126 GiB data, 9.1 GiB used, 706 GiB / 715 GiB avail; 3.2 MiB/s rd, 1.3 MiB/s wr, 4 op/s 2024-04-07T16:52:57.238 INFO:tasks.workunit.client.0.smithi100.stdout: 833: throughput=4.99MB/sec pending data=0 2024-04-07T16:52:57.238 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-hShModTapz5Os63 off=306823 len=1821587 2024-04-07T16:52:57.243 INFO:tasks.workunit.client.0.smithi100.stdout:op 4313 completed, throughput=5MB/sec 2024-04-07T16:52:57.243 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-mdnu2O1BqOa7yI- off=2518521 len=312268 2024-04-07T16:52:57.245 INFO:tasks.workunit.client.0.smithi100.stdout:op 4314 completed, throughput=5MB/sec 2024-04-07T16:52:57.245 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-5xcd7vuYCZuu_e8 off=12074435 len=700616 2024-04-07T16:52:57.252 INFO:tasks.workunit.client.0.smithi100.stdout:op 4315 completed, throughput=5MB/sec 2024-04-07T16:52:57.252 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-df0dPcM3lAHer-L off=11616880 len=516425 2024-04-07T16:52:57.259 INFO:tasks.workunit.client.0.smithi100.stdout:op 4316 completed, throughput=5MB/sec 2024-04-07T16:52:57.259 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-LLO6o6UEhKzLdgE off=7605275 len=1773540 2024-04-07T16:52:57.264 INFO:tasks.workunit.client.0.smithi100.stdout:op 4317 completed, throughput=5MB/sec 2024-04-07T16:52:57.948 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:52:57 smithi100 bash[76202]: cluster 2024-04-07T16:52:57.511691+0000 mgr.y (mgr.25175) 1128 : cluster [DBG] pgmap v1770: 105 pgs: 105 active+clean; 126 GiB data, 9.1 GiB used, 706 GiB / 715 GiB avail; 3.1 MiB/s rd, 1.7 MiB/s wr, 4 op/s 2024-04-07T16:52:57.948 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:52:57 smithi100 bash[74942]: cluster 2024-04-07T16:52:57.511691+0000 mgr.y (mgr.25175) 1128 : cluster [DBG] pgmap v1770: 105 pgs: 105 active+clean; 126 GiB data, 9.1 GiB used, 706 GiB / 715 GiB avail; 3.1 MiB/s rd, 1.7 MiB/s wr, 4 op/s 2024-04-07T16:52:58.112 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:52:57 smithi144 bash[66654]: cluster 2024-04-07T16:52:57.511691+0000 mgr.y (mgr.25175) 1128 : cluster [DBG] pgmap v1770: 105 pgs: 105 active+clean; 126 GiB data, 9.1 GiB used, 706 GiB / 715 GiB avail; 3.1 MiB/s rd, 1.7 MiB/s wr, 4 op/s 2024-04-07T16:52:58.264 INFO:tasks.workunit.client.0.smithi100.stdout: 834: throughput=4.99MB/sec pending data=0 2024-04-07T16:52:58.264 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-CPoz2Z1yzJrY2JA off=4199409 len=2073241 2024-04-07T16:52:58.269 INFO:tasks.workunit.client.0.smithi100.stdout:op 4318 completed, throughput=5MB/sec 2024-04-07T16:52:58.269 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-OKs_OPumEmeOw9a off=15300971 len=1141340 2024-04-07T16:52:58.279 INFO:tasks.workunit.client.0.smithi100.stdout:op 4319 completed, throughput=5MB/sec 2024-04-07T16:52:58.279 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-5N94LQ3U2ib34r4 off=4303828 len=1054446 2024-04-07T16:52:58.281 INFO:tasks.workunit.client.0.smithi100.stdout:op 4320 completed, throughput=5MB/sec 2024-04-07T16:52:58.281 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-g0gEnhMye9TEzIT off=23006469 len=1354493 2024-04-07T16:52:58.291 INFO:tasks.workunit.client.0.smithi100.stdout:op 4321 completed, throughput=5MB/sec 2024-04-07T16:52:58.698 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:52:58 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:52:58] "GET /metrics HTTP/1.1" 200 34070 "" "Prometheus/2.43.0" 2024-04-07T16:52:59.292 INFO:tasks.workunit.client.0.smithi100.stdout: 835: throughput=4.99MB/sec pending data=0 2024-04-07T16:52:59.292 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-5Mz6bOEvmnzkRoq off=7959664 len=1002291 2024-04-07T16:52:59.301 INFO:tasks.workunit.client.0.smithi100.stdout:op 4322 completed, throughput=5MB/sec 2024-04-07T16:52:59.301 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-vzAHP-dJ1szoDaV off=5865006 len=1703052 2024-04-07T16:52:59.305 INFO:tasks.workunit.client.0.smithi100.stdout:op 4323 completed, throughput=5MB/sec 2024-04-07T16:52:59.305 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-9HouQLiiqE5mRuh off=1640516 len=744744 2024-04-07T16:52:59.307 INFO:tasks.workunit.client.0.smithi100.stdout:op 4324 completed, throughput=5MB/sec 2024-04-07T16:52:59.307 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Lw3OBpq_wNjHIwS off=7786573 len=1307415 2024-04-07T16:52:59.319 INFO:tasks.workunit.client.0.smithi100.stdout:op 4325 completed, throughput=5MB/sec 2024-04-07T16:52:59.319 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-fs2RRzVRtzSIuBo off=13751256 len=568968 2024-04-07T16:52:59.321 INFO:tasks.workunit.client.0.smithi100.stdout:op 4326 completed, throughput=5MB/sec 2024-04-07T16:53:00.321 INFO:tasks.workunit.client.0.smithi100.stdout: 836: throughput=4.99MB/sec pending data=0 2024-04-07T16:53:00.321 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-0CzGjGKh15WAQJ_ off=9877940 len=1467221 2024-04-07T16:53:00.334 INFO:tasks.workunit.client.0.smithi100.stdout:op 4327 completed, throughput=5MB/sec 2024-04-07T16:53:00.334 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-xELTUdB0-TsUSOK off=7355822 len=106410 2024-04-07T16:53:00.336 INFO:tasks.workunit.client.0.smithi100.stdout:op 4328 completed, throughput=5MB/sec 2024-04-07T16:53:00.336 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-AqjEDplrfG5ei8l off=13132815 len=322561 2024-04-07T16:53:00.337 INFO:tasks.workunit.client.0.smithi100.stdout:op 4329 completed, throughput=5MB/sec 2024-04-07T16:53:00.338 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-4VqM7w1ugLUq_yB off=7269105 len=152874 2024-04-07T16:53:00.339 INFO:tasks.workunit.client.0.smithi100.stdout:op 4330 completed, throughput=5MB/sec 2024-04-07T16:53:00.339 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-WfhzmJNWwVKnwIh off=12332038 len=2016765 2024-04-07T16:53:00.354 INFO:tasks.workunit.client.0.smithi100.stdout:op 4331 completed, throughput=5MB/sec 2024-04-07T16:53:00.354 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-c0lxRG3QFAvXVcp off=17001595 len=1249543 2024-04-07T16:53:00.365 INFO:tasks.workunit.client.0.smithi100.stdout:op 4332 completed, throughput=5MB/sec 2024-04-07T16:53:00.862 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:53:00 smithi144 bash[66654]: cluster 2024-04-07T16:52:59.512582+0000 mgr.y (mgr.25175) 1129 : cluster [DBG] pgmap v1771: 105 pgs: 105 active+clean; 126 GiB data, 9.1 GiB used, 706 GiB / 715 GiB avail; 2.8 MiB/s rd, 1.8 MiB/s wr, 4 op/s 2024-04-07T16:53:00.948 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:53:00 smithi100 bash[74942]: cluster 2024-04-07T16:52:59.512582+0000 mgr.y (mgr.25175) 1129 : cluster [DBG] pgmap v1771: 105 pgs: 105 active+clean; 126 GiB data, 9.1 GiB used, 706 GiB / 715 GiB avail; 2.8 MiB/s rd, 1.8 MiB/s wr, 4 op/s 2024-04-07T16:53:00.948 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:53:00 smithi100 bash[76202]: cluster 2024-04-07T16:52:59.512582+0000 mgr.y (mgr.25175) 1129 : cluster [DBG] pgmap v1771: 105 pgs: 105 active+clean; 126 GiB data, 9.1 GiB used, 706 GiB / 715 GiB avail; 2.8 MiB/s rd, 1.8 MiB/s wr, 4 op/s 2024-04-07T16:53:01.365 INFO:tasks.workunit.client.0.smithi100.stdout: 837: throughput=4.99MB/sec pending data=0 2024-04-07T16:53:01.365 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-ItlLN81_b1uOKAp off=7917189 len=975149 2024-04-07T16:53:01.374 INFO:tasks.workunit.client.0.smithi100.stdout:op 4333 completed, throughput=5MB/sec 2024-04-07T16:53:01.374 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-qKPjLetNtJ6oKl5 off=5143780 len=1785633 2024-04-07T16:53:01.378 INFO:tasks.workunit.client.0.smithi100.stdout:op 4334 completed, throughput=5MB/sec 2024-04-07T16:53:01.378 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-0Z7b8gY2DpyPIjI off=20242128 len=241087 2024-04-07T16:53:01.383 INFO:tasks.workunit.client.0.smithi100.stdout:op 4335 completed, throughput=5MB/sec 2024-04-07T16:53:01.383 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-oymR-cOryexU0lL off=1462730 len=1093394 2024-04-07T16:53:01.394 INFO:tasks.workunit.client.0.smithi100.stdout:op 4336 completed, throughput=5MB/sec 2024-04-07T16:53:01.394 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-aIP5Xiq_W6wDd0M off=22320110 len=347776 2024-04-07T16:53:01.395 INFO:tasks.workunit.client.0.smithi100.stdout:op 4337 completed, throughput=5MB/sec 2024-04-07T16:53:01.395 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-9LFgGUURE0ah9Er off=207621 len=2014564 2024-04-07T16:53:01.400 INFO:tasks.workunit.client.0.smithi100.stdout:op 4338 completed, throughput=5MB/sec 2024-04-07T16:53:02.400 INFO:tasks.workunit.client.0.smithi100.stdout: 838: throughput=5MB/sec pending data=0 2024-04-07T16:53:02.400 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-k69DK-9gH99LP2f off=5282297 len=1066051 2024-04-07T16:53:02.409 INFO:tasks.workunit.client.0.smithi100.stdout:op 4339 completed, throughput=5MB/sec 2024-04-07T16:53:02.409 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-F9NyNcFo4S9SfnB off=7223967 len=503355 2024-04-07T16:53:02.411 INFO:tasks.workunit.client.0.smithi100.stdout:op 4340 completed, throughput=5MB/sec 2024-04-07T16:53:02.411 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-2ExLALuaV5tZn6F off=2551116 len=262176 2024-04-07T16:53:02.413 INFO:tasks.workunit.client.0.smithi100.stdout:op 4341 completed, throughput=5MB/sec 2024-04-07T16:53:02.413 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-PUwgQdXFiK1YjSy off=4572603 len=636840 2024-04-07T16:53:02.420 INFO:tasks.workunit.client.0.smithi100.stdout:op 4342 completed, throughput=5MB/sec 2024-04-07T16:53:02.420 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj--EmE61dV9EcirgB off=1671504 len=693519 2024-04-07T16:53:02.422 INFO:tasks.workunit.client.0.smithi100.stdout:op 4343 completed, throughput=5MB/sec 2024-04-07T16:53:02.422 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-ScFj6EJMwP37p97 off=4640996 len=1811964 2024-04-07T16:53:02.426 INFO:tasks.workunit.client.0.smithi100.stdout:op 4344 completed, throughput=5MB/sec 2024-04-07T16:53:02.571 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:53:02 smithi144 bash[95233]: ts=2024-04-07T16:53:02.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:53:02.862 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:53:02 smithi144 bash[66654]: cluster 2024-04-07T16:53:01.513444+0000 mgr.y (mgr.25175) 1130 : cluster [DBG] pgmap v1772: 105 pgs: 105 active+clean; 126 GiB data, 9.1 GiB used, 706 GiB / 715 GiB avail; 2.9 MiB/s rd, 1.9 MiB/s wr, 4 op/s 2024-04-07T16:53:02.862 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:53:02 smithi144 bash[66654]: audit 2024-04-07T16:53:01.780668+0000 mon.a (mon.0) 2166 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:53:02.948 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:53:02 smithi100 bash[74942]: cluster 2024-04-07T16:53:01.513444+0000 mgr.y (mgr.25175) 1130 : cluster [DBG] pgmap v1772: 105 pgs: 105 active+clean; 126 GiB data, 9.1 GiB used, 706 GiB / 715 GiB avail; 2.9 MiB/s rd, 1.9 MiB/s wr, 4 op/s 2024-04-07T16:53:02.948 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:53:02 smithi100 bash[74942]: audit 2024-04-07T16:53:01.780668+0000 mon.a (mon.0) 2166 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:53:02.948 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:53:02 smithi100 bash[76202]: cluster 2024-04-07T16:53:01.513444+0000 mgr.y (mgr.25175) 1130 : cluster [DBG] pgmap v1772: 105 pgs: 105 active+clean; 126 GiB data, 9.1 GiB used, 706 GiB / 715 GiB avail; 2.9 MiB/s rd, 1.9 MiB/s wr, 4 op/s 2024-04-07T16:53:02.948 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:53:02 smithi100 bash[76202]: audit 2024-04-07T16:53:01.780668+0000 mon.a (mon.0) 2166 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:53:03.426 INFO:tasks.workunit.client.0.smithi100.stdout: 839: throughput=4.99MB/sec pending data=0 2024-04-07T16:53:03.427 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-uHrd30LgusZvSN5 off=899558 len=400490 2024-04-07T16:53:03.433 INFO:tasks.workunit.client.0.smithi100.stdout:op 4345 completed, throughput=5MB/sec 2024-04-07T16:53:03.433 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-J5qvXJSaBfcwMpi off=10998119 len=1343007 2024-04-07T16:53:03.437 INFO:tasks.workunit.client.0.smithi100.stdout:op 4346 completed, throughput=5MB/sec 2024-04-07T16:53:03.437 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-9pju3O55Br6zFLj off=8431646 len=1376133 2024-04-07T16:53:03.448 INFO:tasks.workunit.client.0.smithi100.stdout:op 4347 completed, throughput=5MB/sec 2024-04-07T16:53:03.448 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-ETqYxWZDU9EOVPz off=20153880 len=1875439 2024-04-07T16:53:03.450 INFO:tasks.workunit.client.0.smithi100.stdout:op 4348 completed, throughput=5MB/sec 2024-04-07T16:53:04.451 INFO:tasks.workunit.client.0.smithi100.stdout: 840: throughput=4.99MB/sec pending data=0 2024-04-07T16:53:04.451 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-XBvkX8y17MUKzKt off=2385055 len=1611562 2024-04-07T16:53:04.464 INFO:tasks.workunit.client.0.smithi100.stdout:op 4349 completed, throughput=5MB/sec 2024-04-07T16:53:04.464 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Z1jBayL6xhz7J7N off=10190992 len=735070 2024-04-07T16:53:04.466 INFO:tasks.workunit.client.0.smithi100.stdout:op 4350 completed, throughput=5MB/sec 2024-04-07T16:53:04.466 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-F-9B0GAchUr9b1h off=1500909 len=839802 2024-04-07T16:53:04.469 INFO:tasks.workunit.client.0.smithi100.stdout:op 4351 completed, throughput=5MB/sec 2024-04-07T16:53:04.469 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-uxKy9D_h0ZwpiBv off=2786052 len=466588 2024-04-07T16:53:04.470 INFO:tasks.workunit.client.0.smithi100.stdout:op 4352 completed, throughput=5MB/sec 2024-04-07T16:53:04.470 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-svYEbzH8SdSmAE8 off=17898983 len=44416 2024-04-07T16:53:04.475 INFO:tasks.workunit.client.0.smithi100.stdout:op 4353 completed, throughput=5MB/sec 2024-04-07T16:53:04.475 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-OE3fs8BP9kVy45O off=108918 len=1278636 2024-04-07T16:53:04.489 INFO:tasks.workunit.client.0.smithi100.stdout:op 4354 completed, throughput=5MB/sec 2024-04-07T16:53:04.489 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-fW0aKqgY5fjwX2i off=9777452 len=2014289 2024-04-07T16:53:04.505 INFO:tasks.workunit.client.0.smithi100.stdout:op 4355 completed, throughput=5MB/sec 2024-04-07T16:53:04.862 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:53:04 smithi144 bash[66654]: cluster 2024-04-07T16:53:03.514538+0000 mgr.y (mgr.25175) 1131 : cluster [DBG] pgmap v1773: 105 pgs: 105 active+clean; 126 GiB data, 9.2 GiB used, 706 GiB / 715 GiB avail; 3.1 MiB/s rd, 2.4 MiB/s wr, 5 op/s 2024-04-07T16:53:04.948 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:53:04 smithi100 bash[74942]: cluster 2024-04-07T16:53:03.514538+0000 mgr.y (mgr.25175) 1131 : cluster [DBG] pgmap v1773: 105 pgs: 105 active+clean; 126 GiB data, 9.2 GiB used, 706 GiB / 715 GiB avail; 3.1 MiB/s rd, 2.4 MiB/s wr, 5 op/s 2024-04-07T16:53:04.948 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:53:04 smithi100 bash[76202]: cluster 2024-04-07T16:53:03.514538+0000 mgr.y (mgr.25175) 1131 : cluster [DBG] pgmap v1773: 105 pgs: 105 active+clean; 126 GiB data, 9.2 GiB used, 706 GiB / 715 GiB avail; 3.1 MiB/s rd, 2.4 MiB/s wr, 5 op/s 2024-04-07T16:53:05.505 INFO:tasks.workunit.client.0.smithi100.stdout: 841: throughput=5MB/sec pending data=0 2024-04-07T16:53:05.505 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Po3zwMXnl54gf68 off=315036 len=2095186 2024-04-07T16:53:05.510 INFO:tasks.workunit.client.0.smithi100.stdout:op 4356 completed, throughput=5MB/sec 2024-04-07T16:53:05.510 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-fjiCf5AS_4miK1e off=5026165 len=819109 2024-04-07T16:53:05.519 INFO:tasks.workunit.client.0.smithi100.stdout:op 4357 completed, throughput=5MB/sec 2024-04-07T16:53:05.519 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-5bdO6ARdxdLPlTN off=425373 len=1016112 2024-04-07T16:53:05.523 INFO:tasks.workunit.client.0.smithi100.stdout:op 4358 completed, throughput=5MB/sec 2024-04-07T16:53:06.523 INFO:tasks.workunit.client.0.smithi100.stdout: 842: throughput=4.99MB/sec pending data=0 2024-04-07T16:53:06.523 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-9BMzde8cx81RfFU off=1214884 len=325889 2024-04-07T16:53:06.528 INFO:tasks.workunit.client.0.smithi100.stdout:op 4359 completed, throughput=4.99MB/sec 2024-04-07T16:53:06.528 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-vhxMnNPBKmyK_gf off=4494738 len=551353 2024-04-07T16:53:06.533 INFO:tasks.workunit.client.0.smithi100.stdout:op 4360 completed, throughput=5MB/sec 2024-04-07T16:53:06.533 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-IGTebyOg3tas9n3 off=2692194 len=1383861 2024-04-07T16:53:06.544 INFO:tasks.workunit.client.0.smithi100.stdout:op 4361 completed, throughput=5MB/sec 2024-04-07T16:53:06.544 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-HoDWBZOmYUR1J1C off=8611560 len=1903511 2024-04-07T16:53:06.549 INFO:tasks.workunit.client.0.smithi100.stdout:op 4362 completed, throughput=5MB/sec 2024-04-07T16:53:06.549 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-EO70cbFPX8qY9zV off=11757849 len=616399 2024-04-07T16:53:06.555 INFO:tasks.workunit.client.0.smithi100.stdout:op 4363 completed, throughput=5MB/sec 2024-04-07T16:53:06.555 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-ZqfO9zKFoKCh9e9 off=15718347 len=734092 2024-04-07T16:53:06.557 INFO:tasks.workunit.client.0.smithi100.stdout:op 4364 completed, throughput=5MB/sec 2024-04-07T16:53:06.862 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:53:06 smithi144 bash[66654]: cluster 2024-04-07T16:53:05.515674+0000 mgr.y (mgr.25175) 1132 : cluster [DBG] pgmap v1774: 105 pgs: 105 active+clean; 126 GiB data, 9.2 GiB used, 706 GiB / 715 GiB avail; 3.0 MiB/s rd, 2.6 MiB/s wr, 5 op/s 2024-04-07T16:53:06.862 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:53:06 smithi144 bash[66654]: audit 2024-04-07T16:53:06.156485+0000 mon.a (mon.0) 2167 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:53:06.948 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:53:06 smithi100 bash[76202]: cluster 2024-04-07T16:53:05.515674+0000 mgr.y (mgr.25175) 1132 : cluster [DBG] pgmap v1774: 105 pgs: 105 active+clean; 126 GiB data, 9.2 GiB used, 706 GiB / 715 GiB avail; 3.0 MiB/s rd, 2.6 MiB/s wr, 5 op/s 2024-04-07T16:53:06.948 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:53:06 smithi100 bash[76202]: audit 2024-04-07T16:53:06.156485+0000 mon.a (mon.0) 2167 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:53:06.949 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:53:06 smithi100 bash[74942]: cluster 2024-04-07T16:53:05.515674+0000 mgr.y (mgr.25175) 1132 : cluster [DBG] pgmap v1774: 105 pgs: 105 active+clean; 126 GiB data, 9.2 GiB used, 706 GiB / 715 GiB avail; 3.0 MiB/s rd, 2.6 MiB/s wr, 5 op/s 2024-04-07T16:53:06.949 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:53:06 smithi100 bash[74942]: audit 2024-04-07T16:53:06.156485+0000 mon.a (mon.0) 2167 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:53:07.557 INFO:tasks.workunit.client.0.smithi100.stdout: 843: throughput=4.99MB/sec pending data=0 2024-04-07T16:53:07.557 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Pzf6HGRAYzkqg_x off=18527922 len=550392 2024-04-07T16:53:07.564 INFO:tasks.workunit.client.0.smithi100.stdout:op 4365 completed, throughput=5MB/sec 2024-04-07T16:53:07.565 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-5ZI1hbgQUUioJ-M off=753607 len=1569766 2024-04-07T16:53:07.568 INFO:tasks.workunit.client.0.smithi100.stdout:op 4366 completed, throughput=5MB/sec 2024-04-07T16:53:07.568 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-HiHh7EHacx7D55h off=11475007 len=1212582 2024-04-07T16:53:07.580 INFO:tasks.workunit.client.0.smithi100.stdout:op 4367 completed, throughput=5MB/sec 2024-04-07T16:53:07.580 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-J7C2sVeMRn_lSSf off=1337625 len=1184328 2024-04-07T16:53:07.584 INFO:tasks.workunit.client.0.smithi100.stdout:op 4368 completed, throughput=5MB/sec 2024-04-07T16:53:07.584 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-AM4A6yVYw6ftf0R off=4944174 len=320320 2024-04-07T16:53:07.590 INFO:tasks.workunit.client.0.smithi100.stdout:op 4369 completed, throughput=5MB/sec 2024-04-07T16:53:07.591 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-cEhErk5pMLZlskb off=2513600 len=1635014 2024-04-07T16:53:07.594 INFO:tasks.workunit.client.0.smithi100.stdout:op 4370 completed, throughput=5MB/sec 2024-04-07T16:53:07.862 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:53:07 smithi144 bash[66654]: audit 2024-04-07T16:53:06.893605+0000 mon.a (mon.0) 2168 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:53:07.862 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:53:07 smithi144 bash[66654]: audit 2024-04-07T16:53:06.895224+0000 mon.a (mon.0) 2169 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:53:07.863 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:53:07 smithi144 bash[66654]: audit 2024-04-07T16:53:06.906347+0000 mon.a (mon.0) 2170 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:53:07.948 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:53:07 smithi100 bash[74942]: audit 2024-04-07T16:53:06.893605+0000 mon.a (mon.0) 2168 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:53:07.949 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:53:07 smithi100 bash[74942]: audit 2024-04-07T16:53:06.895224+0000 mon.a (mon.0) 2169 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:53:07.949 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:53:07 smithi100 bash[74942]: audit 2024-04-07T16:53:06.906347+0000 mon.a (mon.0) 2170 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:53:07.949 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:53:07 smithi100 bash[76202]: audit 2024-04-07T16:53:06.893605+0000 mon.a (mon.0) 2168 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:53:07.949 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:53:07 smithi100 bash[76202]: audit 2024-04-07T16:53:06.895224+0000 mon.a (mon.0) 2169 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:53:07.949 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:53:07 smithi100 bash[76202]: audit 2024-04-07T16:53:06.906347+0000 mon.a (mon.0) 2170 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:53:08.595 INFO:tasks.workunit.client.0.smithi100.stdout: 844: throughput=5MB/sec pending data=0 2024-04-07T16:53:08.595 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-5ffJcRmBM84X97- off=16896618 len=1682749 2024-04-07T16:53:08.610 INFO:tasks.workunit.client.0.smithi100.stdout:op 4371 completed, throughput=5MB/sec 2024-04-07T16:53:08.610 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-GgNJtVuWzCLd3N_ off=2504203 len=1267568 2024-04-07T16:53:08.619 INFO:tasks.workunit.client.0.smithi100.stdout:op 4372 completed, throughput=5MB/sec 2024-04-07T16:53:08.619 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-A5JMUYMdwHyysH4 off=549147 len=864633 2024-04-07T16:53:08.622 INFO:tasks.workunit.client.0.smithi100.stdout:op 4373 completed, throughput=5MB/sec 2024-04-07T16:53:08.622 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-gvEIABDx1NPQomW off=7077863 len=1932774 2024-04-07T16:53:08.627 INFO:tasks.workunit.client.0.smithi100.stdout:op 4374 completed, throughput=5MB/sec 2024-04-07T16:53:08.698 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:53:08 smithi100 bash[74942]: cluster 2024-04-07T16:53:07.516702+0000 mgr.y (mgr.25175) 1133 : cluster [DBG] pgmap v1775: 105 pgs: 105 active+clean; 126 GiB data, 9.2 GiB used, 706 GiB / 715 GiB avail; 2.6 MiB/s rd, 2.5 MiB/s wr, 4 op/s 2024-04-07T16:53:08.699 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:53:08 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:53:08] "GET /metrics HTTP/1.1" 200 34071 "" "Prometheus/2.43.0" 2024-04-07T16:53:08.699 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:53:08 smithi100 bash[76202]: cluster 2024-04-07T16:53:07.516702+0000 mgr.y (mgr.25175) 1133 : cluster [DBG] pgmap v1775: 105 pgs: 105 active+clean; 126 GiB data, 9.2 GiB used, 706 GiB / 715 GiB avail; 2.6 MiB/s rd, 2.5 MiB/s wr, 4 op/s 2024-04-07T16:53:08.862 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:53:08 smithi144 bash[66654]: cluster 2024-04-07T16:53:07.516702+0000 mgr.y (mgr.25175) 1133 : cluster [DBG] pgmap v1775: 105 pgs: 105 active+clean; 126 GiB data, 9.2 GiB used, 706 GiB / 715 GiB avail; 2.6 MiB/s rd, 2.5 MiB/s wr, 4 op/s 2024-04-07T16:53:09.628 INFO:tasks.workunit.client.0.smithi100.stdout: 845: throughput=5MB/sec pending data=0 2024-04-07T16:53:09.628 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-p3gp4MByPCnSZJ- off=563214 len=571796 2024-04-07T16:53:09.630 INFO:tasks.workunit.client.0.smithi100.stdout:op 4375 completed, throughput=5MB/sec 2024-04-07T16:53:09.630 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-8k99TxK2Lw-hH2n off=7938575 len=252631 2024-04-07T16:53:09.632 INFO:tasks.workunit.client.0.smithi100.stdout:op 4376 completed, throughput=5MB/sec 2024-04-07T16:53:09.632 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-SDYJdINLFhGug0a off=454994 len=1112949 2024-04-07T16:53:09.642 INFO:tasks.workunit.client.0.smithi100.stdout:op 4377 completed, throughput=5MB/sec 2024-04-07T16:53:09.642 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-6KRQSLYiSfmOuYa off=2817907 len=542485 2024-04-07T16:53:09.648 INFO:tasks.workunit.client.0.smithi100.stdout:op 4378 completed, throughput=5MB/sec 2024-04-07T16:53:09.649 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-_Cx5uzVT3IcVRiq off=5297266 len=459648 2024-04-07T16:53:09.650 INFO:tasks.workunit.client.0.smithi100.stdout:op 4379 completed, throughput=5MB/sec 2024-04-07T16:53:09.651 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-9l-BZ6yDvhSy78i off=2671572 len=418183 2024-04-07T16:53:09.652 INFO:tasks.workunit.client.0.smithi100.stdout:op 4380 completed, throughput=5MB/sec 2024-04-07T16:53:09.652 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-x6oMNwbRoIACrgF off=7212897 len=1690616 2024-04-07T16:53:09.657 INFO:tasks.workunit.client.0.smithi100.stdout:op 4381 completed, throughput=5MB/sec 2024-04-07T16:53:10.657 INFO:tasks.workunit.client.0.smithi100.stdout: 846: throughput=5MB/sec pending data=0 2024-04-07T16:53:10.658 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-8YYEhWSgdGuBNrT off=3820777 len=492697 2024-04-07T16:53:10.664 INFO:tasks.workunit.client.0.smithi100.stdout:op 4382 completed, throughput=5MB/sec 2024-04-07T16:53:10.664 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-XUMwgf1ASj0zjtC off=2999792 len=631155 2024-04-07T16:53:10.672 INFO:tasks.workunit.client.0.smithi100.stdout:op 4383 completed, throughput=5MB/sec 2024-04-07T16:53:10.672 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-10ercpgJdT6Qr_i off=1046320 len=1122429 2024-04-07T16:53:10.676 INFO:tasks.workunit.client.0.smithi100.stdout:op 4384 completed, throughput=5MB/sec 2024-04-07T16:53:10.676 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-pPBIdN3mcnkYCyH off=15754487 len=742321 2024-04-07T16:53:10.677 INFO:tasks.workunit.client.0.smithi100.stdout:op 4385 completed, throughput=5MB/sec 2024-04-07T16:53:10.678 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-iQDtWeCjNUrn3Ck off=5400851 len=849679 2024-04-07T16:53:10.680 INFO:tasks.workunit.client.0.smithi100.stdout:op 4386 completed, throughput=5MB/sec 2024-04-07T16:53:10.680 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Oxt-eGV83FHlRze off=3247883 len=2067139 2024-04-07T16:53:10.685 INFO:tasks.workunit.client.0.smithi100.stdout:op 4387 completed, throughput=5MB/sec 2024-04-07T16:53:10.862 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:53:10 smithi144 bash[66654]: cluster 2024-04-07T16:53:09.517644+0000 mgr.y (mgr.25175) 1134 : cluster [DBG] pgmap v1776: 105 pgs: 105 active+clean; 126 GiB data, 9.2 GiB used, 706 GiB / 715 GiB avail; 2.6 MiB/s rd, 2.3 MiB/s wr, 4 op/s 2024-04-07T16:53:10.948 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:53:10 smithi100 bash[74942]: cluster 2024-04-07T16:53:09.517644+0000 mgr.y (mgr.25175) 1134 : cluster [DBG] pgmap v1776: 105 pgs: 105 active+clean; 126 GiB data, 9.2 GiB used, 706 GiB / 715 GiB avail; 2.6 MiB/s rd, 2.3 MiB/s wr, 4 op/s 2024-04-07T16:53:10.948 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:53:10 smithi100 bash[76202]: cluster 2024-04-07T16:53:09.517644+0000 mgr.y (mgr.25175) 1134 : cluster [DBG] pgmap v1776: 105 pgs: 105 active+clean; 126 GiB data, 9.2 GiB used, 706 GiB / 715 GiB avail; 2.6 MiB/s rd, 2.3 MiB/s wr, 4 op/s 2024-04-07T16:53:11.685 INFO:tasks.workunit.client.0.smithi100.stdout: 847: throughput=5MB/sec pending data=0 2024-04-07T16:53:11.686 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-IWEV7RhBY_ZavPm off=16096820 len=378826 2024-04-07T16:53:11.692 INFO:tasks.workunit.client.0.smithi100.stdout:op 4388 completed, throughput=5MB/sec 2024-04-07T16:53:11.692 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Qsg66r03K02yR8o off=7035727 len=654218 2024-04-07T16:53:11.693 INFO:tasks.workunit.client.0.smithi100.stdout:op 4389 completed, throughput=5MB/sec 2024-04-07T16:53:11.693 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-A6H-bJhjrUMaL9L off=0 len=1124474 2024-04-07T16:53:11.696 INFO:tasks.workunit.client.0.smithi100.stdout:op 4390 completed, throughput=5MB/sec 2024-04-07T16:53:11.696 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-KAbOW-4_H-FYzwl off=7133433 len=617647 2024-04-07T16:53:11.699 INFO:tasks.workunit.client.0.smithi100.stdout:op 4391 completed, throughput=5MB/sec 2024-04-07T16:53:11.699 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Oc6Je097kYXeD5a off=4812046 len=1389472 2024-04-07T16:53:11.710 INFO:tasks.workunit.client.0.smithi100.stdout:op 4392 completed, throughput=5MB/sec 2024-04-07T16:53:12.591 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:53:12 smithi144 bash[95233]: ts=2024-04-07T16:53:12.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:53:12.711 INFO:tasks.workunit.client.0.smithi100.stdout: 848: throughput=5MB/sec pending data=0 2024-04-07T16:53:12.711 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-AS6QyVAfxwDPbKe off=19642073 len=1579094 2024-04-07T16:53:12.715 INFO:tasks.workunit.client.0.smithi100.stdout:op 4393 completed, throughput=5MB/sec 2024-04-07T16:53:12.715 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-1EmpHFCcfAk8hAo off=3236692 len=1136307 2024-04-07T16:53:12.718 INFO:tasks.workunit.client.0.smithi100.stdout:op 4394 completed, throughput=5MB/sec 2024-04-07T16:53:12.718 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-AqBaoF0brMNJRe7 off=1271704 len=2054167 2024-04-07T16:53:12.735 INFO:tasks.workunit.client.0.smithi100.stdout:op 4395 completed, throughput=5MB/sec 2024-04-07T16:53:12.862 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:53:12 smithi144 bash[66654]: cluster 2024-04-07T16:53:11.518418+0000 mgr.y (mgr.25175) 1135 : cluster [DBG] pgmap v1777: 105 pgs: 105 active+clean; 126 GiB data, 9.2 GiB used, 706 GiB / 715 GiB avail; 2.8 MiB/s rd, 2.3 MiB/s wr, 5 op/s 2024-04-07T16:53:12.948 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:53:12 smithi100 bash[74942]: cluster 2024-04-07T16:53:11.518418+0000 mgr.y (mgr.25175) 1135 : cluster [DBG] pgmap v1777: 105 pgs: 105 active+clean; 126 GiB data, 9.2 GiB used, 706 GiB / 715 GiB avail; 2.8 MiB/s rd, 2.3 MiB/s wr, 5 op/s 2024-04-07T16:53:12.948 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:53:12 smithi100 bash[76202]: cluster 2024-04-07T16:53:11.518418+0000 mgr.y (mgr.25175) 1135 : cluster [DBG] pgmap v1777: 105 pgs: 105 active+clean; 126 GiB data, 9.2 GiB used, 706 GiB / 715 GiB avail; 2.8 MiB/s rd, 2.3 MiB/s wr, 5 op/s 2024-04-07T16:53:13.736 INFO:tasks.workunit.client.0.smithi100.stdout: 849: throughput=4.99MB/sec pending data=0 2024-04-07T16:53:13.736 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-zuLmqR0-DWrYcRt off=3562323 len=1452724 2024-04-07T16:53:13.748 INFO:tasks.workunit.client.0.smithi100.stdout:op 4396 completed, throughput=5MB/sec 2024-04-07T16:53:13.748 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-VuffuGdJ9yPAeBu off=22738144 len=22107 2024-04-07T16:53:13.749 INFO:tasks.workunit.client.0.smithi100.stdout:op 4397 completed, throughput=5MB/sec 2024-04-07T16:53:13.749 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-j7mJk8FZvEDQqKD off=199715 len=944734 2024-04-07T16:53:13.751 INFO:tasks.workunit.client.0.smithi100.stdout:op 4398 completed, throughput=5MB/sec 2024-04-07T16:53:13.751 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-7TzB_YCecNUC0-I off=2152052 len=935637 2024-04-07T16:53:13.761 INFO:tasks.workunit.client.0.smithi100.stdout:op 4399 completed, throughput=5MB/sec 2024-04-07T16:53:13.761 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-m-jKoZz0yFakljz off=20115048 len=363985 2024-04-07T16:53:13.767 INFO:tasks.workunit.client.0.smithi100.stdout:op 4400 completed, throughput=5MB/sec 2024-04-07T16:53:13.767 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-pD6_QgzX47TghaW off=424913 len=1368606 2024-04-07T16:53:13.772 INFO:tasks.workunit.client.0.smithi100.stdout:op 4401 completed, throughput=5MB/sec 2024-04-07T16:53:13.772 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-LbZwxBvL2_rAFHQ off=478081 len=980061 2024-04-07T16:53:13.775 INFO:tasks.workunit.client.0.smithi100.stdout:op 4402 completed, throughput=5MB/sec 2024-04-07T16:53:14.775 INFO:tasks.workunit.client.0.smithi100.stdout: 850: throughput=5MB/sec pending data=0 2024-04-07T16:53:14.775 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-36vj164feVN_i-y off=1410506 len=142157 2024-04-07T16:53:14.780 INFO:tasks.workunit.client.0.smithi100.stdout:op 4403 completed, throughput=5MB/sec 2024-04-07T16:53:14.781 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-HAxGo2xaAOjO8u5 off=2922548 len=841109 2024-04-07T16:53:14.790 INFO:tasks.workunit.client.0.smithi100.stdout:op 4404 completed, throughput=5MB/sec 2024-04-07T16:53:14.790 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-4i-VtQDvM1ffj6D off=1338547 len=109987 2024-04-07T16:53:14.794 INFO:tasks.workunit.client.0.smithi100.stdout:op 4405 completed, throughput=5MB/sec 2024-04-07T16:53:14.794 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-TKjJFXyZq1rbvgg off=7382181 len=1421482 2024-04-07T16:53:14.807 INFO:tasks.workunit.client.0.smithi100.stdout:op 4406 completed, throughput=5MB/sec 2024-04-07T16:53:14.807 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Q4pNwkjmjjqIFf_ off=23692547 len=1582963 2024-04-07T16:53:14.811 INFO:tasks.workunit.client.0.smithi100.stdout:op 4407 completed, throughput=5MB/sec 2024-04-07T16:53:14.811 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-rAKsj0NW1Uvwm3e off=4003538 len=1346562 2024-04-07T16:53:14.821 INFO:tasks.workunit.client.0.smithi100.stdout:op 4408 completed, throughput=5MB/sec 2024-04-07T16:53:14.862 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:53:14 smithi144 bash[66654]: cluster 2024-04-07T16:53:13.519601+0000 mgr.y (mgr.25175) 1136 : cluster [DBG] pgmap v1778: 105 pgs: 105 active+clean; 126 GiB data, 9.2 GiB used, 706 GiB / 715 GiB avail; 3.3 MiB/s rd, 2.5 MiB/s wr, 5 op/s 2024-04-07T16:53:14.948 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:53:14 smithi100 bash[74942]: cluster 2024-04-07T16:53:13.519601+0000 mgr.y (mgr.25175) 1136 : cluster [DBG] pgmap v1778: 105 pgs: 105 active+clean; 126 GiB data, 9.2 GiB used, 706 GiB / 715 GiB avail; 3.3 MiB/s rd, 2.5 MiB/s wr, 5 op/s 2024-04-07T16:53:14.949 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:53:14 smithi100 bash[76202]: cluster 2024-04-07T16:53:13.519601+0000 mgr.y (mgr.25175) 1136 : cluster [DBG] pgmap v1778: 105 pgs: 105 active+clean; 126 GiB data, 9.2 GiB used, 706 GiB / 715 GiB avail; 3.3 MiB/s rd, 2.5 MiB/s wr, 5 op/s 2024-04-07T16:53:15.822 INFO:tasks.workunit.client.0.smithi100.stdout: 851: throughput=5MB/sec pending data=0 2024-04-07T16:53:15.822 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-FoFBn7yxuoORVer off=1185277 len=1064399 2024-04-07T16:53:15.825 INFO:tasks.workunit.client.0.smithi100.stdout:op 4409 completed, throughput=5MB/sec 2024-04-07T16:53:15.825 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-br_VijxPD3X_xN1 off=4678321 len=1974604 2024-04-07T16:53:15.830 INFO:tasks.workunit.client.0.smithi100.stdout:op 4410 completed, throughput=5MB/sec 2024-04-07T16:53:15.830 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-AYZlCQd32m-d36A off=15518454 len=1990829 2024-04-07T16:53:15.847 INFO:tasks.workunit.client.0.smithi100.stdout:op 4411 completed, throughput=5MB/sec 2024-04-07T16:53:16.847 INFO:tasks.workunit.client.0.smithi100.stdout: 852: throughput=4.99MB/sec pending data=0 2024-04-07T16:53:16.848 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-c0gWdcnrl0jtqMf off=398011 len=1323930 2024-04-07T16:53:16.858 INFO:tasks.workunit.client.0.smithi100.stdout:op 4412 completed, throughput=5MB/sec 2024-04-07T16:53:16.858 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj--H8GdKBES7emqMq off=2203981 len=170782 2024-04-07T16:53:16.862 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:53:16 smithi144 bash[66654]: cluster 2024-04-07T16:53:15.520540+0000 mgr.y (mgr.25175) 1137 : cluster [DBG] pgmap v1779: 105 pgs: 105 active+clean; 126 GiB data, 9.2 GiB used, 706 GiB / 715 GiB avail; 2.7 MiB/s rd, 1.9 MiB/s wr, 4 op/s 2024-04-07T16:53:16.864 INFO:tasks.workunit.client.0.smithi100.stdout:op 4413 completed, throughput=5MB/sec 2024-04-07T16:53:16.864 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-MVdVdBkePozIxXa off=171817 len=26114 2024-04-07T16:53:16.865 INFO:tasks.workunit.client.0.smithi100.stdout:op 4414 completed, throughput=5MB/sec 2024-04-07T16:53:16.865 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-9mtgCFMm85Hd-Sq off=2547576 len=291464 2024-04-07T16:53:16.867 INFO:tasks.workunit.client.0.smithi100.stdout:op 4415 completed, throughput=5MB/sec 2024-04-07T16:53:16.867 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-jigEXTU70kl5MTF off=8763597 len=1472270 2024-04-07T16:53:16.872 INFO:tasks.workunit.client.0.smithi100.stdout:op 4416 completed, throughput=5MB/sec 2024-04-07T16:53:16.872 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-HAEfSe9NEoSeIAM off=8218729 len=1219693 2024-04-07T16:53:16.881 INFO:tasks.workunit.client.0.smithi100.stdout:op 4417 completed, throughput=5MB/sec 2024-04-07T16:53:16.881 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-e6alTxkfcVe5Cwc off=13254081 len=1318377 2024-04-07T16:53:16.891 INFO:tasks.workunit.client.0.smithi100.stdout:op 4418 completed, throughput=5MB/sec 2024-04-07T16:53:16.948 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:53:16 smithi100 bash[74942]: cluster 2024-04-07T16:53:15.520540+0000 mgr.y (mgr.25175) 1137 : cluster [DBG] pgmap v1779: 105 pgs: 105 active+clean; 126 GiB data, 9.2 GiB used, 706 GiB / 715 GiB avail; 2.7 MiB/s rd, 1.9 MiB/s wr, 4 op/s 2024-04-07T16:53:16.948 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:53:16 smithi100 bash[76202]: cluster 2024-04-07T16:53:15.520540+0000 mgr.y (mgr.25175) 1137 : cluster [DBG] pgmap v1779: 105 pgs: 105 active+clean; 126 GiB data, 9.2 GiB used, 706 GiB / 715 GiB avail; 2.7 MiB/s rd, 1.9 MiB/s wr, 4 op/s 2024-04-07T16:53:17.862 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:53:17 smithi144 bash[66654]: audit 2024-04-07T16:53:16.781498+0000 mon.a (mon.0) 2171 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:53:17.892 INFO:tasks.workunit.client.0.smithi100.stdout: 853: throughput=5MB/sec pending data=0 2024-04-07T16:53:17.892 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-RI-cOghPI14KOoB off=1927455 len=2070742 2024-04-07T16:53:17.907 INFO:tasks.workunit.client.0.smithi100.stdout:op 4419 completed, throughput=5MB/sec 2024-04-07T16:53:17.907 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-ukhSb4MXgRFZtYf off=772078 len=1782523 2024-04-07T16:53:17.913 INFO:tasks.workunit.client.0.smithi100.stdout:op 4420 completed, throughput=5MB/sec 2024-04-07T16:53:17.913 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-O9D0Ake6QS6ucn6 off=3446496 len=2050164 2024-04-07T16:53:17.917 INFO:tasks.workunit.client.0.smithi100.stdout:op 4421 completed, throughput=5MB/sec 2024-04-07T16:53:17.948 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:53:17 smithi100 bash[74942]: audit 2024-04-07T16:53:16.781498+0000 mon.a (mon.0) 2171 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:53:17.948 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:53:17 smithi100 bash[76202]: audit 2024-04-07T16:53:16.781498+0000 mon.a (mon.0) 2171 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:53:18.698 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:53:18 smithi100 bash[74942]: cluster 2024-04-07T16:53:17.521557+0000 mgr.y (mgr.25175) 1138 : cluster [DBG] pgmap v1780: 105 pgs: 105 active+clean; 126 GiB data, 9.2 GiB used, 706 GiB / 715 GiB avail; 2.8 MiB/s rd, 2.1 MiB/s wr, 5 op/s 2024-04-07T16:53:18.699 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:53:18 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:53:18] "GET /metrics HTTP/1.1" 200 34067 "" "Prometheus/2.43.0" 2024-04-07T16:53:18.699 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:53:18 smithi100 bash[76202]: cluster 2024-04-07T16:53:17.521557+0000 mgr.y (mgr.25175) 1138 : cluster [DBG] pgmap v1780: 105 pgs: 105 active+clean; 126 GiB data, 9.2 GiB used, 706 GiB / 715 GiB avail; 2.8 MiB/s rd, 2.1 MiB/s wr, 5 op/s 2024-04-07T16:53:18.862 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:53:18 smithi144 bash[66654]: cluster 2024-04-07T16:53:17.521557+0000 mgr.y (mgr.25175) 1138 : cluster [DBG] pgmap v1780: 105 pgs: 105 active+clean; 126 GiB data, 9.2 GiB used, 706 GiB / 715 GiB avail; 2.8 MiB/s rd, 2.1 MiB/s wr, 5 op/s 2024-04-07T16:53:18.918 INFO:tasks.workunit.client.0.smithi100.stdout: 854: throughput=5MB/sec pending data=0 2024-04-07T16:53:18.918 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-1gLQYG-giD7ripU off=10629446 len=1779460 2024-04-07T16:53:18.934 INFO:tasks.workunit.client.0.smithi100.stdout:op 4422 completed, throughput=5MB/sec 2024-04-07T16:53:18.934 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-uJVBksCYVZ8E_rB off=7465227 len=1825847 2024-04-07T16:53:18.940 INFO:tasks.workunit.client.0.smithi100.stdout:op 4423 completed, throughput=5MB/sec 2024-04-07T16:53:18.940 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-q1umw-nnt1kaMvQ off=5145763 len=1472390 2024-04-07T16:53:18.945 INFO:tasks.workunit.client.0.smithi100.stdout:op 4424 completed, throughput=5MB/sec 2024-04-07T16:53:19.945 INFO:tasks.workunit.client.0.smithi100.stdout: 855: throughput=5MB/sec pending data=0 2024-04-07T16:53:19.945 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-7vupFi0ZFluCH_O off=444137 len=1113053 2024-04-07T16:53:19.955 INFO:tasks.workunit.client.0.smithi100.stdout:op 4425 completed, throughput=5MB/sec 2024-04-07T16:53:19.955 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-RoLQaGdmwiG-e8q off=58835 len=518605 2024-04-07T16:53:19.962 INFO:tasks.workunit.client.0.smithi100.stdout:op 4426 completed, throughput=5MB/sec 2024-04-07T16:53:19.962 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-mIcVY1zlRJzJT9n off=4772015 len=1970642 2024-04-07T16:53:19.967 INFO:tasks.workunit.client.0.smithi100.stdout:op 4427 completed, throughput=5MB/sec 2024-04-07T16:53:19.967 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-C1qzYVzVjGoRVQf off=7806191 len=1567917 2024-04-07T16:53:19.969 INFO:tasks.workunit.client.0.smithi100.stdout:op 4428 completed, throughput=5MB/sec 2024-04-07T16:53:20.948 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:53:20 smithi100 bash[74942]: cluster 2024-04-07T16:53:19.522522+0000 mgr.y (mgr.25175) 1139 : cluster [DBG] pgmap v1781: 105 pgs: 105 active+clean; 126 GiB data, 9.3 GiB used, 706 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.1 MiB/s wr, 4 op/s 2024-04-07T16:53:20.948 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:53:20 smithi100 bash[76202]: cluster 2024-04-07T16:53:19.522522+0000 mgr.y (mgr.25175) 1139 : cluster [DBG] pgmap v1781: 105 pgs: 105 active+clean; 126 GiB data, 9.3 GiB used, 706 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.1 MiB/s wr, 4 op/s 2024-04-07T16:53:20.969 INFO:tasks.workunit.client.0.smithi100.stdout: 856: throughput=5MB/sec pending data=0 2024-04-07T16:53:20.969 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Nt0CygW_Qf4YMM4 off=4829753 len=1686763 2024-04-07T16:53:20.984 INFO:tasks.workunit.client.0.smithi100.stdout:op 4429 completed, throughput=5MB/sec 2024-04-07T16:53:20.985 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-dm0oPO-4uKbe7me off=6069582 len=1854296 2024-04-07T16:53:20.989 INFO:tasks.workunit.client.0.smithi100.stdout:op 4430 completed, throughput=5MB/sec 2024-04-07T16:53:20.989 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-7qhH16YaHes0kxW off=2731171 len=1321198 2024-04-07T16:53:21.001 INFO:tasks.workunit.client.0.smithi100.stdout:op 4431 completed, throughput=5MB/sec 2024-04-07T16:53:21.112 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:53:20 smithi144 bash[66654]: cluster 2024-04-07T16:53:19.522522+0000 mgr.y (mgr.25175) 1139 : cluster [DBG] pgmap v1781: 105 pgs: 105 active+clean; 126 GiB data, 9.3 GiB used, 706 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.1 MiB/s wr, 4 op/s 2024-04-07T16:53:22.001 INFO:tasks.workunit.client.0.smithi100.stdout: 857: throughput=4.99MB/sec pending data=0 2024-04-07T16:53:22.001 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-aQ-9z2IGoJlKz3h off=3959425 len=2068019 2024-04-07T16:53:22.018 INFO:tasks.workunit.client.0.smithi100.stdout:op 4432 completed, throughput=5MB/sec 2024-04-07T16:53:22.018 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Qw5e4-huMVHJ-Gr off=15461416 len=2023139 2024-04-07T16:53:22.031 INFO:tasks.workunit.client.0.smithi100.stdout:op 4433 completed, throughput=5MB/sec 2024-04-07T16:53:22.031 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-s1R3PG4omJwwwTm off=11748932 len=15436 2024-04-07T16:53:22.036 INFO:tasks.workunit.client.0.smithi100.stdout:op 4434 completed, throughput=5MB/sec 2024-04-07T16:53:22.036 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-PmaANZ2ZFqf2v_c off=18141672 len=1595921 2024-04-07T16:53:22.040 INFO:tasks.workunit.client.0.smithi100.stdout:op 4435 completed, throughput=5MB/sec 2024-04-07T16:53:22.612 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:53:22 smithi144 bash[95233]: ts=2024-04-07T16:53:22.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:53:22.948 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:53:22 smithi100 bash[74942]: cluster 2024-04-07T16:53:21.523348+0000 mgr.y (mgr.25175) 1140 : cluster [DBG] pgmap v1782: 105 pgs: 105 active+clean; 126 GiB data, 9.3 GiB used, 706 GiB / 715 GiB avail; 2.8 MiB/s rd, 2.0 MiB/s wr, 4 op/s 2024-04-07T16:53:22.948 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:53:22 smithi100 bash[76202]: cluster 2024-04-07T16:53:21.523348+0000 mgr.y (mgr.25175) 1140 : cluster [DBG] pgmap v1782: 105 pgs: 105 active+clean; 126 GiB data, 9.3 GiB used, 706 GiB / 715 GiB avail; 2.8 MiB/s rd, 2.0 MiB/s wr, 4 op/s 2024-04-07T16:53:23.040 INFO:tasks.workunit.client.0.smithi100.stdout: 858: throughput=4.99MB/sec pending data=0 2024-04-07T16:53:23.040 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Z0e66pB4UoBy3jZ off=13159139 len=1246382 2024-04-07T16:53:23.051 INFO:tasks.workunit.client.0.smithi100.stdout:op 4436 completed, throughput=5MB/sec 2024-04-07T16:53:23.051 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Ilx5ySA_F0U3c7n off=3318340 len=1524595 2024-04-07T16:53:23.055 INFO:tasks.workunit.client.0.smithi100.stdout:op 4437 completed, throughput=5MB/sec 2024-04-07T16:53:23.055 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj--hj1U3dSG0V2UN7 off=19050549 len=1458930 2024-04-07T16:53:23.068 INFO:tasks.workunit.client.0.smithi100.stdout:op 4438 completed, throughput=5MB/sec 2024-04-07T16:53:23.068 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-2Hhhn0q-G6GjKjx off=10408820 len=1918841 2024-04-07T16:53:23.085 INFO:tasks.workunit.client.0.smithi100.stdout:op 4439 completed, throughput=5MB/sec 2024-04-07T16:53:23.112 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:53:22 smithi144 bash[66654]: cluster 2024-04-07T16:53:21.523348+0000 mgr.y (mgr.25175) 1140 : cluster [DBG] pgmap v1782: 105 pgs: 105 active+clean; 126 GiB data, 9.3 GiB used, 706 GiB / 715 GiB avail; 2.8 MiB/s rd, 2.0 MiB/s wr, 4 op/s 2024-04-07T16:53:24.086 INFO:tasks.workunit.client.0.smithi100.stdout: 859: throughput=5MB/sec pending data=0 2024-04-07T16:53:24.086 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-DFBHHtCofuso79u off=1095866 len=2094563 2024-04-07T16:53:24.103 INFO:tasks.workunit.client.0.smithi100.stdout:op 4440 completed, throughput=5MB/sec 2024-04-07T16:53:24.103 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-mMxVDvB0DDdgB7S off=1269490 len=1018585 2024-04-07T16:53:24.114 INFO:tasks.workunit.client.0.smithi100.stdout:op 4441 completed, throughput=5MB/sec 2024-04-07T16:53:24.114 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-OeCB3kCqbK8n5EF off=19796690 len=1445192 2024-04-07T16:53:24.118 INFO:tasks.workunit.client.0.smithi100.stdout:op 4442 completed, throughput=5MB/sec 2024-04-07T16:53:24.948 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:53:24 smithi100 bash[74942]: cluster 2024-04-07T16:53:23.524495+0000 mgr.y (mgr.25175) 1141 : cluster [DBG] pgmap v1783: 105 pgs: 105 active+clean; 126 GiB data, 9.3 GiB used, 706 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.8 MiB/s wr, 4 op/s 2024-04-07T16:53:24.949 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:53:24 smithi100 bash[76202]: cluster 2024-04-07T16:53:23.524495+0000 mgr.y (mgr.25175) 1141 : cluster [DBG] pgmap v1783: 105 pgs: 105 active+clean; 126 GiB data, 9.3 GiB used, 706 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.8 MiB/s wr, 4 op/s 2024-04-07T16:53:25.112 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:53:24 smithi144 bash[66654]: cluster 2024-04-07T16:53:23.524495+0000 mgr.y (mgr.25175) 1141 : cluster [DBG] pgmap v1783: 105 pgs: 105 active+clean; 126 GiB data, 9.3 GiB used, 706 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.8 MiB/s wr, 4 op/s 2024-04-07T16:53:25.118 INFO:tasks.workunit.client.0.smithi100.stdout: 860: throughput=4.99MB/sec pending data=0 2024-04-07T16:53:25.119 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-BjplzSyvnksfIqT off=709265 len=459035 2024-04-07T16:53:25.125 INFO:tasks.workunit.client.0.smithi100.stdout:op 4443 completed, throughput=5MB/sec 2024-04-07T16:53:25.125 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-KmN9hh0i90TJweJ off=2148653 len=73264 2024-04-07T16:53:25.126 INFO:tasks.workunit.client.0.smithi100.stdout:op 4444 completed, throughput=5MB/sec 2024-04-07T16:53:25.126 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-nJyGI1Rm0_af9w0 off=499394 len=891196 2024-04-07T16:53:25.134 INFO:tasks.workunit.client.0.smithi100.stdout:op 4445 completed, throughput=5MB/sec 2024-04-07T16:53:25.134 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Qh2op-povw1PAIN off=8993242 len=1326495 2024-04-07T16:53:25.146 INFO:tasks.workunit.client.0.smithi100.stdout:op 4446 completed, throughput=5MB/sec 2024-04-07T16:53:25.146 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-9_dlOIiufGaC9mz off=571203 len=1748055 2024-04-07T16:53:25.159 INFO:tasks.workunit.client.0.smithi100.stdout:op 4447 completed, throughput=5MB/sec 2024-04-07T16:53:25.160 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-wXaPVUsdT8hwEFP off=11413647 len=128669 2024-04-07T16:53:25.160 INFO:tasks.workunit.client.0.smithi100.stdout:op 4448 completed, throughput=5MB/sec 2024-04-07T16:53:25.160 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-eGp0E1pXpU83Q2b off=15341220 len=198260 2024-04-07T16:53:25.161 INFO:tasks.workunit.client.0.smithi100.stdout:op 4449 completed, throughput=5MB/sec 2024-04-07T16:53:25.162 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-HvJkQHxImrpIN5R off=2043447 len=1188323 2024-04-07T16:53:25.164 INFO:tasks.workunit.client.0.smithi100.stdout:op 4450 completed, throughput=5MB/sec 2024-04-07T16:53:26.164 INFO:tasks.workunit.client.0.smithi100.stdout: 861: throughput=5MB/sec pending data=0 2024-04-07T16:53:26.164 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-zAK2afx6LXVPCx2 off=1398892 len=1383063 2024-04-07T16:53:26.177 INFO:tasks.workunit.client.0.smithi100.stdout:op 4451 completed, throughput=5MB/sec 2024-04-07T16:53:26.178 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-QrARshO0hwvmosE off=2376108 len=44584 2024-04-07T16:53:26.179 INFO:tasks.workunit.client.0.smithi100.stdout:op 4452 completed, throughput=5MB/sec 2024-04-07T16:53:26.179 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-uRAxgsPRk_igAXX off=0 len=1709435 2024-04-07T16:53:26.192 INFO:tasks.workunit.client.0.smithi100.stdout:op 4453 completed, throughput=5MB/sec 2024-04-07T16:53:26.192 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-xCJK31wRuI3Nwj5 off=8322243 len=1128102 2024-04-07T16:53:26.201 INFO:tasks.workunit.client.0.smithi100.stdout:op 4454 completed, throughput=5MB/sec 2024-04-07T16:53:26.201 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-d33o_qnHwhsGPLM off=0 len=1278615 2024-04-07T16:53:26.212 INFO:tasks.workunit.client.0.smithi100.stdout:op 4455 completed, throughput=5MB/sec 2024-04-07T16:53:26.948 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:53:26 smithi100 bash[74942]: cluster 2024-04-07T16:53:25.525485+0000 mgr.y (mgr.25175) 1142 : cluster [DBG] pgmap v1784: 105 pgs: 105 active+clean; 126 GiB data, 9.3 GiB used, 706 GiB / 715 GiB avail; 2.4 MiB/s rd, 2.4 MiB/s wr, 4 op/s 2024-04-07T16:53:26.949 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:53:26 smithi100 bash[76202]: cluster 2024-04-07T16:53:25.525485+0000 mgr.y (mgr.25175) 1142 : cluster [DBG] pgmap v1784: 105 pgs: 105 active+clean; 126 GiB data, 9.3 GiB used, 706 GiB / 715 GiB avail; 2.4 MiB/s rd, 2.4 MiB/s wr, 4 op/s 2024-04-07T16:53:27.112 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:53:26 smithi144 bash[66654]: cluster 2024-04-07T16:53:25.525485+0000 mgr.y (mgr.25175) 1142 : cluster [DBG] pgmap v1784: 105 pgs: 105 active+clean; 126 GiB data, 9.3 GiB used, 706 GiB / 715 GiB avail; 2.4 MiB/s rd, 2.4 MiB/s wr, 4 op/s 2024-04-07T16:53:27.213 INFO:tasks.workunit.client.0.smithi100.stdout: 862: throughput=5MB/sec pending data=0 2024-04-07T16:53:27.213 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-KAeoMu0xVnqqmJu off=5890528 len=1975675 2024-04-07T16:53:27.219 INFO:tasks.workunit.client.0.smithi100.stdout:op 4456 completed, throughput=5MB/sec 2024-04-07T16:53:27.219 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-_zM680Ua0qeszrs off=11393562 len=249297 2024-04-07T16:53:27.225 INFO:tasks.workunit.client.0.smithi100.stdout:op 4457 completed, throughput=5MB/sec 2024-04-07T16:53:27.225 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-puxIJNL26htvSWp off=7880507 len=1892485 2024-04-07T16:53:27.230 INFO:tasks.workunit.client.0.smithi100.stdout:op 4458 completed, throughput=5MB/sec 2024-04-07T16:53:27.230 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-mYewQhNGkyMgD3M off=299224 len=66537 2024-04-07T16:53:27.235 INFO:tasks.workunit.client.0.smithi100.stdout:op 4459 completed, throughput=5MB/sec 2024-04-07T16:53:27.235 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-3OWtTJ-bTul2swu off=5706268 len=1542452 2024-04-07T16:53:27.239 INFO:tasks.workunit.client.0.smithi100.stdout:op 4460 completed, throughput=5MB/sec 2024-04-07T16:53:28.239 INFO:tasks.workunit.client.0.smithi100.stdout: 863: throughput=5MB/sec pending data=0 2024-04-07T16:53:28.239 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-kh8j9o6YiGxca8D off=653106 len=1700391 2024-04-07T16:53:28.254 INFO:tasks.workunit.client.0.smithi100.stdout:op 4461 completed, throughput=5MB/sec 2024-04-07T16:53:28.254 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-S3bZcgzKa5CGQCE off=1524277 len=1688318 2024-04-07T16:53:28.270 INFO:tasks.workunit.client.0.smithi100.stdout:op 4462 completed, throughput=5MB/sec 2024-04-07T16:53:28.270 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-7EbFtxoDSnEVszi off=8339442 len=1562297 2024-04-07T16:53:28.274 INFO:tasks.workunit.client.0.smithi100.stdout:op 4463 completed, throughput=5MB/sec 2024-04-07T16:53:28.698 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:53:28 smithi100 bash[74942]: cluster 2024-04-07T16:53:27.526563+0000 mgr.y (mgr.25175) 1143 : cluster [DBG] pgmap v1785: 105 pgs: 105 active+clean; 126 GiB data, 9.3 GiB used, 706 GiB / 715 GiB avail; 2.2 MiB/s rd, 3.1 MiB/s wr, 4 op/s 2024-04-07T16:53:28.699 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:53:28 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:53:28] "GET /metrics HTTP/1.1" 200 34067 "" "Prometheus/2.43.0" 2024-04-07T16:53:28.699 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:53:28 smithi100 bash[76202]: cluster 2024-04-07T16:53:27.526563+0000 mgr.y (mgr.25175) 1143 : cluster [DBG] pgmap v1785: 105 pgs: 105 active+clean; 126 GiB data, 9.3 GiB used, 706 GiB / 715 GiB avail; 2.2 MiB/s rd, 3.1 MiB/s wr, 4 op/s 2024-04-07T16:53:29.112 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:53:28 smithi144 bash[66654]: cluster 2024-04-07T16:53:27.526563+0000 mgr.y (mgr.25175) 1143 : cluster [DBG] pgmap v1785: 105 pgs: 105 active+clean; 126 GiB data, 9.3 GiB used, 706 GiB / 715 GiB avail; 2.2 MiB/s rd, 3.1 MiB/s wr, 4 op/s 2024-04-07T16:53:29.274 INFO:tasks.workunit.client.0.smithi100.stdout: 864: throughput=5MB/sec pending data=0 2024-04-07T16:53:29.274 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-b4I5fBH8Z0WZV7o off=141470 len=289891 2024-04-07T16:53:29.276 INFO:tasks.workunit.client.0.smithi100.stdout:op 4464 completed, throughput=5MB/sec 2024-04-07T16:53:29.276 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-IIm9qhSSdx5OyEp off=8690744 len=52108 2024-04-07T16:53:29.281 INFO:tasks.workunit.client.0.smithi100.stdout:op 4465 completed, throughput=5MB/sec 2024-04-07T16:53:29.281 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-MRuZ8J6abjE_GK9 off=15591009 len=245700 2024-04-07T16:53:29.287 INFO:tasks.workunit.client.0.smithi100.stdout:op 4466 completed, throughput=5MB/sec 2024-04-07T16:53:29.287 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-rG16wxs5ESXzyHb off=11411079 len=220875 2024-04-07T16:53:29.288 INFO:tasks.workunit.client.0.smithi100.stdout:op 4467 completed, throughput=5MB/sec 2024-04-07T16:53:29.289 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-1fT9xOPZtTukWTo off=14444904 len=1131217 2024-04-07T16:53:29.300 INFO:tasks.workunit.client.0.smithi100.stdout:op 4468 completed, throughput=5MB/sec 2024-04-07T16:53:29.300 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-YeNMJWHF7H2rJ8p off=7200846 len=1108410 2024-04-07T16:53:29.303 INFO:tasks.workunit.client.0.smithi100.stdout:op 4469 completed, throughput=5MB/sec 2024-04-07T16:53:29.303 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-wOJr9MrJ7Mcqxgh off=19444652 len=807696 2024-04-07T16:53:29.311 INFO:tasks.workunit.client.0.smithi100.stdout:op 4470 completed, throughput=5MB/sec 2024-04-07T16:53:29.311 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-dc1AZON0xgCKJC2 off=13247534 len=2017786 2024-04-07T16:53:29.325 INFO:tasks.workunit.client.0.smithi100.stdout:op 4471 completed, throughput=5MB/sec 2024-04-07T16:53:30.325 INFO:tasks.workunit.client.0.smithi100.stdout: 865: throughput=5MB/sec pending data=0 2024-04-07T16:53:30.325 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-L98Ss4CUVFOTi3M off=5588163 len=537851 2024-04-07T16:53:30.327 INFO:tasks.workunit.client.0.smithi100.stdout:op 4472 completed, throughput=5MB/sec 2024-04-07T16:53:30.327 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-RIpVI7rizughby7 off=2179815 len=115795 2024-04-07T16:53:30.333 INFO:tasks.workunit.client.0.smithi100.stdout:op 4473 completed, throughput=5MB/sec 2024-04-07T16:53:30.333 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-RWQmqe_1XRaAdnh off=18474698 len=1198015 2024-04-07T16:53:30.343 INFO:tasks.workunit.client.0.smithi100.stdout:op 4474 completed, throughput=5MB/sec 2024-04-07T16:53:30.343 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-58RVWzeWY_NIP9E off=5811027 len=1211546 2024-04-07T16:53:30.346 INFO:tasks.workunit.client.0.smithi100.stdout:op 4475 completed, throughput=5MB/sec 2024-04-07T16:53:30.346 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-_ve-5KOV8ciBm4j off=19420729 len=1323798 2024-04-07T16:53:30.357 INFO:tasks.workunit.client.0.smithi100.stdout:op 4476 completed, throughput=5MB/sec 2024-04-07T16:53:30.948 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:53:30 smithi100 bash[74942]: cluster 2024-04-07T16:53:29.527578+0000 mgr.y (mgr.25175) 1144 : cluster [DBG] pgmap v1786: 105 pgs: 105 active+clean; 126 GiB data, 9.3 GiB used, 706 GiB / 715 GiB avail; 1.7 MiB/s rd, 2.9 MiB/s wr, 3 op/s 2024-04-07T16:53:30.948 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:53:30 smithi100 bash[76202]: cluster 2024-04-07T16:53:29.527578+0000 mgr.y (mgr.25175) 1144 : cluster [DBG] pgmap v1786: 105 pgs: 105 active+clean; 126 GiB data, 9.3 GiB used, 706 GiB / 715 GiB avail; 1.7 MiB/s rd, 2.9 MiB/s wr, 3 op/s 2024-04-07T16:53:31.112 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:53:30 smithi144 bash[66654]: cluster 2024-04-07T16:53:29.527578+0000 mgr.y (mgr.25175) 1144 : cluster [DBG] pgmap v1786: 105 pgs: 105 active+clean; 126 GiB data, 9.3 GiB used, 706 GiB / 715 GiB avail; 1.7 MiB/s rd, 2.9 MiB/s wr, 3 op/s 2024-04-07T16:53:31.357 INFO:tasks.workunit.client.0.smithi100.stdout: 866: throughput=4.99MB/sec pending data=0 2024-04-07T16:53:31.358 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-uhHadAkxUrtczpJ off=1542880 len=1476074 2024-04-07T16:53:31.361 INFO:tasks.workunit.client.0.smithi100.stdout:op 4477 completed, throughput=5MB/sec 2024-04-07T16:53:31.361 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-ZHKc7M5tt7BojI0 off=123368 len=670587 2024-04-07T16:53:31.369 INFO:tasks.workunit.client.0.smithi100.stdout:op 4478 completed, throughput=5MB/sec 2024-04-07T16:53:31.370 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj--5-JbV0TQrZMTfu off=7014790 len=533163 2024-04-07T16:53:31.376 INFO:tasks.workunit.client.0.smithi100.stdout:op 4479 completed, throughput=5MB/sec 2024-04-07T16:53:31.376 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-aeMISH7hdig7VX6 off=9999373 len=608995 2024-04-07T16:53:31.382 INFO:tasks.workunit.client.0.smithi100.stdout:op 4480 completed, throughput=5MB/sec 2024-04-07T16:53:31.383 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-kgGiCaYSgheT4md off=2119277 len=1066294 2024-04-07T16:53:31.392 INFO:tasks.workunit.client.0.smithi100.stdout:op 4481 completed, throughput=5MB/sec 2024-04-07T16:53:31.392 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-yi1mHl25nv6hgHk off=1719183 len=1379961 2024-04-07T16:53:31.406 INFO:tasks.workunit.client.0.smithi100.stdout:op 4482 completed, throughput=5MB/sec 2024-04-07T16:53:32.406 INFO:tasks.workunit.client.0.smithi100.stdout: 867: throughput=4.99MB/sec pending data=0 2024-04-07T16:53:32.406 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-WyVXXT4ACeXlOaD off=14813205 len=702378 2024-04-07T16:53:32.408 INFO:tasks.workunit.client.0.smithi100.stdout:op 4483 completed, throughput=5MB/sec 2024-04-07T16:53:32.408 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-CfXFFXC5vSYOxtq off=14750580 len=507589 2024-04-07T16:53:32.411 INFO:tasks.workunit.client.0.smithi100.stdout:op 4484 completed, throughput=5MB/sec 2024-04-07T16:53:32.411 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-W69xegjtfSljNID off=95040 len=1649847 2024-04-07T16:53:32.415 INFO:tasks.workunit.client.0.smithi100.stdout:op 4485 completed, throughput=5MB/sec 2024-04-07T16:53:32.415 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-mPTaY2XRn-hJv0S off=1814140 len=1216664 2024-04-07T16:53:32.417 INFO:tasks.workunit.client.0.smithi100.stdout:op 4486 completed, throughput=5MB/sec 2024-04-07T16:53:32.417 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-YHy64xjywMIQqV1 off=1407664 len=1806496 2024-04-07T16:53:32.431 INFO:tasks.workunit.client.0.smithi100.stdout:op 4487 completed, throughput=5MB/sec 2024-04-07T16:53:32.612 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:53:32 smithi144 bash[95233]: ts=2024-04-07T16:53:32.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:53:32.948 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:53:32 smithi100 bash[74942]: cluster 2024-04-07T16:53:31.528389+0000 mgr.y (mgr.25175) 1145 : cluster [DBG] pgmap v1787: 105 pgs: 105 active+clean; 126 GiB data, 9.4 GiB used, 706 GiB / 715 GiB avail; 1.8 MiB/s rd, 2.9 MiB/s wr, 3 op/s 2024-04-07T16:53:32.949 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:53:32 smithi100 bash[74942]: audit 2024-04-07T16:53:31.781556+0000 mon.a (mon.0) 2172 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:53:32.949 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:53:32 smithi100 bash[76202]: cluster 2024-04-07T16:53:31.528389+0000 mgr.y (mgr.25175) 1145 : cluster [DBG] pgmap v1787: 105 pgs: 105 active+clean; 126 GiB data, 9.4 GiB used, 706 GiB / 715 GiB avail; 1.8 MiB/s rd, 2.9 MiB/s wr, 3 op/s 2024-04-07T16:53:32.949 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:53:32 smithi100 bash[76202]: audit 2024-04-07T16:53:31.781556+0000 mon.a (mon.0) 2172 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:53:33.112 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:53:32 smithi144 bash[66654]: cluster 2024-04-07T16:53:31.528389+0000 mgr.y (mgr.25175) 1145 : cluster [DBG] pgmap v1787: 105 pgs: 105 active+clean; 126 GiB data, 9.4 GiB used, 706 GiB / 715 GiB avail; 1.8 MiB/s rd, 2.9 MiB/s wr, 3 op/s 2024-04-07T16:53:33.112 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:53:32 smithi144 bash[66654]: audit 2024-04-07T16:53:31.781556+0000 mon.a (mon.0) 2172 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:53:33.432 INFO:tasks.workunit.client.0.smithi100.stdout: 868: throughput=5MB/sec pending data=0 2024-04-07T16:53:33.432 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-PUF2ZnyeHwQZ_ZC off=6939872 len=58091 2024-04-07T16:53:33.436 INFO:tasks.workunit.client.0.smithi100.stdout:op 4488 completed, throughput=5MB/sec 2024-04-07T16:53:33.436 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-whSb8L7HdwzPWSw off=1671434 len=63208 2024-04-07T16:53:33.437 INFO:tasks.workunit.client.0.smithi100.stdout:op 4489 completed, throughput=5MB/sec 2024-04-07T16:53:33.437 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-WWeyQgkYZuqbE8N off=11800846 len=2047210 2024-04-07T16:53:33.442 INFO:tasks.workunit.client.0.smithi100.stdout:op 4490 completed, throughput=5MB/sec 2024-04-07T16:53:33.442 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-KrCkz8a5WIWu1QQ off=5045406 len=1539574 2024-04-07T16:53:33.445 INFO:tasks.workunit.client.0.smithi100.stdout:op 4491 completed, throughput=5MB/sec 2024-04-07T16:53:33.446 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-vJbEw7QqNBpO5LK off=8126203 len=1485836 2024-04-07T16:53:33.449 INFO:tasks.workunit.client.0.smithi100.stdout:op 4492 completed, throughput=5MB/sec 2024-04-07T16:53:34.449 INFO:tasks.workunit.client.0.smithi100.stdout: 869: throughput=5MB/sec pending data=0 2024-04-07T16:53:34.450 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-asH8Yrt3vkBxI5B off=13500179 len=746440 2024-04-07T16:53:34.458 INFO:tasks.workunit.client.0.smithi100.stdout:op 4493 completed, throughput=5MB/sec 2024-04-07T16:53:34.458 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-HDLXCco6ILra8-v off=3484136 len=217375 2024-04-07T16:53:34.464 INFO:tasks.workunit.client.0.smithi100.stdout:op 4494 completed, throughput=5MB/sec 2024-04-07T16:53:34.464 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-RFI1u6QLt0bIU08 off=1612371 len=1232655 2024-04-07T16:53:34.476 INFO:tasks.workunit.client.0.smithi100.stdout:op 4495 completed, throughput=5MB/sec 2024-04-07T16:53:34.476 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-1bYVZdsyazrm8y- off=111783 len=1632253 2024-04-07T16:53:34.489 INFO:tasks.workunit.client.0.smithi100.stdout:op 4496 completed, throughput=5MB/sec 2024-04-07T16:53:34.489 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-mTE8Q5lOMSPcngo off=3473068 len=1009060 2024-04-07T16:53:34.498 INFO:tasks.workunit.client.0.smithi100.stdout:op 4497 completed, throughput=5MB/sec 2024-04-07T16:53:34.498 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-p657OD9fqF37dr1 off=14427980 len=889173 2024-04-07T16:53:34.500 INFO:tasks.workunit.client.0.smithi100.stdout:op 4498 completed, throughput=5MB/sec 2024-04-07T16:53:34.948 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:53:34 smithi100 bash[74942]: cluster 2024-04-07T16:53:33.529495+0000 mgr.y (mgr.25175) 1146 : cluster [DBG] pgmap v1788: 105 pgs: 105 active+clean; 126 GiB data, 9.4 GiB used, 706 GiB / 715 GiB avail; 2.3 MiB/s rd, 3.4 MiB/s wr, 4 op/s 2024-04-07T16:53:34.949 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:53:34 smithi100 bash[76202]: cluster 2024-04-07T16:53:33.529495+0000 mgr.y (mgr.25175) 1146 : cluster [DBG] pgmap v1788: 105 pgs: 105 active+clean; 126 GiB data, 9.4 GiB used, 706 GiB / 715 GiB avail; 2.3 MiB/s rd, 3.4 MiB/s wr, 4 op/s 2024-04-07T16:53:35.112 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:53:34 smithi144 bash[66654]: cluster 2024-04-07T16:53:33.529495+0000 mgr.y (mgr.25175) 1146 : cluster [DBG] pgmap v1788: 105 pgs: 105 active+clean; 126 GiB data, 9.4 GiB used, 706 GiB / 715 GiB avail; 2.3 MiB/s rd, 3.4 MiB/s wr, 4 op/s 2024-04-07T16:53:35.500 INFO:tasks.workunit.client.0.smithi100.stdout: 870: throughput=5MB/sec pending data=0 2024-04-07T16:53:35.500 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-bfvqvePEtqmw5dV off=5987677 len=953370 2024-04-07T16:53:35.503 INFO:tasks.workunit.client.0.smithi100.stdout:op 4499 completed, throughput=5MB/sec 2024-04-07T16:53:35.503 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-uJKFSa_MsmJIxOx off=1040990 len=414546 2024-04-07T16:53:35.505 INFO:tasks.workunit.client.0.smithi100.stdout:op 4500 completed, throughput=5MB/sec 2024-04-07T16:53:35.505 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-7_u2XrDgR_ykuQJ off=5517279 len=302070 2024-04-07T16:53:35.511 INFO:tasks.workunit.client.0.smithi100.stdout:op 4501 completed, throughput=5MB/sec 2024-04-07T16:53:35.511 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-AL-uZ14q4aVq9f1 off=16149552 len=241561 2024-04-07T16:53:35.512 INFO:tasks.workunit.client.0.smithi100.stdout:op 4502 completed, throughput=5MB/sec 2024-04-07T16:53:35.512 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-RW8KpiMDHs6n1DE off=3623107 len=1156740 2024-04-07T16:53:35.522 INFO:tasks.workunit.client.0.smithi100.stdout:op 4503 completed, throughput=5MB/sec 2024-04-07T16:53:35.522 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-FNepOsT_STlOHMf off=24194174 len=1275011 2024-04-07T16:53:35.526 INFO:tasks.workunit.client.0.smithi100.stdout:op 4504 completed, throughput=5MB/sec 2024-04-07T16:53:35.526 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-sC7dtxm_f2E2ddJ off=11892899 len=208652 2024-04-07T16:53:35.531 INFO:tasks.workunit.client.0.smithi100.stdout:op 4505 completed, throughput=5MB/sec 2024-04-07T16:53:36.531 INFO:tasks.workunit.client.0.smithi100.stdout: 871: throughput=4.99MB/sec pending data=0 2024-04-07T16:53:36.531 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Zq09cKo-YxY5yPY off=534778 len=2027750 2024-04-07T16:53:36.547 INFO:tasks.workunit.client.0.smithi100.stdout:op 4506 completed, throughput=5MB/sec 2024-04-07T16:53:36.547 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-mbBwDnqJpbrJ3CW off=6370594 len=1164523 2024-04-07T16:53:36.550 INFO:tasks.workunit.client.0.smithi100.stdout:op 4507 completed, throughput=5MB/sec 2024-04-07T16:53:36.550 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-pR2g9fwoWeQASp4 off=1571151 len=1039070 2024-04-07T16:53:36.552 INFO:tasks.workunit.client.0.smithi100.stdout:op 4508 completed, throughput=5MB/sec 2024-04-07T16:53:36.552 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Kbez4SCNtEBCiGY off=4151054 len=649555 2024-04-07T16:53:36.560 INFO:tasks.workunit.client.0.smithi100.stdout:op 4509 completed, throughput=5MB/sec 2024-04-07T16:53:36.560 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-mbNWFPrK0Ouuwdt off=124301 len=736232 2024-04-07T16:53:36.567 INFO:tasks.workunit.client.0.smithi100.stdout:op 4510 completed, throughput=5MB/sec 2024-04-07T16:53:36.948 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:53:36 smithi100 bash[74942]: cluster 2024-04-07T16:53:35.530510+0000 mgr.y (mgr.25175) 1147 : cluster [DBG] pgmap v1789: 105 pgs: 105 active+clean; 126 GiB data, 9.4 GiB used, 706 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.9 MiB/s wr, 5 op/s 2024-04-07T16:53:36.949 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:53:36 smithi100 bash[76202]: cluster 2024-04-07T16:53:35.530510+0000 mgr.y (mgr.25175) 1147 : cluster [DBG] pgmap v1789: 105 pgs: 105 active+clean; 126 GiB data, 9.4 GiB used, 706 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.9 MiB/s wr, 5 op/s 2024-04-07T16:53:37.112 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:53:36 smithi144 bash[66654]: cluster 2024-04-07T16:53:35.530510+0000 mgr.y (mgr.25175) 1147 : cluster [DBG] pgmap v1789: 105 pgs: 105 active+clean; 126 GiB data, 9.4 GiB used, 706 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.9 MiB/s wr, 5 op/s 2024-04-07T16:53:37.567 INFO:tasks.workunit.client.0.smithi100.stdout: 872: throughput=4.99MB/sec pending data=0 2024-04-07T16:53:37.567 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-mJeubM5KfE-WHrd off=8451909 len=672648 2024-04-07T16:53:37.575 INFO:tasks.workunit.client.0.smithi100.stdout:op 4511 completed, throughput=5MB/sec 2024-04-07T16:53:37.575 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-iy0rlKmG2XXVPj1 off=4653759 len=1568413 2024-04-07T16:53:37.588 INFO:tasks.workunit.client.0.smithi100.stdout:op 4512 completed, throughput=5MB/sec 2024-04-07T16:53:37.588 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-NC_dj7VUxFBNDAK off=585020 len=669889 2024-04-07T16:53:37.595 INFO:tasks.workunit.client.0.smithi100.stdout:op 4513 completed, throughput=5MB/sec 2024-04-07T16:53:37.595 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-GjTZmnMeVEyTAdu off=11559 len=182826 2024-04-07T16:53:37.601 INFO:tasks.workunit.client.0.smithi100.stdout:op 4514 completed, throughput=5MB/sec 2024-04-07T16:53:37.601 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-rDDR8q6aj01mF2S off=11651323 len=63982 2024-04-07T16:53:37.601 INFO:tasks.workunit.client.0.smithi100.stdout:op 4515 completed, throughput=5MB/sec 2024-04-07T16:53:37.602 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-rjhSaFaKlUxsTOQ off=1044450 len=1061414 2024-04-07T16:53:37.610 INFO:tasks.workunit.client.0.smithi100.stdout:op 4516 completed, throughput=5MB/sec 2024-04-07T16:53:37.611 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-f0wR5P6I7gMlBuR off=131375 len=881462 2024-04-07T16:53:37.613 INFO:tasks.workunit.client.0.smithi100.stdout:op 4517 completed, throughput=5MB/sec 2024-04-07T16:53:37.613 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Y0GQfT54qskDfGb off=405766 len=344065 2024-04-07T16:53:37.615 INFO:tasks.workunit.client.0.smithi100.stdout:op 4518 completed, throughput=5MB/sec 2024-04-07T16:53:38.615 INFO:tasks.workunit.client.0.smithi100.stdout: 873: throughput=4.99MB/sec pending data=0 2024-04-07T16:53:38.615 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-D5EKc24-KTItXgr off=83571 len=1405780 2024-04-07T16:53:38.626 INFO:tasks.workunit.client.0.smithi100.stdout:op 4519 completed, throughput=5MB/sec 2024-04-07T16:53:38.626 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-sSuhdZKpoa8kiuH off=2295467 len=1311335 2024-04-07T16:53:38.636 INFO:tasks.workunit.client.0.smithi100.stdout:op 4520 completed, throughput=5MB/sec 2024-04-07T16:53:38.636 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-f7b4B4CCeEDyT7Z off=18361335 len=1561229 2024-04-07T16:53:38.640 INFO:tasks.workunit.client.0.smithi100.stdout:op 4521 completed, throughput=5MB/sec 2024-04-07T16:53:38.640 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-d3JBkBODkE-QenH off=4200655 len=1150013 2024-04-07T16:53:38.643 INFO:tasks.workunit.client.0.smithi100.stdout:op 4522 completed, throughput=5MB/sec 2024-04-07T16:53:38.698 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:53:38 smithi100 bash[74942]: cluster 2024-04-07T16:53:37.531554+0000 mgr.y (mgr.25175) 1148 : cluster [DBG] pgmap v1790: 105 pgs: 105 active+clean; 126 GiB data, 9.4 GiB used, 706 GiB / 715 GiB avail; 1.9 MiB/s rd, 2.9 MiB/s wr, 4 op/s 2024-04-07T16:53:38.699 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:53:38 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:53:38] "GET /metrics HTTP/1.1" 200 34075 "" "Prometheus/2.43.0" 2024-04-07T16:53:38.699 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:53:38 smithi100 bash[76202]: cluster 2024-04-07T16:53:37.531554+0000 mgr.y (mgr.25175) 1148 : cluster [DBG] pgmap v1790: 105 pgs: 105 active+clean; 126 GiB data, 9.4 GiB used, 706 GiB / 715 GiB avail; 1.9 MiB/s rd, 2.9 MiB/s wr, 4 op/s 2024-04-07T16:53:39.112 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:53:38 smithi144 bash[66654]: cluster 2024-04-07T16:53:37.531554+0000 mgr.y (mgr.25175) 1148 : cluster [DBG] pgmap v1790: 105 pgs: 105 active+clean; 126 GiB data, 9.4 GiB used, 706 GiB / 715 GiB avail; 1.9 MiB/s rd, 2.9 MiB/s wr, 4 op/s 2024-04-07T16:53:39.644 INFO:tasks.workunit.client.0.smithi100.stdout: 874: throughput=4.99MB/sec pending data=0 2024-04-07T16:53:39.644 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-6kvpyi0L_iJWLeq off=1021310 len=333472 2024-04-07T16:53:39.650 INFO:tasks.workunit.client.0.smithi100.stdout:op 4523 completed, throughput=4.99MB/sec 2024-04-07T16:53:39.650 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-QkwmksJ_JcVEdgA off=1250519 len=1912265 2024-04-07T16:53:39.665 INFO:tasks.workunit.client.0.smithi100.stdout:op 4524 completed, throughput=5MB/sec 2024-04-07T16:53:39.665 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-lc3v61EvyfRS4nn off=16298398 len=746441 2024-04-07T16:53:39.672 INFO:tasks.workunit.client.0.smithi100.stdout:op 4525 completed, throughput=5MB/sec 2024-04-07T16:53:39.672 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-HlsBR5rY8SGH7w7 off=5413005 len=729576 2024-04-07T16:53:39.676 INFO:tasks.workunit.client.0.smithi100.stdout:op 4526 completed, throughput=5MB/sec 2024-04-07T16:53:39.676 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-JQnEI-5Rd0aY_2L off=15271424 len=1119655 2024-04-07T16:53:39.679 INFO:tasks.workunit.client.0.smithi100.stdout:op 4527 completed, throughput=5MB/sec 2024-04-07T16:53:39.679 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-3Nb51X5cg1S3TjX off=5543233 len=1792746 2024-04-07T16:53:39.684 INFO:tasks.workunit.client.0.smithi100.stdout:op 4528 completed, throughput=5MB/sec 2024-04-07T16:53:40.685 INFO:tasks.workunit.client.0.smithi100.stdout: 875: throughput=5MB/sec pending data=0 2024-04-07T16:53:40.685 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-CrXLCUAScSPkDaW off=9544207 len=400885 2024-04-07T16:53:40.691 INFO:tasks.workunit.client.0.smithi100.stdout:op 4529 completed, throughput=5MB/sec 2024-04-07T16:53:40.691 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-_seXtFzOBOTtj4t off=7742455 len=1130080 2024-04-07T16:53:40.694 INFO:tasks.workunit.client.0.smithi100.stdout:op 4530 completed, throughput=5MB/sec 2024-04-07T16:53:40.694 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-8bc0BZc5kpCOALX off=1718663 len=1102972 2024-04-07T16:53:40.704 INFO:tasks.workunit.client.0.smithi100.stdout:op 4531 completed, throughput=5MB/sec 2024-04-07T16:53:40.704 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-LBX1BGrPXNQmZ43 off=3271484 len=733748 2024-04-07T16:53:40.706 INFO:tasks.workunit.client.0.smithi100.stdout:op 4532 completed, throughput=5MB/sec 2024-04-07T16:53:40.706 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-ozr1tdQSR0l4bG5 off=629902 len=538573 2024-04-07T16:53:40.712 INFO:tasks.workunit.client.0.smithi100.stdout:op 4533 completed, throughput=5MB/sec 2024-04-07T16:53:40.712 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-DWRWlDg0Vm5zcQK off=8537486 len=1068237 2024-04-07T16:53:40.714 INFO:tasks.workunit.client.0.smithi100.stdout:op 4534 completed, throughput=5MB/sec 2024-04-07T16:53:40.948 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:53:40 smithi100 bash[76202]: cluster 2024-04-07T16:53:39.532437+0000 mgr.y (mgr.25175) 1149 : cluster [DBG] pgmap v1791: 105 pgs: 105 active+clean; 126 GiB data, 9.4 GiB used, 706 GiB / 715 GiB avail; 2.2 MiB/s rd, 2.5 MiB/s wr, 5 op/s 2024-04-07T16:53:40.949 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:53:40 smithi100 bash[74942]: cluster 2024-04-07T16:53:39.532437+0000 mgr.y (mgr.25175) 1149 : cluster [DBG] pgmap v1791: 105 pgs: 105 active+clean; 126 GiB data, 9.4 GiB used, 706 GiB / 715 GiB avail; 2.2 MiB/s rd, 2.5 MiB/s wr, 5 op/s 2024-04-07T16:53:41.112 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:53:40 smithi144 bash[66654]: cluster 2024-04-07T16:53:39.532437+0000 mgr.y (mgr.25175) 1149 : cluster [DBG] pgmap v1791: 105 pgs: 105 active+clean; 126 GiB data, 9.4 GiB used, 706 GiB / 715 GiB avail; 2.2 MiB/s rd, 2.5 MiB/s wr, 5 op/s 2024-04-07T16:53:41.715 INFO:tasks.workunit.client.0.smithi100.stdout: 876: throughput=5MB/sec pending data=0 2024-04-07T16:53:41.715 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-97FqjuSvnQcgxQT off=12044530 len=653813 2024-04-07T16:53:41.722 INFO:tasks.workunit.client.0.smithi100.stdout:op 4535 completed, throughput=5MB/sec 2024-04-07T16:53:41.722 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-iK-GNJLins1ahRn off=15196309 len=1211957 2024-04-07T16:53:41.732 INFO:tasks.workunit.client.0.smithi100.stdout:op 4536 completed, throughput=5MB/sec 2024-04-07T16:53:41.733 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-AdkJq35cS6PZXoB off=4854403 len=1859469 2024-04-07T16:53:41.738 INFO:tasks.workunit.client.0.smithi100.stdout:op 4537 completed, throughput=5MB/sec 2024-04-07T16:53:41.738 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-hiEDlQ9ANwyZ_Df off=5228427 len=1974577 2024-04-07T16:53:41.753 INFO:tasks.workunit.client.0.smithi100.stdout:op 4538 completed, throughput=5MB/sec 2024-04-07T16:53:42.612 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:53:42 smithi144 bash[95233]: ts=2024-04-07T16:53:42.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:53:42.753 INFO:tasks.workunit.client.0.smithi100.stdout: 877: throughput=5MB/sec pending data=0 2024-04-07T16:53:42.754 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-rI-UlA81GMdQ_Mm off=4201119 len=1517778 2024-04-07T16:53:42.766 INFO:tasks.workunit.client.0.smithi100.stdout:op 4539 completed, throughput=5MB/sec 2024-04-07T16:53:42.766 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-RdCczqVqH3a4uuK off=1383563 len=249523 2024-04-07T16:53:42.772 INFO:tasks.workunit.client.0.smithi100.stdout:op 4540 completed, throughput=5MB/sec 2024-04-07T16:53:42.772 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-_SOjwutquHW_QRA off=13764807 len=398025 2024-04-07T16:53:42.774 INFO:tasks.workunit.client.0.smithi100.stdout:op 4541 completed, throughput=5MB/sec 2024-04-07T16:53:42.774 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-2jjJxTrKQpUtNbE off=906467 len=354159 2024-04-07T16:53:42.776 INFO:tasks.workunit.client.0.smithi100.stdout:op 4542 completed, throughput=5MB/sec 2024-04-07T16:53:42.776 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-H4TxiyRbsM3BQJC off=276843 len=1098330 2024-04-07T16:53:42.785 INFO:tasks.workunit.client.0.smithi100.stdout:op 4543 completed, throughput=5MB/sec 2024-04-07T16:53:42.785 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-KTdM_sFicyhl8lc off=3440956 len=2065403 2024-04-07T16:53:42.800 INFO:tasks.workunit.client.0.smithi100.stdout:op 4544 completed, throughput=5MB/sec 2024-04-07T16:53:42.948 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:53:42 smithi100 bash[74942]: cluster 2024-04-07T16:53:41.533260+0000 mgr.y (mgr.25175) 1150 : cluster [DBG] pgmap v1792: 105 pgs: 105 active+clean; 126 GiB data, 9.4 GiB used, 706 GiB / 715 GiB avail; 2.3 MiB/s rd, 2.7 MiB/s wr, 5 op/s 2024-04-07T16:53:42.949 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:53:42 smithi100 bash[76202]: cluster 2024-04-07T16:53:41.533260+0000 mgr.y (mgr.25175) 1150 : cluster [DBG] pgmap v1792: 105 pgs: 105 active+clean; 126 GiB data, 9.4 GiB used, 706 GiB / 715 GiB avail; 2.3 MiB/s rd, 2.7 MiB/s wr, 5 op/s 2024-04-07T16:53:43.112 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:53:42 smithi144 bash[66654]: cluster 2024-04-07T16:53:41.533260+0000 mgr.y (mgr.25175) 1150 : cluster [DBG] pgmap v1792: 105 pgs: 105 active+clean; 126 GiB data, 9.4 GiB used, 706 GiB / 715 GiB avail; 2.3 MiB/s rd, 2.7 MiB/s wr, 5 op/s 2024-04-07T16:53:43.800 INFO:tasks.workunit.client.0.smithi100.stdout: 878: throughput=5MB/sec pending data=0 2024-04-07T16:53:43.800 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-ojyccHY8CwKiaM_ off=4362906 len=1490927 2024-04-07T16:53:43.812 INFO:tasks.workunit.client.0.smithi100.stdout:op 4545 completed, throughput=5MB/sec 2024-04-07T16:53:43.812 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-0iyKFrOBZkxUrLD off=14877893 len=1992210 2024-04-07T16:53:43.826 INFO:tasks.workunit.client.0.smithi100.stdout:op 4546 completed, throughput=5MB/sec 2024-04-07T16:53:43.826 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-oZKnr7YdCkSLur6 off=845988 len=223379 2024-04-07T16:53:43.832 INFO:tasks.workunit.client.0.smithi100.stdout:op 4547 completed, throughput=5MB/sec 2024-04-07T16:53:43.832 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-xwz2azwirNktmzh off=5294693 len=279185 2024-04-07T16:53:43.834 INFO:tasks.workunit.client.0.smithi100.stdout:op 4548 completed, throughput=5MB/sec 2024-04-07T16:53:43.834 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-ECszPhJcCvxXmzk off=2376350 len=215785 2024-04-07T16:53:43.840 INFO:tasks.workunit.client.0.smithi100.stdout:op 4549 completed, throughput=5MB/sec 2024-04-07T16:53:44.840 INFO:tasks.workunit.client.0.smithi100.stdout: 879: throughput=4.99MB/sec pending data=0 2024-04-07T16:53:44.841 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-YxE23EWtVRBDYVR off=478396 len=1634751 2024-04-07T16:53:44.844 INFO:tasks.workunit.client.0.smithi100.stdout:op 4550 completed, throughput=5MB/sec 2024-04-07T16:53:44.844 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-5xNxkLNOtBApjmX off=1965887 len=1996741 2024-04-07T16:53:44.849 INFO:tasks.workunit.client.0.smithi100.stdout:op 4551 completed, throughput=5MB/sec 2024-04-07T16:53:44.849 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-1vu5qoF1V_hRfWB off=131632 len=1521081 2024-04-07T16:53:44.862 INFO:tasks.workunit.client.0.smithi100.stdout:op 4552 completed, throughput=5MB/sec 2024-04-07T16:53:44.862 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-a1F0v6cuqtXG9k7 off=653047 len=511501 2024-04-07T16:53:44.864 INFO:tasks.workunit.client.0.smithi100.stdout:op 4553 completed, throughput=5MB/sec 2024-04-07T16:53:44.948 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:53:44 smithi100 bash[74942]: cluster 2024-04-07T16:53:43.534406+0000 mgr.y (mgr.25175) 1151 : cluster [DBG] pgmap v1793: 105 pgs: 105 active+clean; 126 GiB data, 9.5 GiB used, 706 GiB / 715 GiB avail; 2.9 MiB/s rd, 3.1 MiB/s wr, 6 op/s 2024-04-07T16:53:44.949 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:53:44 smithi100 bash[76202]: cluster 2024-04-07T16:53:43.534406+0000 mgr.y (mgr.25175) 1151 : cluster [DBG] pgmap v1793: 105 pgs: 105 active+clean; 126 GiB data, 9.5 GiB used, 706 GiB / 715 GiB avail; 2.9 MiB/s rd, 3.1 MiB/s wr, 6 op/s 2024-04-07T16:53:45.112 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:53:44 smithi144 bash[66654]: cluster 2024-04-07T16:53:43.534406+0000 mgr.y (mgr.25175) 1151 : cluster [DBG] pgmap v1793: 105 pgs: 105 active+clean; 126 GiB data, 9.5 GiB used, 706 GiB / 715 GiB avail; 2.9 MiB/s rd, 3.1 MiB/s wr, 6 op/s 2024-04-07T16:53:45.865 INFO:tasks.workunit.client.0.smithi100.stdout: 880: throughput=4.99MB/sec pending data=0 2024-04-07T16:53:45.865 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-NCW12DCPiRZz-HU off=3116082 len=907816 2024-04-07T16:53:45.869 INFO:tasks.workunit.client.0.smithi100.stdout:op 4554 completed, throughput=5MB/sec 2024-04-07T16:53:45.869 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-_lzCrYQOllQ4Bdc off=6207823 len=821505 2024-04-07T16:53:45.877 INFO:tasks.workunit.client.0.smithi100.stdout:op 4555 completed, throughput=5MB/sec 2024-04-07T16:53:45.877 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-f0zxJLaV1P7eFF6 off=10506570 len=1819151 2024-04-07T16:53:45.882 INFO:tasks.workunit.client.0.smithi100.stdout:op 4556 completed, throughput=5MB/sec 2024-04-07T16:53:45.882 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Y0G5X4GT_9tZdIP off=1479038 len=2040069 2024-04-07T16:53:45.885 INFO:tasks.workunit.client.0.smithi100.stdout:op 4557 completed, throughput=5MB/sec 2024-04-07T16:53:46.885 INFO:tasks.workunit.client.0.smithi100.stdout: 881: throughput=5MB/sec pending data=0 2024-04-07T16:53:46.885 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-tkbikMxiwmgmoBB off=12128298 len=452421 2024-04-07T16:53:46.892 INFO:tasks.workunit.client.0.smithi100.stdout:op 4558 completed, throughput=5MB/sec 2024-04-07T16:53:46.893 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-LyqsKOqNcCzmZto off=3032237 len=1404134 2024-04-07T16:53:46.896 INFO:tasks.workunit.client.0.smithi100.stdout:op 4559 completed, throughput=5MB/sec 2024-04-07T16:53:46.896 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-pEIqiQIlwczQRZZ off=4188451 len=833327 2024-04-07T16:53:46.899 INFO:tasks.workunit.client.0.smithi100.stdout:op 4560 completed, throughput=5MB/sec 2024-04-07T16:53:46.899 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-NgDI2XaJqd7HaVC off=1767370 len=652806 2024-04-07T16:53:46.902 INFO:tasks.workunit.client.0.smithi100.stdout:op 4561 completed, throughput=5MB/sec 2024-04-07T16:53:46.902 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-BogfboiFG236tHh off=7650864 len=389895 2024-04-07T16:53:46.904 INFO:tasks.workunit.client.0.smithi100.stdout:op 4562 completed, throughput=5MB/sec 2024-04-07T16:53:46.904 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-6VjGTww-HYxeBkf off=3568019 len=1161896 2024-04-07T16:53:46.914 INFO:tasks.workunit.client.0.smithi100.stdout:op 4563 completed, throughput=5MB/sec 2024-04-07T16:53:46.948 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:53:46 smithi100 bash[74942]: cluster 2024-04-07T16:53:45.535410+0000 mgr.y (mgr.25175) 1152 : cluster [DBG] pgmap v1794: 105 pgs: 105 active+clean; 126 GiB data, 9.5 GiB used, 706 GiB / 715 GiB avail; 2.1 MiB/s rd, 2.9 MiB/s wr, 5 op/s 2024-04-07T16:53:46.949 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:53:46 smithi100 bash[76202]: cluster 2024-04-07T16:53:45.535410+0000 mgr.y (mgr.25175) 1152 : cluster [DBG] pgmap v1794: 105 pgs: 105 active+clean; 126 GiB data, 9.5 GiB used, 706 GiB / 715 GiB avail; 2.1 MiB/s rd, 2.9 MiB/s wr, 5 op/s 2024-04-07T16:53:47.112 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:53:46 smithi144 bash[66654]: cluster 2024-04-07T16:53:45.535410+0000 mgr.y (mgr.25175) 1152 : cluster [DBG] pgmap v1794: 105 pgs: 105 active+clean; 126 GiB data, 9.5 GiB used, 706 GiB / 715 GiB avail; 2.1 MiB/s rd, 2.9 MiB/s wr, 5 op/s 2024-04-07T16:53:47.914 INFO:tasks.workunit.client.0.smithi100.stdout: 882: throughput=4.99MB/sec pending data=0 2024-04-07T16:53:47.915 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-vGRtw0YYAB1hyq- off=6971122 len=462267 2024-04-07T16:53:47.916 INFO:tasks.workunit.client.0.smithi100.stdout:op 4564 completed, throughput=5MB/sec 2024-04-07T16:53:47.916 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-9nrWM5_Y1jOWIbz off=3525469 len=619216 2024-04-07T16:53:47.918 INFO:tasks.workunit.client.0.smithi100.stdout:op 4565 completed, throughput=5MB/sec 2024-04-07T16:53:47.918 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-02a_VRtVQo9lc0G off=7255778 len=425986 2024-04-07T16:53:47.925 INFO:tasks.workunit.client.0.smithi100.stdout:op 4566 completed, throughput=5MB/sec 2024-04-07T16:53:47.925 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-kgFPMHwCssaAZ70 off=10840817 len=597941 2024-04-07T16:53:47.932 INFO:tasks.workunit.client.0.smithi100.stdout:op 4567 completed, throughput=5MB/sec 2024-04-07T16:53:47.932 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-XUh9UCkfO7JhOf6 off=6685100 len=2040941 2024-04-07T16:53:47.937 INFO:tasks.workunit.client.0.smithi100.stdout:op 4568 completed, throughput=5MB/sec 2024-04-07T16:53:47.937 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-8yvN0XT7pLNzCfm off=4364603 len=1297312 2024-04-07T16:53:47.941 INFO:tasks.workunit.client.0.smithi100.stdout:op 4569 completed, throughput=5MB/sec 2024-04-07T16:53:47.948 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:53:47 smithi100 bash[74942]: audit 2024-04-07T16:53:46.781921+0000 mon.a (mon.0) 2173 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:53:47.948 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:53:47 smithi100 bash[76202]: audit 2024-04-07T16:53:46.781921+0000 mon.a (mon.0) 2173 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:53:48.112 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:53:47 smithi144 bash[66654]: audit 2024-04-07T16:53:46.781921+0000 mon.a (mon.0) 2173 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:53:48.684 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:53:48 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:53:48] "GET /metrics HTTP/1.1" 200 34076 "" "Prometheus/2.43.0" 2024-04-07T16:53:48.941 INFO:tasks.workunit.client.0.smithi100.stdout: 883: throughput=4.99MB/sec pending data=0 2024-04-07T16:53:48.941 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-B-csZB16wx8k438 off=8230400 len=1816496 2024-04-07T16:53:48.948 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:53:48 smithi100 bash[74942]: cluster 2024-04-07T16:53:47.536474+0000 mgr.y (mgr.25175) 1153 : cluster [DBG] pgmap v1795: 105 pgs: 105 active+clean; 126 GiB data, 9.5 GiB used, 706 GiB / 715 GiB avail; 1.9 MiB/s rd, 2.5 MiB/s wr, 4 op/s 2024-04-07T16:53:48.948 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:53:48 smithi100 bash[76202]: cluster 2024-04-07T16:53:47.536474+0000 mgr.y (mgr.25175) 1153 : cluster [DBG] pgmap v1795: 105 pgs: 105 active+clean; 126 GiB data, 9.5 GiB used, 706 GiB / 715 GiB avail; 1.9 MiB/s rd, 2.5 MiB/s wr, 4 op/s 2024-04-07T16:53:48.956 INFO:tasks.workunit.client.0.smithi100.stdout:op 4570 completed, throughput=5MB/sec 2024-04-07T16:53:48.956 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-wvUXcWs-mIV33EB off=203356 len=792012 2024-04-07T16:53:48.964 INFO:tasks.workunit.client.0.smithi100.stdout:op 4571 completed, throughput=5MB/sec 2024-04-07T16:53:48.965 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-q8yMfhb-Ou_1fHj off=1627647 len=320564 2024-04-07T16:53:48.970 INFO:tasks.workunit.client.0.smithi100.stdout:op 4572 completed, throughput=5MB/sec 2024-04-07T16:53:48.970 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-fpn95LlGn1FXoE8 off=6184972 len=967640 2024-04-07T16:53:48.972 INFO:tasks.workunit.client.0.smithi100.stdout:op 4573 completed, throughput=5MB/sec 2024-04-07T16:53:48.973 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-3uCmQmVUkDrZgXN off=3258209 len=1706299 2024-04-07T16:53:48.977 INFO:tasks.workunit.client.0.smithi100.stdout:op 4574 completed, throughput=5MB/sec 2024-04-07T16:53:49.112 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:53:48 smithi144 bash[66654]: cluster 2024-04-07T16:53:47.536474+0000 mgr.y (mgr.25175) 1153 : cluster [DBG] pgmap v1795: 105 pgs: 105 active+clean; 126 GiB data, 9.5 GiB used, 706 GiB / 715 GiB avail; 1.9 MiB/s rd, 2.5 MiB/s wr, 4 op/s 2024-04-07T16:53:49.977 INFO:tasks.workunit.client.0.smithi100.stdout: 884: throughput=4.99MB/sec pending data=0 2024-04-07T16:53:49.977 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-X327F7FKVA9oTzE off=11589992 len=924124 2024-04-07T16:53:49.981 INFO:tasks.workunit.client.0.smithi100.stdout:op 4575 completed, throughput=5MB/sec 2024-04-07T16:53:49.981 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-4IU17cVHADaw1V3 off=10117894 len=1186643 2024-04-07T16:53:49.985 INFO:tasks.workunit.client.0.smithi100.stdout:op 4576 completed, throughput=5MB/sec 2024-04-07T16:53:49.985 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-wDhGbD7y2djSwm8 off=2641580 len=702858 2024-04-07T16:53:49.993 INFO:tasks.workunit.client.0.smithi100.stdout:op 4577 completed, throughput=5MB/sec 2024-04-07T16:53:49.993 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-fR6zjnFu1ISzkcP off=1223871 len=1613709 2024-04-07T16:53:50.008 INFO:tasks.workunit.client.0.smithi100.stdout:op 4578 completed, throughput=5MB/sec 2024-04-07T16:53:50.008 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-tjkZbnych4e1cjL off=4082768 len=1978845 2024-04-07T16:53:50.026 INFO:tasks.workunit.client.0.smithi100.stdout:op 4579 completed, throughput=5MB/sec 2024-04-07T16:53:50.948 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:53:50 smithi100 bash[74942]: cluster 2024-04-07T16:53:49.537409+0000 mgr.y (mgr.25175) 1154 : cluster [DBG] pgmap v1796: 105 pgs: 105 active+clean; 126 GiB data, 9.5 GiB used, 706 GiB / 715 GiB avail; 2.3 MiB/s rd, 2.4 MiB/s wr, 4 op/s 2024-04-07T16:53:50.949 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:53:50 smithi100 bash[76202]: cluster 2024-04-07T16:53:49.537409+0000 mgr.y (mgr.25175) 1154 : cluster [DBG] pgmap v1796: 105 pgs: 105 active+clean; 126 GiB data, 9.5 GiB used, 706 GiB / 715 GiB avail; 2.3 MiB/s rd, 2.4 MiB/s wr, 4 op/s 2024-04-07T16:53:51.026 INFO:tasks.workunit.client.0.smithi100.stdout: 885: throughput=5MB/sec pending data=0 2024-04-07T16:53:51.027 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-i2XRwHdlcyA763Q off=162697 len=606708 2024-04-07T16:53:51.029 INFO:tasks.workunit.client.0.smithi100.stdout:op 4580 completed, throughput=5MB/sec 2024-04-07T16:53:51.029 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Gr1QrCdGAVbXms9 off=1307163 len=1639079 2024-04-07T16:53:51.033 INFO:tasks.workunit.client.0.smithi100.stdout:op 4581 completed, throughput=5MB/sec 2024-04-07T16:53:51.033 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Oj_Amc9YYzHEGGq off=8003699 len=249809 2024-04-07T16:53:51.039 INFO:tasks.workunit.client.0.smithi100.stdout:op 4582 completed, throughput=5MB/sec 2024-04-07T16:53:51.039 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-22YI0sqwtL_c4tp off=132981 len=827682 2024-04-07T16:53:51.046 INFO:tasks.workunit.client.0.smithi100.stdout:op 4583 completed, throughput=5MB/sec 2024-04-07T16:53:51.046 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Jyn_NDvxegippUV off=4413908 len=1227626 2024-04-07T16:53:51.049 INFO:tasks.workunit.client.0.smithi100.stdout:op 4584 completed, throughput=5MB/sec 2024-04-07T16:53:51.112 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:53:50 smithi144 bash[66654]: cluster 2024-04-07T16:53:49.537409+0000 mgr.y (mgr.25175) 1154 : cluster [DBG] pgmap v1796: 105 pgs: 105 active+clean; 126 GiB data, 9.5 GiB used, 706 GiB / 715 GiB avail; 2.3 MiB/s rd, 2.4 MiB/s wr, 4 op/s 2024-04-07T16:53:52.049 INFO:tasks.workunit.client.0.smithi100.stdout: 886: throughput=5MB/sec pending data=0 2024-04-07T16:53:52.050 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-xx8JWFvDwtn-o53 off=16603209 len=1508191 2024-04-07T16:53:52.063 INFO:tasks.workunit.client.0.smithi100.stdout:op 4585 completed, throughput=5MB/sec 2024-04-07T16:53:52.063 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-esnbVQXtWK0jsvf off=4386724 len=13242 2024-04-07T16:53:52.064 INFO:tasks.workunit.client.0.smithi100.stdout:op 4586 completed, throughput=5MB/sec 2024-04-07T16:53:52.064 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-d6wzmshVhdG2JWr off=12325450 len=663948 2024-04-07T16:53:52.066 INFO:tasks.workunit.client.0.smithi100.stdout:op 4587 completed, throughput=5MB/sec 2024-04-07T16:53:52.066 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-by6wHyheyEINGjg off=21615383 len=1076727 2024-04-07T16:53:52.070 INFO:tasks.workunit.client.0.smithi100.stdout:op 4588 completed, throughput=5MB/sec 2024-04-07T16:53:52.070 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-EhCY3ScYOqmMB17 off=471771 len=1408902 2024-04-07T16:53:52.074 INFO:tasks.workunit.client.0.smithi100.stdout:op 4589 completed, throughput=5MB/sec 2024-04-07T16:53:52.074 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Xe38hH97ItBzHK0 off=1045746 len=502567 2024-04-07T16:53:52.076 INFO:tasks.workunit.client.0.smithi100.stdout:op 4590 completed, throughput=5MB/sec 2024-04-07T16:53:52.612 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:53:52 smithi144 bash[95233]: ts=2024-04-07T16:53:52.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:53:52.948 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:53:52 smithi100 bash[74942]: cluster 2024-04-07T16:53:51.538270+0000 mgr.y (mgr.25175) 1155 : cluster [DBG] pgmap v1797: 105 pgs: 105 active+clean; 126 GiB data, 9.5 GiB used, 706 GiB / 715 GiB avail; 2.4 MiB/s rd, 2.3 MiB/s wr, 4 op/s 2024-04-07T16:53:52.949 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:53:52 smithi100 bash[76202]: cluster 2024-04-07T16:53:51.538270+0000 mgr.y (mgr.25175) 1155 : cluster [DBG] pgmap v1797: 105 pgs: 105 active+clean; 126 GiB data, 9.5 GiB used, 706 GiB / 715 GiB avail; 2.4 MiB/s rd, 2.3 MiB/s wr, 4 op/s 2024-04-07T16:53:53.076 INFO:tasks.workunit.client.0.smithi100.stdout: 887: throughput=4.99MB/sec pending data=0 2024-04-07T16:53:53.076 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-uLaOrCDi1WwcpPn off=19643185 len=1364327 2024-04-07T16:53:53.080 INFO:tasks.workunit.client.0.smithi100.stdout:op 4591 completed, throughput=5MB/sec 2024-04-07T16:53:53.080 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-oG-ceZDp49zpixS off=3340987 len=449264 2024-04-07T16:53:53.086 INFO:tasks.workunit.client.0.smithi100.stdout:op 4592 completed, throughput=5MB/sec 2024-04-07T16:53:53.086 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-_VyCqsjHaQC8L2b off=2269352 len=880572 2024-04-07T16:53:53.090 INFO:tasks.workunit.client.0.smithi100.stdout:op 4593 completed, throughput=5MB/sec 2024-04-07T16:53:53.090 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-yB0pjFk_ZvDDz_l off=187025 len=975545 2024-04-07T16:53:53.093 INFO:tasks.workunit.client.0.smithi100.stdout:op 4594 completed, throughput=5MB/sec 2024-04-07T16:53:53.093 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-vWhAzs2eINOaGwf off=297418 len=244113 2024-04-07T16:53:53.094 INFO:tasks.workunit.client.0.smithi100.stdout:op 4595 completed, throughput=5MB/sec 2024-04-07T16:53:53.094 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-iNXHoT-iqmC9gxJ off=2442895 len=1200656 2024-04-07T16:53:53.097 INFO:tasks.workunit.client.0.smithi100.stdout:op 4596 completed, throughput=5MB/sec 2024-04-07T16:53:53.112 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:53:52 smithi144 bash[66654]: cluster 2024-04-07T16:53:51.538270+0000 mgr.y (mgr.25175) 1155 : cluster [DBG] pgmap v1797: 105 pgs: 105 active+clean; 126 GiB data, 9.5 GiB used, 706 GiB / 715 GiB avail; 2.4 MiB/s rd, 2.3 MiB/s wr, 4 op/s 2024-04-07T16:53:54.097 INFO:tasks.workunit.client.0.smithi100.stdout: 888: throughput=4.99MB/sec pending data=0 2024-04-07T16:53:54.098 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-8oj4O_P46-MltuT off=4402223 len=1551298 2024-04-07T16:53:54.111 INFO:tasks.workunit.client.0.smithi100.stdout:op 4597 completed, throughput=5MB/sec 2024-04-07T16:53:54.111 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-NlJRRTm1EKvkn4q off=16504325 len=1530888 2024-04-07T16:53:54.114 INFO:tasks.workunit.client.0.smithi100.stdout:op 4598 completed, throughput=5MB/sec 2024-04-07T16:53:54.114 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-1lF1aTftgnPveu_ off=1387259 len=326585 2024-04-07T16:53:54.116 INFO:tasks.workunit.client.0.smithi100.stdout:op 4599 completed, throughput=5MB/sec 2024-04-07T16:53:54.116 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-IuNnb4s4WF5onIc off=0 len=1830371 2024-04-07T16:53:54.129 INFO:tasks.workunit.client.0.smithi100.stdout:op 4600 completed, throughput=5MB/sec 2024-04-07T16:53:54.129 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Z7JHyG_oZv32qM1 off=6959899 len=2051543 2024-04-07T16:53:54.136 INFO:tasks.workunit.client.0.smithi100.stdout:op 4601 completed, throughput=5MB/sec 2024-04-07T16:53:54.948 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:53:54 smithi100 bash[74942]: cluster 2024-04-07T16:53:53.539439+0000 mgr.y (mgr.25175) 1156 : cluster [DBG] pgmap v1798: 105 pgs: 105 active+clean; 126 GiB data, 9.5 GiB used, 706 GiB / 715 GiB avail; 2.8 MiB/s rd, 2.4 MiB/s wr, 5 op/s 2024-04-07T16:53:54.949 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:53:54 smithi100 bash[76202]: cluster 2024-04-07T16:53:53.539439+0000 mgr.y (mgr.25175) 1156 : cluster [DBG] pgmap v1798: 105 pgs: 105 active+clean; 126 GiB data, 9.5 GiB used, 706 GiB / 715 GiB avail; 2.8 MiB/s rd, 2.4 MiB/s wr, 5 op/s 2024-04-07T16:53:55.112 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:53:54 smithi144 bash[66654]: cluster 2024-04-07T16:53:53.539439+0000 mgr.y (mgr.25175) 1156 : cluster [DBG] pgmap v1798: 105 pgs: 105 active+clean; 126 GiB data, 9.5 GiB used, 706 GiB / 715 GiB avail; 2.8 MiB/s rd, 2.4 MiB/s wr, 5 op/s 2024-04-07T16:53:55.136 INFO:tasks.workunit.client.0.smithi100.stdout: 889: throughput=5MB/sec pending data=0 2024-04-07T16:53:55.136 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-U5v2mqsL670_mya off=11495215 len=1038057 2024-04-07T16:53:55.139 INFO:tasks.workunit.client.0.smithi100.stdout:op 4602 completed, throughput=5MB/sec 2024-04-07T16:53:55.139 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-bXhpGen1l1MZVid off=1703869 len=255837 2024-04-07T16:53:55.145 INFO:tasks.workunit.client.0.smithi100.stdout:op 4603 completed, throughput=5MB/sec 2024-04-07T16:53:55.145 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-A9Lr5GYzC8J2OSt off=1080043 len=129301 2024-04-07T16:53:55.150 INFO:tasks.workunit.client.0.smithi100.stdout:op 4604 completed, throughput=5MB/sec 2024-04-07T16:53:55.150 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-EVBp4mxgbP78m3i off=1107490 len=793046 2024-04-07T16:53:55.152 INFO:tasks.workunit.client.0.smithi100.stdout:op 4605 completed, throughput=5MB/sec 2024-04-07T16:53:55.152 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Hr0ReuT6L4m3uBX off=20625908 len=1566861 2024-04-07T16:53:55.165 INFO:tasks.workunit.client.0.smithi100.stdout:op 4606 completed, throughput=5MB/sec 2024-04-07T16:53:56.112 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:53:55 smithi144 bash[66654]: cluster 2024-04-07T16:53:55.540400+0000 mgr.y (mgr.25175) 1157 : cluster [DBG] pgmap v1799: 105 pgs: 105 active+clean; 126 GiB data, 9.5 GiB used, 706 GiB / 715 GiB avail; 2.6 MiB/s rd, 2.0 MiB/s wr, 4 op/s 2024-04-07T16:53:56.165 INFO:tasks.workunit.client.0.smithi100.stdout: 890: throughput=4.99MB/sec pending data=0 2024-04-07T16:53:56.166 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-RAOPFvRaRYyzcYM off=6133048 len=291684 2024-04-07T16:53:56.171 INFO:tasks.workunit.client.0.smithi100.stdout:op 4607 completed, throughput=5MB/sec 2024-04-07T16:53:56.171 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-IFTVWmL5ZMUK3Pn off=18942799 len=1983985 2024-04-07T16:53:56.188 INFO:tasks.workunit.client.0.smithi100.stdout:op 4608 completed, throughput=5MB/sec 2024-04-07T16:53:56.188 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-TXM4xhBM5Eujzzz off=10928205 len=260384 2024-04-07T16:53:56.189 INFO:tasks.workunit.client.0.smithi100.stdout:op 4609 completed, throughput=5MB/sec 2024-04-07T16:53:56.189 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-R_8tqeMq9nDFW05 off=4775207 len=826884 2024-04-07T16:53:56.192 INFO:tasks.workunit.client.0.smithi100.stdout:op 4610 completed, throughput=5MB/sec 2024-04-07T16:53:56.192 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-TXiTHQQELNnZqOJ off=9888254 len=456157 2024-04-07T16:53:56.194 INFO:tasks.workunit.client.0.smithi100.stdout:op 4611 completed, throughput=5MB/sec 2024-04-07T16:53:56.194 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-JGYRp_GAaNzmQkI off=1984087 len=1640884 2024-04-07T16:53:56.197 INFO:tasks.workunit.client.0.smithi100.stdout:op 4612 completed, throughput=5MB/sec 2024-04-07T16:53:56.198 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:53:55 smithi100 bash[74942]: cluster 2024-04-07T16:53:55.540400+0000 mgr.y (mgr.25175) 1157 : cluster [DBG] pgmap v1799: 105 pgs: 105 active+clean; 126 GiB data, 9.5 GiB used, 706 GiB / 715 GiB avail; 2.6 MiB/s rd, 2.0 MiB/s wr, 4 op/s 2024-04-07T16:53:56.201 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:53:55 smithi100 bash[76202]: cluster 2024-04-07T16:53:55.540400+0000 mgr.y (mgr.25175) 1157 : cluster [DBG] pgmap v1799: 105 pgs: 105 active+clean; 126 GiB data, 9.5 GiB used, 706 GiB / 715 GiB avail; 2.6 MiB/s rd, 2.0 MiB/s wr, 4 op/s 2024-04-07T16:53:57.198 INFO:tasks.workunit.client.0.smithi100.stdout: 891: throughput=4.99MB/sec pending data=0 2024-04-07T16:53:57.198 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-KO_HInAkj-GSJES off=8409620 len=580487 2024-04-07T16:53:57.200 INFO:tasks.workunit.client.0.smithi100.stdout:op 4613 completed, throughput=5MB/sec 2024-04-07T16:53:57.200 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-cE4LuZi5SmytuvT off=4654758 len=1813 2024-04-07T16:53:57.204 INFO:tasks.workunit.client.0.smithi100.stdout:op 4614 completed, throughput=5MB/sec 2024-04-07T16:53:57.204 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-eoeC5Z8rxk3W2aa off=4902453 len=1735378 2024-04-07T16:53:57.219 INFO:tasks.workunit.client.0.smithi100.stdout:op 4615 completed, throughput=5MB/sec 2024-04-07T16:53:57.219 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-k8sJFTBnBCFdShw off=4695076 len=609480 2024-04-07T16:53:57.227 INFO:tasks.workunit.client.0.smithi100.stdout:op 4616 completed, throughput=5MB/sec 2024-04-07T16:53:57.227 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-fy98nyg_eG8m2N8 off=5244849 len=1764195 2024-04-07T16:53:57.232 INFO:tasks.workunit.client.0.smithi100.stdout:op 4617 completed, throughput=5MB/sec 2024-04-07T16:53:57.232 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Nmqx1Azn1fM15lX off=6252400 len=1864560 2024-04-07T16:53:57.246 INFO:tasks.workunit.client.0.smithi100.stdout:op 4618 completed, throughput=5MB/sec 2024-04-07T16:53:58.247 INFO:tasks.workunit.client.0.smithi100.stdout: 892: throughput=5MB/sec pending data=0 2024-04-07T16:53:58.247 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-U6_H7zI7PWJOtuT off=3520243 len=438284 2024-04-07T16:53:58.253 INFO:tasks.workunit.client.0.smithi100.stdout:op 4619 completed, throughput=5MB/sec 2024-04-07T16:53:58.253 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-RHIgKbia2twUKjP off=3732760 len=1590856 2024-04-07T16:53:58.259 INFO:tasks.workunit.client.0.smithi100.stdout:op 4620 completed, throughput=5MB/sec 2024-04-07T16:53:58.259 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-CAHz-N3nRUxfmRQ off=10713174 len=1406831 2024-04-07T16:53:58.271 INFO:tasks.workunit.client.0.smithi100.stdout:op 4621 completed, throughput=5MB/sec 2024-04-07T16:53:58.271 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-nZCmiQfKwusTA0h off=11966784 len=1814989 2024-04-07T16:53:58.275 INFO:tasks.workunit.client.0.smithi100.stdout:op 4622 completed, throughput=5MB/sec 2024-04-07T16:53:58.698 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:53:58 smithi100 bash[74942]: cluster 2024-04-07T16:53:57.541566+0000 mgr.y (mgr.25175) 1158 : cluster [DBG] pgmap v1800: 105 pgs: 105 active+clean; 126 GiB data, 9.5 GiB used, 706 GiB / 715 GiB avail; 2.9 MiB/s rd, 1.7 MiB/s wr, 4 op/s 2024-04-07T16:53:58.699 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:53:58 smithi100 bash[76202]: cluster 2024-04-07T16:53:57.541566+0000 mgr.y (mgr.25175) 1158 : cluster [DBG] pgmap v1800: 105 pgs: 105 active+clean; 126 GiB data, 9.5 GiB used, 706 GiB / 715 GiB avail; 2.9 MiB/s rd, 1.7 MiB/s wr, 4 op/s 2024-04-07T16:53:58.699 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:53:58 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:53:58] "GET /metrics HTTP/1.1" 200 34076 "" "Prometheus/2.43.0" 2024-04-07T16:53:58.862 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:53:58 smithi144 bash[66654]: cluster 2024-04-07T16:53:57.541566+0000 mgr.y (mgr.25175) 1158 : cluster [DBG] pgmap v1800: 105 pgs: 105 active+clean; 126 GiB data, 9.5 GiB used, 706 GiB / 715 GiB avail; 2.9 MiB/s rd, 1.7 MiB/s wr, 4 op/s 2024-04-07T16:53:59.276 INFO:tasks.workunit.client.0.smithi100.stdout: 893: throughput=5MB/sec pending data=0 2024-04-07T16:53:59.276 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-tgjf18zkWJg3J-T off=6536632 len=1918288 2024-04-07T16:53:59.290 INFO:tasks.workunit.client.0.smithi100.stdout:op 4623 completed, throughput=5MB/sec 2024-04-07T16:53:59.290 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-j1V_y5Fanb7_BAL off=11759121 len=706497 2024-04-07T16:53:59.297 INFO:tasks.workunit.client.0.smithi100.stdout:op 4624 completed, throughput=5MB/sec 2024-04-07T16:53:59.298 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-UFSAWsrrgogDasZ off=481894 len=640645 2024-04-07T16:53:59.305 INFO:tasks.workunit.client.0.smithi100.stdout:op 4625 completed, throughput=5MB/sec 2024-04-07T16:53:59.306 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-W1pHKm-v8BojPHR off=1920538 len=924737 2024-04-07T16:53:59.314 INFO:tasks.workunit.client.0.smithi100.stdout:op 4626 completed, throughput=5MB/sec 2024-04-07T16:53:59.314 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-4qpYS35Fe6dTcxk off=6892979 len=1143980 2024-04-07T16:53:59.322 INFO:tasks.workunit.client.0.smithi100.stdout:op 4627 completed, throughput=5MB/sec 2024-04-07T16:54:00.323 INFO:tasks.workunit.client.0.smithi100.stdout: 894: throughput=5MB/sec pending data=0 2024-04-07T16:54:00.323 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-8LhNbJsbU7A7SJQ off=8810920 len=514282 2024-04-07T16:54:00.325 INFO:tasks.workunit.client.0.smithi100.stdout:op 4628 completed, throughput=5MB/sec 2024-04-07T16:54:00.325 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-j-OIkfGSUEj5Al1 off=22377095 len=1627752 2024-04-07T16:54:00.331 INFO:tasks.workunit.client.0.smithi100.stdout:op 4629 completed, throughput=5MB/sec 2024-04-07T16:54:00.331 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-JYHXTiZaWE7ZTuu off=6577164 len=111290 2024-04-07T16:54:00.336 INFO:tasks.workunit.client.0.smithi100.stdout:op 4630 completed, throughput=5MB/sec 2024-04-07T16:54:00.336 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-7ise0ZBLJ2Immbi off=12156075 len=1732143 2024-04-07T16:54:00.341 INFO:tasks.workunit.client.0.smithi100.stdout:op 4631 completed, throughput=5MB/sec 2024-04-07T16:54:00.341 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-88xASSW0dsP5poI off=5183707 len=371899 2024-04-07T16:54:00.348 INFO:tasks.workunit.client.0.smithi100.stdout:op 4632 completed, throughput=5MB/sec 2024-04-07T16:54:00.862 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:54:00 smithi144 bash[66654]: cluster 2024-04-07T16:53:59.542478+0000 mgr.y (mgr.25175) 1159 : cluster [DBG] pgmap v1801: 105 pgs: 105 active+clean; 126 GiB data, 9.5 GiB used, 706 GiB / 715 GiB avail; 2.8 MiB/s rd, 1.7 MiB/s wr, 4 op/s 2024-04-07T16:54:00.948 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:54:00 smithi100 bash[74942]: cluster 2024-04-07T16:53:59.542478+0000 mgr.y (mgr.25175) 1159 : cluster [DBG] pgmap v1801: 105 pgs: 105 active+clean; 126 GiB data, 9.5 GiB used, 706 GiB / 715 GiB avail; 2.8 MiB/s rd, 1.7 MiB/s wr, 4 op/s 2024-04-07T16:54:00.949 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:54:00 smithi100 bash[76202]: cluster 2024-04-07T16:53:59.542478+0000 mgr.y (mgr.25175) 1159 : cluster [DBG] pgmap v1801: 105 pgs: 105 active+clean; 126 GiB data, 9.5 GiB used, 706 GiB / 715 GiB avail; 2.8 MiB/s rd, 1.7 MiB/s wr, 4 op/s 2024-04-07T16:54:01.348 INFO:tasks.workunit.client.0.smithi100.stdout: 895: throughput=4.99MB/sec pending data=0 2024-04-07T16:54:01.349 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-tJ6_TuZ5DWZw2Pf off=15149150 len=854972 2024-04-07T16:54:01.356 INFO:tasks.workunit.client.0.smithi100.stdout:op 4633 completed, throughput=5MB/sec 2024-04-07T16:54:01.356 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-eAKWnzvLvK0tWLw off=3280317 len=1673847 2024-04-07T16:54:01.369 INFO:tasks.workunit.client.0.smithi100.stdout:op 4634 completed, throughput=5MB/sec 2024-04-07T16:54:01.369 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-MLJ0sHaoPu45OTY off=4852661 len=1855632 2024-04-07T16:54:01.376 INFO:tasks.workunit.client.0.smithi100.stdout:op 4635 completed, throughput=5MB/sec 2024-04-07T16:54:01.376 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-JC44gvC_j5uH4u6 off=0 len=1632287 2024-04-07T16:54:01.378 INFO:tasks.workunit.client.0.smithi100.stdout:op 4636 completed, throughput=5MB/sec 2024-04-07T16:54:02.378 INFO:tasks.workunit.client.0.smithi100.stdout: 896: throughput=5MB/sec pending data=0 2024-04-07T16:54:02.379 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-qnRltXRKWleNMpe off=9466499 len=1345816 2024-04-07T16:54:02.391 INFO:tasks.workunit.client.0.smithi100.stdout:op 4637 completed, throughput=5MB/sec 2024-04-07T16:54:02.391 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-KRExT6kfuPKPNfQ off=4229384 len=852688 2024-04-07T16:54:02.400 INFO:tasks.workunit.client.0.smithi100.stdout:op 4638 completed, throughput=5MB/sec 2024-04-07T16:54:02.400 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-ityiTXKJeN9nQD3 off=4943077 len=1082134 2024-04-07T16:54:02.403 INFO:tasks.workunit.client.0.smithi100.stdout:op 4639 completed, throughput=5MB/sec 2024-04-07T16:54:02.403 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-U-2MnFPcePcVsid off=722152 len=1337314 2024-04-07T16:54:02.406 INFO:tasks.workunit.client.0.smithi100.stdout:op 4640 completed, throughput=5MB/sec 2024-04-07T16:54:02.407 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-gtBTZXoDSvk14R8 off=12854115 len=1270243 2024-04-07T16:54:02.418 INFO:tasks.workunit.client.0.smithi100.stdout:op 4641 completed, throughput=5MB/sec 2024-04-07T16:54:02.602 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:54:02 smithi144 bash[95233]: ts=2024-04-07T16:54:02.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:54:02.862 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:54:02 smithi144 bash[66654]: cluster 2024-04-07T16:54:01.543285+0000 mgr.y (mgr.25175) 1160 : cluster [DBG] pgmap v1802: 105 pgs: 105 active+clean; 126 GiB data, 9.6 GiB used, 706 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.1 MiB/s wr, 5 op/s 2024-04-07T16:54:02.862 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:54:02 smithi144 bash[66654]: audit 2024-04-07T16:54:01.782405+0000 mon.a (mon.0) 2174 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:54:02.949 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:54:02 smithi100 bash[74942]: cluster 2024-04-07T16:54:01.543285+0000 mgr.y (mgr.25175) 1160 : cluster [DBG] pgmap v1802: 105 pgs: 105 active+clean; 126 GiB data, 9.6 GiB used, 706 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.1 MiB/s wr, 5 op/s 2024-04-07T16:54:02.949 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:54:02 smithi100 bash[74942]: audit 2024-04-07T16:54:01.782405+0000 mon.a (mon.0) 2174 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:54:02.949 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:54:02 smithi100 bash[76202]: cluster 2024-04-07T16:54:01.543285+0000 mgr.y (mgr.25175) 1160 : cluster [DBG] pgmap v1802: 105 pgs: 105 active+clean; 126 GiB data, 9.6 GiB used, 706 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.1 MiB/s wr, 5 op/s 2024-04-07T16:54:02.949 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:54:02 smithi100 bash[76202]: audit 2024-04-07T16:54:01.782405+0000 mon.a (mon.0) 2174 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:54:03.418 INFO:tasks.workunit.client.0.smithi100.stdout: 897: throughput=5MB/sec pending data=0 2024-04-07T16:54:03.620 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-rhcQ9802oCzo5Cg off=3599344 len=1947730 2024-04-07T16:54:03.621 INFO:tasks.workunit.client.0.smithi100.stdout:op 4642 completed, throughput=5MB/sec 2024-04-07T16:54:03.621 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Wdg5Qv1IweQkVE_ off=7180397 len=1056087 2024-04-07T16:54:03.621 INFO:tasks.workunit.client.0.smithi100.stdout:op 4643 completed, throughput=5MB/sec 2024-04-07T16:54:03.621 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-15my5hmVYmE0yKs off=4513365 len=197544 2024-04-07T16:54:03.621 INFO:tasks.workunit.client.0.smithi100.stdout:op 4644 completed, throughput=5MB/sec 2024-04-07T16:54:03.622 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-YlRB6GBZNu7WtHV off=6076890 len=247134 2024-04-07T16:54:03.622 INFO:tasks.workunit.client.0.smithi100.stdout:op 4645 completed, throughput=5MB/sec 2024-04-07T16:54:03.622 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-7eszuOtbhHh2Kbq off=15650398 len=104456 2024-04-07T16:54:03.622 INFO:tasks.workunit.client.0.smithi100.stdout:op 4646 completed, throughput=5MB/sec 2024-04-07T16:54:03.622 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-CFwzKobuSaNMZBE off=4866250 len=1496710 2024-04-07T16:54:03.622 INFO:tasks.workunit.client.0.smithi100.stdout:op 4647 completed, throughput=5MB/sec 2024-04-07T16:54:04.454 INFO:tasks.workunit.client.0.smithi100.stdout: 898: throughput=5MB/sec pending data=0 2024-04-07T16:54:04.454 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-6bakz5c8aBh_n9L off=1080792 len=81398 2024-04-07T16:54:04.455 INFO:tasks.workunit.client.0.smithi100.stdout:op 4648 completed, throughput=5MB/sec 2024-04-07T16:54:04.455 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-UNDqSmfl553N2il off=14396514 len=529662 2024-04-07T16:54:04.462 INFO:tasks.workunit.client.0.smithi100.stdout:op 4649 completed, throughput=5MB/sec 2024-04-07T16:54:04.462 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-crD9Gc5yR8Ysz4x off=4619987 len=973654 2024-04-07T16:54:04.464 INFO:tasks.workunit.client.0.smithi100.stdout:op 4650 completed, throughput=5MB/sec 2024-04-07T16:54:04.465 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-RxuUSFbcpiW7muJ off=2728726 len=887058 2024-04-07T16:54:04.473 INFO:tasks.workunit.client.0.smithi100.stdout:op 4651 completed, throughput=5MB/sec 2024-04-07T16:54:04.473 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-JTpL3rG0BWd8HYJ off=15416522 len=35430 2024-04-07T16:54:04.477 INFO:tasks.workunit.client.0.smithi100.stdout:op 4652 completed, throughput=5MB/sec 2024-04-07T16:54:04.477 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-gelt3NhiZj3bBMg off=10054162 len=657307 2024-04-07T16:54:04.485 INFO:tasks.workunit.client.0.smithi100.stdout:op 4653 completed, throughput=5MB/sec 2024-04-07T16:54:04.485 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-XUIugN_8-D9AAz9 off=19926790 len=1240810 2024-04-07T16:54:04.495 INFO:tasks.workunit.client.0.smithi100.stdout:op 4654 completed, throughput=5MB/sec 2024-04-07T16:54:04.495 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-J-oMtRDrgGM7IVX off=1282223 len=1180326 2024-04-07T16:54:04.498 INFO:tasks.workunit.client.0.smithi100.stdout:op 4655 completed, throughput=5MB/sec 2024-04-07T16:54:04.862 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:54:04 smithi144 bash[66654]: cluster 2024-04-07T16:54:03.544434+0000 mgr.y (mgr.25175) 1161 : cluster [DBG] pgmap v1803: 105 pgs: 105 active+clean; 126 GiB data, 9.6 GiB used, 706 GiB / 715 GiB avail; 3.1 MiB/s rd, 2.6 MiB/s wr, 5 op/s 2024-04-07T16:54:04.948 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:54:04 smithi100 bash[74942]: cluster 2024-04-07T16:54:03.544434+0000 mgr.y (mgr.25175) 1161 : cluster [DBG] pgmap v1803: 105 pgs: 105 active+clean; 126 GiB data, 9.6 GiB used, 706 GiB / 715 GiB avail; 3.1 MiB/s rd, 2.6 MiB/s wr, 5 op/s 2024-04-07T16:54:04.949 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:54:04 smithi100 bash[76202]: cluster 2024-04-07T16:54:03.544434+0000 mgr.y (mgr.25175) 1161 : cluster [DBG] pgmap v1803: 105 pgs: 105 active+clean; 126 GiB data, 9.6 GiB used, 706 GiB / 715 GiB avail; 3.1 MiB/s rd, 2.6 MiB/s wr, 5 op/s 2024-04-07T16:54:05.499 INFO:tasks.workunit.client.0.smithi100.stdout: 899: throughput=5MB/sec pending data=0 2024-04-07T16:54:05.499 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-iKuMV2FBmd33I5I off=1487636 len=381874 2024-04-07T16:54:05.501 INFO:tasks.workunit.client.0.smithi100.stdout:op 4656 completed, throughput=5MB/sec 2024-04-07T16:54:05.501 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-SdJ_qd15AkmeYXt off=7371328 len=832497 2024-04-07T16:54:05.509 INFO:tasks.workunit.client.0.smithi100.stdout:op 4657 completed, throughput=5MB/sec 2024-04-07T16:54:05.510 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-l0D8KSMvWcsO3g3 off=8712671 len=1313631 2024-04-07T16:54:05.513 INFO:tasks.workunit.client.0.smithi100.stdout:op 4658 completed, throughput=5MB/sec 2024-04-07T16:54:05.513 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-FnpAR2otOu2b9W3 off=16972509 len=570630 2024-04-07T16:54:05.515 INFO:tasks.workunit.client.0.smithi100.stdout:op 4659 completed, throughput=5MB/sec 2024-04-07T16:54:05.515 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-_Cx5uzVT3IcVRiq off=1219062 len=1162851 2024-04-07T16:54:05.518 INFO:tasks.workunit.client.0.smithi100.stdout:op 4660 completed, throughput=5MB/sec 2024-04-07T16:54:05.518 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-CXUrOR4NJ4ZIuCY off=3028275 len=165517 2024-04-07T16:54:05.524 INFO:tasks.workunit.client.0.smithi100.stdout:op 4661 completed, throughput=5MB/sec 2024-04-07T16:54:05.524 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-wgGqUptxVPyfP9C off=14477701 len=1219201 2024-04-07T16:54:05.534 INFO:tasks.workunit.client.0.smithi100.stdout:op 4662 completed, throughput=5MB/sec 2024-04-07T16:54:06.535 INFO:tasks.workunit.client.0.smithi100.stdout: 900: throughput=5MB/sec pending data=0 2024-04-07T16:54:06.535 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Li8Brok9cEvlYr1 off=10976389 len=239547 2024-04-07T16:54:06.536 INFO:tasks.workunit.client.0.smithi100.stdout:op 4663 completed, throughput=5MB/sec 2024-04-07T16:54:06.536 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-MP8DRduViC_BN1_ off=9622984 len=52784 2024-04-07T16:54:06.537 INFO:tasks.workunit.client.0.smithi100.stdout:op 4664 completed, throughput=5MB/sec 2024-04-07T16:54:06.537 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-m959Zc9BONTRoiZ off=1531715 len=667543 2024-04-07T16:54:06.540 INFO:tasks.workunit.client.0.smithi100.stdout:op 4665 completed, throughput=5MB/sec 2024-04-07T16:54:06.541 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-VJU1PMED1lixg82 off=1652372 len=705160 2024-04-07T16:54:06.543 INFO:tasks.workunit.client.0.smithi100.stdout:op 4666 completed, throughput=5MB/sec 2024-04-07T16:54:06.543 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-sUe4Vtvsk9D2pax off=2269125 len=933395 2024-04-07T16:54:06.545 INFO:tasks.workunit.client.0.smithi100.stdout:op 4667 completed, throughput=5MB/sec 2024-04-07T16:54:06.546 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-t90Yz2kvLlrMkwB off=8160822 len=1020907 2024-04-07T16:54:06.554 INFO:tasks.workunit.client.0.smithi100.stdout:op 4668 completed, throughput=5MB/sec 2024-04-07T16:54:06.554 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-J7HapWXXJZ1whxT off=5041190 len=893792 2024-04-07T16:54:06.563 INFO:tasks.workunit.client.0.smithi100.stdout:op 4669 completed, throughput=5MB/sec 2024-04-07T16:54:06.862 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:54:06 smithi144 bash[66654]: cluster 2024-04-07T16:54:05.545441+0000 mgr.y (mgr.25175) 1162 : cluster [DBG] pgmap v1804: 105 pgs: 105 active+clean; 126 GiB data, 9.6 GiB used, 706 GiB / 715 GiB avail; 2.8 MiB/s rd, 2.3 MiB/s wr, 5 op/s 2024-04-07T16:54:06.948 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:54:06 smithi100 bash[74942]: cluster 2024-04-07T16:54:05.545441+0000 mgr.y (mgr.25175) 1162 : cluster [DBG] pgmap v1804: 105 pgs: 105 active+clean; 126 GiB data, 9.6 GiB used, 706 GiB / 715 GiB avail; 2.8 MiB/s rd, 2.3 MiB/s wr, 5 op/s 2024-04-07T16:54:06.949 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:54:06 smithi100 bash[76202]: cluster 2024-04-07T16:54:05.545441+0000 mgr.y (mgr.25175) 1162 : cluster [DBG] pgmap v1804: 105 pgs: 105 active+clean; 126 GiB data, 9.6 GiB used, 706 GiB / 715 GiB avail; 2.8 MiB/s rd, 2.3 MiB/s wr, 5 op/s 2024-04-07T16:54:07.563 INFO:tasks.workunit.client.0.smithi100.stdout: 901: throughput=4.99MB/sec pending data=0 2024-04-07T16:54:07.563 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-zckANJQgnjSrOm- off=6050601 len=1203351 2024-04-07T16:54:07.565 INFO:tasks.workunit.client.0.smithi100.stdout:op 4670 completed, throughput=5MB/sec 2024-04-07T16:54:07.565 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-OGZ-c1c3vUoZbXm off=7409788 len=1572338 2024-04-07T16:54:07.574 INFO:tasks.workunit.client.0.smithi100.stdout:op 4671 completed, throughput=5MB/sec 2024-04-07T16:54:07.574 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-8JN-6-3mg9hYjTH off=17495081 len=1283859 2024-04-07T16:54:07.576 INFO:tasks.workunit.client.0.smithi100.stdout:op 4672 completed, throughput=5MB/sec 2024-04-07T16:54:07.576 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-t3HqTS4vSUsL8kd off=2722252 len=36931 2024-04-07T16:54:07.579 INFO:tasks.workunit.client.0.smithi100.stdout:op 4673 completed, throughput=5MB/sec 2024-04-07T16:54:07.579 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-JTeN7hkfauzsR8- off=20992184 len=148754 2024-04-07T16:54:07.580 INFO:tasks.workunit.client.0.smithi100.stdout:op 4674 completed, throughput=5MB/sec 2024-04-07T16:54:07.580 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-_JfpKr7_BUybJMz off=2447426 len=809254 2024-04-07T16:54:07.581 INFO:tasks.workunit.client.0.smithi100.stdout:op 4675 completed, throughput=5MB/sec 2024-04-07T16:54:07.581 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-zHljeTQjmFCyB5q off=1587619 len=1938698 2024-04-07T16:54:07.583 INFO:tasks.workunit.client.0.smithi100.stdout:op 4676 completed, throughput=5MB/sec 2024-04-07T16:54:07.862 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:54:07 smithi144 bash[66654]: audit 2024-04-07T16:54:06.993961+0000 mon.a (mon.0) 2175 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:54:07.862 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:54:07 smithi144 bash[66654]: audit 2024-04-07T16:54:07.518759+0000 mon.a (mon.0) 2176 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:54:07.863 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:54:07 smithi144 bash[66654]: audit 2024-04-07T16:54:07.526374+0000 mon.a (mon.0) 2177 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:54:07.948 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:54:07 smithi100 bash[76202]: audit 2024-04-07T16:54:06.993961+0000 mon.a (mon.0) 2175 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:54:07.949 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:54:07 smithi100 bash[76202]: audit 2024-04-07T16:54:07.518759+0000 mon.a (mon.0) 2176 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:54:07.949 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:54:07 smithi100 bash[76202]: audit 2024-04-07T16:54:07.526374+0000 mon.a (mon.0) 2177 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:54:07.949 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:54:07 smithi100 bash[74942]: audit 2024-04-07T16:54:06.993961+0000 mon.a (mon.0) 2175 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:54:07.949 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:54:07 smithi100 bash[74942]: audit 2024-04-07T16:54:07.518759+0000 mon.a (mon.0) 2176 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:54:07.949 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:54:07 smithi100 bash[74942]: audit 2024-04-07T16:54:07.526374+0000 mon.a (mon.0) 2177 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:54:08.584 INFO:tasks.workunit.client.0.smithi100.stdout: 902: throughput=5MB/sec pending data=0 2024-04-07T16:54:08.584 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-7aoc4Z439PEzDST off=1936160 len=1333583 2024-04-07T16:54:08.595 INFO:tasks.workunit.client.0.smithi100.stdout:op 4677 completed, throughput=5MB/sec 2024-04-07T16:54:08.596 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-HIqpkV0hSZ5ahiT off=10909531 len=1044746 2024-04-07T16:54:08.599 INFO:tasks.workunit.client.0.smithi100.stdout:op 4678 completed, throughput=5MB/sec 2024-04-07T16:54:08.599 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-T4Hnp2vlK2DSF7w off=2337689 len=377761 2024-04-07T16:54:08.601 INFO:tasks.workunit.client.0.smithi100.stdout:op 4679 completed, throughput=5MB/sec 2024-04-07T16:54:08.601 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-ax7ylhHjEPof7_1 off=10142389 len=1062464 2024-04-07T16:54:08.605 INFO:tasks.workunit.client.0.smithi100.stdout:op 4680 completed, throughput=5MB/sec 2024-04-07T16:54:08.699 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:54:08 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:54:08] "GET /metrics HTTP/1.1" 200 34072 "" "Prometheus/2.43.0" 2024-04-07T16:54:08.699 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:54:08 smithi100 bash[74942]: cluster 2024-04-07T16:54:07.546004+0000 mgr.y (mgr.25175) 1163 : cluster [DBG] pgmap v1805: 105 pgs: 105 active+clean; 126 GiB data, 9.6 GiB used, 706 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.4 MiB/s wr, 5 op/s 2024-04-07T16:54:08.699 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:54:08 smithi100 bash[74942]: audit 2024-04-07T16:54:08.105950+0000 mon.a (mon.0) 2178 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:54:08.699 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:54:08 smithi100 bash[74942]: audit 2024-04-07T16:54:08.107405+0000 mon.a (mon.0) 2179 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:54:08.699 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:54:08 smithi100 bash[74942]: audit 2024-04-07T16:54:08.116220+0000 mon.a (mon.0) 2180 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:54:08.699 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:54:08 smithi100 bash[76202]: cluster 2024-04-07T16:54:07.546004+0000 mgr.y (mgr.25175) 1163 : cluster [DBG] pgmap v1805: 105 pgs: 105 active+clean; 126 GiB data, 9.6 GiB used, 706 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.4 MiB/s wr, 5 op/s 2024-04-07T16:54:08.700 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:54:08 smithi100 bash[76202]: audit 2024-04-07T16:54:08.105950+0000 mon.a (mon.0) 2178 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:54:08.700 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:54:08 smithi100 bash[76202]: audit 2024-04-07T16:54:08.107405+0000 mon.a (mon.0) 2179 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:54:08.700 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:54:08 smithi100 bash[76202]: audit 2024-04-07T16:54:08.116220+0000 mon.a (mon.0) 2180 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:54:09.112 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:54:08 smithi144 bash[66654]: cluster 2024-04-07T16:54:07.546004+0000 mgr.y (mgr.25175) 1163 : cluster [DBG] pgmap v1805: 105 pgs: 105 active+clean; 126 GiB data, 9.6 GiB used, 706 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.4 MiB/s wr, 5 op/s 2024-04-07T16:54:09.112 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:54:08 smithi144 bash[66654]: audit 2024-04-07T16:54:08.105950+0000 mon.a (mon.0) 2178 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:54:09.112 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:54:08 smithi144 bash[66654]: audit 2024-04-07T16:54:08.107405+0000 mon.a (mon.0) 2179 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:54:09.112 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:54:08 smithi144 bash[66654]: audit 2024-04-07T16:54:08.116220+0000 mon.a (mon.0) 2180 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:54:09.605 INFO:tasks.workunit.client.0.smithi100.stdout: 903: throughput=4.99MB/sec pending data=0 2024-04-07T16:54:09.605 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-yS_S2eCmOerQc2s off=7649417 len=1745967 2024-04-07T16:54:09.621 INFO:tasks.workunit.client.0.smithi100.stdout:op 4681 completed, throughput=5MB/sec 2024-04-07T16:54:09.621 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Dybo13JXpdIHhU- off=10572817 len=699613 2024-04-07T16:54:09.623 INFO:tasks.workunit.client.0.smithi100.stdout:op 4682 completed, throughput=5MB/sec 2024-04-07T16:54:09.623 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-MGYdSlVRiksjWfD off=1858984 len=1431130 2024-04-07T16:54:09.636 INFO:tasks.workunit.client.0.smithi100.stdout:op 4683 completed, throughput=5MB/sec 2024-04-07T16:54:09.636 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-_ZVOckoik4Nn66O off=12767891 len=976261 2024-04-07T16:54:09.646 INFO:tasks.workunit.client.0.smithi100.stdout:op 4684 completed, throughput=5MB/sec 2024-04-07T16:54:09.646 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-rDDR8q6aj01mF2S off=17770317 len=1741415 2024-04-07T16:54:09.659 INFO:tasks.workunit.client.0.smithi100.stdout:op 4685 completed, throughput=5MB/sec 2024-04-07T16:54:10.660 INFO:tasks.workunit.client.0.smithi100.stdout: 904: throughput=5MB/sec pending data=0 2024-04-07T16:54:10.660 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-TKjJFXyZq1rbvgg off=271071 len=1783738 2024-04-07T16:54:10.676 INFO:tasks.workunit.client.0.smithi100.stdout:op 4686 completed, throughput=5MB/sec 2024-04-07T16:54:10.676 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-_YWOfHATk4_fLVD off=8119047 len=403254 2024-04-07T16:54:10.677 INFO:tasks.workunit.client.0.smithi100.stdout:op 4687 completed, throughput=5MB/sec 2024-04-07T16:54:10.677 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-vOELolSaALsHvCf off=5996280 len=1370282 2024-04-07T16:54:10.681 INFO:tasks.workunit.client.0.smithi100.stdout:op 4688 completed, throughput=5MB/sec 2024-04-07T16:54:10.681 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-6bakz5c8aBh_n9L off=6127152 len=1695611 2024-04-07T16:54:10.686 INFO:tasks.workunit.client.0.smithi100.stdout:op 4689 completed, throughput=5MB/sec 2024-04-07T16:54:10.949 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:54:10 smithi100 bash[74942]: cluster 2024-04-07T16:54:09.546914+0000 mgr.y (mgr.25175) 1164 : cluster [DBG] pgmap v1806: 105 pgs: 105 active+clean; 126 GiB data, 9.6 GiB used, 706 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.4 MiB/s wr, 5 op/s 2024-04-07T16:54:10.949 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:54:10 smithi100 bash[76202]: cluster 2024-04-07T16:54:09.546914+0000 mgr.y (mgr.25175) 1164 : cluster [DBG] pgmap v1806: 105 pgs: 105 active+clean; 126 GiB data, 9.6 GiB used, 706 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.4 MiB/s wr, 5 op/s 2024-04-07T16:54:11.112 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:54:10 smithi144 bash[66654]: cluster 2024-04-07T16:54:09.546914+0000 mgr.y (mgr.25175) 1164 : cluster [DBG] pgmap v1806: 105 pgs: 105 active+clean; 126 GiB data, 9.6 GiB used, 706 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.4 MiB/s wr, 5 op/s 2024-04-07T16:54:11.686 INFO:tasks.workunit.client.0.smithi100.stdout: 905: throughput=5MB/sec pending data=0 2024-04-07T16:54:11.686 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-RF5bp2ZxxfL9G7Z off=24295878 len=229363 2024-04-07T16:54:11.692 INFO:tasks.workunit.client.0.smithi100.stdout:op 4690 completed, throughput=5MB/sec 2024-04-07T16:54:11.692 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-lsuKMG0OHxhm84Y off=9238544 len=1775929 2024-04-07T16:54:11.705 INFO:tasks.workunit.client.0.smithi100.stdout:op 4691 completed, throughput=5MB/sec 2024-04-07T16:54:11.705 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-INDmOGLCTTab0FU off=15299988 len=206860 2024-04-07T16:54:11.710 INFO:tasks.workunit.client.0.smithi100.stdout:op 4692 completed, throughput=5MB/sec 2024-04-07T16:54:11.710 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-oXqsuslwl6c_GXV off=5047545 len=246242 2024-04-07T16:54:11.717 INFO:tasks.workunit.client.0.smithi100.stdout:op 4693 completed, throughput=5MB/sec 2024-04-07T16:54:11.717 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-UGRxcSHe-0kCr_v off=14013670 len=432256 2024-04-07T16:54:11.723 INFO:tasks.workunit.client.0.smithi100.stdout:op 4694 completed, throughput=5MB/sec 2024-04-07T16:54:11.723 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-11Ho5NQDYwrYLuN off=6838541 len=2049081 2024-04-07T16:54:11.739 INFO:tasks.workunit.client.0.smithi100.stdout:op 4695 completed, throughput=5MB/sec 2024-04-07T16:54:12.612 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:54:12 smithi144 bash[95233]: ts=2024-04-07T16:54:12.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:54:12.740 INFO:tasks.workunit.client.0.smithi100.stdout: 906: throughput=5MB/sec pending data=0 2024-04-07T16:54:12.740 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-xBFvOP4icfsHfvE off=2145675 len=414706 2024-04-07T16:54:12.746 INFO:tasks.workunit.client.0.smithi100.stdout:op 4696 completed, throughput=5MB/sec 2024-04-07T16:54:12.746 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-FKUt4Qg-ksn5mzo off=0 len=1455362 2024-04-07T16:54:12.758 INFO:tasks.workunit.client.0.smithi100.stdout:op 4697 completed, throughput=5MB/sec 2024-04-07T16:54:12.758 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-c65XicjtU0lv9Xw off=16751569 len=1341641 2024-04-07T16:54:12.768 INFO:tasks.workunit.client.0.smithi100.stdout:op 4698 completed, throughput=5MB/sec 2024-04-07T16:54:12.769 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-rvQ84kEh-xT63mW off=18478954 len=1581710 2024-04-07T16:54:12.782 INFO:tasks.workunit.client.0.smithi100.stdout:op 4699 completed, throughput=5MB/sec 2024-04-07T16:54:12.782 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-1FE6D7WXEz94Qd3 off=586405 len=2075245 2024-04-07T16:54:12.796 INFO:tasks.workunit.client.0.smithi100.stdout:op 4700 completed, throughput=5MB/sec 2024-04-07T16:54:12.949 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:54:12 smithi100 bash[74942]: cluster 2024-04-07T16:54:11.547749+0000 mgr.y (mgr.25175) 1165 : cluster [DBG] pgmap v1807: 105 pgs: 105 active+clean; 126 GiB data, 9.6 GiB used, 706 GiB / 715 GiB avail; 2.7 MiB/s rd, 3.0 MiB/s wr, 5 op/s 2024-04-07T16:54:12.949 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:54:12 smithi100 bash[76202]: cluster 2024-04-07T16:54:11.547749+0000 mgr.y (mgr.25175) 1165 : cluster [DBG] pgmap v1807: 105 pgs: 105 active+clean; 126 GiB data, 9.6 GiB used, 706 GiB / 715 GiB avail; 2.7 MiB/s rd, 3.0 MiB/s wr, 5 op/s 2024-04-07T16:54:13.112 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:54:12 smithi144 bash[66654]: cluster 2024-04-07T16:54:11.547749+0000 mgr.y (mgr.25175) 1165 : cluster [DBG] pgmap v1807: 105 pgs: 105 active+clean; 126 GiB data, 9.6 GiB used, 706 GiB / 715 GiB avail; 2.7 MiB/s rd, 3.0 MiB/s wr, 5 op/s 2024-04-07T16:54:13.796 INFO:tasks.workunit.client.0.smithi100.stdout: 907: throughput=5MB/sec pending data=0 2024-04-07T16:54:13.797 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-yiaLbL0rNJ4v-se off=463344 len=705541 2024-04-07T16:54:13.799 INFO:tasks.workunit.client.0.smithi100.stdout:op 4701 completed, throughput=5MB/sec 2024-04-07T16:54:13.799 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Vb7Isk3TJPTb9ri off=0 len=1388634 2024-04-07T16:54:13.810 INFO:tasks.workunit.client.0.smithi100.stdout:op 4702 completed, throughput=5MB/sec 2024-04-07T16:54:13.810 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-MhmXWRyvSVMwYn3 off=1084046 len=616317 2024-04-07T16:54:13.812 INFO:tasks.workunit.client.0.smithi100.stdout:op 4703 completed, throughput=5MB/sec 2024-04-07T16:54:13.812 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-UNGDFARDjx18MMP off=5916552 len=1261722 2024-04-07T16:54:13.824 INFO:tasks.workunit.client.0.smithi100.stdout:op 4704 completed, throughput=5MB/sec 2024-04-07T16:54:14.824 INFO:tasks.workunit.client.0.smithi100.stdout: 908: throughput=5MB/sec pending data=0 2024-04-07T16:54:14.824 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-hb71lqlnTw2ihrH off=1143500 len=141905 2024-04-07T16:54:14.825 INFO:tasks.workunit.client.0.smithi100.stdout:op 4705 completed, throughput=5MB/sec 2024-04-07T16:54:14.825 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-tiakL33_kV3bLlk off=5952931 len=635476 2024-04-07T16:54:14.828 INFO:tasks.workunit.client.0.smithi100.stdout:op 4706 completed, throughput=5MB/sec 2024-04-07T16:54:14.828 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-csOtPSYTZvZ3z0C off=23385144 len=1009474 2024-04-07T16:54:14.838 INFO:tasks.workunit.client.0.smithi100.stdout:op 4707 completed, throughput=5MB/sec 2024-04-07T16:54:14.838 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-vBWRrrIwLpbCGQx off=11439812 len=106779 2024-04-07T16:54:14.839 INFO:tasks.workunit.client.0.smithi100.stdout:op 4708 completed, throughput=5MB/sec 2024-04-07T16:54:14.839 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-cHhiIgzu-byIEke off=11434490 len=1657933 2024-04-07T16:54:14.844 INFO:tasks.workunit.client.0.smithi100.stdout:op 4709 completed, throughput=5MB/sec 2024-04-07T16:54:14.845 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-10XaXwyOEyPyOt- off=2479532 len=257087 2024-04-07T16:54:14.846 INFO:tasks.workunit.client.0.smithi100.stdout:op 4710 completed, throughput=5MB/sec 2024-04-07T16:54:14.846 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-PZ9kMzdQx4MPfsq off=6426077 len=1101363 2024-04-07T16:54:14.855 INFO:tasks.workunit.client.0.smithi100.stdout:op 4711 completed, throughput=5MB/sec 2024-04-07T16:54:14.948 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:54:14 smithi100 bash[74942]: cluster 2024-04-07T16:54:13.548807+0000 mgr.y (mgr.25175) 1166 : cluster [DBG] pgmap v1808: 105 pgs: 105 active+clean; 126 GiB data, 9.7 GiB used, 706 GiB / 715 GiB avail; 3.0 MiB/s rd, 3.1 MiB/s wr, 6 op/s 2024-04-07T16:54:14.949 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:54:14 smithi100 bash[76202]: cluster 2024-04-07T16:54:13.548807+0000 mgr.y (mgr.25175) 1166 : cluster [DBG] pgmap v1808: 105 pgs: 105 active+clean; 126 GiB data, 9.7 GiB used, 706 GiB / 715 GiB avail; 3.0 MiB/s rd, 3.1 MiB/s wr, 6 op/s 2024-04-07T16:54:15.112 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:54:14 smithi144 bash[66654]: cluster 2024-04-07T16:54:13.548807+0000 mgr.y (mgr.25175) 1166 : cluster [DBG] pgmap v1808: 105 pgs: 105 active+clean; 126 GiB data, 9.7 GiB used, 706 GiB / 715 GiB avail; 3.0 MiB/s rd, 3.1 MiB/s wr, 6 op/s 2024-04-07T16:54:15.856 INFO:tasks.workunit.client.0.smithi100.stdout: 909: throughput=4.99MB/sec pending data=0 2024-04-07T16:54:15.856 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-AYrgUqfjZEUoEJe off=2643009 len=1000532 2024-04-07T16:54:15.865 INFO:tasks.workunit.client.0.smithi100.stdout:op 4712 completed, throughput=5MB/sec 2024-04-07T16:54:15.865 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-_K3JO4IecbuoTq3 off=5711873 len=1148399 2024-04-07T16:54:15.868 INFO:tasks.workunit.client.0.smithi100.stdout:op 4713 completed, throughput=5MB/sec 2024-04-07T16:54:15.868 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-IeaRK8lnOuB8sy5 off=11024879 len=808312 2024-04-07T16:54:15.876 INFO:tasks.workunit.client.0.smithi100.stdout:op 4714 completed, throughput=5MB/sec 2024-04-07T16:54:15.876 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-KWQw_M7Soj8MXY_ off=1041198 len=30043 2024-04-07T16:54:15.877 INFO:tasks.workunit.client.0.smithi100.stdout:op 4715 completed, throughput=5MB/sec 2024-04-07T16:54:15.877 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj--sB1bEQZVCN2Rou off=941810 len=1538799 2024-04-07T16:54:15.890 INFO:tasks.workunit.client.0.smithi100.stdout:op 4716 completed, throughput=5MB/sec 2024-04-07T16:54:15.890 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-OQD8EE8HS8Qmm8h off=18578005 len=1781095 2024-04-07T16:54:15.893 INFO:tasks.workunit.client.0.smithi100.stdout:op 4717 completed, throughput=5MB/sec 2024-04-07T16:54:16.893 INFO:tasks.workunit.client.0.smithi100.stdout: 910: throughput=5MB/sec pending data=0 2024-04-07T16:54:16.893 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-DIiyNwBJW0tpO6i off=8612289 len=1334456 2024-04-07T16:54:16.905 INFO:tasks.workunit.client.0.smithi100.stdout:op 4718 completed, throughput=5MB/sec 2024-04-07T16:54:16.905 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-3pLTQMK_SgSlEil off=862343 len=416302 2024-04-07T16:54:16.907 INFO:tasks.workunit.client.0.smithi100.stdout:op 4719 completed, throughput=5MB/sec 2024-04-07T16:54:16.907 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-01RapDuhywAPerW off=9091492 len=780454 2024-04-07T16:54:16.916 INFO:tasks.workunit.client.0.smithi100.stdout:op 4720 completed, throughput=5MB/sec 2024-04-07T16:54:16.916 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-hbNUtsbIvnGO8hm off=1958649 len=1730992 2024-04-07T16:54:16.920 INFO:tasks.workunit.client.0.smithi100.stdout:op 4721 completed, throughput=5MB/sec 2024-04-07T16:54:16.920 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-7U2b2z2kznvwWyt off=10233161 len=783809 2024-04-07T16:54:16.922 INFO:tasks.workunit.client.0.smithi100.stdout:op 4722 completed, throughput=5MB/sec 2024-04-07T16:54:16.948 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:54:16 smithi100 bash[74942]: cluster 2024-04-07T16:54:15.549817+0000 mgr.y (mgr.25175) 1167 : cluster [DBG] pgmap v1809: 105 pgs: 105 active+clean; 126 GiB data, 9.7 GiB used, 706 GiB / 715 GiB avail; 2.1 MiB/s rd, 3.0 MiB/s wr, 5 op/s 2024-04-07T16:54:16.949 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:54:16 smithi100 bash[76202]: cluster 2024-04-07T16:54:15.549817+0000 mgr.y (mgr.25175) 1167 : cluster [DBG] pgmap v1809: 105 pgs: 105 active+clean; 126 GiB data, 9.7 GiB used, 706 GiB / 715 GiB avail; 2.1 MiB/s rd, 3.0 MiB/s wr, 5 op/s 2024-04-07T16:54:17.112 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:54:16 smithi144 bash[66654]: cluster 2024-04-07T16:54:15.549817+0000 mgr.y (mgr.25175) 1167 : cluster [DBG] pgmap v1809: 105 pgs: 105 active+clean; 126 GiB data, 9.7 GiB used, 706 GiB / 715 GiB avail; 2.1 MiB/s rd, 3.0 MiB/s wr, 5 op/s 2024-04-07T16:54:17.923 INFO:tasks.workunit.client.0.smithi100.stdout: 911: throughput=5MB/sec pending data=0 2024-04-07T16:54:17.923 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-GXyRQqQCdku5XAH off=2613195 len=1999554 2024-04-07T16:54:17.941 INFO:tasks.workunit.client.0.smithi100.stdout:op 4723 completed, throughput=5MB/sec 2024-04-07T16:54:17.941 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-shudATNpCAhLZsg off=5271853 len=685086 2024-04-07T16:54:17.948 INFO:tasks.workunit.client.0.smithi100.stdout:op 4724 completed, throughput=5MB/sec 2024-04-07T16:54:17.948 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-VfQNZTj6ZZvhL4l off=2673432 len=1460333 2024-04-07T16:54:17.949 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:54:17 smithi100 bash[74942]: audit 2024-04-07T16:54:16.782971+0000 mon.a (mon.0) 2181 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:54:17.949 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:54:17 smithi100 bash[76202]: audit 2024-04-07T16:54:16.782971+0000 mon.a (mon.0) 2181 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:54:17.959 INFO:tasks.workunit.client.0.smithi100.stdout:op 4725 completed, throughput=5MB/sec 2024-04-07T16:54:17.959 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-USgYfYZFCBIj6iH off=18089054 len=551521 2024-04-07T16:54:17.961 INFO:tasks.workunit.client.0.smithi100.stdout:op 4726 completed, throughput=5MB/sec 2024-04-07T16:54:17.961 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-qh5UwTIXljXhQJ5 off=8845727 len=1090761 2024-04-07T16:54:17.969 INFO:tasks.workunit.client.0.smithi100.stdout:op 4727 completed, throughput=5MB/sec 2024-04-07T16:54:18.112 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:54:17 smithi144 bash[66654]: audit 2024-04-07T16:54:16.782971+0000 mon.a (mon.0) 2181 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:54:18.698 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:54:18 smithi100 bash[74942]: cluster 2024-04-07T16:54:17.550828+0000 mgr.y (mgr.25175) 1168 : cluster [DBG] pgmap v1810: 105 pgs: 105 active+clean; 126 GiB data, 9.7 GiB used, 706 GiB / 715 GiB avail; 2.1 MiB/s rd, 2.9 MiB/s wr, 5 op/s 2024-04-07T16:54:18.699 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:54:18 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:54:18] "GET /metrics HTTP/1.1" 200 34071 "" "Prometheus/2.43.0" 2024-04-07T16:54:18.699 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:54:18 smithi100 bash[76202]: cluster 2024-04-07T16:54:17.550828+0000 mgr.y (mgr.25175) 1168 : cluster [DBG] pgmap v1810: 105 pgs: 105 active+clean; 126 GiB data, 9.7 GiB used, 706 GiB / 715 GiB avail; 2.1 MiB/s rd, 2.9 MiB/s wr, 5 op/s 2024-04-07T16:54:18.969 INFO:tasks.workunit.client.0.smithi100.stdout: 912: throughput=5MB/sec pending data=0 2024-04-07T16:54:18.970 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-uBhkeAoL4M-mrRM off=428610 len=1706505 2024-04-07T16:54:18.974 INFO:tasks.workunit.client.0.smithi100.stdout:op 4728 completed, throughput=5MB/sec 2024-04-07T16:54:18.974 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-sDxHISCqkR_shyW off=9158144 len=330312 2024-04-07T16:54:18.981 INFO:tasks.workunit.client.0.smithi100.stdout:op 4729 completed, throughput=5MB/sec 2024-04-07T16:54:18.981 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-4QWeqHc06fsdF0j off=12743741 len=1881551 2024-04-07T16:54:18.986 INFO:tasks.workunit.client.0.smithi100.stdout:op 4730 completed, throughput=5MB/sec 2024-04-07T16:54:18.986 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-SBYHsTg-qEMs3Mj off=11855439 len=541802 2024-04-07T16:54:18.993 INFO:tasks.workunit.client.0.smithi100.stdout:op 4731 completed, throughput=5MB/sec 2024-04-07T16:54:18.993 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-mfMDPNtMBHBKydY off=9906694 len=1530798 2024-04-07T16:54:19.006 INFO:tasks.workunit.client.0.smithi100.stdout:op 4732 completed, throughput=5MB/sec 2024-04-07T16:54:19.112 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:54:18 smithi144 bash[66654]: cluster 2024-04-07T16:54:17.550828+0000 mgr.y (mgr.25175) 1168 : cluster [DBG] pgmap v1810: 105 pgs: 105 active+clean; 126 GiB data, 9.7 GiB used, 706 GiB / 715 GiB avail; 2.1 MiB/s rd, 2.9 MiB/s wr, 5 op/s 2024-04-07T16:54:20.006 INFO:tasks.workunit.client.0.smithi100.stdout: 913: throughput=5MB/sec pending data=0 2024-04-07T16:54:20.006 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-cE4LuZi5SmytuvT off=2199585 len=291028 2024-04-07T16:54:20.007 INFO:tasks.workunit.client.0.smithi100.stdout:op 4733 completed, throughput=5MB/sec 2024-04-07T16:54:20.007 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj--zhYfx7gxx8rDIo off=14599243 len=1507817 2024-04-07T16:54:20.011 INFO:tasks.workunit.client.0.smithi100.stdout:op 4734 completed, throughput=5MB/sec 2024-04-07T16:54:20.011 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-MLyX78p9wn-S294 off=8733420 len=361947 2024-04-07T16:54:20.017 INFO:tasks.workunit.client.0.smithi100.stdout:op 4735 completed, throughput=5MB/sec 2024-04-07T16:54:20.017 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-YCYqeVwlqRooZQG off=12433158 len=590553 2024-04-07T16:54:20.019 INFO:tasks.workunit.client.0.smithi100.stdout:op 4736 completed, throughput=5MB/sec 2024-04-07T16:54:20.019 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-PxVIEcJQxSDkYg6 off=3665404 len=445411 2024-04-07T16:54:20.026 INFO:tasks.workunit.client.0.smithi100.stdout:op 4737 completed, throughput=5MB/sec 2024-04-07T16:54:20.027 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-eMkNsVh9JunalKg off=5133553 len=1529549 2024-04-07T16:54:20.038 INFO:tasks.workunit.client.0.smithi100.stdout:op 4738 completed, throughput=5MB/sec 2024-04-07T16:54:20.949 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:54:20 smithi100 bash[74942]: cluster 2024-04-07T16:54:19.551798+0000 mgr.y (mgr.25175) 1169 : cluster [DBG] pgmap v1811: 105 pgs: 105 active+clean; 126 GiB data, 9.7 GiB used, 706 GiB / 715 GiB avail; 1.8 MiB/s rd, 2.7 MiB/s wr, 4 op/s 2024-04-07T16:54:20.949 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:54:20 smithi100 bash[76202]: cluster 2024-04-07T16:54:19.551798+0000 mgr.y (mgr.25175) 1169 : cluster [DBG] pgmap v1811: 105 pgs: 105 active+clean; 126 GiB data, 9.7 GiB used, 706 GiB / 715 GiB avail; 1.8 MiB/s rd, 2.7 MiB/s wr, 4 op/s 2024-04-07T16:54:21.038 INFO:tasks.workunit.client.0.smithi100.stdout: 914: throughput=5MB/sec pending data=0 2024-04-07T16:54:21.038 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-EQX-ra5wpmMr21m off=5968179 len=137063 2024-04-07T16:54:21.044 INFO:tasks.workunit.client.0.smithi100.stdout:op 4739 completed, throughput=5MB/sec 2024-04-07T16:54:21.044 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-5VcWXbDpY8fjXz7 off=2339742 len=2087707 2024-04-07T16:54:21.061 INFO:tasks.workunit.client.0.smithi100.stdout:op 4740 completed, throughput=5MB/sec 2024-04-07T16:54:21.062 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-eLKldFU5WlpnDIm off=21909466 len=1152711 2024-04-07T16:54:21.071 INFO:tasks.workunit.client.0.smithi100.stdout:op 4741 completed, throughput=5MB/sec 2024-04-07T16:54:21.071 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-0puY_DtQ1W2sMvL off=1267701 len=1339903 2024-04-07T16:54:21.085 INFO:tasks.workunit.client.0.smithi100.stdout:op 4742 completed, throughput=5MB/sec 2024-04-07T16:54:21.085 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-I0Wgar4MBcJtlgW off=687292 len=857918 2024-04-07T16:54:21.093 INFO:tasks.workunit.client.0.smithi100.stdout:op 4743 completed, throughput=5MB/sec 2024-04-07T16:54:21.112 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:54:20 smithi144 bash[66654]: cluster 2024-04-07T16:54:19.551798+0000 mgr.y (mgr.25175) 1169 : cluster [DBG] pgmap v1811: 105 pgs: 105 active+clean; 126 GiB data, 9.7 GiB used, 706 GiB / 715 GiB avail; 1.8 MiB/s rd, 2.7 MiB/s wr, 4 op/s 2024-04-07T16:54:22.093 INFO:tasks.workunit.client.0.smithi100.stdout: 915: throughput=5MB/sec pending data=0 2024-04-07T16:54:22.093 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-UWQrYnj3MO2Ztxz off=10779990 len=1497973 2024-04-07T16:54:22.105 INFO:tasks.workunit.client.0.smithi100.stdout:op 4744 completed, throughput=5MB/sec 2024-04-07T16:54:22.105 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-xXhOSgLX2JpANFP off=13857082 len=1810415 2024-04-07T16:54:22.107 INFO:tasks.workunit.client.0.smithi100.stdout:op 4745 completed, throughput=5MB/sec 2024-04-07T16:54:22.107 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-vOsaP-sxEoHQy0W off=353237 len=773404 2024-04-07T16:54:22.116 INFO:tasks.workunit.client.0.smithi100.stdout:op 4746 completed, throughput=5MB/sec 2024-04-07T16:54:22.116 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-q1NLy1fq4_42fTZ off=20788900 len=352395 2024-04-07T16:54:22.118 INFO:tasks.workunit.client.0.smithi100.stdout:op 4747 completed, throughput=5MB/sec 2024-04-07T16:54:22.118 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Eet0qJm5lRS0r_O off=3192399 len=1401554 2024-04-07T16:54:22.129 INFO:tasks.workunit.client.0.smithi100.stdout:op 4748 completed, throughput=5MB/sec 2024-04-07T16:54:22.612 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:54:22 smithi144 bash[95233]: ts=2024-04-07T16:54:22.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:54:22.949 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:54:22 smithi100 bash[74942]: cluster 2024-04-07T16:54:21.552568+0000 mgr.y (mgr.25175) 1170 : cluster [DBG] pgmap v1812: 105 pgs: 105 active+clean; 126 GiB data, 9.7 GiB used, 706 GiB / 715 GiB avail; 1.6 MiB/s rd, 2.8 MiB/s wr, 4 op/s 2024-04-07T16:54:22.949 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:54:22 smithi100 bash[76202]: cluster 2024-04-07T16:54:21.552568+0000 mgr.y (mgr.25175) 1170 : cluster [DBG] pgmap v1812: 105 pgs: 105 active+clean; 126 GiB data, 9.7 GiB used, 706 GiB / 715 GiB avail; 1.6 MiB/s rd, 2.8 MiB/s wr, 4 op/s 2024-04-07T16:54:23.112 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:54:22 smithi144 bash[66654]: cluster 2024-04-07T16:54:21.552568+0000 mgr.y (mgr.25175) 1170 : cluster [DBG] pgmap v1812: 105 pgs: 105 active+clean; 126 GiB data, 9.7 GiB used, 706 GiB / 715 GiB avail; 1.6 MiB/s rd, 2.8 MiB/s wr, 4 op/s 2024-04-07T16:54:23.129 INFO:tasks.workunit.client.0.smithi100.stdout: 916: throughput=5MB/sec pending data=0 2024-04-07T16:54:23.129 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-CECWZkWN-iM9wsb off=950195 len=1412899 2024-04-07T16:54:23.143 INFO:tasks.workunit.client.0.smithi100.stdout:op 4749 completed, throughput=5MB/sec 2024-04-07T16:54:23.143 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-A0YzvLn-JUbLTu1 off=1142275 len=796495 2024-04-07T16:54:23.145 INFO:tasks.workunit.client.0.smithi100.stdout:op 4750 completed, throughput=5MB/sec 2024-04-07T16:54:23.145 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-odZW9y0TaC8aflp off=10454147 len=1333240 2024-04-07T16:54:23.156 INFO:tasks.workunit.client.0.smithi100.stdout:op 4751 completed, throughput=5MB/sec 2024-04-07T16:54:23.156 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-c-KbcQmm6s7Wi1b off=5596588 len=871259 2024-04-07T16:54:23.159 INFO:tasks.workunit.client.0.smithi100.stdout:op 4752 completed, throughput=5MB/sec 2024-04-07T16:54:24.159 INFO:tasks.workunit.client.0.smithi100.stdout: 917: throughput=4.99MB/sec pending data=0 2024-04-07T16:54:24.160 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-TJ2oTRMxCr_J9RY off=8718141 len=1608097 2024-04-07T16:54:24.166 INFO:tasks.workunit.client.0.smithi100.stdout:op 4753 completed, throughput=5MB/sec 2024-04-07T16:54:24.166 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-cWOpXsJsMBHLQcr off=8688646 len=561078 2024-04-07T16:54:24.167 INFO:tasks.workunit.client.0.smithi100.stdout:op 4754 completed, throughput=5MB/sec 2024-04-07T16:54:24.167 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-o5pvSrMoXs4gWvD off=5178324 len=643040 2024-04-07T16:54:24.170 INFO:tasks.workunit.client.0.smithi100.stdout:op 4755 completed, throughput=5MB/sec 2024-04-07T16:54:24.170 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-z6XmDGGqj4SskyX off=5420647 len=1848473 2024-04-07T16:54:24.186 INFO:tasks.workunit.client.0.smithi100.stdout:op 4756 completed, throughput=5MB/sec 2024-04-07T16:54:24.187 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Tzu9_vz_yIBUWDS off=11639293 len=1429945 2024-04-07T16:54:24.190 INFO:tasks.workunit.client.0.smithi100.stdout:op 4757 completed, throughput=5MB/sec 2024-04-07T16:54:24.949 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:54:24 smithi100 bash[74942]: cluster 2024-04-07T16:54:23.553797+0000 mgr.y (mgr.25175) 1171 : cluster [DBG] pgmap v1813: 105 pgs: 105 active+clean; 126 GiB data, 9.7 GiB used, 705 GiB / 715 GiB avail; 1.9 MiB/s rd, 3.2 MiB/s wr, 5 op/s 2024-04-07T16:54:24.949 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:54:24 smithi100 bash[76202]: cluster 2024-04-07T16:54:23.553797+0000 mgr.y (mgr.25175) 1171 : cluster [DBG] pgmap v1813: 105 pgs: 105 active+clean; 126 GiB data, 9.7 GiB used, 705 GiB / 715 GiB avail; 1.9 MiB/s rd, 3.2 MiB/s wr, 5 op/s 2024-04-07T16:54:25.112 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:54:24 smithi144 bash[66654]: cluster 2024-04-07T16:54:23.553797+0000 mgr.y (mgr.25175) 1171 : cluster [DBG] pgmap v1813: 105 pgs: 105 active+clean; 126 GiB data, 9.7 GiB used, 705 GiB / 715 GiB avail; 1.9 MiB/s rd, 3.2 MiB/s wr, 5 op/s 2024-04-07T16:54:25.190 INFO:tasks.workunit.client.0.smithi100.stdout: 918: throughput=5MB/sec pending data=0 2024-04-07T16:54:25.191 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj--JFUUnIjDUXkY40 off=5168369 len=732651 2024-04-07T16:54:25.198 INFO:tasks.workunit.client.0.smithi100.stdout:op 4758 completed, throughput=5MB/sec 2024-04-07T16:54:25.199 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-i5ULsv6kuF2wb4V off=3650493 len=707573 2024-04-07T16:54:25.206 INFO:tasks.workunit.client.0.smithi100.stdout:op 4759 completed, throughput=5MB/sec 2024-04-07T16:54:25.206 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-vFmqymJOM4XsCv9 off=1239862 len=306918 2024-04-07T16:54:25.207 INFO:tasks.workunit.client.0.smithi100.stdout:op 4760 completed, throughput=5MB/sec 2024-04-07T16:54:25.207 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-IyuCxmQ_UjUhcbM off=21403616 len=525840 2024-04-07T16:54:25.211 INFO:tasks.workunit.client.0.smithi100.stdout:op 4761 completed, throughput=5MB/sec 2024-04-07T16:54:25.211 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Z7JHyG_oZv32qM1 off=9751018 len=1608718 2024-04-07T16:54:25.223 INFO:tasks.workunit.client.0.smithi100.stdout:op 4762 completed, throughput=5MB/sec 2024-04-07T16:54:25.224 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-9FVMFp_n-fGhpko off=5887990 len=912090 2024-04-07T16:54:25.226 INFO:tasks.workunit.client.0.smithi100.stdout:op 4763 completed, throughput=5MB/sec 2024-04-07T16:54:26.227 INFO:tasks.workunit.client.0.smithi100.stdout: 919: throughput=4.99MB/sec pending data=0 2024-04-07T16:54:26.227 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-6E1WvoAGQ71p2BX off=2191932 len=1478415 2024-04-07T16:54:26.242 INFO:tasks.workunit.client.0.smithi100.stdout:op 4764 completed, throughput=5MB/sec 2024-04-07T16:54:26.242 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-0q1JTczTYYUDpNi off=6414158 len=1293141 2024-04-07T16:54:26.245 INFO:tasks.workunit.client.0.smithi100.stdout:op 4765 completed, throughput=5MB/sec 2024-04-07T16:54:26.245 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-soiNeMXm9F5vFNp off=15147164 len=1610134 2024-04-07T16:54:26.259 INFO:tasks.workunit.client.0.smithi100.stdout:op 4766 completed, throughput=5MB/sec 2024-04-07T16:54:26.260 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-A3N7mOsHll0iIKD off=1172500 len=2045870 2024-04-07T16:54:26.264 INFO:tasks.workunit.client.0.smithi100.stdout:op 4767 completed, throughput=5MB/sec 2024-04-07T16:54:26.949 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:54:26 smithi100 bash[74942]: cluster 2024-04-07T16:54:25.554933+0000 mgr.y (mgr.25175) 1172 : cluster [DBG] pgmap v1814: 105 pgs: 105 active+clean; 126 GiB data, 9.8 GiB used, 705 GiB / 715 GiB avail; 1.6 MiB/s rd, 2.9 MiB/s wr, 4 op/s 2024-04-07T16:54:26.949 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:54:26 smithi100 bash[76202]: cluster 2024-04-07T16:54:25.554933+0000 mgr.y (mgr.25175) 1172 : cluster [DBG] pgmap v1814: 105 pgs: 105 active+clean; 126 GiB data, 9.8 GiB used, 705 GiB / 715 GiB avail; 1.6 MiB/s rd, 2.9 MiB/s wr, 4 op/s 2024-04-07T16:54:27.112 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:54:26 smithi144 bash[66654]: cluster 2024-04-07T16:54:25.554933+0000 mgr.y (mgr.25175) 1172 : cluster [DBG] pgmap v1814: 105 pgs: 105 active+clean; 126 GiB data, 9.8 GiB used, 705 GiB / 715 GiB avail; 1.6 MiB/s rd, 2.9 MiB/s wr, 4 op/s 2024-04-07T16:54:27.264 INFO:tasks.workunit.client.0.smithi100.stdout: 920: throughput=5MB/sec pending data=0 2024-04-07T16:54:27.264 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-zpKaojNVmBnUMCS off=6919577 len=1166813 2024-04-07T16:54:27.275 INFO:tasks.workunit.client.0.smithi100.stdout:op 4768 completed, throughput=5MB/sec 2024-04-07T16:54:27.275 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-ycueEra0J6Bh4WG off=8898006 len=594312 2024-04-07T16:54:27.277 INFO:tasks.workunit.client.0.smithi100.stdout:op 4769 completed, throughput=5MB/sec 2024-04-07T16:54:27.277 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-00UrrsI-2iA5gzH off=16145285 len=298617 2024-04-07T16:54:27.283 INFO:tasks.workunit.client.0.smithi100.stdout:op 4770 completed, throughput=5MB/sec 2024-04-07T16:54:27.283 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-ZX4H9YnYXjB-dvP off=1218293 len=1689733 2024-04-07T16:54:27.286 INFO:tasks.workunit.client.0.smithi100.stdout:op 4771 completed, throughput=5MB/sec 2024-04-07T16:54:27.286 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-zuLmqR0-DWrYcRt off=5208187 len=227288 2024-04-07T16:54:27.292 INFO:tasks.workunit.client.0.smithi100.stdout:op 4772 completed, throughput=5MB/sec 2024-04-07T16:54:27.292 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-sfCwfJnWKAXgemj off=1398905 len=1701393 2024-04-07T16:54:27.307 INFO:tasks.workunit.client.0.smithi100.stdout:op 4773 completed, throughput=5MB/sec 2024-04-07T16:54:28.308 INFO:tasks.workunit.client.0.smithi100.stdout: 921: throughput=5MB/sec pending data=0 2024-04-07T16:54:28.308 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-B_m5C58w9FyYrup off=11896018 len=1178378 2024-04-07T16:54:28.311 INFO:tasks.workunit.client.0.smithi100.stdout:op 4774 completed, throughput=5MB/sec 2024-04-07T16:54:28.311 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-oqJ4FGBuYVKWLOz off=3074576 len=1775541 2024-04-07T16:54:28.328 INFO:tasks.workunit.client.0.smithi100.stdout:op 4775 completed, throughput=5MB/sec 2024-04-07T16:54:28.328 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-5fnWWuKLJM8Nx3J off=17513972 len=1234172 2024-04-07T16:54:28.331 INFO:tasks.workunit.client.0.smithi100.stdout:op 4776 completed, throughput=5MB/sec 2024-04-07T16:54:28.699 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:54:28 smithi100 bash[76202]: cluster 2024-04-07T16:54:27.556097+0000 mgr.y (mgr.25175) 1173 : cluster [DBG] pgmap v1815: 105 pgs: 105 active+clean; 126 GiB data, 9.8 GiB used, 705 GiB / 715 GiB avail; 1.8 MiB/s rd, 3.0 MiB/s wr, 4 op/s 2024-04-07T16:54:28.699 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:54:28 smithi100 bash[74942]: cluster 2024-04-07T16:54:27.556097+0000 mgr.y (mgr.25175) 1173 : cluster [DBG] pgmap v1815: 105 pgs: 105 active+clean; 126 GiB data, 9.8 GiB used, 705 GiB / 715 GiB avail; 1.8 MiB/s rd, 3.0 MiB/s wr, 4 op/s 2024-04-07T16:54:28.699 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:54:28 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:54:28] "GET /metrics HTTP/1.1" 200 34071 "" "Prometheus/2.43.0" 2024-04-07T16:54:28.862 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:54:28 smithi144 bash[66654]: cluster 2024-04-07T16:54:27.556097+0000 mgr.y (mgr.25175) 1173 : cluster [DBG] pgmap v1815: 105 pgs: 105 active+clean; 126 GiB data, 9.8 GiB used, 705 GiB / 715 GiB avail; 1.8 MiB/s rd, 3.0 MiB/s wr, 4 op/s 2024-04-07T16:54:29.332 INFO:tasks.workunit.client.0.smithi100.stdout: 922: throughput=5MB/sec pending data=0 2024-04-07T16:54:29.332 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-h_v5r-Uy5m0O7BC off=4707888 len=1573266 2024-04-07T16:54:29.337 INFO:tasks.workunit.client.0.smithi100.stdout:op 4777 completed, throughput=5MB/sec 2024-04-07T16:54:29.337 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-WX2jvTMx4YFRKZ3 off=17517989 len=497133 2024-04-07T16:54:29.344 INFO:tasks.workunit.client.0.smithi100.stdout:op 4778 completed, throughput=5MB/sec 2024-04-07T16:54:29.344 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-NA35Pyj7kOSu0Mn off=237730 len=439621 2024-04-07T16:54:29.350 INFO:tasks.workunit.client.0.smithi100.stdout:op 4779 completed, throughput=5MB/sec 2024-04-07T16:54:29.350 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-VfMc80rb4EgOaxg off=10400482 len=1136695 2024-04-07T16:54:29.359 INFO:tasks.workunit.client.0.smithi100.stdout:op 4780 completed, throughput=5MB/sec 2024-04-07T16:54:29.359 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-nrMWK1uzc_Z-zwi off=15240128 len=250452 2024-04-07T16:54:29.364 INFO:tasks.workunit.client.0.smithi100.stdout:op 4781 completed, throughput=5MB/sec 2024-04-07T16:54:29.364 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-1yZuMW6L5MZPn_B off=1528793 len=335688 2024-04-07T16:54:29.371 INFO:tasks.workunit.client.0.smithi100.stdout:op 4782 completed, throughput=5MB/sec 2024-04-07T16:54:29.371 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-J-oMtRDrgGM7IVX off=3028928 len=1169874 2024-04-07T16:54:29.382 INFO:tasks.workunit.client.0.smithi100.stdout:op 4783 completed, throughput=5MB/sec 2024-04-07T16:54:30.383 INFO:tasks.workunit.client.0.smithi100.stdout: 923: throughput=4.99MB/sec pending data=0 2024-04-07T16:54:30.383 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-AWvmw3og_p5jY_T off=0 len=1095677 2024-04-07T16:54:30.386 INFO:tasks.workunit.client.0.smithi100.stdout:op 4784 completed, throughput=5MB/sec 2024-04-07T16:54:30.386 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-BAHmKyKvaPjkg6K off=4299162 len=1982841 2024-04-07T16:54:30.391 INFO:tasks.workunit.client.0.smithi100.stdout:op 4785 completed, throughput=5MB/sec 2024-04-07T16:54:30.391 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-lHp9vFweQk_saG5 off=362984 len=768174 2024-04-07T16:54:30.393 INFO:tasks.workunit.client.0.smithi100.stdout:op 4786 completed, throughput=5MB/sec 2024-04-07T16:54:30.393 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Ku6Hls8OOdZ4QIT off=5387054 len=2054555 2024-04-07T16:54:30.418 INFO:tasks.workunit.client.0.smithi100.stdout:op 4787 completed, throughput=5MB/sec 2024-04-07T16:54:30.862 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:54:30 smithi144 bash[66654]: cluster 2024-04-07T16:54:29.556985+0000 mgr.y (mgr.25175) 1174 : cluster [DBG] pgmap v1816: 105 pgs: 105 active+clean; 126 GiB data, 9.8 GiB used, 705 GiB / 715 GiB avail; 1.8 MiB/s rd, 2.6 MiB/s wr, 4 op/s 2024-04-07T16:54:30.949 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:54:30 smithi100 bash[74942]: cluster 2024-04-07T16:54:29.556985+0000 mgr.y (mgr.25175) 1174 : cluster [DBG] pgmap v1816: 105 pgs: 105 active+clean; 126 GiB data, 9.8 GiB used, 705 GiB / 715 GiB avail; 1.8 MiB/s rd, 2.6 MiB/s wr, 4 op/s 2024-04-07T16:54:30.949 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:54:30 smithi100 bash[76202]: cluster 2024-04-07T16:54:29.556985+0000 mgr.y (mgr.25175) 1174 : cluster [DBG] pgmap v1816: 105 pgs: 105 active+clean; 126 GiB data, 9.8 GiB used, 705 GiB / 715 GiB avail; 1.8 MiB/s rd, 2.6 MiB/s wr, 4 op/s 2024-04-07T16:54:31.418 INFO:tasks.workunit.client.0.smithi100.stdout: 924: throughput=5MB/sec pending data=0 2024-04-07T16:54:31.418 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-cwoUL4s1FALjUUB off=15833476 len=378558 2024-04-07T16:54:31.420 INFO:tasks.workunit.client.0.smithi100.stdout:op 4788 completed, throughput=5MB/sec 2024-04-07T16:54:31.420 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Vdo61Ki8BdrsliF off=7881501 len=1653635 2024-04-07T16:54:31.434 INFO:tasks.workunit.client.0.smithi100.stdout:op 4789 completed, throughput=5MB/sec 2024-04-07T16:54:31.434 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Rn-ssH7I3dwUGpZ off=21401459 len=217993 2024-04-07T16:54:31.440 INFO:tasks.workunit.client.0.smithi100.stdout:op 4790 completed, throughput=5MB/sec 2024-04-07T16:54:31.440 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-7qKU7kotv87CZyb off=15909944 len=1101652 2024-04-07T16:54:31.443 INFO:tasks.workunit.client.0.smithi100.stdout:op 4791 completed, throughput=5MB/sec 2024-04-07T16:54:31.443 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-TJqIfxAnrsLX7vo off=3777838 len=1901418 2024-04-07T16:54:31.447 INFO:tasks.workunit.client.0.smithi100.stdout:op 4792 completed, throughput=5MB/sec 2024-04-07T16:54:32.448 INFO:tasks.workunit.client.0.smithi100.stdout: 925: throughput=5MB/sec pending data=0 2024-04-07T16:54:32.448 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-AL-uZ14q4aVq9f1 off=7743681 len=115363 2024-04-07T16:54:32.449 INFO:tasks.workunit.client.0.smithi100.stdout:op 4793 completed, throughput=5MB/sec 2024-04-07T16:54:32.449 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-tk-Vs2d3218QNhV off=2047525 len=1753350 2024-04-07T16:54:32.461 INFO:tasks.workunit.client.0.smithi100.stdout:op 4794 completed, throughput=5MB/sec 2024-04-07T16:54:32.462 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-JvQAPj-iAd1DbsK off=11790420 len=1590156 2024-04-07T16:54:32.464 INFO:tasks.workunit.client.0.smithi100.stdout:op 4795 completed, throughput=5MB/sec 2024-04-07T16:54:32.464 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-m83XYkvQ7zYtX1_ off=9582056 len=1828251 2024-04-07T16:54:32.477 INFO:tasks.workunit.client.0.smithi100.stdout:op 4796 completed, throughput=5MB/sec 2024-04-07T16:54:32.612 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:54:32 smithi144 bash[95233]: ts=2024-04-07T16:54:32.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:54:32.949 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:54:32 smithi100 bash[74942]: cluster 2024-04-07T16:54:31.557832+0000 mgr.y (mgr.25175) 1175 : cluster [DBG] pgmap v1817: 105 pgs: 105 active+clean; 126 GiB data, 9.8 GiB used, 705 GiB / 715 GiB avail; 2.1 MiB/s rd, 2.7 MiB/s wr, 4 op/s 2024-04-07T16:54:32.949 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:54:32 smithi100 bash[74942]: audit 2024-04-07T16:54:31.783386+0000 mon.a (mon.0) 2182 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:54:32.949 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:54:32 smithi100 bash[76202]: cluster 2024-04-07T16:54:31.557832+0000 mgr.y (mgr.25175) 1175 : cluster [DBG] pgmap v1817: 105 pgs: 105 active+clean; 126 GiB data, 9.8 GiB used, 705 GiB / 715 GiB avail; 2.1 MiB/s rd, 2.7 MiB/s wr, 4 op/s 2024-04-07T16:54:32.949 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:54:32 smithi100 bash[76202]: audit 2024-04-07T16:54:31.783386+0000 mon.a (mon.0) 2182 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:54:33.112 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:54:32 smithi144 bash[66654]: cluster 2024-04-07T16:54:31.557832+0000 mgr.y (mgr.25175) 1175 : cluster [DBG] pgmap v1817: 105 pgs: 105 active+clean; 126 GiB data, 9.8 GiB used, 705 GiB / 715 GiB avail; 2.1 MiB/s rd, 2.7 MiB/s wr, 4 op/s 2024-04-07T16:54:33.112 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:54:32 smithi144 bash[66654]: audit 2024-04-07T16:54:31.783386+0000 mon.a (mon.0) 2182 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:54:33.478 INFO:tasks.workunit.client.0.smithi100.stdout: 926: throughput=5MB/sec pending data=0 2024-04-07T16:54:33.478 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-dS408lnQrcc6wtM off=10435897 len=1528910 2024-04-07T16:54:33.483 INFO:tasks.workunit.client.0.smithi100.stdout:op 4797 completed, throughput=5MB/sec 2024-04-07T16:54:33.484 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-s5PCOMEx3sqHNlB off=4118720 len=147253 2024-04-07T16:54:33.485 INFO:tasks.workunit.client.0.smithi100.stdout:op 4798 completed, throughput=5MB/sec 2024-04-07T16:54:33.485 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-YmYDYmND47mqfyX off=3420827 len=75801 2024-04-07T16:54:33.486 INFO:tasks.workunit.client.0.smithi100.stdout:op 4799 completed, throughput=5MB/sec 2024-04-07T16:54:33.486 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Z-vWdSZIoRARWpd off=6641514 len=429611 2024-04-07T16:54:33.492 INFO:tasks.workunit.client.0.smithi100.stdout:op 4800 completed, throughput=5MB/sec 2024-04-07T16:54:33.492 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-EzvKjk8oDRO7zQQ off=8325983 len=1978946 2024-04-07T16:54:33.508 INFO:tasks.workunit.client.0.smithi100.stdout:op 4801 completed, throughput=5MB/sec 2024-04-07T16:54:33.508 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-DFMyJzjoyUmL2R- off=7023079 len=1751248 2024-04-07T16:54:33.512 INFO:tasks.workunit.client.0.smithi100.stdout:op 4802 completed, throughput=5MB/sec 2024-04-07T16:54:34.513 INFO:tasks.workunit.client.0.smithi100.stdout: 927: throughput=5MB/sec pending data=0 2024-04-07T16:54:34.513 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-JNcFdpC6jxPoze5 off=3444271 len=768305 2024-04-07T16:54:34.522 INFO:tasks.workunit.client.0.smithi100.stdout:op 4803 completed, throughput=5MB/sec 2024-04-07T16:54:34.523 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-WSmYq77Asoc6XzA off=3472917 len=650651 2024-04-07T16:54:34.525 INFO:tasks.workunit.client.0.smithi100.stdout:op 4804 completed, throughput=5MB/sec 2024-04-07T16:54:34.525 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-ZOHoXqmQglpE971 off=1108372 len=523576 2024-04-07T16:54:34.532 INFO:tasks.workunit.client.0.smithi100.stdout:op 4805 completed, throughput=5MB/sec 2024-04-07T16:54:34.532 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-KvFXNyCKZlmpNcH off=1970048 len=1321570 2024-04-07T16:54:34.544 INFO:tasks.workunit.client.0.smithi100.stdout:op 4806 completed, throughput=5MB/sec 2024-04-07T16:54:34.544 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-YCBNkzrKKTy0WaU off=1698533 len=1863076 2024-04-07T16:54:34.549 INFO:tasks.workunit.client.0.smithi100.stdout:op 4807 completed, throughput=5MB/sec 2024-04-07T16:54:34.949 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:54:34 smithi100 bash[74942]: cluster 2024-04-07T16:54:33.558945+0000 mgr.y (mgr.25175) 1176 : cluster [DBG] pgmap v1818: 105 pgs: 105 active+clean; 126 GiB data, 9.8 GiB used, 705 GiB / 715 GiB avail; 2.6 MiB/s rd, 3.3 MiB/s wr, 5 op/s 2024-04-07T16:54:34.949 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:54:34 smithi100 bash[76202]: cluster 2024-04-07T16:54:33.558945+0000 mgr.y (mgr.25175) 1176 : cluster [DBG] pgmap v1818: 105 pgs: 105 active+clean; 126 GiB data, 9.8 GiB used, 705 GiB / 715 GiB avail; 2.6 MiB/s rd, 3.3 MiB/s wr, 5 op/s 2024-04-07T16:54:35.112 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:54:34 smithi144 bash[66654]: cluster 2024-04-07T16:54:33.558945+0000 mgr.y (mgr.25175) 1176 : cluster [DBG] pgmap v1818: 105 pgs: 105 active+clean; 126 GiB data, 9.8 GiB used, 705 GiB / 715 GiB avail; 2.6 MiB/s rd, 3.3 MiB/s wr, 5 op/s 2024-04-07T16:54:35.549 INFO:tasks.workunit.client.0.smithi100.stdout: 928: throughput=5MB/sec pending data=0 2024-04-07T16:54:35.549 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Sa3NNWIAc2-BJHi off=7027621 len=635688 2024-04-07T16:54:35.551 INFO:tasks.workunit.client.0.smithi100.stdout:op 4808 completed, throughput=5MB/sec 2024-04-07T16:54:35.551 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-ZyMu5OvveBEvfk9 off=3305917 len=1363746 2024-04-07T16:54:35.562 INFO:tasks.workunit.client.0.smithi100.stdout:op 4809 completed, throughput=5MB/sec 2024-04-07T16:54:35.562 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj--emvXYr7n3vwrYJ off=14703882 len=1320158 2024-04-07T16:54:35.568 INFO:tasks.workunit.client.0.smithi100.stdout:op 4810 completed, throughput=5MB/sec 2024-04-07T16:54:35.568 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-dhxeUnb-0G8CxuV off=13489979 len=1196347 2024-04-07T16:54:35.578 INFO:tasks.workunit.client.0.smithi100.stdout:op 4811 completed, throughput=5MB/sec 2024-04-07T16:54:35.578 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-_83KVpsLEN57xfJ off=1184605 len=1505643 2024-04-07T16:54:35.589 INFO:tasks.workunit.client.0.smithi100.stdout:op 4812 completed, throughput=5MB/sec 2024-04-07T16:54:36.590 INFO:tasks.workunit.client.0.smithi100.stdout: 929: throughput=5MB/sec pending data=0 2024-04-07T16:54:36.590 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-WCTw6iKgGYqZII4 off=16929337 len=1596440 2024-04-07T16:54:36.603 INFO:tasks.workunit.client.0.smithi100.stdout:op 4813 completed, throughput=5MB/sec 2024-04-07T16:54:36.603 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-N64OYg_TKsgDBaa off=2361093 len=1396808 2024-04-07T16:54:36.606 INFO:tasks.workunit.client.0.smithi100.stdout:op 4814 completed, throughput=5MB/sec 2024-04-07T16:54:36.606 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-aPQGhQvqJCOyeXA off=7423539 len=731141 2024-04-07T16:54:36.615 INFO:tasks.workunit.client.0.smithi100.stdout:op 4815 completed, throughput=5MB/sec 2024-04-07T16:54:36.615 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Vxl6u2kTAcHBMwC off=10412456 len=269278 2024-04-07T16:54:36.621 INFO:tasks.workunit.client.0.smithi100.stdout:op 4816 completed, throughput=5MB/sec 2024-04-07T16:54:36.621 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-DJHZbzpVvqTO5F5 off=1126393 len=131174 2024-04-07T16:54:36.627 INFO:tasks.workunit.client.0.smithi100.stdout:op 4817 completed, throughput=5MB/sec 2024-04-07T16:54:36.627 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-7IqLtJmrsUGJm7O off=20666836 len=551036 2024-04-07T16:54:36.634 INFO:tasks.workunit.client.0.smithi100.stdout:op 4818 completed, throughput=5MB/sec 2024-04-07T16:54:36.949 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:54:36 smithi100 bash[74942]: cluster 2024-04-07T16:54:35.560378+0000 mgr.y (mgr.25175) 1177 : cluster [DBG] pgmap v1819: 105 pgs: 105 active+clean; 126 GiB data, 9.8 GiB used, 705 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.8 MiB/s wr, 5 op/s 2024-04-07T16:54:36.949 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:54:36 smithi100 bash[76202]: cluster 2024-04-07T16:54:35.560378+0000 mgr.y (mgr.25175) 1177 : cluster [DBG] pgmap v1819: 105 pgs: 105 active+clean; 126 GiB data, 9.8 GiB used, 705 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.8 MiB/s wr, 5 op/s 2024-04-07T16:54:37.112 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:54:36 smithi144 bash[66654]: cluster 2024-04-07T16:54:35.560378+0000 mgr.y (mgr.25175) 1177 : cluster [DBG] pgmap v1819: 105 pgs: 105 active+clean; 126 GiB data, 9.8 GiB used, 705 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.8 MiB/s wr, 5 op/s 2024-04-07T16:54:37.634 INFO:tasks.workunit.client.0.smithi100.stdout: 930: throughput=5MB/sec pending data=0 2024-04-07T16:54:37.634 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-0oTQ338Vev9havo off=18421955 len=1340735 2024-04-07T16:54:37.647 INFO:tasks.workunit.client.0.smithi100.stdout:op 4819 completed, throughput=5MB/sec 2024-04-07T16:54:37.647 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-k9AsolqRHX70XbR off=10962609 len=1146672 2024-04-07T16:54:37.657 INFO:tasks.workunit.client.0.smithi100.stdout:op 4820 completed, throughput=5MB/sec 2024-04-07T16:54:37.657 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-lzt1mexEFehHnRd off=7088848 len=1051959 2024-04-07T16:54:37.659 INFO:tasks.workunit.client.0.smithi100.stdout:op 4821 completed, throughput=5MB/sec 2024-04-07T16:54:37.659 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Lyd-LFzHN2fcLbg off=9874208 len=205098 2024-04-07T16:54:37.664 INFO:tasks.workunit.client.0.smithi100.stdout:op 4822 completed, throughput=5MB/sec 2024-04-07T16:54:37.664 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-0q66lX1oMIOLBRg off=19935673 len=1293721 2024-04-07T16:54:37.675 INFO:tasks.workunit.client.0.smithi100.stdout:op 4823 completed, throughput=5MB/sec 2024-04-07T16:54:37.676 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-HyMJMSfz3uxf0PM off=9646541 len=1133943 2024-04-07T16:54:37.677 INFO:tasks.workunit.client.0.smithi100.stdout:op 4824 completed, throughput=5MB/sec 2024-04-07T16:54:38.678 INFO:tasks.workunit.client.0.smithi100.stdout: 931: throughput=5MB/sec pending data=0 2024-04-07T16:54:38.678 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-L-yfiMaA_uVv5SX off=7948186 len=2071513 2024-04-07T16:54:38.695 INFO:tasks.workunit.client.0.smithi100.stdout:op 4825 completed, throughput=5MB/sec 2024-04-07T16:54:38.695 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-ZXrHrM8iX4Izl3E off=586039 len=1308460 2024-04-07T16:54:38.698 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:54:38 smithi100 bash[74942]: cluster 2024-04-07T16:54:37.561589+0000 mgr.y (mgr.25175) 1178 : cluster [DBG] pgmap v1820: 105 pgs: 105 active+clean; 126 GiB data, 9.8 GiB used, 705 GiB / 715 GiB avail; 2.4 MiB/s rd, 2.7 MiB/s wr, 5 op/s 2024-04-07T16:54:38.699 INFO:tasks.workunit.client.0.smithi100.stdout:op 4826 completed, throughput=5MB/sec 2024-04-07T16:54:38.699 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:54:38 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:54:38] "GET /metrics HTTP/1.1" 200 34070 "" "Prometheus/2.43.0" 2024-04-07T16:54:38.699 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:54:38 smithi100 bash[76202]: cluster 2024-04-07T16:54:37.561589+0000 mgr.y (mgr.25175) 1178 : cluster [DBG] pgmap v1820: 105 pgs: 105 active+clean; 126 GiB data, 9.8 GiB used, 705 GiB / 715 GiB avail; 2.4 MiB/s rd, 2.7 MiB/s wr, 5 op/s 2024-04-07T16:54:38.699 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-rZWMPVk_sUYwCJF off=5812814 len=1380269 2024-04-07T16:54:38.702 INFO:tasks.workunit.client.0.smithi100.stdout:op 4827 completed, throughput=5MB/sec 2024-04-07T16:54:39.112 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:54:38 smithi144 bash[66654]: cluster 2024-04-07T16:54:37.561589+0000 mgr.y (mgr.25175) 1178 : cluster [DBG] pgmap v1820: 105 pgs: 105 active+clean; 126 GiB data, 9.8 GiB used, 705 GiB / 715 GiB avail; 2.4 MiB/s rd, 2.7 MiB/s wr, 5 op/s 2024-04-07T16:54:39.716 INFO:tasks.workunit.client.0.smithi100.stdout: 932: throughput=5MB/sec pending data=0 2024-04-07T16:54:39.716 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-wFxdBaW6mCvU2Pg off=2273826 len=1245984 2024-04-07T16:54:39.718 INFO:tasks.workunit.client.0.smithi100.stdout:op 4828 completed, throughput=5MB/sec 2024-04-07T16:54:39.718 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-azHGzooPo5BZccH off=995772 len=978302 2024-04-07T16:54:39.719 INFO:tasks.workunit.client.0.smithi100.stdout:op 4829 completed, throughput=5MB/sec 2024-04-07T16:54:39.719 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-uhHadAkxUrtczpJ off=1817611 len=370046 2024-04-07T16:54:39.719 INFO:tasks.workunit.client.0.smithi100.stdout:op 4830 completed, throughput=5MB/sec 2024-04-07T16:54:39.719 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-a7dZDPrLz8eTLrB off=56752 len=1539169 2024-04-07T16:54:39.730 INFO:tasks.workunit.client.0.smithi100.stdout:op 4831 completed, throughput=5MB/sec 2024-04-07T16:54:39.730 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Twp15MLmCEYmv2N off=4746917 len=1063951 2024-04-07T16:54:39.739 INFO:tasks.workunit.client.0.smithi100.stdout:op 4832 completed, throughput=5MB/sec 2024-04-07T16:54:40.739 INFO:tasks.workunit.client.0.smithi100.stdout: 933: throughput=4.99MB/sec pending data=0 2024-04-07T16:54:40.739 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-zmCM9iFXx7xcVha off=8981757 len=60006 2024-04-07T16:54:40.744 INFO:tasks.workunit.client.0.smithi100.stdout:op 4833 completed, throughput=5MB/sec 2024-04-07T16:54:40.744 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-ah_ABzNYLR8364G off=13602044 len=1604120 2024-04-07T16:54:40.757 INFO:tasks.workunit.client.0.smithi100.stdout:op 4834 completed, throughput=5MB/sec 2024-04-07T16:54:40.757 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-QcwYF2cleLQ8Bb7 off=7177603 len=1112866 2024-04-07T16:54:40.765 INFO:tasks.workunit.client.0.smithi100.stdout:op 4835 completed, throughput=5MB/sec 2024-04-07T16:54:40.766 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-RoLQaGdmwiG-e8q off=12799946 len=1440049 2024-04-07T16:54:40.770 INFO:tasks.workunit.client.0.smithi100.stdout:op 4836 completed, throughput=5MB/sec 2024-04-07T16:54:40.771 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-tYwywUvUJGt0aVT off=2434223 len=615256 2024-04-07T16:54:40.774 INFO:tasks.workunit.client.0.smithi100.stdout:op 4837 completed, throughput=5MB/sec 2024-04-07T16:54:40.774 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-dVT84LlKgrl2YQ8 off=12954471 len=1817862 2024-04-07T16:54:40.787 INFO:tasks.workunit.client.0.smithi100.stdout:op 4838 completed, throughput=5MB/sec 2024-04-07T16:54:40.939 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:54:40 smithi144 bash[66654]: cluster 2024-04-07T16:54:39.562582+0000 mgr.y (mgr.25175) 1179 : cluster [DBG] pgmap v1821: 105 pgs: 105 active+clean; 126 GiB data, 9.9 GiB used, 705 GiB / 715 GiB avail; 2.3 MiB/s rd, 2.4 MiB/s wr, 4 op/s 2024-04-07T16:54:40.948 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:54:40 smithi100 bash[74942]: cluster 2024-04-07T16:54:39.562582+0000 mgr.y (mgr.25175) 1179 : cluster [DBG] pgmap v1821: 105 pgs: 105 active+clean; 126 GiB data, 9.9 GiB used, 705 GiB / 715 GiB avail; 2.3 MiB/s rd, 2.4 MiB/s wr, 4 op/s 2024-04-07T16:54:40.949 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:54:40 smithi100 bash[76202]: cluster 2024-04-07T16:54:39.562582+0000 mgr.y (mgr.25175) 1179 : cluster [DBG] pgmap v1821: 105 pgs: 105 active+clean; 126 GiB data, 9.9 GiB used, 705 GiB / 715 GiB avail; 2.3 MiB/s rd, 2.4 MiB/s wr, 4 op/s 2024-04-07T16:54:41.788 INFO:tasks.workunit.client.0.smithi100.stdout: 934: throughput=5MB/sec pending data=0 2024-04-07T16:54:41.788 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-0vp9S2besu96Y2e off=577689 len=2023702 2024-04-07T16:54:41.792 INFO:tasks.workunit.client.0.smithi100.stdout:op 4839 completed, throughput=5MB/sec 2024-04-07T16:54:41.792 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-QC1yTPrYfAgSiLs off=5789024 len=1367386 2024-04-07T16:54:41.796 INFO:tasks.workunit.client.0.smithi100.stdout:op 4840 completed, throughput=5MB/sec 2024-04-07T16:54:41.796 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-9_atSwtpeS2WXb5 off=3454478 len=305142 2024-04-07T16:54:41.802 INFO:tasks.workunit.client.0.smithi100.stdout:op 4841 completed, throughput=5MB/sec 2024-04-07T16:54:41.802 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-iIFRfB5cACgwdlm off=22008227 len=1936949 2024-04-07T16:54:41.818 INFO:tasks.workunit.client.0.smithi100.stdout:op 4842 completed, throughput=5MB/sec 2024-04-07T16:54:42.612 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:54:42 smithi144 bash[95233]: ts=2024-04-07T16:54:42.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:54:42.818 INFO:tasks.workunit.client.0.smithi100.stdout: 935: throughput=5MB/sec pending data=0 2024-04-07T16:54:42.818 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-tX-FNfOmYLHJYsn off=10302044 len=147689 2024-04-07T16:54:42.819 INFO:tasks.workunit.client.0.smithi100.stdout:op 4843 completed, throughput=5MB/sec 2024-04-07T16:54:42.819 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-ck_fsWe7_t25jEs off=16370582 len=1650956 2024-04-07T16:54:42.823 INFO:tasks.workunit.client.0.smithi100.stdout:op 4844 completed, throughput=5MB/sec 2024-04-07T16:54:42.823 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-sC7dtxm_f2E2ddJ off=3733753 len=1783772 2024-04-07T16:54:42.827 INFO:tasks.workunit.client.0.smithi100.stdout:op 4845 completed, throughput=5MB/sec 2024-04-07T16:54:42.828 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-bbtyafYni92fmaE off=6611690 len=1056813 2024-04-07T16:54:42.831 INFO:tasks.workunit.client.0.smithi100.stdout:op 4846 completed, throughput=5MB/sec 2024-04-07T16:54:42.949 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:54:42 smithi100 bash[74942]: cluster 2024-04-07T16:54:41.563348+0000 mgr.y (mgr.25175) 1180 : cluster [DBG] pgmap v1822: 105 pgs: 105 active+clean; 126 GiB data, 9.9 GiB used, 705 GiB / 715 GiB avail; 2.4 MiB/s rd, 2.7 MiB/s wr, 5 op/s 2024-04-07T16:54:42.949 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:54:42 smithi100 bash[76202]: cluster 2024-04-07T16:54:41.563348+0000 mgr.y (mgr.25175) 1180 : cluster [DBG] pgmap v1822: 105 pgs: 105 active+clean; 126 GiB data, 9.9 GiB used, 705 GiB / 715 GiB avail; 2.4 MiB/s rd, 2.7 MiB/s wr, 5 op/s 2024-04-07T16:54:43.112 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:54:42 smithi144 bash[66654]: cluster 2024-04-07T16:54:41.563348+0000 mgr.y (mgr.25175) 1180 : cluster [DBG] pgmap v1822: 105 pgs: 105 active+clean; 126 GiB data, 9.9 GiB used, 705 GiB / 715 GiB avail; 2.4 MiB/s rd, 2.7 MiB/s wr, 5 op/s 2024-04-07T16:54:43.831 INFO:tasks.workunit.client.0.smithi100.stdout: 936: throughput=5MB/sec pending data=0 2024-04-07T16:54:43.831 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-aT82LE2J6sX5AVb off=1493221 len=239370 2024-04-07T16:54:43.836 INFO:tasks.workunit.client.0.smithi100.stdout:op 4847 completed, throughput=5MB/sec 2024-04-07T16:54:43.837 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-ityiTXKJeN9nQD3 off=2702903 len=1838153 2024-04-07T16:54:43.853 INFO:tasks.workunit.client.0.smithi100.stdout:op 4848 completed, throughput=5MB/sec 2024-04-07T16:54:43.853 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Kdp8ie9ViFBs_qk off=8279482 len=2017818 2024-04-07T16:54:43.860 INFO:tasks.workunit.client.0.smithi100.stdout:op 4849 completed, throughput=5MB/sec 2024-04-07T16:54:43.860 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-3arSGui_wCW-8pZ off=15189572 len=748148 2024-04-07T16:54:43.869 INFO:tasks.workunit.client.0.smithi100.stdout:op 4850 completed, throughput=5MB/sec 2024-04-07T16:54:44.869 INFO:tasks.workunit.client.0.smithi100.stdout: 937: throughput=5MB/sec pending data=0 2024-04-07T16:54:44.869 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-goU9l5ZJ3dD7waB off=18027647 len=1792064 2024-04-07T16:54:44.873 INFO:tasks.workunit.client.0.smithi100.stdout:op 4851 completed, throughput=5MB/sec 2024-04-07T16:54:44.873 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-WQFu7jHaDsV0K6a off=275676 len=2068863 2024-04-07T16:54:44.889 INFO:tasks.workunit.client.0.smithi100.stdout:op 4852 completed, throughput=5MB/sec 2024-04-07T16:54:44.889 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-HQ9drjHo2wtZ9Ny off=834928 len=698670 2024-04-07T16:54:44.896 INFO:tasks.workunit.client.0.smithi100.stdout:op 4853 completed, throughput=5MB/sec 2024-04-07T16:54:44.896 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-0c-w7Tlk_Thhh9p off=9065698 len=539290 2024-04-07T16:54:44.901 INFO:tasks.workunit.client.0.smithi100.stdout:op 4854 completed, throughput=5MB/sec 2024-04-07T16:54:44.902 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-tEhVnmgXBpARhu_ off=3225287 len=1703272 2024-04-07T16:54:44.913 INFO:tasks.workunit.client.0.smithi100.stdout:op 4855 completed, throughput=5MB/sec 2024-04-07T16:54:44.949 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:54:44 smithi100 bash[74942]: cluster 2024-04-07T16:54:43.564534+0000 mgr.y (mgr.25175) 1181 : cluster [DBG] pgmap v1823: 105 pgs: 105 active+clean; 126 GiB data, 9.9 GiB used, 705 GiB / 715 GiB avail; 3.0 MiB/s rd, 3.1 MiB/s wr, 5 op/s 2024-04-07T16:54:44.949 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:54:44 smithi100 bash[76202]: cluster 2024-04-07T16:54:43.564534+0000 mgr.y (mgr.25175) 1181 : cluster [DBG] pgmap v1823: 105 pgs: 105 active+clean; 126 GiB data, 9.9 GiB used, 705 GiB / 715 GiB avail; 3.0 MiB/s rd, 3.1 MiB/s wr, 5 op/s 2024-04-07T16:54:45.112 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:54:44 smithi144 bash[66654]: cluster 2024-04-07T16:54:43.564534+0000 mgr.y (mgr.25175) 1181 : cluster [DBG] pgmap v1823: 105 pgs: 105 active+clean; 126 GiB data, 9.9 GiB used, 705 GiB / 715 GiB avail; 3.0 MiB/s rd, 3.1 MiB/s wr, 5 op/s 2024-04-07T16:54:45.913 INFO:tasks.workunit.client.0.smithi100.stdout: 938: throughput=5MB/sec pending data=0 2024-04-07T16:54:45.913 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-o3NmGTNGOKsO0Kn off=0 len=1205532 2024-04-07T16:54:45.923 INFO:tasks.workunit.client.0.smithi100.stdout:op 4856 completed, throughput=5MB/sec 2024-04-07T16:54:45.924 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-F26-vVNYlozHHrW off=13054255 len=1334895 2024-04-07T16:54:45.936 INFO:tasks.workunit.client.0.smithi100.stdout:op 4857 completed, throughput=5MB/sec 2024-04-07T16:54:45.936 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Nmqx1Azn1fM15lX off=6977613 len=750175 2024-04-07T16:54:45.940 INFO:tasks.workunit.client.0.smithi100.stdout:op 4858 completed, throughput=5MB/sec 2024-04-07T16:54:45.941 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-aeot9X3OnTa69dD off=16708167 len=1494985 2024-04-07T16:54:45.945 INFO:tasks.workunit.client.0.smithi100.stdout:op 4859 completed, throughput=5MB/sec 2024-04-07T16:54:46.946 INFO:tasks.workunit.client.0.smithi100.stdout: 939: throughput=5MB/sec pending data=0 2024-04-07T16:54:46.946 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-aY9hwCn6RZj9-nr off=20380920 len=2075072 2024-04-07T16:54:46.948 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:54:46 smithi100 bash[74942]: cluster 2024-04-07T16:54:45.565509+0000 mgr.y (mgr.25175) 1182 : cluster [DBG] pgmap v1824: 105 pgs: 105 active+clean; 126 GiB data, 9.9 GiB used, 705 GiB / 715 GiB avail; 2.2 MiB/s rd, 2.7 MiB/s wr, 4 op/s 2024-04-07T16:54:46.949 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:54:46 smithi100 bash[76202]: cluster 2024-04-07T16:54:45.565509+0000 mgr.y (mgr.25175) 1182 : cluster [DBG] pgmap v1824: 105 pgs: 105 active+clean; 126 GiB data, 9.9 GiB used, 705 GiB / 715 GiB avail; 2.2 MiB/s rd, 2.7 MiB/s wr, 4 op/s 2024-04-07T16:54:46.951 INFO:tasks.workunit.client.0.smithi100.stdout:op 4860 completed, throughput=5MB/sec 2024-04-07T16:54:46.951 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-jITvSFyp8zEMy-r off=8843771 len=802912 2024-04-07T16:54:46.959 INFO:tasks.workunit.client.0.smithi100.stdout:op 4861 completed, throughput=5MB/sec 2024-04-07T16:54:46.959 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-1q0r7VEH1J5t8Z3 off=5795511 len=581172 2024-04-07T16:54:46.965 INFO:tasks.workunit.client.0.smithi100.stdout:op 4862 completed, throughput=5MB/sec 2024-04-07T16:54:46.965 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-XCD0NlpcChvwve9 off=4645997 len=1514990 2024-04-07T16:54:46.968 INFO:tasks.workunit.client.0.smithi100.stdout:op 4863 completed, throughput=5MB/sec 2024-04-07T16:54:47.112 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:54:46 smithi144 bash[66654]: cluster 2024-04-07T16:54:45.565509+0000 mgr.y (mgr.25175) 1182 : cluster [DBG] pgmap v1824: 105 pgs: 105 active+clean; 126 GiB data, 9.9 GiB used, 705 GiB / 715 GiB avail; 2.2 MiB/s rd, 2.7 MiB/s wr, 4 op/s 2024-04-07T16:54:47.949 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:54:47 smithi100 bash[74942]: audit 2024-04-07T16:54:46.783857+0000 mon.a (mon.0) 2183 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:54:47.949 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:54:47 smithi100 bash[76202]: audit 2024-04-07T16:54:46.783857+0000 mon.a (mon.0) 2183 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:54:47.969 INFO:tasks.workunit.client.0.smithi100.stdout: 940: throughput=5MB/sec pending data=0 2024-04-07T16:54:47.969 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-HMWQYY8sWlA_RB- off=10588835 len=569979 2024-04-07T16:54:47.972 INFO:tasks.workunit.client.0.smithi100.stdout:op 4864 completed, throughput=5MB/sec 2024-04-07T16:54:47.972 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj--Im6oeHe7-e9itC off=2708156 len=207562 2024-04-07T16:54:47.978 INFO:tasks.workunit.client.0.smithi100.stdout:op 4865 completed, throughput=5MB/sec 2024-04-07T16:54:47.978 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-271UT9S0B4oNoS8 off=7923638 len=710621 2024-04-07T16:54:47.981 INFO:tasks.workunit.client.0.smithi100.stdout:op 4866 completed, throughput=5MB/sec 2024-04-07T16:54:47.981 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-kIJ8Eh1BIoEJcxJ off=6250832 len=94417 2024-04-07T16:54:47.982 INFO:tasks.workunit.client.0.smithi100.stdout:op 4867 completed, throughput=5MB/sec 2024-04-07T16:54:47.982 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-IDXbYwmpYu7ZG66 off=6350210 len=403873 2024-04-07T16:54:47.983 INFO:tasks.workunit.client.0.smithi100.stdout:op 4868 completed, throughput=5MB/sec 2024-04-07T16:54:47.983 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-BQuDmiSxmTgKKE6 off=2192079 len=1580987 2024-04-07T16:54:47.990 INFO:tasks.workunit.client.0.smithi100.stdout:op 4869 completed, throughput=5MB/sec 2024-04-07T16:54:47.990 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-UYRlbDrG0PI77Xe off=4617590 len=225962 2024-04-07T16:54:47.990 INFO:tasks.workunit.client.0.smithi100.stdout:op 4870 completed, throughput=5MB/sec 2024-04-07T16:54:47.990 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-gnjfpmQvaGe32x9 off=1588608 len=95872 2024-04-07T16:54:47.993 INFO:tasks.workunit.client.0.smithi100.stdout:op 4871 completed, throughput=5MB/sec 2024-04-07T16:54:47.993 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-VR6lggY4CqnidXo off=1451657 len=794146 2024-04-07T16:54:48.002 INFO:tasks.workunit.client.0.smithi100.stdout:op 4872 completed, throughput=5MB/sec 2024-04-07T16:54:48.002 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-O3SR6I7kVkplBWm off=588297 len=672959 2024-04-07T16:54:48.008 INFO:tasks.workunit.client.0.smithi100.stdout:op 4873 completed, throughput=5MB/sec 2024-04-07T16:54:48.112 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:54:47 smithi144 bash[66654]: audit 2024-04-07T16:54:46.783857+0000 mon.a (mon.0) 2183 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:54:48.699 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:54:48 smithi100 bash[76202]: cluster 2024-04-07T16:54:47.566582+0000 mgr.y (mgr.25175) 1183 : cluster [DBG] pgmap v1825: 105 pgs: 105 active+clean; 126 GiB data, 9.9 GiB used, 705 GiB / 715 GiB avail; 2.2 MiB/s rd, 2.4 MiB/s wr, 4 op/s 2024-04-07T16:54:48.699 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:54:48 smithi100 bash[74942]: cluster 2024-04-07T16:54:47.566582+0000 mgr.y (mgr.25175) 1183 : cluster [DBG] pgmap v1825: 105 pgs: 105 active+clean; 126 GiB data, 9.9 GiB used, 705 GiB / 715 GiB avail; 2.2 MiB/s rd, 2.4 MiB/s wr, 4 op/s 2024-04-07T16:54:48.699 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:54:48 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:54:48] "GET /metrics HTTP/1.1" 200 34074 "" "Prometheus/2.43.0" 2024-04-07T16:54:49.009 INFO:tasks.workunit.client.0.smithi100.stdout: 941: throughput=5MB/sec pending data=0 2024-04-07T16:54:49.009 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-5l2lPD9GNyW2FpQ off=4453387 len=2029820 2024-04-07T16:54:49.025 INFO:tasks.workunit.client.0.smithi100.stdout:op 4874 completed, throughput=5MB/sec 2024-04-07T16:54:49.025 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-by6wHyheyEINGjg off=1939143 len=345672 2024-04-07T16:54:49.032 INFO:tasks.workunit.client.0.smithi100.stdout:op 4875 completed, throughput=5MB/sec 2024-04-07T16:54:49.032 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-gsz_ONzFRdkhtB- off=6575366 len=1439266 2024-04-07T16:54:49.043 INFO:tasks.workunit.client.0.smithi100.stdout:op 4876 completed, throughput=5MB/sec 2024-04-07T16:54:49.044 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-SM2tcd_CUAduEhH off=4481312 len=1464714 2024-04-07T16:54:49.053 INFO:tasks.workunit.client.0.smithi100.stdout:op 4877 completed, throughput=5MB/sec 2024-04-07T16:54:49.112 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:54:48 smithi144 bash[66654]: cluster 2024-04-07T16:54:47.566582+0000 mgr.y (mgr.25175) 1183 : cluster [DBG] pgmap v1825: 105 pgs: 105 active+clean; 126 GiB data, 9.9 GiB used, 705 GiB / 715 GiB avail; 2.2 MiB/s rd, 2.4 MiB/s wr, 4 op/s 2024-04-07T16:54:50.053 INFO:tasks.workunit.client.0.smithi100.stdout: 942: throughput=5MB/sec pending data=0 2024-04-07T16:54:50.053 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-YK1Ooqban_9ADji off=21964007 len=1306694 2024-04-07T16:54:50.065 INFO:tasks.workunit.client.0.smithi100.stdout:op 4878 completed, throughput=5MB/sec 2024-04-07T16:54:50.065 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-QCNhiGBvmyebU3w off=1820280 len=1152405 2024-04-07T16:54:50.074 INFO:tasks.workunit.client.0.smithi100.stdout:op 4879 completed, throughput=5MB/sec 2024-04-07T16:54:50.074 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-gDLkYJy7sAlcKoM off=5327470 len=371408 2024-04-07T16:54:50.076 INFO:tasks.workunit.client.0.smithi100.stdout:op 4880 completed, throughput=5MB/sec 2024-04-07T16:54:50.076 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-KkXJlimuSmZkXjs off=1816503 len=724997 2024-04-07T16:54:50.084 INFO:tasks.workunit.client.0.smithi100.stdout:op 4881 completed, throughput=5MB/sec 2024-04-07T16:54:50.084 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-aKGqlavafQUD6M5 off=9854194 len=186748 2024-04-07T16:54:50.089 INFO:tasks.workunit.client.0.smithi100.stdout:op 4882 completed, throughput=5MB/sec 2024-04-07T16:54:50.089 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-2RbgNH9euQuxlrl off=22827146 len=1370964 2024-04-07T16:54:50.093 INFO:tasks.workunit.client.0.smithi100.stdout:op 4883 completed, throughput=5MB/sec 2024-04-07T16:54:50.093 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-puxIJNL26htvSWp off=13250252 len=807957 2024-04-07T16:54:50.101 INFO:tasks.workunit.client.0.smithi100.stdout:op 4884 completed, throughput=5MB/sec 2024-04-07T16:54:50.949 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:54:50 smithi100 bash[74942]: cluster 2024-04-07T16:54:49.567548+0000 mgr.y (mgr.25175) 1184 : cluster [DBG] pgmap v1826: 105 pgs: 105 active+clean; 126 GiB data, 9.9 GiB used, 705 GiB / 715 GiB avail; 2.4 MiB/s rd, 2.2 MiB/s wr, 3 op/s 2024-04-07T16:54:50.949 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:54:50 smithi100 bash[76202]: cluster 2024-04-07T16:54:49.567548+0000 mgr.y (mgr.25175) 1184 : cluster [DBG] pgmap v1826: 105 pgs: 105 active+clean; 126 GiB data, 9.9 GiB used, 705 GiB / 715 GiB avail; 2.4 MiB/s rd, 2.2 MiB/s wr, 3 op/s 2024-04-07T16:54:51.101 INFO:tasks.workunit.client.0.smithi100.stdout: 943: throughput=5MB/sec pending data=0 2024-04-07T16:54:51.101 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-ABJtacG_U7y9Gh_ off=17860467 len=778003 2024-04-07T16:54:51.103 INFO:tasks.workunit.client.0.smithi100.stdout:op 4885 completed, throughput=5MB/sec 2024-04-07T16:54:51.104 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-eAsNwJd2p5hTtHy off=1120611 len=1818000 2024-04-07T16:54:51.112 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:54:50 smithi144 bash[66654]: cluster 2024-04-07T16:54:49.567548+0000 mgr.y (mgr.25175) 1184 : cluster [DBG] pgmap v1826: 105 pgs: 105 active+clean; 126 GiB data, 9.9 GiB used, 705 GiB / 715 GiB avail; 2.4 MiB/s rd, 2.2 MiB/s wr, 3 op/s 2024-04-07T16:54:51.119 INFO:tasks.workunit.client.0.smithi100.stdout:op 4886 completed, throughput=5MB/sec 2024-04-07T16:54:51.120 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-zj4oQwU0ru__gAb off=5343234 len=238309 2024-04-07T16:54:51.121 INFO:tasks.workunit.client.0.smithi100.stdout:op 4887 completed, throughput=5MB/sec 2024-04-07T16:54:51.121 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-W1nrYTsO0eDY8_9 off=8722907 len=560362 2024-04-07T16:54:51.128 INFO:tasks.workunit.client.0.smithi100.stdout:op 4888 completed, throughput=5MB/sec 2024-04-07T16:54:51.129 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-2vxhGOF0bJKc0fR off=10101946 len=1099384 2024-04-07T16:54:51.132 INFO:tasks.workunit.client.0.smithi100.stdout:op 4889 completed, throughput=5MB/sec 2024-04-07T16:54:51.132 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-CPxzso5RA9XZGjA off=677304 len=2031315 2024-04-07T16:54:51.136 INFO:tasks.workunit.client.0.smithi100.stdout:op 4890 completed, throughput=5MB/sec 2024-04-07T16:54:52.137 INFO:tasks.workunit.client.0.smithi100.stdout: 944: throughput=5MB/sec pending data=0 2024-04-07T16:54:52.137 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-8wfONl6v4d7WQKL off=10999070 len=74122 2024-04-07T16:54:52.138 INFO:tasks.workunit.client.0.smithi100.stdout:op 4891 completed, throughput=5MB/sec 2024-04-07T16:54:52.138 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-cOpZ1Kbu-wxq-QA off=13523567 len=22432 2024-04-07T16:54:52.139 INFO:tasks.workunit.client.0.smithi100.stdout:op 4892 completed, throughput=5MB/sec 2024-04-07T16:54:52.140 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-MHGGzEONbY4Yu0K off=7035421 len=517307 2024-04-07T16:54:52.142 INFO:tasks.workunit.client.0.smithi100.stdout:op 4893 completed, throughput=5MB/sec 2024-04-07T16:54:52.142 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Vz5cZJS22BUW10_ off=9169318 len=1994313 2024-04-07T16:54:52.147 INFO:tasks.workunit.client.0.smithi100.stdout:op 4894 completed, throughput=5MB/sec 2024-04-07T16:54:52.147 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-ZhruzCyLWAg1zn5 off=22325321 len=284515 2024-04-07T16:54:52.153 INFO:tasks.workunit.client.0.smithi100.stdout:op 4895 completed, throughput=5MB/sec 2024-04-07T16:54:52.153 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-aaEE84zzj3ZYFa7 off=810056 len=1108692 2024-04-07T16:54:52.162 INFO:tasks.workunit.client.0.smithi100.stdout:op 4896 completed, throughput=5MB/sec 2024-04-07T16:54:52.612 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:54:52 smithi144 bash[95233]: ts=2024-04-07T16:54:52.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:54:52.949 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:54:52 smithi100 bash[74942]: cluster 2024-04-07T16:54:51.568458+0000 mgr.y (mgr.25175) 1185 : cluster [DBG] pgmap v1827: 105 pgs: 105 active+clean; 126 GiB data, 9.9 GiB used, 705 GiB / 715 GiB avail; 2.2 MiB/s rd, 2.4 MiB/s wr, 3 op/s 2024-04-07T16:54:52.949 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:54:52 smithi100 bash[76202]: cluster 2024-04-07T16:54:51.568458+0000 mgr.y (mgr.25175) 1185 : cluster [DBG] pgmap v1827: 105 pgs: 105 active+clean; 126 GiB data, 9.9 GiB used, 705 GiB / 715 GiB avail; 2.2 MiB/s rd, 2.4 MiB/s wr, 3 op/s 2024-04-07T16:54:53.112 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:54:52 smithi144 bash[66654]: cluster 2024-04-07T16:54:51.568458+0000 mgr.y (mgr.25175) 1185 : cluster [DBG] pgmap v1827: 105 pgs: 105 active+clean; 126 GiB data, 9.9 GiB used, 705 GiB / 715 GiB avail; 2.2 MiB/s rd, 2.4 MiB/s wr, 3 op/s 2024-04-07T16:54:53.163 INFO:tasks.workunit.client.0.smithi100.stdout: 945: throughput=5MB/sec pending data=0 2024-04-07T16:54:53.163 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-ILNSO-iCDTJ4uGQ off=11138749 len=1134352 2024-04-07T16:54:53.166 INFO:tasks.workunit.client.0.smithi100.stdout:op 4897 completed, throughput=5MB/sec 2024-04-07T16:54:53.166 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-uJL1H_QnVRh72bD off=20468283 len=1357592 2024-04-07T16:54:53.170 INFO:tasks.workunit.client.0.smithi100.stdout:op 4898 completed, throughput=5MB/sec 2024-04-07T16:54:53.170 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-3b74EAvMgaJsWZd off=3849954 len=614116 2024-04-07T16:54:53.177 INFO:tasks.workunit.client.0.smithi100.stdout:op 4899 completed, throughput=5MB/sec 2024-04-07T16:54:53.177 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-UEwtYhjv9W40isE off=5722715 len=462259 2024-04-07T16:54:53.183 INFO:tasks.workunit.client.0.smithi100.stdout:op 4900 completed, throughput=5MB/sec 2024-04-07T16:54:53.184 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-BFeKjFF5VrgU_q_ off=4267172 len=334213 2024-04-07T16:54:53.189 INFO:tasks.workunit.client.0.smithi100.stdout:op 4901 completed, throughput=5MB/sec 2024-04-07T16:54:53.189 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-V76NZNSqme8_gEs off=4327561 len=1881393 2024-04-07T16:54:53.203 INFO:tasks.workunit.client.0.smithi100.stdout:op 4902 completed, throughput=5MB/sec 2024-04-07T16:54:54.204 INFO:tasks.workunit.client.0.smithi100.stdout: 946: throughput=5MB/sec pending data=0 2024-04-07T16:54:54.204 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-A9p1ufz8UYMeKzv off=5421104 len=1198392 2024-04-07T16:54:54.207 INFO:tasks.workunit.client.0.smithi100.stdout:op 4903 completed, throughput=5MB/sec 2024-04-07T16:54:54.207 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-ulkRXJD5TaR_NcO off=484122 len=1333813 2024-04-07T16:54:54.212 INFO:tasks.workunit.client.0.smithi100.stdout:op 4904 completed, throughput=5MB/sec 2024-04-07T16:54:54.212 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-09JjSfJYuDtHHKX off=7141181 len=185612 2024-04-07T16:54:54.214 INFO:tasks.workunit.client.0.smithi100.stdout:op 4905 completed, throughput=5MB/sec 2024-04-07T16:54:54.214 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-0SkfILSHkJldhOL off=1521574 len=235135 2024-04-07T16:54:54.220 INFO:tasks.workunit.client.0.smithi100.stdout:op 4906 completed, throughput=5MB/sec 2024-04-07T16:54:54.221 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-k69DK-9gH99LP2f off=10885234 len=993850 2024-04-07T16:54:54.230 INFO:tasks.workunit.client.0.smithi100.stdout:op 4907 completed, throughput=5MB/sec 2024-04-07T16:54:54.230 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-x4Taih1_dLowjF6 off=6252755 len=1251876 2024-04-07T16:54:54.233 INFO:tasks.workunit.client.0.smithi100.stdout:op 4908 completed, throughput=5MB/sec 2024-04-07T16:54:54.950 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:54:54 smithi100 bash[74942]: cluster 2024-04-07T16:54:53.569554+0000 mgr.y (mgr.25175) 1186 : cluster [DBG] pgmap v1828: 105 pgs: 105 active+clean; 126 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 2.3 MiB/s rd, 2.8 MiB/s wr, 4 op/s 2024-04-07T16:54:54.951 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:54:54 smithi100 bash[76202]: cluster 2024-04-07T16:54:53.569554+0000 mgr.y (mgr.25175) 1186 : cluster [DBG] pgmap v1828: 105 pgs: 105 active+clean; 126 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 2.3 MiB/s rd, 2.8 MiB/s wr, 4 op/s 2024-04-07T16:54:55.112 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:54:54 smithi144 bash[66654]: cluster 2024-04-07T16:54:53.569554+0000 mgr.y (mgr.25175) 1186 : cluster [DBG] pgmap v1828: 105 pgs: 105 active+clean; 126 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 2.3 MiB/s rd, 2.8 MiB/s wr, 4 op/s 2024-04-07T16:54:55.233 INFO:tasks.workunit.client.0.smithi100.stdout: 947: throughput=5MB/sec pending data=0 2024-04-07T16:54:55.233 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-PI2FbNcvIDx0R3Z off=4761811 len=1887701 2024-04-07T16:54:55.238 INFO:tasks.workunit.client.0.smithi100.stdout:op 4909 completed, throughput=5MB/sec 2024-04-07T16:54:55.238 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-ZT-FB_jm-2wxpaX off=1979884 len=1860408 2024-04-07T16:54:55.252 INFO:tasks.workunit.client.0.smithi100.stdout:op 4910 completed, throughput=5MB/sec 2024-04-07T16:54:55.252 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-koZA__3f67fHHQS off=6653254 len=1980875 2024-04-07T16:54:55.268 INFO:tasks.workunit.client.0.smithi100.stdout:op 4911 completed, throughput=5MB/sec 2024-04-07T16:54:56.268 INFO:tasks.workunit.client.0.smithi100.stdout: 948: throughput=5MB/sec pending data=0 2024-04-07T16:54:56.268 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj--SiW3LOn7isJ_1I off=488145 len=719764 2024-04-07T16:54:56.270 INFO:tasks.workunit.client.0.smithi100.stdout:op 4912 completed, throughput=5MB/sec 2024-04-07T16:54:56.270 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj--MnYX1DOh0dOfDF off=8543568 len=1893344 2024-04-07T16:54:56.275 INFO:tasks.workunit.client.0.smithi100.stdout:op 4913 completed, throughput=5MB/sec 2024-04-07T16:54:56.275 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-15EjOVqPfKR5rhQ off=989514 len=1382495 2024-04-07T16:54:56.290 INFO:tasks.workunit.client.0.smithi100.stdout:op 4914 completed, throughput=5MB/sec 2024-04-07T16:54:56.290 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-NxEuc9fSyL5fpj7 off=2550188 len=263134 2024-04-07T16:54:56.295 INFO:tasks.workunit.client.0.smithi100.stdout:op 4915 completed, throughput=5MB/sec 2024-04-07T16:54:56.295 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-6-z7m1GoQLLsDyS off=12219893 len=1923547 2024-04-07T16:54:56.300 INFO:tasks.workunit.client.0.smithi100.stdout:op 4916 completed, throughput=5MB/sec 2024-04-07T16:54:56.949 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:54:56 smithi100 bash[74942]: cluster 2024-04-07T16:54:55.570538+0000 mgr.y (mgr.25175) 1187 : cluster [DBG] pgmap v1829: 105 pgs: 105 active+clean; 126 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 1.9 MiB/s rd, 2.4 MiB/s wr, 4 op/s 2024-04-07T16:54:56.949 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:54:56 smithi100 bash[76202]: cluster 2024-04-07T16:54:55.570538+0000 mgr.y (mgr.25175) 1187 : cluster [DBG] pgmap v1829: 105 pgs: 105 active+clean; 126 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 1.9 MiB/s rd, 2.4 MiB/s wr, 4 op/s 2024-04-07T16:54:57.112 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:54:56 smithi144 bash[66654]: cluster 2024-04-07T16:54:55.570538+0000 mgr.y (mgr.25175) 1187 : cluster [DBG] pgmap v1829: 105 pgs: 105 active+clean; 126 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 1.9 MiB/s rd, 2.4 MiB/s wr, 4 op/s 2024-04-07T16:54:57.300 INFO:tasks.workunit.client.0.smithi100.stdout: 949: throughput=5MB/sec pending data=0 2024-04-07T16:54:57.301 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-biUATuybCfLimT9 off=5513961 len=93103 2024-04-07T16:54:57.305 INFO:tasks.workunit.client.0.smithi100.stdout:op 4917 completed, throughput=5MB/sec 2024-04-07T16:54:57.305 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-ZW0KB9Zpwo-GJHR off=1035035 len=671714 2024-04-07T16:54:57.312 INFO:tasks.workunit.client.0.smithi100.stdout:op 4918 completed, throughput=5MB/sec 2024-04-07T16:54:57.313 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Qgj0enORZbuH1Ou off=8425566 len=1571407 2024-04-07T16:54:57.324 INFO:tasks.workunit.client.0.smithi100.stdout:op 4919 completed, throughput=5MB/sec 2024-04-07T16:54:57.324 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-1gLQYG-giD7ripU off=11453054 len=530061 2024-04-07T16:54:57.331 INFO:tasks.workunit.client.0.smithi100.stdout:op 4920 completed, throughput=5MB/sec 2024-04-07T16:54:57.331 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-W4zvSnks-ey6XYc off=4971591 len=1743177 2024-04-07T16:54:57.336 INFO:tasks.workunit.client.0.smithi100.stdout:op 4921 completed, throughput=5MB/sec 2024-04-07T16:54:58.336 INFO:tasks.workunit.client.0.smithi100.stdout: 950: throughput=5MB/sec pending data=0 2024-04-07T16:54:58.337 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-YPrpGPExP_Zi-AF off=6431963 len=85455 2024-04-07T16:54:58.342 INFO:tasks.workunit.client.0.smithi100.stdout:op 4922 completed, throughput=5MB/sec 2024-04-07T16:54:58.342 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-n6QJY1a_sQAHyXC off=11573322 len=1513802 2024-04-07T16:54:58.347 INFO:tasks.workunit.client.0.smithi100.stdout:op 4923 completed, throughput=5MB/sec 2024-04-07T16:54:58.347 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-z0TaWxMC_8wZ4ce off=9749495 len=1644434 2024-04-07T16:54:58.360 INFO:tasks.workunit.client.0.smithi100.stdout:op 4924 completed, throughput=5MB/sec 2024-04-07T16:54:58.360 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-SntoQIFJ5ZS5lR_ off=2356284 len=1064813 2024-04-07T16:54:58.362 INFO:tasks.workunit.client.0.smithi100.stdout:op 4925 completed, throughput=5MB/sec 2024-04-07T16:54:58.362 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-ogiS0LoyPhKuqgA off=19131232 len=238452 2024-04-07T16:54:58.367 INFO:tasks.workunit.client.0.smithi100.stdout:op 4926 completed, throughput=5MB/sec 2024-04-07T16:54:58.368 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-c3gLl6HzY6e2yuV off=1151103 len=1231222 2024-04-07T16:54:58.378 INFO:tasks.workunit.client.0.smithi100.stdout:op 4927 completed, throughput=5MB/sec 2024-04-07T16:54:58.682 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:54:58 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:54:58] "GET /metrics HTTP/1.1" 200 34074 "" "Prometheus/2.43.0" 2024-04-07T16:54:58.949 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:54:58 smithi100 bash[74942]: cluster 2024-04-07T16:54:57.571614+0000 mgr.y (mgr.25175) 1188 : cluster [DBG] pgmap v1830: 105 pgs: 105 active+clean; 126 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 2.1 MiB/s rd, 2.5 MiB/s wr, 4 op/s 2024-04-07T16:54:58.949 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:54:58 smithi100 bash[76202]: cluster 2024-04-07T16:54:57.571614+0000 mgr.y (mgr.25175) 1188 : cluster [DBG] pgmap v1830: 105 pgs: 105 active+clean; 126 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 2.1 MiB/s rd, 2.5 MiB/s wr, 4 op/s 2024-04-07T16:54:59.112 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:54:58 smithi144 bash[66654]: cluster 2024-04-07T16:54:57.571614+0000 mgr.y (mgr.25175) 1188 : cluster [DBG] pgmap v1830: 105 pgs: 105 active+clean; 126 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 2.1 MiB/s rd, 2.5 MiB/s wr, 4 op/s 2024-04-07T16:54:59.378 INFO:tasks.workunit.client.0.smithi100.stdout: 951: throughput=5MB/sec pending data=0 2024-04-07T16:54:59.379 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-xVEGzEcgznSKN-M off=1221184 len=1769521 2024-04-07T16:54:59.393 INFO:tasks.workunit.client.0.smithi100.stdout:op 4928 completed, throughput=5MB/sec 2024-04-07T16:54:59.393 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-05MYR7aVxJXI7_l off=611099 len=93672 2024-04-07T16:54:59.399 INFO:tasks.workunit.client.0.smithi100.stdout:op 4929 completed, throughput=5MB/sec 2024-04-07T16:54:59.399 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-R6yJtGlv0IdQbsg off=590336 len=1660082 2024-04-07T16:54:59.403 INFO:tasks.workunit.client.0.smithi100.stdout:op 4930 completed, throughput=5MB/sec 2024-04-07T16:54:59.403 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-DXeUGjaiiPeQEZF off=14191362 len=1154718 2024-04-07T16:54:59.406 INFO:tasks.workunit.client.0.smithi100.stdout:op 4931 completed, throughput=5MB/sec 2024-04-07T16:55:00.407 INFO:tasks.workunit.client.0.smithi100.stdout: 952: throughput=5MB/sec pending data=0 2024-04-07T16:55:00.407 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-271UT9S0B4oNoS8 off=5427211 len=1572557 2024-04-07T16:55:00.420 INFO:tasks.workunit.client.0.smithi100.stdout:op 4932 completed, throughput=5MB/sec 2024-04-07T16:55:00.420 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-YHG0wYoZIHjdiI7 off=510184 len=101226 2024-04-07T16:55:00.425 INFO:tasks.workunit.client.0.smithi100.stdout:op 4933 completed, throughput=5MB/sec 2024-04-07T16:55:00.425 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-suCEPlBmQ8Ru3Cw off=197992 len=1475706 2024-04-07T16:55:00.429 INFO:tasks.workunit.client.0.smithi100.stdout:op 4934 completed, throughput=5MB/sec 2024-04-07T16:55:00.429 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-LpFeZw8D7NgPlUS off=6652951 len=2021352 2024-04-07T16:55:00.436 INFO:tasks.workunit.client.0.smithi100.stdout:op 4935 completed, throughput=5MB/sec 2024-04-07T16:55:00.949 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:55:00 smithi100 bash[74942]: cluster 2024-04-07T16:54:59.572597+0000 mgr.y (mgr.25175) 1189 : cluster [DBG] pgmap v1831: 105 pgs: 105 active+clean; 126 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 2.2 MiB/s rd, 2.4 MiB/s wr, 4 op/s 2024-04-07T16:55:00.949 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:55:00 smithi100 bash[76202]: cluster 2024-04-07T16:54:59.572597+0000 mgr.y (mgr.25175) 1189 : cluster [DBG] pgmap v1831: 105 pgs: 105 active+clean; 126 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 2.2 MiB/s rd, 2.4 MiB/s wr, 4 op/s 2024-04-07T16:55:01.112 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:55:00 smithi144 bash[66654]: cluster 2024-04-07T16:54:59.572597+0000 mgr.y (mgr.25175) 1189 : cluster [DBG] pgmap v1831: 105 pgs: 105 active+clean; 126 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 2.2 MiB/s rd, 2.4 MiB/s wr, 4 op/s 2024-04-07T16:55:01.436 INFO:tasks.workunit.client.0.smithi100.stdout: 953: throughput=5MB/sec pending data=0 2024-04-07T16:55:01.436 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-JLKRKUIdLfdGdkv off=2165244 len=1567412 2024-04-07T16:55:01.440 INFO:tasks.workunit.client.0.smithi100.stdout:op 4936 completed, throughput=5MB/sec 2024-04-07T16:55:01.440 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Get02KlaV3VOx4U off=6492521 len=1858600 2024-04-07T16:55:01.454 INFO:tasks.workunit.client.0.smithi100.stdout:op 4937 completed, throughput=5MB/sec 2024-04-07T16:55:01.455 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-__3vodv4Rvxy2FE off=15849065 len=1155179 2024-04-07T16:55:01.458 INFO:tasks.workunit.client.0.smithi100.stdout:op 4938 completed, throughput=5MB/sec 2024-04-07T16:55:01.458 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-lrfM5lRJmaqenIz off=11184520 len=86600 2024-04-07T16:55:01.462 INFO:tasks.workunit.client.0.smithi100.stdout:op 4939 completed, throughput=5MB/sec 2024-04-07T16:55:01.463 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-NKipEBb2d0wspA- off=252614 len=588244 2024-04-07T16:55:01.469 INFO:tasks.workunit.client.0.smithi100.stdout:op 4940 completed, throughput=5MB/sec 2024-04-07T16:55:02.469 INFO:tasks.workunit.client.0.smithi100.stdout: 954: throughput=4.99MB/sec pending data=0 2024-04-07T16:55:02.469 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Tuyo1OLTDAX1OiU off=1388149 len=107976 2024-04-07T16:55:02.470 INFO:tasks.workunit.client.0.smithi100.stdout:op 4941 completed, throughput=5MB/sec 2024-04-07T16:55:02.470 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-NTgW8ypVj4NdWiQ off=2890481 len=70974 2024-04-07T16:55:02.476 INFO:tasks.workunit.client.0.smithi100.stdout:op 4942 completed, throughput=5MB/sec 2024-04-07T16:55:02.476 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-7aoc4Z439PEzDST off=588848 len=1496384 2024-04-07T16:55:02.480 INFO:tasks.workunit.client.0.smithi100.stdout:op 4943 completed, throughput=5MB/sec 2024-04-07T16:55:02.480 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-H-zsBkWWIwbAa_D off=675792 len=34397 2024-04-07T16:55:02.481 INFO:tasks.workunit.client.0.smithi100.stdout:op 4944 completed, throughput=5MB/sec 2024-04-07T16:55:02.481 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-zFJFjWqRuJijz9w off=2085109 len=254133 2024-04-07T16:55:02.483 INFO:tasks.workunit.client.0.smithi100.stdout:op 4945 completed, throughput=5MB/sec 2024-04-07T16:55:02.483 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-oPqv80k5tIjz4aF off=4471704 len=1786544 2024-04-07T16:55:02.487 INFO:tasks.workunit.client.0.smithi100.stdout:op 4946 completed, throughput=5MB/sec 2024-04-07T16:55:02.487 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-tgGB6Y0D_J-nO_q off=7753186 len=2020150 2024-04-07T16:55:02.504 INFO:tasks.workunit.client.0.smithi100.stdout:op 4947 completed, throughput=5MB/sec 2024-04-07T16:55:02.612 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:55:02 smithi144 bash[95233]: ts=2024-04-07T16:55:02.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:55:02.949 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:55:02 smithi100 bash[74942]: cluster 2024-04-07T16:55:01.573422+0000 mgr.y (mgr.25175) 1190 : cluster [DBG] pgmap v1832: 105 pgs: 105 active+clean; 126 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 2.1 MiB/s rd, 3.0 MiB/s wr, 5 op/s 2024-04-07T16:55:02.949 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:55:02 smithi100 bash[74942]: audit 2024-04-07T16:55:01.784551+0000 mon.a (mon.0) 2184 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:55:02.949 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:55:02 smithi100 bash[76202]: cluster 2024-04-07T16:55:01.573422+0000 mgr.y (mgr.25175) 1190 : cluster [DBG] pgmap v1832: 105 pgs: 105 active+clean; 126 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 2.1 MiB/s rd, 3.0 MiB/s wr, 5 op/s 2024-04-07T16:55:02.949 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:55:02 smithi100 bash[76202]: audit 2024-04-07T16:55:01.784551+0000 mon.a (mon.0) 2184 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:55:03.112 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:55:02 smithi144 bash[66654]: cluster 2024-04-07T16:55:01.573422+0000 mgr.y (mgr.25175) 1190 : cluster [DBG] pgmap v1832: 105 pgs: 105 active+clean; 126 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 2.1 MiB/s rd, 3.0 MiB/s wr, 5 op/s 2024-04-07T16:55:03.113 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:55:02 smithi144 bash[66654]: audit 2024-04-07T16:55:01.784551+0000 mon.a (mon.0) 2184 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:55:03.504 INFO:tasks.workunit.client.0.smithi100.stdout: 955: throughput=5MB/sec pending data=0 2024-04-07T16:55:03.504 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-fJizwU5IgpnU7lG off=6030670 len=790589 2024-04-07T16:55:03.507 INFO:tasks.workunit.client.0.smithi100.stdout:op 4948 completed, throughput=5MB/sec 2024-04-07T16:55:03.507 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-H7uUYc3o-u5a_PX off=4932297 len=1702474 2024-04-07T16:55:03.511 INFO:tasks.workunit.client.0.smithi100.stdout:op 4949 completed, throughput=5MB/sec 2024-04-07T16:55:03.511 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-BccTIaVJK_mlcWy off=8443044 len=1600416 2024-04-07T16:55:03.515 INFO:tasks.workunit.client.0.smithi100.stdout:op 4950 completed, throughput=5MB/sec 2024-04-07T16:55:03.516 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-mOxXfGLK9o8C4jq off=16493462 len=213104 2024-04-07T16:55:03.517 INFO:tasks.workunit.client.0.smithi100.stdout:op 4951 completed, throughput=5MB/sec 2024-04-07T16:55:03.517 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-6BPBM0hqw-2is6t off=20902578 len=2081438 2024-04-07T16:55:03.532 INFO:tasks.workunit.client.0.smithi100.stdout:op 4952 completed, throughput=5MB/sec 2024-04-07T16:55:04.532 INFO:tasks.workunit.client.0.smithi100.stdout: 956: throughput=5MB/sec pending data=0 2024-04-07T16:55:04.532 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Xa_egBgoeuauJ87 off=6635395 len=1197459 2024-04-07T16:55:04.535 INFO:tasks.workunit.client.0.smithi100.stdout:op 4953 completed, throughput=5MB/sec 2024-04-07T16:55:04.535 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-RNCEpr0HeC-Dj09 off=4425762 len=138714 2024-04-07T16:55:04.537 INFO:tasks.workunit.client.0.smithi100.stdout:op 4954 completed, throughput=5MB/sec 2024-04-07T16:55:04.537 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-7gmZv4zqhRjouS3 off=9768985 len=1716919 2024-04-07T16:55:04.541 INFO:tasks.workunit.client.0.smithi100.stdout:op 4955 completed, throughput=5MB/sec 2024-04-07T16:55:04.541 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Yt_vtbCd2kU_uP3 off=5828081 len=1245659 2024-04-07T16:55:04.543 INFO:tasks.workunit.client.0.smithi100.stdout:op 4956 completed, throughput=5MB/sec 2024-04-07T16:55:04.949 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:55:04 smithi100 bash[74942]: cluster 2024-04-07T16:55:03.574659+0000 mgr.y (mgr.25175) 1191 : cluster [DBG] pgmap v1833: 105 pgs: 105 active+clean; 126 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 3.1 MiB/s rd, 3.0 MiB/s wr, 6 op/s 2024-04-07T16:55:04.949 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:55:04 smithi100 bash[76202]: cluster 2024-04-07T16:55:03.574659+0000 mgr.y (mgr.25175) 1191 : cluster [DBG] pgmap v1833: 105 pgs: 105 active+clean; 126 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 3.1 MiB/s rd, 3.0 MiB/s wr, 6 op/s 2024-04-07T16:55:05.113 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:55:04 smithi144 bash[66654]: cluster 2024-04-07T16:55:03.574659+0000 mgr.y (mgr.25175) 1191 : cluster [DBG] pgmap v1833: 105 pgs: 105 active+clean; 126 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 3.1 MiB/s rd, 3.0 MiB/s wr, 6 op/s 2024-04-07T16:55:05.550 INFO:tasks.workunit.client.0.smithi100.stdout: 957: throughput=5MB/sec pending data=0 2024-04-07T16:55:05.550 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Gl3T4I3BHSE6ywP off=6537363 len=1285032 2024-04-07T16:55:05.553 INFO:tasks.workunit.client.0.smithi100.stdout:op 4957 completed, throughput=5MB/sec 2024-04-07T16:55:05.554 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-ZMBYOexC5rItzeS off=6574080 len=739584 2024-04-07T16:55:05.554 INFO:tasks.workunit.client.0.smithi100.stdout:op 4958 completed, throughput=5MB/sec 2024-04-07T16:55:05.554 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-2DklIDumv2tSwzB off=6543162 len=1301338 2024-04-07T16:55:05.563 INFO:tasks.workunit.client.0.smithi100.stdout:op 4959 completed, throughput=5MB/sec 2024-04-07T16:55:05.563 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-VQMG6WK9G0ewEzl off=7869161 len=167166 2024-04-07T16:55:05.564 INFO:tasks.workunit.client.0.smithi100.stdout:op 4960 completed, throughput=5MB/sec 2024-04-07T16:55:05.564 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-jhiRBowsDd6KxhK off=3225499 len=737055 2024-04-07T16:55:05.566 INFO:tasks.workunit.client.0.smithi100.stdout:op 4961 completed, throughput=5MB/sec 2024-04-07T16:55:05.567 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-lCorZ7DLB4WOM8a off=2563274 len=1631394 2024-04-07T16:55:05.571 INFO:tasks.workunit.client.0.smithi100.stdout:op 4962 completed, throughput=5MB/sec 2024-04-07T16:55:06.571 INFO:tasks.workunit.client.0.smithi100.stdout: 958: throughput=5MB/sec pending data=0 2024-04-07T16:55:06.571 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-h5SrD1pCNHO_eS5 off=2982653 len=1855995 2024-04-07T16:55:06.576 INFO:tasks.workunit.client.0.smithi100.stdout:op 4963 completed, throughput=5MB/sec 2024-04-07T16:55:06.577 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-HvANC4NutB2-xaN off=95804 len=1642163 2024-04-07T16:55:06.581 INFO:tasks.workunit.client.0.smithi100.stdout:op 4964 completed, throughput=5MB/sec 2024-04-07T16:55:06.581 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-3POXh9N6Eq0068F off=2160777 len=790458 2024-04-07T16:55:06.588 INFO:tasks.workunit.client.0.smithi100.stdout:op 4965 completed, throughput=5MB/sec 2024-04-07T16:55:06.589 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-IkhaWT6nEhAtKW0 off=3961675 len=1488557 2024-04-07T16:55:06.592 INFO:tasks.workunit.client.0.smithi100.stdout:op 4966 completed, throughput=5MB/sec 2024-04-07T16:55:06.949 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:55:06 smithi100 bash[74942]: cluster 2024-04-07T16:55:05.575685+0000 mgr.y (mgr.25175) 1192 : cluster [DBG] pgmap v1834: 105 pgs: 105 active+clean; 126 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 3.5 MiB/s rd, 2.3 MiB/s wr, 5 op/s 2024-04-07T16:55:06.949 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:55:06 smithi100 bash[76202]: cluster 2024-04-07T16:55:05.575685+0000 mgr.y (mgr.25175) 1192 : cluster [DBG] pgmap v1834: 105 pgs: 105 active+clean; 126 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 3.5 MiB/s rd, 2.3 MiB/s wr, 5 op/s 2024-04-07T16:55:07.112 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:55:06 smithi144 bash[66654]: cluster 2024-04-07T16:55:05.575685+0000 mgr.y (mgr.25175) 1192 : cluster [DBG] pgmap v1834: 105 pgs: 105 active+clean; 126 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 3.5 MiB/s rd, 2.3 MiB/s wr, 5 op/s 2024-04-07T16:55:07.592 INFO:tasks.workunit.client.0.smithi100.stdout: 959: throughput=5MB/sec pending data=0 2024-04-07T16:55:07.592 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-J_U_5BNXDZfxJxI off=2216877 len=1605562 2024-04-07T16:55:07.606 INFO:tasks.workunit.client.0.smithi100.stdout:op 4967 completed, throughput=5MB/sec 2024-04-07T16:55:07.606 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-zkCaximuaysMgdO off=6575424 len=675539 2024-04-07T16:55:07.613 INFO:tasks.workunit.client.0.smithi100.stdout:op 4968 completed, throughput=5MB/sec 2024-04-07T16:55:07.613 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-yco2ojnDPlUFqGd off=3474836 len=88784 2024-04-07T16:55:07.615 INFO:tasks.workunit.client.0.smithi100.stdout:op 4969 completed, throughput=5MB/sec 2024-04-07T16:55:07.615 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-fhiAHJynl9VyiQF off=11442705 len=1697876 2024-04-07T16:55:07.621 INFO:tasks.workunit.client.0.smithi100.stdout:op 4970 completed, throughput=5MB/sec 2024-04-07T16:55:07.621 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj--JFUUnIjDUXkY40 off=3496092 len=2065393 2024-04-07T16:55:07.627 INFO:tasks.workunit.client.0.smithi100.stdout:op 4971 completed, throughput=5MB/sec 2024-04-07T16:55:08.627 INFO:tasks.workunit.client.0.smithi100.stdout: 960: throughput=5MB/sec pending data=0 2024-04-07T16:55:08.627 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-IuNnb4s4WF5onIc off=1673871 len=77752 2024-04-07T16:55:08.632 INFO:tasks.workunit.client.0.smithi100.stdout:op 4972 completed, throughput=5MB/sec 2024-04-07T16:55:08.632 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-I1o9QyEx9JNr6h3 off=13056932 len=30417 2024-04-07T16:55:08.636 INFO:tasks.workunit.client.0.smithi100.stdout:op 4973 completed, throughput=5MB/sec 2024-04-07T16:55:08.636 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-NdEVAlyaSDl_I9E off=4676597 len=1335560 2024-04-07T16:55:08.639 INFO:tasks.workunit.client.0.smithi100.stdout:op 4974 completed, throughput=5MB/sec 2024-04-07T16:55:08.639 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-URwzNCeNuZXff9J off=10820547 len=203599 2024-04-07T16:55:08.641 INFO:tasks.workunit.client.0.smithi100.stdout:op 4975 completed, throughput=5MB/sec 2024-04-07T16:55:08.641 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-OMRB3Ck5rdgOci5 off=5226085 len=620850 2024-04-07T16:55:08.643 INFO:tasks.workunit.client.0.smithi100.stdout:op 4976 completed, throughput=5MB/sec 2024-04-07T16:55:08.643 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-oG6ZlJ-5DBXetZ0 off=776834 len=957292 2024-04-07T16:55:08.645 INFO:tasks.workunit.client.0.smithi100.stdout:op 4977 completed, throughput=5MB/sec 2024-04-07T16:55:08.645 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-igpFb_08cLYoL8g off=22234275 len=1603050 2024-04-07T16:55:08.649 INFO:tasks.workunit.client.0.smithi100.stdout:op 4978 completed, throughput=5MB/sec 2024-04-07T16:55:08.699 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:55:08 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:55:08] "GET /metrics HTTP/1.1" 200 34069 "" "Prometheus/2.43.0" 2024-04-07T16:55:09.112 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:55:08 smithi144 bash[66654]: cluster 2024-04-07T16:55:07.576679+0000 mgr.y (mgr.25175) 1193 : cluster [DBG] pgmap v1835: 105 pgs: 105 active+clean; 126 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 3.1 MiB/s rd, 2.2 MiB/s wr, 5 op/s 2024-04-07T16:55:09.112 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:55:08 smithi144 bash[66654]: audit 2024-04-07T16:55:08.207700+0000 mon.a (mon.0) 2185 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:55:09.199 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:55:08 smithi100 bash[74942]: cluster 2024-04-07T16:55:07.576679+0000 mgr.y (mgr.25175) 1193 : cluster [DBG] pgmap v1835: 105 pgs: 105 active+clean; 126 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 3.1 MiB/s rd, 2.2 MiB/s wr, 5 op/s 2024-04-07T16:55:09.199 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:55:08 smithi100 bash[74942]: audit 2024-04-07T16:55:08.207700+0000 mon.a (mon.0) 2185 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:55:09.199 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:55:08 smithi100 bash[76202]: cluster 2024-04-07T16:55:07.576679+0000 mgr.y (mgr.25175) 1193 : cluster [DBG] pgmap v1835: 105 pgs: 105 active+clean; 126 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 3.1 MiB/s rd, 2.2 MiB/s wr, 5 op/s 2024-04-07T16:55:09.199 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:55:08 smithi100 bash[76202]: audit 2024-04-07T16:55:08.207700+0000 mon.a (mon.0) 2185 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:55:09.649 INFO:tasks.workunit.client.0.smithi100.stdout: 961: throughput=5MB/sec pending data=0 2024-04-07T16:55:09.649 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-9Mh9AyIvCOnPLAV off=11319108 len=1994092 2024-04-07T16:55:09.667 INFO:tasks.workunit.client.0.smithi100.stdout:op 4979 completed, throughput=5MB/sec 2024-04-07T16:55:09.668 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj--cHlabAvLtFe3LL off=3588410 len=476597 2024-04-07T16:55:09.669 INFO:tasks.workunit.client.0.smithi100.stdout:op 4980 completed, throughput=5MB/sec 2024-04-07T16:55:09.669 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-23ySYIZPxZbdET5 off=18892540 len=238846 2024-04-07T16:55:09.675 INFO:tasks.workunit.client.0.smithi100.stdout:op 4981 completed, throughput=5MB/sec 2024-04-07T16:55:09.675 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-8Y0SQ_DEba8A12T off=9600846 len=1240801 2024-04-07T16:55:09.685 INFO:tasks.workunit.client.0.smithi100.stdout:op 4982 completed, throughput=5MB/sec 2024-04-07T16:55:09.686 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-ZO2IHlEdQKYkh_9 off=10802386 len=402010 2024-04-07T16:55:09.689 INFO:tasks.workunit.client.0.smithi100.stdout:op 4983 completed, throughput=5MB/sec 2024-04-07T16:55:10.113 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:55:09 smithi144 bash[66654]: audit 2024-04-07T16:55:08.786997+0000 mon.a (mon.0) 2186 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:55:10.113 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:55:09 smithi144 bash[66654]: audit 2024-04-07T16:55:08.796890+0000 mon.a (mon.0) 2187 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:55:10.113 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:55:09 smithi144 bash[66654]: audit 2024-04-07T16:55:09.439052+0000 mon.a (mon.0) 2188 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:55:10.113 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:55:09 smithi144 bash[66654]: audit 2024-04-07T16:55:09.440814+0000 mon.a (mon.0) 2189 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:55:10.113 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:55:09 smithi144 bash[66654]: audit 2024-04-07T16:55:09.450225+0000 mon.a (mon.0) 2190 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:55:10.113 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:55:09 smithi144 bash[66654]: cluster 2024-04-07T16:55:09.577716+0000 mgr.y (mgr.25175) 1194 : cluster [DBG] pgmap v1836: 105 pgs: 105 active+clean; 126 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 3.2 MiB/s rd, 1.9 MiB/s wr, 4 op/s 2024-04-07T16:55:10.199 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:55:09 smithi100 bash[74942]: audit 2024-04-07T16:55:08.786997+0000 mon.a (mon.0) 2186 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:55:10.199 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:55:09 smithi100 bash[74942]: audit 2024-04-07T16:55:08.796890+0000 mon.a (mon.0) 2187 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:55:10.199 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:55:09 smithi100 bash[74942]: audit 2024-04-07T16:55:09.439052+0000 mon.a (mon.0) 2188 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:55:10.199 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:55:09 smithi100 bash[74942]: audit 2024-04-07T16:55:09.440814+0000 mon.a (mon.0) 2189 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:55:10.199 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:55:09 smithi100 bash[74942]: audit 2024-04-07T16:55:09.450225+0000 mon.a (mon.0) 2190 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:55:10.199 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:55:09 smithi100 bash[74942]: cluster 2024-04-07T16:55:09.577716+0000 mgr.y (mgr.25175) 1194 : cluster [DBG] pgmap v1836: 105 pgs: 105 active+clean; 126 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 3.2 MiB/s rd, 1.9 MiB/s wr, 4 op/s 2024-04-07T16:55:10.200 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:55:09 smithi100 bash[76202]: audit 2024-04-07T16:55:08.786997+0000 mon.a (mon.0) 2186 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:55:10.200 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:55:09 smithi100 bash[76202]: audit 2024-04-07T16:55:08.796890+0000 mon.a (mon.0) 2187 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:55:10.200 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:55:09 smithi100 bash[76202]: audit 2024-04-07T16:55:09.439052+0000 mon.a (mon.0) 2188 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:55:10.200 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:55:09 smithi100 bash[76202]: audit 2024-04-07T16:55:09.440814+0000 mon.a (mon.0) 2189 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:55:10.200 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:55:09 smithi100 bash[76202]: audit 2024-04-07T16:55:09.450225+0000 mon.a (mon.0) 2190 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:55:10.200 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:55:09 smithi100 bash[76202]: cluster 2024-04-07T16:55:09.577716+0000 mgr.y (mgr.25175) 1194 : cluster [DBG] pgmap v1836: 105 pgs: 105 active+clean; 126 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 3.2 MiB/s rd, 1.9 MiB/s wr, 4 op/s 2024-04-07T16:55:10.687 INFO:tasks.workunit.client.0.smithi100.stdout: 962: throughput=5MB/sec pending data=0 2024-04-07T16:55:10.687 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-gmqCQlH_zCk6wyC off=1345202 len=188203 2024-04-07T16:55:10.688 INFO:tasks.workunit.client.0.smithi100.stdout:op 4984 completed, throughput=5MB/sec 2024-04-07T16:55:10.688 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-3o73Tr9B-BuQvsv off=101832 len=670837 2024-04-07T16:55:10.691 INFO:tasks.workunit.client.0.smithi100.stdout:op 4985 completed, throughput=5MB/sec 2024-04-07T16:55:10.691 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-OI8OjvTFpQIwMO2 off=9304003 len=1347179 2024-04-07T16:55:10.703 INFO:tasks.workunit.client.0.smithi100.stdout:op 4986 completed, throughput=5MB/sec 2024-04-07T16:55:10.703 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Qt1Wx4OQ5T54WyN off=3793169 len=1890353 2024-04-07T16:55:10.707 INFO:tasks.workunit.client.0.smithi100.stdout:op 4987 completed, throughput=5MB/sec 2024-04-07T16:55:10.707 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-rc1GdsgSTGJja0W off=1853899 len=636990 2024-04-07T16:55:10.709 INFO:tasks.workunit.client.0.smithi100.stdout:op 4988 completed, throughput=5MB/sec 2024-04-07T16:55:10.709 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Sj9HNPAhn2MyVMB off=2881747 len=1322637 2024-04-07T16:55:10.720 INFO:tasks.workunit.client.0.smithi100.stdout:op 4989 completed, throughput=5MB/sec 2024-04-07T16:55:11.720 INFO:tasks.workunit.client.0.smithi100.stdout: 963: throughput=5MB/sec pending data=0 2024-04-07T16:55:11.720 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-AaxWvIqzoiDM30b off=1181723 len=676210 2024-04-07T16:55:11.723 INFO:tasks.workunit.client.0.smithi100.stdout:op 4990 completed, throughput=5MB/sec 2024-04-07T16:55:11.723 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Qnx_I4L45o6bSOg off=10141113 len=3842 2024-04-07T16:55:11.724 INFO:tasks.workunit.client.0.smithi100.stdout:op 4991 completed, throughput=5MB/sec 2024-04-07T16:55:11.725 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-nF96FCcjE8ptP4O off=0 len=1126341 2024-04-07T16:55:11.729 INFO:tasks.workunit.client.0.smithi100.stdout:op 4992 completed, throughput=5MB/sec 2024-04-07T16:55:11.729 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-QPB9AIGzSw7Xugu off=9902860 len=1094278 2024-04-07T16:55:11.730 INFO:tasks.workunit.client.0.smithi100.stdout:op 4993 completed, throughput=5MB/sec 2024-04-07T16:55:11.730 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-_PB_IBl_ttREe_p off=20132663 len=1169299 2024-04-07T16:55:11.734 INFO:tasks.workunit.client.0.smithi100.stdout:op 4994 completed, throughput=5MB/sec 2024-04-07T16:55:11.734 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-aeot9X3OnTa69dD off=13401219 len=248800 2024-04-07T16:55:11.737 INFO:tasks.workunit.client.0.smithi100.stdout:op 4995 completed, throughput=5MB/sec 2024-04-07T16:55:11.737 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-feFJAdJtH9pY3vn off=3427946 len=1343552 2024-04-07T16:55:11.740 INFO:tasks.workunit.client.0.smithi100.stdout:op 4996 completed, throughput=5MB/sec 2024-04-07T16:55:12.613 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:55:12 smithi144 bash[95233]: ts=2024-04-07T16:55:12.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:55:12.740 INFO:tasks.workunit.client.0.smithi100.stdout: 964: throughput=5MB/sec pending data=0 2024-04-07T16:55:12.740 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-EznaNp24JxNPLQR off=414319 len=1265159 2024-04-07T16:55:12.754 INFO:tasks.workunit.client.0.smithi100.stdout:op 4997 completed, throughput=5MB/sec 2024-04-07T16:55:12.754 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-q1EMFCMEA9uuJMX off=533616 len=571669 2024-04-07T16:55:12.759 INFO:tasks.workunit.client.0.smithi100.stdout:op 4998 completed, throughput=5MB/sec 2024-04-07T16:55:12.759 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-WWeyQgkYZuqbE8N off=20019635 len=1205383 2024-04-07T16:55:12.762 INFO:tasks.workunit.client.0.smithi100.stdout:op 4999 completed, throughput=5MB/sec 2024-04-07T16:55:12.762 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Fd8vf853oy8wZ8X off=8923999 len=214999 2024-04-07T16:55:12.764 INFO:tasks.workunit.client.0.smithi100.stdout:op 5000 completed, throughput=5MB/sec 2024-04-07T16:55:12.764 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-1A-Nn5vO9s61xOT off=1639961 len=1249581 2024-04-07T16:55:12.775 INFO:tasks.workunit.client.0.smithi100.stdout:op 5001 completed, throughput=5MB/sec 2024-04-07T16:55:12.949 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:55:12 smithi100 bash[74942]: cluster 2024-04-07T16:55:11.578500+0000 mgr.y (mgr.25175) 1195 : cluster [DBG] pgmap v1837: 105 pgs: 105 active+clean; 126 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 3.2 MiB/s rd, 2.1 MiB/s wr, 5 op/s 2024-04-07T16:55:12.949 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:55:12 smithi100 bash[76202]: cluster 2024-04-07T16:55:11.578500+0000 mgr.y (mgr.25175) 1195 : cluster [DBG] pgmap v1837: 105 pgs: 105 active+clean; 126 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 3.2 MiB/s rd, 2.1 MiB/s wr, 5 op/s 2024-04-07T16:55:13.112 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:55:12 smithi144 bash[66654]: cluster 2024-04-07T16:55:11.578500+0000 mgr.y (mgr.25175) 1195 : cluster [DBG] pgmap v1837: 105 pgs: 105 active+clean; 126 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 3.2 MiB/s rd, 2.1 MiB/s wr, 5 op/s 2024-04-07T16:55:13.777 INFO:tasks.workunit.client.0.smithi100.stdout: 965: throughput=5MB/sec pending data=0 2024-04-07T16:55:13.778 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-B9awlVbmgHnWvnZ off=13455334 len=300581 2024-04-07T16:55:13.781 INFO:tasks.workunit.client.0.smithi100.stdout:op 5002 completed, throughput=5MB/sec 2024-04-07T16:55:13.781 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-NKipEBb2d0wspA- off=114993 len=1786235 2024-04-07T16:55:13.786 INFO:tasks.workunit.client.0.smithi100.stdout:op 5003 completed, throughput=5MB/sec 2024-04-07T16:55:13.786 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj--R75EVj8e3WXzCQ off=10354556 len=1535421 2024-04-07T16:55:13.790 INFO:tasks.workunit.client.0.smithi100.stdout:op 5004 completed, throughput=5MB/sec 2024-04-07T16:55:13.790 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-1McpQN1fAEszle4 off=3404078 len=1842358 2024-04-07T16:55:13.805 INFO:tasks.workunit.client.0.smithi100.stdout:op 5005 completed, throughput=5MB/sec 2024-04-07T16:55:14.806 INFO:tasks.workunit.client.0.smithi100.stdout: 966: throughput=5MB/sec pending data=0 2024-04-07T16:55:14.806 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-mMxgNmcPWJYhlJO off=3688141 len=1833213 2024-04-07T16:55:14.822 INFO:tasks.workunit.client.0.smithi100.stdout:op 5006 completed, throughput=5MB/sec 2024-04-07T16:55:14.822 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-pPBIdN3mcnkYCyH off=6084274 len=440288 2024-04-07T16:55:14.829 INFO:tasks.workunit.client.0.smithi100.stdout:op 5007 completed, throughput=5MB/sec 2024-04-07T16:55:14.829 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Mfl8YMJJITMnMJA off=172373 len=885953 2024-04-07T16:55:14.837 INFO:tasks.workunit.client.0.smithi100.stdout:op 5008 completed, throughput=5MB/sec 2024-04-07T16:55:14.837 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-KTTBiQ7Uo_2AjCH off=255786 len=1202148 2024-04-07T16:55:14.847 INFO:tasks.workunit.client.0.smithi100.stdout:op 5009 completed, throughput=5MB/sec 2024-04-07T16:55:14.848 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-lp14Lf1jWizfMmm off=234711 len=1586158 2024-04-07T16:55:14.850 INFO:tasks.workunit.client.0.smithi100.stdout:op 5010 completed, throughput=5MB/sec 2024-04-07T16:55:14.949 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:55:14 smithi100 bash[74942]: cluster 2024-04-07T16:55:13.579653+0000 mgr.y (mgr.25175) 1196 : cluster [DBG] pgmap v1838: 105 pgs: 105 active+clean; 126 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 3.9 MiB/s rd, 1.6 MiB/s wr, 5 op/s 2024-04-07T16:55:14.949 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:55:14 smithi100 bash[76202]: cluster 2024-04-07T16:55:13.579653+0000 mgr.y (mgr.25175) 1196 : cluster [DBG] pgmap v1838: 105 pgs: 105 active+clean; 126 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 3.9 MiB/s rd, 1.6 MiB/s wr, 5 op/s 2024-04-07T16:55:15.112 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:55:14 smithi144 bash[66654]: cluster 2024-04-07T16:55:13.579653+0000 mgr.y (mgr.25175) 1196 : cluster [DBG] pgmap v1838: 105 pgs: 105 active+clean; 126 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 3.9 MiB/s rd, 1.6 MiB/s wr, 5 op/s 2024-04-07T16:55:15.851 INFO:tasks.workunit.client.0.smithi100.stdout: 967: throughput=5MB/sec pending data=0 2024-04-07T16:55:15.851 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-b1l3BwmSfC4Xib5 off=110436 len=1721740 2024-04-07T16:55:15.865 INFO:tasks.workunit.client.0.smithi100.stdout:op 5011 completed, throughput=5MB/sec 2024-04-07T16:55:15.865 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-8JfAySLvvJo7bk7 off=4044154 len=211546 2024-04-07T16:55:15.871 INFO:tasks.workunit.client.0.smithi100.stdout:op 5012 completed, throughput=5MB/sec 2024-04-07T16:55:15.871 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj--O0tWYyKZj3jFtl off=855770 len=275562 2024-04-07T16:55:15.872 INFO:tasks.workunit.client.0.smithi100.stdout:op 5013 completed, throughput=5MB/sec 2024-04-07T16:55:15.872 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-HWamIFsRT8u2Ksn off=6901344 len=927753 2024-04-07T16:55:15.875 INFO:tasks.workunit.client.0.smithi100.stdout:op 5014 completed, throughput=5MB/sec 2024-04-07T16:55:15.875 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-FNk6u7pEaWvN6D0 off=13050402 len=194996 2024-04-07T16:55:15.876 INFO:tasks.workunit.client.0.smithi100.stdout:op 5015 completed, throughput=5MB/sec 2024-04-07T16:55:15.876 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-uD6EpyVuCNYOJk3 off=18664067 len=1524435 2024-04-07T16:55:15.882 INFO:tasks.workunit.client.0.smithi100.stdout:op 5016 completed, throughput=5MB/sec 2024-04-07T16:55:16.882 INFO:tasks.workunit.client.0.smithi100.stdout: 968: throughput=5MB/sec pending data=0 2024-04-07T16:55:16.883 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-alvUtSY9iTTIRu5 off=0 len=1509312 2024-04-07T16:55:16.887 INFO:tasks.workunit.client.0.smithi100.stdout:op 5017 completed, throughput=5MB/sec 2024-04-07T16:55:16.887 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-dvQd_9lO-VwKMCw off=4635219 len=939315 2024-04-07T16:55:16.890 INFO:tasks.workunit.client.0.smithi100.stdout:op 5018 completed, throughput=5MB/sec 2024-04-07T16:55:16.890 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-hVNLGrWz2476foF off=2256652 len=1261925 2024-04-07T16:55:16.894 INFO:tasks.workunit.client.0.smithi100.stdout:op 5019 completed, throughput=5MB/sec 2024-04-07T16:55:16.894 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-_DfR9b-zpKIu-Jz off=5053059 len=1940611 2024-04-07T16:55:16.911 INFO:tasks.workunit.client.0.smithi100.stdout:op 5020 completed, throughput=5MB/sec 2024-04-07T16:55:16.949 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:55:16 smithi100 bash[74942]: cluster 2024-04-07T16:55:15.580657+0000 mgr.y (mgr.25175) 1197 : cluster [DBG] pgmap v1839: 105 pgs: 105 active+clean; 126 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 3.5 MiB/s rd, 1.3 MiB/s wr, 4 op/s 2024-04-07T16:55:16.949 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:55:16 smithi100 bash[76202]: cluster 2024-04-07T16:55:15.580657+0000 mgr.y (mgr.25175) 1197 : cluster [DBG] pgmap v1839: 105 pgs: 105 active+clean; 126 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 3.5 MiB/s rd, 1.3 MiB/s wr, 4 op/s 2024-04-07T16:55:17.112 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:55:16 smithi144 bash[66654]: cluster 2024-04-07T16:55:15.580657+0000 mgr.y (mgr.25175) 1197 : cluster [DBG] pgmap v1839: 105 pgs: 105 active+clean; 126 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 3.5 MiB/s rd, 1.3 MiB/s wr, 4 op/s 2024-04-07T16:55:17.912 INFO:tasks.workunit.client.0.smithi100.stdout: 969: throughput=5MB/sec pending data=0 2024-04-07T16:55:17.912 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-WZQ8rN4V-sl4GXX off=11299746 len=1160279 2024-04-07T16:55:17.915 INFO:tasks.workunit.client.0.smithi100.stdout:op 5021 completed, throughput=5MB/sec 2024-04-07T16:55:17.915 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-sc1KV_9g3vJiUeu off=10442613 len=1567319 2024-04-07T16:55:17.929 INFO:tasks.workunit.client.0.smithi100.stdout:op 5022 completed, throughput=5MB/sec 2024-04-07T16:55:17.930 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-4XmKuyBlQvXHlXJ off=7495782 len=974036 2024-04-07T16:55:17.939 INFO:tasks.workunit.client.0.smithi100.stdout:op 5023 completed, throughput=5MB/sec 2024-04-07T16:55:17.939 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-KRPsieumtVHuKom off=1172790 len=1488143 2024-04-07T16:55:17.942 INFO:tasks.workunit.client.0.smithi100.stdout:op 5024 completed, throughput=5MB/sec 2024-04-07T16:55:17.949 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:55:17 smithi100 bash[74942]: audit 2024-04-07T16:55:16.784895+0000 mon.a (mon.0) 2191 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:55:17.949 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:55:17 smithi100 bash[76202]: audit 2024-04-07T16:55:16.784895+0000 mon.a (mon.0) 2191 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:55:18.112 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:55:17 smithi144 bash[66654]: audit 2024-04-07T16:55:16.784895+0000 mon.a (mon.0) 2191 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:55:18.699 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:55:18 smithi100 bash[74942]: cluster 2024-04-07T16:55:17.581788+0000 mgr.y (mgr.25175) 1198 : cluster [DBG] pgmap v1840: 105 pgs: 105 active+clean; 126 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 3.1 MiB/s rd, 1.7 MiB/s wr, 4 op/s 2024-04-07T16:55:18.699 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:55:18 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:55:18] "GET /metrics HTTP/1.1" 200 34073 "" "Prometheus/2.43.0" 2024-04-07T16:55:18.699 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:55:18 smithi100 bash[76202]: cluster 2024-04-07T16:55:17.581788+0000 mgr.y (mgr.25175) 1198 : cluster [DBG] pgmap v1840: 105 pgs: 105 active+clean; 126 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 3.1 MiB/s rd, 1.7 MiB/s wr, 4 op/s 2024-04-07T16:55:18.943 INFO:tasks.workunit.client.0.smithi100.stdout: 970: throughput=5MB/sec pending data=0 2024-04-07T16:55:18.943 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-jIapnkiq9eb9LaM off=95183 len=437250 2024-04-07T16:55:18.945 INFO:tasks.workunit.client.0.smithi100.stdout:op 5025 completed, throughput=5MB/sec 2024-04-07T16:55:18.945 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-ILxCq7wk_k4flQi off=16593841 len=1023329 2024-04-07T16:55:18.955 INFO:tasks.workunit.client.0.smithi100.stdout:op 5026 completed, throughput=5MB/sec 2024-04-07T16:55:18.955 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-zcY9f1HYcZkgG5G off=1363588 len=1171231 2024-04-07T16:55:18.966 INFO:tasks.workunit.client.0.smithi100.stdout:op 5027 completed, throughput=5MB/sec 2024-04-07T16:55:18.966 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-bVWIxZj-YOc79zx off=13866195 len=407050 2024-04-07T16:55:18.972 INFO:tasks.workunit.client.0.smithi100.stdout:op 5028 completed, throughput=5MB/sec 2024-04-07T16:55:18.972 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-ADd3Mafky4kEcOv off=3165450 len=1532864 2024-04-07T16:55:18.985 INFO:tasks.workunit.client.0.smithi100.stdout:op 5029 completed, throughput=5MB/sec 2024-04-07T16:55:18.985 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-OZlMYSrUDwTRbju off=583569 len=544204 2024-04-07T16:55:18.987 INFO:tasks.workunit.client.0.smithi100.stdout:op 5030 completed, throughput=5MB/sec 2024-04-07T16:55:19.112 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:55:18 smithi144 bash[66654]: cluster 2024-04-07T16:55:17.581788+0000 mgr.y (mgr.25175) 1198 : cluster [DBG] pgmap v1840: 105 pgs: 105 active+clean; 126 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 3.1 MiB/s rd, 1.7 MiB/s wr, 4 op/s 2024-04-07T16:55:19.988 INFO:tasks.workunit.client.0.smithi100.stdout: 971: throughput=5MB/sec pending data=0 2024-04-07T16:55:19.988 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-NN1xtWjYbpKfYcS off=11662031 len=833000 2024-04-07T16:55:19.990 INFO:tasks.workunit.client.0.smithi100.stdout:op 5031 completed, throughput=5MB/sec 2024-04-07T16:55:19.990 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-yfRND_XRbXaWSJw off=168879 len=993644 2024-04-07T16:55:19.994 INFO:tasks.workunit.client.0.smithi100.stdout:op 5032 completed, throughput=5MB/sec 2024-04-07T16:55:19.994 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Snw4qucB8nXx2gC off=475348 len=407441 2024-04-07T16:55:19.996 INFO:tasks.workunit.client.0.smithi100.stdout:op 5033 completed, throughput=5MB/sec 2024-04-07T16:55:19.996 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-RNCEpr0HeC-Dj09 off=66208 len=1437225 2024-04-07T16:55:19.999 INFO:tasks.workunit.client.0.smithi100.stdout:op 5034 completed, throughput=5MB/sec 2024-04-07T16:55:19.999 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj--lOM1sjQ-TtGlZN off=2713100 len=1320072 2024-04-07T16:55:20.004 INFO:tasks.workunit.client.0.smithi100.stdout:op 5035 completed, throughput=5MB/sec 2024-04-07T16:55:20.004 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-i_Ev96MAzndcYgD off=422766 len=2035290 2024-04-07T16:55:20.009 INFO:tasks.workunit.client.0.smithi100.stdout:op 5036 completed, throughput=5MB/sec 2024-04-07T16:55:20.949 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:55:20 smithi100 bash[74942]: cluster 2024-04-07T16:55:19.582743+0000 mgr.y (mgr.25175) 1199 : cluster [DBG] pgmap v1841: 105 pgs: 105 active+clean; 126 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 3.0 MiB/s rd, 1.7 MiB/s wr, 4 op/s 2024-04-07T16:55:20.949 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:55:20 smithi100 bash[76202]: cluster 2024-04-07T16:55:19.582743+0000 mgr.y (mgr.25175) 1199 : cluster [DBG] pgmap v1841: 105 pgs: 105 active+clean; 126 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 3.0 MiB/s rd, 1.7 MiB/s wr, 4 op/s 2024-04-07T16:55:21.009 INFO:tasks.workunit.client.0.smithi100.stdout: 972: throughput=5MB/sec pending data=0 2024-04-07T16:55:21.010 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-DZvO855G21DIiUg off=12996401 len=1540999 2024-04-07T16:55:21.015 INFO:tasks.workunit.client.0.smithi100.stdout:op 5037 completed, throughput=5MB/sec 2024-04-07T16:55:21.015 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-2RGK-5bkkK33SgO off=13424741 len=261158 2024-04-07T16:55:21.017 INFO:tasks.workunit.client.0.smithi100.stdout:op 5038 completed, throughput=5MB/sec 2024-04-07T16:55:21.017 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-mwFHGBpd6izXkLg off=6432668 len=1541576 2024-04-07T16:55:21.021 INFO:tasks.workunit.client.0.smithi100.stdout:op 5039 completed, throughput=5MB/sec 2024-04-07T16:55:21.021 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-HOLYcd3ZLSZ9tcT off=6995440 len=341594 2024-04-07T16:55:21.027 INFO:tasks.workunit.client.0.smithi100.stdout:op 5040 completed, throughput=5MB/sec 2024-04-07T16:55:21.113 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:55:20 smithi144 bash[66654]: cluster 2024-04-07T16:55:19.582743+0000 mgr.y (mgr.25175) 1199 : cluster [DBG] pgmap v1841: 105 pgs: 105 active+clean; 126 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 3.0 MiB/s rd, 1.7 MiB/s wr, 4 op/s 2024-04-07T16:55:22.027 INFO:tasks.workunit.client.0.smithi100.stdout: 973: throughput=5MB/sec pending data=0 2024-04-07T16:55:22.027 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-0oFHJsf8jsSLJ_J off=5757436 len=1579003 2024-04-07T16:55:22.031 INFO:tasks.workunit.client.0.smithi100.stdout:op 5041 completed, throughput=5MB/sec 2024-04-07T16:55:22.031 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-IBYF1Xb5EfshKcV off=9036888 len=1407884 2024-04-07T16:55:22.044 INFO:tasks.workunit.client.0.smithi100.stdout:op 5042 completed, throughput=5MB/sec 2024-04-07T16:55:22.044 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-FwqxEGiJrZy2LD5 off=1996572 len=413554 2024-04-07T16:55:22.046 INFO:tasks.workunit.client.0.smithi100.stdout:op 5043 completed, throughput=5MB/sec 2024-04-07T16:55:22.046 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-9PlJSZtInIkEu5O off=1173169 len=457976 2024-04-07T16:55:22.053 INFO:tasks.workunit.client.0.smithi100.stdout:op 5044 completed, throughput=5MB/sec 2024-04-07T16:55:22.053 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Im_LMPBAxY4L7UZ off=6539625 len=1366796 2024-04-07T16:55:22.055 INFO:tasks.workunit.client.0.smithi100.stdout:op 5045 completed, throughput=5MB/sec 2024-04-07T16:55:22.055 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-C9pVGBqx3ZS85CS off=6356276 len=1981689 2024-04-07T16:55:22.071 INFO:tasks.workunit.client.0.smithi100.stdout:op 5046 completed, throughput=5MB/sec 2024-04-07T16:55:22.613 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:55:22 smithi144 bash[95233]: ts=2024-04-07T16:55:22.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:55:22.949 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:55:22 smithi100 bash[74942]: cluster 2024-04-07T16:55:21.583663+0000 mgr.y (mgr.25175) 1200 : cluster [DBG] pgmap v1842: 105 pgs: 105 active+clean; 126 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 2.8 MiB/s rd, 2.0 MiB/s wr, 5 op/s 2024-04-07T16:55:22.949 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:55:22 smithi100 bash[76202]: cluster 2024-04-07T16:55:21.583663+0000 mgr.y (mgr.25175) 1200 : cluster [DBG] pgmap v1842: 105 pgs: 105 active+clean; 126 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 2.8 MiB/s rd, 2.0 MiB/s wr, 5 op/s 2024-04-07T16:55:23.071 INFO:tasks.workunit.client.0.smithi100.stdout: 974: throughput=5MB/sec pending data=0 2024-04-07T16:55:23.071 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-IOMbfvwYeGf7DKc off=5865444 len=1565961 2024-04-07T16:55:23.086 INFO:tasks.workunit.client.0.smithi100.stdout:op 5047 completed, throughput=5MB/sec 2024-04-07T16:55:23.086 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-c5hU8gLFrCJGedn off=7950191 len=182877 2024-04-07T16:55:23.087 INFO:tasks.workunit.client.0.smithi100.stdout:op 5048 completed, throughput=5MB/sec 2024-04-07T16:55:23.087 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-jK1QOivKTlSoSyJ off=6251181 len=603871 2024-04-07T16:55:23.089 INFO:tasks.workunit.client.0.smithi100.stdout:op 5049 completed, throughput=5MB/sec 2024-04-07T16:55:23.089 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-UXkwwd2PeM1XLnU off=11891653 len=1192237 2024-04-07T16:55:23.099 INFO:tasks.workunit.client.0.smithi100.stdout:op 5050 completed, throughput=5MB/sec 2024-04-07T16:55:23.099 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-IUPd00xnzXc5Wmr off=12295795 len=89564 2024-04-07T16:55:23.100 INFO:tasks.workunit.client.0.smithi100.stdout:op 5051 completed, throughput=5MB/sec 2024-04-07T16:55:23.112 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:55:22 smithi144 bash[66654]: cluster 2024-04-07T16:55:21.583663+0000 mgr.y (mgr.25175) 1200 : cluster [DBG] pgmap v1842: 105 pgs: 105 active+clean; 126 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 2.8 MiB/s rd, 2.0 MiB/s wr, 5 op/s 2024-04-07T16:55:24.101 INFO:tasks.workunit.client.0.smithi100.stdout: 975: throughput=5MB/sec pending data=0 2024-04-07T16:55:24.101 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-YgEyuMLDLVbeJA1 off=6096622 len=1973182 2024-04-07T16:55:24.105 INFO:tasks.workunit.client.0.smithi100.stdout:op 5052 completed, throughput=5MB/sec 2024-04-07T16:55:24.105 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-lMNjbLzZrR7GCSi off=1015668 len=82398 2024-04-07T16:55:24.107 INFO:tasks.workunit.client.0.smithi100.stdout:op 5053 completed, throughput=5MB/sec 2024-04-07T16:55:24.107 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-00UrrsI-2iA5gzH off=12884171 len=366847 2024-04-07T16:55:24.113 INFO:tasks.workunit.client.0.smithi100.stdout:op 5054 completed, throughput=5MB/sec 2024-04-07T16:55:24.113 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-1gsci6oxalFHYcn off=21582751 len=1088702 2024-04-07T16:55:24.123 INFO:tasks.workunit.client.0.smithi100.stdout:op 5055 completed, throughput=5MB/sec 2024-04-07T16:55:24.124 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-UGxlELlCiAmjGkP off=7051238 len=1978743 2024-04-07T16:55:24.139 INFO:tasks.workunit.client.0.smithi100.stdout:op 5056 completed, throughput=5MB/sec 2024-04-07T16:55:24.949 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:55:24 smithi100 bash[74942]: cluster 2024-04-07T16:55:23.584885+0000 mgr.y (mgr.25175) 1201 : cluster [DBG] pgmap v1843: 105 pgs: 105 active+clean; 126 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 3.4 MiB/s rd, 2.0 MiB/s wr, 5 op/s 2024-04-07T16:55:24.949 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:55:24 smithi100 bash[76202]: cluster 2024-04-07T16:55:23.584885+0000 mgr.y (mgr.25175) 1201 : cluster [DBG] pgmap v1843: 105 pgs: 105 active+clean; 126 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 3.4 MiB/s rd, 2.0 MiB/s wr, 5 op/s 2024-04-07T16:55:25.112 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:55:24 smithi144 bash[66654]: cluster 2024-04-07T16:55:23.584885+0000 mgr.y (mgr.25175) 1201 : cluster [DBG] pgmap v1843: 105 pgs: 105 active+clean; 126 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 3.4 MiB/s rd, 2.0 MiB/s wr, 5 op/s 2024-04-07T16:55:25.139 INFO:tasks.workunit.client.0.smithi100.stdout: 976: throughput=5MB/sec pending data=0 2024-04-07T16:55:25.139 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-iY14QJQnyZOrEVt off=5800617 len=281503 2024-04-07T16:55:25.145 INFO:tasks.workunit.client.0.smithi100.stdout:op 5057 completed, throughput=5MB/sec 2024-04-07T16:55:25.145 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-L3gMoGoNHn6ptYc off=12165844 len=1111248 2024-04-07T16:55:25.148 INFO:tasks.workunit.client.0.smithi100.stdout:op 5058 completed, throughput=5MB/sec 2024-04-07T16:55:25.148 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-6hfq0O7hjkHPo6B off=1219199 len=704307 2024-04-07T16:55:25.151 INFO:tasks.workunit.client.0.smithi100.stdout:op 5059 completed, throughput=5MB/sec 2024-04-07T16:55:25.151 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-8n9wcgkRJtIUG9J off=1224899 len=218655 2024-04-07T16:55:25.152 INFO:tasks.workunit.client.0.smithi100.stdout:op 5060 completed, throughput=5MB/sec 2024-04-07T16:55:25.152 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-q5ZsM_qxgfrXgF4 off=1288121 len=124818 2024-04-07T16:55:25.157 INFO:tasks.workunit.client.0.smithi100.stdout:op 5061 completed, throughput=5MB/sec 2024-04-07T16:55:25.157 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-msy8SiH_5igOJmR off=1907025 len=532983 2024-04-07T16:55:25.160 INFO:tasks.workunit.client.0.smithi100.stdout:op 5062 completed, throughput=5MB/sec 2024-04-07T16:55:25.160 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-7JF-OzcIIGrKuqq off=6849173 len=184873 2024-04-07T16:55:25.161 INFO:tasks.workunit.client.0.smithi100.stdout:op 5063 completed, throughput=5MB/sec 2024-04-07T16:55:25.161 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-kxHaJzPYL5YSO1R off=5126946 len=1012223 2024-04-07T16:55:25.164 INFO:tasks.workunit.client.0.smithi100.stdout:op 5064 completed, throughput=5MB/sec 2024-04-07T16:55:25.164 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-jXTugkeKi6LPS9c off=3041517 len=1011121 2024-04-07T16:55:25.167 INFO:tasks.workunit.client.0.smithi100.stdout:op 5065 completed, throughput=5MB/sec 2024-04-07T16:55:25.167 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-e9tgHM6KrMtsXb9 off=4053952 len=1710018 2024-04-07T16:55:25.172 INFO:tasks.workunit.client.0.smithi100.stdout:op 5066 completed, throughput=5MB/sec 2024-04-07T16:55:26.172 INFO:tasks.workunit.client.0.smithi100.stdout: 977: throughput=5MB/sec pending data=0 2024-04-07T16:55:26.173 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-SlAUQFhKvPhUvc6 off=2255154 len=2068128 2024-04-07T16:55:26.181 INFO:tasks.workunit.client.0.smithi100.stdout:op 5067 completed, throughput=5MB/sec 2024-04-07T16:55:26.181 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-t0J0ZPu3EPuYr1c off=2629070 len=2084353 2024-04-07T16:55:26.197 INFO:tasks.workunit.client.0.smithi100.stdout:op 5068 completed, throughput=5MB/sec 2024-04-07T16:55:26.949 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:55:26 smithi100 bash[74942]: cluster 2024-04-07T16:55:25.585938+0000 mgr.y (mgr.25175) 1202 : cluster [DBG] pgmap v1844: 105 pgs: 105 active+clean; 126 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 2.5 MiB/s rd, 1.9 MiB/s wr, 4 op/s 2024-04-07T16:55:26.949 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:55:26 smithi100 bash[76202]: cluster 2024-04-07T16:55:25.585938+0000 mgr.y (mgr.25175) 1202 : cluster [DBG] pgmap v1844: 105 pgs: 105 active+clean; 126 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 2.5 MiB/s rd, 1.9 MiB/s wr, 4 op/s 2024-04-07T16:55:27.112 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:55:26 smithi144 bash[66654]: cluster 2024-04-07T16:55:25.585938+0000 mgr.y (mgr.25175) 1202 : cluster [DBG] pgmap v1844: 105 pgs: 105 active+clean; 126 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 2.5 MiB/s rd, 1.9 MiB/s wr, 4 op/s 2024-04-07T16:55:27.198 INFO:tasks.workunit.client.0.smithi100.stdout: 978: throughput=5MB/sec pending data=0 2024-04-07T16:55:27.198 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-ae5Wu-BYu8UrWvV off=10046345 len=1233958 2024-04-07T16:55:27.207 INFO:tasks.workunit.client.0.smithi100.stdout:op 5069 completed, throughput=5MB/sec 2024-04-07T16:55:27.208 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-g9xV1JUjI0x0OhE off=6674543 len=1756477 2024-04-07T16:55:27.220 INFO:tasks.workunit.client.0.smithi100.stdout:op 5070 completed, throughput=5MB/sec 2024-04-07T16:55:27.220 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-_5ZKHHqfApuet5q off=20612534 len=1752538 2024-04-07T16:55:27.225 INFO:tasks.workunit.client.0.smithi100.stdout:op 5071 completed, throughput=5MB/sec 2024-04-07T16:55:27.225 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-1gylMo-t2_79S7D off=2744568 len=20036 2024-04-07T16:55:27.226 INFO:tasks.workunit.client.0.smithi100.stdout:op 5072 completed, throughput=5MB/sec 2024-04-07T16:55:27.226 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-eengo2TfA0XJhYk off=9663009 len=927324 2024-04-07T16:55:27.228 INFO:tasks.workunit.client.0.smithi100.stdout:op 5073 completed, throughput=5MB/sec 2024-04-07T16:55:28.229 INFO:tasks.workunit.client.0.smithi100.stdout: 979: throughput=5MB/sec pending data=0 2024-04-07T16:55:28.229 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-YJbR2F68mSLgKUR off=10950044 len=714005 2024-04-07T16:55:28.237 INFO:tasks.workunit.client.0.smithi100.stdout:op 5074 completed, throughput=5MB/sec 2024-04-07T16:55:28.237 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-FTWM5FoVWghalYu off=4562435 len=1737774 2024-04-07T16:55:28.251 INFO:tasks.workunit.client.0.smithi100.stdout:op 5075 completed, throughput=5MB/sec 2024-04-07T16:55:28.251 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-GJ6T4fVGbju5ekS off=15574681 len=1369587 2024-04-07T16:55:28.265 INFO:tasks.workunit.client.0.smithi100.stdout:op 5076 completed, throughput=5MB/sec 2024-04-07T16:55:28.265 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Uy2fAVUj9QgWze8 off=117526 len=472689 2024-04-07T16:55:28.272 INFO:tasks.workunit.client.0.smithi100.stdout:op 5077 completed, throughput=5MB/sec 2024-04-07T16:55:28.272 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-8zinMBN38okE0XY off=6457463 len=1794796 2024-04-07T16:55:28.276 INFO:tasks.workunit.client.0.smithi100.stdout:op 5078 completed, throughput=5MB/sec 2024-04-07T16:55:28.699 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:55:28 smithi100 bash[74942]: cluster 2024-04-07T16:55:27.586993+0000 mgr.y (mgr.25175) 1203 : cluster [DBG] pgmap v1845: 105 pgs: 105 active+clean; 126 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.1 MiB/s wr, 4 op/s 2024-04-07T16:55:28.699 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:55:28 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:55:28] "GET /metrics HTTP/1.1" 200 34073 "" "Prometheus/2.43.0" 2024-04-07T16:55:28.699 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:55:28 smithi100 bash[76202]: cluster 2024-04-07T16:55:27.586993+0000 mgr.y (mgr.25175) 1203 : cluster [DBG] pgmap v1845: 105 pgs: 105 active+clean; 126 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.1 MiB/s wr, 4 op/s 2024-04-07T16:55:29.112 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:55:28 smithi144 bash[66654]: cluster 2024-04-07T16:55:27.586993+0000 mgr.y (mgr.25175) 1203 : cluster [DBG] pgmap v1845: 105 pgs: 105 active+clean; 126 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.1 MiB/s wr, 4 op/s 2024-04-07T16:55:29.276 INFO:tasks.workunit.client.0.smithi100.stdout: 980: throughput=5MB/sec pending data=0 2024-04-07T16:55:29.277 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-b7EGHmx1m51g9Lx off=16286514 len=693888 2024-04-07T16:55:29.280 INFO:tasks.workunit.client.0.smithi100.stdout:op 5079 completed, throughput=5MB/sec 2024-04-07T16:55:29.280 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-MVRBDFNMy8-2uv2 off=0 len=1326782 2024-04-07T16:55:29.285 INFO:tasks.workunit.client.0.smithi100.stdout:op 5080 completed, throughput=5MB/sec 2024-04-07T16:55:29.285 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-rAGT4ScxDS5RVMd off=3336680 len=916519 2024-04-07T16:55:29.293 INFO:tasks.workunit.client.0.smithi100.stdout:op 5081 completed, throughput=5MB/sec 2024-04-07T16:55:29.294 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-kuScHAwvGGoFiG- off=6541735 len=996719 2024-04-07T16:55:29.302 INFO:tasks.workunit.client.0.smithi100.stdout:op 5082 completed, throughput=5MB/sec 2024-04-07T16:55:29.302 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-GA3XxKaCZ_1RjPo off=11908822 len=397906 2024-04-07T16:55:29.304 INFO:tasks.workunit.client.0.smithi100.stdout:op 5083 completed, throughput=5MB/sec 2024-04-07T16:55:30.304 INFO:tasks.workunit.client.0.smithi100.stdout: 981: throughput=5MB/sec pending data=0 2024-04-07T16:55:30.304 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-KcqNLLuBfL4Z7uh off=2044429 len=947250 2024-04-07T16:55:30.307 INFO:tasks.workunit.client.0.smithi100.stdout:op 5084 completed, throughput=5MB/sec 2024-04-07T16:55:30.307 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-GKbd9gzK6dtGU5J off=2453116 len=1401686 2024-04-07T16:55:30.319 INFO:tasks.workunit.client.0.smithi100.stdout:op 5085 completed, throughput=5MB/sec 2024-04-07T16:55:30.319 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Qe9G94nNEXq1HNY off=12017401 len=1718308 2024-04-07T16:55:30.324 INFO:tasks.workunit.client.0.smithi100.stdout:op 5086 completed, throughput=5MB/sec 2024-04-07T16:55:30.324 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-MLyX78p9wn-S294 off=8599702 len=648438 2024-04-07T16:55:30.331 INFO:tasks.workunit.client.0.smithi100.stdout:op 5087 completed, throughput=5MB/sec 2024-04-07T16:55:30.331 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-4pMDm99DMYwxjP0 off=4145484 len=1538736 2024-04-07T16:55:30.334 INFO:tasks.workunit.client.0.smithi100.stdout:op 5088 completed, throughput=5MB/sec 2024-04-07T16:55:30.949 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:55:30 smithi100 bash[74942]: cluster 2024-04-07T16:55:29.587930+0000 mgr.y (mgr.25175) 1204 : cluster [DBG] pgmap v1846: 105 pgs: 105 active+clean; 126 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 2.4 MiB/s rd, 2.0 MiB/s wr, 4 op/s 2024-04-07T16:55:30.949 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:55:30 smithi100 bash[76202]: cluster 2024-04-07T16:55:29.587930+0000 mgr.y (mgr.25175) 1204 : cluster [DBG] pgmap v1846: 105 pgs: 105 active+clean; 126 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 2.4 MiB/s rd, 2.0 MiB/s wr, 4 op/s 2024-04-07T16:55:31.112 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:55:30 smithi144 bash[66654]: cluster 2024-04-07T16:55:29.587930+0000 mgr.y (mgr.25175) 1204 : cluster [DBG] pgmap v1846: 105 pgs: 105 active+clean; 126 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 2.4 MiB/s rd, 2.0 MiB/s wr, 4 op/s 2024-04-07T16:55:31.335 INFO:tasks.workunit.client.0.smithi100.stdout: 982: throughput=5MB/sec pending data=0 2024-04-07T16:55:31.335 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-wACiUGML38iL4JE off=6611733 len=694477 2024-04-07T16:55:31.342 INFO:tasks.workunit.client.0.smithi100.stdout:op 5089 completed, throughput=5MB/sec 2024-04-07T16:55:31.342 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-tCEWP1gZszxy7xX off=5050364 len=481217 2024-04-07T16:55:31.344 INFO:tasks.workunit.client.0.smithi100.stdout:op 5090 completed, throughput=5MB/sec 2024-04-07T16:55:31.344 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-9ALiB0BqNFUdROH off=3699881 len=2050713 2024-04-07T16:55:31.351 INFO:tasks.workunit.client.0.smithi100.stdout:op 5091 completed, throughput=5MB/sec 2024-04-07T16:55:31.351 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Ywj8NyCwsMFO428 off=6026318 len=447072 2024-04-07T16:55:31.352 INFO:tasks.workunit.client.0.smithi100.stdout:op 5092 completed, throughput=5MB/sec 2024-04-07T16:55:31.353 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-s5TkutAMCBPEJXR off=5420541 len=823613 2024-04-07T16:55:31.361 INFO:tasks.workunit.client.0.smithi100.stdout:op 5093 completed, throughput=5MB/sec 2024-04-07T16:55:32.361 INFO:tasks.workunit.client.0.smithi100.stdout: 983: throughput=5MB/sec pending data=0 2024-04-07T16:55:32.361 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-lkHrRzK8zjDvPk2 off=8462469 len=1903563 2024-04-07T16:55:32.377 INFO:tasks.workunit.client.0.smithi100.stdout:op 5094 completed, throughput=5MB/sec 2024-04-07T16:55:32.377 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-51ABbILOrdtxT45 off=946805 len=1103294 2024-04-07T16:55:32.385 INFO:tasks.workunit.client.0.smithi100.stdout:op 5095 completed, throughput=5MB/sec 2024-04-07T16:55:32.385 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-59Nq8yIHsL8flpJ off=664207 len=881835 2024-04-07T16:55:32.388 INFO:tasks.workunit.client.0.smithi100.stdout:op 5096 completed, throughput=5MB/sec 2024-04-07T16:55:32.388 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-X-sfNyKW_rtiMPw off=1611556 len=1944105 2024-04-07T16:55:32.392 INFO:tasks.workunit.client.0.smithi100.stdout:op 5097 completed, throughput=5MB/sec 2024-04-07T16:55:32.614 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:55:32 smithi144 bash[95233]: ts=2024-04-07T16:55:32.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:55:32.949 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:55:32 smithi100 bash[74942]: cluster 2024-04-07T16:55:31.588703+0000 mgr.y (mgr.25175) 1205 : cluster [DBG] pgmap v1847: 105 pgs: 105 active+clean; 126 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 2.6 MiB/s rd, 2.0 MiB/s wr, 4 op/s 2024-04-07T16:55:32.949 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:55:32 smithi100 bash[74942]: audit 2024-04-07T16:55:31.785355+0000 mon.a (mon.0) 2192 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:55:32.949 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:55:32 smithi100 bash[76202]: cluster 2024-04-07T16:55:31.588703+0000 mgr.y (mgr.25175) 1205 : cluster [DBG] pgmap v1847: 105 pgs: 105 active+clean; 126 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 2.6 MiB/s rd, 2.0 MiB/s wr, 4 op/s 2024-04-07T16:55:32.949 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:55:32 smithi100 bash[76202]: audit 2024-04-07T16:55:31.785355+0000 mon.a (mon.0) 2192 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:55:33.112 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:55:32 smithi144 bash[66654]: cluster 2024-04-07T16:55:31.588703+0000 mgr.y (mgr.25175) 1205 : cluster [DBG] pgmap v1847: 105 pgs: 105 active+clean; 126 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 2.6 MiB/s rd, 2.0 MiB/s wr, 4 op/s 2024-04-07T16:55:33.113 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:55:32 smithi144 bash[66654]: audit 2024-04-07T16:55:31.785355+0000 mon.a (mon.0) 2192 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:55:33.393 INFO:tasks.workunit.client.0.smithi100.stdout: 984: throughput=5MB/sec pending data=0 2024-04-07T16:55:33.393 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-LtwizUwrneUprH3 off=7490075 len=1311790 2024-04-07T16:55:33.405 INFO:tasks.workunit.client.0.smithi100.stdout:op 5098 completed, throughput=5MB/sec 2024-04-07T16:55:33.405 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-FYV-YW6W7OT50_q off=3730902 len=672147 2024-04-07T16:55:33.412 INFO:tasks.workunit.client.0.smithi100.stdout:op 5099 completed, throughput=5MB/sec 2024-04-07T16:55:33.412 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-SaPCjtyep8E8TZT off=1861947 len=44151 2024-04-07T16:55:33.417 INFO:tasks.workunit.client.0.smithi100.stdout:op 5100 completed, throughput=5MB/sec 2024-04-07T16:55:33.417 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-gAfL1NL8noml8uC off=13398806 len=1750374 2024-04-07T16:55:33.430 INFO:tasks.workunit.client.0.smithi100.stdout:op 5101 completed, throughput=5MB/sec 2024-04-07T16:55:33.430 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-L98Ss4CUVFOTi3M off=11818935 len=208719 2024-04-07T16:55:33.436 INFO:tasks.workunit.client.0.smithi100.stdout:op 5102 completed, throughput=5MB/sec 2024-04-07T16:55:33.436 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-aeMISH7hdig7VX6 off=379158 len=993638 2024-04-07T16:55:33.438 INFO:tasks.workunit.client.0.smithi100.stdout:op 5103 completed, throughput=5MB/sec 2024-04-07T16:55:34.439 INFO:tasks.workunit.client.0.smithi100.stdout: 985: throughput=5MB/sec pending data=0 2024-04-07T16:55:34.439 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-YUcf1NQS5k-el6L off=8407974 len=1343083 2024-04-07T16:55:34.442 INFO:tasks.workunit.client.0.smithi100.stdout:op 5104 completed, throughput=5MB/sec 2024-04-07T16:55:34.442 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Krufd9fwizT-lIp off=1893829 len=481557 2024-04-07T16:55:34.444 INFO:tasks.workunit.client.0.smithi100.stdout:op 5105 completed, throughput=5MB/sec 2024-04-07T16:55:34.444 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-wpgpHtWfLyJqtSJ off=11665331 len=152331 2024-04-07T16:55:34.445 INFO:tasks.workunit.client.0.smithi100.stdout:op 5106 completed, throughput=5MB/sec 2024-04-07T16:55:34.446 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-OWgUgnawwHT5QaM off=6328756 len=1798520 2024-04-07T16:55:34.452 INFO:tasks.workunit.client.0.smithi100.stdout:op 5107 completed, throughput=5MB/sec 2024-04-07T16:55:34.452 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-HmlFN5fqjKqU_XU off=10289388 len=283336 2024-04-07T16:55:34.457 INFO:tasks.workunit.client.0.smithi100.stdout:op 5108 completed, throughput=5MB/sec 2024-04-07T16:55:34.457 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-P-tYc83i7c_6YeX off=979020 len=459546 2024-04-07T16:55:34.464 INFO:tasks.workunit.client.0.smithi100.stdout:op 5109 completed, throughput=5MB/sec 2024-04-07T16:55:34.464 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-fs2RRzVRtzSIuBo off=13265837 len=672157 2024-04-07T16:55:34.471 INFO:tasks.workunit.client.0.smithi100.stdout:op 5110 completed, throughput=5MB/sec 2024-04-07T16:55:34.471 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-mq0fFmf4M_G_9uy off=24500616 len=434628 2024-04-07T16:55:34.477 INFO:tasks.workunit.client.0.smithi100.stdout:op 5111 completed, throughput=5MB/sec 2024-04-07T16:55:34.949 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:55:34 smithi100 bash[74942]: cluster 2024-04-07T16:55:33.589830+0000 mgr.y (mgr.25175) 1206 : cluster [DBG] pgmap v1848: 105 pgs: 105 active+clean; 126 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 3.4 MiB/s rd, 2.2 MiB/s wr, 5 op/s 2024-04-07T16:55:34.949 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:55:34 smithi100 bash[76202]: cluster 2024-04-07T16:55:33.589830+0000 mgr.y (mgr.25175) 1206 : cluster [DBG] pgmap v1848: 105 pgs: 105 active+clean; 126 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 3.4 MiB/s rd, 2.2 MiB/s wr, 5 op/s 2024-04-07T16:55:35.112 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:55:34 smithi144 bash[66654]: cluster 2024-04-07T16:55:33.589830+0000 mgr.y (mgr.25175) 1206 : cluster [DBG] pgmap v1848: 105 pgs: 105 active+clean; 126 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 3.4 MiB/s rd, 2.2 MiB/s wr, 5 op/s 2024-04-07T16:55:35.478 INFO:tasks.workunit.client.0.smithi100.stdout: 986: throughput=5MB/sec pending data=0 2024-04-07T16:55:35.478 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-8nYt0yD1dL3lPnH off=729152 len=1650969 2024-04-07T16:55:35.482 INFO:tasks.workunit.client.0.smithi100.stdout:op 5112 completed, throughput=5MB/sec 2024-04-07T16:55:35.482 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Kjc9ZMyHjWhn8Yb off=1191663 len=469217 2024-04-07T16:55:35.489 INFO:tasks.workunit.client.0.smithi100.stdout:op 5113 completed, throughput=5MB/sec 2024-04-07T16:55:35.489 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-qpnh-sVN7d_-WrX off=8413458 len=748522 2024-04-07T16:55:35.491 INFO:tasks.workunit.client.0.smithi100.stdout:op 5114 completed, throughput=5MB/sec 2024-04-07T16:55:35.491 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-hbNUtsbIvnGO8hm off=5320488 len=624025 2024-04-07T16:55:35.493 INFO:tasks.workunit.client.0.smithi100.stdout:op 5115 completed, throughput=5MB/sec 2024-04-07T16:55:35.494 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-uyol5mFZ43gcdRv off=22845123 len=1443711 2024-04-07T16:55:35.505 INFO:tasks.workunit.client.0.smithi100.stdout:op 5116 completed, throughput=5MB/sec 2024-04-07T16:55:35.505 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Ae0uIq0LW9Pcerr off=3481963 len=474647 2024-04-07T16:55:35.511 INFO:tasks.workunit.client.0.smithi100.stdout:op 5117 completed, throughput=5MB/sec 2024-04-07T16:55:36.512 INFO:tasks.workunit.client.0.smithi100.stdout: 987: throughput=5MB/sec pending data=0 2024-04-07T16:55:36.512 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-J7HapWXXJZ1whxT off=4725477 len=1331776 2024-04-07T16:55:36.515 INFO:tasks.workunit.client.0.smithi100.stdout:op 5118 completed, throughput=5MB/sec 2024-04-07T16:55:36.515 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-IFcm1vNfbg_wjmh off=6269213 len=980487 2024-04-07T16:55:36.524 INFO:tasks.workunit.client.0.smithi100.stdout:op 5119 completed, throughput=5MB/sec 2024-04-07T16:55:36.524 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Pzf6HGRAYzkqg_x off=2744443 len=1623906 2024-04-07T16:55:36.539 INFO:tasks.workunit.client.0.smithi100.stdout:op 5120 completed, throughput=5MB/sec 2024-04-07T16:55:36.539 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-WjO8VRbfaFwohhC off=3793577 len=1934524 2024-04-07T16:55:36.542 INFO:tasks.workunit.client.0.smithi100.stdout:op 5121 completed, throughput=5MB/sec 2024-04-07T16:55:36.949 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:55:36 smithi100 bash[74942]: cluster 2024-04-07T16:55:35.590841+0000 mgr.y (mgr.25175) 1207 : cluster [DBG] pgmap v1849: 105 pgs: 105 active+clean; 126 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 2.8 MiB/s rd, 2.2 MiB/s wr, 5 op/s 2024-04-07T16:55:36.949 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:55:36 smithi100 bash[76202]: cluster 2024-04-07T16:55:35.590841+0000 mgr.y (mgr.25175) 1207 : cluster [DBG] pgmap v1849: 105 pgs: 105 active+clean; 126 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 2.8 MiB/s rd, 2.2 MiB/s wr, 5 op/s 2024-04-07T16:55:37.113 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:55:36 smithi144 bash[66654]: cluster 2024-04-07T16:55:35.590841+0000 mgr.y (mgr.25175) 1207 : cluster [DBG] pgmap v1849: 105 pgs: 105 active+clean; 126 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 2.8 MiB/s rd, 2.2 MiB/s wr, 5 op/s 2024-04-07T16:55:37.542 INFO:tasks.workunit.client.0.smithi100.stdout: 988: throughput=5MB/sec pending data=0 2024-04-07T16:55:37.542 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-qhyBwPxZBDTktVn off=2022423 len=1339921 2024-04-07T16:55:37.545 INFO:tasks.workunit.client.0.smithi100.stdout:op 5122 completed, throughput=5MB/sec 2024-04-07T16:55:37.545 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-QJtRrpikGlZim2D off=11426885 len=1434787 2024-04-07T16:55:37.557 INFO:tasks.workunit.client.0.smithi100.stdout:op 5123 completed, throughput=5MB/sec 2024-04-07T16:55:37.557 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-VLV92FEMvhGhg0N off=998333 len=732652 2024-04-07T16:55:37.560 INFO:tasks.workunit.client.0.smithi100.stdout:op 5124 completed, throughput=5MB/sec 2024-04-07T16:55:37.560 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-FlLg5p-Rw_Tho39 off=2933104 len=1818820 2024-04-07T16:55:37.571 INFO:tasks.workunit.client.0.smithi100.stdout:op 5125 completed, throughput=5MB/sec 2024-04-07T16:55:38.572 INFO:tasks.workunit.client.0.smithi100.stdout: 989: throughput=5MB/sec pending data=0 2024-04-07T16:55:38.572 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-zR440fSHRHw34RX off=15875133 len=406926 2024-04-07T16:55:38.579 INFO:tasks.workunit.client.0.smithi100.stdout:op 5126 completed, throughput=5MB/sec 2024-04-07T16:55:38.579 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-VZP2108ocdRH4B9 off=5478113 len=961246 2024-04-07T16:55:38.583 INFO:tasks.workunit.client.0.smithi100.stdout:op 5127 completed, throughput=5MB/sec 2024-04-07T16:55:38.583 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-L98Ss4CUVFOTi3M off=13766328 len=865689 2024-04-07T16:55:38.591 INFO:tasks.workunit.client.0.smithi100.stdout:op 5128 completed, throughput=5MB/sec 2024-04-07T16:55:38.591 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-fQy40uBEqQUr3Hk off=2914778 len=586761 2024-04-07T16:55:38.593 INFO:tasks.workunit.client.0.smithi100.stdout:op 5129 completed, throughput=5MB/sec 2024-04-07T16:55:38.593 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-FvY1hM0ZxcEGxUJ off=23136255 len=205532 2024-04-07T16:55:38.598 INFO:tasks.workunit.client.0.smithi100.stdout:op 5130 completed, throughput=5MB/sec 2024-04-07T16:55:38.598 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-X1g7wsr3-57HdPv off=4581320 len=722184 2024-04-07T16:55:38.599 INFO:tasks.workunit.client.0.smithi100.stdout:op 5131 completed, throughput=5MB/sec 2024-04-07T16:55:38.600 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-hhqwcjEEjGjHOEo off=2702918 len=836306 2024-04-07T16:55:38.601 INFO:tasks.workunit.client.0.smithi100.stdout:op 5132 completed, throughput=5MB/sec 2024-04-07T16:55:38.601 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-oBk1kwO1XkBAPzE off=13676871 len=279522 2024-04-07T16:55:38.607 INFO:tasks.workunit.client.0.smithi100.stdout:op 5133 completed, throughput=5MB/sec 2024-04-07T16:55:38.693 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:55:38 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:55:38] "GET /metrics HTTP/1.1" 200 34072 "" "Prometheus/2.43.0" 2024-04-07T16:55:38.949 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:55:38 smithi100 bash[74942]: cluster 2024-04-07T16:55:37.592032+0000 mgr.y (mgr.25175) 1208 : cluster [DBG] pgmap v1850: 105 pgs: 105 active+clean; 126 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 2.9 MiB/s rd, 2.4 MiB/s wr, 5 op/s 2024-04-07T16:55:38.950 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:55:38 smithi100 bash[76202]: cluster 2024-04-07T16:55:37.592032+0000 mgr.y (mgr.25175) 1208 : cluster [DBG] pgmap v1850: 105 pgs: 105 active+clean; 126 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 2.9 MiB/s rd, 2.4 MiB/s wr, 5 op/s 2024-04-07T16:55:39.112 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:55:38 smithi144 bash[66654]: cluster 2024-04-07T16:55:37.592032+0000 mgr.y (mgr.25175) 1208 : cluster [DBG] pgmap v1850: 105 pgs: 105 active+clean; 126 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 2.9 MiB/s rd, 2.4 MiB/s wr, 5 op/s 2024-04-07T16:55:39.607 INFO:tasks.workunit.client.0.smithi100.stdout: 990: throughput=5MB/sec pending data=0 2024-04-07T16:55:39.608 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-MeGMP7HPGcCC2-N off=6528334 len=786706 2024-04-07T16:55:39.611 INFO:tasks.workunit.client.0.smithi100.stdout:op 5134 completed, throughput=5MB/sec 2024-04-07T16:55:39.611 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-IEKMmI_l7rY30tC off=250661 len=1970405 2024-04-07T16:55:39.627 INFO:tasks.workunit.client.0.smithi100.stdout:op 5135 completed, throughput=5MB/sec 2024-04-07T16:55:39.627 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-TgDmNtxc3hafG6c off=16707138 len=251925 2024-04-07T16:55:39.632 INFO:tasks.workunit.client.0.smithi100.stdout:op 5136 completed, throughput=5MB/sec 2024-04-07T16:55:39.632 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-j-qpsA0fJs7uk5c off=3755 len=1475282 2024-04-07T16:55:39.637 INFO:tasks.workunit.client.0.smithi100.stdout:op 5137 completed, throughput=5MB/sec 2024-04-07T16:55:39.637 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-CDhCLVlv71chJtI off=933321 len=237372 2024-04-07T16:55:39.642 INFO:tasks.workunit.client.0.smithi100.stdout:op 5138 completed, throughput=5MB/sec 2024-04-07T16:55:39.642 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-bBjytGl5HAusQ59 off=7908692 len=595759 2024-04-07T16:55:39.649 INFO:tasks.workunit.client.0.smithi100.stdout:op 5139 completed, throughput=5MB/sec 2024-04-07T16:55:39.650 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Sy6RZyRgychmKea off=10912 len=797610 2024-04-07T16:55:39.657 INFO:tasks.workunit.client.0.smithi100.stdout:op 5140 completed, throughput=5MB/sec 2024-04-07T16:55:40.658 INFO:tasks.workunit.client.0.smithi100.stdout: 991: throughput=5MB/sec pending data=0 2024-04-07T16:55:40.658 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-xiF_ie_ZBpROeP2 off=1012217 len=1972439 2024-04-07T16:55:40.674 INFO:tasks.workunit.client.0.smithi100.stdout:op 5141 completed, throughput=5MB/sec 2024-04-07T16:55:40.675 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-6RVXgiMsdKpGoBU off=9992727 len=1033433 2024-04-07T16:55:40.683 INFO:tasks.workunit.client.0.smithi100.stdout:op 5142 completed, throughput=5MB/sec 2024-04-07T16:55:40.684 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-fXHhN6K_RmqPoME off=6283907 len=341215 2024-04-07T16:55:40.685 INFO:tasks.workunit.client.0.smithi100.stdout:op 5143 completed, throughput=5MB/sec 2024-04-07T16:55:40.685 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-MxmWXv-rSUkRgRb off=1896963 len=2085967 2024-04-07T16:55:40.702 INFO:tasks.workunit.client.0.smithi100.stdout:op 5144 completed, throughput=5MB/sec 2024-04-07T16:55:40.949 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:55:40 smithi100 bash[74942]: cluster 2024-04-07T16:55:39.593046+0000 mgr.y (mgr.25175) 1209 : cluster [DBG] pgmap v1851: 105 pgs: 105 active+clean; 126 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.3 MiB/s wr, 5 op/s 2024-04-07T16:55:40.949 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:55:40 smithi100 bash[76202]: cluster 2024-04-07T16:55:39.593046+0000 mgr.y (mgr.25175) 1209 : cluster [DBG] pgmap v1851: 105 pgs: 105 active+clean; 126 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.3 MiB/s wr, 5 op/s 2024-04-07T16:55:41.112 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:55:40 smithi144 bash[66654]: cluster 2024-04-07T16:55:39.593046+0000 mgr.y (mgr.25175) 1209 : cluster [DBG] pgmap v1851: 105 pgs: 105 active+clean; 126 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.3 MiB/s wr, 5 op/s 2024-04-07T16:55:41.702 INFO:tasks.workunit.client.0.smithi100.stdout: 992: throughput=5MB/sec pending data=0 2024-04-07T16:55:41.702 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-GrcvDl9MtYeULqr off=5909605 len=2079868 2024-04-07T16:55:41.717 INFO:tasks.workunit.client.0.smithi100.stdout:op 5145 completed, throughput=5MB/sec 2024-04-07T16:55:41.717 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-MDFU_L-UqCKl_aE off=9909133 len=1751534 2024-04-07T16:55:41.729 INFO:tasks.workunit.client.0.smithi100.stdout:op 5146 completed, throughput=5MB/sec 2024-04-07T16:55:41.730 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-CZQ2YOJMga4CF1r off=5025307 len=75673 2024-04-07T16:55:41.734 INFO:tasks.workunit.client.0.smithi100.stdout:op 5147 completed, throughput=5MB/sec 2024-04-07T16:55:41.734 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-6HjLTmuZpM0rd7J off=2357124 len=1743471 2024-04-07T16:55:41.749 INFO:tasks.workunit.client.0.smithi100.stdout:op 5148 completed, throughput=5MB/sec 2024-04-07T16:55:42.613 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:55:42 smithi144 bash[95233]: ts=2024-04-07T16:55:42.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:55:42.749 INFO:tasks.workunit.client.0.smithi100.stdout: 993: throughput=5MB/sec pending data=0 2024-04-07T16:55:42.749 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Sj9HNPAhn2MyVMB off=3261305 len=384563 2024-04-07T16:55:42.756 INFO:tasks.workunit.client.0.smithi100.stdout:op 5149 completed, throughput=5MB/sec 2024-04-07T16:55:42.756 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-ba0WCg0G8KayGan off=11080505 len=1140337 2024-04-07T16:55:42.759 INFO:tasks.workunit.client.0.smithi100.stdout:op 5150 completed, throughput=5MB/sec 2024-04-07T16:55:42.759 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-EdPRsKS-T6_Ug8X off=3242825 len=1038201 2024-04-07T16:55:42.762 INFO:tasks.workunit.client.0.smithi100.stdout:op 5151 completed, throughput=5MB/sec 2024-04-07T16:55:42.762 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-EIupUI2qtdTRQRd off=13848 len=1852487 2024-04-07T16:55:42.766 INFO:tasks.workunit.client.0.smithi100.stdout:op 5152 completed, throughput=5MB/sec 2024-04-07T16:55:42.767 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-eL88adJdbodthhh off=5349847 len=1362928 2024-04-07T16:55:42.777 INFO:tasks.workunit.client.0.smithi100.stdout:op 5153 completed, throughput=5MB/sec 2024-04-07T16:55:43.112 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:55:42 smithi144 bash[66654]: cluster 2024-04-07T16:55:41.593877+0000 mgr.y (mgr.25175) 1210 : cluster [DBG] pgmap v1852: 105 pgs: 105 active+clean; 126 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 2.6 MiB/s rd, 2.6 MiB/s wr, 5 op/s 2024-04-07T16:55:43.199 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:55:42 smithi100 bash[74942]: cluster 2024-04-07T16:55:41.593877+0000 mgr.y (mgr.25175) 1210 : cluster [DBG] pgmap v1852: 105 pgs: 105 active+clean; 126 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 2.6 MiB/s rd, 2.6 MiB/s wr, 5 op/s 2024-04-07T16:55:43.199 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:55:42 smithi100 bash[76202]: cluster 2024-04-07T16:55:41.593877+0000 mgr.y (mgr.25175) 1210 : cluster [DBG] pgmap v1852: 105 pgs: 105 active+clean; 126 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 2.6 MiB/s rd, 2.6 MiB/s wr, 5 op/s 2024-04-07T16:55:43.777 INFO:tasks.workunit.client.0.smithi100.stdout: 994: throughput=5MB/sec pending data=0 2024-04-07T16:55:43.777 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-nhiPxYrsPbknqom off=11893356 len=1183174 2024-04-07T16:55:43.787 INFO:tasks.workunit.client.0.smithi100.stdout:op 5154 completed, throughput=5MB/sec 2024-04-07T16:55:43.787 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-R2KHPrYZ-OxR3lW off=14530011 len=621128 2024-04-07T16:55:43.789 INFO:tasks.workunit.client.0.smithi100.stdout:op 5155 completed, throughput=5MB/sec 2024-04-07T16:55:43.789 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-bfOolJGkG5ULcn4 off=3461381 len=1286422 2024-04-07T16:55:43.802 INFO:tasks.workunit.client.0.smithi100.stdout:op 5156 completed, throughput=5MB/sec 2024-04-07T16:55:43.802 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Dcki4Jr8n4d-Fqo off=3547684 len=765198 2024-04-07T16:55:43.810 INFO:tasks.workunit.client.0.smithi100.stdout:op 5157 completed, throughput=5MB/sec 2024-04-07T16:55:43.810 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-7d3GVnaThnnxKGu off=3231080 len=939281 2024-04-07T16:55:43.819 INFO:tasks.workunit.client.0.smithi100.stdout:op 5158 completed, throughput=5MB/sec 2024-04-07T16:55:44.820 INFO:tasks.workunit.client.0.smithi100.stdout: 995: throughput=5MB/sec pending data=0 2024-04-07T16:55:44.820 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-sjWpIjmH4IIXRNs off=1305750 len=659143 2024-04-07T16:55:44.822 INFO:tasks.workunit.client.0.smithi100.stdout:op 5159 completed, throughput=5MB/sec 2024-04-07T16:55:44.822 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-flEVYSJRAfZZGot off=14072902 len=440188 2024-04-07T16:55:44.829 INFO:tasks.workunit.client.0.smithi100.stdout:op 5160 completed, throughput=5MB/sec 2024-04-07T16:55:44.829 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-vIcNpiBmgS30poT off=4156483 len=273482 2024-04-07T16:55:44.831 INFO:tasks.workunit.client.0.smithi100.stdout:op 5161 completed, throughput=5MB/sec 2024-04-07T16:55:44.831 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-t4uXCMK2o4eMl2b off=15734450 len=803934 2024-04-07T16:55:44.838 INFO:tasks.workunit.client.0.smithi100.stdout:op 5162 completed, throughput=5MB/sec 2024-04-07T16:55:44.839 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-i69-OSHAuYKX3s- off=5472600 len=916948 2024-04-07T16:55:44.841 INFO:tasks.workunit.client.0.smithi100.stdout:op 5163 completed, throughput=5MB/sec 2024-04-07T16:55:44.842 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-LS0GNoeIzK0OgPa off=5675156 len=397004 2024-04-07T16:55:44.843 INFO:tasks.workunit.client.0.smithi100.stdout:op 5164 completed, throughput=5MB/sec 2024-04-07T16:55:44.843 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-OWdAOKychE4DQ-Z off=177299 len=703038 2024-04-07T16:55:44.851 INFO:tasks.workunit.client.0.smithi100.stdout:op 5165 completed, throughput=5MB/sec 2024-04-07T16:55:44.852 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-4I3NS55mYgRevgN off=3043063 len=747010 2024-04-07T16:55:44.854 INFO:tasks.workunit.client.0.smithi100.stdout:op 5166 completed, throughput=5MB/sec 2024-04-07T16:55:45.112 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:55:44 smithi144 bash[66654]: cluster 2024-04-07T16:55:43.594921+0000 mgr.y (mgr.25175) 1211 : cluster [DBG] pgmap v1853: 105 pgs: 105 active+clean; 126 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 2.8 MiB/s rd, 2.9 MiB/s wr, 6 op/s 2024-04-07T16:55:45.199 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:55:44 smithi100 bash[74942]: cluster 2024-04-07T16:55:43.594921+0000 mgr.y (mgr.25175) 1211 : cluster [DBG] pgmap v1853: 105 pgs: 105 active+clean; 126 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 2.8 MiB/s rd, 2.9 MiB/s wr, 6 op/s 2024-04-07T16:55:45.199 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:55:44 smithi100 bash[76202]: cluster 2024-04-07T16:55:43.594921+0000 mgr.y (mgr.25175) 1211 : cluster [DBG] pgmap v1853: 105 pgs: 105 active+clean; 126 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 2.8 MiB/s rd, 2.9 MiB/s wr, 6 op/s 2024-04-07T16:55:45.854 INFO:tasks.workunit.client.0.smithi100.stdout: 996: throughput=5MB/sec pending data=0 2024-04-07T16:55:45.855 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-jwNxD4BI8nyh4x9 off=294791 len=1838640 2024-04-07T16:55:45.860 INFO:tasks.workunit.client.0.smithi100.stdout:op 5167 completed, throughput=5MB/sec 2024-04-07T16:55:45.860 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Sr2TLfBgPdunQkN off=10375398 len=1371903 2024-04-07T16:55:45.864 INFO:tasks.workunit.client.0.smithi100.stdout:op 5168 completed, throughput=5MB/sec 2024-04-07T16:55:45.864 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-kSDjAvzeVKyITD1 off=6942783 len=1055392 2024-04-07T16:55:45.875 INFO:tasks.workunit.client.0.smithi100.stdout:op 5169 completed, throughput=5MB/sec 2024-04-07T16:55:45.875 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-z_d_-ZnTs0lD1Ww off=5442693 len=2012785 2024-04-07T16:55:45.893 INFO:tasks.workunit.client.0.smithi100.stdout:op 5170 completed, throughput=5MB/sec 2024-04-07T16:55:46.893 INFO:tasks.workunit.client.0.smithi100.stdout: 997: throughput=5MB/sec pending data=0 2024-04-07T16:55:46.893 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-23ySYIZPxZbdET5 off=14128590 len=1688992 2024-04-07T16:55:46.907 INFO:tasks.workunit.client.0.smithi100.stdout:op 5171 completed, throughput=5MB/sec 2024-04-07T16:55:46.907 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-qSlfamtnDYFmnxm off=2514493 len=367813 2024-04-07T16:55:46.908 INFO:tasks.workunit.client.0.smithi100.stdout:op 5172 completed, throughput=5MB/sec 2024-04-07T16:55:46.909 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-dj0Cwqd2IjSPhGx off=878735 len=140589 2024-04-07T16:55:46.914 INFO:tasks.workunit.client.0.smithi100.stdout:op 5173 completed, throughput=5MB/sec 2024-04-07T16:55:46.914 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-d2wbMNh5t370naU off=5120269 len=777975 2024-04-07T16:55:46.921 INFO:tasks.workunit.client.0.smithi100.stdout:op 5174 completed, throughput=5MB/sec 2024-04-07T16:55:46.921 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-xiUervXqICRLt30 off=2858811 len=799877 2024-04-07T16:55:46.924 INFO:tasks.workunit.client.0.smithi100.stdout:op 5175 completed, throughput=5MB/sec 2024-04-07T16:55:46.924 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-rAGT4ScxDS5RVMd off=2615981 len=1637218 2024-04-07T16:55:46.930 INFO:tasks.workunit.client.0.smithi100.stdout:op 5176 completed, throughput=5MB/sec 2024-04-07T16:55:47.112 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:55:46 smithi144 bash[66654]: cluster 2024-04-07T16:55:45.595929+0000 mgr.y (mgr.25175) 1212 : cluster [DBG] pgmap v1854: 105 pgs: 105 active+clean; 126 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 2.0 MiB/s rd, 2.9 MiB/s wr, 5 op/s 2024-04-07T16:55:47.199 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:55:46 smithi100 bash[74942]: cluster 2024-04-07T16:55:45.595929+0000 mgr.y (mgr.25175) 1212 : cluster [DBG] pgmap v1854: 105 pgs: 105 active+clean; 126 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 2.0 MiB/s rd, 2.9 MiB/s wr, 5 op/s 2024-04-07T16:55:47.200 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:55:46 smithi100 bash[76202]: cluster 2024-04-07T16:55:45.595929+0000 mgr.y (mgr.25175) 1212 : cluster [DBG] pgmap v1854: 105 pgs: 105 active+clean; 126 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 2.0 MiB/s rd, 2.9 MiB/s wr, 5 op/s 2024-04-07T16:55:47.930 INFO:tasks.workunit.client.0.smithi100.stdout: 998: throughput=5MB/sec pending data=0 2024-04-07T16:55:47.930 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-H9Gc9ZkYqc864BR off=1063850 len=596991 2024-04-07T16:55:47.937 INFO:tasks.workunit.client.0.smithi100.stdout:op 5177 completed, throughput=5MB/sec 2024-04-07T16:55:47.937 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-LWag0cyDNe4Oc5g off=601352 len=199054 2024-04-07T16:55:47.942 INFO:tasks.workunit.client.0.smithi100.stdout:op 5178 completed, throughput=5MB/sec 2024-04-07T16:55:47.942 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-1fNCeOyPzqJxiYv off=2132673 len=293668 2024-04-07T16:55:47.948 INFO:tasks.workunit.client.0.smithi100.stdout:op 5179 completed, throughput=5MB/sec 2024-04-07T16:55:47.948 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-ruWkFdpsVvL9TM3 off=1447475 len=1757059 2024-04-07T16:55:47.952 INFO:tasks.workunit.client.0.smithi100.stdout:op 5180 completed, throughput=5MB/sec 2024-04-07T16:55:47.952 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-SiMFDxO1zioePD2 off=16119848 len=1318042 2024-04-07T16:55:47.956 INFO:tasks.workunit.client.0.smithi100.stdout:op 5181 completed, throughput=5MB/sec 2024-04-07T16:55:47.956 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Zpq2u96HuZ-0Ij9 off=6666700 len=1043043 2024-04-07T16:55:47.964 INFO:tasks.workunit.client.0.smithi100.stdout:op 5182 completed, throughput=5MB/sec 2024-04-07T16:55:48.112 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:55:47 smithi144 bash[66654]: audit 2024-04-07T16:55:46.785816+0000 mon.a (mon.0) 2193 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:55:48.199 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:55:47 smithi100 bash[74942]: audit 2024-04-07T16:55:46.785816+0000 mon.a (mon.0) 2193 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:55:48.199 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:55:47 smithi100 bash[76202]: audit 2024-04-07T16:55:46.785816+0000 mon.a (mon.0) 2193 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:55:48.699 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:55:48 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:55:48] "GET /metrics HTTP/1.1" 200 34072 "" "Prometheus/2.43.0" 2024-04-07T16:55:48.965 INFO:tasks.workunit.client.0.smithi100.stdout: 999: throughput=5MB/sec pending data=0 2024-04-07T16:55:48.965 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-W267mKfx74l2HwA off=1299673 len=556764 2024-04-07T16:55:48.973 INFO:tasks.workunit.client.0.smithi100.stdout:op 5183 completed, throughput=5MB/sec 2024-04-07T16:55:48.973 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-4ZJWwdoC5-a34Oq off=10837800 len=1346590 2024-04-07T16:55:48.976 INFO:tasks.workunit.client.0.smithi100.stdout:op 5184 completed, throughput=5MB/sec 2024-04-07T16:55:48.976 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-FkPYBU0hiEH1KO9 off=5236566 len=815747 2024-04-07T16:55:48.984 INFO:tasks.workunit.client.0.smithi100.stdout:op 5185 completed, throughput=5MB/sec 2024-04-07T16:55:48.984 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-BckVEUQXA4pplFF off=7629811 len=954881 2024-04-07T16:55:48.987 INFO:tasks.workunit.client.0.smithi100.stdout:op 5186 completed, throughput=5MB/sec 2024-04-07T16:55:48.987 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-d4f-9RvHWtP98cV off=6307258 len=825854 2024-04-07T16:55:48.989 INFO:tasks.workunit.client.0.smithi100.stdout:op 5187 completed, throughput=5MB/sec 2024-04-07T16:55:48.989 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-FGdPH3dIh6ESDP_ off=839604 len=183228 2024-04-07T16:55:48.994 INFO:tasks.workunit.client.0.smithi100.stdout:op 5188 completed, throughput=5MB/sec 2024-04-07T16:55:48.994 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-QlWiBVJkSq8DKer off=2081427 len=1222791 2024-04-07T16:55:49.004 INFO:tasks.workunit.client.0.smithi100.stdout:op 5189 completed, throughput=5MB/sec 2024-04-07T16:55:49.112 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:55:48 smithi144 bash[66654]: cluster 2024-04-07T16:55:47.597001+0000 mgr.y (mgr.25175) 1213 : cluster [DBG] pgmap v1855: 105 pgs: 105 active+clean; 126 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 2.0 MiB/s rd, 3.2 MiB/s wr, 5 op/s 2024-04-07T16:55:49.199 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:55:48 smithi100 bash[74942]: cluster 2024-04-07T16:55:47.597001+0000 mgr.y (mgr.25175) 1213 : cluster [DBG] pgmap v1855: 105 pgs: 105 active+clean; 126 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 2.0 MiB/s rd, 3.2 MiB/s wr, 5 op/s 2024-04-07T16:55:49.199 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:55:48 smithi100 bash[76202]: cluster 2024-04-07T16:55:47.597001+0000 mgr.y (mgr.25175) 1213 : cluster [DBG] pgmap v1855: 105 pgs: 105 active+clean; 126 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 2.0 MiB/s rd, 3.2 MiB/s wr, 5 op/s 2024-04-07T16:55:50.004 INFO:tasks.workunit.client.0.smithi100.stdout: 1000: throughput=5MB/sec pending data=0 2024-04-07T16:55:50.004 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-PUZXVj-smuyyo3P off=7338038 len=1057761 2024-04-07T16:55:50.007 INFO:tasks.workunit.client.0.smithi100.stdout:op 5190 completed, throughput=5MB/sec 2024-04-07T16:55:50.007 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-MQTwhbC7-cLlizt off=2463067 len=1547228 2024-04-07T16:55:50.020 INFO:tasks.workunit.client.0.smithi100.stdout:op 5191 completed, throughput=5MB/sec 2024-04-07T16:55:50.021 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-GePpjs1jtXw-ESC off=3148609 len=801448 2024-04-07T16:55:50.029 INFO:tasks.workunit.client.0.smithi100.stdout:op 5192 completed, throughput=5MB/sec 2024-04-07T16:55:50.029 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Xa_egBgoeuauJ87 off=952283 len=2000108 2024-04-07T16:55:50.046 INFO:tasks.workunit.client.0.smithi100.stdout:op 5193 completed, throughput=5MB/sec 2024-04-07T16:55:51.047 INFO:tasks.workunit.client.0.smithi100.stdout: 1001: throughput=5MB/sec pending data=0 2024-04-07T16:55:51.048 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-QCeOJ6dTX-A81WM off=584310 len=2060148 2024-04-07T16:55:51.062 INFO:tasks.workunit.client.0.smithi100.stdout:op 5194 completed, throughput=5MB/sec 2024-04-07T16:55:51.062 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj--H8GdKBES7emqMq off=3673246 len=945449 2024-04-07T16:55:51.072 INFO:tasks.workunit.client.0.smithi100.stdout:op 5195 completed, throughput=5MB/sec 2024-04-07T16:55:51.072 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-KsRg7i5Nbq5ArMP off=6553976 len=612992 2024-04-07T16:55:51.079 INFO:tasks.workunit.client.0.smithi100.stdout:op 5196 completed, throughput=5MB/sec 2024-04-07T16:55:51.079 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-BNLvde2pMSni-IQ off=17379246 len=1742902 2024-04-07T16:55:51.092 INFO:tasks.workunit.client.0.smithi100.stdout:op 5197 completed, throughput=5MB/sec 2024-04-07T16:55:51.112 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:55:50 smithi144 bash[66654]: cluster 2024-04-07T16:55:49.597979+0000 mgr.y (mgr.25175) 1214 : cluster [DBG] pgmap v1856: 105 pgs: 105 active+clean; 126 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 1.8 MiB/s rd, 2.9 MiB/s wr, 4 op/s 2024-04-07T16:55:51.199 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:55:50 smithi100 bash[74942]: cluster 2024-04-07T16:55:49.597979+0000 mgr.y (mgr.25175) 1214 : cluster [DBG] pgmap v1856: 105 pgs: 105 active+clean; 126 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 1.8 MiB/s rd, 2.9 MiB/s wr, 4 op/s 2024-04-07T16:55:51.199 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:55:50 smithi100 bash[76202]: cluster 2024-04-07T16:55:49.597979+0000 mgr.y (mgr.25175) 1214 : cluster [DBG] pgmap v1856: 105 pgs: 105 active+clean; 126 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 1.8 MiB/s rd, 2.9 MiB/s wr, 4 op/s 2024-04-07T16:55:52.092 INFO:tasks.workunit.client.0.smithi100.stdout: 1002: throughput=5MB/sec pending data=0 2024-04-07T16:55:52.093 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-5Xw2hBmgwW0Q4Vx off=3322775 len=1403561 2024-04-07T16:55:52.104 INFO:tasks.workunit.client.0.smithi100.stdout:op 5198 completed, throughput=5MB/sec 2024-04-07T16:55:52.105 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-v643O5U9HHyPNdx off=7570713 len=1509892 2024-04-07T16:55:52.108 INFO:tasks.workunit.client.0.smithi100.stdout:op 5199 completed, throughput=5MB/sec 2024-04-07T16:55:52.108 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Dd4sQv_4Ny0_JQ6 off=14184695 len=1460161 2024-04-07T16:55:52.112 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:55:51 smithi144 bash[66654]: cluster 2024-04-07T16:55:51.598776+0000 mgr.y (mgr.25175) 1215 : cluster [DBG] pgmap v1857: 105 pgs: 105 active+clean; 126 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 1.8 MiB/s rd, 2.6 MiB/s wr, 4 op/s 2024-04-07T16:55:52.120 INFO:tasks.workunit.client.0.smithi100.stdout:op 5200 completed, throughput=5MB/sec 2024-04-07T16:55:52.120 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-YF3GTVmLyqQ9cde off=4345725 len=189172 2024-04-07T16:55:52.125 INFO:tasks.workunit.client.0.smithi100.stdout:op 5201 completed, throughput=5MB/sec 2024-04-07T16:55:52.199 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:55:51 smithi100 bash[74942]: cluster 2024-04-07T16:55:51.598776+0000 mgr.y (mgr.25175) 1215 : cluster [DBG] pgmap v1857: 105 pgs: 105 active+clean; 126 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 1.8 MiB/s rd, 2.6 MiB/s wr, 4 op/s 2024-04-07T16:55:52.200 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:55:51 smithi100 bash[76202]: cluster 2024-04-07T16:55:51.598776+0000 mgr.y (mgr.25175) 1215 : cluster [DBG] pgmap v1857: 105 pgs: 105 active+clean; 126 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 1.8 MiB/s rd, 2.6 MiB/s wr, 4 op/s 2024-04-07T16:55:52.612 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:55:52 smithi144 bash[95233]: ts=2024-04-07T16:55:52.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:55:53.125 INFO:tasks.workunit.client.0.smithi100.stdout: 1003: throughput=5MB/sec pending data=0 2024-04-07T16:55:53.126 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-WfLGIM6GOKH4c_7 off=5138736 len=1471771 2024-04-07T16:55:53.138 INFO:tasks.workunit.client.0.smithi100.stdout:op 5202 completed, throughput=5MB/sec 2024-04-07T16:55:53.138 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-JmqqRFYG_O1na6g off=16636636 len=1735666 2024-04-07T16:55:53.140 INFO:tasks.workunit.client.0.smithi100.stdout:op 5203 completed, throughput=5MB/sec 2024-04-07T16:55:53.141 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-dLpAsIzKas_7fRZ off=7984083 len=2088565 2024-04-07T16:55:53.146 INFO:tasks.workunit.client.0.smithi100.stdout:op 5204 completed, throughput=5MB/sec 2024-04-07T16:55:53.146 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-WXMdbkp5pi2IqyE off=7837155 len=1494063 2024-04-07T16:55:53.158 INFO:tasks.workunit.client.0.smithi100.stdout:op 5205 completed, throughput=5MB/sec 2024-04-07T16:55:54.158 INFO:tasks.workunit.client.0.smithi100.stdout: 1004: throughput=5MB/sec pending data=0 2024-04-07T16:55:54.159 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-uAxCe2Xf5yEDG3Y off=3194696 len=1243463 2024-04-07T16:55:54.162 INFO:tasks.workunit.client.0.smithi100.stdout:op 5206 completed, throughput=5MB/sec 2024-04-07T16:55:54.162 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-uYg4bUvjxKMDfcO off=1664669 len=1940228 2024-04-07T16:55:54.178 INFO:tasks.workunit.client.0.smithi100.stdout:op 5207 completed, throughput=5MB/sec 2024-04-07T16:55:54.179 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-yLNiyvS-gjFVHEb off=5324394 len=1200193 2024-04-07T16:55:54.187 INFO:tasks.workunit.client.0.smithi100.stdout:op 5208 completed, throughput=5MB/sec 2024-04-07T16:55:54.949 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:55:54 smithi100 bash[74942]: cluster 2024-04-07T16:55:53.599871+0000 mgr.y (mgr.25175) 1216 : cluster [DBG] pgmap v1858: 105 pgs: 105 active+clean; 126 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 2.0 MiB/s rd, 2.9 MiB/s wr, 5 op/s 2024-04-07T16:55:54.949 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:55:54 smithi100 bash[76202]: cluster 2024-04-07T16:55:53.599871+0000 mgr.y (mgr.25175) 1216 : cluster [DBG] pgmap v1858: 105 pgs: 105 active+clean; 126 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 2.0 MiB/s rd, 2.9 MiB/s wr, 5 op/s 2024-04-07T16:55:55.112 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:55:54 smithi144 bash[66654]: cluster 2024-04-07T16:55:53.599871+0000 mgr.y (mgr.25175) 1216 : cluster [DBG] pgmap v1858: 105 pgs: 105 active+clean; 126 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 2.0 MiB/s rd, 2.9 MiB/s wr, 5 op/s 2024-04-07T16:55:55.187 INFO:tasks.workunit.client.0.smithi100.stdout: 1005: throughput=5MB/sec pending data=0 2024-04-07T16:55:55.188 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-vCAZ0L9PaWcCVnW off=6271616 len=1680950 2024-04-07T16:55:55.202 INFO:tasks.workunit.client.0.smithi100.stdout:op 5209 completed, throughput=5MB/sec 2024-04-07T16:55:55.203 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-92rMFHO1CDxV9aD off=2117138 len=681000 2024-04-07T16:55:55.205 INFO:tasks.workunit.client.0.smithi100.stdout:op 5210 completed, throughput=5MB/sec 2024-04-07T16:55:55.205 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-r7OBI6NCLh60LFf off=13968700 len=243554 2024-04-07T16:55:55.206 INFO:tasks.workunit.client.0.smithi100.stdout:op 5211 completed, throughput=5MB/sec 2024-04-07T16:55:55.206 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-fdtvVJpJ6d67Lyv off=2628497 len=1635269 2024-04-07T16:55:55.210 INFO:tasks.workunit.client.0.smithi100.stdout:op 5212 completed, throughput=5MB/sec 2024-04-07T16:55:55.210 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-kqKvs_-lbAUHkOR off=4631551 len=1602095 2024-04-07T16:55:55.214 INFO:tasks.workunit.client.0.smithi100.stdout:op 5213 completed, throughput=5MB/sec 2024-04-07T16:55:56.214 INFO:tasks.workunit.client.0.smithi100.stdout: 1006: throughput=5MB/sec pending data=0 2024-04-07T16:55:56.215 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-UUFVoFY7AWRLwYh off=13841284 len=1103792 2024-04-07T16:55:56.224 INFO:tasks.workunit.client.0.smithi100.stdout:op 5214 completed, throughput=5MB/sec 2024-04-07T16:55:56.224 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-_aJ8x79qW_zjeIl off=1802346 len=1228385 2024-04-07T16:55:56.227 INFO:tasks.workunit.client.0.smithi100.stdout:op 5215 completed, throughput=5MB/sec 2024-04-07T16:55:56.227 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-0ei7Wq2muYA2AyH off=492614 len=2086077 2024-04-07T16:55:56.234 INFO:tasks.workunit.client.0.smithi100.stdout:op 5216 completed, throughput=5MB/sec 2024-04-07T16:55:56.235 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-lNDin6ewa3pkSHl off=3113251 len=927769 2024-04-07T16:55:56.244 INFO:tasks.workunit.client.0.smithi100.stdout:op 5217 completed, throughput=5MB/sec 2024-04-07T16:55:56.949 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:55:56 smithi100 bash[74942]: cluster 2024-04-07T16:55:55.600931+0000 mgr.y (mgr.25175) 1217 : cluster [DBG] pgmap v1859: 105 pgs: 105 active+clean; 126 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 2.0 MiB/s rd, 3.0 MiB/s wr, 4 op/s 2024-04-07T16:55:56.949 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:55:56 smithi100 bash[76202]: cluster 2024-04-07T16:55:55.600931+0000 mgr.y (mgr.25175) 1217 : cluster [DBG] pgmap v1859: 105 pgs: 105 active+clean; 126 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 2.0 MiB/s rd, 3.0 MiB/s wr, 4 op/s 2024-04-07T16:55:57.113 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:55:56 smithi144 bash[66654]: cluster 2024-04-07T16:55:55.600931+0000 mgr.y (mgr.25175) 1217 : cluster [DBG] pgmap v1859: 105 pgs: 105 active+clean; 126 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 2.0 MiB/s rd, 3.0 MiB/s wr, 4 op/s 2024-04-07T16:55:57.245 INFO:tasks.workunit.client.0.smithi100.stdout: 1007: throughput=5MB/sec pending data=0 2024-04-07T16:55:57.245 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-C9pVGBqx3ZS85CS off=7377619 len=1133953 2024-04-07T16:55:57.255 INFO:tasks.workunit.client.0.smithi100.stdout:op 5218 completed, throughput=5MB/sec 2024-04-07T16:55:57.255 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-uB6Ndj5pEbcCX5J off=7421551 len=668774 2024-04-07T16:55:57.262 INFO:tasks.workunit.client.0.smithi100.stdout:op 5219 completed, throughput=5MB/sec 2024-04-07T16:55:57.262 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-gKFWFnnW1OibRxF off=11613306 len=897247 2024-04-07T16:55:57.265 INFO:tasks.workunit.client.0.smithi100.stdout:op 5220 completed, throughput=5MB/sec 2024-04-07T16:55:57.265 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-ELCbuxoppU0pJUk off=16377073 len=1945783 2024-04-07T16:55:57.282 INFO:tasks.workunit.client.0.smithi100.stdout:op 5221 completed, throughput=5MB/sec 2024-04-07T16:55:57.282 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-5Xw2hBmgwW0Q4Vx off=216516 len=1473911 2024-04-07T16:55:57.287 INFO:tasks.workunit.client.0.smithi100.stdout:op 5222 completed, throughput=5MB/sec 2024-04-07T16:55:58.287 INFO:tasks.workunit.client.0.smithi100.stdout: 1008: throughput=5MB/sec pending data=0 2024-04-07T16:55:58.287 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-97uktZq_zUPh2TA off=17173154 len=1870925 2024-04-07T16:55:58.301 INFO:tasks.workunit.client.0.smithi100.stdout:op 5223 completed, throughput=5MB/sec 2024-04-07T16:55:58.301 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-jRwqjDUoCGVwQmn off=4073331 len=882883 2024-04-07T16:55:58.305 INFO:tasks.workunit.client.0.smithi100.stdout:op 5224 completed, throughput=5MB/sec 2024-04-07T16:55:58.305 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-1mjJAXwhiA6GLo6 off=14937178 len=1410353 2024-04-07T16:55:58.316 INFO:tasks.workunit.client.0.smithi100.stdout:op 5225 completed, throughput=5MB/sec 2024-04-07T16:55:58.699 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:55:58 smithi100 bash[74942]: cluster 2024-04-07T16:55:57.602036+0000 mgr.y (mgr.25175) 1218 : cluster [DBG] pgmap v1860: 105 pgs: 105 active+clean; 126 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 2.0 MiB/s rd, 2.6 MiB/s wr, 4 op/s 2024-04-07T16:55:58.699 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:55:58 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:55:58] "GET /metrics HTTP/1.1" 200 34072 "" "Prometheus/2.43.0" 2024-04-07T16:55:58.700 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:55:58 smithi100 bash[76202]: cluster 2024-04-07T16:55:57.602036+0000 mgr.y (mgr.25175) 1218 : cluster [DBG] pgmap v1860: 105 pgs: 105 active+clean; 126 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 2.0 MiB/s rd, 2.6 MiB/s wr, 4 op/s 2024-04-07T16:55:59.112 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:55:58 smithi144 bash[66654]: cluster 2024-04-07T16:55:57.602036+0000 mgr.y (mgr.25175) 1218 : cluster [DBG] pgmap v1860: 105 pgs: 105 active+clean; 126 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 2.0 MiB/s rd, 2.6 MiB/s wr, 4 op/s 2024-04-07T16:55:59.316 INFO:tasks.workunit.client.0.smithi100.stdout: 1009: throughput=5MB/sec pending data=0 2024-04-07T16:55:59.316 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-bbtyafYni92fmaE off=5056639 len=1002026 2024-04-07T16:55:59.319 INFO:tasks.workunit.client.0.smithi100.stdout:op 5226 completed, throughput=5MB/sec 2024-04-07T16:55:59.319 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-BRoQVuOrGK0cWQl off=6166047 len=1545815 2024-04-07T16:55:59.324 INFO:tasks.workunit.client.0.smithi100.stdout:op 5227 completed, throughput=5MB/sec 2024-04-07T16:55:59.325 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-yl_rvSbbq8OemOP off=2791601 len=807607 2024-04-07T16:55:59.332 INFO:tasks.workunit.client.0.smithi100.stdout:op 5228 completed, throughput=5MB/sec 2024-04-07T16:55:59.333 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-5Dd_b7kgNaRac59 off=5439869 len=419106 2024-04-07T16:55:59.339 INFO:tasks.workunit.client.0.smithi100.stdout:op 5229 completed, throughput=5MB/sec 2024-04-07T16:55:59.339 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-PpK5NBzoGkZPWyl off=21755184 len=126041 2024-04-07T16:55:59.343 INFO:tasks.workunit.client.0.smithi100.stdout:op 5230 completed, throughput=5MB/sec 2024-04-07T16:55:59.343 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-GUAZKEQuBkaBkKV off=5112155 len=600499 2024-04-07T16:55:59.345 INFO:tasks.workunit.client.0.smithi100.stdout:op 5231 completed, throughput=5MB/sec 2024-04-07T16:55:59.345 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-9_PEo9PwsYZTqtP off=1449608 len=1009633 2024-04-07T16:55:59.354 INFO:tasks.workunit.client.0.smithi100.stdout:op 5232 completed, throughput=5MB/sec 2024-04-07T16:56:00.355 INFO:tasks.workunit.client.0.smithi100.stdout: 1010: throughput=5MB/sec pending data=0 2024-04-07T16:56:00.355 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-yUFmGIXJBDI3fGc off=5612831 len=1525256 2024-04-07T16:56:00.360 INFO:tasks.workunit.client.0.smithi100.stdout:op 5233 completed, throughput=5MB/sec 2024-04-07T16:56:00.360 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-ViTcwYCwIggoH14 off=3137155 len=396488 2024-04-07T16:56:00.362 INFO:tasks.workunit.client.0.smithi100.stdout:op 5234 completed, throughput=5MB/sec 2024-04-07T16:56:00.362 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-bLIdFRalv2MH-oO off=4004279 len=135568 2024-04-07T16:56:00.367 INFO:tasks.workunit.client.0.smithi100.stdout:op 5235 completed, throughput=5MB/sec 2024-04-07T16:56:00.367 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-VV0TedwmZOswPEF off=2527229 len=435538 2024-04-07T16:56:00.373 INFO:tasks.workunit.client.0.smithi100.stdout:op 5236 completed, throughput=5MB/sec 2024-04-07T16:56:00.373 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-OKs_OPumEmeOw9a off=6099479 len=1819332 2024-04-07T16:56:00.377 INFO:tasks.workunit.client.0.smithi100.stdout:op 5237 completed, throughput=5MB/sec 2024-04-07T16:56:00.378 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-F_fmPd7SyxDNsMJ off=2363788 len=108721 2024-04-07T16:56:00.378 INFO:tasks.workunit.client.0.smithi100.stdout:op 5238 completed, throughput=5MB/sec 2024-04-07T16:56:00.379 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-1MukubLcFckEhX8 off=1836038 len=45728 2024-04-07T16:56:00.380 INFO:tasks.workunit.client.0.smithi100.stdout:op 5239 completed, throughput=5MB/sec 2024-04-07T16:56:00.380 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-EUCfNUgwwv07-v3 off=5374214 len=2085506 2024-04-07T16:56:00.384 INFO:tasks.workunit.client.0.smithi100.stdout:op 5240 completed, throughput=5MB/sec 2024-04-07T16:56:00.949 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:56:00 smithi100 bash[74942]: cluster 2024-04-07T16:55:59.603189+0000 mgr.y (mgr.25175) 1219 : cluster [DBG] pgmap v1861: 105 pgs: 105 active+clean; 126 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 1.8 MiB/s rd, 2.5 MiB/s wr, 3 op/s 2024-04-07T16:56:00.949 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:56:00 smithi100 bash[76202]: cluster 2024-04-07T16:55:59.603189+0000 mgr.y (mgr.25175) 1219 : cluster [DBG] pgmap v1861: 105 pgs: 105 active+clean; 126 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 1.8 MiB/s rd, 2.5 MiB/s wr, 3 op/s 2024-04-07T16:56:01.112 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:56:00 smithi144 bash[66654]: cluster 2024-04-07T16:55:59.603189+0000 mgr.y (mgr.25175) 1219 : cluster [DBG] pgmap v1861: 105 pgs: 105 active+clean; 126 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 1.8 MiB/s rd, 2.5 MiB/s wr, 3 op/s 2024-04-07T16:56:01.385 INFO:tasks.workunit.client.0.smithi100.stdout: 1011: throughput=5MB/sec pending data=0 2024-04-07T16:56:01.385 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-1OapbGCF-ndYLe7 off=2289581 len=1364902 2024-04-07T16:56:01.397 INFO:tasks.workunit.client.0.smithi100.stdout:op 5241 completed, throughput=5MB/sec 2024-04-07T16:56:01.397 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-7-4y97IxEeWJ4EU off=10735933 len=1815619 2024-04-07T16:56:01.415 INFO:tasks.workunit.client.0.smithi100.stdout:op 5242 completed, throughput=5MB/sec 2024-04-07T16:56:01.415 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-RxuUSFbcpiW7muJ off=214533 len=178965 2024-04-07T16:56:01.420 INFO:tasks.workunit.client.0.smithi100.stdout:op 5243 completed, throughput=5MB/sec 2024-04-07T16:56:01.420 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-p9awTzCWoEYUpP1 off=2056511 len=781688 2024-04-07T16:56:01.429 INFO:tasks.workunit.client.0.smithi100.stdout:op 5244 completed, throughput=5MB/sec 2024-04-07T16:56:02.429 INFO:tasks.workunit.client.0.smithi100.stdout: 1012: throughput=5MB/sec pending data=0 2024-04-07T16:56:02.430 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-J0QMDnOoYkBQpFS off=1587474 len=226257 2024-04-07T16:56:02.435 INFO:tasks.workunit.client.0.smithi100.stdout:op 5245 completed, throughput=5MB/sec 2024-04-07T16:56:02.435 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-AEA0r44zCutQ32O off=5858030 len=177634 2024-04-07T16:56:02.436 INFO:tasks.workunit.client.0.smithi100.stdout:op 5246 completed, throughput=5MB/sec 2024-04-07T16:56:02.437 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-bEOqVI362DnEe7Y off=1405109 len=1753419 2024-04-07T16:56:02.449 INFO:tasks.workunit.client.0.smithi100.stdout:op 5247 completed, throughput=5MB/sec 2024-04-07T16:56:02.450 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-R0OTNd5v6ssLuEn off=11783145 len=4575 2024-04-07T16:56:02.454 INFO:tasks.workunit.client.0.smithi100.stdout:op 5248 completed, throughput=5MB/sec 2024-04-07T16:56:02.454 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-qPbJdzJgIvkSG4g off=11704793 len=136263 2024-04-07T16:56:02.455 INFO:tasks.workunit.client.0.smithi100.stdout:op 5249 completed, throughput=5MB/sec 2024-04-07T16:56:02.455 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-ptos64hiXSYQn6j off=5050427 len=1220825 2024-04-07T16:56:02.460 INFO:tasks.workunit.client.0.smithi100.stdout:op 5250 completed, throughput=5MB/sec 2024-04-07T16:56:02.460 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-9G3DYDvJScilKSH off=1892284 len=1101099 2024-04-07T16:56:02.463 INFO:tasks.workunit.client.0.smithi100.stdout:op 5251 completed, throughput=5MB/sec 2024-04-07T16:56:02.463 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-pR1htuW5EpVp6XA off=107830 len=548184 2024-04-07T16:56:02.465 INFO:tasks.workunit.client.0.smithi100.stdout:op 5252 completed, throughput=5MB/sec 2024-04-07T16:56:02.465 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-kX3xNrH3yMNXgwO off=657772 len=686445 2024-04-07T16:56:02.472 INFO:tasks.workunit.client.0.smithi100.stdout:op 5253 completed, throughput=5MB/sec 2024-04-07T16:56:02.613 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:56:02 smithi144 bash[95233]: ts=2024-04-07T16:56:02.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:56:02.949 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:56:02 smithi100 bash[74942]: cluster 2024-04-07T16:56:01.604172+0000 mgr.y (mgr.25175) 1220 : cluster [DBG] pgmap v1862: 105 pgs: 105 active+clean; 126 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 2.0 MiB/s rd, 2.7 MiB/s wr, 4 op/s 2024-04-07T16:56:02.949 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:56:02 smithi100 bash[74942]: audit 2024-04-07T16:56:01.786226+0000 mon.a (mon.0) 2194 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:56:02.949 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:56:02 smithi100 bash[76202]: cluster 2024-04-07T16:56:01.604172+0000 mgr.y (mgr.25175) 1220 : cluster [DBG] pgmap v1862: 105 pgs: 105 active+clean; 126 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 2.0 MiB/s rd, 2.7 MiB/s wr, 4 op/s 2024-04-07T16:56:02.950 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:56:02 smithi100 bash[76202]: audit 2024-04-07T16:56:01.786226+0000 mon.a (mon.0) 2194 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:56:03.113 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:56:02 smithi144 bash[66654]: cluster 2024-04-07T16:56:01.604172+0000 mgr.y (mgr.25175) 1220 : cluster [DBG] pgmap v1862: 105 pgs: 105 active+clean; 126 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 2.0 MiB/s rd, 2.7 MiB/s wr, 4 op/s 2024-04-07T16:56:03.113 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:56:02 smithi144 bash[66654]: audit 2024-04-07T16:56:01.786226+0000 mon.a (mon.0) 2194 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:56:03.473 INFO:tasks.workunit.client.0.smithi100.stdout: 1013: throughput=5MB/sec pending data=0 2024-04-07T16:56:03.473 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-deS9E30zRIFG26s off=3786394 len=1608533 2024-04-07T16:56:03.486 INFO:tasks.workunit.client.0.smithi100.stdout:op 5254 completed, throughput=5MB/sec 2024-04-07T16:56:03.486 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-IJF0cosa3DKWyf9 off=5996130 len=1805310 2024-04-07T16:56:03.492 INFO:tasks.workunit.client.0.smithi100.stdout:op 5255 completed, throughput=5MB/sec 2024-04-07T16:56:03.492 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-U1hiZfpg07MV7iD off=17470511 len=2014423 2024-04-07T16:56:03.510 INFO:tasks.workunit.client.0.smithi100.stdout:op 5256 completed, throughput=5MB/sec 2024-04-07T16:56:04.510 INFO:tasks.workunit.client.0.smithi100.stdout: 1014: throughput=5MB/sec pending data=0 2024-04-07T16:56:04.510 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-g5S6icEweOTZ7QO off=19539227 len=2066474 2024-04-07T16:56:04.514 INFO:tasks.workunit.client.0.smithi100.stdout:op 5257 completed, throughput=5MB/sec 2024-04-07T16:56:04.515 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-vmPa1GJ2bAiaub_ off=18995689 len=1662070 2024-04-07T16:56:04.520 INFO:tasks.workunit.client.0.smithi100.stdout:op 5258 completed, throughput=5MB/sec 2024-04-07T16:56:04.520 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-LwV99TyOiUhNwSr off=1765699 len=1533188 2024-04-07T16:56:04.525 INFO:tasks.workunit.client.0.smithi100.stdout:op 5259 completed, throughput=5MB/sec 2024-04-07T16:56:04.949 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:56:04 smithi100 bash[76202]: cluster 2024-04-07T16:56:03.605457+0000 mgr.y (mgr.25175) 1221 : cluster [DBG] pgmap v1863: 105 pgs: 105 active+clean; 126 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 2.7 MiB/s rd, 3.3 MiB/s wr, 5 op/s 2024-04-07T16:56:04.949 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:56:04 smithi100 bash[74942]: cluster 2024-04-07T16:56:03.605457+0000 mgr.y (mgr.25175) 1221 : cluster [DBG] pgmap v1863: 105 pgs: 105 active+clean; 126 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 2.7 MiB/s rd, 3.3 MiB/s wr, 5 op/s 2024-04-07T16:56:05.112 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:56:04 smithi144 bash[66654]: cluster 2024-04-07T16:56:03.605457+0000 mgr.y (mgr.25175) 1221 : cluster [DBG] pgmap v1863: 105 pgs: 105 active+clean; 126 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 2.7 MiB/s rd, 3.3 MiB/s wr, 5 op/s 2024-04-07T16:56:05.525 INFO:tasks.workunit.client.0.smithi100.stdout: 1015: throughput=5MB/sec pending data=0 2024-04-07T16:56:05.525 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-SQb-4KONggjUnIT off=13007537 len=87105 2024-04-07T16:56:05.526 INFO:tasks.workunit.client.0.smithi100.stdout:op 5260 completed, throughput=5MB/sec 2024-04-07T16:56:05.526 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-r42E-0GAJNVZege off=22682573 len=1181044 2024-04-07T16:56:05.529 INFO:tasks.workunit.client.0.smithi100.stdout:op 5261 completed, throughput=5MB/sec 2024-04-07T16:56:05.529 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-akMTSJODG0z9sRk off=11977378 len=1712607 2024-04-07T16:56:05.534 INFO:tasks.workunit.client.0.smithi100.stdout:op 5262 completed, throughput=5MB/sec 2024-04-07T16:56:05.534 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-aHCKNPf68jWoVlF off=3559253 len=116982 2024-04-07T16:56:05.538 INFO:tasks.workunit.client.0.smithi100.stdout:op 5263 completed, throughput=5MB/sec 2024-04-07T16:56:05.538 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-nnnTnKdJFjN4hHG off=1413343 len=1456333 2024-04-07T16:56:05.538 INFO:tasks.workunit.client.0.smithi100.stdout:op 5264 completed, throughput=5MB/sec 2024-04-07T16:56:05.538 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-4k8yGS9z7-oRznd off=18133084 len=851013 2024-04-07T16:56:05.540 INFO:tasks.workunit.client.0.smithi100.stdout:op 5265 completed, throughput=5MB/sec 2024-04-07T16:56:06.541 INFO:tasks.workunit.client.0.smithi100.stdout: 1016: throughput=5MB/sec pending data=0 2024-04-07T16:56:06.541 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-TPrHDx8FbiRE8Mn off=1464417 len=1197685 2024-04-07T16:56:06.543 INFO:tasks.workunit.client.0.smithi100.stdout:op 5266 completed, throughput=5MB/sec 2024-04-07T16:56:06.543 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-h4kMwhvv3EvlrSx off=15036551 len=833155 2024-04-07T16:56:06.552 INFO:tasks.workunit.client.0.smithi100.stdout:op 5267 completed, throughput=5MB/sec 2024-04-07T16:56:06.552 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-3_OSQKDLgd-2xV8 off=125998 len=1166707 2024-04-07T16:56:06.556 INFO:tasks.workunit.client.0.smithi100.stdout:op 5268 completed, throughput=5MB/sec 2024-04-07T16:56:06.556 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-AIzwdkxFGdjaGiJ off=405764 len=1779749 2024-04-07T16:56:06.560 INFO:tasks.workunit.client.0.smithi100.stdout:op 5269 completed, throughput=5MB/sec 2024-04-07T16:56:06.949 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:56:06 smithi100 bash[74942]: cluster 2024-04-07T16:56:05.606572+0000 mgr.y (mgr.25175) 1222 : cluster [DBG] pgmap v1864: 105 pgs: 105 active+clean; 126 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.9 MiB/s wr, 4 op/s 2024-04-07T16:56:06.949 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:56:06 smithi100 bash[76202]: cluster 2024-04-07T16:56:05.606572+0000 mgr.y (mgr.25175) 1222 : cluster [DBG] pgmap v1864: 105 pgs: 105 active+clean; 126 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.9 MiB/s wr, 4 op/s 2024-04-07T16:56:07.112 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:56:06 smithi144 bash[66654]: cluster 2024-04-07T16:56:05.606572+0000 mgr.y (mgr.25175) 1222 : cluster [DBG] pgmap v1864: 105 pgs: 105 active+clean; 126 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.9 MiB/s wr, 4 op/s 2024-04-07T16:56:07.562 INFO:tasks.workunit.client.0.smithi100.stdout: 1017: throughput=5MB/sec pending data=0 2024-04-07T16:56:07.562 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-SGPvjz1MGFoms6i off=2090714 len=551818 2024-04-07T16:56:07.563 INFO:tasks.workunit.client.0.smithi100.stdout:op 5270 completed, throughput=5MB/sec 2024-04-07T16:56:07.563 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-0HuAUSI6VXdmokp off=365731 len=678951 2024-04-07T16:56:07.573 INFO:tasks.workunit.client.0.smithi100.stdout:op 5271 completed, throughput=5MB/sec 2024-04-07T16:56:07.573 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-9pB9opJWB_2fDu_ off=1291235 len=1711621 2024-04-07T16:56:07.577 INFO:tasks.workunit.client.0.smithi100.stdout:op 5272 completed, throughput=5MB/sec 2024-04-07T16:56:07.578 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-ntdqdmxqIozykx1 off=282885 len=686687 2024-04-07T16:56:07.583 INFO:tasks.workunit.client.0.smithi100.stdout:op 5273 completed, throughput=5MB/sec 2024-04-07T16:56:07.583 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-ovCQDU9guUA3otN off=598483 len=1754611 2024-04-07T16:56:07.584 INFO:tasks.workunit.client.0.smithi100.stdout:op 5274 completed, throughput=5MB/sec 2024-04-07T16:56:08.584 INFO:tasks.workunit.client.0.smithi100.stdout: 1018: throughput=5MB/sec pending data=0 2024-04-07T16:56:08.584 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-eJrI0kN7EX5PAgo off=16921687 len=988006 2024-04-07T16:56:08.588 INFO:tasks.workunit.client.0.smithi100.stdout:op 5275 completed, throughput=5MB/sec 2024-04-07T16:56:08.588 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Nzns7WhPkF5s366 off=2033327 len=1489769 2024-04-07T16:56:08.600 INFO:tasks.workunit.client.0.smithi100.stdout:op 5276 completed, throughput=5MB/sec 2024-04-07T16:56:08.600 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-TXiTHQQELNnZqOJ off=18207161 len=5763 2024-04-07T16:56:08.604 INFO:tasks.workunit.client.0.smithi100.stdout:op 5277 completed, throughput=5MB/sec 2024-04-07T16:56:08.604 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-VHHHPsPjdwsxvvo off=11362547 len=807190 2024-04-07T16:56:08.606 INFO:tasks.workunit.client.0.smithi100.stdout:op 5278 completed, throughput=5MB/sec 2024-04-07T16:56:08.606 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-vmyjORa6AqP0bi5 off=8634953 len=930691 2024-04-07T16:56:08.615 INFO:tasks.workunit.client.0.smithi100.stdout:op 5279 completed, throughput=5MB/sec 2024-04-07T16:56:08.616 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-7322um-TYrsk7cp off=231702 len=872354 2024-04-07T16:56:08.625 INFO:tasks.workunit.client.0.smithi100.stdout:op 5280 completed, throughput=5MB/sec 2024-04-07T16:56:08.625 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-ssGrshbgNtEci1w off=21035599 len=1232068 2024-04-07T16:56:08.631 INFO:tasks.workunit.client.0.smithi100.stdout:op 5281 completed, throughput=5MB/sec 2024-04-07T16:56:08.686 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:56:08 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:56:08] "GET /metrics HTTP/1.1" 200 34075 "" "Prometheus/2.43.0" 2024-04-07T16:56:08.949 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:56:08 smithi100 bash[74942]: cluster 2024-04-07T16:56:07.607852+0000 mgr.y (mgr.25175) 1223 : cluster [DBG] pgmap v1865: 105 pgs: 105 active+clean; 126 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.3 MiB/s wr, 4 op/s 2024-04-07T16:56:08.949 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:56:08 smithi100 bash[76202]: cluster 2024-04-07T16:56:07.607852+0000 mgr.y (mgr.25175) 1223 : cluster [DBG] pgmap v1865: 105 pgs: 105 active+clean; 126 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.3 MiB/s wr, 4 op/s 2024-04-07T16:56:09.112 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:56:08 smithi144 bash[66654]: cluster 2024-04-07T16:56:07.607852+0000 mgr.y (mgr.25175) 1223 : cluster [DBG] pgmap v1865: 105 pgs: 105 active+clean; 126 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.3 MiB/s wr, 4 op/s 2024-04-07T16:56:09.629 INFO:tasks.workunit.client.0.smithi100.stdout: 1019: throughput=5MB/sec pending data=0 2024-04-07T16:56:09.629 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-qdidzmysWWqzvCc off=8848603 len=1034760 2024-04-07T16:56:09.632 INFO:tasks.workunit.client.0.smithi100.stdout:op 5282 completed, throughput=5MB/sec 2024-04-07T16:56:09.633 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-NF6OEj_wIAE2LJ7 off=5633860 len=959850 2024-04-07T16:56:09.636 INFO:tasks.workunit.client.0.smithi100.stdout:op 5283 completed, throughput=5MB/sec 2024-04-07T16:56:09.636 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-nBydzPxALQEMvUI off=12570418 len=1964115 2024-04-07T16:56:09.652 INFO:tasks.workunit.client.0.smithi100.stdout:op 5284 completed, throughput=5MB/sec 2024-04-07T16:56:09.652 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-XH2Owj0iKmI-l3G off=4219583 len=1347945 2024-04-07T16:56:09.655 INFO:tasks.workunit.client.0.smithi100.stdout:op 5285 completed, throughput=5MB/sec 2024-04-07T16:56:09.949 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:56:09 smithi100 bash[74942]: audit 2024-04-07T16:56:09.531802+0000 mon.a (mon.0) 2195 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:56:09.949 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:56:09 smithi100 bash[76202]: audit 2024-04-07T16:56:09.531802+0000 mon.a (mon.0) 2195 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:56:10.112 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:56:09 smithi144 bash[66654]: audit 2024-04-07T16:56:09.531802+0000 mon.a (mon.0) 2195 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:56:10.655 INFO:tasks.workunit.client.0.smithi100.stdout: 1020: throughput=5MB/sec pending data=0 2024-04-07T16:56:10.656 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-nXVZjLx1VWa2ajK off=9583091 len=1366027 2024-04-07T16:56:10.659 INFO:tasks.workunit.client.0.smithi100.stdout:op 5286 completed, throughput=5MB/sec 2024-04-07T16:56:10.660 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-2Gj-m9GqXLAc1eM off=15909640 len=1208587 2024-04-07T16:56:10.670 INFO:tasks.workunit.client.0.smithi100.stdout:op 5287 completed, throughput=5MB/sec 2024-04-07T16:56:10.670 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-a_1k2YHg2iu6Y4b off=479642 len=56825 2024-04-07T16:56:10.673 INFO:tasks.workunit.client.0.smithi100.stdout:op 5288 completed, throughput=5MB/sec 2024-04-07T16:56:10.674 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-ShNI8gyGlFNA2e5 off=1032528 len=946018 2024-04-07T16:56:10.680 INFO:tasks.workunit.client.0.smithi100.stdout:op 5289 completed, throughput=5MB/sec 2024-04-07T16:56:10.680 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-whmrNh2GUMlBqeS off=4641104 len=1512810 2024-04-07T16:56:10.684 INFO:tasks.workunit.client.0.smithi100.stdout:op 5290 completed, throughput=5MB/sec 2024-04-07T16:56:10.949 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:56:10 smithi100 bash[74942]: cluster 2024-04-07T16:56:09.608907+0000 mgr.y (mgr.25175) 1224 : cluster [DBG] pgmap v1866: 105 pgs: 105 active+clean; 126 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 2.6 MiB/s rd, 2.3 MiB/s wr, 4 op/s 2024-04-07T16:56:10.949 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:56:10 smithi100 bash[74942]: audit 2024-04-07T16:56:10.335814+0000 mon.a (mon.0) 2196 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:56:10.949 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:56:10 smithi100 bash[74942]: audit 2024-04-07T16:56:10.337577+0000 mon.a (mon.0) 2197 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:56:10.950 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:56:10 smithi100 bash[74942]: audit 2024-04-07T16:56:10.347365+0000 mon.a (mon.0) 2198 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:56:10.950 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:56:10 smithi100 bash[76202]: cluster 2024-04-07T16:56:09.608907+0000 mgr.y (mgr.25175) 1224 : cluster [DBG] pgmap v1866: 105 pgs: 105 active+clean; 126 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 2.6 MiB/s rd, 2.3 MiB/s wr, 4 op/s 2024-04-07T16:56:10.950 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:56:10 smithi100 bash[76202]: audit 2024-04-07T16:56:10.335814+0000 mon.a (mon.0) 2196 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:56:10.950 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:56:10 smithi100 bash[76202]: audit 2024-04-07T16:56:10.337577+0000 mon.a (mon.0) 2197 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:56:10.950 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:56:10 smithi100 bash[76202]: audit 2024-04-07T16:56:10.347365+0000 mon.a (mon.0) 2198 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:56:11.113 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:56:10 smithi144 bash[66654]: cluster 2024-04-07T16:56:09.608907+0000 mgr.y (mgr.25175) 1224 : cluster [DBG] pgmap v1866: 105 pgs: 105 active+clean; 126 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 2.6 MiB/s rd, 2.3 MiB/s wr, 4 op/s 2024-04-07T16:56:11.113 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:56:10 smithi144 bash[66654]: audit 2024-04-07T16:56:10.335814+0000 mon.a (mon.0) 2196 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:56:11.113 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:56:10 smithi144 bash[66654]: audit 2024-04-07T16:56:10.337577+0000 mon.a (mon.0) 2197 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:56:11.113 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:56:10 smithi144 bash[66654]: audit 2024-04-07T16:56:10.347365+0000 mon.a (mon.0) 2198 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:56:11.684 INFO:tasks.workunit.client.0.smithi100.stdout: 1021: throughput=5MB/sec pending data=0 2024-04-07T16:56:11.684 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-LB1sTDJmiwjTgkB off=2990250 len=1190251 2024-04-07T16:56:11.688 INFO:tasks.workunit.client.0.smithi100.stdout:op 5291 completed, throughput=5MB/sec 2024-04-07T16:56:11.688 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-dX-4IBuoTsQAN4D off=14914048 len=199249 2024-04-07T16:56:11.689 INFO:tasks.workunit.client.0.smithi100.stdout:op 5292 completed, throughput=5MB/sec 2024-04-07T16:56:11.689 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-APBr_nwigZKKgqw off=1415267 len=675021 2024-04-07T16:56:11.697 INFO:tasks.workunit.client.0.smithi100.stdout:op 5293 completed, throughput=5MB/sec 2024-04-07T16:56:11.697 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-yqWnC9Jk0gdZjep off=18076670 len=687773 2024-04-07T16:56:11.699 INFO:tasks.workunit.client.0.smithi100.stdout:op 5294 completed, throughput=5MB/sec 2024-04-07T16:56:11.699 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-BgUagh3dXFQnrKQ off=1041624 len=1986757 2024-04-07T16:56:11.717 INFO:tasks.workunit.client.0.smithi100.stdout:op 5295 completed, throughput=5MB/sec 2024-04-07T16:56:11.717 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Sls3dDfmhEzenVe off=10685490 len=951676 2024-04-07T16:56:11.720 INFO:tasks.workunit.client.0.smithi100.stdout:op 5296 completed, throughput=5MB/sec 2024-04-07T16:56:12.613 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:56:12 smithi144 bash[95233]: ts=2024-04-07T16:56:12.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:56:12.720 INFO:tasks.workunit.client.0.smithi100.stdout: 1022: throughput=5MB/sec pending data=0 2024-04-07T16:56:12.720 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Dd4sQv_4Ny0_JQ6 off=11967891 len=820656 2024-04-07T16:56:12.723 INFO:tasks.workunit.client.0.smithi100.stdout:op 5297 completed, throughput=5MB/sec 2024-04-07T16:56:12.723 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-aFVV6_TGMr_7F2E off=10251907 len=311550 2024-04-07T16:56:12.729 INFO:tasks.workunit.client.0.smithi100.stdout:op 5298 completed, throughput=5MB/sec 2024-04-07T16:56:12.729 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-oXKnCf9JFZEGzNn off=3988683 len=32710 2024-04-07T16:56:12.733 INFO:tasks.workunit.client.0.smithi100.stdout:op 5299 completed, throughput=5MB/sec 2024-04-07T16:56:12.733 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-9PlJSZtInIkEu5O off=1349968 len=1803589 2024-04-07T16:56:12.738 INFO:tasks.workunit.client.0.smithi100.stdout:op 5300 completed, throughput=5MB/sec 2024-04-07T16:56:12.738 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-9YE0_xiz3dNq3lt off=10961722 len=1620480 2024-04-07T16:56:12.744 INFO:tasks.workunit.client.0.smithi100.stdout:op 5301 completed, throughput=5MB/sec 2024-04-07T16:56:13.112 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:56:12 smithi144 bash[66654]: cluster 2024-04-07T16:56:11.609730+0000 mgr.y (mgr.25175) 1225 : cluster [DBG] pgmap v1867: 105 pgs: 105 active+clean; 126 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 3.2 MiB/s rd, 2.0 MiB/s wr, 5 op/s 2024-04-07T16:56:13.199 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:56:12 smithi100 bash[74942]: cluster 2024-04-07T16:56:11.609730+0000 mgr.y (mgr.25175) 1225 : cluster [DBG] pgmap v1867: 105 pgs: 105 active+clean; 126 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 3.2 MiB/s rd, 2.0 MiB/s wr, 5 op/s 2024-04-07T16:56:13.199 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:56:12 smithi100 bash[76202]: cluster 2024-04-07T16:56:11.609730+0000 mgr.y (mgr.25175) 1225 : cluster [DBG] pgmap v1867: 105 pgs: 105 active+clean; 126 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 3.2 MiB/s rd, 2.0 MiB/s wr, 5 op/s 2024-04-07T16:56:13.745 INFO:tasks.workunit.client.0.smithi100.stdout: 1023: throughput=5MB/sec pending data=0 2024-04-07T16:56:13.745 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-5MuxriVZLyXzxpd off=4087632 len=1345986 2024-04-07T16:56:13.756 INFO:tasks.workunit.client.0.smithi100.stdout:op 5302 completed, throughput=5MB/sec 2024-04-07T16:56:13.756 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-V9NM_kf10-yMZcZ off=10843118 len=1414478 2024-04-07T16:56:13.767 INFO:tasks.workunit.client.0.smithi100.stdout:op 5303 completed, throughput=5MB/sec 2024-04-07T16:56:13.768 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-E0Y91H0srElbjVs off=21592410 len=1736557 2024-04-07T16:56:13.772 INFO:tasks.workunit.client.0.smithi100.stdout:op 5304 completed, throughput=5MB/sec 2024-04-07T16:56:13.772 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-SsO2MT4hVntMltT off=6972070 len=47529 2024-04-07T16:56:13.773 INFO:tasks.workunit.client.0.smithi100.stdout:op 5305 completed, throughput=5MB/sec 2024-04-07T16:56:13.773 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-mHZyXpJO9wp-aNS off=1209777 len=907550 2024-04-07T16:56:13.780 INFO:tasks.workunit.client.0.smithi100.stdout:op 5306 completed, throughput=5MB/sec 2024-04-07T16:56:14.781 INFO:tasks.workunit.client.0.smithi100.stdout: 1024: throughput=5MB/sec pending data=0 2024-04-07T16:56:14.781 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-dWwSyisbyO_0q9X off=14484382 len=1778713 2024-04-07T16:56:14.798 INFO:tasks.workunit.client.0.smithi100.stdout:op 5307 completed, throughput=5MB/sec 2024-04-07T16:56:14.798 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-b83OY5YJDhyAmDQ off=6643027 len=1008226 2024-04-07T16:56:14.807 INFO:tasks.workunit.client.0.smithi100.stdout:op 5308 completed, throughput=5MB/sec 2024-04-07T16:56:14.807 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-DrTVz0NUmIQ0djz off=10968517 len=318568 2024-04-07T16:56:14.809 INFO:tasks.workunit.client.0.smithi100.stdout:op 5309 completed, throughput=5MB/sec 2024-04-07T16:56:14.809 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-NX-zclG-gDoORiC off=4267242 len=1072470 2024-04-07T16:56:14.818 INFO:tasks.workunit.client.0.smithi100.stdout:op 5310 completed, throughput=5MB/sec 2024-04-07T16:56:14.818 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-8vOk-SXwkbUH_IV off=2070605 len=283355 2024-04-07T16:56:14.824 INFO:tasks.workunit.client.0.smithi100.stdout:op 5311 completed, throughput=5MB/sec 2024-04-07T16:56:14.825 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-zvBRSG4xxt6R0wJ off=55438 len=1734072 2024-04-07T16:56:14.838 INFO:tasks.workunit.client.0.smithi100.stdout:op 5312 completed, throughput=5MB/sec 2024-04-07T16:56:15.113 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:56:14 smithi144 bash[66654]: cluster 2024-04-07T16:56:13.610823+0000 mgr.y (mgr.25175) 1226 : cluster [DBG] pgmap v1868: 105 pgs: 105 active+clean; 126 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 3.8 MiB/s rd, 2.3 MiB/s wr, 6 op/s 2024-04-07T16:56:15.199 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:56:14 smithi100 bash[74942]: cluster 2024-04-07T16:56:13.610823+0000 mgr.y (mgr.25175) 1226 : cluster [DBG] pgmap v1868: 105 pgs: 105 active+clean; 126 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 3.8 MiB/s rd, 2.3 MiB/s wr, 6 op/s 2024-04-07T16:56:15.199 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:56:14 smithi100 bash[76202]: cluster 2024-04-07T16:56:13.610823+0000 mgr.y (mgr.25175) 1226 : cluster [DBG] pgmap v1868: 105 pgs: 105 active+clean; 126 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 3.8 MiB/s rd, 2.3 MiB/s wr, 6 op/s 2024-04-07T16:56:15.839 INFO:tasks.workunit.client.0.smithi100.stdout: 1025: throughput=5MB/sec pending data=0 2024-04-07T16:56:15.839 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-9_dlOIiufGaC9mz off=952995 len=1427505 2024-04-07T16:56:15.850 INFO:tasks.workunit.client.0.smithi100.stdout:op 5313 completed, throughput=5MB/sec 2024-04-07T16:56:15.850 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-MsWM604doA4Ktyd off=3148884 len=808312 2024-04-07T16:56:15.852 INFO:tasks.workunit.client.0.smithi100.stdout:op 5314 completed, throughput=5MB/sec 2024-04-07T16:56:15.853 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-kS6WDUpkJ3JYUGY off=2050847 len=367829 2024-04-07T16:56:15.859 INFO:tasks.workunit.client.0.smithi100.stdout:op 5315 completed, throughput=5MB/sec 2024-04-07T16:56:15.859 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-EDNaWyVWXBIhnK_ off=6989189 len=1569919 2024-04-07T16:56:15.870 INFO:tasks.workunit.client.0.smithi100.stdout:op 5316 completed, throughput=5MB/sec 2024-04-07T16:56:15.871 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-1i79A6DFqYq6l46 off=3522555 len=1850268 2024-04-07T16:56:15.884 INFO:tasks.workunit.client.0.smithi100.stdout:op 5317 completed, throughput=5MB/sec 2024-04-07T16:56:16.885 INFO:tasks.workunit.client.0.smithi100.stdout: 1026: throughput=5MB/sec pending data=0 2024-04-07T16:56:16.885 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-abJGBjZ4UA_QIcu off=2683898 len=2004023 2024-04-07T16:56:16.899 INFO:tasks.workunit.client.0.smithi100.stdout:op 5318 completed, throughput=5MB/sec 2024-04-07T16:56:16.900 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-dOSzgg-PNfq1R_E off=2814510 len=1168216 2024-04-07T16:56:16.902 INFO:tasks.workunit.client.0.smithi100.stdout:op 5319 completed, throughput=5MB/sec 2024-04-07T16:56:16.902 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-5y4vdArYb-4vs45 off=15472005 len=1166655 2024-04-07T16:56:16.911 INFO:tasks.workunit.client.0.smithi100.stdout:op 5320 completed, throughput=5MB/sec 2024-04-07T16:56:17.112 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:56:16 smithi144 bash[66654]: cluster 2024-04-07T16:56:15.611792+0000 mgr.y (mgr.25175) 1227 : cluster [DBG] pgmap v1869: 105 pgs: 105 active+clean; 126 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 2.9 MiB/s rd, 1.9 MiB/s wr, 4 op/s 2024-04-07T16:56:17.199 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:56:16 smithi100 bash[74942]: cluster 2024-04-07T16:56:15.611792+0000 mgr.y (mgr.25175) 1227 : cluster [DBG] pgmap v1869: 105 pgs: 105 active+clean; 126 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 2.9 MiB/s rd, 1.9 MiB/s wr, 4 op/s 2024-04-07T16:56:17.200 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:56:16 smithi100 bash[76202]: cluster 2024-04-07T16:56:15.611792+0000 mgr.y (mgr.25175) 1227 : cluster [DBG] pgmap v1869: 105 pgs: 105 active+clean; 126 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 2.9 MiB/s rd, 1.9 MiB/s wr, 4 op/s 2024-04-07T16:56:17.912 INFO:tasks.workunit.client.0.smithi100.stdout: 1027: throughput=5MB/sec pending data=0 2024-04-07T16:56:17.912 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-OWgUgnawwHT5QaM off=11945406 len=658976 2024-04-07T16:56:17.915 INFO:tasks.workunit.client.0.smithi100.stdout:op 5321 completed, throughput=5MB/sec 2024-04-07T16:56:17.915 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj--qp8w4wrCFs103w off=9758190 len=1339197 2024-04-07T16:56:17.919 INFO:tasks.workunit.client.0.smithi100.stdout:op 5322 completed, throughput=5MB/sec 2024-04-07T16:56:17.919 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-UFSAWsrrgogDasZ off=7838516 len=731839 2024-04-07T16:56:17.927 INFO:tasks.workunit.client.0.smithi100.stdout:op 5323 completed, throughput=5MB/sec 2024-04-07T16:56:17.927 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-B8jeWwNPDwZZ3yC off=4239027 len=477090 2024-04-07T16:56:17.929 INFO:tasks.workunit.client.0.smithi100.stdout:op 5324 completed, throughput=5MB/sec 2024-04-07T16:56:17.929 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-5IH-74LWPsKxVg3 off=8643444 len=2048204 2024-04-07T16:56:17.934 INFO:tasks.workunit.client.0.smithi100.stdout:op 5325 completed, throughput=5MB/sec 2024-04-07T16:56:18.113 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:56:17 smithi144 bash[66654]: audit 2024-04-07T16:56:16.786719+0000 mon.a (mon.0) 2199 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:56:18.199 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:56:17 smithi100 bash[74942]: audit 2024-04-07T16:56:16.786719+0000 mon.a (mon.0) 2199 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:56:18.199 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:56:17 smithi100 bash[76202]: audit 2024-04-07T16:56:16.786719+0000 mon.a (mon.0) 2199 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:56:18.699 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:56:18 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:56:18] "GET /metrics HTTP/1.1" 200 34073 "" "Prometheus/2.43.0" 2024-04-07T16:56:18.934 INFO:tasks.workunit.client.0.smithi100.stdout: 1028: throughput=5MB/sec pending data=0 2024-04-07T16:56:18.935 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-QupPEj4-sYwTADV off=992697 len=321961 2024-04-07T16:56:18.936 INFO:tasks.workunit.client.0.smithi100.stdout:op 5326 completed, throughput=5MB/sec 2024-04-07T16:56:18.936 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-_Qr3I5vmFOIFSRn off=290260 len=1912391 2024-04-07T16:56:18.941 INFO:tasks.workunit.client.0.smithi100.stdout:op 5327 completed, throughput=5MB/sec 2024-04-07T16:56:18.941 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-74jp29vtGDMldZR off=7369665 len=355943 2024-04-07T16:56:18.947 INFO:tasks.workunit.client.0.smithi100.stdout:op 5328 completed, throughput=5MB/sec 2024-04-07T16:56:18.948 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-BKFJuo5jPW-GGYA off=13617684 len=1776482 2024-04-07T16:56:18.954 INFO:tasks.workunit.client.0.smithi100.stdout:op 5329 completed, throughput=5MB/sec 2024-04-07T16:56:18.954 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-FekJcY1roMF6hML off=6033065 len=2063470 2024-04-07T16:56:18.959 INFO:tasks.workunit.client.0.smithi100.stdout:op 5330 completed, throughput=5MB/sec 2024-04-07T16:56:19.113 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:56:18 smithi144 bash[66654]: cluster 2024-04-07T16:56:17.612752+0000 mgr.y (mgr.25175) 1228 : cluster [DBG] pgmap v1870: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 705 GiB / 715 GiB avail; 2.9 MiB/s rd, 1.9 MiB/s wr, 4 op/s 2024-04-07T16:56:19.199 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:56:18 smithi100 bash[74942]: cluster 2024-04-07T16:56:17.612752+0000 mgr.y (mgr.25175) 1228 : cluster [DBG] pgmap v1870: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 705 GiB / 715 GiB avail; 2.9 MiB/s rd, 1.9 MiB/s wr, 4 op/s 2024-04-07T16:56:19.199 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:56:18 smithi100 bash[76202]: cluster 2024-04-07T16:56:17.612752+0000 mgr.y (mgr.25175) 1228 : cluster [DBG] pgmap v1870: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 705 GiB / 715 GiB avail; 2.9 MiB/s rd, 1.9 MiB/s wr, 4 op/s 2024-04-07T16:56:19.959 INFO:tasks.workunit.client.0.smithi100.stdout: 1029: throughput=5MB/sec pending data=0 2024-04-07T16:56:19.959 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-napoG72ri8kXig0 off=17320385 len=398785 2024-04-07T16:56:19.961 INFO:tasks.workunit.client.0.smithi100.stdout:op 5331 completed, throughput=5MB/sec 2024-04-07T16:56:19.961 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-0o_5yyyi22LxLj- off=0 len=1089546 2024-04-07T16:56:19.965 INFO:tasks.workunit.client.0.smithi100.stdout:op 5332 completed, throughput=5MB/sec 2024-04-07T16:56:19.965 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-HhW_HOpNpF8wMhw off=16335353 len=1118522 2024-04-07T16:56:19.974 INFO:tasks.workunit.client.0.smithi100.stdout:op 5333 completed, throughput=5MB/sec 2024-04-07T16:56:19.975 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Dwt1F8ZXzZqai4V off=4276303 len=448138 2024-04-07T16:56:19.981 INFO:tasks.workunit.client.0.smithi100.stdout:op 5334 completed, throughput=5MB/sec 2024-04-07T16:56:19.981 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-qTAXPigYkT47V2u off=6209947 len=1777812 2024-04-07T16:56:19.986 INFO:tasks.workunit.client.0.smithi100.stdout:op 5335 completed, throughput=5MB/sec 2024-04-07T16:56:20.986 INFO:tasks.workunit.client.0.smithi100.stdout: 1030: throughput=5MB/sec pending data=0 2024-04-07T16:56:20.986 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-zj5YUhF8yGfT6mh off=898277 len=689192 2024-04-07T16:56:20.993 INFO:tasks.workunit.client.0.smithi100.stdout:op 5336 completed, throughput=5MB/sec 2024-04-07T16:56:20.993 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-R7kORH8ZCBH2ZNa off=5388894 len=1721766 2024-04-07T16:56:20.997 INFO:tasks.workunit.client.0.smithi100.stdout:op 5337 completed, throughput=5MB/sec 2024-04-07T16:56:20.997 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-WZQ8rN4V-sl4GXX off=538520 len=1364606 2024-04-07T16:56:21.007 INFO:tasks.workunit.client.0.smithi100.stdout:op 5338 completed, throughput=5MB/sec 2024-04-07T16:56:21.007 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj--VxkzWKJvOXRyUw off=3305557 len=349584 2024-04-07T16:56:21.009 INFO:tasks.workunit.client.0.smithi100.stdout:op 5339 completed, throughput=5MB/sec 2024-04-07T16:56:21.009 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-LeluGZD1vSeGVHE off=4611503 len=53295 2024-04-07T16:56:21.014 INFO:tasks.workunit.client.0.smithi100.stdout:op 5340 completed, throughput=5MB/sec 2024-04-07T16:56:21.014 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-sJHK5g5q6P20r_7 off=1150950 len=832980 2024-04-07T16:56:21.022 INFO:tasks.workunit.client.0.smithi100.stdout:op 5341 completed, throughput=5MB/sec 2024-04-07T16:56:21.113 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:56:20 smithi144 bash[66654]: cluster 2024-04-07T16:56:19.613704+0000 mgr.y (mgr.25175) 1229 : cluster [DBG] pgmap v1871: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 705 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.1 MiB/s wr, 4 op/s 2024-04-07T16:56:21.199 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:56:20 smithi100 bash[74942]: cluster 2024-04-07T16:56:19.613704+0000 mgr.y (mgr.25175) 1229 : cluster [DBG] pgmap v1871: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 705 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.1 MiB/s wr, 4 op/s 2024-04-07T16:56:21.199 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:56:20 smithi100 bash[76202]: cluster 2024-04-07T16:56:19.613704+0000 mgr.y (mgr.25175) 1229 : cluster [DBG] pgmap v1871: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 705 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.1 MiB/s wr, 4 op/s 2024-04-07T16:56:22.022 INFO:tasks.workunit.client.0.smithi100.stdout: 1031: throughput=5MB/sec pending data=0 2024-04-07T16:56:22.022 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-A_xQGr4Ru9hOadq off=1092836 len=728088 2024-04-07T16:56:22.026 INFO:tasks.workunit.client.0.smithi100.stdout:op 5342 completed, throughput=5MB/sec 2024-04-07T16:56:22.026 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-ZDkUV8xakn-J4xa off=3107522 len=1616605 2024-04-07T16:56:22.039 INFO:tasks.workunit.client.0.smithi100.stdout:op 5343 completed, throughput=5MB/sec 2024-04-07T16:56:22.039 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Tp022O5361S_K3d off=196323 len=1027828 2024-04-07T16:56:22.050 INFO:tasks.workunit.client.0.smithi100.stdout:op 5344 completed, throughput=5MB/sec 2024-04-07T16:56:22.050 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-DXeUGjaiiPeQEZF off=1571058 len=2034426 2024-04-07T16:56:22.053 INFO:tasks.workunit.client.0.smithi100.stdout:op 5345 completed, throughput=5MB/sec 2024-04-07T16:56:22.113 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:56:21 smithi144 bash[66654]: cluster 2024-04-07T16:56:21.614508+0000 mgr.y (mgr.25175) 1230 : cluster [DBG] pgmap v1872: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 705 GiB / 715 GiB avail; 2.6 MiB/s rd, 2.3 MiB/s wr, 4 op/s 2024-04-07T16:56:22.199 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:56:21 smithi100 bash[74942]: cluster 2024-04-07T16:56:21.614508+0000 mgr.y (mgr.25175) 1230 : cluster [DBG] pgmap v1872: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 705 GiB / 715 GiB avail; 2.6 MiB/s rd, 2.3 MiB/s wr, 4 op/s 2024-04-07T16:56:22.199 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:56:21 smithi100 bash[76202]: cluster 2024-04-07T16:56:21.614508+0000 mgr.y (mgr.25175) 1230 : cluster [DBG] pgmap v1872: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 705 GiB / 715 GiB avail; 2.6 MiB/s rd, 2.3 MiB/s wr, 4 op/s 2024-04-07T16:56:22.613 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:56:22 smithi144 bash[95233]: ts=2024-04-07T16:56:22.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:56:23.053 INFO:tasks.workunit.client.0.smithi100.stdout: 1032: throughput=5MB/sec pending data=0 2024-04-07T16:56:23.053 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-GSJkTkrhaQ22xvY off=3612109 len=17290 2024-04-07T16:56:23.054 INFO:tasks.workunit.client.0.smithi100.stdout:op 5346 completed, throughput=5MB/sec 2024-04-07T16:56:23.055 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-hbKM3oZTAq3838Z off=5031299 len=2048977 2024-04-07T16:56:23.070 INFO:tasks.workunit.client.0.smithi100.stdout:op 5347 completed, throughput=5MB/sec 2024-04-07T16:56:23.070 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-wGR7xkEXTXrka2M off=8472092 len=1644780 2024-04-07T16:56:23.081 INFO:tasks.workunit.client.0.smithi100.stdout:op 5348 completed, throughput=5MB/sec 2024-04-07T16:56:23.081 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-A_XFzntqKPYhrpZ off=7423889 len=479290 2024-04-07T16:56:23.083 INFO:tasks.workunit.client.0.smithi100.stdout:op 5349 completed, throughput=5MB/sec 2024-04-07T16:56:23.083 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-LhFPcfgASEI698n off=12830145 len=336927 2024-04-07T16:56:23.084 INFO:tasks.workunit.client.0.smithi100.stdout:op 5350 completed, throughput=5MB/sec 2024-04-07T16:56:23.085 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-z3wp9HzXW5QOFe2 off=6238718 len=1550313 2024-04-07T16:56:23.096 INFO:tasks.workunit.client.0.smithi100.stdout:op 5351 completed, throughput=5MB/sec 2024-04-07T16:56:24.096 INFO:tasks.workunit.client.0.smithi100.stdout: 1033: throughput=5MB/sec pending data=0 2024-04-07T16:56:24.096 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-hiNpwxOKfVIaTzE off=175389 len=596348 2024-04-07T16:56:24.104 INFO:tasks.workunit.client.0.smithi100.stdout:op 5352 completed, throughput=5MB/sec 2024-04-07T16:56:24.104 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-AK1wOmiZaWCk--j off=16998033 len=2090543 2024-04-07T16:56:24.108 INFO:tasks.workunit.client.0.smithi100.stdout:op 5353 completed, throughput=5MB/sec 2024-04-07T16:56:24.109 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-PB60wvQyRKwwIV5 off=4156808 len=927902 2024-04-07T16:56:24.111 INFO:tasks.workunit.client.0.smithi100.stdout:op 5354 completed, throughput=5MB/sec 2024-04-07T16:56:24.111 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-_jditG32Vbi4ARW off=21753443 len=1765126 2024-04-07T16:56:24.125 INFO:tasks.workunit.client.0.smithi100.stdout:op 5355 completed, throughput=5MB/sec 2024-04-07T16:56:24.949 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:56:24 smithi100 bash[76202]: cluster 2024-04-07T16:56:23.615597+0000 mgr.y (mgr.25175) 1231 : cluster [DBG] pgmap v1873: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 705 GiB / 715 GiB avail; 2.8 MiB/s rd, 2.7 MiB/s wr, 5 op/s 2024-04-07T16:56:24.949 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:56:24 smithi100 bash[74942]: cluster 2024-04-07T16:56:23.615597+0000 mgr.y (mgr.25175) 1231 : cluster [DBG] pgmap v1873: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 705 GiB / 715 GiB avail; 2.8 MiB/s rd, 2.7 MiB/s wr, 5 op/s 2024-04-07T16:56:25.113 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:56:24 smithi144 bash[66654]: cluster 2024-04-07T16:56:23.615597+0000 mgr.y (mgr.25175) 1231 : cluster [DBG] pgmap v1873: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 705 GiB / 715 GiB avail; 2.8 MiB/s rd, 2.7 MiB/s wr, 5 op/s 2024-04-07T16:56:25.126 INFO:tasks.workunit.client.0.smithi100.stdout: 1034: throughput=5MB/sec pending data=0 2024-04-07T16:56:25.126 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-NS1exCTNABhUzek off=7578225 len=265106 2024-04-07T16:56:25.127 INFO:tasks.workunit.client.0.smithi100.stdout:op 5356 completed, throughput=5MB/sec 2024-04-07T16:56:25.128 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-e24MYICiUaRUOes off=14307712 len=885690 2024-04-07T16:56:25.135 INFO:tasks.workunit.client.0.smithi100.stdout:op 5357 completed, throughput=5MB/sec 2024-04-07T16:56:25.136 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-yUFmGIXJBDI3fGc off=10156637 len=525332 2024-04-07T16:56:25.143 INFO:tasks.workunit.client.0.smithi100.stdout:op 5358 completed, throughput=5MB/sec 2024-04-07T16:56:25.143 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-xDFBSy60LrHcLL- off=222879 len=1006370 2024-04-07T16:56:25.146 INFO:tasks.workunit.client.0.smithi100.stdout:op 5359 completed, throughput=5MB/sec 2024-04-07T16:56:25.146 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-hTJd0RDUm4IlU_W off=13819508 len=940224 2024-04-07T16:56:25.156 INFO:tasks.workunit.client.0.smithi100.stdout:op 5360 completed, throughput=5MB/sec 2024-04-07T16:56:25.156 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-rILrXyd7IOC7FXo off=8470035 len=15105 2024-04-07T16:56:25.160 INFO:tasks.workunit.client.0.smithi100.stdout:op 5361 completed, throughput=5MB/sec 2024-04-07T16:56:25.160 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-rYyC_MuKqBqsOFK off=2527975 len=1353191 2024-04-07T16:56:25.166 INFO:tasks.workunit.client.0.smithi100.stdout:op 5362 completed, throughput=5MB/sec 2024-04-07T16:56:26.164 INFO:tasks.workunit.client.0.smithi100.stdout: 1035: throughput=5MB/sec pending data=0 2024-04-07T16:56:26.164 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-2HsxjixLweMaGRY off=833866 len=1766085 2024-04-07T16:56:26.169 INFO:tasks.workunit.client.0.smithi100.stdout:op 5363 completed, throughput=5MB/sec 2024-04-07T16:56:26.170 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-tWYs2No0IkXMfvG off=423952 len=852919 2024-04-07T16:56:26.177 INFO:tasks.workunit.client.0.smithi100.stdout:op 5364 completed, throughput=5MB/sec 2024-04-07T16:56:26.178 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-EkRBXaAap_igGYb off=24578826 len=950586 2024-04-07T16:56:26.186 INFO:tasks.workunit.client.0.smithi100.stdout:op 5365 completed, throughput=5MB/sec 2024-04-07T16:56:26.187 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-zyrA-n9v24q7YuW off=2554092 len=633917 2024-04-07T16:56:26.194 INFO:tasks.workunit.client.0.smithi100.stdout:op 5366 completed, throughput=5MB/sec 2024-04-07T16:56:26.194 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-6IHqFe-lF8molSY off=4180036 len=1078756 2024-04-07T16:56:26.198 INFO:tasks.workunit.client.0.smithi100.stdout:op 5367 completed, throughput=5MB/sec 2024-04-07T16:56:26.949 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:56:26 smithi100 bash[74942]: cluster 2024-04-07T16:56:25.616556+0000 mgr.y (mgr.25175) 1232 : cluster [DBG] pgmap v1874: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 705 GiB / 715 GiB avail; 2.2 MiB/s rd, 2.4 MiB/s wr, 4 op/s 2024-04-07T16:56:26.949 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:56:26 smithi100 bash[76202]: cluster 2024-04-07T16:56:25.616556+0000 mgr.y (mgr.25175) 1232 : cluster [DBG] pgmap v1874: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 705 GiB / 715 GiB avail; 2.2 MiB/s rd, 2.4 MiB/s wr, 4 op/s 2024-04-07T16:56:27.113 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:56:26 smithi144 bash[66654]: cluster 2024-04-07T16:56:25.616556+0000 mgr.y (mgr.25175) 1232 : cluster [DBG] pgmap v1874: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 705 GiB / 715 GiB avail; 2.2 MiB/s rd, 2.4 MiB/s wr, 4 op/s 2024-04-07T16:56:27.199 INFO:tasks.workunit.client.0.smithi100.stdout: 1036: throughput=5MB/sec pending data=0 2024-04-07T16:56:27.199 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-OqW5aQTLfWRmoxD off=11005397 len=1638732 2024-04-07T16:56:27.214 INFO:tasks.workunit.client.0.smithi100.stdout:op 5368 completed, throughput=5MB/sec 2024-04-07T16:56:27.214 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-vFwoQSC_LAx_uAE off=18923044 len=1098553 2024-04-07T16:56:27.217 INFO:tasks.workunit.client.0.smithi100.stdout:op 5369 completed, throughput=5MB/sec 2024-04-07T16:56:27.217 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Mt54j109G9dnEWj off=3343412 len=26129 2024-04-07T16:56:27.218 INFO:tasks.workunit.client.0.smithi100.stdout:op 5370 completed, throughput=5MB/sec 2024-04-07T16:56:27.221 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-mBsWotsBDjMD3GW off=21701482 len=1459791 2024-04-07T16:56:27.230 INFO:tasks.workunit.client.0.smithi100.stdout:op 5371 completed, throughput=5MB/sec 2024-04-07T16:56:27.230 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-tBn2tlatssPjtPf off=11351335 len=1667754 2024-04-07T16:56:27.232 INFO:tasks.workunit.client.0.smithi100.stdout:op 5372 completed, throughput=5MB/sec 2024-04-07T16:56:28.233 INFO:tasks.workunit.client.0.smithi100.stdout: 1037: throughput=5MB/sec pending data=0 2024-04-07T16:56:28.233 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-vAi6hSV8pyWxKp5 off=2175057 len=431226 2024-04-07T16:56:28.235 INFO:tasks.workunit.client.0.smithi100.stdout:op 5373 completed, throughput=5MB/sec 2024-04-07T16:56:28.235 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-xBI7vY9sV-D4jBO off=2921155 len=627855 2024-04-07T16:56:28.237 INFO:tasks.workunit.client.0.smithi100.stdout:op 5374 completed, throughput=5MB/sec 2024-04-07T16:56:28.238 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-YFrrb4waUE0rCeg off=979798 len=527474 2024-04-07T16:56:28.243 INFO:tasks.workunit.client.0.smithi100.stdout:op 5375 completed, throughput=5MB/sec 2024-04-07T16:56:28.244 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-0F5FCRnknY5lJM7 off=58859 len=615187 2024-04-07T16:56:28.251 INFO:tasks.workunit.client.0.smithi100.stdout:op 5376 completed, throughput=5MB/sec 2024-04-07T16:56:28.251 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-A4Ev1sVraUGVQ9d off=11545344 len=1140701 2024-04-07T16:56:28.255 INFO:tasks.workunit.client.0.smithi100.stdout:op 5377 completed, throughput=5MB/sec 2024-04-07T16:56:28.255 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-CMJj5b64_2IVawK off=4296644 len=926033 2024-04-07T16:56:28.258 INFO:tasks.workunit.client.0.smithi100.stdout:op 5378 completed, throughput=5MB/sec 2024-04-07T16:56:28.258 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Oyl6PCAa7JD4kA9 off=2062135 len=1041821 2024-04-07T16:56:28.266 INFO:tasks.workunit.client.0.smithi100.stdout:op 5379 completed, throughput=5MB/sec 2024-04-07T16:56:28.676 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:56:28 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:56:28] "GET /metrics HTTP/1.1" 200 34073 "" "Prometheus/2.43.0" 2024-04-07T16:56:28.949 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:56:28 smithi100 bash[74942]: cluster 2024-04-07T16:56:27.617653+0000 mgr.y (mgr.25175) 1233 : cluster [DBG] pgmap v1875: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 705 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.4 MiB/s wr, 4 op/s 2024-04-07T16:56:28.949 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:56:28 smithi100 bash[76202]: cluster 2024-04-07T16:56:27.617653+0000 mgr.y (mgr.25175) 1233 : cluster [DBG] pgmap v1875: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 705 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.4 MiB/s wr, 4 op/s 2024-04-07T16:56:29.113 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:56:28 smithi144 bash[66654]: cluster 2024-04-07T16:56:27.617653+0000 mgr.y (mgr.25175) 1233 : cluster [DBG] pgmap v1875: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 705 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.4 MiB/s wr, 4 op/s 2024-04-07T16:56:29.267 INFO:tasks.workunit.client.0.smithi100.stdout: 1038: throughput=5MB/sec pending data=0 2024-04-07T16:56:29.267 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-NGUKffdU9whU50e off=9868989 len=736268 2024-04-07T16:56:29.269 INFO:tasks.workunit.client.0.smithi100.stdout:op 5380 completed, throughput=5MB/sec 2024-04-07T16:56:29.269 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-RfN3MeipR8mIrt7 off=2726875 len=1996877 2024-04-07T16:56:29.288 INFO:tasks.workunit.client.0.smithi100.stdout:op 5381 completed, throughput=5MB/sec 2024-04-07T16:56:29.288 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-MIbgknCww8bUMbY off=550065 len=1663585 2024-04-07T16:56:29.301 INFO:tasks.workunit.client.0.smithi100.stdout:op 5382 completed, throughput=5MB/sec 2024-04-07T16:56:29.301 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-4ShyAsByY6yVA8G off=14635373 len=86985 2024-04-07T16:56:29.305 INFO:tasks.workunit.client.0.smithi100.stdout:op 5383 completed, throughput=5MB/sec 2024-04-07T16:56:29.305 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-oXHDUHheJ174ZwC off=7445757 len=687657 2024-04-07T16:56:29.308 INFO:tasks.workunit.client.0.smithi100.stdout:op 5384 completed, throughput=5MB/sec 2024-04-07T16:56:30.308 INFO:tasks.workunit.client.0.smithi100.stdout: 1039: throughput=5MB/sec pending data=0 2024-04-07T16:56:30.308 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-hPVUlLGnOjmTw8N off=93005 len=1693278 2024-04-07T16:56:30.321 INFO:tasks.workunit.client.0.smithi100.stdout:op 5385 completed, throughput=5MB/sec 2024-04-07T16:56:30.322 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-BGyD2X4yL8P47AL off=19556615 len=1866563 2024-04-07T16:56:30.326 INFO:tasks.workunit.client.0.smithi100.stdout:op 5386 completed, throughput=5MB/sec 2024-04-07T16:56:30.326 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Xv0Sbkyk5ZTNXgW off=11242712 len=861811 2024-04-07T16:56:30.334 INFO:tasks.workunit.client.0.smithi100.stdout:op 5387 completed, throughput=5MB/sec 2024-04-07T16:56:30.334 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-R2KHPrYZ-OxR3lW off=17222683 len=2011300 2024-04-07T16:56:30.339 INFO:tasks.workunit.client.0.smithi100.stdout:op 5388 completed, throughput=5MB/sec 2024-04-07T16:56:30.949 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:56:30 smithi100 bash[74942]: cluster 2024-04-07T16:56:29.618649+0000 mgr.y (mgr.25175) 1234 : cluster [DBG] pgmap v1876: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 705 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.3 MiB/s wr, 4 op/s 2024-04-07T16:56:30.949 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:56:30 smithi100 bash[76202]: cluster 2024-04-07T16:56:29.618649+0000 mgr.y (mgr.25175) 1234 : cluster [DBG] pgmap v1876: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 705 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.3 MiB/s wr, 4 op/s 2024-04-07T16:56:31.113 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:56:30 smithi144 bash[66654]: cluster 2024-04-07T16:56:29.618649+0000 mgr.y (mgr.25175) 1234 : cluster [DBG] pgmap v1876: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 705 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.3 MiB/s wr, 4 op/s 2024-04-07T16:56:31.339 INFO:tasks.workunit.client.0.smithi100.stdout: 1040: throughput=5MB/sec pending data=0 2024-04-07T16:56:31.339 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-wtucn7jl4961lCI off=1045603 len=976551 2024-04-07T16:56:31.349 INFO:tasks.workunit.client.0.smithi100.stdout:op 5389 completed, throughput=5MB/sec 2024-04-07T16:56:31.349 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-27Z80roi9nJA0WE off=11731515 len=191161 2024-04-07T16:56:31.355 INFO:tasks.workunit.client.0.smithi100.stdout:op 5390 completed, throughput=5MB/sec 2024-04-07T16:56:31.355 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-SMjMdBa6Wq9T5ue off=12015445 len=800555 2024-04-07T16:56:31.358 INFO:tasks.workunit.client.0.smithi100.stdout:op 5391 completed, throughput=5MB/sec 2024-04-07T16:56:31.358 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-UQX2LOvyLhbTu1G off=7572773 len=2088734 2024-04-07T16:56:31.366 INFO:tasks.workunit.client.0.smithi100.stdout:op 5392 completed, throughput=5MB/sec 2024-04-07T16:56:31.366 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-ZyY2S5ap91Q7U5K off=5948338 len=1791908 2024-04-07T16:56:31.370 INFO:tasks.workunit.client.0.smithi100.stdout:op 5393 completed, throughput=5MB/sec 2024-04-07T16:56:32.370 INFO:tasks.workunit.client.0.smithi100.stdout: 1041: throughput=5MB/sec pending data=0 2024-04-07T16:56:32.371 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-RyHi2W5nu-YXBSW off=9392389 len=1299952 2024-04-07T16:56:32.381 INFO:tasks.workunit.client.0.smithi100.stdout:op 5394 completed, throughput=5MB/sec 2024-04-07T16:56:32.381 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-2JY6rPh8HD72j1o off=16221111 len=236709 2024-04-07T16:56:32.386 INFO:tasks.workunit.client.0.smithi100.stdout:op 5395 completed, throughput=5MB/sec 2024-04-07T16:56:32.386 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-G4uBMPDRxv1cHw1 off=11760761 len=1978338 2024-04-07T16:56:32.402 INFO:tasks.workunit.client.0.smithi100.stdout:op 5396 completed, throughput=5MB/sec 2024-04-07T16:56:32.402 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-lgn26_V49niXdav off=11813763 len=90949 2024-04-07T16:56:32.407 INFO:tasks.workunit.client.0.smithi100.stdout:op 5397 completed, throughput=5MB/sec 2024-04-07T16:56:32.407 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Xfn2c0F-Lewhueq off=1422880 len=590990 2024-04-07T16:56:32.410 INFO:tasks.workunit.client.0.smithi100.stdout:op 5398 completed, throughput=5MB/sec 2024-04-07T16:56:32.613 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:56:32 smithi144 bash[95233]: ts=2024-04-07T16:56:32.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:56:32.949 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:56:32 smithi100 bash[76202]: cluster 2024-04-07T16:56:31.619506+0000 mgr.y (mgr.25175) 1235 : cluster [DBG] pgmap v1877: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 705 GiB / 715 GiB avail; 2.6 MiB/s rd, 2.3 MiB/s wr, 4 op/s 2024-04-07T16:56:32.949 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:56:32 smithi100 bash[76202]: audit 2024-04-07T16:56:31.787166+0000 mon.a (mon.0) 2200 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:56:32.950 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:56:32 smithi100 bash[74942]: cluster 2024-04-07T16:56:31.619506+0000 mgr.y (mgr.25175) 1235 : cluster [DBG] pgmap v1877: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 705 GiB / 715 GiB avail; 2.6 MiB/s rd, 2.3 MiB/s wr, 4 op/s 2024-04-07T16:56:32.950 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:56:32 smithi100 bash[74942]: audit 2024-04-07T16:56:31.787166+0000 mon.a (mon.0) 2200 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:56:33.113 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:56:32 smithi144 bash[66654]: cluster 2024-04-07T16:56:31.619506+0000 mgr.y (mgr.25175) 1235 : cluster [DBG] pgmap v1877: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 705 GiB / 715 GiB avail; 2.6 MiB/s rd, 2.3 MiB/s wr, 4 op/s 2024-04-07T16:56:33.113 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:56:32 smithi144 bash[66654]: audit 2024-04-07T16:56:31.787166+0000 mon.a (mon.0) 2200 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:56:33.411 INFO:tasks.workunit.client.0.smithi100.stdout: 1042: throughput=5MB/sec pending data=0 2024-04-07T16:56:33.411 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-L1kfKVQIq9Y1m37 off=14000766 len=687793 2024-04-07T16:56:33.426 INFO:tasks.workunit.client.0.smithi100.stdout:op 5399 completed, throughput=5MB/sec 2024-04-07T16:56:33.426 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-wtucn7jl4961lCI off=9462055 len=289841 2024-04-07T16:56:33.431 INFO:tasks.workunit.client.0.smithi100.stdout:op 5400 completed, throughput=5MB/sec 2024-04-07T16:56:33.431 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-GtC5FMvKHa5ohMJ off=14013014 len=1910364 2024-04-07T16:56:33.448 INFO:tasks.workunit.client.0.smithi100.stdout:op 5401 completed, throughput=5MB/sec 2024-04-07T16:56:33.448 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-447ZqW7TdPb4xBk off=976316 len=81943 2024-04-07T16:56:33.453 INFO:tasks.workunit.client.0.smithi100.stdout:op 5402 completed, throughput=5MB/sec 2024-04-07T16:56:33.453 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-m1qiH8D3MCkF42O off=4533585 len=360791 2024-04-07T16:56:33.454 INFO:tasks.workunit.client.0.smithi100.stdout:op 5403 completed, throughput=5MB/sec 2024-04-07T16:56:33.455 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-LsePYUZGwjDPcBI off=290947 len=850646 2024-04-07T16:56:33.463 INFO:tasks.workunit.client.0.smithi100.stdout:op 5404 completed, throughput=5MB/sec 2024-04-07T16:56:33.463 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-B4-VPoMttLkQly7 off=10113 len=1802700 2024-04-07T16:56:33.476 INFO:tasks.workunit.client.0.smithi100.stdout:op 5405 completed, throughput=5MB/sec 2024-04-07T16:56:34.477 INFO:tasks.workunit.client.0.smithi100.stdout: 1043: throughput=5MB/sec pending data=0 2024-04-07T16:56:34.477 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-K971uV0oVRk_2Ze off=11528269 len=1296964 2024-04-07T16:56:34.486 INFO:tasks.workunit.client.0.smithi100.stdout:op 5406 completed, throughput=5MB/sec 2024-04-07T16:56:34.487 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-zsHYv2xY0cki2ro off=6682487 len=906130 2024-04-07T16:56:34.491 INFO:tasks.workunit.client.0.smithi100.stdout:op 5407 completed, throughput=5MB/sec 2024-04-07T16:56:34.491 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-xdpPJ_o0T5fITBj off=9057126 len=1423810 2024-04-07T16:56:34.495 INFO:tasks.workunit.client.0.smithi100.stdout:op 5408 completed, throughput=5MB/sec 2024-04-07T16:56:34.495 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-X2B4NBDUW9-lABS off=1874140 len=932182 2024-04-07T16:56:34.505 INFO:tasks.workunit.client.0.smithi100.stdout:op 5409 completed, throughput=5MB/sec 2024-04-07T16:56:34.950 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:56:34 smithi100 bash[74942]: cluster 2024-04-07T16:56:33.620643+0000 mgr.y (mgr.25175) 1236 : cluster [DBG] pgmap v1878: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 705 GiB / 715 GiB avail; 2.9 MiB/s rd, 2.5 MiB/s wr, 5 op/s 2024-04-07T16:56:34.950 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:56:34 smithi100 bash[76202]: cluster 2024-04-07T16:56:33.620643+0000 mgr.y (mgr.25175) 1236 : cluster [DBG] pgmap v1878: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 705 GiB / 715 GiB avail; 2.9 MiB/s rd, 2.5 MiB/s wr, 5 op/s 2024-04-07T16:56:35.113 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:56:34 smithi144 bash[66654]: cluster 2024-04-07T16:56:33.620643+0000 mgr.y (mgr.25175) 1236 : cluster [DBG] pgmap v1878: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 705 GiB / 715 GiB avail; 2.9 MiB/s rd, 2.5 MiB/s wr, 5 op/s 2024-04-07T16:56:35.505 INFO:tasks.workunit.client.0.smithi100.stdout: 1044: throughput=5MB/sec pending data=0 2024-04-07T16:56:35.505 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-aBfGcaWki6K2ZVX off=4748322 len=770830 2024-04-07T16:56:35.508 INFO:tasks.workunit.client.0.smithi100.stdout:op 5410 completed, throughput=5MB/sec 2024-04-07T16:56:35.508 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-e_gpFKfupCzx_QJ off=19968019 len=1542266 2024-04-07T16:56:35.521 INFO:tasks.workunit.client.0.smithi100.stdout:op 5411 completed, throughput=5MB/sec 2024-04-07T16:56:35.521 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-rXuH7ivyEVPrC4I off=1060250 len=2032594 2024-04-07T16:56:35.538 INFO:tasks.workunit.client.0.smithi100.stdout:op 5412 completed, throughput=5MB/sec 2024-04-07T16:56:35.538 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-BAozPKjmp0MMM1u off=0 len=1160190 2024-04-07T16:56:35.543 INFO:tasks.workunit.client.0.smithi100.stdout:op 5413 completed, throughput=5MB/sec 2024-04-07T16:56:36.543 INFO:tasks.workunit.client.0.smithi100.stdout: 1045: throughput=5MB/sec pending data=0 2024-04-07T16:56:36.543 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-un-QmT_UKt33fZE off=9686483 len=1167758 2024-04-07T16:56:36.547 INFO:tasks.workunit.client.0.smithi100.stdout:op 5414 completed, throughput=5MB/sec 2024-04-07T16:56:36.547 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-cOfyrWaFR2Dq0io off=8721019 len=1812897 2024-04-07T16:56:36.563 INFO:tasks.workunit.client.0.smithi100.stdout:op 5415 completed, throughput=5MB/sec 2024-04-07T16:56:36.563 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-3Z2LPK5rc9-D3kj off=1464329 len=2019240 2024-04-07T16:56:36.577 INFO:tasks.workunit.client.0.smithi100.stdout:op 5416 completed, throughput=5MB/sec 2024-04-07T16:56:36.577 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Lw3OBpq_wNjHIwS off=5470881 len=1659889 2024-04-07T16:56:36.582 INFO:tasks.workunit.client.0.smithi100.stdout:op 5417 completed, throughput=5MB/sec 2024-04-07T16:56:36.949 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:56:36 smithi100 bash[74942]: cluster 2024-04-07T16:56:35.621677+0000 mgr.y (mgr.25175) 1237 : cluster [DBG] pgmap v1879: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 705 GiB / 715 GiB avail; 2.3 MiB/s rd, 2.5 MiB/s wr, 4 op/s 2024-04-07T16:56:36.949 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:56:36 smithi100 bash[76202]: cluster 2024-04-07T16:56:35.621677+0000 mgr.y (mgr.25175) 1237 : cluster [DBG] pgmap v1879: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 705 GiB / 715 GiB avail; 2.3 MiB/s rd, 2.5 MiB/s wr, 4 op/s 2024-04-07T16:56:37.113 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:56:36 smithi144 bash[66654]: cluster 2024-04-07T16:56:35.621677+0000 mgr.y (mgr.25175) 1237 : cluster [DBG] pgmap v1879: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 705 GiB / 715 GiB avail; 2.3 MiB/s rd, 2.5 MiB/s wr, 4 op/s 2024-04-07T16:56:37.583 INFO:tasks.workunit.client.0.smithi100.stdout: 1046: throughput=5MB/sec pending data=0 2024-04-07T16:56:37.583 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-M8_oO9y-4HUwDRE off=17022326 len=376444 2024-04-07T16:56:37.590 INFO:tasks.workunit.client.0.smithi100.stdout:op 5418 completed, throughput=5MB/sec 2024-04-07T16:56:37.590 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-KNFR7I4KaGs6vcH off=7182426 len=1783162 2024-04-07T16:56:37.596 INFO:tasks.workunit.client.0.smithi100.stdout:op 5419 completed, throughput=5MB/sec 2024-04-07T16:56:37.596 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-xZ9iDPWawtVq70F off=4752145 len=202121 2024-04-07T16:56:37.601 INFO:tasks.workunit.client.0.smithi100.stdout:op 5420 completed, throughput=5MB/sec 2024-04-07T16:56:37.601 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-qnygpLxMMpplDwd off=10673279 len=663918 2024-04-07T16:56:37.609 INFO:tasks.workunit.client.0.smithi100.stdout:op 5421 completed, throughput=5MB/sec 2024-04-07T16:56:37.609 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-4p7pslIMVBelgsC off=6737254 len=312020 2024-04-07T16:56:37.611 INFO:tasks.workunit.client.0.smithi100.stdout:op 5422 completed, throughput=5MB/sec 2024-04-07T16:56:37.611 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-PT1ZojGLuSLD8dK off=108004 len=238113 2024-04-07T16:56:37.613 INFO:tasks.workunit.client.0.smithi100.stdout:op 5423 completed, throughput=5MB/sec 2024-04-07T16:56:37.613 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-hFxYQ0HYtuInzyo off=16305083 len=1460024 2024-04-07T16:56:37.626 INFO:tasks.workunit.client.0.smithi100.stdout:op 5424 completed, throughput=5MB/sec 2024-04-07T16:56:38.626 INFO:tasks.workunit.client.0.smithi100.stdout: 1047: throughput=5MB/sec pending data=0 2024-04-07T16:56:38.627 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-ME9MUb_Y0JN4aJF off=2146058 len=496326 2024-04-07T16:56:38.629 INFO:tasks.workunit.client.0.smithi100.stdout:op 5425 completed, throughput=5MB/sec 2024-04-07T16:56:38.629 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-_UbQXGXvWwqL2Rq off=2368625 len=1335817 2024-04-07T16:56:38.641 INFO:tasks.workunit.client.0.smithi100.stdout:op 5426 completed, throughput=5MB/sec 2024-04-07T16:56:38.641 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-tCAb7PRzNNuRE_8 off=11299448 len=787475 2024-04-07T16:56:38.649 INFO:tasks.workunit.client.0.smithi100.stdout:op 5427 completed, throughput=5MB/sec 2024-04-07T16:56:38.649 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-uVQ8UFE-_sW-Qo5 off=1378847 len=838676 2024-04-07T16:56:38.651 INFO:tasks.workunit.client.0.smithi100.stdout:op 5428 completed, throughput=5MB/sec 2024-04-07T16:56:38.651 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-IUfYdHgvFHpBxKv off=6208629 len=1345922 2024-04-07T16:56:38.662 INFO:tasks.workunit.client.0.smithi100.stdout:op 5429 completed, throughput=5MB/sec 2024-04-07T16:56:38.698 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:56:38 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:56:38] "GET /metrics HTTP/1.1" 200 34072 "" "Prometheus/2.43.0" 2024-04-07T16:56:38.949 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:56:38 smithi100 bash[76202]: cluster 2024-04-07T16:56:37.622715+0000 mgr.y (mgr.25175) 1238 : cluster [DBG] pgmap v1880: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 705 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.3 MiB/s wr, 5 op/s 2024-04-07T16:56:38.949 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:56:38 smithi100 bash[74942]: cluster 2024-04-07T16:56:37.622715+0000 mgr.y (mgr.25175) 1238 : cluster [DBG] pgmap v1880: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 705 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.3 MiB/s wr, 5 op/s 2024-04-07T16:56:39.113 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:56:38 smithi144 bash[66654]: cluster 2024-04-07T16:56:37.622715+0000 mgr.y (mgr.25175) 1238 : cluster [DBG] pgmap v1880: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 705 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.3 MiB/s wr, 5 op/s 2024-04-07T16:56:39.662 INFO:tasks.workunit.client.0.smithi100.stdout: 1048: throughput=5MB/sec pending data=0 2024-04-07T16:56:39.662 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Ce6Q5GDSRUwH7DG off=17846985 len=29083 2024-04-07T16:56:39.663 INFO:tasks.workunit.client.0.smithi100.stdout:op 5430 completed, throughput=5MB/sec 2024-04-07T16:56:39.663 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-MvBtjqBnV7QPUgy off=770914 len=1433515 2024-04-07T16:56:39.674 INFO:tasks.workunit.client.0.smithi100.stdout:op 5431 completed, throughput=5MB/sec 2024-04-07T16:56:39.674 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-1daZ9xKD1zFgxA5 off=39601 len=892567 2024-04-07T16:56:39.677 INFO:tasks.workunit.client.0.smithi100.stdout:op 5432 completed, throughput=5MB/sec 2024-04-07T16:56:39.677 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-y1lNDvP0_6y367l off=3451569 len=87644 2024-04-07T16:56:39.682 INFO:tasks.workunit.client.0.smithi100.stdout:op 5433 completed, throughput=5MB/sec 2024-04-07T16:56:39.682 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Xo6FB73Bx8kpf3K off=2618987 len=1802349 2024-04-07T16:56:39.698 INFO:tasks.workunit.client.0.smithi100.stdout:op 5434 completed, throughput=5MB/sec 2024-04-07T16:56:39.699 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-iOvMOhlliLVIRJv off=3367036 len=187671 2024-04-07T16:56:39.699 INFO:tasks.workunit.client.0.smithi100.stdout:op 5435 completed, throughput=5MB/sec 2024-04-07T16:56:39.700 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-TF2ss9guKXviBxT off=6228394 len=1209804 2024-04-07T16:56:39.710 INFO:tasks.workunit.client.0.smithi100.stdout:op 5436 completed, throughput=5MB/sec 2024-04-07T16:56:40.709 INFO:tasks.workunit.client.0.smithi100.stdout: 1049: throughput=5MB/sec pending data=0 2024-04-07T16:56:40.709 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-RPZhLJQAYf6pFAH off=4761025 len=1819459 2024-04-07T16:56:40.724 INFO:tasks.workunit.client.0.smithi100.stdout:op 5437 completed, throughput=5MB/sec 2024-04-07T16:56:40.724 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-bBByoixXnaJnEfD off=16971388 len=1482234 2024-04-07T16:56:40.726 INFO:tasks.workunit.client.0.smithi100.stdout:op 5438 completed, throughput=5MB/sec 2024-04-07T16:56:40.726 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-1dhQsWSYCNRkDtE off=7450265 len=592593 2024-04-07T16:56:40.733 INFO:tasks.workunit.client.0.smithi100.stdout:op 5439 completed, throughput=5MB/sec 2024-04-07T16:56:40.733 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Iy5wguMGEjhgyyO off=18156538 len=1416508 2024-04-07T16:56:40.744 INFO:tasks.workunit.client.0.smithi100.stdout:op 5440 completed, throughput=5MB/sec 2024-04-07T16:56:41.113 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:56:40 smithi144 bash[66654]: cluster 2024-04-07T16:56:39.623762+0000 mgr.y (mgr.25175) 1239 : cluster [DBG] pgmap v1881: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 705 GiB / 715 GiB avail; 2.1 MiB/s rd, 2.4 MiB/s wr, 4 op/s 2024-04-07T16:56:41.199 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:56:40 smithi100 bash[74942]: cluster 2024-04-07T16:56:39.623762+0000 mgr.y (mgr.25175) 1239 : cluster [DBG] pgmap v1881: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 705 GiB / 715 GiB avail; 2.1 MiB/s rd, 2.4 MiB/s wr, 4 op/s 2024-04-07T16:56:41.199 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:56:40 smithi100 bash[76202]: cluster 2024-04-07T16:56:39.623762+0000 mgr.y (mgr.25175) 1239 : cluster [DBG] pgmap v1881: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 705 GiB / 715 GiB avail; 2.1 MiB/s rd, 2.4 MiB/s wr, 4 op/s 2024-04-07T16:56:41.744 INFO:tasks.workunit.client.0.smithi100.stdout: 1050: throughput=5MB/sec pending data=0 2024-04-07T16:56:41.745 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-PXyPqPvjJe2IjoA off=5151775 len=852922 2024-04-07T16:56:41.754 INFO:tasks.workunit.client.0.smithi100.stdout:op 5441 completed, throughput=5MB/sec 2024-04-07T16:56:41.754 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-mt5XR8goDtsSlhb off=2986426 len=26243 2024-04-07T16:56:41.754 INFO:tasks.workunit.client.0.smithi100.stdout:op 5442 completed, throughput=5MB/sec 2024-04-07T16:56:41.754 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-3f9bGXtsEwItu6o off=6268941 len=575301 2024-04-07T16:56:41.756 INFO:tasks.workunit.client.0.smithi100.stdout:op 5443 completed, throughput=5MB/sec 2024-04-07T16:56:41.757 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-J_U_5BNXDZfxJxI off=16372165 len=573094 2024-04-07T16:56:41.758 INFO:tasks.workunit.client.0.smithi100.stdout:op 5444 completed, throughput=5MB/sec 2024-04-07T16:56:41.758 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-1vu5qoF1V_hRfWB off=381980 len=1818815 2024-04-07T16:56:41.762 INFO:tasks.workunit.client.0.smithi100.stdout:op 5445 completed, throughput=5MB/sec 2024-04-07T16:56:41.762 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-AVLKysR7PbJTBHq off=16149572 len=1345510 2024-04-07T16:56:41.765 INFO:tasks.workunit.client.0.smithi100.stdout:op 5446 completed, throughput=5MB/sec 2024-04-07T16:56:42.613 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:56:42 smithi144 bash[95233]: ts=2024-04-07T16:56:42.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:56:42.765 INFO:tasks.workunit.client.0.smithi100.stdout: 1051: throughput=5MB/sec pending data=0 2024-04-07T16:56:42.766 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-b51hVHx3VQBAVmz off=13272575 len=1823855 2024-04-07T16:56:42.771 INFO:tasks.workunit.client.0.smithi100.stdout:op 5447 completed, throughput=5MB/sec 2024-04-07T16:56:42.771 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-g5eNNY9C77l_neU off=8550910 len=216222 2024-04-07T16:56:42.777 INFO:tasks.workunit.client.0.smithi100.stdout:op 5448 completed, throughput=5MB/sec 2024-04-07T16:56:42.777 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-k7bSd33JbUmyit6 off=7060700 len=1084785 2024-04-07T16:56:42.780 INFO:tasks.workunit.client.0.smithi100.stdout:op 5449 completed, throughput=5MB/sec 2024-04-07T16:56:42.780 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-kXOtLWXmoykR9jt off=2767021 len=353071 2024-04-07T16:56:42.781 INFO:tasks.workunit.client.0.smithi100.stdout:op 5450 completed, throughput=5MB/sec 2024-04-07T16:56:42.781 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-WhOkiBU-28ZtLfI off=1668927 len=1432616 2024-04-07T16:56:42.796 INFO:tasks.workunit.client.0.smithi100.stdout:op 5451 completed, throughput=5MB/sec 2024-04-07T16:56:42.796 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Qe9G94nNEXq1HNY off=7340291 len=1882983 2024-04-07T16:56:42.800 INFO:tasks.workunit.client.0.smithi100.stdout:op 5452 completed, throughput=5MB/sec 2024-04-07T16:56:43.113 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:56:42 smithi144 bash[66654]: cluster 2024-04-07T16:56:41.624540+0000 mgr.y (mgr.25175) 1240 : cluster [DBG] pgmap v1882: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 705 GiB / 715 GiB avail; 1.9 MiB/s rd, 2.9 MiB/s wr, 4 op/s 2024-04-07T16:56:43.199 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:56:42 smithi100 bash[74942]: cluster 2024-04-07T16:56:41.624540+0000 mgr.y (mgr.25175) 1240 : cluster [DBG] pgmap v1882: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 705 GiB / 715 GiB avail; 1.9 MiB/s rd, 2.9 MiB/s wr, 4 op/s 2024-04-07T16:56:43.199 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:56:42 smithi100 bash[76202]: cluster 2024-04-07T16:56:41.624540+0000 mgr.y (mgr.25175) 1240 : cluster [DBG] pgmap v1882: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 705 GiB / 715 GiB avail; 1.9 MiB/s rd, 2.9 MiB/s wr, 4 op/s 2024-04-07T16:56:43.801 INFO:tasks.workunit.client.0.smithi100.stdout: 1052: throughput=5MB/sec pending data=0 2024-04-07T16:56:43.801 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-3ewxiyqvo5Hj5He off=4583332 len=617960 2024-04-07T16:56:43.808 INFO:tasks.workunit.client.0.smithi100.stdout:op 5453 completed, throughput=5MB/sec 2024-04-07T16:56:43.808 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-_qU2kvGdD7PUGMu off=9860424 len=786229 2024-04-07T16:56:43.816 INFO:tasks.workunit.client.0.smithi100.stdout:op 5454 completed, throughput=5MB/sec 2024-04-07T16:56:43.816 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-F6L6OiRB4OewYbc off=7687486 len=988390 2024-04-07T16:56:43.819 INFO:tasks.workunit.client.0.smithi100.stdout:op 5455 completed, throughput=5MB/sec 2024-04-07T16:56:43.819 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-lddaQcIIW267pi6 off=3555427 len=211773 2024-04-07T16:56:43.825 INFO:tasks.workunit.client.0.smithi100.stdout:op 5456 completed, throughput=5MB/sec 2024-04-07T16:56:43.825 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-aIuiLLWPerETn4a off=37959 len=375858 2024-04-07T16:56:43.826 INFO:tasks.workunit.client.0.smithi100.stdout:op 5457 completed, throughput=5MB/sec 2024-04-07T16:56:43.826 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-9FX5PQpwHYsziJY off=235993 len=1097842 2024-04-07T16:56:43.829 INFO:tasks.workunit.client.0.smithi100.stdout:op 5458 completed, throughput=5MB/sec 2024-04-07T16:56:44.829 INFO:tasks.workunit.client.0.smithi100.stdout: 1053: throughput=5MB/sec pending data=0 2024-04-07T16:56:44.830 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-k-sRYka4kr_ABQ1 off=4985443 len=1317963 2024-04-07T16:56:44.840 INFO:tasks.workunit.client.0.smithi100.stdout:op 5459 completed, throughput=5MB/sec 2024-04-07T16:56:44.840 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-i-MEah46gglNrnq off=411639 len=1451681 2024-04-07T16:56:44.844 INFO:tasks.workunit.client.0.smithi100.stdout:op 5460 completed, throughput=5MB/sec 2024-04-07T16:56:44.844 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-uEyDLv3_Zi_gq_5 off=877363 len=1386445 2024-04-07T16:56:44.849 INFO:tasks.workunit.client.0.smithi100.stdout:op 5461 completed, throughput=5MB/sec 2024-04-07T16:56:44.849 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-lEag11Suvxdrm-Q off=8037125 len=130235 2024-04-07T16:56:44.850 INFO:tasks.workunit.client.0.smithi100.stdout:op 5462 completed, throughput=5MB/sec 2024-04-07T16:56:44.850 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-wGR7xkEXTXrka2M off=4503053 len=272693 2024-04-07T16:56:44.852 INFO:tasks.workunit.client.0.smithi100.stdout:op 5463 completed, throughput=5MB/sec 2024-04-07T16:56:44.852 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-JRgekhOnAHqzIP6 off=16476285 len=1581302 2024-04-07T16:56:44.864 INFO:tasks.workunit.client.0.smithi100.stdout:op 5464 completed, throughput=5MB/sec 2024-04-07T16:56:45.113 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:56:44 smithi144 bash[66654]: cluster 2024-04-07T16:56:43.625668+0000 mgr.y (mgr.25175) 1241 : cluster [DBG] pgmap v1883: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 705 GiB / 715 GiB avail; 2.0 MiB/s rd, 3.3 MiB/s wr, 5 op/s 2024-04-07T16:56:45.199 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:56:44 smithi100 bash[74942]: cluster 2024-04-07T16:56:43.625668+0000 mgr.y (mgr.25175) 1241 : cluster [DBG] pgmap v1883: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 705 GiB / 715 GiB avail; 2.0 MiB/s rd, 3.3 MiB/s wr, 5 op/s 2024-04-07T16:56:45.199 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:56:44 smithi100 bash[76202]: cluster 2024-04-07T16:56:43.625668+0000 mgr.y (mgr.25175) 1241 : cluster [DBG] pgmap v1883: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 705 GiB / 715 GiB avail; 2.0 MiB/s rd, 3.3 MiB/s wr, 5 op/s 2024-04-07T16:56:45.865 INFO:tasks.workunit.client.0.smithi100.stdout: 1054: throughput=5MB/sec pending data=0 2024-04-07T16:56:45.865 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-M7j1xMyD7H6vkaO off=15239564 len=1591074 2024-04-07T16:56:45.879 INFO:tasks.workunit.client.0.smithi100.stdout:op 5465 completed, throughput=5MB/sec 2024-04-07T16:56:45.879 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-1q0r7VEH1J5t8Z3 off=7833912 len=973869 2024-04-07T16:56:45.881 INFO:tasks.workunit.client.0.smithi100.stdout:op 5466 completed, throughput=5MB/sec 2024-04-07T16:56:45.882 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Xfn2c0F-Lewhueq off=1557059 len=2054269 2024-04-07T16:56:45.901 INFO:tasks.workunit.client.0.smithi100.stdout:op 5467 completed, throughput=5MB/sec 2024-04-07T16:56:46.901 INFO:tasks.workunit.client.0.smithi100.stdout: 1055: throughput=5MB/sec pending data=0 2024-04-07T16:56:46.901 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-B06RsYc9hPlUhET off=16220440 len=1797797 2024-04-07T16:56:46.906 INFO:tasks.workunit.client.0.smithi100.stdout:op 5468 completed, throughput=5MB/sec 2024-04-07T16:56:46.906 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-MQfv8UAVpQ3hBIT off=9430581 len=1491103 2024-04-07T16:56:46.918 INFO:tasks.workunit.client.0.smithi100.stdout:op 5469 completed, throughput=5MB/sec 2024-04-07T16:56:46.918 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-qGMBtfhtgknMg_3 off=9676119 len=61375 2024-04-07T16:56:46.922 INFO:tasks.workunit.client.0.smithi100.stdout:op 5470 completed, throughput=5MB/sec 2024-04-07T16:56:46.922 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-IPJFOJtyohlYAVF off=1138872 len=523996 2024-04-07T16:56:46.925 INFO:tasks.workunit.client.0.smithi100.stdout:op 5471 completed, throughput=5MB/sec 2024-04-07T16:56:46.925 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-aoRGsRJ3wG6Ue7l off=1621011 len=1811321 2024-04-07T16:56:46.929 INFO:tasks.workunit.client.0.smithi100.stdout:op 5472 completed, throughput=5MB/sec 2024-04-07T16:56:47.113 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:56:46 smithi144 bash[66654]: cluster 2024-04-07T16:56:45.626635+0000 mgr.y (mgr.25175) 1242 : cluster [DBG] pgmap v1884: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 2.0 MiB/s rd, 3.0 MiB/s wr, 5 op/s 2024-04-07T16:56:47.199 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:56:46 smithi100 bash[74942]: cluster 2024-04-07T16:56:45.626635+0000 mgr.y (mgr.25175) 1242 : cluster [DBG] pgmap v1884: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 2.0 MiB/s rd, 3.0 MiB/s wr, 5 op/s 2024-04-07T16:56:47.199 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:56:46 smithi100 bash[76202]: cluster 2024-04-07T16:56:45.626635+0000 mgr.y (mgr.25175) 1242 : cluster [DBG] pgmap v1884: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 2.0 MiB/s rd, 3.0 MiB/s wr, 5 op/s 2024-04-07T16:56:47.929 INFO:tasks.workunit.client.0.smithi100.stdout: 1056: throughput=5MB/sec pending data=0 2024-04-07T16:56:47.929 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-sW31kG_e4oNWzo_ off=1021388 len=923393 2024-04-07T16:56:47.932 INFO:tasks.workunit.client.0.smithi100.stdout:op 5473 completed, throughput=5MB/sec 2024-04-07T16:56:47.932 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-qSEg3d1XyMgmTcX off=186554 len=2022562 2024-04-07T16:56:47.937 INFO:tasks.workunit.client.0.smithi100.stdout:op 5474 completed, throughput=5MB/sec 2024-04-07T16:56:47.937 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-s9zzKWGCJjdCdeA off=1891369 len=1434086 2024-04-07T16:56:47.941 INFO:tasks.workunit.client.0.smithi100.stdout:op 5475 completed, throughput=5MB/sec 2024-04-07T16:56:47.941 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-gAWKJNUxPcoQZE5 off=463875 len=1951578 2024-04-07T16:56:47.946 INFO:tasks.workunit.client.0.smithi100.stdout:op 5476 completed, throughput=5MB/sec 2024-04-07T16:56:48.113 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:56:47 smithi144 bash[66654]: audit 2024-04-07T16:56:46.787625+0000 mon.a (mon.0) 2201 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:56:48.200 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:56:47 smithi100 bash[74942]: audit 2024-04-07T16:56:46.787625+0000 mon.a (mon.0) 2201 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:56:48.200 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:56:47 smithi100 bash[76202]: audit 2024-04-07T16:56:46.787625+0000 mon.a (mon.0) 2201 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:56:48.699 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:56:48 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:56:48] "GET /metrics HTTP/1.1" 200 34070 "" "Prometheus/2.43.0" 2024-04-07T16:56:48.946 INFO:tasks.workunit.client.0.smithi100.stdout: 1057: throughput=5MB/sec pending data=0 2024-04-07T16:56:48.947 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-7ukxeEp8Oe1klD7 off=8539777 len=77938 2024-04-07T16:56:48.948 INFO:tasks.workunit.client.0.smithi100.stdout:op 5477 completed, throughput=5MB/sec 2024-04-07T16:56:48.948 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-PAWJ3yvOdLGgvzU off=1021714 len=1671728 2024-04-07T16:56:48.951 INFO:tasks.workunit.client.0.smithi100.stdout:op 5478 completed, throughput=5MB/sec 2024-04-07T16:56:48.952 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-cFb2Hq7jIXZho6- off=2198464 len=647273 2024-04-07T16:56:48.954 INFO:tasks.workunit.client.0.smithi100.stdout:op 5479 completed, throughput=5MB/sec 2024-04-07T16:56:48.954 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-UkA2lJlT1LGBiVf off=4056906 len=1859715 2024-04-07T16:56:48.972 INFO:tasks.workunit.client.0.smithi100.stdout:op 5480 completed, throughput=5MB/sec 2024-04-07T16:56:49.113 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:56:48 smithi144 bash[66654]: cluster 2024-04-07T16:56:47.627695+0000 mgr.y (mgr.25175) 1243 : cluster [DBG] pgmap v1885: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 2.2 MiB/s rd, 3.0 MiB/s wr, 5 op/s 2024-04-07T16:56:49.199 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:56:48 smithi100 bash[74942]: cluster 2024-04-07T16:56:47.627695+0000 mgr.y (mgr.25175) 1243 : cluster [DBG] pgmap v1885: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 2.2 MiB/s rd, 3.0 MiB/s wr, 5 op/s 2024-04-07T16:56:49.199 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:56:48 smithi100 bash[76202]: cluster 2024-04-07T16:56:47.627695+0000 mgr.y (mgr.25175) 1243 : cluster [DBG] pgmap v1885: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 2.2 MiB/s rd, 3.0 MiB/s wr, 5 op/s 2024-04-07T16:56:49.972 INFO:tasks.workunit.client.0.smithi100.stdout: 1058: throughput=5MB/sec pending data=0 2024-04-07T16:56:49.972 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-l93MiKphu1ocGeT off=19047346 len=1400655 2024-04-07T16:56:49.983 INFO:tasks.workunit.client.0.smithi100.stdout:op 5481 completed, throughput=5MB/sec 2024-04-07T16:56:49.984 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-5XHA1UO5In3Ipwm off=13509428 len=513449 2024-04-07T16:56:49.990 INFO:tasks.workunit.client.0.smithi100.stdout:op 5482 completed, throughput=5MB/sec 2024-04-07T16:56:49.990 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-D-fTnBvTZ2tgfAz off=1653250 len=919526 2024-04-07T16:56:49.999 INFO:tasks.workunit.client.0.smithi100.stdout:op 5483 completed, throughput=5MB/sec 2024-04-07T16:56:49.999 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-xZQ2672EYSCtGSA off=11258759 len=438563 2024-04-07T16:56:50.005 INFO:tasks.workunit.client.0.smithi100.stdout:op 5484 completed, throughput=5MB/sec 2024-04-07T16:56:50.005 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-hbuUnVL5yYXdoG5 off=15188205 len=133634 2024-04-07T16:56:50.011 INFO:tasks.workunit.client.0.smithi100.stdout:op 5485 completed, throughput=5MB/sec 2024-04-07T16:56:50.011 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-B1OTtcKd9Wg86sq off=4963005 len=305513 2024-04-07T16:56:50.017 INFO:tasks.workunit.client.0.smithi100.stdout:op 5486 completed, throughput=5MB/sec 2024-04-07T16:56:50.017 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-2Frp7AAvw9YGUdF off=1757260 len=1835777 2024-04-07T16:56:50.030 INFO:tasks.workunit.client.0.smithi100.stdout:op 5487 completed, throughput=5MB/sec 2024-04-07T16:56:50.113 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:56:49 smithi144 bash[66654]: cluster 2024-04-07T16:56:49.628643+0000 mgr.y (mgr.25175) 1244 : cluster [DBG] pgmap v1886: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 2.0 MiB/s rd, 2.7 MiB/s wr, 4 op/s 2024-04-07T16:56:50.199 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:56:49 smithi100 bash[74942]: cluster 2024-04-07T16:56:49.628643+0000 mgr.y (mgr.25175) 1244 : cluster [DBG] pgmap v1886: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 2.0 MiB/s rd, 2.7 MiB/s wr, 4 op/s 2024-04-07T16:56:50.199 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:56:49 smithi100 bash[76202]: cluster 2024-04-07T16:56:49.628643+0000 mgr.y (mgr.25175) 1244 : cluster [DBG] pgmap v1886: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 2.0 MiB/s rd, 2.7 MiB/s wr, 4 op/s 2024-04-07T16:56:51.031 INFO:tasks.workunit.client.0.smithi100.stdout: 1059: throughput=5MB/sec pending data=0 2024-04-07T16:56:51.031 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-e6Shnjf1hLKpGA1 off=1048985 len=306288 2024-04-07T16:56:51.032 INFO:tasks.workunit.client.0.smithi100.stdout:op 5488 completed, throughput=5MB/sec 2024-04-07T16:56:51.032 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-dlJourOyRLSceQG off=10689823 len=69400 2024-04-07T16:56:51.037 INFO:tasks.workunit.client.0.smithi100.stdout:op 5489 completed, throughput=5MB/sec 2024-04-07T16:56:51.037 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-fRDMXgXQvY2aNlU off=3430896 len=225866 2024-04-07T16:56:51.042 INFO:tasks.workunit.client.0.smithi100.stdout:op 5490 completed, throughput=5MB/sec 2024-04-07T16:56:51.042 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-l_Balum2NSmmGq8 off=500543 len=1236112 2024-04-07T16:56:51.053 INFO:tasks.workunit.client.0.smithi100.stdout:op 5491 completed, throughput=5MB/sec 2024-04-07T16:56:51.053 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-i97K-FLNA7_2q-1 off=1990054 len=910174 2024-04-07T16:56:51.056 INFO:tasks.workunit.client.0.smithi100.stdout:op 5492 completed, throughput=5MB/sec 2024-04-07T16:56:51.056 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-QMFgvjxtqFpsZvU off=2107843 len=64339 2024-04-07T16:56:51.060 INFO:tasks.workunit.client.0.smithi100.stdout:op 5493 completed, throughput=5MB/sec 2024-04-07T16:56:51.060 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-hxah47FDJSUpF9R off=1255378 len=1643752 2024-04-07T16:56:51.065 INFO:tasks.workunit.client.0.smithi100.stdout:op 5494 completed, throughput=5MB/sec 2024-04-07T16:56:51.065 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-4J7xS0qXkPOGcF9 off=226040 len=1276637 2024-04-07T16:56:51.071 INFO:tasks.workunit.client.0.smithi100.stdout:op 5495 completed, throughput=5MB/sec 2024-04-07T16:56:52.071 INFO:tasks.workunit.client.0.smithi100.stdout: 1060: throughput=5MB/sec pending data=0 2024-04-07T16:56:52.071 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-i0ZilSbxPXSsby8 off=6952042 len=322493 2024-04-07T16:56:52.077 INFO:tasks.workunit.client.0.smithi100.stdout:op 5496 completed, throughput=5MB/sec 2024-04-07T16:56:52.077 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-i-DQZGAHsj2opsS off=10097986 len=740526 2024-04-07T16:56:52.080 INFO:tasks.workunit.client.0.smithi100.stdout:op 5497 completed, throughput=5MB/sec 2024-04-07T16:56:52.080 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-1gsci6oxalFHYcn off=1321670 len=1424877 2024-04-07T16:56:52.094 INFO:tasks.workunit.client.0.smithi100.stdout:op 5498 completed, throughput=5MB/sec 2024-04-07T16:56:52.094 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-q0IHl9yBtwzdYxp off=4860932 len=1706807 2024-04-07T16:56:52.108 INFO:tasks.workunit.client.0.smithi100.stdout:op 5499 completed, throughput=5MB/sec 2024-04-07T16:56:52.108 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-bexO6dQ_DlUXKOk off=14435364 len=1559343 2024-04-07T16:56:52.121 INFO:tasks.workunit.client.0.smithi100.stdout:op 5500 completed, throughput=5MB/sec 2024-04-07T16:56:52.613 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:56:52 smithi144 bash[95233]: ts=2024-04-07T16:56:52.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:56:52.949 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:56:52 smithi100 bash[74942]: cluster 2024-04-07T16:56:51.629498+0000 mgr.y (mgr.25175) 1245 : cluster [DBG] pgmap v1887: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 2.2 MiB/s rd, 2.7 MiB/s wr, 4 op/s 2024-04-07T16:56:52.950 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:56:52 smithi100 bash[76202]: cluster 2024-04-07T16:56:51.629498+0000 mgr.y (mgr.25175) 1245 : cluster [DBG] pgmap v1887: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 2.2 MiB/s rd, 2.7 MiB/s wr, 4 op/s 2024-04-07T16:56:53.113 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:56:52 smithi144 bash[66654]: cluster 2024-04-07T16:56:51.629498+0000 mgr.y (mgr.25175) 1245 : cluster [DBG] pgmap v1887: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 2.2 MiB/s rd, 2.7 MiB/s wr, 4 op/s 2024-04-07T16:56:53.121 INFO:tasks.workunit.client.0.smithi100.stdout: 1061: throughput=5MB/sec pending data=0 2024-04-07T16:56:53.121 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-A7k7nwQ8K0TbXjc off=4427870 len=895346 2024-04-07T16:56:53.125 INFO:tasks.workunit.client.0.smithi100.stdout:op 5501 completed, throughput=5MB/sec 2024-04-07T16:56:53.125 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-tgjf18zkWJg3J-T off=5868821 len=99138 2024-04-07T16:56:53.127 INFO:tasks.workunit.client.0.smithi100.stdout:op 5502 completed, throughput=5MB/sec 2024-04-07T16:56:53.127 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-tM5Z5KaD-vQJcoe off=15677304 len=1293056 2024-04-07T16:56:53.131 INFO:tasks.workunit.client.0.smithi100.stdout:op 5503 completed, throughput=5MB/sec 2024-04-07T16:56:53.131 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-VS0UGAgkcYNaqef off=4657835 len=2029348 2024-04-07T16:56:53.147 INFO:tasks.workunit.client.0.smithi100.stdout:op 5504 completed, throughput=5MB/sec 2024-04-07T16:56:53.147 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-zgOsYnDGXT_IZdg off=3108185 len=560255 2024-04-07T16:56:53.149 INFO:tasks.workunit.client.0.smithi100.stdout:op 5505 completed, throughput=5MB/sec 2024-04-07T16:56:54.150 INFO:tasks.workunit.client.0.smithi100.stdout: 1062: throughput=5MB/sec pending data=0 2024-04-07T16:56:54.150 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-oulPEMNyBVOy7LT off=17758543 len=511729 2024-04-07T16:56:54.152 INFO:tasks.workunit.client.0.smithi100.stdout:op 5506 completed, throughput=5MB/sec 2024-04-07T16:56:54.152 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-tM5Z5KaD-vQJcoe off=23594992 len=121799 2024-04-07T16:56:54.157 INFO:tasks.workunit.client.0.smithi100.stdout:op 5507 completed, throughput=5MB/sec 2024-04-07T16:56:54.157 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-XEgOMQlp74x4rdU off=7418192 len=908204 2024-04-07T16:56:54.159 INFO:tasks.workunit.client.0.smithi100.stdout:op 5508 completed, throughput=5MB/sec 2024-04-07T16:56:54.160 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-xkRO1sv8FC1lTGS off=4986751 len=1543496 2024-04-07T16:56:54.163 INFO:tasks.workunit.client.0.smithi100.stdout:op 5509 completed, throughput=5MB/sec 2024-04-07T16:56:54.163 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-eM882wH90DiLyfN off=2743158 len=1386461 2024-04-07T16:56:54.167 INFO:tasks.workunit.client.0.smithi100.stdout:op 5510 completed, throughput=5MB/sec 2024-04-07T16:56:54.167 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-R2KHPrYZ-OxR3lW off=4363045 len=1965310 2024-04-07T16:56:54.183 INFO:tasks.workunit.client.0.smithi100.stdout:op 5511 completed, throughput=5MB/sec 2024-04-07T16:56:54.949 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:56:54 smithi100 bash[74942]: cluster 2024-04-07T16:56:53.630612+0000 mgr.y (mgr.25175) 1246 : cluster [DBG] pgmap v1888: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 2.9 MiB/s rd, 2.3 MiB/s wr, 5 op/s 2024-04-07T16:56:54.949 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:56:54 smithi100 bash[76202]: cluster 2024-04-07T16:56:53.630612+0000 mgr.y (mgr.25175) 1246 : cluster [DBG] pgmap v1888: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 2.9 MiB/s rd, 2.3 MiB/s wr, 5 op/s 2024-04-07T16:56:55.113 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:56:54 smithi144 bash[66654]: cluster 2024-04-07T16:56:53.630612+0000 mgr.y (mgr.25175) 1246 : cluster [DBG] pgmap v1888: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 2.9 MiB/s rd, 2.3 MiB/s wr, 5 op/s 2024-04-07T16:56:55.184 INFO:tasks.workunit.client.0.smithi100.stdout: 1063: throughput=5MB/sec pending data=0 2024-04-07T16:56:55.184 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-UhzcSyt6I1aZwti off=6247806 len=1367042 2024-04-07T16:56:55.196 INFO:tasks.workunit.client.0.smithi100.stdout:op 5512 completed, throughput=5MB/sec 2024-04-07T16:56:55.196 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-hIpoC9MkPqPbYWW off=16872709 len=1473509 2024-04-07T16:56:55.200 INFO:tasks.workunit.client.0.smithi100.stdout:op 5513 completed, throughput=5MB/sec 2024-04-07T16:56:55.200 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Dr92P1xcXaz-bQI off=11414342 len=35305 2024-04-07T16:56:55.201 INFO:tasks.workunit.client.0.smithi100.stdout:op 5514 completed, throughput=5MB/sec 2024-04-07T16:56:55.201 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-1uNXTsGKm-Rf7JH off=5448587 len=913124 2024-04-07T16:56:55.211 INFO:tasks.workunit.client.0.smithi100.stdout:op 5515 completed, throughput=5MB/sec 2024-04-07T16:56:55.211 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-98a1p80mU8Se68W off=1601871 len=123204 2024-04-07T16:56:55.212 INFO:tasks.workunit.client.0.smithi100.stdout:op 5516 completed, throughput=5MB/sec 2024-04-07T16:56:55.212 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-80ehkdreTpAeSgX off=3840794 len=1384620 2024-04-07T16:56:55.223 INFO:tasks.workunit.client.0.smithi100.stdout:op 5517 completed, throughput=5MB/sec 2024-04-07T16:56:56.224 INFO:tasks.workunit.client.0.smithi100.stdout: 1064: throughput=5MB/sec pending data=0 2024-04-07T16:56:56.224 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-gmlmpmgX9CaMovg off=1531400 len=1012240 2024-04-07T16:56:56.227 INFO:tasks.workunit.client.0.smithi100.stdout:op 5518 completed, throughput=5MB/sec 2024-04-07T16:56:56.227 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-TQ6yq1lbb6CFW45 off=1085739 len=1153553 2024-04-07T16:56:56.232 INFO:tasks.workunit.client.0.smithi100.stdout:op 5519 completed, throughput=5MB/sec 2024-04-07T16:56:56.232 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-qfimu-WswKoCh5S off=3374136 len=818100 2024-04-07T16:56:56.235 INFO:tasks.workunit.client.0.smithi100.stdout:op 5520 completed, throughput=5MB/sec 2024-04-07T16:56:56.235 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-mIriXEK-yHxRBAU off=20815 len=165358 2024-04-07T16:56:56.236 INFO:tasks.workunit.client.0.smithi100.stdout:op 5521 completed, throughput=5MB/sec 2024-04-07T16:56:56.236 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Xfn2c0F-Lewhueq off=3036572 len=1620928 2024-04-07T16:56:56.241 INFO:tasks.workunit.client.0.smithi100.stdout:op 5522 completed, throughput=5MB/sec 2024-04-07T16:56:56.949 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:56:56 smithi100 bash[74942]: cluster 2024-04-07T16:56:55.631626+0000 mgr.y (mgr.25175) 1247 : cluster [DBG] pgmap v1889: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 2.9 MiB/s rd, 2.0 MiB/s wr, 5 op/s 2024-04-07T16:56:56.949 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:56:56 smithi100 bash[76202]: cluster 2024-04-07T16:56:55.631626+0000 mgr.y (mgr.25175) 1247 : cluster [DBG] pgmap v1889: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 2.9 MiB/s rd, 2.0 MiB/s wr, 5 op/s 2024-04-07T16:56:57.113 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:56:56 smithi144 bash[66654]: cluster 2024-04-07T16:56:55.631626+0000 mgr.y (mgr.25175) 1247 : cluster [DBG] pgmap v1889: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 2.9 MiB/s rd, 2.0 MiB/s wr, 5 op/s 2024-04-07T16:56:57.241 INFO:tasks.workunit.client.0.smithi100.stdout: 1065: throughput=5MB/sec pending data=0 2024-04-07T16:56:57.242 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-KX6H5ntPwpCE5nz off=4856461 len=10234 2024-04-07T16:56:57.242 INFO:tasks.workunit.client.0.smithi100.stdout:op 5523 completed, throughput=5MB/sec 2024-04-07T16:56:57.243 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-giU91g-73bC2UcB off=5995596 len=1398736 2024-04-07T16:56:57.255 INFO:tasks.workunit.client.0.smithi100.stdout:op 5524 completed, throughput=5MB/sec 2024-04-07T16:56:57.255 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-u5dJSCH_J_JgpND off=2772399 len=1714142 2024-04-07T16:56:57.259 INFO:tasks.workunit.client.0.smithi100.stdout:op 5525 completed, throughput=5MB/sec 2024-04-07T16:56:57.259 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-yi1mHl25nv6hgHk off=12291083 len=700604 2024-04-07T16:56:57.267 INFO:tasks.workunit.client.0.smithi100.stdout:op 5526 completed, throughput=5MB/sec 2024-04-07T16:56:57.267 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-60DCVDZeY5Aap5T off=4932229 len=1749909 2024-04-07T16:56:57.269 INFO:tasks.workunit.client.0.smithi100.stdout:op 5527 completed, throughput=5MB/sec 2024-04-07T16:56:58.271 INFO:tasks.workunit.client.0.smithi100.stdout: 1066: throughput=5MB/sec pending data=0 2024-04-07T16:56:58.271 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-f0zxJLaV1P7eFF6 off=4445911 len=470110 2024-04-07T16:56:58.277 INFO:tasks.workunit.client.0.smithi100.stdout:op 5528 completed, throughput=5MB/sec 2024-04-07T16:56:58.277 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-HQ9drjHo2wtZ9Ny off=4233218 len=493326 2024-04-07T16:56:58.280 INFO:tasks.workunit.client.0.smithi100.stdout:op 5529 completed, throughput=5MB/sec 2024-04-07T16:56:58.280 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-TlP8-NLzfV8P4cd off=612843 len=1690820 2024-04-07T16:56:58.292 INFO:tasks.workunit.client.0.smithi100.stdout:op 5530 completed, throughput=5MB/sec 2024-04-07T16:56:58.292 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-wbGBYMYVRGlL9sk off=872590 len=282801 2024-04-07T16:56:58.300 INFO:tasks.workunit.client.0.smithi100.stdout:op 5531 completed, throughput=5MB/sec 2024-04-07T16:56:58.300 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj--8VFUBTYdGGdtdi off=171678 len=1291921 2024-04-07T16:56:58.302 INFO:tasks.workunit.client.0.smithi100.stdout:op 5532 completed, throughput=5MB/sec 2024-04-07T16:56:58.302 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-uDb-G1mtacMkhKJ off=12336359 len=1859300 2024-04-07T16:56:58.317 INFO:tasks.workunit.client.0.smithi100.stdout:op 5533 completed, throughput=5MB/sec 2024-04-07T16:56:58.694 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:56:58 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:56:58] "GET /metrics HTTP/1.1" 200 34070 "" "Prometheus/2.43.0" 2024-04-07T16:56:58.949 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:56:58 smithi100 bash[74942]: cluster 2024-04-07T16:56:57.632626+0000 mgr.y (mgr.25175) 1248 : cluster [DBG] pgmap v1890: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 2.8 MiB/s rd, 2.1 MiB/s wr, 4 op/s 2024-04-07T16:56:58.949 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:56:58 smithi100 bash[76202]: cluster 2024-04-07T16:56:57.632626+0000 mgr.y (mgr.25175) 1248 : cluster [DBG] pgmap v1890: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 2.8 MiB/s rd, 2.1 MiB/s wr, 4 op/s 2024-04-07T16:56:59.113 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:56:58 smithi144 bash[66654]: cluster 2024-04-07T16:56:57.632626+0000 mgr.y (mgr.25175) 1248 : cluster [DBG] pgmap v1890: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 2.8 MiB/s rd, 2.1 MiB/s wr, 4 op/s 2024-04-07T16:56:59.317 INFO:tasks.workunit.client.0.smithi100.stdout: 1067: throughput=5MB/sec pending data=0 2024-04-07T16:56:59.318 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Nz_DJZQcKV5vqfl off=3567734 len=399895 2024-04-07T16:56:59.319 INFO:tasks.workunit.client.0.smithi100.stdout:op 5534 completed, throughput=5MB/sec 2024-04-07T16:56:59.319 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-nhf8j-M965OJ8n5 off=6400607 len=668776 2024-04-07T16:56:59.326 INFO:tasks.workunit.client.0.smithi100.stdout:op 5535 completed, throughput=5MB/sec 2024-04-07T16:56:59.327 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Z0p5wUb8_r09AmW off=5815404 len=883987 2024-04-07T16:56:59.328 INFO:tasks.workunit.client.0.smithi100.stdout:op 5536 completed, throughput=5MB/sec 2024-04-07T16:56:59.328 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-apdrlHZL72icgF1 off=12507158 len=1747881 2024-04-07T16:56:59.344 INFO:tasks.workunit.client.0.smithi100.stdout:op 5537 completed, throughput=5MB/sec 2024-04-07T16:56:59.344 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-kM6OpHIiXZK1vPL off=15789220 len=704647 2024-04-07T16:56:59.345 INFO:tasks.workunit.client.0.smithi100.stdout:op 5538 completed, throughput=5MB/sec 2024-04-07T16:57:00.346 INFO:tasks.workunit.client.0.smithi100.stdout: 1068: throughput=5MB/sec pending data=0 2024-04-07T16:57:00.346 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-fUlOo_k3w3S2fZU off=13199133 len=39648 2024-04-07T16:57:00.350 INFO:tasks.workunit.client.0.smithi100.stdout:op 5539 completed, throughput=5MB/sec 2024-04-07T16:57:00.350 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-GydkTbhwZad1SNi off=2797674 len=257737 2024-04-07T16:57:00.356 INFO:tasks.workunit.client.0.smithi100.stdout:op 5540 completed, throughput=5MB/sec 2024-04-07T16:57:00.357 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-pW11VG0r2VyWIMe off=1591112 len=470977 2024-04-07T16:57:00.358 INFO:tasks.workunit.client.0.smithi100.stdout:op 5541 completed, throughput=5MB/sec 2024-04-07T16:57:00.358 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-jJhF9RUHQC-w2kf off=2685581 len=1138567 2024-04-07T16:57:00.361 INFO:tasks.workunit.client.0.smithi100.stdout:op 5542 completed, throughput=5MB/sec 2024-04-07T16:57:00.361 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-m0hqcN8MgvBF_lb off=8568304 len=474045 2024-04-07T16:57:00.363 INFO:tasks.workunit.client.0.smithi100.stdout:op 5543 completed, throughput=5MB/sec 2024-04-07T16:57:00.363 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-3NZLWJJV29uB1C7 off=3673585 len=163040 2024-04-07T16:57:00.368 INFO:tasks.workunit.client.0.smithi100.stdout:op 5544 completed, throughput=5MB/sec 2024-04-07T16:57:00.368 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-n7zSfZBxlzGiuuB off=102652 len=1833770 2024-04-07T16:57:00.383 INFO:tasks.workunit.client.0.smithi100.stdout:op 5545 completed, throughput=5MB/sec 2024-04-07T16:57:00.383 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-xHpFlJ7lVIlQtir off=607208 len=1319561 2024-04-07T16:57:00.392 INFO:tasks.workunit.client.0.smithi100.stdout:op 5546 completed, throughput=5MB/sec 2024-04-07T16:57:00.949 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:57:00 smithi100 bash[74942]: cluster 2024-04-07T16:56:59.633696+0000 mgr.y (mgr.25175) 1249 : cluster [DBG] pgmap v1891: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 2.6 MiB/s rd, 2.0 MiB/s wr, 4 op/s 2024-04-07T16:57:00.950 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:57:00 smithi100 bash[76202]: cluster 2024-04-07T16:56:59.633696+0000 mgr.y (mgr.25175) 1249 : cluster [DBG] pgmap v1891: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 2.6 MiB/s rd, 2.0 MiB/s wr, 4 op/s 2024-04-07T16:57:01.113 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:57:00 smithi144 bash[66654]: cluster 2024-04-07T16:56:59.633696+0000 mgr.y (mgr.25175) 1249 : cluster [DBG] pgmap v1891: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 2.6 MiB/s rd, 2.0 MiB/s wr, 4 op/s 2024-04-07T16:57:01.393 INFO:tasks.workunit.client.0.smithi100.stdout: 1069: throughput=5MB/sec pending data=0 2024-04-07T16:57:01.393 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-mMo4Ao8i9Kv9C3n off=8797890 len=996801 2024-04-07T16:57:01.395 INFO:tasks.workunit.client.0.smithi100.stdout:op 5547 completed, throughput=5MB/sec 2024-04-07T16:57:01.395 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-zMA8XWDRsAWEsnR off=15795824 len=1632582 2024-04-07T16:57:01.399 INFO:tasks.workunit.client.0.smithi100.stdout:op 5548 completed, throughput=5MB/sec 2024-04-07T16:57:01.399 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-VjfBnjV9Viq0gTP off=2058079 len=1045742 2024-04-07T16:57:01.402 INFO:tasks.workunit.client.0.smithi100.stdout:op 5549 completed, throughput=5MB/sec 2024-04-07T16:57:01.402 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-kFqcytkA2BJ3eEM off=11390001 len=687925 2024-04-07T16:57:01.410 INFO:tasks.workunit.client.0.smithi100.stdout:op 5550 completed, throughput=5MB/sec 2024-04-07T16:57:01.410 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-q3rq9OlzPeP_8v_ off=3563057 len=1986643 2024-04-07T16:57:01.414 INFO:tasks.workunit.client.0.smithi100.stdout:op 5551 completed, throughput=5MB/sec 2024-04-07T16:57:02.415 INFO:tasks.workunit.client.0.smithi100.stdout: 1070: throughput=5MB/sec pending data=0 2024-04-07T16:57:02.415 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-ETZ2U-5OrSi_IKj off=11724033 len=282124 2024-04-07T16:57:02.416 INFO:tasks.workunit.client.0.smithi100.stdout:op 5552 completed, throughput=5MB/sec 2024-04-07T16:57:02.417 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-iAH93a4Luctnp_9 off=3694780 len=326967 2024-04-07T16:57:02.423 INFO:tasks.workunit.client.0.smithi100.stdout:op 5553 completed, throughput=5MB/sec 2024-04-07T16:57:02.423 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-SjPtYa_2QRLYYze off=13676478 len=866212 2024-04-07T16:57:02.431 INFO:tasks.workunit.client.0.smithi100.stdout:op 5554 completed, throughput=5MB/sec 2024-04-07T16:57:02.432 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-oJWDHc1Zuhbis_D off=5077429 len=752618 2024-04-07T16:57:02.434 INFO:tasks.workunit.client.0.smithi100.stdout:op 5555 completed, throughput=5MB/sec 2024-04-07T16:57:02.434 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-uLYmsRJtfR-j8ca off=7781031 len=1415868 2024-04-07T16:57:02.448 INFO:tasks.workunit.client.0.smithi100.stdout:op 5556 completed, throughput=5MB/sec 2024-04-07T16:57:02.448 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-0OVgRKZN3i3h8pl off=1112041 len=1986381 2024-04-07T16:57:02.453 INFO:tasks.workunit.client.0.smithi100.stdout:op 5557 completed, throughput=5MB/sec 2024-04-07T16:57:02.613 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:57:02 smithi144 bash[95233]: ts=2024-04-07T16:57:02.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:57:03.113 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:57:02 smithi144 bash[66654]: cluster 2024-04-07T16:57:01.634662+0000 mgr.y (mgr.25175) 1250 : cluster [DBG] pgmap v1892: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.4 MiB/s wr, 5 op/s 2024-04-07T16:57:03.113 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:57:02 smithi144 bash[66654]: audit 2024-04-07T16:57:01.788054+0000 mon.a (mon.0) 2202 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:57:03.199 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:57:02 smithi100 bash[74942]: cluster 2024-04-07T16:57:01.634662+0000 mgr.y (mgr.25175) 1250 : cluster [DBG] pgmap v1892: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.4 MiB/s wr, 5 op/s 2024-04-07T16:57:03.200 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:57:02 smithi100 bash[74942]: audit 2024-04-07T16:57:01.788054+0000 mon.a (mon.0) 2202 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:57:03.200 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:57:02 smithi100 bash[76202]: cluster 2024-04-07T16:57:01.634662+0000 mgr.y (mgr.25175) 1250 : cluster [DBG] pgmap v1892: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.4 MiB/s wr, 5 op/s 2024-04-07T16:57:03.200 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:57:02 smithi100 bash[76202]: audit 2024-04-07T16:57:01.788054+0000 mon.a (mon.0) 2202 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:57:03.453 INFO:tasks.workunit.client.0.smithi100.stdout: 1071: throughput=5MB/sec pending data=0 2024-04-07T16:57:03.453 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-9jBdo9G5SLlpMkJ off=5232447 len=498229 2024-04-07T16:57:03.460 INFO:tasks.workunit.client.0.smithi100.stdout:op 5558 completed, throughput=5MB/sec 2024-04-07T16:57:03.460 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-u6Lbe1lJE937Srd off=3388399 len=573833 2024-04-07T16:57:03.467 INFO:tasks.workunit.client.0.smithi100.stdout:op 5559 completed, throughput=5MB/sec 2024-04-07T16:57:03.467 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-NE9YbaydCOMp1nq off=719812 len=285191 2024-04-07T16:57:03.473 INFO:tasks.workunit.client.0.smithi100.stdout:op 5560 completed, throughput=5MB/sec 2024-04-07T16:57:03.473 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-zi5jElbm2RQySIO off=488062 len=382239 2024-04-07T16:57:03.475 INFO:tasks.workunit.client.0.smithi100.stdout:op 5561 completed, throughput=5MB/sec 2024-04-07T16:57:03.475 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-SdxbQT36hxMfIu0 off=5737917 len=52344 2024-04-07T16:57:03.479 INFO:tasks.workunit.client.0.smithi100.stdout:op 5562 completed, throughput=5MB/sec 2024-04-07T16:57:03.479 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-WCfGqvm8ZwjqzsB off=15422406 len=1790081 2024-04-07T16:57:03.492 INFO:tasks.workunit.client.0.smithi100.stdout:op 5563 completed, throughput=5MB/sec 2024-04-07T16:57:03.492 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-1Ovjrlk632mW6QB off=3871895 len=877409 2024-04-07T16:57:03.495 INFO:tasks.workunit.client.0.smithi100.stdout:op 5564 completed, throughput=5MB/sec 2024-04-07T16:57:04.495 INFO:tasks.workunit.client.0.smithi100.stdout: 1072: throughput=5MB/sec pending data=0 2024-04-07T16:57:04.495 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-8bc0BZc5kpCOALX off=2924170 len=826608 2024-04-07T16:57:04.498 INFO:tasks.workunit.client.0.smithi100.stdout:op 5565 completed, throughput=5MB/sec 2024-04-07T16:57:04.498 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-3DW1GicLstwH8w_ off=4719114 len=1290031 2024-04-07T16:57:04.509 INFO:tasks.workunit.client.0.smithi100.stdout:op 5566 completed, throughput=5MB/sec 2024-04-07T16:57:04.509 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-5J4rOBRnaas-7eT off=3372393 len=553501 2024-04-07T16:57:04.511 INFO:tasks.workunit.client.0.smithi100.stdout:op 5567 completed, throughput=5MB/sec 2024-04-07T16:57:04.511 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-7tLo14hWqDsJ9_Y off=18632658 len=1499206 2024-04-07T16:57:04.515 INFO:tasks.workunit.client.0.smithi100.stdout:op 5568 completed, throughput=5MB/sec 2024-04-07T16:57:04.515 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-hafIPVqstdnSV__ off=23624659 len=1045290 2024-04-07T16:57:04.524 INFO:tasks.workunit.client.0.smithi100.stdout:op 5569 completed, throughput=5MB/sec 2024-04-07T16:57:05.113 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:57:04 smithi144 bash[66654]: cluster 2024-04-07T16:57:03.635815+0000 mgr.y (mgr.25175) 1251 : cluster [DBG] pgmap v1893: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 2.8 MiB/s rd, 2.8 MiB/s wr, 6 op/s 2024-04-07T16:57:05.199 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:57:04 smithi100 bash[74942]: cluster 2024-04-07T16:57:03.635815+0000 mgr.y (mgr.25175) 1251 : cluster [DBG] pgmap v1893: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 2.8 MiB/s rd, 2.8 MiB/s wr, 6 op/s 2024-04-07T16:57:05.200 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:57:04 smithi100 bash[76202]: cluster 2024-04-07T16:57:03.635815+0000 mgr.y (mgr.25175) 1251 : cluster [DBG] pgmap v1893: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 2.8 MiB/s rd, 2.8 MiB/s wr, 6 op/s 2024-04-07T16:57:05.524 INFO:tasks.workunit.client.0.smithi100.stdout: 1073: throughput=5MB/sec pending data=0 2024-04-07T16:57:05.524 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-YNS9VJr9IEffGBP off=17406673 len=1990340 2024-04-07T16:57:05.528 INFO:tasks.workunit.client.0.smithi100.stdout:op 5570 completed, throughput=5MB/sec 2024-04-07T16:57:05.529 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-D7pTa-sgZRknCQz off=12508447 len=269613 2024-04-07T16:57:05.534 INFO:tasks.workunit.client.0.smithi100.stdout:op 5571 completed, throughput=5MB/sec 2024-04-07T16:57:05.535 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-8wrPqmhHwUdX1DY off=2413474 len=722219 2024-04-07T16:57:05.543 INFO:tasks.workunit.client.0.smithi100.stdout:op 5572 completed, throughput=5MB/sec 2024-04-07T16:57:05.543 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-8haHYepEdl56syO off=4563961 len=1132935 2024-04-07T16:57:05.547 INFO:tasks.workunit.client.0.smithi100.stdout:op 5573 completed, throughput=5MB/sec 2024-04-07T16:57:05.548 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-kNadrv2SglSRklz off=1531783 len=1574491 2024-04-07T16:57:05.560 INFO:tasks.workunit.client.0.smithi100.stdout:op 5574 completed, throughput=5MB/sec 2024-04-07T16:57:06.560 INFO:tasks.workunit.client.0.smithi100.stdout: 1074: throughput=5MB/sec pending data=0 2024-04-07T16:57:06.560 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-O5ozYUeW3mvwK9W off=4873900 len=1151415 2024-04-07T16:57:06.564 INFO:tasks.workunit.client.0.smithi100.stdout:op 5575 completed, throughput=5MB/sec 2024-04-07T16:57:06.564 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-9621VIoyPnodFof off=2858393 len=1885580 2024-04-07T16:57:06.579 INFO:tasks.workunit.client.0.smithi100.stdout:op 5576 completed, throughput=5MB/sec 2024-04-07T16:57:06.579 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-VMTBzOR2uiEBmWd off=1266483 len=1328135 2024-04-07T16:57:06.590 INFO:tasks.workunit.client.0.smithi100.stdout:op 5577 completed, throughput=5MB/sec 2024-04-07T16:57:06.590 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-s5TkutAMCBPEJXR off=6403489 len=710822 2024-04-07T16:57:06.592 INFO:tasks.workunit.client.0.smithi100.stdout:op 5578 completed, throughput=5MB/sec 2024-04-07T16:57:07.113 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:57:06 smithi144 bash[66654]: cluster 2024-04-07T16:57:05.636783+0000 mgr.y (mgr.25175) 1252 : cluster [DBG] pgmap v1894: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 2.6 MiB/s rd, 2.7 MiB/s wr, 5 op/s 2024-04-07T16:57:07.199 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:57:06 smithi100 bash[74942]: cluster 2024-04-07T16:57:05.636783+0000 mgr.y (mgr.25175) 1252 : cluster [DBG] pgmap v1894: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 2.6 MiB/s rd, 2.7 MiB/s wr, 5 op/s 2024-04-07T16:57:07.199 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:57:06 smithi100 bash[76202]: cluster 2024-04-07T16:57:05.636783+0000 mgr.y (mgr.25175) 1252 : cluster [DBG] pgmap v1894: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 2.6 MiB/s rd, 2.7 MiB/s wr, 5 op/s 2024-04-07T16:57:07.592 INFO:tasks.workunit.client.0.smithi100.stdout: 1075: throughput=5MB/sec pending data=0 2024-04-07T16:57:07.592 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-DmEe99GfcWXjhHn off=8325373 len=1533509 2024-04-07T16:57:07.596 INFO:tasks.workunit.client.0.smithi100.stdout:op 5579 completed, throughput=5MB/sec 2024-04-07T16:57:07.596 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-PSpDOSZdAo5Wt4q off=11242404 len=841358 2024-04-07T16:57:07.605 INFO:tasks.workunit.client.0.smithi100.stdout:op 5580 completed, throughput=5MB/sec 2024-04-07T16:57:07.605 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-QiXONvOaGAH-Fho off=553193 len=55854 2024-04-07T16:57:07.610 INFO:tasks.workunit.client.0.smithi100.stdout:op 5581 completed, throughput=5MB/sec 2024-04-07T16:57:07.610 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-9D3bgncW61xPDLk off=5892105 len=1653508 2024-04-07T16:57:07.612 INFO:tasks.workunit.client.0.smithi100.stdout:op 5582 completed, throughput=5MB/sec 2024-04-07T16:57:07.612 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-0MkEdQEwD-c2Wrd off=1828115 len=1388092 2024-04-07T16:57:07.616 INFO:tasks.workunit.client.0.smithi100.stdout:op 5583 completed, throughput=5MB/sec 2024-04-07T16:57:08.617 INFO:tasks.workunit.client.0.smithi100.stdout: 1076: throughput=5MB/sec pending data=0 2024-04-07T16:57:08.617 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-qWnuz7VzV4WGqP2 off=9692951 len=968115 2024-04-07T16:57:08.626 INFO:tasks.workunit.client.0.smithi100.stdout:op 5584 completed, throughput=5MB/sec 2024-04-07T16:57:08.626 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-0US5owbi9Mjc0s1 off=121611 len=573513 2024-04-07T16:57:08.634 INFO:tasks.workunit.client.0.smithi100.stdout:op 5585 completed, throughput=5MB/sec 2024-04-07T16:57:08.634 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-DU1LTwR5mzdyjwu off=11536613 len=540375 2024-04-07T16:57:08.640 INFO:tasks.workunit.client.0.smithi100.stdout:op 5586 completed, throughput=5MB/sec 2024-04-07T16:57:08.641 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-1hXESxHI7HGHC44 off=6831953 len=1273572 2024-04-07T16:57:08.650 INFO:tasks.workunit.client.0.smithi100.stdout:op 5587 completed, throughput=5MB/sec 2024-04-07T16:57:08.651 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-0jDA8vfK2WR-MuS off=7905786 len=1725079 2024-04-07T16:57:08.654 INFO:tasks.workunit.client.0.smithi100.stdout:op 5588 completed, throughput=5MB/sec 2024-04-07T16:57:08.699 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:57:08 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:57:08] "GET /metrics HTTP/1.1" 200 34074 "" "Prometheus/2.43.0" 2024-04-07T16:57:09.113 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:57:08 smithi144 bash[66654]: cluster 2024-04-07T16:57:07.637843+0000 mgr.y (mgr.25175) 1253 : cluster [DBG] pgmap v1895: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 2.6 MiB/s rd, 2.5 MiB/s wr, 5 op/s 2024-04-07T16:57:09.200 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:57:08 smithi100 bash[74942]: cluster 2024-04-07T16:57:07.637843+0000 mgr.y (mgr.25175) 1253 : cluster [DBG] pgmap v1895: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 2.6 MiB/s rd, 2.5 MiB/s wr, 5 op/s 2024-04-07T16:57:09.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:57:08 smithi100 bash[76202]: cluster 2024-04-07T16:57:07.637843+0000 mgr.y (mgr.25175) 1253 : cluster [DBG] pgmap v1895: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 2.6 MiB/s rd, 2.5 MiB/s wr, 5 op/s 2024-04-07T16:57:09.656 INFO:tasks.workunit.client.0.smithi100.stdout: 1077: throughput=5MB/sec pending data=0 2024-04-07T16:57:09.656 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-NJ5WKTEPnZmoIhd off=75052 len=848898 2024-04-07T16:57:09.664 INFO:tasks.workunit.client.0.smithi100.stdout:op 5589 completed, throughput=5MB/sec 2024-04-07T16:57:09.664 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-wTKz_-M24XwTkhf off=1477388 len=349664 2024-04-07T16:57:09.669 INFO:tasks.workunit.client.0.smithi100.stdout:op 5590 completed, throughput=5MB/sec 2024-04-07T16:57:09.670 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-P_QZz3RbbsXOjly off=2683384 len=181883 2024-04-07T16:57:09.671 INFO:tasks.workunit.client.0.smithi100.stdout:op 5591 completed, throughput=5MB/sec 2024-04-07T16:57:09.671 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-FhvCjP0Hg5BwvtT off=281608 len=1069852 2024-04-07T16:57:09.674 INFO:tasks.workunit.client.0.smithi100.stdout:op 5592 completed, throughput=5MB/sec 2024-04-07T16:57:09.674 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-lOpxrW9I3dgSRRK off=10897693 len=752992 2024-04-07T16:57:09.682 INFO:tasks.workunit.client.0.smithi100.stdout:op 5593 completed, throughput=5MB/sec 2024-04-07T16:57:09.682 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-7Gqc2aQAC2vaUDC off=2027650 len=30482 2024-04-07T16:57:09.687 INFO:tasks.workunit.client.0.smithi100.stdout:op 5594 completed, throughput=5MB/sec 2024-04-07T16:57:09.687 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-zaX-PQgQmD7UXoS off=14728066 len=596815 2024-04-07T16:57:09.688 INFO:tasks.workunit.client.0.smithi100.stdout:op 5595 completed, throughput=5MB/sec 2024-04-07T16:57:09.688 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-JXBZweG3F1N0yyM off=5068968 len=212054 2024-04-07T16:57:09.690 INFO:tasks.workunit.client.0.smithi100.stdout:op 5596 completed, throughput=5MB/sec 2024-04-07T16:57:09.690 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-1gymYl_a8zmTXYr off=3455381 len=782239 2024-04-07T16:57:09.693 INFO:tasks.workunit.client.0.smithi100.stdout:op 5597 completed, throughput=5MB/sec 2024-04-07T16:57:09.693 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-0IILJOkewFRNAqL off=14035298 len=1410236 2024-04-07T16:57:09.696 INFO:tasks.workunit.client.0.smithi100.stdout:op 5598 completed, throughput=5MB/sec 2024-04-07T16:57:10.696 INFO:tasks.workunit.client.0.smithi100.stdout: 1078: throughput=5MB/sec pending data=0 2024-04-07T16:57:10.696 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-7TzB_YCecNUC0-I off=1590647 len=2086213 2024-04-07T16:57:10.702 INFO:tasks.workunit.client.0.smithi100.stdout:op 5599 completed, throughput=5MB/sec 2024-04-07T16:57:10.702 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-vMynpn8R1sM_qW9 off=375068 len=883835 2024-04-07T16:57:10.704 INFO:tasks.workunit.client.0.smithi100.stdout:op 5600 completed, throughput=5MB/sec 2024-04-07T16:57:10.705 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-13Hqvy9Fge2ua1p off=10322059 len=566828 2024-04-07T16:57:10.710 INFO:tasks.workunit.client.0.smithi100.stdout:op 5601 completed, throughput=5MB/sec 2024-04-07T16:57:10.710 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-2I5MzIEZkb_RHkj off=5187056 len=71070 2024-04-07T16:57:10.715 INFO:tasks.workunit.client.0.smithi100.stdout:op 5602 completed, throughput=5MB/sec 2024-04-07T16:57:10.715 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-77IIA_OTT0qcCt_ off=1920026 len=1133526 2024-04-07T16:57:10.717 INFO:tasks.workunit.client.0.smithi100.stdout:op 5603 completed, throughput=5MB/sec 2024-04-07T16:57:11.113 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:57:10 smithi144 bash[66654]: cluster 2024-04-07T16:57:09.638774+0000 mgr.y (mgr.25175) 1254 : cluster [DBG] pgmap v1896: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 2.4 MiB/s rd, 2.4 MiB/s wr, 5 op/s 2024-04-07T16:57:11.113 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:57:10 smithi144 bash[66654]: audit 2024-04-07T16:57:10.438668+0000 mon.a (mon.0) 2203 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:57:11.199 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:57:10 smithi100 bash[76202]: cluster 2024-04-07T16:57:09.638774+0000 mgr.y (mgr.25175) 1254 : cluster [DBG] pgmap v1896: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 2.4 MiB/s rd, 2.4 MiB/s wr, 5 op/s 2024-04-07T16:57:11.199 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:57:10 smithi100 bash[76202]: audit 2024-04-07T16:57:10.438668+0000 mon.a (mon.0) 2203 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:57:11.200 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:57:10 smithi100 bash[74942]: cluster 2024-04-07T16:57:09.638774+0000 mgr.y (mgr.25175) 1254 : cluster [DBG] pgmap v1896: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 2.4 MiB/s rd, 2.4 MiB/s wr, 5 op/s 2024-04-07T16:57:11.200 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:57:10 smithi100 bash[74942]: audit 2024-04-07T16:57:10.438668+0000 mon.a (mon.0) 2203 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:57:11.718 INFO:tasks.workunit.client.0.smithi100.stdout: 1079: throughput=5MB/sec pending data=0 2024-04-07T16:57:11.718 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-unpEKhfh-PoNvT1 off=475936 len=314474 2024-04-07T16:57:11.719 INFO:tasks.workunit.client.0.smithi100.stdout:op 5604 completed, throughput=5MB/sec 2024-04-07T16:57:11.719 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-gCau_Ul7WeclzIP off=4691667 len=1920019 2024-04-07T16:57:11.734 INFO:tasks.workunit.client.0.smithi100.stdout:op 5605 completed, throughput=5MB/sec 2024-04-07T16:57:11.734 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-9hnjwsSxce0LHCt off=8437242 len=341641 2024-04-07T16:57:11.736 INFO:tasks.workunit.client.0.smithi100.stdout:op 5606 completed, throughput=5MB/sec 2024-04-07T16:57:11.736 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-iGmXFaK7iK4S4aq off=15057715 len=1186495 2024-04-07T16:57:11.746 INFO:tasks.workunit.client.0.smithi100.stdout:op 5607 completed, throughput=5MB/sec 2024-04-07T16:57:11.746 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-fdhWrdET9IiWIdx off=17622203 len=231953 2024-04-07T16:57:11.752 INFO:tasks.workunit.client.0.smithi100.stdout:op 5608 completed, throughput=5MB/sec 2024-04-07T16:57:11.752 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-gBHdSVGmRfJ9yau off=10374897 len=589163 2024-04-07T16:57:11.754 INFO:tasks.workunit.client.0.smithi100.stdout:op 5609 completed, throughput=5MB/sec 2024-04-07T16:57:11.754 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-IdHbc6J1S2Wrkh1 off=12939428 len=1179274 2024-04-07T16:57:11.764 INFO:tasks.workunit.client.0.smithi100.stdout:op 5610 completed, throughput=5MB/sec 2024-04-07T16:57:12.113 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:57:11 smithi144 bash[66654]: audit 2024-04-07T16:57:11.166931+0000 mon.a (mon.0) 2204 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:57:12.113 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:57:11 smithi144 bash[66654]: audit 2024-04-07T16:57:11.168993+0000 mon.a (mon.0) 2205 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:57:12.113 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:57:11 smithi144 bash[66654]: audit 2024-04-07T16:57:11.180783+0000 mon.a (mon.0) 2206 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:57:12.199 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:57:11 smithi100 bash[74942]: audit 2024-04-07T16:57:11.166931+0000 mon.a (mon.0) 2204 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:57:12.199 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:57:11 smithi100 bash[74942]: audit 2024-04-07T16:57:11.168993+0000 mon.a (mon.0) 2205 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:57:12.199 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:57:11 smithi100 bash[74942]: audit 2024-04-07T16:57:11.180783+0000 mon.a (mon.0) 2206 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:57:12.199 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:57:11 smithi100 bash[76202]: audit 2024-04-07T16:57:11.166931+0000 mon.a (mon.0) 2204 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:57:12.200 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:57:11 smithi100 bash[76202]: audit 2024-04-07T16:57:11.168993+0000 mon.a (mon.0) 2205 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:57:12.200 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:57:11 smithi100 bash[76202]: audit 2024-04-07T16:57:11.180783+0000 mon.a (mon.0) 2206 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:57:12.613 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:57:12 smithi144 bash[95233]: ts=2024-04-07T16:57:12.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:57:12.764 INFO:tasks.workunit.client.0.smithi100.stdout: 1080: throughput=5MB/sec pending data=0 2024-04-07T16:57:12.765 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-YmDp_POJWqN6EGH off=17423655 len=461579 2024-04-07T16:57:12.772 INFO:tasks.workunit.client.0.smithi100.stdout:op 5611 completed, throughput=5MB/sec 2024-04-07T16:57:12.772 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-eQqE5Ax7HoD0_DA off=12740438 len=1476730 2024-04-07T16:57:12.776 INFO:tasks.workunit.client.0.smithi100.stdout:op 5612 completed, throughput=5MB/sec 2024-04-07T16:57:12.776 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-7PQ_b6OKZKX1VDt off=8660310 len=1365082 2024-04-07T16:57:12.794 INFO:tasks.workunit.client.0.smithi100.stdout:op 5613 completed, throughput=5MB/sec 2024-04-07T16:57:12.795 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-a4Af3bXqdWh2MeA off=4843877 len=1802105 2024-04-07T16:57:12.808 INFO:tasks.workunit.client.0.smithi100.stdout:op 5614 completed, throughput=5MB/sec 2024-04-07T16:57:13.113 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:57:12 smithi144 bash[66654]: cluster 2024-04-07T16:57:11.639688+0000 mgr.y (mgr.25175) 1255 : cluster [DBG] pgmap v1897: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.3 MiB/s wr, 5 op/s 2024-04-07T16:57:13.199 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:57:12 smithi100 bash[74942]: cluster 2024-04-07T16:57:11.639688+0000 mgr.y (mgr.25175) 1255 : cluster [DBG] pgmap v1897: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.3 MiB/s wr, 5 op/s 2024-04-07T16:57:13.199 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:57:12 smithi100 bash[76202]: cluster 2024-04-07T16:57:11.639688+0000 mgr.y (mgr.25175) 1255 : cluster [DBG] pgmap v1897: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.3 MiB/s wr, 5 op/s 2024-04-07T16:57:13.808 INFO:tasks.workunit.client.0.smithi100.stdout: 1081: throughput=5MB/sec pending data=0 2024-04-07T16:57:13.808 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-65udYmIEiDw2eGU off=86217 len=1080236 2024-04-07T16:57:13.811 INFO:tasks.workunit.client.0.smithi100.stdout:op 5615 completed, throughput=5MB/sec 2024-04-07T16:57:13.811 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-SMXL1GXCaRNL0YG off=15656236 len=507950 2024-04-07T16:57:13.818 INFO:tasks.workunit.client.0.smithi100.stdout:op 5616 completed, throughput=5MB/sec 2024-04-07T16:57:13.818 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-NJ5WKTEPnZmoIhd off=627965 len=1509968 2024-04-07T16:57:13.823 INFO:tasks.workunit.client.0.smithi100.stdout:op 5617 completed, throughput=5MB/sec 2024-04-07T16:57:13.823 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-EO70cbFPX8qY9zV off=1475569 len=907551 2024-04-07T16:57:13.831 INFO:tasks.workunit.client.0.smithi100.stdout:op 5618 completed, throughput=5MB/sec 2024-04-07T16:57:13.831 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-iGf2gBQicUZSqd1 off=527003 len=226632 2024-04-07T16:57:13.837 INFO:tasks.workunit.client.0.smithi100.stdout:op 5619 completed, throughput=5MB/sec 2024-04-07T16:57:13.837 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-eWWCbw6yi8oxYDF off=716436 len=632062 2024-04-07T16:57:13.840 INFO:tasks.workunit.client.0.smithi100.stdout:op 5620 completed, throughput=5MB/sec 2024-04-07T16:57:13.840 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-l1iMQa3RofHWc3f off=2234793 len=509183 2024-04-07T16:57:13.841 INFO:tasks.workunit.client.0.smithi100.stdout:op 5621 completed, throughput=5MB/sec 2024-04-07T16:57:14.842 INFO:tasks.workunit.client.0.smithi100.stdout: 1082: throughput=5MB/sec pending data=0 2024-04-07T16:57:14.842 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-vGRtw0YYAB1hyq- off=3226831 len=203693 2024-04-07T16:57:14.843 INFO:tasks.workunit.client.0.smithi100.stdout:op 5622 completed, throughput=5MB/sec 2024-04-07T16:57:14.843 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-yBDxyJyiKsd0pRO off=21616487 len=730716 2024-04-07T16:57:14.845 INFO:tasks.workunit.client.0.smithi100.stdout:op 5623 completed, throughput=5MB/sec 2024-04-07T16:57:14.845 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-4fwJBnrhWEf__C0 off=363029 len=644269 2024-04-07T16:57:14.847 INFO:tasks.workunit.client.0.smithi100.stdout:op 5624 completed, throughput=5MB/sec 2024-04-07T16:57:14.848 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-m959Zc9BONTRoiZ off=1350878 len=403129 2024-04-07T16:57:14.850 INFO:tasks.workunit.client.0.smithi100.stdout:op 5625 completed, throughput=5MB/sec 2024-04-07T16:57:14.850 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-9f3oxVXynlY7ip_ off=1275795 len=956410 2024-04-07T16:57:14.853 INFO:tasks.workunit.client.0.smithi100.stdout:op 5626 completed, throughput=5MB/sec 2024-04-07T16:57:14.853 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-x9dTlA7LJpOxk02 off=6035108 len=2055166 2024-04-07T16:57:14.870 INFO:tasks.workunit.client.0.smithi100.stdout:op 5627 completed, throughput=5MB/sec 2024-04-07T16:57:14.870 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-6wUTfwVhWGF6qd4 off=7057499 len=1331861 2024-04-07T16:57:14.882 INFO:tasks.workunit.client.0.smithi100.stdout:op 5628 completed, throughput=5MB/sec 2024-04-07T16:57:15.113 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:57:14 smithi144 bash[66654]: cluster 2024-04-07T16:57:13.641137+0000 mgr.y (mgr.25175) 1256 : cluster [DBG] pgmap v1898: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 2.9 MiB/s rd, 2.5 MiB/s wr, 6 op/s 2024-04-07T16:57:15.199 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:57:14 smithi100 bash[74942]: cluster 2024-04-07T16:57:13.641137+0000 mgr.y (mgr.25175) 1256 : cluster [DBG] pgmap v1898: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 2.9 MiB/s rd, 2.5 MiB/s wr, 6 op/s 2024-04-07T16:57:15.199 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:57:14 smithi100 bash[76202]: cluster 2024-04-07T16:57:13.641137+0000 mgr.y (mgr.25175) 1256 : cluster [DBG] pgmap v1898: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 2.9 MiB/s rd, 2.5 MiB/s wr, 6 op/s 2024-04-07T16:57:15.882 INFO:tasks.workunit.client.0.smithi100.stdout: 1083: throughput=5MB/sec pending data=0 2024-04-07T16:57:15.883 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-pHcdh8ie2A32GLc off=11489469 len=439597 2024-04-07T16:57:15.889 INFO:tasks.workunit.client.0.smithi100.stdout:op 5629 completed, throughput=5MB/sec 2024-04-07T16:57:15.889 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-BAozPKjmp0MMM1u off=0 len=1160190 2024-04-07T16:57:15.891 INFO:tasks.workunit.client.0.smithi100.stdout:op 5630 completed, throughput=5MB/sec 2024-04-07T16:57:15.892 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-MfnsMNoeL1a_Tua off=178597 len=2397 2024-04-07T16:57:15.893 INFO:tasks.workunit.client.0.smithi100.stdout:op 5631 completed, throughput=5MB/sec 2024-04-07T16:57:15.893 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-UdVemXc9OAeYXnx off=8540569 len=1808905 2024-04-07T16:57:15.898 INFO:tasks.workunit.client.0.smithi100.stdout:op 5632 completed, throughput=5MB/sec 2024-04-07T16:57:15.898 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-aeot9X3OnTa69dD off=2155079 len=1887190 2024-04-07T16:57:15.903 INFO:tasks.workunit.client.0.smithi100.stdout:op 5633 completed, throughput=5MB/sec 2024-04-07T16:57:16.903 INFO:tasks.workunit.client.0.smithi100.stdout: 1084: throughput=5MB/sec pending data=0 2024-04-07T16:57:16.904 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Y5j9XibrWS59grP off=3145115 len=552815 2024-04-07T16:57:16.906 INFO:tasks.workunit.client.0.smithi100.stdout:op 5634 completed, throughput=5MB/sec 2024-04-07T16:57:16.906 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-qUkV4KmkdewP5vy off=8269463 len=1236135 2024-04-07T16:57:16.909 INFO:tasks.workunit.client.0.smithi100.stdout:op 5635 completed, throughput=5MB/sec 2024-04-07T16:57:16.909 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-dleZWmZ7JnI21Ob off=10668109 len=2013694 2024-04-07T16:57:16.914 INFO:tasks.workunit.client.0.smithi100.stdout:op 5636 completed, throughput=5MB/sec 2024-04-07T16:57:16.914 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-lsuKMG0OHxhm84Y off=16563403 len=583715 2024-04-07T16:57:16.916 INFO:tasks.workunit.client.0.smithi100.stdout:op 5637 completed, throughput=5MB/sec 2024-04-07T16:57:17.113 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:57:16 smithi144 bash[66654]: cluster 2024-04-07T16:57:15.642258+0000 mgr.y (mgr.25175) 1257 : cluster [DBG] pgmap v1899: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 2.8 MiB/s rd, 2.2 MiB/s wr, 5 op/s 2024-04-07T16:57:17.199 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:57:16 smithi100 bash[74942]: cluster 2024-04-07T16:57:15.642258+0000 mgr.y (mgr.25175) 1257 : cluster [DBG] pgmap v1899: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 2.8 MiB/s rd, 2.2 MiB/s wr, 5 op/s 2024-04-07T16:57:17.200 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:57:16 smithi100 bash[76202]: cluster 2024-04-07T16:57:15.642258+0000 mgr.y (mgr.25175) 1257 : cluster [DBG] pgmap v1899: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 2.8 MiB/s rd, 2.2 MiB/s wr, 5 op/s 2024-04-07T16:57:17.917 INFO:tasks.workunit.client.0.smithi100.stdout: 1085: throughput=5MB/sec pending data=0 2024-04-07T16:57:17.917 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-XUaTolfBnp6mozU off=15435537 len=1996588 2024-04-07T16:57:17.932 INFO:tasks.workunit.client.0.smithi100.stdout:op 5638 completed, throughput=5MB/sec 2024-04-07T16:57:17.932 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-2Lxh5kNOZTQGe-y off=43414 len=807592 2024-04-07T16:57:17.936 INFO:tasks.workunit.client.0.smithi100.stdout:op 5639 completed, throughput=5MB/sec 2024-04-07T16:57:17.936 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-uU2cM90YKdJGz1c off=2881099 len=1143587 2024-04-07T16:57:17.946 INFO:tasks.workunit.client.0.smithi100.stdout:op 5640 completed, throughput=5MB/sec 2024-04-07T16:57:17.946 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-yOmWzlNeKCLiYmc off=5968611 len=58966 2024-04-07T16:57:17.949 INFO:tasks.workunit.client.0.smithi100.stdout:op 5641 completed, throughput=5MB/sec 2024-04-07T16:57:17.949 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-IBYF1Xb5EfshKcV off=5682284 len=1785798 2024-04-07T16:57:17.960 INFO:tasks.workunit.client.0.smithi100.stdout:op 5642 completed, throughput=5MB/sec 2024-04-07T16:57:18.113 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:57:17 smithi144 bash[66654]: audit 2024-04-07T16:57:16.788642+0000 mon.a (mon.0) 2207 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:57:18.199 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:57:17 smithi100 bash[74942]: audit 2024-04-07T16:57:16.788642+0000 mon.a (mon.0) 2207 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:57:18.200 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:57:17 smithi100 bash[76202]: audit 2024-04-07T16:57:16.788642+0000 mon.a (mon.0) 2207 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:57:18.699 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:57:18 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:57:18] "GET /metrics HTTP/1.1" 200 34073 "" "Prometheus/2.43.0" 2024-04-07T16:57:18.961 INFO:tasks.workunit.client.0.smithi100.stdout: 1086: throughput=5MB/sec pending data=0 2024-04-07T16:57:18.961 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-d3Wn-gt7SYaTIy0 off=334076 len=1876140 2024-04-07T16:57:18.976 INFO:tasks.workunit.client.0.smithi100.stdout:op 5643 completed, throughput=5MB/sec 2024-04-07T16:57:18.976 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj--lOM1sjQ-TtGlZN off=19439217 len=583976 2024-04-07T16:57:18.984 INFO:tasks.workunit.client.0.smithi100.stdout:op 5644 completed, throughput=5MB/sec 2024-04-07T16:57:18.984 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-h8Nwm15qzRQstzj off=4505938 len=1454720 2024-04-07T16:57:18.988 INFO:tasks.workunit.client.0.smithi100.stdout:op 5645 completed, throughput=5MB/sec 2024-04-07T16:57:18.988 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-kkB6552SVwj8VHq off=1683061 len=816337 2024-04-07T16:57:18.996 INFO:tasks.workunit.client.0.smithi100.stdout:op 5646 completed, throughput=5MB/sec 2024-04-07T16:57:18.996 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-ZM1aQn0kd1JkIN2 off=1246474 len=1941661 2024-04-07T16:57:19.010 INFO:tasks.workunit.client.0.smithi100.stdout:op 5647 completed, throughput=5MB/sec 2024-04-07T16:57:19.113 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:57:18 smithi144 bash[66654]: cluster 2024-04-07T16:57:17.643349+0000 mgr.y (mgr.25175) 1258 : cluster [DBG] pgmap v1900: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 3.1 MiB/s rd, 2.3 MiB/s wr, 5 op/s 2024-04-07T16:57:19.199 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:57:18 smithi100 bash[74942]: cluster 2024-04-07T16:57:17.643349+0000 mgr.y (mgr.25175) 1258 : cluster [DBG] pgmap v1900: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 3.1 MiB/s rd, 2.3 MiB/s wr, 5 op/s 2024-04-07T16:57:19.199 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:57:18 smithi100 bash[76202]: cluster 2024-04-07T16:57:17.643349+0000 mgr.y (mgr.25175) 1258 : cluster [DBG] pgmap v1900: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 3.1 MiB/s rd, 2.3 MiB/s wr, 5 op/s 2024-04-07T16:57:20.014 INFO:tasks.workunit.client.0.smithi100.stdout: 1087: throughput=5MB/sec pending data=0 2024-04-07T16:57:20.014 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-OkzIHBRTORx5sxp off=7105060 len=1673014 2024-04-07T16:57:20.015 INFO:tasks.workunit.client.0.smithi100.stdout:op 5648 completed, throughput=5MB/sec 2024-04-07T16:57:20.015 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-90S8IkH0O5vfJP_ off=292954 len=392832 2024-04-07T16:57:20.021 INFO:tasks.workunit.client.0.smithi100.stdout:op 5649 completed, throughput=5MB/sec 2024-04-07T16:57:20.021 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-dhxeUnb-0G8CxuV off=19395445 len=298472 2024-04-07T16:57:20.027 INFO:tasks.workunit.client.0.smithi100.stdout:op 5650 completed, throughput=5MB/sec 2024-04-07T16:57:20.027 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-n89XCFN8SxERoCN off=557689 len=876606 2024-04-07T16:57:20.029 INFO:tasks.workunit.client.0.smithi100.stdout:op 5651 completed, throughput=5MB/sec 2024-04-07T16:57:20.029 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-iJuG28AwDmS5dlc off=2338344 len=234247 2024-04-07T16:57:20.030 INFO:tasks.workunit.client.0.smithi100.stdout:op 5652 completed, throughput=5MB/sec 2024-04-07T16:57:20.030 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-DrTVz0NUmIQ0djz off=5648747 len=1618208 2024-04-07T16:57:20.044 INFO:tasks.workunit.client.0.smithi100.stdout:op 5653 completed, throughput=5MB/sec 2024-04-07T16:57:20.113 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:57:19 smithi144 bash[66654]: cluster 2024-04-07T16:57:19.644270+0000 mgr.y (mgr.25175) 1259 : cluster [DBG] pgmap v1901: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 2.9 MiB/s rd, 2.1 MiB/s wr, 5 op/s 2024-04-07T16:57:20.199 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:57:19 smithi100 bash[74942]: cluster 2024-04-07T16:57:19.644270+0000 mgr.y (mgr.25175) 1259 : cluster [DBG] pgmap v1901: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 2.9 MiB/s rd, 2.1 MiB/s wr, 5 op/s 2024-04-07T16:57:20.200 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:57:19 smithi100 bash[76202]: cluster 2024-04-07T16:57:19.644270+0000 mgr.y (mgr.25175) 1259 : cluster [DBG] pgmap v1901: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 2.9 MiB/s rd, 2.1 MiB/s wr, 5 op/s 2024-04-07T16:57:21.045 INFO:tasks.workunit.client.0.smithi100.stdout: 1088: throughput=5MB/sec pending data=0 2024-04-07T16:57:21.045 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-3b74EAvMgaJsWZd off=4832155 len=736787 2024-04-07T16:57:21.053 INFO:tasks.workunit.client.0.smithi100.stdout:op 5654 completed, throughput=5MB/sec 2024-04-07T16:57:21.053 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-NGJp7cLHD1mSew2 off=11315707 len=257435 2024-04-07T16:57:21.055 INFO:tasks.workunit.client.0.smithi100.stdout:op 5655 completed, throughput=5MB/sec 2024-04-07T16:57:21.055 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-7tb9vGYDPaJ-vdB off=2858191 len=462847 2024-04-07T16:57:21.056 INFO:tasks.workunit.client.0.smithi100.stdout:op 5656 completed, throughput=5MB/sec 2024-04-07T16:57:21.056 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-LBX1BGrPXNQmZ43 off=933791 len=1115926 2024-04-07T16:57:21.060 INFO:tasks.workunit.client.0.smithi100.stdout:op 5657 completed, throughput=5MB/sec 2024-04-07T16:57:21.060 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-9LFgGUURE0ah9Er off=666236 len=1555949 2024-04-07T16:57:21.063 INFO:tasks.workunit.client.0.smithi100.stdout:op 5658 completed, throughput=5MB/sec 2024-04-07T16:57:22.063 INFO:tasks.workunit.client.0.smithi100.stdout: 1089: throughput=5MB/sec pending data=0 2024-04-07T16:57:22.063 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-EDZxK7UM30GUFyy off=1417356 len=1942350 2024-04-07T16:57:22.069 INFO:tasks.workunit.client.0.smithi100.stdout:op 5659 completed, throughput=5MB/sec 2024-04-07T16:57:22.069 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-JPrtg6-XZsUQ4Al off=4188617 len=712709 2024-04-07T16:57:22.072 INFO:tasks.workunit.client.0.smithi100.stdout:op 5660 completed, throughput=5MB/sec 2024-04-07T16:57:22.072 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-fpCIPojG9B58Wnk off=17910818 len=610930 2024-04-07T16:57:22.073 INFO:tasks.workunit.client.0.smithi100.stdout:op 5661 completed, throughput=5MB/sec 2024-04-07T16:57:22.073 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-9A3WWgoX9JWPmdN off=2158817 len=99781 2024-04-07T16:57:22.074 INFO:tasks.workunit.client.0.smithi100.stdout:op 5662 completed, throughput=5MB/sec 2024-04-07T16:57:22.074 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-z1n5lzJQORPVoHO off=7067262 len=1966284 2024-04-07T16:57:22.080 INFO:tasks.workunit.client.0.smithi100.stdout:op 5663 completed, throughput=5MB/sec 2024-04-07T16:57:22.613 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:57:22 smithi144 bash[95233]: ts=2024-04-07T16:57:22.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:57:22.949 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:57:22 smithi100 bash[74942]: cluster 2024-04-07T16:57:21.645050+0000 mgr.y (mgr.25175) 1260 : cluster [DBG] pgmap v1902: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.2 MiB/s wr, 5 op/s 2024-04-07T16:57:22.950 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:57:22 smithi100 bash[76202]: cluster 2024-04-07T16:57:21.645050+0000 mgr.y (mgr.25175) 1260 : cluster [DBG] pgmap v1902: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.2 MiB/s wr, 5 op/s 2024-04-07T16:57:23.080 INFO:tasks.workunit.client.0.smithi100.stdout: 1090: throughput=5MB/sec pending data=0 2024-04-07T16:57:23.080 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-xusAYN4eZERAKfc off=8009346 len=563625 2024-04-07T16:57:23.082 INFO:tasks.workunit.client.0.smithi100.stdout:op 5664 completed, throughput=5MB/sec 2024-04-07T16:57:23.082 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-84IVXImCLwPDmHn off=1642738 len=706613 2024-04-07T16:57:23.085 INFO:tasks.workunit.client.0.smithi100.stdout:op 5665 completed, throughput=5MB/sec 2024-04-07T16:57:23.085 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-o4VAeRYyhE-PxYO off=6356917 len=375068 2024-04-07T16:57:23.091 INFO:tasks.workunit.client.0.smithi100.stdout:op 5666 completed, throughput=5MB/sec 2024-04-07T16:57:23.091 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-ljxgqs27Q-eWjXp off=10003816 len=984251 2024-04-07T16:57:23.095 INFO:tasks.workunit.client.0.smithi100.stdout:op 5667 completed, throughput=5MB/sec 2024-04-07T16:57:23.095 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-VR6lggY4CqnidXo off=3165059 len=280468 2024-04-07T16:57:23.097 INFO:tasks.workunit.client.0.smithi100.stdout:op 5668 completed, throughput=5MB/sec 2024-04-07T16:57:23.097 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-hbKM3oZTAq3838Z off=1361426 len=76414 2024-04-07T16:57:23.098 INFO:tasks.workunit.client.0.smithi100.stdout:op 5669 completed, throughput=5MB/sec 2024-04-07T16:57:23.098 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-ib3NAmiolnTnZhv off=11948027 len=605275 2024-04-07T16:57:23.100 INFO:tasks.workunit.client.0.smithi100.stdout:op 5670 completed, throughput=5MB/sec 2024-04-07T16:57:23.100 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-PT1ZojGLuSLD8dK off=255919 len=1063724 2024-04-07T16:57:23.110 INFO:tasks.workunit.client.0.smithi100.stdout:op 5671 completed, throughput=5MB/sec 2024-04-07T16:57:23.110 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-CauLcA6J8-Nm1A_ off=3187957 len=2000201 2024-04-07T16:57:23.113 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:57:22 smithi144 bash[66654]: cluster 2024-04-07T16:57:21.645050+0000 mgr.y (mgr.25175) 1260 : cluster [DBG] pgmap v1902: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.2 MiB/s wr, 5 op/s 2024-04-07T16:57:23.128 INFO:tasks.workunit.client.0.smithi100.stdout:op 5672 completed, throughput=5MB/sec 2024-04-07T16:57:24.129 INFO:tasks.workunit.client.0.smithi100.stdout: 1091: throughput=5MB/sec pending data=0 2024-04-07T16:57:24.129 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-AoTeM-qIEtnpl6Y off=9061132 len=428812 2024-04-07T16:57:24.130 INFO:tasks.workunit.client.0.smithi100.stdout:op 5673 completed, throughput=5MB/sec 2024-04-07T16:57:24.131 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-EXgUoiPgm8845IK off=6362784 len=1794758 2024-04-07T16:57:24.135 INFO:tasks.workunit.client.0.smithi100.stdout:op 5674 completed, throughput=5MB/sec 2024-04-07T16:57:24.135 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-lkbSnX-f0fUasvh off=293194 len=1115298 2024-04-07T16:57:24.138 INFO:tasks.workunit.client.0.smithi100.stdout:op 5675 completed, throughput=5MB/sec 2024-04-07T16:57:24.138 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Tf6oFScemyo1Igo off=0 len=1425164 2024-04-07T16:57:24.150 INFO:tasks.workunit.client.0.smithi100.stdout:op 5676 completed, throughput=5MB/sec 2024-04-07T16:57:25.113 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:57:24 smithi144 bash[66654]: cluster 2024-04-07T16:57:23.646262+0000 mgr.y (mgr.25175) 1261 : cluster [DBG] pgmap v1903: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.5 MiB/s wr, 5 op/s 2024-04-07T16:57:25.150 INFO:tasks.workunit.client.0.smithi100.stdout: 1092: throughput=5MB/sec pending data=0 2024-04-07T16:57:25.151 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-i_dCf8F--7TCRyw off=13345379 len=1108789 2024-04-07T16:57:25.160 INFO:tasks.workunit.client.0.smithi100.stdout:op 5677 completed, throughput=5MB/sec 2024-04-07T16:57:25.160 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-95CQYUYAa0_en2r off=2537793 len=1286636 2024-04-07T16:57:25.163 INFO:tasks.workunit.client.0.smithi100.stdout:op 5678 completed, throughput=5MB/sec 2024-04-07T16:57:25.163 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-BwjVUrkPdKRATYy off=9503320 len=1245793 2024-04-07T16:57:25.168 INFO:tasks.workunit.client.0.smithi100.stdout:op 5679 completed, throughput=5MB/sec 2024-04-07T16:57:25.168 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-mD1I9y1zeeoKrL4 off=632808 len=861617 2024-04-07T16:57:25.176 INFO:tasks.workunit.client.0.smithi100.stdout:op 5680 completed, throughput=5MB/sec 2024-04-07T16:57:25.176 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-xcbx8TuQVZHfpa7 off=7034049 len=1241127 2024-04-07T16:57:25.186 INFO:tasks.workunit.client.0.smithi100.stdout:op 5681 completed, throughput=5MB/sec 2024-04-07T16:57:25.199 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:57:24 smithi100 bash[74942]: cluster 2024-04-07T16:57:23.646262+0000 mgr.y (mgr.25175) 1261 : cluster [DBG] pgmap v1903: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.5 MiB/s wr, 5 op/s 2024-04-07T16:57:25.199 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:57:24 smithi100 bash[76202]: cluster 2024-04-07T16:57:23.646262+0000 mgr.y (mgr.25175) 1261 : cluster [DBG] pgmap v1903: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.5 MiB/s wr, 5 op/s 2024-04-07T16:57:26.187 INFO:tasks.workunit.client.0.smithi100.stdout: 1093: throughput=5MB/sec pending data=0 2024-04-07T16:57:26.187 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-HyFKIOf8lJ13U1b off=4464037 len=1146668 2024-04-07T16:57:26.190 INFO:tasks.workunit.client.0.smithi100.stdout:op 5682 completed, throughput=5MB/sec 2024-04-07T16:57:26.190 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-vYxtV4_Jv-ga83B off=807916 len=1984402 2024-04-07T16:57:26.207 INFO:tasks.workunit.client.0.smithi100.stdout:op 5683 completed, throughput=5MB/sec 2024-04-07T16:57:26.207 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-OpvM2BCPP8ISFmM off=6431110 len=223226 2024-04-07T16:57:26.208 INFO:tasks.workunit.client.0.smithi100.stdout:op 5684 completed, throughput=5MB/sec 2024-04-07T16:57:26.208 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-2MsD-e0gJX5yWhL off=2341149 len=1394577 2024-04-07T16:57:26.220 INFO:tasks.workunit.client.0.smithi100.stdout:op 5685 completed, throughput=5MB/sec 2024-04-07T16:57:27.113 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:57:26 smithi144 bash[66654]: cluster 2024-04-07T16:57:25.647233+0000 mgr.y (mgr.25175) 1262 : cluster [DBG] pgmap v1904: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 2.4 MiB/s rd, 2.0 MiB/s wr, 4 op/s 2024-04-07T16:57:27.199 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:57:26 smithi100 bash[74942]: cluster 2024-04-07T16:57:25.647233+0000 mgr.y (mgr.25175) 1262 : cluster [DBG] pgmap v1904: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 2.4 MiB/s rd, 2.0 MiB/s wr, 4 op/s 2024-04-07T16:57:27.200 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:57:26 smithi100 bash[76202]: cluster 2024-04-07T16:57:25.647233+0000 mgr.y (mgr.25175) 1262 : cluster [DBG] pgmap v1904: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 2.4 MiB/s rd, 2.0 MiB/s wr, 4 op/s 2024-04-07T16:57:27.220 INFO:tasks.workunit.client.0.smithi100.stdout: 1094: throughput=5MB/sec pending data=0 2024-04-07T16:57:27.220 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-W0EsZS6XYsy6NhQ off=4112735 len=1188319 2024-04-07T16:57:27.232 INFO:tasks.workunit.client.0.smithi100.stdout:op 5686 completed, throughput=5MB/sec 2024-04-07T16:57:27.232 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-h-uC9HoKaQEGiV3 off=3462985 len=742578 2024-04-07T16:57:27.234 INFO:tasks.workunit.client.0.smithi100.stdout:op 5687 completed, throughput=5MB/sec 2024-04-07T16:57:27.234 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-PLwOmLg5bnKng3W off=11790472 len=236443 2024-04-07T16:57:27.235 INFO:tasks.workunit.client.0.smithi100.stdout:op 5688 completed, throughput=5MB/sec 2024-04-07T16:57:27.235 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-hez77Mw9NOBglMI off=1883409 len=367888 2024-04-07T16:57:27.237 INFO:tasks.workunit.client.0.smithi100.stdout:op 5689 completed, throughput=5MB/sec 2024-04-07T16:57:27.237 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Snw4qucB8nXx2gC off=1922561 len=390387 2024-04-07T16:57:27.239 INFO:tasks.workunit.client.0.smithi100.stdout:op 5690 completed, throughput=5MB/sec 2024-04-07T16:57:27.239 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-gb4zYBwN6Y2HO_D off=5354180 len=293942 2024-04-07T16:57:27.240 INFO:tasks.workunit.client.0.smithi100.stdout:op 5691 completed, throughput=5MB/sec 2024-04-07T16:57:27.240 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-1vqZCh14UINKKhN off=23665305 len=1762644 2024-04-07T16:57:27.254 INFO:tasks.workunit.client.0.smithi100.stdout:op 5692 completed, throughput=5MB/sec 2024-04-07T16:57:27.254 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-KNDoqkJuVuIlpOY off=1981142 len=1253064 2024-04-07T16:57:27.264 INFO:tasks.workunit.client.0.smithi100.stdout:op 5693 completed, throughput=5MB/sec 2024-04-07T16:57:28.265 INFO:tasks.workunit.client.0.smithi100.stdout: 1095: throughput=5MB/sec pending data=0 2024-04-07T16:57:28.265 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-HRl-8MkWSJU7O7e off=12911501 len=861082 2024-04-07T16:57:28.273 INFO:tasks.workunit.client.0.smithi100.stdout:op 5694 completed, throughput=5MB/sec 2024-04-07T16:57:28.273 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-rJ0CK5vEscHhWMR off=1025722 len=747849 2024-04-07T16:57:28.275 INFO:tasks.workunit.client.0.smithi100.stdout:op 5695 completed, throughput=5MB/sec 2024-04-07T16:57:28.276 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Vjh739l2RWQG5nt off=3142918 len=1190514 2024-04-07T16:57:28.287 INFO:tasks.workunit.client.0.smithi100.stdout:op 5696 completed, throughput=5MB/sec 2024-04-07T16:57:28.288 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-0PuC4ueRhlAHi-D off=568561 len=1970386 2024-04-07T16:57:28.292 INFO:tasks.workunit.client.0.smithi100.stdout:op 5697 completed, throughput=5MB/sec 2024-04-07T16:57:28.699 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:57:28 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:57:28] "GET /metrics HTTP/1.1" 200 34073 "" "Prometheus/2.43.0" 2024-04-07T16:57:29.113 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:57:28 smithi144 bash[66654]: cluster 2024-04-07T16:57:27.648277+0000 mgr.y (mgr.25175) 1263 : cluster [DBG] pgmap v1905: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 2.9 MiB/s rd, 2.0 MiB/s wr, 4 op/s 2024-04-07T16:57:29.199 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:57:28 smithi100 bash[74942]: cluster 2024-04-07T16:57:27.648277+0000 mgr.y (mgr.25175) 1263 : cluster [DBG] pgmap v1905: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 2.9 MiB/s rd, 2.0 MiB/s wr, 4 op/s 2024-04-07T16:57:29.199 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:57:28 smithi100 bash[76202]: cluster 2024-04-07T16:57:27.648277+0000 mgr.y (mgr.25175) 1263 : cluster [DBG] pgmap v1905: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 2.9 MiB/s rd, 2.0 MiB/s wr, 4 op/s 2024-04-07T16:57:29.292 INFO:tasks.workunit.client.0.smithi100.stdout: 1096: throughput=5MB/sec pending data=0 2024-04-07T16:57:29.292 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-yGayoxRl9I1iHxx off=7551517 len=134777 2024-04-07T16:57:29.298 INFO:tasks.workunit.client.0.smithi100.stdout:op 5698 completed, throughput=5MB/sec 2024-04-07T16:57:29.298 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-b7EGHmx1m51g9Lx off=20447729 len=754680 2024-04-07T16:57:29.301 INFO:tasks.workunit.client.0.smithi100.stdout:op 5699 completed, throughput=5MB/sec 2024-04-07T16:57:29.301 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-10PHW3AsiBOT00g off=388631 len=1098479 2024-04-07T16:57:29.311 INFO:tasks.workunit.client.0.smithi100.stdout:op 5700 completed, throughput=5MB/sec 2024-04-07T16:57:29.311 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-xOtbMp2Lm9AHicO off=2343026 len=609392 2024-04-07T16:57:29.319 INFO:tasks.workunit.client.0.smithi100.stdout:op 5701 completed, throughput=5MB/sec 2024-04-07T16:57:29.319 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-tEhVnmgXBpARhu_ off=2787896 len=641904 2024-04-07T16:57:29.327 INFO:tasks.workunit.client.0.smithi100.stdout:op 5702 completed, throughput=5MB/sec 2024-04-07T16:57:29.328 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-IJ3DBB4p8b7XCLB off=17424370 len=1881788 2024-04-07T16:57:29.343 INFO:tasks.workunit.client.0.smithi100.stdout:op 5703 completed, throughput=5MB/sec 2024-04-07T16:57:30.343 INFO:tasks.workunit.client.0.smithi100.stdout: 1097: throughput=5MB/sec pending data=0 2024-04-07T16:57:30.344 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-i4Nllu54OsU6NCw off=425387 len=535083 2024-04-07T16:57:30.346 INFO:tasks.workunit.client.0.smithi100.stdout:op 5704 completed, throughput=5MB/sec 2024-04-07T16:57:30.346 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-BxecSydJy0ZoSXS off=3630521 len=320108 2024-04-07T16:57:30.352 INFO:tasks.workunit.client.0.smithi100.stdout:op 5705 completed, throughput=5MB/sec 2024-04-07T16:57:30.352 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-A9Lr5GYzC8J2OSt off=2427000 len=349792 2024-04-07T16:57:30.358 INFO:tasks.workunit.client.0.smithi100.stdout:op 5706 completed, throughput=5MB/sec 2024-04-07T16:57:30.358 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-8fM6dbdj5Inv9Vu off=2770 len=1764643 2024-04-07T16:57:30.371 INFO:tasks.workunit.client.0.smithi100.stdout:op 5707 completed, throughput=5MB/sec 2024-04-07T16:57:30.371 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-OugYPDA8p3T0Dq2 off=171645 len=324465 2024-04-07T16:57:30.377 INFO:tasks.workunit.client.0.smithi100.stdout:op 5708 completed, throughput=5MB/sec 2024-04-07T16:57:30.377 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-ViTcwYCwIggoH14 off=2223300 len=550952 2024-04-07T16:57:30.379 INFO:tasks.workunit.client.0.smithi100.stdout:op 5709 completed, throughput=5MB/sec 2024-04-07T16:57:30.379 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-W49ELPkNirH9zdL off=22840288 len=1965273 2024-04-07T16:57:30.392 INFO:tasks.workunit.client.0.smithi100.stdout:op 5710 completed, throughput=5MB/sec 2024-04-07T16:57:31.113 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:57:30 smithi144 bash[66654]: cluster 2024-04-07T16:57:29.649218+0000 mgr.y (mgr.25175) 1264 : cluster [DBG] pgmap v1906: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 2.2 MiB/s rd, 1.9 MiB/s wr, 4 op/s 2024-04-07T16:57:31.199 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:57:30 smithi100 bash[74942]: cluster 2024-04-07T16:57:29.649218+0000 mgr.y (mgr.25175) 1264 : cluster [DBG] pgmap v1906: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 2.2 MiB/s rd, 1.9 MiB/s wr, 4 op/s 2024-04-07T16:57:31.199 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:57:30 smithi100 bash[76202]: cluster 2024-04-07T16:57:29.649218+0000 mgr.y (mgr.25175) 1264 : cluster [DBG] pgmap v1906: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 2.2 MiB/s rd, 1.9 MiB/s wr, 4 op/s 2024-04-07T16:57:31.392 INFO:tasks.workunit.client.0.smithi100.stdout: 1098: throughput=5MB/sec pending data=0 2024-04-07T16:57:31.392 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-X7jS0AVCtSGyyIk off=9420633 len=138740 2024-04-07T16:57:31.397 INFO:tasks.workunit.client.0.smithi100.stdout:op 5711 completed, throughput=5MB/sec 2024-04-07T16:57:31.398 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-nBEJ1Y8DwiBhYZh off=8380903 len=172800 2024-04-07T16:57:31.399 INFO:tasks.workunit.client.0.smithi100.stdout:op 5712 completed, throughput=5MB/sec 2024-04-07T16:57:31.399 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-1dhQsWSYCNRkDtE off=4459050 len=500581 2024-04-07T16:57:31.405 INFO:tasks.workunit.client.0.smithi100.stdout:op 5713 completed, throughput=5MB/sec 2024-04-07T16:57:31.405 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-vmPa1GJ2bAiaub_ off=10249273 len=2060055 2024-04-07T16:57:31.410 INFO:tasks.workunit.client.0.smithi100.stdout:op 5714 completed, throughput=5MB/sec 2024-04-07T16:57:31.410 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-PxVIEcJQxSDkYg6 off=3531762 len=568441 2024-04-07T16:57:31.417 INFO:tasks.workunit.client.0.smithi100.stdout:op 5715 completed, throughput=5MB/sec 2024-04-07T16:57:31.417 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-zyrA-n9v24q7YuW off=5807025 len=1301877 2024-04-07T16:57:31.428 INFO:tasks.workunit.client.0.smithi100.stdout:op 5716 completed, throughput=5MB/sec 2024-04-07T16:57:31.428 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-1mWGyrAUPZLHVY4 off=637299 len=1700534 2024-04-07T16:57:31.446 INFO:tasks.workunit.client.0.smithi100.stdout:op 5717 completed, throughput=5MB/sec 2024-04-07T16:57:32.443 INFO:tasks.workunit.client.0.smithi100.stdout: 1099: throughput=5MB/sec pending data=0 2024-04-07T16:57:32.443 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-LlUfdkHrrvlggWz off=9585947 len=1776077 2024-04-07T16:57:32.447 INFO:tasks.workunit.client.0.smithi100.stdout:op 5718 completed, throughput=5MB/sec 2024-04-07T16:57:32.447 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-DsfviVlF2-8Kiwc off=4179628 len=536221 2024-04-07T16:57:32.455 INFO:tasks.workunit.client.0.smithi100.stdout:op 5719 completed, throughput=5MB/sec 2024-04-07T16:57:32.455 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-67r8TCJVt5kflQX off=631502 len=1151745 2024-04-07T16:57:32.459 INFO:tasks.workunit.client.0.smithi100.stdout:op 5720 completed, throughput=5MB/sec 2024-04-07T16:57:32.459 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-zwonKCRP5muYCiD off=8857350 len=1170629 2024-04-07T16:57:32.470 INFO:tasks.workunit.client.0.smithi100.stdout:op 5721 completed, throughput=5MB/sec 2024-04-07T16:57:32.613 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:57:32 smithi144 bash[95233]: ts=2024-04-07T16:57:32.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:57:33.113 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:57:32 smithi144 bash[66654]: cluster 2024-04-07T16:57:31.650064+0000 mgr.y (mgr.25175) 1265 : cluster [DBG] pgmap v1907: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 2.4 MiB/s rd, 2.1 MiB/s wr, 4 op/s 2024-04-07T16:57:33.113 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:57:32 smithi144 bash[66654]: audit 2024-04-07T16:57:31.789035+0000 mon.a (mon.0) 2208 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:57:33.199 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:57:32 smithi100 bash[74942]: cluster 2024-04-07T16:57:31.650064+0000 mgr.y (mgr.25175) 1265 : cluster [DBG] pgmap v1907: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 2.4 MiB/s rd, 2.1 MiB/s wr, 4 op/s 2024-04-07T16:57:33.199 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:57:32 smithi100 bash[74942]: audit 2024-04-07T16:57:31.789035+0000 mon.a (mon.0) 2208 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:57:33.200 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:57:32 smithi100 bash[76202]: cluster 2024-04-07T16:57:31.650064+0000 mgr.y (mgr.25175) 1265 : cluster [DBG] pgmap v1907: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 2.4 MiB/s rd, 2.1 MiB/s wr, 4 op/s 2024-04-07T16:57:33.200 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:57:32 smithi100 bash[76202]: audit 2024-04-07T16:57:31.789035+0000 mon.a (mon.0) 2208 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:57:33.470 INFO:tasks.workunit.client.0.smithi100.stdout: 1100: throughput=5MB/sec pending data=0 2024-04-07T16:57:33.470 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-SPMn9HP5w2DRkey off=732191 len=1114198 2024-04-07T16:57:33.480 INFO:tasks.workunit.client.0.smithi100.stdout:op 5722 completed, throughput=5MB/sec 2024-04-07T16:57:33.480 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-x7SGfFquqBblGoK off=1311802 len=1294516 2024-04-07T16:57:33.483 INFO:tasks.workunit.client.0.smithi100.stdout:op 5723 completed, throughput=5MB/sec 2024-04-07T16:57:33.483 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-ZYl_KYUZj-Vv6iV off=14668049 len=1583827 2024-04-07T16:57:33.487 INFO:tasks.workunit.client.0.smithi100.stdout:op 5724 completed, throughput=5MB/sec 2024-04-07T16:57:33.488 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-xb0xt4c4Z4VavrM off=8423069 len=1826727 2024-04-07T16:57:33.494 INFO:tasks.workunit.client.0.smithi100.stdout:op 5725 completed, throughput=5MB/sec 2024-04-07T16:57:34.494 INFO:tasks.workunit.client.0.smithi100.stdout: 1101: throughput=5MB/sec pending data=0 2024-04-07T16:57:34.494 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-jjD72q-9DjGAXnH off=2761108 len=1959581 2024-04-07T16:57:34.499 INFO:tasks.workunit.client.0.smithi100.stdout:op 5726 completed, throughput=5MB/sec 2024-04-07T16:57:34.499 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-sW31kG_e4oNWzo_ off=15441365 len=1841755 2024-04-07T16:57:34.513 INFO:tasks.workunit.client.0.smithi100.stdout:op 5727 completed, throughput=5MB/sec 2024-04-07T16:57:34.513 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Rb4LWfNRjvv-IdH off=4557942 len=1865769 2024-04-07T16:57:34.517 INFO:tasks.workunit.client.0.smithi100.stdout:op 5728 completed, throughput=5MB/sec 2024-04-07T16:57:35.113 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:57:34 smithi144 bash[66654]: cluster 2024-04-07T16:57:33.651151+0000 mgr.y (mgr.25175) 1266 : cluster [DBG] pgmap v1908: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.8 MiB/s wr, 5 op/s 2024-04-07T16:57:35.199 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:57:34 smithi100 bash[74942]: cluster 2024-04-07T16:57:33.651151+0000 mgr.y (mgr.25175) 1266 : cluster [DBG] pgmap v1908: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.8 MiB/s wr, 5 op/s 2024-04-07T16:57:35.200 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:57:34 smithi100 bash[76202]: cluster 2024-04-07T16:57:33.651151+0000 mgr.y (mgr.25175) 1266 : cluster [DBG] pgmap v1908: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.8 MiB/s wr, 5 op/s 2024-04-07T16:57:35.517 INFO:tasks.workunit.client.0.smithi100.stdout: 1102: throughput=5MB/sec pending data=0 2024-04-07T16:57:35.517 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-mjiBldhSMHusy-1 off=11634356 len=745791 2024-04-07T16:57:35.520 INFO:tasks.workunit.client.0.smithi100.stdout:op 5729 completed, throughput=5MB/sec 2024-04-07T16:57:35.520 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-B4-VPoMttLkQly7 off=5569456 len=1959990 2024-04-07T16:57:35.534 INFO:tasks.workunit.client.0.smithi100.stdout:op 5730 completed, throughput=5MB/sec 2024-04-07T16:57:35.534 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-BUHYrygDnPXlBw3 off=3024140 len=1800721 2024-04-07T16:57:35.540 INFO:tasks.workunit.client.0.smithi100.stdout:op 5731 completed, throughput=5MB/sec 2024-04-07T16:57:36.540 INFO:tasks.workunit.client.0.smithi100.stdout: 1103: throughput=5MB/sec pending data=0 2024-04-07T16:57:36.541 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-YhHGXY8G8TS992F off=6217465 len=817618 2024-04-07T16:57:36.544 INFO:tasks.workunit.client.0.smithi100.stdout:op 5732 completed, throughput=5MB/sec 2024-04-07T16:57:36.544 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Ejv8NepDe8nb0cT off=7642276 len=1855284 2024-04-07T16:57:36.559 INFO:tasks.workunit.client.0.smithi100.stdout:op 5733 completed, throughput=5MB/sec 2024-04-07T16:57:36.559 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-JdtAwTLGOAD6zUt off=7753985 len=626618 2024-04-07T16:57:36.561 INFO:tasks.workunit.client.0.smithi100.stdout:op 5734 completed, throughput=5MB/sec 2024-04-07T16:57:36.561 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-peGYTX0L3ukqkxq off=18080501 len=1468508 2024-04-07T16:57:36.575 INFO:tasks.workunit.client.0.smithi100.stdout:op 5735 completed, throughput=5MB/sec 2024-04-07T16:57:36.575 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-ZxNKoNbDsDgux8h off=270083 len=567532 2024-04-07T16:57:36.577 INFO:tasks.workunit.client.0.smithi100.stdout:op 5736 completed, throughput=5MB/sec 2024-04-07T16:57:37.113 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:57:36 smithi144 bash[66654]: cluster 2024-04-07T16:57:35.652153+0000 mgr.y (mgr.25175) 1267 : cluster [DBG] pgmap v1909: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.5 MiB/s wr, 5 op/s 2024-04-07T16:57:37.199 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:57:36 smithi100 bash[74942]: cluster 2024-04-07T16:57:35.652153+0000 mgr.y (mgr.25175) 1267 : cluster [DBG] pgmap v1909: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.5 MiB/s wr, 5 op/s 2024-04-07T16:57:37.199 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:57:36 smithi100 bash[76202]: cluster 2024-04-07T16:57:35.652153+0000 mgr.y (mgr.25175) 1267 : cluster [DBG] pgmap v1909: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.5 MiB/s wr, 5 op/s 2024-04-07T16:57:37.577 INFO:tasks.workunit.client.0.smithi100.stdout: 1104: throughput=5MB/sec pending data=0 2024-04-07T16:57:37.577 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-7EzeAorbgf5ClI0 off=7190408 len=10969 2024-04-07T16:57:37.582 INFO:tasks.workunit.client.0.smithi100.stdout:op 5737 completed, throughput=5MB/sec 2024-04-07T16:57:37.582 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-jXTugkeKi6LPS9c off=12472781 len=1316878 2024-04-07T16:57:37.585 INFO:tasks.workunit.client.0.smithi100.stdout:op 5738 completed, throughput=5MB/sec 2024-04-07T16:57:37.585 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-5Jp8n0HD7gnJrNh off=6514409 len=9420 2024-04-07T16:57:37.586 INFO:tasks.workunit.client.0.smithi100.stdout:op 5739 completed, throughput=5MB/sec 2024-04-07T16:57:37.586 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-wV0Jhc7bIC92DeG off=570825 len=710027 2024-04-07T16:57:37.594 INFO:tasks.workunit.client.0.smithi100.stdout:op 5740 completed, throughput=5MB/sec 2024-04-07T16:57:37.595 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-2uWiJ2oK3B3tll3 off=16395154 len=2060449 2024-04-07T16:57:37.608 INFO:tasks.workunit.client.0.smithi100.stdout:op 5741 completed, throughput=5MB/sec 2024-04-07T16:57:37.609 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-3bTopyq5oHJM-OW off=230008 len=420531 2024-04-07T16:57:37.610 INFO:tasks.workunit.client.0.smithi100.stdout:op 5742 completed, throughput=5MB/sec 2024-04-07T16:57:37.610 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-X3mtOS-KTiwx1xN off=2364754 len=1298527 2024-04-07T16:57:37.614 INFO:tasks.workunit.client.0.smithi100.stdout:op 5743 completed, throughput=5MB/sec 2024-04-07T16:57:38.614 INFO:tasks.workunit.client.0.smithi100.stdout: 1105: throughput=5MB/sec pending data=0 2024-04-07T16:57:38.614 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-0ly9-HjpjubNR4V off=6109662 len=756609 2024-04-07T16:57:38.617 INFO:tasks.workunit.client.0.smithi100.stdout:op 5744 completed, throughput=5MB/sec 2024-04-07T16:57:38.618 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-dwphwGpwRbFwu8u off=490646 len=309111 2024-04-07T16:57:38.623 INFO:tasks.workunit.client.0.smithi100.stdout:op 5745 completed, throughput=5MB/sec 2024-04-07T16:57:38.623 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-J14Fc2ORuvOrFC8 off=5671774 len=179015 2024-04-07T16:57:38.624 INFO:tasks.workunit.client.0.smithi100.stdout:op 5746 completed, throughput=5MB/sec 2024-04-07T16:57:38.624 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-mIriXEK-yHxRBAU off=1217205 len=1870828 2024-04-07T16:57:38.629 INFO:tasks.workunit.client.0.smithi100.stdout:op 5747 completed, throughput=5MB/sec 2024-04-07T16:57:38.629 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj--A1inBDnmqe5ZOO off=5568486 len=1710364 2024-04-07T16:57:38.632 INFO:tasks.workunit.client.0.smithi100.stdout:op 5748 completed, throughput=5MB/sec 2024-04-07T16:57:38.699 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:57:38 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:57:38] "GET /metrics HTTP/1.1" 200 34073 "" "Prometheus/2.43.0" 2024-04-07T16:57:39.113 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:57:38 smithi144 bash[66654]: cluster 2024-04-07T16:57:37.653245+0000 mgr.y (mgr.25175) 1268 : cluster [DBG] pgmap v1910: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.7 MiB/s wr, 5 op/s 2024-04-07T16:57:39.199 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:57:38 smithi100 bash[74942]: cluster 2024-04-07T16:57:37.653245+0000 mgr.y (mgr.25175) 1268 : cluster [DBG] pgmap v1910: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.7 MiB/s wr, 5 op/s 2024-04-07T16:57:39.199 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:57:38 smithi100 bash[76202]: cluster 2024-04-07T16:57:37.653245+0000 mgr.y (mgr.25175) 1268 : cluster [DBG] pgmap v1910: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.7 MiB/s wr, 5 op/s 2024-04-07T16:57:39.632 INFO:tasks.workunit.client.0.smithi100.stdout: 1106: throughput=5MB/sec pending data=0 2024-04-07T16:57:39.633 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-xm-yYdkRU4_2sUp off=6376321 len=577663 2024-04-07T16:57:39.634 INFO:tasks.workunit.client.0.smithi100.stdout:op 5749 completed, throughput=5MB/sec 2024-04-07T16:57:39.634 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-N_6M9x8TRLbuVSK off=1334736 len=1069442 2024-04-07T16:57:39.638 INFO:tasks.workunit.client.0.smithi100.stdout:op 5750 completed, throughput=5MB/sec 2024-04-07T16:57:39.638 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-9YghhgEa66AtiEB off=4474645 len=1066328 2024-04-07T16:57:39.649 INFO:tasks.workunit.client.0.smithi100.stdout:op 5751 completed, throughput=5MB/sec 2024-04-07T16:57:39.649 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-nnnTnKdJFjN4hHG off=4315486 len=941159 2024-04-07T16:57:39.652 INFO:tasks.workunit.client.0.smithi100.stdout:op 5752 completed, throughput=5MB/sec 2024-04-07T16:57:39.652 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Lo3r1am1dSb7PhP off=3893594 len=1903683 2024-04-07T16:57:39.667 INFO:tasks.workunit.client.0.smithi100.stdout:op 5753 completed, throughput=5MB/sec 2024-04-07T16:57:40.667 INFO:tasks.workunit.client.0.smithi100.stdout: 1107: throughput=5MB/sec pending data=0 2024-04-07T16:57:40.668 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-I743VQOyPLBbEWz off=2756139 len=1710257 2024-04-07T16:57:40.681 INFO:tasks.workunit.client.0.smithi100.stdout:op 5754 completed, throughput=5MB/sec 2024-04-07T16:57:40.681 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-bAy4p-K4j89oKxx off=10573199 len=119528 2024-04-07T16:57:40.682 INFO:tasks.workunit.client.0.smithi100.stdout:op 5755 completed, throughput=5MB/sec 2024-04-07T16:57:40.682 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-pSLcym5QDgMvEkK off=1142791 len=1089391 2024-04-07T16:57:40.685 INFO:tasks.workunit.client.0.smithi100.stdout:op 5756 completed, throughput=5MB/sec 2024-04-07T16:57:40.685 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-1rS-G9bEsEfdwaK off=7354658 len=1675382 2024-04-07T16:57:40.691 INFO:tasks.workunit.client.0.smithi100.stdout:op 5757 completed, throughput=5MB/sec 2024-04-07T16:57:40.691 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-bwHfZBspmsxAIEU off=6695827 len=1629000 2024-04-07T16:57:40.700 INFO:tasks.workunit.client.0.smithi100.stdout:op 5758 completed, throughput=5MB/sec 2024-04-07T16:57:41.113 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:57:40 smithi144 bash[66654]: cluster 2024-04-07T16:57:39.654265+0000 mgr.y (mgr.25175) 1269 : cluster [DBG] pgmap v1911: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 2.3 MiB/s rd, 2.9 MiB/s wr, 5 op/s 2024-04-07T16:57:41.199 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:57:40 smithi100 bash[74942]: cluster 2024-04-07T16:57:39.654265+0000 mgr.y (mgr.25175) 1269 : cluster [DBG] pgmap v1911: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 2.3 MiB/s rd, 2.9 MiB/s wr, 5 op/s 2024-04-07T16:57:41.200 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:57:40 smithi100 bash[76202]: cluster 2024-04-07T16:57:39.654265+0000 mgr.y (mgr.25175) 1269 : cluster [DBG] pgmap v1911: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 2.3 MiB/s rd, 2.9 MiB/s wr, 5 op/s 2024-04-07T16:57:41.700 INFO:tasks.workunit.client.0.smithi100.stdout: 1108: throughput=5MB/sec pending data=0 2024-04-07T16:57:41.701 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-ilT40qBOZDLJTvL off=1294771 len=1962801 2024-04-07T16:57:41.705 INFO:tasks.workunit.client.0.smithi100.stdout:op 5759 completed, throughput=5MB/sec 2024-04-07T16:57:41.705 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-bPvnNU5fRgsAa7c off=6437481 len=1974677 2024-04-07T16:57:41.719 INFO:tasks.workunit.client.0.smithi100.stdout:op 5760 completed, throughput=5MB/sec 2024-04-07T16:57:41.719 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-X7YmvvV_vnRreQG off=3366262 len=142752 2024-04-07T16:57:41.723 INFO:tasks.workunit.client.0.smithi100.stdout:op 5761 completed, throughput=5MB/sec 2024-04-07T16:57:41.724 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-vuQKF1cJ6qZaBKB off=2939033 len=1636035 2024-04-07T16:57:41.729 INFO:tasks.workunit.client.0.smithi100.stdout:op 5762 completed, throughput=5MB/sec 2024-04-07T16:57:42.613 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:57:42 smithi144 bash[95233]: ts=2024-04-07T16:57:42.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:57:42.729 INFO:tasks.workunit.client.0.smithi100.stdout: 1109: throughput=5MB/sec pending data=0 2024-04-07T16:57:42.730 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-V76NZNSqme8_gEs off=2669133 len=1271070 2024-04-07T16:57:42.741 INFO:tasks.workunit.client.0.smithi100.stdout:op 5763 completed, throughput=5MB/sec 2024-04-07T16:57:42.741 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-ILbM8dnOyI1-Uso off=98028 len=1866341 2024-04-07T16:57:42.748 INFO:tasks.workunit.client.0.smithi100.stdout:op 5764 completed, throughput=5MB/sec 2024-04-07T16:57:42.749 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-PcF_Ac0J_qg2p4x off=3910445 len=1549426 2024-04-07T16:57:42.761 INFO:tasks.workunit.client.0.smithi100.stdout:op 5765 completed, throughput=5MB/sec 2024-04-07T16:57:43.113 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:57:42 smithi144 bash[66654]: cluster 2024-04-07T16:57:41.655102+0000 mgr.y (mgr.25175) 1270 : cluster [DBG] pgmap v1912: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.9 MiB/s wr, 5 op/s 2024-04-07T16:57:43.199 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:57:42 smithi100 bash[74942]: cluster 2024-04-07T16:57:41.655102+0000 mgr.y (mgr.25175) 1270 : cluster [DBG] pgmap v1912: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.9 MiB/s wr, 5 op/s 2024-04-07T16:57:43.200 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:57:42 smithi100 bash[76202]: cluster 2024-04-07T16:57:41.655102+0000 mgr.y (mgr.25175) 1270 : cluster [DBG] pgmap v1912: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.9 MiB/s wr, 5 op/s 2024-04-07T16:57:43.761 INFO:tasks.workunit.client.0.smithi100.stdout: 1110: throughput=5MB/sec pending data=0 2024-04-07T16:57:43.761 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-2AOiGGP5X7Ml5JL off=1070551 len=1210645 2024-04-07T16:57:43.765 INFO:tasks.workunit.client.0.smithi100.stdout:op 5766 completed, throughput=5MB/sec 2024-04-07T16:57:43.765 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-aX90OAmYsrEPNTf off=3256070 len=403756 2024-04-07T16:57:43.771 INFO:tasks.workunit.client.0.smithi100.stdout:op 5767 completed, throughput=5MB/sec 2024-04-07T16:57:43.771 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-SBIFf1yc1fshROo off=640909 len=1454078 2024-04-07T16:57:43.775 INFO:tasks.workunit.client.0.smithi100.stdout:op 5768 completed, throughput=5MB/sec 2024-04-07T16:57:43.775 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-bIFvcXXi6PW-h_l off=19663 len=342293 2024-04-07T16:57:43.776 INFO:tasks.workunit.client.0.smithi100.stdout:op 5769 completed, throughput=5MB/sec 2024-04-07T16:57:43.776 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-3Vacwvuqmepdd0o off=14852481 len=943434 2024-04-07T16:57:43.779 INFO:tasks.workunit.client.0.smithi100.stdout:op 5770 completed, throughput=5MB/sec 2024-04-07T16:57:43.779 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-OgnXIfOk_fkD3UG off=597566 len=2052905 2024-04-07T16:57:43.795 INFO:tasks.workunit.client.0.smithi100.stdout:op 5771 completed, throughput=5MB/sec 2024-04-07T16:57:44.113 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:57:43 smithi144 bash[66654]: cluster 2024-04-07T16:57:43.656239+0000 mgr.y (mgr.25175) 1271 : cluster [DBG] pgmap v1913: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 3.1 MiB/s rd, 2.8 MiB/s wr, 5 op/s 2024-04-07T16:57:44.199 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:57:43 smithi100 bash[74942]: cluster 2024-04-07T16:57:43.656239+0000 mgr.y (mgr.25175) 1271 : cluster [DBG] pgmap v1913: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 3.1 MiB/s rd, 2.8 MiB/s wr, 5 op/s 2024-04-07T16:57:44.200 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:57:43 smithi100 bash[76202]: cluster 2024-04-07T16:57:43.656239+0000 mgr.y (mgr.25175) 1271 : cluster [DBG] pgmap v1913: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 3.1 MiB/s rd, 2.8 MiB/s wr, 5 op/s 2024-04-07T16:57:44.795 INFO:tasks.workunit.client.0.smithi100.stdout: 1111: throughput=5MB/sec pending data=0 2024-04-07T16:57:44.796 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-xkgflNorcmAmTJb off=16520647 len=853840 2024-04-07T16:57:44.804 INFO:tasks.workunit.client.0.smithi100.stdout:op 5772 completed, throughput=5MB/sec 2024-04-07T16:57:44.804 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-3lBSUIS48yqqBRP off=5130698 len=213446 2024-04-07T16:57:44.809 INFO:tasks.workunit.client.0.smithi100.stdout:op 5773 completed, throughput=5MB/sec 2024-04-07T16:57:44.809 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-LUFsAs1uWoyfP4L off=23273813 len=1399779 2024-04-07T16:57:44.820 INFO:tasks.workunit.client.0.smithi100.stdout:op 5774 completed, throughput=5MB/sec 2024-04-07T16:57:44.820 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-cvttg9bwXFqDvQc off=480540 len=1907767 2024-04-07T16:57:44.835 INFO:tasks.workunit.client.0.smithi100.stdout:op 5775 completed, throughput=5MB/sec 2024-04-07T16:57:45.835 INFO:tasks.workunit.client.0.smithi100.stdout: 1112: throughput=5MB/sec pending data=0 2024-04-07T16:57:45.836 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-YN7n2OekwmwW8D9 off=19640696 len=1333517 2024-04-07T16:57:45.848 INFO:tasks.workunit.client.0.smithi100.stdout:op 5776 completed, throughput=5MB/sec 2024-04-07T16:57:45.848 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Lyd-LFzHN2fcLbg off=7681671 len=1161636 2024-04-07T16:57:45.856 INFO:tasks.workunit.client.0.smithi100.stdout:op 5777 completed, throughput=5MB/sec 2024-04-07T16:57:45.856 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-UYp8NJqh7fy9qCi off=9296111 len=1591158 2024-04-07T16:57:45.860 INFO:tasks.workunit.client.0.smithi100.stdout:op 5778 completed, throughput=5MB/sec 2024-04-07T16:57:45.860 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-489cQamw8UceI6Y off=9830526 len=536749 2024-04-07T16:57:45.863 INFO:tasks.workunit.client.0.smithi100.stdout:op 5779 completed, throughput=5MB/sec 2024-04-07T16:57:45.863 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-e45XR_6lpsavE3d off=1154796 len=1713752 2024-04-07T16:57:45.875 INFO:tasks.workunit.client.0.smithi100.stdout:op 5780 completed, throughput=5MB/sec 2024-04-07T16:57:46.876 INFO:tasks.workunit.client.0.smithi100.stdout: 1113: throughput=5MB/sec pending data=0 2024-04-07T16:57:46.876 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-uNEnHn6vErivj-7 off=16639160 len=1130107 2024-04-07T16:57:46.879 INFO:tasks.workunit.client.0.smithi100.stdout:op 5781 completed, throughput=5MB/sec 2024-04-07T16:57:46.879 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-57jlwlmu3igcUAq off=18982448 len=677944 2024-04-07T16:57:46.882 INFO:tasks.workunit.client.0.smithi100.stdout:op 5782 completed, throughput=5MB/sec 2024-04-07T16:57:46.882 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-_OKk-YsJTCBvmvu off=4411838 len=743112 2024-04-07T16:57:46.890 INFO:tasks.workunit.client.0.smithi100.stdout:op 5783 completed, throughput=5MB/sec 2024-04-07T16:57:46.890 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-mRuLogdiA9NdA6N off=16643435 len=1033914 2024-04-07T16:57:46.893 INFO:tasks.workunit.client.0.smithi100.stdout:op 5784 completed, throughput=5MB/sec 2024-04-07T16:57:46.893 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Z2BVpRQE6an9bUS off=9099060 len=1414464 2024-04-07T16:57:46.897 INFO:tasks.workunit.client.0.smithi100.stdout:op 5785 completed, throughput=5MB/sec 2024-04-07T16:57:47.113 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:57:46 smithi144 bash[66654]: cluster 2024-04-07T16:57:45.657202+0000 mgr.y (mgr.25175) 1272 : cluster [DBG] pgmap v1914: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 3.0 MiB/s rd, 2.2 MiB/s wr, 4 op/s 2024-04-07T16:57:47.200 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:57:46 smithi100 bash[74942]: cluster 2024-04-07T16:57:45.657202+0000 mgr.y (mgr.25175) 1272 : cluster [DBG] pgmap v1914: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 3.0 MiB/s rd, 2.2 MiB/s wr, 4 op/s 2024-04-07T16:57:47.200 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:57:46 smithi100 bash[76202]: cluster 2024-04-07T16:57:45.657202+0000 mgr.y (mgr.25175) 1272 : cluster [DBG] pgmap v1914: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 3.0 MiB/s rd, 2.2 MiB/s wr, 4 op/s 2024-04-07T16:57:47.898 INFO:tasks.workunit.client.0.smithi100.stdout: 1114: throughput=5MB/sec pending data=0 2024-04-07T16:57:47.898 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-NxEuc9fSyL5fpj7 off=3604900 len=580976 2024-04-07T16:57:47.900 INFO:tasks.workunit.client.0.smithi100.stdout:op 5786 completed, throughput=5MB/sec 2024-04-07T16:57:47.900 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-drWs3qk1MloYk6I off=313496 len=1439790 2024-04-07T16:57:47.905 INFO:tasks.workunit.client.0.smithi100.stdout:op 5787 completed, throughput=5MB/sec 2024-04-07T16:57:47.905 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-eleosf_B5EuXZTO off=16374241 len=1844720 2024-04-07T16:57:47.919 INFO:tasks.workunit.client.0.smithi100.stdout:op 5788 completed, throughput=5MB/sec 2024-04-07T16:57:47.919 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-D2YuI9ZZTFm_vsA off=2090770 len=60937 2024-04-07T16:57:47.920 INFO:tasks.workunit.client.0.smithi100.stdout:op 5789 completed, throughput=5MB/sec 2024-04-07T16:57:47.921 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-hQBh5oRknMnQYZg off=2066160 len=1650199 2024-04-07T16:57:47.923 INFO:tasks.workunit.client.0.smithi100.stdout:op 5790 completed, throughput=5MB/sec 2024-04-07T16:57:48.113 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:57:47 smithi144 bash[66654]: audit 2024-04-07T16:57:46.789473+0000 mon.a (mon.0) 2209 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:57:48.199 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:57:47 smithi100 bash[74942]: audit 2024-04-07T16:57:46.789473+0000 mon.a (mon.0) 2209 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:57:48.200 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:57:47 smithi100 bash[76202]: audit 2024-04-07T16:57:46.789473+0000 mon.a (mon.0) 2209 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:57:48.699 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:57:48 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:57:48] "GET /metrics HTTP/1.1" 200 34074 "" "Prometheus/2.43.0" 2024-04-07T16:57:48.924 INFO:tasks.workunit.client.0.smithi100.stdout: 1115: throughput=5MB/sec pending data=0 2024-04-07T16:57:48.924 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-EirQjjOAfCq40Ei off=2696602 len=908557 2024-04-07T16:57:48.932 INFO:tasks.workunit.client.0.smithi100.stdout:op 5791 completed, throughput=5MB/sec 2024-04-07T16:57:48.932 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-nvc-0z5HAZsLCET off=9212944 len=1197359 2024-04-07T16:57:48.935 INFO:tasks.workunit.client.0.smithi100.stdout:op 5792 completed, throughput=5MB/sec 2024-04-07T16:57:48.935 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-EUCfNUgwwv07-v3 off=10630097 len=802099 2024-04-07T16:57:48.937 INFO:tasks.workunit.client.0.smithi100.stdout:op 5793 completed, throughput=5MB/sec 2024-04-07T16:57:48.937 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-CeXPzbt_wc8RD9p off=1073012 len=1505126 2024-04-07T16:57:48.942 INFO:tasks.workunit.client.0.smithi100.stdout:op 5794 completed, throughput=5MB/sec 2024-04-07T16:57:49.113 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:57:48 smithi144 bash[66654]: cluster 2024-04-07T16:57:47.658280+0000 mgr.y (mgr.25175) 1273 : cluster [DBG] pgmap v1915: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 2.8 MiB/s rd, 2.5 MiB/s wr, 4 op/s 2024-04-07T16:57:49.199 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:57:48 smithi100 bash[74942]: cluster 2024-04-07T16:57:47.658280+0000 mgr.y (mgr.25175) 1273 : cluster [DBG] pgmap v1915: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 2.8 MiB/s rd, 2.5 MiB/s wr, 4 op/s 2024-04-07T16:57:49.199 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:57:48 smithi100 bash[76202]: cluster 2024-04-07T16:57:47.658280+0000 mgr.y (mgr.25175) 1273 : cluster [DBG] pgmap v1915: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 2.8 MiB/s rd, 2.5 MiB/s wr, 4 op/s 2024-04-07T16:57:49.943 INFO:tasks.workunit.client.0.smithi100.stdout: 1116: throughput=5MB/sec pending data=0 2024-04-07T16:57:49.943 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-9C9IaV-8GCmOvy8 off=13158489 len=1472745 2024-04-07T16:57:49.947 INFO:tasks.workunit.client.0.smithi100.stdout:op 5795 completed, throughput=5MB/sec 2024-04-07T16:57:49.947 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-AOvzazVGK0NaUH0 off=2247875 len=1082617 2024-04-07T16:57:49.949 INFO:tasks.workunit.client.0.smithi100.stdout:op 5796 completed, throughput=5MB/sec 2024-04-07T16:57:49.949 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-t0J0ZPu3EPuYr1c off=832822 len=604087 2024-04-07T16:57:49.951 INFO:tasks.workunit.client.0.smithi100.stdout:op 5797 completed, throughput=5MB/sec 2024-04-07T16:57:49.951 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-EDLBzEXdQ5ESLax off=12460198 len=1352695 2024-04-07T16:57:49.965 INFO:tasks.workunit.client.0.smithi100.stdout:op 5798 completed, throughput=5MB/sec 2024-04-07T16:57:49.965 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-vZC8KHkgZsTPWh7 off=2017242 len=404681 2024-04-07T16:57:49.971 INFO:tasks.workunit.client.0.smithi100.stdout:op 5799 completed, throughput=5MB/sec 2024-04-07T16:57:49.971 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-IeaRK8lnOuB8sy5 off=5498960 len=1401336 2024-04-07T16:57:49.983 INFO:tasks.workunit.client.0.smithi100.stdout:op 5800 completed, throughput=5MB/sec 2024-04-07T16:57:50.983 INFO:tasks.workunit.client.0.smithi100.stdout: 1117: throughput=5MB/sec pending data=0 2024-04-07T16:57:50.983 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Yr5KudL5JBXEGw- off=16929717 len=333771 2024-04-07T16:57:50.989 INFO:tasks.workunit.client.0.smithi100.stdout:op 5801 completed, throughput=5MB/sec 2024-04-07T16:57:50.990 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-nocbdgcIe7jbwY7 off=12052260 len=1368460 2024-04-07T16:57:51.001 INFO:tasks.workunit.client.0.smithi100.stdout:op 5802 completed, throughput=5MB/sec 2024-04-07T16:57:51.001 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-zOSm7juujfJnxpQ off=5146245 len=1096903 2024-04-07T16:57:51.004 INFO:tasks.workunit.client.0.smithi100.stdout:op 5803 completed, throughput=5MB/sec 2024-04-07T16:57:51.004 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-VR6lggY4CqnidXo off=5548770 len=344903 2024-04-07T16:57:51.011 INFO:tasks.workunit.client.0.smithi100.stdout:op 5804 completed, throughput=5MB/sec 2024-04-07T16:57:51.011 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-bmVMeLkidnNyG0B off=17042461 len=1989855 2024-04-07T16:57:51.016 INFO:tasks.workunit.client.0.smithi100.stdout:op 5805 completed, throughput=5MB/sec 2024-04-07T16:57:51.113 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:57:50 smithi144 bash[66654]: cluster 2024-04-07T16:57:49.659261+0000 mgr.y (mgr.25175) 1274 : cluster [DBG] pgmap v1916: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 2.3 MiB/s rd, 2.6 MiB/s wr, 4 op/s 2024-04-07T16:57:51.199 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:57:50 smithi100 bash[74942]: cluster 2024-04-07T16:57:49.659261+0000 mgr.y (mgr.25175) 1274 : cluster [DBG] pgmap v1916: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 2.3 MiB/s rd, 2.6 MiB/s wr, 4 op/s 2024-04-07T16:57:51.200 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:57:50 smithi100 bash[76202]: cluster 2024-04-07T16:57:49.659261+0000 mgr.y (mgr.25175) 1274 : cluster [DBG] pgmap v1916: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 2.3 MiB/s rd, 2.6 MiB/s wr, 4 op/s 2024-04-07T16:57:52.016 INFO:tasks.workunit.client.0.smithi100.stdout: 1118: throughput=5MB/sec pending data=0 2024-04-07T16:57:52.016 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-8xMrZmXWCpZX8x3 off=11285556 len=1983842 2024-04-07T16:57:52.035 INFO:tasks.workunit.client.0.smithi100.stdout:op 5806 completed, throughput=5MB/sec 2024-04-07T16:57:52.035 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-l04MA-AT9R9hqCG off=6065297 len=1703602 2024-04-07T16:57:52.049 INFO:tasks.workunit.client.0.smithi100.stdout:op 5807 completed, throughput=5MB/sec 2024-04-07T16:57:52.049 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-AVkKmdPlZpnIWkd off=16804328 len=1376788 2024-04-07T16:57:52.052 INFO:tasks.workunit.client.0.smithi100.stdout:op 5808 completed, throughput=5MB/sec 2024-04-07T16:57:52.613 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:57:52 smithi144 bash[95233]: ts=2024-04-07T16:57:52.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:57:53.053 INFO:tasks.workunit.client.0.smithi100.stdout: 1119: throughput=5MB/sec pending data=0 2024-04-07T16:57:53.053 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-JLw_aVeWS9UWNeE off=6075561 len=1446665 2024-04-07T16:57:53.057 INFO:tasks.workunit.client.0.smithi100.stdout:op 5809 completed, throughput=5MB/sec 2024-04-07T16:57:53.058 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-PjnDa2cQ23vcftg off=7433803 len=396614 2024-04-07T16:57:53.065 INFO:tasks.workunit.client.0.smithi100.stdout:op 5810 completed, throughput=5MB/sec 2024-04-07T16:57:53.065 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-lYEJRRsexraWSct off=1763563 len=484680 2024-04-07T16:57:53.073 INFO:tasks.workunit.client.0.smithi100.stdout:op 5811 completed, throughput=5MB/sec 2024-04-07T16:57:53.073 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-256VuYzQd2wo-a3 off=3006213 len=1208403 2024-04-07T16:57:53.076 INFO:tasks.workunit.client.0.smithi100.stdout:op 5812 completed, throughput=5MB/sec 2024-04-07T16:57:53.076 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-gRmsJgjz-Knh93g off=921535 len=829348 2024-04-07T16:57:53.078 INFO:tasks.workunit.client.0.smithi100.stdout:op 5813 completed, throughput=5MB/sec 2024-04-07T16:57:53.078 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-nJO9kcSgj_ojWum off=809883 len=813088 2024-04-07T16:57:53.086 INFO:tasks.workunit.client.0.smithi100.stdout:op 5814 completed, throughput=5MB/sec 2024-04-07T16:57:53.113 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:57:52 smithi144 bash[66654]: cluster 2024-04-07T16:57:51.660070+0000 mgr.y (mgr.25175) 1275 : cluster [DBG] pgmap v1917: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.2 MiB/s wr, 4 op/s 2024-04-07T16:57:53.199 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:57:52 smithi100 bash[74942]: cluster 2024-04-07T16:57:51.660070+0000 mgr.y (mgr.25175) 1275 : cluster [DBG] pgmap v1917: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.2 MiB/s wr, 4 op/s 2024-04-07T16:57:53.200 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:57:52 smithi100 bash[76202]: cluster 2024-04-07T16:57:51.660070+0000 mgr.y (mgr.25175) 1275 : cluster [DBG] pgmap v1917: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.2 MiB/s wr, 4 op/s 2024-04-07T16:57:54.087 INFO:tasks.workunit.client.0.smithi100.stdout: 1120: throughput=5MB/sec pending data=0 2024-04-07T16:57:54.087 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Vxl6u2kTAcHBMwC off=6102871 len=1968842 2024-04-07T16:57:54.103 INFO:tasks.workunit.client.0.smithi100.stdout:op 5815 completed, throughput=5MB/sec 2024-04-07T16:57:54.103 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-jVfmm2O2ilqRqdo off=16796360 len=85488 2024-04-07T16:57:54.104 INFO:tasks.workunit.client.0.smithi100.stdout:op 5816 completed, throughput=5MB/sec 2024-04-07T16:57:54.104 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-oDNUeSr2JfwE1Un off=9293737 len=517906 2024-04-07T16:57:54.111 INFO:tasks.workunit.client.0.smithi100.stdout:op 5817 completed, throughput=5MB/sec 2024-04-07T16:57:54.111 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-47trYNsSmP2yQBa off=16266266 len=838021 2024-04-07T16:57:54.119 INFO:tasks.workunit.client.0.smithi100.stdout:op 5818 completed, throughput=5MB/sec 2024-04-07T16:57:54.120 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-YzEHtsHmYHbp7Kg off=3449726 len=1790641 2024-04-07T16:57:54.131 INFO:tasks.workunit.client.0.smithi100.stdout:op 5819 completed, throughput=5MB/sec 2024-04-07T16:57:55.113 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:57:54 smithi144 bash[66654]: cluster 2024-04-07T16:57:53.661336+0000 mgr.y (mgr.25175) 1276 : cluster [DBG] pgmap v1918: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 2.6 MiB/s rd, 2.4 MiB/s wr, 4 op/s 2024-04-07T16:57:55.131 INFO:tasks.workunit.client.0.smithi100.stdout: 1121: throughput=5MB/sec pending data=0 2024-04-07T16:57:55.131 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-3bjM6jPDrDroMl8 off=60762 len=1857979 2024-04-07T16:57:55.136 INFO:tasks.workunit.client.0.smithi100.stdout:op 5820 completed, throughput=5MB/sec 2024-04-07T16:57:55.136 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Kp-XB0qyddUL8j2 off=15315020 len=564011 2024-04-07T16:57:55.138 INFO:tasks.workunit.client.0.smithi100.stdout:op 5821 completed, throughput=5MB/sec 2024-04-07T16:57:55.138 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-d4f-9RvHWtP98cV off=1286881 len=2018826 2024-04-07T16:57:55.155 INFO:tasks.workunit.client.0.smithi100.stdout:op 5822 completed, throughput=5MB/sec 2024-04-07T16:57:55.155 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj--5MNbk23Z8oBHzD off=24361281 len=992012 2024-04-07T16:57:55.158 INFO:tasks.workunit.client.0.smithi100.stdout:op 5823 completed, throughput=5MB/sec 2024-04-07T16:57:55.199 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:57:54 smithi100 bash[74942]: cluster 2024-04-07T16:57:53.661336+0000 mgr.y (mgr.25175) 1276 : cluster [DBG] pgmap v1918: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 2.6 MiB/s rd, 2.4 MiB/s wr, 4 op/s 2024-04-07T16:57:55.199 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:57:54 smithi100 bash[76202]: cluster 2024-04-07T16:57:53.661336+0000 mgr.y (mgr.25175) 1276 : cluster [DBG] pgmap v1918: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 2.6 MiB/s rd, 2.4 MiB/s wr, 4 op/s 2024-04-07T16:57:56.159 INFO:tasks.workunit.client.0.smithi100.stdout: 1122: throughput=5MB/sec pending data=0 2024-04-07T16:57:56.159 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-9IB15rkpC9xN9Un off=2716952 len=1726735 2024-04-07T16:57:56.163 INFO:tasks.workunit.client.0.smithi100.stdout:op 5824 completed, throughput=5MB/sec 2024-04-07T16:57:56.163 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-KRExT6kfuPKPNfQ off=3803634 len=953004 2024-04-07T16:57:56.166 INFO:tasks.workunit.client.0.smithi100.stdout:op 5825 completed, throughput=5MB/sec 2024-04-07T16:57:56.166 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-T8V3t3N-r4MYeKZ off=2916319 len=1655401 2024-04-07T16:57:56.179 INFO:tasks.workunit.client.0.smithi100.stdout:op 5826 completed, throughput=5MB/sec 2024-04-07T16:57:56.179 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-oNH7hXVQpYiVjpO off=3503395 len=1042942 2024-04-07T16:57:56.183 INFO:tasks.workunit.client.0.smithi100.stdout:op 5827 completed, throughput=5MB/sec 2024-04-07T16:57:57.113 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:57:56 smithi144 bash[66654]: cluster 2024-04-07T16:57:55.662346+0000 mgr.y (mgr.25175) 1277 : cluster [DBG] pgmap v1919: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 2.2 MiB/s rd, 2.3 MiB/s wr, 4 op/s 2024-04-07T16:57:57.184 INFO:tasks.workunit.client.0.smithi100.stdout: 1123: throughput=5MB/sec pending data=0 2024-04-07T16:57:57.184 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-qvHdON4KOsjhNg9 off=1577096 len=977052 2024-04-07T16:57:57.187 INFO:tasks.workunit.client.0.smithi100.stdout:op 5828 completed, throughput=5MB/sec 2024-04-07T16:57:57.187 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-7IC3R9tjmOdvnXl off=8754914 len=2084120 2024-04-07T16:57:57.191 INFO:tasks.workunit.client.0.smithi100.stdout:op 5829 completed, throughput=5MB/sec 2024-04-07T16:57:57.191 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-MJvsSl-8MqbYq5b off=808920 len=372994 2024-04-07T16:57:57.198 INFO:tasks.workunit.client.0.smithi100.stdout:op 5830 completed, throughput=5MB/sec 2024-04-07T16:57:57.198 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-4A56bB4bKF-WMZg off=1311361 len=1816618 2024-04-07T16:57:57.199 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:57:56 smithi100 bash[74942]: cluster 2024-04-07T16:57:55.662346+0000 mgr.y (mgr.25175) 1277 : cluster [DBG] pgmap v1919: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 2.2 MiB/s rd, 2.3 MiB/s wr, 4 op/s 2024-04-07T16:57:57.199 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:57:56 smithi100 bash[76202]: cluster 2024-04-07T16:57:55.662346+0000 mgr.y (mgr.25175) 1277 : cluster [DBG] pgmap v1919: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 2.2 MiB/s rd, 2.3 MiB/s wr, 4 op/s 2024-04-07T16:57:57.203 INFO:tasks.workunit.client.0.smithi100.stdout:op 5831 completed, throughput=5MB/sec 2024-04-07T16:57:58.203 INFO:tasks.workunit.client.0.smithi100.stdout: 1124: throughput=5MB/sec pending data=0 2024-04-07T16:57:58.204 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-32IKDRbNvcRr3CD off=836665 len=1480404 2024-04-07T16:57:58.215 INFO:tasks.workunit.client.0.smithi100.stdout:op 5832 completed, throughput=5MB/sec 2024-04-07T16:57:58.215 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-MrWjj8g5py-ez_5 off=559447 len=340799 2024-04-07T16:57:58.221 INFO:tasks.workunit.client.0.smithi100.stdout:op 5833 completed, throughput=5MB/sec 2024-04-07T16:57:58.221 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-FNk6u7pEaWvN6D0 off=22275653 len=840148 2024-04-07T16:57:58.229 INFO:tasks.workunit.client.0.smithi100.stdout:op 5834 completed, throughput=5MB/sec 2024-04-07T16:57:58.229 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-aNCF0kLCpEisHUf off=4546754 len=1972334 2024-04-07T16:57:58.233 INFO:tasks.workunit.client.0.smithi100.stdout:op 5835 completed, throughput=5MB/sec 2024-04-07T16:57:58.233 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-EVBp4mxgbP78m3i off=1869425 len=700641 2024-04-07T16:57:58.236 INFO:tasks.workunit.client.0.smithi100.stdout:op 5836 completed, throughput=5MB/sec 2024-04-07T16:57:58.699 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:57:58 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:57:58] "GET /metrics HTTP/1.1" 200 34074 "" "Prometheus/2.43.0" 2024-04-07T16:57:59.113 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:57:58 smithi144 bash[66654]: cluster 2024-04-07T16:57:57.663391+0000 mgr.y (mgr.25175) 1278 : cluster [DBG] pgmap v1920: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.0 MiB/s wr, 4 op/s 2024-04-07T16:57:59.199 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:57:58 smithi100 bash[74942]: cluster 2024-04-07T16:57:57.663391+0000 mgr.y (mgr.25175) 1278 : cluster [DBG] pgmap v1920: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.0 MiB/s wr, 4 op/s 2024-04-07T16:57:59.200 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:57:58 smithi100 bash[76202]: cluster 2024-04-07T16:57:57.663391+0000 mgr.y (mgr.25175) 1278 : cluster [DBG] pgmap v1920: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.0 MiB/s wr, 4 op/s 2024-04-07T16:57:59.236 INFO:tasks.workunit.client.0.smithi100.stdout: 1125: throughput=5MB/sec pending data=0 2024-04-07T16:57:59.236 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-vdhthNdUzgl8aW3 off=957573 len=561179 2024-04-07T16:57:59.243 INFO:tasks.workunit.client.0.smithi100.stdout:op 5837 completed, throughput=5MB/sec 2024-04-07T16:57:59.243 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-jjD72q-9DjGAXnH off=921386 len=1468680 2024-04-07T16:57:59.255 INFO:tasks.workunit.client.0.smithi100.stdout:op 5838 completed, throughput=5MB/sec 2024-04-07T16:57:59.255 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-0PrKswW-dME9S8d off=1506381 len=1281652 2024-04-07T16:57:59.266 INFO:tasks.workunit.client.0.smithi100.stdout:op 5839 completed, throughput=5MB/sec 2024-04-07T16:57:59.266 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-SybbZwPRsHhLuvn off=469134 len=1159995 2024-04-07T16:57:59.275 INFO:tasks.workunit.client.0.smithi100.stdout:op 5840 completed, throughput=5MB/sec 2024-04-07T16:57:59.275 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Jv9uh6XEYwzlRqn off=1120550 len=1988631 2024-04-07T16:57:59.277 INFO:tasks.workunit.client.0.smithi100.stdout:op 5841 completed, throughput=5MB/sec 2024-04-07T16:58:00.113 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:57:59 smithi144 bash[66654]: cluster 2024-04-07T16:57:59.664365+0000 mgr.y (mgr.25175) 1279 : cluster [DBG] pgmap v1921: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 2.2 MiB/s rd, 1.9 MiB/s wr, 3 op/s 2024-04-07T16:58:00.199 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:57:59 smithi100 bash[74942]: cluster 2024-04-07T16:57:59.664365+0000 mgr.y (mgr.25175) 1279 : cluster [DBG] pgmap v1921: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 2.2 MiB/s rd, 1.9 MiB/s wr, 3 op/s 2024-04-07T16:58:00.200 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:57:59 smithi100 bash[76202]: cluster 2024-04-07T16:57:59.664365+0000 mgr.y (mgr.25175) 1279 : cluster [DBG] pgmap v1921: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 2.2 MiB/s rd, 1.9 MiB/s wr, 3 op/s 2024-04-07T16:58:00.277 INFO:tasks.workunit.client.0.smithi100.stdout: 1126: throughput=5MB/sec pending data=0 2024-04-07T16:58:00.278 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-60dWq99Ts2Jpc2_ off=20067327 len=624782 2024-04-07T16:58:00.280 INFO:tasks.workunit.client.0.smithi100.stdout:op 5842 completed, throughput=5MB/sec 2024-04-07T16:58:00.280 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-JtWo1diB6198gM6 off=4076318 len=1480705 2024-04-07T16:58:00.283 INFO:tasks.workunit.client.0.smithi100.stdout:op 5843 completed, throughput=5MB/sec 2024-04-07T16:58:00.283 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-A1_qiGTk6HMzT9W off=6060515 len=1397633 2024-04-07T16:58:00.287 INFO:tasks.workunit.client.0.smithi100.stdout:op 5844 completed, throughput=5MB/sec 2024-04-07T16:58:00.287 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-AWkFO1S5x5AveVD off=646045 len=515328 2024-04-07T16:58:00.289 INFO:tasks.workunit.client.0.smithi100.stdout:op 5845 completed, throughput=5MB/sec 2024-04-07T16:58:00.289 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-oj2DyFgY_Rk0qU_ off=9772146 len=2094146 2024-04-07T16:58:00.308 INFO:tasks.workunit.client.0.smithi100.stdout:op 5846 completed, throughput=5MB/sec 2024-04-07T16:58:01.309 INFO:tasks.workunit.client.0.smithi100.stdout: 1127: throughput=5MB/sec pending data=0 2024-04-07T16:58:01.310 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-UrH3nBLhuzlczJ8 off=1278562 len=505512 2024-04-07T16:58:01.315 INFO:tasks.workunit.client.0.smithi100.stdout:op 5847 completed, throughput=5MB/sec 2024-04-07T16:58:01.315 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-9OI2olpHIAWRFS0 off=2080256 len=1015275 2024-04-07T16:58:01.325 INFO:tasks.workunit.client.0.smithi100.stdout:op 5848 completed, throughput=5MB/sec 2024-04-07T16:58:01.325 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-SDrqWZGnwRLYVMh off=312939 len=459339 2024-04-07T16:58:01.327 INFO:tasks.workunit.client.0.smithi100.stdout:op 5849 completed, throughput=5MB/sec 2024-04-07T16:58:01.327 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-6ErhQNh2JXlllft off=5142706 len=243529 2024-04-07T16:58:01.333 INFO:tasks.workunit.client.0.smithi100.stdout:op 5850 completed, throughput=5MB/sec 2024-04-07T16:58:01.333 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-AFG4XLVuOBwjGb6 off=7662026 len=1177906 2024-04-07T16:58:01.344 INFO:tasks.workunit.client.0.smithi100.stdout:op 5851 completed, throughput=5MB/sec 2024-04-07T16:58:01.344 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj--O6Mu4Vaago3LL3 off=1551852 len=1836557 2024-04-07T16:58:01.348 INFO:tasks.workunit.client.0.smithi100.stdout:op 5852 completed, throughput=5MB/sec 2024-04-07T16:58:02.349 INFO:tasks.workunit.client.0.smithi100.stdout: 1128: throughput=5MB/sec pending data=0 2024-04-07T16:58:02.349 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-GUAZKEQuBkaBkKV off=4480834 len=1998013 2024-04-07T16:58:02.355 INFO:tasks.workunit.client.0.smithi100.stdout:op 5853 completed, throughput=5MB/sec 2024-04-07T16:58:02.355 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-dE1KTJMsGo5RMEq off=4856106 len=1980079 2024-04-07T16:58:02.361 INFO:tasks.workunit.client.0.smithi100.stdout:op 5854 completed, throughput=5MB/sec 2024-04-07T16:58:02.613 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:58:02 smithi144 bash[95233]: ts=2024-04-07T16:58:02.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:58:03.113 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:58:02 smithi144 bash[66654]: cluster 2024-04-07T16:58:01.665134+0000 mgr.y (mgr.25175) 1280 : cluster [DBG] pgmap v1922: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 2.9 MiB/s rd, 2.0 MiB/s wr, 4 op/s 2024-04-07T16:58:03.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:58:02 smithi144 bash[66654]: audit 2024-04-07T16:58:01.789888+0000 mon.a (mon.0) 2210 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:58:03.199 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:58:02 smithi100 bash[74942]: cluster 2024-04-07T16:58:01.665134+0000 mgr.y (mgr.25175) 1280 : cluster [DBG] pgmap v1922: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 2.9 MiB/s rd, 2.0 MiB/s wr, 4 op/s 2024-04-07T16:58:03.200 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:58:02 smithi100 bash[74942]: audit 2024-04-07T16:58:01.789888+0000 mon.a (mon.0) 2210 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:58:03.200 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:58:02 smithi100 bash[76202]: cluster 2024-04-07T16:58:01.665134+0000 mgr.y (mgr.25175) 1280 : cluster [DBG] pgmap v1922: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 2.9 MiB/s rd, 2.0 MiB/s wr, 4 op/s 2024-04-07T16:58:03.200 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:58:02 smithi100 bash[76202]: audit 2024-04-07T16:58:01.789888+0000 mon.a (mon.0) 2210 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:58:03.362 INFO:tasks.workunit.client.0.smithi100.stdout: 1129: throughput=5MB/sec pending data=0 2024-04-07T16:58:03.362 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-YPhxZECfszznyEu off=17808918 len=1299698 2024-04-07T16:58:03.365 INFO:tasks.workunit.client.0.smithi100.stdout:op 5855 completed, throughput=5MB/sec 2024-04-07T16:58:03.366 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-XuElLwwcdrZvNxS off=1260882 len=1455597 2024-04-07T16:58:03.369 INFO:tasks.workunit.client.0.smithi100.stdout:op 5856 completed, throughput=5MB/sec 2024-04-07T16:58:03.369 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-dMDKyZbjsY3VNRs off=9803114 len=439557 2024-04-07T16:58:03.375 INFO:tasks.workunit.client.0.smithi100.stdout:op 5857 completed, throughput=5MB/sec 2024-04-07T16:58:03.376 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-7zsFoSHlMN7RpZJ off=4485473 len=1253328 2024-04-07T16:58:03.380 INFO:tasks.workunit.client.0.smithi100.stdout:op 5858 completed, throughput=5MB/sec 2024-04-07T16:58:03.380 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-z3WFheE14YhoKam off=13717798 len=61619 2024-04-07T16:58:03.381 INFO:tasks.workunit.client.0.smithi100.stdout:op 5859 completed, throughput=5MB/sec 2024-04-07T16:58:03.381 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-eSQDYZfW0MrybHE off=528186 len=1596844 2024-04-07T16:58:03.394 INFO:tasks.workunit.client.0.smithi100.stdout:op 5860 completed, throughput=5MB/sec 2024-04-07T16:58:04.395 INFO:tasks.workunit.client.0.smithi100.stdout: 1130: throughput=5MB/sec pending data=0 2024-04-07T16:58:04.395 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-o7OLx9msuOUIkOu off=2520777 len=66447 2024-04-07T16:58:04.400 INFO:tasks.workunit.client.0.smithi100.stdout:op 5861 completed, throughput=5MB/sec 2024-04-07T16:58:04.400 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-r_tYmSz7EAG_88y off=388384 len=802116 2024-04-07T16:58:04.409 INFO:tasks.workunit.client.0.smithi100.stdout:op 5862 completed, throughput=5MB/sec 2024-04-07T16:58:04.409 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-5iAf6JjsZFGJuzh off=6497964 len=471293 2024-04-07T16:58:04.411 INFO:tasks.workunit.client.0.smithi100.stdout:op 5863 completed, throughput=5MB/sec 2024-04-07T16:58:04.412 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-dKxqEnY4NZUf-Hv off=15023159 len=211008 2024-04-07T16:58:04.417 INFO:tasks.workunit.client.0.smithi100.stdout:op 5864 completed, throughput=5MB/sec 2024-04-07T16:58:04.417 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-xr_AU4wCuWEZMPm off=3812763 len=643827 2024-04-07T16:58:04.425 INFO:tasks.workunit.client.0.smithi100.stdout:op 5865 completed, throughput=5MB/sec 2024-04-07T16:58:04.425 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-QtZn3z7pQh9g_dA off=3126511 len=1277667 2024-04-07T16:58:04.428 INFO:tasks.workunit.client.0.smithi100.stdout:op 5866 completed, throughput=5MB/sec 2024-04-07T16:58:04.428 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Wlwc2On4U7bvYZB off=7500327 len=1144672 2024-04-07T16:58:04.438 INFO:tasks.workunit.client.0.smithi100.stdout:op 5867 completed, throughput=5MB/sec 2024-04-07T16:58:05.113 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:58:04 smithi144 bash[66654]: cluster 2024-04-07T16:58:03.666310+0000 mgr.y (mgr.25175) 1281 : cluster [DBG] pgmap v1923: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 2.8 MiB/s rd, 2.5 MiB/s wr, 4 op/s 2024-04-07T16:58:05.199 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:58:04 smithi100 bash[74942]: cluster 2024-04-07T16:58:03.666310+0000 mgr.y (mgr.25175) 1281 : cluster [DBG] pgmap v1923: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 2.8 MiB/s rd, 2.5 MiB/s wr, 4 op/s 2024-04-07T16:58:05.200 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:58:04 smithi100 bash[76202]: cluster 2024-04-07T16:58:03.666310+0000 mgr.y (mgr.25175) 1281 : cluster [DBG] pgmap v1923: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 2.8 MiB/s rd, 2.5 MiB/s wr, 4 op/s 2024-04-07T16:58:05.438 INFO:tasks.workunit.client.0.smithi100.stdout: 1131: throughput=5MB/sec pending data=0 2024-04-07T16:58:05.439 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-eIHJIc_9cWbUN9b off=1086868 len=414750 2024-04-07T16:58:05.444 INFO:tasks.workunit.client.0.smithi100.stdout:op 5868 completed, throughput=5MB/sec 2024-04-07T16:58:05.445 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-_Fw-7oDXehtx9k7 off=6014285 len=42379 2024-04-07T16:58:05.449 INFO:tasks.workunit.client.0.smithi100.stdout:op 5869 completed, throughput=5MB/sec 2024-04-07T16:58:05.449 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-XKmZHHSzP-LDFTz off=8023808 len=2042602 2024-04-07T16:58:05.454 INFO:tasks.workunit.client.0.smithi100.stdout:op 5870 completed, throughput=5MB/sec 2024-04-07T16:58:05.454 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-rcGTuAAgMZbQ3hG off=6373777 len=684104 2024-04-07T16:58:05.457 INFO:tasks.workunit.client.0.smithi100.stdout:op 5871 completed, throughput=5MB/sec 2024-04-07T16:58:05.457 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-6wZC-Va7zl4AE-q off=367025 len=1254102 2024-04-07T16:58:05.462 INFO:tasks.workunit.client.0.smithi100.stdout:op 5872 completed, throughput=5MB/sec 2024-04-07T16:58:05.462 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-pD4LLn0Z5AodxmE off=18525989 len=958569 2024-04-07T16:58:05.465 INFO:tasks.workunit.client.0.smithi100.stdout:op 5873 completed, throughput=5MB/sec 2024-04-07T16:58:06.466 INFO:tasks.workunit.client.0.smithi100.stdout: 1132: throughput=5MB/sec pending data=0 2024-04-07T16:58:06.466 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-hiNpwxOKfVIaTzE off=1646252 len=1992595 2024-04-07T16:58:06.483 INFO:tasks.workunit.client.0.smithi100.stdout:op 5874 completed, throughput=5MB/sec 2024-04-07T16:58:06.483 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-eUSz4siUDZNjaxu off=4884191 len=1477951 2024-04-07T16:58:06.486 INFO:tasks.workunit.client.0.smithi100.stdout:op 5875 completed, throughput=5MB/sec 2024-04-07T16:58:06.487 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-0US5owbi9Mjc0s1 off=167612 len=1612172 2024-04-07T16:58:06.492 INFO:tasks.workunit.client.0.smithi100.stdout:op 5876 completed, throughput=5MB/sec 2024-04-07T16:58:06.492 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-O5Ne7vWEehEIOY6 off=631629 len=623981 2024-04-07T16:58:06.500 INFO:tasks.workunit.client.0.smithi100.stdout:op 5877 completed, throughput=5MB/sec 2024-04-07T16:58:07.113 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:58:06 smithi144 bash[66654]: cluster 2024-04-07T16:58:05.667294+0000 mgr.y (mgr.25175) 1282 : cluster [DBG] pgmap v1924: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 2.6 MiB/s rd, 2.2 MiB/s wr, 4 op/s 2024-04-07T16:58:07.199 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:58:06 smithi100 bash[74942]: cluster 2024-04-07T16:58:05.667294+0000 mgr.y (mgr.25175) 1282 : cluster [DBG] pgmap v1924: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 2.6 MiB/s rd, 2.2 MiB/s wr, 4 op/s 2024-04-07T16:58:07.200 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:58:06 smithi100 bash[76202]: cluster 2024-04-07T16:58:05.667294+0000 mgr.y (mgr.25175) 1282 : cluster [DBG] pgmap v1924: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 2.6 MiB/s rd, 2.2 MiB/s wr, 4 op/s 2024-04-07T16:58:07.500 INFO:tasks.workunit.client.0.smithi100.stdout: 1133: throughput=5MB/sec pending data=0 2024-04-07T16:58:07.500 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-hx18q47zN7aSjMU off=2188458 len=761507 2024-04-07T16:58:07.508 INFO:tasks.workunit.client.0.smithi100.stdout:op 5878 completed, throughput=5MB/sec 2024-04-07T16:58:07.509 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj--WR2c8NfwVWQKpC off=22110661 len=212393 2024-04-07T16:58:07.510 INFO:tasks.workunit.client.0.smithi100.stdout:op 5879 completed, throughput=5MB/sec 2024-04-07T16:58:07.511 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj--zaZz8OUI_BLxzN off=6042018 len=1131635 2024-04-07T16:58:07.520 INFO:tasks.workunit.client.0.smithi100.stdout:op 5880 completed, throughput=5MB/sec 2024-04-07T16:58:07.520 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-cVKeW-oG5N2QDmo off=4466002 len=1349181 2024-04-07T16:58:07.530 INFO:tasks.workunit.client.0.smithi100.stdout:op 5881 completed, throughput=5MB/sec 2024-04-07T16:58:07.530 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-X3Yay11W3In3HuR off=343940 len=1375086 2024-04-07T16:58:07.539 INFO:tasks.workunit.client.0.smithi100.stdout:op 5882 completed, throughput=5MB/sec 2024-04-07T16:58:07.539 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-XsfrFO2hM6TpQQ7 off=215949 len=1698273 2024-04-07T16:58:07.543 INFO:tasks.workunit.client.0.smithi100.stdout:op 5883 completed, throughput=5MB/sec 2024-04-07T16:58:08.544 INFO:tasks.workunit.client.0.smithi100.stdout: 1134: throughput=5MB/sec pending data=0 2024-04-07T16:58:08.544 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-j8BXRVGr7qawqVI off=12074614 len=202755 2024-04-07T16:58:08.550 INFO:tasks.workunit.client.0.smithi100.stdout:op 5884 completed, throughput=5MB/sec 2024-04-07T16:58:08.550 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-iDBvvSPhfFNf1Ly off=7849732 len=824010 2024-04-07T16:58:08.552 INFO:tasks.workunit.client.0.smithi100.stdout:op 5885 completed, throughput=5MB/sec 2024-04-07T16:58:08.552 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-155kWxOl9LGYjvO off=17832060 len=1258884 2024-04-07T16:58:08.556 INFO:tasks.workunit.client.0.smithi100.stdout:op 5886 completed, throughput=5MB/sec 2024-04-07T16:58:08.556 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-sTXtATRTwuHVWf1 off=5975648 len=1119583 2024-04-07T16:58:08.565 INFO:tasks.workunit.client.0.smithi100.stdout:op 5887 completed, throughput=5MB/sec 2024-04-07T16:58:08.566 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Qi2ysdvCNiGcrL1 off=3533576 len=134115 2024-04-07T16:58:08.571 INFO:tasks.workunit.client.0.smithi100.stdout:op 5888 completed, throughput=5MB/sec 2024-04-07T16:58:08.571 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-KiDHuc0Q6vG1d2D off=204761 len=437751 2024-04-07T16:58:08.572 INFO:tasks.workunit.client.0.smithi100.stdout:op 5889 completed, throughput=5MB/sec 2024-04-07T16:58:08.572 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-WB9pd9reCLTA8Ad off=311832 len=454362 2024-04-07T16:58:08.574 INFO:tasks.workunit.client.0.smithi100.stdout:op 5890 completed, throughput=5MB/sec 2024-04-07T16:58:08.699 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:58:08 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:58:08] "GET /metrics HTTP/1.1" 200 34071 "" "Prometheus/2.43.0" 2024-04-07T16:58:09.113 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:58:08 smithi144 bash[66654]: cluster 2024-04-07T16:58:07.668406+0000 mgr.y (mgr.25175) 1283 : cluster [DBG] pgmap v1925: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 3.2 MiB/s rd, 2.3 MiB/s wr, 4 op/s 2024-04-07T16:58:09.199 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:58:08 smithi100 bash[74942]: cluster 2024-04-07T16:58:07.668406+0000 mgr.y (mgr.25175) 1283 : cluster [DBG] pgmap v1925: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 3.2 MiB/s rd, 2.3 MiB/s wr, 4 op/s 2024-04-07T16:58:09.200 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:58:08 smithi100 bash[76202]: cluster 2024-04-07T16:58:07.668406+0000 mgr.y (mgr.25175) 1283 : cluster [DBG] pgmap v1925: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 3.2 MiB/s rd, 2.3 MiB/s wr, 4 op/s 2024-04-07T16:58:09.574 INFO:tasks.workunit.client.0.smithi100.stdout: 1135: throughput=5MB/sec pending data=0 2024-04-07T16:58:09.575 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-u1wxcGPS2Y79e2X off=2848062 len=417963 2024-04-07T16:58:09.581 INFO:tasks.workunit.client.0.smithi100.stdout:op 5891 completed, throughput=5MB/sec 2024-04-07T16:58:09.581 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-RngLj8GyfIpoYCw off=9020834 len=1072189 2024-04-07T16:58:09.591 INFO:tasks.workunit.client.0.smithi100.stdout:op 5892 completed, throughput=5MB/sec 2024-04-07T16:58:09.591 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-CQ2QXL8jgH5-SfJ off=1589367 len=184831 2024-04-07T16:58:09.592 INFO:tasks.workunit.client.0.smithi100.stdout:op 5893 completed, throughput=5MB/sec 2024-04-07T16:58:09.592 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-6B6KJZDpIJFCpXD off=4684941 len=1437088 2024-04-07T16:58:09.604 INFO:tasks.workunit.client.0.smithi100.stdout:op 5894 completed, throughput=5MB/sec 2024-04-07T16:58:09.604 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-ggLOZ-kz0j5Hdwg off=12263866 len=1216579 2024-04-07T16:58:09.615 INFO:tasks.workunit.client.0.smithi100.stdout:op 5895 completed, throughput=5MB/sec 2024-04-07T16:58:09.616 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-DYcEhRBeY3wrtE7 off=797526 len=943971 2024-04-07T16:58:09.620 INFO:tasks.workunit.client.0.smithi100.stdout:op 5896 completed, throughput=5MB/sec 2024-04-07T16:58:10.620 INFO:tasks.workunit.client.0.smithi100.stdout: 1136: throughput=5MB/sec pending data=0 2024-04-07T16:58:10.620 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-uquf6fM8wFl2EwR off=2380162 len=1040957 2024-04-07T16:58:10.623 INFO:tasks.workunit.client.0.smithi100.stdout:op 5897 completed, throughput=5MB/sec 2024-04-07T16:58:10.623 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-rL1jrmQCC2TlSc9 off=873779 len=1669582 2024-04-07T16:58:10.636 INFO:tasks.workunit.client.0.smithi100.stdout:op 5898 completed, throughput=5MB/sec 2024-04-07T16:58:10.636 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-wLtrrNnu0MFwoLW off=11053370 len=1079940 2024-04-07T16:58:10.647 INFO:tasks.workunit.client.0.smithi100.stdout:op 5899 completed, throughput=5MB/sec 2024-04-07T16:58:10.647 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-f4mNCDb35mqPVWj off=5973619 len=2062897 2024-04-07T16:58:10.649 INFO:tasks.workunit.client.0.smithi100.stdout:op 5900 completed, throughput=5MB/sec 2024-04-07T16:58:11.113 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:58:10 smithi144 bash[66654]: cluster 2024-04-07T16:58:09.669420+0000 mgr.y (mgr.25175) 1284 : cluster [DBG] pgmap v1926: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 2.9 MiB/s rd, 2.5 MiB/s wr, 4 op/s 2024-04-07T16:58:11.200 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:58:10 smithi100 bash[74942]: cluster 2024-04-07T16:58:09.669420+0000 mgr.y (mgr.25175) 1284 : cluster [DBG] pgmap v1926: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 2.9 MiB/s rd, 2.5 MiB/s wr, 4 op/s 2024-04-07T16:58:11.225 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:58:10 smithi100 bash[76202]: cluster 2024-04-07T16:58:09.669420+0000 mgr.y (mgr.25175) 1284 : cluster [DBG] pgmap v1926: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 2.9 MiB/s rd, 2.5 MiB/s wr, 4 op/s 2024-04-07T16:58:11.650 INFO:tasks.workunit.client.0.smithi100.stdout: 1137: throughput=5MB/sec pending data=0 2024-04-07T16:58:11.651 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-KWNN5RpKObwQ-BT off=249864 len=735603 2024-04-07T16:58:11.653 INFO:tasks.workunit.client.0.smithi100.stdout:op 5901 completed, throughput=5MB/sec 2024-04-07T16:58:11.653 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-DfZ96UZ73ntOiOw off=8535447 len=843945 2024-04-07T16:58:11.661 INFO:tasks.workunit.client.0.smithi100.stdout:op 5902 completed, throughput=5MB/sec 2024-04-07T16:58:11.661 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-ynS6XJfDFeG7mWI off=10861335 len=503111 2024-04-07T16:58:11.667 INFO:tasks.workunit.client.0.smithi100.stdout:op 5903 completed, throughput=5MB/sec 2024-04-07T16:58:11.667 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-oG6ZlJ-5DBXetZ0 off=159122 len=297633 2024-04-07T16:58:11.673 INFO:tasks.workunit.client.0.smithi100.stdout:op 5904 completed, throughput=5MB/sec 2024-04-07T16:58:11.673 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-8hUz0g1CAWk0a8b off=4152513 len=663445 2024-04-07T16:58:11.679 INFO:tasks.workunit.client.0.smithi100.stdout:op 5905 completed, throughput=5MB/sec 2024-04-07T16:58:11.679 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-n7zSfZBxlzGiuuB off=13364599 len=731927 2024-04-07T16:58:11.680 INFO:tasks.workunit.client.0.smithi100.stdout:op 5906 completed, throughput=5MB/sec 2024-04-07T16:58:11.680 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-rZSAegLUEoyoVYn off=12710133 len=1885410 2024-04-07T16:58:11.694 INFO:tasks.workunit.client.0.smithi100.stdout:op 5907 completed, throughput=5MB/sec 2024-04-07T16:58:12.113 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:58:11 smithi144 bash[66654]: audit 2024-04-07T16:58:11.272447+0000 mon.a (mon.0) 2211 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:58:12.199 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:58:11 smithi100 bash[74942]: audit 2024-04-07T16:58:11.272447+0000 mon.a (mon.0) 2211 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:58:12.199 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:58:11 smithi100 bash[76202]: audit 2024-04-07T16:58:11.272447+0000 mon.a (mon.0) 2211 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:58:12.613 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:58:12 smithi144 bash[95233]: ts=2024-04-07T16:58:12.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:58:12.694 INFO:tasks.workunit.client.0.smithi100.stdout: 1138: throughput=5MB/sec pending data=0 2024-04-07T16:58:12.694 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-F1zbZHlxFVaEQHq off=4257998 len=1990187 2024-04-07T16:58:12.708 INFO:tasks.workunit.client.0.smithi100.stdout:op 5908 completed, throughput=5MB/sec 2024-04-07T16:58:12.708 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-K2b2l9_P1NrfRWI off=16493668 len=865831 2024-04-07T16:58:12.711 INFO:tasks.workunit.client.0.smithi100.stdout:op 5909 completed, throughput=5MB/sec 2024-04-07T16:58:12.711 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-kxk2uHJvrM2jh3W off=12367772 len=1883655 2024-04-07T16:58:12.723 INFO:tasks.workunit.client.0.smithi100.stdout:op 5910 completed, throughput=5MB/sec 2024-04-07T16:58:13.113 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:58:12 smithi144 bash[66654]: cluster 2024-04-07T16:58:11.670150+0000 mgr.y (mgr.25175) 1285 : cluster [DBG] pgmap v1927: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 2.9 MiB/s rd, 2.4 MiB/s wr, 5 op/s 2024-04-07T16:58:13.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:58:12 smithi144 bash[66654]: audit 2024-04-07T16:58:12.417226+0000 mon.a (mon.0) 2212 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:58:13.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:58:12 smithi144 bash[66654]: audit 2024-04-07T16:58:12.427467+0000 mon.a (mon.0) 2213 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:58:13.199 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:58:12 smithi100 bash[74942]: cluster 2024-04-07T16:58:11.670150+0000 mgr.y (mgr.25175) 1285 : cluster [DBG] pgmap v1927: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 2.9 MiB/s rd, 2.4 MiB/s wr, 5 op/s 2024-04-07T16:58:13.199 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:58:12 smithi100 bash[74942]: audit 2024-04-07T16:58:12.417226+0000 mon.a (mon.0) 2212 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:58:13.200 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:58:12 smithi100 bash[74942]: audit 2024-04-07T16:58:12.427467+0000 mon.a (mon.0) 2213 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:58:13.200 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:58:12 smithi100 bash[76202]: cluster 2024-04-07T16:58:11.670150+0000 mgr.y (mgr.25175) 1285 : cluster [DBG] pgmap v1927: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 2.9 MiB/s rd, 2.4 MiB/s wr, 5 op/s 2024-04-07T16:58:13.200 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:58:12 smithi100 bash[76202]: audit 2024-04-07T16:58:12.417226+0000 mon.a (mon.0) 2212 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:58:13.200 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:58:12 smithi100 bash[76202]: audit 2024-04-07T16:58:12.427467+0000 mon.a (mon.0) 2213 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:58:13.724 INFO:tasks.workunit.client.0.smithi100.stdout: 1139: throughput=5MB/sec pending data=0 2024-04-07T16:58:13.724 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-y8WQBLPW7wN4cg_ off=9220409 len=1383263 2024-04-07T16:58:13.734 INFO:tasks.workunit.client.0.smithi100.stdout:op 5911 completed, throughput=5MB/sec 2024-04-07T16:58:13.734 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-3GAsP7Ys5BnFY8O off=6384153 len=150180 2024-04-07T16:58:13.739 INFO:tasks.workunit.client.0.smithi100.stdout:op 5912 completed, throughput=5MB/sec 2024-04-07T16:58:13.739 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-2YvgXQQuZta7V0y off=1486761 len=557220 2024-04-07T16:58:13.746 INFO:tasks.workunit.client.0.smithi100.stdout:op 5913 completed, throughput=5MB/sec 2024-04-07T16:58:13.747 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Oc02j8ytI_bbI90 off=10252717 len=338772 2024-04-07T16:58:13.754 INFO:tasks.workunit.client.0.smithi100.stdout:op 5914 completed, throughput=5MB/sec 2024-04-07T16:58:13.754 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-yGayoxRl9I1iHxx off=3638788 len=253132 2024-04-07T16:58:13.755 INFO:tasks.workunit.client.0.smithi100.stdout:op 5915 completed, throughput=5MB/sec 2024-04-07T16:58:13.755 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-PT1ZojGLuSLD8dK off=1016797 len=302846 2024-04-07T16:58:13.757 INFO:tasks.workunit.client.0.smithi100.stdout:op 5916 completed, throughput=5MB/sec 2024-04-07T16:58:13.757 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-_oKQWsNxXm1nmp7 off=4729520 len=1950725 2024-04-07T16:58:13.762 INFO:tasks.workunit.client.0.smithi100.stdout:op 5917 completed, throughput=5MB/sec 2024-04-07T16:58:13.762 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-4XmKuyBlQvXHlXJ off=7524491 len=800465 2024-04-07T16:58:13.765 INFO:tasks.workunit.client.0.smithi100.stdout:op 5918 completed, throughput=5MB/sec 2024-04-07T16:58:14.113 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:58:13 smithi144 bash[66654]: cluster 2024-04-07T16:58:13.671266+0000 mgr.y (mgr.25175) 1286 : cluster [DBG] pgmap v1928: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 2.6 MiB/s rd, 2.9 MiB/s wr, 5 op/s 2024-04-07T16:58:14.199 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:58:13 smithi100 bash[74942]: cluster 2024-04-07T16:58:13.671266+0000 mgr.y (mgr.25175) 1286 : cluster [DBG] pgmap v1928: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 2.6 MiB/s rd, 2.9 MiB/s wr, 5 op/s 2024-04-07T16:58:14.200 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:58:13 smithi100 bash[76202]: cluster 2024-04-07T16:58:13.671266+0000 mgr.y (mgr.25175) 1286 : cluster [DBG] pgmap v1928: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 2.6 MiB/s rd, 2.9 MiB/s wr, 5 op/s 2024-04-07T16:58:14.765 INFO:tasks.workunit.client.0.smithi100.stdout: 1140: throughput=5MB/sec pending data=0 2024-04-07T16:58:14.766 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-HP24jJ-Go7uI26k off=673091 len=2022267 2024-04-07T16:58:14.782 INFO:tasks.workunit.client.0.smithi100.stdout:op 5919 completed, throughput=5MB/sec 2024-04-07T16:58:14.782 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-fUbsotUFKVz-rv_ off=693489 len=815097 2024-04-07T16:58:14.790 INFO:tasks.workunit.client.0.smithi100.stdout:op 5920 completed, throughput=5MB/sec 2024-04-07T16:58:14.791 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-HGvUrh-FnZJFrkf off=7683607 len=2022903 2024-04-07T16:58:14.795 INFO:tasks.workunit.client.0.smithi100.stdout:op 5921 completed, throughput=5MB/sec 2024-04-07T16:58:14.795 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj--BEgPv0mrQ6Sn4f off=4432617 len=1937772 2024-04-07T16:58:14.797 INFO:tasks.workunit.client.0.smithi100.stdout:op 5922 completed, throughput=5MB/sec 2024-04-07T16:58:15.798 INFO:tasks.workunit.client.0.smithi100.stdout: 1141: throughput=5MB/sec pending data=0 2024-04-07T16:58:15.798 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-inT8O5usHU5nfNL off=1826713 len=1500158 2024-04-07T16:58:15.802 INFO:tasks.workunit.client.0.smithi100.stdout:op 5923 completed, throughput=5MB/sec 2024-04-07T16:58:15.802 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-eBpYCu9E_kkUCAU off=7361088 len=1673863 2024-04-07T16:58:15.814 INFO:tasks.workunit.client.0.smithi100.stdout:op 5924 completed, throughput=5MB/sec 2024-04-07T16:58:15.814 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-8wfONl6v4d7WQKL off=3871899 len=1157881 2024-04-07T16:58:15.816 INFO:tasks.workunit.client.0.smithi100.stdout:op 5925 completed, throughput=5MB/sec 2024-04-07T16:58:16.817 INFO:tasks.workunit.client.0.smithi100.stdout: 1142: throughput=5MB/sec pending data=0 2024-04-07T16:58:16.817 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-k7bSd33JbUmyit6 off=5456089 len=761595 2024-04-07T16:58:16.825 INFO:tasks.workunit.client.0.smithi100.stdout:op 5926 completed, throughput=5MB/sec 2024-04-07T16:58:16.825 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-W267mKfx74l2HwA off=915475 len=940962 2024-04-07T16:58:16.834 INFO:tasks.workunit.client.0.smithi100.stdout:op 5927 completed, throughput=5MB/sec 2024-04-07T16:58:16.834 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-4ktRoaPsEd05t-3 off=222875 len=1379082 2024-04-07T16:58:16.837 INFO:tasks.workunit.client.0.smithi100.stdout:op 5928 completed, throughput=5MB/sec 2024-04-07T16:58:16.838 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Jd5cBIQ4IbedES9 off=6406074 len=753394 2024-04-07T16:58:16.845 INFO:tasks.workunit.client.0.smithi100.stdout:op 5929 completed, throughput=5MB/sec 2024-04-07T16:58:16.846 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-8IFLLDKqQZh0sDi off=7443809 len=1451532 2024-04-07T16:58:16.857 INFO:tasks.workunit.client.0.smithi100.stdout:op 5930 completed, throughput=5MB/sec 2024-04-07T16:58:17.113 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:58:16 smithi144 bash[66654]: cluster 2024-04-07T16:58:15.672281+0000 mgr.y (mgr.25175) 1287 : cluster [DBG] pgmap v1929: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 2.9 MiB/s rd, 2.3 MiB/s wr, 5 op/s 2024-04-07T16:58:17.199 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:58:16 smithi100 bash[74942]: cluster 2024-04-07T16:58:15.672281+0000 mgr.y (mgr.25175) 1287 : cluster [DBG] pgmap v1929: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 2.9 MiB/s rd, 2.3 MiB/s wr, 5 op/s 2024-04-07T16:58:17.200 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:58:16 smithi100 bash[76202]: cluster 2024-04-07T16:58:15.672281+0000 mgr.y (mgr.25175) 1287 : cluster [DBG] pgmap v1929: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 2.9 MiB/s rd, 2.3 MiB/s wr, 5 op/s 2024-04-07T16:58:17.858 INFO:tasks.workunit.client.0.smithi100.stdout: 1143: throughput=5MB/sec pending data=0 2024-04-07T16:58:17.858 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-UDXeVJ2uu1lyqqB off=3077484 len=1381143 2024-04-07T16:58:17.860 INFO:tasks.workunit.client.0.smithi100.stdout:op 5931 completed, throughput=5MB/sec 2024-04-07T16:58:17.861 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-HiHh7EHacx7D55h off=3396545 len=1256320 2024-04-07T16:58:17.865 INFO:tasks.workunit.client.0.smithi100.stdout:op 5932 completed, throughput=5MB/sec 2024-04-07T16:58:17.865 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-1hzPVOeFQ8DndFb off=18475771 len=1379182 2024-04-07T16:58:17.877 INFO:tasks.workunit.client.0.smithi100.stdout:op 5933 completed, throughput=5MB/sec 2024-04-07T16:58:17.877 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-tLNQYETV9X5AEzx off=6898720 len=455764 2024-04-07T16:58:17.879 INFO:tasks.workunit.client.0.smithi100.stdout:op 5934 completed, throughput=5MB/sec 2024-04-07T16:58:17.879 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-3x6UPByTGRf5MKg off=9095891 len=497972 2024-04-07T16:58:17.885 INFO:tasks.workunit.client.0.smithi100.stdout:op 5935 completed, throughput=5MB/sec 2024-04-07T16:58:18.113 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:58:17 smithi144 bash[66654]: audit 2024-04-07T16:58:16.790322+0000 mon.a (mon.0) 2214 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:58:18.199 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:58:17 smithi100 bash[74942]: audit 2024-04-07T16:58:16.790322+0000 mon.a (mon.0) 2214 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:58:18.200 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:58:17 smithi100 bash[76202]: audit 2024-04-07T16:58:16.790322+0000 mon.a (mon.0) 2214 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:58:18.685 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:58:18 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:58:18] "GET /metrics HTTP/1.1" 200 34075 "" "Prometheus/2.43.0" 2024-04-07T16:58:18.886 INFO:tasks.workunit.client.0.smithi100.stdout: 1144: throughput=5MB/sec pending data=0 2024-04-07T16:58:18.886 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-LzOQ-6P0HCv37DF off=3022455 len=389561 2024-04-07T16:58:18.893 INFO:tasks.workunit.client.0.smithi100.stdout:op 5936 completed, throughput=5MB/sec 2024-04-07T16:58:18.893 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-_9NBRNgAc7bcwst off=3561820 len=1342093 2024-04-07T16:58:18.904 INFO:tasks.workunit.client.0.smithi100.stdout:op 5937 completed, throughput=5MB/sec 2024-04-07T16:58:18.904 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-BxA-fiUZmL6o2vi off=13927446 len=1529145 2024-04-07T16:58:18.908 INFO:tasks.workunit.client.0.smithi100.stdout:op 5938 completed, throughput=5MB/sec 2024-04-07T16:58:18.908 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-lPyeN6Hla4Gp2fS off=8417914 len=95897 2024-04-07T16:58:18.913 INFO:tasks.workunit.client.0.smithi100.stdout:op 5939 completed, throughput=5MB/sec 2024-04-07T16:58:18.913 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-WTjXYTDL5Xtsz2o off=8334549 len=243775 2024-04-07T16:58:18.914 INFO:tasks.workunit.client.0.smithi100.stdout:op 5940 completed, throughput=5MB/sec 2024-04-07T16:58:18.914 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-5VcWXbDpY8fjXz7 off=2422715 len=482307 2024-04-07T16:58:18.916 INFO:tasks.workunit.client.0.smithi100.stdout:op 5941 completed, throughput=5MB/sec 2024-04-07T16:58:18.916 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-AO2pPsXU6ncnPMm off=1343003 len=1654350 2024-04-07T16:58:18.920 INFO:tasks.workunit.client.0.smithi100.stdout:op 5942 completed, throughput=5MB/sec 2024-04-07T16:58:18.949 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:58:18 smithi100 bash[74942]: cluster 2024-04-07T16:58:17.673255+0000 mgr.y (mgr.25175) 1288 : cluster [DBG] pgmap v1930: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.6 MiB/s wr, 5 op/s 2024-04-07T16:58:18.950 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:58:18 smithi100 bash[76202]: cluster 2024-04-07T16:58:17.673255+0000 mgr.y (mgr.25175) 1288 : cluster [DBG] pgmap v1930: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.6 MiB/s wr, 5 op/s 2024-04-07T16:58:19.113 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:58:18 smithi144 bash[66654]: cluster 2024-04-07T16:58:17.673255+0000 mgr.y (mgr.25175) 1288 : cluster [DBG] pgmap v1930: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.6 MiB/s wr, 5 op/s 2024-04-07T16:58:19.114 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 16:58:18 smithi144 bash[98866]: logger=cleanup t=2024-04-07T16:58:18.851542586Z level=info msg="Completed cleanup jobs" duration=432.060786ms 2024-04-07T16:58:19.921 INFO:tasks.workunit.client.0.smithi100.stdout: 1145: throughput=5MB/sec pending data=0 2024-04-07T16:58:19.921 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-LB1sTDJmiwjTgkB off=1639135 len=80805 2024-04-07T16:58:19.922 INFO:tasks.workunit.client.0.smithi100.stdout:op 5943 completed, throughput=5MB/sec 2024-04-07T16:58:19.922 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-jp9-f5URB15sAZE off=1472667 len=2017087 2024-04-07T16:58:19.926 INFO:tasks.workunit.client.0.smithi100.stdout:op 5944 completed, throughput=5MB/sec 2024-04-07T16:58:19.926 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-MGN3ZE-d1TARFzH off=3680441 len=44010 2024-04-07T16:58:19.927 INFO:tasks.workunit.client.0.smithi100.stdout:op 5945 completed, throughput=5MB/sec 2024-04-07T16:58:19.927 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Y5j9XibrWS59grP off=4364032 len=336706 2024-04-07T16:58:19.929 INFO:tasks.workunit.client.0.smithi100.stdout:op 5946 completed, throughput=5MB/sec 2024-04-07T16:58:19.929 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-RemyBWipgyFlBhW off=452749 len=786420 2024-04-07T16:58:19.938 INFO:tasks.workunit.client.0.smithi100.stdout:op 5947 completed, throughput=5MB/sec 2024-04-07T16:58:19.938 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-XG_BMHAtTt1GLWb off=3039456 len=1939374 2024-04-07T16:58:19.953 INFO:tasks.workunit.client.0.smithi100.stdout:op 5948 completed, throughput=5MB/sec 2024-04-07T16:58:20.954 INFO:tasks.workunit.client.0.smithi100.stdout: 1146: throughput=5MB/sec pending data=0 2024-04-07T16:58:20.954 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-skygdTcyLgo-BTB off=3370623 len=47094 2024-04-07T16:58:20.954 INFO:tasks.workunit.client.0.smithi100.stdout:op 5949 completed, throughput=5MB/sec 2024-04-07T16:58:20.954 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-SQ7jZTIXSbGP69r off=8639173 len=331047 2024-04-07T16:58:20.955 INFO:tasks.workunit.client.0.smithi100.stdout:op 5950 completed, throughput=5MB/sec 2024-04-07T16:58:20.956 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-R6yJtGlv0IdQbsg off=5569238 len=1912288 2024-04-07T16:58:20.968 INFO:tasks.workunit.client.0.smithi100.stdout:op 5951 completed, throughput=5MB/sec 2024-04-07T16:58:20.968 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-LTJSlG5gFBCTmF8 off=9198309 len=1643657 2024-04-07T16:58:20.970 INFO:tasks.workunit.client.0.smithi100.stdout:op 5952 completed, throughput=5MB/sec 2024-04-07T16:58:20.970 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-JXiqgTi3JO3fbY2 off=10823574 len=165178 2024-04-07T16:58:20.971 INFO:tasks.workunit.client.0.smithi100.stdout:op 5953 completed, throughput=5MB/sec 2024-04-07T16:58:20.972 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-oAPz6nu22PexoAq off=16838384 len=1523807 2024-04-07T16:58:20.973 INFO:tasks.workunit.client.0.smithi100.stdout:op 5954 completed, throughput=5MB/sec 2024-04-07T16:58:21.113 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:58:20 smithi144 bash[66654]: cluster 2024-04-07T16:58:19.674239+0000 mgr.y (mgr.25175) 1289 : cluster [DBG] pgmap v1931: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1.8 MiB/s rd, 2.6 MiB/s wr, 4 op/s 2024-04-07T16:58:21.142 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:58:20 smithi100 bash[76202]: cluster 2024-04-07T16:58:19.674239+0000 mgr.y (mgr.25175) 1289 : cluster [DBG] pgmap v1931: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1.8 MiB/s rd, 2.6 MiB/s wr, 4 op/s 2024-04-07T16:58:21.143 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:58:20 smithi100 bash[74942]: cluster 2024-04-07T16:58:19.674239+0000 mgr.y (mgr.25175) 1289 : cluster [DBG] pgmap v1931: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1.8 MiB/s rd, 2.6 MiB/s wr, 4 op/s 2024-04-07T16:58:21.974 INFO:tasks.workunit.client.0.smithi100.stdout: 1147: throughput=5MB/sec pending data=0 2024-04-07T16:58:21.974 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-_YwPzIf3U9nSlzM off=1571788 len=1708876 2024-04-07T16:58:21.989 INFO:tasks.workunit.client.0.smithi100.stdout:op 5955 completed, throughput=5MB/sec 2024-04-07T16:58:21.989 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-rJ0CK5vEscHhWMR off=1251818 len=1295882 2024-04-07T16:58:21.997 INFO:tasks.workunit.client.0.smithi100.stdout:op 5956 completed, throughput=5MB/sec 2024-04-07T16:58:21.998 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Mro8frTK3nLbT6M off=321297 len=1396155 2024-04-07T16:58:22.009 INFO:tasks.workunit.client.0.smithi100.stdout:op 5957 completed, throughput=5MB/sec 2024-04-07T16:58:22.009 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-c65XicjtU0lv9Xw off=4780855 len=1355901 2024-04-07T16:58:22.010 INFO:tasks.workunit.client.0.smithi100.stdout:op 5958 completed, throughput=5MB/sec 2024-04-07T16:58:22.613 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:58:22 smithi144 bash[95233]: ts=2024-04-07T16:58:22.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:58:23.009 INFO:tasks.workunit.client.0.smithi100.stdout: 1148: throughput=5MB/sec pending data=0 2024-04-07T16:58:23.009 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-fdtvVJpJ6d67Lyv off=4465433 len=2091224 2024-04-07T16:58:23.024 INFO:tasks.workunit.client.0.smithi100.stdout:op 5959 completed, throughput=5MB/sec 2024-04-07T16:58:23.024 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Be0DnOufEUXwYav off=8116237 len=79259 2024-04-07T16:58:23.028 INFO:tasks.workunit.client.0.smithi100.stdout:op 5960 completed, throughput=5MB/sec 2024-04-07T16:58:23.028 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-dc1AZON0xgCKJC2 off=10070100 len=1205456 2024-04-07T16:58:23.031 INFO:tasks.workunit.client.0.smithi100.stdout:op 5961 completed, throughput=5MB/sec 2024-04-07T16:58:23.031 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-MjnQTq8vio_F6pQ off=4285130 len=1550551 2024-04-07T16:58:23.044 INFO:tasks.workunit.client.0.smithi100.stdout:op 5962 completed, throughput=5MB/sec 2024-04-07T16:58:23.113 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:58:22 smithi144 bash[66654]: cluster 2024-04-07T16:58:21.675082+0000 mgr.y (mgr.25175) 1290 : cluster [DBG] pgmap v1932: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 2.3 MiB/s rd, 2.5 MiB/s wr, 4 op/s 2024-04-07T16:58:23.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:58:22 smithi144 bash[66654]: audit 2024-04-07T16:58:21.935789+0000 mon.a (mon.0) 2215 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:58:23.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:58:22 smithi144 bash[66654]: audit 2024-04-07T16:58:21.949224+0000 mon.a (mon.0) 2216 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:58:23.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:58:22 smithi144 bash[66654]: audit 2024-04-07T16:58:21.951652+0000 mon.a (mon.0) 2217 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:58:23.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:58:22 smithi144 bash[66654]: audit 2024-04-07T16:58:21.953579+0000 mon.a (mon.0) 2218 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:58:23.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:58:22 smithi144 bash[66654]: audit 2024-04-07T16:58:21.966182+0000 mon.a (mon.0) 2219 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:58:23.199 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:58:22 smithi100 bash[74942]: cluster 2024-04-07T16:58:21.675082+0000 mgr.y (mgr.25175) 1290 : cluster [DBG] pgmap v1932: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 2.3 MiB/s rd, 2.5 MiB/s wr, 4 op/s 2024-04-07T16:58:23.200 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:58:22 smithi100 bash[74942]: audit 2024-04-07T16:58:21.935789+0000 mon.a (mon.0) 2215 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:58:23.200 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:58:22 smithi100 bash[74942]: audit 2024-04-07T16:58:21.949224+0000 mon.a (mon.0) 2216 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:58:23.200 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:58:22 smithi100 bash[74942]: audit 2024-04-07T16:58:21.951652+0000 mon.a (mon.0) 2217 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:58:23.200 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:58:22 smithi100 bash[74942]: audit 2024-04-07T16:58:21.953579+0000 mon.a (mon.0) 2218 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:58:23.200 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:58:22 smithi100 bash[74942]: audit 2024-04-07T16:58:21.966182+0000 mon.a (mon.0) 2219 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:58:23.200 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:58:22 smithi100 bash[76202]: cluster 2024-04-07T16:58:21.675082+0000 mgr.y (mgr.25175) 1290 : cluster [DBG] pgmap v1932: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 2.3 MiB/s rd, 2.5 MiB/s wr, 4 op/s 2024-04-07T16:58:23.200 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:58:22 smithi100 bash[76202]: audit 2024-04-07T16:58:21.935789+0000 mon.a (mon.0) 2215 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:58:23.200 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:58:22 smithi100 bash[76202]: audit 2024-04-07T16:58:21.949224+0000 mon.a (mon.0) 2216 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:58:23.200 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:58:22 smithi100 bash[76202]: audit 2024-04-07T16:58:21.951652+0000 mon.a (mon.0) 2217 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:58:23.200 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:58:22 smithi100 bash[76202]: audit 2024-04-07T16:58:21.953579+0000 mon.a (mon.0) 2218 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:58:23.200 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:58:22 smithi100 bash[76202]: audit 2024-04-07T16:58:21.966182+0000 mon.a (mon.0) 2219 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:58:24.045 INFO:tasks.workunit.client.0.smithi100.stdout: 1149: throughput=5MB/sec pending data=0 2024-04-07T16:58:24.045 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-g2nSNOL6kfbvp45 off=4147384 len=1884372 2024-04-07T16:58:24.058 INFO:tasks.workunit.client.0.smithi100.stdout:op 5963 completed, throughput=5MB/sec 2024-04-07T16:58:24.058 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-ldMSRYNM7LmAsSo off=5142719 len=1401346 2024-04-07T16:58:24.068 INFO:tasks.workunit.client.0.smithi100.stdout:op 5964 completed, throughput=5MB/sec 2024-04-07T16:58:24.068 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-4q-DwkG71fMXb8O off=15307740 len=1281343 2024-04-07T16:58:24.080 INFO:tasks.workunit.client.0.smithi100.stdout:op 5965 completed, throughput=5MB/sec 2024-04-07T16:58:24.080 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-uVQ8UFE-_sW-Qo5 off=6270334 len=1179622 2024-04-07T16:58:24.083 INFO:tasks.workunit.client.0.smithi100.stdout:op 5966 completed, throughput=5MB/sec 2024-04-07T16:58:25.084 INFO:tasks.workunit.client.0.smithi100.stdout: 1150: throughput=5MB/sec pending data=0 2024-04-07T16:58:25.084 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-PcDYCxNdahEvtPG off=1694012 len=175813 2024-04-07T16:58:25.089 INFO:tasks.workunit.client.0.smithi100.stdout:op 5967 completed, throughput=5MB/sec 2024-04-07T16:58:25.089 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-1qonWpDTcLGOshD off=3120979 len=1631100 2024-04-07T16:58:25.102 INFO:tasks.workunit.client.0.smithi100.stdout:op 5968 completed, throughput=5MB/sec 2024-04-07T16:58:25.102 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-3or1dLM6J4TTY0G off=21349324 len=549154 2024-04-07T16:58:25.105 INFO:tasks.workunit.client.0.smithi100.stdout:op 5969 completed, throughput=5MB/sec 2024-04-07T16:58:25.105 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-gVsCyilB1dNIDl2 off=0 len=1272429 2024-04-07T16:58:25.109 INFO:tasks.workunit.client.0.smithi100.stdout:op 5970 completed, throughput=5MB/sec 2024-04-07T16:58:25.109 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-hK6VJV8oMMCs8Hf off=11532493 len=1915442 2024-04-07T16:58:25.113 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:58:24 smithi144 bash[66654]: cluster 2024-04-07T16:58:23.676236+0000 mgr.y (mgr.25175) 1291 : cluster [DBG] pgmap v1933: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 2.3 MiB/s rd, 2.8 MiB/s wr, 5 op/s 2024-04-07T16:58:25.125 INFO:tasks.workunit.client.0.smithi100.stdout:op 5971 completed, throughput=5MB/sec 2024-04-07T16:58:25.199 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:58:24 smithi100 bash[74942]: cluster 2024-04-07T16:58:23.676236+0000 mgr.y (mgr.25175) 1291 : cluster [DBG] pgmap v1933: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 2.3 MiB/s rd, 2.8 MiB/s wr, 5 op/s 2024-04-07T16:58:25.200 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:58:24 smithi100 bash[76202]: cluster 2024-04-07T16:58:23.676236+0000 mgr.y (mgr.25175) 1291 : cluster [DBG] pgmap v1933: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 2.3 MiB/s rd, 2.8 MiB/s wr, 5 op/s 2024-04-07T16:58:26.113 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:58:25 smithi144 bash[66654]: cluster 2024-04-07T16:58:25.677265+0000 mgr.y (mgr.25175) 1292 : cluster [DBG] pgmap v1934: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.2 MiB/s wr, 4 op/s 2024-04-07T16:58:26.125 INFO:tasks.workunit.client.0.smithi100.stdout: 1151: throughput=5MB/sec pending data=0 2024-04-07T16:58:26.125 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-dLoJIOESz_d2Xgn off=7969461 len=1187710 2024-04-07T16:58:26.136 INFO:tasks.workunit.client.0.smithi100.stdout:op 5972 completed, throughput=5MB/sec 2024-04-07T16:58:26.136 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-KlsPWUaMn3S5FML off=2099478 len=272528 2024-04-07T16:58:26.142 INFO:tasks.workunit.client.0.smithi100.stdout:op 5973 completed, throughput=5MB/sec 2024-04-07T16:58:26.142 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Mh8kC1KNiG3V9go off=1891699 len=1927863 2024-04-07T16:58:26.157 INFO:tasks.workunit.client.0.smithi100.stdout:op 5974 completed, throughput=5MB/sec 2024-04-07T16:58:26.157 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-oeURkVsuq8Jpmjj off=1508996 len=1669944 2024-04-07T16:58:26.176 INFO:tasks.workunit.client.0.smithi100.stdout:op 5975 completed, throughput=5MB/sec 2024-04-07T16:58:26.199 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:58:25 smithi100 bash[74942]: cluster 2024-04-07T16:58:25.677265+0000 mgr.y (mgr.25175) 1292 : cluster [DBG] pgmap v1934: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.2 MiB/s wr, 4 op/s 2024-04-07T16:58:26.200 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:58:25 smithi100 bash[76202]: cluster 2024-04-07T16:58:25.677265+0000 mgr.y (mgr.25175) 1292 : cluster [DBG] pgmap v1934: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.2 MiB/s wr, 4 op/s 2024-04-07T16:58:27.176 INFO:tasks.workunit.client.0.smithi100.stdout: 1152: throughput=5MB/sec pending data=0 2024-04-07T16:58:27.176 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-sNdMIEn_IZm9dO6 off=3052612 len=723695 2024-04-07T16:58:27.179 INFO:tasks.workunit.client.0.smithi100.stdout:op 5976 completed, throughput=5MB/sec 2024-04-07T16:58:27.179 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-VWxzKJfEhOOcCMa off=1341381 len=1595462 2024-04-07T16:58:27.193 INFO:tasks.workunit.client.0.smithi100.stdout:op 5977 completed, throughput=5MB/sec 2024-04-07T16:58:27.193 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-jLjG9h9TdkZChBe off=2560248 len=1363498 2024-04-07T16:58:27.204 INFO:tasks.workunit.client.0.smithi100.stdout:op 5978 completed, throughput=5MB/sec 2024-04-07T16:58:27.204 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-zbgliWXc1tMEyvs off=5026968 len=1777819 2024-04-07T16:58:27.208 INFO:tasks.workunit.client.0.smithi100.stdout:op 5979 completed, throughput=5MB/sec 2024-04-07T16:58:28.209 INFO:tasks.workunit.client.0.smithi100.stdout: 1153: throughput=5MB/sec pending data=0 2024-04-07T16:58:28.209 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-ovU2OiSAzqyDI7b off=7589662 len=469113 2024-04-07T16:58:28.216 INFO:tasks.workunit.client.0.smithi100.stdout:op 5980 completed, throughput=5MB/sec 2024-04-07T16:58:28.216 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-fniGFn6VX1tg0Yh off=4593275 len=1829931 2024-04-07T16:58:28.220 INFO:tasks.workunit.client.0.smithi100.stdout:op 5981 completed, throughput=5MB/sec 2024-04-07T16:58:28.220 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-RRw710aZd8PleQP off=16803562 len=26097 2024-04-07T16:58:28.225 INFO:tasks.workunit.client.0.smithi100.stdout:op 5982 completed, throughput=5MB/sec 2024-04-07T16:58:28.225 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-ds6JlQrAvLprMLh off=4453284 len=955364 2024-04-07T16:58:28.228 INFO:tasks.workunit.client.0.smithi100.stdout:op 5983 completed, throughput=5MB/sec 2024-04-07T16:58:28.229 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-dfJhU00bX8eeL_c off=21990381 len=72760 2024-04-07T16:58:28.230 INFO:tasks.workunit.client.0.smithi100.stdout:op 5984 completed, throughput=5MB/sec 2024-04-07T16:58:28.230 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-GiPyL5bLdna0agD off=2372863 len=1421666 2024-04-07T16:58:28.239 INFO:tasks.workunit.client.0.smithi100.stdout:op 5985 completed, throughput=5MB/sec 2024-04-07T16:58:28.239 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-2ylcXwEfoKBETex off=2924768 len=108603 2024-04-07T16:58:28.244 INFO:tasks.workunit.client.0.smithi100.stdout:op 5986 completed, throughput=5MB/sec 2024-04-07T16:58:28.244 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-xm-yYdkRU4_2sUp off=13540992 len=326526 2024-04-07T16:58:28.249 INFO:tasks.workunit.client.0.smithi100.stdout:op 5987 completed, throughput=5MB/sec 2024-04-07T16:58:28.699 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:58:28 smithi100 bash[74942]: cluster 2024-04-07T16:58:27.678336+0000 mgr.y (mgr.25175) 1293 : cluster [DBG] pgmap v1935: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1.8 MiB/s rd, 2.6 MiB/s wr, 4 op/s 2024-04-07T16:58:28.700 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:58:28 smithi100 bash[76202]: cluster 2024-04-07T16:58:27.678336+0000 mgr.y (mgr.25175) 1293 : cluster [DBG] pgmap v1935: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1.8 MiB/s rd, 2.6 MiB/s wr, 4 op/s 2024-04-07T16:58:28.700 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:58:28 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:58:28] "GET /metrics HTTP/1.1" 200 34075 "" "Prometheus/2.43.0" 2024-04-07T16:58:28.863 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:58:28 smithi144 bash[66654]: cluster 2024-04-07T16:58:27.678336+0000 mgr.y (mgr.25175) 1293 : cluster [DBG] pgmap v1935: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1.8 MiB/s rd, 2.6 MiB/s wr, 4 op/s 2024-04-07T16:58:29.249 INFO:tasks.workunit.client.0.smithi100.stdout: 1154: throughput=5MB/sec pending data=0 2024-04-07T16:58:29.249 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-c3gLl6HzY6e2yuV off=4692638 len=1338927 2024-04-07T16:58:29.254 INFO:tasks.workunit.client.0.smithi100.stdout:op 5988 completed, throughput=5MB/sec 2024-04-07T16:58:29.254 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-1beE2yR2gksNAH5 off=11893584 len=1246042 2024-04-07T16:58:29.264 INFO:tasks.workunit.client.0.smithi100.stdout:op 5989 completed, throughput=5MB/sec 2024-04-07T16:58:29.264 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-JBE_L2PABoUw-KJ off=6181991 len=1778424 2024-04-07T16:58:29.269 INFO:tasks.workunit.client.0.smithi100.stdout:op 5990 completed, throughput=5MB/sec 2024-04-07T16:58:29.269 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-bd2ieSx9EDwDzh4 off=9156591 len=1365290 2024-04-07T16:58:29.279 INFO:tasks.workunit.client.0.smithi100.stdout:op 5991 completed, throughput=5MB/sec 2024-04-07T16:58:30.280 INFO:tasks.workunit.client.0.smithi100.stdout: 1155: throughput=5MB/sec pending data=0 2024-04-07T16:58:30.280 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-uDb-G1mtacMkhKJ off=17066810 len=1809043 2024-04-07T16:58:30.293 INFO:tasks.workunit.client.0.smithi100.stdout:op 5992 completed, throughput=5MB/sec 2024-04-07T16:58:30.294 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-TsfSUgsNy5u5cLa off=4482797 len=1798106 2024-04-07T16:58:30.296 INFO:tasks.workunit.client.0.smithi100.stdout:op 5993 completed, throughput=5MB/sec 2024-04-07T16:58:30.296 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-Atea1Zt2Bk1qEwx off=3211398 len=657044 2024-04-07T16:58:30.304 INFO:tasks.workunit.client.0.smithi100.stdout:op 5994 completed, throughput=5MB/sec 2024-04-07T16:58:30.304 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-QIQ0C7Q8xWIb0U0 off=1175934 len=11803 2024-04-07T16:58:30.308 INFO:tasks.workunit.client.0.smithi100.stdout:op 5995 completed, throughput=5MB/sec 2024-04-07T16:58:30.309 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-jwLbI_BX90wp43S off=1877615 len=1623281 2024-04-07T16:58:30.319 INFO:tasks.workunit.client.0.smithi100.stdout:op 5996 completed, throughput=5MB/sec 2024-04-07T16:58:31.113 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:58:30 smithi144 bash[66654]: cluster 2024-04-07T16:58:29.679333+0000 mgr.y (mgr.25175) 1294 : cluster [DBG] pgmap v1936: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1.8 MiB/s rd, 2.5 MiB/s wr, 4 op/s 2024-04-07T16:58:31.199 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:58:30 smithi100 bash[74942]: cluster 2024-04-07T16:58:29.679333+0000 mgr.y (mgr.25175) 1294 : cluster [DBG] pgmap v1936: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1.8 MiB/s rd, 2.5 MiB/s wr, 4 op/s 2024-04-07T16:58:31.200 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:58:30 smithi100 bash[76202]: cluster 2024-04-07T16:58:29.679333+0000 mgr.y (mgr.25175) 1294 : cluster [DBG] pgmap v1936: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1.8 MiB/s rd, 2.5 MiB/s wr, 4 op/s 2024-04-07T16:58:31.319 INFO:tasks.workunit.client.0.smithi100.stdout: 1156: throughput=5MB/sec pending data=0 2024-04-07T16:58:31.319 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-tbqRgc4hQBHa7gC off=1723746 len=2084460 2024-04-07T16:58:31.335 INFO:tasks.workunit.client.0.smithi100.stdout:op 5997 completed, throughput=5MB/sec 2024-04-07T16:58:31.335 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-986x-pEw70bQj52 off=832547 len=117204 2024-04-07T16:58:31.336 INFO:tasks.workunit.client.0.smithi100.stdout:op 5998 completed, throughput=5MB/sec 2024-04-07T16:58:31.336 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-UFSAWsrrgogDasZ off=2661726 len=698074 2024-04-07T16:58:31.339 INFO:tasks.workunit.client.0.smithi100.stdout:op 5999 completed, throughput=5MB/sec 2024-04-07T16:58:31.339 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-YOlwN_7QSaXG5SD off=6293848 len=1244686 2024-04-07T16:58:31.342 INFO:tasks.workunit.client.0.smithi100.stdout:op 6000 completed, throughput=5MB/sec 2024-04-07T16:58:31.342 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-zNrU03-VpwQVzwr off=2115131 len=846444 2024-04-07T16:58:31.350 INFO:tasks.workunit.client.0.smithi100.stdout:op 6001 completed, throughput=5MB/sec 2024-04-07T16:58:32.351 INFO:tasks.workunit.client.0.smithi100.stdout: 1157: throughput=5MB/sec pending data=0 2024-04-07T16:58:32.351 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Xyftuvdwgb3QmnQ off=4809880 len=1662989 2024-04-07T16:58:32.356 INFO:tasks.workunit.client.0.smithi100.stdout:op 6002 completed, throughput=5MB/sec 2024-04-07T16:58:32.356 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-oIbF1z5oTFG0kUN off=4036761 len=268158 2024-04-07T16:58:32.362 INFO:tasks.workunit.client.0.smithi100.stdout:op 6003 completed, throughput=5MB/sec 2024-04-07T16:58:32.362 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-oUA1t6wiJxgtjAL off=9279217 len=1754250 2024-04-07T16:58:32.375 INFO:tasks.workunit.client.0.smithi100.stdout:op 6004 completed, throughput=5MB/sec 2024-04-07T16:58:32.375 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-VMTBzOR2uiEBmWd off=679173 len=1182584 2024-04-07T16:58:32.386 INFO:tasks.workunit.client.0.smithi100.stdout:op 6005 completed, throughput=5MB/sec 2024-04-07T16:58:32.386 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-2ImP8--eOnXvqxl off=13719256 len=1591917 2024-04-07T16:58:32.388 INFO:tasks.workunit.client.0.smithi100.stdout:op 6006 completed, throughput=5MB/sec 2024-04-07T16:58:32.613 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:58:32 smithi144 bash[95233]: ts=2024-04-07T16:58:32.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:58:33.113 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:58:32 smithi144 bash[66654]: cluster 2024-04-07T16:58:31.680144+0000 mgr.y (mgr.25175) 1295 : cluster [DBG] pgmap v1937: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 2.1 MiB/s rd, 2.9 MiB/s wr, 4 op/s 2024-04-07T16:58:33.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:58:32 smithi144 bash[66654]: audit 2024-04-07T16:58:31.790741+0000 mon.a (mon.0) 2220 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:58:33.199 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:58:32 smithi100 bash[74942]: cluster 2024-04-07T16:58:31.680144+0000 mgr.y (mgr.25175) 1295 : cluster [DBG] pgmap v1937: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 2.1 MiB/s rd, 2.9 MiB/s wr, 4 op/s 2024-04-07T16:58:33.199 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:58:32 smithi100 bash[74942]: audit 2024-04-07T16:58:31.790741+0000 mon.a (mon.0) 2220 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:58:33.200 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:58:32 smithi100 bash[76202]: cluster 2024-04-07T16:58:31.680144+0000 mgr.y (mgr.25175) 1295 : cluster [DBG] pgmap v1937: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 2.1 MiB/s rd, 2.9 MiB/s wr, 4 op/s 2024-04-07T16:58:33.200 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:58:32 smithi100 bash[76202]: audit 2024-04-07T16:58:31.790741+0000 mon.a (mon.0) 2220 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:58:33.388 INFO:tasks.workunit.client.0.smithi100.stdout: 1158: throughput=5MB/sec pending data=0 2024-04-07T16:58:33.388 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-upqcR0kpuGjRHYx off=4904262 len=1532404 2024-04-07T16:58:33.392 INFO:tasks.workunit.client.0.smithi100.stdout:op 6007 completed, throughput=5MB/sec 2024-04-07T16:58:33.392 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-bOdSY0L67MvTCJa off=244982 len=839913 2024-04-07T16:58:33.394 INFO:tasks.workunit.client.0.smithi100.stdout:op 6008 completed, throughput=5MB/sec 2024-04-07T16:58:33.395 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-qUEtZPx37x7DND1 off=1946154 len=574400 2024-04-07T16:58:33.401 INFO:tasks.workunit.client.0.smithi100.stdout:op 6009 completed, throughput=5MB/sec 2024-04-07T16:58:33.402 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Yhdtm9aST2kNO5- off=644190 len=1096459 2024-04-07T16:58:33.403 INFO:tasks.workunit.client.0.smithi100.stdout:op 6010 completed, throughput=5MB/sec 2024-04-07T16:58:34.404 INFO:tasks.workunit.client.0.smithi100.stdout: 1159: throughput=5MB/sec pending data=0 2024-04-07T16:58:34.404 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-z7WMQLJyNvRES6C off=5904485 len=1790776 2024-04-07T16:58:34.419 INFO:tasks.workunit.client.0.smithi100.stdout:op 6011 completed, throughput=5MB/sec 2024-04-07T16:58:34.419 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-k98-eocymhcnZcf off=7318447 len=795526 2024-04-07T16:58:34.427 INFO:tasks.workunit.client.0.smithi100.stdout:op 6012 completed, throughput=5MB/sec 2024-04-07T16:58:34.427 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-B4AJfhQuziaEwCc off=615475 len=1762289 2024-04-07T16:58:34.433 INFO:tasks.workunit.client.0.smithi100.stdout:op 6013 completed, throughput=5MB/sec 2024-04-07T16:58:34.433 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-FDMxYOn2GJhDkCZ off=11562350 len=39533 2024-04-07T16:58:34.437 INFO:tasks.workunit.client.0.smithi100.stdout:op 6014 completed, throughput=5MB/sec 2024-04-07T16:58:34.438 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-3x6UPByTGRf5MKg off=2927073 len=1817076 2024-04-07T16:58:34.452 INFO:tasks.workunit.client.0.smithi100.stdout:op 6015 completed, throughput=5MB/sec 2024-04-07T16:58:35.113 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:58:34 smithi144 bash[66654]: cluster 2024-04-07T16:58:33.681280+0000 mgr.y (mgr.25175) 1296 : cluster [DBG] pgmap v1938: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1.9 MiB/s rd, 3.0 MiB/s wr, 4 op/s 2024-04-07T16:58:35.199 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:58:34 smithi100 bash[74942]: cluster 2024-04-07T16:58:33.681280+0000 mgr.y (mgr.25175) 1296 : cluster [DBG] pgmap v1938: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1.9 MiB/s rd, 3.0 MiB/s wr, 4 op/s 2024-04-07T16:58:35.200 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:58:34 smithi100 bash[76202]: cluster 2024-04-07T16:58:33.681280+0000 mgr.y (mgr.25175) 1296 : cluster [DBG] pgmap v1938: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1.9 MiB/s rd, 3.0 MiB/s wr, 4 op/s 2024-04-07T16:58:35.453 INFO:tasks.workunit.client.0.smithi100.stdout: 1160: throughput=5MB/sec pending data=0 2024-04-07T16:58:35.453 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-s0orXXYq1f3JxdD off=2410833 len=326864 2024-04-07T16:58:35.459 INFO:tasks.workunit.client.0.smithi100.stdout:op 6016 completed, throughput=5MB/sec 2024-04-07T16:58:35.459 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-sWOrWDnVdnvsUem off=8880777 len=26845 2024-04-07T16:58:35.460 INFO:tasks.workunit.client.0.smithi100.stdout:op 6017 completed, throughput=5MB/sec 2024-04-07T16:58:35.460 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-5c4iOUdUc0CF54M off=2657049 len=1222755 2024-04-07T16:58:35.470 INFO:tasks.workunit.client.0.smithi100.stdout:op 6018 completed, throughput=5MB/sec 2024-04-07T16:58:35.470 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-BFRCzKMAmmLyy2l off=400520 len=1364357 2024-04-07T16:58:35.484 INFO:tasks.workunit.client.0.smithi100.stdout:op 6019 completed, throughput=5MB/sec 2024-04-07T16:58:35.484 INFO:tasks.workunit.client.0.smithi100.stdout:WRITE : oid=obj-npE7Kns2609jlXz off=9796565 len=97564 2024-04-07T16:58:35.488 INFO:tasks.workunit.client.0.smithi100.stdout:op 6020 completed, throughput=5MB/sec 2024-04-07T16:58:35.488 INFO:tasks.workunit.client.0.smithi100.stdout:READ : oid=obj-Z-c6i6prbtfMFyj off=18169890 len=1712965 2024-04-07T16:58:35.492 INFO:tasks.workunit.client.0.smithi100.stdout:op 6021 completed, throughput=5MB/sec 2024-04-07T16:58:36.492 INFO:tasks.workunit.client.0.smithi100.stdout:waiting for all operations to complete 2024-04-07T16:58:36.492 INFO:tasks.workunit.client.0.smithi100.stdout:cleaning up objects 2024-04-07T16:58:37.113 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:58:36 smithi144 bash[66654]: cluster 2024-04-07T16:58:35.682267+0000 mgr.y (mgr.25175) 1297 : cluster [DBG] pgmap v1939: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 2.2 MiB/s rd, 3.1 MiB/s wr, 4 op/s 2024-04-07T16:58:37.199 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:58:36 smithi100 bash[74942]: cluster 2024-04-07T16:58:35.682267+0000 mgr.y (mgr.25175) 1297 : cluster [DBG] pgmap v1939: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 2.2 MiB/s rd, 3.1 MiB/s wr, 4 op/s 2024-04-07T16:58:37.200 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:58:36 smithi100 bash[76202]: cluster 2024-04-07T16:58:35.682267+0000 mgr.y (mgr.25175) 1297 : cluster [DBG] pgmap v1939: 105 pgs: 105 active+clean; 126 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 2.2 MiB/s rd, 3.1 MiB/s wr, 4 op/s 2024-04-07T16:58:38.699 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:58:38 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:58:38] "GET /metrics HTTP/1.1" 200 34073 "" "Prometheus/2.43.0" 2024-04-07T16:58:39.113 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:58:38 smithi144 bash[66654]: cluster 2024-04-07T16:58:37.683318+0000 mgr.y (mgr.25175) 1298 : cluster [DBG] pgmap v1940: 105 pgs: 105 active+clean; 125 GiB data, 12 GiB used, 704 GiB / 715 GiB avail; 1.9 MiB/s rd, 3.0 MiB/s wr, 8 op/s 2024-04-07T16:58:39.199 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:58:38 smithi100 bash[74942]: cluster 2024-04-07T16:58:37.683318+0000 mgr.y (mgr.25175) 1298 : cluster [DBG] pgmap v1940: 105 pgs: 105 active+clean; 125 GiB data, 12 GiB used, 704 GiB / 715 GiB avail; 1.9 MiB/s rd, 3.0 MiB/s wr, 8 op/s 2024-04-07T16:58:39.200 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:58:38 smithi100 bash[76202]: cluster 2024-04-07T16:58:37.683318+0000 mgr.y (mgr.25175) 1298 : cluster [DBG] pgmap v1940: 105 pgs: 105 active+clean; 125 GiB data, 12 GiB used, 704 GiB / 715 GiB avail; 1.9 MiB/s rd, 3.0 MiB/s wr, 8 op/s 2024-04-07T16:58:40.113 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:58:39 smithi144 bash[66654]: cluster 2024-04-07T16:58:39.684265+0000 mgr.y (mgr.25175) 1299 : cluster [DBG] pgmap v1941: 105 pgs: 105 active+clean; 124 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 2.0 MiB/s rd, 2.6 MiB/s wr, 16 op/s 2024-04-07T16:58:40.199 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:58:39 smithi100 bash[74942]: cluster 2024-04-07T16:58:39.684265+0000 mgr.y (mgr.25175) 1299 : cluster [DBG] pgmap v1941: 105 pgs: 105 active+clean; 124 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 2.0 MiB/s rd, 2.6 MiB/s wr, 16 op/s 2024-04-07T16:58:40.200 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:58:39 smithi100 bash[76202]: cluster 2024-04-07T16:58:39.684265+0000 mgr.y (mgr.25175) 1299 : cluster [DBG] pgmap v1941: 105 pgs: 105 active+clean; 124 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 2.0 MiB/s rd, 2.6 MiB/s wr, 16 op/s 2024-04-07T16:58:42.614 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:58:42 smithi144 bash[95233]: ts=2024-04-07T16:58:42.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:58:43.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:58:42 smithi144 bash[66654]: cluster 2024-04-07T16:58:41.685241+0000 mgr.y (mgr.25175) 1300 : cluster [DBG] pgmap v1942: 105 pgs: 105 active+clean; 120 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1.8 MiB/s rd, 2.4 MiB/s wr, 42 op/s 2024-04-07T16:58:43.200 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:58:42 smithi100 bash[74942]: cluster 2024-04-07T16:58:41.685241+0000 mgr.y (mgr.25175) 1300 : cluster [DBG] pgmap v1942: 105 pgs: 105 active+clean; 120 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1.8 MiB/s rd, 2.4 MiB/s wr, 42 op/s 2024-04-07T16:58:43.200 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:58:42 smithi100 bash[76202]: cluster 2024-04-07T16:58:41.685241+0000 mgr.y (mgr.25175) 1300 : cluster [DBG] pgmap v1942: 105 pgs: 105 active+clean; 120 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1.8 MiB/s rd, 2.4 MiB/s wr, 42 op/s 2024-04-07T16:58:45.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:58:44 smithi144 bash[66654]: cluster 2024-04-07T16:58:43.686387+0000 mgr.y (mgr.25175) 1301 : cluster [DBG] pgmap v1943: 105 pgs: 105 active+clean; 110 GiB data, 11 GiB used, 705 GiB / 715 GiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 109 op/s 2024-04-07T16:58:45.199 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:58:44 smithi100 bash[74942]: cluster 2024-04-07T16:58:43.686387+0000 mgr.y (mgr.25175) 1301 : cluster [DBG] pgmap v1943: 105 pgs: 105 active+clean; 110 GiB data, 11 GiB used, 705 GiB / 715 GiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 109 op/s 2024-04-07T16:58:45.200 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:58:44 smithi100 bash[76202]: cluster 2024-04-07T16:58:43.686387+0000 mgr.y (mgr.25175) 1301 : cluster [DBG] pgmap v1943: 105 pgs: 105 active+clean; 110 GiB data, 11 GiB used, 705 GiB / 715 GiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 109 op/s 2024-04-07T16:58:47.113 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:58:46 smithi144 bash[66654]: cluster 2024-04-07T16:58:45.687354+0000 mgr.y (mgr.25175) 1302 : cluster [DBG] pgmap v1944: 105 pgs: 105 active+clean; 104 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 1007 KiB/s rd, 1.3 MiB/s wr, 149 op/s 2024-04-07T16:58:47.199 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:58:46 smithi100 bash[74942]: cluster 2024-04-07T16:58:45.687354+0000 mgr.y (mgr.25175) 1302 : cluster [DBG] pgmap v1944: 105 pgs: 105 active+clean; 104 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 1007 KiB/s rd, 1.3 MiB/s wr, 149 op/s 2024-04-07T16:58:47.200 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:58:46 smithi100 bash[76202]: cluster 2024-04-07T16:58:45.687354+0000 mgr.y (mgr.25175) 1302 : cluster [DBG] pgmap v1944: 105 pgs: 105 active+clean; 104 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 1007 KiB/s rd, 1.3 MiB/s wr, 149 op/s 2024-04-07T16:58:48.113 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:58:47 smithi144 bash[66654]: audit 2024-04-07T16:58:46.791268+0000 mon.a (mon.0) 2221 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:58:48.199 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:58:47 smithi100 bash[74942]: audit 2024-04-07T16:58:46.791268+0000 mon.a (mon.0) 2221 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:58:48.200 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:58:47 smithi100 bash[76202]: audit 2024-04-07T16:58:46.791268+0000 mon.a (mon.0) 2221 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:58:48.700 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:58:48 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:58:48] "GET /metrics HTTP/1.1" 200 34071 "" "Prometheus/2.43.0" 2024-04-07T16:58:49.113 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:58:48 smithi144 bash[66654]: cluster 2024-04-07T16:58:47.688454+0000 mgr.y (mgr.25175) 1303 : cluster [DBG] pgmap v1945: 105 pgs: 105 active+clean; 98 GiB data, 9.7 GiB used, 706 GiB / 715 GiB avail; 420 KiB/s rd, 623 KiB/s wr, 188 op/s 2024-04-07T16:58:49.199 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:58:48 smithi100 bash[74942]: cluster 2024-04-07T16:58:47.688454+0000 mgr.y (mgr.25175) 1303 : cluster [DBG] pgmap v1945: 105 pgs: 105 active+clean; 98 GiB data, 9.7 GiB used, 706 GiB / 715 GiB avail; 420 KiB/s rd, 623 KiB/s wr, 188 op/s 2024-04-07T16:58:49.200 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:58:48 smithi100 bash[76202]: cluster 2024-04-07T16:58:47.688454+0000 mgr.y (mgr.25175) 1303 : cluster [DBG] pgmap v1945: 105 pgs: 105 active+clean; 98 GiB data, 9.7 GiB used, 706 GiB / 715 GiB avail; 420 KiB/s rd, 623 KiB/s wr, 188 op/s 2024-04-07T16:58:50.113 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:58:49 smithi144 bash[66654]: cluster 2024-04-07T16:58:49.689482+0000 mgr.y (mgr.25175) 1304 : cluster [DBG] pgmap v1946: 105 pgs: 105 active+clean; 94 GiB data, 9.3 GiB used, 706 GiB / 715 GiB avail; 139 KiB/s rd, 272 KiB/s wr, 209 op/s 2024-04-07T16:58:50.199 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:58:49 smithi100 bash[74942]: cluster 2024-04-07T16:58:49.689482+0000 mgr.y (mgr.25175) 1304 : cluster [DBG] pgmap v1946: 105 pgs: 105 active+clean; 94 GiB data, 9.3 GiB used, 706 GiB / 715 GiB avail; 139 KiB/s rd, 272 KiB/s wr, 209 op/s 2024-04-07T16:58:50.200 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:58:49 smithi100 bash[76202]: cluster 2024-04-07T16:58:49.689482+0000 mgr.y (mgr.25175) 1304 : cluster [DBG] pgmap v1946: 105 pgs: 105 active+clean; 94 GiB data, 9.3 GiB used, 706 GiB / 715 GiB avail; 139 KiB/s rd, 272 KiB/s wr, 209 op/s 2024-04-07T16:58:52.615 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:58:52 smithi144 bash[95233]: ts=2024-04-07T16:58:52.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:58:53.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:58:52 smithi144 bash[66654]: cluster 2024-04-07T16:58:51.690369+0000 mgr.y (mgr.25175) 1305 : cluster [DBG] pgmap v1947: 105 pgs: 105 active+clean; 88 GiB data, 9.1 GiB used, 706 GiB / 715 GiB avail; 100 KiB/s wr, 240 op/s 2024-04-07T16:58:53.200 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:58:52 smithi100 bash[74942]: cluster 2024-04-07T16:58:51.690369+0000 mgr.y (mgr.25175) 1305 : cluster [DBG] pgmap v1947: 105 pgs: 105 active+clean; 88 GiB data, 9.1 GiB used, 706 GiB / 715 GiB avail; 100 KiB/s wr, 240 op/s 2024-04-07T16:58:53.201 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:58:52 smithi100 bash[76202]: cluster 2024-04-07T16:58:51.690369+0000 mgr.y (mgr.25175) 1305 : cluster [DBG] pgmap v1947: 105 pgs: 105 active+clean; 88 GiB data, 9.1 GiB used, 706 GiB / 715 GiB avail; 100 KiB/s wr, 240 op/s 2024-04-07T16:58:55.113 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:58:54 smithi144 bash[66654]: cluster 2024-04-07T16:58:53.691749+0000 mgr.y (mgr.25175) 1306 : cluster [DBG] pgmap v1948: 105 pgs: 105 active+clean; 79 GiB data, 8.5 GiB used, 707 GiB / 715 GiB avail; 0 B/s wr, 277 op/s 2024-04-07T16:58:55.199 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:58:54 smithi100 bash[74942]: cluster 2024-04-07T16:58:53.691749+0000 mgr.y (mgr.25175) 1306 : cluster [DBG] pgmap v1948: 105 pgs: 105 active+clean; 79 GiB data, 8.5 GiB used, 707 GiB / 715 GiB avail; 0 B/s wr, 277 op/s 2024-04-07T16:58:55.200 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:58:54 smithi100 bash[76202]: cluster 2024-04-07T16:58:53.691749+0000 mgr.y (mgr.25175) 1306 : cluster [DBG] pgmap v1948: 105 pgs: 105 active+clean; 79 GiB data, 8.5 GiB used, 707 GiB / 715 GiB avail; 0 B/s wr, 277 op/s 2024-04-07T16:58:57.113 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:58:56 smithi144 bash[66654]: cluster 2024-04-07T16:58:55.692852+0000 mgr.y (mgr.25175) 1307 : cluster [DBG] pgmap v1949: 105 pgs: 105 active+clean; 74 GiB data, 8.1 GiB used, 707 GiB / 715 GiB avail; 0 B/s wr, 248 op/s 2024-04-07T16:58:57.199 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:58:56 smithi100 bash[74942]: cluster 2024-04-07T16:58:55.692852+0000 mgr.y (mgr.25175) 1307 : cluster [DBG] pgmap v1949: 105 pgs: 105 active+clean; 74 GiB data, 8.1 GiB used, 707 GiB / 715 GiB avail; 0 B/s wr, 248 op/s 2024-04-07T16:58:57.200 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:58:56 smithi100 bash[76202]: cluster 2024-04-07T16:58:55.692852+0000 mgr.y (mgr.25175) 1307 : cluster [DBG] pgmap v1949: 105 pgs: 105 active+clean; 74 GiB data, 8.1 GiB used, 707 GiB / 715 GiB avail; 0 B/s wr, 248 op/s 2024-04-07T16:58:58.702 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:58:58 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:58:58] "GET /metrics HTTP/1.1" 200 34071 "" "Prometheus/2.43.0" 2024-04-07T16:58:59.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:58:58 smithi144 bash[66654]: cluster 2024-04-07T16:58:57.694000+0000 mgr.y (mgr.25175) 1308 : cluster [DBG] pgmap v1950: 105 pgs: 105 active+clean; 69 GiB data, 7.7 GiB used, 707 GiB / 715 GiB avail; 0 B/s wr, 243 op/s 2024-04-07T16:58:59.199 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:58:58 smithi100 bash[74942]: cluster 2024-04-07T16:58:57.694000+0000 mgr.y (mgr.25175) 1308 : cluster [DBG] pgmap v1950: 105 pgs: 105 active+clean; 69 GiB data, 7.7 GiB used, 707 GiB / 715 GiB avail; 0 B/s wr, 243 op/s 2024-04-07T16:58:59.200 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:58:58 smithi100 bash[76202]: cluster 2024-04-07T16:58:57.694000+0000 mgr.y (mgr.25175) 1308 : cluster [DBG] pgmap v1950: 105 pgs: 105 active+clean; 69 GiB data, 7.7 GiB used, 707 GiB / 715 GiB avail; 0 B/s wr, 243 op/s 2024-04-07T16:59:00.113 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:58:59 smithi144 bash[66654]: cluster 2024-04-07T16:58:59.694975+0000 mgr.y (mgr.25175) 1309 : cluster [DBG] pgmap v1951: 105 pgs: 105 active+clean; 65 GiB data, 7.3 GiB used, 708 GiB / 715 GiB avail; 0 B/s wr, 228 op/s 2024-04-07T16:59:00.199 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:58:59 smithi100 bash[74942]: cluster 2024-04-07T16:58:59.694975+0000 mgr.y (mgr.25175) 1309 : cluster [DBG] pgmap v1951: 105 pgs: 105 active+clean; 65 GiB data, 7.3 GiB used, 708 GiB / 715 GiB avail; 0 B/s wr, 228 op/s 2024-04-07T16:59:00.200 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:58:59 smithi100 bash[76202]: cluster 2024-04-07T16:58:59.694975+0000 mgr.y (mgr.25175) 1309 : cluster [DBG] pgmap v1951: 105 pgs: 105 active+clean; 65 GiB data, 7.3 GiB used, 708 GiB / 715 GiB avail; 0 B/s wr, 228 op/s 2024-04-07T16:59:02.614 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:59:02 smithi144 bash[95233]: ts=2024-04-07T16:59:02.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:59:03.113 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:59:02 smithi144 bash[66654]: cluster 2024-04-07T16:59:01.695802+0000 mgr.y (mgr.25175) 1310 : cluster [DBG] pgmap v1952: 105 pgs: 105 active+clean; 60 GiB data, 7.2 GiB used, 708 GiB / 715 GiB avail; 0 B/s wr, 236 op/s 2024-04-07T16:59:03.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:59:02 smithi144 bash[66654]: audit 2024-04-07T16:59:01.791853+0000 mon.a (mon.0) 2222 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:59:03.202 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:59:02 smithi100 bash[74942]: cluster 2024-04-07T16:59:01.695802+0000 mgr.y (mgr.25175) 1310 : cluster [DBG] pgmap v1952: 105 pgs: 105 active+clean; 60 GiB data, 7.2 GiB used, 708 GiB / 715 GiB avail; 0 B/s wr, 236 op/s 2024-04-07T16:59:03.202 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:59:02 smithi100 bash[74942]: audit 2024-04-07T16:59:01.791853+0000 mon.a (mon.0) 2222 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:59:03.202 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:59:02 smithi100 bash[76202]: cluster 2024-04-07T16:59:01.695802+0000 mgr.y (mgr.25175) 1310 : cluster [DBG] pgmap v1952: 105 pgs: 105 active+clean; 60 GiB data, 7.2 GiB used, 708 GiB / 715 GiB avail; 0 B/s wr, 236 op/s 2024-04-07T16:59:03.202 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:59:02 smithi100 bash[76202]: audit 2024-04-07T16:59:01.791853+0000 mon.a (mon.0) 2222 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:59:05.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:59:04 smithi144 bash[66654]: cluster 2024-04-07T16:59:03.697009+0000 mgr.y (mgr.25175) 1311 : cluster [DBG] pgmap v1953: 105 pgs: 105 active+clean; 51 GiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 0 B/s wr, 260 op/s 2024-04-07T16:59:05.199 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:59:04 smithi100 bash[74942]: cluster 2024-04-07T16:59:03.697009+0000 mgr.y (mgr.25175) 1311 : cluster [DBG] pgmap v1953: 105 pgs: 105 active+clean; 51 GiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 0 B/s wr, 260 op/s 2024-04-07T16:59:05.200 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:59:04 smithi100 bash[76202]: cluster 2024-04-07T16:59:03.697009+0000 mgr.y (mgr.25175) 1311 : cluster [DBG] pgmap v1953: 105 pgs: 105 active+clean; 51 GiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 0 B/s wr, 260 op/s 2024-04-07T16:59:07.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:59:06 smithi144 bash[66654]: cluster 2024-04-07T16:59:05.697978+0000 mgr.y (mgr.25175) 1312 : cluster [DBG] pgmap v1954: 105 pgs: 105 active+clean; 46 GiB data, 6.2 GiB used, 709 GiB / 715 GiB avail; 0 B/s wr, 231 op/s 2024-04-07T16:59:07.199 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:59:06 smithi100 bash[74942]: cluster 2024-04-07T16:59:05.697978+0000 mgr.y (mgr.25175) 1312 : cluster [DBG] pgmap v1954: 105 pgs: 105 active+clean; 46 GiB data, 6.2 GiB used, 709 GiB / 715 GiB avail; 0 B/s wr, 231 op/s 2024-04-07T16:59:07.200 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:59:06 smithi100 bash[76202]: cluster 2024-04-07T16:59:05.697978+0000 mgr.y (mgr.25175) 1312 : cluster [DBG] pgmap v1954: 105 pgs: 105 active+clean; 46 GiB data, 6.2 GiB used, 709 GiB / 715 GiB avail; 0 B/s wr, 231 op/s 2024-04-07T16:59:08.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:59:07 smithi144 bash[66654]: cluster 2024-04-07T16:59:07.698996+0000 mgr.y (mgr.25175) 1313 : cluster [DBG] pgmap v1955: 105 pgs: 105 active+clean; 41 GiB data, 5.7 GiB used, 709 GiB / 715 GiB avail; 0 B/s wr, 228 op/s 2024-04-07T16:59:08.205 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:59:07 smithi100 bash[74942]: cluster 2024-04-07T16:59:07.698996+0000 mgr.y (mgr.25175) 1313 : cluster [DBG] pgmap v1955: 105 pgs: 105 active+clean; 41 GiB data, 5.7 GiB used, 709 GiB / 715 GiB avail; 0 B/s wr, 228 op/s 2024-04-07T16:59:08.205 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:59:07 smithi100 bash[76202]: cluster 2024-04-07T16:59:07.698996+0000 mgr.y (mgr.25175) 1313 : cluster [DBG] pgmap v1955: 105 pgs: 105 active+clean; 41 GiB data, 5.7 GiB used, 709 GiB / 715 GiB avail; 0 B/s wr, 228 op/s 2024-04-07T16:59:08.699 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:59:08 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:59:08] "GET /metrics HTTP/1.1" 200 34068 "" "Prometheus/2.43.0" 2024-04-07T16:59:11.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:59:10 smithi144 bash[66654]: cluster 2024-04-07T16:59:09.699945+0000 mgr.y (mgr.25175) 1314 : cluster [DBG] pgmap v1956: 105 pgs: 105 active+clean; 37 GiB data, 5.4 GiB used, 710 GiB / 715 GiB avail; 0 B/s wr, 215 op/s 2024-04-07T16:59:11.199 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:59:10 smithi100 bash[74942]: cluster 2024-04-07T16:59:09.699945+0000 mgr.y (mgr.25175) 1314 : cluster [DBG] pgmap v1956: 105 pgs: 105 active+clean; 37 GiB data, 5.4 GiB used, 710 GiB / 715 GiB avail; 0 B/s wr, 215 op/s 2024-04-07T16:59:11.199 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:59:10 smithi100 bash[76202]: cluster 2024-04-07T16:59:09.699945+0000 mgr.y (mgr.25175) 1314 : cluster [DBG] pgmap v1956: 105 pgs: 105 active+clean; 37 GiB data, 5.4 GiB used, 710 GiB / 715 GiB avail; 0 B/s wr, 215 op/s 2024-04-07T16:59:12.614 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:59:12 smithi144 bash[95233]: ts=2024-04-07T16:59:12.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:59:13.113 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:59:12 smithi144 bash[66654]: cluster 2024-04-07T16:59:11.700825+0000 mgr.y (mgr.25175) 1315 : cluster [DBG] pgmap v1957: 105 pgs: 105 active+clean; 32 GiB data, 5.2 GiB used, 710 GiB / 715 GiB avail; 0 B/s wr, 224 op/s 2024-04-07T16:59:13.199 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:59:12 smithi100 bash[76202]: cluster 2024-04-07T16:59:11.700825+0000 mgr.y (mgr.25175) 1315 : cluster [DBG] pgmap v1957: 105 pgs: 105 active+clean; 32 GiB data, 5.2 GiB used, 710 GiB / 715 GiB avail; 0 B/s wr, 224 op/s 2024-04-07T16:59:13.200 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:59:12 smithi100 bash[74942]: cluster 2024-04-07T16:59:11.700825+0000 mgr.y (mgr.25175) 1315 : cluster [DBG] pgmap v1957: 105 pgs: 105 active+clean; 32 GiB data, 5.2 GiB used, 710 GiB / 715 GiB avail; 0 B/s wr, 224 op/s 2024-04-07T16:59:14.113 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:59:13 smithi144 bash[66654]: cluster 2024-04-07T16:59:13.701978+0000 mgr.y (mgr.25175) 1316 : cluster [DBG] pgmap v1958: 105 pgs: 105 active+clean; 23 GiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 0 B/s wr, 251 op/s 2024-04-07T16:59:14.199 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:59:13 smithi100 bash[74942]: cluster 2024-04-07T16:59:13.701978+0000 mgr.y (mgr.25175) 1316 : cluster [DBG] pgmap v1958: 105 pgs: 105 active+clean; 23 GiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 0 B/s wr, 251 op/s 2024-04-07T16:59:14.200 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:59:13 smithi100 bash[76202]: cluster 2024-04-07T16:59:13.701978+0000 mgr.y (mgr.25175) 1316 : cluster [DBG] pgmap v1958: 105 pgs: 105 active+clean; 23 GiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 0 B/s wr, 251 op/s 2024-04-07T16:59:17.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:59:16 smithi144 bash[66654]: cluster 2024-04-07T16:59:15.703019+0000 mgr.y (mgr.25175) 1317 : cluster [DBG] pgmap v1959: 105 pgs: 105 active+clean; 18 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 0 B/s wr, 220 op/s 2024-04-07T16:59:17.199 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:59:16 smithi100 bash[74942]: cluster 2024-04-07T16:59:15.703019+0000 mgr.y (mgr.25175) 1317 : cluster [DBG] pgmap v1959: 105 pgs: 105 active+clean; 18 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 0 B/s wr, 220 op/s 2024-04-07T16:59:17.200 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:59:16 smithi100 bash[76202]: cluster 2024-04-07T16:59:15.703019+0000 mgr.y (mgr.25175) 1317 : cluster [DBG] pgmap v1959: 105 pgs: 105 active+clean; 18 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 0 B/s wr, 220 op/s 2024-04-07T16:59:18.113 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:59:17 smithi144 bash[66654]: audit 2024-04-07T16:59:16.792536+0000 mon.a (mon.0) 2223 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:59:18.199 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:59:17 smithi100 bash[74942]: audit 2024-04-07T16:59:16.792536+0000 mon.a (mon.0) 2223 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:59:18.200 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:59:17 smithi100 bash[76202]: audit 2024-04-07T16:59:16.792536+0000 mon.a (mon.0) 2223 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:59:18.699 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:59:18 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:59:18] "GET /metrics HTTP/1.1" 200 34064 "" "Prometheus/2.43.0" 2024-04-07T16:59:19.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:59:18 smithi144 bash[66654]: cluster 2024-04-07T16:59:17.704451+0000 mgr.y (mgr.25175) 1318 : cluster [DBG] pgmap v1960: 105 pgs: 105 active+clean; 13 GiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 0 B/s wr, 219 op/s 2024-04-07T16:59:19.199 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:59:18 smithi100 bash[74942]: cluster 2024-04-07T16:59:17.704451+0000 mgr.y (mgr.25175) 1318 : cluster [DBG] pgmap v1960: 105 pgs: 105 active+clean; 13 GiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 0 B/s wr, 219 op/s 2024-04-07T16:59:19.200 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:59:18 smithi100 bash[76202]: cluster 2024-04-07T16:59:17.704451+0000 mgr.y (mgr.25175) 1318 : cluster [DBG] pgmap v1960: 105 pgs: 105 active+clean; 13 GiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 0 B/s wr, 219 op/s 2024-04-07T16:59:20.094 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-07T16:59:20.095 DEBUG:teuthology.orchestra.run.smithi100:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-07T16:59:20.105 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:59:19 smithi100 bash[74942]: cluster 2024-04-07T16:59:19.705396+0000 mgr.y (mgr.25175) 1319 : cluster [DBG] pgmap v1961: 105 pgs: 105 active+clean; 9.6 GiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 0 B/s wr, 206 op/s 2024-04-07T16:59:20.105 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:59:19 smithi100 bash[76202]: cluster 2024-04-07T16:59:19.705396+0000 mgr.y (mgr.25175) 1319 : cluster [DBG] pgmap v1961: 105 pgs: 105 active+clean; 9.6 GiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 0 B/s wr, 206 op/s 2024-04-07T16:59:20.108 INFO:tasks.workunit:Stopping ['rados/load-gen-big.sh'] on client.0... 2024-04-07T16:59:20.108 DEBUG:teuthology.orchestra.run.smithi100:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-04-07T16:59:20.113 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:59:19 smithi144 bash[66654]: cluster 2024-04-07T16:59:19.705396+0000 mgr.y (mgr.25175) 1319 : cluster [DBG] pgmap v1961: 105 pgs: 105 active+clean; 9.6 GiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 0 B/s wr, 206 op/s 2024-04-07T16:59:20.388 DEBUG:teuthology.parallel:result is None 2024-04-07T16:59:20.388 DEBUG:teuthology.orchestra.run.smithi100:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-04-07T16:59:20.401 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-04-07T16:59:20.401 DEBUG:teuthology.orchestra.run.smithi100:> rmdir -- /home/ubuntu/cephtest/mnt.0 2024-04-07T16:59:20.448 INFO:tasks.workunit:Deleted artificial mount point /home/ubuntu/cephtest/mnt.0/client.0 2024-04-07T16:59:20.448 INFO:teuthology.task.full_sequential:In full_sequential, running task print... 2024-04-07T16:59:20.448 INFO:teuthology.task.print:**** done end rados_loadgenbig.yaml 2024-04-07T16:59:20.448 INFO:teuthology.task.full_sequential:In full_sequential, running task print... 2024-04-07T16:59:20.448 INFO:teuthology.task.print:**** done start rbd_import_export.yaml 2024-04-07T16:59:20.448 INFO:teuthology.task.full_sequential:In full_sequential, running task workunit... 2024-04-07T16:59:20.449 INFO:tasks.workunit:Pulling workunits from ref reef 2024-04-07T16:59:20.449 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-04-07T16:59:20.449 DEBUG:teuthology.orchestra.run.smithi144:> stat -- /home/ubuntu/cephtest/mnt.1 2024-04-07T16:59:20.455 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-07T16:59:20.456 INFO:teuthology.orchestra.run.smithi144.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-04-07T16:59:20.456 DEBUG:teuthology.orchestra.run.smithi144:> mkdir -- /home/ubuntu/cephtest/mnt.1 2024-04-07T16:59:20.502 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.1 2024-04-07T16:59:20.502 DEBUG:teuthology.orchestra.run.smithi144:> cd -- /home/ubuntu/cephtest/mnt.1 && mkdir -- client.1 2024-04-07T16:59:20.551 INFO:tasks.workunit:timeout=3h 2024-04-07T16:59:20.551 INFO:tasks.workunit:cleanup=True 2024-04-07T16:59:20.551 DEBUG:teuthology.orchestra.run.smithi144:> rm -rf /home/ubuntu/cephtest/clone.client.1 && git clone --depth 1 --branch reef https://github.com/ceph/ceph-ci.git /home/ubuntu/cephtest/clone.client.1 && cd /home/ubuntu/cephtest/clone.client.1 2024-04-07T16:59:20.604 INFO:tasks.workunit.client.1.smithi144.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.1'... 2024-04-07T16:59:22.614 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:59:22 smithi144 bash[95233]: ts=2024-04-07T16:59:22.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:59:23.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:59:22 smithi144 bash[66654]: cluster 2024-04-07T16:59:21.706188+0000 mgr.y (mgr.25175) 1320 : cluster [DBG] pgmap v1962: 105 pgs: 105 active+clean; 5.1 GiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 0 B/s wr, 212 op/s 2024-04-07T16:59:23.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:59:22 smithi144 bash[66654]: audit 2024-04-07T16:59:22.048925+0000 mon.a (mon.0) 2224 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:59:23.200 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:59:22 smithi100 bash[74942]: cluster 2024-04-07T16:59:21.706188+0000 mgr.y (mgr.25175) 1320 : cluster [DBG] pgmap v1962: 105 pgs: 105 active+clean; 5.1 GiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 0 B/s wr, 212 op/s 2024-04-07T16:59:23.200 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:59:22 smithi100 bash[74942]: audit 2024-04-07T16:59:22.048925+0000 mon.a (mon.0) 2224 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:59:23.200 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:59:22 smithi100 bash[76202]: cluster 2024-04-07T16:59:21.706188+0000 mgr.y (mgr.25175) 1320 : cluster [DBG] pgmap v1962: 105 pgs: 105 active+clean; 5.1 GiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 0 B/s wr, 212 op/s 2024-04-07T16:59:23.200 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:59:22 smithi100 bash[76202]: audit 2024-04-07T16:59:22.048925+0000 mon.a (mon.0) 2224 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T16:59:24.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:59:23 smithi144 bash[66654]: cluster 2024-04-07T16:59:23.707337+0000 mgr.y (mgr.25175) 1321 : cluster [DBG] pgmap v1963: 105 pgs: 105 active+clean; 62 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 0 B/s wr, 211 op/s 2024-04-07T16:59:24.200 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:59:23 smithi100 bash[74942]: cluster 2024-04-07T16:59:23.707337+0000 mgr.y (mgr.25175) 1321 : cluster [DBG] pgmap v1963: 105 pgs: 105 active+clean; 62 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 0 B/s wr, 211 op/s 2024-04-07T16:59:24.200 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:59:23 smithi100 bash[76202]: cluster 2024-04-07T16:59:23.707337+0000 mgr.y (mgr.25175) 1321 : cluster [DBG] pgmap v1963: 105 pgs: 105 active+clean; 62 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 0 B/s wr, 211 op/s 2024-04-07T16:59:27.113 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:59:26 smithi144 bash[66654]: cluster 2024-04-07T16:59:25.708309+0000 mgr.y (mgr.25175) 1322 : cluster [DBG] pgmap v1964: 105 pgs: 105 active+clean; 583 KiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 0 B/s wr, 152 op/s 2024-04-07T16:59:27.127 DEBUG:teuthology.orchestra.run.smithi144:> cd -- /home/ubuntu/cephtest/clone.client.1/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.1 2024-04-07T16:59:27.173 INFO:tasks.workunit.client.1.smithi144.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-04-07T16:59:27.174 INFO:tasks.workunit.client.1.smithi144.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/direct_io' 2024-04-07T16:59:27.174 INFO:tasks.workunit.client.1.smithi144.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-04-07T16:59:27.199 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:59:26 smithi100 bash[74942]: cluster 2024-04-07T16:59:25.708309+0000 mgr.y (mgr.25175) 1322 : cluster [DBG] pgmap v1964: 105 pgs: 105 active+clean; 583 KiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 0 B/s wr, 152 op/s 2024-04-07T16:59:27.199 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:59:26 smithi100 bash[76202]: cluster 2024-04-07T16:59:25.708309+0000 mgr.y (mgr.25175) 1322 : cluster [DBG] pgmap v1964: 105 pgs: 105 active+clean; 583 KiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 0 B/s wr, 152 op/s 2024-04-07T16:59:27.942 INFO:tasks.workunit.client.1.smithi144.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-04-07T16:59:28.017 INFO:tasks.workunit.client.1.smithi144.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-04-07T16:59:28.061 INFO:tasks.workunit.client.1.smithi144.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/direct_io' 2024-04-07T16:59:28.063 INFO:tasks.workunit.client.1.smithi144.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/fs' 2024-04-07T16:59:28.063 INFO:tasks.workunit.client.1.smithi144.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-04-07T16:59:28.105 INFO:tasks.workunit.client.1.smithi144.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/fs' 2024-04-07T16:59:28.108 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-04-07T16:59:28.108 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/home/ubuntu/cephtest/workunits.list.client.1 of=/dev/stdout 2024-04-07T16:59:28.155 INFO:tasks.workunit:Running workunits matching rbd/import_export.sh on client.1... 2024-04-07T16:59:28.157 INFO:tasks.workunit:Running workunit rbd/import_export.sh... 2024-04-07T16:59:28.157 DEBUG:teuthology.orchestra.run.smithi144:workunit test rbd/import_export.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && cd -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=reef TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="1" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.1 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.1 CEPH_MNT=/home/ubuntu/cephtest/mnt.1 RBD_CREATE_ARGS=--new-format adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.1/qa/workunits/rbd/import_export.sh 2024-04-07T16:59:28.204 INFO:tasks.workunit.client.1.smithi144.stderr:+ export RBD_FORCE_ALLOW_V1=1 2024-04-07T16:59:28.204 INFO:tasks.workunit.client.1.smithi144.stderr:+ TMPDIR=/tmp/rbd_import_export_103522 2024-04-07T16:59:28.204 INFO:tasks.workunit.client.1.smithi144.stderr:+ rm -rf /tmp/rbd_import_export_103522 2024-04-07T16:59:28.205 INFO:tasks.workunit.client.1.smithi144.stderr:+ mkdir /tmp/rbd_import_export_103522 2024-04-07T16:59:28.205 INFO:tasks.workunit.client.1.smithi144.stderr:+ trap rm -rf /tmp/rbd_import_export_103522 INT TERM EXIT 2024-04-07T16:59:28.205 INFO:tasks.workunit.client.1.smithi144.stderr:+ mkdir foo.103522 2024-04-07T16:59:28.206 INFO:tasks.workunit.client.1.smithi144.stderr:+ rbd import foo.103522 foo.dir 2024-04-07T16:59:28.277 INFO:tasks.workunit.client.1.smithi144.stderr:rbd: cannot import a directory 2024-04-07T16:59:28.277 INFO:tasks.workunit.client.1.smithi144.stderr: Importing image: 0% complete...failed. 2024-04-07T16:59:28.277 INFO:tasks.workunit.client.1.smithi144.stderr:rbd: import failed: (21) Is a directory 2024-04-07T16:59:28.282 INFO:tasks.workunit.client.1.smithi144.stderr:+ true 2024-04-07T16:59:28.282 INFO:tasks.workunit.client.1.smithi144.stderr:+ rmdir foo.103522 2024-04-07T16:59:28.290 INFO:tasks.workunit.client.1.smithi144.stderr:+ dd if=/bin/sh of=/tmp/rbd_import_export_103522/img bs=1k count=1 seek=10 2024-04-07T16:59:28.291 INFO:tasks.workunit.client.1.smithi144.stderr:1+0 records in 2024-04-07T16:59:28.291 INFO:tasks.workunit.client.1.smithi144.stderr:1+0 records out 2024-04-07T16:59:28.291 INFO:tasks.workunit.client.1.smithi144.stderr:1024 bytes (1.0 kB, 1.0 KiB) copied, 8.3326e-05 s, 12.3 MB/s 2024-04-07T16:59:28.291 INFO:tasks.workunit.client.1.smithi144.stderr:+ dd if=/bin/dd of=/tmp/rbd_import_export_103522/img bs=1k count=10 seek=100 2024-04-07T16:59:28.292 INFO:tasks.workunit.client.1.smithi144.stderr:10+0 records in 2024-04-07T16:59:28.292 INFO:tasks.workunit.client.1.smithi144.stderr:10+0 records out 2024-04-07T16:59:28.292 INFO:tasks.workunit.client.1.smithi144.stderr:10240 bytes (10 kB, 10 KiB) copied, 8.3449e-05 s, 123 MB/s 2024-04-07T16:59:28.292 INFO:tasks.workunit.client.1.smithi144.stderr:+ dd if=/bin/rm of=/tmp/rbd_import_export_103522/img bs=1k count=100 seek=1000 2024-04-07T16:59:28.293 INFO:tasks.workunit.client.1.smithi144.stderr:58+1 records in 2024-04-07T16:59:28.293 INFO:tasks.workunit.client.1.smithi144.stderr:58+1 records out 2024-04-07T16:59:28.293 INFO:tasks.workunit.client.1.smithi144.stderr:59904 bytes (60 kB, 58 KiB) copied, 0.00019977 s, 300 MB/s 2024-04-07T16:59:28.293 INFO:tasks.workunit.client.1.smithi144.stderr:+ dd if=/bin/ls of=/tmp/rbd_import_export_103522/img bs=1k seek=10000 2024-04-07T16:59:28.294 INFO:tasks.workunit.client.1.smithi144.stderr:134+1 records in 2024-04-07T16:59:28.294 INFO:tasks.workunit.client.1.smithi144.stderr:134+1 records out 2024-04-07T16:59:28.294 INFO:tasks.workunit.client.1.smithi144.stderr:138208 bytes (138 kB, 135 KiB) copied, 0.000554248 s, 249 MB/s 2024-04-07T16:59:28.294 INFO:tasks.workunit.client.1.smithi144.stderr:+ dd if=/bin/ln of=/tmp/rbd_import_export_103522/img bs=1k seek=100000 2024-04-07T16:59:28.294 INFO:tasks.workunit.client.1.smithi144.stderr:58+1 records in 2024-04-07T16:59:28.294 INFO:tasks.workunit.client.1.smithi144.stderr:58+1 records out 2024-04-07T16:59:28.294 INFO:tasks.workunit.client.1.smithi144.stderr:59904 bytes (60 kB, 58 KiB) copied, 0.000209083 s, 287 MB/s 2024-04-07T16:59:28.294 INFO:tasks.workunit.client.1.smithi144.stderr:+ dd if=/bin/grep of=/tmp/rbd_import_export_103522/img bs=1k seek=1000000 2024-04-07T16:59:28.295 INFO:tasks.workunit.client.1.smithi144.stderr:178+1 records in 2024-04-07T16:59:28.295 INFO:tasks.workunit.client.1.smithi144.stderr:178+1 records out 2024-04-07T16:59:28.295 INFO:tasks.workunit.client.1.smithi144.stderr:182728 bytes (183 kB, 178 KiB) copied, 0.000476919 s, 383 MB/s 2024-04-07T16:59:28.295 INFO:tasks.workunit.client.1.smithi144.stderr:+ rbd rm testimg 2024-04-07T16:59:28.365 INFO:tasks.workunit.client.1.smithi144.stderr: Removing image: 0% complete...failed. 2024-04-07T16:59:28.365 INFO:tasks.workunit.client.1.smithi144.stderr:rbd: delete error: (2) No such file or directory 2024-04-07T16:59:28.368 INFO:tasks.workunit.client.1.smithi144.stderr:+ true 2024-04-07T16:59:28.368 INFO:tasks.workunit.client.1.smithi144.stderr:+ rbd import --new-format /tmp/rbd_import_export_103522/img testimg 2024-04-07T16:59:28.687 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:59:28 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:59:28] "GET /metrics HTTP/1.1" 200 34064 "" "Prometheus/2.43.0" 2024-04-07T16:59:28.972 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:59:28 smithi100 bash[74942]: audit 2024-04-07T16:59:27.689604+0000 mon.a (mon.0) 2225 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:59:28.972 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:59:28 smithi100 bash[74942]: audit 2024-04-07T16:59:27.696497+0000 mon.a (mon.0) 2226 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:59:28.972 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:59:28 smithi100 bash[74942]: cluster 2024-04-07T16:59:27.708903+0000 mgr.y (mgr.25175) 1323 : cluster [DBG] pgmap v1965: 105 pgs: 105 active+clean; 583 KiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 0 B/s wr, 119 op/s 2024-04-07T16:59:28.972 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:59:28 smithi100 bash[74942]: audit 2024-04-07T16:59:27.894272+0000 mon.a (mon.0) 2227 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:59:28.972 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:59:28 smithi100 bash[74942]: audit 2024-04-07T16:59:27.906240+0000 mon.a (mon.0) 2228 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:59:28.972 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:59:28 smithi100 bash[76202]: audit 2024-04-07T16:59:27.689604+0000 mon.a (mon.0) 2225 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:59:28.972 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:59:28 smithi100 bash[76202]: audit 2024-04-07T16:59:27.696497+0000 mon.a (mon.0) 2226 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:59:28.972 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:59:28 smithi100 bash[76202]: cluster 2024-04-07T16:59:27.708903+0000 mgr.y (mgr.25175) 1323 : cluster [DBG] pgmap v1965: 105 pgs: 105 active+clean; 583 KiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 0 B/s wr, 119 op/s 2024-04-07T16:59:28.973 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:59:28 smithi100 bash[76202]: audit 2024-04-07T16:59:27.894272+0000 mon.a (mon.0) 2227 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:59:28.973 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:59:28 smithi100 bash[76202]: audit 2024-04-07T16:59:27.906240+0000 mon.a (mon.0) 2228 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:59:29.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:59:28 smithi144 bash[66654]: audit 2024-04-07T16:59:27.689604+0000 mon.a (mon.0) 2225 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:59:29.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:59:28 smithi144 bash[66654]: audit 2024-04-07T16:59:27.696497+0000 mon.a (mon.0) 2226 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:59:29.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:59:28 smithi144 bash[66654]: cluster 2024-04-07T16:59:27.708903+0000 mgr.y (mgr.25175) 1323 : cluster [DBG] pgmap v1965: 105 pgs: 105 active+clean; 583 KiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 0 B/s wr, 119 op/s 2024-04-07T16:59:29.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:59:28 smithi144 bash[66654]: audit 2024-04-07T16:59:27.894272+0000 mon.a (mon.0) 2227 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:59:29.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:59:28 smithi144 bash[66654]: audit 2024-04-07T16:59:27.906240+0000 mon.a (mon.0) 2228 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:59:30.199 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:59:29 smithi100 bash[74942]: cluster 2024-04-07T16:59:28.911338+0000 mon.a (mon.0) 2229 : cluster [DBG] osdmap e1068: 8 total, 8 up, 8 in 2024-04-07T16:59:30.200 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:59:29 smithi100 bash[74942]: audit 2024-04-07T16:59:28.923865+0000 mon.a (mon.0) 2230 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:59:30.200 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:59:29 smithi100 bash[74942]: audit 2024-04-07T16:59:28.935714+0000 mon.a (mon.0) 2231 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:59:30.200 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:59:29 smithi100 bash[74942]: cluster 2024-04-07T16:59:29.709671+0000 mgr.y (mgr.25175) 1324 : cluster [DBG] pgmap v1967: 105 pgs: 105 active+clean; 583 KiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 0 B/s wr, 76 op/s 2024-04-07T16:59:30.200 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:59:29 smithi100 bash[76202]: cluster 2024-04-07T16:59:28.911338+0000 mon.a (mon.0) 2229 : cluster [DBG] osdmap e1068: 8 total, 8 up, 8 in 2024-04-07T16:59:30.200 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:59:29 smithi100 bash[76202]: audit 2024-04-07T16:59:28.923865+0000 mon.a (mon.0) 2230 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:59:30.200 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:59:29 smithi100 bash[76202]: audit 2024-04-07T16:59:28.935714+0000 mon.a (mon.0) 2231 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:59:30.200 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:59:29 smithi100 bash[76202]: cluster 2024-04-07T16:59:29.709671+0000 mgr.y (mgr.25175) 1324 : cluster [DBG] pgmap v1967: 105 pgs: 105 active+clean; 583 KiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 0 B/s wr, 76 op/s 2024-04-07T16:59:30.364 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:59:29 smithi144 bash[66654]: cluster 2024-04-07T16:59:28.911338+0000 mon.a (mon.0) 2229 : cluster [DBG] osdmap e1068: 8 total, 8 up, 8 in 2024-04-07T16:59:30.364 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:59:29 smithi144 bash[66654]: audit 2024-04-07T16:59:28.923865+0000 mon.a (mon.0) 2230 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:59:30.364 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:59:29 smithi144 bash[66654]: audit 2024-04-07T16:59:28.935714+0000 mon.a (mon.0) 2231 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:59:30.364 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:59:29 smithi144 bash[66654]: cluster 2024-04-07T16:59:29.709671+0000 mgr.y (mgr.25175) 1324 : cluster [DBG] pgmap v1967: 105 pgs: 105 active+clean; 583 KiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 0 B/s wr, 76 op/s 2024-04-07T16:59:30.577 INFO:tasks.workunit.client.1.smithi144.stderr: Importing image: 1% complete... Importing image: 2% complete... Importing image: 3% complete... Importing image: 4% complete... Importing image: 5% complete... Importing image: 6% complete... Importing image: 7% complete... Importing image: 8% complete... Importing image: 9% complete... Importing image: 10% complete... Importing image: 11% complete... Importing image: 12% complete... Importing image: 13% complete... Importing image: 14% complete... Importing image: 15% complete... Importing image: 16% complete... Importing image: 17% complete... Importing image: 18% complete... Importing image: 19% complete... Importing image: 20% complete... Importing image: 21% complete... Importing image: 22% complete... Importing image: 23% complete... Importing image: 24% complete... Importing image: 25% complete... Importing image: 26% complete... Importing image: 27% complete... Importing image: 28% complete... Importing image: 29% complete... Importing image: 30% complete... Importing image: 31% complete... Importing image: 32% complete... Importing image: 33% complete... Importing image: 34% complete... Importing image: 35% complete... Importing image: 36% complete... Importing image: 37% complete... Importing image: 38% complete... Importing image: 39% complete... Importing image: 40% complete... Importing image: 41% complete... Importing image: 42% complete... Importing image: 43% complete... Importing image: 44% complete... Importing image: 45% complete... Importing image: 46% complete... Importing image: 47% complete... Importing image: 48% complete... Importing image: 49% complete... Importing image: 50% complete... Importing image: 51% complete... Importing image: 52% complete... Importing image: 53% complete... Importing image: 54% complete... Importing image: 55% complete... Importing image: 56% complete... Importing image: 57% complete... Importing image: 58% complete... Importing image: 59% complete... Importing image: 60% complete... Importing image: 61% complete... Importing image: 62% complete... Importing image: 63% complete... Importing image: 64% complete... Importing image: 65% complete... Importing image: 66% complete... Importing image: 67% complete... Importing image: 68% complete... Importing image: 69% complete... Importing image: 70% complete... Importing image: 71% complete... Importing image: 72% complete... Importing image: 73% complete... Importing image: 74% complete... Importing image: 75% complete... Importing image: 76% complete... Importing image: 77% complete... Importing image: 78% complete... Importing image: 79% complete... Importing image: 80% complete... Importing image: 81% complete... Importing image: 82% complete... Importing image: 83% complete... Importing image: 84% complete... Importing image: 85% complete... Importing image: 86% complete... Importing image: 87% complete... Importing image: 88% complete... Importing image: 89% complete... Importing image: 90% complete... Importing image: 91% complete... Importing image: 92% complete... Importing image: 93% complete... Importing image: 94% complete... Importing image: 95% complete... Importing image: 96% complete... Importing image: 97% complete... Importing image: 98% complete... Importing image: 99% complete... Importing image: 100% complete...done. 2024-04-07T16:59:30.577 INFO:tasks.workunit.client.1.smithi144.stderr:rbd: --new-format is deprecated, use --image-format 2 2024-04-07T16:59:30.581 INFO:tasks.workunit.client.1.smithi144.stderr:+ rbd export testimg /tmp/rbd_import_export_103522/img2 2024-04-07T16:59:30.797 INFO:tasks.workunit.client.1.smithi144.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2024-04-07T16:59:30.803 INFO:tasks.workunit.client.1.smithi144.stderr:+ rbd export testimg - 2024-04-07T16:59:31.199 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:59:30 smithi100 bash[74942]: cluster 2024-04-07T16:59:29.927407+0000 mon.a (mon.0) 2232 : cluster [DBG] osdmap e1069: 8 total, 8 up, 8 in 2024-04-07T16:59:31.200 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:59:30 smithi100 bash[76202]: cluster 2024-04-07T16:59:29.927407+0000 mon.a (mon.0) 2232 : cluster [DBG] osdmap e1069: 8 total, 8 up, 8 in 2024-04-07T16:59:31.364 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:59:30 smithi144 bash[66654]: cluster 2024-04-07T16:59:29.927407+0000 mon.a (mon.0) 2232 : cluster [DBG] osdmap e1069: 8 total, 8 up, 8 in 2024-04-07T16:59:31.504 INFO:tasks.workunit.client.1.smithi144.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2024-04-07T16:59:31.510 INFO:tasks.workunit.client.1.smithi144.stderr:+ rbd rm testimg 2024-04-07T16:59:31.769 INFO:tasks.workunit.client.1.smithi144.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-04-07T16:59:31.776 INFO:tasks.workunit.client.1.smithi144.stderr:+ cmp /tmp/rbd_import_export_103522/img /tmp/rbd_import_export_103522/img2 2024-04-07T16:59:32.200 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:59:31 smithi100 bash[74942]: cluster 2024-04-07T16:59:31.710349+0000 mgr.y (mgr.25175) 1325 : cluster [DBG] pgmap v1969: 105 pgs: 105 active+clean; 583 KiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 127 B/s wr, 0 op/s 2024-04-07T16:59:32.200 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:59:31 smithi100 bash[74942]: audit 2024-04-07T16:59:31.792616+0000 mon.a (mon.0) 2233 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:59:32.200 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:59:31 smithi100 bash[76202]: cluster 2024-04-07T16:59:31.710349+0000 mgr.y (mgr.25175) 1325 : cluster [DBG] pgmap v1969: 105 pgs: 105 active+clean; 583 KiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 127 B/s wr, 0 op/s 2024-04-07T16:59:32.200 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:59:31 smithi100 bash[76202]: audit 2024-04-07T16:59:31.792616+0000 mon.a (mon.0) 2233 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:59:32.313 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:59:31 smithi144 bash[66654]: cluster 2024-04-07T16:59:31.710349+0000 mgr.y (mgr.25175) 1325 : cluster [DBG] pgmap v1969: 105 pgs: 105 active+clean; 583 KiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 127 B/s wr, 0 op/s 2024-04-07T16:59:32.313 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:59:31 smithi144 bash[66654]: audit 2024-04-07T16:59:31.792616+0000 mon.a (mon.0) 2233 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:59:32.477 INFO:tasks.workunit.client.1.smithi144.stderr:+ cmp /tmp/rbd_import_export_103522/img /tmp/rbd_import_export_103522/img3 2024-04-07T16:59:32.614 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:59:32 smithi144 bash[95233]: ts=2024-04-07T16:59:32.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:59:32.984 INFO:tasks.workunit.client.1.smithi144.stderr:+ rm /tmp/rbd_import_export_103522/img2 /tmp/rbd_import_export_103522/img3 2024-04-07T16:59:33.162 INFO:tasks.workunit.client.1.smithi144.stderr:+ rbd import --new-format - testimg 2024-04-07T16:59:33.699 INFO:tasks.workunit.client.1.smithi144.stderr: Importing image: 100% complete...done. 2024-04-07T16:59:33.700 INFO:tasks.workunit.client.1.smithi144.stderr:rbd: --new-format is deprecated, use --image-format 2 2024-04-07T16:59:33.704 INFO:tasks.workunit.client.1.smithi144.stderr:+ rbd export testimg /tmp/rbd_import_export_103522/img2 2024-04-07T16:59:33.930 INFO:tasks.workunit.client.1.smithi144.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2024-04-07T16:59:33.936 INFO:tasks.workunit.client.1.smithi144.stderr:+ rbd export testimg - 2024-04-07T16:59:34.113 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:59:33 smithi144 bash[66654]: cluster 2024-04-07T16:59:33.711521+0000 mgr.y (mgr.25175) 1326 : cluster [DBG] pgmap v1970: 105 pgs: 105 active+clean; 583 KiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 117 KiB/s rd, 50 KiB/s wr, 26 op/s 2024-04-07T16:59:34.199 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:59:33 smithi100 bash[74942]: cluster 2024-04-07T16:59:33.711521+0000 mgr.y (mgr.25175) 1326 : cluster [DBG] pgmap v1970: 105 pgs: 105 active+clean; 583 KiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 117 KiB/s rd, 50 KiB/s wr, 26 op/s 2024-04-07T16:59:34.200 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:59:33 smithi100 bash[76202]: cluster 2024-04-07T16:59:33.711521+0000 mgr.y (mgr.25175) 1326 : cluster [DBG] pgmap v1970: 105 pgs: 105 active+clean; 583 KiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 117 KiB/s rd, 50 KiB/s wr, 26 op/s 2024-04-07T16:59:34.659 INFO:tasks.workunit.client.1.smithi144.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2024-04-07T16:59:34.665 INFO:tasks.workunit.client.1.smithi144.stderr:+ rbd rm testimg 2024-04-07T16:59:34.827 INFO:tasks.workunit.client.1.smithi144.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-04-07T16:59:34.834 INFO:tasks.workunit.client.1.smithi144.stderr:+ cmp /tmp/rbd_import_export_103522/img /tmp/rbd_import_export_103522/img2 2024-04-07T16:59:35.529 INFO:tasks.workunit.client.1.smithi144.stderr:+ cmp /tmp/rbd_import_export_103522/img /tmp/rbd_import_export_103522/img3 2024-04-07T16:59:36.006 INFO:tasks.workunit.client.1.smithi144.stderr:+ rm /tmp/rbd_import_export_103522/img /tmp/rbd_import_export_103522/img2 /tmp/rbd_import_export_103522/img3 2024-04-07T16:59:36.113 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:59:35 smithi144 bash[66654]: cluster 2024-04-07T16:59:35.712455+0000 mgr.y (mgr.25175) 1327 : cluster [DBG] pgmap v1971: 105 pgs: 105 active+clean; 583 KiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 142 KiB/s rd, 59 KiB/s wr, 41 op/s 2024-04-07T16:59:36.200 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:59:35 smithi100 bash[74942]: cluster 2024-04-07T16:59:35.712455+0000 mgr.y (mgr.25175) 1327 : cluster [DBG] pgmap v1971: 105 pgs: 105 active+clean; 583 KiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 142 KiB/s rd, 59 KiB/s wr, 41 op/s 2024-04-07T16:59:36.200 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:59:35 smithi100 bash[76202]: cluster 2024-04-07T16:59:35.712455+0000 mgr.y (mgr.25175) 1327 : cluster [DBG] pgmap v1971: 105 pgs: 105 active+clean; 583 KiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 142 KiB/s rd, 59 KiB/s wr, 41 op/s 2024-04-07T16:59:38.700 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:59:38 smithi100 bash[74942]: cluster 2024-04-07T16:59:37.489736+0000 mon.a (mon.0) 2234 : cluster [DBG] osdmap e1070: 8 total, 8 up, 8 in 2024-04-07T16:59:38.700 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:59:38 smithi100 bash[74942]: cluster 2024-04-07T16:59:37.713129+0000 mgr.y (mgr.25175) 1328 : cluster [DBG] pgmap v1973: 105 pgs: 105 active+clean; 583 KiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 261 KiB/s rd, 103 KiB/s wr, 98 op/s 2024-04-07T16:59:38.700 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:59:38 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:59:38] "GET /metrics HTTP/1.1" 200 34044 "" "Prometheus/2.43.0" 2024-04-07T16:59:38.700 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:59:38 smithi100 bash[76202]: cluster 2024-04-07T16:59:37.489736+0000 mon.a (mon.0) 2234 : cluster [DBG] osdmap e1070: 8 total, 8 up, 8 in 2024-04-07T16:59:38.701 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:59:38 smithi100 bash[76202]: cluster 2024-04-07T16:59:37.713129+0000 mgr.y (mgr.25175) 1328 : cluster [DBG] pgmap v1973: 105 pgs: 105 active+clean; 583 KiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 261 KiB/s rd, 103 KiB/s wr, 98 op/s 2024-04-07T16:59:38.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:59:38 smithi144 bash[66654]: cluster 2024-04-07T16:59:37.489736+0000 mon.a (mon.0) 2234 : cluster [DBG] osdmap e1070: 8 total, 8 up, 8 in 2024-04-07T16:59:38.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:59:38 smithi144 bash[66654]: cluster 2024-04-07T16:59:37.713129+0000 mgr.y (mgr.25175) 1328 : cluster [DBG] pgmap v1973: 105 pgs: 105 active+clean; 583 KiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 261 KiB/s rd, 103 KiB/s wr, 98 op/s 2024-04-07T16:59:40.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:59:39 smithi144 bash[66654]: cluster 2024-04-07T16:59:39.714123+0000 mgr.y (mgr.25175) 1329 : cluster [DBG] pgmap v1974: 105 pgs: 105 active+clean; 583 KiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 242 KiB/s rd, 98 KiB/s wr, 82 op/s 2024-04-07T16:59:40.199 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:59:39 smithi100 bash[74942]: cluster 2024-04-07T16:59:39.714123+0000 mgr.y (mgr.25175) 1329 : cluster [DBG] pgmap v1974: 105 pgs: 105 active+clean; 583 KiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 242 KiB/s rd, 98 KiB/s wr, 82 op/s 2024-04-07T16:59:40.200 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:59:39 smithi100 bash[76202]: cluster 2024-04-07T16:59:39.714123+0000 mgr.y (mgr.25175) 1329 : cluster [DBG] pgmap v1974: 105 pgs: 105 active+clean; 583 KiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 242 KiB/s rd, 98 KiB/s wr, 82 op/s 2024-04-07T16:59:41.850 INFO:tasks.workunit.client.1.smithi144.stderr:+ rbd help export 2024-04-07T16:59:41.851 INFO:tasks.workunit.client.1.smithi144.stderr:+ grep -q export-format 2024-04-07T16:59:41.872 INFO:tasks.workunit.client.1.smithi144.stderr:+ dd if=/bin/dd of=/tmp/rbd_import_export_103522/img bs=1k count=10 seek=100 2024-04-07T16:59:41.873 INFO:tasks.workunit.client.1.smithi144.stderr:10+0 records in 2024-04-07T16:59:41.873 INFO:tasks.workunit.client.1.smithi144.stderr:10+0 records out 2024-04-07T16:59:41.873 INFO:tasks.workunit.client.1.smithi144.stderr:10240 bytes (10 kB, 10 KiB) copied, 0.000140479 s, 72.9 MB/s 2024-04-07T16:59:41.873 INFO:tasks.workunit.client.1.smithi144.stderr:+ rbd import --new-format /tmp/rbd_import_export_103522/img testimg 2024-04-07T16:59:41.986 INFO:tasks.workunit.client.1.smithi144.stderr: Importing image: 100% complete...done. 2024-04-07T16:59:41.987 INFO:tasks.workunit.client.1.smithi144.stderr:rbd: --new-format is deprecated, use --image-format 2 2024-04-07T16:59:41.991 INFO:tasks.workunit.client.1.smithi144.stderr:+ rbd snap create testimg@snap 2024-04-07T16:59:42.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:59:41 smithi144 bash[66654]: cluster 2024-04-07T16:59:41.714891+0000 mgr.y (mgr.25175) 1330 : cluster [DBG] pgmap v1975: 105 pgs: 105 active+clean; 583 KiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 237 KiB/s rd, 96 KiB/s wr, 80 op/s 2024-04-07T16:59:42.200 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:59:41 smithi100 bash[74942]: cluster 2024-04-07T16:59:41.714891+0000 mgr.y (mgr.25175) 1330 : cluster [DBG] pgmap v1975: 105 pgs: 105 active+clean; 583 KiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 237 KiB/s rd, 96 KiB/s wr, 80 op/s 2024-04-07T16:59:42.200 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:59:41 smithi100 bash[76202]: cluster 2024-04-07T16:59:41.714891+0000 mgr.y (mgr.25175) 1330 : cluster [DBG] pgmap v1975: 105 pgs: 105 active+clean; 583 KiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 237 KiB/s rd, 96 KiB/s wr, 80 op/s 2024-04-07T16:59:42.614 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:59:42 smithi144 bash[95233]: ts=2024-04-07T16:59:42.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:59:42.857 INFO:tasks.workunit.client.1.smithi144.stderr: Creating snap: 10% complete... Creating snap: 100% complete...done. 2024-04-07T16:59:42.876 INFO:tasks.workunit.client.1.smithi144.stderr:+ rbd image-meta set testimg key1 value1 2024-04-07T16:59:42.965 INFO:tasks.workunit.client.1.smithi144.stderr:+ rbd image-meta list testimg 2024-04-07T16:59:43.026 INFO:tasks.workunit.client.1.smithi144.stderr:+ IMAGEMETA_BEFORE=There is 1 metadatum on this image: 2024-04-07T16:59:43.027 INFO:tasks.workunit.client.1.smithi144.stderr: 2024-04-07T16:59:43.027 INFO:tasks.workunit.client.1.smithi144.stderr:Key Value 2024-04-07T16:59:43.027 INFO:tasks.workunit.client.1.smithi144.stderr:key1 value1 2024-04-07T16:59:43.027 INFO:tasks.workunit.client.1.smithi144.stderr:+ rbd export --export-format 2 testimg /tmp/rbd_import_export_103522/img_v2 2024-04-07T16:59:43.080 INFO:tasks.workunit.client.1.smithi144.stderr: Exporting image: 100% complete... Exporting image: 100% complete...done. 2024-04-07T16:59:43.085 INFO:tasks.workunit.client.1.smithi144.stderr:+ rbd import --export-format 2 /tmp/rbd_import_export_103522/img_v2 testimg_import 2024-04-07T16:59:43.903 INFO:tasks.workunit.client.1.smithi144.stderr: Importing image: 50% complete... Importing image: 100% complete... Importing image: 100% complete...done. 2024-04-07T16:59:43.912 INFO:tasks.workunit.client.1.smithi144.stderr:+ rbd info testimg_import 2024-04-07T16:59:43.964 INFO:tasks.workunit.client.1.smithi144.stdout:rbd image 'testimg_import': 2024-04-07T16:59:43.964 INFO:tasks.workunit.client.1.smithi144.stdout: size 110 KiB in 1 objects 2024-04-07T16:59:43.964 INFO:tasks.workunit.client.1.smithi144.stdout: order 22 (4 MiB objects) 2024-04-07T16:59:43.965 INFO:tasks.workunit.client.1.smithi144.stdout: snapshot_count: 1 2024-04-07T16:59:43.965 INFO:tasks.workunit.client.1.smithi144.stdout: id: d51f48101d6c 2024-04-07T16:59:43.965 INFO:tasks.workunit.client.1.smithi144.stdout: block_name_prefix: rbd_data.d51f48101d6c 2024-04-07T16:59:43.965 INFO:tasks.workunit.client.1.smithi144.stdout: format: 2 2024-04-07T16:59:43.965 INFO:tasks.workunit.client.1.smithi144.stdout: features: layering, exclusive-lock, object-map, fast-diff, deep-flatten 2024-04-07T16:59:43.965 INFO:tasks.workunit.client.1.smithi144.stdout: op_features: 2024-04-07T16:59:43.965 INFO:tasks.workunit.client.1.smithi144.stdout: flags: 2024-04-07T16:59:43.965 INFO:tasks.workunit.client.1.smithi144.stdout: create_timestamp: Sun Apr 7 16:59:43 2024 2024-04-07T16:59:43.965 INFO:tasks.workunit.client.1.smithi144.stdout: access_timestamp: Sun Apr 7 16:59:43 2024 2024-04-07T16:59:43.965 INFO:tasks.workunit.client.1.smithi144.stdout: modify_timestamp: Sun Apr 7 16:59:43 2024 2024-04-07T16:59:43.968 INFO:tasks.workunit.client.1.smithi144.stderr:+ rbd info testimg_import@snap 2024-04-07T16:59:44.018 INFO:tasks.workunit.client.1.smithi144.stdout:rbd image 'testimg_import': 2024-04-07T16:59:44.018 INFO:tasks.workunit.client.1.smithi144.stdout: size 110 KiB in 1 objects 2024-04-07T16:59:44.018 INFO:tasks.workunit.client.1.smithi144.stdout: order 22 (4 MiB objects) 2024-04-07T16:59:44.018 INFO:tasks.workunit.client.1.smithi144.stdout: snapshot_count: 1 2024-04-07T16:59:44.018 INFO:tasks.workunit.client.1.smithi144.stdout: id: d51f48101d6c 2024-04-07T16:59:44.018 INFO:tasks.workunit.client.1.smithi144.stdout: block_name_prefix: rbd_data.d51f48101d6c 2024-04-07T16:59:44.018 INFO:tasks.workunit.client.1.smithi144.stdout: format: 2 2024-04-07T16:59:44.018 INFO:tasks.workunit.client.1.smithi144.stdout: features: layering, exclusive-lock, object-map, fast-diff, deep-flatten 2024-04-07T16:59:44.019 INFO:tasks.workunit.client.1.smithi144.stdout: op_features: 2024-04-07T16:59:44.019 INFO:tasks.workunit.client.1.smithi144.stdout: flags: 2024-04-07T16:59:44.019 INFO:tasks.workunit.client.1.smithi144.stdout: create_timestamp: Sun Apr 7 16:59:43 2024 2024-04-07T16:59:44.019 INFO:tasks.workunit.client.1.smithi144.stdout: access_timestamp: Sun Apr 7 16:59:43 2024 2024-04-07T16:59:44.019 INFO:tasks.workunit.client.1.smithi144.stdout: modify_timestamp: Sun Apr 7 16:59:43 2024 2024-04-07T16:59:44.019 INFO:tasks.workunit.client.1.smithi144.stdout: protected: False 2024-04-07T16:59:44.022 INFO:tasks.workunit.client.1.smithi144.stderr:+ rbd image-meta list testimg_import 2024-04-07T16:59:44.083 INFO:tasks.workunit.client.1.smithi144.stderr:+ IMAGEMETA_AFTER=There is 1 metadatum on this image: 2024-04-07T16:59:44.084 INFO:tasks.workunit.client.1.smithi144.stderr: 2024-04-07T16:59:44.084 INFO:tasks.workunit.client.1.smithi144.stderr:Key Value 2024-04-07T16:59:44.084 INFO:tasks.workunit.client.1.smithi144.stderr:key1 value1 2024-04-07T16:59:44.084 INFO:tasks.workunit.client.1.smithi144.stderr:+ [ There is 1 metadatum on this image: 2024-04-07T16:59:44.084 INFO:tasks.workunit.client.1.smithi144.stderr: 2024-04-07T16:59:44.084 INFO:tasks.workunit.client.1.smithi144.stderr:Key Value 2024-04-07T16:59:44.084 INFO:tasks.workunit.client.1.smithi144.stderr:key1 value1 = There is 1 metadatum on this image: 2024-04-07T16:59:44.084 INFO:tasks.workunit.client.1.smithi144.stderr: 2024-04-07T16:59:44.084 INFO:tasks.workunit.client.1.smithi144.stderr:Key Value 2024-04-07T16:59:44.084 INFO:tasks.workunit.client.1.smithi144.stderr:key1 value1 ] 2024-04-07T16:59:44.085 INFO:tasks.workunit.client.1.smithi144.stderr:+ rbd export testimg_import /tmp/rbd_import_export_103522/img_import 2024-04-07T16:59:44.113 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:59:43 smithi144 bash[66654]: cluster 2024-04-07T16:59:42.834848+0000 mon.a (mon.0) 2235 : cluster [DBG] osdmap e1071: 8 total, 8 up, 8 in 2024-04-07T16:59:44.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:59:43 smithi144 bash[66654]: cluster 2024-04-07T16:59:43.715718+0000 mgr.y (mgr.25175) 1331 : cluster [DBG] pgmap v1977: 105 pgs: 105 active+clean; 583 KiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 154 KiB/s rd, 61 KiB/s wr, 59 op/s 2024-04-07T16:59:44.131 INFO:tasks.workunit.client.1.smithi144.stderr: Exporting image: 100% complete...done. 2024-04-07T16:59:44.135 INFO:tasks.workunit.client.1.smithi144.stderr:+ compare_files_and_ondisk_sizes /tmp/rbd_import_export_103522/img /tmp/rbd_import_export_103522/img_import 2024-04-07T16:59:44.135 INFO:tasks.workunit.client.1.smithi144.stderr:+ cmp -l /tmp/rbd_import_export_103522/img /tmp/rbd_import_export_103522/img_import 2024-04-07T16:59:44.136 INFO:tasks.workunit.client.1.smithi144.stderr:+ stat /tmp/rbd_import_export_103522/img --format %b 2024-04-07T16:59:44.136 INFO:tasks.workunit.client.1.smithi144.stderr:+ origsize=24 2024-04-07T16:59:44.137 INFO:tasks.workunit.client.1.smithi144.stderr:+ stat /tmp/rbd_import_export_103522/img_import --format %b 2024-04-07T16:59:44.137 INFO:tasks.workunit.client.1.smithi144.stderr:+ exportsize=224 2024-04-07T16:59:44.137 INFO:tasks.workunit.client.1.smithi144.stderr:+ difference=200 2024-04-07T16:59:44.137 INFO:tasks.workunit.client.1.smithi144.stderr:+ difference=200 2024-04-07T16:59:44.137 INFO:tasks.workunit.client.1.smithi144.stderr:+ test 200 -ge 0 -a 200 -lt 4096 2024-04-07T16:59:44.137 INFO:tasks.workunit.client.1.smithi144.stderr:+ rbd export testimg@snap /tmp/rbd_import_export_103522/img_snap 2024-04-07T16:59:44.185 INFO:tasks.workunit.client.1.smithi144.stderr: Exporting image: 100% complete...done. 2024-04-07T16:59:44.188 INFO:tasks.workunit.client.1.smithi144.stderr:+ rbd export testimg_import@snap /tmp/rbd_import_export_103522/img_snap_import 2024-04-07T16:59:44.199 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:59:43 smithi100 bash[74942]: cluster 2024-04-07T16:59:42.834848+0000 mon.a (mon.0) 2235 : cluster [DBG] osdmap e1071: 8 total, 8 up, 8 in 2024-04-07T16:59:44.200 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:59:43 smithi100 bash[74942]: cluster 2024-04-07T16:59:43.715718+0000 mgr.y (mgr.25175) 1331 : cluster [DBG] pgmap v1977: 105 pgs: 105 active+clean; 583 KiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 154 KiB/s rd, 61 KiB/s wr, 59 op/s 2024-04-07T16:59:44.200 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:59:43 smithi100 bash[76202]: cluster 2024-04-07T16:59:42.834848+0000 mon.a (mon.0) 2235 : cluster [DBG] osdmap e1071: 8 total, 8 up, 8 in 2024-04-07T16:59:44.200 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:59:43 smithi100 bash[76202]: cluster 2024-04-07T16:59:43.715718+0000 mgr.y (mgr.25175) 1331 : cluster [DBG] pgmap v1977: 105 pgs: 105 active+clean; 583 KiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 154 KiB/s rd, 61 KiB/s wr, 59 op/s 2024-04-07T16:59:44.236 INFO:tasks.workunit.client.1.smithi144.stderr: Exporting image: 100% complete...done. 2024-04-07T16:59:44.240 INFO:tasks.workunit.client.1.smithi144.stderr:+ compare_files_and_ondisk_sizes /tmp/rbd_import_export_103522/img_snap /tmp/rbd_import_export_103522/img_snap_import 2024-04-07T16:59:44.240 INFO:tasks.workunit.client.1.smithi144.stderr:+ cmp -l /tmp/rbd_import_export_103522/img_snap /tmp/rbd_import_export_103522/img_snap_import 2024-04-07T16:59:44.241 INFO:tasks.workunit.client.1.smithi144.stderr:+ stat /tmp/rbd_import_export_103522/img_snap --format %b 2024-04-07T16:59:44.242 INFO:tasks.workunit.client.1.smithi144.stderr:+ origsize=224 2024-04-07T16:59:44.242 INFO:tasks.workunit.client.1.smithi144.stderr:+ stat /tmp/rbd_import_export_103522/img_snap_import --format %b 2024-04-07T16:59:44.242 INFO:tasks.workunit.client.1.smithi144.stderr:+ exportsize=224 2024-04-07T16:59:44.243 INFO:tasks.workunit.client.1.smithi144.stderr:+ difference=0 2024-04-07T16:59:44.243 INFO:tasks.workunit.client.1.smithi144.stderr:+ difference=0 2024-04-07T16:59:44.243 INFO:tasks.workunit.client.1.smithi144.stderr:+ test 0 -ge 0 -a 0 -lt 4096 2024-04-07T16:59:44.243 INFO:tasks.workunit.client.1.smithi144.stderr:+ rm /tmp/rbd_import_export_103522/img_v2 2024-04-07T16:59:44.243 INFO:tasks.workunit.client.1.smithi144.stderr:+ rm /tmp/rbd_import_export_103522/img_import 2024-04-07T16:59:44.243 INFO:tasks.workunit.client.1.smithi144.stderr:+ rm /tmp/rbd_import_export_103522/img_snap 2024-04-07T16:59:44.244 INFO:tasks.workunit.client.1.smithi144.stderr:+ rm /tmp/rbd_import_export_103522/img_snap_import 2024-04-07T16:59:44.244 INFO:tasks.workunit.client.1.smithi144.stderr:+ rbd snap rm testimg_import@snap 2024-04-07T16:59:44.866 INFO:tasks.workunit.client.1.smithi144.stderr: Removing snap: 100% complete...done. 2024-04-07T16:59:44.886 INFO:tasks.workunit.client.1.smithi144.stderr:+ rbd remove testimg_import 2024-04-07T16:59:45.113 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:59:44 smithi144 bash[66654]: cluster 2024-04-07T16:59:43.841770+0000 mon.a (mon.0) 2236 : cluster [DBG] osdmap e1072: 8 total, 8 up, 8 in 2024-04-07T16:59:45.122 INFO:tasks.workunit.client.1.smithi144.stderr: Removing image: 100% complete...done. 2024-04-07T16:59:45.126 INFO:tasks.workunit.client.1.smithi144.stderr:+ rbd snap rm testimg@snap 2024-04-07T16:59:45.199 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:59:44 smithi100 bash[74942]: cluster 2024-04-07T16:59:43.841770+0000 mon.a (mon.0) 2236 : cluster [DBG] osdmap e1072: 8 total, 8 up, 8 in 2024-04-07T16:59:45.200 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:59:44 smithi100 bash[76202]: cluster 2024-04-07T16:59:43.841770+0000 mon.a (mon.0) 2236 : cluster [DBG] osdmap e1072: 8 total, 8 up, 8 in 2024-04-07T16:59:45.881 INFO:tasks.workunit.client.1.smithi144.stderr: Removing snap: 100% complete...done. 2024-04-07T16:59:45.908 INFO:tasks.workunit.client.1.smithi144.stderr:+ rbd rm testimg 2024-04-07T16:59:46.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:59:45 smithi144 bash[66654]: cluster 2024-04-07T16:59:44.842971+0000 mon.a (mon.0) 2237 : cluster [DBG] osdmap e1073: 8 total, 8 up, 8 in 2024-04-07T16:59:46.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:59:45 smithi144 bash[66654]: cluster 2024-04-07T16:59:45.716484+0000 mgr.y (mgr.25175) 1332 : cluster [DBG] pgmap v1980: 105 pgs: 105 active+clean; 693 KiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 80 KiB/s rd, 5.0 KiB/s wr, 95 op/s 2024-04-07T16:59:46.147 INFO:tasks.workunit.client.1.smithi144.stderr: Removing image: 100% complete...done. 2024-04-07T16:59:46.152 INFO:tasks.workunit.client.1.smithi144.stderr:+ rbd import --order 20 /tmp/rbd_import_export_103522/img testimg 2024-04-07T16:59:46.199 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:59:45 smithi100 bash[74942]: cluster 2024-04-07T16:59:44.842971+0000 mon.a (mon.0) 2237 : cluster [DBG] osdmap e1073: 8 total, 8 up, 8 in 2024-04-07T16:59:46.200 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:59:45 smithi100 bash[74942]: cluster 2024-04-07T16:59:45.716484+0000 mgr.y (mgr.25175) 1332 : cluster [DBG] pgmap v1980: 105 pgs: 105 active+clean; 693 KiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 80 KiB/s rd, 5.0 KiB/s wr, 95 op/s 2024-04-07T16:59:46.200 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:59:45 smithi100 bash[76202]: cluster 2024-04-07T16:59:44.842971+0000 mon.a (mon.0) 2237 : cluster [DBG] osdmap e1073: 8 total, 8 up, 8 in 2024-04-07T16:59:46.200 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:59:45 smithi100 bash[76202]: cluster 2024-04-07T16:59:45.716484+0000 mgr.y (mgr.25175) 1332 : cluster [DBG] pgmap v1980: 105 pgs: 105 active+clean; 693 KiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 80 KiB/s rd, 5.0 KiB/s wr, 95 op/s 2024-04-07T16:59:46.263 INFO:tasks.workunit.client.1.smithi144.stderr: Importing image: 100% complete...done. 2024-04-07T16:59:46.264 INFO:tasks.workunit.client.1.smithi144.stderr:rbd: --order is deprecated, use --object-size 2024-04-07T16:59:46.269 INFO:tasks.workunit.client.1.smithi144.stderr:+ rbd export --export-format 2 testimg /tmp/rbd_import_export_103522/img_v2 2024-04-07T16:59:46.327 INFO:tasks.workunit.client.1.smithi144.stderr: Exporting image: 100% complete... Exporting image: 100% complete...done. 2024-04-07T16:59:46.332 INFO:tasks.workunit.client.1.smithi144.stderr:+ rbd import --export-format 2 /tmp/rbd_import_export_103522/img_v2 testimg_import 2024-04-07T16:59:46.470 INFO:tasks.workunit.client.1.smithi144.stderr: Importing image: 100% complete... Importing image: 100% complete...done. 2024-04-07T16:59:46.477 INFO:tasks.workunit.client.1.smithi144.stderr:+ rbd info testimg_import 2024-04-07T16:59:46.478 INFO:tasks.workunit.client.1.smithi144.stderr:+ grep order 2024-04-07T16:59:46.478 INFO:tasks.workunit.client.1.smithi144.stderr:+ awk {print $2} 2024-04-07T16:59:46.478 INFO:tasks.workunit.client.1.smithi144.stderr:+ grep 20 2024-04-07T16:59:46.538 INFO:tasks.workunit.client.1.smithi144.stdout:20 2024-04-07T16:59:46.538 INFO:tasks.workunit.client.1.smithi144.stderr:+ rm /tmp/rbd_import_export_103522/img_v2 2024-04-07T16:59:46.538 INFO:tasks.workunit.client.1.smithi144.stderr:+ rbd remove testimg_import 2024-04-07T16:59:46.794 INFO:tasks.workunit.client.1.smithi144.stderr: Removing image: 100% complete...done. 2024-04-07T16:59:46.801 INFO:tasks.workunit.client.1.smithi144.stderr:+ rbd remove testimg 2024-04-07T16:59:47.063 INFO:tasks.workunit.client.1.smithi144.stderr: Removing image: 100% complete...done. 2024-04-07T16:59:47.072 INFO:tasks.workunit.client.1.smithi144.stderr:+ rbd import --image-feature layering /tmp/rbd_import_export_103522/img testimg 2024-04-07T16:59:47.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:59:46 smithi144 bash[66654]: cluster 2024-04-07T16:59:45.867101+0000 mon.a (mon.0) 2238 : cluster [DBG] osdmap e1074: 8 total, 8 up, 8 in 2024-04-07T16:59:47.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:59:46 smithi144 bash[66654]: audit 2024-04-07T16:59:46.060176+0000 mon.a (mon.0) 2239 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:59:47.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:59:46 smithi144 bash[66654]: audit 2024-04-07T16:59:46.074686+0000 mon.a (mon.0) 2240 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:59:47.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:59:46 smithi144 bash[66654]: audit 2024-04-07T16:59:46.077316+0000 mon.a (mon.0) 2241 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:59:47.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:59:46 smithi144 bash[66654]: audit 2024-04-07T16:59:46.079322+0000 mon.a (mon.0) 2242 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:59:47.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:59:46 smithi144 bash[66654]: audit 2024-04-07T16:59:46.090351+0000 mon.a (mon.0) 2243 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:59:47.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:59:46 smithi144 bash[66654]: audit 2024-04-07T16:59:46.793334+0000 mon.a (mon.0) 2244 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:59:47.166 INFO:tasks.workunit.client.1.smithi144.stderr: Importing image: 100% complete...done. 2024-04-07T16:59:47.171 INFO:tasks.workunit.client.1.smithi144.stderr:+ rbd info testimg 2024-04-07T16:59:47.171 INFO:tasks.workunit.client.1.smithi144.stderr:+ grep features 2024-04-07T16:59:47.200 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:59:46 smithi100 bash[74942]: cluster 2024-04-07T16:59:45.867101+0000 mon.a (mon.0) 2238 : cluster [DBG] osdmap e1074: 8 total, 8 up, 8 in 2024-04-07T16:59:47.200 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:59:46 smithi100 bash[74942]: audit 2024-04-07T16:59:46.060176+0000 mon.a (mon.0) 2239 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:59:47.200 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:59:46 smithi100 bash[74942]: audit 2024-04-07T16:59:46.074686+0000 mon.a (mon.0) 2240 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:59:47.200 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:59:46 smithi100 bash[74942]: audit 2024-04-07T16:59:46.077316+0000 mon.a (mon.0) 2241 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:59:47.200 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:59:46 smithi100 bash[74942]: audit 2024-04-07T16:59:46.079322+0000 mon.a (mon.0) 2242 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:59:47.200 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:59:46 smithi100 bash[74942]: audit 2024-04-07T16:59:46.090351+0000 mon.a (mon.0) 2243 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:59:47.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:59:46 smithi100 bash[74942]: audit 2024-04-07T16:59:46.793334+0000 mon.a (mon.0) 2244 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:59:47.201 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:59:46 smithi100 bash[76202]: cluster 2024-04-07T16:59:45.867101+0000 mon.a (mon.0) 2238 : cluster [DBG] osdmap e1074: 8 total, 8 up, 8 in 2024-04-07T16:59:47.201 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:59:46 smithi100 bash[76202]: audit 2024-04-07T16:59:46.060176+0000 mon.a (mon.0) 2239 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:59:47.201 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:59:46 smithi100 bash[76202]: audit 2024-04-07T16:59:46.074686+0000 mon.a (mon.0) 2240 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:59:47.201 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:59:46 smithi100 bash[76202]: audit 2024-04-07T16:59:46.077316+0000 mon.a (mon.0) 2241 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T16:59:47.201 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:59:46 smithi100 bash[76202]: audit 2024-04-07T16:59:46.079322+0000 mon.a (mon.0) 2242 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T16:59:47.201 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:59:46 smithi100 bash[76202]: audit 2024-04-07T16:59:46.090351+0000 mon.a (mon.0) 2243 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T16:59:47.201 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:59:46 smithi100 bash[76202]: audit 2024-04-07T16:59:46.793334+0000 mon.a (mon.0) 2244 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T16:59:47.223 INFO:tasks.workunit.client.1.smithi144.stderr:+ FEATURES_BEFORE= features: layering 2024-04-07T16:59:47.223 INFO:tasks.workunit.client.1.smithi144.stderr: op_features: 2024-04-07T16:59:47.223 INFO:tasks.workunit.client.1.smithi144.stderr:+ rbd export --export-format 2 testimg /tmp/rbd_import_export_103522/img_v2 2024-04-07T16:59:47.278 INFO:tasks.workunit.client.1.smithi144.stderr: Exporting image: 100% complete... Exporting image: 100% complete...done. 2024-04-07T16:59:47.282 INFO:tasks.workunit.client.1.smithi144.stderr:+ rbd import --export-format 2 /tmp/rbd_import_export_103522/img_v2 testimg_import 2024-04-07T16:59:47.391 INFO:tasks.workunit.client.1.smithi144.stderr: Importing image: 100% complete... Importing image: 100% complete...done. 2024-04-07T16:59:47.396 INFO:tasks.workunit.client.1.smithi144.stderr:+ rbd info testimg_import 2024-04-07T16:59:47.396 INFO:tasks.workunit.client.1.smithi144.stderr:+ grep features 2024-04-07T16:59:47.452 INFO:tasks.workunit.client.1.smithi144.stderr:+ FEATURES_AFTER= features: layering 2024-04-07T16:59:47.452 INFO:tasks.workunit.client.1.smithi144.stderr: op_features: 2024-04-07T16:59:47.452 INFO:tasks.workunit.client.1.smithi144.stderr:+ [ features: layering 2024-04-07T16:59:47.452 INFO:tasks.workunit.client.1.smithi144.stderr: op_features: != features: layering 2024-04-07T16:59:47.452 INFO:tasks.workunit.client.1.smithi144.stderr: op_features: ] 2024-04-07T16:59:47.453 INFO:tasks.workunit.client.1.smithi144.stderr:+ rm /tmp/rbd_import_export_103522/img_v2 2024-04-07T16:59:47.453 INFO:tasks.workunit.client.1.smithi144.stderr:+ rbd remove testimg_import 2024-04-07T16:59:47.633 INFO:tasks.workunit.client.1.smithi144.stderr: Removing image: 100% complete...done. 2024-04-07T16:59:47.639 INFO:tasks.workunit.client.1.smithi144.stderr:+ rbd remove testimg 2024-04-07T16:59:47.825 INFO:tasks.workunit.client.1.smithi144.stderr: Removing image: 100% complete...done. 2024-04-07T16:59:47.832 INFO:tasks.workunit.client.1.smithi144.stderr:+ rbd import --stripe-count 1000 --stripe-unit 4096 /tmp/rbd_import_export_103522/img testimg 2024-04-07T16:59:47.962 INFO:tasks.workunit.client.1.smithi144.stderr: Importing image: 100% complete...done. 2024-04-07T16:59:47.967 INFO:tasks.workunit.client.1.smithi144.stderr:+ rbd export --export-format 2 testimg /tmp/rbd_import_export_103522/img_v2 2024-04-07T16:59:48.022 INFO:tasks.workunit.client.1.smithi144.stderr: Exporting image: 100% complete... Exporting image: 100% complete...done. 2024-04-07T16:59:48.026 INFO:tasks.workunit.client.1.smithi144.stderr:+ rbd import --export-format 2 /tmp/rbd_import_export_103522/img_v2 testimg_import 2024-04-07T16:59:48.180 INFO:tasks.workunit.client.1.smithi144.stderr: Importing image: 100% complete... Importing image: 100% complete...done. 2024-04-07T16:59:48.187 INFO:tasks.workunit.client.1.smithi144.stderr:+ rbd info testimg_import 2024-04-07T16:59:48.187 INFO:tasks.workunit.client.1.smithi144.stderr:+ grep stripe unit 2024-04-07T16:59:48.187 INFO:tasks.workunit.client.1.smithi144.stderr:+ grep -Ei (4 KiB|4096) 2024-04-07T16:59:48.199 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:59:47 smithi100 bash[74942]: cluster 2024-04-07T16:59:47.717622+0000 mgr.y (mgr.25175) 1333 : cluster [DBG] pgmap v1982: 105 pgs: 105 active+clean; 693 KiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 122 KiB/s rd, 8.8 KiB/s wr, 150 op/s 2024-04-07T16:59:48.200 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:59:47 smithi100 bash[76202]: cluster 2024-04-07T16:59:47.717622+0000 mgr.y (mgr.25175) 1333 : cluster [DBG] pgmap v1982: 105 pgs: 105 active+clean; 693 KiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 122 KiB/s rd, 8.8 KiB/s wr, 150 op/s 2024-04-07T16:59:48.247 INFO:tasks.workunit.client.1.smithi144.stdout: stripe unit: 4 KiB 2024-04-07T16:59:48.248 INFO:tasks.workunit.client.1.smithi144.stderr:+ rbd info testimg_import 2024-04-07T16:59:48.248 INFO:tasks.workunit.client.1.smithi144.stderr:+ + awk {print $3} 2024-04-07T16:59:48.248 INFO:tasks.workunit.client.1.smithi144.stderr:grep stripe count 2024-04-07T16:59:48.248 INFO:tasks.workunit.client.1.smithi144.stderr:+ grep 1000 2024-04-07T16:59:48.300 INFO:tasks.workunit.client.1.smithi144.stdout:1000 2024-04-07T16:59:48.300 INFO:tasks.workunit.client.1.smithi144.stderr:+ rm /tmp/rbd_import_export_103522/img_v2 2024-04-07T16:59:48.300 INFO:tasks.workunit.client.1.smithi144.stderr:+ rbd remove testimg_import 2024-04-07T16:59:48.364 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:59:47 smithi144 bash[66654]: cluster 2024-04-07T16:59:47.717622+0000 mgr.y (mgr.25175) 1333 : cluster [DBG] pgmap v1982: 105 pgs: 105 active+clean; 693 KiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 122 KiB/s rd, 8.8 KiB/s wr, 150 op/s 2024-04-07T16:59:48.550 INFO:tasks.workunit.client.1.smithi144.stderr: Removing image: 3% complete... Removing image: 7% complete... Removing image: 10% complete... Removing image: 14% complete... Removing image: 17% complete... Removing image: 21% complete... Removing image: 25% complete... Removing image: 28% complete... Removing image: 32% complete... Removing image: 35% complete... Removing image: 39% complete... Removing image: 42% complete... Removing image: 46% complete... Removing image: 50% complete... Removing image: 53% complete... Removing image: 57% complete... Removing image: 60% complete... Removing image: 64% complete... Removing image: 67% complete... Removing image: 71% complete... Removing image: 75% complete... Removing image: 78% complete... Removing image: 82% complete... Removing image: 85% complete... Removing image: 89% complete... Removing image: 92% complete... Removing image: 96% complete... Removing image: 100% complete...done. 2024-04-07T16:59:48.557 INFO:tasks.workunit.client.1.smithi144.stderr:+ rbd remove testimg 2024-04-07T16:59:48.700 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:59:48 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:59:48] "GET /metrics HTTP/1.1" 200 34049 "" "Prometheus/2.43.0" 2024-04-07T16:59:48.809 INFO:tasks.workunit.client.1.smithi144.stderr: Removing image: 3% complete... Removing image: 7% complete... Removing image: 10% complete... Removing image: 14% complete... Removing image: 17% complete... Removing image: 21% complete... Removing image: 25% complete... Removing image: 28% complete... Removing image: 32% complete... Removing image: 35% complete... Removing image: 39% complete... Removing image: 42% complete... Removing image: 46% complete... Removing image: 50% complete... Removing image: 53% complete... Removing image: 57% complete... Removing image: 60% complete... Removing image: 64% complete... Removing image: 67% complete... Removing image: 71% complete... Removing image: 75% complete... Removing image: 78% complete... Removing image: 82% complete... Removing image: 85% complete... Removing image: 89% complete... Removing image: 92% complete... Removing image: 96% complete... Removing image: 100% complete...done. 2024-04-07T16:59:48.816 INFO:tasks.workunit.client.1.smithi144.stderr:+ rbd import --image-format=2 /tmp/rbd_import_export_103522/img testimg 2024-04-07T16:59:48.933 INFO:tasks.workunit.client.1.smithi144.stderr: Importing image: 100% complete...done. 2024-04-07T16:59:48.937 INFO:tasks.workunit.client.1.smithi144.stderr:+ rbd snap create testimg@snap1 2024-04-07T16:59:49.105 INFO:tasks.workunit.client.1.smithi144.stderr: Creating snap: 10% complete... Creating snap: 100% complete...done. 2024-04-07T16:59:49.126 INFO:tasks.workunit.client.1.smithi144.stderr:+ rbd snap create testimg@snap2 2024-04-07T16:59:50.108 INFO:tasks.workunit.client.1.smithi144.stderr: Creating snap: 10% complete... Creating snap: 100% complete...done. 2024-04-07T16:59:50.128 INFO:tasks.workunit.client.1.smithi144.stderr:+ rbd snap protect testimg@snap2 2024-04-07T16:59:50.195 INFO:tasks.workunit.client.1.smithi144.stderr:+ rbd export --export-format 2 testimg /tmp/rbd_import_export_103522/snap_protect 2024-04-07T16:59:50.252 INFO:tasks.workunit.client.1.smithi144.stderr: Exporting image: 33% complete... Exporting image: 100% complete... Exporting image: 100% complete...done. 2024-04-07T16:59:50.256 INFO:tasks.workunit.client.1.smithi144.stderr:+ rbd import --export-format 2 /tmp/rbd_import_export_103522/snap_protect testimg_import 2024-04-07T16:59:50.364 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:59:50 smithi144 bash[66654]: cluster 2024-04-07T16:59:49.075960+0000 mon.a (mon.0) 2245 : cluster [DBG] osdmap e1075: 8 total, 8 up, 8 in 2024-04-07T16:59:50.364 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:59:50 smithi144 bash[66654]: cluster 2024-04-07T16:59:49.718517+0000 mgr.y (mgr.25175) 1334 : cluster [DBG] pgmap v1984: 105 pgs: 105 active+clean; 693 KiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 141 KiB/s rd, 16 KiB/s wr, 181 op/s 2024-04-07T16:59:50.450 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:59:50 smithi100 bash[74942]: cluster 2024-04-07T16:59:49.075960+0000 mon.a (mon.0) 2245 : cluster [DBG] osdmap e1075: 8 total, 8 up, 8 in 2024-04-07T16:59:50.450 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:59:50 smithi100 bash[74942]: cluster 2024-04-07T16:59:49.718517+0000 mgr.y (mgr.25175) 1334 : cluster [DBG] pgmap v1984: 105 pgs: 105 active+clean; 693 KiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 141 KiB/s rd, 16 KiB/s wr, 181 op/s 2024-04-07T16:59:50.451 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:59:50 smithi100 bash[76202]: cluster 2024-04-07T16:59:49.075960+0000 mon.a (mon.0) 2245 : cluster [DBG] osdmap e1075: 8 total, 8 up, 8 in 2024-04-07T16:59:50.451 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:59:50 smithi100 bash[76202]: cluster 2024-04-07T16:59:49.718517+0000 mgr.y (mgr.25175) 1334 : cluster [DBG] pgmap v1984: 105 pgs: 105 active+clean; 693 KiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 141 KiB/s rd, 16 KiB/s wr, 181 op/s 2024-04-07T16:59:51.364 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:59:51 smithi144 bash[66654]: cluster 2024-04-07T16:59:50.082614+0000 mon.a (mon.0) 2246 : cluster [DBG] osdmap e1076: 8 total, 8 up, 8 in 2024-04-07T16:59:51.450 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:59:51 smithi100 bash[74942]: cluster 2024-04-07T16:59:50.082614+0000 mon.a (mon.0) 2246 : cluster [DBG] osdmap e1076: 8 total, 8 up, 8 in 2024-04-07T16:59:51.450 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:59:51 smithi100 bash[76202]: cluster 2024-04-07T16:59:50.082614+0000 mon.a (mon.0) 2246 : cluster [DBG] osdmap e1076: 8 total, 8 up, 8 in 2024-04-07T16:59:52.178 INFO:tasks.workunit.client.1.smithi144.stderr: Importing image: 33% complete... Importing image: 66% complete... Importing image: 100% complete... Importing image: 100% complete...done. 2024-04-07T16:59:52.186 INFO:tasks.workunit.client.1.smithi144.stderr:+ rbd info testimg_import@snap1 2024-04-07T16:59:52.186 INFO:tasks.workunit.client.1.smithi144.stderr:+ grep protected: False 2024-04-07T16:59:52.243 INFO:tasks.workunit.client.1.smithi144.stdout: protected: False 2024-04-07T16:59:52.244 INFO:tasks.workunit.client.1.smithi144.stderr:+ rbd info testimg_import@snap2 2024-04-07T16:59:52.244 INFO:tasks.workunit.client.1.smithi144.stderr:+ grep protected: True 2024-04-07T16:59:52.292 INFO:tasks.workunit.client.1.smithi144.stdout: protected: True 2024-04-07T16:59:52.293 INFO:tasks.workunit.client.1.smithi144.stderr:+ rm /tmp/rbd_import_export_103522/snap_protect 2024-04-07T16:59:52.293 INFO:tasks.workunit.client.1.smithi144.stderr:+ rbd snap unprotect testimg@snap2 2024-04-07T16:59:52.363 INFO:tasks.workunit.client.1.smithi144.stderr:+ rbd snap unprotect testimg_import@snap2 2024-04-07T16:59:52.364 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:59:52 smithi144 bash[66654]: cluster 2024-04-07T16:59:51.092754+0000 mon.a (mon.0) 2247 : cluster [DBG] osdmap e1077: 8 total, 8 up, 8 in 2024-04-07T16:59:52.364 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:59:52 smithi144 bash[66654]: cluster 2024-04-07T16:59:51.719203+0000 mgr.y (mgr.25175) 1335 : cluster [DBG] pgmap v1987: 105 pgs: 105 active+clean; 583 KiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 238 KiB/s rd, 25 KiB/s wr, 315 op/s 2024-04-07T16:59:52.364 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 16:59:52 smithi144 bash[95233]: ts=2024-04-07T16:59:52.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T16:59:52.421 INFO:tasks.workunit.client.1.smithi144.stderr:+ rbd snap purge testimg 2024-04-07T16:59:52.449 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:59:52 smithi100 bash[74942]: cluster 2024-04-07T16:59:51.092754+0000 mon.a (mon.0) 2247 : cluster [DBG] osdmap e1077: 8 total, 8 up, 8 in 2024-04-07T16:59:52.450 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:59:52 smithi100 bash[74942]: cluster 2024-04-07T16:59:51.719203+0000 mgr.y (mgr.25175) 1335 : cluster [DBG] pgmap v1987: 105 pgs: 105 active+clean; 583 KiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 238 KiB/s rd, 25 KiB/s wr, 315 op/s 2024-04-07T16:59:52.450 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:59:52 smithi100 bash[76202]: cluster 2024-04-07T16:59:51.092754+0000 mon.a (mon.0) 2247 : cluster [DBG] osdmap e1077: 8 total, 8 up, 8 in 2024-04-07T16:59:52.450 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:59:52 smithi100 bash[76202]: cluster 2024-04-07T16:59:51.719203+0000 mgr.y (mgr.25175) 1335 : cluster [DBG] pgmap v1987: 105 pgs: 105 active+clean; 583 KiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 238 KiB/s rd, 25 KiB/s wr, 315 op/s 2024-04-07T16:59:53.364 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:59:53 smithi144 bash[66654]: cluster 2024-04-07T16:59:52.105736+0000 mon.a (mon.0) 2248 : cluster [DBG] osdmap e1078: 8 total, 8 up, 8 in 2024-04-07T16:59:53.364 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:59:53 smithi144 bash[66654]: cluster 2024-04-07T16:59:52.496310+0000 mon.a (mon.0) 2249 : cluster [DBG] osdmap e1079: 8 total, 8 up, 8 in 2024-04-07T16:59:53.450 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:59:53 smithi100 bash[74942]: cluster 2024-04-07T16:59:52.105736+0000 mon.a (mon.0) 2248 : cluster [DBG] osdmap e1078: 8 total, 8 up, 8 in 2024-04-07T16:59:53.450 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:59:53 smithi100 bash[74942]: cluster 2024-04-07T16:59:52.496310+0000 mon.a (mon.0) 2249 : cluster [DBG] osdmap e1079: 8 total, 8 up, 8 in 2024-04-07T16:59:53.450 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:59:53 smithi100 bash[76202]: cluster 2024-04-07T16:59:52.105736+0000 mon.a (mon.0) 2248 : cluster [DBG] osdmap e1078: 8 total, 8 up, 8 in 2024-04-07T16:59:53.450 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:59:53 smithi100 bash[76202]: cluster 2024-04-07T16:59:52.496310+0000 mon.a (mon.0) 2249 : cluster [DBG] osdmap e1079: 8 total, 8 up, 8 in 2024-04-07T16:59:54.523 INFO:tasks.workunit.client.1.smithi144.stderr: Removing all snapshots: 50% complete... Removing all snapshots: 100% complete... Removing all snapshots: 100% complete...done. 2024-04-07T16:59:54.553 INFO:tasks.workunit.client.1.smithi144.stderr:+ rbd snap purge testimg_import 2024-04-07T16:59:54.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:59:54 smithi144 bash[66654]: cluster 2024-04-07T16:59:53.498031+0000 mon.a (mon.0) 2250 : cluster [DBG] osdmap e1080: 8 total, 8 up, 8 in 2024-04-07T16:59:54.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:59:54 smithi144 bash[66654]: cluster 2024-04-07T16:59:53.719912+0000 mgr.y (mgr.25175) 1336 : cluster [DBG] pgmap v1991: 105 pgs: 105 active+clean; 693 KiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 96 KiB/s rd, 19 KiB/s wr, 142 op/s 2024-04-07T16:59:54.950 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:59:54 smithi100 bash[74942]: cluster 2024-04-07T16:59:53.498031+0000 mon.a (mon.0) 2250 : cluster [DBG] osdmap e1080: 8 total, 8 up, 8 in 2024-04-07T16:59:54.950 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:59:54 smithi100 bash[74942]: cluster 2024-04-07T16:59:53.719912+0000 mgr.y (mgr.25175) 1336 : cluster [DBG] pgmap v1991: 105 pgs: 105 active+clean; 693 KiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 96 KiB/s rd, 19 KiB/s wr, 142 op/s 2024-04-07T16:59:54.950 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:59:54 smithi100 bash[76202]: cluster 2024-04-07T16:59:53.498031+0000 mon.a (mon.0) 2250 : cluster [DBG] osdmap e1080: 8 total, 8 up, 8 in 2024-04-07T16:59:54.950 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:59:54 smithi100 bash[76202]: cluster 2024-04-07T16:59:53.719912+0000 mgr.y (mgr.25175) 1336 : cluster [DBG] pgmap v1991: 105 pgs: 105 active+clean; 693 KiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 96 KiB/s rd, 19 KiB/s wr, 142 op/s 2024-04-07T16:59:55.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:59:55 smithi144 bash[66654]: cluster 2024-04-07T16:59:54.512423+0000 mon.a (mon.0) 2251 : cluster [DBG] osdmap e1081: 8 total, 8 up, 8 in 2024-04-07T16:59:55.950 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:59:55 smithi100 bash[74942]: cluster 2024-04-07T16:59:54.512423+0000 mon.a (mon.0) 2251 : cluster [DBG] osdmap e1081: 8 total, 8 up, 8 in 2024-04-07T16:59:55.950 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:59:55 smithi100 bash[76202]: cluster 2024-04-07T16:59:54.512423+0000 mon.a (mon.0) 2251 : cluster [DBG] osdmap e1081: 8 total, 8 up, 8 in 2024-04-07T16:59:56.544 INFO:tasks.workunit.client.1.smithi144.stderr: Removing all snapshots: 50% complete... Removing all snapshots: 100% complete... Removing all snapshots: 100% complete...done. 2024-04-07T16:59:56.586 INFO:tasks.workunit.client.1.smithi144.stderr:+ rbd remove testimg 2024-04-07T16:59:56.851 INFO:tasks.workunit.client.1.smithi144.stderr: Removing image: 100% complete...done. 2024-04-07T16:59:56.857 INFO:tasks.workunit.client.1.smithi144.stderr:+ rbd remove testimg_import 2024-04-07T16:59:56.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:59:56 smithi144 bash[66654]: cluster 2024-04-07T16:59:55.518156+0000 mon.a (mon.0) 2252 : cluster [DBG] osdmap e1082: 8 total, 8 up, 8 in 2024-04-07T16:59:56.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:59:56 smithi144 bash[66654]: cluster 2024-04-07T16:59:55.720622+0000 mgr.y (mgr.25175) 1337 : cluster [DBG] pgmap v1994: 105 pgs: 105 active+clean; 803 KiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 19 KiB/s rd, 8.6 KiB/s wr, 35 op/s 2024-04-07T16:59:56.950 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:59:56 smithi100 bash[76202]: cluster 2024-04-07T16:59:55.518156+0000 mon.a (mon.0) 2252 : cluster [DBG] osdmap e1082: 8 total, 8 up, 8 in 2024-04-07T16:59:56.950 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:59:56 smithi100 bash[76202]: cluster 2024-04-07T16:59:55.720622+0000 mgr.y (mgr.25175) 1337 : cluster [DBG] pgmap v1994: 105 pgs: 105 active+clean; 803 KiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 19 KiB/s rd, 8.6 KiB/s wr, 35 op/s 2024-04-07T16:59:56.950 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:59:56 smithi100 bash[74942]: cluster 2024-04-07T16:59:55.518156+0000 mon.a (mon.0) 2252 : cluster [DBG] osdmap e1082: 8 total, 8 up, 8 in 2024-04-07T16:59:56.950 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:59:56 smithi100 bash[74942]: cluster 2024-04-07T16:59:55.720622+0000 mgr.y (mgr.25175) 1337 : cluster [DBG] pgmap v1994: 105 pgs: 105 active+clean; 803 KiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 19 KiB/s rd, 8.6 KiB/s wr, 35 op/s 2024-04-07T16:59:57.044 INFO:tasks.workunit.client.1.smithi144.stderr: Removing image: 100% complete...done. 2024-04-07T16:59:57.050 INFO:tasks.workunit.client.1.smithi144.stderr:+ tiered=0 2024-04-07T16:59:57.051 INFO:tasks.workunit.client.1.smithi144.stderr:+ ceph osd dump 2024-04-07T16:59:57.051 INFO:tasks.workunit.client.1.smithi144.stderr:+ grep ^pool 2024-04-07T16:59:57.051 INFO:tasks.workunit.client.1.smithi144.stderr:+ grep 'rbd' 2024-04-07T16:59:57.051 INFO:tasks.workunit.client.1.smithi144.stderr:+ grep tier 2024-04-07T16:59:57.480 INFO:tasks.workunit.client.1.smithi144.stderr:+ dd if=/dev/urandom bs=1M seek=1 count=1 of=/tmp/rbd_import_export_103522/sparse1 2024-04-07T16:59:57.485 INFO:tasks.workunit.client.1.smithi144.stderr:1+0 records in 2024-04-07T16:59:57.485 INFO:tasks.workunit.client.1.smithi144.stderr:1+0 records out 2024-04-07T16:59:57.485 INFO:tasks.workunit.client.1.smithi144.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00511982 s, 205 MB/s 2024-04-07T16:59:57.485 INFO:tasks.workunit.client.1.smithi144.stderr:+ dd if=/dev/urandom bs=1M count=1 of=/tmp/rbd_import_export_103522/sparse2 2024-04-07T16:59:57.493 INFO:tasks.workunit.client.1.smithi144.stderr:1+0 records in 2024-04-07T16:59:57.493 INFO:tasks.workunit.client.1.smithi144.stderr:1+0 records out 2024-04-07T16:59:57.493 INFO:tasks.workunit.client.1.smithi144.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00664648 s, 158 MB/s 2024-04-07T16:59:57.493 INFO:tasks.workunit.client.1.smithi144.stderr:+ truncate /tmp/rbd_import_export_103522/sparse2 -s 2M 2024-04-07T16:59:57.507 INFO:tasks.workunit.client.1.smithi144.stderr:+ rbd rm sparse1 2024-04-07T16:59:57.595 INFO:tasks.workunit.client.1.smithi144.stderr: Removing image: 0% complete...failed. 2024-04-07T16:59:57.595 INFO:tasks.workunit.client.1.smithi144.stderr:rbd: delete error: (2) No such file or directory 2024-04-07T16:59:57.600 INFO:tasks.workunit.client.1.smithi144.stderr:+ true 2024-04-07T16:59:57.600 INFO:tasks.workunit.client.1.smithi144.stderr:+ rbd import --new-format --order 20 /tmp/rbd_import_export_103522/sparse1 2024-04-07T16:59:57.692 INFO:tasks.workunit.client.1.smithi144.stderr: Importing image: 50% complete... Importing image: 100% complete...done. 2024-04-07T16:59:57.693 INFO:tasks.workunit.client.1.smithi144.stderr:rbd: --new-format is deprecated, use --image-format 2 2024-04-07T16:59:57.693 INFO:tasks.workunit.client.1.smithi144.stderr:rbd: --order is deprecated, use --object-size 2024-04-07T16:59:57.697 INFO:tasks.workunit.client.1.smithi144.stderr:+ rbd ls -l 2024-04-07T16:59:57.697 INFO:tasks.workunit.client.1.smithi144.stderr:+ grep sparse1 2024-04-07T16:59:57.697 INFO:tasks.workunit.client.1.smithi144.stderr:+ grep -Ei (2 MiB|2048k) 2024-04-07T16:59:57.742 INFO:tasks.workunit.client.1.smithi144.stdout:sparse1 2 MiB 2 2024-04-07T16:59:57.743 INFO:tasks.workunit.client.1.smithi144.stderr:+ objects sparse1 2024-04-07T16:59:57.743 INFO:tasks.workunit.client.1.smithi144.stderr:+ image=sparse1 2024-04-07T16:59:57.743 INFO:tasks.workunit.client.1.smithi144.stderr:+ rbd info sparse1 2024-04-07T16:59:57.743 INFO:tasks.workunit.client.1.smithi144.stderr:+ + awk {print $NF;} 2024-04-07T16:59:57.744 INFO:tasks.workunit.client.1.smithi144.stderr:grep block_name_prefix 2024-04-07T16:59:57.789 INFO:tasks.workunit.client.1.smithi144.stderr:+ prefix=rbd_data.d639a0ccd914 2024-04-07T16:59:57.789 INFO:tasks.workunit.client.1.smithi144.stderr:+ grep rbd_data.d639a0ccd914 2024-04-07T16:59:57.789 INFO:tasks.workunit.client.1.smithi144.stderr:+ get_image_data_pool sparse1 2024-04-07T16:59:57.789 INFO:tasks.workunit.client.1.smithi144.stderr:+ image=sparse1 2024-04-07T16:59:57.789 INFO:tasks.workunit.client.1.smithi144.stderr:+ sed -e s/rbd_data.d639a0ccd914\.// -e s/^0*\([0-9a-f]\)/\1/ 2024-04-07T16:59:57.789 INFO:tasks.workunit.client.1.smithi144.stderr:+ sort -u 2024-04-07T16:59:57.789 INFO:tasks.workunit.client.1.smithi144.stderr:+ rbd info sparse1 2024-04-07T16:59:57.789 INFO:tasks.workunit.client.1.smithi144.stderr:+ grep data_pool: 2024-04-07T16:59:57.789 INFO:tasks.workunit.client.1.smithi144.stderr:+ awk -F: { print $NF } 2024-04-07T16:59:57.833 INFO:tasks.workunit.client.1.smithi144.stderr:+ data_pool= 2024-04-07T16:59:57.833 INFO:tasks.workunit.client.1.smithi144.stderr:+ [ -z ] 2024-04-07T16:59:57.833 INFO:tasks.workunit.client.1.smithi144.stderr:+ data_pool=rbd 2024-04-07T16:59:57.834 INFO:tasks.workunit.client.1.smithi144.stderr:+ echo rbd 2024-04-07T16:59:57.834 INFO:tasks.workunit.client.1.smithi144.stderr:+ rados ls -p rbd 2024-04-07T16:59:57.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:59:57 smithi144 bash[66654]: cluster 2024-04-07T16:59:56.529233+0000 mon.a (mon.0) 2253 : cluster [DBG] osdmap e1083: 8 total, 8 up, 8 in 2024-04-07T16:59:57.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:59:57 smithi144 bash[66654]: audit 2024-04-07T16:59:57.357984+0000 mon.b (mon.1) 11 : audit [DBG] from='client.? 172.21.15.144:0/3929816160' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-07T16:59:57.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:59:57 smithi144 bash[66654]: audit 2024-04-07T16:59:57.463437+0000 mon.b (mon.1) 12 : audit [DBG] from='client.? 172.21.15.144:0/3929816160' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-07T16:59:57.950 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:59:57 smithi100 bash[74942]: cluster 2024-04-07T16:59:56.529233+0000 mon.a (mon.0) 2253 : cluster [DBG] osdmap e1083: 8 total, 8 up, 8 in 2024-04-07T16:59:57.950 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:59:57 smithi100 bash[74942]: audit 2024-04-07T16:59:57.357984+0000 mon.b (mon.1) 11 : audit [DBG] from='client.? 172.21.15.144:0/3929816160' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-07T16:59:57.950 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:59:57 smithi100 bash[74942]: audit 2024-04-07T16:59:57.463437+0000 mon.b (mon.1) 12 : audit [DBG] from='client.? 172.21.15.144:0/3929816160' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-07T16:59:57.950 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:59:57 smithi100 bash[76202]: cluster 2024-04-07T16:59:56.529233+0000 mon.a (mon.0) 2253 : cluster [DBG] osdmap e1083: 8 total, 8 up, 8 in 2024-04-07T16:59:57.950 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:59:57 smithi100 bash[76202]: audit 2024-04-07T16:59:57.357984+0000 mon.b (mon.1) 11 : audit [DBG] from='client.? 172.21.15.144:0/3929816160' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-07T16:59:57.950 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:59:57 smithi100 bash[76202]: audit 2024-04-07T16:59:57.463437+0000 mon.b (mon.1) 12 : audit [DBG] from='client.? 172.21.15.144:0/3929816160' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-07T16:59:57.982 INFO:tasks.workunit.client.1.smithi144.stderr:+ objects=1 2024-04-07T16:59:57.982 INFO:tasks.workunit.client.1.smithi144.stderr:+ echo 1 2024-04-07T16:59:57.982 INFO:tasks.workunit.client.1.smithi144.stderr:+ [ 0 -eq 1 -o 1 = 1 ] 2024-04-07T16:59:57.983 INFO:tasks.workunit.client.1.smithi144.stderr:+ rbd export sparse1 /tmp/rbd_import_export_103522/sparse1.out 2024-04-07T16:59:58.033 INFO:tasks.workunit.client.1.smithi144.stderr: Exporting image: 50% complete... Exporting image: 100% complete...done. 2024-04-07T16:59:58.038 INFO:tasks.workunit.client.1.smithi144.stderr:+ compare_files_and_ondisk_sizes /tmp/rbd_import_export_103522/sparse1 /tmp/rbd_import_export_103522/sparse1.out 2024-04-07T16:59:58.038 INFO:tasks.workunit.client.1.smithi144.stderr:+ cmp -l /tmp/rbd_import_export_103522/sparse1 /tmp/rbd_import_export_103522/sparse1.out 2024-04-07T16:59:58.041 INFO:tasks.workunit.client.1.smithi144.stderr:+ stat /tmp/rbd_import_export_103522/sparse1 --format %b 2024-04-07T16:59:58.041 INFO:tasks.workunit.client.1.smithi144.stderr:+ origsize=2048 2024-04-07T16:59:58.041 INFO:tasks.workunit.client.1.smithi144.stderr:+ stat /tmp/rbd_import_export_103522/sparse1.out --format %b 2024-04-07T16:59:58.042 INFO:tasks.workunit.client.1.smithi144.stderr:+ exportsize=2048 2024-04-07T16:59:58.042 INFO:tasks.workunit.client.1.smithi144.stderr:+ difference=0 2024-04-07T16:59:58.042 INFO:tasks.workunit.client.1.smithi144.stderr:+ difference=0 2024-04-07T16:59:58.042 INFO:tasks.workunit.client.1.smithi144.stderr:+ test 0 -ge 0 -a 0 -lt 4096 2024-04-07T16:59:58.043 INFO:tasks.workunit.client.1.smithi144.stderr:+ rm /tmp/rbd_import_export_103522/sparse1.out 2024-04-07T16:59:58.043 INFO:tasks.workunit.client.1.smithi144.stderr:+ rbd rm sparse1 2024-04-07T16:59:58.203 INFO:tasks.workunit.client.1.smithi144.stderr: Removing image: 50% complete... Removing image: 100% complete...done. 2024-04-07T16:59:58.208 INFO:tasks.workunit.client.1.smithi144.stderr:+ rbd rm sparse2 2024-04-07T16:59:58.287 INFO:tasks.workunit.client.1.smithi144.stderr: Removing image: 0% complete...failed. 2024-04-07T16:59:58.287 INFO:tasks.workunit.client.1.smithi144.stderr:rbd: delete error: (2) No such file or directory 2024-04-07T16:59:58.291 INFO:tasks.workunit.client.1.smithi144.stderr:+ true 2024-04-07T16:59:58.292 INFO:tasks.workunit.client.1.smithi144.stderr:+ rbd import --new-format --order 20 /tmp/rbd_import_export_103522/sparse2 2024-04-07T16:59:58.422 INFO:tasks.workunit.client.1.smithi144.stderr: Importing image: 50% complete... Importing image: 100% complete...done. 2024-04-07T16:59:58.423 INFO:tasks.workunit.client.1.smithi144.stderr:rbd: --new-format is deprecated, use --image-format 2 2024-04-07T16:59:58.423 INFO:tasks.workunit.client.1.smithi144.stderr:rbd: --order is deprecated, use --object-size 2024-04-07T16:59:58.429 INFO:tasks.workunit.client.1.smithi144.stderr:+ rbd ls -l 2024-04-07T16:59:58.429 INFO:tasks.workunit.client.1.smithi144.stderr:+ grep sparse2 2024-04-07T16:59:58.429 INFO:tasks.workunit.client.1.smithi144.stderr:+ grep -Ei (2 MiB|2048k) 2024-04-07T16:59:58.488 INFO:tasks.workunit.client.1.smithi144.stdout:sparse2 2 MiB 2 2024-04-07T16:59:58.489 INFO:tasks.workunit.client.1.smithi144.stderr:+ objects sparse2 2024-04-07T16:59:58.489 INFO:tasks.workunit.client.1.smithi144.stderr:+ image=sparse2 2024-04-07T16:59:58.489 INFO:tasks.workunit.client.1.smithi144.stderr:+ rbd info sparse2 2024-04-07T16:59:58.489 INFO:tasks.workunit.client.1.smithi144.stderr:+ grep block_name_prefix 2024-04-07T16:59:58.489 INFO:tasks.workunit.client.1.smithi144.stderr:+ awk {print $NF;} 2024-04-07T16:59:58.542 INFO:tasks.workunit.client.1.smithi144.stderr:+ prefix=rbd_data.d66c7ad48450 2024-04-07T16:59:58.542 INFO:tasks.workunit.client.1.smithi144.stderr:+ grep rbd_data.d66c7ad48450 2024-04-07T16:59:58.542 INFO:tasks.workunit.client.1.smithi144.stderr:+ get_image_data_pool sparse2 2024-04-07T16:59:58.542 INFO:tasks.workunit.client.1.smithi144.stderr:+ image=sparse2 2024-04-07T16:59:58.543 INFO:tasks.workunit.client.1.smithi144.stderr:+ sed -e s/rbd_data.d66c7ad48450\.// -e s/^0*\([0-9a-f]\)/\1/ 2024-04-07T16:59:58.543 INFO:tasks.workunit.client.1.smithi144.stderr:+ sort -u 2024-04-07T16:59:58.543 INFO:tasks.workunit.client.1.smithi144.stderr:+ rbd info sparse2 2024-04-07T16:59:58.543 INFO:tasks.workunit.client.1.smithi144.stderr:+ grep data_pool: 2024-04-07T16:59:58.543 INFO:tasks.workunit.client.1.smithi144.stderr:+ awk -F: { print $NF } 2024-04-07T16:59:58.592 INFO:tasks.workunit.client.1.smithi144.stderr:+ data_pool= 2024-04-07T16:59:58.593 INFO:tasks.workunit.client.1.smithi144.stderr:+ [ -z ] 2024-04-07T16:59:58.593 INFO:tasks.workunit.client.1.smithi144.stderr:+ data_pool=rbd 2024-04-07T16:59:58.593 INFO:tasks.workunit.client.1.smithi144.stderr:+ echo rbd 2024-04-07T16:59:58.593 INFO:tasks.workunit.client.1.smithi144.stderr:+ rados ls -p rbd 2024-04-07T16:59:58.700 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:59:58 smithi100 bash[74942]: cluster 2024-04-07T16:59:57.721680+0000 mgr.y (mgr.25175) 1338 : cluster [DBG] pgmap v1996: 105 pgs: 105 active+clean; 693 KiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 119 KiB/s rd, 9.2 KiB/s wr, 154 op/s 2024-04-07T16:59:58.700 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 16:59:58 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:16:59:58] "GET /metrics HTTP/1.1" 200 34049 "" "Prometheus/2.43.0" 2024-04-07T16:59:58.700 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:59:58 smithi100 bash[76202]: cluster 2024-04-07T16:59:57.721680+0000 mgr.y (mgr.25175) 1338 : cluster [DBG] pgmap v1996: 105 pgs: 105 active+clean; 693 KiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 119 KiB/s rd, 9.2 KiB/s wr, 154 op/s 2024-04-07T16:59:58.723 INFO:tasks.workunit.client.1.smithi144.stderr:+ objects=0 2024-04-07T16:59:58.723 INFO:tasks.workunit.client.1.smithi144.stderr:+ echo 0 2024-04-07T16:59:58.723 INFO:tasks.workunit.client.1.smithi144.stderr:+ [ 0 -eq 1 -o 0 = 0 ] 2024-04-07T16:59:58.723 INFO:tasks.workunit.client.1.smithi144.stderr:+ rbd export sparse2 /tmp/rbd_import_export_103522/sparse2.out 2024-04-07T16:59:58.783 INFO:tasks.workunit.client.1.smithi144.stderr: Exporting image: 50% complete... Exporting image: 100% complete...done. 2024-04-07T16:59:58.787 INFO:tasks.workunit.client.1.smithi144.stderr:+ compare_files_and_ondisk_sizes /tmp/rbd_import_export_103522/sparse2 /tmp/rbd_import_export_103522/sparse2.out 2024-04-07T16:59:58.787 INFO:tasks.workunit.client.1.smithi144.stderr:+ cmp -l /tmp/rbd_import_export_103522/sparse2 /tmp/rbd_import_export_103522/sparse2.out 2024-04-07T16:59:58.790 INFO:tasks.workunit.client.1.smithi144.stderr:+ stat /tmp/rbd_import_export_103522/sparse2 --format %b 2024-04-07T16:59:58.791 INFO:tasks.workunit.client.1.smithi144.stderr:+ origsize=2048 2024-04-07T16:59:58.791 INFO:tasks.workunit.client.1.smithi144.stderr:+ stat /tmp/rbd_import_export_103522/sparse2.out --format %b 2024-04-07T16:59:58.791 INFO:tasks.workunit.client.1.smithi144.stderr:+ exportsize=2048 2024-04-07T16:59:58.791 INFO:tasks.workunit.client.1.smithi144.stderr:+ difference=0 2024-04-07T16:59:58.791 INFO:tasks.workunit.client.1.smithi144.stderr:+ difference=0 2024-04-07T16:59:58.791 INFO:tasks.workunit.client.1.smithi144.stderr:+ test 0 -ge 0 -a 0 -lt 4096 2024-04-07T16:59:58.791 INFO:tasks.workunit.client.1.smithi144.stderr:+ rm /tmp/rbd_import_export_103522/sparse2.out 2024-04-07T16:59:58.792 INFO:tasks.workunit.client.1.smithi144.stderr:+ rbd rm sparse2 2024-04-07T16:59:58.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:59:58 smithi144 bash[66654]: cluster 2024-04-07T16:59:57.721680+0000 mgr.y (mgr.25175) 1338 : cluster [DBG] pgmap v1996: 105 pgs: 105 active+clean; 693 KiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 119 KiB/s rd, 9.2 KiB/s wr, 154 op/s 2024-04-07T16:59:59.068 INFO:tasks.workunit.client.1.smithi144.stderr: Removing image: 50% complete... Removing image: 100% complete...done. 2024-04-07T16:59:59.074 INFO:tasks.workunit.client.1.smithi144.stderr:+ truncate /tmp/rbd_import_export_103522/sparse1 -s 10M 2024-04-07T16:59:59.075 INFO:tasks.workunit.client.1.smithi144.stderr:+ rbd import --new-format --order 20 - sparse1 2024-04-07T16:59:59.291 INFO:tasks.workunit.client.1.smithi144.stderr: Importing image: 100% complete...done. 2024-04-07T16:59:59.293 INFO:tasks.workunit.client.1.smithi144.stderr:rbd: --new-format is deprecated, use --image-format 2 2024-04-07T16:59:59.293 INFO:tasks.workunit.client.1.smithi144.stderr:rbd: --order is deprecated, use --object-size 2024-04-07T16:59:59.299 INFO:tasks.workunit.client.1.smithi144.stderr:+ rbd ls -l 2024-04-07T16:59:59.299 INFO:tasks.workunit.client.1.smithi144.stderr:+ grep sparse1 2024-04-07T16:59:59.299 INFO:tasks.workunit.client.1.smithi144.stderr:+ grep -Ei (10 MiB|10240k) 2024-04-07T16:59:59.355 INFO:tasks.workunit.client.1.smithi144.stdout:sparse1 10 MiB 2 2024-04-07T16:59:59.356 INFO:tasks.workunit.client.1.smithi144.stderr:+ objects sparse1 2024-04-07T16:59:59.356 INFO:tasks.workunit.client.1.smithi144.stderr:+ image=sparse1 2024-04-07T16:59:59.357 INFO:tasks.workunit.client.1.smithi144.stderr:+ rbd info sparse1 2024-04-07T16:59:59.357 INFO:tasks.workunit.client.1.smithi144.stderr:+ grep block_name_prefix 2024-04-07T16:59:59.357 INFO:tasks.workunit.client.1.smithi144.stderr:+ awk {print $NF;} 2024-04-07T16:59:59.411 INFO:tasks.workunit.client.1.smithi144.stderr:+ prefix=rbd_data.d699ea73c1e8 2024-04-07T16:59:59.412 INFO:tasks.workunit.client.1.smithi144.stderr:+ grep rbd_data.d699ea73c1e8 2024-04-07T16:59:59.412 INFO:tasks.workunit.client.1.smithi144.stderr:+ get_image_data_pool sparse1 2024-04-07T16:59:59.412 INFO:tasks.workunit.client.1.smithi144.stderr:+ image=sparse1 2024-04-07T16:59:59.412 INFO:tasks.workunit.client.1.smithi144.stderr:+ sed -e s/rbd_data.d699ea73c1e8\.// -e s/^0*\([0-9a-f]\)/\1/ 2024-04-07T16:59:59.412 INFO:tasks.workunit.client.1.smithi144.stderr:+ sort -u 2024-04-07T16:59:59.412 INFO:tasks.workunit.client.1.smithi144.stderr:+ rbd info sparse1 2024-04-07T16:59:59.412 INFO:tasks.workunit.client.1.smithi144.stderr:+ grep data_pool: 2024-04-07T16:59:59.412 INFO:tasks.workunit.client.1.smithi144.stderr:+ awk -F: { print $NF } 2024-04-07T16:59:59.462 INFO:tasks.workunit.client.1.smithi144.stderr:+ data_pool= 2024-04-07T16:59:59.462 INFO:tasks.workunit.client.1.smithi144.stderr:+ [ -z ] 2024-04-07T16:59:59.462 INFO:tasks.workunit.client.1.smithi144.stderr:+ data_pool=rbd 2024-04-07T16:59:59.462 INFO:tasks.workunit.client.1.smithi144.stderr:+ echo rbd 2024-04-07T16:59:59.462 INFO:tasks.workunit.client.1.smithi144.stderr:+ rados ls -p rbd 2024-04-07T16:59:59.581 INFO:tasks.workunit.client.1.smithi144.stderr:+ objects=1 2024-04-07T16:59:59.581 INFO:tasks.workunit.client.1.smithi144.stderr:+ echo 1 2024-04-07T16:59:59.581 INFO:tasks.workunit.client.1.smithi144.stderr:+ [ 0 -eq 1 -o 1 = 1 ] 2024-04-07T16:59:59.581 INFO:tasks.workunit.client.1.smithi144.stderr:+ rbd export sparse1 /tmp/rbd_import_export_103522/sparse1.out 2024-04-07T16:59:59.640 INFO:tasks.workunit.client.1.smithi144.stderr: Exporting image: 10% complete... Exporting image: 20% complete... Exporting image: 30% complete... Exporting image: 40% complete... Exporting image: 50% complete... Exporting image: 60% complete... Exporting image: 70% complete... Exporting image: 80% complete... Exporting image: 90% complete... Exporting image: 100% complete...done. 2024-04-07T16:59:59.645 INFO:tasks.workunit.client.1.smithi144.stderr:+ compare_files_and_ondisk_sizes /tmp/rbd_import_export_103522/sparse1 /tmp/rbd_import_export_103522/sparse1.out 2024-04-07T16:59:59.645 INFO:tasks.workunit.client.1.smithi144.stderr:+ cmp -l /tmp/rbd_import_export_103522/sparse1 /tmp/rbd_import_export_103522/sparse1.out 2024-04-07T16:59:59.654 INFO:tasks.workunit.client.1.smithi144.stderr:+ stat /tmp/rbd_import_export_103522/sparse1 --format %b 2024-04-07T16:59:59.655 INFO:tasks.workunit.client.1.smithi144.stderr:+ origsize=2048 2024-04-07T16:59:59.655 INFO:tasks.workunit.client.1.smithi144.stderr:+ stat /tmp/rbd_import_export_103522/sparse1.out --format %b 2024-04-07T16:59:59.656 INFO:tasks.workunit.client.1.smithi144.stderr:+ exportsize=2048 2024-04-07T16:59:59.656 INFO:tasks.workunit.client.1.smithi144.stderr:+ difference=0 2024-04-07T16:59:59.656 INFO:tasks.workunit.client.1.smithi144.stderr:+ difference=0 2024-04-07T16:59:59.656 INFO:tasks.workunit.client.1.smithi144.stderr:+ test 0 -ge 0 -a 0 -lt 4096 2024-04-07T16:59:59.656 INFO:tasks.workunit.client.1.smithi144.stderr:+ rm /tmp/rbd_import_export_103522/sparse1.out 2024-04-07T16:59:59.657 INFO:tasks.workunit.client.1.smithi144.stderr:+ rbd rm sparse1 2024-04-07T16:59:59.906 INFO:tasks.workunit.client.1.smithi144.stderr: Removing image: 10% complete... Removing image: 20% complete... Removing image: 30% complete... Removing image: 40% complete... Removing image: 50% complete... Removing image: 60% complete... Removing image: 70% complete... Removing image: 80% complete... Removing image: 90% complete... Removing image: 100% complete...done. 2024-04-07T16:59:59.911 INFO:tasks.workunit.client.1.smithi144.stderr:+ dd if=/dev/urandom bs=2M count=1 of=/tmp/rbd_import_export_103522/sparse2 oflag=append conv=notrunc 2024-04-07T16:59:59.923 INFO:tasks.workunit.client.1.smithi144.stderr:1+0 records in 2024-04-07T16:59:59.923 INFO:tasks.workunit.client.1.smithi144.stderr:1+0 records out 2024-04-07T16:59:59.923 INFO:tasks.workunit.client.1.smithi144.stderr:2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.0102422 s, 205 MB/s 2024-04-07T16:59:59.923 INFO:tasks.workunit.client.1.smithi144.stderr:+ rbd import --new-format --order 20 - sparse2 2024-04-07T17:00:00.095 INFO:tasks.workunit.client.1.smithi144.stderr: Importing image: 100% complete...done. 2024-04-07T17:00:00.096 INFO:tasks.workunit.client.1.smithi144.stderr:rbd: --new-format is deprecated, use --image-format 2 2024-04-07T17:00:00.096 INFO:tasks.workunit.client.1.smithi144.stderr:rbd: --order is deprecated, use --object-size 2024-04-07T17:00:00.102 INFO:tasks.workunit.client.1.smithi144.stderr:+ + rbd ls -l 2024-04-07T17:00:00.103 INFO:tasks.workunit.client.1.smithi144.stderr:grep sparse2 2024-04-07T17:00:00.103 INFO:tasks.workunit.client.1.smithi144.stderr:+ grep -Ei (4 MiB|4096k) 2024-04-07T17:00:00.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 16:59:59 smithi144 bash[66654]: cluster 2024-04-07T16:59:59.722760+0000 mgr.y (mgr.25175) 1339 : cluster [DBG] pgmap v1997: 105 pgs: 105 active+clean; 693 KiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 90 KiB/s rd, 8.3 KiB/s wr, 120 op/s 2024-04-07T17:00:00.159 INFO:tasks.workunit.client.1.smithi144.stdout:sparse2 4 MiB 2 2024-04-07T17:00:00.160 INFO:tasks.workunit.client.1.smithi144.stderr:+ objects sparse2 2024-04-07T17:00:00.160 INFO:tasks.workunit.client.1.smithi144.stderr:+ image=sparse2 2024-04-07T17:00:00.160 INFO:tasks.workunit.client.1.smithi144.stderr:+ rbd info sparse2 2024-04-07T17:00:00.160 INFO:tasks.workunit.client.1.smithi144.stderr:+ grep block_name_prefix 2024-04-07T17:00:00.161 INFO:tasks.workunit.client.1.smithi144.stderr:+ awk {print $NF;} 2024-04-07T17:00:00.200 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 16:59:59 smithi100 bash[76202]: cluster 2024-04-07T16:59:59.722760+0000 mgr.y (mgr.25175) 1339 : cluster [DBG] pgmap v1997: 105 pgs: 105 active+clean; 693 KiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 90 KiB/s rd, 8.3 KiB/s wr, 120 op/s 2024-04-07T17:00:00.200 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 16:59:59 smithi100 bash[74942]: cluster 2024-04-07T16:59:59.722760+0000 mgr.y (mgr.25175) 1339 : cluster [DBG] pgmap v1997: 105 pgs: 105 active+clean; 693 KiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 90 KiB/s rd, 8.3 KiB/s wr, 120 op/s 2024-04-07T17:00:00.212 INFO:tasks.workunit.client.1.smithi144.stderr:+ prefix=rbd_data.d6c6e476c56e 2024-04-07T17:00:00.214 INFO:tasks.workunit.client.1.smithi144.stderr:+ grep rbd_data.d6c6e476c56e 2024-04-07T17:00:00.214 INFO:tasks.workunit.client.1.smithi144.stderr:+ get_image_data_pool sparse2 2024-04-07T17:00:00.214 INFO:tasks.workunit.client.1.smithi144.stderr:+ image=sparse2 2024-04-07T17:00:00.214 INFO:tasks.workunit.client.1.smithi144.stderr:+ sed -e s/rbd_data.d6c6e476c56e\.// -e s/^0*\([0-9a-f]\)/\1/ 2024-04-07T17:00:00.214 INFO:tasks.workunit.client.1.smithi144.stderr:+ sort -u 2024-04-07T17:00:00.214 INFO:tasks.workunit.client.1.smithi144.stderr:+ rbd info sparse2 2024-04-07T17:00:00.214 INFO:tasks.workunit.client.1.smithi144.stderr:+ grep data_pool: 2024-04-07T17:00:00.214 INFO:tasks.workunit.client.1.smithi144.stderr:+ awk -F: { print $NF } 2024-04-07T17:00:00.265 INFO:tasks.workunit.client.1.smithi144.stderr:+ data_pool= 2024-04-07T17:00:00.265 INFO:tasks.workunit.client.1.smithi144.stderr:+ [ -z ] 2024-04-07T17:00:00.265 INFO:tasks.workunit.client.1.smithi144.stderr:+ data_pool=rbd 2024-04-07T17:00:00.265 INFO:tasks.workunit.client.1.smithi144.stderr:+ echo rbd 2024-04-07T17:00:00.265 INFO:tasks.workunit.client.1.smithi144.stderr:+ rados ls -p rbd 2024-04-07T17:00:00.384 INFO:tasks.workunit.client.1.smithi144.stderr:+ objects=0 2024-04-07T17:00:00.385 INFO:tasks.workunit.client.1.smithi144.stderr:2 2024-04-07T17:00:00.385 INFO:tasks.workunit.client.1.smithi144.stderr:3 2024-04-07T17:00:00.385 INFO:tasks.workunit.client.1.smithi144.stderr:+ echo 0 2 3 2024-04-07T17:00:00.385 INFO:tasks.workunit.client.1.smithi144.stderr:+ [ 0 -eq 1 -o 0 2 3 = 0 2 3 ] 2024-04-07T17:00:00.385 INFO:tasks.workunit.client.1.smithi144.stderr:+ rbd export sparse2 /tmp/rbd_import_export_103522/sparse2.out 2024-04-07T17:00:00.447 INFO:tasks.workunit.client.1.smithi144.stderr: Exporting image: 25% complete... Exporting image: 50% complete... Exporting image: 75% complete... Exporting image: 100% complete...done. 2024-04-07T17:00:00.452 INFO:tasks.workunit.client.1.smithi144.stderr:+ compare_files_and_ondisk_sizes /tmp/rbd_import_export_103522/sparse2 /tmp/rbd_import_export_103522/sparse2.out 2024-04-07T17:00:00.452 INFO:tasks.workunit.client.1.smithi144.stderr:+ cmp -l /tmp/rbd_import_export_103522/sparse2 /tmp/rbd_import_export_103522/sparse2.out 2024-04-07T17:00:00.455 INFO:tasks.workunit.client.1.smithi144.stderr:+ stat /tmp/rbd_import_export_103522/sparse2 --format %b 2024-04-07T17:00:00.456 INFO:tasks.workunit.client.1.smithi144.stderr:+ origsize=6144 2024-04-07T17:00:00.456 INFO:tasks.workunit.client.1.smithi144.stderr:+ stat /tmp/rbd_import_export_103522/sparse2.out --format %b 2024-04-07T17:00:00.457 INFO:tasks.workunit.client.1.smithi144.stderr:+ exportsize=6144 2024-04-07T17:00:00.457 INFO:tasks.workunit.client.1.smithi144.stderr:+ difference=0 2024-04-07T17:00:00.457 INFO:tasks.workunit.client.1.smithi144.stderr:+ difference=0 2024-04-07T17:00:00.457 INFO:tasks.workunit.client.1.smithi144.stderr:+ test 0 -ge 0 -a 0 -lt 4096 2024-04-07T17:00:00.457 INFO:tasks.workunit.client.1.smithi144.stderr:+ rm /tmp/rbd_import_export_103522/sparse2.out 2024-04-07T17:00:00.458 INFO:tasks.workunit.client.1.smithi144.stderr:+ rbd rm sparse2 2024-04-07T17:00:00.697 INFO:tasks.workunit.client.1.smithi144.stderr: Removing image: 25% complete... Removing image: 50% complete... Removing image: 75% complete... Removing image: 100% complete...done. 2024-04-07T17:00:00.706 INFO:tasks.workunit.client.1.smithi144.stdout:partially-sparse file imports to partially-sparse image 2024-04-07T17:00:00.706 INFO:tasks.workunit.client.1.smithi144.stderr:+ echo partially-sparse file imports to partially-sparse image 2024-04-07T17:00:00.706 INFO:tasks.workunit.client.1.smithi144.stderr:+ rbd import --new-format --order 20 /tmp/rbd_import_export_103522/sparse1 sparse 2024-04-07T17:00:00.835 INFO:tasks.workunit.client.1.smithi144.stderr: Importing image: 10% complete... Importing image: 20% complete... Importing image: 30% complete... Importing image: 40% complete... Importing image: 50% complete... Importing image: 60% complete... Importing image: 70% complete... Importing image: 80% complete... Importing image: 90% complete... Importing image: 100% complete...done. 2024-04-07T17:00:00.836 INFO:tasks.workunit.client.1.smithi144.stderr:rbd: --new-format is deprecated, use --image-format 2 2024-04-07T17:00:00.836 INFO:tasks.workunit.client.1.smithi144.stderr:rbd: --order is deprecated, use --object-size 2024-04-07T17:00:00.842 INFO:tasks.workunit.client.1.smithi144.stderr:+ objects sparse 2024-04-07T17:00:00.842 INFO:tasks.workunit.client.1.smithi144.stderr:+ image=sparse 2024-04-07T17:00:00.842 INFO:tasks.workunit.client.1.smithi144.stderr:+ rbd info+ sparse 2024-04-07T17:00:00.843 INFO:tasks.workunit.client.1.smithi144.stderr:grep block_name_prefix 2024-04-07T17:00:00.843 INFO:tasks.workunit.client.1.smithi144.stderr:+ awk {print $NF;} 2024-04-07T17:00:00.902 INFO:tasks.workunit.client.1.smithi144.stderr:+ prefix=rbd_data.d6f3a0957fef 2024-04-07T17:00:00.903 INFO:tasks.workunit.client.1.smithi144.stderr:+ grep rbd_data.d6f3a0957fef 2024-04-07T17:00:00.903 INFO:tasks.workunit.client.1.smithi144.stderr:+ get_image_data_pool sparse 2024-04-07T17:00:00.903 INFO:tasks.workunit.client.1.smithi144.stderr:+ + image=sparse 2024-04-07T17:00:00.903 INFO:tasks.workunit.client.1.smithi144.stderr:sed -e s/rbd_data.d6f3a0957fef\.// -e s/^0*\([0-9a-f]\)/\1/ 2024-04-07T17:00:00.903 INFO:tasks.workunit.client.1.smithi144.stderr:+ sort -u 2024-04-07T17:00:00.904 INFO:tasks.workunit.client.1.smithi144.stderr:+ rbd info sparse 2024-04-07T17:00:00.904 INFO:tasks.workunit.client.1.smithi144.stderr:+ grep data_pool: 2024-04-07T17:00:00.904 INFO:tasks.workunit.client.1.smithi144.stderr:+ awk -F: { print $NF } 2024-04-07T17:00:00.956 INFO:tasks.workunit.client.1.smithi144.stderr:+ data_pool= 2024-04-07T17:00:00.956 INFO:tasks.workunit.client.1.smithi144.stderr:+ [ -z ] 2024-04-07T17:00:00.956 INFO:tasks.workunit.client.1.smithi144.stderr:+ data_pool=rbd 2024-04-07T17:00:00.957 INFO:tasks.workunit.client.1.smithi144.stderr:+ echo rbd 2024-04-07T17:00:00.957 INFO:tasks.workunit.client.1.smithi144.stderr:+ rados ls -p rbd 2024-04-07T17:00:01.079 INFO:tasks.workunit.client.1.smithi144.stderr:+ objects=1 2024-04-07T17:00:01.079 INFO:tasks.workunit.client.1.smithi144.stderr:+ echo 1 2024-04-07T17:00:01.079 INFO:tasks.workunit.client.1.smithi144.stderr:+ [ 0 -eq 1 -o 1 = 1 ] 2024-04-07T17:00:01.079 INFO:tasks.workunit.client.1.smithi144.stderr:+ rbd rm sparse 2024-04-07T17:00:01.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:00:00 smithi144 bash[66654]: cluster 2024-04-07T17:00:00.000167+0000 mon.a (mon.0) 2254 : cluster [INF] overall HEALTH_OK 2024-04-07T17:00:01.200 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:00:00 smithi100 bash[74942]: cluster 2024-04-07T17:00:00.000167+0000 mon.a (mon.0) 2254 : cluster [INF] overall HEALTH_OK 2024-04-07T17:00:01.200 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:00:00 smithi100 bash[76202]: cluster 2024-04-07T17:00:00.000167+0000 mon.a (mon.0) 2254 : cluster [INF] overall HEALTH_OK 2024-04-07T17:00:01.318 INFO:tasks.workunit.client.1.smithi144.stderr: Removing image: 10% complete... Removing image: 20% complete... Removing image: 30% complete... Removing image: 40% complete... Removing image: 50% complete... Removing image: 60% complete... Removing image: 70% complete... Removing image: 80% complete... Removing image: 90% complete... Removing image: 100% complete...done. 2024-04-07T17:00:01.322 INFO:tasks.workunit.client.1.smithi144.stderr:+ echo zeros import through stdin to sparse image 2024-04-07T17:00:01.322 INFO:tasks.workunit.client.1.smithi144.stdout:zeros import through stdin to sparse image 2024-04-07T17:00:01.323 INFO:tasks.workunit.client.1.smithi144.stderr:+ dd if=/dev/zero bs=1M count=4 2024-04-07T17:00:01.323 INFO:tasks.workunit.client.1.smithi144.stderr:+ rbd import --new-format - sparse 2024-04-07T17:00:01.406 INFO:tasks.workunit.client.1.smithi144.stderr:4+0 records in 2024-04-07T17:00:01.407 INFO:tasks.workunit.client.1.smithi144.stderr:4+0 records out 2024-04-07T17:00:01.407 INFO:tasks.workunit.client.1.smithi144.stderr:4194304 bytes (4.2 MB, 4.0 MiB) copied, 0.0834683 s, 50.3 MB/s 2024-04-07T17:00:01.434 INFO:tasks.workunit.client.1.smithi144.stderr: Importing image: 100% complete...done. 2024-04-07T17:00:01.434 INFO:tasks.workunit.client.1.smithi144.stderr:rbd: --new-format is deprecated, use --image-format 2 2024-04-07T17:00:01.439 INFO:tasks.workunit.client.1.smithi144.stderr:+ objects sparse 2024-04-07T17:00:01.439 INFO:tasks.workunit.client.1.smithi144.stderr:+ image=sparse 2024-04-07T17:00:01.439 INFO:tasks.workunit.client.1.smithi144.stderr:+ rbd info sparse 2024-04-07T17:00:01.440 INFO:tasks.workunit.client.1.smithi144.stderr:+ grep block_name_prefix 2024-04-07T17:00:01.440 INFO:tasks.workunit.client.1.smithi144.stderr:+ awk {print $NF;} 2024-04-07T17:00:01.495 INFO:tasks.workunit.client.1.smithi144.stderr:+ prefix=rbd_data.d714a1d7c865 2024-04-07T17:00:01.495 INFO:tasks.workunit.client.1.smithi144.stderr:+ grep rbd_data.d714a1d7c865 2024-04-07T17:00:01.495 INFO:tasks.workunit.client.1.smithi144.stderr:+ get_image_data_pool sparse 2024-04-07T17:00:01.495 INFO:tasks.workunit.client.1.smithi144.stderr:+ image=sparse 2024-04-07T17:00:01.495 INFO:tasks.workunit.client.1.smithi144.stderr:+ sed -e s/rbd_data.d714a1d7c865\.// -e s/^0*\([0-9a-f]\)/\1/ 2024-04-07T17:00:01.495 INFO:tasks.workunit.client.1.smithi144.stderr:+ sort -u 2024-04-07T17:00:01.495 INFO:tasks.workunit.client.1.smithi144.stderr:+ rbd info sparse 2024-04-07T17:00:01.495 INFO:tasks.workunit.client.1.smithi144.stderr:+ grep data_pool: 2024-04-07T17:00:01.495 INFO:tasks.workunit.client.1.smithi144.stderr:+ awk -F: { print $NF } 2024-04-07T17:00:01.546 INFO:tasks.workunit.client.1.smithi144.stderr:+ data_pool= 2024-04-07T17:00:01.546 INFO:tasks.workunit.client.1.smithi144.stderr:+ [ -z ] 2024-04-07T17:00:01.546 INFO:tasks.workunit.client.1.smithi144.stderr:+ data_pool=rbd 2024-04-07T17:00:01.547 INFO:tasks.workunit.client.1.smithi144.stderr:+ echo rbd 2024-04-07T17:00:01.547 INFO:tasks.workunit.client.1.smithi144.stderr:+ rados ls -p rbd 2024-04-07T17:00:01.670 INFO:tasks.workunit.client.1.smithi144.stderr:+ objects= 2024-04-07T17:00:01.670 INFO:tasks.workunit.client.1.smithi144.stderr:+ echo 2024-04-07T17:00:01.670 INFO:tasks.workunit.client.1.smithi144.stderr:+ [ 0 -eq 1 -o = ] 2024-04-07T17:00:01.670 INFO:tasks.workunit.client.1.smithi144.stderr:+ rbd rm sparse 2024-04-07T17:00:01.919 INFO:tasks.workunit.client.1.smithi144.stderr: Removing image: 100% complete...done. 2024-04-07T17:00:01.927 INFO:tasks.workunit.client.1.smithi144.stdout:zeros export to sparse file 2024-04-07T17:00:01.927 INFO:tasks.workunit.client.1.smithi144.stderr:+ echo zeros export to sparse file 2024-04-07T17:00:01.927 INFO:tasks.workunit.client.1.smithi144.stderr:+ rbd create --new-format sparse --size 4 2024-04-07T17:00:01.994 INFO:tasks.workunit.client.1.smithi144.stderr:rbd: --new-format is deprecated, use --image-format 2 2024-04-07T17:00:01.998 INFO:tasks.workunit.client.1.smithi144.stderr:+ rbd info sparse 2024-04-07T17:00:01.998 INFO:tasks.workunit.client.1.smithi144.stderr:+ grep block_name_prefix 2024-04-07T17:00:01.998 INFO:tasks.workunit.client.1.smithi144.stderr:+ awk {print $NF;} 2024-04-07T17:00:02.054 INFO:tasks.workunit.client.1.smithi144.stderr:+ prefix=rbd_data.d73572a0ee87 2024-04-07T17:00:02.054 INFO:tasks.workunit.client.1.smithi144.stderr:+ dd if=/dev/zero bs=4M count=1 2024-04-07T17:00:02.054 INFO:tasks.workunit.client.1.smithi144.stderr:+ get_image_data_pool sparse 2024-04-07T17:00:02.054 INFO:tasks.workunit.client.1.smithi144.stderr:+ image=sparse 2024-04-07T17:00:02.054 INFO:tasks.workunit.client.1.smithi144.stderr:+ rbd info sparse 2024-04-07T17:00:02.055 INFO:tasks.workunit.client.1.smithi144.stderr:+ grep data_pool: 2024-04-07T17:00:02.055 INFO:tasks.workunit.client.1.smithi144.stderr:+ awk -F: { print $NF } 2024-04-07T17:00:02.108 INFO:tasks.workunit.client.1.smithi144.stderr:+ data_pool= 2024-04-07T17:00:02.109 INFO:tasks.workunit.client.1.smithi144.stderr:+ [ -z ] 2024-04-07T17:00:02.109 INFO:tasks.workunit.client.1.smithi144.stderr:+ data_pool=rbd 2024-04-07T17:00:02.109 INFO:tasks.workunit.client.1.smithi144.stderr:+ echo rbd 2024-04-07T17:00:02.109 INFO:tasks.workunit.client.1.smithi144.stderr:+ rados -p rbd put rbd_data.d73572a0ee87.000000000000 - 2024-04-07T17:00:02.113 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:00:01 smithi144 bash[66654]: cluster 2024-04-07T17:00:01.723663+0000 mgr.y (mgr.25175) 1340 : cluster [DBG] pgmap v1998: 105 pgs: 105 active+clean; 583 KiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 127 KiB/s rd, 9.7 KiB/s wr, 166 op/s 2024-04-07T17:00:02.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:00:01 smithi144 bash[66654]: audit 2024-04-07T17:00:01.793656+0000 mon.a (mon.0) 2255 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:00:02.150 INFO:tasks.workunit.client.1.smithi144.stderr:1+0 records in 2024-04-07T17:00:02.150 INFO:tasks.workunit.client.1.smithi144.stderr:1+0 records out 2024-04-07T17:00:02.150 INFO:tasks.workunit.client.1.smithi144.stderr:4194304 bytes (4.2 MB, 4.0 MiB) copied, 0.0967135 s, 43.4 MB/s 2024-04-07T17:00:02.174 INFO:tasks.workunit.client.1.smithi144.stderr:+ objects sparse 2024-04-07T17:00:02.174 INFO:tasks.workunit.client.1.smithi144.stderr:+ image=sparse 2024-04-07T17:00:02.175 INFO:tasks.workunit.client.1.smithi144.stderr:+ rbd info sparse 2024-04-07T17:00:02.175 INFO:tasks.workunit.client.1.smithi144.stderr:+ grep block_name_prefix 2024-04-07T17:00:02.175 INFO:tasks.workunit.client.1.smithi144.stderr:+ awk {print $NF;} 2024-04-07T17:00:02.200 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:00:01 smithi100 bash[74942]: cluster 2024-04-07T17:00:01.723663+0000 mgr.y (mgr.25175) 1340 : cluster [DBG] pgmap v1998: 105 pgs: 105 active+clean; 583 KiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 127 KiB/s rd, 9.7 KiB/s wr, 166 op/s 2024-04-07T17:00:02.200 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:00:01 smithi100 bash[74942]: audit 2024-04-07T17:00:01.793656+0000 mon.a (mon.0) 2255 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:00:02.200 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:00:01 smithi100 bash[76202]: cluster 2024-04-07T17:00:01.723663+0000 mgr.y (mgr.25175) 1340 : cluster [DBG] pgmap v1998: 105 pgs: 105 active+clean; 583 KiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 127 KiB/s rd, 9.7 KiB/s wr, 166 op/s 2024-04-07T17:00:02.200 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:00:01 smithi100 bash[76202]: audit 2024-04-07T17:00:01.793656+0000 mon.a (mon.0) 2255 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:00:02.223 INFO:tasks.workunit.client.1.smithi144.stderr:+ prefix=rbd_data.d73572a0ee87 2024-04-07T17:00:02.225 INFO:tasks.workunit.client.1.smithi144.stderr:+ grep rbd_data.d73572a0ee87 2024-04-07T17:00:02.225 INFO:tasks.workunit.client.1.smithi144.stderr:+ get_image_data_pool sparse 2024-04-07T17:00:02.225 INFO:tasks.workunit.client.1.smithi144.stderr:+ image=sparse 2024-04-07T17:00:02.225 INFO:tasks.workunit.client.1.smithi144.stderr:+ sed -e s/rbd_data.d73572a0ee87\.// -e s/^0*\([0-9a-f]\)/\1/ 2024-04-07T17:00:02.225 INFO:tasks.workunit.client.1.smithi144.stderr:+ sort -u 2024-04-07T17:00:02.225 INFO:tasks.workunit.client.1.smithi144.stderr:+ rbd info sparse 2024-04-07T17:00:02.225 INFO:tasks.workunit.client.1.smithi144.stderr:+ grep data_pool: 2024-04-07T17:00:02.225 INFO:tasks.workunit.client.1.smithi144.stderr:+ awk -F: { print $NF } 2024-04-07T17:00:02.276 INFO:tasks.workunit.client.1.smithi144.stderr:+ data_pool= 2024-04-07T17:00:02.277 INFO:tasks.workunit.client.1.smithi144.stderr:+ [ -z ] 2024-04-07T17:00:02.277 INFO:tasks.workunit.client.1.smithi144.stderr:+ data_pool=rbd 2024-04-07T17:00:02.277 INFO:tasks.workunit.client.1.smithi144.stderr:+ echo rbd 2024-04-07T17:00:02.277 INFO:tasks.workunit.client.1.smithi144.stderr:+ rados ls -p rbd 2024-04-07T17:00:02.377 INFO:tasks.workunit.client.1.smithi144.stderr:+ objects=0 2024-04-07T17:00:02.377 INFO:tasks.workunit.client.1.smithi144.stderr:+ echo 0 2024-04-07T17:00:02.377 INFO:tasks.workunit.client.1.smithi144.stderr:+ [ 0 -eq 1 -o 0 = 0 ] 2024-04-07T17:00:02.377 INFO:tasks.workunit.client.1.smithi144.stderr:+ rm /tmp/rbd_import_export_103522/sparse 2024-04-07T17:00:02.377 INFO:tasks.workunit.client.1.smithi144.stderr:rm: cannot remove '/tmp/rbd_import_export_103522/sparse': No such file or directory 2024-04-07T17:00:02.377 INFO:tasks.workunit.client.1.smithi144.stderr:+ true 2024-04-07T17:00:02.377 INFO:tasks.workunit.client.1.smithi144.stderr:+ rbd export sparse /tmp/rbd_import_export_103522/sparse 2024-04-07T17:00:02.431 INFO:tasks.workunit.client.1.smithi144.stderr: Exporting image: 100% complete...done. 2024-04-07T17:00:02.435 INFO:tasks.workunit.client.1.smithi144.stderr:+ stat /tmp/rbd_import_export_103522/sparse --format=%b 2024-04-07T17:00:02.436 INFO:tasks.workunit.client.1.smithi144.stderr:+ [ 0 = 0 ] 2024-04-07T17:00:02.436 INFO:tasks.workunit.client.1.smithi144.stderr:+ rbd rm sparse 2024-04-07T17:00:02.614 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:00:02 smithi144 bash[95233]: ts=2024-04-07T17:00:02.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:00:02.676 INFO:tasks.workunit.client.1.smithi144.stderr: Removing image: 100% complete...done. 2024-04-07T17:00:02.682 INFO:tasks.workunit.client.1.smithi144.stderr:+ rm /tmp/rbd_import_export_103522/sparse /tmp/rbd_import_export_103522/sparse1 /tmp/rbd_import_export_103522/sparse2 /tmp/rbd_import_export_103522/sparse3 2024-04-07T17:00:02.686 INFO:tasks.workunit.client.1.smithi144.stderr:rm: cannot remove '/tmp/rbd_import_export_103522/sparse3': No such file or directory 2024-04-07T17:00:02.686 INFO:tasks.workunit.client.1.smithi144.stderr:+ true 2024-04-07T17:00:02.686 INFO:tasks.workunit.client.1.smithi144.stderr:+ echo OK 2024-04-07T17:00:02.687 INFO:tasks.workunit.client.1.smithi144.stdout:OK 2024-04-07T17:00:02.687 INFO:tasks.workunit.client.1.smithi144.stderr:+ rm -rf /tmp/rbd_import_export_103522 2024-04-07T17:00:02.688 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-07T17:00:02.689 DEBUG:teuthology.orchestra.run.smithi144:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.1/client.1/tmp 2024-04-07T17:00:02.742 INFO:tasks.workunit:Stopping ['rbd/import_export.sh'] on client.1... 2024-04-07T17:00:02.743 DEBUG:teuthology.orchestra.run.smithi144:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.1 /home/ubuntu/cephtest/clone.client.1 2024-04-07T17:00:03.015 DEBUG:teuthology.parallel:result is None 2024-04-07T17:00:03.016 DEBUG:teuthology.orchestra.run.smithi144:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.1/client.1 2024-04-07T17:00:03.028 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.1/client.1 2024-04-07T17:00:03.028 DEBUG:teuthology.orchestra.run.smithi144:> rmdir -- /home/ubuntu/cephtest/mnt.1 2024-04-07T17:00:03.074 INFO:tasks.workunit:Deleted artificial mount point /home/ubuntu/cephtest/mnt.1/client.1 2024-04-07T17:00:03.074 INFO:teuthology.task.full_sequential:In full_sequential, running task print... 2024-04-07T17:00:03.074 INFO:teuthology.task.print:**** done end rbd_import_export.yaml 2024-04-07T17:00:03.074 INFO:teuthology.task.full_sequential:In full_sequential, running task print... 2024-04-07T17:00:03.074 INFO:teuthology.task.print:**** done start test_rbd_api.yaml 2024-04-07T17:00:03.074 INFO:teuthology.task.full_sequential:In full_sequential, running task workunit... 2024-04-07T17:00:03.075 INFO:tasks.workunit:Pulling workunits from ref reef 2024-04-07T17:00:03.075 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-04-07T17:00:03.075 DEBUG:teuthology.orchestra.run.smithi100:> stat -- /home/ubuntu/cephtest/mnt.0 2024-04-07T17:00:03.082 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-07T17:00:03.082 INFO:teuthology.orchestra.run.smithi100.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-04-07T17:00:03.082 DEBUG:teuthology.orchestra.run.smithi100:> mkdir -- /home/ubuntu/cephtest/mnt.0 2024-04-07T17:00:03.129 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2024-04-07T17:00:03.129 DEBUG:teuthology.orchestra.run.smithi100:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2024-04-07T17:00:03.177 INFO:tasks.workunit:timeout=3h 2024-04-07T17:00:03.177 INFO:tasks.workunit:cleanup=True 2024-04-07T17:00:03.177 DEBUG:teuthology.orchestra.run.smithi100:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone --depth 1 --branch reef https://github.com/ceph/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 2024-04-07T17:00:03.227 INFO:tasks.workunit.client.0.smithi100.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-04-07T17:00:03.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:00:03 smithi144 bash[66654]: cluster 2024-04-07T17:00:02.503181+0000 mon.a (mon.0) 2256 : cluster [DBG] osdmap e1084: 8 total, 8 up, 8 in 2024-04-07T17:00:03.950 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:00:03 smithi100 bash[74942]: cluster 2024-04-07T17:00:02.503181+0000 mon.a (mon.0) 2256 : cluster [DBG] osdmap e1084: 8 total, 8 up, 8 in 2024-04-07T17:00:03.950 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:00:03 smithi100 bash[76202]: cluster 2024-04-07T17:00:02.503181+0000 mon.a (mon.0) 2256 : cluster [DBG] osdmap e1084: 8 total, 8 up, 8 in 2024-04-07T17:00:04.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:00:04 smithi144 bash[66654]: cluster 2024-04-07T17:00:03.724422+0000 mgr.y (mgr.25175) 1341 : cluster [DBG] pgmap v2000: 105 pgs: 105 active+clean; 583 KiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 591 KiB/s rd, 525 KiB/s wr, 275 op/s 2024-04-07T17:00:04.950 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:00:04 smithi100 bash[74942]: cluster 2024-04-07T17:00:03.724422+0000 mgr.y (mgr.25175) 1341 : cluster [DBG] pgmap v2000: 105 pgs: 105 active+clean; 583 KiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 591 KiB/s rd, 525 KiB/s wr, 275 op/s 2024-04-07T17:00:04.950 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:00:04 smithi100 bash[76202]: cluster 2024-04-07T17:00:03.724422+0000 mgr.y (mgr.25175) 1341 : cluster [DBG] pgmap v2000: 105 pgs: 105 active+clean; 583 KiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 591 KiB/s rd, 525 KiB/s wr, 275 op/s 2024-04-07T17:00:06.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:00:05 smithi144 bash[66654]: cluster 2024-04-07T17:00:05.725394+0000 mgr.y (mgr.25175) 1342 : cluster [DBG] pgmap v2001: 105 pgs: 105 active+clean; 583 KiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 916 KiB/s rd, 794 KiB/s wr, 331 op/s 2024-04-07T17:00:06.200 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:00:05 smithi100 bash[74942]: cluster 2024-04-07T17:00:05.725394+0000 mgr.y (mgr.25175) 1342 : cluster [DBG] pgmap v2001: 105 pgs: 105 active+clean; 583 KiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 916 KiB/s rd, 794 KiB/s wr, 331 op/s 2024-04-07T17:00:06.200 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:00:05 smithi100 bash[76202]: cluster 2024-04-07T17:00:05.725394+0000 mgr.y (mgr.25175) 1342 : cluster [DBG] pgmap v2001: 105 pgs: 105 active+clean; 583 KiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 916 KiB/s rd, 794 KiB/s wr, 331 op/s 2024-04-07T17:00:08.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:00:07 smithi144 bash[66654]: cluster 2024-04-07T17:00:07.726366+0000 mgr.y (mgr.25175) 1343 : cluster [DBG] pgmap v2002: 105 pgs: 105 active+clean; 4.6 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 795 KiB/s rd, 1.1 MiB/s wr, 243 op/s 2024-04-07T17:00:08.200 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:00:07 smithi100 bash[76202]: cluster 2024-04-07T17:00:07.726366+0000 mgr.y (mgr.25175) 1343 : cluster [DBG] pgmap v2002: 105 pgs: 105 active+clean; 4.6 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 795 KiB/s rd, 1.1 MiB/s wr, 243 op/s 2024-04-07T17:00:08.200 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:00:07 smithi100 bash[74942]: cluster 2024-04-07T17:00:07.726366+0000 mgr.y (mgr.25175) 1343 : cluster [DBG] pgmap v2002: 105 pgs: 105 active+clean; 4.6 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 795 KiB/s rd, 1.1 MiB/s wr, 243 op/s 2024-04-07T17:00:08.700 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:00:08 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:00:08] "GET /metrics HTTP/1.1" 200 34047 "" "Prometheus/2.43.0" 2024-04-07T17:00:09.356 DEBUG:teuthology.orchestra.run.smithi100:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-04-07T17:00:09.405 INFO:tasks.workunit.client.0.smithi100.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-04-07T17:00:09.407 INFO:tasks.workunit.client.0.smithi100.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-07T17:00:09.407 INFO:tasks.workunit.client.0.smithi100.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-04-07T17:00:09.465 INFO:tasks.workunit.client.0.smithi100.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-04-07T17:00:09.522 INFO:tasks.workunit.client.0.smithi100.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-04-07T17:00:09.568 INFO:tasks.workunit.client.0.smithi100.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-07T17:00:09.569 INFO:tasks.workunit.client.0.smithi100.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-07T17:00:09.570 INFO:tasks.workunit.client.0.smithi100.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-04-07T17:00:09.611 INFO:tasks.workunit.client.0.smithi100.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-07T17:00:09.614 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-07T17:00:09.614 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-04-07T17:00:09.660 INFO:tasks.workunit:Running workunits matching rbd/test_librbd.sh on client.0... 2024-04-07T17:00:09.661 INFO:tasks.workunit:Running workunit rbd/test_librbd.sh... 2024-04-07T17:00:09.661 DEBUG:teuthology.orchestra.run.smithi100:workunit test rbd/test_librbd.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=reef TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 RBD_FEATURES=61 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rbd/test_librbd.sh 2024-04-07T17:00:09.774 INFO:tasks.workunit.client.0.smithi100.stderr:warning: failed to enable lockdep 2024-04-07T17:00:09.774 INFO:tasks.workunit.client.0.smithi100.stdout:seed 115303 2024-04-07T17:00:09.775 INFO:tasks.workunit.client.0.smithi100.stdout:[==========] Running 480 tests from 24 test suites. 2024-04-07T17:00:09.775 INFO:tasks.workunit.client.0.smithi100.stdout:[----------] Global test environment set-up. 2024-04-07T17:00:09.775 INFO:tasks.workunit.client.0.smithi100.stdout:[----------] 1 test from TestJournalStress 2024-04-07T17:00:10.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:00:09 smithi144 bash[66654]: cluster 2024-04-07T17:00:09.727316+0000 mgr.y (mgr.25175) 1344 : cluster [DBG] pgmap v2003: 105 pgs: 105 active+clean; 4.6 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 791 KiB/s rd, 1.1 MiB/s wr, 236 op/s 2024-04-07T17:00:10.200 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:00:09 smithi100 bash[74942]: cluster 2024-04-07T17:00:09.727316+0000 mgr.y (mgr.25175) 1344 : cluster [DBG] pgmap v2003: 105 pgs: 105 active+clean; 4.6 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 791 KiB/s rd, 1.1 MiB/s wr, 236 op/s 2024-04-07T17:00:10.200 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:00:09 smithi100 bash[76202]: cluster 2024-04-07T17:00:09.727316+0000 mgr.y (mgr.25175) 1344 : cluster [DBG] pgmap v2003: 105 pgs: 105 active+clean; 4.6 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 791 KiB/s rd, 1.1 MiB/s wr, 236 op/s 2024-04-07T17:00:11.806 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestJournalStress.DiscardWithPruneWriteOverlap 2024-04-07T17:00:12.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:00:11 smithi144 bash[66654]: cluster 2024-04-07T17:00:10.791388+0000 mon.a (mon.0) 2257 : cluster [DBG] osdmap e1085: 8 total, 8 up, 8 in 2024-04-07T17:00:12.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:00:11 smithi144 bash[66654]: audit 2024-04-07T17:00:10.797631+0000 mon.c (mon.2) 60 : audit [INF] from='client.? 172.21.15.100:0/946505682' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:00:12.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:00:11 smithi144 bash[66654]: audit 2024-04-07T17:00:10.798329+0000 mon.a (mon.0) 2258 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:00:12.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:00:11 smithi144 bash[66654]: cluster 2024-04-07T17:00:11.728085+0000 mgr.y (mgr.25175) 1345 : cluster [DBG] pgmap v2005: 137 pgs: 32 unknown, 105 active+clean; 4.6 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 455 KiB/s rd, 895 KiB/s wr, 159 op/s 2024-04-07T17:00:12.200 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:00:11 smithi100 bash[74942]: cluster 2024-04-07T17:00:10.791388+0000 mon.a (mon.0) 2257 : cluster [DBG] osdmap e1085: 8 total, 8 up, 8 in 2024-04-07T17:00:12.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:00:11 smithi100 bash[74942]: audit 2024-04-07T17:00:10.797631+0000 mon.c (mon.2) 60 : audit [INF] from='client.? 172.21.15.100:0/946505682' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:00:12.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:00:11 smithi100 bash[74942]: audit 2024-04-07T17:00:10.798329+0000 mon.a (mon.0) 2258 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:00:12.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:00:11 smithi100 bash[74942]: cluster 2024-04-07T17:00:11.728085+0000 mgr.y (mgr.25175) 1345 : cluster [DBG] pgmap v2005: 137 pgs: 32 unknown, 105 active+clean; 4.6 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 455 KiB/s rd, 895 KiB/s wr, 159 op/s 2024-04-07T17:00:12.201 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:00:11 smithi100 bash[76202]: cluster 2024-04-07T17:00:10.791388+0000 mon.a (mon.0) 2257 : cluster [DBG] osdmap e1085: 8 total, 8 up, 8 in 2024-04-07T17:00:12.201 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:00:11 smithi100 bash[76202]: audit 2024-04-07T17:00:10.797631+0000 mon.c (mon.2) 60 : audit [INF] from='client.? 172.21.15.100:0/946505682' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:00:12.201 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:00:11 smithi100 bash[76202]: audit 2024-04-07T17:00:10.798329+0000 mon.a (mon.0) 2258 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:00:12.201 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:00:11 smithi100 bash[76202]: cluster 2024-04-07T17:00:11.728085+0000 mgr.y (mgr.25175) 1345 : cluster [DBG] pgmap v2005: 137 pgs: 32 unknown, 105 active+clean; 4.6 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 455 KiB/s rd, 895 KiB/s wr, 159 op/s 2024-04-07T17:00:12.614 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:00:12 smithi144 bash[95233]: ts=2024-04-07T17:00:12.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:00:13.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:00:12 smithi144 bash[66654]: cluster 2024-04-07T17:00:11.783328+0000 mon.a (mon.0) 2259 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T17:00:13.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:00:12 smithi144 bash[66654]: audit 2024-04-07T17:00:11.793501+0000 mon.a (mon.0) 2260 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T17:00:13.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:00:12 smithi144 bash[66654]: cluster 2024-04-07T17:00:11.803751+0000 mon.a (mon.0) 2261 : cluster [DBG] osdmap e1086: 8 total, 8 up, 8 in 2024-04-07T17:00:13.200 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:00:12 smithi100 bash[74942]: cluster 2024-04-07T17:00:11.783328+0000 mon.a (mon.0) 2259 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T17:00:13.200 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:00:12 smithi100 bash[74942]: audit 2024-04-07T17:00:11.793501+0000 mon.a (mon.0) 2260 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T17:00:13.200 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:00:12 smithi100 bash[74942]: cluster 2024-04-07T17:00:11.803751+0000 mon.a (mon.0) 2261 : cluster [DBG] osdmap e1086: 8 total, 8 up, 8 in 2024-04-07T17:00:13.200 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:00:12 smithi100 bash[76202]: cluster 2024-04-07T17:00:11.783328+0000 mon.a (mon.0) 2259 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T17:00:13.201 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:00:12 smithi100 bash[76202]: audit 2024-04-07T17:00:11.793501+0000 mon.a (mon.0) 2260 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T17:00:13.201 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:00:12 smithi100 bash[76202]: cluster 2024-04-07T17:00:11.803751+0000 mon.a (mon.0) 2261 : cluster [DBG] osdmap e1086: 8 total, 8 up, 8 in 2024-04-07T17:00:13.877 INFO:tasks.workunit.client.0.smithi100.stdout:./src/test/librbd/journal/test_Stress.cc:37: Skipped 2024-04-07T17:00:13.877 INFO:tasks.workunit.client.0.smithi100.stdout:Skipping due to unmet REQUIRE 2024-04-07T17:00:13.877 INFO:tasks.workunit.client.0.smithi100.stdout:[ SKIPPED ] TestJournalStress.DiscardWithPruneWriteOverlap (2071 ms) 2024-04-07T17:00:14.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:00:13 smithi144 bash[66654]: cluster 2024-04-07T17:00:12.800381+0000 mon.a (mon.0) 2262 : cluster [DBG] osdmap e1087: 8 total, 8 up, 8 in 2024-04-07T17:00:14.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:00:13 smithi144 bash[66654]: cluster 2024-04-07T17:00:13.728887+0000 mgr.y (mgr.25175) 1346 : cluster [DBG] pgmap v2008: 137 pgs: 13 unknown, 124 active+clean; 4.6 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-04-07T17:00:14.200 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:00:13 smithi100 bash[74942]: cluster 2024-04-07T17:00:12.800381+0000 mon.a (mon.0) 2262 : cluster [DBG] osdmap e1087: 8 total, 8 up, 8 in 2024-04-07T17:00:14.200 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:00:13 smithi100 bash[74942]: cluster 2024-04-07T17:00:13.728887+0000 mgr.y (mgr.25175) 1346 : cluster [DBG] pgmap v2008: 137 pgs: 13 unknown, 124 active+clean; 4.6 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-04-07T17:00:14.200 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:00:13 smithi100 bash[76202]: cluster 2024-04-07T17:00:12.800381+0000 mon.a (mon.0) 2262 : cluster [DBG] osdmap e1087: 8 total, 8 up, 8 in 2024-04-07T17:00:14.200 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:00:13 smithi100 bash[76202]: cluster 2024-04-07T17:00:13.728887+0000 mgr.y (mgr.25175) 1346 : cluster [DBG] pgmap v2008: 137 pgs: 13 unknown, 124 active+clean; 4.6 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-04-07T17:00:14.834 INFO:tasks.workunit.client.0.smithi100.stdout:[----------] 1 test from TestJournalStress (2071 ms total) 2024-04-07T17:00:14.835 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:00:14.835 INFO:tasks.workunit.client.0.smithi100.stdout:[----------] 16 tests from TestJournalReplay 2024-04-07T17:00:15.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:00:14 smithi144 bash[66654]: cluster 2024-04-07T17:00:13.807059+0000 mon.a (mon.0) 2263 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T17:00:15.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:00:14 smithi144 bash[66654]: cluster 2024-04-07T17:00:13.807117+0000 mon.a (mon.0) 2264 : cluster [INF] Cluster is now healthy 2024-04-07T17:00:15.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:00:14 smithi144 bash[66654]: cluster 2024-04-07T17:00:13.827507+0000 mon.a (mon.0) 2265 : cluster [DBG] osdmap e1088: 8 total, 8 up, 8 in 2024-04-07T17:00:15.200 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:00:14 smithi100 bash[74942]: cluster 2024-04-07T17:00:13.807059+0000 mon.a (mon.0) 2263 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T17:00:15.200 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:00:14 smithi100 bash[74942]: cluster 2024-04-07T17:00:13.807117+0000 mon.a (mon.0) 2264 : cluster [INF] Cluster is now healthy 2024-04-07T17:00:15.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:00:14 smithi100 bash[74942]: cluster 2024-04-07T17:00:13.827507+0000 mon.a (mon.0) 2265 : cluster [DBG] osdmap e1088: 8 total, 8 up, 8 in 2024-04-07T17:00:15.201 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:00:14 smithi100 bash[76202]: cluster 2024-04-07T17:00:13.807059+0000 mon.a (mon.0) 2263 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T17:00:15.201 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:00:14 smithi100 bash[76202]: cluster 2024-04-07T17:00:13.807117+0000 mon.a (mon.0) 2264 : cluster [INF] Cluster is now healthy 2024-04-07T17:00:15.201 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:00:14 smithi100 bash[76202]: cluster 2024-04-07T17:00:13.827507+0000 mon.a (mon.0) 2265 : cluster [DBG] osdmap e1088: 8 total, 8 up, 8 in 2024-04-07T17:00:16.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:00:15 smithi144 bash[66654]: cluster 2024-04-07T17:00:14.830363+0000 mon.a (mon.0) 2266 : cluster [DBG] osdmap e1089: 8 total, 8 up, 8 in 2024-04-07T17:00:16.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:00:15 smithi144 bash[66654]: cluster 2024-04-07T17:00:15.729697+0000 mgr.y (mgr.25175) 1347 : cluster [DBG] pgmap v2011: 105 pgs: 105 active+clean; 4.6 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-04-07T17:00:16.200 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:00:15 smithi100 bash[74942]: cluster 2024-04-07T17:00:14.830363+0000 mon.a (mon.0) 2266 : cluster [DBG] osdmap e1089: 8 total, 8 up, 8 in 2024-04-07T17:00:16.200 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:00:15 smithi100 bash[74942]: cluster 2024-04-07T17:00:15.729697+0000 mgr.y (mgr.25175) 1347 : cluster [DBG] pgmap v2011: 105 pgs: 105 active+clean; 4.6 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-04-07T17:00:16.200 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:00:15 smithi100 bash[76202]: cluster 2024-04-07T17:00:14.830363+0000 mon.a (mon.0) 2266 : cluster [DBG] osdmap e1089: 8 total, 8 up, 8 in 2024-04-07T17:00:16.200 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:00:15 smithi100 bash[76202]: cluster 2024-04-07T17:00:15.729697+0000 mgr.y (mgr.25175) 1347 : cluster [DBG] pgmap v2011: 105 pgs: 105 active+clean; 4.6 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-04-07T17:00:16.848 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestJournalReplay.AioDiscardEvent 2024-04-07T17:00:17.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:00:16 smithi144 bash[66654]: cluster 2024-04-07T17:00:15.850093+0000 mon.a (mon.0) 2267 : cluster [DBG] osdmap e1090: 8 total, 8 up, 8 in 2024-04-07T17:00:17.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:00:16 smithi144 bash[66654]: audit 2024-04-07T17:00:15.853961+0000 mon.c (mon.2) 61 : audit [INF] from='client.? 172.21.15.100:0/3666298268' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:00:17.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:00:16 smithi144 bash[66654]: audit 2024-04-07T17:00:15.855230+0000 mon.a (mon.0) 2268 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:00:17.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:00:16 smithi144 bash[66654]: audit 2024-04-07T17:00:16.793949+0000 mon.a (mon.0) 2269 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:00:17.200 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:00:16 smithi100 bash[74942]: cluster 2024-04-07T17:00:15.850093+0000 mon.a (mon.0) 2267 : cluster [DBG] osdmap e1090: 8 total, 8 up, 8 in 2024-04-07T17:00:17.200 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:00:16 smithi100 bash[74942]: audit 2024-04-07T17:00:15.853961+0000 mon.c (mon.2) 61 : audit [INF] from='client.? 172.21.15.100:0/3666298268' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:00:17.200 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:00:16 smithi100 bash[74942]: audit 2024-04-07T17:00:15.855230+0000 mon.a (mon.0) 2268 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:00:17.200 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:00:16 smithi100 bash[74942]: audit 2024-04-07T17:00:16.793949+0000 mon.a (mon.0) 2269 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:00:17.200 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:00:16 smithi100 bash[76202]: cluster 2024-04-07T17:00:15.850093+0000 mon.a (mon.0) 2267 : cluster [DBG] osdmap e1090: 8 total, 8 up, 8 in 2024-04-07T17:00:17.201 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:00:16 smithi100 bash[76202]: audit 2024-04-07T17:00:15.853961+0000 mon.c (mon.2) 61 : audit [INF] from='client.? 172.21.15.100:0/3666298268' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:00:17.201 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:00:16 smithi100 bash[76202]: audit 2024-04-07T17:00:15.855230+0000 mon.a (mon.0) 2268 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:00:17.201 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:00:16 smithi100 bash[76202]: audit 2024-04-07T17:00:16.793949+0000 mon.a (mon.0) 2269 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:00:18.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:00:17 smithi144 bash[66654]: audit 2024-04-07T17:00:16.840332+0000 mon.a (mon.0) 2270 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T17:00:18.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:00:17 smithi144 bash[66654]: cluster 2024-04-07T17:00:16.850465+0000 mon.a (mon.0) 2271 : cluster [DBG] osdmap e1091: 8 total, 8 up, 8 in 2024-04-07T17:00:18.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:00:17 smithi144 bash[66654]: cluster 2024-04-07T17:00:17.730803+0000 mgr.y (mgr.25175) 1348 : cluster [DBG] pgmap v2014: 137 pgs: 20 unknown, 117 active+clean; 4.6 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-04-07T17:00:18.200 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:00:17 smithi100 bash[74942]: audit 2024-04-07T17:00:16.840332+0000 mon.a (mon.0) 2270 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T17:00:18.200 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:00:17 smithi100 bash[74942]: cluster 2024-04-07T17:00:16.850465+0000 mon.a (mon.0) 2271 : cluster [DBG] osdmap e1091: 8 total, 8 up, 8 in 2024-04-07T17:00:18.200 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:00:17 smithi100 bash[74942]: cluster 2024-04-07T17:00:17.730803+0000 mgr.y (mgr.25175) 1348 : cluster [DBG] pgmap v2014: 137 pgs: 20 unknown, 117 active+clean; 4.6 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-04-07T17:00:18.200 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:00:17 smithi100 bash[76202]: audit 2024-04-07T17:00:16.840332+0000 mon.a (mon.0) 2270 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T17:00:18.200 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:00:17 smithi100 bash[76202]: cluster 2024-04-07T17:00:16.850465+0000 mon.a (mon.0) 2271 : cluster [DBG] osdmap e1091: 8 total, 8 up, 8 in 2024-04-07T17:00:18.201 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:00:17 smithi100 bash[76202]: cluster 2024-04-07T17:00:17.730803+0000 mgr.y (mgr.25175) 1348 : cluster [DBG] pgmap v2014: 137 pgs: 20 unknown, 117 active+clean; 4.6 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-04-07T17:00:18.700 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:00:18 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:00:18] "GET /metrics HTTP/1.1" 200 36423 "" "Prometheus/2.43.0" 2024-04-07T17:00:18.922 INFO:tasks.workunit.client.0.smithi100.stdout:./src/test/librbd/journal/test_Replay.cc:109: Skipped 2024-04-07T17:00:18.922 INFO:tasks.workunit.client.0.smithi100.stdout:Skipping due to unmet REQUIRE 2024-04-07T17:00:18.922 INFO:tasks.workunit.client.0.smithi100.stdout:[ SKIPPED ] TestJournalReplay.AioDiscardEvent (2074 ms) 2024-04-07T17:00:18.923 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestJournalReplay.AioWriteEvent 2024-04-07T17:00:18.953 INFO:tasks.workunit.client.0.smithi100.stdout:./src/test/librbd/journal/test_Replay.cc:200: Skipped 2024-04-07T17:00:18.953 INFO:tasks.workunit.client.0.smithi100.stdout:Skipping due to unmet REQUIRE 2024-04-07T17:00:18.953 INFO:tasks.workunit.client.0.smithi100.stdout:[ SKIPPED ] TestJournalReplay.AioWriteEvent (31 ms) 2024-04-07T17:00:18.954 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestJournalReplay.AioFlushEvent 2024-04-07T17:00:18.976 INFO:tasks.workunit.client.0.smithi100.stdout:./src/test/librbd/journal/test_Replay.cc:261: Skipped 2024-04-07T17:00:18.977 INFO:tasks.workunit.client.0.smithi100.stdout:Skipping due to unmet REQUIRE 2024-04-07T17:00:18.977 INFO:tasks.workunit.client.0.smithi100.stdout:[ SKIPPED ] TestJournalReplay.AioFlushEvent (24 ms) 2024-04-07T17:00:18.977 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestJournalReplay.SnapCreate 2024-04-07T17:00:18.997 INFO:tasks.workunit.client.0.smithi100.stdout:./src/test/librbd/journal/test_Replay.cc:307: Skipped 2024-04-07T17:00:18.998 INFO:tasks.workunit.client.0.smithi100.stdout:Skipping due to unmet REQUIRE 2024-04-07T17:00:18.998 INFO:tasks.workunit.client.0.smithi100.stdout:[ SKIPPED ] TestJournalReplay.SnapCreate (21 ms) 2024-04-07T17:00:18.998 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestJournalReplay.SnapProtect 2024-04-07T17:00:19.019 INFO:tasks.workunit.client.0.smithi100.stdout:./src/test/librbd/journal/test_Replay.cc:348: Skipped 2024-04-07T17:00:19.019 INFO:tasks.workunit.client.0.smithi100.stdout:Skipping due to unmet REQUIRE 2024-04-07T17:00:19.019 INFO:tasks.workunit.client.0.smithi100.stdout:[ SKIPPED ] TestJournalReplay.SnapProtect (21 ms) 2024-04-07T17:00:19.019 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestJournalReplay.SnapUnprotect 2024-04-07T17:00:19.040 INFO:tasks.workunit.client.0.smithi100.stdout:./src/test/librbd/journal/test_Replay.cc:394: Skipped 2024-04-07T17:00:19.040 INFO:tasks.workunit.client.0.smithi100.stdout:Skipping due to unmet REQUIRE 2024-04-07T17:00:19.040 INFO:tasks.workunit.client.0.smithi100.stdout:[ SKIPPED ] TestJournalReplay.SnapUnprotect (21 ms) 2024-04-07T17:00:19.040 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestJournalReplay.SnapRename 2024-04-07T17:00:19.062 INFO:tasks.workunit.client.0.smithi100.stdout:./src/test/librbd/journal/test_Replay.cc:450: Skipped 2024-04-07T17:00:19.062 INFO:tasks.workunit.client.0.smithi100.stdout:Skipping due to unmet REQUIRE 2024-04-07T17:00:19.062 INFO:tasks.workunit.client.0.smithi100.stdout:[ SKIPPED ] TestJournalReplay.SnapRename (22 ms) 2024-04-07T17:00:19.062 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestJournalReplay.SnapRollback 2024-04-07T17:00:19.089 INFO:tasks.workunit.client.0.smithi100.stdout:./src/test/librbd/journal/test_Replay.cc:500: Skipped 2024-04-07T17:00:19.090 INFO:tasks.workunit.client.0.smithi100.stdout:Skipping due to unmet REQUIRE 2024-04-07T17:00:19.090 INFO:tasks.workunit.client.0.smithi100.stdout:[ SKIPPED ] TestJournalReplay.SnapRollback (27 ms) 2024-04-07T17:00:19.090 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestJournalReplay.SnapRemove 2024-04-07T17:00:19.111 INFO:tasks.workunit.client.0.smithi100.stdout:./src/test/librbd/journal/test_Replay.cc:541: Skipped 2024-04-07T17:00:19.112 INFO:tasks.workunit.client.0.smithi100.stdout:Skipping due to unmet REQUIRE 2024-04-07T17:00:19.112 INFO:tasks.workunit.client.0.smithi100.stdout:[ SKIPPED ] TestJournalReplay.SnapRemove (22 ms) 2024-04-07T17:00:19.112 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestJournalReplay.Rename 2024-04-07T17:00:19.132 INFO:tasks.workunit.client.0.smithi100.stdout:./src/test/librbd/journal/test_Replay.cc:590: Skipped 2024-04-07T17:00:19.132 INFO:tasks.workunit.client.0.smithi100.stdout:Skipping due to unmet REQUIRE 2024-04-07T17:00:19.132 INFO:tasks.workunit.client.0.smithi100.stdout:[ SKIPPED ] TestJournalReplay.Rename (21 ms) 2024-04-07T17:00:19.132 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestJournalReplay.Resize 2024-04-07T17:00:19.152 INFO:tasks.workunit.client.0.smithi100.stdout:./src/test/librbd/journal/test_Replay.cc:624: Skipped 2024-04-07T17:00:19.152 INFO:tasks.workunit.client.0.smithi100.stdout:Skipping due to unmet REQUIRE 2024-04-07T17:00:19.153 INFO:tasks.workunit.client.0.smithi100.stdout:[ SKIPPED ] TestJournalReplay.Resize (20 ms) 2024-04-07T17:00:19.153 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestJournalReplay.Flatten 2024-04-07T17:00:19.174 INFO:tasks.workunit.client.0.smithi100.stdout:./src/test/librbd/journal/test_Replay.cc:657: Skipped 2024-04-07T17:00:19.174 INFO:tasks.workunit.client.0.smithi100.stdout:Skipping due to unmet REQUIRE 2024-04-07T17:00:19.174 INFO:tasks.workunit.client.0.smithi100.stdout:[ SKIPPED ] TestJournalReplay.Flatten (22 ms) 2024-04-07T17:00:19.174 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestJournalReplay.UpdateFeatures 2024-04-07T17:00:19.196 INFO:tasks.workunit.client.0.smithi100.stdout:./src/test/librbd/journal/test_Replay.cc:704: Skipped 2024-04-07T17:00:19.196 INFO:tasks.workunit.client.0.smithi100.stdout:Skipping due to unmet REQUIRE 2024-04-07T17:00:19.196 INFO:tasks.workunit.client.0.smithi100.stdout:[ SKIPPED ] TestJournalReplay.UpdateFeatures (22 ms) 2024-04-07T17:00:19.196 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestJournalReplay.MetadataSet 2024-04-07T17:00:19.200 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:00:18 smithi100 bash[74942]: cluster 2024-04-07T17:00:17.875170+0000 mon.a (mon.0) 2272 : cluster [DBG] osdmap e1092: 8 total, 8 up, 8 in 2024-04-07T17:00:19.200 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:00:18 smithi100 bash[76202]: cluster 2024-04-07T17:00:17.875170+0000 mon.a (mon.0) 2272 : cluster [DBG] osdmap e1092: 8 total, 8 up, 8 in 2024-04-07T17:00:19.216 INFO:tasks.workunit.client.0.smithi100.stdout:./src/test/librbd/journal/test_Replay.cc:741: Skipped 2024-04-07T17:00:19.216 INFO:tasks.workunit.client.0.smithi100.stdout:Skipping due to unmet REQUIRE 2024-04-07T17:00:19.216 INFO:tasks.workunit.client.0.smithi100.stdout:[ SKIPPED ] TestJournalReplay.MetadataSet (20 ms) 2024-04-07T17:00:19.216 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestJournalReplay.MetadataRemove 2024-04-07T17:00:19.237 INFO:tasks.workunit.client.0.smithi100.stdout:./src/test/librbd/journal/test_Replay.cc:781: Skipped 2024-04-07T17:00:19.237 INFO:tasks.workunit.client.0.smithi100.stdout:Skipping due to unmet REQUIRE 2024-04-07T17:00:19.237 INFO:tasks.workunit.client.0.smithi100.stdout:[ SKIPPED ] TestJournalReplay.MetadataRemove (21 ms) 2024-04-07T17:00:19.237 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestJournalReplay.ObjectPosition 2024-04-07T17:00:19.258 INFO:tasks.workunit.client.0.smithi100.stdout:./src/test/librbd/journal/test_Replay.cc:824: Skipped 2024-04-07T17:00:19.258 INFO:tasks.workunit.client.0.smithi100.stdout:Skipping due to unmet REQUIRE 2024-04-07T17:00:19.258 INFO:tasks.workunit.client.0.smithi100.stdout:[ SKIPPED ] TestJournalReplay.ObjectPosition (21 ms) 2024-04-07T17:00:19.364 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:00:18 smithi144 bash[66654]: cluster 2024-04-07T17:00:17.875170+0000 mon.a (mon.0) 2272 : cluster [DBG] osdmap e1092: 8 total, 8 up, 8 in 2024-04-07T17:00:19.906 INFO:tasks.workunit.client.0.smithi100.stdout:[----------] 16 tests from TestJournalReplay (2410 ms total) 2024-04-07T17:00:19.906 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:00:19.907 INFO:tasks.workunit.client.0.smithi100.stdout:[----------] 4 tests from TestJournalEntries 2024-04-07T17:00:20.200 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:00:19 smithi100 bash[74942]: cluster 2024-04-07T17:00:18.882245+0000 mon.a (mon.0) 2273 : cluster [DBG] osdmap e1093: 8 total, 8 up, 8 in 2024-04-07T17:00:20.200 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:00:19 smithi100 bash[74942]: cluster 2024-04-07T17:00:19.731791+0000 mgr.y (mgr.25175) 1349 : cluster [DBG] pgmap v2017: 137 pgs: 4 unknown, 133 active+clean; 4.6 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 4.2 KiB/s rd, 3.5 KiB/s wr, 9 op/s 2024-04-07T17:00:20.200 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:00:19 smithi100 bash[76202]: cluster 2024-04-07T17:00:18.882245+0000 mon.a (mon.0) 2273 : cluster [DBG] osdmap e1093: 8 total, 8 up, 8 in 2024-04-07T17:00:20.200 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:00:19 smithi100 bash[76202]: cluster 2024-04-07T17:00:19.731791+0000 mgr.y (mgr.25175) 1349 : cluster [DBG] pgmap v2017: 137 pgs: 4 unknown, 133 active+clean; 4.6 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 4.2 KiB/s rd, 3.5 KiB/s wr, 9 op/s 2024-04-07T17:00:20.364 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:00:19 smithi144 bash[66654]: cluster 2024-04-07T17:00:18.882245+0000 mon.a (mon.0) 2273 : cluster [DBG] osdmap e1093: 8 total, 8 up, 8 in 2024-04-07T17:00:20.364 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:00:19 smithi144 bash[66654]: cluster 2024-04-07T17:00:19.731791+0000 mgr.y (mgr.25175) 1349 : cluster [DBG] pgmap v2017: 137 pgs: 4 unknown, 133 active+clean; 4.6 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 4.2 KiB/s rd, 3.5 KiB/s wr, 9 op/s 2024-04-07T17:00:21.200 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:00:20 smithi100 bash[74942]: cluster 2024-04-07T17:00:19.903361+0000 mon.a (mon.0) 2274 : cluster [DBG] osdmap e1094: 8 total, 8 up, 8 in 2024-04-07T17:00:21.200 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:00:20 smithi100 bash[76202]: cluster 2024-04-07T17:00:19.903361+0000 mon.a (mon.0) 2274 : cluster [DBG] osdmap e1094: 8 total, 8 up, 8 in 2024-04-07T17:00:21.364 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:00:20 smithi144 bash[66654]: cluster 2024-04-07T17:00:19.903361+0000 mon.a (mon.0) 2274 : cluster [DBG] osdmap e1094: 8 total, 8 up, 8 in 2024-04-07T17:00:21.905 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestJournalEntries.AioWrite 2024-04-07T17:00:22.202 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:00:21 smithi100 bash[74942]: cluster 2024-04-07T17:00:20.904320+0000 mon.a (mon.0) 2275 : cluster [DBG] osdmap e1095: 8 total, 8 up, 8 in 2024-04-07T17:00:22.202 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:00:21 smithi100 bash[74942]: audit 2024-04-07T17:00:20.906358+0000 mon.c (mon.2) 62 : audit [INF] from='client.? 172.21.15.100:0/3705131467' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:00:22.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:00:21 smithi100 bash[74942]: audit 2024-04-07T17:00:20.918025+0000 mon.a (mon.0) 2276 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:00:22.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:00:21 smithi100 bash[74942]: cluster 2024-04-07T17:00:21.732659+0000 mgr.y (mgr.25175) 1350 : cluster [DBG] pgmap v2020: 137 pgs: 32 unknown, 105 active+clean; 4.6 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-04-07T17:00:22.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:00:21 smithi100 bash[74942]: audit 2024-04-07T17:00:21.895530+0000 mon.a (mon.0) 2277 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T17:00:22.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:00:21 smithi100 bash[74942]: cluster 2024-04-07T17:00:21.902845+0000 mon.a (mon.0) 2278 : cluster [DBG] osdmap e1096: 8 total, 8 up, 8 in 2024-04-07T17:00:22.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:00:21 smithi100 bash[76202]: cluster 2024-04-07T17:00:20.904320+0000 mon.a (mon.0) 2275 : cluster [DBG] osdmap e1095: 8 total, 8 up, 8 in 2024-04-07T17:00:22.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:00:21 smithi100 bash[76202]: audit 2024-04-07T17:00:20.906358+0000 mon.c (mon.2) 62 : audit [INF] from='client.? 172.21.15.100:0/3705131467' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:00:22.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:00:21 smithi100 bash[76202]: audit 2024-04-07T17:00:20.918025+0000 mon.a (mon.0) 2276 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:00:22.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:00:21 smithi100 bash[76202]: cluster 2024-04-07T17:00:21.732659+0000 mgr.y (mgr.25175) 1350 : cluster [DBG] pgmap v2020: 137 pgs: 32 unknown, 105 active+clean; 4.6 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-04-07T17:00:22.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:00:21 smithi100 bash[76202]: audit 2024-04-07T17:00:21.895530+0000 mon.a (mon.0) 2277 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T17:00:22.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:00:21 smithi100 bash[76202]: cluster 2024-04-07T17:00:21.902845+0000 mon.a (mon.0) 2278 : cluster [DBG] osdmap e1096: 8 total, 8 up, 8 in 2024-04-07T17:00:22.314 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:00:21 smithi144 bash[66654]: cluster 2024-04-07T17:00:20.904320+0000 mon.a (mon.0) 2275 : cluster [DBG] osdmap e1095: 8 total, 8 up, 8 in 2024-04-07T17:00:22.314 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:00:21 smithi144 bash[66654]: audit 2024-04-07T17:00:20.906358+0000 mon.c (mon.2) 62 : audit [INF] from='client.? 172.21.15.100:0/3705131467' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:00:22.314 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:00:21 smithi144 bash[66654]: audit 2024-04-07T17:00:20.918025+0000 mon.a (mon.0) 2276 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:00:22.314 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:00:21 smithi144 bash[66654]: cluster 2024-04-07T17:00:21.732659+0000 mgr.y (mgr.25175) 1350 : cluster [DBG] pgmap v2020: 137 pgs: 32 unknown, 105 active+clean; 4.6 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-04-07T17:00:22.314 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:00:21 smithi144 bash[66654]: audit 2024-04-07T17:00:21.895530+0000 mon.a (mon.0) 2277 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T17:00:22.314 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:00:21 smithi144 bash[66654]: cluster 2024-04-07T17:00:21.902845+0000 mon.a (mon.0) 2278 : cluster [DBG] osdmap e1096: 8 total, 8 up, 8 in 2024-04-07T17:00:22.614 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:00:22 smithi144 bash[95233]: ts=2024-04-07T17:00:22.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:00:23.200 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:00:22 smithi100 bash[74942]: cluster 2024-04-07T17:00:21.909304+0000 mon.a (mon.0) 2279 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T17:00:23.200 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:00:22 smithi100 bash[76202]: cluster 2024-04-07T17:00:21.909304+0000 mon.a (mon.0) 2279 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T17:00:23.364 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:00:22 smithi144 bash[66654]: cluster 2024-04-07T17:00:21.909304+0000 mon.a (mon.0) 2279 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T17:00:23.981 INFO:tasks.workunit.client.0.smithi100.stdout:./src/test/librbd/journal/test_Entries.cc:118: Skipped 2024-04-07T17:00:23.981 INFO:tasks.workunit.client.0.smithi100.stdout:Skipping due to unmet REQUIRE 2024-04-07T17:00:23.981 INFO:tasks.workunit.client.0.smithi100.stdout:[ SKIPPED ] TestJournalEntries.AioWrite (2076 ms) 2024-04-07T17:00:23.982 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestJournalEntries.AioDiscard 2024-04-07T17:00:24.009 INFO:tasks.workunit.client.0.smithi100.stdout:./src/test/librbd/journal/test_Entries.cc:163: Skipped 2024-04-07T17:00:24.009 INFO:tasks.workunit.client.0.smithi100.stdout:Skipping due to unmet REQUIRE 2024-04-07T17:00:24.009 INFO:tasks.workunit.client.0.smithi100.stdout:[ SKIPPED ] TestJournalEntries.AioDiscard (28 ms) 2024-04-07T17:00:24.009 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestJournalEntries.AioDiscardWithPrune 2024-04-07T17:00:24.030 INFO:tasks.workunit.client.0.smithi100.stdout:./src/test/librbd/journal/test_Entries.cc:200: Skipped 2024-04-07T17:00:24.030 INFO:tasks.workunit.client.0.smithi100.stdout:Skipping due to unmet REQUIRE 2024-04-07T17:00:24.030 INFO:tasks.workunit.client.0.smithi100.stdout:[ SKIPPED ] TestJournalEntries.AioDiscardWithPrune (21 ms) 2024-04-07T17:00:24.030 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestJournalEntries.AioFlush 2024-04-07T17:00:24.051 INFO:tasks.workunit.client.0.smithi100.stdout:./src/test/librbd/journal/test_Entries.cc:263: Skipped 2024-04-07T17:00:24.052 INFO:tasks.workunit.client.0.smithi100.stdout:Skipping due to unmet REQUIRE 2024-04-07T17:00:24.052 INFO:tasks.workunit.client.0.smithi100.stdout:[ SKIPPED ] TestJournalEntries.AioFlush (21 ms) 2024-04-07T17:00:24.200 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:00:23 smithi100 bash[74942]: cluster 2024-04-07T17:00:22.934821+0000 mon.a (mon.0) 2280 : cluster [DBG] osdmap e1097: 8 total, 8 up, 8 in 2024-04-07T17:00:24.200 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:00:23 smithi100 bash[74942]: cluster 2024-04-07T17:00:23.733527+0000 mgr.y (mgr.25175) 1351 : cluster [DBG] pgmap v2023: 137 pgs: 2 creating+activating, 9 unknown, 126 active+clean; 4.6 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-04-07T17:00:24.200 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:00:23 smithi100 bash[76202]: cluster 2024-04-07T17:00:22.934821+0000 mon.a (mon.0) 2280 : cluster [DBG] osdmap e1097: 8 total, 8 up, 8 in 2024-04-07T17:00:24.201 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:00:23 smithi100 bash[76202]: cluster 2024-04-07T17:00:23.733527+0000 mgr.y (mgr.25175) 1351 : cluster [DBG] pgmap v2023: 137 pgs: 2 creating+activating, 9 unknown, 126 active+clean; 4.6 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-04-07T17:00:24.364 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:00:23 smithi144 bash[66654]: cluster 2024-04-07T17:00:22.934821+0000 mon.a (mon.0) 2280 : cluster [DBG] osdmap e1097: 8 total, 8 up, 8 in 2024-04-07T17:00:24.364 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:00:23 smithi144 bash[66654]: cluster 2024-04-07T17:00:23.733527+0000 mgr.y (mgr.25175) 1351 : cluster [DBG] pgmap v2023: 137 pgs: 2 creating+activating, 9 unknown, 126 active+clean; 4.6 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-04-07T17:00:24.953 INFO:tasks.workunit.client.0.smithi100.stdout:[----------] 4 tests from TestJournalEntries (2146 ms total) 2024-04-07T17:00:24.954 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:00:24.954 INFO:tasks.workunit.client.0.smithi100.stdout:[----------] 2 tests from TestTrash 2024-04-07T17:00:25.364 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:00:24 smithi144 bash[66654]: cluster 2024-04-07T17:00:23.928123+0000 mon.a (mon.0) 2281 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T17:00:25.364 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:00:24 smithi144 bash[66654]: cluster 2024-04-07T17:00:23.928224+0000 mon.a (mon.0) 2282 : cluster [INF] Cluster is now healthy 2024-04-07T17:00:25.364 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:00:24 smithi144 bash[66654]: cluster 2024-04-07T17:00:23.946402+0000 mon.a (mon.0) 2283 : cluster [DBG] osdmap e1098: 8 total, 8 up, 8 in 2024-04-07T17:00:25.450 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:00:24 smithi100 bash[74942]: cluster 2024-04-07T17:00:23.928123+0000 mon.a (mon.0) 2281 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T17:00:25.450 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:00:24 smithi100 bash[74942]: cluster 2024-04-07T17:00:23.928224+0000 mon.a (mon.0) 2282 : cluster [INF] Cluster is now healthy 2024-04-07T17:00:25.450 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:00:24 smithi100 bash[74942]: cluster 2024-04-07T17:00:23.946402+0000 mon.a (mon.0) 2283 : cluster [DBG] osdmap e1098: 8 total, 8 up, 8 in 2024-04-07T17:00:25.451 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:00:24 smithi100 bash[76202]: cluster 2024-04-07T17:00:23.928123+0000 mon.a (mon.0) 2281 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T17:00:25.451 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:00:24 smithi100 bash[76202]: cluster 2024-04-07T17:00:23.928224+0000 mon.a (mon.0) 2282 : cluster [INF] Cluster is now healthy 2024-04-07T17:00:25.451 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:00:24 smithi100 bash[76202]: cluster 2024-04-07T17:00:23.946402+0000 mon.a (mon.0) 2283 : cluster [DBG] osdmap e1098: 8 total, 8 up, 8 in 2024-04-07T17:00:26.364 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:00:25 smithi144 bash[66654]: cluster 2024-04-07T17:00:24.946819+0000 mon.a (mon.0) 2284 : cluster [DBG] osdmap e1099: 8 total, 8 up, 8 in 2024-04-07T17:00:26.364 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:00:25 smithi144 bash[66654]: cluster 2024-04-07T17:00:25.734320+0000 mgr.y (mgr.25175) 1352 : cluster [DBG] pgmap v2026: 105 pgs: 105 active+clean; 4.6 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-04-07T17:00:26.450 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:00:25 smithi100 bash[74942]: cluster 2024-04-07T17:00:24.946819+0000 mon.a (mon.0) 2284 : cluster [DBG] osdmap e1099: 8 total, 8 up, 8 in 2024-04-07T17:00:26.450 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:00:25 smithi100 bash[74942]: cluster 2024-04-07T17:00:25.734320+0000 mgr.y (mgr.25175) 1352 : cluster [DBG] pgmap v2026: 105 pgs: 105 active+clean; 4.6 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-04-07T17:00:26.451 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:00:25 smithi100 bash[76202]: cluster 2024-04-07T17:00:24.946819+0000 mon.a (mon.0) 2284 : cluster [DBG] osdmap e1099: 8 total, 8 up, 8 in 2024-04-07T17:00:26.451 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:00:25 smithi100 bash[76202]: cluster 2024-04-07T17:00:25.734320+0000 mgr.y (mgr.25175) 1352 : cluster [DBG] pgmap v2026: 105 pgs: 105 active+clean; 4.6 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-04-07T17:00:26.996 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestTrash.UserRemovingSource 2024-04-07T17:00:27.364 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:00:26 smithi144 bash[66654]: cluster 2024-04-07T17:00:25.987081+0000 mon.a (mon.0) 2285 : cluster [DBG] osdmap e1100: 8 total, 8 up, 8 in 2024-04-07T17:00:27.364 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:00:26 smithi144 bash[66654]: audit 2024-04-07T17:00:25.989476+0000 mon.a (mon.0) 2286 : audit [INF] from='client.? 172.21.15.100:0/2820282484' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:00:27.450 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:00:26 smithi100 bash[74942]: cluster 2024-04-07T17:00:25.987081+0000 mon.a (mon.0) 2285 : cluster [DBG] osdmap e1100: 8 total, 8 up, 8 in 2024-04-07T17:00:27.450 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:00:26 smithi100 bash[74942]: audit 2024-04-07T17:00:25.989476+0000 mon.a (mon.0) 2286 : audit [INF] from='client.? 172.21.15.100:0/2820282484' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:00:27.450 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:00:26 smithi100 bash[76202]: cluster 2024-04-07T17:00:25.987081+0000 mon.a (mon.0) 2285 : cluster [DBG] osdmap e1100: 8 total, 8 up, 8 in 2024-04-07T17:00:27.450 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:00:26 smithi100 bash[76202]: audit 2024-04-07T17:00:25.989476+0000 mon.a (mon.0) 2286 : audit [INF] from='client.? 172.21.15.100:0/2820282484' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:00:28.364 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:00:28 smithi144 bash[66654]: audit 2024-04-07T17:00:26.988248+0000 mon.a (mon.0) 2287 : audit [INF] from='client.? 172.21.15.100:0/2820282484' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T17:00:28.364 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:00:28 smithi144 bash[66654]: cluster 2024-04-07T17:00:26.996451+0000 mon.a (mon.0) 2288 : cluster [DBG] osdmap e1101: 8 total, 8 up, 8 in 2024-04-07T17:00:28.364 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:00:28 smithi144 bash[66654]: cluster 2024-04-07T17:00:27.734992+0000 mgr.y (mgr.25175) 1353 : cluster [DBG] pgmap v2029: 137 pgs: 10 creating+peering, 22 unknown, 105 active+clean; 4.6 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-04-07T17:00:28.425 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:00:28 smithi100 bash[74942]: audit 2024-04-07T17:00:26.988248+0000 mon.a (mon.0) 2287 : audit [INF] from='client.? 172.21.15.100:0/2820282484' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T17:00:28.425 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:00:28 smithi100 bash[74942]: cluster 2024-04-07T17:00:26.996451+0000 mon.a (mon.0) 2288 : cluster [DBG] osdmap e1101: 8 total, 8 up, 8 in 2024-04-07T17:00:28.425 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:00:28 smithi100 bash[74942]: cluster 2024-04-07T17:00:27.734992+0000 mgr.y (mgr.25175) 1353 : cluster [DBG] pgmap v2029: 137 pgs: 10 creating+peering, 22 unknown, 105 active+clean; 4.6 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-04-07T17:00:28.426 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:00:28 smithi100 bash[76202]: audit 2024-04-07T17:00:26.988248+0000 mon.a (mon.0) 2287 : audit [INF] from='client.? 172.21.15.100:0/2820282484' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T17:00:28.426 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:00:28 smithi100 bash[76202]: cluster 2024-04-07T17:00:26.996451+0000 mon.a (mon.0) 2288 : cluster [DBG] osdmap e1101: 8 total, 8 up, 8 in 2024-04-07T17:00:28.426 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:00:28 smithi100 bash[76202]: cluster 2024-04-07T17:00:27.734992+0000 mgr.y (mgr.25175) 1353 : cluster [DBG] pgmap v2029: 137 pgs: 10 creating+peering, 22 unknown, 105 active+clean; 4.6 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-04-07T17:00:28.700 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:00:28 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:00:28] "GET /metrics HTTP/1.1" 200 36423 "" "Prometheus/2.43.0" 2024-04-07T17:00:29.205 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestTrash.UserRemovingSource (2209 ms) 2024-04-07T17:00:29.205 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestTrash.RestoreMirroringSource 2024-04-07T17:00:29.313 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestTrash.RestoreMirroringSource (107 ms) 2024-04-07T17:00:29.364 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:00:29 smithi144 bash[66654]: cluster 2024-04-07T17:00:28.008696+0000 mon.a (mon.0) 2289 : cluster [DBG] osdmap e1102: 8 total, 8 up, 8 in 2024-04-07T17:00:29.450 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:00:29 smithi100 bash[74942]: cluster 2024-04-07T17:00:28.008696+0000 mon.a (mon.0) 2289 : cluster [DBG] osdmap e1102: 8 total, 8 up, 8 in 2024-04-07T17:00:29.450 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:00:29 smithi100 bash[76202]: cluster 2024-04-07T17:00:28.008696+0000 mon.a (mon.0) 2289 : cluster [DBG] osdmap e1102: 8 total, 8 up, 8 in 2024-04-07T17:00:30.031 INFO:tasks.workunit.client.0.smithi100.stdout:[----------] 2 tests from TestTrash (2316 ms total) 2024-04-07T17:00:30.031 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:00:30.031 INFO:tasks.workunit.client.0.smithi100.stdout:[----------] 1 test from TestOperations 2024-04-07T17:00:30.364 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:00:30 smithi144 bash[66654]: cluster 2024-04-07T17:00:29.013592+0000 mon.a (mon.0) 2290 : cluster [DBG] osdmap e1103: 8 total, 8 up, 8 in 2024-04-07T17:00:30.364 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:00:30 smithi144 bash[66654]: cluster 2024-04-07T17:00:29.735893+0000 mgr.y (mgr.25175) 1354 : cluster [DBG] pgmap v2032: 137 pgs: 10 creating+peering, 3 unknown, 124 active+clean; 4.6 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-04-07T17:00:30.450 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:00:30 smithi100 bash[76202]: cluster 2024-04-07T17:00:29.013592+0000 mon.a (mon.0) 2290 : cluster [DBG] osdmap e1103: 8 total, 8 up, 8 in 2024-04-07T17:00:30.450 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:00:30 smithi100 bash[76202]: cluster 2024-04-07T17:00:29.735893+0000 mgr.y (mgr.25175) 1354 : cluster [DBG] pgmap v2032: 137 pgs: 10 creating+peering, 3 unknown, 124 active+clean; 4.6 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-04-07T17:00:30.450 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:00:30 smithi100 bash[74942]: cluster 2024-04-07T17:00:29.013592+0000 mon.a (mon.0) 2290 : cluster [DBG] osdmap e1103: 8 total, 8 up, 8 in 2024-04-07T17:00:30.451 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:00:30 smithi100 bash[74942]: cluster 2024-04-07T17:00:29.735893+0000 mgr.y (mgr.25175) 1354 : cluster [DBG] pgmap v2032: 137 pgs: 10 creating+peering, 3 unknown, 124 active+clean; 4.6 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-04-07T17:00:31.364 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:00:31 smithi144 bash[66654]: cluster 2024-04-07T17:00:30.026524+0000 mon.a (mon.0) 2291 : cluster [DBG] osdmap e1104: 8 total, 8 up, 8 in 2024-04-07T17:00:31.450 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:00:31 smithi100 bash[74942]: cluster 2024-04-07T17:00:30.026524+0000 mon.a (mon.0) 2291 : cluster [DBG] osdmap e1104: 8 total, 8 up, 8 in 2024-04-07T17:00:31.450 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:00:31 smithi100 bash[76202]: cluster 2024-04-07T17:00:30.026524+0000 mon.a (mon.0) 2291 : cluster [DBG] osdmap e1104: 8 total, 8 up, 8 in 2024-04-07T17:00:32.032 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestOperations.DisableJournalingCorrupt 2024-04-07T17:00:32.314 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:00:32 smithi144 bash[66654]: cluster 2024-04-07T17:00:31.030998+0000 mon.a (mon.0) 2292 : cluster [DBG] osdmap e1105: 8 total, 8 up, 8 in 2024-04-07T17:00:32.314 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:00:32 smithi144 bash[66654]: audit 2024-04-07T17:00:31.034872+0000 mon.c (mon.2) 63 : audit [INF] from='client.? 172.21.15.100:0/3476281725' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:00:32.314 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:00:32 smithi144 bash[66654]: audit 2024-04-07T17:00:31.036267+0000 mon.a (mon.0) 2293 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:00:32.314 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:00:32 smithi144 bash[66654]: cluster 2024-04-07T17:00:31.736646+0000 mgr.y (mgr.25175) 1355 : cluster [DBG] pgmap v2035: 137 pgs: 32 unknown, 105 active+clean; 4.6 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-04-07T17:00:32.314 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:00:32 smithi144 bash[66654]: audit 2024-04-07T17:00:31.794346+0000 mon.a (mon.0) 2294 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:00:32.314 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:00:32 smithi144 bash[66654]: audit 2024-04-07T17:00:32.022952+0000 mon.a (mon.0) 2295 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T17:00:32.314 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:00:32 smithi144 bash[66654]: cluster 2024-04-07T17:00:32.031695+0000 mon.a (mon.0) 2296 : cluster [DBG] osdmap e1106: 8 total, 8 up, 8 in 2024-04-07T17:00:32.450 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:00:32 smithi100 bash[74942]: cluster 2024-04-07T17:00:31.030998+0000 mon.a (mon.0) 2292 : cluster [DBG] osdmap e1105: 8 total, 8 up, 8 in 2024-04-07T17:00:32.450 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:00:32 smithi100 bash[74942]: audit 2024-04-07T17:00:31.034872+0000 mon.c (mon.2) 63 : audit [INF] from='client.? 172.21.15.100:0/3476281725' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:00:32.450 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:00:32 smithi100 bash[74942]: audit 2024-04-07T17:00:31.036267+0000 mon.a (mon.0) 2293 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:00:32.451 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:00:32 smithi100 bash[74942]: cluster 2024-04-07T17:00:31.736646+0000 mgr.y (mgr.25175) 1355 : cluster [DBG] pgmap v2035: 137 pgs: 32 unknown, 105 active+clean; 4.6 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-04-07T17:00:32.451 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:00:32 smithi100 bash[74942]: audit 2024-04-07T17:00:31.794346+0000 mon.a (mon.0) 2294 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:00:32.451 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:00:32 smithi100 bash[74942]: audit 2024-04-07T17:00:32.022952+0000 mon.a (mon.0) 2295 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T17:00:32.451 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:00:32 smithi100 bash[74942]: cluster 2024-04-07T17:00:32.031695+0000 mon.a (mon.0) 2296 : cluster [DBG] osdmap e1106: 8 total, 8 up, 8 in 2024-04-07T17:00:32.451 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:00:32 smithi100 bash[76202]: cluster 2024-04-07T17:00:31.030998+0000 mon.a (mon.0) 2292 : cluster [DBG] osdmap e1105: 8 total, 8 up, 8 in 2024-04-07T17:00:32.451 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:00:32 smithi100 bash[76202]: audit 2024-04-07T17:00:31.034872+0000 mon.c (mon.2) 63 : audit [INF] from='client.? 172.21.15.100:0/3476281725' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:00:32.451 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:00:32 smithi100 bash[76202]: audit 2024-04-07T17:00:31.036267+0000 mon.a (mon.0) 2293 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:00:32.451 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:00:32 smithi100 bash[76202]: cluster 2024-04-07T17:00:31.736646+0000 mgr.y (mgr.25175) 1355 : cluster [DBG] pgmap v2035: 137 pgs: 32 unknown, 105 active+clean; 4.6 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-04-07T17:00:32.451 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:00:32 smithi100 bash[76202]: audit 2024-04-07T17:00:31.794346+0000 mon.a (mon.0) 2294 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:00:32.451 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:00:32 smithi100 bash[76202]: audit 2024-04-07T17:00:32.022952+0000 mon.a (mon.0) 2295 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T17:00:32.451 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:00:32 smithi100 bash[76202]: cluster 2024-04-07T17:00:32.031695+0000 mon.a (mon.0) 2296 : cluster [DBG] osdmap e1106: 8 total, 8 up, 8 in 2024-04-07T17:00:32.614 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:00:32 smithi144 bash[95233]: ts=2024-04-07T17:00:32.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:00:33.364 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:00:33 smithi144 bash[66654]: cluster 2024-04-07T17:00:32.039990+0000 mon.a (mon.0) 2297 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T17:00:33.450 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:00:33 smithi100 bash[74942]: cluster 2024-04-07T17:00:32.039990+0000 mon.a (mon.0) 2297 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T17:00:33.450 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:00:33 smithi100 bash[76202]: cluster 2024-04-07T17:00:32.039990+0000 mon.a (mon.0) 2297 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T17:00:34.117 INFO:tasks.workunit.client.0.smithi100.stdout:./src/test/librbd/test_Operations.cc:18: Skipped 2024-04-07T17:00:34.117 INFO:tasks.workunit.client.0.smithi100.stdout:Skipping due to unmet REQUIRE 2024-04-07T17:00:34.117 INFO:tasks.workunit.client.0.smithi100.stdout:[ SKIPPED ] TestOperations.DisableJournalingCorrupt (2085 ms) 2024-04-07T17:00:34.364 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:00:34 smithi144 bash[66654]: cluster 2024-04-07T17:00:33.060308+0000 mon.a (mon.0) 2298 : cluster [DBG] osdmap e1107: 8 total, 8 up, 8 in 2024-04-07T17:00:34.364 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:00:34 smithi144 bash[66654]: cluster 2024-04-07T17:00:33.737346+0000 mgr.y (mgr.25175) 1356 : cluster [DBG] pgmap v2038: 137 pgs: 14 creating+peering, 16 unknown, 107 active+clean; 4.6 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-04-07T17:00:34.450 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:00:34 smithi100 bash[74942]: cluster 2024-04-07T17:00:33.060308+0000 mon.a (mon.0) 2298 : cluster [DBG] osdmap e1107: 8 total, 8 up, 8 in 2024-04-07T17:00:34.450 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:00:34 smithi100 bash[74942]: cluster 2024-04-07T17:00:33.737346+0000 mgr.y (mgr.25175) 1356 : cluster [DBG] pgmap v2038: 137 pgs: 14 creating+peering, 16 unknown, 107 active+clean; 4.6 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-04-07T17:00:34.450 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:00:34 smithi100 bash[76202]: cluster 2024-04-07T17:00:33.060308+0000 mon.a (mon.0) 2298 : cluster [DBG] osdmap e1107: 8 total, 8 up, 8 in 2024-04-07T17:00:34.451 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:00:34 smithi100 bash[76202]: cluster 2024-04-07T17:00:33.737346+0000 mgr.y (mgr.25175) 1356 : cluster [DBG] pgmap v2038: 137 pgs: 14 creating+peering, 16 unknown, 107 active+clean; 4.6 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-04-07T17:00:35.099 INFO:tasks.workunit.client.0.smithi100.stdout:[----------] 1 test from TestOperations (2085 ms total) 2024-04-07T17:00:35.099 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:00:35.099 INFO:tasks.workunit.client.0.smithi100.stdout:[----------] 4 tests from TestObjectMap 2024-04-07T17:00:35.364 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:00:35 smithi144 bash[66654]: cluster 2024-04-07T17:00:34.053218+0000 mon.a (mon.0) 2299 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T17:00:35.364 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:00:35 smithi144 bash[66654]: cluster 2024-04-07T17:00:34.053275+0000 mon.a (mon.0) 2300 : cluster [INF] Cluster is now healthy 2024-04-07T17:00:35.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:00:35 smithi144 bash[66654]: cluster 2024-04-07T17:00:34.072116+0000 mon.a (mon.0) 2301 : cluster [DBG] osdmap e1108: 8 total, 8 up, 8 in 2024-04-07T17:00:35.450 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:00:35 smithi100 bash[74942]: cluster 2024-04-07T17:00:34.053218+0000 mon.a (mon.0) 2299 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T17:00:35.450 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:00:35 smithi100 bash[74942]: cluster 2024-04-07T17:00:34.053275+0000 mon.a (mon.0) 2300 : cluster [INF] Cluster is now healthy 2024-04-07T17:00:35.450 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:00:35 smithi100 bash[74942]: cluster 2024-04-07T17:00:34.072116+0000 mon.a (mon.0) 2301 : cluster [DBG] osdmap e1108: 8 total, 8 up, 8 in 2024-04-07T17:00:35.451 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:00:35 smithi100 bash[76202]: cluster 2024-04-07T17:00:34.053218+0000 mon.a (mon.0) 2299 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T17:00:35.451 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:00:35 smithi100 bash[76202]: cluster 2024-04-07T17:00:34.053275+0000 mon.a (mon.0) 2300 : cluster [INF] Cluster is now healthy 2024-04-07T17:00:35.451 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:00:35 smithi100 bash[76202]: cluster 2024-04-07T17:00:34.072116+0000 mon.a (mon.0) 2301 : cluster [DBG] osdmap e1108: 8 total, 8 up, 8 in 2024-04-07T17:00:36.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:00:36 smithi144 bash[66654]: cluster 2024-04-07T17:00:35.086384+0000 mon.a (mon.0) 2302 : cluster [DBG] osdmap e1109: 8 total, 8 up, 8 in 2024-04-07T17:00:36.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:00:36 smithi144 bash[66654]: cluster 2024-04-07T17:00:35.738084+0000 mgr.y (mgr.25175) 1357 : cluster [DBG] pgmap v2041: 105 pgs: 105 active+clean; 4.6 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-04-07T17:00:36.450 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:00:36 smithi100 bash[74942]: cluster 2024-04-07T17:00:35.086384+0000 mon.a (mon.0) 2302 : cluster [DBG] osdmap e1109: 8 total, 8 up, 8 in 2024-04-07T17:00:36.450 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:00:36 smithi100 bash[74942]: cluster 2024-04-07T17:00:35.738084+0000 mgr.y (mgr.25175) 1357 : cluster [DBG] pgmap v2041: 105 pgs: 105 active+clean; 4.6 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-04-07T17:00:36.450 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:00:36 smithi100 bash[76202]: cluster 2024-04-07T17:00:35.086384+0000 mon.a (mon.0) 2302 : cluster [DBG] osdmap e1109: 8 total, 8 up, 8 in 2024-04-07T17:00:36.451 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:00:36 smithi100 bash[76202]: cluster 2024-04-07T17:00:35.738084+0000 mgr.y (mgr.25175) 1357 : cluster [DBG] pgmap v2041: 105 pgs: 105 active+clean; 4.6 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-04-07T17:00:37.110 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestObjectMap.RefreshInvalidatesWhenCorrupt 2024-04-07T17:00:37.364 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:00:37 smithi144 bash[66654]: cluster 2024-04-07T17:00:36.100815+0000 mon.a (mon.0) 2303 : cluster [DBG] osdmap e1110: 8 total, 8 up, 8 in 2024-04-07T17:00:37.364 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:00:37 smithi144 bash[66654]: audit 2024-04-07T17:00:36.102506+0000 mon.a (mon.0) 2304 : audit [INF] from='client.? 172.21.15.100:0/369476332' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:00:37.450 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:00:37 smithi100 bash[76202]: cluster 2024-04-07T17:00:36.100815+0000 mon.a (mon.0) 2303 : cluster [DBG] osdmap e1110: 8 total, 8 up, 8 in 2024-04-07T17:00:37.450 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:00:37 smithi100 bash[76202]: audit 2024-04-07T17:00:36.102506+0000 mon.a (mon.0) 2304 : audit [INF] from='client.? 172.21.15.100:0/369476332' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:00:37.450 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:00:37 smithi100 bash[74942]: cluster 2024-04-07T17:00:36.100815+0000 mon.a (mon.0) 2303 : cluster [DBG] osdmap e1110: 8 total, 8 up, 8 in 2024-04-07T17:00:37.451 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:00:37 smithi100 bash[74942]: audit 2024-04-07T17:00:36.102506+0000 mon.a (mon.0) 2304 : audit [INF] from='client.? 172.21.15.100:0/369476332' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:00:38.425 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:00:38 smithi100 bash[74942]: audit 2024-04-07T17:00:37.100321+0000 mon.a (mon.0) 2305 : audit [INF] from='client.? 172.21.15.100:0/369476332' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T17:00:38.425 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:00:38 smithi100 bash[74942]: cluster 2024-04-07T17:00:37.110513+0000 mon.a (mon.0) 2306 : cluster [DBG] osdmap e1111: 8 total, 8 up, 8 in 2024-04-07T17:00:38.425 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:00:38 smithi100 bash[74942]: cluster 2024-04-07T17:00:37.738817+0000 mgr.y (mgr.25175) 1358 : cluster [DBG] pgmap v2044: 137 pgs: 12 creating+peering, 20 unknown, 105 active+clean; 4.6 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-04-07T17:00:38.426 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:00:38 smithi100 bash[76202]: audit 2024-04-07T17:00:37.100321+0000 mon.a (mon.0) 2305 : audit [INF] from='client.? 172.21.15.100:0/369476332' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T17:00:38.426 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:00:38 smithi100 bash[76202]: cluster 2024-04-07T17:00:37.110513+0000 mon.a (mon.0) 2306 : cluster [DBG] osdmap e1111: 8 total, 8 up, 8 in 2024-04-07T17:00:38.426 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:00:38 smithi100 bash[76202]: cluster 2024-04-07T17:00:37.738817+0000 mgr.y (mgr.25175) 1358 : cluster [DBG] pgmap v2044: 137 pgs: 12 creating+peering, 20 unknown, 105 active+clean; 4.6 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-04-07T17:00:38.614 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:00:38 smithi144 bash[66654]: audit 2024-04-07T17:00:37.100321+0000 mon.a (mon.0) 2305 : audit [INF] from='client.? 172.21.15.100:0/369476332' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T17:00:38.614 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:00:38 smithi144 bash[66654]: cluster 2024-04-07T17:00:37.110513+0000 mon.a (mon.0) 2306 : cluster [DBG] osdmap e1111: 8 total, 8 up, 8 in 2024-04-07T17:00:38.614 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:00:38 smithi144 bash[66654]: cluster 2024-04-07T17:00:37.738817+0000 mgr.y (mgr.25175) 1358 : cluster [DBG] pgmap v2044: 137 pgs: 12 creating+peering, 20 unknown, 105 active+clean; 4.6 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-04-07T17:00:38.700 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:00:38 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:00:38] "GET /metrics HTTP/1.1" 200 36422 "" "Prometheus/2.43.0" 2024-04-07T17:00:39.221 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestObjectMap.RefreshInvalidatesWhenCorrupt (2111 ms) 2024-04-07T17:00:39.221 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestObjectMap.RefreshInvalidatesWhenTooSmall 2024-04-07T17:00:39.275 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestObjectMap.RefreshInvalidatesWhenTooSmall (54 ms) 2024-04-07T17:00:39.275 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestObjectMap.InvalidateFlagOnDisk 2024-04-07T17:00:39.358 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestObjectMap.InvalidateFlagOnDisk (83 ms) 2024-04-07T17:00:39.358 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestObjectMap.AcquireLockInvalidatesWhenTooSmall 2024-04-07T17:00:39.419 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestObjectMap.AcquireLockInvalidatesWhenTooSmall (61 ms) 2024-04-07T17:00:39.450 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:00:39 smithi100 bash[74942]: cluster 2024-04-07T17:00:38.126711+0000 mon.a (mon.0) 2307 : cluster [DBG] osdmap e1112: 8 total, 8 up, 8 in 2024-04-07T17:00:39.450 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:00:39 smithi100 bash[76202]: cluster 2024-04-07T17:00:38.126711+0000 mon.a (mon.0) 2307 : cluster [DBG] osdmap e1112: 8 total, 8 up, 8 in 2024-04-07T17:00:39.614 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:00:39 smithi144 bash[66654]: cluster 2024-04-07T17:00:38.126711+0000 mon.a (mon.0) 2307 : cluster [DBG] osdmap e1112: 8 total, 8 up, 8 in 2024-04-07T17:00:40.162 INFO:tasks.workunit.client.0.smithi100.stdout:[----------] 4 tests from TestObjectMap (2309 ms total) 2024-04-07T17:00:40.162 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:00:40.162 INFO:tasks.workunit.client.0.smithi100.stdout:[----------] 2 tests from TestMirroringWatcher 2024-04-07T17:00:40.450 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:00:40 smithi100 bash[74942]: cluster 2024-04-07T17:00:39.128312+0000 mon.a (mon.0) 2308 : cluster [DBG] osdmap e1113: 8 total, 8 up, 8 in 2024-04-07T17:00:40.450 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:00:40 smithi100 bash[74942]: cluster 2024-04-07T17:00:39.739659+0000 mgr.y (mgr.25175) 1359 : cluster [DBG] pgmap v2047: 137 pgs: 12 creating+peering, 5 unknown, 120 active+clean; 4.6 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-04-07T17:00:40.451 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:00:40 smithi100 bash[76202]: cluster 2024-04-07T17:00:39.128312+0000 mon.a (mon.0) 2308 : cluster [DBG] osdmap e1113: 8 total, 8 up, 8 in 2024-04-07T17:00:40.451 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:00:40 smithi100 bash[76202]: cluster 2024-04-07T17:00:39.739659+0000 mgr.y (mgr.25175) 1359 : cluster [DBG] pgmap v2047: 137 pgs: 12 creating+peering, 5 unknown, 120 active+clean; 4.6 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-04-07T17:00:40.614 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:00:40 smithi144 bash[66654]: cluster 2024-04-07T17:00:39.128312+0000 mon.a (mon.0) 2308 : cluster [DBG] osdmap e1113: 8 total, 8 up, 8 in 2024-04-07T17:00:40.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:00:40 smithi144 bash[66654]: cluster 2024-04-07T17:00:39.739659+0000 mgr.y (mgr.25175) 1359 : cluster [DBG] pgmap v2047: 137 pgs: 12 creating+peering, 5 unknown, 120 active+clean; 4.6 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-04-07T17:00:41.450 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:00:41 smithi100 bash[74942]: cluster 2024-04-07T17:00:40.157009+0000 mon.a (mon.0) 2309 : cluster [DBG] osdmap e1114: 8 total, 8 up, 8 in 2024-04-07T17:00:41.450 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:00:41 smithi100 bash[76202]: cluster 2024-04-07T17:00:40.157009+0000 mon.a (mon.0) 2309 : cluster [DBG] osdmap e1114: 8 total, 8 up, 8 in 2024-04-07T17:00:41.614 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:00:41 smithi144 bash[66654]: cluster 2024-04-07T17:00:40.157009+0000 mon.a (mon.0) 2309 : cluster [DBG] osdmap e1114: 8 total, 8 up, 8 in 2024-04-07T17:00:42.164 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestMirroringWatcher.ModeUpdated 2024-04-07T17:00:42.450 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:00:42 smithi100 bash[74942]: cluster 2024-04-07T17:00:41.160705+0000 mon.a (mon.0) 2310 : cluster [DBG] osdmap e1115: 8 total, 8 up, 8 in 2024-04-07T17:00:42.451 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:00:42 smithi100 bash[74942]: audit 2024-04-07T17:00:41.166696+0000 mon.c (mon.2) 64 : audit [INF] from='client.? 172.21.15.100:0/3566081067' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:00:42.451 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:00:42 smithi100 bash[74942]: audit 2024-04-07T17:00:41.178223+0000 mon.a (mon.0) 2311 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:00:42.451 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:00:42 smithi100 bash[74942]: cluster 2024-04-07T17:00:41.740473+0000 mgr.y (mgr.25175) 1360 : cluster [DBG] pgmap v2050: 137 pgs: 32 unknown, 105 active+clean; 4.6 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-04-07T17:00:42.451 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:00:42 smithi100 bash[74942]: audit 2024-04-07T17:00:42.153139+0000 mon.a (mon.0) 2312 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T17:00:42.451 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:00:42 smithi100 bash[74942]: cluster 2024-04-07T17:00:42.163174+0000 mon.a (mon.0) 2313 : cluster [DBG] osdmap e1116: 8 total, 8 up, 8 in 2024-04-07T17:00:42.451 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:00:42 smithi100 bash[76202]: cluster 2024-04-07T17:00:41.160705+0000 mon.a (mon.0) 2310 : cluster [DBG] osdmap e1115: 8 total, 8 up, 8 in 2024-04-07T17:00:42.451 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:00:42 smithi100 bash[76202]: audit 2024-04-07T17:00:41.166696+0000 mon.c (mon.2) 64 : audit [INF] from='client.? 172.21.15.100:0/3566081067' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:00:42.451 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:00:42 smithi100 bash[76202]: audit 2024-04-07T17:00:41.178223+0000 mon.a (mon.0) 2311 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:00:42.451 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:00:42 smithi100 bash[76202]: cluster 2024-04-07T17:00:41.740473+0000 mgr.y (mgr.25175) 1360 : cluster [DBG] pgmap v2050: 137 pgs: 32 unknown, 105 active+clean; 4.6 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-04-07T17:00:42.451 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:00:42 smithi100 bash[76202]: audit 2024-04-07T17:00:42.153139+0000 mon.a (mon.0) 2312 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T17:00:42.451 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:00:42 smithi100 bash[76202]: cluster 2024-04-07T17:00:42.163174+0000 mon.a (mon.0) 2313 : cluster [DBG] osdmap e1116: 8 total, 8 up, 8 in 2024-04-07T17:00:42.614 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:00:42 smithi144 bash[95233]: ts=2024-04-07T17:00:42.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:00:42.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:00:42 smithi144 bash[66654]: cluster 2024-04-07T17:00:41.160705+0000 mon.a (mon.0) 2310 : cluster [DBG] osdmap e1115: 8 total, 8 up, 8 in 2024-04-07T17:00:42.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:00:42 smithi144 bash[66654]: audit 2024-04-07T17:00:41.166696+0000 mon.c (mon.2) 64 : audit [INF] from='client.? 172.21.15.100:0/3566081067' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:00:42.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:00:42 smithi144 bash[66654]: audit 2024-04-07T17:00:41.178223+0000 mon.a (mon.0) 2311 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:00:42.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:00:42 smithi144 bash[66654]: cluster 2024-04-07T17:00:41.740473+0000 mgr.y (mgr.25175) 1360 : cluster [DBG] pgmap v2050: 137 pgs: 32 unknown, 105 active+clean; 4.6 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-04-07T17:00:42.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:00:42 smithi144 bash[66654]: audit 2024-04-07T17:00:42.153139+0000 mon.a (mon.0) 2312 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T17:00:42.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:00:42 smithi144 bash[66654]: cluster 2024-04-07T17:00:42.163174+0000 mon.a (mon.0) 2313 : cluster [DBG] osdmap e1116: 8 total, 8 up, 8 in 2024-04-07T17:00:43.450 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:00:43 smithi100 bash[74942]: cluster 2024-04-07T17:00:42.173024+0000 mon.a (mon.0) 2314 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T17:00:43.450 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:00:43 smithi100 bash[76202]: cluster 2024-04-07T17:00:42.173024+0000 mon.a (mon.0) 2314 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T17:00:43.614 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:00:43 smithi144 bash[66654]: cluster 2024-04-07T17:00:42.173024+0000 mon.a (mon.0) 2314 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T17:00:44.289 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestMirroringWatcher.ModeUpdated (2125 ms) 2024-04-07T17:00:44.289 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestMirroringWatcher.ImageStatusUpdated 2024-04-07T17:00:44.345 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestMirroringWatcher.ImageStatusUpdated (55 ms) 2024-04-07T17:00:44.450 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:00:44 smithi100 bash[74942]: cluster 2024-04-07T17:00:43.187645+0000 mon.a (mon.0) 2315 : cluster [DBG] osdmap e1117: 8 total, 8 up, 8 in 2024-04-07T17:00:44.450 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:00:44 smithi100 bash[74942]: cluster 2024-04-07T17:00:43.741139+0000 mgr.y (mgr.25175) 1361 : cluster [DBG] pgmap v2053: 137 pgs: 14 creating+peering, 13 unknown, 110 active+clean; 4.6 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-04-07T17:00:44.450 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:00:44 smithi100 bash[76202]: cluster 2024-04-07T17:00:43.187645+0000 mon.a (mon.0) 2315 : cluster [DBG] osdmap e1117: 8 total, 8 up, 8 in 2024-04-07T17:00:44.451 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:00:44 smithi100 bash[76202]: cluster 2024-04-07T17:00:43.741139+0000 mgr.y (mgr.25175) 1361 : cluster [DBG] pgmap v2053: 137 pgs: 14 creating+peering, 13 unknown, 110 active+clean; 4.6 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-04-07T17:00:44.614 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:00:44 smithi144 bash[66654]: cluster 2024-04-07T17:00:43.187645+0000 mon.a (mon.0) 2315 : cluster [DBG] osdmap e1117: 8 total, 8 up, 8 in 2024-04-07T17:00:44.614 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:00:44 smithi144 bash[66654]: cluster 2024-04-07T17:00:43.741139+0000 mgr.y (mgr.25175) 1361 : cluster [DBG] pgmap v2053: 137 pgs: 14 creating+peering, 13 unknown, 110 active+clean; 4.6 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-04-07T17:00:45.220 INFO:tasks.workunit.client.0.smithi100.stdout:[----------] 2 tests from TestMirroringWatcher (2182 ms total) 2024-04-07T17:00:45.221 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:00:45.221 INFO:tasks.workunit.client.0.smithi100.stdout:[----------] 57 tests from TestMigration 2024-04-07T17:00:45.450 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:00:45 smithi100 bash[74942]: cluster 2024-04-07T17:00:44.193920+0000 mon.a (mon.0) 2316 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T17:00:45.450 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:00:45 smithi100 bash[74942]: cluster 2024-04-07T17:00:44.193988+0000 mon.a (mon.0) 2317 : cluster [INF] Cluster is now healthy 2024-04-07T17:00:45.450 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:00:45 smithi100 bash[74942]: cluster 2024-04-07T17:00:44.216957+0000 mon.a (mon.0) 2318 : cluster [DBG] osdmap e1118: 8 total, 8 up, 8 in 2024-04-07T17:00:45.451 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:00:45 smithi100 bash[76202]: cluster 2024-04-07T17:00:44.193920+0000 mon.a (mon.0) 2316 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T17:00:45.451 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:00:45 smithi100 bash[76202]: cluster 2024-04-07T17:00:44.193988+0000 mon.a (mon.0) 2317 : cluster [INF] Cluster is now healthy 2024-04-07T17:00:45.451 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:00:45 smithi100 bash[76202]: cluster 2024-04-07T17:00:44.216957+0000 mon.a (mon.0) 2318 : cluster [DBG] osdmap e1118: 8 total, 8 up, 8 in 2024-04-07T17:00:45.614 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:00:45 smithi144 bash[66654]: cluster 2024-04-07T17:00:44.193920+0000 mon.a (mon.0) 2316 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T17:00:45.614 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:00:45 smithi144 bash[66654]: cluster 2024-04-07T17:00:44.193988+0000 mon.a (mon.0) 2317 : cluster [INF] Cluster is now healthy 2024-04-07T17:00:45.614 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:00:45 smithi144 bash[66654]: cluster 2024-04-07T17:00:44.216957+0000 mon.a (mon.0) 2318 : cluster [DBG] osdmap e1118: 8 total, 8 up, 8 in 2024-04-07T17:00:46.614 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:00:46 smithi144 bash[66654]: cluster 2024-04-07T17:00:45.217165+0000 mon.a (mon.0) 2319 : cluster [DBG] osdmap e1119: 8 total, 8 up, 8 in 2024-04-07T17:00:46.614 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:00:46 smithi144 bash[66654]: cluster 2024-04-07T17:00:45.741813+0000 mgr.y (mgr.25175) 1362 : cluster [DBG] pgmap v2056: 105 pgs: 105 active+clean; 4.6 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-04-07T17:00:46.614 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:00:46 smithi144 bash[66654]: audit 2024-04-07T17:00:46.182321+0000 mon.a (mon.0) 2320 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:00:46.700 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:00:46 smithi100 bash[74942]: cluster 2024-04-07T17:00:45.217165+0000 mon.a (mon.0) 2319 : cluster [DBG] osdmap e1119: 8 total, 8 up, 8 in 2024-04-07T17:00:46.700 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:00:46 smithi100 bash[74942]: cluster 2024-04-07T17:00:45.741813+0000 mgr.y (mgr.25175) 1362 : cluster [DBG] pgmap v2056: 105 pgs: 105 active+clean; 4.6 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-04-07T17:00:46.700 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:00:46 smithi100 bash[74942]: audit 2024-04-07T17:00:46.182321+0000 mon.a (mon.0) 2320 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:00:46.700 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:00:46 smithi100 bash[76202]: cluster 2024-04-07T17:00:45.217165+0000 mon.a (mon.0) 2319 : cluster [DBG] osdmap e1119: 8 total, 8 up, 8 in 2024-04-07T17:00:46.701 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:00:46 smithi100 bash[76202]: cluster 2024-04-07T17:00:45.741813+0000 mgr.y (mgr.25175) 1362 : cluster [DBG] pgmap v2056: 105 pgs: 105 active+clean; 4.6 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-04-07T17:00:46.701 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:00:46 smithi100 bash[76202]: audit 2024-04-07T17:00:46.182321+0000 mon.a (mon.0) 2320 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:00:47.614 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:00:47 smithi144 bash[66654]: cluster 2024-04-07T17:00:46.233557+0000 mon.a (mon.0) 2321 : cluster [DBG] osdmap e1120: 8 total, 8 up, 8 in 2024-04-07T17:00:47.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:00:47 smithi144 bash[66654]: audit 2024-04-07T17:00:46.238865+0000 mon.a (mon.0) 2322 : audit [INF] from='client.? 172.21.15.100:0/3416171856' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:00:47.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:00:47 smithi144 bash[66654]: audit 2024-04-07T17:00:46.796874+0000 mon.a (mon.0) 2323 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:00:47.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:00:47 smithi144 bash[66654]: audit 2024-04-07T17:00:46.921693+0000 mon.a (mon.0) 2324 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:00:47.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:00:47 smithi144 bash[66654]: audit 2024-04-07T17:00:46.923578+0000 mon.a (mon.0) 2325 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:00:47.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:00:47 smithi144 bash[66654]: audit 2024-04-07T17:00:46.933617+0000 mon.a (mon.0) 2326 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:00:47.701 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:00:47 smithi100 bash[74942]: cluster 2024-04-07T17:00:46.233557+0000 mon.a (mon.0) 2321 : cluster [DBG] osdmap e1120: 8 total, 8 up, 8 in 2024-04-07T17:00:47.701 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:00:47 smithi100 bash[74942]: audit 2024-04-07T17:00:46.238865+0000 mon.a (mon.0) 2322 : audit [INF] from='client.? 172.21.15.100:0/3416171856' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:00:47.701 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:00:47 smithi100 bash[74942]: audit 2024-04-07T17:00:46.796874+0000 mon.a (mon.0) 2323 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:00:47.701 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:00:47 smithi100 bash[74942]: audit 2024-04-07T17:00:46.921693+0000 mon.a (mon.0) 2324 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:00:47.701 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:00:47 smithi100 bash[74942]: audit 2024-04-07T17:00:46.923578+0000 mon.a (mon.0) 2325 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:00:47.702 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:00:47 smithi100 bash[74942]: audit 2024-04-07T17:00:46.933617+0000 mon.a (mon.0) 2326 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:00:47.702 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:00:47 smithi100 bash[76202]: cluster 2024-04-07T17:00:46.233557+0000 mon.a (mon.0) 2321 : cluster [DBG] osdmap e1120: 8 total, 8 up, 8 in 2024-04-07T17:00:47.702 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:00:47 smithi100 bash[76202]: audit 2024-04-07T17:00:46.238865+0000 mon.a (mon.0) 2322 : audit [INF] from='client.? 172.21.15.100:0/3416171856' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:00:47.702 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:00:47 smithi100 bash[76202]: audit 2024-04-07T17:00:46.796874+0000 mon.a (mon.0) 2323 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:00:47.703 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:00:47 smithi100 bash[76202]: audit 2024-04-07T17:00:46.921693+0000 mon.a (mon.0) 2324 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:00:47.703 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:00:47 smithi100 bash[76202]: audit 2024-04-07T17:00:46.923578+0000 mon.a (mon.0) 2325 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:00:47.703 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:00:47 smithi100 bash[76202]: audit 2024-04-07T17:00:46.933617+0000 mon.a (mon.0) 2326 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:00:48.244 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestMigration.Empty 2024-04-07T17:00:48.614 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:00:48 smithi144 bash[66654]: audit 2024-04-07T17:00:47.224963+0000 mon.a (mon.0) 2327 : audit [INF] from='client.? 172.21.15.100:0/3416171856' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T17:00:48.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:00:48 smithi144 bash[66654]: cluster 2024-04-07T17:00:47.233025+0000 mon.a (mon.0) 2328 : cluster [DBG] osdmap e1121: 8 total, 8 up, 8 in 2024-04-07T17:00:48.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:00:48 smithi144 bash[66654]: cluster 2024-04-07T17:00:47.742440+0000 mgr.y (mgr.25175) 1363 : cluster [DBG] pgmap v2059: 137 pgs: 11 creating+peering, 21 unknown, 105 active+clean; 4.6 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-04-07T17:00:48.700 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:00:48 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:00:48] "GET /metrics HTTP/1.1" 200 36421 "" "Prometheus/2.43.0" 2024-04-07T17:00:48.700 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:00:48 smithi100 bash[74942]: audit 2024-04-07T17:00:47.224963+0000 mon.a (mon.0) 2327 : audit [INF] from='client.? 172.21.15.100:0/3416171856' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T17:00:48.701 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:00:48 smithi100 bash[74942]: cluster 2024-04-07T17:00:47.233025+0000 mon.a (mon.0) 2328 : cluster [DBG] osdmap e1121: 8 total, 8 up, 8 in 2024-04-07T17:00:48.701 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:00:48 smithi100 bash[74942]: cluster 2024-04-07T17:00:47.742440+0000 mgr.y (mgr.25175) 1363 : cluster [DBG] pgmap v2059: 137 pgs: 11 creating+peering, 21 unknown, 105 active+clean; 4.6 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-04-07T17:00:48.701 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:00:48 smithi100 bash[76202]: audit 2024-04-07T17:00:47.224963+0000 mon.a (mon.0) 2327 : audit [INF] from='client.? 172.21.15.100:0/3416171856' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T17:00:48.701 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:00:48 smithi100 bash[76202]: cluster 2024-04-07T17:00:47.233025+0000 mon.a (mon.0) 2328 : cluster [DBG] osdmap e1121: 8 total, 8 up, 8 in 2024-04-07T17:00:48.701 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:00:48 smithi100 bash[76202]: cluster 2024-04-07T17:00:47.742440+0000 mgr.y (mgr.25175) 1363 : cluster [DBG] pgmap v2059: 137 pgs: 11 creating+peering, 21 unknown, 105 active+clean; 4.6 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-04-07T17:00:49.614 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:00:49 smithi144 bash[66654]: cluster 2024-04-07T17:00:48.242511+0000 mon.a (mon.0) 2329 : cluster [DBG] osdmap e1122: 8 total, 8 up, 8 in 2024-04-07T17:00:49.614 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:00:49 smithi144 bash[66654]: cluster 2024-04-07T17:00:49.241314+0000 mon.a (mon.0) 2330 : cluster [DBG] osdmap e1123: 8 total, 8 up, 8 in 2024-04-07T17:00:49.700 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:00:49 smithi100 bash[74942]: cluster 2024-04-07T17:00:48.242511+0000 mon.a (mon.0) 2329 : cluster [DBG] osdmap e1122: 8 total, 8 up, 8 in 2024-04-07T17:00:49.700 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:00:49 smithi100 bash[74942]: cluster 2024-04-07T17:00:49.241314+0000 mon.a (mon.0) 2330 : cluster [DBG] osdmap e1123: 8 total, 8 up, 8 in 2024-04-07T17:00:49.701 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:00:49 smithi100 bash[76202]: cluster 2024-04-07T17:00:48.242511+0000 mon.a (mon.0) 2329 : cluster [DBG] osdmap e1122: 8 total, 8 up, 8 in 2024-04-07T17:00:49.701 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:00:49 smithi100 bash[76202]: cluster 2024-04-07T17:00:49.241314+0000 mon.a (mon.0) 2330 : cluster [DBG] osdmap e1123: 8 total, 8 up, 8 in 2024-04-07T17:00:50.345 INFO:tasks.workunit.client.0.smithi100.stdout:resize: 83886080 2024-04-07T17:00:50.367 INFO:tasks.workunit.client.0.smithi100.stdout:migration_prepare 2024-04-07T17:00:50.614 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:00:50 smithi144 bash[66654]: cluster 2024-04-07T17:00:49.743361+0000 mgr.y (mgr.25175) 1364 : cluster [DBG] pgmap v2062: 169 pgs: 18 creating+peering, 28 unknown, 123 active+clean; 4.6 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-04-07T17:00:50.614 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:00:50 smithi144 bash[66654]: cluster 2024-04-07T17:00:50.243713+0000 mon.a (mon.0) 2331 : cluster [DBG] osdmap e1124: 8 total, 8 up, 8 in 2024-04-07T17:00:50.700 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:00:50 smithi100 bash[74942]: cluster 2024-04-07T17:00:49.743361+0000 mgr.y (mgr.25175) 1364 : cluster [DBG] pgmap v2062: 169 pgs: 18 creating+peering, 28 unknown, 123 active+clean; 4.6 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-04-07T17:00:50.700 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:00:50 smithi100 bash[74942]: cluster 2024-04-07T17:00:50.243713+0000 mon.a (mon.0) 2331 : cluster [DBG] osdmap e1124: 8 total, 8 up, 8 in 2024-04-07T17:00:50.700 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:00:50 smithi100 bash[76202]: cluster 2024-04-07T17:00:49.743361+0000 mgr.y (mgr.25175) 1364 : cluster [DBG] pgmap v2062: 169 pgs: 18 creating+peering, 28 unknown, 123 active+clean; 4.6 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-04-07T17:00:50.700 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:00:50 smithi100 bash[76202]: cluster 2024-04-07T17:00:50.243713+0000 mon.a (mon.0) 2331 : cluster [DBG] osdmap e1124: 8 total, 8 up, 8 in 2024-04-07T17:00:50.701 INFO:tasks.workunit.client.0.smithi100.stdout:migration_execute 2024-04-07T17:00:50.912 INFO:tasks.workunit.client.0.smithi100.stdout:migration_commit 2024-04-07T17:00:51.133 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestMigration.Empty (2889 ms) 2024-04-07T17:00:51.133 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestMigration.OtherName 2024-04-07T17:00:51.217 INFO:tasks.workunit.client.0.smithi100.stdout:resize: 83886080 2024-04-07T17:00:51.245 INFO:tasks.workunit.client.0.smithi100.stdout:migration_prepare 2024-04-07T17:00:51.584 INFO:tasks.workunit.client.0.smithi100.stdout:migration_execute 2024-04-07T17:00:51.614 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:00:51 smithi144 bash[66654]: cluster 2024-04-07T17:00:50.266140+0000 mon.a (mon.0) 2332 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T17:00:51.700 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:00:51 smithi100 bash[74942]: cluster 2024-04-07T17:00:50.266140+0000 mon.a (mon.0) 2332 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T17:00:51.700 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:00:51 smithi100 bash[76202]: cluster 2024-04-07T17:00:50.266140+0000 mon.a (mon.0) 2332 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T17:00:51.771 INFO:tasks.workunit.client.0.smithi100.stdout:migration_commit 2024-04-07T17:00:51.986 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestMigration.OtherName (854 ms) 2024-04-07T17:00:51.987 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestMigration.OtherPool 2024-04-07T17:00:52.079 INFO:tasks.workunit.client.0.smithi100.stdout:resize: 83886080 2024-04-07T17:00:52.113 INFO:tasks.workunit.client.0.smithi100.stdout:migration_prepare 2024-04-07T17:00:52.614 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:00:52 smithi144 bash[66654]: cluster 2024-04-07T17:00:51.744168+0000 mgr.y (mgr.25175) 1365 : cluster [DBG] pgmap v2064: 169 pgs: 18 creating+peering, 21 unknown, 130 active+clean; 4.6 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-04-07T17:00:52.614 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:00:52 smithi144 bash[95233]: ts=2024-04-07T17:00:52.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:00:52.700 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:00:52 smithi100 bash[74942]: cluster 2024-04-07T17:00:51.744168+0000 mgr.y (mgr.25175) 1365 : cluster [DBG] pgmap v2064: 169 pgs: 18 creating+peering, 21 unknown, 130 active+clean; 4.6 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-04-07T17:00:52.700 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:00:52 smithi100 bash[76202]: cluster 2024-04-07T17:00:51.744168+0000 mgr.y (mgr.25175) 1365 : cluster [DBG] pgmap v2064: 169 pgs: 18 creating+peering, 21 unknown, 130 active+clean; 4.6 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-04-07T17:00:53.613 INFO:tasks.workunit.client.0.smithi100.stdout:migration_execute 2024-04-07T17:00:53.614 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:00:53 smithi144 bash[66654]: cluster 2024-04-07T17:00:52.300972+0000 mon.a (mon.0) 2333 : cluster [DBG] osdmap e1125: 8 total, 8 up, 8 in 2024-04-07T17:00:53.700 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:00:53 smithi100 bash[74942]: cluster 2024-04-07T17:00:52.300972+0000 mon.a (mon.0) 2333 : cluster [DBG] osdmap e1125: 8 total, 8 up, 8 in 2024-04-07T17:00:53.700 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:00:53 smithi100 bash[76202]: cluster 2024-04-07T17:00:52.300972+0000 mon.a (mon.0) 2333 : cluster [DBG] osdmap e1125: 8 total, 8 up, 8 in 2024-04-07T17:00:53.808 INFO:tasks.workunit.client.0.smithi100.stdout:migration_commit 2024-04-07T17:00:54.019 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestMigration.OtherPool (2033 ms) 2024-04-07T17:00:54.019 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestMigration.OtherNamespace 2024-04-07T17:00:54.103 INFO:tasks.workunit.client.0.smithi100.stdout:resize: 83886080 2024-04-07T17:00:54.139 INFO:tasks.workunit.client.0.smithi100.stdout:migration_prepare 2024-04-07T17:00:54.510 INFO:tasks.workunit.client.0.smithi100.stdout:migration_execute 2024-04-07T17:00:54.614 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:00:54 smithi144 bash[66654]: cluster 2024-04-07T17:00:53.300903+0000 mon.a (mon.0) 2334 : cluster [DBG] osdmap e1126: 8 total, 8 up, 8 in 2024-04-07T17:00:54.614 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:00:54 smithi144 bash[66654]: cluster 2024-04-07T17:00:53.745120+0000 mgr.y (mgr.25175) 1366 : cluster [DBG] pgmap v2067: 169 pgs: 7 creating+peering, 162 active+clean; 4.6 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 223 KiB/s rd, 11 KiB/s wr, 279 op/s 2024-04-07T17:00:54.700 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:00:54 smithi100 bash[74942]: cluster 2024-04-07T17:00:53.300903+0000 mon.a (mon.0) 2334 : cluster [DBG] osdmap e1126: 8 total, 8 up, 8 in 2024-04-07T17:00:54.700 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:00:54 smithi100 bash[74942]: cluster 2024-04-07T17:00:53.745120+0000 mgr.y (mgr.25175) 1366 : cluster [DBG] pgmap v2067: 169 pgs: 7 creating+peering, 162 active+clean; 4.6 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 223 KiB/s rd, 11 KiB/s wr, 279 op/s 2024-04-07T17:00:54.700 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:00:54 smithi100 bash[76202]: cluster 2024-04-07T17:00:53.300903+0000 mon.a (mon.0) 2334 : cluster [DBG] osdmap e1126: 8 total, 8 up, 8 in 2024-04-07T17:00:54.701 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:00:54 smithi100 bash[76202]: cluster 2024-04-07T17:00:53.745120+0000 mgr.y (mgr.25175) 1366 : cluster [DBG] pgmap v2067: 169 pgs: 7 creating+peering, 162 active+clean; 4.6 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 223 KiB/s rd, 11 KiB/s wr, 279 op/s 2024-04-07T17:00:54.707 INFO:tasks.workunit.client.0.smithi100.stdout:migration_commit 2024-04-07T17:00:54.914 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestMigration.OtherNamespace (895 ms) 2024-04-07T17:00:54.914 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestMigration.DataPool 2024-04-07T17:00:54.997 INFO:tasks.workunit.client.0.smithi100.stdout:resize: 83886080 2024-04-07T17:00:55.028 INFO:tasks.workunit.client.0.smithi100.stdout:migration_prepare 2024-04-07T17:00:55.385 INFO:tasks.workunit.client.0.smithi100.stdout:migration_execute 2024-04-07T17:00:55.586 INFO:tasks.workunit.client.0.smithi100.stdout:migration_commit 2024-04-07T17:00:55.807 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestMigration.DataPool (893 ms) 2024-04-07T17:00:55.808 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestMigration.AbortAfterPrepare 2024-04-07T17:00:55.898 INFO:tasks.workunit.client.0.smithi100.stdout:resize: 83886080 2024-04-07T17:00:55.926 INFO:tasks.workunit.client.0.smithi100.stdout:migration_prepare 2024-04-07T17:00:56.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:00:55 smithi144 bash[66654]: cluster 2024-04-07T17:00:55.746756+0000 mgr.y (mgr.25175) 1367 : cluster [DBG] pgmap v2068: 169 pgs: 169 active+clean; 4.6 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 276 KiB/s rd, 15 KiB/s wr, 347 op/s 2024-04-07T17:00:56.200 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:00:55 smithi100 bash[74942]: cluster 2024-04-07T17:00:55.746756+0000 mgr.y (mgr.25175) 1367 : cluster [DBG] pgmap v2068: 169 pgs: 169 active+clean; 4.6 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 276 KiB/s rd, 15 KiB/s wr, 347 op/s 2024-04-07T17:00:56.200 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:00:55 smithi100 bash[76202]: cluster 2024-04-07T17:00:55.746756+0000 mgr.y (mgr.25175) 1367 : cluster [DBG] pgmap v2068: 169 pgs: 169 active+clean; 4.6 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 276 KiB/s rd, 15 KiB/s wr, 347 op/s 2024-04-07T17:00:56.308 INFO:tasks.workunit.client.0.smithi100.stdout:migration_abort 2024-04-07T17:00:56.567 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestMigration.AbortAfterPrepare (759 ms) 2024-04-07T17:00:56.567 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestMigration.AbortAfterFailedPrepare 2024-04-07T17:00:56.650 INFO:tasks.workunit.client.0.smithi100.stdout:resize: 83886080 2024-04-07T17:00:56.681 INFO:tasks.workunit.client.0.smithi100.stdout:migration_prepare 2024-04-07T17:00:56.934 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestMigration.AbortAfterFailedPrepare (368 ms) 2024-04-07T17:00:56.935 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestMigration.AbortAfterExecute 2024-04-07T17:00:57.016 INFO:tasks.workunit.client.0.smithi100.stdout:resize: 83886080 2024-04-07T17:00:57.044 INFO:tasks.workunit.client.0.smithi100.stdout:migration_prepare 2024-04-07T17:00:57.420 INFO:tasks.workunit.client.0.smithi100.stdout:migration_execute 2024-04-07T17:00:57.628 INFO:tasks.workunit.client.0.smithi100.stdout:migration_abort 2024-04-07T17:00:57.839 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestMigration.AbortAfterExecute (905 ms) 2024-04-07T17:00:57.839 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestMigration.OtherPoolAbortAfterExecute 2024-04-07T17:00:57.921 INFO:tasks.workunit.client.0.smithi100.stdout:resize: 83886080 2024-04-07T17:00:57.950 INFO:tasks.workunit.client.0.smithi100.stdout:migration_prepare 2024-04-07T17:00:58.297 INFO:tasks.workunit.client.0.smithi100.stdout:migration_execute 2024-04-07T17:00:58.480 INFO:tasks.workunit.client.0.smithi100.stdout:migration_abort 2024-04-07T17:00:58.700 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:00:58 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:00:58] "GET /metrics HTTP/1.1" 200 36421 "" "Prometheus/2.43.0" 2024-04-07T17:00:58.701 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:00:58 smithi100 bash[74942]: cluster 2024-04-07T17:00:57.514060+0000 mon.a (mon.0) 2335 : cluster [DBG] osdmap e1127: 8 total, 8 up, 8 in 2024-04-07T17:00:58.701 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:00:58 smithi100 bash[74942]: cluster 2024-04-07T17:00:57.747537+0000 mgr.y (mgr.25175) 1368 : cluster [DBG] pgmap v2070: 169 pgs: 169 active+clean; 4.6 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 409 KiB/s rd, 24 KiB/s wr, 518 op/s 2024-04-07T17:00:58.701 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:00:58 smithi100 bash[76202]: cluster 2024-04-07T17:00:57.514060+0000 mon.a (mon.0) 2335 : cluster [DBG] osdmap e1127: 8 total, 8 up, 8 in 2024-04-07T17:00:58.701 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:00:58 smithi100 bash[76202]: cluster 2024-04-07T17:00:57.747537+0000 mgr.y (mgr.25175) 1368 : cluster [DBG] pgmap v2070: 169 pgs: 169 active+clean; 4.6 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 409 KiB/s rd, 24 KiB/s wr, 518 op/s 2024-04-07T17:00:58.737 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestMigration.OtherPoolAbortAfterExecute (898 ms) 2024-04-07T17:00:58.738 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestMigration.OtherNamespaceAbortAfterExecute 2024-04-07T17:00:58.822 INFO:tasks.workunit.client.0.smithi100.stdout:resize: 83886080 2024-04-07T17:00:58.856 INFO:tasks.workunit.client.0.smithi100.stdout:migration_prepare 2024-04-07T17:00:58.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:00:58 smithi144 bash[66654]: cluster 2024-04-07T17:00:57.514060+0000 mon.a (mon.0) 2335 : cluster [DBG] osdmap e1127: 8 total, 8 up, 8 in 2024-04-07T17:00:58.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:00:58 smithi144 bash[66654]: cluster 2024-04-07T17:00:57.747537+0000 mgr.y (mgr.25175) 1368 : cluster [DBG] pgmap v2070: 169 pgs: 169 active+clean; 4.6 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 409 KiB/s rd, 24 KiB/s wr, 518 op/s 2024-04-07T17:00:59.205 INFO:tasks.workunit.client.0.smithi100.stdout:migration_execute 2024-04-07T17:00:59.384 INFO:tasks.workunit.client.0.smithi100.stdout:migration_abort 2024-04-07T17:00:59.657 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestMigration.OtherNamespaceAbortAfterExecute (919 ms) 2024-04-07T17:00:59.657 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestMigration.MirroringSamePool 2024-04-07T17:00:59.744 INFO:tasks.workunit.client.0.smithi100.stdout:resize: 83886080 2024-04-07T17:00:59.774 INFO:tasks.workunit.client.0.smithi100.stdout:./src/test/librbd/test_Migration.cc:635: Skipped 2024-04-07T17:00:59.774 INFO:tasks.workunit.client.0.smithi100.stdout:Skipping due to unmet REQUIRE 2024-04-07T17:00:59.793 INFO:tasks.workunit.client.0.smithi100.stdout:[ SKIPPED ] TestMigration.MirroringSamePool (136 ms) 2024-04-07T17:00:59.793 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestMigration.MirroringAbort 2024-04-07T17:00:59.878 INFO:tasks.workunit.client.0.smithi100.stdout:resize: 83886080 2024-04-07T17:00:59.908 INFO:tasks.workunit.client.0.smithi100.stdout:./src/test/librbd/test_Migration.cc:653: Skipped 2024-04-07T17:00:59.908 INFO:tasks.workunit.client.0.smithi100.stdout:Skipping due to unmet REQUIRE 2024-04-07T17:00:59.926 INFO:tasks.workunit.client.0.smithi100.stdout:[ SKIPPED ] TestMigration.MirroringAbort (134 ms) 2024-04-07T17:00:59.927 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestMigration.MirroringOtherPoolDisabled 2024-04-07T17:01:00.012 INFO:tasks.workunit.client.0.smithi100.stdout:resize: 83886080 2024-04-07T17:01:00.036 INFO:tasks.workunit.client.0.smithi100.stdout:./src/test/librbd/test_Migration.cc:676: Skipped 2024-04-07T17:01:00.036 INFO:tasks.workunit.client.0.smithi100.stdout:Skipping due to unmet REQUIRE 2024-04-07T17:01:00.054 INFO:tasks.workunit.client.0.smithi100.stdout:[ SKIPPED ] TestMigration.MirroringOtherPoolDisabled (128 ms) 2024-04-07T17:01:00.054 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestMigration.MirroringOtherPoolEnabled 2024-04-07T17:01:00.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:00:59 smithi144 bash[66654]: cluster 2024-04-07T17:00:59.748796+0000 mgr.y (mgr.25175) 1369 : cluster [DBG] pgmap v2071: 169 pgs: 169 active+clean; 4.6 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 459 KiB/s rd, 26 KiB/s wr, 589 op/s 2024-04-07T17:01:00.133 INFO:tasks.workunit.client.0.smithi100.stdout:resize: 83886080 2024-04-07T17:01:00.157 INFO:tasks.workunit.client.0.smithi100.stdout:./src/test/librbd/test_Migration.cc:694: Skipped 2024-04-07T17:01:00.157 INFO:tasks.workunit.client.0.smithi100.stdout:Skipping due to unmet REQUIRE 2024-04-07T17:01:00.172 INFO:tasks.workunit.client.0.smithi100.stdout:[ SKIPPED ] TestMigration.MirroringOtherPoolEnabled (118 ms) 2024-04-07T17:01:00.172 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestMigration.MirroringPool 2024-04-07T17:01:00.200 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:00:59 smithi100 bash[74942]: cluster 2024-04-07T17:00:59.748796+0000 mgr.y (mgr.25175) 1369 : cluster [DBG] pgmap v2071: 169 pgs: 169 active+clean; 4.6 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 459 KiB/s rd, 26 KiB/s wr, 589 op/s 2024-04-07T17:01:00.200 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:00:59 smithi100 bash[76202]: cluster 2024-04-07T17:00:59.748796+0000 mgr.y (mgr.25175) 1369 : cluster [DBG] pgmap v2071: 169 pgs: 169 active+clean; 4.6 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 459 KiB/s rd, 26 KiB/s wr, 589 op/s 2024-04-07T17:01:00.246 INFO:tasks.workunit.client.0.smithi100.stdout:resize: 83886080 2024-04-07T17:01:00.276 INFO:tasks.workunit.client.0.smithi100.stdout:./src/test/librbd/test_Migration.cc:714: Skipped 2024-04-07T17:01:00.276 INFO:tasks.workunit.client.0.smithi100.stdout:Skipping due to unmet REQUIRE 2024-04-07T17:01:00.293 INFO:tasks.workunit.client.0.smithi100.stdout:[ SKIPPED ] TestMigration.MirroringPool (121 ms) 2024-04-07T17:01:00.293 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestMigration.Group 2024-04-07T17:01:00.372 INFO:tasks.workunit.client.0.smithi100.stdout:resize: 83886080 2024-04-07T17:01:00.424 INFO:tasks.workunit.client.0.smithi100.stdout:migration_prepare 2024-04-07T17:01:00.813 INFO:tasks.workunit.client.0.smithi100.stdout:migration_execute 2024-04-07T17:01:01.008 INFO:tasks.workunit.client.0.smithi100.stdout:migration_commit 2024-04-07T17:01:01.248 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestMigration.Group (955 ms) 2024-04-07T17:01:01.248 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestMigration.GroupAbort 2024-04-07T17:01:01.332 INFO:tasks.workunit.client.0.smithi100.stdout:resize: 83886080 2024-04-07T17:01:01.387 INFO:tasks.workunit.client.0.smithi100.stdout:migration_prepare 2024-04-07T17:01:01.788 INFO:tasks.workunit.client.0.smithi100.stdout:migration_abort 2024-04-07T17:01:02.100 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestMigration.GroupAbort (852 ms) 2024-04-07T17:01:02.100 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestMigration.NoSnaps 2024-04-07T17:01:02.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:01:01 smithi144 bash[66654]: cluster 2024-04-07T17:01:01.749856+0000 mgr.y (mgr.25175) 1370 : cluster [DBG] pgmap v2072: 169 pgs: 169 active+clean; 4.6 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 452 KiB/s rd, 27 KiB/s wr, 576 op/s 2024-04-07T17:01:02.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:01:01 smithi144 bash[66654]: audit 2024-04-07T17:01:01.804194+0000 mon.a (mon.0) 2336 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:01:02.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:01:01 smithi144 bash[66654]: audit 2024-04-07T17:01:01.807111+0000 mon.a (mon.0) 2337 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:01:02.188 INFO:tasks.workunit.client.0.smithi100.stdout:resize: 83886080 2024-04-07T17:01:02.200 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:01:01 smithi100 bash[74942]: cluster 2024-04-07T17:01:01.749856+0000 mgr.y (mgr.25175) 1370 : cluster [DBG] pgmap v2072: 169 pgs: 169 active+clean; 4.6 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 452 KiB/s rd, 27 KiB/s wr, 576 op/s 2024-04-07T17:01:02.200 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:01:01 smithi100 bash[74942]: audit 2024-04-07T17:01:01.804194+0000 mon.a (mon.0) 2336 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:01:02.200 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:01:01 smithi100 bash[74942]: audit 2024-04-07T17:01:01.807111+0000 mon.a (mon.0) 2337 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:01:02.200 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:01:01 smithi100 bash[76202]: cluster 2024-04-07T17:01:01.749856+0000 mgr.y (mgr.25175) 1370 : cluster [DBG] pgmap v2072: 169 pgs: 169 active+clean; 4.6 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 452 KiB/s rd, 27 KiB/s wr, 576 op/s 2024-04-07T17:01:02.201 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:01:01 smithi100 bash[76202]: audit 2024-04-07T17:01:01.804194+0000 mon.a (mon.0) 2336 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:01:02.201 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:01:01 smithi100 bash[76202]: audit 2024-04-07T17:01:01.807111+0000 mon.a (mon.0) 2337 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:01:02.217 INFO:tasks.workunit.client.0.smithi100.stdout:write: 1 0~8388609 2024-04-07T17:01:02.331 INFO:tasks.workunit.client.0.smithi100.stdout:write: 1 16777218~8388609 2024-04-07T17:01:02.410 INFO:tasks.workunit.client.0.smithi100.stdout:migration_prepare 2024-04-07T17:01:02.614 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:01:02 smithi144 bash[95233]: ts=2024-04-07T17:01:02.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:01:02.891 INFO:tasks.workunit.client.0.smithi100.stdout:migration_execute 2024-04-07T17:01:03.129 INFO:tasks.workunit.client.0.smithi100.stdout:migration_commit 2024-04-07T17:01:03.402 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestMigration.NoSnaps (1302 ms) 2024-04-07T17:01:03.403 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestMigration.NoSnapsOtherPool 2024-04-07T17:01:03.488 INFO:tasks.workunit.client.0.smithi100.stdout:resize: 83886080 2024-04-07T17:01:03.519 INFO:tasks.workunit.client.0.smithi100.stdout:write: 1 0~8388609 2024-04-07T17:01:03.618 INFO:tasks.workunit.client.0.smithi100.stdout:write: 1 16777218~8388609 2024-04-07T17:01:03.686 INFO:tasks.workunit.client.0.smithi100.stdout:write: 1 0~8388609 2024-04-07T17:01:03.702 INFO:tasks.workunit.client.0.smithi100.stdout:write: 1 16777218~8388609 2024-04-07T17:01:03.767 INFO:tasks.workunit.client.0.smithi100.stdout:migration_prepare 2024-04-07T17:01:03.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:01:03 smithi144 bash[66654]: cluster 2024-04-07T17:01:02.517114+0000 mon.a (mon.0) 2338 : cluster [DBG] osdmap e1128: 8 total, 8 up, 8 in 2024-04-07T17:01:03.950 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:01:03 smithi100 bash[74942]: cluster 2024-04-07T17:01:02.517114+0000 mon.a (mon.0) 2338 : cluster [DBG] osdmap e1128: 8 total, 8 up, 8 in 2024-04-07T17:01:03.950 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:01:03 smithi100 bash[76202]: cluster 2024-04-07T17:01:02.517114+0000 mon.a (mon.0) 2338 : cluster [DBG] osdmap e1128: 8 total, 8 up, 8 in 2024-04-07T17:01:04.147 INFO:tasks.workunit.client.0.smithi100.stdout:migration_execute 2024-04-07T17:01:04.376 INFO:tasks.workunit.client.0.smithi100.stdout:migration_commit 2024-04-07T17:01:04.637 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestMigration.NoSnapsOtherPool (1234 ms) 2024-04-07T17:01:04.637 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestMigration.NoSnapsDataPool 2024-04-07T17:01:04.719 INFO:tasks.workunit.client.0.smithi100.stdout:resize: 83886080 2024-04-07T17:01:04.748 INFO:tasks.workunit.client.0.smithi100.stdout:write: 1 0~8388609 2024-04-07T17:01:04.858 INFO:tasks.workunit.client.0.smithi100.stdout:write: 1 16777218~8388609 2024-04-07T17:01:04.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:01:04 smithi144 bash[66654]: cluster 2024-04-07T17:01:03.750846+0000 mgr.y (mgr.25175) 1371 : cluster [DBG] pgmap v2074: 169 pgs: 169 active+clean; 13 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.7 MiB/s rd, 1.0 MiB/s wr, 901 op/s 2024-04-07T17:01:04.933 INFO:tasks.workunit.client.0.smithi100.stdout:migration_prepare 2024-04-07T17:01:04.950 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:01:04 smithi100 bash[74942]: cluster 2024-04-07T17:01:03.750846+0000 mgr.y (mgr.25175) 1371 : cluster [DBG] pgmap v2074: 169 pgs: 169 active+clean; 13 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.7 MiB/s rd, 1.0 MiB/s wr, 901 op/s 2024-04-07T17:01:04.950 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:01:04 smithi100 bash[76202]: cluster 2024-04-07T17:01:03.750846+0000 mgr.y (mgr.25175) 1371 : cluster [DBG] pgmap v2074: 169 pgs: 169 active+clean; 13 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.7 MiB/s rd, 1.0 MiB/s wr, 901 op/s 2024-04-07T17:01:05.389 INFO:tasks.workunit.client.0.smithi100.stdout:migration_execute 2024-04-07T17:01:05.660 INFO:tasks.workunit.client.0.smithi100.stdout:migration_commit 2024-04-07T17:01:05.940 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestMigration.NoSnapsDataPool (1304 ms) 2024-04-07T17:01:05.940 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestMigration.NoSnapsShrinkAfterPrepare 2024-04-07T17:01:06.024 INFO:tasks.workunit.client.0.smithi100.stdout:resize: 83886080 2024-04-07T17:01:06.055 INFO:tasks.workunit.client.0.smithi100.stdout:write: 1 0~8388609 2024-04-07T17:01:06.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:01:05 smithi144 bash[66654]: cluster 2024-04-07T17:01:05.752103+0000 mgr.y (mgr.25175) 1372 : cluster [DBG] pgmap v2075: 169 pgs: 169 active+clean; 57 MiB data, 3.1 GiB used, 712 GiB / 715 GiB avail; 8.5 MiB/s rd, 10 MiB/s wr, 927 op/s 2024-04-07T17:01:06.144 INFO:tasks.workunit.client.0.smithi100.stdout:write: 1 16777218~8388609 2024-04-07T17:01:06.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:01:05 smithi100 bash[74942]: cluster 2024-04-07T17:01:05.752103+0000 mgr.y (mgr.25175) 1372 : cluster [DBG] pgmap v2075: 169 pgs: 169 active+clean; 57 MiB data, 3.1 GiB used, 712 GiB / 715 GiB avail; 8.5 MiB/s rd, 10 MiB/s wr, 927 op/s 2024-04-07T17:01:06.201 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:01:05 smithi100 bash[76202]: cluster 2024-04-07T17:01:05.752103+0000 mgr.y (mgr.25175) 1372 : cluster [DBG] pgmap v2075: 169 pgs: 169 active+clean; 57 MiB data, 3.1 GiB used, 712 GiB / 715 GiB avail; 8.5 MiB/s rd, 10 MiB/s wr, 927 op/s 2024-04-07T17:01:06.222 INFO:tasks.workunit.client.0.smithi100.stdout:migration_prepare 2024-04-07T17:01:06.671 INFO:tasks.workunit.client.0.smithi100.stdout:resize: 41943040 2024-04-07T17:01:06.694 INFO:tasks.workunit.client.0.smithi100.stdout:migration_execute 2024-04-07T17:01:06.990 INFO:tasks.workunit.client.0.smithi100.stdout:migration_commit 2024-04-07T17:01:07.241 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestMigration.NoSnapsShrinkAfterPrepare (1301 ms) 2024-04-07T17:01:07.241 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestMigration.NoSnapsShrinkToZeroBeforePrepare 2024-04-07T17:01:07.327 INFO:tasks.workunit.client.0.smithi100.stdout:resize: 83886080 2024-04-07T17:01:07.358 INFO:tasks.workunit.client.0.smithi100.stdout:write: 1 0~8388609 2024-04-07T17:01:07.451 INFO:tasks.workunit.client.0.smithi100.stdout:write: 1 16777218~8388609 2024-04-07T17:01:07.535 INFO:tasks.workunit.client.0.smithi100.stdout:resize: 0 2024-04-07T17:01:07.573 INFO:tasks.workunit.client.0.smithi100.stdout:migration_prepare 2024-04-07T17:01:07.873 INFO:tasks.workunit.client.0.smithi100.stdout:migration_execute 2024-04-07T17:01:08.075 INFO:tasks.workunit.client.0.smithi100.stdout:migration_commit 2024-04-07T17:01:08.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:01:07 smithi144 bash[66654]: cluster 2024-04-07T17:01:07.753458+0000 mgr.y (mgr.25175) 1373 : cluster [DBG] pgmap v2076: 169 pgs: 169 active+clean; 125 MiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 17 MiB/s rd, 16 MiB/s wr, 880 op/s 2024-04-07T17:01:08.200 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:01:07 smithi100 bash[74942]: cluster 2024-04-07T17:01:07.753458+0000 mgr.y (mgr.25175) 1373 : cluster [DBG] pgmap v2076: 169 pgs: 169 active+clean; 125 MiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 17 MiB/s rd, 16 MiB/s wr, 880 op/s 2024-04-07T17:01:08.200 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:01:07 smithi100 bash[76202]: cluster 2024-04-07T17:01:07.753458+0000 mgr.y (mgr.25175) 1373 : cluster [DBG] pgmap v2076: 169 pgs: 169 active+clean; 125 MiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 17 MiB/s rd, 16 MiB/s wr, 880 op/s 2024-04-07T17:01:08.208 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestMigration.NoSnapsShrinkToZeroBeforePrepare (967 ms) 2024-04-07T17:01:08.208 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestMigration.NoSnapsShrinkToZeroAfterPrepare 2024-04-07T17:01:08.290 INFO:tasks.workunit.client.0.smithi100.stdout:resize: 83886080 2024-04-07T17:01:08.319 INFO:tasks.workunit.client.0.smithi100.stdout:write: 1 0~8388609 2024-04-07T17:01:08.408 INFO:tasks.workunit.client.0.smithi100.stdout:write: 1 16777218~8388609 2024-04-07T17:01:08.477 INFO:tasks.workunit.client.0.smithi100.stdout:migration_prepare 2024-04-07T17:01:08.700 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:01:08 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:01:08] "GET /metrics HTTP/1.1" 200 38870 "" "Prometheus/2.43.0" 2024-04-07T17:01:08.936 INFO:tasks.workunit.client.0.smithi100.stdout:resize: 0 2024-04-07T17:01:08.971 INFO:tasks.workunit.client.0.smithi100.stdout:migration_execute 2024-04-07T17:01:09.171 INFO:tasks.workunit.client.0.smithi100.stdout:migration_commit 2024-04-07T17:01:09.330 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestMigration.NoSnapsShrinkToZeroAfterPrepare (1122 ms) 2024-04-07T17:01:09.331 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestMigration.NoSnapsExpandAfterPrepare 2024-04-07T17:01:09.414 INFO:tasks.workunit.client.0.smithi100.stdout:resize: 83886080 2024-04-07T17:01:09.445 INFO:tasks.workunit.client.0.smithi100.stdout:write: 1 0~8388609 2024-04-07T17:01:09.539 INFO:tasks.workunit.client.0.smithi100.stdout:write: 1 16777218~8388609 2024-04-07T17:01:09.615 INFO:tasks.workunit.client.0.smithi100.stdout:migration_prepare 2024-04-07T17:01:10.072 INFO:tasks.workunit.client.0.smithi100.stdout:resize: 167772160 2024-04-07T17:01:10.092 INFO:tasks.workunit.client.0.smithi100.stdout:migration_execute 2024-04-07T17:01:10.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:01:09 smithi144 bash[66654]: cluster 2024-04-07T17:01:09.754770+0000 mgr.y (mgr.25175) 1374 : cluster [DBG] pgmap v2077: 169 pgs: 169 active+clean; 157 MiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 31 MiB/s rd, 24 MiB/s wr, 927 op/s 2024-04-07T17:01:10.200 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:01:09 smithi100 bash[74942]: cluster 2024-04-07T17:01:09.754770+0000 mgr.y (mgr.25175) 1374 : cluster [DBG] pgmap v2077: 169 pgs: 169 active+clean; 157 MiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 31 MiB/s rd, 24 MiB/s wr, 927 op/s 2024-04-07T17:01:10.201 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:01:09 smithi100 bash[76202]: cluster 2024-04-07T17:01:09.754770+0000 mgr.y (mgr.25175) 1374 : cluster [DBG] pgmap v2077: 169 pgs: 169 active+clean; 157 MiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 31 MiB/s rd, 24 MiB/s wr, 927 op/s 2024-04-07T17:01:10.346 INFO:tasks.workunit.client.0.smithi100.stdout:migration_commit 2024-04-07T17:01:10.666 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestMigration.NoSnapsExpandAfterPrepare (1336 ms) 2024-04-07T17:01:10.666 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestMigration.NoSnapsSnapAfterPrepare 2024-04-07T17:01:10.753 INFO:tasks.workunit.client.0.smithi100.stdout:resize: 83886080 2024-04-07T17:01:10.781 INFO:tasks.workunit.client.0.smithi100.stdout:write: 1 0~8388609 2024-04-07T17:01:10.879 INFO:tasks.workunit.client.0.smithi100.stdout:write: 1 16777218~8388609 2024-04-07T17:01:10.955 INFO:tasks.workunit.client.0.smithi100.stdout:migration_prepare 2024-04-07T17:01:11.419 INFO:tasks.workunit.client.0.smithi100.stdout:snap_create: after_prepare_snap 2024-04-07T17:01:12.526 INFO:tasks.workunit.client.0.smithi100.stdout:resize: 41943040 2024-04-07T17:01:12.581 INFO:tasks.workunit.client.0.smithi100.stdout:write: * 0~1000 2024-04-07T17:01:12.595 INFO:tasks.workunit.client.0.smithi100.stdout:migration_execute 2024-04-07T17:01:12.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:01:12 smithi144 bash[66654]: cluster 2024-04-07T17:01:11.491196+0000 mon.a (mon.0) 2339 : cluster [DBG] osdmap e1129: 8 total, 8 up, 8 in 2024-04-07T17:01:12.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:01:12 smithi144 bash[66654]: cluster 2024-04-07T17:01:11.755594+0000 mgr.y (mgr.25175) 1375 : cluster [DBG] pgmap v2079: 169 pgs: 169 active+clean; 157 MiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 35 MiB/s rd, 27 MiB/s wr, 672 op/s 2024-04-07T17:01:12.615 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:01:12 smithi144 bash[95233]: ts=2024-04-07T17:01:12.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:01:12.840 INFO:tasks.workunit.client.0.smithi100.stdout:migration_commit 2024-04-07T17:01:12.950 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:01:12 smithi100 bash[74942]: cluster 2024-04-07T17:01:11.491196+0000 mon.a (mon.0) 2339 : cluster [DBG] osdmap e1129: 8 total, 8 up, 8 in 2024-04-07T17:01:12.950 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:01:12 smithi100 bash[74942]: cluster 2024-04-07T17:01:11.755594+0000 mgr.y (mgr.25175) 1375 : cluster [DBG] pgmap v2079: 169 pgs: 169 active+clean; 157 MiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 35 MiB/s rd, 27 MiB/s wr, 672 op/s 2024-04-07T17:01:12.951 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:01:12 smithi100 bash[76202]: cluster 2024-04-07T17:01:11.491196+0000 mon.a (mon.0) 2339 : cluster [DBG] osdmap e1129: 8 total, 8 up, 8 in 2024-04-07T17:01:12.951 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:01:12 smithi100 bash[76202]: cluster 2024-04-07T17:01:11.755594+0000 mgr.y (mgr.25175) 1375 : cluster [DBG] pgmap v2079: 169 pgs: 169 active+clean; 157 MiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 35 MiB/s rd, 27 MiB/s wr, 672 op/s 2024-04-07T17:01:13.188 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestMigration.NoSnapsSnapAfterPrepare (2522 ms) 2024-04-07T17:01:13.189 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestMigration.Snaps 2024-04-07T17:01:13.270 INFO:tasks.workunit.client.0.smithi100.stdout:resize: 83886080 2024-04-07T17:01:13.301 INFO:tasks.workunit.client.0.smithi100.stdout:write: 1 0~8388609 2024-04-07T17:01:13.391 INFO:tasks.workunit.client.0.smithi100.stdout:snap_create: snap1 2024-04-07T17:01:13.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:01:13 smithi144 bash[66654]: cluster 2024-04-07T17:01:12.504403+0000 mon.a (mon.0) 2340 : cluster [DBG] osdmap e1130: 8 total, 8 up, 8 in 2024-04-07T17:01:13.950 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:01:13 smithi100 bash[74942]: cluster 2024-04-07T17:01:12.504403+0000 mon.a (mon.0) 2340 : cluster [DBG] osdmap e1130: 8 total, 8 up, 8 in 2024-04-07T17:01:13.951 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:01:13 smithi100 bash[76202]: cluster 2024-04-07T17:01:12.504403+0000 mon.a (mon.0) 2340 : cluster [DBG] osdmap e1130: 8 total, 8 up, 8 in 2024-04-07T17:01:14.547 INFO:tasks.workunit.client.0.smithi100.stdout:write: 1 8388609~8388609 2024-04-07T17:01:14.594 INFO:tasks.workunit.client.0.smithi100.stdout:write: X 0~1000 2024-04-07T17:01:14.594 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 1010~1000 2024-04-07T17:01:14.594 INFO:tasks.workunit.client.0.smithi100.stdout:snap_create: snap2 2024-04-07T17:01:14.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:01:14 smithi144 bash[66654]: cluster 2024-04-07T17:01:13.511360+0000 mon.a (mon.0) 2341 : cluster [DBG] osdmap e1131: 8 total, 8 up, 8 in 2024-04-07T17:01:14.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:01:14 smithi144 bash[66654]: cluster 2024-04-07T17:01:13.756339+0000 mgr.y (mgr.25175) 1376 : cluster [DBG] pgmap v2082: 169 pgs: 169 active+clean; 185 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 43 MiB/s rd, 29 MiB/s wr, 914 op/s 2024-04-07T17:01:14.950 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:01:14 smithi100 bash[74942]: cluster 2024-04-07T17:01:13.511360+0000 mon.a (mon.0) 2341 : cluster [DBG] osdmap e1131: 8 total, 8 up, 8 in 2024-04-07T17:01:14.951 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:01:14 smithi100 bash[74942]: cluster 2024-04-07T17:01:13.756339+0000 mgr.y (mgr.25175) 1376 : cluster [DBG] pgmap v2082: 169 pgs: 169 active+clean; 185 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 43 MiB/s rd, 29 MiB/s wr, 914 op/s 2024-04-07T17:01:14.951 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:01:14 smithi100 bash[76202]: cluster 2024-04-07T17:01:13.511360+0000 mon.a (mon.0) 2341 : cluster [DBG] osdmap e1131: 8 total, 8 up, 8 in 2024-04-07T17:01:14.951 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:01:14 smithi100 bash[76202]: cluster 2024-04-07T17:01:13.756339+0000 mgr.y (mgr.25175) 1376 : cluster [DBG] pgmap v2082: 169 pgs: 169 active+clean; 185 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 43 MiB/s rd, 29 MiB/s wr, 914 op/s 2024-04-07T17:01:15.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:01:15 smithi144 bash[66654]: cluster 2024-04-07T17:01:14.521509+0000 mon.a (mon.0) 2342 : cluster [DBG] osdmap e1132: 8 total, 8 up, 8 in 2024-04-07T17:01:15.950 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:01:15 smithi100 bash[74942]: cluster 2024-04-07T17:01:14.521509+0000 mon.a (mon.0) 2342 : cluster [DBG] osdmap e1132: 8 total, 8 up, 8 in 2024-04-07T17:01:15.951 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:01:15 smithi100 bash[76202]: cluster 2024-04-07T17:01:14.521509+0000 mon.a (mon.0) 2342 : cluster [DBG] osdmap e1132: 8 total, 8 up, 8 in 2024-04-07T17:01:16.566 INFO:tasks.workunit.client.0.smithi100.stdout:write: X 8388609~1000 2024-04-07T17:01:16.581 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 16777228~1000 2024-04-07T17:01:16.581 INFO:tasks.workunit.client.0.smithi100.stdout:resize: 167772160 2024-04-07T17:01:16.603 INFO:tasks.workunit.client.0.smithi100.stdout:write: 2 83886079~8388609 2024-04-07T17:01:16.661 INFO:tasks.workunit.client.0.smithi100.stdout:snap_create: snap3 2024-04-07T17:01:16.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:01:16 smithi144 bash[66654]: cluster 2024-04-07T17:01:15.523445+0000 mon.a (mon.0) 2343 : cluster [DBG] osdmap e1133: 8 total, 8 up, 8 in 2024-04-07T17:01:16.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:01:16 smithi144 bash[66654]: cluster 2024-04-07T17:01:15.757104+0000 mgr.y (mgr.25175) 1377 : cluster [DBG] pgmap v2085: 169 pgs: 169 active+clean; 205 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 46 MiB/s rd, 35 MiB/s wr, 871 op/s 2024-04-07T17:01:16.950 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:01:16 smithi100 bash[74942]: cluster 2024-04-07T17:01:15.523445+0000 mon.a (mon.0) 2343 : cluster [DBG] osdmap e1133: 8 total, 8 up, 8 in 2024-04-07T17:01:16.950 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:01:16 smithi100 bash[74942]: cluster 2024-04-07T17:01:15.757104+0000 mgr.y (mgr.25175) 1377 : cluster [DBG] pgmap v2085: 169 pgs: 169 active+clean; 205 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 46 MiB/s rd, 35 MiB/s wr, 871 op/s 2024-04-07T17:01:16.951 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:01:16 smithi100 bash[76202]: cluster 2024-04-07T17:01:15.523445+0000 mon.a (mon.0) 2343 : cluster [DBG] osdmap e1133: 8 total, 8 up, 8 in 2024-04-07T17:01:16.951 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:01:16 smithi100 bash[76202]: cluster 2024-04-07T17:01:15.757104+0000 mgr.y (mgr.25175) 1377 : cluster [DBG] pgmap v2085: 169 pgs: 169 active+clean; 205 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 46 MiB/s rd, 35 MiB/s wr, 871 op/s 2024-04-07T17:01:17.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:01:17 smithi144 bash[66654]: cluster 2024-04-07T17:01:16.549382+0000 mon.a (mon.0) 2344 : cluster [DBG] osdmap e1134: 8 total, 8 up, 8 in 2024-04-07T17:01:17.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:01:17 smithi144 bash[66654]: audit 2024-04-07T17:01:16.798280+0000 mon.a (mon.0) 2345 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:01:17.950 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:01:17 smithi100 bash[74942]: cluster 2024-04-07T17:01:16.549382+0000 mon.a (mon.0) 2344 : cluster [DBG] osdmap e1134: 8 total, 8 up, 8 in 2024-04-07T17:01:17.951 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:01:17 smithi100 bash[74942]: audit 2024-04-07T17:01:16.798280+0000 mon.a (mon.0) 2345 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:01:17.951 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:01:17 smithi100 bash[76202]: cluster 2024-04-07T17:01:16.549382+0000 mon.a (mon.0) 2344 : cluster [DBG] osdmap e1134: 8 total, 8 up, 8 in 2024-04-07T17:01:17.951 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:01:17 smithi100 bash[76202]: audit 2024-04-07T17:01:16.798280+0000 mon.a (mon.0) 2345 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:01:18.602 INFO:tasks.workunit.client.0.smithi100.stdout:resize: 83886080 2024-04-07T17:01:18.665 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 83886079~1 2024-04-07T17:01:18.695 INFO:tasks.workunit.client.0.smithi100.stdout:migration_prepare 2024-04-07T17:01:18.700 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:01:18 smithi100 bash[74942]: cluster 2024-04-07T17:01:17.559812+0000 mon.a (mon.0) 2346 : cluster [DBG] osdmap e1135: 8 total, 8 up, 8 in 2024-04-07T17:01:18.700 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:01:18 smithi100 bash[74942]: cluster 2024-04-07T17:01:17.757924+0000 mgr.y (mgr.25175) 1378 : cluster [DBG] pgmap v2088: 169 pgs: 169 active+clean; 221 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 32 MiB/s rd, 18 MiB/s wr, 415 op/s 2024-04-07T17:01:18.700 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:01:18 smithi100 bash[74942]: audit 2024-04-07T17:01:18.320373+0000 mon.a (mon.0) 2347 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "85.16", "id": [5, 0]}]: dispatch 2024-04-07T17:01:18.700 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:01:18 smithi100 bash[74942]: audit 2024-04-07T17:01:18.321095+0000 mon.a (mon.0) 2348 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "85.19", "id": [5, 0]}]: dispatch 2024-04-07T17:01:18.700 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:01:18 smithi100 bash[74942]: audit 2024-04-07T17:01:18.321548+0000 mon.a (mon.0) 2349 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "85.1f", "id": [5, 2]}]: dispatch 2024-04-07T17:01:18.701 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:01:18 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:01:18] "GET /metrics HTTP/1.1" 200 38921 "" "Prometheus/2.43.0" 2024-04-07T17:01:18.701 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:01:18 smithi100 bash[76202]: cluster 2024-04-07T17:01:17.559812+0000 mon.a (mon.0) 2346 : cluster [DBG] osdmap e1135: 8 total, 8 up, 8 in 2024-04-07T17:01:18.701 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:01:18 smithi100 bash[76202]: cluster 2024-04-07T17:01:17.757924+0000 mgr.y (mgr.25175) 1378 : cluster [DBG] pgmap v2088: 169 pgs: 169 active+clean; 221 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 32 MiB/s rd, 18 MiB/s wr, 415 op/s 2024-04-07T17:01:18.701 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:01:18 smithi100 bash[76202]: audit 2024-04-07T17:01:18.320373+0000 mon.a (mon.0) 2347 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "85.16", "id": [5, 0]}]: dispatch 2024-04-07T17:01:18.701 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:01:18 smithi100 bash[76202]: audit 2024-04-07T17:01:18.321095+0000 mon.a (mon.0) 2348 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "85.19", "id": [5, 0]}]: dispatch 2024-04-07T17:01:18.701 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:01:18 smithi100 bash[76202]: audit 2024-04-07T17:01:18.321548+0000 mon.a (mon.0) 2349 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "85.1f", "id": [5, 2]}]: dispatch 2024-04-07T17:01:18.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:01:18 smithi144 bash[66654]: cluster 2024-04-07T17:01:17.559812+0000 mon.a (mon.0) 2346 : cluster [DBG] osdmap e1135: 8 total, 8 up, 8 in 2024-04-07T17:01:18.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:01:18 smithi144 bash[66654]: cluster 2024-04-07T17:01:17.757924+0000 mgr.y (mgr.25175) 1378 : cluster [DBG] pgmap v2088: 169 pgs: 169 active+clean; 221 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 32 MiB/s rd, 18 MiB/s wr, 415 op/s 2024-04-07T17:01:18.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:01:18 smithi144 bash[66654]: audit 2024-04-07T17:01:18.320373+0000 mon.a (mon.0) 2347 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "85.16", "id": [5, 0]}]: dispatch 2024-04-07T17:01:18.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:01:18 smithi144 bash[66654]: audit 2024-04-07T17:01:18.321095+0000 mon.a (mon.0) 2348 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "85.19", "id": [5, 0]}]: dispatch 2024-04-07T17:01:18.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:01:18 smithi144 bash[66654]: audit 2024-04-07T17:01:18.321548+0000 mon.a (mon.0) 2349 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "85.1f", "id": [5, 2]}]: dispatch 2024-04-07T17:01:19.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:01:19 smithi144 bash[66654]: audit 2024-04-07T17:01:18.558963+0000 mon.a (mon.0) 2350 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "85.16", "id": [5, 0]}]': finished 2024-04-07T17:01:19.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:01:19 smithi144 bash[66654]: audit 2024-04-07T17:01:18.559183+0000 mon.a (mon.0) 2351 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "85.19", "id": [5, 0]}]': finished 2024-04-07T17:01:19.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:01:19 smithi144 bash[66654]: audit 2024-04-07T17:01:18.559416+0000 mon.a (mon.0) 2352 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "85.1f", "id": [5, 2]}]': finished 2024-04-07T17:01:19.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:01:19 smithi144 bash[66654]: cluster 2024-04-07T17:01:18.569182+0000 mon.a (mon.0) 2353 : cluster [DBG] osdmap e1136: 8 total, 8 up, 8 in 2024-04-07T17:01:19.950 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:01:19 smithi100 bash[74942]: audit 2024-04-07T17:01:18.558963+0000 mon.a (mon.0) 2350 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "85.16", "id": [5, 0]}]': finished 2024-04-07T17:01:19.950 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:01:19 smithi100 bash[74942]: audit 2024-04-07T17:01:18.559183+0000 mon.a (mon.0) 2351 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "85.19", "id": [5, 0]}]': finished 2024-04-07T17:01:19.951 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:01:19 smithi100 bash[74942]: audit 2024-04-07T17:01:18.559416+0000 mon.a (mon.0) 2352 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "85.1f", "id": [5, 2]}]': finished 2024-04-07T17:01:19.951 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:01:19 smithi100 bash[74942]: cluster 2024-04-07T17:01:18.569182+0000 mon.a (mon.0) 2353 : cluster [DBG] osdmap e1136: 8 total, 8 up, 8 in 2024-04-07T17:01:19.951 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:01:19 smithi100 bash[76202]: audit 2024-04-07T17:01:18.558963+0000 mon.a (mon.0) 2350 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "85.16", "id": [5, 0]}]': finished 2024-04-07T17:01:19.951 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:01:19 smithi100 bash[76202]: audit 2024-04-07T17:01:18.559183+0000 mon.a (mon.0) 2351 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "85.19", "id": [5, 0]}]': finished 2024-04-07T17:01:19.951 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:01:19 smithi100 bash[76202]: audit 2024-04-07T17:01:18.559416+0000 mon.a (mon.0) 2352 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "85.1f", "id": [5, 2]}]': finished 2024-04-07T17:01:19.951 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:01:19 smithi100 bash[76202]: cluster 2024-04-07T17:01:18.569182+0000 mon.a (mon.0) 2353 : cluster [DBG] osdmap e1136: 8 total, 8 up, 8 in 2024-04-07T17:01:20.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:01:20 smithi144 bash[66654]: cluster 2024-04-07T17:01:19.571539+0000 mon.a (mon.0) 2354 : cluster [DBG] osdmap e1137: 8 total, 8 up, 8 in 2024-04-07T17:01:20.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:01:20 smithi144 bash[66654]: cluster 2024-04-07T17:01:19.758750+0000 mgr.y (mgr.25175) 1379 : cluster [DBG] pgmap v2091: 169 pgs: 1 peering, 168 active+clean; 253 MiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 11 MiB/s rd, 11 MiB/s wr, 277 op/s 2024-04-07T17:01:20.950 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:01:20 smithi100 bash[74942]: cluster 2024-04-07T17:01:19.571539+0000 mon.a (mon.0) 2354 : cluster [DBG] osdmap e1137: 8 total, 8 up, 8 in 2024-04-07T17:01:20.951 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:01:20 smithi100 bash[74942]: cluster 2024-04-07T17:01:19.758750+0000 mgr.y (mgr.25175) 1379 : cluster [DBG] pgmap v2091: 169 pgs: 1 peering, 168 active+clean; 253 MiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 11 MiB/s rd, 11 MiB/s wr, 277 op/s 2024-04-07T17:01:20.951 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:01:20 smithi100 bash[76202]: cluster 2024-04-07T17:01:19.571539+0000 mon.a (mon.0) 2354 : cluster [DBG] osdmap e1137: 8 total, 8 up, 8 in 2024-04-07T17:01:20.951 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:01:20 smithi100 bash[76202]: cluster 2024-04-07T17:01:19.758750+0000 mgr.y (mgr.25175) 1379 : cluster [DBG] pgmap v2091: 169 pgs: 1 peering, 168 active+clean; 253 MiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 11 MiB/s rd, 11 MiB/s wr, 277 op/s 2024-04-07T17:01:21.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:01:21 smithi144 bash[66654]: cluster 2024-04-07T17:01:20.597820+0000 mon.a (mon.0) 2355 : cluster [DBG] osdmap e1138: 8 total, 8 up, 8 in 2024-04-07T17:01:21.950 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:01:21 smithi100 bash[76202]: cluster 2024-04-07T17:01:20.597820+0000 mon.a (mon.0) 2355 : cluster [DBG] osdmap e1138: 8 total, 8 up, 8 in 2024-04-07T17:01:21.951 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:01:21 smithi100 bash[74942]: cluster 2024-04-07T17:01:20.597820+0000 mon.a (mon.0) 2355 : cluster [DBG] osdmap e1138: 8 total, 8 up, 8 in 2024-04-07T17:01:22.565 INFO:tasks.workunit.client.0.smithi100.stdout:migration_execute 2024-04-07T17:01:22.614 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:01:22 smithi144 bash[95233]: ts=2024-04-07T17:01:22.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:01:22.850 INFO:tasks.workunit.client.0.smithi100.stdout:migration_commit 2024-04-07T17:01:22.950 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:01:22 smithi100 bash[74942]: cluster 2024-04-07T17:01:21.598417+0000 mon.a (mon.0) 2356 : cluster [DBG] osdmap e1139: 8 total, 8 up, 8 in 2024-04-07T17:01:22.950 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:01:22 smithi100 bash[74942]: cluster 2024-04-07T17:01:21.759725+0000 mgr.y (mgr.25175) 1380 : cluster [DBG] pgmap v2094: 169 pgs: 1 peering, 168 active+clean; 253 MiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 104 KiB/s rd, 7.0 MiB/s wr, 150 op/s 2024-04-07T17:01:22.951 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:01:22 smithi100 bash[76202]: cluster 2024-04-07T17:01:21.598417+0000 mon.a (mon.0) 2356 : cluster [DBG] osdmap e1139: 8 total, 8 up, 8 in 2024-04-07T17:01:22.951 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:01:22 smithi100 bash[76202]: cluster 2024-04-07T17:01:21.759725+0000 mgr.y (mgr.25175) 1380 : cluster [DBG] pgmap v2094: 169 pgs: 1 peering, 168 active+clean; 253 MiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 104 KiB/s rd, 7.0 MiB/s wr, 150 op/s 2024-04-07T17:01:23.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:01:22 smithi144 bash[66654]: cluster 2024-04-07T17:01:21.598417+0000 mon.a (mon.0) 2356 : cluster [DBG] osdmap e1139: 8 total, 8 up, 8 in 2024-04-07T17:01:23.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:01:22 smithi144 bash[66654]: cluster 2024-04-07T17:01:21.759725+0000 mgr.y (mgr.25175) 1380 : cluster [DBG] pgmap v2094: 169 pgs: 1 peering, 168 active+clean; 253 MiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 104 KiB/s rd, 7.0 MiB/s wr, 150 op/s 2024-04-07T17:01:24.951 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:01:24 smithi100 bash[74942]: cluster 2024-04-07T17:01:23.627626+0000 mon.a (mon.0) 2357 : cluster [DBG] osdmap e1140: 8 total, 8 up, 8 in 2024-04-07T17:01:24.951 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:01:24 smithi100 bash[74942]: cluster 2024-04-07T17:01:23.760520+0000 mgr.y (mgr.25175) 1381 : cluster [DBG] pgmap v2096: 169 pgs: 1 peering, 168 active+clean; 261 MiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 12 MiB/s rd, 1.9 MiB/s wr, 254 op/s; 976 KiB/s, 0 objects/s recovering 2024-04-07T17:01:24.951 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:01:24 smithi100 bash[76202]: cluster 2024-04-07T17:01:23.627626+0000 mon.a (mon.0) 2357 : cluster [DBG] osdmap e1140: 8 total, 8 up, 8 in 2024-04-07T17:01:24.951 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:01:24 smithi100 bash[76202]: cluster 2024-04-07T17:01:23.760520+0000 mgr.y (mgr.25175) 1381 : cluster [DBG] pgmap v2096: 169 pgs: 1 peering, 168 active+clean; 261 MiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 12 MiB/s rd, 1.9 MiB/s wr, 254 op/s; 976 KiB/s, 0 objects/s recovering 2024-04-07T17:01:25.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:01:24 smithi144 bash[66654]: cluster 2024-04-07T17:01:23.627626+0000 mon.a (mon.0) 2357 : cluster [DBG] osdmap e1140: 8 total, 8 up, 8 in 2024-04-07T17:01:25.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:01:24 smithi144 bash[66654]: cluster 2024-04-07T17:01:23.760520+0000 mgr.y (mgr.25175) 1381 : cluster [DBG] pgmap v2096: 169 pgs: 1 peering, 168 active+clean; 261 MiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 12 MiB/s rd, 1.9 MiB/s wr, 254 op/s; 976 KiB/s, 0 objects/s recovering 2024-04-07T17:01:25.950 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:01:25 smithi100 bash[74942]: cluster 2024-04-07T17:01:24.638137+0000 mon.a (mon.0) 2358 : cluster [DBG] osdmap e1141: 8 total, 8 up, 8 in 2024-04-07T17:01:25.951 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:01:25 smithi100 bash[76202]: cluster 2024-04-07T17:01:24.638137+0000 mon.a (mon.0) 2358 : cluster [DBG] osdmap e1141: 8 total, 8 up, 8 in 2024-04-07T17:01:26.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:01:25 smithi144 bash[66654]: cluster 2024-04-07T17:01:24.638137+0000 mon.a (mon.0) 2358 : cluster [DBG] osdmap e1141: 8 total, 8 up, 8 in 2024-04-07T17:01:26.255 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestMigration.Snaps (13066 ms) 2024-04-07T17:01:26.255 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestMigration.SnapsOtherPool 2024-04-07T17:01:26.346 INFO:tasks.workunit.client.0.smithi100.stdout:resize: 83886080 2024-04-07T17:01:26.376 INFO:tasks.workunit.client.0.smithi100.stdout:write: 1 0~8388609 2024-04-07T17:01:26.499 INFO:tasks.workunit.client.0.smithi100.stdout:snap_create: snap1 2024-04-07T17:01:26.950 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:01:26 smithi100 bash[74942]: cluster 2024-04-07T17:01:25.646666+0000 mon.a (mon.0) 2359 : cluster [DBG] osdmap e1142: 8 total, 8 up, 8 in 2024-04-07T17:01:26.951 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:01:26 smithi100 bash[74942]: cluster 2024-04-07T17:01:25.761362+0000 mgr.y (mgr.25175) 1382 : cluster [DBG] pgmap v2099: 169 pgs: 169 active+clean; 261 MiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 34 MiB/s rd, 2.9 MiB/s wr, 282 op/s; 983 KiB/s, 0 objects/s recovering 2024-04-07T17:01:26.951 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:01:26 smithi100 bash[76202]: cluster 2024-04-07T17:01:25.646666+0000 mon.a (mon.0) 2359 : cluster [DBG] osdmap e1142: 8 total, 8 up, 8 in 2024-04-07T17:01:26.951 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:01:26 smithi100 bash[76202]: cluster 2024-04-07T17:01:25.761362+0000 mgr.y (mgr.25175) 1382 : cluster [DBG] pgmap v2099: 169 pgs: 169 active+clean; 261 MiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 34 MiB/s rd, 2.9 MiB/s wr, 282 op/s; 983 KiB/s, 0 objects/s recovering 2024-04-07T17:01:27.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:01:26 smithi144 bash[66654]: cluster 2024-04-07T17:01:25.646666+0000 mon.a (mon.0) 2359 : cluster [DBG] osdmap e1142: 8 total, 8 up, 8 in 2024-04-07T17:01:27.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:01:26 smithi144 bash[66654]: cluster 2024-04-07T17:01:25.761362+0000 mgr.y (mgr.25175) 1382 : cluster [DBG] pgmap v2099: 169 pgs: 169 active+clean; 261 MiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 34 MiB/s rd, 2.9 MiB/s wr, 282 op/s; 983 KiB/s, 0 objects/s recovering 2024-04-07T17:01:27.697 INFO:tasks.workunit.client.0.smithi100.stdout:write: 1 8388609~8388609 2024-04-07T17:01:27.749 INFO:tasks.workunit.client.0.smithi100.stdout:write: X 0~1000 2024-04-07T17:01:27.750 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 1010~1000 2024-04-07T17:01:27.750 INFO:tasks.workunit.client.0.smithi100.stdout:snap_create: snap2 2024-04-07T17:01:27.950 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:01:27 smithi100 bash[74942]: cluster 2024-04-07T17:01:26.659954+0000 mon.a (mon.0) 2360 : cluster [DBG] osdmap e1143: 8 total, 8 up, 8 in 2024-04-07T17:01:27.951 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:01:27 smithi100 bash[76202]: cluster 2024-04-07T17:01:26.659954+0000 mon.a (mon.0) 2360 : cluster [DBG] osdmap e1143: 8 total, 8 up, 8 in 2024-04-07T17:01:28.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:01:27 smithi144 bash[66654]: cluster 2024-04-07T17:01:26.659954+0000 mon.a (mon.0) 2360 : cluster [DBG] osdmap e1143: 8 total, 8 up, 8 in 2024-04-07T17:01:28.681 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:01:28 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:01:28] "GET /metrics HTTP/1.1" 200 38921 "" "Prometheus/2.43.0" 2024-04-07T17:01:28.950 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:01:28 smithi100 bash[74942]: cluster 2024-04-07T17:01:27.670093+0000 mon.a (mon.0) 2361 : cluster [DBG] osdmap e1144: 8 total, 8 up, 8 in 2024-04-07T17:01:28.950 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:01:28 smithi100 bash[74942]: cluster 2024-04-07T17:01:27.762333+0000 mgr.y (mgr.25175) 1383 : cluster [DBG] pgmap v2102: 169 pgs: 169 active+clean; 281 MiB data, 3.7 GiB used, 711 GiB / 715 GiB avail; 34 MiB/s rd, 6.0 MiB/s wr, 222 op/s; 0 B/s, 0 objects/s recovering 2024-04-07T17:01:28.951 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:01:28 smithi100 bash[76202]: cluster 2024-04-07T17:01:27.670093+0000 mon.a (mon.0) 2361 : cluster [DBG] osdmap e1144: 8 total, 8 up, 8 in 2024-04-07T17:01:28.951 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:01:28 smithi100 bash[76202]: cluster 2024-04-07T17:01:27.762333+0000 mgr.y (mgr.25175) 1383 : cluster [DBG] pgmap v2102: 169 pgs: 169 active+clean; 281 MiB data, 3.7 GiB used, 711 GiB / 715 GiB avail; 34 MiB/s rd, 6.0 MiB/s wr, 222 op/s; 0 B/s, 0 objects/s recovering 2024-04-07T17:01:29.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:01:28 smithi144 bash[66654]: cluster 2024-04-07T17:01:27.670093+0000 mon.a (mon.0) 2361 : cluster [DBG] osdmap e1144: 8 total, 8 up, 8 in 2024-04-07T17:01:29.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:01:28 smithi144 bash[66654]: cluster 2024-04-07T17:01:27.762333+0000 mgr.y (mgr.25175) 1383 : cluster [DBG] pgmap v2102: 169 pgs: 169 active+clean; 281 MiB data, 3.7 GiB used, 711 GiB / 715 GiB avail; 34 MiB/s rd, 6.0 MiB/s wr, 222 op/s; 0 B/s, 0 objects/s recovering 2024-04-07T17:01:29.729 INFO:tasks.workunit.client.0.smithi100.stdout:write: X 8388609~1000 2024-04-07T17:01:29.752 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 16777228~1000 2024-04-07T17:01:29.752 INFO:tasks.workunit.client.0.smithi100.stdout:resize: 167772160 2024-04-07T17:01:29.776 INFO:tasks.workunit.client.0.smithi100.stdout:write: 2 83886079~8388609 2024-04-07T17:01:29.824 INFO:tasks.workunit.client.0.smithi100.stdout:snap_create: snap3 2024-04-07T17:01:29.950 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:01:29 smithi100 bash[74942]: cluster 2024-04-07T17:01:28.675874+0000 mon.a (mon.0) 2362 : cluster [DBG] osdmap e1145: 8 total, 8 up, 8 in 2024-04-07T17:01:29.951 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:01:29 smithi100 bash[76202]: cluster 2024-04-07T17:01:28.675874+0000 mon.a (mon.0) 2362 : cluster [DBG] osdmap e1145: 8 total, 8 up, 8 in 2024-04-07T17:01:30.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:01:29 smithi144 bash[66654]: cluster 2024-04-07T17:01:28.675874+0000 mon.a (mon.0) 2362 : cluster [DBG] osdmap e1145: 8 total, 8 up, 8 in 2024-04-07T17:01:31.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:01:30 smithi144 bash[66654]: cluster 2024-04-07T17:01:29.692564+0000 mon.a (mon.0) 2363 : cluster [DBG] osdmap e1146: 8 total, 8 up, 8 in 2024-04-07T17:01:31.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:01:30 smithi144 bash[66654]: cluster 2024-04-07T17:01:29.763356+0000 mgr.y (mgr.25175) 1384 : cluster [DBG] pgmap v2105: 169 pgs: 169 active+clean; 277 MiB data, 3.7 GiB used, 711 GiB / 715 GiB avail; 32 MiB/s rd, 9.0 MiB/s wr, 276 op/s 2024-04-07T17:01:31.200 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:01:30 smithi100 bash[74942]: cluster 2024-04-07T17:01:29.692564+0000 mon.a (mon.0) 2363 : cluster [DBG] osdmap e1146: 8 total, 8 up, 8 in 2024-04-07T17:01:31.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:01:30 smithi100 bash[74942]: cluster 2024-04-07T17:01:29.763356+0000 mgr.y (mgr.25175) 1384 : cluster [DBG] pgmap v2105: 169 pgs: 169 active+clean; 277 MiB data, 3.7 GiB used, 711 GiB / 715 GiB avail; 32 MiB/s rd, 9.0 MiB/s wr, 276 op/s 2024-04-07T17:01:31.201 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:01:30 smithi100 bash[76202]: cluster 2024-04-07T17:01:29.692564+0000 mon.a (mon.0) 2363 : cluster [DBG] osdmap e1146: 8 total, 8 up, 8 in 2024-04-07T17:01:31.201 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:01:30 smithi100 bash[76202]: cluster 2024-04-07T17:01:29.763356+0000 mgr.y (mgr.25175) 1384 : cluster [DBG] pgmap v2105: 169 pgs: 169 active+clean; 277 MiB data, 3.7 GiB used, 711 GiB / 715 GiB avail; 32 MiB/s rd, 9.0 MiB/s wr, 276 op/s 2024-04-07T17:01:31.753 INFO:tasks.workunit.client.0.smithi100.stdout:resize: 83886080 2024-04-07T17:01:31.815 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 83886079~1 2024-04-07T17:01:31.839 INFO:tasks.workunit.client.0.smithi100.stdout:write: 1 0~8388609 2024-04-07T17:01:31.870 INFO:tasks.workunit.client.0.smithi100.stdout:write: 1 16777218~8388609 2024-04-07T17:01:31.960 INFO:tasks.workunit.client.0.smithi100.stdout:migration_prepare 2024-04-07T17:01:32.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:01:31 smithi144 bash[66654]: cluster 2024-04-07T17:01:30.716635+0000 mon.a (mon.0) 2364 : cluster [DBG] osdmap e1147: 8 total, 8 up, 8 in 2024-04-07T17:01:32.200 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:01:31 smithi100 bash[74942]: cluster 2024-04-07T17:01:30.716635+0000 mon.a (mon.0) 2364 : cluster [DBG] osdmap e1147: 8 total, 8 up, 8 in 2024-04-07T17:01:32.201 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:01:31 smithi100 bash[76202]: cluster 2024-04-07T17:01:30.716635+0000 mon.a (mon.0) 2364 : cluster [DBG] osdmap e1147: 8 total, 8 up, 8 in 2024-04-07T17:01:32.615 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:01:32 smithi144 bash[95233]: ts=2024-04-07T17:01:32.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:01:33.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:01:32 smithi144 bash[66654]: cluster 2024-04-07T17:01:31.717255+0000 mon.a (mon.0) 2365 : cluster [DBG] osdmap e1148: 8 total, 8 up, 8 in 2024-04-07T17:01:33.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:01:32 smithi144 bash[66654]: cluster 2024-04-07T17:01:31.764186+0000 mgr.y (mgr.25175) 1385 : cluster [DBG] pgmap v2108: 169 pgs: 169 active+clean; 277 MiB data, 3.7 GiB used, 711 GiB / 715 GiB avail; 24 MiB/s rd, 4.0 MiB/s wr, 86 op/s 2024-04-07T17:01:33.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:01:32 smithi144 bash[66654]: audit 2024-04-07T17:01:31.798509+0000 mon.a (mon.0) 2366 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:01:33.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:01:32 smithi144 bash[66654]: cluster 2024-04-07T17:01:32.530363+0000 mon.a (mon.0) 2367 : cluster [DBG] osdmap e1149: 8 total, 8 up, 8 in 2024-04-07T17:01:33.200 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:01:32 smithi100 bash[74942]: cluster 2024-04-07T17:01:31.717255+0000 mon.a (mon.0) 2365 : cluster [DBG] osdmap e1148: 8 total, 8 up, 8 in 2024-04-07T17:01:33.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:01:32 smithi100 bash[74942]: cluster 2024-04-07T17:01:31.764186+0000 mgr.y (mgr.25175) 1385 : cluster [DBG] pgmap v2108: 169 pgs: 169 active+clean; 277 MiB data, 3.7 GiB used, 711 GiB / 715 GiB avail; 24 MiB/s rd, 4.0 MiB/s wr, 86 op/s 2024-04-07T17:01:33.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:01:32 smithi100 bash[74942]: audit 2024-04-07T17:01:31.798509+0000 mon.a (mon.0) 2366 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:01:33.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:01:32 smithi100 bash[74942]: cluster 2024-04-07T17:01:32.530363+0000 mon.a (mon.0) 2367 : cluster [DBG] osdmap e1149: 8 total, 8 up, 8 in 2024-04-07T17:01:33.201 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:01:32 smithi100 bash[76202]: cluster 2024-04-07T17:01:31.717255+0000 mon.a (mon.0) 2365 : cluster [DBG] osdmap e1148: 8 total, 8 up, 8 in 2024-04-07T17:01:33.201 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:01:32 smithi100 bash[76202]: cluster 2024-04-07T17:01:31.764186+0000 mgr.y (mgr.25175) 1385 : cluster [DBG] pgmap v2108: 169 pgs: 169 active+clean; 277 MiB data, 3.7 GiB used, 711 GiB / 715 GiB avail; 24 MiB/s rd, 4.0 MiB/s wr, 86 op/s 2024-04-07T17:01:33.201 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:01:32 smithi100 bash[76202]: audit 2024-04-07T17:01:31.798509+0000 mon.a (mon.0) 2366 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:01:33.201 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:01:32 smithi100 bash[76202]: cluster 2024-04-07T17:01:32.530363+0000 mon.a (mon.0) 2367 : cluster [DBG] osdmap e1149: 8 total, 8 up, 8 in 2024-04-07T17:01:34.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:01:34 smithi144 bash[66654]: cluster 2024-04-07T17:01:33.527683+0000 mon.a (mon.0) 2368 : cluster [DBG] osdmap e1150: 8 total, 8 up, 8 in 2024-04-07T17:01:34.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:01:34 smithi144 bash[66654]: cluster 2024-04-07T17:01:33.765049+0000 mgr.y (mgr.25175) 1386 : cluster [DBG] pgmap v2111: 169 pgs: 169 active+clean; 309 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 3.1 MiB/s rd, 7.0 MiB/s wr, 97 op/s 2024-04-07T17:01:34.950 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:01:34 smithi100 bash[76202]: cluster 2024-04-07T17:01:33.527683+0000 mon.a (mon.0) 2368 : cluster [DBG] osdmap e1150: 8 total, 8 up, 8 in 2024-04-07T17:01:34.951 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:01:34 smithi100 bash[76202]: cluster 2024-04-07T17:01:33.765049+0000 mgr.y (mgr.25175) 1386 : cluster [DBG] pgmap v2111: 169 pgs: 169 active+clean; 309 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 3.1 MiB/s rd, 7.0 MiB/s wr, 97 op/s 2024-04-07T17:01:34.951 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:01:34 smithi100 bash[74942]: cluster 2024-04-07T17:01:33.527683+0000 mon.a (mon.0) 2368 : cluster [DBG] osdmap e1150: 8 total, 8 up, 8 in 2024-04-07T17:01:34.951 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:01:34 smithi100 bash[74942]: cluster 2024-04-07T17:01:33.765049+0000 mgr.y (mgr.25175) 1386 : cluster [DBG] pgmap v2111: 169 pgs: 169 active+clean; 309 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 3.1 MiB/s rd, 7.0 MiB/s wr, 97 op/s 2024-04-07T17:01:35.516 INFO:tasks.workunit.client.0.smithi100.stdout:migration_execute 2024-04-07T17:01:35.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:01:35 smithi144 bash[66654]: cluster 2024-04-07T17:01:34.538428+0000 mon.a (mon.0) 2369 : cluster [DBG] osdmap e1151: 8 total, 8 up, 8 in 2024-04-07T17:01:35.886 INFO:tasks.workunit.client.0.smithi100.stdout:migration_commit 2024-04-07T17:01:35.950 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:01:35 smithi100 bash[74942]: cluster 2024-04-07T17:01:34.538428+0000 mon.a (mon.0) 2369 : cluster [DBG] osdmap e1151: 8 total, 8 up, 8 in 2024-04-07T17:01:35.951 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:01:35 smithi100 bash[76202]: cluster 2024-04-07T17:01:34.538428+0000 mon.a (mon.0) 2369 : cluster [DBG] osdmap e1151: 8 total, 8 up, 8 in 2024-04-07T17:01:36.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:01:36 smithi144 bash[66654]: cluster 2024-04-07T17:01:35.765952+0000 mgr.y (mgr.25175) 1387 : cluster [DBG] pgmap v2113: 169 pgs: 169 active+clean; 341 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 134 KiB/s rd, 14 MiB/s wr, 195 op/s 2024-04-07T17:01:36.950 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:01:36 smithi100 bash[74942]: cluster 2024-04-07T17:01:35.765952+0000 mgr.y (mgr.25175) 1387 : cluster [DBG] pgmap v2113: 169 pgs: 169 active+clean; 341 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 134 KiB/s rd, 14 MiB/s wr, 195 op/s 2024-04-07T17:01:36.951 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:01:36 smithi100 bash[76202]: cluster 2024-04-07T17:01:35.765952+0000 mgr.y (mgr.25175) 1387 : cluster [DBG] pgmap v2113: 169 pgs: 169 active+clean; 341 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 134 KiB/s rd, 14 MiB/s wr, 195 op/s 2024-04-07T17:01:37.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:01:37 smithi144 bash[66654]: cluster 2024-04-07T17:01:36.549017+0000 mon.a (mon.0) 2370 : cluster [DBG] osdmap e1152: 8 total, 8 up, 8 in 2024-04-07T17:01:37.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:01:37 smithi144 bash[66654]: cluster 2024-04-07T17:01:37.527119+0000 mon.a (mon.0) 2371 : cluster [DBG] osdmap e1153: 8 total, 8 up, 8 in 2024-04-07T17:01:37.951 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:01:37 smithi100 bash[74942]: cluster 2024-04-07T17:01:36.549017+0000 mon.a (mon.0) 2370 : cluster [DBG] osdmap e1152: 8 total, 8 up, 8 in 2024-04-07T17:01:37.951 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:01:37 smithi100 bash[74942]: cluster 2024-04-07T17:01:37.527119+0000 mon.a (mon.0) 2371 : cluster [DBG] osdmap e1153: 8 total, 8 up, 8 in 2024-04-07T17:01:37.951 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:01:37 smithi100 bash[76202]: cluster 2024-04-07T17:01:36.549017+0000 mon.a (mon.0) 2370 : cluster [DBG] osdmap e1152: 8 total, 8 up, 8 in 2024-04-07T17:01:37.951 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:01:37 smithi100 bash[76202]: cluster 2024-04-07T17:01:37.527119+0000 mon.a (mon.0) 2371 : cluster [DBG] osdmap e1153: 8 total, 8 up, 8 in 2024-04-07T17:01:38.700 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:01:38 smithi100 bash[74942]: cluster 2024-04-07T17:01:37.766717+0000 mgr.y (mgr.25175) 1388 : cluster [DBG] pgmap v2116: 169 pgs: 169 active+clean; 357 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 15 MiB/s rd, 9.5 MiB/s wr, 290 op/s 2024-04-07T17:01:38.701 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:01:38 smithi100 bash[76202]: cluster 2024-04-07T17:01:37.766717+0000 mgr.y (mgr.25175) 1388 : cluster [DBG] pgmap v2116: 169 pgs: 169 active+clean; 357 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 15 MiB/s rd, 9.5 MiB/s wr, 290 op/s 2024-04-07T17:01:38.701 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:01:38 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:01:38] "GET /metrics HTTP/1.1" 200 38918 "" "Prometheus/2.43.0" 2024-04-07T17:01:38.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:01:38 smithi144 bash[66654]: cluster 2024-04-07T17:01:37.766717+0000 mgr.y (mgr.25175) 1388 : cluster [DBG] pgmap v2116: 169 pgs: 169 active+clean; 357 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 15 MiB/s rd, 9.5 MiB/s wr, 290 op/s 2024-04-07T17:01:39.232 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestMigration.SnapsOtherPool (12978 ms) 2024-04-07T17:01:39.233 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestMigration.SnapsDataPool 2024-04-07T17:01:39.326 INFO:tasks.workunit.client.0.smithi100.stdout:resize: 83886080 2024-04-07T17:01:39.357 INFO:tasks.workunit.client.0.smithi100.stdout:write: 1 0~8388609 2024-04-07T17:01:39.465 INFO:tasks.workunit.client.0.smithi100.stdout:snap_create: snap1 2024-04-07T17:01:39.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:01:39 smithi144 bash[66654]: cluster 2024-04-07T17:01:38.563872+0000 mon.a (mon.0) 2372 : cluster [DBG] osdmap e1154: 8 total, 8 up, 8 in 2024-04-07T17:01:39.950 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:01:39 smithi100 bash[74942]: cluster 2024-04-07T17:01:38.563872+0000 mon.a (mon.0) 2372 : cluster [DBG] osdmap e1154: 8 total, 8 up, 8 in 2024-04-07T17:01:39.951 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:01:39 smithi100 bash[76202]: cluster 2024-04-07T17:01:38.563872+0000 mon.a (mon.0) 2372 : cluster [DBG] osdmap e1154: 8 total, 8 up, 8 in 2024-04-07T17:01:40.604 INFO:tasks.workunit.client.0.smithi100.stdout:write: 1 8388609~8388609 2024-04-07T17:01:40.643 INFO:tasks.workunit.client.0.smithi100.stdout:write: X 0~1000 2024-04-07T17:01:40.643 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 1010~1000 2024-04-07T17:01:40.643 INFO:tasks.workunit.client.0.smithi100.stdout:snap_create: snap2 2024-04-07T17:01:40.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:01:40 smithi144 bash[66654]: cluster 2024-04-07T17:01:39.569110+0000 mon.a (mon.0) 2373 : cluster [DBG] osdmap e1155: 8 total, 8 up, 8 in 2024-04-07T17:01:40.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:01:40 smithi144 bash[66654]: cluster 2024-04-07T17:01:39.767526+0000 mgr.y (mgr.25175) 1389 : cluster [DBG] pgmap v2119: 169 pgs: 169 active+clean; 345 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 54 MiB/s rd, 7.0 MiB/s wr, 395 op/s 2024-04-07T17:01:40.950 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:01:40 smithi100 bash[74942]: cluster 2024-04-07T17:01:39.569110+0000 mon.a (mon.0) 2373 : cluster [DBG] osdmap e1155: 8 total, 8 up, 8 in 2024-04-07T17:01:40.951 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:01:40 smithi100 bash[74942]: cluster 2024-04-07T17:01:39.767526+0000 mgr.y (mgr.25175) 1389 : cluster [DBG] pgmap v2119: 169 pgs: 169 active+clean; 345 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 54 MiB/s rd, 7.0 MiB/s wr, 395 op/s 2024-04-07T17:01:40.951 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:01:40 smithi100 bash[76202]: cluster 2024-04-07T17:01:39.569110+0000 mon.a (mon.0) 2373 : cluster [DBG] osdmap e1155: 8 total, 8 up, 8 in 2024-04-07T17:01:40.951 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:01:40 smithi100 bash[76202]: cluster 2024-04-07T17:01:39.767526+0000 mgr.y (mgr.25175) 1389 : cluster [DBG] pgmap v2119: 169 pgs: 169 active+clean; 345 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 54 MiB/s rd, 7.0 MiB/s wr, 395 op/s 2024-04-07T17:01:41.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:01:41 smithi144 bash[66654]: cluster 2024-04-07T17:01:40.579089+0000 mon.a (mon.0) 2374 : cluster [DBG] osdmap e1156: 8 total, 8 up, 8 in 2024-04-07T17:01:41.950 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:01:41 smithi100 bash[74942]: cluster 2024-04-07T17:01:40.579089+0000 mon.a (mon.0) 2374 : cluster [DBG] osdmap e1156: 8 total, 8 up, 8 in 2024-04-07T17:01:41.951 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:01:41 smithi100 bash[76202]: cluster 2024-04-07T17:01:40.579089+0000 mon.a (mon.0) 2374 : cluster [DBG] osdmap e1156: 8 total, 8 up, 8 in 2024-04-07T17:01:42.597 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:01:42 smithi144 bash[95233]: ts=2024-04-07T17:01:42.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:01:42.634 INFO:tasks.workunit.client.0.smithi100.stdout:write: X 8388609~1000 2024-04-07T17:01:42.652 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 16777228~1000 2024-04-07T17:01:42.652 INFO:tasks.workunit.client.0.smithi100.stdout:resize: 167772160 2024-04-07T17:01:42.685 INFO:tasks.workunit.client.0.smithi100.stdout:write: 2 83886079~8388609 2024-04-07T17:01:42.739 INFO:tasks.workunit.client.0.smithi100.stdout:snap_create: snap3 2024-04-07T17:01:42.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:01:42 smithi144 bash[66654]: cluster 2024-04-07T17:01:41.593787+0000 mon.a (mon.0) 2375 : cluster [DBG] osdmap e1157: 8 total, 8 up, 8 in 2024-04-07T17:01:42.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:01:42 smithi144 bash[66654]: cluster 2024-04-07T17:01:41.768328+0000 mgr.y (mgr.25175) 1390 : cluster [DBG] pgmap v2122: 169 pgs: 169 active+clean; 353 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 47 MiB/s rd, 7.0 MiB/s wr, 199 op/s 2024-04-07T17:01:42.950 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:01:42 smithi100 bash[74942]: cluster 2024-04-07T17:01:41.593787+0000 mon.a (mon.0) 2375 : cluster [DBG] osdmap e1157: 8 total, 8 up, 8 in 2024-04-07T17:01:42.950 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:01:42 smithi100 bash[74942]: cluster 2024-04-07T17:01:41.768328+0000 mgr.y (mgr.25175) 1390 : cluster [DBG] pgmap v2122: 169 pgs: 169 active+clean; 353 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 47 MiB/s rd, 7.0 MiB/s wr, 199 op/s 2024-04-07T17:01:42.951 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:01:42 smithi100 bash[76202]: cluster 2024-04-07T17:01:41.593787+0000 mon.a (mon.0) 2375 : cluster [DBG] osdmap e1157: 8 total, 8 up, 8 in 2024-04-07T17:01:42.951 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:01:42 smithi100 bash[76202]: cluster 2024-04-07T17:01:41.768328+0000 mgr.y (mgr.25175) 1390 : cluster [DBG] pgmap v2122: 169 pgs: 169 active+clean; 353 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 47 MiB/s rd, 7.0 MiB/s wr, 199 op/s 2024-04-07T17:01:43.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:01:43 smithi144 bash[66654]: cluster 2024-04-07T17:01:42.604745+0000 mon.a (mon.0) 2376 : cluster [DBG] osdmap e1158: 8 total, 8 up, 8 in 2024-04-07T17:01:43.950 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:01:43 smithi100 bash[74942]: cluster 2024-04-07T17:01:42.604745+0000 mon.a (mon.0) 2376 : cluster [DBG] osdmap e1158: 8 total, 8 up, 8 in 2024-04-07T17:01:43.951 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:01:43 smithi100 bash[76202]: cluster 2024-04-07T17:01:42.604745+0000 mon.a (mon.0) 2376 : cluster [DBG] osdmap e1158: 8 total, 8 up, 8 in 2024-04-07T17:01:44.653 INFO:tasks.workunit.client.0.smithi100.stdout:resize: 83886080 2024-04-07T17:01:44.713 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 83886079~1 2024-04-07T17:01:44.734 INFO:tasks.workunit.client.0.smithi100.stdout:migration_prepare 2024-04-07T17:01:44.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:01:44 smithi144 bash[66654]: cluster 2024-04-07T17:01:43.611448+0000 mon.a (mon.0) 2377 : cluster [DBG] osdmap e1159: 8 total, 8 up, 8 in 2024-04-07T17:01:44.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:01:44 smithi144 bash[66654]: cluster 2024-04-07T17:01:43.769141+0000 mgr.y (mgr.25175) 1391 : cluster [DBG] pgmap v2125: 169 pgs: 169 active+clean; 357 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 27 MiB/s rd, 7.0 MiB/s wr, 70 op/s 2024-04-07T17:01:44.950 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:01:44 smithi100 bash[74942]: cluster 2024-04-07T17:01:43.611448+0000 mon.a (mon.0) 2377 : cluster [DBG] osdmap e1159: 8 total, 8 up, 8 in 2024-04-07T17:01:44.951 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:01:44 smithi100 bash[74942]: cluster 2024-04-07T17:01:43.769141+0000 mgr.y (mgr.25175) 1391 : cluster [DBG] pgmap v2125: 169 pgs: 169 active+clean; 357 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 27 MiB/s rd, 7.0 MiB/s wr, 70 op/s 2024-04-07T17:01:44.951 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:01:44 smithi100 bash[76202]: cluster 2024-04-07T17:01:43.611448+0000 mon.a (mon.0) 2377 : cluster [DBG] osdmap e1159: 8 total, 8 up, 8 in 2024-04-07T17:01:44.951 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:01:44 smithi100 bash[76202]: cluster 2024-04-07T17:01:43.769141+0000 mgr.y (mgr.25175) 1391 : cluster [DBG] pgmap v2125: 169 pgs: 169 active+clean; 357 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 27 MiB/s rd, 7.0 MiB/s wr, 70 op/s 2024-04-07T17:01:45.950 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:01:45 smithi100 bash[74942]: cluster 2024-04-07T17:01:44.623820+0000 mon.a (mon.0) 2378 : cluster [DBG] osdmap e1160: 8 total, 8 up, 8 in 2024-04-07T17:01:45.951 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:01:45 smithi100 bash[76202]: cluster 2024-04-07T17:01:44.623820+0000 mon.a (mon.0) 2378 : cluster [DBG] osdmap e1160: 8 total, 8 up, 8 in 2024-04-07T17:01:46.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:01:45 smithi144 bash[66654]: cluster 2024-04-07T17:01:44.623820+0000 mon.a (mon.0) 2378 : cluster [DBG] osdmap e1160: 8 total, 8 up, 8 in 2024-04-07T17:01:46.950 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:01:46 smithi100 bash[74942]: cluster 2024-04-07T17:01:45.633913+0000 mon.a (mon.0) 2379 : cluster [DBG] osdmap e1161: 8 total, 8 up, 8 in 2024-04-07T17:01:46.951 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:01:46 smithi100 bash[74942]: cluster 2024-04-07T17:01:45.769997+0000 mgr.y (mgr.25175) 1392 : cluster [DBG] pgmap v2128: 169 pgs: 169 active+clean; 389 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 19 MiB/s rd, 10 MiB/s wr, 185 op/s 2024-04-07T17:01:46.951 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:01:46 smithi100 bash[76202]: cluster 2024-04-07T17:01:45.633913+0000 mon.a (mon.0) 2379 : cluster [DBG] osdmap e1161: 8 total, 8 up, 8 in 2024-04-07T17:01:46.951 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:01:46 smithi100 bash[76202]: cluster 2024-04-07T17:01:45.769997+0000 mgr.y (mgr.25175) 1392 : cluster [DBG] pgmap v2128: 169 pgs: 169 active+clean; 389 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 19 MiB/s rd, 10 MiB/s wr, 185 op/s 2024-04-07T17:01:47.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:01:46 smithi144 bash[66654]: cluster 2024-04-07T17:01:45.633913+0000 mon.a (mon.0) 2379 : cluster [DBG] osdmap e1161: 8 total, 8 up, 8 in 2024-04-07T17:01:47.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:01:46 smithi144 bash[66654]: cluster 2024-04-07T17:01:45.769997+0000 mgr.y (mgr.25175) 1392 : cluster [DBG] pgmap v2128: 169 pgs: 169 active+clean; 389 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 19 MiB/s rd, 10 MiB/s wr, 185 op/s 2024-04-07T17:01:47.951 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:01:47 smithi100 bash[74942]: cluster 2024-04-07T17:01:46.648339+0000 mon.a (mon.0) 2380 : cluster [DBG] osdmap e1162: 8 total, 8 up, 8 in 2024-04-07T17:01:47.968 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:01:47 smithi100 bash[74942]: audit 2024-04-07T17:01:46.799962+0000 mon.a (mon.0) 2381 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:01:47.968 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:01:47 smithi100 bash[74942]: audit 2024-04-07T17:01:47.025023+0000 mon.a (mon.0) 2382 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:01:47.968 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:01:47 smithi100 bash[74942]: cluster 2024-04-07T17:01:47.525545+0000 mon.a (mon.0) 2383 : cluster [DBG] osdmap e1163: 8 total, 8 up, 8 in 2024-04-07T17:01:47.968 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:01:47 smithi100 bash[76202]: cluster 2024-04-07T17:01:46.648339+0000 mon.a (mon.0) 2380 : cluster [DBG] osdmap e1162: 8 total, 8 up, 8 in 2024-04-07T17:01:47.968 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:01:47 smithi100 bash[76202]: audit 2024-04-07T17:01:46.799962+0000 mon.a (mon.0) 2381 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:01:47.968 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:01:47 smithi100 bash[76202]: audit 2024-04-07T17:01:47.025023+0000 mon.a (mon.0) 2382 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:01:47.968 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:01:47 smithi100 bash[76202]: cluster 2024-04-07T17:01:47.525545+0000 mon.a (mon.0) 2383 : cluster [DBG] osdmap e1163: 8 total, 8 up, 8 in 2024-04-07T17:01:48.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:01:47 smithi144 bash[66654]: cluster 2024-04-07T17:01:46.648339+0000 mon.a (mon.0) 2380 : cluster [DBG] osdmap e1162: 8 total, 8 up, 8 in 2024-04-07T17:01:48.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:01:47 smithi144 bash[66654]: audit 2024-04-07T17:01:46.799962+0000 mon.a (mon.0) 2381 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:01:48.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:01:47 smithi144 bash[66654]: audit 2024-04-07T17:01:47.025023+0000 mon.a (mon.0) 2382 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:01:48.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:01:47 smithi144 bash[66654]: cluster 2024-04-07T17:01:47.525545+0000 mon.a (mon.0) 2383 : cluster [DBG] osdmap e1163: 8 total, 8 up, 8 in 2024-04-07T17:01:48.297 INFO:tasks.workunit.client.0.smithi100.stdout:migration_execute 2024-04-07T17:01:48.615 INFO:tasks.workunit.client.0.smithi100.stdout:migration_commit 2024-04-07T17:01:48.700 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:01:48 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:01:48] "GET /metrics HTTP/1.1" 200 38922 "" "Prometheus/2.43.0" 2024-04-07T17:01:48.701 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:01:48 smithi100 bash[74942]: audit 2024-04-07T17:01:47.756921+0000 mon.a (mon.0) 2384 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:01:48.701 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:01:48 smithi100 bash[74942]: audit 2024-04-07T17:01:47.757754+0000 mon.a (mon.0) 2385 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:01:48.701 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:01:48 smithi100 bash[74942]: audit 2024-04-07T17:01:47.766567+0000 mon.a (mon.0) 2386 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:01:48.701 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:01:48 smithi100 bash[74942]: cluster 2024-04-07T17:01:47.770489+0000 mgr.y (mgr.25175) 1393 : cluster [DBG] pgmap v2131: 169 pgs: 169 active+clean; 397 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 1.1 MiB/s rd, 8.0 MiB/s wr, 128 op/s 2024-04-07T17:01:48.701 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:01:48 smithi100 bash[76202]: audit 2024-04-07T17:01:47.756921+0000 mon.a (mon.0) 2384 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:01:48.701 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:01:48 smithi100 bash[76202]: audit 2024-04-07T17:01:47.757754+0000 mon.a (mon.0) 2385 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:01:48.701 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:01:48 smithi100 bash[76202]: audit 2024-04-07T17:01:47.766567+0000 mon.a (mon.0) 2386 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:01:48.701 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:01:48 smithi100 bash[76202]: cluster 2024-04-07T17:01:47.770489+0000 mgr.y (mgr.25175) 1393 : cluster [DBG] pgmap v2131: 169 pgs: 169 active+clean; 397 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 1.1 MiB/s rd, 8.0 MiB/s wr, 128 op/s 2024-04-07T17:01:49.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:01:48 smithi144 bash[66654]: audit 2024-04-07T17:01:47.756921+0000 mon.a (mon.0) 2384 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:01:49.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:01:48 smithi144 bash[66654]: audit 2024-04-07T17:01:47.757754+0000 mon.a (mon.0) 2385 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:01:49.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:01:48 smithi144 bash[66654]: audit 2024-04-07T17:01:47.766567+0000 mon.a (mon.0) 2386 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:01:49.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:01:48 smithi144 bash[66654]: cluster 2024-04-07T17:01:47.770489+0000 mgr.y (mgr.25175) 1393 : cluster [DBG] pgmap v2131: 169 pgs: 169 active+clean; 397 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 1.1 MiB/s rd, 8.0 MiB/s wr, 128 op/s 2024-04-07T17:01:50.950 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:01:50 smithi100 bash[74942]: cluster 2024-04-07T17:01:49.652883+0000 mon.a (mon.0) 2387 : cluster [DBG] osdmap e1164: 8 total, 8 up, 8 in 2024-04-07T17:01:50.951 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:01:50 smithi100 bash[74942]: cluster 2024-04-07T17:01:49.771211+0000 mgr.y (mgr.25175) 1394 : cluster [DBG] pgmap v2133: 169 pgs: 169 active+clean; 401 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 20 MiB/s rd, 2.9 MiB/s wr, 279 op/s 2024-04-07T17:01:50.951 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:01:50 smithi100 bash[76202]: cluster 2024-04-07T17:01:49.652883+0000 mon.a (mon.0) 2387 : cluster [DBG] osdmap e1164: 8 total, 8 up, 8 in 2024-04-07T17:01:50.951 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:01:50 smithi100 bash[76202]: cluster 2024-04-07T17:01:49.771211+0000 mgr.y (mgr.25175) 1394 : cluster [DBG] pgmap v2133: 169 pgs: 169 active+clean; 401 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 20 MiB/s rd, 2.9 MiB/s wr, 279 op/s 2024-04-07T17:01:51.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:01:50 smithi144 bash[66654]: cluster 2024-04-07T17:01:49.652883+0000 mon.a (mon.0) 2387 : cluster [DBG] osdmap e1164: 8 total, 8 up, 8 in 2024-04-07T17:01:51.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:01:50 smithi144 bash[66654]: cluster 2024-04-07T17:01:49.771211+0000 mgr.y (mgr.25175) 1394 : cluster [DBG] pgmap v2133: 169 pgs: 169 active+clean; 401 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 20 MiB/s rd, 2.9 MiB/s wr, 279 op/s 2024-04-07T17:01:51.950 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:01:51 smithi100 bash[74942]: cluster 2024-04-07T17:01:50.663864+0000 mon.a (mon.0) 2388 : cluster [DBG] osdmap e1165: 8 total, 8 up, 8 in 2024-04-07T17:01:51.951 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:01:51 smithi100 bash[76202]: cluster 2024-04-07T17:01:50.663864+0000 mon.a (mon.0) 2388 : cluster [DBG] osdmap e1165: 8 total, 8 up, 8 in 2024-04-07T17:01:52.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:01:51 smithi144 bash[66654]: cluster 2024-04-07T17:01:50.663864+0000 mon.a (mon.0) 2388 : cluster [DBG] osdmap e1165: 8 total, 8 up, 8 in 2024-04-07T17:01:52.310 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestMigration.SnapsDataPool (13078 ms) 2024-04-07T17:01:52.310 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestMigration.SnapsShrinkAfterPrepare 2024-04-07T17:01:52.399 INFO:tasks.workunit.client.0.smithi100.stdout:resize: 83886080 2024-04-07T17:01:52.432 INFO:tasks.workunit.client.0.smithi100.stdout:write: 1 0~8388609 2024-04-07T17:01:52.551 INFO:tasks.workunit.client.0.smithi100.stdout:snap_create: snap1 2024-04-07T17:01:52.614 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:01:52 smithi144 bash[95233]: ts=2024-04-07T17:01:52.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:01:52.951 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:01:52 smithi100 bash[74942]: cluster 2024-04-07T17:01:51.675366+0000 mon.a (mon.0) 2389 : cluster [DBG] osdmap e1166: 8 total, 8 up, 8 in 2024-04-07T17:01:52.951 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:01:52 smithi100 bash[74942]: cluster 2024-04-07T17:01:51.772097+0000 mgr.y (mgr.25175) 1395 : cluster [DBG] pgmap v2136: 169 pgs: 169 active+clean; 405 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 22 MiB/s rd, 1.9 MiB/s wr, 413 op/s 2024-04-07T17:01:52.951 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:01:52 smithi100 bash[76202]: cluster 2024-04-07T17:01:51.675366+0000 mon.a (mon.0) 2389 : cluster [DBG] osdmap e1166: 8 total, 8 up, 8 in 2024-04-07T17:01:52.951 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:01:52 smithi100 bash[76202]: cluster 2024-04-07T17:01:51.772097+0000 mgr.y (mgr.25175) 1395 : cluster [DBG] pgmap v2136: 169 pgs: 169 active+clean; 405 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 22 MiB/s rd, 1.9 MiB/s wr, 413 op/s 2024-04-07T17:01:53.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:01:52 smithi144 bash[66654]: cluster 2024-04-07T17:01:51.675366+0000 mon.a (mon.0) 2389 : cluster [DBG] osdmap e1166: 8 total, 8 up, 8 in 2024-04-07T17:01:53.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:01:52 smithi144 bash[66654]: cluster 2024-04-07T17:01:51.772097+0000 mgr.y (mgr.25175) 1395 : cluster [DBG] pgmap v2136: 169 pgs: 169 active+clean; 405 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 22 MiB/s rd, 1.9 MiB/s wr, 413 op/s 2024-04-07T17:01:53.717 INFO:tasks.workunit.client.0.smithi100.stdout:write: 1 8388609~8388609 2024-04-07T17:01:53.757 INFO:tasks.workunit.client.0.smithi100.stdout:write: X 0~1000 2024-04-07T17:01:53.758 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 1010~1000 2024-04-07T17:01:53.758 INFO:tasks.workunit.client.0.smithi100.stdout:snap_create: snap2 2024-04-07T17:01:53.950 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:01:53 smithi100 bash[74942]: cluster 2024-04-07T17:01:52.686413+0000 mon.a (mon.0) 2390 : cluster [DBG] osdmap e1167: 8 total, 8 up, 8 in 2024-04-07T17:01:53.951 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:01:53 smithi100 bash[76202]: cluster 2024-04-07T17:01:52.686413+0000 mon.a (mon.0) 2390 : cluster [DBG] osdmap e1167: 8 total, 8 up, 8 in 2024-04-07T17:01:54.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:01:53 smithi144 bash[66654]: cluster 2024-04-07T17:01:52.686413+0000 mon.a (mon.0) 2390 : cluster [DBG] osdmap e1167: 8 total, 8 up, 8 in 2024-04-07T17:01:55.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:01:54 smithi144 bash[66654]: cluster 2024-04-07T17:01:53.695338+0000 mon.a (mon.0) 2391 : cluster [DBG] osdmap e1168: 8 total, 8 up, 8 in 2024-04-07T17:01:55.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:01:54 smithi144 bash[66654]: cluster 2024-04-07T17:01:53.773053+0000 mgr.y (mgr.25175) 1396 : cluster [DBG] pgmap v2139: 169 pgs: 169 active+clean; 413 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 29 MiB/s rd, 5.0 MiB/s wr, 215 op/s 2024-04-07T17:01:55.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:01:54 smithi100 bash[74942]: cluster 2024-04-07T17:01:53.695338+0000 mon.a (mon.0) 2391 : cluster [DBG] osdmap e1168: 8 total, 8 up, 8 in 2024-04-07T17:01:55.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:01:54 smithi100 bash[74942]: cluster 2024-04-07T17:01:53.773053+0000 mgr.y (mgr.25175) 1396 : cluster [DBG] pgmap v2139: 169 pgs: 169 active+clean; 413 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 29 MiB/s rd, 5.0 MiB/s wr, 215 op/s 2024-04-07T17:01:55.201 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:01:54 smithi100 bash[76202]: cluster 2024-04-07T17:01:53.695338+0000 mon.a (mon.0) 2391 : cluster [DBG] osdmap e1168: 8 total, 8 up, 8 in 2024-04-07T17:01:55.201 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:01:54 smithi100 bash[76202]: cluster 2024-04-07T17:01:53.773053+0000 mgr.y (mgr.25175) 1396 : cluster [DBG] pgmap v2139: 169 pgs: 169 active+clean; 413 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 29 MiB/s rd, 5.0 MiB/s wr, 215 op/s 2024-04-07T17:01:55.745 INFO:tasks.workunit.client.0.smithi100.stdout:write: X 8388609~1000 2024-04-07T17:01:55.763 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 16777228~1000 2024-04-07T17:01:55.764 INFO:tasks.workunit.client.0.smithi100.stdout:resize: 167772160 2024-04-07T17:01:55.787 INFO:tasks.workunit.client.0.smithi100.stdout:write: 2 83886079~8388609 2024-04-07T17:01:55.844 INFO:tasks.workunit.client.0.smithi100.stdout:snap_create: snap3 2024-04-07T17:01:56.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:01:55 smithi144 bash[66654]: cluster 2024-04-07T17:01:54.697593+0000 mon.a (mon.0) 2392 : cluster [DBG] osdmap e1169: 8 total, 8 up, 8 in 2024-04-07T17:01:56.200 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:01:55 smithi100 bash[74942]: cluster 2024-04-07T17:01:54.697593+0000 mon.a (mon.0) 2392 : cluster [DBG] osdmap e1169: 8 total, 8 up, 8 in 2024-04-07T17:01:56.201 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:01:55 smithi100 bash[76202]: cluster 2024-04-07T17:01:54.697593+0000 mon.a (mon.0) 2392 : cluster [DBG] osdmap e1169: 8 total, 8 up, 8 in 2024-04-07T17:01:57.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:01:56 smithi144 bash[66654]: cluster 2024-04-07T17:01:55.724573+0000 mon.a (mon.0) 2393 : cluster [DBG] osdmap e1170: 8 total, 8 up, 8 in 2024-04-07T17:01:57.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:01:56 smithi144 bash[66654]: cluster 2024-04-07T17:01:55.774017+0000 mgr.y (mgr.25175) 1397 : cluster [DBG] pgmap v2142: 169 pgs: 169 active+clean; 409 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 37 MiB/s rd, 7.0 MiB/s wr, 140 op/s 2024-04-07T17:01:57.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:01:56 smithi100 bash[74942]: cluster 2024-04-07T17:01:55.724573+0000 mon.a (mon.0) 2393 : cluster [DBG] osdmap e1170: 8 total, 8 up, 8 in 2024-04-07T17:01:57.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:01:56 smithi100 bash[74942]: cluster 2024-04-07T17:01:55.774017+0000 mgr.y (mgr.25175) 1397 : cluster [DBG] pgmap v2142: 169 pgs: 169 active+clean; 409 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 37 MiB/s rd, 7.0 MiB/s wr, 140 op/s 2024-04-07T17:01:57.201 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:01:56 smithi100 bash[76202]: cluster 2024-04-07T17:01:55.724573+0000 mon.a (mon.0) 2393 : cluster [DBG] osdmap e1170: 8 total, 8 up, 8 in 2024-04-07T17:01:57.201 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:01:56 smithi100 bash[76202]: cluster 2024-04-07T17:01:55.774017+0000 mgr.y (mgr.25175) 1397 : cluster [DBG] pgmap v2142: 169 pgs: 169 active+clean; 409 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 37 MiB/s rd, 7.0 MiB/s wr, 140 op/s 2024-04-07T17:01:57.567 INFO:tasks.workunit.client.0.smithi100.stdout:resize: 83886080 2024-04-07T17:01:57.628 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 83886079~1 2024-04-07T17:01:57.651 INFO:tasks.workunit.client.0.smithi100.stdout:migration_prepare 2024-04-07T17:01:58.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:01:57 smithi144 bash[66654]: cluster 2024-04-07T17:01:56.731102+0000 mon.a (mon.0) 2394 : cluster [DBG] osdmap e1171: 8 total, 8 up, 8 in 2024-04-07T17:01:58.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:01:57 smithi144 bash[66654]: cluster 2024-04-07T17:01:57.536869+0000 mon.a (mon.0) 2395 : cluster [DBG] osdmap e1172: 8 total, 8 up, 8 in 2024-04-07T17:01:58.200 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:01:57 smithi100 bash[74942]: cluster 2024-04-07T17:01:56.731102+0000 mon.a (mon.0) 2394 : cluster [DBG] osdmap e1171: 8 total, 8 up, 8 in 2024-04-07T17:01:58.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:01:57 smithi100 bash[74942]: cluster 2024-04-07T17:01:57.536869+0000 mon.a (mon.0) 2395 : cluster [DBG] osdmap e1172: 8 total, 8 up, 8 in 2024-04-07T17:01:58.201 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:01:57 smithi100 bash[76202]: cluster 2024-04-07T17:01:56.731102+0000 mon.a (mon.0) 2394 : cluster [DBG] osdmap e1171: 8 total, 8 up, 8 in 2024-04-07T17:01:58.201 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:01:57 smithi100 bash[76202]: cluster 2024-04-07T17:01:57.536869+0000 mon.a (mon.0) 2395 : cluster [DBG] osdmap e1172: 8 total, 8 up, 8 in 2024-04-07T17:01:58.701 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:01:58 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:01:58] "GET /metrics HTTP/1.1" 200 38922 "" "Prometheus/2.43.0" 2024-04-07T17:01:59.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:01:58 smithi144 bash[66654]: cluster 2024-04-07T17:01:57.775068+0000 mgr.y (mgr.25175) 1398 : cluster [DBG] pgmap v2145: 169 pgs: 169 active+clean; 429 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 20 MiB/s rd, 8.0 MiB/s wr, 128 op/s 2024-04-07T17:01:59.200 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:01:58 smithi100 bash[74942]: cluster 2024-04-07T17:01:57.775068+0000 mgr.y (mgr.25175) 1398 : cluster [DBG] pgmap v2145: 169 pgs: 169 active+clean; 429 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 20 MiB/s rd, 8.0 MiB/s wr, 128 op/s 2024-04-07T17:01:59.201 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:01:58 smithi100 bash[76202]: cluster 2024-04-07T17:01:57.775068+0000 mgr.y (mgr.25175) 1398 : cluster [DBG] pgmap v2145: 169 pgs: 169 active+clean; 429 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 20 MiB/s rd, 8.0 MiB/s wr, 128 op/s 2024-04-07T17:02:00.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:01:59 smithi144 bash[66654]: cluster 2024-04-07T17:01:58.745813+0000 mon.a (mon.0) 2396 : cluster [DBG] osdmap e1173: 8 total, 8 up, 8 in 2024-04-07T17:02:00.200 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:01:59 smithi100 bash[76202]: cluster 2024-04-07T17:01:58.745813+0000 mon.a (mon.0) 2396 : cluster [DBG] osdmap e1173: 8 total, 8 up, 8 in 2024-04-07T17:02:00.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:01:59 smithi100 bash[74942]: cluster 2024-04-07T17:01:58.745813+0000 mon.a (mon.0) 2396 : cluster [DBG] osdmap e1173: 8 total, 8 up, 8 in 2024-04-07T17:02:01.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:02:00 smithi144 bash[66654]: cluster 2024-04-07T17:01:59.745860+0000 mon.a (mon.0) 2397 : cluster [DBG] osdmap e1174: 8 total, 8 up, 8 in 2024-04-07T17:02:01.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:02:00 smithi144 bash[66654]: cluster 2024-04-07T17:01:59.775869+0000 mgr.y (mgr.25175) 1399 : cluster [DBG] pgmap v2148: 169 pgs: 169 active+clean; 453 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 10 MiB/s rd, 9.0 MiB/s wr, 138 op/s 2024-04-07T17:02:01.200 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:02:00 smithi100 bash[76202]: cluster 2024-04-07T17:01:59.745860+0000 mon.a (mon.0) 2397 : cluster [DBG] osdmap e1174: 8 total, 8 up, 8 in 2024-04-07T17:02:01.201 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:02:00 smithi100 bash[76202]: cluster 2024-04-07T17:01:59.775869+0000 mgr.y (mgr.25175) 1399 : cluster [DBG] pgmap v2148: 169 pgs: 169 active+clean; 453 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 10 MiB/s rd, 9.0 MiB/s wr, 138 op/s 2024-04-07T17:02:01.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:02:00 smithi100 bash[74942]: cluster 2024-04-07T17:01:59.745860+0000 mon.a (mon.0) 2397 : cluster [DBG] osdmap e1174: 8 total, 8 up, 8 in 2024-04-07T17:02:01.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:02:00 smithi100 bash[74942]: cluster 2024-04-07T17:01:59.775869+0000 mgr.y (mgr.25175) 1399 : cluster [DBG] pgmap v2148: 169 pgs: 169 active+clean; 453 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 10 MiB/s rd, 9.0 MiB/s wr, 138 op/s 2024-04-07T17:02:01.731 INFO:tasks.workunit.client.0.smithi100.stdout:resize: 41943040 2024-04-07T17:02:01.799 INFO:tasks.workunit.client.0.smithi100.stdout:migration_execute 2024-04-07T17:02:02.088 INFO:tasks.workunit.client.0.smithi100.stdout:migration_commit 2024-04-07T17:02:02.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:02:01 smithi144 bash[66654]: cluster 2024-04-07T17:02:00.768935+0000 mon.a (mon.0) 2398 : cluster [DBG] osdmap e1175: 8 total, 8 up, 8 in 2024-04-07T17:02:02.200 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:02:01 smithi100 bash[74942]: cluster 2024-04-07T17:02:00.768935+0000 mon.a (mon.0) 2398 : cluster [DBG] osdmap e1175: 8 total, 8 up, 8 in 2024-04-07T17:02:02.201 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:02:01 smithi100 bash[76202]: cluster 2024-04-07T17:02:00.768935+0000 mon.a (mon.0) 2398 : cluster [DBG] osdmap e1175: 8 total, 8 up, 8 in 2024-04-07T17:02:02.614 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:02:02 smithi144 bash[95233]: ts=2024-04-07T17:02:02.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:02:03.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:02:02 smithi144 bash[66654]: cluster 2024-04-07T17:02:01.776846+0000 mgr.y (mgr.25175) 1400 : cluster [DBG] pgmap v2150: 169 pgs: 169 active+clean; 453 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 68 KiB/s rd, 3.8 MiB/s wr, 100 op/s 2024-04-07T17:02:03.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:02:02 smithi144 bash[66654]: audit 2024-04-07T17:02:01.799938+0000 mon.a (mon.0) 2399 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:02:03.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:02:02 smithi100 bash[74942]: cluster 2024-04-07T17:02:01.776846+0000 mgr.y (mgr.25175) 1400 : cluster [DBG] pgmap v2150: 169 pgs: 169 active+clean; 453 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 68 KiB/s rd, 3.8 MiB/s wr, 100 op/s 2024-04-07T17:02:03.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:02:02 smithi100 bash[74942]: audit 2024-04-07T17:02:01.799938+0000 mon.a (mon.0) 2399 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:02:03.201 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:02:02 smithi100 bash[76202]: cluster 2024-04-07T17:02:01.776846+0000 mgr.y (mgr.25175) 1400 : cluster [DBG] pgmap v2150: 169 pgs: 169 active+clean; 453 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 68 KiB/s rd, 3.8 MiB/s wr, 100 op/s 2024-04-07T17:02:03.201 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:02:02 smithi100 bash[76202]: audit 2024-04-07T17:02:01.799938+0000 mon.a (mon.0) 2399 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:02:04.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:02:03 smithi144 bash[66654]: cluster 2024-04-07T17:02:02.771342+0000 mon.a (mon.0) 2400 : cluster [DBG] osdmap e1176: 8 total, 8 up, 8 in 2024-04-07T17:02:04.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:02:03 smithi100 bash[74942]: cluster 2024-04-07T17:02:02.771342+0000 mon.a (mon.0) 2400 : cluster [DBG] osdmap e1176: 8 total, 8 up, 8 in 2024-04-07T17:02:04.201 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:02:03 smithi100 bash[76202]: cluster 2024-04-07T17:02:02.771342+0000 mon.a (mon.0) 2400 : cluster [DBG] osdmap e1176: 8 total, 8 up, 8 in 2024-04-07T17:02:05.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:02:04 smithi144 bash[66654]: cluster 2024-04-07T17:02:03.778080+0000 mgr.y (mgr.25175) 1401 : cluster [DBG] pgmap v2152: 169 pgs: 169 active+clean; 457 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 19 MiB/s rd, 830 KiB/s wr, 216 op/s 2024-04-07T17:02:05.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:02:04 smithi144 bash[66654]: cluster 2024-04-07T17:02:03.797288+0000 mon.a (mon.0) 2401 : cluster [DBG] osdmap e1177: 8 total, 8 up, 8 in 2024-04-07T17:02:05.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:02:04 smithi100 bash[74942]: cluster 2024-04-07T17:02:03.778080+0000 mgr.y (mgr.25175) 1401 : cluster [DBG] pgmap v2152: 169 pgs: 169 active+clean; 457 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 19 MiB/s rd, 830 KiB/s wr, 216 op/s 2024-04-07T17:02:05.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:02:04 smithi100 bash[74942]: cluster 2024-04-07T17:02:03.797288+0000 mon.a (mon.0) 2401 : cluster [DBG] osdmap e1177: 8 total, 8 up, 8 in 2024-04-07T17:02:05.201 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:02:04 smithi100 bash[76202]: cluster 2024-04-07T17:02:03.778080+0000 mgr.y (mgr.25175) 1401 : cluster [DBG] pgmap v2152: 169 pgs: 169 active+clean; 457 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 19 MiB/s rd, 830 KiB/s wr, 216 op/s 2024-04-07T17:02:05.201 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:02:04 smithi100 bash[76202]: cluster 2024-04-07T17:02:03.797288+0000 mon.a (mon.0) 2401 : cluster [DBG] osdmap e1177: 8 total, 8 up, 8 in 2024-04-07T17:02:05.407 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestMigration.SnapsShrinkAfterPrepare (13097 ms) 2024-04-07T17:02:05.407 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestMigration.SnapsShrinkToZeroBeforePrepare 2024-04-07T17:02:05.496 INFO:tasks.workunit.client.0.smithi100.stdout:resize: 83886080 2024-04-07T17:02:05.528 INFO:tasks.workunit.client.0.smithi100.stdout:write: 1 0~8388609 2024-04-07T17:02:05.639 INFO:tasks.workunit.client.0.smithi100.stdout:snap_create: snap1 2024-04-07T17:02:06.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:02:05 smithi144 bash[66654]: cluster 2024-04-07T17:02:04.806494+0000 mon.a (mon.0) 2402 : cluster [DBG] osdmap e1178: 8 total, 8 up, 8 in 2024-04-07T17:02:06.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:02:05 smithi144 bash[66654]: cluster 2024-04-07T17:02:05.779127+0000 mgr.y (mgr.25175) 1402 : cluster [DBG] pgmap v2155: 169 pgs: 169 active+clean; 469 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 31 MiB/s rd, 4.0 MiB/s wr, 331 op/s 2024-04-07T17:02:06.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:02:05 smithi100 bash[74942]: cluster 2024-04-07T17:02:04.806494+0000 mon.a (mon.0) 2402 : cluster [DBG] osdmap e1178: 8 total, 8 up, 8 in 2024-04-07T17:02:06.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:02:05 smithi100 bash[74942]: cluster 2024-04-07T17:02:05.779127+0000 mgr.y (mgr.25175) 1402 : cluster [DBG] pgmap v2155: 169 pgs: 169 active+clean; 469 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 31 MiB/s rd, 4.0 MiB/s wr, 331 op/s 2024-04-07T17:02:06.201 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:02:05 smithi100 bash[76202]: cluster 2024-04-07T17:02:04.806494+0000 mon.a (mon.0) 2402 : cluster [DBG] osdmap e1178: 8 total, 8 up, 8 in 2024-04-07T17:02:06.201 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:02:05 smithi100 bash[76202]: cluster 2024-04-07T17:02:05.779127+0000 mgr.y (mgr.25175) 1402 : cluster [DBG] pgmap v2155: 169 pgs: 169 active+clean; 469 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 31 MiB/s rd, 4.0 MiB/s wr, 331 op/s 2024-04-07T17:02:06.850 INFO:tasks.workunit.client.0.smithi100.stdout:write: 1 8388609~8388609 2024-04-07T17:02:06.895 INFO:tasks.workunit.client.0.smithi100.stdout:write: X 0~1000 2024-04-07T17:02:06.896 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 1010~1000 2024-04-07T17:02:06.896 INFO:tasks.workunit.client.0.smithi100.stdout:snap_create: snap2 2024-04-07T17:02:07.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:02:06 smithi144 bash[66654]: cluster 2024-04-07T17:02:05.818803+0000 mon.a (mon.0) 2403 : cluster [DBG] osdmap e1179: 8 total, 8 up, 8 in 2024-04-07T17:02:07.200 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:02:06 smithi100 bash[74942]: cluster 2024-04-07T17:02:05.818803+0000 mon.a (mon.0) 2403 : cluster [DBG] osdmap e1179: 8 total, 8 up, 8 in 2024-04-07T17:02:07.201 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:02:06 smithi100 bash[76202]: cluster 2024-04-07T17:02:05.818803+0000 mon.a (mon.0) 2403 : cluster [DBG] osdmap e1179: 8 total, 8 up, 8 in 2024-04-07T17:02:08.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:02:07 smithi144 bash[66654]: cluster 2024-04-07T17:02:06.822742+0000 mon.a (mon.0) 2404 : cluster [DBG] osdmap e1180: 8 total, 8 up, 8 in 2024-04-07T17:02:08.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:02:07 smithi144 bash[66654]: cluster 2024-04-07T17:02:07.780542+0000 mgr.y (mgr.25175) 1403 : cluster [DBG] pgmap v2158: 169 pgs: 169 active+clean; 473 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 25 MiB/s rd, 6.0 MiB/s wr, 309 op/s 2024-04-07T17:02:08.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:02:07 smithi100 bash[74942]: cluster 2024-04-07T17:02:06.822742+0000 mon.a (mon.0) 2404 : cluster [DBG] osdmap e1180: 8 total, 8 up, 8 in 2024-04-07T17:02:08.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:02:07 smithi100 bash[74942]: cluster 2024-04-07T17:02:07.780542+0000 mgr.y (mgr.25175) 1403 : cluster [DBG] pgmap v2158: 169 pgs: 169 active+clean; 473 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 25 MiB/s rd, 6.0 MiB/s wr, 309 op/s 2024-04-07T17:02:08.201 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:02:07 smithi100 bash[76202]: cluster 2024-04-07T17:02:06.822742+0000 mon.a (mon.0) 2404 : cluster [DBG] osdmap e1180: 8 total, 8 up, 8 in 2024-04-07T17:02:08.201 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:02:07 smithi100 bash[76202]: cluster 2024-04-07T17:02:07.780542+0000 mgr.y (mgr.25175) 1403 : cluster [DBG] pgmap v2158: 169 pgs: 169 active+clean; 473 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 25 MiB/s rd, 6.0 MiB/s wr, 309 op/s 2024-04-07T17:02:08.700 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:02:08 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:02:08] "GET /metrics HTTP/1.1" 200 38915 "" "Prometheus/2.43.0" 2024-04-07T17:02:08.881 INFO:tasks.workunit.client.0.smithi100.stdout:write: X 8388609~1000 2024-04-07T17:02:08.897 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 16777228~1000 2024-04-07T17:02:08.897 INFO:tasks.workunit.client.0.smithi100.stdout:resize: 167772160 2024-04-07T17:02:08.923 INFO:tasks.workunit.client.0.smithi100.stdout:write: 2 83886079~8388609 2024-04-07T17:02:08.975 INFO:tasks.workunit.client.0.smithi100.stdout:snap_create: snap3 2024-04-07T17:02:09.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:02:08 smithi144 bash[66654]: cluster 2024-04-07T17:02:07.850669+0000 mon.a (mon.0) 2405 : cluster [DBG] osdmap e1181: 8 total, 8 up, 8 in 2024-04-07T17:02:09.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:02:08 smithi100 bash[74942]: cluster 2024-04-07T17:02:07.850669+0000 mon.a (mon.0) 2405 : cluster [DBG] osdmap e1181: 8 total, 8 up, 8 in 2024-04-07T17:02:09.201 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:02:08 smithi100 bash[76202]: cluster 2024-04-07T17:02:07.850669+0000 mon.a (mon.0) 2405 : cluster [DBG] osdmap e1181: 8 total, 8 up, 8 in 2024-04-07T17:02:10.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:02:09 smithi100 bash[74942]: cluster 2024-04-07T17:02:08.850907+0000 mon.a (mon.0) 2406 : cluster [DBG] osdmap e1182: 8 total, 8 up, 8 in 2024-04-07T17:02:10.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:02:09 smithi100 bash[74942]: cluster 2024-04-07T17:02:09.781693+0000 mgr.y (mgr.25175) 1404 : cluster [DBG] pgmap v2161: 169 pgs: 169 active+clean; 485 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 28 MiB/s rd, 8.0 MiB/s wr, 192 op/s 2024-04-07T17:02:10.201 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:02:09 smithi100 bash[76202]: cluster 2024-04-07T17:02:08.850907+0000 mon.a (mon.0) 2406 : cluster [DBG] osdmap e1182: 8 total, 8 up, 8 in 2024-04-07T17:02:10.201 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:02:09 smithi100 bash[76202]: cluster 2024-04-07T17:02:09.781693+0000 mgr.y (mgr.25175) 1404 : cluster [DBG] pgmap v2161: 169 pgs: 169 active+clean; 485 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 28 MiB/s rd, 8.0 MiB/s wr, 192 op/s 2024-04-07T17:02:10.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:02:09 smithi144 bash[66654]: cluster 2024-04-07T17:02:08.850907+0000 mon.a (mon.0) 2406 : cluster [DBG] osdmap e1182: 8 total, 8 up, 8 in 2024-04-07T17:02:10.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:02:09 smithi144 bash[66654]: cluster 2024-04-07T17:02:09.781693+0000 mgr.y (mgr.25175) 1404 : cluster [DBG] pgmap v2161: 169 pgs: 169 active+clean; 485 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 28 MiB/s rd, 8.0 MiB/s wr, 192 op/s 2024-04-07T17:02:10.899 INFO:tasks.workunit.client.0.smithi100.stdout:resize: 83886080 2024-04-07T17:02:10.961 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 83886079~1 2024-04-07T17:02:10.986 INFO:tasks.workunit.client.0.smithi100.stdout:resize: 0 2024-04-07T17:02:11.030 INFO:tasks.workunit.client.0.smithi100.stdout:migration_prepare 2024-04-07T17:02:11.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:02:10 smithi100 bash[74942]: cluster 2024-04-07T17:02:09.874865+0000 mon.a (mon.0) 2407 : cluster [DBG] osdmap e1183: 8 total, 8 up, 8 in 2024-04-07T17:02:11.201 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:02:10 smithi100 bash[76202]: cluster 2024-04-07T17:02:09.874865+0000 mon.a (mon.0) 2407 : cluster [DBG] osdmap e1183: 8 total, 8 up, 8 in 2024-04-07T17:02:11.364 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:02:10 smithi144 bash[66654]: cluster 2024-04-07T17:02:09.874865+0000 mon.a (mon.0) 2407 : cluster [DBG] osdmap e1183: 8 total, 8 up, 8 in 2024-04-07T17:02:12.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:02:11 smithi100 bash[74942]: cluster 2024-04-07T17:02:10.876126+0000 mon.a (mon.0) 2408 : cluster [DBG] osdmap e1184: 8 total, 8 up, 8 in 2024-04-07T17:02:12.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:02:11 smithi100 bash[74942]: cluster 2024-04-07T17:02:11.782551+0000 mgr.y (mgr.25175) 1405 : cluster [DBG] pgmap v2164: 169 pgs: 169 active+clean; 489 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 21 MiB/s rd, 7.0 MiB/s wr, 89 op/s 2024-04-07T17:02:12.201 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:02:11 smithi100 bash[76202]: cluster 2024-04-07T17:02:10.876126+0000 mon.a (mon.0) 2408 : cluster [DBG] osdmap e1184: 8 total, 8 up, 8 in 2024-04-07T17:02:12.201 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:02:11 smithi100 bash[76202]: cluster 2024-04-07T17:02:11.782551+0000 mgr.y (mgr.25175) 1405 : cluster [DBG] pgmap v2164: 169 pgs: 169 active+clean; 489 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 21 MiB/s rd, 7.0 MiB/s wr, 89 op/s 2024-04-07T17:02:12.313 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:02:11 smithi144 bash[66654]: cluster 2024-04-07T17:02:10.876126+0000 mon.a (mon.0) 2408 : cluster [DBG] osdmap e1184: 8 total, 8 up, 8 in 2024-04-07T17:02:12.314 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:02:11 smithi144 bash[66654]: cluster 2024-04-07T17:02:11.782551+0000 mgr.y (mgr.25175) 1405 : cluster [DBG] pgmap v2164: 169 pgs: 169 active+clean; 489 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 21 MiB/s rd, 7.0 MiB/s wr, 89 op/s 2024-04-07T17:02:12.614 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:02:12 smithi144 bash[95233]: ts=2024-04-07T17:02:12.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:02:13.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:02:12 smithi100 bash[74942]: cluster 2024-04-07T17:02:11.898802+0000 mon.a (mon.0) 2409 : cluster [DBG] osdmap e1185: 8 total, 8 up, 8 in 2024-04-07T17:02:13.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:02:12 smithi100 bash[74942]: cluster 2024-04-07T17:02:12.537937+0000 mon.a (mon.0) 2410 : cluster [DBG] osdmap e1186: 8 total, 8 up, 8 in 2024-04-07T17:02:13.201 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:02:12 smithi100 bash[76202]: cluster 2024-04-07T17:02:11.898802+0000 mon.a (mon.0) 2409 : cluster [DBG] osdmap e1185: 8 total, 8 up, 8 in 2024-04-07T17:02:13.201 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:02:12 smithi100 bash[76202]: cluster 2024-04-07T17:02:12.537937+0000 mon.a (mon.0) 2410 : cluster [DBG] osdmap e1186: 8 total, 8 up, 8 in 2024-04-07T17:02:13.364 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:02:12 smithi144 bash[66654]: cluster 2024-04-07T17:02:11.898802+0000 mon.a (mon.0) 2409 : cluster [DBG] osdmap e1185: 8 total, 8 up, 8 in 2024-04-07T17:02:13.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:02:12 smithi144 bash[66654]: cluster 2024-04-07T17:02:12.537937+0000 mon.a (mon.0) 2410 : cluster [DBG] osdmap e1186: 8 total, 8 up, 8 in 2024-04-07T17:02:14.350 INFO:tasks.workunit.client.0.smithi100.stdout:migration_execute 2024-04-07T17:02:14.631 INFO:tasks.workunit.client.0.smithi100.stdout:migration_commit 2024-04-07T17:02:14.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:02:14 smithi144 bash[66654]: cluster 2024-04-07T17:02:13.542677+0000 mon.a (mon.0) 2411 : cluster [DBG] osdmap e1187: 8 total, 8 up, 8 in 2024-04-07T17:02:14.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:02:14 smithi144 bash[66654]: cluster 2024-04-07T17:02:13.783349+0000 mgr.y (mgr.25175) 1406 : cluster [DBG] pgmap v2168: 169 pgs: 169 active+clean; 505 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 3.2 MiB/s rd, 4.1 MiB/s wr, 172 op/s 2024-04-07T17:02:14.950 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:02:14 smithi100 bash[74942]: cluster 2024-04-07T17:02:13.542677+0000 mon.a (mon.0) 2411 : cluster [DBG] osdmap e1187: 8 total, 8 up, 8 in 2024-04-07T17:02:14.951 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:02:14 smithi100 bash[74942]: cluster 2024-04-07T17:02:13.783349+0000 mgr.y (mgr.25175) 1406 : cluster [DBG] pgmap v2168: 169 pgs: 169 active+clean; 505 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 3.2 MiB/s rd, 4.1 MiB/s wr, 172 op/s 2024-04-07T17:02:14.951 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:02:14 smithi100 bash[76202]: cluster 2024-04-07T17:02:13.542677+0000 mon.a (mon.0) 2411 : cluster [DBG] osdmap e1187: 8 total, 8 up, 8 in 2024-04-07T17:02:14.951 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:02:14 smithi100 bash[76202]: cluster 2024-04-07T17:02:13.783349+0000 mgr.y (mgr.25175) 1406 : cluster [DBG] pgmap v2168: 169 pgs: 169 active+clean; 505 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 3.2 MiB/s rd, 4.1 MiB/s wr, 172 op/s 2024-04-07T17:02:16.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:02:16 smithi144 bash[66654]: cluster 2024-04-07T17:02:15.575016+0000 mon.a (mon.0) 2412 : cluster [DBG] osdmap e1188: 8 total, 8 up, 8 in 2024-04-07T17:02:16.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:02:16 smithi144 bash[66654]: cluster 2024-04-07T17:02:15.784157+0000 mgr.y (mgr.25175) 1407 : cluster [DBG] pgmap v2170: 169 pgs: 169 active+clean; 513 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 9.2 MiB/s rd, 5.0 MiB/s wr, 233 op/s 2024-04-07T17:02:16.951 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:02:16 smithi100 bash[74942]: cluster 2024-04-07T17:02:15.575016+0000 mon.a (mon.0) 2412 : cluster [DBG] osdmap e1188: 8 total, 8 up, 8 in 2024-04-07T17:02:16.951 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:02:16 smithi100 bash[74942]: cluster 2024-04-07T17:02:15.784157+0000 mgr.y (mgr.25175) 1407 : cluster [DBG] pgmap v2170: 169 pgs: 169 active+clean; 513 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 9.2 MiB/s rd, 5.0 MiB/s wr, 233 op/s 2024-04-07T17:02:16.951 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:02:16 smithi100 bash[76202]: cluster 2024-04-07T17:02:15.575016+0000 mon.a (mon.0) 2412 : cluster [DBG] osdmap e1188: 8 total, 8 up, 8 in 2024-04-07T17:02:16.951 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:02:16 smithi100 bash[76202]: cluster 2024-04-07T17:02:15.784157+0000 mgr.y (mgr.25175) 1407 : cluster [DBG] pgmap v2170: 169 pgs: 169 active+clean; 513 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 9.2 MiB/s rd, 5.0 MiB/s wr, 233 op/s 2024-04-07T17:02:17.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:02:17 smithi144 bash[66654]: cluster 2024-04-07T17:02:16.588226+0000 mon.a (mon.0) 2413 : cluster [DBG] osdmap e1189: 8 total, 8 up, 8 in 2024-04-07T17:02:17.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:02:17 smithi144 bash[66654]: audit 2024-04-07T17:02:16.799922+0000 mon.a (mon.0) 2414 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:02:17.951 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:02:17 smithi100 bash[74942]: cluster 2024-04-07T17:02:16.588226+0000 mon.a (mon.0) 2413 : cluster [DBG] osdmap e1189: 8 total, 8 up, 8 in 2024-04-07T17:02:17.951 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:02:17 smithi100 bash[74942]: audit 2024-04-07T17:02:16.799922+0000 mon.a (mon.0) 2414 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:02:17.951 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:02:17 smithi100 bash[76202]: cluster 2024-04-07T17:02:16.588226+0000 mon.a (mon.0) 2413 : cluster [DBG] osdmap e1189: 8 total, 8 up, 8 in 2024-04-07T17:02:17.951 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:02:17 smithi100 bash[76202]: audit 2024-04-07T17:02:16.799922+0000 mon.a (mon.0) 2414 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:02:18.031 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestMigration.SnapsShrinkToZeroBeforePrepare (12624 ms) 2024-04-07T17:02:18.032 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestMigration.SnapsShrinkToZeroAfterPrepare 2024-04-07T17:02:18.124 INFO:tasks.workunit.client.0.smithi100.stdout:resize: 83886080 2024-04-07T17:02:18.155 INFO:tasks.workunit.client.0.smithi100.stdout:write: 1 0~8388609 2024-04-07T17:02:18.272 INFO:tasks.workunit.client.0.smithi100.stdout:snap_create: snap1 2024-04-07T17:02:18.701 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:02:18 smithi100 bash[76202]: cluster 2024-04-07T17:02:17.590665+0000 mon.a (mon.0) 2415 : cluster [DBG] osdmap e1190: 8 total, 8 up, 8 in 2024-04-07T17:02:18.701 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:02:18 smithi100 bash[76202]: cluster 2024-04-07T17:02:17.784982+0000 mgr.y (mgr.25175) 1408 : cluster [DBG] pgmap v2173: 169 pgs: 169 active+clean; 521 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 13 MiB/s rd, 3.8 MiB/s wr, 100 op/s 2024-04-07T17:02:18.701 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:02:18 smithi100 bash[74942]: cluster 2024-04-07T17:02:17.590665+0000 mon.a (mon.0) 2415 : cluster [DBG] osdmap e1190: 8 total, 8 up, 8 in 2024-04-07T17:02:18.701 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:02:18 smithi100 bash[74942]: cluster 2024-04-07T17:02:17.784982+0000 mgr.y (mgr.25175) 1408 : cluster [DBG] pgmap v2173: 169 pgs: 169 active+clean; 521 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 13 MiB/s rd, 3.8 MiB/s wr, 100 op/s 2024-04-07T17:02:18.701 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:02:18 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:02:18] "GET /metrics HTTP/1.1" 200 38921 "" "Prometheus/2.43.0" 2024-04-07T17:02:18.866 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:02:18 smithi144 bash[66654]: cluster 2024-04-07T17:02:17.590665+0000 mon.a (mon.0) 2415 : cluster [DBG] osdmap e1190: 8 total, 8 up, 8 in 2024-04-07T17:02:19.168 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:02:18 smithi144 bash[66654]: cluster 2024-04-07T17:02:17.784982+0000 mgr.y (mgr.25175) 1408 : cluster [DBG] pgmap v2173: 169 pgs: 169 active+clean; 521 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 13 MiB/s rd, 3.8 MiB/s wr, 100 op/s 2024-04-07T17:02:19.659 INFO:tasks.workunit.client.0.smithi100.stdout:write: 1 8388609~8388609 2024-04-07T17:02:19.704 INFO:tasks.workunit.client.0.smithi100.stdout:write: X 0~1000 2024-04-07T17:02:19.704 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 1010~1000 2024-04-07T17:02:19.705 INFO:tasks.workunit.client.0.smithi100.stdout:snap_create: snap2 2024-04-07T17:02:19.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:02:19 smithi144 bash[66654]: cluster 2024-04-07T17:02:18.612877+0000 mon.a (mon.0) 2416 : cluster [DBG] osdmap e1191: 8 total, 8 up, 8 in 2024-04-07T17:02:19.951 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:02:19 smithi100 bash[74942]: cluster 2024-04-07T17:02:18.612877+0000 mon.a (mon.0) 2416 : cluster [DBG] osdmap e1191: 8 total, 8 up, 8 in 2024-04-07T17:02:19.951 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:02:19 smithi100 bash[76202]: cluster 2024-04-07T17:02:18.612877+0000 mon.a (mon.0) 2416 : cluster [DBG] osdmap e1191: 8 total, 8 up, 8 in 2024-04-07T17:02:20.951 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:02:20 smithi100 bash[74942]: cluster 2024-04-07T17:02:19.622343+0000 mon.a (mon.0) 2417 : cluster [DBG] osdmap e1192: 8 total, 8 up, 8 in 2024-04-07T17:02:20.951 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:02:20 smithi100 bash[74942]: cluster 2024-04-07T17:02:19.785798+0000 mgr.y (mgr.25175) 1409 : cluster [DBG] pgmap v2176: 169 pgs: 169 active+clean; 529 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 35 MiB/s rd, 9.0 MiB/s wr, 380 op/s 2024-04-07T17:02:20.951 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:02:20 smithi100 bash[76202]: cluster 2024-04-07T17:02:19.622343+0000 mon.a (mon.0) 2417 : cluster [DBG] osdmap e1192: 8 total, 8 up, 8 in 2024-04-07T17:02:20.951 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:02:20 smithi100 bash[76202]: cluster 2024-04-07T17:02:19.785798+0000 mgr.y (mgr.25175) 1409 : cluster [DBG] pgmap v2176: 169 pgs: 169 active+clean; 529 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 35 MiB/s rd, 9.0 MiB/s wr, 380 op/s 2024-04-07T17:02:21.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:02:20 smithi144 bash[66654]: cluster 2024-04-07T17:02:19.622343+0000 mon.a (mon.0) 2417 : cluster [DBG] osdmap e1192: 8 total, 8 up, 8 in 2024-04-07T17:02:21.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:02:20 smithi144 bash[66654]: cluster 2024-04-07T17:02:19.785798+0000 mgr.y (mgr.25175) 1409 : cluster [DBG] pgmap v2176: 169 pgs: 169 active+clean; 529 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 35 MiB/s rd, 9.0 MiB/s wr, 380 op/s 2024-04-07T17:02:21.660 INFO:tasks.workunit.client.0.smithi100.stdout:write: X 8388609~1000 2024-04-07T17:02:21.678 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 16777228~1000 2024-04-07T17:02:21.678 INFO:tasks.workunit.client.0.smithi100.stdout:resize: 167772160 2024-04-07T17:02:21.707 INFO:tasks.workunit.client.0.smithi100.stdout:write: 2 83886079~8388609 2024-04-07T17:02:21.764 INFO:tasks.workunit.client.0.smithi100.stdout:snap_create: snap3 2024-04-07T17:02:21.950 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:02:21 smithi100 bash[74942]: cluster 2024-04-07T17:02:20.633897+0000 mon.a (mon.0) 2418 : cluster [DBG] osdmap e1193: 8 total, 8 up, 8 in 2024-04-07T17:02:21.951 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:02:21 smithi100 bash[76202]: cluster 2024-04-07T17:02:20.633897+0000 mon.a (mon.0) 2418 : cluster [DBG] osdmap e1193: 8 total, 8 up, 8 in 2024-04-07T17:02:22.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:02:21 smithi144 bash[66654]: cluster 2024-04-07T17:02:20.633897+0000 mon.a (mon.0) 2418 : cluster [DBG] osdmap e1193: 8 total, 8 up, 8 in 2024-04-07T17:02:22.615 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:02:22 smithi144 bash[95233]: ts=2024-04-07T17:02:22.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:02:22.950 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:02:22 smithi100 bash[74942]: cluster 2024-04-07T17:02:21.634417+0000 mon.a (mon.0) 2419 : cluster [DBG] osdmap e1194: 8 total, 8 up, 8 in 2024-04-07T17:02:22.951 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:02:22 smithi100 bash[74942]: cluster 2024-04-07T17:02:21.786737+0000 mgr.y (mgr.25175) 1410 : cluster [DBG] pgmap v2179: 169 pgs: 169 active+clean; 529 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 35 MiB/s rd, 7.0 MiB/s wr, 345 op/s 2024-04-07T17:02:22.951 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:02:22 smithi100 bash[76202]: cluster 2024-04-07T17:02:21.634417+0000 mon.a (mon.0) 2419 : cluster [DBG] osdmap e1194: 8 total, 8 up, 8 in 2024-04-07T17:02:22.951 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:02:22 smithi100 bash[76202]: cluster 2024-04-07T17:02:21.786737+0000 mgr.y (mgr.25175) 1410 : cluster [DBG] pgmap v2179: 169 pgs: 169 active+clean; 529 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 35 MiB/s rd, 7.0 MiB/s wr, 345 op/s 2024-04-07T17:02:23.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:02:22 smithi144 bash[66654]: cluster 2024-04-07T17:02:21.634417+0000 mon.a (mon.0) 2419 : cluster [DBG] osdmap e1194: 8 total, 8 up, 8 in 2024-04-07T17:02:23.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:02:22 smithi144 bash[66654]: cluster 2024-04-07T17:02:21.786737+0000 mgr.y (mgr.25175) 1410 : cluster [DBG] pgmap v2179: 169 pgs: 169 active+clean; 529 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 35 MiB/s rd, 7.0 MiB/s wr, 345 op/s 2024-04-07T17:02:23.692 INFO:tasks.workunit.client.0.smithi100.stdout:resize: 83886080 2024-04-07T17:02:23.759 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 83886079~1 2024-04-07T17:02:23.782 INFO:tasks.workunit.client.0.smithi100.stdout:migration_prepare 2024-04-07T17:02:23.951 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:02:23 smithi100 bash[74942]: cluster 2024-04-07T17:02:22.659385+0000 mon.a (mon.0) 2420 : cluster [DBG] osdmap e1195: 8 total, 8 up, 8 in 2024-04-07T17:02:23.951 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:02:23 smithi100 bash[76202]: cluster 2024-04-07T17:02:22.659385+0000 mon.a (mon.0) 2420 : cluster [DBG] osdmap e1195: 8 total, 8 up, 8 in 2024-04-07T17:02:24.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:02:23 smithi144 bash[66654]: cluster 2024-04-07T17:02:22.659385+0000 mon.a (mon.0) 2420 : cluster [DBG] osdmap e1195: 8 total, 8 up, 8 in 2024-04-07T17:02:24.951 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:02:24 smithi100 bash[74942]: cluster 2024-04-07T17:02:23.664186+0000 mon.a (mon.0) 2421 : cluster [DBG] osdmap e1196: 8 total, 8 up, 8 in 2024-04-07T17:02:24.951 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:02:24 smithi100 bash[74942]: cluster 2024-04-07T17:02:23.787746+0000 mgr.y (mgr.25175) 1411 : cluster [DBG] pgmap v2182: 169 pgs: 169 active+clean; 549 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 10 MiB/s rd, 4.0 MiB/s wr, 121 op/s 2024-04-07T17:02:24.951 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:02:24 smithi100 bash[76202]: cluster 2024-04-07T17:02:23.664186+0000 mon.a (mon.0) 2421 : cluster [DBG] osdmap e1196: 8 total, 8 up, 8 in 2024-04-07T17:02:24.951 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:02:24 smithi100 bash[76202]: cluster 2024-04-07T17:02:23.787746+0000 mgr.y (mgr.25175) 1411 : cluster [DBG] pgmap v2182: 169 pgs: 169 active+clean; 549 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 10 MiB/s rd, 4.0 MiB/s wr, 121 op/s 2024-04-07T17:02:25.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:02:24 smithi144 bash[66654]: cluster 2024-04-07T17:02:23.664186+0000 mon.a (mon.0) 2421 : cluster [DBG] osdmap e1196: 8 total, 8 up, 8 in 2024-04-07T17:02:25.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:02:24 smithi144 bash[66654]: cluster 2024-04-07T17:02:23.787746+0000 mgr.y (mgr.25175) 1411 : cluster [DBG] pgmap v2182: 169 pgs: 169 active+clean; 549 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 10 MiB/s rd, 4.0 MiB/s wr, 121 op/s 2024-04-07T17:02:25.950 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:02:25 smithi100 bash[74942]: cluster 2024-04-07T17:02:24.688269+0000 mon.a (mon.0) 2422 : cluster [DBG] osdmap e1197: 8 total, 8 up, 8 in 2024-04-07T17:02:25.951 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:02:25 smithi100 bash[76202]: cluster 2024-04-07T17:02:24.688269+0000 mon.a (mon.0) 2422 : cluster [DBG] osdmap e1197: 8 total, 8 up, 8 in 2024-04-07T17:02:26.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:02:25 smithi144 bash[66654]: cluster 2024-04-07T17:02:24.688269+0000 mon.a (mon.0) 2422 : cluster [DBG] osdmap e1197: 8 total, 8 up, 8 in 2024-04-07T17:02:27.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:02:26 smithi144 bash[66654]: cluster 2024-04-07T17:02:25.687991+0000 mon.a (mon.0) 2423 : cluster [DBG] osdmap e1198: 8 total, 8 up, 8 in 2024-04-07T17:02:27.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:02:26 smithi144 bash[66654]: cluster 2024-04-07T17:02:25.788545+0000 mgr.y (mgr.25175) 1412 : cluster [DBG] pgmap v2185: 169 pgs: 169 active+clean; 565 MiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 5.1 MiB/s rd, 8.0 MiB/s wr, 161 op/s 2024-04-07T17:02:27.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:02:26 smithi100 bash[74942]: cluster 2024-04-07T17:02:25.687991+0000 mon.a (mon.0) 2423 : cluster [DBG] osdmap e1198: 8 total, 8 up, 8 in 2024-04-07T17:02:27.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:02:26 smithi100 bash[74942]: cluster 2024-04-07T17:02:25.788545+0000 mgr.y (mgr.25175) 1412 : cluster [DBG] pgmap v2185: 169 pgs: 169 active+clean; 565 MiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 5.1 MiB/s rd, 8.0 MiB/s wr, 161 op/s 2024-04-07T17:02:27.201 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:02:26 smithi100 bash[76202]: cluster 2024-04-07T17:02:25.687991+0000 mon.a (mon.0) 2423 : cluster [DBG] osdmap e1198: 8 total, 8 up, 8 in 2024-04-07T17:02:27.201 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:02:26 smithi100 bash[76202]: cluster 2024-04-07T17:02:25.788545+0000 mgr.y (mgr.25175) 1412 : cluster [DBG] pgmap v2185: 169 pgs: 169 active+clean; 565 MiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 5.1 MiB/s rd, 8.0 MiB/s wr, 161 op/s 2024-04-07T17:02:27.748 INFO:tasks.workunit.client.0.smithi100.stdout:resize: 0 2024-04-07T17:02:27.888 INFO:tasks.workunit.client.0.smithi100.stdout:migration_execute 2024-04-07T17:02:28.090 INFO:tasks.workunit.client.0.smithi100.stdout:migration_commit 2024-04-07T17:02:28.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:02:27 smithi144 bash[66654]: cluster 2024-04-07T17:02:26.714865+0000 mon.a (mon.0) 2424 : cluster [DBG] osdmap e1199: 8 total, 8 up, 8 in 2024-04-07T17:02:28.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:02:27 smithi144 bash[66654]: cluster 2024-04-07T17:02:27.547090+0000 mon.a (mon.0) 2425 : cluster [DBG] osdmap e1200: 8 total, 8 up, 8 in 2024-04-07T17:02:28.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:02:27 smithi100 bash[74942]: cluster 2024-04-07T17:02:26.714865+0000 mon.a (mon.0) 2424 : cluster [DBG] osdmap e1199: 8 total, 8 up, 8 in 2024-04-07T17:02:28.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:02:27 smithi100 bash[74942]: cluster 2024-04-07T17:02:27.547090+0000 mon.a (mon.0) 2425 : cluster [DBG] osdmap e1200: 8 total, 8 up, 8 in 2024-04-07T17:02:28.201 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:02:27 smithi100 bash[76202]: cluster 2024-04-07T17:02:26.714865+0000 mon.a (mon.0) 2424 : cluster [DBG] osdmap e1199: 8 total, 8 up, 8 in 2024-04-07T17:02:28.201 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:02:27 smithi100 bash[76202]: cluster 2024-04-07T17:02:27.547090+0000 mon.a (mon.0) 2425 : cluster [DBG] osdmap e1200: 8 total, 8 up, 8 in 2024-04-07T17:02:28.701 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:02:28 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:02:28] "GET /metrics HTTP/1.1" 200 38921 "" "Prometheus/2.43.0" 2024-04-07T17:02:29.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:02:28 smithi144 bash[66654]: cluster 2024-04-07T17:02:27.789668+0000 mgr.y (mgr.25175) 1413 : cluster [DBG] pgmap v2188: 169 pgs: 169 active+clean; 565 MiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 96 KiB/s rd, 4.0 MiB/s wr, 134 op/s 2024-04-07T17:02:29.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:02:28 smithi100 bash[74942]: cluster 2024-04-07T17:02:27.789668+0000 mgr.y (mgr.25175) 1413 : cluster [DBG] pgmap v2188: 169 pgs: 169 active+clean; 565 MiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 96 KiB/s rd, 4.0 MiB/s wr, 134 op/s 2024-04-07T17:02:29.201 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:02:28 smithi100 bash[76202]: cluster 2024-04-07T17:02:27.789668+0000 mgr.y (mgr.25175) 1413 : cluster [DBG] pgmap v2188: 169 pgs: 169 active+clean; 565 MiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 96 KiB/s rd, 4.0 MiB/s wr, 134 op/s 2024-04-07T17:02:30.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:02:29 smithi144 bash[66654]: cluster 2024-04-07T17:02:28.712016+0000 mon.a (mon.0) 2426 : cluster [DBG] osdmap e1201: 8 total, 8 up, 8 in 2024-04-07T17:02:30.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:02:29 smithi100 bash[74942]: cluster 2024-04-07T17:02:28.712016+0000 mon.a (mon.0) 2426 : cluster [DBG] osdmap e1201: 8 total, 8 up, 8 in 2024-04-07T17:02:30.201 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:02:29 smithi100 bash[76202]: cluster 2024-04-07T17:02:28.712016+0000 mon.a (mon.0) 2426 : cluster [DBG] osdmap e1201: 8 total, 8 up, 8 in 2024-04-07T17:02:31.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:02:30 smithi144 bash[66654]: cluster 2024-04-07T17:02:29.733925+0000 mon.a (mon.0) 2427 : cluster [DBG] osdmap e1202: 8 total, 8 up, 8 in 2024-04-07T17:02:31.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:02:30 smithi144 bash[66654]: cluster 2024-04-07T17:02:29.790319+0000 mgr.y (mgr.25175) 1414 : cluster [DBG] pgmap v2191: 169 pgs: 169 active+clean; 585 MiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 26 MiB/s rd, 5.0 MiB/s wr, 172 op/s 2024-04-07T17:02:31.201 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:02:30 smithi100 bash[76202]: cluster 2024-04-07T17:02:29.733925+0000 mon.a (mon.0) 2427 : cluster [DBG] osdmap e1202: 8 total, 8 up, 8 in 2024-04-07T17:02:31.201 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:02:30 smithi100 bash[76202]: cluster 2024-04-07T17:02:29.790319+0000 mgr.y (mgr.25175) 1414 : cluster [DBG] pgmap v2191: 169 pgs: 169 active+clean; 585 MiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 26 MiB/s rd, 5.0 MiB/s wr, 172 op/s 2024-04-07T17:02:31.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:02:30 smithi100 bash[74942]: cluster 2024-04-07T17:02:29.733925+0000 mon.a (mon.0) 2427 : cluster [DBG] osdmap e1202: 8 total, 8 up, 8 in 2024-04-07T17:02:31.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:02:30 smithi100 bash[74942]: cluster 2024-04-07T17:02:29.790319+0000 mgr.y (mgr.25175) 1414 : cluster [DBG] pgmap v2191: 169 pgs: 169 active+clean; 585 MiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 26 MiB/s rd, 5.0 MiB/s wr, 172 op/s 2024-04-07T17:02:31.266 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestMigration.SnapsShrinkToZeroAfterPrepare (13234 ms) 2024-04-07T17:02:31.266 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestMigration.SnapsExpandAfterPrepare 2024-04-07T17:02:31.351 INFO:tasks.workunit.client.0.smithi100.stdout:resize: 83886080 2024-04-07T17:02:31.381 INFO:tasks.workunit.client.0.smithi100.stdout:write: 1 0~8388609 2024-04-07T17:02:31.493 INFO:tasks.workunit.client.0.smithi100.stdout:snap_create: snap1 2024-04-07T17:02:32.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:02:31 smithi144 bash[66654]: cluster 2024-04-07T17:02:30.747539+0000 mon.a (mon.0) 2428 : cluster [DBG] osdmap e1203: 8 total, 8 up, 8 in 2024-04-07T17:02:32.200 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:02:31 smithi100 bash[74942]: cluster 2024-04-07T17:02:30.747539+0000 mon.a (mon.0) 2428 : cluster [DBG] osdmap e1203: 8 total, 8 up, 8 in 2024-04-07T17:02:32.201 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:02:31 smithi100 bash[76202]: cluster 2024-04-07T17:02:30.747539+0000 mon.a (mon.0) 2428 : cluster [DBG] osdmap e1203: 8 total, 8 up, 8 in 2024-04-07T17:02:32.615 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:02:32 smithi144 bash[95233]: ts=2024-04-07T17:02:32.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:02:32.804 INFO:tasks.workunit.client.0.smithi100.stdout:write: 1 8388609~8388609 2024-04-07T17:02:32.836 INFO:tasks.workunit.client.0.smithi100.stdout:write: X 0~1000 2024-04-07T17:02:32.836 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 1010~1000 2024-04-07T17:02:32.836 INFO:tasks.workunit.client.0.smithi100.stdout:snap_create: snap2 2024-04-07T17:02:33.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:02:32 smithi144 bash[66654]: cluster 2024-04-07T17:02:31.778048+0000 mon.a (mon.0) 2429 : cluster [DBG] osdmap e1204: 8 total, 8 up, 8 in 2024-04-07T17:02:33.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:02:32 smithi144 bash[66654]: cluster 2024-04-07T17:02:31.791206+0000 mgr.y (mgr.25175) 1415 : cluster [DBG] pgmap v2194: 169 pgs: 169 active+clean; 581 MiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 29 MiB/s rd, 5.0 MiB/s wr, 245 op/s 2024-04-07T17:02:33.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:02:32 smithi144 bash[66654]: audit 2024-04-07T17:02:31.800371+0000 mon.a (mon.0) 2430 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:02:33.201 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:02:32 smithi100 bash[76202]: cluster 2024-04-07T17:02:31.778048+0000 mon.a (mon.0) 2429 : cluster [DBG] osdmap e1204: 8 total, 8 up, 8 in 2024-04-07T17:02:33.201 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:02:32 smithi100 bash[76202]: cluster 2024-04-07T17:02:31.791206+0000 mgr.y (mgr.25175) 1415 : cluster [DBG] pgmap v2194: 169 pgs: 169 active+clean; 581 MiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 29 MiB/s rd, 5.0 MiB/s wr, 245 op/s 2024-04-07T17:02:33.201 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:02:32 smithi100 bash[76202]: audit 2024-04-07T17:02:31.800371+0000 mon.a (mon.0) 2430 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:02:33.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:02:32 smithi100 bash[74942]: cluster 2024-04-07T17:02:31.778048+0000 mon.a (mon.0) 2429 : cluster [DBG] osdmap e1204: 8 total, 8 up, 8 in 2024-04-07T17:02:33.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:02:32 smithi100 bash[74942]: cluster 2024-04-07T17:02:31.791206+0000 mgr.y (mgr.25175) 1415 : cluster [DBG] pgmap v2194: 169 pgs: 169 active+clean; 581 MiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 29 MiB/s rd, 5.0 MiB/s wr, 245 op/s 2024-04-07T17:02:33.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:02:32 smithi100 bash[74942]: audit 2024-04-07T17:02:31.800371+0000 mon.a (mon.0) 2430 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:02:34.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:02:33 smithi144 bash[66654]: cluster 2024-04-07T17:02:32.781095+0000 mon.a (mon.0) 2431 : cluster [DBG] osdmap e1205: 8 total, 8 up, 8 in 2024-04-07T17:02:34.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:02:33 smithi100 bash[74942]: cluster 2024-04-07T17:02:32.781095+0000 mon.a (mon.0) 2431 : cluster [DBG] osdmap e1205: 8 total, 8 up, 8 in 2024-04-07T17:02:34.201 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:02:33 smithi100 bash[76202]: cluster 2024-04-07T17:02:32.781095+0000 mon.a (mon.0) 2431 : cluster [DBG] osdmap e1205: 8 total, 8 up, 8 in 2024-04-07T17:02:34.838 INFO:tasks.workunit.client.0.smithi100.stdout:write: X 8388609~1000 2024-04-07T17:02:34.855 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 16777228~1000 2024-04-07T17:02:34.855 INFO:tasks.workunit.client.0.smithi100.stdout:resize: 167772160 2024-04-07T17:02:34.878 INFO:tasks.workunit.client.0.smithi100.stdout:write: 2 83886079~8388609 2024-04-07T17:02:34.934 INFO:tasks.workunit.client.0.smithi100.stdout:snap_create: snap3 2024-04-07T17:02:35.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:02:34 smithi144 bash[66654]: cluster 2024-04-07T17:02:33.792455+0000 mgr.y (mgr.25175) 1416 : cluster [DBG] pgmap v2196: 169 pgs: 169 active+clean; 581 MiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 22 MiB/s rd, 3.0 MiB/s wr, 424 op/s 2024-04-07T17:02:35.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:02:34 smithi144 bash[66654]: cluster 2024-04-07T17:02:33.804939+0000 mon.a (mon.0) 2432 : cluster [DBG] osdmap e1206: 8 total, 8 up, 8 in 2024-04-07T17:02:35.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:02:34 smithi100 bash[74942]: cluster 2024-04-07T17:02:33.792455+0000 mgr.y (mgr.25175) 1416 : cluster [DBG] pgmap v2196: 169 pgs: 169 active+clean; 581 MiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 22 MiB/s rd, 3.0 MiB/s wr, 424 op/s 2024-04-07T17:02:35.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:02:34 smithi100 bash[74942]: cluster 2024-04-07T17:02:33.804939+0000 mon.a (mon.0) 2432 : cluster [DBG] osdmap e1206: 8 total, 8 up, 8 in 2024-04-07T17:02:35.201 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:02:34 smithi100 bash[76202]: cluster 2024-04-07T17:02:33.792455+0000 mgr.y (mgr.25175) 1416 : cluster [DBG] pgmap v2196: 169 pgs: 169 active+clean; 581 MiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 22 MiB/s rd, 3.0 MiB/s wr, 424 op/s 2024-04-07T17:02:35.201 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:02:34 smithi100 bash[76202]: cluster 2024-04-07T17:02:33.804939+0000 mon.a (mon.0) 2432 : cluster [DBG] osdmap e1206: 8 total, 8 up, 8 in 2024-04-07T17:02:36.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:02:35 smithi144 bash[66654]: cluster 2024-04-07T17:02:34.806849+0000 mon.a (mon.0) 2433 : cluster [DBG] osdmap e1207: 8 total, 8 up, 8 in 2024-04-07T17:02:36.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:02:35 smithi144 bash[66654]: cluster 2024-04-07T17:02:35.793519+0000 mgr.y (mgr.25175) 1417 : cluster [DBG] pgmap v2199: 169 pgs: 169 active+clean; 593 MiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 37 MiB/s rd, 8.0 MiB/s wr, 307 op/s 2024-04-07T17:02:36.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:02:35 smithi100 bash[74942]: cluster 2024-04-07T17:02:34.806849+0000 mon.a (mon.0) 2433 : cluster [DBG] osdmap e1207: 8 total, 8 up, 8 in 2024-04-07T17:02:36.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:02:35 smithi100 bash[74942]: cluster 2024-04-07T17:02:35.793519+0000 mgr.y (mgr.25175) 1417 : cluster [DBG] pgmap v2199: 169 pgs: 169 active+clean; 593 MiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 37 MiB/s rd, 8.0 MiB/s wr, 307 op/s 2024-04-07T17:02:36.201 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:02:35 smithi100 bash[76202]: cluster 2024-04-07T17:02:34.806849+0000 mon.a (mon.0) 2433 : cluster [DBG] osdmap e1207: 8 total, 8 up, 8 in 2024-04-07T17:02:36.201 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:02:35 smithi100 bash[76202]: cluster 2024-04-07T17:02:35.793519+0000 mgr.y (mgr.25175) 1417 : cluster [DBG] pgmap v2199: 169 pgs: 169 active+clean; 593 MiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 37 MiB/s rd, 8.0 MiB/s wr, 307 op/s 2024-04-07T17:02:36.851 INFO:tasks.workunit.client.0.smithi100.stdout:resize: 83886080 2024-04-07T17:02:36.918 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 83886079~1 2024-04-07T17:02:36.942 INFO:tasks.workunit.client.0.smithi100.stdout:migration_prepare 2024-04-07T17:02:37.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:02:36 smithi144 bash[66654]: cluster 2024-04-07T17:02:35.827303+0000 mon.a (mon.0) 2434 : cluster [DBG] osdmap e1208: 8 total, 8 up, 8 in 2024-04-07T17:02:37.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:02:36 smithi100 bash[74942]: cluster 2024-04-07T17:02:35.827303+0000 mon.a (mon.0) 2434 : cluster [DBG] osdmap e1208: 8 total, 8 up, 8 in 2024-04-07T17:02:37.201 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:02:36 smithi100 bash[76202]: cluster 2024-04-07T17:02:35.827303+0000 mon.a (mon.0) 2434 : cluster [DBG] osdmap e1208: 8 total, 8 up, 8 in 2024-04-07T17:02:38.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:02:37 smithi144 bash[66654]: cluster 2024-04-07T17:02:36.829871+0000 mon.a (mon.0) 2435 : cluster [DBG] osdmap e1209: 8 total, 8 up, 8 in 2024-04-07T17:02:38.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:02:37 smithi144 bash[66654]: cluster 2024-04-07T17:02:37.550569+0000 mon.a (mon.0) 2436 : cluster [DBG] osdmap e1210: 8 total, 8 up, 8 in 2024-04-07T17:02:38.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:02:37 smithi144 bash[66654]: cluster 2024-04-07T17:02:37.794299+0000 mgr.y (mgr.25175) 1418 : cluster [DBG] pgmap v2203: 169 pgs: 169 active+clean; 601 MiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 18 MiB/s rd, 7.0 MiB/s wr, 148 op/s 2024-04-07T17:02:38.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:02:37 smithi100 bash[74942]: cluster 2024-04-07T17:02:36.829871+0000 mon.a (mon.0) 2435 : cluster [DBG] osdmap e1209: 8 total, 8 up, 8 in 2024-04-07T17:02:38.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:02:37 smithi100 bash[74942]: cluster 2024-04-07T17:02:37.550569+0000 mon.a (mon.0) 2436 : cluster [DBG] osdmap e1210: 8 total, 8 up, 8 in 2024-04-07T17:02:38.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:02:37 smithi100 bash[74942]: cluster 2024-04-07T17:02:37.794299+0000 mgr.y (mgr.25175) 1418 : cluster [DBG] pgmap v2203: 169 pgs: 169 active+clean; 601 MiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 18 MiB/s rd, 7.0 MiB/s wr, 148 op/s 2024-04-07T17:02:38.201 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:02:37 smithi100 bash[76202]: cluster 2024-04-07T17:02:36.829871+0000 mon.a (mon.0) 2435 : cluster [DBG] osdmap e1209: 8 total, 8 up, 8 in 2024-04-07T17:02:38.201 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:02:37 smithi100 bash[76202]: cluster 2024-04-07T17:02:37.550569+0000 mon.a (mon.0) 2436 : cluster [DBG] osdmap e1210: 8 total, 8 up, 8 in 2024-04-07T17:02:38.201 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:02:37 smithi100 bash[76202]: cluster 2024-04-07T17:02:37.794299+0000 mgr.y (mgr.25175) 1418 : cluster [DBG] pgmap v2203: 169 pgs: 169 active+clean; 601 MiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 18 MiB/s rd, 7.0 MiB/s wr, 148 op/s 2024-04-07T17:02:38.701 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:02:38 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:02:38] "GET /metrics HTTP/1.1" 200 38924 "" "Prometheus/2.43.0" 2024-04-07T17:02:39.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:02:39 smithi144 bash[66654]: cluster 2024-04-07T17:02:38.557457+0000 mon.a (mon.0) 2437 : cluster [DBG] osdmap e1211: 8 total, 8 up, 8 in 2024-04-07T17:02:39.951 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:02:39 smithi100 bash[74942]: cluster 2024-04-07T17:02:38.557457+0000 mon.a (mon.0) 2437 : cluster [DBG] osdmap e1211: 8 total, 8 up, 8 in 2024-04-07T17:02:39.951 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:02:39 smithi100 bash[76202]: cluster 2024-04-07T17:02:38.557457+0000 mon.a (mon.0) 2437 : cluster [DBG] osdmap e1211: 8 total, 8 up, 8 in 2024-04-07T17:02:40.535 INFO:tasks.workunit.client.0.smithi100.stdout:resize: 167772160 2024-04-07T17:02:40.566 INFO:tasks.workunit.client.0.smithi100.stdout:write: * 83886080~1000 2024-04-07T17:02:40.633 INFO:tasks.workunit.client.0.smithi100.stdout:migration_execute 2024-04-07T17:02:40.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:02:40 smithi144 bash[66654]: cluster 2024-04-07T17:02:39.562856+0000 mon.a (mon.0) 2438 : cluster [DBG] osdmap e1212: 8 total, 8 up, 8 in 2024-04-07T17:02:40.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:02:40 smithi144 bash[66654]: cluster 2024-04-07T17:02:39.795169+0000 mgr.y (mgr.25175) 1419 : cluster [DBG] pgmap v2206: 169 pgs: 169 active+clean; 621 MiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 119 KiB/s rd, 5.1 MiB/s wr, 168 op/s 2024-04-07T17:02:40.906 INFO:tasks.workunit.client.0.smithi100.stdout:migration_commit 2024-04-07T17:02:40.950 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:02:40 smithi100 bash[76202]: cluster 2024-04-07T17:02:39.562856+0000 mon.a (mon.0) 2438 : cluster [DBG] osdmap e1212: 8 total, 8 up, 8 in 2024-04-07T17:02:40.951 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:02:40 smithi100 bash[76202]: cluster 2024-04-07T17:02:39.795169+0000 mgr.y (mgr.25175) 1419 : cluster [DBG] pgmap v2206: 169 pgs: 169 active+clean; 621 MiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 119 KiB/s rd, 5.1 MiB/s wr, 168 op/s 2024-04-07T17:02:40.951 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:02:40 smithi100 bash[74942]: cluster 2024-04-07T17:02:39.562856+0000 mon.a (mon.0) 2438 : cluster [DBG] osdmap e1212: 8 total, 8 up, 8 in 2024-04-07T17:02:40.951 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:02:40 smithi100 bash[74942]: cluster 2024-04-07T17:02:39.795169+0000 mgr.y (mgr.25175) 1419 : cluster [DBG] pgmap v2206: 169 pgs: 169 active+clean; 621 MiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 119 KiB/s rd, 5.1 MiB/s wr, 168 op/s 2024-04-07T17:02:42.589 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:02:42 smithi144 bash[95233]: ts=2024-04-07T17:02:42.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:02:42.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:02:42 smithi144 bash[66654]: cluster 2024-04-07T17:02:41.585739+0000 mon.a (mon.0) 2439 : cluster [DBG] osdmap e1213: 8 total, 8 up, 8 in 2024-04-07T17:02:42.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:02:42 smithi144 bash[66654]: cluster 2024-04-07T17:02:41.796063+0000 mgr.y (mgr.25175) 1420 : cluster [DBG] pgmap v2208: 169 pgs: 169 active+clean; 621 MiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 1.9 MiB/s rd, 2.8 MiB/s wr, 59 op/s 2024-04-07T17:02:42.951 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:02:42 smithi100 bash[74942]: cluster 2024-04-07T17:02:41.585739+0000 mon.a (mon.0) 2439 : cluster [DBG] osdmap e1213: 8 total, 8 up, 8 in 2024-04-07T17:02:42.951 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:02:42 smithi100 bash[74942]: cluster 2024-04-07T17:02:41.796063+0000 mgr.y (mgr.25175) 1420 : cluster [DBG] pgmap v2208: 169 pgs: 169 active+clean; 621 MiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 1.9 MiB/s rd, 2.8 MiB/s wr, 59 op/s 2024-04-07T17:02:42.951 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:02:42 smithi100 bash[76202]: cluster 2024-04-07T17:02:41.585739+0000 mon.a (mon.0) 2439 : cluster [DBG] osdmap e1213: 8 total, 8 up, 8 in 2024-04-07T17:02:42.951 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:02:42 smithi100 bash[76202]: cluster 2024-04-07T17:02:41.796063+0000 mgr.y (mgr.25175) 1420 : cluster [DBG] pgmap v2208: 169 pgs: 169 active+clean; 621 MiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 1.9 MiB/s rd, 2.8 MiB/s wr, 59 op/s 2024-04-07T17:02:43.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:02:43 smithi144 bash[66654]: cluster 2024-04-07T17:02:42.600609+0000 mon.a (mon.0) 2440 : cluster [DBG] osdmap e1214: 8 total, 8 up, 8 in 2024-04-07T17:02:43.951 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:02:43 smithi100 bash[74942]: cluster 2024-04-07T17:02:42.600609+0000 mon.a (mon.0) 2440 : cluster [DBG] osdmap e1214: 8 total, 8 up, 8 in 2024-04-07T17:02:43.951 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:02:43 smithi100 bash[76202]: cluster 2024-04-07T17:02:42.600609+0000 mon.a (mon.0) 2440 : cluster [DBG] osdmap e1214: 8 total, 8 up, 8 in 2024-04-07T17:02:44.360 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestMigration.SnapsExpandAfterPrepare (13094 ms) 2024-04-07T17:02:44.360 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestMigration.SnapsExpandAfterPrepare2 2024-04-07T17:02:44.448 INFO:tasks.workunit.client.0.smithi100.stdout:resize: 83886080 2024-04-07T17:02:44.478 INFO:tasks.workunit.client.0.smithi100.stdout:write: X 41943040~10 2024-04-07T17:02:44.503 INFO:tasks.workunit.client.0.smithi100.stdout:snap_create: snap1 2024-04-07T17:02:44.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:02:44 smithi144 bash[66654]: cluster 2024-04-07T17:02:43.599714+0000 mon.a (mon.0) 2441 : cluster [DBG] osdmap e1215: 8 total, 8 up, 8 in 2024-04-07T17:02:44.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:02:44 smithi144 bash[66654]: cluster 2024-04-07T17:02:43.796673+0000 mgr.y (mgr.25175) 1421 : cluster [DBG] pgmap v2211: 169 pgs: 169 active+clean; 633 MiB data, 4.7 GiB used, 711 GiB / 715 GiB avail; 14 MiB/s rd, 2.8 MiB/s wr, 245 op/s 2024-04-07T17:02:44.951 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:02:44 smithi100 bash[74942]: cluster 2024-04-07T17:02:43.599714+0000 mon.a (mon.0) 2441 : cluster [DBG] osdmap e1215: 8 total, 8 up, 8 in 2024-04-07T17:02:44.951 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:02:44 smithi100 bash[74942]: cluster 2024-04-07T17:02:43.796673+0000 mgr.y (mgr.25175) 1421 : cluster [DBG] pgmap v2211: 169 pgs: 169 active+clean; 633 MiB data, 4.7 GiB used, 711 GiB / 715 GiB avail; 14 MiB/s rd, 2.8 MiB/s wr, 245 op/s 2024-04-07T17:02:44.951 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:02:44 smithi100 bash[76202]: cluster 2024-04-07T17:02:43.599714+0000 mon.a (mon.0) 2441 : cluster [DBG] osdmap e1215: 8 total, 8 up, 8 in 2024-04-07T17:02:44.951 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:02:44 smithi100 bash[76202]: cluster 2024-04-07T17:02:43.796673+0000 mgr.y (mgr.25175) 1421 : cluster [DBG] pgmap v2211: 169 pgs: 169 active+clean; 633 MiB data, 4.7 GiB used, 711 GiB / 715 GiB avail; 14 MiB/s rd, 2.8 MiB/s wr, 245 op/s 2024-04-07T17:02:45.650 INFO:tasks.workunit.client.0.smithi100.stdout:resize: 41943040 2024-04-07T17:02:45.713 INFO:tasks.workunit.client.0.smithi100.stdout:migration_prepare 2024-04-07T17:02:45.951 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:02:45 smithi100 bash[74942]: cluster 2024-04-07T17:02:44.624019+0000 mon.a (mon.0) 2442 : cluster [DBG] osdmap e1216: 8 total, 8 up, 8 in 2024-04-07T17:02:45.951 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:02:45 smithi100 bash[76202]: cluster 2024-04-07T17:02:44.624019+0000 mon.a (mon.0) 2442 : cluster [DBG] osdmap e1216: 8 total, 8 up, 8 in 2024-04-07T17:02:46.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:02:45 smithi144 bash[66654]: cluster 2024-04-07T17:02:44.624019+0000 mon.a (mon.0) 2442 : cluster [DBG] osdmap e1216: 8 total, 8 up, 8 in 2024-04-07T17:02:46.951 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:02:46 smithi100 bash[74942]: cluster 2024-04-07T17:02:45.630623+0000 mon.a (mon.0) 2443 : cluster [DBG] osdmap e1217: 8 total, 8 up, 8 in 2024-04-07T17:02:46.951 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:02:46 smithi100 bash[74942]: cluster 2024-04-07T17:02:45.797402+0000 mgr.y (mgr.25175) 1422 : cluster [DBG] pgmap v2214: 169 pgs: 169 active+clean; 629 MiB data, 4.7 GiB used, 711 GiB / 715 GiB avail; 45 MiB/s rd, 6.0 MiB/s wr, 458 op/s 2024-04-07T17:02:46.951 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:02:46 smithi100 bash[76202]: cluster 2024-04-07T17:02:45.630623+0000 mon.a (mon.0) 2443 : cluster [DBG] osdmap e1217: 8 total, 8 up, 8 in 2024-04-07T17:02:46.951 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:02:46 smithi100 bash[76202]: cluster 2024-04-07T17:02:45.797402+0000 mgr.y (mgr.25175) 1422 : cluster [DBG] pgmap v2214: 169 pgs: 169 active+clean; 629 MiB data, 4.7 GiB used, 711 GiB / 715 GiB avail; 45 MiB/s rd, 6.0 MiB/s wr, 458 op/s 2024-04-07T17:02:47.051 INFO:tasks.workunit.client.0.smithi100.stdout:resize: 83886080 2024-04-07T17:02:47.083 INFO:tasks.workunit.client.0.smithi100.stdout:write: Y 41943040~5 2024-04-07T17:02:47.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:02:46 smithi144 bash[66654]: cluster 2024-04-07T17:02:45.630623+0000 mon.a (mon.0) 2443 : cluster [DBG] osdmap e1217: 8 total, 8 up, 8 in 2024-04-07T17:02:47.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:02:46 smithi144 bash[66654]: cluster 2024-04-07T17:02:45.797402+0000 mgr.y (mgr.25175) 1422 : cluster [DBG] pgmap v2214: 169 pgs: 169 active+clean; 629 MiB data, 4.7 GiB used, 711 GiB / 715 GiB avail; 45 MiB/s rd, 6.0 MiB/s wr, 458 op/s 2024-04-07T17:02:47.347 INFO:tasks.workunit.client.0.smithi100.stdout:migration_execute 2024-04-07T17:02:47.584 INFO:tasks.workunit.client.0.smithi100.stdout:migration_commit 2024-04-07T17:02:47.950 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:02:47 smithi100 bash[76202]: cluster 2024-04-07T17:02:46.640674+0000 mon.a (mon.0) 2444 : cluster [DBG] osdmap e1218: 8 total, 8 up, 8 in 2024-04-07T17:02:47.951 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:02:47 smithi100 bash[76202]: audit 2024-04-07T17:02:46.800966+0000 mon.a (mon.0) 2445 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:02:47.951 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:02:47 smithi100 bash[74942]: cluster 2024-04-07T17:02:46.640674+0000 mon.a (mon.0) 2444 : cluster [DBG] osdmap e1218: 8 total, 8 up, 8 in 2024-04-07T17:02:47.951 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:02:47 smithi100 bash[74942]: audit 2024-04-07T17:02:46.800966+0000 mon.a (mon.0) 2445 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:02:48.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:02:47 smithi144 bash[66654]: cluster 2024-04-07T17:02:46.640674+0000 mon.a (mon.0) 2444 : cluster [DBG] osdmap e1218: 8 total, 8 up, 8 in 2024-04-07T17:02:48.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:02:47 smithi144 bash[66654]: audit 2024-04-07T17:02:46.800966+0000 mon.a (mon.0) 2445 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:02:48.701 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:02:48 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:02:48] "GET /metrics HTTP/1.1" 200 38919 "" "Prometheus/2.43.0" 2024-04-07T17:02:48.701 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:02:48 smithi100 bash[76202]: cluster 2024-04-07T17:02:47.798819+0000 mgr.y (mgr.25175) 1423 : cluster [DBG] pgmap v2216: 169 pgs: 169 active+clean; 625 MiB data, 4.7 GiB used, 711 GiB / 715 GiB avail; 38 MiB/s rd, 2.9 MiB/s wr, 296 op/s 2024-04-07T17:02:48.701 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:02:48 smithi100 bash[76202]: audit 2024-04-07T17:02:47.845589+0000 mon.a (mon.0) 2446 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:02:48.701 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:02:48 smithi100 bash[76202]: audit 2024-04-07T17:02:48.635219+0000 mon.a (mon.0) 2447 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:02:48.701 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:02:48 smithi100 bash[76202]: audit 2024-04-07T17:02:48.636719+0000 mon.a (mon.0) 2448 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:02:48.701 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:02:48 smithi100 bash[76202]: audit 2024-04-07T17:02:48.646722+0000 mon.a (mon.0) 2449 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:02:48.701 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:02:48 smithi100 bash[74942]: cluster 2024-04-07T17:02:47.798819+0000 mgr.y (mgr.25175) 1423 : cluster [DBG] pgmap v2216: 169 pgs: 169 active+clean; 625 MiB data, 4.7 GiB used, 711 GiB / 715 GiB avail; 38 MiB/s rd, 2.9 MiB/s wr, 296 op/s 2024-04-07T17:02:48.702 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:02:48 smithi100 bash[74942]: audit 2024-04-07T17:02:47.845589+0000 mon.a (mon.0) 2446 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:02:48.702 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:02:48 smithi100 bash[74942]: audit 2024-04-07T17:02:48.635219+0000 mon.a (mon.0) 2447 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:02:48.702 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:02:48 smithi100 bash[74942]: audit 2024-04-07T17:02:48.636719+0000 mon.a (mon.0) 2448 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:02:48.702 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:02:48 smithi100 bash[74942]: audit 2024-04-07T17:02:48.646722+0000 mon.a (mon.0) 2449 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:02:48.831 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestMigration.SnapsExpandAfterPrepare2 (4471 ms) 2024-04-07T17:02:48.831 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestMigration.SnapsSnapAfterPrepare 2024-04-07T17:02:48.903 INFO:tasks.workunit.client.0.smithi100.stdout:resize: 83886080 2024-04-07T17:02:48.931 INFO:tasks.workunit.client.0.smithi100.stdout:write: 1 0~8388609 2024-04-07T17:02:49.047 INFO:tasks.workunit.client.0.smithi100.stdout:snap_create: snap1 2024-04-07T17:02:49.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:02:48 smithi144 bash[66654]: cluster 2024-04-07T17:02:47.798819+0000 mgr.y (mgr.25175) 1423 : cluster [DBG] pgmap v2216: 169 pgs: 169 active+clean; 625 MiB data, 4.7 GiB used, 711 GiB / 715 GiB avail; 38 MiB/s rd, 2.9 MiB/s wr, 296 op/s 2024-04-07T17:02:49.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:02:48 smithi144 bash[66654]: audit 2024-04-07T17:02:47.845589+0000 mon.a (mon.0) 2446 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:02:49.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:02:48 smithi144 bash[66654]: audit 2024-04-07T17:02:48.635219+0000 mon.a (mon.0) 2447 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:02:49.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:02:48 smithi144 bash[66654]: audit 2024-04-07T17:02:48.636719+0000 mon.a (mon.0) 2448 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:02:49.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:02:48 smithi144 bash[66654]: audit 2024-04-07T17:02:48.646722+0000 mon.a (mon.0) 2449 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:02:49.951 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:02:49 smithi100 bash[74942]: cluster 2024-04-07T17:02:48.667281+0000 mon.a (mon.0) 2450 : cluster [DBG] osdmap e1219: 8 total, 8 up, 8 in 2024-04-07T17:02:49.951 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:02:49 smithi100 bash[76202]: cluster 2024-04-07T17:02:48.667281+0000 mon.a (mon.0) 2450 : cluster [DBG] osdmap e1219: 8 total, 8 up, 8 in 2024-04-07T17:02:50.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:02:49 smithi144 bash[66654]: cluster 2024-04-07T17:02:48.667281+0000 mon.a (mon.0) 2450 : cluster [DBG] osdmap e1219: 8 total, 8 up, 8 in 2024-04-07T17:02:50.717 INFO:tasks.workunit.client.0.smithi100.stdout:write: 1 8388609~8388609 2024-04-07T17:02:50.758 INFO:tasks.workunit.client.0.smithi100.stdout:write: X 0~1000 2024-04-07T17:02:50.758 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 1010~1000 2024-04-07T17:02:50.758 INFO:tasks.workunit.client.0.smithi100.stdout:snap_create: snap2 2024-04-07T17:02:50.951 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:02:50 smithi100 bash[74942]: cluster 2024-04-07T17:02:49.674390+0000 mon.a (mon.0) 2451 : cluster [DBG] osdmap e1220: 8 total, 8 up, 8 in 2024-04-07T17:02:50.951 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:02:50 smithi100 bash[74942]: cluster 2024-04-07T17:02:49.799736+0000 mgr.y (mgr.25175) 1424 : cluster [DBG] pgmap v2219: 169 pgs: 169 active+clean; 629 MiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 22 MiB/s rd, 1.9 MiB/s wr, 310 op/s 2024-04-07T17:02:50.951 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:02:50 smithi100 bash[76202]: cluster 2024-04-07T17:02:49.674390+0000 mon.a (mon.0) 2451 : cluster [DBG] osdmap e1220: 8 total, 8 up, 8 in 2024-04-07T17:02:50.951 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:02:50 smithi100 bash[76202]: cluster 2024-04-07T17:02:49.799736+0000 mgr.y (mgr.25175) 1424 : cluster [DBG] pgmap v2219: 169 pgs: 169 active+clean; 629 MiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 22 MiB/s rd, 1.9 MiB/s wr, 310 op/s 2024-04-07T17:02:51.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:02:50 smithi144 bash[66654]: cluster 2024-04-07T17:02:49.674390+0000 mon.a (mon.0) 2451 : cluster [DBG] osdmap e1220: 8 total, 8 up, 8 in 2024-04-07T17:02:51.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:02:50 smithi144 bash[66654]: cluster 2024-04-07T17:02:49.799736+0000 mgr.y (mgr.25175) 1424 : cluster [DBG] pgmap v2219: 169 pgs: 169 active+clean; 629 MiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 22 MiB/s rd, 1.9 MiB/s wr, 310 op/s 2024-04-07T17:02:51.951 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:02:51 smithi100 bash[74942]: cluster 2024-04-07T17:02:50.686022+0000 mon.a (mon.0) 2452 : cluster [DBG] osdmap e1221: 8 total, 8 up, 8 in 2024-04-07T17:02:51.951 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:02:51 smithi100 bash[76202]: cluster 2024-04-07T17:02:50.686022+0000 mon.a (mon.0) 2452 : cluster [DBG] osdmap e1221: 8 total, 8 up, 8 in 2024-04-07T17:02:52.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:02:51 smithi144 bash[66654]: cluster 2024-04-07T17:02:50.686022+0000 mon.a (mon.0) 2452 : cluster [DBG] osdmap e1221: 8 total, 8 up, 8 in 2024-04-07T17:02:52.588 INFO:tasks.workunit.client.0.smithi100.stdout:write: X 8388609~1000 2024-04-07T17:02:52.605 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 16777228~1000 2024-04-07T17:02:52.606 INFO:tasks.workunit.client.0.smithi100.stdout:resize: 167772160 2024-04-07T17:02:52.614 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:02:52 smithi144 bash[95233]: ts=2024-04-07T17:02:52.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:02:52.627 INFO:tasks.workunit.client.0.smithi100.stdout:write: 2 83886079~8388609 2024-04-07T17:02:52.673 INFO:tasks.workunit.client.0.smithi100.stdout:snap_create: snap3 2024-04-07T17:02:53.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:02:52 smithi144 bash[66654]: cluster 2024-04-07T17:02:51.686692+0000 mon.a (mon.0) 2453 : cluster [DBG] osdmap e1222: 8 total, 8 up, 8 in 2024-04-07T17:02:53.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:02:52 smithi144 bash[66654]: cluster 2024-04-07T17:02:51.800594+0000 mgr.y (mgr.25175) 1425 : cluster [DBG] pgmap v2222: 169 pgs: 169 active+clean; 629 MiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 15 MiB/s rd, 2.0 MiB/s wr, 219 op/s 2024-04-07T17:02:53.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:02:52 smithi144 bash[66654]: cluster 2024-04-07T17:02:52.560459+0000 mon.a (mon.0) 2454 : cluster [DBG] osdmap e1223: 8 total, 8 up, 8 in 2024-04-07T17:02:53.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:02:52 smithi100 bash[74942]: cluster 2024-04-07T17:02:51.686692+0000 mon.a (mon.0) 2453 : cluster [DBG] osdmap e1222: 8 total, 8 up, 8 in 2024-04-07T17:02:53.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:02:52 smithi100 bash[74942]: cluster 2024-04-07T17:02:51.800594+0000 mgr.y (mgr.25175) 1425 : cluster [DBG] pgmap v2222: 169 pgs: 169 active+clean; 629 MiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 15 MiB/s rd, 2.0 MiB/s wr, 219 op/s 2024-04-07T17:02:53.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:02:52 smithi100 bash[74942]: cluster 2024-04-07T17:02:52.560459+0000 mon.a (mon.0) 2454 : cluster [DBG] osdmap e1223: 8 total, 8 up, 8 in 2024-04-07T17:02:53.201 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:02:52 smithi100 bash[76202]: cluster 2024-04-07T17:02:51.686692+0000 mon.a (mon.0) 2453 : cluster [DBG] osdmap e1222: 8 total, 8 up, 8 in 2024-04-07T17:02:53.201 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:02:52 smithi100 bash[76202]: cluster 2024-04-07T17:02:51.800594+0000 mgr.y (mgr.25175) 1425 : cluster [DBG] pgmap v2222: 169 pgs: 169 active+clean; 629 MiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 15 MiB/s rd, 2.0 MiB/s wr, 219 op/s 2024-04-07T17:02:53.202 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:02:52 smithi100 bash[76202]: cluster 2024-04-07T17:02:52.560459+0000 mon.a (mon.0) 2454 : cluster [DBG] osdmap e1223: 8 total, 8 up, 8 in 2024-04-07T17:02:54.754 INFO:tasks.workunit.client.0.smithi100.stdout:resize: 83886080 2024-04-07T17:02:54.816 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 83886079~1 2024-04-07T17:02:54.840 INFO:tasks.workunit.client.0.smithi100.stdout:migration_prepare 2024-04-07T17:02:55.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:02:54 smithi144 bash[66654]: cluster 2024-04-07T17:02:53.718460+0000 mon.a (mon.0) 2455 : cluster [DBG] osdmap e1224: 8 total, 8 up, 8 in 2024-04-07T17:02:55.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:02:54 smithi144 bash[66654]: cluster 2024-04-07T17:02:53.801411+0000 mgr.y (mgr.25175) 1426 : cluster [DBG] pgmap v2225: 169 pgs: 169 active+clean; 649 MiB data, 4.7 GiB used, 711 GiB / 715 GiB avail; 145 KiB/s rd, 5.0 MiB/s wr, 196 op/s 2024-04-07T17:02:55.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:02:54 smithi100 bash[74942]: cluster 2024-04-07T17:02:53.718460+0000 mon.a (mon.0) 2455 : cluster [DBG] osdmap e1224: 8 total, 8 up, 8 in 2024-04-07T17:02:55.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:02:54 smithi100 bash[74942]: cluster 2024-04-07T17:02:53.801411+0000 mgr.y (mgr.25175) 1426 : cluster [DBG] pgmap v2225: 169 pgs: 169 active+clean; 649 MiB data, 4.7 GiB used, 711 GiB / 715 GiB avail; 145 KiB/s rd, 5.0 MiB/s wr, 196 op/s 2024-04-07T17:02:55.201 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:02:54 smithi100 bash[76202]: cluster 2024-04-07T17:02:53.718460+0000 mon.a (mon.0) 2455 : cluster [DBG] osdmap e1224: 8 total, 8 up, 8 in 2024-04-07T17:02:55.201 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:02:54 smithi100 bash[76202]: cluster 2024-04-07T17:02:53.801411+0000 mgr.y (mgr.25175) 1426 : cluster [DBG] pgmap v2225: 169 pgs: 169 active+clean; 649 MiB data, 4.7 GiB used, 711 GiB / 715 GiB avail; 145 KiB/s rd, 5.0 MiB/s wr, 196 op/s 2024-04-07T17:02:56.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:02:55 smithi144 bash[66654]: cluster 2024-04-07T17:02:54.730213+0000 mon.a (mon.0) 2456 : cluster [DBG] osdmap e1225: 8 total, 8 up, 8 in 2024-04-07T17:02:56.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:02:55 smithi100 bash[74942]: cluster 2024-04-07T17:02:54.730213+0000 mon.a (mon.0) 2456 : cluster [DBG] osdmap e1225: 8 total, 8 up, 8 in 2024-04-07T17:02:56.201 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:02:55 smithi100 bash[76202]: cluster 2024-04-07T17:02:54.730213+0000 mon.a (mon.0) 2456 : cluster [DBG] osdmap e1225: 8 total, 8 up, 8 in 2024-04-07T17:02:57.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:02:56 smithi144 bash[66654]: cluster 2024-04-07T17:02:55.736458+0000 mon.a (mon.0) 2457 : cluster [DBG] osdmap e1226: 8 total, 8 up, 8 in 2024-04-07T17:02:57.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:02:56 smithi144 bash[66654]: cluster 2024-04-07T17:02:55.802224+0000 mgr.y (mgr.25175) 1427 : cluster [DBG] pgmap v2228: 169 pgs: 169 active+clean; 661 MiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 164 KiB/s rd, 8.0 MiB/s wr, 221 op/s 2024-04-07T17:02:57.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:02:56 smithi100 bash[74942]: cluster 2024-04-07T17:02:55.736458+0000 mon.a (mon.0) 2457 : cluster [DBG] osdmap e1226: 8 total, 8 up, 8 in 2024-04-07T17:02:57.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:02:56 smithi100 bash[74942]: cluster 2024-04-07T17:02:55.802224+0000 mgr.y (mgr.25175) 1427 : cluster [DBG] pgmap v2228: 169 pgs: 169 active+clean; 661 MiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 164 KiB/s rd, 8.0 MiB/s wr, 221 op/s 2024-04-07T17:02:57.201 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:02:56 smithi100 bash[76202]: cluster 2024-04-07T17:02:55.736458+0000 mon.a (mon.0) 2457 : cluster [DBG] osdmap e1226: 8 total, 8 up, 8 in 2024-04-07T17:02:57.201 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:02:56 smithi100 bash[76202]: cluster 2024-04-07T17:02:55.802224+0000 mgr.y (mgr.25175) 1427 : cluster [DBG] pgmap v2228: 169 pgs: 169 active+clean; 661 MiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 164 KiB/s rd, 8.0 MiB/s wr, 221 op/s 2024-04-07T17:02:58.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:02:57 smithi144 bash[66654]: cluster 2024-04-07T17:02:56.746950+0000 mon.a (mon.0) 2458 : cluster [DBG] osdmap e1227: 8 total, 8 up, 8 in 2024-04-07T17:02:58.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:02:57 smithi144 bash[66654]: cluster 2024-04-07T17:02:57.565850+0000 mon.a (mon.0) 2459 : cluster [DBG] osdmap e1228: 8 total, 8 up, 8 in 2024-04-07T17:02:58.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:02:57 smithi100 bash[74942]: cluster 2024-04-07T17:02:56.746950+0000 mon.a (mon.0) 2458 : cluster [DBG] osdmap e1227: 8 total, 8 up, 8 in 2024-04-07T17:02:58.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:02:57 smithi100 bash[74942]: cluster 2024-04-07T17:02:57.565850+0000 mon.a (mon.0) 2459 : cluster [DBG] osdmap e1228: 8 total, 8 up, 8 in 2024-04-07T17:02:58.201 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:02:57 smithi100 bash[76202]: cluster 2024-04-07T17:02:56.746950+0000 mon.a (mon.0) 2458 : cluster [DBG] osdmap e1227: 8 total, 8 up, 8 in 2024-04-07T17:02:58.201 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:02:57 smithi100 bash[76202]: cluster 2024-04-07T17:02:57.565850+0000 mon.a (mon.0) 2459 : cluster [DBG] osdmap e1228: 8 total, 8 up, 8 in 2024-04-07T17:02:58.701 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:02:58 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:02:58] "GET /metrics HTTP/1.1" 200 38919 "" "Prometheus/2.43.0" 2024-04-07T17:02:58.795 INFO:tasks.workunit.client.0.smithi100.stdout:snap_create: after_prepare_snap 2024-04-07T17:02:59.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:02:58 smithi144 bash[66654]: cluster 2024-04-07T17:02:57.803208+0000 mgr.y (mgr.25175) 1428 : cluster [DBG] pgmap v2231: 169 pgs: 169 active+clean; 677 MiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 39 KiB/s rd, 5.0 MiB/s wr, 57 op/s 2024-04-07T17:02:59.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:02:58 smithi100 bash[74942]: cluster 2024-04-07T17:02:57.803208+0000 mgr.y (mgr.25175) 1428 : cluster [DBG] pgmap v2231: 169 pgs: 169 active+clean; 677 MiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 39 KiB/s rd, 5.0 MiB/s wr, 57 op/s 2024-04-07T17:02:59.201 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:02:58 smithi100 bash[76202]: cluster 2024-04-07T17:02:57.803208+0000 mgr.y (mgr.25175) 1428 : cluster [DBG] pgmap v2231: 169 pgs: 169 active+clean; 677 MiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 39 KiB/s rd, 5.0 MiB/s wr, 57 op/s 2024-04-07T17:03:00.801 INFO:tasks.workunit.client.0.smithi100.stdout:resize: 41943040 2024-04-07T17:03:00.883 INFO:tasks.workunit.client.0.smithi100.stdout:write: * 0~1000 2024-04-07T17:03:00.900 INFO:tasks.workunit.client.0.smithi100.stdout:migration_execute 2024-04-07T17:03:01.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:03:00 smithi144 bash[66654]: cluster 2024-04-07T17:02:59.763332+0000 mon.a (mon.0) 2460 : cluster [DBG] osdmap e1229: 8 total, 8 up, 8 in 2024-04-07T17:03:01.177 INFO:tasks.workunit.client.0.smithi100.stdout:migration_commit 2024-04-07T17:03:01.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:03:00 smithi100 bash[74942]: cluster 2024-04-07T17:02:59.763332+0000 mon.a (mon.0) 2460 : cluster [DBG] osdmap e1229: 8 total, 8 up, 8 in 2024-04-07T17:03:01.201 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:03:00 smithi100 bash[76202]: cluster 2024-04-07T17:02:59.763332+0000 mon.a (mon.0) 2460 : cluster [DBG] osdmap e1229: 8 total, 8 up, 8 in 2024-04-07T17:03:02.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:03:01 smithi144 bash[66654]: cluster 2024-04-07T17:02:59.804066+0000 mgr.y (mgr.25175) 1429 : cluster [DBG] pgmap v2233: 169 pgs: 169 active+clean; 677 MiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 21 MiB/s rd, 2.0 MiB/s wr, 199 op/s 2024-04-07T17:03:02.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:03:01 smithi144 bash[66654]: cluster 2024-04-07T17:03:00.775830+0000 mon.a (mon.0) 2461 : cluster [DBG] osdmap e1230: 8 total, 8 up, 8 in 2024-04-07T17:03:02.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:03:01 smithi100 bash[74942]: cluster 2024-04-07T17:02:59.804066+0000 mgr.y (mgr.25175) 1429 : cluster [DBG] pgmap v2233: 169 pgs: 169 active+clean; 677 MiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 21 MiB/s rd, 2.0 MiB/s wr, 199 op/s 2024-04-07T17:03:02.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:03:01 smithi100 bash[74942]: cluster 2024-04-07T17:03:00.775830+0000 mon.a (mon.0) 2461 : cluster [DBG] osdmap e1230: 8 total, 8 up, 8 in 2024-04-07T17:03:02.201 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:03:01 smithi100 bash[76202]: cluster 2024-04-07T17:02:59.804066+0000 mgr.y (mgr.25175) 1429 : cluster [DBG] pgmap v2233: 169 pgs: 169 active+clean; 677 MiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 21 MiB/s rd, 2.0 MiB/s wr, 199 op/s 2024-04-07T17:03:02.201 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:03:01 smithi100 bash[76202]: cluster 2024-04-07T17:03:00.775830+0000 mon.a (mon.0) 2461 : cluster [DBG] osdmap e1230: 8 total, 8 up, 8 in 2024-04-07T17:03:02.615 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:03:02 smithi144 bash[95233]: ts=2024-04-07T17:03:02.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:03:03.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:03:02 smithi144 bash[66654]: cluster 2024-04-07T17:03:01.774138+0000 mon.a (mon.0) 2462 : cluster [DBG] osdmap e1231: 8 total, 8 up, 8 in 2024-04-07T17:03:03.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:03:02 smithi144 bash[66654]: audit 2024-04-07T17:03:01.801584+0000 mon.a (mon.0) 2463 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:03:03.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:03:02 smithi100 bash[74942]: cluster 2024-04-07T17:03:01.774138+0000 mon.a (mon.0) 2462 : cluster [DBG] osdmap e1231: 8 total, 8 up, 8 in 2024-04-07T17:03:03.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:03:02 smithi100 bash[74942]: audit 2024-04-07T17:03:01.801584+0000 mon.a (mon.0) 2463 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:03:03.201 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:03:02 smithi100 bash[76202]: cluster 2024-04-07T17:03:01.774138+0000 mon.a (mon.0) 2462 : cluster [DBG] osdmap e1231: 8 total, 8 up, 8 in 2024-04-07T17:03:03.201 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:03:02 smithi100 bash[76202]: audit 2024-04-07T17:03:01.801584+0000 mon.a (mon.0) 2463 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:03:04.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:03:03 smithi144 bash[66654]: cluster 2024-04-07T17:03:01.804799+0000 mgr.y (mgr.25175) 1430 : cluster [DBG] pgmap v2236: 169 pgs: 169 active+clean; 677 MiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 20 MiB/s rd, 7.5 KiB/s wr, 168 op/s 2024-04-07T17:03:04.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:03:03 smithi144 bash[66654]: cluster 2024-04-07T17:03:02.788555+0000 mon.a (mon.0) 2464 : cluster [DBG] osdmap e1232: 8 total, 8 up, 8 in 2024-04-07T17:03:04.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:03:03 smithi100 bash[74942]: cluster 2024-04-07T17:03:01.804799+0000 mgr.y (mgr.25175) 1430 : cluster [DBG] pgmap v2236: 169 pgs: 169 active+clean; 677 MiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 20 MiB/s rd, 7.5 KiB/s wr, 168 op/s 2024-04-07T17:03:04.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:03:03 smithi100 bash[74942]: cluster 2024-04-07T17:03:02.788555+0000 mon.a (mon.0) 2464 : cluster [DBG] osdmap e1232: 8 total, 8 up, 8 in 2024-04-07T17:03:04.201 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:03:03 smithi100 bash[76202]: cluster 2024-04-07T17:03:01.804799+0000 mgr.y (mgr.25175) 1430 : cluster [DBG] pgmap v2236: 169 pgs: 169 active+clean; 677 MiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 20 MiB/s rd, 7.5 KiB/s wr, 168 op/s 2024-04-07T17:03:04.201 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:03:03 smithi100 bash[76202]: cluster 2024-04-07T17:03:02.788555+0000 mon.a (mon.0) 2464 : cluster [DBG] osdmap e1232: 8 total, 8 up, 8 in 2024-04-07T17:03:04.534 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestMigration.SnapsSnapAfterPrepare (15703 ms) 2024-04-07T17:03:04.534 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestMigration.SnapsSnapExpandAfterPrepare 2024-04-07T17:03:04.626 INFO:tasks.workunit.client.0.smithi100.stdout:resize: 83886080 2024-04-07T17:03:04.659 INFO:tasks.workunit.client.0.smithi100.stdout:write: 1 0~8388609 2024-04-07T17:03:04.778 INFO:tasks.workunit.client.0.smithi100.stdout:snap_create: snap1 2024-04-07T17:03:05.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:03:04 smithi144 bash[66654]: cluster 2024-04-07T17:03:03.810727+0000 mon.a (mon.0) 2465 : cluster [DBG] osdmap e1233: 8 total, 8 up, 8 in 2024-04-07T17:03:05.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:03:04 smithi100 bash[74942]: cluster 2024-04-07T17:03:03.810727+0000 mon.a (mon.0) 2465 : cluster [DBG] osdmap e1233: 8 total, 8 up, 8 in 2024-04-07T17:03:05.201 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:03:04 smithi100 bash[76202]: cluster 2024-04-07T17:03:03.810727+0000 mon.a (mon.0) 2465 : cluster [DBG] osdmap e1233: 8 total, 8 up, 8 in 2024-04-07T17:03:05.847 INFO:tasks.workunit.client.0.smithi100.stdout:write: 1 8388609~8388609 2024-04-07T17:03:05.895 INFO:tasks.workunit.client.0.smithi100.stdout:write: X 0~1000 2024-04-07T17:03:05.895 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 1010~1000 2024-04-07T17:03:05.896 INFO:tasks.workunit.client.0.smithi100.stdout:snap_create: snap2 2024-04-07T17:03:06.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:03:05 smithi144 bash[66654]: cluster 2024-04-07T17:03:03.806149+0000 mgr.y (mgr.25175) 1431 : cluster [DBG] pgmap v2239: 169 pgs: 169 active+clean; 693 MiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 26 MiB/s rd, 5.0 MiB/s wr, 392 op/s 2024-04-07T17:03:06.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:03:05 smithi144 bash[66654]: cluster 2024-04-07T17:03:04.819130+0000 mon.a (mon.0) 2466 : cluster [DBG] osdmap e1234: 8 total, 8 up, 8 in 2024-04-07T17:03:06.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:03:05 smithi100 bash[74942]: cluster 2024-04-07T17:03:03.806149+0000 mgr.y (mgr.25175) 1431 : cluster [DBG] pgmap v2239: 169 pgs: 169 active+clean; 693 MiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 26 MiB/s rd, 5.0 MiB/s wr, 392 op/s 2024-04-07T17:03:06.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:03:05 smithi100 bash[74942]: cluster 2024-04-07T17:03:04.819130+0000 mon.a (mon.0) 2466 : cluster [DBG] osdmap e1234: 8 total, 8 up, 8 in 2024-04-07T17:03:06.201 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:03:05 smithi100 bash[76202]: cluster 2024-04-07T17:03:03.806149+0000 mgr.y (mgr.25175) 1431 : cluster [DBG] pgmap v2239: 169 pgs: 169 active+clean; 693 MiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 26 MiB/s rd, 5.0 MiB/s wr, 392 op/s 2024-04-07T17:03:06.201 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:03:05 smithi100 bash[76202]: cluster 2024-04-07T17:03:04.819130+0000 mon.a (mon.0) 2466 : cluster [DBG] osdmap e1234: 8 total, 8 up, 8 in 2024-04-07T17:03:07.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:03:06 smithi144 bash[66654]: cluster 2024-04-07T17:03:05.807312+0000 mgr.y (mgr.25175) 1432 : cluster [DBG] pgmap v2241: 169 pgs: 169 active+clean; 693 MiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 32 MiB/s rd, 7.0 MiB/s wr, 469 op/s 2024-04-07T17:03:07.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:03:06 smithi144 bash[66654]: cluster 2024-04-07T17:03:05.823353+0000 mon.a (mon.0) 2467 : cluster [DBG] osdmap e1235: 8 total, 8 up, 8 in 2024-04-07T17:03:07.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:03:06 smithi100 bash[74942]: cluster 2024-04-07T17:03:05.807312+0000 mgr.y (mgr.25175) 1432 : cluster [DBG] pgmap v2241: 169 pgs: 169 active+clean; 693 MiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 32 MiB/s rd, 7.0 MiB/s wr, 469 op/s 2024-04-07T17:03:07.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:03:06 smithi100 bash[74942]: cluster 2024-04-07T17:03:05.823353+0000 mon.a (mon.0) 2467 : cluster [DBG] osdmap e1235: 8 total, 8 up, 8 in 2024-04-07T17:03:07.201 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:03:06 smithi100 bash[76202]: cluster 2024-04-07T17:03:05.807312+0000 mgr.y (mgr.25175) 1432 : cluster [DBG] pgmap v2241: 169 pgs: 169 active+clean; 693 MiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 32 MiB/s rd, 7.0 MiB/s wr, 469 op/s 2024-04-07T17:03:07.201 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:03:06 smithi100 bash[76202]: cluster 2024-04-07T17:03:05.823353+0000 mon.a (mon.0) 2467 : cluster [DBG] osdmap e1235: 8 total, 8 up, 8 in 2024-04-07T17:03:07.598 INFO:tasks.workunit.client.0.smithi100.stdout:write: X 8388609~1000 2024-04-07T17:03:07.616 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 16777228~1000 2024-04-07T17:03:07.616 INFO:tasks.workunit.client.0.smithi100.stdout:resize: 167772160 2024-04-07T17:03:07.637 INFO:tasks.workunit.client.0.smithi100.stdout:write: 2 83886079~8388609 2024-04-07T17:03:07.693 INFO:tasks.workunit.client.0.smithi100.stdout:snap_create: snap3 2024-04-07T17:03:08.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:03:07 smithi144 bash[66654]: cluster 2024-04-07T17:03:06.833484+0000 mon.a (mon.0) 2468 : cluster [DBG] osdmap e1236: 8 total, 8 up, 8 in 2024-04-07T17:03:08.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:03:07 smithi144 bash[66654]: cluster 2024-04-07T17:03:07.564936+0000 mon.a (mon.0) 2469 : cluster [DBG] osdmap e1237: 8 total, 8 up, 8 in 2024-04-07T17:03:08.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:03:07 smithi100 bash[74942]: cluster 2024-04-07T17:03:06.833484+0000 mon.a (mon.0) 2468 : cluster [DBG] osdmap e1236: 8 total, 8 up, 8 in 2024-04-07T17:03:08.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:03:07 smithi100 bash[74942]: cluster 2024-04-07T17:03:07.564936+0000 mon.a (mon.0) 2469 : cluster [DBG] osdmap e1237: 8 total, 8 up, 8 in 2024-04-07T17:03:08.201 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:03:07 smithi100 bash[76202]: cluster 2024-04-07T17:03:06.833484+0000 mon.a (mon.0) 2468 : cluster [DBG] osdmap e1236: 8 total, 8 up, 8 in 2024-04-07T17:03:08.201 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:03:07 smithi100 bash[76202]: cluster 2024-04-07T17:03:07.564936+0000 mon.a (mon.0) 2469 : cluster [DBG] osdmap e1237: 8 total, 8 up, 8 in 2024-04-07T17:03:08.701 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:03:08 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:03:08] "GET /metrics HTTP/1.1" 200 38920 "" "Prometheus/2.43.0" 2024-04-07T17:03:09.603 INFO:tasks.workunit.client.0.smithi100.stdout:resize: 83886080 2024-04-07T17:03:09.657 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 83886079~1 2024-04-07T17:03:09.680 INFO:tasks.workunit.client.0.smithi100.stdout:migration_prepare 2024-04-07T17:03:09.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:03:09 smithi144 bash[66654]: cluster 2024-04-07T17:03:07.808130+0000 mgr.y (mgr.25175) 1433 : cluster [DBG] pgmap v2245: 169 pgs: 169 active+clean; 697 MiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 22 MiB/s rd, 5.0 MiB/s wr, 158 op/s 2024-04-07T17:03:09.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:03:09 smithi144 bash[66654]: cluster 2024-04-07T17:03:08.575390+0000 mon.a (mon.0) 2470 : cluster [DBG] osdmap e1238: 8 total, 8 up, 8 in 2024-04-07T17:03:09.951 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:03:09 smithi100 bash[74942]: cluster 2024-04-07T17:03:07.808130+0000 mgr.y (mgr.25175) 1433 : cluster [DBG] pgmap v2245: 169 pgs: 169 active+clean; 697 MiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 22 MiB/s rd, 5.0 MiB/s wr, 158 op/s 2024-04-07T17:03:09.951 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:03:09 smithi100 bash[74942]: cluster 2024-04-07T17:03:08.575390+0000 mon.a (mon.0) 2470 : cluster [DBG] osdmap e1238: 8 total, 8 up, 8 in 2024-04-07T17:03:09.951 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:03:09 smithi100 bash[76202]: cluster 2024-04-07T17:03:07.808130+0000 mgr.y (mgr.25175) 1433 : cluster [DBG] pgmap v2245: 169 pgs: 169 active+clean; 697 MiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 22 MiB/s rd, 5.0 MiB/s wr, 158 op/s 2024-04-07T17:03:09.952 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:03:09 smithi100 bash[76202]: cluster 2024-04-07T17:03:08.575390+0000 mon.a (mon.0) 2470 : cluster [DBG] osdmap e1238: 8 total, 8 up, 8 in 2024-04-07T17:03:10.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:03:10 smithi144 bash[66654]: cluster 2024-04-07T17:03:09.577774+0000 mon.a (mon.0) 2471 : cluster [DBG] osdmap e1239: 8 total, 8 up, 8 in 2024-04-07T17:03:10.951 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:03:10 smithi100 bash[74942]: cluster 2024-04-07T17:03:09.577774+0000 mon.a (mon.0) 2471 : cluster [DBG] osdmap e1239: 8 total, 8 up, 8 in 2024-04-07T17:03:10.951 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:03:10 smithi100 bash[76202]: cluster 2024-04-07T17:03:09.577774+0000 mon.a (mon.0) 2471 : cluster [DBG] osdmap e1239: 8 total, 8 up, 8 in 2024-04-07T17:03:11.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:03:11 smithi144 bash[66654]: cluster 2024-04-07T17:03:09.809130+0000 mgr.y (mgr.25175) 1434 : cluster [DBG] pgmap v2248: 169 pgs: 169 active+clean; 713 MiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 37 MiB/s rd, 7.0 MiB/s wr, 107 op/s 2024-04-07T17:03:11.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:03:11 smithi144 bash[66654]: cluster 2024-04-07T17:03:10.591134+0000 mon.a (mon.0) 2472 : cluster [DBG] osdmap e1240: 8 total, 8 up, 8 in 2024-04-07T17:03:11.951 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:03:11 smithi100 bash[74942]: cluster 2024-04-07T17:03:09.809130+0000 mgr.y (mgr.25175) 1434 : cluster [DBG] pgmap v2248: 169 pgs: 169 active+clean; 713 MiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 37 MiB/s rd, 7.0 MiB/s wr, 107 op/s 2024-04-07T17:03:11.951 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:03:11 smithi100 bash[74942]: cluster 2024-04-07T17:03:10.591134+0000 mon.a (mon.0) 2472 : cluster [DBG] osdmap e1240: 8 total, 8 up, 8 in 2024-04-07T17:03:11.951 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:03:11 smithi100 bash[76202]: cluster 2024-04-07T17:03:09.809130+0000 mgr.y (mgr.25175) 1434 : cluster [DBG] pgmap v2248: 169 pgs: 169 active+clean; 713 MiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 37 MiB/s rd, 7.0 MiB/s wr, 107 op/s 2024-04-07T17:03:11.952 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:03:11 smithi100 bash[76202]: cluster 2024-04-07T17:03:10.591134+0000 mon.a (mon.0) 2472 : cluster [DBG] osdmap e1240: 8 total, 8 up, 8 in 2024-04-07T17:03:12.615 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:03:12 smithi144 bash[95233]: ts=2024-04-07T17:03:12.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:03:12.951 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:03:12 smithi100 bash[74942]: cluster 2024-04-07T17:03:11.614148+0000 mon.a (mon.0) 2473 : cluster [DBG] osdmap e1241: 8 total, 8 up, 8 in 2024-04-07T17:03:12.951 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:03:12 smithi100 bash[74942]: cluster 2024-04-07T17:03:12.565607+0000 mon.a (mon.0) 2474 : cluster [DBG] osdmap e1242: 8 total, 8 up, 8 in 2024-04-07T17:03:12.951 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:03:12 smithi100 bash[76202]: cluster 2024-04-07T17:03:11.614148+0000 mon.a (mon.0) 2473 : cluster [DBG] osdmap e1241: 8 total, 8 up, 8 in 2024-04-07T17:03:12.952 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:03:12 smithi100 bash[76202]: cluster 2024-04-07T17:03:12.565607+0000 mon.a (mon.0) 2474 : cluster [DBG] osdmap e1242: 8 total, 8 up, 8 in 2024-04-07T17:03:13.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:03:12 smithi144 bash[66654]: cluster 2024-04-07T17:03:11.614148+0000 mon.a (mon.0) 2473 : cluster [DBG] osdmap e1241: 8 total, 8 up, 8 in 2024-04-07T17:03:13.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:03:12 smithi144 bash[66654]: cluster 2024-04-07T17:03:12.565607+0000 mon.a (mon.0) 2474 : cluster [DBG] osdmap e1242: 8 total, 8 up, 8 in 2024-04-07T17:03:13.550 INFO:tasks.workunit.client.0.smithi100.stdout:snap_create: after_prepare_snap 2024-04-07T17:03:13.951 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:03:13 smithi100 bash[74942]: cluster 2024-04-07T17:03:11.809977+0000 mgr.y (mgr.25175) 1435 : cluster [DBG] pgmap v2251: 169 pgs: 169 active+clean; 717 MiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 21 MiB/s rd, 5.0 MiB/s wr, 68 op/s 2024-04-07T17:03:13.951 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:03:13 smithi100 bash[76202]: cluster 2024-04-07T17:03:11.809977+0000 mgr.y (mgr.25175) 1435 : cluster [DBG] pgmap v2251: 169 pgs: 169 active+clean; 717 MiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 21 MiB/s rd, 5.0 MiB/s wr, 68 op/s 2024-04-07T17:03:14.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:03:13 smithi144 bash[66654]: cluster 2024-04-07T17:03:11.809977+0000 mgr.y (mgr.25175) 1435 : cluster [DBG] pgmap v2251: 169 pgs: 169 active+clean; 717 MiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 21 MiB/s rd, 5.0 MiB/s wr, 68 op/s 2024-04-07T17:03:14.682 INFO:tasks.workunit.client.0.smithi100.stdout:resize: 167772160 2024-04-07T17:03:14.724 INFO:tasks.workunit.client.0.smithi100.stdout:write: * 83886080~1000 2024-04-07T17:03:14.746 INFO:tasks.workunit.client.0.smithi100.stdout:migration_execute 2024-04-07T17:03:14.951 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:03:14 smithi100 bash[74942]: cluster 2024-04-07T17:03:13.636081+0000 mon.a (mon.0) 2475 : cluster [DBG] osdmap e1243: 8 total, 8 up, 8 in 2024-04-07T17:03:14.951 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:03:14 smithi100 bash[76202]: cluster 2024-04-07T17:03:13.636081+0000 mon.a (mon.0) 2475 : cluster [DBG] osdmap e1243: 8 total, 8 up, 8 in 2024-04-07T17:03:15.044 INFO:tasks.workunit.client.0.smithi100.stdout:migration_commit 2024-04-07T17:03:15.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:03:14 smithi144 bash[66654]: cluster 2024-04-07T17:03:13.636081+0000 mon.a (mon.0) 2475 : cluster [DBG] osdmap e1243: 8 total, 8 up, 8 in 2024-04-07T17:03:15.951 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:03:15 smithi100 bash[74942]: cluster 2024-04-07T17:03:13.810954+0000 mgr.y (mgr.25175) 1436 : cluster [DBG] pgmap v2254: 169 pgs: 169 active+clean; 733 MiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 1.1 MiB/s rd, 4.0 MiB/s wr, 177 op/s 2024-04-07T17:03:15.951 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:03:15 smithi100 bash[74942]: cluster 2024-04-07T17:03:14.648437+0000 mon.a (mon.0) 2476 : cluster [DBG] osdmap e1244: 8 total, 8 up, 8 in 2024-04-07T17:03:15.951 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:03:15 smithi100 bash[76202]: cluster 2024-04-07T17:03:13.810954+0000 mgr.y (mgr.25175) 1436 : cluster [DBG] pgmap v2254: 169 pgs: 169 active+clean; 733 MiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 1.1 MiB/s rd, 4.0 MiB/s wr, 177 op/s 2024-04-07T17:03:15.952 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:03:15 smithi100 bash[76202]: cluster 2024-04-07T17:03:14.648437+0000 mon.a (mon.0) 2476 : cluster [DBG] osdmap e1244: 8 total, 8 up, 8 in 2024-04-07T17:03:16.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:03:15 smithi144 bash[66654]: cluster 2024-04-07T17:03:13.810954+0000 mgr.y (mgr.25175) 1436 : cluster [DBG] pgmap v2254: 169 pgs: 169 active+clean; 733 MiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 1.1 MiB/s rd, 4.0 MiB/s wr, 177 op/s 2024-04-07T17:03:16.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:03:15 smithi144 bash[66654]: cluster 2024-04-07T17:03:14.648437+0000 mon.a (mon.0) 2476 : cluster [DBG] osdmap e1244: 8 total, 8 up, 8 in 2024-04-07T17:03:16.951 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:03:16 smithi100 bash[74942]: cluster 2024-04-07T17:03:15.658891+0000 mon.a (mon.0) 2477 : cluster [DBG] osdmap e1245: 8 total, 8 up, 8 in 2024-04-07T17:03:16.951 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:03:16 smithi100 bash[76202]: cluster 2024-04-07T17:03:15.658891+0000 mon.a (mon.0) 2477 : cluster [DBG] osdmap e1245: 8 total, 8 up, 8 in 2024-04-07T17:03:17.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:03:16 smithi144 bash[66654]: cluster 2024-04-07T17:03:15.658891+0000 mon.a (mon.0) 2477 : cluster [DBG] osdmap e1245: 8 total, 8 up, 8 in 2024-04-07T17:03:17.951 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:03:17 smithi100 bash[74942]: cluster 2024-04-07T17:03:15.811843+0000 mgr.y (mgr.25175) 1437 : cluster [DBG] pgmap v2257: 169 pgs: 169 active+clean; 737 MiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 18 MiB/s rd, 4.0 MiB/s wr, 170 op/s 2024-04-07T17:03:17.951 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:03:17 smithi100 bash[74942]: cluster 2024-04-07T17:03:16.660446+0000 mon.a (mon.0) 2478 : cluster [DBG] osdmap e1246: 8 total, 8 up, 8 in 2024-04-07T17:03:17.951 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:03:17 smithi100 bash[74942]: audit 2024-04-07T17:03:16.801751+0000 mon.a (mon.0) 2479 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:03:17.952 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:03:17 smithi100 bash[76202]: cluster 2024-04-07T17:03:15.811843+0000 mgr.y (mgr.25175) 1437 : cluster [DBG] pgmap v2257: 169 pgs: 169 active+clean; 737 MiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 18 MiB/s rd, 4.0 MiB/s wr, 170 op/s 2024-04-07T17:03:17.952 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:03:17 smithi100 bash[76202]: cluster 2024-04-07T17:03:16.660446+0000 mon.a (mon.0) 2478 : cluster [DBG] osdmap e1246: 8 total, 8 up, 8 in 2024-04-07T17:03:17.952 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:03:17 smithi100 bash[76202]: audit 2024-04-07T17:03:16.801751+0000 mon.a (mon.0) 2479 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:03:18.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:03:17 smithi144 bash[66654]: cluster 2024-04-07T17:03:15.811843+0000 mgr.y (mgr.25175) 1437 : cluster [DBG] pgmap v2257: 169 pgs: 169 active+clean; 737 MiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 18 MiB/s rd, 4.0 MiB/s wr, 170 op/s 2024-04-07T17:03:18.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:03:17 smithi144 bash[66654]: cluster 2024-04-07T17:03:16.660446+0000 mon.a (mon.0) 2478 : cluster [DBG] osdmap e1246: 8 total, 8 up, 8 in 2024-04-07T17:03:18.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:03:17 smithi144 bash[66654]: audit 2024-04-07T17:03:16.801751+0000 mon.a (mon.0) 2479 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:03:18.495 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestMigration.SnapsSnapExpandAfterPrepare (13961 ms) 2024-04-07T17:03:18.496 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestMigration.Clone 2024-04-07T17:03:18.583 INFO:tasks.workunit.client.0.smithi100.stdout:resize: 83886080 2024-04-07T17:03:18.613 INFO:tasks.workunit.client.0.smithi100.stdout:write: X 0~8388609 2024-04-07T17:03:18.693 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:03:18 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:03:18] "GET /metrics HTTP/1.1" 200 38922 "" "Prometheus/2.43.0" 2024-04-07T17:03:18.740 INFO:tasks.workunit.client.0.smithi100.stdout:write: X 16777218~8388609 2024-04-07T17:03:18.817 INFO:tasks.workunit.client.0.smithi100.stdout:snap_create: snap 2024-04-07T17:03:18.951 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:03:18 smithi100 bash[74942]: cluster 2024-04-07T17:03:17.684327+0000 mon.a (mon.0) 2480 : cluster [DBG] osdmap e1247: 8 total, 8 up, 8 in 2024-04-07T17:03:18.951 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:03:18 smithi100 bash[76202]: cluster 2024-04-07T17:03:17.684327+0000 mon.a (mon.0) 2480 : cluster [DBG] osdmap e1247: 8 total, 8 up, 8 in 2024-04-07T17:03:19.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:03:18 smithi144 bash[66654]: cluster 2024-04-07T17:03:17.684327+0000 mon.a (mon.0) 2480 : cluster [DBG] osdmap e1247: 8 total, 8 up, 8 in 2024-04-07T17:03:20.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:03:19 smithi144 bash[66654]: cluster 2024-04-07T17:03:17.812555+0000 mgr.y (mgr.25175) 1438 : cluster [DBG] pgmap v2260: 169 pgs: 169 active+clean; 737 MiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 33 MiB/s rd, 2.0 MiB/s wr, 446 op/s 2024-04-07T17:03:20.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:03:19 smithi100 bash[74942]: cluster 2024-04-07T17:03:17.812555+0000 mgr.y (mgr.25175) 1438 : cluster [DBG] pgmap v2260: 169 pgs: 169 active+clean; 737 MiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 33 MiB/s rd, 2.0 MiB/s wr, 446 op/s 2024-04-07T17:03:20.201 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:03:19 smithi100 bash[76202]: cluster 2024-04-07T17:03:17.812555+0000 mgr.y (mgr.25175) 1438 : cluster [DBG] pgmap v2260: 169 pgs: 169 active+clean; 737 MiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 33 MiB/s rd, 2.0 MiB/s wr, 446 op/s 2024-04-07T17:03:20.760 INFO:tasks.workunit.client.0.smithi100.stdout:snap_protect: snap 2024-04-07T17:03:20.795 INFO:tasks.workunit.client.0.smithi100.stdout:clone image105 -> image108 2024-04-07T17:03:21.067 INFO:tasks.workunit.client.0.smithi100.stdout:write: X 0~1000 2024-04-07T17:03:21.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:03:20 smithi144 bash[66654]: cluster 2024-04-07T17:03:19.704531+0000 mon.a (mon.0) 2481 : cluster [DBG] osdmap e1248: 8 total, 8 up, 8 in 2024-04-07T17:03:21.164 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 1010~1000 2024-04-07T17:03:21.170 INFO:tasks.workunit.client.0.smithi100.stdout:snap_create: snap 2024-04-07T17:03:21.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:03:20 smithi100 bash[74942]: cluster 2024-04-07T17:03:19.704531+0000 mon.a (mon.0) 2481 : cluster [DBG] osdmap e1248: 8 total, 8 up, 8 in 2024-04-07T17:03:21.201 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:03:20 smithi100 bash[76202]: cluster 2024-04-07T17:03:19.704531+0000 mon.a (mon.0) 2481 : cluster [DBG] osdmap e1248: 8 total, 8 up, 8 in 2024-04-07T17:03:22.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:03:21 smithi144 bash[66654]: cluster 2024-04-07T17:03:19.813273+0000 mgr.y (mgr.25175) 1439 : cluster [DBG] pgmap v2262: 169 pgs: 169 active+clean; 753 MiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 31 MiB/s rd, 6.8 MiB/s wr, 492 op/s 2024-04-07T17:03:22.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:03:21 smithi144 bash[66654]: cluster 2024-04-07T17:03:20.730480+0000 mon.a (mon.0) 2482 : cluster [DBG] osdmap e1249: 8 total, 8 up, 8 in 2024-04-07T17:03:22.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:03:21 smithi100 bash[74942]: cluster 2024-04-07T17:03:19.813273+0000 mgr.y (mgr.25175) 1439 : cluster [DBG] pgmap v2262: 169 pgs: 169 active+clean; 753 MiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 31 MiB/s rd, 6.8 MiB/s wr, 492 op/s 2024-04-07T17:03:22.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:03:21 smithi100 bash[74942]: cluster 2024-04-07T17:03:20.730480+0000 mon.a (mon.0) 2482 : cluster [DBG] osdmap e1249: 8 total, 8 up, 8 in 2024-04-07T17:03:22.202 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:03:21 smithi100 bash[76202]: cluster 2024-04-07T17:03:19.813273+0000 mgr.y (mgr.25175) 1439 : cluster [DBG] pgmap v2262: 169 pgs: 169 active+clean; 753 MiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 31 MiB/s rd, 6.8 MiB/s wr, 492 op/s 2024-04-07T17:03:22.202 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:03:21 smithi100 bash[76202]: cluster 2024-04-07T17:03:20.730480+0000 mon.a (mon.0) 2482 : cluster [DBG] osdmap e1249: 8 total, 8 up, 8 in 2024-04-07T17:03:22.615 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:03:22 smithi144 bash[95233]: ts=2024-04-07T17:03:22.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:03:22.791 INFO:tasks.workunit.client.0.smithi100.stdout:snap_protect: snap 2024-04-07T17:03:22.823 INFO:tasks.workunit.client.0.smithi100.stdout:clone image108 -> image110 2024-04-07T17:03:23.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:03:22 smithi144 bash[66654]: cluster 2024-04-07T17:03:21.734454+0000 mon.a (mon.0) 2483 : cluster [DBG] osdmap e1250: 8 total, 8 up, 8 in 2024-04-07T17:03:23.179 INFO:tasks.workunit.client.0.smithi100.stdout:write: X 1000~1000 2024-04-07T17:03:23.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:03:22 smithi100 bash[74942]: cluster 2024-04-07T17:03:21.734454+0000 mon.a (mon.0) 2483 : cluster [DBG] osdmap e1250: 8 total, 8 up, 8 in 2024-04-07T17:03:23.201 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:03:22 smithi100 bash[76202]: cluster 2024-04-07T17:03:21.734454+0000 mon.a (mon.0) 2483 : cluster [DBG] osdmap e1250: 8 total, 8 up, 8 in 2024-04-07T17:03:23.277 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 2010~1000 2024-04-07T17:03:23.283 INFO:tasks.workunit.client.0.smithi100.stdout:migration_prepare 2024-04-07T17:03:24.087 INFO:tasks.workunit.client.0.smithi100.stdout:migration_execute 2024-04-07T17:03:24.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:03:23 smithi144 bash[66654]: cluster 2024-04-07T17:03:21.814134+0000 mgr.y (mgr.25175) 1440 : cluster [DBG] pgmap v2265: 169 pgs: 169 active+clean; 757 MiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 25 MiB/s rd, 7.8 MiB/s wr, 125 op/s 2024-04-07T17:03:24.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:03:23 smithi144 bash[66654]: cluster 2024-04-07T17:03:22.758119+0000 mon.a (mon.0) 2484 : cluster [DBG] osdmap e1251: 8 total, 8 up, 8 in 2024-04-07T17:03:24.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:03:23 smithi100 bash[74942]: cluster 2024-04-07T17:03:21.814134+0000 mgr.y (mgr.25175) 1440 : cluster [DBG] pgmap v2265: 169 pgs: 169 active+clean; 757 MiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 25 MiB/s rd, 7.8 MiB/s wr, 125 op/s 2024-04-07T17:03:24.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:03:23 smithi100 bash[74942]: cluster 2024-04-07T17:03:22.758119+0000 mon.a (mon.0) 2484 : cluster [DBG] osdmap e1251: 8 total, 8 up, 8 in 2024-04-07T17:03:24.201 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:03:23 smithi100 bash[76202]: cluster 2024-04-07T17:03:21.814134+0000 mgr.y (mgr.25175) 1440 : cluster [DBG] pgmap v2265: 169 pgs: 169 active+clean; 757 MiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 25 MiB/s rd, 7.8 MiB/s wr, 125 op/s 2024-04-07T17:03:24.202 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:03:23 smithi100 bash[76202]: cluster 2024-04-07T17:03:22.758119+0000 mon.a (mon.0) 2484 : cluster [DBG] osdmap e1251: 8 total, 8 up, 8 in 2024-04-07T17:03:24.580 INFO:tasks.workunit.client.0.smithi100.stdout:migration_commit 2024-04-07T17:03:25.030 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestMigration.Clone (6535 ms) 2024-04-07T17:03:25.030 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestMigration.CloneParent 2024-04-07T17:03:25.118 INFO:tasks.workunit.client.0.smithi100.stdout:resize: 83886080 2024-04-07T17:03:25.150 INFO:tasks.workunit.client.0.smithi100.stdout:snap_create: snap 2024-04-07T17:03:26.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:03:25 smithi144 bash[66654]: cluster 2024-04-07T17:03:23.815079+0000 mgr.y (mgr.25175) 1441 : cluster [DBG] pgmap v2267: 169 pgs: 169 active+clean; 769 MiB data, 5.1 GiB used, 710 GiB / 715 GiB avail; 28 MiB/s rd, 6.8 MiB/s wr, 100 op/s 2024-04-07T17:03:26.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:03:25 smithi100 bash[74942]: cluster 2024-04-07T17:03:23.815079+0000 mgr.y (mgr.25175) 1441 : cluster [DBG] pgmap v2267: 169 pgs: 169 active+clean; 769 MiB data, 5.1 GiB used, 710 GiB / 715 GiB avail; 28 MiB/s rd, 6.8 MiB/s wr, 100 op/s 2024-04-07T17:03:26.201 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:03:25 smithi100 bash[76202]: cluster 2024-04-07T17:03:23.815079+0000 mgr.y (mgr.25175) 1441 : cluster [DBG] pgmap v2267: 169 pgs: 169 active+clean; 769 MiB data, 5.1 GiB used, 710 GiB / 715 GiB avail; 28 MiB/s rd, 6.8 MiB/s wr, 100 op/s 2024-04-07T17:03:26.819 INFO:tasks.workunit.client.0.smithi100.stdout:snap_protect: snap 2024-04-07T17:03:26.852 INFO:tasks.workunit.client.0.smithi100.stdout:clone image111 -> image114 2024-04-07T17:03:27.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:03:26 smithi144 bash[66654]: cluster 2024-04-07T17:03:25.761606+0000 mon.a (mon.0) 2485 : cluster [DBG] osdmap e1252: 8 total, 8 up, 8 in 2024-04-07T17:03:27.120 INFO:tasks.workunit.client.0.smithi100.stdout:migration_prepare 2024-04-07T17:03:27.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:03:26 smithi100 bash[74942]: cluster 2024-04-07T17:03:25.761606+0000 mon.a (mon.0) 2485 : cluster [DBG] osdmap e1252: 8 total, 8 up, 8 in 2024-04-07T17:03:27.201 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:03:26 smithi100 bash[76202]: cluster 2024-04-07T17:03:25.761606+0000 mon.a (mon.0) 2485 : cluster [DBG] osdmap e1252: 8 total, 8 up, 8 in 2024-04-07T17:03:27.549 INFO:tasks.workunit.client.0.smithi100.stdout:migration_abort 2024-04-07T17:03:27.928 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestMigration.CloneParent (2898 ms) 2024-04-07T17:03:27.928 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestMigration.CloneUpdateAfterPrepare 2024-04-07T17:03:28.013 INFO:tasks.workunit.client.0.smithi100.stdout:resize: 83886080 2024-04-07T17:03:28.048 INFO:tasks.workunit.client.0.smithi100.stdout:write: X 0~10 2024-04-07T17:03:28.072 INFO:tasks.workunit.client.0.smithi100.stdout:snap_create: snap 2024-04-07T17:03:28.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:03:27 smithi144 bash[66654]: cluster 2024-04-07T17:03:25.815625+0000 mgr.y (mgr.25175) 1442 : cluster [DBG] pgmap v2269: 169 pgs: 169 active+clean; 773 MiB data, 5.1 GiB used, 710 GiB / 715 GiB avail; 20 MiB/s rd, 4.7 MiB/s wr, 292 op/s 2024-04-07T17:03:28.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:03:27 smithi144 bash[66654]: cluster 2024-04-07T17:03:26.788685+0000 mon.a (mon.0) 2486 : cluster [DBG] osdmap e1253: 8 total, 8 up, 8 in 2024-04-07T17:03:28.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:03:27 smithi144 bash[66654]: cluster 2024-04-07T17:03:27.572465+0000 mon.a (mon.0) 2487 : cluster [DBG] osdmap e1254: 8 total, 8 up, 8 in 2024-04-07T17:03:28.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:03:27 smithi100 bash[74942]: cluster 2024-04-07T17:03:25.815625+0000 mgr.y (mgr.25175) 1442 : cluster [DBG] pgmap v2269: 169 pgs: 169 active+clean; 773 MiB data, 5.1 GiB used, 710 GiB / 715 GiB avail; 20 MiB/s rd, 4.7 MiB/s wr, 292 op/s 2024-04-07T17:03:28.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:03:27 smithi100 bash[74942]: cluster 2024-04-07T17:03:26.788685+0000 mon.a (mon.0) 2486 : cluster [DBG] osdmap e1253: 8 total, 8 up, 8 in 2024-04-07T17:03:28.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:03:27 smithi100 bash[74942]: cluster 2024-04-07T17:03:27.572465+0000 mon.a (mon.0) 2487 : cluster [DBG] osdmap e1254: 8 total, 8 up, 8 in 2024-04-07T17:03:28.202 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:03:27 smithi100 bash[76202]: cluster 2024-04-07T17:03:25.815625+0000 mgr.y (mgr.25175) 1442 : cluster [DBG] pgmap v2269: 169 pgs: 169 active+clean; 773 MiB data, 5.1 GiB used, 710 GiB / 715 GiB avail; 20 MiB/s rd, 4.7 MiB/s wr, 292 op/s 2024-04-07T17:03:28.202 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:03:27 smithi100 bash[76202]: cluster 2024-04-07T17:03:26.788685+0000 mon.a (mon.0) 2486 : cluster [DBG] osdmap e1253: 8 total, 8 up, 8 in 2024-04-07T17:03:28.202 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:03:27 smithi100 bash[76202]: cluster 2024-04-07T17:03:27.572465+0000 mon.a (mon.0) 2487 : cluster [DBG] osdmap e1254: 8 total, 8 up, 8 in 2024-04-07T17:03:28.701 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:03:28 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:03:28] "GET /metrics HTTP/1.1" 200 38922 "" "Prometheus/2.43.0" 2024-04-07T17:03:29.824 INFO:tasks.workunit.client.0.smithi100.stdout:snap_protect: snap 2024-04-07T17:03:29.865 INFO:tasks.workunit.client.0.smithi100.stdout:clone image115 -> image118 2024-04-07T17:03:30.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:03:29 smithi144 bash[66654]: cluster 2024-04-07T17:03:27.816349+0000 mgr.y (mgr.25175) 1443 : cluster [DBG] pgmap v2272: 169 pgs: 169 active+clean; 773 MiB data, 5.1 GiB used, 710 GiB / 715 GiB avail; 16 MiB/s rd, 823 KiB/s wr, 481 op/s 2024-04-07T17:03:30.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:03:29 smithi144 bash[66654]: cluster 2024-04-07T17:03:28.795839+0000 mon.a (mon.0) 2488 : cluster [DBG] osdmap e1255: 8 total, 8 up, 8 in 2024-04-07T17:03:30.128 INFO:tasks.workunit.client.0.smithi100.stdout:migration_prepare 2024-04-07T17:03:30.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:03:29 smithi100 bash[74942]: cluster 2024-04-07T17:03:27.816349+0000 mgr.y (mgr.25175) 1443 : cluster [DBG] pgmap v2272: 169 pgs: 169 active+clean; 773 MiB data, 5.1 GiB used, 710 GiB / 715 GiB avail; 16 MiB/s rd, 823 KiB/s wr, 481 op/s 2024-04-07T17:03:30.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:03:29 smithi100 bash[74942]: cluster 2024-04-07T17:03:28.795839+0000 mon.a (mon.0) 2488 : cluster [DBG] osdmap e1255: 8 total, 8 up, 8 in 2024-04-07T17:03:30.202 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:03:29 smithi100 bash[76202]: cluster 2024-04-07T17:03:27.816349+0000 mgr.y (mgr.25175) 1443 : cluster [DBG] pgmap v2272: 169 pgs: 169 active+clean; 773 MiB data, 5.1 GiB used, 710 GiB / 715 GiB avail; 16 MiB/s rd, 823 KiB/s wr, 481 op/s 2024-04-07T17:03:30.202 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:03:29 smithi100 bash[76202]: cluster 2024-04-07T17:03:28.795839+0000 mon.a (mon.0) 2488 : cluster [DBG] osdmap e1255: 8 total, 8 up, 8 in 2024-04-07T17:03:30.546 INFO:tasks.workunit.client.0.smithi100.stdout:write: Y 0~1 2024-04-07T17:03:30.616 INFO:tasks.workunit.client.0.smithi100.stdout:migration_execute 2024-04-07T17:03:30.827 INFO:tasks.workunit.client.0.smithi100.stdout:migration_commit 2024-04-07T17:03:31.099 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestMigration.CloneUpdateAfterPrepare (3171 ms) 2024-04-07T17:03:31.099 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestMigration.TriggerAssertSnapcSeq 2024-04-07T17:03:31.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:03:30 smithi144 bash[66654]: cluster 2024-04-07T17:03:29.798851+0000 mon.a (mon.0) 2489 : cluster [DBG] osdmap e1256: 8 total, 8 up, 8 in 2024-04-07T17:03:31.189 INFO:tasks.workunit.client.0.smithi100.stdout:resize: 83886080 2024-04-07T17:03:31.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:03:30 smithi100 bash[74942]: cluster 2024-04-07T17:03:29.798851+0000 mon.a (mon.0) 2489 : cluster [DBG] osdmap e1256: 8 total, 8 up, 8 in 2024-04-07T17:03:31.201 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:03:30 smithi100 bash[76202]: cluster 2024-04-07T17:03:29.798851+0000 mon.a (mon.0) 2489 : cluster [DBG] osdmap e1256: 8 total, 8 up, 8 in 2024-04-07T17:03:31.219 INFO:tasks.workunit.client.0.smithi100.stdout:write: A 41943040~10 2024-04-07T17:03:31.244 INFO:tasks.workunit.client.0.smithi100.stdout:snap_create: snap1 2024-04-07T17:03:32.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:03:31 smithi144 bash[66654]: cluster 2024-04-07T17:03:29.817196+0000 mgr.y (mgr.25175) 1444 : cluster [DBG] pgmap v2275: 169 pgs: 169 active+clean; 781 MiB data, 5.1 GiB used, 710 GiB / 715 GiB avail; 14 MiB/s rd, 3.0 MiB/s wr, 652 op/s 2024-04-07T17:03:32.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:03:31 smithi144 bash[66654]: audit 2024-04-07T17:03:31.802086+0000 mon.a (mon.0) 2490 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:03:32.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:03:31 smithi100 bash[74942]: cluster 2024-04-07T17:03:29.817196+0000 mgr.y (mgr.25175) 1444 : cluster [DBG] pgmap v2275: 169 pgs: 169 active+clean; 781 MiB data, 5.1 GiB used, 710 GiB / 715 GiB avail; 14 MiB/s rd, 3.0 MiB/s wr, 652 op/s 2024-04-07T17:03:32.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:03:31 smithi100 bash[74942]: audit 2024-04-07T17:03:31.802086+0000 mon.a (mon.0) 2490 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:03:32.202 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:03:31 smithi100 bash[76202]: cluster 2024-04-07T17:03:29.817196+0000 mgr.y (mgr.25175) 1444 : cluster [DBG] pgmap v2275: 169 pgs: 169 active+clean; 781 MiB data, 5.1 GiB used, 710 GiB / 715 GiB avail; 14 MiB/s rd, 3.0 MiB/s wr, 652 op/s 2024-04-07T17:03:32.202 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:03:31 smithi100 bash[76202]: audit 2024-04-07T17:03:31.802086+0000 mon.a (mon.0) 2490 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:03:32.615 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:03:32 smithi144 bash[95233]: ts=2024-04-07T17:03:32.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:03:32.863 INFO:tasks.workunit.client.0.smithi100.stdout:write: B 41943041~10 2024-04-07T17:03:32.883 INFO:tasks.workunit.client.0.smithi100.stdout:migration_prepare 2024-04-07T17:03:33.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:03:32 smithi144 bash[66654]: cluster 2024-04-07T17:03:31.818279+0000 mgr.y (mgr.25175) 1445 : cluster [DBG] pgmap v2276: 169 pgs: 169 active+clean; 781 MiB data, 5.1 GiB used, 710 GiB / 715 GiB avail; 12 MiB/s rd, 2.4 MiB/s wr, 591 op/s 2024-04-07T17:03:33.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:03:32 smithi144 bash[66654]: cluster 2024-04-07T17:03:31.828063+0000 mon.a (mon.0) 2491 : cluster [DBG] osdmap e1257: 8 total, 8 up, 8 in 2024-04-07T17:03:33.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:03:32 smithi100 bash[74942]: cluster 2024-04-07T17:03:31.818279+0000 mgr.y (mgr.25175) 1445 : cluster [DBG] pgmap v2276: 169 pgs: 169 active+clean; 781 MiB data, 5.1 GiB used, 710 GiB / 715 GiB avail; 12 MiB/s rd, 2.4 MiB/s wr, 591 op/s 2024-04-07T17:03:33.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:03:32 smithi100 bash[74942]: cluster 2024-04-07T17:03:31.828063+0000 mon.a (mon.0) 2491 : cluster [DBG] osdmap e1257: 8 total, 8 up, 8 in 2024-04-07T17:03:33.202 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:03:32 smithi100 bash[76202]: cluster 2024-04-07T17:03:31.818279+0000 mgr.y (mgr.25175) 1445 : cluster [DBG] pgmap v2276: 169 pgs: 169 active+clean; 781 MiB data, 5.1 GiB used, 710 GiB / 715 GiB avail; 12 MiB/s rd, 2.4 MiB/s wr, 591 op/s 2024-04-07T17:03:33.202 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:03:32 smithi100 bash[76202]: cluster 2024-04-07T17:03:31.828063+0000 mon.a (mon.0) 2491 : cluster [DBG] osdmap e1257: 8 total, 8 up, 8 in 2024-04-07T17:03:34.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:03:33 smithi144 bash[66654]: cluster 2024-04-07T17:03:32.836974+0000 mon.a (mon.0) 2492 : cluster [DBG] osdmap e1258: 8 total, 8 up, 8 in 2024-04-07T17:03:34.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:03:33 smithi100 bash[74942]: cluster 2024-04-07T17:03:32.836974+0000 mon.a (mon.0) 2492 : cluster [DBG] osdmap e1258: 8 total, 8 up, 8 in 2024-04-07T17:03:34.201 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:03:33 smithi100 bash[76202]: cluster 2024-04-07T17:03:32.836974+0000 mon.a (mon.0) 2492 : cluster [DBG] osdmap e1258: 8 total, 8 up, 8 in 2024-04-07T17:03:34.208 INFO:tasks.workunit.client.0.smithi100.stdout:write: C 41943042~10 2024-04-07T17:03:34.246 INFO:tasks.workunit.client.0.smithi100.stdout:snap_create: snap2 2024-04-07T17:03:35.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:03:34 smithi144 bash[66654]: cluster 2024-04-07T17:03:33.819205+0000 mgr.y (mgr.25175) 1446 : cluster [DBG] pgmap v2279: 169 pgs: 169 active+clean; 785 MiB data, 5.1 GiB used, 710 GiB / 715 GiB avail; 3.5 MiB/s rd, 1.6 MiB/s wr, 390 op/s 2024-04-07T17:03:35.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:03:34 smithi144 bash[66654]: cluster 2024-04-07T17:03:33.858693+0000 mon.a (mon.0) 2493 : cluster [DBG] osdmap e1259: 8 total, 8 up, 8 in 2024-04-07T17:03:35.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:03:34 smithi100 bash[74942]: cluster 2024-04-07T17:03:33.819205+0000 mgr.y (mgr.25175) 1446 : cluster [DBG] pgmap v2279: 169 pgs: 169 active+clean; 785 MiB data, 5.1 GiB used, 710 GiB / 715 GiB avail; 3.5 MiB/s rd, 1.6 MiB/s wr, 390 op/s 2024-04-07T17:03:35.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:03:34 smithi100 bash[74942]: cluster 2024-04-07T17:03:33.858693+0000 mon.a (mon.0) 2493 : cluster [DBG] osdmap e1259: 8 total, 8 up, 8 in 2024-04-07T17:03:35.202 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:03:34 smithi100 bash[76202]: cluster 2024-04-07T17:03:33.819205+0000 mgr.y (mgr.25175) 1446 : cluster [DBG] pgmap v2279: 169 pgs: 169 active+clean; 785 MiB data, 5.1 GiB used, 710 GiB / 715 GiB avail; 3.5 MiB/s rd, 1.6 MiB/s wr, 390 op/s 2024-04-07T17:03:35.202 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:03:34 smithi100 bash[76202]: cluster 2024-04-07T17:03:33.858693+0000 mon.a (mon.0) 2493 : cluster [DBG] osdmap e1259: 8 total, 8 up, 8 in 2024-04-07T17:03:35.899 INFO:tasks.workunit.client.0.smithi100.stdout:resize: 41943040 2024-04-07T17:03:35.976 INFO:tasks.workunit.client.0.smithi100.stdout:migration_execute 2024-04-07T17:03:36.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:03:35 smithi144 bash[66654]: cluster 2024-04-07T17:03:34.863480+0000 mon.a (mon.0) 2494 : cluster [DBG] osdmap e1260: 8 total, 8 up, 8 in 2024-04-07T17:03:36.144 INFO:tasks.workunit.client.0.smithi100.stdout:migration_commit 2024-04-07T17:03:36.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:03:35 smithi100 bash[74942]: cluster 2024-04-07T17:03:34.863480+0000 mon.a (mon.0) 2494 : cluster [DBG] osdmap e1260: 8 total, 8 up, 8 in 2024-04-07T17:03:36.201 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:03:35 smithi100 bash[76202]: cluster 2024-04-07T17:03:34.863480+0000 mon.a (mon.0) 2494 : cluster [DBG] osdmap e1260: 8 total, 8 up, 8 in 2024-04-07T17:03:37.121 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestMigration.TriggerAssertSnapcSeq (6022 ms) 2024-04-07T17:03:37.122 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestMigration.SnapTrimBeforePrepare 2024-04-07T17:03:37.139 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:03:36 smithi100 bash[74942]: cluster 2024-04-07T17:03:35.820172+0000 mgr.y (mgr.25175) 1447 : cluster [DBG] pgmap v2282: 169 pgs: 169 active+clean; 785 MiB data, 5.1 GiB used, 710 GiB / 715 GiB avail; 1.4 MiB/s rd, 1.0 MiB/s wr, 554 op/s 2024-04-07T17:03:37.139 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:03:36 smithi100 bash[74942]: cluster 2024-04-07T17:03:35.876468+0000 mon.a (mon.0) 2495 : cluster [DBG] osdmap e1261: 8 total, 8 up, 8 in 2024-04-07T17:03:37.189 INFO:tasks.workunit.client.0.smithi100.stdout:resize: 83886080 2024-04-07T17:03:37.215 INFO:tasks.workunit.client.0.smithi100.stdout:write: A 41943040~10 2024-04-07T17:03:37.237 INFO:tasks.workunit.client.0.smithi100.stdout:snap_create: snap1 2024-04-07T17:03:37.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:03:36 smithi144 bash[66654]: cluster 2024-04-07T17:03:35.820172+0000 mgr.y (mgr.25175) 1447 : cluster [DBG] pgmap v2282: 169 pgs: 169 active+clean; 785 MiB data, 5.1 GiB used, 710 GiB / 715 GiB avail; 1.4 MiB/s rd, 1.0 MiB/s wr, 554 op/s 2024-04-07T17:03:37.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:03:36 smithi144 bash[66654]: cluster 2024-04-07T17:03:35.876468+0000 mon.a (mon.0) 2495 : cluster [DBG] osdmap e1261: 8 total, 8 up, 8 in 2024-04-07T17:03:37.451 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:03:37 smithi100 bash[76202]: cluster 2024-04-07T17:03:35.820172+0000 mgr.y (mgr.25175) 1447 : cluster [DBG] pgmap v2282: 169 pgs: 169 active+clean; 785 MiB data, 5.1 GiB used, 710 GiB / 715 GiB avail; 1.4 MiB/s rd, 1.0 MiB/s wr, 554 op/s 2024-04-07T17:03:37.451 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:03:37 smithi100 bash[76202]: cluster 2024-04-07T17:03:35.876468+0000 mon.a (mon.0) 2495 : cluster [DBG] osdmap e1261: 8 total, 8 up, 8 in 2024-04-07T17:03:38.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:03:37 smithi100 bash[74942]: cluster 2024-04-07T17:03:36.882115+0000 mon.a (mon.0) 2496 : cluster [DBG] osdmap e1262: 8 total, 8 up, 8 in 2024-04-07T17:03:38.201 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:03:37 smithi100 bash[76202]: cluster 2024-04-07T17:03:36.882115+0000 mon.a (mon.0) 2496 : cluster [DBG] osdmap e1262: 8 total, 8 up, 8 in 2024-04-07T17:03:38.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:03:37 smithi144 bash[66654]: cluster 2024-04-07T17:03:36.882115+0000 mon.a (mon.0) 2496 : cluster [DBG] osdmap e1262: 8 total, 8 up, 8 in 2024-04-07T17:03:38.701 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:03:38 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:03:38] "GET /metrics HTTP/1.1" 200 38925 "" "Prometheus/2.43.0" 2024-04-07T17:03:38.931 INFO:tasks.workunit.client.0.smithi100.stdout:resize: 41943040 2024-04-07T17:03:38.986 INFO:tasks.workunit.client.0.smithi100.stdout:migration_prepare 2024-04-07T17:03:39.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:03:38 smithi100 bash[74942]: cluster 2024-04-07T17:03:37.821543+0000 mgr.y (mgr.25175) 1448 : cluster [DBG] pgmap v2285: 169 pgs: 169 active+clean; 785 MiB data, 5.1 GiB used, 710 GiB / 715 GiB avail; 325 KiB/s rd, 20 KiB/s wr, 414 op/s 2024-04-07T17:03:39.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:03:38 smithi100 bash[74942]: cluster 2024-04-07T17:03:37.893524+0000 mon.a (mon.0) 2497 : cluster [DBG] osdmap e1263: 8 total, 8 up, 8 in 2024-04-07T17:03:39.202 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:03:38 smithi100 bash[76202]: cluster 2024-04-07T17:03:37.821543+0000 mgr.y (mgr.25175) 1448 : cluster [DBG] pgmap v2285: 169 pgs: 169 active+clean; 785 MiB data, 5.1 GiB used, 710 GiB / 715 GiB avail; 325 KiB/s rd, 20 KiB/s wr, 414 op/s 2024-04-07T17:03:39.202 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:03:38 smithi100 bash[76202]: cluster 2024-04-07T17:03:37.893524+0000 mon.a (mon.0) 2497 : cluster [DBG] osdmap e1263: 8 total, 8 up, 8 in 2024-04-07T17:03:39.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:03:38 smithi144 bash[66654]: cluster 2024-04-07T17:03:37.821543+0000 mgr.y (mgr.25175) 1448 : cluster [DBG] pgmap v2285: 169 pgs: 169 active+clean; 785 MiB data, 5.1 GiB used, 710 GiB / 715 GiB avail; 325 KiB/s rd, 20 KiB/s wr, 414 op/s 2024-04-07T17:03:39.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:03:38 smithi144 bash[66654]: cluster 2024-04-07T17:03:37.893524+0000 mon.a (mon.0) 2497 : cluster [DBG] osdmap e1263: 8 total, 8 up, 8 in 2024-04-07T17:03:40.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:03:39 smithi100 bash[74942]: cluster 2024-04-07T17:03:38.908726+0000 mon.a (mon.0) 2498 : cluster [DBG] osdmap e1264: 8 total, 8 up, 8 in 2024-04-07T17:03:40.201 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:03:39 smithi100 bash[76202]: cluster 2024-04-07T17:03:38.908726+0000 mon.a (mon.0) 2498 : cluster [DBG] osdmap e1264: 8 total, 8 up, 8 in 2024-04-07T17:03:40.234 INFO:tasks.workunit.client.0.smithi100.stdout:resize: 83886080 2024-04-07T17:03:40.266 INFO:tasks.workunit.client.0.smithi100.stdout:snap_create: snap3 2024-04-07T17:03:40.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:03:39 smithi144 bash[66654]: cluster 2024-04-07T17:03:38.908726+0000 mon.a (mon.0) 2498 : cluster [DBG] osdmap e1264: 8 total, 8 up, 8 in 2024-04-07T17:03:41.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:03:40 smithi100 bash[74942]: cluster 2024-04-07T17:03:39.822648+0000 mgr.y (mgr.25175) 1449 : cluster [DBG] pgmap v2288: 169 pgs: 169 active+clean; 785 MiB data, 5.1 GiB used, 710 GiB / 715 GiB avail; 197 KiB/s rd, 21 KiB/s wr, 272 op/s 2024-04-07T17:03:41.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:03:40 smithi100 bash[74942]: cluster 2024-04-07T17:03:39.933341+0000 mon.a (mon.0) 2499 : cluster [DBG] osdmap e1265: 8 total, 8 up, 8 in 2024-04-07T17:03:41.202 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:03:40 smithi100 bash[76202]: cluster 2024-04-07T17:03:39.822648+0000 mgr.y (mgr.25175) 1449 : cluster [DBG] pgmap v2288: 169 pgs: 169 active+clean; 785 MiB data, 5.1 GiB used, 710 GiB / 715 GiB avail; 197 KiB/s rd, 21 KiB/s wr, 272 op/s 2024-04-07T17:03:41.202 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:03:40 smithi100 bash[76202]: cluster 2024-04-07T17:03:39.933341+0000 mon.a (mon.0) 2499 : cluster [DBG] osdmap e1265: 8 total, 8 up, 8 in 2024-04-07T17:03:41.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:03:40 smithi144 bash[66654]: cluster 2024-04-07T17:03:39.822648+0000 mgr.y (mgr.25175) 1449 : cluster [DBG] pgmap v2288: 169 pgs: 169 active+clean; 785 MiB data, 5.1 GiB used, 710 GiB / 715 GiB avail; 197 KiB/s rd, 21 KiB/s wr, 272 op/s 2024-04-07T17:03:41.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:03:40 smithi144 bash[66654]: cluster 2024-04-07T17:03:39.933341+0000 mon.a (mon.0) 2499 : cluster [DBG] osdmap e1265: 8 total, 8 up, 8 in 2024-04-07T17:03:41.984 INFO:tasks.workunit.client.0.smithi100.stdout:write: B 41943040~10 2024-04-07T17:03:41.999 INFO:tasks.workunit.client.0.smithi100.stdout:snap_create: snap4 2024-04-07T17:03:42.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:03:41 smithi100 bash[74942]: cluster 2024-04-07T17:03:40.940229+0000 mon.a (mon.0) 2500 : cluster [DBG] osdmap e1266: 8 total, 8 up, 8 in 2024-04-07T17:03:42.201 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:03:41 smithi100 bash[76202]: cluster 2024-04-07T17:03:40.940229+0000 mon.a (mon.0) 2500 : cluster [DBG] osdmap e1266: 8 total, 8 up, 8 in 2024-04-07T17:03:42.314 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:03:41 smithi144 bash[66654]: cluster 2024-04-07T17:03:40.940229+0000 mon.a (mon.0) 2500 : cluster [DBG] osdmap e1266: 8 total, 8 up, 8 in 2024-04-07T17:03:42.615 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:03:42 smithi144 bash[95233]: ts=2024-04-07T17:03:42.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:03:43.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:03:42 smithi100 bash[74942]: cluster 2024-04-07T17:03:41.823438+0000 mgr.y (mgr.25175) 1450 : cluster [DBG] pgmap v2291: 169 pgs: 169 active+clean; 785 MiB data, 5.1 GiB used, 710 GiB / 715 GiB avail; 172 KiB/s rd, 15 KiB/s wr, 228 op/s 2024-04-07T17:03:43.202 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:03:42 smithi100 bash[74942]: cluster 2024-04-07T17:03:41.952159+0000 mon.a (mon.0) 2501 : cluster [DBG] osdmap e1267: 8 total, 8 up, 8 in 2024-04-07T17:03:43.202 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:03:42 smithi100 bash[74942]: cluster 2024-04-07T17:03:42.583011+0000 mon.a (mon.0) 2502 : cluster [DBG] osdmap e1268: 8 total, 8 up, 8 in 2024-04-07T17:03:43.202 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:03:42 smithi100 bash[76202]: cluster 2024-04-07T17:03:41.823438+0000 mgr.y (mgr.25175) 1450 : cluster [DBG] pgmap v2291: 169 pgs: 169 active+clean; 785 MiB data, 5.1 GiB used, 710 GiB / 715 GiB avail; 172 KiB/s rd, 15 KiB/s wr, 228 op/s 2024-04-07T17:03:43.202 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:03:42 smithi100 bash[76202]: cluster 2024-04-07T17:03:41.952159+0000 mon.a (mon.0) 2501 : cluster [DBG] osdmap e1267: 8 total, 8 up, 8 in 2024-04-07T17:03:43.202 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:03:42 smithi100 bash[76202]: cluster 2024-04-07T17:03:42.583011+0000 mon.a (mon.0) 2502 : cluster [DBG] osdmap e1268: 8 total, 8 up, 8 in 2024-04-07T17:03:43.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:03:42 smithi144 bash[66654]: cluster 2024-04-07T17:03:41.823438+0000 mgr.y (mgr.25175) 1450 : cluster [DBG] pgmap v2291: 169 pgs: 169 active+clean; 785 MiB data, 5.1 GiB used, 710 GiB / 715 GiB avail; 172 KiB/s rd, 15 KiB/s wr, 228 op/s 2024-04-07T17:03:43.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:03:42 smithi144 bash[66654]: cluster 2024-04-07T17:03:41.952159+0000 mon.a (mon.0) 2501 : cluster [DBG] osdmap e1267: 8 total, 8 up, 8 in 2024-04-07T17:03:43.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:03:42 smithi144 bash[66654]: cluster 2024-04-07T17:03:42.583011+0000 mon.a (mon.0) 2502 : cluster [DBG] osdmap e1268: 8 total, 8 up, 8 in 2024-04-07T17:03:43.610 INFO:tasks.workunit.client.0.smithi100.stdout:resize: 41943040 2024-04-07T17:03:43.667 INFO:tasks.workunit.client.0.smithi100.stdout:migration_execute 2024-04-07T17:03:43.836 INFO:tasks.workunit.client.0.smithi100.stdout:migration_commit 2024-04-07T17:03:44.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:03:44 smithi144 bash[66654]: cluster 2024-04-07T17:03:43.583134+0000 mon.a (mon.0) 2503 : cluster [DBG] osdmap e1269: 8 total, 8 up, 8 in 2024-04-07T17:03:44.910 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestMigration.SnapTrimBeforePrepare (7789 ms) 2024-04-07T17:03:44.911 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestMigration.AbortInUseImage 2024-04-07T17:03:44.951 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:03:44 smithi100 bash[74942]: cluster 2024-04-07T17:03:43.583134+0000 mon.a (mon.0) 2503 : cluster [DBG] osdmap e1269: 8 total, 8 up, 8 in 2024-04-07T17:03:44.951 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:03:44 smithi100 bash[76202]: cluster 2024-04-07T17:03:43.583134+0000 mon.a (mon.0) 2503 : cluster [DBG] osdmap e1269: 8 total, 8 up, 8 in 2024-04-07T17:03:44.993 INFO:tasks.workunit.client.0.smithi100.stdout:resize: 83886080 2024-04-07T17:03:45.025 INFO:tasks.workunit.client.0.smithi100.stdout:migration_prepare 2024-04-07T17:03:45.522 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestMigration.AbortInUseImage (611 ms) 2024-04-07T17:03:45.522 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestMigration.AbortWithoutSnapshots 2024-04-07T17:03:45.608 INFO:tasks.workunit.client.0.smithi100.stdout:resize: 83886080 2024-04-07T17:03:45.639 INFO:tasks.workunit.client.0.smithi100.stdout:write: 1 0~8388609 2024-04-07T17:03:45.744 INFO:tasks.workunit.client.0.smithi100.stdout:write: 1 16777218~8388609 2024-04-07T17:03:45.820 INFO:tasks.workunit.client.0.smithi100.stdout:migration_prepare 2024-04-07T17:03:45.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:03:45 smithi144 bash[66654]: cluster 2024-04-07T17:03:43.824192+0000 mgr.y (mgr.25175) 1451 : cluster [DBG] pgmap v2295: 169 pgs: 169 active+clean; 785 MiB data, 5.1 GiB used, 710 GiB / 715 GiB avail; 133 KiB/s rd, 12 KiB/s wr, 174 op/s 2024-04-07T17:03:45.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:03:45 smithi144 bash[66654]: cluster 2024-04-07T17:03:44.593511+0000 mon.a (mon.0) 2504 : cluster [DBG] osdmap e1270: 8 total, 8 up, 8 in 2024-04-07T17:03:45.951 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:03:45 smithi100 bash[74942]: cluster 2024-04-07T17:03:43.824192+0000 mgr.y (mgr.25175) 1451 : cluster [DBG] pgmap v2295: 169 pgs: 169 active+clean; 785 MiB data, 5.1 GiB used, 710 GiB / 715 GiB avail; 133 KiB/s rd, 12 KiB/s wr, 174 op/s 2024-04-07T17:03:45.951 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:03:45 smithi100 bash[74942]: cluster 2024-04-07T17:03:44.593511+0000 mon.a (mon.0) 2504 : cluster [DBG] osdmap e1270: 8 total, 8 up, 8 in 2024-04-07T17:03:45.952 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:03:45 smithi100 bash[76202]: cluster 2024-04-07T17:03:43.824192+0000 mgr.y (mgr.25175) 1451 : cluster [DBG] pgmap v2295: 169 pgs: 169 active+clean; 785 MiB data, 5.1 GiB used, 710 GiB / 715 GiB avail; 133 KiB/s rd, 12 KiB/s wr, 174 op/s 2024-04-07T17:03:45.952 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:03:45 smithi100 bash[76202]: cluster 2024-04-07T17:03:44.593511+0000 mon.a (mon.0) 2504 : cluster [DBG] osdmap e1270: 8 total, 8 up, 8 in 2024-04-07T17:03:46.276 INFO:tasks.workunit.client.0.smithi100.stdout:write: 1 0~8388609 2024-04-07T17:03:46.347 INFO:tasks.workunit.client.0.smithi100.stdout:write: 1 16777218~8388609 2024-04-07T17:03:46.445 INFO:tasks.workunit.client.0.smithi100.stdout:migration_abort 2024-04-07T17:03:46.838 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestMigration.AbortWithoutSnapshots (1317 ms) 2024-04-07T17:03:46.839 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestMigration.AbortWithSnapshots 2024-04-07T17:03:46.929 INFO:tasks.workunit.client.0.smithi100.stdout:resize: 83886080 2024-04-07T17:03:46.960 INFO:tasks.workunit.client.0.smithi100.stdout:write: 1 0~8388609 2024-04-07T17:03:47.063 INFO:tasks.workunit.client.0.smithi100.stdout:snap_create: snap1 2024-04-07T17:03:47.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:03:47 smithi144 bash[66654]: cluster 2024-04-07T17:03:45.825174+0000 mgr.y (mgr.25175) 1452 : cluster [DBG] pgmap v2297: 169 pgs: 169 active+clean; 785 MiB data, 5.1 GiB used, 710 GiB / 715 GiB avail; 146 KiB/s rd, 18 KiB/s wr, 197 op/s 2024-04-07T17:03:47.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:03:47 smithi144 bash[66654]: audit 2024-04-07T17:03:46.802690+0000 mon.a (mon.0) 2505 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:03:47.951 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:03:47 smithi100 bash[74942]: cluster 2024-04-07T17:03:45.825174+0000 mgr.y (mgr.25175) 1452 : cluster [DBG] pgmap v2297: 169 pgs: 169 active+clean; 785 MiB data, 5.1 GiB used, 710 GiB / 715 GiB avail; 146 KiB/s rd, 18 KiB/s wr, 197 op/s 2024-04-07T17:03:47.951 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:03:47 smithi100 bash[74942]: audit 2024-04-07T17:03:46.802690+0000 mon.a (mon.0) 2505 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:03:47.952 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:03:47 smithi100 bash[76202]: cluster 2024-04-07T17:03:45.825174+0000 mgr.y (mgr.25175) 1452 : cluster [DBG] pgmap v2297: 169 pgs: 169 active+clean; 785 MiB data, 5.1 GiB used, 710 GiB / 715 GiB avail; 146 KiB/s rd, 18 KiB/s wr, 197 op/s 2024-04-07T17:03:47.952 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:03:47 smithi100 bash[76202]: audit 2024-04-07T17:03:46.802690+0000 mon.a (mon.0) 2505 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:03:48.659 INFO:tasks.workunit.client.0.smithi100.stdout:write: 1 8388609~8388609 2024-04-07T17:03:48.701 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:03:48 smithi100 bash[74942]: cluster 2024-04-07T17:03:47.615121+0000 mon.a (mon.0) 2506 : cluster [DBG] osdmap e1271: 8 total, 8 up, 8 in 2024-04-07T17:03:48.702 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:03:48 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:03:48] "GET /metrics HTTP/1.1" 200 38922 "" "Prometheus/2.43.0" 2024-04-07T17:03:48.702 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:03:48 smithi100 bash[76202]: cluster 2024-04-07T17:03:47.615121+0000 mon.a (mon.0) 2506 : cluster [DBG] osdmap e1271: 8 total, 8 up, 8 in 2024-04-07T17:03:48.706 INFO:tasks.workunit.client.0.smithi100.stdout:write: X 0~1000 2024-04-07T17:03:48.706 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 1010~1000 2024-04-07T17:03:48.706 INFO:tasks.workunit.client.0.smithi100.stdout:snap_create: snap2 2024-04-07T17:03:49.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:03:48 smithi144 bash[66654]: cluster 2024-04-07T17:03:47.615121+0000 mon.a (mon.0) 2506 : cluster [DBG] osdmap e1271: 8 total, 8 up, 8 in 2024-04-07T17:03:49.952 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:03:49 smithi100 bash[76202]: cluster 2024-04-07T17:03:47.825993+0000 mgr.y (mgr.25175) 1453 : cluster [DBG] pgmap v2299: 169 pgs: 169 active+clean; 793 MiB data, 5.2 GiB used, 710 GiB / 715 GiB avail; 4.0 MiB/s rd, 3.8 MiB/s wr, 239 op/s 2024-04-07T17:03:49.952 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:03:49 smithi100 bash[76202]: cluster 2024-04-07T17:03:48.626307+0000 mon.a (mon.0) 2507 : cluster [DBG] osdmap e1272: 8 total, 8 up, 8 in 2024-04-07T17:03:49.952 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:03:49 smithi100 bash[76202]: audit 2024-04-07T17:03:48.734085+0000 mon.a (mon.0) 2508 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:03:49.952 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:03:49 smithi100 bash[76202]: audit 2024-04-07T17:03:49.463764+0000 mon.a (mon.0) 2509 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:03:49.952 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:03:49 smithi100 bash[76202]: audit 2024-04-07T17:03:49.465786+0000 mon.a (mon.0) 2510 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:03:49.952 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:03:49 smithi100 bash[76202]: audit 2024-04-07T17:03:49.476245+0000 mon.a (mon.0) 2511 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:03:49.952 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:03:49 smithi100 bash[74942]: cluster 2024-04-07T17:03:47.825993+0000 mgr.y (mgr.25175) 1453 : cluster [DBG] pgmap v2299: 169 pgs: 169 active+clean; 793 MiB data, 5.2 GiB used, 710 GiB / 715 GiB avail; 4.0 MiB/s rd, 3.8 MiB/s wr, 239 op/s 2024-04-07T17:03:49.952 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:03:49 smithi100 bash[74942]: cluster 2024-04-07T17:03:48.626307+0000 mon.a (mon.0) 2507 : cluster [DBG] osdmap e1272: 8 total, 8 up, 8 in 2024-04-07T17:03:49.952 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:03:49 smithi100 bash[74942]: audit 2024-04-07T17:03:48.734085+0000 mon.a (mon.0) 2508 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:03:49.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:03:49 smithi100 bash[74942]: audit 2024-04-07T17:03:49.463764+0000 mon.a (mon.0) 2509 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:03:49.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:03:49 smithi100 bash[74942]: audit 2024-04-07T17:03:49.465786+0000 mon.a (mon.0) 2510 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:03:49.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:03:49 smithi100 bash[74942]: audit 2024-04-07T17:03:49.476245+0000 mon.a (mon.0) 2511 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:03:50.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:03:49 smithi144 bash[66654]: cluster 2024-04-07T17:03:47.825993+0000 mgr.y (mgr.25175) 1453 : cluster [DBG] pgmap v2299: 169 pgs: 169 active+clean; 793 MiB data, 5.2 GiB used, 710 GiB / 715 GiB avail; 4.0 MiB/s rd, 3.8 MiB/s wr, 239 op/s 2024-04-07T17:03:50.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:03:49 smithi144 bash[66654]: cluster 2024-04-07T17:03:48.626307+0000 mon.a (mon.0) 2507 : cluster [DBG] osdmap e1272: 8 total, 8 up, 8 in 2024-04-07T17:03:50.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:03:49 smithi144 bash[66654]: audit 2024-04-07T17:03:48.734085+0000 mon.a (mon.0) 2508 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:03:50.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:03:49 smithi144 bash[66654]: audit 2024-04-07T17:03:49.463764+0000 mon.a (mon.0) 2509 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:03:50.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:03:49 smithi144 bash[66654]: audit 2024-04-07T17:03:49.465786+0000 mon.a (mon.0) 2510 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:03:50.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:03:49 smithi144 bash[66654]: audit 2024-04-07T17:03:49.476245+0000 mon.a (mon.0) 2511 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:03:50.664 INFO:tasks.workunit.client.0.smithi100.stdout:write: X 8388609~1000 2024-04-07T17:03:50.683 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 16777228~1000 2024-04-07T17:03:50.683 INFO:tasks.workunit.client.0.smithi100.stdout:resize: 167772160 2024-04-07T17:03:50.710 INFO:tasks.workunit.client.0.smithi100.stdout:write: 2 83886079~8388609 2024-04-07T17:03:50.774 INFO:tasks.workunit.client.0.smithi100.stdout:snap_create: snap3 2024-04-07T17:03:50.951 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:03:50 smithi100 bash[74942]: cluster 2024-04-07T17:03:49.636618+0000 mon.a (mon.0) 2512 : cluster [DBG] osdmap e1273: 8 total, 8 up, 8 in 2024-04-07T17:03:50.952 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:03:50 smithi100 bash[76202]: cluster 2024-04-07T17:03:49.636618+0000 mon.a (mon.0) 2512 : cluster [DBG] osdmap e1273: 8 total, 8 up, 8 in 2024-04-07T17:03:51.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:03:50 smithi144 bash[66654]: cluster 2024-04-07T17:03:49.636618+0000 mon.a (mon.0) 2512 : cluster [DBG] osdmap e1273: 8 total, 8 up, 8 in 2024-04-07T17:03:51.951 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:03:51 smithi100 bash[74942]: cluster 2024-04-07T17:03:49.826880+0000 mgr.y (mgr.25175) 1454 : cluster [DBG] pgmap v2302: 169 pgs: 169 active+clean; 821 MiB data, 5.3 GiB used, 710 GiB / 715 GiB avail; 13 MiB/s rd, 15 MiB/s wr, 413 op/s 2024-04-07T17:03:51.952 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:03:51 smithi100 bash[74942]: cluster 2024-04-07T17:03:50.647957+0000 mon.a (mon.0) 2513 : cluster [DBG] osdmap e1274: 8 total, 8 up, 8 in 2024-04-07T17:03:51.952 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:03:51 smithi100 bash[76202]: cluster 2024-04-07T17:03:49.826880+0000 mgr.y (mgr.25175) 1454 : cluster [DBG] pgmap v2302: 169 pgs: 169 active+clean; 821 MiB data, 5.3 GiB used, 710 GiB / 715 GiB avail; 13 MiB/s rd, 15 MiB/s wr, 413 op/s 2024-04-07T17:03:51.952 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:03:51 smithi100 bash[76202]: cluster 2024-04-07T17:03:50.647957+0000 mon.a (mon.0) 2513 : cluster [DBG] osdmap e1274: 8 total, 8 up, 8 in 2024-04-07T17:03:52.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:03:51 smithi144 bash[66654]: cluster 2024-04-07T17:03:49.826880+0000 mgr.y (mgr.25175) 1454 : cluster [DBG] pgmap v2302: 169 pgs: 169 active+clean; 821 MiB data, 5.3 GiB used, 710 GiB / 715 GiB avail; 13 MiB/s rd, 15 MiB/s wr, 413 op/s 2024-04-07T17:03:52.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:03:51 smithi144 bash[66654]: cluster 2024-04-07T17:03:50.647957+0000 mon.a (mon.0) 2513 : cluster [DBG] osdmap e1274: 8 total, 8 up, 8 in 2024-04-07T17:03:52.616 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:03:52 smithi144 bash[95233]: ts=2024-04-07T17:03:52.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:03:52.621 INFO:tasks.workunit.client.0.smithi100.stdout:resize: 83886080 2024-04-07T17:03:52.685 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 83886079~1 2024-04-07T17:03:52.707 INFO:tasks.workunit.client.0.smithi100.stdout:migration_prepare 2024-04-07T17:03:52.951 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:03:52 smithi100 bash[74942]: cluster 2024-04-07T17:03:51.654302+0000 mon.a (mon.0) 2514 : cluster [DBG] osdmap e1275: 8 total, 8 up, 8 in 2024-04-07T17:03:52.952 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:03:52 smithi100 bash[74942]: cluster 2024-04-07T17:03:52.588046+0000 mon.a (mon.0) 2515 : cluster [DBG] osdmap e1276: 8 total, 8 up, 8 in 2024-04-07T17:03:52.952 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:03:52 smithi100 bash[76202]: cluster 2024-04-07T17:03:51.654302+0000 mon.a (mon.0) 2514 : cluster [DBG] osdmap e1275: 8 total, 8 up, 8 in 2024-04-07T17:03:52.952 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:03:52 smithi100 bash[76202]: cluster 2024-04-07T17:03:52.588046+0000 mon.a (mon.0) 2515 : cluster [DBG] osdmap e1276: 8 total, 8 up, 8 in 2024-04-07T17:03:53.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:03:52 smithi144 bash[66654]: cluster 2024-04-07T17:03:51.654302+0000 mon.a (mon.0) 2514 : cluster [DBG] osdmap e1275: 8 total, 8 up, 8 in 2024-04-07T17:03:53.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:03:52 smithi144 bash[66654]: cluster 2024-04-07T17:03:52.588046+0000 mon.a (mon.0) 2515 : cluster [DBG] osdmap e1276: 8 total, 8 up, 8 in 2024-04-07T17:03:53.951 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:03:53 smithi100 bash[74942]: cluster 2024-04-07T17:03:51.827854+0000 mgr.y (mgr.25175) 1455 : cluster [DBG] pgmap v2305: 169 pgs: 169 active+clean; 833 MiB data, 5.3 GiB used, 710 GiB / 715 GiB avail; 16 MiB/s rd, 20 MiB/s wr, 434 op/s 2024-04-07T17:03:53.952 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:03:53 smithi100 bash[76202]: cluster 2024-04-07T17:03:51.827854+0000 mgr.y (mgr.25175) 1455 : cluster [DBG] pgmap v2305: 169 pgs: 169 active+clean; 833 MiB data, 5.3 GiB used, 710 GiB / 715 GiB avail; 16 MiB/s rd, 20 MiB/s wr, 434 op/s 2024-04-07T17:03:54.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:03:53 smithi144 bash[66654]: cluster 2024-04-07T17:03:51.827854+0000 mgr.y (mgr.25175) 1455 : cluster [DBG] pgmap v2305: 169 pgs: 169 active+clean; 833 MiB data, 5.3 GiB used, 710 GiB / 715 GiB avail; 16 MiB/s rd, 20 MiB/s wr, 434 op/s 2024-04-07T17:03:54.951 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:03:54 smithi100 bash[74942]: cluster 2024-04-07T17:03:53.665558+0000 mon.a (mon.0) 2516 : cluster [DBG] osdmap e1277: 8 total, 8 up, 8 in 2024-04-07T17:03:54.951 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:03:54 smithi100 bash[76202]: cluster 2024-04-07T17:03:53.665558+0000 mon.a (mon.0) 2516 : cluster [DBG] osdmap e1277: 8 total, 8 up, 8 in 2024-04-07T17:03:55.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:03:54 smithi144 bash[66654]: cluster 2024-04-07T17:03:53.665558+0000 mon.a (mon.0) 2516 : cluster [DBG] osdmap e1277: 8 total, 8 up, 8 in 2024-04-07T17:03:55.951 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:03:55 smithi100 bash[74942]: cluster 2024-04-07T17:03:53.828604+0000 mgr.y (mgr.25175) 1456 : cluster [DBG] pgmap v2308: 169 pgs: 169 active+clean; 853 MiB data, 5.4 GiB used, 710 GiB / 715 GiB avail; 5.1 MiB/s rd, 10 MiB/s wr, 119 op/s 2024-04-07T17:03:55.951 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:03:55 smithi100 bash[74942]: cluster 2024-04-07T17:03:54.690878+0000 mon.a (mon.0) 2517 : cluster [DBG] osdmap e1278: 8 total, 8 up, 8 in 2024-04-07T17:03:55.952 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:03:55 smithi100 bash[76202]: cluster 2024-04-07T17:03:53.828604+0000 mgr.y (mgr.25175) 1456 : cluster [DBG] pgmap v2308: 169 pgs: 169 active+clean; 853 MiB data, 5.4 GiB used, 710 GiB / 715 GiB avail; 5.1 MiB/s rd, 10 MiB/s wr, 119 op/s 2024-04-07T17:03:55.952 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:03:55 smithi100 bash[76202]: cluster 2024-04-07T17:03:54.690878+0000 mon.a (mon.0) 2517 : cluster [DBG] osdmap e1278: 8 total, 8 up, 8 in 2024-04-07T17:03:56.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:03:55 smithi144 bash[66654]: cluster 2024-04-07T17:03:53.828604+0000 mgr.y (mgr.25175) 1456 : cluster [DBG] pgmap v2308: 169 pgs: 169 active+clean; 853 MiB data, 5.4 GiB used, 710 GiB / 715 GiB avail; 5.1 MiB/s rd, 10 MiB/s wr, 119 op/s 2024-04-07T17:03:56.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:03:55 smithi144 bash[66654]: cluster 2024-04-07T17:03:54.690878+0000 mon.a (mon.0) 2517 : cluster [DBG] osdmap e1278: 8 total, 8 up, 8 in 2024-04-07T17:03:56.672 INFO:tasks.workunit.client.0.smithi100.stdout:write: 1 0~8388609 2024-04-07T17:03:56.825 INFO:tasks.workunit.client.0.smithi100.stdout:write: 1 16777218~8388609 2024-04-07T17:03:57.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:03:56 smithi144 bash[66654]: cluster 2024-04-07T17:03:55.700785+0000 mon.a (mon.0) 2518 : cluster [DBG] osdmap e1279: 8 total, 8 up, 8 in 2024-04-07T17:03:57.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:03:56 smithi100 bash[74942]: cluster 2024-04-07T17:03:55.700785+0000 mon.a (mon.0) 2518 : cluster [DBG] osdmap e1279: 8 total, 8 up, 8 in 2024-04-07T17:03:57.202 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:03:56 smithi100 bash[76202]: cluster 2024-04-07T17:03:55.700785+0000 mon.a (mon.0) 2518 : cluster [DBG] osdmap e1279: 8 total, 8 up, 8 in 2024-04-07T17:03:57.758 INFO:tasks.workunit.client.0.smithi100.stdout:write: 1 0~8388609 2024-04-07T17:03:57.806 INFO:tasks.workunit.client.0.smithi100.stdout:write: 1 16777218~8388609 2024-04-07T17:03:57.925 INFO:tasks.workunit.client.0.smithi100.stdout:migration_abort 2024-04-07T17:03:58.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:03:57 smithi144 bash[66654]: cluster 2024-04-07T17:03:55.829391+0000 mgr.y (mgr.25175) 1457 : cluster [DBG] pgmap v2311: 169 pgs: 169 active+clean; 873 MiB data, 5.4 GiB used, 710 GiB / 715 GiB avail; 116 KiB/s rd, 8.0 MiB/s wr, 168 op/s 2024-04-07T17:03:58.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:03:57 smithi100 bash[74942]: cluster 2024-04-07T17:03:55.829391+0000 mgr.y (mgr.25175) 1457 : cluster [DBG] pgmap v2311: 169 pgs: 169 active+clean; 873 MiB data, 5.4 GiB used, 710 GiB / 715 GiB avail; 116 KiB/s rd, 8.0 MiB/s wr, 168 op/s 2024-04-07T17:03:58.202 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:03:57 smithi100 bash[76202]: cluster 2024-04-07T17:03:55.829391+0000 mgr.y (mgr.25175) 1457 : cluster [DBG] pgmap v2311: 169 pgs: 169 active+clean; 873 MiB data, 5.4 GiB used, 710 GiB / 715 GiB avail; 116 KiB/s rd, 8.0 MiB/s wr, 168 op/s 2024-04-07T17:03:58.701 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:03:58 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:03:58] "GET /metrics HTTP/1.1" 200 38922 "" "Prometheus/2.43.0" 2024-04-07T17:03:59.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:03:58 smithi144 bash[66654]: cluster 2024-04-07T17:03:57.719386+0000 mon.a (mon.0) 2519 : cluster [DBG] osdmap e1280: 8 total, 8 up, 8 in 2024-04-07T17:03:59.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:03:58 smithi100 bash[74942]: cluster 2024-04-07T17:03:57.719386+0000 mon.a (mon.0) 2519 : cluster [DBG] osdmap e1280: 8 total, 8 up, 8 in 2024-04-07T17:03:59.202 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:03:58 smithi100 bash[76202]: cluster 2024-04-07T17:03:57.719386+0000 mon.a (mon.0) 2519 : cluster [DBG] osdmap e1280: 8 total, 8 up, 8 in 2024-04-07T17:04:00.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:03:59 smithi144 bash[66654]: cluster 2024-04-07T17:03:57.830281+0000 mgr.y (mgr.25175) 1458 : cluster [DBG] pgmap v2313: 169 pgs: 169 active+clean; 877 MiB data, 5.4 GiB used, 710 GiB / 715 GiB avail; 4.9 MiB/s rd, 4.8 MiB/s wr, 205 op/s 2024-04-07T17:04:00.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:03:59 smithi144 bash[66654]: cluster 2024-04-07T17:03:58.739059+0000 mon.a (mon.0) 2520 : cluster [DBG] osdmap e1281: 8 total, 8 up, 8 in 2024-04-07T17:04:00.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:03:59 smithi100 bash[74942]: cluster 2024-04-07T17:03:57.830281+0000 mgr.y (mgr.25175) 1458 : cluster [DBG] pgmap v2313: 169 pgs: 169 active+clean; 877 MiB data, 5.4 GiB used, 710 GiB / 715 GiB avail; 4.9 MiB/s rd, 4.8 MiB/s wr, 205 op/s 2024-04-07T17:04:00.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:03:59 smithi100 bash[74942]: cluster 2024-04-07T17:03:58.739059+0000 mon.a (mon.0) 2520 : cluster [DBG] osdmap e1281: 8 total, 8 up, 8 in 2024-04-07T17:04:00.202 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:03:59 smithi100 bash[76202]: cluster 2024-04-07T17:03:57.830281+0000 mgr.y (mgr.25175) 1458 : cluster [DBG] pgmap v2313: 169 pgs: 169 active+clean; 877 MiB data, 5.4 GiB used, 710 GiB / 715 GiB avail; 4.9 MiB/s rd, 4.8 MiB/s wr, 205 op/s 2024-04-07T17:04:00.202 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:03:59 smithi100 bash[76202]: cluster 2024-04-07T17:03:58.739059+0000 mon.a (mon.0) 2520 : cluster [DBG] osdmap e1281: 8 total, 8 up, 8 in 2024-04-07T17:04:01.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:04:00 smithi144 bash[66654]: cluster 2024-04-07T17:03:59.748800+0000 mon.a (mon.0) 2521 : cluster [DBG] osdmap e1282: 8 total, 8 up, 8 in 2024-04-07T17:04:01.201 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:04:00 smithi100 bash[76202]: cluster 2024-04-07T17:03:59.748800+0000 mon.a (mon.0) 2521 : cluster [DBG] osdmap e1282: 8 total, 8 up, 8 in 2024-04-07T17:04:01.202 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:04:00 smithi100 bash[74942]: cluster 2024-04-07T17:03:59.748800+0000 mon.a (mon.0) 2521 : cluster [DBG] osdmap e1282: 8 total, 8 up, 8 in 2024-04-07T17:04:02.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:04:01 smithi144 bash[66654]: cluster 2024-04-07T17:03:59.830905+0000 mgr.y (mgr.25175) 1459 : cluster [DBG] pgmap v2316: 169 pgs: 169 active+clean; 929 MiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 33 MiB/s rd, 15 MiB/s wr, 254 op/s 2024-04-07T17:04:02.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:04:01 smithi144 bash[66654]: cluster 2024-04-07T17:04:00.760187+0000 mon.a (mon.0) 2522 : cluster [DBG] osdmap e1283: 8 total, 8 up, 8 in 2024-04-07T17:04:02.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:04:01 smithi100 bash[74942]: cluster 2024-04-07T17:03:59.830905+0000 mgr.y (mgr.25175) 1459 : cluster [DBG] pgmap v2316: 169 pgs: 169 active+clean; 929 MiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 33 MiB/s rd, 15 MiB/s wr, 254 op/s 2024-04-07T17:04:02.202 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:04:01 smithi100 bash[74942]: cluster 2024-04-07T17:04:00.760187+0000 mon.a (mon.0) 2522 : cluster [DBG] osdmap e1283: 8 total, 8 up, 8 in 2024-04-07T17:04:02.202 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:04:01 smithi100 bash[76202]: cluster 2024-04-07T17:03:59.830905+0000 mgr.y (mgr.25175) 1459 : cluster [DBG] pgmap v2316: 169 pgs: 169 active+clean; 929 MiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 33 MiB/s rd, 15 MiB/s wr, 254 op/s 2024-04-07T17:04:02.202 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:04:01 smithi100 bash[76202]: cluster 2024-04-07T17:04:00.760187+0000 mon.a (mon.0) 2522 : cluster [DBG] osdmap e1283: 8 total, 8 up, 8 in 2024-04-07T17:04:02.501 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestMigration.AbortWithSnapshots (15662 ms) 2024-04-07T17:04:02.501 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestMigration.CloneV1Parent 2024-04-07T17:04:02.587 INFO:tasks.workunit.client.0.smithi100.stdout:resize: 83886080 2024-04-07T17:04:02.615 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:04:02 smithi144 bash[95233]: ts=2024-04-07T17:04:02.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:04:02.617 INFO:tasks.workunit.client.0.smithi100.stdout:write: A 0~10 2024-04-07T17:04:02.640 INFO:tasks.workunit.client.0.smithi100.stdout:snap_create: snap1 2024-04-07T17:04:03.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:04:02 smithi144 bash[66654]: cluster 2024-04-07T17:04:01.770163+0000 mon.a (mon.0) 2523 : cluster [DBG] osdmap e1284: 8 total, 8 up, 8 in 2024-04-07T17:04:03.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:04:02 smithi144 bash[66654]: audit 2024-04-07T17:04:01.802710+0000 mon.a (mon.0) 2524 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:04:03.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:04:02 smithi100 bash[74942]: cluster 2024-04-07T17:04:01.770163+0000 mon.a (mon.0) 2523 : cluster [DBG] osdmap e1284: 8 total, 8 up, 8 in 2024-04-07T17:04:03.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:04:02 smithi100 bash[74942]: audit 2024-04-07T17:04:01.802710+0000 mon.a (mon.0) 2524 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:04:03.202 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:04:02 smithi100 bash[76202]: cluster 2024-04-07T17:04:01.770163+0000 mon.a (mon.0) 2523 : cluster [DBG] osdmap e1284: 8 total, 8 up, 8 in 2024-04-07T17:04:03.202 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:04:02 smithi100 bash[76202]: audit 2024-04-07T17:04:01.802710+0000 mon.a (mon.0) 2524 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:04:03.825 INFO:tasks.workunit.client.0.smithi100.stdout:snap_protect: snap1 2024-04-07T17:04:03.977 INFO:tasks.workunit.client.0.smithi100.stdout:migration_prepare 2024-04-07T17:04:04.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:04:03 smithi144 bash[66654]: cluster 2024-04-07T17:04:01.831639+0000 mgr.y (mgr.25175) 1460 : cluster [DBG] pgmap v2319: 169 pgs: 169 active+clean; 937 MiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 34 MiB/s rd, 19 MiB/s wr, 225 op/s 2024-04-07T17:04:04.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:04:03 smithi144 bash[66654]: cluster 2024-04-07T17:04:02.777743+0000 mon.a (mon.0) 2525 : cluster [DBG] osdmap e1285: 8 total, 8 up, 8 in 2024-04-07T17:04:04.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:04:03 smithi100 bash[74942]: cluster 2024-04-07T17:04:01.831639+0000 mgr.y (mgr.25175) 1460 : cluster [DBG] pgmap v2319: 169 pgs: 169 active+clean; 937 MiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 34 MiB/s rd, 19 MiB/s wr, 225 op/s 2024-04-07T17:04:04.202 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:04:03 smithi100 bash[74942]: cluster 2024-04-07T17:04:02.777743+0000 mon.a (mon.0) 2525 : cluster [DBG] osdmap e1285: 8 total, 8 up, 8 in 2024-04-07T17:04:04.202 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:04:03 smithi100 bash[76202]: cluster 2024-04-07T17:04:01.831639+0000 mgr.y (mgr.25175) 1460 : cluster [DBG] pgmap v2319: 169 pgs: 169 active+clean; 937 MiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 34 MiB/s rd, 19 MiB/s wr, 225 op/s 2024-04-07T17:04:04.202 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:04:03 smithi100 bash[76202]: cluster 2024-04-07T17:04:02.777743+0000 mon.a (mon.0) 2525 : cluster [DBG] osdmap e1285: 8 total, 8 up, 8 in 2024-04-07T17:04:05.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:04:04 smithi144 bash[66654]: cluster 2024-04-07T17:04:03.779438+0000 mon.a (mon.0) 2526 : cluster [DBG] osdmap e1286: 8 total, 8 up, 8 in 2024-04-07T17:04:05.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:04:04 smithi100 bash[74942]: cluster 2024-04-07T17:04:03.779438+0000 mon.a (mon.0) 2526 : cluster [DBG] osdmap e1286: 8 total, 8 up, 8 in 2024-04-07T17:04:05.202 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:04:04 smithi100 bash[76202]: cluster 2024-04-07T17:04:03.779438+0000 mon.a (mon.0) 2526 : cluster [DBG] osdmap e1286: 8 total, 8 up, 8 in 2024-04-07T17:04:05.306 INFO:tasks.workunit.client.0.smithi100.stdout:migration_execute 2024-04-07T17:04:05.534 INFO:tasks.workunit.client.0.smithi100.stdout:migration_commit 2024-04-07T17:04:06.099 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestMigration.CloneV1Parent (3599 ms) 2024-04-07T17:04:06.099 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestMigration.CloneV2Parent 2024-04-07T17:04:06.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:04:05 smithi144 bash[66654]: cluster 2024-04-07T17:04:03.832442+0000 mgr.y (mgr.25175) 1461 : cluster [DBG] pgmap v2322: 169 pgs: 169 active+clean; 925 MiB data, 5.6 GiB used, 710 GiB / 715 GiB avail; 18 MiB/s rd, 7.0 MiB/s wr, 188 op/s 2024-04-07T17:04:06.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:04:05 smithi144 bash[66654]: cluster 2024-04-07T17:04:04.800939+0000 mon.a (mon.0) 2527 : cluster [DBG] osdmap e1287: 8 total, 8 up, 8 in 2024-04-07T17:04:06.185 INFO:tasks.workunit.client.0.smithi100.stdout:resize: 83886080 2024-04-07T17:04:06.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:04:05 smithi100 bash[74942]: cluster 2024-04-07T17:04:03.832442+0000 mgr.y (mgr.25175) 1461 : cluster [DBG] pgmap v2322: 169 pgs: 169 active+clean; 925 MiB data, 5.6 GiB used, 710 GiB / 715 GiB avail; 18 MiB/s rd, 7.0 MiB/s wr, 188 op/s 2024-04-07T17:04:06.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:04:05 smithi100 bash[74942]: cluster 2024-04-07T17:04:04.800939+0000 mon.a (mon.0) 2527 : cluster [DBG] osdmap e1287: 8 total, 8 up, 8 in 2024-04-07T17:04:06.201 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:04:05 smithi100 bash[76202]: cluster 2024-04-07T17:04:03.832442+0000 mgr.y (mgr.25175) 1461 : cluster [DBG] pgmap v2322: 169 pgs: 169 active+clean; 925 MiB data, 5.6 GiB used, 710 GiB / 715 GiB avail; 18 MiB/s rd, 7.0 MiB/s wr, 188 op/s 2024-04-07T17:04:06.201 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:04:05 smithi100 bash[76202]: cluster 2024-04-07T17:04:04.800939+0000 mon.a (mon.0) 2527 : cluster [DBG] osdmap e1287: 8 total, 8 up, 8 in 2024-04-07T17:04:06.215 INFO:tasks.workunit.client.0.smithi100.stdout:write: A 0~10 2024-04-07T17:04:06.240 INFO:tasks.workunit.client.0.smithi100.stdout:snap_create: snap1 2024-04-07T17:04:07.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:04:06 smithi144 bash[66654]: cluster 2024-04-07T17:04:05.812958+0000 mon.a (mon.0) 2528 : cluster [DBG] osdmap e1288: 8 total, 8 up, 8 in 2024-04-07T17:04:07.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:04:06 smithi100 bash[74942]: cluster 2024-04-07T17:04:05.812958+0000 mon.a (mon.0) 2528 : cluster [DBG] osdmap e1288: 8 total, 8 up, 8 in 2024-04-07T17:04:07.202 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:04:06 smithi100 bash[76202]: cluster 2024-04-07T17:04:05.812958+0000 mon.a (mon.0) 2528 : cluster [DBG] osdmap e1288: 8 total, 8 up, 8 in 2024-04-07T17:04:07.629 INFO:tasks.workunit.client.0.smithi100.stdout:snap_protect: snap1 2024-04-07T17:04:07.785 INFO:tasks.workunit.client.0.smithi100.stdout:migration_prepare 2024-04-07T17:04:08.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:04:07 smithi144 bash[66654]: cluster 2024-04-07T17:04:05.833542+0000 mgr.y (mgr.25175) 1462 : cluster [DBG] pgmap v2325: 169 pgs: 169 active+clean; 905 MiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 35 MiB/s rd, 3.0 MiB/s wr, 210 op/s 2024-04-07T17:04:08.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:04:07 smithi144 bash[66654]: cluster 2024-04-07T17:04:06.829092+0000 mon.a (mon.0) 2529 : cluster [DBG] osdmap e1289: 8 total, 8 up, 8 in 2024-04-07T17:04:08.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:04:07 smithi144 bash[66654]: cluster 2024-04-07T17:04:07.593111+0000 mon.a (mon.0) 2530 : cluster [DBG] osdmap e1290: 8 total, 8 up, 8 in 2024-04-07T17:04:08.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:04:07 smithi100 bash[74942]: cluster 2024-04-07T17:04:05.833542+0000 mgr.y (mgr.25175) 1462 : cluster [DBG] pgmap v2325: 169 pgs: 169 active+clean; 905 MiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 35 MiB/s rd, 3.0 MiB/s wr, 210 op/s 2024-04-07T17:04:08.202 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:04:07 smithi100 bash[74942]: cluster 2024-04-07T17:04:06.829092+0000 mon.a (mon.0) 2529 : cluster [DBG] osdmap e1289: 8 total, 8 up, 8 in 2024-04-07T17:04:08.202 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:04:07 smithi100 bash[74942]: cluster 2024-04-07T17:04:07.593111+0000 mon.a (mon.0) 2530 : cluster [DBG] osdmap e1290: 8 total, 8 up, 8 in 2024-04-07T17:04:08.202 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:04:07 smithi100 bash[76202]: cluster 2024-04-07T17:04:05.833542+0000 mgr.y (mgr.25175) 1462 : cluster [DBG] pgmap v2325: 169 pgs: 169 active+clean; 905 MiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 35 MiB/s rd, 3.0 MiB/s wr, 210 op/s 2024-04-07T17:04:08.202 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:04:07 smithi100 bash[76202]: cluster 2024-04-07T17:04:06.829092+0000 mon.a (mon.0) 2529 : cluster [DBG] osdmap e1289: 8 total, 8 up, 8 in 2024-04-07T17:04:08.202 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:04:07 smithi100 bash[76202]: cluster 2024-04-07T17:04:07.593111+0000 mon.a (mon.0) 2530 : cluster [DBG] osdmap e1290: 8 total, 8 up, 8 in 2024-04-07T17:04:08.701 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:04:08 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:04:08] "GET /metrics HTTP/1.1" 200 38919 "" "Prometheus/2.43.0" 2024-04-07T17:04:09.347 INFO:tasks.workunit.client.0.smithi100.stdout:migration_execute 2024-04-07T17:04:09.590 INFO:tasks.workunit.client.0.smithi100.stdout:migration_commit 2024-04-07T17:04:10.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:04:09 smithi144 bash[66654]: cluster 2024-04-07T17:04:07.834345+0000 mgr.y (mgr.25175) 1463 : cluster [DBG] pgmap v2328: 169 pgs: 169 active+clean; 905 MiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 26 MiB/s rd, 13 KiB/s wr, 230 op/s 2024-04-07T17:04:10.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:04:09 smithi144 bash[66654]: cluster 2024-04-07T17:04:08.835964+0000 mon.a (mon.0) 2531 : cluster [DBG] osdmap e1291: 8 total, 8 up, 8 in 2024-04-07T17:04:10.123 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestMigration.CloneV2Parent (4023 ms) 2024-04-07T17:04:10.123 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestMigration.CloneV1ParentAbort 2024-04-07T17:04:10.201 INFO:tasks.workunit.client.0.smithi100.stdout:resize: 83886080 2024-04-07T17:04:10.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:04:09 smithi100 bash[74942]: cluster 2024-04-07T17:04:07.834345+0000 mgr.y (mgr.25175) 1463 : cluster [DBG] pgmap v2328: 169 pgs: 169 active+clean; 905 MiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 26 MiB/s rd, 13 KiB/s wr, 230 op/s 2024-04-07T17:04:10.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:04:09 smithi100 bash[74942]: cluster 2024-04-07T17:04:08.835964+0000 mon.a (mon.0) 2531 : cluster [DBG] osdmap e1291: 8 total, 8 up, 8 in 2024-04-07T17:04:10.202 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:04:09 smithi100 bash[76202]: cluster 2024-04-07T17:04:07.834345+0000 mgr.y (mgr.25175) 1463 : cluster [DBG] pgmap v2328: 169 pgs: 169 active+clean; 905 MiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 26 MiB/s rd, 13 KiB/s wr, 230 op/s 2024-04-07T17:04:10.202 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:04:09 smithi100 bash[76202]: cluster 2024-04-07T17:04:08.835964+0000 mon.a (mon.0) 2531 : cluster [DBG] osdmap e1291: 8 total, 8 up, 8 in 2024-04-07T17:04:10.232 INFO:tasks.workunit.client.0.smithi100.stdout:write: A 0~10 2024-04-07T17:04:10.256 INFO:tasks.workunit.client.0.smithi100.stdout:snap_create: snap1 2024-04-07T17:04:11.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:04:10 smithi144 bash[66654]: cluster 2024-04-07T17:04:09.835492+0000 mgr.y (mgr.25175) 1464 : cluster [DBG] pgmap v2331: 169 pgs: 169 active+clean; 905 MiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 4.4 MiB/s rd, 23 KiB/s wr, 515 op/s 2024-04-07T17:04:11.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:04:10 smithi144 bash[66654]: cluster 2024-04-07T17:04:09.837736+0000 mon.a (mon.0) 2532 : cluster [DBG] osdmap e1292: 8 total, 8 up, 8 in 2024-04-07T17:04:11.202 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:04:10 smithi100 bash[74942]: cluster 2024-04-07T17:04:09.835492+0000 mgr.y (mgr.25175) 1464 : cluster [DBG] pgmap v2331: 169 pgs: 169 active+clean; 905 MiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 4.4 MiB/s rd, 23 KiB/s wr, 515 op/s 2024-04-07T17:04:11.202 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:04:10 smithi100 bash[74942]: cluster 2024-04-07T17:04:09.837736+0000 mon.a (mon.0) 2532 : cluster [DBG] osdmap e1292: 8 total, 8 up, 8 in 2024-04-07T17:04:11.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:04:10 smithi100 bash[76202]: cluster 2024-04-07T17:04:09.835492+0000 mgr.y (mgr.25175) 1464 : cluster [DBG] pgmap v2331: 169 pgs: 169 active+clean; 905 MiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 4.4 MiB/s rd, 23 KiB/s wr, 515 op/s 2024-04-07T17:04:11.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:04:10 smithi100 bash[76202]: cluster 2024-04-07T17:04:09.837736+0000 mon.a (mon.0) 2532 : cluster [DBG] osdmap e1292: 8 total, 8 up, 8 in 2024-04-07T17:04:11.891 INFO:tasks.workunit.client.0.smithi100.stdout:snap_protect: snap1 2024-04-07T17:04:12.038 INFO:tasks.workunit.client.0.smithi100.stdout:migration_prepare 2024-04-07T17:04:12.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:04:11 smithi144 bash[66654]: cluster 2024-04-07T17:04:10.851871+0000 mon.a (mon.0) 2533 : cluster [DBG] osdmap e1293: 8 total, 8 up, 8 in 2024-04-07T17:04:12.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:04:11 smithi100 bash[74942]: cluster 2024-04-07T17:04:10.851871+0000 mon.a (mon.0) 2533 : cluster [DBG] osdmap e1293: 8 total, 8 up, 8 in 2024-04-07T17:04:12.202 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:04:11 smithi100 bash[76202]: cluster 2024-04-07T17:04:10.851871+0000 mon.a (mon.0) 2533 : cluster [DBG] osdmap e1293: 8 total, 8 up, 8 in 2024-04-07T17:04:12.615 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:04:12 smithi144 bash[95233]: ts=2024-04-07T17:04:12.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:04:13.021 INFO:tasks.workunit.client.0.smithi100.stdout:migration_abort 2024-04-07T17:04:13.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:04:12 smithi144 bash[66654]: cluster 2024-04-07T17:04:11.836300+0000 mgr.y (mgr.25175) 1465 : cluster [DBG] pgmap v2333: 169 pgs: 169 active+clean; 905 MiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 265 KiB/s rd, 17 KiB/s wr, 341 op/s 2024-04-07T17:04:13.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:04:12 smithi144 bash[66654]: cluster 2024-04-07T17:04:11.863100+0000 mon.a (mon.0) 2534 : cluster [DBG] osdmap e1294: 8 total, 8 up, 8 in 2024-04-07T17:04:13.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:04:12 smithi144 bash[66654]: cluster 2024-04-07T17:04:12.595394+0000 mon.a (mon.0) 2535 : cluster [DBG] osdmap e1295: 8 total, 8 up, 8 in 2024-04-07T17:04:13.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:04:12 smithi100 bash[74942]: cluster 2024-04-07T17:04:11.836300+0000 mgr.y (mgr.25175) 1465 : cluster [DBG] pgmap v2333: 169 pgs: 169 active+clean; 905 MiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 265 KiB/s rd, 17 KiB/s wr, 341 op/s 2024-04-07T17:04:13.202 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:04:12 smithi100 bash[74942]: cluster 2024-04-07T17:04:11.863100+0000 mon.a (mon.0) 2534 : cluster [DBG] osdmap e1294: 8 total, 8 up, 8 in 2024-04-07T17:04:13.202 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:04:12 smithi100 bash[74942]: cluster 2024-04-07T17:04:12.595394+0000 mon.a (mon.0) 2535 : cluster [DBG] osdmap e1295: 8 total, 8 up, 8 in 2024-04-07T17:04:13.202 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:04:12 smithi100 bash[76202]: cluster 2024-04-07T17:04:11.836300+0000 mgr.y (mgr.25175) 1465 : cluster [DBG] pgmap v2333: 169 pgs: 169 active+clean; 905 MiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 265 KiB/s rd, 17 KiB/s wr, 341 op/s 2024-04-07T17:04:13.202 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:04:12 smithi100 bash[76202]: cluster 2024-04-07T17:04:11.863100+0000 mon.a (mon.0) 2534 : cluster [DBG] osdmap e1294: 8 total, 8 up, 8 in 2024-04-07T17:04:13.202 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:04:12 smithi100 bash[76202]: cluster 2024-04-07T17:04:12.595394+0000 mon.a (mon.0) 2535 : cluster [DBG] osdmap e1295: 8 total, 8 up, 8 in 2024-04-07T17:04:14.160 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestMigration.CloneV1ParentAbort (4038 ms) 2024-04-07T17:04:14.161 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestMigration.CloneV2ParentAbort 2024-04-07T17:04:14.255 INFO:tasks.workunit.client.0.smithi100.stdout:resize: 83886080 2024-04-07T17:04:14.286 INFO:tasks.workunit.client.0.smithi100.stdout:write: A 0~10 2024-04-07T17:04:14.312 INFO:tasks.workunit.client.0.smithi100.stdout:snap_create: snap1 2024-04-07T17:04:15.202 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:04:14 smithi100 bash[74942]: cluster 2024-04-07T17:04:13.837305+0000 mgr.y (mgr.25175) 1466 : cluster [DBG] pgmap v2336: 169 pgs: 169 active+clean; 905 MiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 169 KiB/s rd, 13 KiB/s wr, 215 op/s 2024-04-07T17:04:15.202 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:04:14 smithi100 bash[74942]: cluster 2024-04-07T17:04:13.876413+0000 mon.a (mon.0) 2536 : cluster [DBG] osdmap e1296: 8 total, 8 up, 8 in 2024-04-07T17:04:15.202 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:04:14 smithi100 bash[76202]: cluster 2024-04-07T17:04:13.837305+0000 mgr.y (mgr.25175) 1466 : cluster [DBG] pgmap v2336: 169 pgs: 169 active+clean; 905 MiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 169 KiB/s rd, 13 KiB/s wr, 215 op/s 2024-04-07T17:04:15.202 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:04:14 smithi100 bash[76202]: cluster 2024-04-07T17:04:13.876413+0000 mon.a (mon.0) 2536 : cluster [DBG] osdmap e1296: 8 total, 8 up, 8 in 2024-04-07T17:04:15.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:04:14 smithi144 bash[66654]: cluster 2024-04-07T17:04:13.837305+0000 mgr.y (mgr.25175) 1466 : cluster [DBG] pgmap v2336: 169 pgs: 169 active+clean; 905 MiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 169 KiB/s rd, 13 KiB/s wr, 215 op/s 2024-04-07T17:04:15.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:04:14 smithi144 bash[66654]: cluster 2024-04-07T17:04:13.876413+0000 mon.a (mon.0) 2536 : cluster [DBG] osdmap e1296: 8 total, 8 up, 8 in 2024-04-07T17:04:15.919 INFO:tasks.workunit.client.0.smithi100.stdout:snap_protect: snap1 2024-04-07T17:04:16.064 INFO:tasks.workunit.client.0.smithi100.stdout:migration_prepare 2024-04-07T17:04:16.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:04:15 smithi100 bash[74942]: cluster 2024-04-07T17:04:14.886940+0000 mon.a (mon.0) 2537 : cluster [DBG] osdmap e1297: 8 total, 8 up, 8 in 2024-04-07T17:04:16.202 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:04:15 smithi100 bash[76202]: cluster 2024-04-07T17:04:14.886940+0000 mon.a (mon.0) 2537 : cluster [DBG] osdmap e1297: 8 total, 8 up, 8 in 2024-04-07T17:04:16.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:04:15 smithi144 bash[66654]: cluster 2024-04-07T17:04:14.886940+0000 mon.a (mon.0) 2537 : cluster [DBG] osdmap e1297: 8 total, 8 up, 8 in 2024-04-07T17:04:17.202 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:04:16 smithi100 bash[76202]: cluster 2024-04-07T17:04:15.838243+0000 mgr.y (mgr.25175) 1467 : cluster [DBG] pgmap v2339: 169 pgs: 169 active+clean; 905 MiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 431 KiB/s rd, 24 KiB/s wr, 538 op/s 2024-04-07T17:04:17.202 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:04:16 smithi100 bash[76202]: cluster 2024-04-07T17:04:15.897381+0000 mon.a (mon.0) 2538 : cluster [DBG] osdmap e1298: 8 total, 8 up, 8 in 2024-04-07T17:04:17.202 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:04:16 smithi100 bash[76202]: audit 2024-04-07T17:04:16.803363+0000 mon.a (mon.0) 2539 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:04:17.202 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:04:16 smithi100 bash[74942]: cluster 2024-04-07T17:04:15.838243+0000 mgr.y (mgr.25175) 1467 : cluster [DBG] pgmap v2339: 169 pgs: 169 active+clean; 905 MiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 431 KiB/s rd, 24 KiB/s wr, 538 op/s 2024-04-07T17:04:17.202 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:04:16 smithi100 bash[74942]: cluster 2024-04-07T17:04:15.897381+0000 mon.a (mon.0) 2538 : cluster [DBG] osdmap e1298: 8 total, 8 up, 8 in 2024-04-07T17:04:17.202 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:04:16 smithi100 bash[74942]: audit 2024-04-07T17:04:16.803363+0000 mon.a (mon.0) 2539 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:04:17.312 INFO:tasks.workunit.client.0.smithi100.stdout:migration_abort 2024-04-07T17:04:17.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:04:16 smithi144 bash[66654]: cluster 2024-04-07T17:04:15.838243+0000 mgr.y (mgr.25175) 1467 : cluster [DBG] pgmap v2339: 169 pgs: 169 active+clean; 905 MiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 431 KiB/s rd, 24 KiB/s wr, 538 op/s 2024-04-07T17:04:17.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:04:16 smithi144 bash[66654]: cluster 2024-04-07T17:04:15.897381+0000 mon.a (mon.0) 2538 : cluster [DBG] osdmap e1298: 8 total, 8 up, 8 in 2024-04-07T17:04:17.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:04:16 smithi144 bash[66654]: audit 2024-04-07T17:04:16.803363+0000 mon.a (mon.0) 2539 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:04:17.881 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestMigration.CloneV2ParentAbort (3721 ms) 2024-04-07T17:04:17.881 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestMigration.CloneV1ParentAbortFixIncompleteChildReattach 2024-04-07T17:04:17.964 INFO:tasks.workunit.client.0.smithi100.stdout:resize: 83886080 2024-04-07T17:04:17.994 INFO:tasks.workunit.client.0.smithi100.stdout:write: A 0~10 2024-04-07T17:04:18.018 INFO:tasks.workunit.client.0.smithi100.stdout:snap_create: snap1 2024-04-07T17:04:18.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:04:17 smithi100 bash[74942]: cluster 2024-04-07T17:04:16.905235+0000 mon.a (mon.0) 2540 : cluster [DBG] osdmap e1299: 8 total, 8 up, 8 in 2024-04-07T17:04:18.202 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:04:17 smithi100 bash[74942]: cluster 2024-04-07T17:04:17.593179+0000 mon.a (mon.0) 2541 : cluster [DBG] osdmap e1300: 8 total, 8 up, 8 in 2024-04-07T17:04:18.202 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:04:17 smithi100 bash[76202]: cluster 2024-04-07T17:04:16.905235+0000 mon.a (mon.0) 2540 : cluster [DBG] osdmap e1299: 8 total, 8 up, 8 in 2024-04-07T17:04:18.202 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:04:17 smithi100 bash[76202]: cluster 2024-04-07T17:04:17.593179+0000 mon.a (mon.0) 2541 : cluster [DBG] osdmap e1300: 8 total, 8 up, 8 in 2024-04-07T17:04:18.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:04:17 smithi144 bash[66654]: cluster 2024-04-07T17:04:16.905235+0000 mon.a (mon.0) 2540 : cluster [DBG] osdmap e1299: 8 total, 8 up, 8 in 2024-04-07T17:04:18.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:04:17 smithi144 bash[66654]: cluster 2024-04-07T17:04:17.593179+0000 mon.a (mon.0) 2541 : cluster [DBG] osdmap e1300: 8 total, 8 up, 8 in 2024-04-07T17:04:18.701 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:04:18 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:04:18] "GET /metrics HTTP/1.1" 200 38917 "" "Prometheus/2.43.0" 2024-04-07T17:04:19.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:04:19 smithi144 bash[66654]: cluster 2024-04-07T17:04:17.838941+0000 mgr.y (mgr.25175) 1468 : cluster [DBG] pgmap v2343: 169 pgs: 169 active+clean; 905 MiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 405 KiB/s rd, 24 KiB/s wr, 506 op/s 2024-04-07T17:04:19.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:04:19 smithi144 bash[66654]: cluster 2024-04-07T17:04:18.912225+0000 mon.a (mon.0) 2542 : cluster [DBG] osdmap e1301: 8 total, 8 up, 8 in 2024-04-07T17:04:19.938 INFO:tasks.workunit.client.0.smithi100.stdout:snap_protect: snap1 2024-04-07T17:04:19.951 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:04:19 smithi100 bash[74942]: cluster 2024-04-07T17:04:17.838941+0000 mgr.y (mgr.25175) 1468 : cluster [DBG] pgmap v2343: 169 pgs: 169 active+clean; 905 MiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 405 KiB/s rd, 24 KiB/s wr, 506 op/s 2024-04-07T17:04:19.951 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:04:19 smithi100 bash[74942]: cluster 2024-04-07T17:04:18.912225+0000 mon.a (mon.0) 2542 : cluster [DBG] osdmap e1301: 8 total, 8 up, 8 in 2024-04-07T17:04:19.952 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:04:19 smithi100 bash[76202]: cluster 2024-04-07T17:04:17.838941+0000 mgr.y (mgr.25175) 1468 : cluster [DBG] pgmap v2343: 169 pgs: 169 active+clean; 905 MiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 405 KiB/s rd, 24 KiB/s wr, 506 op/s 2024-04-07T17:04:19.952 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:04:19 smithi100 bash[76202]: cluster 2024-04-07T17:04:18.912225+0000 mon.a (mon.0) 2542 : cluster [DBG] osdmap e1301: 8 total, 8 up, 8 in 2024-04-07T17:04:20.093 INFO:tasks.workunit.client.0.smithi100.stdout:migration_prepare 2024-04-07T17:04:21.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:04:20 smithi100 bash[74942]: cluster 2024-04-07T17:04:19.840082+0000 mgr.y (mgr.25175) 1469 : cluster [DBG] pgmap v2345: 169 pgs: 169 active+clean; 905 MiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 173 KiB/s rd, 18 KiB/s wr, 243 op/s 2024-04-07T17:04:21.202 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:04:20 smithi100 bash[74942]: cluster 2024-04-07T17:04:19.913146+0000 mon.a (mon.0) 2543 : cluster [DBG] osdmap e1302: 8 total, 8 up, 8 in 2024-04-07T17:04:21.202 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:04:20 smithi100 bash[76202]: cluster 2024-04-07T17:04:19.840082+0000 mgr.y (mgr.25175) 1469 : cluster [DBG] pgmap v2345: 169 pgs: 169 active+clean; 905 MiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 173 KiB/s rd, 18 KiB/s wr, 243 op/s 2024-04-07T17:04:21.202 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:04:20 smithi100 bash[76202]: cluster 2024-04-07T17:04:19.913146+0000 mon.a (mon.0) 2543 : cluster [DBG] osdmap e1302: 8 total, 8 up, 8 in 2024-04-07T17:04:21.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:04:20 smithi144 bash[66654]: cluster 2024-04-07T17:04:19.840082+0000 mgr.y (mgr.25175) 1469 : cluster [DBG] pgmap v2345: 169 pgs: 169 active+clean; 905 MiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 173 KiB/s rd, 18 KiB/s wr, 243 op/s 2024-04-07T17:04:21.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:04:20 smithi144 bash[66654]: cluster 2024-04-07T17:04:19.913146+0000 mon.a (mon.0) 2543 : cluster [DBG] osdmap e1302: 8 total, 8 up, 8 in 2024-04-07T17:04:21.371 INFO:tasks.workunit.client.0.smithi100.stdout:migration_abort 2024-04-07T17:04:22.198 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestMigration.CloneV1ParentAbortFixIncompleteChildReattach (4316 ms) 2024-04-07T17:04:22.198 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestMigration.CloneV1ParentAbortFixParentReattach 2024-04-07T17:04:22.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:04:21 smithi100 bash[74942]: cluster 2024-04-07T17:04:20.918125+0000 mon.a (mon.0) 2544 : cluster [DBG] osdmap e1303: 8 total, 8 up, 8 in 2024-04-07T17:04:22.202 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:04:21 smithi100 bash[76202]: cluster 2024-04-07T17:04:20.918125+0000 mon.a (mon.0) 2544 : cluster [DBG] osdmap e1303: 8 total, 8 up, 8 in 2024-04-07T17:04:22.282 INFO:tasks.workunit.client.0.smithi100.stdout:resize: 83886080 2024-04-07T17:04:22.310 INFO:tasks.workunit.client.0.smithi100.stdout:write: A 0~10 2024-04-07T17:04:22.314 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:04:21 smithi144 bash[66654]: cluster 2024-04-07T17:04:20.918125+0000 mon.a (mon.0) 2544 : cluster [DBG] osdmap e1303: 8 total, 8 up, 8 in 2024-04-07T17:04:22.332 INFO:tasks.workunit.client.0.smithi100.stdout:snap_create: snap1 2024-04-07T17:04:22.615 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:04:22 smithi144 bash[95233]: ts=2024-04-07T17:04:22.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:04:23.202 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:04:22 smithi100 bash[74942]: cluster 2024-04-07T17:04:21.840902+0000 mgr.y (mgr.25175) 1470 : cluster [DBG] pgmap v2348: 169 pgs: 169 active+clean; 905 MiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 78 KiB/s rd, 9.6 KiB/s wr, 114 op/s 2024-04-07T17:04:23.202 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:04:22 smithi100 bash[74942]: cluster 2024-04-07T17:04:21.941354+0000 mon.a (mon.0) 2545 : cluster [DBG] osdmap e1304: 8 total, 8 up, 8 in 2024-04-07T17:04:23.202 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:04:22 smithi100 bash[74942]: cluster 2024-04-07T17:04:22.597839+0000 mon.a (mon.0) 2546 : cluster [DBG] osdmap e1305: 8 total, 8 up, 8 in 2024-04-07T17:04:23.202 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:04:22 smithi100 bash[76202]: cluster 2024-04-07T17:04:21.840902+0000 mgr.y (mgr.25175) 1470 : cluster [DBG] pgmap v2348: 169 pgs: 169 active+clean; 905 MiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 78 KiB/s rd, 9.6 KiB/s wr, 114 op/s 2024-04-07T17:04:23.202 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:04:22 smithi100 bash[76202]: cluster 2024-04-07T17:04:21.941354+0000 mon.a (mon.0) 2545 : cluster [DBG] osdmap e1304: 8 total, 8 up, 8 in 2024-04-07T17:04:23.202 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:04:22 smithi100 bash[76202]: cluster 2024-04-07T17:04:22.597839+0000 mon.a (mon.0) 2546 : cluster [DBG] osdmap e1305: 8 total, 8 up, 8 in 2024-04-07T17:04:23.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:04:22 smithi144 bash[66654]: cluster 2024-04-07T17:04:21.840902+0000 mgr.y (mgr.25175) 1470 : cluster [DBG] pgmap v2348: 169 pgs: 169 active+clean; 905 MiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 78 KiB/s rd, 9.6 KiB/s wr, 114 op/s 2024-04-07T17:04:23.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:04:22 smithi144 bash[66654]: cluster 2024-04-07T17:04:21.941354+0000 mon.a (mon.0) 2545 : cluster [DBG] osdmap e1304: 8 total, 8 up, 8 in 2024-04-07T17:04:23.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:04:22 smithi144 bash[66654]: cluster 2024-04-07T17:04:22.597839+0000 mon.a (mon.0) 2546 : cluster [DBG] osdmap e1305: 8 total, 8 up, 8 in 2024-04-07T17:04:23.636 INFO:tasks.workunit.client.0.smithi100.stdout:snap_protect: snap1 2024-04-07T17:04:23.787 INFO:tasks.workunit.client.0.smithi100.stdout:migration_prepare 2024-04-07T17:04:24.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:04:24 smithi144 bash[66654]: cluster 2024-04-07T17:04:23.600652+0000 mon.a (mon.0) 2547 : cluster [DBG] osdmap e1306: 8 total, 8 up, 8 in 2024-04-07T17:04:24.951 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:04:24 smithi100 bash[74942]: cluster 2024-04-07T17:04:23.600652+0000 mon.a (mon.0) 2547 : cluster [DBG] osdmap e1306: 8 total, 8 up, 8 in 2024-04-07T17:04:24.952 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:04:24 smithi100 bash[76202]: cluster 2024-04-07T17:04:23.600652+0000 mon.a (mon.0) 2547 : cluster [DBG] osdmap e1306: 8 total, 8 up, 8 in 2024-04-07T17:04:25.050 INFO:tasks.workunit.client.0.smithi100.stdout:migration_abort 2024-04-07T17:04:25.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:04:25 smithi144 bash[66654]: cluster 2024-04-07T17:04:23.841719+0000 mgr.y (mgr.25175) 1471 : cluster [DBG] pgmap v2352: 169 pgs: 169 active+clean; 905 MiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 387 KiB/s rd, 21 KiB/s wr, 483 op/s 2024-04-07T17:04:25.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:04:25 smithi144 bash[66654]: cluster 2024-04-07T17:04:24.609835+0000 mon.a (mon.0) 2548 : cluster [DBG] osdmap e1307: 8 total, 8 up, 8 in 2024-04-07T17:04:25.925 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestMigration.CloneV1ParentAbortFixParentReattach (3728 ms) 2024-04-07T17:04:25.926 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestMigration.CloneV1ParentAbortRelinkNotNeeded 2024-04-07T17:04:25.951 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:04:25 smithi100 bash[74942]: cluster 2024-04-07T17:04:23.841719+0000 mgr.y (mgr.25175) 1471 : cluster [DBG] pgmap v2352: 169 pgs: 169 active+clean; 905 MiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 387 KiB/s rd, 21 KiB/s wr, 483 op/s 2024-04-07T17:04:25.952 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:04:25 smithi100 bash[74942]: cluster 2024-04-07T17:04:24.609835+0000 mon.a (mon.0) 2548 : cluster [DBG] osdmap e1307: 8 total, 8 up, 8 in 2024-04-07T17:04:25.952 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:04:25 smithi100 bash[76202]: cluster 2024-04-07T17:04:23.841719+0000 mgr.y (mgr.25175) 1471 : cluster [DBG] pgmap v2352: 169 pgs: 169 active+clean; 905 MiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 387 KiB/s rd, 21 KiB/s wr, 483 op/s 2024-04-07T17:04:25.952 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:04:25 smithi100 bash[76202]: cluster 2024-04-07T17:04:24.609835+0000 mon.a (mon.0) 2548 : cluster [DBG] osdmap e1307: 8 total, 8 up, 8 in 2024-04-07T17:04:26.010 INFO:tasks.workunit.client.0.smithi100.stdout:resize: 83886080 2024-04-07T17:04:26.040 INFO:tasks.workunit.client.0.smithi100.stdout:write: A 0~10 2024-04-07T17:04:26.064 INFO:tasks.workunit.client.0.smithi100.stdout:snap_create: snap1 2024-04-07T17:04:26.951 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:04:26 smithi100 bash[74942]: cluster 2024-04-07T17:04:25.621364+0000 mon.a (mon.0) 2549 : cluster [DBG] osdmap e1308: 8 total, 8 up, 8 in 2024-04-07T17:04:26.952 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:04:26 smithi100 bash[76202]: cluster 2024-04-07T17:04:25.621364+0000 mon.a (mon.0) 2549 : cluster [DBG] osdmap e1308: 8 total, 8 up, 8 in 2024-04-07T17:04:27.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:04:26 smithi144 bash[66654]: cluster 2024-04-07T17:04:25.621364+0000 mon.a (mon.0) 2549 : cluster [DBG] osdmap e1308: 8 total, 8 up, 8 in 2024-04-07T17:04:27.629 INFO:tasks.workunit.client.0.smithi100.stdout:snap_protect: snap1 2024-04-07T17:04:27.776 INFO:tasks.workunit.client.0.smithi100.stdout:migration_prepare 2024-04-07T17:04:27.952 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:04:27 smithi100 bash[74942]: cluster 2024-04-07T17:04:25.842540+0000 mgr.y (mgr.25175) 1472 : cluster [DBG] pgmap v2355: 169 pgs: 169 active+clean; 905 MiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 416 KiB/s rd, 25 KiB/s wr, 526 op/s 2024-04-07T17:04:27.952 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:04:27 smithi100 bash[74942]: cluster 2024-04-07T17:04:26.630898+0000 mon.a (mon.0) 2550 : cluster [DBG] osdmap e1309: 8 total, 8 up, 8 in 2024-04-07T17:04:27.952 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:04:27 smithi100 bash[74942]: cluster 2024-04-07T17:04:27.598111+0000 mon.a (mon.0) 2551 : cluster [DBG] osdmap e1310: 8 total, 8 up, 8 in 2024-04-07T17:04:27.952 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:04:27 smithi100 bash[76202]: cluster 2024-04-07T17:04:25.842540+0000 mgr.y (mgr.25175) 1472 : cluster [DBG] pgmap v2355: 169 pgs: 169 active+clean; 905 MiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 416 KiB/s rd, 25 KiB/s wr, 526 op/s 2024-04-07T17:04:27.952 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:04:27 smithi100 bash[76202]: cluster 2024-04-07T17:04:26.630898+0000 mon.a (mon.0) 2550 : cluster [DBG] osdmap e1309: 8 total, 8 up, 8 in 2024-04-07T17:04:27.952 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:04:27 smithi100 bash[76202]: cluster 2024-04-07T17:04:27.598111+0000 mon.a (mon.0) 2551 : cluster [DBG] osdmap e1310: 8 total, 8 up, 8 in 2024-04-07T17:04:28.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:04:27 smithi144 bash[66654]: cluster 2024-04-07T17:04:25.842540+0000 mgr.y (mgr.25175) 1472 : cluster [DBG] pgmap v2355: 169 pgs: 169 active+clean; 905 MiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 416 KiB/s rd, 25 KiB/s wr, 526 op/s 2024-04-07T17:04:28.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:04:27 smithi144 bash[66654]: cluster 2024-04-07T17:04:26.630898+0000 mon.a (mon.0) 2550 : cluster [DBG] osdmap e1309: 8 total, 8 up, 8 in 2024-04-07T17:04:28.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:04:27 smithi144 bash[66654]: cluster 2024-04-07T17:04:27.598111+0000 mon.a (mon.0) 2551 : cluster [DBG] osdmap e1310: 8 total, 8 up, 8 in 2024-04-07T17:04:28.701 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:04:28 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:04:28] "GET /metrics HTTP/1.1" 200 38917 "" "Prometheus/2.43.0" 2024-04-07T17:04:29.104 INFO:tasks.workunit.client.0.smithi100.stdout:migration_abort 2024-04-07T17:04:29.931 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestMigration.CloneV1ParentAbortRelinkNotNeeded (4006 ms) 2024-04-07T17:04:29.932 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestMigration.CloneV2ParentAbortFixIncompleteChildReattach 2024-04-07T17:04:29.951 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:04:29 smithi100 bash[74942]: cluster 2024-04-07T17:04:27.843337+0000 mgr.y (mgr.25175) 1473 : cluster [DBG] pgmap v2358: 169 pgs: 169 active+clean; 905 MiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 102 KiB/s rd, 12 KiB/s wr, 140 op/s 2024-04-07T17:04:29.951 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:04:29 smithi100 bash[74942]: cluster 2024-04-07T17:04:28.641339+0000 mon.a (mon.0) 2552 : cluster [DBG] osdmap e1311: 8 total, 8 up, 8 in 2024-04-07T17:04:29.952 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:04:29 smithi100 bash[76202]: cluster 2024-04-07T17:04:27.843337+0000 mgr.y (mgr.25175) 1473 : cluster [DBG] pgmap v2358: 169 pgs: 169 active+clean; 905 MiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 102 KiB/s rd, 12 KiB/s wr, 140 op/s 2024-04-07T17:04:29.952 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:04:29 smithi100 bash[76202]: cluster 2024-04-07T17:04:28.641339+0000 mon.a (mon.0) 2552 : cluster [DBG] osdmap e1311: 8 total, 8 up, 8 in 2024-04-07T17:04:30.018 INFO:tasks.workunit.client.0.smithi100.stdout:resize: 83886080 2024-04-07T17:04:30.049 INFO:tasks.workunit.client.0.smithi100.stdout:write: A 0~10 2024-04-07T17:04:30.074 INFO:tasks.workunit.client.0.smithi100.stdout:snap_create: snap1 2024-04-07T17:04:30.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:04:29 smithi144 bash[66654]: cluster 2024-04-07T17:04:27.843337+0000 mgr.y (mgr.25175) 1473 : cluster [DBG] pgmap v2358: 169 pgs: 169 active+clean; 905 MiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 102 KiB/s rd, 12 KiB/s wr, 140 op/s 2024-04-07T17:04:30.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:04:29 smithi144 bash[66654]: cluster 2024-04-07T17:04:28.641339+0000 mon.a (mon.0) 2552 : cluster [DBG] osdmap e1311: 8 total, 8 up, 8 in 2024-04-07T17:04:30.952 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:04:30 smithi100 bash[74942]: cluster 2024-04-07T17:04:29.652445+0000 mon.a (mon.0) 2553 : cluster [DBG] osdmap e1312: 8 total, 8 up, 8 in 2024-04-07T17:04:30.952 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:04:30 smithi100 bash[76202]: cluster 2024-04-07T17:04:29.652445+0000 mon.a (mon.0) 2553 : cluster [DBG] osdmap e1312: 8 total, 8 up, 8 in 2024-04-07T17:04:31.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:04:30 smithi144 bash[66654]: cluster 2024-04-07T17:04:29.652445+0000 mon.a (mon.0) 2553 : cluster [DBG] osdmap e1312: 8 total, 8 up, 8 in 2024-04-07T17:04:31.693 INFO:tasks.workunit.client.0.smithi100.stdout:snap_protect: snap1 2024-04-07T17:04:31.846 INFO:tasks.workunit.client.0.smithi100.stdout:migration_prepare 2024-04-07T17:04:31.951 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:04:31 smithi100 bash[74942]: cluster 2024-04-07T17:04:29.844323+0000 mgr.y (mgr.25175) 1474 : cluster [DBG] pgmap v2361: 169 pgs: 169 active+clean; 905 MiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 280 KiB/s rd, 18 KiB/s wr, 363 op/s 2024-04-07T17:04:31.952 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:04:31 smithi100 bash[74942]: cluster 2024-04-07T17:04:30.666968+0000 mon.a (mon.0) 2554 : cluster [DBG] osdmap e1313: 8 total, 8 up, 8 in 2024-04-07T17:04:31.952 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:04:31 smithi100 bash[76202]: cluster 2024-04-07T17:04:29.844323+0000 mgr.y (mgr.25175) 1474 : cluster [DBG] pgmap v2361: 169 pgs: 169 active+clean; 905 MiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 280 KiB/s rd, 18 KiB/s wr, 363 op/s 2024-04-07T17:04:31.952 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:04:31 smithi100 bash[76202]: cluster 2024-04-07T17:04:30.666968+0000 mon.a (mon.0) 2554 : cluster [DBG] osdmap e1313: 8 total, 8 up, 8 in 2024-04-07T17:04:32.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:04:31 smithi144 bash[66654]: cluster 2024-04-07T17:04:29.844323+0000 mgr.y (mgr.25175) 1474 : cluster [DBG] pgmap v2361: 169 pgs: 169 active+clean; 905 MiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 280 KiB/s rd, 18 KiB/s wr, 363 op/s 2024-04-07T17:04:32.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:04:31 smithi144 bash[66654]: cluster 2024-04-07T17:04:30.666968+0000 mon.a (mon.0) 2554 : cluster [DBG] osdmap e1313: 8 total, 8 up, 8 in 2024-04-07T17:04:32.615 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:04:32 smithi144 bash[95233]: ts=2024-04-07T17:04:32.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:04:32.952 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:04:32 smithi100 bash[74942]: cluster 2024-04-07T17:04:31.669753+0000 mon.a (mon.0) 2555 : cluster [DBG] osdmap e1314: 8 total, 8 up, 8 in 2024-04-07T17:04:32.952 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:04:32 smithi100 bash[74942]: audit 2024-04-07T17:04:31.803905+0000 mon.a (mon.0) 2556 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:04:32.952 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:04:32 smithi100 bash[76202]: cluster 2024-04-07T17:04:31.669753+0000 mon.a (mon.0) 2555 : cluster [DBG] osdmap e1314: 8 total, 8 up, 8 in 2024-04-07T17:04:32.952 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:04:32 smithi100 bash[76202]: audit 2024-04-07T17:04:31.803905+0000 mon.a (mon.0) 2556 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:04:33.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:04:32 smithi144 bash[66654]: cluster 2024-04-07T17:04:31.669753+0000 mon.a (mon.0) 2555 : cluster [DBG] osdmap e1314: 8 total, 8 up, 8 in 2024-04-07T17:04:33.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:04:32 smithi144 bash[66654]: audit 2024-04-07T17:04:31.803905+0000 mon.a (mon.0) 2556 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:04:33.125 INFO:tasks.workunit.client.0.smithi100.stdout:migration_abort 2024-04-07T17:04:33.952 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:04:33 smithi100 bash[74942]: cluster 2024-04-07T17:04:31.864693+0000 mgr.y (mgr.25175) 1475 : cluster [DBG] pgmap v2364: 169 pgs: 169 active+clean; 905 MiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 364 KiB/s rd, 17 KiB/s wr, 452 op/s 2024-04-07T17:04:33.952 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:04:33 smithi100 bash[74942]: cluster 2024-04-07T17:04:32.693531+0000 mon.a (mon.0) 2557 : cluster [DBG] osdmap e1315: 8 total, 8 up, 8 in 2024-04-07T17:04:33.952 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:04:33 smithi100 bash[76202]: cluster 2024-04-07T17:04:31.864693+0000 mgr.y (mgr.25175) 1475 : cluster [DBG] pgmap v2364: 169 pgs: 169 active+clean; 905 MiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 364 KiB/s rd, 17 KiB/s wr, 452 op/s 2024-04-07T17:04:33.952 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:04:33 smithi100 bash[76202]: cluster 2024-04-07T17:04:32.693531+0000 mon.a (mon.0) 2557 : cluster [DBG] osdmap e1315: 8 total, 8 up, 8 in 2024-04-07T17:04:34.036 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestMigration.CloneV2ParentAbortFixIncompleteChildReattach (4104 ms) 2024-04-07T17:04:34.036 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestMigration.CloneV2ParentAbortFixParentReattach 2024-04-07T17:04:34.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:04:33 smithi144 bash[66654]: cluster 2024-04-07T17:04:31.864693+0000 mgr.y (mgr.25175) 1475 : cluster [DBG] pgmap v2364: 169 pgs: 169 active+clean; 905 MiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 364 KiB/s rd, 17 KiB/s wr, 452 op/s 2024-04-07T17:04:34.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:04:33 smithi144 bash[66654]: cluster 2024-04-07T17:04:32.693531+0000 mon.a (mon.0) 2557 : cluster [DBG] osdmap e1315: 8 total, 8 up, 8 in 2024-04-07T17:04:34.130 INFO:tasks.workunit.client.0.smithi100.stdout:resize: 83886080 2024-04-07T17:04:34.163 INFO:tasks.workunit.client.0.smithi100.stdout:write: A 0~10 2024-04-07T17:04:34.189 INFO:tasks.workunit.client.0.smithi100.stdout:snap_create: snap1 2024-04-07T17:04:35.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:04:34 smithi144 bash[66654]: cluster 2024-04-07T17:04:33.705687+0000 mon.a (mon.0) 2558 : cluster [DBG] osdmap e1316: 8 total, 8 up, 8 in 2024-04-07T17:04:35.202 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:04:34 smithi100 bash[74942]: cluster 2024-04-07T17:04:33.705687+0000 mon.a (mon.0) 2558 : cluster [DBG] osdmap e1316: 8 total, 8 up, 8 in 2024-04-07T17:04:35.202 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:04:34 smithi100 bash[76202]: cluster 2024-04-07T17:04:33.705687+0000 mon.a (mon.0) 2558 : cluster [DBG] osdmap e1316: 8 total, 8 up, 8 in 2024-04-07T17:04:35.754 INFO:tasks.workunit.client.0.smithi100.stdout:snap_protect: snap1 2024-04-07T17:04:35.919 INFO:tasks.workunit.client.0.smithi100.stdout:migration_prepare 2024-04-07T17:04:36.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:04:35 smithi144 bash[66654]: cluster 2024-04-07T17:04:33.865535+0000 mgr.y (mgr.25175) 1476 : cluster [DBG] pgmap v2367: 169 pgs: 169 active+clean; 905 MiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 270 KiB/s rd, 16 KiB/s wr, 342 op/s 2024-04-07T17:04:36.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:04:35 smithi144 bash[66654]: cluster 2024-04-07T17:04:34.716896+0000 mon.a (mon.0) 2559 : cluster [DBG] osdmap e1317: 8 total, 8 up, 8 in 2024-04-07T17:04:36.202 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:04:35 smithi100 bash[74942]: cluster 2024-04-07T17:04:33.865535+0000 mgr.y (mgr.25175) 1476 : cluster [DBG] pgmap v2367: 169 pgs: 169 active+clean; 905 MiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 270 KiB/s rd, 16 KiB/s wr, 342 op/s 2024-04-07T17:04:36.202 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:04:35 smithi100 bash[74942]: cluster 2024-04-07T17:04:34.716896+0000 mon.a (mon.0) 2559 : cluster [DBG] osdmap e1317: 8 total, 8 up, 8 in 2024-04-07T17:04:36.202 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:04:35 smithi100 bash[76202]: cluster 2024-04-07T17:04:33.865535+0000 mgr.y (mgr.25175) 1476 : cluster [DBG] pgmap v2367: 169 pgs: 169 active+clean; 905 MiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 270 KiB/s rd, 16 KiB/s wr, 342 op/s 2024-04-07T17:04:36.202 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:04:35 smithi100 bash[76202]: cluster 2024-04-07T17:04:34.716896+0000 mon.a (mon.0) 2559 : cluster [DBG] osdmap e1317: 8 total, 8 up, 8 in 2024-04-07T17:04:37.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:04:36 smithi144 bash[66654]: cluster 2024-04-07T17:04:35.727799+0000 mon.a (mon.0) 2560 : cluster [DBG] osdmap e1318: 8 total, 8 up, 8 in 2024-04-07T17:04:37.185 INFO:tasks.workunit.client.0.smithi100.stdout:migration_abort 2024-04-07T17:04:37.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:04:36 smithi100 bash[74942]: cluster 2024-04-07T17:04:35.727799+0000 mon.a (mon.0) 2560 : cluster [DBG] osdmap e1318: 8 total, 8 up, 8 in 2024-04-07T17:04:37.201 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:04:36 smithi100 bash[76202]: cluster 2024-04-07T17:04:35.727799+0000 mon.a (mon.0) 2560 : cluster [DBG] osdmap e1318: 8 total, 8 up, 8 in 2024-04-07T17:04:37.893 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestMigration.CloneV2ParentAbortFixParentReattach (3857 ms) 2024-04-07T17:04:37.893 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestMigration.CloneV2ParentAbortRelinkNotNeeded 2024-04-07T17:04:37.979 INFO:tasks.workunit.client.0.smithi100.stdout:resize: 83886080 2024-04-07T17:04:38.009 INFO:tasks.workunit.client.0.smithi100.stdout:write: A 0~10 2024-04-07T17:04:38.033 INFO:tasks.workunit.client.0.smithi100.stdout:snap_create: snap1 2024-04-07T17:04:38.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:04:37 smithi144 bash[66654]: cluster 2024-04-07T17:04:35.866307+0000 mgr.y (mgr.25175) 1477 : cluster [DBG] pgmap v2370: 169 pgs: 169 active+clean; 905 MiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 284 KiB/s rd, 23 KiB/s wr, 371 op/s 2024-04-07T17:04:38.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:04:37 smithi144 bash[66654]: cluster 2024-04-07T17:04:36.739720+0000 mon.a (mon.0) 2561 : cluster [DBG] osdmap e1319: 8 total, 8 up, 8 in 2024-04-07T17:04:38.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:04:37 smithi144 bash[66654]: cluster 2024-04-07T17:04:37.607416+0000 mon.a (mon.0) 2562 : cluster [DBG] osdmap e1320: 8 total, 8 up, 8 in 2024-04-07T17:04:38.202 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:04:37 smithi100 bash[74942]: cluster 2024-04-07T17:04:35.866307+0000 mgr.y (mgr.25175) 1477 : cluster [DBG] pgmap v2370: 169 pgs: 169 active+clean; 905 MiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 284 KiB/s rd, 23 KiB/s wr, 371 op/s 2024-04-07T17:04:38.202 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:04:37 smithi100 bash[74942]: cluster 2024-04-07T17:04:36.739720+0000 mon.a (mon.0) 2561 : cluster [DBG] osdmap e1319: 8 total, 8 up, 8 in 2024-04-07T17:04:38.202 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:04:37 smithi100 bash[74942]: cluster 2024-04-07T17:04:37.607416+0000 mon.a (mon.0) 2562 : cluster [DBG] osdmap e1320: 8 total, 8 up, 8 in 2024-04-07T17:04:38.202 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:04:37 smithi100 bash[76202]: cluster 2024-04-07T17:04:35.866307+0000 mgr.y (mgr.25175) 1477 : cluster [DBG] pgmap v2370: 169 pgs: 169 active+clean; 905 MiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 284 KiB/s rd, 23 KiB/s wr, 371 op/s 2024-04-07T17:04:38.202 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:04:37 smithi100 bash[76202]: cluster 2024-04-07T17:04:36.739720+0000 mon.a (mon.0) 2561 : cluster [DBG] osdmap e1319: 8 total, 8 up, 8 in 2024-04-07T17:04:38.202 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:04:37 smithi100 bash[76202]: cluster 2024-04-07T17:04:37.607416+0000 mon.a (mon.0) 2562 : cluster [DBG] osdmap e1320: 8 total, 8 up, 8 in 2024-04-07T17:04:38.702 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:04:38 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:04:38] "GET /metrics HTTP/1.1" 200 38923 "" "Prometheus/2.43.0" 2024-04-07T17:04:39.785 INFO:tasks.workunit.client.0.smithi100.stdout:snap_protect: snap1 2024-04-07T17:04:39.935 INFO:tasks.workunit.client.0.smithi100.stdout:migration_prepare 2024-04-07T17:04:40.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:04:39 smithi144 bash[66654]: cluster 2024-04-07T17:04:37.867133+0000 mgr.y (mgr.25175) 1478 : cluster [DBG] pgmap v2373: 169 pgs: 169 active+clean; 905 MiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 271 KiB/s rd, 21 KiB/s wr, 348 op/s 2024-04-07T17:04:40.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:04:39 smithi144 bash[66654]: cluster 2024-04-07T17:04:38.750069+0000 mon.a (mon.0) 2563 : cluster [DBG] osdmap e1321: 8 total, 8 up, 8 in 2024-04-07T17:04:40.202 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:04:39 smithi100 bash[74942]: cluster 2024-04-07T17:04:37.867133+0000 mgr.y (mgr.25175) 1478 : cluster [DBG] pgmap v2373: 169 pgs: 169 active+clean; 905 MiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 271 KiB/s rd, 21 KiB/s wr, 348 op/s 2024-04-07T17:04:40.202 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:04:39 smithi100 bash[74942]: cluster 2024-04-07T17:04:38.750069+0000 mon.a (mon.0) 2563 : cluster [DBG] osdmap e1321: 8 total, 8 up, 8 in 2024-04-07T17:04:40.202 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:04:39 smithi100 bash[76202]: cluster 2024-04-07T17:04:37.867133+0000 mgr.y (mgr.25175) 1478 : cluster [DBG] pgmap v2373: 169 pgs: 169 active+clean; 905 MiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 271 KiB/s rd, 21 KiB/s wr, 348 op/s 2024-04-07T17:04:40.202 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:04:39 smithi100 bash[76202]: cluster 2024-04-07T17:04:38.750069+0000 mon.a (mon.0) 2563 : cluster [DBG] osdmap e1321: 8 total, 8 up, 8 in 2024-04-07T17:04:41.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:04:40 smithi144 bash[66654]: cluster 2024-04-07T17:04:39.752439+0000 mon.a (mon.0) 2564 : cluster [DBG] osdmap e1322: 8 total, 8 up, 8 in 2024-04-07T17:04:41.202 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:04:40 smithi100 bash[74942]: cluster 2024-04-07T17:04:39.752439+0000 mon.a (mon.0) 2564 : cluster [DBG] osdmap e1322: 8 total, 8 up, 8 in 2024-04-07T17:04:41.202 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:04:40 smithi100 bash[76202]: cluster 2024-04-07T17:04:39.752439+0000 mon.a (mon.0) 2564 : cluster [DBG] osdmap e1322: 8 total, 8 up, 8 in 2024-04-07T17:04:41.210 INFO:tasks.workunit.client.0.smithi100.stdout:migration_abort 2024-04-07T17:04:42.073 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestMigration.CloneV2ParentAbortRelinkNotNeeded (4180 ms) 2024-04-07T17:04:42.073 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestMigration.StressNoMigrate 2024-04-07T17:04:42.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:04:41 smithi144 bash[66654]: cluster 2024-04-07T17:04:39.867977+0000 mgr.y (mgr.25175) 1479 : cluster [DBG] pgmap v2376: 169 pgs: 169 active+clean; 905 MiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 206 KiB/s rd, 18 KiB/s wr, 275 op/s 2024-04-07T17:04:42.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:04:41 smithi144 bash[66654]: cluster 2024-04-07T17:04:40.775667+0000 mon.a (mon.0) 2565 : cluster [DBG] osdmap e1323: 8 total, 8 up, 8 in 2024-04-07T17:04:42.158 INFO:tasks.workunit.client.0.smithi100.stdout:resize: 83886080 2024-04-07T17:04:42.189 INFO:tasks.workunit.client.0.smithi100.stdout:write: A 1685838~6846506 2024-04-07T17:04:42.201 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:04:41 smithi100 bash[76202]: cluster 2024-04-07T17:04:39.867977+0000 mgr.y (mgr.25175) 1479 : cluster [DBG] pgmap v2376: 169 pgs: 169 active+clean; 905 MiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 206 KiB/s rd, 18 KiB/s wr, 275 op/s 2024-04-07T17:04:42.202 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:04:41 smithi100 bash[76202]: cluster 2024-04-07T17:04:40.775667+0000 mon.a (mon.0) 2565 : cluster [DBG] osdmap e1323: 8 total, 8 up, 8 in 2024-04-07T17:04:42.202 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:04:41 smithi100 bash[74942]: cluster 2024-04-07T17:04:39.867977+0000 mgr.y (mgr.25175) 1479 : cluster [DBG] pgmap v2376: 169 pgs: 169 active+clean; 905 MiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 206 KiB/s rd, 18 KiB/s wr, 275 op/s 2024-04-07T17:04:42.202 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:04:41 smithi100 bash[74942]: cluster 2024-04-07T17:04:40.775667+0000 mon.a (mon.0) 2565 : cluster [DBG] osdmap e1323: 8 total, 8 up, 8 in 2024-04-07T17:04:42.285 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 21863189~1151929 2024-04-07T17:04:42.285 INFO:tasks.workunit.client.0.smithi100.stdout:write: A 8692931~8119119 2024-04-07T17:04:42.362 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 30233360~535856 2024-04-07T17:04:42.362 INFO:tasks.workunit.client.0.smithi100.stdout:write: A 27775063~5058968 2024-04-07T17:04:42.428 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 75798067~882453 2024-04-07T17:04:42.428 INFO:tasks.workunit.client.0.smithi100.stdout:write: A 70312274~886871 2024-04-07T17:04:42.450 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 32132680~768930 2024-04-07T17:04:42.456 INFO:tasks.workunit.client.0.smithi100.stdout:snap_create: snap0 2024-04-07T17:04:42.615 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:04:42 smithi144 bash[95233]: ts=2024-04-07T17:04:42.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:04:43.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:04:42 smithi144 bash[66654]: cluster 2024-04-07T17:04:41.776881+0000 mon.a (mon.0) 2566 : cluster [DBG] osdmap e1324: 8 total, 8 up, 8 in 2024-04-07T17:04:43.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:04:42 smithi144 bash[66654]: cluster 2024-04-07T17:04:42.604311+0000 mon.a (mon.0) 2567 : cluster [DBG] osdmap e1325: 8 total, 8 up, 8 in 2024-04-07T17:04:43.202 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:04:42 smithi100 bash[74942]: cluster 2024-04-07T17:04:41.776881+0000 mon.a (mon.0) 2566 : cluster [DBG] osdmap e1324: 8 total, 8 up, 8 in 2024-04-07T17:04:43.202 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:04:42 smithi100 bash[74942]: cluster 2024-04-07T17:04:42.604311+0000 mon.a (mon.0) 2567 : cluster [DBG] osdmap e1325: 8 total, 8 up, 8 in 2024-04-07T17:04:43.202 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:04:42 smithi100 bash[76202]: cluster 2024-04-07T17:04:41.776881+0000 mon.a (mon.0) 2566 : cluster [DBG] osdmap e1324: 8 total, 8 up, 8 in 2024-04-07T17:04:43.202 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:04:42 smithi100 bash[76202]: cluster 2024-04-07T17:04:42.604311+0000 mon.a (mon.0) 2567 : cluster [DBG] osdmap e1325: 8 total, 8 up, 8 in 2024-04-07T17:04:43.626 INFO:tasks.workunit.client.0.smithi100.stdout:write: B 69803031~4217659 2024-04-07T17:04:43.653 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 36407181~7408122 2024-04-07T17:04:43.653 INFO:tasks.workunit.client.0.smithi100.stdout:write: B 48597614~6674371 2024-04-07T17:04:43.674 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 65248102~2586806 2024-04-07T17:04:43.674 INFO:tasks.workunit.client.0.smithi100.stdout:write: B 59631549~6560142 2024-04-07T17:04:43.687 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 82467175~1213321 2024-04-07T17:04:43.687 INFO:tasks.workunit.client.0.smithi100.stdout:write: B 58391476~7193026 2024-04-07T17:04:43.743 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 16733564~266933 2024-04-07T17:04:43.743 INFO:tasks.workunit.client.0.smithi100.stdout:snap_create: snap1 2024-04-07T17:04:44.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:04:43 smithi144 bash[66654]: cluster 2024-04-07T17:04:41.868738+0000 mgr.y (mgr.25175) 1480 : cluster [DBG] pgmap v2379: 169 pgs: 169 active+clean; 905 MiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 86 KiB/s rd, 11 KiB/s wr, 119 op/s 2024-04-07T17:04:44.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:04:43 smithi144 bash[66654]: cluster 2024-04-07T17:04:43.606756+0000 mon.a (mon.0) 2568 : cluster [DBG] osdmap e1326: 8 total, 8 up, 8 in 2024-04-07T17:04:44.202 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:04:43 smithi100 bash[74942]: cluster 2024-04-07T17:04:41.868738+0000 mgr.y (mgr.25175) 1480 : cluster [DBG] pgmap v2379: 169 pgs: 169 active+clean; 905 MiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 86 KiB/s rd, 11 KiB/s wr, 119 op/s 2024-04-07T17:04:44.202 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:04:43 smithi100 bash[74942]: cluster 2024-04-07T17:04:43.606756+0000 mon.a (mon.0) 2568 : cluster [DBG] osdmap e1326: 8 total, 8 up, 8 in 2024-04-07T17:04:44.202 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:04:43 smithi100 bash[76202]: cluster 2024-04-07T17:04:41.868738+0000 mgr.y (mgr.25175) 1480 : cluster [DBG] pgmap v2379: 169 pgs: 169 active+clean; 905 MiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 86 KiB/s rd, 11 KiB/s wr, 119 op/s 2024-04-07T17:04:44.202 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:04:43 smithi100 bash[76202]: cluster 2024-04-07T17:04:43.606756+0000 mon.a (mon.0) 2568 : cluster [DBG] osdmap e1326: 8 total, 8 up, 8 in 2024-04-07T17:04:45.643 INFO:tasks.workunit.client.0.smithi100.stdout:snap_protect: snap1 2024-04-07T17:04:45.680 INFO:tasks.workunit.client.0.smithi100.stdout:clone image159 -> image162 2024-04-07T17:04:45.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:04:45 smithi144 bash[66654]: cluster 2024-04-07T17:04:43.869521+0000 mgr.y (mgr.25175) 1481 : cluster [DBG] pgmap v2382: 169 pgs: 169 active+clean; 909 MiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 305 KiB/s rd, 411 KiB/s wr, 390 op/s 2024-04-07T17:04:45.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:04:45 smithi144 bash[66654]: cluster 2024-04-07T17:04:44.612420+0000 mon.a (mon.0) 2569 : cluster [DBG] osdmap e1327: 8 total, 8 up, 8 in 2024-04-07T17:04:45.945 INFO:tasks.workunit.client.0.smithi100.stdout:write: C 22070265~260671 2024-04-07T17:04:45.951 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:04:45 smithi100 bash[74942]: cluster 2024-04-07T17:04:43.869521+0000 mgr.y (mgr.25175) 1481 : cluster [DBG] pgmap v2382: 169 pgs: 169 active+clean; 909 MiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 305 KiB/s rd, 411 KiB/s wr, 390 op/s 2024-04-07T17:04:45.951 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:04:45 smithi100 bash[74942]: cluster 2024-04-07T17:04:44.612420+0000 mon.a (mon.0) 2569 : cluster [DBG] osdmap e1327: 8 total, 8 up, 8 in 2024-04-07T17:04:45.952 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:04:45 smithi100 bash[76202]: cluster 2024-04-07T17:04:43.869521+0000 mgr.y (mgr.25175) 1481 : cluster [DBG] pgmap v2382: 169 pgs: 169 active+clean; 909 MiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 305 KiB/s rd, 411 KiB/s wr, 390 op/s 2024-04-07T17:04:45.952 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:04:45 smithi100 bash[76202]: cluster 2024-04-07T17:04:44.612420+0000 mon.a (mon.0) 2569 : cluster [DBG] osdmap e1327: 8 total, 8 up, 8 in 2024-04-07T17:04:45.978 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 78821100~5064980 2024-04-07T17:04:45.995 INFO:tasks.workunit.client.0.smithi100.stdout:write: C 47890473~7902288 2024-04-07T17:04:46.093 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 42343592~5620234 2024-04-07T17:04:46.107 INFO:tasks.workunit.client.0.smithi100.stdout:write: C 39943596~762235 2024-04-07T17:04:46.127 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 23727563~7367408 2024-04-07T17:04:46.166 INFO:tasks.workunit.client.0.smithi100.stdout:write: C 11585067~5284400 2024-04-07T17:04:46.257 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 63024170~1255906 2024-04-07T17:04:46.308 INFO:tasks.workunit.client.0.smithi100.stdout:snap_create: snap2 2024-04-07T17:04:46.952 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:04:46 smithi100 bash[74942]: cluster 2024-04-07T17:04:45.620611+0000 mon.a (mon.0) 2570 : cluster [DBG] osdmap e1328: 8 total, 8 up, 8 in 2024-04-07T17:04:46.952 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:04:46 smithi100 bash[76202]: cluster 2024-04-07T17:04:45.620611+0000 mon.a (mon.0) 2570 : cluster [DBG] osdmap e1328: 8 total, 8 up, 8 in 2024-04-07T17:04:47.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:04:46 smithi144 bash[66654]: cluster 2024-04-07T17:04:45.620611+0000 mon.a (mon.0) 2570 : cluster [DBG] osdmap e1328: 8 total, 8 up, 8 in 2024-04-07T17:04:47.637 INFO:tasks.workunit.client.0.smithi100.stdout:write: D 8636014~2569880 2024-04-07T17:04:47.665 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 67850279~6069516 2024-04-07T17:04:47.665 INFO:tasks.workunit.client.0.smithi100.stdout:write: D 32448661~1158700 2024-04-07T17:04:47.667 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 58683374~7711118 2024-04-07T17:04:47.667 INFO:tasks.workunit.client.0.smithi100.stdout:write: D 16366660~6954057 2024-04-07T17:04:47.684 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 70431506~8308074 2024-04-07T17:04:47.766 INFO:tasks.workunit.client.0.smithi100.stdout:write: D 50511786~7975496 2024-04-07T17:04:47.776 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 80149341~227083 2024-04-07T17:04:47.776 INFO:tasks.workunit.client.0.smithi100.stdout:snap_create: snap3 2024-04-07T17:04:47.952 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:04:47 smithi100 bash[74942]: cluster 2024-04-07T17:04:45.870337+0000 mgr.y (mgr.25175) 1482 : cluster [DBG] pgmap v2385: 169 pgs: 169 active+clean; 955 MiB data, 5.6 GiB used, 710 GiB / 715 GiB avail; 334 KiB/s rd, 11 MiB/s wr, 444 op/s 2024-04-07T17:04:47.952 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:04:47 smithi100 bash[74942]: cluster 2024-04-07T17:04:46.632226+0000 mon.a (mon.0) 2571 : cluster [DBG] osdmap e1329: 8 total, 8 up, 8 in 2024-04-07T17:04:47.952 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:04:47 smithi100 bash[74942]: audit 2024-04-07T17:04:46.804464+0000 mon.a (mon.0) 2572 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:04:47.952 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:04:47 smithi100 bash[74942]: cluster 2024-04-07T17:04:47.608067+0000 mon.a (mon.0) 2573 : cluster [DBG] osdmap e1330: 8 total, 8 up, 8 in 2024-04-07T17:04:47.952 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:04:47 smithi100 bash[76202]: cluster 2024-04-07T17:04:45.870337+0000 mgr.y (mgr.25175) 1482 : cluster [DBG] pgmap v2385: 169 pgs: 169 active+clean; 955 MiB data, 5.6 GiB used, 710 GiB / 715 GiB avail; 334 KiB/s rd, 11 MiB/s wr, 444 op/s 2024-04-07T17:04:47.952 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:04:47 smithi100 bash[76202]: cluster 2024-04-07T17:04:46.632226+0000 mon.a (mon.0) 2571 : cluster [DBG] osdmap e1329: 8 total, 8 up, 8 in 2024-04-07T17:04:47.952 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:04:47 smithi100 bash[76202]: audit 2024-04-07T17:04:46.804464+0000 mon.a (mon.0) 2572 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:04:47.953 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:04:47 smithi100 bash[76202]: cluster 2024-04-07T17:04:47.608067+0000 mon.a (mon.0) 2573 : cluster [DBG] osdmap e1330: 8 total, 8 up, 8 in 2024-04-07T17:04:48.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:04:47 smithi144 bash[66654]: cluster 2024-04-07T17:04:45.870337+0000 mgr.y (mgr.25175) 1482 : cluster [DBG] pgmap v2385: 169 pgs: 169 active+clean; 955 MiB data, 5.6 GiB used, 710 GiB / 715 GiB avail; 334 KiB/s rd, 11 MiB/s wr, 444 op/s 2024-04-07T17:04:48.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:04:47 smithi144 bash[66654]: cluster 2024-04-07T17:04:46.632226+0000 mon.a (mon.0) 2571 : cluster [DBG] osdmap e1329: 8 total, 8 up, 8 in 2024-04-07T17:04:48.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:04:47 smithi144 bash[66654]: audit 2024-04-07T17:04:46.804464+0000 mon.a (mon.0) 2572 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:04:48.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:04:47 smithi144 bash[66654]: cluster 2024-04-07T17:04:47.608067+0000 mon.a (mon.0) 2573 : cluster [DBG] osdmap e1330: 8 total, 8 up, 8 in 2024-04-07T17:04:48.702 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:04:48 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:04:48] "GET /metrics HTTP/1.1" 200 38923 "" "Prometheus/2.43.0" 2024-04-07T17:04:49.680 INFO:tasks.workunit.client.0.smithi100.stdout:snap_protect: snap3 2024-04-07T17:04:49.710 INFO:tasks.workunit.client.0.smithi100.stdout:clone image162 -> image164 2024-04-07T17:04:49.952 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:04:49 smithi100 bash[74942]: cluster 2024-04-07T17:04:47.871244+0000 mgr.y (mgr.25175) 1483 : cluster [DBG] pgmap v2388: 169 pgs: 169 active+clean; 1020 MiB data, 5.7 GiB used, 709 GiB / 715 GiB avail; 1.1 MiB/s rd, 21 MiB/s wr, 216 op/s 2024-04-07T17:04:49.952 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:04:49 smithi100 bash[74942]: cluster 2024-04-07T17:04:48.637813+0000 mon.a (mon.0) 2574 : cluster [DBG] osdmap e1331: 8 total, 8 up, 8 in 2024-04-07T17:04:49.952 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:04:49 smithi100 bash[74942]: audit 2024-04-07T17:04:49.567232+0000 mon.a (mon.0) 2575 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:04:49.952 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:04:49 smithi100 bash[76202]: cluster 2024-04-07T17:04:47.871244+0000 mgr.y (mgr.25175) 1483 : cluster [DBG] pgmap v2388: 169 pgs: 169 active+clean; 1020 MiB data, 5.7 GiB used, 709 GiB / 715 GiB avail; 1.1 MiB/s rd, 21 MiB/s wr, 216 op/s 2024-04-07T17:04:49.952 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:04:49 smithi100 bash[76202]: cluster 2024-04-07T17:04:48.637813+0000 mon.a (mon.0) 2574 : cluster [DBG] osdmap e1331: 8 total, 8 up, 8 in 2024-04-07T17:04:49.952 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:04:49 smithi100 bash[76202]: audit 2024-04-07T17:04:49.567232+0000 mon.a (mon.0) 2575 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:04:50.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:04:49 smithi144 bash[66654]: cluster 2024-04-07T17:04:47.871244+0000 mgr.y (mgr.25175) 1483 : cluster [DBG] pgmap v2388: 169 pgs: 169 active+clean; 1020 MiB data, 5.7 GiB used, 709 GiB / 715 GiB avail; 1.1 MiB/s rd, 21 MiB/s wr, 216 op/s 2024-04-07T17:04:50.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:04:49 smithi144 bash[66654]: cluster 2024-04-07T17:04:48.637813+0000 mon.a (mon.0) 2574 : cluster [DBG] osdmap e1331: 8 total, 8 up, 8 in 2024-04-07T17:04:50.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:04:49 smithi144 bash[66654]: audit 2024-04-07T17:04:49.567232+0000 mon.a (mon.0) 2575 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:04:50.193 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestMigration.StressNoMigrate (8120 ms) 2024-04-07T17:04:50.193 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestMigration.Stress 2024-04-07T17:04:50.244 INFO:tasks.workunit.client.0.smithi100.stdout:resize: 83886080 2024-04-07T17:04:50.262 INFO:tasks.workunit.client.0.smithi100.stdout:write: A 69840268~2210694 2024-04-07T17:04:50.292 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 28138754~141316 2024-04-07T17:04:50.292 INFO:tasks.workunit.client.0.smithi100.stdout:write: A 24285940~731960 2024-04-07T17:04:50.307 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 14404969~1534669 2024-04-07T17:04:50.307 INFO:tasks.workunit.client.0.smithi100.stdout:write: A 11179184~2316575 2024-04-07T17:04:50.330 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 33531608~1931667 2024-04-07T17:04:50.330 INFO:tasks.workunit.client.0.smithi100.stdout:write: A 4501548~2332246 2024-04-07T17:04:50.358 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 75510626~224582 2024-04-07T17:04:50.358 INFO:tasks.workunit.client.0.smithi100.stdout:snap_create: snap0 2024-04-07T17:04:50.952 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:04:50 smithi100 bash[74942]: cluster 2024-04-07T17:04:49.657941+0000 mon.a (mon.0) 2576 : cluster [DBG] osdmap e1332: 8 total, 8 up, 8 in 2024-04-07T17:04:50.952 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:04:50 smithi100 bash[74942]: audit 2024-04-07T17:04:50.197454+0000 mon.a (mon.0) 2577 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:04:50.952 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:04:50 smithi100 bash[74942]: audit 2024-04-07T17:04:50.207275+0000 mon.a (mon.0) 2578 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:04:50.952 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:04:50 smithi100 bash[76202]: cluster 2024-04-07T17:04:49.657941+0000 mon.a (mon.0) 2576 : cluster [DBG] osdmap e1332: 8 total, 8 up, 8 in 2024-04-07T17:04:50.952 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:04:50 smithi100 bash[76202]: audit 2024-04-07T17:04:50.197454+0000 mon.a (mon.0) 2577 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:04:50.952 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:04:50 smithi100 bash[76202]: audit 2024-04-07T17:04:50.207275+0000 mon.a (mon.0) 2578 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:04:51.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:04:50 smithi144 bash[66654]: cluster 2024-04-07T17:04:49.657941+0000 mon.a (mon.0) 2576 : cluster [DBG] osdmap e1332: 8 total, 8 up, 8 in 2024-04-07T17:04:51.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:04:50 smithi144 bash[66654]: audit 2024-04-07T17:04:50.197454+0000 mon.a (mon.0) 2577 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:04:51.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:04:50 smithi144 bash[66654]: audit 2024-04-07T17:04:50.207275+0000 mon.a (mon.0) 2578 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:04:51.952 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:04:51 smithi100 bash[74942]: cluster 2024-04-07T17:04:49.872218+0000 mgr.y (mgr.25175) 1484 : cluster [DBG] pgmap v2391: 169 pgs: 169 active+clean; 1.1 GiB data, 5.8 GiB used, 709 GiB / 715 GiB avail; 8.1 MiB/s rd, 27 MiB/s wr, 305 op/s 2024-04-07T17:04:51.952 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:04:51 smithi100 bash[74942]: audit 2024-04-07T17:04:50.854270+0000 mon.a (mon.0) 2579 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:04:51.952 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:04:51 smithi100 bash[74942]: audit 2024-04-07T17:04:50.856182+0000 mon.a (mon.0) 2580 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:04:51.952 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:04:51 smithi100 bash[74942]: audit 2024-04-07T17:04:50.866436+0000 mon.a (mon.0) 2581 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:04:51.952 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:04:51 smithi100 bash[74942]: cluster 2024-04-07T17:04:51.216670+0000 mon.a (mon.0) 2582 : cluster [DBG] osdmap e1333: 8 total, 8 up, 8 in 2024-04-07T17:04:51.953 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:04:51 smithi100 bash[76202]: cluster 2024-04-07T17:04:49.872218+0000 mgr.y (mgr.25175) 1484 : cluster [DBG] pgmap v2391: 169 pgs: 169 active+clean; 1.1 GiB data, 5.8 GiB used, 709 GiB / 715 GiB avail; 8.1 MiB/s rd, 27 MiB/s wr, 305 op/s 2024-04-07T17:04:51.953 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:04:51 smithi100 bash[76202]: audit 2024-04-07T17:04:50.854270+0000 mon.a (mon.0) 2579 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:04:51.953 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:04:51 smithi100 bash[76202]: audit 2024-04-07T17:04:50.856182+0000 mon.a (mon.0) 2580 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:04:51.953 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:04:51 smithi100 bash[76202]: audit 2024-04-07T17:04:50.866436+0000 mon.a (mon.0) 2581 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:04:51.953 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:04:51 smithi100 bash[76202]: cluster 2024-04-07T17:04:51.216670+0000 mon.a (mon.0) 2582 : cluster [DBG] osdmap e1333: 8 total, 8 up, 8 in 2024-04-07T17:04:52.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:04:51 smithi144 bash[66654]: cluster 2024-04-07T17:04:49.872218+0000 mgr.y (mgr.25175) 1484 : cluster [DBG] pgmap v2391: 169 pgs: 169 active+clean; 1.1 GiB data, 5.8 GiB used, 709 GiB / 715 GiB avail; 8.1 MiB/s rd, 27 MiB/s wr, 305 op/s 2024-04-07T17:04:52.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:04:51 smithi144 bash[66654]: audit 2024-04-07T17:04:50.854270+0000 mon.a (mon.0) 2579 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:04:52.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:04:51 smithi144 bash[66654]: audit 2024-04-07T17:04:50.856182+0000 mon.a (mon.0) 2580 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:04:52.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:04:51 smithi144 bash[66654]: audit 2024-04-07T17:04:50.866436+0000 mon.a (mon.0) 2581 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:04:52.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:04:51 smithi144 bash[66654]: cluster 2024-04-07T17:04:51.216670+0000 mon.a (mon.0) 2582 : cluster [DBG] osdmap e1333: 8 total, 8 up, 8 in 2024-04-07T17:04:52.275 INFO:tasks.workunit.client.0.smithi100.stdout:snap_protect: snap0 2024-04-07T17:04:52.304 INFO:tasks.workunit.client.0.smithi100.stdout:clone image165 -> image168 2024-04-07T17:04:52.566 INFO:tasks.workunit.client.0.smithi100.stdout:write: B 54897121~7295991 2024-04-07T17:04:52.615 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:04:52 smithi144 bash[95233]: ts=2024-04-07T17:04:52.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:04:52.666 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 14250049~1346401 2024-04-07T17:04:52.694 INFO:tasks.workunit.client.0.smithi100.stdout:write: B 9654475~4154917 2024-04-07T17:04:52.747 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 37296238~795475 2024-04-07T17:04:52.764 INFO:tasks.workunit.client.0.smithi100.stdout:write: B 76520031~1446005 2024-04-07T17:04:52.791 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 57277431~5067 2024-04-07T17:04:52.798 INFO:tasks.workunit.client.0.smithi100.stdout:write: B 27381585~763323 2024-04-07T17:04:52.819 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 34459071~1288579 2024-04-07T17:04:52.825 INFO:tasks.workunit.client.0.smithi100.stdout:snap_create: snap1 2024-04-07T17:04:53.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:04:53 smithi144 bash[66654]: cluster 2024-04-07T17:04:51.872997+0000 mgr.y (mgr.25175) 1485 : cluster [DBG] pgmap v2393: 169 pgs: 169 active+clean; 1.1 GiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 6.9 MiB/s rd, 18 MiB/s wr, 225 op/s 2024-04-07T17:04:53.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:04:53 smithi144 bash[66654]: cluster 2024-04-07T17:04:52.220624+0000 mon.a (mon.0) 2583 : cluster [DBG] osdmap e1334: 8 total, 8 up, 8 in 2024-04-07T17:04:53.702 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:04:53 smithi100 bash[74942]: cluster 2024-04-07T17:04:51.872997+0000 mgr.y (mgr.25175) 1485 : cluster [DBG] pgmap v2393: 169 pgs: 169 active+clean; 1.1 GiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 6.9 MiB/s rd, 18 MiB/s wr, 225 op/s 2024-04-07T17:04:53.703 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:04:53 smithi100 bash[74942]: cluster 2024-04-07T17:04:52.220624+0000 mon.a (mon.0) 2583 : cluster [DBG] osdmap e1334: 8 total, 8 up, 8 in 2024-04-07T17:04:53.703 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:04:53 smithi100 bash[76202]: cluster 2024-04-07T17:04:51.872997+0000 mgr.y (mgr.25175) 1485 : cluster [DBG] pgmap v2393: 169 pgs: 169 active+clean; 1.1 GiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 6.9 MiB/s rd, 18 MiB/s wr, 225 op/s 2024-04-07T17:04:53.703 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:04:53 smithi100 bash[76202]: cluster 2024-04-07T17:04:52.220624+0000 mon.a (mon.0) 2583 : cluster [DBG] osdmap e1334: 8 total, 8 up, 8 in 2024-04-07T17:04:54.707 INFO:tasks.workunit.client.0.smithi100.stdout:snap_protect: snap1 2024-04-07T17:04:54.739 INFO:tasks.workunit.client.0.smithi100.stdout:clone image168 -> image170 2024-04-07T17:04:54.952 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:04:54 smithi100 bash[74942]: cluster 2024-04-07T17:04:53.684221+0000 mon.a (mon.0) 2584 : cluster [DBG] osdmap e1335: 8 total, 8 up, 8 in 2024-04-07T17:04:54.952 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:04:54 smithi100 bash[76202]: cluster 2024-04-07T17:04:53.684221+0000 mon.a (mon.0) 2584 : cluster [DBG] osdmap e1335: 8 total, 8 up, 8 in 2024-04-07T17:04:55.071 INFO:tasks.workunit.client.0.smithi100.stdout:resize: 92299443 2024-04-07T17:04:55.101 INFO:tasks.workunit.client.0.smithi100.stdout:write: C 16276978~6723361 2024-04-07T17:04:55.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:04:54 smithi144 bash[66654]: cluster 2024-04-07T17:04:53.684221+0000 mon.a (mon.0) 2584 : cluster [DBG] osdmap e1335: 8 total, 8 up, 8 in 2024-04-07T17:04:55.203 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 17357935~2341330 2024-04-07T17:04:55.209 INFO:tasks.workunit.client.0.smithi100.stdout:write: C 18971947~1579960 2024-04-07T17:04:55.235 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 88841868~3457575 2024-04-07T17:04:55.235 INFO:tasks.workunit.client.0.smithi100.stdout:write: C 19926130~2543089 2024-04-07T17:04:55.260 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 37406538~2935863 2024-04-07T17:04:55.281 INFO:tasks.workunit.client.0.smithi100.stdout:write: C 32345345~2378112 2024-04-07T17:04:55.311 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 87026671~5272772 2024-04-07T17:04:55.311 INFO:tasks.workunit.client.0.smithi100.stdout:snap_create: snap2 2024-04-07T17:04:55.952 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:04:55 smithi100 bash[74942]: cluster 2024-04-07T17:04:53.873766+0000 mgr.y (mgr.25175) 1486 : cluster [DBG] pgmap v2396: 169 pgs: 169 active+clean; 1.1 GiB data, 6.0 GiB used, 709 GiB / 715 GiB avail; 9.1 MiB/s rd, 8.6 MiB/s wr, 224 op/s 2024-04-07T17:04:55.952 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:04:55 smithi100 bash[74942]: cluster 2024-04-07T17:04:54.690339+0000 mon.a (mon.0) 2585 : cluster [DBG] osdmap e1336: 8 total, 8 up, 8 in 2024-04-07T17:04:55.952 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:04:55 smithi100 bash[76202]: cluster 2024-04-07T17:04:53.873766+0000 mgr.y (mgr.25175) 1486 : cluster [DBG] pgmap v2396: 169 pgs: 169 active+clean; 1.1 GiB data, 6.0 GiB used, 709 GiB / 715 GiB avail; 9.1 MiB/s rd, 8.6 MiB/s wr, 224 op/s 2024-04-07T17:04:55.952 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:04:55 smithi100 bash[76202]: cluster 2024-04-07T17:04:54.690339+0000 mon.a (mon.0) 2585 : cluster [DBG] osdmap e1336: 8 total, 8 up, 8 in 2024-04-07T17:04:56.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:04:55 smithi144 bash[66654]: cluster 2024-04-07T17:04:53.873766+0000 mgr.y (mgr.25175) 1486 : cluster [DBG] pgmap v2396: 169 pgs: 169 active+clean; 1.1 GiB data, 6.0 GiB used, 709 GiB / 715 GiB avail; 9.1 MiB/s rd, 8.6 MiB/s wr, 224 op/s 2024-04-07T17:04:56.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:04:55 smithi144 bash[66654]: cluster 2024-04-07T17:04:54.690339+0000 mon.a (mon.0) 2585 : cluster [DBG] osdmap e1336: 8 total, 8 up, 8 in 2024-04-07T17:04:56.740 INFO:tasks.workunit.client.0.smithi100.stdout:snap_protect: snap2 2024-04-07T17:04:56.772 INFO:tasks.workunit.client.0.smithi100.stdout:clone image170 -> image172 2024-04-07T17:04:57.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:04:56 smithi144 bash[66654]: cluster 2024-04-07T17:04:55.705895+0000 mon.a (mon.0) 2586 : cluster [DBG] osdmap e1337: 8 total, 8 up, 8 in 2024-04-07T17:04:57.193 INFO:tasks.workunit.client.0.smithi100.stdout:write: D 90573717~646421 2024-04-07T17:04:57.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:04:56 smithi100 bash[74942]: cluster 2024-04-07T17:04:55.705895+0000 mon.a (mon.0) 2586 : cluster [DBG] osdmap e1337: 8 total, 8 up, 8 in 2024-04-07T17:04:57.202 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:04:56 smithi100 bash[76202]: cluster 2024-04-07T17:04:55.705895+0000 mon.a (mon.0) 2586 : cluster [DBG] osdmap e1337: 8 total, 8 up, 8 in 2024-04-07T17:04:57.236 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 68257989~7984645 2024-04-07T17:04:57.307 INFO:tasks.workunit.client.0.smithi100.stdout:write: D 14757143~7014519 2024-04-07T17:04:57.410 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 25957986~2432303 2024-04-07T17:04:57.432 INFO:tasks.workunit.client.0.smithi100.stdout:write: D 3213167~7830399 2024-04-07T17:04:57.516 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 34769213~6030364 2024-04-07T17:04:57.548 INFO:tasks.workunit.client.0.smithi100.stdout:write: D 37424449~3361697 2024-04-07T17:04:57.581 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 55564330~1110623 2024-04-07T17:04:57.634 INFO:tasks.workunit.client.0.smithi100.stdout:snap_create: snap3 2024-04-07T17:04:58.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:04:57 smithi144 bash[66654]: cluster 2024-04-07T17:04:55.874536+0000 mgr.y (mgr.25175) 1487 : cluster [DBG] pgmap v2399: 169 pgs: 169 active+clean; 1.2 GiB data, 6.1 GiB used, 709 GiB / 715 GiB avail; 20 MiB/s rd, 12 MiB/s wr, 400 op/s 2024-04-07T17:04:58.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:04:57 smithi144 bash[66654]: cluster 2024-04-07T17:04:56.713322+0000 mon.a (mon.0) 2587 : cluster [DBG] osdmap e1338: 8 total, 8 up, 8 in 2024-04-07T17:04:58.202 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:04:57 smithi100 bash[74942]: cluster 2024-04-07T17:04:55.874536+0000 mgr.y (mgr.25175) 1487 : cluster [DBG] pgmap v2399: 169 pgs: 169 active+clean; 1.2 GiB data, 6.1 GiB used, 709 GiB / 715 GiB avail; 20 MiB/s rd, 12 MiB/s wr, 400 op/s 2024-04-07T17:04:58.202 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:04:57 smithi100 bash[74942]: cluster 2024-04-07T17:04:56.713322+0000 mon.a (mon.0) 2587 : cluster [DBG] osdmap e1338: 8 total, 8 up, 8 in 2024-04-07T17:04:58.202 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:04:57 smithi100 bash[76202]: cluster 2024-04-07T17:04:55.874536+0000 mgr.y (mgr.25175) 1487 : cluster [DBG] pgmap v2399: 169 pgs: 169 active+clean; 1.2 GiB data, 6.1 GiB used, 709 GiB / 715 GiB avail; 20 MiB/s rd, 12 MiB/s wr, 400 op/s 2024-04-07T17:04:58.202 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:04:57 smithi100 bash[76202]: cluster 2024-04-07T17:04:56.713322+0000 mon.a (mon.0) 2587 : cluster [DBG] osdmap e1338: 8 total, 8 up, 8 in 2024-04-07T17:04:58.702 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:04:58 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:04:58] "GET /metrics HTTP/1.1" 200 38923 "" "Prometheus/2.43.0" 2024-04-07T17:04:58.776 INFO:tasks.workunit.client.0.smithi100.stdout:snap_protect: snap3 2024-04-07T17:04:58.813 INFO:tasks.workunit.client.0.smithi100.stdout:clone image172 -> image174 2024-04-07T17:04:59.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:04:58 smithi144 bash[66654]: cluster 2024-04-07T17:04:57.737455+0000 mon.a (mon.0) 2588 : cluster [DBG] osdmap e1339: 8 total, 8 up, 8 in 2024-04-07T17:04:59.202 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:04:58 smithi100 bash[74942]: cluster 2024-04-07T17:04:57.737455+0000 mon.a (mon.0) 2588 : cluster [DBG] osdmap e1339: 8 total, 8 up, 8 in 2024-04-07T17:04:59.202 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:04:58 smithi100 bash[76202]: cluster 2024-04-07T17:04:57.737455+0000 mon.a (mon.0) 2588 : cluster [DBG] osdmap e1339: 8 total, 8 up, 8 in 2024-04-07T17:04:59.307 INFO:tasks.workunit.client.0.smithi100.stdout:resize: 133825899 2024-04-07T17:04:59.347 INFO:tasks.workunit.client.0.smithi100.stdout:migration_prepare 2024-04-07T17:05:00.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:04:59 smithi144 bash[66654]: cluster 2024-04-07T17:04:57.875306+0000 mgr.y (mgr.25175) 1488 : cluster [DBG] pgmap v2402: 169 pgs: 169 active+clean; 1.2 GiB data, 6.1 GiB used, 709 GiB / 715 GiB avail; 11 MiB/s rd, 9.5 MiB/s wr, 329 op/s 2024-04-07T17:05:00.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:04:59 smithi144 bash[66654]: cluster 2024-04-07T17:04:58.747406+0000 mon.a (mon.0) 2589 : cluster [DBG] osdmap e1340: 8 total, 8 up, 8 in 2024-04-07T17:05:00.202 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:04:59 smithi100 bash[74942]: cluster 2024-04-07T17:04:57.875306+0000 mgr.y (mgr.25175) 1488 : cluster [DBG] pgmap v2402: 169 pgs: 169 active+clean; 1.2 GiB data, 6.1 GiB used, 709 GiB / 715 GiB avail; 11 MiB/s rd, 9.5 MiB/s wr, 329 op/s 2024-04-07T17:05:00.202 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:04:59 smithi100 bash[74942]: cluster 2024-04-07T17:04:58.747406+0000 mon.a (mon.0) 2589 : cluster [DBG] osdmap e1340: 8 total, 8 up, 8 in 2024-04-07T17:05:00.202 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:04:59 smithi100 bash[76202]: cluster 2024-04-07T17:04:57.875306+0000 mgr.y (mgr.25175) 1488 : cluster [DBG] pgmap v2402: 169 pgs: 169 active+clean; 1.2 GiB data, 6.1 GiB used, 709 GiB / 715 GiB avail; 11 MiB/s rd, 9.5 MiB/s wr, 329 op/s 2024-04-07T17:05:00.202 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:04:59 smithi100 bash[76202]: cluster 2024-04-07T17:04:58.747406+0000 mon.a (mon.0) 2589 : cluster [DBG] osdmap e1340: 8 total, 8 up, 8 in 2024-04-07T17:05:00.507 INFO:tasks.workunit.client.0.smithi100.stdout:migration_execute 2024-04-07T17:05:01.144 INFO:tasks.workunit.client.0.smithi100.stdout:migration_commit 2024-04-07T17:05:01.809 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestMigration.Stress (11616 ms) 2024-04-07T17:05:01.809 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestMigration.Stress2 2024-04-07T17:05:01.900 INFO:tasks.workunit.client.0.smithi100.stdout:resize: 83886080 2024-04-07T17:05:01.931 INFO:tasks.workunit.client.0.smithi100.stdout:write: A 42326523~559413 2024-04-07T17:05:01.951 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:05:01 smithi100 bash[74942]: cluster 2024-04-07T17:04:59.876398+0000 mgr.y (mgr.25175) 1489 : cluster [DBG] pgmap v2404: 169 pgs: 169 active+clean; 1.3 GiB data, 6.2 GiB used, 709 GiB / 715 GiB avail; 6.7 MiB/s rd, 19 MiB/s wr, 417 op/s 2024-04-07T17:05:01.952 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:05:01 smithi100 bash[76202]: cluster 2024-04-07T17:04:59.876398+0000 mgr.y (mgr.25175) 1489 : cluster [DBG] pgmap v2404: 169 pgs: 169 active+clean; 1.3 GiB data, 6.2 GiB used, 709 GiB / 715 GiB avail; 6.7 MiB/s rd, 19 MiB/s wr, 417 op/s 2024-04-07T17:05:01.960 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 71139006~2176015 2024-04-07T17:05:01.960 INFO:tasks.workunit.client.0.smithi100.stdout:write: A 6102386~3319346 2024-04-07T17:05:02.002 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 52763723~6408254 2024-04-07T17:05:02.003 INFO:tasks.workunit.client.0.smithi100.stdout:write: A 56590848~2986551 2024-04-07T17:05:02.041 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 59856839~5699721 2024-04-07T17:05:02.046 INFO:tasks.workunit.client.0.smithi100.stdout:write: A 29486389~6905621 2024-04-07T17:05:02.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:05:01 smithi144 bash[66654]: cluster 2024-04-07T17:04:59.876398+0000 mgr.y (mgr.25175) 1489 : cluster [DBG] pgmap v2404: 169 pgs: 169 active+clean; 1.3 GiB data, 6.2 GiB used, 709 GiB / 715 GiB avail; 6.7 MiB/s rd, 19 MiB/s wr, 417 op/s 2024-04-07T17:05:02.118 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 185993~732620 2024-04-07T17:05:02.118 INFO:tasks.workunit.client.0.smithi100.stdout:snap_create: snap0 2024-04-07T17:05:02.615 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:05:02 smithi144 bash[95233]: ts=2024-04-07T17:05:02.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:05:03.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:05:02 smithi144 bash[66654]: audit 2024-04-07T17:05:01.804778+0000 mon.a (mon.0) 2590 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:05:03.202 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:05:02 smithi100 bash[74942]: audit 2024-04-07T17:05:01.804778+0000 mon.a (mon.0) 2590 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:05:03.202 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:05:02 smithi100 bash[76202]: audit 2024-04-07T17:05:01.804778+0000 mon.a (mon.0) 2590 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:05:03.801 INFO:tasks.workunit.client.0.smithi100.stdout:resize: 145224632 2024-04-07T17:05:03.839 INFO:tasks.workunit.client.0.smithi100.stdout:write: B 128909920~3788148 2024-04-07T17:05:03.862 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 82214272~5955238 2024-04-07T17:05:03.862 INFO:tasks.workunit.client.0.smithi100.stdout:write: B 100040767~8178563 2024-04-07T17:05:03.888 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 31353299~5795021 2024-04-07T17:05:03.888 INFO:tasks.workunit.client.0.smithi100.stdout:write: B 35434155~6631354 2024-04-07T17:05:03.932 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 49274760~3591162 2024-04-07T17:05:03.932 INFO:tasks.workunit.client.0.smithi100.stdout:write: B 55907454~3456272 2024-04-07T17:05:03.934 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 19560758~51687 2024-04-07T17:05:03.935 INFO:tasks.workunit.client.0.smithi100.stdout:snap_create: snap1 2024-04-07T17:05:04.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:05:03 smithi144 bash[66654]: cluster 2024-04-07T17:05:01.877436+0000 mgr.y (mgr.25175) 1490 : cluster [DBG] pgmap v2405: 169 pgs: 169 active+clean; 1.3 GiB data, 6.3 GiB used, 709 GiB / 715 GiB avail; 5.0 MiB/s rd, 14 MiB/s wr, 457 op/s 2024-04-07T17:05:04.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:05:03 smithi144 bash[66654]: cluster 2024-04-07T17:05:02.765334+0000 mon.a (mon.0) 2591 : cluster [DBG] osdmap e1341: 8 total, 8 up, 8 in 2024-04-07T17:05:04.202 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:05:03 smithi100 bash[74942]: cluster 2024-04-07T17:05:01.877436+0000 mgr.y (mgr.25175) 1490 : cluster [DBG] pgmap v2405: 169 pgs: 169 active+clean; 1.3 GiB data, 6.3 GiB used, 709 GiB / 715 GiB avail; 5.0 MiB/s rd, 14 MiB/s wr, 457 op/s 2024-04-07T17:05:04.202 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:05:03 smithi100 bash[74942]: cluster 2024-04-07T17:05:02.765334+0000 mon.a (mon.0) 2591 : cluster [DBG] osdmap e1341: 8 total, 8 up, 8 in 2024-04-07T17:05:04.202 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:05:03 smithi100 bash[76202]: cluster 2024-04-07T17:05:01.877436+0000 mgr.y (mgr.25175) 1490 : cluster [DBG] pgmap v2405: 169 pgs: 169 active+clean; 1.3 GiB data, 6.3 GiB used, 709 GiB / 715 GiB avail; 5.0 MiB/s rd, 14 MiB/s wr, 457 op/s 2024-04-07T17:05:04.202 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:05:03 smithi100 bash[76202]: cluster 2024-04-07T17:05:02.765334+0000 mon.a (mon.0) 2591 : cluster [DBG] osdmap e1341: 8 total, 8 up, 8 in 2024-04-07T17:05:05.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:05:04 smithi144 bash[66654]: cluster 2024-04-07T17:05:03.768254+0000 mon.a (mon.0) 2592 : cluster [DBG] osdmap e1342: 8 total, 8 up, 8 in 2024-04-07T17:05:05.202 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:05:04 smithi100 bash[74942]: cluster 2024-04-07T17:05:03.768254+0000 mon.a (mon.0) 2592 : cluster [DBG] osdmap e1342: 8 total, 8 up, 8 in 2024-04-07T17:05:05.202 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:05:04 smithi100 bash[76202]: cluster 2024-04-07T17:05:03.768254+0000 mon.a (mon.0) 2592 : cluster [DBG] osdmap e1342: 8 total, 8 up, 8 in 2024-04-07T17:05:05.827 INFO:tasks.workunit.client.0.smithi100.stdout:write: C 46966235~5506061 2024-04-07T17:05:05.852 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 106174829~752017 2024-04-07T17:05:05.852 INFO:tasks.workunit.client.0.smithi100.stdout:write: C 62743736~7037180 2024-04-07T17:05:05.882 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 125162825~1893666 2024-04-07T17:05:05.882 INFO:tasks.workunit.client.0.smithi100.stdout:write: C 123205907~2621220 2024-04-07T17:05:05.885 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 135724781~5915613 2024-04-07T17:05:05.885 INFO:tasks.workunit.client.0.smithi100.stdout:write: C 76812626~4697955 2024-04-07T17:05:05.892 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 108214777~8223286 2024-04-07T17:05:05.892 INFO:tasks.workunit.client.0.smithi100.stdout:snap_create: snap2 2024-04-07T17:05:06.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:05:05 smithi144 bash[66654]: cluster 2024-04-07T17:05:03.878653+0000 mgr.y (mgr.25175) 1491 : cluster [DBG] pgmap v2408: 169 pgs: 169 active+clean; 1.3 GiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 20 MiB/s rd, 17 MiB/s wr, 603 op/s 2024-04-07T17:05:06.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:05:05 smithi144 bash[66654]: cluster 2024-04-07T17:05:04.788100+0000 mon.a (mon.0) 2593 : cluster [DBG] osdmap e1343: 8 total, 8 up, 8 in 2024-04-07T17:05:06.202 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:05:05 smithi100 bash[74942]: cluster 2024-04-07T17:05:03.878653+0000 mgr.y (mgr.25175) 1491 : cluster [DBG] pgmap v2408: 169 pgs: 169 active+clean; 1.3 GiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 20 MiB/s rd, 17 MiB/s wr, 603 op/s 2024-04-07T17:05:06.202 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:05:05 smithi100 bash[74942]: cluster 2024-04-07T17:05:04.788100+0000 mon.a (mon.0) 2593 : cluster [DBG] osdmap e1343: 8 total, 8 up, 8 in 2024-04-07T17:05:06.202 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:05:05 smithi100 bash[76202]: cluster 2024-04-07T17:05:03.878653+0000 mgr.y (mgr.25175) 1491 : cluster [DBG] pgmap v2408: 169 pgs: 169 active+clean; 1.3 GiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 20 MiB/s rd, 17 MiB/s wr, 603 op/s 2024-04-07T17:05:06.202 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:05:05 smithi100 bash[76202]: cluster 2024-04-07T17:05:04.788100+0000 mon.a (mon.0) 2593 : cluster [DBG] osdmap e1343: 8 total, 8 up, 8 in 2024-04-07T17:05:07.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:05:06 smithi144 bash[66654]: cluster 2024-04-07T17:05:05.800977+0000 mon.a (mon.0) 2594 : cluster [DBG] osdmap e1344: 8 total, 8 up, 8 in 2024-04-07T17:05:07.202 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:05:06 smithi100 bash[74942]: cluster 2024-04-07T17:05:05.800977+0000 mon.a (mon.0) 2594 : cluster [DBG] osdmap e1344: 8 total, 8 up, 8 in 2024-04-07T17:05:07.202 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:05:06 smithi100 bash[76202]: cluster 2024-04-07T17:05:05.800977+0000 mon.a (mon.0) 2594 : cluster [DBG] osdmap e1344: 8 total, 8 up, 8 in 2024-04-07T17:05:07.846 INFO:tasks.workunit.client.0.smithi100.stdout:snap_protect: snap2 2024-04-07T17:05:07.886 INFO:tasks.workunit.client.0.smithi100.stdout:clone image175 -> image178 2024-04-07T17:05:08.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:05:07 smithi144 bash[66654]: cluster 2024-04-07T17:05:05.879632+0000 mgr.y (mgr.25175) 1492 : cluster [DBG] pgmap v2411: 169 pgs: 169 active+clean; 1.4 GiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 39 MiB/s rd, 12 MiB/s wr, 897 op/s 2024-04-07T17:05:08.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:05:07 smithi144 bash[66654]: cluster 2024-04-07T17:05:06.810976+0000 mon.a (mon.0) 2595 : cluster [DBG] osdmap e1345: 8 total, 8 up, 8 in 2024-04-07T17:05:08.175 INFO:tasks.workunit.client.0.smithi100.stdout:resize: 177802824 2024-04-07T17:05:08.202 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:05:07 smithi100 bash[76202]: cluster 2024-04-07T17:05:05.879632+0000 mgr.y (mgr.25175) 1492 : cluster [DBG] pgmap v2411: 169 pgs: 169 active+clean; 1.4 GiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 39 MiB/s rd, 12 MiB/s wr, 897 op/s 2024-04-07T17:05:08.202 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:05:07 smithi100 bash[76202]: cluster 2024-04-07T17:05:06.810976+0000 mon.a (mon.0) 2595 : cluster [DBG] osdmap e1345: 8 total, 8 up, 8 in 2024-04-07T17:05:08.202 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:05:07 smithi100 bash[74942]: cluster 2024-04-07T17:05:05.879632+0000 mgr.y (mgr.25175) 1492 : cluster [DBG] pgmap v2411: 169 pgs: 169 active+clean; 1.4 GiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 39 MiB/s rd, 12 MiB/s wr, 897 op/s 2024-04-07T17:05:08.202 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:05:07 smithi100 bash[74942]: cluster 2024-04-07T17:05:06.810976+0000 mon.a (mon.0) 2595 : cluster [DBG] osdmap e1345: 8 total, 8 up, 8 in 2024-04-07T17:05:08.208 INFO:tasks.workunit.client.0.smithi100.stdout:write: D 168297372~151472 2024-04-07T17:05:08.235 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 169356763~1435658 2024-04-07T17:05:08.241 INFO:tasks.workunit.client.0.smithi100.stdout:write: D 128602891~4116375 2024-04-07T17:05:08.303 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 117321253~6504604 2024-04-07T17:05:08.355 INFO:tasks.workunit.client.0.smithi100.stdout:write: D 163343906~7751626 2024-04-07T17:05:08.431 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 128240428~6996405 2024-04-07T17:05:08.444 INFO:tasks.workunit.client.0.smithi100.stdout:write: D 90756217~2719863 2024-04-07T17:05:08.480 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 2990732~6269109 2024-04-07T17:05:08.506 INFO:tasks.workunit.client.0.smithi100.stdout:snap_create: snap3 2024-04-07T17:05:08.702 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:05:08 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:05:08] "GET /metrics HTTP/1.1" 200 38929 "" "Prometheus/2.43.0" 2024-04-07T17:05:09.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:05:08 smithi144 bash[66654]: cluster 2024-04-07T17:05:07.820075+0000 mon.a (mon.0) 2596 : cluster [DBG] osdmap e1346: 8 total, 8 up, 8 in 2024-04-07T17:05:09.202 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:05:08 smithi100 bash[74942]: cluster 2024-04-07T17:05:07.820075+0000 mon.a (mon.0) 2596 : cluster [DBG] osdmap e1346: 8 total, 8 up, 8 in 2024-04-07T17:05:09.202 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:05:08 smithi100 bash[76202]: cluster 2024-04-07T17:05:07.820075+0000 mon.a (mon.0) 2596 : cluster [DBG] osdmap e1346: 8 total, 8 up, 8 in 2024-04-07T17:05:09.875 INFO:tasks.workunit.client.0.smithi100.stdout:snap_protect: snap3 2024-04-07T17:05:09.903 INFO:tasks.workunit.client.0.smithi100.stdout:clone image178 -> image180 2024-04-07T17:05:10.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:05:09 smithi144 bash[66654]: cluster 2024-04-07T17:05:07.881167+0000 mgr.y (mgr.25175) 1493 : cluster [DBG] pgmap v2414: 169 pgs: 169 active+clean; 1.4 GiB data, 6.5 GiB used, 709 GiB / 715 GiB avail; 15 MiB/s rd, 10 MiB/s wr, 696 op/s 2024-04-07T17:05:10.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:05:09 smithi144 bash[66654]: cluster 2024-04-07T17:05:08.836182+0000 mon.a (mon.0) 2597 : cluster [DBG] osdmap e1347: 8 total, 8 up, 8 in 2024-04-07T17:05:10.202 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:05:09 smithi100 bash[74942]: cluster 2024-04-07T17:05:07.881167+0000 mgr.y (mgr.25175) 1493 : cluster [DBG] pgmap v2414: 169 pgs: 169 active+clean; 1.4 GiB data, 6.5 GiB used, 709 GiB / 715 GiB avail; 15 MiB/s rd, 10 MiB/s wr, 696 op/s 2024-04-07T17:05:10.202 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:05:09 smithi100 bash[74942]: cluster 2024-04-07T17:05:08.836182+0000 mon.a (mon.0) 2597 : cluster [DBG] osdmap e1347: 8 total, 8 up, 8 in 2024-04-07T17:05:10.202 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:05:09 smithi100 bash[76202]: cluster 2024-04-07T17:05:07.881167+0000 mgr.y (mgr.25175) 1493 : cluster [DBG] pgmap v2414: 169 pgs: 169 active+clean; 1.4 GiB data, 6.5 GiB used, 709 GiB / 715 GiB avail; 15 MiB/s rd, 10 MiB/s wr, 696 op/s 2024-04-07T17:05:10.202 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:05:09 smithi100 bash[76202]: cluster 2024-04-07T17:05:08.836182+0000 mon.a (mon.0) 2597 : cluster [DBG] osdmap e1347: 8 total, 8 up, 8 in 2024-04-07T17:05:10.249 INFO:tasks.workunit.client.0.smithi100.stdout:resize: 254900582 2024-04-07T17:05:10.290 INFO:tasks.workunit.client.0.smithi100.stdout:migration_prepare 2024-04-07T17:05:11.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:05:10 smithi144 bash[66654]: cluster 2024-04-07T17:05:09.842944+0000 mon.a (mon.0) 2598 : cluster [DBG] osdmap e1348: 8 total, 8 up, 8 in 2024-04-07T17:05:11.202 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:05:10 smithi100 bash[74942]: cluster 2024-04-07T17:05:09.842944+0000 mon.a (mon.0) 2598 : cluster [DBG] osdmap e1348: 8 total, 8 up, 8 in 2024-04-07T17:05:11.202 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:05:10 smithi100 bash[76202]: cluster 2024-04-07T17:05:09.842944+0000 mon.a (mon.0) 2598 : cluster [DBG] osdmap e1348: 8 total, 8 up, 8 in 2024-04-07T17:05:11.456 INFO:tasks.workunit.client.0.smithi100.stdout:write: a 145562164~1562967 2024-04-07T17:05:11.525 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 90591748~1681233 2024-04-07T17:05:11.558 INFO:tasks.workunit.client.0.smithi100.stdout:write: a 229629262~3897375 2024-04-07T17:05:11.591 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 250809533~405126 2024-04-07T17:05:11.598 INFO:tasks.workunit.client.0.smithi100.stdout:write: a 103813481~255165 2024-04-07T17:05:11.647 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 59491497~3356687 2024-04-07T17:05:11.686 INFO:tasks.workunit.client.0.smithi100.stdout:write: a 160588615~3301657 2024-04-07T17:05:11.765 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 68878092~4883736 2024-04-07T17:05:11.810 INFO:tasks.workunit.client.0.smithi100.stdout:snap_create: user0 2024-04-07T17:05:12.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:05:11 smithi144 bash[66654]: cluster 2024-04-07T17:05:09.882087+0000 mgr.y (mgr.25175) 1494 : cluster [DBG] pgmap v2417: 169 pgs: 169 active+clean; 1.5 GiB data, 6.7 GiB used, 709 GiB / 715 GiB avail; 2.1 MiB/s rd, 22 MiB/s wr, 265 op/s 2024-04-07T17:05:12.202 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:05:11 smithi100 bash[74942]: cluster 2024-04-07T17:05:09.882087+0000 mgr.y (mgr.25175) 1494 : cluster [DBG] pgmap v2417: 169 pgs: 169 active+clean; 1.5 GiB data, 6.7 GiB used, 709 GiB / 715 GiB avail; 2.1 MiB/s rd, 22 MiB/s wr, 265 op/s 2024-04-07T17:05:12.202 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:05:11 smithi100 bash[76202]: cluster 2024-04-07T17:05:09.882087+0000 mgr.y (mgr.25175) 1494 : cluster [DBG] pgmap v2417: 169 pgs: 169 active+clean; 1.5 GiB data, 6.7 GiB used, 709 GiB / 715 GiB avail; 2.1 MiB/s rd, 22 MiB/s wr, 265 op/s 2024-04-07T17:05:12.615 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:05:12 smithi144 bash[95233]: ts=2024-04-07T17:05:12.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:05:12.906 INFO:tasks.workunit.client.0.smithi100.stdout:write: b 174142068~4195067 2024-04-07T17:05:12.925 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 230050501~7700687 2024-04-07T17:05:12.926 INFO:tasks.workunit.client.0.smithi100.stdout:write: b 66932665~1679787 2024-04-07T17:05:12.931 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 168929468~686570 2024-04-07T17:05:12.932 INFO:tasks.workunit.client.0.smithi100.stdout:write: b 121719355~6253742 2024-04-07T17:05:12.960 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 72521941~2107559 2024-04-07T17:05:12.960 INFO:tasks.workunit.client.0.smithi100.stdout:write: b 97791669~6724781 2024-04-07T17:05:12.984 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 120070931~6410538 2024-04-07T17:05:13.118 INFO:tasks.workunit.client.0.smithi100.stdout:snap_create: user1 2024-04-07T17:05:13.202 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:05:12 smithi100 bash[74942]: cluster 2024-04-07T17:05:11.854507+0000 mon.a (mon.0) 2599 : cluster [DBG] osdmap e1349: 8 total, 8 up, 8 in 2024-04-07T17:05:13.202 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:05:12 smithi100 bash[74942]: cluster 2024-04-07T17:05:11.883136+0000 mgr.y (mgr.25175) 1495 : cluster [DBG] pgmap v2419: 169 pgs: 169 active+clean; 1.5 GiB data, 6.7 GiB used, 709 GiB / 715 GiB avail; 2.0 MiB/s rd, 16 MiB/s wr, 162 op/s 2024-04-07T17:05:13.202 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:05:12 smithi100 bash[76202]: cluster 2024-04-07T17:05:11.854507+0000 mon.a (mon.0) 2599 : cluster [DBG] osdmap e1349: 8 total, 8 up, 8 in 2024-04-07T17:05:13.202 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:05:12 smithi100 bash[76202]: cluster 2024-04-07T17:05:11.883136+0000 mgr.y (mgr.25175) 1495 : cluster [DBG] pgmap v2419: 169 pgs: 169 active+clean; 1.5 GiB data, 6.7 GiB used, 709 GiB / 715 GiB avail; 2.0 MiB/s rd, 16 MiB/s wr, 162 op/s 2024-04-07T17:05:13.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:05:12 smithi144 bash[66654]: cluster 2024-04-07T17:05:11.854507+0000 mon.a (mon.0) 2599 : cluster [DBG] osdmap e1349: 8 total, 8 up, 8 in 2024-04-07T17:05:13.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:05:12 smithi144 bash[66654]: cluster 2024-04-07T17:05:11.883136+0000 mgr.y (mgr.25175) 1495 : cluster [DBG] pgmap v2419: 169 pgs: 169 active+clean; 1.5 GiB data, 6.7 GiB used, 709 GiB / 715 GiB avail; 2.0 MiB/s rd, 16 MiB/s wr, 162 op/s 2024-04-07T17:05:14.202 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:05:13 smithi100 bash[74942]: cluster 2024-04-07T17:05:12.872851+0000 mon.a (mon.0) 2600 : cluster [DBG] osdmap e1350: 8 total, 8 up, 8 in 2024-04-07T17:05:14.202 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:05:13 smithi100 bash[76202]: cluster 2024-04-07T17:05:12.872851+0000 mon.a (mon.0) 2600 : cluster [DBG] osdmap e1350: 8 total, 8 up, 8 in 2024-04-07T17:05:14.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:05:13 smithi144 bash[66654]: cluster 2024-04-07T17:05:12.872851+0000 mon.a (mon.0) 2600 : cluster [DBG] osdmap e1350: 8 total, 8 up, 8 in 2024-04-07T17:05:14.942 INFO:tasks.workunit.client.0.smithi100.stdout:resize: 262305017 2024-04-07T17:05:14.972 INFO:tasks.workunit.client.0.smithi100.stdout:write: c 61071236~4623870 2024-04-07T17:05:15.011 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 256399380~5905637 2024-04-07T17:05:15.012 INFO:tasks.workunit.client.0.smithi100.stdout:write: c 155534428~1636113 2024-04-07T17:05:15.012 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 243193416~3324003 2024-04-07T17:05:15.013 INFO:tasks.workunit.client.0.smithi100.stdout:write: c 225309710~3968871 2024-04-07T17:05:15.025 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 47101494~1579318 2024-04-07T17:05:15.025 INFO:tasks.workunit.client.0.smithi100.stdout:write: c 57720134~6815569 2024-04-07T17:05:15.141 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 177128010~6396236 2024-04-07T17:05:15.142 INFO:tasks.workunit.client.0.smithi100.stdout:snap_create: user2 2024-04-07T17:05:15.202 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:05:14 smithi100 bash[74942]: cluster 2024-04-07T17:05:13.884433+0000 mgr.y (mgr.25175) 1496 : cluster [DBG] pgmap v2422: 169 pgs: 169 active+clean; 1.6 GiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 23 MiB/s rd, 13 MiB/s wr, 438 op/s 2024-04-07T17:05:15.202 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:05:14 smithi100 bash[74942]: cluster 2024-04-07T17:05:13.886567+0000 mon.a (mon.0) 2601 : cluster [DBG] osdmap e1351: 8 total, 8 up, 8 in 2024-04-07T17:05:15.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:05:14 smithi100 bash[76202]: cluster 2024-04-07T17:05:13.884433+0000 mgr.y (mgr.25175) 1496 : cluster [DBG] pgmap v2422: 169 pgs: 169 active+clean; 1.6 GiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 23 MiB/s rd, 13 MiB/s wr, 438 op/s 2024-04-07T17:05:15.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:05:14 smithi100 bash[76202]: cluster 2024-04-07T17:05:13.886567+0000 mon.a (mon.0) 2601 : cluster [DBG] osdmap e1351: 8 total, 8 up, 8 in 2024-04-07T17:05:15.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:05:14 smithi144 bash[66654]: cluster 2024-04-07T17:05:13.884433+0000 mgr.y (mgr.25175) 1496 : cluster [DBG] pgmap v2422: 169 pgs: 169 active+clean; 1.6 GiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 23 MiB/s rd, 13 MiB/s wr, 438 op/s 2024-04-07T17:05:15.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:05:14 smithi144 bash[66654]: cluster 2024-04-07T17:05:13.886567+0000 mon.a (mon.0) 2601 : cluster [DBG] osdmap e1351: 8 total, 8 up, 8 in 2024-04-07T17:05:16.202 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:05:15 smithi100 bash[74942]: cluster 2024-04-07T17:05:14.910550+0000 mon.a (mon.0) 2602 : cluster [DBG] osdmap e1352: 8 total, 8 up, 8 in 2024-04-07T17:05:16.202 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:05:15 smithi100 bash[76202]: cluster 2024-04-07T17:05:14.910550+0000 mon.a (mon.0) 2602 : cluster [DBG] osdmap e1352: 8 total, 8 up, 8 in 2024-04-07T17:05:16.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:05:15 smithi144 bash[66654]: cluster 2024-04-07T17:05:14.910550+0000 mon.a (mon.0) 2602 : cluster [DBG] osdmap e1352: 8 total, 8 up, 8 in 2024-04-07T17:05:16.958 INFO:tasks.workunit.client.0.smithi100.stdout:resize: 388161657 2024-04-07T17:05:16.998 INFO:tasks.workunit.client.0.smithi100.stdout:write: d 141951080~4047525 2024-04-07T17:05:17.041 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 361868655~4087066 2024-04-07T17:05:17.041 INFO:tasks.workunit.client.0.smithi100.stdout:write: d 353944783~197504 2024-04-07T17:05:17.042 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 226557634~6760231 2024-04-07T17:05:17.042 INFO:tasks.workunit.client.0.smithi100.stdout:write: d 96302701~7903758 2024-04-07T17:05:17.067 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 348369986~7648780 2024-04-07T17:05:17.068 INFO:tasks.workunit.client.0.smithi100.stdout:write: d 118284718~1894064 2024-04-07T17:05:17.069 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 206840605~4286465 2024-04-07T17:05:17.080 INFO:tasks.workunit.client.0.smithi100.stdout:snap_create: user3 2024-04-07T17:05:17.202 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:05:16 smithi100 bash[74942]: cluster 2024-04-07T17:05:15.885469+0000 mgr.y (mgr.25175) 1497 : cluster [DBG] pgmap v2424: 169 pgs: 169 active+clean; 1.6 GiB data, 7.0 GiB used, 708 GiB / 715 GiB avail; 37 MiB/s rd, 29 MiB/s wr, 678 op/s 2024-04-07T17:05:17.202 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:05:16 smithi100 bash[74942]: cluster 2024-04-07T17:05:15.921872+0000 mon.a (mon.0) 2603 : cluster [DBG] osdmap e1353: 8 total, 8 up, 8 in 2024-04-07T17:05:17.202 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:05:16 smithi100 bash[74942]: audit 2024-04-07T17:05:16.805334+0000 mon.a (mon.0) 2604 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:05:17.202 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:05:16 smithi100 bash[76202]: cluster 2024-04-07T17:05:15.885469+0000 mgr.y (mgr.25175) 1497 : cluster [DBG] pgmap v2424: 169 pgs: 169 active+clean; 1.6 GiB data, 7.0 GiB used, 708 GiB / 715 GiB avail; 37 MiB/s rd, 29 MiB/s wr, 678 op/s 2024-04-07T17:05:17.202 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:05:16 smithi100 bash[76202]: cluster 2024-04-07T17:05:15.921872+0000 mon.a (mon.0) 2603 : cluster [DBG] osdmap e1353: 8 total, 8 up, 8 in 2024-04-07T17:05:17.202 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:05:16 smithi100 bash[76202]: audit 2024-04-07T17:05:16.805334+0000 mon.a (mon.0) 2604 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:05:17.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:05:16 smithi144 bash[66654]: cluster 2024-04-07T17:05:15.885469+0000 mgr.y (mgr.25175) 1497 : cluster [DBG] pgmap v2424: 169 pgs: 169 active+clean; 1.6 GiB data, 7.0 GiB used, 708 GiB / 715 GiB avail; 37 MiB/s rd, 29 MiB/s wr, 678 op/s 2024-04-07T17:05:17.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:05:16 smithi144 bash[66654]: cluster 2024-04-07T17:05:15.921872+0000 mon.a (mon.0) 2603 : cluster [DBG] osdmap e1353: 8 total, 8 up, 8 in 2024-04-07T17:05:17.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:05:16 smithi144 bash[66654]: audit 2024-04-07T17:05:16.805334+0000 mon.a (mon.0) 2604 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:05:18.202 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:05:17 smithi100 bash[74942]: cluster 2024-04-07T17:05:16.932771+0000 mon.a (mon.0) 2605 : cluster [DBG] osdmap e1354: 8 total, 8 up, 8 in 2024-04-07T17:05:18.202 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:05:17 smithi100 bash[76202]: cluster 2024-04-07T17:05:16.932771+0000 mon.a (mon.0) 2605 : cluster [DBG] osdmap e1354: 8 total, 8 up, 8 in 2024-04-07T17:05:18.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:05:17 smithi144 bash[66654]: cluster 2024-04-07T17:05:16.932771+0000 mon.a (mon.0) 2605 : cluster [DBG] osdmap e1354: 8 total, 8 up, 8 in 2024-04-07T17:05:18.702 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:05:18 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:05:18] "GET /metrics HTTP/1.1" 200 38923 "" "Prometheus/2.43.0" 2024-04-07T17:05:18.990 INFO:tasks.workunit.client.0.smithi100.stdout:resize: 581121805 2024-04-07T17:05:19.045 INFO:tasks.workunit.client.0.smithi100.stdout:write: 1 58112180~14528045 2024-04-07T17:05:19.113 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 61744191~7264022 2024-04-07T17:05:19.202 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:05:18 smithi100 bash[74942]: cluster 2024-04-07T17:05:17.886899+0000 mgr.y (mgr.25175) 1498 : cluster [DBG] pgmap v2427: 169 pgs: 169 active+clean; 1.7 GiB data, 7.1 GiB used, 708 GiB / 715 GiB avail; 15 MiB/s rd, 29 MiB/s wr, 353 op/s 2024-04-07T17:05:19.202 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:05:18 smithi100 bash[74942]: cluster 2024-04-07T17:05:17.946281+0000 mon.a (mon.0) 2606 : cluster [DBG] osdmap e1355: 8 total, 8 up, 8 in 2024-04-07T17:05:19.202 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:05:18 smithi100 bash[76202]: cluster 2024-04-07T17:05:17.886899+0000 mgr.y (mgr.25175) 1498 : cluster [DBG] pgmap v2427: 169 pgs: 169 active+clean; 1.7 GiB data, 7.1 GiB used, 708 GiB / 715 GiB avail; 15 MiB/s rd, 29 MiB/s wr, 353 op/s 2024-04-07T17:05:19.202 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:05:18 smithi100 bash[76202]: cluster 2024-04-07T17:05:17.946281+0000 mon.a (mon.0) 2606 : cluster [DBG] osdmap e1355: 8 total, 8 up, 8 in 2024-04-07T17:05:19.206 INFO:tasks.workunit.client.0.smithi100.stdout:write: 2 116224361~14528045 2024-04-07T17:05:19.237 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 119856372~7264022 2024-04-07T17:05:19.345 INFO:tasks.workunit.client.0.smithi100.stdout:write: 3 174336541~14528045 2024-04-07T17:05:19.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:05:18 smithi144 bash[66654]: cluster 2024-04-07T17:05:17.886899+0000 mgr.y (mgr.25175) 1498 : cluster [DBG] pgmap v2427: 169 pgs: 169 active+clean; 1.7 GiB data, 7.1 GiB used, 708 GiB / 715 GiB avail; 15 MiB/s rd, 29 MiB/s wr, 353 op/s 2024-04-07T17:05:19.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:05:18 smithi144 bash[66654]: cluster 2024-04-07T17:05:17.946281+0000 mon.a (mon.0) 2606 : cluster [DBG] osdmap e1355: 8 total, 8 up, 8 in 2024-04-07T17:05:19.376 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 177968552~7264022 2024-04-07T17:05:19.474 INFO:tasks.workunit.client.0.smithi100.stdout:write: 4 232448722~14528045 2024-04-07T17:05:19.481 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 236080733~7264022 2024-04-07T17:05:19.552 INFO:tasks.workunit.client.0.smithi100.stdout:write: 5 290560902~14528045 2024-04-07T17:05:19.581 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 294192913~7264022 2024-04-07T17:05:20.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:05:19 smithi144 bash[66654]: cluster 2024-04-07T17:05:18.956115+0000 mon.a (mon.0) 2607 : cluster [DBG] osdmap e1356: 8 total, 8 up, 8 in 2024-04-07T17:05:20.452 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:05:19 smithi100 bash[74942]: cluster 2024-04-07T17:05:18.956115+0000 mon.a (mon.0) 2607 : cluster [DBG] osdmap e1356: 8 total, 8 up, 8 in 2024-04-07T17:05:20.452 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:05:19 smithi100 bash[76202]: cluster 2024-04-07T17:05:18.956115+0000 mon.a (mon.0) 2607 : cluster [DBG] osdmap e1356: 8 total, 8 up, 8 in 2024-04-07T17:05:21.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:05:20 smithi144 bash[66654]: cluster 2024-04-07T17:05:19.888410+0000 mgr.y (mgr.25175) 1499 : cluster [DBG] pgmap v2430: 169 pgs: 169 active+clean; 1.7 GiB data, 7.3 GiB used, 708 GiB / 715 GiB avail; 8.1 MiB/s rd, 27 MiB/s wr, 300 op/s 2024-04-07T17:05:21.452 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:05:20 smithi100 bash[74942]: cluster 2024-04-07T17:05:19.888410+0000 mgr.y (mgr.25175) 1499 : cluster [DBG] pgmap v2430: 169 pgs: 169 active+clean; 1.7 GiB data, 7.3 GiB used, 708 GiB / 715 GiB avail; 8.1 MiB/s rd, 27 MiB/s wr, 300 op/s 2024-04-07T17:05:21.452 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:05:20 smithi100 bash[76202]: cluster 2024-04-07T17:05:19.888410+0000 mgr.y (mgr.25175) 1499 : cluster [DBG] pgmap v2430: 169 pgs: 169 active+clean; 1.7 GiB data, 7.3 GiB used, 708 GiB / 715 GiB avail; 8.1 MiB/s rd, 27 MiB/s wr, 300 op/s 2024-04-07T17:05:22.615 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:05:22 smithi144 bash[95233]: ts=2024-04-07T17:05:22.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:05:22.857 INFO:tasks.workunit.client.0.smithi100.stdout:migration_execute 2024-04-07T17:05:23.349 INFO:tasks.workunit.client.0.smithi100.stdout:migration_commit 2024-04-07T17:05:23.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:05:22 smithi144 bash[66654]: cluster 2024-04-07T17:05:21.889552+0000 mgr.y (mgr.25175) 1500 : cluster [DBG] pgmap v2431: 169 pgs: 169 active+clean; 1.8 GiB data, 7.3 GiB used, 708 GiB / 715 GiB avail; 11 MiB/s rd, 26 MiB/s wr, 221 op/s 2024-04-07T17:05:23.452 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:05:22 smithi100 bash[74942]: cluster 2024-04-07T17:05:21.889552+0000 mgr.y (mgr.25175) 1500 : cluster [DBG] pgmap v2431: 169 pgs: 169 active+clean; 1.8 GiB data, 7.3 GiB used, 708 GiB / 715 GiB avail; 11 MiB/s rd, 26 MiB/s wr, 221 op/s 2024-04-07T17:05:23.452 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:05:22 smithi100 bash[76202]: cluster 2024-04-07T17:05:21.889552+0000 mgr.y (mgr.25175) 1500 : cluster [DBG] pgmap v2431: 169 pgs: 169 active+clean; 1.8 GiB data, 7.3 GiB used, 708 GiB / 715 GiB avail; 11 MiB/s rd, 26 MiB/s wr, 221 op/s 2024-04-07T17:05:25.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:05:24 smithi144 bash[66654]: cluster 2024-04-07T17:05:23.891204+0000 mgr.y (mgr.25175) 1501 : cluster [DBG] pgmap v2432: 169 pgs: 169 active+clean; 1.8 GiB data, 7.6 GiB used, 708 GiB / 715 GiB avail; 66 MiB/s rd, 33 MiB/s wr, 350 op/s 2024-04-07T17:05:25.452 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:05:24 smithi100 bash[74942]: cluster 2024-04-07T17:05:23.891204+0000 mgr.y (mgr.25175) 1501 : cluster [DBG] pgmap v2432: 169 pgs: 169 active+clean; 1.8 GiB data, 7.6 GiB used, 708 GiB / 715 GiB avail; 66 MiB/s rd, 33 MiB/s wr, 350 op/s 2024-04-07T17:05:25.452 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:05:24 smithi100 bash[76202]: cluster 2024-04-07T17:05:23.891204+0000 mgr.y (mgr.25175) 1501 : cluster [DBG] pgmap v2432: 169 pgs: 169 active+clean; 1.8 GiB data, 7.6 GiB used, 708 GiB / 715 GiB avail; 66 MiB/s rd, 33 MiB/s wr, 350 op/s 2024-04-07T17:05:26.648 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestMigration.Stress2 (24839 ms) 2024-04-07T17:05:26.648 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestMigration.StressLive 2024-04-07T17:05:26.734 INFO:tasks.workunit.client.0.smithi100.stdout:resize: 83886080 2024-04-07T17:05:26.764 INFO:tasks.workunit.client.0.smithi100.stdout:write: A 57507010~78308 2024-04-07T17:05:26.790 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 45990220~2618160 2024-04-07T17:05:26.790 INFO:tasks.workunit.client.0.smithi100.stdout:write: A 12586087~5992656 2024-04-07T17:05:26.857 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 29585441~5977432 2024-04-07T17:05:26.857 INFO:tasks.workunit.client.0.smithi100.stdout:write: A 12373668~1583256 2024-04-07T17:05:26.879 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 2527119~2158658 2024-04-07T17:05:26.879 INFO:tasks.workunit.client.0.smithi100.stdout:write: A 64926439~3672476 2024-04-07T17:05:26.921 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 66479798~4157552 2024-04-07T17:05:26.928 INFO:tasks.workunit.client.0.smithi100.stdout:snap_create: snap0 2024-04-07T17:05:27.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:05:26 smithi144 bash[66654]: cluster 2024-04-07T17:05:25.892698+0000 mgr.y (mgr.25175) 1502 : cluster [DBG] pgmap v2433: 169 pgs: 169 active+clean; 1.9 GiB data, 7.6 GiB used, 708 GiB / 715 GiB avail; 120 MiB/s rd, 29 MiB/s wr, 387 op/s 2024-04-07T17:05:27.452 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:05:26 smithi100 bash[74942]: cluster 2024-04-07T17:05:25.892698+0000 mgr.y (mgr.25175) 1502 : cluster [DBG] pgmap v2433: 169 pgs: 169 active+clean; 1.9 GiB data, 7.6 GiB used, 708 GiB / 715 GiB avail; 120 MiB/s rd, 29 MiB/s wr, 387 op/s 2024-04-07T17:05:27.452 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:05:26 smithi100 bash[76202]: cluster 2024-04-07T17:05:25.892698+0000 mgr.y (mgr.25175) 1502 : cluster [DBG] pgmap v2433: 169 pgs: 169 active+clean; 1.9 GiB data, 7.6 GiB used, 708 GiB / 715 GiB avail; 120 MiB/s rd, 29 MiB/s wr, 387 op/s 2024-04-07T17:05:28.032 INFO:tasks.workunit.client.0.smithi100.stdout:snap_protect: snap0 2024-04-07T17:05:28.061 INFO:tasks.workunit.client.0.smithi100.stdout:clone image181 -> image184 2024-04-07T17:05:28.316 INFO:tasks.workunit.client.0.smithi100.stdout:resize: 139783287 2024-04-07T17:05:28.347 INFO:tasks.workunit.client.0.smithi100.stdout:write: B 61931146~6490466 2024-04-07T17:05:28.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:05:28 smithi144 bash[66654]: cluster 2024-04-07T17:05:26.983503+0000 mon.a (mon.0) 2608 : cluster [DBG] osdmap e1357: 8 total, 8 up, 8 in 2024-04-07T17:05:28.427 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:05:28 smithi100 bash[74942]: cluster 2024-04-07T17:05:26.983503+0000 mon.a (mon.0) 2608 : cluster [DBG] osdmap e1357: 8 total, 8 up, 8 in 2024-04-07T17:05:28.427 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:05:28 smithi100 bash[76202]: cluster 2024-04-07T17:05:26.983503+0000 mon.a (mon.0) 2608 : cluster [DBG] osdmap e1357: 8 total, 8 up, 8 in 2024-04-07T17:05:28.427 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 28371999~5080709 2024-04-07T17:05:28.445 INFO:tasks.workunit.client.0.smithi100.stdout:write: B 108730062~2148907 2024-04-07T17:05:28.482 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 126538128~4778084 2024-04-07T17:05:28.482 INFO:tasks.workunit.client.0.smithi100.stdout:write: B 32178051~3884084 2024-04-07T17:05:28.526 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 15613838~1495701 2024-04-07T17:05:28.593 INFO:tasks.workunit.client.0.smithi100.stdout:write: B 73037417~4267486 2024-04-07T17:05:28.639 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 128739305~4036214 2024-04-07T17:05:28.640 INFO:tasks.workunit.client.0.smithi100.stdout:snap_create: snap1 2024-04-07T17:05:28.702 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:05:28 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:05:28] "GET /metrics HTTP/1.1" 200 38923 "" "Prometheus/2.43.0" 2024-04-07T17:05:29.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:05:29 smithi144 bash[66654]: cluster 2024-04-07T17:05:27.894188+0000 mgr.y (mgr.25175) 1503 : cluster [DBG] pgmap v2435: 169 pgs: 169 active+clean; 1.9 GiB data, 7.6 GiB used, 708 GiB / 715 GiB avail; 140 MiB/s rd, 22 MiB/s wr, 357 op/s 2024-04-07T17:05:29.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:05:29 smithi144 bash[66654]: cluster 2024-04-07T17:05:28.008612+0000 mon.a (mon.0) 2609 : cluster [DBG] osdmap e1358: 8 total, 8 up, 8 in 2024-04-07T17:05:29.452 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:05:29 smithi100 bash[74942]: cluster 2024-04-07T17:05:27.894188+0000 mgr.y (mgr.25175) 1503 : cluster [DBG] pgmap v2435: 169 pgs: 169 active+clean; 1.9 GiB data, 7.6 GiB used, 708 GiB / 715 GiB avail; 140 MiB/s rd, 22 MiB/s wr, 357 op/s 2024-04-07T17:05:29.452 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:05:29 smithi100 bash[74942]: cluster 2024-04-07T17:05:28.008612+0000 mon.a (mon.0) 2609 : cluster [DBG] osdmap e1358: 8 total, 8 up, 8 in 2024-04-07T17:05:29.452 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:05:29 smithi100 bash[76202]: cluster 2024-04-07T17:05:27.894188+0000 mgr.y (mgr.25175) 1503 : cluster [DBG] pgmap v2435: 169 pgs: 169 active+clean; 1.9 GiB data, 7.6 GiB used, 708 GiB / 715 GiB avail; 140 MiB/s rd, 22 MiB/s wr, 357 op/s 2024-04-07T17:05:29.452 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:05:29 smithi100 bash[76202]: cluster 2024-04-07T17:05:28.008612+0000 mon.a (mon.0) 2609 : cluster [DBG] osdmap e1358: 8 total, 8 up, 8 in 2024-04-07T17:05:30.051 INFO:tasks.workunit.client.0.smithi100.stdout:snap_protect: snap1 2024-04-07T17:05:30.085 INFO:tasks.workunit.client.0.smithi100.stdout:clone image184 -> image186 2024-04-07T17:05:30.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:05:30 smithi144 bash[66654]: cluster 2024-04-07T17:05:29.020792+0000 mon.a (mon.0) 2610 : cluster [DBG] osdmap e1359: 8 total, 8 up, 8 in 2024-04-07T17:05:30.429 INFO:tasks.workunit.client.0.smithi100.stdout:resize: 184887307 2024-04-07T17:05:30.452 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:05:30 smithi100 bash[74942]: cluster 2024-04-07T17:05:29.020792+0000 mon.a (mon.0) 2610 : cluster [DBG] osdmap e1359: 8 total, 8 up, 8 in 2024-04-07T17:05:30.452 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:05:30 smithi100 bash[76202]: cluster 2024-04-07T17:05:29.020792+0000 mon.a (mon.0) 2610 : cluster [DBG] osdmap e1359: 8 total, 8 up, 8 in 2024-04-07T17:05:30.461 INFO:tasks.workunit.client.0.smithi100.stdout:write: C 92013375~2306121 2024-04-07T17:05:30.520 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 128942175~1499943 2024-04-07T17:05:30.538 INFO:tasks.workunit.client.0.smithi100.stdout:write: C 5172419~3955461 2024-04-07T17:05:30.619 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 6085428~2282354 2024-04-07T17:05:30.626 INFO:tasks.workunit.client.0.smithi100.stdout:write: C 25150827~4543353 2024-04-07T17:05:30.692 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 75528121~4079463 2024-04-07T17:05:30.715 INFO:tasks.workunit.client.0.smithi100.stdout:write: C 131107662~114447 2024-04-07T17:05:30.723 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 10035351~6801013 2024-04-07T17:05:30.760 INFO:tasks.workunit.client.0.smithi100.stdout:snap_create: snap2 2024-04-07T17:05:31.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:05:31 smithi144 bash[66654]: cluster 2024-04-07T17:05:29.895163+0000 mgr.y (mgr.25175) 1504 : cluster [DBG] pgmap v2438: 169 pgs: 169 active+clean; 1.9 GiB data, 7.7 GiB used, 708 GiB / 715 GiB avail; 202 MiB/s rd, 14 MiB/s wr, 526 op/s 2024-04-07T17:05:31.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:05:31 smithi144 bash[66654]: cluster 2024-04-07T17:05:30.030128+0000 mon.a (mon.0) 2611 : cluster [DBG] osdmap e1360: 8 total, 8 up, 8 in 2024-04-07T17:05:31.452 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:05:31 smithi100 bash[74942]: cluster 2024-04-07T17:05:29.895163+0000 mgr.y (mgr.25175) 1504 : cluster [DBG] pgmap v2438: 169 pgs: 169 active+clean; 1.9 GiB data, 7.7 GiB used, 708 GiB / 715 GiB avail; 202 MiB/s rd, 14 MiB/s wr, 526 op/s 2024-04-07T17:05:31.452 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:05:31 smithi100 bash[74942]: cluster 2024-04-07T17:05:30.030128+0000 mon.a (mon.0) 2611 : cluster [DBG] osdmap e1360: 8 total, 8 up, 8 in 2024-04-07T17:05:31.452 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:05:31 smithi100 bash[76202]: cluster 2024-04-07T17:05:29.895163+0000 mgr.y (mgr.25175) 1504 : cluster [DBG] pgmap v2438: 169 pgs: 169 active+clean; 1.9 GiB data, 7.7 GiB used, 708 GiB / 715 GiB avail; 202 MiB/s rd, 14 MiB/s wr, 526 op/s 2024-04-07T17:05:31.452 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:05:31 smithi100 bash[76202]: cluster 2024-04-07T17:05:30.030128+0000 mon.a (mon.0) 2611 : cluster [DBG] osdmap e1360: 8 total, 8 up, 8 in 2024-04-07T17:05:32.099 INFO:tasks.workunit.client.0.smithi100.stdout:write: D 11432813~7776551 2024-04-07T17:05:32.151 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 128730379~1894108 2024-04-07T17:05:32.151 INFO:tasks.workunit.client.0.smithi100.stdout:write: D 26112469~8257147 2024-04-07T17:05:32.175 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 107851174~7844607 2024-04-07T17:05:32.182 INFO:tasks.workunit.client.0.smithi100.stdout:write: D 165802494~7348377 2024-04-07T17:05:32.225 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 147096350~6426106 2024-04-07T17:05:32.242 INFO:tasks.workunit.client.0.smithi100.stdout:write: D 166764055~5285548 2024-04-07T17:05:32.260 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 123212210~233196 2024-04-07T17:05:32.260 INFO:tasks.workunit.client.0.smithi100.stdout:snap_create: snap3 2024-04-07T17:05:32.315 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:05:32 smithi144 bash[66654]: cluster 2024-04-07T17:05:31.039947+0000 mon.a (mon.0) 2612 : cluster [DBG] osdmap e1361: 8 total, 8 up, 8 in 2024-04-07T17:05:32.315 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:05:32 smithi144 bash[66654]: audit 2024-04-07T17:05:31.805897+0000 mon.a (mon.0) 2613 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:05:32.452 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:05:32 smithi100 bash[74942]: cluster 2024-04-07T17:05:31.039947+0000 mon.a (mon.0) 2612 : cluster [DBG] osdmap e1361: 8 total, 8 up, 8 in 2024-04-07T17:05:32.452 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:05:32 smithi100 bash[74942]: audit 2024-04-07T17:05:31.805897+0000 mon.a (mon.0) 2613 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:05:32.452 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:05:32 smithi100 bash[76202]: cluster 2024-04-07T17:05:31.039947+0000 mon.a (mon.0) 2612 : cluster [DBG] osdmap e1361: 8 total, 8 up, 8 in 2024-04-07T17:05:32.452 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:05:32 smithi100 bash[76202]: audit 2024-04-07T17:05:31.805897+0000 mon.a (mon.0) 2613 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:05:32.615 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:05:32 smithi144 bash[95233]: ts=2024-04-07T17:05:32.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:05:33.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:05:33 smithi144 bash[66654]: cluster 2024-04-07T17:05:31.895927+0000 mgr.y (mgr.25175) 1505 : cluster [DBG] pgmap v2441: 169 pgs: 169 active+clean; 1.9 GiB data, 7.7 GiB used, 708 GiB / 715 GiB avail; 116 MiB/s rd, 11 MiB/s wr, 404 op/s 2024-04-07T17:05:33.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:05:33 smithi144 bash[66654]: cluster 2024-04-07T17:05:32.058764+0000 mon.a (mon.0) 2614 : cluster [DBG] osdmap e1362: 8 total, 8 up, 8 in 2024-04-07T17:05:33.452 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:05:33 smithi100 bash[74942]: cluster 2024-04-07T17:05:31.895927+0000 mgr.y (mgr.25175) 1505 : cluster [DBG] pgmap v2441: 169 pgs: 169 active+clean; 1.9 GiB data, 7.7 GiB used, 708 GiB / 715 GiB avail; 116 MiB/s rd, 11 MiB/s wr, 404 op/s 2024-04-07T17:05:33.452 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:05:33 smithi100 bash[74942]: cluster 2024-04-07T17:05:32.058764+0000 mon.a (mon.0) 2614 : cluster [DBG] osdmap e1362: 8 total, 8 up, 8 in 2024-04-07T17:05:33.452 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:05:33 smithi100 bash[76202]: cluster 2024-04-07T17:05:31.895927+0000 mgr.y (mgr.25175) 1505 : cluster [DBG] pgmap v2441: 169 pgs: 169 active+clean; 1.9 GiB data, 7.7 GiB used, 708 GiB / 715 GiB avail; 116 MiB/s rd, 11 MiB/s wr, 404 op/s 2024-04-07T17:05:33.452 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:05:33 smithi100 bash[76202]: cluster 2024-04-07T17:05:32.058764+0000 mon.a (mon.0) 2614 : cluster [DBG] osdmap e1362: 8 total, 8 up, 8 in 2024-04-07T17:05:34.123 INFO:tasks.workunit.client.0.smithi100.stdout:snap_protect: snap3 2024-04-07T17:05:34.160 INFO:tasks.workunit.client.0.smithi100.stdout:clone image186 -> image188 2024-04-07T17:05:34.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:05:34 smithi144 bash[66654]: cluster 2024-04-07T17:05:33.078364+0000 mon.a (mon.0) 2615 : cluster [DBG] osdmap e1363: 8 total, 8 up, 8 in 2024-04-07T17:05:34.452 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:05:34 smithi100 bash[74942]: cluster 2024-04-07T17:05:33.078364+0000 mon.a (mon.0) 2615 : cluster [DBG] osdmap e1363: 8 total, 8 up, 8 in 2024-04-07T17:05:34.452 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:05:34 smithi100 bash[76202]: cluster 2024-04-07T17:05:33.078364+0000 mon.a (mon.0) 2615 : cluster [DBG] osdmap e1363: 8 total, 8 up, 8 in 2024-04-07T17:05:34.601 INFO:tasks.workunit.client.0.smithi100.stdout:migration_prepare 2024-04-07T17:05:35.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:05:35 smithi144 bash[66654]: cluster 2024-04-07T17:05:33.896707+0000 mgr.y (mgr.25175) 1506 : cluster [DBG] pgmap v2444: 169 pgs: 169 active+clean; 2.0 GiB data, 7.8 GiB used, 707 GiB / 715 GiB avail; 19 MiB/s rd, 16 MiB/s wr, 238 op/s 2024-04-07T17:05:35.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:05:35 smithi144 bash[66654]: cluster 2024-04-07T17:05:34.093472+0000 mon.a (mon.0) 2616 : cluster [DBG] osdmap e1364: 8 total, 8 up, 8 in 2024-04-07T17:05:35.452 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:05:35 smithi100 bash[74942]: cluster 2024-04-07T17:05:33.896707+0000 mgr.y (mgr.25175) 1506 : cluster [DBG] pgmap v2444: 169 pgs: 169 active+clean; 2.0 GiB data, 7.8 GiB used, 707 GiB / 715 GiB avail; 19 MiB/s rd, 16 MiB/s wr, 238 op/s 2024-04-07T17:05:35.452 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:05:35 smithi100 bash[74942]: cluster 2024-04-07T17:05:34.093472+0000 mon.a (mon.0) 2616 : cluster [DBG] osdmap e1364: 8 total, 8 up, 8 in 2024-04-07T17:05:35.452 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:05:35 smithi100 bash[76202]: cluster 2024-04-07T17:05:33.896707+0000 mgr.y (mgr.25175) 1506 : cluster [DBG] pgmap v2444: 169 pgs: 169 active+clean; 2.0 GiB data, 7.8 GiB used, 707 GiB / 715 GiB avail; 19 MiB/s rd, 16 MiB/s wr, 238 op/s 2024-04-07T17:05:35.452 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:05:35 smithi100 bash[76202]: cluster 2024-04-07T17:05:34.093472+0000 mon.a (mon.0) 2616 : cluster [DBG] osdmap e1364: 8 total, 8 up, 8 in 2024-04-07T17:05:35.774 INFO:tasks.workunit.client.0.smithi100.stdout:migration_execute 2024-04-07T17:05:35.774 INFO:tasks.workunit.client.0.smithi100.stdout:write: a 155044317~3569302 2024-04-07T17:05:35.853 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 65819118~3913064 2024-04-07T17:05:35.897 INFO:tasks.workunit.client.0.smithi100.stdout:write: a 63671404~714734 2024-04-07T17:05:35.905 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 148444815~1444351 2024-04-07T17:05:35.935 INFO:tasks.workunit.client.0.smithi100.stdout:write: a 156536861~6572116 2024-04-07T17:05:35.997 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 47216067~3827648 2024-04-07T17:05:36.038 INFO:tasks.workunit.client.0.smithi100.stdout:write: a 1525936~6255389 2024-04-07T17:05:36.108 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 32977391~5673451 2024-04-07T17:05:36.152 INFO:tasks.workunit.client.0.smithi100.stdout:snap_create: user0 2024-04-07T17:05:37.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:05:37 smithi144 bash[66654]: cluster 2024-04-07T17:05:35.898044+0000 mgr.y (mgr.25175) 1507 : cluster [DBG] pgmap v2446: 169 pgs: 169 active+clean; 2.1 GiB data, 7.9 GiB used, 707 GiB / 715 GiB avail; 3.5 MiB/s rd, 17 MiB/s wr, 309 op/s 2024-04-07T17:05:37.452 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:05:37 smithi100 bash[74942]: cluster 2024-04-07T17:05:35.898044+0000 mgr.y (mgr.25175) 1507 : cluster [DBG] pgmap v2446: 169 pgs: 169 active+clean; 2.1 GiB data, 7.9 GiB used, 707 GiB / 715 GiB avail; 3.5 MiB/s rd, 17 MiB/s wr, 309 op/s 2024-04-07T17:05:37.452 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:05:37 smithi100 bash[76202]: cluster 2024-04-07T17:05:35.898044+0000 mgr.y (mgr.25175) 1507 : cluster [DBG] pgmap v2446: 169 pgs: 169 active+clean; 2.1 GiB data, 7.9 GiB used, 707 GiB / 715 GiB avail; 3.5 MiB/s rd, 17 MiB/s wr, 309 op/s 2024-04-07T17:05:38.151 INFO:tasks.workunit.client.0.smithi100.stdout:write: b 59089860~5541991 2024-04-07T17:05:38.183 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 64075310~6248712 2024-04-07T17:05:38.216 INFO:tasks.workunit.client.0.smithi100.stdout:write: b 94188217~2032821 2024-04-07T17:05:38.218 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 151494689~3730483 2024-04-07T17:05:38.219 INFO:tasks.workunit.client.0.smithi100.stdout:write: b 3736171~2527710 2024-04-07T17:05:38.228 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 140489551~1070872 2024-04-07T17:05:38.228 INFO:tasks.workunit.client.0.smithi100.stdout:write: b 93816977~493892 2024-04-07T17:05:38.275 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 155876141~4493896 2024-04-07T17:05:38.275 INFO:tasks.workunit.client.0.smithi100.stdout:snap_create: user1 2024-04-07T17:05:38.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:05:38 smithi144 bash[66654]: cluster 2024-04-07T17:05:37.096463+0000 mon.a (mon.0) 2617 : cluster [DBG] osdmap e1365: 8 total, 8 up, 8 in 2024-04-07T17:05:38.427 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:05:38 smithi100 bash[74942]: cluster 2024-04-07T17:05:37.096463+0000 mon.a (mon.0) 2617 : cluster [DBG] osdmap e1365: 8 total, 8 up, 8 in 2024-04-07T17:05:38.427 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:05:38 smithi100 bash[76202]: cluster 2024-04-07T17:05:37.096463+0000 mon.a (mon.0) 2617 : cluster [DBG] osdmap e1365: 8 total, 8 up, 8 in 2024-04-07T17:05:38.702 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:05:38 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:05:38] "GET /metrics HTTP/1.1" 200 38925 "" "Prometheus/2.43.0" 2024-04-07T17:05:39.452 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:05:39 smithi100 bash[74942]: cluster 2024-04-07T17:05:37.898844+0000 mgr.y (mgr.25175) 1508 : cluster [DBG] pgmap v2448: 169 pgs: 169 active+clean; 2.1 GiB data, 8.1 GiB used, 707 GiB / 715 GiB avail; 9.4 MiB/s rd, 16 MiB/s wr, 329 op/s 2024-04-07T17:05:39.452 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:05:39 smithi100 bash[74942]: cluster 2024-04-07T17:05:38.117322+0000 mon.a (mon.0) 2618 : cluster [DBG] osdmap e1366: 8 total, 8 up, 8 in 2024-04-07T17:05:39.452 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:05:39 smithi100 bash[76202]: cluster 2024-04-07T17:05:37.898844+0000 mgr.y (mgr.25175) 1508 : cluster [DBG] pgmap v2448: 169 pgs: 169 active+clean; 2.1 GiB data, 8.1 GiB used, 707 GiB / 715 GiB avail; 9.4 MiB/s rd, 16 MiB/s wr, 329 op/s 2024-04-07T17:05:39.452 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:05:39 smithi100 bash[76202]: cluster 2024-04-07T17:05:38.117322+0000 mon.a (mon.0) 2618 : cluster [DBG] osdmap e1366: 8 total, 8 up, 8 in 2024-04-07T17:05:39.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:05:39 smithi144 bash[66654]: cluster 2024-04-07T17:05:37.898844+0000 mgr.y (mgr.25175) 1508 : cluster [DBG] pgmap v2448: 169 pgs: 169 active+clean; 2.1 GiB data, 8.1 GiB used, 707 GiB / 715 GiB avail; 9.4 MiB/s rd, 16 MiB/s wr, 329 op/s 2024-04-07T17:05:39.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:05:39 smithi144 bash[66654]: cluster 2024-04-07T17:05:38.117322+0000 mon.a (mon.0) 2618 : cluster [DBG] osdmap e1366: 8 total, 8 up, 8 in 2024-04-07T17:05:40.176 INFO:tasks.workunit.client.0.smithi100.stdout:write: c 108624744~4217944 2024-04-07T17:05:40.198 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 63706689~387905 2024-04-07T17:05:40.199 INFO:tasks.workunit.client.0.smithi100.stdout:write: c 90170240~739952 2024-04-07T17:05:40.200 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 3215877~5519052 2024-04-07T17:05:40.201 INFO:tasks.workunit.client.0.smithi100.stdout:write: c 66777457~5872027 2024-04-07T17:05:40.216 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 153475952~7783192 2024-04-07T17:05:40.220 INFO:tasks.workunit.client.0.smithi100.stdout:write: c 21274858~3146724 2024-04-07T17:05:40.248 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 45121437~310253 2024-04-07T17:05:40.250 INFO:tasks.workunit.client.0.smithi100.stdout:snap_create: user2 2024-04-07T17:05:40.452 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:05:40 smithi100 bash[74942]: cluster 2024-04-07T17:05:39.117682+0000 mon.a (mon.0) 2619 : cluster [DBG] osdmap e1367: 8 total, 8 up, 8 in 2024-04-07T17:05:40.452 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:05:40 smithi100 bash[76202]: cluster 2024-04-07T17:05:39.117682+0000 mon.a (mon.0) 2619 : cluster [DBG] osdmap e1367: 8 total, 8 up, 8 in 2024-04-07T17:05:40.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:05:40 smithi144 bash[66654]: cluster 2024-04-07T17:05:39.117682+0000 mon.a (mon.0) 2619 : cluster [DBG] osdmap e1367: 8 total, 8 up, 8 in 2024-04-07T17:05:41.452 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:05:41 smithi100 bash[74942]: cluster 2024-04-07T17:05:39.899906+0000 mgr.y (mgr.25175) 1509 : cluster [DBG] pgmap v2451: 169 pgs: 169 active+clean; 2.2 GiB data, 8.2 GiB used, 707 GiB / 715 GiB avail; 18 MiB/s rd, 20 MiB/s wr, 572 op/s 2024-04-07T17:05:41.452 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:05:41 smithi100 bash[74942]: cluster 2024-04-07T17:05:40.141230+0000 mon.a (mon.0) 2620 : cluster [DBG] osdmap e1368: 8 total, 8 up, 8 in 2024-04-07T17:05:41.452 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:05:41 smithi100 bash[76202]: cluster 2024-04-07T17:05:39.899906+0000 mgr.y (mgr.25175) 1509 : cluster [DBG] pgmap v2451: 169 pgs: 169 active+clean; 2.2 GiB data, 8.2 GiB used, 707 GiB / 715 GiB avail; 18 MiB/s rd, 20 MiB/s wr, 572 op/s 2024-04-07T17:05:41.452 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:05:41 smithi100 bash[76202]: cluster 2024-04-07T17:05:40.141230+0000 mon.a (mon.0) 2620 : cluster [DBG] osdmap e1368: 8 total, 8 up, 8 in 2024-04-07T17:05:41.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:05:41 smithi144 bash[66654]: cluster 2024-04-07T17:05:39.899906+0000 mgr.y (mgr.25175) 1509 : cluster [DBG] pgmap v2451: 169 pgs: 169 active+clean; 2.2 GiB data, 8.2 GiB used, 707 GiB / 715 GiB avail; 18 MiB/s rd, 20 MiB/s wr, 572 op/s 2024-04-07T17:05:41.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:05:41 smithi144 bash[66654]: cluster 2024-04-07T17:05:40.141230+0000 mon.a (mon.0) 2620 : cluster [DBG] osdmap e1368: 8 total, 8 up, 8 in 2024-04-07T17:05:42.197 INFO:tasks.workunit.client.0.smithi100.stdout:resize: 358662706 2024-04-07T17:05:42.231 INFO:tasks.workunit.client.0.smithi100.stdout:write: d 1289563~8252598 2024-04-07T17:05:42.298 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 38350974~1345114 2024-04-07T17:05:42.299 INFO:tasks.workunit.client.0.smithi100.stdout:write: d 354845433~3817273 2024-04-07T17:05:42.306 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 67556999~1772538 2024-04-07T17:05:42.307 INFO:tasks.workunit.client.0.smithi100.stdout:write: d 132662333~5913109 2024-04-07T17:05:42.327 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 19640068~847156 2024-04-07T17:05:42.327 INFO:tasks.workunit.client.0.smithi100.stdout:write: d 292126354~7816055 2024-04-07T17:05:42.348 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 200943089~5019917 2024-04-07T17:05:42.351 INFO:tasks.workunit.client.0.smithi100.stdout:snap_create: user3 2024-04-07T17:05:42.452 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:05:42 smithi100 bash[74942]: cluster 2024-04-07T17:05:41.142360+0000 mon.a (mon.0) 2621 : cluster [DBG] osdmap e1369: 8 total, 8 up, 8 in 2024-04-07T17:05:42.452 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:05:42 smithi100 bash[76202]: cluster 2024-04-07T17:05:41.142360+0000 mon.a (mon.0) 2621 : cluster [DBG] osdmap e1369: 8 total, 8 up, 8 in 2024-04-07T17:05:42.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:05:42 smithi144 bash[66654]: cluster 2024-04-07T17:05:41.142360+0000 mon.a (mon.0) 2621 : cluster [DBG] osdmap e1369: 8 total, 8 up, 8 in 2024-04-07T17:05:42.616 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:05:42 smithi144 bash[95233]: ts=2024-04-07T17:05:42.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:05:43.452 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:05:43 smithi100 bash[74942]: cluster 2024-04-07T17:05:41.900678+0000 mgr.y (mgr.25175) 1510 : cluster [DBG] pgmap v2454: 169 pgs: 169 active+clean; 2.2 GiB data, 8.3 GiB used, 707 GiB / 715 GiB avail; 17 MiB/s rd, 26 MiB/s wr, 556 op/s 2024-04-07T17:05:43.452 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:05:43 smithi100 bash[74942]: cluster 2024-04-07T17:05:42.167686+0000 mon.a (mon.0) 2622 : cluster [DBG] osdmap e1370: 8 total, 8 up, 8 in 2024-04-07T17:05:43.452 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:05:43 smithi100 bash[76202]: cluster 2024-04-07T17:05:41.900678+0000 mgr.y (mgr.25175) 1510 : cluster [DBG] pgmap v2454: 169 pgs: 169 active+clean; 2.2 GiB data, 8.3 GiB used, 707 GiB / 715 GiB avail; 17 MiB/s rd, 26 MiB/s wr, 556 op/s 2024-04-07T17:05:43.452 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:05:43 smithi100 bash[76202]: cluster 2024-04-07T17:05:42.167686+0000 mon.a (mon.0) 2622 : cluster [DBG] osdmap e1370: 8 total, 8 up, 8 in 2024-04-07T17:05:43.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:05:43 smithi144 bash[66654]: cluster 2024-04-07T17:05:41.900678+0000 mgr.y (mgr.25175) 1510 : cluster [DBG] pgmap v2454: 169 pgs: 169 active+clean; 2.2 GiB data, 8.3 GiB used, 707 GiB / 715 GiB avail; 17 MiB/s rd, 26 MiB/s wr, 556 op/s 2024-04-07T17:05:43.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:05:43 smithi144 bash[66654]: cluster 2024-04-07T17:05:42.167686+0000 mon.a (mon.0) 2622 : cluster [DBG] osdmap e1370: 8 total, 8 up, 8 in 2024-04-07T17:05:44.227 INFO:tasks.workunit.client.0.smithi100.stdout:resize: 282723009 2024-04-07T17:05:44.310 INFO:tasks.workunit.client.0.smithi100.stdout:write: 1 28272300~7068075 2024-04-07T17:05:44.336 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 30039318~3534037 2024-04-07T17:05:44.386 INFO:tasks.workunit.client.0.smithi100.stdout:write: 2 56544601~7068075 2024-04-07T17:05:44.398 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 58311619~3534037 2024-04-07T17:05:44.452 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:05:44 smithi100 bash[74942]: cluster 2024-04-07T17:05:43.166931+0000 mon.a (mon.0) 2623 : cluster [DBG] osdmap e1371: 8 total, 8 up, 8 in 2024-04-07T17:05:44.452 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:05:44 smithi100 bash[76202]: cluster 2024-04-07T17:05:43.166931+0000 mon.a (mon.0) 2623 : cluster [DBG] osdmap e1371: 8 total, 8 up, 8 in 2024-04-07T17:05:44.455 INFO:tasks.workunit.client.0.smithi100.stdout:write: 3 84816902~7068075 2024-04-07T17:05:44.460 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 86583920~3534037 2024-04-07T17:05:44.499 INFO:tasks.workunit.client.0.smithi100.stdout:write: 4 113089203~7068075 2024-04-07T17:05:44.502 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 114856221~3534037 2024-04-07T17:05:44.549 INFO:tasks.workunit.client.0.smithi100.stdout:write: 5 141361504~7068075 2024-04-07T17:05:44.552 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 143128522~3534037 2024-04-07T17:05:44.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:05:44 smithi144 bash[66654]: cluster 2024-04-07T17:05:43.166931+0000 mon.a (mon.0) 2623 : cluster [DBG] osdmap e1371: 8 total, 8 up, 8 in 2024-04-07T17:05:44.823 INFO:tasks.workunit.client.0.smithi100.stdout:migration_commit 2024-04-07T17:05:45.452 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:05:45 smithi100 bash[74942]: cluster 2024-04-07T17:05:43.901487+0000 mgr.y (mgr.25175) 1511 : cluster [DBG] pgmap v2457: 169 pgs: 169 active+clean; 2.2 GiB data, 8.4 GiB used, 707 GiB / 715 GiB avail; 3.7 MiB/s rd, 16 MiB/s wr, 231 op/s 2024-04-07T17:05:45.452 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:05:45 smithi100 bash[74942]: cluster 2024-04-07T17:05:44.198612+0000 mon.a (mon.0) 2624 : cluster [DBG] osdmap e1372: 8 total, 8 up, 8 in 2024-04-07T17:05:45.452 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:05:45 smithi100 bash[76202]: cluster 2024-04-07T17:05:43.901487+0000 mgr.y (mgr.25175) 1511 : cluster [DBG] pgmap v2457: 169 pgs: 169 active+clean; 2.2 GiB data, 8.4 GiB used, 707 GiB / 715 GiB avail; 3.7 MiB/s rd, 16 MiB/s wr, 231 op/s 2024-04-07T17:05:45.452 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:05:45 smithi100 bash[76202]: cluster 2024-04-07T17:05:44.198612+0000 mon.a (mon.0) 2624 : cluster [DBG] osdmap e1372: 8 total, 8 up, 8 in 2024-04-07T17:05:45.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:05:45 smithi144 bash[66654]: cluster 2024-04-07T17:05:43.901487+0000 mgr.y (mgr.25175) 1511 : cluster [DBG] pgmap v2457: 169 pgs: 169 active+clean; 2.2 GiB data, 8.4 GiB used, 707 GiB / 715 GiB avail; 3.7 MiB/s rd, 16 MiB/s wr, 231 op/s 2024-04-07T17:05:45.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:05:45 smithi144 bash[66654]: cluster 2024-04-07T17:05:44.198612+0000 mon.a (mon.0) 2624 : cluster [DBG] osdmap e1372: 8 total, 8 up, 8 in 2024-04-07T17:05:47.452 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:05:47 smithi100 bash[74942]: cluster 2024-04-07T17:05:45.902434+0000 mgr.y (mgr.25175) 1512 : cluster [DBG] pgmap v2459: 169 pgs: 169 active+clean; 2.3 GiB data, 8.5 GiB used, 707 GiB / 715 GiB avail; 3.5 MiB/s rd, 19 MiB/s wr, 262 op/s 2024-04-07T17:05:47.452 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:05:47 smithi100 bash[74942]: audit 2024-04-07T17:05:46.806545+0000 mon.a (mon.0) 2625 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:05:47.452 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:05:47 smithi100 bash[76202]: cluster 2024-04-07T17:05:45.902434+0000 mgr.y (mgr.25175) 1512 : cluster [DBG] pgmap v2459: 169 pgs: 169 active+clean; 2.3 GiB data, 8.5 GiB used, 707 GiB / 715 GiB avail; 3.5 MiB/s rd, 19 MiB/s wr, 262 op/s 2024-04-07T17:05:47.452 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:05:47 smithi100 bash[76202]: audit 2024-04-07T17:05:46.806545+0000 mon.a (mon.0) 2625 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:05:47.507 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestMigration.StressLive (20859 ms) 2024-04-07T17:05:47.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:05:47 smithi144 bash[66654]: cluster 2024-04-07T17:05:45.902434+0000 mgr.y (mgr.25175) 1512 : cluster [DBG] pgmap v2459: 169 pgs: 169 active+clean; 2.3 GiB data, 8.5 GiB used, 707 GiB / 715 GiB avail; 3.5 MiB/s rd, 19 MiB/s wr, 262 op/s 2024-04-07T17:05:47.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:05:47 smithi144 bash[66654]: audit 2024-04-07T17:05:46.806545+0000 mon.a (mon.0) 2625 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:05:48.702 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:05:48 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:05:48] "GET /metrics HTTP/1.1" 200 38923 "" "Prometheus/2.43.0" 2024-04-07T17:05:49.233 INFO:tasks.workunit.client.0.smithi100.stdout:[----------] 57 tests from TestMigration (299262 ms total) 2024-04-07T17:05:49.234 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:05:49.234 INFO:tasks.workunit.client.0.smithi100.stdout:[----------] 6 tests from TestGroup 2024-04-07T17:05:49.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:05:49 smithi144 bash[66654]: cluster 2024-04-07T17:05:47.903841+0000 mgr.y (mgr.25175) 1513 : cluster [DBG] pgmap v2460: 169 pgs: 169 active+clean; 2.4 GiB data, 8.7 GiB used, 707 GiB / 715 GiB avail; 25 MiB/s rd, 22 MiB/s wr, 312 op/s 2024-04-07T17:05:49.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:05:49 smithi144 bash[66654]: cluster 2024-04-07T17:05:48.212563+0000 mon.a (mon.0) 2626 : cluster [DBG] osdmap e1373: 8 total, 8 up, 8 in 2024-04-07T17:05:49.702 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:05:49 smithi100 bash[74942]: cluster 2024-04-07T17:05:47.903841+0000 mgr.y (mgr.25175) 1513 : cluster [DBG] pgmap v2460: 169 pgs: 169 active+clean; 2.4 GiB data, 8.7 GiB used, 707 GiB / 715 GiB avail; 25 MiB/s rd, 22 MiB/s wr, 312 op/s 2024-04-07T17:05:49.702 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:05:49 smithi100 bash[74942]: cluster 2024-04-07T17:05:48.212563+0000 mon.a (mon.0) 2626 : cluster [DBG] osdmap e1373: 8 total, 8 up, 8 in 2024-04-07T17:05:49.703 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:05:49 smithi100 bash[76202]: cluster 2024-04-07T17:05:47.903841+0000 mgr.y (mgr.25175) 1513 : cluster [DBG] pgmap v2460: 169 pgs: 169 active+clean; 2.4 GiB data, 8.7 GiB used, 707 GiB / 715 GiB avail; 25 MiB/s rd, 22 MiB/s wr, 312 op/s 2024-04-07T17:05:49.703 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:05:49 smithi100 bash[76202]: cluster 2024-04-07T17:05:48.212563+0000 mon.a (mon.0) 2626 : cluster [DBG] osdmap e1373: 8 total, 8 up, 8 in 2024-04-07T17:05:50.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:05:50 smithi144 bash[66654]: cluster 2024-04-07T17:05:49.215655+0000 mon.a (mon.0) 2627 : cluster [DBG] osdmap e1374: 8 total, 8 up, 8 in 2024-04-07T17:05:50.702 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:05:50 smithi100 bash[74942]: cluster 2024-04-07T17:05:49.215655+0000 mon.a (mon.0) 2627 : cluster [DBG] osdmap e1374: 8 total, 8 up, 8 in 2024-04-07T17:05:50.702 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:05:50 smithi100 bash[76202]: cluster 2024-04-07T17:05:49.215655+0000 mon.a (mon.0) 2627 : cluster [DBG] osdmap e1374: 8 total, 8 up, 8 in 2024-04-07T17:05:51.251 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestGroup.group_create 2024-04-07T17:05:51.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:05:51 smithi144 bash[66654]: cluster 2024-04-07T17:05:49.904470+0000 mgr.y (mgr.25175) 1514 : cluster [DBG] pgmap v2463: 105 pgs: 105 active+clean; 4.6 MiB data, 8.6 GiB used, 707 GiB / 715 GiB avail 2024-04-07T17:05:51.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:05:51 smithi144 bash[66654]: cluster 2024-04-07T17:05:50.225994+0000 mon.a (mon.0) 2628 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T17:05:51.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:05:51 smithi144 bash[66654]: cluster 2024-04-07T17:05:50.226073+0000 mon.a (mon.0) 2629 : cluster [INF] Cluster is now healthy 2024-04-07T17:05:51.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:05:51 smithi144 bash[66654]: cluster 2024-04-07T17:05:50.245061+0000 mon.a (mon.0) 2630 : cluster [DBG] osdmap e1375: 8 total, 8 up, 8 in 2024-04-07T17:05:51.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:05:51 smithi144 bash[66654]: audit 2024-04-07T17:05:50.247000+0000 mon.a (mon.0) 2631 : audit [INF] from='client.? 172.21.15.100:0/528113017' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:05:51.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:05:51 smithi144 bash[66654]: audit 2024-04-07T17:05:50.958134+0000 mon.a (mon.0) 2632 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:05:51.702 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:05:51 smithi100 bash[74942]: cluster 2024-04-07T17:05:49.904470+0000 mgr.y (mgr.25175) 1514 : cluster [DBG] pgmap v2463: 105 pgs: 105 active+clean; 4.6 MiB data, 8.6 GiB used, 707 GiB / 715 GiB avail 2024-04-07T17:05:51.702 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:05:51 smithi100 bash[74942]: cluster 2024-04-07T17:05:50.225994+0000 mon.a (mon.0) 2628 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T17:05:51.703 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:05:51 smithi100 bash[74942]: cluster 2024-04-07T17:05:50.226073+0000 mon.a (mon.0) 2629 : cluster [INF] Cluster is now healthy 2024-04-07T17:05:51.703 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:05:51 smithi100 bash[74942]: cluster 2024-04-07T17:05:50.245061+0000 mon.a (mon.0) 2630 : cluster [DBG] osdmap e1375: 8 total, 8 up, 8 in 2024-04-07T17:05:51.703 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:05:51 smithi100 bash[74942]: audit 2024-04-07T17:05:50.247000+0000 mon.a (mon.0) 2631 : audit [INF] from='client.? 172.21.15.100:0/528113017' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:05:51.703 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:05:51 smithi100 bash[74942]: audit 2024-04-07T17:05:50.958134+0000 mon.a (mon.0) 2632 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:05:51.703 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:05:51 smithi100 bash[76202]: cluster 2024-04-07T17:05:49.904470+0000 mgr.y (mgr.25175) 1514 : cluster [DBG] pgmap v2463: 105 pgs: 105 active+clean; 4.6 MiB data, 8.6 GiB used, 707 GiB / 715 GiB avail 2024-04-07T17:05:51.703 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:05:51 smithi100 bash[76202]: cluster 2024-04-07T17:05:50.225994+0000 mon.a (mon.0) 2628 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T17:05:51.703 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:05:51 smithi100 bash[76202]: cluster 2024-04-07T17:05:50.226073+0000 mon.a (mon.0) 2629 : cluster [INF] Cluster is now healthy 2024-04-07T17:05:51.703 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:05:51 smithi100 bash[76202]: cluster 2024-04-07T17:05:50.245061+0000 mon.a (mon.0) 2630 : cluster [DBG] osdmap e1375: 8 total, 8 up, 8 in 2024-04-07T17:05:51.703 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:05:51 smithi100 bash[76202]: audit 2024-04-07T17:05:50.247000+0000 mon.a (mon.0) 2631 : audit [INF] from='client.? 172.21.15.100:0/528113017' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:05:51.703 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:05:51 smithi100 bash[76202]: audit 2024-04-07T17:05:50.958134+0000 mon.a (mon.0) 2632 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:05:52.615 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:05:52 smithi144 bash[95233]: ts=2024-04-07T17:05:52.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:05:52.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:05:52 smithi144 bash[66654]: audit 2024-04-07T17:05:51.240684+0000 mon.a (mon.0) 2633 : audit [INF] from='client.? 172.21.15.100:0/528113017' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T17:05:52.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:05:52 smithi144 bash[66654]: cluster 2024-04-07T17:05:51.247404+0000 mon.a (mon.0) 2634 : cluster [DBG] osdmap e1376: 8 total, 8 up, 8 in 2024-04-07T17:05:52.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:05:52 smithi144 bash[66654]: audit 2024-04-07T17:05:51.534559+0000 mon.a (mon.0) 2635 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:05:52.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:05:52 smithi144 bash[66654]: audit 2024-04-07T17:05:51.542026+0000 mon.a (mon.0) 2636 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:05:52.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:05:52 smithi144 bash[66654]: audit 2024-04-07T17:05:52.222782+0000 mon.a (mon.0) 2637 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:05:52.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:05:52 smithi144 bash[66654]: audit 2024-04-07T17:05:52.224660+0000 mon.a (mon.0) 2638 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:05:52.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:05:52 smithi144 bash[66654]: audit 2024-04-07T17:05:52.234863+0000 mon.a (mon.0) 2639 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:05:52.702 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:05:52 smithi100 bash[74942]: audit 2024-04-07T17:05:51.240684+0000 mon.a (mon.0) 2633 : audit [INF] from='client.? 172.21.15.100:0/528113017' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T17:05:52.702 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:05:52 smithi100 bash[74942]: cluster 2024-04-07T17:05:51.247404+0000 mon.a (mon.0) 2634 : cluster [DBG] osdmap e1376: 8 total, 8 up, 8 in 2024-04-07T17:05:52.703 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:05:52 smithi100 bash[74942]: audit 2024-04-07T17:05:51.534559+0000 mon.a (mon.0) 2635 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:05:52.703 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:05:52 smithi100 bash[74942]: audit 2024-04-07T17:05:51.542026+0000 mon.a (mon.0) 2636 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:05:52.703 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:05:52 smithi100 bash[74942]: audit 2024-04-07T17:05:52.222782+0000 mon.a (mon.0) 2637 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:05:52.703 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:05:52 smithi100 bash[74942]: audit 2024-04-07T17:05:52.224660+0000 mon.a (mon.0) 2638 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:05:52.703 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:05:52 smithi100 bash[74942]: audit 2024-04-07T17:05:52.234863+0000 mon.a (mon.0) 2639 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:05:52.703 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:05:52 smithi100 bash[76202]: audit 2024-04-07T17:05:51.240684+0000 mon.a (mon.0) 2633 : audit [INF] from='client.? 172.21.15.100:0/528113017' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T17:05:52.703 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:05:52 smithi100 bash[76202]: cluster 2024-04-07T17:05:51.247404+0000 mon.a (mon.0) 2634 : cluster [DBG] osdmap e1376: 8 total, 8 up, 8 in 2024-04-07T17:05:52.703 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:05:52 smithi100 bash[76202]: audit 2024-04-07T17:05:51.534559+0000 mon.a (mon.0) 2635 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:05:52.703 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:05:52 smithi100 bash[76202]: audit 2024-04-07T17:05:51.542026+0000 mon.a (mon.0) 2636 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:05:52.703 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:05:52 smithi100 bash[76202]: audit 2024-04-07T17:05:52.222782+0000 mon.a (mon.0) 2637 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:05:52.703 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:05:52 smithi100 bash[76202]: audit 2024-04-07T17:05:52.224660+0000 mon.a (mon.0) 2638 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:05:52.703 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:05:52 smithi100 bash[76202]: audit 2024-04-07T17:05:52.234863+0000 mon.a (mon.0) 2639 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:05:53.343 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestGroup.group_create (2092 ms) 2024-04-07T17:05:53.343 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestGroup.group_createPP 2024-04-07T17:05:53.378 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestGroup.group_createPP (35 ms) 2024-04-07T17:05:53.378 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestGroup.add_image 2024-04-07T17:05:53.450 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestGroup.add_image (72 ms) 2024-04-07T17:05:53.450 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestGroup.add_imagePP 2024-04-07T17:05:53.520 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestGroup.add_imagePP (70 ms) 2024-04-07T17:05:53.520 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestGroup.add_snapshot 2024-04-07T17:05:53.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:05:53 smithi144 bash[66654]: cluster 2024-04-07T17:05:51.905194+0000 mgr.y (mgr.25175) 1515 : cluster [DBG] pgmap v2466: 137 pgs: 32 unknown, 105 active+clean; 4.6 MiB data, 8.6 GiB used, 707 GiB / 715 GiB avail 2024-04-07T17:05:53.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:05:53 smithi144 bash[66654]: cluster 2024-04-07T17:05:52.251164+0000 mon.a (mon.0) 2640 : cluster [DBG] osdmap e1377: 8 total, 8 up, 8 in 2024-04-07T17:05:53.702 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:05:53 smithi100 bash[74942]: cluster 2024-04-07T17:05:51.905194+0000 mgr.y (mgr.25175) 1515 : cluster [DBG] pgmap v2466: 137 pgs: 32 unknown, 105 active+clean; 4.6 MiB data, 8.6 GiB used, 707 GiB / 715 GiB avail 2024-04-07T17:05:53.702 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:05:53 smithi100 bash[74942]: cluster 2024-04-07T17:05:52.251164+0000 mon.a (mon.0) 2640 : cluster [DBG] osdmap e1377: 8 total, 8 up, 8 in 2024-04-07T17:05:53.703 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:05:53 smithi100 bash[76202]: cluster 2024-04-07T17:05:51.905194+0000 mgr.y (mgr.25175) 1515 : cluster [DBG] pgmap v2466: 137 pgs: 32 unknown, 105 active+clean; 4.6 MiB data, 8.6 GiB used, 707 GiB / 715 GiB avail 2024-04-07T17:05:53.703 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:05:53 smithi100 bash[76202]: cluster 2024-04-07T17:05:52.251164+0000 mon.a (mon.0) 2640 : cluster [DBG] osdmap e1377: 8 total, 8 up, 8 in 2024-04-07T17:05:54.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:05:54 smithi144 bash[66654]: cluster 2024-04-07T17:05:53.279925+0000 mon.a (mon.0) 2641 : cluster [DBG] osdmap e1378: 8 total, 8 up, 8 in 2024-04-07T17:05:54.702 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:05:54 smithi100 bash[74942]: cluster 2024-04-07T17:05:53.279925+0000 mon.a (mon.0) 2641 : cluster [DBG] osdmap e1378: 8 total, 8 up, 8 in 2024-04-07T17:05:54.702 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:05:54 smithi100 bash[76202]: cluster 2024-04-07T17:05:53.279925+0000 mon.a (mon.0) 2641 : cluster [DBG] osdmap e1378: 8 total, 8 up, 8 in 2024-04-07T17:05:55.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:05:55 smithi144 bash[66654]: cluster 2024-04-07T17:05:53.905899+0000 mgr.y (mgr.25175) 1516 : cluster [DBG] pgmap v2469: 137 pgs: 17 unknown, 120 active+clean; 4.6 MiB data, 5.5 GiB used, 710 GiB / 715 GiB avail 2024-04-07T17:05:55.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:05:55 smithi144 bash[66654]: cluster 2024-04-07T17:05:54.285383+0000 mon.a (mon.0) 2642 : cluster [DBG] osdmap e1379: 8 total, 8 up, 8 in 2024-04-07T17:05:55.702 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:05:55 smithi100 bash[74942]: cluster 2024-04-07T17:05:53.905899+0000 mgr.y (mgr.25175) 1516 : cluster [DBG] pgmap v2469: 137 pgs: 17 unknown, 120 active+clean; 4.6 MiB data, 5.5 GiB used, 710 GiB / 715 GiB avail 2024-04-07T17:05:55.702 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:05:55 smithi100 bash[74942]: cluster 2024-04-07T17:05:54.285383+0000 mon.a (mon.0) 2642 : cluster [DBG] osdmap e1379: 8 total, 8 up, 8 in 2024-04-07T17:05:55.702 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:05:55 smithi100 bash[76202]: cluster 2024-04-07T17:05:53.905899+0000 mgr.y (mgr.25175) 1516 : cluster [DBG] pgmap v2469: 137 pgs: 17 unknown, 120 active+clean; 4.6 MiB data, 5.5 GiB used, 710 GiB / 715 GiB avail 2024-04-07T17:05:55.703 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:05:55 smithi100 bash[76202]: cluster 2024-04-07T17:05:54.285383+0000 mon.a (mon.0) 2642 : cluster [DBG] osdmap e1379: 8 total, 8 up, 8 in 2024-04-07T17:05:56.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:05:56 smithi144 bash[66654]: cluster 2024-04-07T17:05:55.297396+0000 mon.a (mon.0) 2643 : cluster [DBG] osdmap e1380: 8 total, 8 up, 8 in 2024-04-07T17:05:56.702 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:05:56 smithi100 bash[74942]: cluster 2024-04-07T17:05:55.297396+0000 mon.a (mon.0) 2643 : cluster [DBG] osdmap e1380: 8 total, 8 up, 8 in 2024-04-07T17:05:56.702 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:05:56 smithi100 bash[76202]: cluster 2024-04-07T17:05:55.297396+0000 mon.a (mon.0) 2643 : cluster [DBG] osdmap e1380: 8 total, 8 up, 8 in 2024-04-07T17:05:57.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:05:57 smithi144 bash[66654]: cluster 2024-04-07T17:05:55.906714+0000 mgr.y (mgr.25175) 1517 : cluster [DBG] pgmap v2472: 137 pgs: 137 active+clean; 4.6 MiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 77 KiB/s rd, 8.5 KiB/s wr, 105 op/s 2024-04-07T17:05:57.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:05:57 smithi144 bash[66654]: cluster 2024-04-07T17:05:56.306092+0000 mon.a (mon.0) 2644 : cluster [DBG] osdmap e1381: 8 total, 8 up, 8 in 2024-04-07T17:05:57.702 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:05:57 smithi100 bash[74942]: cluster 2024-04-07T17:05:55.906714+0000 mgr.y (mgr.25175) 1517 : cluster [DBG] pgmap v2472: 137 pgs: 137 active+clean; 4.6 MiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 77 KiB/s rd, 8.5 KiB/s wr, 105 op/s 2024-04-07T17:05:57.702 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:05:57 smithi100 bash[74942]: cluster 2024-04-07T17:05:56.306092+0000 mon.a (mon.0) 2644 : cluster [DBG] osdmap e1381: 8 total, 8 up, 8 in 2024-04-07T17:05:57.703 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:05:57 smithi100 bash[76202]: cluster 2024-04-07T17:05:55.906714+0000 mgr.y (mgr.25175) 1517 : cluster [DBG] pgmap v2472: 137 pgs: 137 active+clean; 4.6 MiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 77 KiB/s rd, 8.5 KiB/s wr, 105 op/s 2024-04-07T17:05:57.703 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:05:57 smithi100 bash[76202]: cluster 2024-04-07T17:05:56.306092+0000 mon.a (mon.0) 2644 : cluster [DBG] osdmap e1381: 8 total, 8 up, 8 in 2024-04-07T17:05:58.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:05:58 smithi144 bash[66654]: cluster 2024-04-07T17:05:57.315824+0000 mon.a (mon.0) 2645 : cluster [DBG] osdmap e1382: 8 total, 8 up, 8 in 2024-04-07T17:05:58.702 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:05:58 smithi100 bash[74942]: cluster 2024-04-07T17:05:57.315824+0000 mon.a (mon.0) 2645 : cluster [DBG] osdmap e1382: 8 total, 8 up, 8 in 2024-04-07T17:05:58.703 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:05:58 smithi100 bash[76202]: cluster 2024-04-07T17:05:57.315824+0000 mon.a (mon.0) 2645 : cluster [DBG] osdmap e1382: 8 total, 8 up, 8 in 2024-04-07T17:05:58.703 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:05:58 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:05:58] "GET /metrics HTTP/1.1" 200 38923 "" "Prometheus/2.43.0" 2024-04-07T17:05:59.416 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestGroup.add_snapshot (5896 ms) 2024-04-07T17:05:59.416 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestGroup.add_snapshotPP 2024-04-07T17:05:59.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:05:59 smithi144 bash[66654]: cluster 2024-04-07T17:05:57.907594+0000 mgr.y (mgr.25175) 1518 : cluster [DBG] pgmap v2475: 137 pgs: 137 active+clean; 4.6 MiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 93 KiB/s rd, 11 KiB/s wr, 128 op/s 2024-04-07T17:05:59.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:05:59 smithi144 bash[66654]: cluster 2024-04-07T17:05:58.335080+0000 mon.a (mon.0) 2646 : cluster [DBG] osdmap e1383: 8 total, 8 up, 8 in 2024-04-07T17:05:59.702 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:05:59 smithi100 bash[74942]: cluster 2024-04-07T17:05:57.907594+0000 mgr.y (mgr.25175) 1518 : cluster [DBG] pgmap v2475: 137 pgs: 137 active+clean; 4.6 MiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 93 KiB/s rd, 11 KiB/s wr, 128 op/s 2024-04-07T17:05:59.702 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:05:59 smithi100 bash[74942]: cluster 2024-04-07T17:05:58.335080+0000 mon.a (mon.0) 2646 : cluster [DBG] osdmap e1383: 8 total, 8 up, 8 in 2024-04-07T17:05:59.702 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:05:59 smithi100 bash[76202]: cluster 2024-04-07T17:05:57.907594+0000 mgr.y (mgr.25175) 1518 : cluster [DBG] pgmap v2475: 137 pgs: 137 active+clean; 4.6 MiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 93 KiB/s rd, 11 KiB/s wr, 128 op/s 2024-04-07T17:05:59.703 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:05:59 smithi100 bash[76202]: cluster 2024-04-07T17:05:58.335080+0000 mon.a (mon.0) 2646 : cluster [DBG] osdmap e1383: 8 total, 8 up, 8 in 2024-04-07T17:06:00.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:06:00 smithi144 bash[66654]: cluster 2024-04-07T17:05:59.332268+0000 mon.a (mon.0) 2647 : cluster [DBG] osdmap e1384: 8 total, 8 up, 8 in 2024-04-07T17:06:00.702 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:06:00 smithi100 bash[74942]: cluster 2024-04-07T17:05:59.332268+0000 mon.a (mon.0) 2647 : cluster [DBG] osdmap e1384: 8 total, 8 up, 8 in 2024-04-07T17:06:00.702 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:06:00 smithi100 bash[76202]: cluster 2024-04-07T17:05:59.332268+0000 mon.a (mon.0) 2647 : cluster [DBG] osdmap e1384: 8 total, 8 up, 8 in 2024-04-07T17:06:01.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:06:01 smithi144 bash[66654]: cluster 2024-04-07T17:05:59.908471+0000 mgr.y (mgr.25175) 1519 : cluster [DBG] pgmap v2478: 137 pgs: 137 active+clean; 4.6 MiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 54 KiB/s rd, 8.2 KiB/s wr, 76 op/s 2024-04-07T17:06:01.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:06:01 smithi144 bash[66654]: cluster 2024-04-07T17:06:00.353404+0000 mon.a (mon.0) 2648 : cluster [DBG] osdmap e1385: 8 total, 8 up, 8 in 2024-04-07T17:06:01.702 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:06:01 smithi100 bash[74942]: cluster 2024-04-07T17:05:59.908471+0000 mgr.y (mgr.25175) 1519 : cluster [DBG] pgmap v2478: 137 pgs: 137 active+clean; 4.6 MiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 54 KiB/s rd, 8.2 KiB/s wr, 76 op/s 2024-04-07T17:06:01.702 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:06:01 smithi100 bash[74942]: cluster 2024-04-07T17:06:00.353404+0000 mon.a (mon.0) 2648 : cluster [DBG] osdmap e1385: 8 total, 8 up, 8 in 2024-04-07T17:06:01.703 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:06:01 smithi100 bash[76202]: cluster 2024-04-07T17:05:59.908471+0000 mgr.y (mgr.25175) 1519 : cluster [DBG] pgmap v2478: 137 pgs: 137 active+clean; 4.6 MiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 54 KiB/s rd, 8.2 KiB/s wr, 76 op/s 2024-04-07T17:06:01.703 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:06:01 smithi100 bash[76202]: cluster 2024-04-07T17:06:00.353404+0000 mon.a (mon.0) 2648 : cluster [DBG] osdmap e1385: 8 total, 8 up, 8 in 2024-04-07T17:06:02.615 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:06:02 smithi144 bash[95233]: ts=2024-04-07T17:06:02.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:06:02.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:06:02 smithi144 bash[66654]: cluster 2024-04-07T17:06:01.364406+0000 mon.a (mon.0) 2649 : cluster [DBG] osdmap e1386: 8 total, 8 up, 8 in 2024-04-07T17:06:02.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:06:02 smithi144 bash[66654]: audit 2024-04-07T17:06:01.812489+0000 mon.a (mon.0) 2650 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:06:02.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:06:02 smithi144 bash[66654]: audit 2024-04-07T17:06:01.814776+0000 mon.a (mon.0) 2651 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:06:02.702 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:06:02 smithi100 bash[74942]: cluster 2024-04-07T17:06:01.364406+0000 mon.a (mon.0) 2649 : cluster [DBG] osdmap e1386: 8 total, 8 up, 8 in 2024-04-07T17:06:02.702 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:06:02 smithi100 bash[74942]: audit 2024-04-07T17:06:01.812489+0000 mon.a (mon.0) 2650 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:06:02.702 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:06:02 smithi100 bash[74942]: audit 2024-04-07T17:06:01.814776+0000 mon.a (mon.0) 2651 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:06:02.702 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:06:02 smithi100 bash[76202]: cluster 2024-04-07T17:06:01.364406+0000 mon.a (mon.0) 2649 : cluster [DBG] osdmap e1386: 8 total, 8 up, 8 in 2024-04-07T17:06:02.703 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:06:02 smithi100 bash[76202]: audit 2024-04-07T17:06:01.812489+0000 mon.a (mon.0) 2650 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:06:02.703 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:06:02 smithi100 bash[76202]: audit 2024-04-07T17:06:01.814776+0000 mon.a (mon.0) 2651 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:06:03.702 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:06:03 smithi100 bash[74942]: cluster 2024-04-07T17:06:01.909321+0000 mgr.y (mgr.25175) 1520 : cluster [DBG] pgmap v2481: 137 pgs: 137 active+clean; 4.6 MiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 87 KiB/s rd, 10 KiB/s wr, 116 op/s 2024-04-07T17:06:03.702 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:06:03 smithi100 bash[74942]: cluster 2024-04-07T17:06:02.374606+0000 mon.a (mon.0) 2652 : cluster [DBG] osdmap e1387: 8 total, 8 up, 8 in 2024-04-07T17:06:03.703 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:06:03 smithi100 bash[74942]: cluster 2024-04-07T17:06:02.625109+0000 mon.a (mon.0) 2653 : cluster [DBG] osdmap e1388: 8 total, 8 up, 8 in 2024-04-07T17:06:03.703 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:06:03 smithi100 bash[76202]: cluster 2024-04-07T17:06:01.909321+0000 mgr.y (mgr.25175) 1520 : cluster [DBG] pgmap v2481: 137 pgs: 137 active+clean; 4.6 MiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 87 KiB/s rd, 10 KiB/s wr, 116 op/s 2024-04-07T17:06:03.703 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:06:03 smithi100 bash[76202]: cluster 2024-04-07T17:06:02.374606+0000 mon.a (mon.0) 2652 : cluster [DBG] osdmap e1387: 8 total, 8 up, 8 in 2024-04-07T17:06:03.703 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:06:03 smithi100 bash[76202]: cluster 2024-04-07T17:06:02.625109+0000 mon.a (mon.0) 2653 : cluster [DBG] osdmap e1388: 8 total, 8 up, 8 in 2024-04-07T17:06:03.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:06:03 smithi144 bash[66654]: cluster 2024-04-07T17:06:01.909321+0000 mgr.y (mgr.25175) 1520 : cluster [DBG] pgmap v2481: 137 pgs: 137 active+clean; 4.6 MiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 87 KiB/s rd, 10 KiB/s wr, 116 op/s 2024-04-07T17:06:03.866 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:06:03 smithi144 bash[66654]: cluster 2024-04-07T17:06:02.374606+0000 mon.a (mon.0) 2652 : cluster [DBG] osdmap e1387: 8 total, 8 up, 8 in 2024-04-07T17:06:03.866 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:06:03 smithi144 bash[66654]: cluster 2024-04-07T17:06:02.625109+0000 mon.a (mon.0) 2653 : cluster [DBG] osdmap e1388: 8 total, 8 up, 8 in 2024-04-07T17:06:04.712 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestGroup.add_snapshotPP (5296 ms) 2024-04-07T17:06:04.952 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:06:04 smithi100 bash[74942]: cluster 2024-04-07T17:06:03.627092+0000 mon.a (mon.0) 2654 : cluster [DBG] osdmap e1389: 8 total, 8 up, 8 in 2024-04-07T17:06:04.952 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:06:04 smithi100 bash[76202]: cluster 2024-04-07T17:06:03.627092+0000 mon.a (mon.0) 2654 : cluster [DBG] osdmap e1389: 8 total, 8 up, 8 in 2024-04-07T17:06:05.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:06:04 smithi144 bash[66654]: cluster 2024-04-07T17:06:03.627092+0000 mon.a (mon.0) 2654 : cluster [DBG] osdmap e1389: 8 total, 8 up, 8 in 2024-04-07T17:06:05.651 INFO:tasks.workunit.client.0.smithi100.stdout:[----------] 6 tests from TestGroup (13461 ms total) 2024-04-07T17:06:05.651 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:06:05.651 INFO:tasks.workunit.client.0.smithi100.stdout:[----------] 31 tests from TestDeepCopy 2024-04-07T17:06:05.952 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:06:05 smithi100 bash[74942]: cluster 2024-04-07T17:06:03.910092+0000 mgr.y (mgr.25175) 1521 : cluster [DBG] pgmap v2485: 137 pgs: 137 active+clean; 4.6 MiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 88 KiB/s rd, 7.6 KiB/s wr, 117 op/s 2024-04-07T17:06:05.952 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:06:05 smithi100 bash[74942]: cluster 2024-04-07T17:06:04.637972+0000 mon.a (mon.0) 2655 : cluster [DBG] osdmap e1390: 8 total, 8 up, 8 in 2024-04-07T17:06:05.953 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:06:05 smithi100 bash[76202]: cluster 2024-04-07T17:06:03.910092+0000 mgr.y (mgr.25175) 1521 : cluster [DBG] pgmap v2485: 137 pgs: 137 active+clean; 4.6 MiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 88 KiB/s rd, 7.6 KiB/s wr, 117 op/s 2024-04-07T17:06:05.953 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:06:05 smithi100 bash[76202]: cluster 2024-04-07T17:06:04.637972+0000 mon.a (mon.0) 2655 : cluster [DBG] osdmap e1390: 8 total, 8 up, 8 in 2024-04-07T17:06:06.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:06:05 smithi144 bash[66654]: cluster 2024-04-07T17:06:03.910092+0000 mgr.y (mgr.25175) 1521 : cluster [DBG] pgmap v2485: 137 pgs: 137 active+clean; 4.6 MiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 88 KiB/s rd, 7.6 KiB/s wr, 117 op/s 2024-04-07T17:06:06.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:06:05 smithi144 bash[66654]: cluster 2024-04-07T17:06:04.637972+0000 mon.a (mon.0) 2655 : cluster [DBG] osdmap e1390: 8 total, 8 up, 8 in 2024-04-07T17:06:06.952 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:06:06 smithi100 bash[74942]: cluster 2024-04-07T17:06:05.643950+0000 mon.a (mon.0) 2656 : cluster [DBG] osdmap e1391: 8 total, 8 up, 8 in 2024-04-07T17:06:06.952 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:06:06 smithi100 bash[76202]: cluster 2024-04-07T17:06:05.643950+0000 mon.a (mon.0) 2656 : cluster [DBG] osdmap e1391: 8 total, 8 up, 8 in 2024-04-07T17:06:07.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:06:06 smithi144 bash[66654]: cluster 2024-04-07T17:06:05.643950+0000 mon.a (mon.0) 2656 : cluster [DBG] osdmap e1391: 8 total, 8 up, 8 in 2024-04-07T17:06:07.694 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestDeepCopy.Empty 2024-04-07T17:06:07.952 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:06:07 smithi100 bash[74942]: cluster 2024-04-07T17:06:05.910745+0000 mgr.y (mgr.25175) 1522 : cluster [DBG] pgmap v2488: 105 pgs: 105 active+clean; 4.6 MiB data, 3.2 GiB used, 712 GiB / 715 GiB avail 2024-04-07T17:06:07.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:06:07 smithi100 bash[74942]: cluster 2024-04-07T17:06:06.683647+0000 mon.a (mon.0) 2657 : cluster [DBG] osdmap e1392: 8 total, 8 up, 8 in 2024-04-07T17:06:07.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:06:07 smithi100 bash[74942]: audit 2024-04-07T17:06:06.686898+0000 mon.a (mon.0) 2658 : audit [INF] from='client.? 172.21.15.100:0/3276138578' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:06:07.953 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:06:07 smithi100 bash[76202]: cluster 2024-04-07T17:06:05.910745+0000 mgr.y (mgr.25175) 1522 : cluster [DBG] pgmap v2488: 105 pgs: 105 active+clean; 4.6 MiB data, 3.2 GiB used, 712 GiB / 715 GiB avail 2024-04-07T17:06:07.953 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:06:07 smithi100 bash[76202]: cluster 2024-04-07T17:06:06.683647+0000 mon.a (mon.0) 2657 : cluster [DBG] osdmap e1392: 8 total, 8 up, 8 in 2024-04-07T17:06:07.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:06:07 smithi100 bash[76202]: audit 2024-04-07T17:06:06.686898+0000 mon.a (mon.0) 2658 : audit [INF] from='client.? 172.21.15.100:0/3276138578' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:06:08.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:06:07 smithi144 bash[66654]: cluster 2024-04-07T17:06:05.910745+0000 mgr.y (mgr.25175) 1522 : cluster [DBG] pgmap v2488: 105 pgs: 105 active+clean; 4.6 MiB data, 3.2 GiB used, 712 GiB / 715 GiB avail 2024-04-07T17:06:08.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:06:07 smithi144 bash[66654]: cluster 2024-04-07T17:06:06.683647+0000 mon.a (mon.0) 2657 : cluster [DBG] osdmap e1392: 8 total, 8 up, 8 in 2024-04-07T17:06:08.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:06:07 smithi144 bash[66654]: audit 2024-04-07T17:06:06.686898+0000 mon.a (mon.0) 2658 : audit [INF] from='client.? 172.21.15.100:0/3276138578' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:06:08.696 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:06:08 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:06:08] "GET /metrics HTTP/1.1" 200 36469 "" "Prometheus/2.43.0" 2024-04-07T17:06:08.952 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:06:08 smithi100 bash[74942]: audit 2024-04-07T17:06:07.684937+0000 mon.a (mon.0) 2659 : audit [INF] from='client.? 172.21.15.100:0/3276138578' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T17:06:08.952 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:06:08 smithi100 bash[74942]: cluster 2024-04-07T17:06:07.692948+0000 mon.a (mon.0) 2660 : cluster [DBG] osdmap e1393: 8 total, 8 up, 8 in 2024-04-07T17:06:08.953 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:06:08 smithi100 bash[76202]: audit 2024-04-07T17:06:07.684937+0000 mon.a (mon.0) 2659 : audit [INF] from='client.? 172.21.15.100:0/3276138578' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T17:06:08.953 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:06:08 smithi100 bash[76202]: cluster 2024-04-07T17:06:07.692948+0000 mon.a (mon.0) 2660 : cluster [DBG] osdmap e1393: 8 total, 8 up, 8 in 2024-04-07T17:06:09.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:06:08 smithi144 bash[66654]: audit 2024-04-07T17:06:07.684937+0000 mon.a (mon.0) 2659 : audit [INF] from='client.? 172.21.15.100:0/3276138578' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T17:06:09.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:06:08 smithi144 bash[66654]: cluster 2024-04-07T17:06:07.692948+0000 mon.a (mon.0) 2660 : cluster [DBG] osdmap e1393: 8 total, 8 up, 8 in 2024-04-07T17:06:09.971 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestDeepCopy.Empty (2276 ms) 2024-04-07T17:06:09.971 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestDeepCopy.NoSnaps 2024-04-07T17:06:10.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:06:09 smithi144 bash[66654]: cluster 2024-04-07T17:06:07.911492+0000 mgr.y (mgr.25175) 1523 : cluster [DBG] pgmap v2491: 137 pgs: 18 creating+peering, 14 unknown, 105 active+clean; 4.6 MiB data, 3.2 GiB used, 712 GiB / 715 GiB avail 2024-04-07T17:06:10.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:06:09 smithi144 bash[66654]: cluster 2024-04-07T17:06:08.704213+0000 mon.a (mon.0) 2661 : cluster [DBG] osdmap e1394: 8 total, 8 up, 8 in 2024-04-07T17:06:10.202 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:06:09 smithi100 bash[74942]: cluster 2024-04-07T17:06:07.911492+0000 mgr.y (mgr.25175) 1523 : cluster [DBG] pgmap v2491: 137 pgs: 18 creating+peering, 14 unknown, 105 active+clean; 4.6 MiB data, 3.2 GiB used, 712 GiB / 715 GiB avail 2024-04-07T17:06:10.202 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:06:09 smithi100 bash[74942]: cluster 2024-04-07T17:06:08.704213+0000 mon.a (mon.0) 2661 : cluster [DBG] osdmap e1394: 8 total, 8 up, 8 in 2024-04-07T17:06:10.202 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:06:09 smithi100 bash[76202]: cluster 2024-04-07T17:06:07.911492+0000 mgr.y (mgr.25175) 1523 : cluster [DBG] pgmap v2491: 137 pgs: 18 creating+peering, 14 unknown, 105 active+clean; 4.6 MiB data, 3.2 GiB used, 712 GiB / 715 GiB avail 2024-04-07T17:06:10.202 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:06:09 smithi100 bash[76202]: cluster 2024-04-07T17:06:08.704213+0000 mon.a (mon.0) 2661 : cluster [DBG] osdmap e1394: 8 total, 8 up, 8 in 2024-04-07T17:06:10.458 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestDeepCopy.NoSnaps (488 ms) 2024-04-07T17:06:10.458 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestDeepCopy.Snaps 2024-04-07T17:06:11.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:06:10 smithi144 bash[66654]: cluster 2024-04-07T17:06:09.703323+0000 mon.a (mon.0) 2662 : cluster [DBG] osdmap e1395: 8 total, 8 up, 8 in 2024-04-07T17:06:11.202 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:06:10 smithi100 bash[74942]: cluster 2024-04-07T17:06:09.703323+0000 mon.a (mon.0) 2662 : cluster [DBG] osdmap e1395: 8 total, 8 up, 8 in 2024-04-07T17:06:11.202 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:06:10 smithi100 bash[76202]: cluster 2024-04-07T17:06:09.703323+0000 mon.a (mon.0) 2662 : cluster [DBG] osdmap e1395: 8 total, 8 up, 8 in 2024-04-07T17:06:12.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:06:11 smithi144 bash[66654]: cluster 2024-04-07T17:06:09.912271+0000 mgr.y (mgr.25175) 1524 : cluster [DBG] pgmap v2494: 137 pgs: 18 creating+peering, 3 unknown, 116 active+clean; 4.6 MiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 255 B/s wr, 0 op/s 2024-04-07T17:06:12.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:06:11 smithi144 bash[66654]: cluster 2024-04-07T17:06:10.733600+0000 mon.a (mon.0) 2663 : cluster [DBG] osdmap e1396: 8 total, 8 up, 8 in 2024-04-07T17:06:12.202 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:06:11 smithi100 bash[74942]: cluster 2024-04-07T17:06:09.912271+0000 mgr.y (mgr.25175) 1524 : cluster [DBG] pgmap v2494: 137 pgs: 18 creating+peering, 3 unknown, 116 active+clean; 4.6 MiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 255 B/s wr, 0 op/s 2024-04-07T17:06:12.202 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:06:11 smithi100 bash[74942]: cluster 2024-04-07T17:06:10.733600+0000 mon.a (mon.0) 2663 : cluster [DBG] osdmap e1396: 8 total, 8 up, 8 in 2024-04-07T17:06:12.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:06:11 smithi100 bash[76202]: cluster 2024-04-07T17:06:09.912271+0000 mgr.y (mgr.25175) 1524 : cluster [DBG] pgmap v2494: 137 pgs: 18 creating+peering, 3 unknown, 116 active+clean; 4.6 MiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 255 B/s wr, 0 op/s 2024-04-07T17:06:12.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:06:11 smithi100 bash[76202]: cluster 2024-04-07T17:06:10.733600+0000 mon.a (mon.0) 2663 : cluster [DBG] osdmap e1396: 8 total, 8 up, 8 in 2024-04-07T17:06:12.615 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:06:12 smithi144 bash[95233]: ts=2024-04-07T17:06:12.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:06:13.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:06:12 smithi144 bash[66654]: cluster 2024-04-07T17:06:11.734088+0000 mon.a (mon.0) 2664 : cluster [DBG] osdmap e1397: 8 total, 8 up, 8 in 2024-04-07T17:06:13.202 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:06:12 smithi100 bash[74942]: cluster 2024-04-07T17:06:11.734088+0000 mon.a (mon.0) 2664 : cluster [DBG] osdmap e1397: 8 total, 8 up, 8 in 2024-04-07T17:06:13.202 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:06:12 smithi100 bash[76202]: cluster 2024-04-07T17:06:11.734088+0000 mon.a (mon.0) 2664 : cluster [DBG] osdmap e1397: 8 total, 8 up, 8 in 2024-04-07T17:06:14.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:06:13 smithi144 bash[66654]: cluster 2024-04-07T17:06:11.913054+0000 mgr.y (mgr.25175) 1525 : cluster [DBG] pgmap v2497: 137 pgs: 18 creating+peering, 119 active+clean; 4.6 MiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 767 B/s rd, 767 B/s wr, 3 op/s 2024-04-07T17:06:14.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:06:13 smithi144 bash[66654]: cluster 2024-04-07T17:06:12.763727+0000 mon.a (mon.0) 2665 : cluster [DBG] osdmap e1398: 8 total, 8 up, 8 in 2024-04-07T17:06:14.202 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:06:13 smithi100 bash[74942]: cluster 2024-04-07T17:06:11.913054+0000 mgr.y (mgr.25175) 1525 : cluster [DBG] pgmap v2497: 137 pgs: 18 creating+peering, 119 active+clean; 4.6 MiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 767 B/s rd, 767 B/s wr, 3 op/s 2024-04-07T17:06:14.202 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:06:13 smithi100 bash[74942]: cluster 2024-04-07T17:06:12.763727+0000 mon.a (mon.0) 2665 : cluster [DBG] osdmap e1398: 8 total, 8 up, 8 in 2024-04-07T17:06:14.202 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:06:13 smithi100 bash[76202]: cluster 2024-04-07T17:06:11.913054+0000 mgr.y (mgr.25175) 1525 : cluster [DBG] pgmap v2497: 137 pgs: 18 creating+peering, 119 active+clean; 4.6 MiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 767 B/s rd, 767 B/s wr, 3 op/s 2024-04-07T17:06:14.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:06:13 smithi100 bash[76202]: cluster 2024-04-07T17:06:12.763727+0000 mon.a (mon.0) 2665 : cluster [DBG] osdmap e1398: 8 total, 8 up, 8 in 2024-04-07T17:06:14.285 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestDeepCopy.Snaps (3827 ms) 2024-04-07T17:06:14.285 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestDeepCopy.SnapDiscard 2024-04-07T17:06:15.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:06:14 smithi144 bash[66654]: cluster 2024-04-07T17:06:13.760502+0000 mon.a (mon.0) 2666 : cluster [DBG] osdmap e1399: 8 total, 8 up, 8 in 2024-04-07T17:06:15.202 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:06:14 smithi100 bash[74942]: cluster 2024-04-07T17:06:13.760502+0000 mon.a (mon.0) 2666 : cluster [DBG] osdmap e1399: 8 total, 8 up, 8 in 2024-04-07T17:06:15.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:06:14 smithi100 bash[76202]: cluster 2024-04-07T17:06:13.760502+0000 mon.a (mon.0) 2666 : cluster [DBG] osdmap e1399: 8 total, 8 up, 8 in 2024-04-07T17:06:16.018 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestDeepCopy.SnapDiscard (1733 ms) 2024-04-07T17:06:16.018 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestDeepCopy.CloneDiscard 2024-04-07T17:06:16.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:06:15 smithi144 bash[66654]: cluster 2024-04-07T17:06:13.913818+0000 mgr.y (mgr.25175) 1526 : cluster [DBG] pgmap v2500: 137 pgs: 137 active+clean; 17 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 5.1 MiB/s rd, 3.0 MiB/s wr, 109 op/s 2024-04-07T17:06:16.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:06:15 smithi144 bash[66654]: cluster 2024-04-07T17:06:14.782559+0000 mon.a (mon.0) 2667 : cluster [DBG] osdmap e1400: 8 total, 8 up, 8 in 2024-04-07T17:06:16.202 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:06:15 smithi100 bash[74942]: cluster 2024-04-07T17:06:13.913818+0000 mgr.y (mgr.25175) 1526 : cluster [DBG] pgmap v2500: 137 pgs: 137 active+clean; 17 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 5.1 MiB/s rd, 3.0 MiB/s wr, 109 op/s 2024-04-07T17:06:16.202 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:06:15 smithi100 bash[74942]: cluster 2024-04-07T17:06:14.782559+0000 mon.a (mon.0) 2667 : cluster [DBG] osdmap e1400: 8 total, 8 up, 8 in 2024-04-07T17:06:16.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:06:15 smithi100 bash[76202]: cluster 2024-04-07T17:06:13.913818+0000 mgr.y (mgr.25175) 1526 : cluster [DBG] pgmap v2500: 137 pgs: 137 active+clean; 17 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 5.1 MiB/s rd, 3.0 MiB/s wr, 109 op/s 2024-04-07T17:06:16.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:06:15 smithi100 bash[76202]: cluster 2024-04-07T17:06:14.782559+0000 mon.a (mon.0) 2667 : cluster [DBG] osdmap e1400: 8 total, 8 up, 8 in 2024-04-07T17:06:17.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:06:16 smithi144 bash[66654]: cluster 2024-04-07T17:06:15.792767+0000 mon.a (mon.0) 2668 : cluster [DBG] osdmap e1401: 8 total, 8 up, 8 in 2024-04-07T17:06:17.202 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:06:16 smithi100 bash[74942]: cluster 2024-04-07T17:06:15.792767+0000 mon.a (mon.0) 2668 : cluster [DBG] osdmap e1401: 8 total, 8 up, 8 in 2024-04-07T17:06:17.202 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:06:16 smithi100 bash[76202]: cluster 2024-04-07T17:06:15.792767+0000 mon.a (mon.0) 2668 : cluster [DBG] osdmap e1401: 8 total, 8 up, 8 in 2024-04-07T17:06:17.307 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestDeepCopy.CloneDiscard (1288 ms) 2024-04-07T17:06:17.307 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestDeepCopy.CloneShrink 2024-04-07T17:06:18.059 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestDeepCopy.CloneShrink (753 ms) 2024-04-07T17:06:18.060 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestDeepCopy.CloneExpand 2024-04-07T17:06:18.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:06:17 smithi144 bash[66654]: cluster 2024-04-07T17:06:15.914523+0000 mgr.y (mgr.25175) 1527 : cluster [DBG] pgmap v2503: 137 pgs: 137 active+clean; 53 MiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 22 MiB/s rd, 12 MiB/s wr, 202 op/s 2024-04-07T17:06:18.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:06:17 smithi144 bash[66654]: cluster 2024-04-07T17:06:16.803715+0000 mon.a (mon.0) 2669 : cluster [DBG] osdmap e1402: 8 total, 8 up, 8 in 2024-04-07T17:06:18.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:06:17 smithi144 bash[66654]: audit 2024-04-07T17:06:16.807960+0000 mon.a (mon.0) 2670 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:06:18.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:06:17 smithi144 bash[66654]: cluster 2024-04-07T17:06:17.630328+0000 mon.a (mon.0) 2671 : cluster [DBG] osdmap e1403: 8 total, 8 up, 8 in 2024-04-07T17:06:18.202 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:06:17 smithi100 bash[74942]: cluster 2024-04-07T17:06:15.914523+0000 mgr.y (mgr.25175) 1527 : cluster [DBG] pgmap v2503: 137 pgs: 137 active+clean; 53 MiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 22 MiB/s rd, 12 MiB/s wr, 202 op/s 2024-04-07T17:06:18.202 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:06:17 smithi100 bash[74942]: cluster 2024-04-07T17:06:16.803715+0000 mon.a (mon.0) 2669 : cluster [DBG] osdmap e1402: 8 total, 8 up, 8 in 2024-04-07T17:06:18.202 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:06:17 smithi100 bash[74942]: audit 2024-04-07T17:06:16.807960+0000 mon.a (mon.0) 2670 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:06:18.202 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:06:17 smithi100 bash[74942]: cluster 2024-04-07T17:06:17.630328+0000 mon.a (mon.0) 2671 : cluster [DBG] osdmap e1403: 8 total, 8 up, 8 in 2024-04-07T17:06:18.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:06:17 smithi100 bash[76202]: cluster 2024-04-07T17:06:15.914523+0000 mgr.y (mgr.25175) 1527 : cluster [DBG] pgmap v2503: 137 pgs: 137 active+clean; 53 MiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 22 MiB/s rd, 12 MiB/s wr, 202 op/s 2024-04-07T17:06:18.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:06:17 smithi100 bash[76202]: cluster 2024-04-07T17:06:16.803715+0000 mon.a (mon.0) 2669 : cluster [DBG] osdmap e1402: 8 total, 8 up, 8 in 2024-04-07T17:06:18.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:06:17 smithi100 bash[76202]: audit 2024-04-07T17:06:16.807960+0000 mon.a (mon.0) 2670 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:06:18.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:06:17 smithi100 bash[76202]: cluster 2024-04-07T17:06:17.630328+0000 mon.a (mon.0) 2671 : cluster [DBG] osdmap e1403: 8 total, 8 up, 8 in 2024-04-07T17:06:18.702 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:06:18 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:06:18] "GET /metrics HTTP/1.1" 200 36484 "" "Prometheus/2.43.0" 2024-04-07T17:06:19.378 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestDeepCopy.CloneExpand (1319 ms) 2024-04-07T17:06:19.378 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestDeepCopy.CloneHideParent 2024-04-07T17:06:20.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:06:19 smithi144 bash[66654]: cluster 2024-04-07T17:06:17.915454+0000 mgr.y (mgr.25175) 1528 : cluster [DBG] pgmap v2506: 137 pgs: 137 active+clean; 65 MiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 25 MiB/s rd, 12 MiB/s wr, 187 op/s 2024-04-07T17:06:20.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:06:19 smithi144 bash[66654]: cluster 2024-04-07T17:06:18.812886+0000 mon.a (mon.0) 2672 : cluster [DBG] osdmap e1404: 8 total, 8 up, 8 in 2024-04-07T17:06:20.202 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:06:19 smithi100 bash[74942]: cluster 2024-04-07T17:06:17.915454+0000 mgr.y (mgr.25175) 1528 : cluster [DBG] pgmap v2506: 137 pgs: 137 active+clean; 65 MiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 25 MiB/s rd, 12 MiB/s wr, 187 op/s 2024-04-07T17:06:20.202 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:06:19 smithi100 bash[74942]: cluster 2024-04-07T17:06:18.812886+0000 mon.a (mon.0) 2672 : cluster [DBG] osdmap e1404: 8 total, 8 up, 8 in 2024-04-07T17:06:20.202 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:06:19 smithi100 bash[76202]: cluster 2024-04-07T17:06:17.915454+0000 mgr.y (mgr.25175) 1528 : cluster [DBG] pgmap v2506: 137 pgs: 137 active+clean; 65 MiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 25 MiB/s rd, 12 MiB/s wr, 187 op/s 2024-04-07T17:06:20.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:06:19 smithi100 bash[76202]: cluster 2024-04-07T17:06:18.812886+0000 mon.a (mon.0) 2672 : cluster [DBG] osdmap e1404: 8 total, 8 up, 8 in 2024-04-07T17:06:21.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:06:20 smithi144 bash[66654]: cluster 2024-04-07T17:06:19.815367+0000 mon.a (mon.0) 2673 : cluster [DBG] osdmap e1405: 8 total, 8 up, 8 in 2024-04-07T17:06:21.202 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:06:20 smithi100 bash[74942]: cluster 2024-04-07T17:06:19.815367+0000 mon.a (mon.0) 2673 : cluster [DBG] osdmap e1405: 8 total, 8 up, 8 in 2024-04-07T17:06:21.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:06:20 smithi100 bash[76202]: cluster 2024-04-07T17:06:19.815367+0000 mon.a (mon.0) 2673 : cluster [DBG] osdmap e1405: 8 total, 8 up, 8 in 2024-04-07T17:06:22.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:06:21 smithi144 bash[66654]: cluster 2024-04-07T17:06:19.916174+0000 mgr.y (mgr.25175) 1529 : cluster [DBG] pgmap v2509: 137 pgs: 137 active+clean; 69 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 14 MiB/s rd, 4.0 MiB/s wr, 270 op/s 2024-04-07T17:06:22.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:06:21 smithi144 bash[66654]: cluster 2024-04-07T17:06:20.838428+0000 mon.a (mon.0) 2674 : cluster [DBG] osdmap e1406: 8 total, 8 up, 8 in 2024-04-07T17:06:22.202 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:06:21 smithi100 bash[74942]: cluster 2024-04-07T17:06:19.916174+0000 mgr.y (mgr.25175) 1529 : cluster [DBG] pgmap v2509: 137 pgs: 137 active+clean; 69 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 14 MiB/s rd, 4.0 MiB/s wr, 270 op/s 2024-04-07T17:06:22.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:06:21 smithi100 bash[74942]: cluster 2024-04-07T17:06:20.838428+0000 mon.a (mon.0) 2674 : cluster [DBG] osdmap e1406: 8 total, 8 up, 8 in 2024-04-07T17:06:22.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:06:21 smithi100 bash[76202]: cluster 2024-04-07T17:06:19.916174+0000 mgr.y (mgr.25175) 1529 : cluster [DBG] pgmap v2509: 137 pgs: 137 active+clean; 69 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 14 MiB/s rd, 4.0 MiB/s wr, 270 op/s 2024-04-07T17:06:22.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:06:21 smithi100 bash[76202]: cluster 2024-04-07T17:06:20.838428+0000 mon.a (mon.0) 2674 : cluster [DBG] osdmap e1406: 8 total, 8 up, 8 in 2024-04-07T17:06:22.615 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:06:22 smithi144 bash[95233]: ts=2024-04-07T17:06:22.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:06:23.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:06:22 smithi144 bash[66654]: cluster 2024-04-07T17:06:21.852650+0000 mon.a (mon.0) 2675 : cluster [DBG] osdmap e1407: 8 total, 8 up, 8 in 2024-04-07T17:06:23.202 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:06:22 smithi100 bash[74942]: cluster 2024-04-07T17:06:21.852650+0000 mon.a (mon.0) 2675 : cluster [DBG] osdmap e1407: 8 total, 8 up, 8 in 2024-04-07T17:06:23.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:06:22 smithi100 bash[76202]: cluster 2024-04-07T17:06:21.852650+0000 mon.a (mon.0) 2675 : cluster [DBG] osdmap e1407: 8 total, 8 up, 8 in 2024-04-07T17:06:24.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:06:23 smithi144 bash[66654]: cluster 2024-04-07T17:06:21.917023+0000 mgr.y (mgr.25175) 1530 : cluster [DBG] pgmap v2512: 137 pgs: 137 active+clean; 69 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 6.2 MiB/s rd, 1.0 MiB/s wr, 260 op/s 2024-04-07T17:06:24.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:06:23 smithi144 bash[66654]: cluster 2024-04-07T17:06:22.865043+0000 mon.a (mon.0) 2676 : cluster [DBG] osdmap e1408: 8 total, 8 up, 8 in 2024-04-07T17:06:24.202 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:06:23 smithi100 bash[74942]: cluster 2024-04-07T17:06:21.917023+0000 mgr.y (mgr.25175) 1530 : cluster [DBG] pgmap v2512: 137 pgs: 137 active+clean; 69 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 6.2 MiB/s rd, 1.0 MiB/s wr, 260 op/s 2024-04-07T17:06:24.202 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:06:23 smithi100 bash[74942]: cluster 2024-04-07T17:06:22.865043+0000 mon.a (mon.0) 2676 : cluster [DBG] osdmap e1408: 8 total, 8 up, 8 in 2024-04-07T17:06:24.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:06:23 smithi100 bash[76202]: cluster 2024-04-07T17:06:21.917023+0000 mgr.y (mgr.25175) 1530 : cluster [DBG] pgmap v2512: 137 pgs: 137 active+clean; 69 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 6.2 MiB/s rd, 1.0 MiB/s wr, 260 op/s 2024-04-07T17:06:24.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:06:23 smithi100 bash[76202]: cluster 2024-04-07T17:06:22.865043+0000 mon.a (mon.0) 2676 : cluster [DBG] osdmap e1408: 8 total, 8 up, 8 in 2024-04-07T17:06:25.202 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:06:24 smithi100 bash[76202]: cluster 2024-04-07T17:06:23.874379+0000 mon.a (mon.0) 2677 : cluster [DBG] osdmap e1409: 8 total, 8 up, 8 in 2024-04-07T17:06:25.202 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:06:24 smithi100 bash[76202]: cluster 2024-04-07T17:06:23.917917+0000 mgr.y (mgr.25175) 1531 : cluster [DBG] pgmap v2515: 137 pgs: 137 active+clean; 69 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 243 KiB/s rd, 23 KiB/s wr, 336 op/s 2024-04-07T17:06:25.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:06:24 smithi100 bash[74942]: cluster 2024-04-07T17:06:23.874379+0000 mon.a (mon.0) 2677 : cluster [DBG] osdmap e1409: 8 total, 8 up, 8 in 2024-04-07T17:06:25.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:06:24 smithi100 bash[74942]: cluster 2024-04-07T17:06:23.917917+0000 mgr.y (mgr.25175) 1531 : cluster [DBG] pgmap v2515: 137 pgs: 137 active+clean; 69 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 243 KiB/s rd, 23 KiB/s wr, 336 op/s 2024-04-07T17:06:25.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:06:24 smithi144 bash[66654]: cluster 2024-04-07T17:06:23.874379+0000 mon.a (mon.0) 2677 : cluster [DBG] osdmap e1409: 8 total, 8 up, 8 in 2024-04-07T17:06:25.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:06:24 smithi144 bash[66654]: cluster 2024-04-07T17:06:23.917917+0000 mgr.y (mgr.25175) 1531 : cluster [DBG] pgmap v2515: 137 pgs: 137 active+clean; 69 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 243 KiB/s rd, 23 KiB/s wr, 336 op/s 2024-04-07T17:06:26.148 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestDeepCopy.CloneHideParent (6769 ms) 2024-04-07T17:06:26.148 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestDeepCopy.Clone 2024-04-07T17:06:26.202 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:06:25 smithi100 bash[74942]: cluster 2024-04-07T17:06:24.878442+0000 mon.a (mon.0) 2678 : cluster [DBG] osdmap e1410: 8 total, 8 up, 8 in 2024-04-07T17:06:26.202 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:06:25 smithi100 bash[76202]: cluster 2024-04-07T17:06:24.878442+0000 mon.a (mon.0) 2678 : cluster [DBG] osdmap e1410: 8 total, 8 up, 8 in 2024-04-07T17:06:26.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:06:25 smithi144 bash[66654]: cluster 2024-04-07T17:06:24.878442+0000 mon.a (mon.0) 2678 : cluster [DBG] osdmap e1410: 8 total, 8 up, 8 in 2024-04-07T17:06:27.202 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:06:26 smithi100 bash[74942]: cluster 2024-04-07T17:06:25.893750+0000 mon.a (mon.0) 2679 : cluster [DBG] osdmap e1411: 8 total, 8 up, 8 in 2024-04-07T17:06:27.202 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:06:26 smithi100 bash[74942]: cluster 2024-04-07T17:06:25.918617+0000 mgr.y (mgr.25175) 1532 : cluster [DBG] pgmap v2518: 137 pgs: 137 active+clean; 69 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 242 KiB/s rd, 24 KiB/s wr, 341 op/s 2024-04-07T17:06:27.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:06:26 smithi100 bash[76202]: cluster 2024-04-07T17:06:25.893750+0000 mon.a (mon.0) 2679 : cluster [DBG] osdmap e1411: 8 total, 8 up, 8 in 2024-04-07T17:06:27.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:06:26 smithi100 bash[76202]: cluster 2024-04-07T17:06:25.918617+0000 mgr.y (mgr.25175) 1532 : cluster [DBG] pgmap v2518: 137 pgs: 137 active+clean; 69 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 242 KiB/s rd, 24 KiB/s wr, 341 op/s 2024-04-07T17:06:27.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:06:26 smithi144 bash[66654]: cluster 2024-04-07T17:06:25.893750+0000 mon.a (mon.0) 2679 : cluster [DBG] osdmap e1411: 8 total, 8 up, 8 in 2024-04-07T17:06:27.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:06:26 smithi144 bash[66654]: cluster 2024-04-07T17:06:25.918617+0000 mgr.y (mgr.25175) 1532 : cluster [DBG] pgmap v2518: 137 pgs: 137 active+clean; 69 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 242 KiB/s rd, 24 KiB/s wr, 341 op/s 2024-04-07T17:06:28.202 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:06:27 smithi100 bash[74942]: cluster 2024-04-07T17:06:26.921764+0000 mon.a (mon.0) 2680 : cluster [DBG] osdmap e1412: 8 total, 8 up, 8 in 2024-04-07T17:06:28.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:06:27 smithi100 bash[76202]: cluster 2024-04-07T17:06:26.921764+0000 mon.a (mon.0) 2680 : cluster [DBG] osdmap e1412: 8 total, 8 up, 8 in 2024-04-07T17:06:28.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:06:27 smithi144 bash[66654]: cluster 2024-04-07T17:06:26.921764+0000 mon.a (mon.0) 2680 : cluster [DBG] osdmap e1412: 8 total, 8 up, 8 in 2024-04-07T17:06:28.681 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestDeepCopy.Clone (2533 ms) 2024-04-07T17:06:28.681 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestDeepCopy.CloneFlatten 2024-04-07T17:06:28.702 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:06:28 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:06:28] "GET /metrics HTTP/1.1" 200 36484 "" "Prometheus/2.43.0" 2024-04-07T17:06:29.202 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:06:28 smithi100 bash[74942]: cluster 2024-04-07T17:06:27.919492+0000 mgr.y (mgr.25175) 1533 : cluster [DBG] pgmap v2521: 137 pgs: 137 active+clean; 73 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 139 KiB/s rd, 1.0 MiB/s wr, 194 op/s 2024-04-07T17:06:29.202 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:06:28 smithi100 bash[74942]: cluster 2024-04-07T17:06:27.923820+0000 mon.a (mon.0) 2681 : cluster [DBG] osdmap e1413: 8 total, 8 up, 8 in 2024-04-07T17:06:29.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:06:28 smithi100 bash[76202]: cluster 2024-04-07T17:06:27.919492+0000 mgr.y (mgr.25175) 1533 : cluster [DBG] pgmap v2521: 137 pgs: 137 active+clean; 73 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 139 KiB/s rd, 1.0 MiB/s wr, 194 op/s 2024-04-07T17:06:29.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:06:28 smithi100 bash[76202]: cluster 2024-04-07T17:06:27.923820+0000 mon.a (mon.0) 2681 : cluster [DBG] osdmap e1413: 8 total, 8 up, 8 in 2024-04-07T17:06:29.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:06:28 smithi144 bash[66654]: cluster 2024-04-07T17:06:27.919492+0000 mgr.y (mgr.25175) 1533 : cluster [DBG] pgmap v2521: 137 pgs: 137 active+clean; 73 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 139 KiB/s rd, 1.0 MiB/s wr, 194 op/s 2024-04-07T17:06:29.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:06:28 smithi144 bash[66654]: cluster 2024-04-07T17:06:27.923820+0000 mon.a (mon.0) 2681 : cluster [DBG] osdmap e1413: 8 total, 8 up, 8 in 2024-04-07T17:06:30.202 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:06:29 smithi100 bash[74942]: cluster 2024-04-07T17:06:28.947101+0000 mon.a (mon.0) 2682 : cluster [DBG] osdmap e1414: 8 total, 8 up, 8 in 2024-04-07T17:06:30.202 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:06:29 smithi100 bash[76202]: cluster 2024-04-07T17:06:28.947101+0000 mon.a (mon.0) 2682 : cluster [DBG] osdmap e1414: 8 total, 8 up, 8 in 2024-04-07T17:06:30.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:06:29 smithi144 bash[66654]: cluster 2024-04-07T17:06:28.947101+0000 mon.a (mon.0) 2682 : cluster [DBG] osdmap e1414: 8 total, 8 up, 8 in 2024-04-07T17:06:30.828 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestDeepCopy.CloneFlatten (2148 ms) 2024-04-07T17:06:30.829 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestDeepCopy.Stress 2024-04-07T17:06:30.895 INFO:tasks.workunit.client.0.smithi100.stdout:write: A 29634924~3696948 2024-04-07T17:06:30.928 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 63118870~6685663 2024-04-07T17:06:30.931 INFO:tasks.workunit.client.0.smithi100.stdout:write: A 28882096~8286374 2024-04-07T17:06:30.969 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 46582673~6171153 2024-04-07T17:06:30.970 INFO:tasks.workunit.client.0.smithi100.stdout:write: A 13203222~1240774 2024-04-07T17:06:30.982 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 78399291~5486789 2024-04-07T17:06:30.983 INFO:tasks.workunit.client.0.smithi100.stdout:write: A 18940497~4242061 2024-04-07T17:06:31.007 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 54389062~2981230 2024-04-07T17:06:31.007 INFO:tasks.workunit.client.0.smithi100.stdout:snap: snap0 2024-04-07T17:06:31.202 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:06:30 smithi100 bash[74942]: cluster 2024-04-07T17:06:29.920457+0000 mgr.y (mgr.25175) 1534 : cluster [DBG] pgmap v2523: 137 pgs: 137 active+clean; 97 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 8.1 MiB/s rd, 7.0 MiB/s wr, 197 op/s 2024-04-07T17:06:31.202 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:06:30 smithi100 bash[74942]: cluster 2024-04-07T17:06:29.958125+0000 mon.a (mon.0) 2683 : cluster [DBG] osdmap e1415: 8 total, 8 up, 8 in 2024-04-07T17:06:31.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:06:30 smithi100 bash[76202]: cluster 2024-04-07T17:06:29.920457+0000 mgr.y (mgr.25175) 1534 : cluster [DBG] pgmap v2523: 137 pgs: 137 active+clean; 97 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 8.1 MiB/s rd, 7.0 MiB/s wr, 197 op/s 2024-04-07T17:06:31.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:06:30 smithi100 bash[76202]: cluster 2024-04-07T17:06:29.958125+0000 mon.a (mon.0) 2683 : cluster [DBG] osdmap e1415: 8 total, 8 up, 8 in 2024-04-07T17:06:31.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:06:30 smithi144 bash[66654]: cluster 2024-04-07T17:06:29.920457+0000 mgr.y (mgr.25175) 1534 : cluster [DBG] pgmap v2523: 137 pgs: 137 active+clean; 97 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 8.1 MiB/s rd, 7.0 MiB/s wr, 197 op/s 2024-04-07T17:06:31.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:06:30 smithi144 bash[66654]: cluster 2024-04-07T17:06:29.958125+0000 mon.a (mon.0) 2683 : cluster [DBG] osdmap e1415: 8 total, 8 up, 8 in 2024-04-07T17:06:32.011 INFO:tasks.workunit.client.0.smithi100.stdout:clone image234 -> image235 2024-04-07T17:06:32.137 INFO:tasks.workunit.client.0.smithi100.stdout:write: B 12359909~2444181 2024-04-07T17:06:32.184 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 6702975~2291355 2024-04-07T17:06:32.192 INFO:tasks.workunit.client.0.smithi100.stdout:write: B 77059713~6826367 2024-04-07T17:06:32.226 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 37993981~5730032 2024-04-07T17:06:32.234 INFO:tasks.workunit.client.0.smithi100.stdout:write: B 63787598~2756939 2024-04-07T17:06:32.256 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 31102300~8272507 2024-04-07T17:06:32.283 INFO:tasks.workunit.client.0.smithi100.stdout:write: B 79078320~2019283 2024-04-07T17:06:32.293 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 17093554~2016969 2024-04-07T17:06:32.313 INFO:tasks.workunit.client.0.smithi100.stdout:snap: snap1 2024-04-07T17:06:32.315 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:06:31 smithi144 bash[66654]: audit 2024-04-07T17:06:31.807351+0000 mon.a (mon.0) 2684 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:06:32.452 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:06:31 smithi100 bash[74942]: audit 2024-04-07T17:06:31.807351+0000 mon.a (mon.0) 2684 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:06:32.453 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:06:31 smithi100 bash[76202]: audit 2024-04-07T17:06:31.807351+0000 mon.a (mon.0) 2684 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:06:32.616 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:06:32 smithi144 bash[95233]: ts=2024-04-07T17:06:32.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:06:33.028 INFO:tasks.workunit.client.0.smithi100.stdout:clone image235 -> image236 2024-04-07T17:06:33.185 INFO:tasks.workunit.client.0.smithi100.stdout:resize: 129861731 2024-04-07T17:06:33.201 INFO:tasks.workunit.client.0.smithi100.stdout:write: C 105448755~5762007 2024-04-07T17:06:33.235 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 95109473~5273385 2024-04-07T17:06:33.236 INFO:tasks.workunit.client.0.smithi100.stdout:write: C 59862658~3652550 2024-04-07T17:06:33.268 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 61150248~6430075 2024-04-07T17:06:33.279 INFO:tasks.workunit.client.0.smithi100.stdout:write: C 45982237~6215927 2024-04-07T17:06:33.319 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 116913867~1929292 2024-04-07T17:06:33.320 INFO:tasks.workunit.client.0.smithi100.stdout:write: C 81721451~4543063 2024-04-07T17:06:33.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:06:32 smithi144 bash[66654]: cluster 2024-04-07T17:06:31.921257+0000 mgr.y (mgr.25175) 1535 : cluster [DBG] pgmap v2525: 137 pgs: 137 active+clean; 101 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 8.2 MiB/s rd, 6.4 MiB/s wr, 231 op/s 2024-04-07T17:06:33.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:06:32 smithi144 bash[66654]: cluster 2024-04-07T17:06:31.970712+0000 mon.a (mon.0) 2685 : cluster [DBG] osdmap e1416: 8 total, 8 up, 8 in 2024-04-07T17:06:33.368 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 96749048~471166 2024-04-07T17:06:33.368 INFO:tasks.workunit.client.0.smithi100.stdout:snap: snap2 2024-04-07T17:06:33.452 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:06:32 smithi100 bash[74942]: cluster 2024-04-07T17:06:31.921257+0000 mgr.y (mgr.25175) 1535 : cluster [DBG] pgmap v2525: 137 pgs: 137 active+clean; 101 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 8.2 MiB/s rd, 6.4 MiB/s wr, 231 op/s 2024-04-07T17:06:33.452 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:06:32 smithi100 bash[74942]: cluster 2024-04-07T17:06:31.970712+0000 mon.a (mon.0) 2685 : cluster [DBG] osdmap e1416: 8 total, 8 up, 8 in 2024-04-07T17:06:33.453 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:06:32 smithi100 bash[76202]: cluster 2024-04-07T17:06:31.921257+0000 mgr.y (mgr.25175) 1535 : cluster [DBG] pgmap v2525: 137 pgs: 137 active+clean; 101 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 8.2 MiB/s rd, 6.4 MiB/s wr, 231 op/s 2024-04-07T17:06:33.453 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:06:32 smithi100 bash[76202]: cluster 2024-04-07T17:06:31.970712+0000 mon.a (mon.0) 2685 : cluster [DBG] osdmap e1416: 8 total, 8 up, 8 in 2024-04-07T17:06:34.016 INFO:tasks.workunit.client.0.smithi100.stdout:resize: 149424889 2024-04-07T17:06:34.029 INFO:tasks.workunit.client.0.smithi100.stdout:write: D 97019208~389293 2024-04-07T17:06:34.036 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 134984310~5266196 2024-04-07T17:06:34.037 INFO:tasks.workunit.client.0.smithi100.stdout:write: D 124187658~4400776 2024-04-07T17:06:34.038 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 135880364~3854041 2024-04-07T17:06:34.044 INFO:tasks.workunit.client.0.smithi100.stdout:write: D 43279547~7601818 2024-04-07T17:06:34.044 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 45188861~8332480 2024-04-07T17:06:34.095 INFO:tasks.workunit.client.0.smithi100.stdout:write: D 13810183~818295 2024-04-07T17:06:34.095 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 63790008~130598 2024-04-07T17:06:34.121 INFO:tasks.workunit.client.0.smithi100.stdout:snap: snap3 2024-04-07T17:06:34.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:06:33 smithi144 bash[66654]: cluster 2024-04-07T17:06:32.982308+0000 mon.a (mon.0) 2686 : cluster [DBG] osdmap e1417: 8 total, 8 up, 8 in 2024-04-07T17:06:34.452 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:06:33 smithi100 bash[74942]: cluster 2024-04-07T17:06:32.982308+0000 mon.a (mon.0) 2686 : cluster [DBG] osdmap e1417: 8 total, 8 up, 8 in 2024-04-07T17:06:34.452 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:06:33 smithi100 bash[76202]: cluster 2024-04-07T17:06:32.982308+0000 mon.a (mon.0) 2686 : cluster [DBG] osdmap e1417: 8 total, 8 up, 8 in 2024-04-07T17:06:35.044 INFO:tasks.workunit.client.0.smithi100.stdout:clone image236 -> image237 2024-04-07T17:06:35.249 INFO:tasks.workunit.client.0.smithi100.stdout:resize: 84933855 2024-04-07T17:06:35.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:06:34 smithi144 bash[66654]: cluster 2024-04-07T17:06:33.922243+0000 mgr.y (mgr.25175) 1536 : cluster [DBG] pgmap v2528: 137 pgs: 137 active+clean; 175 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 28 MiB/s rd, 20 MiB/s wr, 391 op/s 2024-04-07T17:06:35.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:06:34 smithi144 bash[66654]: cluster 2024-04-07T17:06:33.995067+0000 mon.a (mon.0) 2687 : cluster [DBG] osdmap e1418: 8 total, 8 up, 8 in 2024-04-07T17:06:35.452 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:06:34 smithi100 bash[74942]: cluster 2024-04-07T17:06:33.922243+0000 mgr.y (mgr.25175) 1536 : cluster [DBG] pgmap v2528: 137 pgs: 137 active+clean; 175 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 28 MiB/s rd, 20 MiB/s wr, 391 op/s 2024-04-07T17:06:35.452 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:06:34 smithi100 bash[74942]: cluster 2024-04-07T17:06:33.995067+0000 mon.a (mon.0) 2687 : cluster [DBG] osdmap e1418: 8 total, 8 up, 8 in 2024-04-07T17:06:35.452 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:06:34 smithi100 bash[76202]: cluster 2024-04-07T17:06:33.922243+0000 mgr.y (mgr.25175) 1536 : cluster [DBG] pgmap v2528: 137 pgs: 137 active+clean; 175 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 28 MiB/s rd, 20 MiB/s wr, 391 op/s 2024-04-07T17:06:35.453 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:06:34 smithi100 bash[76202]: cluster 2024-04-07T17:06:33.995067+0000 mon.a (mon.0) 2687 : cluster [DBG] osdmap e1418: 8 total, 8 up, 8 in 2024-04-07T17:06:35.797 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestDeepCopy.Stress (4969 ms) 2024-04-07T17:06:35.797 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestDeepCopy.NoSnaps_LargerDstObjSize 2024-04-07T17:06:36.284 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestDeepCopy.NoSnaps_LargerDstObjSize (487 ms) 2024-04-07T17:06:36.284 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestDeepCopy.Snaps_LargerDstObjSize 2024-04-07T17:06:36.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:06:35 smithi144 bash[66654]: cluster 2024-04-07T17:06:35.000989+0000 mon.a (mon.0) 2688 : cluster [DBG] osdmap e1419: 8 total, 8 up, 8 in 2024-04-07T17:06:36.452 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:06:35 smithi100 bash[74942]: cluster 2024-04-07T17:06:35.000989+0000 mon.a (mon.0) 2688 : cluster [DBG] osdmap e1419: 8 total, 8 up, 8 in 2024-04-07T17:06:36.452 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:06:35 smithi100 bash[76202]: cluster 2024-04-07T17:06:35.000989+0000 mon.a (mon.0) 2688 : cluster [DBG] osdmap e1419: 8 total, 8 up, 8 in 2024-04-07T17:06:37.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:06:37 smithi144 bash[66654]: cluster 2024-04-07T17:06:35.923219+0000 mgr.y (mgr.25175) 1537 : cluster [DBG] pgmap v2531: 137 pgs: 137 active+clean; 247 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 39 MiB/s rd, 36 MiB/s wr, 440 op/s 2024-04-07T17:06:37.452 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:06:37 smithi100 bash[74942]: cluster 2024-04-07T17:06:35.923219+0000 mgr.y (mgr.25175) 1537 : cluster [DBG] pgmap v2531: 137 pgs: 137 active+clean; 247 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 39 MiB/s rd, 36 MiB/s wr, 440 op/s 2024-04-07T17:06:37.452 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:06:37 smithi100 bash[76202]: cluster 2024-04-07T17:06:35.923219+0000 mgr.y (mgr.25175) 1537 : cluster [DBG] pgmap v2531: 137 pgs: 137 active+clean; 247 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 39 MiB/s rd, 36 MiB/s wr, 440 op/s 2024-04-07T17:06:38.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:06:38 smithi144 bash[66654]: cluster 2024-04-07T17:06:37.015828+0000 mon.a (mon.0) 2689 : cluster [DBG] osdmap e1420: 8 total, 8 up, 8 in 2024-04-07T17:06:38.426 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:06:38 smithi100 bash[74942]: cluster 2024-04-07T17:06:37.015828+0000 mon.a (mon.0) 2689 : cluster [DBG] osdmap e1420: 8 total, 8 up, 8 in 2024-04-07T17:06:38.426 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:06:38 smithi100 bash[76202]: cluster 2024-04-07T17:06:37.015828+0000 mon.a (mon.0) 2689 : cluster [DBG] osdmap e1420: 8 total, 8 up, 8 in 2024-04-07T17:06:38.702 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:06:38 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:06:38] "GET /metrics HTTP/1.1" 200 36491 "" "Prometheus/2.43.0" 2024-04-07T17:06:39.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:06:39 smithi144 bash[66654]: cluster 2024-04-07T17:06:37.924435+0000 mgr.y (mgr.25175) 1538 : cluster [DBG] pgmap v2533: 137 pgs: 137 active+clean; 316 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 25 MiB/s rd, 28 MiB/s wr, 342 op/s 2024-04-07T17:06:39.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:06:39 smithi144 bash[66654]: cluster 2024-04-07T17:06:38.036847+0000 mon.a (mon.0) 2690 : cluster [DBG] osdmap e1421: 8 total, 8 up, 8 in 2024-04-07T17:06:39.452 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:06:39 smithi100 bash[74942]: cluster 2024-04-07T17:06:37.924435+0000 mgr.y (mgr.25175) 1538 : cluster [DBG] pgmap v2533: 137 pgs: 137 active+clean; 316 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 25 MiB/s rd, 28 MiB/s wr, 342 op/s 2024-04-07T17:06:39.452 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:06:39 smithi100 bash[74942]: cluster 2024-04-07T17:06:38.036847+0000 mon.a (mon.0) 2690 : cluster [DBG] osdmap e1421: 8 total, 8 up, 8 in 2024-04-07T17:06:39.452 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:06:39 smithi100 bash[76202]: cluster 2024-04-07T17:06:37.924435+0000 mgr.y (mgr.25175) 1538 : cluster [DBG] pgmap v2533: 137 pgs: 137 active+clean; 316 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 25 MiB/s rd, 28 MiB/s wr, 342 op/s 2024-04-07T17:06:39.453 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:06:39 smithi100 bash[76202]: cluster 2024-04-07T17:06:38.036847+0000 mon.a (mon.0) 2690 : cluster [DBG] osdmap e1421: 8 total, 8 up, 8 in 2024-04-07T17:06:40.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:06:40 smithi144 bash[66654]: cluster 2024-04-07T17:06:39.047767+0000 mon.a (mon.0) 2691 : cluster [DBG] osdmap e1422: 8 total, 8 up, 8 in 2024-04-07T17:06:40.452 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:06:40 smithi100 bash[74942]: cluster 2024-04-07T17:06:39.047767+0000 mon.a (mon.0) 2691 : cluster [DBG] osdmap e1422: 8 total, 8 up, 8 in 2024-04-07T17:06:40.453 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:06:40 smithi100 bash[76202]: cluster 2024-04-07T17:06:39.047767+0000 mon.a (mon.0) 2691 : cluster [DBG] osdmap e1422: 8 total, 8 up, 8 in 2024-04-07T17:06:40.622 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestDeepCopy.Snaps_LargerDstObjSize (4338 ms) 2024-04-07T17:06:40.622 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestDeepCopy.Clone_LargerDstObjSize 2024-04-07T17:06:41.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:06:41 smithi144 bash[66654]: cluster 2024-04-07T17:06:39.925335+0000 mgr.y (mgr.25175) 1539 : cluster [DBG] pgmap v2536: 137 pgs: 137 active+clean; 340 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 19 MiB/s rd, 20 MiB/s wr, 420 op/s 2024-04-07T17:06:41.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:06:41 smithi144 bash[66654]: cluster 2024-04-07T17:06:40.059353+0000 mon.a (mon.0) 2692 : cluster [DBG] osdmap e1423: 8 total, 8 up, 8 in 2024-04-07T17:06:41.452 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:06:41 smithi100 bash[74942]: cluster 2024-04-07T17:06:39.925335+0000 mgr.y (mgr.25175) 1539 : cluster [DBG] pgmap v2536: 137 pgs: 137 active+clean; 340 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 19 MiB/s rd, 20 MiB/s wr, 420 op/s 2024-04-07T17:06:41.453 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:06:41 smithi100 bash[74942]: cluster 2024-04-07T17:06:40.059353+0000 mon.a (mon.0) 2692 : cluster [DBG] osdmap e1423: 8 total, 8 up, 8 in 2024-04-07T17:06:41.453 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:06:41 smithi100 bash[76202]: cluster 2024-04-07T17:06:39.925335+0000 mgr.y (mgr.25175) 1539 : cluster [DBG] pgmap v2536: 137 pgs: 137 active+clean; 340 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 19 MiB/s rd, 20 MiB/s wr, 420 op/s 2024-04-07T17:06:41.453 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:06:41 smithi100 bash[76202]: cluster 2024-04-07T17:06:40.059353+0000 mon.a (mon.0) 2692 : cluster [DBG] osdmap e1423: 8 total, 8 up, 8 in 2024-04-07T17:06:42.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:06:42 smithi144 bash[66654]: cluster 2024-04-07T17:06:41.069593+0000 mon.a (mon.0) 2693 : cluster [DBG] osdmap e1424: 8 total, 8 up, 8 in 2024-04-07T17:06:42.366 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:06:42 smithi144 bash[95233]: ts=2024-04-07T17:06:42.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:06:42.452 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:06:42 smithi100 bash[74942]: cluster 2024-04-07T17:06:41.069593+0000 mon.a (mon.0) 2693 : cluster [DBG] osdmap e1424: 8 total, 8 up, 8 in 2024-04-07T17:06:42.453 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:06:42 smithi100 bash[76202]: cluster 2024-04-07T17:06:41.069593+0000 mon.a (mon.0) 2693 : cluster [DBG] osdmap e1424: 8 total, 8 up, 8 in 2024-04-07T17:06:42.874 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestDeepCopy.Clone_LargerDstObjSize (2251 ms) 2024-04-07T17:06:42.874 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestDeepCopy.CloneFlatten_LargerDstObjSize 2024-04-07T17:06:43.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:06:43 smithi144 bash[66654]: cluster 2024-04-07T17:06:41.926145+0000 mgr.y (mgr.25175) 1540 : cluster [DBG] pgmap v2539: 137 pgs: 137 active+clean; 344 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 7.7 MiB/s rd, 6.4 MiB/s wr, 290 op/s 2024-04-07T17:06:43.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:06:43 smithi144 bash[66654]: cluster 2024-04-07T17:06:42.085521+0000 mon.a (mon.0) 2694 : cluster [DBG] osdmap e1425: 8 total, 8 up, 8 in 2024-04-07T17:06:43.452 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:06:43 smithi100 bash[74942]: cluster 2024-04-07T17:06:41.926145+0000 mgr.y (mgr.25175) 1540 : cluster [DBG] pgmap v2539: 137 pgs: 137 active+clean; 344 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 7.7 MiB/s rd, 6.4 MiB/s wr, 290 op/s 2024-04-07T17:06:43.453 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:06:43 smithi100 bash[74942]: cluster 2024-04-07T17:06:42.085521+0000 mon.a (mon.0) 2694 : cluster [DBG] osdmap e1425: 8 total, 8 up, 8 in 2024-04-07T17:06:43.453 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:06:43 smithi100 bash[76202]: cluster 2024-04-07T17:06:41.926145+0000 mgr.y (mgr.25175) 1540 : cluster [DBG] pgmap v2539: 137 pgs: 137 active+clean; 344 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 7.7 MiB/s rd, 6.4 MiB/s wr, 290 op/s 2024-04-07T17:06:43.453 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:06:43 smithi100 bash[76202]: cluster 2024-04-07T17:06:42.085521+0000 mon.a (mon.0) 2694 : cluster [DBG] osdmap e1425: 8 total, 8 up, 8 in 2024-04-07T17:06:44.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:06:44 smithi144 bash[66654]: cluster 2024-04-07T17:06:43.091858+0000 mon.a (mon.0) 2695 : cluster [DBG] osdmap e1426: 8 total, 8 up, 8 in 2024-04-07T17:06:44.452 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:06:44 smithi100 bash[74942]: cluster 2024-04-07T17:06:43.091858+0000 mon.a (mon.0) 2695 : cluster [DBG] osdmap e1426: 8 total, 8 up, 8 in 2024-04-07T17:06:44.453 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:06:44 smithi100 bash[76202]: cluster 2024-04-07T17:06:43.091858+0000 mon.a (mon.0) 2695 : cluster [DBG] osdmap e1426: 8 total, 8 up, 8 in 2024-04-07T17:06:45.008 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestDeepCopy.CloneFlatten_LargerDstObjSize (2134 ms) 2024-04-07T17:06:45.008 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestDeepCopy.Stress_LargerDstObjSize 2024-04-07T17:06:45.072 INFO:tasks.workunit.client.0.smithi100.stdout:write: A 49704074~1118699 2024-04-07T17:06:45.091 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 57350940~7689720 2024-04-07T17:06:45.095 INFO:tasks.workunit.client.0.smithi100.stdout:write: A 60878798~8232201 2024-04-07T17:06:45.135 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 12328170~2969037 2024-04-07T17:06:45.136 INFO:tasks.workunit.client.0.smithi100.stdout:write: A 20135546~2357397 2024-04-07T17:06:45.153 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 24400809~2403862 2024-04-07T17:06:45.157 INFO:tasks.workunit.client.0.smithi100.stdout:write: A 40359070~3631523 2024-04-07T17:06:45.175 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 32651388~6520465 2024-04-07T17:06:45.178 INFO:tasks.workunit.client.0.smithi100.stdout:snap: snap0 2024-04-07T17:06:45.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:06:45 smithi144 bash[66654]: cluster 2024-04-07T17:06:43.926910+0000 mgr.y (mgr.25175) 1541 : cluster [DBG] pgmap v2542: 137 pgs: 137 active+clean; 372 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 11 MiB/s rd, 8.0 MiB/s wr, 127 op/s 2024-04-07T17:06:45.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:06:45 smithi144 bash[66654]: cluster 2024-04-07T17:06:44.103379+0000 mon.a (mon.0) 2696 : cluster [DBG] osdmap e1427: 8 total, 8 up, 8 in 2024-04-07T17:06:45.452 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:06:45 smithi100 bash[74942]: cluster 2024-04-07T17:06:43.926910+0000 mgr.y (mgr.25175) 1541 : cluster [DBG] pgmap v2542: 137 pgs: 137 active+clean; 372 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 11 MiB/s rd, 8.0 MiB/s wr, 127 op/s 2024-04-07T17:06:45.452 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:06:45 smithi100 bash[74942]: cluster 2024-04-07T17:06:44.103379+0000 mon.a (mon.0) 2696 : cluster [DBG] osdmap e1427: 8 total, 8 up, 8 in 2024-04-07T17:06:45.453 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:06:45 smithi100 bash[76202]: cluster 2024-04-07T17:06:43.926910+0000 mgr.y (mgr.25175) 1541 : cluster [DBG] pgmap v2542: 137 pgs: 137 active+clean; 372 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 11 MiB/s rd, 8.0 MiB/s wr, 127 op/s 2024-04-07T17:06:45.453 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:06:45 smithi100 bash[76202]: cluster 2024-04-07T17:06:44.103379+0000 mon.a (mon.0) 2696 : cluster [DBG] osdmap e1427: 8 total, 8 up, 8 in 2024-04-07T17:06:46.142 INFO:tasks.workunit.client.0.smithi100.stdout:resize: 119982364 2024-04-07T17:06:46.156 INFO:tasks.workunit.client.0.smithi100.stdout:write: B 3643096~4777648 2024-04-07T17:06:46.163 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 56523833~3360147 2024-04-07T17:06:46.163 INFO:tasks.workunit.client.0.smithi100.stdout:write: B 51361703~721486 2024-04-07T17:06:46.163 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 103734329~4649464 2024-04-07T17:06:46.166 INFO:tasks.workunit.client.0.smithi100.stdout:write: B 46544963~7414231 2024-04-07T17:06:46.184 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 110208436~5279396 2024-04-07T17:06:46.186 INFO:tasks.workunit.client.0.smithi100.stdout:write: B 64135131~4957570 2024-04-07T17:06:46.186 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 59766658~1473392 2024-04-07T17:06:46.231 INFO:tasks.workunit.client.0.smithi100.stdout:snap: snap1 2024-04-07T17:06:47.152 INFO:tasks.workunit.client.0.smithi100.stdout:write: C 5893126~6557291 2024-04-07T17:06:47.158 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 61010322~5989331 2024-04-07T17:06:47.159 INFO:tasks.workunit.client.0.smithi100.stdout:write: C 64454929~2450232 2024-04-07T17:06:47.170 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 114749174~5158537 2024-04-07T17:06:47.171 INFO:tasks.workunit.client.0.smithi100.stdout:write: C 52555646~3356174 2024-04-07T17:06:47.171 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 65015557~5700038 2024-04-07T17:06:47.207 INFO:tasks.workunit.client.0.smithi100.stdout:write: C 24460355~6797346 2024-04-07T17:06:47.207 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 106247441~1292129 2024-04-07T17:06:47.239 INFO:tasks.workunit.client.0.smithi100.stdout:snap: snap2 2024-04-07T17:06:47.452 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:06:47 smithi100 bash[74942]: cluster 2024-04-07T17:06:45.927935+0000 mgr.y (mgr.25175) 1542 : cluster [DBG] pgmap v2544: 137 pgs: 137 active+clean; 416 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 31 MiB/s rd, 15 MiB/s wr, 195 op/s 2024-04-07T17:06:47.453 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:06:47 smithi100 bash[74942]: cluster 2024-04-07T17:06:46.116382+0000 mon.a (mon.0) 2697 : cluster [DBG] osdmap e1428: 8 total, 8 up, 8 in 2024-04-07T17:06:47.453 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:06:47 smithi100 bash[74942]: audit 2024-04-07T17:06:46.807850+0000 mon.a (mon.0) 2698 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:06:47.453 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:06:47 smithi100 bash[76202]: cluster 2024-04-07T17:06:45.927935+0000 mgr.y (mgr.25175) 1542 : cluster [DBG] pgmap v2544: 137 pgs: 137 active+clean; 416 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 31 MiB/s rd, 15 MiB/s wr, 195 op/s 2024-04-07T17:06:47.453 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:06:47 smithi100 bash[76202]: cluster 2024-04-07T17:06:46.116382+0000 mon.a (mon.0) 2697 : cluster [DBG] osdmap e1428: 8 total, 8 up, 8 in 2024-04-07T17:06:47.453 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:06:47 smithi100 bash[76202]: audit 2024-04-07T17:06:46.807850+0000 mon.a (mon.0) 2698 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:06:47.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:06:47 smithi144 bash[66654]: cluster 2024-04-07T17:06:45.927935+0000 mgr.y (mgr.25175) 1542 : cluster [DBG] pgmap v2544: 137 pgs: 137 active+clean; 416 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 31 MiB/s rd, 15 MiB/s wr, 195 op/s 2024-04-07T17:06:47.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:06:47 smithi144 bash[66654]: cluster 2024-04-07T17:06:46.116382+0000 mon.a (mon.0) 2697 : cluster [DBG] osdmap e1428: 8 total, 8 up, 8 in 2024-04-07T17:06:47.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:06:47 smithi144 bash[66654]: audit 2024-04-07T17:06:46.807850+0000 mon.a (mon.0) 2698 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:06:48.168 INFO:tasks.workunit.client.0.smithi100.stdout:resize: 133711764 2024-04-07T17:06:48.183 INFO:tasks.workunit.client.0.smithi100.stdout:write: D 67856436~1237217 2024-04-07T17:06:48.192 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 33783660~7160963 2024-04-07T17:06:48.194 INFO:tasks.workunit.client.0.smithi100.stdout:write: D 73619103~2991185 2024-04-07T17:06:48.194 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 58986584~1645050 2024-04-07T17:06:48.195 INFO:tasks.workunit.client.0.smithi100.stdout:write: D 68507236~2418027 2024-04-07T17:06:48.219 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 1709066~3749816 2024-04-07T17:06:48.220 INFO:tasks.workunit.client.0.smithi100.stdout:write: D 2491074~3401988 2024-04-07T17:06:48.230 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 64661186~3650685 2024-04-07T17:06:48.245 INFO:tasks.workunit.client.0.smithi100.stdout:snap: snap3 2024-04-07T17:06:48.427 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:06:48 smithi100 bash[74942]: cluster 2024-04-07T17:06:47.117510+0000 mon.a (mon.0) 2699 : cluster [DBG] osdmap e1429: 8 total, 8 up, 8 in 2024-04-07T17:06:48.427 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:06:48 smithi100 bash[76202]: cluster 2024-04-07T17:06:47.117510+0000 mon.a (mon.0) 2699 : cluster [DBG] osdmap e1429: 8 total, 8 up, 8 in 2024-04-07T17:06:48.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:06:48 smithi144 bash[66654]: cluster 2024-04-07T17:06:47.117510+0000 mon.a (mon.0) 2699 : cluster [DBG] osdmap e1429: 8 total, 8 up, 8 in 2024-04-07T17:06:48.702 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:06:48 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:06:48] "GET /metrics HTTP/1.1" 200 36494 "" "Prometheus/2.43.0" 2024-04-07T17:06:49.193 INFO:tasks.workunit.client.0.smithi100.stdout:clone image256 -> image257 2024-04-07T17:06:49.452 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:06:49 smithi100 bash[74942]: cluster 2024-04-07T17:06:47.928732+0000 mgr.y (mgr.25175) 1543 : cluster [DBG] pgmap v2547: 137 pgs: 137 active+clean; 522 MiB data, 4.7 GiB used, 711 GiB / 715 GiB avail; 44 MiB/s rd, 28 MiB/s wr, 321 op/s 2024-04-07T17:06:49.453 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:06:49 smithi100 bash[74942]: cluster 2024-04-07T17:06:48.141754+0000 mon.a (mon.0) 2700 : cluster [DBG] osdmap e1430: 8 total, 8 up, 8 in 2024-04-07T17:06:49.453 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:06:49 smithi100 bash[76202]: cluster 2024-04-07T17:06:47.928732+0000 mgr.y (mgr.25175) 1543 : cluster [DBG] pgmap v2547: 137 pgs: 137 active+clean; 522 MiB data, 4.7 GiB used, 711 GiB / 715 GiB avail; 44 MiB/s rd, 28 MiB/s wr, 321 op/s 2024-04-07T17:06:49.453 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:06:49 smithi100 bash[76202]: cluster 2024-04-07T17:06:48.141754+0000 mon.a (mon.0) 2700 : cluster [DBG] osdmap e1430: 8 total, 8 up, 8 in 2024-04-07T17:06:49.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:06:49 smithi144 bash[66654]: cluster 2024-04-07T17:06:47.928732+0000 mgr.y (mgr.25175) 1543 : cluster [DBG] pgmap v2547: 137 pgs: 137 active+clean; 522 MiB data, 4.7 GiB used, 711 GiB / 715 GiB avail; 44 MiB/s rd, 28 MiB/s wr, 321 op/s 2024-04-07T17:06:49.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:06:49 smithi144 bash[66654]: cluster 2024-04-07T17:06:48.141754+0000 mon.a (mon.0) 2700 : cluster [DBG] osdmap e1430: 8 total, 8 up, 8 in 2024-04-07T17:06:49.814 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestDeepCopy.Stress_LargerDstObjSize (4806 ms) 2024-04-07T17:06:49.814 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestDeepCopy.NoSnaps_SmallerDstObjSize 2024-04-07T17:06:50.234 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestDeepCopy.NoSnaps_SmallerDstObjSize (421 ms) 2024-04-07T17:06:50.235 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestDeepCopy.Snaps_SmallerDstObjSize 2024-04-07T17:06:50.452 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:06:50 smithi100 bash[74942]: cluster 2024-04-07T17:06:49.150256+0000 mon.a (mon.0) 2701 : cluster [DBG] osdmap e1431: 8 total, 8 up, 8 in 2024-04-07T17:06:50.453 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:06:50 smithi100 bash[76202]: cluster 2024-04-07T17:06:49.150256+0000 mon.a (mon.0) 2701 : cluster [DBG] osdmap e1431: 8 total, 8 up, 8 in 2024-04-07T17:06:50.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:06:50 smithi144 bash[66654]: cluster 2024-04-07T17:06:49.150256+0000 mon.a (mon.0) 2701 : cluster [DBG] osdmap e1431: 8 total, 8 up, 8 in 2024-04-07T17:06:51.452 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:06:51 smithi100 bash[74942]: cluster 2024-04-07T17:06:49.929932+0000 mgr.y (mgr.25175) 1544 : cluster [DBG] pgmap v2550: 137 pgs: 137 active+clean; 559 MiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 35 MiB/s rd, 32 MiB/s wr, 450 op/s 2024-04-07T17:06:51.453 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:06:51 smithi100 bash[76202]: cluster 2024-04-07T17:06:49.929932+0000 mgr.y (mgr.25175) 1544 : cluster [DBG] pgmap v2550: 137 pgs: 137 active+clean; 559 MiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 35 MiB/s rd, 32 MiB/s wr, 450 op/s 2024-04-07T17:06:51.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:06:51 smithi144 bash[66654]: cluster 2024-04-07T17:06:49.929932+0000 mgr.y (mgr.25175) 1544 : cluster [DBG] pgmap v2550: 137 pgs: 137 active+clean; 559 MiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 35 MiB/s rd, 32 MiB/s wr, 450 op/s 2024-04-07T17:06:52.452 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:06:52 smithi100 bash[74942]: cluster 2024-04-07T17:06:51.173484+0000 mon.a (mon.0) 2702 : cluster [DBG] osdmap e1432: 8 total, 8 up, 8 in 2024-04-07T17:06:52.453 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:06:52 smithi100 bash[76202]: cluster 2024-04-07T17:06:51.173484+0000 mon.a (mon.0) 2702 : cluster [DBG] osdmap e1432: 8 total, 8 up, 8 in 2024-04-07T17:06:52.616 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:06:52 smithi144 bash[95233]: ts=2024-04-07T17:06:52.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:06:52.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:06:52 smithi144 bash[66654]: cluster 2024-04-07T17:06:51.173484+0000 mon.a (mon.0) 2702 : cluster [DBG] osdmap e1432: 8 total, 8 up, 8 in 2024-04-07T17:06:53.453 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:06:53 smithi100 bash[74942]: cluster 2024-04-07T17:06:51.930646+0000 mgr.y (mgr.25175) 1545 : cluster [DBG] pgmap v2552: 137 pgs: 137 active+clean; 563 MiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 9.6 MiB/s rd, 7.9 MiB/s wr, 149 op/s 2024-04-07T17:06:53.453 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:06:53 smithi100 bash[74942]: cluster 2024-04-07T17:06:52.182536+0000 mon.a (mon.0) 2703 : cluster [DBG] osdmap e1433: 8 total, 8 up, 8 in 2024-04-07T17:06:53.453 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:06:53 smithi100 bash[74942]: audit 2024-04-07T17:06:52.320516+0000 mon.a (mon.0) 2704 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:06:53.453 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:06:53 smithi100 bash[74942]: audit 2024-04-07T17:06:53.048495+0000 mon.a (mon.0) 2705 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:06:53.453 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:06:53 smithi100 bash[74942]: audit 2024-04-07T17:06:53.050281+0000 mon.a (mon.0) 2706 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:06:53.453 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:06:53 smithi100 bash[74942]: audit 2024-04-07T17:06:53.060322+0000 mon.a (mon.0) 2707 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:06:53.453 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:06:53 smithi100 bash[76202]: cluster 2024-04-07T17:06:51.930646+0000 mgr.y (mgr.25175) 1545 : cluster [DBG] pgmap v2552: 137 pgs: 137 active+clean; 563 MiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 9.6 MiB/s rd, 7.9 MiB/s wr, 149 op/s 2024-04-07T17:06:53.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:06:53 smithi100 bash[76202]: cluster 2024-04-07T17:06:52.182536+0000 mon.a (mon.0) 2703 : cluster [DBG] osdmap e1433: 8 total, 8 up, 8 in 2024-04-07T17:06:53.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:06:53 smithi100 bash[76202]: audit 2024-04-07T17:06:52.320516+0000 mon.a (mon.0) 2704 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:06:53.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:06:53 smithi100 bash[76202]: audit 2024-04-07T17:06:53.048495+0000 mon.a (mon.0) 2705 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:06:53.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:06:53 smithi100 bash[76202]: audit 2024-04-07T17:06:53.050281+0000 mon.a (mon.0) 2706 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:06:53.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:06:53 smithi100 bash[76202]: audit 2024-04-07T17:06:53.060322+0000 mon.a (mon.0) 2707 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:06:53.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:06:53 smithi144 bash[66654]: cluster 2024-04-07T17:06:51.930646+0000 mgr.y (mgr.25175) 1545 : cluster [DBG] pgmap v2552: 137 pgs: 137 active+clean; 563 MiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 9.6 MiB/s rd, 7.9 MiB/s wr, 149 op/s 2024-04-07T17:06:53.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:06:53 smithi144 bash[66654]: cluster 2024-04-07T17:06:52.182536+0000 mon.a (mon.0) 2703 : cluster [DBG] osdmap e1433: 8 total, 8 up, 8 in 2024-04-07T17:06:53.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:06:53 smithi144 bash[66654]: audit 2024-04-07T17:06:52.320516+0000 mon.a (mon.0) 2704 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:06:53.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:06:53 smithi144 bash[66654]: audit 2024-04-07T17:06:53.048495+0000 mon.a (mon.0) 2705 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:06:53.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:06:53 smithi144 bash[66654]: audit 2024-04-07T17:06:53.050281+0000 mon.a (mon.0) 2706 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:06:53.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:06:53 smithi144 bash[66654]: audit 2024-04-07T17:06:53.060322+0000 mon.a (mon.0) 2707 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:06:54.452 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:06:54 smithi100 bash[74942]: cluster 2024-04-07T17:06:53.184729+0000 mon.a (mon.0) 2708 : cluster [DBG] osdmap e1434: 8 total, 8 up, 8 in 2024-04-07T17:06:54.452 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:06:54 smithi100 bash[76202]: cluster 2024-04-07T17:06:53.184729+0000 mon.a (mon.0) 2708 : cluster [DBG] osdmap e1434: 8 total, 8 up, 8 in 2024-04-07T17:06:54.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:06:54 smithi144 bash[66654]: cluster 2024-04-07T17:06:53.184729+0000 mon.a (mon.0) 2708 : cluster [DBG] osdmap e1434: 8 total, 8 up, 8 in 2024-04-07T17:06:54.661 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestDeepCopy.Snaps_SmallerDstObjSize (4427 ms) 2024-04-07T17:06:54.662 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestDeepCopy.Clone_SmallerDstObjSize 2024-04-07T17:06:55.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:06:55 smithi100 bash[74942]: cluster 2024-04-07T17:06:53.931296+0000 mgr.y (mgr.25175) 1546 : cluster [DBG] pgmap v2555: 137 pgs: 137 active+clean; 627 MiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 24 MiB/s rd, 15 MiB/s wr, 136 op/s 2024-04-07T17:06:55.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:06:55 smithi100 bash[74942]: cluster 2024-04-07T17:06:54.206832+0000 mon.a (mon.0) 2709 : cluster [DBG] osdmap e1435: 8 total, 8 up, 8 in 2024-04-07T17:06:55.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:06:55 smithi100 bash[76202]: cluster 2024-04-07T17:06:53.931296+0000 mgr.y (mgr.25175) 1546 : cluster [DBG] pgmap v2555: 137 pgs: 137 active+clean; 627 MiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 24 MiB/s rd, 15 MiB/s wr, 136 op/s 2024-04-07T17:06:55.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:06:55 smithi100 bash[76202]: cluster 2024-04-07T17:06:54.206832+0000 mon.a (mon.0) 2709 : cluster [DBG] osdmap e1435: 8 total, 8 up, 8 in 2024-04-07T17:06:55.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:06:55 smithi144 bash[66654]: cluster 2024-04-07T17:06:53.931296+0000 mgr.y (mgr.25175) 1546 : cluster [DBG] pgmap v2555: 137 pgs: 137 active+clean; 627 MiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 24 MiB/s rd, 15 MiB/s wr, 136 op/s 2024-04-07T17:06:55.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:06:55 smithi144 bash[66654]: cluster 2024-04-07T17:06:54.206832+0000 mon.a (mon.0) 2709 : cluster [DBG] osdmap e1435: 8 total, 8 up, 8 in 2024-04-07T17:06:56.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:06:56 smithi144 bash[66654]: cluster 2024-04-07T17:06:55.218877+0000 mon.a (mon.0) 2710 : cluster [DBG] osdmap e1436: 8 total, 8 up, 8 in 2024-04-07T17:06:56.702 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:06:56 smithi100 bash[76202]: cluster 2024-04-07T17:06:55.218877+0000 mon.a (mon.0) 2710 : cluster [DBG] osdmap e1436: 8 total, 8 up, 8 in 2024-04-07T17:06:56.703 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:06:56 smithi100 bash[74942]: cluster 2024-04-07T17:06:55.218877+0000 mon.a (mon.0) 2710 : cluster [DBG] osdmap e1436: 8 total, 8 up, 8 in 2024-04-07T17:06:56.926 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestDeepCopy.Clone_SmallerDstObjSize (2265 ms) 2024-04-07T17:06:56.927 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestDeepCopy.CloneFlatten_SmallerDstObjSize 2024-04-07T17:06:57.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:06:57 smithi144 bash[66654]: cluster 2024-04-07T17:06:55.931950+0000 mgr.y (mgr.25175) 1547 : cluster [DBG] pgmap v2558: 137 pgs: 137 active+clean; 649 MiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 32 MiB/s rd, 20 MiB/s wr, 352 op/s 2024-04-07T17:06:57.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:06:57 smithi144 bash[66654]: cluster 2024-04-07T17:06:56.228560+0000 mon.a (mon.0) 2711 : cluster [DBG] osdmap e1437: 8 total, 8 up, 8 in 2024-04-07T17:06:57.702 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:06:57 smithi100 bash[74942]: cluster 2024-04-07T17:06:55.931950+0000 mgr.y (mgr.25175) 1547 : cluster [DBG] pgmap v2558: 137 pgs: 137 active+clean; 649 MiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 32 MiB/s rd, 20 MiB/s wr, 352 op/s 2024-04-07T17:06:57.703 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:06:57 smithi100 bash[74942]: cluster 2024-04-07T17:06:56.228560+0000 mon.a (mon.0) 2711 : cluster [DBG] osdmap e1437: 8 total, 8 up, 8 in 2024-04-07T17:06:57.703 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:06:57 smithi100 bash[76202]: cluster 2024-04-07T17:06:55.931950+0000 mgr.y (mgr.25175) 1547 : cluster [DBG] pgmap v2558: 137 pgs: 137 active+clean; 649 MiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 32 MiB/s rd, 20 MiB/s wr, 352 op/s 2024-04-07T17:06:57.703 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:06:57 smithi100 bash[76202]: cluster 2024-04-07T17:06:56.228560+0000 mon.a (mon.0) 2711 : cluster [DBG] osdmap e1437: 8 total, 8 up, 8 in 2024-04-07T17:06:58.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:06:58 smithi144 bash[66654]: cluster 2024-04-07T17:06:57.236661+0000 mon.a (mon.0) 2712 : cluster [DBG] osdmap e1438: 8 total, 8 up, 8 in 2024-04-07T17:06:58.702 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:06:58 smithi100 bash[74942]: cluster 2024-04-07T17:06:57.236661+0000 mon.a (mon.0) 2712 : cluster [DBG] osdmap e1438: 8 total, 8 up, 8 in 2024-04-07T17:06:58.703 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:06:58 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:06:58] "GET /metrics HTTP/1.1" 200 36494 "" "Prometheus/2.43.0" 2024-04-07T17:06:58.703 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:06:58 smithi100 bash[76202]: cluster 2024-04-07T17:06:57.236661+0000 mon.a (mon.0) 2712 : cluster [DBG] osdmap e1438: 8 total, 8 up, 8 in 2024-04-07T17:06:59.001 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestDeepCopy.CloneFlatten_SmallerDstObjSize (2074 ms) 2024-04-07T17:06:59.001 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestDeepCopy.Stress_SmallerDstObjSize 2024-04-07T17:06:59.066 INFO:tasks.workunit.client.0.smithi100.stdout:write: A 63144367~5954425 2024-04-07T17:06:59.105 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 29528987~4543809 2024-04-07T17:06:59.108 INFO:tasks.workunit.client.0.smithi100.stdout:write: A 39390370~6513190 2024-04-07T17:06:59.141 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 61821198~6132048 2024-04-07T17:06:59.154 INFO:tasks.workunit.client.0.smithi100.stdout:write: A 64599037~7849553 2024-04-07T17:06:59.181 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 79547921~4338159 2024-04-07T17:06:59.182 INFO:tasks.workunit.client.0.smithi100.stdout:write: A 34580071~4651136 2024-04-07T17:06:59.200 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 55927420~163919 2024-04-07T17:06:59.200 INFO:tasks.workunit.client.0.smithi100.stdout:snap: snap0 2024-04-07T17:06:59.303 INFO:tasks.workunit.client.0.smithi100.stdout:clone image276 -> image277 2024-04-07T17:06:59.437 INFO:tasks.workunit.client.0.smithi100.stdout:resize: 159008743 2024-04-07T17:06:59.455 INFO:tasks.workunit.client.0.smithi100.stdout:write: B 122482692~7903994 2024-04-07T17:06:59.491 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 130995348~5675153 2024-04-07T17:06:59.496 INFO:tasks.workunit.client.0.smithi100.stdout:write: B 129015205~6964321 2024-04-07T17:06:59.522 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 135148959~7846376 2024-04-07T17:06:59.525 INFO:tasks.workunit.client.0.smithi100.stdout:write: B 4529980~1423625 2024-04-07T17:06:59.538 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 93275173~3546203 2024-04-07T17:06:59.541 INFO:tasks.workunit.client.0.smithi100.stdout:write: B 1830560~7111125 2024-04-07T17:06:59.569 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 130414036~1421395 2024-04-07T17:06:59.573 INFO:tasks.workunit.client.0.smithi100.stdout:snap: snap1 2024-04-07T17:06:59.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:06:59 smithi144 bash[66654]: cluster 2024-04-07T17:06:57.932666+0000 mgr.y (mgr.25175) 1548 : cluster [DBG] pgmap v2561: 137 pgs: 137 active+clean; 672 MiB data, 5.1 GiB used, 710 GiB / 715 GiB avail; 27 MiB/s rd, 11 MiB/s wr, 461 op/s 2024-04-07T17:06:59.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:06:59 smithi144 bash[66654]: cluster 2024-04-07T17:06:58.247119+0000 mon.a (mon.0) 2713 : cluster [DBG] osdmap e1439: 8 total, 8 up, 8 in 2024-04-07T17:06:59.702 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:06:59 smithi100 bash[74942]: cluster 2024-04-07T17:06:57.932666+0000 mgr.y (mgr.25175) 1548 : cluster [DBG] pgmap v2561: 137 pgs: 137 active+clean; 672 MiB data, 5.1 GiB used, 710 GiB / 715 GiB avail; 27 MiB/s rd, 11 MiB/s wr, 461 op/s 2024-04-07T17:06:59.702 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:06:59 smithi100 bash[74942]: cluster 2024-04-07T17:06:58.247119+0000 mon.a (mon.0) 2713 : cluster [DBG] osdmap e1439: 8 total, 8 up, 8 in 2024-04-07T17:06:59.703 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:06:59 smithi100 bash[76202]: cluster 2024-04-07T17:06:57.932666+0000 mgr.y (mgr.25175) 1548 : cluster [DBG] pgmap v2561: 137 pgs: 137 active+clean; 672 MiB data, 5.1 GiB used, 710 GiB / 715 GiB avail; 27 MiB/s rd, 11 MiB/s wr, 461 op/s 2024-04-07T17:06:59.703 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:06:59 smithi100 bash[76202]: cluster 2024-04-07T17:06:58.247119+0000 mon.a (mon.0) 2713 : cluster [DBG] osdmap e1439: 8 total, 8 up, 8 in 2024-04-07T17:07:00.298 INFO:tasks.workunit.client.0.smithi100.stdout:write: C 152176984~829121 2024-04-07T17:07:00.307 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 34122580~6718279 2024-04-07T17:07:00.309 INFO:tasks.workunit.client.0.smithi100.stdout:write: C 106428354~5145390 2024-04-07T17:07:00.309 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 143012421~2970430 2024-04-07T17:07:00.311 INFO:tasks.workunit.client.0.smithi100.stdout:write: C 102714935~4054932 2024-04-07T17:07:00.355 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 106385401~453111 2024-04-07T17:07:00.371 INFO:tasks.workunit.client.0.smithi100.stdout:write: C 25509130~8191394 2024-04-07T17:07:00.371 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 14241356~6641386 2024-04-07T17:07:00.412 INFO:tasks.workunit.client.0.smithi100.stdout:snap: snap2 2024-04-07T17:07:00.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:07:00 smithi144 bash[66654]: cluster 2024-04-07T17:06:59.256472+0000 mon.a (mon.0) 2714 : cluster [DBG] osdmap e1440: 8 total, 8 up, 8 in 2024-04-07T17:07:00.702 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:07:00 smithi100 bash[74942]: cluster 2024-04-07T17:06:59.256472+0000 mon.a (mon.0) 2714 : cluster [DBG] osdmap e1440: 8 total, 8 up, 8 in 2024-04-07T17:07:00.703 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:07:00 smithi100 bash[76202]: cluster 2024-04-07T17:06:59.256472+0000 mon.a (mon.0) 2714 : cluster [DBG] osdmap e1440: 8 total, 8 up, 8 in 2024-04-07T17:07:01.329 INFO:tasks.workunit.client.0.smithi100.stdout:clone image277 -> image278 2024-04-07T17:07:01.495 INFO:tasks.workunit.client.0.smithi100.stdout:resize: 227142641 2024-04-07T17:07:01.512 INFO:tasks.workunit.client.0.smithi100.stdout:write: D 143606733~2343639 2024-04-07T17:07:01.539 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 157682091~7838796 2024-04-07T17:07:01.547 INFO:tasks.workunit.client.0.smithi100.stdout:write: D 46009197~5196826 2024-04-07T17:07:01.606 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 185426558~2222856 2024-04-07T17:07:01.608 INFO:tasks.workunit.client.0.smithi100.stdout:write: D 42700001~6137777 2024-04-07T17:07:01.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:07:01 smithi144 bash[66654]: cluster 2024-04-07T17:06:59.933375+0000 mgr.y (mgr.25175) 1549 : cluster [DBG] pgmap v2564: 137 pgs: 137 active+clean; 735 MiB data, 5.2 GiB used, 710 GiB / 715 GiB avail; 44 MiB/s rd, 22 MiB/s wr, 470 op/s 2024-04-07T17:07:01.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:07:01 smithi144 bash[66654]: cluster 2024-04-07T17:07:00.268495+0000 mon.a (mon.0) 2715 : cluster [DBG] osdmap e1441: 8 total, 8 up, 8 in 2024-04-07T17:07:01.630 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 186304754~2484821 2024-04-07T17:07:01.632 INFO:tasks.workunit.client.0.smithi100.stdout:write: D 144597938~6137657 2024-04-07T17:07:01.660 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 44449316~3336175 2024-04-07T17:07:01.663 INFO:tasks.workunit.client.0.smithi100.stdout:snap: snap3 2024-04-07T17:07:01.702 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:07:01 smithi100 bash[74942]: cluster 2024-04-07T17:06:59.933375+0000 mgr.y (mgr.25175) 1549 : cluster [DBG] pgmap v2564: 137 pgs: 137 active+clean; 735 MiB data, 5.2 GiB used, 710 GiB / 715 GiB avail; 44 MiB/s rd, 22 MiB/s wr, 470 op/s 2024-04-07T17:07:01.702 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:07:01 smithi100 bash[74942]: cluster 2024-04-07T17:07:00.268495+0000 mon.a (mon.0) 2715 : cluster [DBG] osdmap e1441: 8 total, 8 up, 8 in 2024-04-07T17:07:01.703 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:07:01 smithi100 bash[76202]: cluster 2024-04-07T17:06:59.933375+0000 mgr.y (mgr.25175) 1549 : cluster [DBG] pgmap v2564: 137 pgs: 137 active+clean; 735 MiB data, 5.2 GiB used, 710 GiB / 715 GiB avail; 44 MiB/s rd, 22 MiB/s wr, 470 op/s 2024-04-07T17:07:01.703 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:07:01 smithi100 bash[76202]: cluster 2024-04-07T17:07:00.268495+0000 mon.a (mon.0) 2715 : cluster [DBG] osdmap e1441: 8 total, 8 up, 8 in 2024-04-07T17:07:02.333 INFO:tasks.workunit.client.0.smithi100.stdout:clone image278 -> image279 2024-04-07T17:07:02.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:07:02 smithi144 bash[66654]: cluster 2024-04-07T17:07:01.275756+0000 mon.a (mon.0) 2716 : cluster [DBG] osdmap e1442: 8 total, 8 up, 8 in 2024-04-07T17:07:02.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:07:02 smithi144 bash[66654]: audit 2024-04-07T17:07:01.808371+0000 mon.a (mon.0) 2717 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:07:02.616 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:07:02 smithi144 bash[95233]: ts=2024-04-07T17:07:02.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:07:02.702 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:07:02 smithi100 bash[74942]: cluster 2024-04-07T17:07:01.275756+0000 mon.a (mon.0) 2716 : cluster [DBG] osdmap e1442: 8 total, 8 up, 8 in 2024-04-07T17:07:02.702 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:07:02 smithi100 bash[74942]: audit 2024-04-07T17:07:01.808371+0000 mon.a (mon.0) 2717 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:07:02.703 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:07:02 smithi100 bash[76202]: cluster 2024-04-07T17:07:01.275756+0000 mon.a (mon.0) 2716 : cluster [DBG] osdmap e1442: 8 total, 8 up, 8 in 2024-04-07T17:07:02.703 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:07:02 smithi100 bash[76202]: audit 2024-04-07T17:07:01.808371+0000 mon.a (mon.0) 2717 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:07:03.274 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestDeepCopy.Stress_SmallerDstObjSize (4274 ms) 2024-04-07T17:07:03.274 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestDeepCopy.NoSnaps_StrippingLargerDstObjSize 2024-04-07T17:07:03.345 INFO:tasks.workunit.client.0.smithi100.stdout:./src/test/librbd/test_DeepCopy.cc:621: Skipped 2024-04-07T17:07:03.345 INFO:tasks.workunit.client.0.smithi100.stdout:Skipping due to unmet REQUIRE 2024-04-07T17:07:03.525 INFO:tasks.workunit.client.0.smithi100.stdout:[ SKIPPED ] TestDeepCopy.NoSnaps_StrippingLargerDstObjSize (251 ms) 2024-04-07T17:07:03.525 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestDeepCopy.Snaps_StrippingLargerDstObjSize 2024-04-07T17:07:03.586 INFO:tasks.workunit.client.0.smithi100.stdout:./src/test/librbd/test_DeepCopy.cc:635: Skipped 2024-04-07T17:07:03.586 INFO:tasks.workunit.client.0.smithi100.stdout:Skipping due to unmet REQUIRE 2024-04-07T17:07:03.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:07:03 smithi144 bash[66654]: cluster 2024-04-07T17:07:01.934079+0000 mgr.y (mgr.25175) 1550 : cluster [DBG] pgmap v2567: 137 pgs: 137 active+clean; 749 MiB data, 5.3 GiB used, 710 GiB / 715 GiB avail; 30 MiB/s rd, 19 MiB/s wr, 312 op/s 2024-04-07T17:07:03.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:07:03 smithi144 bash[66654]: cluster 2024-04-07T17:07:02.286871+0000 mon.a (mon.0) 2718 : cluster [DBG] osdmap e1443: 8 total, 8 up, 8 in 2024-04-07T17:07:03.702 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:07:03 smithi100 bash[74942]: cluster 2024-04-07T17:07:01.934079+0000 mgr.y (mgr.25175) 1550 : cluster [DBG] pgmap v2567: 137 pgs: 137 active+clean; 749 MiB data, 5.3 GiB used, 710 GiB / 715 GiB avail; 30 MiB/s rd, 19 MiB/s wr, 312 op/s 2024-04-07T17:07:03.702 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:07:03 smithi100 bash[74942]: cluster 2024-04-07T17:07:02.286871+0000 mon.a (mon.0) 2718 : cluster [DBG] osdmap e1443: 8 total, 8 up, 8 in 2024-04-07T17:07:03.703 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:07:03 smithi100 bash[76202]: cluster 2024-04-07T17:07:01.934079+0000 mgr.y (mgr.25175) 1550 : cluster [DBG] pgmap v2567: 137 pgs: 137 active+clean; 749 MiB data, 5.3 GiB used, 710 GiB / 715 GiB avail; 30 MiB/s rd, 19 MiB/s wr, 312 op/s 2024-04-07T17:07:03.703 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:07:03 smithi100 bash[76202]: cluster 2024-04-07T17:07:02.286871+0000 mon.a (mon.0) 2718 : cluster [DBG] osdmap e1443: 8 total, 8 up, 8 in 2024-04-07T17:07:03.769 INFO:tasks.workunit.client.0.smithi100.stdout:[ SKIPPED ] TestDeepCopy.Snaps_StrippingLargerDstObjSize (244 ms) 2024-04-07T17:07:03.770 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestDeepCopy.Clone_StrippingLargerDstObjSize 2024-04-07T17:07:03.834 INFO:tasks.workunit.client.0.smithi100.stdout:./src/test/librbd/test_DeepCopy.cc:649: Skipped 2024-04-07T17:07:03.834 INFO:tasks.workunit.client.0.smithi100.stdout:Skipping due to unmet REQUIRE 2024-04-07T17:07:04.016 INFO:tasks.workunit.client.0.smithi100.stdout:[ SKIPPED ] TestDeepCopy.Clone_StrippingLargerDstObjSize (246 ms) 2024-04-07T17:07:04.016 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestDeepCopy.CloneFlatten_StrippingLargerDstObjSize 2024-04-07T17:07:04.084 INFO:tasks.workunit.client.0.smithi100.stdout:./src/test/librbd/test_DeepCopy.cc:663: Skipped 2024-04-07T17:07:04.085 INFO:tasks.workunit.client.0.smithi100.stdout:Skipping due to unmet REQUIRE 2024-04-07T17:07:04.270 INFO:tasks.workunit.client.0.smithi100.stdout:[ SKIPPED ] TestDeepCopy.CloneFlatten_StrippingLargerDstObjSize (254 ms) 2024-04-07T17:07:04.270 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestDeepCopy.Stress_StrippingLargerDstObjSize 2024-04-07T17:07:04.337 INFO:tasks.workunit.client.0.smithi100.stdout:./src/test/librbd/test_DeepCopy.cc:679: Skipped 2024-04-07T17:07:04.337 INFO:tasks.workunit.client.0.smithi100.stdout:Skipping due to unmet REQUIRE 2024-04-07T17:07:04.521 INFO:tasks.workunit.client.0.smithi100.stdout:[ SKIPPED ] TestDeepCopy.Stress_StrippingLargerDstObjSize (251 ms) 2024-04-07T17:07:04.521 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestDeepCopy.NoSnaps_StrippingSmallerDstObjSize 2024-04-07T17:07:04.583 INFO:tasks.workunit.client.0.smithi100.stdout:./src/test/librbd/test_DeepCopy.cc:693: Skipped 2024-04-07T17:07:04.583 INFO:tasks.workunit.client.0.smithi100.stdout:Skipping due to unmet REQUIRE 2024-04-07T17:07:04.770 INFO:tasks.workunit.client.0.smithi100.stdout:[ SKIPPED ] TestDeepCopy.NoSnaps_StrippingSmallerDstObjSize (248 ms) 2024-04-07T17:07:04.770 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestDeepCopy.Snaps_StrippingSmallerDstObjSize 2024-04-07T17:07:04.835 INFO:tasks.workunit.client.0.smithi100.stdout:./src/test/librbd/test_DeepCopy.cc:707: Skipped 2024-04-07T17:07:04.835 INFO:tasks.workunit.client.0.smithi100.stdout:Skipping due to unmet REQUIRE 2024-04-07T17:07:05.025 INFO:tasks.workunit.client.0.smithi100.stdout:[ SKIPPED ] TestDeepCopy.Snaps_StrippingSmallerDstObjSize (256 ms) 2024-04-07T17:07:05.025 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestDeepCopy.Clone_StrippingSmallerDstObjSize 2024-04-07T17:07:05.088 INFO:tasks.workunit.client.0.smithi100.stdout:./src/test/librbd/test_DeepCopy.cc:721: Skipped 2024-04-07T17:07:05.088 INFO:tasks.workunit.client.0.smithi100.stdout:Skipping due to unmet REQUIRE 2024-04-07T17:07:05.275 INFO:tasks.workunit.client.0.smithi100.stdout:[ SKIPPED ] TestDeepCopy.Clone_StrippingSmallerDstObjSize (251 ms) 2024-04-07T17:07:05.275 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestDeepCopy.CloneFlatten_StrippingSmallerDstObjSize 2024-04-07T17:07:05.341 INFO:tasks.workunit.client.0.smithi100.stdout:./src/test/librbd/test_DeepCopy.cc:735: Skipped 2024-04-07T17:07:05.341 INFO:tasks.workunit.client.0.smithi100.stdout:Skipping due to unmet REQUIRE 2024-04-07T17:07:05.527 INFO:tasks.workunit.client.0.smithi100.stdout:[ SKIPPED ] TestDeepCopy.CloneFlatten_StrippingSmallerDstObjSize (251 ms) 2024-04-07T17:07:05.527 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestDeepCopy.Stress_StrippingSmallerDstObjSize 2024-04-07T17:07:05.591 INFO:tasks.workunit.client.0.smithi100.stdout:./src/test/librbd/test_DeepCopy.cc:751: Skipped 2024-04-07T17:07:05.591 INFO:tasks.workunit.client.0.smithi100.stdout:Skipping due to unmet REQUIRE 2024-04-07T17:07:05.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:07:05 smithi144 bash[66654]: cluster 2024-04-07T17:07:03.935048+0000 mgr.y (mgr.25175) 1551 : cluster [DBG] pgmap v2569: 137 pgs: 137 active+clean; 866 MiB data, 5.6 GiB used, 710 GiB / 715 GiB avail; 22 MiB/s rd, 28 MiB/s wr, 379 op/s 2024-04-07T17:07:05.702 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:07:05 smithi100 bash[74942]: cluster 2024-04-07T17:07:03.935048+0000 mgr.y (mgr.25175) 1551 : cluster [DBG] pgmap v2569: 137 pgs: 137 active+clean; 866 MiB data, 5.6 GiB used, 710 GiB / 715 GiB avail; 22 MiB/s rd, 28 MiB/s wr, 379 op/s 2024-04-07T17:07:05.703 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:07:05 smithi100 bash[76202]: cluster 2024-04-07T17:07:03.935048+0000 mgr.y (mgr.25175) 1551 : cluster [DBG] pgmap v2569: 137 pgs: 137 active+clean; 866 MiB data, 5.6 GiB used, 710 GiB / 715 GiB avail; 22 MiB/s rd, 28 MiB/s wr, 379 op/s 2024-04-07T17:07:05.775 INFO:tasks.workunit.client.0.smithi100.stdout:[ SKIPPED ] TestDeepCopy.Stress_StrippingSmallerDstObjSize (249 ms) 2024-04-07T17:07:06.321 INFO:tasks.workunit.client.0.smithi100.stdout:[----------] 31 tests from TestDeepCopy (58082 ms total) 2024-04-07T17:07:06.322 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:07:06.322 INFO:tasks.workunit.client.0.smithi100.stdout:[----------] 51 tests from TestMirroring 2024-04-07T17:07:07.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:07:07 smithi144 bash[66654]: cluster 2024-04-07T17:07:05.936199+0000 mgr.y (mgr.25175) 1552 : cluster [DBG] pgmap v2570: 137 pgs: 137 active+clean; 898 MiB data, 5.7 GiB used, 709 GiB / 715 GiB avail; 25 MiB/s rd, 27 MiB/s wr, 443 op/s 2024-04-07T17:07:07.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:07:07 smithi144 bash[66654]: cluster 2024-04-07T17:07:06.301753+0000 mon.a (mon.0) 2719 : cluster [DBG] osdmap e1444: 8 total, 8 up, 8 in 2024-04-07T17:07:07.702 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:07:07 smithi100 bash[74942]: cluster 2024-04-07T17:07:05.936199+0000 mgr.y (mgr.25175) 1552 : cluster [DBG] pgmap v2570: 137 pgs: 137 active+clean; 898 MiB data, 5.7 GiB used, 709 GiB / 715 GiB avail; 25 MiB/s rd, 27 MiB/s wr, 443 op/s 2024-04-07T17:07:07.703 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:07:07 smithi100 bash[74942]: cluster 2024-04-07T17:07:06.301753+0000 mon.a (mon.0) 2719 : cluster [DBG] osdmap e1444: 8 total, 8 up, 8 in 2024-04-07T17:07:07.703 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:07:07 smithi100 bash[76202]: cluster 2024-04-07T17:07:05.936199+0000 mgr.y (mgr.25175) 1552 : cluster [DBG] pgmap v2570: 137 pgs: 137 active+clean; 898 MiB data, 5.7 GiB used, 709 GiB / 715 GiB avail; 25 MiB/s rd, 27 MiB/s wr, 443 op/s 2024-04-07T17:07:07.703 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:07:07 smithi100 bash[76202]: cluster 2024-04-07T17:07:06.301753+0000 mon.a (mon.0) 2719 : cluster [DBG] osdmap e1444: 8 total, 8 up, 8 in 2024-04-07T17:07:08.317 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestMirroring.EnableImageMirror_In_MirrorModeImage 2024-04-07T17:07:08.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:07:08 smithi144 bash[66654]: cluster 2024-04-07T17:07:07.309021+0000 mon.a (mon.0) 2720 : cluster [DBG] osdmap e1445: 8 total, 8 up, 8 in 2024-04-07T17:07:08.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:07:08 smithi144 bash[66654]: audit 2024-04-07T17:07:07.330452+0000 mon.a (mon.0) 2721 : audit [INF] from='client.? 172.21.15.100:0/3748045073' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:07:08.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:07:08 smithi144 bash[66654]: audit 2024-04-07T17:07:08.302613+0000 mon.a (mon.0) 2722 : audit [INF] from='client.? 172.21.15.100:0/3748045073' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T17:07:08.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:07:08 smithi144 bash[66654]: cluster 2024-04-07T17:07:08.311729+0000 mon.a (mon.0) 2723 : cluster [DBG] osdmap e1446: 8 total, 8 up, 8 in 2024-04-07T17:07:08.702 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:07:08 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:07:08] "GET /metrics HTTP/1.1" 200 36494 "" "Prometheus/2.43.0" 2024-04-07T17:07:08.703 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:07:08 smithi100 bash[74942]: cluster 2024-04-07T17:07:07.309021+0000 mon.a (mon.0) 2720 : cluster [DBG] osdmap e1445: 8 total, 8 up, 8 in 2024-04-07T17:07:08.703 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:07:08 smithi100 bash[74942]: audit 2024-04-07T17:07:07.330452+0000 mon.a (mon.0) 2721 : audit [INF] from='client.? 172.21.15.100:0/3748045073' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:07:08.703 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:07:08 smithi100 bash[74942]: audit 2024-04-07T17:07:08.302613+0000 mon.a (mon.0) 2722 : audit [INF] from='client.? 172.21.15.100:0/3748045073' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T17:07:08.703 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:07:08 smithi100 bash[74942]: cluster 2024-04-07T17:07:08.311729+0000 mon.a (mon.0) 2723 : cluster [DBG] osdmap e1446: 8 total, 8 up, 8 in 2024-04-07T17:07:08.703 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:07:08 smithi100 bash[76202]: cluster 2024-04-07T17:07:07.309021+0000 mon.a (mon.0) 2720 : cluster [DBG] osdmap e1445: 8 total, 8 up, 8 in 2024-04-07T17:07:08.703 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:07:08 smithi100 bash[76202]: audit 2024-04-07T17:07:07.330452+0000 mon.a (mon.0) 2721 : audit [INF] from='client.? 172.21.15.100:0/3748045073' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:07:08.703 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:07:08 smithi100 bash[76202]: audit 2024-04-07T17:07:08.302613+0000 mon.a (mon.0) 2722 : audit [INF] from='client.? 172.21.15.100:0/3748045073' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T17:07:08.703 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:07:08 smithi100 bash[76202]: cluster 2024-04-07T17:07:08.311729+0000 mon.a (mon.0) 2723 : cluster [DBG] osdmap e1446: 8 total, 8 up, 8 in 2024-04-07T17:07:09.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:07:09 smithi144 bash[66654]: cluster 2024-04-07T17:07:07.936957+0000 mgr.y (mgr.25175) 1553 : cluster [DBG] pgmap v2573: 137 pgs: 32 unknown, 105 active+clean; 4.6 MiB data, 5.2 GiB used, 710 GiB / 715 GiB avail 2024-04-07T17:07:09.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:07:09 smithi144 bash[66654]: cluster 2024-04-07T17:07:08.326062+0000 mon.a (mon.0) 2724 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T17:07:09.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:07:09 smithi144 bash[66654]: cluster 2024-04-07T17:07:09.312941+0000 mon.a (mon.0) 2725 : cluster [DBG] osdmap e1447: 8 total, 8 up, 8 in 2024-04-07T17:07:09.702 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:07:09 smithi100 bash[74942]: cluster 2024-04-07T17:07:07.936957+0000 mgr.y (mgr.25175) 1553 : cluster [DBG] pgmap v2573: 137 pgs: 32 unknown, 105 active+clean; 4.6 MiB data, 5.2 GiB used, 710 GiB / 715 GiB avail 2024-04-07T17:07:09.703 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:07:09 smithi100 bash[74942]: cluster 2024-04-07T17:07:08.326062+0000 mon.a (mon.0) 2724 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T17:07:09.703 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:07:09 smithi100 bash[74942]: cluster 2024-04-07T17:07:09.312941+0000 mon.a (mon.0) 2725 : cluster [DBG] osdmap e1447: 8 total, 8 up, 8 in 2024-04-07T17:07:09.703 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:07:09 smithi100 bash[76202]: cluster 2024-04-07T17:07:07.936957+0000 mgr.y (mgr.25175) 1553 : cluster [DBG] pgmap v2573: 137 pgs: 32 unknown, 105 active+clean; 4.6 MiB data, 5.2 GiB used, 710 GiB / 715 GiB avail 2024-04-07T17:07:09.703 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:07:09 smithi100 bash[76202]: cluster 2024-04-07T17:07:08.326062+0000 mon.a (mon.0) 2724 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T17:07:09.703 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:07:09 smithi100 bash[76202]: cluster 2024-04-07T17:07:09.312941+0000 mon.a (mon.0) 2725 : cluster [DBG] osdmap e1447: 8 total, 8 up, 8 in 2024-04-07T17:07:10.518 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestMirroring.EnableImageMirror_In_MirrorModeImage (2201 ms) 2024-04-07T17:07:10.519 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestMirroring.EnableImageMirror_In_MirrorModePool 2024-04-07T17:07:10.708 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestMirroring.EnableImageMirror_In_MirrorModePool (189 ms) 2024-04-07T17:07:10.708 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestMirroring.EnableImageMirror_In_MirrorModeDisabled 2024-04-07T17:07:10.898 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestMirroring.EnableImageMirror_In_MirrorModeDisabled (190 ms) 2024-04-07T17:07:10.898 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestMirroring.DisableImageMirror_In_MirrorModeImage 2024-04-07T17:07:11.183 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestMirroring.DisableImageMirror_In_MirrorModeImage (285 ms) 2024-04-07T17:07:11.183 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestMirroring.DisableImageMirror_In_MirrorModeImage_NoObjectMap 2024-04-07T17:07:11.433 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestMirroring.DisableImageMirror_In_MirrorModeImage_NoObjectMap (251 ms) 2024-04-07T17:07:11.434 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestMirroring.DisableImageMirror_In_MirrorModePool 2024-04-07T17:07:11.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:07:11 smithi144 bash[66654]: cluster 2024-04-07T17:07:09.937794+0000 mgr.y (mgr.25175) 1554 : cluster [DBG] pgmap v2576: 137 pgs: 7 creating+peering, 11 unknown, 119 active+clean; 4.6 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail 2024-04-07T17:07:11.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:07:11 smithi144 bash[66654]: cluster 2024-04-07T17:07:10.314548+0000 mon.a (mon.0) 2726 : cluster [DBG] osdmap e1448: 8 total, 8 up, 8 in 2024-04-07T17:07:11.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:07:11 smithi144 bash[66654]: cluster 2024-04-07T17:07:10.335950+0000 mon.a (mon.0) 2727 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T17:07:11.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:07:11 smithi144 bash[66654]: cluster 2024-04-07T17:07:10.336038+0000 mon.a (mon.0) 2728 : cluster [INF] Cluster is now healthy 2024-04-07T17:07:11.627 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestMirroring.DisableImageMirror_In_MirrorModePool (194 ms) 2024-04-07T17:07:11.627 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestMirroring.DisableImageMirror_In_MirrorModeDisabled 2024-04-07T17:07:11.702 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:07:11 smithi100 bash[74942]: cluster 2024-04-07T17:07:09.937794+0000 mgr.y (mgr.25175) 1554 : cluster [DBG] pgmap v2576: 137 pgs: 7 creating+peering, 11 unknown, 119 active+clean; 4.6 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail 2024-04-07T17:07:11.702 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:07:11 smithi100 bash[74942]: cluster 2024-04-07T17:07:10.314548+0000 mon.a (mon.0) 2726 : cluster [DBG] osdmap e1448: 8 total, 8 up, 8 in 2024-04-07T17:07:11.702 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:07:11 smithi100 bash[74942]: cluster 2024-04-07T17:07:10.335950+0000 mon.a (mon.0) 2727 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T17:07:11.702 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:07:11 smithi100 bash[74942]: cluster 2024-04-07T17:07:10.336038+0000 mon.a (mon.0) 2728 : cluster [INF] Cluster is now healthy 2024-04-07T17:07:11.703 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:07:11 smithi100 bash[76202]: cluster 2024-04-07T17:07:09.937794+0000 mgr.y (mgr.25175) 1554 : cluster [DBG] pgmap v2576: 137 pgs: 7 creating+peering, 11 unknown, 119 active+clean; 4.6 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail 2024-04-07T17:07:11.703 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:07:11 smithi100 bash[76202]: cluster 2024-04-07T17:07:10.314548+0000 mon.a (mon.0) 2726 : cluster [DBG] osdmap e1448: 8 total, 8 up, 8 in 2024-04-07T17:07:11.703 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:07:11 smithi100 bash[76202]: cluster 2024-04-07T17:07:10.335950+0000 mon.a (mon.0) 2727 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T17:07:11.703 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:07:11 smithi100 bash[76202]: cluster 2024-04-07T17:07:10.336038+0000 mon.a (mon.0) 2728 : cluster [INF] Cluster is now healthy 2024-04-07T17:07:11.816 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestMirroring.DisableImageMirror_In_MirrorModeDisabled (188 ms) 2024-04-07T17:07:11.816 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestMirroring.DisableImageMirrorWithPeer 2024-04-07T17:07:11.816 INFO:tasks.workunit.client.0.smithi100.stdout:./src/test/librbd/test_mirroring.cc:482: Skipped 2024-04-07T17:07:11.816 INFO:tasks.workunit.client.0.smithi100.stdout:Skipping due to unmet REQUIRE 2024-04-07T17:07:11.816 INFO:tasks.workunit.client.0.smithi100.stdout:[ SKIPPED ] TestMirroring.DisableImageMirrorWithPeer (1 ms) 2024-04-07T17:07:11.816 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestMirroring.DisableJournalingWithPeer 2024-04-07T17:07:11.817 INFO:tasks.workunit.client.0.smithi100.stdout:./src/test/librbd/test_mirroring.cc:520: Skipped 2024-04-07T17:07:11.817 INFO:tasks.workunit.client.0.smithi100.stdout:Skipping due to unmet REQUIRE 2024-04-07T17:07:11.817 INFO:tasks.workunit.client.0.smithi100.stdout:[ SKIPPED ] TestMirroring.DisableJournalingWithPeer (0 ms) 2024-04-07T17:07:11.817 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestMirroring.EnableImageMirror_In_MirrorModeDisabled_WithoutJournaling 2024-04-07T17:07:11.956 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestMirroring.EnableImageMirror_In_MirrorModeDisabled_WithoutJournaling (139 ms) 2024-04-07T17:07:11.956 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestMirroring.EnableImageMirror_In_MirrorModePool_WithoutJournaling 2024-04-07T17:07:12.145 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestMirroring.EnableImageMirror_In_MirrorModePool_WithoutJournaling (190 ms) 2024-04-07T17:07:12.146 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestMirroring.EnableImageMirror_In_MirrorModeImage_WithoutJournaling 2024-04-07T17:07:12.616 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:07:12 smithi144 bash[95233]: ts=2024-04-07T17:07:12.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:07:13.457 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestMirroring.EnableImageMirror_In_MirrorModeImage_WithoutJournaling (1312 ms) 2024-04-07T17:07:13.458 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestMirroring.EnableImageMirror_In_MirrorModeImage_WithoutExclusiveLock 2024-04-07T17:07:13.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:07:13 smithi144 bash[66654]: cluster 2024-04-07T17:07:11.938270+0000 mgr.y (mgr.25175) 1555 : cluster [DBG] pgmap v2578: 137 pgs: 7 creating+peering, 7 unknown, 123 active+clean; 4.6 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 220 B/s wr, 0 op/s 2024-04-07T17:07:13.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:07:13 smithi144 bash[66654]: cluster 2024-04-07T17:07:12.326006+0000 mon.a (mon.0) 2729 : cluster [DBG] osdmap e1449: 8 total, 8 up, 8 in 2024-04-07T17:07:13.702 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:07:13 smithi100 bash[74942]: cluster 2024-04-07T17:07:11.938270+0000 mgr.y (mgr.25175) 1555 : cluster [DBG] pgmap v2578: 137 pgs: 7 creating+peering, 7 unknown, 123 active+clean; 4.6 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 220 B/s wr, 0 op/s 2024-04-07T17:07:13.702 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:07:13 smithi100 bash[74942]: cluster 2024-04-07T17:07:12.326006+0000 mon.a (mon.0) 2729 : cluster [DBG] osdmap e1449: 8 total, 8 up, 8 in 2024-04-07T17:07:13.703 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:07:13 smithi100 bash[76202]: cluster 2024-04-07T17:07:11.938270+0000 mgr.y (mgr.25175) 1555 : cluster [DBG] pgmap v2578: 137 pgs: 7 creating+peering, 7 unknown, 123 active+clean; 4.6 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 220 B/s wr, 0 op/s 2024-04-07T17:07:13.703 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:07:13 smithi100 bash[76202]: cluster 2024-04-07T17:07:12.326006+0000 mon.a (mon.0) 2729 : cluster [DBG] osdmap e1449: 8 total, 8 up, 8 in 2024-04-07T17:07:14.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:07:14 smithi144 bash[66654]: cluster 2024-04-07T17:07:13.342595+0000 mon.a (mon.0) 2730 : cluster [DBG] osdmap e1450: 8 total, 8 up, 8 in 2024-04-07T17:07:14.702 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:07:14 smithi100 bash[74942]: cluster 2024-04-07T17:07:13.342595+0000 mon.a (mon.0) 2730 : cluster [DBG] osdmap e1450: 8 total, 8 up, 8 in 2024-04-07T17:07:14.703 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:07:14 smithi100 bash[76202]: cluster 2024-04-07T17:07:13.342595+0000 mon.a (mon.0) 2730 : cluster [DBG] osdmap e1450: 8 total, 8 up, 8 in 2024-04-07T17:07:15.465 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestMirroring.EnableImageMirror_In_MirrorModeImage_WithoutExclusiveLock (2008 ms) 2024-04-07T17:07:15.465 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestMirroring.CreateImage_In_MirrorModeDisabled 2024-04-07T17:07:15.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:07:15 smithi144 bash[66654]: cluster 2024-04-07T17:07:13.938946+0000 mgr.y (mgr.25175) 1556 : cluster [DBG] pgmap v2581: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 83 KiB/s rd, 19 KiB/s wr, 142 op/s 2024-04-07T17:07:15.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:07:15 smithi144 bash[66654]: cluster 2024-04-07T17:07:14.353685+0000 mon.a (mon.0) 2731 : cluster [DBG] osdmap e1451: 8 total, 8 up, 8 in 2024-04-07T17:07:15.689 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestMirroring.CreateImage_In_MirrorModeDisabled (223 ms) 2024-04-07T17:07:15.689 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestMirroring.CreateImage_In_MirrorModeImage 2024-04-07T17:07:15.702 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:07:15 smithi100 bash[74942]: cluster 2024-04-07T17:07:13.938946+0000 mgr.y (mgr.25175) 1556 : cluster [DBG] pgmap v2581: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 83 KiB/s rd, 19 KiB/s wr, 142 op/s 2024-04-07T17:07:15.702 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:07:15 smithi100 bash[74942]: cluster 2024-04-07T17:07:14.353685+0000 mon.a (mon.0) 2731 : cluster [DBG] osdmap e1451: 8 total, 8 up, 8 in 2024-04-07T17:07:15.703 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:07:15 smithi100 bash[76202]: cluster 2024-04-07T17:07:13.938946+0000 mgr.y (mgr.25175) 1556 : cluster [DBG] pgmap v2581: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 83 KiB/s rd, 19 KiB/s wr, 142 op/s 2024-04-07T17:07:15.703 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:07:15 smithi100 bash[76202]: cluster 2024-04-07T17:07:14.353685+0000 mon.a (mon.0) 2731 : cluster [DBG] osdmap e1451: 8 total, 8 up, 8 in 2024-04-07T17:07:15.949 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestMirroring.CreateImage_In_MirrorModeImage (260 ms) 2024-04-07T17:07:15.949 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestMirroring.CreateImage_In_MirrorModePool 2024-04-07T17:07:16.150 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestMirroring.CreateImage_In_MirrorModePool (202 ms) 2024-04-07T17:07:16.150 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestMirroring.CreateImage_In_MirrorModePool_WithoutJournaling 2024-04-07T17:07:16.310 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestMirroring.CreateImage_In_MirrorModePool_WithoutJournaling (159 ms) 2024-04-07T17:07:16.310 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestMirroring.CreateImage_In_MirrorModeImage_WithoutJournaling 2024-04-07T17:07:16.524 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestMirroring.CreateImage_In_MirrorModeImage_WithoutJournaling (214 ms) 2024-04-07T17:07:16.524 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestMirroring.EnableJournaling_In_MirrorModeDisabled 2024-04-07T17:07:16.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:07:16 smithi144 bash[66654]: cluster 2024-04-07T17:07:15.362215+0000 mon.a (mon.0) 2732 : cluster [DBG] osdmap e1452: 8 total, 8 up, 8 in 2024-04-07T17:07:16.702 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:07:16 smithi100 bash[74942]: cluster 2024-04-07T17:07:15.362215+0000 mon.a (mon.0) 2732 : cluster [DBG] osdmap e1452: 8 total, 8 up, 8 in 2024-04-07T17:07:16.703 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:07:16 smithi100 bash[76202]: cluster 2024-04-07T17:07:15.362215+0000 mon.a (mon.0) 2732 : cluster [DBG] osdmap e1452: 8 total, 8 up, 8 in 2024-04-07T17:07:16.788 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestMirroring.EnableJournaling_In_MirrorModeDisabled (265 ms) 2024-04-07T17:07:16.788 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestMirroring.EnableJournaling_In_MirrorModeImage 2024-04-07T17:07:17.084 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestMirroring.EnableJournaling_In_MirrorModeImage (295 ms) 2024-04-07T17:07:17.084 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestMirroring.EnableJournaling_In_MirrorModeImage_SnapshotMirroringEnabled 2024-04-07T17:07:17.702 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:07:17 smithi100 bash[74942]: cluster 2024-04-07T17:07:15.939621+0000 mgr.y (mgr.25175) 1557 : cluster [DBG] pgmap v2584: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 409 KiB/s rd, 53 KiB/s wr, 589 op/s 2024-04-07T17:07:17.703 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:07:17 smithi100 bash[74942]: audit 2024-04-07T17:07:16.809022+0000 mon.a (mon.0) 2733 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:07:17.703 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:07:17 smithi100 bash[76202]: cluster 2024-04-07T17:07:15.939621+0000 mgr.y (mgr.25175) 1557 : cluster [DBG] pgmap v2584: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 409 KiB/s rd, 53 KiB/s wr, 589 op/s 2024-04-07T17:07:17.703 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:07:17 smithi100 bash[76202]: audit 2024-04-07T17:07:16.809022+0000 mon.a (mon.0) 2733 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:07:17.803 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestMirroring.EnableJournaling_In_MirrorModeImage_SnapshotMirroringEnabled (720 ms) 2024-04-07T17:07:17.804 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestMirroring.EnableJournaling_In_MirrorModePool 2024-04-07T17:07:17.866 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:07:17 smithi144 bash[66654]: cluster 2024-04-07T17:07:15.939621+0000 mgr.y (mgr.25175) 1557 : cluster [DBG] pgmap v2584: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 409 KiB/s rd, 53 KiB/s wr, 589 op/s 2024-04-07T17:07:17.866 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:07:17 smithi144 bash[66654]: audit 2024-04-07T17:07:16.809022+0000 mon.a (mon.0) 2733 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:07:18.040 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestMirroring.EnableJournaling_In_MirrorModePool (237 ms) 2024-04-07T17:07:18.041 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestMirroring.DisableJournaling_In_MirrorModePool 2024-04-07T17:07:18.290 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestMirroring.DisableJournaling_In_MirrorModePool (250 ms) 2024-04-07T17:07:18.291 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestMirroring.DisableJournaling_In_MirrorModeImage 2024-04-07T17:07:18.540 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestMirroring.DisableJournaling_In_MirrorModeImage (250 ms) 2024-04-07T17:07:18.541 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestMirroring.MirrorModeSet_DisabledMode_To_PoolMode 2024-04-07T17:07:18.702 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:07:18 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:07:18] "GET /metrics HTTP/1.1" 200 36465 "" "Prometheus/2.43.0" 2024-04-07T17:07:18.702 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:07:18 smithi100 bash[74942]: cluster 2024-04-07T17:07:17.371185+0000 mon.a (mon.0) 2734 : cluster [DBG] osdmap e1453: 8 total, 8 up, 8 in 2024-04-07T17:07:18.703 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:07:18 smithi100 bash[74942]: cluster 2024-04-07T17:07:17.649186+0000 mon.a (mon.0) 2735 : cluster [DBG] osdmap e1454: 8 total, 8 up, 8 in 2024-04-07T17:07:18.703 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:07:18 smithi100 bash[76202]: cluster 2024-04-07T17:07:17.371185+0000 mon.a (mon.0) 2734 : cluster [DBG] osdmap e1453: 8 total, 8 up, 8 in 2024-04-07T17:07:18.703 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:07:18 smithi100 bash[76202]: cluster 2024-04-07T17:07:17.649186+0000 mon.a (mon.0) 2735 : cluster [DBG] osdmap e1454: 8 total, 8 up, 8 in 2024-04-07T17:07:18.849 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestMirroring.MirrorModeSet_DisabledMode_To_PoolMode (308 ms) 2024-04-07T17:07:18.849 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestMirroring.MirrorModeSet_PoolMode_To_DisabledMode 2024-04-07T17:07:18.866 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:07:18 smithi144 bash[66654]: cluster 2024-04-07T17:07:17.371185+0000 mon.a (mon.0) 2734 : cluster [DBG] osdmap e1453: 8 total, 8 up, 8 in 2024-04-07T17:07:18.866 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:07:18 smithi144 bash[66654]: cluster 2024-04-07T17:07:17.649186+0000 mon.a (mon.0) 2735 : cluster [DBG] osdmap e1454: 8 total, 8 up, 8 in 2024-04-07T17:07:19.121 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestMirroring.MirrorModeSet_PoolMode_To_DisabledMode (273 ms) 2024-04-07T17:07:19.122 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestMirroring.MirrorModeSet_DisabledMode_To_ImageMode 2024-04-07T17:07:19.466 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestMirroring.MirrorModeSet_DisabledMode_To_ImageMode (344 ms) 2024-04-07T17:07:19.466 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestMirroring.MirrorModeSet_PoolMode_To_ImageMode 2024-04-07T17:07:19.870 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestMirroring.MirrorModeSet_PoolMode_To_ImageMode (404 ms) 2024-04-07T17:07:19.870 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestMirroring.MirrorModeSet_ImageMode_To_PoolMode 2024-04-07T17:07:19.952 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:07:19 smithi100 bash[74942]: cluster 2024-04-07T17:07:17.940554+0000 mgr.y (mgr.25175) 1558 : cluster [DBG] pgmap v2587: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 425 KiB/s rd, 51 KiB/s wr, 597 op/s 2024-04-07T17:07:19.952 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:07:19 smithi100 bash[76202]: cluster 2024-04-07T17:07:17.940554+0000 mgr.y (mgr.25175) 1558 : cluster [DBG] pgmap v2587: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 425 KiB/s rd, 51 KiB/s wr, 597 op/s 2024-04-07T17:07:20.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:07:19 smithi144 bash[66654]: cluster 2024-04-07T17:07:17.940554+0000 mgr.y (mgr.25175) 1558 : cluster [DBG] pgmap v2587: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 425 KiB/s rd, 51 KiB/s wr, 597 op/s 2024-04-07T17:07:20.195 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestMirroring.MirrorModeSet_ImageMode_To_PoolMode (326 ms) 2024-04-07T17:07:20.195 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestMirroring.MirrorModeSet_ImageMode_To_DisabledMode 2024-04-07T17:07:20.523 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestMirroring.MirrorModeSet_ImageMode_To_DisabledMode (327 ms) 2024-04-07T17:07:20.523 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestMirroring.RemoveImage_With_MirrorImageEnabled 2024-04-07T17:07:20.665 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestMirroring.RemoveImage_With_MirrorImageEnabled (142 ms) 2024-04-07T17:07:20.665 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestMirroring.RemoveImage_With_MirrorImageDisabled 2024-04-07T17:07:20.846 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestMirroring.RemoveImage_With_MirrorImageDisabled (181 ms) 2024-04-07T17:07:20.846 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestMirroring.RemoveImage_With_ImageWithoutJournal 2024-04-07T17:07:21.029 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestMirroring.RemoveImage_With_ImageWithoutJournal (183 ms) 2024-04-07T17:07:21.029 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestMirroring.RemoveImage_With_MirrorImageDemoted 2024-04-07T17:07:21.254 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestMirroring.RemoveImage_With_MirrorImageDemoted (225 ms) 2024-04-07T17:07:21.254 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestMirroring.TrashMoveRestore_PoolMode 2024-04-07T17:07:21.441 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestMirroring.TrashMoveRestore_PoolMode (187 ms) 2024-04-07T17:07:21.441 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestMirroring.TrashMoveRestore_ImageMode_MirroringDisabled 2024-04-07T17:07:21.640 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestMirroring.TrashMoveRestore_ImageMode_MirroringDisabled (199 ms) 2024-04-07T17:07:21.640 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestMirroring.TrashMoveRestore_ImageMode_MirroringEnabled 2024-04-07T17:07:21.913 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestMirroring.TrashMoveRestore_ImageMode_MirroringEnabled (274 ms) 2024-04-07T17:07:21.913 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestMirroring.MirrorStatusList 2024-04-07T17:07:21.952 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:07:21 smithi100 bash[74942]: cluster 2024-04-07T17:07:19.941355+0000 mgr.y (mgr.25175) 1559 : cluster [DBG] pgmap v2588: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 402 KiB/s rd, 51 KiB/s wr, 584 op/s 2024-04-07T17:07:21.953 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:07:21 smithi100 bash[76202]: cluster 2024-04-07T17:07:19.941355+0000 mgr.y (mgr.25175) 1559 : cluster [DBG] pgmap v2588: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 402 KiB/s rd, 51 KiB/s wr, 584 op/s 2024-04-07T17:07:22.078 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestMirroring.MirrorStatusList (165 ms) 2024-04-07T17:07:22.078 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestMirroring.RemoveBootstrapped 2024-04-07T17:07:22.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:07:21 smithi144 bash[66654]: cluster 2024-04-07T17:07:19.941355+0000 mgr.y (mgr.25175) 1559 : cluster [DBG] pgmap v2588: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 402 KiB/s rd, 51 KiB/s wr, 584 op/s 2024-04-07T17:07:22.227 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestMirroring.RemoveBootstrapped (148 ms) 2024-04-07T17:07:22.227 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestMirroring.AioPromoteDemote 2024-04-07T17:07:22.616 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:07:22 smithi144 bash[95233]: ts=2024-04-07T17:07:22.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:07:22.832 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestMirroring.AioPromoteDemote (606 ms) 2024-04-07T17:07:22.832 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestMirroring.AioGetInfo 2024-04-07T17:07:23.508 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestMirroring.AioGetInfo (675 ms) 2024-04-07T17:07:23.508 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestMirroring.AioGetStatus 2024-04-07T17:07:23.918 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestMirroring.AioGetStatus (411 ms) 2024-04-07T17:07:23.919 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestMirroring.SiteName 2024-04-07T17:07:23.929 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestMirroring.SiteName (10 ms) 2024-04-07T17:07:23.929 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestMirroring.Bootstrap 2024-04-07T17:07:23.952 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:07:23 smithi100 bash[74942]: cluster 2024-04-07T17:07:21.942313+0000 mgr.y (mgr.25175) 1560 : cluster [DBG] pgmap v2589: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 332 KiB/s rd, 44 KiB/s wr, 483 op/s 2024-04-07T17:07:23.952 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:07:23 smithi100 bash[74942]: cluster 2024-04-07T17:07:22.647363+0000 mon.a (mon.0) 2736 : cluster [DBG] osdmap e1455: 8 total, 8 up, 8 in 2024-04-07T17:07:23.952 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:07:23 smithi100 bash[76202]: cluster 2024-04-07T17:07:21.942313+0000 mgr.y (mgr.25175) 1560 : cluster [DBG] pgmap v2589: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 332 KiB/s rd, 44 KiB/s wr, 483 op/s 2024-04-07T17:07:23.953 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:07:23 smithi100 bash[76202]: cluster 2024-04-07T17:07:22.647363+0000 mon.a (mon.0) 2736 : cluster [DBG] osdmap e1455: 8 total, 8 up, 8 in 2024-04-07T17:07:24.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:07:23 smithi144 bash[66654]: cluster 2024-04-07T17:07:21.942313+0000 mgr.y (mgr.25175) 1560 : cluster [DBG] pgmap v2589: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 332 KiB/s rd, 44 KiB/s wr, 483 op/s 2024-04-07T17:07:24.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:07:23 smithi144 bash[66654]: cluster 2024-04-07T17:07:22.647363+0000 mon.a (mon.0) 2736 : cluster [DBG] osdmap e1455: 8 total, 8 up, 8 in 2024-04-07T17:07:25.202 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:07:24 smithi100 bash[74942]: audit 2024-04-07T17:07:23.923934+0000 mon.a (mon.0) 2737 : audit [INF] from='client.? 172.21.15.100:0/3748045073' entity='client.admin' 2024-04-07T17:07:25.202 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:07:24 smithi100 bash[74942]: audit 2024-04-07T17:07:23.924521+0000 mon.a (mon.0) 2738 : audit [DBG] from='client.? 172.21.15.100:0/3748045073' entity='client.admin' cmd=[{"prefix": "config-key get", "key": "rbd/mirror/site_name"}]: dispatch 2024-04-07T17:07:25.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:07:24 smithi100 bash[74942]: audit 2024-04-07T17:07:23.924929+0000 mon.a (mon.0) 2739 : audit [INF] from='client.? 172.21.15.100:0/3748045073' entity='client.admin' cmd=[{"prefix": "config-key rm", "key": "rbd/mirror/site_name"}]: dispatch 2024-04-07T17:07:25.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:07:24 smithi100 bash[74942]: audit 2024-04-07T17:07:23.929209+0000 mon.a (mon.0) 2740 : audit [INF] from='client.? 172.21.15.100:0/3748045073' entity='client.admin' cmd='[{"prefix": "config-key rm", "key": "rbd/mirror/site_name"}]': finished 2024-04-07T17:07:25.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:07:24 smithi100 bash[74942]: audit 2024-04-07T17:07:23.929662+0000 mon.a (mon.0) 2741 : audit [DBG] from='client.? 172.21.15.100:0/3748045073' entity='client.admin' cmd=[{"prefix": "config-key get", "key": "rbd/mirror/site_name"}]: dispatch 2024-04-07T17:07:25.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:07:24 smithi100 bash[74942]: cluster 2024-04-07T17:07:23.943006+0000 mgr.y (mgr.25175) 1561 : cluster [DBG] pgmap v2591: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 583 KiB/s rd, 67 KiB/s wr, 819 op/s 2024-04-07T17:07:25.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:07:24 smithi100 bash[76202]: audit 2024-04-07T17:07:23.923934+0000 mon.a (mon.0) 2737 : audit [INF] from='client.? 172.21.15.100:0/3748045073' entity='client.admin' 2024-04-07T17:07:25.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:07:24 smithi100 bash[76202]: audit 2024-04-07T17:07:23.924521+0000 mon.a (mon.0) 2738 : audit [DBG] from='client.? 172.21.15.100:0/3748045073' entity='client.admin' cmd=[{"prefix": "config-key get", "key": "rbd/mirror/site_name"}]: dispatch 2024-04-07T17:07:25.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:07:24 smithi100 bash[76202]: audit 2024-04-07T17:07:23.924929+0000 mon.a (mon.0) 2739 : audit [INF] from='client.? 172.21.15.100:0/3748045073' entity='client.admin' cmd=[{"prefix": "config-key rm", "key": "rbd/mirror/site_name"}]: dispatch 2024-04-07T17:07:25.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:07:24 smithi100 bash[76202]: audit 2024-04-07T17:07:23.929209+0000 mon.a (mon.0) 2740 : audit [INF] from='client.? 172.21.15.100:0/3748045073' entity='client.admin' cmd='[{"prefix": "config-key rm", "key": "rbd/mirror/site_name"}]': finished 2024-04-07T17:07:25.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:07:24 smithi100 bash[76202]: audit 2024-04-07T17:07:23.929662+0000 mon.a (mon.0) 2741 : audit [DBG] from='client.? 172.21.15.100:0/3748045073' entity='client.admin' cmd=[{"prefix": "config-key get", "key": "rbd/mirror/site_name"}]: dispatch 2024-04-07T17:07:25.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:07:24 smithi100 bash[76202]: cluster 2024-04-07T17:07:23.943006+0000 mgr.y (mgr.25175) 1561 : cluster [DBG] pgmap v2591: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 583 KiB/s rd, 67 KiB/s wr, 819 op/s 2024-04-07T17:07:25.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:07:24 smithi144 bash[66654]: audit 2024-04-07T17:07:23.923934+0000 mon.a (mon.0) 2737 : audit [INF] from='client.? 172.21.15.100:0/3748045073' entity='client.admin' 2024-04-07T17:07:25.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:07:24 smithi144 bash[66654]: audit 2024-04-07T17:07:23.924521+0000 mon.a (mon.0) 2738 : audit [DBG] from='client.? 172.21.15.100:0/3748045073' entity='client.admin' cmd=[{"prefix": "config-key get", "key": "rbd/mirror/site_name"}]: dispatch 2024-04-07T17:07:25.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:07:24 smithi144 bash[66654]: audit 2024-04-07T17:07:23.924929+0000 mon.a (mon.0) 2739 : audit [INF] from='client.? 172.21.15.100:0/3748045073' entity='client.admin' cmd=[{"prefix": "config-key rm", "key": "rbd/mirror/site_name"}]: dispatch 2024-04-07T17:07:25.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:07:24 smithi144 bash[66654]: audit 2024-04-07T17:07:23.929209+0000 mon.a (mon.0) 2740 : audit [INF] from='client.? 172.21.15.100:0/3748045073' entity='client.admin' cmd='[{"prefix": "config-key rm", "key": "rbd/mirror/site_name"}]': finished 2024-04-07T17:07:25.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:07:24 smithi144 bash[66654]: audit 2024-04-07T17:07:23.929662+0000 mon.a (mon.0) 2741 : audit [DBG] from='client.? 172.21.15.100:0/3748045073' entity='client.admin' cmd=[{"prefix": "config-key get", "key": "rbd/mirror/site_name"}]: dispatch 2024-04-07T17:07:25.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:07:24 smithi144 bash[66654]: cluster 2024-04-07T17:07:23.943006+0000 mgr.y (mgr.25175) 1561 : cluster [DBG] pgmap v2591: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 583 KiB/s rd, 67 KiB/s wr, 819 op/s 2024-04-07T17:07:25.474 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestMirroring.Bootstrap (1546 ms) 2024-04-07T17:07:25.475 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestMirroring.PeerDirection 2024-04-07T17:07:25.487 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestMirroring.PeerDirection (13 ms) 2024-04-07T17:07:25.487 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestMirroring.Snapshot 2024-04-07T17:07:26.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:07:25 smithi100 bash[74942]: audit 2024-04-07T17:07:25.455899+0000 mon.a (mon.0) 2742 : audit [DBG] from='client.? 172.21.15.100:0/3748045073' entity='client.admin' cmd=[{"prefix": "config-key get", "key": "rbd/mirror/peer_client_id"}]: dispatch 2024-04-07T17:07:26.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:07:25 smithi100 bash[74942]: audit 2024-04-07T17:07:25.459670+0000 mon.a (mon.0) 2743 : audit [INF] from='client.? 172.21.15.100:0/3748045073' entity='client.admin' 2024-04-07T17:07:26.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:07:25 smithi100 bash[74942]: audit 2024-04-07T17:07:25.460063+0000 mon.a (mon.0) 2744 : audit [INF] from='client.? 172.21.15.100:0/3748045073' entity='client.admin' cmd=[{ "prefix": "auth get-or-create", "entity": "client.rbd-mirror-peer", "caps": [ "mon", "profile rbd-mirror-peer", "osd", "profile rbd"], "format": "json"}]: dispatch 2024-04-07T17:07:26.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:07:25 smithi100 bash[74942]: audit 2024-04-07T17:07:25.463891+0000 mon.a (mon.0) 2745 : audit [INF] from='client.? 172.21.15.100:0/3748045073' entity='client.admin' cmd='[{ "prefix": "auth get-or-create", "entity": "client.rbd-mirror-peer", "caps": [ "mon", "profile rbd-mirror-peer", "osd", "profile rbd"], "format": "json"}]': finished 2024-04-07T17:07:26.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:07:25 smithi100 bash[74942]: audit 2024-04-07T17:07:25.465912+0000 mon.a (mon.0) 2746 : audit [DBG] from='client.? 172.21.15.100:0/3748045073' entity='client.admin' cmd=[{"prefix": "config-key get", "key": "rbd/mirror/site_name"}]: dispatch 2024-04-07T17:07:26.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:07:25 smithi100 bash[74942]: audit 2024-04-07T17:07:25.480745+0000 mon.a (mon.0) 2747 : audit [INF] from='client.? 172.21.15.100:0/3748045073' entity='client.admin' cmd=[{"prefix": "config-key rm", "key": "rbd/mirror/peer/88/0a726471-ce1f-442a-9f2d-d0d88e5bcabf"}]: dispatch 2024-04-07T17:07:26.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:07:25 smithi100 bash[74942]: audit 2024-04-07T17:07:25.485627+0000 mon.a (mon.0) 2748 : audit [INF] from='client.? 172.21.15.100:0/3748045073' entity='client.admin' cmd='[{"prefix": "config-key rm", "key": "rbd/mirror/peer/88/0a726471-ce1f-442a-9f2d-d0d88e5bcabf"}]': finished 2024-04-07T17:07:26.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:07:25 smithi100 bash[76202]: audit 2024-04-07T17:07:25.455899+0000 mon.a (mon.0) 2742 : audit [DBG] from='client.? 172.21.15.100:0/3748045073' entity='client.admin' cmd=[{"prefix": "config-key get", "key": "rbd/mirror/peer_client_id"}]: dispatch 2024-04-07T17:07:26.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:07:25 smithi100 bash[76202]: audit 2024-04-07T17:07:25.459670+0000 mon.a (mon.0) 2743 : audit [INF] from='client.? 172.21.15.100:0/3748045073' entity='client.admin' 2024-04-07T17:07:26.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:07:25 smithi100 bash[76202]: audit 2024-04-07T17:07:25.460063+0000 mon.a (mon.0) 2744 : audit [INF] from='client.? 172.21.15.100:0/3748045073' entity='client.admin' cmd=[{ "prefix": "auth get-or-create", "entity": "client.rbd-mirror-peer", "caps": [ "mon", "profile rbd-mirror-peer", "osd", "profile rbd"], "format": "json"}]: dispatch 2024-04-07T17:07:26.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:07:25 smithi100 bash[76202]: audit 2024-04-07T17:07:25.463891+0000 mon.a (mon.0) 2745 : audit [INF] from='client.? 172.21.15.100:0/3748045073' entity='client.admin' cmd='[{ "prefix": "auth get-or-create", "entity": "client.rbd-mirror-peer", "caps": [ "mon", "profile rbd-mirror-peer", "osd", "profile rbd"], "format": "json"}]': finished 2024-04-07T17:07:26.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:07:25 smithi100 bash[76202]: audit 2024-04-07T17:07:25.465912+0000 mon.a (mon.0) 2746 : audit [DBG] from='client.? 172.21.15.100:0/3748045073' entity='client.admin' cmd=[{"prefix": "config-key get", "key": "rbd/mirror/site_name"}]: dispatch 2024-04-07T17:07:26.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:07:25 smithi100 bash[76202]: audit 2024-04-07T17:07:25.480745+0000 mon.a (mon.0) 2747 : audit [INF] from='client.? 172.21.15.100:0/3748045073' entity='client.admin' cmd=[{"prefix": "config-key rm", "key": "rbd/mirror/peer/88/0a726471-ce1f-442a-9f2d-d0d88e5bcabf"}]: dispatch 2024-04-07T17:07:26.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:07:25 smithi100 bash[76202]: audit 2024-04-07T17:07:25.485627+0000 mon.a (mon.0) 2748 : audit [INF] from='client.? 172.21.15.100:0/3748045073' entity='client.admin' cmd='[{"prefix": "config-key rm", "key": "rbd/mirror/peer/88/0a726471-ce1f-442a-9f2d-d0d88e5bcabf"}]': finished 2024-04-07T17:07:26.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:07:25 smithi144 bash[66654]: audit 2024-04-07T17:07:25.455899+0000 mon.a (mon.0) 2742 : audit [DBG] from='client.? 172.21.15.100:0/3748045073' entity='client.admin' cmd=[{"prefix": "config-key get", "key": "rbd/mirror/peer_client_id"}]: dispatch 2024-04-07T17:07:26.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:07:25 smithi144 bash[66654]: audit 2024-04-07T17:07:25.459670+0000 mon.a (mon.0) 2743 : audit [INF] from='client.? 172.21.15.100:0/3748045073' entity='client.admin' 2024-04-07T17:07:26.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:07:25 smithi144 bash[66654]: audit 2024-04-07T17:07:25.460063+0000 mon.a (mon.0) 2744 : audit [INF] from='client.? 172.21.15.100:0/3748045073' entity='client.admin' cmd=[{ "prefix": "auth get-or-create", "entity": "client.rbd-mirror-peer", "caps": [ "mon", "profile rbd-mirror-peer", "osd", "profile rbd"], "format": "json"}]: dispatch 2024-04-07T17:07:26.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:07:25 smithi144 bash[66654]: audit 2024-04-07T17:07:25.463891+0000 mon.a (mon.0) 2745 : audit [INF] from='client.? 172.21.15.100:0/3748045073' entity='client.admin' cmd='[{ "prefix": "auth get-or-create", "entity": "client.rbd-mirror-peer", "caps": [ "mon", "profile rbd-mirror-peer", "osd", "profile rbd"], "format": "json"}]': finished 2024-04-07T17:07:26.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:07:25 smithi144 bash[66654]: audit 2024-04-07T17:07:25.465912+0000 mon.a (mon.0) 2746 : audit [DBG] from='client.? 172.21.15.100:0/3748045073' entity='client.admin' cmd=[{"prefix": "config-key get", "key": "rbd/mirror/site_name"}]: dispatch 2024-04-07T17:07:26.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:07:25 smithi144 bash[66654]: audit 2024-04-07T17:07:25.480745+0000 mon.a (mon.0) 2747 : audit [INF] from='client.? 172.21.15.100:0/3748045073' entity='client.admin' cmd=[{"prefix": "config-key rm", "key": "rbd/mirror/peer/88/0a726471-ce1f-442a-9f2d-d0d88e5bcabf"}]: dispatch 2024-04-07T17:07:26.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:07:25 smithi144 bash[66654]: audit 2024-04-07T17:07:25.485627+0000 mon.a (mon.0) 2748 : audit [INF] from='client.? 172.21.15.100:0/3748045073' entity='client.admin' cmd='[{"prefix": "config-key rm", "key": "rbd/mirror/peer/88/0a726471-ce1f-442a-9f2d-d0d88e5bcabf"}]': finished 2024-04-07T17:07:27.866 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:07:27 smithi144 bash[66654]: cluster 2024-04-07T17:07:25.944081+0000 mgr.y (mgr.25175) 1562 : cluster [DBG] pgmap v2592: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 892 KiB/s rd, 131 KiB/s wr, 1.23k op/s 2024-04-07T17:07:27.866 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:07:27 smithi144 bash[66654]: cluster 2024-04-07T17:07:26.495376+0000 mon.a (mon.0) 2749 : cluster [DBG] osdmap e1456: 8 total, 8 up, 8 in 2024-04-07T17:07:27.952 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:07:27 smithi100 bash[74942]: cluster 2024-04-07T17:07:25.944081+0000 mgr.y (mgr.25175) 1562 : cluster [DBG] pgmap v2592: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 892 KiB/s rd, 131 KiB/s wr, 1.23k op/s 2024-04-07T17:07:27.952 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:07:27 smithi100 bash[74942]: cluster 2024-04-07T17:07:26.495376+0000 mon.a (mon.0) 2749 : cluster [DBG] osdmap e1456: 8 total, 8 up, 8 in 2024-04-07T17:07:27.953 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:07:27 smithi100 bash[76202]: cluster 2024-04-07T17:07:25.944081+0000 mgr.y (mgr.25175) 1562 : cluster [DBG] pgmap v2592: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 892 KiB/s rd, 131 KiB/s wr, 1.23k op/s 2024-04-07T17:07:27.953 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:07:27 smithi100 bash[76202]: cluster 2024-04-07T17:07:26.495376+0000 mon.a (mon.0) 2749 : cluster [DBG] osdmap e1456: 8 total, 8 up, 8 in 2024-04-07T17:07:28.702 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:07:28 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:07:28] "GET /metrics HTTP/1.1" 200 36465 "" "Prometheus/2.43.0" 2024-04-07T17:07:28.703 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:07:28 smithi100 bash[74942]: cluster 2024-04-07T17:07:27.503520+0000 mon.a (mon.0) 2750 : cluster [DBG] osdmap e1457: 8 total, 8 up, 8 in 2024-04-07T17:07:28.703 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:07:28 smithi100 bash[74942]: cluster 2024-04-07T17:07:27.650202+0000 mon.a (mon.0) 2751 : cluster [DBG] osdmap e1458: 8 total, 8 up, 8 in 2024-04-07T17:07:28.703 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:07:28 smithi100 bash[76202]: cluster 2024-04-07T17:07:27.503520+0000 mon.a (mon.0) 2750 : cluster [DBG] osdmap e1457: 8 total, 8 up, 8 in 2024-04-07T17:07:28.703 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:07:28 smithi100 bash[76202]: cluster 2024-04-07T17:07:27.650202+0000 mon.a (mon.0) 2751 : cluster [DBG] osdmap e1458: 8 total, 8 up, 8 in 2024-04-07T17:07:28.866 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:07:28 smithi144 bash[66654]: cluster 2024-04-07T17:07:27.503520+0000 mon.a (mon.0) 2750 : cluster [DBG] osdmap e1457: 8 total, 8 up, 8 in 2024-04-07T17:07:28.866 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:07:28 smithi144 bash[66654]: cluster 2024-04-07T17:07:27.650202+0000 mon.a (mon.0) 2751 : cluster [DBG] osdmap e1458: 8 total, 8 up, 8 in 2024-04-07T17:07:29.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:07:29 smithi100 bash[74942]: cluster 2024-04-07T17:07:27.944985+0000 mgr.y (mgr.25175) 1563 : cluster [DBG] pgmap v2596: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 1.0 MiB/s rd, 173 KiB/s wr, 1.43k op/s 2024-04-07T17:07:29.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:07:29 smithi100 bash[74942]: cluster 2024-04-07T17:07:28.658898+0000 mon.a (mon.0) 2752 : cluster [DBG] osdmap e1459: 8 total, 8 up, 8 in 2024-04-07T17:07:29.953 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:07:29 smithi100 bash[76202]: cluster 2024-04-07T17:07:27.944985+0000 mgr.y (mgr.25175) 1563 : cluster [DBG] pgmap v2596: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 1.0 MiB/s rd, 173 KiB/s wr, 1.43k op/s 2024-04-07T17:07:29.953 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:07:29 smithi100 bash[76202]: cluster 2024-04-07T17:07:28.658898+0000 mon.a (mon.0) 2752 : cluster [DBG] osdmap e1459: 8 total, 8 up, 8 in 2024-04-07T17:07:30.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:07:29 smithi144 bash[66654]: cluster 2024-04-07T17:07:27.944985+0000 mgr.y (mgr.25175) 1563 : cluster [DBG] pgmap v2596: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 1.0 MiB/s rd, 173 KiB/s wr, 1.43k op/s 2024-04-07T17:07:30.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:07:29 smithi144 bash[66654]: cluster 2024-04-07T17:07:28.658898+0000 mon.a (mon.0) 2752 : cluster [DBG] osdmap e1459: 8 total, 8 up, 8 in 2024-04-07T17:07:30.952 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:07:30 smithi100 bash[74942]: cluster 2024-04-07T17:07:29.669168+0000 mon.a (mon.0) 2753 : cluster [DBG] osdmap e1460: 8 total, 8 up, 8 in 2024-04-07T17:07:30.953 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:07:30 smithi100 bash[76202]: cluster 2024-04-07T17:07:29.669168+0000 mon.a (mon.0) 2753 : cluster [DBG] osdmap e1460: 8 total, 8 up, 8 in 2024-04-07T17:07:31.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:07:30 smithi144 bash[66654]: cluster 2024-04-07T17:07:29.669168+0000 mon.a (mon.0) 2753 : cluster [DBG] osdmap e1460: 8 total, 8 up, 8 in 2024-04-07T17:07:31.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:07:31 smithi100 bash[74942]: cluster 2024-04-07T17:07:29.945805+0000 mgr.y (mgr.25175) 1564 : cluster [DBG] pgmap v2599: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 594 KiB/s rd, 85 KiB/s wr, 821 op/s 2024-04-07T17:07:31.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:07:31 smithi100 bash[74942]: cluster 2024-04-07T17:07:30.678794+0000 mon.a (mon.0) 2754 : cluster [DBG] osdmap e1461: 8 total, 8 up, 8 in 2024-04-07T17:07:31.953 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:07:31 smithi100 bash[76202]: cluster 2024-04-07T17:07:29.945805+0000 mgr.y (mgr.25175) 1564 : cluster [DBG] pgmap v2599: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 594 KiB/s rd, 85 KiB/s wr, 821 op/s 2024-04-07T17:07:31.953 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:07:31 smithi100 bash[76202]: cluster 2024-04-07T17:07:30.678794+0000 mon.a (mon.0) 2754 : cluster [DBG] osdmap e1461: 8 total, 8 up, 8 in 2024-04-07T17:07:32.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:07:31 smithi144 bash[66654]: cluster 2024-04-07T17:07:29.945805+0000 mgr.y (mgr.25175) 1564 : cluster [DBG] pgmap v2599: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 594 KiB/s rd, 85 KiB/s wr, 821 op/s 2024-04-07T17:07:32.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:07:31 smithi144 bash[66654]: cluster 2024-04-07T17:07:30.678794+0000 mon.a (mon.0) 2754 : cluster [DBG] osdmap e1461: 8 total, 8 up, 8 in 2024-04-07T17:07:32.616 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:07:32 smithi144 bash[95233]: ts=2024-04-07T17:07:32.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:07:32.952 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:07:32 smithi100 bash[74942]: cluster 2024-04-07T17:07:31.690402+0000 mon.a (mon.0) 2755 : cluster [DBG] osdmap e1462: 8 total, 8 up, 8 in 2024-04-07T17:07:32.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:07:32 smithi100 bash[74942]: audit 2024-04-07T17:07:31.809255+0000 mon.a (mon.0) 2756 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:07:32.953 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:07:32 smithi100 bash[76202]: cluster 2024-04-07T17:07:31.690402+0000 mon.a (mon.0) 2755 : cluster [DBG] osdmap e1462: 8 total, 8 up, 8 in 2024-04-07T17:07:32.953 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:07:32 smithi100 bash[76202]: audit 2024-04-07T17:07:31.809255+0000 mon.a (mon.0) 2756 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:07:33.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:07:32 smithi144 bash[66654]: cluster 2024-04-07T17:07:31.690402+0000 mon.a (mon.0) 2755 : cluster [DBG] osdmap e1462: 8 total, 8 up, 8 in 2024-04-07T17:07:33.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:07:32 smithi144 bash[66654]: audit 2024-04-07T17:07:31.809255+0000 mon.a (mon.0) 2756 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:07:33.952 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:07:33 smithi100 bash[74942]: cluster 2024-04-07T17:07:31.946506+0000 mgr.y (mgr.25175) 1565 : cluster [DBG] pgmap v2602: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 242 KiB/s rd, 26 KiB/s wr, 337 op/s 2024-04-07T17:07:33.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:07:33 smithi100 bash[74942]: cluster 2024-04-07T17:07:32.702377+0000 mon.a (mon.0) 2757 : cluster [DBG] osdmap e1463: 8 total, 8 up, 8 in 2024-04-07T17:07:33.953 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:07:33 smithi100 bash[76202]: cluster 2024-04-07T17:07:31.946506+0000 mgr.y (mgr.25175) 1565 : cluster [DBG] pgmap v2602: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 242 KiB/s rd, 26 KiB/s wr, 337 op/s 2024-04-07T17:07:33.953 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:07:33 smithi100 bash[76202]: cluster 2024-04-07T17:07:32.702377+0000 mon.a (mon.0) 2757 : cluster [DBG] osdmap e1463: 8 total, 8 up, 8 in 2024-04-07T17:07:34.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:07:33 smithi144 bash[66654]: cluster 2024-04-07T17:07:31.946506+0000 mgr.y (mgr.25175) 1565 : cluster [DBG] pgmap v2602: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 242 KiB/s rd, 26 KiB/s wr, 337 op/s 2024-04-07T17:07:34.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:07:33 smithi144 bash[66654]: cluster 2024-04-07T17:07:32.702377+0000 mon.a (mon.0) 2757 : cluster [DBG] osdmap e1463: 8 total, 8 up, 8 in 2024-04-07T17:07:35.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:07:34 smithi144 bash[66654]: cluster 2024-04-07T17:07:33.715992+0000 mon.a (mon.0) 2758 : cluster [DBG] osdmap e1464: 8 total, 8 up, 8 in 2024-04-07T17:07:35.202 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:07:34 smithi100 bash[74942]: cluster 2024-04-07T17:07:33.715992+0000 mon.a (mon.0) 2758 : cluster [DBG] osdmap e1464: 8 total, 8 up, 8 in 2024-04-07T17:07:35.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:07:34 smithi100 bash[76202]: cluster 2024-04-07T17:07:33.715992+0000 mon.a (mon.0) 2758 : cluster [DBG] osdmap e1464: 8 total, 8 up, 8 in 2024-04-07T17:07:36.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:07:35 smithi144 bash[66654]: cluster 2024-04-07T17:07:33.947205+0000 mgr.y (mgr.25175) 1566 : cluster [DBG] pgmap v2605: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 73 KiB/s rd, 6.7 KiB/s wr, 94 op/s 2024-04-07T17:07:36.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:07:35 smithi144 bash[66654]: cluster 2024-04-07T17:07:34.721567+0000 mon.a (mon.0) 2759 : cluster [DBG] osdmap e1465: 8 total, 8 up, 8 in 2024-04-07T17:07:36.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:07:35 smithi100 bash[74942]: cluster 2024-04-07T17:07:33.947205+0000 mgr.y (mgr.25175) 1566 : cluster [DBG] pgmap v2605: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 73 KiB/s rd, 6.7 KiB/s wr, 94 op/s 2024-04-07T17:07:36.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:07:35 smithi100 bash[74942]: cluster 2024-04-07T17:07:34.721567+0000 mon.a (mon.0) 2759 : cluster [DBG] osdmap e1465: 8 total, 8 up, 8 in 2024-04-07T17:07:36.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:07:35 smithi100 bash[76202]: cluster 2024-04-07T17:07:33.947205+0000 mgr.y (mgr.25175) 1566 : cluster [DBG] pgmap v2605: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 73 KiB/s rd, 6.7 KiB/s wr, 94 op/s 2024-04-07T17:07:36.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:07:35 smithi100 bash[76202]: cluster 2024-04-07T17:07:34.721567+0000 mon.a (mon.0) 2759 : cluster [DBG] osdmap e1465: 8 total, 8 up, 8 in 2024-04-07T17:07:37.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:07:36 smithi144 bash[66654]: cluster 2024-04-07T17:07:35.729547+0000 mon.a (mon.0) 2760 : cluster [DBG] osdmap e1466: 8 total, 8 up, 8 in 2024-04-07T17:07:37.202 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:07:36 smithi100 bash[74942]: cluster 2024-04-07T17:07:35.729547+0000 mon.a (mon.0) 2760 : cluster [DBG] osdmap e1466: 8 total, 8 up, 8 in 2024-04-07T17:07:37.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:07:36 smithi100 bash[76202]: cluster 2024-04-07T17:07:35.729547+0000 mon.a (mon.0) 2760 : cluster [DBG] osdmap e1466: 8 total, 8 up, 8 in 2024-04-07T17:07:38.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:07:37 smithi144 bash[66654]: cluster 2024-04-07T17:07:35.948038+0000 mgr.y (mgr.25175) 1567 : cluster [DBG] pgmap v2608: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 64 KiB/s rd, 7.2 KiB/s wr, 85 op/s 2024-04-07T17:07:38.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:07:37 smithi144 bash[66654]: cluster 2024-04-07T17:07:36.740340+0000 mon.a (mon.0) 2761 : cluster [DBG] osdmap e1467: 8 total, 8 up, 8 in 2024-04-07T17:07:38.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:07:37 smithi144 bash[66654]: cluster 2024-04-07T17:07:37.653811+0000 mon.a (mon.0) 2762 : cluster [DBG] osdmap e1468: 8 total, 8 up, 8 in 2024-04-07T17:07:38.202 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:07:37 smithi100 bash[74942]: cluster 2024-04-07T17:07:35.948038+0000 mgr.y (mgr.25175) 1567 : cluster [DBG] pgmap v2608: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 64 KiB/s rd, 7.2 KiB/s wr, 85 op/s 2024-04-07T17:07:38.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:07:37 smithi100 bash[74942]: cluster 2024-04-07T17:07:36.740340+0000 mon.a (mon.0) 2761 : cluster [DBG] osdmap e1467: 8 total, 8 up, 8 in 2024-04-07T17:07:38.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:07:37 smithi100 bash[74942]: cluster 2024-04-07T17:07:37.653811+0000 mon.a (mon.0) 2762 : cluster [DBG] osdmap e1468: 8 total, 8 up, 8 in 2024-04-07T17:07:38.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:07:37 smithi100 bash[76202]: cluster 2024-04-07T17:07:35.948038+0000 mgr.y (mgr.25175) 1567 : cluster [DBG] pgmap v2608: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 64 KiB/s rd, 7.2 KiB/s wr, 85 op/s 2024-04-07T17:07:38.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:07:37 smithi100 bash[76202]: cluster 2024-04-07T17:07:36.740340+0000 mon.a (mon.0) 2761 : cluster [DBG] osdmap e1467: 8 total, 8 up, 8 in 2024-04-07T17:07:38.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:07:37 smithi100 bash[76202]: cluster 2024-04-07T17:07:37.653811+0000 mon.a (mon.0) 2762 : cluster [DBG] osdmap e1468: 8 total, 8 up, 8 in 2024-04-07T17:07:38.702 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:07:38 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:07:38] "GET /metrics HTTP/1.1" 200 36473 "" "Prometheus/2.43.0" 2024-04-07T17:07:39.952 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:07:39 smithi100 bash[74942]: cluster 2024-04-07T17:07:37.948954+0000 mgr.y (mgr.25175) 1568 : cluster [DBG] pgmap v2611: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 73 KiB/s rd, 7.0 KiB/s wr, 98 op/s 2024-04-07T17:07:39.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:07:39 smithi100 bash[74942]: cluster 2024-04-07T17:07:38.675794+0000 mon.a (mon.0) 2763 : cluster [DBG] osdmap e1469: 8 total, 8 up, 8 in 2024-04-07T17:07:39.953 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:07:39 smithi100 bash[76202]: cluster 2024-04-07T17:07:37.948954+0000 mgr.y (mgr.25175) 1568 : cluster [DBG] pgmap v2611: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 73 KiB/s rd, 7.0 KiB/s wr, 98 op/s 2024-04-07T17:07:39.953 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:07:39 smithi100 bash[76202]: cluster 2024-04-07T17:07:38.675794+0000 mon.a (mon.0) 2763 : cluster [DBG] osdmap e1469: 8 total, 8 up, 8 in 2024-04-07T17:07:40.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:07:39 smithi144 bash[66654]: cluster 2024-04-07T17:07:37.948954+0000 mgr.y (mgr.25175) 1568 : cluster [DBG] pgmap v2611: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 73 KiB/s rd, 7.0 KiB/s wr, 98 op/s 2024-04-07T17:07:40.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:07:39 smithi144 bash[66654]: cluster 2024-04-07T17:07:38.675794+0000 mon.a (mon.0) 2763 : cluster [DBG] osdmap e1469: 8 total, 8 up, 8 in 2024-04-07T17:07:40.933 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestMirroring.Snapshot (15445 ms) 2024-04-07T17:07:40.933 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestMirroring.SnapshotRemoveOnDisable 2024-04-07T17:07:40.952 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:07:40 smithi100 bash[74942]: cluster 2024-04-07T17:07:39.677770+0000 mon.a (mon.0) 2764 : cluster [DBG] osdmap e1470: 8 total, 8 up, 8 in 2024-04-07T17:07:40.952 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:07:40 smithi100 bash[76202]: cluster 2024-04-07T17:07:39.677770+0000 mon.a (mon.0) 2764 : cluster [DBG] osdmap e1470: 8 total, 8 up, 8 in 2024-04-07T17:07:41.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:07:40 smithi144 bash[66654]: cluster 2024-04-07T17:07:39.677770+0000 mon.a (mon.0) 2764 : cluster [DBG] osdmap e1470: 8 total, 8 up, 8 in 2024-04-07T17:07:41.953 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:07:41 smithi100 bash[76202]: cluster 2024-04-07T17:07:39.949679+0000 mgr.y (mgr.25175) 1569 : cluster [DBG] pgmap v2614: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 67 KiB/s rd, 5.5 KiB/s wr, 90 op/s 2024-04-07T17:07:41.953 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:07:41 smithi100 bash[76202]: cluster 2024-04-07T17:07:40.705080+0000 mon.a (mon.0) 2765 : cluster [DBG] osdmap e1471: 8 total, 8 up, 8 in 2024-04-07T17:07:41.953 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:07:41 smithi100 bash[76202]: audit 2024-04-07T17:07:40.915079+0000 mon.a (mon.0) 2766 : audit [INF] from='client.? 172.21.15.100:0/3748045073' entity='client.admin' cmd=[{"prefix": "config-key rm", "key": "rbd/mirror/peer/88/68e53edb-ead7-4136-859d-2d87809cad72"}]: dispatch 2024-04-07T17:07:41.953 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:07:41 smithi100 bash[76202]: audit 2024-04-07T17:07:40.921770+0000 mon.a (mon.0) 2767 : audit [INF] from='client.? 172.21.15.100:0/3748045073' entity='client.admin' cmd='[{"prefix": "config-key rm", "key": "rbd/mirror/peer/88/68e53edb-ead7-4136-859d-2d87809cad72"}]': finished 2024-04-07T17:07:41.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:07:41 smithi100 bash[74942]: cluster 2024-04-07T17:07:39.949679+0000 mgr.y (mgr.25175) 1569 : cluster [DBG] pgmap v2614: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 67 KiB/s rd, 5.5 KiB/s wr, 90 op/s 2024-04-07T17:07:41.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:07:41 smithi100 bash[74942]: cluster 2024-04-07T17:07:40.705080+0000 mon.a (mon.0) 2765 : cluster [DBG] osdmap e1471: 8 total, 8 up, 8 in 2024-04-07T17:07:41.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:07:41 smithi100 bash[74942]: audit 2024-04-07T17:07:40.915079+0000 mon.a (mon.0) 2766 : audit [INF] from='client.? 172.21.15.100:0/3748045073' entity='client.admin' cmd=[{"prefix": "config-key rm", "key": "rbd/mirror/peer/88/68e53edb-ead7-4136-859d-2d87809cad72"}]: dispatch 2024-04-07T17:07:41.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:07:41 smithi100 bash[74942]: audit 2024-04-07T17:07:40.921770+0000 mon.a (mon.0) 2767 : audit [INF] from='client.? 172.21.15.100:0/3748045073' entity='client.admin' cmd='[{"prefix": "config-key rm", "key": "rbd/mirror/peer/88/68e53edb-ead7-4136-859d-2d87809cad72"}]': finished 2024-04-07T17:07:42.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:07:41 smithi144 bash[66654]: cluster 2024-04-07T17:07:39.949679+0000 mgr.y (mgr.25175) 1569 : cluster [DBG] pgmap v2614: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 67 KiB/s rd, 5.5 KiB/s wr, 90 op/s 2024-04-07T17:07:42.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:07:41 smithi144 bash[66654]: cluster 2024-04-07T17:07:40.705080+0000 mon.a (mon.0) 2765 : cluster [DBG] osdmap e1471: 8 total, 8 up, 8 in 2024-04-07T17:07:42.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:07:41 smithi144 bash[66654]: audit 2024-04-07T17:07:40.915079+0000 mon.a (mon.0) 2766 : audit [INF] from='client.? 172.21.15.100:0/3748045073' entity='client.admin' cmd=[{"prefix": "config-key rm", "key": "rbd/mirror/peer/88/68e53edb-ead7-4136-859d-2d87809cad72"}]: dispatch 2024-04-07T17:07:42.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:07:41 smithi144 bash[66654]: audit 2024-04-07T17:07:40.921770+0000 mon.a (mon.0) 2767 : audit [INF] from='client.? 172.21.15.100:0/3748045073' entity='client.admin' cmd='[{"prefix": "config-key rm", "key": "rbd/mirror/peer/88/68e53edb-ead7-4136-859d-2d87809cad72"}]': finished 2024-04-07T17:07:42.616 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:07:42 smithi144 bash[95233]: ts=2024-04-07T17:07:42.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:07:43.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:07:42 smithi100 bash[74942]: cluster 2024-04-07T17:07:41.930036+0000 mon.a (mon.0) 2768 : cluster [DBG] osdmap e1472: 8 total, 8 up, 8 in 2024-04-07T17:07:43.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:07:42 smithi100 bash[74942]: cluster 2024-04-07T17:07:41.950461+0000 mgr.y (mgr.25175) 1570 : cluster [DBG] pgmap v2617: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 4.7 KiB/s rd, 0 B/s wr, 8 op/s 2024-04-07T17:07:43.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:07:42 smithi100 bash[74942]: cluster 2024-04-07T17:07:42.661708+0000 mon.a (mon.0) 2769 : cluster [DBG] osdmap e1473: 8 total, 8 up, 8 in 2024-04-07T17:07:43.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:07:42 smithi100 bash[76202]: cluster 2024-04-07T17:07:41.930036+0000 mon.a (mon.0) 2768 : cluster [DBG] osdmap e1472: 8 total, 8 up, 8 in 2024-04-07T17:07:43.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:07:42 smithi100 bash[76202]: cluster 2024-04-07T17:07:41.950461+0000 mgr.y (mgr.25175) 1570 : cluster [DBG] pgmap v2617: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 4.7 KiB/s rd, 0 B/s wr, 8 op/s 2024-04-07T17:07:43.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:07:42 smithi100 bash[76202]: cluster 2024-04-07T17:07:42.661708+0000 mon.a (mon.0) 2769 : cluster [DBG] osdmap e1473: 8 total, 8 up, 8 in 2024-04-07T17:07:43.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:07:42 smithi144 bash[66654]: cluster 2024-04-07T17:07:41.930036+0000 mon.a (mon.0) 2768 : cluster [DBG] osdmap e1472: 8 total, 8 up, 8 in 2024-04-07T17:07:43.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:07:42 smithi144 bash[66654]: cluster 2024-04-07T17:07:41.950461+0000 mgr.y (mgr.25175) 1570 : cluster [DBG] pgmap v2617: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 4.7 KiB/s rd, 0 B/s wr, 8 op/s 2024-04-07T17:07:43.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:07:42 smithi144 bash[66654]: cluster 2024-04-07T17:07:42.661708+0000 mon.a (mon.0) 2769 : cluster [DBG] osdmap e1473: 8 total, 8 up, 8 in 2024-04-07T17:07:44.912 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestMirroring.SnapshotRemoveOnDisable (3979 ms) 2024-04-07T17:07:44.912 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestMirroring.SnapshotUnlinkPeer 2024-04-07T17:07:44.952 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:07:44 smithi100 bash[74942]: cluster 2024-04-07T17:07:43.659016+0000 mon.a (mon.0) 2770 : cluster [DBG] osdmap e1474: 8 total, 8 up, 8 in 2024-04-07T17:07:44.953 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:07:44 smithi100 bash[76202]: cluster 2024-04-07T17:07:43.659016+0000 mon.a (mon.0) 2770 : cluster [DBG] osdmap e1474: 8 total, 8 up, 8 in 2024-04-07T17:07:45.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:07:44 smithi144 bash[66654]: cluster 2024-04-07T17:07:43.659016+0000 mon.a (mon.0) 2770 : cluster [DBG] osdmap e1474: 8 total, 8 up, 8 in 2024-04-07T17:07:45.952 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:07:45 smithi100 bash[74942]: cluster 2024-04-07T17:07:43.951204+0000 mgr.y (mgr.25175) 1571 : cluster [DBG] pgmap v2620: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 51 KiB/s rd, 4.2 KiB/s wr, 70 op/s 2024-04-07T17:07:45.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:07:45 smithi100 bash[74942]: cluster 2024-04-07T17:07:44.669147+0000 mon.a (mon.0) 2771 : cluster [DBG] osdmap e1475: 8 total, 8 up, 8 in 2024-04-07T17:07:45.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:07:45 smithi100 bash[74942]: audit 2024-04-07T17:07:44.894012+0000 mon.a (mon.0) 2772 : audit [INF] from='client.? 172.21.15.100:0/3748045073' entity='client.admin' cmd=[{"prefix": "config-key rm", "key": "rbd/mirror/peer/88/0a241a4d-ff19-4098-8ed3-2a46a6b61fc1"}]: dispatch 2024-04-07T17:07:45.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:07:45 smithi100 bash[74942]: audit 2024-04-07T17:07:44.900280+0000 mon.a (mon.0) 2773 : audit [INF] from='client.? 172.21.15.100:0/3748045073' entity='client.admin' cmd='[{"prefix": "config-key rm", "key": "rbd/mirror/peer/88/0a241a4d-ff19-4098-8ed3-2a46a6b61fc1"}]': finished 2024-04-07T17:07:45.953 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:07:45 smithi100 bash[76202]: cluster 2024-04-07T17:07:43.951204+0000 mgr.y (mgr.25175) 1571 : cluster [DBG] pgmap v2620: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 51 KiB/s rd, 4.2 KiB/s wr, 70 op/s 2024-04-07T17:07:45.953 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:07:45 smithi100 bash[76202]: cluster 2024-04-07T17:07:44.669147+0000 mon.a (mon.0) 2771 : cluster [DBG] osdmap e1475: 8 total, 8 up, 8 in 2024-04-07T17:07:45.953 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:07:45 smithi100 bash[76202]: audit 2024-04-07T17:07:44.894012+0000 mon.a (mon.0) 2772 : audit [INF] from='client.? 172.21.15.100:0/3748045073' entity='client.admin' cmd=[{"prefix": "config-key rm", "key": "rbd/mirror/peer/88/0a241a4d-ff19-4098-8ed3-2a46a6b61fc1"}]: dispatch 2024-04-07T17:07:45.953 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:07:45 smithi100 bash[76202]: audit 2024-04-07T17:07:44.900280+0000 mon.a (mon.0) 2773 : audit [INF] from='client.? 172.21.15.100:0/3748045073' entity='client.admin' cmd='[{"prefix": "config-key rm", "key": "rbd/mirror/peer/88/0a241a4d-ff19-4098-8ed3-2a46a6b61fc1"}]': finished 2024-04-07T17:07:46.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:07:45 smithi144 bash[66654]: cluster 2024-04-07T17:07:43.951204+0000 mgr.y (mgr.25175) 1571 : cluster [DBG] pgmap v2620: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 51 KiB/s rd, 4.2 KiB/s wr, 70 op/s 2024-04-07T17:07:46.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:07:45 smithi144 bash[66654]: cluster 2024-04-07T17:07:44.669147+0000 mon.a (mon.0) 2771 : cluster [DBG] osdmap e1475: 8 total, 8 up, 8 in 2024-04-07T17:07:46.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:07:45 smithi144 bash[66654]: audit 2024-04-07T17:07:44.894012+0000 mon.a (mon.0) 2772 : audit [INF] from='client.? 172.21.15.100:0/3748045073' entity='client.admin' cmd=[{"prefix": "config-key rm", "key": "rbd/mirror/peer/88/0a241a4d-ff19-4098-8ed3-2a46a6b61fc1"}]: dispatch 2024-04-07T17:07:46.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:07:45 smithi144 bash[66654]: audit 2024-04-07T17:07:44.900280+0000 mon.a (mon.0) 2773 : audit [INF] from='client.? 172.21.15.100:0/3748045073' entity='client.admin' cmd='[{"prefix": "config-key rm", "key": "rbd/mirror/peer/88/0a241a4d-ff19-4098-8ed3-2a46a6b61fc1"}]': finished 2024-04-07T17:07:47.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:07:46 smithi100 bash[74942]: cluster 2024-04-07T17:07:45.905449+0000 mon.a (mon.0) 2774 : cluster [DBG] osdmap e1476: 8 total, 8 up, 8 in 2024-04-07T17:07:47.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:07:46 smithi100 bash[74942]: cluster 2024-04-07T17:07:45.952047+0000 mgr.y (mgr.25175) 1572 : cluster [DBG] pgmap v2623: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 97 KiB/s rd, 11 KiB/s wr, 136 op/s 2024-04-07T17:07:47.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:07:46 smithi100 bash[74942]: audit 2024-04-07T17:07:46.809798+0000 mon.a (mon.0) 2775 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:07:47.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:07:46 smithi100 bash[76202]: cluster 2024-04-07T17:07:45.905449+0000 mon.a (mon.0) 2774 : cluster [DBG] osdmap e1476: 8 total, 8 up, 8 in 2024-04-07T17:07:47.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:07:46 smithi100 bash[76202]: cluster 2024-04-07T17:07:45.952047+0000 mgr.y (mgr.25175) 1572 : cluster [DBG] pgmap v2623: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 97 KiB/s rd, 11 KiB/s wr, 136 op/s 2024-04-07T17:07:47.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:07:46 smithi100 bash[76202]: audit 2024-04-07T17:07:46.809798+0000 mon.a (mon.0) 2775 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:07:47.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:07:46 smithi144 bash[66654]: cluster 2024-04-07T17:07:45.905449+0000 mon.a (mon.0) 2774 : cluster [DBG] osdmap e1476: 8 total, 8 up, 8 in 2024-04-07T17:07:47.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:07:46 smithi144 bash[66654]: cluster 2024-04-07T17:07:45.952047+0000 mgr.y (mgr.25175) 1572 : cluster [DBG] pgmap v2623: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 97 KiB/s rd, 11 KiB/s wr, 136 op/s 2024-04-07T17:07:47.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:07:46 smithi144 bash[66654]: audit 2024-04-07T17:07:46.809798+0000 mon.a (mon.0) 2775 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:07:48.202 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:07:47 smithi100 bash[76202]: cluster 2024-04-07T17:07:46.908081+0000 mon.a (mon.0) 2776 : cluster [DBG] osdmap e1477: 8 total, 8 up, 8 in 2024-04-07T17:07:48.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:07:47 smithi100 bash[76202]: cluster 2024-04-07T17:07:47.659945+0000 mon.a (mon.0) 2777 : cluster [DBG] osdmap e1478: 8 total, 8 up, 8 in 2024-04-07T17:07:48.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:07:47 smithi100 bash[74942]: cluster 2024-04-07T17:07:46.908081+0000 mon.a (mon.0) 2776 : cluster [DBG] osdmap e1477: 8 total, 8 up, 8 in 2024-04-07T17:07:48.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:07:47 smithi100 bash[74942]: cluster 2024-04-07T17:07:47.659945+0000 mon.a (mon.0) 2777 : cluster [DBG] osdmap e1478: 8 total, 8 up, 8 in 2024-04-07T17:07:48.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:07:47 smithi144 bash[66654]: cluster 2024-04-07T17:07:46.908081+0000 mon.a (mon.0) 2776 : cluster [DBG] osdmap e1477: 8 total, 8 up, 8 in 2024-04-07T17:07:48.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:07:47 smithi144 bash[66654]: cluster 2024-04-07T17:07:47.659945+0000 mon.a (mon.0) 2777 : cluster [DBG] osdmap e1478: 8 total, 8 up, 8 in 2024-04-07T17:07:48.703 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:07:48 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:07:48] "GET /metrics HTTP/1.1" 200 36474 "" "Prometheus/2.43.0" 2024-04-07T17:07:49.952 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:07:49 smithi100 bash[74942]: cluster 2024-04-07T17:07:47.952945+0000 mgr.y (mgr.25175) 1573 : cluster [DBG] pgmap v2626: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 53 KiB/s rd, 9.0 KiB/s wr, 81 op/s 2024-04-07T17:07:49.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:07:49 smithi100 bash[74942]: cluster 2024-04-07T17:07:48.661356+0000 mon.a (mon.0) 2778 : cluster [DBG] osdmap e1479: 8 total, 8 up, 8 in 2024-04-07T17:07:49.953 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:07:49 smithi100 bash[76202]: cluster 2024-04-07T17:07:47.952945+0000 mgr.y (mgr.25175) 1573 : cluster [DBG] pgmap v2626: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 53 KiB/s rd, 9.0 KiB/s wr, 81 op/s 2024-04-07T17:07:49.953 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:07:49 smithi100 bash[76202]: cluster 2024-04-07T17:07:48.661356+0000 mon.a (mon.0) 2778 : cluster [DBG] osdmap e1479: 8 total, 8 up, 8 in 2024-04-07T17:07:50.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:07:49 smithi144 bash[66654]: cluster 2024-04-07T17:07:47.952945+0000 mgr.y (mgr.25175) 1573 : cluster [DBG] pgmap v2626: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 53 KiB/s rd, 9.0 KiB/s wr, 81 op/s 2024-04-07T17:07:50.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:07:49 smithi144 bash[66654]: cluster 2024-04-07T17:07:48.661356+0000 mon.a (mon.0) 2778 : cluster [DBG] osdmap e1479: 8 total, 8 up, 8 in 2024-04-07T17:07:50.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:07:50 smithi100 bash[74942]: cluster 2024-04-07T17:07:49.675722+0000 mon.a (mon.0) 2779 : cluster [DBG] osdmap e1480: 8 total, 8 up, 8 in 2024-04-07T17:07:50.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:07:50 smithi100 bash[74942]: audit 2024-04-07T17:07:49.728166+0000 mon.a (mon.0) 2780 : audit [INF] from='client.? 172.21.15.100:0/3748045073' entity='client.admin' cmd=[{"prefix": "config-key rm", "key": "rbd/mirror/peer/88/0204a0f8-ac5f-4783-bf96-6ca31d2fe144"}]: dispatch 2024-04-07T17:07:50.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:07:50 smithi100 bash[74942]: audit 2024-04-07T17:07:49.734663+0000 mon.a (mon.0) 2781 : audit [INF] from='client.? 172.21.15.100:0/3748045073' entity='client.admin' cmd='[{"prefix": "config-key rm", "key": "rbd/mirror/peer/88/0204a0f8-ac5f-4783-bf96-6ca31d2fe144"}]': finished 2024-04-07T17:07:50.953 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:07:50 smithi100 bash[76202]: cluster 2024-04-07T17:07:49.675722+0000 mon.a (mon.0) 2779 : cluster [DBG] osdmap e1480: 8 total, 8 up, 8 in 2024-04-07T17:07:50.953 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:07:50 smithi100 bash[76202]: audit 2024-04-07T17:07:49.728166+0000 mon.a (mon.0) 2780 : audit [INF] from='client.? 172.21.15.100:0/3748045073' entity='client.admin' cmd=[{"prefix": "config-key rm", "key": "rbd/mirror/peer/88/0204a0f8-ac5f-4783-bf96-6ca31d2fe144"}]: dispatch 2024-04-07T17:07:50.953 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:07:50 smithi100 bash[76202]: audit 2024-04-07T17:07:49.734663+0000 mon.a (mon.0) 2781 : audit [INF] from='client.? 172.21.15.100:0/3748045073' entity='client.admin' cmd='[{"prefix": "config-key rm", "key": "rbd/mirror/peer/88/0204a0f8-ac5f-4783-bf96-6ca31d2fe144"}]': finished 2024-04-07T17:07:51.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:07:50 smithi144 bash[66654]: cluster 2024-04-07T17:07:49.675722+0000 mon.a (mon.0) 2779 : cluster [DBG] osdmap e1480: 8 total, 8 up, 8 in 2024-04-07T17:07:51.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:07:50 smithi144 bash[66654]: audit 2024-04-07T17:07:49.728166+0000 mon.a (mon.0) 2780 : audit [INF] from='client.? 172.21.15.100:0/3748045073' entity='client.admin' cmd=[{"prefix": "config-key rm", "key": "rbd/mirror/peer/88/0204a0f8-ac5f-4783-bf96-6ca31d2fe144"}]: dispatch 2024-04-07T17:07:51.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:07:50 smithi144 bash[66654]: audit 2024-04-07T17:07:49.734663+0000 mon.a (mon.0) 2781 : audit [INF] from='client.? 172.21.15.100:0/3748045073' entity='client.admin' cmd='[{"prefix": "config-key rm", "key": "rbd/mirror/peer/88/0204a0f8-ac5f-4783-bf96-6ca31d2fe144"}]': finished 2024-04-07T17:07:52.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:07:51 smithi144 bash[66654]: cluster 2024-04-07T17:07:49.953967+0000 mgr.y (mgr.25175) 1574 : cluster [DBG] pgmap v2629: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 144 KiB/s rd, 12 KiB/s wr, 192 op/s 2024-04-07T17:07:52.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:07:51 smithi144 bash[66654]: cluster 2024-04-07T17:07:50.745262+0000 mon.a (mon.0) 2782 : cluster [DBG] osdmap e1481: 8 total, 8 up, 8 in 2024-04-07T17:07:52.202 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:07:51 smithi100 bash[74942]: cluster 2024-04-07T17:07:49.953967+0000 mgr.y (mgr.25175) 1574 : cluster [DBG] pgmap v2629: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 144 KiB/s rd, 12 KiB/s wr, 192 op/s 2024-04-07T17:07:52.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:07:51 smithi100 bash[74942]: cluster 2024-04-07T17:07:50.745262+0000 mon.a (mon.0) 2782 : cluster [DBG] osdmap e1481: 8 total, 8 up, 8 in 2024-04-07T17:07:52.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:07:51 smithi100 bash[76202]: cluster 2024-04-07T17:07:49.953967+0000 mgr.y (mgr.25175) 1574 : cluster [DBG] pgmap v2629: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 144 KiB/s rd, 12 KiB/s wr, 192 op/s 2024-04-07T17:07:52.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:07:51 smithi100 bash[76202]: cluster 2024-04-07T17:07:50.745262+0000 mon.a (mon.0) 2782 : cluster [DBG] osdmap e1481: 8 total, 8 up, 8 in 2024-04-07T17:07:52.616 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:07:52 smithi144 bash[95233]: ts=2024-04-07T17:07:52.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:07:53.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:07:52 smithi144 bash[66654]: cluster 2024-04-07T17:07:51.757457+0000 mon.a (mon.0) 2783 : cluster [DBG] osdmap e1482: 8 total, 8 up, 8 in 2024-04-07T17:07:53.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:07:52 smithi144 bash[66654]: cluster 2024-04-07T17:07:52.662482+0000 mon.a (mon.0) 2784 : cluster [DBG] osdmap e1483: 8 total, 8 up, 8 in 2024-04-07T17:07:53.202 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:07:52 smithi100 bash[74942]: cluster 2024-04-07T17:07:51.757457+0000 mon.a (mon.0) 2783 : cluster [DBG] osdmap e1482: 8 total, 8 up, 8 in 2024-04-07T17:07:53.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:07:52 smithi100 bash[74942]: cluster 2024-04-07T17:07:52.662482+0000 mon.a (mon.0) 2784 : cluster [DBG] osdmap e1483: 8 total, 8 up, 8 in 2024-04-07T17:07:53.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:07:52 smithi100 bash[76202]: cluster 2024-04-07T17:07:51.757457+0000 mon.a (mon.0) 2783 : cluster [DBG] osdmap e1482: 8 total, 8 up, 8 in 2024-04-07T17:07:53.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:07:52 smithi100 bash[76202]: cluster 2024-04-07T17:07:52.662482+0000 mon.a (mon.0) 2784 : cluster [DBG] osdmap e1483: 8 total, 8 up, 8 in 2024-04-07T17:07:54.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:07:53 smithi144 bash[66654]: cluster 2024-04-07T17:07:51.954711+0000 mgr.y (mgr.25175) 1575 : cluster [DBG] pgmap v2632: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 140 KiB/s rd, 10 KiB/s wr, 182 op/s 2024-04-07T17:07:54.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:07:53 smithi144 bash[66654]: audit 2024-04-07T17:07:53.152194+0000 mon.a (mon.0) 2785 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:07:54.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:07:53 smithi100 bash[74942]: cluster 2024-04-07T17:07:51.954711+0000 mgr.y (mgr.25175) 1575 : cluster [DBG] pgmap v2632: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 140 KiB/s rd, 10 KiB/s wr, 182 op/s 2024-04-07T17:07:54.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:07:53 smithi100 bash[74942]: audit 2024-04-07T17:07:53.152194+0000 mon.a (mon.0) 2785 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:07:54.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:07:53 smithi100 bash[76202]: cluster 2024-04-07T17:07:51.954711+0000 mgr.y (mgr.25175) 1575 : cluster [DBG] pgmap v2632: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 140 KiB/s rd, 10 KiB/s wr, 182 op/s 2024-04-07T17:07:54.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:07:53 smithi100 bash[76202]: audit 2024-04-07T17:07:53.152194+0000 mon.a (mon.0) 2785 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:07:54.949 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestMirroring.SnapshotUnlinkPeer (10038 ms) 2024-04-07T17:07:54.950 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestMirroring.SnapshotImageState 2024-04-07T17:07:55.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:07:54 smithi144 bash[66654]: cluster 2024-04-07T17:07:53.763693+0000 mon.a (mon.0) 2786 : cluster [DBG] osdmap e1484: 8 total, 8 up, 8 in 2024-04-07T17:07:55.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:07:54 smithi144 bash[66654]: audit 2024-04-07T17:07:53.892963+0000 mon.a (mon.0) 2787 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:07:55.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:07:54 smithi144 bash[66654]: audit 2024-04-07T17:07:53.894641+0000 mon.a (mon.0) 2788 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:07:55.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:07:54 smithi144 bash[66654]: audit 2024-04-07T17:07:53.902106+0000 mon.a (mon.0) 2789 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:07:55.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:07:54 smithi100 bash[74942]: cluster 2024-04-07T17:07:53.763693+0000 mon.a (mon.0) 2786 : cluster [DBG] osdmap e1484: 8 total, 8 up, 8 in 2024-04-07T17:07:55.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:07:54 smithi100 bash[74942]: audit 2024-04-07T17:07:53.892963+0000 mon.a (mon.0) 2787 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:07:55.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:07:54 smithi100 bash[74942]: audit 2024-04-07T17:07:53.894641+0000 mon.a (mon.0) 2788 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:07:55.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:07:54 smithi100 bash[74942]: audit 2024-04-07T17:07:53.902106+0000 mon.a (mon.0) 2789 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:07:55.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:07:54 smithi100 bash[76202]: cluster 2024-04-07T17:07:53.763693+0000 mon.a (mon.0) 2786 : cluster [DBG] osdmap e1484: 8 total, 8 up, 8 in 2024-04-07T17:07:55.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:07:54 smithi100 bash[76202]: audit 2024-04-07T17:07:53.892963+0000 mon.a (mon.0) 2787 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:07:55.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:07:54 smithi100 bash[76202]: audit 2024-04-07T17:07:53.894641+0000 mon.a (mon.0) 2788 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:07:55.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:07:54 smithi100 bash[76202]: audit 2024-04-07T17:07:53.902106+0000 mon.a (mon.0) 2789 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:07:56.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:07:55 smithi144 bash[66654]: cluster 2024-04-07T17:07:53.955335+0000 mgr.y (mgr.25175) 1576 : cluster [DBG] pgmap v2635: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 4.5 KiB/s rd, 1023 B/s wr, 8 op/s 2024-04-07T17:07:56.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:07:55 smithi144 bash[66654]: cluster 2024-04-07T17:07:54.789649+0000 mon.a (mon.0) 2790 : cluster [DBG] osdmap e1485: 8 total, 8 up, 8 in 2024-04-07T17:07:56.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:07:55 smithi144 bash[66654]: audit 2024-04-07T17:07:54.921101+0000 mon.a (mon.0) 2791 : audit [INF] from='client.? 172.21.15.100:0/3748045073' entity='client.admin' cmd=[{"prefix": "config-key rm", "key": "rbd/mirror/peer/88/d51fc9ea-3b78-4472-bf25-3735713f1a40"}]: dispatch 2024-04-07T17:07:56.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:07:55 smithi144 bash[66654]: audit 2024-04-07T17:07:54.928648+0000 mon.a (mon.0) 2792 : audit [INF] from='client.? 172.21.15.100:0/3748045073' entity='client.admin' cmd='[{"prefix": "config-key rm", "key": "rbd/mirror/peer/88/d51fc9ea-3b78-4472-bf25-3735713f1a40"}]': finished 2024-04-07T17:07:56.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:07:55 smithi144 bash[66654]: audit 2024-04-07T17:07:54.933563+0000 mon.a (mon.0) 2793 : audit [INF] from='client.? 172.21.15.100:0/3748045073' entity='client.admin' cmd=[{"prefix": "config-key rm", "key": "rbd/mirror/peer/88/b073cd42-7a9b-4669-96be-b6d8579a1705"}]: dispatch 2024-04-07T17:07:56.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:07:55 smithi144 bash[66654]: audit 2024-04-07T17:07:54.941171+0000 mon.a (mon.0) 2794 : audit [INF] from='client.? 172.21.15.100:0/3748045073' entity='client.admin' cmd='[{"prefix": "config-key rm", "key": "rbd/mirror/peer/88/b073cd42-7a9b-4669-96be-b6d8579a1705"}]': finished 2024-04-07T17:07:56.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:07:55 smithi100 bash[74942]: cluster 2024-04-07T17:07:53.955335+0000 mgr.y (mgr.25175) 1576 : cluster [DBG] pgmap v2635: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 4.5 KiB/s rd, 1023 B/s wr, 8 op/s 2024-04-07T17:07:56.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:07:55 smithi100 bash[74942]: cluster 2024-04-07T17:07:54.789649+0000 mon.a (mon.0) 2790 : cluster [DBG] osdmap e1485: 8 total, 8 up, 8 in 2024-04-07T17:07:56.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:07:55 smithi100 bash[74942]: audit 2024-04-07T17:07:54.921101+0000 mon.a (mon.0) 2791 : audit [INF] from='client.? 172.21.15.100:0/3748045073' entity='client.admin' cmd=[{"prefix": "config-key rm", "key": "rbd/mirror/peer/88/d51fc9ea-3b78-4472-bf25-3735713f1a40"}]: dispatch 2024-04-07T17:07:56.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:07:55 smithi100 bash[74942]: audit 2024-04-07T17:07:54.928648+0000 mon.a (mon.0) 2792 : audit [INF] from='client.? 172.21.15.100:0/3748045073' entity='client.admin' cmd='[{"prefix": "config-key rm", "key": "rbd/mirror/peer/88/d51fc9ea-3b78-4472-bf25-3735713f1a40"}]': finished 2024-04-07T17:07:56.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:07:55 smithi100 bash[74942]: audit 2024-04-07T17:07:54.933563+0000 mon.a (mon.0) 2793 : audit [INF] from='client.? 172.21.15.100:0/3748045073' entity='client.admin' cmd=[{"prefix": "config-key rm", "key": "rbd/mirror/peer/88/b073cd42-7a9b-4669-96be-b6d8579a1705"}]: dispatch 2024-04-07T17:07:56.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:07:55 smithi100 bash[74942]: audit 2024-04-07T17:07:54.941171+0000 mon.a (mon.0) 2794 : audit [INF] from='client.? 172.21.15.100:0/3748045073' entity='client.admin' cmd='[{"prefix": "config-key rm", "key": "rbd/mirror/peer/88/b073cd42-7a9b-4669-96be-b6d8579a1705"}]': finished 2024-04-07T17:07:56.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:07:55 smithi100 bash[76202]: cluster 2024-04-07T17:07:53.955335+0000 mgr.y (mgr.25175) 1576 : cluster [DBG] pgmap v2635: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 4.5 KiB/s rd, 1023 B/s wr, 8 op/s 2024-04-07T17:07:56.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:07:55 smithi100 bash[76202]: cluster 2024-04-07T17:07:54.789649+0000 mon.a (mon.0) 2790 : cluster [DBG] osdmap e1485: 8 total, 8 up, 8 in 2024-04-07T17:07:56.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:07:55 smithi100 bash[76202]: audit 2024-04-07T17:07:54.921101+0000 mon.a (mon.0) 2791 : audit [INF] from='client.? 172.21.15.100:0/3748045073' entity='client.admin' cmd=[{"prefix": "config-key rm", "key": "rbd/mirror/peer/88/d51fc9ea-3b78-4472-bf25-3735713f1a40"}]: dispatch 2024-04-07T17:07:56.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:07:55 smithi100 bash[76202]: audit 2024-04-07T17:07:54.928648+0000 mon.a (mon.0) 2792 : audit [INF] from='client.? 172.21.15.100:0/3748045073' entity='client.admin' cmd='[{"prefix": "config-key rm", "key": "rbd/mirror/peer/88/d51fc9ea-3b78-4472-bf25-3735713f1a40"}]': finished 2024-04-07T17:07:56.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:07:55 smithi100 bash[76202]: audit 2024-04-07T17:07:54.933563+0000 mon.a (mon.0) 2793 : audit [INF] from='client.? 172.21.15.100:0/3748045073' entity='client.admin' cmd=[{"prefix": "config-key rm", "key": "rbd/mirror/peer/88/b073cd42-7a9b-4669-96be-b6d8579a1705"}]: dispatch 2024-04-07T17:07:56.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:07:55 smithi100 bash[76202]: audit 2024-04-07T17:07:54.941171+0000 mon.a (mon.0) 2794 : audit [INF] from='client.? 172.21.15.100:0/3748045073' entity='client.admin' cmd='[{"prefix": "config-key rm", "key": "rbd/mirror/peer/88/b073cd42-7a9b-4669-96be-b6d8579a1705"}]': finished 2024-04-07T17:07:57.202 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:07:56 smithi100 bash[74942]: cluster 2024-04-07T17:07:55.946966+0000 mon.a (mon.0) 2795 : cluster [DBG] osdmap e1486: 8 total, 8 up, 8 in 2024-04-07T17:07:57.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:07:56 smithi100 bash[74942]: cluster 2024-04-07T17:07:55.956052+0000 mgr.y (mgr.25175) 1577 : cluster [DBG] pgmap v2638: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 129 KiB/s rd, 8.0 KiB/s wr, 176 op/s 2024-04-07T17:07:57.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:07:56 smithi100 bash[76202]: cluster 2024-04-07T17:07:55.946966+0000 mon.a (mon.0) 2795 : cluster [DBG] osdmap e1486: 8 total, 8 up, 8 in 2024-04-07T17:07:57.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:07:56 smithi100 bash[76202]: cluster 2024-04-07T17:07:55.956052+0000 mgr.y (mgr.25175) 1577 : cluster [DBG] pgmap v2638: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 129 KiB/s rd, 8.0 KiB/s wr, 176 op/s 2024-04-07T17:07:57.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:07:56 smithi144 bash[66654]: cluster 2024-04-07T17:07:55.946966+0000 mon.a (mon.0) 2795 : cluster [DBG] osdmap e1486: 8 total, 8 up, 8 in 2024-04-07T17:07:57.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:07:56 smithi144 bash[66654]: cluster 2024-04-07T17:07:55.956052+0000 mgr.y (mgr.25175) 1577 : cluster [DBG] pgmap v2638: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 129 KiB/s rd, 8.0 KiB/s wr, 176 op/s 2024-04-07T17:07:58.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:07:57 smithi144 bash[66654]: cluster 2024-04-07T17:07:56.958641+0000 mon.a (mon.0) 2796 : cluster [DBG] osdmap e1487: 8 total, 8 up, 8 in 2024-04-07T17:07:58.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:07:57 smithi144 bash[66654]: cluster 2024-04-07T17:07:57.664066+0000 mon.a (mon.0) 2797 : cluster [DBG] osdmap e1488: 8 total, 8 up, 8 in 2024-04-07T17:07:58.427 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:07:57 smithi100 bash[74942]: cluster 2024-04-07T17:07:56.958641+0000 mon.a (mon.0) 2796 : cluster [DBG] osdmap e1487: 8 total, 8 up, 8 in 2024-04-07T17:07:58.427 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:07:57 smithi100 bash[74942]: cluster 2024-04-07T17:07:57.664066+0000 mon.a (mon.0) 2797 : cluster [DBG] osdmap e1488: 8 total, 8 up, 8 in 2024-04-07T17:07:58.427 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:07:57 smithi100 bash[76202]: cluster 2024-04-07T17:07:56.958641+0000 mon.a (mon.0) 2796 : cluster [DBG] osdmap e1487: 8 total, 8 up, 8 in 2024-04-07T17:07:58.427 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:07:57 smithi100 bash[76202]: cluster 2024-04-07T17:07:57.664066+0000 mon.a (mon.0) 2797 : cluster [DBG] osdmap e1488: 8 total, 8 up, 8 in 2024-04-07T17:07:58.703 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:07:58 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:07:58] "GET /metrics HTTP/1.1" 200 36474 "" "Prometheus/2.43.0" 2024-04-07T17:07:58.792 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestMirroring.SnapshotImageState (3842 ms) 2024-04-07T17:07:58.792 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestMirroring.SnapshotPromoteDemote 2024-04-07T17:07:59.952 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:07:59 smithi100 bash[74942]: cluster 2024-04-07T17:07:57.956892+0000 mgr.y (mgr.25175) 1578 : cluster [DBG] pgmap v2641: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 166 KiB/s rd, 13 KiB/s wr, 232 op/s 2024-04-07T17:07:59.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:07:59 smithi100 bash[74942]: cluster 2024-04-07T17:07:58.666367+0000 mon.a (mon.0) 2798 : cluster [DBG] osdmap e1489: 8 total, 8 up, 8 in 2024-04-07T17:07:59.953 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:07:59 smithi100 bash[76202]: cluster 2024-04-07T17:07:57.956892+0000 mgr.y (mgr.25175) 1578 : cluster [DBG] pgmap v2641: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 166 KiB/s rd, 13 KiB/s wr, 232 op/s 2024-04-07T17:07:59.953 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:07:59 smithi100 bash[76202]: cluster 2024-04-07T17:07:58.666367+0000 mon.a (mon.0) 2798 : cluster [DBG] osdmap e1489: 8 total, 8 up, 8 in 2024-04-07T17:08:00.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:07:59 smithi144 bash[66654]: cluster 2024-04-07T17:07:57.956892+0000 mgr.y (mgr.25175) 1578 : cluster [DBG] pgmap v2641: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 166 KiB/s rd, 13 KiB/s wr, 232 op/s 2024-04-07T17:08:00.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:07:59 smithi144 bash[66654]: cluster 2024-04-07T17:07:58.666367+0000 mon.a (mon.0) 2798 : cluster [DBG] osdmap e1489: 8 total, 8 up, 8 in 2024-04-07T17:08:00.952 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:08:00 smithi100 bash[74942]: cluster 2024-04-07T17:07:59.678935+0000 mon.a (mon.0) 2799 : cluster [DBG] osdmap e1490: 8 total, 8 up, 8 in 2024-04-07T17:08:00.953 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:08:00 smithi100 bash[76202]: cluster 2024-04-07T17:07:59.678935+0000 mon.a (mon.0) 2799 : cluster [DBG] osdmap e1490: 8 total, 8 up, 8 in 2024-04-07T17:08:01.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:08:00 smithi144 bash[66654]: cluster 2024-04-07T17:07:59.678935+0000 mon.a (mon.0) 2799 : cluster [DBG] osdmap e1490: 8 total, 8 up, 8 in 2024-04-07T17:08:01.953 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:08:01 smithi100 bash[76202]: cluster 2024-04-07T17:07:59.957769+0000 mgr.y (mgr.25175) 1579 : cluster [DBG] pgmap v2644: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 80 KiB/s rd, 17 KiB/s wr, 130 op/s 2024-04-07T17:08:01.953 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:08:01 smithi100 bash[76202]: cluster 2024-04-07T17:08:00.687966+0000 mon.a (mon.0) 2800 : cluster [DBG] osdmap e1491: 8 total, 8 up, 8 in 2024-04-07T17:08:01.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:08:01 smithi100 bash[74942]: cluster 2024-04-07T17:07:59.957769+0000 mgr.y (mgr.25175) 1579 : cluster [DBG] pgmap v2644: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 80 KiB/s rd, 17 KiB/s wr, 130 op/s 2024-04-07T17:08:01.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:08:01 smithi100 bash[74942]: cluster 2024-04-07T17:08:00.687966+0000 mon.a (mon.0) 2800 : cluster [DBG] osdmap e1491: 8 total, 8 up, 8 in 2024-04-07T17:08:02.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:08:01 smithi144 bash[66654]: cluster 2024-04-07T17:07:59.957769+0000 mgr.y (mgr.25175) 1579 : cluster [DBG] pgmap v2644: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 80 KiB/s rd, 17 KiB/s wr, 130 op/s 2024-04-07T17:08:02.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:08:01 smithi144 bash[66654]: cluster 2024-04-07T17:08:00.687966+0000 mon.a (mon.0) 2800 : cluster [DBG] osdmap e1491: 8 total, 8 up, 8 in 2024-04-07T17:08:02.616 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:08:02 smithi144 bash[95233]: ts=2024-04-07T17:08:02.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:08:03.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:08:02 smithi144 bash[66654]: cluster 2024-04-07T17:08:01.695463+0000 mon.a (mon.0) 2801 : cluster [DBG] osdmap e1492: 8 total, 8 up, 8 in 2024-04-07T17:08:03.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:08:02 smithi144 bash[66654]: audit 2024-04-07T17:08:01.810191+0000 mon.a (mon.0) 2802 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:08:03.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:08:02 smithi144 bash[66654]: cluster 2024-04-07T17:08:02.665466+0000 mon.a (mon.0) 2803 : cluster [DBG] osdmap e1493: 8 total, 8 up, 8 in 2024-04-07T17:08:03.202 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:08:02 smithi100 bash[74942]: cluster 2024-04-07T17:08:01.695463+0000 mon.a (mon.0) 2801 : cluster [DBG] osdmap e1492: 8 total, 8 up, 8 in 2024-04-07T17:08:03.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:08:02 smithi100 bash[74942]: audit 2024-04-07T17:08:01.810191+0000 mon.a (mon.0) 2802 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:08:03.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:08:02 smithi100 bash[74942]: cluster 2024-04-07T17:08:02.665466+0000 mon.a (mon.0) 2803 : cluster [DBG] osdmap e1493: 8 total, 8 up, 8 in 2024-04-07T17:08:03.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:08:02 smithi100 bash[76202]: cluster 2024-04-07T17:08:01.695463+0000 mon.a (mon.0) 2801 : cluster [DBG] osdmap e1492: 8 total, 8 up, 8 in 2024-04-07T17:08:03.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:08:02 smithi100 bash[76202]: audit 2024-04-07T17:08:01.810191+0000 mon.a (mon.0) 2802 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:08:03.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:08:02 smithi100 bash[76202]: cluster 2024-04-07T17:08:02.665466+0000 mon.a (mon.0) 2803 : cluster [DBG] osdmap e1493: 8 total, 8 up, 8 in 2024-04-07T17:08:04.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:08:03 smithi144 bash[66654]: cluster 2024-04-07T17:08:01.958472+0000 mgr.y (mgr.25175) 1580 : cluster [DBG] pgmap v2647: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 39 KiB/s rd, 11 KiB/s wr, 66 op/s 2024-04-07T17:08:04.202 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:08:03 smithi100 bash[74942]: cluster 2024-04-07T17:08:01.958472+0000 mgr.y (mgr.25175) 1580 : cluster [DBG] pgmap v2647: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 39 KiB/s rd, 11 KiB/s wr, 66 op/s 2024-04-07T17:08:04.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:08:03 smithi100 bash[76202]: cluster 2024-04-07T17:08:01.958472+0000 mgr.y (mgr.25175) 1580 : cluster [DBG] pgmap v2647: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 39 KiB/s rd, 11 KiB/s wr, 66 op/s 2024-04-07T17:08:05.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:08:04 smithi144 bash[66654]: cluster 2024-04-07T17:08:03.723205+0000 mon.a (mon.0) 2804 : cluster [DBG] osdmap e1494: 8 total, 8 up, 8 in 2024-04-07T17:08:05.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:08:04 smithi100 bash[74942]: cluster 2024-04-07T17:08:03.723205+0000 mon.a (mon.0) 2804 : cluster [DBG] osdmap e1494: 8 total, 8 up, 8 in 2024-04-07T17:08:05.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:08:04 smithi100 bash[76202]: cluster 2024-04-07T17:08:03.723205+0000 mon.a (mon.0) 2804 : cluster [DBG] osdmap e1494: 8 total, 8 up, 8 in 2024-04-07T17:08:06.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:08:05 smithi144 bash[66654]: cluster 2024-04-07T17:08:03.959167+0000 mgr.y (mgr.25175) 1581 : cluster [DBG] pgmap v2650: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 98 KiB/s rd, 10 KiB/s wr, 131 op/s 2024-04-07T17:08:06.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:08:05 smithi144 bash[66654]: cluster 2024-04-07T17:08:04.737057+0000 mon.a (mon.0) 2805 : cluster [DBG] osdmap e1495: 8 total, 8 up, 8 in 2024-04-07T17:08:06.202 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:08:05 smithi100 bash[74942]: cluster 2024-04-07T17:08:03.959167+0000 mgr.y (mgr.25175) 1581 : cluster [DBG] pgmap v2650: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 98 KiB/s rd, 10 KiB/s wr, 131 op/s 2024-04-07T17:08:06.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:08:05 smithi100 bash[74942]: cluster 2024-04-07T17:08:04.737057+0000 mon.a (mon.0) 2805 : cluster [DBG] osdmap e1495: 8 total, 8 up, 8 in 2024-04-07T17:08:06.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:08:05 smithi100 bash[76202]: cluster 2024-04-07T17:08:03.959167+0000 mgr.y (mgr.25175) 1581 : cluster [DBG] pgmap v2650: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 98 KiB/s rd, 10 KiB/s wr, 131 op/s 2024-04-07T17:08:06.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:08:05 smithi100 bash[76202]: cluster 2024-04-07T17:08:04.737057+0000 mon.a (mon.0) 2805 : cluster [DBG] osdmap e1495: 8 total, 8 up, 8 in 2024-04-07T17:08:07.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:08:06 smithi144 bash[66654]: cluster 2024-04-07T17:08:05.737363+0000 mon.a (mon.0) 2806 : cluster [DBG] osdmap e1496: 8 total, 8 up, 8 in 2024-04-07T17:08:07.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:08:06 smithi100 bash[74942]: cluster 2024-04-07T17:08:05.737363+0000 mon.a (mon.0) 2806 : cluster [DBG] osdmap e1496: 8 total, 8 up, 8 in 2024-04-07T17:08:07.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:08:06 smithi100 bash[76202]: cluster 2024-04-07T17:08:05.737363+0000 mon.a (mon.0) 2806 : cluster [DBG] osdmap e1496: 8 total, 8 up, 8 in 2024-04-07T17:08:08.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:08:07 smithi144 bash[66654]: cluster 2024-04-07T17:08:05.960009+0000 mgr.y (mgr.25175) 1582 : cluster [DBG] pgmap v2653: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 215 KiB/s rd, 18 KiB/s wr, 283 op/s 2024-04-07T17:08:08.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:08:07 smithi144 bash[66654]: cluster 2024-04-07T17:08:06.766659+0000 mon.a (mon.0) 2807 : cluster [DBG] osdmap e1497: 8 total, 8 up, 8 in 2024-04-07T17:08:08.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:08:07 smithi144 bash[66654]: cluster 2024-04-07T17:08:07.670684+0000 mon.a (mon.0) 2808 : cluster [DBG] osdmap e1498: 8 total, 8 up, 8 in 2024-04-07T17:08:08.202 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:08:07 smithi100 bash[74942]: cluster 2024-04-07T17:08:05.960009+0000 mgr.y (mgr.25175) 1582 : cluster [DBG] pgmap v2653: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 215 KiB/s rd, 18 KiB/s wr, 283 op/s 2024-04-07T17:08:08.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:08:07 smithi100 bash[74942]: cluster 2024-04-07T17:08:06.766659+0000 mon.a (mon.0) 2807 : cluster [DBG] osdmap e1497: 8 total, 8 up, 8 in 2024-04-07T17:08:08.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:08:07 smithi100 bash[74942]: cluster 2024-04-07T17:08:07.670684+0000 mon.a (mon.0) 2808 : cluster [DBG] osdmap e1498: 8 total, 8 up, 8 in 2024-04-07T17:08:08.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:08:07 smithi100 bash[76202]: cluster 2024-04-07T17:08:05.960009+0000 mgr.y (mgr.25175) 1582 : cluster [DBG] pgmap v2653: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 215 KiB/s rd, 18 KiB/s wr, 283 op/s 2024-04-07T17:08:08.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:08:07 smithi100 bash[76202]: cluster 2024-04-07T17:08:06.766659+0000 mon.a (mon.0) 2807 : cluster [DBG] osdmap e1497: 8 total, 8 up, 8 in 2024-04-07T17:08:08.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:08:07 smithi100 bash[76202]: cluster 2024-04-07T17:08:07.670684+0000 mon.a (mon.0) 2808 : cluster [DBG] osdmap e1498: 8 total, 8 up, 8 in 2024-04-07T17:08:08.703 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:08:08 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:08:08] "GET /metrics HTTP/1.1" 200 36475 "" "Prometheus/2.43.0" 2024-04-07T17:08:08.819 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestMirroring.SnapshotPromoteDemote (10027 ms) 2024-04-07T17:08:08.819 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestMirroring.AioSnapshotCreate 2024-04-07T17:08:09.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:08:09 smithi100 bash[74942]: cluster 2024-04-07T17:08:07.960899+0000 mgr.y (mgr.25175) 1583 : cluster [DBG] pgmap v2656: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 121 KiB/s rd, 8.5 KiB/s wr, 159 op/s 2024-04-07T17:08:09.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:08:09 smithi100 bash[74942]: cluster 2024-04-07T17:08:08.671737+0000 mon.a (mon.0) 2809 : cluster [DBG] osdmap e1499: 8 total, 8 up, 8 in 2024-04-07T17:08:09.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:08:09 smithi100 bash[74942]: audit 2024-04-07T17:08:08.803076+0000 mon.a (mon.0) 2810 : audit [INF] from='client.? 172.21.15.100:0/3748045073' entity='client.admin' cmd=[{"prefix": "config-key rm", "key": "rbd/mirror/peer/88/01a1ddfb-48dd-4fb3-8922-339ff46030a3"}]: dispatch 2024-04-07T17:08:09.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:08:09 smithi100 bash[74942]: audit 2024-04-07T17:08:08.809394+0000 mon.a (mon.0) 2811 : audit [INF] from='client.? 172.21.15.100:0/3748045073' entity='client.admin' cmd='[{"prefix": "config-key rm", "key": "rbd/mirror/peer/88/01a1ddfb-48dd-4fb3-8922-339ff46030a3"}]': finished 2024-04-07T17:08:09.953 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:08:09 smithi100 bash[76202]: cluster 2024-04-07T17:08:07.960899+0000 mgr.y (mgr.25175) 1583 : cluster [DBG] pgmap v2656: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 121 KiB/s rd, 8.5 KiB/s wr, 159 op/s 2024-04-07T17:08:09.953 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:08:09 smithi100 bash[76202]: cluster 2024-04-07T17:08:08.671737+0000 mon.a (mon.0) 2809 : cluster [DBG] osdmap e1499: 8 total, 8 up, 8 in 2024-04-07T17:08:09.953 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:08:09 smithi100 bash[76202]: audit 2024-04-07T17:08:08.803076+0000 mon.a (mon.0) 2810 : audit [INF] from='client.? 172.21.15.100:0/3748045073' entity='client.admin' cmd=[{"prefix": "config-key rm", "key": "rbd/mirror/peer/88/01a1ddfb-48dd-4fb3-8922-339ff46030a3"}]: dispatch 2024-04-07T17:08:09.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:08:09 smithi100 bash[76202]: audit 2024-04-07T17:08:08.809394+0000 mon.a (mon.0) 2811 : audit [INF] from='client.? 172.21.15.100:0/3748045073' entity='client.admin' cmd='[{"prefix": "config-key rm", "key": "rbd/mirror/peer/88/01a1ddfb-48dd-4fb3-8922-339ff46030a3"}]': finished 2024-04-07T17:08:10.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:08:09 smithi144 bash[66654]: cluster 2024-04-07T17:08:07.960899+0000 mgr.y (mgr.25175) 1583 : cluster [DBG] pgmap v2656: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 121 KiB/s rd, 8.5 KiB/s wr, 159 op/s 2024-04-07T17:08:10.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:08:09 smithi144 bash[66654]: cluster 2024-04-07T17:08:08.671737+0000 mon.a (mon.0) 2809 : cluster [DBG] osdmap e1499: 8 total, 8 up, 8 in 2024-04-07T17:08:10.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:08:09 smithi144 bash[66654]: audit 2024-04-07T17:08:08.803076+0000 mon.a (mon.0) 2810 : audit [INF] from='client.? 172.21.15.100:0/3748045073' entity='client.admin' cmd=[{"prefix": "config-key rm", "key": "rbd/mirror/peer/88/01a1ddfb-48dd-4fb3-8922-339ff46030a3"}]: dispatch 2024-04-07T17:08:10.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:08:09 smithi144 bash[66654]: audit 2024-04-07T17:08:08.809394+0000 mon.a (mon.0) 2811 : audit [INF] from='client.? 172.21.15.100:0/3748045073' entity='client.admin' cmd='[{"prefix": "config-key rm", "key": "rbd/mirror/peer/88/01a1ddfb-48dd-4fb3-8922-339ff46030a3"}]': finished 2024-04-07T17:08:11.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:08:10 smithi144 bash[66654]: cluster 2024-04-07T17:08:09.815511+0000 mon.a (mon.0) 2812 : cluster [DBG] osdmap e1500: 8 total, 8 up, 8 in 2024-04-07T17:08:11.202 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:08:10 smithi100 bash[74942]: cluster 2024-04-07T17:08:09.815511+0000 mon.a (mon.0) 2812 : cluster [DBG] osdmap e1500: 8 total, 8 up, 8 in 2024-04-07T17:08:11.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:08:10 smithi100 bash[76202]: cluster 2024-04-07T17:08:09.815511+0000 mon.a (mon.0) 2812 : cluster [DBG] osdmap e1500: 8 total, 8 up, 8 in 2024-04-07T17:08:12.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:08:11 smithi144 bash[66654]: cluster 2024-04-07T17:08:09.961793+0000 mgr.y (mgr.25175) 1584 : cluster [DBG] pgmap v2659: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 35 KiB/s rd, 6.5 KiB/s wr, 56 op/s 2024-04-07T17:08:12.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:08:11 smithi144 bash[66654]: cluster 2024-04-07T17:08:10.833073+0000 mon.a (mon.0) 2813 : cluster [DBG] osdmap e1501: 8 total, 8 up, 8 in 2024-04-07T17:08:12.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:08:11 smithi100 bash[74942]: cluster 2024-04-07T17:08:09.961793+0000 mgr.y (mgr.25175) 1584 : cluster [DBG] pgmap v2659: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 35 KiB/s rd, 6.5 KiB/s wr, 56 op/s 2024-04-07T17:08:12.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:08:11 smithi100 bash[74942]: cluster 2024-04-07T17:08:10.833073+0000 mon.a (mon.0) 2813 : cluster [DBG] osdmap e1501: 8 total, 8 up, 8 in 2024-04-07T17:08:12.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:08:11 smithi100 bash[76202]: cluster 2024-04-07T17:08:09.961793+0000 mgr.y (mgr.25175) 1584 : cluster [DBG] pgmap v2659: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 35 KiB/s rd, 6.5 KiB/s wr, 56 op/s 2024-04-07T17:08:12.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:08:11 smithi100 bash[76202]: cluster 2024-04-07T17:08:10.833073+0000 mon.a (mon.0) 2813 : cluster [DBG] osdmap e1501: 8 total, 8 up, 8 in 2024-04-07T17:08:12.616 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:08:12 smithi144 bash[95233]: ts=2024-04-07T17:08:12.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:08:13.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:08:12 smithi144 bash[66654]: cluster 2024-04-07T17:08:11.829269+0000 mon.a (mon.0) 2814 : cluster [DBG] osdmap e1502: 8 total, 8 up, 8 in 2024-04-07T17:08:13.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:08:12 smithi144 bash[66654]: cluster 2024-04-07T17:08:12.671411+0000 mon.a (mon.0) 2815 : cluster [DBG] osdmap e1503: 8 total, 8 up, 8 in 2024-04-07T17:08:13.202 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:08:12 smithi100 bash[74942]: cluster 2024-04-07T17:08:11.829269+0000 mon.a (mon.0) 2814 : cluster [DBG] osdmap e1502: 8 total, 8 up, 8 in 2024-04-07T17:08:13.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:08:12 smithi100 bash[74942]: cluster 2024-04-07T17:08:12.671411+0000 mon.a (mon.0) 2815 : cluster [DBG] osdmap e1503: 8 total, 8 up, 8 in 2024-04-07T17:08:13.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:08:12 smithi100 bash[76202]: cluster 2024-04-07T17:08:11.829269+0000 mon.a (mon.0) 2814 : cluster [DBG] osdmap e1502: 8 total, 8 up, 8 in 2024-04-07T17:08:13.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:08:12 smithi100 bash[76202]: cluster 2024-04-07T17:08:12.671411+0000 mon.a (mon.0) 2815 : cluster [DBG] osdmap e1503: 8 total, 8 up, 8 in 2024-04-07T17:08:14.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:08:13 smithi144 bash[66654]: cluster 2024-04-07T17:08:11.962639+0000 mgr.y (mgr.25175) 1585 : cluster [DBG] pgmap v2662: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 31 KiB/s rd, 6.2 KiB/s wr, 48 op/s 2024-04-07T17:08:14.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:08:13 smithi144 bash[66654]: cluster 2024-04-07T17:08:13.674054+0000 mon.a (mon.0) 2816 : cluster [DBG] osdmap e1504: 8 total, 8 up, 8 in 2024-04-07T17:08:14.202 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:08:13 smithi100 bash[74942]: cluster 2024-04-07T17:08:11.962639+0000 mgr.y (mgr.25175) 1585 : cluster [DBG] pgmap v2662: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 31 KiB/s rd, 6.2 KiB/s wr, 48 op/s 2024-04-07T17:08:14.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:08:13 smithi100 bash[74942]: cluster 2024-04-07T17:08:13.674054+0000 mon.a (mon.0) 2816 : cluster [DBG] osdmap e1504: 8 total, 8 up, 8 in 2024-04-07T17:08:14.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:08:13 smithi100 bash[76202]: cluster 2024-04-07T17:08:11.962639+0000 mgr.y (mgr.25175) 1585 : cluster [DBG] pgmap v2662: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 31 KiB/s rd, 6.2 KiB/s wr, 48 op/s 2024-04-07T17:08:14.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:08:13 smithi100 bash[76202]: cluster 2024-04-07T17:08:13.674054+0000 mon.a (mon.0) 2816 : cluster [DBG] osdmap e1504: 8 total, 8 up, 8 in 2024-04-07T17:08:15.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:08:15 smithi100 bash[74942]: cluster 2024-04-07T17:08:13.963569+0000 mgr.y (mgr.25175) 1586 : cluster [DBG] pgmap v2665: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 23 KiB/s rd, 7.0 KiB/s wr, 41 op/s 2024-04-07T17:08:15.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:08:15 smithi100 bash[74942]: cluster 2024-04-07T17:08:14.675846+0000 mon.a (mon.0) 2817 : cluster [DBG] osdmap e1505: 8 total, 8 up, 8 in 2024-04-07T17:08:15.953 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:08:15 smithi100 bash[76202]: cluster 2024-04-07T17:08:13.963569+0000 mgr.y (mgr.25175) 1586 : cluster [DBG] pgmap v2665: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 23 KiB/s rd, 7.0 KiB/s wr, 41 op/s 2024-04-07T17:08:15.953 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:08:15 smithi100 bash[76202]: cluster 2024-04-07T17:08:14.675846+0000 mon.a (mon.0) 2817 : cluster [DBG] osdmap e1505: 8 total, 8 up, 8 in 2024-04-07T17:08:16.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:08:15 smithi144 bash[66654]: cluster 2024-04-07T17:08:13.963569+0000 mgr.y (mgr.25175) 1586 : cluster [DBG] pgmap v2665: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 23 KiB/s rd, 7.0 KiB/s wr, 41 op/s 2024-04-07T17:08:16.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:08:15 smithi144 bash[66654]: cluster 2024-04-07T17:08:14.675846+0000 mon.a (mon.0) 2817 : cluster [DBG] osdmap e1505: 8 total, 8 up, 8 in 2024-04-07T17:08:16.952 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:08:16 smithi100 bash[74942]: cluster 2024-04-07T17:08:15.687832+0000 mon.a (mon.0) 2818 : cluster [DBG] osdmap e1506: 8 total, 8 up, 8 in 2024-04-07T17:08:16.953 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:08:16 smithi100 bash[76202]: cluster 2024-04-07T17:08:15.687832+0000 mon.a (mon.0) 2818 : cluster [DBG] osdmap e1506: 8 total, 8 up, 8 in 2024-04-07T17:08:17.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:08:16 smithi144 bash[66654]: cluster 2024-04-07T17:08:15.687832+0000 mon.a (mon.0) 2818 : cluster [DBG] osdmap e1506: 8 total, 8 up, 8 in 2024-04-07T17:08:17.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:08:17 smithi100 bash[74942]: cluster 2024-04-07T17:08:15.964241+0000 mgr.y (mgr.25175) 1587 : cluster [DBG] pgmap v2668: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 66 KiB/s rd, 12 KiB/s wr, 97 op/s 2024-04-07T17:08:17.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:08:17 smithi100 bash[74942]: cluster 2024-04-07T17:08:16.697360+0000 mon.a (mon.0) 2819 : cluster [DBG] osdmap e1507: 8 total, 8 up, 8 in 2024-04-07T17:08:17.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:08:17 smithi100 bash[74942]: audit 2024-04-07T17:08:16.810610+0000 mon.a (mon.0) 2820 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:08:17.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:08:17 smithi100 bash[74942]: cluster 2024-04-07T17:08:17.678012+0000 mon.a (mon.0) 2821 : cluster [DBG] osdmap e1508: 8 total, 8 up, 8 in 2024-04-07T17:08:17.953 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:08:17 smithi100 bash[76202]: cluster 2024-04-07T17:08:15.964241+0000 mgr.y (mgr.25175) 1587 : cluster [DBG] pgmap v2668: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 66 KiB/s rd, 12 KiB/s wr, 97 op/s 2024-04-07T17:08:17.953 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:08:17 smithi100 bash[76202]: cluster 2024-04-07T17:08:16.697360+0000 mon.a (mon.0) 2819 : cluster [DBG] osdmap e1507: 8 total, 8 up, 8 in 2024-04-07T17:08:17.953 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:08:17 smithi100 bash[76202]: audit 2024-04-07T17:08:16.810610+0000 mon.a (mon.0) 2820 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:08:17.953 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:08:17 smithi100 bash[76202]: cluster 2024-04-07T17:08:17.678012+0000 mon.a (mon.0) 2821 : cluster [DBG] osdmap e1508: 8 total, 8 up, 8 in 2024-04-07T17:08:18.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:08:17 smithi144 bash[66654]: cluster 2024-04-07T17:08:15.964241+0000 mgr.y (mgr.25175) 1587 : cluster [DBG] pgmap v2668: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 66 KiB/s rd, 12 KiB/s wr, 97 op/s 2024-04-07T17:08:18.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:08:17 smithi144 bash[66654]: cluster 2024-04-07T17:08:16.697360+0000 mon.a (mon.0) 2819 : cluster [DBG] osdmap e1507: 8 total, 8 up, 8 in 2024-04-07T17:08:18.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:08:17 smithi144 bash[66654]: audit 2024-04-07T17:08:16.810610+0000 mon.a (mon.0) 2820 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:08:18.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:08:17 smithi144 bash[66654]: cluster 2024-04-07T17:08:17.678012+0000 mon.a (mon.0) 2821 : cluster [DBG] osdmap e1508: 8 total, 8 up, 8 in 2024-04-07T17:08:18.703 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:08:18 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:08:18] "GET /metrics HTTP/1.1" 200 36478 "" "Prometheus/2.43.0" 2024-04-07T17:08:18.866 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 17:08:18 smithi144 bash[98866]: logger=cleanup t=2024-04-07T17:08:18.47560528Z level=info msg="Completed cleanup jobs" duration=56.703275ms 2024-04-07T17:08:20.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:08:19 smithi144 bash[66654]: cluster 2024-04-07T17:08:17.965136+0000 mgr.y (mgr.25175) 1588 : cluster [DBG] pgmap v2671: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 77 KiB/s rd, 10 KiB/s wr, 108 op/s 2024-04-07T17:08:20.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:08:19 smithi144 bash[66654]: cluster 2024-04-07T17:08:18.712710+0000 mon.a (mon.0) 2822 : cluster [DBG] osdmap e1509: 8 total, 8 up, 8 in 2024-04-07T17:08:20.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:08:19 smithi100 bash[74942]: cluster 2024-04-07T17:08:17.965136+0000 mgr.y (mgr.25175) 1588 : cluster [DBG] pgmap v2671: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 77 KiB/s rd, 10 KiB/s wr, 108 op/s 2024-04-07T17:08:20.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:08:19 smithi100 bash[74942]: cluster 2024-04-07T17:08:18.712710+0000 mon.a (mon.0) 2822 : cluster [DBG] osdmap e1509: 8 total, 8 up, 8 in 2024-04-07T17:08:20.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:08:19 smithi100 bash[76202]: cluster 2024-04-07T17:08:17.965136+0000 mgr.y (mgr.25175) 1588 : cluster [DBG] pgmap v2671: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 77 KiB/s rd, 10 KiB/s wr, 108 op/s 2024-04-07T17:08:20.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:08:19 smithi100 bash[76202]: cluster 2024-04-07T17:08:18.712710+0000 mon.a (mon.0) 2822 : cluster [DBG] osdmap e1509: 8 total, 8 up, 8 in 2024-04-07T17:08:21.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:08:20 smithi144 bash[66654]: cluster 2024-04-07T17:08:19.721478+0000 mon.a (mon.0) 2823 : cluster [DBG] osdmap e1510: 8 total, 8 up, 8 in 2024-04-07T17:08:21.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:08:20 smithi100 bash[74942]: cluster 2024-04-07T17:08:19.721478+0000 mon.a (mon.0) 2823 : cluster [DBG] osdmap e1510: 8 total, 8 up, 8 in 2024-04-07T17:08:21.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:08:20 smithi100 bash[76202]: cluster 2024-04-07T17:08:19.721478+0000 mon.a (mon.0) 2823 : cluster [DBG] osdmap e1510: 8 total, 8 up, 8 in 2024-04-07T17:08:22.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:08:21 smithi144 bash[66654]: cluster 2024-04-07T17:08:19.966022+0000 mgr.y (mgr.25175) 1589 : cluster [DBG] pgmap v2674: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 111 KiB/s rd, 16 KiB/s wr, 154 op/s 2024-04-07T17:08:22.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:08:21 smithi144 bash[66654]: cluster 2024-04-07T17:08:20.737042+0000 mon.a (mon.0) 2824 : cluster [DBG] osdmap e1511: 8 total, 8 up, 8 in 2024-04-07T17:08:22.202 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:08:21 smithi100 bash[74942]: cluster 2024-04-07T17:08:19.966022+0000 mgr.y (mgr.25175) 1589 : cluster [DBG] pgmap v2674: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 111 KiB/s rd, 16 KiB/s wr, 154 op/s 2024-04-07T17:08:22.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:08:21 smithi100 bash[74942]: cluster 2024-04-07T17:08:20.737042+0000 mon.a (mon.0) 2824 : cluster [DBG] osdmap e1511: 8 total, 8 up, 8 in 2024-04-07T17:08:22.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:08:21 smithi100 bash[76202]: cluster 2024-04-07T17:08:19.966022+0000 mgr.y (mgr.25175) 1589 : cluster [DBG] pgmap v2674: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 111 KiB/s rd, 16 KiB/s wr, 154 op/s 2024-04-07T17:08:22.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:08:21 smithi100 bash[76202]: cluster 2024-04-07T17:08:20.737042+0000 mon.a (mon.0) 2824 : cluster [DBG] osdmap e1511: 8 total, 8 up, 8 in 2024-04-07T17:08:22.616 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:08:22 smithi144 bash[95233]: ts=2024-04-07T17:08:22.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:08:23.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:08:22 smithi144 bash[66654]: cluster 2024-04-07T17:08:21.745413+0000 mon.a (mon.0) 2825 : cluster [DBG] osdmap e1512: 8 total, 8 up, 8 in 2024-04-07T17:08:23.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:08:22 smithi100 bash[74942]: cluster 2024-04-07T17:08:21.745413+0000 mon.a (mon.0) 2825 : cluster [DBG] osdmap e1512: 8 total, 8 up, 8 in 2024-04-07T17:08:23.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:08:22 smithi100 bash[76202]: cluster 2024-04-07T17:08:21.745413+0000 mon.a (mon.0) 2825 : cluster [DBG] osdmap e1512: 8 total, 8 up, 8 in 2024-04-07T17:08:24.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:08:23 smithi144 bash[66654]: cluster 2024-04-07T17:08:21.966722+0000 mgr.y (mgr.25175) 1590 : cluster [DBG] pgmap v2677: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 77 KiB/s rd, 11 KiB/s wr, 101 op/s 2024-04-07T17:08:24.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:08:23 smithi144 bash[66654]: cluster 2024-04-07T17:08:22.760206+0000 mon.a (mon.0) 2826 : cluster [DBG] osdmap e1513: 8 total, 8 up, 8 in 2024-04-07T17:08:24.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:08:23 smithi100 bash[74942]: cluster 2024-04-07T17:08:21.966722+0000 mgr.y (mgr.25175) 1590 : cluster [DBG] pgmap v2677: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 77 KiB/s rd, 11 KiB/s wr, 101 op/s 2024-04-07T17:08:24.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:08:23 smithi100 bash[74942]: cluster 2024-04-07T17:08:22.760206+0000 mon.a (mon.0) 2826 : cluster [DBG] osdmap e1513: 8 total, 8 up, 8 in 2024-04-07T17:08:24.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:08:23 smithi100 bash[76202]: cluster 2024-04-07T17:08:21.966722+0000 mgr.y (mgr.25175) 1590 : cluster [DBG] pgmap v2677: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 77 KiB/s rd, 11 KiB/s wr, 101 op/s 2024-04-07T17:08:24.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:08:23 smithi100 bash[76202]: cluster 2024-04-07T17:08:22.760206+0000 mon.a (mon.0) 2826 : cluster [DBG] osdmap e1513: 8 total, 8 up, 8 in 2024-04-07T17:08:25.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:08:24 smithi144 bash[66654]: cluster 2024-04-07T17:08:23.767683+0000 mon.a (mon.0) 2827 : cluster [DBG] osdmap e1514: 8 total, 8 up, 8 in 2024-04-07T17:08:25.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:08:24 smithi100 bash[74942]: cluster 2024-04-07T17:08:23.767683+0000 mon.a (mon.0) 2827 : cluster [DBG] osdmap e1514: 8 total, 8 up, 8 in 2024-04-07T17:08:25.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:08:24 smithi100 bash[76202]: cluster 2024-04-07T17:08:23.767683+0000 mon.a (mon.0) 2827 : cluster [DBG] osdmap e1514: 8 total, 8 up, 8 in 2024-04-07T17:08:26.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:08:25 smithi144 bash[66654]: cluster 2024-04-07T17:08:23.967658+0000 mgr.y (mgr.25175) 1591 : cluster [DBG] pgmap v2680: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 98 KiB/s rd, 13 KiB/s wr, 137 op/s 2024-04-07T17:08:26.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:08:25 smithi144 bash[66654]: cluster 2024-04-07T17:08:24.781131+0000 mon.a (mon.0) 2828 : cluster [DBG] osdmap e1515: 8 total, 8 up, 8 in 2024-04-07T17:08:26.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:08:25 smithi100 bash[74942]: cluster 2024-04-07T17:08:23.967658+0000 mgr.y (mgr.25175) 1591 : cluster [DBG] pgmap v2680: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 98 KiB/s rd, 13 KiB/s wr, 137 op/s 2024-04-07T17:08:26.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:08:25 smithi100 bash[74942]: cluster 2024-04-07T17:08:24.781131+0000 mon.a (mon.0) 2828 : cluster [DBG] osdmap e1515: 8 total, 8 up, 8 in 2024-04-07T17:08:26.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:08:25 smithi100 bash[76202]: cluster 2024-04-07T17:08:23.967658+0000 mgr.y (mgr.25175) 1591 : cluster [DBG] pgmap v2680: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 98 KiB/s rd, 13 KiB/s wr, 137 op/s 2024-04-07T17:08:26.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:08:25 smithi100 bash[76202]: cluster 2024-04-07T17:08:24.781131+0000 mon.a (mon.0) 2828 : cluster [DBG] osdmap e1515: 8 total, 8 up, 8 in 2024-04-07T17:08:27.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:08:26 smithi144 bash[66654]: cluster 2024-04-07T17:08:25.783671+0000 mon.a (mon.0) 2829 : cluster [DBG] osdmap e1516: 8 total, 8 up, 8 in 2024-04-07T17:08:27.202 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:08:26 smithi100 bash[74942]: cluster 2024-04-07T17:08:25.783671+0000 mon.a (mon.0) 2829 : cluster [DBG] osdmap e1516: 8 total, 8 up, 8 in 2024-04-07T17:08:27.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:08:26 smithi100 bash[76202]: cluster 2024-04-07T17:08:25.783671+0000 mon.a (mon.0) 2829 : cluster [DBG] osdmap e1516: 8 total, 8 up, 8 in 2024-04-07T17:08:28.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:08:27 smithi144 bash[66654]: cluster 2024-04-07T17:08:25.968428+0000 mgr.y (mgr.25175) 1592 : cluster [DBG] pgmap v2683: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 169 KiB/s rd, 18 KiB/s wr, 233 op/s 2024-04-07T17:08:28.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:08:27 smithi144 bash[66654]: cluster 2024-04-07T17:08:26.811447+0000 mon.a (mon.0) 2830 : cluster [DBG] osdmap e1517: 8 total, 8 up, 8 in 2024-04-07T17:08:28.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:08:27 smithi144 bash[66654]: cluster 2024-04-07T17:08:27.681985+0000 mon.a (mon.0) 2831 : cluster [DBG] osdmap e1518: 8 total, 8 up, 8 in 2024-04-07T17:08:28.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:08:27 smithi100 bash[74942]: cluster 2024-04-07T17:08:25.968428+0000 mgr.y (mgr.25175) 1592 : cluster [DBG] pgmap v2683: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 169 KiB/s rd, 18 KiB/s wr, 233 op/s 2024-04-07T17:08:28.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:08:27 smithi100 bash[74942]: cluster 2024-04-07T17:08:26.811447+0000 mon.a (mon.0) 2830 : cluster [DBG] osdmap e1517: 8 total, 8 up, 8 in 2024-04-07T17:08:28.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:08:27 smithi100 bash[74942]: cluster 2024-04-07T17:08:27.681985+0000 mon.a (mon.0) 2831 : cluster [DBG] osdmap e1518: 8 total, 8 up, 8 in 2024-04-07T17:08:28.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:08:27 smithi100 bash[76202]: cluster 2024-04-07T17:08:25.968428+0000 mgr.y (mgr.25175) 1592 : cluster [DBG] pgmap v2683: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 169 KiB/s rd, 18 KiB/s wr, 233 op/s 2024-04-07T17:08:28.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:08:27 smithi100 bash[76202]: cluster 2024-04-07T17:08:26.811447+0000 mon.a (mon.0) 2830 : cluster [DBG] osdmap e1517: 8 total, 8 up, 8 in 2024-04-07T17:08:28.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:08:27 smithi100 bash[76202]: cluster 2024-04-07T17:08:27.681985+0000 mon.a (mon.0) 2831 : cluster [DBG] osdmap e1518: 8 total, 8 up, 8 in 2024-04-07T17:08:28.702 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:08:28 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:08:28] "GET /metrics HTTP/1.1" 200 36478 "" "Prometheus/2.43.0" 2024-04-07T17:08:30.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:08:29 smithi144 bash[66654]: cluster 2024-04-07T17:08:27.969408+0000 mgr.y (mgr.25175) 1593 : cluster [DBG] pgmap v2686: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 87 KiB/s rd, 7.5 KiB/s wr, 121 op/s 2024-04-07T17:08:30.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:08:29 smithi144 bash[66654]: cluster 2024-04-07T17:08:28.843532+0000 mon.a (mon.0) 2832 : cluster [DBG] osdmap e1519: 8 total, 8 up, 8 in 2024-04-07T17:08:30.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:08:29 smithi100 bash[74942]: cluster 2024-04-07T17:08:27.969408+0000 mgr.y (mgr.25175) 1593 : cluster [DBG] pgmap v2686: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 87 KiB/s rd, 7.5 KiB/s wr, 121 op/s 2024-04-07T17:08:30.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:08:29 smithi100 bash[74942]: cluster 2024-04-07T17:08:28.843532+0000 mon.a (mon.0) 2832 : cluster [DBG] osdmap e1519: 8 total, 8 up, 8 in 2024-04-07T17:08:30.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:08:29 smithi100 bash[76202]: cluster 2024-04-07T17:08:27.969408+0000 mgr.y (mgr.25175) 1593 : cluster [DBG] pgmap v2686: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 87 KiB/s rd, 7.5 KiB/s wr, 121 op/s 2024-04-07T17:08:30.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:08:29 smithi100 bash[76202]: cluster 2024-04-07T17:08:28.843532+0000 mon.a (mon.0) 2832 : cluster [DBG] osdmap e1519: 8 total, 8 up, 8 in 2024-04-07T17:08:31.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:08:30 smithi144 bash[66654]: cluster 2024-04-07T17:08:29.845724+0000 mon.a (mon.0) 2833 : cluster [DBG] osdmap e1520: 8 total, 8 up, 8 in 2024-04-07T17:08:31.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:08:30 smithi100 bash[74942]: cluster 2024-04-07T17:08:29.845724+0000 mon.a (mon.0) 2833 : cluster [DBG] osdmap e1520: 8 total, 8 up, 8 in 2024-04-07T17:08:31.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:08:30 smithi100 bash[76202]: cluster 2024-04-07T17:08:29.845724+0000 mon.a (mon.0) 2833 : cluster [DBG] osdmap e1520: 8 total, 8 up, 8 in 2024-04-07T17:08:32.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:08:31 smithi144 bash[66654]: cluster 2024-04-07T17:08:29.970437+0000 mgr.y (mgr.25175) 1594 : cluster [DBG] pgmap v2689: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 66 KiB/s rd, 6.5 KiB/s wr, 100 op/s 2024-04-07T17:08:32.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:08:31 smithi144 bash[66654]: cluster 2024-04-07T17:08:30.869556+0000 mon.a (mon.0) 2834 : cluster [DBG] osdmap e1521: 8 total, 8 up, 8 in 2024-04-07T17:08:32.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:08:31 smithi144 bash[66654]: audit 2024-04-07T17:08:31.811273+0000 mon.a (mon.0) 2835 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:08:32.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:08:31 smithi100 bash[74942]: cluster 2024-04-07T17:08:29.970437+0000 mgr.y (mgr.25175) 1594 : cluster [DBG] pgmap v2689: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 66 KiB/s rd, 6.5 KiB/s wr, 100 op/s 2024-04-07T17:08:32.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:08:31 smithi100 bash[74942]: cluster 2024-04-07T17:08:30.869556+0000 mon.a (mon.0) 2834 : cluster [DBG] osdmap e1521: 8 total, 8 up, 8 in 2024-04-07T17:08:32.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:08:31 smithi100 bash[74942]: audit 2024-04-07T17:08:31.811273+0000 mon.a (mon.0) 2835 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:08:32.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:08:31 smithi100 bash[76202]: cluster 2024-04-07T17:08:29.970437+0000 mgr.y (mgr.25175) 1594 : cluster [DBG] pgmap v2689: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 66 KiB/s rd, 6.5 KiB/s wr, 100 op/s 2024-04-07T17:08:32.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:08:31 smithi100 bash[76202]: cluster 2024-04-07T17:08:30.869556+0000 mon.a (mon.0) 2834 : cluster [DBG] osdmap e1521: 8 total, 8 up, 8 in 2024-04-07T17:08:32.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:08:31 smithi100 bash[76202]: audit 2024-04-07T17:08:31.811273+0000 mon.a (mon.0) 2835 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:08:32.616 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:08:32 smithi144 bash[95233]: ts=2024-04-07T17:08:32.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:08:33.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:08:32 smithi100 bash[74942]: cluster 2024-04-07T17:08:31.881632+0000 mon.a (mon.0) 2836 : cluster [DBG] osdmap e1522: 8 total, 8 up, 8 in 2024-04-07T17:08:33.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:08:32 smithi100 bash[74942]: cluster 2024-04-07T17:08:32.685381+0000 mon.a (mon.0) 2837 : cluster [DBG] osdmap e1523: 8 total, 8 up, 8 in 2024-04-07T17:08:33.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:08:32 smithi100 bash[76202]: cluster 2024-04-07T17:08:31.881632+0000 mon.a (mon.0) 2836 : cluster [DBG] osdmap e1522: 8 total, 8 up, 8 in 2024-04-07T17:08:33.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:08:32 smithi100 bash[76202]: cluster 2024-04-07T17:08:32.685381+0000 mon.a (mon.0) 2837 : cluster [DBG] osdmap e1523: 8 total, 8 up, 8 in 2024-04-07T17:08:33.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:08:32 smithi144 bash[66654]: cluster 2024-04-07T17:08:31.881632+0000 mon.a (mon.0) 2836 : cluster [DBG] osdmap e1522: 8 total, 8 up, 8 in 2024-04-07T17:08:33.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:08:32 smithi144 bash[66654]: cluster 2024-04-07T17:08:32.685381+0000 mon.a (mon.0) 2837 : cluster [DBG] osdmap e1523: 8 total, 8 up, 8 in 2024-04-07T17:08:34.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:08:33 smithi100 bash[74942]: cluster 2024-04-07T17:08:31.971072+0000 mgr.y (mgr.25175) 1595 : cluster [DBG] pgmap v2692: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 50 KiB/s rd, 3.7 KiB/s wr, 73 op/s 2024-04-07T17:08:34.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:08:33 smithi100 bash[74942]: cluster 2024-04-07T17:08:33.690400+0000 mon.a (mon.0) 2838 : cluster [DBG] osdmap e1524: 8 total, 8 up, 8 in 2024-04-07T17:08:34.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:08:33 smithi100 bash[76202]: cluster 2024-04-07T17:08:31.971072+0000 mgr.y (mgr.25175) 1595 : cluster [DBG] pgmap v2692: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 50 KiB/s rd, 3.7 KiB/s wr, 73 op/s 2024-04-07T17:08:34.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:08:33 smithi100 bash[76202]: cluster 2024-04-07T17:08:33.690400+0000 mon.a (mon.0) 2838 : cluster [DBG] osdmap e1524: 8 total, 8 up, 8 in 2024-04-07T17:08:34.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:08:33 smithi144 bash[66654]: cluster 2024-04-07T17:08:31.971072+0000 mgr.y (mgr.25175) 1595 : cluster [DBG] pgmap v2692: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 50 KiB/s rd, 3.7 KiB/s wr, 73 op/s 2024-04-07T17:08:34.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:08:33 smithi144 bash[66654]: cluster 2024-04-07T17:08:33.690400+0000 mon.a (mon.0) 2838 : cluster [DBG] osdmap e1524: 8 total, 8 up, 8 in 2024-04-07T17:08:35.202 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:08:34 smithi100 bash[74942]: cluster 2024-04-07T17:08:33.971951+0000 mgr.y (mgr.25175) 1596 : cluster [DBG] pgmap v2695: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 112 KiB/s rd, 9.2 KiB/s wr, 158 op/s 2024-04-07T17:08:35.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:08:34 smithi100 bash[76202]: cluster 2024-04-07T17:08:33.971951+0000 mgr.y (mgr.25175) 1596 : cluster [DBG] pgmap v2695: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 112 KiB/s rd, 9.2 KiB/s wr, 158 op/s 2024-04-07T17:08:35.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:08:34 smithi144 bash[66654]: cluster 2024-04-07T17:08:33.971951+0000 mgr.y (mgr.25175) 1596 : cluster [DBG] pgmap v2695: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 112 KiB/s rd, 9.2 KiB/s wr, 158 op/s 2024-04-07T17:08:36.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:08:35 smithi100 bash[74942]: cluster 2024-04-07T17:08:34.905522+0000 mon.a (mon.0) 2839 : cluster [DBG] osdmap e1525: 8 total, 8 up, 8 in 2024-04-07T17:08:36.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:08:35 smithi100 bash[76202]: cluster 2024-04-07T17:08:34.905522+0000 mon.a (mon.0) 2839 : cluster [DBG] osdmap e1525: 8 total, 8 up, 8 in 2024-04-07T17:08:36.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:08:35 smithi144 bash[66654]: cluster 2024-04-07T17:08:34.905522+0000 mon.a (mon.0) 2839 : cluster [DBG] osdmap e1525: 8 total, 8 up, 8 in 2024-04-07T17:08:37.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:08:36 smithi100 bash[74942]: cluster 2024-04-07T17:08:35.917359+0000 mon.a (mon.0) 2840 : cluster [DBG] osdmap e1526: 8 total, 8 up, 8 in 2024-04-07T17:08:37.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:08:36 smithi100 bash[74942]: cluster 2024-04-07T17:08:35.972561+0000 mgr.y (mgr.25175) 1597 : cluster [DBG] pgmap v2698: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 119 KiB/s rd, 9.2 KiB/s wr, 168 op/s 2024-04-07T17:08:37.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:08:37 smithi100 bash[76202]: cluster 2024-04-07T17:08:35.917359+0000 mon.a (mon.0) 2840 : cluster [DBG] osdmap e1526: 8 total, 8 up, 8 in 2024-04-07T17:08:37.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:08:37 smithi100 bash[76202]: cluster 2024-04-07T17:08:35.972561+0000 mgr.y (mgr.25175) 1597 : cluster [DBG] pgmap v2698: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 119 KiB/s rd, 9.2 KiB/s wr, 168 op/s 2024-04-07T17:08:37.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:08:36 smithi144 bash[66654]: cluster 2024-04-07T17:08:35.917359+0000 mon.a (mon.0) 2840 : cluster [DBG] osdmap e1526: 8 total, 8 up, 8 in 2024-04-07T17:08:37.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:08:36 smithi144 bash[66654]: cluster 2024-04-07T17:08:35.972561+0000 mgr.y (mgr.25175) 1597 : cluster [DBG] pgmap v2698: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 119 KiB/s rd, 9.2 KiB/s wr, 168 op/s 2024-04-07T17:08:38.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:08:37 smithi100 bash[74942]: cluster 2024-04-07T17:08:36.932937+0000 mon.a (mon.0) 2841 : cluster [DBG] osdmap e1527: 8 total, 8 up, 8 in 2024-04-07T17:08:38.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:08:37 smithi100 bash[74942]: cluster 2024-04-07T17:08:37.686399+0000 mon.a (mon.0) 2842 : cluster [DBG] osdmap e1528: 8 total, 8 up, 8 in 2024-04-07T17:08:38.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:08:37 smithi100 bash[76202]: cluster 2024-04-07T17:08:36.932937+0000 mon.a (mon.0) 2841 : cluster [DBG] osdmap e1527: 8 total, 8 up, 8 in 2024-04-07T17:08:38.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:08:37 smithi100 bash[76202]: cluster 2024-04-07T17:08:37.686399+0000 mon.a (mon.0) 2842 : cluster [DBG] osdmap e1528: 8 total, 8 up, 8 in 2024-04-07T17:08:38.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:08:37 smithi144 bash[66654]: cluster 2024-04-07T17:08:36.932937+0000 mon.a (mon.0) 2841 : cluster [DBG] osdmap e1527: 8 total, 8 up, 8 in 2024-04-07T17:08:38.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:08:37 smithi144 bash[66654]: cluster 2024-04-07T17:08:37.686399+0000 mon.a (mon.0) 2842 : cluster [DBG] osdmap e1528: 8 total, 8 up, 8 in 2024-04-07T17:08:38.702 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:08:38 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:08:38] "GET /metrics HTTP/1.1" 200 36472 "" "Prometheus/2.43.0" 2024-04-07T17:08:39.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:08:38 smithi100 bash[74942]: cluster 2024-04-07T17:08:37.973486+0000 mgr.y (mgr.25175) 1598 : cluster [DBG] pgmap v2701: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 38 KiB/s rd, 3.2 KiB/s wr, 57 op/s 2024-04-07T17:08:39.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:08:38 smithi100 bash[76202]: cluster 2024-04-07T17:08:37.973486+0000 mgr.y (mgr.25175) 1598 : cluster [DBG] pgmap v2701: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 38 KiB/s rd, 3.2 KiB/s wr, 57 op/s 2024-04-07T17:08:39.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:08:38 smithi144 bash[66654]: cluster 2024-04-07T17:08:37.973486+0000 mgr.y (mgr.25175) 1598 : cluster [DBG] pgmap v2701: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 38 KiB/s rd, 3.2 KiB/s wr, 57 op/s 2024-04-07T17:08:40.070 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestMirroring.AioSnapshotCreate (31251 ms) 2024-04-07T17:08:40.202 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:08:39 smithi100 bash[74942]: cluster 2024-04-07T17:08:38.943006+0000 mon.a (mon.0) 2843 : cluster [DBG] osdmap e1529: 8 total, 8 up, 8 in 2024-04-07T17:08:40.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:08:39 smithi100 bash[76202]: cluster 2024-04-07T17:08:38.943006+0000 mon.a (mon.0) 2843 : cluster [DBG] osdmap e1529: 8 total, 8 up, 8 in 2024-04-07T17:08:40.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:08:39 smithi144 bash[66654]: cluster 2024-04-07T17:08:38.943006+0000 mon.a (mon.0) 2843 : cluster [DBG] osdmap e1529: 8 total, 8 up, 8 in 2024-04-07T17:08:41.072 INFO:tasks.workunit.client.0.smithi100.stdout:[----------] 51 tests from TestMirroring (91752 ms total) 2024-04-07T17:08:41.072 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:08:41.073 INFO:tasks.workunit.client.0.smithi100.stdout:[----------] 34 tests from TestInternal 2024-04-07T17:08:41.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:08:40 smithi144 bash[66654]: cluster 2024-04-07T17:08:39.940742+0000 mon.a (mon.0) 2844 : cluster [DBG] osdmap e1530: 8 total, 8 up, 8 in 2024-04-07T17:08:41.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:08:40 smithi144 bash[66654]: cluster 2024-04-07T17:08:39.974903+0000 mgr.y (mgr.25175) 1599 : cluster [DBG] pgmap v2704: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 82 KiB/s rd, 7.5 KiB/s wr, 121 op/s 2024-04-07T17:08:41.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:08:40 smithi144 bash[66654]: audit 2024-04-07T17:08:40.054416+0000 mon.a (mon.0) 2845 : audit [INF] from='client.? 172.21.15.100:0/3748045073' entity='client.admin' cmd=[{"prefix": "config-key rm", "key": "rbd/mirror/peer/88/fb9927a4-5eea-4563-a22f-801b91fb7b73"}]: dispatch 2024-04-07T17:08:41.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:08:40 smithi144 bash[66654]: audit 2024-04-07T17:08:40.060113+0000 mon.a (mon.0) 2846 : audit [INF] from='client.? 172.21.15.100:0/3748045073' entity='client.admin' cmd='[{"prefix": "config-key rm", "key": "rbd/mirror/peer/88/fb9927a4-5eea-4563-a22f-801b91fb7b73"}]': finished 2024-04-07T17:08:41.453 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:08:40 smithi100 bash[74942]: cluster 2024-04-07T17:08:39.940742+0000 mon.a (mon.0) 2844 : cluster [DBG] osdmap e1530: 8 total, 8 up, 8 in 2024-04-07T17:08:41.453 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:08:40 smithi100 bash[74942]: cluster 2024-04-07T17:08:39.974903+0000 mgr.y (mgr.25175) 1599 : cluster [DBG] pgmap v2704: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 82 KiB/s rd, 7.5 KiB/s wr, 121 op/s 2024-04-07T17:08:41.453 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:08:40 smithi100 bash[74942]: audit 2024-04-07T17:08:40.054416+0000 mon.a (mon.0) 2845 : audit [INF] from='client.? 172.21.15.100:0/3748045073' entity='client.admin' cmd=[{"prefix": "config-key rm", "key": "rbd/mirror/peer/88/fb9927a4-5eea-4563-a22f-801b91fb7b73"}]: dispatch 2024-04-07T17:08:41.453 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:08:40 smithi100 bash[74942]: audit 2024-04-07T17:08:40.060113+0000 mon.a (mon.0) 2846 : audit [INF] from='client.? 172.21.15.100:0/3748045073' entity='client.admin' cmd='[{"prefix": "config-key rm", "key": "rbd/mirror/peer/88/fb9927a4-5eea-4563-a22f-801b91fb7b73"}]': finished 2024-04-07T17:08:41.453 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:08:40 smithi100 bash[76202]: cluster 2024-04-07T17:08:39.940742+0000 mon.a (mon.0) 2844 : cluster [DBG] osdmap e1530: 8 total, 8 up, 8 in 2024-04-07T17:08:41.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:08:40 smithi100 bash[76202]: cluster 2024-04-07T17:08:39.974903+0000 mgr.y (mgr.25175) 1599 : cluster [DBG] pgmap v2704: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 82 KiB/s rd, 7.5 KiB/s wr, 121 op/s 2024-04-07T17:08:41.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:08:40 smithi100 bash[76202]: audit 2024-04-07T17:08:40.054416+0000 mon.a (mon.0) 2845 : audit [INF] from='client.? 172.21.15.100:0/3748045073' entity='client.admin' cmd=[{"prefix": "config-key rm", "key": "rbd/mirror/peer/88/fb9927a4-5eea-4563-a22f-801b91fb7b73"}]: dispatch 2024-04-07T17:08:41.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:08:40 smithi100 bash[76202]: audit 2024-04-07T17:08:40.060113+0000 mon.a (mon.0) 2846 : audit [INF] from='client.? 172.21.15.100:0/3748045073' entity='client.admin' cmd='[{"prefix": "config-key rm", "key": "rbd/mirror/peer/88/fb9927a4-5eea-4563-a22f-801b91fb7b73"}]': finished 2024-04-07T17:08:42.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:08:42 smithi144 bash[66654]: cluster 2024-04-07T17:08:41.065871+0000 mon.a (mon.0) 2847 : cluster [DBG] osdmap e1531: 8 total, 8 up, 8 in 2024-04-07T17:08:42.366 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:08:42 smithi144 bash[95233]: ts=2024-04-07T17:08:42.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:08:42.452 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:08:42 smithi100 bash[74942]: cluster 2024-04-07T17:08:41.065871+0000 mon.a (mon.0) 2847 : cluster [DBG] osdmap e1531: 8 total, 8 up, 8 in 2024-04-07T17:08:42.453 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:08:42 smithi100 bash[76202]: cluster 2024-04-07T17:08:41.065871+0000 mon.a (mon.0) 2847 : cluster [DBG] osdmap e1531: 8 total, 8 up, 8 in 2024-04-07T17:08:43.092 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestInternal.OpenByID 2024-04-07T17:08:43.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:08:43 smithi144 bash[66654]: cluster 2024-04-07T17:08:41.975698+0000 mgr.y (mgr.25175) 1600 : cluster [DBG] pgmap v2706: 105 pgs: 105 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail 2024-04-07T17:08:43.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:08:43 smithi144 bash[66654]: cluster 2024-04-07T17:08:42.082641+0000 mon.a (mon.0) 2848 : cluster [DBG] osdmap e1532: 8 total, 8 up, 8 in 2024-04-07T17:08:43.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:08:43 smithi144 bash[66654]: audit 2024-04-07T17:08:42.085437+0000 mon.c (mon.2) 65 : audit [INF] from='client.? 172.21.15.100:0/137326563' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:08:43.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:08:43 smithi144 bash[66654]: audit 2024-04-07T17:08:42.088722+0000 mon.a (mon.0) 2849 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:08:43.453 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:08:43 smithi100 bash[74942]: cluster 2024-04-07T17:08:41.975698+0000 mgr.y (mgr.25175) 1600 : cluster [DBG] pgmap v2706: 105 pgs: 105 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail 2024-04-07T17:08:43.453 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:08:43 smithi100 bash[74942]: cluster 2024-04-07T17:08:42.082641+0000 mon.a (mon.0) 2848 : cluster [DBG] osdmap e1532: 8 total, 8 up, 8 in 2024-04-07T17:08:43.453 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:08:43 smithi100 bash[74942]: audit 2024-04-07T17:08:42.085437+0000 mon.c (mon.2) 65 : audit [INF] from='client.? 172.21.15.100:0/137326563' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:08:43.453 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:08:43 smithi100 bash[74942]: audit 2024-04-07T17:08:42.088722+0000 mon.a (mon.0) 2849 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:08:43.453 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:08:43 smithi100 bash[76202]: cluster 2024-04-07T17:08:41.975698+0000 mgr.y (mgr.25175) 1600 : cluster [DBG] pgmap v2706: 105 pgs: 105 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail 2024-04-07T17:08:43.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:08:43 smithi100 bash[76202]: cluster 2024-04-07T17:08:42.082641+0000 mon.a (mon.0) 2848 : cluster [DBG] osdmap e1532: 8 total, 8 up, 8 in 2024-04-07T17:08:43.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:08:43 smithi100 bash[76202]: audit 2024-04-07T17:08:42.085437+0000 mon.c (mon.2) 65 : audit [INF] from='client.? 172.21.15.100:0/137326563' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:08:43.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:08:43 smithi100 bash[76202]: audit 2024-04-07T17:08:42.088722+0000 mon.a (mon.0) 2849 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:08:44.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:08:44 smithi144 bash[66654]: audit 2024-04-07T17:08:43.083232+0000 mon.a (mon.0) 2850 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T17:08:44.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:08:44 smithi144 bash[66654]: cluster 2024-04-07T17:08:43.090728+0000 mon.a (mon.0) 2851 : cluster [DBG] osdmap e1533: 8 total, 8 up, 8 in 2024-04-07T17:08:44.453 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:08:44 smithi100 bash[74942]: audit 2024-04-07T17:08:43.083232+0000 mon.a (mon.0) 2850 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T17:08:44.453 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:08:44 smithi100 bash[74942]: cluster 2024-04-07T17:08:43.090728+0000 mon.a (mon.0) 2851 : cluster [DBG] osdmap e1533: 8 total, 8 up, 8 in 2024-04-07T17:08:44.453 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:08:44 smithi100 bash[76202]: audit 2024-04-07T17:08:43.083232+0000 mon.a (mon.0) 2850 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T17:08:44.453 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:08:44 smithi100 bash[76202]: cluster 2024-04-07T17:08:43.090728+0000 mon.a (mon.0) 2851 : cluster [DBG] osdmap e1533: 8 total, 8 up, 8 in 2024-04-07T17:08:45.195 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestInternal.OpenByID (2103 ms) 2024-04-07T17:08:45.195 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestInternal.OpenSnapDNE 2024-04-07T17:08:45.237 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestInternal.OpenSnapDNE (43 ms) 2024-04-07T17:08:45.238 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestInternal.IsExclusiveLockOwner 2024-04-07T17:08:45.296 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestInternal.IsExclusiveLockOwner (58 ms) 2024-04-07T17:08:45.296 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestInternal.ResizeLocksImage 2024-04-07T17:08:45.361 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestInternal.ResizeLocksImage (65 ms) 2024-04-07T17:08:45.361 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestInternal.ResizeFailsToLockImage 2024-04-07T17:08:45.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:08:45 smithi144 bash[66654]: cluster 2024-04-07T17:08:43.976607+0000 mgr.y (mgr.25175) 1601 : cluster [DBG] pgmap v2709: 137 pgs: 5 creating+peering, 26 unknown, 106 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail 2024-04-07T17:08:45.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:08:45 smithi144 bash[66654]: cluster 2024-04-07T17:08:44.104321+0000 mon.a (mon.0) 2852 : cluster [DBG] osdmap e1534: 8 total, 8 up, 8 in 2024-04-07T17:08:45.401 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestInternal.ResizeFailsToLockImage (41 ms) 2024-04-07T17:08:45.401 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestInternal.SnapCreateLocksImage 2024-04-07T17:08:45.453 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:08:45 smithi100 bash[74942]: cluster 2024-04-07T17:08:43.976607+0000 mgr.y (mgr.25175) 1601 : cluster [DBG] pgmap v2709: 137 pgs: 5 creating+peering, 26 unknown, 106 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail 2024-04-07T17:08:45.453 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:08:45 smithi100 bash[74942]: cluster 2024-04-07T17:08:44.104321+0000 mon.a (mon.0) 2852 : cluster [DBG] osdmap e1534: 8 total, 8 up, 8 in 2024-04-07T17:08:45.453 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:08:45 smithi100 bash[76202]: cluster 2024-04-07T17:08:43.976607+0000 mgr.y (mgr.25175) 1601 : cluster [DBG] pgmap v2709: 137 pgs: 5 creating+peering, 26 unknown, 106 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail 2024-04-07T17:08:45.453 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:08:45 smithi100 bash[76202]: cluster 2024-04-07T17:08:44.104321+0000 mon.a (mon.0) 2852 : cluster [DBG] osdmap e1534: 8 total, 8 up, 8 in 2024-04-07T17:08:46.453 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:08:46 smithi100 bash[74942]: cluster 2024-04-07T17:08:45.106995+0000 mon.a (mon.0) 2853 : cluster [DBG] osdmap e1535: 8 total, 8 up, 8 in 2024-04-07T17:08:46.453 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:08:46 smithi100 bash[76202]: cluster 2024-04-07T17:08:45.106995+0000 mon.a (mon.0) 2853 : cluster [DBG] osdmap e1535: 8 total, 8 up, 8 in 2024-04-07T17:08:46.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:08:46 smithi144 bash[66654]: cluster 2024-04-07T17:08:45.106995+0000 mon.a (mon.0) 2853 : cluster [DBG] osdmap e1535: 8 total, 8 up, 8 in 2024-04-07T17:08:47.189 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestInternal.SnapCreateLocksImage (1787 ms) 2024-04-07T17:08:47.189 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestInternal.SnapCreateFailsToLockImage 2024-04-07T17:08:47.247 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestInternal.SnapCreateFailsToLockImage (58 ms) 2024-04-07T17:08:47.247 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestInternal.SnapRollbackLocksImage 2024-04-07T17:08:47.453 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:08:47 smithi100 bash[74942]: cluster 2024-04-07T17:08:45.977354+0000 mgr.y (mgr.25175) 1602 : cluster [DBG] pgmap v2712: 137 pgs: 5 creating+peering, 16 unknown, 116 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail 2024-04-07T17:08:47.453 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:08:47 smithi100 bash[74942]: cluster 2024-04-07T17:08:46.133026+0000 mon.a (mon.0) 2854 : cluster [DBG] osdmap e1536: 8 total, 8 up, 8 in 2024-04-07T17:08:47.453 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:08:47 smithi100 bash[74942]: audit 2024-04-07T17:08:46.811765+0000 mon.a (mon.0) 2855 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:08:47.453 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:08:47 smithi100 bash[76202]: cluster 2024-04-07T17:08:45.977354+0000 mgr.y (mgr.25175) 1602 : cluster [DBG] pgmap v2712: 137 pgs: 5 creating+peering, 16 unknown, 116 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail 2024-04-07T17:08:47.453 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:08:47 smithi100 bash[76202]: cluster 2024-04-07T17:08:46.133026+0000 mon.a (mon.0) 2854 : cluster [DBG] osdmap e1536: 8 total, 8 up, 8 in 2024-04-07T17:08:47.453 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:08:47 smithi100 bash[76202]: audit 2024-04-07T17:08:46.811765+0000 mon.a (mon.0) 2855 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:08:47.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:08:47 smithi144 bash[66654]: cluster 2024-04-07T17:08:45.977354+0000 mgr.y (mgr.25175) 1602 : cluster [DBG] pgmap v2712: 137 pgs: 5 creating+peering, 16 unknown, 116 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail 2024-04-07T17:08:47.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:08:47 smithi144 bash[66654]: cluster 2024-04-07T17:08:46.133026+0000 mon.a (mon.0) 2854 : cluster [DBG] osdmap e1536: 8 total, 8 up, 8 in 2024-04-07T17:08:47.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:08:47 smithi144 bash[66654]: audit 2024-04-07T17:08:46.811765+0000 mon.a (mon.0) 2855 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:08:48.427 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:08:48 smithi100 bash[74942]: cluster 2024-04-07T17:08:47.142832+0000 mon.a (mon.0) 2856 : cluster [DBG] osdmap e1537: 8 total, 8 up, 8 in 2024-04-07T17:08:48.427 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:08:48 smithi100 bash[76202]: cluster 2024-04-07T17:08:47.142832+0000 mon.a (mon.0) 2856 : cluster [DBG] osdmap e1537: 8 total, 8 up, 8 in 2024-04-07T17:08:48.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:08:48 smithi144 bash[66654]: cluster 2024-04-07T17:08:47.142832+0000 mon.a (mon.0) 2856 : cluster [DBG] osdmap e1537: 8 total, 8 up, 8 in 2024-04-07T17:08:48.702 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:08:48 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:08:48] "GET /metrics HTTP/1.1" 200 36426 "" "Prometheus/2.43.0" 2024-04-07T17:08:49.210 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestInternal.SnapRollbackLocksImage (1963 ms) 2024-04-07T17:08:49.210 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestInternal.SnapRollbackFailsToLockImage 2024-04-07T17:08:49.453 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:08:49 smithi100 bash[74942]: cluster 2024-04-07T17:08:47.978183+0000 mgr.y (mgr.25175) 1603 : cluster [DBG] pgmap v2715: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 67 KiB/s rd, 8.5 KiB/s wr, 98 op/s 2024-04-07T17:08:49.453 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:08:49 smithi100 bash[74942]: cluster 2024-04-07T17:08:48.153975+0000 mon.a (mon.0) 2857 : cluster [DBG] osdmap e1538: 8 total, 8 up, 8 in 2024-04-07T17:08:49.453 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:08:49 smithi100 bash[76202]: cluster 2024-04-07T17:08:47.978183+0000 mgr.y (mgr.25175) 1603 : cluster [DBG] pgmap v2715: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 67 KiB/s rd, 8.5 KiB/s wr, 98 op/s 2024-04-07T17:08:49.453 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:08:49 smithi100 bash[76202]: cluster 2024-04-07T17:08:48.153975+0000 mon.a (mon.0) 2857 : cluster [DBG] osdmap e1538: 8 total, 8 up, 8 in 2024-04-07T17:08:49.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:08:49 smithi144 bash[66654]: cluster 2024-04-07T17:08:47.978183+0000 mgr.y (mgr.25175) 1603 : cluster [DBG] pgmap v2715: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 67 KiB/s rd, 8.5 KiB/s wr, 98 op/s 2024-04-07T17:08:49.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:08:49 smithi144 bash[66654]: cluster 2024-04-07T17:08:48.153975+0000 mon.a (mon.0) 2857 : cluster [DBG] osdmap e1538: 8 total, 8 up, 8 in 2024-04-07T17:08:50.453 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:08:50 smithi100 bash[74942]: cluster 2024-04-07T17:08:49.154112+0000 mon.a (mon.0) 2858 : cluster [DBG] osdmap e1539: 8 total, 8 up, 8 in 2024-04-07T17:08:50.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:08:50 smithi100 bash[76202]: cluster 2024-04-07T17:08:49.154112+0000 mon.a (mon.0) 2858 : cluster [DBG] osdmap e1539: 8 total, 8 up, 8 in 2024-04-07T17:08:50.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:08:50 smithi144 bash[66654]: cluster 2024-04-07T17:08:49.154112+0000 mon.a (mon.0) 2858 : cluster [DBG] osdmap e1539: 8 total, 8 up, 8 in 2024-04-07T17:08:51.214 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestInternal.SnapRollbackFailsToLockImage (2004 ms) 2024-04-07T17:08:51.214 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestInternal.SnapSetReleasesLock 2024-04-07T17:08:51.453 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:08:51 smithi100 bash[74942]: cluster 2024-04-07T17:08:49.979184+0000 mgr.y (mgr.25175) 1604 : cluster [DBG] pgmap v2718: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 96 KiB/s rd, 11 KiB/s wr, 137 op/s 2024-04-07T17:08:51.453 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:08:51 smithi100 bash[74942]: cluster 2024-04-07T17:08:50.179152+0000 mon.a (mon.0) 2859 : cluster [DBG] osdmap e1540: 8 total, 8 up, 8 in 2024-04-07T17:08:51.453 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:08:51 smithi100 bash[76202]: cluster 2024-04-07T17:08:49.979184+0000 mgr.y (mgr.25175) 1604 : cluster [DBG] pgmap v2718: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 96 KiB/s rd, 11 KiB/s wr, 137 op/s 2024-04-07T17:08:51.453 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:08:51 smithi100 bash[76202]: cluster 2024-04-07T17:08:50.179152+0000 mon.a (mon.0) 2859 : cluster [DBG] osdmap e1540: 8 total, 8 up, 8 in 2024-04-07T17:08:51.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:08:51 smithi144 bash[66654]: cluster 2024-04-07T17:08:49.979184+0000 mgr.y (mgr.25175) 1604 : cluster [DBG] pgmap v2718: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 96 KiB/s rd, 11 KiB/s wr, 137 op/s 2024-04-07T17:08:51.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:08:51 smithi144 bash[66654]: cluster 2024-04-07T17:08:50.179152+0000 mon.a (mon.0) 2859 : cluster [DBG] osdmap e1540: 8 total, 8 up, 8 in 2024-04-07T17:08:52.453 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:08:52 smithi100 bash[74942]: cluster 2024-04-07T17:08:51.178244+0000 mon.a (mon.0) 2860 : cluster [DBG] osdmap e1541: 8 total, 8 up, 8 in 2024-04-07T17:08:52.453 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:08:52 smithi100 bash[76202]: cluster 2024-04-07T17:08:51.178244+0000 mon.a (mon.0) 2860 : cluster [DBG] osdmap e1541: 8 total, 8 up, 8 in 2024-04-07T17:08:52.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:08:52 smithi144 bash[66654]: cluster 2024-04-07T17:08:51.178244+0000 mon.a (mon.0) 2860 : cluster [DBG] osdmap e1541: 8 total, 8 up, 8 in 2024-04-07T17:08:52.616 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:08:52 smithi144 bash[95233]: ts=2024-04-07T17:08:52.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:08:52.739 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestInternal.SnapSetReleasesLock (1525 ms) 2024-04-07T17:08:52.739 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestInternal.FlattenLocksImage 2024-04-07T17:08:53.453 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:08:53 smithi100 bash[74942]: cluster 2024-04-07T17:08:51.980148+0000 mgr.y (mgr.25175) 1605 : cluster [DBG] pgmap v2721: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 40 KiB/s rd, 3.2 KiB/s wr, 53 op/s 2024-04-07T17:08:53.453 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:08:53 smithi100 bash[74942]: cluster 2024-04-07T17:08:52.202392+0000 mon.a (mon.0) 2861 : cluster [DBG] osdmap e1542: 8 total, 8 up, 8 in 2024-04-07T17:08:53.453 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:08:53 smithi100 bash[74942]: cluster 2024-04-07T17:08:52.693345+0000 mon.a (mon.0) 2862 : cluster [DBG] osdmap e1543: 8 total, 8 up, 8 in 2024-04-07T17:08:53.453 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:08:53 smithi100 bash[76202]: cluster 2024-04-07T17:08:51.980148+0000 mgr.y (mgr.25175) 1605 : cluster [DBG] pgmap v2721: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 40 KiB/s rd, 3.2 KiB/s wr, 53 op/s 2024-04-07T17:08:53.453 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:08:53 smithi100 bash[76202]: cluster 2024-04-07T17:08:52.202392+0000 mon.a (mon.0) 2861 : cluster [DBG] osdmap e1542: 8 total, 8 up, 8 in 2024-04-07T17:08:53.453 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:08:53 smithi100 bash[76202]: cluster 2024-04-07T17:08:52.693345+0000 mon.a (mon.0) 2862 : cluster [DBG] osdmap e1543: 8 total, 8 up, 8 in 2024-04-07T17:08:53.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:08:53 smithi144 bash[66654]: cluster 2024-04-07T17:08:51.980148+0000 mgr.y (mgr.25175) 1605 : cluster [DBG] pgmap v2721: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 40 KiB/s rd, 3.2 KiB/s wr, 53 op/s 2024-04-07T17:08:53.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:08:53 smithi144 bash[66654]: cluster 2024-04-07T17:08:52.202392+0000 mon.a (mon.0) 2861 : cluster [DBG] osdmap e1542: 8 total, 8 up, 8 in 2024-04-07T17:08:53.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:08:53 smithi144 bash[66654]: cluster 2024-04-07T17:08:52.693345+0000 mon.a (mon.0) 2862 : cluster [DBG] osdmap e1543: 8 total, 8 up, 8 in 2024-04-07T17:08:54.756 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestInternal.FlattenLocksImage (2017 ms) 2024-04-07T17:08:54.756 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestInternal.FlattenFailsToLockImage 2024-04-07T17:08:54.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:08:54 smithi100 bash[74942]: cluster 2024-04-07T17:08:53.695616+0000 mon.a (mon.0) 2863 : cluster [DBG] osdmap e1544: 8 total, 8 up, 8 in 2024-04-07T17:08:54.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:08:54 smithi100 bash[74942]: audit 2024-04-07T17:08:53.995708+0000 mon.a (mon.0) 2864 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:08:54.953 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:08:54 smithi100 bash[76202]: cluster 2024-04-07T17:08:53.695616+0000 mon.a (mon.0) 2863 : cluster [DBG] osdmap e1544: 8 total, 8 up, 8 in 2024-04-07T17:08:54.953 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:08:54 smithi100 bash[76202]: audit 2024-04-07T17:08:53.995708+0000 mon.a (mon.0) 2864 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:08:55.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:08:54 smithi144 bash[66654]: cluster 2024-04-07T17:08:53.695616+0000 mon.a (mon.0) 2863 : cluster [DBG] osdmap e1544: 8 total, 8 up, 8 in 2024-04-07T17:08:55.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:08:54 smithi144 bash[66654]: audit 2024-04-07T17:08:53.995708+0000 mon.a (mon.0) 2864 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:08:56.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:08:55 smithi144 bash[66654]: cluster 2024-04-07T17:08:53.980996+0000 mgr.y (mgr.25175) 1606 : cluster [DBG] pgmap v2725: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 48 KiB/s rd, 5.2 KiB/s wr, 72 op/s 2024-04-07T17:08:56.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:08:55 smithi144 bash[66654]: cluster 2024-04-07T17:08:54.704930+0000 mon.a (mon.0) 2865 : cluster [DBG] osdmap e1545: 8 total, 8 up, 8 in 2024-04-07T17:08:56.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:08:55 smithi144 bash[66654]: audit 2024-04-07T17:08:54.791535+0000 mon.a (mon.0) 2866 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:08:56.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:08:55 smithi144 bash[66654]: audit 2024-04-07T17:08:54.792706+0000 mon.a (mon.0) 2867 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:08:56.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:08:55 smithi144 bash[66654]: audit 2024-04-07T17:08:54.799211+0000 mon.a (mon.0) 2868 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:08:56.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:08:55 smithi100 bash[74942]: cluster 2024-04-07T17:08:53.980996+0000 mgr.y (mgr.25175) 1606 : cluster [DBG] pgmap v2725: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 48 KiB/s rd, 5.2 KiB/s wr, 72 op/s 2024-04-07T17:08:56.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:08:55 smithi100 bash[74942]: cluster 2024-04-07T17:08:54.704930+0000 mon.a (mon.0) 2865 : cluster [DBG] osdmap e1545: 8 total, 8 up, 8 in 2024-04-07T17:08:56.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:08:55 smithi100 bash[74942]: audit 2024-04-07T17:08:54.791535+0000 mon.a (mon.0) 2866 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:08:56.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:08:55 smithi100 bash[74942]: audit 2024-04-07T17:08:54.792706+0000 mon.a (mon.0) 2867 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:08:56.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:08:55 smithi100 bash[74942]: audit 2024-04-07T17:08:54.799211+0000 mon.a (mon.0) 2868 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:08:56.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:08:55 smithi100 bash[76202]: cluster 2024-04-07T17:08:53.980996+0000 mgr.y (mgr.25175) 1606 : cluster [DBG] pgmap v2725: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 48 KiB/s rd, 5.2 KiB/s wr, 72 op/s 2024-04-07T17:08:56.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:08:55 smithi100 bash[76202]: cluster 2024-04-07T17:08:54.704930+0000 mon.a (mon.0) 2865 : cluster [DBG] osdmap e1545: 8 total, 8 up, 8 in 2024-04-07T17:08:56.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:08:55 smithi100 bash[76202]: audit 2024-04-07T17:08:54.791535+0000 mon.a (mon.0) 2866 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:08:56.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:08:55 smithi100 bash[76202]: audit 2024-04-07T17:08:54.792706+0000 mon.a (mon.0) 2867 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:08:56.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:08:55 smithi100 bash[76202]: audit 2024-04-07T17:08:54.799211+0000 mon.a (mon.0) 2868 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:08:56.776 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestInternal.FlattenFailsToLockImage (2020 ms) 2024-04-07T17:08:56.776 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestInternal.WriteFailsToLockImageBlocklisted 2024-04-07T17:08:57.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:08:56 smithi144 bash[66654]: cluster 2024-04-07T17:08:55.704722+0000 mon.a (mon.0) 2869 : cluster [DBG] osdmap e1546: 8 total, 8 up, 8 in 2024-04-07T17:08:57.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:08:56 smithi100 bash[74942]: cluster 2024-04-07T17:08:55.704722+0000 mon.a (mon.0) 2869 : cluster [DBG] osdmap e1546: 8 total, 8 up, 8 in 2024-04-07T17:08:57.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:08:56 smithi100 bash[76202]: cluster 2024-04-07T17:08:55.704722+0000 mon.a (mon.0) 2869 : cluster [DBG] osdmap e1546: 8 total, 8 up, 8 in 2024-04-07T17:08:57.706 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestInternal.WriteFailsToLockImageBlocklisted (930 ms) 2024-04-07T17:08:57.706 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestInternal.WriteFailsToLockImageBlocklistedWatch 2024-04-07T17:08:58.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:08:57 smithi144 bash[66654]: cluster 2024-04-07T17:08:55.981833+0000 mgr.y (mgr.25175) 1607 : cluster [DBG] pgmap v2728: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 103 KiB/s rd, 6.6 KiB/s wr, 141 op/s 2024-04-07T17:08:58.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:08:57 smithi144 bash[66654]: cluster 2024-04-07T17:08:56.722465+0000 mon.a (mon.0) 2870 : cluster [DBG] osdmap e1547: 8 total, 8 up, 8 in 2024-04-07T17:08:58.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:08:57 smithi144 bash[66654]: audit 2024-04-07T17:08:56.859645+0000 mon.c (mon.2) 66 : audit [INF] from='client.? 172.21.15.100:0/594324814' entity='client.admin' cmd=[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.100:0/594324814"}]: dispatch 2024-04-07T17:08:58.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:08:57 smithi144 bash[66654]: audit 2024-04-07T17:08:56.860056+0000 mon.a (mon.0) 2871 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.100:0/594324814"}]: dispatch 2024-04-07T17:08:58.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:08:57 smithi144 bash[66654]: audit 2024-04-07T17:08:57.687091+0000 mon.a (mon.0) 2872 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.100:0/594324814"}]': finished 2024-04-07T17:08:58.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:08:57 smithi144 bash[66654]: cluster 2024-04-07T17:08:57.694324+0000 mon.a (mon.0) 2873 : cluster [DBG] osdmap e1548: 8 total, 8 up, 8 in 2024-04-07T17:08:58.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:08:57 smithi100 bash[74942]: cluster 2024-04-07T17:08:55.981833+0000 mgr.y (mgr.25175) 1607 : cluster [DBG] pgmap v2728: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 103 KiB/s rd, 6.6 KiB/s wr, 141 op/s 2024-04-07T17:08:58.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:08:57 smithi100 bash[74942]: cluster 2024-04-07T17:08:56.722465+0000 mon.a (mon.0) 2870 : cluster [DBG] osdmap e1547: 8 total, 8 up, 8 in 2024-04-07T17:08:58.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:08:57 smithi100 bash[74942]: audit 2024-04-07T17:08:56.859645+0000 mon.c (mon.2) 66 : audit [INF] from='client.? 172.21.15.100:0/594324814' entity='client.admin' cmd=[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.100:0/594324814"}]: dispatch 2024-04-07T17:08:58.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:08:57 smithi100 bash[74942]: audit 2024-04-07T17:08:56.860056+0000 mon.a (mon.0) 2871 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.100:0/594324814"}]: dispatch 2024-04-07T17:08:58.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:08:57 smithi100 bash[74942]: audit 2024-04-07T17:08:57.687091+0000 mon.a (mon.0) 2872 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.100:0/594324814"}]': finished 2024-04-07T17:08:58.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:08:57 smithi100 bash[74942]: cluster 2024-04-07T17:08:57.694324+0000 mon.a (mon.0) 2873 : cluster [DBG] osdmap e1548: 8 total, 8 up, 8 in 2024-04-07T17:08:58.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:08:57 smithi100 bash[76202]: cluster 2024-04-07T17:08:55.981833+0000 mgr.y (mgr.25175) 1607 : cluster [DBG] pgmap v2728: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 103 KiB/s rd, 6.6 KiB/s wr, 141 op/s 2024-04-07T17:08:58.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:08:57 smithi100 bash[76202]: cluster 2024-04-07T17:08:56.722465+0000 mon.a (mon.0) 2870 : cluster [DBG] osdmap e1547: 8 total, 8 up, 8 in 2024-04-07T17:08:58.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:08:57 smithi100 bash[76202]: audit 2024-04-07T17:08:56.859645+0000 mon.c (mon.2) 66 : audit [INF] from='client.? 172.21.15.100:0/594324814' entity='client.admin' cmd=[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.100:0/594324814"}]: dispatch 2024-04-07T17:08:58.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:08:57 smithi100 bash[76202]: audit 2024-04-07T17:08:56.860056+0000 mon.a (mon.0) 2871 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.100:0/594324814"}]: dispatch 2024-04-07T17:08:58.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:08:57 smithi100 bash[76202]: audit 2024-04-07T17:08:57.687091+0000 mon.a (mon.0) 2872 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.100:0/594324814"}]': finished 2024-04-07T17:08:58.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:08:57 smithi100 bash[76202]: cluster 2024-04-07T17:08:57.694324+0000 mon.a (mon.0) 2873 : cluster [DBG] osdmap e1548: 8 total, 8 up, 8 in 2024-04-07T17:08:58.703 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:08:58 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:08:58] "GET /metrics HTTP/1.1" 200 36426 "" "Prometheus/2.43.0" 2024-04-07T17:08:59.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:08:58 smithi144 bash[66654]: audit 2024-04-07T17:08:57.773779+0000 mon.a (mon.0) 2874 : audit [INF] from='client.? 172.21.15.100:0/3095497165' entity='client.admin' cmd=[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.100:0/3095497165"}]: dispatch 2024-04-07T17:08:59.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:08:58 smithi100 bash[74942]: audit 2024-04-07T17:08:57.773779+0000 mon.a (mon.0) 2874 : audit [INF] from='client.? 172.21.15.100:0/3095497165' entity='client.admin' cmd=[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.100:0/3095497165"}]: dispatch 2024-04-07T17:08:59.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:08:58 smithi100 bash[76202]: audit 2024-04-07T17:08:57.773779+0000 mon.a (mon.0) 2874 : audit [INF] from='client.? 172.21.15.100:0/3095497165' entity='client.admin' cmd=[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.100:0/3095497165"}]: dispatch 2024-04-07T17:08:59.745 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestInternal.WriteFailsToLockImageBlocklistedWatch (2039 ms) 2024-04-07T17:08:59.745 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestInternal.AioWriteRequestsLock 2024-04-07T17:08:59.811 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestInternal.AioWriteRequestsLock (66 ms) 2024-04-07T17:08:59.811 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestInternal.AioDiscardRequestsLock 2024-04-07T17:08:59.876 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestInternal.AioDiscardRequestsLock (66 ms) 2024-04-07T17:08:59.877 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestInternal.CancelAsyncResize 2024-04-07T17:09:00.004 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestInternal.CancelAsyncResize (127 ms) 2024-04-07T17:09:00.004 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestInternal.MultipleResize 2024-04-07T17:09:00.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:08:59 smithi144 bash[66654]: cluster 2024-04-07T17:08:57.982665+0000 mgr.y (mgr.25175) 1608 : cluster [DBG] pgmap v2731: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 175 KiB/s rd, 12 KiB/s wr, 234 op/s 2024-04-07T17:09:00.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:08:59 smithi144 bash[66654]: audit 2024-04-07T17:08:58.730528+0000 mon.a (mon.0) 2875 : audit [INF] from='client.? 172.21.15.100:0/3095497165' entity='client.admin' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.100:0/3095497165"}]': finished 2024-04-07T17:09:00.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:08:59 smithi144 bash[66654]: cluster 2024-04-07T17:08:58.737966+0000 mon.a (mon.0) 2876 : cluster [DBG] osdmap e1549: 8 total, 8 up, 8 in 2024-04-07T17:09:00.164 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestInternal.MultipleResize (160 ms) 2024-04-07T17:09:00.164 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestInternal.Metadata 2024-04-07T17:09:00.202 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:08:59 smithi100 bash[76202]: cluster 2024-04-07T17:08:57.982665+0000 mgr.y (mgr.25175) 1608 : cluster [DBG] pgmap v2731: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 175 KiB/s rd, 12 KiB/s wr, 234 op/s 2024-04-07T17:09:00.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:08:59 smithi100 bash[76202]: audit 2024-04-07T17:08:58.730528+0000 mon.a (mon.0) 2875 : audit [INF] from='client.? 172.21.15.100:0/3095497165' entity='client.admin' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.100:0/3095497165"}]': finished 2024-04-07T17:09:00.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:08:59 smithi100 bash[76202]: cluster 2024-04-07T17:08:58.737966+0000 mon.a (mon.0) 2876 : cluster [DBG] osdmap e1549: 8 total, 8 up, 8 in 2024-04-07T17:09:00.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:08:59 smithi100 bash[74942]: cluster 2024-04-07T17:08:57.982665+0000 mgr.y (mgr.25175) 1608 : cluster [DBG] pgmap v2731: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 175 KiB/s rd, 12 KiB/s wr, 234 op/s 2024-04-07T17:09:00.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:08:59 smithi100 bash[74942]: audit 2024-04-07T17:08:58.730528+0000 mon.a (mon.0) 2875 : audit [INF] from='client.? 172.21.15.100:0/3095497165' entity='client.admin' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.100:0/3095497165"}]': finished 2024-04-07T17:09:00.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:08:59 smithi100 bash[74942]: cluster 2024-04-07T17:08:58.737966+0000 mon.a (mon.0) 2876 : cluster [DBG] osdmap e1549: 8 total, 8 up, 8 in 2024-04-07T17:09:00.304 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestInternal.Metadata (140 ms) 2024-04-07T17:09:00.304 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestInternal.MetadataConfApply 2024-04-07T17:09:00.398 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestInternal.MetadataConfApply (94 ms) 2024-04-07T17:09:00.398 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestInternal.SnapshotCopyup 2024-04-07T17:09:02.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:09:01 smithi144 bash[66654]: cluster 2024-04-07T17:08:59.983879+0000 mgr.y (mgr.25175) 1609 : cluster [DBG] pgmap v2733: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 127 KiB/s rd, 8.6 KiB/s wr, 174 op/s 2024-04-07T17:09:02.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:09:01 smithi144 bash[66654]: cluster 2024-04-07T17:09:00.762572+0000 mon.a (mon.0) 2877 : cluster [DBG] osdmap e1550: 8 total, 8 up, 8 in 2024-04-07T17:09:02.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:09:01 smithi100 bash[74942]: cluster 2024-04-07T17:08:59.983879+0000 mgr.y (mgr.25175) 1609 : cluster [DBG] pgmap v2733: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 127 KiB/s rd, 8.6 KiB/s wr, 174 op/s 2024-04-07T17:09:02.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:09:01 smithi100 bash[74942]: cluster 2024-04-07T17:09:00.762572+0000 mon.a (mon.0) 2877 : cluster [DBG] osdmap e1550: 8 total, 8 up, 8 in 2024-04-07T17:09:02.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:09:01 smithi100 bash[76202]: cluster 2024-04-07T17:08:59.983879+0000 mgr.y (mgr.25175) 1609 : cluster [DBG] pgmap v2733: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 127 KiB/s rd, 8.6 KiB/s wr, 174 op/s 2024-04-07T17:09:02.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:09:01 smithi100 bash[76202]: cluster 2024-04-07T17:09:00.762572+0000 mon.a (mon.0) 2877 : cluster [DBG] osdmap e1550: 8 total, 8 up, 8 in 2024-04-07T17:09:02.616 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:09:02 smithi144 bash[95233]: ts=2024-04-07T17:09:02.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:09:02.910 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestInternal.SnapshotCopyup (2512 ms) 2024-04-07T17:09:02.910 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestInternal.SnapshotCopyupZeros 2024-04-07T17:09:03.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:09:02 smithi144 bash[66654]: cluster 2024-04-07T17:09:01.772037+0000 mon.a (mon.0) 2878 : cluster [DBG] osdmap e1551: 8 total, 8 up, 8 in 2024-04-07T17:09:03.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:09:02 smithi144 bash[66654]: audit 2024-04-07T17:09:01.812208+0000 mon.a (mon.0) 2879 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:09:03.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:09:02 smithi144 bash[66654]: cluster 2024-04-07T17:09:02.695951+0000 mon.a (mon.0) 2880 : cluster [DBG] osdmap e1552: 8 total, 8 up, 8 in 2024-04-07T17:09:03.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:09:02 smithi100 bash[74942]: cluster 2024-04-07T17:09:01.772037+0000 mon.a (mon.0) 2878 : cluster [DBG] osdmap e1551: 8 total, 8 up, 8 in 2024-04-07T17:09:03.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:09:02 smithi100 bash[74942]: audit 2024-04-07T17:09:01.812208+0000 mon.a (mon.0) 2879 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:09:03.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:09:02 smithi100 bash[74942]: cluster 2024-04-07T17:09:02.695951+0000 mon.a (mon.0) 2880 : cluster [DBG] osdmap e1552: 8 total, 8 up, 8 in 2024-04-07T17:09:03.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:09:02 smithi100 bash[76202]: cluster 2024-04-07T17:09:01.772037+0000 mon.a (mon.0) 2878 : cluster [DBG] osdmap e1551: 8 total, 8 up, 8 in 2024-04-07T17:09:03.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:09:02 smithi100 bash[76202]: audit 2024-04-07T17:09:01.812208+0000 mon.a (mon.0) 2879 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:09:03.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:09:02 smithi100 bash[76202]: cluster 2024-04-07T17:09:02.695951+0000 mon.a (mon.0) 2880 : cluster [DBG] osdmap e1552: 8 total, 8 up, 8 in 2024-04-07T17:09:04.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:09:03 smithi144 bash[66654]: cluster 2024-04-07T17:09:01.984737+0000 mgr.y (mgr.25175) 1610 : cluster [DBG] pgmap v2736: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 112 KiB/s rd, 5.8 KiB/s wr, 147 op/s 2024-04-07T17:09:04.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:09:03 smithi100 bash[74942]: cluster 2024-04-07T17:09:01.984737+0000 mgr.y (mgr.25175) 1610 : cluster [DBG] pgmap v2736: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 112 KiB/s rd, 5.8 KiB/s wr, 147 op/s 2024-04-07T17:09:04.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:09:03 smithi100 bash[76202]: cluster 2024-04-07T17:09:01.984737+0000 mgr.y (mgr.25175) 1610 : cluster [DBG] pgmap v2736: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 112 KiB/s rd, 5.8 KiB/s wr, 147 op/s 2024-04-07T17:09:04.871 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestInternal.SnapshotCopyupZeros (1961 ms) 2024-04-07T17:09:04.871 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestInternal.SnapshotCopyupZerosMigration 2024-04-07T17:09:05.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:09:04 smithi144 bash[66654]: cluster 2024-04-07T17:09:03.785641+0000 mon.a (mon.0) 2881 : cluster [DBG] osdmap e1553: 8 total, 8 up, 8 in 2024-04-07T17:09:05.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:09:04 smithi100 bash[74942]: cluster 2024-04-07T17:09:03.785641+0000 mon.a (mon.0) 2881 : cluster [DBG] osdmap e1553: 8 total, 8 up, 8 in 2024-04-07T17:09:05.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:09:04 smithi100 bash[76202]: cluster 2024-04-07T17:09:03.785641+0000 mon.a (mon.0) 2881 : cluster [DBG] osdmap e1553: 8 total, 8 up, 8 in 2024-04-07T17:09:05.878 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestInternal.SnapshotCopyupZerosMigration (1007 ms) 2024-04-07T17:09:05.878 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestInternal.ResizeCopyup 2024-04-07T17:09:06.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:09:05 smithi144 bash[66654]: cluster 2024-04-07T17:09:03.985486+0000 mgr.y (mgr.25175) 1611 : cluster [DBG] pgmap v2739: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 157 KiB/s rd, 23 KiB/s wr, 225 op/s 2024-04-07T17:09:06.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:09:05 smithi144 bash[66654]: cluster 2024-04-07T17:09:04.806192+0000 mon.a (mon.0) 2882 : cluster [DBG] osdmap e1554: 8 total, 8 up, 8 in 2024-04-07T17:09:06.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:09:05 smithi100 bash[74942]: cluster 2024-04-07T17:09:03.985486+0000 mgr.y (mgr.25175) 1611 : cluster [DBG] pgmap v2739: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 157 KiB/s rd, 23 KiB/s wr, 225 op/s 2024-04-07T17:09:06.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:09:05 smithi100 bash[74942]: cluster 2024-04-07T17:09:04.806192+0000 mon.a (mon.0) 2882 : cluster [DBG] osdmap e1554: 8 total, 8 up, 8 in 2024-04-07T17:09:06.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:09:05 smithi100 bash[76202]: cluster 2024-04-07T17:09:03.985486+0000 mgr.y (mgr.25175) 1611 : cluster [DBG] pgmap v2739: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 157 KiB/s rd, 23 KiB/s wr, 225 op/s 2024-04-07T17:09:06.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:09:05 smithi100 bash[76202]: cluster 2024-04-07T17:09:04.806192+0000 mon.a (mon.0) 2882 : cluster [DBG] osdmap e1554: 8 total, 8 up, 8 in 2024-04-07T17:09:07.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:09:06 smithi144 bash[66654]: cluster 2024-04-07T17:09:05.807771+0000 mon.a (mon.0) 2883 : cluster [DBG] osdmap e1555: 8 total, 8 up, 8 in 2024-04-07T17:09:07.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:09:06 smithi100 bash[74942]: cluster 2024-04-07T17:09:05.807771+0000 mon.a (mon.0) 2883 : cluster [DBG] osdmap e1555: 8 total, 8 up, 8 in 2024-04-07T17:09:07.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:09:06 smithi100 bash[76202]: cluster 2024-04-07T17:09:05.807771+0000 mon.a (mon.0) 2883 : cluster [DBG] osdmap e1555: 8 total, 8 up, 8 in 2024-04-07T17:09:07.949 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestInternal.ResizeCopyup (2071 ms) 2024-04-07T17:09:07.949 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestInternal.DiscardCopyup 2024-04-07T17:09:07.972 INFO:tasks.workunit.client.0.smithi100.stdout:./src/test/librbd/test_internal.cc:1056: Skipped 2024-04-07T17:09:07.972 INFO:tasks.workunit.client.0.smithi100.stdout:Skipping due to unmet REQUIRE 2024-04-07T17:09:07.972 INFO:tasks.workunit.client.0.smithi100.stdout:[ SKIPPED ] TestInternal.DiscardCopyup (23 ms) 2024-04-07T17:09:07.972 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestInternal.ImageOptions 2024-04-07T17:09:07.992 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestInternal.ImageOptions (21 ms) 2024-04-07T17:09:07.993 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestInternal.WriteFullCopyup 2024-04-07T17:09:08.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:09:07 smithi144 bash[66654]: cluster 2024-04-07T17:09:05.986156+0000 mgr.y (mgr.25175) 1612 : cluster [DBG] pgmap v2742: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 104 KiB/s rd, 24 KiB/s wr, 171 op/s 2024-04-07T17:09:08.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:09:07 smithi144 bash[66654]: cluster 2024-04-07T17:09:06.828226+0000 mon.a (mon.0) 2884 : cluster [DBG] osdmap e1556: 8 total, 8 up, 8 in 2024-04-07T17:09:08.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:09:07 smithi100 bash[74942]: cluster 2024-04-07T17:09:05.986156+0000 mgr.y (mgr.25175) 1612 : cluster [DBG] pgmap v2742: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 104 KiB/s rd, 24 KiB/s wr, 171 op/s 2024-04-07T17:09:08.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:09:07 smithi100 bash[74942]: cluster 2024-04-07T17:09:06.828226+0000 mon.a (mon.0) 2884 : cluster [DBG] osdmap e1556: 8 total, 8 up, 8 in 2024-04-07T17:09:08.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:09:07 smithi100 bash[76202]: cluster 2024-04-07T17:09:05.986156+0000 mgr.y (mgr.25175) 1612 : cluster [DBG] pgmap v2742: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 104 KiB/s rd, 24 KiB/s wr, 171 op/s 2024-04-07T17:09:08.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:09:07 smithi100 bash[76202]: cluster 2024-04-07T17:09:06.828226+0000 mon.a (mon.0) 2884 : cluster [DBG] osdmap e1556: 8 total, 8 up, 8 in 2024-04-07T17:09:08.703 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:09:08 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:09:08] "GET /metrics HTTP/1.1" 200 36469 "" "Prometheus/2.43.0" 2024-04-07T17:09:09.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:09:08 smithi144 bash[66654]: cluster 2024-04-07T17:09:07.830878+0000 mon.a (mon.0) 2885 : cluster [DBG] osdmap e1557: 8 total, 8 up, 8 in 2024-04-07T17:09:09.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:09:08 smithi100 bash[74942]: cluster 2024-04-07T17:09:07.830878+0000 mon.a (mon.0) 2885 : cluster [DBG] osdmap e1557: 8 total, 8 up, 8 in 2024-04-07T17:09:09.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:09:08 smithi100 bash[76202]: cluster 2024-04-07T17:09:07.830878+0000 mon.a (mon.0) 2885 : cluster [DBG] osdmap e1557: 8 total, 8 up, 8 in 2024-04-07T17:09:10.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:09:09 smithi100 bash[74942]: cluster 2024-04-07T17:09:07.987088+0000 mgr.y (mgr.25175) 1613 : cluster [DBG] pgmap v2745: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 244 KiB/s rd, 23 KiB/s wr, 332 op/s 2024-04-07T17:09:10.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:09:09 smithi100 bash[74942]: cluster 2024-04-07T17:09:08.855768+0000 mon.a (mon.0) 2886 : cluster [DBG] osdmap e1558: 8 total, 8 up, 8 in 2024-04-07T17:09:10.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:09:09 smithi100 bash[76202]: cluster 2024-04-07T17:09:07.987088+0000 mgr.y (mgr.25175) 1613 : cluster [DBG] pgmap v2745: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 244 KiB/s rd, 23 KiB/s wr, 332 op/s 2024-04-07T17:09:10.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:09:09 smithi100 bash[76202]: cluster 2024-04-07T17:09:08.855768+0000 mon.a (mon.0) 2886 : cluster [DBG] osdmap e1558: 8 total, 8 up, 8 in 2024-04-07T17:09:10.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:09:09 smithi144 bash[66654]: cluster 2024-04-07T17:09:07.987088+0000 mgr.y (mgr.25175) 1613 : cluster [DBG] pgmap v2745: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 244 KiB/s rd, 23 KiB/s wr, 332 op/s 2024-04-07T17:09:10.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:09:09 smithi144 bash[66654]: cluster 2024-04-07T17:09:08.855768+0000 mon.a (mon.0) 2886 : cluster [DBG] osdmap e1558: 8 total, 8 up, 8 in 2024-04-07T17:09:11.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:09:10 smithi100 bash[74942]: cluster 2024-04-07T17:09:09.863896+0000 mon.a (mon.0) 2887 : cluster [DBG] osdmap e1559: 8 total, 8 up, 8 in 2024-04-07T17:09:11.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:09:10 smithi100 bash[76202]: cluster 2024-04-07T17:09:09.863896+0000 mon.a (mon.0) 2887 : cluster [DBG] osdmap e1559: 8 total, 8 up, 8 in 2024-04-07T17:09:11.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:09:10 smithi144 bash[66654]: cluster 2024-04-07T17:09:09.863896+0000 mon.a (mon.0) 2887 : cluster [DBG] osdmap e1559: 8 total, 8 up, 8 in 2024-04-07T17:09:11.934 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestInternal.WriteFullCopyup (3941 ms) 2024-04-07T17:09:11.934 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestInternal.DiffIterateCloneOverwrite 2024-04-07T17:09:12.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:09:11 smithi100 bash[74942]: cluster 2024-04-07T17:09:09.988201+0000 mgr.y (mgr.25175) 1614 : cluster [DBG] pgmap v2748: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 286 KiB/s rd, 28 KiB/s wr, 385 op/s 2024-04-07T17:09:12.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:09:11 smithi100 bash[74942]: cluster 2024-04-07T17:09:10.884919+0000 mon.a (mon.0) 2888 : cluster [DBG] osdmap e1560: 8 total, 8 up, 8 in 2024-04-07T17:09:12.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:09:11 smithi100 bash[76202]: cluster 2024-04-07T17:09:09.988201+0000 mgr.y (mgr.25175) 1614 : cluster [DBG] pgmap v2748: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 286 KiB/s rd, 28 KiB/s wr, 385 op/s 2024-04-07T17:09:12.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:09:11 smithi100 bash[76202]: cluster 2024-04-07T17:09:10.884919+0000 mon.a (mon.0) 2888 : cluster [DBG] osdmap e1560: 8 total, 8 up, 8 in 2024-04-07T17:09:12.316 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:09:11 smithi144 bash[66654]: cluster 2024-04-07T17:09:09.988201+0000 mgr.y (mgr.25175) 1614 : cluster [DBG] pgmap v2748: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 286 KiB/s rd, 28 KiB/s wr, 385 op/s 2024-04-07T17:09:12.316 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:09:11 smithi144 bash[66654]: cluster 2024-04-07T17:09:10.884919+0000 mon.a (mon.0) 2888 : cluster [DBG] osdmap e1560: 8 total, 8 up, 8 in 2024-04-07T17:09:12.616 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:09:12 smithi144 bash[95233]: ts=2024-04-07T17:09:12.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:09:13.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:09:12 smithi100 bash[74942]: cluster 2024-04-07T17:09:11.900131+0000 mon.a (mon.0) 2889 : cluster [DBG] osdmap e1561: 8 total, 8 up, 8 in 2024-04-07T17:09:13.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:09:12 smithi100 bash[76202]: cluster 2024-04-07T17:09:11.900131+0000 mon.a (mon.0) 2889 : cluster [DBG] osdmap e1561: 8 total, 8 up, 8 in 2024-04-07T17:09:13.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:09:12 smithi144 bash[66654]: cluster 2024-04-07T17:09:11.900131+0000 mon.a (mon.0) 2889 : cluster [DBG] osdmap e1561: 8 total, 8 up, 8 in 2024-04-07T17:09:13.975 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestInternal.DiffIterateCloneOverwrite (2041 ms) 2024-04-07T17:09:13.975 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestInternal.TestCoR 2024-04-07T17:09:13.998 INFO:tasks.workunit.client.0.smithi100.stdout:./src/test/librbd/test_internal.cc:1313: Skipped 2024-04-07T17:09:13.999 INFO:tasks.workunit.client.0.smithi100.stdout:Skipping due to disabled copy-on-read 2024-04-07T17:09:13.999 INFO:tasks.workunit.client.0.smithi100.stdout:[ SKIPPED ] TestInternal.TestCoR (24 ms) 2024-04-07T17:09:13.999 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestInternal.FlattenNoEmptyObjects 2024-04-07T17:09:14.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:09:13 smithi100 bash[74942]: cluster 2024-04-07T17:09:11.989038+0000 mgr.y (mgr.25175) 1615 : cluster [DBG] pgmap v2751: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 100 KiB/s rd, 15 KiB/s wr, 142 op/s 2024-04-07T17:09:14.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:09:13 smithi100 bash[74942]: cluster 2024-04-07T17:09:12.899764+0000 mon.a (mon.0) 2890 : cluster [DBG] osdmap e1562: 8 total, 8 up, 8 in 2024-04-07T17:09:14.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:09:13 smithi100 bash[76202]: cluster 2024-04-07T17:09:11.989038+0000 mgr.y (mgr.25175) 1615 : cluster [DBG] pgmap v2751: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 100 KiB/s rd, 15 KiB/s wr, 142 op/s 2024-04-07T17:09:14.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:09:13 smithi100 bash[76202]: cluster 2024-04-07T17:09:12.899764+0000 mon.a (mon.0) 2890 : cluster [DBG] osdmap e1562: 8 total, 8 up, 8 in 2024-04-07T17:09:14.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:09:13 smithi144 bash[66654]: cluster 2024-04-07T17:09:11.989038+0000 mgr.y (mgr.25175) 1615 : cluster [DBG] pgmap v2751: 137 pgs: 137 active+clean; 4.6 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 100 KiB/s rd, 15 KiB/s wr, 142 op/s 2024-04-07T17:09:14.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:09:13 smithi144 bash[66654]: cluster 2024-04-07T17:09:12.899764+0000 mon.a (mon.0) 2890 : cluster [DBG] osdmap e1562: 8 total, 8 up, 8 in 2024-04-07T17:09:14.601 INFO:tasks.workunit.client.0.smithi100.stdout:made parent image "image390": 4194304K (1024 * 4K) 2024-04-07T17:09:14.601 INFO:tasks.workunit.client.0.smithi100.stdout:generated random write map: 2024-04-07T17:09:14.601 INFO:tasks.workunit.client.0.smithi100.stdout: [20 , 82913 ] 2024-04-07T17:09:14.601 INFO:tasks.workunit.client.0.smithi100.stdout: [27 , 112152 ] 2024-04-07T17:09:14.601 INFO:tasks.workunit.client.0.smithi100.stdout: [28 , 114815 ] 2024-04-07T17:09:14.602 INFO:tasks.workunit.client.0.smithi100.stdout: [29 , 119077 ] 2024-04-07T17:09:14.602 INFO:tasks.workunit.client.0.smithi100.stdout: [43 , 177505 ] 2024-04-07T17:09:14.602 INFO:tasks.workunit.client.0.smithi100.stdout: [50 , 207554 ] 2024-04-07T17:09:14.602 INFO:tasks.workunit.client.0.smithi100.stdout: [74 , 306242 ] 2024-04-07T17:09:14.602 INFO:tasks.workunit.client.0.smithi100.stdout: [82 , 336454 ] 2024-04-07T17:09:14.602 INFO:tasks.workunit.client.0.smithi100.stdout: [98 , 402801 ] 2024-04-07T17:09:14.602 INFO:tasks.workunit.client.0.smithi100.stdout: [99 , 406455 ] 2024-04-07T17:09:14.602 INFO:tasks.workunit.client.0.smithi100.stdout: [107 , 439507 ] 2024-04-07T17:09:14.602 INFO:tasks.workunit.client.0.smithi100.stdout: [117 , 482127 ] 2024-04-07T17:09:14.602 INFO:tasks.workunit.client.0.smithi100.stdout: [122 , 502517 ] 2024-04-07T17:09:14.602 INFO:tasks.workunit.client.0.smithi100.stdout: [125 , 515274 ] 2024-04-07T17:09:14.603 INFO:tasks.workunit.client.0.smithi100.stdout: [149 , 612955 ] 2024-04-07T17:09:14.603 INFO:tasks.workunit.client.0.smithi100.stdout: [154 , 633920 ] 2024-04-07T17:09:14.603 INFO:tasks.workunit.client.0.smithi100.stdout: [156 , 642241 ] 2024-04-07T17:09:14.603 INFO:tasks.workunit.client.0.smithi100.stdout: [160 , 656801 ] 2024-04-07T17:09:14.603 INFO:tasks.workunit.client.0.smithi100.stdout: [162 , 663828 ] 2024-04-07T17:09:14.603 INFO:tasks.workunit.client.0.smithi100.stdout: [166 , 681959 ] 2024-04-07T17:09:14.603 INFO:tasks.workunit.client.0.smithi100.stdout: [175 , 719123 ] 2024-04-07T17:09:14.603 INFO:tasks.workunit.client.0.smithi100.stdout: [179 , 736328 ] 2024-04-07T17:09:14.603 INFO:tasks.workunit.client.0.smithi100.stdout: [181 , 741639 ] 2024-04-07T17:09:14.603 INFO:tasks.workunit.client.0.smithi100.stdout: [182 , 747765 ] 2024-04-07T17:09:14.603 INFO:tasks.workunit.client.0.smithi100.stdout: [207 , 848091 ] 2024-04-07T17:09:14.603 INFO:tasks.workunit.client.0.smithi100.stdout: [214 , 878581 ] 2024-04-07T17:09:14.603 INFO:tasks.workunit.client.0.smithi100.stdout: [232 , 951923 ] 2024-04-07T17:09:14.603 INFO:tasks.workunit.client.0.smithi100.stdout: [233 , 955980 ] 2024-04-07T17:09:14.603 INFO:tasks.workunit.client.0.smithi100.stdout: [236 , 968113 ] 2024-04-07T17:09:14.604 INFO:tasks.workunit.client.0.smithi100.stdout: [276 , 1131903 ] 2024-04-07T17:09:14.604 INFO:tasks.workunit.client.0.smithi100.stdout: [282 , 1156214 ] 2024-04-07T17:09:14.604 INFO:tasks.workunit.client.0.smithi100.stdout: [298 , 1222906 ] 2024-04-07T17:09:14.604 INFO:tasks.workunit.client.0.smithi100.stdout: [314 , 1287182 ] 2024-04-07T17:09:14.604 INFO:tasks.workunit.client.0.smithi100.stdout: [315 , 1293544 ] 2024-04-07T17:09:14.604 INFO:tasks.workunit.client.0.smithi100.stdout: [317 , 1300621 ] 2024-04-07T17:09:14.604 INFO:tasks.workunit.client.0.smithi100.stdout: [326 , 1335314 ] 2024-04-07T17:09:14.604 INFO:tasks.workunit.client.0.smithi100.stdout: [329 , 1348663 ] 2024-04-07T17:09:14.604 INFO:tasks.workunit.client.0.smithi100.stdout: [339 , 1391828 ] 2024-04-07T17:09:14.604 INFO:tasks.workunit.client.0.smithi100.stdout: [347 , 1422992 ] 2024-04-07T17:09:14.604 INFO:tasks.workunit.client.0.smithi100.stdout: [366 , 1499436 ] 2024-04-07T17:09:14.604 INFO:tasks.workunit.client.0.smithi100.stdout: [371 , 1520440 ] 2024-04-07T17:09:14.604 INFO:tasks.workunit.client.0.smithi100.stdout: [381 , 1562583 ] 2024-04-07T17:09:14.604 INFO:tasks.workunit.client.0.smithi100.stdout: [386 , 1581438 ] 2024-04-07T17:09:14.604 INFO:tasks.workunit.client.0.smithi100.stdout: [387 , 1587798 ] 2024-04-07T17:09:14.604 INFO:tasks.workunit.client.0.smithi100.stdout: [396 , 1624024 ] 2024-04-07T17:09:14.605 INFO:tasks.workunit.client.0.smithi100.stdout: [399 , 1636149 ] 2024-04-07T17:09:14.605 INFO:tasks.workunit.client.0.smithi100.stdout: [415 , 1702523 ] 2024-04-07T17:09:14.605 INFO:tasks.workunit.client.0.smithi100.stdout: [428 , 1755573 ] 2024-04-07T17:09:14.605 INFO:tasks.workunit.client.0.smithi100.stdout: [430 , 1763786 ] 2024-04-07T17:09:14.605 INFO:tasks.workunit.client.0.smithi100.stdout: [436 , 1788015 ] 2024-04-07T17:09:14.605 INFO:tasks.workunit.client.0.smithi100.stdout: [448 , 1837595 ] 2024-04-07T17:09:14.605 INFO:tasks.workunit.client.0.smithi100.stdout: [461 , 1890415 ] 2024-04-07T17:09:14.605 INFO:tasks.workunit.client.0.smithi100.stdout: [469 , 1922822 ] 2024-04-07T17:09:14.605 INFO:tasks.workunit.client.0.smithi100.stdout: [497 , 2038296 ] 2024-04-07T17:09:14.605 INFO:tasks.workunit.client.0.smithi100.stdout: [498 , 2040725 ] 2024-04-07T17:09:14.605 INFO:tasks.workunit.client.0.smithi100.stdout: [499 , 2044038 ] 2024-04-07T17:09:14.605 INFO:tasks.workunit.client.0.smithi100.stdout: [508 , 2082299 ] 2024-04-07T17:09:14.605 INFO:tasks.workunit.client.0.smithi100.stdout: [518 , 2122351 ] 2024-04-07T17:09:14.605 INFO:tasks.workunit.client.0.smithi100.stdout: [524 , 2146847 ] 2024-04-07T17:09:14.605 INFO:tasks.workunit.client.0.smithi100.stdout: [550 , 2256167 ] 2024-04-07T17:09:14.605 INFO:tasks.workunit.client.0.smithi100.stdout: [555 , 2273600 ] 2024-04-07T17:09:14.606 INFO:tasks.workunit.client.0.smithi100.stdout: [571 , 2341740 ] 2024-04-07T17:09:14.606 INFO:tasks.workunit.client.0.smithi100.stdout: [579 , 2373861 ] 2024-04-07T17:09:14.606 INFO:tasks.workunit.client.0.smithi100.stdout: [580 , 2376350 ] 2024-04-07T17:09:14.606 INFO:tasks.workunit.client.0.smithi100.stdout: [595 , 2438710 ] 2024-04-07T17:09:14.606 INFO:tasks.workunit.client.0.smithi100.stdout: [605 , 2479207 ] 2024-04-07T17:09:14.606 INFO:tasks.workunit.client.0.smithi100.stdout: [622 , 2551010 ] 2024-04-07T17:09:14.606 INFO:tasks.workunit.client.0.smithi100.stdout: [627 , 2569894 ] 2024-04-07T17:09:14.606 INFO:tasks.workunit.client.0.smithi100.stdout: [670 , 2744548 ] 2024-04-07T17:09:14.606 INFO:tasks.workunit.client.0.smithi100.stdout: [673 , 2758180 ] 2024-04-07T17:09:14.606 INFO:tasks.workunit.client.0.smithi100.stdout: [710 , 2910790 ] 2024-04-07T17:09:14.606 INFO:tasks.workunit.client.0.smithi100.stdout: [719 , 2947773 ] 2024-04-07T17:09:14.606 INFO:tasks.workunit.client.0.smithi100.stdout: [763 , 3126500 ] 2024-04-07T17:09:14.606 INFO:tasks.workunit.client.0.smithi100.stdout: [782 , 3203812 ] 2024-04-07T17:09:14.606 INFO:tasks.workunit.client.0.smithi100.stdout: [783 , 3210135 ] 2024-04-07T17:09:14.606 INFO:tasks.workunit.client.0.smithi100.stdout: [793 , 3251597 ] 2024-04-07T17:09:14.607 INFO:tasks.workunit.client.0.smithi100.stdout: [801 , 3281666 ] 2024-04-07T17:09:14.607 INFO:tasks.workunit.client.0.smithi100.stdout: [805 , 3299909 ] 2024-04-07T17:09:14.607 INFO:tasks.workunit.client.0.smithi100.stdout: [824 , 3376231 ] 2024-04-07T17:09:14.607 INFO:tasks.workunit.client.0.smithi100.stdout: [828 , 3393182 ] 2024-04-07T17:09:14.607 INFO:tasks.workunit.client.0.smithi100.stdout: [839 , 3438941 ] 2024-04-07T17:09:14.607 INFO:tasks.workunit.client.0.smithi100.stdout: [841 , 3446039 ] 2024-04-07T17:09:14.607 INFO:tasks.workunit.client.0.smithi100.stdout: [859 , 3520132 ] 2024-04-07T17:09:14.607 INFO:tasks.workunit.client.0.smithi100.stdout: [862 , 3534014 ] 2024-04-07T17:09:14.607 INFO:tasks.workunit.client.0.smithi100.stdout: [870 , 3566232 ] 2024-04-07T17:09:14.607 INFO:tasks.workunit.client.0.smithi100.stdout: [882 , 3615461 ] 2024-04-07T17:09:14.607 INFO:tasks.workunit.client.0.smithi100.stdout: [893 , 3658888 ] 2024-04-07T17:09:14.607 INFO:tasks.workunit.client.0.smithi100.stdout: [906 , 3711931 ] 2024-04-07T17:09:14.607 INFO:tasks.workunit.client.0.smithi100.stdout: [917 , 3756165 ] 2024-04-07T17:09:14.607 INFO:tasks.workunit.client.0.smithi100.stdout: [922 , 3777561 ] 2024-04-07T17:09:14.607 INFO:tasks.workunit.client.0.smithi100.stdout: [940 , 3851250 ] 2024-04-07T17:09:14.608 INFO:tasks.workunit.client.0.smithi100.stdout: [947 , 3881967 ] 2024-04-07T17:09:14.608 INFO:tasks.workunit.client.0.smithi100.stdout: [982 , 4023205 ] 2024-04-07T17:09:14.608 INFO:tasks.workunit.client.0.smithi100.stdout: [989 , 4051543 ] 2024-04-07T17:09:14.608 INFO:tasks.workunit.client.0.smithi100.stdout: [1002 , 4105910 ] 2024-04-07T17:09:14.608 INFO:tasks.workunit.client.0.smithi100.stdout: [1005 , 4118567 ] 2024-04-07T17:09:14.608 INFO:tasks.workunit.client.0.smithi100.stdout: [1013 , 4151252 ] 2024-04-07T17:09:14.608 INFO:tasks.workunit.client.0.smithi100.stdout:write data based on random map 2024-04-07T17:09:14.608 INFO:tasks.workunit.client.0.smithi100.stdout: write object-20 2024-04-07T17:09:14.608 INFO:tasks.workunit.client.0.smithi100.stdout: write object-27 2024-04-07T17:09:14.608 INFO:tasks.workunit.client.0.smithi100.stdout: write object-28 2024-04-07T17:09:14.608 INFO:tasks.workunit.client.0.smithi100.stdout: write object-29 2024-04-07T17:09:14.608 INFO:tasks.workunit.client.0.smithi100.stdout: write object-43 2024-04-07T17:09:14.608 INFO:tasks.workunit.client.0.smithi100.stdout: write object-50 2024-04-07T17:09:14.608 INFO:tasks.workunit.client.0.smithi100.stdout: write object-74 2024-04-07T17:09:14.608 INFO:tasks.workunit.client.0.smithi100.stdout: write object-82 2024-04-07T17:09:14.609 INFO:tasks.workunit.client.0.smithi100.stdout: write object-98 2024-04-07T17:09:14.609 INFO:tasks.workunit.client.0.smithi100.stdout: write object-99 2024-04-07T17:09:14.609 INFO:tasks.workunit.client.0.smithi100.stdout: write object-107 2024-04-07T17:09:14.609 INFO:tasks.workunit.client.0.smithi100.stdout: write object-117 2024-04-07T17:09:14.609 INFO:tasks.workunit.client.0.smithi100.stdout: write object-122 2024-04-07T17:09:14.609 INFO:tasks.workunit.client.0.smithi100.stdout: write object-125 2024-04-07T17:09:14.609 INFO:tasks.workunit.client.0.smithi100.stdout: write object-149 2024-04-07T17:09:14.609 INFO:tasks.workunit.client.0.smithi100.stdout: write object-154 2024-04-07T17:09:14.609 INFO:tasks.workunit.client.0.smithi100.stdout: write object-156 2024-04-07T17:09:14.609 INFO:tasks.workunit.client.0.smithi100.stdout: write object-160 2024-04-07T17:09:14.609 INFO:tasks.workunit.client.0.smithi100.stdout: write object-162 2024-04-07T17:09:14.609 INFO:tasks.workunit.client.0.smithi100.stdout: write object-166 2024-04-07T17:09:14.609 INFO:tasks.workunit.client.0.smithi100.stdout: write object-175 2024-04-07T17:09:14.609 INFO:tasks.workunit.client.0.smithi100.stdout: write object-179 2024-04-07T17:09:14.610 INFO:tasks.workunit.client.0.smithi100.stdout: write object-181 2024-04-07T17:09:14.610 INFO:tasks.workunit.client.0.smithi100.stdout: write object-182 2024-04-07T17:09:14.610 INFO:tasks.workunit.client.0.smithi100.stdout: write object-207 2024-04-07T17:09:14.610 INFO:tasks.workunit.client.0.smithi100.stdout: write object-214 2024-04-07T17:09:14.610 INFO:tasks.workunit.client.0.smithi100.stdout: write object-232 2024-04-07T17:09:14.610 INFO:tasks.workunit.client.0.smithi100.stdout: write object-233 2024-04-07T17:09:14.610 INFO:tasks.workunit.client.0.smithi100.stdout: write object-236 2024-04-07T17:09:14.610 INFO:tasks.workunit.client.0.smithi100.stdout: write object-276 2024-04-07T17:09:14.610 INFO:tasks.workunit.client.0.smithi100.stdout: write object-282 2024-04-07T17:09:14.610 INFO:tasks.workunit.client.0.smithi100.stdout: write object-298 2024-04-07T17:09:14.610 INFO:tasks.workunit.client.0.smithi100.stdout: write object-314 2024-04-07T17:09:14.610 INFO:tasks.workunit.client.0.smithi100.stdout: write object-315 2024-04-07T17:09:14.610 INFO:tasks.workunit.client.0.smithi100.stdout: write object-317 2024-04-07T17:09:14.610 INFO:tasks.workunit.client.0.smithi100.stdout: write object-326 2024-04-07T17:09:14.611 INFO:tasks.workunit.client.0.smithi100.stdout: write object-329 2024-04-07T17:09:14.611 INFO:tasks.workunit.client.0.smithi100.stdout: write object-339 2024-04-07T17:09:14.611 INFO:tasks.workunit.client.0.smithi100.stdout: write object-347 2024-04-07T17:09:14.611 INFO:tasks.workunit.client.0.smithi100.stdout: write object-366 2024-04-07T17:09:14.611 INFO:tasks.workunit.client.0.smithi100.stdout: write object-371 2024-04-07T17:09:14.611 INFO:tasks.workunit.client.0.smithi100.stdout: write object-381 2024-04-07T17:09:14.611 INFO:tasks.workunit.client.0.smithi100.stdout: write object-386 2024-04-07T17:09:14.611 INFO:tasks.workunit.client.0.smithi100.stdout: write object-387 2024-04-07T17:09:14.611 INFO:tasks.workunit.client.0.smithi100.stdout: write object-396 2024-04-07T17:09:14.611 INFO:tasks.workunit.client.0.smithi100.stdout: write object-399 2024-04-07T17:09:14.611 INFO:tasks.workunit.client.0.smithi100.stdout: write object-415 2024-04-07T17:09:14.611 INFO:tasks.workunit.client.0.smithi100.stdout: write object-428 2024-04-07T17:09:14.611 INFO:tasks.workunit.client.0.smithi100.stdout: write object-430 2024-04-07T17:09:14.611 INFO:tasks.workunit.client.0.smithi100.stdout: write object-436 2024-04-07T17:09:14.611 INFO:tasks.workunit.client.0.smithi100.stdout: write object-448 2024-04-07T17:09:14.612 INFO:tasks.workunit.client.0.smithi100.stdout: write object-461 2024-04-07T17:09:14.612 INFO:tasks.workunit.client.0.smithi100.stdout: write object-469 2024-04-07T17:09:14.612 INFO:tasks.workunit.client.0.smithi100.stdout: write object-497 2024-04-07T17:09:14.612 INFO:tasks.workunit.client.0.smithi100.stdout: write object-498 2024-04-07T17:09:14.612 INFO:tasks.workunit.client.0.smithi100.stdout: write object-499 2024-04-07T17:09:14.612 INFO:tasks.workunit.client.0.smithi100.stdout: write object-508 2024-04-07T17:09:14.612 INFO:tasks.workunit.client.0.smithi100.stdout: write object-518 2024-04-07T17:09:14.612 INFO:tasks.workunit.client.0.smithi100.stdout: write object-524 2024-04-07T17:09:14.612 INFO:tasks.workunit.client.0.smithi100.stdout: write object-550 2024-04-07T17:09:14.612 INFO:tasks.workunit.client.0.smithi100.stdout: write object-555 2024-04-07T17:09:14.612 INFO:tasks.workunit.client.0.smithi100.stdout: write object-571 2024-04-07T17:09:14.612 INFO:tasks.workunit.client.0.smithi100.stdout: write object-579 2024-04-07T17:09:14.612 INFO:tasks.workunit.client.0.smithi100.stdout: write object-580 2024-04-07T17:09:14.612 INFO:tasks.workunit.client.0.smithi100.stdout: write object-595 2024-04-07T17:09:14.612 INFO:tasks.workunit.client.0.smithi100.stdout: write object-605 2024-04-07T17:09:14.612 INFO:tasks.workunit.client.0.smithi100.stdout: write object-622 2024-04-07T17:09:14.612 INFO:tasks.workunit.client.0.smithi100.stdout: write object-627 2024-04-07T17:09:14.613 INFO:tasks.workunit.client.0.smithi100.stdout: write object-670 2024-04-07T17:09:14.613 INFO:tasks.workunit.client.0.smithi100.stdout: write object-673 2024-04-07T17:09:14.613 INFO:tasks.workunit.client.0.smithi100.stdout: write object-710 2024-04-07T17:09:14.613 INFO:tasks.workunit.client.0.smithi100.stdout: write object-719 2024-04-07T17:09:14.613 INFO:tasks.workunit.client.0.smithi100.stdout: write object-763 2024-04-07T17:09:14.613 INFO:tasks.workunit.client.0.smithi100.stdout: write object-782 2024-04-07T17:09:14.613 INFO:tasks.workunit.client.0.smithi100.stdout: write object-783 2024-04-07T17:09:14.613 INFO:tasks.workunit.client.0.smithi100.stdout: write object-793 2024-04-07T17:09:14.613 INFO:tasks.workunit.client.0.smithi100.stdout: write object-801 2024-04-07T17:09:14.613 INFO:tasks.workunit.client.0.smithi100.stdout: write object-805 2024-04-07T17:09:14.613 INFO:tasks.workunit.client.0.smithi100.stdout: write object-824 2024-04-07T17:09:14.613 INFO:tasks.workunit.client.0.smithi100.stdout: write object-828 2024-04-07T17:09:14.613 INFO:tasks.workunit.client.0.smithi100.stdout: write object-839 2024-04-07T17:09:14.613 INFO:tasks.workunit.client.0.smithi100.stdout: write object-841 2024-04-07T17:09:14.613 INFO:tasks.workunit.client.0.smithi100.stdout: write object-859 2024-04-07T17:09:14.613 INFO:tasks.workunit.client.0.smithi100.stdout: write object-862 2024-04-07T17:09:14.613 INFO:tasks.workunit.client.0.smithi100.stdout: write object-870 2024-04-07T17:09:14.614 INFO:tasks.workunit.client.0.smithi100.stdout: write object-882 2024-04-07T17:09:14.614 INFO:tasks.workunit.client.0.smithi100.stdout: write object-893 2024-04-07T17:09:15.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:09:14 smithi100 bash[74942]: cluster 2024-04-07T17:09:13.913922+0000 mon.a (mon.0) 2891 : cluster [DBG] osdmap e1563: 8 total, 8 up, 8 in 2024-04-07T17:09:15.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:09:14 smithi100 bash[74942]: cluster 2024-04-07T17:09:13.989799+0000 mgr.y (mgr.25175) 1616 : cluster [DBG] pgmap v2754: 137 pgs: 137 active+clean; 8.6 MiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 3.1 MiB/s rd, 3.0 MiB/s wr, 148 op/s 2024-04-07T17:09:15.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:09:14 smithi100 bash[76202]: cluster 2024-04-07T17:09:13.913922+0000 mon.a (mon.0) 2891 : cluster [DBG] osdmap e1563: 8 total, 8 up, 8 in 2024-04-07T17:09:15.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:09:14 smithi100 bash[76202]: cluster 2024-04-07T17:09:13.989799+0000 mgr.y (mgr.25175) 1616 : cluster [DBG] pgmap v2754: 137 pgs: 137 active+clean; 8.6 MiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 3.1 MiB/s rd, 3.0 MiB/s wr, 148 op/s 2024-04-07T17:09:15.310 INFO:tasks.workunit.client.0.smithi100.stdout: write object-906 2024-04-07T17:09:15.310 INFO:tasks.workunit.client.0.smithi100.stdout: write object-917 2024-04-07T17:09:15.310 INFO:tasks.workunit.client.0.smithi100.stdout: write object-922 2024-04-07T17:09:15.310 INFO:tasks.workunit.client.0.smithi100.stdout: write object-940 2024-04-07T17:09:15.311 INFO:tasks.workunit.client.0.smithi100.stdout: write object-947 2024-04-07T17:09:15.311 INFO:tasks.workunit.client.0.smithi100.stdout: write object-982 2024-04-07T17:09:15.311 INFO:tasks.workunit.client.0.smithi100.stdout: write object-989 2024-04-07T17:09:15.311 INFO:tasks.workunit.client.0.smithi100.stdout: write object-1002 2024-04-07T17:09:15.311 INFO:tasks.workunit.client.0.smithi100.stdout: write object-1005 2024-04-07T17:09:15.311 INFO:tasks.workunit.client.0.smithi100.stdout: write object-1013 2024-04-07T17:09:15.311 INFO:tasks.workunit.client.0.smithi100.stdout:verify written data by reading 2024-04-07T17:09:15.311 INFO:tasks.workunit.client.0.smithi100.stdout: read object-20 2024-04-07T17:09:15.311 INFO:tasks.workunit.client.0.smithi100.stdout:made snapshot "image390@parent_snap" and protect it 2024-04-07T17:09:15.311 INFO:tasks.workunit.client.0.smithi100.stdout:made and opened clone "image391" 2024-04-07T17:09:15.311 INFO:tasks.workunit.client.0.smithi100.stdout:flattening clone: "image391" 2024-04-07T17:09:15.311 INFO:tasks.workunit.client.0.smithi100.stdout:check whether child image has the same set of objects as parent 2024-04-07T17:09:15.311 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestInternal.FlattenNoEmptyObjects (1311 ms) 2024-04-07T17:09:15.312 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestInternal.PoolMetadataConfApply 2024-04-07T17:09:15.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:09:14 smithi144 bash[66654]: cluster 2024-04-07T17:09:13.913922+0000 mon.a (mon.0) 2891 : cluster [DBG] osdmap e1563: 8 total, 8 up, 8 in 2024-04-07T17:09:15.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:09:14 smithi144 bash[66654]: cluster 2024-04-07T17:09:13.989799+0000 mgr.y (mgr.25175) 1616 : cluster [DBG] pgmap v2754: 137 pgs: 137 active+clean; 8.6 MiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 3.1 MiB/s rd, 3.0 MiB/s wr, 148 op/s 2024-04-07T17:09:15.422 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestInternal.PoolMetadataConfApply (113 ms) 2024-04-07T17:09:15.422 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestInternal.Sparsify 2024-04-07T17:09:15.453 INFO:tasks.workunit.client.0.smithi100.stdout:sparsify_supported=1 2024-04-07T17:09:15.454 INFO:tasks.workunit.client.0.smithi100.stdout:sparse_read_supported=1 2024-04-07T17:09:15.490 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestInternal.Sparsify (67 ms) 2024-04-07T17:09:15.490 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestInternal.SparsifyClone 2024-04-07T17:09:15.514 INFO:tasks.workunit.client.0.smithi100.stdout:sparsify_supported=1 2024-04-07T17:09:15.703 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:09:15 smithi100 bash[68137]: [07/Apr/2024:17:09:15] ENGINE Bus STOPPING 2024-04-07T17:09:15.703 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:09:15 smithi100 bash[68137]: [07/Apr/2024:17:09:15] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-04-07T17:09:15.703 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:09:15 smithi100 bash[68137]: [07/Apr/2024:17:09:15] ENGINE Bus STOPPED 2024-04-07T17:09:15.703 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:09:15 smithi100 bash[68137]: [07/Apr/2024:17:09:15] ENGINE Bus STARTING 2024-04-07T17:09:15.703 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:09:15 smithi100 bash[68137]: [07/Apr/2024:17:09:15] ENGINE Serving on http://:::9283 2024-04-07T17:09:15.703 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:09:15 smithi100 bash[68137]: [07/Apr/2024:17:09:15] ENGINE Bus STARTED 2024-04-07T17:09:16.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:09:15 smithi100 bash[74942]: cluster 2024-04-07T17:09:14.938559+0000 mon.a (mon.0) 2892 : cluster [DBG] osdmap e1564: 8 total, 8 up, 8 in 2024-04-07T17:09:16.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:09:15 smithi100 bash[74942]: audit 2024-04-07T17:09:15.339653+0000 mon.a (mon.0) 2893 : audit [INF] from='client.? ' entity='client.admin' 2024-04-07T17:09:16.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:09:15 smithi100 bash[74942]: cluster 2024-04-07T17:09:15.368750+0000 mgr.y (mgr.25175) 1617 : cluster [DBG] pgmap v2756: 137 pgs: 137 active+clean; 8.7 MiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 3.6 MiB/s rd, 3.5 MiB/s wr, 245 op/s 2024-04-07T17:09:16.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:09:15 smithi100 bash[74942]: audit 2024-04-07T17:09:15.370773+0000 mon.a (mon.0) 2894 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:09:16.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:09:15 smithi100 bash[74942]: audit 2024-04-07T17:09:15.380687+0000 mon.a (mon.0) 2895 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:09:16.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:09:15 smithi100 bash[74942]: audit 2024-04-07T17:09:15.382390+0000 mon.a (mon.0) 2896 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:09:16.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:09:15 smithi100 bash[74942]: audit 2024-04-07T17:09:15.383195+0000 mon.a (mon.0) 2897 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:09:16.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:09:15 smithi100 bash[74942]: audit 2024-04-07T17:09:15.388747+0000 mon.a (mon.0) 2898 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:09:16.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:09:15 smithi100 bash[76202]: cluster 2024-04-07T17:09:14.938559+0000 mon.a (mon.0) 2892 : cluster [DBG] osdmap e1564: 8 total, 8 up, 8 in 2024-04-07T17:09:16.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:09:15 smithi100 bash[76202]: audit 2024-04-07T17:09:15.339653+0000 mon.a (mon.0) 2893 : audit [INF] from='client.? ' entity='client.admin' 2024-04-07T17:09:16.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:09:15 smithi100 bash[76202]: cluster 2024-04-07T17:09:15.368750+0000 mgr.y (mgr.25175) 1617 : cluster [DBG] pgmap v2756: 137 pgs: 137 active+clean; 8.7 MiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 3.6 MiB/s rd, 3.5 MiB/s wr, 245 op/s 2024-04-07T17:09:16.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:09:15 smithi100 bash[76202]: audit 2024-04-07T17:09:15.370773+0000 mon.a (mon.0) 2894 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:09:16.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:09:15 smithi100 bash[76202]: audit 2024-04-07T17:09:15.380687+0000 mon.a (mon.0) 2895 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:09:16.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:09:15 smithi100 bash[76202]: audit 2024-04-07T17:09:15.382390+0000 mon.a (mon.0) 2896 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:09:16.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:09:15 smithi100 bash[76202]: audit 2024-04-07T17:09:15.383195+0000 mon.a (mon.0) 2897 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:09:16.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:09:15 smithi100 bash[76202]: audit 2024-04-07T17:09:15.388747+0000 mon.a (mon.0) 2898 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:09:16.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:09:15 smithi144 bash[66654]: cluster 2024-04-07T17:09:14.938559+0000 mon.a (mon.0) 2892 : cluster [DBG] osdmap e1564: 8 total, 8 up, 8 in 2024-04-07T17:09:16.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:09:15 smithi144 bash[66654]: audit 2024-04-07T17:09:15.339653+0000 mon.a (mon.0) 2893 : audit [INF] from='client.? ' entity='client.admin' 2024-04-07T17:09:16.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:09:15 smithi144 bash[66654]: cluster 2024-04-07T17:09:15.368750+0000 mgr.y (mgr.25175) 1617 : cluster [DBG] pgmap v2756: 137 pgs: 137 active+clean; 8.7 MiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 3.6 MiB/s rd, 3.5 MiB/s wr, 245 op/s 2024-04-07T17:09:16.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:09:15 smithi144 bash[66654]: audit 2024-04-07T17:09:15.370773+0000 mon.a (mon.0) 2894 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:09:16.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:09:15 smithi144 bash[66654]: audit 2024-04-07T17:09:15.380687+0000 mon.a (mon.0) 2895 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:09:16.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:09:15 smithi144 bash[66654]: audit 2024-04-07T17:09:15.382390+0000 mon.a (mon.0) 2896 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:09:16.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:09:15 smithi144 bash[66654]: audit 2024-04-07T17:09:15.383195+0000 mon.a (mon.0) 2897 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:09:16.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:09:15 smithi144 bash[66654]: audit 2024-04-07T17:09:15.388747+0000 mon.a (mon.0) 2898 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:09:17.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:09:16 smithi100 bash[74942]: cluster 2024-04-07T17:09:16.337285+0000 mon.a (mon.0) 2899 : cluster [WRN] Health check failed: Telemetry requires re-opt-in (TELEMETRY_CHANGED) 2024-04-07T17:09:17.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:09:16 smithi100 bash[74942]: cluster 2024-04-07T17:09:16.397179+0000 mon.a (mon.0) 2900 : cluster [DBG] osdmap e1565: 8 total, 8 up, 8 in 2024-04-07T17:09:17.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:09:16 smithi100 bash[74942]: audit 2024-04-07T17:09:16.819082+0000 mon.a (mon.0) 2901 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:09:17.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:09:16 smithi100 bash[74942]: audit 2024-04-07T17:09:16.821742+0000 mon.a (mon.0) 2902 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:09:17.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:09:16 smithi100 bash[76202]: cluster 2024-04-07T17:09:16.337285+0000 mon.a (mon.0) 2899 : cluster [WRN] Health check failed: Telemetry requires re-opt-in (TELEMETRY_CHANGED) 2024-04-07T17:09:17.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:09:16 smithi100 bash[76202]: cluster 2024-04-07T17:09:16.397179+0000 mon.a (mon.0) 2900 : cluster [DBG] osdmap e1565: 8 total, 8 up, 8 in 2024-04-07T17:09:17.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:09:16 smithi100 bash[76202]: audit 2024-04-07T17:09:16.819082+0000 mon.a (mon.0) 2901 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:09:17.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:09:16 smithi100 bash[76202]: audit 2024-04-07T17:09:16.821742+0000 mon.a (mon.0) 2902 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:09:17.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:09:16 smithi144 bash[66654]: cluster 2024-04-07T17:09:16.337285+0000 mon.a (mon.0) 2899 : cluster [WRN] Health check failed: Telemetry requires re-opt-in (TELEMETRY_CHANGED) 2024-04-07T17:09:17.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:09:16 smithi144 bash[66654]: cluster 2024-04-07T17:09:16.397179+0000 mon.a (mon.0) 2900 : cluster [DBG] osdmap e1565: 8 total, 8 up, 8 in 2024-04-07T17:09:17.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:09:16 smithi144 bash[66654]: audit 2024-04-07T17:09:16.819082+0000 mon.a (mon.0) 2901 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:09:17.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:09:16 smithi144 bash[66654]: audit 2024-04-07T17:09:16.821742+0000 mon.a (mon.0) 2902 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:09:17.752 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestInternal.SparsifyClone (2262 ms) 2024-04-07T17:09:17.752 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestInternal.MissingDataPool 2024-04-07T17:09:18.703 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:09:18 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:09:18] "GET /metrics HTTP/1.1" 200 36558 "" "Prometheus/2.43.0" 2024-04-07T17:09:19.003 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestInternal.MissingDataPool (1251 ms) 2024-04-07T17:09:19.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:09:18 smithi144 bash[66654]: cluster 2024-04-07T17:09:17.369954+0000 mgr.y (mgr.25175) 1618 : cluster [DBG] pgmap v2758: 137 pgs: 137 active+clean; 8.9 MiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 431 KiB/s rd, 144 KiB/s wr, 672 op/s 2024-04-07T17:09:19.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:09:18 smithi144 bash[66654]: cluster 2024-04-07T17:09:17.700671+0000 mon.a (mon.0) 2903 : cluster [DBG] osdmap e1566: 8 total, 8 up, 8 in 2024-04-07T17:09:19.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:09:18 smithi100 bash[74942]: cluster 2024-04-07T17:09:17.369954+0000 mgr.y (mgr.25175) 1618 : cluster [DBG] pgmap v2758: 137 pgs: 137 active+clean; 8.9 MiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 431 KiB/s rd, 144 KiB/s wr, 672 op/s 2024-04-07T17:09:19.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:09:18 smithi100 bash[74942]: cluster 2024-04-07T17:09:17.700671+0000 mon.a (mon.0) 2903 : cluster [DBG] osdmap e1566: 8 total, 8 up, 8 in 2024-04-07T17:09:19.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:09:18 smithi100 bash[76202]: cluster 2024-04-07T17:09:17.369954+0000 mgr.y (mgr.25175) 1618 : cluster [DBG] pgmap v2758: 137 pgs: 137 active+clean; 8.9 MiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 431 KiB/s rd, 144 KiB/s wr, 672 op/s 2024-04-07T17:09:19.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:09:18 smithi100 bash[76202]: cluster 2024-04-07T17:09:17.700671+0000 mon.a (mon.0) 2903 : cluster [DBG] osdmap e1566: 8 total, 8 up, 8 in 2024-04-07T17:09:19.737 INFO:tasks.workunit.client.0.smithi100.stdout:[----------] 34 tests from TestInternal (35911 ms total) 2024-04-07T17:09:19.737 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:09:19.737 INFO:tasks.workunit.client.0.smithi100.stdout:[----------] 17 tests from TestImageWatcher 2024-04-07T17:09:20.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:09:19 smithi144 bash[66654]: cluster 2024-04-07T17:09:18.704127+0000 mon.a (mon.0) 2904 : cluster [DBG] osdmap e1567: 8 total, 8 up, 8 in 2024-04-07T17:09:20.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:09:19 smithi144 bash[66654]: audit 2024-04-07T17:09:19.012276+0000 mon.a (mon.0) 2905 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "89.c", "id": [7, 2]}]: dispatch 2024-04-07T17:09:20.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:09:19 smithi144 bash[66654]: audit 2024-04-07T17:09:19.012722+0000 mon.a (mon.0) 2906 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "89.1b", "id": [7, 4]}]: dispatch 2024-04-07T17:09:20.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:09:19 smithi100 bash[74942]: cluster 2024-04-07T17:09:18.704127+0000 mon.a (mon.0) 2904 : cluster [DBG] osdmap e1567: 8 total, 8 up, 8 in 2024-04-07T17:09:20.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:09:19 smithi100 bash[74942]: audit 2024-04-07T17:09:19.012276+0000 mon.a (mon.0) 2905 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "89.c", "id": [7, 2]}]: dispatch 2024-04-07T17:09:20.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:09:19 smithi100 bash[74942]: audit 2024-04-07T17:09:19.012722+0000 mon.a (mon.0) 2906 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "89.1b", "id": [7, 4]}]: dispatch 2024-04-07T17:09:20.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:09:19 smithi100 bash[76202]: cluster 2024-04-07T17:09:18.704127+0000 mon.a (mon.0) 2904 : cluster [DBG] osdmap e1567: 8 total, 8 up, 8 in 2024-04-07T17:09:20.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:09:19 smithi100 bash[76202]: audit 2024-04-07T17:09:19.012276+0000 mon.a (mon.0) 2905 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "89.c", "id": [7, 2]}]: dispatch 2024-04-07T17:09:20.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:09:19 smithi100 bash[76202]: audit 2024-04-07T17:09:19.012722+0000 mon.a (mon.0) 2906 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "89.1b", "id": [7, 4]}]: dispatch 2024-04-07T17:09:21.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:09:20 smithi144 bash[66654]: cluster 2024-04-07T17:09:19.370902+0000 mgr.y (mgr.25175) 1619 : cluster [DBG] pgmap v2761: 137 pgs: 137 active+clean; 9.0 MiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 481 KiB/s rd, 159 KiB/s wr, 750 op/s 2024-04-07T17:09:21.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:09:20 smithi144 bash[66654]: audit 2024-04-07T17:09:19.719253+0000 mon.a (mon.0) 2907 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "89.c", "id": [7, 2]}]': finished 2024-04-07T17:09:21.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:09:20 smithi144 bash[66654]: audit 2024-04-07T17:09:19.719522+0000 mon.a (mon.0) 2908 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "89.1b", "id": [7, 4]}]': finished 2024-04-07T17:09:21.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:09:20 smithi144 bash[66654]: cluster 2024-04-07T17:09:19.729487+0000 mon.a (mon.0) 2909 : cluster [DBG] osdmap e1568: 8 total, 8 up, 8 in 2024-04-07T17:09:21.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:09:20 smithi100 bash[76202]: cluster 2024-04-07T17:09:19.370902+0000 mgr.y (mgr.25175) 1619 : cluster [DBG] pgmap v2761: 137 pgs: 137 active+clean; 9.0 MiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 481 KiB/s rd, 159 KiB/s wr, 750 op/s 2024-04-07T17:09:21.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:09:20 smithi100 bash[76202]: audit 2024-04-07T17:09:19.719253+0000 mon.a (mon.0) 2907 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "89.c", "id": [7, 2]}]': finished 2024-04-07T17:09:21.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:09:20 smithi100 bash[76202]: audit 2024-04-07T17:09:19.719522+0000 mon.a (mon.0) 2908 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "89.1b", "id": [7, 4]}]': finished 2024-04-07T17:09:21.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:09:20 smithi100 bash[76202]: cluster 2024-04-07T17:09:19.729487+0000 mon.a (mon.0) 2909 : cluster [DBG] osdmap e1568: 8 total, 8 up, 8 in 2024-04-07T17:09:21.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:09:20 smithi100 bash[74942]: cluster 2024-04-07T17:09:19.370902+0000 mgr.y (mgr.25175) 1619 : cluster [DBG] pgmap v2761: 137 pgs: 137 active+clean; 9.0 MiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 481 KiB/s rd, 159 KiB/s wr, 750 op/s 2024-04-07T17:09:21.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:09:20 smithi100 bash[74942]: audit 2024-04-07T17:09:19.719253+0000 mon.a (mon.0) 2907 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "89.c", "id": [7, 2]}]': finished 2024-04-07T17:09:21.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:09:20 smithi100 bash[74942]: audit 2024-04-07T17:09:19.719522+0000 mon.a (mon.0) 2908 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "89.1b", "id": [7, 4]}]': finished 2024-04-07T17:09:21.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:09:20 smithi100 bash[74942]: cluster 2024-04-07T17:09:19.729487+0000 mon.a (mon.0) 2909 : cluster [DBG] osdmap e1568: 8 total, 8 up, 8 in 2024-04-07T17:09:21.737 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestImageWatcher.NotifyHeaderUpdate 2024-04-07T17:09:22.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:09:21 smithi144 bash[66654]: cluster 2024-04-07T17:09:20.731959+0000 mon.a (mon.0) 2910 : cluster [DBG] osdmap e1569: 8 total, 8 up, 8 in 2024-04-07T17:09:22.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:09:21 smithi144 bash[66654]: audit 2024-04-07T17:09:20.734252+0000 mon.a (mon.0) 2911 : audit [INF] from='client.? 172.21.15.100:0/1378285109' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:09:22.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:09:21 smithi144 bash[66654]: audit 2024-04-07T17:09:21.726003+0000 mon.a (mon.0) 2912 : audit [INF] from='client.? 172.21.15.100:0/1378285109' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T17:09:22.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:09:21 smithi144 bash[66654]: cluster 2024-04-07T17:09:21.734706+0000 mon.a (mon.0) 2913 : cluster [DBG] osdmap e1570: 8 total, 8 up, 8 in 2024-04-07T17:09:22.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:09:21 smithi100 bash[74942]: cluster 2024-04-07T17:09:20.731959+0000 mon.a (mon.0) 2910 : cluster [DBG] osdmap e1569: 8 total, 8 up, 8 in 2024-04-07T17:09:22.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:09:21 smithi100 bash[74942]: audit 2024-04-07T17:09:20.734252+0000 mon.a (mon.0) 2911 : audit [INF] from='client.? 172.21.15.100:0/1378285109' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:09:22.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:09:21 smithi100 bash[74942]: audit 2024-04-07T17:09:21.726003+0000 mon.a (mon.0) 2912 : audit [INF] from='client.? 172.21.15.100:0/1378285109' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T17:09:22.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:09:21 smithi100 bash[74942]: cluster 2024-04-07T17:09:21.734706+0000 mon.a (mon.0) 2913 : cluster [DBG] osdmap e1570: 8 total, 8 up, 8 in 2024-04-07T17:09:22.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:09:21 smithi100 bash[76202]: cluster 2024-04-07T17:09:20.731959+0000 mon.a (mon.0) 2910 : cluster [DBG] osdmap e1569: 8 total, 8 up, 8 in 2024-04-07T17:09:22.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:09:21 smithi100 bash[76202]: audit 2024-04-07T17:09:20.734252+0000 mon.a (mon.0) 2911 : audit [INF] from='client.? 172.21.15.100:0/1378285109' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:09:22.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:09:21 smithi100 bash[76202]: audit 2024-04-07T17:09:21.726003+0000 mon.a (mon.0) 2912 : audit [INF] from='client.? 172.21.15.100:0/1378285109' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T17:09:22.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:09:21 smithi100 bash[76202]: cluster 2024-04-07T17:09:21.734706+0000 mon.a (mon.0) 2913 : cluster [DBG] osdmap e1570: 8 total, 8 up, 8 in 2024-04-07T17:09:22.616 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:09:22 smithi144 bash[95233]: ts=2024-04-07T17:09:22.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:09:23.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:09:22 smithi144 bash[66654]: cluster 2024-04-07T17:09:21.371775+0000 mgr.y (mgr.25175) 1620 : cluster [DBG] pgmap v2764: 137 pgs: 32 unknown, 105 active+clean; 4.6 MiB data, 3.4 GiB used, 712 GiB / 715 GiB avail 2024-04-07T17:09:23.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:09:22 smithi144 bash[66654]: cluster 2024-04-07T17:09:21.741557+0000 mon.a (mon.0) 2914 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T17:09:23.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:09:22 smithi144 bash[66654]: cluster 2024-04-07T17:09:22.735353+0000 mon.a (mon.0) 2915 : cluster [DBG] osdmap e1571: 8 total, 8 up, 8 in 2024-04-07T17:09:23.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:09:22 smithi100 bash[74942]: cluster 2024-04-07T17:09:21.371775+0000 mgr.y (mgr.25175) 1620 : cluster [DBG] pgmap v2764: 137 pgs: 32 unknown, 105 active+clean; 4.6 MiB data, 3.4 GiB used, 712 GiB / 715 GiB avail 2024-04-07T17:09:23.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:09:22 smithi100 bash[74942]: cluster 2024-04-07T17:09:21.741557+0000 mon.a (mon.0) 2914 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T17:09:23.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:09:22 smithi100 bash[74942]: cluster 2024-04-07T17:09:22.735353+0000 mon.a (mon.0) 2915 : cluster [DBG] osdmap e1571: 8 total, 8 up, 8 in 2024-04-07T17:09:23.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:09:22 smithi100 bash[76202]: cluster 2024-04-07T17:09:21.371775+0000 mgr.y (mgr.25175) 1620 : cluster [DBG] pgmap v2764: 137 pgs: 32 unknown, 105 active+clean; 4.6 MiB data, 3.4 GiB used, 712 GiB / 715 GiB avail 2024-04-07T17:09:23.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:09:22 smithi100 bash[76202]: cluster 2024-04-07T17:09:21.741557+0000 mon.a (mon.0) 2914 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T17:09:23.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:09:22 smithi100 bash[76202]: cluster 2024-04-07T17:09:22.735353+0000 mon.a (mon.0) 2915 : cluster [DBG] osdmap e1571: 8 total, 8 up, 8 in 2024-04-07T17:09:23.822 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestImageWatcher.NotifyHeaderUpdate (2085 ms) 2024-04-07T17:09:23.822 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestImageWatcher.NotifyFlatten 2024-04-07T17:09:23.877 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestImageWatcher.NotifyFlatten (54 ms) 2024-04-07T17:09:23.878 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestImageWatcher.NotifyResize 2024-04-07T17:09:23.921 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestImageWatcher.NotifyResize (45 ms) 2024-04-07T17:09:23.921 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestImageWatcher.NotifyRebuildObjectMap 2024-04-07T17:09:23.969 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestImageWatcher.NotifyRebuildObjectMap (48 ms) 2024-04-07T17:09:23.969 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestImageWatcher.NotifyUpdateFeatures 2024-04-07T17:09:24.020 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestImageWatcher.NotifyUpdateFeatures (51 ms) 2024-04-07T17:09:24.020 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestImageWatcher.NotifySnapCreate 2024-04-07T17:09:24.072 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestImageWatcher.NotifySnapCreate (52 ms) 2024-04-07T17:09:24.072 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestImageWatcher.NotifySnapCreateError 2024-04-07T17:09:24.113 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestImageWatcher.NotifySnapCreateError (41 ms) 2024-04-07T17:09:24.113 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestImageWatcher.NotifySnapRename 2024-04-07T17:09:24.154 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestImageWatcher.NotifySnapRename (42 ms) 2024-04-07T17:09:24.155 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestImageWatcher.NotifySnapRenameError 2024-04-07T17:09:24.205 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestImageWatcher.NotifySnapRenameError (51 ms) 2024-04-07T17:09:24.206 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestImageWatcher.NotifySnapRemove 2024-04-07T17:09:24.248 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestImageWatcher.NotifySnapRemove (42 ms) 2024-04-07T17:09:24.248 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestImageWatcher.NotifySnapProtect 2024-04-07T17:09:24.300 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestImageWatcher.NotifySnapProtect (52 ms) 2024-04-07T17:09:24.300 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestImageWatcher.NotifySnapUnprotect 2024-04-07T17:09:24.352 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestImageWatcher.NotifySnapUnprotect (52 ms) 2024-04-07T17:09:24.352 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestImageWatcher.NotifyRename 2024-04-07T17:09:24.407 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestImageWatcher.NotifyRename (56 ms) 2024-04-07T17:09:24.408 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestImageWatcher.NotifyAsyncTimedOut 2024-04-07T17:09:24.461 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestImageWatcher.NotifyAsyncTimedOut (53 ms) 2024-04-07T17:09:24.461 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestImageWatcher.NotifyAsyncError 2024-04-07T17:09:24.503 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestImageWatcher.NotifyAsyncError (43 ms) 2024-04-07T17:09:24.504 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestImageWatcher.NotifyAsyncCompleteError 2024-04-07T17:09:24.552 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestImageWatcher.NotifyAsyncCompleteError (48 ms) 2024-04-07T17:09:24.552 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestImageWatcher.NotifyAsyncRequestTimedOut 2024-04-07T17:09:24.602 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestImageWatcher.NotifyAsyncRequestTimedOut (50 ms) 2024-04-07T17:09:24.767 INFO:tasks.workunit.client.0.smithi100.stdout:[----------] 17 tests from TestImageWatcher (2865 ms total) 2024-04-07T17:09:24.767 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:09:24.767 INFO:tasks.workunit.client.0.smithi100.stdout:[----------] 168 tests from TestLibRBD 2024-04-07T17:09:24.823 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.CreateAndStat 2024-04-07T17:09:25.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:09:24 smithi144 bash[66654]: cluster 2024-04-07T17:09:23.372722+0000 mgr.y (mgr.25175) 1621 : cluster [DBG] pgmap v2767: 137 pgs: 13 unknown, 124 active+clean; 4.6 MiB data, 3.4 GiB used, 712 GiB / 715 GiB avail 2024-04-07T17:09:25.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:09:24 smithi144 bash[66654]: cluster 2024-04-07T17:09:23.738113+0000 mon.a (mon.0) 2916 : cluster [DBG] osdmap e1572: 8 total, 8 up, 8 in 2024-04-07T17:09:25.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:09:24 smithi144 bash[66654]: cluster 2024-04-07T17:09:23.746671+0000 mon.a (mon.0) 2917 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T17:09:25.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:09:24 smithi100 bash[74942]: cluster 2024-04-07T17:09:23.372722+0000 mgr.y (mgr.25175) 1621 : cluster [DBG] pgmap v2767: 137 pgs: 13 unknown, 124 active+clean; 4.6 MiB data, 3.4 GiB used, 712 GiB / 715 GiB avail 2024-04-07T17:09:25.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:09:24 smithi100 bash[74942]: cluster 2024-04-07T17:09:23.738113+0000 mon.a (mon.0) 2916 : cluster [DBG] osdmap e1572: 8 total, 8 up, 8 in 2024-04-07T17:09:25.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:09:24 smithi100 bash[74942]: cluster 2024-04-07T17:09:23.746671+0000 mon.a (mon.0) 2917 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T17:09:25.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:09:24 smithi100 bash[76202]: cluster 2024-04-07T17:09:23.372722+0000 mgr.y (mgr.25175) 1621 : cluster [DBG] pgmap v2767: 137 pgs: 13 unknown, 124 active+clean; 4.6 MiB data, 3.4 GiB used, 712 GiB / 715 GiB avail 2024-04-07T17:09:25.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:09:24 smithi100 bash[76202]: cluster 2024-04-07T17:09:23.738113+0000 mon.a (mon.0) 2916 : cluster [DBG] osdmap e1572: 8 total, 8 up, 8 in 2024-04-07T17:09:25.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:09:24 smithi100 bash[76202]: cluster 2024-04-07T17:09:23.746671+0000 mon.a (mon.0) 2917 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T17:09:26.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:09:25 smithi144 bash[66654]: cluster 2024-04-07T17:09:24.758898+0000 mon.a (mon.0) 2918 : cluster [DBG] osdmap e1573: 8 total, 8 up, 8 in 2024-04-07T17:09:26.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:09:25 smithi100 bash[74942]: cluster 2024-04-07T17:09:24.758898+0000 mon.a (mon.0) 2918 : cluster [DBG] osdmap e1573: 8 total, 8 up, 8 in 2024-04-07T17:09:26.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:09:25 smithi100 bash[76202]: cluster 2024-04-07T17:09:24.758898+0000 mon.a (mon.0) 2918 : cluster [DBG] osdmap e1573: 8 total, 8 up, 8 in 2024-04-07T17:09:26.806 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:09:27.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:09:26 smithi144 bash[66654]: cluster 2024-04-07T17:09:25.373728+0000 mgr.y (mgr.25175) 1622 : cluster [DBG] pgmap v2770: 105 pgs: 105 active+clean; 4.6 MiB data, 3.4 GiB used, 712 GiB / 715 GiB avail 2024-04-07T17:09:27.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:09:26 smithi144 bash[66654]: cluster 2024-04-07T17:09:25.774075+0000 mon.a (mon.0) 2919 : cluster [DBG] osdmap e1574: 8 total, 8 up, 8 in 2024-04-07T17:09:27.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:09:26 smithi144 bash[66654]: audit 2024-04-07T17:09:25.781992+0000 mon.a (mon.0) 2920 : audit [INF] from='client.? 172.21.15.100:0/1817382777' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:09:27.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:09:26 smithi100 bash[74942]: cluster 2024-04-07T17:09:25.373728+0000 mgr.y (mgr.25175) 1622 : cluster [DBG] pgmap v2770: 105 pgs: 105 active+clean; 4.6 MiB data, 3.4 GiB used, 712 GiB / 715 GiB avail 2024-04-07T17:09:27.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:09:26 smithi100 bash[74942]: cluster 2024-04-07T17:09:25.774075+0000 mon.a (mon.0) 2919 : cluster [DBG] osdmap e1574: 8 total, 8 up, 8 in 2024-04-07T17:09:27.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:09:26 smithi100 bash[74942]: audit 2024-04-07T17:09:25.781992+0000 mon.a (mon.0) 2920 : audit [INF] from='client.? 172.21.15.100:0/1817382777' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:09:27.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:09:26 smithi100 bash[76202]: cluster 2024-04-07T17:09:25.373728+0000 mgr.y (mgr.25175) 1622 : cluster [DBG] pgmap v2770: 105 pgs: 105 active+clean; 4.6 MiB data, 3.4 GiB used, 712 GiB / 715 GiB avail 2024-04-07T17:09:27.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:09:26 smithi100 bash[76202]: cluster 2024-04-07T17:09:25.774075+0000 mon.a (mon.0) 2919 : cluster [DBG] osdmap e1574: 8 total, 8 up, 8 in 2024-04-07T17:09:27.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:09:26 smithi100 bash[76202]: audit 2024-04-07T17:09:25.781992+0000 mon.a (mon.0) 2920 : audit [INF] from='client.? 172.21.15.100:0/1817382777' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:09:28.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:09:27 smithi144 bash[66654]: audit 2024-04-07T17:09:26.767607+0000 mon.a (mon.0) 2921 : audit [INF] from='client.? 172.21.15.100:0/1817382777' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T17:09:28.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:09:27 smithi144 bash[66654]: cluster 2024-04-07T17:09:26.776112+0000 mon.a (mon.0) 2922 : cluster [DBG] osdmap e1575: 8 total, 8 up, 8 in 2024-04-07T17:09:28.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:09:27 smithi100 bash[74942]: audit 2024-04-07T17:09:26.767607+0000 mon.a (mon.0) 2921 : audit [INF] from='client.? 172.21.15.100:0/1817382777' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T17:09:28.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:09:27 smithi100 bash[74942]: cluster 2024-04-07T17:09:26.776112+0000 mon.a (mon.0) 2922 : cluster [DBG] osdmap e1575: 8 total, 8 up, 8 in 2024-04-07T17:09:28.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:09:27 smithi100 bash[76202]: audit 2024-04-07T17:09:26.767607+0000 mon.a (mon.0) 2921 : audit [INF] from='client.? 172.21.15.100:0/1817382777' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T17:09:28.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:09:27 smithi100 bash[76202]: cluster 2024-04-07T17:09:26.776112+0000 mon.a (mon.0) 2922 : cluster [DBG] osdmap e1575: 8 total, 8 up, 8 in 2024-04-07T17:09:28.703 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:09:28 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:09:28] "GET /metrics HTTP/1.1" 200 36558 "" "Prometheus/2.43.0" 2024-04-07T17:09:28.905 INFO:tasks.workunit.client.0.smithi100.stdout:image has size 2097152 and order 22 2024-04-07T17:09:28.905 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.CreateAndStat (4081 ms) 2024-04-07T17:09:28.905 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.CreateWithSameDataPool 2024-04-07T17:09:28.906 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:09:28.939 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.CreateWithSameDataPool (35 ms) 2024-04-07T17:09:28.940 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.CreateAndStatPP 2024-04-07T17:09:28.940 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:09:28.978 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.CreateAndStatPP (38 ms) 2024-04-07T17:09:28.978 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.GetId 2024-04-07T17:09:28.978 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:09:29.024 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.GetId (46 ms) 2024-04-07T17:09:29.128 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.GetIdPP 2024-04-07T17:09:29.128 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:09:29.128 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:09:28 smithi144 bash[66654]: cluster 2024-04-07T17:09:27.374882+0000 mgr.y (mgr.25175) 1623 : cluster [DBG] pgmap v2773: 137 pgs: 13 unknown, 124 active+clean; 4.6 MiB data, 3.4 GiB used, 712 GiB / 715 GiB avail 2024-04-07T17:09:29.129 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:09:28 smithi144 bash[66654]: cluster 2024-04-07T17:09:27.810440+0000 mon.a (mon.0) 2923 : cluster [DBG] osdmap e1576: 8 total, 8 up, 8 in 2024-04-07T17:09:29.129 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.GetIdPP (58 ms) 2024-04-07T17:09:29.129 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.GetBlockNamePrefix 2024-04-07T17:09:29.129 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:09:29.129 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.GetBlockNamePrefix (36 ms) 2024-04-07T17:09:29.129 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.GetBlockNamePrefixPP 2024-04-07T17:09:29.129 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:09:29.147 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.GetBlockNamePrefixPP (29 ms) 2024-04-07T17:09:29.147 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.TestGetCreateTimestamp 2024-04-07T17:09:29.147 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:09:29.175 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.TestGetCreateTimestamp (26 ms) 2024-04-07T17:09:29.175 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.GetCreateTimestampPP 2024-04-07T17:09:29.175 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:09:29.202 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.GetCreateTimestampPP (29 ms) 2024-04-07T17:09:29.202 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.OpenAio 2024-04-07T17:09:29.202 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:09:29.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:09:28 smithi100 bash[74942]: cluster 2024-04-07T17:09:27.374882+0000 mgr.y (mgr.25175) 1623 : cluster [DBG] pgmap v2773: 137 pgs: 13 unknown, 124 active+clean; 4.6 MiB data, 3.4 GiB used, 712 GiB / 715 GiB avail 2024-04-07T17:09:29.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:09:28 smithi100 bash[74942]: cluster 2024-04-07T17:09:27.810440+0000 mon.a (mon.0) 2923 : cluster [DBG] osdmap e1576: 8 total, 8 up, 8 in 2024-04-07T17:09:29.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:09:28 smithi100 bash[76202]: cluster 2024-04-07T17:09:27.374882+0000 mgr.y (mgr.25175) 1623 : cluster [DBG] pgmap v2773: 137 pgs: 13 unknown, 124 active+clean; 4.6 MiB data, 3.4 GiB used, 712 GiB / 715 GiB avail 2024-04-07T17:09:29.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:09:28 smithi100 bash[76202]: cluster 2024-04-07T17:09:27.810440+0000 mon.a (mon.0) 2923 : cluster [DBG] osdmap e1576: 8 total, 8 up, 8 in 2024-04-07T17:09:29.237 INFO:tasks.workunit.client.0.smithi100.stdout:image has size 2097152 and order 22 2024-04-07T17:09:29.238 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.OpenAio (36 ms) 2024-04-07T17:09:29.238 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.OpenAioFail 2024-04-07T17:09:29.246 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.OpenAioFail (6 ms) 2024-04-07T17:09:29.246 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.OpenAioPP 2024-04-07T17:09:29.246 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:09:29.281 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.OpenAioPP (37 ms) 2024-04-07T17:09:29.281 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.OpenAioFailPP 2024-04-07T17:09:29.285 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.OpenAioFailPP (4 ms) 2024-04-07T17:09:29.285 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.ResizeAndStat 2024-04-07T17:09:29.285 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:09:29.346 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.ResizeAndStat (61 ms) 2024-04-07T17:09:29.346 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.ResizeAndStatPP 2024-04-07T17:09:29.346 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:09:29.423 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.ResizeAndStatPP (78 ms) 2024-04-07T17:09:29.424 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.UpdateWatchAndResize 2024-04-07T17:09:29.424 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:09:29.514 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.UpdateWatchAndResize (90 ms) 2024-04-07T17:09:29.514 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.UpdateWatchAndResizePP 2024-04-07T17:09:29.514 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:09:29.592 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.UpdateWatchAndResizePP (78 ms) 2024-04-07T17:09:29.592 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.TestCreateLsDelete 2024-04-07T17:09:30.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:09:29 smithi144 bash[66654]: cluster 2024-04-07T17:09:28.822799+0000 mon.a (mon.0) 2924 : cluster [DBG] osdmap e1577: 8 total, 8 up, 8 in 2024-04-07T17:09:30.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:09:29 smithi144 bash[66654]: audit 2024-04-07T17:09:29.057371+0000 mon.a (mon.0) 2925 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config-key dump", "format": "json"}]: dispatch 2024-04-07T17:09:30.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:09:29 smithi144 bash[66654]: audit 2024-04-07T17:09:29.201211+0000 mon.a (mon.0) 2926 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:09:30.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:09:29 smithi144 bash[66654]: audit 2024-04-07T17:09:29.201698+0000 mon.a (mon.0) 2927 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:09:30.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:09:29 smithi144 bash[66654]: audit 2024-04-07T17:09:29.201916+0000 mon.a (mon.0) 2928 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:09:30.118 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:09:29 smithi144 bash[66654]: audit 2024-04-07T17:09:29.204657+0000 mon.a (mon.0) 2929 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:09:30.118 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:09:29 smithi144 bash[66654]: audit 2024-04-07T17:09:29.204915+0000 mon.a (mon.0) 2930 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:09:30.118 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:09:29 smithi144 bash[66654]: audit 2024-04-07T17:09:29.205195+0000 mon.a (mon.0) 2931 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:09:30.118 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:09:29 smithi144 bash[66654]: audit 2024-04-07T17:09:29.206779+0000 mon.a (mon.0) 2932 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:09:30.118 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:09:29 smithi144 bash[66654]: audit 2024-04-07T17:09:29.206974+0000 mon.a (mon.0) 2933 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:09:30.118 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:09:29 smithi144 bash[66654]: audit 2024-04-07T17:09:29.207244+0000 mon.a (mon.0) 2934 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:09:30.118 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:09:29 smithi144 bash[66654]: audit 2024-04-07T17:09:29.219721+0000 mon.a (mon.0) 2935 : audit [INF] from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T17:09:30.118 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:09:29 smithi144 bash[66654]: audit 2024-04-07T17:09:29.219804+0000 mon.a (mon.0) 2936 : audit [INF] from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T17:09:30.118 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:09:29 smithi144 bash[66654]: audit 2024-04-07T17:09:29.219999+0000 mon.a (mon.0) 2937 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:09:30.118 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:09:29 smithi144 bash[66654]: audit 2024-04-07T17:09:29.220239+0000 mon.a (mon.0) 2938 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:09:30.118 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:09:29 smithi144 bash[66654]: audit 2024-04-07T17:09:29.220467+0000 mon.a (mon.0) 2939 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:09:30.118 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:09:29 smithi144 bash[66654]: audit 2024-04-07T17:09:29.222517+0000 mon.a (mon.0) 2940 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:09:30.118 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:09:29 smithi144 bash[66654]: audit 2024-04-07T17:09:29.222633+0000 mon.b (mon.1) 13 : audit [INF] from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T17:09:30.118 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:09:29 smithi144 bash[66654]: audit 2024-04-07T17:09:29.222726+0000 mon.b (mon.1) 14 : audit [INF] from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T17:09:30.119 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:09:29 smithi144 bash[66654]: audit 2024-04-07T17:09:29.222802+0000 mon.a (mon.0) 2941 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:09:30.119 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:09:29 smithi144 bash[66654]: audit 2024-04-07T17:09:29.223154+0000 mon.a (mon.0) 2942 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:09:30.119 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:09:29 smithi144 bash[66654]: audit 2024-04-07T17:09:29.224624+0000 mon.c (mon.2) 67 : audit [INF] from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T17:09:30.119 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:09:29 smithi144 bash[66654]: audit 2024-04-07T17:09:29.224769+0000 mon.c (mon.2) 68 : audit [INF] from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T17:09:30.119 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:09:29 smithi144 bash[66654]: audit 2024-04-07T17:09:29.225031+0000 mon.a (mon.0) 2943 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:09:30.119 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:09:29 smithi144 bash[66654]: audit 2024-04-07T17:09:29.225706+0000 mon.a (mon.0) 2944 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:09:30.119 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:09:29 smithi144 bash[66654]: audit 2024-04-07T17:09:29.225979+0000 mon.a (mon.0) 2945 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:09:30.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:09:29 smithi100 bash[74942]: cluster 2024-04-07T17:09:28.822799+0000 mon.a (mon.0) 2924 : cluster [DBG] osdmap e1577: 8 total, 8 up, 8 in 2024-04-07T17:09:30.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:09:29 smithi100 bash[74942]: audit 2024-04-07T17:09:29.057371+0000 mon.a (mon.0) 2925 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config-key dump", "format": "json"}]: dispatch 2024-04-07T17:09:30.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:09:29 smithi100 bash[74942]: audit 2024-04-07T17:09:29.201211+0000 mon.a (mon.0) 2926 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:09:30.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:09:29 smithi100 bash[74942]: audit 2024-04-07T17:09:29.201698+0000 mon.a (mon.0) 2927 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:09:30.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:09:29 smithi100 bash[74942]: audit 2024-04-07T17:09:29.201916+0000 mon.a (mon.0) 2928 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:09:30.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:09:29 smithi100 bash[74942]: audit 2024-04-07T17:09:29.204657+0000 mon.a (mon.0) 2929 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:09:30.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:09:29 smithi100 bash[74942]: audit 2024-04-07T17:09:29.204915+0000 mon.a (mon.0) 2930 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:09:30.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:09:29 smithi100 bash[74942]: audit 2024-04-07T17:09:29.205195+0000 mon.a (mon.0) 2931 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:09:30.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:09:29 smithi100 bash[74942]: audit 2024-04-07T17:09:29.206779+0000 mon.a (mon.0) 2932 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:09:30.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:09:29 smithi100 bash[74942]: audit 2024-04-07T17:09:29.206974+0000 mon.a (mon.0) 2933 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:09:30.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:09:29 smithi100 bash[74942]: audit 2024-04-07T17:09:29.207244+0000 mon.a (mon.0) 2934 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:09:30.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:09:29 smithi100 bash[74942]: audit 2024-04-07T17:09:29.219721+0000 mon.a (mon.0) 2935 : audit [INF] from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T17:09:30.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:09:29 smithi100 bash[74942]: audit 2024-04-07T17:09:29.219804+0000 mon.a (mon.0) 2936 : audit [INF] from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T17:09:30.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:09:29 smithi100 bash[74942]: audit 2024-04-07T17:09:29.219999+0000 mon.a (mon.0) 2937 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:09:30.205 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:09:29 smithi100 bash[74942]: audit 2024-04-07T17:09:29.220239+0000 mon.a (mon.0) 2938 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:09:30.205 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:09:29 smithi100 bash[74942]: audit 2024-04-07T17:09:29.220467+0000 mon.a (mon.0) 2939 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:09:30.205 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:09:29 smithi100 bash[74942]: audit 2024-04-07T17:09:29.222517+0000 mon.a (mon.0) 2940 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:09:30.205 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:09:29 smithi100 bash[74942]: audit 2024-04-07T17:09:29.222633+0000 mon.b (mon.1) 13 : audit [INF] from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T17:09:30.205 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:09:29 smithi100 bash[74942]: audit 2024-04-07T17:09:29.222726+0000 mon.b (mon.1) 14 : audit [INF] from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T17:09:30.205 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:09:29 smithi100 bash[76202]: cluster 2024-04-07T17:09:28.822799+0000 mon.a (mon.0) 2924 : cluster [DBG] osdmap e1577: 8 total, 8 up, 8 in 2024-04-07T17:09:30.205 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:09:29 smithi100 bash[76202]: audit 2024-04-07T17:09:29.057371+0000 mon.a (mon.0) 2925 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config-key dump", "format": "json"}]: dispatch 2024-04-07T17:09:30.205 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:09:29 smithi100 bash[76202]: audit 2024-04-07T17:09:29.201211+0000 mon.a (mon.0) 2926 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:09:30.206 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:09:29 smithi100 bash[76202]: audit 2024-04-07T17:09:29.201698+0000 mon.a (mon.0) 2927 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:09:30.206 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:09:29 smithi100 bash[76202]: audit 2024-04-07T17:09:29.201916+0000 mon.a (mon.0) 2928 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:09:30.206 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:09:29 smithi100 bash[76202]: audit 2024-04-07T17:09:29.204657+0000 mon.a (mon.0) 2929 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:09:30.206 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:09:29 smithi100 bash[76202]: audit 2024-04-07T17:09:29.204915+0000 mon.a (mon.0) 2930 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:09:30.206 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:09:29 smithi100 bash[76202]: audit 2024-04-07T17:09:29.205195+0000 mon.a (mon.0) 2931 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:09:30.206 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:09:29 smithi100 bash[76202]: audit 2024-04-07T17:09:29.206779+0000 mon.a (mon.0) 2932 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:09:30.206 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:09:29 smithi100 bash[76202]: audit 2024-04-07T17:09:29.206974+0000 mon.a (mon.0) 2933 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:09:30.206 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:09:29 smithi100 bash[76202]: audit 2024-04-07T17:09:29.207244+0000 mon.a (mon.0) 2934 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:09:30.206 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:09:29 smithi100 bash[76202]: audit 2024-04-07T17:09:29.219721+0000 mon.a (mon.0) 2935 : audit [INF] from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T17:09:30.206 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:09:29 smithi100 bash[76202]: audit 2024-04-07T17:09:29.219804+0000 mon.a (mon.0) 2936 : audit [INF] from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T17:09:30.206 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:09:29 smithi100 bash[76202]: audit 2024-04-07T17:09:29.219999+0000 mon.a (mon.0) 2937 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:09:30.206 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:09:29 smithi100 bash[76202]: audit 2024-04-07T17:09:29.220239+0000 mon.a (mon.0) 2938 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:09:30.206 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:09:29 smithi100 bash[76202]: audit 2024-04-07T17:09:29.220467+0000 mon.a (mon.0) 2939 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:09:30.206 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:09:29 smithi100 bash[76202]: audit 2024-04-07T17:09:29.222517+0000 mon.a (mon.0) 2940 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:09:30.207 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:09:29 smithi100 bash[76202]: audit 2024-04-07T17:09:29.222633+0000 mon.b (mon.1) 13 : audit [INF] from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T17:09:30.207 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:09:29 smithi100 bash[76202]: audit 2024-04-07T17:09:29.222726+0000 mon.b (mon.1) 14 : audit [INF] from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T17:09:30.207 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:09:29 smithi100 bash[76202]: audit 2024-04-07T17:09:29.222802+0000 mon.a (mon.0) 2941 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:09:30.207 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:09:29 smithi100 bash[76202]: audit 2024-04-07T17:09:29.223154+0000 mon.a (mon.0) 2942 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:09:30.207 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:09:29 smithi100 bash[76202]: audit 2024-04-07T17:09:29.224624+0000 mon.c (mon.2) 67 : audit [INF] from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T17:09:30.207 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:09:29 smithi100 bash[76202]: audit 2024-04-07T17:09:29.224769+0000 mon.c (mon.2) 68 : audit [INF] from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T17:09:30.207 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:09:29 smithi100 bash[76202]: audit 2024-04-07T17:09:29.225031+0000 mon.a (mon.0) 2943 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:09:30.207 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:09:29 smithi100 bash[76202]: audit 2024-04-07T17:09:29.225706+0000 mon.a (mon.0) 2944 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:09:30.207 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:09:29 smithi100 bash[76202]: audit 2024-04-07T17:09:29.225979+0000 mon.a (mon.0) 2945 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:09:30.207 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:09:29 smithi100 bash[74942]: audit 2024-04-07T17:09:29.222802+0000 mon.a (mon.0) 2941 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:09:30.207 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:09:29 smithi100 bash[74942]: audit 2024-04-07T17:09:29.223154+0000 mon.a (mon.0) 2942 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:09:30.207 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:09:29 smithi100 bash[74942]: audit 2024-04-07T17:09:29.224624+0000 mon.c (mon.2) 67 : audit [INF] from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T17:09:30.207 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:09:29 smithi100 bash[74942]: audit 2024-04-07T17:09:29.224769+0000 mon.c (mon.2) 68 : audit [INF] from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T17:09:30.208 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:09:29 smithi100 bash[74942]: audit 2024-04-07T17:09:29.225031+0000 mon.a (mon.0) 2943 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:09:30.208 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:09:29 smithi100 bash[74942]: audit 2024-04-07T17:09:29.225706+0000 mon.a (mon.0) 2944 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:09:30.208 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:09:29 smithi100 bash[74942]: audit 2024-04-07T17:09:29.225979+0000 mon.a (mon.0) 2945 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:09:30.888 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:09:31.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:09:30 smithi144 bash[66654]: cluster 2024-04-07T17:09:29.375517+0000 mgr.y (mgr.25175) 1624 : cluster [DBG] pgmap v2776: 137 pgs: 2 unknown, 135 active+clean; 4.6 MiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 11 KiB/s rd, 2.7 KiB/s wr, 15 op/s 2024-04-07T17:09:31.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:09:30 smithi144 bash[66654]: cluster 2024-04-07T17:09:29.834273+0000 mon.a (mon.0) 2946 : cluster [DBG] osdmap e1578: 8 total, 8 up, 8 in 2024-04-07T17:09:31.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:09:30 smithi144 bash[66654]: audit 2024-04-07T17:09:29.840666+0000 mon.c (mon.2) 69 : audit [INF] from='client.? 172.21.15.100:0/1893978270' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:09:31.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:09:30 smithi144 bash[66654]: audit 2024-04-07T17:09:29.841785+0000 mon.a (mon.0) 2947 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:09:31.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:09:30 smithi100 bash[74942]: cluster 2024-04-07T17:09:29.375517+0000 mgr.y (mgr.25175) 1624 : cluster [DBG] pgmap v2776: 137 pgs: 2 unknown, 135 active+clean; 4.6 MiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 11 KiB/s rd, 2.7 KiB/s wr, 15 op/s 2024-04-07T17:09:31.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:09:30 smithi100 bash[74942]: cluster 2024-04-07T17:09:29.834273+0000 mon.a (mon.0) 2946 : cluster [DBG] osdmap e1578: 8 total, 8 up, 8 in 2024-04-07T17:09:31.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:09:30 smithi100 bash[74942]: audit 2024-04-07T17:09:29.840666+0000 mon.c (mon.2) 69 : audit [INF] from='client.? 172.21.15.100:0/1893978270' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:09:31.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:09:30 smithi100 bash[74942]: audit 2024-04-07T17:09:29.841785+0000 mon.a (mon.0) 2947 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:09:31.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:09:30 smithi100 bash[76202]: cluster 2024-04-07T17:09:29.375517+0000 mgr.y (mgr.25175) 1624 : cluster [DBG] pgmap v2776: 137 pgs: 2 unknown, 135 active+clean; 4.6 MiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 11 KiB/s rd, 2.7 KiB/s wr, 15 op/s 2024-04-07T17:09:31.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:09:30 smithi100 bash[76202]: cluster 2024-04-07T17:09:29.834273+0000 mon.a (mon.0) 2946 : cluster [DBG] osdmap e1578: 8 total, 8 up, 8 in 2024-04-07T17:09:31.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:09:30 smithi100 bash[76202]: audit 2024-04-07T17:09:29.840666+0000 mon.c (mon.2) 69 : audit [INF] from='client.? 172.21.15.100:0/1893978270' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:09:31.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:09:30 smithi100 bash[76202]: audit 2024-04-07T17:09:29.841785+0000 mon.a (mon.0) 2947 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:09:32.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:09:31 smithi144 bash[66654]: audit 2024-04-07T17:09:30.834852+0000 mon.a (mon.0) 2948 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T17:09:32.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:09:31 smithi144 bash[66654]: cluster 2024-04-07T17:09:30.842837+0000 mon.a (mon.0) 2949 : cluster [DBG] osdmap e1579: 8 total, 8 up, 8 in 2024-04-07T17:09:32.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:09:31 smithi144 bash[66654]: audit 2024-04-07T17:09:31.813047+0000 mon.a (mon.0) 2950 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:09:32.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:09:31 smithi100 bash[74942]: audit 2024-04-07T17:09:30.834852+0000 mon.a (mon.0) 2948 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T17:09:32.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:09:31 smithi100 bash[74942]: cluster 2024-04-07T17:09:30.842837+0000 mon.a (mon.0) 2949 : cluster [DBG] osdmap e1579: 8 total, 8 up, 8 in 2024-04-07T17:09:32.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:09:31 smithi100 bash[74942]: audit 2024-04-07T17:09:31.813047+0000 mon.a (mon.0) 2950 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:09:32.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:09:31 smithi100 bash[76202]: audit 2024-04-07T17:09:30.834852+0000 mon.a (mon.0) 2948 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T17:09:32.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:09:31 smithi100 bash[76202]: cluster 2024-04-07T17:09:30.842837+0000 mon.a (mon.0) 2949 : cluster [DBG] osdmap e1579: 8 total, 8 up, 8 in 2024-04-07T17:09:32.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:09:31 smithi100 bash[76202]: audit 2024-04-07T17:09:31.813047+0000 mon.a (mon.0) 2950 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:09:32.616 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:09:32 smithi144 bash[95233]: ts=2024-04-07T17:09:32.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:09:32.926 INFO:tasks.workunit.client.0.smithi100.stdout:image: image18 2024-04-07T17:09:32.926 INFO:tasks.workunit.client.0.smithi100.stdout:expected = image18 2024-04-07T17:09:32.926 INFO:tasks.workunit.client.0.smithi100.stdout:found image18 2024-04-07T17:09:32.926 INFO:tasks.workunit.client.0.smithi100.stdout:erased image18 2024-04-07T17:09:32.926 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:09:33.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:09:32 smithi144 bash[66654]: cluster 2024-04-07T17:09:31.376199+0000 mgr.y (mgr.25175) 1625 : cluster [DBG] pgmap v2779: 169 pgs: 1 creating+peering, 31 unknown, 137 active+clean; 4.6 MiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 28 KiB/s rd, 4.5 KiB/s wr, 39 op/s 2024-04-07T17:09:33.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:09:32 smithi144 bash[66654]: cluster 2024-04-07T17:09:31.852763+0000 mon.a (mon.0) 2951 : cluster [DBG] osdmap e1580: 8 total, 8 up, 8 in 2024-04-07T17:09:33.142 INFO:tasks.workunit.client.0.smithi100.stdout:image: image18 2024-04-07T17:09:33.142 INFO:tasks.workunit.client.0.smithi100.stdout:image: image19 2024-04-07T17:09:33.142 INFO:tasks.workunit.client.0.smithi100.stdout:expected = image18 2024-04-07T17:09:33.142 INFO:tasks.workunit.client.0.smithi100.stdout:found image18 2024-04-07T17:09:33.142 INFO:tasks.workunit.client.0.smithi100.stdout:erased image18 2024-04-07T17:09:33.142 INFO:tasks.workunit.client.0.smithi100.stdout:expected = image19 2024-04-07T17:09:33.142 INFO:tasks.workunit.client.0.smithi100.stdout:found image19 2024-04-07T17:09:33.142 INFO:tasks.workunit.client.0.smithi100.stdout:erased image19 2024-04-07T17:09:33.143 INFO:tasks.workunit.client.0.smithi100.stdout:image: image19 2024-04-07T17:09:33.143 INFO:tasks.workunit.client.0.smithi100.stdout:expected = image19 2024-04-07T17:09:33.143 INFO:tasks.workunit.client.0.smithi100.stdout:found image19 2024-04-07T17:09:33.143 INFO:tasks.workunit.client.0.smithi100.stdout:erased image19 2024-04-07T17:09:33.143 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.TestCreateLsDelete (3550 ms) 2024-04-07T17:09:33.143 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.TestCreateLsDeletePP 2024-04-07T17:09:33.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:09:32 smithi100 bash[74942]: cluster 2024-04-07T17:09:31.376199+0000 mgr.y (mgr.25175) 1625 : cluster [DBG] pgmap v2779: 169 pgs: 1 creating+peering, 31 unknown, 137 active+clean; 4.6 MiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 28 KiB/s rd, 4.5 KiB/s wr, 39 op/s 2024-04-07T17:09:33.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:09:32 smithi100 bash[74942]: cluster 2024-04-07T17:09:31.852763+0000 mon.a (mon.0) 2951 : cluster [DBG] osdmap e1580: 8 total, 8 up, 8 in 2024-04-07T17:09:33.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:09:32 smithi100 bash[76202]: cluster 2024-04-07T17:09:31.376199+0000 mgr.y (mgr.25175) 1625 : cluster [DBG] pgmap v2779: 169 pgs: 1 creating+peering, 31 unknown, 137 active+clean; 4.6 MiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 28 KiB/s rd, 4.5 KiB/s wr, 39 op/s 2024-04-07T17:09:33.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:09:32 smithi100 bash[76202]: cluster 2024-04-07T17:09:31.852763+0000 mon.a (mon.0) 2951 : cluster [DBG] osdmap e1580: 8 total, 8 up, 8 in 2024-04-07T17:09:34.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:09:33 smithi100 bash[74942]: cluster 2024-04-07T17:09:32.872684+0000 mon.a (mon.0) 2952 : cluster [DBG] osdmap e1581: 8 total, 8 up, 8 in 2024-04-07T17:09:34.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:09:33 smithi100 bash[76202]: cluster 2024-04-07T17:09:32.872684+0000 mon.a (mon.0) 2952 : cluster [DBG] osdmap e1581: 8 total, 8 up, 8 in 2024-04-07T17:09:34.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:09:33 smithi144 bash[66654]: cluster 2024-04-07T17:09:32.872684+0000 mon.a (mon.0) 2952 : cluster [DBG] osdmap e1581: 8 total, 8 up, 8 in 2024-04-07T17:09:34.897 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:09:35.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:09:34 smithi100 bash[74942]: cluster 2024-04-07T17:09:33.377163+0000 mgr.y (mgr.25175) 1626 : cluster [DBG] pgmap v2782: 169 pgs: 1 creating+peering, 6 unknown, 162 active+clean; 4.6 MiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 143 KiB/s rd, 15 KiB/s wr, 201 op/s 2024-04-07T17:09:35.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:09:34 smithi100 bash[74942]: cluster 2024-04-07T17:09:33.892131+0000 mon.a (mon.0) 2953 : cluster [DBG] osdmap e1582: 8 total, 8 up, 8 in 2024-04-07T17:09:35.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:09:34 smithi100 bash[74942]: audit 2024-04-07T17:09:33.894473+0000 mon.c (mon.2) 70 : audit [INF] from='client.? 172.21.15.100:0/1840083403' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:09:35.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:09:34 smithi100 bash[74942]: audit 2024-04-07T17:09:33.899860+0000 mon.a (mon.0) 2954 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:09:35.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:09:34 smithi100 bash[76202]: cluster 2024-04-07T17:09:33.377163+0000 mgr.y (mgr.25175) 1626 : cluster [DBG] pgmap v2782: 169 pgs: 1 creating+peering, 6 unknown, 162 active+clean; 4.6 MiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 143 KiB/s rd, 15 KiB/s wr, 201 op/s 2024-04-07T17:09:35.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:09:34 smithi100 bash[76202]: cluster 2024-04-07T17:09:33.892131+0000 mon.a (mon.0) 2953 : cluster [DBG] osdmap e1582: 8 total, 8 up, 8 in 2024-04-07T17:09:35.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:09:34 smithi100 bash[76202]: audit 2024-04-07T17:09:33.894473+0000 mon.c (mon.2) 70 : audit [INF] from='client.? 172.21.15.100:0/1840083403' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:09:35.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:09:34 smithi100 bash[76202]: audit 2024-04-07T17:09:33.899860+0000 mon.a (mon.0) 2954 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:09:35.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:09:34 smithi144 bash[66654]: cluster 2024-04-07T17:09:33.377163+0000 mgr.y (mgr.25175) 1626 : cluster [DBG] pgmap v2782: 169 pgs: 1 creating+peering, 6 unknown, 162 active+clean; 4.6 MiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 143 KiB/s rd, 15 KiB/s wr, 201 op/s 2024-04-07T17:09:35.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:09:34 smithi144 bash[66654]: cluster 2024-04-07T17:09:33.892131+0000 mon.a (mon.0) 2953 : cluster [DBG] osdmap e1582: 8 total, 8 up, 8 in 2024-04-07T17:09:35.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:09:34 smithi144 bash[66654]: audit 2024-04-07T17:09:33.894473+0000 mon.c (mon.2) 70 : audit [INF] from='client.? 172.21.15.100:0/1840083403' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:09:35.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:09:34 smithi144 bash[66654]: audit 2024-04-07T17:09:33.899860+0000 mon.a (mon.0) 2954 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:09:36.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:09:35 smithi100 bash[76202]: audit 2024-04-07T17:09:34.880055+0000 mon.a (mon.0) 2955 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T17:09:36.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:09:35 smithi100 bash[76202]: cluster 2024-04-07T17:09:34.888522+0000 mon.a (mon.0) 2956 : cluster [DBG] osdmap e1583: 8 total, 8 up, 8 in 2024-04-07T17:09:36.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:09:35 smithi100 bash[76202]: cluster 2024-04-07T17:09:35.377976+0000 mgr.y (mgr.25175) 1627 : cluster [DBG] pgmap v2785: 201 pgs: 9 creating+peering, 23 unknown, 169 active+clean; 4.6 MiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 156 KiB/s rd, 16 KiB/s wr, 218 op/s 2024-04-07T17:09:36.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:09:35 smithi100 bash[74942]: audit 2024-04-07T17:09:34.880055+0000 mon.a (mon.0) 2955 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T17:09:36.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:09:35 smithi100 bash[74942]: cluster 2024-04-07T17:09:34.888522+0000 mon.a (mon.0) 2956 : cluster [DBG] osdmap e1583: 8 total, 8 up, 8 in 2024-04-07T17:09:36.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:09:35 smithi100 bash[74942]: cluster 2024-04-07T17:09:35.377976+0000 mgr.y (mgr.25175) 1627 : cluster [DBG] pgmap v2785: 201 pgs: 9 creating+peering, 23 unknown, 169 active+clean; 4.6 MiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 156 KiB/s rd, 16 KiB/s wr, 218 op/s 2024-04-07T17:09:36.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:09:35 smithi144 bash[66654]: audit 2024-04-07T17:09:34.880055+0000 mon.a (mon.0) 2955 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T17:09:36.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:09:35 smithi144 bash[66654]: cluster 2024-04-07T17:09:34.888522+0000 mon.a (mon.0) 2956 : cluster [DBG] osdmap e1583: 8 total, 8 up, 8 in 2024-04-07T17:09:36.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:09:35 smithi144 bash[66654]: cluster 2024-04-07T17:09:35.377976+0000 mgr.y (mgr.25175) 1627 : cluster [DBG] pgmap v2785: 201 pgs: 9 creating+peering, 23 unknown, 169 active+clean; 4.6 MiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 156 KiB/s rd, 16 KiB/s wr, 218 op/s 2024-04-07T17:09:36.992 INFO:tasks.workunit.client.0.smithi100.stdout:num images is: 1 2024-04-07T17:09:36.992 INFO:tasks.workunit.client.0.smithi100.stdout:expected: 1 2024-04-07T17:09:36.992 INFO:tasks.workunit.client.0.smithi100.stdout:image: image20 2024-04-07T17:09:36.992 INFO:tasks.workunit.client.0.smithi100.stdout:expected = image20 2024-04-07T17:09:36.992 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:09:37.017 INFO:tasks.workunit.client.0.smithi100.stdout:num images is: 2 2024-04-07T17:09:37.018 INFO:tasks.workunit.client.0.smithi100.stdout:expected: 2 2024-04-07T17:09:37.018 INFO:tasks.workunit.client.0.smithi100.stdout:image: image20 2024-04-07T17:09:37.018 INFO:tasks.workunit.client.0.smithi100.stdout:image: image21 2024-04-07T17:09:37.018 INFO:tasks.workunit.client.0.smithi100.stdout:expected = image20 2024-04-07T17:09:37.018 INFO:tasks.workunit.client.0.smithi100.stdout:expected = image21 2024-04-07T17:09:37.178 INFO:tasks.workunit.client.0.smithi100.stdout:num images is: 1 2024-04-07T17:09:37.178 INFO:tasks.workunit.client.0.smithi100.stdout:expected: 1 2024-04-07T17:09:37.178 INFO:tasks.workunit.client.0.smithi100.stdout:image: image21 2024-04-07T17:09:37.178 INFO:tasks.workunit.client.0.smithi100.stdout:expected = image21 2024-04-07T17:09:37.178 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.TestCreateLsDeletePP (4036 ms) 2024-04-07T17:09:37.179 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.TestCopy 2024-04-07T17:09:37.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:09:36 smithi100 bash[74942]: cluster 2024-04-07T17:09:35.917882+0000 mon.a (mon.0) 2957 : cluster [DBG] osdmap e1584: 8 total, 8 up, 8 in 2024-04-07T17:09:37.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:09:36 smithi100 bash[76202]: cluster 2024-04-07T17:09:35.917882+0000 mon.a (mon.0) 2957 : cluster [DBG] osdmap e1584: 8 total, 8 up, 8 in 2024-04-07T17:09:37.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:09:36 smithi144 bash[66654]: cluster 2024-04-07T17:09:35.917882+0000 mon.a (mon.0) 2957 : cluster [DBG] osdmap e1584: 8 total, 8 up, 8 in 2024-04-07T17:09:38.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:09:37 smithi100 bash[74942]: cluster 2024-04-07T17:09:36.932557+0000 mon.a (mon.0) 2958 : cluster [DBG] osdmap e1585: 8 total, 8 up, 8 in 2024-04-07T17:09:38.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:09:37 smithi100 bash[74942]: cluster 2024-04-07T17:09:37.379239+0000 mgr.y (mgr.25175) 1628 : cluster [DBG] pgmap v2788: 201 pgs: 9 creating+peering, 9 unknown, 183 active+clean; 4.6 MiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 55 KiB/s rd, 6.0 KiB/s wr, 76 op/s 2024-04-07T17:09:38.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:09:37 smithi100 bash[74942]: cluster 2024-04-07T17:09:37.713218+0000 mon.a (mon.0) 2959 : cluster [DBG] osdmap e1586: 8 total, 8 up, 8 in 2024-04-07T17:09:38.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:09:37 smithi100 bash[74942]: audit 2024-04-07T17:09:37.719279+0000 mon.c (mon.2) 71 : audit [INF] from='client.? 172.21.15.100:0/3236497731' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:09:38.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:09:37 smithi100 bash[74942]: audit 2024-04-07T17:09:37.720424+0000 mon.a (mon.0) 2960 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:09:38.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:09:37 smithi100 bash[76202]: cluster 2024-04-07T17:09:36.932557+0000 mon.a (mon.0) 2958 : cluster [DBG] osdmap e1585: 8 total, 8 up, 8 in 2024-04-07T17:09:38.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:09:37 smithi100 bash[76202]: cluster 2024-04-07T17:09:37.379239+0000 mgr.y (mgr.25175) 1628 : cluster [DBG] pgmap v2788: 201 pgs: 9 creating+peering, 9 unknown, 183 active+clean; 4.6 MiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 55 KiB/s rd, 6.0 KiB/s wr, 76 op/s 2024-04-07T17:09:38.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:09:37 smithi100 bash[76202]: cluster 2024-04-07T17:09:37.713218+0000 mon.a (mon.0) 2959 : cluster [DBG] osdmap e1586: 8 total, 8 up, 8 in 2024-04-07T17:09:38.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:09:37 smithi100 bash[76202]: audit 2024-04-07T17:09:37.719279+0000 mon.c (mon.2) 71 : audit [INF] from='client.? 172.21.15.100:0/3236497731' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:09:38.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:09:37 smithi100 bash[76202]: audit 2024-04-07T17:09:37.720424+0000 mon.a (mon.0) 2960 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:09:38.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:09:37 smithi144 bash[66654]: cluster 2024-04-07T17:09:36.932557+0000 mon.a (mon.0) 2958 : cluster [DBG] osdmap e1585: 8 total, 8 up, 8 in 2024-04-07T17:09:38.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:09:37 smithi144 bash[66654]: cluster 2024-04-07T17:09:37.379239+0000 mgr.y (mgr.25175) 1628 : cluster [DBG] pgmap v2788: 201 pgs: 9 creating+peering, 9 unknown, 183 active+clean; 4.6 MiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 55 KiB/s rd, 6.0 KiB/s wr, 76 op/s 2024-04-07T17:09:38.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:09:37 smithi144 bash[66654]: cluster 2024-04-07T17:09:37.713218+0000 mon.a (mon.0) 2959 : cluster [DBG] osdmap e1586: 8 total, 8 up, 8 in 2024-04-07T17:09:38.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:09:37 smithi144 bash[66654]: audit 2024-04-07T17:09:37.719279+0000 mon.c (mon.2) 71 : audit [INF] from='client.? 172.21.15.100:0/3236497731' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:09:38.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:09:37 smithi144 bash[66654]: audit 2024-04-07T17:09:37.720424+0000 mon.a (mon.0) 2960 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:09:38.703 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:09:38 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:09:38] "GET /metrics HTTP/1.1" 200 38898 "" "Prometheus/2.43.0" 2024-04-07T17:09:38.723 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:09:40.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:09:39 smithi144 bash[66654]: audit 2024-04-07T17:09:38.705803+0000 mon.a (mon.0) 2961 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T17:09:40.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:09:39 smithi144 bash[66654]: cluster 2024-04-07T17:09:38.717449+0000 mon.a (mon.0) 2962 : cluster [DBG] osdmap e1587: 8 total, 8 up, 8 in 2024-04-07T17:09:40.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:09:39 smithi100 bash[74942]: audit 2024-04-07T17:09:38.705803+0000 mon.a (mon.0) 2961 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T17:09:40.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:09:39 smithi100 bash[74942]: cluster 2024-04-07T17:09:38.717449+0000 mon.a (mon.0) 2962 : cluster [DBG] osdmap e1587: 8 total, 8 up, 8 in 2024-04-07T17:09:40.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:09:39 smithi100 bash[76202]: audit 2024-04-07T17:09:38.705803+0000 mon.a (mon.0) 2961 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T17:09:40.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:09:39 smithi100 bash[76202]: cluster 2024-04-07T17:09:38.717449+0000 mon.a (mon.0) 2962 : cluster [DBG] osdmap e1587: 8 total, 8 up, 8 in 2024-04-07T17:09:41.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:09:40 smithi144 bash[66654]: cluster 2024-04-07T17:09:39.380455+0000 mgr.y (mgr.25175) 1629 : cluster [DBG] pgmap v2791: 233 pgs: 13 creating+peering, 19 unknown, 201 active+clean; 4.6 MiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 29 KiB/s rd, 4.5 KiB/s wr, 41 op/s 2024-04-07T17:09:41.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:09:40 smithi144 bash[66654]: cluster 2024-04-07T17:09:39.715982+0000 mon.a (mon.0) 2963 : cluster [DBG] osdmap e1588: 8 total, 8 up, 8 in 2024-04-07T17:09:41.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:09:40 smithi100 bash[74942]: cluster 2024-04-07T17:09:39.380455+0000 mgr.y (mgr.25175) 1629 : cluster [DBG] pgmap v2791: 233 pgs: 13 creating+peering, 19 unknown, 201 active+clean; 4.6 MiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 29 KiB/s rd, 4.5 KiB/s wr, 41 op/s 2024-04-07T17:09:41.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:09:40 smithi100 bash[74942]: cluster 2024-04-07T17:09:39.715982+0000 mon.a (mon.0) 2963 : cluster [DBG] osdmap e1588: 8 total, 8 up, 8 in 2024-04-07T17:09:41.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:09:40 smithi100 bash[76202]: cluster 2024-04-07T17:09:39.380455+0000 mgr.y (mgr.25175) 1629 : cluster [DBG] pgmap v2791: 233 pgs: 13 creating+peering, 19 unknown, 201 active+clean; 4.6 MiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 29 KiB/s rd, 4.5 KiB/s wr, 41 op/s 2024-04-07T17:09:41.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:09:40 smithi100 bash[76202]: cluster 2024-04-07T17:09:39.715982+0000 mon.a (mon.0) 2963 : cluster [DBG] osdmap e1588: 8 total, 8 up, 8 in 2024-04-07T17:09:41.810 INFO:tasks.workunit.client.0.smithi100.stdout:image: image22 2024-04-07T17:09:41.810 INFO:tasks.workunit.client.0.smithi100.stdout:expected = image22 2024-04-07T17:09:41.811 INFO:tasks.workunit.client.0.smithi100.stdout:found image22 2024-04-07T17:09:41.811 INFO:tasks.workunit.client.0.smithi100.stdout:erased image22 2024-04-07T17:09:41.811 INFO:tasks.workunit.client.0.smithi100.stdout:image: image22 2024-04-07T17:09:41.811 INFO:tasks.workunit.client.0.smithi100.stdout:image: image23 2024-04-07T17:09:41.811 INFO:tasks.workunit.client.0.smithi100.stdout:expected = image22 2024-04-07T17:09:41.811 INFO:tasks.workunit.client.0.smithi100.stdout:found image22 2024-04-07T17:09:41.811 INFO:tasks.workunit.client.0.smithi100.stdout:erased image22 2024-04-07T17:09:41.811 INFO:tasks.workunit.client.0.smithi100.stdout:expected = image23 2024-04-07T17:09:41.811 INFO:tasks.workunit.client.0.smithi100.stdout:found image23 2024-04-07T17:09:41.811 INFO:tasks.workunit.client.0.smithi100.stdout:erased image23 2024-04-07T17:09:41.811 INFO:tasks.workunit.client.0.smithi100.stdout:100.00% done 2024-04-07T17:09:41.811 INFO:tasks.workunit.client.0.smithi100.stdout:image: image22 2024-04-07T17:09:41.811 INFO:tasks.workunit.client.0.smithi100.stdout:image: image23 2024-04-07T17:09:41.811 INFO:tasks.workunit.client.0.smithi100.stdout:image: image24 2024-04-07T17:09:41.811 INFO:tasks.workunit.client.0.smithi100.stdout:expected = image22 2024-04-07T17:09:41.812 INFO:tasks.workunit.client.0.smithi100.stdout:found image22 2024-04-07T17:09:41.812 INFO:tasks.workunit.client.0.smithi100.stdout:erased image22 2024-04-07T17:09:41.812 INFO:tasks.workunit.client.0.smithi100.stdout:expected = image23 2024-04-07T17:09:41.812 INFO:tasks.workunit.client.0.smithi100.stdout:found image23 2024-04-07T17:09:41.812 INFO:tasks.workunit.client.0.smithi100.stdout:erased image23 2024-04-07T17:09:41.812 INFO:tasks.workunit.client.0.smithi100.stdout:expected = image24 2024-04-07T17:09:41.812 INFO:tasks.workunit.client.0.smithi100.stdout:found image24 2024-04-07T17:09:41.812 INFO:tasks.workunit.client.0.smithi100.stdout:erased image24 2024-04-07T17:09:41.812 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.TestCopy (4632 ms) 2024-04-07T17:09:41.812 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.TestCopyPP 2024-04-07T17:09:42.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:09:41 smithi144 bash[66654]: cluster 2024-04-07T17:09:40.743738+0000 mon.a (mon.0) 2964 : cluster [DBG] osdmap e1589: 8 total, 8 up, 8 in 2024-04-07T17:09:42.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:09:41 smithi100 bash[74942]: cluster 2024-04-07T17:09:40.743738+0000 mon.a (mon.0) 2964 : cluster [DBG] osdmap e1589: 8 total, 8 up, 8 in 2024-04-07T17:09:42.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:09:41 smithi100 bash[76202]: cluster 2024-04-07T17:09:40.743738+0000 mon.a (mon.0) 2964 : cluster [DBG] osdmap e1589: 8 total, 8 up, 8 in 2024-04-07T17:09:42.616 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:09:42 smithi144 bash[95233]: ts=2024-04-07T17:09:42.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:09:43.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:09:42 smithi144 bash[66654]: cluster 2024-04-07T17:09:41.381511+0000 mgr.y (mgr.25175) 1630 : cluster [DBG] pgmap v2794: 233 pgs: 7 creating+peering, 15 unknown, 211 active+clean; 4.6 MiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 5.5 KiB/s rd, 2.2 KiB/s wr, 12 op/s 2024-04-07T17:09:43.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:09:42 smithi144 bash[66654]: cluster 2024-04-07T17:09:42.718854+0000 mon.a (mon.0) 2965 : cluster [DBG] osdmap e1590: 8 total, 8 up, 8 in 2024-04-07T17:09:43.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:09:42 smithi144 bash[66654]: audit 2024-04-07T17:09:42.721695+0000 mon.a (mon.0) 2966 : audit [INF] from='client.? 172.21.15.100:0/3443176463' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:09:43.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:09:42 smithi100 bash[74942]: cluster 2024-04-07T17:09:41.381511+0000 mgr.y (mgr.25175) 1630 : cluster [DBG] pgmap v2794: 233 pgs: 7 creating+peering, 15 unknown, 211 active+clean; 4.6 MiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 5.5 KiB/s rd, 2.2 KiB/s wr, 12 op/s 2024-04-07T17:09:43.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:09:42 smithi100 bash[74942]: cluster 2024-04-07T17:09:42.718854+0000 mon.a (mon.0) 2965 : cluster [DBG] osdmap e1590: 8 total, 8 up, 8 in 2024-04-07T17:09:43.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:09:42 smithi100 bash[74942]: audit 2024-04-07T17:09:42.721695+0000 mon.a (mon.0) 2966 : audit [INF] from='client.? 172.21.15.100:0/3443176463' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:09:43.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:09:42 smithi100 bash[76202]: cluster 2024-04-07T17:09:41.381511+0000 mgr.y (mgr.25175) 1630 : cluster [DBG] pgmap v2794: 233 pgs: 7 creating+peering, 15 unknown, 211 active+clean; 4.6 MiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 5.5 KiB/s rd, 2.2 KiB/s wr, 12 op/s 2024-04-07T17:09:43.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:09:42 smithi100 bash[76202]: cluster 2024-04-07T17:09:42.718854+0000 mon.a (mon.0) 2965 : cluster [DBG] osdmap e1590: 8 total, 8 up, 8 in 2024-04-07T17:09:43.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:09:42 smithi100 bash[76202]: audit 2024-04-07T17:09:42.721695+0000 mon.a (mon.0) 2966 : audit [INF] from='client.? 172.21.15.100:0/3443176463' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:09:43.734 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:09:45.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:09:44 smithi144 bash[66654]: cluster 2024-04-07T17:09:43.382788+0000 mgr.y (mgr.25175) 1631 : cluster [DBG] pgmap v2796: 265 pgs: 3 creating+peering, 29 unknown, 233 active+clean; 4.6 MiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 271 KiB/s rd, 19 KiB/s wr, 325 op/s 2024-04-07T17:09:45.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:09:44 smithi144 bash[66654]: audit 2024-04-07T17:09:43.709659+0000 mon.a (mon.0) 2967 : audit [INF] from='client.? 172.21.15.100:0/3443176463' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T17:09:45.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:09:44 smithi144 bash[66654]: cluster 2024-04-07T17:09:43.718322+0000 mon.a (mon.0) 2968 : cluster [DBG] osdmap e1591: 8 total, 8 up, 8 in 2024-04-07T17:09:45.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:09:44 smithi144 bash[66654]: cluster 2024-04-07T17:09:43.747581+0000 mon.a (mon.0) 2969 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T17:09:45.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:09:44 smithi100 bash[74942]: cluster 2024-04-07T17:09:43.382788+0000 mgr.y (mgr.25175) 1631 : cluster [DBG] pgmap v2796: 265 pgs: 3 creating+peering, 29 unknown, 233 active+clean; 4.6 MiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 271 KiB/s rd, 19 KiB/s wr, 325 op/s 2024-04-07T17:09:45.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:09:44 smithi100 bash[74942]: audit 2024-04-07T17:09:43.709659+0000 mon.a (mon.0) 2967 : audit [INF] from='client.? 172.21.15.100:0/3443176463' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T17:09:45.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:09:44 smithi100 bash[74942]: cluster 2024-04-07T17:09:43.718322+0000 mon.a (mon.0) 2968 : cluster [DBG] osdmap e1591: 8 total, 8 up, 8 in 2024-04-07T17:09:45.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:09:44 smithi100 bash[74942]: cluster 2024-04-07T17:09:43.747581+0000 mon.a (mon.0) 2969 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T17:09:45.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:09:44 smithi100 bash[76202]: cluster 2024-04-07T17:09:43.382788+0000 mgr.y (mgr.25175) 1631 : cluster [DBG] pgmap v2796: 265 pgs: 3 creating+peering, 29 unknown, 233 active+clean; 4.6 MiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 271 KiB/s rd, 19 KiB/s wr, 325 op/s 2024-04-07T17:09:45.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:09:44 smithi100 bash[76202]: audit 2024-04-07T17:09:43.709659+0000 mon.a (mon.0) 2967 : audit [INF] from='client.? 172.21.15.100:0/3443176463' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T17:09:45.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:09:44 smithi100 bash[76202]: cluster 2024-04-07T17:09:43.718322+0000 mon.a (mon.0) 2968 : cluster [DBG] osdmap e1591: 8 total, 8 up, 8 in 2024-04-07T17:09:45.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:09:44 smithi100 bash[76202]: cluster 2024-04-07T17:09:43.747581+0000 mon.a (mon.0) 2969 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T17:09:46.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:09:45 smithi144 bash[66654]: cluster 2024-04-07T17:09:44.729335+0000 mon.a (mon.0) 2970 : cluster [DBG] osdmap e1592: 8 total, 8 up, 8 in 2024-04-07T17:09:46.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:09:45 smithi100 bash[74942]: cluster 2024-04-07T17:09:44.729335+0000 mon.a (mon.0) 2970 : cluster [DBG] osdmap e1592: 8 total, 8 up, 8 in 2024-04-07T17:09:46.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:09:45 smithi100 bash[76202]: cluster 2024-04-07T17:09:44.729335+0000 mon.a (mon.0) 2970 : cluster [DBG] osdmap e1592: 8 total, 8 up, 8 in 2024-04-07T17:09:46.487 INFO:tasks.workunit.client.0.smithi100.stdout:num images is: 1 2024-04-07T17:09:46.488 INFO:tasks.workunit.client.0.smithi100.stdout:expected: 1 2024-04-07T17:09:46.488 INFO:tasks.workunit.client.0.smithi100.stdout:image: image25 2024-04-07T17:09:46.488 INFO:tasks.workunit.client.0.smithi100.stdout:expected = image25 2024-04-07T17:09:46.543 INFO:tasks.workunit.client.0.smithi100.stdout:num images is: 2 2024-04-07T17:09:46.543 INFO:tasks.workunit.client.0.smithi100.stdout:expected: 2 2024-04-07T17:09:46.543 INFO:tasks.workunit.client.0.smithi100.stdout:image: image25 2024-04-07T17:09:46.543 INFO:tasks.workunit.client.0.smithi100.stdout:image: image26 2024-04-07T17:09:46.543 INFO:tasks.workunit.client.0.smithi100.stdout:expected = image25 2024-04-07T17:09:46.543 INFO:tasks.workunit.client.0.smithi100.stdout:expected = image26 2024-04-07T17:09:46.680 INFO:tasks.workunit.client.0.smithi100.stdout:100.00% done 2024-04-07T17:09:46.680 INFO:tasks.workunit.client.0.smithi100.stdout:num images is: 3 2024-04-07T17:09:46.680 INFO:tasks.workunit.client.0.smithi100.stdout:expected: 3 2024-04-07T17:09:46.680 INFO:tasks.workunit.client.0.smithi100.stdout:image: image25 2024-04-07T17:09:46.680 INFO:tasks.workunit.client.0.smithi100.stdout:image: image26 2024-04-07T17:09:46.680 INFO:tasks.workunit.client.0.smithi100.stdout:image: image27 2024-04-07T17:09:46.680 INFO:tasks.workunit.client.0.smithi100.stdout:expected = image25 2024-04-07T17:09:46.680 INFO:tasks.workunit.client.0.smithi100.stdout:expected = image26 2024-04-07T17:09:46.680 INFO:tasks.workunit.client.0.smithi100.stdout:expected = image27 2024-04-07T17:09:46.790 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.TestCopyPP (4978 ms) 2024-04-07T17:09:46.790 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.TestDeepCopy 2024-04-07T17:09:47.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:09:46 smithi144 bash[66654]: cluster 2024-04-07T17:09:45.384003+0000 mgr.y (mgr.25175) 1632 : cluster [DBG] pgmap v2799: 265 pgs: 3 creating+peering, 16 unknown, 246 active+clean; 4.6 MiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 313 KiB/s rd, 20 KiB/s wr, 367 op/s 2024-04-07T17:09:47.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:09:46 smithi144 bash[66654]: cluster 2024-04-07T17:09:45.721543+0000 mon.a (mon.0) 2971 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T17:09:47.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:09:46 smithi144 bash[66654]: cluster 2024-04-07T17:09:45.740051+0000 mon.a (mon.0) 2972 : cluster [DBG] osdmap e1593: 8 total, 8 up, 8 in 2024-04-07T17:09:47.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:09:46 smithi100 bash[74942]: cluster 2024-04-07T17:09:45.384003+0000 mgr.y (mgr.25175) 1632 : cluster [DBG] pgmap v2799: 265 pgs: 3 creating+peering, 16 unknown, 246 active+clean; 4.6 MiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 313 KiB/s rd, 20 KiB/s wr, 367 op/s 2024-04-07T17:09:47.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:09:46 smithi100 bash[74942]: cluster 2024-04-07T17:09:45.721543+0000 mon.a (mon.0) 2971 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T17:09:47.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:09:46 smithi100 bash[74942]: cluster 2024-04-07T17:09:45.740051+0000 mon.a (mon.0) 2972 : cluster [DBG] osdmap e1593: 8 total, 8 up, 8 in 2024-04-07T17:09:47.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:09:46 smithi100 bash[76202]: cluster 2024-04-07T17:09:45.384003+0000 mgr.y (mgr.25175) 1632 : cluster [DBG] pgmap v2799: 265 pgs: 3 creating+peering, 16 unknown, 246 active+clean; 4.6 MiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 313 KiB/s rd, 20 KiB/s wr, 367 op/s 2024-04-07T17:09:47.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:09:46 smithi100 bash[76202]: cluster 2024-04-07T17:09:45.721543+0000 mon.a (mon.0) 2971 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T17:09:47.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:09:46 smithi100 bash[76202]: cluster 2024-04-07T17:09:45.740051+0000 mon.a (mon.0) 2972 : cluster [DBG] osdmap e1593: 8 total, 8 up, 8 in 2024-04-07T17:09:48.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:09:47 smithi144 bash[66654]: audit 2024-04-07T17:09:46.813678+0000 mon.a (mon.0) 2973 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:09:48.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:09:47 smithi144 bash[66654]: cluster 2024-04-07T17:09:47.722782+0000 mon.a (mon.0) 2974 : cluster [DBG] osdmap e1594: 8 total, 8 up, 8 in 2024-04-07T17:09:48.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:09:47 smithi144 bash[66654]: audit 2024-04-07T17:09:47.725047+0000 mon.a (mon.0) 2975 : audit [INF] from='client.? 172.21.15.100:0/407587632' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:09:48.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:09:47 smithi100 bash[74942]: audit 2024-04-07T17:09:46.813678+0000 mon.a (mon.0) 2973 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:09:48.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:09:47 smithi100 bash[74942]: cluster 2024-04-07T17:09:47.722782+0000 mon.a (mon.0) 2974 : cluster [DBG] osdmap e1594: 8 total, 8 up, 8 in 2024-04-07T17:09:48.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:09:47 smithi100 bash[74942]: audit 2024-04-07T17:09:47.725047+0000 mon.a (mon.0) 2975 : audit [INF] from='client.? 172.21.15.100:0/407587632' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:09:48.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:09:47 smithi100 bash[76202]: audit 2024-04-07T17:09:46.813678+0000 mon.a (mon.0) 2973 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:09:48.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:09:47 smithi100 bash[76202]: cluster 2024-04-07T17:09:47.722782+0000 mon.a (mon.0) 2974 : cluster [DBG] osdmap e1594: 8 total, 8 up, 8 in 2024-04-07T17:09:48.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:09:47 smithi100 bash[76202]: audit 2024-04-07T17:09:47.725047+0000 mon.a (mon.0) 2975 : audit [INF] from='client.? 172.21.15.100:0/407587632' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:09:48.703 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:09:48 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:09:48] "GET /metrics HTTP/1.1" 200 46136 "" "Prometheus/2.43.0" 2024-04-07T17:09:48.732 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:09:49.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:09:48 smithi144 bash[66654]: cluster 2024-04-07T17:09:47.385799+0000 mgr.y (mgr.25175) 1633 : cluster [DBG] pgmap v2801: 265 pgs: 3 creating+peering, 262 active+clean; 4.6 MiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 537 KiB/s rd, 34 KiB/s wr, 627 op/s 2024-04-07T17:09:49.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:09:48 smithi144 bash[66654]: audit 2024-04-07T17:09:48.713503+0000 mon.a (mon.0) 2976 : audit [INF] from='client.? 172.21.15.100:0/407587632' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T17:09:49.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:09:48 smithi144 bash[66654]: cluster 2024-04-07T17:09:48.720092+0000 mon.a (mon.0) 2977 : cluster [DBG] osdmap e1595: 8 total, 8 up, 8 in 2024-04-07T17:09:49.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:09:48 smithi100 bash[74942]: cluster 2024-04-07T17:09:47.385799+0000 mgr.y (mgr.25175) 1633 : cluster [DBG] pgmap v2801: 265 pgs: 3 creating+peering, 262 active+clean; 4.6 MiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 537 KiB/s rd, 34 KiB/s wr, 627 op/s 2024-04-07T17:09:49.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:09:48 smithi100 bash[74942]: audit 2024-04-07T17:09:48.713503+0000 mon.a (mon.0) 2976 : audit [INF] from='client.? 172.21.15.100:0/407587632' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T17:09:49.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:09:48 smithi100 bash[74942]: cluster 2024-04-07T17:09:48.720092+0000 mon.a (mon.0) 2977 : cluster [DBG] osdmap e1595: 8 total, 8 up, 8 in 2024-04-07T17:09:49.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:09:48 smithi100 bash[76202]: cluster 2024-04-07T17:09:47.385799+0000 mgr.y (mgr.25175) 1633 : cluster [DBG] pgmap v2801: 265 pgs: 3 creating+peering, 262 active+clean; 4.6 MiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 537 KiB/s rd, 34 KiB/s wr, 627 op/s 2024-04-07T17:09:49.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:09:48 smithi100 bash[76202]: audit 2024-04-07T17:09:48.713503+0000 mon.a (mon.0) 2976 : audit [INF] from='client.? 172.21.15.100:0/407587632' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T17:09:49.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:09:48 smithi100 bash[76202]: cluster 2024-04-07T17:09:48.720092+0000 mon.a (mon.0) 2977 : cluster [DBG] osdmap e1595: 8 total, 8 up, 8 in 2024-04-07T17:09:51.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:09:50 smithi144 bash[66654]: cluster 2024-04-07T17:09:49.387191+0000 mgr.y (mgr.25175) 1634 : cluster [DBG] pgmap v2804: 297 pgs: 3 creating+peering, 18 unknown, 276 active+clean; 4.6 MiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 274 KiB/s rd, 18 KiB/s wr, 324 op/s 2024-04-07T17:09:51.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:09:50 smithi144 bash[66654]: cluster 2024-04-07T17:09:49.725655+0000 mon.a (mon.0) 2978 : cluster [DBG] osdmap e1596: 8 total, 8 up, 8 in 2024-04-07T17:09:51.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:09:50 smithi100 bash[74942]: cluster 2024-04-07T17:09:49.387191+0000 mgr.y (mgr.25175) 1634 : cluster [DBG] pgmap v2804: 297 pgs: 3 creating+peering, 18 unknown, 276 active+clean; 4.6 MiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 274 KiB/s rd, 18 KiB/s wr, 324 op/s 2024-04-07T17:09:51.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:09:50 smithi100 bash[74942]: cluster 2024-04-07T17:09:49.725655+0000 mon.a (mon.0) 2978 : cluster [DBG] osdmap e1596: 8 total, 8 up, 8 in 2024-04-07T17:09:51.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:09:50 smithi100 bash[76202]: cluster 2024-04-07T17:09:49.387191+0000 mgr.y (mgr.25175) 1634 : cluster [DBG] pgmap v2804: 297 pgs: 3 creating+peering, 18 unknown, 276 active+clean; 4.6 MiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 274 KiB/s rd, 18 KiB/s wr, 324 op/s 2024-04-07T17:09:51.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:09:50 smithi100 bash[76202]: cluster 2024-04-07T17:09:49.725655+0000 mon.a (mon.0) 2978 : cluster [DBG] osdmap e1596: 8 total, 8 up, 8 in 2024-04-07T17:09:52.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:09:51 smithi144 bash[66654]: cluster 2024-04-07T17:09:50.726295+0000 mon.a (mon.0) 2979 : cluster [DBG] osdmap e1597: 8 total, 8 up, 8 in 2024-04-07T17:09:52.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:09:51 smithi100 bash[74942]: cluster 2024-04-07T17:09:50.726295+0000 mon.a (mon.0) 2979 : cluster [DBG] osdmap e1597: 8 total, 8 up, 8 in 2024-04-07T17:09:52.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:09:51 smithi100 bash[76202]: cluster 2024-04-07T17:09:50.726295+0000 mon.a (mon.0) 2979 : cluster [DBG] osdmap e1597: 8 total, 8 up, 8 in 2024-04-07T17:09:52.616 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:09:52 smithi144 bash[95233]: ts=2024-04-07T17:09:52.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:09:53.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:09:52 smithi144 bash[66654]: cluster 2024-04-07T17:09:51.388079+0000 mgr.y (mgr.25175) 1635 : cluster [DBG] pgmap v2807: 297 pgs: 3 creating+peering, 16 unknown, 278 active+clean; 4.6 MiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 30 KiB/s rd, 1.7 KiB/s wr, 36 op/s 2024-04-07T17:09:53.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:09:52 smithi144 bash[66654]: cluster 2024-04-07T17:09:52.719859+0000 mon.a (mon.0) 2980 : cluster [DBG] osdmap e1598: 8 total, 8 up, 8 in 2024-04-07T17:09:53.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:09:52 smithi100 bash[74942]: cluster 2024-04-07T17:09:51.388079+0000 mgr.y (mgr.25175) 1635 : cluster [DBG] pgmap v2807: 297 pgs: 3 creating+peering, 16 unknown, 278 active+clean; 4.6 MiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 30 KiB/s rd, 1.7 KiB/s wr, 36 op/s 2024-04-07T17:09:53.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:09:52 smithi100 bash[74942]: cluster 2024-04-07T17:09:52.719859+0000 mon.a (mon.0) 2980 : cluster [DBG] osdmap e1598: 8 total, 8 up, 8 in 2024-04-07T17:09:53.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:09:52 smithi100 bash[76202]: cluster 2024-04-07T17:09:51.388079+0000 mgr.y (mgr.25175) 1635 : cluster [DBG] pgmap v2807: 297 pgs: 3 creating+peering, 16 unknown, 278 active+clean; 4.6 MiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 30 KiB/s rd, 1.7 KiB/s wr, 36 op/s 2024-04-07T17:09:53.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:09:52 smithi100 bash[76202]: cluster 2024-04-07T17:09:52.719859+0000 mon.a (mon.0) 2980 : cluster [DBG] osdmap e1598: 8 total, 8 up, 8 in 2024-04-07T17:09:55.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:09:54 smithi144 bash[66654]: cluster 2024-04-07T17:09:53.389280+0000 mgr.y (mgr.25175) 1636 : cluster [DBG] pgmap v2809: 297 pgs: 297 active+clean; 4.6 MiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 301 KiB/s rd, 20 KiB/s wr, 356 op/s 2024-04-07T17:09:55.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:09:54 smithi144 bash[66654]: cluster 2024-04-07T17:09:53.765459+0000 mon.a (mon.0) 2981 : cluster [DBG] osdmap e1599: 8 total, 8 up, 8 in 2024-04-07T17:09:55.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:09:54 smithi100 bash[74942]: cluster 2024-04-07T17:09:53.389280+0000 mgr.y (mgr.25175) 1636 : cluster [DBG] pgmap v2809: 297 pgs: 297 active+clean; 4.6 MiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 301 KiB/s rd, 20 KiB/s wr, 356 op/s 2024-04-07T17:09:55.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:09:54 smithi100 bash[74942]: cluster 2024-04-07T17:09:53.765459+0000 mon.a (mon.0) 2981 : cluster [DBG] osdmap e1599: 8 total, 8 up, 8 in 2024-04-07T17:09:55.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:09:54 smithi100 bash[76202]: cluster 2024-04-07T17:09:53.389280+0000 mgr.y (mgr.25175) 1636 : cluster [DBG] pgmap v2809: 297 pgs: 297 active+clean; 4.6 MiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 301 KiB/s rd, 20 KiB/s wr, 356 op/s 2024-04-07T17:09:55.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:09:54 smithi100 bash[76202]: cluster 2024-04-07T17:09:53.765459+0000 mon.a (mon.0) 2981 : cluster [DBG] osdmap e1599: 8 total, 8 up, 8 in 2024-04-07T17:09:55.980 INFO:tasks.workunit.client.0.smithi100.stdout:image: image28 2024-04-07T17:09:55.980 INFO:tasks.workunit.client.0.smithi100.stdout:expected = image28 2024-04-07T17:09:55.980 INFO:tasks.workunit.client.0.smithi100.stdout:found image28 2024-04-07T17:09:55.980 INFO:tasks.workunit.client.0.smithi100.stdout:erased image28 2024-04-07T17:09:55.980 INFO:tasks.workunit.client.0.smithi100.stdout:image: image28 2024-04-07T17:09:55.980 INFO:tasks.workunit.client.0.smithi100.stdout:image: image29 2024-04-07T17:09:55.980 INFO:tasks.workunit.client.0.smithi100.stdout:expected = image28 2024-04-07T17:09:55.980 INFO:tasks.workunit.client.0.smithi100.stdout:found image28 2024-04-07T17:09:55.980 INFO:tasks.workunit.client.0.smithi100.stdout:erased image28 2024-04-07T17:09:55.980 INFO:tasks.workunit.client.0.smithi100.stdout:expected = image29 2024-04-07T17:09:55.980 INFO:tasks.workunit.client.0.smithi100.stdout:found image29 2024-04-07T17:09:55.981 INFO:tasks.workunit.client.0.smithi100.stdout:erased image29 2024-04-07T17:09:55.981 INFO:tasks.workunit.client.0.smithi100.stdout:100.00% done 2024-04-07T17:09:55.981 INFO:tasks.workunit.client.0.smithi100.stdout:image: image28 2024-04-07T17:09:55.981 INFO:tasks.workunit.client.0.smithi100.stdout:image: image29 2024-04-07T17:09:55.981 INFO:tasks.workunit.client.0.smithi100.stdout:image: image30 2024-04-07T17:09:55.981 INFO:tasks.workunit.client.0.smithi100.stdout:expected = image28 2024-04-07T17:09:55.981 INFO:tasks.workunit.client.0.smithi100.stdout:found image28 2024-04-07T17:09:55.981 INFO:tasks.workunit.client.0.smithi100.stdout:erased image28 2024-04-07T17:09:55.981 INFO:tasks.workunit.client.0.smithi100.stdout:expected = image29 2024-04-07T17:09:55.981 INFO:tasks.workunit.client.0.smithi100.stdout:found image29 2024-04-07T17:09:55.981 INFO:tasks.workunit.client.0.smithi100.stdout:erased image29 2024-04-07T17:09:55.981 INFO:tasks.workunit.client.0.smithi100.stdout:expected = image30 2024-04-07T17:09:55.981 INFO:tasks.workunit.client.0.smithi100.stdout:found image30 2024-04-07T17:09:55.981 INFO:tasks.workunit.client.0.smithi100.stdout:erased image30 2024-04-07T17:09:55.982 INFO:tasks.workunit.client.0.smithi100.stdout:image: image28 2024-04-07T17:09:55.982 INFO:tasks.workunit.client.0.smithi100.stdout:image: image29 2024-04-07T17:09:55.982 INFO:tasks.workunit.client.0.smithi100.stdout:image: image30 2024-04-07T17:09:55.982 INFO:tasks.workunit.client.0.smithi100.stdout:image: image31 2024-04-07T17:09:55.982 INFO:tasks.workunit.client.0.smithi100.stdout:expected = image28 2024-04-07T17:09:55.982 INFO:tasks.workunit.client.0.smithi100.stdout:found image28 2024-04-07T17:09:55.982 INFO:tasks.workunit.client.0.smithi100.stdout:erased image28 2024-04-07T17:09:55.982 INFO:tasks.workunit.client.0.smithi100.stdout:expected = image29 2024-04-07T17:09:55.982 INFO:tasks.workunit.client.0.smithi100.stdout:found image29 2024-04-07T17:09:55.982 INFO:tasks.workunit.client.0.smithi100.stdout:erased image29 2024-04-07T17:09:55.982 INFO:tasks.workunit.client.0.smithi100.stdout:expected = image30 2024-04-07T17:09:55.982 INFO:tasks.workunit.client.0.smithi100.stdout:found image30 2024-04-07T17:09:55.982 INFO:tasks.workunit.client.0.smithi100.stdout:erased image30 2024-04-07T17:09:55.982 INFO:tasks.workunit.client.0.smithi100.stdout:expected = image31 2024-04-07T17:09:55.982 INFO:tasks.workunit.client.0.smithi100.stdout:found image31 2024-04-07T17:09:55.983 INFO:tasks.workunit.client.0.smithi100.stdout:erased image31 2024-04-07T17:09:55.983 INFO:tasks.workunit.client.0.smithi100.stdout:image: image28 2024-04-07T17:09:55.983 INFO:tasks.workunit.client.0.smithi100.stdout:image: image29 2024-04-07T17:09:55.983 INFO:tasks.workunit.client.0.smithi100.stdout:image: image30 2024-04-07T17:09:55.983 INFO:tasks.workunit.client.0.smithi100.stdout:image: image31 2024-04-07T17:09:55.983 INFO:tasks.workunit.client.0.smithi100.stdout:image: image32 2024-04-07T17:09:55.983 INFO:tasks.workunit.client.0.smithi100.stdout:expected = image28 2024-04-07T17:09:55.983 INFO:tasks.workunit.client.0.smithi100.stdout:found image28 2024-04-07T17:09:55.983 INFO:tasks.workunit.client.0.smithi100.stdout:erased image28 2024-04-07T17:09:55.983 INFO:tasks.workunit.client.0.smithi100.stdout:expected = image29 2024-04-07T17:09:55.983 INFO:tasks.workunit.client.0.smithi100.stdout:found image29 2024-04-07T17:09:55.983 INFO:tasks.workunit.client.0.smithi100.stdout:erased image29 2024-04-07T17:09:55.983 INFO:tasks.workunit.client.0.smithi100.stdout:expected = image30 2024-04-07T17:09:55.983 INFO:tasks.workunit.client.0.smithi100.stdout:found image30 2024-04-07T17:09:55.983 INFO:tasks.workunit.client.0.smithi100.stdout:erased image30 2024-04-07T17:09:55.984 INFO:tasks.workunit.client.0.smithi100.stdout:expected = image31 2024-04-07T17:09:55.984 INFO:tasks.workunit.client.0.smithi100.stdout:found image31 2024-04-07T17:09:55.984 INFO:tasks.workunit.client.0.smithi100.stdout:erased image31 2024-04-07T17:09:55.984 INFO:tasks.workunit.client.0.smithi100.stdout:expected = image32 2024-04-07T17:09:55.984 INFO:tasks.workunit.client.0.smithi100.stdout:found image32 2024-04-07T17:09:55.984 INFO:tasks.workunit.client.0.smithi100.stdout:erased image32 2024-04-07T17:09:55.984 INFO:tasks.workunit.client.0.smithi100.stdout:100.00% done 2024-04-07T17:09:55.984 INFO:tasks.workunit.client.0.smithi100.stdout:image: image28 2024-04-07T17:09:55.984 INFO:tasks.workunit.client.0.smithi100.stdout:image: image29 2024-04-07T17:09:55.984 INFO:tasks.workunit.client.0.smithi100.stdout:image: image30 2024-04-07T17:09:55.984 INFO:tasks.workunit.client.0.smithi100.stdout:image: image31 2024-04-07T17:09:55.984 INFO:tasks.workunit.client.0.smithi100.stdout:image: image32 2024-04-07T17:09:55.984 INFO:tasks.workunit.client.0.smithi100.stdout:image: image33 2024-04-07T17:09:55.984 INFO:tasks.workunit.client.0.smithi100.stdout:expected = image28 2024-04-07T17:09:55.984 INFO:tasks.workunit.client.0.smithi100.stdout:found image28 2024-04-07T17:09:55.985 INFO:tasks.workunit.client.0.smithi100.stdout:erased image28 2024-04-07T17:09:55.985 INFO:tasks.workunit.client.0.smithi100.stdout:expected = image29 2024-04-07T17:09:55.985 INFO:tasks.workunit.client.0.smithi100.stdout:found image29 2024-04-07T17:09:55.985 INFO:tasks.workunit.client.0.smithi100.stdout:erased image29 2024-04-07T17:09:55.985 INFO:tasks.workunit.client.0.smithi100.stdout:expected = image30 2024-04-07T17:09:55.985 INFO:tasks.workunit.client.0.smithi100.stdout:found image30 2024-04-07T17:09:55.985 INFO:tasks.workunit.client.0.smithi100.stdout:erased image30 2024-04-07T17:09:55.985 INFO:tasks.workunit.client.0.smithi100.stdout:expected = image31 2024-04-07T17:09:55.985 INFO:tasks.workunit.client.0.smithi100.stdout:found image31 2024-04-07T17:09:55.985 INFO:tasks.workunit.client.0.smithi100.stdout:erased image31 2024-04-07T17:09:55.985 INFO:tasks.workunit.client.0.smithi100.stdout:expected = image32 2024-04-07T17:09:55.985 INFO:tasks.workunit.client.0.smithi100.stdout:found image32 2024-04-07T17:09:55.985 INFO:tasks.workunit.client.0.smithi100.stdout:erased image32 2024-04-07T17:09:55.985 INFO:tasks.workunit.client.0.smithi100.stdout:expected = image33 2024-04-07T17:09:55.985 INFO:tasks.workunit.client.0.smithi100.stdout:found image33 2024-04-07T17:09:55.986 INFO:tasks.workunit.client.0.smithi100.stdout:erased image33 2024-04-07T17:09:55.986 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.TestDeepCopy (9191 ms) 2024-04-07T17:09:55.986 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.TestDeepCopyPP 2024-04-07T17:09:56.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:09:55 smithi144 bash[66654]: cluster 2024-04-07T17:09:54.778445+0000 mon.a (mon.0) 2982 : cluster [DBG] osdmap e1600: 8 total, 8 up, 8 in 2024-04-07T17:09:56.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:09:55 smithi100 bash[74942]: cluster 2024-04-07T17:09:54.778445+0000 mon.a (mon.0) 2982 : cluster [DBG] osdmap e1600: 8 total, 8 up, 8 in 2024-04-07T17:09:56.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:09:55 smithi100 bash[76202]: cluster 2024-04-07T17:09:54.778445+0000 mon.a (mon.0) 2982 : cluster [DBG] osdmap e1600: 8 total, 8 up, 8 in 2024-04-07T17:09:57.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:09:56 smithi144 bash[66654]: cluster 2024-04-07T17:09:55.390451+0000 mgr.y (mgr.25175) 1637 : cluster [DBG] pgmap v2812: 297 pgs: 297 active+clean; 4.6 MiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 308 KiB/s rd, 21 KiB/s wr, 366 op/s 2024-04-07T17:09:57.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:09:56 smithi144 bash[66654]: cluster 2024-04-07T17:09:55.785991+0000 mon.a (mon.0) 2983 : cluster [DBG] osdmap e1601: 8 total, 8 up, 8 in 2024-04-07T17:09:57.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:09:56 smithi100 bash[74942]: cluster 2024-04-07T17:09:55.390451+0000 mgr.y (mgr.25175) 1637 : cluster [DBG] pgmap v2812: 297 pgs: 297 active+clean; 4.6 MiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 308 KiB/s rd, 21 KiB/s wr, 366 op/s 2024-04-07T17:09:57.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:09:56 smithi100 bash[74942]: cluster 2024-04-07T17:09:55.785991+0000 mon.a (mon.0) 2983 : cluster [DBG] osdmap e1601: 8 total, 8 up, 8 in 2024-04-07T17:09:57.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:09:56 smithi100 bash[76202]: cluster 2024-04-07T17:09:55.390451+0000 mgr.y (mgr.25175) 1637 : cluster [DBG] pgmap v2812: 297 pgs: 297 active+clean; 4.6 MiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 308 KiB/s rd, 21 KiB/s wr, 366 op/s 2024-04-07T17:09:57.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:09:56 smithi100 bash[76202]: cluster 2024-04-07T17:09:55.785991+0000 mon.a (mon.0) 2983 : cluster [DBG] osdmap e1601: 8 total, 8 up, 8 in 2024-04-07T17:09:57.732 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:09:58.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:09:57 smithi144 bash[66654]: cluster 2024-04-07T17:09:56.791306+0000 mon.a (mon.0) 2984 : cluster [DBG] osdmap e1602: 8 total, 8 up, 8 in 2024-04-07T17:09:58.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:09:57 smithi144 bash[66654]: audit 2024-04-07T17:09:56.814364+0000 mon.a (mon.0) 2985 : audit [INF] from='client.? 172.21.15.100:0/2152605383' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:09:58.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:09:57 smithi144 bash[66654]: audit 2024-04-07T17:09:57.715934+0000 mon.a (mon.0) 2986 : audit [INF] from='client.? 172.21.15.100:0/2152605383' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T17:09:58.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:09:57 smithi144 bash[66654]: cluster 2024-04-07T17:09:57.725010+0000 mon.a (mon.0) 2987 : cluster [DBG] osdmap e1603: 8 total, 8 up, 8 in 2024-04-07T17:09:58.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:09:57 smithi100 bash[74942]: cluster 2024-04-07T17:09:56.791306+0000 mon.a (mon.0) 2984 : cluster [DBG] osdmap e1602: 8 total, 8 up, 8 in 2024-04-07T17:09:58.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:09:57 smithi100 bash[74942]: audit 2024-04-07T17:09:56.814364+0000 mon.a (mon.0) 2985 : audit [INF] from='client.? 172.21.15.100:0/2152605383' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:09:58.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:09:57 smithi100 bash[74942]: audit 2024-04-07T17:09:57.715934+0000 mon.a (mon.0) 2986 : audit [INF] from='client.? 172.21.15.100:0/2152605383' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T17:09:58.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:09:57 smithi100 bash[74942]: cluster 2024-04-07T17:09:57.725010+0000 mon.a (mon.0) 2987 : cluster [DBG] osdmap e1603: 8 total, 8 up, 8 in 2024-04-07T17:09:58.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:09:57 smithi100 bash[76202]: cluster 2024-04-07T17:09:56.791306+0000 mon.a (mon.0) 2984 : cluster [DBG] osdmap e1602: 8 total, 8 up, 8 in 2024-04-07T17:09:58.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:09:57 smithi100 bash[76202]: audit 2024-04-07T17:09:56.814364+0000 mon.a (mon.0) 2985 : audit [INF] from='client.? 172.21.15.100:0/2152605383' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:09:58.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:09:57 smithi100 bash[76202]: audit 2024-04-07T17:09:57.715934+0000 mon.a (mon.0) 2986 : audit [INF] from='client.? 172.21.15.100:0/2152605383' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T17:09:58.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:09:57 smithi100 bash[76202]: cluster 2024-04-07T17:09:57.725010+0000 mon.a (mon.0) 2987 : cluster [DBG] osdmap e1603: 8 total, 8 up, 8 in 2024-04-07T17:09:58.703 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:09:58 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:09:58] "GET /metrics HTTP/1.1" 200 46136 "" "Prometheus/2.43.0" 2024-04-07T17:09:59.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:09:58 smithi144 bash[66654]: cluster 2024-04-07T17:09:57.392564+0000 mgr.y (mgr.25175) 1638 : cluster [DBG] pgmap v2815: 329 pgs: 17 creating+peering, 15 unknown, 297 active+clean; 4.6 MiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 131 KiB/s rd, 5.2 KiB/s wr, 157 op/s 2024-04-07T17:09:59.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:09:58 smithi144 bash[66654]: cluster 2024-04-07T17:09:57.802840+0000 mon.a (mon.0) 2988 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T17:09:59.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:09:58 smithi144 bash[66654]: cluster 2024-04-07T17:09:58.725664+0000 mon.a (mon.0) 2989 : cluster [DBG] osdmap e1604: 8 total, 8 up, 8 in 2024-04-07T17:09:59.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:09:58 smithi100 bash[74942]: cluster 2024-04-07T17:09:57.392564+0000 mgr.y (mgr.25175) 1638 : cluster [DBG] pgmap v2815: 329 pgs: 17 creating+peering, 15 unknown, 297 active+clean; 4.6 MiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 131 KiB/s rd, 5.2 KiB/s wr, 157 op/s 2024-04-07T17:09:59.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:09:58 smithi100 bash[74942]: cluster 2024-04-07T17:09:57.802840+0000 mon.a (mon.0) 2988 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T17:09:59.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:09:58 smithi100 bash[74942]: cluster 2024-04-07T17:09:58.725664+0000 mon.a (mon.0) 2989 : cluster [DBG] osdmap e1604: 8 total, 8 up, 8 in 2024-04-07T17:09:59.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:09:58 smithi100 bash[76202]: cluster 2024-04-07T17:09:57.392564+0000 mgr.y (mgr.25175) 1638 : cluster [DBG] pgmap v2815: 329 pgs: 17 creating+peering, 15 unknown, 297 active+clean; 4.6 MiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 131 KiB/s rd, 5.2 KiB/s wr, 157 op/s 2024-04-07T17:09:59.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:09:58 smithi100 bash[76202]: cluster 2024-04-07T17:09:57.802840+0000 mon.a (mon.0) 2988 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T17:09:59.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:09:58 smithi100 bash[76202]: cluster 2024-04-07T17:09:58.725664+0000 mon.a (mon.0) 2989 : cluster [DBG] osdmap e1604: 8 total, 8 up, 8 in 2024-04-07T17:10:00.397 INFO:tasks.workunit.client.0.smithi100.stdout:num images is: 1 2024-04-07T17:10:00.397 INFO:tasks.workunit.client.0.smithi100.stdout:expected: 1 2024-04-07T17:10:00.397 INFO:tasks.workunit.client.0.smithi100.stdout:image: image34 2024-04-07T17:10:00.397 INFO:tasks.workunit.client.0.smithi100.stdout:expected = image34 2024-04-07T17:10:00.473 INFO:tasks.workunit.client.0.smithi100.stdout:num images is: 2 2024-04-07T17:10:00.473 INFO:tasks.workunit.client.0.smithi100.stdout:expected: 2 2024-04-07T17:10:00.473 INFO:tasks.workunit.client.0.smithi100.stdout:image: image34 2024-04-07T17:10:00.474 INFO:tasks.workunit.client.0.smithi100.stdout:image: image35 2024-04-07T17:10:00.474 INFO:tasks.workunit.client.0.smithi100.stdout:expected = image34 2024-04-07T17:10:00.474 INFO:tasks.workunit.client.0.smithi100.stdout:expected = image35 2024-04-07T17:10:00.630 INFO:tasks.workunit.client.0.smithi100.stdout:100.00% done 2024-04-07T17:10:00.630 INFO:tasks.workunit.client.0.smithi100.stdout:num images is: 3 2024-04-07T17:10:00.630 INFO:tasks.workunit.client.0.smithi100.stdout:expected: 3 2024-04-07T17:10:00.630 INFO:tasks.workunit.client.0.smithi100.stdout:image: image34 2024-04-07T17:10:00.630 INFO:tasks.workunit.client.0.smithi100.stdout:image: image35 2024-04-07T17:10:00.630 INFO:tasks.workunit.client.0.smithi100.stdout:image: image36 2024-04-07T17:10:00.630 INFO:tasks.workunit.client.0.smithi100.stdout:expected = image34 2024-04-07T17:10:00.630 INFO:tasks.workunit.client.0.smithi100.stdout:expected = image35 2024-04-07T17:10:00.631 INFO:tasks.workunit.client.0.smithi100.stdout:expected = image36 2024-04-07T17:10:00.720 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.TestDeepCopyPP (4740 ms) 2024-04-07T17:10:00.720 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.TestCreateLsDeleteSnap 2024-04-07T17:10:00.720 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:10:01.118 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:00 smithi144 bash[66654]: cluster 2024-04-07T17:09:59.393986+0000 mgr.y (mgr.25175) 1639 : cluster [DBG] pgmap v2818: 329 pgs: 17 creating+peering, 4 unknown, 308 active+clean; 4.6 MiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 187 KiB/s rd, 7.5 KiB/s wr, 221 op/s 2024-04-07T17:10:01.118 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:00 smithi144 bash[66654]: cluster 2024-04-07T17:09:59.730049+0000 mon.a (mon.0) 2990 : cluster [DBG] osdmap e1605: 8 total, 8 up, 8 in 2024-04-07T17:10:01.118 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:00 smithi144 bash[66654]: cluster 2024-04-07T17:09:59.812837+0000 mon.a (mon.0) 2991 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T17:10:01.118 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:00 smithi144 bash[66654]: cluster 2024-04-07T17:10:00.000152+0000 mon.a (mon.0) 2992 : cluster [WRN] Health detail: HEALTH_WARN Telemetry requires re-opt-in 2024-04-07T17:10:01.118 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:00 smithi144 bash[66654]: cluster 2024-04-07T17:10:00.000229+0000 mon.a (mon.0) 2993 : cluster [WRN] [WRN] TELEMETRY_CHANGED: Telemetry requires re-opt-in 2024-04-07T17:10:01.118 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:00 smithi144 bash[66654]: cluster 2024-04-07T17:10:00.000290+0000 mon.a (mon.0) 2994 : cluster [WRN] telemetry module includes new collections; please re-opt-in to new collections with `ceph telemetry on` 2024-04-07T17:10:01.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:00 smithi100 bash[74942]: cluster 2024-04-07T17:09:59.393986+0000 mgr.y (mgr.25175) 1639 : cluster [DBG] pgmap v2818: 329 pgs: 17 creating+peering, 4 unknown, 308 active+clean; 4.6 MiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 187 KiB/s rd, 7.5 KiB/s wr, 221 op/s 2024-04-07T17:10:01.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:00 smithi100 bash[74942]: cluster 2024-04-07T17:09:59.730049+0000 mon.a (mon.0) 2990 : cluster [DBG] osdmap e1605: 8 total, 8 up, 8 in 2024-04-07T17:10:01.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:00 smithi100 bash[74942]: cluster 2024-04-07T17:09:59.812837+0000 mon.a (mon.0) 2991 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T17:10:01.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:00 smithi100 bash[74942]: cluster 2024-04-07T17:10:00.000152+0000 mon.a (mon.0) 2992 : cluster [WRN] Health detail: HEALTH_WARN Telemetry requires re-opt-in 2024-04-07T17:10:01.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:00 smithi100 bash[74942]: cluster 2024-04-07T17:10:00.000229+0000 mon.a (mon.0) 2993 : cluster [WRN] [WRN] TELEMETRY_CHANGED: Telemetry requires re-opt-in 2024-04-07T17:10:01.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:00 smithi100 bash[74942]: cluster 2024-04-07T17:10:00.000290+0000 mon.a (mon.0) 2994 : cluster [WRN] telemetry module includes new collections; please re-opt-in to new collections with `ceph telemetry on` 2024-04-07T17:10:01.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:00 smithi100 bash[76202]: cluster 2024-04-07T17:09:59.393986+0000 mgr.y (mgr.25175) 1639 : cluster [DBG] pgmap v2818: 329 pgs: 17 creating+peering, 4 unknown, 308 active+clean; 4.6 MiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 187 KiB/s rd, 7.5 KiB/s wr, 221 op/s 2024-04-07T17:10:01.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:00 smithi100 bash[76202]: cluster 2024-04-07T17:09:59.730049+0000 mon.a (mon.0) 2990 : cluster [DBG] osdmap e1605: 8 total, 8 up, 8 in 2024-04-07T17:10:01.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:00 smithi100 bash[76202]: cluster 2024-04-07T17:09:59.812837+0000 mon.a (mon.0) 2991 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T17:10:01.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:00 smithi100 bash[76202]: cluster 2024-04-07T17:10:00.000152+0000 mon.a (mon.0) 2992 : cluster [WRN] Health detail: HEALTH_WARN Telemetry requires re-opt-in 2024-04-07T17:10:01.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:00 smithi100 bash[76202]: cluster 2024-04-07T17:10:00.000229+0000 mon.a (mon.0) 2993 : cluster [WRN] [WRN] TELEMETRY_CHANGED: Telemetry requires re-opt-in 2024-04-07T17:10:01.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:00 smithi100 bash[76202]: cluster 2024-04-07T17:10:00.000290+0000 mon.a (mon.0) 2994 : cluster [WRN] telemetry module includes new collections; please re-opt-in to new collections with `ceph telemetry on` 2024-04-07T17:10:02.616 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:10:02 smithi144 bash[95233]: ts=2024-04-07T17:10:02.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:10:03.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:02 smithi144 bash[66654]: cluster 2024-04-07T17:10:01.395185+0000 mgr.y (mgr.25175) 1640 : cluster [DBG] pgmap v2820: 329 pgs: 17 creating+peering, 312 active+clean; 4.6 MiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 164 KiB/s rd, 6.9 KiB/s wr, 194 op/s 2024-04-07T17:10:03.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:02 smithi144 bash[66654]: cluster 2024-04-07T17:10:01.739975+0000 mon.a (mon.0) 2995 : cluster [DBG] osdmap e1606: 8 total, 8 up, 8 in 2024-04-07T17:10:03.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:02 smithi144 bash[66654]: audit 2024-04-07T17:10:01.814181+0000 mon.a (mon.0) 2996 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:10:03.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:02 smithi100 bash[74942]: cluster 2024-04-07T17:10:01.395185+0000 mgr.y (mgr.25175) 1640 : cluster [DBG] pgmap v2820: 329 pgs: 17 creating+peering, 312 active+clean; 4.6 MiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 164 KiB/s rd, 6.9 KiB/s wr, 194 op/s 2024-04-07T17:10:03.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:02 smithi100 bash[74942]: cluster 2024-04-07T17:10:01.739975+0000 mon.a (mon.0) 2995 : cluster [DBG] osdmap e1606: 8 total, 8 up, 8 in 2024-04-07T17:10:03.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:02 smithi100 bash[74942]: audit 2024-04-07T17:10:01.814181+0000 mon.a (mon.0) 2996 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:10:03.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:02 smithi100 bash[76202]: cluster 2024-04-07T17:10:01.395185+0000 mgr.y (mgr.25175) 1640 : cluster [DBG] pgmap v2820: 329 pgs: 17 creating+peering, 312 active+clean; 4.6 MiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 164 KiB/s rd, 6.9 KiB/s wr, 194 op/s 2024-04-07T17:10:03.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:02 smithi100 bash[76202]: cluster 2024-04-07T17:10:01.739975+0000 mon.a (mon.0) 2995 : cluster [DBG] osdmap e1606: 8 total, 8 up, 8 in 2024-04-07T17:10:03.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:02 smithi100 bash[76202]: audit 2024-04-07T17:10:01.814181+0000 mon.a (mon.0) 2996 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:10:04.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:03 smithi144 bash[66654]: cluster 2024-04-07T17:10:02.743605+0000 mon.a (mon.0) 2997 : cluster [DBG] osdmap e1607: 8 total, 8 up, 8 in 2024-04-07T17:10:04.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:03 smithi100 bash[74942]: cluster 2024-04-07T17:10:02.743605+0000 mon.a (mon.0) 2997 : cluster [DBG] osdmap e1607: 8 total, 8 up, 8 in 2024-04-07T17:10:04.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:03 smithi100 bash[76202]: cluster 2024-04-07T17:10:02.743605+0000 mon.a (mon.0) 2997 : cluster [DBG] osdmap e1607: 8 total, 8 up, 8 in 2024-04-07T17:10:04.827 INFO:tasks.workunit.client.0.smithi100.stdout:num snaps is: 1 2024-04-07T17:10:04.827 INFO:tasks.workunit.client.0.smithi100.stdout:expected: 1 2024-04-07T17:10:04.827 INFO:tasks.workunit.client.0.smithi100.stdout:snap: snap1 2024-04-07T17:10:04.827 INFO:tasks.workunit.client.0.smithi100.stdout:found snap1 with size 2097152 2024-04-07T17:10:04.827 INFO:tasks.workunit.client.0.smithi100.stdout:num snaps is: 2 2024-04-07T17:10:04.827 INFO:tasks.workunit.client.0.smithi100.stdout:expected: 2 2024-04-07T17:10:04.828 INFO:tasks.workunit.client.0.smithi100.stdout:snap: snap1 2024-04-07T17:10:04.828 INFO:tasks.workunit.client.0.smithi100.stdout:snap: snap2 2024-04-07T17:10:04.828 INFO:tasks.workunit.client.0.smithi100.stdout:found snap1 with size 2097152 2024-04-07T17:10:04.828 INFO:tasks.workunit.client.0.smithi100.stdout:found snap2 with size 4194304 2024-04-07T17:10:04.828 INFO:tasks.workunit.client.0.smithi100.stdout:num snaps is: 1 2024-04-07T17:10:04.828 INFO:tasks.workunit.client.0.smithi100.stdout:expected: 1 2024-04-07T17:10:04.828 INFO:tasks.workunit.client.0.smithi100.stdout:snap: snap2 2024-04-07T17:10:04.828 INFO:tasks.workunit.client.0.smithi100.stdout:found snap2 with size 4194304 2024-04-07T17:10:04.828 INFO:tasks.workunit.client.0.smithi100.stdout:num snaps is: 0 2024-04-07T17:10:04.828 INFO:tasks.workunit.client.0.smithi100.stdout:expected: 0 2024-04-07T17:10:04.828 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.TestCreateLsDeleteSnap (4107 ms) 2024-04-07T17:10:04.828 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.TestGetSnapShotTimeStamp 2024-04-07T17:10:04.828 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:10:05.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:04 smithi144 bash[66654]: cluster 2024-04-07T17:10:03.396452+0000 mgr.y (mgr.25175) 1641 : cluster [DBG] pgmap v2823: 329 pgs: 329 active+clean; 4.6 MiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 320 KiB/s rd, 22 KiB/s wr, 383 op/s 2024-04-07T17:10:05.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:04 smithi144 bash[66654]: cluster 2024-04-07T17:10:03.770069+0000 mon.a (mon.0) 2998 : cluster [DBG] osdmap e1608: 8 total, 8 up, 8 in 2024-04-07T17:10:05.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:04 smithi100 bash[74942]: cluster 2024-04-07T17:10:03.396452+0000 mgr.y (mgr.25175) 1641 : cluster [DBG] pgmap v2823: 329 pgs: 329 active+clean; 4.6 MiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 320 KiB/s rd, 22 KiB/s wr, 383 op/s 2024-04-07T17:10:05.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:04 smithi100 bash[74942]: cluster 2024-04-07T17:10:03.770069+0000 mon.a (mon.0) 2998 : cluster [DBG] osdmap e1608: 8 total, 8 up, 8 in 2024-04-07T17:10:05.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:04 smithi100 bash[76202]: cluster 2024-04-07T17:10:03.396452+0000 mgr.y (mgr.25175) 1641 : cluster [DBG] pgmap v2823: 329 pgs: 329 active+clean; 4.6 MiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 320 KiB/s rd, 22 KiB/s wr, 383 op/s 2024-04-07T17:10:05.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:04 smithi100 bash[76202]: cluster 2024-04-07T17:10:03.770069+0000 mon.a (mon.0) 2998 : cluster [DBG] osdmap e1608: 8 total, 8 up, 8 in 2024-04-07T17:10:06.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:05 smithi144 bash[66654]: cluster 2024-04-07T17:10:04.770988+0000 mon.a (mon.0) 2999 : cluster [DBG] osdmap e1609: 8 total, 8 up, 8 in 2024-04-07T17:10:06.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:05 smithi100 bash[74942]: cluster 2024-04-07T17:10:04.770988+0000 mon.a (mon.0) 2999 : cluster [DBG] osdmap e1609: 8 total, 8 up, 8 in 2024-04-07T17:10:06.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:05 smithi100 bash[76202]: cluster 2024-04-07T17:10:04.770988+0000 mon.a (mon.0) 2999 : cluster [DBG] osdmap e1609: 8 total, 8 up, 8 in 2024-04-07T17:10:06.862 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.TestGetSnapShotTimeStamp (2035 ms) 2024-04-07T17:10:06.863 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.TestCreateLsDeleteSnapPP 2024-04-07T17:10:06.863 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:10:07.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:06 smithi144 bash[66654]: cluster 2024-04-07T17:10:05.397621+0000 mgr.y (mgr.25175) 1642 : cluster [DBG] pgmap v2826: 329 pgs: 329 active+clean; 4.6 MiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 365 KiB/s rd, 24 KiB/s wr, 435 op/s 2024-04-07T17:10:07.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:06 smithi144 bash[66654]: cluster 2024-04-07T17:10:05.799231+0000 mon.a (mon.0) 3000 : cluster [DBG] osdmap e1610: 8 total, 8 up, 8 in 2024-04-07T17:10:07.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:06 smithi100 bash[74942]: cluster 2024-04-07T17:10:05.397621+0000 mgr.y (mgr.25175) 1642 : cluster [DBG] pgmap v2826: 329 pgs: 329 active+clean; 4.6 MiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 365 KiB/s rd, 24 KiB/s wr, 435 op/s 2024-04-07T17:10:07.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:06 smithi100 bash[74942]: cluster 2024-04-07T17:10:05.799231+0000 mon.a (mon.0) 3000 : cluster [DBG] osdmap e1610: 8 total, 8 up, 8 in 2024-04-07T17:10:07.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:06 smithi100 bash[76202]: cluster 2024-04-07T17:10:05.397621+0000 mgr.y (mgr.25175) 1642 : cluster [DBG] pgmap v2826: 329 pgs: 329 active+clean; 4.6 MiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 365 KiB/s rd, 24 KiB/s wr, 435 op/s 2024-04-07T17:10:07.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:06 smithi100 bash[76202]: cluster 2024-04-07T17:10:05.799231+0000 mon.a (mon.0) 3000 : cluster [DBG] osdmap e1610: 8 total, 8 up, 8 in 2024-04-07T17:10:07.774 INFO:tasks.workunit.client.0.smithi100.stdout:num snaps is: 1 2024-04-07T17:10:07.774 INFO:tasks.workunit.client.0.smithi100.stdout:expected: 1 2024-04-07T17:10:07.774 INFO:tasks.workunit.client.0.smithi100.stdout:snap: snap1 2024-04-07T17:10:07.774 INFO:tasks.workunit.client.0.smithi100.stdout:found snap1 with size 2097152 2024-04-07T17:10:08.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:07 smithi144 bash[66654]: cluster 2024-04-07T17:10:06.800573+0000 mon.a (mon.0) 3001 : cluster [DBG] osdmap e1611: 8 total, 8 up, 8 in 2024-04-07T17:10:08.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:07 smithi144 bash[66654]: cluster 2024-04-07T17:10:07.732145+0000 mon.a (mon.0) 3002 : cluster [DBG] osdmap e1612: 8 total, 8 up, 8 in 2024-04-07T17:10:08.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:07 smithi100 bash[74942]: cluster 2024-04-07T17:10:06.800573+0000 mon.a (mon.0) 3001 : cluster [DBG] osdmap e1611: 8 total, 8 up, 8 in 2024-04-07T17:10:08.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:07 smithi100 bash[74942]: cluster 2024-04-07T17:10:07.732145+0000 mon.a (mon.0) 3002 : cluster [DBG] osdmap e1612: 8 total, 8 up, 8 in 2024-04-07T17:10:08.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:07 smithi100 bash[76202]: cluster 2024-04-07T17:10:06.800573+0000 mon.a (mon.0) 3001 : cluster [DBG] osdmap e1611: 8 total, 8 up, 8 in 2024-04-07T17:10:08.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:07 smithi100 bash[76202]: cluster 2024-04-07T17:10:07.732145+0000 mon.a (mon.0) 3002 : cluster [DBG] osdmap e1612: 8 total, 8 up, 8 in 2024-04-07T17:10:08.703 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:10:08 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:10:08] "GET /metrics HTTP/1.1" 200 50966 "" "Prometheus/2.43.0" 2024-04-07T17:10:08.768 INFO:tasks.workunit.client.0.smithi100.stdout:num snaps is: 2 2024-04-07T17:10:08.768 INFO:tasks.workunit.client.0.smithi100.stdout:expected: 2 2024-04-07T17:10:08.768 INFO:tasks.workunit.client.0.smithi100.stdout:snap: snap1 2024-04-07T17:10:08.768 INFO:tasks.workunit.client.0.smithi100.stdout:snap: snap2 2024-04-07T17:10:08.768 INFO:tasks.workunit.client.0.smithi100.stdout:found snap1 with size 2097152 2024-04-07T17:10:08.768 INFO:tasks.workunit.client.0.smithi100.stdout:found snap2 with size 4194304 2024-04-07T17:10:09.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:08 smithi144 bash[66654]: cluster 2024-04-07T17:10:07.399758+0000 mgr.y (mgr.25175) 1643 : cluster [DBG] pgmap v2829: 329 pgs: 329 active+clean; 4.6 MiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 61 KiB/s rd, 5.2 KiB/s wr, 79 op/s 2024-04-07T17:10:09.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:08 smithi144 bash[66654]: cluster 2024-04-07T17:10:08.730884+0000 mon.a (mon.0) 3003 : cluster [DBG] osdmap e1613: 8 total, 8 up, 8 in 2024-04-07T17:10:09.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:08 smithi100 bash[74942]: cluster 2024-04-07T17:10:07.399758+0000 mgr.y (mgr.25175) 1643 : cluster [DBG] pgmap v2829: 329 pgs: 329 active+clean; 4.6 MiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 61 KiB/s rd, 5.2 KiB/s wr, 79 op/s 2024-04-07T17:10:09.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:08 smithi100 bash[74942]: cluster 2024-04-07T17:10:08.730884+0000 mon.a (mon.0) 3003 : cluster [DBG] osdmap e1613: 8 total, 8 up, 8 in 2024-04-07T17:10:09.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:08 smithi100 bash[76202]: cluster 2024-04-07T17:10:07.399758+0000 mgr.y (mgr.25175) 1643 : cluster [DBG] pgmap v2829: 329 pgs: 329 active+clean; 4.6 MiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 61 KiB/s rd, 5.2 KiB/s wr, 79 op/s 2024-04-07T17:10:09.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:08 smithi100 bash[76202]: cluster 2024-04-07T17:10:08.730884+0000 mon.a (mon.0) 3003 : cluster [DBG] osdmap e1613: 8 total, 8 up, 8 in 2024-04-07T17:10:09.773 INFO:tasks.workunit.client.0.smithi100.stdout:num snaps is: 1 2024-04-07T17:10:09.773 INFO:tasks.workunit.client.0.smithi100.stdout:expected: 1 2024-04-07T17:10:09.773 INFO:tasks.workunit.client.0.smithi100.stdout:snap: snap2 2024-04-07T17:10:09.773 INFO:tasks.workunit.client.0.smithi100.stdout:found snap2 with size 4194304 2024-04-07T17:10:10.788 INFO:tasks.workunit.client.0.smithi100.stdout:num snaps is: 0 2024-04-07T17:10:10.788 INFO:tasks.workunit.client.0.smithi100.stdout:expected: 0 2024-04-07T17:10:10.795 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.TestCreateLsDeleteSnapPP (3934 ms) 2024-04-07T17:10:10.795 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.TestGetNameIdSnapPP 2024-04-07T17:10:10.796 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:10:11.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:10 smithi144 bash[66654]: cluster 2024-04-07T17:10:09.401187+0000 mgr.y (mgr.25175) 1644 : cluster [DBG] pgmap v2832: 329 pgs: 329 active+clean; 4.6 MiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 74 KiB/s rd, 8.5 KiB/s wr, 102 op/s 2024-04-07T17:10:11.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:10 smithi144 bash[66654]: cluster 2024-04-07T17:10:09.732977+0000 mon.a (mon.0) 3004 : cluster [DBG] osdmap e1614: 8 total, 8 up, 8 in 2024-04-07T17:10:11.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:10 smithi100 bash[74942]: cluster 2024-04-07T17:10:09.401187+0000 mgr.y (mgr.25175) 1644 : cluster [DBG] pgmap v2832: 329 pgs: 329 active+clean; 4.6 MiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 74 KiB/s rd, 8.5 KiB/s wr, 102 op/s 2024-04-07T17:10:11.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:10 smithi100 bash[74942]: cluster 2024-04-07T17:10:09.732977+0000 mon.a (mon.0) 3004 : cluster [DBG] osdmap e1614: 8 total, 8 up, 8 in 2024-04-07T17:10:11.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:10 smithi100 bash[76202]: cluster 2024-04-07T17:10:09.401187+0000 mgr.y (mgr.25175) 1644 : cluster [DBG] pgmap v2832: 329 pgs: 329 active+clean; 4.6 MiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 74 KiB/s rd, 8.5 KiB/s wr, 102 op/s 2024-04-07T17:10:11.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:10 smithi100 bash[76202]: cluster 2024-04-07T17:10:09.732977+0000 mon.a (mon.0) 3004 : cluster [DBG] osdmap e1614: 8 total, 8 up, 8 in 2024-04-07T17:10:12.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:11 smithi144 bash[66654]: cluster 2024-04-07T17:10:10.750959+0000 mon.a (mon.0) 3005 : cluster [DBG] osdmap e1615: 8 total, 8 up, 8 in 2024-04-07T17:10:12.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:11 smithi100 bash[74942]: cluster 2024-04-07T17:10:10.750959+0000 mon.a (mon.0) 3005 : cluster [DBG] osdmap e1615: 8 total, 8 up, 8 in 2024-04-07T17:10:12.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:11 smithi100 bash[76202]: cluster 2024-04-07T17:10:10.750959+0000 mon.a (mon.0) 3005 : cluster [DBG] osdmap e1615: 8 total, 8 up, 8 in 2024-04-07T17:10:12.616 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:10:12 smithi144 bash[95233]: ts=2024-04-07T17:10:12.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:10:13.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:12 smithi144 bash[66654]: cluster 2024-04-07T17:10:11.402021+0000 mgr.y (mgr.25175) 1645 : cluster [DBG] pgmap v2835: 329 pgs: 329 active+clean; 4.6 MiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 49 KiB/s rd, 5.7 KiB/s wr, 67 op/s 2024-04-07T17:10:13.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:12 smithi144 bash[66654]: cluster 2024-04-07T17:10:11.755982+0000 mon.a (mon.0) 3006 : cluster [DBG] osdmap e1616: 8 total, 8 up, 8 in 2024-04-07T17:10:13.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:12 smithi144 bash[66654]: cluster 2024-04-07T17:10:12.729457+0000 mon.a (mon.0) 3007 : cluster [DBG] osdmap e1617: 8 total, 8 up, 8 in 2024-04-07T17:10:13.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:12 smithi100 bash[74942]: cluster 2024-04-07T17:10:11.402021+0000 mgr.y (mgr.25175) 1645 : cluster [DBG] pgmap v2835: 329 pgs: 329 active+clean; 4.6 MiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 49 KiB/s rd, 5.7 KiB/s wr, 67 op/s 2024-04-07T17:10:13.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:12 smithi100 bash[74942]: cluster 2024-04-07T17:10:11.755982+0000 mon.a (mon.0) 3006 : cluster [DBG] osdmap e1616: 8 total, 8 up, 8 in 2024-04-07T17:10:13.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:12 smithi100 bash[74942]: cluster 2024-04-07T17:10:12.729457+0000 mon.a (mon.0) 3007 : cluster [DBG] osdmap e1617: 8 total, 8 up, 8 in 2024-04-07T17:10:13.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:12 smithi100 bash[76202]: cluster 2024-04-07T17:10:11.402021+0000 mgr.y (mgr.25175) 1645 : cluster [DBG] pgmap v2835: 329 pgs: 329 active+clean; 4.6 MiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 49 KiB/s rd, 5.7 KiB/s wr, 67 op/s 2024-04-07T17:10:13.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:12 smithi100 bash[76202]: cluster 2024-04-07T17:10:11.755982+0000 mon.a (mon.0) 3006 : cluster [DBG] osdmap e1616: 8 total, 8 up, 8 in 2024-04-07T17:10:13.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:12 smithi100 bash[76202]: cluster 2024-04-07T17:10:12.729457+0000 mon.a (mon.0) 3007 : cluster [DBG] osdmap e1617: 8 total, 8 up, 8 in 2024-04-07T17:10:15.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:14 smithi144 bash[66654]: cluster 2024-04-07T17:10:13.403294+0000 mgr.y (mgr.25175) 1646 : cluster [DBG] pgmap v2838: 329 pgs: 329 active+clean; 4.6 MiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 4.5 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-04-07T17:10:15.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:14 smithi144 bash[66654]: cluster 2024-04-07T17:10:13.769439+0000 mon.a (mon.0) 3008 : cluster [DBG] osdmap e1618: 8 total, 8 up, 8 in 2024-04-07T17:10:15.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:14 smithi100 bash[74942]: cluster 2024-04-07T17:10:13.403294+0000 mgr.y (mgr.25175) 1646 : cluster [DBG] pgmap v2838: 329 pgs: 329 active+clean; 4.6 MiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 4.5 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-04-07T17:10:15.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:14 smithi100 bash[74942]: cluster 2024-04-07T17:10:13.769439+0000 mon.a (mon.0) 3008 : cluster [DBG] osdmap e1618: 8 total, 8 up, 8 in 2024-04-07T17:10:15.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:14 smithi100 bash[76202]: cluster 2024-04-07T17:10:13.403294+0000 mgr.y (mgr.25175) 1646 : cluster [DBG] pgmap v2838: 329 pgs: 329 active+clean; 4.6 MiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 4.5 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-04-07T17:10:15.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:14 smithi100 bash[76202]: cluster 2024-04-07T17:10:13.769439+0000 mon.a (mon.0) 3008 : cluster [DBG] osdmap e1618: 8 total, 8 up, 8 in 2024-04-07T17:10:16.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:15 smithi144 bash[66654]: cluster 2024-04-07T17:10:14.778408+0000 mon.a (mon.0) 3009 : cluster [DBG] osdmap e1619: 8 total, 8 up, 8 in 2024-04-07T17:10:16.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:15 smithi144 bash[66654]: audit 2024-04-07T17:10:15.449953+0000 mon.a (mon.0) 3010 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:10:16.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:15 smithi100 bash[74942]: cluster 2024-04-07T17:10:14.778408+0000 mon.a (mon.0) 3009 : cluster [DBG] osdmap e1619: 8 total, 8 up, 8 in 2024-04-07T17:10:16.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:15 smithi100 bash[74942]: audit 2024-04-07T17:10:15.449953+0000 mon.a (mon.0) 3010 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:10:16.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:15 smithi100 bash[76202]: cluster 2024-04-07T17:10:14.778408+0000 mon.a (mon.0) 3009 : cluster [DBG] osdmap e1619: 8 total, 8 up, 8 in 2024-04-07T17:10:16.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:15 smithi100 bash[76202]: audit 2024-04-07T17:10:15.449953+0000 mon.a (mon.0) 3010 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:10:16.842 INFO:tasks.workunit.client.0.smithi100.stdout:num snaps is: 0 2024-04-07T17:10:16.843 INFO:tasks.workunit.client.0.smithi100.stdout:expected: 0 2024-04-07T17:10:16.857 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.TestGetNameIdSnapPP (6061 ms) 2024-04-07T17:10:16.857 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.TestCreateLsRenameSnapPP 2024-04-07T17:10:16.857 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:10:17.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:16 smithi144 bash[66654]: cluster 2024-04-07T17:10:15.404533+0000 mgr.y (mgr.25175) 1647 : cluster [DBG] pgmap v2841: 329 pgs: 329 active+clean; 4.6 MiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 54 KiB/s rd, 5.7 KiB/s wr, 73 op/s 2024-04-07T17:10:17.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:16 smithi144 bash[66654]: cluster 2024-04-07T17:10:15.789883+0000 mon.a (mon.0) 3011 : cluster [DBG] osdmap e1620: 8 total, 8 up, 8 in 2024-04-07T17:10:17.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:16 smithi100 bash[74942]: cluster 2024-04-07T17:10:15.404533+0000 mgr.y (mgr.25175) 1647 : cluster [DBG] pgmap v2841: 329 pgs: 329 active+clean; 4.6 MiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 54 KiB/s rd, 5.7 KiB/s wr, 73 op/s 2024-04-07T17:10:17.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:16 smithi100 bash[74942]: cluster 2024-04-07T17:10:15.789883+0000 mon.a (mon.0) 3011 : cluster [DBG] osdmap e1620: 8 total, 8 up, 8 in 2024-04-07T17:10:17.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:16 smithi100 bash[76202]: cluster 2024-04-07T17:10:15.404533+0000 mgr.y (mgr.25175) 1647 : cluster [DBG] pgmap v2841: 329 pgs: 329 active+clean; 4.6 MiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 54 KiB/s rd, 5.7 KiB/s wr, 73 op/s 2024-04-07T17:10:17.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:16 smithi100 bash[76202]: cluster 2024-04-07T17:10:15.789883+0000 mon.a (mon.0) 3011 : cluster [DBG] osdmap e1620: 8 total, 8 up, 8 in 2024-04-07T17:10:17.783 INFO:tasks.workunit.client.0.smithi100.stdout:num snaps is: 1 2024-04-07T17:10:17.783 INFO:tasks.workunit.client.0.smithi100.stdout:expected: 1 2024-04-07T17:10:17.783 INFO:tasks.workunit.client.0.smithi100.stdout:snap: snap1 2024-04-07T17:10:17.784 INFO:tasks.workunit.client.0.smithi100.stdout:found snap1 with size 2097152 2024-04-07T17:10:18.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:17 smithi144 bash[66654]: cluster 2024-04-07T17:10:16.799075+0000 mon.a (mon.0) 3012 : cluster [DBG] osdmap e1621: 8 total, 8 up, 8 in 2024-04-07T17:10:18.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:17 smithi144 bash[66654]: audit 2024-04-07T17:10:16.817467+0000 mon.a (mon.0) 3013 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:10:18.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:17 smithi144 bash[66654]: cluster 2024-04-07T17:10:17.732043+0000 mon.a (mon.0) 3014 : cluster [DBG] osdmap e1622: 8 total, 8 up, 8 in 2024-04-07T17:10:18.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:17 smithi100 bash[74942]: cluster 2024-04-07T17:10:16.799075+0000 mon.a (mon.0) 3012 : cluster [DBG] osdmap e1621: 8 total, 8 up, 8 in 2024-04-07T17:10:18.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:17 smithi100 bash[74942]: audit 2024-04-07T17:10:16.817467+0000 mon.a (mon.0) 3013 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:10:18.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:17 smithi100 bash[74942]: cluster 2024-04-07T17:10:17.732043+0000 mon.a (mon.0) 3014 : cluster [DBG] osdmap e1622: 8 total, 8 up, 8 in 2024-04-07T17:10:18.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:17 smithi100 bash[76202]: cluster 2024-04-07T17:10:16.799075+0000 mon.a (mon.0) 3012 : cluster [DBG] osdmap e1621: 8 total, 8 up, 8 in 2024-04-07T17:10:18.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:17 smithi100 bash[76202]: audit 2024-04-07T17:10:16.817467+0000 mon.a (mon.0) 3013 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:10:18.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:17 smithi100 bash[76202]: cluster 2024-04-07T17:10:17.732043+0000 mon.a (mon.0) 3014 : cluster [DBG] osdmap e1622: 8 total, 8 up, 8 in 2024-04-07T17:10:18.703 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:10:18 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:10:18] "GET /metrics HTTP/1.1" 200 50981 "" "Prometheus/2.43.0" 2024-04-07T17:10:18.848 INFO:tasks.workunit.client.0.smithi100.stdout:num snaps is: 2 2024-04-07T17:10:18.848 INFO:tasks.workunit.client.0.smithi100.stdout:expected: 2 2024-04-07T17:10:18.848 INFO:tasks.workunit.client.0.smithi100.stdout:snap: snap1 2024-04-07T17:10:18.848 INFO:tasks.workunit.client.0.smithi100.stdout:snap: snap2 2024-04-07T17:10:18.848 INFO:tasks.workunit.client.0.smithi100.stdout:found snap1 with size 2097152 2024-04-07T17:10:18.849 INFO:tasks.workunit.client.0.smithi100.stdout:found snap2 with size 4194304 2024-04-07T17:10:18.862 INFO:tasks.workunit.client.0.smithi100.stdout:num snaps is: 2 2024-04-07T17:10:18.863 INFO:tasks.workunit.client.0.smithi100.stdout:expected: 2 2024-04-07T17:10:18.863 INFO:tasks.workunit.client.0.smithi100.stdout:snap: snap1-rename 2024-04-07T17:10:18.863 INFO:tasks.workunit.client.0.smithi100.stdout:snap: snap2 2024-04-07T17:10:18.863 INFO:tasks.workunit.client.0.smithi100.stdout:found snap1-rename with size 2097152 2024-04-07T17:10:18.863 INFO:tasks.workunit.client.0.smithi100.stdout:found snap2 with size 4194304 2024-04-07T17:10:19.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:18 smithi144 bash[66654]: cluster 2024-04-07T17:10:17.406602+0000 mgr.y (mgr.25175) 1648 : cluster [DBG] pgmap v2844: 329 pgs: 329 active+clean; 4.6 MiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 55 KiB/s rd, 5.7 KiB/s wr, 75 op/s 2024-04-07T17:10:19.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:18 smithi144 bash[66654]: cluster 2024-04-07T17:10:18.807452+0000 mon.a (mon.0) 3015 : cluster [DBG] osdmap e1623: 8 total, 8 up, 8 in 2024-04-07T17:10:19.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:18 smithi100 bash[74942]: cluster 2024-04-07T17:10:17.406602+0000 mgr.y (mgr.25175) 1648 : cluster [DBG] pgmap v2844: 329 pgs: 329 active+clean; 4.6 MiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 55 KiB/s rd, 5.7 KiB/s wr, 75 op/s 2024-04-07T17:10:19.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:18 smithi100 bash[74942]: cluster 2024-04-07T17:10:18.807452+0000 mon.a (mon.0) 3015 : cluster [DBG] osdmap e1623: 8 total, 8 up, 8 in 2024-04-07T17:10:19.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:18 smithi100 bash[76202]: cluster 2024-04-07T17:10:17.406602+0000 mgr.y (mgr.25175) 1648 : cluster [DBG] pgmap v2844: 329 pgs: 329 active+clean; 4.6 MiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 55 KiB/s rd, 5.7 KiB/s wr, 75 op/s 2024-04-07T17:10:19.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:18 smithi100 bash[76202]: cluster 2024-04-07T17:10:18.807452+0000 mon.a (mon.0) 3015 : cluster [DBG] osdmap e1623: 8 total, 8 up, 8 in 2024-04-07T17:10:19.907 INFO:tasks.workunit.client.0.smithi100.stdout:num snaps is: 1 2024-04-07T17:10:19.907 INFO:tasks.workunit.client.0.smithi100.stdout:expected: 1 2024-04-07T17:10:19.907 INFO:tasks.workunit.client.0.smithi100.stdout:snap: snap2-rename 2024-04-07T17:10:19.907 INFO:tasks.workunit.client.0.smithi100.stdout:found snap2-rename with size 4194304 2024-04-07T17:10:20.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:19 smithi144 bash[66654]: audit 2024-04-07T17:10:19.792265+0000 mon.a (mon.0) 3016 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "93.1", "id": [1, 4]}]: dispatch 2024-04-07T17:10:20.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:19 smithi144 bash[66654]: audit 2024-04-07T17:10:19.792783+0000 mon.a (mon.0) 3017 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "93.9", "id": [2, 4]}]: dispatch 2024-04-07T17:10:20.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:19 smithi144 bash[66654]: audit 2024-04-07T17:10:19.793213+0000 mon.a (mon.0) 3018 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "93.10", "id": [2, 4]}]: dispatch 2024-04-07T17:10:20.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:19 smithi144 bash[66654]: audit 2024-04-07T17:10:19.793645+0000 mon.a (mon.0) 3019 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "93.13", "id": [2, 4]}]: dispatch 2024-04-07T17:10:20.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:19 smithi144 bash[66654]: audit 2024-04-07T17:10:19.794056+0000 mon.a (mon.0) 3020 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "93.16", "id": [2, 3]}]: dispatch 2024-04-07T17:10:20.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:19 smithi144 bash[66654]: audit 2024-04-07T17:10:19.794490+0000 mon.a (mon.0) 3021 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "95.0", "id": [6, 4]}]: dispatch 2024-04-07T17:10:20.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:19 smithi144 bash[66654]: audit 2024-04-07T17:10:19.794893+0000 mon.a (mon.0) 3022 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "95.17", "id": [1, 2]}]: dispatch 2024-04-07T17:10:20.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:19 smithi144 bash[66654]: audit 2024-04-07T17:10:19.795302+0000 mon.a (mon.0) 3023 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "96.15", "id": [0, 3]}]: dispatch 2024-04-07T17:10:20.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:19 smithi144 bash[66654]: audit 2024-04-07T17:10:19.795757+0000 mon.a (mon.0) 3024 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "97.5", "id": [0, 2]}]: dispatch 2024-04-07T17:10:20.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:19 smithi144 bash[66654]: audit 2024-04-07T17:10:19.796170+0000 mon.a (mon.0) 3025 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "97.b", "id": [0, 7]}]: dispatch 2024-04-07T17:10:20.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:19 smithi100 bash[74942]: audit 2024-04-07T17:10:19.792265+0000 mon.a (mon.0) 3016 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "93.1", "id": [1, 4]}]: dispatch 2024-04-07T17:10:20.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:19 smithi100 bash[74942]: audit 2024-04-07T17:10:19.792783+0000 mon.a (mon.0) 3017 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "93.9", "id": [2, 4]}]: dispatch 2024-04-07T17:10:20.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:19 smithi100 bash[74942]: audit 2024-04-07T17:10:19.793213+0000 mon.a (mon.0) 3018 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "93.10", "id": [2, 4]}]: dispatch 2024-04-07T17:10:20.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:19 smithi100 bash[74942]: audit 2024-04-07T17:10:19.793645+0000 mon.a (mon.0) 3019 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "93.13", "id": [2, 4]}]: dispatch 2024-04-07T17:10:20.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:19 smithi100 bash[74942]: audit 2024-04-07T17:10:19.794056+0000 mon.a (mon.0) 3020 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "93.16", "id": [2, 3]}]: dispatch 2024-04-07T17:10:20.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:19 smithi100 bash[74942]: audit 2024-04-07T17:10:19.794490+0000 mon.a (mon.0) 3021 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "95.0", "id": [6, 4]}]: dispatch 2024-04-07T17:10:20.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:19 smithi100 bash[74942]: audit 2024-04-07T17:10:19.794893+0000 mon.a (mon.0) 3022 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "95.17", "id": [1, 2]}]: dispatch 2024-04-07T17:10:20.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:19 smithi100 bash[74942]: audit 2024-04-07T17:10:19.795302+0000 mon.a (mon.0) 3023 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "96.15", "id": [0, 3]}]: dispatch 2024-04-07T17:10:20.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:19 smithi100 bash[74942]: audit 2024-04-07T17:10:19.795757+0000 mon.a (mon.0) 3024 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "97.5", "id": [0, 2]}]: dispatch 2024-04-07T17:10:20.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:19 smithi100 bash[74942]: audit 2024-04-07T17:10:19.796170+0000 mon.a (mon.0) 3025 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "97.b", "id": [0, 7]}]: dispatch 2024-04-07T17:10:20.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:19 smithi100 bash[76202]: audit 2024-04-07T17:10:19.792265+0000 mon.a (mon.0) 3016 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "93.1", "id": [1, 4]}]: dispatch 2024-04-07T17:10:20.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:19 smithi100 bash[76202]: audit 2024-04-07T17:10:19.792783+0000 mon.a (mon.0) 3017 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "93.9", "id": [2, 4]}]: dispatch 2024-04-07T17:10:20.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:19 smithi100 bash[76202]: audit 2024-04-07T17:10:19.793213+0000 mon.a (mon.0) 3018 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "93.10", "id": [2, 4]}]: dispatch 2024-04-07T17:10:20.205 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:19 smithi100 bash[76202]: audit 2024-04-07T17:10:19.793645+0000 mon.a (mon.0) 3019 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "93.13", "id": [2, 4]}]: dispatch 2024-04-07T17:10:20.205 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:19 smithi100 bash[76202]: audit 2024-04-07T17:10:19.794056+0000 mon.a (mon.0) 3020 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "93.16", "id": [2, 3]}]: dispatch 2024-04-07T17:10:20.205 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:19 smithi100 bash[76202]: audit 2024-04-07T17:10:19.794490+0000 mon.a (mon.0) 3021 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "95.0", "id": [6, 4]}]: dispatch 2024-04-07T17:10:20.205 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:19 smithi100 bash[76202]: audit 2024-04-07T17:10:19.794893+0000 mon.a (mon.0) 3022 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "95.17", "id": [1, 2]}]: dispatch 2024-04-07T17:10:20.205 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:19 smithi100 bash[76202]: audit 2024-04-07T17:10:19.795302+0000 mon.a (mon.0) 3023 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "96.15", "id": [0, 3]}]: dispatch 2024-04-07T17:10:20.205 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:19 smithi100 bash[76202]: audit 2024-04-07T17:10:19.795757+0000 mon.a (mon.0) 3024 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "97.5", "id": [0, 2]}]: dispatch 2024-04-07T17:10:20.205 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:19 smithi100 bash[76202]: audit 2024-04-07T17:10:19.796170+0000 mon.a (mon.0) 3025 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "97.b", "id": [0, 7]}]: dispatch 2024-04-07T17:10:20.870 INFO:tasks.workunit.client.0.smithi100.stdout:num snaps is: 0 2024-04-07T17:10:20.870 INFO:tasks.workunit.client.0.smithi100.stdout:expected: 0 2024-04-07T17:10:20.874 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.TestCreateLsRenameSnapPP (4017 ms) 2024-04-07T17:10:20.874 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.ConcurrentCreatesUnvalidatedPool 2024-04-07T17:10:21.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:20 smithi144 bash[66654]: cluster 2024-04-07T17:10:19.408111+0000 mgr.y (mgr.25175) 1649 : cluster [DBG] pgmap v2847: 329 pgs: 329 active+clean; 4.6 MiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 6.2 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2024-04-07T17:10:21.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:20 smithi144 bash[66654]: audit 2024-04-07T17:10:19.825493+0000 mon.a (mon.0) 3026 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "93.1", "id": [1, 4]}]': finished 2024-04-07T17:10:21.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:20 smithi144 bash[66654]: audit 2024-04-07T17:10:19.825956+0000 mon.a (mon.0) 3027 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "93.9", "id": [2, 4]}]': finished 2024-04-07T17:10:21.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:20 smithi144 bash[66654]: audit 2024-04-07T17:10:19.826074+0000 mon.a (mon.0) 3028 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "93.10", "id": [2, 4]}]': finished 2024-04-07T17:10:21.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:20 smithi144 bash[66654]: audit 2024-04-07T17:10:19.826217+0000 mon.a (mon.0) 3029 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "93.13", "id": [2, 4]}]': finished 2024-04-07T17:10:21.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:20 smithi144 bash[66654]: audit 2024-04-07T17:10:19.826412+0000 mon.a (mon.0) 3030 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "93.16", "id": [2, 3]}]': finished 2024-04-07T17:10:21.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:20 smithi144 bash[66654]: audit 2024-04-07T17:10:19.826594+0000 mon.a (mon.0) 3031 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "95.0", "id": [6, 4]}]': finished 2024-04-07T17:10:21.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:20 smithi144 bash[66654]: audit 2024-04-07T17:10:19.826951+0000 mon.a (mon.0) 3032 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "95.17", "id": [1, 2]}]': finished 2024-04-07T17:10:21.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:20 smithi144 bash[66654]: audit 2024-04-07T17:10:19.827153+0000 mon.a (mon.0) 3033 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "96.15", "id": [0, 3]}]': finished 2024-04-07T17:10:21.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:20 smithi144 bash[66654]: audit 2024-04-07T17:10:19.827347+0000 mon.a (mon.0) 3034 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "97.5", "id": [0, 2]}]': finished 2024-04-07T17:10:21.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:20 smithi144 bash[66654]: audit 2024-04-07T17:10:19.827685+0000 mon.a (mon.0) 3035 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "97.b", "id": [0, 7]}]': finished 2024-04-07T17:10:21.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:20 smithi144 bash[66654]: cluster 2024-04-07T17:10:19.836237+0000 mon.a (mon.0) 3036 : cluster [DBG] osdmap e1624: 8 total, 8 up, 8 in 2024-04-07T17:10:21.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:20 smithi100 bash[74942]: cluster 2024-04-07T17:10:19.408111+0000 mgr.y (mgr.25175) 1649 : cluster [DBG] pgmap v2847: 329 pgs: 329 active+clean; 4.6 MiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 6.2 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2024-04-07T17:10:21.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:20 smithi100 bash[74942]: audit 2024-04-07T17:10:19.825493+0000 mon.a (mon.0) 3026 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "93.1", "id": [1, 4]}]': finished 2024-04-07T17:10:21.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:20 smithi100 bash[74942]: audit 2024-04-07T17:10:19.825956+0000 mon.a (mon.0) 3027 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "93.9", "id": [2, 4]}]': finished 2024-04-07T17:10:21.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:20 smithi100 bash[74942]: audit 2024-04-07T17:10:19.826074+0000 mon.a (mon.0) 3028 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "93.10", "id": [2, 4]}]': finished 2024-04-07T17:10:21.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:20 smithi100 bash[74942]: audit 2024-04-07T17:10:19.826217+0000 mon.a (mon.0) 3029 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "93.13", "id": [2, 4]}]': finished 2024-04-07T17:10:21.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:20 smithi100 bash[74942]: audit 2024-04-07T17:10:19.826412+0000 mon.a (mon.0) 3030 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "93.16", "id": [2, 3]}]': finished 2024-04-07T17:10:21.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:20 smithi100 bash[74942]: audit 2024-04-07T17:10:19.826594+0000 mon.a (mon.0) 3031 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "95.0", "id": [6, 4]}]': finished 2024-04-07T17:10:21.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:20 smithi100 bash[74942]: audit 2024-04-07T17:10:19.826951+0000 mon.a (mon.0) 3032 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "95.17", "id": [1, 2]}]': finished 2024-04-07T17:10:21.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:20 smithi100 bash[74942]: audit 2024-04-07T17:10:19.827153+0000 mon.a (mon.0) 3033 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "96.15", "id": [0, 3]}]': finished 2024-04-07T17:10:21.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:20 smithi100 bash[74942]: audit 2024-04-07T17:10:19.827347+0000 mon.a (mon.0) 3034 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "97.5", "id": [0, 2]}]': finished 2024-04-07T17:10:21.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:20 smithi100 bash[74942]: audit 2024-04-07T17:10:19.827685+0000 mon.a (mon.0) 3035 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "97.b", "id": [0, 7]}]': finished 2024-04-07T17:10:21.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:20 smithi100 bash[74942]: cluster 2024-04-07T17:10:19.836237+0000 mon.a (mon.0) 3036 : cluster [DBG] osdmap e1624: 8 total, 8 up, 8 in 2024-04-07T17:10:21.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:20 smithi100 bash[76202]: cluster 2024-04-07T17:10:19.408111+0000 mgr.y (mgr.25175) 1649 : cluster [DBG] pgmap v2847: 329 pgs: 329 active+clean; 4.6 MiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 6.2 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2024-04-07T17:10:21.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:20 smithi100 bash[76202]: audit 2024-04-07T17:10:19.825493+0000 mon.a (mon.0) 3026 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "93.1", "id": [1, 4]}]': finished 2024-04-07T17:10:21.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:20 smithi100 bash[76202]: audit 2024-04-07T17:10:19.825956+0000 mon.a (mon.0) 3027 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "93.9", "id": [2, 4]}]': finished 2024-04-07T17:10:21.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:20 smithi100 bash[76202]: audit 2024-04-07T17:10:19.826074+0000 mon.a (mon.0) 3028 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "93.10", "id": [2, 4]}]': finished 2024-04-07T17:10:21.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:20 smithi100 bash[76202]: audit 2024-04-07T17:10:19.826217+0000 mon.a (mon.0) 3029 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "93.13", "id": [2, 4]}]': finished 2024-04-07T17:10:21.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:20 smithi100 bash[76202]: audit 2024-04-07T17:10:19.826412+0000 mon.a (mon.0) 3030 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "93.16", "id": [2, 3]}]': finished 2024-04-07T17:10:21.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:20 smithi100 bash[76202]: audit 2024-04-07T17:10:19.826594+0000 mon.a (mon.0) 3031 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "95.0", "id": [6, 4]}]': finished 2024-04-07T17:10:21.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:20 smithi100 bash[76202]: audit 2024-04-07T17:10:19.826951+0000 mon.a (mon.0) 3032 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "95.17", "id": [1, 2]}]': finished 2024-04-07T17:10:21.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:20 smithi100 bash[76202]: audit 2024-04-07T17:10:19.827153+0000 mon.a (mon.0) 3033 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "96.15", "id": [0, 3]}]': finished 2024-04-07T17:10:21.205 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:20 smithi100 bash[76202]: audit 2024-04-07T17:10:19.827347+0000 mon.a (mon.0) 3034 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "97.5", "id": [0, 2]}]': finished 2024-04-07T17:10:21.205 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:20 smithi100 bash[76202]: audit 2024-04-07T17:10:19.827685+0000 mon.a (mon.0) 3035 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "97.b", "id": [0, 7]}]': finished 2024-04-07T17:10:21.205 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:20 smithi100 bash[76202]: cluster 2024-04-07T17:10:19.836237+0000 mon.a (mon.0) 3036 : cluster [DBG] osdmap e1624: 8 total, 8 up, 8 in 2024-04-07T17:10:22.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:21 smithi144 bash[66654]: cluster 2024-04-07T17:10:20.831827+0000 mon.a (mon.0) 3037 : cluster [DBG] osdmap e1625: 8 total, 8 up, 8 in 2024-04-07T17:10:22.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:21 smithi144 bash[66654]: audit 2024-04-07T17:10:21.174134+0000 mon.a (mon.0) 3038 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:10:22.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:21 smithi144 bash[66654]: audit 2024-04-07T17:10:21.179839+0000 mon.a (mon.0) 3039 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:10:22.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:21 smithi144 bash[66654]: audit 2024-04-07T17:10:21.434168+0000 mon.a (mon.0) 3040 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:10:22.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:21 smithi144 bash[66654]: audit 2024-04-07T17:10:21.446325+0000 mon.a (mon.0) 3041 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:10:22.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:21 smithi100 bash[74942]: cluster 2024-04-07T17:10:20.831827+0000 mon.a (mon.0) 3037 : cluster [DBG] osdmap e1625: 8 total, 8 up, 8 in 2024-04-07T17:10:22.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:21 smithi100 bash[74942]: audit 2024-04-07T17:10:21.174134+0000 mon.a (mon.0) 3038 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:10:22.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:21 smithi100 bash[74942]: audit 2024-04-07T17:10:21.179839+0000 mon.a (mon.0) 3039 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:10:22.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:21 smithi100 bash[74942]: audit 2024-04-07T17:10:21.434168+0000 mon.a (mon.0) 3040 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:10:22.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:21 smithi100 bash[74942]: audit 2024-04-07T17:10:21.446325+0000 mon.a (mon.0) 3041 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:10:22.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:21 smithi100 bash[76202]: cluster 2024-04-07T17:10:20.831827+0000 mon.a (mon.0) 3037 : cluster [DBG] osdmap e1625: 8 total, 8 up, 8 in 2024-04-07T17:10:22.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:21 smithi100 bash[76202]: audit 2024-04-07T17:10:21.174134+0000 mon.a (mon.0) 3038 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:10:22.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:21 smithi100 bash[76202]: audit 2024-04-07T17:10:21.179839+0000 mon.a (mon.0) 3039 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:10:22.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:21 smithi100 bash[76202]: audit 2024-04-07T17:10:21.434168+0000 mon.a (mon.0) 3040 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:10:22.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:21 smithi100 bash[76202]: audit 2024-04-07T17:10:21.446325+0000 mon.a (mon.0) 3041 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:10:22.616 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:10:22 smithi144 bash[95233]: ts=2024-04-07T17:10:22.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:10:22.751 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:10:22.752 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:10:22.752 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:10:22.752 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:10:23.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:22 smithi144 bash[66654]: cluster 2024-04-07T17:10:21.408988+0000 mgr.y (mgr.25175) 1650 : cluster [DBG] pgmap v2850: 329 pgs: 1 peering, 328 active+clean; 4.6 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 46 KiB/s rd, 4.7 KiB/s wr, 61 op/s 2024-04-07T17:10:23.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:22 smithi144 bash[66654]: cluster 2024-04-07T17:10:21.872520+0000 mon.a (mon.0) 3042 : cluster [DBG] osdmap e1626: 8 total, 8 up, 8 in 2024-04-07T17:10:23.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:22 smithi144 bash[66654]: audit 2024-04-07T17:10:21.880123+0000 mon.a (mon.0) 3043 : audit [INF] from='client.? 172.21.15.100:0/1052846239' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:10:23.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:22 smithi144 bash[66654]: audit 2024-04-07T17:10:22.030384+0000 mon.a (mon.0) 3044 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:10:23.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:22 smithi144 bash[66654]: audit 2024-04-07T17:10:22.032353+0000 mon.a (mon.0) 3045 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:10:23.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:22 smithi144 bash[66654]: audit 2024-04-07T17:10:22.042747+0000 mon.a (mon.0) 3046 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:10:23.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:22 smithi144 bash[66654]: audit 2024-04-07T17:10:22.727797+0000 mon.a (mon.0) 3047 : audit [INF] from='client.? 172.21.15.100:0/1052846239' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T17:10:23.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:22 smithi144 bash[66654]: cluster 2024-04-07T17:10:22.738968+0000 mon.a (mon.0) 3048 : cluster [DBG] osdmap e1627: 8 total, 8 up, 8 in 2024-04-07T17:10:23.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:22 smithi100 bash[74942]: cluster 2024-04-07T17:10:21.408988+0000 mgr.y (mgr.25175) 1650 : cluster [DBG] pgmap v2850: 329 pgs: 1 peering, 328 active+clean; 4.6 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 46 KiB/s rd, 4.7 KiB/s wr, 61 op/s 2024-04-07T17:10:23.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:22 smithi100 bash[74942]: cluster 2024-04-07T17:10:21.872520+0000 mon.a (mon.0) 3042 : cluster [DBG] osdmap e1626: 8 total, 8 up, 8 in 2024-04-07T17:10:23.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:22 smithi100 bash[74942]: audit 2024-04-07T17:10:21.880123+0000 mon.a (mon.0) 3043 : audit [INF] from='client.? 172.21.15.100:0/1052846239' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:10:23.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:22 smithi100 bash[74942]: audit 2024-04-07T17:10:22.030384+0000 mon.a (mon.0) 3044 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:10:23.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:22 smithi100 bash[74942]: audit 2024-04-07T17:10:22.032353+0000 mon.a (mon.0) 3045 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:10:23.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:22 smithi100 bash[74942]: audit 2024-04-07T17:10:22.042747+0000 mon.a (mon.0) 3046 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:10:23.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:22 smithi100 bash[74942]: audit 2024-04-07T17:10:22.727797+0000 mon.a (mon.0) 3047 : audit [INF] from='client.? 172.21.15.100:0/1052846239' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T17:10:23.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:22 smithi100 bash[74942]: cluster 2024-04-07T17:10:22.738968+0000 mon.a (mon.0) 3048 : cluster [DBG] osdmap e1627: 8 total, 8 up, 8 in 2024-04-07T17:10:23.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:22 smithi100 bash[76202]: cluster 2024-04-07T17:10:21.408988+0000 mgr.y (mgr.25175) 1650 : cluster [DBG] pgmap v2850: 329 pgs: 1 peering, 328 active+clean; 4.6 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 46 KiB/s rd, 4.7 KiB/s wr, 61 op/s 2024-04-07T17:10:23.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:22 smithi100 bash[76202]: cluster 2024-04-07T17:10:21.872520+0000 mon.a (mon.0) 3042 : cluster [DBG] osdmap e1626: 8 total, 8 up, 8 in 2024-04-07T17:10:23.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:22 smithi100 bash[76202]: audit 2024-04-07T17:10:21.880123+0000 mon.a (mon.0) 3043 : audit [INF] from='client.? 172.21.15.100:0/1052846239' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:10:23.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:22 smithi100 bash[76202]: audit 2024-04-07T17:10:22.030384+0000 mon.a (mon.0) 3044 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:10:23.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:22 smithi100 bash[76202]: audit 2024-04-07T17:10:22.032353+0000 mon.a (mon.0) 3045 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:10:23.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:22 smithi100 bash[76202]: audit 2024-04-07T17:10:22.042747+0000 mon.a (mon.0) 3046 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:10:23.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:22 smithi100 bash[76202]: audit 2024-04-07T17:10:22.727797+0000 mon.a (mon.0) 3047 : audit [INF] from='client.? 172.21.15.100:0/1052846239' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T17:10:23.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:22 smithi100 bash[76202]: cluster 2024-04-07T17:10:22.738968+0000 mon.a (mon.0) 3048 : cluster [DBG] osdmap e1627: 8 total, 8 up, 8 in 2024-04-07T17:10:25.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:24 smithi144 bash[66654]: cluster 2024-04-07T17:10:23.410295+0000 mgr.y (mgr.25175) 1651 : cluster [DBG] pgmap v2853: 361 pgs: 15 creating+peering, 9 unknown, 1 peering, 336 active+clean; 4.6 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 50 KiB/s rd, 4.5 KiB/s wr, 66 op/s; 8 B/s, 0 objects/s recovering 2024-04-07T17:10:25.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:24 smithi144 bash[66654]: cluster 2024-04-07T17:10:23.742166+0000 mon.a (mon.0) 3049 : cluster [DBG] osdmap e1628: 8 total, 8 up, 8 in 2024-04-07T17:10:25.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:24 smithi100 bash[74942]: cluster 2024-04-07T17:10:23.410295+0000 mgr.y (mgr.25175) 1651 : cluster [DBG] pgmap v2853: 361 pgs: 15 creating+peering, 9 unknown, 1 peering, 336 active+clean; 4.6 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 50 KiB/s rd, 4.5 KiB/s wr, 66 op/s; 8 B/s, 0 objects/s recovering 2024-04-07T17:10:25.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:24 smithi100 bash[74942]: cluster 2024-04-07T17:10:23.742166+0000 mon.a (mon.0) 3049 : cluster [DBG] osdmap e1628: 8 total, 8 up, 8 in 2024-04-07T17:10:25.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:24 smithi100 bash[76202]: cluster 2024-04-07T17:10:23.410295+0000 mgr.y (mgr.25175) 1651 : cluster [DBG] pgmap v2853: 361 pgs: 15 creating+peering, 9 unknown, 1 peering, 336 active+clean; 4.6 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 50 KiB/s rd, 4.5 KiB/s wr, 66 op/s; 8 B/s, 0 objects/s recovering 2024-04-07T17:10:25.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:24 smithi100 bash[76202]: cluster 2024-04-07T17:10:23.742166+0000 mon.a (mon.0) 3049 : cluster [DBG] osdmap e1628: 8 total, 8 up, 8 in 2024-04-07T17:10:25.309 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.ConcurrentCreatesUnvalidatedPool (4435 ms) 2024-04-07T17:10:25.309 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.RemoveFullTry 2024-04-07T17:10:26.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:25 smithi144 bash[66654]: cluster 2024-04-07T17:10:24.748977+0000 mon.a (mon.0) 3050 : cluster [DBG] osdmap e1629: 8 total, 8 up, 8 in 2024-04-07T17:10:26.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:25 smithi100 bash[74942]: cluster 2024-04-07T17:10:24.748977+0000 mon.a (mon.0) 3050 : cluster [DBG] osdmap e1629: 8 total, 8 up, 8 in 2024-04-07T17:10:26.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:25 smithi100 bash[76202]: cluster 2024-04-07T17:10:24.748977+0000 mon.a (mon.0) 3050 : cluster [DBG] osdmap e1629: 8 total, 8 up, 8 in 2024-04-07T17:10:26.834 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:10:27.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:26 smithi144 bash[66654]: cluster 2024-04-07T17:10:25.411489+0000 mgr.y (mgr.25175) 1652 : cluster [DBG] pgmap v2856: 361 pgs: 15 creating+peering, 346 active+clean; 4.6 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 22 KiB/s rd, 2.5 KiB/s wr, 33 op/s; 8 B/s, 0 objects/s recovering 2024-04-07T17:10:27.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:26 smithi144 bash[66654]: cluster 2024-04-07T17:10:25.762758+0000 mon.a (mon.0) 3051 : cluster [DBG] osdmap e1630: 8 total, 8 up, 8 in 2024-04-07T17:10:27.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:26 smithi144 bash[66654]: audit 2024-04-07T17:10:25.768890+0000 mon.c (mon.2) 72 : audit [INF] from='client.? 172.21.15.100:0/2090705731' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:10:27.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:26 smithi144 bash[66654]: audit 2024-04-07T17:10:25.770800+0000 mon.a (mon.0) 3052 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:10:27.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:26 smithi100 bash[74942]: cluster 2024-04-07T17:10:25.411489+0000 mgr.y (mgr.25175) 1652 : cluster [DBG] pgmap v2856: 361 pgs: 15 creating+peering, 346 active+clean; 4.6 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 22 KiB/s rd, 2.5 KiB/s wr, 33 op/s; 8 B/s, 0 objects/s recovering 2024-04-07T17:10:27.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:26 smithi100 bash[74942]: cluster 2024-04-07T17:10:25.762758+0000 mon.a (mon.0) 3051 : cluster [DBG] osdmap e1630: 8 total, 8 up, 8 in 2024-04-07T17:10:27.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:26 smithi100 bash[74942]: audit 2024-04-07T17:10:25.768890+0000 mon.c (mon.2) 72 : audit [INF] from='client.? 172.21.15.100:0/2090705731' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:10:27.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:26 smithi100 bash[74942]: audit 2024-04-07T17:10:25.770800+0000 mon.a (mon.0) 3052 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:10:27.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:26 smithi100 bash[76202]: cluster 2024-04-07T17:10:25.411489+0000 mgr.y (mgr.25175) 1652 : cluster [DBG] pgmap v2856: 361 pgs: 15 creating+peering, 346 active+clean; 4.6 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 22 KiB/s rd, 2.5 KiB/s wr, 33 op/s; 8 B/s, 0 objects/s recovering 2024-04-07T17:10:27.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:26 smithi100 bash[76202]: cluster 2024-04-07T17:10:25.762758+0000 mon.a (mon.0) 3051 : cluster [DBG] osdmap e1630: 8 total, 8 up, 8 in 2024-04-07T17:10:27.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:26 smithi100 bash[76202]: audit 2024-04-07T17:10:25.768890+0000 mon.c (mon.2) 72 : audit [INF] from='client.? 172.21.15.100:0/2090705731' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:10:27.205 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:26 smithi100 bash[76202]: audit 2024-04-07T17:10:25.770800+0000 mon.a (mon.0) 3052 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:10:27.205 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:10:26 smithi100 bash[68137]: [07/Apr/2024:17:10:26] ENGINE Bus STOPPING 2024-04-07T17:10:27.205 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:10:26 smithi100 bash[68137]: [07/Apr/2024:17:10:26] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-04-07T17:10:27.205 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:10:26 smithi100 bash[68137]: [07/Apr/2024:17:10:26] ENGINE Bus STOPPED 2024-04-07T17:10:27.205 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:10:26 smithi100 bash[68137]: [07/Apr/2024:17:10:26] ENGINE Bus STARTING 2024-04-07T17:10:27.205 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:10:26 smithi100 bash[68137]: [07/Apr/2024:17:10:26] ENGINE Serving on http://:::9283 2024-04-07T17:10:27.205 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:10:26 smithi100 bash[68137]: [07/Apr/2024:17:10:26] ENGINE Bus STARTED 2024-04-07T17:10:28.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:27 smithi144 bash[66654]: audit 2024-04-07T17:10:26.755516+0000 mon.a (mon.0) 3053 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T17:10:28.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:27 smithi144 bash[66654]: cluster 2024-04-07T17:10:26.766153+0000 mon.a (mon.0) 3054 : cluster [DBG] osdmap e1631: 8 total, 8 up, 8 in 2024-04-07T17:10:28.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:27 smithi144 bash[66654]: audit 2024-04-07T17:10:26.830875+0000 mon.a (mon.0) 3055 : audit [INF] from='client.? ' entity='client.admin' 2024-04-07T17:10:28.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:27 smithi144 bash[66654]: audit 2024-04-07T17:10:26.864269+0000 mon.a (mon.0) 3056 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:10:28.118 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:27 smithi144 bash[66654]: audit 2024-04-07T17:10:26.866771+0000 mon.a (mon.0) 3057 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:10:28.118 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:27 smithi144 bash[66654]: audit 2024-04-07T17:10:26.867663+0000 mon.a (mon.0) 3058 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:10:28.118 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:27 smithi144 bash[66654]: audit 2024-04-07T17:10:26.873844+0000 mon.a (mon.0) 3059 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:10:28.118 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:27 smithi144 bash[66654]: audit 2024-04-07T17:10:26.896503+0000 mon.a (mon.0) 3060 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:10:28.118 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:27 smithi144 bash[66654]: audit 2024-04-07T17:10:26.903881+0000 mon.a (mon.0) 3061 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config-key dump", "format": "json"}]: dispatch 2024-04-07T17:10:28.118 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:27 smithi144 bash[66654]: audit 2024-04-07T17:10:27.077880+0000 mon.a (mon.0) 3062 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:10:28.118 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:27 smithi144 bash[66654]: audit 2024-04-07T17:10:27.078424+0000 mon.a (mon.0) 3063 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:10:28.118 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:27 smithi144 bash[66654]: audit 2024-04-07T17:10:27.078837+0000 mon.a (mon.0) 3064 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:10:28.118 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:27 smithi144 bash[66654]: audit 2024-04-07T17:10:27.082013+0000 mon.a (mon.0) 3065 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:10:28.118 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:27 smithi144 bash[66654]: audit 2024-04-07T17:10:27.082591+0000 mon.a (mon.0) 3066 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:10:28.118 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:27 smithi144 bash[66654]: audit 2024-04-07T17:10:27.082973+0000 mon.a (mon.0) 3067 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:10:28.118 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:27 smithi144 bash[66654]: audit 2024-04-07T17:10:27.085224+0000 mon.a (mon.0) 3068 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:10:28.118 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:27 smithi144 bash[66654]: audit 2024-04-07T17:10:27.085677+0000 mon.a (mon.0) 3069 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:10:28.118 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:27 smithi144 bash[66654]: audit 2024-04-07T17:10:27.086188+0000 mon.a (mon.0) 3070 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:10:28.119 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:27 smithi144 bash[66654]: audit 2024-04-07T17:10:27.101155+0000 mon.a (mon.0) 3071 : audit [INF] from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T17:10:28.119 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:27 smithi144 bash[66654]: audit 2024-04-07T17:10:27.101270+0000 mon.a (mon.0) 3072 : audit [INF] from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T17:10:28.119 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:27 smithi144 bash[66654]: audit 2024-04-07T17:10:27.101540+0000 mon.a (mon.0) 3073 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:10:28.119 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:27 smithi144 bash[66654]: audit 2024-04-07T17:10:27.101756+0000 mon.a (mon.0) 3074 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:10:28.119 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:27 smithi144 bash[66654]: audit 2024-04-07T17:10:27.102059+0000 mon.a (mon.0) 3075 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:10:28.119 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:27 smithi144 bash[66654]: audit 2024-04-07T17:10:27.104672+0000 mon.b (mon.1) 15 : audit [INF] from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T17:10:28.119 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:27 smithi144 bash[66654]: audit 2024-04-07T17:10:27.104784+0000 mon.b (mon.1) 16 : audit [INF] from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T17:10:28.119 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:27 smithi144 bash[66654]: audit 2024-04-07T17:10:27.104915+0000 mon.a (mon.0) 3076 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:10:28.119 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:27 smithi144 bash[66654]: audit 2024-04-07T17:10:27.105610+0000 mon.a (mon.0) 3077 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:10:28.119 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:27 smithi144 bash[66654]: audit 2024-04-07T17:10:27.105927+0000 mon.a (mon.0) 3078 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:10:28.119 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:27 smithi144 bash[66654]: audit 2024-04-07T17:10:27.108738+0000 mon.c (mon.2) 73 : audit [INF] from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T17:10:28.119 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:27 smithi144 bash[66654]: audit 2024-04-07T17:10:27.108820+0000 mon.c (mon.2) 74 : audit [INF] from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T17:10:28.119 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:27 smithi144 bash[66654]: audit 2024-04-07T17:10:27.109262+0000 mon.a (mon.0) 3079 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:10:28.119 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:27 smithi144 bash[66654]: audit 2024-04-07T17:10:27.109884+0000 mon.a (mon.0) 3080 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:10:28.120 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:27 smithi144 bash[66654]: audit 2024-04-07T17:10:27.110135+0000 mon.a (mon.0) 3081 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:10:28.120 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:27 smithi144 bash[66654]: cluster 2024-04-07T17:10:27.739756+0000 mon.a (mon.0) 3082 : cluster [DBG] osdmap e1632: 8 total, 8 up, 8 in 2024-04-07T17:10:28.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:27 smithi100 bash[74942]: audit 2024-04-07T17:10:26.755516+0000 mon.a (mon.0) 3053 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T17:10:28.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:27 smithi100 bash[74942]: cluster 2024-04-07T17:10:26.766153+0000 mon.a (mon.0) 3054 : cluster [DBG] osdmap e1631: 8 total, 8 up, 8 in 2024-04-07T17:10:28.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:27 smithi100 bash[74942]: audit 2024-04-07T17:10:26.830875+0000 mon.a (mon.0) 3055 : audit [INF] from='client.? ' entity='client.admin' 2024-04-07T17:10:28.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:27 smithi100 bash[74942]: audit 2024-04-07T17:10:26.864269+0000 mon.a (mon.0) 3056 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:10:28.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:27 smithi100 bash[74942]: audit 2024-04-07T17:10:26.866771+0000 mon.a (mon.0) 3057 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:10:28.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:27 smithi100 bash[74942]: audit 2024-04-07T17:10:26.867663+0000 mon.a (mon.0) 3058 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:10:28.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:27 smithi100 bash[74942]: audit 2024-04-07T17:10:26.873844+0000 mon.a (mon.0) 3059 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:10:28.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:27 smithi100 bash[74942]: audit 2024-04-07T17:10:26.896503+0000 mon.a (mon.0) 3060 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:10:28.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:27 smithi100 bash[74942]: audit 2024-04-07T17:10:26.903881+0000 mon.a (mon.0) 3061 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config-key dump", "format": "json"}]: dispatch 2024-04-07T17:10:28.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:27 smithi100 bash[74942]: audit 2024-04-07T17:10:27.077880+0000 mon.a (mon.0) 3062 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:10:28.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:27 smithi100 bash[74942]: audit 2024-04-07T17:10:27.078424+0000 mon.a (mon.0) 3063 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:10:28.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:27 smithi100 bash[74942]: audit 2024-04-07T17:10:27.078837+0000 mon.a (mon.0) 3064 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:10:28.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:27 smithi100 bash[74942]: audit 2024-04-07T17:10:27.082013+0000 mon.a (mon.0) 3065 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:10:28.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:27 smithi100 bash[74942]: audit 2024-04-07T17:10:27.082591+0000 mon.a (mon.0) 3066 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:10:28.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:27 smithi100 bash[74942]: audit 2024-04-07T17:10:27.082973+0000 mon.a (mon.0) 3067 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:10:28.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:27 smithi100 bash[74942]: audit 2024-04-07T17:10:27.085224+0000 mon.a (mon.0) 3068 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:10:28.205 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:27 smithi100 bash[74942]: audit 2024-04-07T17:10:27.085677+0000 mon.a (mon.0) 3069 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:10:28.205 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:27 smithi100 bash[74942]: audit 2024-04-07T17:10:27.086188+0000 mon.a (mon.0) 3070 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:10:28.205 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:27 smithi100 bash[76202]: audit 2024-04-07T17:10:26.755516+0000 mon.a (mon.0) 3053 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T17:10:28.205 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:27 smithi100 bash[76202]: cluster 2024-04-07T17:10:26.766153+0000 mon.a (mon.0) 3054 : cluster [DBG] osdmap e1631: 8 total, 8 up, 8 in 2024-04-07T17:10:28.205 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:27 smithi100 bash[76202]: audit 2024-04-07T17:10:26.830875+0000 mon.a (mon.0) 3055 : audit [INF] from='client.? ' entity='client.admin' 2024-04-07T17:10:28.205 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:27 smithi100 bash[76202]: audit 2024-04-07T17:10:26.864269+0000 mon.a (mon.0) 3056 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:10:28.205 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:27 smithi100 bash[76202]: audit 2024-04-07T17:10:26.866771+0000 mon.a (mon.0) 3057 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:10:28.205 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:27 smithi100 bash[76202]: audit 2024-04-07T17:10:26.867663+0000 mon.a (mon.0) 3058 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:10:28.205 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:27 smithi100 bash[76202]: audit 2024-04-07T17:10:26.873844+0000 mon.a (mon.0) 3059 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:10:28.205 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:27 smithi100 bash[76202]: audit 2024-04-07T17:10:26.896503+0000 mon.a (mon.0) 3060 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:10:28.206 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:27 smithi100 bash[76202]: audit 2024-04-07T17:10:26.903881+0000 mon.a (mon.0) 3061 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config-key dump", "format": "json"}]: dispatch 2024-04-07T17:10:28.206 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:27 smithi100 bash[76202]: audit 2024-04-07T17:10:27.077880+0000 mon.a (mon.0) 3062 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:10:28.206 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:27 smithi100 bash[76202]: audit 2024-04-07T17:10:27.078424+0000 mon.a (mon.0) 3063 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:10:28.206 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:27 smithi100 bash[76202]: audit 2024-04-07T17:10:27.078837+0000 mon.a (mon.0) 3064 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:10:28.206 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:27 smithi100 bash[76202]: audit 2024-04-07T17:10:27.082013+0000 mon.a (mon.0) 3065 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:10:28.206 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:27 smithi100 bash[76202]: audit 2024-04-07T17:10:27.082591+0000 mon.a (mon.0) 3066 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:10:28.206 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:27 smithi100 bash[76202]: audit 2024-04-07T17:10:27.082973+0000 mon.a (mon.0) 3067 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:10:28.206 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:27 smithi100 bash[76202]: audit 2024-04-07T17:10:27.085224+0000 mon.a (mon.0) 3068 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:10:28.206 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:27 smithi100 bash[76202]: audit 2024-04-07T17:10:27.085677+0000 mon.a (mon.0) 3069 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:10:28.206 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:27 smithi100 bash[76202]: audit 2024-04-07T17:10:27.086188+0000 mon.a (mon.0) 3070 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:10:28.206 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:27 smithi100 bash[76202]: audit 2024-04-07T17:10:27.101155+0000 mon.a (mon.0) 3071 : audit [INF] from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T17:10:28.206 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:27 smithi100 bash[76202]: audit 2024-04-07T17:10:27.101270+0000 mon.a (mon.0) 3072 : audit [INF] from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T17:10:28.206 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:27 smithi100 bash[76202]: audit 2024-04-07T17:10:27.101540+0000 mon.a (mon.0) 3073 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:10:28.206 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:27 smithi100 bash[76202]: audit 2024-04-07T17:10:27.101756+0000 mon.a (mon.0) 3074 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:10:28.206 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:27 smithi100 bash[76202]: audit 2024-04-07T17:10:27.102059+0000 mon.a (mon.0) 3075 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:10:28.207 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:27 smithi100 bash[76202]: audit 2024-04-07T17:10:27.104672+0000 mon.b (mon.1) 15 : audit [INF] from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T17:10:28.207 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:27 smithi100 bash[76202]: audit 2024-04-07T17:10:27.104784+0000 mon.b (mon.1) 16 : audit [INF] from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T17:10:28.207 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:27 smithi100 bash[76202]: audit 2024-04-07T17:10:27.104915+0000 mon.a (mon.0) 3076 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:10:28.207 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:27 smithi100 bash[76202]: audit 2024-04-07T17:10:27.105610+0000 mon.a (mon.0) 3077 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:10:28.207 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:27 smithi100 bash[76202]: audit 2024-04-07T17:10:27.105927+0000 mon.a (mon.0) 3078 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:10:28.207 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:27 smithi100 bash[76202]: audit 2024-04-07T17:10:27.108738+0000 mon.c (mon.2) 73 : audit [INF] from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T17:10:28.207 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:27 smithi100 bash[76202]: audit 2024-04-07T17:10:27.108820+0000 mon.c (mon.2) 74 : audit [INF] from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T17:10:28.207 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:27 smithi100 bash[76202]: audit 2024-04-07T17:10:27.109262+0000 mon.a (mon.0) 3079 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:10:28.207 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:27 smithi100 bash[76202]: audit 2024-04-07T17:10:27.109884+0000 mon.a (mon.0) 3080 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:10:28.207 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:27 smithi100 bash[76202]: audit 2024-04-07T17:10:27.110135+0000 mon.a (mon.0) 3081 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:10:28.207 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:27 smithi100 bash[76202]: cluster 2024-04-07T17:10:27.739756+0000 mon.a (mon.0) 3082 : cluster [DBG] osdmap e1632: 8 total, 8 up, 8 in 2024-04-07T17:10:28.207 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:27 smithi100 bash[74942]: audit 2024-04-07T17:10:27.101155+0000 mon.a (mon.0) 3071 : audit [INF] from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T17:10:28.208 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:27 smithi100 bash[74942]: audit 2024-04-07T17:10:27.101270+0000 mon.a (mon.0) 3072 : audit [INF] from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T17:10:28.208 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:27 smithi100 bash[74942]: audit 2024-04-07T17:10:27.101540+0000 mon.a (mon.0) 3073 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:10:28.208 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:27 smithi100 bash[74942]: audit 2024-04-07T17:10:27.101756+0000 mon.a (mon.0) 3074 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:10:28.208 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:27 smithi100 bash[74942]: audit 2024-04-07T17:10:27.102059+0000 mon.a (mon.0) 3075 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:10:28.208 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:27 smithi100 bash[74942]: audit 2024-04-07T17:10:27.104672+0000 mon.b (mon.1) 15 : audit [INF] from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T17:10:28.208 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:27 smithi100 bash[74942]: audit 2024-04-07T17:10:27.104784+0000 mon.b (mon.1) 16 : audit [INF] from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T17:10:28.208 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:27 smithi100 bash[74942]: audit 2024-04-07T17:10:27.104915+0000 mon.a (mon.0) 3076 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:10:28.208 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:27 smithi100 bash[74942]: audit 2024-04-07T17:10:27.105610+0000 mon.a (mon.0) 3077 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:10:28.208 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:27 smithi100 bash[74942]: audit 2024-04-07T17:10:27.105927+0000 mon.a (mon.0) 3078 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:10:28.208 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:27 smithi100 bash[74942]: audit 2024-04-07T17:10:27.108738+0000 mon.c (mon.2) 73 : audit [INF] from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T17:10:28.208 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:27 smithi100 bash[74942]: audit 2024-04-07T17:10:27.108820+0000 mon.c (mon.2) 74 : audit [INF] from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T17:10:28.208 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:27 smithi100 bash[74942]: audit 2024-04-07T17:10:27.109262+0000 mon.a (mon.0) 3079 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:10:28.208 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:27 smithi100 bash[74942]: audit 2024-04-07T17:10:27.109884+0000 mon.a (mon.0) 3080 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:10:28.208 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:27 smithi100 bash[74942]: audit 2024-04-07T17:10:27.110135+0000 mon.a (mon.0) 3081 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:10:28.209 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:27 smithi100 bash[74942]: cluster 2024-04-07T17:10:27.739756+0000 mon.a (mon.0) 3082 : cluster [DBG] osdmap e1632: 8 total, 8 up, 8 in 2024-04-07T17:10:28.703 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:10:28 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:10:28] "GET /metrics HTTP/1.1" 200 50981 "" "Prometheus/2.43.0" 2024-04-07T17:10:28.973 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:10:29.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:28 smithi100 bash[74942]: cluster 2024-04-07T17:10:27.413844+0000 mgr.y (mgr.25175) 1653 : cluster [DBG] pgmap v2859: 393 pgs: 15 unknown, 378 active+clean; 4.6 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 86 KiB/s rd, 7.7 KiB/s wr, 118 op/s 2024-04-07T17:10:29.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:28 smithi100 bash[74942]: cluster 2024-04-07T17:10:28.756302+0000 mon.a (mon.0) 3083 : cluster [DBG] osdmap e1633: 8 total, 8 up, 8 in 2024-04-07T17:10:29.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:28 smithi100 bash[76202]: cluster 2024-04-07T17:10:27.413844+0000 mgr.y (mgr.25175) 1653 : cluster [DBG] pgmap v2859: 393 pgs: 15 unknown, 378 active+clean; 4.6 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 86 KiB/s rd, 7.7 KiB/s wr, 118 op/s 2024-04-07T17:10:29.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:28 smithi100 bash[76202]: cluster 2024-04-07T17:10:28.756302+0000 mon.a (mon.0) 3083 : cluster [DBG] osdmap e1633: 8 total, 8 up, 8 in 2024-04-07T17:10:29.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:28 smithi144 bash[66654]: cluster 2024-04-07T17:10:27.413844+0000 mgr.y (mgr.25175) 1653 : cluster [DBG] pgmap v2859: 393 pgs: 15 unknown, 378 active+clean; 4.6 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 86 KiB/s rd, 7.7 KiB/s wr, 118 op/s 2024-04-07T17:10:29.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:28 smithi144 bash[66654]: cluster 2024-04-07T17:10:28.756302+0000 mon.a (mon.0) 3083 : cluster [DBG] osdmap e1633: 8 total, 8 up, 8 in 2024-04-07T17:10:30.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:29 smithi100 bash[74942]: audit 2024-04-07T17:10:28.995993+0000 mon.c (mon.2) 75 : audit [INF] from='client.? 172.21.15.100:0/3213618994' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "test-librbd-smithi100-115303-23", "field": "max_bytes", "val": "10485760"}]: dispatch 2024-04-07T17:10:30.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:29 smithi100 bash[74942]: audit 2024-04-07T17:10:28.996696+0000 mon.a (mon.0) 3084 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "test-librbd-smithi100-115303-23", "field": "max_bytes", "val": "10485760"}]: dispatch 2024-04-07T17:10:30.205 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:29 smithi100 bash[76202]: audit 2024-04-07T17:10:28.995993+0000 mon.c (mon.2) 75 : audit [INF] from='client.? 172.21.15.100:0/3213618994' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "test-librbd-smithi100-115303-23", "field": "max_bytes", "val": "10485760"}]: dispatch 2024-04-07T17:10:30.205 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:29 smithi100 bash[76202]: audit 2024-04-07T17:10:28.996696+0000 mon.a (mon.0) 3084 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "test-librbd-smithi100-115303-23", "field": "max_bytes", "val": "10485760"}]: dispatch 2024-04-07T17:10:30.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:29 smithi144 bash[66654]: audit 2024-04-07T17:10:28.995993+0000 mon.c (mon.2) 75 : audit [INF] from='client.? 172.21.15.100:0/3213618994' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "test-librbd-smithi100-115303-23", "field": "max_bytes", "val": "10485760"}]: dispatch 2024-04-07T17:10:30.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:29 smithi144 bash[66654]: audit 2024-04-07T17:10:28.996696+0000 mon.a (mon.0) 3084 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "test-librbd-smithi100-115303-23", "field": "max_bytes", "val": "10485760"}]: dispatch 2024-04-07T17:10:31.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:30 smithi100 bash[74942]: cluster 2024-04-07T17:10:29.415470+0000 mgr.y (mgr.25175) 1654 : cluster [DBG] pgmap v2862: 393 pgs: 6 unknown, 387 active+clean; 4.6 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 97 KiB/s rd, 8.0 KiB/s wr, 135 op/s 2024-04-07T17:10:31.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:30 smithi100 bash[74942]: audit 2024-04-07T17:10:29.890417+0000 mon.a (mon.0) 3085 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "test-librbd-smithi100-115303-23", "field": "max_bytes", "val": "10485760"}]': finished 2024-04-07T17:10:31.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:30 smithi100 bash[74942]: cluster 2024-04-07T17:10:29.898038+0000 mon.a (mon.0) 3086 : cluster [DBG] osdmap e1634: 8 total, 8 up, 8 in 2024-04-07T17:10:31.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:30 smithi100 bash[76202]: cluster 2024-04-07T17:10:29.415470+0000 mgr.y (mgr.25175) 1654 : cluster [DBG] pgmap v2862: 393 pgs: 6 unknown, 387 active+clean; 4.6 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 97 KiB/s rd, 8.0 KiB/s wr, 135 op/s 2024-04-07T17:10:31.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:30 smithi100 bash[76202]: audit 2024-04-07T17:10:29.890417+0000 mon.a (mon.0) 3085 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "test-librbd-smithi100-115303-23", "field": "max_bytes", "val": "10485760"}]': finished 2024-04-07T17:10:31.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:30 smithi100 bash[76202]: cluster 2024-04-07T17:10:29.898038+0000 mon.a (mon.0) 3086 : cluster [DBG] osdmap e1634: 8 total, 8 up, 8 in 2024-04-07T17:10:31.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:30 smithi144 bash[66654]: cluster 2024-04-07T17:10:29.415470+0000 mgr.y (mgr.25175) 1654 : cluster [DBG] pgmap v2862: 393 pgs: 6 unknown, 387 active+clean; 4.6 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 97 KiB/s rd, 8.0 KiB/s wr, 135 op/s 2024-04-07T17:10:31.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:30 smithi144 bash[66654]: audit 2024-04-07T17:10:29.890417+0000 mon.a (mon.0) 3085 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "test-librbd-smithi100-115303-23", "field": "max_bytes", "val": "10485760"}]': finished 2024-04-07T17:10:31.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:30 smithi144 bash[66654]: cluster 2024-04-07T17:10:29.898038+0000 mon.a (mon.0) 3086 : cluster [DBG] osdmap e1634: 8 total, 8 up, 8 in 2024-04-07T17:10:32.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:31 smithi100 bash[74942]: audit 2024-04-07T17:10:31.817152+0000 mon.a (mon.0) 3087 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:10:32.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:31 smithi100 bash[76202]: audit 2024-04-07T17:10:31.817152+0000 mon.a (mon.0) 3087 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:10:32.316 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:31 smithi144 bash[66654]: audit 2024-04-07T17:10:31.817152+0000 mon.a (mon.0) 3087 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:10:32.616 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:10:32 smithi144 bash[95233]: ts=2024-04-07T17:10:32.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:10:33.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:32 smithi100 bash[74942]: cluster 2024-04-07T17:10:31.416355+0000 mgr.y (mgr.25175) 1655 : cluster [DBG] pgmap v2864: 393 pgs: 393 active+clean; 4.6 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 86 KiB/s rd, 7.1 KiB/s wr, 123 op/s 2024-04-07T17:10:33.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:32 smithi100 bash[74942]: cluster 2024-04-07T17:10:32.746155+0000 mon.a (mon.0) 3088 : cluster [DBG] osdmap e1635: 8 total, 8 up, 8 in 2024-04-07T17:10:33.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:32 smithi100 bash[76202]: cluster 2024-04-07T17:10:31.416355+0000 mgr.y (mgr.25175) 1655 : cluster [DBG] pgmap v2864: 393 pgs: 393 active+clean; 4.6 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 86 KiB/s rd, 7.1 KiB/s wr, 123 op/s 2024-04-07T17:10:33.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:32 smithi100 bash[76202]: cluster 2024-04-07T17:10:32.746155+0000 mon.a (mon.0) 3088 : cluster [DBG] osdmap e1635: 8 total, 8 up, 8 in 2024-04-07T17:10:33.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:32 smithi144 bash[66654]: cluster 2024-04-07T17:10:31.416355+0000 mgr.y (mgr.25175) 1655 : cluster [DBG] pgmap v2864: 393 pgs: 393 active+clean; 4.6 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 86 KiB/s rd, 7.1 KiB/s wr, 123 op/s 2024-04-07T17:10:33.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:32 smithi144 bash[66654]: cluster 2024-04-07T17:10:32.746155+0000 mon.a (mon.0) 3088 : cluster [DBG] osdmap e1635: 8 total, 8 up, 8 in 2024-04-07T17:10:34.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:33 smithi144 bash[66654]: cluster 2024-04-07T17:10:33.417786+0000 mgr.y (mgr.25175) 1656 : cluster [DBG] pgmap v2866: 393 pgs: 393 active+clean; 4.6 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 49 KiB/s rd, 3.5 KiB/s wr, 72 op/s 2024-04-07T17:10:34.453 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:33 smithi100 bash[74942]: cluster 2024-04-07T17:10:33.417786+0000 mgr.y (mgr.25175) 1656 : cluster [DBG] pgmap v2866: 393 pgs: 393 active+clean; 4.6 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 49 KiB/s rd, 3.5 KiB/s wr, 72 op/s 2024-04-07T17:10:34.453 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:33 smithi100 bash[76202]: cluster 2024-04-07T17:10:33.417786+0000 mgr.y (mgr.25175) 1656 : cluster [DBG] pgmap v2866: 393 pgs: 393 active+clean; 4.6 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 49 KiB/s rd, 3.5 KiB/s wr, 72 op/s 2024-04-07T17:10:36.866 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:36 smithi144 bash[66654]: cluster 2024-04-07T17:10:35.419714+0000 mgr.y (mgr.25175) 1657 : cluster [DBG] pgmap v2867: 393 pgs: 393 active+clean; 8.6 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 40 KiB/s rd, 617 KiB/s wr, 60 op/s 2024-04-07T17:10:36.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:36 smithi100 bash[74942]: cluster 2024-04-07T17:10:35.419714+0000 mgr.y (mgr.25175) 1657 : cluster [DBG] pgmap v2867: 393 pgs: 393 active+clean; 8.6 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 40 KiB/s rd, 617 KiB/s wr, 60 op/s 2024-04-07T17:10:36.953 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:36 smithi100 bash[76202]: cluster 2024-04-07T17:10:35.419714+0000 mgr.y (mgr.25175) 1657 : cluster [DBG] pgmap v2867: 393 pgs: 393 active+clean; 8.6 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 40 KiB/s rd, 617 KiB/s wr, 60 op/s 2024-04-07T17:10:38.703 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:10:38 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:10:38] "GET /metrics HTTP/1.1" 200 55788 "" "Prometheus/2.43.0" 2024-04-07T17:10:39.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:38 smithi144 bash[66654]: cluster 2024-04-07T17:10:37.422070+0000 mgr.y (mgr.25175) 1658 : cluster [DBG] pgmap v2868: 393 pgs: 393 active+clean; 8.6 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 22 KiB/s rd, 514 KiB/s wr, 34 op/s 2024-04-07T17:10:39.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:38 smithi144 bash[66654]: cluster 2024-04-07T17:10:37.743929+0000 mon.a (mon.0) 3089 : cluster [DBG] osdmap e1636: 8 total, 8 up, 8 in 2024-04-07T17:10:39.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:38 smithi100 bash[74942]: cluster 2024-04-07T17:10:37.422070+0000 mgr.y (mgr.25175) 1658 : cluster [DBG] pgmap v2868: 393 pgs: 393 active+clean; 8.6 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 22 KiB/s rd, 514 KiB/s wr, 34 op/s 2024-04-07T17:10:39.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:38 smithi100 bash[74942]: cluster 2024-04-07T17:10:37.743929+0000 mon.a (mon.0) 3089 : cluster [DBG] osdmap e1636: 8 total, 8 up, 8 in 2024-04-07T17:10:39.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:38 smithi100 bash[76202]: cluster 2024-04-07T17:10:37.422070+0000 mgr.y (mgr.25175) 1658 : cluster [DBG] pgmap v2868: 393 pgs: 393 active+clean; 8.6 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 22 KiB/s rd, 514 KiB/s wr, 34 op/s 2024-04-07T17:10:39.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:38 smithi100 bash[76202]: cluster 2024-04-07T17:10:37.743929+0000 mon.a (mon.0) 3089 : cluster [DBG] osdmap e1636: 8 total, 8 up, 8 in 2024-04-07T17:10:41.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:40 smithi144 bash[66654]: cluster 2024-04-07T17:10:39.424061+0000 mgr.y (mgr.25175) 1659 : cluster [DBG] pgmap v2870: 393 pgs: 393 active+clean; 13 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 21 KiB/s rd, 1.0 MiB/s wr, 31 op/s 2024-04-07T17:10:41.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:40 smithi100 bash[74942]: cluster 2024-04-07T17:10:39.424061+0000 mgr.y (mgr.25175) 1659 : cluster [DBG] pgmap v2870: 393 pgs: 393 active+clean; 13 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 21 KiB/s rd, 1.0 MiB/s wr, 31 op/s 2024-04-07T17:10:41.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:40 smithi100 bash[76202]: cluster 2024-04-07T17:10:39.424061+0000 mgr.y (mgr.25175) 1659 : cluster [DBG] pgmap v2870: 393 pgs: 393 active+clean; 13 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 21 KiB/s rd, 1.0 MiB/s wr, 31 op/s 2024-04-07T17:10:42.616 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:10:42 smithi144 bash[95233]: ts=2024-04-07T17:10:42.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:10:43.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:42 smithi144 bash[66654]: cluster 2024-04-07T17:10:41.425413+0000 mgr.y (mgr.25175) 1660 : cluster [DBG] pgmap v2871: 393 pgs: 393 active+clean; 13 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 2.5 KiB/s rd, 944 KiB/s wr, 5 op/s 2024-04-07T17:10:43.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:42 smithi100 bash[74942]: cluster 2024-04-07T17:10:41.425413+0000 mgr.y (mgr.25175) 1660 : cluster [DBG] pgmap v2871: 393 pgs: 393 active+clean; 13 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 2.5 KiB/s rd, 944 KiB/s wr, 5 op/s 2024-04-07T17:10:43.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:42 smithi100 bash[76202]: cluster 2024-04-07T17:10:41.425413+0000 mgr.y (mgr.25175) 1660 : cluster [DBG] pgmap v2871: 393 pgs: 393 active+clean; 13 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 2.5 KiB/s rd, 944 KiB/s wr, 5 op/s 2024-04-07T17:10:45.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:44 smithi144 bash[66654]: cluster 2024-04-07T17:10:43.428153+0000 mgr.y (mgr.25175) 1661 : cluster [DBG] pgmap v2872: 393 pgs: 393 active+clean; 17 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 2.2 KiB/s rd, 1.2 MiB/s wr, 5 op/s 2024-04-07T17:10:45.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:44 smithi100 bash[74942]: cluster 2024-04-07T17:10:43.428153+0000 mgr.y (mgr.25175) 1661 : cluster [DBG] pgmap v2872: 393 pgs: 393 active+clean; 17 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 2.2 KiB/s rd, 1.2 MiB/s wr, 5 op/s 2024-04-07T17:10:45.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:44 smithi100 bash[76202]: cluster 2024-04-07T17:10:43.428153+0000 mgr.y (mgr.25175) 1661 : cluster [DBG] pgmap v2872: 393 pgs: 393 active+clean; 17 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 2.2 KiB/s rd, 1.2 MiB/s wr, 5 op/s 2024-04-07T17:10:47.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:46 smithi144 bash[66654]: cluster 2024-04-07T17:10:45.430138+0000 mgr.y (mgr.25175) 1662 : cluster [DBG] pgmap v2873: 393 pgs: 393 active+clean; 19 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 1022 B/s rd, 1023 KiB/s wr, 2 op/s 2024-04-07T17:10:47.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:46 smithi100 bash[74942]: cluster 2024-04-07T17:10:45.430138+0000 mgr.y (mgr.25175) 1662 : cluster [DBG] pgmap v2873: 393 pgs: 393 active+clean; 19 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 1022 B/s rd, 1023 KiB/s wr, 2 op/s 2024-04-07T17:10:47.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:46 smithi100 bash[76202]: cluster 2024-04-07T17:10:45.430138+0000 mgr.y (mgr.25175) 1662 : cluster [DBG] pgmap v2873: 393 pgs: 393 active+clean; 19 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 1022 B/s rd, 1023 KiB/s wr, 2 op/s 2024-04-07T17:10:48.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:47 smithi144 bash[66654]: audit 2024-04-07T17:10:46.817386+0000 mon.a (mon.0) 3090 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:10:48.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:47 smithi144 bash[66654]: cluster 2024-04-07T17:10:47.737419+0000 mon.a (mon.0) 3091 : cluster [WRN] pool 'test-librbd-smithi100-115303-23' is full (reached quota's max_bytes: 10 MiB) 2024-04-07T17:10:48.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:47 smithi144 bash[66654]: cluster 2024-04-07T17:10:47.738092+0000 mon.a (mon.0) 3092 : cluster [WRN] Health check failed: 1 pool(s) full (POOL_FULL) 2024-04-07T17:10:48.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:47 smithi144 bash[66654]: cluster 2024-04-07T17:10:47.748886+0000 mon.a (mon.0) 3093 : cluster [DBG] osdmap e1637: 8 total, 8 up, 8 in 2024-04-07T17:10:48.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:47 smithi100 bash[74942]: audit 2024-04-07T17:10:46.817386+0000 mon.a (mon.0) 3090 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:10:48.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:47 smithi100 bash[74942]: cluster 2024-04-07T17:10:47.737419+0000 mon.a (mon.0) 3091 : cluster [WRN] pool 'test-librbd-smithi100-115303-23' is full (reached quota's max_bytes: 10 MiB) 2024-04-07T17:10:48.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:47 smithi100 bash[74942]: cluster 2024-04-07T17:10:47.738092+0000 mon.a (mon.0) 3092 : cluster [WRN] Health check failed: 1 pool(s) full (POOL_FULL) 2024-04-07T17:10:48.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:47 smithi100 bash[74942]: cluster 2024-04-07T17:10:47.748886+0000 mon.a (mon.0) 3093 : cluster [DBG] osdmap e1637: 8 total, 8 up, 8 in 2024-04-07T17:10:48.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:47 smithi100 bash[76202]: audit 2024-04-07T17:10:46.817386+0000 mon.a (mon.0) 3090 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:10:48.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:47 smithi100 bash[76202]: cluster 2024-04-07T17:10:47.737419+0000 mon.a (mon.0) 3091 : cluster [WRN] pool 'test-librbd-smithi100-115303-23' is full (reached quota's max_bytes: 10 MiB) 2024-04-07T17:10:48.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:47 smithi100 bash[76202]: cluster 2024-04-07T17:10:47.738092+0000 mon.a (mon.0) 3092 : cluster [WRN] Health check failed: 1 pool(s) full (POOL_FULL) 2024-04-07T17:10:48.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:47 smithi100 bash[76202]: cluster 2024-04-07T17:10:47.748886+0000 mon.a (mon.0) 3093 : cluster [DBG] osdmap e1637: 8 total, 8 up, 8 in 2024-04-07T17:10:48.305 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.RemoveFullTry (22996 ms) 2024-04-07T17:10:48.305 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.RemoveFullTryDataPool 2024-04-07T17:10:48.703 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:10:48 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:10:48] "GET /metrics HTTP/1.1" 200 55812 "" "Prometheus/2.43.0" 2024-04-07T17:10:49.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:48 smithi144 bash[66654]: cluster 2024-04-07T17:10:47.432575+0000 mgr.y (mgr.25175) 1663 : cluster [DBG] pgmap v2874: 393 pgs: 393 active+clean; 19 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 1022 B/s rd, 1023 KiB/s wr, 2 op/s 2024-04-07T17:10:49.205 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:48 smithi100 bash[74942]: cluster 2024-04-07T17:10:47.432575+0000 mgr.y (mgr.25175) 1663 : cluster [DBG] pgmap v2874: 393 pgs: 393 active+clean; 19 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 1022 B/s rd, 1023 KiB/s wr, 2 op/s 2024-04-07T17:10:49.205 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:48 smithi100 bash[76202]: cluster 2024-04-07T17:10:47.432575+0000 mgr.y (mgr.25175) 1663 : cluster [DBG] pgmap v2874: 393 pgs: 393 active+clean; 19 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 1022 B/s rd, 1023 KiB/s wr, 2 op/s 2024-04-07T17:10:50.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:49 smithi144 bash[66654]: cluster 2024-04-07T17:10:48.798950+0000 mon.a (mon.0) 3094 : cluster [DBG] osdmap e1638: 8 total, 8 up, 8 in 2024-04-07T17:10:50.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:49 smithi144 bash[66654]: audit 2024-04-07T17:10:48.807393+0000 mon.a (mon.0) 3095 : audit [INF] from='client.? 172.21.15.100:0/997283390' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:10:50.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:49 smithi100 bash[74942]: cluster 2024-04-07T17:10:48.798950+0000 mon.a (mon.0) 3094 : cluster [DBG] osdmap e1638: 8 total, 8 up, 8 in 2024-04-07T17:10:50.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:49 smithi100 bash[74942]: audit 2024-04-07T17:10:48.807393+0000 mon.a (mon.0) 3095 : audit [INF] from='client.? 172.21.15.100:0/997283390' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:10:50.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:49 smithi100 bash[76202]: cluster 2024-04-07T17:10:48.798950+0000 mon.a (mon.0) 3094 : cluster [DBG] osdmap e1638: 8 total, 8 up, 8 in 2024-04-07T17:10:50.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:49 smithi100 bash[76202]: audit 2024-04-07T17:10:48.807393+0000 mon.a (mon.0) 3095 : audit [INF] from='client.? 172.21.15.100:0/997283390' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:10:51.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:50 smithi144 bash[66654]: cluster 2024-04-07T17:10:49.434366+0000 mgr.y (mgr.25175) 1664 : cluster [DBG] pgmap v2877: 425 pgs: 7 creating+peering, 25 unknown, 393 active+clean; 13 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 639 B/s rd, 1023 KiB/s wr, 2 op/s 2024-04-07T17:10:51.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:50 smithi144 bash[66654]: cluster 2024-04-07T17:10:49.793255+0000 mon.a (mon.0) 3096 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T17:10:51.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:50 smithi144 bash[66654]: audit 2024-04-07T17:10:49.805916+0000 mon.a (mon.0) 3097 : audit [INF] from='client.? 172.21.15.100:0/997283390' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T17:10:51.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:50 smithi144 bash[66654]: cluster 2024-04-07T17:10:49.817537+0000 mon.a (mon.0) 3098 : cluster [DBG] osdmap e1639: 8 total, 8 up, 8 in 2024-04-07T17:10:51.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:50 smithi100 bash[74942]: cluster 2024-04-07T17:10:49.434366+0000 mgr.y (mgr.25175) 1664 : cluster [DBG] pgmap v2877: 425 pgs: 7 creating+peering, 25 unknown, 393 active+clean; 13 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 639 B/s rd, 1023 KiB/s wr, 2 op/s 2024-04-07T17:10:51.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:50 smithi100 bash[74942]: cluster 2024-04-07T17:10:49.793255+0000 mon.a (mon.0) 3096 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T17:10:51.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:50 smithi100 bash[74942]: audit 2024-04-07T17:10:49.805916+0000 mon.a (mon.0) 3097 : audit [INF] from='client.? 172.21.15.100:0/997283390' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T17:10:51.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:50 smithi100 bash[74942]: cluster 2024-04-07T17:10:49.817537+0000 mon.a (mon.0) 3098 : cluster [DBG] osdmap e1639: 8 total, 8 up, 8 in 2024-04-07T17:10:51.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:50 smithi100 bash[76202]: cluster 2024-04-07T17:10:49.434366+0000 mgr.y (mgr.25175) 1664 : cluster [DBG] pgmap v2877: 425 pgs: 7 creating+peering, 25 unknown, 393 active+clean; 13 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 639 B/s rd, 1023 KiB/s wr, 2 op/s 2024-04-07T17:10:51.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:50 smithi100 bash[76202]: cluster 2024-04-07T17:10:49.793255+0000 mon.a (mon.0) 3096 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T17:10:51.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:50 smithi100 bash[76202]: audit 2024-04-07T17:10:49.805916+0000 mon.a (mon.0) 3097 : audit [INF] from='client.? 172.21.15.100:0/997283390' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T17:10:51.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:50 smithi100 bash[76202]: cluster 2024-04-07T17:10:49.817537+0000 mon.a (mon.0) 3098 : cluster [DBG] osdmap e1639: 8 total, 8 up, 8 in 2024-04-07T17:10:51.881 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:10:52.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:51 smithi144 bash[66654]: cluster 2024-04-07T17:10:50.821336+0000 mon.a (mon.0) 3099 : cluster [DBG] osdmap e1640: 8 total, 8 up, 8 in 2024-04-07T17:10:52.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:51 smithi144 bash[66654]: audit 2024-04-07T17:10:50.829844+0000 mon.c (mon.2) 76 : audit [INF] from='client.? 172.21.15.100:0/3977982330' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:10:52.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:51 smithi144 bash[66654]: audit 2024-04-07T17:10:50.831667+0000 mon.a (mon.0) 3100 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:10:52.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:51 smithi100 bash[74942]: cluster 2024-04-07T17:10:50.821336+0000 mon.a (mon.0) 3099 : cluster [DBG] osdmap e1640: 8 total, 8 up, 8 in 2024-04-07T17:10:52.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:51 smithi100 bash[74942]: audit 2024-04-07T17:10:50.829844+0000 mon.c (mon.2) 76 : audit [INF] from='client.? 172.21.15.100:0/3977982330' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:10:52.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:51 smithi100 bash[74942]: audit 2024-04-07T17:10:50.831667+0000 mon.a (mon.0) 3100 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:10:52.203 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:10:51 smithi100 bash[68137]: [07/Apr/2024:17:10:51] ENGINE Bus STOPPING 2024-04-07T17:10:52.203 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:10:51 smithi100 bash[68137]: [07/Apr/2024:17:10:51] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-04-07T17:10:52.204 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:10:51 smithi100 bash[68137]: [07/Apr/2024:17:10:51] ENGINE Bus STOPPED 2024-04-07T17:10:52.204 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:10:51 smithi100 bash[68137]: [07/Apr/2024:17:10:51] ENGINE Bus STARTING 2024-04-07T17:10:52.204 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:10:52 smithi100 bash[68137]: [07/Apr/2024:17:10:52] ENGINE Serving on http://:::9283 2024-04-07T17:10:52.204 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:10:52 smithi100 bash[68137]: [07/Apr/2024:17:10:52] ENGINE Bus STARTED 2024-04-07T17:10:52.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:51 smithi100 bash[76202]: cluster 2024-04-07T17:10:50.821336+0000 mon.a (mon.0) 3099 : cluster [DBG] osdmap e1640: 8 total, 8 up, 8 in 2024-04-07T17:10:52.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:51 smithi100 bash[76202]: audit 2024-04-07T17:10:50.829844+0000 mon.c (mon.2) 76 : audit [INF] from='client.? 172.21.15.100:0/3977982330' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:10:52.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:51 smithi100 bash[76202]: audit 2024-04-07T17:10:50.831667+0000 mon.a (mon.0) 3100 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:10:52.616 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:10:52 smithi144 bash[95233]: ts=2024-04-07T17:10:52.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:10:53.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:52 smithi144 bash[66654]: cluster 2024-04-07T17:10:51.435491+0000 mgr.y (mgr.25175) 1665 : cluster [DBG] pgmap v2880: 457 pgs: 7 creating+peering, 53 unknown, 397 active+clean; 8.6 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 6.2 KiB/s rd, 1.0 MiB/s wr, 13 op/s 2024-04-07T17:10:53.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:52 smithi144 bash[66654]: audit 2024-04-07T17:10:51.824157+0000 mon.a (mon.0) 3101 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T17:10:53.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:52 smithi144 bash[66654]: cluster 2024-04-07T17:10:51.834791+0000 mon.a (mon.0) 3102 : cluster [DBG] osdmap e1641: 8 total, 8 up, 8 in 2024-04-07T17:10:53.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:52 smithi144 bash[66654]: audit 2024-04-07T17:10:51.877339+0000 mon.a (mon.0) 3103 : audit [INF] from='client.? ' entity='client.admin' 2024-04-07T17:10:53.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:52 smithi144 bash[66654]: audit 2024-04-07T17:10:51.912816+0000 mon.a (mon.0) 3104 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:10:53.118 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:52 smithi144 bash[66654]: audit 2024-04-07T17:10:51.916876+0000 mon.a (mon.0) 3105 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:10:53.118 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:52 smithi144 bash[66654]: audit 2024-04-07T17:10:51.918410+0000 mon.a (mon.0) 3106 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:10:53.118 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:52 smithi144 bash[66654]: audit 2024-04-07T17:10:51.919471+0000 mon.a (mon.0) 3107 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:10:53.118 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:52 smithi144 bash[66654]: audit 2024-04-07T17:10:51.925605+0000 mon.a (mon.0) 3108 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:10:53.118 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:52 smithi144 bash[66654]: audit 2024-04-07T17:10:51.925916+0000 mon.a (mon.0) 3109 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config-key dump", "format": "json"}]: dispatch 2024-04-07T17:10:53.118 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:52 smithi144 bash[66654]: audit 2024-04-07T17:10:52.101534+0000 mon.a (mon.0) 3110 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:10:53.118 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:52 smithi144 bash[66654]: audit 2024-04-07T17:10:52.101744+0000 mon.a (mon.0) 3111 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:10:53.118 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:52 smithi144 bash[66654]: audit 2024-04-07T17:10:52.101898+0000 mon.a (mon.0) 3112 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:10:53.118 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:52 smithi144 bash[66654]: audit 2024-04-07T17:10:52.104004+0000 mon.a (mon.0) 3113 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:10:53.118 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:52 smithi144 bash[66654]: audit 2024-04-07T17:10:52.104144+0000 mon.a (mon.0) 3114 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:10:53.118 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:52 smithi144 bash[66654]: audit 2024-04-07T17:10:52.104284+0000 mon.a (mon.0) 3115 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:10:53.118 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:52 smithi144 bash[66654]: audit 2024-04-07T17:10:52.105564+0000 mon.a (mon.0) 3116 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:10:53.118 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:52 smithi144 bash[66654]: audit 2024-04-07T17:10:52.105777+0000 mon.a (mon.0) 3117 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:10:53.119 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:52 smithi144 bash[66654]: audit 2024-04-07T17:10:52.105912+0000 mon.a (mon.0) 3118 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:10:53.119 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:52 smithi144 bash[66654]: audit 2024-04-07T17:10:52.118079+0000 mon.a (mon.0) 3119 : audit [INF] from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T17:10:53.119 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:52 smithi144 bash[66654]: audit 2024-04-07T17:10:52.118145+0000 mon.a (mon.0) 3120 : audit [INF] from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T17:10:53.119 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:52 smithi144 bash[66654]: audit 2024-04-07T17:10:52.118364+0000 mon.a (mon.0) 3121 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:10:53.119 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:52 smithi144 bash[66654]: audit 2024-04-07T17:10:52.118585+0000 mon.a (mon.0) 3122 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:10:53.119 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:52 smithi144 bash[66654]: audit 2024-04-07T17:10:52.118791+0000 mon.a (mon.0) 3123 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:10:53.119 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:52 smithi144 bash[66654]: audit 2024-04-07T17:10:52.120835+0000 mon.a (mon.0) 3124 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:10:53.119 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:52 smithi144 bash[66654]: audit 2024-04-07T17:10:52.120917+0000 mon.b (mon.1) 17 : audit [INF] from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T17:10:53.119 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:52 smithi144 bash[66654]: audit 2024-04-07T17:10:52.121029+0000 mon.a (mon.0) 3125 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:10:53.119 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:52 smithi144 bash[66654]: audit 2024-04-07T17:10:52.121073+0000 mon.b (mon.1) 18 : audit [INF] from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T17:10:53.120 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:52 smithi144 bash[66654]: audit 2024-04-07T17:10:52.121210+0000 mon.a (mon.0) 3126 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:10:53.120 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:52 smithi144 bash[66654]: audit 2024-04-07T17:10:52.122693+0000 mon.c (mon.2) 77 : audit [INF] from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T17:10:53.120 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:52 smithi144 bash[66654]: audit 2024-04-07T17:10:52.122806+0000 mon.c (mon.2) 78 : audit [INF] from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T17:10:53.120 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:52 smithi144 bash[66654]: audit 2024-04-07T17:10:52.122868+0000 mon.a (mon.0) 3127 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:10:53.120 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:52 smithi144 bash[66654]: audit 2024-04-07T17:10:52.123090+0000 mon.a (mon.0) 3128 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:10:53.120 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:52 smithi144 bash[66654]: audit 2024-04-07T17:10:52.123353+0000 mon.a (mon.0) 3129 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:10:53.120 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:52 smithi144 bash[66654]: cluster 2024-04-07T17:10:52.739992+0000 mon.a (mon.0) 3130 : cluster [INF] pool 'test-librbd-smithi100-115303-23' no longer out of quota; removing NO_QUOTA flag 2024-04-07T17:10:53.120 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:52 smithi144 bash[66654]: cluster 2024-04-07T17:10:52.740962+0000 mon.a (mon.0) 3131 : cluster [INF] Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-04-07T17:10:53.120 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:52 smithi144 bash[66654]: cluster 2024-04-07T17:10:52.754332+0000 mon.a (mon.0) 3132 : cluster [DBG] osdmap e1642: 8 total, 8 up, 8 in 2024-04-07T17:10:53.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:52 smithi100 bash[74942]: cluster 2024-04-07T17:10:51.435491+0000 mgr.y (mgr.25175) 1665 : cluster [DBG] pgmap v2880: 457 pgs: 7 creating+peering, 53 unknown, 397 active+clean; 8.6 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 6.2 KiB/s rd, 1.0 MiB/s wr, 13 op/s 2024-04-07T17:10:53.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:52 smithi100 bash[74942]: audit 2024-04-07T17:10:51.824157+0000 mon.a (mon.0) 3101 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T17:10:53.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:52 smithi100 bash[74942]: cluster 2024-04-07T17:10:51.834791+0000 mon.a (mon.0) 3102 : cluster [DBG] osdmap e1641: 8 total, 8 up, 8 in 2024-04-07T17:10:53.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:52 smithi100 bash[74942]: audit 2024-04-07T17:10:51.877339+0000 mon.a (mon.0) 3103 : audit [INF] from='client.? ' entity='client.admin' 2024-04-07T17:10:53.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:52 smithi100 bash[74942]: audit 2024-04-07T17:10:51.912816+0000 mon.a (mon.0) 3104 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:10:53.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:52 smithi100 bash[74942]: audit 2024-04-07T17:10:51.916876+0000 mon.a (mon.0) 3105 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:10:53.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:52 smithi100 bash[74942]: audit 2024-04-07T17:10:51.918410+0000 mon.a (mon.0) 3106 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:10:53.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:52 smithi100 bash[74942]: audit 2024-04-07T17:10:51.919471+0000 mon.a (mon.0) 3107 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:10:53.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:52 smithi100 bash[74942]: audit 2024-04-07T17:10:51.925605+0000 mon.a (mon.0) 3108 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:10:53.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:52 smithi100 bash[74942]: audit 2024-04-07T17:10:51.925916+0000 mon.a (mon.0) 3109 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config-key dump", "format": "json"}]: dispatch 2024-04-07T17:10:53.205 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:52 smithi100 bash[74942]: audit 2024-04-07T17:10:52.101534+0000 mon.a (mon.0) 3110 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:10:53.205 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:52 smithi100 bash[74942]: audit 2024-04-07T17:10:52.101744+0000 mon.a (mon.0) 3111 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:10:53.205 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:52 smithi100 bash[74942]: audit 2024-04-07T17:10:52.101898+0000 mon.a (mon.0) 3112 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:10:53.205 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:52 smithi100 bash[74942]: audit 2024-04-07T17:10:52.104004+0000 mon.a (mon.0) 3113 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:10:53.205 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:52 smithi100 bash[74942]: audit 2024-04-07T17:10:52.104144+0000 mon.a (mon.0) 3114 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:10:53.205 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:52 smithi100 bash[74942]: audit 2024-04-07T17:10:52.104284+0000 mon.a (mon.0) 3115 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:10:53.205 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:52 smithi100 bash[74942]: audit 2024-04-07T17:10:52.105564+0000 mon.a (mon.0) 3116 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:10:53.205 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:52 smithi100 bash[74942]: audit 2024-04-07T17:10:52.105777+0000 mon.a (mon.0) 3117 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:10:53.205 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:52 smithi100 bash[76202]: cluster 2024-04-07T17:10:51.435491+0000 mgr.y (mgr.25175) 1665 : cluster [DBG] pgmap v2880: 457 pgs: 7 creating+peering, 53 unknown, 397 active+clean; 8.6 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 6.2 KiB/s rd, 1.0 MiB/s wr, 13 op/s 2024-04-07T17:10:53.205 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:52 smithi100 bash[76202]: audit 2024-04-07T17:10:51.824157+0000 mon.a (mon.0) 3101 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T17:10:53.205 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:52 smithi100 bash[76202]: cluster 2024-04-07T17:10:51.834791+0000 mon.a (mon.0) 3102 : cluster [DBG] osdmap e1641: 8 total, 8 up, 8 in 2024-04-07T17:10:53.206 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:52 smithi100 bash[76202]: audit 2024-04-07T17:10:51.877339+0000 mon.a (mon.0) 3103 : audit [INF] from='client.? ' entity='client.admin' 2024-04-07T17:10:53.206 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:52 smithi100 bash[76202]: audit 2024-04-07T17:10:51.912816+0000 mon.a (mon.0) 3104 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:10:53.206 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:52 smithi100 bash[76202]: audit 2024-04-07T17:10:51.916876+0000 mon.a (mon.0) 3105 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:10:53.206 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:52 smithi100 bash[76202]: audit 2024-04-07T17:10:51.918410+0000 mon.a (mon.0) 3106 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:10:53.206 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:52 smithi100 bash[76202]: audit 2024-04-07T17:10:51.919471+0000 mon.a (mon.0) 3107 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:10:53.206 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:52 smithi100 bash[76202]: audit 2024-04-07T17:10:51.925605+0000 mon.a (mon.0) 3108 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:10:53.206 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:52 smithi100 bash[76202]: audit 2024-04-07T17:10:51.925916+0000 mon.a (mon.0) 3109 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config-key dump", "format": "json"}]: dispatch 2024-04-07T17:10:53.206 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:52 smithi100 bash[76202]: audit 2024-04-07T17:10:52.101534+0000 mon.a (mon.0) 3110 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:10:53.206 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:52 smithi100 bash[76202]: audit 2024-04-07T17:10:52.101744+0000 mon.a (mon.0) 3111 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:10:53.206 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:52 smithi100 bash[76202]: audit 2024-04-07T17:10:52.101898+0000 mon.a (mon.0) 3112 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:10:53.206 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:52 smithi100 bash[76202]: audit 2024-04-07T17:10:52.104004+0000 mon.a (mon.0) 3113 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:10:53.206 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:52 smithi100 bash[76202]: audit 2024-04-07T17:10:52.104144+0000 mon.a (mon.0) 3114 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:10:53.207 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:52 smithi100 bash[76202]: audit 2024-04-07T17:10:52.104284+0000 mon.a (mon.0) 3115 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:10:53.207 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:52 smithi100 bash[76202]: audit 2024-04-07T17:10:52.105564+0000 mon.a (mon.0) 3116 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:10:53.207 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:52 smithi100 bash[76202]: audit 2024-04-07T17:10:52.105777+0000 mon.a (mon.0) 3117 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:10:53.207 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:52 smithi100 bash[74942]: audit 2024-04-07T17:10:52.105912+0000 mon.a (mon.0) 3118 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:10:53.207 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:52 smithi100 bash[74942]: audit 2024-04-07T17:10:52.118079+0000 mon.a (mon.0) 3119 : audit [INF] from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T17:10:53.207 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:52 smithi100 bash[74942]: audit 2024-04-07T17:10:52.118145+0000 mon.a (mon.0) 3120 : audit [INF] from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T17:10:53.207 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:52 smithi100 bash[74942]: audit 2024-04-07T17:10:52.118364+0000 mon.a (mon.0) 3121 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:10:53.207 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:52 smithi100 bash[74942]: audit 2024-04-07T17:10:52.118585+0000 mon.a (mon.0) 3122 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:10:53.207 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:52 smithi100 bash[74942]: audit 2024-04-07T17:10:52.118791+0000 mon.a (mon.0) 3123 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:10:53.207 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:52 smithi100 bash[74942]: audit 2024-04-07T17:10:52.120835+0000 mon.a (mon.0) 3124 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:10:53.208 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:52 smithi100 bash[74942]: audit 2024-04-07T17:10:52.120917+0000 mon.b (mon.1) 17 : audit [INF] from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T17:10:53.208 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:52 smithi100 bash[74942]: audit 2024-04-07T17:10:52.121029+0000 mon.a (mon.0) 3125 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:10:53.208 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:52 smithi100 bash[74942]: audit 2024-04-07T17:10:52.121073+0000 mon.b (mon.1) 18 : audit [INF] from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T17:10:53.208 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:52 smithi100 bash[74942]: audit 2024-04-07T17:10:52.121210+0000 mon.a (mon.0) 3126 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:10:53.208 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:52 smithi100 bash[74942]: audit 2024-04-07T17:10:52.122693+0000 mon.c (mon.2) 77 : audit [INF] from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T17:10:53.208 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:52 smithi100 bash[74942]: audit 2024-04-07T17:10:52.122806+0000 mon.c (mon.2) 78 : audit [INF] from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T17:10:53.208 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:52 smithi100 bash[74942]: audit 2024-04-07T17:10:52.122868+0000 mon.a (mon.0) 3127 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:10:53.208 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:52 smithi100 bash[74942]: audit 2024-04-07T17:10:52.123090+0000 mon.a (mon.0) 3128 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:10:53.208 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:52 smithi100 bash[74942]: audit 2024-04-07T17:10:52.123353+0000 mon.a (mon.0) 3129 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:10:53.208 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:52 smithi100 bash[74942]: cluster 2024-04-07T17:10:52.739992+0000 mon.a (mon.0) 3130 : cluster [INF] pool 'test-librbd-smithi100-115303-23' no longer out of quota; removing NO_QUOTA flag 2024-04-07T17:10:53.208 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:52 smithi100 bash[74942]: cluster 2024-04-07T17:10:52.740962+0000 mon.a (mon.0) 3131 : cluster [INF] Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-04-07T17:10:53.208 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:52 smithi100 bash[74942]: cluster 2024-04-07T17:10:52.754332+0000 mon.a (mon.0) 3132 : cluster [DBG] osdmap e1642: 8 total, 8 up, 8 in 2024-04-07T17:10:53.208 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:52 smithi100 bash[76202]: audit 2024-04-07T17:10:52.105912+0000 mon.a (mon.0) 3118 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:10:53.208 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:52 smithi100 bash[76202]: audit 2024-04-07T17:10:52.118079+0000 mon.a (mon.0) 3119 : audit [INF] from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T17:10:53.209 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:52 smithi100 bash[76202]: audit 2024-04-07T17:10:52.118145+0000 mon.a (mon.0) 3120 : audit [INF] from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T17:10:53.209 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:52 smithi100 bash[76202]: audit 2024-04-07T17:10:52.118364+0000 mon.a (mon.0) 3121 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:10:53.209 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:52 smithi100 bash[76202]: audit 2024-04-07T17:10:52.118585+0000 mon.a (mon.0) 3122 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:10:53.209 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:52 smithi100 bash[76202]: audit 2024-04-07T17:10:52.118791+0000 mon.a (mon.0) 3123 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:10:53.209 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:52 smithi100 bash[76202]: audit 2024-04-07T17:10:52.120835+0000 mon.a (mon.0) 3124 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:10:53.209 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:52 smithi100 bash[76202]: audit 2024-04-07T17:10:52.120917+0000 mon.b (mon.1) 17 : audit [INF] from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T17:10:53.209 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:52 smithi100 bash[76202]: audit 2024-04-07T17:10:52.121029+0000 mon.a (mon.0) 3125 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:10:53.209 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:52 smithi100 bash[76202]: audit 2024-04-07T17:10:52.121073+0000 mon.b (mon.1) 18 : audit [INF] from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T17:10:53.209 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:52 smithi100 bash[76202]: audit 2024-04-07T17:10:52.121210+0000 mon.a (mon.0) 3126 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:10:53.209 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:52 smithi100 bash[76202]: audit 2024-04-07T17:10:52.122693+0000 mon.c (mon.2) 77 : audit [INF] from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T17:10:53.209 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:52 smithi100 bash[76202]: audit 2024-04-07T17:10:52.122806+0000 mon.c (mon.2) 78 : audit [INF] from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T17:10:53.209 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:52 smithi100 bash[76202]: audit 2024-04-07T17:10:52.122868+0000 mon.a (mon.0) 3127 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:10:53.209 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:52 smithi100 bash[76202]: audit 2024-04-07T17:10:52.123090+0000 mon.a (mon.0) 3128 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:10:53.209 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:52 smithi100 bash[76202]: audit 2024-04-07T17:10:52.123353+0000 mon.a (mon.0) 3129 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:10:53.210 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:52 smithi100 bash[76202]: cluster 2024-04-07T17:10:52.739992+0000 mon.a (mon.0) 3130 : cluster [INF] pool 'test-librbd-smithi100-115303-23' no longer out of quota; removing NO_QUOTA flag 2024-04-07T17:10:53.210 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:52 smithi100 bash[76202]: cluster 2024-04-07T17:10:52.740962+0000 mon.a (mon.0) 3131 : cluster [INF] Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-04-07T17:10:53.210 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:52 smithi100 bash[76202]: cluster 2024-04-07T17:10:52.754332+0000 mon.a (mon.0) 3132 : cluster [DBG] osdmap e1642: 8 total, 8 up, 8 in 2024-04-07T17:10:55.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:54 smithi144 bash[66654]: cluster 2024-04-07T17:10:53.437103+0000 mgr.y (mgr.25175) 1666 : cluster [DBG] pgmap v2883: 457 pgs: 7 creating+peering, 12 unknown, 438 active+clean; 4.6 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 30 KiB/s rd, 514 KiB/s wr, 41 op/s 2024-04-07T17:10:55.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:54 smithi144 bash[66654]: cluster 2024-04-07T17:10:53.754676+0000 mon.a (mon.0) 3133 : cluster [DBG] osdmap e1643: 8 total, 8 up, 8 in 2024-04-07T17:10:55.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:54 smithi144 bash[66654]: audit 2024-04-07T17:10:53.826420+0000 mon.c (mon.2) 79 : audit [INF] from='client.? 172.21.15.100:0/3213618994' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "test-librbd-smithi100-115303-25", "field": "max_bytes", "val": "10485760"}]: dispatch 2024-04-07T17:10:55.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:54 smithi144 bash[66654]: cluster 2024-04-07T17:10:53.826725+0000 mon.a (mon.0) 3134 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T17:10:55.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:54 smithi144 bash[66654]: audit 2024-04-07T17:10:53.827686+0000 mon.a (mon.0) 3135 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "test-librbd-smithi100-115303-25", "field": "max_bytes", "val": "10485760"}]: dispatch 2024-04-07T17:10:55.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:54 smithi100 bash[74942]: cluster 2024-04-07T17:10:53.437103+0000 mgr.y (mgr.25175) 1666 : cluster [DBG] pgmap v2883: 457 pgs: 7 creating+peering, 12 unknown, 438 active+clean; 4.6 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 30 KiB/s rd, 514 KiB/s wr, 41 op/s 2024-04-07T17:10:55.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:54 smithi100 bash[74942]: cluster 2024-04-07T17:10:53.754676+0000 mon.a (mon.0) 3133 : cluster [DBG] osdmap e1643: 8 total, 8 up, 8 in 2024-04-07T17:10:55.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:54 smithi100 bash[74942]: audit 2024-04-07T17:10:53.826420+0000 mon.c (mon.2) 79 : audit [INF] from='client.? 172.21.15.100:0/3213618994' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "test-librbd-smithi100-115303-25", "field": "max_bytes", "val": "10485760"}]: dispatch 2024-04-07T17:10:55.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:54 smithi100 bash[74942]: cluster 2024-04-07T17:10:53.826725+0000 mon.a (mon.0) 3134 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T17:10:55.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:54 smithi100 bash[74942]: audit 2024-04-07T17:10:53.827686+0000 mon.a (mon.0) 3135 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "test-librbd-smithi100-115303-25", "field": "max_bytes", "val": "10485760"}]: dispatch 2024-04-07T17:10:55.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:54 smithi100 bash[76202]: cluster 2024-04-07T17:10:53.437103+0000 mgr.y (mgr.25175) 1666 : cluster [DBG] pgmap v2883: 457 pgs: 7 creating+peering, 12 unknown, 438 active+clean; 4.6 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 30 KiB/s rd, 514 KiB/s wr, 41 op/s 2024-04-07T17:10:55.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:54 smithi100 bash[76202]: cluster 2024-04-07T17:10:53.754676+0000 mon.a (mon.0) 3133 : cluster [DBG] osdmap e1643: 8 total, 8 up, 8 in 2024-04-07T17:10:55.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:54 smithi100 bash[76202]: audit 2024-04-07T17:10:53.826420+0000 mon.c (mon.2) 79 : audit [INF] from='client.? 172.21.15.100:0/3213618994' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "test-librbd-smithi100-115303-25", "field": "max_bytes", "val": "10485760"}]: dispatch 2024-04-07T17:10:55.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:54 smithi100 bash[76202]: cluster 2024-04-07T17:10:53.826725+0000 mon.a (mon.0) 3134 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T17:10:55.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:54 smithi100 bash[76202]: audit 2024-04-07T17:10:53.827686+0000 mon.a (mon.0) 3135 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "test-librbd-smithi100-115303-25", "field": "max_bytes", "val": "10485760"}]: dispatch 2024-04-07T17:10:56.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:55 smithi144 bash[66654]: audit 2024-04-07T17:10:54.757613+0000 mon.a (mon.0) 3136 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "test-librbd-smithi100-115303-25", "field": "max_bytes", "val": "10485760"}]': finished 2024-04-07T17:10:56.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:55 smithi144 bash[66654]: cluster 2024-04-07T17:10:54.764563+0000 mon.a (mon.0) 3137 : cluster [DBG] osdmap e1644: 8 total, 8 up, 8 in 2024-04-07T17:10:56.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:55 smithi100 bash[74942]: audit 2024-04-07T17:10:54.757613+0000 mon.a (mon.0) 3136 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "test-librbd-smithi100-115303-25", "field": "max_bytes", "val": "10485760"}]': finished 2024-04-07T17:10:56.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:55 smithi100 bash[74942]: cluster 2024-04-07T17:10:54.764563+0000 mon.a (mon.0) 3137 : cluster [DBG] osdmap e1644: 8 total, 8 up, 8 in 2024-04-07T17:10:56.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:55 smithi100 bash[76202]: audit 2024-04-07T17:10:54.757613+0000 mon.a (mon.0) 3136 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "test-librbd-smithi100-115303-25", "field": "max_bytes", "val": "10485760"}]': finished 2024-04-07T17:10:56.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:55 smithi100 bash[76202]: cluster 2024-04-07T17:10:54.764563+0000 mon.a (mon.0) 3137 : cluster [DBG] osdmap e1644: 8 total, 8 up, 8 in 2024-04-07T17:10:57.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:56 smithi144 bash[66654]: cluster 2024-04-07T17:10:55.438621+0000 mgr.y (mgr.25175) 1667 : cluster [DBG] pgmap v2886: 457 pgs: 457 active+clean; 4.6 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 24 KiB/s rd, 1.7 KiB/s wr, 31 op/s 2024-04-07T17:10:57.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:56 smithi100 bash[74942]: cluster 2024-04-07T17:10:55.438621+0000 mgr.y (mgr.25175) 1667 : cluster [DBG] pgmap v2886: 457 pgs: 457 active+clean; 4.6 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 24 KiB/s rd, 1.7 KiB/s wr, 31 op/s 2024-04-07T17:10:57.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:56 smithi100 bash[76202]: cluster 2024-04-07T17:10:55.438621+0000 mgr.y (mgr.25175) 1667 : cluster [DBG] pgmap v2886: 457 pgs: 457 active+clean; 4.6 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 24 KiB/s rd, 1.7 KiB/s wr, 31 op/s 2024-04-07T17:10:58.703 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:10:58 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:10:58] "GET /metrics HTTP/1.1" 200 55812 "" "Prometheus/2.43.0" 2024-04-07T17:10:59.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:10:58 smithi144 bash[66654]: cluster 2024-04-07T17:10:57.441340+0000 mgr.y (mgr.25175) 1668 : cluster [DBG] pgmap v2887: 457 pgs: 457 active+clean; 7.6 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 27 KiB/s rd, 549 KiB/s wr, 37 op/s 2024-04-07T17:10:59.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:10:58 smithi100 bash[74942]: cluster 2024-04-07T17:10:57.441340+0000 mgr.y (mgr.25175) 1668 : cluster [DBG] pgmap v2887: 457 pgs: 457 active+clean; 7.6 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 27 KiB/s rd, 549 KiB/s wr, 37 op/s 2024-04-07T17:10:59.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:10:58 smithi100 bash[76202]: cluster 2024-04-07T17:10:57.441340+0000 mgr.y (mgr.25175) 1668 : cluster [DBG] pgmap v2887: 457 pgs: 457 active+clean; 7.6 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 27 KiB/s rd, 549 KiB/s wr, 37 op/s 2024-04-07T17:11:01.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:11:00 smithi144 bash[66654]: cluster 2024-04-07T17:10:59.443644+0000 mgr.y (mgr.25175) 1669 : cluster [DBG] pgmap v2888: 457 pgs: 457 active+clean; 7.6 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 10 KiB/s rd, 461 KiB/s wr, 17 op/s 2024-04-07T17:11:01.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:11:00 smithi100 bash[74942]: cluster 2024-04-07T17:10:59.443644+0000 mgr.y (mgr.25175) 1669 : cluster [DBG] pgmap v2888: 457 pgs: 457 active+clean; 7.6 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 10 KiB/s rd, 461 KiB/s wr, 17 op/s 2024-04-07T17:11:01.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:11:00 smithi100 bash[76202]: cluster 2024-04-07T17:10:59.443644+0000 mgr.y (mgr.25175) 1669 : cluster [DBG] pgmap v2888: 457 pgs: 457 active+clean; 7.6 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 10 KiB/s rd, 461 KiB/s wr, 17 op/s 2024-04-07T17:11:02.616 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:11:02 smithi144 bash[95233]: ts=2024-04-07T17:11:02.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:11:03.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:11:02 smithi144 bash[66654]: cluster 2024-04-07T17:11:01.445096+0000 mgr.y (mgr.25175) 1670 : cluster [DBG] pgmap v2889: 457 pgs: 457 active+clean; 7.6 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 8.6 KiB/s rd, 385 KiB/s wr, 14 op/s 2024-04-07T17:11:03.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:11:02 smithi144 bash[66654]: audit 2024-04-07T17:11:01.818685+0000 mon.a (mon.0) 3138 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:11:03.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:11:02 smithi144 bash[66654]: cluster 2024-04-07T17:11:02.757019+0000 mon.a (mon.0) 3139 : cluster [DBG] osdmap e1645: 8 total, 8 up, 8 in 2024-04-07T17:11:03.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:11:02 smithi100 bash[74942]: cluster 2024-04-07T17:11:01.445096+0000 mgr.y (mgr.25175) 1670 : cluster [DBG] pgmap v2889: 457 pgs: 457 active+clean; 7.6 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 8.6 KiB/s rd, 385 KiB/s wr, 14 op/s 2024-04-07T17:11:03.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:11:02 smithi100 bash[74942]: audit 2024-04-07T17:11:01.818685+0000 mon.a (mon.0) 3138 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:11:03.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:11:02 smithi100 bash[74942]: cluster 2024-04-07T17:11:02.757019+0000 mon.a (mon.0) 3139 : cluster [DBG] osdmap e1645: 8 total, 8 up, 8 in 2024-04-07T17:11:03.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:11:02 smithi100 bash[76202]: cluster 2024-04-07T17:11:01.445096+0000 mgr.y (mgr.25175) 1670 : cluster [DBG] pgmap v2889: 457 pgs: 457 active+clean; 7.6 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 8.6 KiB/s rd, 385 KiB/s wr, 14 op/s 2024-04-07T17:11:03.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:11:02 smithi100 bash[76202]: audit 2024-04-07T17:11:01.818685+0000 mon.a (mon.0) 3138 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:11:03.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:11:02 smithi100 bash[76202]: cluster 2024-04-07T17:11:02.757019+0000 mon.a (mon.0) 3139 : cluster [DBG] osdmap e1645: 8 total, 8 up, 8 in 2024-04-07T17:11:05.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:11:04 smithi144 bash[66654]: cluster 2024-04-07T17:11:03.446625+0000 mgr.y (mgr.25175) 1671 : cluster [DBG] pgmap v2891: 457 pgs: 457 active+clean; 13 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 7.9 KiB/s rd, 944 KiB/s wr, 13 op/s 2024-04-07T17:11:05.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:11:04 smithi100 bash[74942]: cluster 2024-04-07T17:11:03.446625+0000 mgr.y (mgr.25175) 1671 : cluster [DBG] pgmap v2891: 457 pgs: 457 active+clean; 13 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 7.9 KiB/s rd, 944 KiB/s wr, 13 op/s 2024-04-07T17:11:05.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:11:04 smithi100 bash[76202]: cluster 2024-04-07T17:11:03.446625+0000 mgr.y (mgr.25175) 1671 : cluster [DBG] pgmap v2891: 457 pgs: 457 active+clean; 13 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 7.9 KiB/s rd, 944 KiB/s wr, 13 op/s 2024-04-07T17:11:07.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:11:06 smithi144 bash[66654]: cluster 2024-04-07T17:11:05.448827+0000 mgr.y (mgr.25175) 1672 : cluster [DBG] pgmap v2892: 457 pgs: 457 active+clean; 13 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 7.1 KiB/s rd, 819 KiB/s wr, 11 op/s 2024-04-07T17:11:07.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:11:06 smithi100 bash[74942]: cluster 2024-04-07T17:11:05.448827+0000 mgr.y (mgr.25175) 1672 : cluster [DBG] pgmap v2892: 457 pgs: 457 active+clean; 13 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 7.1 KiB/s rd, 819 KiB/s wr, 11 op/s 2024-04-07T17:11:07.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:11:06 smithi100 bash[76202]: cluster 2024-04-07T17:11:05.448827+0000 mgr.y (mgr.25175) 1672 : cluster [DBG] pgmap v2892: 457 pgs: 457 active+clean; 13 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 7.1 KiB/s rd, 819 KiB/s wr, 11 op/s 2024-04-07T17:11:08.703 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:11:08 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:11:08] "GET /metrics HTTP/1.1" 200 60735 "" "Prometheus/2.43.0" 2024-04-07T17:11:09.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:11:08 smithi144 bash[66654]: cluster 2024-04-07T17:11:07.451601+0000 mgr.y (mgr.25175) 1673 : cluster [DBG] pgmap v2893: 457 pgs: 457 active+clean; 13 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 1.6 KiB/s rd, 512 KiB/s wr, 3 op/s 2024-04-07T17:11:09.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:11:08 smithi100 bash[74942]: cluster 2024-04-07T17:11:07.451601+0000 mgr.y (mgr.25175) 1673 : cluster [DBG] pgmap v2893: 457 pgs: 457 active+clean; 13 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 1.6 KiB/s rd, 512 KiB/s wr, 3 op/s 2024-04-07T17:11:09.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:11:08 smithi100 bash[76202]: cluster 2024-04-07T17:11:07.451601+0000 mgr.y (mgr.25175) 1673 : cluster [DBG] pgmap v2893: 457 pgs: 457 active+clean; 13 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 1.6 KiB/s rd, 512 KiB/s wr, 3 op/s 2024-04-07T17:11:11.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:11:10 smithi144 bash[66654]: cluster 2024-04-07T17:11:09.453744+0000 mgr.y (mgr.25175) 1674 : cluster [DBG] pgmap v2894: 457 pgs: 457 active+clean; 20 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 1022 B/s rd, 1.2 MiB/s wr, 2 op/s 2024-04-07T17:11:11.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:11:10 smithi100 bash[74942]: cluster 2024-04-07T17:11:09.453744+0000 mgr.y (mgr.25175) 1674 : cluster [DBG] pgmap v2894: 457 pgs: 457 active+clean; 20 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 1022 B/s rd, 1.2 MiB/s wr, 2 op/s 2024-04-07T17:11:11.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:11:10 smithi100 bash[76202]: cluster 2024-04-07T17:11:09.453744+0000 mgr.y (mgr.25175) 1674 : cluster [DBG] pgmap v2894: 457 pgs: 457 active+clean; 20 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 1022 B/s rd, 1.2 MiB/s wr, 2 op/s 2024-04-07T17:11:12.617 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:11:12 smithi144 bash[95233]: ts=2024-04-07T17:11:12.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:11:13.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:11:12 smithi144 bash[66654]: cluster 2024-04-07T17:11:11.455238+0000 mgr.y (mgr.25175) 1675 : cluster [DBG] pgmap v2895: 457 pgs: 457 active+clean; 20 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 1022 B/s rd, 1.2 MiB/s wr, 2 op/s 2024-04-07T17:11:13.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:11:12 smithi144 bash[66654]: cluster 2024-04-07T17:11:12.754130+0000 mon.a (mon.0) 3140 : cluster [WRN] pool 'test-librbd-smithi100-115303-25' is full (reached quota's max_bytes: 10 MiB) 2024-04-07T17:11:13.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:11:12 smithi144 bash[66654]: cluster 2024-04-07T17:11:12.754950+0000 mon.a (mon.0) 3141 : cluster [WRN] Health check failed: 1 pool(s) full (POOL_FULL) 2024-04-07T17:11:13.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:11:12 smithi144 bash[66654]: cluster 2024-04-07T17:11:12.765756+0000 mon.a (mon.0) 3142 : cluster [DBG] osdmap e1646: 8 total, 8 up, 8 in 2024-04-07T17:11:13.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:11:12 smithi100 bash[74942]: cluster 2024-04-07T17:11:11.455238+0000 mgr.y (mgr.25175) 1675 : cluster [DBG] pgmap v2895: 457 pgs: 457 active+clean; 20 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 1022 B/s rd, 1.2 MiB/s wr, 2 op/s 2024-04-07T17:11:13.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:11:12 smithi100 bash[74942]: cluster 2024-04-07T17:11:12.754130+0000 mon.a (mon.0) 3140 : cluster [WRN] pool 'test-librbd-smithi100-115303-25' is full (reached quota's max_bytes: 10 MiB) 2024-04-07T17:11:13.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:11:12 smithi100 bash[74942]: cluster 2024-04-07T17:11:12.754950+0000 mon.a (mon.0) 3141 : cluster [WRN] Health check failed: 1 pool(s) full (POOL_FULL) 2024-04-07T17:11:13.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:11:12 smithi100 bash[74942]: cluster 2024-04-07T17:11:12.765756+0000 mon.a (mon.0) 3142 : cluster [DBG] osdmap e1646: 8 total, 8 up, 8 in 2024-04-07T17:11:13.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:11:12 smithi100 bash[76202]: cluster 2024-04-07T17:11:11.455238+0000 mgr.y (mgr.25175) 1675 : cluster [DBG] pgmap v2895: 457 pgs: 457 active+clean; 20 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 1022 B/s rd, 1.2 MiB/s wr, 2 op/s 2024-04-07T17:11:13.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:11:12 smithi100 bash[76202]: cluster 2024-04-07T17:11:12.754130+0000 mon.a (mon.0) 3140 : cluster [WRN] pool 'test-librbd-smithi100-115303-25' is full (reached quota's max_bytes: 10 MiB) 2024-04-07T17:11:13.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:11:12 smithi100 bash[76202]: cluster 2024-04-07T17:11:12.754950+0000 mon.a (mon.0) 3141 : cluster [WRN] Health check failed: 1 pool(s) full (POOL_FULL) 2024-04-07T17:11:13.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:11:12 smithi100 bash[76202]: cluster 2024-04-07T17:11:12.765756+0000 mon.a (mon.0) 3142 : cluster [DBG] osdmap e1646: 8 total, 8 up, 8 in 2024-04-07T17:11:13.257 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.RemoveFullTryDataPool (24952 ms) 2024-04-07T17:11:13.257 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.TestIO 2024-04-07T17:11:13.257 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:11:15.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:11:14 smithi144 bash[66654]: cluster 2024-04-07T17:11:13.456996+0000 mgr.y (mgr.25175) 1676 : cluster [DBG] pgmap v2897: 457 pgs: 457 active+clean; 21 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 1022 B/s rd, 819 KiB/s wr, 2 op/s 2024-04-07T17:11:15.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:11:14 smithi100 bash[74942]: cluster 2024-04-07T17:11:13.456996+0000 mgr.y (mgr.25175) 1676 : cluster [DBG] pgmap v2897: 457 pgs: 457 active+clean; 21 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 1022 B/s rd, 819 KiB/s wr, 2 op/s 2024-04-07T17:11:15.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:11:14 smithi100 bash[76202]: cluster 2024-04-07T17:11:13.456996+0000 mgr.y (mgr.25175) 1676 : cluster [DBG] pgmap v2897: 457 pgs: 457 active+clean; 21 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 1022 B/s rd, 819 KiB/s wr, 2 op/s 2024-04-07T17:11:17.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:11:16 smithi144 bash[66654]: cluster 2024-04-07T17:11:15.458979+0000 mgr.y (mgr.25175) 1677 : cluster [DBG] pgmap v2898: 457 pgs: 457 active+clean; 18 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 4.3 KiB/s rd, 922 KiB/s wr, 8 op/s 2024-04-07T17:11:17.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:11:16 smithi100 bash[74942]: cluster 2024-04-07T17:11:15.458979+0000 mgr.y (mgr.25175) 1677 : cluster [DBG] pgmap v2898: 457 pgs: 457 active+clean; 18 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 4.3 KiB/s rd, 922 KiB/s wr, 8 op/s 2024-04-07T17:11:17.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:11:16 smithi100 bash[76202]: cluster 2024-04-07T17:11:15.458979+0000 mgr.y (mgr.25175) 1677 : cluster [DBG] pgmap v2898: 457 pgs: 457 active+clean; 18 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 4.3 KiB/s rd, 922 KiB/s wr, 8 op/s 2024-04-07T17:11:17.954 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 512 2024-04-07T17:11:17.954 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 512 2024-04-07T17:11:17.954 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 512 2024-04-07T17:11:17.954 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 512 2024-04-07T17:11:17.954 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 512 2024-04-07T17:11:17.954 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:11:17.954 INFO:tasks.workunit.client.0.smithi100.stdout:started write 2024-04-07T17:11:17.955 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:11:17.955 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:11:17.955 INFO:tasks.workunit.client.0.smithi100.stdout:finished write 2024-04-07T17:11:17.955 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:11:17.955 INFO:tasks.workunit.client.0.smithi100.stdout:started write 2024-04-07T17:11:17.955 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:11:17.955 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:11:17.955 INFO:tasks.workunit.client.0.smithi100.stdout:finished write 2024-04-07T17:11:17.955 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:11:17.955 INFO:tasks.workunit.client.0.smithi100.stdout:started write 2024-04-07T17:11:17.955 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:11:17.956 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:11:17.956 INFO:tasks.workunit.client.0.smithi100.stdout:finished write 2024-04-07T17:11:17.956 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:11:17.956 INFO:tasks.workunit.client.0.smithi100.stdout:started write 2024-04-07T17:11:17.956 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:11:17.956 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:11:17.956 INFO:tasks.workunit.client.0.smithi100.stdout:finished write 2024-04-07T17:11:17.956 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:11:17.956 INFO:tasks.workunit.client.0.smithi100.stdout:started write 2024-04-07T17:11:17.956 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:11:17.956 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:11:17.957 INFO:tasks.workunit.client.0.smithi100.stdout:finished write 2024-04-07T17:11:17.957 INFO:tasks.workunit.client.0.smithi100.stdout:start compare and write 2024-04-07T17:11:17.957 INFO:tasks.workunit.client.0.smithi100.stdout:compare and wrote: 512 2024-04-07T17:11:17.957 INFO:tasks.workunit.client.0.smithi100.stdout:start compare and write 2024-04-07T17:11:17.957 INFO:tasks.workunit.client.0.smithi100.stdout:compare and wrote: 512 2024-04-07T17:11:17.957 INFO:tasks.workunit.client.0.smithi100.stdout:start compare and write 2024-04-07T17:11:17.957 INFO:tasks.workunit.client.0.smithi100.stdout:compare and wrote: 512 2024-04-07T17:11:17.957 INFO:tasks.workunit.client.0.smithi100.stdout:start compare and write 2024-04-07T17:11:17.957 INFO:tasks.workunit.client.0.smithi100.stdout:compare and wrote: 512 2024-04-07T17:11:17.957 INFO:tasks.workunit.client.0.smithi100.stdout:start compare and write 2024-04-07T17:11:17.957 INFO:tasks.workunit.client.0.smithi100.stdout:compare and wrote: 512 2024-04-07T17:11:17.957 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:11:17.957 INFO:tasks.workunit.client.0.smithi100.stdout:started aio compare and write 2024-04-07T17:11:17.958 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:11:17.958 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:11:17.958 INFO:tasks.workunit.client.0.smithi100.stdout:finished aio compare and write 2024-04-07T17:11:17.958 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:11:17.958 INFO:tasks.workunit.client.0.smithi100.stdout:started aio compare and write 2024-04-07T17:11:17.958 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:11:17.958 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:11:17.958 INFO:tasks.workunit.client.0.smithi100.stdout:finished aio compare and write 2024-04-07T17:11:17.958 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:11:17.958 INFO:tasks.workunit.client.0.smithi100.stdout:started aio compare and write 2024-04-07T17:11:17.958 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:11:17.958 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:11:17.959 INFO:tasks.workunit.client.0.smithi100.stdout:finished aio compare and write 2024-04-07T17:11:17.959 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:11:17.959 INFO:tasks.workunit.client.0.smithi100.stdout:started aio compare and write 2024-04-07T17:11:17.959 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:11:17.959 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:11:17.959 INFO:tasks.workunit.client.0.smithi100.stdout:finished aio compare and write 2024-04-07T17:11:17.959 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:11:17.959 INFO:tasks.workunit.client.0.smithi100.stdout:started aio compare and write 2024-04-07T17:11:17.959 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:11:17.959 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:11:17.959 INFO:tasks.workunit.client.0.smithi100.stdout:finished aio compare and write 2024-04-07T17:11:17.959 INFO:tasks.workunit.client.0.smithi100.stdout:read: 512 2024-04-07T17:11:17.959 INFO:tasks.workunit.client.0.smithi100.stdout:read: 512 2024-04-07T17:11:17.960 INFO:tasks.workunit.client.0.smithi100.stdout:read: 512 2024-04-07T17:11:17.960 INFO:tasks.workunit.client.0.smithi100.stdout:read: 512 2024-04-07T17:11:17.960 INFO:tasks.workunit.client.0.smithi100.stdout:read: 512 2024-04-07T17:11:17.960 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:11:17.960 INFO:tasks.workunit.client.0.smithi100.stdout:started read 2024-04-07T17:11:17.960 INFO:tasks.workunit.client.0.smithi100.stdout:read completion cb called! 2024-04-07T17:11:17.960 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 512 2024-04-07T17:11:17.960 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:11:17.960 INFO:tasks.workunit.client.0.smithi100.stdout:started read 2024-04-07T17:11:17.960 INFO:tasks.workunit.client.0.smithi100.stdout:read completion cb called! 2024-04-07T17:11:17.960 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 512 2024-04-07T17:11:17.960 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:11:17.961 INFO:tasks.workunit.client.0.smithi100.stdout:started read 2024-04-07T17:11:17.961 INFO:tasks.workunit.client.0.smithi100.stdout:read completion cb called! 2024-04-07T17:11:17.961 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 512 2024-04-07T17:11:17.961 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:11:17.961 INFO:tasks.workunit.client.0.smithi100.stdout:started read 2024-04-07T17:11:17.961 INFO:tasks.workunit.client.0.smithi100.stdout:read completion cb called! 2024-04-07T17:11:17.961 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 512 2024-04-07T17:11:17.961 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:11:17.961 INFO:tasks.workunit.client.0.smithi100.stdout:started read 2024-04-07T17:11:17.961 INFO:tasks.workunit.client.0.smithi100.stdout:read completion cb called! 2024-04-07T17:11:17.961 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 512 2024-04-07T17:11:17.961 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 512~512 = 512 2024-04-07T17:11:17.961 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:11:17.962 INFO:tasks.workunit.client.0.smithi100.stdout:aio discard: 1536~512 = 0 2024-04-07T17:11:17.962 INFO:tasks.workunit.client.0.smithi100.stdout:read: 512 2024-04-07T17:11:17.962 INFO:tasks.workunit.client.0.smithi100.stdout:read: 512 2024-04-07T17:11:17.962 INFO:tasks.workunit.client.0.smithi100.stdout:read: 512 2024-04-07T17:11:17.962 INFO:tasks.workunit.client.0.smithi100.stdout:read: 512 2024-04-07T17:11:17.962 INFO:tasks.workunit.client.0.smithi100.stdout:read: 512 2024-04-07T17:11:17.962 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 0 2024-04-07T17:11:17.962 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:11:17.962 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:11:17.962 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 0 2024-04-07T17:11:17.962 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:11:17.962 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:11:17.963 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 16384 2024-04-07T17:11:17.963 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:11:17.963 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:11:17.963 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 16384 2024-04-07T17:11:17.963 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:11:17.963 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:11:17.963 INFO:tasks.workunit.client.0.smithi100.stdout:expected fail, finished writesame 2024-04-07T17:11:17.963 INFO:tasks.workunit.client.0.smithi100.stdout:expected fail, finished writesame 2024-04-07T17:11:17.963 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 49152 2024-04-07T17:11:17.963 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:11:17.963 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:11:17.963 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 49152 2024-04-07T17:11:17.964 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:11:17.964 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:11:17.964 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 65536 2024-04-07T17:11:17.964 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:11:17.964 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:11:17.964 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 65536 2024-04-07T17:11:17.964 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:11:17.964 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:11:17.964 INFO:tasks.workunit.client.0.smithi100.stdout:expected fail, finished writesame 2024-04-07T17:11:17.964 INFO:tasks.workunit.client.0.smithi100.stdout:expected fail, finished writesame 2024-04-07T17:11:17.964 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 98304 2024-04-07T17:11:17.964 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:11:17.964 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:11:17.965 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 98304 2024-04-07T17:11:17.965 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:11:17.965 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:11:17.965 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 114688 2024-04-07T17:11:17.965 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:11:17.965 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:11:17.965 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 114688 2024-04-07T17:11:17.965 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:11:17.965 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:11:17.965 INFO:tasks.workunit.client.0.smithi100.stdout:expected fail, finished writesame 2024-04-07T17:11:17.965 INFO:tasks.workunit.client.0.smithi100.stdout:expected fail, finished writesame 2024-04-07T17:11:17.965 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 147456 2024-04-07T17:11:17.966 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:11:17.966 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:11:17.966 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 147456 2024-04-07T17:11:17.966 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:11:17.966 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:11:17.966 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 163840 2024-04-07T17:11:17.966 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:11:17.966 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:11:17.966 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 163840 2024-04-07T17:11:17.966 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:11:17.966 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:11:17.966 INFO:tasks.workunit.client.0.smithi100.stdout:expected fail, finished writesame 2024-04-07T17:11:17.966 INFO:tasks.workunit.client.0.smithi100.stdout:expected fail, finished writesame 2024-04-07T17:11:17.967 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 196608 2024-04-07T17:11:17.967 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:11:17.967 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:11:17.967 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 196608 2024-04-07T17:11:17.967 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:11:17.967 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:11:17.967 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 212992 2024-04-07T17:11:17.967 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:11:17.967 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:11:17.967 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 212992 2024-04-07T17:11:17.967 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:11:17.968 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:11:17.968 INFO:tasks.workunit.client.0.smithi100.stdout:expected fail, finished writesame 2024-04-07T17:11:17.968 INFO:tasks.workunit.client.0.smithi100.stdout:expected fail, finished writesame 2024-04-07T17:11:17.968 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:11:17.968 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:11:17.968 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:11:17.968 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:11:17.968 INFO:tasks.workunit.client.0.smithi100.stdout:finished writesame 2024-04-07T17:11:17.968 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:11:17.968 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:11:17.968 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:11:17.968 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:11:17.969 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:11:17.969 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:11:17.969 INFO:tasks.workunit.client.0.smithi100.stdout:finished writesame 2024-04-07T17:11:17.969 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:11:17.969 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:11:17.969 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:11:17.969 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:11:17.969 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:11:17.969 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:11:17.969 INFO:tasks.workunit.client.0.smithi100.stdout:finished writesame 2024-04-07T17:11:17.969 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:11:17.969 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:11:17.970 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:11:17.970 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:11:17.970 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:11:17.970 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:11:17.970 INFO:tasks.workunit.client.0.smithi100.stdout:finished writesame 2024-04-07T17:11:17.970 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:11:17.970 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:11:17.970 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:11:17.970 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:11:17.970 INFO:tasks.workunit.client.0.smithi100.stdout:expected fail, finished writesame 2024-04-07T17:11:17.970 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:11:17.971 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:11:17.971 INFO:tasks.workunit.client.0.smithi100.stdout:expected fail, finished writesame 2024-04-07T17:11:17.971 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:11:17.971 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:11:17.971 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:11:17.971 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:11:17.971 INFO:tasks.workunit.client.0.smithi100.stdout:finished writesame 2024-04-07T17:11:17.971 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:11:17.971 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:11:17.971 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:11:17.971 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:11:17.971 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:11:17.972 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:11:17.972 INFO:tasks.workunit.client.0.smithi100.stdout:finished writesame 2024-04-07T17:11:17.972 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:11:17.972 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:11:17.972 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:11:18.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:11:17 smithi144 bash[66654]: audit 2024-04-07T17:11:16.831381+0000 mon.a (mon.0) 3143 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:11:18.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:11:17 smithi144 bash[66654]: audit 2024-04-07T17:11:16.834833+0000 mon.a (mon.0) 3144 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:11:18.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:11:17 smithi100 bash[74942]: audit 2024-04-07T17:11:16.831381+0000 mon.a (mon.0) 3143 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:11:18.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:11:17 smithi100 bash[74942]: audit 2024-04-07T17:11:16.834833+0000 mon.a (mon.0) 3144 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:11:18.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:11:17 smithi100 bash[76202]: audit 2024-04-07T17:11:16.831381+0000 mon.a (mon.0) 3143 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:11:18.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:11:17 smithi100 bash[76202]: audit 2024-04-07T17:11:16.834833+0000 mon.a (mon.0) 3144 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:11:18.703 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:11:18 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:11:18] "GET /metrics HTTP/1.1" 200 60794 "" "Prometheus/2.43.0" 2024-04-07T17:11:19.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:11:18 smithi144 bash[66654]: cluster 2024-04-07T17:11:17.461723+0000 mgr.y (mgr.25175) 1678 : cluster [DBG] pgmap v2899: 457 pgs: 457 active+clean; 8.6 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 16 KiB/s rd, 1.0 MiB/s wr, 23 op/s 2024-04-07T17:11:19.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:11:18 smithi100 bash[74942]: cluster 2024-04-07T17:11:17.461723+0000 mgr.y (mgr.25175) 1678 : cluster [DBG] pgmap v2899: 457 pgs: 457 active+clean; 8.6 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 16 KiB/s rd, 1.0 MiB/s wr, 23 op/s 2024-04-07T17:11:19.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:11:18 smithi100 bash[76202]: cluster 2024-04-07T17:11:17.461723+0000 mgr.y (mgr.25175) 1678 : cluster [DBG] pgmap v2899: 457 pgs: 457 active+clean; 8.6 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 16 KiB/s rd, 1.0 MiB/s wr, 23 op/s 2024-04-07T17:11:21.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:11:20 smithi100 bash[74942]: cluster 2024-04-07T17:11:19.464418+0000 mgr.y (mgr.25175) 1679 : cluster [DBG] pgmap v2900: 457 pgs: 457 active+clean; 4.6 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 238 KiB/s rd, 421 KiB/s wr, 467 op/s 2024-04-07T17:11:21.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:11:20 smithi100 bash[74942]: audit 2024-04-07T17:11:19.939960+0000 mon.a (mon.0) 3145 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "99.6", "id": [4, 6]}]: dispatch 2024-04-07T17:11:21.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:11:20 smithi100 bash[76202]: cluster 2024-04-07T17:11:19.464418+0000 mgr.y (mgr.25175) 1679 : cluster [DBG] pgmap v2900: 457 pgs: 457 active+clean; 4.6 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 238 KiB/s rd, 421 KiB/s wr, 467 op/s 2024-04-07T17:11:21.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:11:20 smithi100 bash[76202]: audit 2024-04-07T17:11:19.939960+0000 mon.a (mon.0) 3145 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "99.6", "id": [4, 6]}]: dispatch 2024-04-07T17:11:21.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:11:20 smithi144 bash[66654]: cluster 2024-04-07T17:11:19.464418+0000 mgr.y (mgr.25175) 1679 : cluster [DBG] pgmap v2900: 457 pgs: 457 active+clean; 4.6 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 238 KiB/s rd, 421 KiB/s wr, 467 op/s 2024-04-07T17:11:21.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:11:20 smithi144 bash[66654]: audit 2024-04-07T17:11:19.939960+0000 mon.a (mon.0) 3145 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "99.6", "id": [4, 6]}]: dispatch 2024-04-07T17:11:22.029 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:11:22.029 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:11:22.029 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:11:22.029 INFO:tasks.workunit.client.0.smithi100.stdout:finished writesame 2024-04-07T17:11:22.029 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:11:22.029 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:11:22.029 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:11:22.029 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:11:22.029 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:11:22.029 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:11:22.029 INFO:tasks.workunit.client.0.smithi100.stdout:finished writesame 2024-04-07T17:11:22.029 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:11:22.030 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:11:22.030 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:11:22.030 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:11:22.030 INFO:tasks.workunit.client.0.smithi100.stdout:expected fail, finished writesame 2024-04-07T17:11:22.030 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:11:22.030 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:11:22.030 INFO:tasks.workunit.client.0.smithi100.stdout:expected fail, finished writesame 2024-04-07T17:11:22.030 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:11:22.030 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:11:22.030 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:11:22.030 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:11:22.030 INFO:tasks.workunit.client.0.smithi100.stdout:finished writesame 2024-04-07T17:11:22.030 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:11:22.031 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:11:22.031 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:11:22.031 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:11:22.031 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:11:22.031 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:11:22.031 INFO:tasks.workunit.client.0.smithi100.stdout:finished writesame 2024-04-07T17:11:22.031 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:11:22.031 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:11:22.031 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:11:22.031 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:11:22.031 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:11:22.031 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:11:22.031 INFO:tasks.workunit.client.0.smithi100.stdout:finished writesame 2024-04-07T17:11:22.031 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:11:22.032 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:11:22.032 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:11:22.032 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:11:22.032 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:11:22.032 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:11:22.032 INFO:tasks.workunit.client.0.smithi100.stdout:finished writesame 2024-04-07T17:11:22.032 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:11:22.032 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:11:22.032 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:11:22.032 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:11:22.032 INFO:tasks.workunit.client.0.smithi100.stdout:expected fail, finished writesame 2024-04-07T17:11:22.032 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:11:22.032 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:11:22.032 INFO:tasks.workunit.client.0.smithi100.stdout:expected fail, finished writesame 2024-04-07T17:11:22.033 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:11:22.033 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:11:22.033 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:11:22.033 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:11:22.033 INFO:tasks.workunit.client.0.smithi100.stdout:finished writesame 2024-04-07T17:11:22.033 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:11:22.033 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:11:22.033 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:11:22.033 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:11:22.033 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:11:22.033 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:11:22.033 INFO:tasks.workunit.client.0.smithi100.stdout:finished writesame 2024-04-07T17:11:22.033 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:11:22.033 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:11:22.033 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:11:22.034 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:11:22.034 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:11:22.034 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:11:22.034 INFO:tasks.workunit.client.0.smithi100.stdout:finished writesame 2024-04-07T17:11:22.034 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:11:22.034 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:11:22.034 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:11:22.034 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:11:22.034 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:11:22.034 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:11:22.034 INFO:tasks.workunit.client.0.smithi100.stdout:finished writesame 2024-04-07T17:11:22.034 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:11:22.034 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:11:22.034 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:11:22.035 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:11:22.035 INFO:tasks.workunit.client.0.smithi100.stdout:expected fail, finished writesame 2024-04-07T17:11:22.035 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:11:22.035 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:11:22.035 INFO:tasks.workunit.client.0.smithi100.stdout:expected fail, finished writesame 2024-04-07T17:11:22.035 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:11:22.035 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:11:22.035 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:11:22.035 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:11:22.035 INFO:tasks.workunit.client.0.smithi100.stdout:finished writesame 2024-04-07T17:11:22.035 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:11:22.035 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:11:22.035 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:11:22.035 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:11:22.035 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:11:22.036 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:11:22.036 INFO:tasks.workunit.client.0.smithi100.stdout:finished writesame 2024-04-07T17:11:22.036 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:11:22.036 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:11:22.036 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:11:22.036 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:11:22.036 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:11:22.036 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:11:22.036 INFO:tasks.workunit.client.0.smithi100.stdout:finished writesame 2024-04-07T17:11:22.036 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:11:22.036 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:11:22.036 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:11:22.036 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:11:22.036 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:11:22.036 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:11:22.037 INFO:tasks.workunit.client.0.smithi100.stdout:finished writesame 2024-04-07T17:11:22.037 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:11:22.037 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:11:22.037 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:11:22.037 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:11:22.037 INFO:tasks.workunit.client.0.smithi100.stdout:expected fail, finished writesame 2024-04-07T17:11:22.037 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:11:22.037 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:11:22.037 INFO:tasks.workunit.client.0.smithi100.stdout:expected fail, finished writesame 2024-04-07T17:11:22.037 INFO:tasks.workunit.client.0.smithi100.stdout:read completion cb called! 2024-04-07T17:11:22.037 INFO:tasks.workunit.client.0.smithi100.stdout:read completion cb called! 2024-04-07T17:11:22.037 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 512 2024-04-07T17:11:22.037 INFO:tasks.workunit.client.0.smithi100.stdout:read completion cb called! 2024-04-07T17:11:22.037 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.TestIO (8771 ms) 2024-04-07T17:11:22.037 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.TestEncryptionLUKS1 2024-04-07T17:11:22.038 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:11:22.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:11:21 smithi100 bash[76202]: audit 2024-04-07T17:11:20.843665+0000 mon.a (mon.0) 3146 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "99.6", "id": [4, 6]}]': finished 2024-04-07T17:11:22.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:11:21 smithi100 bash[76202]: cluster 2024-04-07T17:11:20.865779+0000 mon.a (mon.0) 3147 : cluster [DBG] osdmap e1647: 8 total, 8 up, 8 in 2024-04-07T17:11:22.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:11:21 smithi100 bash[76202]: cluster 2024-04-07T17:11:21.860383+0000 mon.a (mon.0) 3148 : cluster [DBG] osdmap e1648: 8 total, 8 up, 8 in 2024-04-07T17:11:22.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:11:21 smithi100 bash[74942]: audit 2024-04-07T17:11:20.843665+0000 mon.a (mon.0) 3146 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "99.6", "id": [4, 6]}]': finished 2024-04-07T17:11:22.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:11:21 smithi100 bash[74942]: cluster 2024-04-07T17:11:20.865779+0000 mon.a (mon.0) 3147 : cluster [DBG] osdmap e1647: 8 total, 8 up, 8 in 2024-04-07T17:11:22.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:11:21 smithi100 bash[74942]: cluster 2024-04-07T17:11:21.860383+0000 mon.a (mon.0) 3148 : cluster [DBG] osdmap e1648: 8 total, 8 up, 8 in 2024-04-07T17:11:22.317 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:11:21 smithi144 bash[66654]: audit 2024-04-07T17:11:20.843665+0000 mon.a (mon.0) 3146 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "99.6", "id": [4, 6]}]': finished 2024-04-07T17:11:22.317 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:11:21 smithi144 bash[66654]: cluster 2024-04-07T17:11:20.865779+0000 mon.a (mon.0) 3147 : cluster [DBG] osdmap e1647: 8 total, 8 up, 8 in 2024-04-07T17:11:22.317 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:11:21 smithi144 bash[66654]: cluster 2024-04-07T17:11:21.860383+0000 mon.a (mon.0) 3148 : cluster [DBG] osdmap e1648: 8 total, 8 up, 8 in 2024-04-07T17:11:22.616 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:11:22 smithi144 bash[95233]: ts=2024-04-07T17:11:22.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:11:23.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:11:22 smithi100 bash[76202]: cluster 2024-04-07T17:11:21.465568+0000 mgr.y (mgr.25175) 1680 : cluster [DBG] pgmap v2902: 457 pgs: 457 active+clean; 4.6 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 274 KiB/s rd, 366 KiB/s wr, 537 op/s 2024-04-07T17:11:23.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:11:22 smithi100 bash[76202]: cluster 2024-04-07T17:11:22.758142+0000 mon.a (mon.0) 3149 : cluster [INF] pool 'test-librbd-smithi100-115303-25' no longer out of quota; removing NO_QUOTA flag 2024-04-07T17:11:23.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:11:22 smithi100 bash[76202]: cluster 2024-04-07T17:11:22.758915+0000 mon.a (mon.0) 3150 : cluster [INF] Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-04-07T17:11:23.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:11:22 smithi100 bash[76202]: cluster 2024-04-07T17:11:22.775227+0000 mon.a (mon.0) 3151 : cluster [DBG] osdmap e1649: 8 total, 8 up, 8 in 2024-04-07T17:11:23.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:11:22 smithi100 bash[74942]: cluster 2024-04-07T17:11:21.465568+0000 mgr.y (mgr.25175) 1680 : cluster [DBG] pgmap v2902: 457 pgs: 457 active+clean; 4.6 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 274 KiB/s rd, 366 KiB/s wr, 537 op/s 2024-04-07T17:11:23.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:11:22 smithi100 bash[74942]: cluster 2024-04-07T17:11:22.758142+0000 mon.a (mon.0) 3149 : cluster [INF] pool 'test-librbd-smithi100-115303-25' no longer out of quota; removing NO_QUOTA flag 2024-04-07T17:11:23.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:11:22 smithi100 bash[74942]: cluster 2024-04-07T17:11:22.758915+0000 mon.a (mon.0) 3150 : cluster [INF] Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-04-07T17:11:23.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:11:22 smithi100 bash[74942]: cluster 2024-04-07T17:11:22.775227+0000 mon.a (mon.0) 3151 : cluster [DBG] osdmap e1649: 8 total, 8 up, 8 in 2024-04-07T17:11:23.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:11:22 smithi144 bash[66654]: cluster 2024-04-07T17:11:21.465568+0000 mgr.y (mgr.25175) 1680 : cluster [DBG] pgmap v2902: 457 pgs: 457 active+clean; 4.6 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 274 KiB/s rd, 366 KiB/s wr, 537 op/s 2024-04-07T17:11:23.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:11:22 smithi144 bash[66654]: cluster 2024-04-07T17:11:22.758142+0000 mon.a (mon.0) 3149 : cluster [INF] pool 'test-librbd-smithi100-115303-25' no longer out of quota; removing NO_QUOTA flag 2024-04-07T17:11:23.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:11:22 smithi144 bash[66654]: cluster 2024-04-07T17:11:22.758915+0000 mon.a (mon.0) 3150 : cluster [INF] Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-04-07T17:11:23.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:11:22 smithi144 bash[66654]: cluster 2024-04-07T17:11:22.775227+0000 mon.a (mon.0) 3151 : cluster [DBG] osdmap e1649: 8 total, 8 up, 8 in 2024-04-07T17:11:25.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:11:24 smithi100 bash[74942]: cluster 2024-04-07T17:11:23.467130+0000 mgr.y (mgr.25175) 1681 : cluster [DBG] pgmap v2905: 457 pgs: 457 active+clean; 6.6 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 697 KiB/s rd, 350 KiB/s wr, 1.40k op/s 2024-04-07T17:11:25.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:11:24 smithi100 bash[76202]: cluster 2024-04-07T17:11:23.467130+0000 mgr.y (mgr.25175) 1681 : cluster [DBG] pgmap v2905: 457 pgs: 457 active+clean; 6.6 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 697 KiB/s rd, 350 KiB/s wr, 1.40k op/s 2024-04-07T17:11:25.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:11:24 smithi144 bash[66654]: cluster 2024-04-07T17:11:23.467130+0000 mgr.y (mgr.25175) 1681 : cluster [DBG] pgmap v2905: 457 pgs: 457 active+clean; 6.6 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 697 KiB/s rd, 350 KiB/s wr, 1.40k op/s 2024-04-07T17:11:27.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:11:26 smithi100 bash[74942]: cluster 2024-04-07T17:11:25.469217+0000 mgr.y (mgr.25175) 1682 : cluster [DBG] pgmap v2906: 457 pgs: 457 active+clean; 6.6 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 326 KiB/s rd, 163 KiB/s wr, 658 op/s 2024-04-07T17:11:27.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:11:26 smithi100 bash[76202]: cluster 2024-04-07T17:11:25.469217+0000 mgr.y (mgr.25175) 1682 : cluster [DBG] pgmap v2906: 457 pgs: 457 active+clean; 6.6 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 326 KiB/s rd, 163 KiB/s wr, 658 op/s 2024-04-07T17:11:27.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:11:26 smithi144 bash[66654]: cluster 2024-04-07T17:11:25.469217+0000 mgr.y (mgr.25175) 1682 : cluster [DBG] pgmap v2906: 457 pgs: 457 active+clean; 6.6 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 326 KiB/s rd, 163 KiB/s wr, 658 op/s 2024-04-07T17:11:28.703 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:11:28 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:11:28] "GET /metrics HTTP/1.1" 200 60794 "" "Prometheus/2.43.0" 2024-04-07T17:11:29.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:11:28 smithi100 bash[74942]: cluster 2024-04-07T17:11:27.472178+0000 mgr.y (mgr.25175) 1683 : cluster [DBG] pgmap v2907: 457 pgs: 457 active+clean; 6.6 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 301 KiB/s rd, 149 KiB/s wr, 606 op/s 2024-04-07T17:11:29.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:11:28 smithi100 bash[76202]: cluster 2024-04-07T17:11:27.472178+0000 mgr.y (mgr.25175) 1683 : cluster [DBG] pgmap v2907: 457 pgs: 457 active+clean; 6.6 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 301 KiB/s rd, 149 KiB/s wr, 606 op/s 2024-04-07T17:11:29.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:11:28 smithi144 bash[66654]: cluster 2024-04-07T17:11:27.472178+0000 mgr.y (mgr.25175) 1683 : cluster [DBG] pgmap v2907: 457 pgs: 457 active+clean; 6.6 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 301 KiB/s rd, 149 KiB/s wr, 606 op/s 2024-04-07T17:11:30.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:11:29 smithi144 bash[66654]: cluster 2024-04-07T17:11:29.474651+0000 mgr.y (mgr.25175) 1684 : cluster [DBG] pgmap v2908: 457 pgs: 457 active+clean; 6.7 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 538 KiB/s rd, 286 KiB/s wr, 795 op/s 2024-04-07T17:11:30.453 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:11:29 smithi100 bash[74942]: cluster 2024-04-07T17:11:29.474651+0000 mgr.y (mgr.25175) 1684 : cluster [DBG] pgmap v2908: 457 pgs: 457 active+clean; 6.7 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 538 KiB/s rd, 286 KiB/s wr, 795 op/s 2024-04-07T17:11:30.453 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:11:29 smithi100 bash[76202]: cluster 2024-04-07T17:11:29.474651+0000 mgr.y (mgr.25175) 1684 : cluster [DBG] pgmap v2908: 457 pgs: 457 active+clean; 6.7 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 538 KiB/s rd, 286 KiB/s wr, 795 op/s 2024-04-07T17:11:31.673 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 512 2024-04-07T17:11:31.673 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 512 2024-04-07T17:11:31.673 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 512 2024-04-07T17:11:31.673 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 512 2024-04-07T17:11:31.673 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 512 2024-04-07T17:11:31.673 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:11:31.673 INFO:tasks.workunit.client.0.smithi100.stdout:started write 2024-04-07T17:11:31.673 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:11:31.673 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:11:31.673 INFO:tasks.workunit.client.0.smithi100.stdout:finished write 2024-04-07T17:11:31.673 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:11:31.673 INFO:tasks.workunit.client.0.smithi100.stdout:started write 2024-04-07T17:11:31.673 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:11:31.674 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:11:31.674 INFO:tasks.workunit.client.0.smithi100.stdout:finished write 2024-04-07T17:11:31.674 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:11:31.674 INFO:tasks.workunit.client.0.smithi100.stdout:started write 2024-04-07T17:11:31.674 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:11:31.674 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:11:31.674 INFO:tasks.workunit.client.0.smithi100.stdout:finished write 2024-04-07T17:11:31.674 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:11:31.674 INFO:tasks.workunit.client.0.smithi100.stdout:started write 2024-04-07T17:11:31.674 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:11:31.674 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:11:31.674 INFO:tasks.workunit.client.0.smithi100.stdout:finished write 2024-04-07T17:11:31.674 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:11:31.674 INFO:tasks.workunit.client.0.smithi100.stdout:started write 2024-04-07T17:11:31.675 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:11:31.675 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:11:31.675 INFO:tasks.workunit.client.0.smithi100.stdout:finished write 2024-04-07T17:11:31.675 INFO:tasks.workunit.client.0.smithi100.stdout:start compare and write 2024-04-07T17:11:31.675 INFO:tasks.workunit.client.0.smithi100.stdout:compare and wrote: 512 2024-04-07T17:11:31.675 INFO:tasks.workunit.client.0.smithi100.stdout:start compare and write 2024-04-07T17:11:31.675 INFO:tasks.workunit.client.0.smithi100.stdout:compare and wrote: 512 2024-04-07T17:11:31.675 INFO:tasks.workunit.client.0.smithi100.stdout:start compare and write 2024-04-07T17:11:31.675 INFO:tasks.workunit.client.0.smithi100.stdout:compare and wrote: 512 2024-04-07T17:11:31.675 INFO:tasks.workunit.client.0.smithi100.stdout:start compare and write 2024-04-07T17:11:31.675 INFO:tasks.workunit.client.0.smithi100.stdout:compare and wrote: 512 2024-04-07T17:11:31.675 INFO:tasks.workunit.client.0.smithi100.stdout:start compare and write 2024-04-07T17:11:31.675 INFO:tasks.workunit.client.0.smithi100.stdout:compare and wrote: 512 2024-04-07T17:11:31.675 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:11:31.675 INFO:tasks.workunit.client.0.smithi100.stdout:started aio compare and write 2024-04-07T17:11:31.676 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:11:31.676 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:11:31.676 INFO:tasks.workunit.client.0.smithi100.stdout:finished aio compare and write 2024-04-07T17:11:31.676 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:11:31.676 INFO:tasks.workunit.client.0.smithi100.stdout:started aio compare and write 2024-04-07T17:11:31.676 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:11:31.676 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:11:31.676 INFO:tasks.workunit.client.0.smithi100.stdout:finished aio compare and write 2024-04-07T17:11:31.676 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:11:31.676 INFO:tasks.workunit.client.0.smithi100.stdout:started aio compare and write 2024-04-07T17:11:31.676 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:11:31.676 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:11:31.676 INFO:tasks.workunit.client.0.smithi100.stdout:finished aio compare and write 2024-04-07T17:11:31.676 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:11:31.676 INFO:tasks.workunit.client.0.smithi100.stdout:started aio compare and write 2024-04-07T17:11:31.676 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:11:31.677 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:11:31.677 INFO:tasks.workunit.client.0.smithi100.stdout:finished aio compare and write 2024-04-07T17:11:31.677 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:11:31.677 INFO:tasks.workunit.client.0.smithi100.stdout:started aio compare and write 2024-04-07T17:11:31.677 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:11:31.677 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:11:31.677 INFO:tasks.workunit.client.0.smithi100.stdout:finished aio compare and write 2024-04-07T17:11:31.677 INFO:tasks.workunit.client.0.smithi100.stdout:read: 512 2024-04-07T17:11:31.677 INFO:tasks.workunit.client.0.smithi100.stdout:read: 512 2024-04-07T17:11:31.677 INFO:tasks.workunit.client.0.smithi100.stdout:read: 512 2024-04-07T17:11:31.677 INFO:tasks.workunit.client.0.smithi100.stdout:read: 512 2024-04-07T17:11:31.677 INFO:tasks.workunit.client.0.smithi100.stdout:read: 512 2024-04-07T17:11:31.677 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:11:31.677 INFO:tasks.workunit.client.0.smithi100.stdout:started read 2024-04-07T17:11:31.677 INFO:tasks.workunit.client.0.smithi100.stdout:read completion cb called! 2024-04-07T17:11:31.678 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 512 2024-04-07T17:11:31.678 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:11:31.678 INFO:tasks.workunit.client.0.smithi100.stdout:started read 2024-04-07T17:11:31.678 INFO:tasks.workunit.client.0.smithi100.stdout:read completion cb called! 2024-04-07T17:11:31.678 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 512 2024-04-07T17:11:31.678 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:11:31.678 INFO:tasks.workunit.client.0.smithi100.stdout:started read 2024-04-07T17:11:31.678 INFO:tasks.workunit.client.0.smithi100.stdout:read completion cb called! 2024-04-07T17:11:31.678 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 512 2024-04-07T17:11:31.678 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:11:31.678 INFO:tasks.workunit.client.0.smithi100.stdout:started read 2024-04-07T17:11:31.678 INFO:tasks.workunit.client.0.smithi100.stdout:read completion cb called! 2024-04-07T17:11:31.678 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 512 2024-04-07T17:11:31.678 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:11:31.678 INFO:tasks.workunit.client.0.smithi100.stdout:started read 2024-04-07T17:11:31.679 INFO:tasks.workunit.client.0.smithi100.stdout:read completion cb called! 2024-04-07T17:11:31.679 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 512 2024-04-07T17:11:31.679 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 512~512 = 512 2024-04-07T17:11:31.679 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:11:31.679 INFO:tasks.workunit.client.0.smithi100.stdout:aio discard: 1536~512 = 0 2024-04-07T17:11:31.679 INFO:tasks.workunit.client.0.smithi100.stdout:read: 512 2024-04-07T17:11:31.679 INFO:tasks.workunit.client.0.smithi100.stdout:read: 512 2024-04-07T17:11:31.679 INFO:tasks.workunit.client.0.smithi100.stdout:read: 512 2024-04-07T17:11:31.679 INFO:tasks.workunit.client.0.smithi100.stdout:read: 512 2024-04-07T17:11:31.679 INFO:tasks.workunit.client.0.smithi100.stdout:read: 512 2024-04-07T17:11:31.679 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 0 2024-04-07T17:11:31.679 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:11:31.679 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:11:31.679 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 0 2024-04-07T17:11:31.679 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:11:31.680 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:11:31.680 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 16384 2024-04-07T17:11:31.680 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:11:31.680 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:11:31.680 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 16384 2024-04-07T17:11:31.680 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:11:31.680 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:11:31.680 INFO:tasks.workunit.client.0.smithi100.stdout:expected fail, finished writesame 2024-04-07T17:11:31.680 INFO:tasks.workunit.client.0.smithi100.stdout:expected fail, finished writesame 2024-04-07T17:11:31.680 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 49152 2024-04-07T17:11:31.680 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:11:31.680 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:11:31.680 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 49152 2024-04-07T17:11:31.680 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:11:31.680 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:11:31.680 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 65536 2024-04-07T17:11:31.681 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:11:31.681 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:11:31.681 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 65536 2024-04-07T17:11:31.681 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:11:31.681 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:11:31.681 INFO:tasks.workunit.client.0.smithi100.stdout:expected fail, finished writesame 2024-04-07T17:11:31.681 INFO:tasks.workunit.client.0.smithi100.stdout:expected fail, finished writesame 2024-04-07T17:11:31.681 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 98304 2024-04-07T17:11:31.681 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:11:31.681 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:11:31.681 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 98304 2024-04-07T17:11:31.681 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:11:31.681 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:11:31.681 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 114688 2024-04-07T17:11:31.681 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:11:31.682 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:11:31.682 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 114688 2024-04-07T17:11:31.682 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:11:31.682 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:11:31.682 INFO:tasks.workunit.client.0.smithi100.stdout:expected fail, finished writesame 2024-04-07T17:11:31.682 INFO:tasks.workunit.client.0.smithi100.stdout:expected fail, finished writesame 2024-04-07T17:11:31.682 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 147456 2024-04-07T17:11:31.682 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:11:31.682 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:11:31.682 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 147456 2024-04-07T17:11:31.682 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:11:31.682 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:11:31.682 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 163840 2024-04-07T17:11:31.682 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:11:31.682 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:11:31.683 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 163840 2024-04-07T17:11:31.683 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:11:31.683 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:11:31.683 INFO:tasks.workunit.client.0.smithi100.stdout:expected fail, finished writesame 2024-04-07T17:11:31.683 INFO:tasks.workunit.client.0.smithi100.stdout:expected fail, finished writesame 2024-04-07T17:11:31.683 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 196608 2024-04-07T17:11:31.683 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:11:31.683 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:11:31.683 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 196608 2024-04-07T17:11:31.683 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:11:31.683 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:11:31.683 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 212992 2024-04-07T17:11:31.684 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:11:31.684 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:11:31.684 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 212992 2024-04-07T17:11:31.684 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:11:31.684 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:11:31.684 INFO:tasks.workunit.client.0.smithi100.stdout:expected fail, finished writesame 2024-04-07T17:11:31.684 INFO:tasks.workunit.client.0.smithi100.stdout:expected fail, finished writesame 2024-04-07T17:11:31.684 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:11:31.684 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:11:31.684 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:11:31.684 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:11:31.684 INFO:tasks.workunit.client.0.smithi100.stdout:finished writesame 2024-04-07T17:11:31.684 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:11:31.684 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:11:31.684 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:11:31.685 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:11:31.685 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:11:31.685 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:11:31.685 INFO:tasks.workunit.client.0.smithi100.stdout:finished writesame 2024-04-07T17:11:31.685 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:11:31.685 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:11:31.685 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:11:31.685 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:11:31.685 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:11:31.685 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:11:31.685 INFO:tasks.workunit.client.0.smithi100.stdout:finished writesame 2024-04-07T17:11:31.685 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:11:31.685 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:11:31.685 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:11:31.686 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:11:31.686 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:11:31.686 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:11:31.686 INFO:tasks.workunit.client.0.smithi100.stdout:finished writesame 2024-04-07T17:11:31.686 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:11:31.686 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:11:31.686 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:11:31.686 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:11:31.686 INFO:tasks.workunit.client.0.smithi100.stdout:expected fail, finished writesame 2024-04-07T17:11:31.686 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:11:31.686 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:11:31.686 INFO:tasks.workunit.client.0.smithi100.stdout:expected fail, finished writesame 2024-04-07T17:11:31.687 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:11:31.687 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:11:31.687 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:11:31.687 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:11:31.687 INFO:tasks.workunit.client.0.smithi100.stdout:finished writesame 2024-04-07T17:11:31.687 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:11:31.687 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:11:31.687 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:11:31.687 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:11:31.687 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:11:31.687 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:11:31.687 INFO:tasks.workunit.client.0.smithi100.stdout:finished writesame 2024-04-07T17:11:31.687 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:11:31.687 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:11:31.688 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:11:32.617 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:11:32 smithi144 bash[95233]: ts=2024-04-07T17:11:32.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:11:33.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:11:32 smithi144 bash[66654]: cluster 2024-04-07T17:11:31.476288+0000 mgr.y (mgr.25175) 1685 : cluster [DBG] pgmap v2909: 457 pgs: 457 active+clean; 11 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 448 KiB/s rd, 664 KiB/s wr, 661 op/s 2024-04-07T17:11:33.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:11:32 smithi144 bash[66654]: audit 2024-04-07T17:11:31.826337+0000 mon.a (mon.0) 3152 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:11:33.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:11:32 smithi144 bash[66654]: audit 2024-04-07T17:11:31.829362+0000 mon.a (mon.0) 3153 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:11:33.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:11:32 smithi100 bash[74942]: cluster 2024-04-07T17:11:31.476288+0000 mgr.y (mgr.25175) 1685 : cluster [DBG] pgmap v2909: 457 pgs: 457 active+clean; 11 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 448 KiB/s rd, 664 KiB/s wr, 661 op/s 2024-04-07T17:11:33.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:11:32 smithi100 bash[74942]: audit 2024-04-07T17:11:31.826337+0000 mon.a (mon.0) 3152 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:11:33.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:11:32 smithi100 bash[74942]: audit 2024-04-07T17:11:31.829362+0000 mon.a (mon.0) 3153 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:11:33.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:11:32 smithi100 bash[76202]: cluster 2024-04-07T17:11:31.476288+0000 mgr.y (mgr.25175) 1685 : cluster [DBG] pgmap v2909: 457 pgs: 457 active+clean; 11 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 448 KiB/s rd, 664 KiB/s wr, 661 op/s 2024-04-07T17:11:33.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:11:32 smithi100 bash[76202]: audit 2024-04-07T17:11:31.826337+0000 mon.a (mon.0) 3152 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:11:33.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:11:32 smithi100 bash[76202]: audit 2024-04-07T17:11:31.829362+0000 mon.a (mon.0) 3153 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:11:35.118 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:11:34 smithi144 bash[66654]: cluster 2024-04-07T17:11:33.479634+0000 mgr.y (mgr.25175) 1686 : cluster [DBG] pgmap v2910: 457 pgs: 457 active+clean; 11 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 402 KiB/s rd, 597 KiB/s wr, 594 op/s 2024-04-07T17:11:35.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:11:34 smithi100 bash[74942]: cluster 2024-04-07T17:11:33.479634+0000 mgr.y (mgr.25175) 1686 : cluster [DBG] pgmap v2910: 457 pgs: 457 active+clean; 11 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 402 KiB/s rd, 597 KiB/s wr, 594 op/s 2024-04-07T17:11:35.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:11:34 smithi100 bash[76202]: cluster 2024-04-07T17:11:33.479634+0000 mgr.y (mgr.25175) 1686 : cluster [DBG] pgmap v2910: 457 pgs: 457 active+clean; 11 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 402 KiB/s rd, 597 KiB/s wr, 594 op/s 2024-04-07T17:11:37.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:11:36 smithi144 bash[66654]: cluster 2024-04-07T17:11:35.482084+0000 mgr.y (mgr.25175) 1687 : cluster [DBG] pgmap v2911: 457 pgs: 457 active+clean; 11 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 522 KiB/s rd, 611 KiB/s wr, 529 op/s 2024-04-07T17:11:37.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:11:36 smithi100 bash[76202]: cluster 2024-04-07T17:11:35.482084+0000 mgr.y (mgr.25175) 1687 : cluster [DBG] pgmap v2911: 457 pgs: 457 active+clean; 11 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 522 KiB/s rd, 611 KiB/s wr, 529 op/s 2024-04-07T17:11:37.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:11:36 smithi100 bash[74942]: cluster 2024-04-07T17:11:35.482084+0000 mgr.y (mgr.25175) 1687 : cluster [DBG] pgmap v2911: 457 pgs: 457 active+clean; 11 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 522 KiB/s rd, 611 KiB/s wr, 529 op/s 2024-04-07T17:11:38.703 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:11:38 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:11:38] "GET /metrics HTTP/1.1" 200 60777 "" "Prometheus/2.43.0" 2024-04-07T17:11:39.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:11:38 smithi144 bash[66654]: cluster 2024-04-07T17:11:37.483636+0000 mgr.y (mgr.25175) 1688 : cluster [DBG] pgmap v2912: 457 pgs: 457 active+clean; 15 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 524 KiB/s rd, 611 KiB/s wr, 532 op/s 2024-04-07T17:11:39.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:11:38 smithi100 bash[76202]: cluster 2024-04-07T17:11:37.483636+0000 mgr.y (mgr.25175) 1688 : cluster [DBG] pgmap v2912: 457 pgs: 457 active+clean; 15 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 524 KiB/s rd, 611 KiB/s wr, 532 op/s 2024-04-07T17:11:39.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:11:38 smithi100 bash[74942]: cluster 2024-04-07T17:11:37.483636+0000 mgr.y (mgr.25175) 1688 : cluster [DBG] pgmap v2912: 457 pgs: 457 active+clean; 15 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 524 KiB/s rd, 611 KiB/s wr, 532 op/s 2024-04-07T17:11:40.966 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:11:40.966 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:11:40.966 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:11:40.966 INFO:tasks.workunit.client.0.smithi100.stdout:finished writesame 2024-04-07T17:11:40.966 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:11:40.966 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:11:40.966 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:11:40.966 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:11:40.966 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:11:40.966 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:11:40.966 INFO:tasks.workunit.client.0.smithi100.stdout:finished writesame 2024-04-07T17:11:40.966 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:11:40.967 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:11:40.967 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:11:40.967 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:11:40.967 INFO:tasks.workunit.client.0.smithi100.stdout:expected fail, finished writesame 2024-04-07T17:11:40.967 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:11:40.967 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:11:40.967 INFO:tasks.workunit.client.0.smithi100.stdout:expected fail, finished writesame 2024-04-07T17:11:40.967 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:11:40.967 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:11:40.967 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:11:40.967 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:11:40.967 INFO:tasks.workunit.client.0.smithi100.stdout:finished writesame 2024-04-07T17:11:40.967 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:11:40.967 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:11:40.967 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:11:40.968 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:11:40.968 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:11:40.968 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:11:40.968 INFO:tasks.workunit.client.0.smithi100.stdout:finished writesame 2024-04-07T17:11:40.968 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:11:40.968 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:11:40.968 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:11:40.968 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:11:40.968 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:11:40.968 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:11:40.968 INFO:tasks.workunit.client.0.smithi100.stdout:finished writesame 2024-04-07T17:11:40.968 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:11:40.968 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:11:40.968 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:11:40.969 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:11:40.969 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:11:40.969 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:11:40.969 INFO:tasks.workunit.client.0.smithi100.stdout:finished writesame 2024-04-07T17:11:40.969 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:11:40.969 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:11:40.969 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:11:40.969 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:11:40.969 INFO:tasks.workunit.client.0.smithi100.stdout:expected fail, finished writesame 2024-04-07T17:11:40.969 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:11:40.969 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:11:40.969 INFO:tasks.workunit.client.0.smithi100.stdout:expected fail, finished writesame 2024-04-07T17:11:40.969 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:11:40.969 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:11:40.969 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:11:40.970 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:11:40.970 INFO:tasks.workunit.client.0.smithi100.stdout:finished writesame 2024-04-07T17:11:40.970 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:11:40.970 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:11:40.970 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:11:40.970 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:11:40.970 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:11:40.970 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:11:40.970 INFO:tasks.workunit.client.0.smithi100.stdout:finished writesame 2024-04-07T17:11:40.970 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:11:40.970 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:11:40.970 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:11:40.970 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:11:40.970 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:11:40.970 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:11:40.971 INFO:tasks.workunit.client.0.smithi100.stdout:finished writesame 2024-04-07T17:11:40.971 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:11:40.971 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:11:40.971 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:11:40.971 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:11:40.971 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:11:40.971 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:11:40.971 INFO:tasks.workunit.client.0.smithi100.stdout:finished writesame 2024-04-07T17:11:40.971 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:11:40.971 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:11:40.971 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:11:40.971 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:11:40.971 INFO:tasks.workunit.client.0.smithi100.stdout:expected fail, finished writesame 2024-04-07T17:11:40.971 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:11:40.971 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:11:40.972 INFO:tasks.workunit.client.0.smithi100.stdout:expected fail, finished writesame 2024-04-07T17:11:40.972 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:11:40.972 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:11:40.972 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:11:40.972 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:11:40.972 INFO:tasks.workunit.client.0.smithi100.stdout:finished writesame 2024-04-07T17:11:40.972 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:11:40.972 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:11:40.972 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:11:40.972 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:11:40.972 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:11:40.972 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:11:40.972 INFO:tasks.workunit.client.0.smithi100.stdout:finished writesame 2024-04-07T17:11:40.972 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:11:40.972 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:11:40.972 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:11:40.973 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:11:40.973 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:11:40.973 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:11:40.973 INFO:tasks.workunit.client.0.smithi100.stdout:finished writesame 2024-04-07T17:11:40.973 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:11:40.973 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:11:40.973 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:11:40.973 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:11:40.973 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:11:40.973 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:11:40.973 INFO:tasks.workunit.client.0.smithi100.stdout:finished writesame 2024-04-07T17:11:40.973 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:11:40.973 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:11:40.973 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:11:40.974 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:11:40.974 INFO:tasks.workunit.client.0.smithi100.stdout:expected fail, finished writesame 2024-04-07T17:11:40.974 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:11:40.974 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:11:40.974 INFO:tasks.workunit.client.0.smithi100.stdout:expected fail, finished writesame 2024-04-07T17:11:40.974 INFO:tasks.workunit.client.0.smithi100.stdout:read completion cb called! 2024-04-07T17:11:40.974 INFO:tasks.workunit.client.0.smithi100.stdout:read completion cb called! 2024-04-07T17:11:40.974 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 512 2024-04-07T17:11:40.974 INFO:tasks.workunit.client.0.smithi100.stdout:read completion cb called! 2024-04-07T17:11:40.974 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 4 2024-04-07T17:11:40.974 INFO:tasks.workunit.client.0.smithi100.stdout:read: 4 2024-04-07T17:11:40.974 INFO:tasks.workunit.client.0.smithi100.stdout:read: 4 2024-04-07T17:11:40.974 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.TestEncryptionLUKS1 (18936 ms) 2024-04-07T17:11:40.974 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.TestEncryptionLUKS2 2024-04-07T17:11:40.974 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:11:41.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:11:40 smithi144 bash[66654]: cluster 2024-04-07T17:11:39.484757+0000 mgr.y (mgr.25175) 1689 : cluster [DBG] pgmap v2913: 457 pgs: 457 active+clean; 15 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 700 KiB/s rd, 691 KiB/s wr, 706 op/s 2024-04-07T17:11:41.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:11:40 smithi100 bash[76202]: cluster 2024-04-07T17:11:39.484757+0000 mgr.y (mgr.25175) 1689 : cluster [DBG] pgmap v2913: 457 pgs: 457 active+clean; 15 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 700 KiB/s rd, 691 KiB/s wr, 706 op/s 2024-04-07T17:11:41.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:11:40 smithi100 bash[74942]: cluster 2024-04-07T17:11:39.484757+0000 mgr.y (mgr.25175) 1689 : cluster [DBG] pgmap v2913: 457 pgs: 457 active+clean; 15 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 700 KiB/s rd, 691 KiB/s wr, 706 op/s 2024-04-07T17:11:42.616 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:11:42 smithi144 bash[95233]: ts=2024-04-07T17:11:42.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:11:43.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:11:42 smithi144 bash[66654]: cluster 2024-04-07T17:11:41.486201+0000 mgr.y (mgr.25175) 1690 : cluster [DBG] pgmap v2914: 457 pgs: 457 active+clean; 15 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 1.2 MiB/s rd, 582 KiB/s wr, 511 op/s 2024-04-07T17:11:43.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:11:42 smithi100 bash[74942]: cluster 2024-04-07T17:11:41.486201+0000 mgr.y (mgr.25175) 1690 : cluster [DBG] pgmap v2914: 457 pgs: 457 active+clean; 15 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 1.2 MiB/s rd, 582 KiB/s wr, 511 op/s 2024-04-07T17:11:43.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:11:42 smithi100 bash[76202]: cluster 2024-04-07T17:11:41.486201+0000 mgr.y (mgr.25175) 1690 : cluster [DBG] pgmap v2914: 457 pgs: 457 active+clean; 15 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 1.2 MiB/s rd, 582 KiB/s wr, 511 op/s 2024-04-07T17:11:45.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:11:44 smithi144 bash[66654]: cluster 2024-04-07T17:11:43.488154+0000 mgr.y (mgr.25175) 1691 : cluster [DBG] pgmap v2915: 457 pgs: 457 active+clean; 15 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 1.2 MiB/s rd, 241 KiB/s wr, 516 op/s 2024-04-07T17:11:45.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:11:44 smithi100 bash[74942]: cluster 2024-04-07T17:11:43.488154+0000 mgr.y (mgr.25175) 1691 : cluster [DBG] pgmap v2915: 457 pgs: 457 active+clean; 15 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 1.2 MiB/s rd, 241 KiB/s wr, 516 op/s 2024-04-07T17:11:45.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:11:44 smithi100 bash[76202]: cluster 2024-04-07T17:11:43.488154+0000 mgr.y (mgr.25175) 1691 : cluster [DBG] pgmap v2915: 457 pgs: 457 active+clean; 15 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 1.2 MiB/s rd, 241 KiB/s wr, 516 op/s 2024-04-07T17:11:47.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:11:46 smithi144 bash[66654]: cluster 2024-04-07T17:11:45.489505+0000 mgr.y (mgr.25175) 1692 : cluster [DBG] pgmap v2916: 457 pgs: 457 active+clean; 15 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 1.2 MiB/s rd, 242 KiB/s wr, 516 op/s 2024-04-07T17:11:47.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:11:46 smithi144 bash[66654]: audit 2024-04-07T17:11:46.818777+0000 mon.a (mon.0) 3154 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:11:47.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:11:46 smithi100 bash[74942]: cluster 2024-04-07T17:11:45.489505+0000 mgr.y (mgr.25175) 1692 : cluster [DBG] pgmap v2916: 457 pgs: 457 active+clean; 15 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 1.2 MiB/s rd, 242 KiB/s wr, 516 op/s 2024-04-07T17:11:47.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:11:46 smithi100 bash[74942]: audit 2024-04-07T17:11:46.818777+0000 mon.a (mon.0) 3154 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:11:47.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:11:46 smithi100 bash[76202]: cluster 2024-04-07T17:11:45.489505+0000 mgr.y (mgr.25175) 1692 : cluster [DBG] pgmap v2916: 457 pgs: 457 active+clean; 15 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 1.2 MiB/s rd, 242 KiB/s wr, 516 op/s 2024-04-07T17:11:47.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:11:46 smithi100 bash[76202]: audit 2024-04-07T17:11:46.818777+0000 mon.a (mon.0) 3154 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:11:48.703 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:11:48 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:11:48] "GET /metrics HTTP/1.1" 200 60776 "" "Prometheus/2.43.0" 2024-04-07T17:11:49.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:11:48 smithi144 bash[66654]: cluster 2024-04-07T17:11:47.491159+0000 mgr.y (mgr.25175) 1693 : cluster [DBG] pgmap v2917: 457 pgs: 457 active+clean; 15 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 916 KiB/s rd, 81 KiB/s wr, 189 op/s 2024-04-07T17:11:49.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:11:48 smithi100 bash[74942]: cluster 2024-04-07T17:11:47.491159+0000 mgr.y (mgr.25175) 1693 : cluster [DBG] pgmap v2917: 457 pgs: 457 active+clean; 15 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 916 KiB/s rd, 81 KiB/s wr, 189 op/s 2024-04-07T17:11:49.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:11:48 smithi100 bash[76202]: cluster 2024-04-07T17:11:47.491159+0000 mgr.y (mgr.25175) 1693 : cluster [DBG] pgmap v2917: 457 pgs: 457 active+clean; 15 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 916 KiB/s rd, 81 KiB/s wr, 189 op/s 2024-04-07T17:11:51.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:11:50 smithi144 bash[66654]: cluster 2024-04-07T17:11:49.492532+0000 mgr.y (mgr.25175) 1694 : cluster [DBG] pgmap v2918: 457 pgs: 457 active+clean; 15 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 914 KiB/s rd, 81 KiB/s wr, 186 op/s 2024-04-07T17:11:51.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:11:50 smithi100 bash[74942]: cluster 2024-04-07T17:11:49.492532+0000 mgr.y (mgr.25175) 1694 : cluster [DBG] pgmap v2918: 457 pgs: 457 active+clean; 15 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 914 KiB/s rd, 81 KiB/s wr, 186 op/s 2024-04-07T17:11:51.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:11:50 smithi100 bash[76202]: cluster 2024-04-07T17:11:49.492532+0000 mgr.y (mgr.25175) 1694 : cluster [DBG] pgmap v2918: 457 pgs: 457 active+clean; 15 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 914 KiB/s rd, 81 KiB/s wr, 186 op/s 2024-04-07T17:11:52.616 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:11:52 smithi144 bash[95233]: ts=2024-04-07T17:11:52.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:11:53.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:11:52 smithi144 bash[66654]: cluster 2024-04-07T17:11:51.494122+0000 mgr.y (mgr.25175) 1695 : cluster [DBG] pgmap v2919: 457 pgs: 457 active+clean; 15 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 735 KiB/s rd, 341 B/s wr, 6 op/s 2024-04-07T17:11:53.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:11:52 smithi144 bash[66654]: audit 2024-04-07T17:11:52.066195+0000 mon.a (mon.0) 3155 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:11:53.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:11:52 smithi144 bash[66654]: audit 2024-04-07T17:11:52.800684+0000 mon.a (mon.0) 3156 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:11:53.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:11:52 smithi144 bash[66654]: audit 2024-04-07T17:11:52.802128+0000 mon.a (mon.0) 3157 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:11:53.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:11:52 smithi144 bash[66654]: audit 2024-04-07T17:11:52.808933+0000 mon.a (mon.0) 3158 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:11:53.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:11:52 smithi100 bash[76202]: cluster 2024-04-07T17:11:51.494122+0000 mgr.y (mgr.25175) 1695 : cluster [DBG] pgmap v2919: 457 pgs: 457 active+clean; 15 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 735 KiB/s rd, 341 B/s wr, 6 op/s 2024-04-07T17:11:53.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:11:52 smithi100 bash[76202]: audit 2024-04-07T17:11:52.066195+0000 mon.a (mon.0) 3155 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:11:53.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:11:52 smithi100 bash[76202]: audit 2024-04-07T17:11:52.800684+0000 mon.a (mon.0) 3156 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:11:53.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:11:52 smithi100 bash[76202]: audit 2024-04-07T17:11:52.802128+0000 mon.a (mon.0) 3157 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:11:53.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:11:52 smithi100 bash[76202]: audit 2024-04-07T17:11:52.808933+0000 mon.a (mon.0) 3158 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:11:53.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:11:52 smithi100 bash[74942]: cluster 2024-04-07T17:11:51.494122+0000 mgr.y (mgr.25175) 1695 : cluster [DBG] pgmap v2919: 457 pgs: 457 active+clean; 15 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 735 KiB/s rd, 341 B/s wr, 6 op/s 2024-04-07T17:11:53.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:11:52 smithi100 bash[74942]: audit 2024-04-07T17:11:52.066195+0000 mon.a (mon.0) 3155 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:11:53.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:11:52 smithi100 bash[74942]: audit 2024-04-07T17:11:52.800684+0000 mon.a (mon.0) 3156 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:11:53.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:11:52 smithi100 bash[74942]: audit 2024-04-07T17:11:52.802128+0000 mon.a (mon.0) 3157 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:11:53.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:11:52 smithi100 bash[74942]: audit 2024-04-07T17:11:52.808933+0000 mon.a (mon.0) 3158 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:11:54.859 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 512 2024-04-07T17:11:54.859 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 512 2024-04-07T17:11:54.859 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 512 2024-04-07T17:11:54.859 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 512 2024-04-07T17:11:54.860 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 512 2024-04-07T17:11:54.860 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:11:54.860 INFO:tasks.workunit.client.0.smithi100.stdout:started write 2024-04-07T17:11:54.860 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:11:54.860 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:11:54.860 INFO:tasks.workunit.client.0.smithi100.stdout:finished write 2024-04-07T17:11:54.860 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:11:54.860 INFO:tasks.workunit.client.0.smithi100.stdout:started write 2024-04-07T17:11:54.860 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:11:54.860 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:11:54.860 INFO:tasks.workunit.client.0.smithi100.stdout:finished write 2024-04-07T17:11:54.860 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:11:54.860 INFO:tasks.workunit.client.0.smithi100.stdout:started write 2024-04-07T17:11:54.861 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:11:54.861 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:11:54.861 INFO:tasks.workunit.client.0.smithi100.stdout:finished write 2024-04-07T17:11:54.861 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:11:54.861 INFO:tasks.workunit.client.0.smithi100.stdout:started write 2024-04-07T17:11:54.861 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:11:54.861 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:11:54.861 INFO:tasks.workunit.client.0.smithi100.stdout:finished write 2024-04-07T17:11:54.861 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:11:54.861 INFO:tasks.workunit.client.0.smithi100.stdout:started write 2024-04-07T17:11:54.861 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:11:54.861 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:11:54.861 INFO:tasks.workunit.client.0.smithi100.stdout:finished write 2024-04-07T17:11:54.861 INFO:tasks.workunit.client.0.smithi100.stdout:start compare and write 2024-04-07T17:11:54.861 INFO:tasks.workunit.client.0.smithi100.stdout:compare and wrote: 512 2024-04-07T17:11:54.862 INFO:tasks.workunit.client.0.smithi100.stdout:start compare and write 2024-04-07T17:11:54.862 INFO:tasks.workunit.client.0.smithi100.stdout:compare and wrote: 512 2024-04-07T17:11:54.862 INFO:tasks.workunit.client.0.smithi100.stdout:start compare and write 2024-04-07T17:11:54.862 INFO:tasks.workunit.client.0.smithi100.stdout:compare and wrote: 512 2024-04-07T17:11:54.862 INFO:tasks.workunit.client.0.smithi100.stdout:start compare and write 2024-04-07T17:11:54.862 INFO:tasks.workunit.client.0.smithi100.stdout:compare and wrote: 512 2024-04-07T17:11:54.862 INFO:tasks.workunit.client.0.smithi100.stdout:start compare and write 2024-04-07T17:11:54.862 INFO:tasks.workunit.client.0.smithi100.stdout:compare and wrote: 512 2024-04-07T17:11:54.862 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:11:54.862 INFO:tasks.workunit.client.0.smithi100.stdout:started aio compare and write 2024-04-07T17:11:54.862 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:11:54.862 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:11:54.862 INFO:tasks.workunit.client.0.smithi100.stdout:finished aio compare and write 2024-04-07T17:11:54.863 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:11:54.863 INFO:tasks.workunit.client.0.smithi100.stdout:started aio compare and write 2024-04-07T17:11:54.863 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:11:54.863 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:11:54.863 INFO:tasks.workunit.client.0.smithi100.stdout:finished aio compare and write 2024-04-07T17:11:54.863 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:11:54.863 INFO:tasks.workunit.client.0.smithi100.stdout:started aio compare and write 2024-04-07T17:11:54.863 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:11:54.863 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:11:54.863 INFO:tasks.workunit.client.0.smithi100.stdout:finished aio compare and write 2024-04-07T17:11:54.863 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:11:54.863 INFO:tasks.workunit.client.0.smithi100.stdout:started aio compare and write 2024-04-07T17:11:54.863 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:11:54.863 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:11:54.864 INFO:tasks.workunit.client.0.smithi100.stdout:finished aio compare and write 2024-04-07T17:11:54.864 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:11:54.864 INFO:tasks.workunit.client.0.smithi100.stdout:started aio compare and write 2024-04-07T17:11:54.864 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:11:54.864 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:11:54.864 INFO:tasks.workunit.client.0.smithi100.stdout:finished aio compare and write 2024-04-07T17:11:54.864 INFO:tasks.workunit.client.0.smithi100.stdout:read: 512 2024-04-07T17:11:54.864 INFO:tasks.workunit.client.0.smithi100.stdout:read: 512 2024-04-07T17:11:54.864 INFO:tasks.workunit.client.0.smithi100.stdout:read: 512 2024-04-07T17:11:54.864 INFO:tasks.workunit.client.0.smithi100.stdout:read: 512 2024-04-07T17:11:54.864 INFO:tasks.workunit.client.0.smithi100.stdout:read: 512 2024-04-07T17:11:54.864 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:11:54.864 INFO:tasks.workunit.client.0.smithi100.stdout:started read 2024-04-07T17:11:54.864 INFO:tasks.workunit.client.0.smithi100.stdout:read completion cb called! 2024-04-07T17:11:54.864 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 512 2024-04-07T17:11:54.864 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:11:54.865 INFO:tasks.workunit.client.0.smithi100.stdout:started read 2024-04-07T17:11:54.865 INFO:tasks.workunit.client.0.smithi100.stdout:read completion cb called! 2024-04-07T17:11:54.865 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 512 2024-04-07T17:11:54.865 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:11:54.865 INFO:tasks.workunit.client.0.smithi100.stdout:started read 2024-04-07T17:11:54.865 INFO:tasks.workunit.client.0.smithi100.stdout:read completion cb called! 2024-04-07T17:11:54.865 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 512 2024-04-07T17:11:54.865 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:11:54.865 INFO:tasks.workunit.client.0.smithi100.stdout:started read 2024-04-07T17:11:54.865 INFO:tasks.workunit.client.0.smithi100.stdout:read completion cb called! 2024-04-07T17:11:54.865 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 512 2024-04-07T17:11:54.865 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:11:54.865 INFO:tasks.workunit.client.0.smithi100.stdout:started read 2024-04-07T17:11:54.865 INFO:tasks.workunit.client.0.smithi100.stdout:read completion cb called! 2024-04-07T17:11:54.866 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 512 2024-04-07T17:11:54.866 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 512~512 = 512 2024-04-07T17:11:54.866 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:11:54.866 INFO:tasks.workunit.client.0.smithi100.stdout:aio discard: 1536~512 = 0 2024-04-07T17:11:54.866 INFO:tasks.workunit.client.0.smithi100.stdout:read: 512 2024-04-07T17:11:54.866 INFO:tasks.workunit.client.0.smithi100.stdout:read: 512 2024-04-07T17:11:54.866 INFO:tasks.workunit.client.0.smithi100.stdout:read: 512 2024-04-07T17:11:54.866 INFO:tasks.workunit.client.0.smithi100.stdout:read: 512 2024-04-07T17:11:54.866 INFO:tasks.workunit.client.0.smithi100.stdout:read: 512 2024-04-07T17:11:54.866 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 0 2024-04-07T17:11:54.866 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:11:54.866 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:11:54.866 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 0 2024-04-07T17:11:54.867 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:11:54.867 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:11:54.867 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 16384 2024-04-07T17:11:54.867 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:11:54.867 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:11:54.867 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 16384 2024-04-07T17:11:54.867 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:11:54.867 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:11:54.867 INFO:tasks.workunit.client.0.smithi100.stdout:expected fail, finished writesame 2024-04-07T17:11:54.867 INFO:tasks.workunit.client.0.smithi100.stdout:expected fail, finished writesame 2024-04-07T17:11:54.867 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 49152 2024-04-07T17:11:54.867 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:11:54.868 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:11:54.868 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 49152 2024-04-07T17:11:54.868 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:11:54.868 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:11:54.868 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 65536 2024-04-07T17:11:54.868 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:11:54.868 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:11:54.868 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 65536 2024-04-07T17:11:54.868 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:11:54.868 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:11:54.868 INFO:tasks.workunit.client.0.smithi100.stdout:expected fail, finished writesame 2024-04-07T17:11:54.868 INFO:tasks.workunit.client.0.smithi100.stdout:expected fail, finished writesame 2024-04-07T17:11:54.868 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 98304 2024-04-07T17:11:54.868 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:11:54.869 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:11:54.869 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 98304 2024-04-07T17:11:54.869 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:11:54.869 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:11:54.869 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 114688 2024-04-07T17:11:54.869 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:11:54.869 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:11:54.869 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 114688 2024-04-07T17:11:54.869 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:11:54.869 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:11:54.869 INFO:tasks.workunit.client.0.smithi100.stdout:expected fail, finished writesame 2024-04-07T17:11:54.869 INFO:tasks.workunit.client.0.smithi100.stdout:expected fail, finished writesame 2024-04-07T17:11:54.869 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 147456 2024-04-07T17:11:54.869 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:11:54.870 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:11:54.870 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 147456 2024-04-07T17:11:54.870 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:11:54.870 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:11:54.870 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 163840 2024-04-07T17:11:54.870 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:11:54.870 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:11:54.870 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 163840 2024-04-07T17:11:54.870 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:11:54.870 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:11:54.870 INFO:tasks.workunit.client.0.smithi100.stdout:expected fail, finished writesame 2024-04-07T17:11:54.870 INFO:tasks.workunit.client.0.smithi100.stdout:expected fail, finished writesame 2024-04-07T17:11:54.870 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 196608 2024-04-07T17:11:54.870 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:11:54.871 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:11:54.871 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 196608 2024-04-07T17:11:54.871 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:11:54.871 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:11:54.871 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 212992 2024-04-07T17:11:54.871 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:11:54.871 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:11:54.871 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 212992 2024-04-07T17:11:54.871 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:11:54.871 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:11:54.871 INFO:tasks.workunit.client.0.smithi100.stdout:expected fail, finished writesame 2024-04-07T17:11:54.871 INFO:tasks.workunit.client.0.smithi100.stdout:expected fail, finished writesame 2024-04-07T17:11:54.871 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:11:54.871 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:11:54.872 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:11:54.872 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:11:54.872 INFO:tasks.workunit.client.0.smithi100.stdout:finished writesame 2024-04-07T17:11:54.872 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:11:54.872 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:11:54.872 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:11:54.872 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:11:54.872 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:11:54.872 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:11:54.872 INFO:tasks.workunit.client.0.smithi100.stdout:finished writesame 2024-04-07T17:11:54.872 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:11:54.872 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:11:54.872 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:11:54.872 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:11:54.872 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:11:54.872 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:11:54.873 INFO:tasks.workunit.client.0.smithi100.stdout:finished writesame 2024-04-07T17:11:54.873 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:11:54.873 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:11:54.873 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:11:54.873 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:11:54.873 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:11:54.873 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:11:54.873 INFO:tasks.workunit.client.0.smithi100.stdout:finished writesame 2024-04-07T17:11:54.873 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:11:54.873 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:11:54.873 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:11:54.873 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:11:54.873 INFO:tasks.workunit.client.0.smithi100.stdout:expected fail, finished writesame 2024-04-07T17:11:54.873 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:11:54.873 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:11:54.874 INFO:tasks.workunit.client.0.smithi100.stdout:expected fail, finished writesame 2024-04-07T17:11:54.874 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:11:54.874 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:11:54.874 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:11:54.874 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:11:54.874 INFO:tasks.workunit.client.0.smithi100.stdout:finished writesame 2024-04-07T17:11:54.874 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:11:54.874 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:11:54.874 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:11:54.874 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:11:54.874 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:11:54.874 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:11:54.874 INFO:tasks.workunit.client.0.smithi100.stdout:finished writesame 2024-04-07T17:11:54.874 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:11:54.875 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:11:54.875 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:11:55.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:11:54 smithi100 bash[74942]: cluster 2024-04-07T17:11:53.497328+0000 mgr.y (mgr.25175) 1696 : cluster [DBG] pgmap v2920: 457 pgs: 457 active+clean; 27 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 260 op/s 2024-04-07T17:11:55.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:11:54 smithi100 bash[76202]: cluster 2024-04-07T17:11:53.497328+0000 mgr.y (mgr.25175) 1696 : cluster [DBG] pgmap v2920: 457 pgs: 457 active+clean; 27 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 260 op/s 2024-04-07T17:11:55.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:11:54 smithi144 bash[66654]: cluster 2024-04-07T17:11:53.497328+0000 mgr.y (mgr.25175) 1696 : cluster [DBG] pgmap v2920: 457 pgs: 457 active+clean; 27 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 260 op/s 2024-04-07T17:11:57.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:11:56 smithi100 bash[74942]: cluster 2024-04-07T17:11:55.499538+0000 mgr.y (mgr.25175) 1697 : cluster [DBG] pgmap v2921: 457 pgs: 457 active+clean; 31 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 1.0 MiB/s rd, 1.5 MiB/s wr, 255 op/s 2024-04-07T17:11:57.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:11:56 smithi100 bash[76202]: cluster 2024-04-07T17:11:55.499538+0000 mgr.y (mgr.25175) 1697 : cluster [DBG] pgmap v2921: 457 pgs: 457 active+clean; 31 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 1.0 MiB/s rd, 1.5 MiB/s wr, 255 op/s 2024-04-07T17:11:57.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:11:56 smithi144 bash[66654]: cluster 2024-04-07T17:11:55.499538+0000 mgr.y (mgr.25175) 1697 : cluster [DBG] pgmap v2921: 457 pgs: 457 active+clean; 31 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 1.0 MiB/s rd, 1.5 MiB/s wr, 255 op/s 2024-04-07T17:11:58.703 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:11:58 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:11:58] "GET /metrics HTTP/1.1" 200 60776 "" "Prometheus/2.43.0" 2024-04-07T17:11:59.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:11:58 smithi100 bash[74942]: cluster 2024-04-07T17:11:57.502318+0000 mgr.y (mgr.25175) 1698 : cluster [DBG] pgmap v2922: 457 pgs: 457 active+clean; 31 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 1.0 MiB/s rd, 1.5 MiB/s wr, 254 op/s 2024-04-07T17:11:59.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:11:58 smithi100 bash[76202]: cluster 2024-04-07T17:11:57.502318+0000 mgr.y (mgr.25175) 1698 : cluster [DBG] pgmap v2922: 457 pgs: 457 active+clean; 31 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 1.0 MiB/s rd, 1.5 MiB/s wr, 254 op/s 2024-04-07T17:11:59.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:11:58 smithi144 bash[66654]: cluster 2024-04-07T17:11:57.502318+0000 mgr.y (mgr.25175) 1698 : cluster [DBG] pgmap v2922: 457 pgs: 457 active+clean; 31 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 1.0 MiB/s rd, 1.5 MiB/s wr, 254 op/s 2024-04-07T17:12:01.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:12:00 smithi100 bash[74942]: cluster 2024-04-07T17:11:59.504703+0000 mgr.y (mgr.25175) 1699 : cluster [DBG] pgmap v2923: 457 pgs: 457 active+clean; 31 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 2.4 MiB/s rd, 1.6 MiB/s wr, 584 op/s 2024-04-07T17:12:01.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:12:00 smithi100 bash[76202]: cluster 2024-04-07T17:11:59.504703+0000 mgr.y (mgr.25175) 1699 : cluster [DBG] pgmap v2923: 457 pgs: 457 active+clean; 31 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 2.4 MiB/s rd, 1.6 MiB/s wr, 584 op/s 2024-04-07T17:12:01.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:12:00 smithi144 bash[66654]: cluster 2024-04-07T17:11:59.504703+0000 mgr.y (mgr.25175) 1699 : cluster [DBG] pgmap v2923: 457 pgs: 457 active+clean; 31 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 2.4 MiB/s rd, 1.6 MiB/s wr, 584 op/s 2024-04-07T17:12:02.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:12:01 smithi100 bash[74942]: audit 2024-04-07T17:12:01.819111+0000 mon.a (mon.0) 3159 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:12:02.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:12:01 smithi100 bash[76202]: audit 2024-04-07T17:12:01.819111+0000 mon.a (mon.0) 3159 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:12:02.316 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:12:01 smithi144 bash[66654]: audit 2024-04-07T17:12:01.819111+0000 mon.a (mon.0) 3159 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:12:02.617 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:12:02 smithi144 bash[95233]: ts=2024-04-07T17:12:02.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:12:03.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:12:02 smithi100 bash[74942]: cluster 2024-04-07T17:12:01.505727+0000 mgr.y (mgr.25175) 1700 : cluster [DBG] pgmap v2924: 457 pgs: 457 active+clean; 31 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 2.4 MiB/s rd, 1.6 MiB/s wr, 584 op/s 2024-04-07T17:12:03.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:12:02 smithi100 bash[76202]: cluster 2024-04-07T17:12:01.505727+0000 mgr.y (mgr.25175) 1700 : cluster [DBG] pgmap v2924: 457 pgs: 457 active+clean; 31 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 2.4 MiB/s rd, 1.6 MiB/s wr, 584 op/s 2024-04-07T17:12:03.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:12:02 smithi144 bash[66654]: cluster 2024-04-07T17:12:01.505727+0000 mgr.y (mgr.25175) 1700 : cluster [DBG] pgmap v2924: 457 pgs: 457 active+clean; 31 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 2.4 MiB/s rd, 1.6 MiB/s wr, 584 op/s 2024-04-07T17:12:04.080 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:12:04.080 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:12:04.080 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:12:04.080 INFO:tasks.workunit.client.0.smithi100.stdout:finished writesame 2024-04-07T17:12:04.080 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:12:04.080 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:12:04.080 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:12:04.081 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:12:04.081 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:12:04.081 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:12:04.081 INFO:tasks.workunit.client.0.smithi100.stdout:finished writesame 2024-04-07T17:12:04.081 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:12:04.081 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:12:04.081 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:12:04.081 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:12:04.081 INFO:tasks.workunit.client.0.smithi100.stdout:expected fail, finished writesame 2024-04-07T17:12:04.081 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:12:04.081 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:12:04.081 INFO:tasks.workunit.client.0.smithi100.stdout:expected fail, finished writesame 2024-04-07T17:12:04.081 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:12:04.081 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:12:04.081 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:12:04.081 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:12:04.082 INFO:tasks.workunit.client.0.smithi100.stdout:finished writesame 2024-04-07T17:12:04.082 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:12:04.082 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:12:04.082 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:12:04.082 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:12:04.082 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:12:04.082 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:12:04.082 INFO:tasks.workunit.client.0.smithi100.stdout:finished writesame 2024-04-07T17:12:04.082 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:12:04.082 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:12:04.082 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:12:04.082 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:12:04.082 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:12:04.082 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:12:04.083 INFO:tasks.workunit.client.0.smithi100.stdout:finished writesame 2024-04-07T17:12:04.083 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:12:04.083 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:12:04.083 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:12:04.083 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:12:04.083 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:12:04.083 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:12:04.083 INFO:tasks.workunit.client.0.smithi100.stdout:finished writesame 2024-04-07T17:12:04.083 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:12:04.083 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:12:04.083 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:12:04.083 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:12:04.083 INFO:tasks.workunit.client.0.smithi100.stdout:expected fail, finished writesame 2024-04-07T17:12:04.083 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:12:04.083 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:12:04.083 INFO:tasks.workunit.client.0.smithi100.stdout:expected fail, finished writesame 2024-04-07T17:12:04.084 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:12:04.084 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:12:04.084 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:12:04.084 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:12:04.084 INFO:tasks.workunit.client.0.smithi100.stdout:finished writesame 2024-04-07T17:12:04.084 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:12:04.084 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:12:04.084 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:12:04.084 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:12:04.084 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:12:04.084 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:12:04.084 INFO:tasks.workunit.client.0.smithi100.stdout:finished writesame 2024-04-07T17:12:04.084 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:12:04.084 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:12:04.085 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:12:04.085 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:12:04.085 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:12:04.085 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:12:04.085 INFO:tasks.workunit.client.0.smithi100.stdout:finished writesame 2024-04-07T17:12:04.085 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:12:04.085 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:12:04.085 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:12:04.085 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:12:04.085 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:12:04.085 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:12:04.085 INFO:tasks.workunit.client.0.smithi100.stdout:finished writesame 2024-04-07T17:12:04.085 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:12:04.085 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:12:04.085 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:12:04.086 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:12:04.086 INFO:tasks.workunit.client.0.smithi100.stdout:expected fail, finished writesame 2024-04-07T17:12:04.086 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:12:04.086 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:12:04.086 INFO:tasks.workunit.client.0.smithi100.stdout:expected fail, finished writesame 2024-04-07T17:12:04.086 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:12:04.086 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:12:04.086 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:12:04.086 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:12:04.086 INFO:tasks.workunit.client.0.smithi100.stdout:finished writesame 2024-04-07T17:12:04.086 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:12:04.086 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:12:04.086 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:12:04.087 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:12:04.087 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:12:04.087 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:12:04.087 INFO:tasks.workunit.client.0.smithi100.stdout:finished writesame 2024-04-07T17:12:04.087 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:12:04.087 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:12:04.087 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:12:04.087 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:12:04.087 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:12:04.087 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:12:04.087 INFO:tasks.workunit.client.0.smithi100.stdout:finished writesame 2024-04-07T17:12:04.087 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:12:04.087 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:12:04.087 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:12:04.087 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:12:04.087 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:12:04.088 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:12:04.088 INFO:tasks.workunit.client.0.smithi100.stdout:finished writesame 2024-04-07T17:12:04.088 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:12:04.088 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:12:04.088 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:12:04.088 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:12:04.088 INFO:tasks.workunit.client.0.smithi100.stdout:expected fail, finished writesame 2024-04-07T17:12:04.088 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:12:04.088 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:12:04.088 INFO:tasks.workunit.client.0.smithi100.stdout:expected fail, finished writesame 2024-04-07T17:12:04.088 INFO:tasks.workunit.client.0.smithi100.stdout:read completion cb called! 2024-04-07T17:12:04.088 INFO:tasks.workunit.client.0.smithi100.stdout:read completion cb called! 2024-04-07T17:12:04.088 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 512 2024-04-07T17:12:04.088 INFO:tasks.workunit.client.0.smithi100.stdout:read completion cb called! 2024-04-07T17:12:04.089 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 4 2024-04-07T17:12:04.089 INFO:tasks.workunit.client.0.smithi100.stdout:read: 4 2024-04-07T17:12:04.089 INFO:tasks.workunit.client.0.smithi100.stdout:read: 4 2024-04-07T17:12:04.089 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.TestEncryptionLUKS2 (23116 ms) 2024-04-07T17:12:04.089 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.TestCloneEncryption 2024-04-07T17:12:04.089 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:12:05.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:12:04 smithi100 bash[74942]: cluster 2024-04-07T17:12:03.507563+0000 mgr.y (mgr.25175) 1701 : cluster [DBG] pgmap v2925: 457 pgs: 457 active+clean; 35 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 2.9 MiB/s rd, 1.7 MiB/s wr, 719 op/s 2024-04-07T17:12:05.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:12:04 smithi100 bash[76202]: cluster 2024-04-07T17:12:03.507563+0000 mgr.y (mgr.25175) 1701 : cluster [DBG] pgmap v2925: 457 pgs: 457 active+clean; 35 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 2.9 MiB/s rd, 1.7 MiB/s wr, 719 op/s 2024-04-07T17:12:05.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:12:04 smithi144 bash[66654]: cluster 2024-04-07T17:12:03.507563+0000 mgr.y (mgr.25175) 1701 : cluster [DBG] pgmap v2925: 457 pgs: 457 active+clean; 35 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 2.9 MiB/s rd, 1.7 MiB/s wr, 719 op/s 2024-04-07T17:12:06.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:12:05 smithi100 bash[76202]: cluster 2024-04-07T17:12:04.916214+0000 mon.a (mon.0) 3160 : cluster [DBG] osdmap e1650: 8 total, 8 up, 8 in 2024-04-07T17:12:06.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:12:05 smithi100 bash[74942]: cluster 2024-04-07T17:12:04.916214+0000 mon.a (mon.0) 3160 : cluster [DBG] osdmap e1650: 8 total, 8 up, 8 in 2024-04-07T17:12:06.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:12:05 smithi144 bash[66654]: cluster 2024-04-07T17:12:04.916214+0000 mon.a (mon.0) 3160 : cluster [DBG] osdmap e1650: 8 total, 8 up, 8 in 2024-04-07T17:12:07.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:12:06 smithi100 bash[76202]: cluster 2024-04-07T17:12:05.509031+0000 mgr.y (mgr.25175) 1702 : cluster [DBG] pgmap v2927: 457 pgs: 457 active+clean; 35 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 3.2 MiB/s rd, 278 KiB/s wr, 559 op/s 2024-04-07T17:12:07.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:12:06 smithi100 bash[74942]: cluster 2024-04-07T17:12:05.509031+0000 mgr.y (mgr.25175) 1702 : cluster [DBG] pgmap v2927: 457 pgs: 457 active+clean; 35 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 3.2 MiB/s rd, 278 KiB/s wr, 559 op/s 2024-04-07T17:12:07.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:12:06 smithi144 bash[66654]: cluster 2024-04-07T17:12:05.509031+0000 mgr.y (mgr.25175) 1702 : cluster [DBG] pgmap v2927: 457 pgs: 457 active+clean; 35 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 3.2 MiB/s rd, 278 KiB/s wr, 559 op/s 2024-04-07T17:12:08.703 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:12:08 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:12:08] "GET /metrics HTTP/1.1" 200 60776 "" "Prometheus/2.43.0" 2024-04-07T17:12:09.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:12:08 smithi100 bash[74942]: cluster 2024-04-07T17:12:07.511743+0000 mgr.y (mgr.25175) 1703 : cluster [DBG] pgmap v2928: 457 pgs: 457 active+clean; 35 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 3.2 MiB/s rd, 280 KiB/s wr, 573 op/s 2024-04-07T17:12:09.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:12:08 smithi100 bash[76202]: cluster 2024-04-07T17:12:07.511743+0000 mgr.y (mgr.25175) 1703 : cluster [DBG] pgmap v2928: 457 pgs: 457 active+clean; 35 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 3.2 MiB/s rd, 280 KiB/s wr, 573 op/s 2024-04-07T17:12:09.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:12:08 smithi144 bash[66654]: cluster 2024-04-07T17:12:07.511743+0000 mgr.y (mgr.25175) 1703 : cluster [DBG] pgmap v2928: 457 pgs: 457 active+clean; 35 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 3.2 MiB/s rd, 280 KiB/s wr, 573 op/s 2024-04-07T17:12:11.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:12:10 smithi100 bash[74942]: cluster 2024-04-07T17:12:09.512957+0000 mgr.y (mgr.25175) 1704 : cluster [DBG] pgmap v2929: 457 pgs: 457 active+clean; 35 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 1.5 MiB/s rd, 65 KiB/s wr, 193 op/s 2024-04-07T17:12:11.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:12:10 smithi100 bash[76202]: cluster 2024-04-07T17:12:09.512957+0000 mgr.y (mgr.25175) 1704 : cluster [DBG] pgmap v2929: 457 pgs: 457 active+clean; 35 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 1.5 MiB/s rd, 65 KiB/s wr, 193 op/s 2024-04-07T17:12:11.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:12:10 smithi144 bash[66654]: cluster 2024-04-07T17:12:09.512957+0000 mgr.y (mgr.25175) 1704 : cluster [DBG] pgmap v2929: 457 pgs: 457 active+clean; 35 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 1.5 MiB/s rd, 65 KiB/s wr, 193 op/s 2024-04-07T17:12:12.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:12:11 smithi100 bash[76202]: cluster 2024-04-07T17:12:10.942820+0000 mon.a (mon.0) 3161 : cluster [DBG] osdmap e1651: 8 total, 8 up, 8 in 2024-04-07T17:12:12.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:12:11 smithi100 bash[76202]: cluster 2024-04-07T17:12:11.513985+0000 mgr.y (mgr.25175) 1705 : cluster [DBG] pgmap v2931: 457 pgs: 457 active+clean; 39 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 1.1 MiB/s rd, 514 KiB/s wr, 38 op/s 2024-04-07T17:12:12.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:12:11 smithi100 bash[74942]: cluster 2024-04-07T17:12:10.942820+0000 mon.a (mon.0) 3161 : cluster [DBG] osdmap e1651: 8 total, 8 up, 8 in 2024-04-07T17:12:12.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:12:11 smithi100 bash[74942]: cluster 2024-04-07T17:12:11.513985+0000 mgr.y (mgr.25175) 1705 : cluster [DBG] pgmap v2931: 457 pgs: 457 active+clean; 39 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 1.1 MiB/s rd, 514 KiB/s wr, 38 op/s 2024-04-07T17:12:12.317 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:12:11 smithi144 bash[66654]: cluster 2024-04-07T17:12:10.942820+0000 mon.a (mon.0) 3161 : cluster [DBG] osdmap e1651: 8 total, 8 up, 8 in 2024-04-07T17:12:12.317 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:12:11 smithi144 bash[66654]: cluster 2024-04-07T17:12:11.513985+0000 mgr.y (mgr.25175) 1705 : cluster [DBG] pgmap v2931: 457 pgs: 457 active+clean; 39 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 1.1 MiB/s rd, 514 KiB/s wr, 38 op/s 2024-04-07T17:12:12.617 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:12:12 smithi144 bash[95233]: ts=2024-04-07T17:12:12.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:12:14.867 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:12:14 smithi144 bash[66654]: cluster 2024-04-07T17:12:13.515997+0000 mgr.y (mgr.25175) 1706 : cluster [DBG] pgmap v2932: 457 pgs: 457 active+clean; 39 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 75 KiB/s rd, 479 KiB/s wr, 54 op/s 2024-04-07T17:12:14.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:12:14 smithi100 bash[74942]: cluster 2024-04-07T17:12:13.515997+0000 mgr.y (mgr.25175) 1706 : cluster [DBG] pgmap v2932: 457 pgs: 457 active+clean; 39 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 75 KiB/s rd, 479 KiB/s wr, 54 op/s 2024-04-07T17:12:14.953 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:12:14 smithi100 bash[76202]: cluster 2024-04-07T17:12:13.515997+0000 mgr.y (mgr.25175) 1706 : cluster [DBG] pgmap v2932: 457 pgs: 457 active+clean; 39 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 75 KiB/s rd, 479 KiB/s wr, 54 op/s 2024-04-07T17:12:16.867 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:12:16 smithi144 bash[66654]: cluster 2024-04-07T17:12:15.517350+0000 mgr.y (mgr.25175) 1707 : cluster [DBG] pgmap v2933: 457 pgs: 457 active+clean; 39 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 75 KiB/s rd, 413 KiB/s wr, 60 op/s 2024-04-07T17:12:16.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:12:16 smithi100 bash[74942]: cluster 2024-04-07T17:12:15.517350+0000 mgr.y (mgr.25175) 1707 : cluster [DBG] pgmap v2933: 457 pgs: 457 active+clean; 39 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 75 KiB/s rd, 413 KiB/s wr, 60 op/s 2024-04-07T17:12:16.953 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:12:16 smithi100 bash[76202]: cluster 2024-04-07T17:12:15.517350+0000 mgr.y (mgr.25175) 1707 : cluster [DBG] pgmap v2933: 457 pgs: 457 active+clean; 39 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 75 KiB/s rd, 413 KiB/s wr, 60 op/s 2024-04-07T17:12:17.867 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:12:17 smithi144 bash[66654]: audit 2024-04-07T17:12:16.819387+0000 mon.a (mon.0) 3162 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:12:17.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:12:17 smithi100 bash[74942]: audit 2024-04-07T17:12:16.819387+0000 mon.a (mon.0) 3162 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:12:17.953 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:12:17 smithi100 bash[76202]: audit 2024-04-07T17:12:16.819387+0000 mon.a (mon.0) 3162 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:12:18.703 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:12:18 smithi100 bash[74942]: cluster 2024-04-07T17:12:17.518999+0000 mgr.y (mgr.25175) 1708 : cluster [DBG] pgmap v2934: 457 pgs: 457 active+clean; 47 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 68 KiB/s rd, 1.2 MiB/s wr, 50 op/s 2024-04-07T17:12:18.703 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:12:18 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:12:18] "GET /metrics HTTP/1.1" 200 60778 "" "Prometheus/2.43.0" 2024-04-07T17:12:18.703 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:12:18 smithi100 bash[76202]: cluster 2024-04-07T17:12:17.518999+0000 mgr.y (mgr.25175) 1708 : cluster [DBG] pgmap v2934: 457 pgs: 457 active+clean; 47 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 68 KiB/s rd, 1.2 MiB/s wr, 50 op/s 2024-04-07T17:12:18.866 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:12:18 smithi144 bash[66654]: cluster 2024-04-07T17:12:17.518999+0000 mgr.y (mgr.25175) 1708 : cluster [DBG] pgmap v2934: 457 pgs: 457 active+clean; 47 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 68 KiB/s rd, 1.2 MiB/s wr, 50 op/s 2024-04-07T17:12:20.867 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:12:20 smithi144 bash[66654]: cluster 2024-04-07T17:12:19.521037+0000 mgr.y (mgr.25175) 1709 : cluster [DBG] pgmap v2935: 457 pgs: 457 active+clean; 55 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 83 KiB/s rd, 2.0 MiB/s wr, 34 op/s 2024-04-07T17:12:20.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:12:20 smithi100 bash[74942]: cluster 2024-04-07T17:12:19.521037+0000 mgr.y (mgr.25175) 1709 : cluster [DBG] pgmap v2935: 457 pgs: 457 active+clean; 55 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 83 KiB/s rd, 2.0 MiB/s wr, 34 op/s 2024-04-07T17:12:20.953 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:12:20 smithi100 bash[76202]: cluster 2024-04-07T17:12:19.521037+0000 mgr.y (mgr.25175) 1709 : cluster [DBG] pgmap v2935: 457 pgs: 457 active+clean; 55 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 83 KiB/s rd, 2.0 MiB/s wr, 34 op/s 2024-04-07T17:12:21.867 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:12:21 smithi144 bash[66654]: cluster 2024-04-07T17:12:20.607483+0000 mon.a (mon.0) 3163 : cluster [DBG] osdmap e1652: 8 total, 8 up, 8 in 2024-04-07T17:12:21.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:12:21 smithi100 bash[74942]: cluster 2024-04-07T17:12:20.607483+0000 mon.a (mon.0) 3163 : cluster [DBG] osdmap e1652: 8 total, 8 up, 8 in 2024-04-07T17:12:21.953 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:12:21 smithi100 bash[76202]: cluster 2024-04-07T17:12:20.607483+0000 mon.a (mon.0) 3163 : cluster [DBG] osdmap e1652: 8 total, 8 up, 8 in 2024-04-07T17:12:22.610 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:12:22 smithi144 bash[95233]: ts=2024-04-07T17:12:22.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:12:22.867 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:12:22 smithi144 bash[66654]: cluster 2024-04-07T17:12:21.522044+0000 mgr.y (mgr.25175) 1710 : cluster [DBG] pgmap v2937: 457 pgs: 457 active+clean; 55 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 55 KiB/s rd, 1.6 MiB/s wr, 35 op/s 2024-04-07T17:12:22.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:12:22 smithi100 bash[74942]: cluster 2024-04-07T17:12:21.522044+0000 mgr.y (mgr.25175) 1710 : cluster [DBG] pgmap v2937: 457 pgs: 457 active+clean; 55 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 55 KiB/s rd, 1.6 MiB/s wr, 35 op/s 2024-04-07T17:12:22.953 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:12:22 smithi100 bash[76202]: cluster 2024-04-07T17:12:21.522044+0000 mgr.y (mgr.25175) 1710 : cluster [DBG] pgmap v2937: 457 pgs: 457 active+clean; 55 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 55 KiB/s rd, 1.6 MiB/s wr, 35 op/s 2024-04-07T17:12:24.867 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:12:24 smithi144 bash[66654]: cluster 2024-04-07T17:12:23.524264+0000 mgr.y (mgr.25175) 1711 : cluster [DBG] pgmap v2938: 457 pgs: 457 active+clean; 55 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 55 KiB/s rd, 1.6 MiB/s wr, 34 op/s 2024-04-07T17:12:24.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:12:24 smithi100 bash[74942]: cluster 2024-04-07T17:12:23.524264+0000 mgr.y (mgr.25175) 1711 : cluster [DBG] pgmap v2938: 457 pgs: 457 active+clean; 55 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 55 KiB/s rd, 1.6 MiB/s wr, 34 op/s 2024-04-07T17:12:24.953 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:12:24 smithi100 bash[76202]: cluster 2024-04-07T17:12:23.524264+0000 mgr.y (mgr.25175) 1711 : cluster [DBG] pgmap v2938: 457 pgs: 457 active+clean; 55 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 55 KiB/s rd, 1.6 MiB/s wr, 34 op/s 2024-04-07T17:12:26.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:12:26 smithi100 bash[74942]: cluster 2024-04-07T17:12:25.525600+0000 mgr.y (mgr.25175) 1712 : cluster [DBG] pgmap v2939: 457 pgs: 457 active+clean; 59 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 60 KiB/s rd, 2.0 MiB/s wr, 40 op/s 2024-04-07T17:12:26.953 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:12:26 smithi100 bash[76202]: cluster 2024-04-07T17:12:25.525600+0000 mgr.y (mgr.25175) 1712 : cluster [DBG] pgmap v2939: 457 pgs: 457 active+clean; 59 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 60 KiB/s rd, 2.0 MiB/s wr, 40 op/s 2024-04-07T17:12:27.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:12:26 smithi144 bash[66654]: cluster 2024-04-07T17:12:25.525600+0000 mgr.y (mgr.25175) 1712 : cluster [DBG] pgmap v2939: 457 pgs: 457 active+clean; 59 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 60 KiB/s rd, 2.0 MiB/s wr, 40 op/s 2024-04-07T17:12:28.703 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:12:28 smithi100 bash[74942]: cluster 2024-04-07T17:12:27.527199+0000 mgr.y (mgr.25175) 1713 : cluster [DBG] pgmap v2940: 457 pgs: 457 active+clean; 59 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 58 KiB/s rd, 1.2 MiB/s wr, 36 op/s 2024-04-07T17:12:28.703 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:12:28 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:12:28] "GET /metrics HTTP/1.1" 200 60778 "" "Prometheus/2.43.0" 2024-04-07T17:12:28.703 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:12:28 smithi100 bash[76202]: cluster 2024-04-07T17:12:27.527199+0000 mgr.y (mgr.25175) 1713 : cluster [DBG] pgmap v2940: 457 pgs: 457 active+clean; 59 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 58 KiB/s rd, 1.2 MiB/s wr, 36 op/s 2024-04-07T17:12:29.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:12:28 smithi144 bash[66654]: cluster 2024-04-07T17:12:27.527199+0000 mgr.y (mgr.25175) 1713 : cluster [DBG] pgmap v2940: 457 pgs: 457 active+clean; 59 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 58 KiB/s rd, 1.2 MiB/s wr, 36 op/s 2024-04-07T17:12:30.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:12:30 smithi100 bash[74942]: cluster 2024-04-07T17:12:29.528533+0000 mgr.y (mgr.25175) 1714 : cluster [DBG] pgmap v2941: 457 pgs: 457 active+clean; 63 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 31 KiB/s rd, 821 KiB/s wr, 40 op/s 2024-04-07T17:12:30.953 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:12:30 smithi100 bash[76202]: cluster 2024-04-07T17:12:29.528533+0000 mgr.y (mgr.25175) 1714 : cluster [DBG] pgmap v2941: 457 pgs: 457 active+clean; 63 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 31 KiB/s rd, 821 KiB/s wr, 40 op/s 2024-04-07T17:12:31.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:12:30 smithi144 bash[66654]: cluster 2024-04-07T17:12:29.528533+0000 mgr.y (mgr.25175) 1714 : cluster [DBG] pgmap v2941: 457 pgs: 457 active+clean; 63 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 31 KiB/s rd, 821 KiB/s wr, 40 op/s 2024-04-07T17:12:32.617 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:12:32 smithi144 bash[95233]: ts=2024-04-07T17:12:32.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:12:32.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:12:32 smithi100 bash[74942]: cluster 2024-04-07T17:12:31.529460+0000 mgr.y (mgr.25175) 1715 : cluster [DBG] pgmap v2942: 457 pgs: 457 active+clean; 67 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 55 KiB/s rd, 1.1 MiB/s wr, 36 op/s 2024-04-07T17:12:32.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:12:32 smithi100 bash[74942]: audit 2024-04-07T17:12:31.819696+0000 mon.a (mon.0) 3164 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:12:32.953 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:12:32 smithi100 bash[76202]: cluster 2024-04-07T17:12:31.529460+0000 mgr.y (mgr.25175) 1715 : cluster [DBG] pgmap v2942: 457 pgs: 457 active+clean; 67 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 55 KiB/s rd, 1.1 MiB/s wr, 36 op/s 2024-04-07T17:12:32.953 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:12:32 smithi100 bash[76202]: audit 2024-04-07T17:12:31.819696+0000 mon.a (mon.0) 3164 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:12:33.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:12:32 smithi144 bash[66654]: cluster 2024-04-07T17:12:31.529460+0000 mgr.y (mgr.25175) 1715 : cluster [DBG] pgmap v2942: 457 pgs: 457 active+clean; 67 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 55 KiB/s rd, 1.1 MiB/s wr, 36 op/s 2024-04-07T17:12:33.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:12:32 smithi144 bash[66654]: audit 2024-04-07T17:12:31.819696+0000 mon.a (mon.0) 3164 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:12:34.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:12:34 smithi100 bash[74942]: cluster 2024-04-07T17:12:33.531350+0000 mgr.y (mgr.25175) 1716 : cluster [DBG] pgmap v2943: 457 pgs: 457 active+clean; 75 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 1.4 MiB/s rd, 1.7 MiB/s wr, 34 op/s 2024-04-07T17:12:34.953 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:12:34 smithi100 bash[76202]: cluster 2024-04-07T17:12:33.531350+0000 mgr.y (mgr.25175) 1716 : cluster [DBG] pgmap v2943: 457 pgs: 457 active+clean; 75 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 1.4 MiB/s rd, 1.7 MiB/s wr, 34 op/s 2024-04-07T17:12:35.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:12:34 smithi144 bash[66654]: cluster 2024-04-07T17:12:33.531350+0000 mgr.y (mgr.25175) 1716 : cluster [DBG] pgmap v2943: 457 pgs: 457 active+clean; 75 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 1.4 MiB/s rd, 1.7 MiB/s wr, 34 op/s 2024-04-07T17:12:36.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:12:36 smithi100 bash[74942]: cluster 2024-04-07T17:12:35.532704+0000 mgr.y (mgr.25175) 1717 : cluster [DBG] pgmap v2944: 457 pgs: 457 active+clean; 75 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 1.4 MiB/s rd, 1.7 MiB/s wr, 20 op/s 2024-04-07T17:12:36.953 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:12:36 smithi100 bash[76202]: cluster 2024-04-07T17:12:35.532704+0000 mgr.y (mgr.25175) 1717 : cluster [DBG] pgmap v2944: 457 pgs: 457 active+clean; 75 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 1.4 MiB/s rd, 1.7 MiB/s wr, 20 op/s 2024-04-07T17:12:37.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:12:36 smithi144 bash[66654]: cluster 2024-04-07T17:12:35.532704+0000 mgr.y (mgr.25175) 1717 : cluster [DBG] pgmap v2944: 457 pgs: 457 active+clean; 75 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 1.4 MiB/s rd, 1.7 MiB/s wr, 20 op/s 2024-04-07T17:12:38.703 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:12:38 smithi100 bash[74942]: cluster 2024-04-07T17:12:37.535343+0000 mgr.y (mgr.25175) 1718 : cluster [DBG] pgmap v2945: 457 pgs: 457 active+clean; 75 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 4 op/s 2024-04-07T17:12:38.703 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:12:38 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:12:38] "GET /metrics HTTP/1.1" 200 60782 "" "Prometheus/2.43.0" 2024-04-07T17:12:38.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:12:38 smithi100 bash[76202]: cluster 2024-04-07T17:12:37.535343+0000 mgr.y (mgr.25175) 1718 : cluster [DBG] pgmap v2945: 457 pgs: 457 active+clean; 75 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 4 op/s 2024-04-07T17:12:39.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:12:38 smithi144 bash[66654]: cluster 2024-04-07T17:12:37.535343+0000 mgr.y (mgr.25175) 1718 : cluster [DBG] pgmap v2945: 457 pgs: 457 active+clean; 75 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 4 op/s 2024-04-07T17:12:39.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:12:39 smithi100 bash[74942]: cluster 2024-04-07T17:12:38.659470+0000 mon.a (mon.0) 3165 : cluster [DBG] osdmap e1653: 8 total, 8 up, 8 in 2024-04-07T17:12:39.953 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:12:39 smithi100 bash[76202]: cluster 2024-04-07T17:12:38.659470+0000 mon.a (mon.0) 3165 : cluster [DBG] osdmap e1653: 8 total, 8 up, 8 in 2024-04-07T17:12:40.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:12:39 smithi144 bash[66654]: cluster 2024-04-07T17:12:38.659470+0000 mon.a (mon.0) 3165 : cluster [DBG] osdmap e1653: 8 total, 8 up, 8 in 2024-04-07T17:12:40.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:12:40 smithi100 bash[74942]: cluster 2024-04-07T17:12:39.536534+0000 mgr.y (mgr.25175) 1719 : cluster [DBG] pgmap v2947: 457 pgs: 457 active+clean; 75 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 1.7 MiB/s rd, 1.2 MiB/s wr, 9 op/s 2024-04-07T17:12:40.953 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:12:40 smithi100 bash[76202]: cluster 2024-04-07T17:12:39.536534+0000 mgr.y (mgr.25175) 1719 : cluster [DBG] pgmap v2947: 457 pgs: 457 active+clean; 75 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 1.7 MiB/s rd, 1.2 MiB/s wr, 9 op/s 2024-04-07T17:12:41.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:12:40 smithi144 bash[66654]: cluster 2024-04-07T17:12:39.536534+0000 mgr.y (mgr.25175) 1719 : cluster [DBG] pgmap v2947: 457 pgs: 457 active+clean; 75 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 1.7 MiB/s rd, 1.2 MiB/s wr, 9 op/s 2024-04-07T17:12:42.617 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:12:42 smithi144 bash[95233]: ts=2024-04-07T17:12:42.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:12:42.666 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:12:42 smithi144 bash[66654]: cluster 2024-04-07T17:12:41.537345+0000 mgr.y (mgr.25175) 1720 : cluster [DBG] pgmap v2948: 457 pgs: 457 active+clean; 75 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 1.7 MiB/s rd, 819 KiB/s wr, 16 op/s 2024-04-07T17:12:42.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:12:42 smithi100 bash[74942]: cluster 2024-04-07T17:12:41.537345+0000 mgr.y (mgr.25175) 1720 : cluster [DBG] pgmap v2948: 457 pgs: 457 active+clean; 75 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 1.7 MiB/s rd, 819 KiB/s wr, 16 op/s 2024-04-07T17:12:42.953 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:12:42 smithi100 bash[76202]: cluster 2024-04-07T17:12:41.537345+0000 mgr.y (mgr.25175) 1720 : cluster [DBG] pgmap v2948: 457 pgs: 457 active+clean; 75 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 1.7 MiB/s rd, 819 KiB/s wr, 16 op/s 2024-04-07T17:12:44.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:12:44 smithi100 bash[74942]: cluster 2024-04-07T17:12:43.539159+0000 mgr.y (mgr.25175) 1721 : cluster [DBG] pgmap v2949: 457 pgs: 457 active+clean; 75 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 121 KiB/s rd, 1.1 KiB/s wr, 42 op/s 2024-04-07T17:12:44.953 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:12:44 smithi100 bash[76202]: cluster 2024-04-07T17:12:43.539159+0000 mgr.y (mgr.25175) 1721 : cluster [DBG] pgmap v2949: 457 pgs: 457 active+clean; 75 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 121 KiB/s rd, 1.1 KiB/s wr, 42 op/s 2024-04-07T17:12:45.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:12:44 smithi144 bash[66654]: cluster 2024-04-07T17:12:43.539159+0000 mgr.y (mgr.25175) 1721 : cluster [DBG] pgmap v2949: 457 pgs: 457 active+clean; 75 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 121 KiB/s rd, 1.1 KiB/s wr, 42 op/s 2024-04-07T17:12:46.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:12:46 smithi100 bash[74942]: cluster 2024-04-07T17:12:45.541278+0000 mgr.y (mgr.25175) 1722 : cluster [DBG] pgmap v2950: 457 pgs: 457 active+clean; 75 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 149 KiB/s rd, 1.1 KiB/s wr, 42 op/s 2024-04-07T17:12:46.953 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:12:46 smithi100 bash[76202]: cluster 2024-04-07T17:12:45.541278+0000 mgr.y (mgr.25175) 1722 : cluster [DBG] pgmap v2950: 457 pgs: 457 active+clean; 75 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 149 KiB/s rd, 1.1 KiB/s wr, 42 op/s 2024-04-07T17:12:47.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:12:46 smithi144 bash[66654]: cluster 2024-04-07T17:12:45.541278+0000 mgr.y (mgr.25175) 1722 : cluster [DBG] pgmap v2950: 457 pgs: 457 active+clean; 75 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 149 KiB/s rd, 1.1 KiB/s wr, 42 op/s 2024-04-07T17:12:47.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:12:47 smithi100 bash[74942]: audit 2024-04-07T17:12:46.819782+0000 mon.a (mon.0) 3166 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:12:47.953 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:12:47 smithi100 bash[76202]: audit 2024-04-07T17:12:46.819782+0000 mon.a (mon.0) 3166 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:12:48.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:12:47 smithi144 bash[66654]: audit 2024-04-07T17:12:46.819782+0000 mon.a (mon.0) 3166 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:12:48.686 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:12:48 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:12:48] "GET /metrics HTTP/1.1" 200 60783 "" "Prometheus/2.43.0" 2024-04-07T17:12:48.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:12:48 smithi100 bash[74942]: cluster 2024-04-07T17:12:47.542988+0000 mgr.y (mgr.25175) 1723 : cluster [DBG] pgmap v2951: 457 pgs: 457 active+clean; 199 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 1.0 MiB/s rd, 12 MiB/s wr, 103 op/s 2024-04-07T17:12:48.953 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:12:48 smithi100 bash[76202]: cluster 2024-04-07T17:12:47.542988+0000 mgr.y (mgr.25175) 1723 : cluster [DBG] pgmap v2951: 457 pgs: 457 active+clean; 199 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 1.0 MiB/s rd, 12 MiB/s wr, 103 op/s 2024-04-07T17:12:49.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:12:48 smithi144 bash[66654]: cluster 2024-04-07T17:12:47.542988+0000 mgr.y (mgr.25175) 1723 : cluster [DBG] pgmap v2951: 457 pgs: 457 active+clean; 199 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 1.0 MiB/s rd, 12 MiB/s wr, 103 op/s 2024-04-07T17:12:50.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:12:50 smithi100 bash[74942]: cluster 2024-04-07T17:12:49.545257+0000 mgr.y (mgr.25175) 1724 : cluster [DBG] pgmap v2952: 457 pgs: 457 active+clean; 263 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 924 KiB/s rd, 21 MiB/s wr, 108 op/s 2024-04-07T17:12:50.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:12:50 smithi100 bash[74942]: cluster 2024-04-07T17:12:49.710027+0000 mon.a (mon.0) 3167 : cluster [DBG] osdmap e1654: 8 total, 8 up, 8 in 2024-04-07T17:12:50.953 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:12:50 smithi100 bash[76202]: cluster 2024-04-07T17:12:49.545257+0000 mgr.y (mgr.25175) 1724 : cluster [DBG] pgmap v2952: 457 pgs: 457 active+clean; 263 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 924 KiB/s rd, 21 MiB/s wr, 108 op/s 2024-04-07T17:12:50.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:12:50 smithi100 bash[76202]: cluster 2024-04-07T17:12:49.710027+0000 mon.a (mon.0) 3167 : cluster [DBG] osdmap e1654: 8 total, 8 up, 8 in 2024-04-07T17:12:51.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:12:50 smithi144 bash[66654]: cluster 2024-04-07T17:12:49.545257+0000 mgr.y (mgr.25175) 1724 : cluster [DBG] pgmap v2952: 457 pgs: 457 active+clean; 263 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 924 KiB/s rd, 21 MiB/s wr, 108 op/s 2024-04-07T17:12:51.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:12:50 smithi144 bash[66654]: cluster 2024-04-07T17:12:49.710027+0000 mon.a (mon.0) 3167 : cluster [DBG] osdmap e1654: 8 total, 8 up, 8 in 2024-04-07T17:12:52.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:12:51 smithi144 bash[66654]: cluster 2024-04-07T17:12:50.716003+0000 mon.a (mon.0) 3168 : cluster [DBG] osdmap e1655: 8 total, 8 up, 8 in 2024-04-07T17:12:52.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:12:51 smithi100 bash[74942]: cluster 2024-04-07T17:12:50.716003+0000 mon.a (mon.0) 3168 : cluster [DBG] osdmap e1655: 8 total, 8 up, 8 in 2024-04-07T17:12:52.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:12:51 smithi100 bash[76202]: cluster 2024-04-07T17:12:50.716003+0000 mon.a (mon.0) 3168 : cluster [DBG] osdmap e1655: 8 total, 8 up, 8 in 2024-04-07T17:12:52.617 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:12:52 smithi144 bash[95233]: ts=2024-04-07T17:12:52.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:12:52.970 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 4 2024-04-07T17:12:52.970 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 4 2024-04-07T17:12:52.970 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 4 2024-04-07T17:12:52.970 INFO:tasks.workunit.client.0.smithi100.stdout:read: 4 2024-04-07T17:12:52.970 INFO:tasks.workunit.client.0.smithi100.stdout:read: 4 2024-04-07T17:12:52.970 INFO:tasks.workunit.client.0.smithi100.stdout:read: 4 2024-04-07T17:12:52.971 INFO:tasks.workunit.client.0.smithi100.stdout:read: 4 2024-04-07T17:12:52.971 INFO:tasks.workunit.client.0.smithi100.stdout:read: 4 2024-04-07T17:12:52.971 INFO:tasks.workunit.client.0.smithi100.stdout:read: 4 2024-04-07T17:12:52.971 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.TestCloneEncryption (48889 ms) 2024-04-07T17:12:52.971 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.LUKS1UnderLUKS2WithoutResize 2024-04-07T17:12:52.971 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:12:53.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:12:52 smithi144 bash[66654]: cluster 2024-04-07T17:12:51.546293+0000 mgr.y (mgr.25175) 1725 : cluster [DBG] pgmap v2955: 457 pgs: 457 active+clean; 259 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 2.1 MiB/s rd, 32 MiB/s wr, 99 op/s 2024-04-07T17:12:53.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:12:52 smithi144 bash[66654]: cluster 2024-04-07T17:12:51.730379+0000 mon.a (mon.0) 3169 : cluster [DBG] osdmap e1656: 8 total, 8 up, 8 in 2024-04-07T17:12:53.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:12:52 smithi100 bash[76202]: cluster 2024-04-07T17:12:51.546293+0000 mgr.y (mgr.25175) 1725 : cluster [DBG] pgmap v2955: 457 pgs: 457 active+clean; 259 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 2.1 MiB/s rd, 32 MiB/s wr, 99 op/s 2024-04-07T17:12:53.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:12:52 smithi100 bash[76202]: cluster 2024-04-07T17:12:51.730379+0000 mon.a (mon.0) 3169 : cluster [DBG] osdmap e1656: 8 total, 8 up, 8 in 2024-04-07T17:12:53.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:12:52 smithi100 bash[74942]: cluster 2024-04-07T17:12:51.546293+0000 mgr.y (mgr.25175) 1725 : cluster [DBG] pgmap v2955: 457 pgs: 457 active+clean; 259 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 2.1 MiB/s rd, 32 MiB/s wr, 99 op/s 2024-04-07T17:12:53.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:12:52 smithi100 bash[74942]: cluster 2024-04-07T17:12:51.730379+0000 mon.a (mon.0) 3169 : cluster [DBG] osdmap e1656: 8 total, 8 up, 8 in 2024-04-07T17:12:54.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:12:53 smithi144 bash[66654]: cluster 2024-04-07T17:12:52.729101+0000 mon.a (mon.0) 3170 : cluster [DBG] osdmap e1657: 8 total, 8 up, 8 in 2024-04-07T17:12:54.134 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:12:53 smithi144 bash[66654]: audit 2024-04-07T17:12:52.871665+0000 mon.a (mon.0) 3171 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:12:54.134 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:12:53 smithi144 bash[66654]: audit 2024-04-07T17:12:53.610492+0000 mon.a (mon.0) 3172 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:12:54.134 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:12:53 smithi144 bash[66654]: audit 2024-04-07T17:12:53.612162+0000 mon.a (mon.0) 3173 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:12:54.134 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:12:53 smithi144 bash[66654]: audit 2024-04-07T17:12:53.620245+0000 mon.a (mon.0) 3174 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:12:54.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:12:53 smithi100 bash[74942]: cluster 2024-04-07T17:12:52.729101+0000 mon.a (mon.0) 3170 : cluster [DBG] osdmap e1657: 8 total, 8 up, 8 in 2024-04-07T17:12:54.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:12:53 smithi100 bash[74942]: audit 2024-04-07T17:12:52.871665+0000 mon.a (mon.0) 3171 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:12:54.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:12:53 smithi100 bash[74942]: audit 2024-04-07T17:12:53.610492+0000 mon.a (mon.0) 3172 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:12:54.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:12:53 smithi100 bash[74942]: audit 2024-04-07T17:12:53.612162+0000 mon.a (mon.0) 3173 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:12:54.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:12:53 smithi100 bash[74942]: audit 2024-04-07T17:12:53.620245+0000 mon.a (mon.0) 3174 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:12:54.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:12:53 smithi100 bash[76202]: cluster 2024-04-07T17:12:52.729101+0000 mon.a (mon.0) 3170 : cluster [DBG] osdmap e1657: 8 total, 8 up, 8 in 2024-04-07T17:12:54.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:12:53 smithi100 bash[76202]: audit 2024-04-07T17:12:52.871665+0000 mon.a (mon.0) 3171 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:12:54.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:12:53 smithi100 bash[76202]: audit 2024-04-07T17:12:53.610492+0000 mon.a (mon.0) 3172 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:12:54.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:12:53 smithi100 bash[76202]: audit 2024-04-07T17:12:53.612162+0000 mon.a (mon.0) 3173 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:12:54.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:12:53 smithi100 bash[76202]: audit 2024-04-07T17:12:53.620245+0000 mon.a (mon.0) 3174 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:12:55.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:12:54 smithi144 bash[66654]: cluster 2024-04-07T17:12:53.547244+0000 mgr.y (mgr.25175) 1726 : cluster [DBG] pgmap v2958: 457 pgs: 457 active+clean; 43 MiB data, 3.7 GiB used, 711 GiB / 715 GiB avail; 2.1 MiB/s rd, 7.0 MiB/s wr, 162 op/s 2024-04-07T17:12:55.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:12:54 smithi100 bash[74942]: cluster 2024-04-07T17:12:53.547244+0000 mgr.y (mgr.25175) 1726 : cluster [DBG] pgmap v2958: 457 pgs: 457 active+clean; 43 MiB data, 3.7 GiB used, 711 GiB / 715 GiB avail; 2.1 MiB/s rd, 7.0 MiB/s wr, 162 op/s 2024-04-07T17:12:55.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:12:54 smithi100 bash[76202]: cluster 2024-04-07T17:12:53.547244+0000 mgr.y (mgr.25175) 1726 : cluster [DBG] pgmap v2958: 457 pgs: 457 active+clean; 43 MiB data, 3.7 GiB used, 711 GiB / 715 GiB avail; 2.1 MiB/s rd, 7.0 MiB/s wr, 162 op/s 2024-04-07T17:12:57.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:12:56 smithi144 bash[66654]: cluster 2024-04-07T17:12:55.548428+0000 mgr.y (mgr.25175) 1727 : cluster [DBG] pgmap v2959: 457 pgs: 457 active+clean; 35 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 1.5 MiB/s rd, 4.8 MiB/s wr, 190 op/s 2024-04-07T17:12:57.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:12:56 smithi100 bash[74942]: cluster 2024-04-07T17:12:55.548428+0000 mgr.y (mgr.25175) 1727 : cluster [DBG] pgmap v2959: 457 pgs: 457 active+clean; 35 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 1.5 MiB/s rd, 4.8 MiB/s wr, 190 op/s 2024-04-07T17:12:57.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:12:56 smithi100 bash[76202]: cluster 2024-04-07T17:12:55.548428+0000 mgr.y (mgr.25175) 1727 : cluster [DBG] pgmap v2959: 457 pgs: 457 active+clean; 35 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 1.5 MiB/s rd, 4.8 MiB/s wr, 190 op/s 2024-04-07T17:12:58.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:12:57 smithi144 bash[66654]: cluster 2024-04-07T17:12:56.772512+0000 mon.a (mon.0) 3175 : cluster [DBG] osdmap e1658: 8 total, 8 up, 8 in 2024-04-07T17:12:58.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:12:57 smithi100 bash[74942]: cluster 2024-04-07T17:12:56.772512+0000 mon.a (mon.0) 3175 : cluster [DBG] osdmap e1658: 8 total, 8 up, 8 in 2024-04-07T17:12:58.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:12:57 smithi100 bash[76202]: cluster 2024-04-07T17:12:56.772512+0000 mon.a (mon.0) 3175 : cluster [DBG] osdmap e1658: 8 total, 8 up, 8 in 2024-04-07T17:12:58.703 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:12:58 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:12:58] "GET /metrics HTTP/1.1" 200 60783 "" "Prometheus/2.43.0" 2024-04-07T17:12:59.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:12:58 smithi144 bash[66654]: cluster 2024-04-07T17:12:57.550910+0000 mgr.y (mgr.25175) 1728 : cluster [DBG] pgmap v2961: 457 pgs: 457 active+clean; 52 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 142 KiB/s rd, 2.8 MiB/s wr, 207 op/s 2024-04-07T17:12:59.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:12:58 smithi144 bash[66654]: cluster 2024-04-07T17:12:57.787149+0000 mon.a (mon.0) 3176 : cluster [DBG] osdmap e1659: 8 total, 8 up, 8 in 2024-04-07T17:12:59.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:12:58 smithi100 bash[76202]: cluster 2024-04-07T17:12:57.550910+0000 mgr.y (mgr.25175) 1728 : cluster [DBG] pgmap v2961: 457 pgs: 457 active+clean; 52 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 142 KiB/s rd, 2.8 MiB/s wr, 207 op/s 2024-04-07T17:12:59.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:12:58 smithi100 bash[76202]: cluster 2024-04-07T17:12:57.787149+0000 mon.a (mon.0) 3176 : cluster [DBG] osdmap e1659: 8 total, 8 up, 8 in 2024-04-07T17:12:59.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:12:58 smithi100 bash[74942]: cluster 2024-04-07T17:12:57.550910+0000 mgr.y (mgr.25175) 1728 : cluster [DBG] pgmap v2961: 457 pgs: 457 active+clean; 52 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 142 KiB/s rd, 2.8 MiB/s wr, 207 op/s 2024-04-07T17:12:59.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:12:58 smithi100 bash[74942]: cluster 2024-04-07T17:12:57.787149+0000 mon.a (mon.0) 3176 : cluster [DBG] osdmap e1659: 8 total, 8 up, 8 in 2024-04-07T17:13:01.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:13:00 smithi144 bash[66654]: cluster 2024-04-07T17:12:59.552365+0000 mgr.y (mgr.25175) 1729 : cluster [DBG] pgmap v2963: 457 pgs: 457 active+clean; 60 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 100 KiB/s rd, 3.7 MiB/s wr, 139 op/s 2024-04-07T17:13:01.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:13:00 smithi100 bash[74942]: cluster 2024-04-07T17:12:59.552365+0000 mgr.y (mgr.25175) 1729 : cluster [DBG] pgmap v2963: 457 pgs: 457 active+clean; 60 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 100 KiB/s rd, 3.7 MiB/s wr, 139 op/s 2024-04-07T17:13:01.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:13:00 smithi100 bash[76202]: cluster 2024-04-07T17:12:59.552365+0000 mgr.y (mgr.25175) 1729 : cluster [DBG] pgmap v2963: 457 pgs: 457 active+clean; 60 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 100 KiB/s rd, 3.7 MiB/s wr, 139 op/s 2024-04-07T17:13:02.617 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:13:02 smithi144 bash[95233]: ts=2024-04-07T17:13:02.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:13:03.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:13:02 smithi144 bash[66654]: cluster 2024-04-07T17:13:01.553308+0000 mgr.y (mgr.25175) 1730 : cluster [DBG] pgmap v2964: 457 pgs: 457 active+clean; 64 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 75 KiB/s rd, 3.6 MiB/s wr, 105 op/s 2024-04-07T17:13:03.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:13:02 smithi144 bash[66654]: audit 2024-04-07T17:13:01.820460+0000 mon.a (mon.0) 3177 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:13:03.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:13:02 smithi100 bash[74942]: cluster 2024-04-07T17:13:01.553308+0000 mgr.y (mgr.25175) 1730 : cluster [DBG] pgmap v2964: 457 pgs: 457 active+clean; 64 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 75 KiB/s rd, 3.6 MiB/s wr, 105 op/s 2024-04-07T17:13:03.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:13:02 smithi100 bash[74942]: audit 2024-04-07T17:13:01.820460+0000 mon.a (mon.0) 3177 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:13:03.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:13:02 smithi100 bash[76202]: cluster 2024-04-07T17:13:01.553308+0000 mgr.y (mgr.25175) 1730 : cluster [DBG] pgmap v2964: 457 pgs: 457 active+clean; 64 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 75 KiB/s rd, 3.6 MiB/s wr, 105 op/s 2024-04-07T17:13:03.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:13:02 smithi100 bash[76202]: audit 2024-04-07T17:13:01.820460+0000 mon.a (mon.0) 3177 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:13:05.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:13:04 smithi144 bash[66654]: cluster 2024-04-07T17:13:03.555113+0000 mgr.y (mgr.25175) 1731 : cluster [DBG] pgmap v2965: 457 pgs: 457 active+clean; 68 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 69 KiB/s rd, 4.1 MiB/s wr, 48 op/s 2024-04-07T17:13:05.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:13:04 smithi100 bash[76202]: cluster 2024-04-07T17:13:03.555113+0000 mgr.y (mgr.25175) 1731 : cluster [DBG] pgmap v2965: 457 pgs: 457 active+clean; 68 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 69 KiB/s rd, 4.1 MiB/s wr, 48 op/s 2024-04-07T17:13:05.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:13:04 smithi100 bash[74942]: cluster 2024-04-07T17:13:03.555113+0000 mgr.y (mgr.25175) 1731 : cluster [DBG] pgmap v2965: 457 pgs: 457 active+clean; 68 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 69 KiB/s rd, 4.1 MiB/s wr, 48 op/s 2024-04-07T17:13:06.794 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.LUKS1UnderLUKS2WithoutResize (13825 ms) 2024-04-07T17:13:06.794 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.LUKS2UnderLUKS1WithoutResize 2024-04-07T17:13:06.794 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:13:07.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:13:06 smithi144 bash[66654]: cluster 2024-04-07T17:13:05.557115+0000 mgr.y (mgr.25175) 1732 : cluster [DBG] pgmap v2966: 457 pgs: 457 active+clean; 68 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 96 KiB/s rd, 3.8 MiB/s wr, 44 op/s 2024-04-07T17:13:07.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:13:06 smithi100 bash[76202]: cluster 2024-04-07T17:13:05.557115+0000 mgr.y (mgr.25175) 1732 : cluster [DBG] pgmap v2966: 457 pgs: 457 active+clean; 68 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 96 KiB/s rd, 3.8 MiB/s wr, 44 op/s 2024-04-07T17:13:07.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:13:06 smithi100 bash[74942]: cluster 2024-04-07T17:13:05.557115+0000 mgr.y (mgr.25175) 1732 : cluster [DBG] pgmap v2966: 457 pgs: 457 active+clean; 68 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 96 KiB/s rd, 3.8 MiB/s wr, 44 op/s 2024-04-07T17:13:08.703 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:13:08 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:13:08] "GET /metrics HTTP/1.1" 200 60785 "" "Prometheus/2.43.0" 2024-04-07T17:13:09.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:13:08 smithi144 bash[66654]: cluster 2024-04-07T17:13:07.558516+0000 mgr.y (mgr.25175) 1733 : cluster [DBG] pgmap v2967: 457 pgs: 457 active+clean; 80 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 998 KiB/s rd, 2.8 MiB/s wr, 26 op/s 2024-04-07T17:13:09.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:13:08 smithi100 bash[74942]: cluster 2024-04-07T17:13:07.558516+0000 mgr.y (mgr.25175) 1733 : cluster [DBG] pgmap v2967: 457 pgs: 457 active+clean; 80 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 998 KiB/s rd, 2.8 MiB/s wr, 26 op/s 2024-04-07T17:13:09.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:13:08 smithi100 bash[76202]: cluster 2024-04-07T17:13:07.558516+0000 mgr.y (mgr.25175) 1733 : cluster [DBG] pgmap v2967: 457 pgs: 457 active+clean; 80 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 998 KiB/s rd, 2.8 MiB/s wr, 26 op/s 2024-04-07T17:13:11.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:13:10 smithi144 bash[66654]: cluster 2024-04-07T17:13:09.559999+0000 mgr.y (mgr.25175) 1734 : cluster [DBG] pgmap v2968: 457 pgs: 457 active+clean; 80 MiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 24 op/s 2024-04-07T17:13:11.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:13:10 smithi100 bash[74942]: cluster 2024-04-07T17:13:09.559999+0000 mgr.y (mgr.25175) 1734 : cluster [DBG] pgmap v2968: 457 pgs: 457 active+clean; 80 MiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 24 op/s 2024-04-07T17:13:11.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:13:10 smithi100 bash[76202]: cluster 2024-04-07T17:13:09.559999+0000 mgr.y (mgr.25175) 1734 : cluster [DBG] pgmap v2968: 457 pgs: 457 active+clean; 80 MiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 24 op/s 2024-04-07T17:13:12.617 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:13:12 smithi144 bash[95233]: ts=2024-04-07T17:13:12.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:13:13.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:13:12 smithi144 bash[66654]: cluster 2024-04-07T17:13:11.560939+0000 mgr.y (mgr.25175) 1735 : cluster [DBG] pgmap v2969: 457 pgs: 457 active+clean; 80 MiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 1.1 MiB/s rd, 1.7 MiB/s wr, 8 op/s 2024-04-07T17:13:13.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:13:12 smithi100 bash[74942]: cluster 2024-04-07T17:13:11.560939+0000 mgr.y (mgr.25175) 1735 : cluster [DBG] pgmap v2969: 457 pgs: 457 active+clean; 80 MiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 1.1 MiB/s rd, 1.7 MiB/s wr, 8 op/s 2024-04-07T17:13:13.205 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:13:12 smithi100 bash[76202]: cluster 2024-04-07T17:13:11.560939+0000 mgr.y (mgr.25175) 1735 : cluster [DBG] pgmap v2969: 457 pgs: 457 active+clean; 80 MiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 1.1 MiB/s rd, 1.7 MiB/s wr, 8 op/s 2024-04-07T17:13:15.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:13:14 smithi144 bash[66654]: cluster 2024-04-07T17:13:13.562770+0000 mgr.y (mgr.25175) 1736 : cluster [DBG] pgmap v2970: 457 pgs: 457 active+clean; 80 MiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 8 op/s 2024-04-07T17:13:15.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:13:14 smithi100 bash[76202]: cluster 2024-04-07T17:13:13.562770+0000 mgr.y (mgr.25175) 1736 : cluster [DBG] pgmap v2970: 457 pgs: 457 active+clean; 80 MiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 8 op/s 2024-04-07T17:13:15.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:13:14 smithi100 bash[74942]: cluster 2024-04-07T17:13:13.562770+0000 mgr.y (mgr.25175) 1736 : cluster [DBG] pgmap v2970: 457 pgs: 457 active+clean; 80 MiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 8 op/s 2024-04-07T17:13:16.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:13:15 smithi144 bash[66654]: cluster 2024-04-07T17:13:14.835631+0000 mon.a (mon.0) 3178 : cluster [DBG] osdmap e1660: 8 total, 8 up, 8 in 2024-04-07T17:13:16.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:13:15 smithi100 bash[76202]: cluster 2024-04-07T17:13:14.835631+0000 mon.a (mon.0) 3178 : cluster [DBG] osdmap e1660: 8 total, 8 up, 8 in 2024-04-07T17:13:16.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:13:15 smithi100 bash[74942]: cluster 2024-04-07T17:13:14.835631+0000 mon.a (mon.0) 3178 : cluster [DBG] osdmap e1660: 8 total, 8 up, 8 in 2024-04-07T17:13:17.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:13:16 smithi144 bash[66654]: cluster 2024-04-07T17:13:15.564152+0000 mgr.y (mgr.25175) 1737 : cluster [DBG] pgmap v2972: 457 pgs: 457 active+clean; 80 MiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 15 op/s 2024-04-07T17:13:17.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:13:16 smithi144 bash[66654]: audit 2024-04-07T17:13:16.821592+0000 mon.a (mon.0) 3179 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:13:17.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:13:16 smithi100 bash[74942]: cluster 2024-04-07T17:13:15.564152+0000 mgr.y (mgr.25175) 1737 : cluster [DBG] pgmap v2972: 457 pgs: 457 active+clean; 80 MiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 15 op/s 2024-04-07T17:13:17.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:13:16 smithi100 bash[74942]: audit 2024-04-07T17:13:16.821592+0000 mon.a (mon.0) 3179 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:13:17.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:13:16 smithi100 bash[76202]: cluster 2024-04-07T17:13:15.564152+0000 mgr.y (mgr.25175) 1737 : cluster [DBG] pgmap v2972: 457 pgs: 457 active+clean; 80 MiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 15 op/s 2024-04-07T17:13:17.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:13:16 smithi100 bash[76202]: audit 2024-04-07T17:13:16.821592+0000 mon.a (mon.0) 3179 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:13:18.703 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:13:18 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:13:18] "GET /metrics HTTP/1.1" 200 60784 "" "Prometheus/2.43.0" 2024-04-07T17:13:19.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:13:18 smithi144 bash[66654]: cluster 2024-04-07T17:13:17.566690+0000 mgr.y (mgr.25175) 1738 : cluster [DBG] pgmap v2973: 457 pgs: 457 active+clean; 96 MiB data, 3.7 GiB used, 711 GiB / 715 GiB avail; 418 KiB/s rd, 1.6 MiB/s wr, 13 op/s 2024-04-07T17:13:19.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:13:18 smithi100 bash[76202]: cluster 2024-04-07T17:13:17.566690+0000 mgr.y (mgr.25175) 1738 : cluster [DBG] pgmap v2973: 457 pgs: 457 active+clean; 96 MiB data, 3.7 GiB used, 711 GiB / 715 GiB avail; 418 KiB/s rd, 1.6 MiB/s wr, 13 op/s 2024-04-07T17:13:19.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:13:18 smithi100 bash[74942]: cluster 2024-04-07T17:13:17.566690+0000 mgr.y (mgr.25175) 1738 : cluster [DBG] pgmap v2973: 457 pgs: 457 active+clean; 96 MiB data, 3.7 GiB used, 711 GiB / 715 GiB avail; 418 KiB/s rd, 1.6 MiB/s wr, 13 op/s 2024-04-07T17:13:21.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:13:20 smithi144 bash[66654]: cluster 2024-04-07T17:13:19.568802+0000 mgr.y (mgr.25175) 1739 : cluster [DBG] pgmap v2974: 457 pgs: 457 active+clean; 125 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 48 KiB/s rd, 4.5 MiB/s wr, 29 op/s 2024-04-07T17:13:21.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:13:20 smithi100 bash[74942]: cluster 2024-04-07T17:13:19.568802+0000 mgr.y (mgr.25175) 1739 : cluster [DBG] pgmap v2974: 457 pgs: 457 active+clean; 125 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 48 KiB/s rd, 4.5 MiB/s wr, 29 op/s 2024-04-07T17:13:21.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:13:20 smithi100 bash[76202]: cluster 2024-04-07T17:13:19.568802+0000 mgr.y (mgr.25175) 1739 : cluster [DBG] pgmap v2974: 457 pgs: 457 active+clean; 125 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 48 KiB/s rd, 4.5 MiB/s wr, 29 op/s 2024-04-07T17:13:22.617 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:13:22 smithi144 bash[95233]: ts=2024-04-07T17:13:22.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:13:23.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:13:22 smithi144 bash[66654]: cluster 2024-04-07T17:13:21.569809+0000 mgr.y (mgr.25175) 1740 : cluster [DBG] pgmap v2975: 457 pgs: 457 active+clean; 125 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 53 KiB/s rd, 4.5 MiB/s wr, 34 op/s 2024-04-07T17:13:23.139 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.LUKS2UnderLUKS1WithoutResize (16345 ms) 2024-04-07T17:13:23.139 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.EncryptionFormatNoData 2024-04-07T17:13:23.139 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:13:23.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:13:22 smithi100 bash[74942]: cluster 2024-04-07T17:13:21.569809+0000 mgr.y (mgr.25175) 1740 : cluster [DBG] pgmap v2975: 457 pgs: 457 active+clean; 125 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 53 KiB/s rd, 4.5 MiB/s wr, 34 op/s 2024-04-07T17:13:23.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:13:22 smithi100 bash[76202]: cluster 2024-04-07T17:13:21.569809+0000 mgr.y (mgr.25175) 1740 : cluster [DBG] pgmap v2975: 457 pgs: 457 active+clean; 125 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 53 KiB/s rd, 4.5 MiB/s wr, 34 op/s 2024-04-07T17:13:25.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:13:24 smithi144 bash[66654]: cluster 2024-04-07T17:13:23.572844+0000 mgr.y (mgr.25175) 1741 : cluster [DBG] pgmap v2976: 457 pgs: 457 active+clean; 125 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 83 KiB/s rd, 4.5 MiB/s wr, 36 op/s 2024-04-07T17:13:25.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:13:24 smithi100 bash[74942]: cluster 2024-04-07T17:13:23.572844+0000 mgr.y (mgr.25175) 1741 : cluster [DBG] pgmap v2976: 457 pgs: 457 active+clean; 125 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 83 KiB/s rd, 4.5 MiB/s wr, 36 op/s 2024-04-07T17:13:25.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:13:24 smithi100 bash[76202]: cluster 2024-04-07T17:13:23.572844+0000 mgr.y (mgr.25175) 1741 : cluster [DBG] pgmap v2976: 457 pgs: 457 active+clean; 125 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 83 KiB/s rd, 4.5 MiB/s wr, 36 op/s 2024-04-07T17:13:27.118 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:13:26 smithi144 bash[66654]: cluster 2024-04-07T17:13:25.574962+0000 mgr.y (mgr.25175) 1742 : cluster [DBG] pgmap v2977: 457 pgs: 457 active+clean; 125 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 1.2 MiB/s rd, 4.2 MiB/s wr, 36 op/s 2024-04-07T17:13:27.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:13:26 smithi100 bash[74942]: cluster 2024-04-07T17:13:25.574962+0000 mgr.y (mgr.25175) 1742 : cluster [DBG] pgmap v2977: 457 pgs: 457 active+clean; 125 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 1.2 MiB/s rd, 4.2 MiB/s wr, 36 op/s 2024-04-07T17:13:27.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:13:26 smithi100 bash[76202]: cluster 2024-04-07T17:13:25.574962+0000 mgr.y (mgr.25175) 1742 : cluster [DBG] pgmap v2977: 457 pgs: 457 active+clean; 125 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 1.2 MiB/s rd, 4.2 MiB/s wr, 36 op/s 2024-04-07T17:13:27.292 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.EncryptionFormatNoData (4153 ms) 2024-04-07T17:13:27.292 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.EncryptionLoadBadSize 2024-04-07T17:13:27.292 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:13:28.703 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:13:28 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:13:28] "GET /metrics HTTP/1.1" 200 60784 "" "Prometheus/2.43.0" 2024-04-07T17:13:29.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:13:28 smithi144 bash[66654]: cluster 2024-04-07T17:13:27.577542+0000 mgr.y (mgr.25175) 1743 : cluster [DBG] pgmap v2978: 457 pgs: 457 active+clean; 125 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 1.7 MiB/s rd, 3.7 MiB/s wr, 27 op/s 2024-04-07T17:13:29.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:13:28 smithi100 bash[74942]: cluster 2024-04-07T17:13:27.577542+0000 mgr.y (mgr.25175) 1743 : cluster [DBG] pgmap v2978: 457 pgs: 457 active+clean; 125 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 1.7 MiB/s rd, 3.7 MiB/s wr, 27 op/s 2024-04-07T17:13:29.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:13:28 smithi100 bash[76202]: cluster 2024-04-07T17:13:27.577542+0000 mgr.y (mgr.25175) 1743 : cluster [DBG] pgmap v2978: 457 pgs: 457 active+clean; 125 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 1.7 MiB/s rd, 3.7 MiB/s wr, 27 op/s 2024-04-07T17:13:31.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:13:30 smithi100 bash[74942]: cluster 2024-04-07T17:13:29.579609+0000 mgr.y (mgr.25175) 1744 : cluster [DBG] pgmap v2979: 457 pgs: 457 active+clean; 125 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 2.1 MiB/s rd, 2.4 MiB/s wr, 32 op/s 2024-04-07T17:13:31.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:13:30 smithi100 bash[76202]: cluster 2024-04-07T17:13:29.579609+0000 mgr.y (mgr.25175) 1744 : cluster [DBG] pgmap v2979: 457 pgs: 457 active+clean; 125 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 2.1 MiB/s rd, 2.4 MiB/s wr, 32 op/s 2024-04-07T17:13:31.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:13:30 smithi144 bash[66654]: cluster 2024-04-07T17:13:29.579609+0000 mgr.y (mgr.25175) 1744 : cluster [DBG] pgmap v2979: 457 pgs: 457 active+clean; 125 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 2.1 MiB/s rd, 2.4 MiB/s wr, 32 op/s 2024-04-07T17:13:32.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:13:31 smithi100 bash[76202]: audit 2024-04-07T17:13:31.821893+0000 mon.a (mon.0) 3180 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:13:32.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:13:31 smithi100 bash[74942]: audit 2024-04-07T17:13:31.821893+0000 mon.a (mon.0) 3180 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:13:32.322 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:13:31 smithi144 bash[66654]: audit 2024-04-07T17:13:31.821893+0000 mon.a (mon.0) 3180 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:13:32.617 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:13:32 smithi144 bash[95233]: ts=2024-04-07T17:13:32.317Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:13:33.009 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.EncryptionLoadBadSize (5717 ms) 2024-04-07T17:13:33.009 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.EncryptionLoadBadStripePattern 2024-04-07T17:13:33.010 INFO:tasks.workunit.client.0.smithi100.stdout:./src/test/librbd/test_librbd.cc:2840: Skipped 2024-04-07T17:13:33.010 INFO:tasks.workunit.client.0.smithi100.stdout:Skipping due to unmet REQUIRE 2024-04-07T17:13:33.010 INFO:tasks.workunit.client.0.smithi100.stdout:[ SKIPPED ] TestLibRBD.EncryptionLoadBadStripePattern (0 ms) 2024-04-07T17:13:33.010 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.EncryptionLoadFormatMismatch 2024-04-07T17:13:33.010 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:13:33.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:13:32 smithi100 bash[76202]: cluster 2024-04-07T17:13:31.580940+0000 mgr.y (mgr.25175) 1745 : cluster [DBG] pgmap v2980: 457 pgs: 457 active+clean; 125 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 2.1 MiB/s rd, 1022 B/s wr, 20 op/s 2024-04-07T17:13:33.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:13:32 smithi100 bash[74942]: cluster 2024-04-07T17:13:31.580940+0000 mgr.y (mgr.25175) 1745 : cluster [DBG] pgmap v2980: 457 pgs: 457 active+clean; 125 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 2.1 MiB/s rd, 1022 B/s wr, 20 op/s 2024-04-07T17:13:33.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:13:32 smithi144 bash[66654]: cluster 2024-04-07T17:13:31.580940+0000 mgr.y (mgr.25175) 1745 : cluster [DBG] pgmap v2980: 457 pgs: 457 active+clean; 125 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 2.1 MiB/s rd, 1022 B/s wr, 20 op/s 2024-04-07T17:13:35.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:13:34 smithi100 bash[76202]: cluster 2024-04-07T17:13:33.582657+0000 mgr.y (mgr.25175) 1746 : cluster [DBG] pgmap v2981: 457 pgs: 457 active+clean; 133 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 2.1 MiB/s rd, 683 KiB/s wr, 16 op/s 2024-04-07T17:13:35.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:13:34 smithi100 bash[74942]: cluster 2024-04-07T17:13:33.582657+0000 mgr.y (mgr.25175) 1746 : cluster [DBG] pgmap v2981: 457 pgs: 457 active+clean; 133 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 2.1 MiB/s rd, 683 KiB/s wr, 16 op/s 2024-04-07T17:13:35.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:13:34 smithi144 bash[66654]: cluster 2024-04-07T17:13:33.582657+0000 mgr.y (mgr.25175) 1746 : cluster [DBG] pgmap v2981: 457 pgs: 457 active+clean; 133 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 2.1 MiB/s rd, 683 KiB/s wr, 16 op/s 2024-04-07T17:13:37.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:13:36 smithi100 bash[74942]: cluster 2024-04-07T17:13:35.583821+0000 mgr.y (mgr.25175) 1747 : cluster [DBG] pgmap v2982: 457 pgs: 457 active+clean; 133 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 2.1 MiB/s rd, 683 KiB/s wr, 21 op/s 2024-04-07T17:13:37.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:13:36 smithi100 bash[76202]: cluster 2024-04-07T17:13:35.583821+0000 mgr.y (mgr.25175) 1747 : cluster [DBG] pgmap v2982: 457 pgs: 457 active+clean; 133 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 2.1 MiB/s rd, 683 KiB/s wr, 21 op/s 2024-04-07T17:13:37.369 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:13:36 smithi144 bash[66654]: cluster 2024-04-07T17:13:35.583821+0000 mgr.y (mgr.25175) 1747 : cluster [DBG] pgmap v2982: 457 pgs: 457 active+clean; 133 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 2.1 MiB/s rd, 683 KiB/s wr, 21 op/s 2024-04-07T17:13:38.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:13:37 smithi100 bash[74942]: cluster 2024-04-07T17:13:36.909087+0000 mon.a (mon.0) 3181 : cluster [DBG] osdmap e1661: 8 total, 8 up, 8 in 2024-04-07T17:13:38.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:13:37 smithi100 bash[76202]: cluster 2024-04-07T17:13:36.909087+0000 mon.a (mon.0) 3181 : cluster [DBG] osdmap e1661: 8 total, 8 up, 8 in 2024-04-07T17:13:38.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:13:37 smithi144 bash[66654]: cluster 2024-04-07T17:13:36.909087+0000 mon.a (mon.0) 3181 : cluster [DBG] osdmap e1661: 8 total, 8 up, 8 in 2024-04-07T17:13:38.703 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:13:38 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:13:38] "GET /metrics HTTP/1.1" 200 60785 "" "Prometheus/2.43.0" 2024-04-07T17:13:39.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:13:38 smithi100 bash[76202]: cluster 2024-04-07T17:13:37.586590+0000 mgr.y (mgr.25175) 1748 : cluster [DBG] pgmap v2984: 457 pgs: 457 active+clean; 133 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 597 KiB/s rd, 821 KiB/s wr, 38 op/s 2024-04-07T17:13:39.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:13:38 smithi100 bash[76202]: cluster 2024-04-07T17:13:37.921859+0000 mon.a (mon.0) 3182 : cluster [DBG] osdmap e1662: 8 total, 8 up, 8 in 2024-04-07T17:13:39.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:13:38 smithi100 bash[74942]: cluster 2024-04-07T17:13:37.586590+0000 mgr.y (mgr.25175) 1748 : cluster [DBG] pgmap v2984: 457 pgs: 457 active+clean; 133 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 597 KiB/s rd, 821 KiB/s wr, 38 op/s 2024-04-07T17:13:39.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:13:38 smithi100 bash[74942]: cluster 2024-04-07T17:13:37.921859+0000 mon.a (mon.0) 3182 : cluster [DBG] osdmap e1662: 8 total, 8 up, 8 in 2024-04-07T17:13:39.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:13:38 smithi144 bash[66654]: cluster 2024-04-07T17:13:37.586590+0000 mgr.y (mgr.25175) 1748 : cluster [DBG] pgmap v2984: 457 pgs: 457 active+clean; 133 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 597 KiB/s rd, 821 KiB/s wr, 38 op/s 2024-04-07T17:13:39.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:13:38 smithi144 bash[66654]: cluster 2024-04-07T17:13:37.921859+0000 mon.a (mon.0) 3182 : cluster [DBG] osdmap e1662: 8 total, 8 up, 8 in 2024-04-07T17:13:41.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:13:40 smithi100 bash[74942]: cluster 2024-04-07T17:13:39.588244+0000 mgr.y (mgr.25175) 1749 : cluster [DBG] pgmap v2986: 457 pgs: 457 active+clean; 137 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 116 KiB/s rd, 1.5 MiB/s wr, 58 op/s 2024-04-07T17:13:41.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:13:40 smithi100 bash[76202]: cluster 2024-04-07T17:13:39.588244+0000 mgr.y (mgr.25175) 1749 : cluster [DBG] pgmap v2986: 457 pgs: 457 active+clean; 137 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 116 KiB/s rd, 1.5 MiB/s wr, 58 op/s 2024-04-07T17:13:41.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:13:40 smithi144 bash[66654]: cluster 2024-04-07T17:13:39.588244+0000 mgr.y (mgr.25175) 1749 : cluster [DBG] pgmap v2986: 457 pgs: 457 active+clean; 137 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 116 KiB/s rd, 1.5 MiB/s wr, 58 op/s 2024-04-07T17:13:42.617 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:13:42 smithi144 bash[95233]: ts=2024-04-07T17:13:42.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:13:43.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:13:42 smithi100 bash[74942]: cluster 2024-04-07T17:13:41.589255+0000 mgr.y (mgr.25175) 1750 : cluster [DBG] pgmap v2987: 457 pgs: 457 active+clean; 137 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 78 KiB/s rd, 514 KiB/s wr, 54 op/s 2024-04-07T17:13:43.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:13:42 smithi100 bash[76202]: cluster 2024-04-07T17:13:41.589255+0000 mgr.y (mgr.25175) 1750 : cluster [DBG] pgmap v2987: 457 pgs: 457 active+clean; 137 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 78 KiB/s rd, 514 KiB/s wr, 54 op/s 2024-04-07T17:13:43.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:13:42 smithi144 bash[66654]: cluster 2024-04-07T17:13:41.589255+0000 mgr.y (mgr.25175) 1750 : cluster [DBG] pgmap v2987: 457 pgs: 457 active+clean; 137 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 78 KiB/s rd, 514 KiB/s wr, 54 op/s 2024-04-07T17:13:45.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:13:44 smithi100 bash[74942]: cluster 2024-04-07T17:13:43.591045+0000 mgr.y (mgr.25175) 1751 : cluster [DBG] pgmap v2988: 457 pgs: 457 active+clean; 137 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 93 KiB/s rd, 515 KiB/s wr, 74 op/s 2024-04-07T17:13:45.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:13:44 smithi100 bash[76202]: cluster 2024-04-07T17:13:43.591045+0000 mgr.y (mgr.25175) 1751 : cluster [DBG] pgmap v2988: 457 pgs: 457 active+clean; 137 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 93 KiB/s rd, 515 KiB/s wr, 74 op/s 2024-04-07T17:13:45.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:13:44 smithi144 bash[66654]: cluster 2024-04-07T17:13:43.591045+0000 mgr.y (mgr.25175) 1751 : cluster [DBG] pgmap v2988: 457 pgs: 457 active+clean; 137 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 93 KiB/s rd, 515 KiB/s wr, 74 op/s 2024-04-07T17:13:47.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:13:46 smithi100 bash[74942]: cluster 2024-04-07T17:13:45.592381+0000 mgr.y (mgr.25175) 1752 : cluster [DBG] pgmap v2989: 457 pgs: 457 active+clean; 137 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 85 KiB/s rd, 475 KiB/s wr, 68 op/s 2024-04-07T17:13:47.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:13:46 smithi100 bash[74942]: audit 2024-04-07T17:13:46.821491+0000 mon.a (mon.0) 3183 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:13:47.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:13:46 smithi100 bash[76202]: cluster 2024-04-07T17:13:45.592381+0000 mgr.y (mgr.25175) 1752 : cluster [DBG] pgmap v2989: 457 pgs: 457 active+clean; 137 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 85 KiB/s rd, 475 KiB/s wr, 68 op/s 2024-04-07T17:13:47.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:13:46 smithi100 bash[76202]: audit 2024-04-07T17:13:46.821491+0000 mon.a (mon.0) 3183 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:13:47.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:13:46 smithi144 bash[66654]: cluster 2024-04-07T17:13:45.592381+0000 mgr.y (mgr.25175) 1752 : cluster [DBG] pgmap v2989: 457 pgs: 457 active+clean; 137 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 85 KiB/s rd, 475 KiB/s wr, 68 op/s 2024-04-07T17:13:47.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:13:46 smithi144 bash[66654]: audit 2024-04-07T17:13:46.821491+0000 mon.a (mon.0) 3183 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:13:48.703 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:13:48 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:13:48] "GET /metrics HTTP/1.1" 200 60779 "" "Prometheus/2.43.0" 2024-04-07T17:13:49.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:13:48 smithi100 bash[74942]: cluster 2024-04-07T17:13:47.593972+0000 mgr.y (mgr.25175) 1753 : cluster [DBG] pgmap v2990: 457 pgs: 457 active+clean; 137 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 40 KiB/s rd, 411 KiB/s wr, 50 op/s 2024-04-07T17:13:49.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:13:48 smithi100 bash[76202]: cluster 2024-04-07T17:13:47.593972+0000 mgr.y (mgr.25175) 1753 : cluster [DBG] pgmap v2990: 457 pgs: 457 active+clean; 137 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 40 KiB/s rd, 411 KiB/s wr, 50 op/s 2024-04-07T17:13:49.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:13:48 smithi144 bash[66654]: cluster 2024-04-07T17:13:47.593972+0000 mgr.y (mgr.25175) 1753 : cluster [DBG] pgmap v2990: 457 pgs: 457 active+clean; 137 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 40 KiB/s rd, 411 KiB/s wr, 50 op/s 2024-04-07T17:13:51.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:13:50 smithi100 bash[74942]: cluster 2024-04-07T17:13:49.595313+0000 mgr.y (mgr.25175) 1754 : cluster [DBG] pgmap v2991: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 2.2 MiB/s rd, 1.7 MiB/s wr, 51 op/s 2024-04-07T17:13:51.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:13:50 smithi100 bash[76202]: cluster 2024-04-07T17:13:49.595313+0000 mgr.y (mgr.25175) 1754 : cluster [DBG] pgmap v2991: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 2.2 MiB/s rd, 1.7 MiB/s wr, 51 op/s 2024-04-07T17:13:51.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:13:50 smithi144 bash[66654]: cluster 2024-04-07T17:13:49.595313+0000 mgr.y (mgr.25175) 1754 : cluster [DBG] pgmap v2991: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 2.2 MiB/s rd, 1.7 MiB/s wr, 51 op/s 2024-04-07T17:13:52.617 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:13:52 smithi144 bash[95233]: ts=2024-04-07T17:13:52.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:13:53.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:13:52 smithi100 bash[74942]: cluster 2024-04-07T17:13:51.596188+0000 mgr.y (mgr.25175) 1755 : cluster [DBG] pgmap v2992: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 2.1 MiB/s rd, 1.3 MiB/s wr, 33 op/s 2024-04-07T17:13:53.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:13:52 smithi100 bash[76202]: cluster 2024-04-07T17:13:51.596188+0000 mgr.y (mgr.25175) 1755 : cluster [DBG] pgmap v2992: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 2.1 MiB/s rd, 1.3 MiB/s wr, 33 op/s 2024-04-07T17:13:53.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:13:52 smithi144 bash[66654]: cluster 2024-04-07T17:13:51.596188+0000 mgr.y (mgr.25175) 1755 : cluster [DBG] pgmap v2992: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 2.1 MiB/s rd, 1.3 MiB/s wr, 33 op/s 2024-04-07T17:13:54.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:13:53 smithi100 bash[74942]: audit 2024-04-07T17:13:53.695697+0000 mon.a (mon.0) 3184 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:13:54.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:13:53 smithi100 bash[76202]: audit 2024-04-07T17:13:53.695697+0000 mon.a (mon.0) 3184 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:13:54.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:13:53 smithi144 bash[66654]: audit 2024-04-07T17:13:53.695697+0000 mon.a (mon.0) 3184 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:13:55.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:13:54 smithi100 bash[74942]: cluster 2024-04-07T17:13:53.597769+0000 mgr.y (mgr.25175) 1756 : cluster [DBG] pgmap v2993: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 2.1 MiB/s rd, 1.3 MiB/s wr, 33 op/s 2024-04-07T17:13:55.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:13:54 smithi100 bash[74942]: audit 2024-04-07T17:13:54.422936+0000 mon.a (mon.0) 3185 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:13:55.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:13:54 smithi100 bash[74942]: audit 2024-04-07T17:13:54.423704+0000 mon.a (mon.0) 3186 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:13:55.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:13:54 smithi100 bash[74942]: audit 2024-04-07T17:13:54.429530+0000 mon.a (mon.0) 3187 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:13:55.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:13:54 smithi100 bash[76202]: cluster 2024-04-07T17:13:53.597769+0000 mgr.y (mgr.25175) 1756 : cluster [DBG] pgmap v2993: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 2.1 MiB/s rd, 1.3 MiB/s wr, 33 op/s 2024-04-07T17:13:55.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:13:54 smithi100 bash[76202]: audit 2024-04-07T17:13:54.422936+0000 mon.a (mon.0) 3185 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:13:55.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:13:54 smithi100 bash[76202]: audit 2024-04-07T17:13:54.423704+0000 mon.a (mon.0) 3186 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:13:55.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:13:54 smithi100 bash[76202]: audit 2024-04-07T17:13:54.429530+0000 mon.a (mon.0) 3187 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:13:55.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:13:54 smithi144 bash[66654]: cluster 2024-04-07T17:13:53.597769+0000 mgr.y (mgr.25175) 1756 : cluster [DBG] pgmap v2993: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 2.1 MiB/s rd, 1.3 MiB/s wr, 33 op/s 2024-04-07T17:13:55.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:13:54 smithi144 bash[66654]: audit 2024-04-07T17:13:54.422936+0000 mon.a (mon.0) 3185 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:13:55.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:13:54 smithi144 bash[66654]: audit 2024-04-07T17:13:54.423704+0000 mon.a (mon.0) 3186 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:13:55.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:13:54 smithi144 bash[66654]: audit 2024-04-07T17:13:54.429530+0000 mon.a (mon.0) 3187 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:13:56.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:13:55 smithi144 bash[66654]: cluster 2024-04-07T17:13:55.599815+0000 mgr.y (mgr.25175) 1757 : cluster [DBG] pgmap v2994: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 5.8 MiB/s rd, 1.3 MiB/s wr, 15 op/s 2024-04-07T17:13:56.453 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:13:55 smithi100 bash[74942]: cluster 2024-04-07T17:13:55.599815+0000 mgr.y (mgr.25175) 1757 : cluster [DBG] pgmap v2994: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 5.8 MiB/s rd, 1.3 MiB/s wr, 15 op/s 2024-04-07T17:13:56.453 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:13:55 smithi100 bash[76202]: cluster 2024-04-07T17:13:55.599815+0000 mgr.y (mgr.25175) 1757 : cluster [DBG] pgmap v2994: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 5.8 MiB/s rd, 1.3 MiB/s wr, 15 op/s 2024-04-07T17:13:58.703 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:13:58 smithi100 bash[74942]: cluster 2024-04-07T17:13:57.601446+0000 mgr.y (mgr.25175) 1758 : cluster [DBG] pgmap v2995: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 5.8 MiB/s rd, 1.3 MiB/s wr, 15 op/s 2024-04-07T17:13:58.703 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:13:58 smithi100 bash[76202]: cluster 2024-04-07T17:13:57.601446+0000 mgr.y (mgr.25175) 1758 : cluster [DBG] pgmap v2995: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 5.8 MiB/s rd, 1.3 MiB/s wr, 15 op/s 2024-04-07T17:13:58.704 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:13:58 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:13:58] "GET /metrics HTTP/1.1" 200 60779 "" "Prometheus/2.43.0" 2024-04-07T17:13:59.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:13:58 smithi144 bash[66654]: cluster 2024-04-07T17:13:57.601446+0000 mgr.y (mgr.25175) 1758 : cluster [DBG] pgmap v2995: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 5.8 MiB/s rd, 1.3 MiB/s wr, 15 op/s 2024-04-07T17:14:00.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:14:00 smithi100 bash[74942]: cluster 2024-04-07T17:13:59.602819+0000 mgr.y (mgr.25175) 1759 : cluster [DBG] pgmap v2996: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 6.5 MiB/s rd, 1.3 MiB/s wr, 10 op/s 2024-04-07T17:14:00.953 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:14:00 smithi100 bash[76202]: cluster 2024-04-07T17:13:59.602819+0000 mgr.y (mgr.25175) 1759 : cluster [DBG] pgmap v2996: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 6.5 MiB/s rd, 1.3 MiB/s wr, 10 op/s 2024-04-07T17:14:01.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:14:00 smithi144 bash[66654]: cluster 2024-04-07T17:13:59.602819+0000 mgr.y (mgr.25175) 1759 : cluster [DBG] pgmap v2996: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 6.5 MiB/s rd, 1.3 MiB/s wr, 10 op/s 2024-04-07T17:14:02.617 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:14:02 smithi144 bash[95233]: ts=2024-04-07T17:14:02.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:14:02.953 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:14:02 smithi100 bash[76202]: cluster 2024-04-07T17:14:01.604113+0000 mgr.y (mgr.25175) 1760 : cluster [DBG] pgmap v2997: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 4.4 MiB/s rd, 2 op/s 2024-04-07T17:14:02.953 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:14:02 smithi100 bash[76202]: audit 2024-04-07T17:14:01.822540+0000 mon.a (mon.0) 3188 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:14:02.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:14:02 smithi100 bash[74942]: cluster 2024-04-07T17:14:01.604113+0000 mgr.y (mgr.25175) 1760 : cluster [DBG] pgmap v2997: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 4.4 MiB/s rd, 2 op/s 2024-04-07T17:14:02.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:14:02 smithi100 bash[74942]: audit 2024-04-07T17:14:01.822540+0000 mon.a (mon.0) 3188 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:14:03.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:14:02 smithi144 bash[66654]: cluster 2024-04-07T17:14:01.604113+0000 mgr.y (mgr.25175) 1760 : cluster [DBG] pgmap v2997: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 4.4 MiB/s rd, 2 op/s 2024-04-07T17:14:03.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:14:02 smithi144 bash[66654]: audit 2024-04-07T17:14:01.822540+0000 mon.a (mon.0) 3188 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:14:04.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:14:04 smithi100 bash[74942]: cluster 2024-04-07T17:14:03.605917+0000 mgr.y (mgr.25175) 1761 : cluster [DBG] pgmap v2998: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 4.4 MiB/s rd, 2 op/s 2024-04-07T17:14:04.953 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:14:04 smithi100 bash[76202]: cluster 2024-04-07T17:14:03.605917+0000 mgr.y (mgr.25175) 1761 : cluster [DBG] pgmap v2998: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 4.4 MiB/s rd, 2 op/s 2024-04-07T17:14:05.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:14:04 smithi144 bash[66654]: cluster 2024-04-07T17:14:03.605917+0000 mgr.y (mgr.25175) 1761 : cluster [DBG] pgmap v2998: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 4.4 MiB/s rd, 2 op/s 2024-04-07T17:14:06.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:14:06 smithi100 bash[74942]: cluster 2024-04-07T17:14:05.608015+0000 mgr.y (mgr.25175) 1762 : cluster [DBG] pgmap v2999: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 5.2 MiB/s rd, 3 op/s 2024-04-07T17:14:06.953 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:14:06 smithi100 bash[76202]: cluster 2024-04-07T17:14:05.608015+0000 mgr.y (mgr.25175) 1762 : cluster [DBG] pgmap v2999: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 5.2 MiB/s rd, 3 op/s 2024-04-07T17:14:07.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:14:06 smithi144 bash[66654]: cluster 2024-04-07T17:14:05.608015+0000 mgr.y (mgr.25175) 1762 : cluster [DBG] pgmap v2999: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 5.2 MiB/s rd, 3 op/s 2024-04-07T17:14:08.678 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:14:08 smithi100 bash[74942]: cluster 2024-04-07T17:14:07.609724+0000 mgr.y (mgr.25175) 1763 : cluster [DBG] pgmap v3000: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 1.4 MiB/s rd, 1 op/s 2024-04-07T17:14:08.678 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:14:08 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:14:08] "GET /metrics HTTP/1.1" 200 60784 "" "Prometheus/2.43.0" 2024-04-07T17:14:08.678 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:14:08 smithi100 bash[76202]: cluster 2024-04-07T17:14:07.609724+0000 mgr.y (mgr.25175) 1763 : cluster [DBG] pgmap v3000: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 1.4 MiB/s rd, 1 op/s 2024-04-07T17:14:09.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:14:08 smithi144 bash[66654]: cluster 2024-04-07T17:14:07.609724+0000 mgr.y (mgr.25175) 1763 : cluster [DBG] pgmap v3000: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 1.4 MiB/s rd, 1 op/s 2024-04-07T17:14:10.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:14:10 smithi100 bash[74942]: cluster 2024-04-07T17:14:09.611072+0000 mgr.y (mgr.25175) 1764 : cluster [DBG] pgmap v3001: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 2.2 MiB/s rd, 1 op/s 2024-04-07T17:14:10.953 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:14:10 smithi100 bash[76202]: cluster 2024-04-07T17:14:09.611072+0000 mgr.y (mgr.25175) 1764 : cluster [DBG] pgmap v3001: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 2.2 MiB/s rd, 1 op/s 2024-04-07T17:14:11.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:14:10 smithi144 bash[66654]: cluster 2024-04-07T17:14:09.611072+0000 mgr.y (mgr.25175) 1764 : cluster [DBG] pgmap v3001: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 2.2 MiB/s rd, 1 op/s 2024-04-07T17:14:12.617 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:14:12 smithi144 bash[95233]: ts=2024-04-07T17:14:12.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:14:12.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:14:12 smithi100 bash[74942]: cluster 2024-04-07T17:14:11.612023+0000 mgr.y (mgr.25175) 1765 : cluster [DBG] pgmap v3002: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 1.5 MiB/s rd, 1 op/s 2024-04-07T17:14:12.953 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:14:12 smithi100 bash[76202]: cluster 2024-04-07T17:14:11.612023+0000 mgr.y (mgr.25175) 1765 : cluster [DBG] pgmap v3002: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 1.5 MiB/s rd, 1 op/s 2024-04-07T17:14:13.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:14:12 smithi144 bash[66654]: cluster 2024-04-07T17:14:11.612023+0000 mgr.y (mgr.25175) 1765 : cluster [DBG] pgmap v3002: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 1.5 MiB/s rd, 1 op/s 2024-04-07T17:14:14.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:14:14 smithi100 bash[74942]: cluster 2024-04-07T17:14:13.613815+0000 mgr.y (mgr.25175) 1766 : cluster [DBG] pgmap v3003: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 1.5 MiB/s rd, 1 op/s 2024-04-07T17:14:14.953 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:14:14 smithi100 bash[76202]: cluster 2024-04-07T17:14:13.613815+0000 mgr.y (mgr.25175) 1766 : cluster [DBG] pgmap v3003: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 1.5 MiB/s rd, 1 op/s 2024-04-07T17:14:15.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:14:14 smithi144 bash[66654]: cluster 2024-04-07T17:14:13.613815+0000 mgr.y (mgr.25175) 1766 : cluster [DBG] pgmap v3003: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 1.5 MiB/s rd, 1 op/s 2024-04-07T17:14:16.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:14:16 smithi100 bash[74942]: cluster 2024-04-07T17:14:15.615851+0000 mgr.y (mgr.25175) 1767 : cluster [DBG] pgmap v3004: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 2.2 MiB/s rd, 1 op/s 2024-04-07T17:14:16.953 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:14:16 smithi100 bash[76202]: cluster 2024-04-07T17:14:15.615851+0000 mgr.y (mgr.25175) 1767 : cluster [DBG] pgmap v3004: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 2.2 MiB/s rd, 1 op/s 2024-04-07T17:14:17.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:14:16 smithi144 bash[66654]: cluster 2024-04-07T17:14:15.615851+0000 mgr.y (mgr.25175) 1767 : cluster [DBG] pgmap v3004: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 2.2 MiB/s rd, 1 op/s 2024-04-07T17:14:17.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:14:17 smithi100 bash[74942]: audit 2024-04-07T17:14:16.822111+0000 mon.a (mon.0) 3189 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:14:17.953 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:14:17 smithi100 bash[76202]: audit 2024-04-07T17:14:16.822111+0000 mon.a (mon.0) 3189 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:14:18.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:14:17 smithi144 bash[66654]: audit 2024-04-07T17:14:16.822111+0000 mon.a (mon.0) 3189 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:14:18.703 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:14:18 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:14:18] "GET /metrics HTTP/1.1" 200 60787 "" "Prometheus/2.43.0" 2024-04-07T17:14:19.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:14:18 smithi144 bash[66654]: cluster 2024-04-07T17:14:17.617111+0000 mgr.y (mgr.25175) 1768 : cluster [DBG] pgmap v3005: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 1.5 MiB/s rd, 1 op/s 2024-04-07T17:14:19.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:14:18 smithi100 bash[76202]: cluster 2024-04-07T17:14:17.617111+0000 mgr.y (mgr.25175) 1768 : cluster [DBG] pgmap v3005: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 1.5 MiB/s rd, 1 op/s 2024-04-07T17:14:19.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:14:18 smithi100 bash[74942]: cluster 2024-04-07T17:14:17.617111+0000 mgr.y (mgr.25175) 1768 : cluster [DBG] pgmap v3005: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 1.5 MiB/s rd, 1 op/s 2024-04-07T17:14:21.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:14:20 smithi144 bash[66654]: cluster 2024-04-07T17:14:19.619143+0000 mgr.y (mgr.25175) 1769 : cluster [DBG] pgmap v3006: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 1.9 MiB/s rd, 1 op/s 2024-04-07T17:14:21.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:14:20 smithi100 bash[76202]: cluster 2024-04-07T17:14:19.619143+0000 mgr.y (mgr.25175) 1769 : cluster [DBG] pgmap v3006: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 1.9 MiB/s rd, 1 op/s 2024-04-07T17:14:21.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:14:20 smithi100 bash[74942]: cluster 2024-04-07T17:14:19.619143+0000 mgr.y (mgr.25175) 1769 : cluster [DBG] pgmap v3006: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 1.9 MiB/s rd, 1 op/s 2024-04-07T17:14:22.617 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:14:22 smithi144 bash[95233]: ts=2024-04-07T17:14:22.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:14:23.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:14:22 smithi144 bash[66654]: cluster 2024-04-07T17:14:21.620477+0000 mgr.y (mgr.25175) 1770 : cluster [DBG] pgmap v3007: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 1.1 MiB/s rd, 0 op/s 2024-04-07T17:14:23.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:14:22 smithi100 bash[74942]: cluster 2024-04-07T17:14:21.620477+0000 mgr.y (mgr.25175) 1770 : cluster [DBG] pgmap v3007: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 1.1 MiB/s rd, 0 op/s 2024-04-07T17:14:23.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:14:22 smithi100 bash[76202]: cluster 2024-04-07T17:14:21.620477+0000 mgr.y (mgr.25175) 1770 : cluster [DBG] pgmap v3007: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 1.1 MiB/s rd, 0 op/s 2024-04-07T17:14:25.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:14:24 smithi144 bash[66654]: cluster 2024-04-07T17:14:23.622342+0000 mgr.y (mgr.25175) 1771 : cluster [DBG] pgmap v3008: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 1.1 MiB/s rd, 2 op/s 2024-04-07T17:14:25.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:14:24 smithi100 bash[74942]: cluster 2024-04-07T17:14:23.622342+0000 mgr.y (mgr.25175) 1771 : cluster [DBG] pgmap v3008: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 1.1 MiB/s rd, 2 op/s 2024-04-07T17:14:25.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:14:24 smithi100 bash[76202]: cluster 2024-04-07T17:14:23.622342+0000 mgr.y (mgr.25175) 1771 : cluster [DBG] pgmap v3008: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 1.1 MiB/s rd, 2 op/s 2024-04-07T17:14:27.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:14:26 smithi144 bash[66654]: cluster 2024-04-07T17:14:25.624357+0000 mgr.y (mgr.25175) 1772 : cluster [DBG] pgmap v3009: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 1.2 MiB/s rd, 5 op/s 2024-04-07T17:14:27.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:14:26 smithi100 bash[76202]: cluster 2024-04-07T17:14:25.624357+0000 mgr.y (mgr.25175) 1772 : cluster [DBG] pgmap v3009: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 1.2 MiB/s rd, 5 op/s 2024-04-07T17:14:27.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:14:26 smithi100 bash[74942]: cluster 2024-04-07T17:14:25.624357+0000 mgr.y (mgr.25175) 1772 : cluster [DBG] pgmap v3009: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 1.2 MiB/s rd, 5 op/s 2024-04-07T17:14:28.703 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:14:28 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:14:28] "GET /metrics HTTP/1.1" 200 60787 "" "Prometheus/2.43.0" 2024-04-07T17:14:29.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:14:28 smithi144 bash[66654]: cluster 2024-04-07T17:14:27.626893+0000 mgr.y (mgr.25175) 1773 : cluster [DBG] pgmap v3010: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 468 KiB/s rd, 8 op/s 2024-04-07T17:14:29.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:14:28 smithi100 bash[74942]: cluster 2024-04-07T17:14:27.626893+0000 mgr.y (mgr.25175) 1773 : cluster [DBG] pgmap v3010: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 468 KiB/s rd, 8 op/s 2024-04-07T17:14:29.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:14:28 smithi100 bash[76202]: cluster 2024-04-07T17:14:27.626893+0000 mgr.y (mgr.25175) 1773 : cluster [DBG] pgmap v3010: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 468 KiB/s rd, 8 op/s 2024-04-07T17:14:31.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:14:30 smithi144 bash[66654]: cluster 2024-04-07T17:14:29.628244+0000 mgr.y (mgr.25175) 1774 : cluster [DBG] pgmap v3011: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 518 KiB/s rd, 11 op/s 2024-04-07T17:14:31.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:14:30 smithi100 bash[74942]: cluster 2024-04-07T17:14:29.628244+0000 mgr.y (mgr.25175) 1774 : cluster [DBG] pgmap v3011: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 518 KiB/s rd, 11 op/s 2024-04-07T17:14:31.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:14:30 smithi100 bash[76202]: cluster 2024-04-07T17:14:29.628244+0000 mgr.y (mgr.25175) 1774 : cluster [DBG] pgmap v3011: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 518 KiB/s rd, 11 op/s 2024-04-07T17:14:32.617 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:14:32 smithi144 bash[95233]: ts=2024-04-07T17:14:32.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:14:33.118 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:14:32 smithi144 bash[66654]: cluster 2024-04-07T17:14:31.629543+0000 mgr.y (mgr.25175) 1775 : cluster [DBG] pgmap v3012: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 129 KiB/s rd, 11 op/s 2024-04-07T17:14:33.118 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:14:32 smithi144 bash[66654]: audit 2024-04-07T17:14:31.822343+0000 mon.a (mon.0) 3190 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:14:33.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:14:32 smithi100 bash[74942]: cluster 2024-04-07T17:14:31.629543+0000 mgr.y (mgr.25175) 1775 : cluster [DBG] pgmap v3012: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 129 KiB/s rd, 11 op/s 2024-04-07T17:14:33.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:14:32 smithi100 bash[74942]: audit 2024-04-07T17:14:31.822343+0000 mon.a (mon.0) 3190 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:14:33.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:14:32 smithi100 bash[76202]: cluster 2024-04-07T17:14:31.629543+0000 mgr.y (mgr.25175) 1775 : cluster [DBG] pgmap v3012: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 129 KiB/s rd, 11 op/s 2024-04-07T17:14:33.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:14:32 smithi100 bash[76202]: audit 2024-04-07T17:14:31.822343+0000 mon.a (mon.0) 3190 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:14:35.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:14:34 smithi144 bash[66654]: cluster 2024-04-07T17:14:33.632621+0000 mgr.y (mgr.25175) 1776 : cluster [DBG] pgmap v3013: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 134 KiB/s rd, 16 op/s 2024-04-07T17:14:35.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:14:34 smithi100 bash[74942]: cluster 2024-04-07T17:14:33.632621+0000 mgr.y (mgr.25175) 1776 : cluster [DBG] pgmap v3013: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 134 KiB/s rd, 16 op/s 2024-04-07T17:14:35.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:14:34 smithi100 bash[76202]: cluster 2024-04-07T17:14:33.632621+0000 mgr.y (mgr.25175) 1776 : cluster [DBG] pgmap v3013: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 134 KiB/s rd, 16 op/s 2024-04-07T17:14:37.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:14:36 smithi144 bash[66654]: cluster 2024-04-07T17:14:35.634027+0000 mgr.y (mgr.25175) 1777 : cluster [DBG] pgmap v3014: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 180 KiB/s rd, 14 op/s 2024-04-07T17:14:37.206 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:14:36 smithi100 bash[74942]: cluster 2024-04-07T17:14:35.634027+0000 mgr.y (mgr.25175) 1777 : cluster [DBG] pgmap v3014: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 180 KiB/s rd, 14 op/s 2024-04-07T17:14:37.206 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:14:36 smithi100 bash[76202]: cluster 2024-04-07T17:14:35.634027+0000 mgr.y (mgr.25175) 1777 : cluster [DBG] pgmap v3014: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 180 KiB/s rd, 14 op/s 2024-04-07T17:14:38.703 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:14:38 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:14:38] "GET /metrics HTTP/1.1" 200 60785 "" "Prometheus/2.43.0" 2024-04-07T17:14:39.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:14:38 smithi144 bash[66654]: cluster 2024-04-07T17:14:37.636529+0000 mgr.y (mgr.25175) 1778 : cluster [DBG] pgmap v3015: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 110 KiB/s rd, 85 B/s wr, 16 op/s 2024-04-07T17:14:39.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:14:38 smithi100 bash[74942]: cluster 2024-04-07T17:14:37.636529+0000 mgr.y (mgr.25175) 1778 : cluster [DBG] pgmap v3015: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 110 KiB/s rd, 85 B/s wr, 16 op/s 2024-04-07T17:14:39.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:14:38 smithi100 bash[76202]: cluster 2024-04-07T17:14:37.636529+0000 mgr.y (mgr.25175) 1778 : cluster [DBG] pgmap v3015: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 110 KiB/s rd, 85 B/s wr, 16 op/s 2024-04-07T17:14:41.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:14:40 smithi144 bash[66654]: cluster 2024-04-07T17:14:39.638656+0000 mgr.y (mgr.25175) 1779 : cluster [DBG] pgmap v3016: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 181 KiB/s rd, 85 B/s wr, 16 op/s 2024-04-07T17:14:41.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:14:40 smithi100 bash[74942]: cluster 2024-04-07T17:14:39.638656+0000 mgr.y (mgr.25175) 1779 : cluster [DBG] pgmap v3016: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 181 KiB/s rd, 85 B/s wr, 16 op/s 2024-04-07T17:14:41.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:14:40 smithi100 bash[76202]: cluster 2024-04-07T17:14:39.638656+0000 mgr.y (mgr.25175) 1779 : cluster [DBG] pgmap v3016: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 181 KiB/s rd, 85 B/s wr, 16 op/s 2024-04-07T17:14:42.617 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:14:42 smithi144 bash[95233]: ts=2024-04-07T17:14:42.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:14:43.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:14:42 smithi144 bash[66654]: cluster 2024-04-07T17:14:41.639586+0000 mgr.y (mgr.25175) 1780 : cluster [DBG] pgmap v3017: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 131 KiB/s rd, 85 B/s wr, 13 op/s 2024-04-07T17:14:43.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:14:42 smithi100 bash[76202]: cluster 2024-04-07T17:14:41.639586+0000 mgr.y (mgr.25175) 1780 : cluster [DBG] pgmap v3017: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 131 KiB/s rd, 85 B/s wr, 13 op/s 2024-04-07T17:14:43.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:14:42 smithi100 bash[74942]: cluster 2024-04-07T17:14:41.639586+0000 mgr.y (mgr.25175) 1780 : cluster [DBG] pgmap v3017: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 131 KiB/s rd, 85 B/s wr, 13 op/s 2024-04-07T17:14:45.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:14:44 smithi144 bash[66654]: cluster 2024-04-07T17:14:43.641084+0000 mgr.y (mgr.25175) 1781 : cluster [DBG] pgmap v3018: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 136 KiB/s rd, 170 B/s wr, 19 op/s 2024-04-07T17:14:45.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:14:44 smithi100 bash[74942]: cluster 2024-04-07T17:14:43.641084+0000 mgr.y (mgr.25175) 1781 : cluster [DBG] pgmap v3018: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 136 KiB/s rd, 170 B/s wr, 19 op/s 2024-04-07T17:14:45.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:14:44 smithi100 bash[76202]: cluster 2024-04-07T17:14:43.641084+0000 mgr.y (mgr.25175) 1781 : cluster [DBG] pgmap v3018: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 136 KiB/s rd, 170 B/s wr, 19 op/s 2024-04-07T17:14:47.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:14:46 smithi144 bash[66654]: cluster 2024-04-07T17:14:45.643060+0000 mgr.y (mgr.25175) 1782 : cluster [DBG] pgmap v3019: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 181 KiB/s rd, 255 B/s wr, 17 op/s 2024-04-07T17:14:47.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:14:46 smithi144 bash[66654]: audit 2024-04-07T17:14:46.822712+0000 mon.a (mon.0) 3191 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:14:47.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:14:46 smithi100 bash[74942]: cluster 2024-04-07T17:14:45.643060+0000 mgr.y (mgr.25175) 1782 : cluster [DBG] pgmap v3019: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 181 KiB/s rd, 255 B/s wr, 17 op/s 2024-04-07T17:14:47.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:14:46 smithi100 bash[74942]: audit 2024-04-07T17:14:46.822712+0000 mon.a (mon.0) 3191 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:14:47.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:14:46 smithi100 bash[76202]: cluster 2024-04-07T17:14:45.643060+0000 mgr.y (mgr.25175) 1782 : cluster [DBG] pgmap v3019: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 181 KiB/s rd, 255 B/s wr, 17 op/s 2024-04-07T17:14:47.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:14:46 smithi100 bash[76202]: audit 2024-04-07T17:14:46.822712+0000 mon.a (mon.0) 3191 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:14:48.703 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:14:48 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:14:48] "GET /metrics HTTP/1.1" 200 60790 "" "Prometheus/2.43.0" 2024-04-07T17:14:49.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:14:48 smithi144 bash[66654]: cluster 2024-04-07T17:14:47.644609+0000 mgr.y (mgr.25175) 1783 : cluster [DBG] pgmap v3020: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 138 KiB/s rd, 255 B/s wr, 21 op/s 2024-04-07T17:14:49.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:14:48 smithi100 bash[74942]: cluster 2024-04-07T17:14:47.644609+0000 mgr.y (mgr.25175) 1783 : cluster [DBG] pgmap v3020: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 138 KiB/s rd, 255 B/s wr, 21 op/s 2024-04-07T17:14:49.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:14:48 smithi100 bash[76202]: cluster 2024-04-07T17:14:47.644609+0000 mgr.y (mgr.25175) 1783 : cluster [DBG] pgmap v3020: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 138 KiB/s rd, 255 B/s wr, 21 op/s 2024-04-07T17:14:51.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:14:50 smithi144 bash[66654]: cluster 2024-04-07T17:14:49.645695+0000 mgr.y (mgr.25175) 1784 : cluster [DBG] pgmap v3021: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 208 KiB/s rd, 170 B/s wr, 19 op/s 2024-04-07T17:14:51.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:14:50 smithi100 bash[74942]: cluster 2024-04-07T17:14:49.645695+0000 mgr.y (mgr.25175) 1784 : cluster [DBG] pgmap v3021: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 208 KiB/s rd, 170 B/s wr, 19 op/s 2024-04-07T17:14:51.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:14:50 smithi100 bash[76202]: cluster 2024-04-07T17:14:49.645695+0000 mgr.y (mgr.25175) 1784 : cluster [DBG] pgmap v3021: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 208 KiB/s rd, 170 B/s wr, 19 op/s 2024-04-07T17:14:52.617 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:14:52 smithi144 bash[95233]: ts=2024-04-07T17:14:52.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:14:53.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:14:52 smithi144 bash[66654]: cluster 2024-04-07T17:14:51.647012+0000 mgr.y (mgr.25175) 1785 : cluster [DBG] pgmap v3022: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 134 KiB/s rd, 170 B/s wr, 16 op/s 2024-04-07T17:14:53.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:14:52 smithi100 bash[74942]: cluster 2024-04-07T17:14:51.647012+0000 mgr.y (mgr.25175) 1785 : cluster [DBG] pgmap v3022: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 134 KiB/s rd, 170 B/s wr, 16 op/s 2024-04-07T17:14:53.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:14:52 smithi100 bash[76202]: cluster 2024-04-07T17:14:51.647012+0000 mgr.y (mgr.25175) 1785 : cluster [DBG] pgmap v3022: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 134 KiB/s rd, 170 B/s wr, 16 op/s 2024-04-07T17:14:55.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:14:54 smithi144 bash[66654]: cluster 2024-04-07T17:14:53.648844+0000 mgr.y (mgr.25175) 1786 : cluster [DBG] pgmap v3023: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 134 KiB/s rd, 170 B/s wr, 16 op/s 2024-04-07T17:14:55.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:14:54 smithi144 bash[66654]: audit 2024-04-07T17:14:54.509283+0000 mon.a (mon.0) 3192 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:14:55.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:14:54 smithi100 bash[76202]: cluster 2024-04-07T17:14:53.648844+0000 mgr.y (mgr.25175) 1786 : cluster [DBG] pgmap v3023: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 134 KiB/s rd, 170 B/s wr, 16 op/s 2024-04-07T17:14:55.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:14:54 smithi100 bash[76202]: audit 2024-04-07T17:14:54.509283+0000 mon.a (mon.0) 3192 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:14:55.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:14:54 smithi100 bash[74942]: cluster 2024-04-07T17:14:53.648844+0000 mgr.y (mgr.25175) 1786 : cluster [DBG] pgmap v3023: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 134 KiB/s rd, 170 B/s wr, 16 op/s 2024-04-07T17:14:55.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:14:54 smithi100 bash[74942]: audit 2024-04-07T17:14:54.509283+0000 mon.a (mon.0) 3192 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:14:56.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:14:56 smithi144 bash[66654]: audit 2024-04-07T17:14:55.034299+0000 mon.a (mon.0) 3193 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:14:56.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:14:56 smithi144 bash[66654]: audit 2024-04-07T17:14:55.039830+0000 mon.a (mon.0) 3194 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:14:56.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:14:56 smithi144 bash[66654]: audit 2024-04-07T17:14:55.618814+0000 mon.a (mon.0) 3195 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:14:56.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:14:56 smithi144 bash[66654]: audit 2024-04-07T17:14:55.620044+0000 mon.a (mon.0) 3196 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:14:56.368 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:14:56 smithi144 bash[66654]: audit 2024-04-07T17:14:55.626950+0000 mon.a (mon.0) 3197 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:14:56.368 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:14:56 smithi144 bash[66654]: cluster 2024-04-07T17:14:55.650037+0000 mgr.y (mgr.25175) 1787 : cluster [DBG] pgmap v3024: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 179 KiB/s rd, 85 B/s wr, 14 op/s 2024-04-07T17:14:56.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:14:56 smithi100 bash[74942]: audit 2024-04-07T17:14:55.034299+0000 mon.a (mon.0) 3193 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:14:56.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:14:56 smithi100 bash[74942]: audit 2024-04-07T17:14:55.039830+0000 mon.a (mon.0) 3194 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:14:56.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:14:56 smithi100 bash[74942]: audit 2024-04-07T17:14:55.618814+0000 mon.a (mon.0) 3195 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:14:56.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:14:56 smithi100 bash[74942]: audit 2024-04-07T17:14:55.620044+0000 mon.a (mon.0) 3196 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:14:56.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:14:56 smithi100 bash[74942]: audit 2024-04-07T17:14:55.626950+0000 mon.a (mon.0) 3197 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:14:56.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:14:56 smithi100 bash[74942]: cluster 2024-04-07T17:14:55.650037+0000 mgr.y (mgr.25175) 1787 : cluster [DBG] pgmap v3024: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 179 KiB/s rd, 85 B/s wr, 14 op/s 2024-04-07T17:14:56.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:14:56 smithi100 bash[76202]: audit 2024-04-07T17:14:55.034299+0000 mon.a (mon.0) 3193 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:14:56.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:14:56 smithi100 bash[76202]: audit 2024-04-07T17:14:55.039830+0000 mon.a (mon.0) 3194 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:14:56.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:14:56 smithi100 bash[76202]: audit 2024-04-07T17:14:55.618814+0000 mon.a (mon.0) 3195 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:14:56.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:14:56 smithi100 bash[76202]: audit 2024-04-07T17:14:55.620044+0000 mon.a (mon.0) 3196 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:14:56.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:14:56 smithi100 bash[76202]: audit 2024-04-07T17:14:55.626950+0000 mon.a (mon.0) 3197 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:14:56.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:14:56 smithi100 bash[76202]: cluster 2024-04-07T17:14:55.650037+0000 mgr.y (mgr.25175) 1787 : cluster [DBG] pgmap v3024: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 179 KiB/s rd, 85 B/s wr, 14 op/s 2024-04-07T17:14:58.703 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:14:58 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:14:58] "GET /metrics HTTP/1.1" 200 60790 "" "Prometheus/2.43.0" 2024-04-07T17:14:59.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:14:58 smithi144 bash[66654]: cluster 2024-04-07T17:14:57.651355+0000 mgr.y (mgr.25175) 1788 : cluster [DBG] pgmap v3025: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 134 KiB/s rd, 16 op/s 2024-04-07T17:14:59.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:14:58 smithi100 bash[74942]: cluster 2024-04-07T17:14:57.651355+0000 mgr.y (mgr.25175) 1788 : cluster [DBG] pgmap v3025: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 134 KiB/s rd, 16 op/s 2024-04-07T17:14:59.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:14:58 smithi100 bash[76202]: cluster 2024-04-07T17:14:57.651355+0000 mgr.y (mgr.25175) 1788 : cluster [DBG] pgmap v3025: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 134 KiB/s rd, 16 op/s 2024-04-07T17:15:01.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:15:00 smithi144 bash[66654]: cluster 2024-04-07T17:14:59.652618+0000 mgr.y (mgr.25175) 1789 : cluster [DBG] pgmap v3026: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 203 KiB/s rd, 14 op/s 2024-04-07T17:15:01.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:15:00 smithi100 bash[74942]: cluster 2024-04-07T17:14:59.652618+0000 mgr.y (mgr.25175) 1789 : cluster [DBG] pgmap v3026: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 203 KiB/s rd, 14 op/s 2024-04-07T17:15:01.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:15:00 smithi100 bash[76202]: cluster 2024-04-07T17:14:59.652618+0000 mgr.y (mgr.25175) 1789 : cluster [DBG] pgmap v3026: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 203 KiB/s rd, 14 op/s 2024-04-07T17:15:02.617 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:15:02 smithi144 bash[95233]: ts=2024-04-07T17:15:02.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:15:03.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:15:02 smithi144 bash[66654]: cluster 2024-04-07T17:15:01.653970+0000 mgr.y (mgr.25175) 1790 : cluster [DBG] pgmap v3027: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 129 KiB/s rd, 11 op/s 2024-04-07T17:15:03.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:15:02 smithi144 bash[66654]: audit 2024-04-07T17:15:01.823867+0000 mon.a (mon.0) 3198 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:15:03.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:15:02 smithi100 bash[76202]: cluster 2024-04-07T17:15:01.653970+0000 mgr.y (mgr.25175) 1790 : cluster [DBG] pgmap v3027: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 129 KiB/s rd, 11 op/s 2024-04-07T17:15:03.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:15:02 smithi100 bash[76202]: audit 2024-04-07T17:15:01.823867+0000 mon.a (mon.0) 3198 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:15:03.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:15:02 smithi100 bash[74942]: cluster 2024-04-07T17:15:01.653970+0000 mgr.y (mgr.25175) 1790 : cluster [DBG] pgmap v3027: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 129 KiB/s rd, 11 op/s 2024-04-07T17:15:03.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:15:02 smithi100 bash[74942]: audit 2024-04-07T17:15:01.823867+0000 mon.a (mon.0) 3198 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:15:05.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:15:04 smithi144 bash[66654]: cluster 2024-04-07T17:15:03.656900+0000 mgr.y (mgr.25175) 1791 : cluster [DBG] pgmap v3028: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 134 KiB/s rd, 16 op/s 2024-04-07T17:15:05.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:15:04 smithi100 bash[74942]: cluster 2024-04-07T17:15:03.656900+0000 mgr.y (mgr.25175) 1791 : cluster [DBG] pgmap v3028: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 134 KiB/s rd, 16 op/s 2024-04-07T17:15:05.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:15:04 smithi100 bash[76202]: cluster 2024-04-07T17:15:03.656900+0000 mgr.y (mgr.25175) 1791 : cluster [DBG] pgmap v3028: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 134 KiB/s rd, 16 op/s 2024-04-07T17:15:07.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:15:06 smithi144 bash[66654]: cluster 2024-04-07T17:15:05.658450+0000 mgr.y (mgr.25175) 1792 : cluster [DBG] pgmap v3029: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 181 KiB/s rd, 16 op/s 2024-04-07T17:15:07.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:15:06 smithi100 bash[74942]: cluster 2024-04-07T17:15:05.658450+0000 mgr.y (mgr.25175) 1792 : cluster [DBG] pgmap v3029: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 181 KiB/s rd, 16 op/s 2024-04-07T17:15:07.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:15:06 smithi100 bash[76202]: cluster 2024-04-07T17:15:05.658450+0000 mgr.y (mgr.25175) 1792 : cluster [DBG] pgmap v3029: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 181 KiB/s rd, 16 op/s 2024-04-07T17:15:08.703 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:15:08 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:15:08] "GET /metrics HTTP/1.1" 200 60789 "" "Prometheus/2.43.0" 2024-04-07T17:15:09.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:15:08 smithi144 bash[66654]: cluster 2024-04-07T17:15:07.660998+0000 mgr.y (mgr.25175) 1793 : cluster [DBG] pgmap v3030: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 136 KiB/s rd, 18 op/s 2024-04-07T17:15:09.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:15:08 smithi100 bash[74942]: cluster 2024-04-07T17:15:07.660998+0000 mgr.y (mgr.25175) 1793 : cluster [DBG] pgmap v3030: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 136 KiB/s rd, 18 op/s 2024-04-07T17:15:09.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:15:08 smithi100 bash[76202]: cluster 2024-04-07T17:15:07.660998+0000 mgr.y (mgr.25175) 1793 : cluster [DBG] pgmap v3030: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 136 KiB/s rd, 18 op/s 2024-04-07T17:15:11.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:15:10 smithi144 bash[66654]: cluster 2024-04-07T17:15:09.663586+0000 mgr.y (mgr.25175) 1794 : cluster [DBG] pgmap v3031: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 205 KiB/s rd, 16 op/s 2024-04-07T17:15:11.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:15:10 smithi100 bash[74942]: cluster 2024-04-07T17:15:09.663586+0000 mgr.y (mgr.25175) 1794 : cluster [DBG] pgmap v3031: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 205 KiB/s rd, 16 op/s 2024-04-07T17:15:11.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:15:10 smithi100 bash[76202]: cluster 2024-04-07T17:15:09.663586+0000 mgr.y (mgr.25175) 1794 : cluster [DBG] pgmap v3031: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 205 KiB/s rd, 16 op/s 2024-04-07T17:15:12.617 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:15:12 smithi144 bash[95233]: ts=2024-04-07T17:15:12.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:15:13.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:15:12 smithi144 bash[66654]: cluster 2024-04-07T17:15:11.664550+0000 mgr.y (mgr.25175) 1795 : cluster [DBG] pgmap v3032: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 131 KiB/s rd, 13 op/s 2024-04-07T17:15:13.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:15:12 smithi100 bash[74942]: cluster 2024-04-07T17:15:11.664550+0000 mgr.y (mgr.25175) 1795 : cluster [DBG] pgmap v3032: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 131 KiB/s rd, 13 op/s 2024-04-07T17:15:13.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:15:12 smithi100 bash[76202]: cluster 2024-04-07T17:15:11.664550+0000 mgr.y (mgr.25175) 1795 : cluster [DBG] pgmap v3032: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 131 KiB/s rd, 13 op/s 2024-04-07T17:15:15.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:15:14 smithi144 bash[66654]: cluster 2024-04-07T17:15:13.667533+0000 mgr.y (mgr.25175) 1796 : cluster [DBG] pgmap v3033: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 136 KiB/s rd, 19 op/s 2024-04-07T17:15:15.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:15:14 smithi100 bash[74942]: cluster 2024-04-07T17:15:13.667533+0000 mgr.y (mgr.25175) 1796 : cluster [DBG] pgmap v3033: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 136 KiB/s rd, 19 op/s 2024-04-07T17:15:15.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:15:14 smithi100 bash[76202]: cluster 2024-04-07T17:15:13.667533+0000 mgr.y (mgr.25175) 1796 : cluster [DBG] pgmap v3033: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 136 KiB/s rd, 19 op/s 2024-04-07T17:15:17.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:15:16 smithi144 bash[66654]: cluster 2024-04-07T17:15:15.668717+0000 mgr.y (mgr.25175) 1797 : cluster [DBG] pgmap v3034: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 181 KiB/s rd, 16 op/s 2024-04-07T17:15:17.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:15:16 smithi100 bash[74942]: cluster 2024-04-07T17:15:15.668717+0000 mgr.y (mgr.25175) 1797 : cluster [DBG] pgmap v3034: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 181 KiB/s rd, 16 op/s 2024-04-07T17:15:17.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:15:16 smithi100 bash[76202]: cluster 2024-04-07T17:15:15.668717+0000 mgr.y (mgr.25175) 1797 : cluster [DBG] pgmap v3034: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 181 KiB/s rd, 16 op/s 2024-04-07T17:15:18.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:15:17 smithi144 bash[66654]: audit 2024-04-07T17:15:16.823433+0000 mon.a (mon.0) 3199 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:15:18.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:15:17 smithi100 bash[76202]: audit 2024-04-07T17:15:16.823433+0000 mon.a (mon.0) 3199 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:15:18.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:15:17 smithi100 bash[74942]: audit 2024-04-07T17:15:16.823433+0000 mon.a (mon.0) 3199 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:15:18.703 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:15:18 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:15:18] "GET /metrics HTTP/1.1" 200 60790 "" "Prometheus/2.43.0" 2024-04-07T17:15:19.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:15:18 smithi144 bash[66654]: cluster 2024-04-07T17:15:17.670356+0000 mgr.y (mgr.25175) 1798 : cluster [DBG] pgmap v3035: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 138 KiB/s rd, 21 op/s 2024-04-07T17:15:19.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:15:18 smithi100 bash[74942]: cluster 2024-04-07T17:15:17.670356+0000 mgr.y (mgr.25175) 1798 : cluster [DBG] pgmap v3035: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 138 KiB/s rd, 21 op/s 2024-04-07T17:15:19.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:15:18 smithi100 bash[76202]: cluster 2024-04-07T17:15:17.670356+0000 mgr.y (mgr.25175) 1798 : cluster [DBG] pgmap v3035: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 138 KiB/s rd, 21 op/s 2024-04-07T17:15:21.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:15:20 smithi144 bash[66654]: cluster 2024-04-07T17:15:19.672437+0000 mgr.y (mgr.25175) 1799 : cluster [DBG] pgmap v3036: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 208 KiB/s rd, 19 op/s 2024-04-07T17:15:21.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:15:20 smithi100 bash[76202]: cluster 2024-04-07T17:15:19.672437+0000 mgr.y (mgr.25175) 1799 : cluster [DBG] pgmap v3036: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 208 KiB/s rd, 19 op/s 2024-04-07T17:15:21.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:15:20 smithi100 bash[74942]: cluster 2024-04-07T17:15:19.672437+0000 mgr.y (mgr.25175) 1799 : cluster [DBG] pgmap v3036: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 208 KiB/s rd, 19 op/s 2024-04-07T17:15:22.617 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:15:22 smithi144 bash[95233]: ts=2024-04-07T17:15:22.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:15:23.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:15:22 smithi144 bash[66654]: cluster 2024-04-07T17:15:21.673745+0000 mgr.y (mgr.25175) 1800 : cluster [DBG] pgmap v3037: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 134 KiB/s rd, 16 op/s 2024-04-07T17:15:23.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:15:22 smithi100 bash[74942]: cluster 2024-04-07T17:15:21.673745+0000 mgr.y (mgr.25175) 1800 : cluster [DBG] pgmap v3037: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 134 KiB/s rd, 16 op/s 2024-04-07T17:15:23.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:15:22 smithi100 bash[76202]: cluster 2024-04-07T17:15:21.673745+0000 mgr.y (mgr.25175) 1800 : cluster [DBG] pgmap v3037: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 134 KiB/s rd, 16 op/s 2024-04-07T17:15:25.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:15:24 smithi144 bash[66654]: cluster 2024-04-07T17:15:23.675636+0000 mgr.y (mgr.25175) 1801 : cluster [DBG] pgmap v3038: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 134 KiB/s rd, 16 op/s 2024-04-07T17:15:25.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:15:24 smithi100 bash[74942]: cluster 2024-04-07T17:15:23.675636+0000 mgr.y (mgr.25175) 1801 : cluster [DBG] pgmap v3038: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 134 KiB/s rd, 16 op/s 2024-04-07T17:15:25.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:15:24 smithi100 bash[76202]: cluster 2024-04-07T17:15:23.675636+0000 mgr.y (mgr.25175) 1801 : cluster [DBG] pgmap v3038: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 134 KiB/s rd, 16 op/s 2024-04-07T17:15:27.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:15:26 smithi144 bash[66654]: cluster 2024-04-07T17:15:25.676805+0000 mgr.y (mgr.25175) 1802 : cluster [DBG] pgmap v3039: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 179 KiB/s rd, 14 op/s 2024-04-07T17:15:27.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:15:26 smithi100 bash[74942]: cluster 2024-04-07T17:15:25.676805+0000 mgr.y (mgr.25175) 1802 : cluster [DBG] pgmap v3039: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 179 KiB/s rd, 14 op/s 2024-04-07T17:15:27.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:15:26 smithi100 bash[76202]: cluster 2024-04-07T17:15:25.676805+0000 mgr.y (mgr.25175) 1802 : cluster [DBG] pgmap v3039: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 179 KiB/s rd, 14 op/s 2024-04-07T17:15:28.703 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:15:28 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:15:28] "GET /metrics HTTP/1.1" 200 60790 "" "Prometheus/2.43.0" 2024-04-07T17:15:29.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:15:28 smithi144 bash[66654]: cluster 2024-04-07T17:15:27.679259+0000 mgr.y (mgr.25175) 1803 : cluster [DBG] pgmap v3040: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 134 KiB/s rd, 16 op/s 2024-04-07T17:15:29.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:15:28 smithi100 bash[74942]: cluster 2024-04-07T17:15:27.679259+0000 mgr.y (mgr.25175) 1803 : cluster [DBG] pgmap v3040: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 134 KiB/s rd, 16 op/s 2024-04-07T17:15:29.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:15:28 smithi100 bash[76202]: cluster 2024-04-07T17:15:27.679259+0000 mgr.y (mgr.25175) 1803 : cluster [DBG] pgmap v3040: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 134 KiB/s rd, 16 op/s 2024-04-07T17:15:31.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:15:30 smithi144 bash[66654]: cluster 2024-04-07T17:15:29.680635+0000 mgr.y (mgr.25175) 1804 : cluster [DBG] pgmap v3041: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 203 KiB/s rd, 14 op/s 2024-04-07T17:15:31.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:15:30 smithi100 bash[74942]: cluster 2024-04-07T17:15:29.680635+0000 mgr.y (mgr.25175) 1804 : cluster [DBG] pgmap v3041: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 203 KiB/s rd, 14 op/s 2024-04-07T17:15:31.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:15:30 smithi100 bash[76202]: cluster 2024-04-07T17:15:29.680635+0000 mgr.y (mgr.25175) 1804 : cluster [DBG] pgmap v3041: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 203 KiB/s rd, 14 op/s 2024-04-07T17:15:32.617 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:15:32 smithi144 bash[95233]: ts=2024-04-07T17:15:32.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:15:33.119 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:15:32 smithi144 bash[66654]: cluster 2024-04-07T17:15:31.681951+0000 mgr.y (mgr.25175) 1805 : cluster [DBG] pgmap v3042: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 129 KiB/s rd, 11 op/s 2024-04-07T17:15:33.119 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:15:32 smithi144 bash[66654]: audit 2024-04-07T17:15:31.824570+0000 mon.a (mon.0) 3200 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:15:33.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:15:32 smithi100 bash[76202]: cluster 2024-04-07T17:15:31.681951+0000 mgr.y (mgr.25175) 1805 : cluster [DBG] pgmap v3042: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 129 KiB/s rd, 11 op/s 2024-04-07T17:15:33.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:15:32 smithi100 bash[76202]: audit 2024-04-07T17:15:31.824570+0000 mon.a (mon.0) 3200 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:15:33.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:15:32 smithi100 bash[74942]: cluster 2024-04-07T17:15:31.681951+0000 mgr.y (mgr.25175) 1805 : cluster [DBG] pgmap v3042: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 129 KiB/s rd, 11 op/s 2024-04-07T17:15:33.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:15:32 smithi100 bash[74942]: audit 2024-04-07T17:15:31.824570+0000 mon.a (mon.0) 3200 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:15:35.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:15:34 smithi144 bash[66654]: cluster 2024-04-07T17:15:33.684947+0000 mgr.y (mgr.25175) 1806 : cluster [DBG] pgmap v3043: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 134 KiB/s rd, 16 op/s 2024-04-07T17:15:35.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:15:34 smithi100 bash[74942]: cluster 2024-04-07T17:15:33.684947+0000 mgr.y (mgr.25175) 1806 : cluster [DBG] pgmap v3043: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 134 KiB/s rd, 16 op/s 2024-04-07T17:15:35.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:15:34 smithi100 bash[76202]: cluster 2024-04-07T17:15:33.684947+0000 mgr.y (mgr.25175) 1806 : cluster [DBG] pgmap v3043: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 134 KiB/s rd, 16 op/s 2024-04-07T17:15:37.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:15:36 smithi144 bash[66654]: cluster 2024-04-07T17:15:35.686333+0000 mgr.y (mgr.25175) 1807 : cluster [DBG] pgmap v3044: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 181 KiB/s rd, 16 op/s 2024-04-07T17:15:37.205 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:15:36 smithi100 bash[74942]: cluster 2024-04-07T17:15:35.686333+0000 mgr.y (mgr.25175) 1807 : cluster [DBG] pgmap v3044: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 181 KiB/s rd, 16 op/s 2024-04-07T17:15:37.205 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:15:36 smithi100 bash[76202]: cluster 2024-04-07T17:15:35.686333+0000 mgr.y (mgr.25175) 1807 : cluster [DBG] pgmap v3044: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 181 KiB/s rd, 16 op/s 2024-04-07T17:15:38.704 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:15:38 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:15:38] "GET /metrics HTTP/1.1" 200 60787 "" "Prometheus/2.43.0" 2024-04-07T17:15:39.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:15:38 smithi144 bash[66654]: cluster 2024-04-07T17:15:37.688885+0000 mgr.y (mgr.25175) 1808 : cluster [DBG] pgmap v3045: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 136 KiB/s rd, 85 B/s wr, 18 op/s 2024-04-07T17:15:39.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:15:38 smithi100 bash[74942]: cluster 2024-04-07T17:15:37.688885+0000 mgr.y (mgr.25175) 1808 : cluster [DBG] pgmap v3045: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 136 KiB/s rd, 85 B/s wr, 18 op/s 2024-04-07T17:15:39.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:15:38 smithi100 bash[76202]: cluster 2024-04-07T17:15:37.688885+0000 mgr.y (mgr.25175) 1808 : cluster [DBG] pgmap v3045: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 136 KiB/s rd, 85 B/s wr, 18 op/s 2024-04-07T17:15:40.739 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.EncryptionLoadFormatMismatch (127729 ms) 2024-04-07T17:15:40.739 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.EncryptedResize 2024-04-07T17:15:40.740 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:15:41.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:15:40 smithi144 bash[66654]: cluster 2024-04-07T17:15:39.690970+0000 mgr.y (mgr.25175) 1809 : cluster [DBG] pgmap v3046: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 205 KiB/s rd, 85 B/s wr, 16 op/s 2024-04-07T17:15:41.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:15:40 smithi100 bash[74942]: cluster 2024-04-07T17:15:39.690970+0000 mgr.y (mgr.25175) 1809 : cluster [DBG] pgmap v3046: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 205 KiB/s rd, 85 B/s wr, 16 op/s 2024-04-07T17:15:41.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:15:40 smithi100 bash[76202]: cluster 2024-04-07T17:15:39.690970+0000 mgr.y (mgr.25175) 1809 : cluster [DBG] pgmap v3046: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 205 KiB/s rd, 85 B/s wr, 16 op/s 2024-04-07T17:15:42.617 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:15:42 smithi144 bash[95233]: ts=2024-04-07T17:15:42.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:15:43.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:15:42 smithi144 bash[66654]: cluster 2024-04-07T17:15:41.692367+0000 mgr.y (mgr.25175) 1810 : cluster [DBG] pgmap v3047: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 131 KiB/s rd, 85 B/s wr, 13 op/s 2024-04-07T17:15:43.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:15:42 smithi100 bash[74942]: cluster 2024-04-07T17:15:41.692367+0000 mgr.y (mgr.25175) 1810 : cluster [DBG] pgmap v3047: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 131 KiB/s rd, 85 B/s wr, 13 op/s 2024-04-07T17:15:43.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:15:42 smithi100 bash[76202]: cluster 2024-04-07T17:15:41.692367+0000 mgr.y (mgr.25175) 1810 : cluster [DBG] pgmap v3047: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 131 KiB/s rd, 85 B/s wr, 13 op/s 2024-04-07T17:15:45.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:15:44 smithi144 bash[66654]: cluster 2024-04-07T17:15:43.694338+0000 mgr.y (mgr.25175) 1811 : cluster [DBG] pgmap v3048: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 131 KiB/s rd, 170 B/s wr, 14 op/s 2024-04-07T17:15:45.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:15:44 smithi100 bash[74942]: cluster 2024-04-07T17:15:43.694338+0000 mgr.y (mgr.25175) 1811 : cluster [DBG] pgmap v3048: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 131 KiB/s rd, 170 B/s wr, 14 op/s 2024-04-07T17:15:45.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:15:44 smithi100 bash[76202]: cluster 2024-04-07T17:15:43.694338+0000 mgr.y (mgr.25175) 1811 : cluster [DBG] pgmap v3048: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 131 KiB/s rd, 170 B/s wr, 14 op/s 2024-04-07T17:15:47.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:15:46 smithi144 bash[66654]: cluster 2024-04-07T17:15:45.695716+0000 mgr.y (mgr.25175) 1812 : cluster [DBG] pgmap v3049: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 129 KiB/s rd, 426 B/s wr, 11 op/s 2024-04-07T17:15:47.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:15:46 smithi100 bash[74942]: cluster 2024-04-07T17:15:45.695716+0000 mgr.y (mgr.25175) 1812 : cluster [DBG] pgmap v3049: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 129 KiB/s rd, 426 B/s wr, 11 op/s 2024-04-07T17:15:47.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:15:46 smithi100 bash[76202]: cluster 2024-04-07T17:15:45.695716+0000 mgr.y (mgr.25175) 1812 : cluster [DBG] pgmap v3049: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 129 KiB/s rd, 426 B/s wr, 11 op/s 2024-04-07T17:15:48.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:15:47 smithi144 bash[66654]: audit 2024-04-07T17:15:46.824162+0000 mon.a (mon.0) 3201 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:15:48.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:15:47 smithi100 bash[74942]: audit 2024-04-07T17:15:46.824162+0000 mon.a (mon.0) 3201 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:15:48.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:15:47 smithi100 bash[76202]: audit 2024-04-07T17:15:46.824162+0000 mon.a (mon.0) 3201 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:15:48.703 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:15:48 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:15:48] "GET /metrics HTTP/1.1" 200 60780 "" "Prometheus/2.43.0" 2024-04-07T17:15:49.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:15:48 smithi144 bash[66654]: cluster 2024-04-07T17:15:47.697363+0000 mgr.y (mgr.25175) 1813 : cluster [DBG] pgmap v3050: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 81 KiB/s rd, 426 B/s wr, 11 op/s 2024-04-07T17:15:49.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:15:48 smithi100 bash[74942]: cluster 2024-04-07T17:15:47.697363+0000 mgr.y (mgr.25175) 1813 : cluster [DBG] pgmap v3050: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 81 KiB/s rd, 426 B/s wr, 11 op/s 2024-04-07T17:15:49.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:15:48 smithi100 bash[76202]: cluster 2024-04-07T17:15:47.697363+0000 mgr.y (mgr.25175) 1813 : cluster [DBG] pgmap v3050: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 81 KiB/s rd, 426 B/s wr, 11 op/s 2024-04-07T17:15:51.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:15:50 smithi144 bash[66654]: cluster 2024-04-07T17:15:49.698688+0000 mgr.y (mgr.25175) 1814 : cluster [DBG] pgmap v3051: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 77 KiB/s rd, 341 B/s wr, 6 op/s 2024-04-07T17:15:51.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:15:50 smithi100 bash[74942]: cluster 2024-04-07T17:15:49.698688+0000 mgr.y (mgr.25175) 1814 : cluster [DBG] pgmap v3051: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 77 KiB/s rd, 341 B/s wr, 6 op/s 2024-04-07T17:15:51.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:15:50 smithi100 bash[76202]: cluster 2024-04-07T17:15:49.698688+0000 mgr.y (mgr.25175) 1814 : cluster [DBG] pgmap v3051: 457 pgs: 457 active+clean; 153 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 77 KiB/s rd, 341 B/s wr, 6 op/s 2024-04-07T17:15:52.617 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:15:52 smithi144 bash[95233]: ts=2024-04-07T17:15:52.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:15:53.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:15:52 smithi144 bash[66654]: cluster 2024-04-07T17:15:51.699654+0000 mgr.y (mgr.25175) 1815 : cluster [DBG] pgmap v3052: 457 pgs: 457 active+clean; 157 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 2.3 KiB/s rd, 341 KiB/s wr, 3 op/s 2024-04-07T17:15:53.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:15:52 smithi100 bash[74942]: cluster 2024-04-07T17:15:51.699654+0000 mgr.y (mgr.25175) 1815 : cluster [DBG] pgmap v3052: 457 pgs: 457 active+clean; 157 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 2.3 KiB/s rd, 341 KiB/s wr, 3 op/s 2024-04-07T17:15:53.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:15:52 smithi100 bash[76202]: cluster 2024-04-07T17:15:51.699654+0000 mgr.y (mgr.25175) 1815 : cluster [DBG] pgmap v3052: 457 pgs: 457 active+clean; 157 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 2.3 KiB/s rd, 341 KiB/s wr, 3 op/s 2024-04-07T17:15:55.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:15:54 smithi144 bash[66654]: cluster 2024-04-07T17:15:53.701464+0000 mgr.y (mgr.25175) 1816 : cluster [DBG] pgmap v3053: 457 pgs: 457 active+clean; 161 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 5.4 KiB/s rd, 598 KiB/s wr, 8 op/s 2024-04-07T17:15:55.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:15:54 smithi100 bash[74942]: cluster 2024-04-07T17:15:53.701464+0000 mgr.y (mgr.25175) 1816 : cluster [DBG] pgmap v3053: 457 pgs: 457 active+clean; 161 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 5.4 KiB/s rd, 598 KiB/s wr, 8 op/s 2024-04-07T17:15:55.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:15:54 smithi100 bash[76202]: cluster 2024-04-07T17:15:53.701464+0000 mgr.y (mgr.25175) 1816 : cluster [DBG] pgmap v3053: 457 pgs: 457 active+clean; 161 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 5.4 KiB/s rd, 598 KiB/s wr, 8 op/s 2024-04-07T17:15:56.103 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.EncryptedResize (15364 ms) 2024-04-07T17:15:56.103 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.EncryptedFlattenSmallData 2024-04-07T17:15:56.103 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:15:56.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:15:55 smithi144 bash[66654]: audit 2024-04-07T17:15:55.690980+0000 mon.a (mon.0) 3202 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:15:56.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:15:55 smithi100 bash[76202]: audit 2024-04-07T17:15:55.690980+0000 mon.a (mon.0) 3202 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:15:56.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:15:55 smithi100 bash[74942]: audit 2024-04-07T17:15:55.690980+0000 mon.a (mon.0) 3202 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:15:57.118 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:15:56 smithi144 bash[66654]: cluster 2024-04-07T17:15:55.702776+0000 mgr.y (mgr.25175) 1817 : cluster [DBG] pgmap v3054: 457 pgs: 457 active+clean; 173 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 84 KiB/s rd, 1.6 MiB/s wr, 17 op/s 2024-04-07T17:15:57.118 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:15:56 smithi144 bash[66654]: audit 2024-04-07T17:15:56.213176+0000 mon.a (mon.0) 3203 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:15:57.118 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:15:56 smithi144 bash[66654]: audit 2024-04-07T17:15:56.219658+0000 mon.a (mon.0) 3204 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:15:57.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:15:56 smithi100 bash[76202]: cluster 2024-04-07T17:15:55.702776+0000 mgr.y (mgr.25175) 1817 : cluster [DBG] pgmap v3054: 457 pgs: 457 active+clean; 173 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 84 KiB/s rd, 1.6 MiB/s wr, 17 op/s 2024-04-07T17:15:57.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:15:56 smithi100 bash[76202]: audit 2024-04-07T17:15:56.213176+0000 mon.a (mon.0) 3203 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:15:57.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:15:56 smithi100 bash[76202]: audit 2024-04-07T17:15:56.219658+0000 mon.a (mon.0) 3204 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:15:57.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:15:56 smithi100 bash[74942]: cluster 2024-04-07T17:15:55.702776+0000 mgr.y (mgr.25175) 1817 : cluster [DBG] pgmap v3054: 457 pgs: 457 active+clean; 173 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 84 KiB/s rd, 1.6 MiB/s wr, 17 op/s 2024-04-07T17:15:57.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:15:56 smithi100 bash[74942]: audit 2024-04-07T17:15:56.213176+0000 mon.a (mon.0) 3203 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:15:57.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:15:56 smithi100 bash[74942]: audit 2024-04-07T17:15:56.219658+0000 mon.a (mon.0) 3204 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:15:58.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:15:57 smithi144 bash[66654]: audit 2024-04-07T17:15:56.905802+0000 mon.a (mon.0) 3205 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:15:58.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:15:57 smithi144 bash[66654]: audit 2024-04-07T17:15:56.907187+0000 mon.a (mon.0) 3206 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:15:58.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:15:57 smithi144 bash[66654]: audit 2024-04-07T17:15:56.916595+0000 mon.a (mon.0) 3207 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:15:58.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:15:57 smithi100 bash[74942]: audit 2024-04-07T17:15:56.905802+0000 mon.a (mon.0) 3205 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:15:58.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:15:57 smithi100 bash[74942]: audit 2024-04-07T17:15:56.907187+0000 mon.a (mon.0) 3206 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:15:58.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:15:57 smithi100 bash[74942]: audit 2024-04-07T17:15:56.916595+0000 mon.a (mon.0) 3207 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:15:58.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:15:57 smithi100 bash[76202]: audit 2024-04-07T17:15:56.905802+0000 mon.a (mon.0) 3205 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:15:58.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:15:57 smithi100 bash[76202]: audit 2024-04-07T17:15:56.907187+0000 mon.a (mon.0) 3206 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:15:58.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:15:57 smithi100 bash[76202]: audit 2024-04-07T17:15:56.916595+0000 mon.a (mon.0) 3207 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:15:58.704 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:15:58 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:15:58] "GET /metrics HTTP/1.1" 200 60780 "" "Prometheus/2.43.0" 2024-04-07T17:15:59.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:15:58 smithi100 bash[74942]: cluster 2024-04-07T17:15:57.704681+0000 mgr.y (mgr.25175) 1818 : cluster [DBG] pgmap v3055: 457 pgs: 457 active+clean; 169 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 86 KiB/s rd, 1.6 MiB/s wr, 20 op/s 2024-04-07T17:15:59.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:15:58 smithi100 bash[76202]: cluster 2024-04-07T17:15:57.704681+0000 mgr.y (mgr.25175) 1818 : cluster [DBG] pgmap v3055: 457 pgs: 457 active+clean; 169 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 86 KiB/s rd, 1.6 MiB/s wr, 20 op/s 2024-04-07T17:15:59.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:15:58 smithi144 bash[66654]: cluster 2024-04-07T17:15:57.704681+0000 mgr.y (mgr.25175) 1818 : cluster [DBG] pgmap v3055: 457 pgs: 457 active+clean; 169 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 86 KiB/s rd, 1.6 MiB/s wr, 20 op/s 2024-04-07T17:16:01.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:16:00 smithi100 bash[74942]: cluster 2024-04-07T17:15:59.705903+0000 mgr.y (mgr.25175) 1819 : cluster [DBG] pgmap v3056: 457 pgs: 457 active+clean; 169 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 93 KiB/s rd, 1.6 MiB/s wr, 29 op/s 2024-04-07T17:16:01.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:16:00 smithi100 bash[76202]: cluster 2024-04-07T17:15:59.705903+0000 mgr.y (mgr.25175) 1819 : cluster [DBG] pgmap v3056: 457 pgs: 457 active+clean; 169 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 93 KiB/s rd, 1.6 MiB/s wr, 29 op/s 2024-04-07T17:16:01.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:16:00 smithi144 bash[66654]: cluster 2024-04-07T17:15:59.705903+0000 mgr.y (mgr.25175) 1819 : cluster [DBG] pgmap v3056: 457 pgs: 457 active+clean; 169 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 93 KiB/s rd, 1.6 MiB/s wr, 29 op/s 2024-04-07T17:16:02.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:16:01 smithi100 bash[74942]: audit 2024-04-07T17:16:01.825552+0000 mon.a (mon.0) 3208 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:16:02.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:16:01 smithi100 bash[76202]: audit 2024-04-07T17:16:01.825552+0000 mon.a (mon.0) 3208 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:16:02.317 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:16:01 smithi144 bash[66654]: audit 2024-04-07T17:16:01.825552+0000 mon.a (mon.0) 3208 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:16:02.617 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:16:02 smithi144 bash[95233]: ts=2024-04-07T17:16:02.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:16:03.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:16:02 smithi100 bash[74942]: cluster 2024-04-07T17:16:01.707222+0000 mgr.y (mgr.25175) 1820 : cluster [DBG] pgmap v3057: 457 pgs: 457 active+clean; 169 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 93 KiB/s rd, 1.6 MiB/s wr, 29 op/s 2024-04-07T17:16:03.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:16:02 smithi100 bash[74942]: cluster 2024-04-07T17:16:01.935283+0000 mon.a (mon.0) 3209 : cluster [DBG] osdmap e1663: 8 total, 8 up, 8 in 2024-04-07T17:16:03.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:16:02 smithi100 bash[76202]: cluster 2024-04-07T17:16:01.707222+0000 mgr.y (mgr.25175) 1820 : cluster [DBG] pgmap v3057: 457 pgs: 457 active+clean; 169 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 93 KiB/s rd, 1.6 MiB/s wr, 29 op/s 2024-04-07T17:16:03.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:16:02 smithi100 bash[76202]: cluster 2024-04-07T17:16:01.935283+0000 mon.a (mon.0) 3209 : cluster [DBG] osdmap e1663: 8 total, 8 up, 8 in 2024-04-07T17:16:03.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:16:02 smithi144 bash[66654]: cluster 2024-04-07T17:16:01.707222+0000 mgr.y (mgr.25175) 1820 : cluster [DBG] pgmap v3057: 457 pgs: 457 active+clean; 169 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 93 KiB/s rd, 1.6 MiB/s wr, 29 op/s 2024-04-07T17:16:03.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:16:02 smithi144 bash[66654]: cluster 2024-04-07T17:16:01.935283+0000 mon.a (mon.0) 3209 : cluster [DBG] osdmap e1663: 8 total, 8 up, 8 in 2024-04-07T17:16:05.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:16:04 smithi100 bash[74942]: cluster 2024-04-07T17:16:03.709133+0000 mgr.y (mgr.25175) 1821 : cluster [DBG] pgmap v3059: 457 pgs: 457 active+clean; 177 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 146 KiB/s rd, 2.0 MiB/s wr, 42 op/s 2024-04-07T17:16:05.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:16:04 smithi100 bash[76202]: cluster 2024-04-07T17:16:03.709133+0000 mgr.y (mgr.25175) 1821 : cluster [DBG] pgmap v3059: 457 pgs: 457 active+clean; 177 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 146 KiB/s rd, 2.0 MiB/s wr, 42 op/s 2024-04-07T17:16:05.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:16:04 smithi144 bash[66654]: cluster 2024-04-07T17:16:03.709133+0000 mgr.y (mgr.25175) 1821 : cluster [DBG] pgmap v3059: 457 pgs: 457 active+clean; 177 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 146 KiB/s rd, 2.0 MiB/s wr, 42 op/s 2024-04-07T17:16:07.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:16:06 smithi100 bash[74942]: cluster 2024-04-07T17:16:05.711757+0000 mgr.y (mgr.25175) 1822 : cluster [DBG] pgmap v3060: 457 pgs: 457 active+clean; 185 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 52 KiB/s rd, 1.6 MiB/s wr, 32 op/s 2024-04-07T17:16:07.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:16:06 smithi100 bash[76202]: cluster 2024-04-07T17:16:05.711757+0000 mgr.y (mgr.25175) 1822 : cluster [DBG] pgmap v3060: 457 pgs: 457 active+clean; 185 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 52 KiB/s rd, 1.6 MiB/s wr, 32 op/s 2024-04-07T17:16:07.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:16:06 smithi144 bash[66654]: cluster 2024-04-07T17:16:05.711757+0000 mgr.y (mgr.25175) 1822 : cluster [DBG] pgmap v3060: 457 pgs: 457 active+clean; 185 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 52 KiB/s rd, 1.6 MiB/s wr, 32 op/s 2024-04-07T17:16:07.580 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.EncryptedFlattenSmallData (11477 ms) 2024-04-07T17:16:07.580 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.TestIOWithIOHint 2024-04-07T17:16:07.580 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:16:08.704 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:16:08 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:16:08] "GET /metrics HTTP/1.1" 200 60783 "" "Prometheus/2.43.0" 2024-04-07T17:16:09.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:16:08 smithi100 bash[74942]: cluster 2024-04-07T17:16:07.713705+0000 mgr.y (mgr.25175) 1823 : cluster [DBG] pgmap v3061: 457 pgs: 457 active+clean; 201 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 95 KiB/s rd, 3.2 MiB/s wr, 47 op/s 2024-04-07T17:16:09.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:16:08 smithi100 bash[76202]: cluster 2024-04-07T17:16:07.713705+0000 mgr.y (mgr.25175) 1823 : cluster [DBG] pgmap v3061: 457 pgs: 457 active+clean; 201 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 95 KiB/s rd, 3.2 MiB/s wr, 47 op/s 2024-04-07T17:16:09.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:16:08 smithi144 bash[66654]: cluster 2024-04-07T17:16:07.713705+0000 mgr.y (mgr.25175) 1823 : cluster [DBG] pgmap v3061: 457 pgs: 457 active+clean; 201 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 95 KiB/s rd, 3.2 MiB/s wr, 47 op/s 2024-04-07T17:16:11.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:16:10 smithi100 bash[76202]: cluster 2024-04-07T17:16:09.716139+0000 mgr.y (mgr.25175) 1824 : cluster [DBG] pgmap v3062: 457 pgs: 457 active+clean; 201 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 1.3 MiB/s rd, 3.2 MiB/s wr, 43 op/s 2024-04-07T17:16:11.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:16:10 smithi100 bash[74942]: cluster 2024-04-07T17:16:09.716139+0000 mgr.y (mgr.25175) 1824 : cluster [DBG] pgmap v3062: 457 pgs: 457 active+clean; 201 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 1.3 MiB/s rd, 3.2 MiB/s wr, 43 op/s 2024-04-07T17:16:11.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:16:10 smithi144 bash[66654]: cluster 2024-04-07T17:16:09.716139+0000 mgr.y (mgr.25175) 1824 : cluster [DBG] pgmap v3062: 457 pgs: 457 active+clean; 201 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 1.3 MiB/s rd, 3.2 MiB/s wr, 43 op/s 2024-04-07T17:16:12.617 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:16:12 smithi144 bash[95233]: ts=2024-04-07T17:16:12.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:16:13.111 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 512 2024-04-07T17:16:13.111 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 512 2024-04-07T17:16:13.112 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 512 2024-04-07T17:16:13.112 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 512 2024-04-07T17:16:13.112 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 512 2024-04-07T17:16:13.112 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:16:13.112 INFO:tasks.workunit.client.0.smithi100.stdout:started write 2024-04-07T17:16:13.112 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:16:13.112 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:16:13.112 INFO:tasks.workunit.client.0.smithi100.stdout:finished write 2024-04-07T17:16:13.112 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:16:13.112 INFO:tasks.workunit.client.0.smithi100.stdout:started write 2024-04-07T17:16:13.112 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:16:13.112 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:16:13.112 INFO:tasks.workunit.client.0.smithi100.stdout:finished write 2024-04-07T17:16:13.112 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:16:13.112 INFO:tasks.workunit.client.0.smithi100.stdout:started write 2024-04-07T17:16:13.113 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:16:13.113 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:16:13.113 INFO:tasks.workunit.client.0.smithi100.stdout:finished write 2024-04-07T17:16:13.113 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:16:13.113 INFO:tasks.workunit.client.0.smithi100.stdout:started write 2024-04-07T17:16:13.113 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:16:13.113 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:16:13.113 INFO:tasks.workunit.client.0.smithi100.stdout:finished write 2024-04-07T17:16:13.113 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:16:13.113 INFO:tasks.workunit.client.0.smithi100.stdout:started write 2024-04-07T17:16:13.113 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:16:13.113 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:16:13.113 INFO:tasks.workunit.client.0.smithi100.stdout:finished write 2024-04-07T17:16:13.113 INFO:tasks.workunit.client.0.smithi100.stdout:start compare and write 2024-04-07T17:16:13.113 INFO:tasks.workunit.client.0.smithi100.stdout:compare and wrote: 512 2024-04-07T17:16:13.113 INFO:tasks.workunit.client.0.smithi100.stdout:start compare and write 2024-04-07T17:16:13.114 INFO:tasks.workunit.client.0.smithi100.stdout:compare and wrote: 512 2024-04-07T17:16:13.114 INFO:tasks.workunit.client.0.smithi100.stdout:start compare and write 2024-04-07T17:16:13.114 INFO:tasks.workunit.client.0.smithi100.stdout:compare and wrote: 512 2024-04-07T17:16:13.114 INFO:tasks.workunit.client.0.smithi100.stdout:start compare and write 2024-04-07T17:16:13.114 INFO:tasks.workunit.client.0.smithi100.stdout:compare and wrote: 512 2024-04-07T17:16:13.114 INFO:tasks.workunit.client.0.smithi100.stdout:start compare and write 2024-04-07T17:16:13.114 INFO:tasks.workunit.client.0.smithi100.stdout:compare and wrote: 512 2024-04-07T17:16:13.114 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:16:13.114 INFO:tasks.workunit.client.0.smithi100.stdout:started aio compare and write 2024-04-07T17:16:13.114 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:16:13.114 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:16:13.114 INFO:tasks.workunit.client.0.smithi100.stdout:finished aio compare and write 2024-04-07T17:16:13.114 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:16:13.114 INFO:tasks.workunit.client.0.smithi100.stdout:started aio compare and write 2024-04-07T17:16:13.114 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:16:13.114 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:16:13.115 INFO:tasks.workunit.client.0.smithi100.stdout:finished aio compare and write 2024-04-07T17:16:13.115 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:16:13.115 INFO:tasks.workunit.client.0.smithi100.stdout:started aio compare and write 2024-04-07T17:16:13.115 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:16:13.115 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:16:13.115 INFO:tasks.workunit.client.0.smithi100.stdout:finished aio compare and write 2024-04-07T17:16:13.115 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:16:13.115 INFO:tasks.workunit.client.0.smithi100.stdout:started aio compare and write 2024-04-07T17:16:13.115 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:16:13.115 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:16:13.115 INFO:tasks.workunit.client.0.smithi100.stdout:finished aio compare and write 2024-04-07T17:16:13.115 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:16:13.115 INFO:tasks.workunit.client.0.smithi100.stdout:started aio compare and write 2024-04-07T17:16:13.115 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:16:13.115 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:16:13.115 INFO:tasks.workunit.client.0.smithi100.stdout:finished aio compare and write 2024-04-07T17:16:13.115 INFO:tasks.workunit.client.0.smithi100.stdout:read: 512 2024-04-07T17:16:13.115 INFO:tasks.workunit.client.0.smithi100.stdout:read: 512 2024-04-07T17:16:13.116 INFO:tasks.workunit.client.0.smithi100.stdout:read: 512 2024-04-07T17:16:13.116 INFO:tasks.workunit.client.0.smithi100.stdout:read: 512 2024-04-07T17:16:13.116 INFO:tasks.workunit.client.0.smithi100.stdout:read: 512 2024-04-07T17:16:13.116 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:16:13.116 INFO:tasks.workunit.client.0.smithi100.stdout:started read 2024-04-07T17:16:13.116 INFO:tasks.workunit.client.0.smithi100.stdout:read completion cb called! 2024-04-07T17:16:13.116 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 512 2024-04-07T17:16:13.116 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:16:13.116 INFO:tasks.workunit.client.0.smithi100.stdout:started read 2024-04-07T17:16:13.116 INFO:tasks.workunit.client.0.smithi100.stdout:read completion cb called! 2024-04-07T17:16:13.116 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 512 2024-04-07T17:16:13.116 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:16:13.116 INFO:tasks.workunit.client.0.smithi100.stdout:started read 2024-04-07T17:16:13.116 INFO:tasks.workunit.client.0.smithi100.stdout:read completion cb called! 2024-04-07T17:16:13.116 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 512 2024-04-07T17:16:13.116 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:16:13.117 INFO:tasks.workunit.client.0.smithi100.stdout:started read 2024-04-07T17:16:13.117 INFO:tasks.workunit.client.0.smithi100.stdout:read completion cb called! 2024-04-07T17:16:13.117 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 512 2024-04-07T17:16:13.117 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:16:13.117 INFO:tasks.workunit.client.0.smithi100.stdout:started read 2024-04-07T17:16:13.117 INFO:tasks.workunit.client.0.smithi100.stdout:read completion cb called! 2024-04-07T17:16:13.117 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 512 2024-04-07T17:16:13.117 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 512~512 = 512 2024-04-07T17:16:13.117 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:16:13.117 INFO:tasks.workunit.client.0.smithi100.stdout:aio discard: 1536~512 = 0 2024-04-07T17:16:13.117 INFO:tasks.workunit.client.0.smithi100.stdout:read: 512 2024-04-07T17:16:13.117 INFO:tasks.workunit.client.0.smithi100.stdout:read: 512 2024-04-07T17:16:13.117 INFO:tasks.workunit.client.0.smithi100.stdout:read: 512 2024-04-07T17:16:13.117 INFO:tasks.workunit.client.0.smithi100.stdout:read: 512 2024-04-07T17:16:13.117 INFO:tasks.workunit.client.0.smithi100.stdout:read: 512 2024-04-07T17:16:13.117 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 0 2024-04-07T17:16:13.117 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:16:13.118 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:16:13.118 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 0 2024-04-07T17:16:13.118 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:16:13.118 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:16:13.118 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 16384 2024-04-07T17:16:13.118 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:16:13.118 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:16:13.118 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 16384 2024-04-07T17:16:13.118 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:16:13.118 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:16:13.118 INFO:tasks.workunit.client.0.smithi100.stdout:expected fail, finished writesame 2024-04-07T17:16:13.118 INFO:tasks.workunit.client.0.smithi100.stdout:expected fail, finished writesame 2024-04-07T17:16:13.118 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 49152 2024-04-07T17:16:13.118 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:16:13.118 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:16:13.119 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 49152 2024-04-07T17:16:13.119 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:16:13.119 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:16:13.119 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 65536 2024-04-07T17:16:13.119 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:16:13.119 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:16:13.119 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 65536 2024-04-07T17:16:13.119 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:16:13.119 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:16:13.119 INFO:tasks.workunit.client.0.smithi100.stdout:expected fail, finished writesame 2024-04-07T17:16:13.119 INFO:tasks.workunit.client.0.smithi100.stdout:expected fail, finished writesame 2024-04-07T17:16:13.119 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 98304 2024-04-07T17:16:13.119 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:16:13.119 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:16:13.119 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 98304 2024-04-07T17:16:13.119 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:16:13.119 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:16:13.120 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 114688 2024-04-07T17:16:13.120 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:16:13.120 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:16:13.120 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 114688 2024-04-07T17:16:13.120 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:16:13.120 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:16:13.120 INFO:tasks.workunit.client.0.smithi100.stdout:expected fail, finished writesame 2024-04-07T17:16:13.120 INFO:tasks.workunit.client.0.smithi100.stdout:expected fail, finished writesame 2024-04-07T17:16:13.120 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 147456 2024-04-07T17:16:13.120 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:16:13.120 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:16:13.120 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 147456 2024-04-07T17:16:13.120 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:16:13.120 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:16:13.120 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 163840 2024-04-07T17:16:13.120 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:16:13.120 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:16:13.121 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 163840 2024-04-07T17:16:13.121 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:16:13.121 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:16:13.121 INFO:tasks.workunit.client.0.smithi100.stdout:expected fail, finished writesame 2024-04-07T17:16:13.121 INFO:tasks.workunit.client.0.smithi100.stdout:expected fail, finished writesame 2024-04-07T17:16:13.121 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 196608 2024-04-07T17:16:13.121 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:16:13.121 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:16:13.121 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 196608 2024-04-07T17:16:13.121 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:16:13.121 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:16:13.121 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 212992 2024-04-07T17:16:13.121 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:16:13.121 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:16:13.121 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 212992 2024-04-07T17:16:13.121 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:16:13.122 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:16:13.122 INFO:tasks.workunit.client.0.smithi100.stdout:expected fail, finished writesame 2024-04-07T17:16:13.122 INFO:tasks.workunit.client.0.smithi100.stdout:expected fail, finished writesame 2024-04-07T17:16:13.122 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:16:13.122 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:16:13.122 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:16:13.122 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:16:13.122 INFO:tasks.workunit.client.0.smithi100.stdout:finished writesame 2024-04-07T17:16:13.122 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:16:13.122 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:16:13.122 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:16:13.122 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:16:13.122 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:16:13.123 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:16:13.123 INFO:tasks.workunit.client.0.smithi100.stdout:finished writesame 2024-04-07T17:16:13.123 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:16:13.123 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:16:13.123 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:16:13.123 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:16:13.123 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:16:13.123 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:16:13.123 INFO:tasks.workunit.client.0.smithi100.stdout:finished writesame 2024-04-07T17:16:13.123 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:16:13.123 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:16:13.123 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:16:13.123 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:16:13.123 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:16:13.123 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:16:13.124 INFO:tasks.workunit.client.0.smithi100.stdout:finished writesame 2024-04-07T17:16:13.124 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:16:13.124 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:16:13.124 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:16:13.124 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:16:13.124 INFO:tasks.workunit.client.0.smithi100.stdout:expected fail, finished writesame 2024-04-07T17:16:13.124 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:16:13.124 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:16:13.124 INFO:tasks.workunit.client.0.smithi100.stdout:expected fail, finished writesame 2024-04-07T17:16:13.124 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:16:13.124 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:16:13.124 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:16:13.124 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:16:13.124 INFO:tasks.workunit.client.0.smithi100.stdout:finished writesame 2024-04-07T17:16:13.124 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:16:13.124 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:16:13.125 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:16:13.125 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:16:13.125 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:16:13.125 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:16:13.125 INFO:tasks.workunit.client.0.smithi100.stdout:finished writesame 2024-04-07T17:16:13.125 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:16:13.125 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:16:13.125 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:16:13.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:16:12 smithi144 bash[66654]: cluster 2024-04-07T17:16:11.717739+0000 mgr.y (mgr.25175) 1825 : cluster [DBG] pgmap v3063: 457 pgs: 457 active+clean; 201 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 1.7 MiB/s rd, 3.2 MiB/s wr, 43 op/s 2024-04-07T17:16:13.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:16:12 smithi100 bash[74942]: cluster 2024-04-07T17:16:11.717739+0000 mgr.y (mgr.25175) 1825 : cluster [DBG] pgmap v3063: 457 pgs: 457 active+clean; 201 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 1.7 MiB/s rd, 3.2 MiB/s wr, 43 op/s 2024-04-07T17:16:13.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:16:12 smithi100 bash[76202]: cluster 2024-04-07T17:16:11.717739+0000 mgr.y (mgr.25175) 1825 : cluster [DBG] pgmap v3063: 457 pgs: 457 active+clean; 201 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 1.7 MiB/s rd, 3.2 MiB/s wr, 43 op/s 2024-04-07T17:16:15.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:16:14 smithi144 bash[66654]: cluster 2024-04-07T17:16:13.720900+0000 mgr.y (mgr.25175) 1826 : cluster [DBG] pgmap v3064: 457 pgs: 457 active+clean; 201 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 1.6 MiB/s rd, 2.8 MiB/s wr, 342 op/s 2024-04-07T17:16:15.455 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:16:14 smithi100 bash[74942]: cluster 2024-04-07T17:16:13.720900+0000 mgr.y (mgr.25175) 1826 : cluster [DBG] pgmap v3064: 457 pgs: 457 active+clean; 201 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 1.6 MiB/s rd, 2.8 MiB/s wr, 342 op/s 2024-04-07T17:16:15.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:16:14 smithi100 bash[76202]: cluster 2024-04-07T17:16:13.720900+0000 mgr.y (mgr.25175) 1826 : cluster [DBG] pgmap v3064: 457 pgs: 457 active+clean; 201 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 1.6 MiB/s rd, 2.8 MiB/s wr, 342 op/s 2024-04-07T17:16:17.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:16:16 smithi144 bash[66654]: cluster 2024-04-07T17:16:15.723398+0000 mgr.y (mgr.25175) 1827 : cluster [DBG] pgmap v3065: 457 pgs: 457 active+clean; 201 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 1.6 MiB/s rd, 2.1 MiB/s wr, 324 op/s 2024-04-07T17:16:17.368 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:16:16 smithi144 bash[66654]: audit 2024-04-07T17:16:16.826275+0000 mon.a (mon.0) 3210 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:16:17.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:16:16 smithi100 bash[74942]: cluster 2024-04-07T17:16:15.723398+0000 mgr.y (mgr.25175) 1827 : cluster [DBG] pgmap v3065: 457 pgs: 457 active+clean; 201 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 1.6 MiB/s rd, 2.1 MiB/s wr, 324 op/s 2024-04-07T17:16:17.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:16:16 smithi100 bash[74942]: audit 2024-04-07T17:16:16.826275+0000 mon.a (mon.0) 3210 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:16:17.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:16:16 smithi100 bash[76202]: cluster 2024-04-07T17:16:15.723398+0000 mgr.y (mgr.25175) 1827 : cluster [DBG] pgmap v3065: 457 pgs: 457 active+clean; 201 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 1.6 MiB/s rd, 2.1 MiB/s wr, 324 op/s 2024-04-07T17:16:17.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:16:16 smithi100 bash[76202]: audit 2024-04-07T17:16:16.826275+0000 mon.a (mon.0) 3210 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:16:17.961 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:16:17.961 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:16:17.961 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:16:17.961 INFO:tasks.workunit.client.0.smithi100.stdout:finished writesame 2024-04-07T17:16:17.961 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:16:17.961 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:16:17.961 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:16:17.961 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:16:17.961 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:16:17.961 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:16:17.961 INFO:tasks.workunit.client.0.smithi100.stdout:finished writesame 2024-04-07T17:16:17.962 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:16:17.962 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:16:17.962 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:16:17.962 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:16:17.962 INFO:tasks.workunit.client.0.smithi100.stdout:expected fail, finished writesame 2024-04-07T17:16:17.962 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:16:17.962 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:16:17.962 INFO:tasks.workunit.client.0.smithi100.stdout:expected fail, finished writesame 2024-04-07T17:16:17.962 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:16:17.962 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:16:17.962 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:16:17.962 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:16:17.962 INFO:tasks.workunit.client.0.smithi100.stdout:finished writesame 2024-04-07T17:16:17.962 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:16:17.963 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:16:17.963 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:16:17.963 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:16:17.963 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:16:17.963 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:16:17.963 INFO:tasks.workunit.client.0.smithi100.stdout:finished writesame 2024-04-07T17:16:17.963 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:16:17.963 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:16:17.963 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:16:17.963 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:16:17.963 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:16:17.963 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:16:17.963 INFO:tasks.workunit.client.0.smithi100.stdout:finished writesame 2024-04-07T17:16:17.963 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:16:17.963 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:16:17.964 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:16:17.964 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:16:17.964 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:16:17.964 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:16:17.964 INFO:tasks.workunit.client.0.smithi100.stdout:finished writesame 2024-04-07T17:16:17.964 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:16:17.964 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:16:17.964 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:16:17.964 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:16:17.964 INFO:tasks.workunit.client.0.smithi100.stdout:expected fail, finished writesame 2024-04-07T17:16:17.964 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:16:17.964 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:16:17.964 INFO:tasks.workunit.client.0.smithi100.stdout:expected fail, finished writesame 2024-04-07T17:16:17.964 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:16:17.965 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:16:17.965 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:16:17.965 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:16:17.965 INFO:tasks.workunit.client.0.smithi100.stdout:finished writesame 2024-04-07T17:16:17.965 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:16:17.965 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:16:17.965 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:16:17.965 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:16:17.965 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:16:17.965 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:16:17.965 INFO:tasks.workunit.client.0.smithi100.stdout:finished writesame 2024-04-07T17:16:17.965 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:16:17.965 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:16:17.965 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:16:17.966 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:16:17.966 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:16:17.966 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:16:17.966 INFO:tasks.workunit.client.0.smithi100.stdout:finished writesame 2024-04-07T17:16:17.966 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:16:17.966 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:16:17.966 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:16:17.966 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:16:17.966 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:16:17.966 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:16:17.966 INFO:tasks.workunit.client.0.smithi100.stdout:finished writesame 2024-04-07T17:16:17.966 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:16:17.966 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:16:17.966 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:16:17.967 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:16:17.967 INFO:tasks.workunit.client.0.smithi100.stdout:expected fail, finished writesame 2024-04-07T17:16:17.967 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:16:17.967 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:16:17.967 INFO:tasks.workunit.client.0.smithi100.stdout:expected fail, finished writesame 2024-04-07T17:16:17.967 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:16:17.967 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:16:17.967 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:16:17.967 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:16:17.967 INFO:tasks.workunit.client.0.smithi100.stdout:finished writesame 2024-04-07T17:16:17.967 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:16:17.967 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:16:17.967 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:16:17.967 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:16:17.967 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:16:17.968 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:16:17.968 INFO:tasks.workunit.client.0.smithi100.stdout:finished writesame 2024-04-07T17:16:17.968 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:16:17.968 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:16:17.968 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:16:17.968 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:16:17.968 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:16:17.968 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:16:17.968 INFO:tasks.workunit.client.0.smithi100.stdout:finished writesame 2024-04-07T17:16:17.968 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:16:17.968 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:16:17.968 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:16:17.968 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:16:17.968 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:16:17.968 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:16:17.969 INFO:tasks.workunit.client.0.smithi100.stdout:finished writesame 2024-04-07T17:16:17.969 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:16:17.969 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:16:17.969 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:16:17.969 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:16:17.969 INFO:tasks.workunit.client.0.smithi100.stdout:expected fail, finished writesame 2024-04-07T17:16:17.969 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:16:17.969 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:16:17.969 INFO:tasks.workunit.client.0.smithi100.stdout:expected fail, finished writesame 2024-04-07T17:16:17.969 INFO:tasks.workunit.client.0.smithi100.stdout:read completion cb called! 2024-04-07T17:16:17.969 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 512 2024-04-07T17:16:17.969 INFO:tasks.workunit.client.0.smithi100.stdout:read completion cb called! 2024-04-07T17:16:17.969 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.TestIOWithIOHint (10381 ms) 2024-04-07T17:16:17.969 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.TestDataPoolIO 2024-04-07T17:16:18.704 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:16:18 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:16:18] "GET /metrics HTTP/1.1" 200 60778 "" "Prometheus/2.43.0" 2024-04-07T17:16:19.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:16:18 smithi144 bash[66654]: cluster 2024-04-07T17:16:17.726300+0000 mgr.y (mgr.25175) 1828 : cluster [DBG] pgmap v3066: 457 pgs: 457 active+clean; 201 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 1.7 MiB/s rd, 1.5 MiB/s wr, 660 op/s 2024-04-07T17:16:19.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:16:18 smithi100 bash[74942]: cluster 2024-04-07T17:16:17.726300+0000 mgr.y (mgr.25175) 1828 : cluster [DBG] pgmap v3066: 457 pgs: 457 active+clean; 201 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 1.7 MiB/s rd, 1.5 MiB/s wr, 660 op/s 2024-04-07T17:16:19.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:16:18 smithi100 bash[76202]: cluster 2024-04-07T17:16:17.726300+0000 mgr.y (mgr.25175) 1828 : cluster [DBG] pgmap v3066: 457 pgs: 457 active+clean; 201 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 1.7 MiB/s rd, 1.5 MiB/s wr, 660 op/s 2024-04-07T17:16:20.001 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:16:20.370 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:16:20 smithi144 bash[66654]: cluster 2024-04-07T17:16:18.985470+0000 mon.a (mon.0) 3211 : cluster [DBG] osdmap e1664: 8 total, 8 up, 8 in 2024-04-07T17:16:20.370 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:16:20 smithi144 bash[66654]: audit 2024-04-07T17:16:18.990333+0000 mon.a (mon.0) 3212 : audit [INF] from='client.? 172.21.15.100:0/1175625814' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:16:20.370 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:16:20 smithi144 bash[66654]: cluster 2024-04-07T17:16:19.727959+0000 mgr.y (mgr.25175) 1829 : cluster [DBG] pgmap v3068: 489 pgs: 4 creating+peering, 28 unknown, 457 active+clean; 201 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 793 KiB/s rd, 210 KiB/s wr, 764 op/s 2024-04-07T17:16:20.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:16:19 smithi100 bash[74942]: cluster 2024-04-07T17:16:18.985470+0000 mon.a (mon.0) 3211 : cluster [DBG] osdmap e1664: 8 total, 8 up, 8 in 2024-04-07T17:16:20.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:16:19 smithi100 bash[74942]: audit 2024-04-07T17:16:18.990333+0000 mon.a (mon.0) 3212 : audit [INF] from='client.? 172.21.15.100:0/1175625814' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:16:20.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:16:19 smithi100 bash[74942]: cluster 2024-04-07T17:16:19.727959+0000 mgr.y (mgr.25175) 1829 : cluster [DBG] pgmap v3068: 489 pgs: 4 creating+peering, 28 unknown, 457 active+clean; 201 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 793 KiB/s rd, 210 KiB/s wr, 764 op/s 2024-04-07T17:16:20.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:16:20 smithi100 bash[76202]: cluster 2024-04-07T17:16:18.985470+0000 mon.a (mon.0) 3211 : cluster [DBG] osdmap e1664: 8 total, 8 up, 8 in 2024-04-07T17:16:20.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:16:20 smithi100 bash[76202]: audit 2024-04-07T17:16:18.990333+0000 mon.a (mon.0) 3212 : audit [INF] from='client.? 172.21.15.100:0/1175625814' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:16:20.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:16:20 smithi100 bash[76202]: cluster 2024-04-07T17:16:19.727959+0000 mgr.y (mgr.25175) 1829 : cluster [DBG] pgmap v3068: 489 pgs: 4 creating+peering, 28 unknown, 457 active+clean; 201 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 793 KiB/s rd, 210 KiB/s wr, 764 op/s 2024-04-07T17:16:21.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:16:21 smithi144 bash[66654]: cluster 2024-04-07T17:16:19.978250+0000 mon.a (mon.0) 3213 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T17:16:21.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:16:21 smithi144 bash[66654]: audit 2024-04-07T17:16:19.980061+0000 mon.a (mon.0) 3214 : audit [INF] from='client.? 172.21.15.100:0/1175625814' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-26","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T17:16:21.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:16:21 smithi144 bash[66654]: cluster 2024-04-07T17:16:19.988325+0000 mon.a (mon.0) 3215 : cluster [DBG] osdmap e1665: 8 total, 8 up, 8 in 2024-04-07T17:16:21.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:16:21 smithi100 bash[74942]: cluster 2024-04-07T17:16:19.978250+0000 mon.a (mon.0) 3213 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T17:16:21.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:16:21 smithi100 bash[74942]: audit 2024-04-07T17:16:19.980061+0000 mon.a (mon.0) 3214 : audit [INF] from='client.? 172.21.15.100:0/1175625814' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-26","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T17:16:21.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:16:21 smithi100 bash[74942]: cluster 2024-04-07T17:16:19.988325+0000 mon.a (mon.0) 3215 : cluster [DBG] osdmap e1665: 8 total, 8 up, 8 in 2024-04-07T17:16:21.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:16:21 smithi100 bash[76202]: cluster 2024-04-07T17:16:19.978250+0000 mon.a (mon.0) 3213 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T17:16:21.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:16:21 smithi100 bash[76202]: audit 2024-04-07T17:16:19.980061+0000 mon.a (mon.0) 3214 : audit [INF] from='client.? 172.21.15.100:0/1175625814' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-26","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T17:16:21.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:16:21 smithi100 bash[76202]: cluster 2024-04-07T17:16:19.988325+0000 mon.a (mon.0) 3215 : cluster [DBG] osdmap e1665: 8 total, 8 up, 8 in 2024-04-07T17:16:22.202 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 512 2024-04-07T17:16:22.202 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 512 2024-04-07T17:16:22.202 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 512 2024-04-07T17:16:22.202 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 512 2024-04-07T17:16:22.202 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 512 2024-04-07T17:16:22.202 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:16:22.203 INFO:tasks.workunit.client.0.smithi100.stdout:started write 2024-04-07T17:16:22.203 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:16:22.203 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:16:22.203 INFO:tasks.workunit.client.0.smithi100.stdout:finished write 2024-04-07T17:16:22.203 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:16:22.203 INFO:tasks.workunit.client.0.smithi100.stdout:started write 2024-04-07T17:16:22.203 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:16:22.203 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:16:22.203 INFO:tasks.workunit.client.0.smithi100.stdout:finished write 2024-04-07T17:16:22.203 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:16:22.203 INFO:tasks.workunit.client.0.smithi100.stdout:started write 2024-04-07T17:16:22.203 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:16:22.203 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:16:22.204 INFO:tasks.workunit.client.0.smithi100.stdout:finished write 2024-04-07T17:16:22.204 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:16:22.204 INFO:tasks.workunit.client.0.smithi100.stdout:started write 2024-04-07T17:16:22.204 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:16:22.204 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:16:22.204 INFO:tasks.workunit.client.0.smithi100.stdout:finished write 2024-04-07T17:16:22.204 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:16:22.204 INFO:tasks.workunit.client.0.smithi100.stdout:started write 2024-04-07T17:16:22.204 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:16:22.204 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:16:22.204 INFO:tasks.workunit.client.0.smithi100.stdout:finished write 2024-04-07T17:16:22.204 INFO:tasks.workunit.client.0.smithi100.stdout:read: 512 2024-04-07T17:16:22.204 INFO:tasks.workunit.client.0.smithi100.stdout:read: 512 2024-04-07T17:16:22.204 INFO:tasks.workunit.client.0.smithi100.stdout:read: 512 2024-04-07T17:16:22.205 INFO:tasks.workunit.client.0.smithi100.stdout:read: 512 2024-04-07T17:16:22.205 INFO:tasks.workunit.client.0.smithi100.stdout:read: 512 2024-04-07T17:16:22.205 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:16:22.205 INFO:tasks.workunit.client.0.smithi100.stdout:started read 2024-04-07T17:16:22.205 INFO:tasks.workunit.client.0.smithi100.stdout:read completion cb called! 2024-04-07T17:16:22.205 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 512 2024-04-07T17:16:22.205 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:16:22.205 INFO:tasks.workunit.client.0.smithi100.stdout:started read 2024-04-07T17:16:22.205 INFO:tasks.workunit.client.0.smithi100.stdout:read completion cb called! 2024-04-07T17:16:22.205 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 512 2024-04-07T17:16:22.205 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:16:22.205 INFO:tasks.workunit.client.0.smithi100.stdout:started read 2024-04-07T17:16:22.205 INFO:tasks.workunit.client.0.smithi100.stdout:read completion cb called! 2024-04-07T17:16:22.205 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 512 2024-04-07T17:16:22.206 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:16:22.206 INFO:tasks.workunit.client.0.smithi100.stdout:started read 2024-04-07T17:16:22.206 INFO:tasks.workunit.client.0.smithi100.stdout:read completion cb called! 2024-04-07T17:16:22.206 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 512 2024-04-07T17:16:22.206 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:16:22.206 INFO:tasks.workunit.client.0.smithi100.stdout:started read 2024-04-07T17:16:22.206 INFO:tasks.workunit.client.0.smithi100.stdout:read completion cb called! 2024-04-07T17:16:22.206 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 512 2024-04-07T17:16:22.206 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 512~512 = 512 2024-04-07T17:16:22.206 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:16:22.207 INFO:tasks.workunit.client.0.smithi100.stdout:aio discard: 1536~512 = 0 2024-04-07T17:16:22.207 INFO:tasks.workunit.client.0.smithi100.stdout:read: 512 2024-04-07T17:16:22.207 INFO:tasks.workunit.client.0.smithi100.stdout:read: 512 2024-04-07T17:16:22.207 INFO:tasks.workunit.client.0.smithi100.stdout:read: 512 2024-04-07T17:16:22.207 INFO:tasks.workunit.client.0.smithi100.stdout:read: 512 2024-04-07T17:16:22.207 INFO:tasks.workunit.client.0.smithi100.stdout:read: 512 2024-04-07T17:16:22.207 INFO:tasks.workunit.client.0.smithi100.stdout:read completion cb called! 2024-04-07T17:16:22.207 INFO:tasks.workunit.client.0.smithi100.stdout:read completion cb called! 2024-04-07T17:16:22.207 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.TestDataPoolIO (4241 ms) 2024-04-07T17:16:22.207 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.TestCompareAndWriteMismatch 2024-04-07T17:16:22.207 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:16:22.262 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.TestCompareAndWriteMismatch (60 ms) 2024-04-07T17:16:22.262 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.TestAioCompareAndWriteMismatch 2024-04-07T17:16:22.262 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:16:22.318 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:16:22 smithi144 bash[66654]: cluster 2024-04-07T17:16:21.015632+0000 mon.a (mon.0) 3216 : cluster [DBG] osdmap e1666: 8 total, 8 up, 8 in 2024-04-07T17:16:22.318 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:16:22 smithi144 bash[66654]: cluster 2024-04-07T17:16:21.728929+0000 mgr.y (mgr.25175) 1830 : cluster [DBG] pgmap v3071: 489 pgs: 10 creating+peering, 22 unknown, 457 active+clean; 201 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 324 KiB/s rd, 173 KiB/s wr, 673 op/s 2024-04-07T17:16:22.326 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.TestAioCompareAndWriteMismatch (64 ms) 2024-04-07T17:16:22.326 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.TestCompareAndWriteSuccess 2024-04-07T17:16:22.326 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:16:22.386 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.TestCompareAndWriteSuccess (61 ms) 2024-04-07T17:16:22.387 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.TestAioCompareAndWriteSuccess 2024-04-07T17:16:22.387 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:16:22.451 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.TestAioCompareAndWriteSuccess (64 ms) 2024-04-07T17:16:22.451 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.TestCompareAndWriteStripeUnitUnaligned 2024-04-07T17:16:22.451 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:16:22.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:16:22 smithi100 bash[76202]: cluster 2024-04-07T17:16:21.015632+0000 mon.a (mon.0) 3216 : cluster [DBG] osdmap e1666: 8 total, 8 up, 8 in 2024-04-07T17:16:22.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:16:22 smithi100 bash[76202]: cluster 2024-04-07T17:16:21.728929+0000 mgr.y (mgr.25175) 1830 : cluster [DBG] pgmap v3071: 489 pgs: 10 creating+peering, 22 unknown, 457 active+clean; 201 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 324 KiB/s rd, 173 KiB/s wr, 673 op/s 2024-04-07T17:16:22.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:16:22 smithi100 bash[74942]: cluster 2024-04-07T17:16:21.015632+0000 mon.a (mon.0) 3216 : cluster [DBG] osdmap e1666: 8 total, 8 up, 8 in 2024-04-07T17:16:22.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:16:22 smithi100 bash[74942]: cluster 2024-04-07T17:16:21.728929+0000 mgr.y (mgr.25175) 1830 : cluster [DBG] pgmap v3071: 489 pgs: 10 creating+peering, 22 unknown, 457 active+clean; 201 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 324 KiB/s rd, 173 KiB/s wr, 673 op/s 2024-04-07T17:16:22.581 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.TestCompareAndWriteStripeUnitUnaligned (130 ms) 2024-04-07T17:16:22.582 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.TestAioCompareAndWriteStripeUnitUnaligned 2024-04-07T17:16:22.582 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:16:22.618 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:16:22 smithi144 bash[95233]: ts=2024-04-07T17:16:22.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:16:22.715 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.TestAioCompareAndWriteStripeUnitUnaligned (134 ms) 2024-04-07T17:16:22.715 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.TestCompareAndWriteTooLarge 2024-04-07T17:16:22.715 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:16:22.862 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.TestCompareAndWriteTooLarge (147 ms) 2024-04-07T17:16:22.862 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.TestAioCompareAndWriteTooLarge 2024-04-07T17:16:22.862 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:16:22.989 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.TestAioCompareAndWriteTooLarge (127 ms) 2024-04-07T17:16:22.989 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.TestCompareAndWriteStripeUnitSuccess 2024-04-07T17:16:22.989 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:16:23.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:16:23 smithi144 bash[66654]: cluster 2024-04-07T17:16:22.008576+0000 mon.a (mon.0) 3217 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T17:16:23.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:16:23 smithi144 bash[66654]: cluster 2024-04-07T17:16:22.029085+0000 mon.a (mon.0) 3218 : cluster [DBG] osdmap e1667: 8 total, 8 up, 8 in 2024-04-07T17:16:23.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:16:23 smithi100 bash[74942]: cluster 2024-04-07T17:16:22.008576+0000 mon.a (mon.0) 3217 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T17:16:23.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:16:23 smithi100 bash[74942]: cluster 2024-04-07T17:16:22.029085+0000 mon.a (mon.0) 3218 : cluster [DBG] osdmap e1667: 8 total, 8 up, 8 in 2024-04-07T17:16:23.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:16:23 smithi100 bash[76202]: cluster 2024-04-07T17:16:22.008576+0000 mon.a (mon.0) 3217 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T17:16:23.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:16:23 smithi100 bash[76202]: cluster 2024-04-07T17:16:22.029085+0000 mon.a (mon.0) 3218 : cluster [DBG] osdmap e1667: 8 total, 8 up, 8 in 2024-04-07T17:16:23.597 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.TestCompareAndWriteStripeUnitSuccess (608 ms) 2024-04-07T17:16:23.597 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.TestAioCompareAndWriteStripeUnitSuccess 2024-04-07T17:16:23.597 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:16:24.110 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.TestAioCompareAndWriteStripeUnitSuccess (513 ms) 2024-04-07T17:16:24.110 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.TestAioCompareAndWriteVIovecLenDiffers 2024-04-07T17:16:24.110 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:16:24.172 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.TestAioCompareAndWriteVIovecLenDiffers (62 ms) 2024-04-07T17:16:24.172 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.TestAioCompareAndWriteVMismatch 2024-04-07T17:16:24.172 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:16:24.237 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.TestAioCompareAndWriteVMismatch (65 ms) 2024-04-07T17:16:24.237 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.TestAioCompareAndWriteVSuccess 2024-04-07T17:16:24.237 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:16:24.307 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.TestAioCompareAndWriteVSuccess (70 ms) 2024-04-07T17:16:24.307 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.TestScatterGatherIO 2024-04-07T17:16:24.307 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:16:24.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:16:24 smithi144 bash[66654]: cluster 2024-04-07T17:16:23.731324+0000 mgr.y (mgr.25175) 1831 : cluster [DBG] pgmap v3073: 489 pgs: 10 creating+peering, 7 unknown, 472 active+clean; 207 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 953 KiB/s rd, 864 KiB/s wr, 187 op/s 2024-04-07T17:16:24.382 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.TestScatterGatherIO (75 ms) 2024-04-07T17:16:24.383 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.TestEmptyDiscard 2024-04-07T17:16:24.383 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:16:24.443 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:16:24.443 INFO:tasks.workunit.client.0.smithi100.stdout:aio discard: 0~1048576 = 0 2024-04-07T17:16:24.443 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:16:24.443 INFO:tasks.workunit.client.0.smithi100.stdout:aio discard: 0~4194304 = 0 2024-04-07T17:16:24.443 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:16:24.443 INFO:tasks.workunit.client.0.smithi100.stdout:aio discard: 3145728~1048576 = 0 2024-04-07T17:16:24.443 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.TestEmptyDiscard (61 ms) 2024-04-07T17:16:24.443 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.TestFUA 2024-04-07T17:16:24.443 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:16:24.453 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:16:24 smithi100 bash[74942]: cluster 2024-04-07T17:16:23.731324+0000 mgr.y (mgr.25175) 1831 : cluster [DBG] pgmap v3073: 489 pgs: 10 creating+peering, 7 unknown, 472 active+clean; 207 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 953 KiB/s rd, 864 KiB/s wr, 187 op/s 2024-04-07T17:16:24.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:16:24 smithi100 bash[76202]: cluster 2024-04-07T17:16:23.731324+0000 mgr.y (mgr.25175) 1831 : cluster [DBG] pgmap v3073: 489 pgs: 10 creating+peering, 7 unknown, 472 active+clean; 207 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 953 KiB/s rd, 864 KiB/s wr, 187 op/s 2024-04-07T17:16:24.694 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 512 2024-04-07T17:16:24.694 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 512 2024-04-07T17:16:24.694 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 512 2024-04-07T17:16:24.695 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 512 2024-04-07T17:16:24.695 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 512 2024-04-07T17:16:24.695 INFO:tasks.workunit.client.0.smithi100.stdout:read: 512 2024-04-07T17:16:24.695 INFO:tasks.workunit.client.0.smithi100.stdout:read: 512 2024-04-07T17:16:24.695 INFO:tasks.workunit.client.0.smithi100.stdout:read: 512 2024-04-07T17:16:24.695 INFO:tasks.workunit.client.0.smithi100.stdout:read: 512 2024-04-07T17:16:24.695 INFO:tasks.workunit.client.0.smithi100.stdout:read: 512 2024-04-07T17:16:24.695 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:16:24.695 INFO:tasks.workunit.client.0.smithi100.stdout:started write 2024-04-07T17:16:24.696 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:16:24.696 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:16:24.696 INFO:tasks.workunit.client.0.smithi100.stdout:finished write 2024-04-07T17:16:24.696 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:16:24.696 INFO:tasks.workunit.client.0.smithi100.stdout:started write 2024-04-07T17:16:24.696 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:16:24.696 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:16:24.696 INFO:tasks.workunit.client.0.smithi100.stdout:finished write 2024-04-07T17:16:24.696 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:16:24.697 INFO:tasks.workunit.client.0.smithi100.stdout:started write 2024-04-07T17:16:24.697 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:16:24.697 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:16:24.697 INFO:tasks.workunit.client.0.smithi100.stdout:finished write 2024-04-07T17:16:24.697 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:16:24.697 INFO:tasks.workunit.client.0.smithi100.stdout:started write 2024-04-07T17:16:24.697 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:16:24.697 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:16:24.697 INFO:tasks.workunit.client.0.smithi100.stdout:finished write 2024-04-07T17:16:24.698 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:16:24.698 INFO:tasks.workunit.client.0.smithi100.stdout:started write 2024-04-07T17:16:24.698 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:16:24.698 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:16:24.698 INFO:tasks.workunit.client.0.smithi100.stdout:finished write 2024-04-07T17:16:24.698 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:16:24.698 INFO:tasks.workunit.client.0.smithi100.stdout:started read 2024-04-07T17:16:24.698 INFO:tasks.workunit.client.0.smithi100.stdout:read completion cb called! 2024-04-07T17:16:24.698 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 512 2024-04-07T17:16:24.699 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:16:24.699 INFO:tasks.workunit.client.0.smithi100.stdout:started read 2024-04-07T17:16:24.699 INFO:tasks.workunit.client.0.smithi100.stdout:read completion cb called! 2024-04-07T17:16:24.699 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 512 2024-04-07T17:16:24.699 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:16:24.699 INFO:tasks.workunit.client.0.smithi100.stdout:started read 2024-04-07T17:16:24.699 INFO:tasks.workunit.client.0.smithi100.stdout:read completion cb called! 2024-04-07T17:16:24.699 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 512 2024-04-07T17:16:24.699 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:16:24.700 INFO:tasks.workunit.client.0.smithi100.stdout:started read 2024-04-07T17:16:24.700 INFO:tasks.workunit.client.0.smithi100.stdout:read completion cb called! 2024-04-07T17:16:24.700 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 512 2024-04-07T17:16:24.700 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:16:24.700 INFO:tasks.workunit.client.0.smithi100.stdout:started read 2024-04-07T17:16:24.700 INFO:tasks.workunit.client.0.smithi100.stdout:read completion cb called! 2024-04-07T17:16:24.700 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 512 2024-04-07T17:16:24.700 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.TestFUA (251 ms) 2024-04-07T17:16:24.700 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.TestIOPP 2024-04-07T17:16:24.701 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:16:24.741 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 512 2024-04-07T17:16:24.741 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 512 2024-04-07T17:16:24.741 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 512 2024-04-07T17:16:24.741 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 512 2024-04-07T17:16:24.741 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 512 2024-04-07T17:16:24.741 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:16:24.742 INFO:tasks.workunit.client.0.smithi100.stdout:started write 2024-04-07T17:16:24.742 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:16:24.742 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:16:24.742 INFO:tasks.workunit.client.0.smithi100.stdout:finished write 2024-04-07T17:16:24.742 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:16:24.742 INFO:tasks.workunit.client.0.smithi100.stdout:started write 2024-04-07T17:16:24.742 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:16:24.742 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:16:24.742 INFO:tasks.workunit.client.0.smithi100.stdout:finished write 2024-04-07T17:16:24.742 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:16:24.742 INFO:tasks.workunit.client.0.smithi100.stdout:started write 2024-04-07T17:16:24.742 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:16:24.742 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:16:24.742 INFO:tasks.workunit.client.0.smithi100.stdout:finished write 2024-04-07T17:16:24.743 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:16:24.743 INFO:tasks.workunit.client.0.smithi100.stdout:started write 2024-04-07T17:16:24.743 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:16:24.743 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:16:24.743 INFO:tasks.workunit.client.0.smithi100.stdout:finished write 2024-04-07T17:16:24.743 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:16:24.743 INFO:tasks.workunit.client.0.smithi100.stdout:started write 2024-04-07T17:16:24.743 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:16:24.782 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:16:24.782 INFO:tasks.workunit.client.0.smithi100.stdout:finished write 2024-04-07T17:16:24.782 INFO:tasks.workunit.client.0.smithi100.stdout:start compare and write 2024-04-07T17:16:24.782 INFO:tasks.workunit.client.0.smithi100.stdout:compare and wrote: 512 2024-04-07T17:16:24.782 INFO:tasks.workunit.client.0.smithi100.stdout:start compare and write 2024-04-07T17:16:24.782 INFO:tasks.workunit.client.0.smithi100.stdout:compare and wrote: 512 2024-04-07T17:16:24.782 INFO:tasks.workunit.client.0.smithi100.stdout:start compare and write 2024-04-07T17:16:24.782 INFO:tasks.workunit.client.0.smithi100.stdout:compare and wrote: 512 2024-04-07T17:16:24.783 INFO:tasks.workunit.client.0.smithi100.stdout:start compare and write 2024-04-07T17:16:24.783 INFO:tasks.workunit.client.0.smithi100.stdout:compare and wrote: 512 2024-04-07T17:16:24.783 INFO:tasks.workunit.client.0.smithi100.stdout:start compare and write 2024-04-07T17:16:24.783 INFO:tasks.workunit.client.0.smithi100.stdout:compare and wrote: 512 2024-04-07T17:16:24.783 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:16:24.783 INFO:tasks.workunit.client.0.smithi100.stdout:started aio compare and write 2024-04-07T17:16:24.783 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:16:24.785 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:16:24.785 INFO:tasks.workunit.client.0.smithi100.stdout:finished aio compare and write 2024-04-07T17:16:24.785 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:16:24.785 INFO:tasks.workunit.client.0.smithi100.stdout:started aio compare and write 2024-04-07T17:16:24.785 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:16:24.788 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:16:24.788 INFO:tasks.workunit.client.0.smithi100.stdout:finished aio compare and write 2024-04-07T17:16:24.788 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:16:24.788 INFO:tasks.workunit.client.0.smithi100.stdout:started aio compare and write 2024-04-07T17:16:24.788 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:16:24.791 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:16:24.791 INFO:tasks.workunit.client.0.smithi100.stdout:finished aio compare and write 2024-04-07T17:16:24.791 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:16:24.791 INFO:tasks.workunit.client.0.smithi100.stdout:started aio compare and write 2024-04-07T17:16:24.792 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:16:24.794 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:16:24.794 INFO:tasks.workunit.client.0.smithi100.stdout:finished aio compare and write 2024-04-07T17:16:24.794 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:16:24.795 INFO:tasks.workunit.client.0.smithi100.stdout:started aio compare and write 2024-04-07T17:16:24.795 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:16:24.801 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:16:24.801 INFO:tasks.workunit.client.0.smithi100.stdout:finished aio compare and write 2024-04-07T17:16:24.801 INFO:tasks.workunit.client.0.smithi100.stdout:read: 512 2024-04-07T17:16:24.802 INFO:tasks.workunit.client.0.smithi100.stdout:read: 512 2024-04-07T17:16:24.802 INFO:tasks.workunit.client.0.smithi100.stdout:read: 512 2024-04-07T17:16:24.802 INFO:tasks.workunit.client.0.smithi100.stdout:read: 512 2024-04-07T17:16:24.802 INFO:tasks.workunit.client.0.smithi100.stdout:read: 512 2024-04-07T17:16:24.802 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:16:24.802 INFO:tasks.workunit.client.0.smithi100.stdout:started read 2024-04-07T17:16:24.802 INFO:tasks.workunit.client.0.smithi100.stdout:read completion cb called! 2024-04-07T17:16:24.802 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 512 2024-04-07T17:16:24.802 INFO:tasks.workunit.client.0.smithi100.stdout:finished read 2024-04-07T17:16:24.803 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:16:24.803 INFO:tasks.workunit.client.0.smithi100.stdout:started read 2024-04-07T17:16:24.803 INFO:tasks.workunit.client.0.smithi100.stdout:read completion cb called! 2024-04-07T17:16:24.803 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 512 2024-04-07T17:16:24.803 INFO:tasks.workunit.client.0.smithi100.stdout:finished read 2024-04-07T17:16:24.803 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:16:24.803 INFO:tasks.workunit.client.0.smithi100.stdout:started read 2024-04-07T17:16:24.803 INFO:tasks.workunit.client.0.smithi100.stdout:read completion cb called! 2024-04-07T17:16:24.804 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 512 2024-04-07T17:16:24.804 INFO:tasks.workunit.client.0.smithi100.stdout:finished read 2024-04-07T17:16:24.804 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:16:24.804 INFO:tasks.workunit.client.0.smithi100.stdout:started read 2024-04-07T17:16:24.804 INFO:tasks.workunit.client.0.smithi100.stdout:read completion cb called! 2024-04-07T17:16:24.805 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 512 2024-04-07T17:16:24.805 INFO:tasks.workunit.client.0.smithi100.stdout:finished read 2024-04-07T17:16:24.806 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:16:24.806 INFO:tasks.workunit.client.0.smithi100.stdout:started read 2024-04-07T17:16:24.806 INFO:tasks.workunit.client.0.smithi100.stdout:read completion cb called! 2024-04-07T17:16:24.806 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 512 2024-04-07T17:16:24.806 INFO:tasks.workunit.client.0.smithi100.stdout:finished read 2024-04-07T17:16:24.806 INFO:tasks.workunit.client.0.smithi100.stdout:discard: 512~512 2024-04-07T17:16:24.806 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:16:27.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:16:26 smithi144 bash[66654]: cluster 2024-04-07T17:16:25.733645+0000 mgr.y (mgr.25175) 1832 : cluster [DBG] pgmap v3074: 489 pgs: 489 active+clean; 223 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 3.4 MiB/s rd, 3.3 MiB/s wr, 175 op/s 2024-04-07T17:16:27.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:16:26 smithi100 bash[74942]: cluster 2024-04-07T17:16:25.733645+0000 mgr.y (mgr.25175) 1832 : cluster [DBG] pgmap v3074: 489 pgs: 489 active+clean; 223 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 3.4 MiB/s rd, 3.3 MiB/s wr, 175 op/s 2024-04-07T17:16:27.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:16:26 smithi100 bash[76202]: cluster 2024-04-07T17:16:25.733645+0000 mgr.y (mgr.25175) 1832 : cluster [DBG] pgmap v3074: 489 pgs: 489 active+clean; 223 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 3.4 MiB/s rd, 3.3 MiB/s wr, 175 op/s 2024-04-07T17:16:28.704 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:16:28 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:16:28] "GET /metrics HTTP/1.1" 200 60778 "" "Prometheus/2.43.0" 2024-04-07T17:16:28.801 INFO:tasks.workunit.client.0.smithi100.stdout:read: 512 2024-04-07T17:16:28.802 INFO:tasks.workunit.client.0.smithi100.stdout:read: 512 2024-04-07T17:16:28.802 INFO:tasks.workunit.client.0.smithi100.stdout:read: 512 2024-04-07T17:16:28.802 INFO:tasks.workunit.client.0.smithi100.stdout:read: 512 2024-04-07T17:16:28.802 INFO:tasks.workunit.client.0.smithi100.stdout:read: 512 2024-04-07T17:16:28.802 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 0 2024-04-07T17:16:28.802 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:16:28.802 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:16:28.802 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 0 2024-04-07T17:16:28.802 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:16:28.802 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:16:28.802 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 16384 2024-04-07T17:16:28.802 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:16:28.803 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:16:28.803 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 16384 2024-04-07T17:16:28.803 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:16:28.803 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:16:28.803 INFO:tasks.workunit.client.0.smithi100.stdout:expected fail, finished writesame 2024-04-07T17:16:28.803 INFO:tasks.workunit.client.0.smithi100.stdout:expected fail, finished writesame 2024-04-07T17:16:28.803 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 49152 2024-04-07T17:16:28.803 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:16:28.803 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:16:28.803 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 49152 2024-04-07T17:16:28.803 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:16:28.803 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:16:28.803 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 65536 2024-04-07T17:16:28.803 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:16:28.803 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:16:28.803 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 65536 2024-04-07T17:16:28.804 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:16:28.804 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:16:28.804 INFO:tasks.workunit.client.0.smithi100.stdout:expected fail, finished writesame 2024-04-07T17:16:28.804 INFO:tasks.workunit.client.0.smithi100.stdout:expected fail, finished writesame 2024-04-07T17:16:28.804 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 98304 2024-04-07T17:16:28.804 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:16:28.804 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:16:28.804 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 98304 2024-04-07T17:16:28.804 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:16:28.804 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:16:28.804 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 114688 2024-04-07T17:16:28.804 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:16:28.804 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:16:28.804 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 114688 2024-04-07T17:16:28.804 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:16:28.805 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:16:28.805 INFO:tasks.workunit.client.0.smithi100.stdout:expected fail, finished writesame 2024-04-07T17:16:28.805 INFO:tasks.workunit.client.0.smithi100.stdout:expected fail, finished writesame 2024-04-07T17:16:28.805 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 147456 2024-04-07T17:16:28.805 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:16:28.805 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:16:28.805 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 147456 2024-04-07T17:16:28.805 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:16:28.805 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:16:28.805 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 163840 2024-04-07T17:16:28.805 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:16:28.805 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:16:28.805 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 163840 2024-04-07T17:16:28.805 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:16:28.805 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:16:28.805 INFO:tasks.workunit.client.0.smithi100.stdout:expected fail, finished writesame 2024-04-07T17:16:28.806 INFO:tasks.workunit.client.0.smithi100.stdout:expected fail, finished writesame 2024-04-07T17:16:28.806 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 196608 2024-04-07T17:16:28.806 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:16:28.806 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:16:28.806 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 196608 2024-04-07T17:16:28.806 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:16:28.806 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:16:28.806 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 212992 2024-04-07T17:16:28.806 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:16:28.806 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:16:28.806 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 212992 2024-04-07T17:16:28.806 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:16:28.806 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:16:28.806 INFO:tasks.workunit.client.0.smithi100.stdout:expected fail, finished writesame 2024-04-07T17:16:28.807 INFO:tasks.workunit.client.0.smithi100.stdout:expected fail, finished writesame 2024-04-07T17:16:28.807 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:16:28.807 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:16:28.807 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:16:28.807 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:16:28.807 INFO:tasks.workunit.client.0.smithi100.stdout:finished writesame 2024-04-07T17:16:28.807 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:16:28.807 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:16:28.807 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:16:28.807 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:16:28.807 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:16:28.807 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:16:28.807 INFO:tasks.workunit.client.0.smithi100.stdout:finished writesame 2024-04-07T17:16:28.807 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:16:28.807 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:16:28.807 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:16:28.808 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:16:28.808 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:16:28.832 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:16:28.832 INFO:tasks.workunit.client.0.smithi100.stdout:finished writesame 2024-04-07T17:16:28.832 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:16:28.832 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:16:28.832 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:16:28.832 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:16:28.832 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:16:28.861 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:16:28.861 INFO:tasks.workunit.client.0.smithi100.stdout:finished writesame 2024-04-07T17:16:28.861 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:16:28.861 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:16:28.861 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:16:28.861 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:16:28.861 INFO:tasks.workunit.client.0.smithi100.stdout:expected fail, finished writesame 2024-04-07T17:16:28.861 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:16:28.862 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:16:28.862 INFO:tasks.workunit.client.0.smithi100.stdout:expected fail, finished writesame 2024-04-07T17:16:28.862 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:16:28.862 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:16:28.862 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:16:28.947 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:16:28.947 INFO:tasks.workunit.client.0.smithi100.stdout:finished writesame 2024-04-07T17:16:28.947 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:16:28.947 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:16:28.947 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:16:28.947 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:16:28.947 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:16:29.034 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:16:29.035 INFO:tasks.workunit.client.0.smithi100.stdout:finished writesame 2024-04-07T17:16:29.035 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:16:29.035 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:16:29.035 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:16:29.035 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:16:29.035 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:16:29.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:16:28 smithi144 bash[66654]: cluster 2024-04-07T17:16:27.736467+0000 mgr.y (mgr.25175) 1833 : cluster [DBG] pgmap v3075: 489 pgs: 489 active+clean; 249 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 6.5 MiB/s rd, 6.3 MiB/s wr, 533 op/s 2024-04-07T17:16:29.118 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:16:28 smithi144 bash[66654]: cluster 2024-04-07T17:16:27.852007+0000 mon.a (mon.0) 3219 : cluster [DBG] osdmap e1668: 8 total, 8 up, 8 in 2024-04-07T17:16:29.162 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:16:29.162 INFO:tasks.workunit.client.0.smithi100.stdout:finished writesame 2024-04-07T17:16:29.162 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:16:29.162 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:16:29.162 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:16:29.162 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:16:29.162 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:16:29.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:16:28 smithi100 bash[74942]: cluster 2024-04-07T17:16:27.736467+0000 mgr.y (mgr.25175) 1833 : cluster [DBG] pgmap v3075: 489 pgs: 489 active+clean; 249 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 6.5 MiB/s rd, 6.3 MiB/s wr, 533 op/s 2024-04-07T17:16:29.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:16:28 smithi100 bash[74942]: cluster 2024-04-07T17:16:27.852007+0000 mon.a (mon.0) 3219 : cluster [DBG] osdmap e1668: 8 total, 8 up, 8 in 2024-04-07T17:16:29.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:16:28 smithi100 bash[76202]: cluster 2024-04-07T17:16:27.736467+0000 mgr.y (mgr.25175) 1833 : cluster [DBG] pgmap v3075: 489 pgs: 489 active+clean; 249 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 6.5 MiB/s rd, 6.3 MiB/s wr, 533 op/s 2024-04-07T17:16:29.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:16:28 smithi100 bash[76202]: cluster 2024-04-07T17:16:27.852007+0000 mon.a (mon.0) 3219 : cluster [DBG] osdmap e1668: 8 total, 8 up, 8 in 2024-04-07T17:16:29.282 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:16:29.282 INFO:tasks.workunit.client.0.smithi100.stdout:finished writesame 2024-04-07T17:16:29.282 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:16:29.282 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:16:29.282 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:16:29.282 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:16:29.282 INFO:tasks.workunit.client.0.smithi100.stdout:expected fail, finished writesame 2024-04-07T17:16:29.282 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:16:29.283 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:16:29.283 INFO:tasks.workunit.client.0.smithi100.stdout:expected fail, finished writesame 2024-04-07T17:16:29.283 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:16:29.283 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:16:29.283 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:16:29.455 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:16:29.455 INFO:tasks.workunit.client.0.smithi100.stdout:finished writesame 2024-04-07T17:16:29.455 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:16:29.456 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:16:29.456 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:16:29.456 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:16:29.456 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:16:29.649 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:16:29.649 INFO:tasks.workunit.client.0.smithi100.stdout:finished writesame 2024-04-07T17:16:29.649 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:16:29.649 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:16:29.649 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:16:29.650 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:16:29.650 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:16:29.873 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:16:29.873 INFO:tasks.workunit.client.0.smithi100.stdout:finished writesame 2024-04-07T17:16:29.874 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:16:29.874 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:16:29.874 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:16:29.874 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:16:29.874 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:16:30.080 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:16:30.080 INFO:tasks.workunit.client.0.smithi100.stdout:finished writesame 2024-04-07T17:16:30.080 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:16:30.080 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:16:30.080 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:16:30.080 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:16:30.080 INFO:tasks.workunit.client.0.smithi100.stdout:expected fail, finished writesame 2024-04-07T17:16:30.080 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:16:30.080 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:16:30.080 INFO:tasks.workunit.client.0.smithi100.stdout:expected fail, finished writesame 2024-04-07T17:16:30.080 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:16:30.080 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:16:30.080 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:16:30.360 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:16:30.360 INFO:tasks.workunit.client.0.smithi100.stdout:finished writesame 2024-04-07T17:16:30.360 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:16:30.360 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:16:30.360 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:16:30.360 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:16:30.361 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:16:30.609 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:16:30.609 INFO:tasks.workunit.client.0.smithi100.stdout:finished writesame 2024-04-07T17:16:30.609 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:16:30.609 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:16:30.609 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:16:30.609 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:16:30.609 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:16:30.893 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:16:30.893 INFO:tasks.workunit.client.0.smithi100.stdout:finished writesame 2024-04-07T17:16:30.893 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:16:30.893 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:16:30.893 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:16:30.893 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:16:30.893 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:16:31.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:16:30 smithi144 bash[66654]: cluster 2024-04-07T17:16:29.738816+0000 mgr.y (mgr.25175) 1834 : cluster [DBG] pgmap v3077: 489 pgs: 489 active+clean; 253 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 7.3 MiB/s rd, 7.1 MiB/s wr, 565 op/s 2024-04-07T17:16:31.197 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:16:31.197 INFO:tasks.workunit.client.0.smithi100.stdout:finished writesame 2024-04-07T17:16:31.198 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:16:31.198 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:16:31.198 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:16:31.198 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:16:31.198 INFO:tasks.workunit.client.0.smithi100.stdout:expected fail, finished writesame 2024-04-07T17:16:31.198 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:16:31.198 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:16:31.198 INFO:tasks.workunit.client.0.smithi100.stdout:expected fail, finished writesame 2024-04-07T17:16:31.198 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:16:31.198 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:16:31.198 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:16:31.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:16:30 smithi100 bash[74942]: cluster 2024-04-07T17:16:29.738816+0000 mgr.y (mgr.25175) 1834 : cluster [DBG] pgmap v3077: 489 pgs: 489 active+clean; 253 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 7.3 MiB/s rd, 7.1 MiB/s wr, 565 op/s 2024-04-07T17:16:31.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:16:30 smithi100 bash[76202]: cluster 2024-04-07T17:16:29.738816+0000 mgr.y (mgr.25175) 1834 : cluster [DBG] pgmap v3077: 489 pgs: 489 active+clean; 253 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 7.3 MiB/s rd, 7.1 MiB/s wr, 565 op/s 2024-04-07T17:16:31.635 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:16:31.636 INFO:tasks.workunit.client.0.smithi100.stdout:finished writesame 2024-04-07T17:16:31.636 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:16:31.636 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:16:31.636 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:16:31.636 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:16:31.636 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:16:31.985 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:16:31.985 INFO:tasks.workunit.client.0.smithi100.stdout:finished writesame 2024-04-07T17:16:31.985 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:16:31.985 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:16:31.985 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:16:31.985 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:16:31.985 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:16:32.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:16:31 smithi144 bash[66654]: audit 2024-04-07T17:16:31.826533+0000 mon.a (mon.0) 3220 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:16:32.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:16:31 smithi100 bash[74942]: audit 2024-04-07T17:16:31.826533+0000 mon.a (mon.0) 3220 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:16:32.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:16:31 smithi100 bash[76202]: audit 2024-04-07T17:16:31.826533+0000 mon.a (mon.0) 3220 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:16:32.394 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:16:32.394 INFO:tasks.workunit.client.0.smithi100.stdout:finished writesame 2024-04-07T17:16:32.394 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:16:32.395 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:16:32.395 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:16:32.395 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:16:32.395 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:16:32.617 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:16:32 smithi144 bash[95233]: ts=2024-04-07T17:16:32.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:16:32.801 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:16:32.801 INFO:tasks.workunit.client.0.smithi100.stdout:finished writesame 2024-04-07T17:16:32.801 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:16:32.801 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:16:32.801 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:16:32.801 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:16:32.801 INFO:tasks.workunit.client.0.smithi100.stdout:expected fail, finished writesame 2024-04-07T17:16:32.801 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:16:32.801 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:16:32.801 INFO:tasks.workunit.client.0.smithi100.stdout:expected fail, finished writesame 2024-04-07T17:16:32.802 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.TestIOPP (8107 ms) 2024-04-07T17:16:32.802 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.TestCompareAndWriteCompareTooSmallPP 2024-04-07T17:16:32.802 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:16:32.872 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.TestCompareAndWriteCompareTooSmallPP (71 ms) 2024-04-07T17:16:32.872 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.TestAioCompareAndWriteCompareTooSmallPP 2024-04-07T17:16:32.872 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:16:32.943 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.TestAioCompareAndWriteCompareTooSmallPP (71 ms) 2024-04-07T17:16:32.943 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.TestCompareAndWriteWriteTooSmallPP 2024-04-07T17:16:32.943 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:16:33.008 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.TestCompareAndWriteWriteTooSmallPP (65 ms) 2024-04-07T17:16:33.008 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.TestAioCompareAndWriteWriteTooSmallPP 2024-04-07T17:16:33.008 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:16:33.076 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.TestAioCompareAndWriteWriteTooSmallPP (68 ms) 2024-04-07T17:16:33.077 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.TestCompareAndWriteMismatchPP 2024-04-07T17:16:33.077 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:16:33.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:16:32 smithi144 bash[66654]: cluster 2024-04-07T17:16:31.740420+0000 mgr.y (mgr.25175) 1835 : cluster [DBG] pgmap v3078: 489 pgs: 489 active+clean; 253 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 6.0 MiB/s rd, 5.8 MiB/s wr, 384 op/s 2024-04-07T17:16:33.150 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.TestCompareAndWriteMismatchPP (73 ms) 2024-04-07T17:16:33.150 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.TestAioCompareAndWriteMismatchPP 2024-04-07T17:16:33.150 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:16:33.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:16:32 smithi100 bash[74942]: cluster 2024-04-07T17:16:31.740420+0000 mgr.y (mgr.25175) 1835 : cluster [DBG] pgmap v3078: 489 pgs: 489 active+clean; 253 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 6.0 MiB/s rd, 5.8 MiB/s wr, 384 op/s 2024-04-07T17:16:33.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:16:32 smithi100 bash[76202]: cluster 2024-04-07T17:16:31.740420+0000 mgr.y (mgr.25175) 1835 : cluster [DBG] pgmap v3078: 489 pgs: 489 active+clean; 253 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 6.0 MiB/s rd, 5.8 MiB/s wr, 384 op/s 2024-04-07T17:16:33.210 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:16:33.220 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.TestAioCompareAndWriteMismatchPP (70 ms) 2024-04-07T17:16:33.220 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.TestCompareAndWriteMismatchBufferlistGreaterLenPP 2024-04-07T17:16:33.220 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:16:33.292 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.TestCompareAndWriteMismatchBufferlistGreaterLenPP (72 ms) 2024-04-07T17:16:33.292 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.TestAioCompareAndWriteMismatchBufferlistGreaterLenPP 2024-04-07T17:16:33.292 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:16:33.353 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:16:33.363 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.TestAioCompareAndWriteMismatchBufferlistGreaterLenPP (71 ms) 2024-04-07T17:16:33.363 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.TestCompareAndWriteSuccessPP 2024-04-07T17:16:33.363 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:16:33.434 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.TestCompareAndWriteSuccessPP (71 ms) 2024-04-07T17:16:33.434 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.TestAioCompareAndWriteSuccessPP 2024-04-07T17:16:33.434 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:16:33.499 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:16:33.509 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.TestAioCompareAndWriteSuccessPP (76 ms) 2024-04-07T17:16:33.509 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.TestCompareAndWriteSuccessBufferlistGreaterLenPP 2024-04-07T17:16:33.510 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:16:33.581 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.TestCompareAndWriteSuccessBufferlistGreaterLenPP (71 ms) 2024-04-07T17:16:33.581 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.TestAioCompareAndWriteSuccessBufferlistGreaterLenPP 2024-04-07T17:16:33.581 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:16:33.644 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:16:33.655 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.TestAioCompareAndWriteSuccessBufferlistGreaterLenPP (74 ms) 2024-04-07T17:16:33.655 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.TestCompareAndWriteStripeUnitUnalignedPP 2024-04-07T17:16:33.655 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:16:33.805 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.TestCompareAndWriteStripeUnitUnalignedPP (150 ms) 2024-04-07T17:16:33.805 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.TestAioCompareAndWriteStripeUnitUnalignedPP 2024-04-07T17:16:33.805 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:16:33.900 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:16:33.932 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.TestAioCompareAndWriteStripeUnitUnalignedPP (127 ms) 2024-04-07T17:16:33.933 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.TestCompareAndWriteTooLargePP 2024-04-07T17:16:33.933 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:16:34.082 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.TestCompareAndWriteTooLargePP (150 ms) 2024-04-07T17:16:34.083 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.TestAioCompareAndWriteTooLargePP 2024-04-07T17:16:34.083 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:16:34.203 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:16:34.256 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.TestAioCompareAndWriteTooLargePP (174 ms) 2024-04-07T17:16:34.256 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.TestCompareAndWriteStripeUnitSuccessPP 2024-04-07T17:16:34.256 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:16:34.809 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.TestCompareAndWriteStripeUnitSuccessPP (553 ms) 2024-04-07T17:16:34.809 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.TestAioCompareAndWriteStripeUnitSuccessPP 2024-04-07T17:16:34.810 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:16:35.157 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:16:35.183 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.TestAioCompareAndWriteStripeUnitSuccessPP (374 ms) 2024-04-07T17:16:35.183 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.TestIOPPWithIOHint 2024-04-07T17:16:35.183 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:16:35.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:16:34 smithi100 bash[74942]: cluster 2024-04-07T17:16:33.743614+0000 mgr.y (mgr.25175) 1836 : cluster [DBG] pgmap v3079: 489 pgs: 489 active+clean; 253 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 5.6 MiB/s rd, 5.4 MiB/s wr, 888 op/s 2024-04-07T17:16:35.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:16:34 smithi100 bash[76202]: cluster 2024-04-07T17:16:33.743614+0000 mgr.y (mgr.25175) 1836 : cluster [DBG] pgmap v3079: 489 pgs: 489 active+clean; 253 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 5.6 MiB/s rd, 5.4 MiB/s wr, 888 op/s 2024-04-07T17:16:35.258 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 512 2024-04-07T17:16:35.259 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 512 2024-04-07T17:16:35.259 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 512 2024-04-07T17:16:35.259 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 512 2024-04-07T17:16:35.259 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 512 2024-04-07T17:16:35.259 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:16:35.259 INFO:tasks.workunit.client.0.smithi100.stdout:started write 2024-04-07T17:16:35.259 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:16:35.261 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:16:35.261 INFO:tasks.workunit.client.0.smithi100.stdout:finished write 2024-04-07T17:16:35.261 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:16:35.261 INFO:tasks.workunit.client.0.smithi100.stdout:started write 2024-04-07T17:16:35.261 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:16:35.263 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:16:35.263 INFO:tasks.workunit.client.0.smithi100.stdout:finished write 2024-04-07T17:16:35.263 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:16:35.263 INFO:tasks.workunit.client.0.smithi100.stdout:started write 2024-04-07T17:16:35.263 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:16:35.266 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:16:35.266 INFO:tasks.workunit.client.0.smithi100.stdout:finished write 2024-04-07T17:16:35.266 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:16:35.266 INFO:tasks.workunit.client.0.smithi100.stdout:started write 2024-04-07T17:16:35.266 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:16:35.269 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:16:35.269 INFO:tasks.workunit.client.0.smithi100.stdout:finished write 2024-04-07T17:16:35.269 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:16:35.270 INFO:tasks.workunit.client.0.smithi100.stdout:started write 2024-04-07T17:16:35.270 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:16:35.271 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:16:35.272 INFO:tasks.workunit.client.0.smithi100.stdout:finished write 2024-04-07T17:16:35.272 INFO:tasks.workunit.client.0.smithi100.stdout:read: 512 2024-04-07T17:16:35.272 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:16:35.272 INFO:tasks.workunit.client.0.smithi100.stdout:started read 2024-04-07T17:16:35.272 INFO:tasks.workunit.client.0.smithi100.stdout:read completion cb called! 2024-04-07T17:16:35.272 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 512 2024-04-07T17:16:35.272 INFO:tasks.workunit.client.0.smithi100.stdout:finished read 2024-04-07T17:16:35.273 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:16:35.273 INFO:tasks.workunit.client.0.smithi100.stdout:started read 2024-04-07T17:16:35.273 INFO:tasks.workunit.client.0.smithi100.stdout:read completion cb called! 2024-04-07T17:16:35.274 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 512 2024-04-07T17:16:35.274 INFO:tasks.workunit.client.0.smithi100.stdout:finished read 2024-04-07T17:16:35.274 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:16:35.274 INFO:tasks.workunit.client.0.smithi100.stdout:started read 2024-04-07T17:16:35.274 INFO:tasks.workunit.client.0.smithi100.stdout:read completion cb called! 2024-04-07T17:16:35.275 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 512 2024-04-07T17:16:35.275 INFO:tasks.workunit.client.0.smithi100.stdout:finished read 2024-04-07T17:16:35.275 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:16:35.275 INFO:tasks.workunit.client.0.smithi100.stdout:started read 2024-04-07T17:16:35.275 INFO:tasks.workunit.client.0.smithi100.stdout:read completion cb called! 2024-04-07T17:16:35.276 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 512 2024-04-07T17:16:35.276 INFO:tasks.workunit.client.0.smithi100.stdout:finished read 2024-04-07T17:16:35.276 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:16:35.276 INFO:tasks.workunit.client.0.smithi100.stdout:started read 2024-04-07T17:16:35.276 INFO:tasks.workunit.client.0.smithi100.stdout:read completion cb called! 2024-04-07T17:16:35.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:16:34 smithi144 bash[66654]: cluster 2024-04-07T17:16:33.743614+0000 mgr.y (mgr.25175) 1836 : cluster [DBG] pgmap v3079: 489 pgs: 489 active+clean; 253 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 5.6 MiB/s rd, 5.4 MiB/s wr, 888 op/s 2024-04-07T17:16:37.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:16:36 smithi100 bash[74942]: cluster 2024-04-07T17:16:35.746022+0000 mgr.y (mgr.25175) 1837 : cluster [DBG] pgmap v3080: 489 pgs: 489 active+clean; 253 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 4.1 MiB/s rd, 3.8 MiB/s wr, 918 op/s 2024-04-07T17:16:37.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:16:36 smithi100 bash[76202]: cluster 2024-04-07T17:16:35.746022+0000 mgr.y (mgr.25175) 1837 : cluster [DBG] pgmap v3080: 489 pgs: 489 active+clean; 253 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 4.1 MiB/s rd, 3.8 MiB/s wr, 918 op/s 2024-04-07T17:16:37.368 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:16:36 smithi144 bash[66654]: cluster 2024-04-07T17:16:35.746022+0000 mgr.y (mgr.25175) 1837 : cluster [DBG] pgmap v3080: 489 pgs: 489 active+clean; 253 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 4.1 MiB/s rd, 3.8 MiB/s wr, 918 op/s 2024-04-07T17:16:38.704 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:16:38 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:16:38] "GET /metrics HTTP/1.1" 200 63248 "" "Prometheus/2.43.0" 2024-04-07T17:16:39.200 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 512 2024-04-07T17:16:39.200 INFO:tasks.workunit.client.0.smithi100.stdout:finished read 2024-04-07T17:16:39.201 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 0 2024-04-07T17:16:39.201 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:16:39.201 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:16:39.201 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 0 2024-04-07T17:16:39.201 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:16:39.201 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:16:39.201 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 16384 2024-04-07T17:16:39.201 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:16:39.201 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:16:39.201 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 16384 2024-04-07T17:16:39.201 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:16:39.201 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:16:39.201 INFO:tasks.workunit.client.0.smithi100.stdout:expected fail, finished writesame 2024-04-07T17:16:39.201 INFO:tasks.workunit.client.0.smithi100.stdout:expected fail, finished writesame 2024-04-07T17:16:39.201 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 49152 2024-04-07T17:16:39.202 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:16:39.202 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:16:39.202 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 49152 2024-04-07T17:16:39.202 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:16:39.202 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:16:39.202 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 65536 2024-04-07T17:16:39.202 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:16:39.202 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:16:39.202 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 65536 2024-04-07T17:16:39.202 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:16:39.202 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:16:39.202 INFO:tasks.workunit.client.0.smithi100.stdout:expected fail, finished writesame 2024-04-07T17:16:39.202 INFO:tasks.workunit.client.0.smithi100.stdout:expected fail, finished writesame 2024-04-07T17:16:39.202 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 98304 2024-04-07T17:16:39.202 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:16:39.203 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:16:39.203 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 98304 2024-04-07T17:16:39.203 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:16:39.203 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:16:39.203 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 114688 2024-04-07T17:16:39.203 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:16:39.203 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:16:39.203 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 114688 2024-04-07T17:16:39.203 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:16:39.203 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:16:39.203 INFO:tasks.workunit.client.0.smithi100.stdout:expected fail, finished writesame 2024-04-07T17:16:39.203 INFO:tasks.workunit.client.0.smithi100.stdout:expected fail, finished writesame 2024-04-07T17:16:39.203 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 147456 2024-04-07T17:16:39.203 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:16:39.203 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:16:39.204 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 147456 2024-04-07T17:16:39.204 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:16:39.204 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:16:39.204 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 163840 2024-04-07T17:16:39.204 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:16:39.204 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:16:39.204 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 163840 2024-04-07T17:16:39.204 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:16:39.204 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:16:39.204 INFO:tasks.workunit.client.0.smithi100.stdout:expected fail, finished writesame 2024-04-07T17:16:39.204 INFO:tasks.workunit.client.0.smithi100.stdout:expected fail, finished writesame 2024-04-07T17:16:39.204 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 196608 2024-04-07T17:16:39.205 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:16:39.205 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:16:39.205 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 196608 2024-04-07T17:16:39.205 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:16:39.205 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:16:39.205 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 212992 2024-04-07T17:16:39.205 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:16:39.205 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:16:39.205 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 212992 2024-04-07T17:16:39.205 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:16:39.206 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:16:39.206 INFO:tasks.workunit.client.0.smithi100.stdout:expected fail, finished writesame 2024-04-07T17:16:39.206 INFO:tasks.workunit.client.0.smithi100.stdout:expected fail, finished writesame 2024-04-07T17:16:39.206 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:16:39.206 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:16:39.206 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:16:39.206 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:16:39.206 INFO:tasks.workunit.client.0.smithi100.stdout:finished writesame 2024-04-07T17:16:39.206 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:16:39.206 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:16:39.206 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:16:39.206 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:16:39.206 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:16:39.207 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:16:39.207 INFO:tasks.workunit.client.0.smithi100.stdout:finished writesame 2024-04-07T17:16:39.207 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:16:39.207 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:16:39.207 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:16:39.207 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:16:39.207 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:16:39.208 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:16:38 smithi100 bash[74942]: cluster 2024-04-07T17:16:37.748903+0000 mgr.y (mgr.25175) 1838 : cluster [DBG] pgmap v3081: 489 pgs: 489 active+clean; 289 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 4.9 MiB/s rd, 4.6 MiB/s wr, 932 op/s 2024-04-07T17:16:39.208 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:16:38 smithi100 bash[76202]: cluster 2024-04-07T17:16:37.748903+0000 mgr.y (mgr.25175) 1838 : cluster [DBG] pgmap v3081: 489 pgs: 489 active+clean; 289 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 4.9 MiB/s rd, 4.6 MiB/s wr, 932 op/s 2024-04-07T17:16:39.234 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:16:39.234 INFO:tasks.workunit.client.0.smithi100.stdout:finished writesame 2024-04-07T17:16:39.234 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:16:39.235 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:16:39.235 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:16:39.235 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:16:39.235 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:16:39.265 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:16:39.265 INFO:tasks.workunit.client.0.smithi100.stdout:finished writesame 2024-04-07T17:16:39.265 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:16:39.265 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:16:39.265 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:16:39.265 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:16:39.265 INFO:tasks.workunit.client.0.smithi100.stdout:expected fail, finished writesame 2024-04-07T17:16:39.265 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:16:39.265 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:16:39.265 INFO:tasks.workunit.client.0.smithi100.stdout:expected fail, finished writesame 2024-04-07T17:16:39.265 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:16:39.266 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:16:39.266 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:16:39.358 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:16:39.358 INFO:tasks.workunit.client.0.smithi100.stdout:finished writesame 2024-04-07T17:16:39.358 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:16:39.358 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:16:39.358 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:16:39.358 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:16:39.358 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:16:39.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:16:38 smithi144 bash[66654]: cluster 2024-04-07T17:16:37.748903+0000 mgr.y (mgr.25175) 1838 : cluster [DBG] pgmap v3081: 489 pgs: 489 active+clean; 289 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 4.9 MiB/s rd, 4.6 MiB/s wr, 932 op/s 2024-04-07T17:16:39.449 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:16:39.449 INFO:tasks.workunit.client.0.smithi100.stdout:finished writesame 2024-04-07T17:16:39.449 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:16:39.449 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:16:39.449 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:16:39.450 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:16:39.450 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:16:39.577 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:16:39.577 INFO:tasks.workunit.client.0.smithi100.stdout:finished writesame 2024-04-07T17:16:39.577 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:16:39.577 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:16:39.578 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:16:39.578 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:16:39.578 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:16:39.708 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:16:39.708 INFO:tasks.workunit.client.0.smithi100.stdout:finished writesame 2024-04-07T17:16:39.708 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:16:39.709 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:16:39.709 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:16:39.709 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:16:39.709 INFO:tasks.workunit.client.0.smithi100.stdout:expected fail, finished writesame 2024-04-07T17:16:39.709 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:16:39.709 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:16:39.709 INFO:tasks.workunit.client.0.smithi100.stdout:expected fail, finished writesame 2024-04-07T17:16:39.709 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:16:39.709 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:16:39.709 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:16:39.905 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:16:39.906 INFO:tasks.workunit.client.0.smithi100.stdout:finished writesame 2024-04-07T17:16:39.906 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:16:39.906 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:16:39.906 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:16:39.906 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:16:39.906 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:16:40.083 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:16:40.083 INFO:tasks.workunit.client.0.smithi100.stdout:finished writesame 2024-04-07T17:16:40.083 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:16:40.083 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:16:40.083 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:16:40.084 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:16:40.084 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:16:40.296 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:16:40.296 INFO:tasks.workunit.client.0.smithi100.stdout:finished writesame 2024-04-07T17:16:40.296 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:16:40.296 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:16:40.296 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:16:40.296 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:16:40.296 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:16:40.502 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:16:40.502 INFO:tasks.workunit.client.0.smithi100.stdout:finished writesame 2024-04-07T17:16:40.502 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:16:40.502 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:16:40.502 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:16:40.503 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:16:40.503 INFO:tasks.workunit.client.0.smithi100.stdout:expected fail, finished writesame 2024-04-07T17:16:40.503 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:16:40.503 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:16:40.503 INFO:tasks.workunit.client.0.smithi100.stdout:expected fail, finished writesame 2024-04-07T17:16:40.503 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:16:40.503 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:16:40.503 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:16:40.784 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:16:40.784 INFO:tasks.workunit.client.0.smithi100.stdout:finished writesame 2024-04-07T17:16:40.784 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:16:40.784 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:16:40.784 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:16:40.784 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:16:40.784 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:16:41.053 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:16:41.054 INFO:tasks.workunit.client.0.smithi100.stdout:finished writesame 2024-04-07T17:16:41.054 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:16:41.054 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:16:41.054 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:16:41.054 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:16:41.054 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:16:41.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:16:40 smithi100 bash[74942]: cluster 2024-04-07T17:16:39.751271+0000 mgr.y (mgr.25175) 1839 : cluster [DBG] pgmap v3082: 489 pgs: 489 active+clean; 301 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 5.8 MiB/s rd, 5.5 MiB/s wr, 804 op/s 2024-04-07T17:16:41.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:16:40 smithi100 bash[76202]: cluster 2024-04-07T17:16:39.751271+0000 mgr.y (mgr.25175) 1839 : cluster [DBG] pgmap v3082: 489 pgs: 489 active+clean; 301 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 5.8 MiB/s rd, 5.5 MiB/s wr, 804 op/s 2024-04-07T17:16:41.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:16:40 smithi144 bash[66654]: cluster 2024-04-07T17:16:39.751271+0000 mgr.y (mgr.25175) 1839 : cluster [DBG] pgmap v3082: 489 pgs: 489 active+clean; 301 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 5.8 MiB/s rd, 5.5 MiB/s wr, 804 op/s 2024-04-07T17:16:41.374 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:16:41.374 INFO:tasks.workunit.client.0.smithi100.stdout:finished writesame 2024-04-07T17:16:41.374 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:16:41.374 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:16:41.374 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:16:41.374 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:16:41.374 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:16:41.706 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:16:41.706 INFO:tasks.workunit.client.0.smithi100.stdout:finished writesame 2024-04-07T17:16:41.706 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:16:41.706 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:16:41.706 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:16:41.706 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:16:41.707 INFO:tasks.workunit.client.0.smithi100.stdout:expected fail, finished writesame 2024-04-07T17:16:41.707 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:16:41.707 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:16:41.707 INFO:tasks.workunit.client.0.smithi100.stdout:expected fail, finished writesame 2024-04-07T17:16:41.707 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:16:41.707 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:16:41.707 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:16:42.086 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:16:42.086 INFO:tasks.workunit.client.0.smithi100.stdout:finished writesame 2024-04-07T17:16:42.086 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:16:42.086 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:16:42.087 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:16:42.087 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:16:42.087 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:16:42.432 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:16:42.433 INFO:tasks.workunit.client.0.smithi100.stdout:finished writesame 2024-04-07T17:16:42.433 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:16:42.433 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:16:42.433 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:16:42.433 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:16:42.433 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:16:42.617 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:16:42 smithi144 bash[95233]: ts=2024-04-07T17:16:42.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:16:42.837 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:16:42.838 INFO:tasks.workunit.client.0.smithi100.stdout:finished writesame 2024-04-07T17:16:42.838 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:16:42.838 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:16:42.838 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:16:42.838 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:16:42.838 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:16:43.202 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:16:43.202 INFO:tasks.workunit.client.0.smithi100.stdout:finished writesame 2024-04-07T17:16:43.202 INFO:tasks.workunit.client.0.smithi100.stdout:to verify the data 2024-04-07T17:16:43.202 INFO:tasks.workunit.client.0.smithi100.stdout:verified 2024-04-07T17:16:43.202 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:16:43.202 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:16:43.202 INFO:tasks.workunit.client.0.smithi100.stdout:expected fail, finished writesame 2024-04-07T17:16:43.202 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:16:43.202 INFO:tasks.workunit.client.0.smithi100.stdout:started writesame 2024-04-07T17:16:43.202 INFO:tasks.workunit.client.0.smithi100.stdout:expected fail, finished writesame 2024-04-07T17:16:43.202 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.TestIOPPWithIOHint (8018 ms) 2024-04-07T17:16:43.203 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.TestIOToSnapshot 2024-04-07T17:16:43.203 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:16:43.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:16:42 smithi100 bash[74942]: cluster 2024-04-07T17:16:41.752815+0000 mgr.y (mgr.25175) 1840 : cluster [DBG] pgmap v3083: 489 pgs: 489 active+clean; 301 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 5.0 MiB/s rd, 4.8 MiB/s wr, 764 op/s 2024-04-07T17:16:43.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:16:42 smithi100 bash[76202]: cluster 2024-04-07T17:16:41.752815+0000 mgr.y (mgr.25175) 1840 : cluster [DBG] pgmap v3083: 489 pgs: 489 active+clean; 301 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 5.0 MiB/s rd, 4.8 MiB/s wr, 764 op/s 2024-04-07T17:16:43.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:16:42 smithi144 bash[66654]: cluster 2024-04-07T17:16:41.752815+0000 mgr.y (mgr.25175) 1840 : cluster [DBG] pgmap v3083: 489 pgs: 489 active+clean; 301 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 5.0 MiB/s rd, 4.8 MiB/s wr, 764 op/s 2024-04-07T17:16:45.001 INFO:tasks.workunit.client.0.smithi100.stdout:num snaps is: 0 2024-04-07T17:16:45.001 INFO:tasks.workunit.client.0.smithi100.stdout:expected: 0 2024-04-07T17:16:45.001 INFO:tasks.workunit.client.0.smithi100.stdout:num snaps is: 1 2024-04-07T17:16:45.001 INFO:tasks.workunit.client.0.smithi100.stdout:expected: 1 2024-04-07T17:16:45.001 INFO:tasks.workunit.client.0.smithi100.stdout:snap: orig 2024-04-07T17:16:45.001 INFO:tasks.workunit.client.0.smithi100.stdout:found orig with size 2097152 2024-04-07T17:16:45.001 INFO:tasks.workunit.client.0.smithi100.stdout:read: 80 2024-04-07T17:16:45.001 INFO:tasks.workunit.client.0.smithi100.stdout:write test data! 2024-04-07T17:16:45.001 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 80 2024-04-07T17:16:45.001 INFO:tasks.workunit.client.0.smithi100.stdout:num snaps is: 2 2024-04-07T17:16:45.001 INFO:tasks.workunit.client.0.smithi100.stdout:expected: 2 2024-04-07T17:16:45.002 INFO:tasks.workunit.client.0.smithi100.stdout:snap: orig 2024-04-07T17:16:45.002 INFO:tasks.workunit.client.0.smithi100.stdout:snap: written 2024-04-07T17:16:45.002 INFO:tasks.workunit.client.0.smithi100.stdout:found orig with size 2097152 2024-04-07T17:16:45.002 INFO:tasks.workunit.client.0.smithi100.stdout:found written with size 2097152 2024-04-07T17:16:45.002 INFO:tasks.workunit.client.0.smithi100.stdout:read: 80 2024-04-07T17:16:45.002 INFO:tasks.workunit.client.0.smithi100.stdout:read: 80 2024-04-07T17:16:45.002 INFO:tasks.workunit.client.0.smithi100.stdout:read: 80 2024-04-07T17:16:45.002 INFO:tasks.workunit.client.0.smithi100.stdout:write to snapshot returned -30 2024-04-07T17:16:45.002 INFO:tasks.workunit.client.0.smithi100.stdout:Read-only file system 2024-04-07T17:16:45.077 INFO:tasks.workunit.client.0.smithi100.stdout:read: 80 2024-04-07T17:16:45.077 INFO:tasks.workunit.client.0.smithi100.stdout:read: 80 2024-04-07T17:16:45.077 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 80 2024-04-07T17:16:45.077 INFO:tasks.workunit.client.0.smithi100.stdout:opening testimg@orig 2024-04-07T17:16:45.077 INFO:tasks.workunit.client.0.smithi100.stdout:read: 80 2024-04-07T17:16:45.077 INFO:tasks.workunit.client.0.smithi100.stdout:write to snapshot returned -30 2024-04-07T17:16:45.077 INFO:tasks.workunit.client.0.smithi100.stdout:Read-only file system 2024-04-07T17:16:45.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:16:44 smithi100 bash[74942]: cluster 2024-04-07T17:16:43.756122+0000 mgr.y (mgr.25175) 1841 : cluster [DBG] pgmap v3084: 489 pgs: 489 active+clean; 301 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 5.3 MiB/s rd, 4.9 MiB/s wr, 1.20k op/s 2024-04-07T17:16:45.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:16:44 smithi100 bash[74942]: cluster 2024-04-07T17:16:43.902580+0000 mon.a (mon.0) 3221 : cluster [DBG] osdmap e1669: 8 total, 8 up, 8 in 2024-04-07T17:16:45.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:16:44 smithi100 bash[76202]: cluster 2024-04-07T17:16:43.756122+0000 mgr.y (mgr.25175) 1841 : cluster [DBG] pgmap v3084: 489 pgs: 489 active+clean; 301 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 5.3 MiB/s rd, 4.9 MiB/s wr, 1.20k op/s 2024-04-07T17:16:45.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:16:44 smithi100 bash[76202]: cluster 2024-04-07T17:16:43.902580+0000 mon.a (mon.0) 3221 : cluster [DBG] osdmap e1669: 8 total, 8 up, 8 in 2024-04-07T17:16:45.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:16:44 smithi144 bash[66654]: cluster 2024-04-07T17:16:43.756122+0000 mgr.y (mgr.25175) 1841 : cluster [DBG] pgmap v3084: 489 pgs: 489 active+clean; 301 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 5.3 MiB/s rd, 4.9 MiB/s wr, 1.20k op/s 2024-04-07T17:16:45.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:16:44 smithi144 bash[66654]: cluster 2024-04-07T17:16:43.902580+0000 mon.a (mon.0) 3221 : cluster [DBG] osdmap e1669: 8 total, 8 up, 8 in 2024-04-07T17:16:46.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:16:45 smithi100 bash[74942]: cluster 2024-04-07T17:16:44.905992+0000 mon.a (mon.0) 3222 : cluster [DBG] osdmap e1670: 8 total, 8 up, 8 in 2024-04-07T17:16:46.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:16:45 smithi100 bash[76202]: cluster 2024-04-07T17:16:44.905992+0000 mon.a (mon.0) 3222 : cluster [DBG] osdmap e1670: 8 total, 8 up, 8 in 2024-04-07T17:16:46.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:16:45 smithi144 bash[66654]: cluster 2024-04-07T17:16:44.905992+0000 mon.a (mon.0) 3222 : cluster [DBG] osdmap e1670: 8 total, 8 up, 8 in 2024-04-07T17:16:47.038 INFO:tasks.workunit.client.0.smithi100.stdout:num snaps is: 2 2024-04-07T17:16:47.038 INFO:tasks.workunit.client.0.smithi100.stdout:expected: 2 2024-04-07T17:16:47.038 INFO:tasks.workunit.client.0.smithi100.stdout:snap: orig 2024-04-07T17:16:47.038 INFO:tasks.workunit.client.0.smithi100.stdout:snap: written 2024-04-07T17:16:47.038 INFO:tasks.workunit.client.0.smithi100.stdout:found orig with size 2097152 2024-04-07T17:16:47.038 INFO:tasks.workunit.client.0.smithi100.stdout:found written with size 2097152 2024-04-07T17:16:47.039 INFO:tasks.workunit.client.0.smithi100.stdout:num snaps is: 1 2024-04-07T17:16:47.039 INFO:tasks.workunit.client.0.smithi100.stdout:expected: 1 2024-04-07T17:16:47.039 INFO:tasks.workunit.client.0.smithi100.stdout:snap: orig 2024-04-07T17:16:47.039 INFO:tasks.workunit.client.0.smithi100.stdout:found orig with size 2097152 2024-04-07T17:16:47.039 INFO:tasks.workunit.client.0.smithi100.stdout:num snaps is: 0 2024-04-07T17:16:47.039 INFO:tasks.workunit.client.0.smithi100.stdout:expected: 0 2024-04-07T17:16:47.039 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.TestIOToSnapshot (3837 ms) 2024-04-07T17:16:47.039 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.TestSnapshotDeletedIo 2024-04-07T17:16:47.039 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:16:47.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:16:46 smithi100 bash[74942]: cluster 2024-04-07T17:16:45.757661+0000 mgr.y (mgr.25175) 1842 : cluster [DBG] pgmap v3087: 489 pgs: 489 active+clean; 301 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 2.9 MiB/s rd, 2.7 MiB/s wr, 685 op/s 2024-04-07T17:16:47.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:16:46 smithi100 bash[74942]: cluster 2024-04-07T17:16:45.930909+0000 mon.a (mon.0) 3223 : cluster [DBG] osdmap e1671: 8 total, 8 up, 8 in 2024-04-07T17:16:47.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:16:46 smithi100 bash[74942]: audit 2024-04-07T17:16:46.827668+0000 mon.a (mon.0) 3224 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:16:47.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:16:46 smithi100 bash[76202]: cluster 2024-04-07T17:16:45.757661+0000 mgr.y (mgr.25175) 1842 : cluster [DBG] pgmap v3087: 489 pgs: 489 active+clean; 301 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 2.9 MiB/s rd, 2.7 MiB/s wr, 685 op/s 2024-04-07T17:16:47.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:16:46 smithi100 bash[76202]: cluster 2024-04-07T17:16:45.930909+0000 mon.a (mon.0) 3223 : cluster [DBG] osdmap e1671: 8 total, 8 up, 8 in 2024-04-07T17:16:47.205 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:16:46 smithi100 bash[76202]: audit 2024-04-07T17:16:46.827668+0000 mon.a (mon.0) 3224 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:16:47.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:16:46 smithi144 bash[66654]: cluster 2024-04-07T17:16:45.757661+0000 mgr.y (mgr.25175) 1842 : cluster [DBG] pgmap v3087: 489 pgs: 489 active+clean; 301 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 2.9 MiB/s rd, 2.7 MiB/s wr, 685 op/s 2024-04-07T17:16:47.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:16:46 smithi144 bash[66654]: cluster 2024-04-07T17:16:45.930909+0000 mon.a (mon.0) 3223 : cluster [DBG] osdmap e1671: 8 total, 8 up, 8 in 2024-04-07T17:16:47.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:16:46 smithi144 bash[66654]: audit 2024-04-07T17:16:46.827668+0000 mon.a (mon.0) 3224 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:16:48.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:16:47 smithi100 bash[76202]: cluster 2024-04-07T17:16:46.943334+0000 mon.a (mon.0) 3225 : cluster [DBG] osdmap e1672: 8 total, 8 up, 8 in 2024-04-07T17:16:48.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:16:47 smithi100 bash[74942]: cluster 2024-04-07T17:16:46.943334+0000 mon.a (mon.0) 3225 : cluster [DBG] osdmap e1672: 8 total, 8 up, 8 in 2024-04-07T17:16:48.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:16:47 smithi144 bash[66654]: cluster 2024-04-07T17:16:46.943334+0000 mon.a (mon.0) 3225 : cluster [DBG] osdmap e1672: 8 total, 8 up, 8 in 2024-04-07T17:16:48.704 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:16:48 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:16:48] "GET /metrics HTTP/1.1" 200 63248 "" "Prometheus/2.43.0" 2024-04-07T17:16:49.021 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.TestSnapshotDeletedIo (1983 ms) 2024-04-07T17:16:49.021 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.TestClone 2024-04-07T17:16:49.021 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:16:49.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:16:48 smithi100 bash[74942]: cluster 2024-04-07T17:16:47.761053+0000 mgr.y (mgr.25175) 1843 : cluster [DBG] pgmap v3090: 489 pgs: 489 active+clean; 301 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 144 KiB/s rd, 56 KiB/s wr, 357 op/s 2024-04-07T17:16:49.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:16:48 smithi100 bash[74942]: cluster 2024-04-07T17:16:47.958633+0000 mon.a (mon.0) 3226 : cluster [DBG] osdmap e1673: 8 total, 8 up, 8 in 2024-04-07T17:16:49.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:16:48 smithi100 bash[76202]: cluster 2024-04-07T17:16:47.761053+0000 mgr.y (mgr.25175) 1843 : cluster [DBG] pgmap v3090: 489 pgs: 489 active+clean; 301 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 144 KiB/s rd, 56 KiB/s wr, 357 op/s 2024-04-07T17:16:49.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:16:48 smithi100 bash[76202]: cluster 2024-04-07T17:16:47.958633+0000 mon.a (mon.0) 3226 : cluster [DBG] osdmap e1673: 8 total, 8 up, 8 in 2024-04-07T17:16:49.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:16:48 smithi144 bash[66654]: cluster 2024-04-07T17:16:47.761053+0000 mgr.y (mgr.25175) 1843 : cluster [DBG] pgmap v3090: 489 pgs: 489 active+clean; 301 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 144 KiB/s rd, 56 KiB/s wr, 357 op/s 2024-04-07T17:16:49.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:16:48 smithi144 bash[66654]: cluster 2024-04-07T17:16:47.958633+0000 mon.a (mon.0) 3226 : cluster [DBG] osdmap e1673: 8 total, 8 up, 8 in 2024-04-07T17:16:50.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:16:49 smithi144 bash[66654]: cluster 2024-04-07T17:16:48.966041+0000 mon.a (mon.0) 3227 : cluster [DBG] osdmap e1674: 8 total, 8 up, 8 in 2024-04-07T17:16:50.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:16:49 smithi100 bash[74942]: cluster 2024-04-07T17:16:48.966041+0000 mon.a (mon.0) 3227 : cluster [DBG] osdmap e1674: 8 total, 8 up, 8 in 2024-04-07T17:16:50.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:16:49 smithi100 bash[76202]: cluster 2024-04-07T17:16:48.966041+0000 mon.a (mon.0) 3227 : cluster [DBG] osdmap e1674: 8 total, 8 up, 8 in 2024-04-07T17:16:51.036 INFO:tasks.workunit.client.0.smithi100.stdout:made parent image "parent" 2024-04-07T17:16:51.036 INFO:tasks.workunit.client.0.smithi100.stdout:parent has no parent info 2024-04-07T17:16:51.036 INFO:tasks.workunit.client.0.smithi100.stdout:made snapshot "parent@parent_snap" 2024-04-07T17:16:51.036 INFO:tasks.workunit.client.0.smithi100.stdout:can't unprotect an unprotected snap 2024-04-07T17:16:51.036 INFO:tasks.workunit.client.0.smithi100.stdout:can't protect a protected snap 2024-04-07T17:16:51.036 INFO:tasks.workunit.client.0.smithi100.stdout:made and opened clone "child" 2024-04-07T17:16:51.036 INFO:tasks.workunit.client.0.smithi100.stdout:read: 8 2024-04-07T17:16:51.037 INFO:tasks.workunit.client.0.smithi100.stdout:read: 8 2024-04-07T17:16:51.037 INFO:tasks.workunit.client.0.smithi100.stdout:read: 8 2024-04-07T17:16:51.037 INFO:tasks.workunit.client.0.smithi100.stdout:sizes and overlaps are good between parent and child 2024-04-07T17:16:51.037 INFO:tasks.workunit.client.0.smithi100.stdout:child image successfully cloned all image-meta pairs 2024-04-07T17:16:51.037 INFO:tasks.workunit.client.0.smithi100.stdout:sized down clone, changed overlap 2024-04-07T17:16:51.037 INFO:tasks.workunit.client.0.smithi100.stdout:parent info: size 4194304 obj_size 4194304 parent_pool 18446744073709551615 2024-04-07T17:16:51.037 INFO:tasks.workunit.client.0.smithi100.stdout:sized up clone, changed size but not overlap or parent's size 2024-04-07T17:16:51.037 INFO:tasks.workunit.client.0.smithi100.stdout:can't remove parent while child still exists 2024-04-07T17:16:51.037 INFO:tasks.workunit.client.0.smithi100.stdout:can't remove parent while still protected 2024-04-07T17:16:51.037 INFO:tasks.workunit.client.0.smithi100.stdout:removed parent snap after unprotecting 2024-04-07T17:16:51.037 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.TestClone (2015 ms) 2024-04-07T17:16:51.037 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.TestClone2 2024-04-07T17:16:51.037 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:16:51.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:16:50 smithi144 bash[66654]: cluster 2024-04-07T17:16:49.762596+0000 mgr.y (mgr.25175) 1844 : cluster [DBG] pgmap v3093: 489 pgs: 489 active+clean; 301 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 134 KiB/s rd, 56 KiB/s wr, 349 op/s 2024-04-07T17:16:51.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:16:50 smithi144 bash[66654]: cluster 2024-04-07T17:16:49.983338+0000 mon.a (mon.0) 3228 : cluster [DBG] osdmap e1675: 8 total, 8 up, 8 in 2024-04-07T17:16:51.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:16:50 smithi100 bash[74942]: cluster 2024-04-07T17:16:49.762596+0000 mgr.y (mgr.25175) 1844 : cluster [DBG] pgmap v3093: 489 pgs: 489 active+clean; 301 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 134 KiB/s rd, 56 KiB/s wr, 349 op/s 2024-04-07T17:16:51.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:16:50 smithi100 bash[74942]: cluster 2024-04-07T17:16:49.983338+0000 mon.a (mon.0) 3228 : cluster [DBG] osdmap e1675: 8 total, 8 up, 8 in 2024-04-07T17:16:51.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:16:50 smithi100 bash[76202]: cluster 2024-04-07T17:16:49.762596+0000 mgr.y (mgr.25175) 1844 : cluster [DBG] pgmap v3093: 489 pgs: 489 active+clean; 301 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 134 KiB/s rd, 56 KiB/s wr, 349 op/s 2024-04-07T17:16:51.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:16:50 smithi100 bash[76202]: cluster 2024-04-07T17:16:49.983338+0000 mon.a (mon.0) 3228 : cluster [DBG] osdmap e1675: 8 total, 8 up, 8 in 2024-04-07T17:16:52.317 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:16:52 smithi144 bash[66654]: cluster 2024-04-07T17:16:50.987704+0000 mon.a (mon.0) 3229 : cluster [DBG] osdmap e1676: 8 total, 8 up, 8 in 2024-04-07T17:16:52.317 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:16:52 smithi144 bash[66654]: cluster 2024-04-07T17:16:51.763896+0000 mgr.y (mgr.25175) 1845 : cluster [DBG] pgmap v3096: 489 pgs: 489 active+clean; 301 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 38 KiB/s rd, 3.2 KiB/s wr, 53 op/s 2024-04-07T17:16:52.346 INFO:tasks.workunit.client.0.smithi100.stdout:made parent image "parent" 2024-04-07T17:16:52.346 INFO:tasks.workunit.client.0.smithi100.stdout:parent has no parent info 2024-04-07T17:16:52.346 INFO:tasks.workunit.client.0.smithi100.stdout:made snapshot "parent@parent_snap" 2024-04-07T17:16:52.346 INFO:tasks.workunit.client.0.smithi100.stdout:made and opened clone "child" 2024-04-07T17:16:52.346 INFO:tasks.workunit.client.0.smithi100.stdout:child image successfully cloned all image-meta pairs 2024-04-07T17:16:52.346 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.TestClone2 (1310 ms) 2024-04-07T17:16:52.346 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.ListChildren 2024-04-07T17:16:52.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:16:52 smithi100 bash[74942]: cluster 2024-04-07T17:16:50.987704+0000 mon.a (mon.0) 3229 : cluster [DBG] osdmap e1676: 8 total, 8 up, 8 in 2024-04-07T17:16:52.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:16:52 smithi100 bash[74942]: cluster 2024-04-07T17:16:51.763896+0000 mgr.y (mgr.25175) 1845 : cluster [DBG] pgmap v3096: 489 pgs: 489 active+clean; 301 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 38 KiB/s rd, 3.2 KiB/s wr, 53 op/s 2024-04-07T17:16:52.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:16:52 smithi100 bash[76202]: cluster 2024-04-07T17:16:50.987704+0000 mon.a (mon.0) 3229 : cluster [DBG] osdmap e1676: 8 total, 8 up, 8 in 2024-04-07T17:16:52.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:16:52 smithi100 bash[76202]: cluster 2024-04-07T17:16:51.763896+0000 mgr.y (mgr.25175) 1845 : cluster [DBG] pgmap v3096: 489 pgs: 489 active+clean; 301 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 38 KiB/s rd, 3.2 KiB/s wr, 53 op/s 2024-04-07T17:16:52.617 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:16:52 smithi144 bash[95233]: ts=2024-04-07T17:16:52.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:16:53.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:16:53 smithi144 bash[66654]: cluster 2024-04-07T17:16:52.004806+0000 mon.a (mon.0) 3230 : cluster [DBG] osdmap e1677: 8 total, 8 up, 8 in 2024-04-07T17:16:53.368 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:16:53 smithi144 bash[66654]: cluster 2024-04-07T17:16:52.866316+0000 mon.a (mon.0) 3231 : cluster [DBG] osdmap e1678: 8 total, 8 up, 8 in 2024-04-07T17:16:53.368 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:16:53 smithi144 bash[66654]: audit 2024-04-07T17:16:52.869726+0000 mon.a (mon.0) 3232 : audit [INF] from='client.? 172.21.15.100:0/946567204' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:16:53.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:16:53 smithi100 bash[76202]: cluster 2024-04-07T17:16:52.004806+0000 mon.a (mon.0) 3230 : cluster [DBG] osdmap e1677: 8 total, 8 up, 8 in 2024-04-07T17:16:53.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:16:53 smithi100 bash[76202]: cluster 2024-04-07T17:16:52.866316+0000 mon.a (mon.0) 3231 : cluster [DBG] osdmap e1678: 8 total, 8 up, 8 in 2024-04-07T17:16:53.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:16:53 smithi100 bash[76202]: audit 2024-04-07T17:16:52.869726+0000 mon.a (mon.0) 3232 : audit [INF] from='client.? 172.21.15.100:0/946567204' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:16:53.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:16:53 smithi100 bash[74942]: cluster 2024-04-07T17:16:52.004806+0000 mon.a (mon.0) 3230 : cluster [DBG] osdmap e1677: 8 total, 8 up, 8 in 2024-04-07T17:16:53.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:16:53 smithi100 bash[74942]: cluster 2024-04-07T17:16:52.866316+0000 mon.a (mon.0) 3231 : cluster [DBG] osdmap e1678: 8 total, 8 up, 8 in 2024-04-07T17:16:53.455 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:16:53 smithi100 bash[74942]: audit 2024-04-07T17:16:52.869726+0000 mon.a (mon.0) 3232 : audit [INF] from='client.? 172.21.15.100:0/946567204' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:16:55.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:16:54 smithi144 bash[66654]: cluster 2024-04-07T17:16:53.765568+0000 mgr.y (mgr.25175) 1846 : cluster [DBG] pgmap v3099: 521 pgs: 2 creating+peering, 30 unknown, 489 active+clean; 301 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 488 KiB/s rd, 35 KiB/s wr, 593 op/s 2024-04-07T17:16:55.118 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:16:54 smithi144 bash[66654]: audit 2024-04-07T17:16:53.857477+0000 mon.a (mon.0) 3233 : audit [INF] from='client.? 172.21.15.100:0/946567204' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T17:16:55.118 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:16:54 smithi144 bash[66654]: cluster 2024-04-07T17:16:53.866619+0000 mon.a (mon.0) 3234 : cluster [DBG] osdmap e1679: 8 total, 8 up, 8 in 2024-04-07T17:16:55.118 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:16:54 smithi144 bash[66654]: cluster 2024-04-07T17:16:54.014122+0000 mon.a (mon.0) 3235 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T17:16:55.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:16:54 smithi100 bash[74942]: cluster 2024-04-07T17:16:53.765568+0000 mgr.y (mgr.25175) 1846 : cluster [DBG] pgmap v3099: 521 pgs: 2 creating+peering, 30 unknown, 489 active+clean; 301 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 488 KiB/s rd, 35 KiB/s wr, 593 op/s 2024-04-07T17:16:55.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:16:54 smithi100 bash[74942]: audit 2024-04-07T17:16:53.857477+0000 mon.a (mon.0) 3233 : audit [INF] from='client.? 172.21.15.100:0/946567204' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T17:16:55.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:16:54 smithi100 bash[74942]: cluster 2024-04-07T17:16:53.866619+0000 mon.a (mon.0) 3234 : cluster [DBG] osdmap e1679: 8 total, 8 up, 8 in 2024-04-07T17:16:55.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:16:54 smithi100 bash[74942]: cluster 2024-04-07T17:16:54.014122+0000 mon.a (mon.0) 3235 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T17:16:55.205 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:16:54 smithi100 bash[76202]: cluster 2024-04-07T17:16:53.765568+0000 mgr.y (mgr.25175) 1846 : cluster [DBG] pgmap v3099: 521 pgs: 2 creating+peering, 30 unknown, 489 active+clean; 301 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 488 KiB/s rd, 35 KiB/s wr, 593 op/s 2024-04-07T17:16:55.205 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:16:54 smithi100 bash[76202]: audit 2024-04-07T17:16:53.857477+0000 mon.a (mon.0) 3233 : audit [INF] from='client.? 172.21.15.100:0/946567204' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T17:16:55.205 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:16:54 smithi100 bash[76202]: cluster 2024-04-07T17:16:53.866619+0000 mon.a (mon.0) 3234 : cluster [DBG] osdmap e1679: 8 total, 8 up, 8 in 2024-04-07T17:16:55.205 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:16:54 smithi100 bash[76202]: cluster 2024-04-07T17:16:54.014122+0000 mon.a (mon.0) 3235 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T17:16:55.903 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:16:56.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:16:55 smithi100 bash[74942]: cluster 2024-04-07T17:16:54.881235+0000 mon.a (mon.0) 3236 : cluster [DBG] osdmap e1680: 8 total, 8 up, 8 in 2024-04-07T17:16:56.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:16:55 smithi100 bash[74942]: audit 2024-04-07T17:16:54.891661+0000 mon.a (mon.0) 3237 : audit [INF] from='client.? 172.21.15.100:0/1498727940' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-28","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:16:56.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:16:55 smithi100 bash[76202]: cluster 2024-04-07T17:16:54.881235+0000 mon.a (mon.0) 3236 : cluster [DBG] osdmap e1680: 8 total, 8 up, 8 in 2024-04-07T17:16:56.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:16:55 smithi100 bash[76202]: audit 2024-04-07T17:16:54.891661+0000 mon.a (mon.0) 3237 : audit [INF] from='client.? 172.21.15.100:0/1498727940' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-28","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:16:56.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:16:55 smithi144 bash[66654]: cluster 2024-04-07T17:16:54.881235+0000 mon.a (mon.0) 3236 : cluster [DBG] osdmap e1680: 8 total, 8 up, 8 in 2024-04-07T17:16:56.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:16:55 smithi144 bash[66654]: audit 2024-04-07T17:16:54.891661+0000 mon.a (mon.0) 3237 : audit [INF] from='client.? 172.21.15.100:0/1498727940' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-28","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:16:57.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:16:56 smithi100 bash[74942]: cluster 2024-04-07T17:16:55.767404+0000 mgr.y (mgr.25175) 1847 : cluster [DBG] pgmap v3102: 553 pgs: 4 creating+peering, 47 unknown, 502 active+clean; 301 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 786 KiB/s rd, 56 KiB/s wr, 938 op/s 2024-04-07T17:16:57.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:16:56 smithi100 bash[74942]: audit 2024-04-07T17:16:55.883095+0000 mon.a (mon.0) 3238 : audit [INF] from='client.? 172.21.15.100:0/1498727940' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-28","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T17:16:57.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:16:56 smithi100 bash[74942]: cluster 2024-04-07T17:16:55.893083+0000 mon.a (mon.0) 3239 : cluster [DBG] osdmap e1681: 8 total, 8 up, 8 in 2024-04-07T17:16:57.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:16:56 smithi100 bash[76202]: cluster 2024-04-07T17:16:55.767404+0000 mgr.y (mgr.25175) 1847 : cluster [DBG] pgmap v3102: 553 pgs: 4 creating+peering, 47 unknown, 502 active+clean; 301 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 786 KiB/s rd, 56 KiB/s wr, 938 op/s 2024-04-07T17:16:57.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:16:56 smithi100 bash[76202]: audit 2024-04-07T17:16:55.883095+0000 mon.a (mon.0) 3238 : audit [INF] from='client.? 172.21.15.100:0/1498727940' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-28","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T17:16:57.205 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:16:56 smithi100 bash[76202]: cluster 2024-04-07T17:16:55.893083+0000 mon.a (mon.0) 3239 : cluster [DBG] osdmap e1681: 8 total, 8 up, 8 in 2024-04-07T17:16:57.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:16:56 smithi144 bash[66654]: cluster 2024-04-07T17:16:55.767404+0000 mgr.y (mgr.25175) 1847 : cluster [DBG] pgmap v3102: 553 pgs: 4 creating+peering, 47 unknown, 502 active+clean; 301 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 786 KiB/s rd, 56 KiB/s wr, 938 op/s 2024-04-07T17:16:57.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:16:56 smithi144 bash[66654]: audit 2024-04-07T17:16:55.883095+0000 mon.a (mon.0) 3238 : audit [INF] from='client.? 172.21.15.100:0/1498727940' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-28","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T17:16:57.368 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:16:56 smithi144 bash[66654]: cluster 2024-04-07T17:16:55.893083+0000 mon.a (mon.0) 3239 : cluster [DBG] osdmap e1681: 8 total, 8 up, 8 in 2024-04-07T17:16:58.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:16:57 smithi100 bash[76202]: cluster 2024-04-07T17:16:56.892036+0000 mon.a (mon.0) 3240 : cluster [DBG] osdmap e1682: 8 total, 8 up, 8 in 2024-04-07T17:16:58.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:16:57 smithi100 bash[76202]: audit 2024-04-07T17:16:57.000116+0000 mon.a (mon.0) 3241 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:16:58.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:16:57 smithi100 bash[76202]: audit 2024-04-07T17:16:57.741183+0000 mon.a (mon.0) 3242 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:16:58.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:16:57 smithi100 bash[76202]: audit 2024-04-07T17:16:57.742949+0000 mon.a (mon.0) 3243 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:16:58.205 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:16:57 smithi100 bash[76202]: audit 2024-04-07T17:16:57.751640+0000 mon.a (mon.0) 3244 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:16:58.205 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:16:57 smithi100 bash[76202]: cluster 2024-04-07T17:16:57.866182+0000 mon.a (mon.0) 3245 : cluster [DBG] osdmap e1683: 8 total, 8 up, 8 in 2024-04-07T17:16:58.205 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:16:57 smithi100 bash[74942]: cluster 2024-04-07T17:16:56.892036+0000 mon.a (mon.0) 3240 : cluster [DBG] osdmap e1682: 8 total, 8 up, 8 in 2024-04-07T17:16:58.205 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:16:57 smithi100 bash[74942]: audit 2024-04-07T17:16:57.000116+0000 mon.a (mon.0) 3241 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:16:58.205 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:16:57 smithi100 bash[74942]: audit 2024-04-07T17:16:57.741183+0000 mon.a (mon.0) 3242 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:16:58.205 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:16:57 smithi100 bash[74942]: audit 2024-04-07T17:16:57.742949+0000 mon.a (mon.0) 3243 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:16:58.205 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:16:57 smithi100 bash[74942]: audit 2024-04-07T17:16:57.751640+0000 mon.a (mon.0) 3244 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:16:58.205 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:16:57 smithi100 bash[74942]: cluster 2024-04-07T17:16:57.866182+0000 mon.a (mon.0) 3245 : cluster [DBG] osdmap e1683: 8 total, 8 up, 8 in 2024-04-07T17:16:58.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:16:57 smithi144 bash[66654]: cluster 2024-04-07T17:16:56.892036+0000 mon.a (mon.0) 3240 : cluster [DBG] osdmap e1682: 8 total, 8 up, 8 in 2024-04-07T17:16:58.368 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:16:57 smithi144 bash[66654]: audit 2024-04-07T17:16:57.000116+0000 mon.a (mon.0) 3241 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:16:58.368 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:16:57 smithi144 bash[66654]: audit 2024-04-07T17:16:57.741183+0000 mon.a (mon.0) 3242 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:16:58.368 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:16:57 smithi144 bash[66654]: audit 2024-04-07T17:16:57.742949+0000 mon.a (mon.0) 3243 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:16:58.368 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:16:57 smithi144 bash[66654]: audit 2024-04-07T17:16:57.751640+0000 mon.a (mon.0) 3244 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:16:58.368 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:16:57 smithi144 bash[66654]: cluster 2024-04-07T17:16:57.866182+0000 mon.a (mon.0) 3245 : cluster [DBG] osdmap e1683: 8 total, 8 up, 8 in 2024-04-07T17:16:58.704 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:16:58 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:16:58] "GET /metrics HTTP/1.1" 200 63248 "" "Prometheus/2.43.0" 2024-04-07T17:16:59.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:16:58 smithi100 bash[74942]: cluster 2024-04-07T17:16:57.770467+0000 mgr.y (mgr.25175) 1848 : cluster [DBG] pgmap v3105: 553 pgs: 4 creating+peering, 15 unknown, 534 active+clean; 301 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 336 KiB/s rd, 24 KiB/s wr, 396 op/s 2024-04-07T17:16:59.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:16:58 smithi100 bash[74942]: cluster 2024-04-07T17:16:58.748876+0000 mon.a (mon.0) 3246 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T17:16:59.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:16:58 smithi100 bash[76202]: cluster 2024-04-07T17:16:57.770467+0000 mgr.y (mgr.25175) 1848 : cluster [DBG] pgmap v3105: 553 pgs: 4 creating+peering, 15 unknown, 534 active+clean; 301 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 336 KiB/s rd, 24 KiB/s wr, 396 op/s 2024-04-07T17:16:59.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:16:58 smithi100 bash[76202]: cluster 2024-04-07T17:16:58.748876+0000 mon.a (mon.0) 3246 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T17:16:59.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:16:58 smithi144 bash[66654]: cluster 2024-04-07T17:16:57.770467+0000 mgr.y (mgr.25175) 1848 : cluster [DBG] pgmap v3105: 553 pgs: 4 creating+peering, 15 unknown, 534 active+clean; 301 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 336 KiB/s rd, 24 KiB/s wr, 396 op/s 2024-04-07T17:16:59.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:16:58 smithi144 bash[66654]: cluster 2024-04-07T17:16:58.748876+0000 mon.a (mon.0) 3246 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T17:17:00.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:16:59 smithi100 bash[74942]: cluster 2024-04-07T17:16:58.914913+0000 mon.a (mon.0) 3247 : cluster [DBG] osdmap e1684: 8 total, 8 up, 8 in 2024-04-07T17:17:00.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:16:59 smithi100 bash[76202]: cluster 2024-04-07T17:16:58.914913+0000 mon.a (mon.0) 3247 : cluster [DBG] osdmap e1684: 8 total, 8 up, 8 in 2024-04-07T17:17:00.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:16:59 smithi144 bash[66654]: cluster 2024-04-07T17:16:58.914913+0000 mon.a (mon.0) 3247 : cluster [DBG] osdmap e1684: 8 total, 8 up, 8 in 2024-04-07T17:17:01.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:17:00 smithi100 bash[74942]: cluster 2024-04-07T17:16:59.772702+0000 mgr.y (mgr.25175) 1849 : cluster [DBG] pgmap v3108: 553 pgs: 2 creating+peering, 551 active+clean; 301 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 32 KiB/s rd, 2.0 KiB/s wr, 42 op/s 2024-04-07T17:17:01.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:17:00 smithi100 bash[74942]: cluster 2024-04-07T17:16:59.941832+0000 mon.a (mon.0) 3248 : cluster [DBG] osdmap e1685: 8 total, 8 up, 8 in 2024-04-07T17:17:01.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:17:00 smithi100 bash[76202]: cluster 2024-04-07T17:16:59.772702+0000 mgr.y (mgr.25175) 1849 : cluster [DBG] pgmap v3108: 553 pgs: 2 creating+peering, 551 active+clean; 301 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 32 KiB/s rd, 2.0 KiB/s wr, 42 op/s 2024-04-07T17:17:01.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:17:00 smithi100 bash[76202]: cluster 2024-04-07T17:16:59.941832+0000 mon.a (mon.0) 3248 : cluster [DBG] osdmap e1685: 8 total, 8 up, 8 in 2024-04-07T17:17:01.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:17:00 smithi144 bash[66654]: cluster 2024-04-07T17:16:59.772702+0000 mgr.y (mgr.25175) 1849 : cluster [DBG] pgmap v3108: 553 pgs: 2 creating+peering, 551 active+clean; 301 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 32 KiB/s rd, 2.0 KiB/s wr, 42 op/s 2024-04-07T17:17:01.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:17:00 smithi144 bash[66654]: cluster 2024-04-07T17:16:59.941832+0000 mon.a (mon.0) 3248 : cluster [DBG] osdmap e1685: 8 total, 8 up, 8 in 2024-04-07T17:17:02.236 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:17:02.236 INFO:tasks.workunit.client.0.smithi100.stdout:trying to find test-librbd-smithi100-115303-28/image113 2024-04-07T17:17:02.236 INFO:tasks.workunit.client.0.smithi100.stdout:checking test-librbd-smithi100-115303-28/image113 2024-04-07T17:17:02.236 INFO:tasks.workunit.client.0.smithi100.stdout:found child test-librbd-smithi100-115303-28/image113 2024-04-07T17:17:02.236 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:17:02.236 INFO:tasks.workunit.client.0.smithi100.stdout:num children is: 1 2024-04-07T17:17:02.236 INFO:tasks.workunit.client.0.smithi100.stdout:expected: 1 2024-04-07T17:17:02.236 INFO:tasks.workunit.client.0.smithi100.stdout:child: image113 2024-04-07T17:17:02.236 INFO:tasks.workunit.client.0.smithi100.stdout:found child test-librbd-smithi100-115303-28/image113/b186da1bf797 2024-04-07T17:17:02.237 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:17:02.237 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:17:02.237 INFO:tasks.workunit.client.0.smithi100.stdout:trying to find test-librbd-smithi100-115303-28/image113 2024-04-07T17:17:02.237 INFO:tasks.workunit.client.0.smithi100.stdout:checking test-librbd-smithi100-115303-27/image114 2024-04-07T17:17:02.237 INFO:tasks.workunit.client.0.smithi100.stdout:checking test-librbd-smithi100-115303-28/image113 2024-04-07T17:17:02.237 INFO:tasks.workunit.client.0.smithi100.stdout:found child test-librbd-smithi100-115303-28/image113 2024-04-07T17:17:02.237 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:17:02.237 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:17:02.237 INFO:tasks.workunit.client.0.smithi100.stdout:trying to find test-librbd-smithi100-115303-27/image114 2024-04-07T17:17:02.237 INFO:tasks.workunit.client.0.smithi100.stdout:checking test-librbd-smithi100-115303-27/image114 2024-04-07T17:17:02.237 INFO:tasks.workunit.client.0.smithi100.stdout:found child test-librbd-smithi100-115303-27/image114 2024-04-07T17:17:02.237 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:17:02.237 INFO:tasks.workunit.client.0.smithi100.stdout:num children is: 2 2024-04-07T17:17:02.237 INFO:tasks.workunit.client.0.smithi100.stdout:expected: 2 2024-04-07T17:17:02.237 INFO:tasks.workunit.client.0.smithi100.stdout:child: image114 2024-04-07T17:17:02.238 INFO:tasks.workunit.client.0.smithi100.stdout:child: image113 2024-04-07T17:17:02.238 INFO:tasks.workunit.client.0.smithi100.stdout:found child test-librbd-smithi100-115303-28/image113/b186da1bf797 2024-04-07T17:17:02.238 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:17:02.238 INFO:tasks.workunit.client.0.smithi100.stdout:found child test-librbd-smithi100-115303-27/image114/b186534983c1 2024-04-07T17:17:02.238 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:17:02.238 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:17:02.238 INFO:tasks.workunit.client.0.smithi100.stdout:trying to find test-librbd-smithi100-115303-28/image113 2024-04-07T17:17:02.238 INFO:tasks.workunit.client.0.smithi100.stdout:checking test-librbd-smithi100-115303-27/image114 2024-04-07T17:17:02.238 INFO:tasks.workunit.client.0.smithi100.stdout:checking test-librbd-smithi100-115303-28/image113 2024-04-07T17:17:02.238 INFO:tasks.workunit.client.0.smithi100.stdout:found child test-librbd-smithi100-115303-28/image113 2024-04-07T17:17:02.238 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:17:02.238 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:17:02.238 INFO:tasks.workunit.client.0.smithi100.stdout:trying to find test-librbd-smithi100-115303-27/image114 2024-04-07T17:17:02.238 INFO:tasks.workunit.client.0.smithi100.stdout:checking test-librbd-smithi100-115303-27/image114 2024-04-07T17:17:02.239 INFO:tasks.workunit.client.0.smithi100.stdout:found child test-librbd-smithi100-115303-27/image114 2024-04-07T17:17:02.239 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:17:02.239 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:17:02.239 INFO:tasks.workunit.client.0.smithi100.stdout:trying to find test-librbd-smithi100-115303-28/image115 2024-04-07T17:17:02.239 INFO:tasks.workunit.client.0.smithi100.stdout:checking test-librbd-smithi100-115303-27/image114 2024-04-07T17:17:02.239 INFO:tasks.workunit.client.0.smithi100.stdout:checking test-librbd-smithi100-115303-28/image113 2024-04-07T17:17:02.239 INFO:tasks.workunit.client.0.smithi100.stdout:checking test-librbd-smithi100-115303-28/image115 2024-04-07T17:17:02.239 INFO:tasks.workunit.client.0.smithi100.stdout:found child test-librbd-smithi100-115303-28/image115 2024-04-07T17:17:02.239 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:17:02.239 INFO:tasks.workunit.client.0.smithi100.stdout:num children is: 3 2024-04-07T17:17:02.239 INFO:tasks.workunit.client.0.smithi100.stdout:expected: 3 2024-04-07T17:17:02.239 INFO:tasks.workunit.client.0.smithi100.stdout:child: image114 2024-04-07T17:17:02.239 INFO:tasks.workunit.client.0.smithi100.stdout:child: image113 2024-04-07T17:17:02.239 INFO:tasks.workunit.client.0.smithi100.stdout:child: image115 2024-04-07T17:17:02.239 INFO:tasks.workunit.client.0.smithi100.stdout:found child test-librbd-smithi100-115303-28/image113/b186da1bf797 2024-04-07T17:17:02.240 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:17:02.240 INFO:tasks.workunit.client.0.smithi100.stdout:found child test-librbd-smithi100-115303-27/image114/b186534983c1 2024-04-07T17:17:02.240 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:17:02.240 INFO:tasks.workunit.client.0.smithi100.stdout:found child test-librbd-smithi100-115303-28/image115/b1868acdef62 2024-04-07T17:17:02.240 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:17:02.240 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:17:02.240 INFO:tasks.workunit.client.0.smithi100.stdout:trying to find test-librbd-smithi100-115303-28/image113 2024-04-07T17:17:02.240 INFO:tasks.workunit.client.0.smithi100.stdout:checking test-librbd-smithi100-115303-27/image114 2024-04-07T17:17:02.240 INFO:tasks.workunit.client.0.smithi100.stdout:checking test-librbd-smithi100-115303-28/image113 2024-04-07T17:17:02.240 INFO:tasks.workunit.client.0.smithi100.stdout:found child test-librbd-smithi100-115303-28/image113 2024-04-07T17:17:02.240 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:17:02.240 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:17:02.240 INFO:tasks.workunit.client.0.smithi100.stdout:trying to find test-librbd-smithi100-115303-27/image114 2024-04-07T17:17:02.240 INFO:tasks.workunit.client.0.smithi100.stdout:checking test-librbd-smithi100-115303-27/image114 2024-04-07T17:17:02.240 INFO:tasks.workunit.client.0.smithi100.stdout:found child test-librbd-smithi100-115303-27/image114 2024-04-07T17:17:02.241 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:17:02.241 INFO:tasks.workunit.client.0.smithi100.stdout:num children is: 3 2024-04-07T17:17:02.241 INFO:tasks.workunit.client.0.smithi100.stdout:expected: 3 2024-04-07T17:17:02.241 INFO:tasks.workunit.client.0.smithi100.stdout:child: image114 2024-04-07T17:17:02.241 INFO:tasks.workunit.client.0.smithi100.stdout:child: image113 2024-04-07T17:17:02.241 INFO:tasks.workunit.client.0.smithi100.stdout:child: image115 2024-04-07T17:17:02.241 INFO:tasks.workunit.client.0.smithi100.stdout:found child test-librbd-smithi100-115303-28/image113/b186da1bf797 2024-04-07T17:17:02.241 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:17:02.241 INFO:tasks.workunit.client.0.smithi100.stdout:found child test-librbd-smithi100-115303-27/image114/b186534983c1 2024-04-07T17:17:02.241 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:17:02.241 INFO:tasks.workunit.client.0.smithi100.stdout:found child test-librbd-smithi100-115303-28/image115/b1868acdef62 2024-04-07T17:17:02.241 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:17:02.241 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:17:02.241 INFO:tasks.workunit.client.0.smithi100.stdout:trying to find test-librbd-smithi100-115303-28/image113 2024-04-07T17:17:02.241 INFO:tasks.workunit.client.0.smithi100.stdout:checking test-librbd-smithi100-115303-27/image114 2024-04-07T17:17:02.242 INFO:tasks.workunit.client.0.smithi100.stdout:checking test-librbd-smithi100-115303-28/image113 2024-04-07T17:17:02.242 INFO:tasks.workunit.client.0.smithi100.stdout:found child test-librbd-smithi100-115303-28/image113 2024-04-07T17:17:02.242 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:17:02.242 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:17:02.242 INFO:tasks.workunit.client.0.smithi100.stdout:trying to find test-librbd-smithi100-115303-27/image114 2024-04-07T17:17:02.242 INFO:tasks.workunit.client.0.smithi100.stdout:checking test-librbd-smithi100-115303-27/image114 2024-04-07T17:17:02.242 INFO:tasks.workunit.client.0.smithi100.stdout:found child test-librbd-smithi100-115303-27/image114 2024-04-07T17:17:02.242 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:17:02.242 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:17:02.242 INFO:tasks.workunit.client.0.smithi100.stdout:trying to find test-librbd-smithi100-115303-28/image116 2024-04-07T17:17:02.242 INFO:tasks.workunit.client.0.smithi100.stdout:checking test-librbd-smithi100-115303-27/image114 2024-04-07T17:17:02.242 INFO:tasks.workunit.client.0.smithi100.stdout:checking test-librbd-smithi100-115303-28/image113 2024-04-07T17:17:02.242 INFO:tasks.workunit.client.0.smithi100.stdout:checking test-librbd-smithi100-115303-28/image116 2024-04-07T17:17:02.242 INFO:tasks.workunit.client.0.smithi100.stdout:found child test-librbd-smithi100-115303-28/image116 2024-04-07T17:17:02.242 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:17:02.243 INFO:tasks.workunit.client.0.smithi100.stdout:num children is: 4 2024-04-07T17:17:02.243 INFO:tasks.workunit.client.0.smithi100.stdout:expected: 4 2024-04-07T17:17:02.243 INFO:tasks.workunit.client.0.smithi100.stdout:child: image114 2024-04-07T17:17:02.243 INFO:tasks.workunit.client.0.smithi100.stdout:child: image113 2024-04-07T17:17:02.243 INFO:tasks.workunit.client.0.smithi100.stdout:child: image115 2024-04-07T17:17:02.243 INFO:tasks.workunit.client.0.smithi100.stdout:child: image116 2024-04-07T17:17:02.243 INFO:tasks.workunit.client.0.smithi100.stdout:found child test-librbd-smithi100-115303-28/image113/b186da1bf797 2024-04-07T17:17:02.243 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:17:02.243 INFO:tasks.workunit.client.0.smithi100.stdout:found child test-librbd-smithi100-115303-27/image114/b186534983c1 2024-04-07T17:17:02.243 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:17:02.243 INFO:tasks.workunit.client.0.smithi100.stdout:found child test-librbd-smithi100-115303-28/image115/b1868acdef62 2024-04-07T17:17:02.243 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:17:02.243 INFO:tasks.workunit.client.0.smithi100.stdout:found child test-librbd-smithi100-115303-28/image116/b186e6c3bbbc 2024-04-07T17:17:02.243 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:17:02.243 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:17:02.244 INFO:tasks.workunit.client.0.smithi100.stdout:trying to find test-librbd-smithi100-115303-28/image113 2024-04-07T17:17:02.244 INFO:tasks.workunit.client.0.smithi100.stdout:checking test-librbd-smithi100-115303-27/image114 2024-04-07T17:17:02.244 INFO:tasks.workunit.client.0.smithi100.stdout:checking test-librbd-smithi100-115303-28/image113 2024-04-07T17:17:02.244 INFO:tasks.workunit.client.0.smithi100.stdout:found child test-librbd-smithi100-115303-28/image113 2024-04-07T17:17:02.244 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:17:02.244 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:17:02.244 INFO:tasks.workunit.client.0.smithi100.stdout:trying to find test-librbd-smithi100-115303-27/image114 2024-04-07T17:17:02.244 INFO:tasks.workunit.client.0.smithi100.stdout:checking test-librbd-smithi100-115303-27/image114 2024-04-07T17:17:02.244 INFO:tasks.workunit.client.0.smithi100.stdout:found child test-librbd-smithi100-115303-27/image114 2024-04-07T17:17:02.244 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:17:02.244 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:17:02.244 INFO:tasks.workunit.client.0.smithi100.stdout:trying to find test-librbd-smithi100-115303-28/image115 2024-04-07T17:17:02.244 INFO:tasks.workunit.client.0.smithi100.stdout:checking test-librbd-smithi100-115303-27/image114 2024-04-07T17:17:02.244 INFO:tasks.workunit.client.0.smithi100.stdout:checking test-librbd-smithi100-115303-28/image113 2024-04-07T17:17:02.244 INFO:tasks.workunit.client.0.smithi100.stdout:checking test-librbd-smithi100-115303-28/image115 2024-04-07T17:17:02.244 INFO:tasks.workunit.client.0.smithi100.stdout:found child test-librbd-smithi100-115303-28/image115 2024-04-07T17:17:02.245 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:17:02.245 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:17:02.245 INFO:tasks.workunit.client.0.smithi100.stdout:trying to find test-librbd-smithi100-115303-28/image116 2024-04-07T17:17:02.317 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:17:01 smithi144 bash[66654]: cluster 2024-04-07T17:17:00.942030+0000 mon.a (mon.0) 3249 : cluster [DBG] osdmap e1686: 8 total, 8 up, 8 in 2024-04-07T17:17:02.318 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:17:01 smithi144 bash[66654]: audit 2024-04-07T17:17:01.827565+0000 mon.a (mon.0) 3250 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:17:02.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:17:01 smithi100 bash[74942]: cluster 2024-04-07T17:17:00.942030+0000 mon.a (mon.0) 3249 : cluster [DBG] osdmap e1686: 8 total, 8 up, 8 in 2024-04-07T17:17:02.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:17:01 smithi100 bash[74942]: audit 2024-04-07T17:17:01.827565+0000 mon.a (mon.0) 3250 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:17:02.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:17:01 smithi100 bash[76202]: cluster 2024-04-07T17:17:00.942030+0000 mon.a (mon.0) 3249 : cluster [DBG] osdmap e1686: 8 total, 8 up, 8 in 2024-04-07T17:17:02.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:17:01 smithi100 bash[76202]: audit 2024-04-07T17:17:01.827565+0000 mon.a (mon.0) 3250 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:17:02.617 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:17:02 smithi144 bash[95233]: ts=2024-04-07T17:17:02.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:17:03.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:17:02 smithi144 bash[66654]: cluster 2024-04-07T17:17:01.774138+0000 mgr.y (mgr.25175) 1850 : cluster [DBG] pgmap v3111: 553 pgs: 553 active+clean; 301 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 27 KiB/s rd, 1.2 KiB/s wr, 34 op/s 2024-04-07T17:17:03.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:17:02 smithi100 bash[74942]: cluster 2024-04-07T17:17:01.774138+0000 mgr.y (mgr.25175) 1850 : cluster [DBG] pgmap v3111: 553 pgs: 553 active+clean; 301 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 27 KiB/s rd, 1.2 KiB/s wr, 34 op/s 2024-04-07T17:17:03.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:17:02 smithi100 bash[76202]: cluster 2024-04-07T17:17:01.774138+0000 mgr.y (mgr.25175) 1850 : cluster [DBG] pgmap v3111: 553 pgs: 553 active+clean; 301 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 27 KiB/s rd, 1.2 KiB/s wr, 34 op/s 2024-04-07T17:17:04.231 INFO:tasks.workunit.client.0.smithi100.stdout:checking test-librbd-smithi100-115303-27/image114 2024-04-07T17:17:04.231 INFO:tasks.workunit.client.0.smithi100.stdout:checking test-librbd-smithi100-115303-28/image113 2024-04-07T17:17:04.232 INFO:tasks.workunit.client.0.smithi100.stdout:checking test-librbd-smithi100-115303-28/image115 2024-04-07T17:17:04.232 INFO:tasks.workunit.client.0.smithi100.stdout:checking test-librbd-smithi100-115303-28/image116 2024-04-07T17:17:04.232 INFO:tasks.workunit.client.0.smithi100.stdout:found child test-librbd-smithi100-115303-28/image116 2024-04-07T17:17:04.232 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:17:04.232 INFO:tasks.workunit.client.0.smithi100.stdout:num children is: 4 2024-04-07T17:17:04.232 INFO:tasks.workunit.client.0.smithi100.stdout:expected: 4 2024-04-07T17:17:04.232 INFO:tasks.workunit.client.0.smithi100.stdout:child: image114 2024-04-07T17:17:04.232 INFO:tasks.workunit.client.0.smithi100.stdout:child: image113 2024-04-07T17:17:04.232 INFO:tasks.workunit.client.0.smithi100.stdout:child: image115 2024-04-07T17:17:04.232 INFO:tasks.workunit.client.0.smithi100.stdout:child: image116 2024-04-07T17:17:04.232 INFO:tasks.workunit.client.0.smithi100.stdout:found child test-librbd-smithi100-115303-28/image113/b186da1bf797 2024-04-07T17:17:04.232 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:17:04.232 INFO:tasks.workunit.client.0.smithi100.stdout:found child test-librbd-smithi100-115303-27/image114/b186534983c1 2024-04-07T17:17:04.232 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:17:04.233 INFO:tasks.workunit.client.0.smithi100.stdout:found child test-librbd-smithi100-115303-28/image115/b1868acdef62 2024-04-07T17:17:04.233 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:17:04.233 INFO:tasks.workunit.client.0.smithi100.stdout:found child test-librbd-smithi100-115303-28/image116/b186e6c3bbbc 2024-04-07T17:17:04.233 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:17:04.233 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:17:04.233 INFO:tasks.workunit.client.0.smithi100.stdout:trying to find test-librbd-smithi100-115303-27/image114 2024-04-07T17:17:04.233 INFO:tasks.workunit.client.0.smithi100.stdout:checking test-librbd-smithi100-115303-27/image114 2024-04-07T17:17:04.233 INFO:tasks.workunit.client.0.smithi100.stdout:found child test-librbd-smithi100-115303-27/image114 2024-04-07T17:17:04.233 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:17:04.233 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:17:04.233 INFO:tasks.workunit.client.0.smithi100.stdout:trying to find test-librbd-smithi100-115303-28/image115 2024-04-07T17:17:04.233 INFO:tasks.workunit.client.0.smithi100.stdout:checking test-librbd-smithi100-115303-27/image114 2024-04-07T17:17:04.233 INFO:tasks.workunit.client.0.smithi100.stdout:checking test-librbd-smithi100-115303-28/image115 2024-04-07T17:17:04.233 INFO:tasks.workunit.client.0.smithi100.stdout:found child test-librbd-smithi100-115303-28/image115 2024-04-07T17:17:04.234 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:17:04.234 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:17:04.234 INFO:tasks.workunit.client.0.smithi100.stdout:trying to find test-librbd-smithi100-115303-28/image116 2024-04-07T17:17:04.234 INFO:tasks.workunit.client.0.smithi100.stdout:checking test-librbd-smithi100-115303-27/image114 2024-04-07T17:17:04.234 INFO:tasks.workunit.client.0.smithi100.stdout:checking test-librbd-smithi100-115303-28/image115 2024-04-07T17:17:04.234 INFO:tasks.workunit.client.0.smithi100.stdout:checking test-librbd-smithi100-115303-28/image116 2024-04-07T17:17:04.234 INFO:tasks.workunit.client.0.smithi100.stdout:found child test-librbd-smithi100-115303-28/image116 2024-04-07T17:17:04.234 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:17:04.234 INFO:tasks.workunit.client.0.smithi100.stdout:num children is: 3 2024-04-07T17:17:04.234 INFO:tasks.workunit.client.0.smithi100.stdout:expected: 3 2024-04-07T17:17:04.234 INFO:tasks.workunit.client.0.smithi100.stdout:child: image114 2024-04-07T17:17:04.234 INFO:tasks.workunit.client.0.smithi100.stdout:child: image115 2024-04-07T17:17:04.234 INFO:tasks.workunit.client.0.smithi100.stdout:child: image116 2024-04-07T17:17:04.234 INFO:tasks.workunit.client.0.smithi100.stdout:found child test-librbd-smithi100-115303-27/image114/b186534983c1 2024-04-07T17:17:04.234 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:17:04.235 INFO:tasks.workunit.client.0.smithi100.stdout:found child test-librbd-smithi100-115303-28/image115/b1868acdef62 2024-04-07T17:17:04.235 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:17:04.235 INFO:tasks.workunit.client.0.smithi100.stdout:found child test-librbd-smithi100-115303-28/image116/b186e6c3bbbc 2024-04-07T17:17:04.235 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:17:04.235 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:17:04.235 INFO:tasks.workunit.client.0.smithi100.stdout:trying to find test-librbd-smithi100-115303-27/image114 2024-04-07T17:17:04.235 INFO:tasks.workunit.client.0.smithi100.stdout:checking test-librbd-smithi100-115303-27/image114 2024-04-07T17:17:04.235 INFO:tasks.workunit.client.0.smithi100.stdout:found child test-librbd-smithi100-115303-27/image114 2024-04-07T17:17:04.235 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:17:04.235 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:17:04.235 INFO:tasks.workunit.client.0.smithi100.stdout:trying to find test-librbd-smithi100-115303-28/image116 2024-04-07T17:17:04.235 INFO:tasks.workunit.client.0.smithi100.stdout:checking test-librbd-smithi100-115303-27/image114 2024-04-07T17:17:04.235 INFO:tasks.workunit.client.0.smithi100.stdout:checking test-librbd-smithi100-115303-28/image116 2024-04-07T17:17:04.235 INFO:tasks.workunit.client.0.smithi100.stdout:found child test-librbd-smithi100-115303-28/image116 2024-04-07T17:17:04.235 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:17:04.236 INFO:tasks.workunit.client.0.smithi100.stdout:num children is: 2 2024-04-07T17:17:04.236 INFO:tasks.workunit.client.0.smithi100.stdout:expected: 2 2024-04-07T17:17:04.236 INFO:tasks.workunit.client.0.smithi100.stdout:child: image114 2024-04-07T17:17:04.236 INFO:tasks.workunit.client.0.smithi100.stdout:child: image116 2024-04-07T17:17:04.236 INFO:tasks.workunit.client.0.smithi100.stdout:found child test-librbd-smithi100-115303-27/image114/b186534983c1 2024-04-07T17:17:04.236 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:17:04.236 INFO:tasks.workunit.client.0.smithi100.stdout:found child test-librbd-smithi100-115303-28/image116/b186e6c3bbbc 2024-04-07T17:17:04.236 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:17:04.236 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:17:04.236 INFO:tasks.workunit.client.0.smithi100.stdout:trying to find test-librbd-smithi100-115303-27/image114 2024-04-07T17:17:04.236 INFO:tasks.workunit.client.0.smithi100.stdout:checking test-librbd-smithi100-115303-27/image114 2024-04-07T17:17:04.236 INFO:tasks.workunit.client.0.smithi100.stdout:found child test-librbd-smithi100-115303-27/image114 2024-04-07T17:17:04.236 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:17:04.236 INFO:tasks.workunit.client.0.smithi100.stdout:num children is: 1 2024-04-07T17:17:04.236 INFO:tasks.workunit.client.0.smithi100.stdout:expected: 1 2024-04-07T17:17:04.237 INFO:tasks.workunit.client.0.smithi100.stdout:child: image114 2024-04-07T17:17:04.237 INFO:tasks.workunit.client.0.smithi100.stdout:found child test-librbd-smithi100-115303-27/image114/b186534983c1 2024-04-07T17:17:04.237 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:17:04.237 INFO:tasks.workunit.client.0.smithi100.stdout:num children is: 0 2024-04-07T17:17:04.237 INFO:tasks.workunit.client.0.smithi100.stdout:expected: 0 2024-04-07T17:17:04.237 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.ListChildren (11885 ms) 2024-04-07T17:17:04.237 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.ListChildrenTiered 2024-04-07T17:17:05.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:17:04 smithi144 bash[66654]: cluster 2024-04-07T17:17:03.778305+0000 mgr.y (mgr.25175) 1851 : cluster [DBG] pgmap v3112: 553 pgs: 553 active+clean; 301 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 113 KiB/s rd, 4.4 KiB/s wr, 136 op/s 2024-04-07T17:17:05.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:17:04 smithi144 bash[66654]: cluster 2024-04-07T17:17:03.982991+0000 mon.a (mon.0) 3251 : cluster [DBG] osdmap e1687: 8 total, 8 up, 8 in 2024-04-07T17:17:05.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:17:04 smithi100 bash[74942]: cluster 2024-04-07T17:17:03.778305+0000 mgr.y (mgr.25175) 1851 : cluster [DBG] pgmap v3112: 553 pgs: 553 active+clean; 301 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 113 KiB/s rd, 4.4 KiB/s wr, 136 op/s 2024-04-07T17:17:05.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:17:04 smithi100 bash[74942]: cluster 2024-04-07T17:17:03.982991+0000 mon.a (mon.0) 3251 : cluster [DBG] osdmap e1687: 8 total, 8 up, 8 in 2024-04-07T17:17:05.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:17:04 smithi100 bash[76202]: cluster 2024-04-07T17:17:03.778305+0000 mgr.y (mgr.25175) 1851 : cluster [DBG] pgmap v3112: 553 pgs: 553 active+clean; 301 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 113 KiB/s rd, 4.4 KiB/s wr, 136 op/s 2024-04-07T17:17:05.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:17:04 smithi100 bash[76202]: cluster 2024-04-07T17:17:03.982991+0000 mon.a (mon.0) 3251 : cluster [DBG] osdmap e1687: 8 total, 8 up, 8 in 2024-04-07T17:17:06.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:17:05 smithi144 bash[66654]: cluster 2024-04-07T17:17:04.996035+0000 mon.a (mon.0) 3252 : cluster [DBG] osdmap e1688: 8 total, 8 up, 8 in 2024-04-07T17:17:06.368 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:17:05 smithi144 bash[66654]: audit 2024-04-07T17:17:04.998892+0000 mon.a (mon.0) 3253 : audit [INF] from='client.? 172.21.15.100:0/3056095998' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:17:06.368 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:17:05 smithi144 bash[66654]: cluster 2024-04-07T17:17:05.779565+0000 mgr.y (mgr.25175) 1852 : cluster [DBG] pgmap v3115: 585 pgs: 32 unknown, 553 active+clean; 301 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 299 KiB/s rd, 9.1 KiB/s wr, 367 op/s 2024-04-07T17:17:06.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:17:06 smithi100 bash[74942]: cluster 2024-04-07T17:17:04.996035+0000 mon.a (mon.0) 3252 : cluster [DBG] osdmap e1688: 8 total, 8 up, 8 in 2024-04-07T17:17:06.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:17:06 smithi100 bash[74942]: audit 2024-04-07T17:17:04.998892+0000 mon.a (mon.0) 3253 : audit [INF] from='client.? 172.21.15.100:0/3056095998' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:17:06.455 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:17:06 smithi100 bash[74942]: cluster 2024-04-07T17:17:05.779565+0000 mgr.y (mgr.25175) 1852 : cluster [DBG] pgmap v3115: 585 pgs: 32 unknown, 553 active+clean; 301 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 299 KiB/s rd, 9.1 KiB/s wr, 367 op/s 2024-04-07T17:17:06.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:17:06 smithi100 bash[76202]: cluster 2024-04-07T17:17:04.996035+0000 mon.a (mon.0) 3252 : cluster [DBG] osdmap e1688: 8 total, 8 up, 8 in 2024-04-07T17:17:06.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:17:06 smithi100 bash[76202]: audit 2024-04-07T17:17:04.998892+0000 mon.a (mon.0) 3253 : audit [INF] from='client.? 172.21.15.100:0/3056095998' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:17:06.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:17:06 smithi100 bash[76202]: cluster 2024-04-07T17:17:05.779565+0000 mgr.y (mgr.25175) 1852 : cluster [DBG] pgmap v3115: 585 pgs: 32 unknown, 553 active+clean; 301 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 299 KiB/s rd, 9.1 KiB/s wr, 367 op/s 2024-04-07T17:17:07.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:17:07 smithi144 bash[66654]: cluster 2024-04-07T17:17:05.982851+0000 mon.a (mon.0) 3254 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T17:17:07.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:17:07 smithi144 bash[66654]: audit 2024-04-07T17:17:05.996582+0000 mon.a (mon.0) 3255 : audit [INF] from='client.? 172.21.15.100:0/3056095998' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T17:17:07.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:17:07 smithi144 bash[66654]: cluster 2024-04-07T17:17:06.009784+0000 mon.a (mon.0) 3256 : cluster [DBG] osdmap e1689: 8 total, 8 up, 8 in 2024-04-07T17:17:07.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:17:07 smithi100 bash[74942]: cluster 2024-04-07T17:17:05.982851+0000 mon.a (mon.0) 3254 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T17:17:07.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:17:07 smithi100 bash[74942]: audit 2024-04-07T17:17:05.996582+0000 mon.a (mon.0) 3255 : audit [INF] from='client.? 172.21.15.100:0/3056095998' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T17:17:07.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:17:07 smithi100 bash[74942]: cluster 2024-04-07T17:17:06.009784+0000 mon.a (mon.0) 3256 : cluster [DBG] osdmap e1689: 8 total, 8 up, 8 in 2024-04-07T17:17:07.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:17:07 smithi100 bash[76202]: cluster 2024-04-07T17:17:05.982851+0000 mon.a (mon.0) 3254 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T17:17:07.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:17:07 smithi100 bash[76202]: audit 2024-04-07T17:17:05.996582+0000 mon.a (mon.0) 3255 : audit [INF] from='client.? 172.21.15.100:0/3056095998' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T17:17:07.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:17:07 smithi100 bash[76202]: cluster 2024-04-07T17:17:06.009784+0000 mon.a (mon.0) 3256 : cluster [DBG] osdmap e1689: 8 total, 8 up, 8 in 2024-04-07T17:17:08.368 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:17:08 smithi144 bash[66654]: cluster 2024-04-07T17:17:07.016968+0000 mon.a (mon.0) 3257 : cluster [DBG] osdmap e1690: 8 total, 8 up, 8 in 2024-04-07T17:17:08.368 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:17:08 smithi144 bash[66654]: audit 2024-04-07T17:17:07.022831+0000 mon.c (mon.2) 80 : audit [INF] from='client.? 172.21.15.100:0/3348498784' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:17:08.368 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:17:08 smithi144 bash[66654]: audit 2024-04-07T17:17:07.025643+0000 mon.a (mon.0) 3258 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:17:08.368 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:17:08 smithi144 bash[66654]: cluster 2024-04-07T17:17:07.782359+0000 mgr.y (mgr.25175) 1853 : cluster [DBG] pgmap v3118: 617 pgs: 6 creating+peering, 45 unknown, 566 active+clean; 301 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 392 KiB/s rd, 8.7 KiB/s wr, 481 op/s 2024-04-07T17:17:08.368 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:17:08 smithi144 bash[66654]: audit 2024-04-07T17:17:07.861468+0000 mon.a (mon.0) 3259 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T17:17:08.368 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:17:08 smithi144 bash[66654]: cluster 2024-04-07T17:17:07.869199+0000 mon.a (mon.0) 3260 : cluster [DBG] osdmap e1691: 8 total, 8 up, 8 in 2024-04-07T17:17:08.428 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:17:08 smithi100 bash[74942]: cluster 2024-04-07T17:17:07.016968+0000 mon.a (mon.0) 3257 : cluster [DBG] osdmap e1690: 8 total, 8 up, 8 in 2024-04-07T17:17:08.428 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:17:08 smithi100 bash[74942]: audit 2024-04-07T17:17:07.022831+0000 mon.c (mon.2) 80 : audit [INF] from='client.? 172.21.15.100:0/3348498784' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:17:08.428 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:17:08 smithi100 bash[74942]: audit 2024-04-07T17:17:07.025643+0000 mon.a (mon.0) 3258 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:17:08.429 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:17:08 smithi100 bash[74942]: cluster 2024-04-07T17:17:07.782359+0000 mgr.y (mgr.25175) 1853 : cluster [DBG] pgmap v3118: 617 pgs: 6 creating+peering, 45 unknown, 566 active+clean; 301 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 392 KiB/s rd, 8.7 KiB/s wr, 481 op/s 2024-04-07T17:17:08.429 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:17:08 smithi100 bash[74942]: audit 2024-04-07T17:17:07.861468+0000 mon.a (mon.0) 3259 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T17:17:08.429 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:17:08 smithi100 bash[74942]: cluster 2024-04-07T17:17:07.869199+0000 mon.a (mon.0) 3260 : cluster [DBG] osdmap e1691: 8 total, 8 up, 8 in 2024-04-07T17:17:08.429 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:17:08 smithi100 bash[76202]: cluster 2024-04-07T17:17:07.016968+0000 mon.a (mon.0) 3257 : cluster [DBG] osdmap e1690: 8 total, 8 up, 8 in 2024-04-07T17:17:08.429 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:17:08 smithi100 bash[76202]: audit 2024-04-07T17:17:07.022831+0000 mon.c (mon.2) 80 : audit [INF] from='client.? 172.21.15.100:0/3348498784' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:17:08.429 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:17:08 smithi100 bash[76202]: audit 2024-04-07T17:17:07.025643+0000 mon.a (mon.0) 3258 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:17:08.429 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:17:08 smithi100 bash[76202]: cluster 2024-04-07T17:17:07.782359+0000 mgr.y (mgr.25175) 1853 : cluster [DBG] pgmap v3118: 617 pgs: 6 creating+peering, 45 unknown, 566 active+clean; 301 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 392 KiB/s rd, 8.7 KiB/s wr, 481 op/s 2024-04-07T17:17:08.429 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:17:08 smithi100 bash[76202]: audit 2024-04-07T17:17:07.861468+0000 mon.a (mon.0) 3259 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T17:17:08.429 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:17:08 smithi100 bash[76202]: cluster 2024-04-07T17:17:07.869199+0000 mon.a (mon.0) 3260 : cluster [DBG] osdmap e1691: 8 total, 8 up, 8 in 2024-04-07T17:17:08.704 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:17:08 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:17:08] "GET /metrics HTTP/1.1" 200 68131 "" "Prometheus/2.43.0" 2024-04-07T17:17:10.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:17:09 smithi100 bash[74942]: cluster 2024-04-07T17:17:08.873259+0000 mon.a (mon.0) 3261 : cluster [DBG] osdmap e1692: 8 total, 8 up, 8 in 2024-04-07T17:17:10.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:17:09 smithi100 bash[74942]: audit 2024-04-07T17:17:08.875923+0000 mon.c (mon.2) 81 : audit [INF] from='client.? 172.21.15.100:0/1421404350' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:17:10.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:17:09 smithi100 bash[74942]: audit 2024-04-07T17:17:08.883209+0000 mon.a (mon.0) 3262 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:17:10.205 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:17:09 smithi100 bash[76202]: cluster 2024-04-07T17:17:08.873259+0000 mon.a (mon.0) 3261 : cluster [DBG] osdmap e1692: 8 total, 8 up, 8 in 2024-04-07T17:17:10.205 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:17:09 smithi100 bash[76202]: audit 2024-04-07T17:17:08.875923+0000 mon.c (mon.2) 81 : audit [INF] from='client.? 172.21.15.100:0/1421404350' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:17:10.205 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:17:09 smithi100 bash[76202]: audit 2024-04-07T17:17:08.883209+0000 mon.a (mon.0) 3262 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:17:10.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:17:09 smithi144 bash[66654]: cluster 2024-04-07T17:17:08.873259+0000 mon.a (mon.0) 3261 : cluster [DBG] osdmap e1692: 8 total, 8 up, 8 in 2024-04-07T17:17:10.368 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:17:09 smithi144 bash[66654]: audit 2024-04-07T17:17:08.875923+0000 mon.c (mon.2) 81 : audit [INF] from='client.? 172.21.15.100:0/1421404350' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:17:10.368 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:17:09 smithi144 bash[66654]: audit 2024-04-07T17:17:08.883209+0000 mon.a (mon.0) 3262 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:17:11.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:17:10 smithi100 bash[76202]: cluster 2024-04-07T17:17:09.785299+0000 mgr.y (mgr.25175) 1854 : cluster [DBG] pgmap v3121: 649 pgs: 17 creating+peering, 39 unknown, 593 active+clean; 301 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 130 KiB/s rd, 1.5 KiB/s wr, 157 op/s 2024-04-07T17:17:11.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:17:10 smithi100 bash[76202]: audit 2024-04-07T17:17:09.868641+0000 mon.a (mon.0) 3263 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T17:17:11.205 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:17:10 smithi100 bash[76202]: cluster 2024-04-07T17:17:09.877632+0000 mon.a (mon.0) 3264 : cluster [DBG] osdmap e1693: 8 total, 8 up, 8 in 2024-04-07T17:17:11.205 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:17:10 smithi100 bash[76202]: audit 2024-04-07T17:17:09.920240+0000 mon.c (mon.2) 82 : audit [INF] from='client.? 172.21.15.100:0/3213618994' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-librbd-smithi100-115303-29", "tierpool":"test-librbd-smithi100-115303-31", "force_nonempty":""}]: dispatch 2024-04-07T17:17:11.205 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:17:10 smithi100 bash[76202]: audit 2024-04-07T17:17:09.921569+0000 mon.a (mon.0) 3265 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-librbd-smithi100-115303-29", "tierpool":"test-librbd-smithi100-115303-31", "force_nonempty":""}]: dispatch 2024-04-07T17:17:11.205 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:17:10 smithi100 bash[74942]: cluster 2024-04-07T17:17:09.785299+0000 mgr.y (mgr.25175) 1854 : cluster [DBG] pgmap v3121: 649 pgs: 17 creating+peering, 39 unknown, 593 active+clean; 301 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 130 KiB/s rd, 1.5 KiB/s wr, 157 op/s 2024-04-07T17:17:11.205 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:17:10 smithi100 bash[74942]: audit 2024-04-07T17:17:09.868641+0000 mon.a (mon.0) 3263 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T17:17:11.205 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:17:10 smithi100 bash[74942]: cluster 2024-04-07T17:17:09.877632+0000 mon.a (mon.0) 3264 : cluster [DBG] osdmap e1693: 8 total, 8 up, 8 in 2024-04-07T17:17:11.205 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:17:10 smithi100 bash[74942]: audit 2024-04-07T17:17:09.920240+0000 mon.c (mon.2) 82 : audit [INF] from='client.? 172.21.15.100:0/3213618994' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-librbd-smithi100-115303-29", "tierpool":"test-librbd-smithi100-115303-31", "force_nonempty":""}]: dispatch 2024-04-07T17:17:11.205 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:17:10 smithi100 bash[74942]: audit 2024-04-07T17:17:09.921569+0000 mon.a (mon.0) 3265 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-librbd-smithi100-115303-29", "tierpool":"test-librbd-smithi100-115303-31", "force_nonempty":""}]: dispatch 2024-04-07T17:17:11.368 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:17:10 smithi144 bash[66654]: cluster 2024-04-07T17:17:09.785299+0000 mgr.y (mgr.25175) 1854 : cluster [DBG] pgmap v3121: 649 pgs: 17 creating+peering, 39 unknown, 593 active+clean; 301 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 130 KiB/s rd, 1.5 KiB/s wr, 157 op/s 2024-04-07T17:17:11.368 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:17:10 smithi144 bash[66654]: audit 2024-04-07T17:17:09.868641+0000 mon.a (mon.0) 3263 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T17:17:11.368 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:17:10 smithi144 bash[66654]: cluster 2024-04-07T17:17:09.877632+0000 mon.a (mon.0) 3264 : cluster [DBG] osdmap e1693: 8 total, 8 up, 8 in 2024-04-07T17:17:11.368 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:17:10 smithi144 bash[66654]: audit 2024-04-07T17:17:09.920240+0000 mon.c (mon.2) 82 : audit [INF] from='client.? 172.21.15.100:0/3213618994' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-librbd-smithi100-115303-29", "tierpool":"test-librbd-smithi100-115303-31", "force_nonempty":""}]: dispatch 2024-04-07T17:17:11.368 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:17:10 smithi144 bash[66654]: audit 2024-04-07T17:17:09.921569+0000 mon.a (mon.0) 3265 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-librbd-smithi100-115303-29", "tierpool":"test-librbd-smithi100-115303-31", "force_nonempty":""}]: dispatch 2024-04-07T17:17:12.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:17:11 smithi100 bash[74942]: audit 2024-04-07T17:17:10.902409+0000 mon.a (mon.0) 3266 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-librbd-smithi100-115303-29", "tierpool":"test-librbd-smithi100-115303-31", "force_nonempty":""}]': finished 2024-04-07T17:17:12.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:17:11 smithi100 bash[74942]: cluster 2024-04-07T17:17:10.909562+0000 mon.a (mon.0) 3267 : cluster [DBG] osdmap e1694: 8 total, 8 up, 8 in 2024-04-07T17:17:12.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:17:11 smithi100 bash[74942]: audit 2024-04-07T17:17:10.912586+0000 mon.c (mon.2) 83 : audit [INF] from='client.? 172.21.15.100:0/3213618994' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-librbd-smithi100-115303-31", "mode":"writeback"}]: dispatch 2024-04-07T17:17:12.205 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:17:11 smithi100 bash[74942]: audit 2024-04-07T17:17:10.937335+0000 mon.a (mon.0) 3268 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-librbd-smithi100-115303-31", "mode":"writeback"}]: dispatch 2024-04-07T17:17:12.205 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:17:11 smithi100 bash[76202]: audit 2024-04-07T17:17:10.902409+0000 mon.a (mon.0) 3266 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-librbd-smithi100-115303-29", "tierpool":"test-librbd-smithi100-115303-31", "force_nonempty":""}]': finished 2024-04-07T17:17:12.205 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:17:11 smithi100 bash[76202]: cluster 2024-04-07T17:17:10.909562+0000 mon.a (mon.0) 3267 : cluster [DBG] osdmap e1694: 8 total, 8 up, 8 in 2024-04-07T17:17:12.205 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:17:11 smithi100 bash[76202]: audit 2024-04-07T17:17:10.912586+0000 mon.c (mon.2) 83 : audit [INF] from='client.? 172.21.15.100:0/3213618994' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-librbd-smithi100-115303-31", "mode":"writeback"}]: dispatch 2024-04-07T17:17:12.205 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:17:11 smithi100 bash[76202]: audit 2024-04-07T17:17:10.937335+0000 mon.a (mon.0) 3268 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-librbd-smithi100-115303-31", "mode":"writeback"}]: dispatch 2024-04-07T17:17:12.317 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:17:11 smithi144 bash[66654]: audit 2024-04-07T17:17:10.902409+0000 mon.a (mon.0) 3266 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-librbd-smithi100-115303-29", "tierpool":"test-librbd-smithi100-115303-31", "force_nonempty":""}]': finished 2024-04-07T17:17:12.318 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:17:11 smithi144 bash[66654]: cluster 2024-04-07T17:17:10.909562+0000 mon.a (mon.0) 3267 : cluster [DBG] osdmap e1694: 8 total, 8 up, 8 in 2024-04-07T17:17:12.318 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:17:11 smithi144 bash[66654]: audit 2024-04-07T17:17:10.912586+0000 mon.c (mon.2) 83 : audit [INF] from='client.? 172.21.15.100:0/3213618994' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-librbd-smithi100-115303-31", "mode":"writeback"}]: dispatch 2024-04-07T17:17:12.318 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:17:11 smithi144 bash[66654]: audit 2024-04-07T17:17:10.937335+0000 mon.a (mon.0) 3268 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-librbd-smithi100-115303-31", "mode":"writeback"}]: dispatch 2024-04-07T17:17:12.617 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:17:12 smithi144 bash[95233]: ts=2024-04-07T17:17:12.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:17:12.888 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:17:13.205 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:17:12 smithi100 bash[74942]: cluster 2024-04-07T17:17:11.786459+0000 mgr.y (mgr.25175) 1855 : cluster [DBG] pgmap v3124: 649 pgs: 17 creating+peering, 24 unknown, 608 active+clean; 301 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 36 KiB/s rd, 766 B/s wr, 45 op/s 2024-04-07T17:17:13.205 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:17:12 smithi100 bash[74942]: cluster 2024-04-07T17:17:11.919579+0000 mon.a (mon.0) 3269 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-04-07T17:17:13.205 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:17:12 smithi100 bash[74942]: cluster 2024-04-07T17:17:11.920763+0000 mon.a (mon.0) 3270 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T17:17:13.205 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:17:12 smithi100 bash[74942]: audit 2024-04-07T17:17:11.922306+0000 mon.a (mon.0) 3271 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-librbd-smithi100-115303-31", "mode":"writeback"}]': finished 2024-04-07T17:17:13.205 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:17:12 smithi100 bash[74942]: cluster 2024-04-07T17:17:11.930907+0000 mon.a (mon.0) 3272 : cluster [DBG] osdmap e1695: 8 total, 8 up, 8 in 2024-04-07T17:17:13.205 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:17:12 smithi100 bash[74942]: audit 2024-04-07T17:17:11.932588+0000 mon.c (mon.2) 84 : audit [INF] from='client.? 172.21.15.100:0/3213618994' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-librbd-smithi100-115303-29", "overlaypool":"test-librbd-smithi100-115303-31"}]: dispatch 2024-04-07T17:17:13.205 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:17:12 smithi100 bash[74942]: audit 2024-04-07T17:17:11.936006+0000 mon.a (mon.0) 3273 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-librbd-smithi100-115303-29", "overlaypool":"test-librbd-smithi100-115303-31"}]: dispatch 2024-04-07T17:17:13.205 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:17:12 smithi100 bash[74942]: audit 2024-04-07T17:17:12.864306+0000 mon.a (mon.0) 3274 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-librbd-smithi100-115303-29", "overlaypool":"test-librbd-smithi100-115303-31"}]': finished 2024-04-07T17:17:13.205 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:17:12 smithi100 bash[74942]: cluster 2024-04-07T17:17:12.875335+0000 mon.a (mon.0) 3275 : cluster [DBG] osdmap e1696: 8 total, 8 up, 8 in 2024-04-07T17:17:13.206 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:17:12 smithi100 bash[76202]: cluster 2024-04-07T17:17:11.786459+0000 mgr.y (mgr.25175) 1855 : cluster [DBG] pgmap v3124: 649 pgs: 17 creating+peering, 24 unknown, 608 active+clean; 301 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 36 KiB/s rd, 766 B/s wr, 45 op/s 2024-04-07T17:17:13.206 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:17:12 smithi100 bash[76202]: cluster 2024-04-07T17:17:11.919579+0000 mon.a (mon.0) 3269 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-04-07T17:17:13.206 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:17:12 smithi100 bash[76202]: cluster 2024-04-07T17:17:11.920763+0000 mon.a (mon.0) 3270 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T17:17:13.206 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:17:12 smithi100 bash[76202]: audit 2024-04-07T17:17:11.922306+0000 mon.a (mon.0) 3271 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-librbd-smithi100-115303-31", "mode":"writeback"}]': finished 2024-04-07T17:17:13.206 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:17:12 smithi100 bash[76202]: cluster 2024-04-07T17:17:11.930907+0000 mon.a (mon.0) 3272 : cluster [DBG] osdmap e1695: 8 total, 8 up, 8 in 2024-04-07T17:17:13.206 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:17:12 smithi100 bash[76202]: audit 2024-04-07T17:17:11.932588+0000 mon.c (mon.2) 84 : audit [INF] from='client.? 172.21.15.100:0/3213618994' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-librbd-smithi100-115303-29", "overlaypool":"test-librbd-smithi100-115303-31"}]: dispatch 2024-04-07T17:17:13.206 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:17:12 smithi100 bash[76202]: audit 2024-04-07T17:17:11.936006+0000 mon.a (mon.0) 3273 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-librbd-smithi100-115303-29", "overlaypool":"test-librbd-smithi100-115303-31"}]: dispatch 2024-04-07T17:17:13.206 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:17:12 smithi100 bash[76202]: audit 2024-04-07T17:17:12.864306+0000 mon.a (mon.0) 3274 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-librbd-smithi100-115303-29", "overlaypool":"test-librbd-smithi100-115303-31"}]': finished 2024-04-07T17:17:13.206 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:17:12 smithi100 bash[76202]: cluster 2024-04-07T17:17:12.875335+0000 mon.a (mon.0) 3275 : cluster [DBG] osdmap e1696: 8 total, 8 up, 8 in 2024-04-07T17:17:13.368 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:17:12 smithi144 bash[66654]: cluster 2024-04-07T17:17:11.786459+0000 mgr.y (mgr.25175) 1855 : cluster [DBG] pgmap v3124: 649 pgs: 17 creating+peering, 24 unknown, 608 active+clean; 301 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 36 KiB/s rd, 766 B/s wr, 45 op/s 2024-04-07T17:17:13.368 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:17:12 smithi144 bash[66654]: cluster 2024-04-07T17:17:11.919579+0000 mon.a (mon.0) 3269 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-04-07T17:17:13.368 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:17:12 smithi144 bash[66654]: cluster 2024-04-07T17:17:11.920763+0000 mon.a (mon.0) 3270 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T17:17:13.368 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:17:12 smithi144 bash[66654]: audit 2024-04-07T17:17:11.922306+0000 mon.a (mon.0) 3271 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-librbd-smithi100-115303-31", "mode":"writeback"}]': finished 2024-04-07T17:17:13.368 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:17:12 smithi144 bash[66654]: cluster 2024-04-07T17:17:11.930907+0000 mon.a (mon.0) 3272 : cluster [DBG] osdmap e1695: 8 total, 8 up, 8 in 2024-04-07T17:17:13.368 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:17:12 smithi144 bash[66654]: audit 2024-04-07T17:17:11.932588+0000 mon.c (mon.2) 84 : audit [INF] from='client.? 172.21.15.100:0/3213618994' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-librbd-smithi100-115303-29", "overlaypool":"test-librbd-smithi100-115303-31"}]: dispatch 2024-04-07T17:17:13.368 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:17:12 smithi144 bash[66654]: audit 2024-04-07T17:17:11.936006+0000 mon.a (mon.0) 3273 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-librbd-smithi100-115303-29", "overlaypool":"test-librbd-smithi100-115303-31"}]: dispatch 2024-04-07T17:17:13.368 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:17:12 smithi144 bash[66654]: audit 2024-04-07T17:17:12.864306+0000 mon.a (mon.0) 3274 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-librbd-smithi100-115303-29", "overlaypool":"test-librbd-smithi100-115303-31"}]': finished 2024-04-07T17:17:13.368 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:17:12 smithi144 bash[66654]: cluster 2024-04-07T17:17:12.875335+0000 mon.a (mon.0) 3275 : cluster [DBG] osdmap e1696: 8 total, 8 up, 8 in 2024-04-07T17:17:15.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:17:14 smithi100 bash[74942]: cluster 2024-04-07T17:17:13.788296+0000 mgr.y (mgr.25175) 1856 : cluster [DBG] pgmap v3127: 649 pgs: 11 creating+peering, 638 active+clean; 301 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail 2024-04-07T17:17:15.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:17:14 smithi100 bash[74942]: cluster 2024-04-07T17:17:13.957565+0000 mon.a (mon.0) 3276 : cluster [DBG] osdmap e1697: 8 total, 8 up, 8 in 2024-04-07T17:17:15.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:17:14 smithi100 bash[76202]: cluster 2024-04-07T17:17:13.788296+0000 mgr.y (mgr.25175) 1856 : cluster [DBG] pgmap v3127: 649 pgs: 11 creating+peering, 638 active+clean; 301 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail 2024-04-07T17:17:15.205 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:17:14 smithi100 bash[76202]: cluster 2024-04-07T17:17:13.957565+0000 mon.a (mon.0) 3276 : cluster [DBG] osdmap e1697: 8 total, 8 up, 8 in 2024-04-07T17:17:15.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:17:14 smithi144 bash[66654]: cluster 2024-04-07T17:17:13.788296+0000 mgr.y (mgr.25175) 1856 : cluster [DBG] pgmap v3127: 649 pgs: 11 creating+peering, 638 active+clean; 301 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail 2024-04-07T17:17:15.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:17:14 smithi144 bash[66654]: cluster 2024-04-07T17:17:13.957565+0000 mon.a (mon.0) 3276 : cluster [DBG] osdmap e1697: 8 total, 8 up, 8 in 2024-04-07T17:17:16.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:17:15 smithi144 bash[66654]: cluster 2024-04-07T17:17:14.967663+0000 mon.a (mon.0) 3277 : cluster [DBG] osdmap e1698: 8 total, 8 up, 8 in 2024-04-07T17:17:16.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:17:15 smithi100 bash[74942]: cluster 2024-04-07T17:17:14.967663+0000 mon.a (mon.0) 3277 : cluster [DBG] osdmap e1698: 8 total, 8 up, 8 in 2024-04-07T17:17:16.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:17:15 smithi100 bash[76202]: cluster 2024-04-07T17:17:14.967663+0000 mon.a (mon.0) 3277 : cluster [DBG] osdmap e1698: 8 total, 8 up, 8 in 2024-04-07T17:17:17.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:17:17 smithi144 bash[66654]: cluster 2024-04-07T17:17:15.790157+0000 mgr.y (mgr.25175) 1857 : cluster [DBG] pgmap v3130: 649 pgs: 649 active+clean; 301 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 255 B/s wr, 0 op/s 2024-04-07T17:17:17.368 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:17:17 smithi144 bash[66654]: cluster 2024-04-07T17:17:15.971900+0000 mon.a (mon.0) 3278 : cluster [DBG] osdmap e1699: 8 total, 8 up, 8 in 2024-04-07T17:17:17.368 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:17:17 smithi144 bash[66654]: audit 2024-04-07T17:17:16.834304+0000 mon.a (mon.0) 3279 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:17:17.368 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:17:17 smithi144 bash[66654]: audit 2024-04-07T17:17:16.837583+0000 mon.a (mon.0) 3280 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:17:17.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:17:17 smithi100 bash[74942]: cluster 2024-04-07T17:17:15.790157+0000 mgr.y (mgr.25175) 1857 : cluster [DBG] pgmap v3130: 649 pgs: 649 active+clean; 301 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 255 B/s wr, 0 op/s 2024-04-07T17:17:17.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:17:17 smithi100 bash[74942]: cluster 2024-04-07T17:17:15.971900+0000 mon.a (mon.0) 3278 : cluster [DBG] osdmap e1699: 8 total, 8 up, 8 in 2024-04-07T17:17:17.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:17:17 smithi100 bash[74942]: audit 2024-04-07T17:17:16.834304+0000 mon.a (mon.0) 3279 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:17:17.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:17:17 smithi100 bash[74942]: audit 2024-04-07T17:17:16.837583+0000 mon.a (mon.0) 3280 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:17:17.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:17:17 smithi100 bash[76202]: cluster 2024-04-07T17:17:15.790157+0000 mgr.y (mgr.25175) 1857 : cluster [DBG] pgmap v3130: 649 pgs: 649 active+clean; 301 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 255 B/s wr, 0 op/s 2024-04-07T17:17:17.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:17:17 smithi100 bash[76202]: cluster 2024-04-07T17:17:15.971900+0000 mon.a (mon.0) 3278 : cluster [DBG] osdmap e1699: 8 total, 8 up, 8 in 2024-04-07T17:17:17.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:17:17 smithi100 bash[76202]: audit 2024-04-07T17:17:16.834304+0000 mon.a (mon.0) 3279 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:17:17.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:17:17 smithi100 bash[76202]: audit 2024-04-07T17:17:16.837583+0000 mon.a (mon.0) 3280 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:17:18.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:17:18 smithi144 bash[66654]: cluster 2024-04-07T17:17:16.994041+0000 mon.a (mon.0) 3281 : cluster [DBG] osdmap e1700: 8 total, 8 up, 8 in 2024-04-07T17:17:18.368 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:17:18 smithi144 bash[66654]: cluster 2024-04-07T17:17:17.794267+0000 mgr.y (mgr.25175) 1858 : cluster [DBG] pgmap v3133: 649 pgs: 649 active+clean; 301 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 2.5 KiB/s rd, 767 B/s wr, 5 op/s 2024-04-07T17:17:18.428 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:17:18 smithi100 bash[76202]: cluster 2024-04-07T17:17:16.994041+0000 mon.a (mon.0) 3281 : cluster [DBG] osdmap e1700: 8 total, 8 up, 8 in 2024-04-07T17:17:18.429 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:17:18 smithi100 bash[76202]: cluster 2024-04-07T17:17:17.794267+0000 mgr.y (mgr.25175) 1858 : cluster [DBG] pgmap v3133: 649 pgs: 649 active+clean; 301 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 2.5 KiB/s rd, 767 B/s wr, 5 op/s 2024-04-07T17:17:18.429 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:17:18 smithi100 bash[74942]: cluster 2024-04-07T17:17:16.994041+0000 mon.a (mon.0) 3281 : cluster [DBG] osdmap e1700: 8 total, 8 up, 8 in 2024-04-07T17:17:18.429 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:17:18 smithi100 bash[74942]: cluster 2024-04-07T17:17:17.794267+0000 mgr.y (mgr.25175) 1858 : cluster [DBG] pgmap v3133: 649 pgs: 649 active+clean; 301 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 2.5 KiB/s rd, 767 B/s wr, 5 op/s 2024-04-07T17:17:18.704 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:17:18 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:17:18] "GET /metrics HTTP/1.1" 200 75547 "" "Prometheus/2.43.0" 2024-04-07T17:17:19.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:17:19 smithi144 bash[66654]: cluster 2024-04-07T17:17:18.027943+0000 mon.a (mon.0) 3282 : cluster [DBG] osdmap e1701: 8 total, 8 up, 8 in 2024-04-07T17:17:19.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:17:19 smithi100 bash[74942]: cluster 2024-04-07T17:17:18.027943+0000 mon.a (mon.0) 3282 : cluster [DBG] osdmap e1701: 8 total, 8 up, 8 in 2024-04-07T17:17:19.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:17:19 smithi100 bash[76202]: cluster 2024-04-07T17:17:18.027943+0000 mon.a (mon.0) 3282 : cluster [DBG] osdmap e1701: 8 total, 8 up, 8 in 2024-04-07T17:17:19.492 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:17:19.492 INFO:tasks.workunit.client.0.smithi100.stdout:trying to find test-librbd-smithi100-115303-30/image118 2024-04-07T17:17:19.492 INFO:tasks.workunit.client.0.smithi100.stdout:checking test-librbd-smithi100-115303-30/image118 2024-04-07T17:17:19.492 INFO:tasks.workunit.client.0.smithi100.stdout:found child test-librbd-smithi100-115303-30/image118 2024-04-07T17:17:19.493 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:17:19.493 INFO:tasks.workunit.client.0.smithi100.stdout:num children is: 1 2024-04-07T17:17:19.493 INFO:tasks.workunit.client.0.smithi100.stdout:expected: 1 2024-04-07T17:17:19.493 INFO:tasks.workunit.client.0.smithi100.stdout:child: image118 2024-04-07T17:17:19.493 INFO:tasks.workunit.client.0.smithi100.stdout:found child test-librbd-smithi100-115303-30/image118/b1868f71ca7e 2024-04-07T17:17:19.493 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:17:19.493 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:17:19.493 INFO:tasks.workunit.client.0.smithi100.stdout:trying to find test-librbd-smithi100-115303-30/image118 2024-04-07T17:17:19.493 INFO:tasks.workunit.client.0.smithi100.stdout:checking test-librbd-smithi100-115303-29/image119 2024-04-07T17:17:19.493 INFO:tasks.workunit.client.0.smithi100.stdout:checking test-librbd-smithi100-115303-30/image118 2024-04-07T17:17:19.493 INFO:tasks.workunit.client.0.smithi100.stdout:found child test-librbd-smithi100-115303-30/image118 2024-04-07T17:17:19.493 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:17:19.493 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:17:19.494 INFO:tasks.workunit.client.0.smithi100.stdout:trying to find test-librbd-smithi100-115303-29/image119 2024-04-07T17:17:19.494 INFO:tasks.workunit.client.0.smithi100.stdout:checking test-librbd-smithi100-115303-29/image119 2024-04-07T17:17:19.494 INFO:tasks.workunit.client.0.smithi100.stdout:found child test-librbd-smithi100-115303-29/image119 2024-04-07T17:17:19.494 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:17:19.494 INFO:tasks.workunit.client.0.smithi100.stdout:num children is: 2 2024-04-07T17:17:19.494 INFO:tasks.workunit.client.0.smithi100.stdout:expected: 2 2024-04-07T17:17:19.494 INFO:tasks.workunit.client.0.smithi100.stdout:child: image119 2024-04-07T17:17:19.494 INFO:tasks.workunit.client.0.smithi100.stdout:child: image118 2024-04-07T17:17:19.494 INFO:tasks.workunit.client.0.smithi100.stdout:found child test-librbd-smithi100-115303-30/image118/b1868f71ca7e 2024-04-07T17:17:19.494 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:17:19.494 INFO:tasks.workunit.client.0.smithi100.stdout:found child test-librbd-smithi100-115303-29/image119/b18678173feb 2024-04-07T17:17:19.494 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:17:19.494 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:17:19.494 INFO:tasks.workunit.client.0.smithi100.stdout:trying to find test-librbd-smithi100-115303-30/image118 2024-04-07T17:17:19.495 INFO:tasks.workunit.client.0.smithi100.stdout:checking test-librbd-smithi100-115303-29/image119 2024-04-07T17:17:19.495 INFO:tasks.workunit.client.0.smithi100.stdout:checking test-librbd-smithi100-115303-30/image118 2024-04-07T17:17:19.495 INFO:tasks.workunit.client.0.smithi100.stdout:found child test-librbd-smithi100-115303-30/image118 2024-04-07T17:17:19.495 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:17:19.495 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:17:19.495 INFO:tasks.workunit.client.0.smithi100.stdout:trying to find test-librbd-smithi100-115303-29/image119 2024-04-07T17:17:19.495 INFO:tasks.workunit.client.0.smithi100.stdout:checking test-librbd-smithi100-115303-29/image119 2024-04-07T17:17:19.495 INFO:tasks.workunit.client.0.smithi100.stdout:found child test-librbd-smithi100-115303-29/image119 2024-04-07T17:17:19.495 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:17:19.495 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:17:19.495 INFO:tasks.workunit.client.0.smithi100.stdout:trying to find test-librbd-smithi100-115303-30/image120 2024-04-07T17:17:19.495 INFO:tasks.workunit.client.0.smithi100.stdout:checking test-librbd-smithi100-115303-29/image119 2024-04-07T17:17:19.495 INFO:tasks.workunit.client.0.smithi100.stdout:checking test-librbd-smithi100-115303-30/image118 2024-04-07T17:17:19.495 INFO:tasks.workunit.client.0.smithi100.stdout:checking test-librbd-smithi100-115303-30/image120 2024-04-07T17:17:19.495 INFO:tasks.workunit.client.0.smithi100.stdout:found child test-librbd-smithi100-115303-30/image120 2024-04-07T17:17:19.495 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:17:19.495 INFO:tasks.workunit.client.0.smithi100.stdout:num children is: 3 2024-04-07T17:17:19.496 INFO:tasks.workunit.client.0.smithi100.stdout:expected: 3 2024-04-07T17:17:19.496 INFO:tasks.workunit.client.0.smithi100.stdout:child: image119 2024-04-07T17:17:19.496 INFO:tasks.workunit.client.0.smithi100.stdout:child: image118 2024-04-07T17:17:19.496 INFO:tasks.workunit.client.0.smithi100.stdout:child: image120 2024-04-07T17:17:19.496 INFO:tasks.workunit.client.0.smithi100.stdout:found child test-librbd-smithi100-115303-30/image118/b1868f71ca7e 2024-04-07T17:17:19.496 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:17:19.496 INFO:tasks.workunit.client.0.smithi100.stdout:found child test-librbd-smithi100-115303-29/image119/b18678173feb 2024-04-07T17:17:19.496 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:17:19.496 INFO:tasks.workunit.client.0.smithi100.stdout:found child test-librbd-smithi100-115303-30/image120/b186607ff13a 2024-04-07T17:17:19.496 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:17:19.496 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:17:19.496 INFO:tasks.workunit.client.0.smithi100.stdout:trying to find test-librbd-smithi100-115303-30/image118 2024-04-07T17:17:19.496 INFO:tasks.workunit.client.0.smithi100.stdout:checking test-librbd-smithi100-115303-29/image119 2024-04-07T17:17:19.496 INFO:tasks.workunit.client.0.smithi100.stdout:checking test-librbd-smithi100-115303-30/image118 2024-04-07T17:17:19.496 INFO:tasks.workunit.client.0.smithi100.stdout:found child test-librbd-smithi100-115303-30/image118 2024-04-07T17:17:19.496 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:17:19.497 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:17:19.497 INFO:tasks.workunit.client.0.smithi100.stdout:trying to find test-librbd-smithi100-115303-29/image119 2024-04-07T17:17:19.497 INFO:tasks.workunit.client.0.smithi100.stdout:checking test-librbd-smithi100-115303-29/image119 2024-04-07T17:17:19.497 INFO:tasks.workunit.client.0.smithi100.stdout:found child test-librbd-smithi100-115303-29/image119 2024-04-07T17:17:19.497 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:17:19.497 INFO:tasks.workunit.client.0.smithi100.stdout:num children is: 3 2024-04-07T17:17:19.497 INFO:tasks.workunit.client.0.smithi100.stdout:expected: 3 2024-04-07T17:17:19.497 INFO:tasks.workunit.client.0.smithi100.stdout:child: image119 2024-04-07T17:17:19.497 INFO:tasks.workunit.client.0.smithi100.stdout:child: image118 2024-04-07T17:17:19.497 INFO:tasks.workunit.client.0.smithi100.stdout:child: image120 2024-04-07T17:17:19.497 INFO:tasks.workunit.client.0.smithi100.stdout:found child test-librbd-smithi100-115303-30/image118/b1868f71ca7e 2024-04-07T17:17:19.497 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:17:19.497 INFO:tasks.workunit.client.0.smithi100.stdout:found child test-librbd-smithi100-115303-29/image119/b18678173feb 2024-04-07T17:17:19.497 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:17:19.497 INFO:tasks.workunit.client.0.smithi100.stdout:found child test-librbd-smithi100-115303-30/image120/b186607ff13a 2024-04-07T17:17:19.498 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:17:19.498 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:17:19.498 INFO:tasks.workunit.client.0.smithi100.stdout:trying to find test-librbd-smithi100-115303-30/image118 2024-04-07T17:17:19.498 INFO:tasks.workunit.client.0.smithi100.stdout:checking test-librbd-smithi100-115303-29/image119 2024-04-07T17:17:19.498 INFO:tasks.workunit.client.0.smithi100.stdout:checking test-librbd-smithi100-115303-30/image118 2024-04-07T17:17:19.498 INFO:tasks.workunit.client.0.smithi100.stdout:found child test-librbd-smithi100-115303-30/image118 2024-04-07T17:17:19.498 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:17:19.498 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:17:19.498 INFO:tasks.workunit.client.0.smithi100.stdout:trying to find test-librbd-smithi100-115303-29/image119 2024-04-07T17:17:19.498 INFO:tasks.workunit.client.0.smithi100.stdout:checking test-librbd-smithi100-115303-29/image119 2024-04-07T17:17:19.498 INFO:tasks.workunit.client.0.smithi100.stdout:found child test-librbd-smithi100-115303-29/image119 2024-04-07T17:17:19.498 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:17:19.498 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:17:19.499 INFO:tasks.workunit.client.0.smithi100.stdout:trying to find test-librbd-smithi100-115303-30/image121 2024-04-07T17:17:19.499 INFO:tasks.workunit.client.0.smithi100.stdout:checking test-librbd-smithi100-115303-29/image119 2024-04-07T17:17:19.499 INFO:tasks.workunit.client.0.smithi100.stdout:checking test-librbd-smithi100-115303-30/image118 2024-04-07T17:17:19.499 INFO:tasks.workunit.client.0.smithi100.stdout:checking test-librbd-smithi100-115303-30/image121 2024-04-07T17:17:19.499 INFO:tasks.workunit.client.0.smithi100.stdout:found child test-librbd-smithi100-115303-30/image121 2024-04-07T17:17:19.499 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:17:19.499 INFO:tasks.workunit.client.0.smithi100.stdout:num children is: 4 2024-04-07T17:17:19.499 INFO:tasks.workunit.client.0.smithi100.stdout:expected: 4 2024-04-07T17:17:19.499 INFO:tasks.workunit.client.0.smithi100.stdout:child: image119 2024-04-07T17:17:19.499 INFO:tasks.workunit.client.0.smithi100.stdout:child: image118 2024-04-07T17:17:19.499 INFO:tasks.workunit.client.0.smithi100.stdout:child: image120 2024-04-07T17:17:19.499 INFO:tasks.workunit.client.0.smithi100.stdout:child: image121 2024-04-07T17:17:19.499 INFO:tasks.workunit.client.0.smithi100.stdout:found child test-librbd-smithi100-115303-30/image118/b1868f71ca7e 2024-04-07T17:17:19.499 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:17:19.499 INFO:tasks.workunit.client.0.smithi100.stdout:found child test-librbd-smithi100-115303-29/image119/b18678173feb 2024-04-07T17:17:19.499 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:17:19.500 INFO:tasks.workunit.client.0.smithi100.stdout:found child test-librbd-smithi100-115303-30/image120/b186607ff13a 2024-04-07T17:17:19.500 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:17:19.500 INFO:tasks.workunit.client.0.smithi100.stdout:found child test-librbd-smithi100-115303-30/image121/b186acb0bde0 2024-04-07T17:17:19.500 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:17:19.500 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:17:19.500 INFO:tasks.workunit.client.0.smithi100.stdout:trying to find test-librbd-smithi100-115303-30/image118 2024-04-07T17:17:19.500 INFO:tasks.workunit.client.0.smithi100.stdout:checking test-librbd-smithi100-115303-29/image119 2024-04-07T17:17:19.500 INFO:tasks.workunit.client.0.smithi100.stdout:checking test-librbd-smithi100-115303-30/image118 2024-04-07T17:17:19.500 INFO:tasks.workunit.client.0.smithi100.stdout:found child test-librbd-smithi100-115303-30/image118 2024-04-07T17:17:19.500 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:17:19.500 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:17:19.500 INFO:tasks.workunit.client.0.smithi100.stdout:trying to find test-librbd-smithi100-115303-29/image119 2024-04-07T17:17:19.500 INFO:tasks.workunit.client.0.smithi100.stdout:checking test-librbd-smithi100-115303-29/image119 2024-04-07T17:17:19.500 INFO:tasks.workunit.client.0.smithi100.stdout:found child test-librbd-smithi100-115303-29/image119 2024-04-07T17:17:19.500 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:17:19.500 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:17:19.501 INFO:tasks.workunit.client.0.smithi100.stdout:trying to find test-librbd-smithi100-115303-30/image120 2024-04-07T17:17:19.501 INFO:tasks.workunit.client.0.smithi100.stdout:checking test-librbd-smithi100-115303-29/image119 2024-04-07T17:17:19.501 INFO:tasks.workunit.client.0.smithi100.stdout:checking test-librbd-smithi100-115303-30/image118 2024-04-07T17:17:19.501 INFO:tasks.workunit.client.0.smithi100.stdout:checking test-librbd-smithi100-115303-30/image120 2024-04-07T17:17:19.501 INFO:tasks.workunit.client.0.smithi100.stdout:found child test-librbd-smithi100-115303-30/image120 2024-04-07T17:17:19.501 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:17:19.501 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:17:19.501 INFO:tasks.workunit.client.0.smithi100.stdout:trying to find test-librbd-smithi100-115303-30/image121 2024-04-07T17:17:20.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:17:20 smithi144 bash[66654]: cluster 2024-04-07T17:17:19.796841+0000 mgr.y (mgr.25175) 1859 : cluster [DBG] pgmap v3135: 649 pgs: 649 active+clean; 301 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 37 KiB/s rd, 2.9 KiB/s wr, 50 op/s 2024-04-07T17:17:20.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:17:20 smithi100 bash[74942]: cluster 2024-04-07T17:17:19.796841+0000 mgr.y (mgr.25175) 1859 : cluster [DBG] pgmap v3135: 649 pgs: 649 active+clean; 301 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 37 KiB/s rd, 2.9 KiB/s wr, 50 op/s 2024-04-07T17:17:20.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:17:20 smithi100 bash[76202]: cluster 2024-04-07T17:17:19.796841+0000 mgr.y (mgr.25175) 1859 : cluster [DBG] pgmap v3135: 649 pgs: 649 active+clean; 301 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 37 KiB/s rd, 2.9 KiB/s wr, 50 op/s 2024-04-07T17:17:22.318 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:17:22 smithi144 bash[66654]: cluster 2024-04-07T17:17:21.042118+0000 mon.a (mon.0) 3283 : cluster [DBG] osdmap e1702: 8 total, 8 up, 8 in 2024-04-07T17:17:22.318 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:17:22 smithi144 bash[66654]: audit 2024-04-07T17:17:21.235666+0000 mon.c (mon.2) 85 : audit [INF] from='client.? 172.21.15.100:0/3213618994' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-librbd-smithi100-115303-29"}]: dispatch 2024-04-07T17:17:22.318 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:17:22 smithi144 bash[66654]: audit 2024-04-07T17:17:21.236229+0000 mon.a (mon.0) 3284 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-librbd-smithi100-115303-29"}]: dispatch 2024-04-07T17:17:22.318 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:17:22 smithi144 bash[66654]: audit 2024-04-07T17:17:21.669948+0000 mon.a (mon.0) 3285 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "102.d", "id": [6, 4]}]: dispatch 2024-04-07T17:17:22.318 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:17:22 smithi144 bash[66654]: audit 2024-04-07T17:17:21.670437+0000 mon.a (mon.0) 3286 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "106.19", "id": [2, 5]}]: dispatch 2024-04-07T17:17:22.318 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:17:22 smithi144 bash[66654]: audit 2024-04-07T17:17:21.670837+0000 mon.a (mon.0) 3287 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "107.c", "id": [3, 7]}]: dispatch 2024-04-07T17:17:22.319 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:17:22 smithi144 bash[66654]: cluster 2024-04-07T17:17:21.798022+0000 mgr.y (mgr.25175) 1860 : cluster [DBG] pgmap v3137: 649 pgs: 649 active+clean; 301 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 47 KiB/s rd, 5.0 KiB/s wr, 63 op/s 2024-04-07T17:17:22.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:17:22 smithi100 bash[74942]: cluster 2024-04-07T17:17:21.042118+0000 mon.a (mon.0) 3283 : cluster [DBG] osdmap e1702: 8 total, 8 up, 8 in 2024-04-07T17:17:22.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:17:22 smithi100 bash[74942]: audit 2024-04-07T17:17:21.235666+0000 mon.c (mon.2) 85 : audit [INF] from='client.? 172.21.15.100:0/3213618994' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-librbd-smithi100-115303-29"}]: dispatch 2024-04-07T17:17:22.455 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:17:22 smithi100 bash[74942]: audit 2024-04-07T17:17:21.236229+0000 mon.a (mon.0) 3284 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-librbd-smithi100-115303-29"}]: dispatch 2024-04-07T17:17:22.455 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:17:22 smithi100 bash[74942]: audit 2024-04-07T17:17:21.669948+0000 mon.a (mon.0) 3285 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "102.d", "id": [6, 4]}]: dispatch 2024-04-07T17:17:22.455 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:17:22 smithi100 bash[74942]: audit 2024-04-07T17:17:21.670437+0000 mon.a (mon.0) 3286 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "106.19", "id": [2, 5]}]: dispatch 2024-04-07T17:17:22.455 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:17:22 smithi100 bash[74942]: audit 2024-04-07T17:17:21.670837+0000 mon.a (mon.0) 3287 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "107.c", "id": [3, 7]}]: dispatch 2024-04-07T17:17:22.455 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:17:22 smithi100 bash[74942]: cluster 2024-04-07T17:17:21.798022+0000 mgr.y (mgr.25175) 1860 : cluster [DBG] pgmap v3137: 649 pgs: 649 active+clean; 301 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 47 KiB/s rd, 5.0 KiB/s wr, 63 op/s 2024-04-07T17:17:22.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:17:22 smithi100 bash[76202]: cluster 2024-04-07T17:17:21.042118+0000 mon.a (mon.0) 3283 : cluster [DBG] osdmap e1702: 8 total, 8 up, 8 in 2024-04-07T17:17:22.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:17:22 smithi100 bash[76202]: audit 2024-04-07T17:17:21.235666+0000 mon.c (mon.2) 85 : audit [INF] from='client.? 172.21.15.100:0/3213618994' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-librbd-smithi100-115303-29"}]: dispatch 2024-04-07T17:17:22.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:17:22 smithi100 bash[76202]: audit 2024-04-07T17:17:21.236229+0000 mon.a (mon.0) 3284 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-librbd-smithi100-115303-29"}]: dispatch 2024-04-07T17:17:22.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:17:22 smithi100 bash[76202]: audit 2024-04-07T17:17:21.669948+0000 mon.a (mon.0) 3285 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "102.d", "id": [6, 4]}]: dispatch 2024-04-07T17:17:22.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:17:22 smithi100 bash[76202]: audit 2024-04-07T17:17:21.670437+0000 mon.a (mon.0) 3286 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "106.19", "id": [2, 5]}]: dispatch 2024-04-07T17:17:22.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:17:22 smithi100 bash[76202]: audit 2024-04-07T17:17:21.670837+0000 mon.a (mon.0) 3287 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "107.c", "id": [3, 7]}]: dispatch 2024-04-07T17:17:22.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:17:22 smithi100 bash[76202]: cluster 2024-04-07T17:17:21.798022+0000 mgr.y (mgr.25175) 1860 : cluster [DBG] pgmap v3137: 649 pgs: 649 active+clean; 301 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 47 KiB/s rd, 5.0 KiB/s wr, 63 op/s 2024-04-07T17:17:22.617 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:17:22 smithi144 bash[95233]: ts=2024-04-07T17:17:22.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:17:22.880 INFO:tasks.workunit.client.0.smithi100.stdout:checking test-librbd-smithi100-115303-29/image119 2024-04-07T17:17:22.880 INFO:tasks.workunit.client.0.smithi100.stdout:checking test-librbd-smithi100-115303-30/image118 2024-04-07T17:17:22.880 INFO:tasks.workunit.client.0.smithi100.stdout:checking test-librbd-smithi100-115303-30/image120 2024-04-07T17:17:22.880 INFO:tasks.workunit.client.0.smithi100.stdout:checking test-librbd-smithi100-115303-30/image121 2024-04-07T17:17:22.880 INFO:tasks.workunit.client.0.smithi100.stdout:found child test-librbd-smithi100-115303-30/image121 2024-04-07T17:17:22.880 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:17:22.880 INFO:tasks.workunit.client.0.smithi100.stdout:num children is: 4 2024-04-07T17:17:22.880 INFO:tasks.workunit.client.0.smithi100.stdout:expected: 4 2024-04-07T17:17:22.880 INFO:tasks.workunit.client.0.smithi100.stdout:child: image119 2024-04-07T17:17:22.881 INFO:tasks.workunit.client.0.smithi100.stdout:child: image118 2024-04-07T17:17:22.881 INFO:tasks.workunit.client.0.smithi100.stdout:child: image120 2024-04-07T17:17:22.881 INFO:tasks.workunit.client.0.smithi100.stdout:child: image121 2024-04-07T17:17:22.881 INFO:tasks.workunit.client.0.smithi100.stdout:found child test-librbd-smithi100-115303-30/image118/b1868f71ca7e 2024-04-07T17:17:22.881 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:17:22.881 INFO:tasks.workunit.client.0.smithi100.stdout:found child test-librbd-smithi100-115303-29/image119/b18678173feb 2024-04-07T17:17:22.881 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:17:22.881 INFO:tasks.workunit.client.0.smithi100.stdout:found child test-librbd-smithi100-115303-30/image120/b186607ff13a 2024-04-07T17:17:22.881 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:17:22.881 INFO:tasks.workunit.client.0.smithi100.stdout:found child test-librbd-smithi100-115303-30/image121/b186acb0bde0 2024-04-07T17:17:22.881 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:17:22.881 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:17:22.881 INFO:tasks.workunit.client.0.smithi100.stdout:trying to find test-librbd-smithi100-115303-29/image119 2024-04-07T17:17:22.882 INFO:tasks.workunit.client.0.smithi100.stdout:checking test-librbd-smithi100-115303-29/image119 2024-04-07T17:17:22.882 INFO:tasks.workunit.client.0.smithi100.stdout:found child test-librbd-smithi100-115303-29/image119 2024-04-07T17:17:22.882 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:17:22.882 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:17:22.882 INFO:tasks.workunit.client.0.smithi100.stdout:trying to find test-librbd-smithi100-115303-30/image120 2024-04-07T17:17:22.882 INFO:tasks.workunit.client.0.smithi100.stdout:checking test-librbd-smithi100-115303-29/image119 2024-04-07T17:17:22.882 INFO:tasks.workunit.client.0.smithi100.stdout:checking test-librbd-smithi100-115303-30/image120 2024-04-07T17:17:22.882 INFO:tasks.workunit.client.0.smithi100.stdout:found child test-librbd-smithi100-115303-30/image120 2024-04-07T17:17:22.882 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:17:22.882 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:17:22.882 INFO:tasks.workunit.client.0.smithi100.stdout:trying to find test-librbd-smithi100-115303-30/image121 2024-04-07T17:17:22.882 INFO:tasks.workunit.client.0.smithi100.stdout:checking test-librbd-smithi100-115303-29/image119 2024-04-07T17:17:22.883 INFO:tasks.workunit.client.0.smithi100.stdout:checking test-librbd-smithi100-115303-30/image120 2024-04-07T17:17:22.883 INFO:tasks.workunit.client.0.smithi100.stdout:checking test-librbd-smithi100-115303-30/image121 2024-04-07T17:17:22.883 INFO:tasks.workunit.client.0.smithi100.stdout:found child test-librbd-smithi100-115303-30/image121 2024-04-07T17:17:22.883 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:17:22.883 INFO:tasks.workunit.client.0.smithi100.stdout:num children is: 3 2024-04-07T17:17:22.883 INFO:tasks.workunit.client.0.smithi100.stdout:expected: 3 2024-04-07T17:17:22.883 INFO:tasks.workunit.client.0.smithi100.stdout:child: image119 2024-04-07T17:17:22.883 INFO:tasks.workunit.client.0.smithi100.stdout:child: image120 2024-04-07T17:17:22.883 INFO:tasks.workunit.client.0.smithi100.stdout:child: image121 2024-04-07T17:17:22.883 INFO:tasks.workunit.client.0.smithi100.stdout:found child test-librbd-smithi100-115303-29/image119/b18678173feb 2024-04-07T17:17:22.883 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:17:22.883 INFO:tasks.workunit.client.0.smithi100.stdout:found child test-librbd-smithi100-115303-30/image120/b186607ff13a 2024-04-07T17:17:22.883 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:17:22.884 INFO:tasks.workunit.client.0.smithi100.stdout:found child test-librbd-smithi100-115303-30/image121/b186acb0bde0 2024-04-07T17:17:22.884 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:17:22.884 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:17:22.884 INFO:tasks.workunit.client.0.smithi100.stdout:trying to find test-librbd-smithi100-115303-29/image119 2024-04-07T17:17:22.884 INFO:tasks.workunit.client.0.smithi100.stdout:checking test-librbd-smithi100-115303-29/image119 2024-04-07T17:17:22.884 INFO:tasks.workunit.client.0.smithi100.stdout:found child test-librbd-smithi100-115303-29/image119 2024-04-07T17:17:22.884 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:17:22.884 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:17:22.884 INFO:tasks.workunit.client.0.smithi100.stdout:trying to find test-librbd-smithi100-115303-30/image121 2024-04-07T17:17:22.884 INFO:tasks.workunit.client.0.smithi100.stdout:checking test-librbd-smithi100-115303-29/image119 2024-04-07T17:17:22.884 INFO:tasks.workunit.client.0.smithi100.stdout:checking test-librbd-smithi100-115303-30/image121 2024-04-07T17:17:22.884 INFO:tasks.workunit.client.0.smithi100.stdout:found child test-librbd-smithi100-115303-30/image121 2024-04-07T17:17:22.885 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:17:22.885 INFO:tasks.workunit.client.0.smithi100.stdout:num children is: 2 2024-04-07T17:17:22.885 INFO:tasks.workunit.client.0.smithi100.stdout:expected: 2 2024-04-07T17:17:22.885 INFO:tasks.workunit.client.0.smithi100.stdout:child: image119 2024-04-07T17:17:22.885 INFO:tasks.workunit.client.0.smithi100.stdout:child: image121 2024-04-07T17:17:22.885 INFO:tasks.workunit.client.0.smithi100.stdout:found child test-librbd-smithi100-115303-29/image119/b18678173feb 2024-04-07T17:17:22.885 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:17:22.885 INFO:tasks.workunit.client.0.smithi100.stdout:found child test-librbd-smithi100-115303-30/image121/b186acb0bde0 2024-04-07T17:17:22.885 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:17:22.885 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:17:22.885 INFO:tasks.workunit.client.0.smithi100.stdout:trying to find test-librbd-smithi100-115303-29/image119 2024-04-07T17:17:22.885 INFO:tasks.workunit.client.0.smithi100.stdout:checking test-librbd-smithi100-115303-29/image119 2024-04-07T17:17:22.886 INFO:tasks.workunit.client.0.smithi100.stdout:found child test-librbd-smithi100-115303-29/image119 2024-04-07T17:17:22.886 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:17:22.886 INFO:tasks.workunit.client.0.smithi100.stdout:num children is: 1 2024-04-07T17:17:22.886 INFO:tasks.workunit.client.0.smithi100.stdout:expected: 1 2024-04-07T17:17:22.886 INFO:tasks.workunit.client.0.smithi100.stdout:child: image119 2024-04-07T17:17:22.886 INFO:tasks.workunit.client.0.smithi100.stdout:found child test-librbd-smithi100-115303-29/image119/b18678173feb 2024-04-07T17:17:22.886 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:17:22.886 INFO:tasks.workunit.client.0.smithi100.stdout:num children is: 0 2024-04-07T17:17:22.886 INFO:tasks.workunit.client.0.smithi100.stdout:expected: 0 2024-04-07T17:17:22.886 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.ListChildrenTiered (18648 ms) 2024-04-07T17:17:22.886 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.LockingPP 2024-04-07T17:17:22.886 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:17:23.042 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.LockingPP (163 ms) 2024-04-07T17:17:23.042 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.FlushAio 2024-04-07T17:17:23.042 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:17:23.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:17:23 smithi144 bash[66654]: audit 2024-04-07T17:17:22.045582+0000 mon.a (mon.0) 3288 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-librbd-smithi100-115303-29"}]': finished 2024-04-07T17:17:23.368 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:17:23 smithi144 bash[66654]: audit 2024-04-07T17:17:22.045798+0000 mon.a (mon.0) 3289 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "102.d", "id": [6, 4]}]': finished 2024-04-07T17:17:23.368 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:17:23 smithi144 bash[66654]: audit 2024-04-07T17:17:22.045983+0000 mon.a (mon.0) 3290 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "106.19", "id": [2, 5]}]': finished 2024-04-07T17:17:23.368 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:17:23 smithi144 bash[66654]: audit 2024-04-07T17:17:22.046158+0000 mon.a (mon.0) 3291 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "107.c", "id": [3, 7]}]': finished 2024-04-07T17:17:23.368 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:17:23 smithi144 bash[66654]: cluster 2024-04-07T17:17:22.057738+0000 mon.a (mon.0) 3292 : cluster [DBG] osdmap e1703: 8 total, 8 up, 8 in 2024-04-07T17:17:23.368 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:17:23 smithi144 bash[66654]: audit 2024-04-07T17:17:22.057911+0000 mon.c (mon.2) 86 : audit [INF] from='client.? 172.21.15.100:0/3213618994' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-librbd-smithi100-115303-29", "tierpool":"test-librbd-smithi100-115303-31"}]: dispatch 2024-04-07T17:17:23.368 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:17:23 smithi144 bash[66654]: audit 2024-04-07T17:17:22.061639+0000 mon.a (mon.0) 3293 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-librbd-smithi100-115303-29", "tierpool":"test-librbd-smithi100-115303-31"}]: dispatch 2024-04-07T17:17:23.368 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:17:23 smithi144 bash[66654]: cluster 2024-04-07T17:17:22.866006+0000 mon.a (mon.0) 3294 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-04-07T17:17:23.368 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:17:23 smithi144 bash[66654]: audit 2024-04-07T17:17:22.868532+0000 mon.a (mon.0) 3295 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-librbd-smithi100-115303-29", "tierpool":"test-librbd-smithi100-115303-31"}]': finished 2024-04-07T17:17:23.368 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:17:23 smithi144 bash[66654]: cluster 2024-04-07T17:17:22.878034+0000 mon.a (mon.0) 3296 : cluster [DBG] osdmap e1704: 8 total, 8 up, 8 in 2024-04-07T17:17:23.441 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.FlushAio (399 ms) 2024-04-07T17:17:23.442 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.FlushAioPP 2024-04-07T17:17:23.442 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:17:23.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:17:23 smithi100 bash[76202]: audit 2024-04-07T17:17:22.045582+0000 mon.a (mon.0) 3288 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-librbd-smithi100-115303-29"}]': finished 2024-04-07T17:17:23.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:17:23 smithi100 bash[76202]: audit 2024-04-07T17:17:22.045798+0000 mon.a (mon.0) 3289 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "102.d", "id": [6, 4]}]': finished 2024-04-07T17:17:23.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:17:23 smithi100 bash[76202]: audit 2024-04-07T17:17:22.045983+0000 mon.a (mon.0) 3290 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "106.19", "id": [2, 5]}]': finished 2024-04-07T17:17:23.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:17:23 smithi100 bash[76202]: audit 2024-04-07T17:17:22.046158+0000 mon.a (mon.0) 3291 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "107.c", "id": [3, 7]}]': finished 2024-04-07T17:17:23.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:17:23 smithi100 bash[76202]: cluster 2024-04-07T17:17:22.057738+0000 mon.a (mon.0) 3292 : cluster [DBG] osdmap e1703: 8 total, 8 up, 8 in 2024-04-07T17:17:23.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:17:23 smithi100 bash[76202]: audit 2024-04-07T17:17:22.057911+0000 mon.c (mon.2) 86 : audit [INF] from='client.? 172.21.15.100:0/3213618994' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-librbd-smithi100-115303-29", "tierpool":"test-librbd-smithi100-115303-31"}]: dispatch 2024-04-07T17:17:23.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:17:23 smithi100 bash[76202]: audit 2024-04-07T17:17:22.061639+0000 mon.a (mon.0) 3293 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-librbd-smithi100-115303-29", "tierpool":"test-librbd-smithi100-115303-31"}]: dispatch 2024-04-07T17:17:23.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:17:23 smithi100 bash[76202]: cluster 2024-04-07T17:17:22.866006+0000 mon.a (mon.0) 3294 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-04-07T17:17:23.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:17:23 smithi100 bash[76202]: audit 2024-04-07T17:17:22.868532+0000 mon.a (mon.0) 3295 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-librbd-smithi100-115303-29", "tierpool":"test-librbd-smithi100-115303-31"}]': finished 2024-04-07T17:17:23.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:17:23 smithi100 bash[76202]: cluster 2024-04-07T17:17:22.878034+0000 mon.a (mon.0) 3296 : cluster [DBG] osdmap e1704: 8 total, 8 up, 8 in 2024-04-07T17:17:23.455 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:17:23 smithi100 bash[74942]: audit 2024-04-07T17:17:22.045582+0000 mon.a (mon.0) 3288 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-librbd-smithi100-115303-29"}]': finished 2024-04-07T17:17:23.455 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:17:23 smithi100 bash[74942]: audit 2024-04-07T17:17:22.045798+0000 mon.a (mon.0) 3289 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "102.d", "id": [6, 4]}]': finished 2024-04-07T17:17:23.455 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:17:23 smithi100 bash[74942]: audit 2024-04-07T17:17:22.045983+0000 mon.a (mon.0) 3290 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "106.19", "id": [2, 5]}]': finished 2024-04-07T17:17:23.455 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:17:23 smithi100 bash[74942]: audit 2024-04-07T17:17:22.046158+0000 mon.a (mon.0) 3291 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "107.c", "id": [3, 7]}]': finished 2024-04-07T17:17:23.455 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:17:23 smithi100 bash[74942]: cluster 2024-04-07T17:17:22.057738+0000 mon.a (mon.0) 3292 : cluster [DBG] osdmap e1703: 8 total, 8 up, 8 in 2024-04-07T17:17:23.455 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:17:23 smithi100 bash[74942]: audit 2024-04-07T17:17:22.057911+0000 mon.c (mon.2) 86 : audit [INF] from='client.? 172.21.15.100:0/3213618994' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-librbd-smithi100-115303-29", "tierpool":"test-librbd-smithi100-115303-31"}]: dispatch 2024-04-07T17:17:23.455 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:17:23 smithi100 bash[74942]: audit 2024-04-07T17:17:22.061639+0000 mon.a (mon.0) 3293 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-librbd-smithi100-115303-29", "tierpool":"test-librbd-smithi100-115303-31"}]: dispatch 2024-04-07T17:17:23.455 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:17:23 smithi100 bash[74942]: cluster 2024-04-07T17:17:22.866006+0000 mon.a (mon.0) 3294 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-04-07T17:17:23.455 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:17:23 smithi100 bash[74942]: audit 2024-04-07T17:17:22.868532+0000 mon.a (mon.0) 3295 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-librbd-smithi100-115303-29", "tierpool":"test-librbd-smithi100-115303-31"}]': finished 2024-04-07T17:17:23.455 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:17:23 smithi100 bash[74942]: cluster 2024-04-07T17:17:22.878034+0000 mon.a (mon.0) 3296 : cluster [DBG] osdmap e1704: 8 total, 8 up, 8 in 2024-04-07T17:17:23.716 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.FlushAioPP (275 ms) 2024-04-07T17:17:23.716 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.ZeroLengthWrite 2024-04-07T17:17:23.716 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:17:23.755 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.ZeroLengthWrite (39 ms) 2024-04-07T17:17:23.755 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.ZeroLengthDiscard 2024-04-07T17:17:23.755 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:17:23.802 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.ZeroLengthDiscard (47 ms) 2024-04-07T17:17:23.802 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.ZeroLengthRead 2024-04-07T17:17:23.802 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:17:23.834 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.ZeroLengthRead (32 ms) 2024-04-07T17:17:23.834 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.LargeCacheRead 2024-04-07T17:17:23.834 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:17:23.912 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.LargeCacheRead (78 ms) 2024-04-07T17:17:23.913 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.TestPendingAio 2024-04-07T17:17:23.913 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:17:24.284 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.TestPendingAio (372 ms) 2024-04-07T17:17:24.285 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.CompareAndWriteCopyup 2024-04-07T17:17:24.285 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:17:24.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:17:24 smithi144 bash[66654]: cluster 2024-04-07T17:17:23.800060+0000 mgr.y (mgr.25175) 1861 : cluster [DBG] pgmap v3140: 649 pgs: 649 active+clean; 301 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 383 KiB/s rd, 8.8 KiB/s wr, 460 op/s; 0 B/s, 0 objects/s recovering 2024-04-07T17:17:24.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:17:24 smithi100 bash[74942]: cluster 2024-04-07T17:17:23.800060+0000 mgr.y (mgr.25175) 1861 : cluster [DBG] pgmap v3140: 649 pgs: 649 active+clean; 301 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 383 KiB/s rd, 8.8 KiB/s wr, 460 op/s; 0 B/s, 0 objects/s recovering 2024-04-07T17:17:24.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:17:24 smithi100 bash[76202]: cluster 2024-04-07T17:17:23.800060+0000 mgr.y (mgr.25175) 1861 : cluster [DBG] pgmap v3140: 649 pgs: 649 active+clean; 301 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 383 KiB/s rd, 8.8 KiB/s wr, 460 op/s; 0 B/s, 0 objects/s recovering 2024-04-07T17:17:25.313 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:17:26.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:17:26 smithi144 bash[66654]: cluster 2024-04-07T17:17:25.069408+0000 mon.a (mon.0) 3297 : cluster [DBG] osdmap e1705: 8 total, 8 up, 8 in 2024-04-07T17:17:26.368 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:17:26 smithi144 bash[66654]: cluster 2024-04-07T17:17:25.801257+0000 mgr.y (mgr.25175) 1862 : cluster [DBG] pgmap v3142: 649 pgs: 649 active+clean; 303 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 483 KiB/s rd, 65 KiB/s wr, 645 op/s; 0 B/s, 0 objects/s recovering 2024-04-07T17:17:26.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:17:26 smithi100 bash[74942]: cluster 2024-04-07T17:17:25.069408+0000 mon.a (mon.0) 3297 : cluster [DBG] osdmap e1705: 8 total, 8 up, 8 in 2024-04-07T17:17:26.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:17:26 smithi100 bash[74942]: cluster 2024-04-07T17:17:25.801257+0000 mgr.y (mgr.25175) 1862 : cluster [DBG] pgmap v3142: 649 pgs: 649 active+clean; 303 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 483 KiB/s rd, 65 KiB/s wr, 645 op/s; 0 B/s, 0 objects/s recovering 2024-04-07T17:17:26.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:17:26 smithi100 bash[76202]: cluster 2024-04-07T17:17:25.069408+0000 mon.a (mon.0) 3297 : cluster [DBG] osdmap e1705: 8 total, 8 up, 8 in 2024-04-07T17:17:26.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:17:26 smithi100 bash[76202]: cluster 2024-04-07T17:17:25.801257+0000 mgr.y (mgr.25175) 1862 : cluster [DBG] pgmap v3142: 649 pgs: 649 active+clean; 303 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 483 KiB/s rd, 65 KiB/s wr, 645 op/s; 0 B/s, 0 objects/s recovering 2024-04-07T17:17:27.212 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.CompareAndWriteCopyup (2928 ms) 2024-04-07T17:17:27.212 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.CompareAndWriteCopyupMismatch 2024-04-07T17:17:27.212 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:17:27.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:17:27 smithi144 bash[66654]: cluster 2024-04-07T17:17:26.083052+0000 mon.a (mon.0) 3298 : cluster [DBG] osdmap e1706: 8 total, 8 up, 8 in 2024-04-07T17:17:27.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:17:27 smithi100 bash[74942]: cluster 2024-04-07T17:17:26.083052+0000 mon.a (mon.0) 3298 : cluster [DBG] osdmap e1706: 8 total, 8 up, 8 in 2024-04-07T17:17:27.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:17:27 smithi100 bash[76202]: cluster 2024-04-07T17:17:26.083052+0000 mon.a (mon.0) 3298 : cluster [DBG] osdmap e1706: 8 total, 8 up, 8 in 2024-04-07T17:17:28.137 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:17:28.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:17:28 smithi144 bash[66654]: cluster 2024-04-07T17:17:27.093456+0000 mon.a (mon.0) 3299 : cluster [DBG] osdmap e1707: 8 total, 8 up, 8 in 2024-04-07T17:17:28.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:17:28 smithi144 bash[66654]: cluster 2024-04-07T17:17:27.802548+0000 mgr.y (mgr.25175) 1863 : cluster [DBG] pgmap v3145: 649 pgs: 649 active+clean; 309 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 854 KiB/s rd, 590 KiB/s wr, 711 op/s; 0 B/s, 0 objects/s recovering 2024-04-07T17:17:28.368 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:17:28 smithi144 bash[66654]: cluster 2024-04-07T17:17:27.879540+0000 mon.a (mon.0) 3300 : cluster [DBG] osdmap e1708: 8 total, 8 up, 8 in 2024-04-07T17:17:28.429 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:17:28 smithi100 bash[74942]: cluster 2024-04-07T17:17:27.093456+0000 mon.a (mon.0) 3299 : cluster [DBG] osdmap e1707: 8 total, 8 up, 8 in 2024-04-07T17:17:28.429 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:17:28 smithi100 bash[74942]: cluster 2024-04-07T17:17:27.802548+0000 mgr.y (mgr.25175) 1863 : cluster [DBG] pgmap v3145: 649 pgs: 649 active+clean; 309 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 854 KiB/s rd, 590 KiB/s wr, 711 op/s; 0 B/s, 0 objects/s recovering 2024-04-07T17:17:28.429 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:17:28 smithi100 bash[74942]: cluster 2024-04-07T17:17:27.879540+0000 mon.a (mon.0) 3300 : cluster [DBG] osdmap e1708: 8 total, 8 up, 8 in 2024-04-07T17:17:28.429 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:17:28 smithi100 bash[76202]: cluster 2024-04-07T17:17:27.093456+0000 mon.a (mon.0) 3299 : cluster [DBG] osdmap e1707: 8 total, 8 up, 8 in 2024-04-07T17:17:28.429 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:17:28 smithi100 bash[76202]: cluster 2024-04-07T17:17:27.802548+0000 mgr.y (mgr.25175) 1863 : cluster [DBG] pgmap v3145: 649 pgs: 649 active+clean; 309 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 854 KiB/s rd, 590 KiB/s wr, 711 op/s; 0 B/s, 0 objects/s recovering 2024-04-07T17:17:28.430 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:17:28 smithi100 bash[76202]: cluster 2024-04-07T17:17:27.879540+0000 mon.a (mon.0) 3300 : cluster [DBG] osdmap e1708: 8 total, 8 up, 8 in 2024-04-07T17:17:28.704 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:17:28 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:17:28] "GET /metrics HTTP/1.1" 200 75547 "" "Prometheus/2.43.0" 2024-04-07T17:17:30.244 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.CompareAndWriteCopyupMismatch (3031 ms) 2024-04-07T17:17:30.244 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.Flatten 2024-04-07T17:17:30.244 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:17:30.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:17:30 smithi100 bash[74942]: cluster 2024-04-07T17:17:29.109152+0000 mon.a (mon.0) 3301 : cluster [DBG] osdmap e1709: 8 total, 8 up, 8 in 2024-04-07T17:17:30.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:17:30 smithi100 bash[74942]: cluster 2024-04-07T17:17:29.804503+0000 mgr.y (mgr.25175) 1864 : cluster [DBG] pgmap v3148: 649 pgs: 649 active+clean; 309 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 775 KiB/s rd, 661 KiB/s wr, 471 op/s 2024-04-07T17:17:30.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:17:30 smithi100 bash[76202]: cluster 2024-04-07T17:17:29.109152+0000 mon.a (mon.0) 3301 : cluster [DBG] osdmap e1709: 8 total, 8 up, 8 in 2024-04-07T17:17:30.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:17:30 smithi100 bash[76202]: cluster 2024-04-07T17:17:29.804503+0000 mgr.y (mgr.25175) 1864 : cluster [DBG] pgmap v3148: 649 pgs: 649 active+clean; 309 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 775 KiB/s rd, 661 KiB/s wr, 471 op/s 2024-04-07T17:17:30.617 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:17:30 smithi144 bash[66654]: cluster 2024-04-07T17:17:29.109152+0000 mon.a (mon.0) 3301 : cluster [DBG] osdmap e1709: 8 total, 8 up, 8 in 2024-04-07T17:17:30.617 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:17:30 smithi144 bash[66654]: cluster 2024-04-07T17:17:29.804503+0000 mgr.y (mgr.25175) 1864 : cluster [DBG] pgmap v3148: 649 pgs: 649 active+clean; 309 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 775 KiB/s rd, 661 KiB/s wr, 471 op/s 2024-04-07T17:17:31.399 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.Flatten (1155 ms) 2024-04-07T17:17:31.399 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.Sparsify 2024-04-07T17:17:31.399 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:17:31.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:17:31 smithi100 bash[74942]: cluster 2024-04-07T17:17:30.126992+0000 mon.a (mon.0) 3302 : cluster [DBG] osdmap e1710: 8 total, 8 up, 8 in 2024-04-07T17:17:31.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:17:31 smithi100 bash[76202]: cluster 2024-04-07T17:17:30.126992+0000 mon.a (mon.0) 3302 : cluster [DBG] osdmap e1710: 8 total, 8 up, 8 in 2024-04-07T17:17:31.484 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 32768 2024-04-07T17:17:31.485 INFO:tasks.workunit.client.0.smithi100.stdout:read: 32768 2024-04-07T17:17:31.485 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.Sparsify (86 ms) 2024-04-07T17:17:31.485 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.SparsifyPP 2024-04-07T17:17:31.485 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:17:31.574 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.SparsifyPP (90 ms) 2024-04-07T17:17:31.574 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.SnapshotLimit 2024-04-07T17:17:31.574 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:17:31.617 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:17:31 smithi144 bash[66654]: cluster 2024-04-07T17:17:30.126992+0000 mon.a (mon.0) 3302 : cluster [DBG] osdmap e1710: 8 total, 8 up, 8 in 2024-04-07T17:17:32.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:17:32 smithi100 bash[74942]: cluster 2024-04-07T17:17:31.123696+0000 mon.a (mon.0) 3303 : cluster [DBG] osdmap e1711: 8 total, 8 up, 8 in 2024-04-07T17:17:32.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:17:32 smithi100 bash[74942]: cluster 2024-04-07T17:17:31.805700+0000 mgr.y (mgr.25175) 1865 : cluster [DBG] pgmap v3151: 649 pgs: 649 active+clean; 309 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 55 KiB/s rd, 8.7 KiB/s wr, 77 op/s 2024-04-07T17:17:32.455 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:17:32 smithi100 bash[74942]: audit 2024-04-07T17:17:31.835409+0000 mon.a (mon.0) 3304 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:17:32.455 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:17:32 smithi100 bash[74942]: audit 2024-04-07T17:17:31.838692+0000 mon.a (mon.0) 3305 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:17:32.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:17:32 smithi100 bash[76202]: cluster 2024-04-07T17:17:31.123696+0000 mon.a (mon.0) 3303 : cluster [DBG] osdmap e1711: 8 total, 8 up, 8 in 2024-04-07T17:17:32.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:17:32 smithi100 bash[76202]: cluster 2024-04-07T17:17:31.805700+0000 mgr.y (mgr.25175) 1865 : cluster [DBG] pgmap v3151: 649 pgs: 649 active+clean; 309 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 55 KiB/s rd, 8.7 KiB/s wr, 77 op/s 2024-04-07T17:17:32.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:17:32 smithi100 bash[76202]: audit 2024-04-07T17:17:31.835409+0000 mon.a (mon.0) 3304 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:17:32.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:17:32 smithi100 bash[76202]: audit 2024-04-07T17:17:31.838692+0000 mon.a (mon.0) 3305 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:17:32.618 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:17:32 smithi144 bash[66654]: cluster 2024-04-07T17:17:31.123696+0000 mon.a (mon.0) 3303 : cluster [DBG] osdmap e1711: 8 total, 8 up, 8 in 2024-04-07T17:17:32.618 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:17:32 smithi144 bash[66654]: cluster 2024-04-07T17:17:31.805700+0000 mgr.y (mgr.25175) 1865 : cluster [DBG] pgmap v3151: 649 pgs: 649 active+clean; 309 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 55 KiB/s rd, 8.7 KiB/s wr, 77 op/s 2024-04-07T17:17:32.618 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:17:32 smithi144 bash[66654]: audit 2024-04-07T17:17:31.835409+0000 mon.a (mon.0) 3304 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:17:32.618 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:17:32 smithi144 bash[66654]: audit 2024-04-07T17:17:31.838692+0000 mon.a (mon.0) 3305 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:17:32.618 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:17:32 smithi144 bash[95233]: ts=2024-04-07T17:17:32.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:17:33.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:17:33 smithi100 bash[76202]: cluster 2024-04-07T17:17:32.157398+0000 mon.a (mon.0) 3306 : cluster [DBG] osdmap e1712: 8 total, 8 up, 8 in 2024-04-07T17:17:33.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:17:33 smithi100 bash[74942]: cluster 2024-04-07T17:17:32.157398+0000 mon.a (mon.0) 3306 : cluster [DBG] osdmap e1712: 8 total, 8 up, 8 in 2024-04-07T17:17:33.617 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:17:33 smithi144 bash[66654]: cluster 2024-04-07T17:17:32.157398+0000 mon.a (mon.0) 3306 : cluster [DBG] osdmap e1712: 8 total, 8 up, 8 in 2024-04-07T17:17:34.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:17:34 smithi100 bash[74942]: cluster 2024-04-07T17:17:33.155492+0000 mon.a (mon.0) 3307 : cluster [DBG] osdmap e1713: 8 total, 8 up, 8 in 2024-04-07T17:17:34.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:17:34 smithi100 bash[74942]: cluster 2024-04-07T17:17:33.806898+0000 mgr.y (mgr.25175) 1866 : cluster [DBG] pgmap v3154: 649 pgs: 649 active+clean; 309 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 211 KiB/s rd, 41 KiB/s wr, 265 op/s 2024-04-07T17:17:34.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:17:34 smithi100 bash[76202]: cluster 2024-04-07T17:17:33.155492+0000 mon.a (mon.0) 3307 : cluster [DBG] osdmap e1713: 8 total, 8 up, 8 in 2024-04-07T17:17:34.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:17:34 smithi100 bash[76202]: cluster 2024-04-07T17:17:33.806898+0000 mgr.y (mgr.25175) 1866 : cluster [DBG] pgmap v3154: 649 pgs: 649 active+clean; 309 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 211 KiB/s rd, 41 KiB/s wr, 265 op/s 2024-04-07T17:17:34.617 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:17:34 smithi144 bash[66654]: cluster 2024-04-07T17:17:33.155492+0000 mon.a (mon.0) 3307 : cluster [DBG] osdmap e1713: 8 total, 8 up, 8 in 2024-04-07T17:17:34.618 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:17:34 smithi144 bash[66654]: cluster 2024-04-07T17:17:33.806898+0000 mgr.y (mgr.25175) 1866 : cluster [DBG] pgmap v3154: 649 pgs: 649 active+clean; 309 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 211 KiB/s rd, 41 KiB/s wr, 265 op/s 2024-04-07T17:17:35.455 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:17:35 smithi100 bash[74942]: cluster 2024-04-07T17:17:34.186564+0000 mon.a (mon.0) 3308 : cluster [DBG] osdmap e1714: 8 total, 8 up, 8 in 2024-04-07T17:17:35.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:17:35 smithi100 bash[76202]: cluster 2024-04-07T17:17:34.186564+0000 mon.a (mon.0) 3308 : cluster [DBG] osdmap e1714: 8 total, 8 up, 8 in 2024-04-07T17:17:35.617 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:17:35 smithi144 bash[66654]: cluster 2024-04-07T17:17:34.186564+0000 mon.a (mon.0) 3308 : cluster [DBG] osdmap e1714: 8 total, 8 up, 8 in 2024-04-07T17:17:36.285 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.SnapshotLimit (4710 ms) 2024-04-07T17:17:36.285 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.SnapshotLimitPP 2024-04-07T17:17:36.285 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:17:36.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:17:36 smithi100 bash[74942]: cluster 2024-04-07T17:17:35.195930+0000 mon.a (mon.0) 3309 : cluster [DBG] osdmap e1715: 8 total, 8 up, 8 in 2024-04-07T17:17:36.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:17:36 smithi100 bash[74942]: cluster 2024-04-07T17:17:35.808031+0000 mgr.y (mgr.25175) 1867 : cluster [DBG] pgmap v3157: 649 pgs: 649 active+clean; 309 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 257 KiB/s rd, 45 KiB/s wr, 324 op/s 2024-04-07T17:17:36.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:17:36 smithi100 bash[76202]: cluster 2024-04-07T17:17:35.195930+0000 mon.a (mon.0) 3309 : cluster [DBG] osdmap e1715: 8 total, 8 up, 8 in 2024-04-07T17:17:36.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:17:36 smithi100 bash[76202]: cluster 2024-04-07T17:17:35.808031+0000 mgr.y (mgr.25175) 1867 : cluster [DBG] pgmap v3157: 649 pgs: 649 active+clean; 309 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 257 KiB/s rd, 45 KiB/s wr, 324 op/s 2024-04-07T17:17:36.617 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:17:36 smithi144 bash[66654]: cluster 2024-04-07T17:17:35.195930+0000 mon.a (mon.0) 3309 : cluster [DBG] osdmap e1715: 8 total, 8 up, 8 in 2024-04-07T17:17:36.618 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:17:36 smithi144 bash[66654]: cluster 2024-04-07T17:17:35.808031+0000 mgr.y (mgr.25175) 1867 : cluster [DBG] pgmap v3157: 649 pgs: 649 active+clean; 309 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 257 KiB/s rd, 45 KiB/s wr, 324 op/s 2024-04-07T17:17:37.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:17:37 smithi100 bash[74942]: cluster 2024-04-07T17:17:36.205233+0000 mon.a (mon.0) 3310 : cluster [DBG] osdmap e1716: 8 total, 8 up, 8 in 2024-04-07T17:17:37.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:17:37 smithi100 bash[76202]: cluster 2024-04-07T17:17:36.205233+0000 mon.a (mon.0) 3310 : cluster [DBG] osdmap e1716: 8 total, 8 up, 8 in 2024-04-07T17:17:37.617 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:17:37 smithi144 bash[66654]: cluster 2024-04-07T17:17:36.205233+0000 mon.a (mon.0) 3310 : cluster [DBG] osdmap e1716: 8 total, 8 up, 8 in 2024-04-07T17:17:38.617 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:17:38 smithi144 bash[66654]: cluster 2024-04-07T17:17:37.213017+0000 mon.a (mon.0) 3311 : cluster [DBG] osdmap e1717: 8 total, 8 up, 8 in 2024-04-07T17:17:38.618 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:17:38 smithi144 bash[66654]: cluster 2024-04-07T17:17:37.809169+0000 mgr.y (mgr.25175) 1868 : cluster [DBG] pgmap v3160: 649 pgs: 649 active+clean; 309 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 72 KiB/s rd, 7.0 KiB/s wr, 95 op/s 2024-04-07T17:17:38.704 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:17:38 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:17:38] "GET /metrics HTTP/1.1" 200 75591 "" "Prometheus/2.43.0" 2024-04-07T17:17:38.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:17:38 smithi100 bash[74942]: cluster 2024-04-07T17:17:37.213017+0000 mon.a (mon.0) 3311 : cluster [DBG] osdmap e1717: 8 total, 8 up, 8 in 2024-04-07T17:17:38.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:17:38 smithi100 bash[74942]: cluster 2024-04-07T17:17:37.809169+0000 mgr.y (mgr.25175) 1868 : cluster [DBG] pgmap v3160: 649 pgs: 649 active+clean; 309 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 72 KiB/s rd, 7.0 KiB/s wr, 95 op/s 2024-04-07T17:17:38.705 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:17:38 smithi100 bash[76202]: cluster 2024-04-07T17:17:37.213017+0000 mon.a (mon.0) 3311 : cluster [DBG] osdmap e1717: 8 total, 8 up, 8 in 2024-04-07T17:17:38.705 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:17:38 smithi100 bash[76202]: cluster 2024-04-07T17:17:37.809169+0000 mgr.y (mgr.25175) 1868 : cluster [DBG] pgmap v3160: 649 pgs: 649 active+clean; 309 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 72 KiB/s rd, 7.0 KiB/s wr, 95 op/s 2024-04-07T17:17:39.617 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:17:39 smithi144 bash[66654]: cluster 2024-04-07T17:17:38.217715+0000 mon.a (mon.0) 3312 : cluster [DBG] osdmap e1718: 8 total, 8 up, 8 in 2024-04-07T17:17:39.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:17:39 smithi100 bash[74942]: cluster 2024-04-07T17:17:38.217715+0000 mon.a (mon.0) 3312 : cluster [DBG] osdmap e1718: 8 total, 8 up, 8 in 2024-04-07T17:17:39.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:17:39 smithi100 bash[76202]: cluster 2024-04-07T17:17:38.217715+0000 mon.a (mon.0) 3312 : cluster [DBG] osdmap e1718: 8 total, 8 up, 8 in 2024-04-07T17:17:40.617 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:17:40 smithi144 bash[66654]: cluster 2024-04-07T17:17:39.252440+0000 mon.a (mon.0) 3313 : cluster [DBG] osdmap e1719: 8 total, 8 up, 8 in 2024-04-07T17:17:40.618 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:17:40 smithi144 bash[66654]: cluster 2024-04-07T17:17:39.810463+0000 mgr.y (mgr.25175) 1869 : cluster [DBG] pgmap v3163: 649 pgs: 649 active+clean; 309 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 27 KiB/s rd, 3.5 KiB/s wr, 38 op/s 2024-04-07T17:17:40.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:17:40 smithi100 bash[74942]: cluster 2024-04-07T17:17:39.252440+0000 mon.a (mon.0) 3313 : cluster [DBG] osdmap e1719: 8 total, 8 up, 8 in 2024-04-07T17:17:40.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:17:40 smithi100 bash[74942]: cluster 2024-04-07T17:17:39.810463+0000 mgr.y (mgr.25175) 1869 : cluster [DBG] pgmap v3163: 649 pgs: 649 active+clean; 309 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 27 KiB/s rd, 3.5 KiB/s wr, 38 op/s 2024-04-07T17:17:40.705 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:17:40 smithi100 bash[76202]: cluster 2024-04-07T17:17:39.252440+0000 mon.a (mon.0) 3313 : cluster [DBG] osdmap e1719: 8 total, 8 up, 8 in 2024-04-07T17:17:40.705 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:17:40 smithi100 bash[76202]: cluster 2024-04-07T17:17:39.810463+0000 mgr.y (mgr.25175) 1869 : cluster [DBG] pgmap v3163: 649 pgs: 649 active+clean; 309 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 27 KiB/s rd, 3.5 KiB/s wr, 38 op/s 2024-04-07T17:17:41.338 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.SnapshotLimitPP (5054 ms) 2024-04-07T17:17:41.339 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.RebuildObjectMapViaLockOwner 2024-04-07T17:17:41.339 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:17:41.479 INFO:tasks.workunit.client.0.smithi100.stdout:0.00% done 2024-04-07T17:17:41.479 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.RebuildObjectMapViaLockOwner (140 ms) 2024-04-07T17:17:41.479 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.RenameViaLockOwner 2024-04-07T17:17:41.479 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:17:41.617 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:17:41 smithi144 bash[66654]: cluster 2024-04-07T17:17:40.265974+0000 mon.a (mon.0) 3314 : cluster [DBG] osdmap e1720: 8 total, 8 up, 8 in 2024-04-07T17:17:41.665 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.RenameViaLockOwner (187 ms) 2024-04-07T17:17:41.665 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.SnapCreateViaLockOwner 2024-04-07T17:17:41.665 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:17:41.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:17:41 smithi100 bash[74942]: cluster 2024-04-07T17:17:40.265974+0000 mon.a (mon.0) 3314 : cluster [DBG] osdmap e1720: 8 total, 8 up, 8 in 2024-04-07T17:17:41.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:17:41 smithi100 bash[76202]: cluster 2024-04-07T17:17:40.265974+0000 mon.a (mon.0) 3314 : cluster [DBG] osdmap e1720: 8 total, 8 up, 8 in 2024-04-07T17:17:42.386 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.SnapCreateViaLockOwner (720 ms) 2024-04-07T17:17:42.386 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.SnapRemoveViaLockOwner 2024-04-07T17:17:42.386 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:17:42.617 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:17:42 smithi144 bash[66654]: cluster 2024-04-07T17:17:41.265404+0000 mon.a (mon.0) 3315 : cluster [DBG] osdmap e1721: 8 total, 8 up, 8 in 2024-04-07T17:17:42.618 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:17:42 smithi144 bash[66654]: cluster 2024-04-07T17:17:41.811717+0000 mgr.y (mgr.25175) 1870 : cluster [DBG] pgmap v3166: 649 pgs: 649 active+clean; 309 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 3 op/s 2024-04-07T17:17:42.618 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:17:42 smithi144 bash[95233]: ts=2024-04-07T17:17:42.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:17:42.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:17:42 smithi100 bash[74942]: cluster 2024-04-07T17:17:41.265404+0000 mon.a (mon.0) 3315 : cluster [DBG] osdmap e1721: 8 total, 8 up, 8 in 2024-04-07T17:17:42.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:17:42 smithi100 bash[74942]: cluster 2024-04-07T17:17:41.811717+0000 mgr.y (mgr.25175) 1870 : cluster [DBG] pgmap v3166: 649 pgs: 649 active+clean; 309 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 3 op/s 2024-04-07T17:17:42.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:17:42 smithi100 bash[76202]: cluster 2024-04-07T17:17:41.265404+0000 mon.a (mon.0) 3315 : cluster [DBG] osdmap e1721: 8 total, 8 up, 8 in 2024-04-07T17:17:42.705 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:17:42 smithi100 bash[76202]: cluster 2024-04-07T17:17:41.811717+0000 mgr.y (mgr.25175) 1870 : cluster [DBG] pgmap v3166: 649 pgs: 649 active+clean; 309 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 3 op/s 2024-04-07T17:17:43.617 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:17:43 smithi144 bash[66654]: cluster 2024-04-07T17:17:42.292562+0000 mon.a (mon.0) 3316 : cluster [DBG] osdmap e1722: 8 total, 8 up, 8 in 2024-04-07T17:17:43.617 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:17:43 smithi144 bash[66654]: cluster 2024-04-07T17:17:42.886578+0000 mon.a (mon.0) 3317 : cluster [DBG] osdmap e1723: 8 total, 8 up, 8 in 2024-04-07T17:17:43.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:17:43 smithi100 bash[74942]: cluster 2024-04-07T17:17:42.292562+0000 mon.a (mon.0) 3316 : cluster [DBG] osdmap e1722: 8 total, 8 up, 8 in 2024-04-07T17:17:43.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:17:43 smithi100 bash[74942]: cluster 2024-04-07T17:17:42.886578+0000 mon.a (mon.0) 3317 : cluster [DBG] osdmap e1723: 8 total, 8 up, 8 in 2024-04-07T17:17:43.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:17:43 smithi100 bash[76202]: cluster 2024-04-07T17:17:42.292562+0000 mon.a (mon.0) 3316 : cluster [DBG] osdmap e1722: 8 total, 8 up, 8 in 2024-04-07T17:17:43.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:17:43 smithi100 bash[76202]: cluster 2024-04-07T17:17:42.886578+0000 mon.a (mon.0) 3317 : cluster [DBG] osdmap e1723: 8 total, 8 up, 8 in 2024-04-07T17:17:43.960 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.SnapRemoveViaLockOwner (1575 ms) 2024-04-07T17:17:43.960 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.UpdateFeaturesViaLockOwner 2024-04-07T17:17:43.960 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:17:44.131 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.UpdateFeaturesViaLockOwner (170 ms) 2024-04-07T17:17:44.131 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.EnableJournalingViaLockOwner 2024-04-07T17:17:44.131 INFO:tasks.workunit.client.0.smithi100.stdout:./src/test/librbd/test_librbd.cc:9376: Skipped 2024-04-07T17:17:44.131 INFO:tasks.workunit.client.0.smithi100.stdout:Skipping due to unmet REQUIRE 2024-04-07T17:17:44.131 INFO:tasks.workunit.client.0.smithi100.stdout:[ SKIPPED ] TestLibRBD.EnableJournalingViaLockOwner (1 ms) 2024-04-07T17:17:44.131 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.SnapRemove2 2024-04-07T17:17:44.131 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:17:45.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:17:44 smithi100 bash[74942]: cluster 2024-04-07T17:17:43.813529+0000 mgr.y (mgr.25175) 1871 : cluster [DBG] pgmap v3169: 649 pgs: 649 active+clean; 309 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 36 KiB/s rd, 5.2 KiB/s wr, 51 op/s 2024-04-07T17:17:45.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:17:44 smithi100 bash[74942]: cluster 2024-04-07T17:17:43.888197+0000 mon.a (mon.0) 3318 : cluster [DBG] osdmap e1724: 8 total, 8 up, 8 in 2024-04-07T17:17:45.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:17:44 smithi100 bash[76202]: cluster 2024-04-07T17:17:43.813529+0000 mgr.y (mgr.25175) 1871 : cluster [DBG] pgmap v3169: 649 pgs: 649 active+clean; 309 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 36 KiB/s rd, 5.2 KiB/s wr, 51 op/s 2024-04-07T17:17:45.205 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:17:44 smithi100 bash[76202]: cluster 2024-04-07T17:17:43.888197+0000 mon.a (mon.0) 3318 : cluster [DBG] osdmap e1724: 8 total, 8 up, 8 in 2024-04-07T17:17:45.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:17:44 smithi144 bash[66654]: cluster 2024-04-07T17:17:43.813529+0000 mgr.y (mgr.25175) 1871 : cluster [DBG] pgmap v3169: 649 pgs: 649 active+clean; 309 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 36 KiB/s rd, 5.2 KiB/s wr, 51 op/s 2024-04-07T17:17:45.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:17:44 smithi144 bash[66654]: cluster 2024-04-07T17:17:43.888197+0000 mon.a (mon.0) 3318 : cluster [DBG] osdmap e1724: 8 total, 8 up, 8 in 2024-04-07T17:17:45.985 INFO:tasks.workunit.client.0.smithi100.stdout:100.00% done 2024-04-07T17:17:45.985 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.SnapRemove2 (1853 ms) 2024-04-07T17:17:45.985 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.SnapRenameViaLockOwner 2024-04-07T17:17:45.985 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:17:46.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:17:45 smithi100 bash[74942]: cluster 2024-04-07T17:17:44.897539+0000 mon.a (mon.0) 3319 : cluster [DBG] osdmap e1725: 8 total, 8 up, 8 in 2024-04-07T17:17:46.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:17:45 smithi100 bash[76202]: cluster 2024-04-07T17:17:44.897539+0000 mon.a (mon.0) 3319 : cluster [DBG] osdmap e1725: 8 total, 8 up, 8 in 2024-04-07T17:17:46.368 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:17:45 smithi144 bash[66654]: cluster 2024-04-07T17:17:44.897539+0000 mon.a (mon.0) 3319 : cluster [DBG] osdmap e1725: 8 total, 8 up, 8 in 2024-04-07T17:17:47.047 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.SnapRenameViaLockOwner (1061 ms) 2024-04-07T17:17:47.047 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.SnapProtectViaLockOwner 2024-04-07T17:17:47.047 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:17:47.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:17:46 smithi100 bash[74942]: cluster 2024-04-07T17:17:45.815617+0000 mgr.y (mgr.25175) 1872 : cluster [DBG] pgmap v3172: 649 pgs: 649 active+clean; 309 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 94 KiB/s rd, 10 KiB/s wr, 132 op/s 2024-04-07T17:17:47.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:17:46 smithi100 bash[74942]: cluster 2024-04-07T17:17:45.910827+0000 mon.a (mon.0) 3320 : cluster [DBG] osdmap e1726: 8 total, 8 up, 8 in 2024-04-07T17:17:47.205 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:17:46 smithi100 bash[74942]: audit 2024-04-07T17:17:46.829289+0000 mon.a (mon.0) 3321 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:17:47.205 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:17:46 smithi100 bash[76202]: cluster 2024-04-07T17:17:45.815617+0000 mgr.y (mgr.25175) 1872 : cluster [DBG] pgmap v3172: 649 pgs: 649 active+clean; 309 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 94 KiB/s rd, 10 KiB/s wr, 132 op/s 2024-04-07T17:17:47.205 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:17:46 smithi100 bash[76202]: cluster 2024-04-07T17:17:45.910827+0000 mon.a (mon.0) 3320 : cluster [DBG] osdmap e1726: 8 total, 8 up, 8 in 2024-04-07T17:17:47.205 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:17:46 smithi100 bash[76202]: audit 2024-04-07T17:17:46.829289+0000 mon.a (mon.0) 3321 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:17:47.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:17:46 smithi144 bash[66654]: cluster 2024-04-07T17:17:45.815617+0000 mgr.y (mgr.25175) 1872 : cluster [DBG] pgmap v3172: 649 pgs: 649 active+clean; 309 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 94 KiB/s rd, 10 KiB/s wr, 132 op/s 2024-04-07T17:17:47.368 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:17:46 smithi144 bash[66654]: cluster 2024-04-07T17:17:45.910827+0000 mon.a (mon.0) 3320 : cluster [DBG] osdmap e1726: 8 total, 8 up, 8 in 2024-04-07T17:17:47.368 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:17:46 smithi144 bash[66654]: audit 2024-04-07T17:17:46.829289+0000 mon.a (mon.0) 3321 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:17:48.040 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.SnapProtectViaLockOwner (994 ms) 2024-04-07T17:17:48.040 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.SnapUnprotectViaLockOwner 2024-04-07T17:17:48.040 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:17:48.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:17:47 smithi100 bash[74942]: cluster 2024-04-07T17:17:46.928939+0000 mon.a (mon.0) 3322 : cluster [DBG] osdmap e1727: 8 total, 8 up, 8 in 2024-04-07T17:17:48.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:17:47 smithi100 bash[74942]: cluster 2024-04-07T17:17:47.889094+0000 mon.a (mon.0) 3323 : cluster [DBG] osdmap e1728: 8 total, 8 up, 8 in 2024-04-07T17:17:48.205 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:17:47 smithi100 bash[76202]: cluster 2024-04-07T17:17:46.928939+0000 mon.a (mon.0) 3322 : cluster [DBG] osdmap e1727: 8 total, 8 up, 8 in 2024-04-07T17:17:48.205 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:17:47 smithi100 bash[76202]: cluster 2024-04-07T17:17:47.889094+0000 mon.a (mon.0) 3323 : cluster [DBG] osdmap e1728: 8 total, 8 up, 8 in 2024-04-07T17:17:48.368 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:17:47 smithi144 bash[66654]: cluster 2024-04-07T17:17:46.928939+0000 mon.a (mon.0) 3322 : cluster [DBG] osdmap e1727: 8 total, 8 up, 8 in 2024-04-07T17:17:48.368 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:17:47 smithi144 bash[66654]: cluster 2024-04-07T17:17:47.889094+0000 mon.a (mon.0) 3323 : cluster [DBG] osdmap e1728: 8 total, 8 up, 8 in 2024-04-07T17:17:48.704 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:17:48 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:17:48] "GET /metrics HTTP/1.1" 200 75601 "" "Prometheus/2.43.0" 2024-04-07T17:17:49.094 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.SnapUnprotectViaLockOwner (1053 ms) 2024-04-07T17:17:49.094 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.FlattenViaLockOwner 2024-04-07T17:17:49.094 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:17:49.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:17:48 smithi100 bash[74942]: cluster 2024-04-07T17:17:47.819180+0000 mgr.y (mgr.25175) 1873 : cluster [DBG] pgmap v3175: 649 pgs: 649 active+clean; 309 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 159 KiB/s rd, 13 KiB/s wr, 217 op/s 2024-04-07T17:17:49.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:17:48 smithi100 bash[76202]: cluster 2024-04-07T17:17:47.819180+0000 mgr.y (mgr.25175) 1873 : cluster [DBG] pgmap v3175: 649 pgs: 649 active+clean; 309 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 159 KiB/s rd, 13 KiB/s wr, 217 op/s 2024-04-07T17:17:49.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:17:48 smithi144 bash[66654]: cluster 2024-04-07T17:17:47.819180+0000 mgr.y (mgr.25175) 1873 : cluster [DBG] pgmap v3175: 649 pgs: 649 active+clean; 309 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 159 KiB/s rd, 13 KiB/s wr, 217 op/s 2024-04-07T17:17:50.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:17:49 smithi100 bash[74942]: cluster 2024-04-07T17:17:48.944611+0000 mon.a (mon.0) 3324 : cluster [DBG] osdmap e1729: 8 total, 8 up, 8 in 2024-04-07T17:17:50.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:17:49 smithi100 bash[76202]: cluster 2024-04-07T17:17:48.944611+0000 mon.a (mon.0) 3324 : cluster [DBG] osdmap e1729: 8 total, 8 up, 8 in 2024-04-07T17:17:50.267 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.FlattenViaLockOwner (1173 ms) 2024-04-07T17:17:50.267 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.ResizeViaLockOwner 2024-04-07T17:17:50.267 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:17:50.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:17:49 smithi144 bash[66654]: cluster 2024-04-07T17:17:48.944611+0000 mon.a (mon.0) 3324 : cluster [DBG] osdmap e1729: 8 total, 8 up, 8 in 2024-04-07T17:17:50.373 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.ResizeViaLockOwner (107 ms) 2024-04-07T17:17:50.373 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.SparsifyViaLockOwner 2024-04-07T17:17:50.373 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:17:50.492 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.SparsifyViaLockOwner (119 ms) 2024-04-07T17:17:50.493 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.ObjectMapConsistentSnap 2024-04-07T17:17:50.493 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:17:51.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:17:50 smithi100 bash[74942]: cluster 2024-04-07T17:17:49.821756+0000 mgr.y (mgr.25175) 1874 : cluster [DBG] pgmap v3178: 649 pgs: 649 active+clean; 309 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 157 KiB/s rd, 13 KiB/s wr, 213 op/s 2024-04-07T17:17:51.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:17:50 smithi100 bash[74942]: cluster 2024-04-07T17:17:49.956524+0000 mon.a (mon.0) 3325 : cluster [DBG] osdmap e1730: 8 total, 8 up, 8 in 2024-04-07T17:17:51.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:17:50 smithi100 bash[76202]: cluster 2024-04-07T17:17:49.821756+0000 mgr.y (mgr.25175) 1874 : cluster [DBG] pgmap v3178: 649 pgs: 649 active+clean; 309 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 157 KiB/s rd, 13 KiB/s wr, 213 op/s 2024-04-07T17:17:51.205 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:17:50 smithi100 bash[76202]: cluster 2024-04-07T17:17:49.956524+0000 mon.a (mon.0) 3325 : cluster [DBG] osdmap e1730: 8 total, 8 up, 8 in 2024-04-07T17:17:51.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:17:50 smithi144 bash[66654]: cluster 2024-04-07T17:17:49.821756+0000 mgr.y (mgr.25175) 1874 : cluster [DBG] pgmap v3178: 649 pgs: 649 active+clean; 309 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 157 KiB/s rd, 13 KiB/s wr, 213 op/s 2024-04-07T17:17:51.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:17:50 smithi144 bash[66654]: cluster 2024-04-07T17:17:49.956524+0000 mon.a (mon.0) 3325 : cluster [DBG] osdmap e1730: 8 total, 8 up, 8 in 2024-04-07T17:17:52.318 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:17:51 smithi144 bash[66654]: cluster 2024-04-07T17:17:50.967612+0000 mon.a (mon.0) 3326 : cluster [DBG] osdmap e1731: 8 total, 8 up, 8 in 2024-04-07T17:17:52.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:17:51 smithi100 bash[74942]: cluster 2024-04-07T17:17:50.967612+0000 mon.a (mon.0) 3326 : cluster [DBG] osdmap e1731: 8 total, 8 up, 8 in 2024-04-07T17:17:52.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:17:51 smithi100 bash[76202]: cluster 2024-04-07T17:17:50.967612+0000 mon.a (mon.0) 3326 : cluster [DBG] osdmap e1731: 8 total, 8 up, 8 in 2024-04-07T17:17:52.617 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:17:52 smithi144 bash[95233]: ts=2024-04-07T17:17:52.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:17:53.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:17:52 smithi144 bash[66654]: cluster 2024-04-07T17:17:51.822968+0000 mgr.y (mgr.25175) 1875 : cluster [DBG] pgmap v3181: 649 pgs: 649 active+clean; 309 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 56 KiB/s rd, 5.7 KiB/s wr, 79 op/s 2024-04-07T17:17:53.368 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:17:52 smithi144 bash[66654]: cluster 2024-04-07T17:17:51.977536+0000 mon.a (mon.0) 3327 : cluster [DBG] osdmap e1732: 8 total, 8 up, 8 in 2024-04-07T17:17:53.368 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:17:52 smithi144 bash[66654]: cluster 2024-04-07T17:17:52.889793+0000 mon.a (mon.0) 3328 : cluster [DBG] osdmap e1733: 8 total, 8 up, 8 in 2024-04-07T17:17:53.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:17:52 smithi100 bash[74942]: cluster 2024-04-07T17:17:51.822968+0000 mgr.y (mgr.25175) 1875 : cluster [DBG] pgmap v3181: 649 pgs: 649 active+clean; 309 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 56 KiB/s rd, 5.7 KiB/s wr, 79 op/s 2024-04-07T17:17:53.455 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:17:52 smithi100 bash[74942]: cluster 2024-04-07T17:17:51.977536+0000 mon.a (mon.0) 3327 : cluster [DBG] osdmap e1732: 8 total, 8 up, 8 in 2024-04-07T17:17:53.455 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:17:52 smithi100 bash[74942]: cluster 2024-04-07T17:17:52.889793+0000 mon.a (mon.0) 3328 : cluster [DBG] osdmap e1733: 8 total, 8 up, 8 in 2024-04-07T17:17:53.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:17:52 smithi100 bash[76202]: cluster 2024-04-07T17:17:51.822968+0000 mgr.y (mgr.25175) 1875 : cluster [DBG] pgmap v3181: 649 pgs: 649 active+clean; 309 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 56 KiB/s rd, 5.7 KiB/s wr, 79 op/s 2024-04-07T17:17:53.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:17:52 smithi100 bash[76202]: cluster 2024-04-07T17:17:51.977536+0000 mon.a (mon.0) 3327 : cluster [DBG] osdmap e1732: 8 total, 8 up, 8 in 2024-04-07T17:17:53.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:17:52 smithi100 bash[76202]: cluster 2024-04-07T17:17:52.889793+0000 mon.a (mon.0) 3328 : cluster [DBG] osdmap e1733: 8 total, 8 up, 8 in 2024-04-07T17:17:55.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:17:54 smithi100 bash[74942]: cluster 2024-04-07T17:17:53.824880+0000 mgr.y (mgr.25175) 1876 : cluster [DBG] pgmap v3184: 649 pgs: 649 active+clean; 309 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 197 KiB/s rd, 9.7 KiB/s wr, 258 op/s 2024-04-07T17:17:55.205 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:17:54 smithi100 bash[74942]: cluster 2024-04-07T17:17:53.898140+0000 mon.a (mon.0) 3329 : cluster [DBG] osdmap e1734: 8 total, 8 up, 8 in 2024-04-07T17:17:55.205 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:17:54 smithi100 bash[76202]: cluster 2024-04-07T17:17:53.824880+0000 mgr.y (mgr.25175) 1876 : cluster [DBG] pgmap v3184: 649 pgs: 649 active+clean; 309 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 197 KiB/s rd, 9.7 KiB/s wr, 258 op/s 2024-04-07T17:17:55.205 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:17:54 smithi100 bash[76202]: cluster 2024-04-07T17:17:53.898140+0000 mon.a (mon.0) 3329 : cluster [DBG] osdmap e1734: 8 total, 8 up, 8 in 2024-04-07T17:17:55.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:17:54 smithi144 bash[66654]: cluster 2024-04-07T17:17:53.824880+0000 mgr.y (mgr.25175) 1876 : cluster [DBG] pgmap v3184: 649 pgs: 649 active+clean; 309 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 197 KiB/s rd, 9.7 KiB/s wr, 258 op/s 2024-04-07T17:17:55.368 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:17:54 smithi144 bash[66654]: cluster 2024-04-07T17:17:53.898140+0000 mon.a (mon.0) 3329 : cluster [DBG] osdmap e1734: 8 total, 8 up, 8 in 2024-04-07T17:17:56.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:17:55 smithi100 bash[74942]: cluster 2024-04-07T17:17:54.913389+0000 mon.a (mon.0) 3330 : cluster [DBG] osdmap e1735: 8 total, 8 up, 8 in 2024-04-07T17:17:56.205 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:17:55 smithi100 bash[76202]: cluster 2024-04-07T17:17:54.913389+0000 mon.a (mon.0) 3330 : cluster [DBG] osdmap e1735: 8 total, 8 up, 8 in 2024-04-07T17:17:56.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:17:55 smithi144 bash[66654]: cluster 2024-04-07T17:17:54.913389+0000 mon.a (mon.0) 3330 : cluster [DBG] osdmap e1735: 8 total, 8 up, 8 in 2024-04-07T17:17:57.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:17:56 smithi100 bash[74942]: cluster 2024-04-07T17:17:55.826717+0000 mgr.y (mgr.25175) 1877 : cluster [DBG] pgmap v3187: 649 pgs: 649 active+clean; 309 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 216 KiB/s rd, 11 KiB/s wr, 282 op/s 2024-04-07T17:17:57.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:17:56 smithi100 bash[74942]: cluster 2024-04-07T17:17:55.922858+0000 mon.a (mon.0) 3331 : cluster [DBG] osdmap e1736: 8 total, 8 up, 8 in 2024-04-07T17:17:57.205 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:17:56 smithi100 bash[76202]: cluster 2024-04-07T17:17:55.826717+0000 mgr.y (mgr.25175) 1877 : cluster [DBG] pgmap v3187: 649 pgs: 649 active+clean; 309 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 216 KiB/s rd, 11 KiB/s wr, 282 op/s 2024-04-07T17:17:57.205 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:17:56 smithi100 bash[76202]: cluster 2024-04-07T17:17:55.922858+0000 mon.a (mon.0) 3331 : cluster [DBG] osdmap e1736: 8 total, 8 up, 8 in 2024-04-07T17:17:57.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:17:56 smithi144 bash[66654]: cluster 2024-04-07T17:17:55.826717+0000 mgr.y (mgr.25175) 1877 : cluster [DBG] pgmap v3187: 649 pgs: 649 active+clean; 309 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 216 KiB/s rd, 11 KiB/s wr, 282 op/s 2024-04-07T17:17:57.368 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:17:56 smithi144 bash[66654]: cluster 2024-04-07T17:17:55.922858+0000 mon.a (mon.0) 3331 : cluster [DBG] osdmap e1736: 8 total, 8 up, 8 in 2024-04-07T17:17:58.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:17:57 smithi100 bash[76202]: cluster 2024-04-07T17:17:56.930294+0000 mon.a (mon.0) 3332 : cluster [DBG] osdmap e1737: 8 total, 8 up, 8 in 2024-04-07T17:17:58.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:17:57 smithi100 bash[76202]: audit 2024-04-07T17:17:57.845334+0000 mon.a (mon.0) 3333 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:17:58.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:17:57 smithi100 bash[74942]: cluster 2024-04-07T17:17:56.930294+0000 mon.a (mon.0) 3332 : cluster [DBG] osdmap e1737: 8 total, 8 up, 8 in 2024-04-07T17:17:58.205 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:17:57 smithi100 bash[74942]: audit 2024-04-07T17:17:57.845334+0000 mon.a (mon.0) 3333 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:17:58.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:17:57 smithi144 bash[66654]: cluster 2024-04-07T17:17:56.930294+0000 mon.a (mon.0) 3332 : cluster [DBG] osdmap e1737: 8 total, 8 up, 8 in 2024-04-07T17:17:58.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:17:57 smithi144 bash[66654]: audit 2024-04-07T17:17:57.845334+0000 mon.a (mon.0) 3333 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:17:58.704 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:17:58 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:17:58] "GET /metrics HTTP/1.1" 200 75601 "" "Prometheus/2.43.0" 2024-04-07T17:17:59.368 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:17:58 smithi144 bash[66654]: cluster 2024-04-07T17:17:57.830371+0000 mgr.y (mgr.25175) 1878 : cluster [DBG] pgmap v3190: 649 pgs: 649 active+clean; 309 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 81 KiB/s rd, 5.2 KiB/s wr, 108 op/s 2024-04-07T17:17:59.368 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:17:58 smithi144 bash[66654]: cluster 2024-04-07T17:17:57.940347+0000 mon.a (mon.0) 3334 : cluster [DBG] osdmap e1738: 8 total, 8 up, 8 in 2024-04-07T17:17:59.368 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:17:58 smithi144 bash[66654]: audit 2024-04-07T17:17:58.636481+0000 mon.a (mon.0) 3335 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:17:59.368 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:17:58 smithi144 bash[66654]: audit 2024-04-07T17:17:58.638088+0000 mon.a (mon.0) 3336 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:17:59.368 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:17:58 smithi144 bash[66654]: audit 2024-04-07T17:17:58.649115+0000 mon.a (mon.0) 3337 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:17:59.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:17:58 smithi100 bash[74942]: cluster 2024-04-07T17:17:57.830371+0000 mgr.y (mgr.25175) 1878 : cluster [DBG] pgmap v3190: 649 pgs: 649 active+clean; 309 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 81 KiB/s rd, 5.2 KiB/s wr, 108 op/s 2024-04-07T17:17:59.455 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:17:58 smithi100 bash[74942]: cluster 2024-04-07T17:17:57.940347+0000 mon.a (mon.0) 3334 : cluster [DBG] osdmap e1738: 8 total, 8 up, 8 in 2024-04-07T17:17:59.455 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:17:58 smithi100 bash[74942]: audit 2024-04-07T17:17:58.636481+0000 mon.a (mon.0) 3335 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:17:59.455 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:17:58 smithi100 bash[74942]: audit 2024-04-07T17:17:58.638088+0000 mon.a (mon.0) 3336 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:17:59.455 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:17:58 smithi100 bash[74942]: audit 2024-04-07T17:17:58.649115+0000 mon.a (mon.0) 3337 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:17:59.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:17:58 smithi100 bash[76202]: cluster 2024-04-07T17:17:57.830371+0000 mgr.y (mgr.25175) 1878 : cluster [DBG] pgmap v3190: 649 pgs: 649 active+clean; 309 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 81 KiB/s rd, 5.2 KiB/s wr, 108 op/s 2024-04-07T17:17:59.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:17:58 smithi100 bash[76202]: cluster 2024-04-07T17:17:57.940347+0000 mon.a (mon.0) 3334 : cluster [DBG] osdmap e1738: 8 total, 8 up, 8 in 2024-04-07T17:17:59.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:17:58 smithi100 bash[76202]: audit 2024-04-07T17:17:58.636481+0000 mon.a (mon.0) 3335 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:17:59.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:17:58 smithi100 bash[76202]: audit 2024-04-07T17:17:58.638088+0000 mon.a (mon.0) 3336 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:17:59.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:17:58 smithi100 bash[76202]: audit 2024-04-07T17:17:58.649115+0000 mon.a (mon.0) 3337 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:18:00.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:17:59 smithi144 bash[66654]: cluster 2024-04-07T17:17:58.945772+0000 mon.a (mon.0) 3338 : cluster [DBG] osdmap e1739: 8 total, 8 up, 8 in 2024-04-07T17:18:00.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:17:59 smithi100 bash[74942]: cluster 2024-04-07T17:17:58.945772+0000 mon.a (mon.0) 3338 : cluster [DBG] osdmap e1739: 8 total, 8 up, 8 in 2024-04-07T17:18:00.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:17:59 smithi100 bash[76202]: cluster 2024-04-07T17:17:58.945772+0000 mon.a (mon.0) 3338 : cluster [DBG] osdmap e1739: 8 total, 8 up, 8 in 2024-04-07T17:18:01.202 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.ObjectMapConsistentSnap (10710 ms) 2024-04-07T17:18:01.202 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.Metadata 2024-04-07T17:18:01.202 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:18:01.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:18:00 smithi144 bash[66654]: cluster 2024-04-07T17:17:59.832917+0000 mgr.y (mgr.25175) 1879 : cluster [DBG] pgmap v3193: 649 pgs: 649 active+clean; 309 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 62 KiB/s rd, 3.7 KiB/s wr, 82 op/s 2024-04-07T17:18:01.368 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:18:00 smithi144 bash[66654]: cluster 2024-04-07T17:17:59.971285+0000 mon.a (mon.0) 3339 : cluster [DBG] osdmap e1740: 8 total, 8 up, 8 in 2024-04-07T17:18:01.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:18:00 smithi100 bash[76202]: cluster 2024-04-07T17:17:59.832917+0000 mgr.y (mgr.25175) 1879 : cluster [DBG] pgmap v3193: 649 pgs: 649 active+clean; 309 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 62 KiB/s rd, 3.7 KiB/s wr, 82 op/s 2024-04-07T17:18:01.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:18:00 smithi100 bash[76202]: cluster 2024-04-07T17:17:59.971285+0000 mon.a (mon.0) 3339 : cluster [DBG] osdmap e1740: 8 total, 8 up, 8 in 2024-04-07T17:18:01.455 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:18:00 smithi100 bash[74942]: cluster 2024-04-07T17:17:59.832917+0000 mgr.y (mgr.25175) 1879 : cluster [DBG] pgmap v3193: 649 pgs: 649 active+clean; 309 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 62 KiB/s rd, 3.7 KiB/s wr, 82 op/s 2024-04-07T17:18:01.455 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:18:00 smithi100 bash[74942]: cluster 2024-04-07T17:17:59.971285+0000 mon.a (mon.0) 3339 : cluster [DBG] osdmap e1740: 8 total, 8 up, 8 in 2024-04-07T17:18:02.247 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.Metadata (1045 ms) 2024-04-07T17:18:02.247 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.MetadataPP 2024-04-07T17:18:02.248 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:18:02.318 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:18:01 smithi144 bash[66654]: audit 2024-04-07T17:18:01.829668+0000 mon.a (mon.0) 3340 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:18:02.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:18:01 smithi100 bash[74942]: audit 2024-04-07T17:18:01.829668+0000 mon.a (mon.0) 3340 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:18:02.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:18:01 smithi100 bash[76202]: audit 2024-04-07T17:18:01.829668+0000 mon.a (mon.0) 3340 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:18:02.617 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:18:02 smithi144 bash[95233]: ts=2024-04-07T17:18:02.317Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:18:03.278 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.MetadataPP (1030 ms) 2024-04-07T17:18:03.278 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.UpdateFeatures 2024-04-07T17:18:03.278 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:18:03.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:18:02 smithi144 bash[66654]: cluster 2024-04-07T17:18:01.834710+0000 mgr.y (mgr.25175) 1880 : cluster [DBG] pgmap v3195: 649 pgs: 649 active+clean; 309 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 51 KiB/s rd, 3.1 KiB/s wr, 67 op/s 2024-04-07T17:18:03.368 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:18:02 smithi144 bash[66654]: cluster 2024-04-07T17:18:01.987199+0000 mon.a (mon.0) 3341 : cluster [DBG] osdmap e1741: 8 total, 8 up, 8 in 2024-04-07T17:18:03.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:18:03 smithi100 bash[74942]: cluster 2024-04-07T17:18:01.834710+0000 mgr.y (mgr.25175) 1880 : cluster [DBG] pgmap v3195: 649 pgs: 649 active+clean; 309 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 51 KiB/s rd, 3.1 KiB/s wr, 67 op/s 2024-04-07T17:18:03.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:18:03 smithi100 bash[74942]: cluster 2024-04-07T17:18:01.987199+0000 mon.a (mon.0) 3341 : cluster [DBG] osdmap e1741: 8 total, 8 up, 8 in 2024-04-07T17:18:03.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:18:03 smithi100 bash[76202]: cluster 2024-04-07T17:18:01.834710+0000 mgr.y (mgr.25175) 1880 : cluster [DBG] pgmap v3195: 649 pgs: 649 active+clean; 309 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 51 KiB/s rd, 3.1 KiB/s wr, 67 op/s 2024-04-07T17:18:03.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:18:03 smithi100 bash[76202]: cluster 2024-04-07T17:18:01.987199+0000 mon.a (mon.0) 3341 : cluster [DBG] osdmap e1741: 8 total, 8 up, 8 in 2024-04-07T17:18:03.578 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.UpdateFeatures (301 ms) 2024-04-07T17:18:03.578 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.FeaturesBitmaskString 2024-04-07T17:18:03.578 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.FeaturesBitmaskString (0 ms) 2024-04-07T17:18:03.578 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.RebuildObjectMap 2024-04-07T17:18:03.578 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:18:04.157 INFO:tasks.workunit.client.0.smithi100.stdout:0.00% done 2024-04-07T17:18:04.157 INFO:tasks.workunit.client.0.smithi100.stdout:25.00% done 2024-04-07T17:18:04.157 INFO:tasks.workunit.client.0.smithi100.stdout:50.00% done 2024-04-07T17:18:04.157 INFO:tasks.workunit.client.0.smithi100.stdout:75.00% done 2024-04-07T17:18:04.157 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.RebuildObjectMap (578 ms) 2024-04-07T17:18:04.157 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.RebuildNewObjectMap 2024-04-07T17:18:04.246 INFO:tasks.workunit.client.0.smithi100.stdout:0.00% done 2024-04-07T17:18:04.246 INFO:tasks.workunit.client.0.smithi100.stdout:25.00% done 2024-04-07T17:18:04.246 INFO:tasks.workunit.client.0.smithi100.stdout:50.00% done 2024-04-07T17:18:04.246 INFO:tasks.workunit.client.0.smithi100.stdout:75.00% done 2024-04-07T17:18:04.247 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.RebuildNewObjectMap (90 ms) 2024-04-07T17:18:04.247 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.CheckObjectMap 2024-04-07T17:18:04.247 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:18:04.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:18:04 smithi144 bash[66654]: cluster 2024-04-07T17:18:02.988112+0000 mon.a (mon.0) 3342 : cluster [DBG] osdmap e1742: 8 total, 8 up, 8 in 2024-04-07T17:18:04.368 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:18:04 smithi144 bash[66654]: cluster 2024-04-07T17:18:03.837611+0000 mgr.y (mgr.25175) 1881 : cluster [DBG] pgmap v3198: 649 pgs: 649 active+clean; 309 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 366 KiB/s rd, 145 KiB/s wr, 590 op/s 2024-04-07T17:18:04.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:18:04 smithi100 bash[74942]: cluster 2024-04-07T17:18:02.988112+0000 mon.a (mon.0) 3342 : cluster [DBG] osdmap e1742: 8 total, 8 up, 8 in 2024-04-07T17:18:04.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:18:04 smithi100 bash[74942]: cluster 2024-04-07T17:18:03.837611+0000 mgr.y (mgr.25175) 1881 : cluster [DBG] pgmap v3198: 649 pgs: 649 active+clean; 309 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 366 KiB/s rd, 145 KiB/s wr, 590 op/s 2024-04-07T17:18:04.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:18:04 smithi100 bash[76202]: cluster 2024-04-07T17:18:02.988112+0000 mon.a (mon.0) 3342 : cluster [DBG] osdmap e1742: 8 total, 8 up, 8 in 2024-04-07T17:18:04.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:18:04 smithi100 bash[76202]: cluster 2024-04-07T17:18:03.837611+0000 mgr.y (mgr.25175) 1881 : cluster [DBG] pgmap v3198: 649 pgs: 649 active+clean; 309 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 366 KiB/s rd, 145 KiB/s wr, 590 op/s 2024-04-07T17:18:05.214 INFO:tasks.workunit.client.0.smithi100.stdout:0.00% done 2024-04-07T17:18:05.215 INFO:tasks.workunit.client.0.smithi100.stdout:25.00% done 2024-04-07T17:18:05.215 INFO:tasks.workunit.client.0.smithi100.stdout:50.00% done 2024-04-07T17:18:05.215 INFO:tasks.workunit.client.0.smithi100.stdout:75.00% done 2024-04-07T17:18:05.215 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.CheckObjectMap (968 ms) 2024-04-07T17:18:05.215 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.BlockingAIO 2024-04-07T17:18:05.215 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:18:05.312 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.BlockingAIO (97 ms) 2024-04-07T17:18:05.312 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.ExclusiveLockTransition 2024-04-07T17:18:05.312 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:18:05.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:18:05 smithi144 bash[66654]: cluster 2024-04-07T17:18:04.013817+0000 mon.a (mon.0) 3343 : cluster [DBG] osdmap e1743: 8 total, 8 up, 8 in 2024-04-07T17:18:05.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:18:05 smithi100 bash[74942]: cluster 2024-04-07T17:18:04.013817+0000 mon.a (mon.0) 3343 : cluster [DBG] osdmap e1743: 8 total, 8 up, 8 in 2024-04-07T17:18:05.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:18:05 smithi100 bash[76202]: cluster 2024-04-07T17:18:04.013817+0000 mon.a (mon.0) 3343 : cluster [DBG] osdmap e1743: 8 total, 8 up, 8 in 2024-04-07T17:18:05.586 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.ExclusiveLockTransition (274 ms) 2024-04-07T17:18:05.586 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.ExclusiveLockReadTransition 2024-04-07T17:18:05.586 INFO:tasks.workunit.client.0.smithi100.stdout:./src/test/librbd/test_librbd.cc:10433: Skipped 2024-04-07T17:18:05.586 INFO:tasks.workunit.client.0.smithi100.stdout:Skipping due to unmet REQUIRE 2024-04-07T17:18:05.586 INFO:tasks.workunit.client.0.smithi100.stdout:[ SKIPPED ] TestLibRBD.ExclusiveLockReadTransition (0 ms) 2024-04-07T17:18:05.586 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.CacheMayCopyOnWrite 2024-04-07T17:18:05.586 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:18:06.248 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.CacheMayCopyOnWrite (662 ms) 2024-04-07T17:18:06.248 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.FlushEmptyOpsOnExternalSnapshot 2024-04-07T17:18:06.248 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:18:06.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:18:06 smithi144 bash[66654]: cluster 2024-04-07T17:18:05.023382+0000 mon.a (mon.0) 3344 : cluster [DBG] osdmap e1744: 8 total, 8 up, 8 in 2024-04-07T17:18:06.368 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:18:06 smithi144 bash[66654]: cluster 2024-04-07T17:18:05.838793+0000 mgr.y (mgr.25175) 1882 : cluster [DBG] pgmap v3201: 649 pgs: 649 active+clean; 309 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 535 KiB/s rd, 210 KiB/s wr, 865 op/s 2024-04-07T17:18:06.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:18:06 smithi100 bash[76202]: cluster 2024-04-07T17:18:05.023382+0000 mon.a (mon.0) 3344 : cluster [DBG] osdmap e1744: 8 total, 8 up, 8 in 2024-04-07T17:18:06.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:18:06 smithi100 bash[76202]: cluster 2024-04-07T17:18:05.838793+0000 mgr.y (mgr.25175) 1882 : cluster [DBG] pgmap v3201: 649 pgs: 649 active+clean; 309 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 535 KiB/s rd, 210 KiB/s wr, 865 op/s 2024-04-07T17:18:06.455 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:18:06 smithi100 bash[74942]: cluster 2024-04-07T17:18:05.023382+0000 mon.a (mon.0) 3344 : cluster [DBG] osdmap e1744: 8 total, 8 up, 8 in 2024-04-07T17:18:06.455 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:18:06 smithi100 bash[74942]: cluster 2024-04-07T17:18:05.838793+0000 mgr.y (mgr.25175) 1882 : cluster [DBG] pgmap v3201: 649 pgs: 649 active+clean; 309 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 535 KiB/s rd, 210 KiB/s wr, 865 op/s 2024-04-07T17:18:07.136 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.FlushEmptyOpsOnExternalSnapshot (889 ms) 2024-04-07T17:18:07.137 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.TestImageOptions 2024-04-07T17:18:07.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:18:07 smithi144 bash[66654]: cluster 2024-04-07T17:18:06.037463+0000 mon.a (mon.0) 3345 : cluster [DBG] osdmap e1745: 8 total, 8 up, 8 in 2024-04-07T17:18:07.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:18:07 smithi100 bash[74942]: cluster 2024-04-07T17:18:06.037463+0000 mon.a (mon.0) 3345 : cluster [DBG] osdmap e1745: 8 total, 8 up, 8 in 2024-04-07T17:18:07.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:18:07 smithi100 bash[76202]: cluster 2024-04-07T17:18:06.037463+0000 mon.a (mon.0) 3345 : cluster [DBG] osdmap e1745: 8 total, 8 up, 8 in 2024-04-07T17:18:08.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:18:08 smithi144 bash[66654]: cluster 2024-04-07T17:18:07.036400+0000 mon.a (mon.0) 3346 : cluster [DBG] osdmap e1746: 8 total, 8 up, 8 in 2024-04-07T17:18:08.368 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:18:08 smithi144 bash[66654]: cluster 2024-04-07T17:18:07.841794+0000 mgr.y (mgr.25175) 1883 : cluster [DBG] pgmap v3204: 649 pgs: 649 active+clean; 309 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 413 KiB/s rd, 118 KiB/s wr, 573 op/s 2024-04-07T17:18:08.401 INFO:tasks.workunit.client.0.smithi100.stdout:0.00% done 2024-04-07T17:18:08.401 INFO:tasks.workunit.client.0.smithi100.stdout:100.00% done 2024-04-07T17:18:08.402 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.TestImageOptions (1265 ms) 2024-04-07T17:18:08.402 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.TestImageOptionsPP 2024-04-07T17:18:08.428 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:18:08 smithi100 bash[74942]: cluster 2024-04-07T17:18:07.036400+0000 mon.a (mon.0) 3346 : cluster [DBG] osdmap e1746: 8 total, 8 up, 8 in 2024-04-07T17:18:08.428 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:18:08 smithi100 bash[74942]: cluster 2024-04-07T17:18:07.841794+0000 mgr.y (mgr.25175) 1883 : cluster [DBG] pgmap v3204: 649 pgs: 649 active+clean; 309 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 413 KiB/s rd, 118 KiB/s wr, 573 op/s 2024-04-07T17:18:08.428 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:18:08 smithi100 bash[76202]: cluster 2024-04-07T17:18:07.036400+0000 mon.a (mon.0) 3346 : cluster [DBG] osdmap e1746: 8 total, 8 up, 8 in 2024-04-07T17:18:08.429 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:18:08 smithi100 bash[76202]: cluster 2024-04-07T17:18:07.841794+0000 mgr.y (mgr.25175) 1883 : cluster [DBG] pgmap v3204: 649 pgs: 649 active+clean; 309 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 413 KiB/s rd, 118 KiB/s wr, 573 op/s 2024-04-07T17:18:08.704 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:18:08 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:18:08] "GET /metrics HTTP/1.1" 200 75597 "" "Prometheus/2.43.0" 2024-04-07T17:18:09.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:18:09 smithi144 bash[66654]: cluster 2024-04-07T17:18:08.075234+0000 mon.a (mon.0) 3347 : cluster [DBG] osdmap e1747: 8 total, 8 up, 8 in 2024-04-07T17:18:09.412 INFO:tasks.workunit.client.0.smithi100.stdout:0.00% done 2024-04-07T17:18:09.412 INFO:tasks.workunit.client.0.smithi100.stdout:100.00% done 2024-04-07T17:18:09.412 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.TestImageOptionsPP (1010 ms) 2024-04-07T17:18:09.412 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.EventSocketPipe 2024-04-07T17:18:09.412 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.EventSocketPipe (0 ms) 2024-04-07T17:18:09.412 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.EventSocketEventfd 2024-04-07T17:18:09.412 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.EventSocketEventfd (0 ms) 2024-04-07T17:18:09.412 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.ImagePollIO 2024-04-07T17:18:09.413 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:18:09.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:18:09 smithi100 bash[74942]: cluster 2024-04-07T17:18:08.075234+0000 mon.a (mon.0) 3347 : cluster [DBG] osdmap e1747: 8 total, 8 up, 8 in 2024-04-07T17:18:09.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:18:09 smithi100 bash[76202]: cluster 2024-04-07T17:18:08.075234+0000 mon.a (mon.0) 3347 : cluster [DBG] osdmap e1747: 8 total, 8 up, 8 in 2024-04-07T17:18:09.517 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 512 2024-04-07T17:18:09.517 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 512 2024-04-07T17:18:09.517 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 512 2024-04-07T17:18:09.517 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 512 2024-04-07T17:18:09.517 INFO:tasks.workunit.client.0.smithi100.stdout:wrote: 512 2024-04-07T17:18:09.517 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:18:09.517 INFO:tasks.workunit.client.0.smithi100.stdout:started write 2024-04-07T17:18:09.517 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:18:09.517 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:18:09.517 INFO:tasks.workunit.client.0.smithi100.stdout:finished write 2024-04-07T17:18:09.517 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:18:09.517 INFO:tasks.workunit.client.0.smithi100.stdout:started write 2024-04-07T17:18:09.518 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:18:09.518 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:18:09.518 INFO:tasks.workunit.client.0.smithi100.stdout:finished write 2024-04-07T17:18:09.518 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:18:09.518 INFO:tasks.workunit.client.0.smithi100.stdout:started write 2024-04-07T17:18:09.518 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:18:09.518 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:18:09.518 INFO:tasks.workunit.client.0.smithi100.stdout:finished write 2024-04-07T17:18:09.518 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:18:09.518 INFO:tasks.workunit.client.0.smithi100.stdout:started write 2024-04-07T17:18:09.518 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:18:09.518 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:18:09.518 INFO:tasks.workunit.client.0.smithi100.stdout:finished write 2024-04-07T17:18:09.518 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:18:09.519 INFO:tasks.workunit.client.0.smithi100.stdout:started write 2024-04-07T17:18:09.519 INFO:tasks.workunit.client.0.smithi100.stdout:write completion cb called! 2024-04-07T17:18:09.519 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 0 2024-04-07T17:18:09.519 INFO:tasks.workunit.client.0.smithi100.stdout:finished write 2024-04-07T17:18:09.519 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:18:09.519 INFO:tasks.workunit.client.0.smithi100.stdout:started read 2024-04-07T17:18:09.519 INFO:tasks.workunit.client.0.smithi100.stdout:read completion cb called! 2024-04-07T17:18:09.519 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 512 2024-04-07T17:18:09.519 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:18:09.519 INFO:tasks.workunit.client.0.smithi100.stdout:started read 2024-04-07T17:18:09.519 INFO:tasks.workunit.client.0.smithi100.stdout:read completion cb called! 2024-04-07T17:18:09.519 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 512 2024-04-07T17:18:09.519 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:18:09.519 INFO:tasks.workunit.client.0.smithi100.stdout:started read 2024-04-07T17:18:09.520 INFO:tasks.workunit.client.0.smithi100.stdout:read completion cb called! 2024-04-07T17:18:09.520 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 512 2024-04-07T17:18:09.520 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:18:09.520 INFO:tasks.workunit.client.0.smithi100.stdout:started read 2024-04-07T17:18:09.520 INFO:tasks.workunit.client.0.smithi100.stdout:read completion cb called! 2024-04-07T17:18:09.520 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 512 2024-04-07T17:18:09.520 INFO:tasks.workunit.client.0.smithi100.stdout:created completion 2024-04-07T17:18:09.520 INFO:tasks.workunit.client.0.smithi100.stdout:started read 2024-04-07T17:18:09.520 INFO:tasks.workunit.client.0.smithi100.stdout:read completion cb called! 2024-04-07T17:18:09.520 INFO:tasks.workunit.client.0.smithi100.stdout:return value is: 512 2024-04-07T17:18:09.520 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.ImagePollIO (105 ms) 2024-04-07T17:18:09.520 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.Mirror 2024-04-07T17:18:09.528 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:18:09.551 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:18:10.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:18:10 smithi144 bash[66654]: cluster 2024-04-07T17:18:09.082952+0000 mon.a (mon.0) 3348 : cluster [DBG] osdmap e1748: 8 total, 8 up, 8 in 2024-04-07T17:18:10.368 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:18:10 smithi144 bash[66654]: cluster 2024-04-07T17:18:09.843671+0000 mgr.y (mgr.25175) 1884 : cluster [DBG] pgmap v3207: 649 pgs: 649 active+clean; 311 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 401 KiB/s rd, 499 KiB/s wr, 524 op/s 2024-04-07T17:18:10.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:18:10 smithi100 bash[74942]: cluster 2024-04-07T17:18:09.082952+0000 mon.a (mon.0) 3348 : cluster [DBG] osdmap e1748: 8 total, 8 up, 8 in 2024-04-07T17:18:10.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:18:10 smithi100 bash[74942]: cluster 2024-04-07T17:18:09.843671+0000 mgr.y (mgr.25175) 1884 : cluster [DBG] pgmap v3207: 649 pgs: 649 active+clean; 311 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 401 KiB/s rd, 499 KiB/s wr, 524 op/s 2024-04-07T17:18:10.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:18:10 smithi100 bash[76202]: cluster 2024-04-07T17:18:09.082952+0000 mon.a (mon.0) 3348 : cluster [DBG] osdmap e1748: 8 total, 8 up, 8 in 2024-04-07T17:18:10.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:18:10 smithi100 bash[76202]: cluster 2024-04-07T17:18:09.843671+0000 mgr.y (mgr.25175) 1884 : cluster [DBG] pgmap v3207: 649 pgs: 649 active+clean; 311 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 401 KiB/s rd, 499 KiB/s wr, 524 op/s 2024-04-07T17:18:10.536 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.Mirror (1020 ms) 2024-04-07T17:18:10.536 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.MirrorPeerAttributes 2024-04-07T17:18:10.603 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.MirrorPeerAttributes (67 ms) 2024-04-07T17:18:10.603 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.CreateWithMirrorEnabled 2024-04-07T17:18:11.368 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:18:11 smithi144 bash[66654]: cluster 2024-04-07T17:18:10.094705+0000 mon.a (mon.0) 3349 : cluster [DBG] osdmap e1749: 8 total, 8 up, 8 in 2024-04-07T17:18:11.368 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:18:11 smithi144 bash[66654]: audit 2024-04-07T17:18:10.451176+0000 mon.a (mon.0) 3350 : audit [INF] from='client.? 172.21.15.100:0/1677144245' entity='client.admin' cmd=[{"prefix": "config-key rm", "key": "rbd/mirror/peer/91/uuid4"}]: dispatch 2024-04-07T17:18:11.368 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:18:11 smithi144 bash[66654]: audit 2024-04-07T17:18:10.457832+0000 mon.a (mon.0) 3351 : audit [INF] from='client.? 172.21.15.100:0/1677144245' entity='client.admin' cmd='[{"prefix": "config-key rm", "key": "rbd/mirror/peer/91/uuid4"}]': finished 2024-04-07T17:18:11.368 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:18:11 smithi144 bash[66654]: audit 2024-04-07T17:18:10.463562+0000 mon.a (mon.0) 3352 : audit [INF] from='client.? 172.21.15.100:0/1677144245' entity='client.admin' cmd=[{"prefix": "config-key rm", "key": "rbd/mirror/peer/91/dee0f31f-3fe3-4ccb-865a-77df0fe586dc"}]: dispatch 2024-04-07T17:18:11.368 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:18:11 smithi144 bash[66654]: audit 2024-04-07T17:18:10.472280+0000 mon.a (mon.0) 3353 : audit [INF] from='client.? 172.21.15.100:0/1677144245' entity='client.admin' cmd='[{"prefix": "config-key rm", "key": "rbd/mirror/peer/91/dee0f31f-3fe3-4ccb-865a-77df0fe586dc"}]': finished 2024-04-07T17:18:11.368 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:18:11 smithi144 bash[66654]: audit 2024-04-07T17:18:10.493978+0000 mon.a (mon.0) 3354 : audit [INF] from='client.? 172.21.15.100:0/1677144245' entity='client.admin' cmd=[{"prefix": "config-key rm", "key": "rbd/mirror/peer/91/5962f148-835e-454c-a20a-2dfd6ed4277c"}]: dispatch 2024-04-07T17:18:11.368 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:18:11 smithi144 bash[66654]: audit 2024-04-07T17:18:10.501171+0000 mon.a (mon.0) 3355 : audit [INF] from='client.? 172.21.15.100:0/1677144245' entity='client.admin' cmd='[{"prefix": "config-key rm", "key": "rbd/mirror/peer/91/5962f148-835e-454c-a20a-2dfd6ed4277c"}]': finished 2024-04-07T17:18:11.368 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:18:11 smithi144 bash[66654]: audit 2024-04-07T17:18:10.512320+0000 mon.a (mon.0) 3356 : audit [INF] from='client.? 172.21.15.100:0/1677144245' entity='client.admin' cmd=[{"prefix": "config-key rm", "key": "rbd/mirror/peer/91/69d4623e-37f2-4ccb-a5c4-6eb93f9ba0db"}]: dispatch 2024-04-07T17:18:11.369 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:18:11 smithi144 bash[66654]: audit 2024-04-07T17:18:10.521057+0000 mon.a (mon.0) 3357 : audit [INF] from='client.? 172.21.15.100:0/1677144245' entity='client.admin' cmd='[{"prefix": "config-key rm", "key": "rbd/mirror/peer/91/69d4623e-37f2-4ccb-a5c4-6eb93f9ba0db"}]': finished 2024-04-07T17:18:11.369 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:18:11 smithi144 bash[66654]: audit 2024-04-07T17:18:10.545953+0000 mon.a (mon.0) 3358 : audit [DBG] from='client.? 172.21.15.100:0/1677144245' entity='client.admin' cmd=[{"prefix": "config-key get", "key": "rbd/mirror/peer/91/44e84396-d839-4080-bec3-076d7f9dae90"}]: dispatch 2024-04-07T17:18:11.369 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:18:11 smithi144 bash[66654]: audit 2024-04-07T17:18:10.556570+0000 mon.a (mon.0) 3359 : audit [INF] from='client.? 172.21.15.100:0/1677144245' entity='client.admin' 2024-04-07T17:18:11.369 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:18:11 smithi144 bash[66654]: audit 2024-04-07T17:18:10.557505+0000 mon.a (mon.0) 3360 : audit [DBG] from='client.? 172.21.15.100:0/1677144245' entity='client.admin' cmd=[{"prefix": "config-key get", "key": "rbd/mirror/peer/91/44e84396-d839-4080-bec3-076d7f9dae90"}]: dispatch 2024-04-07T17:18:11.369 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:18:11 smithi144 bash[66654]: audit 2024-04-07T17:18:10.558093+0000 mon.a (mon.0) 3361 : audit [INF] from='client.? 172.21.15.100:0/1677144245' entity='client.admin' cmd=[{"prefix": "config-key rm", "key": "rbd/mirror/peer/91/44e84396-d839-4080-bec3-076d7f9dae90"}]: dispatch 2024-04-07T17:18:11.369 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:18:11 smithi144 bash[66654]: audit 2024-04-07T17:18:10.578487+0000 mon.a (mon.0) 3362 : audit [INF] from='client.? 172.21.15.100:0/1677144245' entity='client.admin' cmd='[{"prefix": "config-key rm", "key": "rbd/mirror/peer/91/44e84396-d839-4080-bec3-076d7f9dae90"}]': finished 2024-04-07T17:18:11.455 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:18:11 smithi100 bash[74942]: cluster 2024-04-07T17:18:10.094705+0000 mon.a (mon.0) 3349 : cluster [DBG] osdmap e1749: 8 total, 8 up, 8 in 2024-04-07T17:18:11.455 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:18:11 smithi100 bash[74942]: audit 2024-04-07T17:18:10.451176+0000 mon.a (mon.0) 3350 : audit [INF] from='client.? 172.21.15.100:0/1677144245' entity='client.admin' cmd=[{"prefix": "config-key rm", "key": "rbd/mirror/peer/91/uuid4"}]: dispatch 2024-04-07T17:18:11.455 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:18:11 smithi100 bash[74942]: audit 2024-04-07T17:18:10.457832+0000 mon.a (mon.0) 3351 : audit [INF] from='client.? 172.21.15.100:0/1677144245' entity='client.admin' cmd='[{"prefix": "config-key rm", "key": "rbd/mirror/peer/91/uuid4"}]': finished 2024-04-07T17:18:11.455 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:18:11 smithi100 bash[74942]: audit 2024-04-07T17:18:10.463562+0000 mon.a (mon.0) 3352 : audit [INF] from='client.? 172.21.15.100:0/1677144245' entity='client.admin' cmd=[{"prefix": "config-key rm", "key": "rbd/mirror/peer/91/dee0f31f-3fe3-4ccb-865a-77df0fe586dc"}]: dispatch 2024-04-07T17:18:11.455 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:18:11 smithi100 bash[74942]: audit 2024-04-07T17:18:10.472280+0000 mon.a (mon.0) 3353 : audit [INF] from='client.? 172.21.15.100:0/1677144245' entity='client.admin' cmd='[{"prefix": "config-key rm", "key": "rbd/mirror/peer/91/dee0f31f-3fe3-4ccb-865a-77df0fe586dc"}]': finished 2024-04-07T17:18:11.455 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:18:11 smithi100 bash[74942]: audit 2024-04-07T17:18:10.493978+0000 mon.a (mon.0) 3354 : audit [INF] from='client.? 172.21.15.100:0/1677144245' entity='client.admin' cmd=[{"prefix": "config-key rm", "key": "rbd/mirror/peer/91/5962f148-835e-454c-a20a-2dfd6ed4277c"}]: dispatch 2024-04-07T17:18:11.455 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:18:11 smithi100 bash[74942]: audit 2024-04-07T17:18:10.501171+0000 mon.a (mon.0) 3355 : audit [INF] from='client.? 172.21.15.100:0/1677144245' entity='client.admin' cmd='[{"prefix": "config-key rm", "key": "rbd/mirror/peer/91/5962f148-835e-454c-a20a-2dfd6ed4277c"}]': finished 2024-04-07T17:18:11.455 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:18:11 smithi100 bash[74942]: audit 2024-04-07T17:18:10.512320+0000 mon.a (mon.0) 3356 : audit [INF] from='client.? 172.21.15.100:0/1677144245' entity='client.admin' cmd=[{"prefix": "config-key rm", "key": "rbd/mirror/peer/91/69d4623e-37f2-4ccb-a5c4-6eb93f9ba0db"}]: dispatch 2024-04-07T17:18:11.455 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:18:11 smithi100 bash[74942]: audit 2024-04-07T17:18:10.521057+0000 mon.a (mon.0) 3357 : audit [INF] from='client.? 172.21.15.100:0/1677144245' entity='client.admin' cmd='[{"prefix": "config-key rm", "key": "rbd/mirror/peer/91/69d4623e-37f2-4ccb-a5c4-6eb93f9ba0db"}]': finished 2024-04-07T17:18:11.455 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:18:11 smithi100 bash[74942]: audit 2024-04-07T17:18:10.545953+0000 mon.a (mon.0) 3358 : audit [DBG] from='client.? 172.21.15.100:0/1677144245' entity='client.admin' cmd=[{"prefix": "config-key get", "key": "rbd/mirror/peer/91/44e84396-d839-4080-bec3-076d7f9dae90"}]: dispatch 2024-04-07T17:18:11.456 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:18:11 smithi100 bash[74942]: audit 2024-04-07T17:18:10.556570+0000 mon.a (mon.0) 3359 : audit [INF] from='client.? 172.21.15.100:0/1677144245' entity='client.admin' 2024-04-07T17:18:11.456 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:18:11 smithi100 bash[74942]: audit 2024-04-07T17:18:10.557505+0000 mon.a (mon.0) 3360 : audit [DBG] from='client.? 172.21.15.100:0/1677144245' entity='client.admin' cmd=[{"prefix": "config-key get", "key": "rbd/mirror/peer/91/44e84396-d839-4080-bec3-076d7f9dae90"}]: dispatch 2024-04-07T17:18:11.456 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:18:11 smithi100 bash[74942]: audit 2024-04-07T17:18:10.558093+0000 mon.a (mon.0) 3361 : audit [INF] from='client.? 172.21.15.100:0/1677144245' entity='client.admin' cmd=[{"prefix": "config-key rm", "key": "rbd/mirror/peer/91/44e84396-d839-4080-bec3-076d7f9dae90"}]: dispatch 2024-04-07T17:18:11.456 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:18:11 smithi100 bash[74942]: audit 2024-04-07T17:18:10.578487+0000 mon.a (mon.0) 3362 : audit [INF] from='client.? 172.21.15.100:0/1677144245' entity='client.admin' cmd='[{"prefix": "config-key rm", "key": "rbd/mirror/peer/91/44e84396-d839-4080-bec3-076d7f9dae90"}]': finished 2024-04-07T17:18:11.456 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:18:11 smithi100 bash[76202]: cluster 2024-04-07T17:18:10.094705+0000 mon.a (mon.0) 3349 : cluster [DBG] osdmap e1749: 8 total, 8 up, 8 in 2024-04-07T17:18:11.456 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:18:11 smithi100 bash[76202]: audit 2024-04-07T17:18:10.451176+0000 mon.a (mon.0) 3350 : audit [INF] from='client.? 172.21.15.100:0/1677144245' entity='client.admin' cmd=[{"prefix": "config-key rm", "key": "rbd/mirror/peer/91/uuid4"}]: dispatch 2024-04-07T17:18:11.456 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:18:11 smithi100 bash[76202]: audit 2024-04-07T17:18:10.457832+0000 mon.a (mon.0) 3351 : audit [INF] from='client.? 172.21.15.100:0/1677144245' entity='client.admin' cmd='[{"prefix": "config-key rm", "key": "rbd/mirror/peer/91/uuid4"}]': finished 2024-04-07T17:18:11.456 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:18:11 smithi100 bash[76202]: audit 2024-04-07T17:18:10.463562+0000 mon.a (mon.0) 3352 : audit [INF] from='client.? 172.21.15.100:0/1677144245' entity='client.admin' cmd=[{"prefix": "config-key rm", "key": "rbd/mirror/peer/91/dee0f31f-3fe3-4ccb-865a-77df0fe586dc"}]: dispatch 2024-04-07T17:18:11.456 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:18:11 smithi100 bash[76202]: audit 2024-04-07T17:18:10.472280+0000 mon.a (mon.0) 3353 : audit [INF] from='client.? 172.21.15.100:0/1677144245' entity='client.admin' cmd='[{"prefix": "config-key rm", "key": "rbd/mirror/peer/91/dee0f31f-3fe3-4ccb-865a-77df0fe586dc"}]': finished 2024-04-07T17:18:11.456 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:18:11 smithi100 bash[76202]: audit 2024-04-07T17:18:10.493978+0000 mon.a (mon.0) 3354 : audit [INF] from='client.? 172.21.15.100:0/1677144245' entity='client.admin' cmd=[{"prefix": "config-key rm", "key": "rbd/mirror/peer/91/5962f148-835e-454c-a20a-2dfd6ed4277c"}]: dispatch 2024-04-07T17:18:11.456 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:18:11 smithi100 bash[76202]: audit 2024-04-07T17:18:10.501171+0000 mon.a (mon.0) 3355 : audit [INF] from='client.? 172.21.15.100:0/1677144245' entity='client.admin' cmd='[{"prefix": "config-key rm", "key": "rbd/mirror/peer/91/5962f148-835e-454c-a20a-2dfd6ed4277c"}]': finished 2024-04-07T17:18:11.456 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:18:11 smithi100 bash[76202]: audit 2024-04-07T17:18:10.512320+0000 mon.a (mon.0) 3356 : audit [INF] from='client.? 172.21.15.100:0/1677144245' entity='client.admin' cmd=[{"prefix": "config-key rm", "key": "rbd/mirror/peer/91/69d4623e-37f2-4ccb-a5c4-6eb93f9ba0db"}]: dispatch 2024-04-07T17:18:11.456 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:18:11 smithi100 bash[76202]: audit 2024-04-07T17:18:10.521057+0000 mon.a (mon.0) 3357 : audit [INF] from='client.? 172.21.15.100:0/1677144245' entity='client.admin' cmd='[{"prefix": "config-key rm", "key": "rbd/mirror/peer/91/69d4623e-37f2-4ccb-a5c4-6eb93f9ba0db"}]': finished 2024-04-07T17:18:11.457 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:18:11 smithi100 bash[76202]: audit 2024-04-07T17:18:10.545953+0000 mon.a (mon.0) 3358 : audit [DBG] from='client.? 172.21.15.100:0/1677144245' entity='client.admin' cmd=[{"prefix": "config-key get", "key": "rbd/mirror/peer/91/44e84396-d839-4080-bec3-076d7f9dae90"}]: dispatch 2024-04-07T17:18:11.457 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:18:11 smithi100 bash[76202]: audit 2024-04-07T17:18:10.556570+0000 mon.a (mon.0) 3359 : audit [INF] from='client.? 172.21.15.100:0/1677144245' entity='client.admin' 2024-04-07T17:18:11.457 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:18:11 smithi100 bash[76202]: audit 2024-04-07T17:18:10.557505+0000 mon.a (mon.0) 3360 : audit [DBG] from='client.? 172.21.15.100:0/1677144245' entity='client.admin' cmd=[{"prefix": "config-key get", "key": "rbd/mirror/peer/91/44e84396-d839-4080-bec3-076d7f9dae90"}]: dispatch 2024-04-07T17:18:11.457 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:18:11 smithi100 bash[76202]: audit 2024-04-07T17:18:10.558093+0000 mon.a (mon.0) 3361 : audit [INF] from='client.? 172.21.15.100:0/1677144245' entity='client.admin' cmd=[{"prefix": "config-key rm", "key": "rbd/mirror/peer/91/44e84396-d839-4080-bec3-076d7f9dae90"}]: dispatch 2024-04-07T17:18:11.457 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:18:11 smithi100 bash[76202]: audit 2024-04-07T17:18:10.578487+0000 mon.a (mon.0) 3362 : audit [INF] from='client.? 172.21.15.100:0/1677144245' entity='client.admin' cmd='[{"prefix": "config-key rm", "key": "rbd/mirror/peer/91/44e84396-d839-4080-bec3-076d7f9dae90"}]': finished 2024-04-07T17:18:12.592 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:18:12 smithi144 bash[95233]: ts=2024-04-07T17:18:12.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:18:12.867 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:18:12 smithi144 bash[66654]: cluster 2024-04-07T17:18:11.584961+0000 mon.a (mon.0) 3363 : cluster [DBG] osdmap e1750: 8 total, 8 up, 8 in 2024-04-07T17:18:12.868 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:18:12 smithi144 bash[66654]: cluster 2024-04-07T17:18:11.844835+0000 mgr.y (mgr.25175) 1885 : cluster [DBG] pgmap v3210: 649 pgs: 649 active+clean; 311 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 103 KiB/s rd, 549 KiB/s wr, 131 op/s 2024-04-07T17:18:12.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:18:12 smithi100 bash[74942]: cluster 2024-04-07T17:18:11.584961+0000 mon.a (mon.0) 3363 : cluster [DBG] osdmap e1750: 8 total, 8 up, 8 in 2024-04-07T17:18:12.955 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:18:12 smithi100 bash[74942]: cluster 2024-04-07T17:18:11.844835+0000 mgr.y (mgr.25175) 1885 : cluster [DBG] pgmap v3210: 649 pgs: 649 active+clean; 311 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 103 KiB/s rd, 549 KiB/s wr, 131 op/s 2024-04-07T17:18:12.955 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:18:12 smithi100 bash[76202]: cluster 2024-04-07T17:18:11.584961+0000 mon.a (mon.0) 3363 : cluster [DBG] osdmap e1750: 8 total, 8 up, 8 in 2024-04-07T17:18:12.955 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:18:12 smithi100 bash[76202]: cluster 2024-04-07T17:18:11.844835+0000 mgr.y (mgr.25175) 1885 : cluster [DBG] pgmap v3210: 649 pgs: 649 active+clean; 311 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 103 KiB/s rd, 549 KiB/s wr, 131 op/s 2024-04-07T17:18:13.867 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:18:13 smithi144 bash[66654]: cluster 2024-04-07T17:18:12.600093+0000 mon.a (mon.0) 3364 : cluster [DBG] osdmap e1751: 8 total, 8 up, 8 in 2024-04-07T17:18:13.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:18:13 smithi100 bash[74942]: cluster 2024-04-07T17:18:12.600093+0000 mon.a (mon.0) 3364 : cluster [DBG] osdmap e1751: 8 total, 8 up, 8 in 2024-04-07T17:18:13.955 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:18:13 smithi100 bash[76202]: cluster 2024-04-07T17:18:12.600093+0000 mon.a (mon.0) 3364 : cluster [DBG] osdmap e1751: 8 total, 8 up, 8 in 2024-04-07T17:18:14.867 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:18:14 smithi144 bash[66654]: cluster 2024-04-07T17:18:13.611535+0000 mon.a (mon.0) 3365 : cluster [DBG] osdmap e1752: 8 total, 8 up, 8 in 2024-04-07T17:18:14.868 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:18:14 smithi144 bash[66654]: cluster 2024-04-07T17:18:13.845973+0000 mgr.y (mgr.25175) 1886 : cluster [DBG] pgmap v3213: 649 pgs: 649 active+clean; 317 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 249 KiB/s rd, 1.7 MiB/s wr, 330 op/s 2024-04-07T17:18:14.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:18:14 smithi100 bash[76202]: cluster 2024-04-07T17:18:13.611535+0000 mon.a (mon.0) 3365 : cluster [DBG] osdmap e1752: 8 total, 8 up, 8 in 2024-04-07T17:18:14.955 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:18:14 smithi100 bash[76202]: cluster 2024-04-07T17:18:13.845973+0000 mgr.y (mgr.25175) 1886 : cluster [DBG] pgmap v3213: 649 pgs: 649 active+clean; 317 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 249 KiB/s rd, 1.7 MiB/s wr, 330 op/s 2024-04-07T17:18:14.955 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:18:14 smithi100 bash[74942]: cluster 2024-04-07T17:18:13.611535+0000 mon.a (mon.0) 3365 : cluster [DBG] osdmap e1752: 8 total, 8 up, 8 in 2024-04-07T17:18:14.955 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:18:14 smithi100 bash[74942]: cluster 2024-04-07T17:18:13.845973+0000 mgr.y (mgr.25175) 1886 : cluster [DBG] pgmap v3213: 649 pgs: 649 active+clean; 317 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 249 KiB/s rd, 1.7 MiB/s wr, 330 op/s 2024-04-07T17:18:15.710 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.CreateWithMirrorEnabled (5106 ms) 2024-04-07T17:18:15.710 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.FlushCacheWithCopyupOnExternalSnapshot 2024-04-07T17:18:15.710 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:18:15.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:18:15 smithi100 bash[74942]: cluster 2024-04-07T17:18:14.621462+0000 mon.a (mon.0) 3366 : cluster [DBG] osdmap e1753: 8 total, 8 up, 8 in 2024-04-07T17:18:15.955 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:18:15 smithi100 bash[76202]: cluster 2024-04-07T17:18:14.621462+0000 mon.a (mon.0) 3366 : cluster [DBG] osdmap e1753: 8 total, 8 up, 8 in 2024-04-07T17:18:16.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:18:15 smithi144 bash[66654]: cluster 2024-04-07T17:18:14.621462+0000 mon.a (mon.0) 3366 : cluster [DBG] osdmap e1753: 8 total, 8 up, 8 in 2024-04-07T17:18:16.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:18:16 smithi100 bash[74942]: cluster 2024-04-07T17:18:15.623714+0000 mon.a (mon.0) 3367 : cluster [DBG] osdmap e1754: 8 total, 8 up, 8 in 2024-04-07T17:18:16.955 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:18:16 smithi100 bash[74942]: cluster 2024-04-07T17:18:15.847136+0000 mgr.y (mgr.25175) 1887 : cluster [DBG] pgmap v3216: 649 pgs: 649 active+clean; 317 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 278 KiB/s rd, 1.5 MiB/s wr, 367 op/s 2024-04-07T17:18:16.955 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:18:16 smithi100 bash[76202]: cluster 2024-04-07T17:18:15.623714+0000 mon.a (mon.0) 3367 : cluster [DBG] osdmap e1754: 8 total, 8 up, 8 in 2024-04-07T17:18:16.955 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:18:16 smithi100 bash[76202]: cluster 2024-04-07T17:18:15.847136+0000 mgr.y (mgr.25175) 1887 : cluster [DBG] pgmap v3216: 649 pgs: 649 active+clean; 317 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 278 KiB/s rd, 1.5 MiB/s wr, 367 op/s 2024-04-07T17:18:17.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:18:16 smithi144 bash[66654]: cluster 2024-04-07T17:18:15.623714+0000 mon.a (mon.0) 3367 : cluster [DBG] osdmap e1754: 8 total, 8 up, 8 in 2024-04-07T17:18:17.118 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:18:16 smithi144 bash[66654]: cluster 2024-04-07T17:18:15.847136+0000 mgr.y (mgr.25175) 1887 : cluster [DBG] pgmap v3216: 649 pgs: 649 active+clean; 317 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 278 KiB/s rd, 1.5 MiB/s wr, 367 op/s 2024-04-07T17:18:17.715 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.FlushCacheWithCopyupOnExternalSnapshot (2006 ms) 2024-04-07T17:18:17.716 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.ExclusiveLock 2024-04-07T17:18:17.716 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:18:17.849 INFO:tasks.workunit.client.0.smithi100.stdout:1: acquiring exclusive lock 2024-04-07T17:18:17.849 INFO:tasks.workunit.client.0.smithi100.stdout:2: acquiring exclusive lock 2024-04-07T17:18:17.859 INFO:tasks.workunit.client.0.smithi100.stdout:2: exclusive lock acquired 2024-04-07T17:18:17.895 INFO:tasks.workunit.client.0.smithi100.stdout:2: releasing exclusive lock 2024-04-07T17:18:17.902 INFO:tasks.workunit.client.0.smithi100.stdout:2: exclusive lock released 2024-04-07T17:18:17.903 INFO:tasks.workunit.client.0.smithi100.stdout:2: acquiring exclusive lock 2024-04-07T17:18:17.912 INFO:tasks.workunit.client.0.smithi100.stdout:2: exclusive lock acquired 2024-04-07T17:18:17.914 INFO:tasks.workunit.client.0.smithi100.stdout:2: releasing exclusive lock 2024-04-07T17:18:17.919 INFO:tasks.workunit.client.0.smithi100.stdout:2: exclusive lock released 2024-04-07T17:18:17.920 INFO:tasks.workunit.client.0.smithi100.stdout:2: acquiring exclusive lock 2024-04-07T17:18:17.928 INFO:tasks.workunit.client.0.smithi100.stdout:2: exclusive lock acquired 2024-04-07T17:18:17.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:18:17 smithi100 bash[74942]: cluster 2024-04-07T17:18:16.651257+0000 mon.a (mon.0) 3368 : cluster [DBG] osdmap e1755: 8 total, 8 up, 8 in 2024-04-07T17:18:17.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:18:17 smithi100 bash[74942]: audit 2024-04-07T17:18:16.829916+0000 mon.a (mon.0) 3369 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:18:17.955 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:18:17 smithi100 bash[76202]: cluster 2024-04-07T17:18:16.651257+0000 mon.a (mon.0) 3368 : cluster [DBG] osdmap e1755: 8 total, 8 up, 8 in 2024-04-07T17:18:17.955 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:18:17 smithi100 bash[76202]: audit 2024-04-07T17:18:16.829916+0000 mon.a (mon.0) 3369 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:18:17.957 INFO:tasks.workunit.client.0.smithi100.stdout:2: releasing exclusive lock 2024-04-07T17:18:17.962 INFO:tasks.workunit.client.0.smithi100.stdout:2: exclusive lock released 2024-04-07T17:18:17.962 INFO:tasks.workunit.client.0.smithi100.stdout:2: acquiring exclusive lock 2024-04-07T17:18:17.969 INFO:tasks.workunit.client.0.smithi100.stdout:1: exclusive lock acquired 2024-04-07T17:18:18.010 INFO:tasks.workunit.client.0.smithi100.stdout:1: releasing exclusive lock 2024-04-07T17:18:18.018 INFO:tasks.workunit.client.0.smithi100.stdout:1: exclusive lock released 2024-04-07T17:18:18.018 INFO:tasks.workunit.client.0.smithi100.stdout:1: acquiring exclusive lock 2024-04-07T17:18:18.024 INFO:tasks.workunit.client.0.smithi100.stdout:2: exclusive lock acquired 2024-04-07T17:18:18.055 INFO:tasks.workunit.client.0.smithi100.stdout:2: releasing exclusive lock 2024-04-07T17:18:18.060 INFO:tasks.workunit.client.0.smithi100.stdout:2: exclusive lock released 2024-04-07T17:18:18.060 INFO:tasks.workunit.client.0.smithi100.stdout:2: acquiring exclusive lock 2024-04-07T17:18:18.069 INFO:tasks.workunit.client.0.smithi100.stdout:1: exclusive lock acquired 2024-04-07T17:18:18.105 INFO:tasks.workunit.client.0.smithi100.stdout:1: releasing exclusive lock 2024-04-07T17:18:18.111 INFO:tasks.workunit.client.0.smithi100.stdout:1: exclusive lock released 2024-04-07T17:18:18.111 INFO:tasks.workunit.client.0.smithi100.stdout:1: acquiring exclusive lock 2024-04-07T17:18:18.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:18:17 smithi144 bash[66654]: cluster 2024-04-07T17:18:16.651257+0000 mon.a (mon.0) 3368 : cluster [DBG] osdmap e1755: 8 total, 8 up, 8 in 2024-04-07T17:18:18.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:18:17 smithi144 bash[66654]: audit 2024-04-07T17:18:16.829916+0000 mon.a (mon.0) 3369 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:18:18.119 INFO:tasks.workunit.client.0.smithi100.stdout:1: exclusive lock acquired 2024-04-07T17:18:18.132 INFO:tasks.workunit.client.0.smithi100.stdout:1: releasing exclusive lock 2024-04-07T17:18:18.137 INFO:tasks.workunit.client.0.smithi100.stdout:1: exclusive lock released 2024-04-07T17:18:18.137 INFO:tasks.workunit.client.0.smithi100.stdout:1: acquiring exclusive lock 2024-04-07T17:18:18.145 INFO:tasks.workunit.client.0.smithi100.stdout:2: exclusive lock acquired 2024-04-07T17:18:18.188 INFO:tasks.workunit.client.0.smithi100.stdout:2: releasing exclusive lock 2024-04-07T17:18:18.194 INFO:tasks.workunit.client.0.smithi100.stdout:2: exclusive lock released 2024-04-07T17:18:18.202 INFO:tasks.workunit.client.0.smithi100.stdout:1: exclusive lock acquired 2024-04-07T17:18:18.246 INFO:tasks.workunit.client.0.smithi100.stdout:1: releasing exclusive lock 2024-04-07T17:18:18.253 INFO:tasks.workunit.client.0.smithi100.stdout:1: exclusive lock released 2024-04-07T17:18:18.253 INFO:tasks.workunit.client.0.smithi100.stdout:1: acquiring exclusive lock 2024-04-07T17:18:18.264 INFO:tasks.workunit.client.0.smithi100.stdout:1: exclusive lock acquired 2024-04-07T17:18:18.264 INFO:tasks.workunit.client.0.smithi100.stdout:1: releasing exclusive lock 2024-04-07T17:18:18.271 INFO:tasks.workunit.client.0.smithi100.stdout:1: exclusive lock released 2024-04-07T17:18:18.312 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.ExclusiveLock (597 ms) 2024-04-07T17:18:18.312 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.BreakLock 2024-04-07T17:18:18.351 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:18:18.681 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:18:18 smithi100 bash[74942]: cluster 2024-04-07T17:18:17.661325+0000 mon.a (mon.0) 3370 : cluster [DBG] osdmap e1756: 8 total, 8 up, 8 in 2024-04-07T17:18:18.681 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:18:18 smithi100 bash[74942]: cluster 2024-04-07T17:18:17.848434+0000 mgr.y (mgr.25175) 1888 : cluster [DBG] pgmap v3219: 649 pgs: 649 active+clean; 318 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 144 KiB/s rd, 75 KiB/s wr, 189 op/s 2024-04-07T17:18:18.681 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:18:18 smithi100 bash[74942]: audit 2024-04-07T17:18:18.417421+0000 mon.c (mon.2) 87 : audit [INF] from='client.? 172.21.15.100:0/3213618994' entity='client.admin' cmd=[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.100:0/1091704426"}]: dispatch 2024-04-07T17:18:18.681 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:18:18 smithi100 bash[74942]: audit 2024-04-07T17:18:18.417820+0000 mon.a (mon.0) 3371 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.100:0/1091704426"}]: dispatch 2024-04-07T17:18:18.682 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:18:18 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:18:18] "GET /metrics HTTP/1.1" 200 75602 "" "Prometheus/2.43.0" 2024-04-07T17:18:18.776 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.BreakLock (463 ms) 2024-04-07T17:18:18.776 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.DiscardAfterWrite 2024-04-07T17:18:18.776 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:18:18.851 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.DiscardAfterWrite (76 ms) 2024-04-07T17:18:18.852 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.DefaultFeatures 2024-04-07T17:18:18.852 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.DefaultFeatures (0 ms) 2024-04-07T17:18:18.852 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.TestTrashMoveAndPurge 2024-04-07T17:18:18.852 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:18:18.867 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 17:18:18 smithi144 bash[98866]: logger=cleanup t=2024-04-07T17:18:18.475962965Z level=info msg="Completed cleanup jobs" duration=56.603236ms 2024-04-07T17:18:18.868 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:18:18 smithi144 bash[66654]: cluster 2024-04-07T17:18:17.661325+0000 mon.a (mon.0) 3370 : cluster [DBG] osdmap e1756: 8 total, 8 up, 8 in 2024-04-07T17:18:18.868 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:18:18 smithi144 bash[66654]: cluster 2024-04-07T17:18:17.848434+0000 mgr.y (mgr.25175) 1888 : cluster [DBG] pgmap v3219: 649 pgs: 649 active+clean; 318 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 144 KiB/s rd, 75 KiB/s wr, 189 op/s 2024-04-07T17:18:18.868 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:18:18 smithi144 bash[66654]: audit 2024-04-07T17:18:18.417421+0000 mon.c (mon.2) 87 : audit [INF] from='client.? 172.21.15.100:0/3213618994' entity='client.admin' cmd=[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.100:0/1091704426"}]: dispatch 2024-04-07T17:18:18.868 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:18:18 smithi144 bash[66654]: audit 2024-04-07T17:18:18.417820+0000 mon.a (mon.0) 3371 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.100:0/1091704426"}]: dispatch 2024-04-07T17:18:18.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:18:18 smithi100 bash[76202]: cluster 2024-04-07T17:18:17.661325+0000 mon.a (mon.0) 3370 : cluster [DBG] osdmap e1756: 8 total, 8 up, 8 in 2024-04-07T17:18:18.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:18:18 smithi100 bash[76202]: cluster 2024-04-07T17:18:17.848434+0000 mgr.y (mgr.25175) 1888 : cluster [DBG] pgmap v3219: 649 pgs: 649 active+clean; 318 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 144 KiB/s rd, 75 KiB/s wr, 189 op/s 2024-04-07T17:18:18.955 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:18:18 smithi100 bash[76202]: audit 2024-04-07T17:18:18.417421+0000 mon.c (mon.2) 87 : audit [INF] from='client.? 172.21.15.100:0/3213618994' entity='client.admin' cmd=[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.100:0/1091704426"}]: dispatch 2024-04-07T17:18:18.955 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:18:18 smithi100 bash[76202]: audit 2024-04-07T17:18:18.417820+0000 mon.a (mon.0) 3371 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.100:0/1091704426"}]: dispatch 2024-04-07T17:18:19.009 INFO:tasks.workunit.client.0.smithi100.stdout:0.00% done 2024-04-07T17:18:19.009 INFO:tasks.workunit.client.0.smithi100.stdout:1.56% done 2024-04-07T17:18:19.010 INFO:tasks.workunit.client.0.smithi100.stdout:3.12% done 2024-04-07T17:18:19.010 INFO:tasks.workunit.client.0.smithi100.stdout:4.69% done 2024-04-07T17:18:19.010 INFO:tasks.workunit.client.0.smithi100.stdout:6.25% done 2024-04-07T17:18:19.010 INFO:tasks.workunit.client.0.smithi100.stdout:7.81% done 2024-04-07T17:18:19.010 INFO:tasks.workunit.client.0.smithi100.stdout:9.38% done 2024-04-07T17:18:19.010 INFO:tasks.workunit.client.0.smithi100.stdout:10.94% done 2024-04-07T17:18:19.010 INFO:tasks.workunit.client.0.smithi100.stdout:12.50% done 2024-04-07T17:18:19.010 INFO:tasks.workunit.client.0.smithi100.stdout:14.06% done 2024-04-07T17:18:19.010 INFO:tasks.workunit.client.0.smithi100.stdout:15.62% done 2024-04-07T17:18:19.010 INFO:tasks.workunit.client.0.smithi100.stdout:17.19% done 2024-04-07T17:18:19.010 INFO:tasks.workunit.client.0.smithi100.stdout:18.75% done 2024-04-07T17:18:19.010 INFO:tasks.workunit.client.0.smithi100.stdout:20.31% done 2024-04-07T17:18:19.010 INFO:tasks.workunit.client.0.smithi100.stdout:21.88% done 2024-04-07T17:18:19.010 INFO:tasks.workunit.client.0.smithi100.stdout:23.44% done 2024-04-07T17:18:19.011 INFO:tasks.workunit.client.0.smithi100.stdout:25.00% done 2024-04-07T17:18:19.011 INFO:tasks.workunit.client.0.smithi100.stdout:26.56% done 2024-04-07T17:18:19.011 INFO:tasks.workunit.client.0.smithi100.stdout:28.12% done 2024-04-07T17:18:19.011 INFO:tasks.workunit.client.0.smithi100.stdout:29.69% done 2024-04-07T17:18:19.011 INFO:tasks.workunit.client.0.smithi100.stdout:31.25% done 2024-04-07T17:18:19.011 INFO:tasks.workunit.client.0.smithi100.stdout:32.81% done 2024-04-07T17:18:19.011 INFO:tasks.workunit.client.0.smithi100.stdout:34.38% done 2024-04-07T17:18:19.011 INFO:tasks.workunit.client.0.smithi100.stdout:35.94% done 2024-04-07T17:18:19.011 INFO:tasks.workunit.client.0.smithi100.stdout:37.50% done 2024-04-07T17:18:19.011 INFO:tasks.workunit.client.0.smithi100.stdout:39.06% done 2024-04-07T17:18:19.011 INFO:tasks.workunit.client.0.smithi100.stdout:40.62% done 2024-04-07T17:18:19.011 INFO:tasks.workunit.client.0.smithi100.stdout:42.19% done 2024-04-07T17:18:19.011 INFO:tasks.workunit.client.0.smithi100.stdout:43.75% done 2024-04-07T17:18:19.011 INFO:tasks.workunit.client.0.smithi100.stdout:45.31% done 2024-04-07T17:18:19.012 INFO:tasks.workunit.client.0.smithi100.stdout:46.88% done 2024-04-07T17:18:19.012 INFO:tasks.workunit.client.0.smithi100.stdout:48.44% done 2024-04-07T17:18:19.012 INFO:tasks.workunit.client.0.smithi100.stdout:50.00% done 2024-04-07T17:18:19.012 INFO:tasks.workunit.client.0.smithi100.stdout:51.56% done 2024-04-07T17:18:19.012 INFO:tasks.workunit.client.0.smithi100.stdout:53.12% done 2024-04-07T17:18:19.012 INFO:tasks.workunit.client.0.smithi100.stdout:54.69% done 2024-04-07T17:18:19.012 INFO:tasks.workunit.client.0.smithi100.stdout:56.25% done 2024-04-07T17:18:19.012 INFO:tasks.workunit.client.0.smithi100.stdout:57.81% done 2024-04-07T17:18:19.012 INFO:tasks.workunit.client.0.smithi100.stdout:59.38% done 2024-04-07T17:18:19.012 INFO:tasks.workunit.client.0.smithi100.stdout:60.94% done 2024-04-07T17:18:19.012 INFO:tasks.workunit.client.0.smithi100.stdout:62.50% done 2024-04-07T17:18:19.012 INFO:tasks.workunit.client.0.smithi100.stdout:64.06% done 2024-04-07T17:18:19.012 INFO:tasks.workunit.client.0.smithi100.stdout:65.62% done 2024-04-07T17:18:19.012 INFO:tasks.workunit.client.0.smithi100.stdout:67.19% done 2024-04-07T17:18:19.012 INFO:tasks.workunit.client.0.smithi100.stdout:68.75% done 2024-04-07T17:18:19.012 INFO:tasks.workunit.client.0.smithi100.stdout:70.31% done 2024-04-07T17:18:19.013 INFO:tasks.workunit.client.0.smithi100.stdout:71.88% done 2024-04-07T17:18:19.013 INFO:tasks.workunit.client.0.smithi100.stdout:73.44% done 2024-04-07T17:18:19.013 INFO:tasks.workunit.client.0.smithi100.stdout:75.00% done 2024-04-07T17:18:19.013 INFO:tasks.workunit.client.0.smithi100.stdout:76.56% done 2024-04-07T17:18:19.013 INFO:tasks.workunit.client.0.smithi100.stdout:78.12% done 2024-04-07T17:18:19.013 INFO:tasks.workunit.client.0.smithi100.stdout:79.69% done 2024-04-07T17:18:19.013 INFO:tasks.workunit.client.0.smithi100.stdout:81.25% done 2024-04-07T17:18:19.013 INFO:tasks.workunit.client.0.smithi100.stdout:82.81% done 2024-04-07T17:18:19.013 INFO:tasks.workunit.client.0.smithi100.stdout:84.38% done 2024-04-07T17:18:19.013 INFO:tasks.workunit.client.0.smithi100.stdout:85.94% done 2024-04-07T17:18:19.013 INFO:tasks.workunit.client.0.smithi100.stdout:87.50% done 2024-04-07T17:18:19.013 INFO:tasks.workunit.client.0.smithi100.stdout:89.06% done 2024-04-07T17:18:19.013 INFO:tasks.workunit.client.0.smithi100.stdout:90.62% done 2024-04-07T17:18:19.013 INFO:tasks.workunit.client.0.smithi100.stdout:92.19% done 2024-04-07T17:18:19.013 INFO:tasks.workunit.client.0.smithi100.stdout:93.75% done 2024-04-07T17:18:19.013 INFO:tasks.workunit.client.0.smithi100.stdout:95.31% done 2024-04-07T17:18:19.013 INFO:tasks.workunit.client.0.smithi100.stdout:96.88% done 2024-04-07T17:18:19.014 INFO:tasks.workunit.client.0.smithi100.stdout:98.44% done 2024-04-07T17:18:19.014 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.TestTrashMoveAndPurge (158 ms) 2024-04-07T17:18:19.014 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.TestTrashMoveAndPurgeNonExpiredDelay 2024-04-07T17:18:19.014 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:18:19.172 INFO:tasks.workunit.client.0.smithi100.stdout:0.00% done 2024-04-07T17:18:19.172 INFO:tasks.workunit.client.0.smithi100.stdout:1.56% done 2024-04-07T17:18:19.173 INFO:tasks.workunit.client.0.smithi100.stdout:3.12% done 2024-04-07T17:18:19.173 INFO:tasks.workunit.client.0.smithi100.stdout:4.69% done 2024-04-07T17:18:19.173 INFO:tasks.workunit.client.0.smithi100.stdout:6.25% done 2024-04-07T17:18:19.173 INFO:tasks.workunit.client.0.smithi100.stdout:7.81% done 2024-04-07T17:18:19.173 INFO:tasks.workunit.client.0.smithi100.stdout:9.38% done 2024-04-07T17:18:19.173 INFO:tasks.workunit.client.0.smithi100.stdout:10.94% done 2024-04-07T17:18:19.173 INFO:tasks.workunit.client.0.smithi100.stdout:12.50% done 2024-04-07T17:18:19.173 INFO:tasks.workunit.client.0.smithi100.stdout:14.06% done 2024-04-07T17:18:19.173 INFO:tasks.workunit.client.0.smithi100.stdout:15.62% done 2024-04-07T17:18:19.173 INFO:tasks.workunit.client.0.smithi100.stdout:17.19% done 2024-04-07T17:18:19.173 INFO:tasks.workunit.client.0.smithi100.stdout:18.75% done 2024-04-07T17:18:19.174 INFO:tasks.workunit.client.0.smithi100.stdout:20.31% done 2024-04-07T17:18:19.174 INFO:tasks.workunit.client.0.smithi100.stdout:21.88% done 2024-04-07T17:18:19.174 INFO:tasks.workunit.client.0.smithi100.stdout:23.44% done 2024-04-07T17:18:19.174 INFO:tasks.workunit.client.0.smithi100.stdout:25.00% done 2024-04-07T17:18:19.174 INFO:tasks.workunit.client.0.smithi100.stdout:26.56% done 2024-04-07T17:18:19.174 INFO:tasks.workunit.client.0.smithi100.stdout:28.12% done 2024-04-07T17:18:19.174 INFO:tasks.workunit.client.0.smithi100.stdout:29.69% done 2024-04-07T17:18:19.174 INFO:tasks.workunit.client.0.smithi100.stdout:31.25% done 2024-04-07T17:18:19.174 INFO:tasks.workunit.client.0.smithi100.stdout:32.81% done 2024-04-07T17:18:19.174 INFO:tasks.workunit.client.0.smithi100.stdout:34.38% done 2024-04-07T17:18:19.174 INFO:tasks.workunit.client.0.smithi100.stdout:35.94% done 2024-04-07T17:18:19.174 INFO:tasks.workunit.client.0.smithi100.stdout:37.50% done 2024-04-07T17:18:19.174 INFO:tasks.workunit.client.0.smithi100.stdout:39.06% done 2024-04-07T17:18:19.174 INFO:tasks.workunit.client.0.smithi100.stdout:40.62% done 2024-04-07T17:18:19.175 INFO:tasks.workunit.client.0.smithi100.stdout:42.19% done 2024-04-07T17:18:19.175 INFO:tasks.workunit.client.0.smithi100.stdout:43.75% done 2024-04-07T17:18:19.175 INFO:tasks.workunit.client.0.smithi100.stdout:45.31% done 2024-04-07T17:18:19.175 INFO:tasks.workunit.client.0.smithi100.stdout:46.88% done 2024-04-07T17:18:19.175 INFO:tasks.workunit.client.0.smithi100.stdout:48.44% done 2024-04-07T17:18:19.175 INFO:tasks.workunit.client.0.smithi100.stdout:50.00% done 2024-04-07T17:18:19.175 INFO:tasks.workunit.client.0.smithi100.stdout:51.56% done 2024-04-07T17:18:19.175 INFO:tasks.workunit.client.0.smithi100.stdout:53.12% done 2024-04-07T17:18:19.175 INFO:tasks.workunit.client.0.smithi100.stdout:54.69% done 2024-04-07T17:18:19.175 INFO:tasks.workunit.client.0.smithi100.stdout:56.25% done 2024-04-07T17:18:19.175 INFO:tasks.workunit.client.0.smithi100.stdout:57.81% done 2024-04-07T17:18:19.175 INFO:tasks.workunit.client.0.smithi100.stdout:59.38% done 2024-04-07T17:18:19.175 INFO:tasks.workunit.client.0.smithi100.stdout:60.94% done 2024-04-07T17:18:19.175 INFO:tasks.workunit.client.0.smithi100.stdout:62.50% done 2024-04-07T17:18:19.175 INFO:tasks.workunit.client.0.smithi100.stdout:64.06% done 2024-04-07T17:18:19.176 INFO:tasks.workunit.client.0.smithi100.stdout:65.62% done 2024-04-07T17:18:19.176 INFO:tasks.workunit.client.0.smithi100.stdout:67.19% done 2024-04-07T17:18:19.176 INFO:tasks.workunit.client.0.smithi100.stdout:68.75% done 2024-04-07T17:18:19.176 INFO:tasks.workunit.client.0.smithi100.stdout:70.31% done 2024-04-07T17:18:19.176 INFO:tasks.workunit.client.0.smithi100.stdout:71.88% done 2024-04-07T17:18:19.176 INFO:tasks.workunit.client.0.smithi100.stdout:73.44% done 2024-04-07T17:18:19.176 INFO:tasks.workunit.client.0.smithi100.stdout:75.00% done 2024-04-07T17:18:19.176 INFO:tasks.workunit.client.0.smithi100.stdout:76.56% done 2024-04-07T17:18:19.176 INFO:tasks.workunit.client.0.smithi100.stdout:78.12% done 2024-04-07T17:18:19.176 INFO:tasks.workunit.client.0.smithi100.stdout:79.69% done 2024-04-07T17:18:19.176 INFO:tasks.workunit.client.0.smithi100.stdout:81.25% done 2024-04-07T17:18:19.176 INFO:tasks.workunit.client.0.smithi100.stdout:82.81% done 2024-04-07T17:18:19.176 INFO:tasks.workunit.client.0.smithi100.stdout:84.38% done 2024-04-07T17:18:19.177 INFO:tasks.workunit.client.0.smithi100.stdout:85.94% done 2024-04-07T17:18:19.177 INFO:tasks.workunit.client.0.smithi100.stdout:87.50% done 2024-04-07T17:18:19.177 INFO:tasks.workunit.client.0.smithi100.stdout:89.06% done 2024-04-07T17:18:19.177 INFO:tasks.workunit.client.0.smithi100.stdout:90.62% done 2024-04-07T17:18:19.177 INFO:tasks.workunit.client.0.smithi100.stdout:92.19% done 2024-04-07T17:18:19.177 INFO:tasks.workunit.client.0.smithi100.stdout:93.75% done 2024-04-07T17:18:19.177 INFO:tasks.workunit.client.0.smithi100.stdout:95.31% done 2024-04-07T17:18:19.177 INFO:tasks.workunit.client.0.smithi100.stdout:96.88% done 2024-04-07T17:18:19.177 INFO:tasks.workunit.client.0.smithi100.stdout:98.44% done 2024-04-07T17:18:19.177 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.TestTrashMoveAndPurgeNonExpiredDelay (163 ms) 2024-04-07T17:18:19.177 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.TestTrashPurge 2024-04-07T17:18:19.177 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:18:19.193 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:18:19.519 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.TestTrashPurge (347 ms) 2024-04-07T17:18:19.519 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.TestTrashMoveAndRestore 2024-04-07T17:18:19.519 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:18:19.647 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.TestTrashMoveAndRestore (128 ms) 2024-04-07T17:18:19.648 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.TestListWatchers 2024-04-07T17:18:19.648 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:18:19.736 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.TestListWatchers (88 ms) 2024-04-07T17:18:19.736 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.TestSetSnapById 2024-04-07T17:18:19.736 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:18:19.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:18:19 smithi100 bash[74942]: audit 2024-04-07T17:18:18.655818+0000 mon.a (mon.0) 3372 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.100:0/1091704426"}]': finished 2024-04-07T17:18:19.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:18:19 smithi100 bash[74942]: cluster 2024-04-07T17:18:18.663384+0000 mon.a (mon.0) 3373 : cluster [DBG] osdmap e1757: 8 total, 8 up, 8 in 2024-04-07T17:18:19.955 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:18:19 smithi100 bash[76202]: audit 2024-04-07T17:18:18.655818+0000 mon.a (mon.0) 3372 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.100:0/1091704426"}]': finished 2024-04-07T17:18:19.955 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:18:19 smithi100 bash[76202]: cluster 2024-04-07T17:18:18.663384+0000 mon.a (mon.0) 3373 : cluster [DBG] osdmap e1757: 8 total, 8 up, 8 in 2024-04-07T17:18:20.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:18:19 smithi144 bash[66654]: audit 2024-04-07T17:18:18.655818+0000 mon.a (mon.0) 3372 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.100:0/1091704426"}]': finished 2024-04-07T17:18:20.118 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:18:19 smithi144 bash[66654]: cluster 2024-04-07T17:18:18.663384+0000 mon.a (mon.0) 3373 : cluster [DBG] osdmap e1757: 8 total, 8 up, 8 in 2024-04-07T17:18:20.781 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.TestSetSnapById (1045 ms) 2024-04-07T17:18:20.781 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.Namespaces 2024-04-07T17:18:20.845 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.Namespaces (65 ms) 2024-04-07T17:18:20.845 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.NamespacesPP 2024-04-07T17:18:20.913 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:18:20.925 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:18:20.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:18:20 smithi100 bash[74942]: cluster 2024-04-07T17:18:19.850879+0000 mgr.y (mgr.25175) 1889 : cluster [DBG] pgmap v3221: 649 pgs: 649 active+clean; 318 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 220 KiB/s rd, 133 KiB/s wr, 211 op/s 2024-04-07T17:18:20.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:18:20 smithi100 bash[76202]: cluster 2024-04-07T17:18:19.850879+0000 mgr.y (mgr.25175) 1889 : cluster [DBG] pgmap v3221: 649 pgs: 649 active+clean; 318 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 220 KiB/s rd, 133 KiB/s wr, 211 op/s 2024-04-07T17:18:21.118 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:18:20 smithi144 bash[66654]: cluster 2024-04-07T17:18:19.850879+0000 mgr.y (mgr.25175) 1889 : cluster [DBG] pgmap v3221: 649 pgs: 649 active+clean; 318 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 220 KiB/s rd, 133 KiB/s wr, 211 op/s 2024-04-07T17:18:21.147 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.NamespacesPP (302 ms) 2024-04-07T17:18:21.147 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.Migration 2024-04-07T17:18:21.147 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:18:21.147 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:18:21.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:18:21 smithi100 bash[74942]: cluster 2024-04-07T17:18:20.692979+0000 mon.a (mon.0) 3374 : cluster [DBG] osdmap e1758: 8 total, 8 up, 8 in 2024-04-07T17:18:21.955 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:18:21 smithi100 bash[76202]: cluster 2024-04-07T17:18:20.692979+0000 mon.a (mon.0) 3374 : cluster [DBG] osdmap e1758: 8 total, 8 up, 8 in 2024-04-07T17:18:22.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:18:21 smithi144 bash[66654]: cluster 2024-04-07T17:18:20.692979+0000 mon.a (mon.0) 3374 : cluster [DBG] osdmap e1758: 8 total, 8 up, 8 in 2024-04-07T17:18:22.334 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.Migration (1186 ms) 2024-04-07T17:18:22.334 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.MigrationPP 2024-04-07T17:18:22.334 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:18:22.334 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:18:22.617 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:18:22 smithi144 bash[95233]: ts=2024-04-07T17:18:22.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:18:22.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:18:22 smithi100 bash[74942]: cluster 2024-04-07T17:18:21.852138+0000 mgr.y (mgr.25175) 1890 : cluster [DBG] pgmap v3223: 649 pgs: 649 active+clean; 318 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 183 KiB/s rd, 110 KiB/s wr, 181 op/s 2024-04-07T17:18:22.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:18:22 smithi100 bash[76202]: cluster 2024-04-07T17:18:21.852138+0000 mgr.y (mgr.25175) 1890 : cluster [DBG] pgmap v3223: 649 pgs: 649 active+clean; 318 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 183 KiB/s rd, 110 KiB/s wr, 181 op/s 2024-04-07T17:18:23.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:18:22 smithi144 bash[66654]: cluster 2024-04-07T17:18:21.852138+0000 mgr.y (mgr.25175) 1890 : cluster [DBG] pgmap v3223: 649 pgs: 649 active+clean; 318 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 183 KiB/s rd, 110 KiB/s wr, 181 op/s 2024-04-07T17:18:23.602 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.MigrationPP (1269 ms) 2024-04-07T17:18:23.603 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.TestGetAccessTimestamp 2024-04-07T17:18:23.603 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:18:23.651 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.TestGetAccessTimestamp (49 ms) 2024-04-07T17:18:23.652 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.TestGetModifyTimestamp 2024-04-07T17:18:23.652 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:18:23.701 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.TestGetModifyTimestamp (49 ms) 2024-04-07T17:18:23.701 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.ZeroOverlapFlatten 2024-04-07T17:18:23.701 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:18:24.123 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.ZeroOverlapFlatten (423 ms) 2024-04-07T17:18:24.123 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.PoolMetadata 2024-04-07T17:18:24.162 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:18:23 smithi100 bash[74942]: cluster 2024-04-07T17:18:22.899949+0000 mon.a (mon.0) 3375 : cluster [DBG] osdmap e1759: 8 total, 8 up, 8 in 2024-04-07T17:18:24.162 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:18:23 smithi100 bash[76202]: cluster 2024-04-07T17:18:22.899949+0000 mon.a (mon.0) 3375 : cluster [DBG] osdmap e1759: 8 total, 8 up, 8 in 2024-04-07T17:18:24.178 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.PoolMetadata (55 ms) 2024-04-07T17:18:24.178 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.PoolMetadataPP 2024-04-07T17:18:24.190 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:18:24 smithi100 bash[68137]: [07/Apr/2024:17:18:24] ENGINE Bus STOPPING 2024-04-07T17:18:24.206 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.PoolMetadataPP (28 ms) 2024-04-07T17:18:24.206 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.Config 2024-04-07T17:18:24.212 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:18:24.269 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.Config (63 ms) 2024-04-07T17:18:24.270 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.ConfigPP 2024-04-07T17:18:24.275 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:18:24.346 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.ConfigPP (76 ms) 2024-04-07T17:18:24.346 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.PoolStatsPP 2024-04-07T17:18:24.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:18:23 smithi144 bash[66654]: cluster 2024-04-07T17:18:22.899949+0000 mon.a (mon.0) 3375 : cluster [DBG] osdmap e1759: 8 total, 8 up, 8 in 2024-04-07T17:18:24.454 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:18:24 smithi100 bash[68137]: [07/Apr/2024:17:18:24] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-04-07T17:18:24.455 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:18:24 smithi100 bash[68137]: [07/Apr/2024:17:18:24] ENGINE Bus STOPPED 2024-04-07T17:18:24.455 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:18:24 smithi100 bash[68137]: [07/Apr/2024:17:18:24] ENGINE Bus STARTING 2024-04-07T17:18:24.455 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:18:24 smithi100 bash[68137]: [07/Apr/2024:17:18:24] ENGINE Serving on http://:::9283 2024-04-07T17:18:24.455 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:18:24 smithi100 bash[68137]: [07/Apr/2024:17:18:24] ENGINE Bus STARTED 2024-04-07T17:18:25.206 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:18:24 smithi100 bash[74942]: cluster 2024-04-07T17:18:23.854176+0000 mgr.y (mgr.25175) 1891 : cluster [DBG] pgmap v3225: 649 pgs: 649 active+clean; 318 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 293 KiB/s rd, 60 KiB/s wr, 358 op/s 2024-04-07T17:18:25.206 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:18:24 smithi100 bash[74942]: cluster 2024-04-07T17:18:23.914617+0000 mon.a (mon.0) 3376 : cluster [DBG] osdmap e1760: 8 total, 8 up, 8 in 2024-04-07T17:18:25.206 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:18:24 smithi100 bash[74942]: audit 2024-04-07T17:18:24.145570+0000 mon.a (mon.0) 3377 : audit [INF] from='client.? ' entity='client.admin' 2024-04-07T17:18:25.206 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:18:24 smithi100 bash[74942]: audit 2024-04-07T17:18:24.188691+0000 mon.a (mon.0) 3378 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:18:25.206 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:18:24 smithi100 bash[74942]: audit 2024-04-07T17:18:24.200306+0000 mon.a (mon.0) 3379 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config-key dump", "format": "json"}]: dispatch 2024-04-07T17:18:25.206 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:18:24 smithi100 bash[74942]: audit 2024-04-07T17:18:24.271474+0000 mon.a (mon.0) 3380 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:18:25.206 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:18:24 smithi100 bash[74942]: audit 2024-04-07T17:18:24.319655+0000 mon.a (mon.0) 3381 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:18:25.207 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:18:24 smithi100 bash[74942]: audit 2024-04-07T17:18:24.320528+0000 mon.a (mon.0) 3382 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:18:25.207 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:18:24 smithi100 bash[74942]: audit 2024-04-07T17:18:24.326619+0000 mon.a (mon.0) 3383 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:18:25.207 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:18:24 smithi100 bash[74942]: audit 2024-04-07T17:18:24.399596+0000 mon.a (mon.0) 3384 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:18:25.207 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:18:24 smithi100 bash[74942]: audit 2024-04-07T17:18:24.399841+0000 mon.a (mon.0) 3385 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:18:25.207 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:18:24 smithi100 bash[74942]: audit 2024-04-07T17:18:24.400079+0000 mon.a (mon.0) 3386 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:18:25.207 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:18:24 smithi100 bash[74942]: audit 2024-04-07T17:18:24.402349+0000 mon.a (mon.0) 3387 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:18:25.207 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:18:24 smithi100 bash[74942]: audit 2024-04-07T17:18:24.402577+0000 mon.a (mon.0) 3388 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:18:25.207 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:18:24 smithi100 bash[74942]: audit 2024-04-07T17:18:24.402771+0000 mon.a (mon.0) 3389 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:18:25.207 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:18:24 smithi100 bash[74942]: audit 2024-04-07T17:18:24.404374+0000 mon.a (mon.0) 3390 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:18:25.207 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:18:24 smithi100 bash[74942]: audit 2024-04-07T17:18:24.404596+0000 mon.a (mon.0) 3391 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:18:25.207 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:18:24 smithi100 bash[74942]: audit 2024-04-07T17:18:24.404811+0000 mon.a (mon.0) 3392 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:18:25.207 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:18:24 smithi100 bash[74942]: audit 2024-04-07T17:18:24.417410+0000 mon.a (mon.0) 3393 : audit [INF] from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T17:18:25.208 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:18:24 smithi100 bash[76202]: cluster 2024-04-07T17:18:23.854176+0000 mgr.y (mgr.25175) 1891 : cluster [DBG] pgmap v3225: 649 pgs: 649 active+clean; 318 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 293 KiB/s rd, 60 KiB/s wr, 358 op/s 2024-04-07T17:18:25.208 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:18:24 smithi100 bash[76202]: cluster 2024-04-07T17:18:23.914617+0000 mon.a (mon.0) 3376 : cluster [DBG] osdmap e1760: 8 total, 8 up, 8 in 2024-04-07T17:18:25.208 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:18:24 smithi100 bash[76202]: audit 2024-04-07T17:18:24.145570+0000 mon.a (mon.0) 3377 : audit [INF] from='client.? ' entity='client.admin' 2024-04-07T17:18:25.208 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:18:24 smithi100 bash[76202]: audit 2024-04-07T17:18:24.188691+0000 mon.a (mon.0) 3378 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:18:25.208 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:18:24 smithi100 bash[76202]: audit 2024-04-07T17:18:24.200306+0000 mon.a (mon.0) 3379 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config-key dump", "format": "json"}]: dispatch 2024-04-07T17:18:25.208 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:18:24 smithi100 bash[76202]: audit 2024-04-07T17:18:24.271474+0000 mon.a (mon.0) 3380 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:18:25.208 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:18:24 smithi100 bash[76202]: audit 2024-04-07T17:18:24.319655+0000 mon.a (mon.0) 3381 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:18:25.208 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:18:24 smithi100 bash[76202]: audit 2024-04-07T17:18:24.320528+0000 mon.a (mon.0) 3382 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:18:25.208 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:18:24 smithi100 bash[76202]: audit 2024-04-07T17:18:24.326619+0000 mon.a (mon.0) 3383 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:18:25.208 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:18:24 smithi100 bash[76202]: audit 2024-04-07T17:18:24.399596+0000 mon.a (mon.0) 3384 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:18:25.208 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:18:24 smithi100 bash[76202]: audit 2024-04-07T17:18:24.399841+0000 mon.a (mon.0) 3385 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:18:25.208 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:18:24 smithi100 bash[76202]: audit 2024-04-07T17:18:24.400079+0000 mon.a (mon.0) 3386 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:18:25.208 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:18:24 smithi100 bash[76202]: audit 2024-04-07T17:18:24.402349+0000 mon.a (mon.0) 3387 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:18:25.209 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:18:24 smithi100 bash[76202]: audit 2024-04-07T17:18:24.402577+0000 mon.a (mon.0) 3388 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:18:25.209 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:18:24 smithi100 bash[76202]: audit 2024-04-07T17:18:24.402771+0000 mon.a (mon.0) 3389 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:18:25.209 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:18:24 smithi100 bash[76202]: audit 2024-04-07T17:18:24.404374+0000 mon.a (mon.0) 3390 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:18:25.209 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:18:24 smithi100 bash[76202]: audit 2024-04-07T17:18:24.404596+0000 mon.a (mon.0) 3391 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:18:25.209 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:18:24 smithi100 bash[76202]: audit 2024-04-07T17:18:24.404811+0000 mon.a (mon.0) 3392 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:18:25.209 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:18:24 smithi100 bash[76202]: audit 2024-04-07T17:18:24.417410+0000 mon.a (mon.0) 3393 : audit [INF] from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T17:18:25.209 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:18:24 smithi100 bash[74942]: audit 2024-04-07T17:18:24.417502+0000 mon.a (mon.0) 3394 : audit [INF] from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T17:18:25.209 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:18:24 smithi100 bash[74942]: audit 2024-04-07T17:18:24.417740+0000 mon.a (mon.0) 3395 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:18:25.209 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:18:24 smithi100 bash[74942]: audit 2024-04-07T17:18:24.417903+0000 mon.a (mon.0) 3396 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:18:25.209 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:18:24 smithi100 bash[74942]: audit 2024-04-07T17:18:24.418082+0000 mon.a (mon.0) 3397 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:18:25.210 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:18:24 smithi100 bash[74942]: audit 2024-04-07T17:18:24.420024+0000 mon.a (mon.0) 3398 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:18:25.210 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:18:24 smithi100 bash[74942]: audit 2024-04-07T17:18:24.420189+0000 mon.a (mon.0) 3399 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:18:25.210 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:18:24 smithi100 bash[74942]: audit 2024-04-07T17:18:24.420404+0000 mon.a (mon.0) 3400 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:18:25.210 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:18:24 smithi100 bash[74942]: audit 2024-04-07T17:18:24.420504+0000 mon.b (mon.1) 19 : audit [INF] from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T17:18:25.210 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:18:24 smithi100 bash[74942]: audit 2024-04-07T17:18:24.420637+0000 mon.b (mon.1) 20 : audit [INF] from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T17:18:25.210 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:18:24 smithi100 bash[74942]: audit 2024-04-07T17:18:24.421569+0000 mon.c (mon.2) 88 : audit [INF] from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T17:18:25.210 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:18:24 smithi100 bash[74942]: audit 2024-04-07T17:18:24.421618+0000 mon.c (mon.2) 89 : audit [INF] from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T17:18:25.210 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:18:24 smithi100 bash[74942]: audit 2024-04-07T17:18:24.421878+0000 mon.a (mon.0) 3401 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:18:25.210 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:18:24 smithi100 bash[74942]: audit 2024-04-07T17:18:24.422223+0000 mon.a (mon.0) 3402 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:18:25.210 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:18:24 smithi100 bash[74942]: audit 2024-04-07T17:18:24.422384+0000 mon.a (mon.0) 3403 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:18:25.210 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:18:24 smithi100 bash[76202]: audit 2024-04-07T17:18:24.417502+0000 mon.a (mon.0) 3394 : audit [INF] from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T17:18:25.210 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:18:24 smithi100 bash[76202]: audit 2024-04-07T17:18:24.417740+0000 mon.a (mon.0) 3395 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:18:25.211 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:18:24 smithi100 bash[76202]: audit 2024-04-07T17:18:24.417903+0000 mon.a (mon.0) 3396 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:18:25.211 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:18:24 smithi100 bash[76202]: audit 2024-04-07T17:18:24.418082+0000 mon.a (mon.0) 3397 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:18:25.211 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:18:24 smithi100 bash[76202]: audit 2024-04-07T17:18:24.420024+0000 mon.a (mon.0) 3398 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:18:25.211 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:18:24 smithi100 bash[76202]: audit 2024-04-07T17:18:24.420189+0000 mon.a (mon.0) 3399 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:18:25.211 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:18:24 smithi100 bash[76202]: audit 2024-04-07T17:18:24.420404+0000 mon.a (mon.0) 3400 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:18:25.211 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:18:24 smithi100 bash[76202]: audit 2024-04-07T17:18:24.420504+0000 mon.b (mon.1) 19 : audit [INF] from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T17:18:25.211 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:18:24 smithi100 bash[76202]: audit 2024-04-07T17:18:24.420637+0000 mon.b (mon.1) 20 : audit [INF] from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T17:18:25.211 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:18:24 smithi100 bash[76202]: audit 2024-04-07T17:18:24.421569+0000 mon.c (mon.2) 88 : audit [INF] from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T17:18:25.211 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:18:24 smithi100 bash[76202]: audit 2024-04-07T17:18:24.421618+0000 mon.c (mon.2) 89 : audit [INF] from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T17:18:25.211 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:18:24 smithi100 bash[76202]: audit 2024-04-07T17:18:24.421878+0000 mon.a (mon.0) 3401 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:18:25.211 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:18:24 smithi100 bash[76202]: audit 2024-04-07T17:18:24.422223+0000 mon.a (mon.0) 3402 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:18:25.211 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:18:24 smithi100 bash[76202]: audit 2024-04-07T17:18:24.422384+0000 mon.a (mon.0) 3403 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:18:25.370 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:18:24 smithi144 bash[66654]: cluster 2024-04-07T17:18:23.854176+0000 mgr.y (mgr.25175) 1891 : cluster [DBG] pgmap v3225: 649 pgs: 649 active+clean; 318 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 293 KiB/s rd, 60 KiB/s wr, 358 op/s 2024-04-07T17:18:25.370 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:18:24 smithi144 bash[66654]: cluster 2024-04-07T17:18:23.914617+0000 mon.a (mon.0) 3376 : cluster [DBG] osdmap e1760: 8 total, 8 up, 8 in 2024-04-07T17:18:25.370 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:18:24 smithi144 bash[66654]: audit 2024-04-07T17:18:24.145570+0000 mon.a (mon.0) 3377 : audit [INF] from='client.? ' entity='client.admin' 2024-04-07T17:18:25.370 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:18:24 smithi144 bash[66654]: audit 2024-04-07T17:18:24.188691+0000 mon.a (mon.0) 3378 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:18:25.370 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:18:24 smithi144 bash[66654]: audit 2024-04-07T17:18:24.200306+0000 mon.a (mon.0) 3379 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config-key dump", "format": "json"}]: dispatch 2024-04-07T17:18:25.370 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:18:24 smithi144 bash[66654]: audit 2024-04-07T17:18:24.271474+0000 mon.a (mon.0) 3380 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:18:25.370 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:18:24 smithi144 bash[66654]: audit 2024-04-07T17:18:24.319655+0000 mon.a (mon.0) 3381 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:18:25.370 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:18:24 smithi144 bash[66654]: audit 2024-04-07T17:18:24.320528+0000 mon.a (mon.0) 3382 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:18:25.370 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:18:24 smithi144 bash[66654]: audit 2024-04-07T17:18:24.326619+0000 mon.a (mon.0) 3383 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:18:25.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:18:24 smithi144 bash[66654]: audit 2024-04-07T17:18:24.399596+0000 mon.a (mon.0) 3384 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:18:25.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:18:24 smithi144 bash[66654]: audit 2024-04-07T17:18:24.399841+0000 mon.a (mon.0) 3385 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:18:25.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:18:24 smithi144 bash[66654]: audit 2024-04-07T17:18:24.400079+0000 mon.a (mon.0) 3386 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:18:25.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:18:24 smithi144 bash[66654]: audit 2024-04-07T17:18:24.402349+0000 mon.a (mon.0) 3387 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:18:25.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:18:24 smithi144 bash[66654]: audit 2024-04-07T17:18:24.402577+0000 mon.a (mon.0) 3388 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:18:25.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:18:24 smithi144 bash[66654]: audit 2024-04-07T17:18:24.402771+0000 mon.a (mon.0) 3389 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:18:25.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:18:24 smithi144 bash[66654]: audit 2024-04-07T17:18:24.404374+0000 mon.a (mon.0) 3390 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:18:25.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:18:24 smithi144 bash[66654]: audit 2024-04-07T17:18:24.404596+0000 mon.a (mon.0) 3391 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:18:25.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:18:24 smithi144 bash[66654]: audit 2024-04-07T17:18:24.404811+0000 mon.a (mon.0) 3392 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:18:25.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:18:24 smithi144 bash[66654]: audit 2024-04-07T17:18:24.417410+0000 mon.a (mon.0) 3393 : audit [INF] from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T17:18:25.371 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:18:24 smithi144 bash[66654]: audit 2024-04-07T17:18:24.417502+0000 mon.a (mon.0) 3394 : audit [INF] from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T17:18:25.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:18:24 smithi144 bash[66654]: audit 2024-04-07T17:18:24.417740+0000 mon.a (mon.0) 3395 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:18:25.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:18:24 smithi144 bash[66654]: audit 2024-04-07T17:18:24.417903+0000 mon.a (mon.0) 3396 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:18:25.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:18:24 smithi144 bash[66654]: audit 2024-04-07T17:18:24.418082+0000 mon.a (mon.0) 3397 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:18:25.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:18:24 smithi144 bash[66654]: audit 2024-04-07T17:18:24.420024+0000 mon.a (mon.0) 3398 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:18:25.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:18:24 smithi144 bash[66654]: audit 2024-04-07T17:18:24.420189+0000 mon.a (mon.0) 3399 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:18:25.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:18:24 smithi144 bash[66654]: audit 2024-04-07T17:18:24.420404+0000 mon.a (mon.0) 3400 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:18:25.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:18:24 smithi144 bash[66654]: audit 2024-04-07T17:18:24.420504+0000 mon.b (mon.1) 19 : audit [INF] from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T17:18:25.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:18:24 smithi144 bash[66654]: audit 2024-04-07T17:18:24.420637+0000 mon.b (mon.1) 20 : audit [INF] from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T17:18:25.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:18:24 smithi144 bash[66654]: audit 2024-04-07T17:18:24.421569+0000 mon.c (mon.2) 88 : audit [INF] from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T17:18:25.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:18:24 smithi144 bash[66654]: audit 2024-04-07T17:18:24.421618+0000 mon.c (mon.2) 89 : audit [INF] from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T17:18:25.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:18:24 smithi144 bash[66654]: audit 2024-04-07T17:18:24.421878+0000 mon.a (mon.0) 3401 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:18:25.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:18:24 smithi144 bash[66654]: audit 2024-04-07T17:18:24.422223+0000 mon.a (mon.0) 3402 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:18:25.372 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:18:24 smithi144 bash[66654]: audit 2024-04-07T17:18:24.422384+0000 mon.a (mon.0) 3403 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:18:26.357 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:18:26.617 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:18:26 smithi144 bash[66654]: cluster 2024-04-07T17:18:25.339019+0000 mon.a (mon.0) 3404 : cluster [DBG] osdmap e1761: 8 total, 8 up, 8 in 2024-04-07T17:18:26.618 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:18:26 smithi144 bash[66654]: audit 2024-04-07T17:18:25.346518+0000 mon.c (mon.2) 90 : audit [INF] from='client.? 172.21.15.100:0/4254735271' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-32","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:18:26.618 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:18:26 smithi144 bash[66654]: audit 2024-04-07T17:18:25.347580+0000 mon.a (mon.0) 3405 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-32","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:18:26.618 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:18:26 smithi144 bash[66654]: cluster 2024-04-07T17:18:25.855560+0000 mgr.y (mgr.25175) 1892 : cluster [DBG] pgmap v3228: 681 pgs: 32 unknown, 649 active+clean; 318 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 387 KiB/s rd, 26 KiB/s wr, 529 op/s 2024-04-07T17:18:26.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:18:26 smithi100 bash[74942]: cluster 2024-04-07T17:18:25.339019+0000 mon.a (mon.0) 3404 : cluster [DBG] osdmap e1761: 8 total, 8 up, 8 in 2024-04-07T17:18:26.705 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:18:26 smithi100 bash[74942]: audit 2024-04-07T17:18:25.346518+0000 mon.c (mon.2) 90 : audit [INF] from='client.? 172.21.15.100:0/4254735271' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-32","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:18:26.705 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:18:26 smithi100 bash[74942]: audit 2024-04-07T17:18:25.347580+0000 mon.a (mon.0) 3405 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-32","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:18:26.705 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:18:26 smithi100 bash[74942]: cluster 2024-04-07T17:18:25.855560+0000 mgr.y (mgr.25175) 1892 : cluster [DBG] pgmap v3228: 681 pgs: 32 unknown, 649 active+clean; 318 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 387 KiB/s rd, 26 KiB/s wr, 529 op/s 2024-04-07T17:18:26.705 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:18:26 smithi100 bash[76202]: cluster 2024-04-07T17:18:25.339019+0000 mon.a (mon.0) 3404 : cluster [DBG] osdmap e1761: 8 total, 8 up, 8 in 2024-04-07T17:18:26.705 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:18:26 smithi100 bash[76202]: audit 2024-04-07T17:18:25.346518+0000 mon.c (mon.2) 90 : audit [INF] from='client.? 172.21.15.100:0/4254735271' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-32","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:18:26.705 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:18:26 smithi100 bash[76202]: audit 2024-04-07T17:18:25.347580+0000 mon.a (mon.0) 3405 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-32","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:18:26.705 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:18:26 smithi100 bash[76202]: cluster 2024-04-07T17:18:25.855560+0000 mgr.y (mgr.25175) 1892 : cluster [DBG] pgmap v3228: 681 pgs: 32 unknown, 649 active+clean; 318 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 387 KiB/s rd, 26 KiB/s wr, 529 op/s 2024-04-07T17:18:27.617 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:18:27 smithi144 bash[66654]: cluster 2024-04-07T17:18:26.327531+0000 mon.a (mon.0) 3406 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T17:18:27.618 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:18:27 smithi144 bash[66654]: audit 2024-04-07T17:18:26.338254+0000 mon.a (mon.0) 3407 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-32","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T17:18:27.618 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:18:27 smithi144 bash[66654]: cluster 2024-04-07T17:18:26.349589+0000 mon.a (mon.0) 3408 : cluster [DBG] osdmap e1762: 8 total, 8 up, 8 in 2024-04-07T17:18:27.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:18:27 smithi100 bash[74942]: cluster 2024-04-07T17:18:26.327531+0000 mon.a (mon.0) 3406 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T17:18:27.705 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:18:27 smithi100 bash[74942]: audit 2024-04-07T17:18:26.338254+0000 mon.a (mon.0) 3407 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-32","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T17:18:27.705 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:18:27 smithi100 bash[74942]: cluster 2024-04-07T17:18:26.349589+0000 mon.a (mon.0) 3408 : cluster [DBG] osdmap e1762: 8 total, 8 up, 8 in 2024-04-07T17:18:27.705 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:18:27 smithi100 bash[76202]: cluster 2024-04-07T17:18:26.327531+0000 mon.a (mon.0) 3406 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T17:18:27.705 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:18:27 smithi100 bash[76202]: audit 2024-04-07T17:18:26.338254+0000 mon.a (mon.0) 3407 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-32","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T17:18:27.705 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:18:27 smithi100 bash[76202]: cluster 2024-04-07T17:18:26.349589+0000 mon.a (mon.0) 3408 : cluster [DBG] osdmap e1762: 8 total, 8 up, 8 in 2024-04-07T17:18:28.574 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:18:28.603 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:18:28.617 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:18:28 smithi144 bash[66654]: cluster 2024-04-07T17:18:27.358019+0000 mon.a (mon.0) 3409 : cluster [DBG] osdmap e1763: 8 total, 8 up, 8 in 2024-04-07T17:18:28.617 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:18:28 smithi144 bash[66654]: cluster 2024-04-07T17:18:27.856860+0000 mgr.y (mgr.25175) 1893 : cluster [DBG] pgmap v3231: 681 pgs: 16 unknown, 665 active+clean; 318 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 506 KiB/s rd, 32 KiB/s wr, 650 op/s 2024-04-07T17:18:28.624 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:18:28.704 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:18:28 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:18:28] "GET /metrics HTTP/1.1" 200 75602 "" "Prometheus/2.43.0" 2024-04-07T17:18:28.705 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:18:28 smithi100 bash[74942]: cluster 2024-04-07T17:18:27.358019+0000 mon.a (mon.0) 3409 : cluster [DBG] osdmap e1763: 8 total, 8 up, 8 in 2024-04-07T17:18:28.705 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:18:28 smithi100 bash[74942]: cluster 2024-04-07T17:18:27.856860+0000 mgr.y (mgr.25175) 1893 : cluster [DBG] pgmap v3231: 681 pgs: 16 unknown, 665 active+clean; 318 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 506 KiB/s rd, 32 KiB/s wr, 650 op/s 2024-04-07T17:18:28.705 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:18:28 smithi100 bash[76202]: cluster 2024-04-07T17:18:27.358019+0000 mon.a (mon.0) 3409 : cluster [DBG] osdmap e1763: 8 total, 8 up, 8 in 2024-04-07T17:18:28.705 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:18:28 smithi100 bash[76202]: cluster 2024-04-07T17:18:27.856860+0000 mgr.y (mgr.25175) 1893 : cluster [DBG] pgmap v3231: 681 pgs: 16 unknown, 665 active+clean; 318 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 506 KiB/s rd, 32 KiB/s wr, 650 op/s 2024-04-07T17:18:29.589 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.PoolStatsPP (5244 ms) 2024-04-07T17:18:29.590 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.ImageSpec 2024-04-07T17:18:29.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:18:29 smithi100 bash[74942]: cluster 2024-04-07T17:18:28.351396+0000 mon.a (mon.0) 3410 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T17:18:29.705 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:18:29 smithi100 bash[74942]: cluster 2024-04-07T17:18:28.373929+0000 mon.a (mon.0) 3411 : cluster [DBG] osdmap e1764: 8 total, 8 up, 8 in 2024-04-07T17:18:29.705 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:18:29 smithi100 bash[76202]: cluster 2024-04-07T17:18:28.351396+0000 mon.a (mon.0) 3410 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T17:18:29.705 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:18:29 smithi100 bash[76202]: cluster 2024-04-07T17:18:28.373929+0000 mon.a (mon.0) 3411 : cluster [DBG] osdmap e1764: 8 total, 8 up, 8 in 2024-04-07T17:18:29.867 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:18:29 smithi144 bash[66654]: cluster 2024-04-07T17:18:28.351396+0000 mon.a (mon.0) 3410 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T17:18:29.868 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:18:29 smithi144 bash[66654]: cluster 2024-04-07T17:18:28.373929+0000 mon.a (mon.0) 3411 : cluster [DBG] osdmap e1764: 8 total, 8 up, 8 in 2024-04-07T17:18:30.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:18:30 smithi100 bash[74942]: cluster 2024-04-07T17:18:29.380272+0000 mon.a (mon.0) 3412 : cluster [DBG] osdmap e1765: 8 total, 8 up, 8 in 2024-04-07T17:18:30.705 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:18:30 smithi100 bash[74942]: cluster 2024-04-07T17:18:29.858086+0000 mgr.y (mgr.25175) 1894 : cluster [DBG] pgmap v3234: 681 pgs: 6 unknown, 675 active+clean; 318 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 370 KiB/s rd, 21 KiB/s wr, 481 op/s 2024-04-07T17:18:30.705 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:18:30 smithi100 bash[76202]: cluster 2024-04-07T17:18:29.380272+0000 mon.a (mon.0) 3412 : cluster [DBG] osdmap e1765: 8 total, 8 up, 8 in 2024-04-07T17:18:30.705 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:18:30 smithi100 bash[76202]: cluster 2024-04-07T17:18:29.858086+0000 mgr.y (mgr.25175) 1894 : cluster [DBG] pgmap v3234: 681 pgs: 6 unknown, 675 active+clean; 318 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 370 KiB/s rd, 21 KiB/s wr, 481 op/s 2024-04-07T17:18:30.867 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:18:30 smithi144 bash[66654]: cluster 2024-04-07T17:18:29.380272+0000 mon.a (mon.0) 3412 : cluster [DBG] osdmap e1765: 8 total, 8 up, 8 in 2024-04-07T17:18:30.868 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:18:30 smithi144 bash[66654]: cluster 2024-04-07T17:18:29.858086+0000 mgr.y (mgr.25175) 1894 : cluster [DBG] pgmap v3234: 681 pgs: 6 unknown, 675 active+clean; 318 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 370 KiB/s rd, 21 KiB/s wr, 481 op/s 2024-04-07T17:18:31.420 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:18:31.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:18:31 smithi100 bash[74942]: cluster 2024-04-07T17:18:30.402738+0000 mon.a (mon.0) 3413 : cluster [DBG] osdmap e1766: 8 total, 8 up, 8 in 2024-04-07T17:18:31.705 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:18:31 smithi100 bash[74942]: audit 2024-04-07T17:18:30.408440+0000 mon.c (mon.2) 91 : audit [INF] from='client.? 172.21.15.100:0/3373773988' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:18:31.705 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:18:31 smithi100 bash[74942]: audit 2024-04-07T17:18:30.414201+0000 mon.a (mon.0) 3414 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:18:31.705 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:18:31 smithi100 bash[76202]: cluster 2024-04-07T17:18:30.402738+0000 mon.a (mon.0) 3413 : cluster [DBG] osdmap e1766: 8 total, 8 up, 8 in 2024-04-07T17:18:31.705 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:18:31 smithi100 bash[76202]: audit 2024-04-07T17:18:30.408440+0000 mon.c (mon.2) 91 : audit [INF] from='client.? 172.21.15.100:0/3373773988' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:18:31.705 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:18:31 smithi100 bash[76202]: audit 2024-04-07T17:18:30.414201+0000 mon.a (mon.0) 3414 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:18:31.868 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:18:31 smithi144 bash[66654]: cluster 2024-04-07T17:18:30.402738+0000 mon.a (mon.0) 3413 : cluster [DBG] osdmap e1766: 8 total, 8 up, 8 in 2024-04-07T17:18:31.868 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:18:31 smithi144 bash[66654]: audit 2024-04-07T17:18:30.408440+0000 mon.c (mon.2) 91 : audit [INF] from='client.? 172.21.15.100:0/3373773988' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:18:31.868 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:18:31 smithi144 bash[66654]: audit 2024-04-07T17:18:30.414201+0000 mon.a (mon.0) 3414 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:18:32.618 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:18:32 smithi144 bash[95233]: ts=2024-04-07T17:18:32.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:18:32.618 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:18:32 smithi144 bash[66654]: audit 2024-04-07T17:18:31.400781+0000 mon.a (mon.0) 3415 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T17:18:32.618 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:18:32 smithi144 bash[66654]: cluster 2024-04-07T17:18:31.409178+0000 mon.a (mon.0) 3416 : cluster [DBG] osdmap e1767: 8 total, 8 up, 8 in 2024-04-07T17:18:32.618 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:18:32 smithi144 bash[66654]: audit 2024-04-07T17:18:31.830438+0000 mon.a (mon.0) 3417 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:18:32.618 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:18:32 smithi144 bash[66654]: cluster 2024-04-07T17:18:31.859263+0000 mgr.y (mgr.25175) 1895 : cluster [DBG] pgmap v3237: 713 pgs: 32 unknown, 681 active+clean; 318 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 74 KiB/s rd, 6.0 KiB/s wr, 99 op/s 2024-04-07T17:18:32.705 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:18:32 smithi100 bash[74942]: audit 2024-04-07T17:18:31.400781+0000 mon.a (mon.0) 3415 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T17:18:32.705 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:18:32 smithi100 bash[74942]: cluster 2024-04-07T17:18:31.409178+0000 mon.a (mon.0) 3416 : cluster [DBG] osdmap e1767: 8 total, 8 up, 8 in 2024-04-07T17:18:32.705 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:18:32 smithi100 bash[74942]: audit 2024-04-07T17:18:31.830438+0000 mon.a (mon.0) 3417 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:18:32.705 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:18:32 smithi100 bash[74942]: cluster 2024-04-07T17:18:31.859263+0000 mgr.y (mgr.25175) 1895 : cluster [DBG] pgmap v3237: 713 pgs: 32 unknown, 681 active+clean; 318 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 74 KiB/s rd, 6.0 KiB/s wr, 99 op/s 2024-04-07T17:18:32.705 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:18:32 smithi100 bash[76202]: audit 2024-04-07T17:18:31.400781+0000 mon.a (mon.0) 3415 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T17:18:32.705 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:18:32 smithi100 bash[76202]: cluster 2024-04-07T17:18:31.409178+0000 mon.a (mon.0) 3416 : cluster [DBG] osdmap e1767: 8 total, 8 up, 8 in 2024-04-07T17:18:32.706 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:18:32 smithi100 bash[76202]: audit 2024-04-07T17:18:31.830438+0000 mon.a (mon.0) 3417 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:18:32.706 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:18:32 smithi100 bash[76202]: cluster 2024-04-07T17:18:31.859263+0000 mgr.y (mgr.25175) 1895 : cluster [DBG] pgmap v3237: 713 pgs: 32 unknown, 681 active+clean; 318 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 74 KiB/s rd, 6.0 KiB/s wr, 99 op/s 2024-04-07T17:18:33.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:18:33 smithi100 bash[74942]: cluster 2024-04-07T17:18:32.412384+0000 mon.a (mon.0) 3418 : cluster [DBG] osdmap e1768: 8 total, 8 up, 8 in 2024-04-07T17:18:33.705 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:18:33 smithi100 bash[76202]: cluster 2024-04-07T17:18:32.412384+0000 mon.a (mon.0) 3418 : cluster [DBG] osdmap e1768: 8 total, 8 up, 8 in 2024-04-07T17:18:33.867 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:18:33 smithi144 bash[66654]: cluster 2024-04-07T17:18:32.412384+0000 mon.a (mon.0) 3418 : cluster [DBG] osdmap e1768: 8 total, 8 up, 8 in 2024-04-07T17:18:34.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:18:34 smithi100 bash[74942]: cluster 2024-04-07T17:18:33.441760+0000 mon.a (mon.0) 3419 : cluster [DBG] osdmap e1769: 8 total, 8 up, 8 in 2024-04-07T17:18:34.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:18:34 smithi100 bash[74942]: cluster 2024-04-07T17:18:33.860514+0000 mgr.y (mgr.25175) 1896 : cluster [DBG] pgmap v3240: 713 pgs: 13 unknown, 700 active+clean; 318 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 37 KiB/s rd, 4.5 KiB/s wr, 50 op/s 2024-04-07T17:18:34.705 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:18:34 smithi100 bash[76202]: cluster 2024-04-07T17:18:33.441760+0000 mon.a (mon.0) 3419 : cluster [DBG] osdmap e1769: 8 total, 8 up, 8 in 2024-04-07T17:18:34.705 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:18:34 smithi100 bash[76202]: cluster 2024-04-07T17:18:33.860514+0000 mgr.y (mgr.25175) 1896 : cluster [DBG] pgmap v3240: 713 pgs: 13 unknown, 700 active+clean; 318 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 37 KiB/s rd, 4.5 KiB/s wr, 50 op/s 2024-04-07T17:18:34.867 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:18:34 smithi144 bash[66654]: cluster 2024-04-07T17:18:33.441760+0000 mon.a (mon.0) 3419 : cluster [DBG] osdmap e1769: 8 total, 8 up, 8 in 2024-04-07T17:18:34.868 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:18:34 smithi144 bash[66654]: cluster 2024-04-07T17:18:33.860514+0000 mgr.y (mgr.25175) 1896 : cluster [DBG] pgmap v3240: 713 pgs: 13 unknown, 700 active+clean; 318 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 37 KiB/s rd, 4.5 KiB/s wr, 50 op/s 2024-04-07T17:18:35.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:18:35 smithi100 bash[74942]: cluster 2024-04-07T17:18:34.457013+0000 mon.a (mon.0) 3420 : cluster [DBG] osdmap e1770: 8 total, 8 up, 8 in 2024-04-07T17:18:35.705 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:18:35 smithi100 bash[76202]: cluster 2024-04-07T17:18:34.457013+0000 mon.a (mon.0) 3420 : cluster [DBG] osdmap e1770: 8 total, 8 up, 8 in 2024-04-07T17:18:35.850 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.ImageSpec (6260 ms) 2024-04-07T17:18:35.850 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.SnapRemoveWithChildMissing 2024-04-07T17:18:35.870 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:18:35 smithi144 bash[66654]: cluster 2024-04-07T17:18:34.457013+0000 mon.a (mon.0) 3420 : cluster [DBG] osdmap e1770: 8 total, 8 up, 8 in 2024-04-07T17:18:36.868 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:18:36 smithi144 bash[66654]: cluster 2024-04-07T17:18:35.471070+0000 mon.a (mon.0) 3421 : cluster [DBG] osdmap e1771: 8 total, 8 up, 8 in 2024-04-07T17:18:36.868 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:18:36 smithi144 bash[66654]: cluster 2024-04-07T17:18:35.861873+0000 mgr.y (mgr.25175) 1897 : cluster [DBG] pgmap v3243: 713 pgs: 713 active+clean; 318 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 42 KiB/s rd, 4.7 KiB/s wr, 55 op/s 2024-04-07T17:18:36.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:18:36 smithi100 bash[74942]: cluster 2024-04-07T17:18:35.471070+0000 mon.a (mon.0) 3421 : cluster [DBG] osdmap e1771: 8 total, 8 up, 8 in 2024-04-07T17:18:36.955 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:18:36 smithi100 bash[74942]: cluster 2024-04-07T17:18:35.861873+0000 mgr.y (mgr.25175) 1897 : cluster [DBG] pgmap v3243: 713 pgs: 713 active+clean; 318 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 42 KiB/s rd, 4.7 KiB/s wr, 55 op/s 2024-04-07T17:18:36.955 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:18:36 smithi100 bash[76202]: cluster 2024-04-07T17:18:35.471070+0000 mon.a (mon.0) 3421 : cluster [DBG] osdmap e1771: 8 total, 8 up, 8 in 2024-04-07T17:18:36.955 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:18:36 smithi100 bash[76202]: cluster 2024-04-07T17:18:35.861873+0000 mgr.y (mgr.25175) 1897 : cluster [DBG] pgmap v3243: 713 pgs: 713 active+clean; 318 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 42 KiB/s rd, 4.7 KiB/s wr, 55 op/s 2024-04-07T17:18:37.504 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:18:37.867 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:18:37 smithi144 bash[66654]: cluster 2024-04-07T17:18:36.485000+0000 mon.a (mon.0) 3422 : cluster [DBG] osdmap e1772: 8 total, 8 up, 8 in 2024-04-07T17:18:37.868 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:18:37 smithi144 bash[66654]: audit 2024-04-07T17:18:36.486986+0000 mon.c (mon.2) 92 : audit [INF] from='client.? 172.21.15.100:0/1844270563' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-34","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:18:37.868 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:18:37 smithi144 bash[66654]: audit 2024-04-07T17:18:36.488607+0000 mon.a (mon.0) 3423 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-34","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:18:37.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:18:37 smithi100 bash[74942]: cluster 2024-04-07T17:18:36.485000+0000 mon.a (mon.0) 3422 : cluster [DBG] osdmap e1772: 8 total, 8 up, 8 in 2024-04-07T17:18:37.955 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:18:37 smithi100 bash[74942]: audit 2024-04-07T17:18:36.486986+0000 mon.c (mon.2) 92 : audit [INF] from='client.? 172.21.15.100:0/1844270563' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-34","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:18:37.955 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:18:37 smithi100 bash[74942]: audit 2024-04-07T17:18:36.488607+0000 mon.a (mon.0) 3423 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-34","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:18:37.956 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:18:37 smithi100 bash[76202]: cluster 2024-04-07T17:18:36.485000+0000 mon.a (mon.0) 3422 : cluster [DBG] osdmap e1772: 8 total, 8 up, 8 in 2024-04-07T17:18:37.956 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:18:37 smithi100 bash[76202]: audit 2024-04-07T17:18:36.486986+0000 mon.c (mon.2) 92 : audit [INF] from='client.? 172.21.15.100:0/1844270563' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-34","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:18:37.956 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:18:37 smithi100 bash[76202]: audit 2024-04-07T17:18:36.488607+0000 mon.a (mon.0) 3423 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-34","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:18:38.704 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:18:38 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:18:38] "GET /metrics HTTP/1.1" 200 80482 "" "Prometheus/2.43.0" 2024-04-07T17:18:38.705 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:18:38 smithi100 bash[74942]: audit 2024-04-07T17:18:37.478692+0000 mon.a (mon.0) 3424 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-34","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T17:18:38.705 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:18:38 smithi100 bash[74942]: cluster 2024-04-07T17:18:37.487148+0000 mon.a (mon.0) 3425 : cluster [DBG] osdmap e1773: 8 total, 8 up, 8 in 2024-04-07T17:18:38.705 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:18:38 smithi100 bash[74942]: cluster 2024-04-07T17:18:37.863088+0000 mgr.y (mgr.25175) 1898 : cluster [DBG] pgmap v3246: 745 pgs: 13 creating+peering, 19 unknown, 713 active+clean; 318 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 120 KiB/s rd, 5.7 KiB/s wr, 153 op/s 2024-04-07T17:18:38.705 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:18:38 smithi100 bash[74942]: cluster 2024-04-07T17:18:37.909310+0000 mon.a (mon.0) 3426 : cluster [DBG] osdmap e1774: 8 total, 8 up, 8 in 2024-04-07T17:18:38.705 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:18:38 smithi100 bash[76202]: audit 2024-04-07T17:18:37.478692+0000 mon.a (mon.0) 3424 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-34","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T17:18:38.705 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:18:38 smithi100 bash[76202]: cluster 2024-04-07T17:18:37.487148+0000 mon.a (mon.0) 3425 : cluster [DBG] osdmap e1773: 8 total, 8 up, 8 in 2024-04-07T17:18:38.705 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:18:38 smithi100 bash[76202]: cluster 2024-04-07T17:18:37.863088+0000 mgr.y (mgr.25175) 1898 : cluster [DBG] pgmap v3246: 745 pgs: 13 creating+peering, 19 unknown, 713 active+clean; 318 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 120 KiB/s rd, 5.7 KiB/s wr, 153 op/s 2024-04-07T17:18:38.706 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:18:38 smithi100 bash[76202]: cluster 2024-04-07T17:18:37.909310+0000 mon.a (mon.0) 3426 : cluster [DBG] osdmap e1774: 8 total, 8 up, 8 in 2024-04-07T17:18:38.868 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:18:38 smithi144 bash[66654]: audit 2024-04-07T17:18:37.478692+0000 mon.a (mon.0) 3424 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-34","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T17:18:38.868 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:18:38 smithi144 bash[66654]: cluster 2024-04-07T17:18:37.487148+0000 mon.a (mon.0) 3425 : cluster [DBG] osdmap e1773: 8 total, 8 up, 8 in 2024-04-07T17:18:38.868 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:18:38 smithi144 bash[66654]: cluster 2024-04-07T17:18:37.863088+0000 mgr.y (mgr.25175) 1898 : cluster [DBG] pgmap v3246: 745 pgs: 13 creating+peering, 19 unknown, 713 active+clean; 318 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 120 KiB/s rd, 5.7 KiB/s wr, 153 op/s 2024-04-07T17:18:38.868 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:18:38 smithi144 bash[66654]: cluster 2024-04-07T17:18:37.909310+0000 mon.a (mon.0) 3426 : cluster [DBG] osdmap e1774: 8 total, 8 up, 8 in 2024-04-07T17:18:40.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:18:39 smithi100 bash[74942]: cluster 2024-04-07T17:18:38.910243+0000 mon.a (mon.0) 3427 : cluster [DBG] osdmap e1775: 8 total, 8 up, 8 in 2024-04-07T17:18:40.205 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:18:39 smithi100 bash[76202]: cluster 2024-04-07T17:18:38.910243+0000 mon.a (mon.0) 3427 : cluster [DBG] osdmap e1775: 8 total, 8 up, 8 in 2024-04-07T17:18:40.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:18:39 smithi144 bash[66654]: cluster 2024-04-07T17:18:38.910243+0000 mon.a (mon.0) 3427 : cluster [DBG] osdmap e1775: 8 total, 8 up, 8 in 2024-04-07T17:18:41.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:18:40 smithi100 bash[74942]: cluster 2024-04-07T17:18:39.865939+0000 mgr.y (mgr.25175) 1899 : cluster [DBG] pgmap v3249: 745 pgs: 13 creating+peering, 6 unknown, 726 active+clean; 318 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 140 KiB/s rd, 6.2 KiB/s wr, 176 op/s 2024-04-07T17:18:41.205 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:18:40 smithi100 bash[74942]: cluster 2024-04-07T17:18:39.930618+0000 mon.a (mon.0) 3428 : cluster [DBG] osdmap e1776: 8 total, 8 up, 8 in 2024-04-07T17:18:41.205 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:18:40 smithi100 bash[76202]: cluster 2024-04-07T17:18:39.865939+0000 mgr.y (mgr.25175) 1899 : cluster [DBG] pgmap v3249: 745 pgs: 13 creating+peering, 6 unknown, 726 active+clean; 318 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 140 KiB/s rd, 6.2 KiB/s wr, 176 op/s 2024-04-07T17:18:41.205 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:18:40 smithi100 bash[76202]: cluster 2024-04-07T17:18:39.930618+0000 mon.a (mon.0) 3428 : cluster [DBG] osdmap e1776: 8 total, 8 up, 8 in 2024-04-07T17:18:41.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:18:40 smithi144 bash[66654]: cluster 2024-04-07T17:18:39.865939+0000 mgr.y (mgr.25175) 1899 : cluster [DBG] pgmap v3249: 745 pgs: 13 creating+peering, 6 unknown, 726 active+clean; 318 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 140 KiB/s rd, 6.2 KiB/s wr, 176 op/s 2024-04-07T17:18:41.368 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:18:40 smithi144 bash[66654]: cluster 2024-04-07T17:18:39.930618+0000 mon.a (mon.0) 3428 : cluster [DBG] osdmap e1776: 8 total, 8 up, 8 in 2024-04-07T17:18:42.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:18:41 smithi100 bash[74942]: cluster 2024-04-07T17:18:40.928318+0000 mon.a (mon.0) 3429 : cluster [DBG] osdmap e1777: 8 total, 8 up, 8 in 2024-04-07T17:18:42.205 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:18:41 smithi100 bash[76202]: cluster 2024-04-07T17:18:40.928318+0000 mon.a (mon.0) 3429 : cluster [DBG] osdmap e1777: 8 total, 8 up, 8 in 2024-04-07T17:18:42.317 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:18:41 smithi144 bash[66654]: cluster 2024-04-07T17:18:40.928318+0000 mon.a (mon.0) 3429 : cluster [DBG] osdmap e1777: 8 total, 8 up, 8 in 2024-04-07T17:18:42.618 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:18:42 smithi144 bash[95233]: ts=2024-04-07T17:18:42.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:18:43.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:18:42 smithi100 bash[74942]: cluster 2024-04-07T17:18:41.867252+0000 mgr.y (mgr.25175) 1900 : cluster [DBG] pgmap v3252: 745 pgs: 13 creating+peering, 732 active+clean; 318 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 24 KiB/s rd, 1.2 KiB/s wr, 28 op/s 2024-04-07T17:18:43.205 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:18:42 smithi100 bash[74942]: cluster 2024-04-07T17:18:41.953271+0000 mon.a (mon.0) 3430 : cluster [DBG] osdmap e1778: 8 total, 8 up, 8 in 2024-04-07T17:18:43.205 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:18:42 smithi100 bash[74942]: cluster 2024-04-07T17:18:42.913178+0000 mon.a (mon.0) 3431 : cluster [DBG] osdmap e1779: 8 total, 8 up, 8 in 2024-04-07T17:18:43.205 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:18:42 smithi100 bash[76202]: cluster 2024-04-07T17:18:41.867252+0000 mgr.y (mgr.25175) 1900 : cluster [DBG] pgmap v3252: 745 pgs: 13 creating+peering, 732 active+clean; 318 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 24 KiB/s rd, 1.2 KiB/s wr, 28 op/s 2024-04-07T17:18:43.205 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:18:42 smithi100 bash[76202]: cluster 2024-04-07T17:18:41.953271+0000 mon.a (mon.0) 3430 : cluster [DBG] osdmap e1778: 8 total, 8 up, 8 in 2024-04-07T17:18:43.205 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:18:42 smithi100 bash[76202]: cluster 2024-04-07T17:18:42.913178+0000 mon.a (mon.0) 3431 : cluster [DBG] osdmap e1779: 8 total, 8 up, 8 in 2024-04-07T17:18:43.368 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:18:42 smithi144 bash[66654]: cluster 2024-04-07T17:18:41.867252+0000 mgr.y (mgr.25175) 1900 : cluster [DBG] pgmap v3252: 745 pgs: 13 creating+peering, 732 active+clean; 318 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 24 KiB/s rd, 1.2 KiB/s wr, 28 op/s 2024-04-07T17:18:43.368 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:18:42 smithi144 bash[66654]: cluster 2024-04-07T17:18:41.953271+0000 mon.a (mon.0) 3430 : cluster [DBG] osdmap e1778: 8 total, 8 up, 8 in 2024-04-07T17:18:43.368 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:18:42 smithi144 bash[66654]: cluster 2024-04-07T17:18:42.913178+0000 mon.a (mon.0) 3431 : cluster [DBG] osdmap e1779: 8 total, 8 up, 8 in 2024-04-07T17:18:44.061 INFO:tasks.workunit.client.0.smithi100.stdout:num children is: 3 2024-04-07T17:18:44.061 INFO:tasks.workunit.client.0.smithi100.stdout:expected: 3 2024-04-07T17:18:44.062 INFO:tasks.workunit.client.0.smithi100.stdout:child: image218 2024-04-07T17:18:44.062 INFO:tasks.workunit.client.0.smithi100.stdout:child: image220 2024-04-07T17:18:44.062 INFO:tasks.workunit.client.0.smithi100.stdout:child: image219 2024-04-07T17:18:44.062 INFO:tasks.workunit.client.0.smithi100.stdout:found child test-librbd-smithi100-115303-15/image218/b186ad51d5ef 2024-04-07T17:18:44.062 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:18:44.062 INFO:tasks.workunit.client.0.smithi100.stdout:found child test-librbd-smithi100-115303-34/image219/b186d0a31417 2024-04-07T17:18:44.062 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:18:44.062 INFO:tasks.workunit.client.0.smithi100.stdout:found child test-librbd-smithi100-115303-15/image220/b186eccac0eb 2024-04-07T17:18:44.062 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:18:44.062 INFO:tasks.workunit.client.0.smithi100.stdout:num children is: 0 2024-04-07T17:18:44.062 INFO:tasks.workunit.client.0.smithi100.stdout:expected: 0 2024-04-07T17:18:44.062 INFO:tasks.workunit.client.0.smithi100.stdout:num snaps is: 0 2024-04-07T17:18:44.062 INFO:tasks.workunit.client.0.smithi100.stdout:expected: 0 2024-04-07T17:18:44.062 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.SnapRemoveWithChildMissing (8212 ms) 2024-04-07T17:18:44.062 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.QuiesceWatch 2024-04-07T17:18:45.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:18:44 smithi144 bash[66654]: cluster 2024-04-07T17:18:43.869228+0000 mgr.y (mgr.25175) 1901 : cluster [DBG] pgmap v3255: 713 pgs: 713 active+clean; 6.6 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail 2024-04-07T17:18:45.368 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:18:44 smithi144 bash[66654]: cluster 2024-04-07T17:18:43.959278+0000 mon.a (mon.0) 3432 : cluster [DBG] osdmap e1780: 8 total, 8 up, 8 in 2024-04-07T17:18:45.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:18:44 smithi100 bash[74942]: cluster 2024-04-07T17:18:43.869228+0000 mgr.y (mgr.25175) 1901 : cluster [DBG] pgmap v3255: 713 pgs: 713 active+clean; 6.6 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail 2024-04-07T17:18:45.455 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:18:44 smithi100 bash[74942]: cluster 2024-04-07T17:18:43.959278+0000 mon.a (mon.0) 3432 : cluster [DBG] osdmap e1780: 8 total, 8 up, 8 in 2024-04-07T17:18:45.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:18:44 smithi100 bash[76202]: cluster 2024-04-07T17:18:43.869228+0000 mgr.y (mgr.25175) 1901 : cluster [DBG] pgmap v3255: 713 pgs: 713 active+clean; 6.6 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail 2024-04-07T17:18:45.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:18:44 smithi100 bash[76202]: cluster 2024-04-07T17:18:43.959278+0000 mon.a (mon.0) 3432 : cluster [DBG] osdmap e1780: 8 total, 8 up, 8 in 2024-04-07T17:18:46.003 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:18:46.368 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:18:45 smithi144 bash[66654]: cluster 2024-04-07T17:18:44.975650+0000 mon.a (mon.0) 3433 : cluster [DBG] osdmap e1781: 8 total, 8 up, 8 in 2024-04-07T17:18:46.368 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:18:45 smithi144 bash[66654]: audit 2024-04-07T17:18:44.984518+0000 mon.c (mon.2) 93 : audit [INF] from='client.? 172.21.15.100:0/2814523829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:18:46.368 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:18:45 smithi144 bash[66654]: audit 2024-04-07T17:18:44.985453+0000 mon.a (mon.0) 3434 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:18:46.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:18:45 smithi100 bash[74942]: cluster 2024-04-07T17:18:44.975650+0000 mon.a (mon.0) 3433 : cluster [DBG] osdmap e1781: 8 total, 8 up, 8 in 2024-04-07T17:18:46.455 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:18:45 smithi100 bash[74942]: audit 2024-04-07T17:18:44.984518+0000 mon.c (mon.2) 93 : audit [INF] from='client.? 172.21.15.100:0/2814523829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:18:46.455 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:18:45 smithi100 bash[74942]: audit 2024-04-07T17:18:44.985453+0000 mon.a (mon.0) 3434 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:18:46.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:18:45 smithi100 bash[76202]: cluster 2024-04-07T17:18:44.975650+0000 mon.a (mon.0) 3433 : cluster [DBG] osdmap e1781: 8 total, 8 up, 8 in 2024-04-07T17:18:46.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:18:45 smithi100 bash[76202]: audit 2024-04-07T17:18:44.984518+0000 mon.c (mon.2) 93 : audit [INF] from='client.? 172.21.15.100:0/2814523829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:18:46.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:18:45 smithi100 bash[76202]: audit 2024-04-07T17:18:44.985453+0000 mon.a (mon.0) 3434 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:18:47.368 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:18:46 smithi144 bash[66654]: cluster 2024-04-07T17:18:45.871459+0000 mgr.y (mgr.25175) 1902 : cluster [DBG] pgmap v3258: 745 pgs: 4 creating+peering, 28 unknown, 713 active+clean; 6.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail 2024-04-07T17:18:47.368 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:18:46 smithi144 bash[66654]: cluster 2024-04-07T17:18:45.968434+0000 mon.a (mon.0) 3435 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T17:18:47.368 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:18:46 smithi144 bash[66654]: audit 2024-04-07T17:18:45.978236+0000 mon.a (mon.0) 3436 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T17:18:47.368 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:18:46 smithi144 bash[66654]: cluster 2024-04-07T17:18:45.989380+0000 mon.a (mon.0) 3437 : cluster [DBG] osdmap e1782: 8 total, 8 up, 8 in 2024-04-07T17:18:47.368 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:18:46 smithi144 bash[66654]: audit 2024-04-07T17:18:46.836528+0000 mon.a (mon.0) 3438 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:18:47.368 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:18:46 smithi144 bash[66654]: audit 2024-04-07T17:18:46.839960+0000 mon.a (mon.0) 3439 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:18:47.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:18:46 smithi100 bash[76202]: cluster 2024-04-07T17:18:45.871459+0000 mgr.y (mgr.25175) 1902 : cluster [DBG] pgmap v3258: 745 pgs: 4 creating+peering, 28 unknown, 713 active+clean; 6.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail 2024-04-07T17:18:47.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:18:46 smithi100 bash[76202]: cluster 2024-04-07T17:18:45.968434+0000 mon.a (mon.0) 3435 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T17:18:47.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:18:46 smithi100 bash[76202]: audit 2024-04-07T17:18:45.978236+0000 mon.a (mon.0) 3436 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T17:18:47.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:18:46 smithi100 bash[76202]: cluster 2024-04-07T17:18:45.989380+0000 mon.a (mon.0) 3437 : cluster [DBG] osdmap e1782: 8 total, 8 up, 8 in 2024-04-07T17:18:47.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:18:46 smithi100 bash[76202]: audit 2024-04-07T17:18:46.836528+0000 mon.a (mon.0) 3438 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:18:47.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:18:46 smithi100 bash[76202]: audit 2024-04-07T17:18:46.839960+0000 mon.a (mon.0) 3439 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:18:47.455 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:18:46 smithi100 bash[74942]: cluster 2024-04-07T17:18:45.871459+0000 mgr.y (mgr.25175) 1902 : cluster [DBG] pgmap v3258: 745 pgs: 4 creating+peering, 28 unknown, 713 active+clean; 6.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail 2024-04-07T17:18:47.456 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:18:46 smithi100 bash[74942]: cluster 2024-04-07T17:18:45.968434+0000 mon.a (mon.0) 3435 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T17:18:47.456 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:18:46 smithi100 bash[74942]: audit 2024-04-07T17:18:45.978236+0000 mon.a (mon.0) 3436 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T17:18:47.456 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:18:46 smithi100 bash[74942]: cluster 2024-04-07T17:18:45.989380+0000 mon.a (mon.0) 3437 : cluster [DBG] osdmap e1782: 8 total, 8 up, 8 in 2024-04-07T17:18:47.456 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:18:46 smithi100 bash[74942]: audit 2024-04-07T17:18:46.836528+0000 mon.a (mon.0) 3438 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:18:47.456 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:18:46 smithi100 bash[74942]: audit 2024-04-07T17:18:46.839960+0000 mon.a (mon.0) 3439 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:18:48.368 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:18:48 smithi144 bash[66654]: cluster 2024-04-07T17:18:46.989864+0000 mon.a (mon.0) 3440 : cluster [DBG] osdmap e1783: 8 total, 8 up, 8 in 2024-04-07T17:18:48.368 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:18:48 smithi144 bash[66654]: cluster 2024-04-07T17:18:47.912562+0000 mon.a (mon.0) 3441 : cluster [DBG] osdmap e1784: 8 total, 8 up, 8 in 2024-04-07T17:18:48.429 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:18:48 smithi100 bash[74942]: cluster 2024-04-07T17:18:46.989864+0000 mon.a (mon.0) 3440 : cluster [DBG] osdmap e1783: 8 total, 8 up, 8 in 2024-04-07T17:18:48.429 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:18:48 smithi100 bash[74942]: cluster 2024-04-07T17:18:47.912562+0000 mon.a (mon.0) 3441 : cluster [DBG] osdmap e1784: 8 total, 8 up, 8 in 2024-04-07T17:18:48.429 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:18:48 smithi100 bash[76202]: cluster 2024-04-07T17:18:46.989864+0000 mon.a (mon.0) 3440 : cluster [DBG] osdmap e1783: 8 total, 8 up, 8 in 2024-04-07T17:18:48.429 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:18:48 smithi100 bash[76202]: cluster 2024-04-07T17:18:47.912562+0000 mon.a (mon.0) 3441 : cluster [DBG] osdmap e1784: 8 total, 8 up, 8 in 2024-04-07T17:18:48.704 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:18:48 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:18:48] "GET /metrics HTTP/1.1" 200 82985 "" "Prometheus/2.43.0" 2024-04-07T17:18:49.368 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:18:49 smithi144 bash[66654]: cluster 2024-04-07T17:18:47.875446+0000 mgr.y (mgr.25175) 1903 : cluster [DBG] pgmap v3261: 745 pgs: 4 creating+peering, 9 unknown, 732 active+clean; 6.6 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 47 KiB/s rd, 3.2 KiB/s wr, 64 op/s 2024-04-07T17:18:49.368 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:18:49 smithi144 bash[66654]: cluster 2024-04-07T17:18:47.994758+0000 mon.a (mon.0) 3442 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T17:18:49.455 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:18:49 smithi100 bash[74942]: cluster 2024-04-07T17:18:47.875446+0000 mgr.y (mgr.25175) 1903 : cluster [DBG] pgmap v3261: 745 pgs: 4 creating+peering, 9 unknown, 732 active+clean; 6.6 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 47 KiB/s rd, 3.2 KiB/s wr, 64 op/s 2024-04-07T17:18:49.455 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:18:49 smithi100 bash[74942]: cluster 2024-04-07T17:18:47.994758+0000 mon.a (mon.0) 3442 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T17:18:49.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:18:49 smithi100 bash[76202]: cluster 2024-04-07T17:18:47.875446+0000 mgr.y (mgr.25175) 1903 : cluster [DBG] pgmap v3261: 745 pgs: 4 creating+peering, 9 unknown, 732 active+clean; 6.6 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 47 KiB/s rd, 3.2 KiB/s wr, 64 op/s 2024-04-07T17:18:49.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:18:49 smithi100 bash[76202]: cluster 2024-04-07T17:18:47.994758+0000 mon.a (mon.0) 3442 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T17:18:50.369 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:18:50 smithi144 bash[66654]: cluster 2024-04-07T17:18:49.028785+0000 mon.a (mon.0) 3443 : cluster [DBG] osdmap e1785: 8 total, 8 up, 8 in 2024-04-07T17:18:50.369 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:18:50 smithi144 bash[66654]: cluster 2024-04-07T17:18:49.877947+0000 mgr.y (mgr.25175) 1904 : cluster [DBG] pgmap v3264: 745 pgs: 4 creating+peering, 741 active+clean; 6.6 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 60 KiB/s rd, 2.7 KiB/s wr, 78 op/s 2024-04-07T17:18:50.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:18:50 smithi100 bash[76202]: cluster 2024-04-07T17:18:49.028785+0000 mon.a (mon.0) 3443 : cluster [DBG] osdmap e1785: 8 total, 8 up, 8 in 2024-04-07T17:18:50.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:18:50 smithi100 bash[76202]: cluster 2024-04-07T17:18:49.877947+0000 mgr.y (mgr.25175) 1904 : cluster [DBG] pgmap v3264: 745 pgs: 4 creating+peering, 741 active+clean; 6.6 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 60 KiB/s rd, 2.7 KiB/s wr, 78 op/s 2024-04-07T17:18:50.455 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:18:50 smithi100 bash[74942]: cluster 2024-04-07T17:18:49.028785+0000 mon.a (mon.0) 3443 : cluster [DBG] osdmap e1785: 8 total, 8 up, 8 in 2024-04-07T17:18:50.455 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:18:50 smithi100 bash[74942]: cluster 2024-04-07T17:18:49.877947+0000 mgr.y (mgr.25175) 1904 : cluster [DBG] pgmap v3264: 745 pgs: 4 creating+peering, 741 active+clean; 6.6 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 60 KiB/s rd, 2.7 KiB/s wr, 78 op/s 2024-04-07T17:18:51.368 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:18:51 smithi144 bash[66654]: cluster 2024-04-07T17:18:50.041690+0000 mon.a (mon.0) 3444 : cluster [DBG] osdmap e1786: 8 total, 8 up, 8 in 2024-04-07T17:18:51.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:18:51 smithi100 bash[76202]: cluster 2024-04-07T17:18:50.041690+0000 mon.a (mon.0) 3444 : cluster [DBG] osdmap e1786: 8 total, 8 up, 8 in 2024-04-07T17:18:51.455 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:18:51 smithi100 bash[74942]: cluster 2024-04-07T17:18:50.041690+0000 mon.a (mon.0) 3444 : cluster [DBG] osdmap e1786: 8 total, 8 up, 8 in 2024-04-07T17:18:52.318 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:18:52 smithi144 bash[66654]: cluster 2024-04-07T17:18:51.045263+0000 mon.a (mon.0) 3445 : cluster [DBG] osdmap e1787: 8 total, 8 up, 8 in 2024-04-07T17:18:52.318 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:18:52 smithi144 bash[66654]: cluster 2024-04-07T17:18:51.879363+0000 mgr.y (mgr.25175) 1905 : cluster [DBG] pgmap v3267: 745 pgs: 745 active+clean; 6.6 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 24 KiB/s rd, 1.2 KiB/s wr, 29 op/s 2024-04-07T17:18:52.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:18:52 smithi100 bash[74942]: cluster 2024-04-07T17:18:51.045263+0000 mon.a (mon.0) 3445 : cluster [DBG] osdmap e1787: 8 total, 8 up, 8 in 2024-04-07T17:18:52.455 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:18:52 smithi100 bash[74942]: cluster 2024-04-07T17:18:51.879363+0000 mgr.y (mgr.25175) 1905 : cluster [DBG] pgmap v3267: 745 pgs: 745 active+clean; 6.6 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 24 KiB/s rd, 1.2 KiB/s wr, 29 op/s 2024-04-07T17:18:52.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:18:52 smithi100 bash[76202]: cluster 2024-04-07T17:18:51.045263+0000 mon.a (mon.0) 3445 : cluster [DBG] osdmap e1787: 8 total, 8 up, 8 in 2024-04-07T17:18:52.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:18:52 smithi100 bash[76202]: cluster 2024-04-07T17:18:51.879363+0000 mgr.y (mgr.25175) 1905 : cluster [DBG] pgmap v3267: 745 pgs: 745 active+clean; 6.6 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 24 KiB/s rd, 1.2 KiB/s wr, 29 op/s 2024-04-07T17:18:52.618 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:18:52 smithi144 bash[95233]: ts=2024-04-07T17:18:52.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:18:53.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:18:53 smithi144 bash[66654]: cluster 2024-04-07T17:18:52.074731+0000 mon.a (mon.0) 3446 : cluster [DBG] osdmap e1788: 8 total, 8 up, 8 in 2024-04-07T17:18:53.368 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:18:53 smithi144 bash[66654]: cluster 2024-04-07T17:18:52.916399+0000 mon.a (mon.0) 3447 : cluster [DBG] osdmap e1789: 8 total, 8 up, 8 in 2024-04-07T17:18:53.455 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:18:53 smithi100 bash[74942]: cluster 2024-04-07T17:18:52.074731+0000 mon.a (mon.0) 3446 : cluster [DBG] osdmap e1788: 8 total, 8 up, 8 in 2024-04-07T17:18:53.455 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:18:53 smithi100 bash[74942]: cluster 2024-04-07T17:18:52.916399+0000 mon.a (mon.0) 3447 : cluster [DBG] osdmap e1789: 8 total, 8 up, 8 in 2024-04-07T17:18:53.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:18:53 smithi100 bash[76202]: cluster 2024-04-07T17:18:52.074731+0000 mon.a (mon.0) 3446 : cluster [DBG] osdmap e1788: 8 total, 8 up, 8 in 2024-04-07T17:18:53.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:18:53 smithi100 bash[76202]: cluster 2024-04-07T17:18:52.916399+0000 mon.a (mon.0) 3447 : cluster [DBG] osdmap e1789: 8 total, 8 up, 8 in 2024-04-07T17:18:54.111 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.QuiesceWatch (10049 ms) 2024-04-07T17:18:54.111 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.QuiesceWatchPP 2024-04-07T17:18:54.112 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:18:55.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:18:54 smithi100 bash[74942]: cluster 2024-04-07T17:18:53.881348+0000 mgr.y (mgr.25175) 1906 : cluster [DBG] pgmap v3270: 745 pgs: 745 active+clean; 6.6 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 4.7 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-04-07T17:18:55.205 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:18:54 smithi100 bash[74942]: cluster 2024-04-07T17:18:53.921565+0000 mon.a (mon.0) 3448 : cluster [DBG] osdmap e1790: 8 total, 8 up, 8 in 2024-04-07T17:18:55.205 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:18:54 smithi100 bash[76202]: cluster 2024-04-07T17:18:53.881348+0000 mgr.y (mgr.25175) 1906 : cluster [DBG] pgmap v3270: 745 pgs: 745 active+clean; 6.6 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 4.7 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-04-07T17:18:55.205 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:18:54 smithi100 bash[76202]: cluster 2024-04-07T17:18:53.921565+0000 mon.a (mon.0) 3448 : cluster [DBG] osdmap e1790: 8 total, 8 up, 8 in 2024-04-07T17:18:55.368 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:18:54 smithi144 bash[66654]: cluster 2024-04-07T17:18:53.881348+0000 mgr.y (mgr.25175) 1906 : cluster [DBG] pgmap v3270: 745 pgs: 745 active+clean; 6.6 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 4.7 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-04-07T17:18:55.368 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:18:54 smithi144 bash[66654]: cluster 2024-04-07T17:18:53.921565+0000 mon.a (mon.0) 3448 : cluster [DBG] osdmap e1790: 8 total, 8 up, 8 in 2024-04-07T17:18:56.368 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:18:56 smithi144 bash[66654]: cluster 2024-04-07T17:18:55.082178+0000 mon.a (mon.0) 3449 : cluster [DBG] osdmap e1791: 8 total, 8 up, 8 in 2024-04-07T17:18:56.368 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:18:56 smithi144 bash[66654]: cluster 2024-04-07T17:18:55.882602+0000 mgr.y (mgr.25175) 1907 : cluster [DBG] pgmap v3273: 745 pgs: 745 active+clean; 6.6 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 60 KiB/s rd, 5.5 KiB/s wr, 81 op/s 2024-04-07T17:18:56.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:18:56 smithi100 bash[74942]: cluster 2024-04-07T17:18:55.082178+0000 mon.a (mon.0) 3449 : cluster [DBG] osdmap e1791: 8 total, 8 up, 8 in 2024-04-07T17:18:56.455 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:18:56 smithi100 bash[74942]: cluster 2024-04-07T17:18:55.882602+0000 mgr.y (mgr.25175) 1907 : cluster [DBG] pgmap v3273: 745 pgs: 745 active+clean; 6.6 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 60 KiB/s rd, 5.5 KiB/s wr, 81 op/s 2024-04-07T17:18:56.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:18:56 smithi100 bash[76202]: cluster 2024-04-07T17:18:55.082178+0000 mon.a (mon.0) 3449 : cluster [DBG] osdmap e1791: 8 total, 8 up, 8 in 2024-04-07T17:18:56.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:18:56 smithi100 bash[76202]: cluster 2024-04-07T17:18:55.882602+0000 mgr.y (mgr.25175) 1907 : cluster [DBG] pgmap v3273: 745 pgs: 745 active+clean; 6.6 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 60 KiB/s rd, 5.5 KiB/s wr, 81 op/s 2024-04-07T17:18:57.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:18:57 smithi144 bash[66654]: cluster 2024-04-07T17:18:56.096992+0000 mon.a (mon.0) 3450 : cluster [DBG] osdmap e1792: 8 total, 8 up, 8 in 2024-04-07T17:18:57.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:18:57 smithi100 bash[74942]: cluster 2024-04-07T17:18:56.096992+0000 mon.a (mon.0) 3450 : cluster [DBG] osdmap e1792: 8 total, 8 up, 8 in 2024-04-07T17:18:57.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:18:57 smithi100 bash[76202]: cluster 2024-04-07T17:18:56.096992+0000 mon.a (mon.0) 3450 : cluster [DBG] osdmap e1792: 8 total, 8 up, 8 in 2024-04-07T17:18:58.368 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:18:58 smithi144 bash[66654]: cluster 2024-04-07T17:18:57.096309+0000 mon.a (mon.0) 3451 : cluster [DBG] osdmap e1793: 8 total, 8 up, 8 in 2024-04-07T17:18:58.368 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:18:58 smithi144 bash[66654]: cluster 2024-04-07T17:18:57.885618+0000 mgr.y (mgr.25175) 1908 : cluster [DBG] pgmap v3276: 745 pgs: 745 active+clean; 6.6 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 95 KiB/s rd, 7.5 KiB/s wr, 128 op/s 2024-04-07T17:18:58.368 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:18:58 smithi144 bash[66654]: cluster 2024-04-07T17:18:57.917894+0000 mon.a (mon.0) 3452 : cluster [DBG] osdmap e1794: 8 total, 8 up, 8 in 2024-04-07T17:18:58.429 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:18:58 smithi100 bash[74942]: cluster 2024-04-07T17:18:57.096309+0000 mon.a (mon.0) 3451 : cluster [DBG] osdmap e1793: 8 total, 8 up, 8 in 2024-04-07T17:18:58.429 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:18:58 smithi100 bash[74942]: cluster 2024-04-07T17:18:57.885618+0000 mgr.y (mgr.25175) 1908 : cluster [DBG] pgmap v3276: 745 pgs: 745 active+clean; 6.6 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 95 KiB/s rd, 7.5 KiB/s wr, 128 op/s 2024-04-07T17:18:58.429 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:18:58 smithi100 bash[74942]: cluster 2024-04-07T17:18:57.917894+0000 mon.a (mon.0) 3452 : cluster [DBG] osdmap e1794: 8 total, 8 up, 8 in 2024-04-07T17:18:58.430 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:18:58 smithi100 bash[76202]: cluster 2024-04-07T17:18:57.096309+0000 mon.a (mon.0) 3451 : cluster [DBG] osdmap e1793: 8 total, 8 up, 8 in 2024-04-07T17:18:58.430 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:18:58 smithi100 bash[76202]: cluster 2024-04-07T17:18:57.885618+0000 mgr.y (mgr.25175) 1908 : cluster [DBG] pgmap v3276: 745 pgs: 745 active+clean; 6.6 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 95 KiB/s rd, 7.5 KiB/s wr, 128 op/s 2024-04-07T17:18:58.430 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:18:58 smithi100 bash[76202]: cluster 2024-04-07T17:18:57.917894+0000 mon.a (mon.0) 3452 : cluster [DBG] osdmap e1794: 8 total, 8 up, 8 in 2024-04-07T17:18:58.704 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:18:58 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:18:58] "GET /metrics HTTP/1.1" 200 82985 "" "Prometheus/2.43.0" 2024-04-07T17:19:00.120 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.QuiesceWatchPP (6009 ms) 2024-04-07T17:19:00.120 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.QuiesceWatchError 2024-04-07T17:19:00.120 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:19:00.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:18:59 smithi100 bash[74942]: cluster 2024-04-07T17:18:58.921473+0000 mon.a (mon.0) 3453 : cluster [DBG] osdmap e1795: 8 total, 8 up, 8 in 2024-04-07T17:19:00.205 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:18:59 smithi100 bash[76202]: cluster 2024-04-07T17:18:58.921473+0000 mon.a (mon.0) 3453 : cluster [DBG] osdmap e1795: 8 total, 8 up, 8 in 2024-04-07T17:19:00.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:18:59 smithi144 bash[66654]: cluster 2024-04-07T17:18:58.921473+0000 mon.a (mon.0) 3453 : cluster [DBG] osdmap e1795: 8 total, 8 up, 8 in 2024-04-07T17:19:01.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:19:00 smithi144 bash[66654]: cluster 2024-04-07T17:18:59.888484+0000 mgr.y (mgr.25175) 1909 : cluster [DBG] pgmap v3279: 745 pgs: 745 active+clean; 6.6 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 38 KiB/s rd, 3.2 KiB/s wr, 55 op/s 2024-04-07T17:19:01.368 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:19:00 smithi144 bash[66654]: cluster 2024-04-07T17:18:59.923681+0000 mon.a (mon.0) 3454 : cluster [DBG] osdmap e1796: 8 total, 8 up, 8 in 2024-04-07T17:19:01.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:19:00 smithi100 bash[74942]: cluster 2024-04-07T17:18:59.888484+0000 mgr.y (mgr.25175) 1909 : cluster [DBG] pgmap v3279: 745 pgs: 745 active+clean; 6.6 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 38 KiB/s rd, 3.2 KiB/s wr, 55 op/s 2024-04-07T17:19:01.455 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:19:00 smithi100 bash[74942]: cluster 2024-04-07T17:18:59.923681+0000 mon.a (mon.0) 3454 : cluster [DBG] osdmap e1796: 8 total, 8 up, 8 in 2024-04-07T17:19:01.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:19:00 smithi100 bash[76202]: cluster 2024-04-07T17:18:59.888484+0000 mgr.y (mgr.25175) 1909 : cluster [DBG] pgmap v3279: 745 pgs: 745 active+clean; 6.6 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 38 KiB/s rd, 3.2 KiB/s wr, 55 op/s 2024-04-07T17:19:01.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:19:00 smithi100 bash[76202]: cluster 2024-04-07T17:18:59.923681+0000 mon.a (mon.0) 3454 : cluster [DBG] osdmap e1796: 8 total, 8 up, 8 in 2024-04-07T17:19:02.318 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:19:01 smithi144 bash[66654]: cluster 2024-04-07T17:19:00.951351+0000 mon.a (mon.0) 3455 : cluster [DBG] osdmap e1797: 8 total, 8 up, 8 in 2024-04-07T17:19:02.318 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:19:01 smithi144 bash[66654]: audit 2024-04-07T17:19:01.836712+0000 mon.a (mon.0) 3456 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:19:02.319 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:19:01 smithi144 bash[66654]: audit 2024-04-07T17:19:01.840101+0000 mon.a (mon.0) 3457 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:19:02.455 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:19:01 smithi100 bash[74942]: cluster 2024-04-07T17:19:00.951351+0000 mon.a (mon.0) 3455 : cluster [DBG] osdmap e1797: 8 total, 8 up, 8 in 2024-04-07T17:19:02.455 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:19:01 smithi100 bash[74942]: audit 2024-04-07T17:19:01.836712+0000 mon.a (mon.0) 3456 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:19:02.455 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:19:01 smithi100 bash[74942]: audit 2024-04-07T17:19:01.840101+0000 mon.a (mon.0) 3457 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:19:02.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:19:01 smithi100 bash[76202]: cluster 2024-04-07T17:19:00.951351+0000 mon.a (mon.0) 3455 : cluster [DBG] osdmap e1797: 8 total, 8 up, 8 in 2024-04-07T17:19:02.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:19:01 smithi100 bash[76202]: audit 2024-04-07T17:19:01.836712+0000 mon.a (mon.0) 3456 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:19:02.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:19:01 smithi100 bash[76202]: audit 2024-04-07T17:19:01.840101+0000 mon.a (mon.0) 3457 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:19:02.617 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:19:02 smithi144 bash[95233]: ts=2024-04-07T17:19:02.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:19:03.368 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:19:02 smithi144 bash[66654]: cluster 2024-04-07T17:19:01.952656+0000 mgr.y (mgr.25175) 1910 : cluster [DBG] pgmap v3282: 745 pgs: 745 active+clean; 6.6 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-04-07T17:19:03.368 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:19:02 smithi144 bash[66654]: cluster 2024-04-07T17:19:01.970203+0000 mon.a (mon.0) 3458 : cluster [DBG] osdmap e1798: 8 total, 8 up, 8 in 2024-04-07T17:19:03.368 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:19:02 smithi144 bash[66654]: cluster 2024-04-07T17:19:02.922478+0000 mon.a (mon.0) 3459 : cluster [DBG] osdmap e1799: 8 total, 8 up, 8 in 2024-04-07T17:19:03.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:19:02 smithi100 bash[74942]: cluster 2024-04-07T17:19:01.952656+0000 mgr.y (mgr.25175) 1910 : cluster [DBG] pgmap v3282: 745 pgs: 745 active+clean; 6.6 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-04-07T17:19:03.455 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:19:02 smithi100 bash[74942]: cluster 2024-04-07T17:19:01.970203+0000 mon.a (mon.0) 3458 : cluster [DBG] osdmap e1798: 8 total, 8 up, 8 in 2024-04-07T17:19:03.455 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:19:02 smithi100 bash[74942]: cluster 2024-04-07T17:19:02.922478+0000 mon.a (mon.0) 3459 : cluster [DBG] osdmap e1799: 8 total, 8 up, 8 in 2024-04-07T17:19:03.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:19:02 smithi100 bash[76202]: cluster 2024-04-07T17:19:01.952656+0000 mgr.y (mgr.25175) 1910 : cluster [DBG] pgmap v3282: 745 pgs: 745 active+clean; 6.6 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-04-07T17:19:03.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:19:02 smithi100 bash[76202]: cluster 2024-04-07T17:19:01.970203+0000 mon.a (mon.0) 3458 : cluster [DBG] osdmap e1798: 8 total, 8 up, 8 in 2024-04-07T17:19:03.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:19:02 smithi100 bash[76202]: cluster 2024-04-07T17:19:02.922478+0000 mon.a (mon.0) 3459 : cluster [DBG] osdmap e1799: 8 total, 8 up, 8 in 2024-04-07T17:19:05.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:19:04 smithi144 bash[66654]: cluster 2024-04-07T17:19:03.955292+0000 mgr.y (mgr.25175) 1911 : cluster [DBG] pgmap v3285: 745 pgs: 745 active+clean; 6.6 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 98 KiB/s rd, 8.1 KiB/s wr, 134 op/s 2024-04-07T17:19:05.368 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:19:04 smithi144 bash[66654]: cluster 2024-04-07T17:19:03.991550+0000 mon.a (mon.0) 3460 : cluster [DBG] osdmap e1800: 8 total, 8 up, 8 in 2024-04-07T17:19:05.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:19:04 smithi100 bash[74942]: cluster 2024-04-07T17:19:03.955292+0000 mgr.y (mgr.25175) 1911 : cluster [DBG] pgmap v3285: 745 pgs: 745 active+clean; 6.6 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 98 KiB/s rd, 8.1 KiB/s wr, 134 op/s 2024-04-07T17:19:05.455 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:19:04 smithi100 bash[74942]: cluster 2024-04-07T17:19:03.991550+0000 mon.a (mon.0) 3460 : cluster [DBG] osdmap e1800: 8 total, 8 up, 8 in 2024-04-07T17:19:05.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:19:04 smithi100 bash[76202]: cluster 2024-04-07T17:19:03.955292+0000 mgr.y (mgr.25175) 1911 : cluster [DBG] pgmap v3285: 745 pgs: 745 active+clean; 6.6 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 98 KiB/s rd, 8.1 KiB/s wr, 134 op/s 2024-04-07T17:19:05.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:19:04 smithi100 bash[76202]: cluster 2024-04-07T17:19:03.991550+0000 mon.a (mon.0) 3460 : cluster [DBG] osdmap e1800: 8 total, 8 up, 8 in 2024-04-07T17:19:06.221 INFO:tasks.workunit.client.0.smithi100.stdout:10.00% done 2024-04-07T17:19:06.221 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.QuiesceWatchError (6101 ms) 2024-04-07T17:19:06.221 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.QuiesceWatchTimeout 2024-04-07T17:19:06.221 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:19:06.257 INFO:tasks.workunit.client.0.smithi100.stderr:test quiesce is not long enough to time out 2024-04-07T17:19:06.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:19:06 smithi144 bash[66654]: cluster 2024-04-07T17:19:05.015115+0000 mon.a (mon.0) 3461 : cluster [DBG] osdmap e1801: 8 total, 8 up, 8 in 2024-04-07T17:19:06.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:19:06 smithi144 bash[66654]: cluster 2024-04-07T17:19:05.957471+0000 mgr.y (mgr.25175) 1912 : cluster [DBG] pgmap v3288: 745 pgs: 745 active+clean; 6.6 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 101 KiB/s rd, 8.2 KiB/s wr, 138 op/s 2024-04-07T17:19:06.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:19:06 smithi100 bash[74942]: cluster 2024-04-07T17:19:05.015115+0000 mon.a (mon.0) 3461 : cluster [DBG] osdmap e1801: 8 total, 8 up, 8 in 2024-04-07T17:19:06.455 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:19:06 smithi100 bash[74942]: cluster 2024-04-07T17:19:05.957471+0000 mgr.y (mgr.25175) 1912 : cluster [DBG] pgmap v3288: 745 pgs: 745 active+clean; 6.6 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 101 KiB/s rd, 8.2 KiB/s wr, 138 op/s 2024-04-07T17:19:06.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:19:06 smithi100 bash[76202]: cluster 2024-04-07T17:19:05.015115+0000 mon.a (mon.0) 3461 : cluster [DBG] osdmap e1801: 8 total, 8 up, 8 in 2024-04-07T17:19:06.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:19:06 smithi100 bash[76202]: cluster 2024-04-07T17:19:05.957471+0000 mgr.y (mgr.25175) 1912 : cluster [DBG] pgmap v3288: 745 pgs: 745 active+clean; 6.6 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 101 KiB/s rd, 8.2 KiB/s wr, 138 op/s 2024-04-07T17:19:07.368 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:19:07 smithi144 bash[66654]: cluster 2024-04-07T17:19:06.027238+0000 mon.a (mon.0) 3462 : cluster [DBG] osdmap e1802: 8 total, 8 up, 8 in 2024-04-07T17:19:07.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:19:07 smithi100 bash[74942]: cluster 2024-04-07T17:19:06.027238+0000 mon.a (mon.0) 3462 : cluster [DBG] osdmap e1802: 8 total, 8 up, 8 in 2024-04-07T17:19:07.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:19:07 smithi100 bash[76202]: cluster 2024-04-07T17:19:06.027238+0000 mon.a (mon.0) 3462 : cluster [DBG] osdmap e1802: 8 total, 8 up, 8 in 2024-04-07T17:19:08.704 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:19:08 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:19:08] "GET /metrics HTTP/1.1" 200 83017 "" "Prometheus/2.43.0" 2024-04-07T17:19:09.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:19:08 smithi100 bash[74942]: cluster 2024-04-07T17:19:07.924245+0000 mon.a (mon.0) 3463 : cluster [DBG] osdmap e1803: 8 total, 8 up, 8 in 2024-04-07T17:19:09.205 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:19:08 smithi100 bash[74942]: cluster 2024-04-07T17:19:07.958501+0000 mgr.y (mgr.25175) 1913 : cluster [DBG] pgmap v3291: 745 pgs: 745 active+clean; 6.6 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 12 KiB/s rd, 1.5 KiB/s wr, 16 op/s 2024-04-07T17:19:09.205 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:19:08 smithi100 bash[76202]: cluster 2024-04-07T17:19:07.924245+0000 mon.a (mon.0) 3463 : cluster [DBG] osdmap e1803: 8 total, 8 up, 8 in 2024-04-07T17:19:09.205 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:19:08 smithi100 bash[76202]: cluster 2024-04-07T17:19:07.958501+0000 mgr.y (mgr.25175) 1913 : cluster [DBG] pgmap v3291: 745 pgs: 745 active+clean; 6.6 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 12 KiB/s rd, 1.5 KiB/s wr, 16 op/s 2024-04-07T17:19:09.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:19:08 smithi144 bash[66654]: cluster 2024-04-07T17:19:07.924245+0000 mon.a (mon.0) 3463 : cluster [DBG] osdmap e1803: 8 total, 8 up, 8 in 2024-04-07T17:19:09.368 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:19:08 smithi144 bash[66654]: cluster 2024-04-07T17:19:07.958501+0000 mgr.y (mgr.25175) 1913 : cluster [DBG] pgmap v3291: 745 pgs: 745 active+clean; 6.6 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 12 KiB/s rd, 1.5 KiB/s wr, 16 op/s 2024-04-07T17:19:10.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:19:10 smithi144 bash[66654]: cluster 2024-04-07T17:19:09.962207+0000 mgr.y (mgr.25175) 1914 : cluster [DBG] pgmap v3292: 745 pgs: 745 active+clean; 6.6 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 35 KiB/s rd, 2.8 KiB/s wr, 49 op/s 2024-04-07T17:19:10.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:19:10 smithi100 bash[74942]: cluster 2024-04-07T17:19:09.962207+0000 mgr.y (mgr.25175) 1914 : cluster [DBG] pgmap v3292: 745 pgs: 745 active+clean; 6.6 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 35 KiB/s rd, 2.8 KiB/s wr, 49 op/s 2024-04-07T17:19:10.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:19:10 smithi100 bash[76202]: cluster 2024-04-07T17:19:09.962207+0000 mgr.y (mgr.25175) 1914 : cluster [DBG] pgmap v3292: 745 pgs: 745 active+clean; 6.6 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 35 KiB/s rd, 2.8 KiB/s wr, 49 op/s 2024-04-07T17:19:12.317 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:19:12 smithi144 bash[66654]: cluster 2024-04-07T17:19:11.964543+0000 mgr.y (mgr.25175) 1915 : cluster [DBG] pgmap v3293: 745 pgs: 745 active+clean; 6.6 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 31 KiB/s rd, 2.4 KiB/s wr, 43 op/s 2024-04-07T17:19:12.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:19:12 smithi100 bash[74942]: cluster 2024-04-07T17:19:11.964543+0000 mgr.y (mgr.25175) 1915 : cluster [DBG] pgmap v3293: 745 pgs: 745 active+clean; 6.6 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 31 KiB/s rd, 2.4 KiB/s wr, 43 op/s 2024-04-07T17:19:12.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:19:12 smithi100 bash[76202]: cluster 2024-04-07T17:19:11.964543+0000 mgr.y (mgr.25175) 1915 : cluster [DBG] pgmap v3293: 745 pgs: 745 active+clean; 6.6 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 31 KiB/s rd, 2.4 KiB/s wr, 43 op/s 2024-04-07T17:19:12.618 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:19:12 smithi144 bash[95233]: ts=2024-04-07T17:19:12.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:19:14.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:19:13 smithi100 bash[74942]: cluster 2024-04-07T17:19:12.930998+0000 mon.a (mon.0) 3464 : cluster [DBG] osdmap e1804: 8 total, 8 up, 8 in 2024-04-07T17:19:14.205 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:19:13 smithi100 bash[76202]: cluster 2024-04-07T17:19:12.930998+0000 mon.a (mon.0) 3464 : cluster [DBG] osdmap e1804: 8 total, 8 up, 8 in 2024-04-07T17:19:14.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:19:13 smithi144 bash[66654]: cluster 2024-04-07T17:19:12.930998+0000 mon.a (mon.0) 3464 : cluster [DBG] osdmap e1804: 8 total, 8 up, 8 in 2024-04-07T17:19:14.994 INFO:tasks.workunit.client.0.smithi100.stderr:test quiesce is timed out 2024-04-07T17:19:15.004 INFO:tasks.workunit.client.0.smithi100.stderr:waiting for timed out ... 0 2024-04-07T17:19:15.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:19:14 smithi100 bash[74942]: cluster 2024-04-07T17:19:13.966687+0000 mgr.y (mgr.25175) 1916 : cluster [DBG] pgmap v3295: 745 pgs: 745 active+clean; 6.6 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 26 KiB/s rd, 2.1 KiB/s wr, 36 op/s 2024-04-07T17:19:15.205 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:19:14 smithi100 bash[76202]: cluster 2024-04-07T17:19:13.966687+0000 mgr.y (mgr.25175) 1916 : cluster [DBG] pgmap v3295: 745 pgs: 745 active+clean; 6.6 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 26 KiB/s rd, 2.1 KiB/s wr, 36 op/s 2024-04-07T17:19:15.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:19:14 smithi144 bash[66654]: cluster 2024-04-07T17:19:13.966687+0000 mgr.y (mgr.25175) 1916 : cluster [DBG] pgmap v3295: 745 pgs: 745 active+clean; 6.6 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 26 KiB/s rd, 2.1 KiB/s wr, 36 op/s 2024-04-07T17:19:16.005 INFO:tasks.workunit.client.0.smithi100.stderr:waiting for timed out ... 1 2024-04-07T17:19:16.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:19:15 smithi100 bash[74942]: cluster 2024-04-07T17:19:14.940286+0000 mon.a (mon.0) 3465 : cluster [DBG] osdmap e1805: 8 total, 8 up, 8 in 2024-04-07T17:19:16.205 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:19:15 smithi100 bash[76202]: cluster 2024-04-07T17:19:14.940286+0000 mon.a (mon.0) 3465 : cluster [DBG] osdmap e1805: 8 total, 8 up, 8 in 2024-04-07T17:19:16.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:19:15 smithi144 bash[66654]: cluster 2024-04-07T17:19:14.940286+0000 mon.a (mon.0) 3465 : cluster [DBG] osdmap e1805: 8 total, 8 up, 8 in 2024-04-07T17:19:17.005 INFO:tasks.workunit.client.0.smithi100.stderr:waiting for timed out ... 2 2024-04-07T17:19:17.205 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:19:16 smithi100 bash[74942]: cluster 2024-04-07T17:19:15.968881+0000 mgr.y (mgr.25175) 1917 : cluster [DBG] pgmap v3297: 745 pgs: 745 active+clean; 6.6 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 21 KiB/s rd, 1.4 KiB/s wr, 29 op/s 2024-04-07T17:19:17.206 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:19:16 smithi100 bash[74942]: audit 2024-04-07T17:19:16.831637+0000 mon.a (mon.0) 3466 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:19:17.206 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:19:16 smithi100 bash[76202]: cluster 2024-04-07T17:19:15.968881+0000 mgr.y (mgr.25175) 1917 : cluster [DBG] pgmap v3297: 745 pgs: 745 active+clean; 6.6 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 21 KiB/s rd, 1.4 KiB/s wr, 29 op/s 2024-04-07T17:19:17.206 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:19:16 smithi100 bash[76202]: audit 2024-04-07T17:19:16.831637+0000 mon.a (mon.0) 3466 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:19:17.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:19:16 smithi144 bash[66654]: cluster 2024-04-07T17:19:15.968881+0000 mgr.y (mgr.25175) 1917 : cluster [DBG] pgmap v3297: 745 pgs: 745 active+clean; 6.6 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 21 KiB/s rd, 1.4 KiB/s wr, 29 op/s 2024-04-07T17:19:17.368 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:19:16 smithi144 bash[66654]: audit 2024-04-07T17:19:16.831637+0000 mon.a (mon.0) 3466 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:19:18.005 INFO:tasks.workunit.client.0.smithi100.stderr:waiting for timed out ... 3 2024-04-07T17:19:18.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:19:18 smithi144 bash[66654]: cluster 2024-04-07T17:19:17.972865+0000 mgr.y (mgr.25175) 1918 : cluster [DBG] pgmap v3298: 745 pgs: 745 active+clean; 6.6 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 4.2 KiB/s rd, 383 B/s wr, 5 op/s 2024-04-07T17:19:18.430 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:19:18 smithi100 bash[76202]: cluster 2024-04-07T17:19:17.972865+0000 mgr.y (mgr.25175) 1918 : cluster [DBG] pgmap v3298: 745 pgs: 745 active+clean; 6.6 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 4.2 KiB/s rd, 383 B/s wr, 5 op/s 2024-04-07T17:19:18.430 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:19:18 smithi100 bash[74942]: cluster 2024-04-07T17:19:17.972865+0000 mgr.y (mgr.25175) 1918 : cluster [DBG] pgmap v3298: 745 pgs: 745 active+clean; 6.6 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 4.2 KiB/s rd, 383 B/s wr, 5 op/s 2024-04-07T17:19:18.704 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:19:18 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:19:18] "GET /metrics HTTP/1.1" 200 83014 "" "Prometheus/2.43.0" 2024-04-07T17:19:19.005 INFO:tasks.workunit.client.0.smithi100.stderr:waiting for timed out ... 4 2024-04-07T17:19:20.005 INFO:tasks.workunit.client.0.smithi100.stderr:waiting for timed out ... 5 2024-04-07T17:19:20.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:19:20 smithi144 bash[66654]: cluster 2024-04-07T17:19:19.976308+0000 mgr.y (mgr.25175) 1919 : cluster [DBG] pgmap v3299: 745 pgs: 745 active+clean; 6.6 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 3.6 KiB/s rd, 383 B/s wr, 4 op/s 2024-04-07T17:19:20.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:19:20 smithi100 bash[74942]: cluster 2024-04-07T17:19:19.976308+0000 mgr.y (mgr.25175) 1919 : cluster [DBG] pgmap v3299: 745 pgs: 745 active+clean; 6.6 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 3.6 KiB/s rd, 383 B/s wr, 4 op/s 2024-04-07T17:19:20.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:19:20 smithi100 bash[76202]: cluster 2024-04-07T17:19:19.976308+0000 mgr.y (mgr.25175) 1919 : cluster [DBG] pgmap v3299: 745 pgs: 745 active+clean; 6.6 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 3.6 KiB/s rd, 383 B/s wr, 4 op/s 2024-04-07T17:19:21.005 INFO:tasks.workunit.client.0.smithi100.stderr:waiting for timed out ... 6 2024-04-07T17:19:22.005 INFO:tasks.workunit.client.0.smithi100.stderr:waiting for timed out ... 7 2024-04-07T17:19:22.317 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:19:22 smithi144 bash[66654]: cluster 2024-04-07T17:19:21.978311+0000 mgr.y (mgr.25175) 1920 : cluster [DBG] pgmap v3300: 745 pgs: 745 active+clean; 6.6 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 3.2 KiB/s rd, 339 B/s wr, 4 op/s 2024-04-07T17:19:22.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:19:22 smithi100 bash[74942]: cluster 2024-04-07T17:19:21.978311+0000 mgr.y (mgr.25175) 1920 : cluster [DBG] pgmap v3300: 745 pgs: 745 active+clean; 6.6 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 3.2 KiB/s rd, 339 B/s wr, 4 op/s 2024-04-07T17:19:22.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:19:22 smithi100 bash[76202]: cluster 2024-04-07T17:19:21.978311+0000 mgr.y (mgr.25175) 1920 : cluster [DBG] pgmap v3300: 745 pgs: 745 active+clean; 6.6 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 3.2 KiB/s rd, 339 B/s wr, 4 op/s 2024-04-07T17:19:22.618 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:19:22 smithi144 bash[95233]: ts=2024-04-07T17:19:22.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:19:23.006 INFO:tasks.workunit.client.0.smithi100.stderr:waiting for timed out ... 8 2024-04-07T17:19:23.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:19:23 smithi144 bash[66654]: audit 2024-04-07T17:19:22.495403+0000 mon.a (mon.0) 3467 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "109.d", "id": [3, 6]}]: dispatch 2024-04-07T17:19:23.368 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:19:23 smithi144 bash[66654]: audit 2024-04-07T17:19:22.495962+0000 mon.a (mon.0) 3468 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "109.1d", "id": [3, 0]}]: dispatch 2024-04-07T17:19:23.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:19:23 smithi100 bash[74942]: audit 2024-04-07T17:19:22.495403+0000 mon.a (mon.0) 3467 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "109.d", "id": [3, 6]}]: dispatch 2024-04-07T17:19:23.455 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:19:23 smithi100 bash[74942]: audit 2024-04-07T17:19:22.495962+0000 mon.a (mon.0) 3468 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "109.1d", "id": [3, 0]}]: dispatch 2024-04-07T17:19:23.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:19:23 smithi100 bash[76202]: audit 2024-04-07T17:19:22.495403+0000 mon.a (mon.0) 3467 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "109.d", "id": [3, 6]}]: dispatch 2024-04-07T17:19:23.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:19:23 smithi100 bash[76202]: audit 2024-04-07T17:19:22.495962+0000 mon.a (mon.0) 3468 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "109.1d", "id": [3, 0]}]: dispatch 2024-04-07T17:19:24.006 INFO:tasks.workunit.client.0.smithi100.stderr:waiting for timed out ... 9 2024-04-07T17:19:24.370 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:19:24 smithi144 bash[66654]: audit 2024-04-07T17:19:23.074063+0000 mon.a (mon.0) 3469 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "109.d", "id": [3, 6]}]': finished 2024-04-07T17:19:24.370 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:19:24 smithi144 bash[66654]: audit 2024-04-07T17:19:23.074231+0000 mon.a (mon.0) 3470 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "109.1d", "id": [3, 0]}]': finished 2024-04-07T17:19:24.370 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:19:24 smithi144 bash[66654]: cluster 2024-04-07T17:19:23.082756+0000 mon.a (mon.0) 3471 : cluster [DBG] osdmap e1806: 8 total, 8 up, 8 in 2024-04-07T17:19:24.370 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:19:24 smithi144 bash[66654]: cluster 2024-04-07T17:19:23.979593+0000 mgr.y (mgr.25175) 1921 : cluster [DBG] pgmap v3302: 745 pgs: 745 active+clean; 6.6 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 3.2 KiB/s rd, 339 B/s wr, 4 op/s 2024-04-07T17:19:24.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:19:24 smithi100 bash[76202]: audit 2024-04-07T17:19:23.074063+0000 mon.a (mon.0) 3469 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "109.d", "id": [3, 6]}]': finished 2024-04-07T17:19:24.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:19:24 smithi100 bash[76202]: audit 2024-04-07T17:19:23.074231+0000 mon.a (mon.0) 3470 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "109.1d", "id": [3, 0]}]': finished 2024-04-07T17:19:24.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:19:24 smithi100 bash[76202]: cluster 2024-04-07T17:19:23.082756+0000 mon.a (mon.0) 3471 : cluster [DBG] osdmap e1806: 8 total, 8 up, 8 in 2024-04-07T17:19:24.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:19:24 smithi100 bash[76202]: cluster 2024-04-07T17:19:23.979593+0000 mgr.y (mgr.25175) 1921 : cluster [DBG] pgmap v3302: 745 pgs: 745 active+clean; 6.6 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 3.2 KiB/s rd, 339 B/s wr, 4 op/s 2024-04-07T17:19:24.455 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:19:24 smithi100 bash[74942]: audit 2024-04-07T17:19:23.074063+0000 mon.a (mon.0) 3469 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "109.d", "id": [3, 6]}]': finished 2024-04-07T17:19:24.455 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:19:24 smithi100 bash[74942]: audit 2024-04-07T17:19:23.074231+0000 mon.a (mon.0) 3470 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "109.1d", "id": [3, 0]}]': finished 2024-04-07T17:19:24.455 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:19:24 smithi100 bash[74942]: cluster 2024-04-07T17:19:23.082756+0000 mon.a (mon.0) 3471 : cluster [DBG] osdmap e1806: 8 total, 8 up, 8 in 2024-04-07T17:19:24.455 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:19:24 smithi100 bash[74942]: cluster 2024-04-07T17:19:23.979593+0000 mgr.y (mgr.25175) 1921 : cluster [DBG] pgmap v3302: 745 pgs: 745 active+clean; 6.6 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 3.2 KiB/s rd, 339 B/s wr, 4 op/s 2024-04-07T17:19:25.006 INFO:tasks.workunit.client.0.smithi100.stderr:waiting for timed out ... 10 2024-04-07T17:19:25.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:19:25 smithi144 bash[66654]: cluster 2024-04-07T17:19:24.096346+0000 mon.a (mon.0) 3472 : cluster [DBG] osdmap e1807: 8 total, 8 up, 8 in 2024-04-07T17:19:25.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:19:25 smithi144 bash[66654]: audit 2024-04-07T17:19:24.407476+0000 mon.a (mon.0) 3473 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:19:25.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:19:25 smithi100 bash[74942]: cluster 2024-04-07T17:19:24.096346+0000 mon.a (mon.0) 3472 : cluster [DBG] osdmap e1807: 8 total, 8 up, 8 in 2024-04-07T17:19:25.455 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:19:25 smithi100 bash[74942]: audit 2024-04-07T17:19:24.407476+0000 mon.a (mon.0) 3473 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:19:25.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:19:25 smithi100 bash[76202]: cluster 2024-04-07T17:19:24.096346+0000 mon.a (mon.0) 3472 : cluster [DBG] osdmap e1807: 8 total, 8 up, 8 in 2024-04-07T17:19:25.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:19:25 smithi100 bash[76202]: audit 2024-04-07T17:19:24.407476+0000 mon.a (mon.0) 3473 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:19:26.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:19:26 smithi144 bash[66654]: audit 2024-04-07T17:19:25.159600+0000 mon.a (mon.0) 3474 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:19:26.368 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:19:26 smithi144 bash[66654]: audit 2024-04-07T17:19:25.161376+0000 mon.a (mon.0) 3475 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:19:26.368 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:19:26 smithi144 bash[66654]: audit 2024-04-07T17:19:25.172281+0000 mon.a (mon.0) 3476 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:19:26.368 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:19:26 smithi144 bash[66654]: cluster 2024-04-07T17:19:25.982391+0000 mgr.y (mgr.25175) 1922 : cluster [DBG] pgmap v3304: 745 pgs: 1 activating+degraded, 744 active+clean; 6.6 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 383 B/s rd, 0 op/s; 1/543 objects degraded (0.184%) 2024-04-07T17:19:26.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:19:26 smithi100 bash[74942]: audit 2024-04-07T17:19:25.159600+0000 mon.a (mon.0) 3474 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:19:26.455 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:19:26 smithi100 bash[74942]: audit 2024-04-07T17:19:25.161376+0000 mon.a (mon.0) 3475 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:19:26.455 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:19:26 smithi100 bash[74942]: audit 2024-04-07T17:19:25.172281+0000 mon.a (mon.0) 3476 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:19:26.455 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:19:26 smithi100 bash[74942]: cluster 2024-04-07T17:19:25.982391+0000 mgr.y (mgr.25175) 1922 : cluster [DBG] pgmap v3304: 745 pgs: 1 activating+degraded, 744 active+clean; 6.6 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 383 B/s rd, 0 op/s; 1/543 objects degraded (0.184%) 2024-04-07T17:19:26.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:19:26 smithi100 bash[76202]: audit 2024-04-07T17:19:25.159600+0000 mon.a (mon.0) 3474 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:19:26.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:19:26 smithi100 bash[76202]: audit 2024-04-07T17:19:25.161376+0000 mon.a (mon.0) 3475 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:19:26.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:19:26 smithi100 bash[76202]: audit 2024-04-07T17:19:25.172281+0000 mon.a (mon.0) 3476 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:19:26.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:19:26 smithi100 bash[76202]: cluster 2024-04-07T17:19:25.982391+0000 mgr.y (mgr.25175) 1922 : cluster [DBG] pgmap v3304: 745 pgs: 1 activating+degraded, 744 active+clean; 6.6 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 383 B/s rd, 0 op/s; 1/543 objects degraded (0.184%) 2024-04-07T17:19:27.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:19:27 smithi144 bash[66654]: cluster 2024-04-07T17:19:26.169328+0000 mon.a (mon.0) 3477 : cluster [WRN] Health check failed: Degraded data redundancy: 1/543 objects degraded (0.184%), 1 pg degraded (PG_DEGRADED) 2024-04-07T17:19:27.455 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:19:27 smithi100 bash[74942]: cluster 2024-04-07T17:19:26.169328+0000 mon.a (mon.0) 3477 : cluster [WRN] Health check failed: Degraded data redundancy: 1/543 objects degraded (0.184%), 1 pg degraded (PG_DEGRADED) 2024-04-07T17:19:27.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:19:27 smithi100 bash[76202]: cluster 2024-04-07T17:19:26.169328+0000 mon.a (mon.0) 3477 : cluster [WRN] Health check failed: Degraded data redundancy: 1/543 objects degraded (0.184%), 1 pg degraded (PG_DEGRADED) 2024-04-07T17:19:28.704 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:19:28 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:19:28] "GET /metrics HTTP/1.1" 200 83014 "" "Prometheus/2.43.0" 2024-04-07T17:19:29.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:19:28 smithi100 bash[74942]: cluster 2024-04-07T17:19:27.986411+0000 mgr.y (mgr.25175) 1923 : cluster [DBG] pgmap v3305: 745 pgs: 1 activating+degraded, 744 active+clean; 6.6 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 1/543 objects degraded (0.184%) 2024-04-07T17:19:29.205 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:19:28 smithi100 bash[76202]: cluster 2024-04-07T17:19:27.986411+0000 mgr.y (mgr.25175) 1923 : cluster [DBG] pgmap v3305: 745 pgs: 1 activating+degraded, 744 active+clean; 6.6 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 1/543 objects degraded (0.184%) 2024-04-07T17:19:29.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:19:28 smithi144 bash[66654]: cluster 2024-04-07T17:19:27.986411+0000 mgr.y (mgr.25175) 1923 : cluster [DBG] pgmap v3305: 745 pgs: 1 activating+degraded, 744 active+clean; 6.6 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 1/543 objects degraded (0.184%) 2024-04-07T17:19:30.368 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:19:30 smithi144 bash[66654]: cluster 2024-04-07T17:19:29.989821+0000 mgr.y (mgr.25175) 1924 : cluster [DBG] pgmap v3306: 745 pgs: 745 active+clean; 6.6 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 4 B/s, 0 objects/s recovering 2024-04-07T17:19:30.368 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:19:30 smithi144 bash[66654]: cluster 2024-04-07T17:19:30.043896+0000 mon.a (mon.0) 3478 : cluster [INF] Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 1/543 objects degraded (0.184%), 1 pg degraded) 2024-04-07T17:19:30.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:19:30 smithi100 bash[74942]: cluster 2024-04-07T17:19:29.989821+0000 mgr.y (mgr.25175) 1924 : cluster [DBG] pgmap v3306: 745 pgs: 745 active+clean; 6.6 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 4 B/s, 0 objects/s recovering 2024-04-07T17:19:30.455 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:19:30 smithi100 bash[74942]: cluster 2024-04-07T17:19:30.043896+0000 mon.a (mon.0) 3478 : cluster [INF] Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 1/543 objects degraded (0.184%), 1 pg degraded) 2024-04-07T17:19:30.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:19:30 smithi100 bash[76202]: cluster 2024-04-07T17:19:29.989821+0000 mgr.y (mgr.25175) 1924 : cluster [DBG] pgmap v3306: 745 pgs: 745 active+clean; 6.6 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 4 B/s, 0 objects/s recovering 2024-04-07T17:19:30.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:19:30 smithi100 bash[76202]: cluster 2024-04-07T17:19:30.043896+0000 mon.a (mon.0) 3478 : cluster [INF] Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 1/543 objects degraded (0.184%), 1 pg degraded) 2024-04-07T17:19:32.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:19:31 smithi100 bash[74942]: audit 2024-04-07T17:19:31.832052+0000 mon.a (mon.0) 3479 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:19:32.205 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:19:31 smithi100 bash[76202]: audit 2024-04-07T17:19:31.832052+0000 mon.a (mon.0) 3479 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:19:32.317 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:19:31 smithi144 bash[66654]: audit 2024-04-07T17:19:31.832052+0000 mon.a (mon.0) 3479 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:19:32.617 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:19:32 smithi144 bash[95233]: ts=2024-04-07T17:19:32.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:19:33.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:19:32 smithi100 bash[74942]: cluster 2024-04-07T17:19:31.991836+0000 mgr.y (mgr.25175) 1925 : cluster [DBG] pgmap v3307: 745 pgs: 745 active+clean; 6.6 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 3 B/s, 0 objects/s recovering 2024-04-07T17:19:33.205 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:19:32 smithi100 bash[76202]: cluster 2024-04-07T17:19:31.991836+0000 mgr.y (mgr.25175) 1925 : cluster [DBG] pgmap v3307: 745 pgs: 745 active+clean; 6.6 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 3 B/s, 0 objects/s recovering 2024-04-07T17:19:33.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:19:32 smithi144 bash[66654]: cluster 2024-04-07T17:19:31.991836+0000 mgr.y (mgr.25175) 1925 : cluster [DBG] pgmap v3307: 745 pgs: 745 active+clean; 6.6 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 3 B/s, 0 objects/s recovering 2024-04-07T17:19:34.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:19:34 smithi144 bash[66654]: cluster 2024-04-07T17:19:33.996262+0000 mgr.y (mgr.25175) 1926 : cluster [DBG] pgmap v3308: 745 pgs: 745 active+clean; 6.6 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 3 B/s, 0 objects/s recovering 2024-04-07T17:19:34.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:19:34 smithi100 bash[74942]: cluster 2024-04-07T17:19:33.996262+0000 mgr.y (mgr.25175) 1926 : cluster [DBG] pgmap v3308: 745 pgs: 745 active+clean; 6.6 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 3 B/s, 0 objects/s recovering 2024-04-07T17:19:34.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:19:34 smithi100 bash[76202]: cluster 2024-04-07T17:19:33.996262+0000 mgr.y (mgr.25175) 1926 : cluster [DBG] pgmap v3308: 745 pgs: 745 active+clean; 6.6 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 3 B/s, 0 objects/s recovering 2024-04-07T17:19:36.007 INFO:tasks.workunit.client.0.smithi100.stderr:test retry succeeds 2024-04-07T17:19:36.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:19:36 smithi144 bash[66654]: cluster 2024-04-07T17:19:35.999677+0000 mgr.y (mgr.25175) 1927 : cluster [DBG] pgmap v3309: 745 pgs: 745 active+clean; 6.6 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 2 B/s, 0 objects/s recovering 2024-04-07T17:19:36.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:19:36 smithi100 bash[74942]: cluster 2024-04-07T17:19:35.999677+0000 mgr.y (mgr.25175) 1927 : cluster [DBG] pgmap v3309: 745 pgs: 745 active+clean; 6.6 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 2 B/s, 0 objects/s recovering 2024-04-07T17:19:36.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:19:36 smithi100 bash[76202]: cluster 2024-04-07T17:19:35.999677+0000 mgr.y (mgr.25175) 1927 : cluster [DBG] pgmap v3309: 745 pgs: 745 active+clean; 6.6 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 2 B/s, 0 objects/s recovering 2024-04-07T17:19:37.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:19:37 smithi144 bash[66654]: cluster 2024-04-07T17:19:36.071480+0000 mon.a (mon.0) 3480 : cluster [DBG] osdmap e1808: 8 total, 8 up, 8 in 2024-04-07T17:19:37.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:19:37 smithi100 bash[74942]: cluster 2024-04-07T17:19:36.071480+0000 mon.a (mon.0) 3480 : cluster [DBG] osdmap e1808: 8 total, 8 up, 8 in 2024-04-07T17:19:37.458 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:19:37 smithi100 bash[76202]: cluster 2024-04-07T17:19:36.071480+0000 mon.a (mon.0) 3480 : cluster [DBG] osdmap e1808: 8 total, 8 up, 8 in 2024-04-07T17:19:38.358 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.QuiesceWatchTimeout (32138 ms) 2024-04-07T17:19:38.358 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.WriteZeroes 2024-04-07T17:19:38.358 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:19:38.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:19:38 smithi144 bash[66654]: cluster 2024-04-07T17:19:37.088318+0000 mon.a (mon.0) 3481 : cluster [DBG] osdmap e1809: 8 total, 8 up, 8 in 2024-04-07T17:19:38.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:19:38 smithi144 bash[66654]: cluster 2024-04-07T17:19:38.003011+0000 mgr.y (mgr.25175) 1928 : cluster [DBG] pgmap v3312: 745 pgs: 745 active+clean; 6.6 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 5.1 KiB/s rd, 894 B/s wr, 7 op/s 2024-04-07T17:19:38.428 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:19:38 smithi100 bash[74942]: cluster 2024-04-07T17:19:37.088318+0000 mon.a (mon.0) 3481 : cluster [DBG] osdmap e1809: 8 total, 8 up, 8 in 2024-04-07T17:19:38.428 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:19:38 smithi100 bash[74942]: cluster 2024-04-07T17:19:38.003011+0000 mgr.y (mgr.25175) 1928 : cluster [DBG] pgmap v3312: 745 pgs: 745 active+clean; 6.6 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 5.1 KiB/s rd, 894 B/s wr, 7 op/s 2024-04-07T17:19:38.429 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:19:38 smithi100 bash[76202]: cluster 2024-04-07T17:19:37.088318+0000 mon.a (mon.0) 3481 : cluster [DBG] osdmap e1809: 8 total, 8 up, 8 in 2024-04-07T17:19:38.429 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:19:38 smithi100 bash[76202]: cluster 2024-04-07T17:19:38.003011+0000 mgr.y (mgr.25175) 1928 : cluster [DBG] pgmap v3312: 745 pgs: 745 active+clean; 6.6 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 5.1 KiB/s rd, 894 B/s wr, 7 op/s 2024-04-07T17:19:38.494 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.WriteZeroes (136 ms) 2024-04-07T17:19:38.495 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.WriteZeroesThickProvision 2024-04-07T17:19:38.495 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:19:38.637 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.WriteZeroesThickProvision (143 ms) 2024-04-07T17:19:38.637 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] TestLibRBD.ConcurentOperations 2024-04-07T17:19:38.637 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:19:38.704 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:19:38 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:19:38] "GET /metrics HTTP/1.1" 200 83023 "" "Prometheus/2.43.0" 2024-04-07T17:19:39.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:19:39 smithi100 bash[74942]: cluster 2024-04-07T17:19:38.117939+0000 mon.a (mon.0) 3482 : cluster [DBG] osdmap e1810: 8 total, 8 up, 8 in 2024-04-07T17:19:39.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:19:39 smithi100 bash[76202]: cluster 2024-04-07T17:19:38.117939+0000 mon.a (mon.0) 3482 : cluster [DBG] osdmap e1810: 8 total, 8 up, 8 in 2024-04-07T17:19:39.617 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:19:39 smithi144 bash[66654]: cluster 2024-04-07T17:19:38.117939+0000 mon.a (mon.0) 3482 : cluster [DBG] osdmap e1810: 8 total, 8 up, 8 in 2024-04-07T17:19:40.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:19:40 smithi100 bash[74942]: cluster 2024-04-07T17:19:39.118041+0000 mon.a (mon.0) 3483 : cluster [DBG] osdmap e1811: 8 total, 8 up, 8 in 2024-04-07T17:19:40.455 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:19:40 smithi100 bash[74942]: cluster 2024-04-07T17:19:40.005201+0000 mgr.y (mgr.25175) 1929 : cluster [DBG] pgmap v3315: 745 pgs: 745 active+clean; 6.6 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 11 KiB/s rd, 1.7 KiB/s wr, 16 op/s 2024-04-07T17:19:40.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:19:40 smithi100 bash[76202]: cluster 2024-04-07T17:19:39.118041+0000 mon.a (mon.0) 3483 : cluster [DBG] osdmap e1811: 8 total, 8 up, 8 in 2024-04-07T17:19:40.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:19:40 smithi100 bash[76202]: cluster 2024-04-07T17:19:40.005201+0000 mgr.y (mgr.25175) 1929 : cluster [DBG] pgmap v3315: 745 pgs: 745 active+clean; 6.6 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 11 KiB/s rd, 1.7 KiB/s wr, 16 op/s 2024-04-07T17:19:40.617 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:19:40 smithi144 bash[66654]: cluster 2024-04-07T17:19:39.118041+0000 mon.a (mon.0) 3483 : cluster [DBG] osdmap e1811: 8 total, 8 up, 8 in 2024-04-07T17:19:40.617 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:19:40 smithi144 bash[66654]: cluster 2024-04-07T17:19:40.005201+0000 mgr.y (mgr.25175) 1929 : cluster [DBG] pgmap v3315: 745 pgs: 745 active+clean; 6.6 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 11 KiB/s rd, 1.7 KiB/s wr, 16 op/s 2024-04-07T17:19:41.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:19:41 smithi100 bash[74942]: cluster 2024-04-07T17:19:40.132744+0000 mon.a (mon.0) 3484 : cluster [DBG] osdmap e1812: 8 total, 8 up, 8 in 2024-04-07T17:19:41.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:19:41 smithi100 bash[76202]: cluster 2024-04-07T17:19:40.132744+0000 mon.a (mon.0) 3484 : cluster [DBG] osdmap e1812: 8 total, 8 up, 8 in 2024-04-07T17:19:41.617 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:19:41 smithi144 bash[66654]: cluster 2024-04-07T17:19:40.132744+0000 mon.a (mon.0) 3484 : cluster [DBG] osdmap e1812: 8 total, 8 up, 8 in 2024-04-07T17:19:42.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:19:42 smithi100 bash[74942]: cluster 2024-04-07T17:19:41.160215+0000 mon.a (mon.0) 3485 : cluster [DBG] osdmap e1813: 8 total, 8 up, 8 in 2024-04-07T17:19:42.455 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:19:42 smithi100 bash[74942]: cluster 2024-04-07T17:19:42.006474+0000 mgr.y (mgr.25175) 1930 : cluster [DBG] pgmap v3318: 745 pgs: 745 active+clean; 6.6 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 8.5 KiB/s rd, 2.0 KiB/s wr, 13 op/s 2024-04-07T17:19:42.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:19:42 smithi100 bash[76202]: cluster 2024-04-07T17:19:41.160215+0000 mon.a (mon.0) 3485 : cluster [DBG] osdmap e1813: 8 total, 8 up, 8 in 2024-04-07T17:19:42.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:19:42 smithi100 bash[76202]: cluster 2024-04-07T17:19:42.006474+0000 mgr.y (mgr.25175) 1930 : cluster [DBG] pgmap v3318: 745 pgs: 745 active+clean; 6.6 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 8.5 KiB/s rd, 2.0 KiB/s wr, 13 op/s 2024-04-07T17:19:42.617 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:19:42 smithi144 bash[66654]: cluster 2024-04-07T17:19:41.160215+0000 mon.a (mon.0) 3485 : cluster [DBG] osdmap e1813: 8 total, 8 up, 8 in 2024-04-07T17:19:42.617 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:19:42 smithi144 bash[66654]: cluster 2024-04-07T17:19:42.006474+0000 mgr.y (mgr.25175) 1930 : cluster [DBG] pgmap v3318: 745 pgs: 745 active+clean; 6.6 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 8.5 KiB/s rd, 2.0 KiB/s wr, 13 op/s 2024-04-07T17:19:42.618 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:19:42 smithi144 bash[95233]: ts=2024-04-07T17:19:42.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:19:43.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:19:43 smithi100 bash[74942]: cluster 2024-04-07T17:19:42.178499+0000 mon.a (mon.0) 3486 : cluster [DBG] osdmap e1814: 8 total, 8 up, 8 in 2024-04-07T17:19:43.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:19:43 smithi100 bash[76202]: cluster 2024-04-07T17:19:42.178499+0000 mon.a (mon.0) 3486 : cluster [DBG] osdmap e1814: 8 total, 8 up, 8 in 2024-04-07T17:19:43.617 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:19:43 smithi144 bash[66654]: cluster 2024-04-07T17:19:42.178499+0000 mon.a (mon.0) 3486 : cluster [DBG] osdmap e1814: 8 total, 8 up, 8 in 2024-04-07T17:19:44.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:19:44 smithi100 bash[74942]: cluster 2024-04-07T17:19:43.176331+0000 mon.a (mon.0) 3487 : cluster [DBG] osdmap e1815: 8 total, 8 up, 8 in 2024-04-07T17:19:44.455 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:19:44 smithi100 bash[74942]: cluster 2024-04-07T17:19:44.007802+0000 mgr.y (mgr.25175) 1931 : cluster [DBG] pgmap v3321: 745 pgs: 745 active+clean; 6.6 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 121 KiB/s rd, 9.7 KiB/s wr, 170 op/s 2024-04-07T17:19:44.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:19:44 smithi100 bash[76202]: cluster 2024-04-07T17:19:43.176331+0000 mon.a (mon.0) 3487 : cluster [DBG] osdmap e1815: 8 total, 8 up, 8 in 2024-04-07T17:19:44.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:19:44 smithi100 bash[76202]: cluster 2024-04-07T17:19:44.007802+0000 mgr.y (mgr.25175) 1931 : cluster [DBG] pgmap v3321: 745 pgs: 745 active+clean; 6.6 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 121 KiB/s rd, 9.7 KiB/s wr, 170 op/s 2024-04-07T17:19:44.617 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:19:44 smithi144 bash[66654]: cluster 2024-04-07T17:19:43.176331+0000 mon.a (mon.0) 3487 : cluster [DBG] osdmap e1815: 8 total, 8 up, 8 in 2024-04-07T17:19:44.617 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:19:44 smithi144 bash[66654]: cluster 2024-04-07T17:19:44.007802+0000 mgr.y (mgr.25175) 1931 : cluster [DBG] pgmap v3321: 745 pgs: 745 active+clean; 6.6 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 121 KiB/s rd, 9.7 KiB/s wr, 170 op/s 2024-04-07T17:19:45.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:19:45 smithi100 bash[74942]: cluster 2024-04-07T17:19:44.203731+0000 mon.a (mon.0) 3488 : cluster [DBG] osdmap e1816: 8 total, 8 up, 8 in 2024-04-07T17:19:45.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:19:45 smithi100 bash[76202]: cluster 2024-04-07T17:19:44.203731+0000 mon.a (mon.0) 3488 : cluster [DBG] osdmap e1816: 8 total, 8 up, 8 in 2024-04-07T17:19:45.617 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:19:45 smithi144 bash[66654]: cluster 2024-04-07T17:19:44.203731+0000 mon.a (mon.0) 3488 : cluster [DBG] osdmap e1816: 8 total, 8 up, 8 in 2024-04-07T17:19:46.617 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:19:46 smithi144 bash[66654]: cluster 2024-04-07T17:19:45.212132+0000 mon.a (mon.0) 3489 : cluster [DBG] osdmap e1817: 8 total, 8 up, 8 in 2024-04-07T17:19:46.617 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:19:46 smithi144 bash[66654]: cluster 2024-04-07T17:19:46.009006+0000 mgr.y (mgr.25175) 1932 : cluster [DBG] pgmap v3324: 745 pgs: 745 active+clean; 6.6 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 113 KiB/s rd, 7.7 KiB/s wr, 159 op/s 2024-04-07T17:19:46.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:19:46 smithi100 bash[74942]: cluster 2024-04-07T17:19:45.212132+0000 mon.a (mon.0) 3489 : cluster [DBG] osdmap e1817: 8 total, 8 up, 8 in 2024-04-07T17:19:46.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:19:46 smithi100 bash[74942]: cluster 2024-04-07T17:19:46.009006+0000 mgr.y (mgr.25175) 1932 : cluster [DBG] pgmap v3324: 745 pgs: 745 active+clean; 6.6 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 113 KiB/s rd, 7.7 KiB/s wr, 159 op/s 2024-04-07T17:19:46.705 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:19:46 smithi100 bash[76202]: cluster 2024-04-07T17:19:45.212132+0000 mon.a (mon.0) 3489 : cluster [DBG] osdmap e1817: 8 total, 8 up, 8 in 2024-04-07T17:19:46.705 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:19:46 smithi100 bash[76202]: cluster 2024-04-07T17:19:46.009006+0000 mgr.y (mgr.25175) 1932 : cluster [DBG] pgmap v3324: 745 pgs: 745 active+clean; 6.6 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 113 KiB/s rd, 7.7 KiB/s wr, 159 op/s 2024-04-07T17:19:47.617 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:19:47 smithi144 bash[66654]: cluster 2024-04-07T17:19:46.226091+0000 mon.a (mon.0) 3490 : cluster [DBG] osdmap e1818: 8 total, 8 up, 8 in 2024-04-07T17:19:47.617 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:19:47 smithi144 bash[66654]: audit 2024-04-07T17:19:46.833650+0000 mon.a (mon.0) 3491 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:19:47.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:19:47 smithi100 bash[74942]: cluster 2024-04-07T17:19:46.226091+0000 mon.a (mon.0) 3490 : cluster [DBG] osdmap e1818: 8 total, 8 up, 8 in 2024-04-07T17:19:47.705 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:19:47 smithi100 bash[74942]: audit 2024-04-07T17:19:46.833650+0000 mon.a (mon.0) 3491 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:19:47.705 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:19:47 smithi100 bash[76202]: cluster 2024-04-07T17:19:46.226091+0000 mon.a (mon.0) 3490 : cluster [DBG] osdmap e1818: 8 total, 8 up, 8 in 2024-04-07T17:19:47.705 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:19:47 smithi100 bash[76202]: audit 2024-04-07T17:19:46.833650+0000 mon.a (mon.0) 3491 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:19:48.617 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:19:48 smithi144 bash[66654]: cluster 2024-04-07T17:19:47.223496+0000 mon.a (mon.0) 3492 : cluster [DBG] osdmap e1819: 8 total, 8 up, 8 in 2024-04-07T17:19:48.617 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:19:48 smithi144 bash[66654]: cluster 2024-04-07T17:19:47.944631+0000 mon.a (mon.0) 3493 : cluster [DBG] osdmap e1820: 8 total, 8 up, 8 in 2024-04-07T17:19:48.617 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:19:48 smithi144 bash[66654]: cluster 2024-04-07T17:19:48.010182+0000 mgr.y (mgr.25175) 1933 : cluster [DBG] pgmap v3328: 745 pgs: 745 active+clean; 6.6 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 12 KiB/s rd, 3.4 KiB/s wr, 16 op/s 2024-04-07T17:19:48.704 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:19:48 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:19:48] "GET /metrics HTTP/1.1" 200 83033 "" "Prometheus/2.43.0" 2024-04-07T17:19:48.705 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:19:48 smithi100 bash[74942]: cluster 2024-04-07T17:19:47.223496+0000 mon.a (mon.0) 3492 : cluster [DBG] osdmap e1819: 8 total, 8 up, 8 in 2024-04-07T17:19:48.705 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:19:48 smithi100 bash[74942]: cluster 2024-04-07T17:19:47.944631+0000 mon.a (mon.0) 3493 : cluster [DBG] osdmap e1820: 8 total, 8 up, 8 in 2024-04-07T17:19:48.705 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:19:48 smithi100 bash[74942]: cluster 2024-04-07T17:19:48.010182+0000 mgr.y (mgr.25175) 1933 : cluster [DBG] pgmap v3328: 745 pgs: 745 active+clean; 6.6 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 12 KiB/s rd, 3.4 KiB/s wr, 16 op/s 2024-04-07T17:19:48.705 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:19:48 smithi100 bash[76202]: cluster 2024-04-07T17:19:47.223496+0000 mon.a (mon.0) 3492 : cluster [DBG] osdmap e1819: 8 total, 8 up, 8 in 2024-04-07T17:19:48.705 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:19:48 smithi100 bash[76202]: cluster 2024-04-07T17:19:47.944631+0000 mon.a (mon.0) 3493 : cluster [DBG] osdmap e1820: 8 total, 8 up, 8 in 2024-04-07T17:19:48.705 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:19:48 smithi100 bash[76202]: cluster 2024-04-07T17:19:48.010182+0000 mgr.y (mgr.25175) 1933 : cluster [DBG] pgmap v3328: 745 pgs: 745 active+clean; 6.6 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 12 KiB/s rd, 3.4 KiB/s wr, 16 op/s 2024-04-07T17:19:50.369 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:19:49 smithi144 bash[66654]: cluster 2024-04-07T17:19:48.955550+0000 mon.a (mon.0) 3494 : cluster [DBG] osdmap e1821: 8 total, 8 up, 8 in 2024-04-07T17:19:50.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:19:49 smithi100 bash[74942]: cluster 2024-04-07T17:19:48.955550+0000 mon.a (mon.0) 3494 : cluster [DBG] osdmap e1821: 8 total, 8 up, 8 in 2024-04-07T17:19:50.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:19:49 smithi100 bash[76202]: cluster 2024-04-07T17:19:48.955550+0000 mon.a (mon.0) 3494 : cluster [DBG] osdmap e1821: 8 total, 8 up, 8 in 2024-04-07T17:19:51.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:19:50 smithi144 bash[66654]: cluster 2024-04-07T17:19:49.959603+0000 mon.a (mon.0) 3495 : cluster [DBG] osdmap e1822: 8 total, 8 up, 8 in 2024-04-07T17:19:51.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:19:50 smithi144 bash[66654]: cluster 2024-04-07T17:19:50.012065+0000 mgr.y (mgr.25175) 1934 : cluster [DBG] pgmap v3331: 745 pgs: 745 active+clean; 6.6 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 12 KiB/s rd, 4.2 KiB/s wr, 17 op/s 2024-04-07T17:19:51.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:19:50 smithi100 bash[74942]: cluster 2024-04-07T17:19:49.959603+0000 mon.a (mon.0) 3495 : cluster [DBG] osdmap e1822: 8 total, 8 up, 8 in 2024-04-07T17:19:51.455 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:19:50 smithi100 bash[74942]: cluster 2024-04-07T17:19:50.012065+0000 mgr.y (mgr.25175) 1934 : cluster [DBG] pgmap v3331: 745 pgs: 745 active+clean; 6.6 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 12 KiB/s rd, 4.2 KiB/s wr, 17 op/s 2024-04-07T17:19:51.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:19:50 smithi100 bash[76202]: cluster 2024-04-07T17:19:49.959603+0000 mon.a (mon.0) 3495 : cluster [DBG] osdmap e1822: 8 total, 8 up, 8 in 2024-04-07T17:19:51.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:19:50 smithi100 bash[76202]: cluster 2024-04-07T17:19:50.012065+0000 mgr.y (mgr.25175) 1934 : cluster [DBG] pgmap v3331: 745 pgs: 745 active+clean; 6.6 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 12 KiB/s rd, 4.2 KiB/s wr, 17 op/s 2024-04-07T17:19:52.317 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:19:52 smithi144 bash[66654]: cluster 2024-04-07T17:19:50.988361+0000 mon.a (mon.0) 3496 : cluster [DBG] osdmap e1823: 8 total, 8 up, 8 in 2024-04-07T17:19:52.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:19:52 smithi100 bash[74942]: cluster 2024-04-07T17:19:50.988361+0000 mon.a (mon.0) 3496 : cluster [DBG] osdmap e1823: 8 total, 8 up, 8 in 2024-04-07T17:19:52.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:19:52 smithi100 bash[76202]: cluster 2024-04-07T17:19:50.988361+0000 mon.a (mon.0) 3496 : cluster [DBG] osdmap e1823: 8 total, 8 up, 8 in 2024-04-07T17:19:52.617 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:19:52 smithi144 bash[95233]: ts=2024-04-07T17:19:52.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:19:53.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:19:53 smithi144 bash[66654]: cluster 2024-04-07T17:19:51.992026+0000 mon.a (mon.0) 3497 : cluster [DBG] osdmap e1824: 8 total, 8 up, 8 in 2024-04-07T17:19:53.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:19:53 smithi100 bash[74942]: cluster 2024-04-07T17:19:51.992026+0000 mon.a (mon.0) 3497 : cluster [DBG] osdmap e1824: 8 total, 8 up, 8 in 2024-04-07T17:19:53.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:19:53 smithi100 bash[76202]: cluster 2024-04-07T17:19:51.992026+0000 mon.a (mon.0) 3497 : cluster [DBG] osdmap e1824: 8 total, 8 up, 8 in 2024-04-07T17:19:54.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:19:54 smithi144 bash[66654]: cluster 2024-04-07T17:19:52.013753+0000 mgr.y (mgr.25175) 1935 : cluster [DBG] pgmap v3334: 745 pgs: 745 active+clean; 6.6 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 2.7 KiB/s rd, 1023 B/s wr, 4 op/s 2024-04-07T17:19:54.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:19:54 smithi144 bash[66654]: cluster 2024-04-07T17:19:53.011066+0000 mon.a (mon.0) 3498 : cluster [DBG] osdmap e1825: 8 total, 8 up, 8 in 2024-04-07T17:19:54.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:19:54 smithi100 bash[74942]: cluster 2024-04-07T17:19:52.013753+0000 mgr.y (mgr.25175) 1935 : cluster [DBG] pgmap v3334: 745 pgs: 745 active+clean; 6.6 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 2.7 KiB/s rd, 1023 B/s wr, 4 op/s 2024-04-07T17:19:54.455 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:19:54 smithi100 bash[74942]: cluster 2024-04-07T17:19:53.011066+0000 mon.a (mon.0) 3498 : cluster [DBG] osdmap e1825: 8 total, 8 up, 8 in 2024-04-07T17:19:54.457 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:19:54 smithi100 bash[76202]: cluster 2024-04-07T17:19:52.013753+0000 mgr.y (mgr.25175) 1935 : cluster [DBG] pgmap v3334: 745 pgs: 745 active+clean; 6.6 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 2.7 KiB/s rd, 1023 B/s wr, 4 op/s 2024-04-07T17:19:54.457 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:19:54 smithi100 bash[76202]: cluster 2024-04-07T17:19:53.011066+0000 mon.a (mon.0) 3498 : cluster [DBG] osdmap e1825: 8 total, 8 up, 8 in 2024-04-07T17:19:55.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:19:55 smithi144 bash[66654]: cluster 2024-04-07T17:19:54.015897+0000 mgr.y (mgr.25175) 1936 : cluster [DBG] pgmap v3336: 745 pgs: 745 active+clean; 6.6 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 179 KiB/s rd, 5.9 KiB/s wr, 247 op/s 2024-04-07T17:19:55.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:19:55 smithi144 bash[66654]: cluster 2024-04-07T17:19:54.050808+0000 mon.a (mon.0) 3499 : cluster [DBG] osdmap e1826: 8 total, 8 up, 8 in 2024-04-07T17:19:55.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:19:55 smithi100 bash[74942]: cluster 2024-04-07T17:19:54.015897+0000 mgr.y (mgr.25175) 1936 : cluster [DBG] pgmap v3336: 745 pgs: 745 active+clean; 6.6 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 179 KiB/s rd, 5.9 KiB/s wr, 247 op/s 2024-04-07T17:19:55.455 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:19:55 smithi100 bash[74942]: cluster 2024-04-07T17:19:54.050808+0000 mon.a (mon.0) 3499 : cluster [DBG] osdmap e1826: 8 total, 8 up, 8 in 2024-04-07T17:19:55.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:19:55 smithi100 bash[76202]: cluster 2024-04-07T17:19:54.015897+0000 mgr.y (mgr.25175) 1936 : cluster [DBG] pgmap v3336: 745 pgs: 745 active+clean; 6.6 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 179 KiB/s rd, 5.9 KiB/s wr, 247 op/s 2024-04-07T17:19:55.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:19:55 smithi100 bash[76202]: cluster 2024-04-07T17:19:54.050808+0000 mon.a (mon.0) 3499 : cluster [DBG] osdmap e1826: 8 total, 8 up, 8 in 2024-04-07T17:19:56.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:19:56 smithi144 bash[66654]: cluster 2024-04-07T17:19:55.056004+0000 mon.a (mon.0) 3500 : cluster [DBG] osdmap e1827: 8 total, 8 up, 8 in 2024-04-07T17:19:56.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:19:56 smithi100 bash[74942]: cluster 2024-04-07T17:19:55.056004+0000 mon.a (mon.0) 3500 : cluster [DBG] osdmap e1827: 8 total, 8 up, 8 in 2024-04-07T17:19:56.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:19:56 smithi100 bash[76202]: cluster 2024-04-07T17:19:55.056004+0000 mon.a (mon.0) 3500 : cluster [DBG] osdmap e1827: 8 total, 8 up, 8 in 2024-04-07T17:19:57.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:19:57 smithi144 bash[66654]: cluster 2024-04-07T17:19:56.017107+0000 mgr.y (mgr.25175) 1937 : cluster [DBG] pgmap v3339: 745 pgs: 745 active+clean; 6.6 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 178 KiB/s rd, 5.7 KiB/s wr, 247 op/s 2024-04-07T17:19:57.368 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:19:57 smithi144 bash[66654]: cluster 2024-04-07T17:19:56.069469+0000 mon.a (mon.0) 3501 : cluster [DBG] osdmap e1828: 8 total, 8 up, 8 in 2024-04-07T17:19:57.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:19:57 smithi100 bash[74942]: cluster 2024-04-07T17:19:56.017107+0000 mgr.y (mgr.25175) 1937 : cluster [DBG] pgmap v3339: 745 pgs: 745 active+clean; 6.6 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 178 KiB/s rd, 5.7 KiB/s wr, 247 op/s 2024-04-07T17:19:57.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:19:57 smithi100 bash[74942]: cluster 2024-04-07T17:19:56.069469+0000 mon.a (mon.0) 3501 : cluster [DBG] osdmap e1828: 8 total, 8 up, 8 in 2024-04-07T17:19:57.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:19:57 smithi100 bash[76202]: cluster 2024-04-07T17:19:56.017107+0000 mgr.y (mgr.25175) 1937 : cluster [DBG] pgmap v3339: 745 pgs: 745 active+clean; 6.6 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 178 KiB/s rd, 5.7 KiB/s wr, 247 op/s 2024-04-07T17:19:57.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:19:57 smithi100 bash[76202]: cluster 2024-04-07T17:19:56.069469+0000 mon.a (mon.0) 3501 : cluster [DBG] osdmap e1828: 8 total, 8 up, 8 in 2024-04-07T17:19:58.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:19:58 smithi144 bash[66654]: cluster 2024-04-07T17:19:57.070910+0000 mon.a (mon.0) 3502 : cluster [DBG] osdmap e1829: 8 total, 8 up, 8 in 2024-04-07T17:19:58.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:19:58 smithi144 bash[66654]: cluster 2024-04-07T17:19:57.950213+0000 mon.a (mon.0) 3503 : cluster [DBG] osdmap e1830: 8 total, 8 up, 8 in 2024-04-07T17:19:58.428 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:19:58 smithi100 bash[74942]: cluster 2024-04-07T17:19:57.070910+0000 mon.a (mon.0) 3502 : cluster [DBG] osdmap e1829: 8 total, 8 up, 8 in 2024-04-07T17:19:58.429 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:19:58 smithi100 bash[74942]: cluster 2024-04-07T17:19:57.950213+0000 mon.a (mon.0) 3503 : cluster [DBG] osdmap e1830: 8 total, 8 up, 8 in 2024-04-07T17:19:58.429 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:19:58 smithi100 bash[76202]: cluster 2024-04-07T17:19:57.070910+0000 mon.a (mon.0) 3502 : cluster [DBG] osdmap e1829: 8 total, 8 up, 8 in 2024-04-07T17:19:58.429 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:19:58 smithi100 bash[76202]: cluster 2024-04-07T17:19:57.950213+0000 mon.a (mon.0) 3503 : cluster [DBG] osdmap e1830: 8 total, 8 up, 8 in 2024-04-07T17:19:58.704 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:19:58 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:19:58] "GET /metrics HTTP/1.1" 200 83033 "" "Prometheus/2.43.0" 2024-04-07T17:20:00.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:19:59 smithi100 bash[74942]: cluster 2024-04-07T17:19:58.018159+0000 mgr.y (mgr.25175) 1938 : cluster [DBG] pgmap v3343: 745 pgs: 745 active+clean; 6.6 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 11 KiB/s rd, 4.0 KiB/s wr, 19 op/s 2024-04-07T17:20:00.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:19:59 smithi100 bash[74942]: cluster 2024-04-07T17:19:59.094379+0000 mon.a (mon.0) 3504 : cluster [DBG] osdmap e1831: 8 total, 8 up, 8 in 2024-04-07T17:20:00.205 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:19:59 smithi100 bash[76202]: cluster 2024-04-07T17:19:58.018159+0000 mgr.y (mgr.25175) 1938 : cluster [DBG] pgmap v3343: 745 pgs: 745 active+clean; 6.6 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 11 KiB/s rd, 4.0 KiB/s wr, 19 op/s 2024-04-07T17:20:00.205 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:19:59 smithi100 bash[76202]: cluster 2024-04-07T17:19:59.094379+0000 mon.a (mon.0) 3504 : cluster [DBG] osdmap e1831: 8 total, 8 up, 8 in 2024-04-07T17:20:00.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:19:59 smithi144 bash[66654]: cluster 2024-04-07T17:19:58.018159+0000 mgr.y (mgr.25175) 1938 : cluster [DBG] pgmap v3343: 745 pgs: 745 active+clean; 6.6 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 11 KiB/s rd, 4.0 KiB/s wr, 19 op/s 2024-04-07T17:20:00.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:19:59 smithi144 bash[66654]: cluster 2024-04-07T17:19:59.094379+0000 mon.a (mon.0) 3504 : cluster [DBG] osdmap e1831: 8 total, 8 up, 8 in 2024-04-07T17:20:01.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:20:00 smithi144 bash[66654]: cluster 2024-04-07T17:20:00.000120+0000 mon.a (mon.0) 3505 : cluster [WRN] overall HEALTH_WARN Telemetry requires re-opt-in 2024-04-07T17:20:01.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:20:00 smithi144 bash[66654]: cluster 2024-04-07T17:20:00.099528+0000 mon.a (mon.0) 3506 : cluster [DBG] osdmap e1832: 8 total, 8 up, 8 in 2024-04-07T17:20:01.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:20:00 smithi100 bash[74942]: cluster 2024-04-07T17:20:00.000120+0000 mon.a (mon.0) 3505 : cluster [WRN] overall HEALTH_WARN Telemetry requires re-opt-in 2024-04-07T17:20:01.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:20:00 smithi100 bash[74942]: cluster 2024-04-07T17:20:00.099528+0000 mon.a (mon.0) 3506 : cluster [DBG] osdmap e1832: 8 total, 8 up, 8 in 2024-04-07T17:20:01.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:20:00 smithi100 bash[76202]: cluster 2024-04-07T17:20:00.000120+0000 mon.a (mon.0) 3505 : cluster [WRN] overall HEALTH_WARN Telemetry requires re-opt-in 2024-04-07T17:20:01.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:20:00 smithi100 bash[76202]: cluster 2024-04-07T17:20:00.099528+0000 mon.a (mon.0) 3506 : cluster [DBG] osdmap e1832: 8 total, 8 up, 8 in 2024-04-07T17:20:02.318 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:20:01 smithi144 bash[66654]: cluster 2024-04-07T17:20:00.020938+0000 mgr.y (mgr.25175) 1939 : cluster [DBG] pgmap v3345: 745 pgs: 745 active+clean; 6.6 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 11 KiB/s rd, 4.0 KiB/s wr, 19 op/s 2024-04-07T17:20:02.318 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:20:01 smithi144 bash[66654]: cluster 2024-04-07T17:20:01.100186+0000 mon.a (mon.0) 3507 : cluster [DBG] osdmap e1833: 8 total, 8 up, 8 in 2024-04-07T17:20:02.318 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:20:01 smithi144 bash[66654]: audit 2024-04-07T17:20:01.833056+0000 mon.a (mon.0) 3508 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:20:02.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:20:01 smithi100 bash[74942]: cluster 2024-04-07T17:20:00.020938+0000 mgr.y (mgr.25175) 1939 : cluster [DBG] pgmap v3345: 745 pgs: 745 active+clean; 6.6 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 11 KiB/s rd, 4.0 KiB/s wr, 19 op/s 2024-04-07T17:20:02.455 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:20:01 smithi100 bash[74942]: cluster 2024-04-07T17:20:01.100186+0000 mon.a (mon.0) 3507 : cluster [DBG] osdmap e1833: 8 total, 8 up, 8 in 2024-04-07T17:20:02.455 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:20:01 smithi100 bash[74942]: audit 2024-04-07T17:20:01.833056+0000 mon.a (mon.0) 3508 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:20:02.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:20:01 smithi100 bash[76202]: cluster 2024-04-07T17:20:00.020938+0000 mgr.y (mgr.25175) 1939 : cluster [DBG] pgmap v3345: 745 pgs: 745 active+clean; 6.6 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 11 KiB/s rd, 4.0 KiB/s wr, 19 op/s 2024-04-07T17:20:02.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:20:01 smithi100 bash[76202]: cluster 2024-04-07T17:20:01.100186+0000 mon.a (mon.0) 3507 : cluster [DBG] osdmap e1833: 8 total, 8 up, 8 in 2024-04-07T17:20:02.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:20:01 smithi100 bash[76202]: audit 2024-04-07T17:20:01.833056+0000 mon.a (mon.0) 3508 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:20:02.617 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:20:02 smithi144 bash[95233]: ts=2024-04-07T17:20:02.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:20:03.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:20:03 smithi144 bash[66654]: cluster 2024-04-07T17:20:02.022204+0000 mgr.y (mgr.25175) 1940 : cluster [DBG] pgmap v3348: 745 pgs: 745 active+clean; 6.6 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 1.5 KiB/s rd, 251 B/s wr, 2 op/s 2024-04-07T17:20:03.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:20:03 smithi144 bash[66654]: cluster 2024-04-07T17:20:02.105139+0000 mon.a (mon.0) 3509 : cluster [DBG] osdmap e1834: 8 total, 8 up, 8 in 2024-04-07T17:20:03.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:20:03 smithi144 bash[66654]: cluster 2024-04-07T17:20:02.950876+0000 mon.a (mon.0) 3510 : cluster [DBG] osdmap e1835: 8 total, 8 up, 8 in 2024-04-07T17:20:03.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:20:03 smithi100 bash[74942]: cluster 2024-04-07T17:20:02.022204+0000 mgr.y (mgr.25175) 1940 : cluster [DBG] pgmap v3348: 745 pgs: 745 active+clean; 6.6 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 1.5 KiB/s rd, 251 B/s wr, 2 op/s 2024-04-07T17:20:03.455 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:20:03 smithi100 bash[74942]: cluster 2024-04-07T17:20:02.105139+0000 mon.a (mon.0) 3509 : cluster [DBG] osdmap e1834: 8 total, 8 up, 8 in 2024-04-07T17:20:03.455 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:20:03 smithi100 bash[74942]: cluster 2024-04-07T17:20:02.950876+0000 mon.a (mon.0) 3510 : cluster [DBG] osdmap e1835: 8 total, 8 up, 8 in 2024-04-07T17:20:03.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:20:03 smithi100 bash[76202]: cluster 2024-04-07T17:20:02.022204+0000 mgr.y (mgr.25175) 1940 : cluster [DBG] pgmap v3348: 745 pgs: 745 active+clean; 6.6 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 1.5 KiB/s rd, 251 B/s wr, 2 op/s 2024-04-07T17:20:03.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:20:03 smithi100 bash[76202]: cluster 2024-04-07T17:20:02.105139+0000 mon.a (mon.0) 3509 : cluster [DBG] osdmap e1834: 8 total, 8 up, 8 in 2024-04-07T17:20:03.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:20:03 smithi100 bash[76202]: cluster 2024-04-07T17:20:02.950876+0000 mon.a (mon.0) 3510 : cluster [DBG] osdmap e1835: 8 total, 8 up, 8 in 2024-04-07T17:20:04.170 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] TestLibRBD.ConcurentOperations (25532 ms) 2024-04-07T17:20:05.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:20:04 smithi144 bash[66654]: cluster 2024-04-07T17:20:03.977767+0000 mon.a (mon.0) 3511 : cluster [DBG] osdmap e1836: 8 total, 8 up, 8 in 2024-04-07T17:20:05.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:20:04 smithi100 bash[74942]: cluster 2024-04-07T17:20:03.977767+0000 mon.a (mon.0) 3511 : cluster [DBG] osdmap e1836: 8 total, 8 up, 8 in 2024-04-07T17:20:05.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:20:04 smithi100 bash[76202]: cluster 2024-04-07T17:20:03.977767+0000 mon.a (mon.0) 3511 : cluster [DBG] osdmap e1836: 8 total, 8 up, 8 in 2024-04-07T17:20:06.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:20:06 smithi144 bash[66654]: cluster 2024-04-07T17:20:04.023284+0000 mgr.y (mgr.25175) 1941 : cluster [DBG] pgmap v3352: 745 pgs: 745 active+clean; 6.6 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 54 KiB/s rd, 3.8 KiB/s wr, 72 op/s 2024-04-07T17:20:06.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:20:06 smithi144 bash[66654]: cluster 2024-04-07T17:20:04.992658+0000 mon.a (mon.0) 3512 : cluster [DBG] osdmap e1837: 8 total, 8 up, 8 in 2024-04-07T17:20:06.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:20:06 smithi100 bash[74942]: cluster 2024-04-07T17:20:04.023284+0000 mgr.y (mgr.25175) 1941 : cluster [DBG] pgmap v3352: 745 pgs: 745 active+clean; 6.6 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 54 KiB/s rd, 3.8 KiB/s wr, 72 op/s 2024-04-07T17:20:06.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:20:06 smithi100 bash[74942]: cluster 2024-04-07T17:20:04.992658+0000 mon.a (mon.0) 3512 : cluster [DBG] osdmap e1837: 8 total, 8 up, 8 in 2024-04-07T17:20:06.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:20:06 smithi100 bash[76202]: cluster 2024-04-07T17:20:04.023284+0000 mgr.y (mgr.25175) 1941 : cluster [DBG] pgmap v3352: 745 pgs: 745 active+clean; 6.6 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 54 KiB/s rd, 3.8 KiB/s wr, 72 op/s 2024-04-07T17:20:06.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:20:06 smithi100 bash[76202]: cluster 2024-04-07T17:20:04.992658+0000 mon.a (mon.0) 3512 : cluster [DBG] osdmap e1837: 8 total, 8 up, 8 in 2024-04-07T17:20:07.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:20:07 smithi144 bash[66654]: cluster 2024-04-07T17:20:05.991177+0000 mon.a (mon.0) 3513 : cluster [DBG] osdmap e1838: 8 total, 8 up, 8 in 2024-04-07T17:20:07.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:20:07 smithi100 bash[74942]: cluster 2024-04-07T17:20:05.991177+0000 mon.a (mon.0) 3513 : cluster [DBG] osdmap e1838: 8 total, 8 up, 8 in 2024-04-07T17:20:07.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:20:07 smithi100 bash[76202]: cluster 2024-04-07T17:20:05.991177+0000 mon.a (mon.0) 3513 : cluster [DBG] osdmap e1838: 8 total, 8 up, 8 in 2024-04-07T17:20:08.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:20:08 smithi144 bash[66654]: cluster 2024-04-07T17:20:06.026879+0000 mgr.y (mgr.25175) 1942 : cluster [DBG] pgmap v3355: 681 pgs: 681 active+clean; 6.6 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 2.0 KiB/s rd, 1 op/s 2024-04-07T17:20:08.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:20:08 smithi144 bash[66654]: cluster 2024-04-07T17:20:06.995624+0000 mon.a (mon.0) 3514 : cluster [DBG] osdmap e1839: 8 total, 8 up, 8 in 2024-04-07T17:20:08.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:20:08 smithi144 bash[66654]: cluster 2024-04-07T17:20:07.957996+0000 mon.a (mon.0) 3515 : cluster [DBG] osdmap e1840: 8 total, 8 up, 8 in 2024-04-07T17:20:08.428 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:20:08 smithi100 bash[74942]: cluster 2024-04-07T17:20:06.026879+0000 mgr.y (mgr.25175) 1942 : cluster [DBG] pgmap v3355: 681 pgs: 681 active+clean; 6.6 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 2.0 KiB/s rd, 1 op/s 2024-04-07T17:20:08.428 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:20:08 smithi100 bash[74942]: cluster 2024-04-07T17:20:06.995624+0000 mon.a (mon.0) 3514 : cluster [DBG] osdmap e1839: 8 total, 8 up, 8 in 2024-04-07T17:20:08.428 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:20:08 smithi100 bash[74942]: cluster 2024-04-07T17:20:07.957996+0000 mon.a (mon.0) 3515 : cluster [DBG] osdmap e1840: 8 total, 8 up, 8 in 2024-04-07T17:20:08.428 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:20:08 smithi100 bash[76202]: cluster 2024-04-07T17:20:06.026879+0000 mgr.y (mgr.25175) 1942 : cluster [DBG] pgmap v3355: 681 pgs: 681 active+clean; 6.6 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 2.0 KiB/s rd, 1 op/s 2024-04-07T17:20:08.428 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:20:08 smithi100 bash[76202]: cluster 2024-04-07T17:20:06.995624+0000 mon.a (mon.0) 3514 : cluster [DBG] osdmap e1839: 8 total, 8 up, 8 in 2024-04-07T17:20:08.429 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:20:08 smithi100 bash[76202]: cluster 2024-04-07T17:20:07.957996+0000 mon.a (mon.0) 3515 : cluster [DBG] osdmap e1840: 8 total, 8 up, 8 in 2024-04-07T17:20:08.704 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:20:08 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:20:08] "GET /metrics HTTP/1.1" 200 83036 "" "Prometheus/2.43.0" 2024-04-07T17:20:10.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:20:09 smithi144 bash[66654]: cluster 2024-04-07T17:20:08.028031+0000 mgr.y (mgr.25175) 1943 : cluster [DBG] pgmap v3358: 617 pgs: 617 active+clean; 6.6 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail 2024-04-07T17:20:10.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:20:09 smithi144 bash[66654]: cluster 2024-04-07T17:20:08.966683+0000 mon.a (mon.0) 3516 : cluster [DBG] osdmap e1841: 8 total, 8 up, 8 in 2024-04-07T17:20:10.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:20:09 smithi100 bash[74942]: cluster 2024-04-07T17:20:08.028031+0000 mgr.y (mgr.25175) 1943 : cluster [DBG] pgmap v3358: 617 pgs: 617 active+clean; 6.6 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail 2024-04-07T17:20:10.455 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:20:09 smithi100 bash[74942]: cluster 2024-04-07T17:20:08.966683+0000 mon.a (mon.0) 3516 : cluster [DBG] osdmap e1841: 8 total, 8 up, 8 in 2024-04-07T17:20:10.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:20:09 smithi100 bash[76202]: cluster 2024-04-07T17:20:08.028031+0000 mgr.y (mgr.25175) 1943 : cluster [DBG] pgmap v3358: 617 pgs: 617 active+clean; 6.6 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail 2024-04-07T17:20:10.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:20:09 smithi100 bash[76202]: cluster 2024-04-07T17:20:08.966683+0000 mon.a (mon.0) 3516 : cluster [DBG] osdmap e1841: 8 total, 8 up, 8 in 2024-04-07T17:20:11.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:20:10 smithi144 bash[66654]: cluster 2024-04-07T17:20:09.972375+0000 mon.a (mon.0) 3517 : cluster [DBG] osdmap e1842: 8 total, 8 up, 8 in 2024-04-07T17:20:11.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:20:10 smithi100 bash[74942]: cluster 2024-04-07T17:20:09.972375+0000 mon.a (mon.0) 3517 : cluster [DBG] osdmap e1842: 8 total, 8 up, 8 in 2024-04-07T17:20:11.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:20:10 smithi100 bash[76202]: cluster 2024-04-07T17:20:09.972375+0000 mon.a (mon.0) 3517 : cluster [DBG] osdmap e1842: 8 total, 8 up, 8 in 2024-04-07T17:20:12.317 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:20:11 smithi144 bash[66654]: cluster 2024-04-07T17:20:10.028754+0000 mgr.y (mgr.25175) 1944 : cluster [DBG] pgmap v3361: 553 pgs: 553 active+clean; 6.6 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail 2024-04-07T17:20:12.318 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:20:11 smithi144 bash[66654]: cluster 2024-04-07T17:20:10.982844+0000 mon.a (mon.0) 3518 : cluster [DBG] osdmap e1843: 8 total, 8 up, 8 in 2024-04-07T17:20:12.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:20:11 smithi100 bash[74942]: cluster 2024-04-07T17:20:10.028754+0000 mgr.y (mgr.25175) 1944 : cluster [DBG] pgmap v3361: 553 pgs: 553 active+clean; 6.6 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail 2024-04-07T17:20:12.455 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:20:11 smithi100 bash[74942]: cluster 2024-04-07T17:20:10.982844+0000 mon.a (mon.0) 3518 : cluster [DBG] osdmap e1843: 8 total, 8 up, 8 in 2024-04-07T17:20:12.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:20:11 smithi100 bash[76202]: cluster 2024-04-07T17:20:10.028754+0000 mgr.y (mgr.25175) 1944 : cluster [DBG] pgmap v3361: 553 pgs: 553 active+clean; 6.6 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail 2024-04-07T17:20:12.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:20:11 smithi100 bash[76202]: cluster 2024-04-07T17:20:10.982844+0000 mon.a (mon.0) 3518 : cluster [DBG] osdmap e1843: 8 total, 8 up, 8 in 2024-04-07T17:20:12.617 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:20:12 smithi144 bash[95233]: ts=2024-04-07T17:20:12.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:20:13.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:20:13 smithi144 bash[66654]: cluster 2024-04-07T17:20:11.992608+0000 mon.a (mon.0) 3519 : cluster [DBG] osdmap e1844: 8 total, 8 up, 8 in 2024-04-07T17:20:13.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:20:13 smithi144 bash[66654]: cluster 2024-04-07T17:20:12.964738+0000 mon.a (mon.0) 3520 : cluster [DBG] osdmap e1845: 8 total, 8 up, 8 in 2024-04-07T17:20:13.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:20:13 smithi100 bash[74942]: cluster 2024-04-07T17:20:11.992608+0000 mon.a (mon.0) 3519 : cluster [DBG] osdmap e1844: 8 total, 8 up, 8 in 2024-04-07T17:20:13.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:20:13 smithi100 bash[74942]: cluster 2024-04-07T17:20:12.964738+0000 mon.a (mon.0) 3520 : cluster [DBG] osdmap e1845: 8 total, 8 up, 8 in 2024-04-07T17:20:13.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:20:13 smithi100 bash[76202]: cluster 2024-04-07T17:20:11.992608+0000 mon.a (mon.0) 3519 : cluster [DBG] osdmap e1844: 8 total, 8 up, 8 in 2024-04-07T17:20:13.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:20:13 smithi100 bash[76202]: cluster 2024-04-07T17:20:12.964738+0000 mon.a (mon.0) 3520 : cluster [DBG] osdmap e1845: 8 total, 8 up, 8 in 2024-04-07T17:20:14.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:20:14 smithi144 bash[66654]: cluster 2024-04-07T17:20:12.029558+0000 mgr.y (mgr.25175) 1945 : cluster [DBG] pgmap v3364: 489 pgs: 489 active+clean; 6.6 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail 2024-04-07T17:20:14.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:20:14 smithi144 bash[66654]: cluster 2024-04-07T17:20:13.964584+0000 mon.a (mon.0) 3521 : cluster [DBG] osdmap e1846: 8 total, 8 up, 8 in 2024-04-07T17:20:14.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:20:14 smithi100 bash[74942]: cluster 2024-04-07T17:20:12.029558+0000 mgr.y (mgr.25175) 1945 : cluster [DBG] pgmap v3364: 489 pgs: 489 active+clean; 6.6 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail 2024-04-07T17:20:14.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:20:14 smithi100 bash[74942]: cluster 2024-04-07T17:20:13.964584+0000 mon.a (mon.0) 3521 : cluster [DBG] osdmap e1846: 8 total, 8 up, 8 in 2024-04-07T17:20:14.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:20:14 smithi100 bash[76202]: cluster 2024-04-07T17:20:12.029558+0000 mgr.y (mgr.25175) 1945 : cluster [DBG] pgmap v3364: 489 pgs: 489 active+clean; 6.6 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail 2024-04-07T17:20:14.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:20:14 smithi100 bash[76202]: cluster 2024-04-07T17:20:13.964584+0000 mon.a (mon.0) 3521 : cluster [DBG] osdmap e1846: 8 total, 8 up, 8 in 2024-04-07T17:20:16.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:20:15 smithi144 bash[66654]: cluster 2024-04-07T17:20:14.030497+0000 mgr.y (mgr.25175) 1946 : cluster [DBG] pgmap v3367: 425 pgs: 425 active+clean; 6.6 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail 2024-04-07T17:20:16.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:20:15 smithi144 bash[66654]: cluster 2024-04-07T17:20:14.967888+0000 mon.a (mon.0) 3522 : cluster [DBG] osdmap e1847: 8 total, 8 up, 8 in 2024-04-07T17:20:16.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:20:15 smithi100 bash[74942]: cluster 2024-04-07T17:20:14.030497+0000 mgr.y (mgr.25175) 1946 : cluster [DBG] pgmap v3367: 425 pgs: 425 active+clean; 6.6 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail 2024-04-07T17:20:16.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:20:15 smithi100 bash[74942]: cluster 2024-04-07T17:20:14.967888+0000 mon.a (mon.0) 3522 : cluster [DBG] osdmap e1847: 8 total, 8 up, 8 in 2024-04-07T17:20:16.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:20:15 smithi100 bash[76202]: cluster 2024-04-07T17:20:14.030497+0000 mgr.y (mgr.25175) 1946 : cluster [DBG] pgmap v3367: 425 pgs: 425 active+clean; 6.6 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail 2024-04-07T17:20:16.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:20:15 smithi100 bash[76202]: cluster 2024-04-07T17:20:14.967888+0000 mon.a (mon.0) 3522 : cluster [DBG] osdmap e1847: 8 total, 8 up, 8 in 2024-04-07T17:20:17.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:20:17 smithi144 bash[66654]: cluster 2024-04-07T17:20:15.970916+0000 mon.a (mon.0) 3523 : cluster [DBG] osdmap e1848: 8 total, 8 up, 8 in 2024-04-07T17:20:17.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:20:17 smithi144 bash[66654]: audit 2024-04-07T17:20:16.833247+0000 mon.a (mon.0) 3524 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:20:17.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:20:17 smithi144 bash[66654]: cluster 2024-04-07T17:20:16.975679+0000 mon.a (mon.0) 3525 : cluster [DBG] osdmap e1849: 8 total, 8 up, 8 in 2024-04-07T17:20:17.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:20:17 smithi100 bash[74942]: cluster 2024-04-07T17:20:15.970916+0000 mon.a (mon.0) 3523 : cluster [DBG] osdmap e1848: 8 total, 8 up, 8 in 2024-04-07T17:20:17.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:20:17 smithi100 bash[74942]: audit 2024-04-07T17:20:16.833247+0000 mon.a (mon.0) 3524 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:20:17.455 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:20:17 smithi100 bash[74942]: cluster 2024-04-07T17:20:16.975679+0000 mon.a (mon.0) 3525 : cluster [DBG] osdmap e1849: 8 total, 8 up, 8 in 2024-04-07T17:20:17.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:20:17 smithi100 bash[76202]: cluster 2024-04-07T17:20:15.970916+0000 mon.a (mon.0) 3523 : cluster [DBG] osdmap e1848: 8 total, 8 up, 8 in 2024-04-07T17:20:17.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:20:17 smithi100 bash[76202]: audit 2024-04-07T17:20:16.833247+0000 mon.a (mon.0) 3524 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:20:17.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:20:17 smithi100 bash[76202]: cluster 2024-04-07T17:20:16.975679+0000 mon.a (mon.0) 3525 : cluster [DBG] osdmap e1849: 8 total, 8 up, 8 in 2024-04-07T17:20:18.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:20:18 smithi144 bash[66654]: cluster 2024-04-07T17:20:16.031339+0000 mgr.y (mgr.25175) 1947 : cluster [DBG] pgmap v3370: 361 pgs: 361 active+clean; 4.6 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail 2024-04-07T17:20:18.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:20:18 smithi144 bash[66654]: cluster 2024-04-07T17:20:17.976835+0000 mon.a (mon.0) 3526 : cluster [DBG] osdmap e1850: 8 total, 8 up, 8 in 2024-04-07T17:20:18.428 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:20:18 smithi100 bash[74942]: cluster 2024-04-07T17:20:16.031339+0000 mgr.y (mgr.25175) 1947 : cluster [DBG] pgmap v3370: 361 pgs: 361 active+clean; 4.6 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail 2024-04-07T17:20:18.428 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:20:18 smithi100 bash[74942]: cluster 2024-04-07T17:20:17.976835+0000 mon.a (mon.0) 3526 : cluster [DBG] osdmap e1850: 8 total, 8 up, 8 in 2024-04-07T17:20:18.428 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:20:18 smithi100 bash[76202]: cluster 2024-04-07T17:20:16.031339+0000 mgr.y (mgr.25175) 1947 : cluster [DBG] pgmap v3370: 361 pgs: 361 active+clean; 4.6 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail 2024-04-07T17:20:18.428 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:20:18 smithi100 bash[76202]: cluster 2024-04-07T17:20:17.976835+0000 mon.a (mon.0) 3526 : cluster [DBG] osdmap e1850: 8 total, 8 up, 8 in 2024-04-07T17:20:18.704 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:20:18 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:20:18] "GET /metrics HTTP/1.1" 200 53942 "" "Prometheus/2.43.0" 2024-04-07T17:20:20.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:20:19 smithi144 bash[66654]: cluster 2024-04-07T17:20:18.032308+0000 mgr.y (mgr.25175) 1948 : cluster [DBG] pgmap v3373: 297 pgs: 297 active+clean; 4.6 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail 2024-04-07T17:20:20.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:20:19 smithi144 bash[66654]: cluster 2024-04-07T17:20:18.978409+0000 mon.a (mon.0) 3527 : cluster [DBG] osdmap e1851: 8 total, 8 up, 8 in 2024-04-07T17:20:20.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:20:19 smithi100 bash[74942]: cluster 2024-04-07T17:20:18.032308+0000 mgr.y (mgr.25175) 1948 : cluster [DBG] pgmap v3373: 297 pgs: 297 active+clean; 4.6 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail 2024-04-07T17:20:20.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:20:19 smithi100 bash[74942]: cluster 2024-04-07T17:20:18.978409+0000 mon.a (mon.0) 3527 : cluster [DBG] osdmap e1851: 8 total, 8 up, 8 in 2024-04-07T17:20:20.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:20:19 smithi100 bash[76202]: cluster 2024-04-07T17:20:18.032308+0000 mgr.y (mgr.25175) 1948 : cluster [DBG] pgmap v3373: 297 pgs: 297 active+clean; 4.6 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail 2024-04-07T17:20:20.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:20:19 smithi100 bash[76202]: cluster 2024-04-07T17:20:18.978409+0000 mon.a (mon.0) 3527 : cluster [DBG] osdmap e1851: 8 total, 8 up, 8 in 2024-04-07T17:20:21.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:20:21 smithi144 bash[66654]: cluster 2024-04-07T17:20:19.981220+0000 mon.a (mon.0) 3528 : cluster [DBG] osdmap e1852: 8 total, 8 up, 8 in 2024-04-07T17:20:21.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:20:21 smithi144 bash[66654]: cluster 2024-04-07T17:20:20.983398+0000 mon.a (mon.0) 3529 : cluster [DBG] osdmap e1853: 8 total, 8 up, 8 in 2024-04-07T17:20:21.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:20:21 smithi100 bash[74942]: cluster 2024-04-07T17:20:19.981220+0000 mon.a (mon.0) 3528 : cluster [DBG] osdmap e1852: 8 total, 8 up, 8 in 2024-04-07T17:20:21.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:20:21 smithi100 bash[74942]: cluster 2024-04-07T17:20:20.983398+0000 mon.a (mon.0) 3529 : cluster [DBG] osdmap e1853: 8 total, 8 up, 8 in 2024-04-07T17:20:21.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:20:21 smithi100 bash[76202]: cluster 2024-04-07T17:20:19.981220+0000 mon.a (mon.0) 3528 : cluster [DBG] osdmap e1852: 8 total, 8 up, 8 in 2024-04-07T17:20:21.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:20:21 smithi100 bash[76202]: cluster 2024-04-07T17:20:20.983398+0000 mon.a (mon.0) 3529 : cluster [DBG] osdmap e1853: 8 total, 8 up, 8 in 2024-04-07T17:20:22.317 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:20:22 smithi144 bash[66654]: cluster 2024-04-07T17:20:20.033150+0000 mgr.y (mgr.25175) 1949 : cluster [DBG] pgmap v3376: 233 pgs: 233 active+clean; 4.6 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail 2024-04-07T17:20:22.318 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:20:22 smithi144 bash[66654]: cluster 2024-04-07T17:20:21.985398+0000 mon.a (mon.0) 3530 : cluster [DBG] osdmap e1854: 8 total, 8 up, 8 in 2024-04-07T17:20:22.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:20:22 smithi100 bash[74942]: cluster 2024-04-07T17:20:20.033150+0000 mgr.y (mgr.25175) 1949 : cluster [DBG] pgmap v3376: 233 pgs: 233 active+clean; 4.6 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail 2024-04-07T17:20:22.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:20:22 smithi100 bash[74942]: cluster 2024-04-07T17:20:21.985398+0000 mon.a (mon.0) 3530 : cluster [DBG] osdmap e1854: 8 total, 8 up, 8 in 2024-04-07T17:20:22.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:20:22 smithi100 bash[76202]: cluster 2024-04-07T17:20:20.033150+0000 mgr.y (mgr.25175) 1949 : cluster [DBG] pgmap v3376: 233 pgs: 233 active+clean; 4.6 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail 2024-04-07T17:20:22.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:20:22 smithi100 bash[76202]: cluster 2024-04-07T17:20:21.985398+0000 mon.a (mon.0) 3530 : cluster [DBG] osdmap e1854: 8 total, 8 up, 8 in 2024-04-07T17:20:22.617 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:20:22 smithi144 bash[95233]: ts=2024-04-07T17:20:22.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:20:24.001 INFO:tasks.workunit.client.0.smithi100.stdout:[----------] 168 tests from TestLibRBD (639345 ms total) 2024-04-07T17:20:24.001 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:20:24.001 INFO:tasks.workunit.client.0.smithi100.stdout:[----------] 7 tests from EncryptedFlattenTest/0, where TypeParam = PlaintextUnderLUKS1 2024-04-07T17:20:24.072 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] EncryptedFlattenTest/0.Simple 2024-04-07T17:20:24.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:20:24 smithi144 bash[66654]: cluster 2024-04-07T17:20:22.033833+0000 mgr.y (mgr.25175) 1950 : cluster [DBG] pgmap v3379: 169 pgs: 169 active+clean; 4.6 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail 2024-04-07T17:20:24.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:20:24 smithi144 bash[66654]: cluster 2024-04-07T17:20:22.989778+0000 mon.a (mon.0) 3531 : cluster [DBG] osdmap e1855: 8 total, 8 up, 8 in 2024-04-07T17:20:24.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:20:24 smithi100 bash[74942]: cluster 2024-04-07T17:20:22.033833+0000 mgr.y (mgr.25175) 1950 : cluster [DBG] pgmap v3379: 169 pgs: 169 active+clean; 4.6 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail 2024-04-07T17:20:24.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:20:24 smithi100 bash[74942]: cluster 2024-04-07T17:20:22.989778+0000 mon.a (mon.0) 3531 : cluster [DBG] osdmap e1855: 8 total, 8 up, 8 in 2024-04-07T17:20:24.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:20:24 smithi100 bash[76202]: cluster 2024-04-07T17:20:22.033833+0000 mgr.y (mgr.25175) 1950 : cluster [DBG] pgmap v3379: 169 pgs: 169 active+clean; 4.6 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail 2024-04-07T17:20:24.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:20:24 smithi100 bash[76202]: cluster 2024-04-07T17:20:22.989778+0000 mon.a (mon.0) 3531 : cluster [DBG] osdmap e1855: 8 total, 8 up, 8 in 2024-04-07T17:20:25.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:20:25 smithi144 bash[66654]: cluster 2024-04-07T17:20:23.992705+0000 mon.a (mon.0) 3532 : cluster [DBG] osdmap e1856: 8 total, 8 up, 8 in 2024-04-07T17:20:25.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:20:25 smithi100 bash[76202]: cluster 2024-04-07T17:20:23.992705+0000 mon.a (mon.0) 3532 : cluster [DBG] osdmap e1856: 8 total, 8 up, 8 in 2024-04-07T17:20:25.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:20:25 smithi100 bash[74942]: cluster 2024-04-07T17:20:23.992705+0000 mon.a (mon.0) 3532 : cluster [DBG] osdmap e1856: 8 total, 8 up, 8 in 2024-04-07T17:20:26.056 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:20:26.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:20:26 smithi144 bash[66654]: cluster 2024-04-07T17:20:24.034405+0000 mgr.y (mgr.25175) 1951 : cluster [DBG] pgmap v3382: 105 pgs: 105 active+clean; 4.6 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail 2024-04-07T17:20:26.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:20:26 smithi144 bash[66654]: cluster 2024-04-07T17:20:25.025139+0000 mon.a (mon.0) 3533 : cluster [DBG] osdmap e1857: 8 total, 8 up, 8 in 2024-04-07T17:20:26.368 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:20:26 smithi144 bash[66654]: audit 2024-04-07T17:20:25.034013+0000 mon.c (mon.2) 94 : audit [INF] from='client.? 172.21.15.100:0/975756040' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-36","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:20:26.368 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:20:26 smithi144 bash[66654]: audit 2024-04-07T17:20:25.035164+0000 mon.a (mon.0) 3534 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-36","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:20:26.368 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:20:26 smithi144 bash[66654]: audit 2024-04-07T17:20:25.261508+0000 mon.a (mon.0) 3535 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:20:26.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:20:26 smithi100 bash[74942]: cluster 2024-04-07T17:20:24.034405+0000 mgr.y (mgr.25175) 1951 : cluster [DBG] pgmap v3382: 105 pgs: 105 active+clean; 4.6 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail 2024-04-07T17:20:26.455 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:20:26 smithi100 bash[74942]: cluster 2024-04-07T17:20:25.025139+0000 mon.a (mon.0) 3533 : cluster [DBG] osdmap e1857: 8 total, 8 up, 8 in 2024-04-07T17:20:26.455 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:20:26 smithi100 bash[74942]: audit 2024-04-07T17:20:25.034013+0000 mon.c (mon.2) 94 : audit [INF] from='client.? 172.21.15.100:0/975756040' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-36","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:20:26.455 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:20:26 smithi100 bash[74942]: audit 2024-04-07T17:20:25.035164+0000 mon.a (mon.0) 3534 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-36","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:20:26.455 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:20:26 smithi100 bash[74942]: audit 2024-04-07T17:20:25.261508+0000 mon.a (mon.0) 3535 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:20:26.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:20:26 smithi100 bash[76202]: cluster 2024-04-07T17:20:24.034405+0000 mgr.y (mgr.25175) 1951 : cluster [DBG] pgmap v3382: 105 pgs: 105 active+clean; 4.6 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail 2024-04-07T17:20:26.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:20:26 smithi100 bash[76202]: cluster 2024-04-07T17:20:25.025139+0000 mon.a (mon.0) 3533 : cluster [DBG] osdmap e1857: 8 total, 8 up, 8 in 2024-04-07T17:20:26.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:20:26 smithi100 bash[76202]: audit 2024-04-07T17:20:25.034013+0000 mon.c (mon.2) 94 : audit [INF] from='client.? 172.21.15.100:0/975756040' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-36","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:20:26.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:20:26 smithi100 bash[76202]: audit 2024-04-07T17:20:25.035164+0000 mon.a (mon.0) 3534 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-36","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:20:26.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:20:26 smithi100 bash[76202]: audit 2024-04-07T17:20:25.261508+0000 mon.a (mon.0) 3535 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:20:27.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:20:27 smithi144 bash[66654]: audit 2024-04-07T17:20:26.032356+0000 mon.a (mon.0) 3536 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-36","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T17:20:27.368 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:20:27 smithi144 bash[66654]: cluster 2024-04-07T17:20:26.035293+0000 mgr.y (mgr.25175) 1952 : cluster [DBG] pgmap v3384: 137 pgs: 5 creating+peering, 27 unknown, 105 active+clean; 4.6 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail 2024-04-07T17:20:27.368 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:20:27 smithi144 bash[66654]: cluster 2024-04-07T17:20:26.042951+0000 mon.a (mon.0) 3537 : cluster [DBG] osdmap e1858: 8 total, 8 up, 8 in 2024-04-07T17:20:27.458 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:20:27 smithi100 bash[74942]: audit 2024-04-07T17:20:26.032356+0000 mon.a (mon.0) 3536 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-36","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T17:20:27.458 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:20:27 smithi100 bash[74942]: cluster 2024-04-07T17:20:26.035293+0000 mgr.y (mgr.25175) 1952 : cluster [DBG] pgmap v3384: 137 pgs: 5 creating+peering, 27 unknown, 105 active+clean; 4.6 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail 2024-04-07T17:20:27.458 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:20:27 smithi100 bash[74942]: cluster 2024-04-07T17:20:26.042951+0000 mon.a (mon.0) 3537 : cluster [DBG] osdmap e1858: 8 total, 8 up, 8 in 2024-04-07T17:20:27.458 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:20:27 smithi100 bash[76202]: audit 2024-04-07T17:20:26.032356+0000 mon.a (mon.0) 3536 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-36","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T17:20:27.459 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:20:27 smithi100 bash[76202]: cluster 2024-04-07T17:20:26.035293+0000 mgr.y (mgr.25175) 1952 : cluster [DBG] pgmap v3384: 137 pgs: 5 creating+peering, 27 unknown, 105 active+clean; 4.6 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail 2024-04-07T17:20:27.459 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:20:27 smithi100 bash[76202]: cluster 2024-04-07T17:20:26.042951+0000 mon.a (mon.0) 3537 : cluster [DBG] osdmap e1858: 8 total, 8 up, 8 in 2024-04-07T17:20:28.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:20:28 smithi144 bash[66654]: cluster 2024-04-07T17:20:27.033398+0000 mon.a (mon.0) 3538 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T17:20:28.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:20:28 smithi144 bash[66654]: cluster 2024-04-07T17:20:27.044091+0000 mon.a (mon.0) 3539 : cluster [DBG] osdmap e1859: 8 total, 8 up, 8 in 2024-04-07T17:20:28.429 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:20:28 smithi100 bash[74942]: cluster 2024-04-07T17:20:27.033398+0000 mon.a (mon.0) 3538 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T17:20:28.429 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:20:28 smithi100 bash[74942]: cluster 2024-04-07T17:20:27.044091+0000 mon.a (mon.0) 3539 : cluster [DBG] osdmap e1859: 8 total, 8 up, 8 in 2024-04-07T17:20:28.429 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:20:28 smithi100 bash[76202]: cluster 2024-04-07T17:20:27.033398+0000 mon.a (mon.0) 3538 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T17:20:28.429 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:20:28 smithi100 bash[76202]: cluster 2024-04-07T17:20:27.044091+0000 mon.a (mon.0) 3539 : cluster [DBG] osdmap e1859: 8 total, 8 up, 8 in 2024-04-07T17:20:28.704 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:20:28 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:20:28] "GET /metrics HTTP/1.1" 200 53942 "" "Prometheus/2.43.0" 2024-04-07T17:20:29.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:20:29 smithi144 bash[66654]: cluster 2024-04-07T17:20:28.036413+0000 mgr.y (mgr.25175) 1953 : cluster [DBG] pgmap v3387: 137 pgs: 5 creating+peering, 11 unknown, 121 active+clean; 4.6 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail 2024-04-07T17:20:29.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:20:29 smithi144 bash[66654]: cluster 2024-04-07T17:20:28.048603+0000 mon.a (mon.0) 3540 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T17:20:29.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:20:29 smithi144 bash[66654]: cluster 2024-04-07T17:20:28.070051+0000 mon.a (mon.0) 3541 : cluster [DBG] osdmap e1860: 8 total, 8 up, 8 in 2024-04-07T17:20:29.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:20:29 smithi100 bash[76202]: cluster 2024-04-07T17:20:28.036413+0000 mgr.y (mgr.25175) 1953 : cluster [DBG] pgmap v3387: 137 pgs: 5 creating+peering, 11 unknown, 121 active+clean; 4.6 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail 2024-04-07T17:20:29.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:20:29 smithi100 bash[76202]: cluster 2024-04-07T17:20:28.048603+0000 mon.a (mon.0) 3540 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T17:20:29.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:20:29 smithi100 bash[76202]: cluster 2024-04-07T17:20:28.070051+0000 mon.a (mon.0) 3541 : cluster [DBG] osdmap e1860: 8 total, 8 up, 8 in 2024-04-07T17:20:29.455 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:20:29 smithi100 bash[74942]: cluster 2024-04-07T17:20:28.036413+0000 mgr.y (mgr.25175) 1953 : cluster [DBG] pgmap v3387: 137 pgs: 5 creating+peering, 11 unknown, 121 active+clean; 4.6 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail 2024-04-07T17:20:29.455 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:20:29 smithi100 bash[74942]: cluster 2024-04-07T17:20:28.048603+0000 mon.a (mon.0) 3540 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T17:20:29.455 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:20:29 smithi100 bash[74942]: cluster 2024-04-07T17:20:28.070051+0000 mon.a (mon.0) 3541 : cluster [DBG] osdmap e1860: 8 total, 8 up, 8 in 2024-04-07T17:20:30.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:20:30 smithi144 bash[66654]: cluster 2024-04-07T17:20:29.071269+0000 mon.a (mon.0) 3542 : cluster [DBG] osdmap e1861: 8 total, 8 up, 8 in 2024-04-07T17:20:30.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:20:30 smithi100 bash[74942]: cluster 2024-04-07T17:20:29.071269+0000 mon.a (mon.0) 3542 : cluster [DBG] osdmap e1861: 8 total, 8 up, 8 in 2024-04-07T17:20:30.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:20:30 smithi100 bash[76202]: cluster 2024-04-07T17:20:29.071269+0000 mon.a (mon.0) 3542 : cluster [DBG] osdmap e1861: 8 total, 8 up, 8 in 2024-04-07T17:20:32.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:20:31 smithi144 bash[66654]: cluster 2024-04-07T17:20:30.037293+0000 mgr.y (mgr.25175) 1954 : cluster [DBG] pgmap v3390: 137 pgs: 5 creating+peering, 132 active+clean; 8.6 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 511 B/s rd, 1024 KiB/s wr, 1 op/s 2024-04-07T17:20:32.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:20:31 smithi144 bash[66654]: audit 2024-04-07T17:20:30.851026+0000 mon.a (mon.0) 3543 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:20:32.118 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:20:31 smithi144 bash[66654]: audit 2024-04-07T17:20:30.856348+0000 mon.a (mon.0) 3544 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:20:32.118 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:20:31 smithi144 bash[66654]: audit 2024-04-07T17:20:31.209024+0000 mon.a (mon.0) 3545 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:20:32.118 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:20:31 smithi144 bash[66654]: audit 2024-04-07T17:20:31.219051+0000 mon.a (mon.0) 3546 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:20:32.118 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:20:31 smithi144 bash[66654]: audit 2024-04-07T17:20:31.798218+0000 mon.a (mon.0) 3547 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:20:32.118 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:20:31 smithi144 bash[66654]: audit 2024-04-07T17:20:31.799896+0000 mon.a (mon.0) 3548 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:20:32.118 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:20:31 smithi144 bash[66654]: audit 2024-04-07T17:20:31.809663+0000 mon.a (mon.0) 3549 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:20:32.118 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:20:31 smithi144 bash[66654]: audit 2024-04-07T17:20:31.832414+0000 mon.a (mon.0) 3550 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:20:32.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:20:31 smithi100 bash[76202]: cluster 2024-04-07T17:20:30.037293+0000 mgr.y (mgr.25175) 1954 : cluster [DBG] pgmap v3390: 137 pgs: 5 creating+peering, 132 active+clean; 8.6 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 511 B/s rd, 1024 KiB/s wr, 1 op/s 2024-04-07T17:20:32.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:20:31 smithi100 bash[76202]: audit 2024-04-07T17:20:30.851026+0000 mon.a (mon.0) 3543 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:20:32.205 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:20:31 smithi100 bash[76202]: audit 2024-04-07T17:20:30.856348+0000 mon.a (mon.0) 3544 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:20:32.205 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:20:31 smithi100 bash[76202]: audit 2024-04-07T17:20:31.209024+0000 mon.a (mon.0) 3545 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:20:32.205 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:20:31 smithi100 bash[76202]: audit 2024-04-07T17:20:31.219051+0000 mon.a (mon.0) 3546 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:20:32.205 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:20:31 smithi100 bash[76202]: audit 2024-04-07T17:20:31.798218+0000 mon.a (mon.0) 3547 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:20:32.205 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:20:31 smithi100 bash[76202]: audit 2024-04-07T17:20:31.799896+0000 mon.a (mon.0) 3548 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:20:32.205 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:20:31 smithi100 bash[76202]: audit 2024-04-07T17:20:31.809663+0000 mon.a (mon.0) 3549 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:20:32.205 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:20:31 smithi100 bash[76202]: audit 2024-04-07T17:20:31.832414+0000 mon.a (mon.0) 3550 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:20:32.205 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:20:31 smithi100 bash[74942]: cluster 2024-04-07T17:20:30.037293+0000 mgr.y (mgr.25175) 1954 : cluster [DBG] pgmap v3390: 137 pgs: 5 creating+peering, 132 active+clean; 8.6 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 511 B/s rd, 1024 KiB/s wr, 1 op/s 2024-04-07T17:20:32.205 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:20:31 smithi100 bash[74942]: audit 2024-04-07T17:20:30.851026+0000 mon.a (mon.0) 3543 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:20:32.205 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:20:31 smithi100 bash[74942]: audit 2024-04-07T17:20:30.856348+0000 mon.a (mon.0) 3544 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:20:32.205 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:20:31 smithi100 bash[74942]: audit 2024-04-07T17:20:31.209024+0000 mon.a (mon.0) 3545 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:20:32.206 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:20:31 smithi100 bash[74942]: audit 2024-04-07T17:20:31.219051+0000 mon.a (mon.0) 3546 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:20:32.206 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:20:31 smithi100 bash[74942]: audit 2024-04-07T17:20:31.798218+0000 mon.a (mon.0) 3547 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:20:32.206 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:20:31 smithi100 bash[74942]: audit 2024-04-07T17:20:31.799896+0000 mon.a (mon.0) 3548 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:20:32.206 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:20:31 smithi100 bash[74942]: audit 2024-04-07T17:20:31.809663+0000 mon.a (mon.0) 3549 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:20:32.206 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:20:31 smithi100 bash[74942]: audit 2024-04-07T17:20:31.832414+0000 mon.a (mon.0) 3550 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:20:32.617 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:20:32 smithi144 bash[95233]: ts=2024-04-07T17:20:32.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:20:34.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:20:33 smithi144 bash[66654]: cluster 2024-04-07T17:20:32.038216+0000 mgr.y (mgr.25175) 1955 : cluster [DBG] pgmap v3391: 137 pgs: 137 active+clean; 17 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 28 KiB/s rd, 2.0 MiB/s wr, 36 op/s 2024-04-07T17:20:34.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:20:33 smithi100 bash[74942]: cluster 2024-04-07T17:20:32.038216+0000 mgr.y (mgr.25175) 1955 : cluster [DBG] pgmap v3391: 137 pgs: 137 active+clean; 17 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 28 KiB/s rd, 2.0 MiB/s wr, 36 op/s 2024-04-07T17:20:34.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:20:33 smithi100 bash[76202]: cluster 2024-04-07T17:20:32.038216+0000 mgr.y (mgr.25175) 1955 : cluster [DBG] pgmap v3391: 137 pgs: 137 active+clean; 17 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 28 KiB/s rd, 2.0 MiB/s wr, 36 op/s 2024-04-07T17:20:36.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:20:35 smithi144 bash[66654]: cluster 2024-04-07T17:20:34.039531+0000 mgr.y (mgr.25175) 1956 : cluster [DBG] pgmap v3392: 137 pgs: 137 active+clean; 30 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 31 KiB/s rd, 3.6 MiB/s wr, 43 op/s 2024-04-07T17:20:36.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:20:35 smithi100 bash[74942]: cluster 2024-04-07T17:20:34.039531+0000 mgr.y (mgr.25175) 1956 : cluster [DBG] pgmap v3392: 137 pgs: 137 active+clean; 30 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 31 KiB/s rd, 3.6 MiB/s wr, 43 op/s 2024-04-07T17:20:36.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:20:35 smithi100 bash[76202]: cluster 2024-04-07T17:20:34.039531+0000 mgr.y (mgr.25175) 1956 : cluster [DBG] pgmap v3392: 137 pgs: 137 active+clean; 30 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 31 KiB/s rd, 3.6 MiB/s wr, 43 op/s 2024-04-07T17:20:37.239 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] EncryptedFlattenTest/0.Simple (13167 ms) 2024-04-07T17:20:37.239 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] EncryptedFlattenTest/0.Grow 2024-04-07T17:20:37.239 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:20:38.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:20:37 smithi100 bash[74942]: cluster 2024-04-07T17:20:36.040567+0000 mgr.y (mgr.25175) 1957 : cluster [DBG] pgmap v3393: 137 pgs: 137 active+clean; 34 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 2.1 MiB/s rd, 3.6 MiB/s wr, 64 op/s 2024-04-07T17:20:38.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:20:37 smithi100 bash[76202]: cluster 2024-04-07T17:20:36.040567+0000 mgr.y (mgr.25175) 1957 : cluster [DBG] pgmap v3393: 137 pgs: 137 active+clean; 34 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 2.1 MiB/s rd, 3.6 MiB/s wr, 64 op/s 2024-04-07T17:20:38.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:20:37 smithi144 bash[66654]: cluster 2024-04-07T17:20:36.040567+0000 mgr.y (mgr.25175) 1957 : cluster [DBG] pgmap v3393: 137 pgs: 137 active+clean; 34 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 2.1 MiB/s rd, 3.6 MiB/s wr, 64 op/s 2024-04-07T17:20:38.704 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:20:38 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:20:38] "GET /metrics HTTP/1.1" 200 36739 "" "Prometheus/2.43.0" 2024-04-07T17:20:39.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:20:38 smithi100 bash[74942]: cluster 2024-04-07T17:20:37.871954+0000 mon.a (mon.0) 3551 : cluster [DBG] osdmap e1862: 8 total, 8 up, 8 in 2024-04-07T17:20:39.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:20:38 smithi100 bash[74942]: cluster 2024-04-07T17:20:37.969900+0000 mon.a (mon.0) 3552 : cluster [DBG] osdmap e1863: 8 total, 8 up, 8 in 2024-04-07T17:20:39.205 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:20:38 smithi100 bash[76202]: cluster 2024-04-07T17:20:37.871954+0000 mon.a (mon.0) 3551 : cluster [DBG] osdmap e1862: 8 total, 8 up, 8 in 2024-04-07T17:20:39.205 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:20:38 smithi100 bash[76202]: cluster 2024-04-07T17:20:37.969900+0000 mon.a (mon.0) 3552 : cluster [DBG] osdmap e1863: 8 total, 8 up, 8 in 2024-04-07T17:20:39.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:20:38 smithi144 bash[66654]: cluster 2024-04-07T17:20:37.871954+0000 mon.a (mon.0) 3551 : cluster [DBG] osdmap e1862: 8 total, 8 up, 8 in 2024-04-07T17:20:39.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:20:38 smithi144 bash[66654]: cluster 2024-04-07T17:20:37.969900+0000 mon.a (mon.0) 3552 : cluster [DBG] osdmap e1863: 8 total, 8 up, 8 in 2024-04-07T17:20:40.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:20:39 smithi144 bash[66654]: cluster 2024-04-07T17:20:38.041258+0000 mgr.y (mgr.25175) 1958 : cluster [DBG] pgmap v3396: 137 pgs: 137 active+clean; 50 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 7.4 MiB/s rd, 5.1 MiB/s wr, 67 op/s 2024-04-07T17:20:40.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:20:39 smithi100 bash[76202]: cluster 2024-04-07T17:20:38.041258+0000 mgr.y (mgr.25175) 1958 : cluster [DBG] pgmap v3396: 137 pgs: 137 active+clean; 50 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 7.4 MiB/s rd, 5.1 MiB/s wr, 67 op/s 2024-04-07T17:20:40.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:20:39 smithi100 bash[74942]: cluster 2024-04-07T17:20:38.041258+0000 mgr.y (mgr.25175) 1958 : cluster [DBG] pgmap v3396: 137 pgs: 137 active+clean; 50 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 7.4 MiB/s rd, 5.1 MiB/s wr, 67 op/s 2024-04-07T17:20:42.317 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:20:41 smithi144 bash[66654]: cluster 2024-04-07T17:20:40.042309+0000 mgr.y (mgr.25175) 1959 : cluster [DBG] pgmap v3397: 137 pgs: 137 active+clean; 71 MiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 11 MiB/s rd, 7.3 MiB/s wr, 60 op/s 2024-04-07T17:20:42.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:20:41 smithi100 bash[74942]: cluster 2024-04-07T17:20:40.042309+0000 mgr.y (mgr.25175) 1959 : cluster [DBG] pgmap v3397: 137 pgs: 137 active+clean; 71 MiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 11 MiB/s rd, 7.3 MiB/s wr, 60 op/s 2024-04-07T17:20:42.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:20:41 smithi100 bash[76202]: cluster 2024-04-07T17:20:40.042309+0000 mgr.y (mgr.25175) 1959 : cluster [DBG] pgmap v3397: 137 pgs: 137 active+clean; 71 MiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 11 MiB/s rd, 7.3 MiB/s wr, 60 op/s 2024-04-07T17:20:42.617 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:20:42 smithi144 bash[95233]: ts=2024-04-07T17:20:42.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:20:44.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:20:43 smithi144 bash[66654]: cluster 2024-04-07T17:20:42.043191+0000 mgr.y (mgr.25175) 1960 : cluster [DBG] pgmap v3398: 137 pgs: 137 active+clean; 71 MiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 11 MiB/s rd, 5.6 MiB/s wr, 75 op/s 2024-04-07T17:20:44.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:20:43 smithi100 bash[74942]: cluster 2024-04-07T17:20:42.043191+0000 mgr.y (mgr.25175) 1960 : cluster [DBG] pgmap v3398: 137 pgs: 137 active+clean; 71 MiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 11 MiB/s rd, 5.6 MiB/s wr, 75 op/s 2024-04-07T17:20:44.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:20:43 smithi100 bash[76202]: cluster 2024-04-07T17:20:42.043191+0000 mgr.y (mgr.25175) 1960 : cluster [DBG] pgmap v3398: 137 pgs: 137 active+clean; 71 MiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 11 MiB/s rd, 5.6 MiB/s wr, 75 op/s 2024-04-07T17:20:46.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:20:45 smithi144 bash[66654]: cluster 2024-04-07T17:20:44.044476+0000 mgr.y (mgr.25175) 1961 : cluster [DBG] pgmap v3399: 137 pgs: 137 active+clean; 88 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 11 MiB/s rd, 7.3 MiB/s wr, 61 op/s 2024-04-07T17:20:46.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:20:45 smithi100 bash[74942]: cluster 2024-04-07T17:20:44.044476+0000 mgr.y (mgr.25175) 1961 : cluster [DBG] pgmap v3399: 137 pgs: 137 active+clean; 88 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 11 MiB/s rd, 7.3 MiB/s wr, 61 op/s 2024-04-07T17:20:46.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:20:45 smithi100 bash[76202]: cluster 2024-04-07T17:20:44.044476+0000 mgr.y (mgr.25175) 1961 : cluster [DBG] pgmap v3399: 137 pgs: 137 active+clean; 88 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 11 MiB/s rd, 7.3 MiB/s wr, 61 op/s 2024-04-07T17:20:46.508 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] EncryptedFlattenTest/0.Grow (9269 ms) 2024-04-07T17:20:46.508 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] EncryptedFlattenTest/0.Shrink 2024-04-07T17:20:46.508 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:20:47.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:20:46 smithi144 bash[66654]: audit 2024-04-07T17:20:46.833070+0000 mon.a (mon.0) 3553 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:20:47.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:20:46 smithi100 bash[74942]: audit 2024-04-07T17:20:46.833070+0000 mon.a (mon.0) 3553 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:20:47.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:20:46 smithi100 bash[76202]: audit 2024-04-07T17:20:46.833070+0000 mon.a (mon.0) 3553 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:20:48.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:20:47 smithi144 bash[66654]: cluster 2024-04-07T17:20:46.045533+0000 mgr.y (mgr.25175) 1962 : cluster [DBG] pgmap v3400: 137 pgs: 137 active+clean; 96 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 7.7 MiB/s rd, 6.1 MiB/s wr, 70 op/s 2024-04-07T17:20:48.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:20:47 smithi144 bash[66654]: cluster 2024-04-07T17:20:46.997302+0000 mon.a (mon.0) 3554 : cluster [DBG] osdmap e1864: 8 total, 8 up, 8 in 2024-04-07T17:20:48.428 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:20:47 smithi100 bash[76202]: cluster 2024-04-07T17:20:46.045533+0000 mgr.y (mgr.25175) 1962 : cluster [DBG] pgmap v3400: 137 pgs: 137 active+clean; 96 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 7.7 MiB/s rd, 6.1 MiB/s wr, 70 op/s 2024-04-07T17:20:48.428 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:20:47 smithi100 bash[76202]: cluster 2024-04-07T17:20:46.997302+0000 mon.a (mon.0) 3554 : cluster [DBG] osdmap e1864: 8 total, 8 up, 8 in 2024-04-07T17:20:48.428 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:20:47 smithi100 bash[74942]: cluster 2024-04-07T17:20:46.045533+0000 mgr.y (mgr.25175) 1962 : cluster [DBG] pgmap v3400: 137 pgs: 137 active+clean; 96 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 7.7 MiB/s rd, 6.1 MiB/s wr, 70 op/s 2024-04-07T17:20:48.428 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:20:47 smithi100 bash[74942]: cluster 2024-04-07T17:20:46.997302+0000 mon.a (mon.0) 3554 : cluster [DBG] osdmap e1864: 8 total, 8 up, 8 in 2024-04-07T17:20:48.704 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:20:48 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:20:48] "GET /metrics HTTP/1.1" 200 36753 "" "Prometheus/2.43.0" 2024-04-07T17:20:50.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:20:49 smithi144 bash[66654]: cluster 2024-04-07T17:20:48.046699+0000 mgr.y (mgr.25175) 1963 : cluster [DBG] pgmap v3402: 137 pgs: 137 active+clean; 122 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 11 MiB/s rd, 8.0 MiB/s wr, 61 op/s 2024-04-07T17:20:50.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:20:49 smithi100 bash[74942]: cluster 2024-04-07T17:20:48.046699+0000 mgr.y (mgr.25175) 1963 : cluster [DBG] pgmap v3402: 137 pgs: 137 active+clean; 122 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 11 MiB/s rd, 8.0 MiB/s wr, 61 op/s 2024-04-07T17:20:50.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:20:49 smithi100 bash[76202]: cluster 2024-04-07T17:20:48.046699+0000 mgr.y (mgr.25175) 1963 : cluster [DBG] pgmap v3402: 137 pgs: 137 active+clean; 122 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 11 MiB/s rd, 8.0 MiB/s wr, 61 op/s 2024-04-07T17:20:52.317 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:20:52 smithi144 bash[66654]: cluster 2024-04-07T17:20:50.047739+0000 mgr.y (mgr.25175) 1964 : cluster [DBG] pgmap v3403: 137 pgs: 137 active+clean; 130 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 11 MiB/s rd, 6.3 MiB/s wr, 87 op/s 2024-04-07T17:20:52.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:20:51 smithi100 bash[74942]: cluster 2024-04-07T17:20:50.047739+0000 mgr.y (mgr.25175) 1964 : cluster [DBG] pgmap v3403: 137 pgs: 137 active+clean; 130 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 11 MiB/s rd, 6.3 MiB/s wr, 87 op/s 2024-04-07T17:20:52.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:20:52 smithi100 bash[76202]: cluster 2024-04-07T17:20:50.047739+0000 mgr.y (mgr.25175) 1964 : cluster [DBG] pgmap v3403: 137 pgs: 137 active+clean; 130 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 11 MiB/s rd, 6.3 MiB/s wr, 87 op/s 2024-04-07T17:20:52.617 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:20:52 smithi144 bash[95233]: ts=2024-04-07T17:20:52.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:20:54.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:20:54 smithi144 bash[66654]: cluster 2024-04-07T17:20:52.048591+0000 mgr.y (mgr.25175) 1965 : cluster [DBG] pgmap v3404: 137 pgs: 137 active+clean; 138 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 12 MiB/s rd, 7.1 MiB/s wr, 74 op/s 2024-04-07T17:20:54.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:20:54 smithi100 bash[74942]: cluster 2024-04-07T17:20:52.048591+0000 mgr.y (mgr.25175) 1965 : cluster [DBG] pgmap v3404: 137 pgs: 137 active+clean; 138 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 12 MiB/s rd, 7.1 MiB/s wr, 74 op/s 2024-04-07T17:20:54.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:20:54 smithi100 bash[76202]: cluster 2024-04-07T17:20:52.048591+0000 mgr.y (mgr.25175) 1965 : cluster [DBG] pgmap v3404: 137 pgs: 137 active+clean; 138 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 12 MiB/s rd, 7.1 MiB/s wr, 74 op/s 2024-04-07T17:20:55.558 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] EncryptedFlattenTest/0.Shrink (9050 ms) 2024-04-07T17:20:55.558 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] EncryptedFlattenTest/0.ShrinkToOne 2024-04-07T17:20:55.558 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:20:56.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:20:56 smithi144 bash[66654]: cluster 2024-04-07T17:20:54.049859+0000 mgr.y (mgr.25175) 1966 : cluster [DBG] pgmap v3405: 137 pgs: 137 active+clean; 138 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 10 MiB/s rd, 5.4 MiB/s wr, 66 op/s 2024-04-07T17:20:56.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:20:56 smithi100 bash[76202]: cluster 2024-04-07T17:20:54.049859+0000 mgr.y (mgr.25175) 1966 : cluster [DBG] pgmap v3405: 137 pgs: 137 active+clean; 138 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 10 MiB/s rd, 5.4 MiB/s wr, 66 op/s 2024-04-07T17:20:56.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:20:56 smithi100 bash[74942]: cluster 2024-04-07T17:20:54.049859+0000 mgr.y (mgr.25175) 1966 : cluster [DBG] pgmap v3405: 137 pgs: 137 active+clean; 138 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 10 MiB/s rd, 5.4 MiB/s wr, 66 op/s 2024-04-07T17:20:57.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:20:57 smithi144 bash[66654]: cluster 2024-04-07T17:20:56.023159+0000 mon.a (mon.0) 3555 : cluster [DBG] osdmap e1865: 8 total, 8 up, 8 in 2024-04-07T17:20:57.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:20:57 smithi100 bash[74942]: cluster 2024-04-07T17:20:56.023159+0000 mon.a (mon.0) 3555 : cluster [DBG] osdmap e1865: 8 total, 8 up, 8 in 2024-04-07T17:20:57.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:20:57 smithi100 bash[76202]: cluster 2024-04-07T17:20:56.023159+0000 mon.a (mon.0) 3555 : cluster [DBG] osdmap e1865: 8 total, 8 up, 8 in 2024-04-07T17:20:58.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:20:58 smithi144 bash[66654]: cluster 2024-04-07T17:20:56.050551+0000 mgr.y (mgr.25175) 1967 : cluster [DBG] pgmap v3407: 137 pgs: 137 active+clean; 163 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 15 MiB/s rd, 8.3 MiB/s wr, 81 op/s 2024-04-07T17:20:58.427 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:20:58 smithi100 bash[74942]: cluster 2024-04-07T17:20:56.050551+0000 mgr.y (mgr.25175) 1967 : cluster [DBG] pgmap v3407: 137 pgs: 137 active+clean; 163 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 15 MiB/s rd, 8.3 MiB/s wr, 81 op/s 2024-04-07T17:20:58.428 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:20:58 smithi100 bash[76202]: cluster 2024-04-07T17:20:56.050551+0000 mgr.y (mgr.25175) 1967 : cluster [DBG] pgmap v3407: 137 pgs: 137 active+clean; 163 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 15 MiB/s rd, 8.3 MiB/s wr, 81 op/s 2024-04-07T17:20:58.704 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:20:58 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:20:58] "GET /metrics HTTP/1.1" 200 36753 "" "Prometheus/2.43.0" 2024-04-07T17:20:59.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:20:59 smithi144 bash[66654]: cluster 2024-04-07T17:20:58.051759+0000 mgr.y (mgr.25175) 1968 : cluster [DBG] pgmap v3408: 137 pgs: 137 active+clean; 172 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 12 MiB/s rd, 5.4 MiB/s wr, 76 op/s 2024-04-07T17:20:59.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:20:59 smithi100 bash[76202]: cluster 2024-04-07T17:20:58.051759+0000 mgr.y (mgr.25175) 1968 : cluster [DBG] pgmap v3408: 137 pgs: 137 active+clean; 172 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 12 MiB/s rd, 5.4 MiB/s wr, 76 op/s 2024-04-07T17:20:59.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:20:59 smithi100 bash[74942]: cluster 2024-04-07T17:20:58.051759+0000 mgr.y (mgr.25175) 1968 : cluster [DBG] pgmap v3408: 137 pgs: 137 active+clean; 172 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 12 MiB/s rd, 5.4 MiB/s wr, 76 op/s 2024-04-07T17:21:01.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:21:01 smithi144 bash[66654]: cluster 2024-04-07T17:21:00.052853+0000 mgr.y (mgr.25175) 1969 : cluster [DBG] pgmap v3409: 137 pgs: 137 active+clean; 188 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 11 MiB/s rd, 6.2 MiB/s wr, 64 op/s 2024-04-07T17:21:01.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:21:01 smithi100 bash[74942]: cluster 2024-04-07T17:21:00.052853+0000 mgr.y (mgr.25175) 1969 : cluster [DBG] pgmap v3409: 137 pgs: 137 active+clean; 188 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 11 MiB/s rd, 6.2 MiB/s wr, 64 op/s 2024-04-07T17:21:01.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:21:01 smithi100 bash[76202]: cluster 2024-04-07T17:21:00.052853+0000 mgr.y (mgr.25175) 1969 : cluster [DBG] pgmap v3409: 137 pgs: 137 active+clean; 188 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 11 MiB/s rd, 6.2 MiB/s wr, 64 op/s 2024-04-07T17:21:02.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:21:02 smithi144 bash[66654]: audit 2024-04-07T17:21:01.833176+0000 mon.a (mon.0) 3556 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:21:02.367 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:21:02 smithi144 bash[95233]: ts=2024-04-07T17:21:02.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:21:02.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:21:02 smithi100 bash[74942]: audit 2024-04-07T17:21:01.833176+0000 mon.a (mon.0) 3556 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:21:02.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:21:02 smithi100 bash[76202]: audit 2024-04-07T17:21:01.833176+0000 mon.a (mon.0) 3556 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:21:03.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:21:03 smithi144 bash[66654]: cluster 2024-04-07T17:21:02.053679+0000 mgr.y (mgr.25175) 1970 : cluster [DBG] pgmap v3410: 137 pgs: 137 active+clean; 196 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 11 MiB/s rd, 6.2 MiB/s wr, 66 op/s 2024-04-07T17:21:03.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:21:03 smithi100 bash[74942]: cluster 2024-04-07T17:21:02.053679+0000 mgr.y (mgr.25175) 1970 : cluster [DBG] pgmap v3410: 137 pgs: 137 active+clean; 196 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 11 MiB/s rd, 6.2 MiB/s wr, 66 op/s 2024-04-07T17:21:03.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:21:03 smithi100 bash[76202]: cluster 2024-04-07T17:21:02.053679+0000 mgr.y (mgr.25175) 1970 : cluster [DBG] pgmap v3410: 137 pgs: 137 active+clean; 196 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 11 MiB/s rd, 6.2 MiB/s wr, 66 op/s 2024-04-07T17:21:04.452 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] EncryptedFlattenTest/0.ShrinkToOne (8894 ms) 2024-04-07T17:21:04.452 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] EncryptedFlattenTest/0.ShrinkToOneAfterSnapshot 2024-04-07T17:21:04.452 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:21:05.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:21:05 smithi144 bash[66654]: cluster 2024-04-07T17:21:04.054964+0000 mgr.y (mgr.25175) 1971 : cluster [DBG] pgmap v3411: 137 pgs: 137 active+clean; 200 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 11 MiB/s rd, 7.0 MiB/s wr, 75 op/s 2024-04-07T17:21:05.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:21:05 smithi100 bash[74942]: cluster 2024-04-07T17:21:04.054964+0000 mgr.y (mgr.25175) 1971 : cluster [DBG] pgmap v3411: 137 pgs: 137 active+clean; 200 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 11 MiB/s rd, 7.0 MiB/s wr, 75 op/s 2024-04-07T17:21:05.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:21:05 smithi100 bash[76202]: cluster 2024-04-07T17:21:04.054964+0000 mgr.y (mgr.25175) 1971 : cluster [DBG] pgmap v3411: 137 pgs: 137 active+clean; 200 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 11 MiB/s rd, 7.0 MiB/s wr, 75 op/s 2024-04-07T17:21:06.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:21:06 smithi100 bash[74942]: cluster 2024-04-07T17:21:05.114452+0000 mon.a (mon.0) 3557 : cluster [DBG] osdmap e1866: 8 total, 8 up, 8 in 2024-04-07T17:21:06.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:21:06 smithi100 bash[76202]: cluster 2024-04-07T17:21:05.114452+0000 mon.a (mon.0) 3557 : cluster [DBG] osdmap e1866: 8 total, 8 up, 8 in 2024-04-07T17:21:06.617 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:21:06 smithi144 bash[66654]: cluster 2024-04-07T17:21:05.114452+0000 mon.a (mon.0) 3557 : cluster [DBG] osdmap e1866: 8 total, 8 up, 8 in 2024-04-07T17:21:07.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:21:07 smithi100 bash[74942]: cluster 2024-04-07T17:21:06.055653+0000 mgr.y (mgr.25175) 1972 : cluster [DBG] pgmap v3413: 137 pgs: 137 active+clean; 204 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 7.1 MiB/s rd, 4.9 MiB/s wr, 58 op/s 2024-04-07T17:21:07.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:21:07 smithi100 bash[76202]: cluster 2024-04-07T17:21:06.055653+0000 mgr.y (mgr.25175) 1972 : cluster [DBG] pgmap v3413: 137 pgs: 137 active+clean; 204 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 7.1 MiB/s rd, 4.9 MiB/s wr, 58 op/s 2024-04-07T17:21:07.617 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:21:07 smithi144 bash[66654]: cluster 2024-04-07T17:21:06.055653+0000 mgr.y (mgr.25175) 1972 : cluster [DBG] pgmap v3413: 137 pgs: 137 active+clean; 204 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 7.1 MiB/s rd, 4.9 MiB/s wr, 58 op/s 2024-04-07T17:21:08.704 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:21:08 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:21:08] "GET /metrics HTTP/1.1" 200 36758 "" "Prometheus/2.43.0" 2024-04-07T17:21:09.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:21:09 smithi100 bash[74942]: cluster 2024-04-07T17:21:08.056747+0000 mgr.y (mgr.25175) 1973 : cluster [DBG] pgmap v3414: 137 pgs: 137 active+clean; 217 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 3.8 MiB/s rd, 5.3 MiB/s wr, 84 op/s 2024-04-07T17:21:09.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:21:09 smithi100 bash[76202]: cluster 2024-04-07T17:21:08.056747+0000 mgr.y (mgr.25175) 1973 : cluster [DBG] pgmap v3414: 137 pgs: 137 active+clean; 217 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 3.8 MiB/s rd, 5.3 MiB/s wr, 84 op/s 2024-04-07T17:21:09.617 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:21:09 smithi144 bash[66654]: cluster 2024-04-07T17:21:08.056747+0000 mgr.y (mgr.25175) 1973 : cluster [DBG] pgmap v3414: 137 pgs: 137 active+clean; 217 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 3.8 MiB/s rd, 5.3 MiB/s wr, 84 op/s 2024-04-07T17:21:11.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:21:11 smithi100 bash[74942]: cluster 2024-04-07T17:21:10.057776+0000 mgr.y (mgr.25175) 1974 : cluster [DBG] pgmap v3415: 137 pgs: 137 active+clean; 225 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 1.3 MiB/s rd, 4.5 MiB/s wr, 65 op/s 2024-04-07T17:21:11.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:21:11 smithi100 bash[76202]: cluster 2024-04-07T17:21:10.057776+0000 mgr.y (mgr.25175) 1974 : cluster [DBG] pgmap v3415: 137 pgs: 137 active+clean; 225 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 1.3 MiB/s rd, 4.5 MiB/s wr, 65 op/s 2024-04-07T17:21:11.617 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:21:11 smithi144 bash[66654]: cluster 2024-04-07T17:21:10.057776+0000 mgr.y (mgr.25175) 1974 : cluster [DBG] pgmap v3415: 137 pgs: 137 active+clean; 225 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 1.3 MiB/s rd, 4.5 MiB/s wr, 65 op/s 2024-04-07T17:21:12.617 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:21:12 smithi144 bash[95233]: ts=2024-04-07T17:21:12.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:21:13.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:21:13 smithi100 bash[74942]: cluster 2024-04-07T17:21:12.058691+0000 mgr.y (mgr.25175) 1975 : cluster [DBG] pgmap v3416: 137 pgs: 137 active+clean; 225 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 917 KiB/s rd, 3.7 MiB/s wr, 56 op/s 2024-04-07T17:21:13.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:21:13 smithi100 bash[74942]: cluster 2024-04-07T17:21:12.148036+0000 mon.a (mon.0) 3558 : cluster [DBG] osdmap e1867: 8 total, 8 up, 8 in 2024-04-07T17:21:13.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:21:13 smithi100 bash[76202]: cluster 2024-04-07T17:21:12.058691+0000 mgr.y (mgr.25175) 1975 : cluster [DBG] pgmap v3416: 137 pgs: 137 active+clean; 225 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 917 KiB/s rd, 3.7 MiB/s wr, 56 op/s 2024-04-07T17:21:13.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:21:13 smithi100 bash[76202]: cluster 2024-04-07T17:21:12.148036+0000 mon.a (mon.0) 3558 : cluster [DBG] osdmap e1867: 8 total, 8 up, 8 in 2024-04-07T17:21:13.617 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:21:13 smithi144 bash[66654]: cluster 2024-04-07T17:21:12.058691+0000 mgr.y (mgr.25175) 1975 : cluster [DBG] pgmap v3416: 137 pgs: 137 active+clean; 225 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 917 KiB/s rd, 3.7 MiB/s wr, 56 op/s 2024-04-07T17:21:13.617 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:21:13 smithi144 bash[66654]: cluster 2024-04-07T17:21:12.148036+0000 mon.a (mon.0) 3558 : cluster [DBG] osdmap e1867: 8 total, 8 up, 8 in 2024-04-07T17:21:14.653 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] EncryptedFlattenTest/0.ShrinkToOneAfterSnapshot (10202 ms) 2024-04-07T17:21:14.653 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] EncryptedFlattenTest/0.MinOverlap 2024-04-07T17:21:14.653 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:21:15.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:21:15 smithi100 bash[74942]: cluster 2024-04-07T17:21:14.059895+0000 mgr.y (mgr.25175) 1976 : cluster [DBG] pgmap v3418: 137 pgs: 137 active+clean; 233 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 505 KiB/s rd, 3.2 MiB/s wr, 65 op/s 2024-04-07T17:21:15.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:21:15 smithi100 bash[76202]: cluster 2024-04-07T17:21:14.059895+0000 mgr.y (mgr.25175) 1976 : cluster [DBG] pgmap v3418: 137 pgs: 137 active+clean; 233 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 505 KiB/s rd, 3.2 MiB/s wr, 65 op/s 2024-04-07T17:21:15.617 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:21:15 smithi144 bash[66654]: cluster 2024-04-07T17:21:14.059895+0000 mgr.y (mgr.25175) 1976 : cluster [DBG] pgmap v3418: 137 pgs: 137 active+clean; 233 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 505 KiB/s rd, 3.2 MiB/s wr, 65 op/s 2024-04-07T17:21:16.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:21:16 smithi100 bash[74942]: cluster 2024-04-07T17:21:15.164164+0000 mon.a (mon.0) 3559 : cluster [DBG] osdmap e1868: 8 total, 8 up, 8 in 2024-04-07T17:21:16.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:21:16 smithi100 bash[76202]: cluster 2024-04-07T17:21:15.164164+0000 mon.a (mon.0) 3559 : cluster [DBG] osdmap e1868: 8 total, 8 up, 8 in 2024-04-07T17:21:16.617 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:21:16 smithi144 bash[66654]: cluster 2024-04-07T17:21:15.164164+0000 mon.a (mon.0) 3559 : cluster [DBG] osdmap e1868: 8 total, 8 up, 8 in 2024-04-07T17:21:17.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:21:17 smithi100 bash[74942]: cluster 2024-04-07T17:21:16.060353+0000 mgr.y (mgr.25175) 1977 : cluster [DBG] pgmap v3420: 137 pgs: 137 active+clean; 266 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 2.1 MiB/s rd, 6.1 MiB/s wr, 35 op/s 2024-04-07T17:21:17.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:21:17 smithi100 bash[74942]: audit 2024-04-07T17:21:16.833785+0000 mon.a (mon.0) 3560 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:21:17.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:21:17 smithi100 bash[76202]: cluster 2024-04-07T17:21:16.060353+0000 mgr.y (mgr.25175) 1977 : cluster [DBG] pgmap v3420: 137 pgs: 137 active+clean; 266 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 2.1 MiB/s rd, 6.1 MiB/s wr, 35 op/s 2024-04-07T17:21:17.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:21:17 smithi100 bash[76202]: audit 2024-04-07T17:21:16.833785+0000 mon.a (mon.0) 3560 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:21:17.617 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:21:17 smithi144 bash[66654]: cluster 2024-04-07T17:21:16.060353+0000 mgr.y (mgr.25175) 1977 : cluster [DBG] pgmap v3420: 137 pgs: 137 active+clean; 266 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 2.1 MiB/s rd, 6.1 MiB/s wr, 35 op/s 2024-04-07T17:21:17.617 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:21:17 smithi144 bash[66654]: audit 2024-04-07T17:21:16.833785+0000 mon.a (mon.0) 3560 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:21:18.704 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:21:18 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:21:18] "GET /metrics HTTP/1.1" 200 36762 "" "Prometheus/2.43.0" 2024-04-07T17:21:19.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:21:19 smithi100 bash[74942]: cluster 2024-04-07T17:21:18.061003+0000 mgr.y (mgr.25175) 1978 : cluster [DBG] pgmap v3421: 137 pgs: 137 active+clean; 299 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 3.8 MiB/s rd, 9.3 MiB/s wr, 74 op/s 2024-04-07T17:21:19.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:21:19 smithi100 bash[76202]: cluster 2024-04-07T17:21:18.061003+0000 mgr.y (mgr.25175) 1978 : cluster [DBG] pgmap v3421: 137 pgs: 137 active+clean; 299 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 3.8 MiB/s rd, 9.3 MiB/s wr, 74 op/s 2024-04-07T17:21:19.617 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:21:19 smithi144 bash[66654]: cluster 2024-04-07T17:21:18.061003+0000 mgr.y (mgr.25175) 1978 : cluster [DBG] pgmap v3421: 137 pgs: 137 active+clean; 299 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 3.8 MiB/s rd, 9.3 MiB/s wr, 74 op/s 2024-04-07T17:21:21.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:21:21 smithi100 bash[74942]: cluster 2024-04-07T17:21:20.062009+0000 mgr.y (mgr.25175) 1979 : cluster [DBG] pgmap v3422: 137 pgs: 137 active+clean; 311 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 3.8 MiB/s rd, 11 MiB/s wr, 93 op/s 2024-04-07T17:21:21.516 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:21:21 smithi100 bash[76202]: cluster 2024-04-07T17:21:20.062009+0000 mgr.y (mgr.25175) 1979 : cluster [DBG] pgmap v3422: 137 pgs: 137 active+clean; 311 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 3.8 MiB/s rd, 11 MiB/s wr, 93 op/s 2024-04-07T17:21:21.617 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:21:21 smithi144 bash[66654]: cluster 2024-04-07T17:21:20.062009+0000 mgr.y (mgr.25175) 1979 : cluster [DBG] pgmap v3422: 137 pgs: 137 active+clean; 311 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 3.8 MiB/s rd, 11 MiB/s wr, 93 op/s 2024-04-07T17:21:22.617 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:21:22 smithi144 bash[95233]: ts=2024-04-07T17:21:22.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:21:23.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:21:23 smithi100 bash[76202]: cluster 2024-04-07T17:21:22.063027+0000 mgr.y (mgr.25175) 1980 : cluster [DBG] pgmap v3423: 137 pgs: 137 active+clean; 311 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 3.1 MiB/s rd, 8.7 MiB/s wr, 70 op/s 2024-04-07T17:21:23.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:21:23 smithi100 bash[74942]: cluster 2024-04-07T17:21:22.063027+0000 mgr.y (mgr.25175) 1980 : cluster [DBG] pgmap v3423: 137 pgs: 137 active+clean; 311 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 3.1 MiB/s rd, 8.7 MiB/s wr, 70 op/s 2024-04-07T17:21:23.617 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:21:23 smithi144 bash[66654]: cluster 2024-04-07T17:21:22.063027+0000 mgr.y (mgr.25175) 1980 : cluster [DBG] pgmap v3423: 137 pgs: 137 active+clean; 311 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 3.1 MiB/s rd, 8.7 MiB/s wr, 70 op/s 2024-04-07T17:21:23.719 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] EncryptedFlattenTest/0.MinOverlap (9066 ms) 2024-04-07T17:21:23.719 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] EncryptedFlattenTest/0.ZeroOverlap 2024-04-07T17:21:23.719 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:21:25.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:21:25 smithi100 bash[74942]: cluster 2024-04-07T17:21:24.064502+0000 mgr.y (mgr.25175) 1981 : cluster [DBG] pgmap v3424: 137 pgs: 137 active+clean; 315 MiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 3.1 MiB/s rd, 8.6 MiB/s wr, 69 op/s 2024-04-07T17:21:25.455 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:21:25 smithi100 bash[74942]: cluster 2024-04-07T17:21:24.200325+0000 mon.a (mon.0) 3561 : cluster [DBG] osdmap e1869: 8 total, 8 up, 8 in 2024-04-07T17:21:25.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:21:25 smithi100 bash[76202]: cluster 2024-04-07T17:21:24.064502+0000 mgr.y (mgr.25175) 1981 : cluster [DBG] pgmap v3424: 137 pgs: 137 active+clean; 315 MiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 3.1 MiB/s rd, 8.6 MiB/s wr, 69 op/s 2024-04-07T17:21:25.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:21:25 smithi100 bash[76202]: cluster 2024-04-07T17:21:24.200325+0000 mon.a (mon.0) 3561 : cluster [DBG] osdmap e1869: 8 total, 8 up, 8 in 2024-04-07T17:21:25.617 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:21:25 smithi144 bash[66654]: cluster 2024-04-07T17:21:24.064502+0000 mgr.y (mgr.25175) 1981 : cluster [DBG] pgmap v3424: 137 pgs: 137 active+clean; 315 MiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 3.1 MiB/s rd, 8.6 MiB/s wr, 69 op/s 2024-04-07T17:21:25.617 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:21:25 smithi144 bash[66654]: cluster 2024-04-07T17:21:24.200325+0000 mon.a (mon.0) 3561 : cluster [DBG] osdmap e1869: 8 total, 8 up, 8 in 2024-04-07T17:21:27.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:21:27 smithi100 bash[74942]: cluster 2024-04-07T17:21:26.065486+0000 mgr.y (mgr.25175) 1982 : cluster [DBG] pgmap v3426: 137 pgs: 137 active+clean; 327 MiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 3.4 MiB/s rd, 6.9 MiB/s wr, 81 op/s 2024-04-07T17:21:27.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:21:27 smithi100 bash[76202]: cluster 2024-04-07T17:21:26.065486+0000 mgr.y (mgr.25175) 1982 : cluster [DBG] pgmap v3426: 137 pgs: 137 active+clean; 327 MiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 3.4 MiB/s rd, 6.9 MiB/s wr, 81 op/s 2024-04-07T17:21:27.618 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:21:27 smithi144 bash[66654]: cluster 2024-04-07T17:21:26.065486+0000 mgr.y (mgr.25175) 1982 : cluster [DBG] pgmap v3426: 137 pgs: 137 active+clean; 327 MiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 3.4 MiB/s rd, 6.9 MiB/s wr, 81 op/s 2024-04-07T17:21:28.704 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:21:28 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:21:28] "GET /metrics HTTP/1.1" 200 36762 "" "Prometheus/2.43.0" 2024-04-07T17:21:29.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:21:29 smithi100 bash[74942]: cluster 2024-04-07T17:21:28.066713+0000 mgr.y (mgr.25175) 1983 : cluster [DBG] pgmap v3427: 137 pgs: 137 active+clean; 335 MiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 2.1 MiB/s rd, 4.4 MiB/s wr, 67 op/s 2024-04-07T17:21:29.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:21:29 smithi100 bash[76202]: cluster 2024-04-07T17:21:28.066713+0000 mgr.y (mgr.25175) 1983 : cluster [DBG] pgmap v3427: 137 pgs: 137 active+clean; 335 MiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 2.1 MiB/s rd, 4.4 MiB/s wr, 67 op/s 2024-04-07T17:21:29.617 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:21:29 smithi144 bash[66654]: cluster 2024-04-07T17:21:28.066713+0000 mgr.y (mgr.25175) 1983 : cluster [DBG] pgmap v3427: 137 pgs: 137 active+clean; 335 MiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 2.1 MiB/s rd, 4.4 MiB/s wr, 67 op/s 2024-04-07T17:21:31.617 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:21:31 smithi144 bash[66654]: cluster 2024-04-07T17:21:30.067434+0000 mgr.y (mgr.25175) 1984 : cluster [DBG] pgmap v3428: 137 pgs: 137 active+clean; 344 MiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 2.1 MiB/s rd, 4.1 MiB/s wr, 63 op/s 2024-04-07T17:21:31.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:21:31 smithi100 bash[74942]: cluster 2024-04-07T17:21:30.067434+0000 mgr.y (mgr.25175) 1984 : cluster [DBG] pgmap v3428: 137 pgs: 137 active+clean; 344 MiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 2.1 MiB/s rd, 4.1 MiB/s wr, 63 op/s 2024-04-07T17:21:31.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:21:31 smithi100 bash[76202]: cluster 2024-04-07T17:21:30.067434+0000 mgr.y (mgr.25175) 1984 : cluster [DBG] pgmap v3428: 137 pgs: 137 active+clean; 344 MiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 2.1 MiB/s rd, 4.1 MiB/s wr, 63 op/s 2024-04-07T17:21:32.617 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:21:32 smithi144 bash[95233]: ts=2024-04-07T17:21:32.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:21:32.617 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:21:32 smithi144 bash[66654]: audit 2024-04-07T17:21:31.834055+0000 mon.a (mon.0) 3562 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:21:32.617 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:21:32 smithi144 bash[66654]: audit 2024-04-07T17:21:31.944963+0000 mon.a (mon.0) 3563 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:21:32.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:21:32 smithi100 bash[74942]: audit 2024-04-07T17:21:31.834055+0000 mon.a (mon.0) 3562 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:21:32.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:21:32 smithi100 bash[74942]: audit 2024-04-07T17:21:31.944963+0000 mon.a (mon.0) 3563 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:21:32.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:21:32 smithi100 bash[76202]: audit 2024-04-07T17:21:31.834055+0000 mon.a (mon.0) 3562 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:21:32.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:21:32 smithi100 bash[76202]: audit 2024-04-07T17:21:31.944963+0000 mon.a (mon.0) 3563 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:21:32.726 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] EncryptedFlattenTest/0.ZeroOverlap (9007 ms) 2024-04-07T17:21:33.617 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:21:33 smithi144 bash[66654]: cluster 2024-04-07T17:21:32.068418+0000 mgr.y (mgr.25175) 1985 : cluster [DBG] pgmap v3429: 137 pgs: 137 active+clean; 344 MiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 2.1 MiB/s rd, 4.1 MiB/s wr, 61 op/s 2024-04-07T17:21:33.617 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:21:33 smithi144 bash[66654]: audit 2024-04-07T17:21:32.675097+0000 mon.a (mon.0) 3564 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:21:33.617 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:21:33 smithi144 bash[66654]: audit 2024-04-07T17:21:32.676733+0000 mon.a (mon.0) 3565 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:21:33.617 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:21:33 smithi144 bash[66654]: audit 2024-04-07T17:21:32.686414+0000 mon.a (mon.0) 3566 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:21:33.698 INFO:tasks.workunit.client.0.smithi100.stdout:[----------] 7 tests from EncryptedFlattenTest/0 (68655 ms total) 2024-04-07T17:21:33.698 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:21:33.698 INFO:tasks.workunit.client.0.smithi100.stdout:[----------] 7 tests from EncryptedFlattenTest/1, where TypeParam = PlaintextUnderLUKS2 2024-04-07T17:21:33.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:21:33 smithi100 bash[76202]: cluster 2024-04-07T17:21:32.068418+0000 mgr.y (mgr.25175) 1985 : cluster [DBG] pgmap v3429: 137 pgs: 137 active+clean; 344 MiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 2.1 MiB/s rd, 4.1 MiB/s wr, 61 op/s 2024-04-07T17:21:33.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:21:33 smithi100 bash[76202]: audit 2024-04-07T17:21:32.675097+0000 mon.a (mon.0) 3564 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:21:33.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:21:33 smithi100 bash[76202]: audit 2024-04-07T17:21:32.676733+0000 mon.a (mon.0) 3565 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:21:33.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:21:33 smithi100 bash[76202]: audit 2024-04-07T17:21:32.686414+0000 mon.a (mon.0) 3566 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:21:33.705 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:21:33 smithi100 bash[74942]: cluster 2024-04-07T17:21:32.068418+0000 mgr.y (mgr.25175) 1985 : cluster [DBG] pgmap v3429: 137 pgs: 137 active+clean; 344 MiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 2.1 MiB/s rd, 4.1 MiB/s wr, 61 op/s 2024-04-07T17:21:33.705 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:21:33 smithi100 bash[74942]: audit 2024-04-07T17:21:32.675097+0000 mon.a (mon.0) 3564 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:21:33.705 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:21:33 smithi100 bash[74942]: audit 2024-04-07T17:21:32.676733+0000 mon.a (mon.0) 3565 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:21:33.705 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:21:33 smithi100 bash[74942]: audit 2024-04-07T17:21:32.686414+0000 mon.a (mon.0) 3566 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:21:33.752 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] EncryptedFlattenTest/1.Simple 2024-04-07T17:21:34.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:21:34 smithi100 bash[74942]: cluster 2024-04-07T17:21:33.692475+0000 mon.a (mon.0) 3567 : cluster [DBG] osdmap e1870: 8 total, 8 up, 8 in 2024-04-07T17:21:34.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:21:34 smithi100 bash[76202]: cluster 2024-04-07T17:21:33.692475+0000 mon.a (mon.0) 3567 : cluster [DBG] osdmap e1870: 8 total, 8 up, 8 in 2024-04-07T17:21:35.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:21:34 smithi144 bash[66654]: cluster 2024-04-07T17:21:33.692475+0000 mon.a (mon.0) 3567 : cluster [DBG] osdmap e1870: 8 total, 8 up, 8 in 2024-04-07T17:21:35.715 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:21:36.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:21:35 smithi144 bash[66654]: cluster 2024-04-07T17:21:34.069230+0000 mgr.y (mgr.25175) 1986 : cluster [DBG] pgmap v3431: 105 pgs: 105 active+clean; 4.6 MiB data, 4.5 GiB used, 711 GiB / 715 GiB avail 2024-04-07T17:21:36.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:21:35 smithi144 bash[66654]: cluster 2024-04-07T17:21:34.706147+0000 mon.a (mon.0) 3568 : cluster [DBG] osdmap e1871: 8 total, 8 up, 8 in 2024-04-07T17:21:36.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:21:35 smithi144 bash[66654]: audit 2024-04-07T17:21:34.709892+0000 mon.a (mon.0) 3569 : audit [INF] from='client.? 172.21.15.100:0/1404628029' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:21:36.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:21:35 smithi100 bash[74942]: cluster 2024-04-07T17:21:34.069230+0000 mgr.y (mgr.25175) 1986 : cluster [DBG] pgmap v3431: 105 pgs: 105 active+clean; 4.6 MiB data, 4.5 GiB used, 711 GiB / 715 GiB avail 2024-04-07T17:21:36.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:21:35 smithi100 bash[74942]: cluster 2024-04-07T17:21:34.706147+0000 mon.a (mon.0) 3568 : cluster [DBG] osdmap e1871: 8 total, 8 up, 8 in 2024-04-07T17:21:36.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:21:35 smithi100 bash[74942]: audit 2024-04-07T17:21:34.709892+0000 mon.a (mon.0) 3569 : audit [INF] from='client.? 172.21.15.100:0/1404628029' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:21:36.205 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:21:35 smithi100 bash[76202]: cluster 2024-04-07T17:21:34.069230+0000 mgr.y (mgr.25175) 1986 : cluster [DBG] pgmap v3431: 105 pgs: 105 active+clean; 4.6 MiB data, 4.5 GiB used, 711 GiB / 715 GiB avail 2024-04-07T17:21:36.205 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:21:35 smithi100 bash[76202]: cluster 2024-04-07T17:21:34.706147+0000 mon.a (mon.0) 3568 : cluster [DBG] osdmap e1871: 8 total, 8 up, 8 in 2024-04-07T17:21:36.205 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:21:35 smithi100 bash[76202]: audit 2024-04-07T17:21:34.709892+0000 mon.a (mon.0) 3569 : audit [INF] from='client.? 172.21.15.100:0/1404628029' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:21:37.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:21:36 smithi144 bash[66654]: audit 2024-04-07T17:21:35.700471+0000 mon.a (mon.0) 3570 : audit [INF] from='client.? 172.21.15.100:0/1404628029' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T17:21:37.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:21:36 smithi144 bash[66654]: cluster 2024-04-07T17:21:35.708357+0000 mon.a (mon.0) 3571 : cluster [DBG] osdmap e1872: 8 total, 8 up, 8 in 2024-04-07T17:21:37.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:21:36 smithi144 bash[66654]: cluster 2024-04-07T17:21:36.713127+0000 mon.a (mon.0) 3572 : cluster [DBG] osdmap e1873: 8 total, 8 up, 8 in 2024-04-07T17:21:37.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:21:36 smithi100 bash[74942]: audit 2024-04-07T17:21:35.700471+0000 mon.a (mon.0) 3570 : audit [INF] from='client.? 172.21.15.100:0/1404628029' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T17:21:37.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:21:36 smithi100 bash[74942]: cluster 2024-04-07T17:21:35.708357+0000 mon.a (mon.0) 3571 : cluster [DBG] osdmap e1872: 8 total, 8 up, 8 in 2024-04-07T17:21:37.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:21:36 smithi100 bash[74942]: cluster 2024-04-07T17:21:36.713127+0000 mon.a (mon.0) 3572 : cluster [DBG] osdmap e1873: 8 total, 8 up, 8 in 2024-04-07T17:21:37.205 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:21:36 smithi100 bash[76202]: audit 2024-04-07T17:21:35.700471+0000 mon.a (mon.0) 3570 : audit [INF] from='client.? 172.21.15.100:0/1404628029' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T17:21:37.205 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:21:36 smithi100 bash[76202]: cluster 2024-04-07T17:21:35.708357+0000 mon.a (mon.0) 3571 : cluster [DBG] osdmap e1872: 8 total, 8 up, 8 in 2024-04-07T17:21:37.205 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:21:36 smithi100 bash[76202]: cluster 2024-04-07T17:21:36.713127+0000 mon.a (mon.0) 3572 : cluster [DBG] osdmap e1873: 8 total, 8 up, 8 in 2024-04-07T17:21:38.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:21:37 smithi144 bash[66654]: cluster 2024-04-07T17:21:36.069978+0000 mgr.y (mgr.25175) 1987 : cluster [DBG] pgmap v3434: 137 pgs: 3 creating+peering, 29 unknown, 105 active+clean; 4.6 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail 2024-04-07T17:21:38.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:21:37 smithi100 bash[74942]: cluster 2024-04-07T17:21:36.069978+0000 mgr.y (mgr.25175) 1987 : cluster [DBG] pgmap v3434: 137 pgs: 3 creating+peering, 29 unknown, 105 active+clean; 4.6 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail 2024-04-07T17:21:38.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:21:37 smithi100 bash[76202]: cluster 2024-04-07T17:21:36.069978+0000 mgr.y (mgr.25175) 1987 : cluster [DBG] pgmap v3434: 137 pgs: 3 creating+peering, 29 unknown, 105 active+clean; 4.6 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail 2024-04-07T17:21:38.704 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:21:38 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:21:38] "GET /metrics HTTP/1.1" 200 36756 "" "Prometheus/2.43.0" 2024-04-07T17:21:39.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:21:38 smithi144 bash[66654]: cluster 2024-04-07T17:21:37.730991+0000 mon.a (mon.0) 3573 : cluster [DBG] osdmap e1874: 8 total, 8 up, 8 in 2024-04-07T17:21:39.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:21:38 smithi100 bash[74942]: cluster 2024-04-07T17:21:37.730991+0000 mon.a (mon.0) 3573 : cluster [DBG] osdmap e1874: 8 total, 8 up, 8 in 2024-04-07T17:21:39.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:21:38 smithi100 bash[76202]: cluster 2024-04-07T17:21:37.730991+0000 mon.a (mon.0) 3573 : cluster [DBG] osdmap e1874: 8 total, 8 up, 8 in 2024-04-07T17:21:40.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:21:39 smithi144 bash[66654]: cluster 2024-04-07T17:21:38.070722+0000 mgr.y (mgr.25175) 1988 : cluster [DBG] pgmap v3437: 137 pgs: 3 creating+peering, 11 unknown, 123 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail 2024-04-07T17:21:40.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:21:39 smithi144 bash[66654]: cluster 2024-04-07T17:21:38.756611+0000 mon.a (mon.0) 3574 : cluster [DBG] osdmap e1875: 8 total, 8 up, 8 in 2024-04-07T17:21:40.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:21:39 smithi100 bash[74942]: cluster 2024-04-07T17:21:38.070722+0000 mgr.y (mgr.25175) 1988 : cluster [DBG] pgmap v3437: 137 pgs: 3 creating+peering, 11 unknown, 123 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail 2024-04-07T17:21:40.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:21:39 smithi100 bash[74942]: cluster 2024-04-07T17:21:38.756611+0000 mon.a (mon.0) 3574 : cluster [DBG] osdmap e1875: 8 total, 8 up, 8 in 2024-04-07T17:21:40.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:21:39 smithi100 bash[76202]: cluster 2024-04-07T17:21:38.070722+0000 mgr.y (mgr.25175) 1988 : cluster [DBG] pgmap v3437: 137 pgs: 3 creating+peering, 11 unknown, 123 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail 2024-04-07T17:21:40.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:21:39 smithi100 bash[76202]: cluster 2024-04-07T17:21:38.756611+0000 mon.a (mon.0) 3574 : cluster [DBG] osdmap e1875: 8 total, 8 up, 8 in 2024-04-07T17:21:42.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:21:41 smithi144 bash[66654]: cluster 2024-04-07T17:21:40.071831+0000 mgr.y (mgr.25175) 1989 : cluster [DBG] pgmap v3439: 137 pgs: 137 active+clean; 17 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 2.5 KiB/s rd, 2.7 MiB/s wr, 4 op/s 2024-04-07T17:21:42.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:21:41 smithi100 bash[74942]: cluster 2024-04-07T17:21:40.071831+0000 mgr.y (mgr.25175) 1989 : cluster [DBG] pgmap v3439: 137 pgs: 137 active+clean; 17 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 2.5 KiB/s rd, 2.7 MiB/s wr, 4 op/s 2024-04-07T17:21:42.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:21:41 smithi100 bash[76202]: cluster 2024-04-07T17:21:40.071831+0000 mgr.y (mgr.25175) 1989 : cluster [DBG] pgmap v3439: 137 pgs: 137 active+clean; 17 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 2.5 KiB/s rd, 2.7 MiB/s wr, 4 op/s 2024-04-07T17:21:42.617 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:21:42 smithi144 bash[95233]: ts=2024-04-07T17:21:42.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:21:44.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:21:43 smithi144 bash[66654]: cluster 2024-04-07T17:21:42.072550+0000 mgr.y (mgr.25175) 1990 : cluster [DBG] pgmap v3440: 137 pgs: 137 active+clean; 22 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 3.3 KiB/s rd, 2.8 MiB/s wr, 6 op/s 2024-04-07T17:21:44.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:21:43 smithi100 bash[74942]: cluster 2024-04-07T17:21:42.072550+0000 mgr.y (mgr.25175) 1990 : cluster [DBG] pgmap v3440: 137 pgs: 137 active+clean; 22 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 3.3 KiB/s rd, 2.8 MiB/s wr, 6 op/s 2024-04-07T17:21:44.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:21:43 smithi100 bash[76202]: cluster 2024-04-07T17:21:42.072550+0000 mgr.y (mgr.25175) 1990 : cluster [DBG] pgmap v3440: 137 pgs: 137 active+clean; 22 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 3.3 KiB/s rd, 2.8 MiB/s wr, 6 op/s 2024-04-07T17:21:46.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:21:45 smithi144 bash[66654]: cluster 2024-04-07T17:21:44.073359+0000 mgr.y (mgr.25175) 1991 : cluster [DBG] pgmap v3441: 137 pgs: 137 active+clean; 30 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 36 KiB/s rd, 3.4 MiB/s wr, 50 op/s 2024-04-07T17:21:46.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:21:45 smithi100 bash[74942]: cluster 2024-04-07T17:21:44.073359+0000 mgr.y (mgr.25175) 1991 : cluster [DBG] pgmap v3441: 137 pgs: 137 active+clean; 30 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 36 KiB/s rd, 3.4 MiB/s wr, 50 op/s 2024-04-07T17:21:46.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:21:45 smithi100 bash[76202]: cluster 2024-04-07T17:21:44.073359+0000 mgr.y (mgr.25175) 1991 : cluster [DBG] pgmap v3441: 137 pgs: 137 active+clean; 30 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 36 KiB/s rd, 3.4 MiB/s wr, 50 op/s 2024-04-07T17:21:48.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:21:47 smithi144 bash[66654]: cluster 2024-04-07T17:21:46.074047+0000 mgr.y (mgr.25175) 1992 : cluster [DBG] pgmap v3442: 137 pgs: 137 active+clean; 30 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 32 KiB/s rd, 3.0 MiB/s wr, 44 op/s 2024-04-07T17:21:48.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:21:47 smithi144 bash[66654]: audit 2024-04-07T17:21:46.833922+0000 mon.a (mon.0) 3575 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:21:48.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:21:47 smithi100 bash[74942]: cluster 2024-04-07T17:21:46.074047+0000 mgr.y (mgr.25175) 1992 : cluster [DBG] pgmap v3442: 137 pgs: 137 active+clean; 30 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 32 KiB/s rd, 3.0 MiB/s wr, 44 op/s 2024-04-07T17:21:48.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:21:47 smithi100 bash[74942]: audit 2024-04-07T17:21:46.833922+0000 mon.a (mon.0) 3575 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:21:48.205 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:21:47 smithi100 bash[76202]: cluster 2024-04-07T17:21:46.074047+0000 mgr.y (mgr.25175) 1992 : cluster [DBG] pgmap v3442: 137 pgs: 137 active+clean; 30 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 32 KiB/s rd, 3.0 MiB/s wr, 44 op/s 2024-04-07T17:21:48.205 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:21:47 smithi100 bash[76202]: audit 2024-04-07T17:21:46.833922+0000 mon.a (mon.0) 3575 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:21:48.704 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:21:48 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:21:48] "GET /metrics HTTP/1.1" 200 36749 "" "Prometheus/2.43.0" 2024-04-07T17:21:49.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:21:48 smithi144 bash[66654]: cluster 2024-04-07T17:21:47.978680+0000 mon.a (mon.0) 3576 : cluster [DBG] osdmap e1876: 8 total, 8 up, 8 in 2024-04-07T17:21:49.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:21:48 smithi100 bash[74942]: cluster 2024-04-07T17:21:47.978680+0000 mon.a (mon.0) 3576 : cluster [DBG] osdmap e1876: 8 total, 8 up, 8 in 2024-04-07T17:21:49.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:21:48 smithi100 bash[76202]: cluster 2024-04-07T17:21:47.978680+0000 mon.a (mon.0) 3576 : cluster [DBG] osdmap e1876: 8 total, 8 up, 8 in 2024-04-07T17:21:50.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:21:49 smithi144 bash[66654]: cluster 2024-04-07T17:21:48.074544+0000 mgr.y (mgr.25175) 1993 : cluster [DBG] pgmap v3444: 137 pgs: 137 active+clean; 30 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 29 KiB/s rd, 2.3 MiB/s wr, 39 op/s 2024-04-07T17:21:50.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:21:49 smithi100 bash[74942]: cluster 2024-04-07T17:21:48.074544+0000 mgr.y (mgr.25175) 1993 : cluster [DBG] pgmap v3444: 137 pgs: 137 active+clean; 30 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 29 KiB/s rd, 2.3 MiB/s wr, 39 op/s 2024-04-07T17:21:50.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:21:49 smithi100 bash[76202]: cluster 2024-04-07T17:21:48.074544+0000 mgr.y (mgr.25175) 1993 : cluster [DBG] pgmap v3444: 137 pgs: 137 active+clean; 30 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 29 KiB/s rd, 2.3 MiB/s wr, 39 op/s 2024-04-07T17:21:52.317 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:21:51 smithi144 bash[66654]: cluster 2024-04-07T17:21:50.075736+0000 mgr.y (mgr.25175) 1994 : cluster [DBG] pgmap v3445: 137 pgs: 137 active+clean; 34 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 28 KiB/s rd, 1.7 MiB/s wr, 39 op/s 2024-04-07T17:21:52.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:21:51 smithi100 bash[74942]: cluster 2024-04-07T17:21:50.075736+0000 mgr.y (mgr.25175) 1994 : cluster [DBG] pgmap v3445: 137 pgs: 137 active+clean; 34 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 28 KiB/s rd, 1.7 MiB/s wr, 39 op/s 2024-04-07T17:21:52.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:21:51 smithi100 bash[76202]: cluster 2024-04-07T17:21:50.075736+0000 mgr.y (mgr.25175) 1994 : cluster [DBG] pgmap v3445: 137 pgs: 137 active+clean; 34 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 28 KiB/s rd, 1.7 MiB/s wr, 39 op/s 2024-04-07T17:21:52.617 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:21:52 smithi144 bash[95233]: ts=2024-04-07T17:21:52.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:21:52.826 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] EncryptedFlattenTest/1.Simple (19074 ms) 2024-04-07T17:21:52.826 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] EncryptedFlattenTest/1.Grow 2024-04-07T17:21:52.826 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:21:54.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:21:53 smithi144 bash[66654]: cluster 2024-04-07T17:21:52.076342+0000 mgr.y (mgr.25175) 1995 : cluster [DBG] pgmap v3446: 137 pgs: 137 active+clean; 38 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 38 op/s 2024-04-07T17:21:54.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:21:53 smithi100 bash[74942]: cluster 2024-04-07T17:21:52.076342+0000 mgr.y (mgr.25175) 1995 : cluster [DBG] pgmap v3446: 137 pgs: 137 active+clean; 38 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 38 op/s 2024-04-07T17:21:54.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:21:53 smithi100 bash[76202]: cluster 2024-04-07T17:21:52.076342+0000 mgr.y (mgr.25175) 1995 : cluster [DBG] pgmap v3446: 137 pgs: 137 active+clean; 38 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 38 op/s 2024-04-07T17:21:55.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:21:55 smithi144 bash[66654]: cluster 2024-04-07T17:21:54.007011+0000 mon.a (mon.0) 3577 : cluster [DBG] osdmap e1877: 8 total, 8 up, 8 in 2024-04-07T17:21:55.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:21:55 smithi100 bash[74942]: cluster 2024-04-07T17:21:54.007011+0000 mon.a (mon.0) 3577 : cluster [DBG] osdmap e1877: 8 total, 8 up, 8 in 2024-04-07T17:21:55.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:21:55 smithi100 bash[76202]: cluster 2024-04-07T17:21:54.007011+0000 mon.a (mon.0) 3577 : cluster [DBG] osdmap e1877: 8 total, 8 up, 8 in 2024-04-07T17:21:56.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:21:56 smithi144 bash[66654]: cluster 2024-04-07T17:21:54.077125+0000 mgr.y (mgr.25175) 1996 : cluster [DBG] pgmap v3448: 137 pgs: 137 active+clean; 68 MiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 8.1 MiB/s rd, 6.3 MiB/s wr, 40 op/s 2024-04-07T17:21:56.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:21:56 smithi100 bash[74942]: cluster 2024-04-07T17:21:54.077125+0000 mgr.y (mgr.25175) 1996 : cluster [DBG] pgmap v3448: 137 pgs: 137 active+clean; 68 MiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 8.1 MiB/s rd, 6.3 MiB/s wr, 40 op/s 2024-04-07T17:21:56.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:21:56 smithi100 bash[76202]: cluster 2024-04-07T17:21:54.077125+0000 mgr.y (mgr.25175) 1996 : cluster [DBG] pgmap v3448: 137 pgs: 137 active+clean; 68 MiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 8.1 MiB/s rd, 6.3 MiB/s wr, 40 op/s 2024-04-07T17:21:58.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:21:58 smithi144 bash[66654]: cluster 2024-04-07T17:21:56.077923+0000 mgr.y (mgr.25175) 1997 : cluster [DBG] pgmap v3449: 137 pgs: 137 active+clean; 84 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 13 MiB/s rd, 8.6 MiB/s wr, 52 op/s 2024-04-07T17:21:58.427 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:21:58 smithi100 bash[74942]: cluster 2024-04-07T17:21:56.077923+0000 mgr.y (mgr.25175) 1997 : cluster [DBG] pgmap v3449: 137 pgs: 137 active+clean; 84 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 13 MiB/s rd, 8.6 MiB/s wr, 52 op/s 2024-04-07T17:21:58.427 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:21:58 smithi100 bash[76202]: cluster 2024-04-07T17:21:56.077923+0000 mgr.y (mgr.25175) 1997 : cluster [DBG] pgmap v3449: 137 pgs: 137 active+clean; 84 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 13 MiB/s rd, 8.6 MiB/s wr, 52 op/s 2024-04-07T17:21:58.704 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:21:58 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:21:58] "GET /metrics HTTP/1.1" 200 36749 "" "Prometheus/2.43.0" 2024-04-07T17:22:00.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:22:00 smithi144 bash[66654]: cluster 2024-04-07T17:21:58.078733+0000 mgr.y (mgr.25175) 1998 : cluster [DBG] pgmap v3450: 137 pgs: 137 active+clean; 96 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 12 MiB/s rd, 8.2 MiB/s wr, 51 op/s 2024-04-07T17:22:00.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:22:00 smithi100 bash[74942]: cluster 2024-04-07T17:21:58.078733+0000 mgr.y (mgr.25175) 1998 : cluster [DBG] pgmap v3450: 137 pgs: 137 active+clean; 96 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 12 MiB/s rd, 8.2 MiB/s wr, 51 op/s 2024-04-07T17:22:00.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:22:00 smithi100 bash[76202]: cluster 2024-04-07T17:21:58.078733+0000 mgr.y (mgr.25175) 1998 : cluster [DBG] pgmap v3450: 137 pgs: 137 active+clean; 96 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 12 MiB/s rd, 8.2 MiB/s wr, 51 op/s 2024-04-07T17:22:02.317 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:22:02 smithi144 bash[66654]: cluster 2024-04-07T17:22:00.079551+0000 mgr.y (mgr.25175) 1999 : cluster [DBG] pgmap v3451: 137 pgs: 137 active+clean; 96 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 12 MiB/s rd, 7.8 MiB/s wr, 62 op/s 2024-04-07T17:22:02.317 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:22:02 smithi144 bash[66654]: audit 2024-04-07T17:22:01.834218+0000 mon.a (mon.0) 3578 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:22:02.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:22:02 smithi100 bash[74942]: cluster 2024-04-07T17:22:00.079551+0000 mgr.y (mgr.25175) 1999 : cluster [DBG] pgmap v3451: 137 pgs: 137 active+clean; 96 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 12 MiB/s rd, 7.8 MiB/s wr, 62 op/s 2024-04-07T17:22:02.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:22:02 smithi100 bash[74942]: audit 2024-04-07T17:22:01.834218+0000 mon.a (mon.0) 3578 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:22:02.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:22:02 smithi100 bash[76202]: cluster 2024-04-07T17:22:00.079551+0000 mgr.y (mgr.25175) 1999 : cluster [DBG] pgmap v3451: 137 pgs: 137 active+clean; 96 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 12 MiB/s rd, 7.8 MiB/s wr, 62 op/s 2024-04-07T17:22:02.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:22:02 smithi100 bash[76202]: audit 2024-04-07T17:22:01.834218+0000 mon.a (mon.0) 3578 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:22:02.617 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:22:02 smithi144 bash[95233]: ts=2024-04-07T17:22:02.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:22:04.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:22:04 smithi144 bash[66654]: cluster 2024-04-07T17:22:02.080151+0000 mgr.y (mgr.25175) 2000 : cluster [DBG] pgmap v3452: 137 pgs: 137 active+clean; 96 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 10 MiB/s rd, 7.0 MiB/s wr, 60 op/s 2024-04-07T17:22:04.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:22:04 smithi100 bash[74942]: cluster 2024-04-07T17:22:02.080151+0000 mgr.y (mgr.25175) 2000 : cluster [DBG] pgmap v3452: 137 pgs: 137 active+clean; 96 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 10 MiB/s rd, 7.0 MiB/s wr, 60 op/s 2024-04-07T17:22:04.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:22:04 smithi100 bash[76202]: cluster 2024-04-07T17:22:02.080151+0000 mgr.y (mgr.25175) 2000 : cluster [DBG] pgmap v3452: 137 pgs: 137 active+clean; 96 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 10 MiB/s rd, 7.0 MiB/s wr, 60 op/s 2024-04-07T17:22:06.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:22:06 smithi144 bash[66654]: cluster 2024-04-07T17:22:04.080764+0000 mgr.y (mgr.25175) 2001 : cluster [DBG] pgmap v3453: 137 pgs: 137 active+clean; 96 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 5.2 MiB/s rd, 3.2 MiB/s wr, 33 op/s 2024-04-07T17:22:06.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:22:06 smithi100 bash[76202]: cluster 2024-04-07T17:22:04.080764+0000 mgr.y (mgr.25175) 2001 : cluster [DBG] pgmap v3453: 137 pgs: 137 active+clean; 96 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 5.2 MiB/s rd, 3.2 MiB/s wr, 33 op/s 2024-04-07T17:22:06.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:22:06 smithi100 bash[74942]: cluster 2024-04-07T17:22:04.080764+0000 mgr.y (mgr.25175) 2001 : cluster [DBG] pgmap v3453: 137 pgs: 137 active+clean; 96 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 5.2 MiB/s rd, 3.2 MiB/s wr, 33 op/s 2024-04-07T17:22:08.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:22:08 smithi144 bash[66654]: cluster 2024-04-07T17:22:06.081426+0000 mgr.y (mgr.25175) 2002 : cluster [DBG] pgmap v3454: 137 pgs: 137 active+clean; 100 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 4.4 MiB/s rd, 3.0 MiB/s wr, 28 op/s 2024-04-07T17:22:08.426 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:22:08 smithi100 bash[74942]: cluster 2024-04-07T17:22:06.081426+0000 mgr.y (mgr.25175) 2002 : cluster [DBG] pgmap v3454: 137 pgs: 137 active+clean; 100 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 4.4 MiB/s rd, 3.0 MiB/s wr, 28 op/s 2024-04-07T17:22:08.427 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:22:08 smithi100 bash[76202]: cluster 2024-04-07T17:22:06.081426+0000 mgr.y (mgr.25175) 2002 : cluster [DBG] pgmap v3454: 137 pgs: 137 active+clean; 100 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 4.4 MiB/s rd, 3.0 MiB/s wr, 28 op/s 2024-04-07T17:22:08.649 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] EncryptedFlattenTest/1.Grow (15823 ms) 2024-04-07T17:22:08.649 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] EncryptedFlattenTest/1.Shrink 2024-04-07T17:22:08.649 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:22:08.704 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:22:08 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:22:08] "GET /metrics HTTP/1.1" 200 36752 "" "Prometheus/2.43.0" 2024-04-07T17:22:10.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:22:10 smithi144 bash[66654]: cluster 2024-04-07T17:22:08.082227+0000 mgr.y (mgr.25175) 2003 : cluster [DBG] pgmap v3455: 137 pgs: 137 active+clean; 121 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 4.5 MiB/s rd, 3.8 MiB/s wr, 40 op/s 2024-04-07T17:22:10.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:22:10 smithi144 bash[66654]: cluster 2024-04-07T17:22:09.048040+0000 mon.a (mon.0) 3579 : cluster [DBG] osdmap e1878: 8 total, 8 up, 8 in 2024-04-07T17:22:10.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:22:10 smithi100 bash[74942]: cluster 2024-04-07T17:22:08.082227+0000 mgr.y (mgr.25175) 2003 : cluster [DBG] pgmap v3455: 137 pgs: 137 active+clean; 121 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 4.5 MiB/s rd, 3.8 MiB/s wr, 40 op/s 2024-04-07T17:22:10.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:22:10 smithi100 bash[74942]: cluster 2024-04-07T17:22:09.048040+0000 mon.a (mon.0) 3579 : cluster [DBG] osdmap e1878: 8 total, 8 up, 8 in 2024-04-07T17:22:10.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:22:10 smithi100 bash[76202]: cluster 2024-04-07T17:22:08.082227+0000 mgr.y (mgr.25175) 2003 : cluster [DBG] pgmap v3455: 137 pgs: 137 active+clean; 121 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 4.5 MiB/s rd, 3.8 MiB/s wr, 40 op/s 2024-04-07T17:22:10.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:22:10 smithi100 bash[76202]: cluster 2024-04-07T17:22:09.048040+0000 mon.a (mon.0) 3579 : cluster [DBG] osdmap e1878: 8 total, 8 up, 8 in 2024-04-07T17:22:12.317 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:22:12 smithi144 bash[66654]: cluster 2024-04-07T17:22:10.083225+0000 mgr.y (mgr.25175) 2004 : cluster [DBG] pgmap v3457: 137 pgs: 137 active+clean; 137 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 10 MiB/s rd, 5.7 MiB/s wr, 33 op/s 2024-04-07T17:22:12.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:22:12 smithi100 bash[74942]: cluster 2024-04-07T17:22:10.083225+0000 mgr.y (mgr.25175) 2004 : cluster [DBG] pgmap v3457: 137 pgs: 137 active+clean; 137 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 10 MiB/s rd, 5.7 MiB/s wr, 33 op/s 2024-04-07T17:22:12.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:22:12 smithi100 bash[76202]: cluster 2024-04-07T17:22:10.083225+0000 mgr.y (mgr.25175) 2004 : cluster [DBG] pgmap v3457: 137 pgs: 137 active+clean; 137 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 10 MiB/s rd, 5.7 MiB/s wr, 33 op/s 2024-04-07T17:22:12.617 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:22:12 smithi144 bash[95233]: ts=2024-04-07T17:22:12.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:22:13.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:22:13 smithi144 bash[66654]: cluster 2024-04-07T17:22:12.083910+0000 mgr.y (mgr.25175) 2005 : cluster [DBG] pgmap v3458: 137 pgs: 137 active+clean; 141 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 10 MiB/s rd, 6.1 MiB/s wr, 40 op/s 2024-04-07T17:22:13.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:22:13 smithi100 bash[74942]: cluster 2024-04-07T17:22:12.083910+0000 mgr.y (mgr.25175) 2005 : cluster [DBG] pgmap v3458: 137 pgs: 137 active+clean; 141 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 10 MiB/s rd, 6.1 MiB/s wr, 40 op/s 2024-04-07T17:22:13.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:22:13 smithi100 bash[76202]: cluster 2024-04-07T17:22:12.083910+0000 mgr.y (mgr.25175) 2005 : cluster [DBG] pgmap v3458: 137 pgs: 137 active+clean; 141 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 10 MiB/s rd, 6.1 MiB/s wr, 40 op/s 2024-04-07T17:22:15.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:22:15 smithi144 bash[66654]: cluster 2024-04-07T17:22:14.084706+0000 mgr.y (mgr.25175) 2006 : cluster [DBG] pgmap v3459: 137 pgs: 137 active+clean; 162 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 12 MiB/s rd, 8.2 MiB/s wr, 68 op/s 2024-04-07T17:22:15.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:22:15 smithi100 bash[74942]: cluster 2024-04-07T17:22:14.084706+0000 mgr.y (mgr.25175) 2006 : cluster [DBG] pgmap v3459: 137 pgs: 137 active+clean; 162 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 12 MiB/s rd, 8.2 MiB/s wr, 68 op/s 2024-04-07T17:22:15.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:22:15 smithi100 bash[76202]: cluster 2024-04-07T17:22:14.084706+0000 mgr.y (mgr.25175) 2006 : cluster [DBG] pgmap v3459: 137 pgs: 137 active+clean; 162 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 12 MiB/s rd, 8.2 MiB/s wr, 68 op/s 2024-04-07T17:22:17.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:22:17 smithi144 bash[66654]: cluster 2024-04-07T17:22:16.085392+0000 mgr.y (mgr.25175) 2007 : cluster [DBG] pgmap v3460: 137 pgs: 137 active+clean; 162 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 12 MiB/s rd, 7.8 MiB/s wr, 68 op/s 2024-04-07T17:22:17.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:22:17 smithi144 bash[66654]: audit 2024-04-07T17:22:16.834524+0000 mon.a (mon.0) 3580 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:22:17.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:22:17 smithi100 bash[74942]: cluster 2024-04-07T17:22:16.085392+0000 mgr.y (mgr.25175) 2007 : cluster [DBG] pgmap v3460: 137 pgs: 137 active+clean; 162 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 12 MiB/s rd, 7.8 MiB/s wr, 68 op/s 2024-04-07T17:22:17.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:22:17 smithi100 bash[74942]: audit 2024-04-07T17:22:16.834524+0000 mon.a (mon.0) 3580 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:22:17.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:22:17 smithi100 bash[76202]: cluster 2024-04-07T17:22:16.085392+0000 mgr.y (mgr.25175) 2007 : cluster [DBG] pgmap v3460: 137 pgs: 137 active+clean; 162 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 12 MiB/s rd, 7.8 MiB/s wr, 68 op/s 2024-04-07T17:22:17.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:22:17 smithi100 bash[76202]: audit 2024-04-07T17:22:16.834524+0000 mon.a (mon.0) 3580 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:22:18.704 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:22:18 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:22:18] "GET /metrics HTTP/1.1" 200 36755 "" "Prometheus/2.43.0" 2024-04-07T17:22:19.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:22:19 smithi144 bash[66654]: cluster 2024-04-07T17:22:18.086212+0000 mgr.y (mgr.25175) 2008 : cluster [DBG] pgmap v3461: 137 pgs: 137 active+clean; 162 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 7.2 MiB/s rd, 4.9 MiB/s wr, 44 op/s 2024-04-07T17:22:19.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:22:19 smithi100 bash[74942]: cluster 2024-04-07T17:22:18.086212+0000 mgr.y (mgr.25175) 2008 : cluster [DBG] pgmap v3461: 137 pgs: 137 active+clean; 162 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 7.2 MiB/s rd, 4.9 MiB/s wr, 44 op/s 2024-04-07T17:22:19.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:22:19 smithi100 bash[76202]: cluster 2024-04-07T17:22:18.086212+0000 mgr.y (mgr.25175) 2008 : cluster [DBG] pgmap v3461: 137 pgs: 137 active+clean; 162 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 7.2 MiB/s rd, 4.9 MiB/s wr, 44 op/s 2024-04-07T17:22:21.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:22:21 smithi144 bash[66654]: cluster 2024-04-07T17:22:20.086923+0000 mgr.y (mgr.25175) 2009 : cluster [DBG] pgmap v3462: 137 pgs: 137 active+clean; 166 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 4.1 MiB/s rd, 3.7 MiB/s wr, 36 op/s 2024-04-07T17:22:21.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:22:21 smithi100 bash[74942]: cluster 2024-04-07T17:22:20.086923+0000 mgr.y (mgr.25175) 2009 : cluster [DBG] pgmap v3462: 137 pgs: 137 active+clean; 166 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 4.1 MiB/s rd, 3.7 MiB/s wr, 36 op/s 2024-04-07T17:22:21.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:22:21 smithi100 bash[76202]: cluster 2024-04-07T17:22:20.086923+0000 mgr.y (mgr.25175) 2009 : cluster [DBG] pgmap v3462: 137 pgs: 137 active+clean; 166 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 4.1 MiB/s rd, 3.7 MiB/s wr, 36 op/s 2024-04-07T17:22:22.617 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:22:22 smithi144 bash[95233]: ts=2024-04-07T17:22:22.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:22:23.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:22:23 smithi144 bash[66654]: cluster 2024-04-07T17:22:22.087527+0000 mgr.y (mgr.25175) 2010 : cluster [DBG] pgmap v3463: 137 pgs: 137 active+clean; 179 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 3.3 MiB/s rd, 3.8 MiB/s wr, 39 op/s 2024-04-07T17:22:23.415 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] EncryptedFlattenTest/1.Shrink (14766 ms) 2024-04-07T17:22:23.415 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] EncryptedFlattenTest/1.ShrinkToOne 2024-04-07T17:22:23.415 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:22:23.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:22:23 smithi100 bash[76202]: cluster 2024-04-07T17:22:22.087527+0000 mgr.y (mgr.25175) 2010 : cluster [DBG] pgmap v3463: 137 pgs: 137 active+clean; 179 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 3.3 MiB/s rd, 3.8 MiB/s wr, 39 op/s 2024-04-07T17:22:23.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:22:23 smithi100 bash[74942]: cluster 2024-04-07T17:22:22.087527+0000 mgr.y (mgr.25175) 2010 : cluster [DBG] pgmap v3463: 137 pgs: 137 active+clean; 179 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 3.3 MiB/s rd, 3.8 MiB/s wr, 39 op/s 2024-04-07T17:22:25.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:22:25 smithi100 bash[74942]: cluster 2024-04-07T17:22:24.088761+0000 mgr.y (mgr.25175) 2011 : cluster [DBG] pgmap v3464: 137 pgs: 137 active+clean; 199 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 8.4 MiB/s rd, 5.8 MiB/s wr, 48 op/s 2024-04-07T17:22:25.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:22:25 smithi100 bash[74942]: cluster 2024-04-07T17:22:24.126653+0000 mon.a (mon.0) 3581 : cluster [DBG] osdmap e1879: 8 total, 8 up, 8 in 2024-04-07T17:22:25.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:22:25 smithi100 bash[76202]: cluster 2024-04-07T17:22:24.088761+0000 mgr.y (mgr.25175) 2011 : cluster [DBG] pgmap v3464: 137 pgs: 137 active+clean; 199 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 8.4 MiB/s rd, 5.8 MiB/s wr, 48 op/s 2024-04-07T17:22:25.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:22:25 smithi100 bash[76202]: cluster 2024-04-07T17:22:24.126653+0000 mon.a (mon.0) 3581 : cluster [DBG] osdmap e1879: 8 total, 8 up, 8 in 2024-04-07T17:22:25.617 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:22:25 smithi144 bash[66654]: cluster 2024-04-07T17:22:24.088761+0000 mgr.y (mgr.25175) 2011 : cluster [DBG] pgmap v3464: 137 pgs: 137 active+clean; 199 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 8.4 MiB/s rd, 5.8 MiB/s wr, 48 op/s 2024-04-07T17:22:25.617 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:22:25 smithi144 bash[66654]: cluster 2024-04-07T17:22:24.126653+0000 mon.a (mon.0) 3581 : cluster [DBG] osdmap e1879: 8 total, 8 up, 8 in 2024-04-07T17:22:27.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:22:27 smithi100 bash[74942]: cluster 2024-04-07T17:22:26.089543+0000 mgr.y (mgr.25175) 2012 : cluster [DBG] pgmap v3466: 137 pgs: 137 active+clean; 207 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 9.6 MiB/s rd, 6.1 MiB/s wr, 59 op/s 2024-04-07T17:22:27.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:22:27 smithi100 bash[76202]: cluster 2024-04-07T17:22:26.089543+0000 mgr.y (mgr.25175) 2012 : cluster [DBG] pgmap v3466: 137 pgs: 137 active+clean; 207 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 9.6 MiB/s rd, 6.1 MiB/s wr, 59 op/s 2024-04-07T17:22:27.617 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:22:27 smithi144 bash[66654]: cluster 2024-04-07T17:22:26.089543+0000 mgr.y (mgr.25175) 2012 : cluster [DBG] pgmap v3466: 137 pgs: 137 active+clean; 207 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 9.6 MiB/s rd, 6.1 MiB/s wr, 59 op/s 2024-04-07T17:22:28.704 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:22:28 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:22:28] "GET /metrics HTTP/1.1" 200 36755 "" "Prometheus/2.43.0" 2024-04-07T17:22:30.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:22:29 smithi144 bash[66654]: cluster 2024-04-07T17:22:28.090410+0000 mgr.y (mgr.25175) 2013 : cluster [DBG] pgmap v3467: 137 pgs: 137 active+clean; 228 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 12 MiB/s rd, 8.2 MiB/s wr, 68 op/s 2024-04-07T17:22:30.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:22:29 smithi100 bash[74942]: cluster 2024-04-07T17:22:28.090410+0000 mgr.y (mgr.25175) 2013 : cluster [DBG] pgmap v3467: 137 pgs: 137 active+clean; 228 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 12 MiB/s rd, 8.2 MiB/s wr, 68 op/s 2024-04-07T17:22:30.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:22:29 smithi100 bash[76202]: cluster 2024-04-07T17:22:28.090410+0000 mgr.y (mgr.25175) 2013 : cluster [DBG] pgmap v3467: 137 pgs: 137 active+clean; 228 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 12 MiB/s rd, 8.2 MiB/s wr, 68 op/s 2024-04-07T17:22:32.317 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:22:31 smithi144 bash[66654]: cluster 2024-04-07T17:22:30.091117+0000 mgr.y (mgr.25175) 2014 : cluster [DBG] pgmap v3468: 137 pgs: 137 active+clean; 228 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 12 MiB/s rd, 7.8 MiB/s wr, 68 op/s 2024-04-07T17:22:32.318 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:22:31 smithi144 bash[66654]: audit 2024-04-07T17:22:31.835842+0000 mon.a (mon.0) 3582 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:22:32.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:22:31 smithi100 bash[74942]: cluster 2024-04-07T17:22:30.091117+0000 mgr.y (mgr.25175) 2014 : cluster [DBG] pgmap v3468: 137 pgs: 137 active+clean; 228 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 12 MiB/s rd, 7.8 MiB/s wr, 68 op/s 2024-04-07T17:22:32.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:22:31 smithi100 bash[74942]: audit 2024-04-07T17:22:31.835842+0000 mon.a (mon.0) 3582 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:22:32.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:22:31 smithi100 bash[76202]: cluster 2024-04-07T17:22:30.091117+0000 mgr.y (mgr.25175) 2014 : cluster [DBG] pgmap v3468: 137 pgs: 137 active+clean; 228 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 12 MiB/s rd, 7.8 MiB/s wr, 68 op/s 2024-04-07T17:22:32.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:22:31 smithi100 bash[76202]: audit 2024-04-07T17:22:31.835842+0000 mon.a (mon.0) 3582 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:22:32.617 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:22:32 smithi144 bash[95233]: ts=2024-04-07T17:22:32.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:22:33.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:22:32 smithi144 bash[66654]: audit 2024-04-07T17:22:32.752676+0000 mon.a (mon.0) 3583 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:22:33.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:22:32 smithi100 bash[74942]: audit 2024-04-07T17:22:32.752676+0000 mon.a (mon.0) 3583 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:22:33.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:22:33 smithi100 bash[76202]: audit 2024-04-07T17:22:32.752676+0000 mon.a (mon.0) 3583 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:22:34.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:22:34 smithi144 bash[66654]: cluster 2024-04-07T17:22:32.091730+0000 mgr.y (mgr.25175) 2015 : cluster [DBG] pgmap v3469: 137 pgs: 137 active+clean; 228 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 9.5 MiB/s rd, 6.1 MiB/s wr, 56 op/s 2024-04-07T17:22:34.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:22:34 smithi144 bash[66654]: audit 2024-04-07T17:22:33.488487+0000 mon.a (mon.0) 3584 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:22:34.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:22:34 smithi144 bash[66654]: audit 2024-04-07T17:22:33.489487+0000 mon.a (mon.0) 3585 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:22:34.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:22:34 smithi144 bash[66654]: audit 2024-04-07T17:22:33.496636+0000 mon.a (mon.0) 3586 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:22:34.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:22:33 smithi100 bash[74942]: cluster 2024-04-07T17:22:32.091730+0000 mgr.y (mgr.25175) 2015 : cluster [DBG] pgmap v3469: 137 pgs: 137 active+clean; 228 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 9.5 MiB/s rd, 6.1 MiB/s wr, 56 op/s 2024-04-07T17:22:34.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:22:33 smithi100 bash[74942]: audit 2024-04-07T17:22:33.488487+0000 mon.a (mon.0) 3584 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:22:34.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:22:33 smithi100 bash[74942]: audit 2024-04-07T17:22:33.489487+0000 mon.a (mon.0) 3585 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:22:34.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:22:34 smithi100 bash[74942]: audit 2024-04-07T17:22:33.496636+0000 mon.a (mon.0) 3586 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:22:34.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:22:34 smithi100 bash[76202]: cluster 2024-04-07T17:22:32.091730+0000 mgr.y (mgr.25175) 2015 : cluster [DBG] pgmap v3469: 137 pgs: 137 active+clean; 228 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 9.5 MiB/s rd, 6.1 MiB/s wr, 56 op/s 2024-04-07T17:22:34.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:22:34 smithi100 bash[76202]: audit 2024-04-07T17:22:33.488487+0000 mon.a (mon.0) 3584 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:22:34.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:22:34 smithi100 bash[76202]: audit 2024-04-07T17:22:33.489487+0000 mon.a (mon.0) 3585 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:22:34.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:22:34 smithi100 bash[76202]: audit 2024-04-07T17:22:33.496636+0000 mon.a (mon.0) 3586 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:22:36.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:22:36 smithi144 bash[66654]: cluster 2024-04-07T17:22:34.092974+0000 mgr.y (mgr.25175) 2016 : cluster [DBG] pgmap v3470: 137 pgs: 137 active+clean; 228 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 3.3 MiB/s rd, 3.3 MiB/s wr, 38 op/s 2024-04-07T17:22:36.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:22:36 smithi100 bash[74942]: cluster 2024-04-07T17:22:34.092974+0000 mgr.y (mgr.25175) 2016 : cluster [DBG] pgmap v3470: 137 pgs: 137 active+clean; 228 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 3.3 MiB/s rd, 3.3 MiB/s wr, 38 op/s 2024-04-07T17:22:36.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:22:36 smithi100 bash[76202]: cluster 2024-04-07T17:22:34.092974+0000 mgr.y (mgr.25175) 2016 : cluster [DBG] pgmap v3470: 137 pgs: 137 active+clean; 228 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 3.3 MiB/s rd, 3.3 MiB/s wr, 38 op/s 2024-04-07T17:22:38.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:22:38 smithi144 bash[66654]: cluster 2024-04-07T17:22:36.093978+0000 mgr.y (mgr.25175) 2017 : cluster [DBG] pgmap v3471: 137 pgs: 137 active+clean; 228 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 2.8 MiB/s rd, 2.8 MiB/s wr, 35 op/s 2024-04-07T17:22:38.403 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] EncryptedFlattenTest/1.ShrinkToOne (14988 ms) 2024-04-07T17:22:38.403 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] EncryptedFlattenTest/1.ShrinkToOneAfterSnapshot 2024-04-07T17:22:38.403 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:22:38.426 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:22:38 smithi100 bash[74942]: cluster 2024-04-07T17:22:36.093978+0000 mgr.y (mgr.25175) 2017 : cluster [DBG] pgmap v3471: 137 pgs: 137 active+clean; 228 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 2.8 MiB/s rd, 2.8 MiB/s wr, 35 op/s 2024-04-07T17:22:38.426 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:22:38 smithi100 bash[76202]: cluster 2024-04-07T17:22:36.093978+0000 mgr.y (mgr.25175) 2017 : cluster [DBG] pgmap v3471: 137 pgs: 137 active+clean; 228 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 2.8 MiB/s rd, 2.8 MiB/s wr, 35 op/s 2024-04-07T17:22:38.704 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:22:38 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:22:38] "GET /metrics HTTP/1.1" 200 36752 "" "Prometheus/2.43.0" 2024-04-07T17:22:40.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:22:40 smithi144 bash[66654]: cluster 2024-04-07T17:22:38.094858+0000 mgr.y (mgr.25175) 2018 : cluster [DBG] pgmap v3472: 137 pgs: 137 active+clean; 248 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 3.5 MiB/s rd, 5.1 MiB/s wr, 27 op/s 2024-04-07T17:22:40.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:22:40 smithi144 bash[66654]: cluster 2024-04-07T17:22:39.026652+0000 mon.a (mon.0) 3587 : cluster [DBG] osdmap e1880: 8 total, 8 up, 8 in 2024-04-07T17:22:40.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:22:40 smithi100 bash[74942]: cluster 2024-04-07T17:22:38.094858+0000 mgr.y (mgr.25175) 2018 : cluster [DBG] pgmap v3472: 137 pgs: 137 active+clean; 248 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 3.5 MiB/s rd, 5.1 MiB/s wr, 27 op/s 2024-04-07T17:22:40.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:22:40 smithi100 bash[74942]: cluster 2024-04-07T17:22:39.026652+0000 mon.a (mon.0) 3587 : cluster [DBG] osdmap e1880: 8 total, 8 up, 8 in 2024-04-07T17:22:40.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:22:40 smithi100 bash[76202]: cluster 2024-04-07T17:22:38.094858+0000 mgr.y (mgr.25175) 2018 : cluster [DBG] pgmap v3472: 137 pgs: 137 active+clean; 248 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 3.5 MiB/s rd, 5.1 MiB/s wr, 27 op/s 2024-04-07T17:22:40.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:22:40 smithi100 bash[76202]: cluster 2024-04-07T17:22:39.026652+0000 mon.a (mon.0) 3587 : cluster [DBG] osdmap e1880: 8 total, 8 up, 8 in 2024-04-07T17:22:42.317 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:22:42 smithi144 bash[66654]: cluster 2024-04-07T17:22:40.096025+0000 mgr.y (mgr.25175) 2019 : cluster [DBG] pgmap v3474: 137 pgs: 137 active+clean; 256 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 2.1 MiB/s rd, 4.8 MiB/s wr, 24 op/s 2024-04-07T17:22:42.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:22:42 smithi100 bash[74942]: cluster 2024-04-07T17:22:40.096025+0000 mgr.y (mgr.25175) 2019 : cluster [DBG] pgmap v3474: 137 pgs: 137 active+clean; 256 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 2.1 MiB/s rd, 4.8 MiB/s wr, 24 op/s 2024-04-07T17:22:42.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:22:42 smithi100 bash[76202]: cluster 2024-04-07T17:22:40.096025+0000 mgr.y (mgr.25175) 2019 : cluster [DBG] pgmap v3474: 137 pgs: 137 active+clean; 256 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 2.1 MiB/s rd, 4.8 MiB/s wr, 24 op/s 2024-04-07T17:22:42.617 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:22:42 smithi144 bash[95233]: ts=2024-04-07T17:22:42.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:22:44.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:22:44 smithi144 bash[66654]: cluster 2024-04-07T17:22:42.096673+0000 mgr.y (mgr.25175) 2020 : cluster [DBG] pgmap v3475: 137 pgs: 137 active+clean; 260 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 2.1 MiB/s rd, 5.2 MiB/s wr, 40 op/s 2024-04-07T17:22:44.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:22:44 smithi100 bash[74942]: cluster 2024-04-07T17:22:42.096673+0000 mgr.y (mgr.25175) 2020 : cluster [DBG] pgmap v3475: 137 pgs: 137 active+clean; 260 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 2.1 MiB/s rd, 5.2 MiB/s wr, 40 op/s 2024-04-07T17:22:44.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:22:44 smithi100 bash[76202]: cluster 2024-04-07T17:22:42.096673+0000 mgr.y (mgr.25175) 2020 : cluster [DBG] pgmap v3475: 137 pgs: 137 active+clean; 260 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 2.1 MiB/s rd, 5.2 MiB/s wr, 40 op/s 2024-04-07T17:22:46.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:22:46 smithi144 bash[66654]: cluster 2024-04-07T17:22:44.097466+0000 mgr.y (mgr.25175) 2021 : cluster [DBG] pgmap v3476: 137 pgs: 137 active+clean; 273 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 2.1 MiB/s rd, 6.5 MiB/s wr, 60 op/s 2024-04-07T17:22:46.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:22:46 smithi100 bash[74942]: cluster 2024-04-07T17:22:44.097466+0000 mgr.y (mgr.25175) 2021 : cluster [DBG] pgmap v3476: 137 pgs: 137 active+clean; 273 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 2.1 MiB/s rd, 6.5 MiB/s wr, 60 op/s 2024-04-07T17:22:46.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:22:46 smithi100 bash[76202]: cluster 2024-04-07T17:22:44.097466+0000 mgr.y (mgr.25175) 2021 : cluster [DBG] pgmap v3476: 137 pgs: 137 active+clean; 273 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 2.1 MiB/s rd, 6.5 MiB/s wr, 60 op/s 2024-04-07T17:22:47.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:22:47 smithi144 bash[66654]: audit 2024-04-07T17:22:46.836027+0000 mon.a (mon.0) 3588 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:22:47.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:22:47 smithi100 bash[74942]: audit 2024-04-07T17:22:46.836027+0000 mon.a (mon.0) 3588 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:22:47.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:22:47 smithi100 bash[76202]: audit 2024-04-07T17:22:46.836027+0000 mon.a (mon.0) 3588 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:22:48.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:22:48 smithi144 bash[66654]: cluster 2024-04-07T17:22:46.098143+0000 mgr.y (mgr.25175) 2022 : cluster [DBG] pgmap v3477: 137 pgs: 137 active+clean; 273 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 2.1 MiB/s rd, 6.5 MiB/s wr, 56 op/s 2024-04-07T17:22:48.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:22:48 smithi144 bash[66654]: cluster 2024-04-07T17:22:47.049190+0000 mon.a (mon.0) 3589 : cluster [DBG] osdmap e1881: 8 total, 8 up, 8 in 2024-04-07T17:22:48.426 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:22:48 smithi100 bash[74942]: cluster 2024-04-07T17:22:46.098143+0000 mgr.y (mgr.25175) 2022 : cluster [DBG] pgmap v3477: 137 pgs: 137 active+clean; 273 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 2.1 MiB/s rd, 6.5 MiB/s wr, 56 op/s 2024-04-07T17:22:48.427 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:22:48 smithi100 bash[74942]: cluster 2024-04-07T17:22:47.049190+0000 mon.a (mon.0) 3589 : cluster [DBG] osdmap e1881: 8 total, 8 up, 8 in 2024-04-07T17:22:48.427 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:22:48 smithi100 bash[76202]: cluster 2024-04-07T17:22:46.098143+0000 mgr.y (mgr.25175) 2022 : cluster [DBG] pgmap v3477: 137 pgs: 137 active+clean; 273 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 2.1 MiB/s rd, 6.5 MiB/s wr, 56 op/s 2024-04-07T17:22:48.427 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:22:48 smithi100 bash[76202]: cluster 2024-04-07T17:22:47.049190+0000 mon.a (mon.0) 3589 : cluster [DBG] osdmap e1881: 8 total, 8 up, 8 in 2024-04-07T17:22:48.704 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:22:48 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:22:48] "GET /metrics HTTP/1.1" 200 36755 "" "Prometheus/2.43.0" 2024-04-07T17:22:49.222 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] EncryptedFlattenTest/1.ShrinkToOneAfterSnapshot (10819 ms) 2024-04-07T17:22:49.222 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] EncryptedFlattenTest/1.MinOverlap 2024-04-07T17:22:49.222 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:22:50.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:22:50 smithi144 bash[66654]: cluster 2024-04-07T17:22:48.099067+0000 mgr.y (mgr.25175) 2023 : cluster [DBG] pgmap v3479: 137 pgs: 137 active+clean; 305 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 1.5 MiB/s rd, 6.3 MiB/s wr, 74 op/s 2024-04-07T17:22:50.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:22:50 smithi100 bash[74942]: cluster 2024-04-07T17:22:48.099067+0000 mgr.y (mgr.25175) 2023 : cluster [DBG] pgmap v3479: 137 pgs: 137 active+clean; 305 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 1.5 MiB/s rd, 6.3 MiB/s wr, 74 op/s 2024-04-07T17:22:50.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:22:50 smithi100 bash[76202]: cluster 2024-04-07T17:22:48.099067+0000 mgr.y (mgr.25175) 2023 : cluster [DBG] pgmap v3479: 137 pgs: 137 active+clean; 305 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 1.5 MiB/s rd, 6.3 MiB/s wr, 74 op/s 2024-04-07T17:22:51.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:22:51 smithi144 bash[66654]: cluster 2024-04-07T17:22:50.061949+0000 mon.a (mon.0) 3590 : cluster [DBG] osdmap e1882: 8 total, 8 up, 8 in 2024-04-07T17:22:51.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:22:51 smithi144 bash[66654]: cluster 2024-04-07T17:22:50.099904+0000 mgr.y (mgr.25175) 2024 : cluster [DBG] pgmap v3481: 137 pgs: 137 active+clean; 338 MiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 3.7 MiB/s rd, 9.8 MiB/s wr, 66 op/s 2024-04-07T17:22:51.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:22:51 smithi100 bash[74942]: cluster 2024-04-07T17:22:50.061949+0000 mon.a (mon.0) 3590 : cluster [DBG] osdmap e1882: 8 total, 8 up, 8 in 2024-04-07T17:22:51.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:22:51 smithi100 bash[74942]: cluster 2024-04-07T17:22:50.099904+0000 mgr.y (mgr.25175) 2024 : cluster [DBG] pgmap v3481: 137 pgs: 137 active+clean; 338 MiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 3.7 MiB/s rd, 9.8 MiB/s wr, 66 op/s 2024-04-07T17:22:51.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:22:51 smithi100 bash[76202]: cluster 2024-04-07T17:22:50.061949+0000 mon.a (mon.0) 3590 : cluster [DBG] osdmap e1882: 8 total, 8 up, 8 in 2024-04-07T17:22:51.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:22:51 smithi100 bash[76202]: cluster 2024-04-07T17:22:50.099904+0000 mgr.y (mgr.25175) 2024 : cluster [DBG] pgmap v3481: 137 pgs: 137 active+clean; 338 MiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 3.7 MiB/s rd, 9.8 MiB/s wr, 66 op/s 2024-04-07T17:22:52.617 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:22:52 smithi144 bash[95233]: ts=2024-04-07T17:22:52.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:22:53.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:22:53 smithi100 bash[74942]: cluster 2024-04-07T17:22:52.100640+0000 mgr.y (mgr.25175) 2025 : cluster [DBG] pgmap v3482: 137 pgs: 137 active+clean; 350 MiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 4.2 MiB/s rd, 9.6 MiB/s wr, 79 op/s 2024-04-07T17:22:53.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:22:53 smithi100 bash[76202]: cluster 2024-04-07T17:22:52.100640+0000 mgr.y (mgr.25175) 2025 : cluster [DBG] pgmap v3482: 137 pgs: 137 active+clean; 350 MiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 4.2 MiB/s rd, 9.6 MiB/s wr, 79 op/s 2024-04-07T17:22:53.617 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:22:53 smithi144 bash[66654]: cluster 2024-04-07T17:22:52.100640+0000 mgr.y (mgr.25175) 2025 : cluster [DBG] pgmap v3482: 137 pgs: 137 active+clean; 350 MiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 4.2 MiB/s rd, 9.6 MiB/s wr, 79 op/s 2024-04-07T17:22:55.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:22:55 smithi100 bash[74942]: cluster 2024-04-07T17:22:54.101461+0000 mgr.y (mgr.25175) 2026 : cluster [DBG] pgmap v3483: 137 pgs: 137 active+clean; 379 MiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 5.3 MiB/s rd, 13 MiB/s wr, 93 op/s 2024-04-07T17:22:55.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:22:55 smithi100 bash[76202]: cluster 2024-04-07T17:22:54.101461+0000 mgr.y (mgr.25175) 2026 : cluster [DBG] pgmap v3483: 137 pgs: 137 active+clean; 379 MiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 5.3 MiB/s rd, 13 MiB/s wr, 93 op/s 2024-04-07T17:22:55.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:22:55 smithi144 bash[66654]: cluster 2024-04-07T17:22:54.101461+0000 mgr.y (mgr.25175) 2026 : cluster [DBG] pgmap v3483: 137 pgs: 137 active+clean; 379 MiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 5.3 MiB/s rd, 13 MiB/s wr, 93 op/s 2024-04-07T17:22:57.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:22:57 smithi100 bash[74942]: cluster 2024-04-07T17:22:56.102138+0000 mgr.y (mgr.25175) 2027 : cluster [DBG] pgmap v3484: 137 pgs: 137 active+clean; 383 MiB data, 4.7 GiB used, 711 GiB / 715 GiB avail; 4.7 MiB/s rd, 12 MiB/s wr, 90 op/s 2024-04-07T17:22:57.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:22:57 smithi100 bash[76202]: cluster 2024-04-07T17:22:56.102138+0000 mgr.y (mgr.25175) 2027 : cluster [DBG] pgmap v3484: 137 pgs: 137 active+clean; 383 MiB data, 4.7 GiB used, 711 GiB / 715 GiB avail; 4.7 MiB/s rd, 12 MiB/s wr, 90 op/s 2024-04-07T17:22:57.617 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:22:57 smithi144 bash[66654]: cluster 2024-04-07T17:22:56.102138+0000 mgr.y (mgr.25175) 2027 : cluster [DBG] pgmap v3484: 137 pgs: 137 active+clean; 383 MiB data, 4.7 GiB used, 711 GiB / 715 GiB avail; 4.7 MiB/s rd, 12 MiB/s wr, 90 op/s 2024-04-07T17:22:58.704 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:22:58 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:22:58] "GET /metrics HTTP/1.1" 200 36755 "" "Prometheus/2.43.0" 2024-04-07T17:22:59.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:22:59 smithi100 bash[74942]: cluster 2024-04-07T17:22:58.102981+0000 mgr.y (mgr.25175) 2028 : cluster [DBG] pgmap v3485: 137 pgs: 137 active+clean; 383 MiB data, 4.7 GiB used, 711 GiB / 715 GiB avail; 2.9 MiB/s rd, 7.8 MiB/s wr, 51 op/s 2024-04-07T17:22:59.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:22:59 smithi100 bash[76202]: cluster 2024-04-07T17:22:58.102981+0000 mgr.y (mgr.25175) 2028 : cluster [DBG] pgmap v3485: 137 pgs: 137 active+clean; 383 MiB data, 4.7 GiB used, 711 GiB / 715 GiB avail; 2.9 MiB/s rd, 7.8 MiB/s wr, 51 op/s 2024-04-07T17:22:59.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:22:59 smithi144 bash[66654]: cluster 2024-04-07T17:22:58.102981+0000 mgr.y (mgr.25175) 2028 : cluster [DBG] pgmap v3485: 137 pgs: 137 active+clean; 383 MiB data, 4.7 GiB used, 711 GiB / 715 GiB avail; 2.9 MiB/s rd, 7.8 MiB/s wr, 51 op/s 2024-04-07T17:23:01.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:23:01 smithi100 bash[74942]: cluster 2024-04-07T17:23:00.103844+0000 mgr.y (mgr.25175) 2029 : cluster [DBG] pgmap v3486: 137 pgs: 137 active+clean; 383 MiB data, 4.7 GiB used, 711 GiB / 715 GiB avail; 1.3 MiB/s rd, 4.5 MiB/s wr, 48 op/s 2024-04-07T17:23:01.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:23:01 smithi100 bash[76202]: cluster 2024-04-07T17:23:00.103844+0000 mgr.y (mgr.25175) 2029 : cluster [DBG] pgmap v3486: 137 pgs: 137 active+clean; 383 MiB data, 4.7 GiB used, 711 GiB / 715 GiB avail; 1.3 MiB/s rd, 4.5 MiB/s wr, 48 op/s 2024-04-07T17:23:01.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:23:01 smithi144 bash[66654]: cluster 2024-04-07T17:23:00.103844+0000 mgr.y (mgr.25175) 2029 : cluster [DBG] pgmap v3486: 137 pgs: 137 active+clean; 383 MiB data, 4.7 GiB used, 711 GiB / 715 GiB avail; 1.3 MiB/s rd, 4.5 MiB/s wr, 48 op/s 2024-04-07T17:23:02.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:23:02 smithi100 bash[74942]: audit 2024-04-07T17:23:01.835731+0000 mon.a (mon.0) 3591 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:23:02.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:23:02 smithi100 bash[76202]: audit 2024-04-07T17:23:01.835731+0000 mon.a (mon.0) 3591 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:23:02.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:23:02 smithi144 bash[66654]: audit 2024-04-07T17:23:01.835731+0000 mon.a (mon.0) 3591 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:23:02.617 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:23:02 smithi144 bash[95233]: ts=2024-04-07T17:23:02.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:23:03.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:23:03 smithi100 bash[74942]: cluster 2024-04-07T17:23:02.104463+0000 mgr.y (mgr.25175) 2030 : cluster [DBG] pgmap v3487: 137 pgs: 137 active+clean; 383 MiB data, 4.7 GiB used, 711 GiB / 715 GiB avail; 1.1 MiB/s rd, 3.8 MiB/s wr, 46 op/s 2024-04-07T17:23:03.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:23:03 smithi100 bash[76202]: cluster 2024-04-07T17:23:02.104463+0000 mgr.y (mgr.25175) 2030 : cluster [DBG] pgmap v3487: 137 pgs: 137 active+clean; 383 MiB data, 4.7 GiB used, 711 GiB / 715 GiB avail; 1.1 MiB/s rd, 3.8 MiB/s wr, 46 op/s 2024-04-07T17:23:03.617 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:23:03 smithi144 bash[66654]: cluster 2024-04-07T17:23:02.104463+0000 mgr.y (mgr.25175) 2030 : cluster [DBG] pgmap v3487: 137 pgs: 137 active+clean; 383 MiB data, 4.7 GiB used, 711 GiB / 715 GiB avail; 1.1 MiB/s rd, 3.8 MiB/s wr, 46 op/s 2024-04-07T17:23:04.691 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] EncryptedFlattenTest/1.MinOverlap (15469 ms) 2024-04-07T17:23:04.691 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] EncryptedFlattenTest/1.ZeroOverlap 2024-04-07T17:23:04.691 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:23:05.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:23:05 smithi100 bash[74942]: cluster 2024-04-07T17:23:04.105245+0000 mgr.y (mgr.25175) 2031 : cluster [DBG] pgmap v3488: 137 pgs: 137 active+clean; 395 MiB data, 4.7 GiB used, 711 GiB / 715 GiB avail; 1.1 MiB/s rd, 4.1 MiB/s wr, 22 op/s 2024-04-07T17:23:05.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:23:05 smithi100 bash[76202]: cluster 2024-04-07T17:23:04.105245+0000 mgr.y (mgr.25175) 2031 : cluster [DBG] pgmap v3488: 137 pgs: 137 active+clean; 395 MiB data, 4.7 GiB used, 711 GiB / 715 GiB avail; 1.1 MiB/s rd, 4.1 MiB/s wr, 22 op/s 2024-04-07T17:23:05.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:23:05 smithi144 bash[66654]: cluster 2024-04-07T17:23:04.105245+0000 mgr.y (mgr.25175) 2031 : cluster [DBG] pgmap v3488: 137 pgs: 137 active+clean; 395 MiB data, 4.7 GiB used, 711 GiB / 715 GiB avail; 1.1 MiB/s rd, 4.1 MiB/s wr, 22 op/s 2024-04-07T17:23:06.617 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:23:06 smithi144 bash[66654]: cluster 2024-04-07T17:23:05.216340+0000 mon.a (mon.0) 3592 : cluster [DBG] osdmap e1883: 8 total, 8 up, 8 in 2024-04-07T17:23:06.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:23:06 smithi100 bash[74942]: cluster 2024-04-07T17:23:05.216340+0000 mon.a (mon.0) 3592 : cluster [DBG] osdmap e1883: 8 total, 8 up, 8 in 2024-04-07T17:23:06.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:23:06 smithi100 bash[76202]: cluster 2024-04-07T17:23:05.216340+0000 mon.a (mon.0) 3592 : cluster [DBG] osdmap e1883: 8 total, 8 up, 8 in 2024-04-07T17:23:07.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:23:07 smithi144 bash[66654]: cluster 2024-04-07T17:23:06.105683+0000 mgr.y (mgr.25175) 2032 : cluster [DBG] pgmap v3490: 137 pgs: 137 active+clean; 399 MiB data, 4.7 GiB used, 711 GiB / 715 GiB avail; 1.3 MiB/s rd, 2.4 MiB/s wr, 21 op/s 2024-04-07T17:23:07.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:23:07 smithi100 bash[74942]: cluster 2024-04-07T17:23:06.105683+0000 mgr.y (mgr.25175) 2032 : cluster [DBG] pgmap v3490: 137 pgs: 137 active+clean; 399 MiB data, 4.7 GiB used, 711 GiB / 715 GiB avail; 1.3 MiB/s rd, 2.4 MiB/s wr, 21 op/s 2024-04-07T17:23:07.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:23:07 smithi100 bash[76202]: cluster 2024-04-07T17:23:06.105683+0000 mgr.y (mgr.25175) 2032 : cluster [DBG] pgmap v3490: 137 pgs: 137 active+clean; 399 MiB data, 4.7 GiB used, 711 GiB / 715 GiB avail; 1.3 MiB/s rd, 2.4 MiB/s wr, 21 op/s 2024-04-07T17:23:08.704 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:23:08 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:23:08] "GET /metrics HTTP/1.1" 200 36762 "" "Prometheus/2.43.0" 2024-04-07T17:23:09.617 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:23:09 smithi144 bash[66654]: cluster 2024-04-07T17:23:08.106638+0000 mgr.y (mgr.25175) 2033 : cluster [DBG] pgmap v3491: 137 pgs: 137 active+clean; 424 MiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 3.3 MiB/s rd, 6.1 MiB/s wr, 60 op/s 2024-04-07T17:23:09.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:23:09 smithi100 bash[74942]: cluster 2024-04-07T17:23:08.106638+0000 mgr.y (mgr.25175) 2033 : cluster [DBG] pgmap v3491: 137 pgs: 137 active+clean; 424 MiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 3.3 MiB/s rd, 6.1 MiB/s wr, 60 op/s 2024-04-07T17:23:09.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:23:09 smithi100 bash[76202]: cluster 2024-04-07T17:23:08.106638+0000 mgr.y (mgr.25175) 2033 : cluster [DBG] pgmap v3491: 137 pgs: 137 active+clean; 424 MiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 3.3 MiB/s rd, 6.1 MiB/s wr, 60 op/s 2024-04-07T17:23:11.617 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:23:11 smithi144 bash[66654]: cluster 2024-04-07T17:23:10.107336+0000 mgr.y (mgr.25175) 2034 : cluster [DBG] pgmap v3492: 137 pgs: 137 active+clean; 428 MiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 3.3 MiB/s rd, 6.5 MiB/s wr, 62 op/s 2024-04-07T17:23:11.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:23:11 smithi100 bash[74942]: cluster 2024-04-07T17:23:10.107336+0000 mgr.y (mgr.25175) 2034 : cluster [DBG] pgmap v3492: 137 pgs: 137 active+clean; 428 MiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 3.3 MiB/s rd, 6.5 MiB/s wr, 62 op/s 2024-04-07T17:23:11.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:23:11 smithi100 bash[76202]: cluster 2024-04-07T17:23:10.107336+0000 mgr.y (mgr.25175) 2034 : cluster [DBG] pgmap v3492: 137 pgs: 137 active+clean; 428 MiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 3.3 MiB/s rd, 6.5 MiB/s wr, 62 op/s 2024-04-07T17:23:12.617 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:23:12 smithi144 bash[95233]: ts=2024-04-07T17:23:12.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:23:13.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:23:13 smithi144 bash[66654]: cluster 2024-04-07T17:23:12.107831+0000 mgr.y (mgr.25175) 2035 : cluster [DBG] pgmap v3493: 137 pgs: 137 active+clean; 428 MiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 3.3 MiB/s rd, 6.5 MiB/s wr, 59 op/s 2024-04-07T17:23:13.703 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:23:13 smithi100 bash[74942]: cluster 2024-04-07T17:23:12.107831+0000 mgr.y (mgr.25175) 2035 : cluster [DBG] pgmap v3493: 137 pgs: 137 active+clean; 428 MiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 3.3 MiB/s rd, 6.5 MiB/s wr, 59 op/s 2024-04-07T17:23:13.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:23:13 smithi100 bash[76202]: cluster 2024-04-07T17:23:12.107831+0000 mgr.y (mgr.25175) 2035 : cluster [DBG] pgmap v3493: 137 pgs: 137 active+clean; 428 MiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 3.3 MiB/s rd, 6.5 MiB/s wr, 59 op/s 2024-04-07T17:23:15.339 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] EncryptedFlattenTest/1.ZeroOverlap (10648 ms) 2024-04-07T17:23:15.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:23:15 smithi144 bash[66654]: cluster 2024-04-07T17:23:14.108632+0000 mgr.y (mgr.25175) 2036 : cluster [DBG] pgmap v3494: 137 pgs: 137 active+clean; 436 MiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 2.9 MiB/s rd, 5.7 MiB/s wr, 64 op/s 2024-04-07T17:23:15.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:23:15 smithi100 bash[74942]: cluster 2024-04-07T17:23:14.108632+0000 mgr.y (mgr.25175) 2036 : cluster [DBG] pgmap v3494: 137 pgs: 137 active+clean; 436 MiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 2.9 MiB/s rd, 5.7 MiB/s wr, 64 op/s 2024-04-07T17:23:15.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:23:15 smithi100 bash[76202]: cluster 2024-04-07T17:23:14.108632+0000 mgr.y (mgr.25175) 2036 : cluster [DBG] pgmap v3494: 137 pgs: 137 active+clean; 436 MiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 2.9 MiB/s rd, 5.7 MiB/s wr, 64 op/s 2024-04-07T17:23:16.263 INFO:tasks.workunit.client.0.smithi100.stdout:[----------] 7 tests from EncryptedFlattenTest/1 (101587 ms total) 2024-04-07T17:23:16.263 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:23:16.263 INFO:tasks.workunit.client.0.smithi100.stdout:[----------] 7 tests from EncryptedFlattenTest/2, where TypeParam = UnformattedLUKS1 2024-04-07T17:23:16.301 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] EncryptedFlattenTest/2.Simple 2024-04-07T17:23:17.617 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:23:17 smithi144 bash[66654]: cluster 2024-04-07T17:23:16.109663+0000 mgr.y (mgr.25175) 2037 : cluster [DBG] pgmap v3495: 137 pgs: 137 active+clean; 444 MiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 2.7 MiB/s rd, 6.0 MiB/s wr, 48 op/s 2024-04-07T17:23:17.617 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:23:17 smithi144 bash[66654]: cluster 2024-04-07T17:23:16.241460+0000 mon.a (mon.0) 3593 : cluster [DBG] osdmap e1884: 8 total, 8 up, 8 in 2024-04-07T17:23:17.617 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:23:17 smithi144 bash[66654]: audit 2024-04-07T17:23:16.837011+0000 mon.a (mon.0) 3594 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:23:17.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:23:17 smithi100 bash[74942]: cluster 2024-04-07T17:23:16.109663+0000 mgr.y (mgr.25175) 2037 : cluster [DBG] pgmap v3495: 137 pgs: 137 active+clean; 444 MiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 2.7 MiB/s rd, 6.0 MiB/s wr, 48 op/s 2024-04-07T17:23:17.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:23:17 smithi100 bash[74942]: cluster 2024-04-07T17:23:16.241460+0000 mon.a (mon.0) 3593 : cluster [DBG] osdmap e1884: 8 total, 8 up, 8 in 2024-04-07T17:23:17.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:23:17 smithi100 bash[74942]: audit 2024-04-07T17:23:16.837011+0000 mon.a (mon.0) 3594 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:23:17.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:23:17 smithi100 bash[76202]: cluster 2024-04-07T17:23:16.109663+0000 mgr.y (mgr.25175) 2037 : cluster [DBG] pgmap v3495: 137 pgs: 137 active+clean; 444 MiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 2.7 MiB/s rd, 6.0 MiB/s wr, 48 op/s 2024-04-07T17:23:17.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:23:17 smithi100 bash[76202]: cluster 2024-04-07T17:23:16.241460+0000 mon.a (mon.0) 3593 : cluster [DBG] osdmap e1884: 8 total, 8 up, 8 in 2024-04-07T17:23:17.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:23:17 smithi100 bash[76202]: audit 2024-04-07T17:23:16.837011+0000 mon.a (mon.0) 3594 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:23:18.284 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:23:18.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:23:18 smithi144 bash[66654]: cluster 2024-04-07T17:23:17.261561+0000 mon.a (mon.0) 3595 : cluster [DBG] osdmap e1885: 8 total, 8 up, 8 in 2024-04-07T17:23:18.617 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:23:18 smithi144 bash[66654]: audit 2024-04-07T17:23:17.267768+0000 mon.a (mon.0) 3596 : audit [INF] from='client.? 172.21.15.100:0/2233294843' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-38","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:23:18.704 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:23:18 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:23:18] "GET /metrics HTTP/1.1" 200 34270 "" "Prometheus/2.43.0" 2024-04-07T17:23:18.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:23:18 smithi100 bash[74942]: cluster 2024-04-07T17:23:17.261561+0000 mon.a (mon.0) 3595 : cluster [DBG] osdmap e1885: 8 total, 8 up, 8 in 2024-04-07T17:23:18.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:23:18 smithi100 bash[74942]: audit 2024-04-07T17:23:17.267768+0000 mon.a (mon.0) 3596 : audit [INF] from='client.? 172.21.15.100:0/2233294843' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-38","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:23:18.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:23:18 smithi100 bash[76202]: cluster 2024-04-07T17:23:17.261561+0000 mon.a (mon.0) 3595 : cluster [DBG] osdmap e1885: 8 total, 8 up, 8 in 2024-04-07T17:23:18.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:23:18 smithi100 bash[76202]: audit 2024-04-07T17:23:17.267768+0000 mon.a (mon.0) 3596 : audit [INF] from='client.? 172.21.15.100:0/2233294843' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-38","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:23:19.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:23:19 smithi144 bash[66654]: cluster 2024-04-07T17:23:18.110570+0000 mgr.y (mgr.25175) 2038 : cluster [DBG] pgmap v3498: 137 pgs: 6 creating+peering, 26 unknown, 105 active+clean; 4.6 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail 2024-04-07T17:23:19.617 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:23:19 smithi144 bash[66654]: cluster 2024-04-07T17:23:18.251521+0000 mon.a (mon.0) 3597 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T17:23:19.617 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:23:19 smithi144 bash[66654]: audit 2024-04-07T17:23:18.263656+0000 mon.a (mon.0) 3598 : audit [INF] from='client.? 172.21.15.100:0/2233294843' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-38","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T17:23:19.617 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:23:19 smithi144 bash[66654]: cluster 2024-04-07T17:23:18.274546+0000 mon.a (mon.0) 3599 : cluster [DBG] osdmap e1886: 8 total, 8 up, 8 in 2024-04-07T17:23:19.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:23:19 smithi100 bash[74942]: cluster 2024-04-07T17:23:18.110570+0000 mgr.y (mgr.25175) 2038 : cluster [DBG] pgmap v3498: 137 pgs: 6 creating+peering, 26 unknown, 105 active+clean; 4.6 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail 2024-04-07T17:23:19.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:23:19 smithi100 bash[74942]: cluster 2024-04-07T17:23:18.251521+0000 mon.a (mon.0) 3597 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T17:23:19.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:23:19 smithi100 bash[74942]: audit 2024-04-07T17:23:18.263656+0000 mon.a (mon.0) 3598 : audit [INF] from='client.? 172.21.15.100:0/2233294843' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-38","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T17:23:19.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:23:19 smithi100 bash[74942]: cluster 2024-04-07T17:23:18.274546+0000 mon.a (mon.0) 3599 : cluster [DBG] osdmap e1886: 8 total, 8 up, 8 in 2024-04-07T17:23:19.705 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:23:19 smithi100 bash[76202]: cluster 2024-04-07T17:23:18.110570+0000 mgr.y (mgr.25175) 2038 : cluster [DBG] pgmap v3498: 137 pgs: 6 creating+peering, 26 unknown, 105 active+clean; 4.6 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail 2024-04-07T17:23:19.705 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:23:19 smithi100 bash[76202]: cluster 2024-04-07T17:23:18.251521+0000 mon.a (mon.0) 3597 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T17:23:19.705 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:23:19 smithi100 bash[76202]: audit 2024-04-07T17:23:18.263656+0000 mon.a (mon.0) 3598 : audit [INF] from='client.? 172.21.15.100:0/2233294843' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-38","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T17:23:19.705 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:23:19 smithi100 bash[76202]: cluster 2024-04-07T17:23:18.274546+0000 mon.a (mon.0) 3599 : cluster [DBG] osdmap e1886: 8 total, 8 up, 8 in 2024-04-07T17:23:20.617 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:23:20 smithi144 bash[66654]: cluster 2024-04-07T17:23:19.275542+0000 mon.a (mon.0) 3600 : cluster [DBG] osdmap e1887: 8 total, 8 up, 8 in 2024-04-07T17:23:20.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:23:20 smithi100 bash[74942]: cluster 2024-04-07T17:23:19.275542+0000 mon.a (mon.0) 3600 : cluster [DBG] osdmap e1887: 8 total, 8 up, 8 in 2024-04-07T17:23:20.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:23:20 smithi100 bash[76202]: cluster 2024-04-07T17:23:19.275542+0000 mon.a (mon.0) 3600 : cluster [DBG] osdmap e1887: 8 total, 8 up, 8 in 2024-04-07T17:23:21.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:23:21 smithi144 bash[66654]: cluster 2024-04-07T17:23:20.111485+0000 mgr.y (mgr.25175) 2039 : cluster [DBG] pgmap v3501: 137 pgs: 6 creating+peering, 10 unknown, 121 active+clean; 4.6 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail 2024-04-07T17:23:21.617 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:23:21 smithi144 bash[66654]: cluster 2024-04-07T17:23:20.279572+0000 mon.a (mon.0) 3601 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T17:23:21.617 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:23:21 smithi144 bash[66654]: cluster 2024-04-07T17:23:20.301247+0000 mon.a (mon.0) 3602 : cluster [DBG] osdmap e1888: 8 total, 8 up, 8 in 2024-04-07T17:23:21.703 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:23:21 smithi100 bash[74942]: cluster 2024-04-07T17:23:20.111485+0000 mgr.y (mgr.25175) 2039 : cluster [DBG] pgmap v3501: 137 pgs: 6 creating+peering, 10 unknown, 121 active+clean; 4.6 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail 2024-04-07T17:23:21.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:23:21 smithi100 bash[74942]: cluster 2024-04-07T17:23:20.279572+0000 mon.a (mon.0) 3601 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T17:23:21.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:23:21 smithi100 bash[74942]: cluster 2024-04-07T17:23:20.301247+0000 mon.a (mon.0) 3602 : cluster [DBG] osdmap e1888: 8 total, 8 up, 8 in 2024-04-07T17:23:21.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:23:21 smithi100 bash[76202]: cluster 2024-04-07T17:23:20.111485+0000 mgr.y (mgr.25175) 2039 : cluster [DBG] pgmap v3501: 137 pgs: 6 creating+peering, 10 unknown, 121 active+clean; 4.6 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail 2024-04-07T17:23:21.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:23:21 smithi100 bash[76202]: cluster 2024-04-07T17:23:20.279572+0000 mon.a (mon.0) 3601 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T17:23:21.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:23:21 smithi100 bash[76202]: cluster 2024-04-07T17:23:20.301247+0000 mon.a (mon.0) 3602 : cluster [DBG] osdmap e1888: 8 total, 8 up, 8 in 2024-04-07T17:23:22.616 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:23:22 smithi144 bash[95233]: ts=2024-04-07T17:23:22.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:23:23.617 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:23:23 smithi144 bash[66654]: cluster 2024-04-07T17:23:22.112028+0000 mgr.y (mgr.25175) 2040 : cluster [DBG] pgmap v3503: 137 pgs: 6 creating+peering, 8 unknown, 123 active+clean; 4.6 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail 2024-04-07T17:23:23.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:23:23 smithi100 bash[74942]: cluster 2024-04-07T17:23:22.112028+0000 mgr.y (mgr.25175) 2040 : cluster [DBG] pgmap v3503: 137 pgs: 6 creating+peering, 8 unknown, 123 active+clean; 4.6 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail 2024-04-07T17:23:23.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:23:23 smithi100 bash[76202]: cluster 2024-04-07T17:23:22.112028+0000 mgr.y (mgr.25175) 2040 : cluster [DBG] pgmap v3503: 137 pgs: 6 creating+peering, 8 unknown, 123 active+clean; 4.6 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail 2024-04-07T17:23:25.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:23:25 smithi144 bash[66654]: cluster 2024-04-07T17:23:24.113179+0000 mgr.y (mgr.25175) 2041 : cluster [DBG] pgmap v3504: 137 pgs: 137 active+clean; 4.6 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 1023 B/s rd, 682 B/s wr, 3 op/s 2024-04-07T17:23:25.617 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:23:25 smithi144 bash[66654]: cluster 2024-04-07T17:23:24.313245+0000 mon.a (mon.0) 3603 : cluster [DBG] osdmap e1889: 8 total, 8 up, 8 in 2024-04-07T17:23:25.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:23:25 smithi100 bash[76202]: cluster 2024-04-07T17:23:24.113179+0000 mgr.y (mgr.25175) 2041 : cluster [DBG] pgmap v3504: 137 pgs: 137 active+clean; 4.6 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 1023 B/s rd, 682 B/s wr, 3 op/s 2024-04-07T17:23:25.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:23:25 smithi100 bash[76202]: cluster 2024-04-07T17:23:24.313245+0000 mon.a (mon.0) 3603 : cluster [DBG] osdmap e1889: 8 total, 8 up, 8 in 2024-04-07T17:23:25.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:23:25 smithi100 bash[74942]: cluster 2024-04-07T17:23:24.113179+0000 mgr.y (mgr.25175) 2041 : cluster [DBG] pgmap v3504: 137 pgs: 137 active+clean; 4.6 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 1023 B/s rd, 682 B/s wr, 3 op/s 2024-04-07T17:23:25.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:23:25 smithi100 bash[74942]: cluster 2024-04-07T17:23:24.313245+0000 mon.a (mon.0) 3603 : cluster [DBG] osdmap e1889: 8 total, 8 up, 8 in 2024-04-07T17:23:27.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:23:27 smithi144 bash[66654]: cluster 2024-04-07T17:23:26.114000+0000 mgr.y (mgr.25175) 2042 : cluster [DBG] pgmap v3506: 137 pgs: 137 active+clean; 13 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 6.6 KiB/s rd, 1.2 MiB/s wr, 10 op/s 2024-04-07T17:23:27.703 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:23:27 smithi100 bash[74942]: cluster 2024-04-07T17:23:26.114000+0000 mgr.y (mgr.25175) 2042 : cluster [DBG] pgmap v3506: 137 pgs: 137 active+clean; 13 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 6.6 KiB/s rd, 1.2 MiB/s wr, 10 op/s 2024-04-07T17:23:27.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:23:27 smithi100 bash[76202]: cluster 2024-04-07T17:23:26.114000+0000 mgr.y (mgr.25175) 2042 : cluster [DBG] pgmap v3506: 137 pgs: 137 active+clean; 13 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 6.6 KiB/s rd, 1.2 MiB/s wr, 10 op/s 2024-04-07T17:23:28.704 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:23:28 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:23:28] "GET /metrics HTTP/1.1" 200 34270 "" "Prometheus/2.43.0" 2024-04-07T17:23:29.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:23:29 smithi144 bash[66654]: cluster 2024-04-07T17:23:28.001827+0000 mon.a (mon.0) 3604 : cluster [DBG] osdmap e1890: 8 total, 8 up, 8 in 2024-04-07T17:23:29.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:23:29 smithi100 bash[76202]: cluster 2024-04-07T17:23:28.001827+0000 mon.a (mon.0) 3604 : cluster [DBG] osdmap e1890: 8 total, 8 up, 8 in 2024-04-07T17:23:29.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:23:29 smithi100 bash[74942]: cluster 2024-04-07T17:23:28.001827+0000 mon.a (mon.0) 3604 : cluster [DBG] osdmap e1890: 8 total, 8 up, 8 in 2024-04-07T17:23:30.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:23:30 smithi144 bash[66654]: cluster 2024-04-07T17:23:28.114467+0000 mgr.y (mgr.25175) 2043 : cluster [DBG] pgmap v3508: 137 pgs: 137 active+clean; 30 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 89 KiB/s rd, 3.2 MiB/s wr, 22 op/s 2024-04-07T17:23:30.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:23:30 smithi100 bash[76202]: cluster 2024-04-07T17:23:28.114467+0000 mgr.y (mgr.25175) 2043 : cluster [DBG] pgmap v3508: 137 pgs: 137 active+clean; 30 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 89 KiB/s rd, 3.2 MiB/s wr, 22 op/s 2024-04-07T17:23:30.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:23:30 smithi100 bash[74942]: cluster 2024-04-07T17:23:28.114467+0000 mgr.y (mgr.25175) 2043 : cluster [DBG] pgmap v3508: 137 pgs: 137 active+clean; 30 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 89 KiB/s rd, 3.2 MiB/s wr, 22 op/s 2024-04-07T17:23:31.083 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] EncryptedFlattenTest/2.Simple (14782 ms) 2024-04-07T17:23:31.083 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] EncryptedFlattenTest/2.Grow 2024-04-07T17:23:31.083 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:23:32.316 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:23:32 smithi144 bash[66654]: cluster 2024-04-07T17:23:30.115573+0000 mgr.y (mgr.25175) 2044 : cluster [DBG] pgmap v3509: 137 pgs: 137 active+clean; 35 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 3.2 MiB/s rd, 3.8 MiB/s wr, 57 op/s 2024-04-07T17:23:32.317 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:23:32 smithi144 bash[66654]: audit 2024-04-07T17:23:31.837366+0000 mon.a (mon.0) 3605 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:23:32.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:23:32 smithi100 bash[74942]: cluster 2024-04-07T17:23:30.115573+0000 mgr.y (mgr.25175) 2044 : cluster [DBG] pgmap v3509: 137 pgs: 137 active+clean; 35 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 3.2 MiB/s rd, 3.8 MiB/s wr, 57 op/s 2024-04-07T17:23:32.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:23:32 smithi100 bash[74942]: audit 2024-04-07T17:23:31.837366+0000 mon.a (mon.0) 3605 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:23:32.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:23:32 smithi100 bash[76202]: cluster 2024-04-07T17:23:30.115573+0000 mgr.y (mgr.25175) 2044 : cluster [DBG] pgmap v3509: 137 pgs: 137 active+clean; 35 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 3.2 MiB/s rd, 3.8 MiB/s wr, 57 op/s 2024-04-07T17:23:32.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:23:32 smithi100 bash[76202]: audit 2024-04-07T17:23:31.837366+0000 mon.a (mon.0) 3605 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:23:32.616 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:23:32 smithi144 bash[95233]: ts=2024-04-07T17:23:32.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:23:34.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:23:34 smithi144 bash[66654]: cluster 2024-04-07T17:23:32.116327+0000 mgr.y (mgr.25175) 2045 : cluster [DBG] pgmap v3510: 137 pgs: 137 active+clean; 35 MiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 3.2 MiB/s rd, 3.8 MiB/s wr, 56 op/s 2024-04-07T17:23:34.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:23:34 smithi144 bash[66654]: audit 2024-04-07T17:23:33.577298+0000 mon.a (mon.0) 3606 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:23:34.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:23:34 smithi100 bash[76202]: cluster 2024-04-07T17:23:32.116327+0000 mgr.y (mgr.25175) 2045 : cluster [DBG] pgmap v3510: 137 pgs: 137 active+clean; 35 MiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 3.2 MiB/s rd, 3.8 MiB/s wr, 56 op/s 2024-04-07T17:23:34.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:23:34 smithi100 bash[76202]: audit 2024-04-07T17:23:33.577298+0000 mon.a (mon.0) 3606 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:23:34.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:23:34 smithi100 bash[74942]: cluster 2024-04-07T17:23:32.116327+0000 mgr.y (mgr.25175) 2045 : cluster [DBG] pgmap v3510: 137 pgs: 137 active+clean; 35 MiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 3.2 MiB/s rd, 3.8 MiB/s wr, 56 op/s 2024-04-07T17:23:34.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:23:34 smithi100 bash[74942]: audit 2024-04-07T17:23:33.577298+0000 mon.a (mon.0) 3606 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:23:35.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:23:35 smithi144 bash[66654]: audit 2024-04-07T17:23:34.250046+0000 mon.a (mon.0) 3607 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:23:35.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:23:35 smithi144 bash[66654]: audit 2024-04-07T17:23:34.251047+0000 mon.a (mon.0) 3608 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:23:35.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:23:35 smithi144 bash[66654]: audit 2024-04-07T17:23:34.258591+0000 mon.a (mon.0) 3609 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:23:35.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:23:35 smithi100 bash[74942]: audit 2024-04-07T17:23:34.250046+0000 mon.a (mon.0) 3607 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:23:35.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:23:35 smithi100 bash[74942]: audit 2024-04-07T17:23:34.251047+0000 mon.a (mon.0) 3608 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:23:35.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:23:35 smithi100 bash[74942]: audit 2024-04-07T17:23:34.258591+0000 mon.a (mon.0) 3609 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:23:35.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:23:35 smithi100 bash[76202]: audit 2024-04-07T17:23:34.250046+0000 mon.a (mon.0) 3607 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:23:35.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:23:35 smithi100 bash[76202]: audit 2024-04-07T17:23:34.251047+0000 mon.a (mon.0) 3608 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:23:35.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:23:35 smithi100 bash[76202]: audit 2024-04-07T17:23:34.258591+0000 mon.a (mon.0) 3609 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:23:36.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:23:36 smithi144 bash[66654]: cluster 2024-04-07T17:23:34.117139+0000 mgr.y (mgr.25175) 2046 : cluster [DBG] pgmap v3511: 137 pgs: 137 active+clean; 63 MiB data, 3.7 GiB used, 711 GiB / 715 GiB avail; 11 MiB/s rd, 5.9 MiB/s wr, 61 op/s 2024-04-07T17:23:36.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:23:36 smithi100 bash[76202]: cluster 2024-04-07T17:23:34.117139+0000 mgr.y (mgr.25175) 2046 : cluster [DBG] pgmap v3511: 137 pgs: 137 active+clean; 63 MiB data, 3.7 GiB used, 711 GiB / 715 GiB avail; 11 MiB/s rd, 5.9 MiB/s wr, 61 op/s 2024-04-07T17:23:36.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:23:36 smithi100 bash[74942]: cluster 2024-04-07T17:23:34.117139+0000 mgr.y (mgr.25175) 2046 : cluster [DBG] pgmap v3511: 137 pgs: 137 active+clean; 63 MiB data, 3.7 GiB used, 711 GiB / 715 GiB avail; 11 MiB/s rd, 5.9 MiB/s wr, 61 op/s 2024-04-07T17:23:37.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:23:37 smithi144 bash[66654]: cluster 2024-04-07T17:23:36.055365+0000 mon.a (mon.0) 3610 : cluster [DBG] osdmap e1891: 8 total, 8 up, 8 in 2024-04-07T17:23:37.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:23:37 smithi144 bash[66654]: cluster 2024-04-07T17:23:36.117837+0000 mgr.y (mgr.25175) 2047 : cluster [DBG] pgmap v3513: 137 pgs: 137 active+clean; 63 MiB data, 3.7 GiB used, 711 GiB / 715 GiB avail; 13 MiB/s rd, 4.1 MiB/s wr, 61 op/s 2024-04-07T17:23:37.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:23:37 smithi100 bash[76202]: cluster 2024-04-07T17:23:36.055365+0000 mon.a (mon.0) 3610 : cluster [DBG] osdmap e1891: 8 total, 8 up, 8 in 2024-04-07T17:23:37.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:23:37 smithi100 bash[76202]: cluster 2024-04-07T17:23:36.117837+0000 mgr.y (mgr.25175) 2047 : cluster [DBG] pgmap v3513: 137 pgs: 137 active+clean; 63 MiB data, 3.7 GiB used, 711 GiB / 715 GiB avail; 13 MiB/s rd, 4.1 MiB/s wr, 61 op/s 2024-04-07T17:23:37.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:23:37 smithi100 bash[74942]: cluster 2024-04-07T17:23:36.055365+0000 mon.a (mon.0) 3610 : cluster [DBG] osdmap e1891: 8 total, 8 up, 8 in 2024-04-07T17:23:37.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:23:37 smithi100 bash[74942]: cluster 2024-04-07T17:23:36.117837+0000 mgr.y (mgr.25175) 2047 : cluster [DBG] pgmap v3513: 137 pgs: 137 active+clean; 63 MiB data, 3.7 GiB used, 711 GiB / 715 GiB avail; 13 MiB/s rd, 4.1 MiB/s wr, 61 op/s 2024-04-07T17:23:38.705 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:23:38 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:23:38] "GET /metrics HTTP/1.1" 200 36750 "" "Prometheus/2.43.0" 2024-04-07T17:23:39.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:23:39 smithi100 bash[74942]: cluster 2024-04-07T17:23:38.119047+0000 mgr.y (mgr.25175) 2048 : cluster [DBG] pgmap v3514: 137 pgs: 137 active+clean; 83 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 10 MiB/s rd, 5.3 MiB/s wr, 80 op/s 2024-04-07T17:23:39.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:23:39 smithi100 bash[76202]: cluster 2024-04-07T17:23:38.119047+0000 mgr.y (mgr.25175) 2048 : cluster [DBG] pgmap v3514: 137 pgs: 137 active+clean; 83 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 10 MiB/s rd, 5.3 MiB/s wr, 80 op/s 2024-04-07T17:23:39.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:23:39 smithi144 bash[66654]: cluster 2024-04-07T17:23:38.119047+0000 mgr.y (mgr.25175) 2048 : cluster [DBG] pgmap v3514: 137 pgs: 137 active+clean; 83 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 10 MiB/s rd, 5.3 MiB/s wr, 80 op/s 2024-04-07T17:23:41.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:23:41 smithi100 bash[74942]: cluster 2024-04-07T17:23:40.119724+0000 mgr.y (mgr.25175) 2049 : cluster [DBG] pgmap v3515: 137 pgs: 137 active+clean; 92 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 8.0 MiB/s rd, 5.7 MiB/s wr, 61 op/s 2024-04-07T17:23:41.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:23:41 smithi100 bash[76202]: cluster 2024-04-07T17:23:40.119724+0000 mgr.y (mgr.25175) 2049 : cluster [DBG] pgmap v3515: 137 pgs: 137 active+clean; 92 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 8.0 MiB/s rd, 5.7 MiB/s wr, 61 op/s 2024-04-07T17:23:41.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:23:41 smithi144 bash[66654]: cluster 2024-04-07T17:23:40.119724+0000 mgr.y (mgr.25175) 2049 : cluster [DBG] pgmap v3515: 137 pgs: 137 active+clean; 92 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 8.0 MiB/s rd, 5.7 MiB/s wr, 61 op/s 2024-04-07T17:23:42.616 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:23:42 smithi144 bash[95233]: ts=2024-04-07T17:23:42.317Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:23:43.150 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] EncryptedFlattenTest/2.Grow (12067 ms) 2024-04-07T17:23:43.150 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] EncryptedFlattenTest/2.Shrink 2024-04-07T17:23:43.150 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:23:43.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:23:43 smithi100 bash[74942]: cluster 2024-04-07T17:23:42.120451+0000 mgr.y (mgr.25175) 2050 : cluster [DBG] pgmap v3516: 137 pgs: 137 active+clean; 92 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 8.0 MiB/s rd, 5.7 MiB/s wr, 60 op/s 2024-04-07T17:23:43.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:23:43 smithi100 bash[76202]: cluster 2024-04-07T17:23:42.120451+0000 mgr.y (mgr.25175) 2050 : cluster [DBG] pgmap v3516: 137 pgs: 137 active+clean; 92 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 8.0 MiB/s rd, 5.7 MiB/s wr, 60 op/s 2024-04-07T17:23:43.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:23:43 smithi144 bash[66654]: cluster 2024-04-07T17:23:42.120451+0000 mgr.y (mgr.25175) 2050 : cluster [DBG] pgmap v3516: 137 pgs: 137 active+clean; 92 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 8.0 MiB/s rd, 5.7 MiB/s wr, 60 op/s 2024-04-07T17:23:45.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:23:45 smithi100 bash[74942]: cluster 2024-04-07T17:23:44.121612+0000 mgr.y (mgr.25175) 2051 : cluster [DBG] pgmap v3517: 137 pgs: 137 active+clean; 109 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 6.7 MiB/s rd, 4.6 MiB/s wr, 57 op/s 2024-04-07T17:23:45.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:23:45 smithi100 bash[76202]: cluster 2024-04-07T17:23:44.121612+0000 mgr.y (mgr.25175) 2051 : cluster [DBG] pgmap v3517: 137 pgs: 137 active+clean; 109 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 6.7 MiB/s rd, 4.6 MiB/s wr, 57 op/s 2024-04-07T17:23:45.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:23:45 smithi144 bash[66654]: cluster 2024-04-07T17:23:44.121612+0000 mgr.y (mgr.25175) 2051 : cluster [DBG] pgmap v3517: 137 pgs: 137 active+clean; 109 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 6.7 MiB/s rd, 4.6 MiB/s wr, 57 op/s 2024-04-07T17:23:47.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:23:47 smithi100 bash[74942]: cluster 2024-04-07T17:23:46.122604+0000 mgr.y (mgr.25175) 2052 : cluster [DBG] pgmap v3518: 137 pgs: 137 active+clean; 121 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 9.2 MiB/s rd, 5.8 MiB/s wr, 60 op/s 2024-04-07T17:23:47.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:23:47 smithi100 bash[74942]: audit 2024-04-07T17:23:46.837458+0000 mon.a (mon.0) 3611 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:23:47.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:23:47 smithi100 bash[76202]: cluster 2024-04-07T17:23:46.122604+0000 mgr.y (mgr.25175) 2052 : cluster [DBG] pgmap v3518: 137 pgs: 137 active+clean; 121 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 9.2 MiB/s rd, 5.8 MiB/s wr, 60 op/s 2024-04-07T17:23:47.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:23:47 smithi100 bash[76202]: audit 2024-04-07T17:23:46.837458+0000 mon.a (mon.0) 3611 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:23:47.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:23:47 smithi144 bash[66654]: cluster 2024-04-07T17:23:46.122604+0000 mgr.y (mgr.25175) 2052 : cluster [DBG] pgmap v3518: 137 pgs: 137 active+clean; 121 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 9.2 MiB/s rd, 5.8 MiB/s wr, 60 op/s 2024-04-07T17:23:47.617 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:23:47 smithi144 bash[66654]: audit 2024-04-07T17:23:46.837458+0000 mon.a (mon.0) 3611 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:23:48.617 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:23:48 smithi144 bash[66654]: cluster 2024-04-07T17:23:47.208888+0000 mon.a (mon.0) 3612 : cluster [DBG] osdmap e1892: 8 total, 8 up, 8 in 2024-04-07T17:23:48.703 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:23:48 smithi100 bash[74942]: cluster 2024-04-07T17:23:47.208888+0000 mon.a (mon.0) 3612 : cluster [DBG] osdmap e1892: 8 total, 8 up, 8 in 2024-04-07T17:23:48.704 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:23:48 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:23:48] "GET /metrics HTTP/1.1" 200 36759 "" "Prometheus/2.43.0" 2024-04-07T17:23:48.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:23:48 smithi100 bash[76202]: cluster 2024-04-07T17:23:47.208888+0000 mon.a (mon.0) 3612 : cluster [DBG] osdmap e1892: 8 total, 8 up, 8 in 2024-04-07T17:23:49.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:23:49 smithi144 bash[66654]: cluster 2024-04-07T17:23:48.123198+0000 mgr.y (mgr.25175) 2053 : cluster [DBG] pgmap v3520: 137 pgs: 137 active+clean; 138 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 11 MiB/s rd, 5.5 MiB/s wr, 38 op/s 2024-04-07T17:23:49.703 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:23:49 smithi100 bash[74942]: cluster 2024-04-07T17:23:48.123198+0000 mgr.y (mgr.25175) 2053 : cluster [DBG] pgmap v3520: 137 pgs: 137 active+clean; 138 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 11 MiB/s rd, 5.5 MiB/s wr, 38 op/s 2024-04-07T17:23:49.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:23:49 smithi100 bash[76202]: cluster 2024-04-07T17:23:48.123198+0000 mgr.y (mgr.25175) 2053 : cluster [DBG] pgmap v3520: 137 pgs: 137 active+clean; 138 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 11 MiB/s rd, 5.5 MiB/s wr, 38 op/s 2024-04-07T17:23:51.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:23:51 smithi144 bash[66654]: cluster 2024-04-07T17:23:50.124222+0000 mgr.y (mgr.25175) 2054 : cluster [DBG] pgmap v3521: 137 pgs: 137 active+clean; 150 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 10 MiB/s rd, 5.8 MiB/s wr, 38 op/s 2024-04-07T17:23:51.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:23:51 smithi100 bash[76202]: cluster 2024-04-07T17:23:50.124222+0000 mgr.y (mgr.25175) 2054 : cluster [DBG] pgmap v3521: 137 pgs: 137 active+clean; 150 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 10 MiB/s rd, 5.8 MiB/s wr, 38 op/s 2024-04-07T17:23:51.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:23:51 smithi100 bash[74942]: cluster 2024-04-07T17:23:50.124222+0000 mgr.y (mgr.25175) 2054 : cluster [DBG] pgmap v3521: 137 pgs: 137 active+clean; 150 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 10 MiB/s rd, 5.8 MiB/s wr, 38 op/s 2024-04-07T17:23:52.617 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:23:52 smithi144 bash[95233]: ts=2024-04-07T17:23:52.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:23:53.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:23:53 smithi144 bash[66654]: cluster 2024-04-07T17:23:52.124866+0000 mgr.y (mgr.25175) 2055 : cluster [DBG] pgmap v3522: 137 pgs: 137 active+clean; 150 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 10 MiB/s rd, 5.8 MiB/s wr, 38 op/s 2024-04-07T17:23:53.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:23:53 smithi100 bash[74942]: cluster 2024-04-07T17:23:52.124866+0000 mgr.y (mgr.25175) 2055 : cluster [DBG] pgmap v3522: 137 pgs: 137 active+clean; 150 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 10 MiB/s rd, 5.8 MiB/s wr, 38 op/s 2024-04-07T17:23:53.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:23:53 smithi100 bash[76202]: cluster 2024-04-07T17:23:52.124866+0000 mgr.y (mgr.25175) 2055 : cluster [DBG] pgmap v3522: 137 pgs: 137 active+clean; 150 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 10 MiB/s rd, 5.8 MiB/s wr, 38 op/s 2024-04-07T17:23:54.342 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] EncryptedFlattenTest/2.Shrink (11192 ms) 2024-04-07T17:23:54.342 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] EncryptedFlattenTest/2.ShrinkToOne 2024-04-07T17:23:54.342 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:23:55.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:23:55 smithi144 bash[66654]: cluster 2024-04-07T17:23:54.126056+0000 mgr.y (mgr.25175) 2056 : cluster [DBG] pgmap v3523: 137 pgs: 137 active+clean; 170 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 9.3 MiB/s rd, 6.1 MiB/s wr, 57 op/s 2024-04-07T17:23:55.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:23:55 smithi100 bash[76202]: cluster 2024-04-07T17:23:54.126056+0000 mgr.y (mgr.25175) 2056 : cluster [DBG] pgmap v3523: 137 pgs: 137 active+clean; 170 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 9.3 MiB/s rd, 6.1 MiB/s wr, 57 op/s 2024-04-07T17:23:55.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:23:55 smithi100 bash[74942]: cluster 2024-04-07T17:23:54.126056+0000 mgr.y (mgr.25175) 2056 : cluster [DBG] pgmap v3523: 137 pgs: 137 active+clean; 170 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 9.3 MiB/s rd, 6.1 MiB/s wr, 57 op/s 2024-04-07T17:23:57.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:23:57 smithi144 bash[66654]: cluster 2024-04-07T17:23:56.127037+0000 mgr.y (mgr.25175) 2057 : cluster [DBG] pgmap v3524: 137 pgs: 137 active+clean; 179 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 10 MiB/s rd, 5.8 MiB/s wr, 64 op/s 2024-04-07T17:23:57.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:23:57 smithi100 bash[74942]: cluster 2024-04-07T17:23:56.127037+0000 mgr.y (mgr.25175) 2057 : cluster [DBG] pgmap v3524: 137 pgs: 137 active+clean; 179 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 10 MiB/s rd, 5.8 MiB/s wr, 64 op/s 2024-04-07T17:23:57.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:23:57 smithi100 bash[76202]: cluster 2024-04-07T17:23:56.127037+0000 mgr.y (mgr.25175) 2057 : cluster [DBG] pgmap v3524: 137 pgs: 137 active+clean; 179 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 10 MiB/s rd, 5.8 MiB/s wr, 64 op/s 2024-04-07T17:23:58.704 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:23:58 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:23:58] "GET /metrics HTTP/1.1" 200 36759 "" "Prometheus/2.43.0" 2024-04-07T17:23:59.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:23:59 smithi144 bash[66654]: cluster 2024-04-07T17:23:58.128465+0000 mgr.y (mgr.25175) 2058 : cluster [DBG] pgmap v3525: 137 pgs: 137 active+clean; 179 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 9.7 MiB/s rd, 4.9 MiB/s wr, 57 op/s 2024-04-07T17:23:59.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:23:59 smithi100 bash[74942]: cluster 2024-04-07T17:23:58.128465+0000 mgr.y (mgr.25175) 2058 : cluster [DBG] pgmap v3525: 137 pgs: 137 active+clean; 179 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 9.7 MiB/s rd, 4.9 MiB/s wr, 57 op/s 2024-04-07T17:23:59.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:23:59 smithi100 bash[76202]: cluster 2024-04-07T17:23:58.128465+0000 mgr.y (mgr.25175) 2058 : cluster [DBG] pgmap v3525: 137 pgs: 137 active+clean; 179 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 9.7 MiB/s rd, 4.9 MiB/s wr, 57 op/s 2024-04-07T17:24:00.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:24:00 smithi144 bash[66654]: cluster 2024-04-07T17:23:59.259234+0000 mon.a (mon.0) 3613 : cluster [DBG] osdmap e1893: 8 total, 8 up, 8 in 2024-04-07T17:24:00.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:24:00 smithi100 bash[74942]: cluster 2024-04-07T17:23:59.259234+0000 mon.a (mon.0) 3613 : cluster [DBG] osdmap e1893: 8 total, 8 up, 8 in 2024-04-07T17:24:00.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:24:00 smithi100 bash[76202]: cluster 2024-04-07T17:23:59.259234+0000 mon.a (mon.0) 3613 : cluster [DBG] osdmap e1893: 8 total, 8 up, 8 in 2024-04-07T17:24:01.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:24:01 smithi144 bash[66654]: cluster 2024-04-07T17:24:00.129403+0000 mgr.y (mgr.25175) 2059 : cluster [DBG] pgmap v3527: 137 pgs: 137 active+clean; 192 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 10 MiB/s rd, 4.2 MiB/s wr, 59 op/s 2024-04-07T17:24:01.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:24:01 smithi100 bash[74942]: cluster 2024-04-07T17:24:00.129403+0000 mgr.y (mgr.25175) 2059 : cluster [DBG] pgmap v3527: 137 pgs: 137 active+clean; 192 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 10 MiB/s rd, 4.2 MiB/s wr, 59 op/s 2024-04-07T17:24:01.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:24:01 smithi100 bash[76202]: cluster 2024-04-07T17:24:00.129403+0000 mgr.y (mgr.25175) 2059 : cluster [DBG] pgmap v3527: 137 pgs: 137 active+clean; 192 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 10 MiB/s rd, 4.2 MiB/s wr, 59 op/s 2024-04-07T17:24:02.617 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:24:02 smithi144 bash[66654]: audit 2024-04-07T17:24:01.837811+0000 mon.a (mon.0) 3614 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:24:02.617 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:24:02 smithi144 bash[95233]: ts=2024-04-07T17:24:02.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:24:02.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:24:02 smithi100 bash[74942]: audit 2024-04-07T17:24:01.837811+0000 mon.a (mon.0) 3614 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:24:02.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:24:02 smithi100 bash[76202]: audit 2024-04-07T17:24:01.837811+0000 mon.a (mon.0) 3614 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:24:03.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:24:03 smithi144 bash[66654]: cluster 2024-04-07T17:24:02.130394+0000 mgr.y (mgr.25175) 2060 : cluster [DBG] pgmap v3528: 137 pgs: 137 active+clean; 192 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 10 MiB/s rd, 4.2 MiB/s wr, 59 op/s 2024-04-07T17:24:03.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:24:03 smithi100 bash[74942]: cluster 2024-04-07T17:24:02.130394+0000 mgr.y (mgr.25175) 2060 : cluster [DBG] pgmap v3528: 137 pgs: 137 active+clean; 192 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 10 MiB/s rd, 4.2 MiB/s wr, 59 op/s 2024-04-07T17:24:03.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:24:03 smithi100 bash[76202]: cluster 2024-04-07T17:24:02.130394+0000 mgr.y (mgr.25175) 2060 : cluster [DBG] pgmap v3528: 137 pgs: 137 active+clean; 192 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 10 MiB/s rd, 4.2 MiB/s wr, 59 op/s 2024-04-07T17:24:05.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:24:05 smithi144 bash[66654]: cluster 2024-04-07T17:24:04.131207+0000 mgr.y (mgr.25175) 2061 : cluster [DBG] pgmap v3529: 137 pgs: 137 active+clean; 208 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 5.2 MiB/s rd, 3.8 MiB/s wr, 53 op/s 2024-04-07T17:24:05.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:24:05 smithi100 bash[74942]: cluster 2024-04-07T17:24:04.131207+0000 mgr.y (mgr.25175) 2061 : cluster [DBG] pgmap v3529: 137 pgs: 137 active+clean; 208 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 5.2 MiB/s rd, 3.8 MiB/s wr, 53 op/s 2024-04-07T17:24:05.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:24:05 smithi100 bash[76202]: cluster 2024-04-07T17:24:04.131207+0000 mgr.y (mgr.25175) 2061 : cluster [DBG] pgmap v3529: 137 pgs: 137 active+clean; 208 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 5.2 MiB/s rd, 3.8 MiB/s wr, 53 op/s 2024-04-07T17:24:06.079 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] EncryptedFlattenTest/2.ShrinkToOne (11737 ms) 2024-04-07T17:24:06.079 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] EncryptedFlattenTest/2.ShrinkToOneAfterSnapshot 2024-04-07T17:24:06.079 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:24:07.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:24:07 smithi144 bash[66654]: cluster 2024-04-07T17:24:06.131777+0000 mgr.y (mgr.25175) 2062 : cluster [DBG] pgmap v3530: 137 pgs: 137 active+clean; 208 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 1.7 MiB/s rd, 2.9 MiB/s wr, 45 op/s 2024-04-07T17:24:07.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:24:07 smithi100 bash[74942]: cluster 2024-04-07T17:24:06.131777+0000 mgr.y (mgr.25175) 2062 : cluster [DBG] pgmap v3530: 137 pgs: 137 active+clean; 208 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 1.7 MiB/s rd, 2.9 MiB/s wr, 45 op/s 2024-04-07T17:24:07.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:24:07 smithi100 bash[76202]: cluster 2024-04-07T17:24:06.131777+0000 mgr.y (mgr.25175) 2062 : cluster [DBG] pgmap v3530: 137 pgs: 137 active+clean; 208 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 1.7 MiB/s rd, 2.9 MiB/s wr, 45 op/s 2024-04-07T17:24:08.704 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:24:08 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:24:08] "GET /metrics HTTP/1.1" 200 36757 "" "Prometheus/2.43.0" 2024-04-07T17:24:09.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:24:09 smithi144 bash[66654]: cluster 2024-04-07T17:24:08.133078+0000 mgr.y (mgr.25175) 2063 : cluster [DBG] pgmap v3531: 137 pgs: 137 active+clean; 216 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 1.7 MiB/s rd, 4.1 MiB/s wr, 54 op/s 2024-04-07T17:24:09.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:24:09 smithi100 bash[74942]: cluster 2024-04-07T17:24:08.133078+0000 mgr.y (mgr.25175) 2063 : cluster [DBG] pgmap v3531: 137 pgs: 137 active+clean; 216 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 1.7 MiB/s rd, 4.1 MiB/s wr, 54 op/s 2024-04-07T17:24:09.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:24:09 smithi100 bash[76202]: cluster 2024-04-07T17:24:08.133078+0000 mgr.y (mgr.25175) 2063 : cluster [DBG] pgmap v3531: 137 pgs: 137 active+clean; 216 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 1.7 MiB/s rd, 4.1 MiB/s wr, 54 op/s 2024-04-07T17:24:11.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:24:11 smithi144 bash[66654]: cluster 2024-04-07T17:24:10.134221+0000 mgr.y (mgr.25175) 2064 : cluster [DBG] pgmap v3532: 137 pgs: 137 active+clean; 216 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 867 KiB/s rd, 3.7 MiB/s wr, 47 op/s 2024-04-07T17:24:11.617 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:24:11 smithi144 bash[66654]: cluster 2024-04-07T17:24:10.314846+0000 mon.a (mon.0) 3615 : cluster [DBG] osdmap e1894: 8 total, 8 up, 8 in 2024-04-07T17:24:11.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:24:11 smithi100 bash[74942]: cluster 2024-04-07T17:24:10.134221+0000 mgr.y (mgr.25175) 2064 : cluster [DBG] pgmap v3532: 137 pgs: 137 active+clean; 216 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 867 KiB/s rd, 3.7 MiB/s wr, 47 op/s 2024-04-07T17:24:11.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:24:11 smithi100 bash[74942]: cluster 2024-04-07T17:24:10.314846+0000 mon.a (mon.0) 3615 : cluster [DBG] osdmap e1894: 8 total, 8 up, 8 in 2024-04-07T17:24:11.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:24:11 smithi100 bash[76202]: cluster 2024-04-07T17:24:10.134221+0000 mgr.y (mgr.25175) 2064 : cluster [DBG] pgmap v3532: 137 pgs: 137 active+clean; 216 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 867 KiB/s rd, 3.7 MiB/s wr, 47 op/s 2024-04-07T17:24:11.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:24:11 smithi100 bash[76202]: cluster 2024-04-07T17:24:10.314846+0000 mon.a (mon.0) 3615 : cluster [DBG] osdmap e1894: 8 total, 8 up, 8 in 2024-04-07T17:24:12.616 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:24:12 smithi144 bash[95233]: ts=2024-04-07T17:24:12.318Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:24:13.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:24:13 smithi144 bash[66654]: cluster 2024-04-07T17:24:12.135075+0000 mgr.y (mgr.25175) 2065 : cluster [DBG] pgmap v3534: 137 pgs: 137 active+clean; 224 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 983 KiB/s rd, 3.6 MiB/s wr, 66 op/s 2024-04-07T17:24:13.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:24:13 smithi100 bash[74942]: cluster 2024-04-07T17:24:12.135075+0000 mgr.y (mgr.25175) 2065 : cluster [DBG] pgmap v3534: 137 pgs: 137 active+clean; 224 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 983 KiB/s rd, 3.6 MiB/s wr, 66 op/s 2024-04-07T17:24:13.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:24:13 smithi100 bash[76202]: cluster 2024-04-07T17:24:12.135075+0000 mgr.y (mgr.25175) 2065 : cluster [DBG] pgmap v3534: 137 pgs: 137 active+clean; 224 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 983 KiB/s rd, 3.6 MiB/s wr, 66 op/s 2024-04-07T17:24:15.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:24:15 smithi144 bash[66654]: cluster 2024-04-07T17:24:14.136271+0000 mgr.y (mgr.25175) 2066 : cluster [DBG] pgmap v3535: 137 pgs: 137 active+clean; 236 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 902 KiB/s rd, 3.2 MiB/s wr, 37 op/s 2024-04-07T17:24:15.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:24:15 smithi100 bash[74942]: cluster 2024-04-07T17:24:14.136271+0000 mgr.y (mgr.25175) 2066 : cluster [DBG] pgmap v3535: 137 pgs: 137 active+clean; 236 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 902 KiB/s rd, 3.2 MiB/s wr, 37 op/s 2024-04-07T17:24:15.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:24:15 smithi100 bash[76202]: cluster 2024-04-07T17:24:14.136271+0000 mgr.y (mgr.25175) 2066 : cluster [DBG] pgmap v3535: 137 pgs: 137 active+clean; 236 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 902 KiB/s rd, 3.2 MiB/s wr, 37 op/s 2024-04-07T17:24:16.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:24:16 smithi144 bash[66654]: cluster 2024-04-07T17:24:15.332888+0000 mon.a (mon.0) 3616 : cluster [DBG] osdmap e1895: 8 total, 8 up, 8 in 2024-04-07T17:24:16.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:24:16 smithi100 bash[76202]: cluster 2024-04-07T17:24:15.332888+0000 mon.a (mon.0) 3616 : cluster [DBG] osdmap e1895: 8 total, 8 up, 8 in 2024-04-07T17:24:16.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:24:16 smithi100 bash[74942]: cluster 2024-04-07T17:24:15.332888+0000 mon.a (mon.0) 3616 : cluster [DBG] osdmap e1895: 8 total, 8 up, 8 in 2024-04-07T17:24:17.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:24:17 smithi144 bash[66654]: cluster 2024-04-07T17:24:16.137048+0000 mgr.y (mgr.25175) 2067 : cluster [DBG] pgmap v3537: 137 pgs: 137 active+clean; 245 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 76 KiB/s rd, 3.6 MiB/s wr, 53 op/s 2024-04-07T17:24:17.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:24:17 smithi144 bash[66654]: audit 2024-04-07T17:24:16.838169+0000 mon.a (mon.0) 3617 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:24:17.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:24:17 smithi100 bash[76202]: cluster 2024-04-07T17:24:16.137048+0000 mgr.y (mgr.25175) 2067 : cluster [DBG] pgmap v3537: 137 pgs: 137 active+clean; 245 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 76 KiB/s rd, 3.6 MiB/s wr, 53 op/s 2024-04-07T17:24:17.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:24:17 smithi100 bash[76202]: audit 2024-04-07T17:24:16.838169+0000 mon.a (mon.0) 3617 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:24:17.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:24:17 smithi100 bash[74942]: cluster 2024-04-07T17:24:16.137048+0000 mgr.y (mgr.25175) 2067 : cluster [DBG] pgmap v3537: 137 pgs: 137 active+clean; 245 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 76 KiB/s rd, 3.6 MiB/s wr, 53 op/s 2024-04-07T17:24:17.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:24:17 smithi100 bash[74942]: audit 2024-04-07T17:24:16.838169+0000 mon.a (mon.0) 3617 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:24:17.943 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] EncryptedFlattenTest/2.ShrinkToOneAfterSnapshot (11865 ms) 2024-04-07T17:24:17.944 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] EncryptedFlattenTest/2.MinOverlap 2024-04-07T17:24:17.944 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:24:18.704 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:24:18 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:24:18] "GET /metrics HTTP/1.1" 200 36756 "" "Prometheus/2.43.0" 2024-04-07T17:24:19.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:24:19 smithi144 bash[66654]: cluster 2024-04-07T17:24:18.137869+0000 mgr.y (mgr.25175) 2068 : cluster [DBG] pgmap v3538: 137 pgs: 137 active+clean; 285 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 3.1 MiB/s rd, 8.6 MiB/s wr, 73 op/s 2024-04-07T17:24:19.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:24:19 smithi100 bash[74942]: cluster 2024-04-07T17:24:18.137869+0000 mgr.y (mgr.25175) 2068 : cluster [DBG] pgmap v3538: 137 pgs: 137 active+clean; 285 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 3.1 MiB/s rd, 8.6 MiB/s wr, 73 op/s 2024-04-07T17:24:19.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:24:19 smithi100 bash[76202]: cluster 2024-04-07T17:24:18.137869+0000 mgr.y (mgr.25175) 2068 : cluster [DBG] pgmap v3538: 137 pgs: 137 active+clean; 285 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 3.1 MiB/s rd, 8.6 MiB/s wr, 73 op/s 2024-04-07T17:24:21.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:24:21 smithi144 bash[66654]: cluster 2024-04-07T17:24:20.138888+0000 mgr.y (mgr.25175) 2069 : cluster [DBG] pgmap v3539: 137 pgs: 137 active+clean; 306 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 3.5 MiB/s rd, 9.2 MiB/s wr, 73 op/s 2024-04-07T17:24:21.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:24:21 smithi100 bash[74942]: cluster 2024-04-07T17:24:20.138888+0000 mgr.y (mgr.25175) 2069 : cluster [DBG] pgmap v3539: 137 pgs: 137 active+clean; 306 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 3.5 MiB/s rd, 9.2 MiB/s wr, 73 op/s 2024-04-07T17:24:21.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:24:21 smithi100 bash[76202]: cluster 2024-04-07T17:24:20.138888+0000 mgr.y (mgr.25175) 2069 : cluster [DBG] pgmap v3539: 137 pgs: 137 active+clean; 306 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 3.5 MiB/s rd, 9.2 MiB/s wr, 73 op/s 2024-04-07T17:24:22.616 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:24:22 smithi144 bash[95233]: ts=2024-04-07T17:24:22.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:24:23.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:24:23 smithi144 bash[66654]: cluster 2024-04-07T17:24:22.139865+0000 mgr.y (mgr.25175) 2070 : cluster [DBG] pgmap v3540: 137 pgs: 137 active+clean; 310 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 3.4 MiB/s rd, 8.6 MiB/s wr, 57 op/s 2024-04-07T17:24:23.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:24:23 smithi144 bash[66654]: cluster 2024-04-07T17:24:22.362624+0000 mon.a (mon.0) 3618 : cluster [DBG] osdmap e1896: 8 total, 8 up, 8 in 2024-04-07T17:24:23.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:24:23 smithi100 bash[74942]: cluster 2024-04-07T17:24:22.139865+0000 mgr.y (mgr.25175) 2070 : cluster [DBG] pgmap v3540: 137 pgs: 137 active+clean; 310 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 3.4 MiB/s rd, 8.6 MiB/s wr, 57 op/s 2024-04-07T17:24:23.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:24:23 smithi100 bash[74942]: cluster 2024-04-07T17:24:22.362624+0000 mon.a (mon.0) 3618 : cluster [DBG] osdmap e1896: 8 total, 8 up, 8 in 2024-04-07T17:24:23.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:24:23 smithi100 bash[76202]: cluster 2024-04-07T17:24:22.139865+0000 mgr.y (mgr.25175) 2070 : cluster [DBG] pgmap v3540: 137 pgs: 137 active+clean; 310 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 3.4 MiB/s rd, 8.6 MiB/s wr, 57 op/s 2024-04-07T17:24:23.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:24:23 smithi100 bash[76202]: cluster 2024-04-07T17:24:22.362624+0000 mon.a (mon.0) 3618 : cluster [DBG] osdmap e1896: 8 total, 8 up, 8 in 2024-04-07T17:24:25.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:24:25 smithi144 bash[66654]: cluster 2024-04-07T17:24:24.140777+0000 mgr.y (mgr.25175) 2071 : cluster [DBG] pgmap v3542: 137 pgs: 137 active+clean; 327 MiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 3.8 MiB/s rd, 9.3 MiB/s wr, 51 op/s 2024-04-07T17:24:25.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:24:25 smithi100 bash[74942]: cluster 2024-04-07T17:24:24.140777+0000 mgr.y (mgr.25175) 2071 : cluster [DBG] pgmap v3542: 137 pgs: 137 active+clean; 327 MiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 3.8 MiB/s rd, 9.3 MiB/s wr, 51 op/s 2024-04-07T17:24:25.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:24:25 smithi100 bash[76202]: cluster 2024-04-07T17:24:24.140777+0000 mgr.y (mgr.25175) 2071 : cluster [DBG] pgmap v3542: 137 pgs: 137 active+clean; 327 MiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 3.8 MiB/s rd, 9.3 MiB/s wr, 51 op/s 2024-04-07T17:24:27.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:24:27 smithi144 bash[66654]: cluster 2024-04-07T17:24:26.141813+0000 mgr.y (mgr.25175) 2072 : cluster [DBG] pgmap v3543: 137 pgs: 137 active+clean; 335 MiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 3.5 MiB/s rd, 9.0 MiB/s wr, 75 op/s 2024-04-07T17:24:27.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:24:27 smithi100 bash[74942]: cluster 2024-04-07T17:24:26.141813+0000 mgr.y (mgr.25175) 2072 : cluster [DBG] pgmap v3543: 137 pgs: 137 active+clean; 335 MiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 3.5 MiB/s rd, 9.0 MiB/s wr, 75 op/s 2024-04-07T17:24:27.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:24:27 smithi100 bash[76202]: cluster 2024-04-07T17:24:26.141813+0000 mgr.y (mgr.25175) 2072 : cluster [DBG] pgmap v3543: 137 pgs: 137 active+clean; 335 MiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 3.5 MiB/s rd, 9.0 MiB/s wr, 75 op/s 2024-04-07T17:24:28.704 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:24:28 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:24:28] "GET /metrics HTTP/1.1" 200 36756 "" "Prometheus/2.43.0" 2024-04-07T17:24:29.094 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] EncryptedFlattenTest/2.MinOverlap (11150 ms) 2024-04-07T17:24:29.094 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] EncryptedFlattenTest/2.ZeroOverlap 2024-04-07T17:24:29.095 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:24:29.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:24:29 smithi100 bash[74942]: cluster 2024-04-07T17:24:28.143116+0000 mgr.y (mgr.25175) 2073 : cluster [DBG] pgmap v3544: 137 pgs: 137 active+clean; 335 MiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 1.0 MiB/s rd, 5.0 MiB/s wr, 54 op/s 2024-04-07T17:24:29.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:24:29 smithi100 bash[76202]: cluster 2024-04-07T17:24:28.143116+0000 mgr.y (mgr.25175) 2073 : cluster [DBG] pgmap v3544: 137 pgs: 137 active+clean; 335 MiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 1.0 MiB/s rd, 5.0 MiB/s wr, 54 op/s 2024-04-07T17:24:29.866 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:24:29 smithi144 bash[66654]: cluster 2024-04-07T17:24:28.143116+0000 mgr.y (mgr.25175) 2073 : cluster [DBG] pgmap v3544: 137 pgs: 137 active+clean; 335 MiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 1.0 MiB/s rd, 5.0 MiB/s wr, 54 op/s 2024-04-07T17:24:31.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:24:31 smithi100 bash[74942]: cluster 2024-04-07T17:24:30.144271+0000 mgr.y (mgr.25175) 2074 : cluster [DBG] pgmap v3545: 137 pgs: 137 active+clean; 343 MiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 1.7 MiB/s rd, 4.1 MiB/s wr, 57 op/s 2024-04-07T17:24:31.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:24:31 smithi100 bash[76202]: cluster 2024-04-07T17:24:30.144271+0000 mgr.y (mgr.25175) 2074 : cluster [DBG] pgmap v3545: 137 pgs: 137 active+clean; 343 MiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 1.7 MiB/s rd, 4.1 MiB/s wr, 57 op/s 2024-04-07T17:24:31.866 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:24:31 smithi144 bash[66654]: cluster 2024-04-07T17:24:30.144271+0000 mgr.y (mgr.25175) 2074 : cluster [DBG] pgmap v3545: 137 pgs: 137 active+clean; 343 MiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 1.7 MiB/s rd, 4.1 MiB/s wr, 57 op/s 2024-04-07T17:24:32.617 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:24:32 smithi144 bash[95233]: ts=2024-04-07T17:24:32.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:24:32.617 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:24:32 smithi144 bash[66654]: audit 2024-04-07T17:24:31.838821+0000 mon.a (mon.0) 3619 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:24:32.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:24:32 smithi100 bash[74942]: audit 2024-04-07T17:24:31.838821+0000 mon.a (mon.0) 3619 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:24:32.705 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:24:32 smithi100 bash[76202]: audit 2024-04-07T17:24:31.838821+0000 mon.a (mon.0) 3619 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:24:33.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:24:33 smithi100 bash[74942]: cluster 2024-04-07T17:24:32.145127+0000 mgr.y (mgr.25175) 2075 : cluster [DBG] pgmap v3546: 137 pgs: 137 active+clean; 343 MiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 1.7 MiB/s rd, 3.7 MiB/s wr, 56 op/s 2024-04-07T17:24:33.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:24:33 smithi100 bash[76202]: cluster 2024-04-07T17:24:32.145127+0000 mgr.y (mgr.25175) 2075 : cluster [DBG] pgmap v3546: 137 pgs: 137 active+clean; 343 MiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 1.7 MiB/s rd, 3.7 MiB/s wr, 56 op/s 2024-04-07T17:24:33.866 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:24:33 smithi144 bash[66654]: cluster 2024-04-07T17:24:32.145127+0000 mgr.y (mgr.25175) 2075 : cluster [DBG] pgmap v3546: 137 pgs: 137 active+clean; 343 MiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 1.7 MiB/s rd, 3.7 MiB/s wr, 56 op/s 2024-04-07T17:24:34.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:24:34 smithi100 bash[76202]: cluster 2024-04-07T17:24:33.393507+0000 mon.a (mon.0) 3620 : cluster [DBG] osdmap e1897: 8 total, 8 up, 8 in 2024-04-07T17:24:34.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:24:34 smithi100 bash[76202]: audit 2024-04-07T17:24:34.321021+0000 mon.a (mon.0) 3621 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:24:34.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:24:34 smithi100 bash[74942]: cluster 2024-04-07T17:24:33.393507+0000 mon.a (mon.0) 3620 : cluster [DBG] osdmap e1897: 8 total, 8 up, 8 in 2024-04-07T17:24:34.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:24:34 smithi100 bash[74942]: audit 2024-04-07T17:24:34.321021+0000 mon.a (mon.0) 3621 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:24:34.866 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:24:34 smithi144 bash[66654]: cluster 2024-04-07T17:24:33.393507+0000 mon.a (mon.0) 3620 : cluster [DBG] osdmap e1897: 8 total, 8 up, 8 in 2024-04-07T17:24:34.867 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:24:34 smithi144 bash[66654]: audit 2024-04-07T17:24:34.321021+0000 mon.a (mon.0) 3621 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:24:35.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:24:35 smithi100 bash[74942]: cluster 2024-04-07T17:24:34.145657+0000 mgr.y (mgr.25175) 2076 : cluster [DBG] pgmap v3548: 137 pgs: 137 active+clean; 348 MiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 2.1 MiB/s rd, 2.5 MiB/s wr, 69 op/s 2024-04-07T17:24:35.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:24:35 smithi100 bash[74942]: audit 2024-04-07T17:24:35.000598+0000 mon.a (mon.0) 3622 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:24:35.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:24:35 smithi100 bash[74942]: audit 2024-04-07T17:24:35.001717+0000 mon.a (mon.0) 3623 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:24:35.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:24:35 smithi100 bash[74942]: audit 2024-04-07T17:24:35.008047+0000 mon.a (mon.0) 3624 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:24:35.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:24:35 smithi100 bash[76202]: cluster 2024-04-07T17:24:34.145657+0000 mgr.y (mgr.25175) 2076 : cluster [DBG] pgmap v3548: 137 pgs: 137 active+clean; 348 MiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 2.1 MiB/s rd, 2.5 MiB/s wr, 69 op/s 2024-04-07T17:24:35.705 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:24:35 smithi100 bash[76202]: audit 2024-04-07T17:24:35.000598+0000 mon.a (mon.0) 3622 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:24:35.705 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:24:35 smithi100 bash[76202]: audit 2024-04-07T17:24:35.001717+0000 mon.a (mon.0) 3623 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:24:35.705 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:24:35 smithi100 bash[76202]: audit 2024-04-07T17:24:35.008047+0000 mon.a (mon.0) 3624 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:24:35.866 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:24:35 smithi144 bash[66654]: cluster 2024-04-07T17:24:34.145657+0000 mgr.y (mgr.25175) 2076 : cluster [DBG] pgmap v3548: 137 pgs: 137 active+clean; 348 MiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 2.1 MiB/s rd, 2.5 MiB/s wr, 69 op/s 2024-04-07T17:24:35.867 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:24:35 smithi144 bash[66654]: audit 2024-04-07T17:24:35.000598+0000 mon.a (mon.0) 3622 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:24:35.867 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:24:35 smithi144 bash[66654]: audit 2024-04-07T17:24:35.001717+0000 mon.a (mon.0) 3623 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:24:35.867 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:24:35 smithi144 bash[66654]: audit 2024-04-07T17:24:35.008047+0000 mon.a (mon.0) 3624 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:24:37.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:24:37 smithi100 bash[74942]: cluster 2024-04-07T17:24:36.146604+0000 mgr.y (mgr.25175) 2077 : cluster [DBG] pgmap v3549: 137 pgs: 137 active+clean; 352 MiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 55 op/s 2024-04-07T17:24:37.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:24:37 smithi100 bash[76202]: cluster 2024-04-07T17:24:36.146604+0000 mgr.y (mgr.25175) 2077 : cluster [DBG] pgmap v3549: 137 pgs: 137 active+clean; 352 MiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 55 op/s 2024-04-07T17:24:37.866 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:24:37 smithi144 bash[66654]: cluster 2024-04-07T17:24:36.146604+0000 mgr.y (mgr.25175) 2077 : cluster [DBG] pgmap v3549: 137 pgs: 137 active+clean; 352 MiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 55 op/s 2024-04-07T17:24:38.704 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:24:38 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:24:38] "GET /metrics HTTP/1.1" 200 36756 "" "Prometheus/2.43.0" 2024-04-07T17:24:39.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:24:39 smithi100 bash[74942]: cluster 2024-04-07T17:24:38.147881+0000 mgr.y (mgr.25175) 2078 : cluster [DBG] pgmap v3550: 137 pgs: 137 active+clean; 372 MiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 2.1 MiB/s rd, 4.1 MiB/s wr, 58 op/s 2024-04-07T17:24:39.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:24:39 smithi100 bash[76202]: cluster 2024-04-07T17:24:38.147881+0000 mgr.y (mgr.25175) 2078 : cluster [DBG] pgmap v3550: 137 pgs: 137 active+clean; 372 MiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 2.1 MiB/s rd, 4.1 MiB/s wr, 58 op/s 2024-04-07T17:24:39.866 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:24:39 smithi144 bash[66654]: cluster 2024-04-07T17:24:38.147881+0000 mgr.y (mgr.25175) 2078 : cluster [DBG] pgmap v3550: 137 pgs: 137 active+clean; 372 MiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 2.1 MiB/s rd, 4.1 MiB/s wr, 58 op/s 2024-04-07T17:24:40.220 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] EncryptedFlattenTest/2.ZeroOverlap (11126 ms) 2024-04-07T17:24:40.441 INFO:tasks.workunit.client.0.smithi100.stdout:[----------] 7 tests from EncryptedFlattenTest/2 (83919 ms total) 2024-04-07T17:24:40.442 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:24:40.442 INFO:tasks.workunit.client.0.smithi100.stdout:[----------] 7 tests from EncryptedFlattenTest/3, where TypeParam = LUKS1UnderLUKS1 2024-04-07T17:24:40.493 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] EncryptedFlattenTest/3.Simple 2024-04-07T17:24:41.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:24:41 smithi100 bash[74942]: cluster 2024-04-07T17:24:40.148956+0000 mgr.y (mgr.25175) 2079 : cluster [DBG] pgmap v3551: 137 pgs: 137 active+clean; 376 MiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 537 KiB/s rd, 3.3 MiB/s wr, 54 op/s 2024-04-07T17:24:41.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:24:41 smithi100 bash[74942]: cluster 2024-04-07T17:24:40.430197+0000 mon.a (mon.0) 3625 : cluster [DBG] osdmap e1898: 8 total, 8 up, 8 in 2024-04-07T17:24:41.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:24:41 smithi100 bash[76202]: cluster 2024-04-07T17:24:40.148956+0000 mgr.y (mgr.25175) 2079 : cluster [DBG] pgmap v3551: 137 pgs: 137 active+clean; 376 MiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 537 KiB/s rd, 3.3 MiB/s wr, 54 op/s 2024-04-07T17:24:41.705 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:24:41 smithi100 bash[76202]: cluster 2024-04-07T17:24:40.430197+0000 mon.a (mon.0) 3625 : cluster [DBG] osdmap e1898: 8 total, 8 up, 8 in 2024-04-07T17:24:41.866 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:24:41 smithi144 bash[66654]: cluster 2024-04-07T17:24:40.148956+0000 mgr.y (mgr.25175) 2079 : cluster [DBG] pgmap v3551: 137 pgs: 137 active+clean; 376 MiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 537 KiB/s rd, 3.3 MiB/s wr, 54 op/s 2024-04-07T17:24:41.867 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:24:41 smithi144 bash[66654]: cluster 2024-04-07T17:24:40.430197+0000 mon.a (mon.0) 3625 : cluster [DBG] osdmap e1898: 8 total, 8 up, 8 in 2024-04-07T17:24:42.471 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:24:42.616 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:24:42 smithi144 bash[95233]: ts=2024-04-07T17:24:42.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:24:42.617 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:24:42 smithi144 bash[66654]: cluster 2024-04-07T17:24:41.448289+0000 mon.a (mon.0) 3626 : cluster [DBG] osdmap e1899: 8 total, 8 up, 8 in 2024-04-07T17:24:42.617 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:24:42 smithi144 bash[66654]: audit 2024-04-07T17:24:41.453158+0000 mon.c (mon.2) 95 : audit [INF] from='client.? 172.21.15.100:0/3599958774' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:24:42.617 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:24:42 smithi144 bash[66654]: audit 2024-04-07T17:24:41.454112+0000 mon.a (mon.0) 3627 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:24:42.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:24:42 smithi100 bash[76202]: cluster 2024-04-07T17:24:41.448289+0000 mon.a (mon.0) 3626 : cluster [DBG] osdmap e1899: 8 total, 8 up, 8 in 2024-04-07T17:24:42.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:24:42 smithi100 bash[76202]: audit 2024-04-07T17:24:41.453158+0000 mon.c (mon.2) 95 : audit [INF] from='client.? 172.21.15.100:0/3599958774' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:24:42.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:24:42 smithi100 bash[76202]: audit 2024-04-07T17:24:41.454112+0000 mon.a (mon.0) 3627 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:24:42.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:24:42 smithi100 bash[74942]: cluster 2024-04-07T17:24:41.448289+0000 mon.a (mon.0) 3626 : cluster [DBG] osdmap e1899: 8 total, 8 up, 8 in 2024-04-07T17:24:42.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:24:42 smithi100 bash[74942]: audit 2024-04-07T17:24:41.453158+0000 mon.c (mon.2) 95 : audit [INF] from='client.? 172.21.15.100:0/3599958774' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:24:42.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:24:42 smithi100 bash[74942]: audit 2024-04-07T17:24:41.454112+0000 mon.a (mon.0) 3627 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:24:43.867 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:24:43 smithi144 bash[66654]: cluster 2024-04-07T17:24:42.149948+0000 mgr.y (mgr.25175) 2080 : cluster [DBG] pgmap v3554: 137 pgs: 6 creating+peering, 26 unknown, 105 active+clean; 4.6 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail 2024-04-07T17:24:43.867 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:24:43 smithi144 bash[66654]: cluster 2024-04-07T17:24:42.440138+0000 mon.a (mon.0) 3628 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T17:24:43.867 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:24:43 smithi144 bash[66654]: audit 2024-04-07T17:24:42.449706+0000 mon.a (mon.0) 3629 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T17:24:43.867 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:24:43 smithi144 bash[66654]: cluster 2024-04-07T17:24:42.461064+0000 mon.a (mon.0) 3630 : cluster [DBG] osdmap e1900: 8 total, 8 up, 8 in 2024-04-07T17:24:43.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:24:43 smithi100 bash[74942]: cluster 2024-04-07T17:24:42.149948+0000 mgr.y (mgr.25175) 2080 : cluster [DBG] pgmap v3554: 137 pgs: 6 creating+peering, 26 unknown, 105 active+clean; 4.6 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail 2024-04-07T17:24:43.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:24:43 smithi100 bash[74942]: cluster 2024-04-07T17:24:42.440138+0000 mon.a (mon.0) 3628 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T17:24:43.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:24:43 smithi100 bash[74942]: audit 2024-04-07T17:24:42.449706+0000 mon.a (mon.0) 3629 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T17:24:43.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:24:43 smithi100 bash[74942]: cluster 2024-04-07T17:24:42.461064+0000 mon.a (mon.0) 3630 : cluster [DBG] osdmap e1900: 8 total, 8 up, 8 in 2024-04-07T17:24:43.955 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:24:43 smithi100 bash[76202]: cluster 2024-04-07T17:24:42.149948+0000 mgr.y (mgr.25175) 2080 : cluster [DBG] pgmap v3554: 137 pgs: 6 creating+peering, 26 unknown, 105 active+clean; 4.6 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail 2024-04-07T17:24:43.955 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:24:43 smithi100 bash[76202]: cluster 2024-04-07T17:24:42.440138+0000 mon.a (mon.0) 3628 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T17:24:43.955 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:24:43 smithi100 bash[76202]: audit 2024-04-07T17:24:42.449706+0000 mon.a (mon.0) 3629 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T17:24:43.955 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:24:43 smithi100 bash[76202]: cluster 2024-04-07T17:24:42.461064+0000 mon.a (mon.0) 3630 : cluster [DBG] osdmap e1900: 8 total, 8 up, 8 in 2024-04-07T17:24:44.866 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:24:44 smithi144 bash[66654]: cluster 2024-04-07T17:24:43.459612+0000 mon.a (mon.0) 3631 : cluster [DBG] osdmap e1901: 8 total, 8 up, 8 in 2024-04-07T17:24:44.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:24:44 smithi100 bash[76202]: cluster 2024-04-07T17:24:43.459612+0000 mon.a (mon.0) 3631 : cluster [DBG] osdmap e1901: 8 total, 8 up, 8 in 2024-04-07T17:24:44.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:24:44 smithi100 bash[74942]: cluster 2024-04-07T17:24:43.459612+0000 mon.a (mon.0) 3631 : cluster [DBG] osdmap e1901: 8 total, 8 up, 8 in 2024-04-07T17:24:45.866 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:24:45 smithi144 bash[66654]: cluster 2024-04-07T17:24:44.151038+0000 mgr.y (mgr.25175) 2081 : cluster [DBG] pgmap v3557: 137 pgs: 15 creating+peering, 4 unknown, 118 active+clean; 4.6 MiB data, 3.7 GiB used, 712 GiB / 715 GiB avail 2024-04-07T17:24:45.866 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:24:45 smithi144 bash[66654]: cluster 2024-04-07T17:24:44.463969+0000 mon.a (mon.0) 3632 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T17:24:45.867 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:24:45 smithi144 bash[66654]: cluster 2024-04-07T17:24:44.481940+0000 mon.a (mon.0) 3633 : cluster [DBG] osdmap e1902: 8 total, 8 up, 8 in 2024-04-07T17:24:45.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:24:45 smithi100 bash[74942]: cluster 2024-04-07T17:24:44.151038+0000 mgr.y (mgr.25175) 2081 : cluster [DBG] pgmap v3557: 137 pgs: 15 creating+peering, 4 unknown, 118 active+clean; 4.6 MiB data, 3.7 GiB used, 712 GiB / 715 GiB avail 2024-04-07T17:24:45.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:24:45 smithi100 bash[74942]: cluster 2024-04-07T17:24:44.463969+0000 mon.a (mon.0) 3632 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T17:24:45.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:24:45 smithi100 bash[74942]: cluster 2024-04-07T17:24:44.481940+0000 mon.a (mon.0) 3633 : cluster [DBG] osdmap e1902: 8 total, 8 up, 8 in 2024-04-07T17:24:45.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:24:45 smithi100 bash[76202]: cluster 2024-04-07T17:24:44.151038+0000 mgr.y (mgr.25175) 2081 : cluster [DBG] pgmap v3557: 137 pgs: 15 creating+peering, 4 unknown, 118 active+clean; 4.6 MiB data, 3.7 GiB used, 712 GiB / 715 GiB avail 2024-04-07T17:24:45.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:24:45 smithi100 bash[76202]: cluster 2024-04-07T17:24:44.463969+0000 mon.a (mon.0) 3632 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T17:24:45.955 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:24:45 smithi100 bash[76202]: cluster 2024-04-07T17:24:44.481940+0000 mon.a (mon.0) 3633 : cluster [DBG] osdmap e1902: 8 total, 8 up, 8 in 2024-04-07T17:24:47.866 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:24:47 smithi144 bash[66654]: cluster 2024-04-07T17:24:46.151926+0000 mgr.y (mgr.25175) 2082 : cluster [DBG] pgmap v3559: 137 pgs: 15 creating+peering, 122 active+clean; 4.6 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 652 B/s rd, 434 B/s wr, 2 op/s 2024-04-07T17:24:47.866 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:24:47 smithi144 bash[66654]: audit 2024-04-07T17:24:46.838948+0000 mon.a (mon.0) 3634 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:24:47.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:24:47 smithi100 bash[76202]: cluster 2024-04-07T17:24:46.151926+0000 mgr.y (mgr.25175) 2082 : cluster [DBG] pgmap v3559: 137 pgs: 15 creating+peering, 122 active+clean; 4.6 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 652 B/s rd, 434 B/s wr, 2 op/s 2024-04-07T17:24:47.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:24:47 smithi100 bash[76202]: audit 2024-04-07T17:24:46.838948+0000 mon.a (mon.0) 3634 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:24:47.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:24:47 smithi100 bash[74942]: cluster 2024-04-07T17:24:46.151926+0000 mgr.y (mgr.25175) 2082 : cluster [DBG] pgmap v3559: 137 pgs: 15 creating+peering, 122 active+clean; 4.6 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 652 B/s rd, 434 B/s wr, 2 op/s 2024-04-07T17:24:47.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:24:47 smithi100 bash[74942]: audit 2024-04-07T17:24:46.838948+0000 mon.a (mon.0) 3634 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:24:48.704 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:24:48 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:24:48] "GET /metrics HTTP/1.1" 200 36725 "" "Prometheus/2.43.0" 2024-04-07T17:24:49.866 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:24:49 smithi144 bash[66654]: cluster 2024-04-07T17:24:48.153122+0000 mgr.y (mgr.25175) 2083 : cluster [DBG] pgmap v3560: 137 pgs: 137 active+clean; 4.6 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 3 op/s 2024-04-07T17:24:49.867 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:24:49 smithi144 bash[66654]: cluster 2024-04-07T17:24:48.493588+0000 mon.a (mon.0) 3635 : cluster [DBG] osdmap e1903: 8 total, 8 up, 8 in 2024-04-07T17:24:49.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:24:49 smithi100 bash[76202]: cluster 2024-04-07T17:24:48.153122+0000 mgr.y (mgr.25175) 2083 : cluster [DBG] pgmap v3560: 137 pgs: 137 active+clean; 4.6 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 3 op/s 2024-04-07T17:24:49.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:24:49 smithi100 bash[76202]: cluster 2024-04-07T17:24:48.493588+0000 mon.a (mon.0) 3635 : cluster [DBG] osdmap e1903: 8 total, 8 up, 8 in 2024-04-07T17:24:49.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:24:49 smithi100 bash[74942]: cluster 2024-04-07T17:24:48.153122+0000 mgr.y (mgr.25175) 2083 : cluster [DBG] pgmap v3560: 137 pgs: 137 active+clean; 4.6 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 3 op/s 2024-04-07T17:24:49.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:24:49 smithi100 bash[74942]: cluster 2024-04-07T17:24:48.493588+0000 mon.a (mon.0) 3635 : cluster [DBG] osdmap e1903: 8 total, 8 up, 8 in 2024-04-07T17:24:51.866 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:24:51 smithi144 bash[66654]: cluster 2024-04-07T17:24:50.154210+0000 mgr.y (mgr.25175) 2084 : cluster [DBG] pgmap v3562: 137 pgs: 137 active+clean; 17 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 38 KiB/s rd, 1.8 MiB/s wr, 50 op/s 2024-04-07T17:24:51.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:24:51 smithi100 bash[76202]: cluster 2024-04-07T17:24:50.154210+0000 mgr.y (mgr.25175) 2084 : cluster [DBG] pgmap v3562: 137 pgs: 137 active+clean; 17 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 38 KiB/s rd, 1.8 MiB/s wr, 50 op/s 2024-04-07T17:24:51.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:24:51 smithi100 bash[74942]: cluster 2024-04-07T17:24:50.154210+0000 mgr.y (mgr.25175) 2084 : cluster [DBG] pgmap v3562: 137 pgs: 137 active+clean; 17 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 38 KiB/s rd, 1.8 MiB/s wr, 50 op/s 2024-04-07T17:24:52.616 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:24:52 smithi144 bash[95233]: ts=2024-04-07T17:24:52.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:24:54.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:24:54 smithi144 bash[66654]: cluster 2024-04-07T17:24:52.155158+0000 mgr.y (mgr.25175) 2085 : cluster [DBG] pgmap v3563: 137 pgs: 137 active+clean; 26 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 32 KiB/s rd, 2.6 MiB/s wr, 43 op/s 2024-04-07T17:24:54.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:24:54 smithi144 bash[66654]: cluster 2024-04-07T17:24:53.020415+0000 mon.a (mon.0) 3636 : cluster [DBG] osdmap e1904: 8 total, 8 up, 8 in 2024-04-07T17:24:54.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:24:54 smithi100 bash[74942]: cluster 2024-04-07T17:24:52.155158+0000 mgr.y (mgr.25175) 2085 : cluster [DBG] pgmap v3563: 137 pgs: 137 active+clean; 26 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 32 KiB/s rd, 2.6 MiB/s wr, 43 op/s 2024-04-07T17:24:54.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:24:54 smithi100 bash[74942]: cluster 2024-04-07T17:24:53.020415+0000 mon.a (mon.0) 3636 : cluster [DBG] osdmap e1904: 8 total, 8 up, 8 in 2024-04-07T17:24:54.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:24:54 smithi100 bash[76202]: cluster 2024-04-07T17:24:52.155158+0000 mgr.y (mgr.25175) 2085 : cluster [DBG] pgmap v3563: 137 pgs: 137 active+clean; 26 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 32 KiB/s rd, 2.6 MiB/s wr, 43 op/s 2024-04-07T17:24:54.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:24:54 smithi100 bash[76202]: cluster 2024-04-07T17:24:53.020415+0000 mon.a (mon.0) 3636 : cluster [DBG] osdmap e1904: 8 total, 8 up, 8 in 2024-04-07T17:24:56.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:24:56 smithi144 bash[66654]: cluster 2024-04-07T17:24:54.156087+0000 mgr.y (mgr.25175) 2086 : cluster [DBG] pgmap v3565: 137 pgs: 137 active+clean; 34 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 41 KiB/s rd, 3.6 MiB/s wr, 55 op/s 2024-04-07T17:24:56.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:24:56 smithi100 bash[74942]: cluster 2024-04-07T17:24:54.156087+0000 mgr.y (mgr.25175) 2086 : cluster [DBG] pgmap v3565: 137 pgs: 137 active+clean; 34 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 41 KiB/s rd, 3.6 MiB/s wr, 55 op/s 2024-04-07T17:24:56.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:24:56 smithi100 bash[76202]: cluster 2024-04-07T17:24:54.156087+0000 mgr.y (mgr.25175) 2086 : cluster [DBG] pgmap v3565: 137 pgs: 137 active+clean; 34 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 41 KiB/s rd, 3.6 MiB/s wr, 55 op/s 2024-04-07T17:24:58.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:24:58 smithi144 bash[66654]: cluster 2024-04-07T17:24:56.156987+0000 mgr.y (mgr.25175) 2087 : cluster [DBG] pgmap v3566: 137 pgs: 137 active+clean; 34 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 40 KiB/s rd, 3.6 MiB/s wr, 54 op/s 2024-04-07T17:24:58.426 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:24:58 smithi100 bash[76202]: cluster 2024-04-07T17:24:56.156987+0000 mgr.y (mgr.25175) 2087 : cluster [DBG] pgmap v3566: 137 pgs: 137 active+clean; 34 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 40 KiB/s rd, 3.6 MiB/s wr, 54 op/s 2024-04-07T17:24:58.427 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:24:58 smithi100 bash[74942]: cluster 2024-04-07T17:24:56.156987+0000 mgr.y (mgr.25175) 2087 : cluster [DBG] pgmap v3566: 137 pgs: 137 active+clean; 34 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 40 KiB/s rd, 3.6 MiB/s wr, 54 op/s 2024-04-07T17:24:58.704 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:24:58 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:24:58] "GET /metrics HTTP/1.1" 200 36725 "" "Prometheus/2.43.0" 2024-04-07T17:24:59.231 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] EncryptedFlattenTest/3.Simple (18738 ms) 2024-04-07T17:24:59.231 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] EncryptedFlattenTest/3.Grow 2024-04-07T17:24:59.231 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:25:00.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:25:00 smithi144 bash[66654]: cluster 2024-04-07T17:24:58.158197+0000 mgr.y (mgr.25175) 2088 : cluster [DBG] pgmap v3567: 137 pgs: 137 active+clean; 50 MiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 5.3 MiB/s rd, 5.1 MiB/s wr, 49 op/s 2024-04-07T17:25:00.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:25:00 smithi100 bash[76202]: cluster 2024-04-07T17:24:58.158197+0000 mgr.y (mgr.25175) 2088 : cluster [DBG] pgmap v3567: 137 pgs: 137 active+clean; 50 MiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 5.3 MiB/s rd, 5.1 MiB/s wr, 49 op/s 2024-04-07T17:25:00.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:25:00 smithi100 bash[74942]: cluster 2024-04-07T17:24:58.158197+0000 mgr.y (mgr.25175) 2088 : cluster [DBG] pgmap v3567: 137 pgs: 137 active+clean; 50 MiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 5.3 MiB/s rd, 5.1 MiB/s wr, 49 op/s 2024-04-07T17:25:02.317 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:25:02 smithi144 bash[66654]: cluster 2024-04-07T17:25:00.159384+0000 mgr.y (mgr.25175) 2089 : cluster [DBG] pgmap v3568: 137 pgs: 137 active+clean; 55 MiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 7.2 MiB/s rd, 4.2 MiB/s wr, 29 op/s 2024-04-07T17:25:02.317 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:25:02 smithi144 bash[66654]: audit 2024-04-07T17:25:01.839455+0000 mon.a (mon.0) 3637 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:25:02.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:25:02 smithi100 bash[74942]: cluster 2024-04-07T17:25:00.159384+0000 mgr.y (mgr.25175) 2089 : cluster [DBG] pgmap v3568: 137 pgs: 137 active+clean; 55 MiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 7.2 MiB/s rd, 4.2 MiB/s wr, 29 op/s 2024-04-07T17:25:02.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:25:02 smithi100 bash[74942]: audit 2024-04-07T17:25:01.839455+0000 mon.a (mon.0) 3637 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:25:02.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:25:02 smithi100 bash[76202]: cluster 2024-04-07T17:25:00.159384+0000 mgr.y (mgr.25175) 2089 : cluster [DBG] pgmap v3568: 137 pgs: 137 active+clean; 55 MiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 7.2 MiB/s rd, 4.2 MiB/s wr, 29 op/s 2024-04-07T17:25:02.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:25:02 smithi100 bash[76202]: audit 2024-04-07T17:25:01.839455+0000 mon.a (mon.0) 3637 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:25:02.616 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:25:02 smithi144 bash[95233]: ts=2024-04-07T17:25:02.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:25:04.369 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:25:04 smithi144 bash[66654]: cluster 2024-04-07T17:25:02.160418+0000 mgr.y (mgr.25175) 2090 : cluster [DBG] pgmap v3569: 137 pgs: 137 active+clean; 63 MiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 8.8 MiB/s rd, 4.1 MiB/s wr, 33 op/s 2024-04-07T17:25:04.370 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:25:04 smithi144 bash[66654]: cluster 2024-04-07T17:25:03.048654+0000 mon.a (mon.0) 3638 : cluster [DBG] osdmap e1905: 8 total, 8 up, 8 in 2024-04-07T17:25:04.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:25:04 smithi100 bash[76202]: cluster 2024-04-07T17:25:02.160418+0000 mgr.y (mgr.25175) 2090 : cluster [DBG] pgmap v3569: 137 pgs: 137 active+clean; 63 MiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 8.8 MiB/s rd, 4.1 MiB/s wr, 33 op/s 2024-04-07T17:25:04.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:25:04 smithi100 bash[76202]: cluster 2024-04-07T17:25:03.048654+0000 mon.a (mon.0) 3638 : cluster [DBG] osdmap e1905: 8 total, 8 up, 8 in 2024-04-07T17:25:04.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:25:04 smithi100 bash[74942]: cluster 2024-04-07T17:25:02.160418+0000 mgr.y (mgr.25175) 2090 : cluster [DBG] pgmap v3569: 137 pgs: 137 active+clean; 63 MiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 8.8 MiB/s rd, 4.1 MiB/s wr, 33 op/s 2024-04-07T17:25:04.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:25:04 smithi100 bash[74942]: cluster 2024-04-07T17:25:03.048654+0000 mon.a (mon.0) 3638 : cluster [DBG] osdmap e1905: 8 total, 8 up, 8 in 2024-04-07T17:25:06.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:25:06 smithi144 bash[66654]: cluster 2024-04-07T17:25:04.161450+0000 mgr.y (mgr.25175) 2091 : cluster [DBG] pgmap v3571: 137 pgs: 137 active+clean; 67 MiB data, 3.7 GiB used, 711 GiB / 715 GiB avail; 10 MiB/s rd, 3.7 MiB/s wr, 30 op/s 2024-04-07T17:25:06.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:25:06 smithi100 bash[74942]: cluster 2024-04-07T17:25:04.161450+0000 mgr.y (mgr.25175) 2091 : cluster [DBG] pgmap v3571: 137 pgs: 137 active+clean; 67 MiB data, 3.7 GiB used, 711 GiB / 715 GiB avail; 10 MiB/s rd, 3.7 MiB/s wr, 30 op/s 2024-04-07T17:25:06.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:25:06 smithi100 bash[76202]: cluster 2024-04-07T17:25:04.161450+0000 mgr.y (mgr.25175) 2091 : cluster [DBG] pgmap v3571: 137 pgs: 137 active+clean; 67 MiB data, 3.7 GiB used, 711 GiB / 715 GiB avail; 10 MiB/s rd, 3.7 MiB/s wr, 30 op/s 2024-04-07T17:25:08.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:25:08 smithi144 bash[66654]: cluster 2024-04-07T17:25:06.162246+0000 mgr.y (mgr.25175) 2092 : cluster [DBG] pgmap v3572: 137 pgs: 137 active+clean; 67 MiB data, 3.7 GiB used, 711 GiB / 715 GiB avail; 10 MiB/s rd, 3.7 MiB/s wr, 37 op/s 2024-04-07T17:25:08.426 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:25:08 smithi100 bash[74942]: cluster 2024-04-07T17:25:06.162246+0000 mgr.y (mgr.25175) 2092 : cluster [DBG] pgmap v3572: 137 pgs: 137 active+clean; 67 MiB data, 3.7 GiB used, 711 GiB / 715 GiB avail; 10 MiB/s rd, 3.7 MiB/s wr, 37 op/s 2024-04-07T17:25:08.427 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:25:08 smithi100 bash[76202]: cluster 2024-04-07T17:25:06.162246+0000 mgr.y (mgr.25175) 2092 : cluster [DBG] pgmap v3572: 137 pgs: 137 active+clean; 67 MiB data, 3.7 GiB used, 711 GiB / 715 GiB avail; 10 MiB/s rd, 3.7 MiB/s wr, 37 op/s 2024-04-07T17:25:08.704 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:25:08 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:25:08] "GET /metrics HTTP/1.1" 200 36753 "" "Prometheus/2.43.0" 2024-04-07T17:25:10.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:25:10 smithi144 bash[66654]: cluster 2024-04-07T17:25:08.163485+0000 mgr.y (mgr.25175) 2093 : cluster [DBG] pgmap v3573: 137 pgs: 137 active+clean; 92 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 5.5 MiB/s rd, 4.2 MiB/s wr, 52 op/s 2024-04-07T17:25:10.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:25:10 smithi100 bash[74942]: cluster 2024-04-07T17:25:08.163485+0000 mgr.y (mgr.25175) 2093 : cluster [DBG] pgmap v3573: 137 pgs: 137 active+clean; 92 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 5.5 MiB/s rd, 4.2 MiB/s wr, 52 op/s 2024-04-07T17:25:10.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:25:10 smithi100 bash[76202]: cluster 2024-04-07T17:25:08.163485+0000 mgr.y (mgr.25175) 2093 : cluster [DBG] pgmap v3573: 137 pgs: 137 active+clean; 92 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 5.5 MiB/s rd, 4.2 MiB/s wr, 52 op/s 2024-04-07T17:25:12.317 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:25:12 smithi144 bash[66654]: cluster 2024-04-07T17:25:10.164520+0000 mgr.y (mgr.25175) 2094 : cluster [DBG] pgmap v3574: 137 pgs: 137 active+clean; 96 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 3.4 MiB/s rd, 4.1 MiB/s wr, 43 op/s 2024-04-07T17:25:12.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:25:12 smithi100 bash[74942]: cluster 2024-04-07T17:25:10.164520+0000 mgr.y (mgr.25175) 2094 : cluster [DBG] pgmap v3574: 137 pgs: 137 active+clean; 96 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 3.4 MiB/s rd, 4.1 MiB/s wr, 43 op/s 2024-04-07T17:25:12.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:25:12 smithi100 bash[76202]: cluster 2024-04-07T17:25:10.164520+0000 mgr.y (mgr.25175) 2094 : cluster [DBG] pgmap v3574: 137 pgs: 137 active+clean; 96 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 3.4 MiB/s rd, 4.1 MiB/s wr, 43 op/s 2024-04-07T17:25:12.616 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:25:12 smithi144 bash[95233]: ts=2024-04-07T17:25:12.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:25:13.780 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] EncryptedFlattenTest/3.Grow (14549 ms) 2024-04-07T17:25:13.781 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] EncryptedFlattenTest/3.Shrink 2024-04-07T17:25:13.781 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:25:14.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:25:14 smithi144 bash[66654]: cluster 2024-04-07T17:25:12.165205+0000 mgr.y (mgr.25175) 2095 : cluster [DBG] pgmap v3575: 137 pgs: 137 active+clean; 112 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 7.6 MiB/s rd, 4.9 MiB/s wr, 46 op/s 2024-04-07T17:25:14.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:25:14 smithi100 bash[76202]: cluster 2024-04-07T17:25:12.165205+0000 mgr.y (mgr.25175) 2095 : cluster [DBG] pgmap v3575: 137 pgs: 137 active+clean; 112 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 7.6 MiB/s rd, 4.9 MiB/s wr, 46 op/s 2024-04-07T17:25:14.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:25:14 smithi100 bash[74942]: cluster 2024-04-07T17:25:12.165205+0000 mgr.y (mgr.25175) 2095 : cluster [DBG] pgmap v3575: 137 pgs: 137 active+clean; 112 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 7.6 MiB/s rd, 4.9 MiB/s wr, 46 op/s 2024-04-07T17:25:16.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:25:16 smithi144 bash[66654]: cluster 2024-04-07T17:25:14.166414+0000 mgr.y (mgr.25175) 2096 : cluster [DBG] pgmap v3576: 137 pgs: 137 active+clean; 117 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 6.9 MiB/s rd, 4.9 MiB/s wr, 56 op/s 2024-04-07T17:25:16.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:25:16 smithi100 bash[76202]: cluster 2024-04-07T17:25:14.166414+0000 mgr.y (mgr.25175) 2096 : cluster [DBG] pgmap v3576: 137 pgs: 137 active+clean; 117 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 6.9 MiB/s rd, 4.9 MiB/s wr, 56 op/s 2024-04-07T17:25:16.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:25:16 smithi100 bash[74942]: cluster 2024-04-07T17:25:14.166414+0000 mgr.y (mgr.25175) 2096 : cluster [DBG] pgmap v3576: 137 pgs: 137 active+clean; 117 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 6.9 MiB/s rd, 4.9 MiB/s wr, 56 op/s 2024-04-07T17:25:17.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:25:17 smithi144 bash[66654]: audit 2024-04-07T17:25:16.839703+0000 mon.a (mon.0) 3639 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:25:17.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:25:17 smithi100 bash[76202]: audit 2024-04-07T17:25:16.839703+0000 mon.a (mon.0) 3639 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:25:17.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:25:17 smithi100 bash[74942]: audit 2024-04-07T17:25:16.839703+0000 mon.a (mon.0) 3639 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:25:18.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:25:18 smithi144 bash[66654]: cluster 2024-04-07T17:25:16.167295+0000 mgr.y (mgr.25175) 2097 : cluster [DBG] pgmap v3577: 137 pgs: 137 active+clean; 121 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 7.0 MiB/s rd, 4.8 MiB/s wr, 48 op/s 2024-04-07T17:25:18.427 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:25:18 smithi100 bash[74942]: cluster 2024-04-07T17:25:16.167295+0000 mgr.y (mgr.25175) 2097 : cluster [DBG] pgmap v3577: 137 pgs: 137 active+clean; 121 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 7.0 MiB/s rd, 4.8 MiB/s wr, 48 op/s 2024-04-07T17:25:18.427 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:25:18 smithi100 bash[76202]: cluster 2024-04-07T17:25:16.167295+0000 mgr.y (mgr.25175) 2097 : cluster [DBG] pgmap v3577: 137 pgs: 137 active+clean; 121 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 7.0 MiB/s rd, 4.8 MiB/s wr, 48 op/s 2024-04-07T17:25:18.704 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:25:18 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:25:18] "GET /metrics HTTP/1.1" 200 36763 "" "Prometheus/2.43.0" 2024-04-07T17:25:19.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:25:19 smithi144 bash[66654]: cluster 2024-04-07T17:25:18.083217+0000 mon.a (mon.0) 3640 : cluster [DBG] osdmap e1906: 8 total, 8 up, 8 in 2024-04-07T17:25:19.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:25:19 smithi144 bash[66654]: cluster 2024-04-07T17:25:18.167985+0000 mgr.y (mgr.25175) 2098 : cluster [DBG] pgmap v3579: 137 pgs: 137 active+clean; 125 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 10 MiB/s rd, 3.7 MiB/s wr, 33 op/s 2024-04-07T17:25:19.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:25:19 smithi100 bash[74942]: cluster 2024-04-07T17:25:18.083217+0000 mon.a (mon.0) 3640 : cluster [DBG] osdmap e1906: 8 total, 8 up, 8 in 2024-04-07T17:25:19.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:25:19 smithi100 bash[74942]: cluster 2024-04-07T17:25:18.167985+0000 mgr.y (mgr.25175) 2098 : cluster [DBG] pgmap v3579: 137 pgs: 137 active+clean; 125 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 10 MiB/s rd, 3.7 MiB/s wr, 33 op/s 2024-04-07T17:25:19.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:25:19 smithi100 bash[76202]: cluster 2024-04-07T17:25:18.083217+0000 mon.a (mon.0) 3640 : cluster [DBG] osdmap e1906: 8 total, 8 up, 8 in 2024-04-07T17:25:19.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:25:19 smithi100 bash[76202]: cluster 2024-04-07T17:25:18.167985+0000 mgr.y (mgr.25175) 2098 : cluster [DBG] pgmap v3579: 137 pgs: 137 active+clean; 125 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 10 MiB/s rd, 3.7 MiB/s wr, 33 op/s 2024-04-07T17:25:21.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:25:21 smithi144 bash[66654]: cluster 2024-04-07T17:25:20.169079+0000 mgr.y (mgr.25175) 2099 : cluster [DBG] pgmap v3580: 137 pgs: 137 active+clean; 133 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 10 MiB/s rd, 4.1 MiB/s wr, 50 op/s 2024-04-07T17:25:21.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:25:21 smithi100 bash[74942]: cluster 2024-04-07T17:25:20.169079+0000 mgr.y (mgr.25175) 2099 : cluster [DBG] pgmap v3580: 137 pgs: 137 active+clean; 133 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 10 MiB/s rd, 4.1 MiB/s wr, 50 op/s 2024-04-07T17:25:21.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:25:21 smithi100 bash[76202]: cluster 2024-04-07T17:25:20.169079+0000 mgr.y (mgr.25175) 2099 : cluster [DBG] pgmap v3580: 137 pgs: 137 active+clean; 133 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 10 MiB/s rd, 4.1 MiB/s wr, 50 op/s 2024-04-07T17:25:22.616 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:25:22 smithi144 bash[95233]: ts=2024-04-07T17:25:22.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:25:23.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:25:23 smithi144 bash[66654]: cluster 2024-04-07T17:25:22.170118+0000 mgr.y (mgr.25175) 2100 : cluster [DBG] pgmap v3581: 137 pgs: 137 active+clean; 146 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 4.7 MiB/s rd, 3.8 MiB/s wr, 52 op/s 2024-04-07T17:25:23.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:25:23 smithi100 bash[74942]: cluster 2024-04-07T17:25:22.170118+0000 mgr.y (mgr.25175) 2100 : cluster [DBG] pgmap v3581: 137 pgs: 137 active+clean; 146 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 4.7 MiB/s rd, 3.8 MiB/s wr, 52 op/s 2024-04-07T17:25:23.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:25:23 smithi100 bash[76202]: cluster 2024-04-07T17:25:22.170118+0000 mgr.y (mgr.25175) 2100 : cluster [DBG] pgmap v3581: 137 pgs: 137 active+clean; 146 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 4.7 MiB/s rd, 3.8 MiB/s wr, 52 op/s 2024-04-07T17:25:25.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:25:25 smithi144 bash[66654]: cluster 2024-04-07T17:25:24.171208+0000 mgr.y (mgr.25175) 2101 : cluster [DBG] pgmap v3582: 137 pgs: 137 active+clean; 154 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 3.4 MiB/s rd, 4.1 MiB/s wr, 44 op/s 2024-04-07T17:25:25.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:25:25 smithi100 bash[74942]: cluster 2024-04-07T17:25:24.171208+0000 mgr.y (mgr.25175) 2101 : cluster [DBG] pgmap v3582: 137 pgs: 137 active+clean; 154 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 3.4 MiB/s rd, 4.1 MiB/s wr, 44 op/s 2024-04-07T17:25:25.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:25:25 smithi100 bash[76202]: cluster 2024-04-07T17:25:24.171208+0000 mgr.y (mgr.25175) 2101 : cluster [DBG] pgmap v3582: 137 pgs: 137 active+clean; 154 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 3.4 MiB/s rd, 4.1 MiB/s wr, 44 op/s 2024-04-07T17:25:27.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:25:27 smithi144 bash[66654]: cluster 2024-04-07T17:25:26.172066+0000 mgr.y (mgr.25175) 2102 : cluster [DBG] pgmap v3583: 137 pgs: 137 active+clean; 154 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 2.2 MiB/s rd, 3.3 MiB/s wr, 43 op/s 2024-04-07T17:25:27.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:25:27 smithi100 bash[76202]: cluster 2024-04-07T17:25:26.172066+0000 mgr.y (mgr.25175) 2102 : cluster [DBG] pgmap v3583: 137 pgs: 137 active+clean; 154 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 2.2 MiB/s rd, 3.3 MiB/s wr, 43 op/s 2024-04-07T17:25:27.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:25:27 smithi100 bash[74942]: cluster 2024-04-07T17:25:26.172066+0000 mgr.y (mgr.25175) 2102 : cluster [DBG] pgmap v3583: 137 pgs: 137 active+clean; 154 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 2.2 MiB/s rd, 3.3 MiB/s wr, 43 op/s 2024-04-07T17:25:28.705 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:25:28 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:25:28] "GET /metrics HTTP/1.1" 200 36763 "" "Prometheus/2.43.0" 2024-04-07T17:25:28.790 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] EncryptedFlattenTest/3.Shrink (15009 ms) 2024-04-07T17:25:28.791 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] EncryptedFlattenTest/3.ShrinkToOne 2024-04-07T17:25:28.791 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:25:29.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:25:29 smithi144 bash[66654]: cluster 2024-04-07T17:25:28.173286+0000 mgr.y (mgr.25175) 2103 : cluster [DBG] pgmap v3584: 137 pgs: 137 active+clean; 167 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 5.2 MiB/s rd, 4.6 MiB/s wr, 44 op/s 2024-04-07T17:25:29.703 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:25:29 smithi100 bash[74942]: cluster 2024-04-07T17:25:28.173286+0000 mgr.y (mgr.25175) 2103 : cluster [DBG] pgmap v3584: 137 pgs: 137 active+clean; 167 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 5.2 MiB/s rd, 4.6 MiB/s wr, 44 op/s 2024-04-07T17:25:29.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:25:29 smithi100 bash[76202]: cluster 2024-04-07T17:25:28.173286+0000 mgr.y (mgr.25175) 2103 : cluster [DBG] pgmap v3584: 137 pgs: 137 active+clean; 167 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 5.2 MiB/s rd, 4.6 MiB/s wr, 44 op/s 2024-04-07T17:25:31.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:25:31 smithi144 bash[66654]: cluster 2024-04-07T17:25:30.174456+0000 mgr.y (mgr.25175) 2104 : cluster [DBG] pgmap v3585: 137 pgs: 137 active+clean; 179 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 7.7 MiB/s rd, 4.8 MiB/s wr, 51 op/s 2024-04-07T17:25:31.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:25:31 smithi100 bash[74942]: cluster 2024-04-07T17:25:30.174456+0000 mgr.y (mgr.25175) 2104 : cluster [DBG] pgmap v3585: 137 pgs: 137 active+clean; 179 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 7.7 MiB/s rd, 4.8 MiB/s wr, 51 op/s 2024-04-07T17:25:31.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:25:31 smithi100 bash[76202]: cluster 2024-04-07T17:25:30.174456+0000 mgr.y (mgr.25175) 2104 : cluster [DBG] pgmap v3585: 137 pgs: 137 active+clean; 179 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 7.7 MiB/s rd, 4.8 MiB/s wr, 51 op/s 2024-04-07T17:25:32.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:25:32 smithi144 bash[66654]: audit 2024-04-07T17:25:31.840291+0000 mon.a (mon.0) 3641 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:25:32.616 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:25:32 smithi144 bash[95233]: ts=2024-04-07T17:25:32.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:25:32.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:25:32 smithi100 bash[74942]: audit 2024-04-07T17:25:31.840291+0000 mon.a (mon.0) 3641 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:25:32.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:25:32 smithi100 bash[76202]: audit 2024-04-07T17:25:31.840291+0000 mon.a (mon.0) 3641 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:25:33.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:25:33 smithi144 bash[66654]: cluster 2024-04-07T17:25:32.175485+0000 mgr.y (mgr.25175) 2105 : cluster [DBG] pgmap v3586: 137 pgs: 137 active+clean; 179 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 8.8 MiB/s rd, 4.2 MiB/s wr, 36 op/s 2024-04-07T17:25:33.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:25:33 smithi100 bash[74942]: cluster 2024-04-07T17:25:32.175485+0000 mgr.y (mgr.25175) 2105 : cluster [DBG] pgmap v3586: 137 pgs: 137 active+clean; 179 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 8.8 MiB/s rd, 4.2 MiB/s wr, 36 op/s 2024-04-07T17:25:33.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:25:33 smithi100 bash[76202]: cluster 2024-04-07T17:25:32.175485+0000 mgr.y (mgr.25175) 2105 : cluster [DBG] pgmap v3586: 137 pgs: 137 active+clean; 179 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 8.8 MiB/s rd, 4.2 MiB/s wr, 36 op/s 2024-04-07T17:25:34.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:25:34 smithi144 bash[66654]: cluster 2024-04-07T17:25:33.268896+0000 mon.a (mon.0) 3642 : cluster [DBG] osdmap e1907: 8 total, 8 up, 8 in 2024-04-07T17:25:34.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:25:34 smithi100 bash[74942]: cluster 2024-04-07T17:25:33.268896+0000 mon.a (mon.0) 3642 : cluster [DBG] osdmap e1907: 8 total, 8 up, 8 in 2024-04-07T17:25:34.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:25:34 smithi100 bash[76202]: cluster 2024-04-07T17:25:33.268896+0000 mon.a (mon.0) 3642 : cluster [DBG] osdmap e1907: 8 total, 8 up, 8 in 2024-04-07T17:25:35.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:25:35 smithi144 bash[66654]: cluster 2024-04-07T17:25:34.176536+0000 mgr.y (mgr.25175) 2106 : cluster [DBG] pgmap v3588: 137 pgs: 137 active+clean; 191 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 10 MiB/s rd, 4.1 MiB/s wr, 33 op/s 2024-04-07T17:25:35.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:25:35 smithi144 bash[66654]: audit 2024-04-07T17:25:35.054294+0000 mon.a (mon.0) 3643 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:25:35.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:25:35 smithi100 bash[74942]: cluster 2024-04-07T17:25:34.176536+0000 mgr.y (mgr.25175) 2106 : cluster [DBG] pgmap v3588: 137 pgs: 137 active+clean; 191 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 10 MiB/s rd, 4.1 MiB/s wr, 33 op/s 2024-04-07T17:25:35.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:25:35 smithi100 bash[74942]: audit 2024-04-07T17:25:35.054294+0000 mon.a (mon.0) 3643 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:25:35.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:25:35 smithi100 bash[76202]: cluster 2024-04-07T17:25:34.176536+0000 mgr.y (mgr.25175) 2106 : cluster [DBG] pgmap v3588: 137 pgs: 137 active+clean; 191 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 10 MiB/s rd, 4.1 MiB/s wr, 33 op/s 2024-04-07T17:25:35.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:25:35 smithi100 bash[76202]: audit 2024-04-07T17:25:35.054294+0000 mon.a (mon.0) 3643 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:25:36.867 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:25:36 smithi144 bash[66654]: audit 2024-04-07T17:25:35.578396+0000 mon.a (mon.0) 3644 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:25:36.867 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:25:36 smithi144 bash[66654]: audit 2024-04-07T17:25:35.584525+0000 mon.a (mon.0) 3645 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:25:36.867 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:25:36 smithi144 bash[66654]: audit 2024-04-07T17:25:36.171492+0000 mon.a (mon.0) 3646 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:25:36.867 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:25:36 smithi144 bash[66654]: audit 2024-04-07T17:25:36.172890+0000 mon.a (mon.0) 3647 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:25:36.867 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:25:36 smithi144 bash[66654]: audit 2024-04-07T17:25:36.183389+0000 mon.a (mon.0) 3648 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:25:36.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:25:36 smithi100 bash[74942]: audit 2024-04-07T17:25:35.578396+0000 mon.a (mon.0) 3644 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:25:36.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:25:36 smithi100 bash[74942]: audit 2024-04-07T17:25:35.584525+0000 mon.a (mon.0) 3645 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:25:36.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:25:36 smithi100 bash[74942]: audit 2024-04-07T17:25:36.171492+0000 mon.a (mon.0) 3646 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:25:36.955 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:25:36 smithi100 bash[74942]: audit 2024-04-07T17:25:36.172890+0000 mon.a (mon.0) 3647 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:25:36.955 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:25:36 smithi100 bash[74942]: audit 2024-04-07T17:25:36.183389+0000 mon.a (mon.0) 3648 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:25:36.955 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:25:36 smithi100 bash[76202]: audit 2024-04-07T17:25:35.578396+0000 mon.a (mon.0) 3644 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:25:36.955 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:25:36 smithi100 bash[76202]: audit 2024-04-07T17:25:35.584525+0000 mon.a (mon.0) 3645 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:25:36.955 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:25:36 smithi100 bash[76202]: audit 2024-04-07T17:25:36.171492+0000 mon.a (mon.0) 3646 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:25:36.955 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:25:36 smithi100 bash[76202]: audit 2024-04-07T17:25:36.172890+0000 mon.a (mon.0) 3647 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:25:36.955 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:25:36 smithi100 bash[76202]: audit 2024-04-07T17:25:36.183389+0000 mon.a (mon.0) 3648 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:25:37.866 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:25:37 smithi144 bash[66654]: cluster 2024-04-07T17:25:36.177125+0000 mgr.y (mgr.25175) 2107 : cluster [DBG] pgmap v3589: 137 pgs: 137 active+clean; 199 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 10 MiB/s rd, 4.9 MiB/s wr, 33 op/s 2024-04-07T17:25:37.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:25:37 smithi100 bash[76202]: cluster 2024-04-07T17:25:36.177125+0000 mgr.y (mgr.25175) 2107 : cluster [DBG] pgmap v3589: 137 pgs: 137 active+clean; 199 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 10 MiB/s rd, 4.9 MiB/s wr, 33 op/s 2024-04-07T17:25:37.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:25:37 smithi100 bash[74942]: cluster 2024-04-07T17:25:36.177125+0000 mgr.y (mgr.25175) 2107 : cluster [DBG] pgmap v3589: 137 pgs: 137 active+clean; 199 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 10 MiB/s rd, 4.9 MiB/s wr, 33 op/s 2024-04-07T17:25:38.704 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:25:38 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:25:38] "GET /metrics HTTP/1.1" 200 36758 "" "Prometheus/2.43.0" 2024-04-07T17:25:40.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:25:40 smithi144 bash[66654]: cluster 2024-04-07T17:25:38.178297+0000 mgr.y (mgr.25175) 2108 : cluster [DBG] pgmap v3590: 137 pgs: 137 active+clean; 208 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 5.3 MiB/s rd, 4.1 MiB/s wr, 54 op/s 2024-04-07T17:25:40.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:25:40 smithi100 bash[74942]: cluster 2024-04-07T17:25:38.178297+0000 mgr.y (mgr.25175) 2108 : cluster [DBG] pgmap v3590: 137 pgs: 137 active+clean; 208 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 5.3 MiB/s rd, 4.1 MiB/s wr, 54 op/s 2024-04-07T17:25:40.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:25:40 smithi100 bash[76202]: cluster 2024-04-07T17:25:38.178297+0000 mgr.y (mgr.25175) 2108 : cluster [DBG] pgmap v3590: 137 pgs: 137 active+clean; 208 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 5.3 MiB/s rd, 4.1 MiB/s wr, 54 op/s 2024-04-07T17:25:42.317 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:25:42 smithi144 bash[66654]: cluster 2024-04-07T17:25:40.179021+0000 mgr.y (mgr.25175) 2109 : cluster [DBG] pgmap v3591: 137 pgs: 137 active+clean; 208 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 1.4 MiB/s rd, 2.9 MiB/s wr, 37 op/s 2024-04-07T17:25:42.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:25:42 smithi100 bash[74942]: cluster 2024-04-07T17:25:40.179021+0000 mgr.y (mgr.25175) 2109 : cluster [DBG] pgmap v3591: 137 pgs: 137 active+clean; 208 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 1.4 MiB/s rd, 2.9 MiB/s wr, 37 op/s 2024-04-07T17:25:42.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:25:42 smithi100 bash[76202]: cluster 2024-04-07T17:25:40.179021+0000 mgr.y (mgr.25175) 2109 : cluster [DBG] pgmap v3591: 137 pgs: 137 active+clean; 208 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 1.4 MiB/s rd, 2.9 MiB/s wr, 37 op/s 2024-04-07T17:25:42.616 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:25:42 smithi144 bash[95233]: ts=2024-04-07T17:25:42.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:25:43.935 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] EncryptedFlattenTest/3.ShrinkToOne (15145 ms) 2024-04-07T17:25:43.935 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] EncryptedFlattenTest/3.ShrinkToOneAfterSnapshot 2024-04-07T17:25:43.935 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:25:44.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:25:44 smithi144 bash[66654]: cluster 2024-04-07T17:25:42.180193+0000 mgr.y (mgr.25175) 2110 : cluster [DBG] pgmap v3592: 137 pgs: 137 active+clean; 212 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 534 KiB/s rd, 3.7 MiB/s wr, 51 op/s 2024-04-07T17:25:44.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:25:44 smithi100 bash[76202]: cluster 2024-04-07T17:25:42.180193+0000 mgr.y (mgr.25175) 2110 : cluster [DBG] pgmap v3592: 137 pgs: 137 active+clean; 212 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 534 KiB/s rd, 3.7 MiB/s wr, 51 op/s 2024-04-07T17:25:44.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:25:44 smithi100 bash[74942]: cluster 2024-04-07T17:25:42.180193+0000 mgr.y (mgr.25175) 2110 : cluster [DBG] pgmap v3592: 137 pgs: 137 active+clean; 212 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 534 KiB/s rd, 3.7 MiB/s wr, 51 op/s 2024-04-07T17:25:46.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:25:46 smithi144 bash[66654]: cluster 2024-04-07T17:25:44.180846+0000 mgr.y (mgr.25175) 2111 : cluster [DBG] pgmap v3593: 137 pgs: 137 active+clean; 216 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 865 KiB/s rd, 4.1 MiB/s wr, 47 op/s 2024-04-07T17:25:46.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:25:46 smithi100 bash[74942]: cluster 2024-04-07T17:25:44.180846+0000 mgr.y (mgr.25175) 2111 : cluster [DBG] pgmap v3593: 137 pgs: 137 active+clean; 216 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 865 KiB/s rd, 4.1 MiB/s wr, 47 op/s 2024-04-07T17:25:46.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:25:46 smithi100 bash[76202]: cluster 2024-04-07T17:25:44.180846+0000 mgr.y (mgr.25175) 2111 : cluster [DBG] pgmap v3593: 137 pgs: 137 active+clean; 216 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 865 KiB/s rd, 4.1 MiB/s wr, 47 op/s 2024-04-07T17:25:47.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:25:47 smithi144 bash[66654]: audit 2024-04-07T17:25:46.840895+0000 mon.a (mon.0) 3649 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:25:47.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:25:47 smithi100 bash[76202]: audit 2024-04-07T17:25:46.840895+0000 mon.a (mon.0) 3649 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:25:47.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:25:47 smithi100 bash[74942]: audit 2024-04-07T17:25:46.840895+0000 mon.a (mon.0) 3649 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:25:48.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:25:48 smithi144 bash[66654]: cluster 2024-04-07T17:25:46.181640+0000 mgr.y (mgr.25175) 2112 : cluster [DBG] pgmap v3594: 137 pgs: 137 active+clean; 216 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 783 KiB/s rd, 2.8 MiB/s wr, 37 op/s 2024-04-07T17:25:48.427 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:25:48 smithi100 bash[74942]: cluster 2024-04-07T17:25:46.181640+0000 mgr.y (mgr.25175) 2112 : cluster [DBG] pgmap v3594: 137 pgs: 137 active+clean; 216 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 783 KiB/s rd, 2.8 MiB/s wr, 37 op/s 2024-04-07T17:25:48.427 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:25:48 smithi100 bash[76202]: cluster 2024-04-07T17:25:46.181640+0000 mgr.y (mgr.25175) 2112 : cluster [DBG] pgmap v3594: 137 pgs: 137 active+clean; 216 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 783 KiB/s rd, 2.8 MiB/s wr, 37 op/s 2024-04-07T17:25:48.703 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:25:48 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:25:48] "GET /metrics HTTP/1.1" 200 36757 "" "Prometheus/2.43.0" 2024-04-07T17:25:49.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:25:49 smithi144 bash[66654]: cluster 2024-04-07T17:25:48.068801+0000 mon.a (mon.0) 3650 : cluster [DBG] osdmap e1908: 8 total, 8 up, 8 in 2024-04-07T17:25:49.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:25:49 smithi100 bash[74942]: cluster 2024-04-07T17:25:48.068801+0000 mon.a (mon.0) 3650 : cluster [DBG] osdmap e1908: 8 total, 8 up, 8 in 2024-04-07T17:25:49.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:25:49 smithi100 bash[76202]: cluster 2024-04-07T17:25:48.068801+0000 mon.a (mon.0) 3650 : cluster [DBG] osdmap e1908: 8 total, 8 up, 8 in 2024-04-07T17:25:50.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:25:50 smithi144 bash[66654]: cluster 2024-04-07T17:25:48.182395+0000 mgr.y (mgr.25175) 2113 : cluster [DBG] pgmap v3596: 137 pgs: 137 active+clean; 216 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 922 KiB/s rd, 1.6 MiB/s wr, 22 op/s 2024-04-07T17:25:50.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:25:50 smithi100 bash[74942]: cluster 2024-04-07T17:25:48.182395+0000 mgr.y (mgr.25175) 2113 : cluster [DBG] pgmap v3596: 137 pgs: 137 active+clean; 216 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 922 KiB/s rd, 1.6 MiB/s wr, 22 op/s 2024-04-07T17:25:50.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:25:50 smithi100 bash[76202]: cluster 2024-04-07T17:25:48.182395+0000 mgr.y (mgr.25175) 2113 : cluster [DBG] pgmap v3596: 137 pgs: 137 active+clean; 216 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 922 KiB/s rd, 1.6 MiB/s wr, 22 op/s 2024-04-07T17:25:51.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:25:51 smithi100 bash[74942]: cluster 2024-04-07T17:25:50.183548+0000 mgr.y (mgr.25175) 2114 : cluster [DBG] pgmap v3597: 137 pgs: 137 active+clean; 233 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 898 KiB/s rd, 3.3 MiB/s wr, 32 op/s 2024-04-07T17:25:51.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:25:51 smithi100 bash[76202]: cluster 2024-04-07T17:25:50.183548+0000 mgr.y (mgr.25175) 2114 : cluster [DBG] pgmap v3597: 137 pgs: 137 active+clean; 233 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 898 KiB/s rd, 3.3 MiB/s wr, 32 op/s 2024-04-07T17:25:51.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:25:51 smithi144 bash[66654]: cluster 2024-04-07T17:25:50.183548+0000 mgr.y (mgr.25175) 2114 : cluster [DBG] pgmap v3597: 137 pgs: 137 active+clean; 233 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 898 KiB/s rd, 3.3 MiB/s wr, 32 op/s 2024-04-07T17:25:52.616 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:25:52 smithi144 bash[95233]: ts=2024-04-07T17:25:52.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:25:53.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:25:53 smithi144 bash[66654]: cluster 2024-04-07T17:25:52.184711+0000 mgr.y (mgr.25175) 2115 : cluster [DBG] pgmap v3598: 137 pgs: 137 active+clean; 245 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 437 KiB/s rd, 3.7 MiB/s wr, 39 op/s 2024-04-07T17:25:53.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:25:53 smithi100 bash[74942]: cluster 2024-04-07T17:25:52.184711+0000 mgr.y (mgr.25175) 2115 : cluster [DBG] pgmap v3598: 137 pgs: 137 active+clean; 245 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 437 KiB/s rd, 3.7 MiB/s wr, 39 op/s 2024-04-07T17:25:53.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:25:53 smithi100 bash[76202]: cluster 2024-04-07T17:25:52.184711+0000 mgr.y (mgr.25175) 2115 : cluster [DBG] pgmap v3598: 137 pgs: 137 active+clean; 245 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 437 KiB/s rd, 3.7 MiB/s wr, 39 op/s 2024-04-07T17:25:55.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:25:55 smithi144 bash[66654]: cluster 2024-04-07T17:25:54.185852+0000 mgr.y (mgr.25175) 2116 : cluster [DBG] pgmap v3599: 137 pgs: 137 active+clean; 249 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 57 KiB/s rd, 3.3 MiB/s wr, 40 op/s 2024-04-07T17:25:55.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:25:55 smithi100 bash[74942]: cluster 2024-04-07T17:25:54.185852+0000 mgr.y (mgr.25175) 2116 : cluster [DBG] pgmap v3599: 137 pgs: 137 active+clean; 249 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 57 KiB/s rd, 3.3 MiB/s wr, 40 op/s 2024-04-07T17:25:55.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:25:55 smithi100 bash[76202]: cluster 2024-04-07T17:25:54.185852+0000 mgr.y (mgr.25175) 2116 : cluster [DBG] pgmap v3599: 137 pgs: 137 active+clean; 249 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 57 KiB/s rd, 3.3 MiB/s wr, 40 op/s 2024-04-07T17:25:57.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:25:57 smithi144 bash[66654]: cluster 2024-04-07T17:25:56.186821+0000 mgr.y (mgr.25175) 2117 : cluster [DBG] pgmap v3600: 137 pgs: 137 active+clean; 249 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 57 KiB/s rd, 3.3 MiB/s wr, 40 op/s 2024-04-07T17:25:57.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:25:57 smithi100 bash[74942]: cluster 2024-04-07T17:25:56.186821+0000 mgr.y (mgr.25175) 2117 : cluster [DBG] pgmap v3600: 137 pgs: 137 active+clean; 249 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 57 KiB/s rd, 3.3 MiB/s wr, 40 op/s 2024-04-07T17:25:57.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:25:57 smithi100 bash[76202]: cluster 2024-04-07T17:25:56.186821+0000 mgr.y (mgr.25175) 2117 : cluster [DBG] pgmap v3600: 137 pgs: 137 active+clean; 249 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 57 KiB/s rd, 3.3 MiB/s wr, 40 op/s 2024-04-07T17:25:58.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:25:58 smithi144 bash[66654]: cluster 2024-04-07T17:25:57.262565+0000 mon.a (mon.0) 3651 : cluster [DBG] osdmap e1909: 8 total, 8 up, 8 in 2024-04-07T17:25:58.704 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:25:58 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:25:58] "GET /metrics HTTP/1.1" 200 36757 "" "Prometheus/2.43.0" 2024-04-07T17:25:58.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:25:58 smithi100 bash[74942]: cluster 2024-04-07T17:25:57.262565+0000 mon.a (mon.0) 3651 : cluster [DBG] osdmap e1909: 8 total, 8 up, 8 in 2024-04-07T17:25:58.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:25:58 smithi100 bash[76202]: cluster 2024-04-07T17:25:57.262565+0000 mon.a (mon.0) 3651 : cluster [DBG] osdmap e1909: 8 total, 8 up, 8 in 2024-04-07T17:25:59.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:25:59 smithi144 bash[66654]: cluster 2024-04-07T17:25:58.187648+0000 mgr.y (mgr.25175) 2118 : cluster [DBG] pgmap v3602: 137 pgs: 137 active+clean; 249 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 60 KiB/s rd, 3.3 MiB/s wr, 43 op/s 2024-04-07T17:25:59.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:25:59 smithi100 bash[74942]: cluster 2024-04-07T17:25:58.187648+0000 mgr.y (mgr.25175) 2118 : cluster [DBG] pgmap v3602: 137 pgs: 137 active+clean; 249 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 60 KiB/s rd, 3.3 MiB/s wr, 43 op/s 2024-04-07T17:25:59.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:25:59 smithi100 bash[76202]: cluster 2024-04-07T17:25:58.187648+0000 mgr.y (mgr.25175) 2118 : cluster [DBG] pgmap v3602: 137 pgs: 137 active+clean; 249 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 60 KiB/s rd, 3.3 MiB/s wr, 43 op/s 2024-04-07T17:25:59.834 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] EncryptedFlattenTest/3.ShrinkToOneAfterSnapshot (15899 ms) 2024-04-07T17:25:59.834 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] EncryptedFlattenTest/3.MinOverlap 2024-04-07T17:25:59.834 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:26:01.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:26:01 smithi144 bash[66654]: cluster 2024-04-07T17:26:00.188694+0000 mgr.y (mgr.25175) 2119 : cluster [DBG] pgmap v3603: 137 pgs: 137 active+clean; 274 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 1.8 MiB/s rd, 4.1 MiB/s wr, 46 op/s 2024-04-07T17:26:01.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:26:01 smithi100 bash[74942]: cluster 2024-04-07T17:26:00.188694+0000 mgr.y (mgr.25175) 2119 : cluster [DBG] pgmap v3603: 137 pgs: 137 active+clean; 274 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 1.8 MiB/s rd, 4.1 MiB/s wr, 46 op/s 2024-04-07T17:26:01.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:26:01 smithi100 bash[76202]: cluster 2024-04-07T17:26:00.188694+0000 mgr.y (mgr.25175) 2119 : cluster [DBG] pgmap v3603: 137 pgs: 137 active+clean; 274 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 1.8 MiB/s rd, 4.1 MiB/s wr, 46 op/s 2024-04-07T17:26:02.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:26:02 smithi144 bash[66654]: audit 2024-04-07T17:26:01.841013+0000 mon.a (mon.0) 3652 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:26:02.616 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:26:02 smithi144 bash[95233]: ts=2024-04-07T17:26:02.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:26:02.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:26:02 smithi100 bash[74942]: audit 2024-04-07T17:26:01.841013+0000 mon.a (mon.0) 3652 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:26:02.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:26:02 smithi100 bash[76202]: audit 2024-04-07T17:26:01.841013+0000 mon.a (mon.0) 3652 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:26:03.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:26:03 smithi144 bash[66654]: cluster 2024-04-07T17:26:02.189831+0000 mgr.y (mgr.25175) 2120 : cluster [DBG] pgmap v3604: 137 pgs: 137 active+clean; 306 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 3.0 MiB/s rd, 6.1 MiB/s wr, 42 op/s 2024-04-07T17:26:03.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:26:03 smithi100 bash[74942]: cluster 2024-04-07T17:26:02.189831+0000 mgr.y (mgr.25175) 2120 : cluster [DBG] pgmap v3604: 137 pgs: 137 active+clean; 306 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 3.0 MiB/s rd, 6.1 MiB/s wr, 42 op/s 2024-04-07T17:26:03.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:26:03 smithi100 bash[76202]: cluster 2024-04-07T17:26:02.189831+0000 mgr.y (mgr.25175) 2120 : cluster [DBG] pgmap v3604: 137 pgs: 137 active+clean; 306 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 3.0 MiB/s rd, 6.1 MiB/s wr, 42 op/s 2024-04-07T17:26:05.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:26:05 smithi144 bash[66654]: cluster 2024-04-07T17:26:04.190937+0000 mgr.y (mgr.25175) 2121 : cluster [DBG] pgmap v3605: 137 pgs: 137 active+clean; 306 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 3.0 MiB/s rd, 5.7 MiB/s wr, 48 op/s 2024-04-07T17:26:05.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:26:05 smithi144 bash[66654]: cluster 2024-04-07T17:26:04.292118+0000 mon.a (mon.0) 3653 : cluster [DBG] osdmap e1910: 8 total, 8 up, 8 in 2024-04-07T17:26:05.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:26:05 smithi100 bash[74942]: cluster 2024-04-07T17:26:04.190937+0000 mgr.y (mgr.25175) 2121 : cluster [DBG] pgmap v3605: 137 pgs: 137 active+clean; 306 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 3.0 MiB/s rd, 5.7 MiB/s wr, 48 op/s 2024-04-07T17:26:05.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:26:05 smithi100 bash[74942]: cluster 2024-04-07T17:26:04.292118+0000 mon.a (mon.0) 3653 : cluster [DBG] osdmap e1910: 8 total, 8 up, 8 in 2024-04-07T17:26:05.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:26:05 smithi100 bash[76202]: cluster 2024-04-07T17:26:04.190937+0000 mgr.y (mgr.25175) 2121 : cluster [DBG] pgmap v3605: 137 pgs: 137 active+clean; 306 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 3.0 MiB/s rd, 5.7 MiB/s wr, 48 op/s 2024-04-07T17:26:05.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:26:05 smithi100 bash[76202]: cluster 2024-04-07T17:26:04.292118+0000 mon.a (mon.0) 3653 : cluster [DBG] osdmap e1910: 8 total, 8 up, 8 in 2024-04-07T17:26:07.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:26:07 smithi144 bash[66654]: cluster 2024-04-07T17:26:06.191791+0000 mgr.y (mgr.25175) 2122 : cluster [DBG] pgmap v3607: 137 pgs: 137 active+clean; 306 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 3.4 MiB/s rd, 6.4 MiB/s wr, 64 op/s 2024-04-07T17:26:07.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:26:07 smithi100 bash[74942]: cluster 2024-04-07T17:26:06.191791+0000 mgr.y (mgr.25175) 2122 : cluster [DBG] pgmap v3607: 137 pgs: 137 active+clean; 306 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 3.4 MiB/s rd, 6.4 MiB/s wr, 64 op/s 2024-04-07T17:26:07.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:26:07 smithi100 bash[76202]: cluster 2024-04-07T17:26:06.191791+0000 mgr.y (mgr.25175) 2122 : cluster [DBG] pgmap v3607: 137 pgs: 137 active+clean; 306 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 3.4 MiB/s rd, 6.4 MiB/s wr, 64 op/s 2024-04-07T17:26:08.704 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:26:08 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:26:08] "GET /metrics HTTP/1.1" 200 36760 "" "Prometheus/2.43.0" 2024-04-07T17:26:09.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:26:09 smithi144 bash[66654]: cluster 2024-04-07T17:26:08.193037+0000 mgr.y (mgr.25175) 2123 : cluster [DBG] pgmap v3608: 137 pgs: 137 active+clean; 335 MiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 3.0 MiB/s rd, 8.6 MiB/s wr, 61 op/s 2024-04-07T17:26:09.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:26:09 smithi100 bash[74942]: cluster 2024-04-07T17:26:08.193037+0000 mgr.y (mgr.25175) 2123 : cluster [DBG] pgmap v3608: 137 pgs: 137 active+clean; 335 MiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 3.0 MiB/s rd, 8.6 MiB/s wr, 61 op/s 2024-04-07T17:26:09.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:26:09 smithi100 bash[76202]: cluster 2024-04-07T17:26:08.193037+0000 mgr.y (mgr.25175) 2123 : cluster [DBG] pgmap v3608: 137 pgs: 137 active+clean; 335 MiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 3.0 MiB/s rd, 8.6 MiB/s wr, 61 op/s 2024-04-07T17:26:11.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:26:11 smithi144 bash[66654]: cluster 2024-04-07T17:26:10.194106+0000 mgr.y (mgr.25175) 2124 : cluster [DBG] pgmap v3609: 137 pgs: 137 active+clean; 335 MiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 1.2 MiB/s rd, 6.1 MiB/s wr, 61 op/s 2024-04-07T17:26:11.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:26:11 smithi100 bash[74942]: cluster 2024-04-07T17:26:10.194106+0000 mgr.y (mgr.25175) 2124 : cluster [DBG] pgmap v3609: 137 pgs: 137 active+clean; 335 MiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 1.2 MiB/s rd, 6.1 MiB/s wr, 61 op/s 2024-04-07T17:26:11.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:26:11 smithi100 bash[76202]: cluster 2024-04-07T17:26:10.194106+0000 mgr.y (mgr.25175) 2124 : cluster [DBG] pgmap v3609: 137 pgs: 137 active+clean; 335 MiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 1.2 MiB/s rd, 6.1 MiB/s wr, 61 op/s 2024-04-07T17:26:12.616 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:26:12 smithi144 bash[95233]: ts=2024-04-07T17:26:12.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:26:13.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:26:13 smithi144 bash[66654]: cluster 2024-04-07T17:26:12.195304+0000 mgr.y (mgr.25175) 2125 : cluster [DBG] pgmap v3610: 137 pgs: 137 active+clean; 339 MiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 89 KiB/s rd, 3.3 MiB/s wr, 42 op/s 2024-04-07T17:26:13.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:26:13 smithi100 bash[74942]: cluster 2024-04-07T17:26:12.195304+0000 mgr.y (mgr.25175) 2125 : cluster [DBG] pgmap v3610: 137 pgs: 137 active+clean; 339 MiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 89 KiB/s rd, 3.3 MiB/s wr, 42 op/s 2024-04-07T17:26:13.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:26:13 smithi100 bash[76202]: cluster 2024-04-07T17:26:12.195304+0000 mgr.y (mgr.25175) 2125 : cluster [DBG] pgmap v3610: 137 pgs: 137 active+clean; 339 MiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 89 KiB/s rd, 3.3 MiB/s wr, 42 op/s 2024-04-07T17:26:14.508 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] EncryptedFlattenTest/3.MinOverlap (14674 ms) 2024-04-07T17:26:14.508 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] EncryptedFlattenTest/3.ZeroOverlap 2024-04-07T17:26:14.508 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:26:15.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:26:15 smithi144 bash[66654]: cluster 2024-04-07T17:26:14.195918+0000 mgr.y (mgr.25175) 2126 : cluster [DBG] pgmap v3611: 137 pgs: 137 active+clean; 339 MiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 92 KiB/s rd, 3.3 MiB/s wr, 44 op/s 2024-04-07T17:26:15.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:26:15 smithi100 bash[74942]: cluster 2024-04-07T17:26:14.195918+0000 mgr.y (mgr.25175) 2126 : cluster [DBG] pgmap v3611: 137 pgs: 137 active+clean; 339 MiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 92 KiB/s rd, 3.3 MiB/s wr, 44 op/s 2024-04-07T17:26:15.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:26:15 smithi100 bash[76202]: cluster 2024-04-07T17:26:14.195918+0000 mgr.y (mgr.25175) 2126 : cluster [DBG] pgmap v3611: 137 pgs: 137 active+clean; 339 MiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 92 KiB/s rd, 3.3 MiB/s wr, 44 op/s 2024-04-07T17:26:17.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:26:17 smithi144 bash[66654]: cluster 2024-04-07T17:26:16.196711+0000 mgr.y (mgr.25175) 2127 : cluster [DBG] pgmap v3612: 137 pgs: 137 active+clean; 339 MiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 79 KiB/s rd, 2.8 MiB/s wr, 39 op/s 2024-04-07T17:26:17.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:26:17 smithi144 bash[66654]: audit 2024-04-07T17:26:16.841485+0000 mon.a (mon.0) 3654 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:26:17.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:26:17 smithi100 bash[74942]: cluster 2024-04-07T17:26:16.196711+0000 mgr.y (mgr.25175) 2127 : cluster [DBG] pgmap v3612: 137 pgs: 137 active+clean; 339 MiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 79 KiB/s rd, 2.8 MiB/s wr, 39 op/s 2024-04-07T17:26:17.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:26:17 smithi100 bash[74942]: audit 2024-04-07T17:26:16.841485+0000 mon.a (mon.0) 3654 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:26:17.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:26:17 smithi100 bash[76202]: cluster 2024-04-07T17:26:16.196711+0000 mgr.y (mgr.25175) 2127 : cluster [DBG] pgmap v3612: 137 pgs: 137 active+clean; 339 MiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 79 KiB/s rd, 2.8 MiB/s wr, 39 op/s 2024-04-07T17:26:17.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:26:17 smithi100 bash[76202]: audit 2024-04-07T17:26:16.841485+0000 mon.a (mon.0) 3654 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:26:18.704 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:26:18 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:26:18] "GET /metrics HTTP/1.1" 200 36757 "" "Prometheus/2.43.0" 2024-04-07T17:26:19.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:26:19 smithi144 bash[66654]: cluster 2024-04-07T17:26:18.197954+0000 mgr.y (mgr.25175) 2128 : cluster [DBG] pgmap v3613: 137 pgs: 137 active+clean; 343 MiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 1.8 MiB/s rd, 3.7 MiB/s wr, 30 op/s 2024-04-07T17:26:19.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:26:19 smithi100 bash[74942]: cluster 2024-04-07T17:26:18.197954+0000 mgr.y (mgr.25175) 2128 : cluster [DBG] pgmap v3613: 137 pgs: 137 active+clean; 343 MiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 1.8 MiB/s rd, 3.7 MiB/s wr, 30 op/s 2024-04-07T17:26:19.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:26:19 smithi100 bash[76202]: cluster 2024-04-07T17:26:18.197954+0000 mgr.y (mgr.25175) 2128 : cluster [DBG] pgmap v3613: 137 pgs: 137 active+clean; 343 MiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 1.8 MiB/s rd, 3.7 MiB/s wr, 30 op/s 2024-04-07T17:26:20.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:26:20 smithi144 bash[66654]: cluster 2024-04-07T17:26:19.330162+0000 mon.a (mon.0) 3655 : cluster [DBG] osdmap e1911: 8 total, 8 up, 8 in 2024-04-07T17:26:20.706 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:26:20 smithi100 bash[74942]: cluster 2024-04-07T17:26:19.330162+0000 mon.a (mon.0) 3655 : cluster [DBG] osdmap e1911: 8 total, 8 up, 8 in 2024-04-07T17:26:20.706 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:26:20 smithi100 bash[76202]: cluster 2024-04-07T17:26:19.330162+0000 mon.a (mon.0) 3655 : cluster [DBG] osdmap e1911: 8 total, 8 up, 8 in 2024-04-07T17:26:21.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:26:21 smithi144 bash[66654]: cluster 2024-04-07T17:26:20.198769+0000 mgr.y (mgr.25175) 2129 : cluster [DBG] pgmap v3615: 137 pgs: 137 active+clean; 351 MiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 2.1 MiB/s rd, 2.4 MiB/s wr, 43 op/s 2024-04-07T17:26:21.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:26:21 smithi100 bash[74942]: cluster 2024-04-07T17:26:20.198769+0000 mgr.y (mgr.25175) 2129 : cluster [DBG] pgmap v3615: 137 pgs: 137 active+clean; 351 MiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 2.1 MiB/s rd, 2.4 MiB/s wr, 43 op/s 2024-04-07T17:26:21.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:26:21 smithi100 bash[76202]: cluster 2024-04-07T17:26:20.198769+0000 mgr.y (mgr.25175) 2129 : cluster [DBG] pgmap v3615: 137 pgs: 137 active+clean; 351 MiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 2.1 MiB/s rd, 2.4 MiB/s wr, 43 op/s 2024-04-07T17:26:22.616 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:26:22 smithi144 bash[95233]: ts=2024-04-07T17:26:22.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:26:23.617 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:26:23 smithi144 bash[66654]: cluster 2024-04-07T17:26:22.199908+0000 mgr.y (mgr.25175) 2130 : cluster [DBG] pgmap v3616: 137 pgs: 137 active+clean; 372 MiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 2.1 MiB/s rd, 4.1 MiB/s wr, 50 op/s 2024-04-07T17:26:23.705 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:26:23 smithi100 bash[74942]: cluster 2024-04-07T17:26:22.199908+0000 mgr.y (mgr.25175) 2130 : cluster [DBG] pgmap v3616: 137 pgs: 137 active+clean; 372 MiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 2.1 MiB/s rd, 4.1 MiB/s wr, 50 op/s 2024-04-07T17:26:23.706 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:26:23 smithi100 bash[76202]: cluster 2024-04-07T17:26:22.199908+0000 mgr.y (mgr.25175) 2130 : cluster [DBG] pgmap v3616: 137 pgs: 137 active+clean; 372 MiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 2.1 MiB/s rd, 4.1 MiB/s wr, 50 op/s 2024-04-07T17:26:25.617 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:26:25 smithi144 bash[66654]: cluster 2024-04-07T17:26:24.200929+0000 mgr.y (mgr.25175) 2131 : cluster [DBG] pgmap v3617: 137 pgs: 137 active+clean; 376 MiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 2.1 MiB/s rd, 4.5 MiB/s wr, 42 op/s 2024-04-07T17:26:25.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:26:25 smithi100 bash[74942]: cluster 2024-04-07T17:26:24.200929+0000 mgr.y (mgr.25175) 2131 : cluster [DBG] pgmap v3617: 137 pgs: 137 active+clean; 376 MiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 2.1 MiB/s rd, 4.5 MiB/s wr, 42 op/s 2024-04-07T17:26:25.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:26:25 smithi100 bash[76202]: cluster 2024-04-07T17:26:24.200929+0000 mgr.y (mgr.25175) 2131 : cluster [DBG] pgmap v3617: 137 pgs: 137 active+clean; 376 MiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 2.1 MiB/s rd, 4.5 MiB/s wr, 42 op/s 2024-04-07T17:26:27.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:26:27 smithi144 bash[66654]: cluster 2024-04-07T17:26:26.201744+0000 mgr.y (mgr.25175) 2132 : cluster [DBG] pgmap v3618: 137 pgs: 137 active+clean; 376 MiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 2.1 MiB/s rd, 4.5 MiB/s wr, 45 op/s 2024-04-07T17:26:27.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:26:27 smithi100 bash[76202]: cluster 2024-04-07T17:26:26.201744+0000 mgr.y (mgr.25175) 2132 : cluster [DBG] pgmap v3618: 137 pgs: 137 active+clean; 376 MiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 2.1 MiB/s rd, 4.5 MiB/s wr, 45 op/s 2024-04-07T17:26:27.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:26:27 smithi100 bash[74942]: cluster 2024-04-07T17:26:26.201744+0000 mgr.y (mgr.25175) 2132 : cluster [DBG] pgmap v3618: 137 pgs: 137 active+clean; 376 MiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 2.1 MiB/s rd, 4.5 MiB/s wr, 45 op/s 2024-04-07T17:26:28.704 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:26:28 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:26:28] "GET /metrics HTTP/1.1" 200 36757 "" "Prometheus/2.43.0" 2024-04-07T17:26:29.882 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] EncryptedFlattenTest/3.ZeroOverlap (15374 ms) 2024-04-07T17:26:30.057 INFO:tasks.workunit.client.0.smithi100.stdout:[----------] 7 tests from EncryptedFlattenTest/3 (109389 ms total) 2024-04-07T17:26:30.057 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:26:30.057 INFO:tasks.workunit.client.0.smithi100.stdout:[----------] 7 tests from EncryptedFlattenTest/4, where TypeParam = LUKS1UnderLUKS2 2024-04-07T17:26:30.100 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] EncryptedFlattenTest/4.Simple 2024-04-07T17:26:30.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:26:30 smithi144 bash[66654]: cluster 2024-04-07T17:26:28.203068+0000 mgr.y (mgr.25175) 2133 : cluster [DBG] pgmap v3619: 137 pgs: 137 active+clean; 376 MiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 88 KiB/s rd, 3.3 MiB/s wr, 42 op/s 2024-04-07T17:26:30.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:26:30 smithi100 bash[74942]: cluster 2024-04-07T17:26:28.203068+0000 mgr.y (mgr.25175) 2133 : cluster [DBG] pgmap v3619: 137 pgs: 137 active+clean; 376 MiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 88 KiB/s rd, 3.3 MiB/s wr, 42 op/s 2024-04-07T17:26:30.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:26:30 smithi100 bash[76202]: cluster 2024-04-07T17:26:28.203068+0000 mgr.y (mgr.25175) 2133 : cluster [DBG] pgmap v3619: 137 pgs: 137 active+clean; 376 MiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 88 KiB/s rd, 3.3 MiB/s wr, 42 op/s 2024-04-07T17:26:31.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:26:31 smithi144 bash[66654]: cluster 2024-04-07T17:26:30.050852+0000 mon.a (mon.0) 3656 : cluster [DBG] osdmap e1912: 8 total, 8 up, 8 in 2024-04-07T17:26:31.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:26:31 smithi100 bash[74942]: cluster 2024-04-07T17:26:30.050852+0000 mon.a (mon.0) 3656 : cluster [DBG] osdmap e1912: 8 total, 8 up, 8 in 2024-04-07T17:26:31.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:26:31 smithi100 bash[76202]: cluster 2024-04-07T17:26:30.050852+0000 mon.a (mon.0) 3656 : cluster [DBG] osdmap e1912: 8 total, 8 up, 8 in 2024-04-07T17:26:32.093 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:26:32.366 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:26:32 smithi144 bash[95233]: ts=2024-04-07T17:26:32.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:26:32.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:26:32 smithi144 bash[66654]: cluster 2024-04-07T17:26:30.203677+0000 mgr.y (mgr.25175) 2134 : cluster [DBG] pgmap v3621: 105 pgs: 105 active+clean; 4.6 MiB data, 4.6 GiB used, 711 GiB / 715 GiB avail 2024-04-07T17:26:32.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:26:32 smithi144 bash[66654]: cluster 2024-04-07T17:26:31.078401+0000 mon.a (mon.0) 3657 : cluster [DBG] osdmap e1913: 8 total, 8 up, 8 in 2024-04-07T17:26:32.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:26:32 smithi144 bash[66654]: audit 2024-04-07T17:26:31.088037+0000 mon.c (mon.2) 96 : audit [INF] from='client.? 172.21.15.100:0/783130163' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-40","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:26:32.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:26:32 smithi144 bash[66654]: audit 2024-04-07T17:26:31.089151+0000 mon.a (mon.0) 3658 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-40","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:26:32.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:26:32 smithi144 bash[66654]: audit 2024-04-07T17:26:31.842153+0000 mon.a (mon.0) 3659 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:26:32.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:26:32 smithi100 bash[74942]: cluster 2024-04-07T17:26:30.203677+0000 mgr.y (mgr.25175) 2134 : cluster [DBG] pgmap v3621: 105 pgs: 105 active+clean; 4.6 MiB data, 4.6 GiB used, 711 GiB / 715 GiB avail 2024-04-07T17:26:32.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:26:32 smithi100 bash[74942]: cluster 2024-04-07T17:26:31.078401+0000 mon.a (mon.0) 3657 : cluster [DBG] osdmap e1913: 8 total, 8 up, 8 in 2024-04-07T17:26:32.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:26:32 smithi100 bash[74942]: audit 2024-04-07T17:26:31.088037+0000 mon.c (mon.2) 96 : audit [INF] from='client.? 172.21.15.100:0/783130163' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-40","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:26:32.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:26:32 smithi100 bash[74942]: audit 2024-04-07T17:26:31.089151+0000 mon.a (mon.0) 3658 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-40","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:26:32.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:26:32 smithi100 bash[74942]: audit 2024-04-07T17:26:31.842153+0000 mon.a (mon.0) 3659 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:26:32.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:26:32 smithi100 bash[76202]: cluster 2024-04-07T17:26:30.203677+0000 mgr.y (mgr.25175) 2134 : cluster [DBG] pgmap v3621: 105 pgs: 105 active+clean; 4.6 MiB data, 4.6 GiB used, 711 GiB / 715 GiB avail 2024-04-07T17:26:32.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:26:32 smithi100 bash[76202]: cluster 2024-04-07T17:26:31.078401+0000 mon.a (mon.0) 3657 : cluster [DBG] osdmap e1913: 8 total, 8 up, 8 in 2024-04-07T17:26:32.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:26:32 smithi100 bash[76202]: audit 2024-04-07T17:26:31.088037+0000 mon.c (mon.2) 96 : audit [INF] from='client.? 172.21.15.100:0/783130163' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-40","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:26:32.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:26:32 smithi100 bash[76202]: audit 2024-04-07T17:26:31.089151+0000 mon.a (mon.0) 3658 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-40","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:26:32.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:26:32 smithi100 bash[76202]: audit 2024-04-07T17:26:31.842153+0000 mon.a (mon.0) 3659 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:26:33.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:26:33 smithi144 bash[66654]: audit 2024-04-07T17:26:32.075411+0000 mon.a (mon.0) 3660 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-40","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T17:26:33.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:26:33 smithi144 bash[66654]: cluster 2024-04-07T17:26:32.084487+0000 mon.a (mon.0) 3661 : cluster [DBG] osdmap e1914: 8 total, 8 up, 8 in 2024-04-07T17:26:33.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:26:33 smithi144 bash[66654]: cluster 2024-04-07T17:26:32.204393+0000 mgr.y (mgr.25175) 2135 : cluster [DBG] pgmap v3624: 137 pgs: 12 creating+peering, 20 unknown, 105 active+clean; 4.6 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail 2024-04-07T17:26:33.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:26:33 smithi100 bash[74942]: audit 2024-04-07T17:26:32.075411+0000 mon.a (mon.0) 3660 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-40","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T17:26:33.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:26:33 smithi100 bash[74942]: cluster 2024-04-07T17:26:32.084487+0000 mon.a (mon.0) 3661 : cluster [DBG] osdmap e1914: 8 total, 8 up, 8 in 2024-04-07T17:26:33.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:26:33 smithi100 bash[74942]: cluster 2024-04-07T17:26:32.204393+0000 mgr.y (mgr.25175) 2135 : cluster [DBG] pgmap v3624: 137 pgs: 12 creating+peering, 20 unknown, 105 active+clean; 4.6 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail 2024-04-07T17:26:33.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:26:33 smithi100 bash[76202]: audit 2024-04-07T17:26:32.075411+0000 mon.a (mon.0) 3660 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-40","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T17:26:33.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:26:33 smithi100 bash[76202]: cluster 2024-04-07T17:26:32.084487+0000 mon.a (mon.0) 3661 : cluster [DBG] osdmap e1914: 8 total, 8 up, 8 in 2024-04-07T17:26:33.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:26:33 smithi100 bash[76202]: cluster 2024-04-07T17:26:32.204393+0000 mgr.y (mgr.25175) 2135 : cluster [DBG] pgmap v3624: 137 pgs: 12 creating+peering, 20 unknown, 105 active+clean; 4.6 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail 2024-04-07T17:26:34.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:26:34 smithi100 bash[74942]: cluster 2024-04-07T17:26:33.118171+0000 mon.a (mon.0) 3662 : cluster [DBG] osdmap e1915: 8 total, 8 up, 8 in 2024-04-07T17:26:34.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:26:34 smithi100 bash[76202]: cluster 2024-04-07T17:26:33.118171+0000 mon.a (mon.0) 3662 : cluster [DBG] osdmap e1915: 8 total, 8 up, 8 in 2024-04-07T17:26:34.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:26:34 smithi144 bash[66654]: cluster 2024-04-07T17:26:33.118171+0000 mon.a (mon.0) 3662 : cluster [DBG] osdmap e1915: 8 total, 8 up, 8 in 2024-04-07T17:26:35.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:26:35 smithi100 bash[74942]: cluster 2024-04-07T17:26:34.119275+0000 mon.a (mon.0) 3663 : cluster [DBG] osdmap e1916: 8 total, 8 up, 8 in 2024-04-07T17:26:35.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:26:35 smithi100 bash[74942]: cluster 2024-04-07T17:26:34.204849+0000 mgr.y (mgr.25175) 2136 : cluster [DBG] pgmap v3627: 137 pgs: 12 creating+peering, 4 unknown, 121 active+clean; 4.6 MiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 255 B/s wr, 0 op/s 2024-04-07T17:26:35.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:26:35 smithi100 bash[76202]: cluster 2024-04-07T17:26:34.119275+0000 mon.a (mon.0) 3663 : cluster [DBG] osdmap e1916: 8 total, 8 up, 8 in 2024-04-07T17:26:35.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:26:35 smithi100 bash[76202]: cluster 2024-04-07T17:26:34.204849+0000 mgr.y (mgr.25175) 2136 : cluster [DBG] pgmap v3627: 137 pgs: 12 creating+peering, 4 unknown, 121 active+clean; 4.6 MiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 255 B/s wr, 0 op/s 2024-04-07T17:26:35.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:26:35 smithi144 bash[66654]: cluster 2024-04-07T17:26:34.119275+0000 mon.a (mon.0) 3663 : cluster [DBG] osdmap e1916: 8 total, 8 up, 8 in 2024-04-07T17:26:35.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:26:35 smithi144 bash[66654]: cluster 2024-04-07T17:26:34.204849+0000 mgr.y (mgr.25175) 2136 : cluster [DBG] pgmap v3627: 137 pgs: 12 creating+peering, 4 unknown, 121 active+clean; 4.6 MiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 255 B/s wr, 0 op/s 2024-04-07T17:26:36.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:26:36 smithi144 bash[66654]: audit 2024-04-07T17:26:36.254029+0000 mon.a (mon.0) 3664 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:26:36.703 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:26:36 smithi100 bash[74942]: audit 2024-04-07T17:26:36.254029+0000 mon.a (mon.0) 3664 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:26:36.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:26:36 smithi100 bash[76202]: audit 2024-04-07T17:26:36.254029+0000 mon.a (mon.0) 3664 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:26:38.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:26:37 smithi144 bash[66654]: cluster 2024-04-07T17:26:36.205622+0000 mgr.y (mgr.25175) 2137 : cluster [DBG] pgmap v3628: 137 pgs: 12 creating+peering, 125 active+clean; 4.6 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 399 B/s rd, 399 B/s wr, 0 op/s 2024-04-07T17:26:38.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:26:37 smithi144 bash[66654]: audit 2024-04-07T17:26:36.780344+0000 mon.a (mon.0) 3665 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:26:38.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:26:37 smithi144 bash[66654]: audit 2024-04-07T17:26:36.788470+0000 mon.a (mon.0) 3666 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:26:38.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:26:37 smithi144 bash[66654]: audit 2024-04-07T17:26:37.448887+0000 mon.a (mon.0) 3667 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:26:38.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:26:37 smithi144 bash[66654]: audit 2024-04-07T17:26:37.450393+0000 mon.a (mon.0) 3668 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:26:38.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:26:37 smithi144 bash[66654]: audit 2024-04-07T17:26:37.459814+0000 mon.a (mon.0) 3669 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:26:38.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:26:37 smithi100 bash[74942]: cluster 2024-04-07T17:26:36.205622+0000 mgr.y (mgr.25175) 2137 : cluster [DBG] pgmap v3628: 137 pgs: 12 creating+peering, 125 active+clean; 4.6 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 399 B/s rd, 399 B/s wr, 0 op/s 2024-04-07T17:26:38.205 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:26:37 smithi100 bash[74942]: audit 2024-04-07T17:26:36.780344+0000 mon.a (mon.0) 3665 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:26:38.205 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:26:37 smithi100 bash[74942]: audit 2024-04-07T17:26:36.788470+0000 mon.a (mon.0) 3666 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:26:38.205 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:26:37 smithi100 bash[74942]: audit 2024-04-07T17:26:37.448887+0000 mon.a (mon.0) 3667 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:26:38.205 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:26:37 smithi100 bash[74942]: audit 2024-04-07T17:26:37.450393+0000 mon.a (mon.0) 3668 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:26:38.205 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:26:37 smithi100 bash[74942]: audit 2024-04-07T17:26:37.459814+0000 mon.a (mon.0) 3669 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:26:38.206 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:26:37 smithi100 bash[76202]: cluster 2024-04-07T17:26:36.205622+0000 mgr.y (mgr.25175) 2137 : cluster [DBG] pgmap v3628: 137 pgs: 12 creating+peering, 125 active+clean; 4.6 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 399 B/s rd, 399 B/s wr, 0 op/s 2024-04-07T17:26:38.206 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:26:37 smithi100 bash[76202]: audit 2024-04-07T17:26:36.780344+0000 mon.a (mon.0) 3665 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:26:38.206 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:26:37 smithi100 bash[76202]: audit 2024-04-07T17:26:36.788470+0000 mon.a (mon.0) 3666 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:26:38.206 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:26:37 smithi100 bash[76202]: audit 2024-04-07T17:26:37.448887+0000 mon.a (mon.0) 3667 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:26:38.206 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:26:37 smithi100 bash[76202]: audit 2024-04-07T17:26:37.450393+0000 mon.a (mon.0) 3668 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:26:38.206 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:26:37 smithi100 bash[76202]: audit 2024-04-07T17:26:37.459814+0000 mon.a (mon.0) 3669 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:26:38.704 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:26:38 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:26:38] "GET /metrics HTTP/1.1" 200 36689 "" "Prometheus/2.43.0" 2024-04-07T17:26:40.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:26:39 smithi144 bash[66654]: cluster 2024-04-07T17:26:38.206999+0000 mgr.y (mgr.25175) 2138 : cluster [DBG] pgmap v3629: 137 pgs: 137 active+clean; 13 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 4.7 KiB/s rd, 1.3 MiB/s wr, 8 op/s 2024-04-07T17:26:40.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:26:39 smithi144 bash[66654]: cluster 2024-04-07T17:26:38.795654+0000 mon.a (mon.0) 3670 : cluster [DBG] osdmap e1917: 8 total, 8 up, 8 in 2024-04-07T17:26:40.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:26:39 smithi100 bash[74942]: cluster 2024-04-07T17:26:38.206999+0000 mgr.y (mgr.25175) 2138 : cluster [DBG] pgmap v3629: 137 pgs: 137 active+clean; 13 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 4.7 KiB/s rd, 1.3 MiB/s wr, 8 op/s 2024-04-07T17:26:40.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:26:39 smithi100 bash[74942]: cluster 2024-04-07T17:26:38.795654+0000 mon.a (mon.0) 3670 : cluster [DBG] osdmap e1917: 8 total, 8 up, 8 in 2024-04-07T17:26:40.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:26:39 smithi100 bash[76202]: cluster 2024-04-07T17:26:38.206999+0000 mgr.y (mgr.25175) 2138 : cluster [DBG] pgmap v3629: 137 pgs: 137 active+clean; 13 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 4.7 KiB/s rd, 1.3 MiB/s wr, 8 op/s 2024-04-07T17:26:40.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:26:39 smithi100 bash[76202]: cluster 2024-04-07T17:26:38.795654+0000 mon.a (mon.0) 3670 : cluster [DBG] osdmap e1917: 8 total, 8 up, 8 in 2024-04-07T17:26:42.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:26:41 smithi144 bash[66654]: cluster 2024-04-07T17:26:40.207693+0000 mgr.y (mgr.25175) 2139 : cluster [DBG] pgmap v3631: 137 pgs: 137 active+clean; 30 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 13 KiB/s rd, 3.5 MiB/s wr, 19 op/s 2024-04-07T17:26:42.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:26:41 smithi100 bash[74942]: cluster 2024-04-07T17:26:40.207693+0000 mgr.y (mgr.25175) 2139 : cluster [DBG] pgmap v3631: 137 pgs: 137 active+clean; 30 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 13 KiB/s rd, 3.5 MiB/s wr, 19 op/s 2024-04-07T17:26:42.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:26:41 smithi100 bash[76202]: cluster 2024-04-07T17:26:40.207693+0000 mgr.y (mgr.25175) 2139 : cluster [DBG] pgmap v3631: 137 pgs: 137 active+clean; 30 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 13 KiB/s rd, 3.5 MiB/s wr, 19 op/s 2024-04-07T17:26:42.617 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:26:42 smithi144 bash[95233]: ts=2024-04-07T17:26:42.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:26:44.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:26:44 smithi144 bash[66654]: cluster 2024-04-07T17:26:42.208476+0000 mgr.y (mgr.25175) 2140 : cluster [DBG] pgmap v3632: 137 pgs: 137 active+clean; 34 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 36 KiB/s rd, 3.6 MiB/s wr, 48 op/s 2024-04-07T17:26:44.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:26:44 smithi144 bash[66654]: cluster 2024-04-07T17:26:43.045299+0000 mon.a (mon.0) 3671 : cluster [DBG] osdmap e1918: 8 total, 8 up, 8 in 2024-04-07T17:26:44.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:26:44 smithi100 bash[74942]: cluster 2024-04-07T17:26:42.208476+0000 mgr.y (mgr.25175) 2140 : cluster [DBG] pgmap v3632: 137 pgs: 137 active+clean; 34 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 36 KiB/s rd, 3.6 MiB/s wr, 48 op/s 2024-04-07T17:26:44.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:26:44 smithi100 bash[74942]: cluster 2024-04-07T17:26:43.045299+0000 mon.a (mon.0) 3671 : cluster [DBG] osdmap e1918: 8 total, 8 up, 8 in 2024-04-07T17:26:44.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:26:44 smithi100 bash[76202]: cluster 2024-04-07T17:26:42.208476+0000 mgr.y (mgr.25175) 2140 : cluster [DBG] pgmap v3632: 137 pgs: 137 active+clean; 34 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 36 KiB/s rd, 3.6 MiB/s wr, 48 op/s 2024-04-07T17:26:44.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:26:44 smithi100 bash[76202]: cluster 2024-04-07T17:26:43.045299+0000 mon.a (mon.0) 3671 : cluster [DBG] osdmap e1918: 8 total, 8 up, 8 in 2024-04-07T17:26:46.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:26:46 smithi144 bash[66654]: cluster 2024-04-07T17:26:44.209135+0000 mgr.y (mgr.25175) 2141 : cluster [DBG] pgmap v3634: 137 pgs: 137 active+clean; 34 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 36 KiB/s rd, 3.6 MiB/s wr, 49 op/s 2024-04-07T17:26:46.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:26:46 smithi100 bash[74942]: cluster 2024-04-07T17:26:44.209135+0000 mgr.y (mgr.25175) 2141 : cluster [DBG] pgmap v3634: 137 pgs: 137 active+clean; 34 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 36 KiB/s rd, 3.6 MiB/s wr, 49 op/s 2024-04-07T17:26:46.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:26:46 smithi100 bash[76202]: cluster 2024-04-07T17:26:44.209135+0000 mgr.y (mgr.25175) 2141 : cluster [DBG] pgmap v3634: 137 pgs: 137 active+clean; 34 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 36 KiB/s rd, 3.6 MiB/s wr, 49 op/s 2024-04-07T17:26:47.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:26:47 smithi144 bash[66654]: audit 2024-04-07T17:26:46.841761+0000 mon.a (mon.0) 3672 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:26:47.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:26:47 smithi100 bash[74942]: audit 2024-04-07T17:26:46.841761+0000 mon.a (mon.0) 3672 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:26:47.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:26:47 smithi100 bash[76202]: audit 2024-04-07T17:26:46.841761+0000 mon.a (mon.0) 3672 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:26:48.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:26:48 smithi144 bash[66654]: cluster 2024-04-07T17:26:46.209726+0000 mgr.y (mgr.25175) 2142 : cluster [DBG] pgmap v3635: 137 pgs: 137 active+clean; 34 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 33 KiB/s rd, 2.6 MiB/s wr, 44 op/s 2024-04-07T17:26:48.427 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:26:48 smithi100 bash[74942]: cluster 2024-04-07T17:26:46.209726+0000 mgr.y (mgr.25175) 2142 : cluster [DBG] pgmap v3635: 137 pgs: 137 active+clean; 34 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 33 KiB/s rd, 2.6 MiB/s wr, 44 op/s 2024-04-07T17:26:48.428 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:26:48 smithi100 bash[76202]: cluster 2024-04-07T17:26:46.209726+0000 mgr.y (mgr.25175) 2142 : cluster [DBG] pgmap v3635: 137 pgs: 137 active+clean; 34 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 33 KiB/s rd, 2.6 MiB/s wr, 44 op/s 2024-04-07T17:26:48.704 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:26:48 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:26:48] "GET /metrics HTTP/1.1" 200 36751 "" "Prometheus/2.43.0" 2024-04-07T17:26:50.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:26:50 smithi144 bash[66654]: cluster 2024-04-07T17:26:48.210927+0000 mgr.y (mgr.25175) 2143 : cluster [DBG] pgmap v3636: 137 pgs: 137 active+clean; 42 MiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 64 KiB/s rd, 3.1 MiB/s wr, 45 op/s 2024-04-07T17:26:50.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:26:50 smithi100 bash[74942]: cluster 2024-04-07T17:26:48.210927+0000 mgr.y (mgr.25175) 2143 : cluster [DBG] pgmap v3636: 137 pgs: 137 active+clean; 42 MiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 64 KiB/s rd, 3.1 MiB/s wr, 45 op/s 2024-04-07T17:26:50.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:26:50 smithi100 bash[76202]: cluster 2024-04-07T17:26:48.210927+0000 mgr.y (mgr.25175) 2143 : cluster [DBG] pgmap v3636: 137 pgs: 137 active+clean; 42 MiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 64 KiB/s rd, 3.1 MiB/s wr, 45 op/s 2024-04-07T17:26:50.813 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] EncryptedFlattenTest/4.Simple (20713 ms) 2024-04-07T17:26:50.813 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] EncryptedFlattenTest/4.Grow 2024-04-07T17:26:50.813 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:26:52.317 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:26:52 smithi144 bash[66654]: cluster 2024-04-07T17:26:50.211662+0000 mgr.y (mgr.25175) 2144 : cluster [DBG] pgmap v3637: 137 pgs: 137 active+clean; 59 MiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 4.0 MiB/s rd, 3.7 MiB/s wr, 45 op/s 2024-04-07T17:26:52.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:26:52 smithi100 bash[74942]: cluster 2024-04-07T17:26:50.211662+0000 mgr.y (mgr.25175) 2144 : cluster [DBG] pgmap v3637: 137 pgs: 137 active+clean; 59 MiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 4.0 MiB/s rd, 3.7 MiB/s wr, 45 op/s 2024-04-07T17:26:52.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:26:52 smithi100 bash[76202]: cluster 2024-04-07T17:26:50.211662+0000 mgr.y (mgr.25175) 2144 : cluster [DBG] pgmap v3637: 137 pgs: 137 active+clean; 59 MiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 4.0 MiB/s rd, 3.7 MiB/s wr, 45 op/s 2024-04-07T17:26:52.616 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:26:52 smithi144 bash[95233]: ts=2024-04-07T17:26:52.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:26:54.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:26:54 smithi144 bash[66654]: cluster 2024-04-07T17:26:52.212686+0000 mgr.y (mgr.25175) 2145 : cluster [DBG] pgmap v3638: 137 pgs: 137 active+clean; 71 MiB data, 3.7 GiB used, 711 GiB / 715 GiB avail; 7.6 MiB/s rd, 4.9 MiB/s wr, 28 op/s 2024-04-07T17:26:54.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:26:54 smithi100 bash[74942]: cluster 2024-04-07T17:26:52.212686+0000 mgr.y (mgr.25175) 2145 : cluster [DBG] pgmap v3638: 137 pgs: 137 active+clean; 71 MiB data, 3.7 GiB used, 711 GiB / 715 GiB avail; 7.6 MiB/s rd, 4.9 MiB/s wr, 28 op/s 2024-04-07T17:26:54.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:26:54 smithi100 bash[76202]: cluster 2024-04-07T17:26:52.212686+0000 mgr.y (mgr.25175) 2145 : cluster [DBG] pgmap v3638: 137 pgs: 137 active+clean; 71 MiB data, 3.7 GiB used, 711 GiB / 715 GiB avail; 7.6 MiB/s rd, 4.9 MiB/s wr, 28 op/s 2024-04-07T17:26:56.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:26:56 smithi144 bash[66654]: cluster 2024-04-07T17:26:54.213714+0000 mgr.y (mgr.25175) 2146 : cluster [DBG] pgmap v3639: 137 pgs: 137 active+clean; 75 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 10 MiB/s rd, 5.1 MiB/s wr, 30 op/s 2024-04-07T17:26:56.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:26:56 smithi144 bash[66654]: cluster 2024-04-07T17:26:55.084776+0000 mon.a (mon.0) 3673 : cluster [DBG] osdmap e1919: 8 total, 8 up, 8 in 2024-04-07T17:26:56.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:26:56 smithi100 bash[74942]: cluster 2024-04-07T17:26:54.213714+0000 mgr.y (mgr.25175) 2146 : cluster [DBG] pgmap v3639: 137 pgs: 137 active+clean; 75 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 10 MiB/s rd, 5.1 MiB/s wr, 30 op/s 2024-04-07T17:26:56.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:26:56 smithi100 bash[74942]: cluster 2024-04-07T17:26:55.084776+0000 mon.a (mon.0) 3673 : cluster [DBG] osdmap e1919: 8 total, 8 up, 8 in 2024-04-07T17:26:56.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:26:56 smithi100 bash[76202]: cluster 2024-04-07T17:26:54.213714+0000 mgr.y (mgr.25175) 2146 : cluster [DBG] pgmap v3639: 137 pgs: 137 active+clean; 75 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 10 MiB/s rd, 5.1 MiB/s wr, 30 op/s 2024-04-07T17:26:56.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:26:56 smithi100 bash[76202]: cluster 2024-04-07T17:26:55.084776+0000 mon.a (mon.0) 3673 : cluster [DBG] osdmap e1919: 8 total, 8 up, 8 in 2024-04-07T17:26:58.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:26:58 smithi144 bash[66654]: cluster 2024-04-07T17:26:56.214384+0000 mgr.y (mgr.25175) 2147 : cluster [DBG] pgmap v3641: 137 pgs: 137 active+clean; 79 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 12 MiB/s rd, 6.1 MiB/s wr, 34 op/s 2024-04-07T17:26:58.426 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:26:58 smithi100 bash[74942]: cluster 2024-04-07T17:26:56.214384+0000 mgr.y (mgr.25175) 2147 : cluster [DBG] pgmap v3641: 137 pgs: 137 active+clean; 79 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 12 MiB/s rd, 6.1 MiB/s wr, 34 op/s 2024-04-07T17:26:58.426 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:26:58 smithi100 bash[76202]: cluster 2024-04-07T17:26:56.214384+0000 mgr.y (mgr.25175) 2147 : cluster [DBG] pgmap v3641: 137 pgs: 137 active+clean; 79 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 12 MiB/s rd, 6.1 MiB/s wr, 34 op/s 2024-04-07T17:26:58.704 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:26:58 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:26:58] "GET /metrics HTTP/1.1" 200 36751 "" "Prometheus/2.43.0" 2024-04-07T17:27:00.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:27:00 smithi144 bash[66654]: cluster 2024-04-07T17:26:58.215267+0000 mgr.y (mgr.25175) 2148 : cluster [DBG] pgmap v3642: 137 pgs: 137 active+clean; 92 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 12 MiB/s rd, 6.6 MiB/s wr, 50 op/s 2024-04-07T17:27:00.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:27:00 smithi100 bash[74942]: cluster 2024-04-07T17:26:58.215267+0000 mgr.y (mgr.25175) 2148 : cluster [DBG] pgmap v3642: 137 pgs: 137 active+clean; 92 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 12 MiB/s rd, 6.6 MiB/s wr, 50 op/s 2024-04-07T17:27:00.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:27:00 smithi100 bash[76202]: cluster 2024-04-07T17:26:58.215267+0000 mgr.y (mgr.25175) 2148 : cluster [DBG] pgmap v3642: 137 pgs: 137 active+clean; 92 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 12 MiB/s rd, 6.6 MiB/s wr, 50 op/s 2024-04-07T17:27:01.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:27:01 smithi100 bash[74942]: cluster 2024-04-07T17:27:00.216006+0000 mgr.y (mgr.25175) 2149 : cluster [DBG] pgmap v3643: 137 pgs: 137 active+clean; 104 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 7.8 MiB/s rd, 5.3 MiB/s wr, 50 op/s 2024-04-07T17:27:01.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:27:01 smithi100 bash[76202]: cluster 2024-04-07T17:27:00.216006+0000 mgr.y (mgr.25175) 2149 : cluster [DBG] pgmap v3643: 137 pgs: 137 active+clean; 104 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 7.8 MiB/s rd, 5.3 MiB/s wr, 50 op/s 2024-04-07T17:27:01.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:27:01 smithi144 bash[66654]: cluster 2024-04-07T17:27:00.216006+0000 mgr.y (mgr.25175) 2149 : cluster [DBG] pgmap v3643: 137 pgs: 137 active+clean; 104 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 7.8 MiB/s rd, 5.3 MiB/s wr, 50 op/s 2024-04-07T17:27:02.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:27:02 smithi100 bash[74942]: audit 2024-04-07T17:27:01.842078+0000 mon.a (mon.0) 3674 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:27:02.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:27:02 smithi100 bash[76202]: audit 2024-04-07T17:27:01.842078+0000 mon.a (mon.0) 3674 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:27:02.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:27:02 smithi144 bash[66654]: audit 2024-04-07T17:27:01.842078+0000 mon.a (mon.0) 3674 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:27:02.616 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:27:02 smithi144 bash[95233]: ts=2024-04-07T17:27:02.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:27:03.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:27:03 smithi100 bash[74942]: cluster 2024-04-07T17:27:02.216908+0000 mgr.y (mgr.25175) 2150 : cluster [DBG] pgmap v3644: 137 pgs: 137 active+clean; 104 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 4.2 MiB/s rd, 3.7 MiB/s wr, 41 op/s 2024-04-07T17:27:03.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:27:03 smithi100 bash[76202]: cluster 2024-04-07T17:27:02.216908+0000 mgr.y (mgr.25175) 2150 : cluster [DBG] pgmap v3644: 137 pgs: 137 active+clean; 104 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 4.2 MiB/s rd, 3.7 MiB/s wr, 41 op/s 2024-04-07T17:27:03.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:27:03 smithi144 bash[66654]: cluster 2024-04-07T17:27:02.216908+0000 mgr.y (mgr.25175) 2150 : cluster [DBG] pgmap v3644: 137 pgs: 137 active+clean; 104 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 4.2 MiB/s rd, 3.7 MiB/s wr, 41 op/s 2024-04-07T17:27:05.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:27:05 smithi144 bash[66654]: cluster 2024-04-07T17:27:04.217631+0000 mgr.y (mgr.25175) 2151 : cluster [DBG] pgmap v3645: 137 pgs: 137 active+clean; 104 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 26 KiB/s rd, 2.9 MiB/s wr, 35 op/s 2024-04-07T17:27:05.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:27:05 smithi100 bash[74942]: cluster 2024-04-07T17:27:04.217631+0000 mgr.y (mgr.25175) 2151 : cluster [DBG] pgmap v3645: 137 pgs: 137 active+clean; 104 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 26 KiB/s rd, 2.9 MiB/s wr, 35 op/s 2024-04-07T17:27:05.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:27:05 smithi100 bash[76202]: cluster 2024-04-07T17:27:04.217631+0000 mgr.y (mgr.25175) 2151 : cluster [DBG] pgmap v3645: 137 pgs: 137 active+clean; 104 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 26 KiB/s rd, 2.9 MiB/s wr, 35 op/s 2024-04-07T17:27:07.618 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:27:07 smithi144 bash[66654]: cluster 2024-04-07T17:27:06.218249+0000 mgr.y (mgr.25175) 2152 : cluster [DBG] pgmap v3646: 137 pgs: 137 active+clean; 104 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 23 KiB/s rd, 2.2 MiB/s wr, 31 op/s 2024-04-07T17:27:07.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:27:07 smithi100 bash[74942]: cluster 2024-04-07T17:27:06.218249+0000 mgr.y (mgr.25175) 2152 : cluster [DBG] pgmap v3646: 137 pgs: 137 active+clean; 104 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 23 KiB/s rd, 2.2 MiB/s wr, 31 op/s 2024-04-07T17:27:07.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:27:07 smithi100 bash[76202]: cluster 2024-04-07T17:27:06.218249+0000 mgr.y (mgr.25175) 2152 : cluster [DBG] pgmap v3646: 137 pgs: 137 active+clean; 104 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 23 KiB/s rd, 2.2 MiB/s wr, 31 op/s 2024-04-07T17:27:08.704 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:27:08 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:27:08] "GET /metrics HTTP/1.1" 200 36755 "" "Prometheus/2.43.0" 2024-04-07T17:27:09.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:27:09 smithi144 bash[66654]: cluster 2024-04-07T17:27:08.219486+0000 mgr.y (mgr.25175) 2153 : cluster [DBG] pgmap v3647: 137 pgs: 137 active+clean; 108 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 26 KiB/s rd, 2.4 MiB/s wr, 34 op/s 2024-04-07T17:27:09.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:27:09 smithi100 bash[74942]: cluster 2024-04-07T17:27:08.219486+0000 mgr.y (mgr.25175) 2153 : cluster [DBG] pgmap v3647: 137 pgs: 137 active+clean; 108 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 26 KiB/s rd, 2.4 MiB/s wr, 34 op/s 2024-04-07T17:27:09.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:27:09 smithi100 bash[76202]: cluster 2024-04-07T17:27:08.219486+0000 mgr.y (mgr.25175) 2153 : cluster [DBG] pgmap v3647: 137 pgs: 137 active+clean; 108 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 26 KiB/s rd, 2.4 MiB/s wr, 34 op/s 2024-04-07T17:27:11.507 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] EncryptedFlattenTest/4.Grow (20694 ms) 2024-04-07T17:27:11.507 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] EncryptedFlattenTest/4.Shrink 2024-04-07T17:27:11.507 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:27:11.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:27:11 smithi144 bash[66654]: cluster 2024-04-07T17:27:10.220242+0000 mgr.y (mgr.25175) 2154 : cluster [DBG] pgmap v3648: 137 pgs: 137 active+clean; 124 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 1.1 MiB/s rd, 3.3 MiB/s wr, 25 op/s 2024-04-07T17:27:11.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:27:11 smithi100 bash[74942]: cluster 2024-04-07T17:27:10.220242+0000 mgr.y (mgr.25175) 2154 : cluster [DBG] pgmap v3648: 137 pgs: 137 active+clean; 124 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 1.1 MiB/s rd, 3.3 MiB/s wr, 25 op/s 2024-04-07T17:27:11.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:27:11 smithi100 bash[76202]: cluster 2024-04-07T17:27:10.220242+0000 mgr.y (mgr.25175) 2154 : cluster [DBG] pgmap v3648: 137 pgs: 137 active+clean; 124 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 1.1 MiB/s rd, 3.3 MiB/s wr, 25 op/s 2024-04-07T17:27:12.616 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:27:12 smithi144 bash[95233]: ts=2024-04-07T17:27:12.317Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:27:13.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:27:13 smithi144 bash[66654]: cluster 2024-04-07T17:27:12.221341+0000 mgr.y (mgr.25175) 2155 : cluster [DBG] pgmap v3649: 137 pgs: 137 active+clean; 133 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 4.4 MiB/s rd, 3.4 MiB/s wr, 23 op/s 2024-04-07T17:27:13.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:27:13 smithi100 bash[74942]: cluster 2024-04-07T17:27:12.221341+0000 mgr.y (mgr.25175) 2155 : cluster [DBG] pgmap v3649: 137 pgs: 137 active+clean; 133 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 4.4 MiB/s rd, 3.4 MiB/s wr, 23 op/s 2024-04-07T17:27:13.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:27:13 smithi100 bash[76202]: cluster 2024-04-07T17:27:12.221341+0000 mgr.y (mgr.25175) 2155 : cluster [DBG] pgmap v3649: 137 pgs: 137 active+clean; 133 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 4.4 MiB/s rd, 3.4 MiB/s wr, 23 op/s 2024-04-07T17:27:15.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:27:15 smithi144 bash[66654]: cluster 2024-04-07T17:27:14.222346+0000 mgr.y (mgr.25175) 2156 : cluster [DBG] pgmap v3650: 137 pgs: 137 active+clean; 145 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 9.4 MiB/s rd, 4.8 MiB/s wr, 30 op/s 2024-04-07T17:27:15.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:27:15 smithi100 bash[74942]: cluster 2024-04-07T17:27:14.222346+0000 mgr.y (mgr.25175) 2156 : cluster [DBG] pgmap v3650: 137 pgs: 137 active+clean; 145 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 9.4 MiB/s rd, 4.8 MiB/s wr, 30 op/s 2024-04-07T17:27:15.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:27:15 smithi100 bash[76202]: cluster 2024-04-07T17:27:14.222346+0000 mgr.y (mgr.25175) 2156 : cluster [DBG] pgmap v3650: 137 pgs: 137 active+clean; 145 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 9.4 MiB/s rd, 4.8 MiB/s wr, 30 op/s 2024-04-07T17:27:17.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:27:17 smithi144 bash[66654]: cluster 2024-04-07T17:27:16.223194+0000 mgr.y (mgr.25175) 2157 : cluster [DBG] pgmap v3651: 137 pgs: 137 active+clean; 145 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 9.7 MiB/s rd, 4.8 MiB/s wr, 30 op/s 2024-04-07T17:27:17.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:27:17 smithi144 bash[66654]: cluster 2024-04-07T17:27:16.312536+0000 mon.a (mon.0) 3675 : cluster [DBG] osdmap e1920: 8 total, 8 up, 8 in 2024-04-07T17:27:17.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:27:17 smithi144 bash[66654]: audit 2024-04-07T17:27:16.843290+0000 mon.a (mon.0) 3676 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:27:17.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:27:17 smithi100 bash[74942]: cluster 2024-04-07T17:27:16.223194+0000 mgr.y (mgr.25175) 2157 : cluster [DBG] pgmap v3651: 137 pgs: 137 active+clean; 145 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 9.7 MiB/s rd, 4.8 MiB/s wr, 30 op/s 2024-04-07T17:27:17.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:27:17 smithi100 bash[74942]: cluster 2024-04-07T17:27:16.312536+0000 mon.a (mon.0) 3675 : cluster [DBG] osdmap e1920: 8 total, 8 up, 8 in 2024-04-07T17:27:17.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:27:17 smithi100 bash[74942]: audit 2024-04-07T17:27:16.843290+0000 mon.a (mon.0) 3676 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:27:17.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:27:17 smithi100 bash[76202]: cluster 2024-04-07T17:27:16.223194+0000 mgr.y (mgr.25175) 2157 : cluster [DBG] pgmap v3651: 137 pgs: 137 active+clean; 145 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 9.7 MiB/s rd, 4.8 MiB/s wr, 30 op/s 2024-04-07T17:27:17.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:27:17 smithi100 bash[76202]: cluster 2024-04-07T17:27:16.312536+0000 mon.a (mon.0) 3675 : cluster [DBG] osdmap e1920: 8 total, 8 up, 8 in 2024-04-07T17:27:17.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:27:17 smithi100 bash[76202]: audit 2024-04-07T17:27:16.843290+0000 mon.a (mon.0) 3676 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:27:18.704 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:27:18 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:27:18] "GET /metrics HTTP/1.1" 200 36758 "" "Prometheus/2.43.0" 2024-04-07T17:27:19.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:27:19 smithi144 bash[66654]: cluster 2024-04-07T17:27:18.224172+0000 mgr.y (mgr.25175) 2158 : cluster [DBG] pgmap v3653: 137 pgs: 137 active+clean; 153 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 12 MiB/s rd, 6.1 MiB/s wr, 43 op/s 2024-04-07T17:27:19.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:27:19 smithi100 bash[74942]: cluster 2024-04-07T17:27:18.224172+0000 mgr.y (mgr.25175) 2158 : cluster [DBG] pgmap v3653: 137 pgs: 137 active+clean; 153 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 12 MiB/s rd, 6.1 MiB/s wr, 43 op/s 2024-04-07T17:27:19.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:27:19 smithi100 bash[76202]: cluster 2024-04-07T17:27:18.224172+0000 mgr.y (mgr.25175) 2158 : cluster [DBG] pgmap v3653: 137 pgs: 137 active+clean; 153 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 12 MiB/s rd, 6.1 MiB/s wr, 43 op/s 2024-04-07T17:27:21.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:27:21 smithi144 bash[66654]: cluster 2024-04-07T17:27:20.224876+0000 mgr.y (mgr.25175) 2159 : cluster [DBG] pgmap v3654: 137 pgs: 137 active+clean; 174 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 10 MiB/s rd, 5.8 MiB/s wr, 52 op/s 2024-04-07T17:27:21.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:27:21 smithi100 bash[74942]: cluster 2024-04-07T17:27:20.224876+0000 mgr.y (mgr.25175) 2159 : cluster [DBG] pgmap v3654: 137 pgs: 137 active+clean; 174 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 10 MiB/s rd, 5.8 MiB/s wr, 52 op/s 2024-04-07T17:27:21.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:27:21 smithi100 bash[76202]: cluster 2024-04-07T17:27:20.224876+0000 mgr.y (mgr.25175) 2159 : cluster [DBG] pgmap v3654: 137 pgs: 137 active+clean; 174 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 10 MiB/s rd, 5.8 MiB/s wr, 52 op/s 2024-04-07T17:27:22.616 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:27:22 smithi144 bash[95233]: ts=2024-04-07T17:27:22.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:27:23.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:27:23 smithi144 bash[66654]: cluster 2024-04-07T17:27:22.225611+0000 mgr.y (mgr.25175) 2160 : cluster [DBG] pgmap v3655: 137 pgs: 137 active+clean; 174 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 6.4 MiB/s rd, 4.5 MiB/s wr, 43 op/s 2024-04-07T17:27:23.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:27:23 smithi100 bash[74942]: cluster 2024-04-07T17:27:22.225611+0000 mgr.y (mgr.25175) 2160 : cluster [DBG] pgmap v3655: 137 pgs: 137 active+clean; 174 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 6.4 MiB/s rd, 4.5 MiB/s wr, 43 op/s 2024-04-07T17:27:23.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:27:23 smithi100 bash[76202]: cluster 2024-04-07T17:27:22.225611+0000 mgr.y (mgr.25175) 2160 : cluster [DBG] pgmap v3655: 137 pgs: 137 active+clean; 174 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 6.4 MiB/s rd, 4.5 MiB/s wr, 43 op/s 2024-04-07T17:27:25.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:27:25 smithi144 bash[66654]: cluster 2024-04-07T17:27:24.226691+0000 mgr.y (mgr.25175) 2161 : cluster [DBG] pgmap v3656: 137 pgs: 137 active+clean; 174 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 436 KiB/s rd, 2.9 MiB/s wr, 35 op/s 2024-04-07T17:27:25.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:27:25 smithi100 bash[74942]: cluster 2024-04-07T17:27:24.226691+0000 mgr.y (mgr.25175) 2161 : cluster [DBG] pgmap v3656: 137 pgs: 137 active+clean; 174 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 436 KiB/s rd, 2.9 MiB/s wr, 35 op/s 2024-04-07T17:27:25.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:27:25 smithi100 bash[76202]: cluster 2024-04-07T17:27:24.226691+0000 mgr.y (mgr.25175) 2161 : cluster [DBG] pgmap v3656: 137 pgs: 137 active+clean; 174 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 436 KiB/s rd, 2.9 MiB/s wr, 35 op/s 2024-04-07T17:27:27.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:27:27 smithi144 bash[66654]: cluster 2024-04-07T17:27:26.227625+0000 mgr.y (mgr.25175) 2162 : cluster [DBG] pgmap v3657: 137 pgs: 137 active+clean; 174 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 26 KiB/s rd, 2.9 MiB/s wr, 35 op/s 2024-04-07T17:27:27.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:27:27 smithi100 bash[74942]: cluster 2024-04-07T17:27:26.227625+0000 mgr.y (mgr.25175) 2162 : cluster [DBG] pgmap v3657: 137 pgs: 137 active+clean; 174 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 26 KiB/s rd, 2.9 MiB/s wr, 35 op/s 2024-04-07T17:27:27.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:27:27 smithi100 bash[76202]: cluster 2024-04-07T17:27:26.227625+0000 mgr.y (mgr.25175) 2162 : cluster [DBG] pgmap v3657: 137 pgs: 137 active+clean; 174 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 26 KiB/s rd, 2.9 MiB/s wr, 35 op/s 2024-04-07T17:27:28.704 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:27:28 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:27:28] "GET /metrics HTTP/1.1" 200 36758 "" "Prometheus/2.43.0" 2024-04-07T17:27:29.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:27:29 smithi144 bash[66654]: cluster 2024-04-07T17:27:28.228857+0000 mgr.y (mgr.25175) 2163 : cluster [DBG] pgmap v3658: 137 pgs: 137 active+clean; 186 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 74 KiB/s rd, 3.4 MiB/s wr, 35 op/s 2024-04-07T17:27:29.706 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:27:29 smithi100 bash[74942]: cluster 2024-04-07T17:27:28.228857+0000 mgr.y (mgr.25175) 2163 : cluster [DBG] pgmap v3658: 137 pgs: 137 active+clean; 186 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 74 KiB/s rd, 3.4 MiB/s wr, 35 op/s 2024-04-07T17:27:29.706 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:27:29 smithi100 bash[76202]: cluster 2024-04-07T17:27:28.228857+0000 mgr.y (mgr.25175) 2163 : cluster [DBG] pgmap v3658: 137 pgs: 137 active+clean; 186 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 74 KiB/s rd, 3.4 MiB/s wr, 35 op/s 2024-04-07T17:27:30.820 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] EncryptedFlattenTest/4.Shrink (19313 ms) 2024-04-07T17:27:30.821 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] EncryptedFlattenTest/4.ShrinkToOne 2024-04-07T17:27:30.821 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:27:31.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:27:31 smithi144 bash[66654]: cluster 2024-04-07T17:27:30.229605+0000 mgr.y (mgr.25175) 2164 : cluster [DBG] pgmap v3659: 137 pgs: 137 active+clean; 194 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 4.1 MiB/s rd, 3.7 MiB/s wr, 29 op/s 2024-04-07T17:27:31.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:27:31 smithi100 bash[76202]: cluster 2024-04-07T17:27:30.229605+0000 mgr.y (mgr.25175) 2164 : cluster [DBG] pgmap v3659: 137 pgs: 137 active+clean; 194 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 4.1 MiB/s rd, 3.7 MiB/s wr, 29 op/s 2024-04-07T17:27:31.705 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:27:31 smithi100 bash[74942]: cluster 2024-04-07T17:27:30.229605+0000 mgr.y (mgr.25175) 2164 : cluster [DBG] pgmap v3659: 137 pgs: 137 active+clean; 194 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 4.1 MiB/s rd, 3.7 MiB/s wr, 29 op/s 2024-04-07T17:27:32.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:27:32 smithi144 bash[66654]: audit 2024-04-07T17:27:31.842739+0000 mon.a (mon.0) 3677 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:27:32.616 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:27:32 smithi144 bash[95233]: ts=2024-04-07T17:27:32.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:27:32.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:27:32 smithi100 bash[74942]: audit 2024-04-07T17:27:31.842739+0000 mon.a (mon.0) 3677 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:27:32.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:27:32 smithi100 bash[76202]: audit 2024-04-07T17:27:31.842739+0000 mon.a (mon.0) 3677 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:27:33.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:27:33 smithi144 bash[66654]: cluster 2024-04-07T17:27:32.230715+0000 mgr.y (mgr.25175) 2165 : cluster [DBG] pgmap v3660: 137 pgs: 137 active+clean; 211 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 8.7 MiB/s rd, 4.4 MiB/s wr, 21 op/s 2024-04-07T17:27:33.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:27:33 smithi100 bash[74942]: cluster 2024-04-07T17:27:32.230715+0000 mgr.y (mgr.25175) 2165 : cluster [DBG] pgmap v3660: 137 pgs: 137 active+clean; 211 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 8.7 MiB/s rd, 4.4 MiB/s wr, 21 op/s 2024-04-07T17:27:33.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:27:33 smithi100 bash[76202]: cluster 2024-04-07T17:27:32.230715+0000 mgr.y (mgr.25175) 2165 : cluster [DBG] pgmap v3660: 137 pgs: 137 active+clean; 211 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 8.7 MiB/s rd, 4.4 MiB/s wr, 21 op/s 2024-04-07T17:27:35.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:27:35 smithi100 bash[74942]: cluster 2024-04-07T17:27:34.231695+0000 mgr.y (mgr.25175) 2166 : cluster [DBG] pgmap v3661: 137 pgs: 137 active+clean; 215 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 9.7 MiB/s rd, 4.8 MiB/s wr, 30 op/s 2024-04-07T17:27:35.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:27:35 smithi100 bash[76202]: cluster 2024-04-07T17:27:34.231695+0000 mgr.y (mgr.25175) 2166 : cluster [DBG] pgmap v3661: 137 pgs: 137 active+clean; 215 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 9.7 MiB/s rd, 4.8 MiB/s wr, 30 op/s 2024-04-07T17:27:35.866 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:27:35 smithi144 bash[66654]: cluster 2024-04-07T17:27:34.231695+0000 mgr.y (mgr.25175) 2166 : cluster [DBG] pgmap v3661: 137 pgs: 137 active+clean; 215 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 9.7 MiB/s rd, 4.8 MiB/s wr, 30 op/s 2024-04-07T17:27:36.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:27:36 smithi100 bash[74942]: cluster 2024-04-07T17:27:35.363102+0000 mon.a (mon.0) 3678 : cluster [DBG] osdmap e1921: 8 total, 8 up, 8 in 2024-04-07T17:27:36.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:27:36 smithi100 bash[76202]: cluster 2024-04-07T17:27:35.363102+0000 mon.a (mon.0) 3678 : cluster [DBG] osdmap e1921: 8 total, 8 up, 8 in 2024-04-07T17:27:36.866 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:27:36 smithi144 bash[66654]: cluster 2024-04-07T17:27:35.363102+0000 mon.a (mon.0) 3678 : cluster [DBG] osdmap e1921: 8 total, 8 up, 8 in 2024-04-07T17:27:37.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:27:37 smithi100 bash[74942]: cluster 2024-04-07T17:27:36.232371+0000 mgr.y (mgr.25175) 2167 : cluster [DBG] pgmap v3663: 137 pgs: 137 active+clean; 223 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 12 MiB/s rd, 6.5 MiB/s wr, 37 op/s 2024-04-07T17:27:37.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:27:37 smithi100 bash[76202]: cluster 2024-04-07T17:27:36.232371+0000 mgr.y (mgr.25175) 2167 : cluster [DBG] pgmap v3663: 137 pgs: 137 active+clean; 223 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 12 MiB/s rd, 6.5 MiB/s wr, 37 op/s 2024-04-07T17:27:37.866 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:27:37 smithi144 bash[66654]: cluster 2024-04-07T17:27:36.232371+0000 mgr.y (mgr.25175) 2167 : cluster [DBG] pgmap v3663: 137 pgs: 137 active+clean; 223 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 12 MiB/s rd, 6.5 MiB/s wr, 37 op/s 2024-04-07T17:27:38.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:27:38 smithi100 bash[74942]: audit 2024-04-07T17:27:37.540624+0000 mon.a (mon.0) 3679 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:27:38.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:27:38 smithi100 bash[74942]: audit 2024-04-07T17:27:38.222754+0000 mon.a (mon.0) 3680 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:27:38.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:27:38 smithi100 bash[74942]: audit 2024-04-07T17:27:38.223592+0000 mon.a (mon.0) 3681 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:27:38.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:27:38 smithi100 bash[74942]: audit 2024-04-07T17:27:38.229714+0000 mon.a (mon.0) 3682 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:27:38.704 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:27:38 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:27:38] "GET /metrics HTTP/1.1" 200 36757 "" "Prometheus/2.43.0" 2024-04-07T17:27:38.705 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:27:38 smithi100 bash[76202]: audit 2024-04-07T17:27:37.540624+0000 mon.a (mon.0) 3679 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:27:38.705 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:27:38 smithi100 bash[76202]: audit 2024-04-07T17:27:38.222754+0000 mon.a (mon.0) 3680 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:27:38.705 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:27:38 smithi100 bash[76202]: audit 2024-04-07T17:27:38.223592+0000 mon.a (mon.0) 3681 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:27:38.705 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:27:38 smithi100 bash[76202]: audit 2024-04-07T17:27:38.229714+0000 mon.a (mon.0) 3682 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:27:38.866 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:27:38 smithi144 bash[66654]: audit 2024-04-07T17:27:37.540624+0000 mon.a (mon.0) 3679 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:27:38.866 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:27:38 smithi144 bash[66654]: audit 2024-04-07T17:27:38.222754+0000 mon.a (mon.0) 3680 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:27:38.866 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:27:38 smithi144 bash[66654]: audit 2024-04-07T17:27:38.223592+0000 mon.a (mon.0) 3681 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:27:38.866 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:27:38 smithi144 bash[66654]: audit 2024-04-07T17:27:38.229714+0000 mon.a (mon.0) 3682 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:27:39.703 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:27:39 smithi100 bash[74942]: cluster 2024-04-07T17:27:38.233042+0000 mgr.y (mgr.25175) 2168 : cluster [DBG] pgmap v3664: 137 pgs: 137 active+clean; 232 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 12 MiB/s rd, 6.2 MiB/s wr, 61 op/s 2024-04-07T17:27:39.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:27:39 smithi100 bash[76202]: cluster 2024-04-07T17:27:38.233042+0000 mgr.y (mgr.25175) 2168 : cluster [DBG] pgmap v3664: 137 pgs: 137 active+clean; 232 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 12 MiB/s rd, 6.2 MiB/s wr, 61 op/s 2024-04-07T17:27:39.866 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:27:39 smithi144 bash[66654]: cluster 2024-04-07T17:27:38.233042+0000 mgr.y (mgr.25175) 2168 : cluster [DBG] pgmap v3664: 137 pgs: 137 active+clean; 232 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 12 MiB/s rd, 6.2 MiB/s wr, 61 op/s 2024-04-07T17:27:41.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:27:41 smithi100 bash[74942]: cluster 2024-04-07T17:27:40.233742+0000 mgr.y (mgr.25175) 2169 : cluster [DBG] pgmap v3665: 137 pgs: 137 active+clean; 244 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 6.8 MiB/s rd, 6.2 MiB/s wr, 58 op/s 2024-04-07T17:27:41.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:27:41 smithi100 bash[76202]: cluster 2024-04-07T17:27:40.233742+0000 mgr.y (mgr.25175) 2169 : cluster [DBG] pgmap v3665: 137 pgs: 137 active+clean; 244 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 6.8 MiB/s rd, 6.2 MiB/s wr, 58 op/s 2024-04-07T17:27:41.866 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:27:41 smithi144 bash[66654]: cluster 2024-04-07T17:27:40.233742+0000 mgr.y (mgr.25175) 2169 : cluster [DBG] pgmap v3665: 137 pgs: 137 active+clean; 244 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 6.8 MiB/s rd, 6.2 MiB/s wr, 58 op/s 2024-04-07T17:27:42.616 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:27:42 smithi144 bash[95233]: ts=2024-04-07T17:27:42.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:27:43.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:27:43 smithi100 bash[74942]: cluster 2024-04-07T17:27:42.234496+0000 mgr.y (mgr.25175) 2170 : cluster [DBG] pgmap v3666: 137 pgs: 137 active+clean; 244 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 1.2 MiB/s rd, 3.3 MiB/s wr, 46 op/s 2024-04-07T17:27:43.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:27:43 smithi100 bash[76202]: cluster 2024-04-07T17:27:42.234496+0000 mgr.y (mgr.25175) 2170 : cluster [DBG] pgmap v3666: 137 pgs: 137 active+clean; 244 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 1.2 MiB/s rd, 3.3 MiB/s wr, 46 op/s 2024-04-07T17:27:43.866 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:27:43 smithi144 bash[66654]: cluster 2024-04-07T17:27:42.234496+0000 mgr.y (mgr.25175) 2170 : cluster [DBG] pgmap v3666: 137 pgs: 137 active+clean; 244 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 1.2 MiB/s rd, 3.3 MiB/s wr, 46 op/s 2024-04-07T17:27:45.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:27:45 smithi100 bash[74942]: cluster 2024-04-07T17:27:44.235219+0000 mgr.y (mgr.25175) 2171 : cluster [DBG] pgmap v3667: 137 pgs: 137 active+clean; 244 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 26 KiB/s rd, 2.9 MiB/s wr, 35 op/s 2024-04-07T17:27:45.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:27:45 smithi100 bash[76202]: cluster 2024-04-07T17:27:44.235219+0000 mgr.y (mgr.25175) 2171 : cluster [DBG] pgmap v3667: 137 pgs: 137 active+clean; 244 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 26 KiB/s rd, 2.9 MiB/s wr, 35 op/s 2024-04-07T17:27:45.866 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:27:45 smithi144 bash[66654]: cluster 2024-04-07T17:27:44.235219+0000 mgr.y (mgr.25175) 2171 : cluster [DBG] pgmap v3667: 137 pgs: 137 active+clean; 244 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 26 KiB/s rd, 2.9 MiB/s wr, 35 op/s 2024-04-07T17:27:47.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:27:47 smithi100 bash[74942]: cluster 2024-04-07T17:27:46.235853+0000 mgr.y (mgr.25175) 2172 : cluster [DBG] pgmap v3668: 137 pgs: 137 active+clean; 244 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 24 KiB/s rd, 1.9 MiB/s wr, 31 op/s 2024-04-07T17:27:47.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:27:47 smithi100 bash[74942]: audit 2024-04-07T17:27:46.843054+0000 mon.a (mon.0) 3683 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:27:47.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:27:47 smithi100 bash[76202]: cluster 2024-04-07T17:27:46.235853+0000 mgr.y (mgr.25175) 2172 : cluster [DBG] pgmap v3668: 137 pgs: 137 active+clean; 244 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 24 KiB/s rd, 1.9 MiB/s wr, 31 op/s 2024-04-07T17:27:47.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:27:47 smithi100 bash[76202]: audit 2024-04-07T17:27:46.843054+0000 mon.a (mon.0) 3683 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:27:47.866 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:27:47 smithi144 bash[66654]: cluster 2024-04-07T17:27:46.235853+0000 mgr.y (mgr.25175) 2172 : cluster [DBG] pgmap v3668: 137 pgs: 137 active+clean; 244 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 24 KiB/s rd, 1.9 MiB/s wr, 31 op/s 2024-04-07T17:27:47.866 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:27:47 smithi144 bash[66654]: audit 2024-04-07T17:27:46.843054+0000 mon.a (mon.0) 3683 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:27:48.704 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:27:48 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:27:48] "GET /metrics HTTP/1.1" 200 36756 "" "Prometheus/2.43.0" 2024-04-07T17:27:50.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:27:50 smithi144 bash[66654]: cluster 2024-04-07T17:27:48.237026+0000 mgr.y (mgr.25175) 2173 : cluster [DBG] pgmap v3669: 137 pgs: 137 active+clean; 248 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 28 KiB/s rd, 2.1 MiB/s wr, 37 op/s 2024-04-07T17:27:50.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:27:50 smithi100 bash[74942]: cluster 2024-04-07T17:27:48.237026+0000 mgr.y (mgr.25175) 2173 : cluster [DBG] pgmap v3669: 137 pgs: 137 active+clean; 248 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 28 KiB/s rd, 2.1 MiB/s wr, 37 op/s 2024-04-07T17:27:50.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:27:50 smithi100 bash[76202]: cluster 2024-04-07T17:27:48.237026+0000 mgr.y (mgr.25175) 2173 : cluster [DBG] pgmap v3669: 137 pgs: 137 active+clean; 248 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 28 KiB/s rd, 2.1 MiB/s wr, 37 op/s 2024-04-07T17:27:51.856 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] EncryptedFlattenTest/4.ShrinkToOne (21036 ms) 2024-04-07T17:27:51.856 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] EncryptedFlattenTest/4.ShrinkToOneAfterSnapshot 2024-04-07T17:27:51.856 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:27:52.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:27:52 smithi144 bash[66654]: cluster 2024-04-07T17:27:50.237777+0000 mgr.y (mgr.25175) 2174 : cluster [DBG] pgmap v3670: 137 pgs: 137 active+clean; 260 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 762 KiB/s rd, 3.0 MiB/s wr, 12 op/s 2024-04-07T17:27:52.366 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:27:52 smithi144 bash[95233]: ts=2024-04-07T17:27:52.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:27:52.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:27:52 smithi100 bash[74942]: cluster 2024-04-07T17:27:50.237777+0000 mgr.y (mgr.25175) 2174 : cluster [DBG] pgmap v3670: 137 pgs: 137 active+clean; 260 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 762 KiB/s rd, 3.0 MiB/s wr, 12 op/s 2024-04-07T17:27:52.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:27:52 smithi100 bash[76202]: cluster 2024-04-07T17:27:50.237777+0000 mgr.y (mgr.25175) 2174 : cluster [DBG] pgmap v3670: 137 pgs: 137 active+clean; 260 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 762 KiB/s rd, 3.0 MiB/s wr, 12 op/s 2024-04-07T17:27:54.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:27:54 smithi144 bash[66654]: cluster 2024-04-07T17:27:52.238415+0000 mgr.y (mgr.25175) 2175 : cluster [DBG] pgmap v3671: 137 pgs: 137 active+clean; 264 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 1.1 MiB/s rd, 2.7 MiB/s wr, 17 op/s 2024-04-07T17:27:54.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:27:54 smithi100 bash[74942]: cluster 2024-04-07T17:27:52.238415+0000 mgr.y (mgr.25175) 2175 : cluster [DBG] pgmap v3671: 137 pgs: 137 active+clean; 264 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 1.1 MiB/s rd, 2.7 MiB/s wr, 17 op/s 2024-04-07T17:27:54.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:27:54 smithi100 bash[76202]: cluster 2024-04-07T17:27:52.238415+0000 mgr.y (mgr.25175) 2175 : cluster [DBG] pgmap v3671: 137 pgs: 137 active+clean; 264 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 1.1 MiB/s rd, 2.7 MiB/s wr, 17 op/s 2024-04-07T17:27:56.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:27:56 smithi144 bash[66654]: cluster 2024-04-07T17:27:54.239051+0000 mgr.y (mgr.25175) 2176 : cluster [DBG] pgmap v3672: 137 pgs: 137 active+clean; 264 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 1.8 MiB/s rd, 3.3 MiB/s wr, 23 op/s 2024-04-07T17:27:56.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:27:56 smithi100 bash[76202]: cluster 2024-04-07T17:27:54.239051+0000 mgr.y (mgr.25175) 2176 : cluster [DBG] pgmap v3672: 137 pgs: 137 active+clean; 264 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 1.8 MiB/s rd, 3.3 MiB/s wr, 23 op/s 2024-04-07T17:27:56.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:27:56 smithi100 bash[74942]: cluster 2024-04-07T17:27:54.239051+0000 mgr.y (mgr.25175) 2176 : cluster [DBG] pgmap v3672: 137 pgs: 137 active+clean; 264 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 1.8 MiB/s rd, 3.3 MiB/s wr, 23 op/s 2024-04-07T17:27:57.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:27:57 smithi144 bash[66654]: cluster 2024-04-07T17:27:56.094685+0000 mon.a (mon.0) 3684 : cluster [DBG] osdmap e1922: 8 total, 8 up, 8 in 2024-04-07T17:27:57.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:27:57 smithi100 bash[74942]: cluster 2024-04-07T17:27:56.094685+0000 mon.a (mon.0) 3684 : cluster [DBG] osdmap e1922: 8 total, 8 up, 8 in 2024-04-07T17:27:57.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:27:57 smithi100 bash[76202]: cluster 2024-04-07T17:27:56.094685+0000 mon.a (mon.0) 3684 : cluster [DBG] osdmap e1922: 8 total, 8 up, 8 in 2024-04-07T17:27:58.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:27:58 smithi144 bash[66654]: cluster 2024-04-07T17:27:56.239751+0000 mgr.y (mgr.25175) 2177 : cluster [DBG] pgmap v3674: 137 pgs: 137 active+clean; 264 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 2.1 MiB/s rd, 4.0 MiB/s wr, 28 op/s 2024-04-07T17:27:58.426 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:27:58 smithi100 bash[74942]: cluster 2024-04-07T17:27:56.239751+0000 mgr.y (mgr.25175) 2177 : cluster [DBG] pgmap v3674: 137 pgs: 137 active+clean; 264 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 2.1 MiB/s rd, 4.0 MiB/s wr, 28 op/s 2024-04-07T17:27:58.426 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:27:58 smithi100 bash[76202]: cluster 2024-04-07T17:27:56.239751+0000 mgr.y (mgr.25175) 2177 : cluster [DBG] pgmap v3674: 137 pgs: 137 active+clean; 264 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 2.1 MiB/s rd, 4.0 MiB/s wr, 28 op/s 2024-04-07T17:27:58.704 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:27:58 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:27:58] "GET /metrics HTTP/1.1" 200 36756 "" "Prometheus/2.43.0" 2024-04-07T17:27:59.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:27:59 smithi144 bash[66654]: cluster 2024-04-07T17:27:58.240516+0000 mgr.y (mgr.25175) 2178 : cluster [DBG] pgmap v3675: 137 pgs: 137 active+clean; 281 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 2.1 MiB/s rd, 5.3 MiB/s wr, 25 op/s 2024-04-07T17:27:59.453 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:27:59 smithi100 bash[74942]: cluster 2024-04-07T17:27:58.240516+0000 mgr.y (mgr.25175) 2178 : cluster [DBG] pgmap v3675: 137 pgs: 137 active+clean; 281 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 2.1 MiB/s rd, 5.3 MiB/s wr, 25 op/s 2024-04-07T17:27:59.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:27:59 smithi100 bash[76202]: cluster 2024-04-07T17:27:58.240516+0000 mgr.y (mgr.25175) 2178 : cluster [DBG] pgmap v3675: 137 pgs: 137 active+clean; 281 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 2.1 MiB/s rd, 5.3 MiB/s wr, 25 op/s 2024-04-07T17:28:01.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:28:01 smithi144 bash[66654]: cluster 2024-04-07T17:28:00.241245+0000 mgr.y (mgr.25175) 2179 : cluster [DBG] pgmap v3676: 137 pgs: 137 active+clean; 293 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 1.2 MiB/s rd, 4.5 MiB/s wr, 51 op/s 2024-04-07T17:28:01.703 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:28:01 smithi100 bash[74942]: cluster 2024-04-07T17:28:00.241245+0000 mgr.y (mgr.25175) 2179 : cluster [DBG] pgmap v3676: 137 pgs: 137 active+clean; 293 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 1.2 MiB/s rd, 4.5 MiB/s wr, 51 op/s 2024-04-07T17:28:01.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:28:01 smithi100 bash[76202]: cluster 2024-04-07T17:28:00.241245+0000 mgr.y (mgr.25175) 2179 : cluster [DBG] pgmap v3676: 137 pgs: 137 active+clean; 293 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 1.2 MiB/s rd, 4.5 MiB/s wr, 51 op/s 2024-04-07T17:28:02.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:28:02 smithi144 bash[66654]: audit 2024-04-07T17:28:01.843370+0000 mon.a (mon.0) 3685 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:28:02.616 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:28:02 smithi144 bash[95233]: ts=2024-04-07T17:28:02.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:28:02.703 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:28:02 smithi100 bash[74942]: audit 2024-04-07T17:28:01.843370+0000 mon.a (mon.0) 3685 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:28:02.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:28:02 smithi100 bash[76202]: audit 2024-04-07T17:28:01.843370+0000 mon.a (mon.0) 3685 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:28:03.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:28:03 smithi144 bash[66654]: cluster 2024-04-07T17:28:02.241984+0000 mgr.y (mgr.25175) 2180 : cluster [DBG] pgmap v3677: 137 pgs: 137 active+clean; 293 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 850 KiB/s rd, 3.7 MiB/s wr, 42 op/s 2024-04-07T17:28:03.703 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:28:03 smithi100 bash[74942]: cluster 2024-04-07T17:28:02.241984+0000 mgr.y (mgr.25175) 2180 : cluster [DBG] pgmap v3677: 137 pgs: 137 active+clean; 293 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 850 KiB/s rd, 3.7 MiB/s wr, 42 op/s 2024-04-07T17:28:03.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:28:03 smithi100 bash[76202]: cluster 2024-04-07T17:28:02.241984+0000 mgr.y (mgr.25175) 2180 : cluster [DBG] pgmap v3677: 137 pgs: 137 active+clean; 293 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 850 KiB/s rd, 3.7 MiB/s wr, 42 op/s 2024-04-07T17:28:05.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:28:05 smithi144 bash[66654]: cluster 2024-04-07T17:28:04.247250+0000 mgr.y (mgr.25175) 2181 : cluster [DBG] pgmap v3678: 137 pgs: 137 active+clean; 293 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 26 KiB/s rd, 2.9 MiB/s wr, 35 op/s 2024-04-07T17:28:05.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:28:05 smithi100 bash[74942]: cluster 2024-04-07T17:28:04.247250+0000 mgr.y (mgr.25175) 2181 : cluster [DBG] pgmap v3678: 137 pgs: 137 active+clean; 293 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 26 KiB/s rd, 2.9 MiB/s wr, 35 op/s 2024-04-07T17:28:05.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:28:05 smithi100 bash[76202]: cluster 2024-04-07T17:28:04.247250+0000 mgr.y (mgr.25175) 2181 : cluster [DBG] pgmap v3678: 137 pgs: 137 active+clean; 293 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 26 KiB/s rd, 2.9 MiB/s wr, 35 op/s 2024-04-07T17:28:07.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:28:07 smithi144 bash[66654]: cluster 2024-04-07T17:28:06.248196+0000 mgr.y (mgr.25175) 2182 : cluster [DBG] pgmap v3679: 137 pgs: 137 active+clean; 293 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 26 KiB/s rd, 2.9 MiB/s wr, 34 op/s 2024-04-07T17:28:07.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:28:07 smithi100 bash[74942]: cluster 2024-04-07T17:28:06.248196+0000 mgr.y (mgr.25175) 2182 : cluster [DBG] pgmap v3679: 137 pgs: 137 active+clean; 293 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 26 KiB/s rd, 2.9 MiB/s wr, 34 op/s 2024-04-07T17:28:07.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:28:07 smithi100 bash[76202]: cluster 2024-04-07T17:28:06.248196+0000 mgr.y (mgr.25175) 2182 : cluster [DBG] pgmap v3679: 137 pgs: 137 active+clean; 293 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 26 KiB/s rd, 2.9 MiB/s wr, 34 op/s 2024-04-07T17:28:08.704 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:28:08 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:28:08] "GET /metrics HTTP/1.1" 200 36757 "" "Prometheus/2.43.0" 2024-04-07T17:28:09.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:28:09 smithi144 bash[66654]: cluster 2024-04-07T17:28:08.249437+0000 mgr.y (mgr.25175) 2183 : cluster [DBG] pgmap v3680: 137 pgs: 137 active+clean; 305 MiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 50 KiB/s rd, 3.4 MiB/s wr, 35 op/s 2024-04-07T17:28:09.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:28:09 smithi144 bash[66654]: cluster 2024-04-07T17:28:08.327057+0000 mon.a (mon.0) 3686 : cluster [DBG] osdmap e1923: 8 total, 8 up, 8 in 2024-04-07T17:28:09.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:28:09 smithi100 bash[74942]: cluster 2024-04-07T17:28:08.249437+0000 mgr.y (mgr.25175) 2183 : cluster [DBG] pgmap v3680: 137 pgs: 137 active+clean; 305 MiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 50 KiB/s rd, 3.4 MiB/s wr, 35 op/s 2024-04-07T17:28:09.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:28:09 smithi100 bash[74942]: cluster 2024-04-07T17:28:08.327057+0000 mon.a (mon.0) 3686 : cluster [DBG] osdmap e1923: 8 total, 8 up, 8 in 2024-04-07T17:28:09.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:28:09 smithi100 bash[76202]: cluster 2024-04-07T17:28:08.249437+0000 mgr.y (mgr.25175) 2183 : cluster [DBG] pgmap v3680: 137 pgs: 137 active+clean; 305 MiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 50 KiB/s rd, 3.4 MiB/s wr, 35 op/s 2024-04-07T17:28:09.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:28:09 smithi100 bash[76202]: cluster 2024-04-07T17:28:08.327057+0000 mon.a (mon.0) 3686 : cluster [DBG] osdmap e1923: 8 total, 8 up, 8 in 2024-04-07T17:28:10.454 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] EncryptedFlattenTest/4.ShrinkToOneAfterSnapshot (18598 ms) 2024-04-07T17:28:10.454 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] EncryptedFlattenTest/4.MinOverlap 2024-04-07T17:28:10.454 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:28:11.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:28:11 smithi144 bash[66654]: cluster 2024-04-07T17:28:10.250194+0000 mgr.y (mgr.25175) 2184 : cluster [DBG] pgmap v3682: 137 pgs: 137 active+clean; 342 MiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 1.3 MiB/s rd, 4.9 MiB/s wr, 14 op/s 2024-04-07T17:28:11.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:28:11 smithi100 bash[76202]: cluster 2024-04-07T17:28:10.250194+0000 mgr.y (mgr.25175) 2184 : cluster [DBG] pgmap v3682: 137 pgs: 137 active+clean; 342 MiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 1.3 MiB/s rd, 4.9 MiB/s wr, 14 op/s 2024-04-07T17:28:11.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:28:11 smithi100 bash[74942]: cluster 2024-04-07T17:28:10.250194+0000 mgr.y (mgr.25175) 2184 : cluster [DBG] pgmap v3682: 137 pgs: 137 active+clean; 342 MiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 1.3 MiB/s rd, 4.9 MiB/s wr, 14 op/s 2024-04-07T17:28:12.616 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:28:12 smithi144 bash[95233]: ts=2024-04-07T17:28:12.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:28:13.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:28:13 smithi144 bash[66654]: cluster 2024-04-07T17:28:12.251248+0000 mgr.y (mgr.25175) 2185 : cluster [DBG] pgmap v3683: 137 pgs: 137 active+clean; 362 MiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 3.0 MiB/s rd, 6.9 MiB/s wr, 34 op/s 2024-04-07T17:28:13.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:28:13 smithi100 bash[74942]: cluster 2024-04-07T17:28:12.251248+0000 mgr.y (mgr.25175) 2185 : cluster [DBG] pgmap v3683: 137 pgs: 137 active+clean; 362 MiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 3.0 MiB/s rd, 6.9 MiB/s wr, 34 op/s 2024-04-07T17:28:13.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:28:13 smithi100 bash[76202]: cluster 2024-04-07T17:28:12.251248+0000 mgr.y (mgr.25175) 2185 : cluster [DBG] pgmap v3683: 137 pgs: 137 active+clean; 362 MiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 3.0 MiB/s rd, 6.9 MiB/s wr, 34 op/s 2024-04-07T17:28:15.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:28:15 smithi144 bash[66654]: cluster 2024-04-07T17:28:14.252353+0000 mgr.y (mgr.25175) 2186 : cluster [DBG] pgmap v3684: 137 pgs: 137 active+clean; 378 MiB data, 4.7 GiB used, 711 GiB / 715 GiB avail; 4.2 MiB/s rd, 8.5 MiB/s wr, 49 op/s 2024-04-07T17:28:15.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:28:15 smithi100 bash[74942]: cluster 2024-04-07T17:28:14.252353+0000 mgr.y (mgr.25175) 2186 : cluster [DBG] pgmap v3684: 137 pgs: 137 active+clean; 378 MiB data, 4.7 GiB used, 711 GiB / 715 GiB avail; 4.2 MiB/s rd, 8.5 MiB/s wr, 49 op/s 2024-04-07T17:28:15.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:28:15 smithi100 bash[76202]: cluster 2024-04-07T17:28:14.252353+0000 mgr.y (mgr.25175) 2186 : cluster [DBG] pgmap v3684: 137 pgs: 137 active+clean; 378 MiB data, 4.7 GiB used, 711 GiB / 715 GiB avail; 4.2 MiB/s rd, 8.5 MiB/s wr, 49 op/s 2024-04-07T17:28:16.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:28:16 smithi100 bash[74942]: cluster 2024-04-07T17:28:15.351710+0000 mon.a (mon.0) 3687 : cluster [DBG] osdmap e1924: 8 total, 8 up, 8 in 2024-04-07T17:28:16.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:28:16 smithi100 bash[76202]: cluster 2024-04-07T17:28:15.351710+0000 mon.a (mon.0) 3687 : cluster [DBG] osdmap e1924: 8 total, 8 up, 8 in 2024-04-07T17:28:16.866 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:28:16 smithi144 bash[66654]: cluster 2024-04-07T17:28:15.351710+0000 mon.a (mon.0) 3687 : cluster [DBG] osdmap e1924: 8 total, 8 up, 8 in 2024-04-07T17:28:17.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:28:17 smithi100 bash[74942]: cluster 2024-04-07T17:28:16.253035+0000 mgr.y (mgr.25175) 2187 : cluster [DBG] pgmap v3686: 137 pgs: 137 active+clean; 386 MiB data, 4.7 GiB used, 711 GiB / 715 GiB avail; 5.2 MiB/s rd, 10 MiB/s wr, 53 op/s 2024-04-07T17:28:17.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:28:17 smithi100 bash[74942]: audit 2024-04-07T17:28:16.843902+0000 mon.a (mon.0) 3688 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:28:17.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:28:17 smithi100 bash[76202]: cluster 2024-04-07T17:28:16.253035+0000 mgr.y (mgr.25175) 2187 : cluster [DBG] pgmap v3686: 137 pgs: 137 active+clean; 386 MiB data, 4.7 GiB used, 711 GiB / 715 GiB avail; 5.2 MiB/s rd, 10 MiB/s wr, 53 op/s 2024-04-07T17:28:17.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:28:17 smithi100 bash[76202]: audit 2024-04-07T17:28:16.843902+0000 mon.a (mon.0) 3688 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:28:17.866 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:28:17 smithi144 bash[66654]: cluster 2024-04-07T17:28:16.253035+0000 mgr.y (mgr.25175) 2187 : cluster [DBG] pgmap v3686: 137 pgs: 137 active+clean; 386 MiB data, 4.7 GiB used, 711 GiB / 715 GiB avail; 5.2 MiB/s rd, 10 MiB/s wr, 53 op/s 2024-04-07T17:28:17.866 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:28:17 smithi144 bash[66654]: audit 2024-04-07T17:28:16.843902+0000 mon.a (mon.0) 3688 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:28:18.704 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:28:18 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:28:18] "GET /metrics HTTP/1.1" 200 36762 "" "Prometheus/2.43.0" 2024-04-07T17:28:18.866 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 17:28:18 smithi144 bash[98866]: logger=cleanup t=2024-04-07T17:28:18.482262296Z level=info msg="Completed cleanup jobs" duration=63.299005ms 2024-04-07T17:28:19.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:28:19 smithi100 bash[74942]: cluster 2024-04-07T17:28:18.253987+0000 mgr.y (mgr.25175) 2188 : cluster [DBG] pgmap v3687: 137 pgs: 137 active+clean; 399 MiB data, 4.7 GiB used, 710 GiB / 715 GiB avail; 4.2 MiB/s rd, 9.5 MiB/s wr, 61 op/s 2024-04-07T17:28:19.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:28:19 smithi100 bash[76202]: cluster 2024-04-07T17:28:18.253987+0000 mgr.y (mgr.25175) 2188 : cluster [DBG] pgmap v3687: 137 pgs: 137 active+clean; 399 MiB data, 4.7 GiB used, 710 GiB / 715 GiB avail; 4.2 MiB/s rd, 9.5 MiB/s wr, 61 op/s 2024-04-07T17:28:19.866 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:28:19 smithi144 bash[66654]: cluster 2024-04-07T17:28:18.253987+0000 mgr.y (mgr.25175) 2188 : cluster [DBG] pgmap v3687: 137 pgs: 137 active+clean; 399 MiB data, 4.7 GiB used, 710 GiB / 715 GiB avail; 4.2 MiB/s rd, 9.5 MiB/s wr, 61 op/s 2024-04-07T17:28:21.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:28:21 smithi100 bash[74942]: cluster 2024-04-07T17:28:20.254707+0000 mgr.y (mgr.25175) 2189 : cluster [DBG] pgmap v3688: 137 pgs: 137 active+clean; 407 MiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 3.0 MiB/s rd, 6.5 MiB/s wr, 70 op/s 2024-04-07T17:28:21.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:28:21 smithi100 bash[76202]: cluster 2024-04-07T17:28:20.254707+0000 mgr.y (mgr.25175) 2189 : cluster [DBG] pgmap v3688: 137 pgs: 137 active+clean; 407 MiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 3.0 MiB/s rd, 6.5 MiB/s wr, 70 op/s 2024-04-07T17:28:21.866 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:28:21 smithi144 bash[66654]: cluster 2024-04-07T17:28:20.254707+0000 mgr.y (mgr.25175) 2189 : cluster [DBG] pgmap v3688: 137 pgs: 137 active+clean; 407 MiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 3.0 MiB/s rd, 6.5 MiB/s wr, 70 op/s 2024-04-07T17:28:22.616 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:28:22 smithi144 bash[95233]: ts=2024-04-07T17:28:22.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:28:23.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:28:23 smithi100 bash[74942]: cluster 2024-04-07T17:28:22.255463+0000 mgr.y (mgr.25175) 2190 : cluster [DBG] pgmap v3689: 137 pgs: 137 active+clean; 415 MiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 1.2 MiB/s rd, 5.3 MiB/s wr, 54 op/s 2024-04-07T17:28:23.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:28:23 smithi100 bash[76202]: cluster 2024-04-07T17:28:22.255463+0000 mgr.y (mgr.25175) 2190 : cluster [DBG] pgmap v3689: 137 pgs: 137 active+clean; 415 MiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 1.2 MiB/s rd, 5.3 MiB/s wr, 54 op/s 2024-04-07T17:28:23.866 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:28:23 smithi144 bash[66654]: cluster 2024-04-07T17:28:22.255463+0000 mgr.y (mgr.25175) 2190 : cluster [DBG] pgmap v3689: 137 pgs: 137 active+clean; 415 MiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 1.2 MiB/s rd, 5.3 MiB/s wr, 54 op/s 2024-04-07T17:28:25.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:28:25 smithi100 bash[74942]: cluster 2024-04-07T17:28:24.256554+0000 mgr.y (mgr.25175) 2191 : cluster [DBG] pgmap v3690: 137 pgs: 137 active+clean; 419 MiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 32 KiB/s rd, 4.1 MiB/s wr, 43 op/s 2024-04-07T17:28:25.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:28:25 smithi100 bash[76202]: cluster 2024-04-07T17:28:24.256554+0000 mgr.y (mgr.25175) 2191 : cluster [DBG] pgmap v3690: 137 pgs: 137 active+clean; 419 MiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 32 KiB/s rd, 4.1 MiB/s wr, 43 op/s 2024-04-07T17:28:25.866 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:28:25 smithi144 bash[66654]: cluster 2024-04-07T17:28:24.256554+0000 mgr.y (mgr.25175) 2191 : cluster [DBG] pgmap v3690: 137 pgs: 137 active+clean; 419 MiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 32 KiB/s rd, 4.1 MiB/s wr, 43 op/s 2024-04-07T17:28:27.645 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] EncryptedFlattenTest/4.MinOverlap (17191 ms) 2024-04-07T17:28:27.645 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] EncryptedFlattenTest/4.ZeroOverlap 2024-04-07T17:28:27.645 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:28:27.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:28:27 smithi100 bash[74942]: cluster 2024-04-07T17:28:26.257195+0000 mgr.y (mgr.25175) 2192 : cluster [DBG] pgmap v3691: 137 pgs: 137 active+clean; 423 MiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 57 KiB/s rd, 3.4 MiB/s wr, 42 op/s 2024-04-07T17:28:27.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:28:27 smithi100 bash[76202]: cluster 2024-04-07T17:28:26.257195+0000 mgr.y (mgr.25175) 2192 : cluster [DBG] pgmap v3691: 137 pgs: 137 active+clean; 423 MiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 57 KiB/s rd, 3.4 MiB/s wr, 42 op/s 2024-04-07T17:28:27.866 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:28:27 smithi144 bash[66654]: cluster 2024-04-07T17:28:26.257195+0000 mgr.y (mgr.25175) 2192 : cluster [DBG] pgmap v3691: 137 pgs: 137 active+clean; 423 MiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 57 KiB/s rd, 3.4 MiB/s wr, 42 op/s 2024-04-07T17:28:28.704 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:28:28 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:28:28] "GET /metrics HTTP/1.1" 200 36762 "" "Prometheus/2.43.0" 2024-04-07T17:28:29.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:28:29 smithi100 bash[74942]: cluster 2024-04-07T17:28:28.258380+0000 mgr.y (mgr.25175) 2193 : cluster [DBG] pgmap v3692: 137 pgs: 137 active+clean; 427 MiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 1.7 MiB/s rd, 4.4 MiB/s wr, 42 op/s 2024-04-07T17:28:29.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:28:29 smithi100 bash[76202]: cluster 2024-04-07T17:28:28.258380+0000 mgr.y (mgr.25175) 2193 : cluster [DBG] pgmap v3692: 137 pgs: 137 active+clean; 427 MiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 1.7 MiB/s rd, 4.4 MiB/s wr, 42 op/s 2024-04-07T17:28:29.866 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:28:29 smithi144 bash[66654]: cluster 2024-04-07T17:28:28.258380+0000 mgr.y (mgr.25175) 2193 : cluster [DBG] pgmap v3692: 137 pgs: 137 active+clean; 427 MiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 1.7 MiB/s rd, 4.4 MiB/s wr, 42 op/s 2024-04-07T17:28:31.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:28:31 smithi100 bash[74942]: cluster 2024-04-07T17:28:30.259399+0000 mgr.y (mgr.25175) 2194 : cluster [DBG] pgmap v3693: 137 pgs: 137 active+clean; 427 MiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 2.8 MiB/s rd, 4.0 MiB/s wr, 35 op/s 2024-04-07T17:28:31.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:28:31 smithi100 bash[76202]: cluster 2024-04-07T17:28:30.259399+0000 mgr.y (mgr.25175) 2194 : cluster [DBG] pgmap v3693: 137 pgs: 137 active+clean; 427 MiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 2.8 MiB/s rd, 4.0 MiB/s wr, 35 op/s 2024-04-07T17:28:31.866 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:28:31 smithi144 bash[66654]: cluster 2024-04-07T17:28:30.259399+0000 mgr.y (mgr.25175) 2194 : cluster [DBG] pgmap v3693: 137 pgs: 137 active+clean; 427 MiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 2.8 MiB/s rd, 4.0 MiB/s wr, 35 op/s 2024-04-07T17:28:32.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:28:32 smithi144 bash[66654]: audit 2024-04-07T17:28:31.844903+0000 mon.a (mon.0) 3689 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:28:32.616 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:28:32 smithi144 bash[95233]: ts=2024-04-07T17:28:32.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:28:32.703 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:28:32 smithi100 bash[74942]: audit 2024-04-07T17:28:31.844903+0000 mon.a (mon.0) 3689 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:28:32.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:28:32 smithi100 bash[76202]: audit 2024-04-07T17:28:31.844903+0000 mon.a (mon.0) 3689 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:28:33.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:28:33 smithi100 bash[74942]: cluster 2024-04-07T17:28:32.260559+0000 mgr.y (mgr.25175) 2195 : cluster [DBG] pgmap v3694: 137 pgs: 137 active+clean; 436 MiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 2.8 MiB/s rd, 4.1 MiB/s wr, 34 op/s 2024-04-07T17:28:33.705 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:28:33 smithi100 bash[74942]: cluster 2024-04-07T17:28:32.410978+0000 mon.a (mon.0) 3690 : cluster [DBG] osdmap e1925: 8 total, 8 up, 8 in 2024-04-07T17:28:33.705 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:28:33 smithi100 bash[76202]: cluster 2024-04-07T17:28:32.260559+0000 mgr.y (mgr.25175) 2195 : cluster [DBG] pgmap v3694: 137 pgs: 137 active+clean; 436 MiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 2.8 MiB/s rd, 4.1 MiB/s wr, 34 op/s 2024-04-07T17:28:33.705 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:28:33 smithi100 bash[76202]: cluster 2024-04-07T17:28:32.410978+0000 mon.a (mon.0) 3690 : cluster [DBG] osdmap e1925: 8 total, 8 up, 8 in 2024-04-07T17:28:33.866 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:28:33 smithi144 bash[66654]: cluster 2024-04-07T17:28:32.260559+0000 mgr.y (mgr.25175) 2195 : cluster [DBG] pgmap v3694: 137 pgs: 137 active+clean; 436 MiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 2.8 MiB/s rd, 4.1 MiB/s wr, 34 op/s 2024-04-07T17:28:33.866 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:28:33 smithi144 bash[66654]: cluster 2024-04-07T17:28:32.410978+0000 mon.a (mon.0) 3690 : cluster [DBG] osdmap e1925: 8 total, 8 up, 8 in 2024-04-07T17:28:35.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:28:35 smithi100 bash[74942]: cluster 2024-04-07T17:28:34.261416+0000 mgr.y (mgr.25175) 2196 : cluster [DBG] pgmap v3696: 137 pgs: 137 active+clean; 452 MiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 3.3 MiB/s rd, 5.3 MiB/s wr, 34 op/s 2024-04-07T17:28:35.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:28:35 smithi100 bash[76202]: cluster 2024-04-07T17:28:34.261416+0000 mgr.y (mgr.25175) 2196 : cluster [DBG] pgmap v3696: 137 pgs: 137 active+clean; 452 MiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 3.3 MiB/s rd, 5.3 MiB/s wr, 34 op/s 2024-04-07T17:28:35.866 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:28:35 smithi144 bash[66654]: cluster 2024-04-07T17:28:34.261416+0000 mgr.y (mgr.25175) 2196 : cluster [DBG] pgmap v3696: 137 pgs: 137 active+clean; 452 MiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 3.3 MiB/s rd, 5.3 MiB/s wr, 34 op/s 2024-04-07T17:28:37.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:28:37 smithi100 bash[74942]: cluster 2024-04-07T17:28:36.262032+0000 mgr.y (mgr.25175) 2197 : cluster [DBG] pgmap v3697: 137 pgs: 137 active+clean; 456 MiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 3.3 MiB/s rd, 5.3 MiB/s wr, 37 op/s 2024-04-07T17:28:37.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:28:37 smithi100 bash[76202]: cluster 2024-04-07T17:28:36.262032+0000 mgr.y (mgr.25175) 2197 : cluster [DBG] pgmap v3697: 137 pgs: 137 active+clean; 456 MiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 3.3 MiB/s rd, 5.3 MiB/s wr, 37 op/s 2024-04-07T17:28:37.866 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:28:37 smithi144 bash[66654]: cluster 2024-04-07T17:28:36.262032+0000 mgr.y (mgr.25175) 2197 : cluster [DBG] pgmap v3697: 137 pgs: 137 active+clean; 456 MiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 3.3 MiB/s rd, 5.3 MiB/s wr, 37 op/s 2024-04-07T17:28:38.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:28:38 smithi100 bash[74942]: audit 2024-04-07T17:28:38.294079+0000 mon.a (mon.0) 3691 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:28:38.704 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:28:38 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:28:38] "GET /metrics HTTP/1.1" 200 36763 "" "Prometheus/2.43.0" 2024-04-07T17:28:38.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:28:38 smithi100 bash[76202]: audit 2024-04-07T17:28:38.294079+0000 mon.a (mon.0) 3691 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:28:38.866 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:28:38 smithi144 bash[66654]: audit 2024-04-07T17:28:38.294079+0000 mon.a (mon.0) 3691 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:28:39.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:28:39 smithi100 bash[74942]: cluster 2024-04-07T17:28:38.262911+0000 mgr.y (mgr.25175) 2198 : cluster [DBG] pgmap v3698: 137 pgs: 137 active+clean; 456 MiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 1.3 MiB/s rd, 3.7 MiB/s wr, 49 op/s 2024-04-07T17:28:39.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:28:39 smithi100 bash[76202]: cluster 2024-04-07T17:28:38.262911+0000 mgr.y (mgr.25175) 2198 : cluster [DBG] pgmap v3698: 137 pgs: 137 active+clean; 456 MiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 1.3 MiB/s rd, 3.7 MiB/s wr, 49 op/s 2024-04-07T17:28:39.866 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:28:39 smithi144 bash[66654]: cluster 2024-04-07T17:28:38.262911+0000 mgr.y (mgr.25175) 2198 : cluster [DBG] pgmap v3698: 137 pgs: 137 active+clean; 456 MiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 1.3 MiB/s rd, 3.7 MiB/s wr, 49 op/s 2024-04-07T17:28:40.866 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:28:40 smithi144 bash[66654]: audit 2024-04-07T17:28:39.487872+0000 mon.a (mon.0) 3692 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:28:40.866 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:28:40 smithi144 bash[66654]: audit 2024-04-07T17:28:39.496763+0000 mon.a (mon.0) 3693 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:28:40.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:28:40 smithi100 bash[74942]: audit 2024-04-07T17:28:39.487872+0000 mon.a (mon.0) 3692 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:28:40.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:28:40 smithi100 bash[74942]: audit 2024-04-07T17:28:39.496763+0000 mon.a (mon.0) 3693 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:28:40.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:28:40 smithi100 bash[76202]: audit 2024-04-07T17:28:39.487872+0000 mon.a (mon.0) 3692 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:28:40.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:28:40 smithi100 bash[76202]: audit 2024-04-07T17:28:39.496763+0000 mon.a (mon.0) 3693 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:28:41.866 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:28:41 smithi144 bash[66654]: cluster 2024-04-07T17:28:40.263660+0000 mgr.y (mgr.25175) 2199 : cluster [DBG] pgmap v3699: 137 pgs: 137 active+clean; 456 MiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 29 KiB/s rd, 2.9 MiB/s wr, 39 op/s 2024-04-07T17:28:41.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:28:41 smithi100 bash[76202]: cluster 2024-04-07T17:28:40.263660+0000 mgr.y (mgr.25175) 2199 : cluster [DBG] pgmap v3699: 137 pgs: 137 active+clean; 456 MiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 29 KiB/s rd, 2.9 MiB/s wr, 39 op/s 2024-04-07T17:28:41.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:28:41 smithi100 bash[74942]: cluster 2024-04-07T17:28:40.263660+0000 mgr.y (mgr.25175) 2199 : cluster [DBG] pgmap v3699: 137 pgs: 137 active+clean; 456 MiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 29 KiB/s rd, 2.9 MiB/s wr, 39 op/s 2024-04-07T17:28:42.616 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:28:42 smithi144 bash[95233]: ts=2024-04-07T17:28:42.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:28:43.866 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:28:43 smithi144 bash[66654]: cluster 2024-04-07T17:28:42.264405+0000 mgr.y (mgr.25175) 2200 : cluster [DBG] pgmap v3700: 137 pgs: 137 active+clean; 456 MiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 19 KiB/s rd, 2.0 MiB/s wr, 24 op/s 2024-04-07T17:28:43.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:28:43 smithi100 bash[74942]: cluster 2024-04-07T17:28:42.264405+0000 mgr.y (mgr.25175) 2200 : cluster [DBG] pgmap v3700: 137 pgs: 137 active+clean; 456 MiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 19 KiB/s rd, 2.0 MiB/s wr, 24 op/s 2024-04-07T17:28:43.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:28:43 smithi100 bash[76202]: cluster 2024-04-07T17:28:42.264405+0000 mgr.y (mgr.25175) 2200 : cluster [DBG] pgmap v3700: 137 pgs: 137 active+clean; 456 MiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 19 KiB/s rd, 2.0 MiB/s wr, 24 op/s 2024-04-07T17:28:45.758 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:28:45 smithi100 bash[74942]: cluster 2024-04-07T17:28:44.265133+0000 mgr.y (mgr.25175) 2201 : cluster [DBG] pgmap v3701: 137 pgs: 137 active+clean; 456 MiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 16 KiB/s rd, 1.7 MiB/s wr, 20 op/s 2024-04-07T17:28:45.759 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:28:45 smithi100 bash[76202]: cluster 2024-04-07T17:28:44.265133+0000 mgr.y (mgr.25175) 2201 : cluster [DBG] pgmap v3701: 137 pgs: 137 active+clean; 456 MiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 16 KiB/s rd, 1.7 MiB/s wr, 20 op/s 2024-04-07T17:28:45.866 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:28:45 smithi144 bash[66654]: cluster 2024-04-07T17:28:44.265133+0000 mgr.y (mgr.25175) 2201 : cluster [DBG] pgmap v3701: 137 pgs: 137 active+clean; 456 MiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 16 KiB/s rd, 1.7 MiB/s wr, 20 op/s 2024-04-07T17:28:47.866 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:28:47 smithi144 bash[66654]: cluster 2024-04-07T17:28:46.265724+0000 mgr.y (mgr.25175) 2202 : cluster [DBG] pgmap v3702: 137 pgs: 137 active+clean; 456 MiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 15 KiB/s rd, 342 KiB/s wr, 19 op/s 2024-04-07T17:28:47.866 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:28:47 smithi144 bash[66654]: audit 2024-04-07T17:28:46.844153+0000 mon.a (mon.0) 3694 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:28:47.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:28:47 smithi100 bash[74942]: cluster 2024-04-07T17:28:46.265724+0000 mgr.y (mgr.25175) 2202 : cluster [DBG] pgmap v3702: 137 pgs: 137 active+clean; 456 MiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 15 KiB/s rd, 342 KiB/s wr, 19 op/s 2024-04-07T17:28:47.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:28:47 smithi100 bash[74942]: audit 2024-04-07T17:28:46.844153+0000 mon.a (mon.0) 3694 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:28:47.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:28:47 smithi100 bash[76202]: cluster 2024-04-07T17:28:46.265724+0000 mgr.y (mgr.25175) 2202 : cluster [DBG] pgmap v3702: 137 pgs: 137 active+clean; 456 MiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 15 KiB/s rd, 342 KiB/s wr, 19 op/s 2024-04-07T17:28:47.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:28:47 smithi100 bash[76202]: audit 2024-04-07T17:28:46.844153+0000 mon.a (mon.0) 3694 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:28:48.704 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:28:48 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:28:48] "GET /metrics HTTP/1.1" 200 36761 "" "Prometheus/2.43.0" 2024-04-07T17:28:50.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:28:49 smithi100 bash[76202]: cluster 2024-04-07T17:28:48.266386+0000 mgr.y (mgr.25175) 2203 : cluster [DBG] pgmap v3703: 137 pgs: 137 active+clean; 460 MiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 37 KiB/s rd, 342 KiB/s wr, 17 op/s 2024-04-07T17:28:50.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:28:49 smithi100 bash[76202]: audit 2024-04-07T17:28:48.936373+0000 mon.a (mon.0) 3695 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:28:50.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:28:49 smithi100 bash[76202]: audit 2024-04-07T17:28:48.944375+0000 mon.a (mon.0) 3696 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:28:50.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:28:49 smithi100 bash[76202]: audit 2024-04-07T17:28:48.945939+0000 mon.a (mon.0) 3697 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:28:50.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:28:49 smithi100 bash[76202]: audit 2024-04-07T17:28:48.947338+0000 mon.a (mon.0) 3698 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:28:50.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:28:49 smithi100 bash[76202]: audit 2024-04-07T17:28:48.956191+0000 mon.a (mon.0) 3699 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:28:50.205 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:28:49 smithi100 bash[74942]: cluster 2024-04-07T17:28:48.266386+0000 mgr.y (mgr.25175) 2203 : cluster [DBG] pgmap v3703: 137 pgs: 137 active+clean; 460 MiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 37 KiB/s rd, 342 KiB/s wr, 17 op/s 2024-04-07T17:28:50.205 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:28:49 smithi100 bash[74942]: audit 2024-04-07T17:28:48.936373+0000 mon.a (mon.0) 3695 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:28:50.205 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:28:49 smithi100 bash[74942]: audit 2024-04-07T17:28:48.944375+0000 mon.a (mon.0) 3696 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:28:50.205 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:28:49 smithi100 bash[74942]: audit 2024-04-07T17:28:48.945939+0000 mon.a (mon.0) 3697 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:28:50.205 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:28:49 smithi100 bash[74942]: audit 2024-04-07T17:28:48.947338+0000 mon.a (mon.0) 3698 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:28:50.205 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:28:49 smithi100 bash[74942]: audit 2024-04-07T17:28:48.956191+0000 mon.a (mon.0) 3699 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:28:50.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:28:49 smithi144 bash[66654]: cluster 2024-04-07T17:28:48.266386+0000 mgr.y (mgr.25175) 2203 : cluster [DBG] pgmap v3703: 137 pgs: 137 active+clean; 460 MiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 37 KiB/s rd, 342 KiB/s wr, 17 op/s 2024-04-07T17:28:50.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:28:49 smithi144 bash[66654]: audit 2024-04-07T17:28:48.936373+0000 mon.a (mon.0) 3695 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:28:50.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:28:49 smithi144 bash[66654]: audit 2024-04-07T17:28:48.944375+0000 mon.a (mon.0) 3696 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:28:50.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:28:49 smithi144 bash[66654]: audit 2024-04-07T17:28:48.945939+0000 mon.a (mon.0) 3697 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:28:50.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:28:49 smithi144 bash[66654]: audit 2024-04-07T17:28:48.947338+0000 mon.a (mon.0) 3698 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:28:50.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:28:49 smithi144 bash[66654]: audit 2024-04-07T17:28:48.956191+0000 mon.a (mon.0) 3699 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:28:52.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:28:51 smithi100 bash[74942]: cluster 2024-04-07T17:28:50.266981+0000 mgr.y (mgr.25175) 2204 : cluster [DBG] pgmap v3704: 137 pgs: 137 active+clean; 472 MiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 30 KiB/s rd, 1.3 MiB/s wr, 9 op/s 2024-04-07T17:28:52.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:28:51 smithi100 bash[76202]: cluster 2024-04-07T17:28:50.266981+0000 mgr.y (mgr.25175) 2204 : cluster [DBG] pgmap v3704: 137 pgs: 137 active+clean; 472 MiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 30 KiB/s rd, 1.3 MiB/s wr, 9 op/s 2024-04-07T17:28:52.316 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:28:51 smithi144 bash[66654]: cluster 2024-04-07T17:28:50.266981+0000 mgr.y (mgr.25175) 2204 : cluster [DBG] pgmap v3704: 137 pgs: 137 active+clean; 472 MiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 30 KiB/s rd, 1.3 MiB/s wr, 9 op/s 2024-04-07T17:28:52.543 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] EncryptedFlattenTest/4.ZeroOverlap (24898 ms) 2024-04-07T17:28:52.616 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:28:52 smithi144 bash[95233]: ts=2024-04-07T17:28:52.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:28:52.974 INFO:tasks.workunit.client.0.smithi100.stdout:[----------] 7 tests from EncryptedFlattenTest/4 (142443 ms total) 2024-04-07T17:28:52.974 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:28:52.974 INFO:tasks.workunit.client.0.smithi100.stdout:[----------] 7 tests from EncryptedFlattenTest/5, where TypeParam = UnformattedLUKS2 2024-04-07T17:28:53.035 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] EncryptedFlattenTest/5.Simple 2024-04-07T17:28:54.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:28:53 smithi144 bash[66654]: cluster 2024-04-07T17:28:52.267732+0000 mgr.y (mgr.25175) 2205 : cluster [DBG] pgmap v3705: 137 pgs: 137 active+clean; 472 MiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 420 KiB/s rd, 1.7 MiB/s wr, 10 op/s 2024-04-07T17:28:54.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:28:53 smithi144 bash[66654]: cluster 2024-04-07T17:28:52.955349+0000 mon.a (mon.0) 3700 : cluster [DBG] osdmap e1926: 8 total, 8 up, 8 in 2024-04-07T17:28:54.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:28:53 smithi100 bash[74942]: cluster 2024-04-07T17:28:52.267732+0000 mgr.y (mgr.25175) 2205 : cluster [DBG] pgmap v3705: 137 pgs: 137 active+clean; 472 MiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 420 KiB/s rd, 1.7 MiB/s wr, 10 op/s 2024-04-07T17:28:54.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:28:53 smithi100 bash[74942]: cluster 2024-04-07T17:28:52.955349+0000 mon.a (mon.0) 3700 : cluster [DBG] osdmap e1926: 8 total, 8 up, 8 in 2024-04-07T17:28:54.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:28:53 smithi100 bash[76202]: cluster 2024-04-07T17:28:52.267732+0000 mgr.y (mgr.25175) 2205 : cluster [DBG] pgmap v3705: 137 pgs: 137 active+clean; 472 MiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 420 KiB/s rd, 1.7 MiB/s wr, 10 op/s 2024-04-07T17:28:54.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:28:53 smithi100 bash[76202]: cluster 2024-04-07T17:28:52.955349+0000 mon.a (mon.0) 3700 : cluster [DBG] osdmap e1926: 8 total, 8 up, 8 in 2024-04-07T17:28:54.991 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:28:55.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:28:54 smithi144 bash[66654]: cluster 2024-04-07T17:28:53.960562+0000 mon.a (mon.0) 3701 : cluster [DBG] osdmap e1927: 8 total, 8 up, 8 in 2024-04-07T17:28:55.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:28:54 smithi144 bash[66654]: audit 2024-04-07T17:28:53.964289+0000 mon.a (mon.0) 3702 : audit [INF] from='client.? 172.21.15.100:0/2066184458' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:28:55.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:28:54 smithi144 bash[66654]: audit 2024-04-07T17:28:54.955505+0000 mon.a (mon.0) 3703 : audit [INF] from='client.? 172.21.15.100:0/2066184458' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T17:28:55.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:28:54 smithi144 bash[66654]: cluster 2024-04-07T17:28:54.968457+0000 mon.a (mon.0) 3704 : cluster [DBG] osdmap e1928: 8 total, 8 up, 8 in 2024-04-07T17:28:55.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:28:54 smithi100 bash[74942]: cluster 2024-04-07T17:28:53.960562+0000 mon.a (mon.0) 3701 : cluster [DBG] osdmap e1927: 8 total, 8 up, 8 in 2024-04-07T17:28:55.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:28:54 smithi100 bash[74942]: audit 2024-04-07T17:28:53.964289+0000 mon.a (mon.0) 3702 : audit [INF] from='client.? 172.21.15.100:0/2066184458' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:28:55.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:28:54 smithi100 bash[74942]: audit 2024-04-07T17:28:54.955505+0000 mon.a (mon.0) 3703 : audit [INF] from='client.? 172.21.15.100:0/2066184458' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T17:28:55.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:28:54 smithi100 bash[74942]: cluster 2024-04-07T17:28:54.968457+0000 mon.a (mon.0) 3704 : cluster [DBG] osdmap e1928: 8 total, 8 up, 8 in 2024-04-07T17:28:55.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:28:54 smithi100 bash[76202]: cluster 2024-04-07T17:28:53.960562+0000 mon.a (mon.0) 3701 : cluster [DBG] osdmap e1927: 8 total, 8 up, 8 in 2024-04-07T17:28:55.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:28:54 smithi100 bash[76202]: audit 2024-04-07T17:28:53.964289+0000 mon.a (mon.0) 3702 : audit [INF] from='client.? 172.21.15.100:0/2066184458' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:28:55.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:28:54 smithi100 bash[76202]: audit 2024-04-07T17:28:54.955505+0000 mon.a (mon.0) 3703 : audit [INF] from='client.? 172.21.15.100:0/2066184458' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T17:28:55.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:28:54 smithi100 bash[76202]: cluster 2024-04-07T17:28:54.968457+0000 mon.a (mon.0) 3704 : cluster [DBG] osdmap e1928: 8 total, 8 up, 8 in 2024-04-07T17:28:56.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:28:55 smithi144 bash[66654]: cluster 2024-04-07T17:28:54.268564+0000 mgr.y (mgr.25175) 2206 : cluster [DBG] pgmap v3708: 137 pgs: 4 creating+peering, 28 unknown, 105 active+clean; 4.6 MiB data, 4.7 GiB used, 710 GiB / 715 GiB avail 2024-04-07T17:28:56.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:28:55 smithi144 bash[66654]: cluster 2024-04-07T17:28:54.971571+0000 mon.a (mon.0) 3705 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T17:28:56.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:28:55 smithi100 bash[74942]: cluster 2024-04-07T17:28:54.268564+0000 mgr.y (mgr.25175) 2206 : cluster [DBG] pgmap v3708: 137 pgs: 4 creating+peering, 28 unknown, 105 active+clean; 4.6 MiB data, 4.7 GiB used, 710 GiB / 715 GiB avail 2024-04-07T17:28:56.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:28:55 smithi100 bash[74942]: cluster 2024-04-07T17:28:54.971571+0000 mon.a (mon.0) 3705 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T17:28:56.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:28:55 smithi100 bash[76202]: cluster 2024-04-07T17:28:54.268564+0000 mgr.y (mgr.25175) 2206 : cluster [DBG] pgmap v3708: 137 pgs: 4 creating+peering, 28 unknown, 105 active+clean; 4.6 MiB data, 4.7 GiB used, 710 GiB / 715 GiB avail 2024-04-07T17:28:56.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:28:55 smithi100 bash[76202]: cluster 2024-04-07T17:28:54.971571+0000 mon.a (mon.0) 3705 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T17:28:57.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:28:57 smithi144 bash[66654]: cluster 2024-04-07T17:28:56.000376+0000 mon.a (mon.0) 3706 : cluster [DBG] osdmap e1929: 8 total, 8 up, 8 in 2024-04-07T17:28:57.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:28:57 smithi100 bash[74942]: cluster 2024-04-07T17:28:56.000376+0000 mon.a (mon.0) 3706 : cluster [DBG] osdmap e1929: 8 total, 8 up, 8 in 2024-04-07T17:28:57.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:28:57 smithi100 bash[76202]: cluster 2024-04-07T17:28:56.000376+0000 mon.a (mon.0) 3706 : cluster [DBG] osdmap e1929: 8 total, 8 up, 8 in 2024-04-07T17:28:58.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:28:58 smithi144 bash[66654]: cluster 2024-04-07T17:28:56.269274+0000 mgr.y (mgr.25175) 2207 : cluster [DBG] pgmap v3711: 137 pgs: 9 creating+peering, 23 unknown, 105 active+clean; 4.6 MiB data, 4.6 GiB used, 711 GiB / 715 GiB avail 2024-04-07T17:28:58.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:28:58 smithi144 bash[66654]: cluster 2024-04-07T17:28:56.993484+0000 mon.a (mon.0) 3707 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T17:28:58.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:28:58 smithi144 bash[66654]: cluster 2024-04-07T17:28:57.011408+0000 mon.a (mon.0) 3708 : cluster [DBG] osdmap e1930: 8 total, 8 up, 8 in 2024-04-07T17:28:58.426 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:28:58 smithi100 bash[74942]: cluster 2024-04-07T17:28:56.269274+0000 mgr.y (mgr.25175) 2207 : cluster [DBG] pgmap v3711: 137 pgs: 9 creating+peering, 23 unknown, 105 active+clean; 4.6 MiB data, 4.6 GiB used, 711 GiB / 715 GiB avail 2024-04-07T17:28:58.426 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:28:58 smithi100 bash[74942]: cluster 2024-04-07T17:28:56.993484+0000 mon.a (mon.0) 3707 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T17:28:58.426 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:28:58 smithi100 bash[74942]: cluster 2024-04-07T17:28:57.011408+0000 mon.a (mon.0) 3708 : cluster [DBG] osdmap e1930: 8 total, 8 up, 8 in 2024-04-07T17:28:58.426 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:28:58 smithi100 bash[76202]: cluster 2024-04-07T17:28:56.269274+0000 mgr.y (mgr.25175) 2207 : cluster [DBG] pgmap v3711: 137 pgs: 9 creating+peering, 23 unknown, 105 active+clean; 4.6 MiB data, 4.6 GiB used, 711 GiB / 715 GiB avail 2024-04-07T17:28:58.426 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:28:58 smithi100 bash[76202]: cluster 2024-04-07T17:28:56.993484+0000 mon.a (mon.0) 3707 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T17:28:58.426 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:28:58 smithi100 bash[76202]: cluster 2024-04-07T17:28:57.011408+0000 mon.a (mon.0) 3708 : cluster [DBG] osdmap e1930: 8 total, 8 up, 8 in 2024-04-07T17:28:58.704 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:28:58 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:28:58] "GET /metrics HTTP/1.1" 200 36761 "" "Prometheus/2.43.0" 2024-04-07T17:29:00.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:29:00 smithi144 bash[66654]: cluster 2024-04-07T17:28:58.269913+0000 mgr.y (mgr.25175) 2208 : cluster [DBG] pgmap v3713: 137 pgs: 9 creating+peering, 4 unknown, 124 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 6.0 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-04-07T17:29:00.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:29:00 smithi100 bash[74942]: cluster 2024-04-07T17:28:58.269913+0000 mgr.y (mgr.25175) 2208 : cluster [DBG] pgmap v3713: 137 pgs: 9 creating+peering, 4 unknown, 124 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 6.0 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-04-07T17:29:00.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:29:00 smithi100 bash[76202]: cluster 2024-04-07T17:28:58.269913+0000 mgr.y (mgr.25175) 2208 : cluster [DBG] pgmap v3713: 137 pgs: 9 creating+peering, 4 unknown, 124 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 6.0 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-04-07T17:29:02.316 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:29:02 smithi144 bash[66654]: cluster 2024-04-07T17:29:00.271081+0000 mgr.y (mgr.25175) 2209 : cluster [DBG] pgmap v3714: 137 pgs: 137 active+clean; 4.6 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 4.8 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-04-07T17:29:02.316 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:29:02 smithi144 bash[66654]: audit 2024-04-07T17:29:01.844706+0000 mon.a (mon.0) 3709 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:29:02.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:29:02 smithi100 bash[74942]: cluster 2024-04-07T17:29:00.271081+0000 mgr.y (mgr.25175) 2209 : cluster [DBG] pgmap v3714: 137 pgs: 137 active+clean; 4.6 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 4.8 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-04-07T17:29:02.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:29:02 smithi100 bash[74942]: audit 2024-04-07T17:29:01.844706+0000 mon.a (mon.0) 3709 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:29:02.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:29:02 smithi100 bash[76202]: cluster 2024-04-07T17:29:00.271081+0000 mgr.y (mgr.25175) 2209 : cluster [DBG] pgmap v3714: 137 pgs: 137 active+clean; 4.6 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 4.8 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-04-07T17:29:02.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:29:02 smithi100 bash[76202]: audit 2024-04-07T17:29:01.844706+0000 mon.a (mon.0) 3709 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:29:02.616 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:29:02 smithi144 bash[95233]: ts=2024-04-07T17:29:02.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:29:04.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:29:04 smithi144 bash[66654]: cluster 2024-04-07T17:29:02.271905+0000 mgr.y (mgr.25175) 2210 : cluster [DBG] pgmap v3715: 137 pgs: 137 active+clean; 4.6 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 4.1 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-04-07T17:29:04.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:29:04 smithi144 bash[66654]: cluster 2024-04-07T17:29:03.088068+0000 mon.a (mon.0) 3710 : cluster [DBG] osdmap e1931: 8 total, 8 up, 8 in 2024-04-07T17:29:04.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:29:04 smithi100 bash[74942]: cluster 2024-04-07T17:29:02.271905+0000 mgr.y (mgr.25175) 2210 : cluster [DBG] pgmap v3715: 137 pgs: 137 active+clean; 4.6 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 4.1 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-04-07T17:29:04.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:29:04 smithi100 bash[74942]: cluster 2024-04-07T17:29:03.088068+0000 mon.a (mon.0) 3710 : cluster [DBG] osdmap e1931: 8 total, 8 up, 8 in 2024-04-07T17:29:04.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:29:04 smithi100 bash[76202]: cluster 2024-04-07T17:29:02.271905+0000 mgr.y (mgr.25175) 2210 : cluster [DBG] pgmap v3715: 137 pgs: 137 active+clean; 4.6 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 4.1 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-04-07T17:29:04.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:29:04 smithi100 bash[76202]: cluster 2024-04-07T17:29:03.088068+0000 mon.a (mon.0) 3710 : cluster [DBG] osdmap e1931: 8 total, 8 up, 8 in 2024-04-07T17:29:06.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:29:06 smithi144 bash[66654]: cluster 2024-04-07T17:29:04.272557+0000 mgr.y (mgr.25175) 2211 : cluster [DBG] pgmap v3717: 137 pgs: 137 active+clean; 4.6 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 3.7 KiB/s rd, 1023 B/s wr, 6 op/s 2024-04-07T17:29:06.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:29:06 smithi100 bash[74942]: cluster 2024-04-07T17:29:04.272557+0000 mgr.y (mgr.25175) 2211 : cluster [DBG] pgmap v3717: 137 pgs: 137 active+clean; 4.6 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 3.7 KiB/s rd, 1023 B/s wr, 6 op/s 2024-04-07T17:29:06.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:29:06 smithi100 bash[76202]: cluster 2024-04-07T17:29:04.272557+0000 mgr.y (mgr.25175) 2211 : cluster [DBG] pgmap v3717: 137 pgs: 137 active+clean; 4.6 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 3.7 KiB/s rd, 1023 B/s wr, 6 op/s 2024-04-07T17:29:07.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:29:07 smithi144 bash[66654]: cluster 2024-04-07T17:29:06.095807+0000 mon.a (mon.0) 3711 : cluster [DBG] osdmap e1932: 8 total, 8 up, 8 in 2024-04-07T17:29:07.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:29:07 smithi100 bash[74942]: cluster 2024-04-07T17:29:06.095807+0000 mon.a (mon.0) 3711 : cluster [DBG] osdmap e1932: 8 total, 8 up, 8 in 2024-04-07T17:29:07.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:29:07 smithi100 bash[76202]: cluster 2024-04-07T17:29:06.095807+0000 mon.a (mon.0) 3711 : cluster [DBG] osdmap e1932: 8 total, 8 up, 8 in 2024-04-07T17:29:08.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:29:08 smithi144 bash[66654]: cluster 2024-04-07T17:29:06.273228+0000 mgr.y (mgr.25175) 2212 : cluster [DBG] pgmap v3719: 137 pgs: 137 active+clean; 4.6 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 511 B/s rd, 383 B/s wr, 1 op/s 2024-04-07T17:29:08.426 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:29:08 smithi100 bash[74942]: cluster 2024-04-07T17:29:06.273228+0000 mgr.y (mgr.25175) 2212 : cluster [DBG] pgmap v3719: 137 pgs: 137 active+clean; 4.6 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 511 B/s rd, 383 B/s wr, 1 op/s 2024-04-07T17:29:08.426 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:29:08 smithi100 bash[76202]: cluster 2024-04-07T17:29:06.273228+0000 mgr.y (mgr.25175) 2212 : cluster [DBG] pgmap v3719: 137 pgs: 137 active+clean; 4.6 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 511 B/s rd, 383 B/s wr, 1 op/s 2024-04-07T17:29:08.704 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:29:08 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:29:08] "GET /metrics HTTP/1.1" 200 36727 "" "Prometheus/2.43.0" 2024-04-07T17:29:10.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:29:10 smithi144 bash[66654]: cluster 2024-04-07T17:29:08.274573+0000 mgr.y (mgr.25175) 2213 : cluster [DBG] pgmap v3720: 137 pgs: 137 active+clean; 29 MiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 23 KiB/s rd, 3.0 MiB/s wr, 29 op/s 2024-04-07T17:29:10.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:29:10 smithi100 bash[76202]: cluster 2024-04-07T17:29:08.274573+0000 mgr.y (mgr.25175) 2213 : cluster [DBG] pgmap v3720: 137 pgs: 137 active+clean; 29 MiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 23 KiB/s rd, 3.0 MiB/s wr, 29 op/s 2024-04-07T17:29:10.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:29:10 smithi100 bash[74942]: cluster 2024-04-07T17:29:08.274573+0000 mgr.y (mgr.25175) 2213 : cluster [DBG] pgmap v3720: 137 pgs: 137 active+clean; 29 MiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 23 KiB/s rd, 3.0 MiB/s wr, 29 op/s 2024-04-07T17:29:11.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:29:11 smithi100 bash[74942]: cluster 2024-04-07T17:29:10.275304+0000 mgr.y (mgr.25175) 2214 : cluster [DBG] pgmap v3721: 137 pgs: 137 active+clean; 46 MiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 111 KiB/s rd, 5.1 MiB/s wr, 52 op/s 2024-04-07T17:29:11.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:29:11 smithi100 bash[76202]: cluster 2024-04-07T17:29:10.275304+0000 mgr.y (mgr.25175) 2214 : cluster [DBG] pgmap v3721: 137 pgs: 137 active+clean; 46 MiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 111 KiB/s rd, 5.1 MiB/s wr, 52 op/s 2024-04-07T17:29:11.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:29:11 smithi144 bash[66654]: cluster 2024-04-07T17:29:10.275304+0000 mgr.y (mgr.25175) 2214 : cluster [DBG] pgmap v3721: 137 pgs: 137 active+clean; 46 MiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 111 KiB/s rd, 5.1 MiB/s wr, 52 op/s 2024-04-07T17:29:12.616 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:29:12 smithi144 bash[95233]: ts=2024-04-07T17:29:12.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:29:12.668 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] EncryptedFlattenTest/5.Simple (19634 ms) 2024-04-07T17:29:12.669 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] EncryptedFlattenTest/5.Grow 2024-04-07T17:29:12.669 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:29:13.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:29:13 smithi144 bash[66654]: cluster 2024-04-07T17:29:12.276066+0000 mgr.y (mgr.25175) 2215 : cluster [DBG] pgmap v3722: 137 pgs: 137 active+clean; 67 MiB data, 3.7 GiB used, 711 GiB / 715 GiB avail; 4.1 MiB/s rd, 6.7 MiB/s wr, 48 op/s 2024-04-07T17:29:13.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:29:13 smithi100 bash[74942]: cluster 2024-04-07T17:29:12.276066+0000 mgr.y (mgr.25175) 2215 : cluster [DBG] pgmap v3722: 137 pgs: 137 active+clean; 67 MiB data, 3.7 GiB used, 711 GiB / 715 GiB avail; 4.1 MiB/s rd, 6.7 MiB/s wr, 48 op/s 2024-04-07T17:29:13.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:29:13 smithi100 bash[76202]: cluster 2024-04-07T17:29:12.276066+0000 mgr.y (mgr.25175) 2215 : cluster [DBG] pgmap v3722: 137 pgs: 137 active+clean; 67 MiB data, 3.7 GiB used, 711 GiB / 715 GiB avail; 4.1 MiB/s rd, 6.7 MiB/s wr, 48 op/s 2024-04-07T17:29:15.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:29:15 smithi144 bash[66654]: cluster 2024-04-07T17:29:14.276886+0000 mgr.y (mgr.25175) 2216 : cluster [DBG] pgmap v3723: 137 pgs: 137 active+clean; 79 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 8.8 MiB/s rd, 7.4 MiB/s wr, 62 op/s 2024-04-07T17:29:15.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:29:15 smithi100 bash[74942]: cluster 2024-04-07T17:29:14.276886+0000 mgr.y (mgr.25175) 2216 : cluster [DBG] pgmap v3723: 137 pgs: 137 active+clean; 79 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 8.8 MiB/s rd, 7.4 MiB/s wr, 62 op/s 2024-04-07T17:29:15.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:29:15 smithi100 bash[76202]: cluster 2024-04-07T17:29:14.276886+0000 mgr.y (mgr.25175) 2216 : cluster [DBG] pgmap v3723: 137 pgs: 137 active+clean; 79 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 8.8 MiB/s rd, 7.4 MiB/s wr, 62 op/s 2024-04-07T17:29:17.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:29:17 smithi144 bash[66654]: cluster 2024-04-07T17:29:16.277496+0000 mgr.y (mgr.25175) 2217 : cluster [DBG] pgmap v3724: 137 pgs: 137 active+clean; 87 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 9.8 MiB/s rd, 8.1 MiB/s wr, 62 op/s 2024-04-07T17:29:17.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:29:17 smithi144 bash[66654]: audit 2024-04-07T17:29:16.845002+0000 mon.a (mon.0) 3712 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:29:17.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:29:17 smithi100 bash[74942]: cluster 2024-04-07T17:29:16.277496+0000 mgr.y (mgr.25175) 2217 : cluster [DBG] pgmap v3724: 137 pgs: 137 active+clean; 87 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 9.8 MiB/s rd, 8.1 MiB/s wr, 62 op/s 2024-04-07T17:29:17.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:29:17 smithi100 bash[74942]: audit 2024-04-07T17:29:16.845002+0000 mon.a (mon.0) 3712 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:29:17.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:29:17 smithi100 bash[76202]: cluster 2024-04-07T17:29:16.277496+0000 mgr.y (mgr.25175) 2217 : cluster [DBG] pgmap v3724: 137 pgs: 137 active+clean; 87 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 9.8 MiB/s rd, 8.1 MiB/s wr, 62 op/s 2024-04-07T17:29:17.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:29:17 smithi100 bash[76202]: audit 2024-04-07T17:29:16.845002+0000 mon.a (mon.0) 3712 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:29:18.704 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:29:18 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:29:18] "GET /metrics HTTP/1.1" 200 36760 "" "Prometheus/2.43.0" 2024-04-07T17:29:19.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:29:19 smithi144 bash[66654]: cluster 2024-04-07T17:29:18.278297+0000 mgr.y (mgr.25175) 2218 : cluster [DBG] pgmap v3725: 137 pgs: 137 active+clean; 87 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 9.8 MiB/s rd, 6.8 MiB/s wr, 57 op/s 2024-04-07T17:29:19.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:29:19 smithi100 bash[74942]: cluster 2024-04-07T17:29:18.278297+0000 mgr.y (mgr.25175) 2218 : cluster [DBG] pgmap v3725: 137 pgs: 137 active+clean; 87 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 9.8 MiB/s rd, 6.8 MiB/s wr, 57 op/s 2024-04-07T17:29:19.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:29:19 smithi100 bash[76202]: cluster 2024-04-07T17:29:18.278297+0000 mgr.y (mgr.25175) 2218 : cluster [DBG] pgmap v3725: 137 pgs: 137 active+clean; 87 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 9.8 MiB/s rd, 6.8 MiB/s wr, 57 op/s 2024-04-07T17:29:21.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:29:21 smithi144 bash[66654]: cluster 2024-04-07T17:29:20.278996+0000 mgr.y (mgr.25175) 2219 : cluster [DBG] pgmap v3726: 137 pgs: 137 active+clean; 87 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 9.8 MiB/s rd, 4.8 MiB/s wr, 37 op/s 2024-04-07T17:29:21.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:29:21 smithi100 bash[74942]: cluster 2024-04-07T17:29:20.278996+0000 mgr.y (mgr.25175) 2219 : cluster [DBG] pgmap v3726: 137 pgs: 137 active+clean; 87 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 9.8 MiB/s rd, 4.8 MiB/s wr, 37 op/s 2024-04-07T17:29:21.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:29:21 smithi100 bash[76202]: cluster 2024-04-07T17:29:20.278996+0000 mgr.y (mgr.25175) 2219 : cluster [DBG] pgmap v3726: 137 pgs: 137 active+clean; 87 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 9.8 MiB/s rd, 4.8 MiB/s wr, 37 op/s 2024-04-07T17:29:22.616 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:29:22 smithi144 bash[95233]: ts=2024-04-07T17:29:22.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:29:23.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:29:23 smithi144 bash[66654]: cluster 2024-04-07T17:29:22.280123+0000 mgr.y (mgr.25175) 2220 : cluster [DBG] pgmap v3727: 137 pgs: 137 active+clean; 87 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 9.7 MiB/s rd, 3.4 MiB/s wr, 21 op/s 2024-04-07T17:29:23.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:29:23 smithi144 bash[66654]: cluster 2024-04-07T17:29:22.353886+0000 mon.a (mon.0) 3713 : cluster [DBG] osdmap e1933: 8 total, 8 up, 8 in 2024-04-07T17:29:23.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:29:23 smithi100 bash[74942]: cluster 2024-04-07T17:29:22.280123+0000 mgr.y (mgr.25175) 2220 : cluster [DBG] pgmap v3727: 137 pgs: 137 active+clean; 87 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 9.7 MiB/s rd, 3.4 MiB/s wr, 21 op/s 2024-04-07T17:29:23.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:29:23 smithi100 bash[74942]: cluster 2024-04-07T17:29:22.353886+0000 mon.a (mon.0) 3713 : cluster [DBG] osdmap e1933: 8 total, 8 up, 8 in 2024-04-07T17:29:23.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:29:23 smithi100 bash[76202]: cluster 2024-04-07T17:29:22.280123+0000 mgr.y (mgr.25175) 2220 : cluster [DBG] pgmap v3727: 137 pgs: 137 active+clean; 87 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 9.7 MiB/s rd, 3.4 MiB/s wr, 21 op/s 2024-04-07T17:29:23.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:29:23 smithi100 bash[76202]: cluster 2024-04-07T17:29:22.353886+0000 mon.a (mon.0) 3713 : cluster [DBG] osdmap e1933: 8 total, 8 up, 8 in 2024-04-07T17:29:24.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:29:24 smithi144 bash[66654]: audit 2024-04-07T17:29:23.655659+0000 mon.a (mon.0) 3714 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "117.6", "id": [5, 4]}]: dispatch 2024-04-07T17:29:24.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:29:24 smithi144 bash[66654]: audit 2024-04-07T17:29:23.655931+0000 mon.a (mon.0) 3715 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "117.8", "id": [5, 4]}]: dispatch 2024-04-07T17:29:24.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:29:24 smithi100 bash[74942]: audit 2024-04-07T17:29:23.655659+0000 mon.a (mon.0) 3714 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "117.6", "id": [5, 4]}]: dispatch 2024-04-07T17:29:24.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:29:24 smithi100 bash[74942]: audit 2024-04-07T17:29:23.655931+0000 mon.a (mon.0) 3715 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "117.8", "id": [5, 4]}]: dispatch 2024-04-07T17:29:24.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:29:24 smithi100 bash[76202]: audit 2024-04-07T17:29:23.655659+0000 mon.a (mon.0) 3714 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "117.6", "id": [5, 4]}]: dispatch 2024-04-07T17:29:24.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:29:24 smithi100 bash[76202]: audit 2024-04-07T17:29:23.655931+0000 mon.a (mon.0) 3715 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "117.8", "id": [5, 4]}]: dispatch 2024-04-07T17:29:25.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:29:25 smithi100 bash[74942]: cluster 2024-04-07T17:29:24.280863+0000 mgr.y (mgr.25175) 2221 : cluster [DBG] pgmap v3729: 137 pgs: 137 active+clean; 104 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 3.0 MiB/s rd, 2.5 MiB/s wr, 18 op/s 2024-04-07T17:29:25.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:29:25 smithi100 bash[74942]: audit 2024-04-07T17:29:24.356939+0000 mon.a (mon.0) 3716 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "117.6", "id": [5, 4]}]': finished 2024-04-07T17:29:25.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:29:25 smithi100 bash[74942]: audit 2024-04-07T17:29:24.357034+0000 mon.a (mon.0) 3717 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "117.8", "id": [5, 4]}]': finished 2024-04-07T17:29:25.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:29:25 smithi100 bash[74942]: cluster 2024-04-07T17:29:24.362543+0000 mon.a (mon.0) 3718 : cluster [DBG] osdmap e1934: 8 total, 8 up, 8 in 2024-04-07T17:29:25.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:29:25 smithi100 bash[76202]: cluster 2024-04-07T17:29:24.280863+0000 mgr.y (mgr.25175) 2221 : cluster [DBG] pgmap v3729: 137 pgs: 137 active+clean; 104 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 3.0 MiB/s rd, 2.5 MiB/s wr, 18 op/s 2024-04-07T17:29:25.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:29:25 smithi100 bash[76202]: audit 2024-04-07T17:29:24.356939+0000 mon.a (mon.0) 3716 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "117.6", "id": [5, 4]}]': finished 2024-04-07T17:29:25.705 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:29:25 smithi100 bash[76202]: audit 2024-04-07T17:29:24.357034+0000 mon.a (mon.0) 3717 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "117.8", "id": [5, 4]}]': finished 2024-04-07T17:29:25.705 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:29:25 smithi100 bash[76202]: cluster 2024-04-07T17:29:24.362543+0000 mon.a (mon.0) 3718 : cluster [DBG] osdmap e1934: 8 total, 8 up, 8 in 2024-04-07T17:29:25.866 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:29:25 smithi144 bash[66654]: cluster 2024-04-07T17:29:24.280863+0000 mgr.y (mgr.25175) 2221 : cluster [DBG] pgmap v3729: 137 pgs: 137 active+clean; 104 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 3.0 MiB/s rd, 2.5 MiB/s wr, 18 op/s 2024-04-07T17:29:25.866 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:29:25 smithi144 bash[66654]: audit 2024-04-07T17:29:24.356939+0000 mon.a (mon.0) 3716 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "117.6", "id": [5, 4]}]': finished 2024-04-07T17:29:25.866 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:29:25 smithi144 bash[66654]: audit 2024-04-07T17:29:24.357034+0000 mon.a (mon.0) 3717 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "117.8", "id": [5, 4]}]': finished 2024-04-07T17:29:25.866 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:29:25 smithi144 bash[66654]: cluster 2024-04-07T17:29:24.362543+0000 mon.a (mon.0) 3718 : cluster [DBG] osdmap e1934: 8 total, 8 up, 8 in 2024-04-07T17:29:26.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:29:26 smithi100 bash[74942]: cluster 2024-04-07T17:29:25.362472+0000 mon.a (mon.0) 3719 : cluster [DBG] osdmap e1935: 8 total, 8 up, 8 in 2024-04-07T17:29:26.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:29:26 smithi100 bash[76202]: cluster 2024-04-07T17:29:25.362472+0000 mon.a (mon.0) 3719 : cluster [DBG] osdmap e1935: 8 total, 8 up, 8 in 2024-04-07T17:29:26.866 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:29:26 smithi144 bash[66654]: cluster 2024-04-07T17:29:25.362472+0000 mon.a (mon.0) 3719 : cluster [DBG] osdmap e1935: 8 total, 8 up, 8 in 2024-04-07T17:29:27.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:29:27 smithi100 bash[74942]: cluster 2024-04-07T17:29:26.281346+0000 mgr.y (mgr.25175) 2222 : cluster [DBG] pgmap v3732: 137 pgs: 137 active+clean; 104 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 63 KiB/s rd, 2.8 MiB/s wr, 20 op/s 2024-04-07T17:29:27.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:29:27 smithi100 bash[76202]: cluster 2024-04-07T17:29:26.281346+0000 mgr.y (mgr.25175) 2222 : cluster [DBG] pgmap v3732: 137 pgs: 137 active+clean; 104 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 63 KiB/s rd, 2.8 MiB/s wr, 20 op/s 2024-04-07T17:29:27.866 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:29:27 smithi144 bash[66654]: cluster 2024-04-07T17:29:26.281346+0000 mgr.y (mgr.25175) 2222 : cluster [DBG] pgmap v3732: 137 pgs: 137 active+clean; 104 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 63 KiB/s rd, 2.8 MiB/s wr, 20 op/s 2024-04-07T17:29:28.704 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:29:28 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:29:28] "GET /metrics HTTP/1.1" 200 36760 "" "Prometheus/2.43.0" 2024-04-07T17:29:29.004 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] EncryptedFlattenTest/5.Grow (16335 ms) 2024-04-07T17:29:29.004 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] EncryptedFlattenTest/5.Shrink 2024-04-07T17:29:29.004 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:29:29.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:29:29 smithi100 bash[74942]: cluster 2024-04-07T17:29:28.282166+0000 mgr.y (mgr.25175) 2223 : cluster [DBG] pgmap v3733: 137 pgs: 137 active+clean; 153 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 9.7 MiB/s rd, 11 MiB/s wr, 87 op/s; 682 KiB/s, 0 objects/s recovering 2024-04-07T17:29:29.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:29:29 smithi100 bash[76202]: cluster 2024-04-07T17:29:28.282166+0000 mgr.y (mgr.25175) 2223 : cluster [DBG] pgmap v3733: 137 pgs: 137 active+clean; 153 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 9.7 MiB/s rd, 11 MiB/s wr, 87 op/s; 682 KiB/s, 0 objects/s recovering 2024-04-07T17:29:29.866 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:29:29 smithi144 bash[66654]: cluster 2024-04-07T17:29:28.282166+0000 mgr.y (mgr.25175) 2223 : cluster [DBG] pgmap v3733: 137 pgs: 137 active+clean; 153 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 9.7 MiB/s rd, 11 MiB/s wr, 87 op/s; 682 KiB/s, 0 objects/s recovering 2024-04-07T17:29:31.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:29:31 smithi100 bash[74942]: cluster 2024-04-07T17:29:30.282881+0000 mgr.y (mgr.25175) 2224 : cluster [DBG] pgmap v3734: 137 pgs: 137 active+clean; 169 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 13 MiB/s rd, 10 MiB/s wr, 85 op/s; 516 KiB/s, 0 objects/s recovering 2024-04-07T17:29:31.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:29:31 smithi100 bash[76202]: cluster 2024-04-07T17:29:30.282881+0000 mgr.y (mgr.25175) 2224 : cluster [DBG] pgmap v3734: 137 pgs: 137 active+clean; 169 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 13 MiB/s rd, 10 MiB/s wr, 85 op/s; 516 KiB/s, 0 objects/s recovering 2024-04-07T17:29:31.866 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:29:31 smithi144 bash[66654]: cluster 2024-04-07T17:29:30.282881+0000 mgr.y (mgr.25175) 2224 : cluster [DBG] pgmap v3734: 137 pgs: 137 active+clean; 169 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 13 MiB/s rd, 10 MiB/s wr, 85 op/s; 516 KiB/s, 0 objects/s recovering 2024-04-07T17:29:32.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:29:32 smithi144 bash[66654]: audit 2024-04-07T17:29:31.845424+0000 mon.a (mon.0) 3720 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:29:32.616 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:29:32 smithi144 bash[95233]: ts=2024-04-07T17:29:32.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:29:32.703 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:29:32 smithi100 bash[74942]: audit 2024-04-07T17:29:31.845424+0000 mon.a (mon.0) 3720 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:29:32.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:29:32 smithi100 bash[76202]: audit 2024-04-07T17:29:31.845424+0000 mon.a (mon.0) 3720 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:29:33.705 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:29:33 smithi100 bash[74942]: cluster 2024-04-07T17:29:32.283682+0000 mgr.y (mgr.25175) 2225 : cluster [DBG] pgmap v3735: 137 pgs: 137 active+clean; 169 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 13 MiB/s rd, 8.1 MiB/s wr, 70 op/s; 512 KiB/s, 0 objects/s recovering 2024-04-07T17:29:33.706 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:29:33 smithi100 bash[76202]: cluster 2024-04-07T17:29:32.283682+0000 mgr.y (mgr.25175) 2225 : cluster [DBG] pgmap v3735: 137 pgs: 137 active+clean; 169 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 13 MiB/s rd, 8.1 MiB/s wr, 70 op/s; 512 KiB/s, 0 objects/s recovering 2024-04-07T17:29:33.866 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:29:33 smithi144 bash[66654]: cluster 2024-04-07T17:29:32.283682+0000 mgr.y (mgr.25175) 2225 : cluster [DBG] pgmap v3735: 137 pgs: 137 active+clean; 169 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 13 MiB/s rd, 8.1 MiB/s wr, 70 op/s; 512 KiB/s, 0 objects/s recovering 2024-04-07T17:29:35.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:29:35 smithi100 bash[74942]: cluster 2024-04-07T17:29:34.284399+0000 mgr.y (mgr.25175) 2226 : cluster [DBG] pgmap v3736: 137 pgs: 137 active+clean; 169 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 12 MiB/s rd, 6.6 MiB/s wr, 57 op/s; 413 KiB/s, 0 objects/s recovering 2024-04-07T17:29:35.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:29:35 smithi100 bash[76202]: cluster 2024-04-07T17:29:34.284399+0000 mgr.y (mgr.25175) 2226 : cluster [DBG] pgmap v3736: 137 pgs: 137 active+clean; 169 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 12 MiB/s rd, 6.6 MiB/s wr, 57 op/s; 413 KiB/s, 0 objects/s recovering 2024-04-07T17:29:35.866 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:29:35 smithi144 bash[66654]: cluster 2024-04-07T17:29:34.284399+0000 mgr.y (mgr.25175) 2226 : cluster [DBG] pgmap v3736: 137 pgs: 137 active+clean; 169 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 12 MiB/s rd, 6.6 MiB/s wr, 57 op/s; 413 KiB/s, 0 objects/s recovering 2024-04-07T17:29:37.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:29:37 smithi100 bash[74942]: cluster 2024-04-07T17:29:36.285014+0000 mgr.y (mgr.25175) 2227 : cluster [DBG] pgmap v3737: 137 pgs: 137 active+clean; 169 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 11 MiB/s rd, 6.0 MiB/s wr, 52 op/s; 375 KiB/s, 0 objects/s recovering 2024-04-07T17:29:37.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:29:37 smithi100 bash[76202]: cluster 2024-04-07T17:29:36.285014+0000 mgr.y (mgr.25175) 2227 : cluster [DBG] pgmap v3737: 137 pgs: 137 active+clean; 169 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 11 MiB/s rd, 6.0 MiB/s wr, 52 op/s; 375 KiB/s, 0 objects/s recovering 2024-04-07T17:29:37.866 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:29:37 smithi144 bash[66654]: cluster 2024-04-07T17:29:36.285014+0000 mgr.y (mgr.25175) 2227 : cluster [DBG] pgmap v3737: 137 pgs: 137 active+clean; 169 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 11 MiB/s rd, 6.0 MiB/s wr, 52 op/s; 375 KiB/s, 0 objects/s recovering 2024-04-07T17:29:38.704 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:29:38 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:29:38] "GET /metrics HTTP/1.1" 200 36770 "" "Prometheus/2.43.0" 2024-04-07T17:29:39.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:29:39 smithi100 bash[74942]: cluster 2024-04-07T17:29:38.285841+0000 mgr.y (mgr.25175) 2228 : cluster [DBG] pgmap v3738: 137 pgs: 137 active+clean; 169 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 9.7 MiB/s rd, 5.4 MiB/s wr, 47 op/s; 341 KiB/s, 0 objects/s recovering 2024-04-07T17:29:39.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:29:39 smithi100 bash[76202]: cluster 2024-04-07T17:29:38.285841+0000 mgr.y (mgr.25175) 2228 : cluster [DBG] pgmap v3738: 137 pgs: 137 active+clean; 169 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 9.7 MiB/s rd, 5.4 MiB/s wr, 47 op/s; 341 KiB/s, 0 objects/s recovering 2024-04-07T17:29:39.866 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:29:39 smithi144 bash[66654]: cluster 2024-04-07T17:29:38.285841+0000 mgr.y (mgr.25175) 2228 : cluster [DBG] pgmap v3738: 137 pgs: 137 active+clean; 169 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 9.7 MiB/s rd, 5.4 MiB/s wr, 47 op/s; 341 KiB/s, 0 objects/s recovering 2024-04-07T17:29:40.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:29:40 smithi100 bash[74942]: cluster 2024-04-07T17:29:39.418683+0000 mon.a (mon.0) 3721 : cluster [DBG] osdmap e1936: 8 total, 8 up, 8 in 2024-04-07T17:29:40.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:29:40 smithi100 bash[76202]: cluster 2024-04-07T17:29:39.418683+0000 mon.a (mon.0) 3721 : cluster [DBG] osdmap e1936: 8 total, 8 up, 8 in 2024-04-07T17:29:40.866 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:29:40 smithi144 bash[66654]: cluster 2024-04-07T17:29:39.418683+0000 mon.a (mon.0) 3721 : cluster [DBG] osdmap e1936: 8 total, 8 up, 8 in 2024-04-07T17:29:41.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:29:41 smithi100 bash[76202]: cluster 2024-04-07T17:29:40.286451+0000 mgr.y (mgr.25175) 2229 : cluster [DBG] pgmap v3740: 137 pgs: 137 active+clean; 178 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 1.7 MiB/s rd, 922 KiB/s wr, 4 op/s 2024-04-07T17:29:41.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:29:41 smithi100 bash[74942]: cluster 2024-04-07T17:29:40.286451+0000 mgr.y (mgr.25175) 2229 : cluster [DBG] pgmap v3740: 137 pgs: 137 active+clean; 178 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 1.7 MiB/s rd, 922 KiB/s wr, 4 op/s 2024-04-07T17:29:41.866 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:29:41 smithi144 bash[66654]: cluster 2024-04-07T17:29:40.286451+0000 mgr.y (mgr.25175) 2229 : cluster [DBG] pgmap v3740: 137 pgs: 137 active+clean; 178 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 1.7 MiB/s rd, 922 KiB/s wr, 4 op/s 2024-04-07T17:29:42.615 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:29:42 smithi144 bash[95233]: ts=2024-04-07T17:29:42.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:29:43.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:29:43 smithi100 bash[74942]: cluster 2024-04-07T17:29:42.287212+0000 mgr.y (mgr.25175) 2230 : cluster [DBG] pgmap v3741: 137 pgs: 137 active+clean; 202 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 1.4 MiB/s rd, 3.3 MiB/s wr, 14 op/s 2024-04-07T17:29:43.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:29:43 smithi100 bash[76202]: cluster 2024-04-07T17:29:42.287212+0000 mgr.y (mgr.25175) 2230 : cluster [DBG] pgmap v3741: 137 pgs: 137 active+clean; 202 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 1.4 MiB/s rd, 3.3 MiB/s wr, 14 op/s 2024-04-07T17:29:43.866 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:29:43 smithi144 bash[66654]: cluster 2024-04-07T17:29:42.287212+0000 mgr.y (mgr.25175) 2230 : cluster [DBG] pgmap v3741: 137 pgs: 137 active+clean; 202 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 1.4 MiB/s rd, 3.3 MiB/s wr, 14 op/s 2024-04-07T17:29:45.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:29:45 smithi100 bash[74942]: cluster 2024-04-07T17:29:44.287955+0000 mgr.y (mgr.25175) 2231 : cluster [DBG] pgmap v3742: 137 pgs: 137 active+clean; 210 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 500 KiB/s rd, 4.1 MiB/s wr, 44 op/s 2024-04-07T17:29:45.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:29:45 smithi100 bash[76202]: cluster 2024-04-07T17:29:44.287955+0000 mgr.y (mgr.25175) 2231 : cluster [DBG] pgmap v3742: 137 pgs: 137 active+clean; 210 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 500 KiB/s rd, 4.1 MiB/s wr, 44 op/s 2024-04-07T17:29:45.866 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:29:45 smithi144 bash[66654]: cluster 2024-04-07T17:29:44.287955+0000 mgr.y (mgr.25175) 2231 : cluster [DBG] pgmap v3742: 137 pgs: 137 active+clean; 210 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 500 KiB/s rd, 4.1 MiB/s wr, 44 op/s 2024-04-07T17:29:46.202 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] EncryptedFlattenTest/5.Shrink (17198 ms) 2024-04-07T17:29:46.202 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] EncryptedFlattenTest/5.ShrinkToOne 2024-04-07T17:29:46.202 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:29:47.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:29:47 smithi100 bash[74942]: cluster 2024-04-07T17:29:46.288929+0000 mgr.y (mgr.25175) 2232 : cluster [DBG] pgmap v3743: 137 pgs: 137 active+clean; 210 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 1.4 MiB/s rd, 4.1 MiB/s wr, 44 op/s 2024-04-07T17:29:47.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:29:47 smithi100 bash[74942]: audit 2024-04-07T17:29:46.846311+0000 mon.a (mon.0) 3722 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:29:47.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:29:47 smithi100 bash[76202]: cluster 2024-04-07T17:29:46.288929+0000 mgr.y (mgr.25175) 2232 : cluster [DBG] pgmap v3743: 137 pgs: 137 active+clean; 210 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 1.4 MiB/s rd, 4.1 MiB/s wr, 44 op/s 2024-04-07T17:29:47.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:29:47 smithi100 bash[76202]: audit 2024-04-07T17:29:46.846311+0000 mon.a (mon.0) 3722 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:29:47.866 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:29:47 smithi144 bash[66654]: cluster 2024-04-07T17:29:46.288929+0000 mgr.y (mgr.25175) 2232 : cluster [DBG] pgmap v3743: 137 pgs: 137 active+clean; 210 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 1.4 MiB/s rd, 4.1 MiB/s wr, 44 op/s 2024-04-07T17:29:47.866 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:29:47 smithi144 bash[66654]: audit 2024-04-07T17:29:46.846311+0000 mon.a (mon.0) 3722 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:29:48.704 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:29:48 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:29:48] "GET /metrics HTTP/1.1" 200 36756 "" "Prometheus/2.43.0" 2024-04-07T17:29:49.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:29:49 smithi100 bash[74942]: cluster 2024-04-07T17:29:48.289926+0000 mgr.y (mgr.25175) 2233 : cluster [DBG] pgmap v3744: 137 pgs: 137 active+clean; 247 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 12 MiB/s rd, 7.8 MiB/s wr, 66 op/s 2024-04-07T17:29:49.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:29:49 smithi100 bash[74942]: audit 2024-04-07T17:29:49.027030+0000 mon.a (mon.0) 3723 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:29:49.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:29:49 smithi100 bash[76202]: cluster 2024-04-07T17:29:48.289926+0000 mgr.y (mgr.25175) 2233 : cluster [DBG] pgmap v3744: 137 pgs: 137 active+clean; 247 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 12 MiB/s rd, 7.8 MiB/s wr, 66 op/s 2024-04-07T17:29:49.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:29:49 smithi100 bash[76202]: audit 2024-04-07T17:29:49.027030+0000 mon.a (mon.0) 3723 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:29:49.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:29:49 smithi144 bash[66654]: cluster 2024-04-07T17:29:48.289926+0000 mgr.y (mgr.25175) 2233 : cluster [DBG] pgmap v3744: 137 pgs: 137 active+clean; 247 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 12 MiB/s rd, 7.8 MiB/s wr, 66 op/s 2024-04-07T17:29:49.866 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:29:49 smithi144 bash[66654]: audit 2024-04-07T17:29:49.027030+0000 mon.a (mon.0) 3723 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:29:51.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:29:51 smithi100 bash[74942]: audit 2024-04-07T17:29:50.183073+0000 mon.a (mon.0) 3724 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:29:51.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:29:51 smithi100 bash[74942]: audit 2024-04-07T17:29:50.190402+0000 mon.a (mon.0) 3725 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:29:51.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:29:51 smithi100 bash[74942]: cluster 2024-04-07T17:29:50.290679+0000 mgr.y (mgr.25175) 2234 : cluster [DBG] pgmap v3745: 137 pgs: 137 active+clean; 251 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 11 MiB/s rd, 7.2 MiB/s wr, 63 op/s 2024-04-07T17:29:51.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:29:51 smithi100 bash[76202]: audit 2024-04-07T17:29:50.183073+0000 mon.a (mon.0) 3724 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:29:51.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:29:51 smithi100 bash[76202]: audit 2024-04-07T17:29:50.190402+0000 mon.a (mon.0) 3725 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:29:51.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:29:51 smithi100 bash[76202]: cluster 2024-04-07T17:29:50.290679+0000 mgr.y (mgr.25175) 2234 : cluster [DBG] pgmap v3745: 137 pgs: 137 active+clean; 251 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 11 MiB/s rd, 7.2 MiB/s wr, 63 op/s 2024-04-07T17:29:51.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:29:51 smithi144 bash[66654]: audit 2024-04-07T17:29:50.183073+0000 mon.a (mon.0) 3724 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:29:51.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:29:51 smithi144 bash[66654]: audit 2024-04-07T17:29:50.190402+0000 mon.a (mon.0) 3725 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:29:51.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:29:51 smithi144 bash[66654]: cluster 2024-04-07T17:29:50.290679+0000 mgr.y (mgr.25175) 2234 : cluster [DBG] pgmap v3745: 137 pgs: 137 active+clean; 251 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 11 MiB/s rd, 7.2 MiB/s wr, 63 op/s 2024-04-07T17:29:52.616 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:29:52 smithi144 bash[95233]: ts=2024-04-07T17:29:52.317Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:29:53.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:29:53 smithi100 bash[74942]: cluster 2024-04-07T17:29:52.291525+0000 mgr.y (mgr.25175) 2235 : cluster [DBG] pgmap v3746: 137 pgs: 137 active+clean; 251 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 9.8 MiB/s rd, 6.1 MiB/s wr, 57 op/s 2024-04-07T17:29:53.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:29:53 smithi100 bash[76202]: cluster 2024-04-07T17:29:52.291525+0000 mgr.y (mgr.25175) 2235 : cluster [DBG] pgmap v3746: 137 pgs: 137 active+clean; 251 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 9.8 MiB/s rd, 6.1 MiB/s wr, 57 op/s 2024-04-07T17:29:53.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:29:53 smithi144 bash[66654]: cluster 2024-04-07T17:29:52.291525+0000 mgr.y (mgr.25175) 2235 : cluster [DBG] pgmap v3746: 137 pgs: 137 active+clean; 251 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 9.8 MiB/s rd, 6.1 MiB/s wr, 57 op/s 2024-04-07T17:29:54.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:29:54 smithi100 bash[74942]: cluster 2024-04-07T17:29:53.361505+0000 mon.a (mon.0) 3726 : cluster [DBG] osdmap e1937: 8 total, 8 up, 8 in 2024-04-07T17:29:54.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:29:54 smithi100 bash[76202]: cluster 2024-04-07T17:29:53.361505+0000 mon.a (mon.0) 3726 : cluster [DBG] osdmap e1937: 8 total, 8 up, 8 in 2024-04-07T17:29:54.866 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:29:54 smithi144 bash[66654]: cluster 2024-04-07T17:29:53.361505+0000 mon.a (mon.0) 3726 : cluster [DBG] osdmap e1937: 8 total, 8 up, 8 in 2024-04-07T17:29:55.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:29:55 smithi144 bash[66654]: cluster 2024-04-07T17:29:54.292173+0000 mgr.y (mgr.25175) 2236 : cluster [DBG] pgmap v3748: 137 pgs: 137 active+clean; 267 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 11 MiB/s rd, 5.7 MiB/s wr, 28 op/s 2024-04-07T17:29:55.703 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:29:55 smithi100 bash[74942]: cluster 2024-04-07T17:29:54.292173+0000 mgr.y (mgr.25175) 2236 : cluster [DBG] pgmap v3748: 137 pgs: 137 active+clean; 267 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 11 MiB/s rd, 5.7 MiB/s wr, 28 op/s 2024-04-07T17:29:55.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:29:55 smithi100 bash[76202]: cluster 2024-04-07T17:29:54.292173+0000 mgr.y (mgr.25175) 2236 : cluster [DBG] pgmap v3748: 137 pgs: 137 active+clean; 267 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 11 MiB/s rd, 5.7 MiB/s wr, 28 op/s 2024-04-07T17:29:57.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:29:57 smithi100 bash[74942]: cluster 2024-04-07T17:29:56.292770+0000 mgr.y (mgr.25175) 2237 : cluster [DBG] pgmap v3749: 137 pgs: 137 active+clean; 280 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 10 MiB/s rd, 7.0 MiB/s wr, 36 op/s 2024-04-07T17:29:57.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:29:57 smithi100 bash[76202]: cluster 2024-04-07T17:29:56.292770+0000 mgr.y (mgr.25175) 2237 : cluster [DBG] pgmap v3749: 137 pgs: 137 active+clean; 280 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 10 MiB/s rd, 7.0 MiB/s wr, 36 op/s 2024-04-07T17:29:57.705 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:29:57 smithi144 bash[66654]: cluster 2024-04-07T17:29:56.292770+0000 mgr.y (mgr.25175) 2237 : cluster [DBG] pgmap v3749: 137 pgs: 137 active+clean; 280 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 10 MiB/s rd, 7.0 MiB/s wr, 36 op/s 2024-04-07T17:29:58.222 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] EncryptedFlattenTest/5.ShrinkToOne (12020 ms) 2024-04-07T17:29:58.222 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] EncryptedFlattenTest/5.ShrinkToOneAfterSnapshot 2024-04-07T17:29:58.222 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:29:58.704 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:29:58 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:29:58] "GET /metrics HTTP/1.1" 200 36756 "" "Prometheus/2.43.0" 2024-04-07T17:29:59.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:29:59 smithi100 bash[74942]: cluster 2024-04-07T17:29:58.294094+0000 mgr.y (mgr.25175) 2238 : cluster [DBG] pgmap v3750: 137 pgs: 137 active+clean; 304 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 888 KiB/s rd, 6.1 MiB/s wr, 53 op/s 2024-04-07T17:29:59.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:29:59 smithi100 bash[76202]: cluster 2024-04-07T17:29:58.294094+0000 mgr.y (mgr.25175) 2238 : cluster [DBG] pgmap v3750: 137 pgs: 137 active+clean; 304 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 888 KiB/s rd, 6.1 MiB/s wr, 53 op/s 2024-04-07T17:29:59.866 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:29:59 smithi144 bash[66654]: cluster 2024-04-07T17:29:58.294094+0000 mgr.y (mgr.25175) 2238 : cluster [DBG] pgmap v3750: 137 pgs: 137 active+clean; 304 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 888 KiB/s rd, 6.1 MiB/s wr, 53 op/s 2024-04-07T17:30:00.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:30:00 smithi100 bash[74942]: audit 2024-04-07T17:29:59.682376+0000 mon.a (mon.0) 3727 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:30:00.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:30:00 smithi100 bash[74942]: audit 2024-04-07T17:29:59.688872+0000 mon.a (mon.0) 3728 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:30:00.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:30:00 smithi100 bash[74942]: audit 2024-04-07T17:29:59.690134+0000 mon.a (mon.0) 3729 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:30:00.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:30:00 smithi100 bash[74942]: audit 2024-04-07T17:29:59.691030+0000 mon.a (mon.0) 3730 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:30:00.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:30:00 smithi100 bash[74942]: audit 2024-04-07T17:29:59.698104+0000 mon.a (mon.0) 3731 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:30:00.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:30:00 smithi100 bash[74942]: cluster 2024-04-07T17:30:00.000121+0000 mon.a (mon.0) 3732 : cluster [WRN] overall HEALTH_WARN Telemetry requires re-opt-in 2024-04-07T17:30:00.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:30:00 smithi100 bash[76202]: audit 2024-04-07T17:29:59.682376+0000 mon.a (mon.0) 3727 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:30:00.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:30:00 smithi100 bash[76202]: audit 2024-04-07T17:29:59.688872+0000 mon.a (mon.0) 3728 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:30:00.955 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:30:00 smithi100 bash[76202]: audit 2024-04-07T17:29:59.690134+0000 mon.a (mon.0) 3729 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:30:00.955 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:30:00 smithi100 bash[76202]: audit 2024-04-07T17:29:59.691030+0000 mon.a (mon.0) 3730 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:30:00.955 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:30:00 smithi100 bash[76202]: audit 2024-04-07T17:29:59.698104+0000 mon.a (mon.0) 3731 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:30:00.955 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:30:00 smithi100 bash[76202]: cluster 2024-04-07T17:30:00.000121+0000 mon.a (mon.0) 3732 : cluster [WRN] overall HEALTH_WARN Telemetry requires re-opt-in 2024-04-07T17:30:01.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:30:00 smithi144 bash[66654]: audit 2024-04-07T17:29:59.682376+0000 mon.a (mon.0) 3727 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:30:01.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:30:00 smithi144 bash[66654]: audit 2024-04-07T17:29:59.688872+0000 mon.a (mon.0) 3728 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:30:01.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:30:00 smithi144 bash[66654]: audit 2024-04-07T17:29:59.690134+0000 mon.a (mon.0) 3729 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:30:01.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:30:00 smithi144 bash[66654]: audit 2024-04-07T17:29:59.691030+0000 mon.a (mon.0) 3730 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:30:01.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:30:00 smithi144 bash[66654]: audit 2024-04-07T17:29:59.698104+0000 mon.a (mon.0) 3731 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:30:01.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:30:00 smithi144 bash[66654]: cluster 2024-04-07T17:30:00.000121+0000 mon.a (mon.0) 3732 : cluster [WRN] overall HEALTH_WARN Telemetry requires re-opt-in 2024-04-07T17:30:02.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:30:01 smithi144 bash[66654]: cluster 2024-04-07T17:30:00.294912+0000 mgr.y (mgr.25175) 2239 : cluster [DBG] pgmap v3751: 137 pgs: 137 active+clean; 312 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 2.1 MiB/s rd, 6.5 MiB/s wr, 57 op/s 2024-04-07T17:30:02.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:30:01 smithi100 bash[74942]: cluster 2024-04-07T17:30:00.294912+0000 mgr.y (mgr.25175) 2239 : cluster [DBG] pgmap v3751: 137 pgs: 137 active+clean; 312 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 2.1 MiB/s rd, 6.5 MiB/s wr, 57 op/s 2024-04-07T17:30:02.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:30:01 smithi100 bash[76202]: cluster 2024-04-07T17:30:00.294912+0000 mgr.y (mgr.25175) 2239 : cluster [DBG] pgmap v3751: 137 pgs: 137 active+clean; 312 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 2.1 MiB/s rd, 6.5 MiB/s wr, 57 op/s 2024-04-07T17:30:02.616 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:30:02 smithi144 bash[95233]: ts=2024-04-07T17:30:02.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:30:03.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:30:02 smithi144 bash[66654]: audit 2024-04-07T17:30:01.845987+0000 mon.a (mon.0) 3733 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:30:03.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:30:02 smithi100 bash[74942]: audit 2024-04-07T17:30:01.845987+0000 mon.a (mon.0) 3733 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:30:03.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:30:02 smithi100 bash[76202]: audit 2024-04-07T17:30:01.845987+0000 mon.a (mon.0) 3733 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:30:04.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:30:03 smithi144 bash[66654]: cluster 2024-04-07T17:30:02.295691+0000 mgr.y (mgr.25175) 2240 : cluster [DBG] pgmap v3752: 137 pgs: 137 active+clean; 312 MiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 2.1 MiB/s rd, 6.5 MiB/s wr, 60 op/s 2024-04-07T17:30:04.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:30:03 smithi100 bash[74942]: cluster 2024-04-07T17:30:02.295691+0000 mgr.y (mgr.25175) 2240 : cluster [DBG] pgmap v3752: 137 pgs: 137 active+clean; 312 MiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 2.1 MiB/s rd, 6.5 MiB/s wr, 60 op/s 2024-04-07T17:30:04.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:30:03 smithi100 bash[76202]: cluster 2024-04-07T17:30:02.295691+0000 mgr.y (mgr.25175) 2240 : cluster [DBG] pgmap v3752: 137 pgs: 137 active+clean; 312 MiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 2.1 MiB/s rd, 6.5 MiB/s wr, 60 op/s 2024-04-07T17:30:06.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:30:05 smithi144 bash[66654]: cluster 2024-04-07T17:30:04.296526+0000 mgr.y (mgr.25175) 2241 : cluster [DBG] pgmap v3753: 137 pgs: 137 active+clean; 312 MiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 1.9 MiB/s rd, 5.2 MiB/s wr, 55 op/s 2024-04-07T17:30:06.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:30:05 smithi100 bash[74942]: cluster 2024-04-07T17:30:04.296526+0000 mgr.y (mgr.25175) 2241 : cluster [DBG] pgmap v3753: 137 pgs: 137 active+clean; 312 MiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 1.9 MiB/s rd, 5.2 MiB/s wr, 55 op/s 2024-04-07T17:30:06.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:30:05 smithi100 bash[76202]: cluster 2024-04-07T17:30:04.296526+0000 mgr.y (mgr.25175) 2241 : cluster [DBG] pgmap v3753: 137 pgs: 137 active+clean; 312 MiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 1.9 MiB/s rd, 5.2 MiB/s wr, 55 op/s 2024-04-07T17:30:07.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:30:06 smithi144 bash[66654]: cluster 2024-04-07T17:30:05.732059+0000 mon.a (mon.0) 3734 : cluster [DBG] osdmap e1938: 8 total, 8 up, 8 in 2024-04-07T17:30:07.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:30:06 smithi100 bash[74942]: cluster 2024-04-07T17:30:05.732059+0000 mon.a (mon.0) 3734 : cluster [DBG] osdmap e1938: 8 total, 8 up, 8 in 2024-04-07T17:30:07.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:30:06 smithi100 bash[76202]: cluster 2024-04-07T17:30:05.732059+0000 mon.a (mon.0) 3734 : cluster [DBG] osdmap e1938: 8 total, 8 up, 8 in 2024-04-07T17:30:08.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:30:07 smithi144 bash[66654]: cluster 2024-04-07T17:30:06.297045+0000 mgr.y (mgr.25175) 2242 : cluster [DBG] pgmap v3755: 137 pgs: 137 active+clean; 312 MiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 2.1 MiB/s rd, 3.6 MiB/s wr, 51 op/s 2024-04-07T17:30:08.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:30:07 smithi100 bash[74942]: cluster 2024-04-07T17:30:06.297045+0000 mgr.y (mgr.25175) 2242 : cluster [DBG] pgmap v3755: 137 pgs: 137 active+clean; 312 MiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 2.1 MiB/s rd, 3.6 MiB/s wr, 51 op/s 2024-04-07T17:30:08.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:30:07 smithi100 bash[76202]: cluster 2024-04-07T17:30:06.297045+0000 mgr.y (mgr.25175) 2242 : cluster [DBG] pgmap v3755: 137 pgs: 137 active+clean; 312 MiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 2.1 MiB/s rd, 3.6 MiB/s wr, 51 op/s 2024-04-07T17:30:08.703 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:30:08 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:30:08] "GET /metrics HTTP/1.1" 200 36758 "" "Prometheus/2.43.0" 2024-04-07T17:30:10.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:30:09 smithi144 bash[66654]: cluster 2024-04-07T17:30:08.297870+0000 mgr.y (mgr.25175) 2243 : cluster [DBG] pgmap v3756: 137 pgs: 137 active+clean; 344 MiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 1.3 MiB/s rd, 4.0 MiB/s wr, 36 op/s 2024-04-07T17:30:10.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:30:09 smithi100 bash[74942]: cluster 2024-04-07T17:30:08.297870+0000 mgr.y (mgr.25175) 2243 : cluster [DBG] pgmap v3756: 137 pgs: 137 active+clean; 344 MiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 1.3 MiB/s rd, 4.0 MiB/s wr, 36 op/s 2024-04-07T17:30:10.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:30:09 smithi100 bash[76202]: cluster 2024-04-07T17:30:08.297870+0000 mgr.y (mgr.25175) 2243 : cluster [DBG] pgmap v3756: 137 pgs: 137 active+clean; 344 MiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 1.3 MiB/s rd, 4.0 MiB/s wr, 36 op/s 2024-04-07T17:30:11.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:30:10 smithi144 bash[66654]: cluster 2024-04-07T17:30:09.760486+0000 mon.a (mon.0) 3735 : cluster [DBG] osdmap e1939: 8 total, 8 up, 8 in 2024-04-07T17:30:11.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:30:10 smithi100 bash[74942]: cluster 2024-04-07T17:30:09.760486+0000 mon.a (mon.0) 3735 : cluster [DBG] osdmap e1939: 8 total, 8 up, 8 in 2024-04-07T17:30:11.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:30:10 smithi100 bash[76202]: cluster 2024-04-07T17:30:09.760486+0000 mon.a (mon.0) 3735 : cluster [DBG] osdmap e1939: 8 total, 8 up, 8 in 2024-04-07T17:30:11.957 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] EncryptedFlattenTest/5.ShrinkToOneAfterSnapshot (13735 ms) 2024-04-07T17:30:11.957 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] EncryptedFlattenTest/5.MinOverlap 2024-04-07T17:30:11.957 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:30:12.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:30:11 smithi144 bash[66654]: cluster 2024-04-07T17:30:10.298508+0000 mgr.y (mgr.25175) 2244 : cluster [DBG] pgmap v3758: 137 pgs: 137 active+clean; 353 MiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 212 KiB/s rd, 5.1 MiB/s wr, 65 op/s 2024-04-07T17:30:12.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:30:11 smithi100 bash[74942]: cluster 2024-04-07T17:30:10.298508+0000 mgr.y (mgr.25175) 2244 : cluster [DBG] pgmap v3758: 137 pgs: 137 active+clean; 353 MiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 212 KiB/s rd, 5.1 MiB/s wr, 65 op/s 2024-04-07T17:30:12.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:30:11 smithi100 bash[76202]: cluster 2024-04-07T17:30:10.298508+0000 mgr.y (mgr.25175) 2244 : cluster [DBG] pgmap v3758: 137 pgs: 137 active+clean; 353 MiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 212 KiB/s rd, 5.1 MiB/s wr, 65 op/s 2024-04-07T17:30:12.616 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:30:12 smithi144 bash[95233]: ts=2024-04-07T17:30:12.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:30:14.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:30:13 smithi144 bash[66654]: cluster 2024-04-07T17:30:12.299596+0000 mgr.y (mgr.25175) 2245 : cluster [DBG] pgmap v3759: 137 pgs: 137 active+clean; 377 MiB data, 4.7 GiB used, 711 GiB / 715 GiB avail; 4.7 MiB/s rd, 8.1 MiB/s wr, 77 op/s 2024-04-07T17:30:14.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:30:13 smithi100 bash[74942]: cluster 2024-04-07T17:30:12.299596+0000 mgr.y (mgr.25175) 2245 : cluster [DBG] pgmap v3759: 137 pgs: 137 active+clean; 377 MiB data, 4.7 GiB used, 711 GiB / 715 GiB avail; 4.7 MiB/s rd, 8.1 MiB/s wr, 77 op/s 2024-04-07T17:30:14.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:30:13 smithi100 bash[76202]: cluster 2024-04-07T17:30:12.299596+0000 mgr.y (mgr.25175) 2245 : cluster [DBG] pgmap v3759: 137 pgs: 137 active+clean; 377 MiB data, 4.7 GiB used, 711 GiB / 715 GiB avail; 4.7 MiB/s rd, 8.1 MiB/s wr, 77 op/s 2024-04-07T17:30:16.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:30:15 smithi144 bash[66654]: cluster 2024-04-07T17:30:14.300201+0000 mgr.y (mgr.25175) 2246 : cluster [DBG] pgmap v3760: 137 pgs: 137 active+clean; 430 MiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 5.9 MiB/s rd, 14 MiB/s wr, 81 op/s 2024-04-07T17:30:16.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:30:15 smithi100 bash[74942]: cluster 2024-04-07T17:30:14.300201+0000 mgr.y (mgr.25175) 2246 : cluster [DBG] pgmap v3760: 137 pgs: 137 active+clean; 430 MiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 5.9 MiB/s rd, 14 MiB/s wr, 81 op/s 2024-04-07T17:30:16.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:30:15 smithi100 bash[76202]: cluster 2024-04-07T17:30:14.300201+0000 mgr.y (mgr.25175) 2246 : cluster [DBG] pgmap v3760: 137 pgs: 137 active+clean; 430 MiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 5.9 MiB/s rd, 14 MiB/s wr, 81 op/s 2024-04-07T17:30:18.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:30:17 smithi144 bash[66654]: cluster 2024-04-07T17:30:16.300780+0000 mgr.y (mgr.25175) 2247 : cluster [DBG] pgmap v3761: 137 pgs: 137 active+clean; 438 MiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 5.8 MiB/s rd, 13 MiB/s wr, 83 op/s 2024-04-07T17:30:18.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:30:17 smithi144 bash[66654]: audit 2024-04-07T17:30:16.846334+0000 mon.a (mon.0) 3736 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:30:18.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:30:17 smithi100 bash[74942]: cluster 2024-04-07T17:30:16.300780+0000 mgr.y (mgr.25175) 2247 : cluster [DBG] pgmap v3761: 137 pgs: 137 active+clean; 438 MiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 5.8 MiB/s rd, 13 MiB/s wr, 83 op/s 2024-04-07T17:30:18.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:30:17 smithi100 bash[74942]: audit 2024-04-07T17:30:16.846334+0000 mon.a (mon.0) 3736 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:30:18.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:30:17 smithi100 bash[76202]: cluster 2024-04-07T17:30:16.300780+0000 mgr.y (mgr.25175) 2247 : cluster [DBG] pgmap v3761: 137 pgs: 137 active+clean; 438 MiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 5.8 MiB/s rd, 13 MiB/s wr, 83 op/s 2024-04-07T17:30:18.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:30:17 smithi100 bash[76202]: audit 2024-04-07T17:30:16.846334+0000 mon.a (mon.0) 3736 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:30:18.704 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:30:18 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:30:18] "GET /metrics HTTP/1.1" 200 36767 "" "Prometheus/2.43.0" 2024-04-07T17:30:20.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:30:19 smithi144 bash[66654]: cluster 2024-04-07T17:30:18.301576+0000 mgr.y (mgr.25175) 2248 : cluster [DBG] pgmap v3762: 137 pgs: 137 active+clean; 438 MiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 5.8 MiB/s rd, 9.4 MiB/s wr, 60 op/s 2024-04-07T17:30:20.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:30:19 smithi100 bash[74942]: cluster 2024-04-07T17:30:18.301576+0000 mgr.y (mgr.25175) 2248 : cluster [DBG] pgmap v3762: 137 pgs: 137 active+clean; 438 MiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 5.8 MiB/s rd, 9.4 MiB/s wr, 60 op/s 2024-04-07T17:30:20.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:30:19 smithi100 bash[76202]: cluster 2024-04-07T17:30:18.301576+0000 mgr.y (mgr.25175) 2248 : cluster [DBG] pgmap v3762: 137 pgs: 137 active+clean; 438 MiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 5.8 MiB/s rd, 9.4 MiB/s wr, 60 op/s 2024-04-07T17:30:22.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:30:21 smithi144 bash[66654]: cluster 2024-04-07T17:30:20.302291+0000 mgr.y (mgr.25175) 2249 : cluster [DBG] pgmap v3763: 137 pgs: 137 active+clean; 438 MiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 5.5 MiB/s rd, 8.5 MiB/s wr, 56 op/s 2024-04-07T17:30:22.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:30:21 smithi100 bash[74942]: cluster 2024-04-07T17:30:20.302291+0000 mgr.y (mgr.25175) 2249 : cluster [DBG] pgmap v3763: 137 pgs: 137 active+clean; 438 MiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 5.5 MiB/s rd, 8.5 MiB/s wr, 56 op/s 2024-04-07T17:30:22.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:30:21 smithi100 bash[76202]: cluster 2024-04-07T17:30:20.302291+0000 mgr.y (mgr.25175) 2249 : cluster [DBG] pgmap v3763: 137 pgs: 137 active+clean; 438 MiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 5.5 MiB/s rd, 8.5 MiB/s wr, 56 op/s 2024-04-07T17:30:22.616 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:30:22 smithi144 bash[95233]: ts=2024-04-07T17:30:22.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:30:23.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:30:22 smithi144 bash[66654]: cluster 2024-04-07T17:30:21.798743+0000 mon.a (mon.0) 3737 : cluster [DBG] osdmap e1940: 8 total, 8 up, 8 in 2024-04-07T17:30:23.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:30:22 smithi100 bash[74942]: cluster 2024-04-07T17:30:21.798743+0000 mon.a (mon.0) 3737 : cluster [DBG] osdmap e1940: 8 total, 8 up, 8 in 2024-04-07T17:30:23.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:30:22 smithi100 bash[76202]: cluster 2024-04-07T17:30:21.798743+0000 mon.a (mon.0) 3737 : cluster [DBG] osdmap e1940: 8 total, 8 up, 8 in 2024-04-07T17:30:24.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:30:24 smithi144 bash[66654]: cluster 2024-04-07T17:30:22.303089+0000 mgr.y (mgr.25175) 2250 : cluster [DBG] pgmap v3765: 137 pgs: 137 active+clean; 451 MiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 2.0 MiB/s rd, 7.4 MiB/s wr, 22 op/s 2024-04-07T17:30:24.453 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:30:24 smithi100 bash[74942]: cluster 2024-04-07T17:30:22.303089+0000 mgr.y (mgr.25175) 2250 : cluster [DBG] pgmap v3765: 137 pgs: 137 active+clean; 451 MiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 2.0 MiB/s rd, 7.4 MiB/s wr, 22 op/s 2024-04-07T17:30:24.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:30:24 smithi100 bash[76202]: cluster 2024-04-07T17:30:22.303089+0000 mgr.y (mgr.25175) 2250 : cluster [DBG] pgmap v3765: 137 pgs: 137 active+clean; 451 MiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 2.0 MiB/s rd, 7.4 MiB/s wr, 22 op/s 2024-04-07T17:30:26.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:30:26 smithi144 bash[66654]: cluster 2024-04-07T17:30:24.303815+0000 mgr.y (mgr.25175) 2251 : cluster [DBG] pgmap v3766: 137 pgs: 137 active+clean; 463 MiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 918 KiB/s rd, 3.3 MiB/s wr, 56 op/s 2024-04-07T17:30:26.453 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:30:26 smithi100 bash[74942]: cluster 2024-04-07T17:30:24.303815+0000 mgr.y (mgr.25175) 2251 : cluster [DBG] pgmap v3766: 137 pgs: 137 active+clean; 463 MiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 918 KiB/s rd, 3.3 MiB/s wr, 56 op/s 2024-04-07T17:30:26.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:30:26 smithi100 bash[76202]: cluster 2024-04-07T17:30:24.303815+0000 mgr.y (mgr.25175) 2251 : cluster [DBG] pgmap v3766: 137 pgs: 137 active+clean; 463 MiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 918 KiB/s rd, 3.3 MiB/s wr, 56 op/s 2024-04-07T17:30:28.328 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] EncryptedFlattenTest/5.MinOverlap (16372 ms) 2024-04-07T17:30:28.329 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] EncryptedFlattenTest/5.ZeroOverlap 2024-04-07T17:30:28.329 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:30:28.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:30:28 smithi144 bash[66654]: cluster 2024-04-07T17:30:26.304518+0000 mgr.y (mgr.25175) 2252 : cluster [DBG] pgmap v3767: 137 pgs: 137 active+clean; 479 MiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 89 KiB/s rd, 4.1 MiB/s wr, 42 op/s 2024-04-07T17:30:28.426 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:30:28 smithi100 bash[74942]: cluster 2024-04-07T17:30:26.304518+0000 mgr.y (mgr.25175) 2252 : cluster [DBG] pgmap v3767: 137 pgs: 137 active+clean; 479 MiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 89 KiB/s rd, 4.1 MiB/s wr, 42 op/s 2024-04-07T17:30:28.426 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:30:28 smithi100 bash[76202]: cluster 2024-04-07T17:30:26.304518+0000 mgr.y (mgr.25175) 2252 : cluster [DBG] pgmap v3767: 137 pgs: 137 active+clean; 479 MiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 89 KiB/s rd, 4.1 MiB/s wr, 42 op/s 2024-04-07T17:30:28.703 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:30:28 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:30:28] "GET /metrics HTTP/1.1" 200 36767 "" "Prometheus/2.43.0" 2024-04-07T17:30:30.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:30:30 smithi144 bash[66654]: cluster 2024-04-07T17:30:28.305245+0000 mgr.y (mgr.25175) 2253 : cluster [DBG] pgmap v3768: 137 pgs: 137 active+clean; 491 MiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 531 KiB/s rd, 5.3 MiB/s wr, 48 op/s 2024-04-07T17:30:30.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:30:30 smithi100 bash[74942]: cluster 2024-04-07T17:30:28.305245+0000 mgr.y (mgr.25175) 2253 : cluster [DBG] pgmap v3768: 137 pgs: 137 active+clean; 491 MiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 531 KiB/s rd, 5.3 MiB/s wr, 48 op/s 2024-04-07T17:30:30.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:30:30 smithi100 bash[76202]: cluster 2024-04-07T17:30:28.305245+0000 mgr.y (mgr.25175) 2253 : cluster [DBG] pgmap v3768: 137 pgs: 137 active+clean; 491 MiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 531 KiB/s rd, 5.3 MiB/s wr, 48 op/s 2024-04-07T17:30:32.453 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:30:32 smithi100 bash[74942]: cluster 2024-04-07T17:30:30.306031+0000 mgr.y (mgr.25175) 2254 : cluster [DBG] pgmap v3769: 137 pgs: 137 active+clean; 499 MiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 3.3 MiB/s rd, 6.5 MiB/s wr, 59 op/s 2024-04-07T17:30:32.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:30:32 smithi100 bash[74942]: audit 2024-04-07T17:30:31.846781+0000 mon.a (mon.0) 3738 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:30:32.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:30:32 smithi100 bash[76202]: cluster 2024-04-07T17:30:30.306031+0000 mgr.y (mgr.25175) 2254 : cluster [DBG] pgmap v3769: 137 pgs: 137 active+clean; 499 MiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 3.3 MiB/s rd, 6.5 MiB/s wr, 59 op/s 2024-04-07T17:30:32.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:30:32 smithi100 bash[76202]: audit 2024-04-07T17:30:31.846781+0000 mon.a (mon.0) 3738 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:30:32.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:30:32 smithi144 bash[66654]: cluster 2024-04-07T17:30:30.306031+0000 mgr.y (mgr.25175) 2254 : cluster [DBG] pgmap v3769: 137 pgs: 137 active+clean; 499 MiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 3.3 MiB/s rd, 6.5 MiB/s wr, 59 op/s 2024-04-07T17:30:32.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:30:32 smithi144 bash[66654]: audit 2024-04-07T17:30:31.846781+0000 mon.a (mon.0) 3738 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:30:32.616 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:30:32 smithi144 bash[95233]: ts=2024-04-07T17:30:32.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:30:34.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:30:34 smithi100 bash[74942]: cluster 2024-04-07T17:30:32.306842+0000 mgr.y (mgr.25175) 2255 : cluster [DBG] pgmap v3770: 137 pgs: 137 active+clean; 499 MiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 3.2 MiB/s rd, 6.2 MiB/s wr, 59 op/s 2024-04-07T17:30:34.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:30:34 smithi100 bash[76202]: cluster 2024-04-07T17:30:32.306842+0000 mgr.y (mgr.25175) 2255 : cluster [DBG] pgmap v3770: 137 pgs: 137 active+clean; 499 MiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 3.2 MiB/s rd, 6.2 MiB/s wr, 59 op/s 2024-04-07T17:30:34.619 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:30:34 smithi144 bash[66654]: cluster 2024-04-07T17:30:32.306842+0000 mgr.y (mgr.25175) 2255 : cluster [DBG] pgmap v3770: 137 pgs: 137 active+clean; 499 MiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 3.2 MiB/s rd, 6.2 MiB/s wr, 59 op/s 2024-04-07T17:30:35.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:30:35 smithi100 bash[74942]: cluster 2024-04-07T17:30:34.307938+0000 mgr.y (mgr.25175) 2256 : cluster [DBG] pgmap v3771: 137 pgs: 137 active+clean; 499 MiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 2.8 MiB/s rd, 4.3 MiB/s wr, 52 op/s 2024-04-07T17:30:35.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:30:35 smithi100 bash[76202]: cluster 2024-04-07T17:30:34.307938+0000 mgr.y (mgr.25175) 2256 : cluster [DBG] pgmap v3771: 137 pgs: 137 active+clean; 499 MiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 2.8 MiB/s rd, 4.3 MiB/s wr, 52 op/s 2024-04-07T17:30:35.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:30:35 smithi144 bash[66654]: cluster 2024-04-07T17:30:34.307938+0000 mgr.y (mgr.25175) 2256 : cluster [DBG] pgmap v3771: 137 pgs: 137 active+clean; 499 MiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 2.8 MiB/s rd, 4.3 MiB/s wr, 52 op/s 2024-04-07T17:30:37.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:30:37 smithi144 bash[66654]: cluster 2024-04-07T17:30:36.308563+0000 mgr.y (mgr.25175) 2257 : cluster [DBG] pgmap v3772: 137 pgs: 137 active+clean; 499 MiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 2.7 MiB/s rd, 3.3 MiB/s wr, 18 op/s 2024-04-07T17:30:37.703 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:30:37 smithi100 bash[74942]: cluster 2024-04-07T17:30:36.308563+0000 mgr.y (mgr.25175) 2257 : cluster [DBG] pgmap v3772: 137 pgs: 137 active+clean; 499 MiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 2.7 MiB/s rd, 3.3 MiB/s wr, 18 op/s 2024-04-07T17:30:37.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:30:37 smithi100 bash[76202]: cluster 2024-04-07T17:30:36.308563+0000 mgr.y (mgr.25175) 2257 : cluster [DBG] pgmap v3772: 137 pgs: 137 active+clean; 499 MiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 2.7 MiB/s rd, 3.3 MiB/s wr, 18 op/s 2024-04-07T17:30:38.705 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:30:38 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:30:38] "GET /metrics HTTP/1.1" 200 36760 "" "Prometheus/2.43.0" 2024-04-07T17:30:39.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:30:39 smithi100 bash[74942]: cluster 2024-04-07T17:30:38.309671+0000 mgr.y (mgr.25175) 2258 : cluster [DBG] pgmap v3773: 137 pgs: 137 active+clean; 499 MiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.0 MiB/s wr, 18 op/s 2024-04-07T17:30:39.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:30:39 smithi100 bash[74942]: cluster 2024-04-07T17:30:38.371944+0000 mon.a (mon.0) 3739 : cluster [DBG] osdmap e1941: 8 total, 8 up, 8 in 2024-04-07T17:30:39.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:30:39 smithi100 bash[76202]: cluster 2024-04-07T17:30:38.309671+0000 mgr.y (mgr.25175) 2258 : cluster [DBG] pgmap v3773: 137 pgs: 137 active+clean; 499 MiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.0 MiB/s wr, 18 op/s 2024-04-07T17:30:39.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:30:39 smithi100 bash[76202]: cluster 2024-04-07T17:30:38.371944+0000 mon.a (mon.0) 3739 : cluster [DBG] osdmap e1941: 8 total, 8 up, 8 in 2024-04-07T17:30:39.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:30:39 smithi144 bash[66654]: cluster 2024-04-07T17:30:38.309671+0000 mgr.y (mgr.25175) 2258 : cluster [DBG] pgmap v3773: 137 pgs: 137 active+clean; 499 MiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.0 MiB/s wr, 18 op/s 2024-04-07T17:30:39.866 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:30:39 smithi144 bash[66654]: cluster 2024-04-07T17:30:38.371944+0000 mon.a (mon.0) 3739 : cluster [DBG] osdmap e1941: 8 total, 8 up, 8 in 2024-04-07T17:30:41.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:30:41 smithi100 bash[74942]: cluster 2024-04-07T17:30:40.310392+0000 mgr.y (mgr.25175) 2259 : cluster [DBG] pgmap v3775: 137 pgs: 137 active+clean; 511 MiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 3.1 KiB/s rd, 1.2 MiB/s wr, 4 op/s 2024-04-07T17:30:41.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:30:41 smithi100 bash[76202]: cluster 2024-04-07T17:30:40.310392+0000 mgr.y (mgr.25175) 2259 : cluster [DBG] pgmap v3775: 137 pgs: 137 active+clean; 511 MiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 3.1 KiB/s rd, 1.2 MiB/s wr, 4 op/s 2024-04-07T17:30:41.866 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:30:41 smithi144 bash[66654]: cluster 2024-04-07T17:30:40.310392+0000 mgr.y (mgr.25175) 2259 : cluster [DBG] pgmap v3775: 137 pgs: 137 active+clean; 511 MiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 3.1 KiB/s rd, 1.2 MiB/s wr, 4 op/s 2024-04-07T17:30:42.616 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:30:42 smithi144 bash[95233]: ts=2024-04-07T17:30:42.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:30:43.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:30:43 smithi100 bash[74942]: cluster 2024-04-07T17:30:42.311160+0000 mgr.y (mgr.25175) 2260 : cluster [DBG] pgmap v3776: 137 pgs: 137 active+clean; 516 MiB data, 5.1 GiB used, 710 GiB / 715 GiB avail; 24 KiB/s rd, 1.7 MiB/s wr, 29 op/s 2024-04-07T17:30:43.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:30:43 smithi100 bash[76202]: cluster 2024-04-07T17:30:42.311160+0000 mgr.y (mgr.25175) 2260 : cluster [DBG] pgmap v3776: 137 pgs: 137 active+clean; 516 MiB data, 5.1 GiB used, 710 GiB / 715 GiB avail; 24 KiB/s rd, 1.7 MiB/s wr, 29 op/s 2024-04-07T17:30:43.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:30:43 smithi144 bash[66654]: cluster 2024-04-07T17:30:42.311160+0000 mgr.y (mgr.25175) 2260 : cluster [DBG] pgmap v3776: 137 pgs: 137 active+clean; 516 MiB data, 5.1 GiB used, 710 GiB / 715 GiB avail; 24 KiB/s rd, 1.7 MiB/s wr, 29 op/s 2024-04-07T17:30:44.792 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] EncryptedFlattenTest/5.ZeroOverlap (16463 ms) 2024-04-07T17:30:45.409 INFO:tasks.workunit.client.0.smithi100.stdout:[----------] 7 tests from EncryptedFlattenTest/5 (111757 ms total) 2024-04-07T17:30:45.409 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:30:45.409 INFO:tasks.workunit.client.0.smithi100.stdout:[----------] 7 tests from EncryptedFlattenTest/6, where TypeParam = LUKS2UnderLUKS2 2024-04-07T17:30:45.491 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] EncryptedFlattenTest/6.Simple 2024-04-07T17:30:45.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:30:45 smithi100 bash[76202]: cluster 2024-04-07T17:30:44.311871+0000 mgr.y (mgr.25175) 2261 : cluster [DBG] pgmap v3777: 137 pgs: 137 active+clean; 544 MiB data, 5.2 GiB used, 710 GiB / 715 GiB avail; 908 KiB/s rd, 4.5 MiB/s wr, 43 op/s 2024-04-07T17:30:45.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:30:45 smithi100 bash[74942]: cluster 2024-04-07T17:30:44.311871+0000 mgr.y (mgr.25175) 2261 : cluster [DBG] pgmap v3777: 137 pgs: 137 active+clean; 544 MiB data, 5.2 GiB used, 710 GiB / 715 GiB avail; 908 KiB/s rd, 4.5 MiB/s wr, 43 op/s 2024-04-07T17:30:45.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:30:45 smithi144 bash[66654]: cluster 2024-04-07T17:30:44.311871+0000 mgr.y (mgr.25175) 2261 : cluster [DBG] pgmap v3777: 137 pgs: 137 active+clean; 544 MiB data, 5.2 GiB used, 710 GiB / 715 GiB avail; 908 KiB/s rd, 4.5 MiB/s wr, 43 op/s 2024-04-07T17:30:46.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:30:46 smithi100 bash[74942]: cluster 2024-04-07T17:30:45.393777+0000 mon.a (mon.0) 3740 : cluster [DBG] osdmap e1942: 8 total, 8 up, 8 in 2024-04-07T17:30:46.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:30:46 smithi100 bash[76202]: cluster 2024-04-07T17:30:45.393777+0000 mon.a (mon.0) 3740 : cluster [DBG] osdmap e1942: 8 total, 8 up, 8 in 2024-04-07T17:30:46.866 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:30:46 smithi144 bash[66654]: cluster 2024-04-07T17:30:45.393777+0000 mon.a (mon.0) 3740 : cluster [DBG] osdmap e1942: 8 total, 8 up, 8 in 2024-04-07T17:30:47.431 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:30:47.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:30:47 smithi100 bash[74942]: cluster 2024-04-07T17:30:46.312553+0000 mgr.y (mgr.25175) 2262 : cluster [DBG] pgmap v3779: 105 pgs: 105 active+clean; 4.6 MiB data, 5.2 GiB used, 710 GiB / 715 GiB avail 2024-04-07T17:30:47.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:30:47 smithi100 bash[74942]: cluster 2024-04-07T17:30:46.409304+0000 mon.a (mon.0) 3741 : cluster [DBG] osdmap e1943: 8 total, 8 up, 8 in 2024-04-07T17:30:47.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:30:47 smithi100 bash[74942]: audit 2024-04-07T17:30:46.414024+0000 mon.c (mon.2) 97 : audit [INF] from='client.? 172.21.15.100:0/3478027334' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-42","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:30:47.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:30:47 smithi100 bash[74942]: audit 2024-04-07T17:30:46.414977+0000 mon.a (mon.0) 3742 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-42","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:30:47.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:30:47 smithi100 bash[74942]: audit 2024-04-07T17:30:46.847850+0000 mon.a (mon.0) 3743 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:30:47.705 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:30:47 smithi100 bash[76202]: cluster 2024-04-07T17:30:46.312553+0000 mgr.y (mgr.25175) 2262 : cluster [DBG] pgmap v3779: 105 pgs: 105 active+clean; 4.6 MiB data, 5.2 GiB used, 710 GiB / 715 GiB avail 2024-04-07T17:30:47.705 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:30:47 smithi100 bash[76202]: cluster 2024-04-07T17:30:46.409304+0000 mon.a (mon.0) 3741 : cluster [DBG] osdmap e1943: 8 total, 8 up, 8 in 2024-04-07T17:30:47.705 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:30:47 smithi100 bash[76202]: audit 2024-04-07T17:30:46.414024+0000 mon.c (mon.2) 97 : audit [INF] from='client.? 172.21.15.100:0/3478027334' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-42","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:30:47.705 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:30:47 smithi100 bash[76202]: audit 2024-04-07T17:30:46.414977+0000 mon.a (mon.0) 3742 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-42","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:30:47.705 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:30:47 smithi100 bash[76202]: audit 2024-04-07T17:30:46.847850+0000 mon.a (mon.0) 3743 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:30:47.866 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:30:47 smithi144 bash[66654]: cluster 2024-04-07T17:30:46.312553+0000 mgr.y (mgr.25175) 2262 : cluster [DBG] pgmap v3779: 105 pgs: 105 active+clean; 4.6 MiB data, 5.2 GiB used, 710 GiB / 715 GiB avail 2024-04-07T17:30:47.866 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:30:47 smithi144 bash[66654]: cluster 2024-04-07T17:30:46.409304+0000 mon.a (mon.0) 3741 : cluster [DBG] osdmap e1943: 8 total, 8 up, 8 in 2024-04-07T17:30:47.866 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:30:47 smithi144 bash[66654]: audit 2024-04-07T17:30:46.414024+0000 mon.c (mon.2) 97 : audit [INF] from='client.? 172.21.15.100:0/3478027334' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-42","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:30:47.866 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:30:47 smithi144 bash[66654]: audit 2024-04-07T17:30:46.414977+0000 mon.a (mon.0) 3742 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-42","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:30:47.866 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:30:47 smithi144 bash[66654]: audit 2024-04-07T17:30:46.847850+0000 mon.a (mon.0) 3743 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:30:48.704 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:30:48 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:30:48] "GET /metrics HTTP/1.1" 200 36697 "" "Prometheus/2.43.0" 2024-04-07T17:30:48.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:30:48 smithi100 bash[74942]: audit 2024-04-07T17:30:47.414605+0000 mon.a (mon.0) 3744 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-42","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T17:30:48.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:30:48 smithi100 bash[74942]: cluster 2024-04-07T17:30:47.423733+0000 mon.a (mon.0) 3745 : cluster [DBG] osdmap e1944: 8 total, 8 up, 8 in 2024-04-07T17:30:48.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:30:48 smithi100 bash[76202]: audit 2024-04-07T17:30:47.414605+0000 mon.a (mon.0) 3744 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-42","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T17:30:48.705 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:30:48 smithi100 bash[76202]: cluster 2024-04-07T17:30:47.423733+0000 mon.a (mon.0) 3745 : cluster [DBG] osdmap e1944: 8 total, 8 up, 8 in 2024-04-07T17:30:48.866 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:30:48 smithi144 bash[66654]: audit 2024-04-07T17:30:47.414605+0000 mon.a (mon.0) 3744 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-42","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T17:30:48.866 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:30:48 smithi144 bash[66654]: cluster 2024-04-07T17:30:47.423733+0000 mon.a (mon.0) 3745 : cluster [DBG] osdmap e1944: 8 total, 8 up, 8 in 2024-04-07T17:30:49.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:30:49 smithi100 bash[74942]: cluster 2024-04-07T17:30:48.313469+0000 mgr.y (mgr.25175) 2263 : cluster [DBG] pgmap v3782: 137 pgs: 15 creating+peering, 11 unknown, 111 active+clean; 4.6 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail 2024-04-07T17:30:49.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:30:49 smithi100 bash[74942]: cluster 2024-04-07T17:30:48.432826+0000 mon.a (mon.0) 3746 : cluster [DBG] osdmap e1945: 8 total, 8 up, 8 in 2024-04-07T17:30:49.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:30:49 smithi100 bash[76202]: cluster 2024-04-07T17:30:48.313469+0000 mgr.y (mgr.25175) 2263 : cluster [DBG] pgmap v3782: 137 pgs: 15 creating+peering, 11 unknown, 111 active+clean; 4.6 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail 2024-04-07T17:30:49.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:30:49 smithi100 bash[76202]: cluster 2024-04-07T17:30:48.432826+0000 mon.a (mon.0) 3746 : cluster [DBG] osdmap e1945: 8 total, 8 up, 8 in 2024-04-07T17:30:49.866 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:30:49 smithi144 bash[66654]: cluster 2024-04-07T17:30:48.313469+0000 mgr.y (mgr.25175) 2263 : cluster [DBG] pgmap v3782: 137 pgs: 15 creating+peering, 11 unknown, 111 active+clean; 4.6 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail 2024-04-07T17:30:49.866 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:30:49 smithi144 bash[66654]: cluster 2024-04-07T17:30:48.432826+0000 mon.a (mon.0) 3746 : cluster [DBG] osdmap e1945: 8 total, 8 up, 8 in 2024-04-07T17:30:50.703 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:30:50 smithi100 bash[74942]: cluster 2024-04-07T17:30:49.433026+0000 mon.a (mon.0) 3747 : cluster [DBG] osdmap e1946: 8 total, 8 up, 8 in 2024-04-07T17:30:50.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:30:50 smithi100 bash[76202]: cluster 2024-04-07T17:30:49.433026+0000 mon.a (mon.0) 3747 : cluster [DBG] osdmap e1946: 8 total, 8 up, 8 in 2024-04-07T17:30:50.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:30:50 smithi144 bash[66654]: cluster 2024-04-07T17:30:49.433026+0000 mon.a (mon.0) 3747 : cluster [DBG] osdmap e1946: 8 total, 8 up, 8 in 2024-04-07T17:30:51.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:30:51 smithi100 bash[74942]: cluster 2024-04-07T17:30:50.314248+0000 mgr.y (mgr.25175) 2264 : cluster [DBG] pgmap v3785: 137 pgs: 15 creating+peering, 122 active+clean; 4.6 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail 2024-04-07T17:30:51.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:30:51 smithi100 bash[76202]: cluster 2024-04-07T17:30:50.314248+0000 mgr.y (mgr.25175) 2264 : cluster [DBG] pgmap v3785: 137 pgs: 15 creating+peering, 122 active+clean; 4.6 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail 2024-04-07T17:30:51.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:30:51 smithi144 bash[66654]: cluster 2024-04-07T17:30:50.314248+0000 mgr.y (mgr.25175) 2264 : cluster [DBG] pgmap v3785: 137 pgs: 15 creating+peering, 122 active+clean; 4.6 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail 2024-04-07T17:30:52.615 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:30:52 smithi144 bash[95233]: ts=2024-04-07T17:30:52.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:30:53.866 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:30:53 smithi144 bash[66654]: cluster 2024-04-07T17:30:52.315128+0000 mgr.y (mgr.25175) 2265 : cluster [DBG] pgmap v3786: 137 pgs: 137 active+clean; 4.6 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 4.7 KiB/s rd, 865 B/s wr, 7 op/s 2024-04-07T17:30:53.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:30:53 smithi100 bash[74942]: cluster 2024-04-07T17:30:52.315128+0000 mgr.y (mgr.25175) 2265 : cluster [DBG] pgmap v3786: 137 pgs: 137 active+clean; 4.6 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 4.7 KiB/s rd, 865 B/s wr, 7 op/s 2024-04-07T17:30:53.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:30:53 smithi100 bash[76202]: cluster 2024-04-07T17:30:52.315128+0000 mgr.y (mgr.25175) 2265 : cluster [DBG] pgmap v3786: 137 pgs: 137 active+clean; 4.6 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 4.7 KiB/s rd, 865 B/s wr, 7 op/s 2024-04-07T17:30:55.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:30:55 smithi144 bash[66654]: cluster 2024-04-07T17:30:54.315846+0000 mgr.y (mgr.25175) 2266 : cluster [DBG] pgmap v3787: 137 pgs: 137 active+clean; 4.6 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 4.3 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-04-07T17:30:55.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:30:55 smithi100 bash[74942]: cluster 2024-04-07T17:30:54.315846+0000 mgr.y (mgr.25175) 2266 : cluster [DBG] pgmap v3787: 137 pgs: 137 active+clean; 4.6 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 4.3 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-04-07T17:30:55.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:30:55 smithi100 bash[76202]: cluster 2024-04-07T17:30:54.315846+0000 mgr.y (mgr.25175) 2266 : cluster [DBG] pgmap v3787: 137 pgs: 137 active+clean; 4.6 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 4.3 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-04-07T17:30:57.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:30:57 smithi144 bash[66654]: cluster 2024-04-07T17:30:56.316427+0000 mgr.y (mgr.25175) 2267 : cluster [DBG] pgmap v3788: 137 pgs: 137 active+clean; 4.6 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 3.7 KiB/s rd, 1023 B/s wr, 6 op/s 2024-04-07T17:30:57.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:30:57 smithi100 bash[74942]: cluster 2024-04-07T17:30:56.316427+0000 mgr.y (mgr.25175) 2267 : cluster [DBG] pgmap v3788: 137 pgs: 137 active+clean; 4.6 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 3.7 KiB/s rd, 1023 B/s wr, 6 op/s 2024-04-07T17:30:57.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:30:57 smithi100 bash[76202]: cluster 2024-04-07T17:30:56.316427+0000 mgr.y (mgr.25175) 2267 : cluster [DBG] pgmap v3788: 137 pgs: 137 active+clean; 4.6 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 3.7 KiB/s rd, 1023 B/s wr, 6 op/s 2024-04-07T17:30:58.704 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:30:58 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:30:58] "GET /metrics HTTP/1.1" 200 36697 "" "Prometheus/2.43.0" 2024-04-07T17:30:59.453 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:30:59 smithi100 bash[76202]: cluster 2024-04-07T17:30:58.112022+0000 mon.a (mon.0) 3748 : cluster [DBG] osdmap e1947: 8 total, 8 up, 8 in 2024-04-07T17:30:59.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:30:59 smithi100 bash[74942]: cluster 2024-04-07T17:30:58.112022+0000 mon.a (mon.0) 3748 : cluster [DBG] osdmap e1947: 8 total, 8 up, 8 in 2024-04-07T17:30:59.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:30:59 smithi144 bash[66654]: cluster 2024-04-07T17:30:58.112022+0000 mon.a (mon.0) 3748 : cluster [DBG] osdmap e1947: 8 total, 8 up, 8 in 2024-04-07T17:31:00.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:31:00 smithi100 bash[76202]: cluster 2024-04-07T17:30:58.316889+0000 mgr.y (mgr.25175) 2268 : cluster [DBG] pgmap v3790: 137 pgs: 137 active+clean; 4.6 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 3.4 KiB/s rd, 921 B/s wr, 5 op/s 2024-04-07T17:31:00.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:31:00 smithi100 bash[76202]: audit 2024-04-07T17:30:59.782857+0000 mon.a (mon.0) 3749 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:31:00.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:31:00 smithi100 bash[74942]: cluster 2024-04-07T17:30:58.316889+0000 mgr.y (mgr.25175) 2268 : cluster [DBG] pgmap v3790: 137 pgs: 137 active+clean; 4.6 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 3.4 KiB/s rd, 921 B/s wr, 5 op/s 2024-04-07T17:31:00.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:31:00 smithi100 bash[74942]: audit 2024-04-07T17:30:59.782857+0000 mon.a (mon.0) 3749 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:31:00.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:31:00 smithi144 bash[66654]: cluster 2024-04-07T17:30:58.316889+0000 mgr.y (mgr.25175) 2268 : cluster [DBG] pgmap v3790: 137 pgs: 137 active+clean; 4.6 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 3.4 KiB/s rd, 921 B/s wr, 5 op/s 2024-04-07T17:31:00.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:31:00 smithi144 bash[66654]: audit 2024-04-07T17:30:59.782857+0000 mon.a (mon.0) 3749 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:31:01.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:31:01 smithi100 bash[74942]: cluster 2024-04-07T17:31:00.127776+0000 mon.a (mon.0) 3750 : cluster [DBG] osdmap e1948: 8 total, 8 up, 8 in 2024-04-07T17:31:01.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:31:01 smithi100 bash[74942]: cluster 2024-04-07T17:31:00.317480+0000 mgr.y (mgr.25175) 2269 : cluster [DBG] pgmap v3792: 137 pgs: 137 active+clean; 13 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 255 B/s rd, 1.0 MiB/s wr, 1 op/s 2024-04-07T17:31:01.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:31:01 smithi100 bash[76202]: cluster 2024-04-07T17:31:00.127776+0000 mon.a (mon.0) 3750 : cluster [DBG] osdmap e1948: 8 total, 8 up, 8 in 2024-04-07T17:31:01.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:31:01 smithi100 bash[76202]: cluster 2024-04-07T17:31:00.317480+0000 mgr.y (mgr.25175) 2269 : cluster [DBG] pgmap v3792: 137 pgs: 137 active+clean; 13 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 255 B/s rd, 1.0 MiB/s wr, 1 op/s 2024-04-07T17:31:01.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:31:01 smithi144 bash[66654]: cluster 2024-04-07T17:31:00.127776+0000 mon.a (mon.0) 3750 : cluster [DBG] osdmap e1948: 8 total, 8 up, 8 in 2024-04-07T17:31:01.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:31:01 smithi144 bash[66654]: cluster 2024-04-07T17:31:00.317480+0000 mgr.y (mgr.25175) 2269 : cluster [DBG] pgmap v3792: 137 pgs: 137 active+clean; 13 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 255 B/s rd, 1.0 MiB/s wr, 1 op/s 2024-04-07T17:31:02.453 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:31:02 smithi100 bash[74942]: audit 2024-04-07T17:31:01.848057+0000 mon.a (mon.0) 3751 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:31:02.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:31:02 smithi100 bash[76202]: audit 2024-04-07T17:31:01.848057+0000 mon.a (mon.0) 3751 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:31:02.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:31:02 smithi144 bash[66654]: audit 2024-04-07T17:31:01.848057+0000 mon.a (mon.0) 3751 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:31:02.616 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:31:02 smithi144 bash[95233]: ts=2024-04-07T17:31:02.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:31:03.453 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:31:03 smithi100 bash[74942]: cluster 2024-04-07T17:31:02.318344+0000 mgr.y (mgr.25175) 2270 : cluster [DBG] pgmap v3793: 137 pgs: 137 active+clean; 46 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 25 KiB/s rd, 5.1 MiB/s wr, 35 op/s 2024-04-07T17:31:03.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:31:03 smithi100 bash[76202]: cluster 2024-04-07T17:31:02.318344+0000 mgr.y (mgr.25175) 2270 : cluster [DBG] pgmap v3793: 137 pgs: 137 active+clean; 46 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 25 KiB/s rd, 5.1 MiB/s wr, 35 op/s 2024-04-07T17:31:03.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:31:03 smithi144 bash[66654]: cluster 2024-04-07T17:31:02.318344+0000 mgr.y (mgr.25175) 2270 : cluster [DBG] pgmap v3793: 137 pgs: 137 active+clean; 46 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 25 KiB/s rd, 5.1 MiB/s wr, 35 op/s 2024-04-07T17:31:05.706 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:31:05 smithi100 bash[74942]: cluster 2024-04-07T17:31:04.318945+0000 mgr.y (mgr.25175) 2271 : cluster [DBG] pgmap v3794: 137 pgs: 137 active+clean; 46 MiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 32 KiB/s rd, 5.1 MiB/s wr, 44 op/s 2024-04-07T17:31:05.707 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:31:05 smithi100 bash[76202]: cluster 2024-04-07T17:31:04.318945+0000 mgr.y (mgr.25175) 2271 : cluster [DBG] pgmap v3794: 137 pgs: 137 active+clean; 46 MiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 32 KiB/s rd, 5.1 MiB/s wr, 44 op/s 2024-04-07T17:31:05.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:31:05 smithi144 bash[66654]: cluster 2024-04-07T17:31:04.318945+0000 mgr.y (mgr.25175) 2271 : cluster [DBG] pgmap v3794: 137 pgs: 137 active+clean; 46 MiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 32 KiB/s rd, 5.1 MiB/s wr, 44 op/s 2024-04-07T17:31:06.866 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:31:06 smithi144 bash[66654]: audit 2024-04-07T17:31:05.524485+0000 mon.a (mon.0) 3752 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:31:06.866 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:31:06 smithi144 bash[66654]: audit 2024-04-07T17:31:05.532720+0000 mon.a (mon.0) 3753 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:31:06.866 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:31:06 smithi144 bash[66654]: audit 2024-04-07T17:31:05.679634+0000 mon.a (mon.0) 3754 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:31:06.866 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:31:06 smithi144 bash[66654]: audit 2024-04-07T17:31:05.687409+0000 mon.a (mon.0) 3755 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:31:06.866 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:31:06 smithi144 bash[66654]: audit 2024-04-07T17:31:06.265192+0000 mon.a (mon.0) 3756 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:31:06.866 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:31:06 smithi144 bash[66654]: audit 2024-04-07T17:31:06.266147+0000 mon.a (mon.0) 3757 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:31:06.866 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:31:06 smithi144 bash[66654]: audit 2024-04-07T17:31:06.272851+0000 mon.a (mon.0) 3758 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:31:06.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:31:06 smithi100 bash[74942]: audit 2024-04-07T17:31:05.524485+0000 mon.a (mon.0) 3752 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:31:06.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:31:06 smithi100 bash[74942]: audit 2024-04-07T17:31:05.532720+0000 mon.a (mon.0) 3753 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:31:06.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:31:06 smithi100 bash[74942]: audit 2024-04-07T17:31:05.679634+0000 mon.a (mon.0) 3754 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:31:06.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:31:06 smithi100 bash[74942]: audit 2024-04-07T17:31:05.687409+0000 mon.a (mon.0) 3755 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:31:06.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:31:06 smithi100 bash[74942]: audit 2024-04-07T17:31:06.265192+0000 mon.a (mon.0) 3756 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:31:06.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:31:06 smithi100 bash[74942]: audit 2024-04-07T17:31:06.266147+0000 mon.a (mon.0) 3757 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:31:06.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:31:06 smithi100 bash[74942]: audit 2024-04-07T17:31:06.272851+0000 mon.a (mon.0) 3758 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:31:06.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:31:06 smithi100 bash[76202]: audit 2024-04-07T17:31:05.524485+0000 mon.a (mon.0) 3752 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:31:06.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:31:06 smithi100 bash[76202]: audit 2024-04-07T17:31:05.532720+0000 mon.a (mon.0) 3753 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:31:06.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:31:06 smithi100 bash[76202]: audit 2024-04-07T17:31:05.679634+0000 mon.a (mon.0) 3754 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:31:06.955 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:31:06 smithi100 bash[76202]: audit 2024-04-07T17:31:05.687409+0000 mon.a (mon.0) 3755 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:31:06.955 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:31:06 smithi100 bash[76202]: audit 2024-04-07T17:31:06.265192+0000 mon.a (mon.0) 3756 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:31:06.955 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:31:06 smithi100 bash[76202]: audit 2024-04-07T17:31:06.266147+0000 mon.a (mon.0) 3757 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:31:06.955 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:31:06 smithi100 bash[76202]: audit 2024-04-07T17:31:06.272851+0000 mon.a (mon.0) 3758 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:31:07.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:31:07 smithi144 bash[66654]: cluster 2024-04-07T17:31:06.319563+0000 mgr.y (mgr.25175) 2272 : cluster [DBG] pgmap v3795: 137 pgs: 137 active+clean; 46 MiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 31 KiB/s rd, 5.0 MiB/s wr, 43 op/s 2024-04-07T17:31:07.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:31:07 smithi100 bash[74942]: cluster 2024-04-07T17:31:06.319563+0000 mgr.y (mgr.25175) 2272 : cluster [DBG] pgmap v3795: 137 pgs: 137 active+clean; 46 MiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 31 KiB/s rd, 5.0 MiB/s wr, 43 op/s 2024-04-07T17:31:07.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:31:07 smithi100 bash[76202]: cluster 2024-04-07T17:31:06.319563+0000 mgr.y (mgr.25175) 2272 : cluster [DBG] pgmap v3795: 137 pgs: 137 active+clean; 46 MiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 31 KiB/s rd, 5.0 MiB/s wr, 43 op/s 2024-04-07T17:31:08.703 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:31:08 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:31:08] "GET /metrics HTTP/1.1" 200 36744 "" "Prometheus/2.43.0" 2024-04-07T17:31:09.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:31:09 smithi144 bash[66654]: cluster 2024-04-07T17:31:08.320372+0000 mgr.y (mgr.25175) 2273 : cluster [DBG] pgmap v3796: 137 pgs: 137 active+clean; 46 MiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 26 KiB/s rd, 4.1 MiB/s wr, 35 op/s 2024-04-07T17:31:09.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:31:09 smithi100 bash[74942]: cluster 2024-04-07T17:31:08.320372+0000 mgr.y (mgr.25175) 2273 : cluster [DBG] pgmap v3796: 137 pgs: 137 active+clean; 46 MiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 26 KiB/s rd, 4.1 MiB/s wr, 35 op/s 2024-04-07T17:31:09.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:31:09 smithi100 bash[76202]: cluster 2024-04-07T17:31:08.320372+0000 mgr.y (mgr.25175) 2273 : cluster [DBG] pgmap v3796: 137 pgs: 137 active+clean; 46 MiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 26 KiB/s rd, 4.1 MiB/s wr, 35 op/s 2024-04-07T17:31:11.866 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:31:11 smithi144 bash[66654]: cluster 2024-04-07T17:31:10.321066+0000 mgr.y (mgr.25175) 2274 : cluster [DBG] pgmap v3797: 137 pgs: 137 active+clean; 58 MiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 56 KiB/s rd, 4.4 MiB/s wr, 38 op/s 2024-04-07T17:31:11.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:31:11 smithi100 bash[74942]: cluster 2024-04-07T17:31:10.321066+0000 mgr.y (mgr.25175) 2274 : cluster [DBG] pgmap v3797: 137 pgs: 137 active+clean; 58 MiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 56 KiB/s rd, 4.4 MiB/s wr, 38 op/s 2024-04-07T17:31:11.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:31:11 smithi100 bash[76202]: cluster 2024-04-07T17:31:10.321066+0000 mgr.y (mgr.25175) 2274 : cluster [DBG] pgmap v3797: 137 pgs: 137 active+clean; 58 MiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 56 KiB/s rd, 4.4 MiB/s wr, 38 op/s 2024-04-07T17:31:12.616 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:31:12 smithi144 bash[95233]: ts=2024-04-07T17:31:12.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:31:13.403 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] EncryptedFlattenTest/6.Simple (27912 ms) 2024-04-07T17:31:13.403 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] EncryptedFlattenTest/6.Grow 2024-04-07T17:31:13.404 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:31:13.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:31:13 smithi144 bash[66654]: cluster 2024-04-07T17:31:12.321841+0000 mgr.y (mgr.25175) 2275 : cluster [DBG] pgmap v3798: 137 pgs: 137 active+clean; 67 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 4.3 MiB/s rd, 4.8 MiB/s wr, 46 op/s 2024-04-07T17:31:13.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:31:13 smithi100 bash[76202]: cluster 2024-04-07T17:31:12.321841+0000 mgr.y (mgr.25175) 2275 : cluster [DBG] pgmap v3798: 137 pgs: 137 active+clean; 67 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 4.3 MiB/s rd, 4.8 MiB/s wr, 46 op/s 2024-04-07T17:31:13.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:31:13 smithi100 bash[74942]: cluster 2024-04-07T17:31:12.321841+0000 mgr.y (mgr.25175) 2275 : cluster [DBG] pgmap v3798: 137 pgs: 137 active+clean; 67 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 4.3 MiB/s rd, 4.8 MiB/s wr, 46 op/s 2024-04-07T17:31:15.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:31:15 smithi144 bash[66654]: cluster 2024-04-07T17:31:14.322894+0000 mgr.y (mgr.25175) 2276 : cluster [DBG] pgmap v3799: 137 pgs: 137 active+clean; 87 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 8.0 MiB/s rd, 4.8 MiB/s wr, 33 op/s 2024-04-07T17:31:15.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:31:15 smithi100 bash[74942]: cluster 2024-04-07T17:31:14.322894+0000 mgr.y (mgr.25175) 2276 : cluster [DBG] pgmap v3799: 137 pgs: 137 active+clean; 87 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 8.0 MiB/s rd, 4.8 MiB/s wr, 33 op/s 2024-04-07T17:31:15.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:31:15 smithi100 bash[76202]: cluster 2024-04-07T17:31:14.322894+0000 mgr.y (mgr.25175) 2276 : cluster [DBG] pgmap v3799: 137 pgs: 137 active+clean; 87 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 8.0 MiB/s rd, 4.8 MiB/s wr, 33 op/s 2024-04-07T17:31:17.866 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:31:17 smithi144 bash[66654]: cluster 2024-04-07T17:31:16.323561+0000 mgr.y (mgr.25175) 2277 : cluster [DBG] pgmap v3800: 137 pgs: 137 active+clean; 87 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 8.7 MiB/s rd, 4.8 MiB/s wr, 27 op/s 2024-04-07T17:31:17.866 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:31:17 smithi144 bash[66654]: audit 2024-04-07T17:31:16.847950+0000 mon.a (mon.0) 3759 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:31:17.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:31:17 smithi100 bash[74942]: cluster 2024-04-07T17:31:16.323561+0000 mgr.y (mgr.25175) 2277 : cluster [DBG] pgmap v3800: 137 pgs: 137 active+clean; 87 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 8.7 MiB/s rd, 4.8 MiB/s wr, 27 op/s 2024-04-07T17:31:17.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:31:17 smithi100 bash[74942]: audit 2024-04-07T17:31:16.847950+0000 mon.a (mon.0) 3759 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:31:17.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:31:17 smithi100 bash[76202]: cluster 2024-04-07T17:31:16.323561+0000 mgr.y (mgr.25175) 2277 : cluster [DBG] pgmap v3800: 137 pgs: 137 active+clean; 87 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 8.7 MiB/s rd, 4.8 MiB/s wr, 27 op/s 2024-04-07T17:31:17.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:31:17 smithi100 bash[76202]: audit 2024-04-07T17:31:16.847950+0000 mon.a (mon.0) 3759 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:31:18.704 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:31:18 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:31:18] "GET /metrics HTTP/1.1" 200 36756 "" "Prometheus/2.43.0" 2024-04-07T17:31:19.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:31:19 smithi144 bash[66654]: cluster 2024-04-07T17:31:18.324374+0000 mgr.y (mgr.25175) 2278 : cluster [DBG] pgmap v3801: 137 pgs: 137 active+clean; 87 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 9.7 MiB/s rd, 4.8 MiB/s wr, 28 op/s 2024-04-07T17:31:19.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:31:19 smithi100 bash[74942]: cluster 2024-04-07T17:31:18.324374+0000 mgr.y (mgr.25175) 2278 : cluster [DBG] pgmap v3801: 137 pgs: 137 active+clean; 87 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 9.7 MiB/s rd, 4.8 MiB/s wr, 28 op/s 2024-04-07T17:31:19.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:31:19 smithi100 bash[76202]: cluster 2024-04-07T17:31:18.324374+0000 mgr.y (mgr.25175) 2278 : cluster [DBG] pgmap v3801: 137 pgs: 137 active+clean; 87 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 9.7 MiB/s rd, 4.8 MiB/s wr, 28 op/s 2024-04-07T17:31:21.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:31:21 smithi144 bash[66654]: cluster 2024-04-07T17:31:20.325077+0000 mgr.y (mgr.25175) 2279 : cluster [DBG] pgmap v3802: 137 pgs: 137 active+clean; 87 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 9.7 MiB/s rd, 4.8 MiB/s wr, 28 op/s 2024-04-07T17:31:21.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:31:21 smithi100 bash[74942]: cluster 2024-04-07T17:31:20.325077+0000 mgr.y (mgr.25175) 2279 : cluster [DBG] pgmap v3802: 137 pgs: 137 active+clean; 87 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 9.7 MiB/s rd, 4.8 MiB/s wr, 28 op/s 2024-04-07T17:31:21.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:31:21 smithi100 bash[76202]: cluster 2024-04-07T17:31:20.325077+0000 mgr.y (mgr.25175) 2279 : cluster [DBG] pgmap v3802: 137 pgs: 137 active+clean; 87 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 9.7 MiB/s rd, 4.8 MiB/s wr, 28 op/s 2024-04-07T17:31:22.615 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:31:22 smithi144 bash[95233]: ts=2024-04-07T17:31:22.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:31:23.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:31:23 smithi144 bash[66654]: cluster 2024-04-07T17:31:22.325820+0000 mgr.y (mgr.25175) 2280 : cluster [DBG] pgmap v3803: 137 pgs: 137 active+clean; 87 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 9.7 MiB/s rd, 3.8 MiB/s wr, 25 op/s 2024-04-07T17:31:23.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:31:23 smithi100 bash[76202]: cluster 2024-04-07T17:31:22.325820+0000 mgr.y (mgr.25175) 2280 : cluster [DBG] pgmap v3803: 137 pgs: 137 active+clean; 87 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 9.7 MiB/s rd, 3.8 MiB/s wr, 25 op/s 2024-04-07T17:31:23.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:31:23 smithi100 bash[74942]: cluster 2024-04-07T17:31:22.325820+0000 mgr.y (mgr.25175) 2280 : cluster [DBG] pgmap v3803: 137 pgs: 137 active+clean; 87 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 9.7 MiB/s rd, 3.8 MiB/s wr, 25 op/s 2024-04-07T17:31:24.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:31:24 smithi144 bash[66654]: cluster 2024-04-07T17:31:23.574284+0000 mon.a (mon.0) 3760 : cluster [DBG] osdmap e1949: 8 total, 8 up, 8 in 2024-04-07T17:31:24.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:31:24 smithi100 bash[74942]: cluster 2024-04-07T17:31:23.574284+0000 mon.a (mon.0) 3760 : cluster [DBG] osdmap e1949: 8 total, 8 up, 8 in 2024-04-07T17:31:24.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:31:24 smithi100 bash[76202]: cluster 2024-04-07T17:31:23.574284+0000 mon.a (mon.0) 3760 : cluster [DBG] osdmap e1949: 8 total, 8 up, 8 in 2024-04-07T17:31:25.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:31:25 smithi144 bash[66654]: cluster 2024-04-07T17:31:24.326715+0000 mgr.y (mgr.25175) 2281 : cluster [DBG] pgmap v3805: 137 pgs: 137 active+clean; 103 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 2.1 MiB/s rd, 1.6 MiB/s wr, 28 op/s 2024-04-07T17:31:25.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:31:25 smithi100 bash[74942]: cluster 2024-04-07T17:31:24.326715+0000 mgr.y (mgr.25175) 2281 : cluster [DBG] pgmap v3805: 137 pgs: 137 active+clean; 103 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 2.1 MiB/s rd, 1.6 MiB/s wr, 28 op/s 2024-04-07T17:31:25.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:31:25 smithi100 bash[76202]: cluster 2024-04-07T17:31:24.326715+0000 mgr.y (mgr.25175) 2281 : cluster [DBG] pgmap v3805: 137 pgs: 137 active+clean; 103 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 2.1 MiB/s rd, 1.6 MiB/s wr, 28 op/s 2024-04-07T17:31:27.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:31:27 smithi144 bash[66654]: cluster 2024-04-07T17:31:26.327338+0000 mgr.y (mgr.25175) 2282 : cluster [DBG] pgmap v3806: 137 pgs: 137 active+clean; 103 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 28 op/s 2024-04-07T17:31:27.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:31:27 smithi100 bash[74942]: cluster 2024-04-07T17:31:26.327338+0000 mgr.y (mgr.25175) 2282 : cluster [DBG] pgmap v3806: 137 pgs: 137 active+clean; 103 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 28 op/s 2024-04-07T17:31:27.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:31:27 smithi100 bash[76202]: cluster 2024-04-07T17:31:26.327338+0000 mgr.y (mgr.25175) 2282 : cluster [DBG] pgmap v3806: 137 pgs: 137 active+clean; 103 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 28 op/s 2024-04-07T17:31:28.704 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:31:28 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:31:28] "GET /metrics HTTP/1.1" 200 36756 "" "Prometheus/2.43.0" 2024-04-07T17:31:29.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:31:29 smithi144 bash[66654]: cluster 2024-04-07T17:31:28.328116+0000 mgr.y (mgr.25175) 2283 : cluster [DBG] pgmap v3807: 137 pgs: 137 active+clean; 128 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 26 KiB/s rd, 4.1 MiB/s wr, 35 op/s 2024-04-07T17:31:29.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:31:29 smithi100 bash[74942]: cluster 2024-04-07T17:31:28.328116+0000 mgr.y (mgr.25175) 2283 : cluster [DBG] pgmap v3807: 137 pgs: 137 active+clean; 128 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 26 KiB/s rd, 4.1 MiB/s wr, 35 op/s 2024-04-07T17:31:29.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:31:29 smithi100 bash[76202]: cluster 2024-04-07T17:31:28.328116+0000 mgr.y (mgr.25175) 2283 : cluster [DBG] pgmap v3807: 137 pgs: 137 active+clean; 128 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 26 KiB/s rd, 4.1 MiB/s wr, 35 op/s 2024-04-07T17:31:31.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:31:31 smithi144 bash[66654]: cluster 2024-04-07T17:31:30.328816+0000 mgr.y (mgr.25175) 2284 : cluster [DBG] pgmap v3808: 137 pgs: 137 active+clean; 128 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 26 KiB/s rd, 4.1 MiB/s wr, 35 op/s 2024-04-07T17:31:31.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:31:31 smithi100 bash[74942]: cluster 2024-04-07T17:31:30.328816+0000 mgr.y (mgr.25175) 2284 : cluster [DBG] pgmap v3808: 137 pgs: 137 active+clean; 128 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 26 KiB/s rd, 4.1 MiB/s wr, 35 op/s 2024-04-07T17:31:31.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:31:31 smithi100 bash[76202]: cluster 2024-04-07T17:31:30.328816+0000 mgr.y (mgr.25175) 2284 : cluster [DBG] pgmap v3808: 137 pgs: 137 active+clean; 128 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 26 KiB/s rd, 4.1 MiB/s wr, 35 op/s 2024-04-07T17:31:32.608 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:31:32 smithi144 bash[95233]: ts=2024-04-07T17:31:32.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:31:32.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:31:32 smithi144 bash[66654]: audit 2024-04-07T17:31:31.848178+0000 mon.a (mon.0) 3761 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:31:32.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:31:32 smithi100 bash[74942]: audit 2024-04-07T17:31:31.848178+0000 mon.a (mon.0) 3761 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:31:32.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:31:32 smithi100 bash[76202]: audit 2024-04-07T17:31:31.848178+0000 mon.a (mon.0) 3761 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:31:33.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:31:33 smithi144 bash[66654]: cluster 2024-04-07T17:31:32.329591+0000 mgr.y (mgr.25175) 2285 : cluster [DBG] pgmap v3809: 137 pgs: 137 active+clean; 128 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 26 KiB/s rd, 4.1 MiB/s wr, 35 op/s 2024-04-07T17:31:33.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:31:33 smithi100 bash[74942]: cluster 2024-04-07T17:31:32.329591+0000 mgr.y (mgr.25175) 2285 : cluster [DBG] pgmap v3809: 137 pgs: 137 active+clean; 128 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 26 KiB/s rd, 4.1 MiB/s wr, 35 op/s 2024-04-07T17:31:33.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:31:33 smithi100 bash[76202]: cluster 2024-04-07T17:31:32.329591+0000 mgr.y (mgr.25175) 2285 : cluster [DBG] pgmap v3809: 137 pgs: 137 active+clean; 128 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 26 KiB/s rd, 4.1 MiB/s wr, 35 op/s 2024-04-07T17:31:35.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:31:35 smithi144 bash[66654]: cluster 2024-04-07T17:31:34.330310+0000 mgr.y (mgr.25175) 2286 : cluster [DBG] pgmap v3810: 137 pgs: 137 active+clean; 132 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 27 KiB/s rd, 4.2 MiB/s wr, 36 op/s 2024-04-07T17:31:35.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:31:35 smithi100 bash[74942]: cluster 2024-04-07T17:31:34.330310+0000 mgr.y (mgr.25175) 2286 : cluster [DBG] pgmap v3810: 137 pgs: 137 active+clean; 132 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 27 KiB/s rd, 4.2 MiB/s wr, 36 op/s 2024-04-07T17:31:35.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:31:35 smithi100 bash[76202]: cluster 2024-04-07T17:31:34.330310+0000 mgr.y (mgr.25175) 2286 : cluster [DBG] pgmap v3810: 137 pgs: 137 active+clean; 132 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 27 KiB/s rd, 4.2 MiB/s wr, 36 op/s 2024-04-07T17:31:37.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:31:37 smithi100 bash[74942]: cluster 2024-04-07T17:31:36.330912+0000 mgr.y (mgr.25175) 2287 : cluster [DBG] pgmap v3811: 137 pgs: 137 active+clean; 132 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 7.8 KiB/s rd, 2.4 MiB/s wr, 10 op/s 2024-04-07T17:31:37.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:31:37 smithi100 bash[76202]: cluster 2024-04-07T17:31:36.330912+0000 mgr.y (mgr.25175) 2287 : cluster [DBG] pgmap v3811: 137 pgs: 137 active+clean; 132 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 7.8 KiB/s rd, 2.4 MiB/s wr, 10 op/s 2024-04-07T17:31:38.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:31:37 smithi144 bash[66654]: cluster 2024-04-07T17:31:36.330912+0000 mgr.y (mgr.25175) 2287 : cluster [DBG] pgmap v3811: 137 pgs: 137 active+clean; 132 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 7.8 KiB/s rd, 2.4 MiB/s wr, 10 op/s 2024-04-07T17:31:38.704 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:31:38 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:31:38] "GET /metrics HTTP/1.1" 200 36756 "" "Prometheus/2.43.0" 2024-04-07T17:31:39.743 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] EncryptedFlattenTest/6.Grow (26341 ms) 2024-04-07T17:31:39.744 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] EncryptedFlattenTest/6.Shrink 2024-04-07T17:31:39.744 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:31:39.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:31:39 smithi100 bash[74942]: cluster 2024-04-07T17:31:38.331760+0000 mgr.y (mgr.25175) 2288 : cluster [DBG] pgmap v3812: 137 pgs: 137 active+clean; 152 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 2.1 MiB/s rd, 4.7 MiB/s wr, 18 op/s 2024-04-07T17:31:39.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:31:39 smithi100 bash[76202]: cluster 2024-04-07T17:31:38.331760+0000 mgr.y (mgr.25175) 2288 : cluster [DBG] pgmap v3812: 137 pgs: 137 active+clean; 152 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 2.1 MiB/s rd, 4.7 MiB/s wr, 18 op/s 2024-04-07T17:31:40.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:31:39 smithi144 bash[66654]: cluster 2024-04-07T17:31:38.331760+0000 mgr.y (mgr.25175) 2288 : cluster [DBG] pgmap v3812: 137 pgs: 137 active+clean; 152 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 2.1 MiB/s rd, 4.7 MiB/s wr, 18 op/s 2024-04-07T17:31:41.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:31:41 smithi100 bash[74942]: cluster 2024-04-07T17:31:40.332344+0000 mgr.y (mgr.25175) 2289 : cluster [DBG] pgmap v3813: 137 pgs: 137 active+clean; 160 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 5.1 MiB/s rd, 3.7 MiB/s wr, 12 op/s 2024-04-07T17:31:41.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:31:41 smithi100 bash[76202]: cluster 2024-04-07T17:31:40.332344+0000 mgr.y (mgr.25175) 2289 : cluster [DBG] pgmap v3813: 137 pgs: 137 active+clean; 160 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 5.1 MiB/s rd, 3.7 MiB/s wr, 12 op/s 2024-04-07T17:31:42.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:31:41 smithi144 bash[66654]: cluster 2024-04-07T17:31:40.332344+0000 mgr.y (mgr.25175) 2289 : cluster [DBG] pgmap v3813: 137 pgs: 137 active+clean; 160 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 5.1 MiB/s rd, 3.7 MiB/s wr, 12 op/s 2024-04-07T17:31:42.615 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:31:42 smithi144 bash[95233]: ts=2024-04-07T17:31:42.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:31:43.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:31:43 smithi100 bash[74942]: cluster 2024-04-07T17:31:42.333146+0000 mgr.y (mgr.25175) 2290 : cluster [DBG] pgmap v3814: 137 pgs: 137 active+clean; 169 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 8.4 MiB/s rd, 4.8 MiB/s wr, 27 op/s 2024-04-07T17:31:43.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:31:43 smithi100 bash[76202]: cluster 2024-04-07T17:31:42.333146+0000 mgr.y (mgr.25175) 2290 : cluster [DBG] pgmap v3814: 137 pgs: 137 active+clean; 169 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 8.4 MiB/s rd, 4.8 MiB/s wr, 27 op/s 2024-04-07T17:31:44.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:31:43 smithi144 bash[66654]: cluster 2024-04-07T17:31:42.333146+0000 mgr.y (mgr.25175) 2290 : cluster [DBG] pgmap v3814: 137 pgs: 137 active+clean; 169 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 8.4 MiB/s rd, 4.8 MiB/s wr, 27 op/s 2024-04-07T17:31:45.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:31:45 smithi100 bash[74942]: cluster 2024-04-07T17:31:44.333866+0000 mgr.y (mgr.25175) 2291 : cluster [DBG] pgmap v3815: 137 pgs: 137 active+clean; 169 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 9.8 MiB/s rd, 4.8 MiB/s wr, 30 op/s 2024-04-07T17:31:45.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:31:45 smithi100 bash[76202]: cluster 2024-04-07T17:31:44.333866+0000 mgr.y (mgr.25175) 2291 : cluster [DBG] pgmap v3815: 137 pgs: 137 active+clean; 169 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 9.8 MiB/s rd, 4.8 MiB/s wr, 30 op/s 2024-04-07T17:31:46.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:31:45 smithi144 bash[66654]: cluster 2024-04-07T17:31:44.333866+0000 mgr.y (mgr.25175) 2291 : cluster [DBG] pgmap v3815: 137 pgs: 137 active+clean; 169 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 9.8 MiB/s rd, 4.8 MiB/s wr, 30 op/s 2024-04-07T17:31:47.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:31:47 smithi100 bash[74942]: cluster 2024-04-07T17:31:46.334453+0000 mgr.y (mgr.25175) 2292 : cluster [DBG] pgmap v3816: 137 pgs: 137 active+clean; 169 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 9.7 MiB/s rd, 4.4 MiB/s wr, 27 op/s 2024-04-07T17:31:47.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:31:47 smithi100 bash[74942]: audit 2024-04-07T17:31:46.848473+0000 mon.a (mon.0) 3762 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:31:47.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:31:47 smithi100 bash[76202]: cluster 2024-04-07T17:31:46.334453+0000 mgr.y (mgr.25175) 2292 : cluster [DBG] pgmap v3816: 137 pgs: 137 active+clean; 169 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 9.7 MiB/s rd, 4.4 MiB/s wr, 27 op/s 2024-04-07T17:31:47.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:31:47 smithi100 bash[76202]: audit 2024-04-07T17:31:46.848473+0000 mon.a (mon.0) 3762 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:31:48.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:31:47 smithi144 bash[66654]: cluster 2024-04-07T17:31:46.334453+0000 mgr.y (mgr.25175) 2292 : cluster [DBG] pgmap v3816: 137 pgs: 137 active+clean; 169 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 9.7 MiB/s rd, 4.4 MiB/s wr, 27 op/s 2024-04-07T17:31:48.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:31:47 smithi144 bash[66654]: audit 2024-04-07T17:31:46.848473+0000 mon.a (mon.0) 3762 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:31:48.703 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:31:48 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:31:48] "GET /metrics HTTP/1.1" 200 36757 "" "Prometheus/2.43.0" 2024-04-07T17:31:49.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:31:49 smithi100 bash[74942]: cluster 2024-04-07T17:31:48.335258+0000 mgr.y (mgr.25175) 2293 : cluster [DBG] pgmap v3817: 137 pgs: 137 active+clean; 169 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 9.7 MiB/s rd, 4.4 MiB/s wr, 27 op/s 2024-04-07T17:31:49.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:31:49 smithi100 bash[76202]: cluster 2024-04-07T17:31:48.335258+0000 mgr.y (mgr.25175) 2293 : cluster [DBG] pgmap v3817: 137 pgs: 137 active+clean; 169 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 9.7 MiB/s rd, 4.4 MiB/s wr, 27 op/s 2024-04-07T17:31:50.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:31:49 smithi144 bash[66654]: cluster 2024-04-07T17:31:48.335258+0000 mgr.y (mgr.25175) 2293 : cluster [DBG] pgmap v3817: 137 pgs: 137 active+clean; 169 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 9.7 MiB/s rd, 4.4 MiB/s wr, 27 op/s 2024-04-07T17:31:50.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:31:50 smithi100 bash[74942]: cluster 2024-04-07T17:31:49.654558+0000 mon.a (mon.0) 3763 : cluster [DBG] osdmap e1950: 8 total, 8 up, 8 in 2024-04-07T17:31:50.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:31:50 smithi100 bash[76202]: cluster 2024-04-07T17:31:49.654558+0000 mon.a (mon.0) 3763 : cluster [DBG] osdmap e1950: 8 total, 8 up, 8 in 2024-04-07T17:31:51.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:31:50 smithi144 bash[66654]: cluster 2024-04-07T17:31:49.654558+0000 mon.a (mon.0) 3763 : cluster [DBG] osdmap e1950: 8 total, 8 up, 8 in 2024-04-07T17:31:51.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:31:51 smithi100 bash[74942]: cluster 2024-04-07T17:31:50.336044+0000 mgr.y (mgr.25175) 2294 : cluster [DBG] pgmap v3819: 137 pgs: 137 active+clean; 177 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 5.6 MiB/s rd, 2.1 MiB/s wr, 21 op/s 2024-04-07T17:31:51.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:31:51 smithi100 bash[76202]: cluster 2024-04-07T17:31:50.336044+0000 mgr.y (mgr.25175) 2294 : cluster [DBG] pgmap v3819: 137 pgs: 137 active+clean; 177 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 5.6 MiB/s rd, 2.1 MiB/s wr, 21 op/s 2024-04-07T17:31:52.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:31:51 smithi144 bash[66654]: cluster 2024-04-07T17:31:50.336044+0000 mgr.y (mgr.25175) 2294 : cluster [DBG] pgmap v3819: 137 pgs: 137 active+clean; 177 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 5.6 MiB/s rd, 2.1 MiB/s wr, 21 op/s 2024-04-07T17:31:52.615 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:31:52 smithi144 bash[95233]: ts=2024-04-07T17:31:52.317Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:31:53.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:31:53 smithi100 bash[74942]: cluster 2024-04-07T17:31:52.336862+0000 mgr.y (mgr.25175) 2295 : cluster [DBG] pgmap v3820: 137 pgs: 137 active+clean; 194 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 1.6 MiB/s rd, 2.5 MiB/s wr, 19 op/s 2024-04-07T17:31:53.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:31:53 smithi100 bash[76202]: cluster 2024-04-07T17:31:52.336862+0000 mgr.y (mgr.25175) 2295 : cluster [DBG] pgmap v3820: 137 pgs: 137 active+clean; 194 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 1.6 MiB/s rd, 2.5 MiB/s wr, 19 op/s 2024-04-07T17:31:54.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:31:53 smithi144 bash[66654]: cluster 2024-04-07T17:31:52.336862+0000 mgr.y (mgr.25175) 2295 : cluster [DBG] pgmap v3820: 137 pgs: 137 active+clean; 194 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 1.6 MiB/s rd, 2.5 MiB/s wr, 19 op/s 2024-04-07T17:31:55.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:31:55 smithi100 bash[74942]: cluster 2024-04-07T17:31:54.337582+0000 mgr.y (mgr.25175) 2296 : cluster [DBG] pgmap v3821: 137 pgs: 137 active+clean; 210 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 26 KiB/s rd, 4.1 MiB/s wr, 35 op/s 2024-04-07T17:31:55.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:31:55 smithi100 bash[76202]: cluster 2024-04-07T17:31:54.337582+0000 mgr.y (mgr.25175) 2296 : cluster [DBG] pgmap v3821: 137 pgs: 137 active+clean; 210 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 26 KiB/s rd, 4.1 MiB/s wr, 35 op/s 2024-04-07T17:31:56.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:31:55 smithi144 bash[66654]: cluster 2024-04-07T17:31:54.337582+0000 mgr.y (mgr.25175) 2296 : cluster [DBG] pgmap v3821: 137 pgs: 137 active+clean; 210 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 26 KiB/s rd, 4.1 MiB/s wr, 35 op/s 2024-04-07T17:31:57.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:31:57 smithi100 bash[74942]: cluster 2024-04-07T17:31:56.338181+0000 mgr.y (mgr.25175) 2297 : cluster [DBG] pgmap v3822: 137 pgs: 137 active+clean; 210 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 26 KiB/s rd, 4.1 MiB/s wr, 35 op/s 2024-04-07T17:31:57.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:31:57 smithi100 bash[76202]: cluster 2024-04-07T17:31:56.338181+0000 mgr.y (mgr.25175) 2297 : cluster [DBG] pgmap v3822: 137 pgs: 137 active+clean; 210 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 26 KiB/s rd, 4.1 MiB/s wr, 35 op/s 2024-04-07T17:31:58.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:31:57 smithi144 bash[66654]: cluster 2024-04-07T17:31:56.338181+0000 mgr.y (mgr.25175) 2297 : cluster [DBG] pgmap v3822: 137 pgs: 137 active+clean; 210 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 26 KiB/s rd, 4.1 MiB/s wr, 35 op/s 2024-04-07T17:31:58.704 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:31:58 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:31:58] "GET /metrics HTTP/1.1" 200 36757 "" "Prometheus/2.43.0" 2024-04-07T17:31:59.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:31:59 smithi100 bash[74942]: cluster 2024-04-07T17:31:58.338998+0000 mgr.y (mgr.25175) 2298 : cluster [DBG] pgmap v3823: 137 pgs: 137 active+clean; 210 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 26 KiB/s rd, 4.1 MiB/s wr, 35 op/s 2024-04-07T17:31:59.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:31:59 smithi100 bash[76202]: cluster 2024-04-07T17:31:58.338998+0000 mgr.y (mgr.25175) 2298 : cluster [DBG] pgmap v3823: 137 pgs: 137 active+clean; 210 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 26 KiB/s rd, 4.1 MiB/s wr, 35 op/s 2024-04-07T17:32:00.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:31:59 smithi144 bash[66654]: cluster 2024-04-07T17:31:58.338998+0000 mgr.y (mgr.25175) 2298 : cluster [DBG] pgmap v3823: 137 pgs: 137 active+clean; 210 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 26 KiB/s rd, 4.1 MiB/s wr, 35 op/s 2024-04-07T17:32:01.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:32:01 smithi100 bash[74942]: cluster 2024-04-07T17:32:00.339739+0000 mgr.y (mgr.25175) 2299 : cluster [DBG] pgmap v3824: 137 pgs: 137 active+clean; 214 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 56 KiB/s rd, 3.8 MiB/s wr, 40 op/s 2024-04-07T17:32:01.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:32:01 smithi100 bash[76202]: cluster 2024-04-07T17:32:00.339739+0000 mgr.y (mgr.25175) 2299 : cluster [DBG] pgmap v3824: 137 pgs: 137 active+clean; 214 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 56 KiB/s rd, 3.8 MiB/s wr, 40 op/s 2024-04-07T17:32:02.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:32:01 smithi144 bash[66654]: cluster 2024-04-07T17:32:00.339739+0000 mgr.y (mgr.25175) 2299 : cluster [DBG] pgmap v3824: 137 pgs: 137 active+clean; 214 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 56 KiB/s rd, 3.8 MiB/s wr, 40 op/s 2024-04-07T17:32:02.615 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:32:02 smithi144 bash[95233]: ts=2024-04-07T17:32:02.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:32:02.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:32:02 smithi100 bash[74942]: audit 2024-04-07T17:32:01.848829+0000 mon.a (mon.0) 3764 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:32:02.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:32:02 smithi100 bash[76202]: audit 2024-04-07T17:32:01.848829+0000 mon.a (mon.0) 3764 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:32:03.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:32:02 smithi144 bash[66654]: audit 2024-04-07T17:32:01.848829+0000 mon.a (mon.0) 3764 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:32:03.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:32:03 smithi100 bash[74942]: cluster 2024-04-07T17:32:02.340489+0000 mgr.y (mgr.25175) 2300 : cluster [DBG] pgmap v3825: 137 pgs: 137 active+clean; 226 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 52 KiB/s rd, 4.1 MiB/s wr, 38 op/s 2024-04-07T17:32:03.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:32:03 smithi100 bash[76202]: cluster 2024-04-07T17:32:02.340489+0000 mgr.y (mgr.25175) 2300 : cluster [DBG] pgmap v3825: 137 pgs: 137 active+clean; 226 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 52 KiB/s rd, 4.1 MiB/s wr, 38 op/s 2024-04-07T17:32:04.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:32:03 smithi144 bash[66654]: cluster 2024-04-07T17:32:02.340489+0000 mgr.y (mgr.25175) 2300 : cluster [DBG] pgmap v3825: 137 pgs: 137 active+clean; 226 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 52 KiB/s rd, 4.1 MiB/s wr, 38 op/s 2024-04-07T17:32:05.066 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] EncryptedFlattenTest/6.Shrink (25322 ms) 2024-04-07T17:32:05.066 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] EncryptedFlattenTest/6.ShrinkToOne 2024-04-07T17:32:05.066 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:32:05.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:32:05 smithi100 bash[74942]: cluster 2024-04-07T17:32:04.341202+0000 mgr.y (mgr.25175) 2301 : cluster [DBG] pgmap v3826: 137 pgs: 137 active+clean; 235 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 2.6 MiB/s rd, 3.8 MiB/s wr, 33 op/s 2024-04-07T17:32:05.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:32:05 smithi100 bash[76202]: cluster 2024-04-07T17:32:04.341202+0000 mgr.y (mgr.25175) 2301 : cluster [DBG] pgmap v3826: 137 pgs: 137 active+clean; 235 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 2.6 MiB/s rd, 3.8 MiB/s wr, 33 op/s 2024-04-07T17:32:06.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:32:05 smithi144 bash[66654]: cluster 2024-04-07T17:32:04.341202+0000 mgr.y (mgr.25175) 2301 : cluster [DBG] pgmap v3826: 137 pgs: 137 active+clean; 235 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 2.6 MiB/s rd, 3.8 MiB/s wr, 33 op/s 2024-04-07T17:32:06.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:32:06 smithi100 bash[74942]: audit 2024-04-07T17:32:06.319458+0000 mon.a (mon.0) 3765 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:32:06.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:32:06 smithi100 bash[76202]: audit 2024-04-07T17:32:06.319458+0000 mon.a (mon.0) 3765 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:32:07.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:32:06 smithi144 bash[66654]: audit 2024-04-07T17:32:06.319458+0000 mon.a (mon.0) 3765 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:32:07.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:32:07 smithi100 bash[76202]: cluster 2024-04-07T17:32:06.341734+0000 mgr.y (mgr.25175) 2302 : cluster [DBG] pgmap v3827: 137 pgs: 137 active+clean; 239 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 3.6 MiB/s rd, 2.8 MiB/s wr, 17 op/s 2024-04-07T17:32:07.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:32:07 smithi100 bash[76202]: audit 2024-04-07T17:32:07.059225+0000 mon.a (mon.0) 3766 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:32:07.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:32:07 smithi100 bash[76202]: audit 2024-04-07T17:32:07.060340+0000 mon.a (mon.0) 3767 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:32:07.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:32:07 smithi100 bash[76202]: audit 2024-04-07T17:32:07.067306+0000 mon.a (mon.0) 3768 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:32:07.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:32:07 smithi100 bash[74942]: cluster 2024-04-07T17:32:06.341734+0000 mgr.y (mgr.25175) 2302 : cluster [DBG] pgmap v3827: 137 pgs: 137 active+clean; 239 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 3.6 MiB/s rd, 2.8 MiB/s wr, 17 op/s 2024-04-07T17:32:07.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:32:07 smithi100 bash[74942]: audit 2024-04-07T17:32:07.059225+0000 mon.a (mon.0) 3766 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:32:07.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:32:07 smithi100 bash[74942]: audit 2024-04-07T17:32:07.060340+0000 mon.a (mon.0) 3767 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:32:07.955 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:32:07 smithi100 bash[74942]: audit 2024-04-07T17:32:07.067306+0000 mon.a (mon.0) 3768 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:32:08.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:32:07 smithi144 bash[66654]: cluster 2024-04-07T17:32:06.341734+0000 mgr.y (mgr.25175) 2302 : cluster [DBG] pgmap v3827: 137 pgs: 137 active+clean; 239 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 3.6 MiB/s rd, 2.8 MiB/s wr, 17 op/s 2024-04-07T17:32:08.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:32:07 smithi144 bash[66654]: audit 2024-04-07T17:32:07.059225+0000 mon.a (mon.0) 3766 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:32:08.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:32:07 smithi144 bash[66654]: audit 2024-04-07T17:32:07.060340+0000 mon.a (mon.0) 3767 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:32:08.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:32:07 smithi144 bash[66654]: audit 2024-04-07T17:32:07.067306+0000 mon.a (mon.0) 3768 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:32:08.703 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:32:08 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:32:08] "GET /metrics HTTP/1.1" 200 36756 "" "Prometheus/2.43.0" 2024-04-07T17:32:09.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:32:09 smithi100 bash[74942]: cluster 2024-04-07T17:32:08.343109+0000 mgr.y (mgr.25175) 2303 : cluster [DBG] pgmap v3828: 137 pgs: 137 active+clean; 251 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 8.7 MiB/s rd, 4.8 MiB/s wr, 29 op/s 2024-04-07T17:32:09.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:32:09 smithi100 bash[76202]: cluster 2024-04-07T17:32:08.343109+0000 mgr.y (mgr.25175) 2303 : cluster [DBG] pgmap v3828: 137 pgs: 137 active+clean; 251 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 8.7 MiB/s rd, 4.8 MiB/s wr, 29 op/s 2024-04-07T17:32:10.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:32:09 smithi144 bash[66654]: cluster 2024-04-07T17:32:08.343109+0000 mgr.y (mgr.25175) 2303 : cluster [DBG] pgmap v3828: 137 pgs: 137 active+clean; 251 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 8.7 MiB/s rd, 4.8 MiB/s wr, 29 op/s 2024-04-07T17:32:11.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:32:11 smithi100 bash[74942]: cluster 2024-04-07T17:32:10.344233+0000 mgr.y (mgr.25175) 2304 : cluster [DBG] pgmap v3829: 137 pgs: 137 active+clean; 251 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 9.8 MiB/s rd, 4.8 MiB/s wr, 31 op/s 2024-04-07T17:32:11.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:32:11 smithi100 bash[76202]: cluster 2024-04-07T17:32:10.344233+0000 mgr.y (mgr.25175) 2304 : cluster [DBG] pgmap v3829: 137 pgs: 137 active+clean; 251 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 9.8 MiB/s rd, 4.8 MiB/s wr, 31 op/s 2024-04-07T17:32:12.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:32:11 smithi144 bash[66654]: cluster 2024-04-07T17:32:10.344233+0000 mgr.y (mgr.25175) 2304 : cluster [DBG] pgmap v3829: 137 pgs: 137 active+clean; 251 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 9.8 MiB/s rd, 4.8 MiB/s wr, 31 op/s 2024-04-07T17:32:12.616 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:32:12 smithi144 bash[95233]: ts=2024-04-07T17:32:12.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:32:14.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:32:13 smithi144 bash[66654]: cluster 2024-04-07T17:32:12.345039+0000 mgr.y (mgr.25175) 2305 : cluster [DBG] pgmap v3830: 137 pgs: 137 active+clean; 251 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 9.7 MiB/s rd, 4.4 MiB/s wr, 24 op/s 2024-04-07T17:32:14.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:32:13 smithi100 bash[74942]: cluster 2024-04-07T17:32:12.345039+0000 mgr.y (mgr.25175) 2305 : cluster [DBG] pgmap v3830: 137 pgs: 137 active+clean; 251 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 9.7 MiB/s rd, 4.4 MiB/s wr, 24 op/s 2024-04-07T17:32:14.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:32:13 smithi100 bash[76202]: cluster 2024-04-07T17:32:12.345039+0000 mgr.y (mgr.25175) 2305 : cluster [DBG] pgmap v3830: 137 pgs: 137 active+clean; 251 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 9.7 MiB/s rd, 4.4 MiB/s wr, 24 op/s 2024-04-07T17:32:16.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:32:15 smithi144 bash[66654]: cluster 2024-04-07T17:32:14.346122+0000 mgr.y (mgr.25175) 2306 : cluster [DBG] pgmap v3831: 137 pgs: 137 active+clean; 255 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 9.7 MiB/s rd, 3.8 MiB/s wr, 22 op/s 2024-04-07T17:32:16.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:32:15 smithi144 bash[66654]: cluster 2024-04-07T17:32:14.717136+0000 mon.a (mon.0) 3769 : cluster [DBG] osdmap e1951: 8 total, 8 up, 8 in 2024-04-07T17:32:16.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:32:15 smithi100 bash[74942]: cluster 2024-04-07T17:32:14.346122+0000 mgr.y (mgr.25175) 2306 : cluster [DBG] pgmap v3831: 137 pgs: 137 active+clean; 255 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 9.7 MiB/s rd, 3.8 MiB/s wr, 22 op/s 2024-04-07T17:32:16.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:32:15 smithi100 bash[74942]: cluster 2024-04-07T17:32:14.717136+0000 mon.a (mon.0) 3769 : cluster [DBG] osdmap e1951: 8 total, 8 up, 8 in 2024-04-07T17:32:16.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:32:15 smithi100 bash[76202]: cluster 2024-04-07T17:32:14.346122+0000 mgr.y (mgr.25175) 2306 : cluster [DBG] pgmap v3831: 137 pgs: 137 active+clean; 255 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 9.7 MiB/s rd, 3.8 MiB/s wr, 22 op/s 2024-04-07T17:32:16.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:32:15 smithi100 bash[76202]: cluster 2024-04-07T17:32:14.717136+0000 mon.a (mon.0) 3769 : cluster [DBG] osdmap e1951: 8 total, 8 up, 8 in 2024-04-07T17:32:18.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:32:17 smithi144 bash[66654]: cluster 2024-04-07T17:32:16.346947+0000 mgr.y (mgr.25175) 2307 : cluster [DBG] pgmap v3833: 137 pgs: 137 active+clean; 259 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 7.4 MiB/s rd, 3.2 MiB/s wr, 25 op/s 2024-04-07T17:32:18.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:32:17 smithi144 bash[66654]: audit 2024-04-07T17:32:16.849122+0000 mon.a (mon.0) 3770 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:32:18.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:32:17 smithi100 bash[74942]: cluster 2024-04-07T17:32:16.346947+0000 mgr.y (mgr.25175) 2307 : cluster [DBG] pgmap v3833: 137 pgs: 137 active+clean; 259 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 7.4 MiB/s rd, 3.2 MiB/s wr, 25 op/s 2024-04-07T17:32:18.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:32:17 smithi100 bash[74942]: audit 2024-04-07T17:32:16.849122+0000 mon.a (mon.0) 3770 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:32:18.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:32:17 smithi100 bash[76202]: cluster 2024-04-07T17:32:16.346947+0000 mgr.y (mgr.25175) 2307 : cluster [DBG] pgmap v3833: 137 pgs: 137 active+clean; 259 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 7.4 MiB/s rd, 3.2 MiB/s wr, 25 op/s 2024-04-07T17:32:18.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:32:17 smithi100 bash[76202]: audit 2024-04-07T17:32:16.849122+0000 mon.a (mon.0) 3770 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:32:18.704 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:32:18 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:32:18] "GET /metrics HTTP/1.1" 200 36755 "" "Prometheus/2.43.0" 2024-04-07T17:32:20.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:32:19 smithi144 bash[66654]: cluster 2024-04-07T17:32:18.347863+0000 mgr.y (mgr.25175) 2308 : cluster [DBG] pgmap v3834: 137 pgs: 137 active+clean; 288 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 1.3 MiB/s rd, 3.7 MiB/s wr, 36 op/s 2024-04-07T17:32:20.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:32:19 smithi100 bash[74942]: cluster 2024-04-07T17:32:18.347863+0000 mgr.y (mgr.25175) 2308 : cluster [DBG] pgmap v3834: 137 pgs: 137 active+clean; 288 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 1.3 MiB/s rd, 3.7 MiB/s wr, 36 op/s 2024-04-07T17:32:20.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:32:19 smithi100 bash[76202]: cluster 2024-04-07T17:32:18.347863+0000 mgr.y (mgr.25175) 2308 : cluster [DBG] pgmap v3834: 137 pgs: 137 active+clean; 288 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 1.3 MiB/s rd, 3.7 MiB/s wr, 36 op/s 2024-04-07T17:32:22.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:32:21 smithi144 bash[66654]: cluster 2024-04-07T17:32:20.348591+0000 mgr.y (mgr.25175) 2309 : cluster [DBG] pgmap v3835: 137 pgs: 137 active+clean; 292 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 26 KiB/s rd, 4.1 MiB/s wr, 35 op/s 2024-04-07T17:32:22.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:32:21 smithi100 bash[74942]: cluster 2024-04-07T17:32:20.348591+0000 mgr.y (mgr.25175) 2309 : cluster [DBG] pgmap v3835: 137 pgs: 137 active+clean; 292 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 26 KiB/s rd, 4.1 MiB/s wr, 35 op/s 2024-04-07T17:32:22.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:32:21 smithi100 bash[76202]: cluster 2024-04-07T17:32:20.348591+0000 mgr.y (mgr.25175) 2309 : cluster [DBG] pgmap v3835: 137 pgs: 137 active+clean; 292 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 26 KiB/s rd, 4.1 MiB/s wr, 35 op/s 2024-04-07T17:32:22.615 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:32:22 smithi144 bash[95233]: ts=2024-04-07T17:32:22.317Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:32:24.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:32:23 smithi144 bash[66654]: cluster 2024-04-07T17:32:22.349505+0000 mgr.y (mgr.25175) 2310 : cluster [DBG] pgmap v3836: 137 pgs: 137 active+clean; 292 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 26 KiB/s rd, 4.1 MiB/s wr, 35 op/s 2024-04-07T17:32:24.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:32:23 smithi100 bash[74942]: cluster 2024-04-07T17:32:22.349505+0000 mgr.y (mgr.25175) 2310 : cluster [DBG] pgmap v3836: 137 pgs: 137 active+clean; 292 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 26 KiB/s rd, 4.1 MiB/s wr, 35 op/s 2024-04-07T17:32:24.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:32:23 smithi100 bash[76202]: cluster 2024-04-07T17:32:22.349505+0000 mgr.y (mgr.25175) 2310 : cluster [DBG] pgmap v3836: 137 pgs: 137 active+clean; 292 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 26 KiB/s rd, 4.1 MiB/s wr, 35 op/s 2024-04-07T17:32:26.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:32:25 smithi144 bash[66654]: cluster 2024-04-07T17:32:24.350404+0000 mgr.y (mgr.25175) 2311 : cluster [DBG] pgmap v3837: 137 pgs: 137 active+clean; 292 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 26 KiB/s rd, 3.7 MiB/s wr, 35 op/s 2024-04-07T17:32:26.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:32:25 smithi100 bash[74942]: cluster 2024-04-07T17:32:24.350404+0000 mgr.y (mgr.25175) 2311 : cluster [DBG] pgmap v3837: 137 pgs: 137 active+clean; 292 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 26 KiB/s rd, 3.7 MiB/s wr, 35 op/s 2024-04-07T17:32:26.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:32:25 smithi100 bash[76202]: cluster 2024-04-07T17:32:24.350404+0000 mgr.y (mgr.25175) 2311 : cluster [DBG] pgmap v3837: 137 pgs: 137 active+clean; 292 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 26 KiB/s rd, 3.7 MiB/s wr, 35 op/s 2024-04-07T17:32:28.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:32:27 smithi144 bash[66654]: cluster 2024-04-07T17:32:26.351282+0000 mgr.y (mgr.25175) 2312 : cluster [DBG] pgmap v3838: 137 pgs: 137 active+clean; 292 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 22 KiB/s rd, 3.2 MiB/s wr, 30 op/s 2024-04-07T17:32:28.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:32:27 smithi100 bash[74942]: cluster 2024-04-07T17:32:26.351282+0000 mgr.y (mgr.25175) 2312 : cluster [DBG] pgmap v3838: 137 pgs: 137 active+clean; 292 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 22 KiB/s rd, 3.2 MiB/s wr, 30 op/s 2024-04-07T17:32:28.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:32:27 smithi100 bash[76202]: cluster 2024-04-07T17:32:26.351282+0000 mgr.y (mgr.25175) 2312 : cluster [DBG] pgmap v3838: 137 pgs: 137 active+clean; 292 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 22 KiB/s rd, 3.2 MiB/s wr, 30 op/s 2024-04-07T17:32:28.704 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:32:28 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:32:28] "GET /metrics HTTP/1.1" 200 36755 "" "Prometheus/2.43.0" 2024-04-07T17:32:30.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:32:29 smithi144 bash[66654]: cluster 2024-04-07T17:32:28.351981+0000 mgr.y (mgr.25175) 2313 : cluster [DBG] pgmap v3839: 137 pgs: 137 active+clean; 304 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 71 KiB/s rd, 3.7 MiB/s wr, 31 op/s 2024-04-07T17:32:30.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:32:29 smithi100 bash[74942]: cluster 2024-04-07T17:32:28.351981+0000 mgr.y (mgr.25175) 2313 : cluster [DBG] pgmap v3839: 137 pgs: 137 active+clean; 304 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 71 KiB/s rd, 3.7 MiB/s wr, 31 op/s 2024-04-07T17:32:30.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:32:29 smithi100 bash[76202]: cluster 2024-04-07T17:32:28.351981+0000 mgr.y (mgr.25175) 2313 : cluster [DBG] pgmap v3839: 137 pgs: 137 active+clean; 304 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 71 KiB/s rd, 3.7 MiB/s wr, 31 op/s 2024-04-07T17:32:30.713 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] EncryptedFlattenTest/6.ShrinkToOne (25647 ms) 2024-04-07T17:32:30.713 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] EncryptedFlattenTest/6.ShrinkToOneAfterSnapshot 2024-04-07T17:32:30.713 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:32:32.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:32:31 smithi144 bash[66654]: cluster 2024-04-07T17:32:30.352665+0000 mgr.y (mgr.25175) 2314 : cluster [DBG] pgmap v3840: 137 pgs: 137 active+clean; 312 MiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 738 KiB/s rd, 2.7 MiB/s wr, 10 op/s 2024-04-07T17:32:32.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:32:31 smithi100 bash[74942]: cluster 2024-04-07T17:32:30.352665+0000 mgr.y (mgr.25175) 2314 : cluster [DBG] pgmap v3840: 137 pgs: 137 active+clean; 312 MiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 738 KiB/s rd, 2.7 MiB/s wr, 10 op/s 2024-04-07T17:32:32.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:32:31 smithi100 bash[76202]: cluster 2024-04-07T17:32:30.352665+0000 mgr.y (mgr.25175) 2314 : cluster [DBG] pgmap v3840: 137 pgs: 137 active+clean; 312 MiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 738 KiB/s rd, 2.7 MiB/s wr, 10 op/s 2024-04-07T17:32:32.615 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:32:32 smithi144 bash[95233]: ts=2024-04-07T17:32:32.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:32:33.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:32:32 smithi144 bash[66654]: audit 2024-04-07T17:32:31.850368+0000 mon.a (mon.0) 3771 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:32:33.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:32:32 smithi100 bash[74942]: audit 2024-04-07T17:32:31.850368+0000 mon.a (mon.0) 3771 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:32:33.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:32:32 smithi100 bash[76202]: audit 2024-04-07T17:32:31.850368+0000 mon.a (mon.0) 3771 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:32:34.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:32:33 smithi144 bash[66654]: cluster 2024-04-07T17:32:32.353533+0000 mgr.y (mgr.25175) 2315 : cluster [DBG] pgmap v3841: 137 pgs: 137 active+clean; 312 MiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 1.1 MiB/s rd, 2.7 MiB/s wr, 15 op/s 2024-04-07T17:32:34.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:32:33 smithi100 bash[74942]: cluster 2024-04-07T17:32:32.353533+0000 mgr.y (mgr.25175) 2315 : cluster [DBG] pgmap v3841: 137 pgs: 137 active+clean; 312 MiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 1.1 MiB/s rd, 2.7 MiB/s wr, 15 op/s 2024-04-07T17:32:34.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:32:33 smithi100 bash[76202]: cluster 2024-04-07T17:32:32.353533+0000 mgr.y (mgr.25175) 2315 : cluster [DBG] pgmap v3841: 137 pgs: 137 active+clean; 312 MiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 1.1 MiB/s rd, 2.7 MiB/s wr, 15 op/s 2024-04-07T17:32:36.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:32:35 smithi144 bash[66654]: cluster 2024-04-07T17:32:34.354261+0000 mgr.y (mgr.25175) 2316 : cluster [DBG] pgmap v3842: 137 pgs: 137 active+clean; 312 MiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 1.8 MiB/s rd, 3.3 MiB/s wr, 23 op/s 2024-04-07T17:32:36.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:32:35 smithi100 bash[74942]: cluster 2024-04-07T17:32:34.354261+0000 mgr.y (mgr.25175) 2316 : cluster [DBG] pgmap v3842: 137 pgs: 137 active+clean; 312 MiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 1.8 MiB/s rd, 3.3 MiB/s wr, 23 op/s 2024-04-07T17:32:36.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:32:35 smithi100 bash[76202]: cluster 2024-04-07T17:32:34.354261+0000 mgr.y (mgr.25175) 2316 : cluster [DBG] pgmap v3842: 137 pgs: 137 active+clean; 312 MiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 1.8 MiB/s rd, 3.3 MiB/s wr, 23 op/s 2024-04-07T17:32:38.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:32:37 smithi144 bash[66654]: cluster 2024-04-07T17:32:36.354850+0000 mgr.y (mgr.25175) 2317 : cluster [DBG] pgmap v3843: 137 pgs: 137 active+clean; 312 MiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 1.8 MiB/s rd, 3.3 MiB/s wr, 23 op/s 2024-04-07T17:32:38.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:32:37 smithi100 bash[74942]: cluster 2024-04-07T17:32:36.354850+0000 mgr.y (mgr.25175) 2317 : cluster [DBG] pgmap v3843: 137 pgs: 137 active+clean; 312 MiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 1.8 MiB/s rd, 3.3 MiB/s wr, 23 op/s 2024-04-07T17:32:38.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:32:37 smithi100 bash[76202]: cluster 2024-04-07T17:32:36.354850+0000 mgr.y (mgr.25175) 2317 : cluster [DBG] pgmap v3843: 137 pgs: 137 active+clean; 312 MiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 1.8 MiB/s rd, 3.3 MiB/s wr, 23 op/s 2024-04-07T17:32:38.703 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:32:38 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:32:38] "GET /metrics HTTP/1.1" 200 36760 "" "Prometheus/2.43.0" 2024-04-07T17:32:40.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:32:39 smithi144 bash[66654]: cluster 2024-04-07T17:32:38.355674+0000 mgr.y (mgr.25175) 2318 : cluster [DBG] pgmap v3844: 137 pgs: 137 active+clean; 312 MiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 1.8 MiB/s rd, 3.3 MiB/s wr, 23 op/s 2024-04-07T17:32:40.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:32:39 smithi100 bash[74942]: cluster 2024-04-07T17:32:38.355674+0000 mgr.y (mgr.25175) 2318 : cluster [DBG] pgmap v3844: 137 pgs: 137 active+clean; 312 MiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 1.8 MiB/s rd, 3.3 MiB/s wr, 23 op/s 2024-04-07T17:32:40.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:32:39 smithi100 bash[76202]: cluster 2024-04-07T17:32:38.355674+0000 mgr.y (mgr.25175) 2318 : cluster [DBG] pgmap v3844: 137 pgs: 137 active+clean; 312 MiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 1.8 MiB/s rd, 3.3 MiB/s wr, 23 op/s 2024-04-07T17:32:42.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:32:41 smithi144 bash[66654]: cluster 2024-04-07T17:32:40.356727+0000 mgr.y (mgr.25175) 2319 : cluster [DBG] pgmap v3845: 137 pgs: 137 active+clean; 312 MiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 1.7 MiB/s rd, 2.3 MiB/s wr, 14 op/s 2024-04-07T17:32:42.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:32:41 smithi144 bash[66654]: cluster 2024-04-07T17:32:40.778611+0000 mon.a (mon.0) 3772 : cluster [DBG] osdmap e1952: 8 total, 8 up, 8 in 2024-04-07T17:32:42.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:32:41 smithi100 bash[74942]: cluster 2024-04-07T17:32:40.356727+0000 mgr.y (mgr.25175) 2319 : cluster [DBG] pgmap v3845: 137 pgs: 137 active+clean; 312 MiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 1.7 MiB/s rd, 2.3 MiB/s wr, 14 op/s 2024-04-07T17:32:42.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:32:41 smithi100 bash[74942]: cluster 2024-04-07T17:32:40.778611+0000 mon.a (mon.0) 3772 : cluster [DBG] osdmap e1952: 8 total, 8 up, 8 in 2024-04-07T17:32:42.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:32:41 smithi100 bash[76202]: cluster 2024-04-07T17:32:40.356727+0000 mgr.y (mgr.25175) 2319 : cluster [DBG] pgmap v3845: 137 pgs: 137 active+clean; 312 MiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 1.7 MiB/s rd, 2.3 MiB/s wr, 14 op/s 2024-04-07T17:32:42.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:32:41 smithi100 bash[76202]: cluster 2024-04-07T17:32:40.778611+0000 mon.a (mon.0) 3772 : cluster [DBG] osdmap e1952: 8 total, 8 up, 8 in 2024-04-07T17:32:42.615 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:32:42 smithi144 bash[95233]: ts=2024-04-07T17:32:42.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:32:44.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:32:43 smithi144 bash[66654]: cluster 2024-04-07T17:32:42.357641+0000 mgr.y (mgr.25175) 2320 : cluster [DBG] pgmap v3847: 137 pgs: 137 active+clean; 324 MiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 845 KiB/s rd, 2.0 MiB/s wr, 33 op/s 2024-04-07T17:32:44.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:32:43 smithi100 bash[74942]: cluster 2024-04-07T17:32:42.357641+0000 mgr.y (mgr.25175) 2320 : cluster [DBG] pgmap v3847: 137 pgs: 137 active+clean; 324 MiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 845 KiB/s rd, 2.0 MiB/s wr, 33 op/s 2024-04-07T17:32:44.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:32:43 smithi100 bash[76202]: cluster 2024-04-07T17:32:42.357641+0000 mgr.y (mgr.25175) 2320 : cluster [DBG] pgmap v3847: 137 pgs: 137 active+clean; 324 MiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 845 KiB/s rd, 2.0 MiB/s wr, 33 op/s 2024-04-07T17:32:46.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:32:45 smithi144 bash[66654]: cluster 2024-04-07T17:32:44.358395+0000 mgr.y (mgr.25175) 2321 : cluster [DBG] pgmap v3848: 137 pgs: 137 active+clean; 352 MiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 26 KiB/s rd, 4.0 MiB/s wr, 35 op/s 2024-04-07T17:32:46.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:32:45 smithi100 bash[74942]: cluster 2024-04-07T17:32:44.358395+0000 mgr.y (mgr.25175) 2321 : cluster [DBG] pgmap v3848: 137 pgs: 137 active+clean; 352 MiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 26 KiB/s rd, 4.0 MiB/s wr, 35 op/s 2024-04-07T17:32:46.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:32:45 smithi100 bash[76202]: cluster 2024-04-07T17:32:44.358395+0000 mgr.y (mgr.25175) 2321 : cluster [DBG] pgmap v3848: 137 pgs: 137 active+clean; 352 MiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 26 KiB/s rd, 4.0 MiB/s wr, 35 op/s 2024-04-07T17:32:48.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:32:47 smithi144 bash[66654]: cluster 2024-04-07T17:32:46.358997+0000 mgr.y (mgr.25175) 2322 : cluster [DBG] pgmap v3849: 137 pgs: 137 active+clean; 353 MiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 26 KiB/s rd, 4.1 MiB/s wr, 35 op/s 2024-04-07T17:32:48.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:32:47 smithi144 bash[66654]: audit 2024-04-07T17:32:46.850069+0000 mon.a (mon.0) 3773 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:32:48.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:32:47 smithi100 bash[74942]: cluster 2024-04-07T17:32:46.358997+0000 mgr.y (mgr.25175) 2322 : cluster [DBG] pgmap v3849: 137 pgs: 137 active+clean; 353 MiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 26 KiB/s rd, 4.1 MiB/s wr, 35 op/s 2024-04-07T17:32:48.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:32:47 smithi100 bash[74942]: audit 2024-04-07T17:32:46.850069+0000 mon.a (mon.0) 3773 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:32:48.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:32:47 smithi100 bash[76202]: cluster 2024-04-07T17:32:46.358997+0000 mgr.y (mgr.25175) 2322 : cluster [DBG] pgmap v3849: 137 pgs: 137 active+clean; 353 MiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 26 KiB/s rd, 4.1 MiB/s wr, 35 op/s 2024-04-07T17:32:48.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:32:47 smithi100 bash[76202]: audit 2024-04-07T17:32:46.850069+0000 mon.a (mon.0) 3773 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:32:48.703 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:32:48 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:32:48] "GET /metrics HTTP/1.1" 200 36762 "" "Prometheus/2.43.0" 2024-04-07T17:32:50.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:32:49 smithi144 bash[66654]: cluster 2024-04-07T17:32:48.359840+0000 mgr.y (mgr.25175) 2323 : cluster [DBG] pgmap v3850: 137 pgs: 137 active+clean; 353 MiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 26 KiB/s rd, 4.1 MiB/s wr, 35 op/s 2024-04-07T17:32:50.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:32:49 smithi100 bash[74942]: cluster 2024-04-07T17:32:48.359840+0000 mgr.y (mgr.25175) 2323 : cluster [DBG] pgmap v3850: 137 pgs: 137 active+clean; 353 MiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 26 KiB/s rd, 4.1 MiB/s wr, 35 op/s 2024-04-07T17:32:50.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:32:49 smithi100 bash[76202]: cluster 2024-04-07T17:32:48.359840+0000 mgr.y (mgr.25175) 2323 : cluster [DBG] pgmap v3850: 137 pgs: 137 active+clean; 353 MiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 26 KiB/s rd, 4.1 MiB/s wr, 35 op/s 2024-04-07T17:32:52.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:32:51 smithi144 bash[66654]: cluster 2024-04-07T17:32:50.360540+0000 mgr.y (mgr.25175) 2324 : cluster [DBG] pgmap v3851: 137 pgs: 137 active+clean; 353 MiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 26 KiB/s rd, 4.1 MiB/s wr, 35 op/s 2024-04-07T17:32:52.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:32:51 smithi100 bash[74942]: cluster 2024-04-07T17:32:50.360540+0000 mgr.y (mgr.25175) 2324 : cluster [DBG] pgmap v3851: 137 pgs: 137 active+clean; 353 MiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 26 KiB/s rd, 4.1 MiB/s wr, 35 op/s 2024-04-07T17:32:52.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:32:51 smithi100 bash[76202]: cluster 2024-04-07T17:32:50.360540+0000 mgr.y (mgr.25175) 2324 : cluster [DBG] pgmap v3851: 137 pgs: 137 active+clean; 353 MiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 26 KiB/s rd, 4.1 MiB/s wr, 35 op/s 2024-04-07T17:32:52.615 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:32:52 smithi144 bash[95233]: ts=2024-04-07T17:32:52.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:32:54.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:32:53 smithi144 bash[66654]: cluster 2024-04-07T17:32:52.361308+0000 mgr.y (mgr.25175) 2325 : cluster [DBG] pgmap v3852: 137 pgs: 137 active+clean; 357 MiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 27 KiB/s rd, 3.9 MiB/s wr, 36 op/s 2024-04-07T17:32:54.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:32:53 smithi100 bash[74942]: cluster 2024-04-07T17:32:52.361308+0000 mgr.y (mgr.25175) 2325 : cluster [DBG] pgmap v3852: 137 pgs: 137 active+clean; 357 MiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 27 KiB/s rd, 3.9 MiB/s wr, 36 op/s 2024-04-07T17:32:54.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:32:53 smithi100 bash[76202]: cluster 2024-04-07T17:32:52.361308+0000 mgr.y (mgr.25175) 2325 : cluster [DBG] pgmap v3852: 137 pgs: 137 active+clean; 357 MiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 27 KiB/s rd, 3.9 MiB/s wr, 36 op/s 2024-04-07T17:32:56.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:32:55 smithi144 bash[66654]: cluster 2024-04-07T17:32:54.362001+0000 mgr.y (mgr.25175) 2326 : cluster [DBG] pgmap v3853: 137 pgs: 137 active+clean; 369 MiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 60 KiB/s rd, 3.8 MiB/s wr, 19 op/s 2024-04-07T17:32:56.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:32:55 smithi100 bash[74942]: cluster 2024-04-07T17:32:54.362001+0000 mgr.y (mgr.25175) 2326 : cluster [DBG] pgmap v3853: 137 pgs: 137 active+clean; 369 MiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 60 KiB/s rd, 3.8 MiB/s wr, 19 op/s 2024-04-07T17:32:56.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:32:55 smithi100 bash[76202]: cluster 2024-04-07T17:32:54.362001+0000 mgr.y (mgr.25175) 2326 : cluster [DBG] pgmap v3853: 137 pgs: 137 active+clean; 369 MiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 60 KiB/s rd, 3.8 MiB/s wr, 19 op/s 2024-04-07T17:32:57.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:32:56 smithi144 bash[66654]: cluster 2024-04-07T17:32:55.806278+0000 mon.a (mon.0) 3774 : cluster [DBG] osdmap e1953: 8 total, 8 up, 8 in 2024-04-07T17:32:57.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:32:56 smithi100 bash[74942]: cluster 2024-04-07T17:32:55.806278+0000 mon.a (mon.0) 3774 : cluster [DBG] osdmap e1953: 8 total, 8 up, 8 in 2024-04-07T17:32:57.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:32:56 smithi100 bash[76202]: cluster 2024-04-07T17:32:55.806278+0000 mon.a (mon.0) 3774 : cluster [DBG] osdmap e1953: 8 total, 8 up, 8 in 2024-04-07T17:32:58.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:32:57 smithi144 bash[66654]: cluster 2024-04-07T17:32:56.362526+0000 mgr.y (mgr.25175) 2327 : cluster [DBG] pgmap v3855: 137 pgs: 137 active+clean; 369 MiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 65 KiB/s rd, 1.6 MiB/s wr, 11 op/s 2024-04-07T17:32:58.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:32:57 smithi100 bash[74942]: cluster 2024-04-07T17:32:56.362526+0000 mgr.y (mgr.25175) 2327 : cluster [DBG] pgmap v3855: 137 pgs: 137 active+clean; 369 MiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 65 KiB/s rd, 1.6 MiB/s wr, 11 op/s 2024-04-07T17:32:58.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:32:57 smithi100 bash[76202]: cluster 2024-04-07T17:32:56.362526+0000 mgr.y (mgr.25175) 2327 : cluster [DBG] pgmap v3855: 137 pgs: 137 active+clean; 369 MiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 65 KiB/s rd, 1.6 MiB/s wr, 11 op/s 2024-04-07T17:32:58.244 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] EncryptedFlattenTest/6.ShrinkToOneAfterSnapshot (27531 ms) 2024-04-07T17:32:58.244 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] EncryptedFlattenTest/6.MinOverlap 2024-04-07T17:32:58.244 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:32:58.704 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:32:58 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:32:58] "GET /metrics HTTP/1.1" 200 36762 "" "Prometheus/2.43.0" 2024-04-07T17:33:00.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:32:59 smithi144 bash[66654]: cluster 2024-04-07T17:32:58.363359+0000 mgr.y (mgr.25175) 2328 : cluster [DBG] pgmap v3856: 137 pgs: 137 active+clean; 397 MiB data, 4.7 GiB used, 710 GiB / 715 GiB avail; 2.1 MiB/s rd, 4.4 MiB/s wr, 31 op/s 2024-04-07T17:33:00.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:32:59 smithi100 bash[74942]: cluster 2024-04-07T17:32:58.363359+0000 mgr.y (mgr.25175) 2328 : cluster [DBG] pgmap v3856: 137 pgs: 137 active+clean; 397 MiB data, 4.7 GiB used, 710 GiB / 715 GiB avail; 2.1 MiB/s rd, 4.4 MiB/s wr, 31 op/s 2024-04-07T17:33:00.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:32:59 smithi100 bash[76202]: cluster 2024-04-07T17:32:58.363359+0000 mgr.y (mgr.25175) 2328 : cluster [DBG] pgmap v3856: 137 pgs: 137 active+clean; 397 MiB data, 4.7 GiB used, 710 GiB / 715 GiB avail; 2.1 MiB/s rd, 4.4 MiB/s wr, 31 op/s 2024-04-07T17:33:02.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:33:01 smithi144 bash[66654]: cluster 2024-04-07T17:33:00.364120+0000 mgr.y (mgr.25175) 2329 : cluster [DBG] pgmap v3857: 137 pgs: 137 active+clean; 438 MiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 4.2 MiB/s rd, 8.5 MiB/s wr, 44 op/s 2024-04-07T17:33:02.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:33:01 smithi100 bash[74942]: cluster 2024-04-07T17:33:00.364120+0000 mgr.y (mgr.25175) 2329 : cluster [DBG] pgmap v3857: 137 pgs: 137 active+clean; 438 MiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 4.2 MiB/s rd, 8.5 MiB/s wr, 44 op/s 2024-04-07T17:33:02.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:33:01 smithi100 bash[76202]: cluster 2024-04-07T17:33:00.364120+0000 mgr.y (mgr.25175) 2329 : cluster [DBG] pgmap v3857: 137 pgs: 137 active+clean; 438 MiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 4.2 MiB/s rd, 8.5 MiB/s wr, 44 op/s 2024-04-07T17:33:02.615 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:33:02 smithi144 bash[95233]: ts=2024-04-07T17:33:02.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:33:03.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:33:02 smithi144 bash[66654]: audit 2024-04-07T17:33:01.850332+0000 mon.a (mon.0) 3775 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:33:03.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:33:02 smithi100 bash[74942]: audit 2024-04-07T17:33:01.850332+0000 mon.a (mon.0) 3775 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:33:03.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:33:02 smithi100 bash[76202]: audit 2024-04-07T17:33:01.850332+0000 mon.a (mon.0) 3775 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:33:04.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:33:03 smithi144 bash[66654]: cluster 2024-04-07T17:33:02.364882+0000 mgr.y (mgr.25175) 2330 : cluster [DBG] pgmap v3858: 137 pgs: 137 active+clean; 438 MiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 4.2 MiB/s rd, 8.1 MiB/s wr, 43 op/s 2024-04-07T17:33:04.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:33:03 smithi100 bash[74942]: cluster 2024-04-07T17:33:02.364882+0000 mgr.y (mgr.25175) 2330 : cluster [DBG] pgmap v3858: 137 pgs: 137 active+clean; 438 MiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 4.2 MiB/s rd, 8.1 MiB/s wr, 43 op/s 2024-04-07T17:33:04.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:33:03 smithi100 bash[76202]: cluster 2024-04-07T17:33:02.364882+0000 mgr.y (mgr.25175) 2330 : cluster [DBG] pgmap v3858: 137 pgs: 137 active+clean; 438 MiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 4.2 MiB/s rd, 8.1 MiB/s wr, 43 op/s 2024-04-07T17:33:06.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:33:05 smithi144 bash[66654]: cluster 2024-04-07T17:33:04.365576+0000 mgr.y (mgr.25175) 2331 : cluster [DBG] pgmap v3859: 137 pgs: 137 active+clean; 438 MiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 4.2 MiB/s rd, 6.9 MiB/s wr, 38 op/s 2024-04-07T17:33:06.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:33:05 smithi100 bash[74942]: cluster 2024-04-07T17:33:04.365576+0000 mgr.y (mgr.25175) 2331 : cluster [DBG] pgmap v3859: 137 pgs: 137 active+clean; 438 MiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 4.2 MiB/s rd, 6.9 MiB/s wr, 38 op/s 2024-04-07T17:33:06.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:33:05 smithi100 bash[76202]: cluster 2024-04-07T17:33:04.365576+0000 mgr.y (mgr.25175) 2331 : cluster [DBG] pgmap v3859: 137 pgs: 137 active+clean; 438 MiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 4.2 MiB/s rd, 6.9 MiB/s wr, 38 op/s 2024-04-07T17:33:08.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:33:07 smithi144 bash[66654]: cluster 2024-04-07T17:33:06.366168+0000 mgr.y (mgr.25175) 2332 : cluster [DBG] pgmap v3860: 137 pgs: 137 active+clean; 438 MiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 3.9 MiB/s rd, 6.5 MiB/s wr, 36 op/s 2024-04-07T17:33:08.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:33:07 smithi144 bash[66654]: audit 2024-04-07T17:33:07.143209+0000 mon.a (mon.0) 3776 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:33:08.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:33:07 smithi144 bash[66654]: audit 2024-04-07T17:33:07.834597+0000 mon.a (mon.0) 3777 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:33:08.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:33:07 smithi144 bash[66654]: audit 2024-04-07T17:33:07.836098+0000 mon.a (mon.0) 3778 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:33:08.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:33:07 smithi100 bash[74942]: cluster 2024-04-07T17:33:06.366168+0000 mgr.y (mgr.25175) 2332 : cluster [DBG] pgmap v3860: 137 pgs: 137 active+clean; 438 MiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 3.9 MiB/s rd, 6.5 MiB/s wr, 36 op/s 2024-04-07T17:33:08.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:33:07 smithi100 bash[74942]: audit 2024-04-07T17:33:07.143209+0000 mon.a (mon.0) 3776 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:33:08.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:33:07 smithi100 bash[74942]: audit 2024-04-07T17:33:07.834597+0000 mon.a (mon.0) 3777 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:33:08.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:33:07 smithi100 bash[74942]: audit 2024-04-07T17:33:07.836098+0000 mon.a (mon.0) 3778 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:33:08.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:33:07 smithi100 bash[76202]: cluster 2024-04-07T17:33:06.366168+0000 mgr.y (mgr.25175) 2332 : cluster [DBG] pgmap v3860: 137 pgs: 137 active+clean; 438 MiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 3.9 MiB/s rd, 6.5 MiB/s wr, 36 op/s 2024-04-07T17:33:08.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:33:07 smithi100 bash[76202]: audit 2024-04-07T17:33:07.143209+0000 mon.a (mon.0) 3776 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:33:08.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:33:07 smithi100 bash[76202]: audit 2024-04-07T17:33:07.834597+0000 mon.a (mon.0) 3777 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:33:08.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:33:07 smithi100 bash[76202]: audit 2024-04-07T17:33:07.836098+0000 mon.a (mon.0) 3778 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:33:08.703 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:33:08 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:33:08] "GET /metrics HTTP/1.1" 200 36764 "" "Prometheus/2.43.0" 2024-04-07T17:33:09.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:33:08 smithi100 bash[74942]: audit 2024-04-07T17:33:07.847443+0000 mon.a (mon.0) 3779 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:33:09.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:33:08 smithi100 bash[74942]: cluster 2024-04-07T17:33:07.857137+0000 mon.a (mon.0) 3780 : cluster [DBG] osdmap e1954: 8 total, 8 up, 8 in 2024-04-07T17:33:09.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:33:08 smithi100 bash[76202]: audit 2024-04-07T17:33:07.847443+0000 mon.a (mon.0) 3779 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:33:09.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:33:08 smithi100 bash[76202]: cluster 2024-04-07T17:33:07.857137+0000 mon.a (mon.0) 3780 : cluster [DBG] osdmap e1954: 8 total, 8 up, 8 in 2024-04-07T17:33:09.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:33:08 smithi144 bash[66654]: audit 2024-04-07T17:33:07.847443+0000 mon.a (mon.0) 3779 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:33:09.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:33:08 smithi144 bash[66654]: cluster 2024-04-07T17:33:07.857137+0000 mon.a (mon.0) 3780 : cluster [DBG] osdmap e1954: 8 total, 8 up, 8 in 2024-04-07T17:33:10.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:33:09 smithi100 bash[74942]: cluster 2024-04-07T17:33:08.367065+0000 mgr.y (mgr.25175) 2333 : cluster [DBG] pgmap v3862: 137 pgs: 137 active+clean; 442 MiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 2.1 MiB/s rd, 4.5 MiB/s wr, 19 op/s 2024-04-07T17:33:10.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:33:09 smithi100 bash[76202]: cluster 2024-04-07T17:33:08.367065+0000 mgr.y (mgr.25175) 2333 : cluster [DBG] pgmap v3862: 137 pgs: 137 active+clean; 442 MiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 2.1 MiB/s rd, 4.5 MiB/s wr, 19 op/s 2024-04-07T17:33:10.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:33:09 smithi144 bash[66654]: cluster 2024-04-07T17:33:08.367065+0000 mgr.y (mgr.25175) 2333 : cluster [DBG] pgmap v3862: 137 pgs: 137 active+clean; 442 MiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 2.1 MiB/s rd, 4.5 MiB/s wr, 19 op/s 2024-04-07T17:33:12.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:33:11 smithi100 bash[74942]: cluster 2024-04-07T17:33:10.367890+0000 mgr.y (mgr.25175) 2334 : cluster [DBG] pgmap v3863: 137 pgs: 137 active+clean; 458 MiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 4.3 KiB/s rd, 2.0 MiB/s wr, 7 op/s 2024-04-07T17:33:12.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:33:11 smithi100 bash[76202]: cluster 2024-04-07T17:33:10.367890+0000 mgr.y (mgr.25175) 2334 : cluster [DBG] pgmap v3863: 137 pgs: 137 active+clean; 458 MiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 4.3 KiB/s rd, 2.0 MiB/s wr, 7 op/s 2024-04-07T17:33:12.316 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:33:11 smithi144 bash[66654]: cluster 2024-04-07T17:33:10.367890+0000 mgr.y (mgr.25175) 2334 : cluster [DBG] pgmap v3863: 137 pgs: 137 active+clean; 458 MiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 4.3 KiB/s rd, 2.0 MiB/s wr, 7 op/s 2024-04-07T17:33:12.615 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:33:12 smithi144 bash[95233]: ts=2024-04-07T17:33:12.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:33:14.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:33:13 smithi100 bash[74942]: cluster 2024-04-07T17:33:12.368684+0000 mgr.y (mgr.25175) 2335 : cluster [DBG] pgmap v3864: 137 pgs: 137 active+clean; 479 MiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 26 KiB/s rd, 4.1 MiB/s wr, 35 op/s 2024-04-07T17:33:14.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:33:13 smithi100 bash[76202]: cluster 2024-04-07T17:33:12.368684+0000 mgr.y (mgr.25175) 2335 : cluster [DBG] pgmap v3864: 137 pgs: 137 active+clean; 479 MiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 26 KiB/s rd, 4.1 MiB/s wr, 35 op/s 2024-04-07T17:33:14.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:33:13 smithi144 bash[66654]: cluster 2024-04-07T17:33:12.368684+0000 mgr.y (mgr.25175) 2335 : cluster [DBG] pgmap v3864: 137 pgs: 137 active+clean; 479 MiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 26 KiB/s rd, 4.1 MiB/s wr, 35 op/s 2024-04-07T17:33:16.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:33:15 smithi100 bash[74942]: cluster 2024-04-07T17:33:14.369408+0000 mgr.y (mgr.25175) 2336 : cluster [DBG] pgmap v3865: 137 pgs: 137 active+clean; 483 MiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 55 KiB/s rd, 4.5 MiB/s wr, 35 op/s 2024-04-07T17:33:16.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:33:15 smithi100 bash[76202]: cluster 2024-04-07T17:33:14.369408+0000 mgr.y (mgr.25175) 2336 : cluster [DBG] pgmap v3865: 137 pgs: 137 active+clean; 483 MiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 55 KiB/s rd, 4.5 MiB/s wr, 35 op/s 2024-04-07T17:33:16.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:33:15 smithi144 bash[66654]: cluster 2024-04-07T17:33:14.369408+0000 mgr.y (mgr.25175) 2336 : cluster [DBG] pgmap v3865: 137 pgs: 137 active+clean; 483 MiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 55 KiB/s rd, 4.5 MiB/s wr, 35 op/s 2024-04-07T17:33:17.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:33:16 smithi100 bash[74942]: audit 2024-04-07T17:33:16.850691+0000 mon.a (mon.0) 3781 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:33:17.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:33:16 smithi100 bash[76202]: audit 2024-04-07T17:33:16.850691+0000 mon.a (mon.0) 3781 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:33:17.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:33:16 smithi144 bash[66654]: audit 2024-04-07T17:33:16.850691+0000 mon.a (mon.0) 3781 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:33:18.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:33:17 smithi100 bash[74942]: cluster 2024-04-07T17:33:16.369995+0000 mgr.y (mgr.25175) 2337 : cluster [DBG] pgmap v3866: 137 pgs: 137 active+clean; 483 MiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 55 KiB/s rd, 4.5 MiB/s wr, 35 op/s 2024-04-07T17:33:18.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:33:17 smithi100 bash[76202]: cluster 2024-04-07T17:33:16.369995+0000 mgr.y (mgr.25175) 2337 : cluster [DBG] pgmap v3866: 137 pgs: 137 active+clean; 483 MiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 55 KiB/s rd, 4.5 MiB/s wr, 35 op/s 2024-04-07T17:33:18.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:33:17 smithi144 bash[66654]: cluster 2024-04-07T17:33:16.369995+0000 mgr.y (mgr.25175) 2337 : cluster [DBG] pgmap v3866: 137 pgs: 137 active+clean; 483 MiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 55 KiB/s rd, 4.5 MiB/s wr, 35 op/s 2024-04-07T17:33:18.704 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:33:18 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:33:18] "GET /metrics HTTP/1.1" 200 36760 "" "Prometheus/2.43.0" 2024-04-07T17:33:19.214 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] EncryptedFlattenTest/6.MinOverlap (20970 ms) 2024-04-07T17:33:19.214 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] EncryptedFlattenTest/6.ZeroOverlap 2024-04-07T17:33:19.214 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:33:20.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:33:19 smithi100 bash[74942]: cluster 2024-04-07T17:33:18.370823+0000 mgr.y (mgr.25175) 2338 : cluster [DBG] pgmap v3867: 137 pgs: 137 active+clean; 495 MiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 449 KiB/s rd, 5.8 MiB/s wr, 45 op/s 2024-04-07T17:33:20.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:33:19 smithi100 bash[76202]: cluster 2024-04-07T17:33:18.370823+0000 mgr.y (mgr.25175) 2338 : cluster [DBG] pgmap v3867: 137 pgs: 137 active+clean; 495 MiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 449 KiB/s rd, 5.8 MiB/s wr, 45 op/s 2024-04-07T17:33:20.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:33:19 smithi144 bash[66654]: cluster 2024-04-07T17:33:18.370823+0000 mgr.y (mgr.25175) 2338 : cluster [DBG] pgmap v3867: 137 pgs: 137 active+clean; 495 MiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 449 KiB/s rd, 5.8 MiB/s wr, 45 op/s 2024-04-07T17:33:22.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:33:21 smithi100 bash[74942]: cluster 2024-04-07T17:33:20.371869+0000 mgr.y (mgr.25175) 2339 : cluster [DBG] pgmap v3868: 137 pgs: 137 active+clean; 495 MiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 782 KiB/s rd, 5.1 MiB/s wr, 39 op/s 2024-04-07T17:33:22.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:33:21 smithi100 bash[76202]: cluster 2024-04-07T17:33:20.371869+0000 mgr.y (mgr.25175) 2339 : cluster [DBG] pgmap v3868: 137 pgs: 137 active+clean; 495 MiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 782 KiB/s rd, 5.1 MiB/s wr, 39 op/s 2024-04-07T17:33:22.315 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:33:21 smithi144 bash[66654]: cluster 2024-04-07T17:33:20.371869+0000 mgr.y (mgr.25175) 2339 : cluster [DBG] pgmap v3868: 137 pgs: 137 active+clean; 495 MiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 782 KiB/s rd, 5.1 MiB/s wr, 39 op/s 2024-04-07T17:33:22.615 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:33:22 smithi144 bash[95233]: ts=2024-04-07T17:33:22.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:33:24.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:33:23 smithi100 bash[74942]: cluster 2024-04-07T17:33:22.372704+0000 mgr.y (mgr.25175) 2340 : cluster [DBG] pgmap v3869: 137 pgs: 137 active+clean; 499 MiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 2.8 MiB/s rd, 5.1 MiB/s wr, 49 op/s 2024-04-07T17:33:24.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:33:23 smithi100 bash[76202]: cluster 2024-04-07T17:33:22.372704+0000 mgr.y (mgr.25175) 2340 : cluster [DBG] pgmap v3869: 137 pgs: 137 active+clean; 499 MiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 2.8 MiB/s rd, 5.1 MiB/s wr, 49 op/s 2024-04-07T17:33:24.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:33:23 smithi144 bash[66654]: cluster 2024-04-07T17:33:22.372704+0000 mgr.y (mgr.25175) 2340 : cluster [DBG] pgmap v3869: 137 pgs: 137 active+clean; 499 MiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 2.8 MiB/s rd, 5.1 MiB/s wr, 49 op/s 2024-04-07T17:33:26.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:33:25 smithi100 bash[74942]: cluster 2024-04-07T17:33:24.373332+0000 mgr.y (mgr.25175) 2341 : cluster [DBG] pgmap v3870: 137 pgs: 137 active+clean; 499 MiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 2.8 MiB/s rd, 3.3 MiB/s wr, 25 op/s 2024-04-07T17:33:26.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:33:25 smithi100 bash[76202]: cluster 2024-04-07T17:33:24.373332+0000 mgr.y (mgr.25175) 2341 : cluster [DBG] pgmap v3870: 137 pgs: 137 active+clean; 499 MiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 2.8 MiB/s rd, 3.3 MiB/s wr, 25 op/s 2024-04-07T17:33:26.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:33:25 smithi144 bash[66654]: cluster 2024-04-07T17:33:24.373332+0000 mgr.y (mgr.25175) 2341 : cluster [DBG] pgmap v3870: 137 pgs: 137 active+clean; 499 MiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 2.8 MiB/s rd, 3.3 MiB/s wr, 25 op/s 2024-04-07T17:33:28.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:33:27 smithi100 bash[74942]: cluster 2024-04-07T17:33:26.373848+0000 mgr.y (mgr.25175) 2342 : cluster [DBG] pgmap v3871: 137 pgs: 137 active+clean; 499 MiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 2.7 MiB/s rd, 3.0 MiB/s wr, 25 op/s 2024-04-07T17:33:28.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:33:27 smithi100 bash[76202]: cluster 2024-04-07T17:33:26.373848+0000 mgr.y (mgr.25175) 2342 : cluster [DBG] pgmap v3871: 137 pgs: 137 active+clean; 499 MiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 2.7 MiB/s rd, 3.0 MiB/s wr, 25 op/s 2024-04-07T17:33:28.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:33:27 smithi144 bash[66654]: cluster 2024-04-07T17:33:26.373848+0000 mgr.y (mgr.25175) 2342 : cluster [DBG] pgmap v3871: 137 pgs: 137 active+clean; 499 MiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 2.7 MiB/s rd, 3.0 MiB/s wr, 25 op/s 2024-04-07T17:33:28.704 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:33:28 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:33:28] "GET /metrics HTTP/1.1" 200 36760 "" "Prometheus/2.43.0" 2024-04-07T17:33:30.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:33:29 smithi100 bash[74942]: cluster 2024-04-07T17:33:28.375095+0000 mgr.y (mgr.25175) 2343 : cluster [DBG] pgmap v3872: 137 pgs: 137 active+clean; 499 MiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 2.7 MiB/s rd, 3.0 MiB/s wr, 25 op/s 2024-04-07T17:33:30.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:33:29 smithi100 bash[74942]: cluster 2024-04-07T17:33:28.936090+0000 mon.a (mon.0) 3782 : cluster [DBG] osdmap e1955: 8 total, 8 up, 8 in 2024-04-07T17:33:30.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:33:29 smithi100 bash[76202]: cluster 2024-04-07T17:33:28.375095+0000 mgr.y (mgr.25175) 2343 : cluster [DBG] pgmap v3872: 137 pgs: 137 active+clean; 499 MiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 2.7 MiB/s rd, 3.0 MiB/s wr, 25 op/s 2024-04-07T17:33:30.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:33:29 smithi100 bash[76202]: cluster 2024-04-07T17:33:28.936090+0000 mon.a (mon.0) 3782 : cluster [DBG] osdmap e1955: 8 total, 8 up, 8 in 2024-04-07T17:33:30.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:33:29 smithi144 bash[66654]: cluster 2024-04-07T17:33:28.375095+0000 mgr.y (mgr.25175) 2343 : cluster [DBG] pgmap v3872: 137 pgs: 137 active+clean; 499 MiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 2.7 MiB/s rd, 3.0 MiB/s wr, 25 op/s 2024-04-07T17:33:30.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:33:29 smithi144 bash[66654]: cluster 2024-04-07T17:33:28.936090+0000 mon.a (mon.0) 3782 : cluster [DBG] osdmap e1955: 8 total, 8 up, 8 in 2024-04-07T17:33:32.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:33:31 smithi100 bash[74942]: cluster 2024-04-07T17:33:30.375864+0000 mgr.y (mgr.25175) 2344 : cluster [DBG] pgmap v3874: 137 pgs: 137 active+clean; 507 MiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 2.4 MiB/s rd, 2.4 MiB/s wr, 18 op/s 2024-04-07T17:33:32.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:33:31 smithi100 bash[74942]: audit 2024-04-07T17:33:31.851158+0000 mon.a (mon.0) 3783 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:33:32.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:33:31 smithi100 bash[76202]: cluster 2024-04-07T17:33:30.375864+0000 mgr.y (mgr.25175) 2344 : cluster [DBG] pgmap v3874: 137 pgs: 137 active+clean; 507 MiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 2.4 MiB/s rd, 2.4 MiB/s wr, 18 op/s 2024-04-07T17:33:32.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:33:31 smithi100 bash[76202]: audit 2024-04-07T17:33:31.851158+0000 mon.a (mon.0) 3783 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:33:32.316 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:33:31 smithi144 bash[66654]: cluster 2024-04-07T17:33:30.375864+0000 mgr.y (mgr.25175) 2344 : cluster [DBG] pgmap v3874: 137 pgs: 137 active+clean; 507 MiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 2.4 MiB/s rd, 2.4 MiB/s wr, 18 op/s 2024-04-07T17:33:32.317 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:33:31 smithi144 bash[66654]: audit 2024-04-07T17:33:31.851158+0000 mon.a (mon.0) 3783 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:33:32.615 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:33:32 smithi144 bash[95233]: ts=2024-04-07T17:33:32.317Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:33:34.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:33:33 smithi100 bash[74942]: cluster 2024-04-07T17:33:32.376626+0000 mgr.y (mgr.25175) 2345 : cluster [DBG] pgmap v3875: 137 pgs: 137 active+clean; 528 MiB data, 5.1 GiB used, 710 GiB / 715 GiB avail; 8.2 KiB/s rd, 2.9 MiB/s wr, 12 op/s 2024-04-07T17:33:34.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:33:33 smithi100 bash[76202]: cluster 2024-04-07T17:33:32.376626+0000 mgr.y (mgr.25175) 2345 : cluster [DBG] pgmap v3875: 137 pgs: 137 active+clean; 528 MiB data, 5.1 GiB used, 710 GiB / 715 GiB avail; 8.2 KiB/s rd, 2.9 MiB/s wr, 12 op/s 2024-04-07T17:33:34.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:33:33 smithi144 bash[66654]: cluster 2024-04-07T17:33:32.376626+0000 mgr.y (mgr.25175) 2345 : cluster [DBG] pgmap v3875: 137 pgs: 137 active+clean; 528 MiB data, 5.1 GiB used, 710 GiB / 715 GiB avail; 8.2 KiB/s rd, 2.9 MiB/s wr, 12 op/s 2024-04-07T17:33:36.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:33:35 smithi100 bash[74942]: cluster 2024-04-07T17:33:34.377266+0000 mgr.y (mgr.25175) 2346 : cluster [DBG] pgmap v3876: 137 pgs: 137 active+clean; 540 MiB data, 5.1 GiB used, 710 GiB / 715 GiB avail; 26 KiB/s rd, 4.1 MiB/s wr, 35 op/s 2024-04-07T17:33:36.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:33:35 smithi100 bash[76202]: cluster 2024-04-07T17:33:34.377266+0000 mgr.y (mgr.25175) 2346 : cluster [DBG] pgmap v3876: 137 pgs: 137 active+clean; 540 MiB data, 5.1 GiB used, 710 GiB / 715 GiB avail; 26 KiB/s rd, 4.1 MiB/s wr, 35 op/s 2024-04-07T17:33:36.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:33:35 smithi144 bash[66654]: cluster 2024-04-07T17:33:34.377266+0000 mgr.y (mgr.25175) 2346 : cluster [DBG] pgmap v3876: 137 pgs: 137 active+clean; 540 MiB data, 5.1 GiB used, 710 GiB / 715 GiB avail; 26 KiB/s rd, 4.1 MiB/s wr, 35 op/s 2024-04-07T17:33:38.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:33:37 smithi100 bash[74942]: cluster 2024-04-07T17:33:36.378071+0000 mgr.y (mgr.25175) 2347 : cluster [DBG] pgmap v3877: 137 pgs: 137 active+clean; 540 MiB data, 5.1 GiB used, 710 GiB / 715 GiB avail; 26 KiB/s rd, 4.1 MiB/s wr, 35 op/s 2024-04-07T17:33:38.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:33:37 smithi100 bash[76202]: cluster 2024-04-07T17:33:36.378071+0000 mgr.y (mgr.25175) 2347 : cluster [DBG] pgmap v3877: 137 pgs: 137 active+clean; 540 MiB data, 5.1 GiB used, 710 GiB / 715 GiB avail; 26 KiB/s rd, 4.1 MiB/s wr, 35 op/s 2024-04-07T17:33:38.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:33:37 smithi144 bash[66654]: cluster 2024-04-07T17:33:36.378071+0000 mgr.y (mgr.25175) 2347 : cluster [DBG] pgmap v3877: 137 pgs: 137 active+clean; 540 MiB data, 5.1 GiB used, 710 GiB / 715 GiB avail; 26 KiB/s rd, 4.1 MiB/s wr, 35 op/s 2024-04-07T17:33:38.703 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:33:38 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:33:38] "GET /metrics HTTP/1.1" 200 36759 "" "Prometheus/2.43.0" 2024-04-07T17:33:40.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:33:39 smithi100 bash[74942]: cluster 2024-04-07T17:33:38.379386+0000 mgr.y (mgr.25175) 2348 : cluster [DBG] pgmap v3878: 137 pgs: 137 active+clean; 540 MiB data, 5.1 GiB used, 710 GiB / 715 GiB avail; 26 KiB/s rd, 4.1 MiB/s wr, 35 op/s 2024-04-07T17:33:40.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:33:39 smithi100 bash[76202]: cluster 2024-04-07T17:33:38.379386+0000 mgr.y (mgr.25175) 2348 : cluster [DBG] pgmap v3878: 137 pgs: 137 active+clean; 540 MiB data, 5.1 GiB used, 710 GiB / 715 GiB avail; 26 KiB/s rd, 4.1 MiB/s wr, 35 op/s 2024-04-07T17:33:40.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:33:39 smithi144 bash[66654]: cluster 2024-04-07T17:33:38.379386+0000 mgr.y (mgr.25175) 2348 : cluster [DBG] pgmap v3878: 137 pgs: 137 active+clean; 540 MiB data, 5.1 GiB used, 710 GiB / 715 GiB avail; 26 KiB/s rd, 4.1 MiB/s wr, 35 op/s 2024-04-07T17:33:42.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:33:41 smithi100 bash[74942]: cluster 2024-04-07T17:33:40.380103+0000 mgr.y (mgr.25175) 2349 : cluster [DBG] pgmap v3879: 137 pgs: 137 active+clean; 544 MiB data, 5.2 GiB used, 710 GiB / 715 GiB avail; 50 KiB/s rd, 3.9 MiB/s wr, 34 op/s 2024-04-07T17:33:42.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:33:41 smithi100 bash[76202]: cluster 2024-04-07T17:33:40.380103+0000 mgr.y (mgr.25175) 2349 : cluster [DBG] pgmap v3879: 137 pgs: 137 active+clean; 544 MiB data, 5.2 GiB used, 710 GiB / 715 GiB avail; 50 KiB/s rd, 3.9 MiB/s wr, 34 op/s 2024-04-07T17:33:42.315 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:33:41 smithi144 bash[66654]: cluster 2024-04-07T17:33:40.380103+0000 mgr.y (mgr.25175) 2349 : cluster [DBG] pgmap v3879: 137 pgs: 137 active+clean; 544 MiB data, 5.2 GiB used, 710 GiB / 715 GiB avail; 50 KiB/s rd, 3.9 MiB/s wr, 34 op/s 2024-04-07T17:33:42.615 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:33:42 smithi144 bash[95233]: ts=2024-04-07T17:33:42.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:33:44.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:33:43 smithi144 bash[66654]: cluster 2024-04-07T17:33:42.380907+0000 mgr.y (mgr.25175) 2350 : cluster [DBG] pgmap v3880: 137 pgs: 137 active+clean; 544 MiB data, 5.2 GiB used, 710 GiB / 715 GiB avail; 73 KiB/s rd, 3.1 MiB/s wr, 34 op/s 2024-04-07T17:33:44.453 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:33:43 smithi100 bash[74942]: cluster 2024-04-07T17:33:42.380907+0000 mgr.y (mgr.25175) 2350 : cluster [DBG] pgmap v3880: 137 pgs: 137 active+clean; 544 MiB data, 5.2 GiB used, 710 GiB / 715 GiB avail; 73 KiB/s rd, 3.1 MiB/s wr, 34 op/s 2024-04-07T17:33:44.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:33:43 smithi100 bash[76202]: cluster 2024-04-07T17:33:42.380907+0000 mgr.y (mgr.25175) 2350 : cluster [DBG] pgmap v3880: 137 pgs: 137 active+clean; 544 MiB data, 5.2 GiB used, 710 GiB / 715 GiB avail; 73 KiB/s rd, 3.1 MiB/s wr, 34 op/s 2024-04-07T17:33:44.501 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] EncryptedFlattenTest/6.ZeroOverlap (25288 ms) 2024-04-07T17:33:44.976 INFO:tasks.workunit.client.0.smithi100.stdout:[----------] 7 tests from EncryptedFlattenTest/6 (179011 ms total) 2024-04-07T17:33:44.976 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:33:44.976 INFO:tasks.workunit.client.0.smithi100.stdout:[----------] 7 tests from EncryptedFlattenTest/7, where TypeParam = LUKS2UnderLUKS1 2024-04-07T17:33:45.035 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] EncryptedFlattenTest/7.Simple 2024-04-07T17:33:46.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:33:45 smithi144 bash[66654]: cluster 2024-04-07T17:33:44.382082+0000 mgr.y (mgr.25175) 2351 : cluster [DBG] pgmap v3881: 137 pgs: 137 active+clean; 556 MiB data, 5.2 GiB used, 710 GiB / 715 GiB avail; 1.1 MiB/s rd, 3.3 MiB/s wr, 34 op/s 2024-04-07T17:33:46.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:33:45 smithi144 bash[66654]: cluster 2024-04-07T17:33:44.966953+0000 mon.a (mon.0) 3784 : cluster [DBG] osdmap e1956: 8 total, 8 up, 8 in 2024-04-07T17:33:46.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:33:45 smithi100 bash[74942]: cluster 2024-04-07T17:33:44.382082+0000 mgr.y (mgr.25175) 2351 : cluster [DBG] pgmap v3881: 137 pgs: 137 active+clean; 556 MiB data, 5.2 GiB used, 710 GiB / 715 GiB avail; 1.1 MiB/s rd, 3.3 MiB/s wr, 34 op/s 2024-04-07T17:33:46.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:33:45 smithi100 bash[74942]: cluster 2024-04-07T17:33:44.966953+0000 mon.a (mon.0) 3784 : cluster [DBG] osdmap e1956: 8 total, 8 up, 8 in 2024-04-07T17:33:46.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:33:45 smithi100 bash[76202]: cluster 2024-04-07T17:33:44.382082+0000 mgr.y (mgr.25175) 2351 : cluster [DBG] pgmap v3881: 137 pgs: 137 active+clean; 556 MiB data, 5.2 GiB used, 710 GiB / 715 GiB avail; 1.1 MiB/s rd, 3.3 MiB/s wr, 34 op/s 2024-04-07T17:33:46.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:33:45 smithi100 bash[76202]: cluster 2024-04-07T17:33:44.966953+0000 mon.a (mon.0) 3784 : cluster [DBG] osdmap e1956: 8 total, 8 up, 8 in 2024-04-07T17:33:47.006 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:33:47.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:33:46 smithi144 bash[66654]: cluster 2024-04-07T17:33:45.984630+0000 mon.a (mon.0) 3785 : cluster [DBG] osdmap e1957: 8 total, 8 up, 8 in 2024-04-07T17:33:47.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:33:46 smithi144 bash[66654]: audit 2024-04-07T17:33:45.997017+0000 mon.c (mon.2) 98 : audit [INF] from='client.? 172.21.15.100:0/645265676' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:33:47.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:33:46 smithi144 bash[66654]: audit 2024-04-07T17:33:45.998555+0000 mon.a (mon.0) 3786 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:33:47.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:33:46 smithi144 bash[66654]: audit 2024-04-07T17:33:46.852214+0000 mon.a (mon.0) 3787 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:33:47.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:33:46 smithi100 bash[74942]: cluster 2024-04-07T17:33:45.984630+0000 mon.a (mon.0) 3785 : cluster [DBG] osdmap e1957: 8 total, 8 up, 8 in 2024-04-07T17:33:47.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:33:46 smithi100 bash[74942]: audit 2024-04-07T17:33:45.997017+0000 mon.c (mon.2) 98 : audit [INF] from='client.? 172.21.15.100:0/645265676' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:33:47.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:33:46 smithi100 bash[74942]: audit 2024-04-07T17:33:45.998555+0000 mon.a (mon.0) 3786 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:33:47.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:33:46 smithi100 bash[74942]: audit 2024-04-07T17:33:46.852214+0000 mon.a (mon.0) 3787 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:33:47.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:33:46 smithi100 bash[76202]: cluster 2024-04-07T17:33:45.984630+0000 mon.a (mon.0) 3785 : cluster [DBG] osdmap e1957: 8 total, 8 up, 8 in 2024-04-07T17:33:47.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:33:46 smithi100 bash[76202]: audit 2024-04-07T17:33:45.997017+0000 mon.c (mon.2) 98 : audit [INF] from='client.? 172.21.15.100:0/645265676' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:33:47.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:33:46 smithi100 bash[76202]: audit 2024-04-07T17:33:45.998555+0000 mon.a (mon.0) 3786 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:33:47.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:33:46 smithi100 bash[76202]: audit 2024-04-07T17:33:46.852214+0000 mon.a (mon.0) 3787 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:33:48.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:33:48 smithi144 bash[66654]: cluster 2024-04-07T17:33:46.382874+0000 mgr.y (mgr.25175) 2352 : cluster [DBG] pgmap v3884: 137 pgs: 32 unknown, 105 active+clean; 4.6 MiB data, 5.2 GiB used, 710 GiB / 715 GiB avail 2024-04-07T17:33:48.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:33:48 smithi144 bash[66654]: cluster 2024-04-07T17:33:46.975863+0000 mon.a (mon.0) 3788 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T17:33:48.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:33:48 smithi144 bash[66654]: audit 2024-04-07T17:33:46.985888+0000 mon.a (mon.0) 3789 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T17:33:48.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:33:48 smithi144 bash[66654]: cluster 2024-04-07T17:33:46.998016+0000 mon.a (mon.0) 3790 : cluster [DBG] osdmap e1958: 8 total, 8 up, 8 in 2024-04-07T17:33:48.426 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:33:48 smithi100 bash[74942]: cluster 2024-04-07T17:33:46.382874+0000 mgr.y (mgr.25175) 2352 : cluster [DBG] pgmap v3884: 137 pgs: 32 unknown, 105 active+clean; 4.6 MiB data, 5.2 GiB used, 710 GiB / 715 GiB avail 2024-04-07T17:33:48.427 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:33:48 smithi100 bash[74942]: cluster 2024-04-07T17:33:46.975863+0000 mon.a (mon.0) 3788 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T17:33:48.427 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:33:48 smithi100 bash[74942]: audit 2024-04-07T17:33:46.985888+0000 mon.a (mon.0) 3789 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T17:33:48.427 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:33:48 smithi100 bash[74942]: cluster 2024-04-07T17:33:46.998016+0000 mon.a (mon.0) 3790 : cluster [DBG] osdmap e1958: 8 total, 8 up, 8 in 2024-04-07T17:33:48.427 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:33:48 smithi100 bash[76202]: cluster 2024-04-07T17:33:46.382874+0000 mgr.y (mgr.25175) 2352 : cluster [DBG] pgmap v3884: 137 pgs: 32 unknown, 105 active+clean; 4.6 MiB data, 5.2 GiB used, 710 GiB / 715 GiB avail 2024-04-07T17:33:48.427 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:33:48 smithi100 bash[76202]: cluster 2024-04-07T17:33:46.975863+0000 mon.a (mon.0) 3788 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T17:33:48.427 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:33:48 smithi100 bash[76202]: audit 2024-04-07T17:33:46.985888+0000 mon.a (mon.0) 3789 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T17:33:48.427 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:33:48 smithi100 bash[76202]: cluster 2024-04-07T17:33:46.998016+0000 mon.a (mon.0) 3790 : cluster [DBG] osdmap e1958: 8 total, 8 up, 8 in 2024-04-07T17:33:48.704 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:33:48 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:33:48] "GET /metrics HTTP/1.1" 200 36695 "" "Prometheus/2.43.0" 2024-04-07T17:33:49.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:33:49 smithi144 bash[66654]: cluster 2024-04-07T17:33:47.997292+0000 mon.a (mon.0) 3791 : cluster [DBG] osdmap e1959: 8 total, 8 up, 8 in 2024-04-07T17:33:49.453 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:33:49 smithi100 bash[76202]: cluster 2024-04-07T17:33:47.997292+0000 mon.a (mon.0) 3791 : cluster [DBG] osdmap e1959: 8 total, 8 up, 8 in 2024-04-07T17:33:49.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:33:49 smithi100 bash[74942]: cluster 2024-04-07T17:33:47.997292+0000 mon.a (mon.0) 3791 : cluster [DBG] osdmap e1959: 8 total, 8 up, 8 in 2024-04-07T17:33:50.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:33:50 smithi144 bash[66654]: cluster 2024-04-07T17:33:48.383623+0000 mgr.y (mgr.25175) 2353 : cluster [DBG] pgmap v3887: 137 pgs: 12 creating+activating, 4 creating+peering, 12 unknown, 109 active+clean; 4.6 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail 2024-04-07T17:33:50.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:33:50 smithi144 bash[66654]: cluster 2024-04-07T17:33:49.001220+0000 mon.a (mon.0) 3792 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T17:33:50.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:33:50 smithi144 bash[66654]: cluster 2024-04-07T17:33:49.024055+0000 mon.a (mon.0) 3793 : cluster [DBG] osdmap e1960: 8 total, 8 up, 8 in 2024-04-07T17:33:50.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:33:50 smithi100 bash[74942]: cluster 2024-04-07T17:33:48.383623+0000 mgr.y (mgr.25175) 2353 : cluster [DBG] pgmap v3887: 137 pgs: 12 creating+activating, 4 creating+peering, 12 unknown, 109 active+clean; 4.6 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail 2024-04-07T17:33:50.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:33:50 smithi100 bash[74942]: cluster 2024-04-07T17:33:49.001220+0000 mon.a (mon.0) 3792 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T17:33:50.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:33:50 smithi100 bash[74942]: cluster 2024-04-07T17:33:49.024055+0000 mon.a (mon.0) 3793 : cluster [DBG] osdmap e1960: 8 total, 8 up, 8 in 2024-04-07T17:33:50.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:33:50 smithi100 bash[76202]: cluster 2024-04-07T17:33:48.383623+0000 mgr.y (mgr.25175) 2353 : cluster [DBG] pgmap v3887: 137 pgs: 12 creating+activating, 4 creating+peering, 12 unknown, 109 active+clean; 4.6 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail 2024-04-07T17:33:50.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:33:50 smithi100 bash[76202]: cluster 2024-04-07T17:33:49.001220+0000 mon.a (mon.0) 3792 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T17:33:50.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:33:50 smithi100 bash[76202]: cluster 2024-04-07T17:33:49.024055+0000 mon.a (mon.0) 3793 : cluster [DBG] osdmap e1960: 8 total, 8 up, 8 in 2024-04-07T17:33:52.315 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:33:52 smithi144 bash[66654]: cluster 2024-04-07T17:33:50.384369+0000 mgr.y (mgr.25175) 2354 : cluster [DBG] pgmap v3889: 137 pgs: 12 creating+activating, 4 creating+peering, 121 active+clean; 4.6 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 232 B/s rd, 232 B/s wr, 1 op/s 2024-04-07T17:33:52.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:33:52 smithi100 bash[74942]: cluster 2024-04-07T17:33:50.384369+0000 mgr.y (mgr.25175) 2354 : cluster [DBG] pgmap v3889: 137 pgs: 12 creating+activating, 4 creating+peering, 121 active+clean; 4.6 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 232 B/s rd, 232 B/s wr, 1 op/s 2024-04-07T17:33:52.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:33:52 smithi100 bash[76202]: cluster 2024-04-07T17:33:50.384369+0000 mgr.y (mgr.25175) 2354 : cluster [DBG] pgmap v3889: 137 pgs: 12 creating+activating, 4 creating+peering, 121 active+clean; 4.6 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 232 B/s rd, 232 B/s wr, 1 op/s 2024-04-07T17:33:52.615 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:33:52 smithi144 bash[95233]: ts=2024-04-07T17:33:52.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:33:54.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:33:54 smithi100 bash[74942]: cluster 2024-04-07T17:33:52.385185+0000 mgr.y (mgr.25175) 2355 : cluster [DBG] pgmap v3890: 137 pgs: 137 active+clean; 4.6 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 5.0 KiB/s rd, 1.3 KiB/s wr, 8 op/s 2024-04-07T17:33:54.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:33:54 smithi100 bash[76202]: cluster 2024-04-07T17:33:52.385185+0000 mgr.y (mgr.25175) 2355 : cluster [DBG] pgmap v3890: 137 pgs: 137 active+clean; 4.6 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 5.0 KiB/s rd, 1.3 KiB/s wr, 8 op/s 2024-04-07T17:33:54.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:33:54 smithi144 bash[66654]: cluster 2024-04-07T17:33:52.385185+0000 mgr.y (mgr.25175) 2355 : cluster [DBG] pgmap v3890: 137 pgs: 137 active+clean; 4.6 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 5.0 KiB/s rd, 1.3 KiB/s wr, 8 op/s 2024-04-07T17:33:56.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:33:56 smithi100 bash[74942]: cluster 2024-04-07T17:33:54.386130+0000 mgr.y (mgr.25175) 2356 : cluster [DBG] pgmap v3891: 137 pgs: 137 active+clean; 4.6 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 4.1 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-04-07T17:33:56.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:33:56 smithi100 bash[76202]: cluster 2024-04-07T17:33:54.386130+0000 mgr.y (mgr.25175) 2356 : cluster [DBG] pgmap v3891: 137 pgs: 137 active+clean; 4.6 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 4.1 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-04-07T17:33:56.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:33:56 smithi144 bash[66654]: cluster 2024-04-07T17:33:54.386130+0000 mgr.y (mgr.25175) 2356 : cluster [DBG] pgmap v3891: 137 pgs: 137 active+clean; 4.6 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 4.1 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-04-07T17:33:58.427 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:33:58 smithi100 bash[74942]: cluster 2024-04-07T17:33:56.386696+0000 mgr.y (mgr.25175) 2357 : cluster [DBG] pgmap v3892: 137 pgs: 137 active+clean; 4.6 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 3.6 KiB/s rd, 975 B/s wr, 6 op/s 2024-04-07T17:33:58.428 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:33:58 smithi100 bash[74942]: cluster 2024-04-07T17:33:58.145063+0000 mon.a (mon.0) 3794 : cluster [DBG] osdmap e1961: 8 total, 8 up, 8 in 2024-04-07T17:33:58.428 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:33:58 smithi100 bash[76202]: cluster 2024-04-07T17:33:56.386696+0000 mgr.y (mgr.25175) 2357 : cluster [DBG] pgmap v3892: 137 pgs: 137 active+clean; 4.6 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 3.6 KiB/s rd, 975 B/s wr, 6 op/s 2024-04-07T17:33:58.428 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:33:58 smithi100 bash[76202]: cluster 2024-04-07T17:33:58.145063+0000 mon.a (mon.0) 3794 : cluster [DBG] osdmap e1961: 8 total, 8 up, 8 in 2024-04-07T17:33:58.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:33:58 smithi144 bash[66654]: cluster 2024-04-07T17:33:56.386696+0000 mgr.y (mgr.25175) 2357 : cluster [DBG] pgmap v3892: 137 pgs: 137 active+clean; 4.6 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 3.6 KiB/s rd, 975 B/s wr, 6 op/s 2024-04-07T17:33:58.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:33:58 smithi144 bash[66654]: cluster 2024-04-07T17:33:58.145063+0000 mon.a (mon.0) 3794 : cluster [DBG] osdmap e1961: 8 total, 8 up, 8 in 2024-04-07T17:33:58.704 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:33:58 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:33:58] "GET /metrics HTTP/1.1" 200 36695 "" "Prometheus/2.43.0" 2024-04-07T17:33:59.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:33:59 smithi100 bash[74942]: cluster 2024-04-07T17:33:58.387149+0000 mgr.y (mgr.25175) 2358 : cluster [DBG] pgmap v3894: 137 pgs: 137 active+clean; 4.6 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 3.2 KiB/s rd, 874 B/s wr, 5 op/s 2024-04-07T17:33:59.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:33:59 smithi100 bash[76202]: cluster 2024-04-07T17:33:58.387149+0000 mgr.y (mgr.25175) 2358 : cluster [DBG] pgmap v3894: 137 pgs: 137 active+clean; 4.6 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 3.2 KiB/s rd, 874 B/s wr, 5 op/s 2024-04-07T17:33:59.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:33:59 smithi144 bash[66654]: cluster 2024-04-07T17:33:58.387149+0000 mgr.y (mgr.25175) 2358 : cluster [DBG] pgmap v3894: 137 pgs: 137 active+clean; 4.6 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 3.2 KiB/s rd, 874 B/s wr, 5 op/s 2024-04-07T17:34:00.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:34:00 smithi100 bash[74942]: cluster 2024-04-07T17:33:59.161967+0000 mon.a (mon.0) 3795 : cluster [DBG] osdmap e1962: 8 total, 8 up, 8 in 2024-04-07T17:34:00.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:34:00 smithi100 bash[76202]: cluster 2024-04-07T17:33:59.161967+0000 mon.a (mon.0) 3795 : cluster [DBG] osdmap e1962: 8 total, 8 up, 8 in 2024-04-07T17:34:00.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:34:00 smithi144 bash[66654]: cluster 2024-04-07T17:33:59.161967+0000 mon.a (mon.0) 3795 : cluster [DBG] osdmap e1962: 8 total, 8 up, 8 in 2024-04-07T17:34:01.453 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:34:01 smithi100 bash[76202]: cluster 2024-04-07T17:34:00.387990+0000 mgr.y (mgr.25175) 2359 : cluster [DBG] pgmap v3896: 137 pgs: 137 active+clean; 26 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 2.6 MiB/s wr, 0 op/s 2024-04-07T17:34:01.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:34:01 smithi100 bash[74942]: cluster 2024-04-07T17:34:00.387990+0000 mgr.y (mgr.25175) 2359 : cluster [DBG] pgmap v3896: 137 pgs: 137 active+clean; 26 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 2.6 MiB/s wr, 0 op/s 2024-04-07T17:34:01.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:34:01 smithi144 bash[66654]: cluster 2024-04-07T17:34:00.387990+0000 mgr.y (mgr.25175) 2359 : cluster [DBG] pgmap v3896: 137 pgs: 137 active+clean; 26 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 2.6 MiB/s wr, 0 op/s 2024-04-07T17:34:02.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:34:02 smithi100 bash[74942]: audit 2024-04-07T17:34:01.852428+0000 mon.a (mon.0) 3796 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:34:02.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:34:02 smithi100 bash[76202]: audit 2024-04-07T17:34:01.852428+0000 mon.a (mon.0) 3796 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:34:02.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:34:02 smithi144 bash[66654]: audit 2024-04-07T17:34:01.852428+0000 mon.a (mon.0) 3796 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:34:02.616 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:34:02 smithi144 bash[95233]: ts=2024-04-07T17:34:02.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:34:03.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:34:03 smithi100 bash[74942]: cluster 2024-04-07T17:34:02.389102+0000 mgr.y (mgr.25175) 2360 : cluster [DBG] pgmap v3897: 137 pgs: 137 active+clean; 38 MiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 26 KiB/s rd, 4.1 MiB/s wr, 36 op/s 2024-04-07T17:34:03.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:34:03 smithi100 bash[76202]: cluster 2024-04-07T17:34:02.389102+0000 mgr.y (mgr.25175) 2360 : cluster [DBG] pgmap v3897: 137 pgs: 137 active+clean; 38 MiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 26 KiB/s rd, 4.1 MiB/s wr, 36 op/s 2024-04-07T17:34:03.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:34:03 smithi144 bash[66654]: cluster 2024-04-07T17:34:02.389102+0000 mgr.y (mgr.25175) 2360 : cluster [DBG] pgmap v3897: 137 pgs: 137 active+clean; 38 MiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 26 KiB/s rd, 4.1 MiB/s wr, 36 op/s 2024-04-07T17:34:05.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:34:05 smithi100 bash[74942]: cluster 2024-04-07T17:34:04.390108+0000 mgr.y (mgr.25175) 2361 : cluster [DBG] pgmap v3898: 137 pgs: 137 active+clean; 50 MiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 69 KiB/s rd, 5.6 MiB/s wr, 46 op/s 2024-04-07T17:34:05.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:34:05 smithi100 bash[76202]: cluster 2024-04-07T17:34:04.390108+0000 mgr.y (mgr.25175) 2361 : cluster [DBG] pgmap v3898: 137 pgs: 137 active+clean; 50 MiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 69 KiB/s rd, 5.6 MiB/s wr, 46 op/s 2024-04-07T17:34:05.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:34:05 smithi144 bash[66654]: cluster 2024-04-07T17:34:04.390108+0000 mgr.y (mgr.25175) 2361 : cluster [DBG] pgmap v3898: 137 pgs: 137 active+clean; 50 MiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 69 KiB/s rd, 5.6 MiB/s wr, 46 op/s 2024-04-07T17:34:07.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:34:07 smithi100 bash[74942]: cluster 2024-04-07T17:34:06.390710+0000 mgr.y (mgr.25175) 2362 : cluster [DBG] pgmap v3899: 137 pgs: 137 active+clean; 50 MiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 67 KiB/s rd, 5.5 MiB/s wr, 44 op/s 2024-04-07T17:34:07.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:34:07 smithi100 bash[76202]: cluster 2024-04-07T17:34:06.390710+0000 mgr.y (mgr.25175) 2362 : cluster [DBG] pgmap v3899: 137 pgs: 137 active+clean; 50 MiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 67 KiB/s rd, 5.5 MiB/s wr, 44 op/s 2024-04-07T17:34:07.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:34:07 smithi144 bash[66654]: cluster 2024-04-07T17:34:06.390710+0000 mgr.y (mgr.25175) 2362 : cluster [DBG] pgmap v3899: 137 pgs: 137 active+clean; 50 MiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 67 KiB/s rd, 5.5 MiB/s wr, 44 op/s 2024-04-07T17:34:08.703 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:34:08 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:34:08] "GET /metrics HTTP/1.1" 200 36742 "" "Prometheus/2.43.0" 2024-04-07T17:34:08.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:34:08 smithi100 bash[76202]: audit 2024-04-07T17:34:07.933921+0000 mon.a (mon.0) 3797 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:34:08.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:34:08 smithi100 bash[74942]: audit 2024-04-07T17:34:07.933921+0000 mon.a (mon.0) 3797 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:34:08.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:34:08 smithi144 bash[66654]: audit 2024-04-07T17:34:07.933921+0000 mon.a (mon.0) 3797 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:34:09.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:34:09 smithi144 bash[66654]: cluster 2024-04-07T17:34:08.391502+0000 mgr.y (mgr.25175) 2363 : cluster [DBG] pgmap v3900: 137 pgs: 137 active+clean; 54 MiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 55 KiB/s rd, 4.9 MiB/s wr, 37 op/s 2024-04-07T17:34:09.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:34:09 smithi144 bash[66654]: audit 2024-04-07T17:34:08.664673+0000 mon.a (mon.0) 3798 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:34:09.866 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:34:09 smithi144 bash[66654]: audit 2024-04-07T17:34:08.666208+0000 mon.a (mon.0) 3799 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:34:09.866 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:34:09 smithi144 bash[66654]: audit 2024-04-07T17:34:08.675766+0000 mon.a (mon.0) 3800 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:34:09.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:34:09 smithi100 bash[74942]: cluster 2024-04-07T17:34:08.391502+0000 mgr.y (mgr.25175) 2363 : cluster [DBG] pgmap v3900: 137 pgs: 137 active+clean; 54 MiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 55 KiB/s rd, 4.9 MiB/s wr, 37 op/s 2024-04-07T17:34:09.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:34:09 smithi100 bash[74942]: audit 2024-04-07T17:34:08.664673+0000 mon.a (mon.0) 3798 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:34:09.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:34:09 smithi100 bash[74942]: audit 2024-04-07T17:34:08.666208+0000 mon.a (mon.0) 3799 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:34:09.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:34:09 smithi100 bash[74942]: audit 2024-04-07T17:34:08.675766+0000 mon.a (mon.0) 3800 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:34:09.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:34:09 smithi100 bash[76202]: cluster 2024-04-07T17:34:08.391502+0000 mgr.y (mgr.25175) 2363 : cluster [DBG] pgmap v3900: 137 pgs: 137 active+clean; 54 MiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 55 KiB/s rd, 4.9 MiB/s wr, 37 op/s 2024-04-07T17:34:09.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:34:09 smithi100 bash[76202]: audit 2024-04-07T17:34:08.664673+0000 mon.a (mon.0) 3798 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:34:09.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:34:09 smithi100 bash[76202]: audit 2024-04-07T17:34:08.666208+0000 mon.a (mon.0) 3799 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:34:09.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:34:09 smithi100 bash[76202]: audit 2024-04-07T17:34:08.675766+0000 mon.a (mon.0) 3800 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:34:11.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:34:11 smithi144 bash[66654]: cluster 2024-04-07T17:34:10.392217+0000 mgr.y (mgr.25175) 2364 : cluster [DBG] pgmap v3901: 137 pgs: 137 active+clean; 54 MiB data, 3.7 GiB used, 711 GiB / 715 GiB avail; 194 KiB/s rd, 2.9 MiB/s wr, 37 op/s 2024-04-07T17:34:11.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:34:11 smithi100 bash[74942]: cluster 2024-04-07T17:34:10.392217+0000 mgr.y (mgr.25175) 2364 : cluster [DBG] pgmap v3901: 137 pgs: 137 active+clean; 54 MiB data, 3.7 GiB used, 711 GiB / 715 GiB avail; 194 KiB/s rd, 2.9 MiB/s wr, 37 op/s 2024-04-07T17:34:11.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:34:11 smithi100 bash[76202]: cluster 2024-04-07T17:34:10.392217+0000 mgr.y (mgr.25175) 2364 : cluster [DBG] pgmap v3901: 137 pgs: 137 active+clean; 54 MiB data, 3.7 GiB used, 711 GiB / 715 GiB avail; 194 KiB/s rd, 2.9 MiB/s wr, 37 op/s 2024-04-07T17:34:12.615 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:34:12 smithi144 bash[95233]: ts=2024-04-07T17:34:12.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:34:13.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:34:13 smithi144 bash[66654]: cluster 2024-04-07T17:34:12.392912+0000 mgr.y (mgr.25175) 2365 : cluster [DBG] pgmap v3902: 137 pgs: 137 active+clean; 58 MiB data, 3.7 GiB used, 711 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.7 MiB/s wr, 39 op/s 2024-04-07T17:34:13.940 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] EncryptedFlattenTest/7.Simple (28905 ms) 2024-04-07T17:34:13.940 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] EncryptedFlattenTest/7.Grow 2024-04-07T17:34:13.941 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:34:13.953 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:34:13 smithi100 bash[76202]: cluster 2024-04-07T17:34:12.392912+0000 mgr.y (mgr.25175) 2365 : cluster [DBG] pgmap v3902: 137 pgs: 137 active+clean; 58 MiB data, 3.7 GiB used, 711 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.7 MiB/s wr, 39 op/s 2024-04-07T17:34:13.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:34:13 smithi100 bash[74942]: cluster 2024-04-07T17:34:12.392912+0000 mgr.y (mgr.25175) 2365 : cluster [DBG] pgmap v3902: 137 pgs: 137 active+clean; 58 MiB data, 3.7 GiB used, 711 GiB / 715 GiB avail; 2.5 MiB/s rd, 2.7 MiB/s wr, 39 op/s 2024-04-07T17:34:15.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:34:15 smithi144 bash[66654]: cluster 2024-04-07T17:34:14.393858+0000 mgr.y (mgr.25175) 2366 : cluster [DBG] pgmap v3903: 137 pgs: 137 active+clean; 78 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 7.4 MiB/s rd, 3.7 MiB/s wr, 30 op/s 2024-04-07T17:34:15.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:34:15 smithi100 bash[74942]: cluster 2024-04-07T17:34:14.393858+0000 mgr.y (mgr.25175) 2366 : cluster [DBG] pgmap v3903: 137 pgs: 137 active+clean; 78 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 7.4 MiB/s rd, 3.7 MiB/s wr, 30 op/s 2024-04-07T17:34:15.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:34:15 smithi100 bash[76202]: cluster 2024-04-07T17:34:14.393858+0000 mgr.y (mgr.25175) 2366 : cluster [DBG] pgmap v3903: 137 pgs: 137 active+clean; 78 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 7.4 MiB/s rd, 3.7 MiB/s wr, 30 op/s 2024-04-07T17:34:17.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:34:17 smithi144 bash[66654]: cluster 2024-04-07T17:34:16.394484+0000 mgr.y (mgr.25175) 2367 : cluster [DBG] pgmap v3904: 137 pgs: 137 active+clean; 78 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 8.0 MiB/s rd, 2.7 MiB/s wr, 23 op/s 2024-04-07T17:34:17.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:34:17 smithi144 bash[66654]: audit 2024-04-07T17:34:16.852248+0000 mon.a (mon.0) 3801 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:34:17.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:34:17 smithi100 bash[74942]: cluster 2024-04-07T17:34:16.394484+0000 mgr.y (mgr.25175) 2367 : cluster [DBG] pgmap v3904: 137 pgs: 137 active+clean; 78 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 8.0 MiB/s rd, 2.7 MiB/s wr, 23 op/s 2024-04-07T17:34:17.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:34:17 smithi100 bash[74942]: audit 2024-04-07T17:34:16.852248+0000 mon.a (mon.0) 3801 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:34:17.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:34:17 smithi100 bash[76202]: cluster 2024-04-07T17:34:16.394484+0000 mgr.y (mgr.25175) 2367 : cluster [DBG] pgmap v3904: 137 pgs: 137 active+clean; 78 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 8.0 MiB/s rd, 2.7 MiB/s wr, 23 op/s 2024-04-07T17:34:17.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:34:17 smithi100 bash[76202]: audit 2024-04-07T17:34:16.852248+0000 mon.a (mon.0) 3801 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:34:18.703 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:34:18 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:34:18] "GET /metrics HTTP/1.1" 200 36756 "" "Prometheus/2.43.0" 2024-04-07T17:34:19.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:34:19 smithi144 bash[66654]: cluster 2024-04-07T17:34:18.395306+0000 mgr.y (mgr.25175) 2368 : cluster [DBG] pgmap v3905: 137 pgs: 137 active+clean; 78 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 8.8 MiB/s rd, 2.7 MiB/s wr, 25 op/s 2024-04-07T17:34:19.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:34:19 smithi100 bash[74942]: cluster 2024-04-07T17:34:18.395306+0000 mgr.y (mgr.25175) 2368 : cluster [DBG] pgmap v3905: 137 pgs: 137 active+clean; 78 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 8.8 MiB/s rd, 2.7 MiB/s wr, 25 op/s 2024-04-07T17:34:19.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:34:19 smithi100 bash[76202]: cluster 2024-04-07T17:34:18.395306+0000 mgr.y (mgr.25175) 2368 : cluster [DBG] pgmap v3905: 137 pgs: 137 active+clean; 78 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 8.8 MiB/s rd, 2.7 MiB/s wr, 25 op/s 2024-04-07T17:34:21.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:34:21 smithi144 bash[66654]: cluster 2024-04-07T17:34:20.396013+0000 mgr.y (mgr.25175) 2369 : cluster [DBG] pgmap v3906: 137 pgs: 137 active+clean; 78 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 8.8 MiB/s rd, 2.3 MiB/s wr, 25 op/s 2024-04-07T17:34:21.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:34:21 smithi100 bash[74942]: cluster 2024-04-07T17:34:20.396013+0000 mgr.y (mgr.25175) 2369 : cluster [DBG] pgmap v3906: 137 pgs: 137 active+clean; 78 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 8.8 MiB/s rd, 2.3 MiB/s wr, 25 op/s 2024-04-07T17:34:21.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:34:21 smithi100 bash[76202]: cluster 2024-04-07T17:34:20.396013+0000 mgr.y (mgr.25175) 2369 : cluster [DBG] pgmap v3906: 137 pgs: 137 active+clean; 78 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 8.8 MiB/s rd, 2.3 MiB/s wr, 25 op/s 2024-04-07T17:34:22.616 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:34:22 smithi144 bash[95233]: ts=2024-04-07T17:34:22.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:34:23.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:34:23 smithi144 bash[66654]: cluster 2024-04-07T17:34:22.396779+0000 mgr.y (mgr.25175) 2370 : cluster [DBG] pgmap v3907: 137 pgs: 137 active+clean; 78 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 8.6 MiB/s rd, 2.3 MiB/s wr, 21 op/s 2024-04-07T17:34:23.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:34:23 smithi100 bash[76202]: cluster 2024-04-07T17:34:22.396779+0000 mgr.y (mgr.25175) 2370 : cluster [DBG] pgmap v3907: 137 pgs: 137 active+clean; 78 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 8.6 MiB/s rd, 2.3 MiB/s wr, 21 op/s 2024-04-07T17:34:23.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:34:23 smithi100 bash[74942]: cluster 2024-04-07T17:34:22.396779+0000 mgr.y (mgr.25175) 2370 : cluster [DBG] pgmap v3907: 137 pgs: 137 active+clean; 78 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 8.6 MiB/s rd, 2.3 MiB/s wr, 21 op/s 2024-04-07T17:34:24.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:34:24 smithi144 bash[66654]: cluster 2024-04-07T17:34:23.498457+0000 mon.a (mon.0) 3802 : cluster [DBG] osdmap e1963: 8 total, 8 up, 8 in 2024-04-07T17:34:24.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:34:24 smithi100 bash[74942]: cluster 2024-04-07T17:34:23.498457+0000 mon.a (mon.0) 3802 : cluster [DBG] osdmap e1963: 8 total, 8 up, 8 in 2024-04-07T17:34:24.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:34:24 smithi100 bash[76202]: cluster 2024-04-07T17:34:23.498457+0000 mon.a (mon.0) 3802 : cluster [DBG] osdmap e1963: 8 total, 8 up, 8 in 2024-04-07T17:34:25.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:34:25 smithi144 bash[66654]: cluster 2024-04-07T17:34:24.397664+0000 mgr.y (mgr.25175) 2371 : cluster [DBG] pgmap v3909: 137 pgs: 137 active+clean; 90 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 1.7 MiB/s rd, 1.2 MiB/s wr, 18 op/s 2024-04-07T17:34:25.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:34:25 smithi144 bash[66654]: audit 2024-04-07T17:34:24.516966+0000 mon.a (mon.0) 3803 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "119.5", "id": [3, 0]}]: dispatch 2024-04-07T17:34:25.866 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:34:25 smithi144 bash[66654]: audit 2024-04-07T17:34:24.517505+0000 mon.a (mon.0) 3804 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "119.18", "id": [3, 0]}]: dispatch 2024-04-07T17:34:25.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:34:25 smithi100 bash[76202]: cluster 2024-04-07T17:34:24.397664+0000 mgr.y (mgr.25175) 2371 : cluster [DBG] pgmap v3909: 137 pgs: 137 active+clean; 90 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 1.7 MiB/s rd, 1.2 MiB/s wr, 18 op/s 2024-04-07T17:34:25.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:34:25 smithi100 bash[76202]: audit 2024-04-07T17:34:24.516966+0000 mon.a (mon.0) 3803 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "119.5", "id": [3, 0]}]: dispatch 2024-04-07T17:34:25.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:34:25 smithi100 bash[76202]: audit 2024-04-07T17:34:24.517505+0000 mon.a (mon.0) 3804 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "119.18", "id": [3, 0]}]: dispatch 2024-04-07T17:34:25.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:34:25 smithi100 bash[74942]: cluster 2024-04-07T17:34:24.397664+0000 mgr.y (mgr.25175) 2371 : cluster [DBG] pgmap v3909: 137 pgs: 137 active+clean; 90 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 1.7 MiB/s rd, 1.2 MiB/s wr, 18 op/s 2024-04-07T17:34:25.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:34:25 smithi100 bash[74942]: audit 2024-04-07T17:34:24.516966+0000 mon.a (mon.0) 3803 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "119.5", "id": [3, 0]}]: dispatch 2024-04-07T17:34:25.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:34:25 smithi100 bash[74942]: audit 2024-04-07T17:34:24.517505+0000 mon.a (mon.0) 3804 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "119.18", "id": [3, 0]}]: dispatch 2024-04-07T17:34:26.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:34:26 smithi144 bash[66654]: audit 2024-04-07T17:34:25.510939+0000 mon.a (mon.0) 3805 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "119.5", "id": [3, 0]}]': finished 2024-04-07T17:34:26.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:34:26 smithi144 bash[66654]: audit 2024-04-07T17:34:25.511176+0000 mon.a (mon.0) 3806 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "119.18", "id": [3, 0]}]': finished 2024-04-07T17:34:26.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:34:26 smithi144 bash[66654]: cluster 2024-04-07T17:34:25.518324+0000 mon.a (mon.0) 3807 : cluster [DBG] osdmap e1964: 8 total, 8 up, 8 in 2024-04-07T17:34:26.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:34:26 smithi100 bash[76202]: audit 2024-04-07T17:34:25.510939+0000 mon.a (mon.0) 3805 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "119.5", "id": [3, 0]}]': finished 2024-04-07T17:34:26.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:34:26 smithi100 bash[76202]: audit 2024-04-07T17:34:25.511176+0000 mon.a (mon.0) 3806 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "119.18", "id": [3, 0]}]': finished 2024-04-07T17:34:26.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:34:26 smithi100 bash[76202]: cluster 2024-04-07T17:34:25.518324+0000 mon.a (mon.0) 3807 : cluster [DBG] osdmap e1964: 8 total, 8 up, 8 in 2024-04-07T17:34:26.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:34:26 smithi100 bash[74942]: audit 2024-04-07T17:34:25.510939+0000 mon.a (mon.0) 3805 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "119.5", "id": [3, 0]}]': finished 2024-04-07T17:34:26.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:34:26 smithi100 bash[74942]: audit 2024-04-07T17:34:25.511176+0000 mon.a (mon.0) 3806 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "119.18", "id": [3, 0]}]': finished 2024-04-07T17:34:26.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:34:26 smithi100 bash[74942]: cluster 2024-04-07T17:34:25.518324+0000 mon.a (mon.0) 3807 : cluster [DBG] osdmap e1964: 8 total, 8 up, 8 in 2024-04-07T17:34:27.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:34:27 smithi144 bash[66654]: cluster 2024-04-07T17:34:26.398319+0000 mgr.y (mgr.25175) 2372 : cluster [DBG] pgmap v3911: 137 pgs: 137 active+clean; 90 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 17 KiB/s rd, 1.5 MiB/s wr, 20 op/s 2024-04-07T17:34:27.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:34:27 smithi144 bash[66654]: cluster 2024-04-07T17:34:26.520679+0000 mon.a (mon.0) 3808 : cluster [DBG] osdmap e1965: 8 total, 8 up, 8 in 2024-04-07T17:34:27.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:34:27 smithi100 bash[76202]: cluster 2024-04-07T17:34:26.398319+0000 mgr.y (mgr.25175) 2372 : cluster [DBG] pgmap v3911: 137 pgs: 137 active+clean; 90 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 17 KiB/s rd, 1.5 MiB/s wr, 20 op/s 2024-04-07T17:34:27.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:34:27 smithi100 bash[76202]: cluster 2024-04-07T17:34:26.520679+0000 mon.a (mon.0) 3808 : cluster [DBG] osdmap e1965: 8 total, 8 up, 8 in 2024-04-07T17:34:27.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:34:27 smithi100 bash[74942]: cluster 2024-04-07T17:34:26.398319+0000 mgr.y (mgr.25175) 2372 : cluster [DBG] pgmap v3911: 137 pgs: 137 active+clean; 90 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 17 KiB/s rd, 1.5 MiB/s wr, 20 op/s 2024-04-07T17:34:27.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:34:27 smithi100 bash[74942]: cluster 2024-04-07T17:34:26.520679+0000 mon.a (mon.0) 3808 : cluster [DBG] osdmap e1965: 8 total, 8 up, 8 in 2024-04-07T17:34:28.703 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:34:28 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:34:28] "GET /metrics HTTP/1.1" 200 36756 "" "Prometheus/2.43.0" 2024-04-07T17:34:30.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:34:30 smithi100 bash[74942]: cluster 2024-04-07T17:34:28.399012+0000 mgr.y (mgr.25175) 2373 : cluster [DBG] pgmap v3913: 137 pgs: 137 active+clean; 119 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 43 KiB/s rd, 6.8 MiB/s wr, 58 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2024-04-07T17:34:30.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:34:30 smithi100 bash[76202]: cluster 2024-04-07T17:34:28.399012+0000 mgr.y (mgr.25175) 2373 : cluster [DBG] pgmap v3913: 137 pgs: 137 active+clean; 119 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 43 KiB/s rd, 6.8 MiB/s wr, 58 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2024-04-07T17:34:30.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:34:30 smithi144 bash[66654]: cluster 2024-04-07T17:34:28.399012+0000 mgr.y (mgr.25175) 2373 : cluster [DBG] pgmap v3913: 137 pgs: 137 active+clean; 119 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 43 KiB/s rd, 6.8 MiB/s wr, 58 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2024-04-07T17:34:32.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:34:32 smithi100 bash[74942]: cluster 2024-04-07T17:34:30.399718+0000 mgr.y (mgr.25175) 2374 : cluster [DBG] pgmap v3914: 137 pgs: 137 active+clean; 123 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 79 KiB/s rd, 6.5 MiB/s wr, 51 op/s; 593 KiB/s, 1 keys/s, 0 objects/s recovering 2024-04-07T17:34:32.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:34:32 smithi100 bash[74942]: audit 2024-04-07T17:34:31.853265+0000 mon.a (mon.0) 3809 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:34:32.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:34:32 smithi100 bash[76202]: cluster 2024-04-07T17:34:30.399718+0000 mgr.y (mgr.25175) 2374 : cluster [DBG] pgmap v3914: 137 pgs: 137 active+clean; 123 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 79 KiB/s rd, 6.5 MiB/s wr, 51 op/s; 593 KiB/s, 1 keys/s, 0 objects/s recovering 2024-04-07T17:34:32.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:34:32 smithi100 bash[76202]: audit 2024-04-07T17:34:31.853265+0000 mon.a (mon.0) 3809 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:34:32.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:34:32 smithi144 bash[66654]: cluster 2024-04-07T17:34:30.399718+0000 mgr.y (mgr.25175) 2374 : cluster [DBG] pgmap v3914: 137 pgs: 137 active+clean; 123 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 79 KiB/s rd, 6.5 MiB/s wr, 51 op/s; 593 KiB/s, 1 keys/s, 0 objects/s recovering 2024-04-07T17:34:32.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:34:32 smithi144 bash[66654]: audit 2024-04-07T17:34:31.853265+0000 mon.a (mon.0) 3809 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:34:32.616 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:34:32 smithi144 bash[95233]: ts=2024-04-07T17:34:32.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:34:33.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:34:33 smithi144 bash[66654]: cluster 2024-04-07T17:34:32.400964+0000 mgr.y (mgr.25175) 2375 : cluster [DBG] pgmap v3915: 137 pgs: 137 active+clean; 123 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 185 KiB/s rd, 4.1 MiB/s wr, 32 op/s; 512 KiB/s, 1 keys/s, 0 objects/s recovering 2024-04-07T17:34:33.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:34:33 smithi100 bash[74942]: cluster 2024-04-07T17:34:32.400964+0000 mgr.y (mgr.25175) 2375 : cluster [DBG] pgmap v3915: 137 pgs: 137 active+clean; 123 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 185 KiB/s rd, 4.1 MiB/s wr, 32 op/s; 512 KiB/s, 1 keys/s, 0 objects/s recovering 2024-04-07T17:34:33.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:34:33 smithi100 bash[76202]: cluster 2024-04-07T17:34:32.400964+0000 mgr.y (mgr.25175) 2375 : cluster [DBG] pgmap v3915: 137 pgs: 137 active+clean; 123 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 185 KiB/s rd, 4.1 MiB/s wr, 32 op/s; 512 KiB/s, 1 keys/s, 0 objects/s recovering 2024-04-07T17:34:35.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:34:35 smithi144 bash[66654]: cluster 2024-04-07T17:34:34.401726+0000 mgr.y (mgr.25175) 2376 : cluster [DBG] pgmap v3916: 137 pgs: 137 active+clean; 127 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 202 KiB/s rd, 4.2 MiB/s wr, 33 op/s; 461 KiB/s, 1 keys/s, 0 objects/s recovering 2024-04-07T17:34:35.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:34:35 smithi100 bash[74942]: cluster 2024-04-07T17:34:34.401726+0000 mgr.y (mgr.25175) 2376 : cluster [DBG] pgmap v3916: 137 pgs: 137 active+clean; 127 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 202 KiB/s rd, 4.2 MiB/s wr, 33 op/s; 461 KiB/s, 1 keys/s, 0 objects/s recovering 2024-04-07T17:34:35.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:34:35 smithi100 bash[76202]: cluster 2024-04-07T17:34:34.401726+0000 mgr.y (mgr.25175) 2376 : cluster [DBG] pgmap v3916: 137 pgs: 137 active+clean; 127 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 202 KiB/s rd, 4.2 MiB/s wr, 33 op/s; 461 KiB/s, 1 keys/s, 0 objects/s recovering 2024-04-07T17:34:37.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:34:37 smithi144 bash[66654]: cluster 2024-04-07T17:34:36.402534+0000 mgr.y (mgr.25175) 2377 : cluster [DBG] pgmap v3917: 137 pgs: 137 active+clean; 127 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 208 KiB/s rd, 3.7 MiB/s wr, 29 op/s; 409 KiB/s, 0 keys/s, 0 objects/s recovering 2024-04-07T17:34:37.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:34:37 smithi100 bash[74942]: cluster 2024-04-07T17:34:36.402534+0000 mgr.y (mgr.25175) 2377 : cluster [DBG] pgmap v3917: 137 pgs: 137 active+clean; 127 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 208 KiB/s rd, 3.7 MiB/s wr, 29 op/s; 409 KiB/s, 0 keys/s, 0 objects/s recovering 2024-04-07T17:34:37.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:34:37 smithi100 bash[76202]: cluster 2024-04-07T17:34:36.402534+0000 mgr.y (mgr.25175) 2377 : cluster [DBG] pgmap v3917: 137 pgs: 137 active+clean; 127 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 208 KiB/s rd, 3.7 MiB/s wr, 29 op/s; 409 KiB/s, 0 keys/s, 0 objects/s recovering 2024-04-07T17:34:38.330 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] EncryptedFlattenTest/7.Grow (24390 ms) 2024-04-07T17:34:38.330 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] EncryptedFlattenTest/7.Shrink 2024-04-07T17:34:38.330 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:34:38.703 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:34:38 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:34:38] "GET /metrics HTTP/1.1" 200 36766 "" "Prometheus/2.43.0" 2024-04-07T17:34:39.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:34:39 smithi144 bash[66654]: cluster 2024-04-07T17:34:38.403810+0000 mgr.y (mgr.25175) 2378 : cluster [DBG] pgmap v3918: 137 pgs: 137 active+clean; 143 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 3.6 MiB/s rd, 4.5 MiB/s wr, 38 op/s; 345 KiB/s, 0 keys/s, 0 objects/s recovering 2024-04-07T17:34:39.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:34:39 smithi100 bash[74942]: cluster 2024-04-07T17:34:38.403810+0000 mgr.y (mgr.25175) 2378 : cluster [DBG] pgmap v3918: 137 pgs: 137 active+clean; 143 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 3.6 MiB/s rd, 4.5 MiB/s wr, 38 op/s; 345 KiB/s, 0 keys/s, 0 objects/s recovering 2024-04-07T17:34:39.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:34:39 smithi100 bash[76202]: cluster 2024-04-07T17:34:38.403810+0000 mgr.y (mgr.25175) 2378 : cluster [DBG] pgmap v3918: 137 pgs: 137 active+clean; 143 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 3.6 MiB/s rd, 4.5 MiB/s wr, 38 op/s; 345 KiB/s, 0 keys/s, 0 objects/s recovering 2024-04-07T17:34:41.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:34:41 smithi144 bash[66654]: cluster 2024-04-07T17:34:40.404598+0000 mgr.y (mgr.25175) 2379 : cluster [DBG] pgmap v3919: 137 pgs: 137 active+clean; 151 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 7.5 MiB/s rd, 3.0 MiB/s wr, 27 op/s; 341 KiB/s, 0 objects/s recovering 2024-04-07T17:34:41.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:34:41 smithi100 bash[74942]: cluster 2024-04-07T17:34:40.404598+0000 mgr.y (mgr.25175) 2379 : cluster [DBG] pgmap v3919: 137 pgs: 137 active+clean; 151 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 7.5 MiB/s rd, 3.0 MiB/s wr, 27 op/s; 341 KiB/s, 0 objects/s recovering 2024-04-07T17:34:41.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:34:41 smithi100 bash[76202]: cluster 2024-04-07T17:34:40.404598+0000 mgr.y (mgr.25175) 2379 : cluster [DBG] pgmap v3919: 137 pgs: 137 active+clean; 151 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 7.5 MiB/s rd, 3.0 MiB/s wr, 27 op/s; 341 KiB/s, 0 objects/s recovering 2024-04-07T17:34:42.615 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:34:42 smithi144 bash[95233]: ts=2024-04-07T17:34:42.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:34:43.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:34:43 smithi144 bash[66654]: cluster 2024-04-07T17:34:42.405385+0000 mgr.y (mgr.25175) 2380 : cluster [DBG] pgmap v3920: 137 pgs: 137 active+clean; 151 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 8.8 MiB/s rd, 2.7 MiB/s wr, 28 op/s 2024-04-07T17:34:43.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:34:43 smithi100 bash[74942]: cluster 2024-04-07T17:34:42.405385+0000 mgr.y (mgr.25175) 2380 : cluster [DBG] pgmap v3920: 137 pgs: 137 active+clean; 151 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 8.8 MiB/s rd, 2.7 MiB/s wr, 28 op/s 2024-04-07T17:34:43.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:34:43 smithi100 bash[76202]: cluster 2024-04-07T17:34:42.405385+0000 mgr.y (mgr.25175) 2380 : cluster [DBG] pgmap v3920: 137 pgs: 137 active+clean; 151 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 8.8 MiB/s rd, 2.7 MiB/s wr, 28 op/s 2024-04-07T17:34:45.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:34:45 smithi144 bash[66654]: cluster 2024-04-07T17:34:44.406081+0000 mgr.y (mgr.25175) 2381 : cluster [DBG] pgmap v3921: 137 pgs: 137 active+clean; 151 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 8.7 MiB/s rd, 2.7 MiB/s wr, 22 op/s 2024-04-07T17:34:45.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:34:45 smithi100 bash[74942]: cluster 2024-04-07T17:34:44.406081+0000 mgr.y (mgr.25175) 2381 : cluster [DBG] pgmap v3921: 137 pgs: 137 active+clean; 151 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 8.7 MiB/s rd, 2.7 MiB/s wr, 22 op/s 2024-04-07T17:34:45.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:34:45 smithi100 bash[76202]: cluster 2024-04-07T17:34:44.406081+0000 mgr.y (mgr.25175) 2381 : cluster [DBG] pgmap v3921: 137 pgs: 137 active+clean; 151 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 8.7 MiB/s rd, 2.7 MiB/s wr, 22 op/s 2024-04-07T17:34:47.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:34:47 smithi144 bash[66654]: cluster 2024-04-07T17:34:46.406651+0000 mgr.y (mgr.25175) 2382 : cluster [DBG] pgmap v3922: 137 pgs: 137 active+clean; 151 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 8.7 MiB/s rd, 2.3 MiB/s wr, 19 op/s 2024-04-07T17:34:47.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:34:47 smithi144 bash[66654]: audit 2024-04-07T17:34:46.852953+0000 mon.a (mon.0) 3810 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:34:47.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:34:47 smithi100 bash[74942]: cluster 2024-04-07T17:34:46.406651+0000 mgr.y (mgr.25175) 2382 : cluster [DBG] pgmap v3922: 137 pgs: 137 active+clean; 151 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 8.7 MiB/s rd, 2.3 MiB/s wr, 19 op/s 2024-04-07T17:34:47.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:34:47 smithi100 bash[74942]: audit 2024-04-07T17:34:46.852953+0000 mon.a (mon.0) 3810 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:34:47.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:34:47 smithi100 bash[76202]: cluster 2024-04-07T17:34:46.406651+0000 mgr.y (mgr.25175) 2382 : cluster [DBG] pgmap v3922: 137 pgs: 137 active+clean; 151 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 8.7 MiB/s rd, 2.3 MiB/s wr, 19 op/s 2024-04-07T17:34:47.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:34:47 smithi100 bash[76202]: audit 2024-04-07T17:34:46.852953+0000 mon.a (mon.0) 3810 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:34:48.704 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:34:48 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:34:48] "GET /metrics HTTP/1.1" 200 36751 "" "Prometheus/2.43.0" 2024-04-07T17:34:48.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:34:48 smithi100 bash[74942]: cluster 2024-04-07T17:34:47.495024+0000 mon.a (mon.0) 3811 : cluster [DBG] osdmap e1966: 8 total, 8 up, 8 in 2024-04-07T17:34:48.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:34:48 smithi100 bash[76202]: cluster 2024-04-07T17:34:47.495024+0000 mon.a (mon.0) 3811 : cluster [DBG] osdmap e1966: 8 total, 8 up, 8 in 2024-04-07T17:34:48.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:34:48 smithi144 bash[66654]: cluster 2024-04-07T17:34:47.495024+0000 mon.a (mon.0) 3811 : cluster [DBG] osdmap e1966: 8 total, 8 up, 8 in 2024-04-07T17:34:49.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:34:49 smithi144 bash[66654]: cluster 2024-04-07T17:34:48.407411+0000 mgr.y (mgr.25175) 2383 : cluster [DBG] pgmap v3924: 137 pgs: 137 active+clean; 152 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 6.3 MiB/s rd, 1.3 MiB/s wr, 7 op/s 2024-04-07T17:34:49.953 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:34:49 smithi100 bash[76202]: cluster 2024-04-07T17:34:48.407411+0000 mgr.y (mgr.25175) 2383 : cluster [DBG] pgmap v3924: 137 pgs: 137 active+clean; 152 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 6.3 MiB/s rd, 1.3 MiB/s wr, 7 op/s 2024-04-07T17:34:49.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:34:49 smithi100 bash[74942]: cluster 2024-04-07T17:34:48.407411+0000 mgr.y (mgr.25175) 2383 : cluster [DBG] pgmap v3924: 137 pgs: 137 active+clean; 152 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 6.3 MiB/s rd, 1.3 MiB/s wr, 7 op/s 2024-04-07T17:34:51.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:34:51 smithi144 bash[66654]: cluster 2024-04-07T17:34:50.408462+0000 mgr.y (mgr.25175) 2384 : cluster [DBG] pgmap v3925: 137 pgs: 137 active+clean; 168 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 27 op/s 2024-04-07T17:34:51.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:34:51 smithi100 bash[74942]: cluster 2024-04-07T17:34:50.408462+0000 mgr.y (mgr.25175) 2384 : cluster [DBG] pgmap v3925: 137 pgs: 137 active+clean; 168 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 27 op/s 2024-04-07T17:34:51.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:34:51 smithi100 bash[76202]: cluster 2024-04-07T17:34:50.408462+0000 mgr.y (mgr.25175) 2384 : cluster [DBG] pgmap v3925: 137 pgs: 137 active+clean; 168 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 27 op/s 2024-04-07T17:34:52.615 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:34:52 smithi144 bash[95233]: ts=2024-04-07T17:34:52.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:34:53.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:34:53 smithi144 bash[66654]: cluster 2024-04-07T17:34:52.409148+0000 mgr.y (mgr.25175) 2385 : cluster [DBG] pgmap v3926: 137 pgs: 137 active+clean; 192 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 26 KiB/s rd, 4.1 MiB/s wr, 35 op/s 2024-04-07T17:34:53.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:34:53 smithi100 bash[74942]: cluster 2024-04-07T17:34:52.409148+0000 mgr.y (mgr.25175) 2385 : cluster [DBG] pgmap v3926: 137 pgs: 137 active+clean; 192 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 26 KiB/s rd, 4.1 MiB/s wr, 35 op/s 2024-04-07T17:34:53.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:34:53 smithi100 bash[76202]: cluster 2024-04-07T17:34:52.409148+0000 mgr.y (mgr.25175) 2385 : cluster [DBG] pgmap v3926: 137 pgs: 137 active+clean; 192 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 26 KiB/s rd, 4.1 MiB/s wr, 35 op/s 2024-04-07T17:34:55.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:34:55 smithi144 bash[66654]: cluster 2024-04-07T17:34:54.409844+0000 mgr.y (mgr.25175) 2386 : cluster [DBG] pgmap v3927: 137 pgs: 137 active+clean; 196 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 55 KiB/s rd, 4.5 MiB/s wr, 37 op/s 2024-04-07T17:34:55.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:34:55 smithi100 bash[76202]: cluster 2024-04-07T17:34:54.409844+0000 mgr.y (mgr.25175) 2386 : cluster [DBG] pgmap v3927: 137 pgs: 137 active+clean; 196 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 55 KiB/s rd, 4.5 MiB/s wr, 37 op/s 2024-04-07T17:34:55.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:34:55 smithi100 bash[74942]: cluster 2024-04-07T17:34:54.409844+0000 mgr.y (mgr.25175) 2386 : cluster [DBG] pgmap v3927: 137 pgs: 137 active+clean; 196 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 55 KiB/s rd, 4.5 MiB/s wr, 37 op/s 2024-04-07T17:34:57.866 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:34:57 smithi144 bash[66654]: cluster 2024-04-07T17:34:56.410683+0000 mgr.y (mgr.25175) 2387 : cluster [DBG] pgmap v3928: 137 pgs: 137 active+clean; 196 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 55 KiB/s rd, 4.5 MiB/s wr, 37 op/s 2024-04-07T17:34:57.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:34:57 smithi100 bash[74942]: cluster 2024-04-07T17:34:56.410683+0000 mgr.y (mgr.25175) 2387 : cluster [DBG] pgmap v3928: 137 pgs: 137 active+clean; 196 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 55 KiB/s rd, 4.5 MiB/s wr, 37 op/s 2024-04-07T17:34:57.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:34:57 smithi100 bash[76202]: cluster 2024-04-07T17:34:56.410683+0000 mgr.y (mgr.25175) 2387 : cluster [DBG] pgmap v3928: 137 pgs: 137 active+clean; 196 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 55 KiB/s rd, 4.5 MiB/s wr, 37 op/s 2024-04-07T17:34:58.704 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:34:58 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:34:58] "GET /metrics HTTP/1.1" 200 36751 "" "Prometheus/2.43.0" 2024-04-07T17:34:59.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:34:59 smithi144 bash[66654]: cluster 2024-04-07T17:34:58.411571+0000 mgr.y (mgr.25175) 2388 : cluster [DBG] pgmap v3929: 137 pgs: 137 active+clean; 200 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 200 KiB/s rd, 4.5 MiB/s wr, 37 op/s 2024-04-07T17:34:59.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:34:59 smithi100 bash[74942]: cluster 2024-04-07T17:34:58.411571+0000 mgr.y (mgr.25175) 2388 : cluster [DBG] pgmap v3929: 137 pgs: 137 active+clean; 200 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 200 KiB/s rd, 4.5 MiB/s wr, 37 op/s 2024-04-07T17:34:59.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:34:59 smithi100 bash[76202]: cluster 2024-04-07T17:34:58.411571+0000 mgr.y (mgr.25175) 2388 : cluster [DBG] pgmap v3929: 137 pgs: 137 active+clean; 200 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 200 KiB/s rd, 4.5 MiB/s wr, 37 op/s 2024-04-07T17:35:01.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:35:01 smithi144 bash[66654]: cluster 2024-04-07T17:35:00.412534+0000 mgr.y (mgr.25175) 2389 : cluster [DBG] pgmap v3930: 137 pgs: 137 active+clean; 200 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 185 KiB/s rd, 4.0 MiB/s wr, 38 op/s 2024-04-07T17:35:01.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:35:01 smithi100 bash[74942]: cluster 2024-04-07T17:35:00.412534+0000 mgr.y (mgr.25175) 2389 : cluster [DBG] pgmap v3930: 137 pgs: 137 active+clean; 200 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 185 KiB/s rd, 4.0 MiB/s wr, 38 op/s 2024-04-07T17:35:01.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:35:01 smithi100 bash[76202]: cluster 2024-04-07T17:35:00.412534+0000 mgr.y (mgr.25175) 2389 : cluster [DBG] pgmap v3930: 137 pgs: 137 active+clean; 200 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 185 KiB/s rd, 4.0 MiB/s wr, 38 op/s 2024-04-07T17:35:02.256 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] EncryptedFlattenTest/7.Shrink (23927 ms) 2024-04-07T17:35:02.257 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] EncryptedFlattenTest/7.ShrinkToOne 2024-04-07T17:35:02.257 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:35:02.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:35:02 smithi144 bash[66654]: audit 2024-04-07T17:35:01.853794+0000 mon.a (mon.0) 3812 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:35:02.615 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:35:02 smithi144 bash[95233]: ts=2024-04-07T17:35:02.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:35:02.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:35:02 smithi100 bash[74942]: audit 2024-04-07T17:35:01.853794+0000 mon.a (mon.0) 3812 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:35:02.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:35:02 smithi100 bash[76202]: audit 2024-04-07T17:35:01.853794+0000 mon.a (mon.0) 3812 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:35:03.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:35:03 smithi144 bash[66654]: cluster 2024-04-07T17:35:02.413696+0000 mgr.y (mgr.25175) 2390 : cluster [DBG] pgmap v3931: 137 pgs: 137 active+clean; 224 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 4.2 MiB/s rd, 4.7 MiB/s wr, 25 op/s 2024-04-07T17:35:03.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:35:03 smithi100 bash[74942]: cluster 2024-04-07T17:35:02.413696+0000 mgr.y (mgr.25175) 2390 : cluster [DBG] pgmap v3931: 137 pgs: 137 active+clean; 224 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 4.2 MiB/s rd, 4.7 MiB/s wr, 25 op/s 2024-04-07T17:35:03.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:35:03 smithi100 bash[76202]: cluster 2024-04-07T17:35:02.413696+0000 mgr.y (mgr.25175) 2390 : cluster [DBG] pgmap v3931: 137 pgs: 137 active+clean; 224 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 4.2 MiB/s rd, 4.7 MiB/s wr, 25 op/s 2024-04-07T17:35:05.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:35:05 smithi144 bash[66654]: cluster 2024-04-07T17:35:04.414525+0000 mgr.y (mgr.25175) 2391 : cluster [DBG] pgmap v3932: 137 pgs: 137 active+clean; 224 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 6.8 MiB/s rd, 3.0 MiB/s wr, 24 op/s 2024-04-07T17:35:05.953 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:35:05 smithi100 bash[76202]: cluster 2024-04-07T17:35:04.414525+0000 mgr.y (mgr.25175) 2391 : cluster [DBG] pgmap v3932: 137 pgs: 137 active+clean; 224 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 6.8 MiB/s rd, 3.0 MiB/s wr, 24 op/s 2024-04-07T17:35:05.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:35:05 smithi100 bash[74942]: cluster 2024-04-07T17:35:04.414525+0000 mgr.y (mgr.25175) 2391 : cluster [DBG] pgmap v3932: 137 pgs: 137 active+clean; 224 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 6.8 MiB/s rd, 3.0 MiB/s wr, 24 op/s 2024-04-07T17:35:07.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:35:07 smithi144 bash[66654]: cluster 2024-04-07T17:35:06.415206+0000 mgr.y (mgr.25175) 2392 : cluster [DBG] pgmap v3933: 137 pgs: 137 active+clean; 224 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 6.8 MiB/s rd, 2.7 MiB/s wr, 23 op/s 2024-04-07T17:35:07.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:35:07 smithi100 bash[74942]: cluster 2024-04-07T17:35:06.415206+0000 mgr.y (mgr.25175) 2392 : cluster [DBG] pgmap v3933: 137 pgs: 137 active+clean; 224 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 6.8 MiB/s rd, 2.7 MiB/s wr, 23 op/s 2024-04-07T17:35:07.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:35:07 smithi100 bash[76202]: cluster 2024-04-07T17:35:06.415206+0000 mgr.y (mgr.25175) 2392 : cluster [DBG] pgmap v3933: 137 pgs: 137 active+clean; 224 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 6.8 MiB/s rd, 2.7 MiB/s wr, 23 op/s 2024-04-07T17:35:08.704 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:35:08 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:35:08] "GET /metrics HTTP/1.1" 200 36758 "" "Prometheus/2.43.0" 2024-04-07T17:35:09.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:35:09 smithi144 bash[66654]: cluster 2024-04-07T17:35:08.416132+0000 mgr.y (mgr.25175) 2393 : cluster [DBG] pgmap v3934: 137 pgs: 137 active+clean; 224 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 8.8 MiB/s rd, 2.7 MiB/s wr, 27 op/s 2024-04-07T17:35:09.866 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:35:09 smithi144 bash[66654]: audit 2024-04-07T17:35:08.721993+0000 mon.a (mon.0) 3813 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:35:09.866 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:35:09 smithi144 bash[66654]: audit 2024-04-07T17:35:09.405008+0000 mon.a (mon.0) 3814 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:35:09.866 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:35:09 smithi144 bash[66654]: audit 2024-04-07T17:35:09.405953+0000 mon.a (mon.0) 3815 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:35:09.866 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:35:09 smithi144 bash[66654]: audit 2024-04-07T17:35:09.412440+0000 mon.a (mon.0) 3816 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:35:09.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:35:09 smithi100 bash[74942]: cluster 2024-04-07T17:35:08.416132+0000 mgr.y (mgr.25175) 2393 : cluster [DBG] pgmap v3934: 137 pgs: 137 active+clean; 224 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 8.8 MiB/s rd, 2.7 MiB/s wr, 27 op/s 2024-04-07T17:35:09.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:35:09 smithi100 bash[74942]: audit 2024-04-07T17:35:08.721993+0000 mon.a (mon.0) 3813 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:35:09.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:35:09 smithi100 bash[74942]: audit 2024-04-07T17:35:09.405008+0000 mon.a (mon.0) 3814 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:35:09.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:35:09 smithi100 bash[74942]: audit 2024-04-07T17:35:09.405953+0000 mon.a (mon.0) 3815 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:35:09.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:35:09 smithi100 bash[74942]: audit 2024-04-07T17:35:09.412440+0000 mon.a (mon.0) 3816 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:35:09.955 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:35:09 smithi100 bash[76202]: cluster 2024-04-07T17:35:08.416132+0000 mgr.y (mgr.25175) 2393 : cluster [DBG] pgmap v3934: 137 pgs: 137 active+clean; 224 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 8.8 MiB/s rd, 2.7 MiB/s wr, 27 op/s 2024-04-07T17:35:09.955 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:35:09 smithi100 bash[76202]: audit 2024-04-07T17:35:08.721993+0000 mon.a (mon.0) 3813 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:35:09.955 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:35:09 smithi100 bash[76202]: audit 2024-04-07T17:35:09.405008+0000 mon.a (mon.0) 3814 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:35:09.955 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:35:09 smithi100 bash[76202]: audit 2024-04-07T17:35:09.405953+0000 mon.a (mon.0) 3815 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:35:09.955 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:35:09 smithi100 bash[76202]: audit 2024-04-07T17:35:09.412440+0000 mon.a (mon.0) 3816 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:35:11.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:35:11 smithi144 bash[66654]: cluster 2024-04-07T17:35:10.416858+0000 mgr.y (mgr.25175) 2394 : cluster [DBG] pgmap v3935: 137 pgs: 137 active+clean; 224 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 8.6 MiB/s rd, 2.3 MiB/s wr, 24 op/s 2024-04-07T17:35:11.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:35:11 smithi100 bash[74942]: cluster 2024-04-07T17:35:10.416858+0000 mgr.y (mgr.25175) 2394 : cluster [DBG] pgmap v3935: 137 pgs: 137 active+clean; 224 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 8.6 MiB/s rd, 2.3 MiB/s wr, 24 op/s 2024-04-07T17:35:11.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:35:11 smithi100 bash[76202]: cluster 2024-04-07T17:35:10.416858+0000 mgr.y (mgr.25175) 2394 : cluster [DBG] pgmap v3935: 137 pgs: 137 active+clean; 224 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 8.6 MiB/s rd, 2.3 MiB/s wr, 24 op/s 2024-04-07T17:35:12.615 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:35:12 smithi144 bash[95233]: ts=2024-04-07T17:35:12.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:35:13.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:35:13 smithi144 bash[66654]: cluster 2024-04-07T17:35:12.418080+0000 mgr.y (mgr.25175) 2395 : cluster [DBG] pgmap v3936: 137 pgs: 137 active+clean; 244 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 8.6 MiB/s rd, 4.0 MiB/s wr, 21 op/s 2024-04-07T17:35:13.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:35:13 smithi144 bash[66654]: cluster 2024-04-07T17:35:12.563256+0000 mon.a (mon.0) 3817 : cluster [DBG] osdmap e1967: 8 total, 8 up, 8 in 2024-04-07T17:35:13.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:35:13 smithi100 bash[74942]: cluster 2024-04-07T17:35:12.418080+0000 mgr.y (mgr.25175) 2395 : cluster [DBG] pgmap v3936: 137 pgs: 137 active+clean; 244 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 8.6 MiB/s rd, 4.0 MiB/s wr, 21 op/s 2024-04-07T17:35:13.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:35:13 smithi100 bash[74942]: cluster 2024-04-07T17:35:12.563256+0000 mon.a (mon.0) 3817 : cluster [DBG] osdmap e1967: 8 total, 8 up, 8 in 2024-04-07T17:35:13.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:35:13 smithi100 bash[76202]: cluster 2024-04-07T17:35:12.418080+0000 mgr.y (mgr.25175) 2395 : cluster [DBG] pgmap v3936: 137 pgs: 137 active+clean; 244 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 8.6 MiB/s rd, 4.0 MiB/s wr, 21 op/s 2024-04-07T17:35:13.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:35:13 smithi100 bash[76202]: cluster 2024-04-07T17:35:12.563256+0000 mon.a (mon.0) 3817 : cluster [DBG] osdmap e1967: 8 total, 8 up, 8 in 2024-04-07T17:35:15.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:35:15 smithi144 bash[66654]: cluster 2024-04-07T17:35:14.419141+0000 mgr.y (mgr.25175) 2396 : cluster [DBG] pgmap v3938: 137 pgs: 137 active+clean; 265 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 2.4 MiB/s rd, 4.1 MiB/s wr, 17 op/s 2024-04-07T17:35:15.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:35:15 smithi100 bash[74942]: cluster 2024-04-07T17:35:14.419141+0000 mgr.y (mgr.25175) 2396 : cluster [DBG] pgmap v3938: 137 pgs: 137 active+clean; 265 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 2.4 MiB/s rd, 4.1 MiB/s wr, 17 op/s 2024-04-07T17:35:15.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:35:15 smithi100 bash[76202]: cluster 2024-04-07T17:35:14.419141+0000 mgr.y (mgr.25175) 2396 : cluster [DBG] pgmap v3938: 137 pgs: 137 active+clean; 265 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 2.4 MiB/s rd, 4.1 MiB/s wr, 17 op/s 2024-04-07T17:35:17.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:35:17 smithi144 bash[66654]: cluster 2024-04-07T17:35:16.420145+0000 mgr.y (mgr.25175) 2397 : cluster [DBG] pgmap v3939: 137 pgs: 137 active+clean; 265 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 2.4 MiB/s rd, 4.1 MiB/s wr, 17 op/s 2024-04-07T17:35:17.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:35:17 smithi144 bash[66654]: audit 2024-04-07T17:35:16.853579+0000 mon.a (mon.0) 3818 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:35:17.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:35:17 smithi100 bash[76202]: cluster 2024-04-07T17:35:16.420145+0000 mgr.y (mgr.25175) 2397 : cluster [DBG] pgmap v3939: 137 pgs: 137 active+clean; 265 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 2.4 MiB/s rd, 4.1 MiB/s wr, 17 op/s 2024-04-07T17:35:17.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:35:17 smithi100 bash[76202]: audit 2024-04-07T17:35:16.853579+0000 mon.a (mon.0) 3818 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:35:17.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:35:17 smithi100 bash[74942]: cluster 2024-04-07T17:35:16.420145+0000 mgr.y (mgr.25175) 2397 : cluster [DBG] pgmap v3939: 137 pgs: 137 active+clean; 265 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 2.4 MiB/s rd, 4.1 MiB/s wr, 17 op/s 2024-04-07T17:35:17.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:35:17 smithi100 bash[74942]: audit 2024-04-07T17:35:16.853579+0000 mon.a (mon.0) 3818 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:35:18.703 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:35:18 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:35:18] "GET /metrics HTTP/1.1" 200 36766 "" "Prometheus/2.43.0" 2024-04-07T17:35:19.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:35:19 smithi144 bash[66654]: cluster 2024-04-07T17:35:18.421504+0000 mgr.y (mgr.25175) 2398 : cluster [DBG] pgmap v3940: 137 pgs: 137 active+clean; 269 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 55 KiB/s rd, 4.5 MiB/s wr, 35 op/s 2024-04-07T17:35:19.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:35:19 smithi100 bash[74942]: cluster 2024-04-07T17:35:18.421504+0000 mgr.y (mgr.25175) 2398 : cluster [DBG] pgmap v3940: 137 pgs: 137 active+clean; 269 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 55 KiB/s rd, 4.5 MiB/s wr, 35 op/s 2024-04-07T17:35:19.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:35:19 smithi100 bash[76202]: cluster 2024-04-07T17:35:18.421504+0000 mgr.y (mgr.25175) 2398 : cluster [DBG] pgmap v3940: 137 pgs: 137 active+clean; 269 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 55 KiB/s rd, 4.5 MiB/s wr, 35 op/s 2024-04-07T17:35:21.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:35:21 smithi144 bash[66654]: cluster 2024-04-07T17:35:20.422290+0000 mgr.y (mgr.25175) 2399 : cluster [DBG] pgmap v3941: 137 pgs: 137 active+clean; 269 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 55 KiB/s rd, 4.5 MiB/s wr, 37 op/s 2024-04-07T17:35:21.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:35:21 smithi100 bash[74942]: cluster 2024-04-07T17:35:20.422290+0000 mgr.y (mgr.25175) 2399 : cluster [DBG] pgmap v3941: 137 pgs: 137 active+clean; 269 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 55 KiB/s rd, 4.5 MiB/s wr, 37 op/s 2024-04-07T17:35:21.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:35:21 smithi100 bash[76202]: cluster 2024-04-07T17:35:20.422290+0000 mgr.y (mgr.25175) 2399 : cluster [DBG] pgmap v3941: 137 pgs: 137 active+clean; 269 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 55 KiB/s rd, 4.5 MiB/s wr, 37 op/s 2024-04-07T17:35:22.615 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:35:22 smithi144 bash[95233]: ts=2024-04-07T17:35:22.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:35:23.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:35:23 smithi144 bash[66654]: cluster 2024-04-07T17:35:22.423458+0000 mgr.y (mgr.25175) 2400 : cluster [DBG] pgmap v3942: 137 pgs: 137 active+clean; 273 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 117 KiB/s rd, 2.9 MiB/s wr, 42 op/s 2024-04-07T17:35:23.953 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:35:23 smithi100 bash[76202]: cluster 2024-04-07T17:35:22.423458+0000 mgr.y (mgr.25175) 2400 : cluster [DBG] pgmap v3942: 137 pgs: 137 active+clean; 273 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 117 KiB/s rd, 2.9 MiB/s wr, 42 op/s 2024-04-07T17:35:23.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:35:23 smithi100 bash[74942]: cluster 2024-04-07T17:35:22.423458+0000 mgr.y (mgr.25175) 2400 : cluster [DBG] pgmap v3942: 137 pgs: 137 active+clean; 273 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 117 KiB/s rd, 2.9 MiB/s wr, 42 op/s 2024-04-07T17:35:25.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:35:25 smithi144 bash[66654]: cluster 2024-04-07T17:35:24.424183+0000 mgr.y (mgr.25175) 2401 : cluster [DBG] pgmap v3943: 137 pgs: 137 active+clean; 273 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 186 KiB/s rd, 2.4 MiB/s wr, 37 op/s 2024-04-07T17:35:25.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:35:25 smithi100 bash[74942]: cluster 2024-04-07T17:35:24.424183+0000 mgr.y (mgr.25175) 2401 : cluster [DBG] pgmap v3943: 137 pgs: 137 active+clean; 273 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 186 KiB/s rd, 2.4 MiB/s wr, 37 op/s 2024-04-07T17:35:25.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:35:25 smithi100 bash[76202]: cluster 2024-04-07T17:35:24.424183+0000 mgr.y (mgr.25175) 2401 : cluster [DBG] pgmap v3943: 137 pgs: 137 active+clean; 273 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 186 KiB/s rd, 2.4 MiB/s wr, 37 op/s 2024-04-07T17:35:27.229 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] EncryptedFlattenTest/7.ShrinkToOne (24972 ms) 2024-04-07T17:35:27.229 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] EncryptedFlattenTest/7.ShrinkToOneAfterSnapshot 2024-04-07T17:35:27.229 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:35:27.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:35:27 smithi144 bash[66654]: cluster 2024-04-07T17:35:26.424677+0000 mgr.y (mgr.25175) 2402 : cluster [DBG] pgmap v3944: 137 pgs: 137 active+clean; 273 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 179 KiB/s rd, 684 KiB/s wr, 28 op/s 2024-04-07T17:35:27.953 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:35:27 smithi100 bash[76202]: cluster 2024-04-07T17:35:26.424677+0000 mgr.y (mgr.25175) 2402 : cluster [DBG] pgmap v3944: 137 pgs: 137 active+clean; 273 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 179 KiB/s rd, 684 KiB/s wr, 28 op/s 2024-04-07T17:35:27.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:35:27 smithi100 bash[74942]: cluster 2024-04-07T17:35:26.424677+0000 mgr.y (mgr.25175) 2402 : cluster [DBG] pgmap v3944: 137 pgs: 137 active+clean; 273 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 179 KiB/s rd, 684 KiB/s wr, 28 op/s 2024-04-07T17:35:28.704 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:35:28 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:35:28] "GET /metrics HTTP/1.1" 200 36766 "" "Prometheus/2.43.0" 2024-04-07T17:35:29.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:35:29 smithi144 bash[66654]: cluster 2024-04-07T17:35:28.426362+0000 mgr.y (mgr.25175) 2403 : cluster [DBG] pgmap v3945: 137 pgs: 137 active+clean; 269 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 914 KiB/s rd, 1.0 MiB/s wr, 35 op/s 2024-04-07T17:35:29.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:35:29 smithi100 bash[74942]: cluster 2024-04-07T17:35:28.426362+0000 mgr.y (mgr.25175) 2403 : cluster [DBG] pgmap v3945: 137 pgs: 137 active+clean; 269 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 914 KiB/s rd, 1.0 MiB/s wr, 35 op/s 2024-04-07T17:35:29.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:35:29 smithi100 bash[76202]: cluster 2024-04-07T17:35:28.426362+0000 mgr.y (mgr.25175) 2403 : cluster [DBG] pgmap v3945: 137 pgs: 137 active+clean; 269 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 914 KiB/s rd, 1.0 MiB/s wr, 35 op/s 2024-04-07T17:35:31.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:35:31 smithi144 bash[66654]: cluster 2024-04-07T17:35:30.427190+0000 mgr.y (mgr.25175) 2404 : cluster [DBG] pgmap v3946: 137 pgs: 137 active+clean; 273 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 877 KiB/s rd, 1.3 MiB/s wr, 19 op/s 2024-04-07T17:35:31.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:35:31 smithi100 bash[74942]: cluster 2024-04-07T17:35:30.427190+0000 mgr.y (mgr.25175) 2404 : cluster [DBG] pgmap v3946: 137 pgs: 137 active+clean; 273 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 877 KiB/s rd, 1.3 MiB/s wr, 19 op/s 2024-04-07T17:35:31.953 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:35:31 smithi100 bash[76202]: cluster 2024-04-07T17:35:30.427190+0000 mgr.y (mgr.25175) 2404 : cluster [DBG] pgmap v3946: 137 pgs: 137 active+clean; 273 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 877 KiB/s rd, 1.3 MiB/s wr, 19 op/s 2024-04-07T17:35:32.604 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:35:32 smithi144 bash[95233]: ts=2024-04-07T17:35:32.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:35:32.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:35:32 smithi144 bash[66654]: audit 2024-04-07T17:35:31.853892+0000 mon.a (mon.0) 3819 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:35:32.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:35:32 smithi100 bash[74942]: audit 2024-04-07T17:35:31.853892+0000 mon.a (mon.0) 3819 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:35:32.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:35:32 smithi100 bash[76202]: audit 2024-04-07T17:35:31.853892+0000 mon.a (mon.0) 3819 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:35:33.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:35:33 smithi144 bash[66654]: cluster 2024-04-07T17:35:32.428003+0000 mgr.y (mgr.25175) 2405 : cluster [DBG] pgmap v3947: 137 pgs: 137 active+clean; 273 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 879 KiB/s rd, 1.3 MiB/s wr, 22 op/s 2024-04-07T17:35:33.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:35:33 smithi100 bash[74942]: cluster 2024-04-07T17:35:32.428003+0000 mgr.y (mgr.25175) 2405 : cluster [DBG] pgmap v3947: 137 pgs: 137 active+clean; 273 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 879 KiB/s rd, 1.3 MiB/s wr, 22 op/s 2024-04-07T17:35:33.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:35:33 smithi100 bash[76202]: cluster 2024-04-07T17:35:32.428003+0000 mgr.y (mgr.25175) 2405 : cluster [DBG] pgmap v3947: 137 pgs: 137 active+clean; 273 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 879 KiB/s rd, 1.3 MiB/s wr, 22 op/s 2024-04-07T17:35:35.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:35:35 smithi144 bash[66654]: cluster 2024-04-07T17:35:34.428711+0000 mgr.y (mgr.25175) 2406 : cluster [DBG] pgmap v3948: 137 pgs: 137 active+clean; 273 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 826 KiB/s rd, 1.0 MiB/s wr, 16 op/s 2024-04-07T17:35:35.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:35:35 smithi100 bash[74942]: cluster 2024-04-07T17:35:34.428711+0000 mgr.y (mgr.25175) 2406 : cluster [DBG] pgmap v3948: 137 pgs: 137 active+clean; 273 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 826 KiB/s rd, 1.0 MiB/s wr, 16 op/s 2024-04-07T17:35:35.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:35:35 smithi100 bash[76202]: cluster 2024-04-07T17:35:34.428711+0000 mgr.y (mgr.25175) 2406 : cluster [DBG] pgmap v3948: 137 pgs: 137 active+clean; 273 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 826 KiB/s rd, 1.0 MiB/s wr, 16 op/s 2024-04-07T17:35:37.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:35:37 smithi100 bash[74942]: cluster 2024-04-07T17:35:36.429301+0000 mgr.y (mgr.25175) 2407 : cluster [DBG] pgmap v3949: 137 pgs: 137 active+clean; 273 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 740 KiB/s rd, 1.0 MiB/s wr, 14 op/s 2024-04-07T17:35:37.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:35:37 smithi100 bash[76202]: cluster 2024-04-07T17:35:36.429301+0000 mgr.y (mgr.25175) 2407 : cluster [DBG] pgmap v3949: 137 pgs: 137 active+clean; 273 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 740 KiB/s rd, 1.0 MiB/s wr, 14 op/s 2024-04-07T17:35:38.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:35:37 smithi144 bash[66654]: cluster 2024-04-07T17:35:36.429301+0000 mgr.y (mgr.25175) 2407 : cluster [DBG] pgmap v3949: 137 pgs: 137 active+clean; 273 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 740 KiB/s rd, 1.0 MiB/s wr, 14 op/s 2024-04-07T17:35:38.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:35:38 smithi100 bash[74942]: cluster 2024-04-07T17:35:37.618432+0000 mon.a (mon.0) 3820 : cluster [DBG] osdmap e1968: 8 total, 8 up, 8 in 2024-04-07T17:35:38.704 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:35:38 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:35:38] "GET /metrics HTTP/1.1" 200 36759 "" "Prometheus/2.43.0" 2024-04-07T17:35:38.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:35:38 smithi100 bash[76202]: cluster 2024-04-07T17:35:37.618432+0000 mon.a (mon.0) 3820 : cluster [DBG] osdmap e1968: 8 total, 8 up, 8 in 2024-04-07T17:35:39.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:35:38 smithi144 bash[66654]: cluster 2024-04-07T17:35:37.618432+0000 mon.a (mon.0) 3820 : cluster [DBG] osdmap e1968: 8 total, 8 up, 8 in 2024-04-07T17:35:39.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:35:39 smithi100 bash[76202]: cluster 2024-04-07T17:35:38.430093+0000 mgr.y (mgr.25175) 2408 : cluster [DBG] pgmap v3951: 137 pgs: 137 active+clean; 310 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 17 KiB/s rd, 4.5 MiB/s wr, 26 op/s 2024-04-07T17:35:39.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:35:39 smithi100 bash[74942]: cluster 2024-04-07T17:35:38.430093+0000 mgr.y (mgr.25175) 2408 : cluster [DBG] pgmap v3951: 137 pgs: 137 active+clean; 310 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 17 KiB/s rd, 4.5 MiB/s wr, 26 op/s 2024-04-07T17:35:40.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:35:39 smithi144 bash[66654]: cluster 2024-04-07T17:35:38.430093+0000 mgr.y (mgr.25175) 2408 : cluster [DBG] pgmap v3951: 137 pgs: 137 active+clean; 310 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 17 KiB/s rd, 4.5 MiB/s wr, 26 op/s 2024-04-07T17:35:41.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:35:41 smithi100 bash[74942]: cluster 2024-04-07T17:35:40.431138+0000 mgr.y (mgr.25175) 2409 : cluster [DBG] pgmap v3952: 137 pgs: 137 active+clean; 314 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 15 KiB/s rd, 4.1 MiB/s wr, 22 op/s 2024-04-07T17:35:41.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:35:41 smithi100 bash[76202]: cluster 2024-04-07T17:35:40.431138+0000 mgr.y (mgr.25175) 2409 : cluster [DBG] pgmap v3952: 137 pgs: 137 active+clean; 314 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 15 KiB/s rd, 4.1 MiB/s wr, 22 op/s 2024-04-07T17:35:42.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:35:41 smithi144 bash[66654]: cluster 2024-04-07T17:35:40.431138+0000 mgr.y (mgr.25175) 2409 : cluster [DBG] pgmap v3952: 137 pgs: 137 active+clean; 314 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 15 KiB/s rd, 4.1 MiB/s wr, 22 op/s 2024-04-07T17:35:42.615 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:35:42 smithi144 bash[95233]: ts=2024-04-07T17:35:42.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:35:43.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:35:43 smithi100 bash[74942]: cluster 2024-04-07T17:35:42.432331+0000 mgr.y (mgr.25175) 2410 : cluster [DBG] pgmap v3953: 137 pgs: 137 active+clean; 318 MiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 55 KiB/s rd, 4.5 MiB/s wr, 36 op/s 2024-04-07T17:35:43.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:35:43 smithi100 bash[76202]: cluster 2024-04-07T17:35:42.432331+0000 mgr.y (mgr.25175) 2410 : cluster [DBG] pgmap v3953: 137 pgs: 137 active+clean; 318 MiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 55 KiB/s rd, 4.5 MiB/s wr, 36 op/s 2024-04-07T17:35:44.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:35:43 smithi144 bash[66654]: cluster 2024-04-07T17:35:42.432331+0000 mgr.y (mgr.25175) 2410 : cluster [DBG] pgmap v3953: 137 pgs: 137 active+clean; 318 MiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 55 KiB/s rd, 4.5 MiB/s wr, 36 op/s 2024-04-07T17:35:45.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:35:45 smithi100 bash[76202]: cluster 2024-04-07T17:35:44.433080+0000 mgr.y (mgr.25175) 2411 : cluster [DBG] pgmap v3954: 137 pgs: 137 active+clean; 318 MiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 55 KiB/s rd, 4.5 MiB/s wr, 37 op/s 2024-04-07T17:35:45.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:35:45 smithi100 bash[74942]: cluster 2024-04-07T17:35:44.433080+0000 mgr.y (mgr.25175) 2411 : cluster [DBG] pgmap v3954: 137 pgs: 137 active+clean; 318 MiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 55 KiB/s rd, 4.5 MiB/s wr, 37 op/s 2024-04-07T17:35:46.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:35:45 smithi144 bash[66654]: cluster 2024-04-07T17:35:44.433080+0000 mgr.y (mgr.25175) 2411 : cluster [DBG] pgmap v3954: 137 pgs: 137 active+clean; 318 MiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 55 KiB/s rd, 4.5 MiB/s wr, 37 op/s 2024-04-07T17:35:47.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:35:47 smithi100 bash[74942]: cluster 2024-04-07T17:35:46.433872+0000 mgr.y (mgr.25175) 2412 : cluster [DBG] pgmap v3955: 137 pgs: 137 active+clean; 318 MiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 55 KiB/s rd, 4.5 MiB/s wr, 37 op/s 2024-04-07T17:35:47.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:35:47 smithi100 bash[74942]: audit 2024-04-07T17:35:46.854729+0000 mon.a (mon.0) 3821 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:35:47.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:35:47 smithi100 bash[76202]: cluster 2024-04-07T17:35:46.433872+0000 mgr.y (mgr.25175) 2412 : cluster [DBG] pgmap v3955: 137 pgs: 137 active+clean; 318 MiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 55 KiB/s rd, 4.5 MiB/s wr, 37 op/s 2024-04-07T17:35:47.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:35:47 smithi100 bash[76202]: audit 2024-04-07T17:35:46.854729+0000 mon.a (mon.0) 3821 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:35:48.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:35:47 smithi144 bash[66654]: cluster 2024-04-07T17:35:46.433872+0000 mgr.y (mgr.25175) 2412 : cluster [DBG] pgmap v3955: 137 pgs: 137 active+clean; 318 MiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 55 KiB/s rd, 4.5 MiB/s wr, 37 op/s 2024-04-07T17:35:48.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:35:47 smithi144 bash[66654]: audit 2024-04-07T17:35:46.854729+0000 mon.a (mon.0) 3821 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:35:48.704 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:35:48 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:35:48] "GET /metrics HTTP/1.1" 200 36755 "" "Prometheus/2.43.0" 2024-04-07T17:35:49.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:35:49 smithi100 bash[74942]: cluster 2024-04-07T17:35:48.434734+0000 mgr.y (mgr.25175) 2413 : cluster [DBG] pgmap v3956: 137 pgs: 137 active+clean; 318 MiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 198 KiB/s rd, 1.8 MiB/s wr, 31 op/s 2024-04-07T17:35:49.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:35:49 smithi100 bash[76202]: cluster 2024-04-07T17:35:48.434734+0000 mgr.y (mgr.25175) 2413 : cluster [DBG] pgmap v3956: 137 pgs: 137 active+clean; 318 MiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 198 KiB/s rd, 1.8 MiB/s wr, 31 op/s 2024-04-07T17:35:50.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:35:49 smithi144 bash[66654]: cluster 2024-04-07T17:35:48.434734+0000 mgr.y (mgr.25175) 2413 : cluster [DBG] pgmap v3956: 137 pgs: 137 active+clean; 318 MiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 198 KiB/s rd, 1.8 MiB/s wr, 31 op/s 2024-04-07T17:35:51.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:35:51 smithi100 bash[74942]: cluster 2024-04-07T17:35:50.435968+0000 mgr.y (mgr.25175) 2414 : cluster [DBG] pgmap v3957: 137 pgs: 137 active+clean; 322 MiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 175 KiB/s rd, 1.0 MiB/s wr, 24 op/s 2024-04-07T17:35:51.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:35:51 smithi100 bash[74942]: cluster 2024-04-07T17:35:50.665666+0000 mon.a (mon.0) 3822 : cluster [DBG] osdmap e1969: 8 total, 8 up, 8 in 2024-04-07T17:35:51.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:35:51 smithi100 bash[76202]: cluster 2024-04-07T17:35:50.435968+0000 mgr.y (mgr.25175) 2414 : cluster [DBG] pgmap v3957: 137 pgs: 137 active+clean; 322 MiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 175 KiB/s rd, 1.0 MiB/s wr, 24 op/s 2024-04-07T17:35:51.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:35:51 smithi100 bash[76202]: cluster 2024-04-07T17:35:50.665666+0000 mon.a (mon.0) 3822 : cluster [DBG] osdmap e1969: 8 total, 8 up, 8 in 2024-04-07T17:35:52.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:35:51 smithi144 bash[66654]: cluster 2024-04-07T17:35:50.435968+0000 mgr.y (mgr.25175) 2414 : cluster [DBG] pgmap v3957: 137 pgs: 137 active+clean; 322 MiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 175 KiB/s rd, 1.0 MiB/s wr, 24 op/s 2024-04-07T17:35:52.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:35:51 smithi144 bash[66654]: cluster 2024-04-07T17:35:50.665666+0000 mon.a (mon.0) 3822 : cluster [DBG] osdmap e1969: 8 total, 8 up, 8 in 2024-04-07T17:35:52.615 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:35:52 smithi144 bash[95233]: ts=2024-04-07T17:35:52.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:35:53.176 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] EncryptedFlattenTest/7.ShrinkToOneAfterSnapshot (25947 ms) 2024-04-07T17:35:53.176 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] EncryptedFlattenTest/7.MinOverlap 2024-04-07T17:35:53.176 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:35:53.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:35:53 smithi100 bash[74942]: cluster 2024-04-07T17:35:52.437163+0000 mgr.y (mgr.25175) 2415 : cluster [DBG] pgmap v3959: 137 pgs: 137 active+clean; 350 MiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 1.8 MiB/s rd, 3.2 MiB/s wr, 21 op/s 2024-04-07T17:35:53.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:35:53 smithi100 bash[76202]: cluster 2024-04-07T17:35:52.437163+0000 mgr.y (mgr.25175) 2415 : cluster [DBG] pgmap v3959: 137 pgs: 137 active+clean; 350 MiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 1.8 MiB/s rd, 3.2 MiB/s wr, 21 op/s 2024-04-07T17:35:54.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:35:53 smithi144 bash[66654]: cluster 2024-04-07T17:35:52.437163+0000 mgr.y (mgr.25175) 2415 : cluster [DBG] pgmap v3959: 137 pgs: 137 active+clean; 350 MiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 1.8 MiB/s rd, 3.2 MiB/s wr, 21 op/s 2024-04-07T17:35:55.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:35:55 smithi100 bash[74942]: cluster 2024-04-07T17:35:54.437978+0000 mgr.y (mgr.25175) 2416 : cluster [DBG] pgmap v3960: 137 pgs: 137 active+clean; 370 MiB data, 4.7 GiB used, 711 GiB / 715 GiB avail; 3.0 MiB/s rd, 5.2 MiB/s wr, 39 op/s 2024-04-07T17:35:55.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:35:55 smithi100 bash[76202]: cluster 2024-04-07T17:35:54.437978+0000 mgr.y (mgr.25175) 2416 : cluster [DBG] pgmap v3960: 137 pgs: 137 active+clean; 370 MiB data, 4.7 GiB used, 711 GiB / 715 GiB avail; 3.0 MiB/s rd, 5.2 MiB/s wr, 39 op/s 2024-04-07T17:35:56.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:35:55 smithi144 bash[66654]: cluster 2024-04-07T17:35:54.437978+0000 mgr.y (mgr.25175) 2416 : cluster [DBG] pgmap v3960: 137 pgs: 137 active+clean; 370 MiB data, 4.7 GiB used, 711 GiB / 715 GiB avail; 3.0 MiB/s rd, 5.2 MiB/s wr, 39 op/s 2024-04-07T17:35:57.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:35:57 smithi100 bash[74942]: cluster 2024-04-07T17:35:56.438620+0000 mgr.y (mgr.25175) 2417 : cluster [DBG] pgmap v3961: 137 pgs: 137 active+clean; 378 MiB data, 4.7 GiB used, 711 GiB / 715 GiB avail; 3.0 MiB/s rd, 6.0 MiB/s wr, 39 op/s 2024-04-07T17:35:57.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:35:57 smithi100 bash[76202]: cluster 2024-04-07T17:35:56.438620+0000 mgr.y (mgr.25175) 2417 : cluster [DBG] pgmap v3961: 137 pgs: 137 active+clean; 378 MiB data, 4.7 GiB used, 711 GiB / 715 GiB avail; 3.0 MiB/s rd, 6.0 MiB/s wr, 39 op/s 2024-04-07T17:35:58.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:35:57 smithi144 bash[66654]: cluster 2024-04-07T17:35:56.438620+0000 mgr.y (mgr.25175) 2417 : cluster [DBG] pgmap v3961: 137 pgs: 137 active+clean; 378 MiB data, 4.7 GiB used, 711 GiB / 715 GiB avail; 3.0 MiB/s rd, 6.0 MiB/s wr, 39 op/s 2024-04-07T17:35:58.703 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:35:58 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:35:58] "GET /metrics HTTP/1.1" 200 36755 "" "Prometheus/2.43.0" 2024-04-07T17:35:59.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:35:59 smithi100 bash[74942]: cluster 2024-04-07T17:35:58.439547+0000 mgr.y (mgr.25175) 2418 : cluster [DBG] pgmap v3962: 137 pgs: 137 active+clean; 378 MiB data, 4.7 GiB used, 711 GiB / 715 GiB avail; 2.9 MiB/s rd, 6.0 MiB/s wr, 35 op/s 2024-04-07T17:35:59.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:35:59 smithi100 bash[76202]: cluster 2024-04-07T17:35:58.439547+0000 mgr.y (mgr.25175) 2418 : cluster [DBG] pgmap v3962: 137 pgs: 137 active+clean; 378 MiB data, 4.7 GiB used, 711 GiB / 715 GiB avail; 2.9 MiB/s rd, 6.0 MiB/s wr, 35 op/s 2024-04-07T17:36:00.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:35:59 smithi144 bash[66654]: cluster 2024-04-07T17:35:58.439547+0000 mgr.y (mgr.25175) 2418 : cluster [DBG] pgmap v3962: 137 pgs: 137 active+clean; 378 MiB data, 4.7 GiB used, 711 GiB / 715 GiB avail; 2.9 MiB/s rd, 6.0 MiB/s wr, 35 op/s 2024-04-07T17:36:01.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:36:01 smithi100 bash[74942]: cluster 2024-04-07T17:36:00.440261+0000 mgr.y (mgr.25175) 2419 : cluster [DBG] pgmap v3963: 137 pgs: 137 active+clean; 378 MiB data, 4.7 GiB used, 711 GiB / 715 GiB avail; 2.9 MiB/s rd, 5.6 MiB/s wr, 35 op/s 2024-04-07T17:36:01.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:36:01 smithi100 bash[76202]: cluster 2024-04-07T17:36:00.440261+0000 mgr.y (mgr.25175) 2419 : cluster [DBG] pgmap v3963: 137 pgs: 137 active+clean; 378 MiB data, 4.7 GiB used, 711 GiB / 715 GiB avail; 2.9 MiB/s rd, 5.6 MiB/s wr, 35 op/s 2024-04-07T17:36:02.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:36:01 smithi144 bash[66654]: cluster 2024-04-07T17:36:00.440261+0000 mgr.y (mgr.25175) 2419 : cluster [DBG] pgmap v3963: 137 pgs: 137 active+clean; 378 MiB data, 4.7 GiB used, 711 GiB / 715 GiB avail; 2.9 MiB/s rd, 5.6 MiB/s wr, 35 op/s 2024-04-07T17:36:02.615 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:36:02 smithi144 bash[95233]: ts=2024-04-07T17:36:02.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:36:02.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:36:02 smithi100 bash[74942]: audit 2024-04-07T17:36:01.854624+0000 mon.a (mon.0) 3823 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:36:02.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:36:02 smithi100 bash[76202]: audit 2024-04-07T17:36:01.854624+0000 mon.a (mon.0) 3823 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:36:03.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:36:02 smithi144 bash[66654]: audit 2024-04-07T17:36:01.854624+0000 mon.a (mon.0) 3823 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:36:03.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:36:03 smithi100 bash[74942]: cluster 2024-04-07T17:36:02.441034+0000 mgr.y (mgr.25175) 2420 : cluster [DBG] pgmap v3964: 137 pgs: 137 active+clean; 378 MiB data, 4.7 GiB used, 711 GiB / 715 GiB avail; 2.4 MiB/s rd, 4.8 MiB/s wr, 29 op/s 2024-04-07T17:36:03.955 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:36:03 smithi100 bash[76202]: cluster 2024-04-07T17:36:02.441034+0000 mgr.y (mgr.25175) 2420 : cluster [DBG] pgmap v3964: 137 pgs: 137 active+clean; 378 MiB data, 4.7 GiB used, 711 GiB / 715 GiB avail; 2.4 MiB/s rd, 4.8 MiB/s wr, 29 op/s 2024-04-07T17:36:04.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:36:03 smithi144 bash[66654]: cluster 2024-04-07T17:36:02.441034+0000 mgr.y (mgr.25175) 2420 : cluster [DBG] pgmap v3964: 137 pgs: 137 active+clean; 378 MiB data, 4.7 GiB used, 711 GiB / 715 GiB avail; 2.4 MiB/s rd, 4.8 MiB/s wr, 29 op/s 2024-04-07T17:36:05.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:36:04 smithi144 bash[66654]: cluster 2024-04-07T17:36:03.707983+0000 mon.a (mon.0) 3824 : cluster [DBG] osdmap e1970: 8 total, 8 up, 8 in 2024-04-07T17:36:05.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:36:04 smithi100 bash[74942]: cluster 2024-04-07T17:36:03.707983+0000 mon.a (mon.0) 3824 : cluster [DBG] osdmap e1970: 8 total, 8 up, 8 in 2024-04-07T17:36:05.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:36:04 smithi100 bash[76202]: cluster 2024-04-07T17:36:03.707983+0000 mon.a (mon.0) 3824 : cluster [DBG] osdmap e1970: 8 total, 8 up, 8 in 2024-04-07T17:36:06.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:36:05 smithi144 bash[66654]: cluster 2024-04-07T17:36:04.441948+0000 mgr.y (mgr.25175) 2421 : cluster [DBG] pgmap v3966: 137 pgs: 137 active+clean; 386 MiB data, 4.7 GiB used, 711 GiB / 715 GiB avail; 3.5 KiB/s rd, 1.6 MiB/s wr, 6 op/s 2024-04-07T17:36:06.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:36:05 smithi100 bash[74942]: cluster 2024-04-07T17:36:04.441948+0000 mgr.y (mgr.25175) 2421 : cluster [DBG] pgmap v3966: 137 pgs: 137 active+clean; 386 MiB data, 4.7 GiB used, 711 GiB / 715 GiB avail; 3.5 KiB/s rd, 1.6 MiB/s wr, 6 op/s 2024-04-07T17:36:06.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:36:05 smithi100 bash[76202]: cluster 2024-04-07T17:36:04.441948+0000 mgr.y (mgr.25175) 2421 : cluster [DBG] pgmap v3966: 137 pgs: 137 active+clean; 386 MiB data, 4.7 GiB used, 711 GiB / 715 GiB avail; 3.5 KiB/s rd, 1.6 MiB/s wr, 6 op/s 2024-04-07T17:36:08.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:36:07 smithi144 bash[66654]: cluster 2024-04-07T17:36:06.442781+0000 mgr.y (mgr.25175) 2422 : cluster [DBG] pgmap v3967: 137 pgs: 137 active+clean; 391 MiB data, 4.7 GiB used, 711 GiB / 715 GiB avail; 3.5 KiB/s rd, 1.3 MiB/s wr, 6 op/s 2024-04-07T17:36:08.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:36:07 smithi100 bash[74942]: cluster 2024-04-07T17:36:06.442781+0000 mgr.y (mgr.25175) 2422 : cluster [DBG] pgmap v3967: 137 pgs: 137 active+clean; 391 MiB data, 4.7 GiB used, 711 GiB / 715 GiB avail; 3.5 KiB/s rd, 1.3 MiB/s wr, 6 op/s 2024-04-07T17:36:08.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:36:07 smithi100 bash[76202]: cluster 2024-04-07T17:36:06.442781+0000 mgr.y (mgr.25175) 2422 : cluster [DBG] pgmap v3967: 137 pgs: 137 active+clean; 391 MiB data, 4.7 GiB used, 711 GiB / 715 GiB avail; 3.5 KiB/s rd, 1.3 MiB/s wr, 6 op/s 2024-04-07T17:36:08.703 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:36:08 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:36:08] "GET /metrics HTTP/1.1" 200 36759 "" "Prometheus/2.43.0" 2024-04-07T17:36:09.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:36:09 smithi100 bash[74942]: cluster 2024-04-07T17:36:08.444074+0000 mgr.y (mgr.25175) 2423 : cluster [DBG] pgmap v3968: 137 pgs: 137 active+clean; 419 MiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 26 KiB/s rd, 4.1 MiB/s wr, 35 op/s 2024-04-07T17:36:09.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:36:09 smithi100 bash[76202]: cluster 2024-04-07T17:36:08.444074+0000 mgr.y (mgr.25175) 2423 : cluster [DBG] pgmap v3968: 137 pgs: 137 active+clean; 419 MiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 26 KiB/s rd, 4.1 MiB/s wr, 35 op/s 2024-04-07T17:36:09.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:36:09 smithi144 bash[66654]: cluster 2024-04-07T17:36:08.444074+0000 mgr.y (mgr.25175) 2423 : cluster [DBG] pgmap v3968: 137 pgs: 137 active+clean; 419 MiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 26 KiB/s rd, 4.1 MiB/s wr, 35 op/s 2024-04-07T17:36:10.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:36:10 smithi100 bash[74942]: audit 2024-04-07T17:36:09.475312+0000 mon.a (mon.0) 3825 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:36:10.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:36:10 smithi100 bash[74942]: audit 2024-04-07T17:36:10.067409+0000 mon.a (mon.0) 3826 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:36:10.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:36:10 smithi100 bash[74942]: audit 2024-04-07T17:36:10.076916+0000 mon.a (mon.0) 3827 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:36:10.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:36:10 smithi100 bash[76202]: audit 2024-04-07T17:36:09.475312+0000 mon.a (mon.0) 3825 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:36:10.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:36:10 smithi100 bash[76202]: audit 2024-04-07T17:36:10.067409+0000 mon.a (mon.0) 3826 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:36:10.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:36:10 smithi100 bash[76202]: audit 2024-04-07T17:36:10.076916+0000 mon.a (mon.0) 3827 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:36:10.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:36:10 smithi144 bash[66654]: audit 2024-04-07T17:36:09.475312+0000 mon.a (mon.0) 3825 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:36:10.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:36:10 smithi144 bash[66654]: audit 2024-04-07T17:36:10.067409+0000 mon.a (mon.0) 3826 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:36:10.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:36:10 smithi144 bash[66654]: audit 2024-04-07T17:36:10.076916+0000 mon.a (mon.0) 3827 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:36:11.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:36:11 smithi100 bash[74942]: cluster 2024-04-07T17:36:10.444914+0000 mgr.y (mgr.25175) 2424 : cluster [DBG] pgmap v3969: 137 pgs: 137 active+clean; 423 MiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 55 KiB/s rd, 4.5 MiB/s wr, 36 op/s 2024-04-07T17:36:11.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:36:11 smithi100 bash[74942]: audit 2024-04-07T17:36:10.722533+0000 mon.a (mon.0) 3828 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:36:11.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:36:11 smithi100 bash[74942]: audit 2024-04-07T17:36:10.723484+0000 mon.a (mon.0) 3829 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:36:11.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:36:11 smithi100 bash[74942]: audit 2024-04-07T17:36:10.729241+0000 mon.a (mon.0) 3830 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:36:11.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:36:11 smithi100 bash[76202]: cluster 2024-04-07T17:36:10.444914+0000 mgr.y (mgr.25175) 2424 : cluster [DBG] pgmap v3969: 137 pgs: 137 active+clean; 423 MiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 55 KiB/s rd, 4.5 MiB/s wr, 36 op/s 2024-04-07T17:36:11.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:36:11 smithi100 bash[76202]: audit 2024-04-07T17:36:10.722533+0000 mon.a (mon.0) 3828 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:36:11.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:36:11 smithi100 bash[76202]: audit 2024-04-07T17:36:10.723484+0000 mon.a (mon.0) 3829 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:36:11.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:36:11 smithi100 bash[76202]: audit 2024-04-07T17:36:10.729241+0000 mon.a (mon.0) 3830 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:36:11.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:36:11 smithi144 bash[66654]: cluster 2024-04-07T17:36:10.444914+0000 mgr.y (mgr.25175) 2424 : cluster [DBG] pgmap v3969: 137 pgs: 137 active+clean; 423 MiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 55 KiB/s rd, 4.5 MiB/s wr, 36 op/s 2024-04-07T17:36:11.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:36:11 smithi144 bash[66654]: audit 2024-04-07T17:36:10.722533+0000 mon.a (mon.0) 3828 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:36:11.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:36:11 smithi144 bash[66654]: audit 2024-04-07T17:36:10.723484+0000 mon.a (mon.0) 3829 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:36:11.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:36:11 smithi144 bash[66654]: audit 2024-04-07T17:36:10.729241+0000 mon.a (mon.0) 3830 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:36:12.615 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:36:12 smithi144 bash[95233]: ts=2024-04-07T17:36:12.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:36:13.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:36:13 smithi144 bash[66654]: cluster 2024-04-07T17:36:12.446062+0000 mgr.y (mgr.25175) 2425 : cluster [DBG] pgmap v3970: 137 pgs: 137 active+clean; 423 MiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 84 KiB/s rd, 4.5 MiB/s wr, 37 op/s 2024-04-07T17:36:13.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:36:13 smithi100 bash[74942]: cluster 2024-04-07T17:36:12.446062+0000 mgr.y (mgr.25175) 2425 : cluster [DBG] pgmap v3970: 137 pgs: 137 active+clean; 423 MiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 84 KiB/s rd, 4.5 MiB/s wr, 37 op/s 2024-04-07T17:36:13.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:36:13 smithi100 bash[76202]: cluster 2024-04-07T17:36:12.446062+0000 mgr.y (mgr.25175) 2425 : cluster [DBG] pgmap v3970: 137 pgs: 137 active+clean; 423 MiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 84 KiB/s rd, 4.5 MiB/s wr, 37 op/s 2024-04-07T17:36:15.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:36:15 smithi144 bash[66654]: cluster 2024-04-07T17:36:14.446797+0000 mgr.y (mgr.25175) 2426 : cluster [DBG] pgmap v3971: 137 pgs: 137 active+clean; 427 MiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 108 KiB/s rd, 4.6 MiB/s wr, 39 op/s 2024-04-07T17:36:15.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:36:15 smithi100 bash[74942]: cluster 2024-04-07T17:36:14.446797+0000 mgr.y (mgr.25175) 2426 : cluster [DBG] pgmap v3971: 137 pgs: 137 active+clean; 427 MiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 108 KiB/s rd, 4.6 MiB/s wr, 39 op/s 2024-04-07T17:36:15.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:36:15 smithi100 bash[76202]: cluster 2024-04-07T17:36:14.446797+0000 mgr.y (mgr.25175) 2426 : cluster [DBG] pgmap v3971: 137 pgs: 137 active+clean; 427 MiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 108 KiB/s rd, 4.6 MiB/s wr, 39 op/s 2024-04-07T17:36:17.866 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:36:17 smithi144 bash[66654]: cluster 2024-04-07T17:36:16.447659+0000 mgr.y (mgr.25175) 2427 : cluster [DBG] pgmap v3972: 137 pgs: 137 active+clean; 427 MiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 182 KiB/s rd, 3.4 MiB/s wr, 34 op/s 2024-04-07T17:36:17.866 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:36:17 smithi144 bash[66654]: audit 2024-04-07T17:36:16.855460+0000 mon.a (mon.0) 3831 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:36:17.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:36:17 smithi100 bash[76202]: cluster 2024-04-07T17:36:16.447659+0000 mgr.y (mgr.25175) 2427 : cluster [DBG] pgmap v3972: 137 pgs: 137 active+clean; 427 MiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 182 KiB/s rd, 3.4 MiB/s wr, 34 op/s 2024-04-07T17:36:17.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:36:17 smithi100 bash[76202]: audit 2024-04-07T17:36:16.855460+0000 mon.a (mon.0) 3831 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:36:17.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:36:17 smithi100 bash[74942]: cluster 2024-04-07T17:36:16.447659+0000 mgr.y (mgr.25175) 2427 : cluster [DBG] pgmap v3972: 137 pgs: 137 active+clean; 427 MiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 182 KiB/s rd, 3.4 MiB/s wr, 34 op/s 2024-04-07T17:36:17.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:36:17 smithi100 bash[74942]: audit 2024-04-07T17:36:16.855460+0000 mon.a (mon.0) 3831 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:36:18.704 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:36:18 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:36:18] "GET /metrics HTTP/1.1" 200 36757 "" "Prometheus/2.43.0" 2024-04-07T17:36:18.716 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] EncryptedFlattenTest/7.MinOverlap (25540 ms) 2024-04-07T17:36:18.716 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] EncryptedFlattenTest/7.ZeroOverlap 2024-04-07T17:36:18.716 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:36:19.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:36:19 smithi144 bash[66654]: cluster 2024-04-07T17:36:18.448455+0000 mgr.y (mgr.25175) 2428 : cluster [DBG] pgmap v3973: 137 pgs: 137 active+clean; 427 MiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 215 KiB/s rd, 3.0 MiB/s wr, 45 op/s 2024-04-07T17:36:19.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:36:19 smithi100 bash[74942]: cluster 2024-04-07T17:36:18.448455+0000 mgr.y (mgr.25175) 2428 : cluster [DBG] pgmap v3973: 137 pgs: 137 active+clean; 427 MiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 215 KiB/s rd, 3.0 MiB/s wr, 45 op/s 2024-04-07T17:36:19.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:36:19 smithi100 bash[76202]: cluster 2024-04-07T17:36:18.448455+0000 mgr.y (mgr.25175) 2428 : cluster [DBG] pgmap v3973: 137 pgs: 137 active+clean; 427 MiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 215 KiB/s rd, 3.0 MiB/s wr, 45 op/s 2024-04-07T17:36:21.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:36:21 smithi144 bash[66654]: cluster 2024-04-07T17:36:20.449268+0000 mgr.y (mgr.25175) 2429 : cluster [DBG] pgmap v3974: 137 pgs: 137 active+clean; 427 MiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 1.9 MiB/s rd, 1.7 MiB/s wr, 21 op/s 2024-04-07T17:36:21.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:36:21 smithi100 bash[74942]: cluster 2024-04-07T17:36:20.449268+0000 mgr.y (mgr.25175) 2429 : cluster [DBG] pgmap v3974: 137 pgs: 137 active+clean; 427 MiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 1.9 MiB/s rd, 1.7 MiB/s wr, 21 op/s 2024-04-07T17:36:21.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:36:21 smithi100 bash[76202]: cluster 2024-04-07T17:36:20.449268+0000 mgr.y (mgr.25175) 2429 : cluster [DBG] pgmap v3974: 137 pgs: 137 active+clean; 427 MiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 1.9 MiB/s rd, 1.7 MiB/s wr, 21 op/s 2024-04-07T17:36:22.615 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:36:22 smithi144 bash[95233]: ts=2024-04-07T17:36:22.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:36:23.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:36:23 smithi144 bash[66654]: cluster 2024-04-07T17:36:22.450504+0000 mgr.y (mgr.25175) 2430 : cluster [DBG] pgmap v3975: 137 pgs: 137 active+clean; 427 MiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 1.9 MiB/s rd, 1.3 MiB/s wr, 24 op/s 2024-04-07T17:36:23.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:36:23 smithi100 bash[74942]: cluster 2024-04-07T17:36:22.450504+0000 mgr.y (mgr.25175) 2430 : cluster [DBG] pgmap v3975: 137 pgs: 137 active+clean; 427 MiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 1.9 MiB/s rd, 1.3 MiB/s wr, 24 op/s 2024-04-07T17:36:23.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:36:23 smithi100 bash[76202]: cluster 2024-04-07T17:36:22.450504+0000 mgr.y (mgr.25175) 2430 : cluster [DBG] pgmap v3975: 137 pgs: 137 active+clean; 427 MiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 1.9 MiB/s rd, 1.3 MiB/s wr, 24 op/s 2024-04-07T17:36:25.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:36:25 smithi144 bash[66654]: cluster 2024-04-07T17:36:24.451290+0000 mgr.y (mgr.25175) 2431 : cluster [DBG] pgmap v3976: 137 pgs: 137 active+clean; 427 MiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 1.8 MiB/s rd, 1.3 MiB/s wr, 24 op/s 2024-04-07T17:36:25.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:36:25 smithi100 bash[74942]: cluster 2024-04-07T17:36:24.451290+0000 mgr.y (mgr.25175) 2431 : cluster [DBG] pgmap v3976: 137 pgs: 137 active+clean; 427 MiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 1.8 MiB/s rd, 1.3 MiB/s wr, 24 op/s 2024-04-07T17:36:25.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:36:25 smithi100 bash[76202]: cluster 2024-04-07T17:36:24.451290+0000 mgr.y (mgr.25175) 2431 : cluster [DBG] pgmap v3976: 137 pgs: 137 active+clean; 427 MiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 1.8 MiB/s rd, 1.3 MiB/s wr, 24 op/s 2024-04-07T17:36:27.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:36:27 smithi144 bash[66654]: cluster 2024-04-07T17:36:26.451999+0000 mgr.y (mgr.25175) 2432 : cluster [DBG] pgmap v3977: 137 pgs: 137 active+clean; 427 MiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 1.8 MiB/s rd, 1.0 MiB/s wr, 19 op/s 2024-04-07T17:36:27.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:36:27 smithi100 bash[74942]: cluster 2024-04-07T17:36:26.451999+0000 mgr.y (mgr.25175) 2432 : cluster [DBG] pgmap v3977: 137 pgs: 137 active+clean; 427 MiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 1.8 MiB/s rd, 1.0 MiB/s wr, 19 op/s 2024-04-07T17:36:27.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:36:27 smithi100 bash[76202]: cluster 2024-04-07T17:36:26.451999+0000 mgr.y (mgr.25175) 2432 : cluster [DBG] pgmap v3977: 137 pgs: 137 active+clean; 427 MiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 1.8 MiB/s rd, 1.0 MiB/s wr, 19 op/s 2024-04-07T17:36:28.704 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:36:28 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:36:28] "GET /metrics HTTP/1.1" 200 36757 "" "Prometheus/2.43.0" 2024-04-07T17:36:29.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:36:29 smithi144 bash[66654]: cluster 2024-04-07T17:36:28.453287+0000 mgr.y (mgr.25175) 2433 : cluster [DBG] pgmap v3978: 137 pgs: 137 active+clean; 427 MiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 1.7 MiB/s rd, 1.0 MiB/s wr, 19 op/s 2024-04-07T17:36:29.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:36:29 smithi144 bash[66654]: cluster 2024-04-07T17:36:28.542580+0000 mon.a (mon.0) 3832 : cluster [DBG] osdmap e1971: 8 total, 8 up, 8 in 2024-04-07T17:36:29.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:36:29 smithi100 bash[76202]: cluster 2024-04-07T17:36:28.453287+0000 mgr.y (mgr.25175) 2433 : cluster [DBG] pgmap v3978: 137 pgs: 137 active+clean; 427 MiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 1.7 MiB/s rd, 1.0 MiB/s wr, 19 op/s 2024-04-07T17:36:29.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:36:29 smithi100 bash[76202]: cluster 2024-04-07T17:36:28.542580+0000 mon.a (mon.0) 3832 : cluster [DBG] osdmap e1971: 8 total, 8 up, 8 in 2024-04-07T17:36:29.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:36:29 smithi100 bash[74942]: cluster 2024-04-07T17:36:28.453287+0000 mgr.y (mgr.25175) 2433 : cluster [DBG] pgmap v3978: 137 pgs: 137 active+clean; 427 MiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 1.7 MiB/s rd, 1.0 MiB/s wr, 19 op/s 2024-04-07T17:36:29.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:36:29 smithi100 bash[74942]: cluster 2024-04-07T17:36:28.542580+0000 mon.a (mon.0) 3832 : cluster [DBG] osdmap e1971: 8 total, 8 up, 8 in 2024-04-07T17:36:31.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:36:31 smithi144 bash[66654]: cluster 2024-04-07T17:36:30.454344+0000 mgr.y (mgr.25175) 2434 : cluster [DBG] pgmap v3980: 137 pgs: 137 active+clean; 439 MiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 7.7 KiB/s rd, 1.2 MiB/s wr, 10 op/s 2024-04-07T17:36:31.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:36:31 smithi100 bash[76202]: cluster 2024-04-07T17:36:30.454344+0000 mgr.y (mgr.25175) 2434 : cluster [DBG] pgmap v3980: 137 pgs: 137 active+clean; 439 MiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 7.7 KiB/s rd, 1.2 MiB/s wr, 10 op/s 2024-04-07T17:36:31.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:36:31 smithi100 bash[74942]: cluster 2024-04-07T17:36:30.454344+0000 mgr.y (mgr.25175) 2434 : cluster [DBG] pgmap v3980: 137 pgs: 137 active+clean; 439 MiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 7.7 KiB/s rd, 1.2 MiB/s wr, 10 op/s 2024-04-07T17:36:32.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:36:32 smithi144 bash[66654]: audit 2024-04-07T17:36:31.855884+0000 mon.a (mon.0) 3833 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:36:32.616 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:36:32 smithi144 bash[95233]: ts=2024-04-07T17:36:32.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:36:32.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:36:32 smithi100 bash[74942]: audit 2024-04-07T17:36:31.855884+0000 mon.a (mon.0) 3833 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:36:32.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:36:32 smithi100 bash[76202]: audit 2024-04-07T17:36:31.855884+0000 mon.a (mon.0) 3833 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:36:33.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:36:33 smithi144 bash[66654]: cluster 2024-04-07T17:36:32.455794+0000 mgr.y (mgr.25175) 2435 : cluster [DBG] pgmap v3981: 137 pgs: 137 active+clean; 460 MiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 25 KiB/s rd, 3.3 MiB/s wr, 34 op/s 2024-04-07T17:36:33.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:36:33 smithi100 bash[76202]: cluster 2024-04-07T17:36:32.455794+0000 mgr.y (mgr.25175) 2435 : cluster [DBG] pgmap v3981: 137 pgs: 137 active+clean; 460 MiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 25 KiB/s rd, 3.3 MiB/s wr, 34 op/s 2024-04-07T17:36:33.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:36:33 smithi100 bash[74942]: cluster 2024-04-07T17:36:32.455794+0000 mgr.y (mgr.25175) 2435 : cluster [DBG] pgmap v3981: 137 pgs: 137 active+clean; 460 MiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 25 KiB/s rd, 3.3 MiB/s wr, 34 op/s 2024-04-07T17:36:35.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:36:35 smithi144 bash[66654]: cluster 2024-04-07T17:36:34.456432+0000 mgr.y (mgr.25175) 2436 : cluster [DBG] pgmap v3982: 137 pgs: 137 active+clean; 468 MiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 26 KiB/s rd, 4.1 MiB/s wr, 36 op/s 2024-04-07T17:36:35.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:36:35 smithi100 bash[74942]: cluster 2024-04-07T17:36:34.456432+0000 mgr.y (mgr.25175) 2436 : cluster [DBG] pgmap v3982: 137 pgs: 137 active+clean; 468 MiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 26 KiB/s rd, 4.1 MiB/s wr, 36 op/s 2024-04-07T17:36:35.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:36:35 smithi100 bash[76202]: cluster 2024-04-07T17:36:34.456432+0000 mgr.y (mgr.25175) 2436 : cluster [DBG] pgmap v3982: 137 pgs: 137 active+clean; 468 MiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 26 KiB/s rd, 4.1 MiB/s wr, 36 op/s 2024-04-07T17:36:37.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:36:37 smithi144 bash[66654]: cluster 2024-04-07T17:36:36.456953+0000 mgr.y (mgr.25175) 2437 : cluster [DBG] pgmap v3983: 137 pgs: 137 active+clean; 468 MiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 26 KiB/s rd, 4.1 MiB/s wr, 36 op/s 2024-04-07T17:36:37.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:36:37 smithi100 bash[76202]: cluster 2024-04-07T17:36:36.456953+0000 mgr.y (mgr.25175) 2437 : cluster [DBG] pgmap v3983: 137 pgs: 137 active+clean; 468 MiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 26 KiB/s rd, 4.1 MiB/s wr, 36 op/s 2024-04-07T17:36:37.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:36:37 smithi100 bash[74942]: cluster 2024-04-07T17:36:36.456953+0000 mgr.y (mgr.25175) 2437 : cluster [DBG] pgmap v3983: 137 pgs: 137 active+clean; 468 MiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 26 KiB/s rd, 4.1 MiB/s wr, 36 op/s 2024-04-07T17:36:38.704 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:36:38 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:36:38] "GET /metrics HTTP/1.1" 200 36759 "" "Prometheus/2.43.0" 2024-04-07T17:36:39.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:36:39 smithi144 bash[66654]: cluster 2024-04-07T17:36:38.458173+0000 mgr.y (mgr.25175) 2438 : cluster [DBG] pgmap v3984: 137 pgs: 137 active+clean; 476 MiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 191 KiB/s rd, 4.9 MiB/s wr, 43 op/s 2024-04-07T17:36:39.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:36:39 smithi100 bash[74942]: cluster 2024-04-07T17:36:38.458173+0000 mgr.y (mgr.25175) 2438 : cluster [DBG] pgmap v3984: 137 pgs: 137 active+clean; 476 MiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 191 KiB/s rd, 4.9 MiB/s wr, 43 op/s 2024-04-07T17:36:39.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:36:39 smithi100 bash[76202]: cluster 2024-04-07T17:36:38.458173+0000 mgr.y (mgr.25175) 2438 : cluster [DBG] pgmap v3984: 137 pgs: 137 active+clean; 476 MiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 191 KiB/s rd, 4.9 MiB/s wr, 43 op/s 2024-04-07T17:36:41.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:36:41 smithi144 bash[66654]: cluster 2024-04-07T17:36:40.458888+0000 mgr.y (mgr.25175) 2439 : cluster [DBG] pgmap v3985: 137 pgs: 137 active+clean; 476 MiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 210 KiB/s rd, 4.1 MiB/s wr, 38 op/s 2024-04-07T17:36:41.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:36:41 smithi100 bash[74942]: cluster 2024-04-07T17:36:40.458888+0000 mgr.y (mgr.25175) 2439 : cluster [DBG] pgmap v3985: 137 pgs: 137 active+clean; 476 MiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 210 KiB/s rd, 4.1 MiB/s wr, 38 op/s 2024-04-07T17:36:41.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:36:41 smithi100 bash[76202]: cluster 2024-04-07T17:36:40.458888+0000 mgr.y (mgr.25175) 2439 : cluster [DBG] pgmap v3985: 137 pgs: 137 active+clean; 476 MiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 210 KiB/s rd, 4.1 MiB/s wr, 38 op/s 2024-04-07T17:36:42.615 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:36:42 smithi144 bash[95233]: ts=2024-04-07T17:36:42.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:36:43.059 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] EncryptedFlattenTest/7.ZeroOverlap (24343 ms) 2024-04-07T17:36:43.596 INFO:tasks.workunit.client.0.smithi100.stdout:[----------] 7 tests from EncryptedFlattenTest/7 (178024 ms total) 2024-04-07T17:36:43.596 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:36:43.596 INFO:tasks.workunit.client.0.smithi100.stdout:[----------] 15 tests from DiffIterateTest/0, where TypeParam = DiffIterateParams 2024-04-07T17:36:43.636 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] DiffIterateTest/0.DiffIterate 2024-04-07T17:36:43.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:36:43 smithi144 bash[66654]: cluster 2024-04-07T17:36:42.459624+0000 mgr.y (mgr.25175) 2440 : cluster [DBG] pgmap v3986: 137 pgs: 137 active+clean; 476 MiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 571 KiB/s rd, 3.4 MiB/s wr, 34 op/s 2024-04-07T17:36:43.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:36:43 smithi100 bash[74942]: cluster 2024-04-07T17:36:42.459624+0000 mgr.y (mgr.25175) 2440 : cluster [DBG] pgmap v3986: 137 pgs: 137 active+clean; 476 MiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 571 KiB/s rd, 3.4 MiB/s wr, 34 op/s 2024-04-07T17:36:43.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:36:43 smithi100 bash[76202]: cluster 2024-04-07T17:36:42.459624+0000 mgr.y (mgr.25175) 2440 : cluster [DBG] pgmap v3986: 137 pgs: 137 active+clean; 476 MiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 571 KiB/s rd, 3.4 MiB/s wr, 34 op/s 2024-04-07T17:36:44.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:36:44 smithi144 bash[66654]: cluster 2024-04-07T17:36:43.582352+0000 mon.a (mon.0) 3834 : cluster [DBG] osdmap e1972: 8 total, 8 up, 8 in 2024-04-07T17:36:44.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:36:44 smithi100 bash[74942]: cluster 2024-04-07T17:36:43.582352+0000 mon.a (mon.0) 3834 : cluster [DBG] osdmap e1972: 8 total, 8 up, 8 in 2024-04-07T17:36:44.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:36:44 smithi100 bash[76202]: cluster 2024-04-07T17:36:43.582352+0000 mon.a (mon.0) 3834 : cluster [DBG] osdmap e1972: 8 total, 8 up, 8 in 2024-04-07T17:36:45.634 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:36:45.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:36:45 smithi100 bash[74942]: cluster 2024-04-07T17:36:44.460431+0000 mgr.y (mgr.25175) 2441 : cluster [DBG] pgmap v3988: 105 pgs: 105 active+clean; 4.6 MiB data, 5.0 GiB used, 710 GiB / 715 GiB avail 2024-04-07T17:36:45.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:36:45 smithi100 bash[74942]: cluster 2024-04-07T17:36:44.608232+0000 mon.a (mon.0) 3835 : cluster [DBG] osdmap e1973: 8 total, 8 up, 8 in 2024-04-07T17:36:45.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:36:45 smithi100 bash[74942]: audit 2024-04-07T17:36:44.621236+0000 mon.c (mon.2) 99 : audit [INF] from='client.? 172.21.15.100:0/2383430443' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-44","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:36:45.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:36:45 smithi100 bash[74942]: audit 2024-04-07T17:36:44.623006+0000 mon.a (mon.0) 3836 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-44","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:36:45.955 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:36:45 smithi100 bash[76202]: cluster 2024-04-07T17:36:44.460431+0000 mgr.y (mgr.25175) 2441 : cluster [DBG] pgmap v3988: 105 pgs: 105 active+clean; 4.6 MiB data, 5.0 GiB used, 710 GiB / 715 GiB avail 2024-04-07T17:36:45.955 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:36:45 smithi100 bash[76202]: cluster 2024-04-07T17:36:44.608232+0000 mon.a (mon.0) 3835 : cluster [DBG] osdmap e1973: 8 total, 8 up, 8 in 2024-04-07T17:36:45.955 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:36:45 smithi100 bash[76202]: audit 2024-04-07T17:36:44.621236+0000 mon.c (mon.2) 99 : audit [INF] from='client.? 172.21.15.100:0/2383430443' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-44","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:36:45.955 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:36:45 smithi100 bash[76202]: audit 2024-04-07T17:36:44.623006+0000 mon.a (mon.0) 3836 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-44","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:36:46.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:36:45 smithi144 bash[66654]: cluster 2024-04-07T17:36:44.460431+0000 mgr.y (mgr.25175) 2441 : cluster [DBG] pgmap v3988: 105 pgs: 105 active+clean; 4.6 MiB data, 5.0 GiB used, 710 GiB / 715 GiB avail 2024-04-07T17:36:46.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:36:45 smithi144 bash[66654]: cluster 2024-04-07T17:36:44.608232+0000 mon.a (mon.0) 3835 : cluster [DBG] osdmap e1973: 8 total, 8 up, 8 in 2024-04-07T17:36:46.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:36:45 smithi144 bash[66654]: audit 2024-04-07T17:36:44.621236+0000 mon.c (mon.2) 99 : audit [INF] from='client.? 172.21.15.100:0/2383430443' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-44","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:36:46.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:36:45 smithi144 bash[66654]: audit 2024-04-07T17:36:44.623006+0000 mon.a (mon.0) 3836 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-44","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:36:46.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:36:46 smithi100 bash[74942]: audit 2024-04-07T17:36:45.604426+0000 mon.a (mon.0) 3837 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-44","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T17:36:46.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:36:46 smithi100 bash[74942]: cluster 2024-04-07T17:36:45.612654+0000 mon.a (mon.0) 3838 : cluster [DBG] osdmap e1974: 8 total, 8 up, 8 in 2024-04-07T17:36:46.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:36:46 smithi100 bash[76202]: audit 2024-04-07T17:36:45.604426+0000 mon.a (mon.0) 3837 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-44","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T17:36:46.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:36:46 smithi100 bash[76202]: cluster 2024-04-07T17:36:45.612654+0000 mon.a (mon.0) 3838 : cluster [DBG] osdmap e1974: 8 total, 8 up, 8 in 2024-04-07T17:36:47.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:36:46 smithi144 bash[66654]: audit 2024-04-07T17:36:45.604426+0000 mon.a (mon.0) 3837 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-44","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T17:36:47.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:36:46 smithi144 bash[66654]: cluster 2024-04-07T17:36:45.612654+0000 mon.a (mon.0) 3838 : cluster [DBG] osdmap e1974: 8 total, 8 up, 8 in 2024-04-07T17:36:47.725 INFO:tasks.workunit.client.0.smithi100.stdout: wrote [288813~67497,10212848~57610,11277250~64390,12336971~29437,12812782~35055,14001186~34231,15078382~74119,15429934~83131,19490669~19337,20705024~87228] 2024-04-07T17:36:47.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:36:47 smithi100 bash[74942]: cluster 2024-04-07T17:36:46.461282+0000 mgr.y (mgr.25175) 2442 : cluster [DBG] pgmap v3991: 137 pgs: 3 creating+peering, 29 unknown, 105 active+clean; 4.6 MiB data, 4.8 GiB used, 710 GiB / 715 GiB avail 2024-04-07T17:36:47.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:36:47 smithi100 bash[74942]: cluster 2024-04-07T17:36:46.639522+0000 mon.a (mon.0) 3839 : cluster [DBG] osdmap e1975: 8 total, 8 up, 8 in 2024-04-07T17:36:47.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:36:47 smithi100 bash[74942]: audit 2024-04-07T17:36:46.856934+0000 mon.a (mon.0) 3840 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:36:47.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:36:47 smithi100 bash[76202]: cluster 2024-04-07T17:36:46.461282+0000 mgr.y (mgr.25175) 2442 : cluster [DBG] pgmap v3991: 137 pgs: 3 creating+peering, 29 unknown, 105 active+clean; 4.6 MiB data, 4.8 GiB used, 710 GiB / 715 GiB avail 2024-04-07T17:36:47.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:36:47 smithi100 bash[76202]: cluster 2024-04-07T17:36:46.639522+0000 mon.a (mon.0) 3839 : cluster [DBG] osdmap e1975: 8 total, 8 up, 8 in 2024-04-07T17:36:47.955 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:36:47 smithi100 bash[76202]: audit 2024-04-07T17:36:46.856934+0000 mon.a (mon.0) 3840 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:36:48.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:36:47 smithi144 bash[66654]: cluster 2024-04-07T17:36:46.461282+0000 mgr.y (mgr.25175) 2442 : cluster [DBG] pgmap v3991: 137 pgs: 3 creating+peering, 29 unknown, 105 active+clean; 4.6 MiB data, 4.8 GiB used, 710 GiB / 715 GiB avail 2024-04-07T17:36:48.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:36:47 smithi144 bash[66654]: cluster 2024-04-07T17:36:46.639522+0000 mon.a (mon.0) 3839 : cluster [DBG] osdmap e1975: 8 total, 8 up, 8 in 2024-04-07T17:36:48.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:36:47 smithi144 bash[66654]: audit 2024-04-07T17:36:46.856934+0000 mon.a (mon.0) 3840 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:36:48.697 INFO:tasks.workunit.client.0.smithi100.stdout: wrote [5352906~779,6806284~17376,8394646~68638,12022717~27852,12376070~15692,12460492~27095,12863716~97086,13625959~90170,15563915~34238,19546100~43026] 2024-04-07T17:36:48.739 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:36:48 smithi100 bash[74942]: cluster 2024-04-07T17:36:47.641583+0000 mon.a (mon.0) 3841 : cluster [DBG] osdmap e1976: 8 total, 8 up, 8 in 2024-04-07T17:36:48.740 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:36:48 smithi100 bash[76202]: cluster 2024-04-07T17:36:47.641583+0000 mon.a (mon.0) 3841 : cluster [DBG] osdmap e1976: 8 total, 8 up, 8 in 2024-04-07T17:36:48.740 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:36:48 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:36:48] "GET /metrics HTTP/1.1" 200 36693 "" "Prometheus/2.43.0" 2024-04-07T17:36:48.741 INFO:tasks.workunit.client.0.smithi100.stdout: diff was [4194304~2629356,8394646~68638,12022717~27852,12366408~121179,12863716~97086,13625959~90170,15513065~85088,19546100~43026] 2024-04-07T17:36:48.741 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] DiffIterateTest/0.DiffIterate (5083 ms) 2024-04-07T17:36:48.741 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] DiffIterateTest/0.DiffIterateDeterministic 2024-04-07T17:36:48.741 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:36:49.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:36:48 smithi144 bash[66654]: cluster 2024-04-07T17:36:47.641583+0000 mon.a (mon.0) 3841 : cluster [DBG] osdmap e1976: 8 total, 8 up, 8 in 2024-04-07T17:36:49.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:36:49 smithi100 bash[74942]: cluster 2024-04-07T17:36:48.462182+0000 mgr.y (mgr.25175) 2443 : cluster [DBG] pgmap v3994: 137 pgs: 3 creating+peering, 9 unknown, 125 active+clean; 4.6 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail 2024-04-07T17:36:49.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:36:49 smithi100 bash[74942]: cluster 2024-04-07T17:36:48.666644+0000 mon.a (mon.0) 3842 : cluster [DBG] osdmap e1977: 8 total, 8 up, 8 in 2024-04-07T17:36:49.955 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:36:49 smithi100 bash[76202]: cluster 2024-04-07T17:36:48.462182+0000 mgr.y (mgr.25175) 2443 : cluster [DBG] pgmap v3994: 137 pgs: 3 creating+peering, 9 unknown, 125 active+clean; 4.6 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail 2024-04-07T17:36:49.955 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:36:49 smithi100 bash[76202]: cluster 2024-04-07T17:36:48.666644+0000 mon.a (mon.0) 3842 : cluster [DBG] osdmap e1977: 8 total, 8 up, 8 in 2024-04-07T17:36:50.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:36:49 smithi144 bash[66654]: cluster 2024-04-07T17:36:48.462182+0000 mgr.y (mgr.25175) 2443 : cluster [DBG] pgmap v3994: 137 pgs: 3 creating+peering, 9 unknown, 125 active+clean; 4.6 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail 2024-04-07T17:36:50.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:36:49 smithi144 bash[66654]: cluster 2024-04-07T17:36:48.666644+0000 mon.a (mon.0) 3842 : cluster [DBG] osdmap e1977: 8 total, 8 up, 8 in 2024-04-07T17:36:50.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:36:50 smithi100 bash[74942]: cluster 2024-04-07T17:36:49.667959+0000 mon.a (mon.0) 3843 : cluster [DBG] osdmap e1978: 8 total, 8 up, 8 in 2024-04-07T17:36:50.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:36:50 smithi100 bash[76202]: cluster 2024-04-07T17:36:49.667959+0000 mon.a (mon.0) 3843 : cluster [DBG] osdmap e1978: 8 total, 8 up, 8 in 2024-04-07T17:36:51.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:36:50 smithi144 bash[66654]: cluster 2024-04-07T17:36:49.667959+0000 mon.a (mon.0) 3843 : cluster [DBG] osdmap e1978: 8 total, 8 up, 8 in 2024-04-07T17:36:51.763 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] DiffIterateTest/0.DiffIterateDeterministic (3045 ms) 2024-04-07T17:36:51.764 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] DiffIterateTest/0.DiffIterateDeterministicPP 2024-04-07T17:36:51.764 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:36:51.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:36:51 smithi100 bash[74942]: cluster 2024-04-07T17:36:50.463315+0000 mgr.y (mgr.25175) 2444 : cluster [DBG] pgmap v3997: 137 pgs: 137 active+clean; 12 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 1.5 KiB/s rd, 110 KiB/s wr, 6 op/s 2024-04-07T17:36:51.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:36:51 smithi100 bash[74942]: cluster 2024-04-07T17:36:50.689903+0000 mon.a (mon.0) 3844 : cluster [DBG] osdmap e1979: 8 total, 8 up, 8 in 2024-04-07T17:36:51.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:36:51 smithi100 bash[76202]: cluster 2024-04-07T17:36:50.463315+0000 mgr.y (mgr.25175) 2444 : cluster [DBG] pgmap v3997: 137 pgs: 137 active+clean; 12 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 1.5 KiB/s rd, 110 KiB/s wr, 6 op/s 2024-04-07T17:36:51.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:36:51 smithi100 bash[76202]: cluster 2024-04-07T17:36:50.689903+0000 mon.a (mon.0) 3844 : cluster [DBG] osdmap e1979: 8 total, 8 up, 8 in 2024-04-07T17:36:52.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:36:51 smithi144 bash[66654]: cluster 2024-04-07T17:36:50.463315+0000 mgr.y (mgr.25175) 2444 : cluster [DBG] pgmap v3997: 137 pgs: 137 active+clean; 12 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 1.5 KiB/s rd, 110 KiB/s wr, 6 op/s 2024-04-07T17:36:52.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:36:51 smithi144 bash[66654]: cluster 2024-04-07T17:36:50.689903+0000 mon.a (mon.0) 3844 : cluster [DBG] osdmap e1979: 8 total, 8 up, 8 in 2024-04-07T17:36:52.615 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:36:52 smithi144 bash[95233]: ts=2024-04-07T17:36:52.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:36:53.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:36:52 smithi144 bash[66654]: cluster 2024-04-07T17:36:51.703925+0000 mon.a (mon.0) 3845 : cluster [DBG] osdmap e1980: 8 total, 8 up, 8 in 2024-04-07T17:36:53.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:36:52 smithi100 bash[74942]: cluster 2024-04-07T17:36:51.703925+0000 mon.a (mon.0) 3845 : cluster [DBG] osdmap e1980: 8 total, 8 up, 8 in 2024-04-07T17:36:53.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:36:52 smithi100 bash[76202]: cluster 2024-04-07T17:36:51.703925+0000 mon.a (mon.0) 3845 : cluster [DBG] osdmap e1980: 8 total, 8 up, 8 in 2024-04-07T17:36:54.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:36:53 smithi144 bash[66654]: cluster 2024-04-07T17:36:52.464721+0000 mgr.y (mgr.25175) 2445 : cluster [DBG] pgmap v4000: 137 pgs: 137 active+clean; 18 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 66 KiB/s rd, 252 KiB/s wr, 107 op/s 2024-04-07T17:36:54.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:36:53 smithi144 bash[66654]: cluster 2024-04-07T17:36:52.714431+0000 mon.a (mon.0) 3846 : cluster [DBG] osdmap e1981: 8 total, 8 up, 8 in 2024-04-07T17:36:54.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:36:53 smithi144 bash[66654]: cluster 2024-04-07T17:36:53.190408+0000 mon.a (mon.0) 3847 : cluster [DBG] osdmap e1982: 8 total, 8 up, 8 in 2024-04-07T17:36:54.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:36:53 smithi100 bash[74942]: cluster 2024-04-07T17:36:52.464721+0000 mgr.y (mgr.25175) 2445 : cluster [DBG] pgmap v4000: 137 pgs: 137 active+clean; 18 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 66 KiB/s rd, 252 KiB/s wr, 107 op/s 2024-04-07T17:36:54.205 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:36:53 smithi100 bash[74942]: cluster 2024-04-07T17:36:52.714431+0000 mon.a (mon.0) 3846 : cluster [DBG] osdmap e1981: 8 total, 8 up, 8 in 2024-04-07T17:36:54.205 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:36:53 smithi100 bash[74942]: cluster 2024-04-07T17:36:53.190408+0000 mon.a (mon.0) 3847 : cluster [DBG] osdmap e1982: 8 total, 8 up, 8 in 2024-04-07T17:36:54.205 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:36:53 smithi100 bash[76202]: cluster 2024-04-07T17:36:52.464721+0000 mgr.y (mgr.25175) 2445 : cluster [DBG] pgmap v4000: 137 pgs: 137 active+clean; 18 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 66 KiB/s rd, 252 KiB/s wr, 107 op/s 2024-04-07T17:36:54.205 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:36:53 smithi100 bash[76202]: cluster 2024-04-07T17:36:52.714431+0000 mon.a (mon.0) 3846 : cluster [DBG] osdmap e1981: 8 total, 8 up, 8 in 2024-04-07T17:36:54.205 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:36:53 smithi100 bash[76202]: cluster 2024-04-07T17:36:53.190408+0000 mon.a (mon.0) 3847 : cluster [DBG] osdmap e1982: 8 total, 8 up, 8 in 2024-04-07T17:36:54.258 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] DiffIterateTest/0.DiffIterateDeterministicPP (2494 ms) 2024-04-07T17:36:54.258 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] DiffIterateTest/0.DiffIterateDiscard 2024-04-07T17:36:54.258 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:36:55.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:36:55 smithi100 bash[74942]: cluster 2024-04-07T17:36:54.193494+0000 mon.a (mon.0) 3848 : cluster [DBG] osdmap e1983: 8 total, 8 up, 8 in 2024-04-07T17:36:55.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:36:55 smithi100 bash[74942]: cluster 2024-04-07T17:36:54.465496+0000 mgr.y (mgr.25175) 2446 : cluster [DBG] pgmap v4004: 137 pgs: 137 active+clean; 18 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 71 KiB/s rd, 152 KiB/s wr, 114 op/s 2024-04-07T17:36:55.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:36:55 smithi100 bash[76202]: cluster 2024-04-07T17:36:54.193494+0000 mon.a (mon.0) 3848 : cluster [DBG] osdmap e1983: 8 total, 8 up, 8 in 2024-04-07T17:36:55.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:36:55 smithi100 bash[76202]: cluster 2024-04-07T17:36:54.465496+0000 mgr.y (mgr.25175) 2446 : cluster [DBG] pgmap v4004: 137 pgs: 137 active+clean; 18 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 71 KiB/s rd, 152 KiB/s wr, 114 op/s 2024-04-07T17:36:55.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:36:55 smithi144 bash[66654]: cluster 2024-04-07T17:36:54.193494+0000 mon.a (mon.0) 3848 : cluster [DBG] osdmap e1983: 8 total, 8 up, 8 in 2024-04-07T17:36:55.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:36:55 smithi144 bash[66654]: cluster 2024-04-07T17:36:54.465496+0000 mgr.y (mgr.25175) 2446 : cluster [DBG] pgmap v4004: 137 pgs: 137 active+clean; 18 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 71 KiB/s rd, 152 KiB/s wr, 114 op/s 2024-04-07T17:36:56.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:36:56 smithi144 bash[66654]: cluster 2024-04-07T17:36:55.204860+0000 mon.a (mon.0) 3849 : cluster [DBG] osdmap e1984: 8 total, 8 up, 8 in 2024-04-07T17:36:56.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:36:56 smithi100 bash[74942]: cluster 2024-04-07T17:36:55.204860+0000 mon.a (mon.0) 3849 : cluster [DBG] osdmap e1984: 8 total, 8 up, 8 in 2024-04-07T17:36:56.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:36:56 smithi100 bash[76202]: cluster 2024-04-07T17:36:55.204860+0000 mon.a (mon.0) 3849 : cluster [DBG] osdmap e1984: 8 total, 8 up, 8 in 2024-04-07T17:36:57.291 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] DiffIterateTest/0.DiffIterateDiscard (3033 ms) 2024-04-07T17:36:57.291 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] DiffIterateTest/0.DiffIterateTruncate 2024-04-07T17:36:57.291 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:36:57.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:36:57 smithi144 bash[66654]: cluster 2024-04-07T17:36:56.205171+0000 mon.a (mon.0) 3850 : cluster [DBG] osdmap e1985: 8 total, 8 up, 8 in 2024-04-07T17:36:57.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:36:57 smithi144 bash[66654]: cluster 2024-04-07T17:36:56.466231+0000 mgr.y (mgr.25175) 2447 : cluster [DBG] pgmap v4007: 137 pgs: 137 active+clean; 18 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 2.9 KiB/s rd, 1.3 KiB/s wr, 11 op/s 2024-04-07T17:36:57.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:36:57 smithi100 bash[74942]: cluster 2024-04-07T17:36:56.205171+0000 mon.a (mon.0) 3850 : cluster [DBG] osdmap e1985: 8 total, 8 up, 8 in 2024-04-07T17:36:57.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:36:57 smithi100 bash[74942]: cluster 2024-04-07T17:36:56.466231+0000 mgr.y (mgr.25175) 2447 : cluster [DBG] pgmap v4007: 137 pgs: 137 active+clean; 18 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 2.9 KiB/s rd, 1.3 KiB/s wr, 11 op/s 2024-04-07T17:36:57.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:36:57 smithi100 bash[76202]: cluster 2024-04-07T17:36:56.205171+0000 mon.a (mon.0) 3850 : cluster [DBG] osdmap e1985: 8 total, 8 up, 8 in 2024-04-07T17:36:57.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:36:57 smithi100 bash[76202]: cluster 2024-04-07T17:36:56.466231+0000 mgr.y (mgr.25175) 2447 : cluster [DBG] pgmap v4007: 137 pgs: 137 active+clean; 18 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 2.9 KiB/s rd, 1.3 KiB/s wr, 11 op/s 2024-04-07T17:36:58.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:36:58 smithi144 bash[66654]: cluster 2024-04-07T17:36:57.226131+0000 mon.a (mon.0) 3851 : cluster [DBG] osdmap e1986: 8 total, 8 up, 8 in 2024-04-07T17:36:58.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:36:58 smithi100 bash[74942]: cluster 2024-04-07T17:36:57.226131+0000 mon.a (mon.0) 3851 : cluster [DBG] osdmap e1986: 8 total, 8 up, 8 in 2024-04-07T17:36:58.704 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:36:58 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:36:58] "GET /metrics HTTP/1.1" 200 36693 "" "Prometheus/2.43.0" 2024-04-07T17:36:58.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:36:58 smithi100 bash[76202]: cluster 2024-04-07T17:36:57.226131+0000 mon.a (mon.0) 3851 : cluster [DBG] osdmap e1986: 8 total, 8 up, 8 in 2024-04-07T17:36:59.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:36:59 smithi144 bash[66654]: cluster 2024-04-07T17:36:58.239549+0000 mon.a (mon.0) 3852 : cluster [DBG] osdmap e1987: 8 total, 8 up, 8 in 2024-04-07T17:36:59.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:36:59 smithi144 bash[66654]: cluster 2024-04-07T17:36:58.466976+0000 mgr.y (mgr.25175) 2448 : cluster [DBG] pgmap v4010: 137 pgs: 137 active+clean; 18 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 62 KiB/s rd, 12 KiB/s wr, 97 op/s 2024-04-07T17:36:59.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:36:59 smithi100 bash[74942]: cluster 2024-04-07T17:36:58.239549+0000 mon.a (mon.0) 3852 : cluster [DBG] osdmap e1987: 8 total, 8 up, 8 in 2024-04-07T17:36:59.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:36:59 smithi100 bash[74942]: cluster 2024-04-07T17:36:58.466976+0000 mgr.y (mgr.25175) 2448 : cluster [DBG] pgmap v4010: 137 pgs: 137 active+clean; 18 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 62 KiB/s rd, 12 KiB/s wr, 97 op/s 2024-04-07T17:36:59.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:36:59 smithi100 bash[76202]: cluster 2024-04-07T17:36:58.239549+0000 mon.a (mon.0) 3852 : cluster [DBG] osdmap e1987: 8 total, 8 up, 8 in 2024-04-07T17:36:59.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:36:59 smithi100 bash[76202]: cluster 2024-04-07T17:36:58.466976+0000 mgr.y (mgr.25175) 2448 : cluster [DBG] pgmap v4010: 137 pgs: 137 active+clean; 18 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 62 KiB/s rd, 12 KiB/s wr, 97 op/s 2024-04-07T17:37:00.342 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] DiffIterateTest/0.DiffIterateTruncate (3051 ms) 2024-04-07T17:37:00.342 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] DiffIterateTest/0.DiffIterateWriteAndTruncate 2024-04-07T17:37:00.342 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:37:00.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:37:00 smithi144 bash[66654]: cluster 2024-04-07T17:36:59.249241+0000 mon.a (mon.0) 3853 : cluster [DBG] osdmap e1988: 8 total, 8 up, 8 in 2024-04-07T17:37:00.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:37:00 smithi100 bash[74942]: cluster 2024-04-07T17:36:59.249241+0000 mon.a (mon.0) 3853 : cluster [DBG] osdmap e1988: 8 total, 8 up, 8 in 2024-04-07T17:37:00.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:37:00 smithi100 bash[76202]: cluster 2024-04-07T17:36:59.249241+0000 mon.a (mon.0) 3853 : cluster [DBG] osdmap e1988: 8 total, 8 up, 8 in 2024-04-07T17:37:01.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:37:01 smithi144 bash[66654]: cluster 2024-04-07T17:37:00.249521+0000 mon.a (mon.0) 3854 : cluster [DBG] osdmap e1989: 8 total, 8 up, 8 in 2024-04-07T17:37:01.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:37:01 smithi144 bash[66654]: cluster 2024-04-07T17:37:00.467851+0000 mgr.y (mgr.25175) 2449 : cluster [DBG] pgmap v4013: 137 pgs: 137 active+clean; 19 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 70 KiB/s rd, 270 KiB/s wr, 107 op/s 2024-04-07T17:37:01.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:37:01 smithi100 bash[74942]: cluster 2024-04-07T17:37:00.249521+0000 mon.a (mon.0) 3854 : cluster [DBG] osdmap e1989: 8 total, 8 up, 8 in 2024-04-07T17:37:01.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:37:01 smithi100 bash[74942]: cluster 2024-04-07T17:37:00.467851+0000 mgr.y (mgr.25175) 2449 : cluster [DBG] pgmap v4013: 137 pgs: 137 active+clean; 19 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 70 KiB/s rd, 270 KiB/s wr, 107 op/s 2024-04-07T17:37:01.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:37:01 smithi100 bash[76202]: cluster 2024-04-07T17:37:00.249521+0000 mon.a (mon.0) 3854 : cluster [DBG] osdmap e1989: 8 total, 8 up, 8 in 2024-04-07T17:37:01.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:37:01 smithi100 bash[76202]: cluster 2024-04-07T17:37:00.467851+0000 mgr.y (mgr.25175) 2449 : cluster [DBG] pgmap v4013: 137 pgs: 137 active+clean; 19 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 70 KiB/s rd, 270 KiB/s wr, 107 op/s 2024-04-07T17:37:02.615 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:37:02 smithi144 bash[95233]: ts=2024-04-07T17:37:02.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:37:02.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:37:02 smithi144 bash[66654]: cluster 2024-04-07T17:37:01.274281+0000 mon.a (mon.0) 3855 : cluster [DBG] osdmap e1990: 8 total, 8 up, 8 in 2024-04-07T17:37:02.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:37:02 smithi144 bash[66654]: audit 2024-04-07T17:37:01.856924+0000 mon.a (mon.0) 3856 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:37:02.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:37:02 smithi100 bash[74942]: cluster 2024-04-07T17:37:01.274281+0000 mon.a (mon.0) 3855 : cluster [DBG] osdmap e1990: 8 total, 8 up, 8 in 2024-04-07T17:37:02.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:37:02 smithi100 bash[74942]: audit 2024-04-07T17:37:01.856924+0000 mon.a (mon.0) 3856 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:37:02.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:37:02 smithi100 bash[76202]: cluster 2024-04-07T17:37:01.274281+0000 mon.a (mon.0) 3855 : cluster [DBG] osdmap e1990: 8 total, 8 up, 8 in 2024-04-07T17:37:02.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:37:02 smithi100 bash[76202]: audit 2024-04-07T17:37:01.856924+0000 mon.a (mon.0) 3856 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:37:03.387 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] DiffIterateTest/0.DiffIterateWriteAndTruncate (3045 ms) 2024-04-07T17:37:03.387 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] DiffIterateTest/0.DiffIterateStress 2024-04-07T17:37:03.387 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:37:03.482 INFO:tasks.workunit.client.0.smithi100.stdout: i=0 exists [23638476~2276597,98502672~3072494,125965573~4384260,182628739~2536501,197989623~74176,207606288~4131451,321502401~5004346,339616250~1118330,366057679~1764425] wrote [23638476~2276597,98502672~3072494,125965573~4384260,182628739~2536501,197989623~74176,207606288~4131451,321502401~5004346,339616250~1118330,366057679~1764425] 2024-04-07T17:37:03.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:37:03 smithi144 bash[66654]: cluster 2024-04-07T17:37:02.285478+0000 mon.a (mon.0) 3857 : cluster [DBG] osdmap e1991: 8 total, 8 up, 8 in 2024-04-07T17:37:03.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:37:03 smithi144 bash[66654]: cluster 2024-04-07T17:37:02.468736+0000 mgr.y (mgr.25175) 2450 : cluster [DBG] pgmap v4016: 137 pgs: 137 active+clean; 19 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 47 KiB/s rd, 262 KiB/s wr, 72 op/s 2024-04-07T17:37:03.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:37:03 smithi100 bash[74942]: cluster 2024-04-07T17:37:02.285478+0000 mon.a (mon.0) 3857 : cluster [DBG] osdmap e1991: 8 total, 8 up, 8 in 2024-04-07T17:37:03.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:37:03 smithi100 bash[74942]: cluster 2024-04-07T17:37:02.468736+0000 mgr.y (mgr.25175) 2450 : cluster [DBG] pgmap v4016: 137 pgs: 137 active+clean; 19 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 47 KiB/s rd, 262 KiB/s wr, 72 op/s 2024-04-07T17:37:03.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:37:03 smithi100 bash[76202]: cluster 2024-04-07T17:37:02.285478+0000 mon.a (mon.0) 3857 : cluster [DBG] osdmap e1991: 8 total, 8 up, 8 in 2024-04-07T17:37:03.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:37:03 smithi100 bash[76202]: cluster 2024-04-07T17:37:02.468736+0000 mgr.y (mgr.25175) 2450 : cluster [DBG] pgmap v4016: 137 pgs: 137 active+clean; 19 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 47 KiB/s rd, 262 KiB/s wr, 72 op/s 2024-04-07T17:37:04.411 INFO:tasks.workunit.client.0.smithi100.stdout: i=1 exists [23638476~2276597,50852715~5853280,67034044~1727684,85466334~5069656,98502672~3072494,125965573~4384260,182628739~2536501,194102759~3585735,197989623~74176,207606288~4131451,253547268~2401828,275605911~4974570,321502401~5004346,339616250~1118330,350406459~6437804,366057679~1764425,381619449~3142051,392232408~7812666] wrote [50852715~5853280,67034044~1727684,85466334~5069656,194102759~3585735,253547268~2401828,275605911~4974570,350406459~6437804,381619449~3142051,392232408~7812666] 2024-04-07T17:37:04.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:37:04 smithi144 bash[66654]: cluster 2024-04-07T17:37:03.298233+0000 mon.a (mon.0) 3858 : cluster [DBG] osdmap e1992: 8 total, 8 up, 8 in 2024-04-07T17:37:04.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:37:04 smithi100 bash[74942]: cluster 2024-04-07T17:37:03.298233+0000 mon.a (mon.0) 3858 : cluster [DBG] osdmap e1992: 8 total, 8 up, 8 in 2024-04-07T17:37:04.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:37:04 smithi100 bash[76202]: cluster 2024-04-07T17:37:03.298233+0000 mon.a (mon.0) 3858 : cluster [DBG] osdmap e1992: 8 total, 8 up, 8 in 2024-04-07T17:37:05.476 INFO:tasks.workunit.client.0.smithi100.stdout: i=2 exists [23638476~4906123,50852715~5853280,67034044~1727684,85466334~5069656,98502672~3072494,108048350~5929155,125965573~4384260,146434166~11449972,180237750~4927490,186904555~1491543,194102759~7364680,207606288~14253952,253547268~2401828,275605911~4974570,321502401~5004346,339616250~1118330,350406459~6437804,366057679~1764425,381619449~3142051,392232408~7812666] wrote [25445275~3099324,108048350~5929155,146434166~11449972,180237750~3536884,186904555~1491543,194866296~6601143,210875736~10984504] 2024-04-07T17:37:05.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:37:05 smithi144 bash[66654]: cluster 2024-04-07T17:37:04.297569+0000 mon.a (mon.0) 3859 : cluster [DBG] osdmap e1993: 8 total, 8 up, 8 in 2024-04-07T17:37:05.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:37:05 smithi144 bash[66654]: cluster 2024-04-07T17:37:04.469512+0000 mgr.y (mgr.25175) 2451 : cluster [DBG] pgmap v4019: 137 pgs: 137 active+clean; 30 MiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 71 KiB/s rd, 1.3 MiB/s wr, 101 op/s 2024-04-07T17:37:05.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:37:05 smithi100 bash[74942]: cluster 2024-04-07T17:37:04.297569+0000 mon.a (mon.0) 3859 : cluster [DBG] osdmap e1993: 8 total, 8 up, 8 in 2024-04-07T17:37:05.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:37:05 smithi100 bash[74942]: cluster 2024-04-07T17:37:04.469512+0000 mgr.y (mgr.25175) 2451 : cluster [DBG] pgmap v4019: 137 pgs: 137 active+clean; 30 MiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 71 KiB/s rd, 1.3 MiB/s wr, 101 op/s 2024-04-07T17:37:05.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:37:05 smithi100 bash[76202]: cluster 2024-04-07T17:37:04.297569+0000 mon.a (mon.0) 3859 : cluster [DBG] osdmap e1993: 8 total, 8 up, 8 in 2024-04-07T17:37:05.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:37:05 smithi100 bash[76202]: cluster 2024-04-07T17:37:04.469512+0000 mgr.y (mgr.25175) 2451 : cluster [DBG] pgmap v4019: 137 pgs: 137 active+clean; 30 MiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 71 KiB/s rd, 1.3 MiB/s wr, 101 op/s 2024-04-07T17:37:06.449 INFO:tasks.workunit.client.0.smithi100.stdout: i=3 exists [20375589~8169010,50852715~5853280,67034044~1727684,81710240~1015114,85466334~5069656,98502672~3072494,108048350~5929155,125965573~4384260,143151737~1037788,146434166~12400590,165813830~1137640,180237750~4927490,185860175~6968289,194102759~7364680,203328670~18531570,243012066~4719194,253547268~2401828,275605911~7066628,321502401~5004346,339616250~1118330,350406459~6437804,366057679~1764425,381619449~3142051,392232408~7812666] wrote [20375589~3479656,81710240~1015114,143151737~1037788,156912913~1921843,165813830~1137640,185860175~6968289,203328670~4346303,216364348~218501,243012066~4719194,276663656~6008883] 2024-04-07T17:37:06.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:37:06 smithi144 bash[66654]: cluster 2024-04-07T17:37:05.328144+0000 mon.a (mon.0) 3860 : cluster [DBG] osdmap e1994: 8 total, 8 up, 8 in 2024-04-07T17:37:06.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:37:06 smithi100 bash[74942]: cluster 2024-04-07T17:37:05.328144+0000 mon.a (mon.0) 3860 : cluster [DBG] osdmap e1994: 8 total, 8 up, 8 in 2024-04-07T17:37:06.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:37:06 smithi100 bash[76202]: cluster 2024-04-07T17:37:05.328144+0000 mon.a (mon.0) 3860 : cluster [DBG] osdmap e1994: 8 total, 8 up, 8 in 2024-04-07T17:37:07.400 INFO:tasks.workunit.client.0.smithi100.stdout: i=4 exists [20375589~8169010,29778841~737427,50852715~5853280,67034044~1727684,81710240~1015114,85466334~5069656,98502672~3072494,107619787~6357718,125965573~4384260,143151737~15683019,165813830~1137640,173876238~1467374,180237750~4927490,185860175~6968289,194102759~7364680,203328670~18531570,243012066~4719194,253547268~4410812,275605911~9036918,309308954~1620369,321502401~5004346,339616250~1118330,350406459~6437804,360049192~84791,366057679~1764425,381619449~3142051,388412399~2413139,392232408~7812666] wrote [29778841~737427,107619787~2326498,143194295~6807063,154799337~811894,173876238~1467374,253764583~4193497,279625672~5017157,309308954~1620369,360049192~84791,388412399~2413139] 2024-04-07T17:37:07.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:37:07 smithi144 bash[66654]: cluster 2024-04-07T17:37:06.334330+0000 mon.a (mon.0) 3861 : cluster [DBG] osdmap e1995: 8 total, 8 up, 8 in 2024-04-07T17:37:07.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:37:07 smithi144 bash[66654]: cluster 2024-04-07T17:37:06.470237+0000 mgr.y (mgr.25175) 2452 : cluster [DBG] pgmap v4022: 137 pgs: 137 active+clean; 45 MiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 86 KiB/s rd, 3.7 MiB/s wr, 145 op/s 2024-04-07T17:37:07.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:37:07 smithi100 bash[74942]: cluster 2024-04-07T17:37:06.334330+0000 mon.a (mon.0) 3861 : cluster [DBG] osdmap e1995: 8 total, 8 up, 8 in 2024-04-07T17:37:07.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:37:07 smithi100 bash[74942]: cluster 2024-04-07T17:37:06.470237+0000 mgr.y (mgr.25175) 2452 : cluster [DBG] pgmap v4022: 137 pgs: 137 active+clean; 45 MiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 86 KiB/s rd, 3.7 MiB/s wr, 145 op/s 2024-04-07T17:37:07.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:37:07 smithi100 bash[76202]: cluster 2024-04-07T17:37:06.334330+0000 mon.a (mon.0) 3861 : cluster [DBG] osdmap e1995: 8 total, 8 up, 8 in 2024-04-07T17:37:07.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:37:07 smithi100 bash[76202]: cluster 2024-04-07T17:37:06.470237+0000 mgr.y (mgr.25175) 2452 : cluster [DBG] pgmap v4022: 137 pgs: 137 active+clean; 45 MiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 86 KiB/s rd, 3.7 MiB/s wr, 145 op/s 2024-04-07T17:37:08.485 INFO:tasks.workunit.client.0.smithi100.stdout: i=5 exists [20375589~8169010,29778841~737427,50852715~5853280,67034044~1727684,81710240~1015114,85466334~5069656,98502672~3072494,107619787~6357718,125965573~4384260,140416111~18418645,160337176~6614294,173876238~1467374,180237750~4927490,185860175~6968289,194102759~7364680,203328670~18531570,243012066~7805483,253547268~4410812,275605911~9036918,301491055~2008066,309308954~1620369,321502401~5004346,326964255~3039511,339616250~1118330,350406459~6437804,360049192~84791,366057679~1764425,374607324~4374722,381619449~3142051,388412399~2413139,392232408~7812666] wrote [140416111~6992252,148109105~141733,160337176~6458877,216361490~3081245,244611976~6205573,301491055~2008066,326964255~3039511,350891716~1451525,374607324~4374722] 2024-04-07T17:37:08.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:37:08 smithi144 bash[66654]: cluster 2024-04-07T17:37:07.345680+0000 mon.a (mon.0) 3862 : cluster [DBG] osdmap e1996: 8 total, 8 up, 8 in 2024-04-07T17:37:08.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:37:08 smithi100 bash[74942]: cluster 2024-04-07T17:37:07.345680+0000 mon.a (mon.0) 3862 : cluster [DBG] osdmap e1996: 8 total, 8 up, 8 in 2024-04-07T17:37:08.704 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:37:08 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:37:08] "GET /metrics HTTP/1.1" 200 36741 "" "Prometheus/2.43.0" 2024-04-07T17:37:08.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:37:08 smithi100 bash[76202]: cluster 2024-04-07T17:37:07.345680+0000 mon.a (mon.0) 3862 : cluster [DBG] osdmap e1996: 8 total, 8 up, 8 in 2024-04-07T17:37:09.491 INFO:tasks.workunit.client.0.smithi100.stdout: i=6 exists [19480873~9063726,29778841~737427,50852715~5853280,67034044~1727684,81710240~1015114,85466334~5069656,98502672~3072494,107619787~6357718,125965573~4384260,140416111~18418645,160337176~7153146,173876238~1467374,180237750~4927490,185860175~6968289,194102759~7364680,203328670~18531570,243012066~7906095,253547268~10579148,273160535~11482294,292398584~2871045,301491055~2008066,309308954~1620369,321502401~5004346,326964255~3039511,337086692~6947322,350406459~6437804,360049192~84791,366057679~1764425,374607324~4374722,381619449~3142051,388412399~2413139,392232408~7812666] wrote [19480873~7079576,166133159~1357163,182034135~1518423,196727604~2108791,210325964~283338,248419159~2499002,256332667~7793749,273160535~7546831,292398584~2871045,337086692~6947322] 2024-04-07T17:37:09.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:37:09 smithi144 bash[66654]: cluster 2024-04-07T17:37:08.349441+0000 mon.a (mon.0) 3863 : cluster [DBG] osdmap e1997: 8 total, 8 up, 8 in 2024-04-07T17:37:09.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:37:09 smithi144 bash[66654]: cluster 2024-04-07T17:37:08.471036+0000 mgr.y (mgr.25175) 2453 : cluster [DBG] pgmap v4025: 137 pgs: 137 active+clean; 156 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 83 KiB/s rd, 22 MiB/s wr, 153 op/s 2024-04-07T17:37:09.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:37:09 smithi100 bash[74942]: cluster 2024-04-07T17:37:08.349441+0000 mon.a (mon.0) 3863 : cluster [DBG] osdmap e1997: 8 total, 8 up, 8 in 2024-04-07T17:37:09.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:37:09 smithi100 bash[74942]: cluster 2024-04-07T17:37:08.471036+0000 mgr.y (mgr.25175) 2453 : cluster [DBG] pgmap v4025: 137 pgs: 137 active+clean; 156 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 83 KiB/s rd, 22 MiB/s wr, 153 op/s 2024-04-07T17:37:09.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:37:09 smithi100 bash[76202]: cluster 2024-04-07T17:37:08.349441+0000 mon.a (mon.0) 3863 : cluster [DBG] osdmap e1997: 8 total, 8 up, 8 in 2024-04-07T17:37:09.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:37:09 smithi100 bash[76202]: cluster 2024-04-07T17:37:08.471036+0000 mgr.y (mgr.25175) 2453 : cluster [DBG] pgmap v4025: 137 pgs: 137 active+clean; 156 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 83 KiB/s rd, 22 MiB/s wr, 153 op/s 2024-04-07T17:37:10.492 INFO:tasks.workunit.client.0.smithi100.stdout: i=7 exists [19480873~9063726,29778841~737427,49566880~7139115,66861224~7774919,81710240~1015114,85466334~5069656,98502672~3072494,107619787~6357718,125965573~4384260,140416111~18418645,160337176~7153146,173876238~1467374,180237750~12590714,194102759~7364680,202420372~19439868,240886088~1607107,243012066~7906095,253547268~10579148,273160535~11482294,292398584~2871045,301491055~2008066,309308954~1620369,321502401~5004346,326964255~3039511,337086692~6947322,345811290~1311192,350406459~6437804,360049192~84791,366057679~5611097,374580519~4455211,381619449~3142051,388412399~2413139,392232408~7812666] wrote [49566880~3318481,66861224~7774919,141097128~3081966,183213271~4693294,202420372~1435645,210854655~6060499,240886088~1607107,345811290~1311192,367170792~4497984,374580519~4455211] 2024-04-07T17:37:10.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:37:10 smithi100 bash[74942]: cluster 2024-04-07T17:37:09.375895+0000 mon.a (mon.0) 3864 : cluster [DBG] osdmap e1998: 8 total, 8 up, 8 in 2024-04-07T17:37:10.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:37:10 smithi100 bash[76202]: cluster 2024-04-07T17:37:09.375895+0000 mon.a (mon.0) 3864 : cluster [DBG] osdmap e1998: 8 total, 8 up, 8 in 2024-04-07T17:37:10.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:37:10 smithi144 bash[66654]: cluster 2024-04-07T17:37:09.375895+0000 mon.a (mon.0) 3864 : cluster [DBG] osdmap e1998: 8 total, 8 up, 8 in 2024-04-07T17:37:11.503 INFO:tasks.workunit.client.0.smithi100.stdout: i=8 exists [19480873~9063726,29778841~737427,49566880~7139115,66861224~7774919,81710240~1015114,85466334~5069656,92999623~8575543,107619787~6357718,125965573~8012021,140416111~18418645,160337176~7153146,173876238~1467374,180237750~12590714,194102759~7364680,202420372~19439868,235153608~5013034,240886088~1607107,243012066~7906095,253228895~10897521,273160535~11482294,292398584~2871045,300063155~4934365,309308954~1620369,321502401~5004346,326964255~3039511,337086692~6947322,345811290~1311192,350406459~6437804,360049192~84791,366057679~5611097,374580519~4455211,381619449~3142051,388412399~2413139,392232408~7812666] wrote [27909341~266890,92999623~7613429,128030792~5946802,149233764~3199220,154895619~3096498,235153608~5013034,253228895~825520,300063155~4934365,378193032~444908] 2024-04-07T17:37:11.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:37:11 smithi100 bash[74942]: cluster 2024-04-07T17:37:10.377868+0000 mon.a (mon.0) 3865 : cluster [DBG] osdmap e1999: 8 total, 8 up, 8 in 2024-04-07T17:37:11.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:37:11 smithi100 bash[74942]: cluster 2024-04-07T17:37:10.472397+0000 mgr.y (mgr.25175) 2454 : cluster [DBG] pgmap v4028: 137 pgs: 137 active+clean; 251 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 141 KiB/s rd, 39 MiB/s wr, 240 op/s 2024-04-07T17:37:11.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:37:11 smithi100 bash[74942]: audit 2024-04-07T17:37:10.809581+0000 mon.a (mon.0) 3866 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:37:11.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:37:11 smithi100 bash[74942]: audit 2024-04-07T17:37:11.336464+0000 mon.a (mon.0) 3867 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:37:11.705 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:37:11 smithi100 bash[74942]: audit 2024-04-07T17:37:11.342555+0000 mon.a (mon.0) 3868 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:37:11.705 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:37:11 smithi100 bash[76202]: cluster 2024-04-07T17:37:10.377868+0000 mon.a (mon.0) 3865 : cluster [DBG] osdmap e1999: 8 total, 8 up, 8 in 2024-04-07T17:37:11.705 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:37:11 smithi100 bash[76202]: cluster 2024-04-07T17:37:10.472397+0000 mgr.y (mgr.25175) 2454 : cluster [DBG] pgmap v4028: 137 pgs: 137 active+clean; 251 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 141 KiB/s rd, 39 MiB/s wr, 240 op/s 2024-04-07T17:37:11.705 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:37:11 smithi100 bash[76202]: audit 2024-04-07T17:37:10.809581+0000 mon.a (mon.0) 3866 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:37:11.705 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:37:11 smithi100 bash[76202]: audit 2024-04-07T17:37:11.336464+0000 mon.a (mon.0) 3867 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:37:11.705 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:37:11 smithi100 bash[76202]: audit 2024-04-07T17:37:11.342555+0000 mon.a (mon.0) 3868 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:37:11.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:37:11 smithi144 bash[66654]: cluster 2024-04-07T17:37:10.377868+0000 mon.a (mon.0) 3865 : cluster [DBG] osdmap e1999: 8 total, 8 up, 8 in 2024-04-07T17:37:11.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:37:11 smithi144 bash[66654]: cluster 2024-04-07T17:37:10.472397+0000 mgr.y (mgr.25175) 2454 : cluster [DBG] pgmap v4028: 137 pgs: 137 active+clean; 251 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 141 KiB/s rd, 39 MiB/s wr, 240 op/s 2024-04-07T17:37:11.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:37:11 smithi144 bash[66654]: audit 2024-04-07T17:37:10.809581+0000 mon.a (mon.0) 3866 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:37:11.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:37:11 smithi144 bash[66654]: audit 2024-04-07T17:37:11.336464+0000 mon.a (mon.0) 3867 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:37:11.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:37:11 smithi144 bash[66654]: audit 2024-04-07T17:37:11.342555+0000 mon.a (mon.0) 3868 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:37:12.556 INFO:tasks.workunit.client.0.smithi100.stdout: i=9 exists [7275769~186849,19095670~9448929,29778841~737427,31769017~7139161,49566880~7139115,66861224~7774919,81710240~1015114,85466334~5069656,92999623~8575543,107619787~6357718,116025937~3770559,125965573~8012021,140416111~18418645,160337176~7153146,173876238~1467374,180237750~12590714,194102759~7364680,202420372~19439868,230813446~9353196,240886088~1607107,243012066~21114350,273160535~13315830,292398584~2871045,300063155~4934365,309308954~1620369,321502401~5004346,326964255~3039511,337086692~6947322,345811290~1311192,350406459~6437804,360049192~84791,366057679~5611097,374580519~4455211,381619449~3142051,388412399~2413139,392232408~7812666] wrote [7275769~186849,19095670~6652777,31769017~7139161,95470042~6051657,116025937~3770559,182747004~7066023,230813446~6715104,250447165~7466990,281234113~5242252] 2024-04-07T17:37:12.615 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:37:12 smithi144 bash[95233]: ts=2024-04-07T17:37:12.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:37:12.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:37:12 smithi144 bash[66654]: cluster 2024-04-07T17:37:11.396247+0000 mon.a (mon.0) 3869 : cluster [DBG] osdmap e2000: 8 total, 8 up, 8 in 2024-04-07T17:37:12.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:37:12 smithi144 bash[66654]: audit 2024-04-07T17:37:12.074242+0000 mon.a (mon.0) 3870 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:37:12.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:37:12 smithi144 bash[66654]: audit 2024-04-07T17:37:12.076166+0000 mon.a (mon.0) 3871 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:37:12.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:37:12 smithi144 bash[66654]: audit 2024-04-07T17:37:12.087014+0000 mon.a (mon.0) 3872 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:37:12.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:37:12 smithi100 bash[74942]: cluster 2024-04-07T17:37:11.396247+0000 mon.a (mon.0) 3869 : cluster [DBG] osdmap e2000: 8 total, 8 up, 8 in 2024-04-07T17:37:12.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:37:12 smithi100 bash[74942]: audit 2024-04-07T17:37:12.074242+0000 mon.a (mon.0) 3870 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:37:12.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:37:12 smithi100 bash[74942]: audit 2024-04-07T17:37:12.076166+0000 mon.a (mon.0) 3871 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:37:12.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:37:12 smithi100 bash[74942]: audit 2024-04-07T17:37:12.087014+0000 mon.a (mon.0) 3872 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:37:12.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:37:12 smithi100 bash[76202]: cluster 2024-04-07T17:37:11.396247+0000 mon.a (mon.0) 3869 : cluster [DBG] osdmap e2000: 8 total, 8 up, 8 in 2024-04-07T17:37:12.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:37:12 smithi100 bash[76202]: audit 2024-04-07T17:37:12.074242+0000 mon.a (mon.0) 3870 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:37:12.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:37:12 smithi100 bash[76202]: audit 2024-04-07T17:37:12.076166+0000 mon.a (mon.0) 3871 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:37:12.705 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:37:12 smithi100 bash[76202]: audit 2024-04-07T17:37:12.087014+0000 mon.a (mon.0) 3872 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:37:13.520 INFO:tasks.workunit.client.0.smithi100.stdout: i=10 exists [2974197~5131400,19095670~9448929,29778841~737427,31769017~7139161,49566880~7139115,66861224~7774919,81710240~1015114,85466334~5069656,92999623~8575543,106500403~7477102,116025937~3770559,121501610~12475984,140416111~18418645,160337176~7153146,173876238~1467374,180237750~12590714,194102759~7364680,202420372~19439868,230813446~33312970,273160535~13315830,287123044~8146585,300063155~4934365,309308954~1620369,321502401~5004346,326964255~3039511,337086692~6947322,345811290~1311192,350406459~6437804,360049192~84791,366057679~7704401,374580519~4455211,381619449~3142051,388412399~2413139,392232408~7812666] wrote [2974197~5131400,27431447~878329,106500403~5285199,118278253~711581,121501610~8078304,186734432~153638,239191876~4760106,287123044~5448918,323691380~1672817,366341667~7420413] 2024-04-07T17:37:13.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:37:13 smithi100 bash[74942]: cluster 2024-04-07T17:37:12.409999+0000 mon.a (mon.0) 3873 : cluster [DBG] osdmap e2001: 8 total, 8 up, 8 in 2024-04-07T17:37:13.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:37:13 smithi100 bash[74942]: cluster 2024-04-07T17:37:12.473484+0000 mgr.y (mgr.25175) 2455 : cluster [DBG] pgmap v4031: 137 pgs: 137 active+clean; 359 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 175 KiB/s rd, 43 MiB/s wr, 285 op/s 2024-04-07T17:37:13.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:37:13 smithi100 bash[76202]: cluster 2024-04-07T17:37:12.409999+0000 mon.a (mon.0) 3873 : cluster [DBG] osdmap e2001: 8 total, 8 up, 8 in 2024-04-07T17:37:13.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:37:13 smithi100 bash[76202]: cluster 2024-04-07T17:37:12.473484+0000 mgr.y (mgr.25175) 2455 : cluster [DBG] pgmap v4031: 137 pgs: 137 active+clean; 359 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 175 KiB/s rd, 43 MiB/s wr, 285 op/s 2024-04-07T17:37:13.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:37:13 smithi144 bash[66654]: cluster 2024-04-07T17:37:12.409999+0000 mon.a (mon.0) 3873 : cluster [DBG] osdmap e2001: 8 total, 8 up, 8 in 2024-04-07T17:37:13.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:37:13 smithi144 bash[66654]: cluster 2024-04-07T17:37:12.473484+0000 mgr.y (mgr.25175) 2455 : cluster [DBG] pgmap v4031: 137 pgs: 137 active+clean; 359 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 175 KiB/s rd, 43 MiB/s wr, 285 op/s 2024-04-07T17:37:14.554 INFO:tasks.workunit.client.0.smithi100.stdout: i=11 exists [2974197~5131400,19095670~9448929,29778841~737427,31769017~10175249,49566880~7139115,62264039~1975360,66861224~7774919,78193873~4609655,85466334~5069656,92999623~8575543,106500403~7477102,116025937~3770559,121501610~12475984,140416111~18418645,160337176~7153146,173876238~1467374,180237750~12590714,194102759~7364680,202420372~19439868,222539835~1385676,225244127~38882289,272896915~13579450,287123044~8146585,300063155~4934365,309308954~1620369,321502401~5004346,326964255~3039511,337086692~6947322,345811290~1311192,350406459~6437804,360049192~84791,366057679~7704401,374580519~10318590,388412399~3633438,392232408~13894185] wrote [37362822~4581444,62264039~1975360,78193873~4609655,218521286~1672472,222539835~1385676,225244127~7197860,272896915~6011986,377322524~7576585,388479465~3566372,398418332~7708261] 2024-04-07T17:37:14.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:37:14 smithi100 bash[74942]: cluster 2024-04-07T17:37:13.412084+0000 mon.a (mon.0) 3874 : cluster [DBG] osdmap e2002: 8 total, 8 up, 8 in 2024-04-07T17:37:14.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:37:14 smithi100 bash[76202]: cluster 2024-04-07T17:37:13.412084+0000 mon.a (mon.0) 3874 : cluster [DBG] osdmap e2002: 8 total, 8 up, 8 in 2024-04-07T17:37:14.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:37:14 smithi144 bash[66654]: cluster 2024-04-07T17:37:13.412084+0000 mon.a (mon.0) 3874 : cluster [DBG] osdmap e2002: 8 total, 8 up, 8 in 2024-04-07T17:37:15.581 INFO:tasks.workunit.client.0.smithi100.stdout: i=12 exists [2974197~5131400,19095670~9448929,29778841~737427,31769017~13613313,45857929~10848066,62264039~1975360,66861224~7774919,78193873~4609655,85466334~5069656,92999623~10564968,106500403~13616197,121501610~12475984,140416111~18418645,160337176~7153146,173876238~1467374,180237750~12590714,194102759~7364680,202420372~19439868,222539835~1385676,225244127~38882289,272896915~13579450,287123044~8471300,298036890~6960630,309308954~1620369,321502401~8501365,337086692~6947322,345811290~1311192,350406459~6437804,360049192~84791,366057679~7704401,374580519~10318590,388412399~3633438,392232408~13894185] wrote [37386125~7996205,45857929~8483438,98683126~4881465,112275487~7841113,294114638~1479706,298036890~6918207,324984001~3125370,380094867~2863217,400878444~1331564] 2024-04-07T17:37:15.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:37:15 smithi100 bash[74942]: cluster 2024-04-07T17:37:14.430111+0000 mon.a (mon.0) 3875 : cluster [DBG] osdmap e2003: 8 total, 8 up, 8 in 2024-04-07T17:37:15.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:37:15 smithi100 bash[74942]: cluster 2024-04-07T17:37:14.474264+0000 mgr.y (mgr.25175) 2456 : cluster [DBG] pgmap v4034: 137 pgs: 137 active+clean; 401 MiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 64 KiB/s rd, 33 MiB/s wr, 102 op/s 2024-04-07T17:37:15.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:37:15 smithi100 bash[76202]: cluster 2024-04-07T17:37:14.430111+0000 mon.a (mon.0) 3875 : cluster [DBG] osdmap e2003: 8 total, 8 up, 8 in 2024-04-07T17:37:15.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:37:15 smithi100 bash[76202]: cluster 2024-04-07T17:37:14.474264+0000 mgr.y (mgr.25175) 2456 : cluster [DBG] pgmap v4034: 137 pgs: 137 active+clean; 401 MiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 64 KiB/s rd, 33 MiB/s wr, 102 op/s 2024-04-07T17:37:15.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:37:15 smithi144 bash[66654]: cluster 2024-04-07T17:37:14.430111+0000 mon.a (mon.0) 3875 : cluster [DBG] osdmap e2003: 8 total, 8 up, 8 in 2024-04-07T17:37:15.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:37:15 smithi144 bash[66654]: cluster 2024-04-07T17:37:14.474264+0000 mgr.y (mgr.25175) 2456 : cluster [DBG] pgmap v4034: 137 pgs: 137 active+clean; 401 MiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 64 KiB/s rd, 33 MiB/s wr, 102 op/s 2024-04-07T17:37:16.556 INFO:tasks.workunit.client.0.smithi100.stdout: i=13 exists [2974197~5131400,19095670~9448929,29778841~737427,31769017~13613313,45857929~10848066,62264039~1975360,66861224~10262977,78193873~4609655,85466334~5069656,92999623~10564968,106500403~13616197,121501610~12475984,136270708~2853668,140416111~18418645,160337176~7153146,173876238~1467374,180237750~12590714,194102759~7364680,202420372~19439868,222539835~1385676,225244127~38882289,267179572~19296793,287123044~8471300,298036890~6960630,309308954~1620369,321502401~8501365,337086692~6947322,345811290~1311192,350406459~6437804,360049192~84791,366057679~7704401,374580519~10318590,388412399~3633438,392232408~13894185,408516200~2525554] wrote [20802751~3327980,27329013~249441,70298794~6825407,136270708~2853668,144181083~2188757,267179572~6811128,394880591~2187459,403289909~2301039,408516200~2525554] 2024-04-07T17:37:16.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:37:16 smithi144 bash[66654]: cluster 2024-04-07T17:37:15.446165+0000 mon.a (mon.0) 3876 : cluster [DBG] osdmap e2004: 8 total, 8 up, 8 in 2024-04-07T17:37:16.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:37:16 smithi100 bash[74942]: cluster 2024-04-07T17:37:15.446165+0000 mon.a (mon.0) 3876 : cluster [DBG] osdmap e2004: 8 total, 8 up, 8 in 2024-04-07T17:37:16.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:37:16 smithi100 bash[76202]: cluster 2024-04-07T17:37:15.446165+0000 mon.a (mon.0) 3876 : cluster [DBG] osdmap e2004: 8 total, 8 up, 8 in 2024-04-07T17:37:17.625 INFO:tasks.workunit.client.0.smithi100.stdout: i=14 exists [2329752~5775845,18121228~10423371,29778841~737427,31769017~13613313,45857929~10848066,62264039~1975360,66861224~10262977,78193873~4609655,85466334~5069656,92999623~10564968,106500403~13616197,121501610~12475984,136270708~2853668,140416111~18418645,160337176~7153146,173876238~1467374,180237750~12590714,194102759~7364680,202420372~21505139,225244127~38882289,267179572~19296793,287123044~8471300,298036890~6960630,309308954~1620369,318962927~11040839,337086692~6947322,345811290~1311192,350406459~6437804,360049192~84791,366057679~7704401,374580519~10318590,388412399~3633438,392232408~13894185,408516200~2525554] wrote [2329752~961414,18121228~5535757,42101226~986344,124240937~6018736,187506017~4831615,213644600~9382452,288216273~6211110,299541775~4799266,318962927~7975475] 2024-04-07T17:37:17.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:37:17 smithi144 bash[66654]: cluster 2024-04-07T17:37:16.461118+0000 mon.a (mon.0) 3877 : cluster [DBG] osdmap e2005: 8 total, 8 up, 8 in 2024-04-07T17:37:17.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:37:17 smithi144 bash[66654]: cluster 2024-04-07T17:37:16.474968+0000 mgr.y (mgr.25175) 2457 : cluster [DBG] pgmap v4037: 137 pgs: 137 active+clean; 442 MiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 120 KiB/s rd, 19 MiB/s wr, 201 op/s 2024-04-07T17:37:17.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:37:17 smithi144 bash[66654]: audit 2024-04-07T17:37:16.857464+0000 mon.a (mon.0) 3878 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:37:17.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:37:17 smithi100 bash[74942]: cluster 2024-04-07T17:37:16.461118+0000 mon.a (mon.0) 3877 : cluster [DBG] osdmap e2005: 8 total, 8 up, 8 in 2024-04-07T17:37:17.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:37:17 smithi100 bash[74942]: cluster 2024-04-07T17:37:16.474968+0000 mgr.y (mgr.25175) 2457 : cluster [DBG] pgmap v4037: 137 pgs: 137 active+clean; 442 MiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 120 KiB/s rd, 19 MiB/s wr, 201 op/s 2024-04-07T17:37:17.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:37:17 smithi100 bash[74942]: audit 2024-04-07T17:37:16.857464+0000 mon.a (mon.0) 3878 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:37:17.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:37:17 smithi100 bash[76202]: cluster 2024-04-07T17:37:16.461118+0000 mon.a (mon.0) 3877 : cluster [DBG] osdmap e2005: 8 total, 8 up, 8 in 2024-04-07T17:37:17.955 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:37:17 smithi100 bash[76202]: cluster 2024-04-07T17:37:16.474968+0000 mgr.y (mgr.25175) 2457 : cluster [DBG] pgmap v4037: 137 pgs: 137 active+clean; 442 MiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 120 KiB/s rd, 19 MiB/s wr, 201 op/s 2024-04-07T17:37:17.955 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:37:17 smithi100 bash[76202]: audit 2024-04-07T17:37:16.857464+0000 mon.a (mon.0) 3878 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:37:18.616 INFO:tasks.workunit.client.0.smithi100.stdout: i=15 exists [2329752~5775845,18121228~10423371,29778841~737427,31769017~13613313,45857929~10848066,62264039~1975360,66861224~16545045,85466334~5069656,92999623~10564968,106500403~13616197,121501610~12475984,136270708~2853668,140416111~18418645,160337176~7153146,168236274~8367316,180237750~12590714,194102759~7364680,202420372~22190370,225244127~38882289,267179572~19296793,287123044~8471300,298036890~6960630,309308954~1620369,314393585~15610181,337086692~6947322,345811290~1311192,350406459~6437804,360049192~84791,366057679~7704401,374580519~10318590,388412399~3633438,392232408~14655911,408516200~2525554] wrote [39996176~4049302,76018526~7387743,168236274~8367316,220978325~3632417,242402883~5824033,255043666~3463088,314393585~7979426,401962264~4926055] 2024-04-07T17:37:18.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:37:18 smithi100 bash[74942]: cluster 2024-04-07T17:37:17.483320+0000 mon.a (mon.0) 3879 : cluster [DBG] osdmap e2006: 8 total, 8 up, 8 in 2024-04-07T17:37:18.704 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:37:18 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:37:18] "GET /metrics HTTP/1.1" 200 36760 "" "Prometheus/2.43.0" 2024-04-07T17:37:18.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:37:18 smithi100 bash[76202]: cluster 2024-04-07T17:37:17.483320+0000 mon.a (mon.0) 3879 : cluster [DBG] osdmap e2006: 8 total, 8 up, 8 in 2024-04-07T17:37:18.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:37:18 smithi144 bash[66654]: cluster 2024-04-07T17:37:17.483320+0000 mon.a (mon.0) 3879 : cluster [DBG] osdmap e2006: 8 total, 8 up, 8 in 2024-04-07T17:37:19.677 INFO:tasks.workunit.client.0.smithi100.stdout: i=16 exists [2329752~5775845,14933611~2822456,18121228~10423371,29778841~737427,31769017~13613313,45857929~10848066,62264039~1975360,66861224~16545045,85466334~5069656,92999623~10564968,106500403~13616197,121501610~12475984,136270708~2853668,140416111~18418645,160337176~7153146,168236274~8367316,180237750~12590714,194102759~7364680,202420372~22190370,225244127~38882289,267179572~28414772,298036890~6960630,309308954~1620369,314393585~15610181,332944411~11089603,345811290~1311192,350406459~6437804,360049192~84791,366057679~7704401,374580519~13389528,388412399~3633438,392232408~14655911,408516200~2525554] wrote [3107878~3847053,14933611~2822456,48881748~4683249,95781519~3680106,235932092~5747954,254075762~1580725,286319045~3681129,323018422~3348552,332944411~6919544,380563114~7406933] 2024-04-07T17:37:19.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:37:19 smithi144 bash[66654]: cluster 2024-04-07T17:37:18.475979+0000 mgr.y (mgr.25175) 2458 : cluster [DBG] pgmap v4039: 137 pgs: 137 active+clean; 589 MiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 211 KiB/s rd, 41 MiB/s wr, 333 op/s 2024-04-07T17:37:19.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:37:19 smithi144 bash[66654]: cluster 2024-04-07T17:37:18.487097+0000 mon.a (mon.0) 3880 : cluster [DBG] osdmap e2007: 8 total, 8 up, 8 in 2024-04-07T17:37:19.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:37:19 smithi100 bash[74942]: cluster 2024-04-07T17:37:18.475979+0000 mgr.y (mgr.25175) 2458 : cluster [DBG] pgmap v4039: 137 pgs: 137 active+clean; 589 MiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 211 KiB/s rd, 41 MiB/s wr, 333 op/s 2024-04-07T17:37:19.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:37:19 smithi100 bash[74942]: cluster 2024-04-07T17:37:18.487097+0000 mon.a (mon.0) 3880 : cluster [DBG] osdmap e2007: 8 total, 8 up, 8 in 2024-04-07T17:37:19.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:37:19 smithi100 bash[76202]: cluster 2024-04-07T17:37:18.475979+0000 mgr.y (mgr.25175) 2458 : cluster [DBG] pgmap v4039: 137 pgs: 137 active+clean; 589 MiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 211 KiB/s rd, 41 MiB/s wr, 333 op/s 2024-04-07T17:37:19.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:37:19 smithi100 bash[76202]: cluster 2024-04-07T17:37:18.487097+0000 mon.a (mon.0) 3880 : cluster [DBG] osdmap e2007: 8 total, 8 up, 8 in 2024-04-07T17:37:20.681 INFO:tasks.workunit.client.0.smithi100.stdout: i=17 exists [2329752~5775845,12468100~16076499,29778841~737427,31769017~13613313,45857929~10848066,62264039~1975360,66861224~16545045,85466334~5069656,92999623~10564968,106500403~13616197,121501610~12475984,136270708~22564048,160337176~7153146,168236274~8367316,180237750~12590714,194102759~7364680,202420372~61706044,267179572~28414772,298036890~6960630,309308954~1620369,314393585~15610181,332944411~11089603,345811290~1311192,350406459~6863082,360049192~84791,366057679~7704401,374580519~13389528,388412399~3633438,392232408~14655911,408516200~2525554] wrote [12468100~7703070,39912210~3191969,68206575~6181429,136996101~4514786,211678512~920259,220705920~8848195,242476971~1393065,282485483~6755344,355806290~1463251] 2024-04-07T17:37:20.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:37:20 smithi144 bash[66654]: cluster 2024-04-07T17:37:19.540289+0000 mon.a (mon.0) 3881 : cluster [DBG] osdmap e2008: 8 total, 8 up, 8 in 2024-04-07T17:37:20.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:37:20 smithi100 bash[74942]: cluster 2024-04-07T17:37:19.540289+0000 mon.a (mon.0) 3881 : cluster [DBG] osdmap e2008: 8 total, 8 up, 8 in 2024-04-07T17:37:20.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:37:20 smithi100 bash[76202]: cluster 2024-04-07T17:37:19.540289+0000 mon.a (mon.0) 3881 : cluster [DBG] osdmap e2008: 8 total, 8 up, 8 in 2024-04-07T17:37:21.683 INFO:tasks.workunit.client.0.smithi100.stdout: i=18 exists [2329752~9105991,12468100~16076499,29778841~737427,31769017~13613313,45857929~10848066,58033882~7969874,66861224~16545045,85466334~5069656,92999623~10564968,106500403~13616197,121501610~12475984,136270708~22564048,160337176~7153146,168236274~8367316,180237750~12590714,194102759~7364680,202420372~61706044,267179572~28414772,298036890~6960630,309308954~1620369,314393585~15610181,332944411~11089603,345811290~1311192,350406459~6863082,360049192~84791,366057679~7704401,374580519~17465318,392232408~14655911,408516200~2525554] wrote [6688790~4746953,58033882~7969874,126602360~5222168,183274874~3283813,210147535~5174157,219514234~5127006,229742904~13365134,298643235~5745703,386437457~2358462] 2024-04-07T17:37:21.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:37:21 smithi144 bash[66654]: cluster 2024-04-07T17:37:20.476851+0000 mgr.y (mgr.25175) 2459 : cluster [DBG] pgmap v4042: 137 pgs: 137 active+clean; 678 MiB data, 5.2 GiB used, 710 GiB / 715 GiB avail; 215 KiB/s rd, 55 MiB/s wr, 345 op/s 2024-04-07T17:37:21.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:37:21 smithi144 bash[66654]: cluster 2024-04-07T17:37:20.555194+0000 mon.a (mon.0) 3882 : cluster [DBG] osdmap e2009: 8 total, 8 up, 8 in 2024-04-07T17:37:21.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:37:21 smithi100 bash[74942]: cluster 2024-04-07T17:37:20.476851+0000 mgr.y (mgr.25175) 2459 : cluster [DBG] pgmap v4042: 137 pgs: 137 active+clean; 678 MiB data, 5.2 GiB used, 710 GiB / 715 GiB avail; 215 KiB/s rd, 55 MiB/s wr, 345 op/s 2024-04-07T17:37:21.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:37:21 smithi100 bash[74942]: cluster 2024-04-07T17:37:20.555194+0000 mon.a (mon.0) 3882 : cluster [DBG] osdmap e2009: 8 total, 8 up, 8 in 2024-04-07T17:37:21.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:37:21 smithi100 bash[76202]: cluster 2024-04-07T17:37:20.476851+0000 mgr.y (mgr.25175) 2459 : cluster [DBG] pgmap v4042: 137 pgs: 137 active+clean; 678 MiB data, 5.2 GiB used, 710 GiB / 715 GiB avail; 215 KiB/s rd, 55 MiB/s wr, 345 op/s 2024-04-07T17:37:21.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:37:21 smithi100 bash[76202]: cluster 2024-04-07T17:37:20.555194+0000 mon.a (mon.0) 3882 : cluster [DBG] osdmap e2009: 8 total, 8 up, 8 in 2024-04-07T17:37:22.568 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:37:22 smithi144 bash[95233]: ts=2024-04-07T17:37:22.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:37:22.704 INFO:tasks.workunit.client.0.smithi100.stdout: i=19 exists [2329752~26214847,29778841~737427,31769017~13613313,45857929~20145827,66861224~16545045,85466334~5069656,92999623~10564968,106500403~13616197,121501610~12475984,136270708~22564048,160337176~7153146,168236274~8367316,180237750~12590714,194102759~7364680,202420372~61706044,267179572~28414772,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350406459~6863082,360049192~84791,366057679~7704401,374580519~17465318,392232408~14655911,408516200~2525554] wrote [9531660~4517757,55380210~7770032,71826911~6724391,141885014~6484550,149552729~4403047,174179109~452103,219630111~5573206,226376149~484931,306903613~5021411] 2024-04-07T17:37:22.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:37:22 smithi144 bash[66654]: cluster 2024-04-07T17:37:21.551523+0000 mon.a (mon.0) 3883 : cluster [DBG] osdmap e2010: 8 total, 8 up, 8 in 2024-04-07T17:37:22.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:37:22 smithi100 bash[74942]: cluster 2024-04-07T17:37:21.551523+0000 mon.a (mon.0) 3883 : cluster [DBG] osdmap e2010: 8 total, 8 up, 8 in 2024-04-07T17:37:22.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:37:22 smithi100 bash[76202]: cluster 2024-04-07T17:37:21.551523+0000 mon.a (mon.0) 3883 : cluster [DBG] osdmap e2010: 8 total, 8 up, 8 in 2024-04-07T17:37:23.604 INFO:tasks.workunit.client.0.smithi100.stdout:from 0 to 1 diff [50852715~5853280,67034044~1727684,85466334~5069656,194102759~3585735,253547268~2401828,275605911~4974570,350406459~6437804,381619449~3142051,392232408~7812666] 2024-04-07T17:37:23.604 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [50852715~5853280,67034044~1727684,85466334~5069656,194102759~3585735,253547268~2401828,275605911~4974570,350406459~6437804,381619449~3142051,392232408~7812666] 2024-04-07T17:37:23.652 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [50331648~6374347,62914560~5847168,83886080~6649910,192937984~4750510,251658240~4290856,272629760~7950721,348127232~8717031,377487360~7274140,390070272~9974802] 2024-04-07T17:37:23.652 INFO:tasks.workunit.client.0.smithi100.stdout:from 0 to 2 diff [25445275~3099324,50852715~5853280,67034044~1727684,85466334~5069656,108048350~5929155,146434166~11449972,180237750~3536884,186904555~1491543,194102759~7364680,210875736~10984504,253547268~2401828,275605911~4974570,350406459~6437804,381619449~3142051,392232408~7812666] 2024-04-07T17:37:23.652 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [25445275~3099324,50852715~5853280,67034044~1727684,85466334~5069656,108048350~5929155,146434166~11449972,180237750~3536884,186904555~1491543,194102759~7364680,210875736~10984504,253547268~2401828,275605911~4974570,350406459~6437804,381619449~3142051,392232408~7812666] 2024-04-07T17:37:23.670 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [25445275~3099324,50331648~6374347,62914560~5847168,83886080~6649910,104857600~9119905,142606336~15277802,176160768~7613866,185165240~3230858,192937984~8529455,210875736~10984504,251658240~4290856,272629760~7950721,348127232~8717031,377487360~7274140,390070272~9974802] 2024-04-07T17:37:23.670 INFO:tasks.workunit.client.0.smithi100.stdout:from 0 to 3 diff [20375589~3479656,25445275~3099324,50852715~5853280,67034044~1727684,81710240~1015114,85466334~5069656,108048350~5929155,143151737~1037788,146434166~12400590,165813830~1137640,180237750~3536884,185860175~6968289,194102759~7364680,203328670~4346303,210875736~10984504,243012066~4719194,253547268~2401828,275605911~7066628,350406459~6437804,381619449~3142051,392232408~7812666] 2024-04-07T17:37:23.670 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [20375589~3479656,25445275~3099324,50852715~5853280,67034044~1727684,81710240~1015114,85466334~5069656,108048350~5929155,143151737~1037788,146434166~12400590,165813830~1137640,180237750~3536884,185860175~6968289,194102759~7364680,203328670~4346303,210875736~10984504,243012066~4719194,253547268~2401828,275605911~7066628,350406459~6437804,381619449~3142051,392232408~7812666] 2024-04-07T17:37:23.687 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [16777216~7078029,25445275~3099324,50331648~6374347,62914560~5847168,79691776~3033578,83886080~6649910,104857600~9119905,142606336~16228420,163577856~3373614,176160768~7613866,185165240~7663224,192937984~14736989,210875736~10984504,239075328~8655932,251658240~4290856,272629760~10042779,348127232~8717031,377487360~7274140,390070272~9974802] 2024-04-07T17:37:23.687 INFO:tasks.workunit.client.0.smithi100.stdout:from 0 to 4 diff [20375589~3479656,25445275~3099324,29778841~737427,50852715~5853280,67034044~1727684,81710240~1015114,85466334~5069656,107619787~6357718,143151737~15683019,165813830~1137640,173876238~1467374,180237750~3536884,185860175~6968289,194102759~7364680,203328670~4346303,210875736~10984504,243012066~4719194,253547268~4410812,275605911~9036918,309308954~1620369,350406459~6437804,360049192~84791,381619449~3142051,388412399~2413139,392232408~7812666] 2024-04-07T17:37:23.687 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [20375589~3479656,25445275~3099324,29778841~737427,50852715~5853280,67034044~1727684,81710240~1015114,85466334~5069656,107619787~6357718,143151737~15683019,165813830~1137640,173876238~1467374,180237750~3536884,185860175~6968289,194102759~7364680,203328670~4346303,210875736~10984504,243012066~4719194,253547268~4410812,275605911~9036918,309308954~1620369,350406459~6437804,360049192~84791,381619449~3142051,388412399~2413139,392232408~7812666] 2024-04-07T17:37:23.701 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [16777216~7078029,25445275~3099324,29360128~1156140,50331648~6374347,62914560~5847168,79691776~3033578,83886080~6649910,104857600~9119905,142606336~16228420,163577856~3373614,171966464~3377148,176160768~7613866,185165240~7663224,192937984~14736989,210875736~10984504,239075328~8655932,251658240~6299840,272629760~12013069,306184192~4745131,348127232~12006751,377487360~7274140,385875968~14169106] 2024-04-07T17:37:23.701 INFO:tasks.workunit.client.0.smithi100.stdout:from 0 to 5 diff [20375589~3479656,25445275~3099324,29778841~737427,50852715~5853280,67034044~1727684,81710240~1015114,85466334~5069656,107619787~6357718,140416111~18418645,160337176~6614294,173876238~1467374,180237750~3536884,185860175~6968289,194102759~7364680,203328670~4346303,210875736~10984504,243012066~7805483,253547268~4410812,275605911~9036918,301491055~2008066,309308954~1620369,326964255~3039511,350406459~6437804,360049192~84791,374607324~4374722,381619449~3142051,388412399~2413139,392232408~7812666] 2024-04-07T17:37:23.701 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [20375589~3479656,25445275~3099324,29778841~737427,50852715~5853280,67034044~1727684,81710240~1015114,85466334~5069656,107619787~6357718,140416111~18418645,160337176~6614294,173876238~1467374,180237750~3536884,185860175~6968289,194102759~7364680,203328670~4346303,210875736~10984504,243012066~7805483,253547268~4410812,275605911~9036918,301491055~2008066,309308954~1620369,326964255~3039511,350406459~6437804,360049192~84791,374607324~4374722,381619449~3142051,388412399~2413139,392232408~7812666] 2024-04-07T17:37:23.714 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [16777216~7078029,25445275~3099324,29360128~1156140,50331648~6374347,62914560~5847168,79691776~3033578,83886080~6649910,104857600~9119905,138412032~20422724,159383552~7567918,171966464~3377148,176160768~7613866,185165240~7663224,192937984~14736989,210875736~10984504,239075328~11742221,251658240~6299840,272629760~12013069,297795584~5703537,306184192~4745131,326506747~3497019,348127232~12006751,373293056~11468444,385875968~14169106] 2024-04-07T17:37:23.714 INFO:tasks.workunit.client.0.smithi100.stdout:from 0 to 6 diff [19480873~9063726,29778841~737427,50852715~5853280,67034044~1727684,81710240~1015114,85466334~5069656,107619787~6357718,140416111~18418645,160337176~7153146,173876238~1467374,180237750~3536884,185860175~6968289,194102759~7364680,203328670~4346303,210325964~283338,210875736~10984504,243012066~7906095,253547268~10579148,273160535~11482294,292398584~2871045,301491055~2008066,309308954~1620369,326964255~3039511,337086692~6947322,350406459~6437804,360049192~84791,374607324~4374722,381619449~3142051,388412399~2413139,392232408~7812666] 2024-04-07T17:37:23.714 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [19480873~9063726,29778841~737427,50852715~5853280,67034044~1727684,81710240~1015114,85466334~5069656,107619787~6357718,140416111~18418645,160337176~7153146,173876238~1467374,180237750~3536884,185860175~6968289,194102759~7364680,203328670~4346303,210325964~283338,210875736~10984504,243012066~7906095,253547268~10579148,273160535~11482294,292398584~2871045,301491055~2008066,309308954~1620369,326964255~3039511,337086692~6947322,350406459~6437804,360049192~84791,374607324~4374722,381619449~3142051,388412399~2413139,392232408~7812666] 2024-04-07T17:37:23.725 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [16777216~11767383,29360128~1156140,50331648~6374347,62914560~5847168,79691776~3033578,83886080~6649910,104857600~9119905,138412032~20422724,159383552~8106770,171966464~3377148,176160768~7613866,185165240~7663224,192937984~14736989,210325964~283338,210875736~10984504,239075328~11842833,251658240~12468176,272629760~12013069,289406976~5862653,297795584~5703537,306184192~4745131,326506747~3497019,337086692~6947322,348127232~12006751,373293056~11468444,385875968~14169106] 2024-04-07T17:37:23.725 INFO:tasks.workunit.client.0.smithi100.stdout:from 0 to 7 diff [19480873~9063726,29778841~737427,49566880~7139115,66861224~7774919,81710240~1015114,85466334~5069656,107619787~6357718,140416111~18418645,160337176~7153146,173876238~1467374,180237750~12590714,194102759~7364680,202420372~5254601,210325964~283338,210854655~11005585,240886088~1607107,243012066~7906095,253547268~10579148,273160535~11482294,292398584~2871045,301491055~2008066,309308954~1620369,326964255~3039511,337086692~6947322,345811290~1311192,350406459~6437804,360049192~84791,367170792~4497984,374580519~4455211,381619449~3142051,388412399~2413139,392232408~7812666] 2024-04-07T17:37:23.725 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [19480873~9063726,29778841~737427,49566880~7139115,66861224~7774919,81710240~1015114,85466334~5069656,107619787~6357718,140416111~18418645,160337176~7153146,173876238~1467374,180237750~12590714,194102759~7364680,202420372~5254601,210325964~283338,210854655~11005585,240886088~1607107,243012066~7906095,253547268~10579148,273160535~11482294,292398584~2871045,301491055~2008066,309308954~1620369,326964255~3039511,337086692~6947322,345811290~1311192,350406459~6437804,360049192~84791,367170792~4497984,374580519~4455211,381619449~3142051,388412399~2413139,392232408~7812666] 2024-04-07T17:37:23.736 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [16777216~11767383,29360128~1156140,46137344~10568651,62914560~11721583,79691776~3033578,83886080~6649910,104857600~9119905,138412032~20422724,159383552~8106770,171966464~3377148,176160768~16667696,192937984~14736989,210325964~283338,210854655~11005585,239075328~11842833,251658240~12468176,272629760~12013069,289406976~5862653,297795584~5703537,306184192~4745131,326506747~3497019,337086692~10035790,348127232~12006751,367170792~4497984,373293056~11468444,385875968~14169106] 2024-04-07T17:37:23.736 INFO:tasks.workunit.client.0.smithi100.stdout:from 0 to 8 diff [19480873~9063726,29778841~737427,49566880~7139115,66861224~7774919,81710240~1015114,85466334~5069656,92999623~7613429,107619787~6357718,128030792~5946802,140416111~18418645,160337176~7153146,173876238~1467374,180237750~12590714,194102759~7364680,202420372~5254601,210325964~283338,210854655~11005585,235153608~5013034,240886088~1607107,243012066~7906095,253228895~10897521,273160535~11482294,292398584~2871045,300063155~4934365,309308954~1620369,326964255~3039511,337086692~6947322,345811290~1311192,350406459~6437804,360049192~84791,367170792~4497984,374580519~4455211,381619449~3142051,388412399~2413139,392232408~7812666] 2024-04-07T17:37:23.736 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [19480873~9063726,29778841~737427,49566880~7139115,66861224~7774919,81710240~1015114,85466334~5069656,92999623~7613429,107619787~6357718,128030792~5946802,140416111~18418645,160337176~7153146,173876238~1467374,180237750~12590714,194102759~7364680,202420372~5254601,210325964~283338,210854655~11005585,235153608~5013034,240886088~1607107,243012066~7906095,253228895~10897521,273160535~11482294,292398584~2871045,300063155~4934365,309308954~1620369,326964255~3039511,337086692~6947322,345811290~1311192,350406459~6437804,360049192~84791,367170792~4497984,374580519~4455211,381619449~3142051,388412399~2413139,392232408~7812666] 2024-04-07T17:37:23.747 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [16777216~11767383,29360128~1156140,46137344~10568651,62914560~11721583,79691776~3033578,83886080~6649910,92274688~8338364,104857600~9119905,128030792~5946802,138412032~20422724,159383552~8106770,171966464~3377148,176160768~16667696,192937984~14736989,210325964~283338,210854655~11005585,234881024~16037137,251658240~12468176,272629760~12013069,289406976~5862653,297795584~7201936,306184192~4745131,326506747~3497019,337086692~10035790,348127232~12006751,367170792~4497984,373293056~11468444,385875968~14169106] 2024-04-07T17:37:23.747 INFO:tasks.workunit.client.0.smithi100.stdout:from 0 to 9 diff [7275769~186849,19095670~9448929,29778841~737427,31769017~7139161,49566880~7139115,66861224~7774919,81710240~1015114,85466334~5069656,92999623~8522076,107619787~6357718,116025937~3770559,128030792~5946802,140416111~18418645,160337176~7153146,173876238~1467374,180237750~12590714,194102759~7364680,202420372~5254601,210325964~283338,210854655~11005585,230813446~9353196,240886088~1607107,243012066~21114350,273160535~13315830,292398584~2871045,300063155~4934365,309308954~1620369,326964255~3039511,337086692~6947322,345811290~1311192,350406459~6437804,360049192~84791,367170792~4497984,374580519~4455211,381619449~3142051,388412399~2413139,392232408~7812666] 2024-04-07T17:37:23.748 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [7275769~186849,19095670~9448929,29778841~737427,31769017~7139161,49566880~7139115,66861224~7774919,81710240~1015114,85466334~5069656,92999623~8522076,107619787~6357718,116025937~3770559,128030792~5946802,140416111~18418645,160337176~7153146,173876238~1467374,180237750~12590714,194102759~7364680,202420372~5254601,210325964~283338,210854655~11005585,230813446~9353196,240886088~1607107,243012066~21114350,273160535~13315830,292398584~2871045,300063155~4934365,309308954~1620369,326964255~3039511,337086692~6947322,345811290~1311192,350406459~6437804,360049192~84791,367170792~4497984,374580519~4455211,381619449~3142051,388412399~2413139,392232408~7812666] 2024-04-07T17:37:23.757 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [4194304~3268314,16777216~11767383,29360128~9548050,46137344~10568651,62914560~11721583,79691776~3033578,83886080~6649910,92274688~9247011,104857600~14938896,128030792~5946802,138412032~20422724,159383552~8106770,171966464~3377148,176160768~16667696,192937984~14736989,210325964~283338,210854655~11005585,230686720~33439696,272629760~13846605,289406976~5862653,297795584~7201936,306184192~4745131,326506747~3497019,337086692~10035790,348127232~12006751,367170792~4497984,373293056~11468444,385875968~14169106] 2024-04-07T17:37:23.757 INFO:tasks.workunit.client.0.smithi100.stdout:from 0 to 10 diff [2974197~5131400,19095670~9448929,29778841~737427,31769017~7139161,49566880~7139115,66861224~7774919,81710240~1015114,85466334~5069656,92999623~8522076,106500403~7477102,116025937~3770559,121501610~12475984,140416111~18418645,160337176~7153146,173876238~1467374,180237750~12590714,194102759~7364680,202420372~5254601,210325964~283338,210854655~11005585,230813446~33312970,273160535~13315830,287123044~8146585,300063155~4934365,309308954~1620369,323691380~1672817,326964255~3039511,337086692~6947322,345811290~1311192,350406459~6437804,360049192~84791,366341667~7420413,374580519~4455211,381619449~3142051,388412399~2413139,392232408~7812666] 2024-04-07T17:37:23.757 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2974197~5131400,19095670~9448929,29778841~737427,31769017~7139161,49566880~7139115,66861224~7774919,81710240~1015114,85466334~5069656,92999623~8522076,106500403~7477102,116025937~3770559,121501610~12475984,140416111~18418645,160337176~7153146,173876238~1467374,180237750~12590714,194102759~7364680,202420372~5254601,210325964~283338,210854655~11005585,230813446~33312970,273160535~13315830,287123044~8146585,300063155~4934365,309308954~1620369,323691380~1672817,326964255~3039511,337086692~6947322,345811290~1311192,350406459~6437804,360049192~84791,366341667~7420413,374580519~4455211,381619449~3142051,388412399~2413139,392232408~7812666] 2024-04-07T17:37:23.766 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~8105597,16777216~11767383,29360128~9548050,46137344~10568651,62914560~11721583,79691776~3033578,83886080~6649910,92274688~9247011,104857600~29119994,138412032~20422724,159383552~8106770,171966464~3377148,176160768~16667696,192937984~14736989,210325964~283338,210854655~11005585,230686720~33439696,272629760~22639869,297795584~7201936,306184192~4745131,323691380~1672817,326506747~3497019,337086692~10035790,348127232~12006751,366341667~18419833,385875968~14169106] 2024-04-07T17:37:23.766 INFO:tasks.workunit.client.0.smithi100.stdout:from 0 to 11 diff [2974197~5131400,19095670~9448929,29778841~737427,31769017~10175249,49566880~7139115,62264039~1975360,66861224~7774919,78193873~4609655,85466334~5069656,92999623~8522076,106500403~7477102,116025937~3770559,121501610~12475984,140416111~18418645,160337176~7153146,173876238~1467374,180237750~12590714,194102759~7364680,202420372~5254601,210325964~283338,210854655~11005585,222539835~1385676,225244127~38882289,272896915~13579450,287123044~8146585,300063155~4934365,309308954~1620369,323691380~1672817,326964255~3039511,337086692~6947322,345811290~1311192,350406459~6437804,360049192~84791,366341667~7420413,374580519~10318590,388412399~3633438,392232408~13894185] 2024-04-07T17:37:23.766 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2974197~5131400,19095670~9448929,29778841~737427,31769017~10175249,49566880~7139115,62264039~1975360,66861224~7774919,78193873~4609655,85466334~5069656,92999623~8522076,106500403~7477102,116025937~3770559,121501610~12475984,140416111~18418645,160337176~7153146,173876238~1467374,180237750~12590714,194102759~7364680,202420372~5254601,210325964~283338,210854655~11005585,222539835~1385676,225244127~38882289,272896915~13579450,287123044~8146585,300063155~4934365,309308954~1620369,323691380~1672817,326964255~3039511,337086692~6947322,345811290~1311192,350406459~6437804,360049192~84791,366341667~7420413,374580519~10318590,388412399~3633438,392232408~13894185] 2024-04-07T17:37:23.776 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~8105597,16777216~11767383,29360128~12584138,46137344~10568651,58720256~15915887,75497472~7306056,83886080~6649910,92274688~9247011,104857600~29119994,138412032~20422724,159383552~8106770,171966464~3377148,176160768~16667696,192937984~14736989,210325964~283338,210854655~11005585,222298112~41828304,272629760~22639869,297795584~7201936,306184192~4745131,323691380~1672817,326506747~3497019,337086692~10035790,348127232~12006751,366341667~18557442,385875968~20250625] 2024-04-07T17:37:23.776 INFO:tasks.workunit.client.0.smithi100.stdout:from 0 to 12 diff [2974197~5131400,19095670~9448929,29778841~737427,31769017~13613313,45857929~10848066,62264039~1975360,66861224~7774919,78193873~4609655,85466334~5069656,92999623~10564968,106500403~13616197,121501610~12475984,140416111~18418645,160337176~7153146,173876238~1467374,180237750~12590714,194102759~7364680,202420372~5254601,210325964~283338,210854655~11005585,222539835~1385676,225244127~38882289,272896915~13579450,287123044~8471300,298036890~6960630,309308954~1620369,323691380~6312386,337086692~6947322,345811290~1311192,350406459~6437804,360049192~84791,366341667~7420413,374580519~10318590,388412399~3633438,392232408~13894185] 2024-04-07T17:37:23.776 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2974197~5131400,19095670~9448929,29778841~737427,31769017~13613313,45857929~10848066,62264039~1975360,66861224~7774919,78193873~4609655,85466334~5069656,92999623~10564968,106500403~13616197,121501610~12475984,140416111~18418645,160337176~7153146,173876238~1467374,180237750~12590714,194102759~7364680,202420372~5254601,210325964~283338,210854655~11005585,222539835~1385676,225244127~38882289,272896915~13579450,287123044~8471300,298036890~6960630,309308954~1620369,323691380~6312386,337086692~6947322,345811290~1311192,350406459~6437804,360049192~84791,366341667~7420413,374580519~10318590,388412399~3633438,392232408~13894185] 2024-04-07T17:37:23.784 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~8105597,16777216~11767383,29360128~27345867,58720256~15915887,75497472~7306056,83886080~6649910,92274688~11289903,104857600~29119994,138412032~20422724,159383552~8106770,171966464~3377148,176160768~16667696,192937984~14736989,210325964~283338,210854655~11005585,222298112~41828304,272629760~22964584,297795584~7201936,306184192~4745131,323691380~6312386,337086692~10035790,348127232~12006751,366341667~18557442,385875968~20250625] 2024-04-07T17:37:23.784 INFO:tasks.workunit.client.0.smithi100.stdout:from 0 to 13 diff [2974197~5131400,19095670~9448929,29778841~737427,31769017~13613313,45857929~10848066,62264039~1975360,66861224~10262977,78193873~4609655,85466334~5069656,92999623~10564968,106500403~13616197,121501610~12475984,136270708~2853668,140416111~18418645,160337176~7153146,173876238~1467374,180237750~12590714,194102759~7364680,202420372~5254601,210325964~283338,210854655~11005585,222539835~1385676,225244127~38882289,267179572~19296793,287123044~8471300,298036890~6960630,309308954~1620369,323691380~6312386,337086692~6947322,345811290~1311192,350406459~6437804,360049192~84791,366341667~7420413,374580519~10318590,388412399~3633438,392232408~13894185,408516200~2525554] 2024-04-07T17:37:23.784 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2974197~5131400,19095670~9448929,29778841~737427,31769017~13613313,45857929~10848066,62264039~1975360,66861224~10262977,78193873~4609655,85466334~5069656,92999623~10564968,106500403~13616197,121501610~12475984,136270708~2853668,140416111~18418645,160337176~7153146,173876238~1467374,180237750~12590714,194102759~7364680,202420372~5254601,210325964~283338,210854655~11005585,222539835~1385676,225244127~38882289,267179572~19296793,287123044~8471300,298036890~6960630,309308954~1620369,323691380~6312386,337086692~6947322,345811290~1311192,350406459~6437804,360049192~84791,366341667~7420413,374580519~10318590,388412399~3633438,392232408~13894185,408516200~2525554] 2024-04-07T17:37:23.792 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~8105597,16777216~11767383,29360128~27345867,58720256~24083272,83886080~6649910,92274688~11289903,104857600~29119994,134217728~24617028,159383552~8106770,171966464~3377148,176160768~16667696,192937984~14736989,210325964~283338,210854655~11005585,222298112~41828304,264241152~31353192,297795584~7201936,306184192~4745131,323691380~6312386,337086692~10035790,348127232~12006751,366341667~18557442,385875968~20250625,406847488~4194266] 2024-04-07T17:37:23.792 INFO:tasks.workunit.client.0.smithi100.stdout:from 0 to 14 diff [2329752~5775845,18121228~10423371,29778841~737427,31769017~13613313,45857929~10848066,62264039~1975360,66861224~10262977,78193873~4609655,85466334~5069656,92999623~10564968,106500403~13616197,121501610~12475984,136270708~2853668,140416111~18418645,160337176~7153146,173876238~1467374,180237750~12590714,194102759~7364680,202420372~5254601,210325964~283338,210854655~13070856,225244127~38882289,267179572~19296793,287123044~8471300,298036890~6960630,309308954~1620369,318962927~11040839,337086692~6947322,345811290~1311192,350406459~6437804,360049192~84791,366341667~7420413,374580519~10318590,388412399~3633438,392232408~13894185,408516200~2525554] 2024-04-07T17:37:23.792 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~5775845,18121228~10423371,29778841~737427,31769017~13613313,45857929~10848066,62264039~1975360,66861224~10262977,78193873~4609655,85466334~5069656,92999623~10564968,106500403~13616197,121501610~12475984,136270708~2853668,140416111~18418645,160337176~7153146,173876238~1467374,180237750~12590714,194102759~7364680,202420372~5254601,210325964~283338,210854655~13070856,225244127~38882289,267179572~19296793,287123044~8471300,298036890~6960630,309308954~1620369,318962927~11040839,337086692~6947322,345811290~1311192,350406459~6437804,360049192~84791,366341667~7420413,374580519~10318590,388412399~3633438,392232408~13894185,408516200~2525554] 2024-04-07T17:37:23.801 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~8105597,16777216~11767383,29360128~27345867,58720256~24083272,83886080~6649910,92274688~11289903,104857600~29119994,134217728~24617028,159383552~8106770,171966464~3377148,176160768~16667696,192937984~14736989,210325964~283338,210854655~53271761,264241152~31353192,297795584~7201936,306184192~4745131,318962927~11040839,337086692~10035790,348127232~12006751,366341667~18557442,385875968~20250625,406847488~4194266] 2024-04-07T17:37:23.801 INFO:tasks.workunit.client.0.smithi100.stdout:from 0 to 15 diff [2329752~5775845,18121228~10423371,29778841~737427,31769017~13613313,45857929~10848066,62264039~1975360,66861224~16545045,85466334~5069656,92999623~10564968,106500403~13616197,121501610~12475984,136270708~2853668,140416111~18418645,160337176~7153146,168236274~8367316,180237750~12590714,194102759~7364680,202420372~5254601,210325964~283338,210854655~13756087,225244127~38882289,267179572~19296793,287123044~8471300,298036890~6960630,309308954~1620369,314393585~15610181,337086692~6947322,345811290~1311192,350406459~6437804,360049192~84791,366341667~7420413,374580519~10318590,388412399~3633438,392232408~14655911,408516200~2525554] 2024-04-07T17:37:23.801 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~5775845,18121228~10423371,29778841~737427,31769017~13613313,45857929~10848066,62264039~1975360,66861224~16545045,85466334~5069656,92999623~10564968,106500403~13616197,121501610~12475984,136270708~2853668,140416111~18418645,160337176~7153146,168236274~8367316,180237750~12590714,194102759~7364680,202420372~5254601,210325964~283338,210854655~13756087,225244127~38882289,267179572~19296793,287123044~8471300,298036890~6960630,309308954~1620369,314393585~15610181,337086692~6947322,345811290~1311192,350406459~6437804,360049192~84791,366341667~7420413,374580519~10318590,388412399~3633438,392232408~14655911,408516200~2525554] 2024-04-07T17:37:23.809 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~8105597,16777216~11767383,29360128~27345867,58720256~24686013,83886080~6649910,92274688~11289903,104857600~29119994,134217728~24617028,159383552~8106770,167772160~25056304,192937984~14736989,210325964~283338,210854655~53271761,264241152~31353192,297795584~7201936,306184192~23819574,337086692~10035790,348127232~12006751,366341667~18557442,385875968~25165786] 2024-04-07T17:37:23.809 INFO:tasks.workunit.client.0.smithi100.stdout:from 0 to 16 diff [2329752~5775845,14933611~2822456,18121228~10423371,29778841~737427,31769017~13613313,45857929~10848066,62264039~1975360,66861224~16545045,85466334~5069656,92999623~10564968,106500403~13616197,121501610~12475984,136270708~2853668,140416111~18418645,160337176~7153146,168236274~8367316,180237750~12590714,194102759~7364680,202420372~5254601,210325964~283338,210854655~13756087,225244127~38882289,267179572~28414772,298036890~6960630,309308954~1620369,314393585~15610181,332944411~11089603,345811290~1311192,350406459~6437804,360049192~84791,366341667~7420413,374580519~13389528,388412399~3633438,392232408~14655911,408516200~2525554] 2024-04-07T17:37:23.809 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~5775845,14933611~2822456,18121228~10423371,29778841~737427,31769017~13613313,45857929~10848066,62264039~1975360,66861224~16545045,85466334~5069656,92999623~10564968,106500403~13616197,121501610~12475984,136270708~2853668,140416111~18418645,160337176~7153146,168236274~8367316,180237750~12590714,194102759~7364680,202420372~5254601,210325964~283338,210854655~13756087,225244127~38882289,267179572~28414772,298036890~6960630,309308954~1620369,314393585~15610181,332944411~11089603,345811290~1311192,350406459~6437804,360049192~84791,366341667~7420413,374580519~13389528,388412399~3633438,392232408~14655911,408516200~2525554] 2024-04-07T17:37:23.817 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~8105597,12582912~15961687,29360128~27345867,58720256~24686013,83886080~6649910,92274688~11289903,104857600~29119994,134217728~24617028,159383552~8106770,167772160~25056304,192937984~14736989,210325964~283338,210854655~53271761,264241152~31353192,297795584~7201936,306184192~23819574,331350016~15772466,348127232~12006751,366341667~44700087] 2024-04-07T17:37:23.817 INFO:tasks.workunit.client.0.smithi100.stdout:from 0 to 17 diff [2329752~5775845,12468100~16076499,29778841~737427,31769017~13613313,45857929~10848066,62264039~1975360,66861224~16545045,85466334~5069656,92999623~10564968,106500403~13616197,121501610~12475984,136270708~22564048,160337176~7153146,168236274~8367316,180237750~12590714,194102759~7364680,202420372~5254601,210325964~283338,210854655~53271761,267179572~28414772,298036890~6960630,309308954~1620369,314393585~15610181,332944411~11089603,345811290~1311192,350406459~6863082,360049192~84791,366341667~7420413,374580519~13389528,388412399~3633438,392232408~14655911,408516200~2525554] 2024-04-07T17:37:23.817 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~5775845,12468100~16076499,29778841~737427,31769017~13613313,45857929~10848066,62264039~1975360,66861224~16545045,85466334~5069656,92999623~10564968,106500403~13616197,121501610~12475984,136270708~22564048,160337176~7153146,168236274~8367316,180237750~12590714,194102759~7364680,202420372~5254601,210325964~283338,210854655~53271761,267179572~28414772,298036890~6960630,309308954~1620369,314393585~15610181,332944411~11089603,345811290~1311192,350406459~6863082,360049192~84791,366341667~7420413,374580519~13389528,388412399~3633438,392232408~14655911,408516200~2525554] 2024-04-07T17:37:23.825 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~8105597,8388608~20155991,29360128~27345867,58720256~24686013,83886080~6649910,92274688~11289903,104857600~29119994,134217728~24617028,159383552~8106770,167772160~25056304,192937984~14736989,210325964~283338,210854655~53271761,264241152~31353192,297795584~7201936,306184192~23819574,331350016~15772466,348127232~12006751,366341667~44700087] 2024-04-07T17:37:23.825 INFO:tasks.workunit.client.0.smithi100.stdout:from 0 to 18 diff [2329752~9105991,12468100~16076499,29778841~737427,31769017~13613313,45857929~10848066,58033882~7969874,66861224~16545045,85466334~5069656,92999623~10564968,106500403~13616197,121501610~12475984,136270708~22564048,160337176~7153146,168236274~8367316,180237750~12590714,194102759~7364680,202420372~5254601,210147535~53978881,267179572~28414772,298036890~6960630,309308954~1620369,314393585~15610181,332944411~11089603,345811290~1311192,350406459~6863082,360049192~84791,366341667~7420413,374580519~17465318,392232408~14655911,408516200~2525554] 2024-04-07T17:37:23.825 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~9105991,12468100~16076499,29778841~737427,31769017~13613313,45857929~10848066,58033882~7969874,66861224~16545045,85466334~5069656,92999623~10564968,106500403~13616197,121501610~12475984,136270708~22564048,160337176~7153146,168236274~8367316,180237750~12590714,194102759~7364680,202420372~5254601,210147535~53978881,267179572~28414772,298036890~6960630,309308954~1620369,314393585~15610181,332944411~11089603,345811290~1311192,350406459~6863082,360049192~84791,366341667~7420413,374580519~17465318,392232408~14655911,408516200~2525554] 2024-04-07T17:37:23.836 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~28544599,29360128~54046141,83886080~6649910,92274688~11289903,104857600~29119994,134217728~24617028,159383552~8106770,167772160~25056304,192937984~14736989,210147535~53978881,264241152~31353192,297795584~7201936,306184192~23819574,331350016~15772466,348127232~12006751,366341667~44700087] 2024-04-07T17:37:23.837 INFO:tasks.workunit.client.0.smithi100.stdout:from 0 to 19 diff [2329752~26214847,29778841~737427,31769017~13613313,45857929~20145827,66861224~16545045,85466334~5069656,92999623~10564968,106500403~13616197,121501610~12475984,136270708~22564048,160337176~7153146,168236274~8367316,180237750~12590714,194102759~7364680,202420372~5254601,210147535~53978881,267179572~28414772,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350406459~6863082,360049192~84791,366341667~7420413,374580519~17465318,392232408~14655911,408516200~2525554] 2024-04-07T17:37:23.837 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~26214847,29778841~737427,31769017~13613313,45857929~20145827,66861224~16545045,85466334~5069656,92999623~10564968,106500403~13616197,121501610~12475984,136270708~22564048,160337176~7153146,168236274~8367316,180237750~12590714,194102759~7364680,202420372~5254601,210147535~53978881,267179572~28414772,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350406459~6863082,360049192~84791,366341667~7420413,374580519~17465318,392232408~14655911,408516200~2525554] 2024-04-07T17:37:23.844 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~28544599,29360128~54046141,83886080~6649910,92274688~11289903,104857600~29119994,134217728~24617028,159383552~8106770,167772160~25056304,192937984~14736989,210147535~53978881,264241152~31353192,297795584~7201936,306184192~23819574,331350016~15772466,348127232~12006751,366341667~44700087] 2024-04-07T17:37:23.844 INFO:tasks.workunit.client.0.smithi100.stdout:from 1 to 2 diff [25445275~3099324,108048350~5929155,146434166~11449972,180237750~3536884,186904555~1491543,194866296~6601143,210875736~10984504] 2024-04-07T17:37:23.844 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [25445275~3099324,108048350~5929155,146434166~11449972,180237750~3536884,186904555~1491543,194866296~6601143,210875736~10984504] 2024-04-07T17:37:23.853 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [25445275~3099324,104857600~9119905,142606336~15277802,176160768~7613866,185165240~3230858,194866296~6601143,210875736~10984504] 2024-04-07T17:37:23.853 INFO:tasks.workunit.client.0.smithi100.stdout:from 1 to 3 diff [20375589~3479656,25445275~3099324,81710240~1015114,108048350~5929155,143151737~1037788,146434166~12400590,165813830~1137640,180237750~3536884,185860175~6968289,194866296~6601143,203328670~4346303,210875736~10984504,243012066~4719194,276663656~6008883] 2024-04-07T17:37:23.854 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [20375589~3479656,25445275~3099324,81710240~1015114,108048350~5929155,143151737~1037788,146434166~12400590,165813830~1137640,180237750~3536884,185860175~6968289,194866296~6601143,203328670~4346303,210875736~10984504,243012066~4719194,276663656~6008883] 2024-04-07T17:37:23.861 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [16777216~7078029,25445275~3099324,79691776~3033578,104857600~9119905,142606336~16228420,163577856~3373614,176160768~7613866,185165240~7663224,194866296~12808677,210875736~10984504,239075328~8655932,276663656~6008883] 2024-04-07T17:37:23.861 INFO:tasks.workunit.client.0.smithi100.stdout:from 1 to 4 diff [20375589~3479656,25445275~3099324,29778841~737427,81710240~1015114,107619787~6357718,143151737~15683019,165813830~1137640,173876238~1467374,180237750~3536884,185860175~6968289,194866296~6601143,203328670~4346303,210875736~10984504,243012066~4719194,253764583~4193497,276663656~7979173,309308954~1620369,360049192~84791,388412399~2413139] 2024-04-07T17:37:23.861 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [20375589~3479656,25445275~3099324,29778841~737427,81710240~1015114,107619787~6357718,143151737~15683019,165813830~1137640,173876238~1467374,180237750~3536884,185860175~6968289,194866296~6601143,203328670~4346303,210875736~10984504,243012066~4719194,253764583~4193497,276663656~7979173,309308954~1620369,360049192~84791,388412399~2413139] 2024-04-07T17:37:23.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:37:23 smithi144 bash[66654]: cluster 2024-04-07T17:37:22.478093+0000 mgr.y (mgr.25175) 2460 : cluster [DBG] pgmap v4045: 137 pgs: 137 active+clean; 811 MiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 249 KiB/s rd, 57 MiB/s wr, 390 op/s 2024-04-07T17:37:23.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:37:23 smithi144 bash[66654]: cluster 2024-04-07T17:37:22.571685+0000 mon.a (mon.0) 3884 : cluster [DBG] osdmap e2011: 8 total, 8 up, 8 in 2024-04-07T17:37:23.868 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [16777216~7078029,25445275~3099324,29360128~1156140,79691776~3033578,104857600~9119905,142606336~16228420,163577856~3373614,171966464~3377148,176160768~7613866,185165240~7663224,194866296~12808677,210875736~10984504,239075328~8655932,253764583~4193497,276663656~7979173,306184192~4745131,356844263~3289720,385875968~4949570] 2024-04-07T17:37:23.868 INFO:tasks.workunit.client.0.smithi100.stdout:from 1 to 5 diff [20375589~3479656,25445275~3099324,29778841~737427,81710240~1015114,107619787~6357718,140416111~18418645,160337176~6614294,173876238~1467374,180237750~3536884,185860175~6968289,194866296~6601143,203328670~4346303,210875736~10984504,243012066~7805483,253764583~4193497,276663656~7979173,301491055~2008066,309308954~1620369,326964255~3039511,350891716~1451525,360049192~84791,374607324~4374722,388412399~2413139] 2024-04-07T17:37:23.868 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [20375589~3479656,25445275~3099324,29778841~737427,81710240~1015114,107619787~6357718,140416111~18418645,160337176~6614294,173876238~1467374,180237750~3536884,185860175~6968289,194866296~6601143,203328670~4346303,210875736~10984504,243012066~7805483,253764583~4193497,276663656~7979173,301491055~2008066,309308954~1620369,326964255~3039511,350891716~1451525,360049192~84791,374607324~4374722,388412399~2413139] 2024-04-07T17:37:23.875 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [16777216~7078029,25445275~3099324,29360128~1156140,79691776~3033578,104857600~9119905,138412032~20422724,159383552~7567918,171966464~3377148,176160768~7613866,185165240~7663224,194866296~12808677,210875736~10984504,239075328~11742221,253764583~4193497,276663656~7979173,297795584~5703537,306184192~4745131,326506747~3497019,350891716~1451525,356844263~3289720,373293056~5688990,385875968~4949570] 2024-04-07T17:37:23.875 INFO:tasks.workunit.client.0.smithi100.stdout:from 1 to 6 diff [19480873~9063726,29778841~737427,81710240~1015114,107619787~6357718,140416111~18418645,160337176~7153146,173876238~1467374,180237750~3536884,185860175~6968289,194866296~6601143,203328670~4346303,210325964~283338,210875736~10984504,243012066~7906095,253764583~10361833,273160535~11482294,292398584~2871045,301491055~2008066,309308954~1620369,326964255~3039511,337086692~6947322,350891716~1451525,360049192~84791,374607324~4374722,388412399~2413139] 2024-04-07T17:37:23.875 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [19480873~9063726,29778841~737427,81710240~1015114,107619787~6357718,140416111~18418645,160337176~7153146,173876238~1467374,180237750~3536884,185860175~6968289,194866296~6601143,203328670~4346303,210325964~283338,210875736~10984504,243012066~7906095,253764583~10361833,273160535~11482294,292398584~2871045,301491055~2008066,309308954~1620369,326964255~3039511,337086692~6947322,350891716~1451525,360049192~84791,374607324~4374722,388412399~2413139] 2024-04-07T17:37:23.884 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [16777216~11767383,29360128~1156140,79691776~3033578,104857600~9119905,138412032~20422724,159383552~8106770,171966464~3377148,176160768~7613866,185165240~7663224,194866296~12808677,210325964~283338,210875736~10984504,239075328~11842833,253764583~10361833,273160535~11482294,289406976~5862653,297795584~5703537,306184192~4745131,326506747~3497019,337086692~6947322,350891716~1451525,356844263~3289720,373293056~5688990,385875968~4949570] 2024-04-07T17:37:23.884 INFO:tasks.workunit.client.0.smithi100.stdout:from 1 to 7 diff [19480873~9063726,29778841~737427,49566880~3318481,66861224~7774919,81710240~1015114,107619787~6357718,140416111~18418645,160337176~7153146,173876238~1467374,180237750~12590714,194866296~6601143,202420372~5254601,210325964~283338,210854655~11005585,240886088~1607107,243012066~7906095,253764583~10361833,273160535~11482294,292398584~2871045,301491055~2008066,309308954~1620369,326964255~3039511,337086692~6947322,345811290~1311192,350891716~1451525,360049192~84791,367170792~4497984,374580519~4455211,388412399~2413139] 2024-04-07T17:37:23.884 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [19480873~9063726,29778841~737427,49566880~3318481,66861224~7774919,81710240~1015114,107619787~6357718,140416111~18418645,160337176~7153146,173876238~1467374,180237750~12590714,194866296~6601143,202420372~5254601,210325964~283338,210854655~11005585,240886088~1607107,243012066~7906095,253764583~10361833,273160535~11482294,292398584~2871045,301491055~2008066,309308954~1620369,326964255~3039511,337086692~6947322,345811290~1311192,350891716~1451525,360049192~84791,367170792~4497984,374580519~4455211,388412399~2413139] 2024-04-07T17:37:23.892 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [16777216~11767383,29360128~1156140,46137344~6748017,66861224~7774919,79691776~3033578,104857600~9119905,138412032~20422724,159383552~8106770,171966464~3377148,176160768~16667696,194866296~12808677,210325964~283338,210854655~11005585,239075328~11842833,253764583~10361833,273160535~11482294,289406976~5862653,297795584~5703537,306184192~4745131,326506747~3497019,337086692~10035790,350891716~1451525,356844263~3289720,367170792~4497984,373293056~5742674,385875968~4949570] 2024-04-07T17:37:23.892 INFO:tasks.workunit.client.0.smithi100.stdout:from 1 to 8 diff [19480873~9063726,29778841~737427,49566880~3318481,66861224~7774919,81710240~1015114,92999623~7613429,107619787~6357718,128030792~5946802,140416111~18418645,160337176~7153146,173876238~1467374,180237750~12590714,194866296~6601143,202420372~5254601,210325964~283338,210854655~11005585,235153608~5013034,240886088~1607107,243012066~7906095,253228895~10897521,273160535~11482294,292398584~2871045,300063155~4934365,309308954~1620369,326964255~3039511,337086692~6947322,345811290~1311192,350891716~1451525,360049192~84791,367170792~4497984,374580519~4455211,388412399~2413139] 2024-04-07T17:37:23.892 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [19480873~9063726,29778841~737427,49566880~3318481,66861224~7774919,81710240~1015114,92999623~7613429,107619787~6357718,128030792~5946802,140416111~18418645,160337176~7153146,173876238~1467374,180237750~12590714,194866296~6601143,202420372~5254601,210325964~283338,210854655~11005585,235153608~5013034,240886088~1607107,243012066~7906095,253228895~10897521,273160535~11482294,292398584~2871045,300063155~4934365,309308954~1620369,326964255~3039511,337086692~6947322,345811290~1311192,350891716~1451525,360049192~84791,367170792~4497984,374580519~4455211,388412399~2413139] 2024-04-07T17:37:23.899 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [16777216~11767383,29360128~1156140,46137344~6748017,66861224~7774919,79691776~3033578,92274688~8338364,104857600~9119905,128030792~5946802,138412032~20422724,159383552~8106770,171966464~3377148,176160768~16667696,194866296~12808677,210325964~283338,210854655~11005585,234881024~16037137,253228895~10897521,273160535~11482294,289406976~5862653,297795584~7201936,306184192~4745131,326506747~3497019,337086692~10035790,350891716~1451525,356844263~3289720,367170792~4497984,373293056~5742674,385875968~4949570] 2024-04-07T17:37:23.900 INFO:tasks.workunit.client.0.smithi100.stdout:from 1 to 9 diff [7275769~186849,19095670~9448929,29778841~737427,31769017~7139161,49566880~3318481,66861224~7774919,81710240~1015114,92999623~8522076,107619787~6357718,116025937~3770559,128030792~5946802,140416111~18418645,160337176~7153146,173876238~1467374,180237750~12590714,194866296~6601143,202420372~5254601,210325964~283338,210854655~11005585,230813446~9353196,240886088~1607107,243012066~21114350,273160535~13315830,292398584~2871045,300063155~4934365,309308954~1620369,326964255~3039511,337086692~6947322,345811290~1311192,350891716~1451525,360049192~84791,367170792~4497984,374580519~4455211,388412399~2413139] 2024-04-07T17:37:23.900 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [7275769~186849,19095670~9448929,29778841~737427,31769017~7139161,49566880~3318481,66861224~7774919,81710240~1015114,92999623~8522076,107619787~6357718,116025937~3770559,128030792~5946802,140416111~18418645,160337176~7153146,173876238~1467374,180237750~12590714,194866296~6601143,202420372~5254601,210325964~283338,210854655~11005585,230813446~9353196,240886088~1607107,243012066~21114350,273160535~13315830,292398584~2871045,300063155~4934365,309308954~1620369,326964255~3039511,337086692~6947322,345811290~1311192,350891716~1451525,360049192~84791,367170792~4497984,374580519~4455211,388412399~2413139] 2024-04-07T17:37:23.907 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [4194304~3268314,16777216~11767383,29360128~9548050,46137344~6748017,66861224~7774919,79691776~3033578,92274688~9247011,104857600~14938896,128030792~5946802,138412032~20422724,159383552~8106770,171966464~3377148,176160768~16667696,194866296~12808677,210325964~283338,210854655~11005585,230686720~33439696,273160535~13315830,289406976~5862653,297795584~7201936,306184192~4745131,326506747~3497019,337086692~10035790,350891716~1451525,356844263~3289720,367170792~4497984,373293056~5742674,385875968~4949570] 2024-04-07T17:37:23.907 INFO:tasks.workunit.client.0.smithi100.stdout:from 1 to 10 diff [2974197~5131400,19095670~9448929,29778841~737427,31769017~7139161,49566880~3318481,66861224~7774919,81710240~1015114,92999623~8522076,106500403~7477102,116025937~3770559,121501610~12475984,140416111~18418645,160337176~7153146,173876238~1467374,180237750~12590714,194866296~6601143,202420372~5254601,210325964~283338,210854655~11005585,230813446~33312970,273160535~13315830,287123044~8146585,300063155~4934365,309308954~1620369,323691380~1672817,326964255~3039511,337086692~6947322,345811290~1311192,350891716~1451525,360049192~84791,366341667~7420413,374580519~4455211,388412399~2413139] 2024-04-07T17:37:23.908 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2974197~5131400,19095670~9448929,29778841~737427,31769017~7139161,49566880~3318481,66861224~7774919,81710240~1015114,92999623~8522076,106500403~7477102,116025937~3770559,121501610~12475984,140416111~18418645,160337176~7153146,173876238~1467374,180237750~12590714,194866296~6601143,202420372~5254601,210325964~283338,210854655~11005585,230813446~33312970,273160535~13315830,287123044~8146585,300063155~4934365,309308954~1620369,323691380~1672817,326964255~3039511,337086692~6947322,345811290~1311192,350891716~1451525,360049192~84791,366341667~7420413,374580519~4455211,388412399~2413139] 2024-04-07T17:37:23.915 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~8105597,16777216~11767383,29360128~9548050,46137344~6748017,66861224~7774919,79691776~3033578,92274688~9247011,104857600~29119994,138412032~20422724,159383552~8106770,171966464~3377148,176160768~16667696,194866296~12808677,210325964~283338,210854655~11005585,230686720~33439696,273160535~22109094,297795584~7201936,306184192~4745131,323691380~1672817,326506747~3497019,337086692~10035790,350891716~1451525,356844263~3289720,366341667~12694063,385875968~4949570] 2024-04-07T17:37:23.915 INFO:tasks.workunit.client.0.smithi100.stdout:from 1 to 11 diff [2974197~5131400,19095670~9448929,29778841~737427,31769017~10175249,49566880~3318481,62264039~1975360,66861224~7774919,78193873~4609655,92999623~8522076,106500403~7477102,116025937~3770559,121501610~12475984,140416111~18418645,160337176~7153146,173876238~1467374,180237750~12590714,194866296~6601143,202420372~5254601,210325964~283338,210854655~11005585,222539835~1385676,225244127~38882289,272896915~13579450,287123044~8146585,300063155~4934365,309308954~1620369,323691380~1672817,326964255~3039511,337086692~6947322,345811290~1311192,350891716~1451525,360049192~84791,366341667~7420413,374580519~10318590,388412399~3633438,398418332~7708261] 2024-04-07T17:37:23.915 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2974197~5131400,19095670~9448929,29778841~737427,31769017~10175249,49566880~3318481,62264039~1975360,66861224~7774919,78193873~4609655,92999623~8522076,106500403~7477102,116025937~3770559,121501610~12475984,140416111~18418645,160337176~7153146,173876238~1467374,180237750~12590714,194866296~6601143,202420372~5254601,210325964~283338,210854655~11005585,222539835~1385676,225244127~38882289,272896915~13579450,287123044~8146585,300063155~4934365,309308954~1620369,323691380~1672817,326964255~3039511,337086692~6947322,345811290~1311192,350891716~1451525,360049192~84791,366341667~7420413,374580519~10318590,388412399~3633438,398418332~7708261] 2024-04-07T17:37:23.923 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~8105597,16777216~11767383,29360128~12584138,46137344~6748017,58720256~5519143,66861224~7774919,75497472~7306056,92274688~9247011,104857600~29119994,138412032~20422724,159383552~8106770,171966464~3377148,176160768~16667696,194866296~12808677,210325964~283338,210854655~11005585,222298112~41828304,272896915~22372714,297795584~7201936,306184192~4745131,323691380~1672817,326506747~3497019,337086692~10035790,350891716~1451525,356844263~3289720,366341667~18557442,385875968~6169869,398418332~7708261] 2024-04-07T17:37:23.923 INFO:tasks.workunit.client.0.smithi100.stdout:from 1 to 12 diff [2974197~5131400,19095670~9448929,29778841~737427,31769017~13613313,45857929~8483438,62264039~1975360,66861224~7774919,78193873~4609655,92999623~10564968,106500403~13616197,121501610~12475984,140416111~18418645,160337176~7153146,173876238~1467374,180237750~12590714,194866296~6601143,202420372~5254601,210325964~283338,210854655~11005585,222539835~1385676,225244127~38882289,272896915~13579450,287123044~8471300,298036890~6960630,309308954~1620369,323691380~6312386,337086692~6947322,345811290~1311192,350891716~1451525,360049192~84791,366341667~7420413,374580519~10318590,388412399~3633438,398418332~7708261] 2024-04-07T17:37:23.923 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2974197~5131400,19095670~9448929,29778841~737427,31769017~13613313,45857929~8483438,62264039~1975360,66861224~7774919,78193873~4609655,92999623~10564968,106500403~13616197,121501610~12475984,140416111~18418645,160337176~7153146,173876238~1467374,180237750~12590714,194866296~6601143,202420372~5254601,210325964~283338,210854655~11005585,222539835~1385676,225244127~38882289,272896915~13579450,287123044~8471300,298036890~6960630,309308954~1620369,323691380~6312386,337086692~6947322,345811290~1311192,350891716~1451525,360049192~84791,366341667~7420413,374580519~10318590,388412399~3633438,398418332~7708261] 2024-04-07T17:37:23.930 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~8105597,16777216~11767383,29360128~24981239,58720256~5519143,66861224~7774919,75497472~7306056,92274688~11289903,104857600~29119994,138412032~20422724,159383552~8106770,171966464~3377148,176160768~16667696,194866296~12808677,210325964~283338,210854655~11005585,222298112~41828304,272896915~22697429,297795584~7201936,306184192~4745131,323691380~6312386,337086692~10035790,350891716~1451525,356844263~3289720,366341667~18557442,385875968~6169869,398418332~7708261] 2024-04-07T17:37:23.930 INFO:tasks.workunit.client.0.smithi100.stdout:from 1 to 13 diff [2974197~5131400,19095670~9448929,29778841~737427,31769017~13613313,45857929~8483438,62264039~1975360,66861224~10262977,78193873~4609655,92999623~10564968,106500403~13616197,121501610~12475984,136270708~2853668,140416111~18418645,160337176~7153146,173876238~1467374,180237750~12590714,194866296~6601143,202420372~5254601,210325964~283338,210854655~11005585,222539835~1385676,225244127~38882289,267179572~19296793,287123044~8471300,298036890~6960630,309308954~1620369,323691380~6312386,337086692~6947322,345811290~1311192,350891716~1451525,360049192~84791,366341667~7420413,374580519~10318590,388412399~3633438,394880591~2187459,398418332~7708261,408516200~2525554] 2024-04-07T17:37:23.930 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2974197~5131400,19095670~9448929,29778841~737427,31769017~13613313,45857929~8483438,62264039~1975360,66861224~10262977,78193873~4609655,92999623~10564968,106500403~13616197,121501610~12475984,136270708~2853668,140416111~18418645,160337176~7153146,173876238~1467374,180237750~12590714,194866296~6601143,202420372~5254601,210325964~283338,210854655~11005585,222539835~1385676,225244127~38882289,267179572~19296793,287123044~8471300,298036890~6960630,309308954~1620369,323691380~6312386,337086692~6947322,345811290~1311192,350891716~1451525,360049192~84791,366341667~7420413,374580519~10318590,388412399~3633438,394880591~2187459,398418332~7708261,408516200~2525554] 2024-04-07T17:37:23.938 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~8105597,16777216~11767383,29360128~24981239,58720256~5519143,66861224~15942304,92274688~11289903,104857600~29119994,134217728~24617028,159383552~8106770,171966464~3377148,176160768~16667696,194866296~12808677,210325964~283338,210854655~11005585,222298112~41828304,264241152~31353192,297795584~7201936,306184192~4745131,323691380~6312386,337086692~10035790,350891716~1451525,356844263~3289720,366341667~18557442,385875968~6169869,394880591~2187459,398418332~7708261,406847488~4194266] 2024-04-07T17:37:23.938 INFO:tasks.workunit.client.0.smithi100.stdout:from 1 to 14 diff [2329752~5775845,18121228~10423371,29778841~737427,31769017~13613313,45857929~8483438,62264039~1975360,66861224~10262977,78193873~4609655,92999623~10564968,106500403~13616197,121501610~12475984,136270708~2853668,140416111~18418645,160337176~7153146,173876238~1467374,180237750~12590714,194866296~6601143,202420372~5254601,210325964~283338,210854655~13070856,225244127~38882289,267179572~19296793,287123044~8471300,298036890~6960630,309308954~1620369,318962927~11040839,337086692~6947322,345811290~1311192,350891716~1451525,360049192~84791,366341667~7420413,374580519~10318590,388412399~3633438,394880591~2187459,398418332~7708261,408516200~2525554] 2024-04-07T17:37:23.938 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~5775845,18121228~10423371,29778841~737427,31769017~13613313,45857929~8483438,62264039~1975360,66861224~10262977,78193873~4609655,92999623~10564968,106500403~13616197,121501610~12475984,136270708~2853668,140416111~18418645,160337176~7153146,173876238~1467374,180237750~12590714,194866296~6601143,202420372~5254601,210325964~283338,210854655~13070856,225244127~38882289,267179572~19296793,287123044~8471300,298036890~6960630,309308954~1620369,318962927~11040839,337086692~6947322,345811290~1311192,350891716~1451525,360049192~84791,366341667~7420413,374580519~10318590,388412399~3633438,394880591~2187459,398418332~7708261,408516200~2525554] 2024-04-07T17:37:23.947 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~8105597,16777216~11767383,29360128~24981239,58720256~5519143,66861224~15942304,92274688~11289903,104857600~29119994,134217728~24617028,159383552~8106770,171966464~3377148,176160768~16667696,194866296~12808677,210325964~283338,210854655~53271761,264241152~31353192,297795584~7201936,306184192~4745131,318962927~11040839,337086692~10035790,350891716~1451525,356844263~3289720,366341667~18557442,385875968~6169869,394880591~2187459,398418332~7708261,406847488~4194266] 2024-04-07T17:37:23.947 INFO:tasks.workunit.client.0.smithi100.stdout:from 1 to 15 diff [2329752~5775845,18121228~10423371,29778841~737427,31769017~13613313,45857929~8483438,62264039~1975360,66861224~16545045,92999623~10564968,106500403~13616197,121501610~12475984,136270708~2853668,140416111~18418645,160337176~7153146,168236274~8367316,180237750~12590714,194866296~6601143,202420372~5254601,210325964~283338,210854655~13756087,225244127~38882289,267179572~19296793,287123044~8471300,298036890~6960630,309308954~1620369,314393585~15610181,337086692~6947322,345811290~1311192,350891716~1451525,360049192~84791,366341667~7420413,374580519~10318590,388412399~3633438,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:23.947 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~5775845,18121228~10423371,29778841~737427,31769017~13613313,45857929~8483438,62264039~1975360,66861224~16545045,92999623~10564968,106500403~13616197,121501610~12475984,136270708~2853668,140416111~18418645,160337176~7153146,168236274~8367316,180237750~12590714,194866296~6601143,202420372~5254601,210325964~283338,210854655~13756087,225244127~38882289,267179572~19296793,287123044~8471300,298036890~6960630,309308954~1620369,314393585~15610181,337086692~6947322,345811290~1311192,350891716~1451525,360049192~84791,366341667~7420413,374580519~10318590,388412399~3633438,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:23.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:37:23 smithi100 bash[76202]: cluster 2024-04-07T17:37:22.478093+0000 mgr.y (mgr.25175) 2460 : cluster [DBG] pgmap v4045: 137 pgs: 137 active+clean; 811 MiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 249 KiB/s rd, 57 MiB/s wr, 390 op/s 2024-04-07T17:37:23.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:37:23 smithi100 bash[76202]: cluster 2024-04-07T17:37:22.571685+0000 mon.a (mon.0) 3884 : cluster [DBG] osdmap e2011: 8 total, 8 up, 8 in 2024-04-07T17:37:23.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:37:23 smithi100 bash[74942]: cluster 2024-04-07T17:37:22.478093+0000 mgr.y (mgr.25175) 2460 : cluster [DBG] pgmap v4045: 137 pgs: 137 active+clean; 811 MiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 249 KiB/s rd, 57 MiB/s wr, 390 op/s 2024-04-07T17:37:23.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:37:23 smithi100 bash[74942]: cluster 2024-04-07T17:37:22.571685+0000 mon.a (mon.0) 3884 : cluster [DBG] osdmap e2011: 8 total, 8 up, 8 in 2024-04-07T17:37:23.955 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~8105597,16777216~11767383,29360128~24981239,58720256~5519143,66861224~16545045,92274688~11289903,104857600~29119994,134217728~24617028,159383552~8106770,167772160~25056304,194866296~12808677,210325964~283338,210854655~53271761,264241152~31353192,297795584~7201936,306184192~23819574,337086692~10035790,350891716~1451525,356844263~3289720,366341667~18557442,385875968~6169869,394880591~2187459,398418332~12623422] 2024-04-07T17:37:23.955 INFO:tasks.workunit.client.0.smithi100.stdout:from 1 to 16 diff [2329752~5775845,14933611~2822456,18121228~10423371,29778841~737427,31769017~13613313,45857929~8483438,62264039~1975360,66861224~16545045,92999623~10564968,106500403~13616197,121501610~12475984,136270708~2853668,140416111~18418645,160337176~7153146,168236274~8367316,180237750~12590714,194866296~6601143,202420372~5254601,210325964~283338,210854655~13756087,225244127~38882289,267179572~28414772,298036890~6960630,309308954~1620369,314393585~15610181,332944411~11089603,345811290~1311192,350891716~1451525,360049192~84791,366341667~7420413,374580519~13389528,388412399~3633438,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:23.955 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~5775845,14933611~2822456,18121228~10423371,29778841~737427,31769017~13613313,45857929~8483438,62264039~1975360,66861224~16545045,92999623~10564968,106500403~13616197,121501610~12475984,136270708~2853668,140416111~18418645,160337176~7153146,168236274~8367316,180237750~12590714,194866296~6601143,202420372~5254601,210325964~283338,210854655~13756087,225244127~38882289,267179572~28414772,298036890~6960630,309308954~1620369,314393585~15610181,332944411~11089603,345811290~1311192,350891716~1451525,360049192~84791,366341667~7420413,374580519~13389528,388412399~3633438,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:23.962 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~8105597,12582912~15961687,29360128~24981239,58720256~5519143,66861224~16545045,92274688~11289903,104857600~29119994,134217728~24617028,159383552~8106770,167772160~25056304,194866296~12808677,210325964~283338,210854655~53271761,264241152~31353192,297795584~7201936,306184192~23819574,331350016~15772466,350891716~1451525,356844263~3289720,366341667~25704170,394880591~2187459,398418332~12623422] 2024-04-07T17:37:23.962 INFO:tasks.workunit.client.0.smithi100.stdout:from 1 to 17 diff [2329752~5775845,12468100~16076499,29778841~737427,31769017~13613313,45857929~8483438,62264039~1975360,66861224~16545045,92999623~10564968,106500403~13616197,121501610~12475984,136270708~22564048,160337176~7153146,168236274~8367316,180237750~12590714,194866296~6601143,202420372~5254601,210325964~283338,210854655~53271761,267179572~28414772,298036890~6960630,309308954~1620369,314393585~15610181,332944411~11089603,345811290~1311192,350891716~1451525,355806290~1463251,360049192~84791,366341667~7420413,374580519~13389528,388412399~3633438,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:23.962 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~5775845,12468100~16076499,29778841~737427,31769017~13613313,45857929~8483438,62264039~1975360,66861224~16545045,92999623~10564968,106500403~13616197,121501610~12475984,136270708~22564048,160337176~7153146,168236274~8367316,180237750~12590714,194866296~6601143,202420372~5254601,210325964~283338,210854655~53271761,267179572~28414772,298036890~6960630,309308954~1620369,314393585~15610181,332944411~11089603,345811290~1311192,350891716~1451525,355806290~1463251,360049192~84791,366341667~7420413,374580519~13389528,388412399~3633438,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:23.970 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~8105597,8388608~20155991,29360128~24981239,58720256~5519143,66861224~16545045,92274688~11289903,104857600~29119994,134217728~24617028,159383552~8106770,167772160~25056304,194866296~12808677,210325964~283338,210854655~53271761,264241152~31353192,297795584~7201936,306184192~23819574,331350016~15772466,350891716~1451525,355806290~4327693,366341667~25704170,394880591~2187459,398418332~12623422] 2024-04-07T17:37:23.970 INFO:tasks.workunit.client.0.smithi100.stdout:from 1 to 18 diff [2329752~9105991,12468100~16076499,29778841~737427,31769017~13613313,45857929~8483438,58033882~7969874,66861224~16545045,92999623~10564968,106500403~13616197,121501610~12475984,136270708~22564048,160337176~7153146,168236274~8367316,180237750~12590714,194866296~6601143,202420372~5254601,210147535~53978881,267179572~28414772,298036890~6960630,309308954~1620369,314393585~15610181,332944411~11089603,345811290~1311192,350891716~1451525,355806290~1463251,360049192~84791,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:23.970 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~9105991,12468100~16076499,29778841~737427,31769017~13613313,45857929~8483438,58033882~7969874,66861224~16545045,92999623~10564968,106500403~13616197,121501610~12475984,136270708~22564048,160337176~7153146,168236274~8367316,180237750~12590714,194866296~6601143,202420372~5254601,210147535~53978881,267179572~28414772,298036890~6960630,309308954~1620369,314393585~15610181,332944411~11089603,345811290~1311192,350891716~1451525,355806290~1463251,360049192~84791,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:23.978 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~28544599,29360128~24981239,56705995~9297761,66861224~16545045,92274688~11289903,104857600~29119994,134217728~24617028,159383552~8106770,167772160~25056304,194866296~12808677,210147535~53978881,264241152~31353192,297795584~7201936,306184192~23819574,331350016~15772466,350891716~1451525,355806290~4327693,366341667~25704170,394880591~2187459,398418332~12623422] 2024-04-07T17:37:23.979 INFO:tasks.workunit.client.0.smithi100.stdout:from 1 to 19 diff [2329752~26214847,29778841~737427,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~13616197,121501610~12475984,136270708~22564048,160337176~7153146,168236274~8367316,180237750~12590714,194866296~6601143,202420372~5254601,210147535~53978881,267179572~28414772,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350891716~1451525,355806290~1463251,360049192~84791,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:23.979 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~26214847,29778841~737427,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~13616197,121501610~12475984,136270708~22564048,160337176~7153146,168236274~8367316,180237750~12590714,194866296~6601143,202420372~5254601,210147535~53978881,267179572~28414772,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350891716~1451525,355806290~1463251,360049192~84791,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:23.986 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~28544599,29360128~24981239,55380210~10623546,66861224~16545045,92274688~11289903,104857600~29119994,134217728~24617028,159383552~8106770,167772160~25056304,194866296~12808677,210147535~53978881,264241152~31353192,297795584~7201936,306184192~23819574,331350016~15772466,350891716~1451525,355806290~4327693,366341667~25704170,394880591~2187459,398418332~12623422] 2024-04-07T17:37:23.987 INFO:tasks.workunit.client.0.smithi100.stdout:from 2 to 3 diff [20375589~3479656,81710240~1015114,143151737~1037788,156912913~1921843,165813830~1137640,185860175~6968289,203328670~4346303,216364348~218501,243012066~4719194,276663656~6008883] 2024-04-07T17:37:23.987 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [20375589~3479656,81710240~1015114,143151737~1037788,156912913~1921843,165813830~1137640,185860175~6968289,203328670~4346303,216364348~218501,243012066~4719194,276663656~6008883] 2024-04-07T17:37:23.994 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [16777216~7078029,79691776~3033578,143151737~1037788,156912913~1921843,163577856~3373614,185860175~6968289,201467439~6207534,216364348~218501,239075328~8655932,276663656~6008883] 2024-04-07T17:37:23.994 INFO:tasks.workunit.client.0.smithi100.stdout:from 2 to 4 diff [20375589~3479656,29778841~737427,81710240~1015114,107619787~2326498,143151737~6849621,154799337~811894,156912913~1921843,165813830~1137640,173876238~1467374,185860175~6968289,203328670~4346303,216364348~218501,243012066~4719194,253764583~4193497,276663656~7979173,309308954~1620369,360049192~84791,388412399~2413139] 2024-04-07T17:37:23.994 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [20375589~3479656,29778841~737427,81710240~1015114,107619787~2326498,143151737~6849621,154799337~811894,156912913~1921843,165813830~1137640,173876238~1467374,185860175~6968289,203328670~4346303,216364348~218501,243012066~4719194,253764583~4193497,276663656~7979173,309308954~1620369,360049192~84791,388412399~2413139] 2024-04-07T17:37:24.001 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [16777216~7078029,29360128~1156140,79691776~3033578,107619787~2326498,143151737~6849621,154799337~811894,156912913~1921843,163577856~3373614,171966464~3377148,185860175~6968289,201467439~6207534,216364348~218501,239075328~8655932,253764583~4193497,276663656~7979173,306184192~4745131,356844263~3289720,385875968~4949570] 2024-04-07T17:37:24.002 INFO:tasks.workunit.client.0.smithi100.stdout:from 2 to 5 diff [20375589~3479656,29778841~737427,81710240~1015114,107619787~2326498,140416111~9585247,154799337~811894,156912913~1921843,160337176~6614294,173876238~1467374,185860175~6968289,203328670~4346303,216361490~3081245,243012066~7805483,253764583~4193497,276663656~7979173,301491055~2008066,309308954~1620369,326964255~3039511,350891716~1451525,360049192~84791,374607324~4374722,388412399~2413139] 2024-04-07T17:37:24.002 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [20375589~3479656,29778841~737427,81710240~1015114,107619787~2326498,140416111~9585247,154799337~811894,156912913~1921843,160337176~6614294,173876238~1467374,185860175~6968289,203328670~4346303,216361490~3081245,243012066~7805483,253764583~4193497,276663656~7979173,301491055~2008066,309308954~1620369,326964255~3039511,350891716~1451525,360049192~84791,374607324~4374722,388412399~2413139] 2024-04-07T17:37:24.009 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [16777216~7078029,29360128~1156140,79691776~3033578,107619787~2326498,138412032~11589326,154799337~811894,156912913~1921843,159383552~7567918,171966464~3377148,185860175~6968289,201467439~6207534,216361490~3081245,239075328~11742221,253764583~4193497,276663656~7979173,297795584~5703537,306184192~4745131,326506747~3497019,350891716~1451525,356844263~3289720,373293056~5688990,385875968~4949570] 2024-04-07T17:37:24.009 INFO:tasks.workunit.client.0.smithi100.stdout:from 2 to 6 diff [19480873~7079576,29778841~737427,81710240~1015114,107619787~2326498,140416111~9585247,154799337~811894,156912913~1921843,160337176~7153146,173876238~1467374,182034135~1518423,185860175~6968289,196727604~2108791,203328670~4346303,210325964~283338,216361490~3081245,243012066~7906095,253764583~10361833,273160535~11482294,292398584~2871045,301491055~2008066,309308954~1620369,326964255~3039511,337086692~6947322,350891716~1451525,360049192~84791,374607324~4374722,388412399~2413139] 2024-04-07T17:37:24.009 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [19480873~7079576,29778841~737427,81710240~1015114,107619787~2326498,140416111~9585247,154799337~811894,156912913~1921843,160337176~7153146,173876238~1467374,182034135~1518423,185860175~6968289,196727604~2108791,203328670~4346303,210325964~283338,216361490~3081245,243012066~7906095,253764583~10361833,273160535~11482294,292398584~2871045,301491055~2008066,309308954~1620369,326964255~3039511,337086692~6947322,350891716~1451525,360049192~84791,374607324~4374722,388412399~2413139] 2024-04-07T17:37:24.016 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [16777216~9783233,29360128~1156140,79691776~3033578,107619787~2326498,138412032~11589326,154799337~811894,156912913~1921843,159383552~8106770,171966464~3377148,182034135~1518423,185860175~6968289,196727604~2108791,201467439~6207534,210325964~283338,216361490~3081245,239075328~11842833,253764583~10361833,273160535~11482294,289406976~5862653,297795584~5703537,306184192~4745131,326506747~3497019,337086692~6947322,350891716~1451525,356844263~3289720,373293056~5688990,385875968~4949570] 2024-04-07T17:37:24.016 INFO:tasks.workunit.client.0.smithi100.stdout:from 2 to 7 diff [19480873~7079576,29778841~737427,49566880~3318481,66861224~7774919,81710240~1015114,107619787~2326498,140416111~9585247,154799337~811894,156912913~1921843,160337176~7153146,173876238~1467374,182034135~10794329,196727604~2108791,202420372~5254601,210325964~283338,210854655~8588080,240886088~1607107,243012066~7906095,253764583~10361833,273160535~11482294,292398584~2871045,301491055~2008066,309308954~1620369,326964255~3039511,337086692~6947322,345811290~1311192,350891716~1451525,360049192~84791,367170792~4497984,374580519~4455211,388412399~2413139] 2024-04-07T17:37:24.016 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [19480873~7079576,29778841~737427,49566880~3318481,66861224~7774919,81710240~1015114,107619787~2326498,140416111~9585247,154799337~811894,156912913~1921843,160337176~7153146,173876238~1467374,182034135~10794329,196727604~2108791,202420372~5254601,210325964~283338,210854655~8588080,240886088~1607107,243012066~7906095,253764583~10361833,273160535~11482294,292398584~2871045,301491055~2008066,309308954~1620369,326964255~3039511,337086692~6947322,345811290~1311192,350891716~1451525,360049192~84791,367170792~4497984,374580519~4455211,388412399~2413139] 2024-04-07T17:37:24.024 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [16777216~9783233,29360128~1156140,46137344~6748017,66861224~7774919,79691776~3033578,107619787~2326498,138412032~11589326,154799337~811894,156912913~1921843,159383552~8106770,171966464~3377148,182034135~10794329,196727604~2108791,201467439~6207534,210325964~283338,210854655~8588080,239075328~11842833,253764583~10361833,273160535~11482294,289406976~5862653,297795584~5703537,306184192~4745131,326506747~3497019,337086692~10035790,350891716~1451525,356844263~3289720,367170792~4497984,373293056~5742674,385875968~4949570] 2024-04-07T17:37:24.024 INFO:tasks.workunit.client.0.smithi100.stdout:from 2 to 8 diff [19480873~7079576,27909341~266890,29778841~737427,49566880~3318481,66861224~7774919,81710240~1015114,92999623~7613429,107619787~2326498,128030792~5946802,140416111~12016873,154799337~4035419,160337176~7153146,173876238~1467374,182034135~10794329,196727604~2108791,202420372~5254601,210325964~283338,210854655~8588080,235153608~5013034,240886088~1607107,243012066~7906095,253228895~10897521,273160535~11482294,292398584~2871045,300063155~4934365,309308954~1620369,326964255~3039511,337086692~6947322,345811290~1311192,350891716~1451525,360049192~84791,367170792~4497984,374580519~4455211,388412399~2413139] 2024-04-07T17:37:24.024 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [19480873~7079576,27909341~266890,29778841~737427,49566880~3318481,66861224~7774919,81710240~1015114,92999623~7613429,107619787~2326498,128030792~5946802,140416111~12016873,154799337~4035419,160337176~7153146,173876238~1467374,182034135~10794329,196727604~2108791,202420372~5254601,210325964~283338,210854655~8588080,235153608~5013034,240886088~1607107,243012066~7906095,253228895~10897521,273160535~11482294,292398584~2871045,300063155~4934365,309308954~1620369,326964255~3039511,337086692~6947322,345811290~1311192,350891716~1451525,360049192~84791,367170792~4497984,374580519~4455211,388412399~2413139] 2024-04-07T17:37:24.032 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [16777216~9783233,27909341~266890,29360128~1156140,46137344~6748017,66861224~7774919,79691776~3033578,92274688~8338364,107619787~2326498,128030792~5946802,138412032~14020952,154799337~4035419,159383552~8106770,171966464~3377148,182034135~10794329,196727604~2108791,201467439~6207534,210325964~283338,210854655~8588080,234881024~16037137,253228895~10897521,273160535~11482294,289406976~5862653,297795584~7201936,306184192~4745131,326506747~3497019,337086692~10035790,350891716~1451525,356844263~3289720,367170792~4497984,373293056~5742674,385875968~4949570] 2024-04-07T17:37:24.032 INFO:tasks.workunit.client.0.smithi100.stdout:from 2 to 9 diff [7275769~186849,19095670~7464779,27909341~266890,29778841~737427,31769017~7139161,49566880~3318481,66861224~7774919,81710240~1015114,92999623~8522076,107619787~2326498,116025937~3770559,128030792~5946802,140416111~12016873,154799337~4035419,160337176~7153146,173876238~1467374,182034135~10794329,196727604~2108791,202420372~5254601,210325964~283338,210854655~8588080,230813446~9353196,240886088~1607107,243012066~21114350,273160535~13315830,292398584~2871045,300063155~4934365,309308954~1620369,326964255~3039511,337086692~6947322,345811290~1311192,350891716~1451525,360049192~84791,367170792~4497984,374580519~4455211,388412399~2413139] 2024-04-07T17:37:24.033 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [7275769~186849,19095670~7464779,27909341~266890,29778841~737427,31769017~7139161,49566880~3318481,66861224~7774919,81710240~1015114,92999623~8522076,107619787~2326498,116025937~3770559,128030792~5946802,140416111~12016873,154799337~4035419,160337176~7153146,173876238~1467374,182034135~10794329,196727604~2108791,202420372~5254601,210325964~283338,210854655~8588080,230813446~9353196,240886088~1607107,243012066~21114350,273160535~13315830,292398584~2871045,300063155~4934365,309308954~1620369,326964255~3039511,337086692~6947322,345811290~1311192,350891716~1451525,360049192~84791,367170792~4497984,374580519~4455211,388412399~2413139] 2024-04-07T17:37:24.040 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [4194304~3268314,16777216~9783233,27909341~266890,29360128~9548050,46137344~6748017,66861224~7774919,79691776~3033578,92274688~9247011,107619787~2326498,113977505~5818991,128030792~5946802,138412032~14020952,154799337~4035419,159383552~8106770,171966464~3377148,182034135~10794329,196727604~2108791,201467439~6207534,210325964~283338,210854655~8588080,230686720~33439696,273160535~13315830,289406976~5862653,297795584~7201936,306184192~4745131,326506747~3497019,337086692~10035790,350891716~1451525,356844263~3289720,367170792~4497984,373293056~5742674,385875968~4949570] 2024-04-07T17:37:24.040 INFO:tasks.workunit.client.0.smithi100.stdout:from 2 to 10 diff [2974197~5131400,19095670~7464779,27431447~878329,29778841~737427,31769017~7139161,49566880~3318481,66861224~7774919,81710240~1015114,92999623~8522076,106500403~5285199,116025937~3770559,121501610~12475984,140416111~12016873,154799337~4035419,160337176~7153146,173876238~1467374,182034135~10794329,196727604~2108791,202420372~5254601,210325964~283338,210854655~8588080,230813446~33312970,273160535~13315830,287123044~8146585,300063155~4934365,309308954~1620369,323691380~1672817,326964255~3039511,337086692~6947322,345811290~1311192,350891716~1451525,360049192~84791,366341667~7420413,374580519~4455211,388412399~2413139] 2024-04-07T17:37:24.041 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2974197~5131400,19095670~7464779,27431447~878329,29778841~737427,31769017~7139161,49566880~3318481,66861224~7774919,81710240~1015114,92999623~8522076,106500403~5285199,116025937~3770559,121501610~12475984,140416111~12016873,154799337~4035419,160337176~7153146,173876238~1467374,182034135~10794329,196727604~2108791,202420372~5254601,210325964~283338,210854655~8588080,230813446~33312970,273160535~13315830,287123044~8146585,300063155~4934365,309308954~1620369,323691380~1672817,326964255~3039511,337086692~6947322,345811290~1311192,350891716~1451525,360049192~84791,366341667~7420413,374580519~4455211,388412399~2413139] 2024-04-07T17:37:24.048 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~8105597,16777216~9783233,27431447~878329,29360128~9548050,46137344~6748017,66861224~7774919,79691776~3033578,92274688~9247011,106500403~5285199,113977505~20000089,138412032~14020952,154799337~4035419,159383552~8106770,171966464~3377148,182034135~10794329,196727604~2108791,201467439~6207534,210325964~283338,210854655~8588080,230686720~33439696,273160535~22109094,297795584~7201936,306184192~4745131,323691380~1672817,326506747~3497019,337086692~10035790,350891716~1451525,356844263~3289720,366341667~12694063,385875968~4949570] 2024-04-07T17:37:24.048 INFO:tasks.workunit.client.0.smithi100.stdout:from 2 to 11 diff [2974197~5131400,19095670~7464779,27431447~878329,29778841~737427,31769017~10175249,49566880~3318481,62264039~1975360,66861224~7774919,78193873~4609655,92999623~8522076,106500403~5285199,116025937~3770559,121501610~12475984,140416111~12016873,154799337~4035419,160337176~7153146,173876238~1467374,182034135~10794329,196727604~2108791,202420372~5254601,210325964~283338,210854655~9339103,222539835~1385676,225244127~38882289,272896915~13579450,287123044~8146585,300063155~4934365,309308954~1620369,323691380~1672817,326964255~3039511,337086692~6947322,345811290~1311192,350891716~1451525,360049192~84791,366341667~7420413,374580519~10318590,388412399~3633438,398418332~7708261] 2024-04-07T17:37:24.048 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2974197~5131400,19095670~7464779,27431447~878329,29778841~737427,31769017~10175249,49566880~3318481,62264039~1975360,66861224~7774919,78193873~4609655,92999623~8522076,106500403~5285199,116025937~3770559,121501610~12475984,140416111~12016873,154799337~4035419,160337176~7153146,173876238~1467374,182034135~10794329,196727604~2108791,202420372~5254601,210325964~283338,210854655~9339103,222539835~1385676,225244127~38882289,272896915~13579450,287123044~8146585,300063155~4934365,309308954~1620369,323691380~1672817,326964255~3039511,337086692~6947322,345811290~1311192,350891716~1451525,360049192~84791,366341667~7420413,374580519~10318590,388412399~3633438,398418332~7708261] 2024-04-07T17:37:24.056 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~8105597,16777216~9783233,27431447~878329,29360128~12584138,46137344~6748017,58720256~5519143,66861224~7774919,75497472~7306056,92274688~9247011,106500403~5285199,113977505~20000089,138412032~14020952,154799337~4035419,159383552~8106770,171966464~3377148,182034135~10794329,196727604~2108791,201467439~6207534,210325964~283338,210854655~9339103,222298112~41828304,272896915~22372714,297795584~7201936,306184192~4745131,323691380~1672817,326506747~3497019,337086692~10035790,350891716~1451525,356844263~3289720,366341667~18557442,385875968~6169869,398418332~7708261] 2024-04-07T17:37:24.056 INFO:tasks.workunit.client.0.smithi100.stdout:from 2 to 12 diff [2974197~5131400,19095670~7464779,27431447~878329,29778841~737427,31769017~13613313,45857929~8483438,62264039~1975360,66861224~7774919,78193873~4609655,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,140416111~12016873,154799337~4035419,160337176~7153146,173876238~1467374,182034135~10794329,196727604~2108791,202420372~5254601,210325964~283338,210854655~9339103,222539835~1385676,225244127~38882289,272896915~13579450,287123044~8471300,298036890~6960630,309308954~1620369,323691380~6312386,337086692~6947322,345811290~1311192,350891716~1451525,360049192~84791,366341667~7420413,374580519~10318590,388412399~3633438,398418332~7708261] 2024-04-07T17:37:24.056 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2974197~5131400,19095670~7464779,27431447~878329,29778841~737427,31769017~13613313,45857929~8483438,62264039~1975360,66861224~7774919,78193873~4609655,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,140416111~12016873,154799337~4035419,160337176~7153146,173876238~1467374,182034135~10794329,196727604~2108791,202420372~5254601,210325964~283338,210854655~9339103,222539835~1385676,225244127~38882289,272896915~13579450,287123044~8471300,298036890~6960630,309308954~1620369,323691380~6312386,337086692~6947322,345811290~1311192,350891716~1451525,360049192~84791,366341667~7420413,374580519~10318590,388412399~3633438,398418332~7708261] 2024-04-07T17:37:24.063 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~8105597,16777216~9783233,27431447~878329,29360128~24981239,58720256~5519143,66861224~7774919,75497472~7306056,92274688~11289903,106500403~5285199,112275487~21702107,138412032~14020952,154799337~4035419,159383552~8106770,171966464~3377148,182034135~10794329,196727604~2108791,201467439~6207534,210325964~283338,210854655~9339103,222298112~41828304,272896915~22697429,297795584~7201936,306184192~4745131,323691380~6312386,337086692~10035790,350891716~1451525,356844263~3289720,366341667~18557442,385875968~6169869,398418332~7708261] 2024-04-07T17:37:24.063 INFO:tasks.workunit.client.0.smithi100.stdout:from 2 to 13 diff [2974197~5131400,19095670~7464779,27329013~980763,29778841~737427,31769017~13613313,45857929~8483438,62264039~1975360,66861224~10262977,78193873~4609655,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~2853668,140416111~12016873,154799337~4035419,160337176~7153146,173876238~1467374,182034135~10794329,196727604~2108791,202420372~5254601,210325964~283338,210854655~9339103,222539835~1385676,225244127~38882289,267179572~19296793,287123044~8471300,298036890~6960630,309308954~1620369,323691380~6312386,337086692~6947322,345811290~1311192,350891716~1451525,360049192~84791,366341667~7420413,374580519~10318590,388412399~3633438,394880591~2187459,398418332~7708261,408516200~2525554] 2024-04-07T17:37:24.064 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2974197~5131400,19095670~7464779,27329013~980763,29778841~737427,31769017~13613313,45857929~8483438,62264039~1975360,66861224~10262977,78193873~4609655,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~2853668,140416111~12016873,154799337~4035419,160337176~7153146,173876238~1467374,182034135~10794329,196727604~2108791,202420372~5254601,210325964~283338,210854655~9339103,222539835~1385676,225244127~38882289,267179572~19296793,287123044~8471300,298036890~6960630,309308954~1620369,323691380~6312386,337086692~6947322,345811290~1311192,350891716~1451525,360049192~84791,366341667~7420413,374580519~10318590,388412399~3633438,394880591~2187459,398418332~7708261,408516200~2525554] 2024-04-07T17:37:24.072 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~8105597,16777216~9783233,27329013~980763,29360128~24981239,58720256~5519143,66861224~15942304,92274688~11289903,106500403~5285199,112275487~21702107,134217728~18215256,154799337~4035419,159383552~8106770,171966464~3377148,182034135~10794329,196727604~2108791,201467439~6207534,210325964~283338,210854655~9339103,222298112~41828304,264241152~31353192,297795584~7201936,306184192~4745131,323691380~6312386,337086692~10035790,350891716~1451525,356844263~3289720,366341667~18557442,385875968~6169869,394880591~2187459,398418332~7708261,406847488~4194266] 2024-04-07T17:37:24.072 INFO:tasks.workunit.client.0.smithi100.stdout:from 2 to 14 diff [2329752~5775845,18121228~8439221,27329013~980763,29778841~737427,31769017~13613313,45857929~8483438,62264039~1975360,66861224~10262977,78193873~4609655,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~2853668,140416111~12016873,154799337~4035419,160337176~7153146,173876238~1467374,182034135~10794329,196727604~2108791,202420372~5254601,210325964~283338,210854655~13070856,225244127~38882289,267179572~19296793,287123044~8471300,298036890~6960630,309308954~1620369,318962927~11040839,337086692~6947322,345811290~1311192,350891716~1451525,360049192~84791,366341667~7420413,374580519~10318590,388412399~3633438,394880591~2187459,398418332~7708261,408516200~2525554] 2024-04-07T17:37:24.072 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~5775845,18121228~8439221,27329013~980763,29778841~737427,31769017~13613313,45857929~8483438,62264039~1975360,66861224~10262977,78193873~4609655,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~2853668,140416111~12016873,154799337~4035419,160337176~7153146,173876238~1467374,182034135~10794329,196727604~2108791,202420372~5254601,210325964~283338,210854655~13070856,225244127~38882289,267179572~19296793,287123044~8471300,298036890~6960630,309308954~1620369,318962927~11040839,337086692~6947322,345811290~1311192,350891716~1451525,360049192~84791,366341667~7420413,374580519~10318590,388412399~3633438,394880591~2187459,398418332~7708261,408516200~2525554] 2024-04-07T17:37:24.080 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~8105597,16777216~9783233,27329013~980763,29360128~24981239,58720256~5519143,66861224~15942304,92274688~11289903,106500403~5285199,112275487~21702107,134217728~18215256,154799337~4035419,159383552~8106770,171966464~3377148,182034135~10794329,196727604~2108791,201467439~6207534,210325964~283338,210854655~53271761,264241152~31353192,297795584~7201936,306184192~4745131,318962927~11040839,337086692~10035790,350891716~1451525,356844263~3289720,366341667~18557442,385875968~6169869,394880591~2187459,398418332~7708261,406847488~4194266] 2024-04-07T17:37:24.080 INFO:tasks.workunit.client.0.smithi100.stdout:from 2 to 15 diff [2329752~5775845,18121228~8439221,27329013~980763,29778841~737427,31769017~13613313,45857929~8483438,62264039~1975360,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~2853668,140416111~12016873,154799337~4035419,160337176~7153146,168236274~8367316,182034135~10794329,196727604~2108791,202420372~5254601,210325964~283338,210854655~13756087,225244127~38882289,267179572~19296793,287123044~8471300,298036890~6960630,309308954~1620369,314393585~15610181,337086692~6947322,345811290~1311192,350891716~1451525,360049192~84791,366341667~7420413,374580519~10318590,388412399~3633438,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:24.080 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~5775845,18121228~8439221,27329013~980763,29778841~737427,31769017~13613313,45857929~8483438,62264039~1975360,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~2853668,140416111~12016873,154799337~4035419,160337176~7153146,168236274~8367316,182034135~10794329,196727604~2108791,202420372~5254601,210325964~283338,210854655~13756087,225244127~38882289,267179572~19296793,287123044~8471300,298036890~6960630,309308954~1620369,314393585~15610181,337086692~6947322,345811290~1311192,350891716~1451525,360049192~84791,366341667~7420413,374580519~10318590,388412399~3633438,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:24.088 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~8105597,16777216~9783233,27329013~980763,29360128~24981239,58720256~5519143,66861224~16545045,92274688~11289903,106500403~5285199,112275487~21702107,134217728~18215256,154799337~4035419,159383552~8106770,167772160~8831430,182034135~10794329,196727604~2108791,201467439~6207534,210325964~283338,210854655~53271761,264241152~31353192,297795584~7201936,306184192~23819574,337086692~10035790,350891716~1451525,356844263~3289720,366341667~18557442,385875968~6169869,394880591~2187459,398418332~12623422] 2024-04-07T17:37:24.088 INFO:tasks.workunit.client.0.smithi100.stdout:from 2 to 16 diff [2329752~5775845,14933611~2822456,18121228~8439221,27329013~980763,29778841~737427,31769017~13613313,45857929~8483438,62264039~1975360,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~2853668,140416111~12016873,154799337~4035419,160337176~7153146,168236274~8367316,182034135~10794329,196727604~2108791,202420372~5254601,210325964~283338,210854655~13756087,225244127~38882289,267179572~28414772,298036890~6960630,309308954~1620369,314393585~15610181,332944411~11089603,345811290~1311192,350891716~1451525,360049192~84791,366341667~7420413,374580519~13389528,388412399~3633438,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:24.088 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~5775845,14933611~2822456,18121228~8439221,27329013~980763,29778841~737427,31769017~13613313,45857929~8483438,62264039~1975360,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~2853668,140416111~12016873,154799337~4035419,160337176~7153146,168236274~8367316,182034135~10794329,196727604~2108791,202420372~5254601,210325964~283338,210854655~13756087,225244127~38882289,267179572~28414772,298036890~6960630,309308954~1620369,314393585~15610181,332944411~11089603,345811290~1311192,350891716~1451525,360049192~84791,366341667~7420413,374580519~13389528,388412399~3633438,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:24.097 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~8105597,12582912~13977537,27329013~980763,29360128~24981239,58720256~5519143,66861224~16545045,92274688~11289903,106500403~5285199,112275487~21702107,134217728~18215256,154799337~4035419,159383552~8106770,167772160~8831430,182034135~10794329,196727604~2108791,201467439~6207534,210325964~283338,210854655~53271761,264241152~31353192,297795584~7201936,306184192~23819574,331350016~15772466,350891716~1451525,356844263~3289720,366341667~25704170,394880591~2187459,398418332~12623422] 2024-04-07T17:37:24.097 INFO:tasks.workunit.client.0.smithi100.stdout:from 2 to 17 diff [2329752~5775845,12468100~14092349,27329013~980763,29778841~737427,31769017~13613313,45857929~8483438,62264039~1975360,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~16162276,154799337~4035419,160337176~7153146,168236274~8367316,182034135~10794329,196727604~2108791,202420372~5254601,210325964~283338,210854655~53271761,267179572~28414772,298036890~6960630,309308954~1620369,314393585~15610181,332944411~11089603,345811290~1311192,350891716~1451525,355806290~1463251,360049192~84791,366341667~7420413,374580519~13389528,388412399~3633438,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:24.097 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~5775845,12468100~14092349,27329013~980763,29778841~737427,31769017~13613313,45857929~8483438,62264039~1975360,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~16162276,154799337~4035419,160337176~7153146,168236274~8367316,182034135~10794329,196727604~2108791,202420372~5254601,210325964~283338,210854655~53271761,267179572~28414772,298036890~6960630,309308954~1620369,314393585~15610181,332944411~11089603,345811290~1311192,350891716~1451525,355806290~1463251,360049192~84791,366341667~7420413,374580519~13389528,388412399~3633438,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:24.105 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~8105597,8388608~18171841,27329013~980763,29360128~24981239,58720256~5519143,66861224~16545045,92274688~11289903,106500403~5285199,112275487~21702107,134217728~18215256,154799337~4035419,159383552~8106770,167772160~8831430,182034135~10794329,196727604~2108791,201467439~6207534,210325964~283338,210854655~53271761,264241152~31353192,297795584~7201936,306184192~23819574,331350016~15772466,350891716~1451525,355806290~4327693,366341667~25704170,394880591~2187459,398418332~12623422] 2024-04-07T17:37:24.105 INFO:tasks.workunit.client.0.smithi100.stdout:from 2 to 18 diff [2329752~9105991,12468100~14092349,27329013~980763,29778841~737427,31769017~13613313,45857929~8483438,58033882~7969874,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~16162276,154799337~4035419,160337176~7153146,168236274~8367316,182034135~10794329,196727604~2108791,202420372~5254601,210147535~53978881,267179572~28414772,298036890~6960630,309308954~1620369,314393585~15610181,332944411~11089603,345811290~1311192,350891716~1451525,355806290~1463251,360049192~84791,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:24.105 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~9105991,12468100~14092349,27329013~980763,29778841~737427,31769017~13613313,45857929~8483438,58033882~7969874,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~16162276,154799337~4035419,160337176~7153146,168236274~8367316,182034135~10794329,196727604~2108791,202420372~5254601,210147535~53978881,267179572~28414772,298036890~6960630,309308954~1620369,314393585~15610181,332944411~11089603,345811290~1311192,350891716~1451525,355806290~1463251,360049192~84791,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:24.112 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~26560449,27329013~980763,29360128~24981239,56705995~9297761,66861224~16545045,92274688~11289903,106500403~5285199,112275487~21702107,134217728~18215256,154799337~4035419,159383552~8106770,167772160~8831430,182034135~10794329,196727604~2108791,201467439~6207534,210147535~53978881,264241152~31353192,297795584~7201936,306184192~23819574,331350016~15772466,350891716~1451525,355806290~4327693,366341667~25704170,394880591~2187459,398418332~12623422] 2024-04-07T17:37:24.112 INFO:tasks.workunit.client.0.smithi100.stdout:from 2 to 19 diff [2329752~24230697,27329013~980763,29778841~737427,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~17685068,154799337~4035419,160337176~7153146,168236274~8367316,182034135~10794329,196727604~2108791,202420372~5254601,210147535~53978881,267179572~28414772,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350891716~1451525,355806290~1463251,360049192~84791,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:24.112 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~24230697,27329013~980763,29778841~737427,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~17685068,154799337~4035419,160337176~7153146,168236274~8367316,182034135~10794329,196727604~2108791,202420372~5254601,210147535~53978881,267179572~28414772,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350891716~1451525,355806290~1463251,360049192~84791,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:24.120 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~26560449,27329013~980763,29360128~24981239,55380210~10623546,66861224~16545045,92274688~11289903,106500403~5285199,112275487~21702107,134217728~19738048,154799337~4035419,159383552~8106770,167772160~8831430,182034135~10794329,196727604~2108791,201467439~6207534,210147535~53978881,264241152~31353192,297795584~7201936,306184192~23819574,331350016~15772466,350891716~1451525,355806290~4327693,366341667~25704170,394880591~2187459,398418332~12623422] 2024-04-07T17:37:24.120 INFO:tasks.workunit.client.0.smithi100.stdout:from 3 to 4 diff [29778841~737427,107619787~2326498,143194295~6807063,154799337~811894,173876238~1467374,253764583~4193497,279625672~5017157,309308954~1620369,360049192~84791,388412399~2413139] 2024-04-07T17:37:24.120 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [29778841~737427,107619787~2326498,143194295~6807063,154799337~811894,173876238~1467374,253764583~4193497,279625672~5017157,309308954~1620369,360049192~84791,388412399~2413139] 2024-04-07T17:37:24.128 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [29360128~1156140,107619787~2326498,143194295~6807063,154799337~811894,171966464~3377148,253764583~4193497,279625672~5017157,306184192~4745131,356844263~3289720,385875968~4949570] 2024-04-07T17:37:24.128 INFO:tasks.workunit.client.0.smithi100.stdout:from 3 to 5 diff [29778841~737427,107619787~2326498,140416111~9585247,154799337~811894,160337176~6458877,173876238~1467374,216361490~3081245,244611976~6205573,253764583~4193497,279625672~5017157,301491055~2008066,309308954~1620369,326964255~3039511,350891716~1451525,360049192~84791,374607324~4374722,388412399~2413139] 2024-04-07T17:37:24.128 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [29778841~737427,107619787~2326498,140416111~9585247,154799337~811894,160337176~6458877,173876238~1467374,216361490~3081245,244611976~6205573,253764583~4193497,279625672~5017157,301491055~2008066,309308954~1620369,326964255~3039511,350891716~1451525,360049192~84791,374607324~4374722,388412399~2413139] 2024-04-07T17:37:24.136 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [29360128~1156140,107619787~2326498,138412032~11589326,154799337~811894,159383552~7412501,171966464~3377148,216361490~3081245,244611976~6205573,253764583~4193497,279625672~5017157,297795584~5703537,306184192~4745131,326506747~3497019,350891716~1451525,356844263~3289720,373293056~5688990,385875968~4949570] 2024-04-07T17:37:24.136 INFO:tasks.workunit.client.0.smithi100.stdout:from 3 to 6 diff [19480873~7079576,29778841~737427,107619787~2326498,140416111~9585247,154799337~811894,160337176~7153146,173876238~1467374,182034135~1518423,196727604~2108791,210325964~283338,216361490~3081245,244611976~6306185,253764583~10361833,273160535~11482294,292398584~2871045,301491055~2008066,309308954~1620369,326964255~3039511,337086692~6947322,350891716~1451525,360049192~84791,374607324~4374722,388412399~2413139] 2024-04-07T17:37:24.136 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [19480873~7079576,29778841~737427,107619787~2326498,140416111~9585247,154799337~811894,160337176~7153146,173876238~1467374,182034135~1518423,196727604~2108791,210325964~283338,216361490~3081245,244611976~6306185,253764583~10361833,273160535~11482294,292398584~2871045,301491055~2008066,309308954~1620369,326964255~3039511,337086692~6947322,350891716~1451525,360049192~84791,374607324~4374722,388412399~2413139] 2024-04-07T17:37:24.143 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [19480873~7079576,29360128~1156140,107619787~2326498,138412032~11589326,154799337~811894,159383552~8106770,171966464~3377148,182034135~1518423,196727604~2108791,210325964~283338,216361490~3081245,244611976~6306185,253764583~10361833,273160535~11482294,289406976~5862653,297795584~5703537,306184192~4745131,326506747~3497019,337086692~6947322,350891716~1451525,356844263~3289720,373293056~5688990,385875968~4949570] 2024-04-07T17:37:24.144 INFO:tasks.workunit.client.0.smithi100.stdout:from 3 to 7 diff [19480873~7079576,29778841~737427,49566880~3318481,66861224~7774919,107619787~2326498,140416111~9585247,154799337~811894,160337176~7153146,173876238~1467374,182034135~5872430,196727604~2108791,202420372~1435645,210325964~283338,210854655~8588080,240886088~1607107,244611976~6306185,253764583~10361833,273160535~11482294,292398584~2871045,301491055~2008066,309308954~1620369,326964255~3039511,337086692~6947322,345811290~1311192,350891716~1451525,360049192~84791,367170792~4497984,374580519~4455211,388412399~2413139] 2024-04-07T17:37:24.144 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [19480873~7079576,29778841~737427,49566880~3318481,66861224~7774919,107619787~2326498,140416111~9585247,154799337~811894,160337176~7153146,173876238~1467374,182034135~5872430,196727604~2108791,202420372~1435645,210325964~283338,210854655~8588080,240886088~1607107,244611976~6306185,253764583~10361833,273160535~11482294,292398584~2871045,301491055~2008066,309308954~1620369,326964255~3039511,337086692~6947322,345811290~1311192,350891716~1451525,360049192~84791,367170792~4497984,374580519~4455211,388412399~2413139] 2024-04-07T17:37:24.151 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [19480873~7079576,29360128~1156140,46137344~6748017,66861224~7774919,107619787~2326498,138412032~11589326,154799337~811894,159383552~8106770,171966464~3377148,182034135~5872430,196727604~2108791,202420372~1435645,210325964~283338,210854655~8588080,240886088~1607107,244611976~6306185,253764583~10361833,273160535~11482294,289406976~5862653,297795584~5703537,306184192~4745131,326506747~3497019,337086692~10035790,350891716~1451525,356844263~3289720,367170792~4497984,373293056~5742674,385875968~4949570] 2024-04-07T17:37:24.151 INFO:tasks.workunit.client.0.smithi100.stdout:from 3 to 8 diff [19480873~7079576,27909341~266890,29778841~737427,49566880~3318481,66861224~7774919,92999623~7613429,107619787~2326498,128030792~5946802,140416111~12016873,154799337~3192780,160337176~7153146,173876238~1467374,182034135~5872430,196727604~2108791,202420372~1435645,210325964~283338,210854655~8588080,235153608~5013034,240886088~1607107,244611976~6306185,253228895~10897521,273160535~11482294,292398584~2871045,300063155~4934365,309308954~1620369,326964255~3039511,337086692~6947322,345811290~1311192,350891716~1451525,360049192~84791,367170792~4497984,374580519~4455211,388412399~2413139] 2024-04-07T17:37:24.151 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [19480873~7079576,27909341~266890,29778841~737427,49566880~3318481,66861224~7774919,92999623~7613429,107619787~2326498,128030792~5946802,140416111~12016873,154799337~3192780,160337176~7153146,173876238~1467374,182034135~5872430,196727604~2108791,202420372~1435645,210325964~283338,210854655~8588080,235153608~5013034,240886088~1607107,244611976~6306185,253228895~10897521,273160535~11482294,292398584~2871045,300063155~4934365,309308954~1620369,326964255~3039511,337086692~6947322,345811290~1311192,350891716~1451525,360049192~84791,367170792~4497984,374580519~4455211,388412399~2413139] 2024-04-07T17:37:24.159 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [19480873~7079576,27909341~266890,29360128~1156140,46137344~6748017,66861224~7774919,92274688~8338364,107619787~2326498,128030792~5946802,138412032~14020952,154799337~3192780,159383552~8106770,171966464~3377148,182034135~5872430,196727604~2108791,202420372~1435645,210325964~283338,210854655~8588080,234881024~5285618,240886088~1607107,244611976~6306185,253228895~10897521,273160535~11482294,289406976~5862653,297795584~7201936,306184192~4745131,326506747~3497019,337086692~10035790,350891716~1451525,356844263~3289720,367170792~4497984,373293056~5742674,385875968~4949570] 2024-04-07T17:37:24.159 INFO:tasks.workunit.client.0.smithi100.stdout:from 3 to 9 diff [7275769~186849,19095670~7464779,27909341~266890,29778841~737427,31769017~7139161,49566880~3318481,66861224~7774919,92999623~8522076,107619787~2326498,116025937~3770559,128030792~5946802,140416111~12016873,154799337~3192780,160337176~7153146,173876238~1467374,182034135~7778892,196727604~2108791,202420372~1435645,210325964~283338,210854655~8588080,230813446~9353196,240886088~1607107,244611976~19514440,273160535~13315830,292398584~2871045,300063155~4934365,309308954~1620369,326964255~3039511,337086692~6947322,345811290~1311192,350891716~1451525,360049192~84791,367170792~4497984,374580519~4455211,388412399~2413139] 2024-04-07T17:37:24.159 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [7275769~186849,19095670~7464779,27909341~266890,29778841~737427,31769017~7139161,49566880~3318481,66861224~7774919,92999623~8522076,107619787~2326498,116025937~3770559,128030792~5946802,140416111~12016873,154799337~3192780,160337176~7153146,173876238~1467374,182034135~7778892,196727604~2108791,202420372~1435645,210325964~283338,210854655~8588080,230813446~9353196,240886088~1607107,244611976~19514440,273160535~13315830,292398584~2871045,300063155~4934365,309308954~1620369,326964255~3039511,337086692~6947322,345811290~1311192,350891716~1451525,360049192~84791,367170792~4497984,374580519~4455211,388412399~2413139] 2024-04-07T17:37:24.167 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [4194304~3268314,19095670~7464779,27909341~266890,29360128~9548050,46137344~6748017,66861224~7774919,92274688~9247011,107619787~2326498,113977505~5818991,128030792~5946802,138412032~14020952,154799337~3192780,159383552~8106770,171966464~3377148,182034135~7778892,196727604~2108791,202420372~1435645,210325964~283338,210854655~8588080,230686720~9479922,240886088~1607107,244611976~19514440,273160535~13315830,289406976~5862653,297795584~7201936,306184192~4745131,326506747~3497019,337086692~10035790,350891716~1451525,356844263~3289720,367170792~4497984,373293056~5742674,385875968~4949570] 2024-04-07T17:37:24.167 INFO:tasks.workunit.client.0.smithi100.stdout:from 3 to 10 diff [2974197~5131400,19095670~7464779,27431447~878329,29778841~737427,31769017~7139161,49566880~3318481,66861224~7774919,92999623~8522076,106500403~5285199,116025937~3770559,121501610~12475984,140416111~12016873,154799337~3192780,160337176~7153146,173876238~1467374,182034135~7778892,196727604~2108791,202420372~1435645,210325964~283338,210854655~8588080,230813446~13138536,244611976~19514440,273160535~13315830,287123044~8146585,300063155~4934365,309308954~1620369,323691380~1672817,326964255~3039511,337086692~6947322,345811290~1311192,350891716~1451525,360049192~84791,366341667~7420413,374580519~4455211,388412399~2413139] 2024-04-07T17:37:24.167 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2974197~5131400,19095670~7464779,27431447~878329,29778841~737427,31769017~7139161,49566880~3318481,66861224~7774919,92999623~8522076,106500403~5285199,116025937~3770559,121501610~12475984,140416111~12016873,154799337~3192780,160337176~7153146,173876238~1467374,182034135~7778892,196727604~2108791,202420372~1435645,210325964~283338,210854655~8588080,230813446~13138536,244611976~19514440,273160535~13315830,287123044~8146585,300063155~4934365,309308954~1620369,323691380~1672817,326964255~3039511,337086692~6947322,345811290~1311192,350891716~1451525,360049192~84791,366341667~7420413,374580519~4455211,388412399~2413139] 2024-04-07T17:37:24.175 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~8105597,19095670~7464779,27431447~878329,29360128~9548050,46137344~6748017,66861224~7774919,92274688~9247011,106500403~5285199,113977505~20000089,138412032~14020952,154799337~3192780,159383552~8106770,171966464~3377148,182034135~7778892,196727604~2108791,202420372~1435645,210325964~283338,210854655~8588080,230686720~13265262,244611976~19514440,273160535~22109094,297795584~7201936,306184192~4745131,323691380~1672817,326506747~3497019,337086692~10035790,350891716~1451525,356844263~3289720,366341667~12694063,385875968~4949570] 2024-04-07T17:37:24.175 INFO:tasks.workunit.client.0.smithi100.stdout:from 3 to 11 diff [2974197~5131400,19095670~7464779,27431447~878329,29778841~737427,31769017~10175249,49566880~3318481,62264039~1975360,66861224~7774919,78193873~4609655,92999623~8522076,106500403~5285199,116025937~3770559,121501610~12475984,140416111~12016873,154799337~3192780,160337176~7153146,173876238~1467374,182034135~7778892,196727604~2108791,202420372~1435645,210325964~283338,210854655~9339103,222539835~1385676,225244127~18707855,244611976~19514440,272896915~13579450,287123044~8146585,300063155~4934365,309308954~1620369,323691380~1672817,326964255~3039511,337086692~6947322,345811290~1311192,350891716~1451525,360049192~84791,366341667~7420413,374580519~10318590,388412399~3633438,398418332~7708261] 2024-04-07T17:37:24.175 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2974197~5131400,19095670~7464779,27431447~878329,29778841~737427,31769017~10175249,49566880~3318481,62264039~1975360,66861224~7774919,78193873~4609655,92999623~8522076,106500403~5285199,116025937~3770559,121501610~12475984,140416111~12016873,154799337~3192780,160337176~7153146,173876238~1467374,182034135~7778892,196727604~2108791,202420372~1435645,210325964~283338,210854655~9339103,222539835~1385676,225244127~18707855,244611976~19514440,272896915~13579450,287123044~8146585,300063155~4934365,309308954~1620369,323691380~1672817,326964255~3039511,337086692~6947322,345811290~1311192,350891716~1451525,360049192~84791,366341667~7420413,374580519~10318590,388412399~3633438,398418332~7708261] 2024-04-07T17:37:24.182 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~8105597,19095670~7464779,27431447~878329,29360128~12584138,46137344~6748017,58720256~5519143,66861224~7774919,75497472~7306056,92274688~9247011,106500403~5285199,113977505~20000089,138412032~14020952,154799337~3192780,159383552~8106770,171966464~3377148,182034135~7778892,196727604~2108791,202420372~1435645,210325964~283338,210854655~9339103,222298112~21653870,244611976~19514440,272896915~22372714,297795584~7201936,306184192~4745131,323691380~1672817,326506747~3497019,337086692~10035790,350891716~1451525,356844263~3289720,366341667~18557442,385875968~6169869,398418332~7708261] 2024-04-07T17:37:24.182 INFO:tasks.workunit.client.0.smithi100.stdout:from 3 to 12 diff [2974197~5131400,19095670~7464779,27431447~878329,29778841~737427,31769017~13613313,45857929~8483438,62264039~1975360,66861224~7774919,78193873~4609655,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,140416111~12016873,154799337~3192780,160337176~7153146,173876238~1467374,182034135~7778892,196727604~2108791,202420372~1435645,210325964~283338,210854655~9339103,222539835~1385676,225244127~18707855,244611976~19514440,272896915~13579450,287123044~8471300,298036890~6960630,309308954~1620369,323691380~6312386,337086692~6947322,345811290~1311192,350891716~1451525,360049192~84791,366341667~7420413,374580519~10318590,388412399~3633438,398418332~7708261] 2024-04-07T17:37:24.182 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2974197~5131400,19095670~7464779,27431447~878329,29778841~737427,31769017~13613313,45857929~8483438,62264039~1975360,66861224~7774919,78193873~4609655,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,140416111~12016873,154799337~3192780,160337176~7153146,173876238~1467374,182034135~7778892,196727604~2108791,202420372~1435645,210325964~283338,210854655~9339103,222539835~1385676,225244127~18707855,244611976~19514440,272896915~13579450,287123044~8471300,298036890~6960630,309308954~1620369,323691380~6312386,337086692~6947322,345811290~1311192,350891716~1451525,360049192~84791,366341667~7420413,374580519~10318590,388412399~3633438,398418332~7708261] 2024-04-07T17:37:24.190 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~8105597,19095670~7464779,27431447~878329,29360128~24981239,58720256~5519143,66861224~7774919,75497472~7306056,92274688~11289903,106500403~5285199,112275487~21702107,138412032~14020952,154799337~3192780,159383552~8106770,171966464~3377148,182034135~7778892,196727604~2108791,202420372~1435645,210325964~283338,210854655~9339103,222298112~21653870,244611976~19514440,272896915~22697429,297795584~7201936,306184192~4745131,323691380~6312386,337086692~10035790,350891716~1451525,356844263~3289720,366341667~18557442,385875968~6169869,398418332~7708261] 2024-04-07T17:37:24.190 INFO:tasks.workunit.client.0.smithi100.stdout:from 3 to 13 diff [2974197~5131400,19095670~7464779,27329013~980763,29778841~737427,31769017~13613313,45857929~8483438,62264039~1975360,66861224~10262977,78193873~4609655,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~2853668,140416111~12016873,154799337~3192780,160337176~7153146,173876238~1467374,182034135~7778892,196727604~2108791,202420372~1435645,210325964~283338,210854655~9339103,222539835~1385676,225244127~18707855,244611976~19514440,267179572~19296793,287123044~8471300,298036890~6960630,309308954~1620369,323691380~6312386,337086692~6947322,345811290~1311192,350891716~1451525,360049192~84791,366341667~7420413,374580519~10318590,388412399~3633438,394880591~2187459,398418332~7708261,408516200~2525554] 2024-04-07T17:37:24.190 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2974197~5131400,19095670~7464779,27329013~980763,29778841~737427,31769017~13613313,45857929~8483438,62264039~1975360,66861224~10262977,78193873~4609655,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~2853668,140416111~12016873,154799337~3192780,160337176~7153146,173876238~1467374,182034135~7778892,196727604~2108791,202420372~1435645,210325964~283338,210854655~9339103,222539835~1385676,225244127~18707855,244611976~19514440,267179572~19296793,287123044~8471300,298036890~6960630,309308954~1620369,323691380~6312386,337086692~6947322,345811290~1311192,350891716~1451525,360049192~84791,366341667~7420413,374580519~10318590,388412399~3633438,394880591~2187459,398418332~7708261,408516200~2525554] 2024-04-07T17:37:24.198 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~8105597,19095670~7464779,27329013~980763,29360128~24981239,58720256~5519143,66861224~15942304,92274688~11289903,106500403~5285199,112275487~21702107,134217728~18215256,154799337~3192780,159383552~8106770,171966464~3377148,182034135~7778892,196727604~2108791,202420372~1435645,210325964~283338,210854655~9339103,222298112~21653870,244611976~19514440,264241152~31353192,297795584~7201936,306184192~4745131,323691380~6312386,337086692~10035790,350891716~1451525,356844263~3289720,366341667~18557442,385875968~6169869,394880591~2187459,398418332~7708261,406847488~4194266] 2024-04-07T17:37:24.198 INFO:tasks.workunit.client.0.smithi100.stdout:from 3 to 14 diff [2329752~5775845,18121228~8439221,27329013~980763,29778841~737427,31769017~13613313,45857929~8483438,62264039~1975360,66861224~10262977,78193873~4609655,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~2853668,140416111~12016873,154799337~3192780,160337176~7153146,173876238~1467374,182034135~10303497,196727604~2108791,202420372~1435645,210325964~283338,210854655~13070856,225244127~18707855,244611976~19514440,267179572~19296793,287123044~8471300,298036890~6960630,309308954~1620369,318962927~11040839,337086692~6947322,345811290~1311192,350891716~1451525,360049192~84791,366341667~7420413,374580519~10318590,388412399~3633438,394880591~2187459,398418332~7708261,408516200~2525554] 2024-04-07T17:37:24.198 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~5775845,18121228~8439221,27329013~980763,29778841~737427,31769017~13613313,45857929~8483438,62264039~1975360,66861224~10262977,78193873~4609655,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~2853668,140416111~12016873,154799337~3192780,160337176~7153146,173876238~1467374,182034135~10303497,196727604~2108791,202420372~1435645,210325964~283338,210854655~13070856,225244127~18707855,244611976~19514440,267179572~19296793,287123044~8471300,298036890~6960630,309308954~1620369,318962927~11040839,337086692~6947322,345811290~1311192,350891716~1451525,360049192~84791,366341667~7420413,374580519~10318590,388412399~3633438,394880591~2187459,398418332~7708261,408516200~2525554] 2024-04-07T17:37:24.205 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~8105597,18121228~8439221,27329013~980763,29360128~24981239,58720256~5519143,66861224~15942304,92274688~11289903,106500403~5285199,112275487~21702107,134217728~18215256,154799337~3192780,159383552~8106770,171966464~3377148,182034135~10303497,196727604~2108791,202420372~1435645,210325964~283338,210854655~33097327,244611976~19514440,264241152~31353192,297795584~7201936,306184192~4745131,318962927~11040839,337086692~10035790,350891716~1451525,356844263~3289720,366341667~18557442,385875968~6169869,394880591~2187459,398418332~7708261,406847488~4194266] 2024-04-07T17:37:24.206 INFO:tasks.workunit.client.0.smithi100.stdout:from 3 to 15 diff [2329752~5775845,18121228~8439221,27329013~980763,29778841~737427,31769017~13613313,45857929~8483438,62264039~1975360,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~2853668,140416111~12016873,154799337~3192780,160337176~7153146,168236274~8367316,182034135~10303497,196727604~2108791,202420372~1435645,210325964~283338,210854655~13756087,225244127~38882289,267179572~19296793,287123044~8471300,298036890~6960630,309308954~1620369,314393585~15610181,337086692~6947322,345811290~1311192,350891716~1451525,360049192~84791,366341667~7420413,374580519~10318590,388412399~3633438,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:24.206 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~5775845,18121228~8439221,27329013~980763,29778841~737427,31769017~13613313,45857929~8483438,62264039~1975360,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~2853668,140416111~12016873,154799337~3192780,160337176~7153146,168236274~8367316,182034135~10303497,196727604~2108791,202420372~1435645,210325964~283338,210854655~13756087,225244127~38882289,267179572~19296793,287123044~8471300,298036890~6960630,309308954~1620369,314393585~15610181,337086692~6947322,345811290~1311192,350891716~1451525,360049192~84791,366341667~7420413,374580519~10318590,388412399~3633438,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:24.214 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~8105597,18121228~8439221,27329013~980763,29360128~24981239,58720256~5519143,66861224~16545045,92274688~11289903,106500403~5285199,112275487~21702107,134217728~18215256,154799337~3192780,159383552~8106770,167772160~8831430,182034135~10303497,196727604~2108791,202420372~1435645,210325964~283338,210854655~53271761,264241152~31353192,297795584~7201936,306184192~23819574,337086692~10035790,350891716~1451525,356844263~3289720,366341667~18557442,385875968~6169869,394880591~2187459,398418332~12623422] 2024-04-07T17:37:24.214 INFO:tasks.workunit.client.0.smithi100.stdout:from 3 to 16 diff [2329752~5775845,14933611~2822456,18121228~8439221,27329013~980763,29778841~737427,31769017~13613313,45857929~8483438,62264039~1975360,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~2853668,140416111~12016873,154799337~3192780,160337176~7153146,168236274~8367316,182034135~10303497,196727604~2108791,202420372~1435645,210325964~283338,210854655~13756087,225244127~38882289,267179572~28414772,298036890~6960630,309308954~1620369,314393585~15610181,332944411~11089603,345811290~1311192,350891716~1451525,360049192~84791,366341667~7420413,374580519~13389528,388412399~3633438,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:24.214 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~5775845,14933611~2822456,18121228~8439221,27329013~980763,29778841~737427,31769017~13613313,45857929~8483438,62264039~1975360,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~2853668,140416111~12016873,154799337~3192780,160337176~7153146,168236274~8367316,182034135~10303497,196727604~2108791,202420372~1435645,210325964~283338,210854655~13756087,225244127~38882289,267179572~28414772,298036890~6960630,309308954~1620369,314393585~15610181,332944411~11089603,345811290~1311192,350891716~1451525,360049192~84791,366341667~7420413,374580519~13389528,388412399~3633438,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:24.222 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~8105597,12582912~5173155,18121228~8439221,27329013~980763,29360128~24981239,58720256~5519143,66861224~16545045,92274688~11289903,106500403~5285199,112275487~21702107,134217728~18215256,154799337~3192780,159383552~8106770,167772160~8831430,182034135~10303497,196727604~2108791,202420372~1435645,210325964~283338,210854655~53271761,264241152~31353192,297795584~7201936,306184192~23819574,331350016~15772466,350891716~1451525,356844263~3289720,366341667~25704170,394880591~2187459,398418332~12623422] 2024-04-07T17:37:24.222 INFO:tasks.workunit.client.0.smithi100.stdout:from 3 to 17 diff [2329752~5775845,12468100~14092349,27329013~980763,29778841~737427,31769017~13613313,45857929~8483438,62264039~1975360,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~16162276,154799337~3192780,160337176~7153146,168236274~8367316,182034135~10303497,196727604~2108791,202420372~1435645,210325964~283338,210854655~53271761,267179572~28414772,298036890~6960630,309308954~1620369,314393585~15610181,332944411~11089603,345811290~1311192,350891716~1451525,355806290~1463251,360049192~84791,366341667~7420413,374580519~13389528,388412399~3633438,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:24.222 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~5775845,12468100~14092349,27329013~980763,29778841~737427,31769017~13613313,45857929~8483438,62264039~1975360,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~16162276,154799337~3192780,160337176~7153146,168236274~8367316,182034135~10303497,196727604~2108791,202420372~1435645,210325964~283338,210854655~53271761,267179572~28414772,298036890~6960630,309308954~1620369,314393585~15610181,332944411~11089603,345811290~1311192,350891716~1451525,355806290~1463251,360049192~84791,366341667~7420413,374580519~13389528,388412399~3633438,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:24.229 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~8105597,8388608~18171841,27329013~980763,29360128~24981239,58720256~5519143,66861224~16545045,92274688~11289903,106500403~5285199,112275487~21702107,134217728~18215256,154799337~3192780,159383552~8106770,167772160~8831430,182034135~10303497,196727604~2108791,202420372~1435645,210325964~283338,210854655~53271761,264241152~31353192,297795584~7201936,306184192~23819574,331350016~15772466,350891716~1451525,355806290~4327693,366341667~25704170,394880591~2187459,398418332~12623422] 2024-04-07T17:37:24.229 INFO:tasks.workunit.client.0.smithi100.stdout:from 3 to 18 diff [2329752~9105991,12468100~14092349,27329013~980763,29778841~737427,31769017~13613313,45857929~8483438,58033882~7969874,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~16162276,154799337~3192780,160337176~7153146,168236274~8367316,182034135~10303497,196727604~2108791,202420372~1435645,210147535~53978881,267179572~28414772,298036890~6960630,309308954~1620369,314393585~15610181,332944411~11089603,345811290~1311192,350891716~1451525,355806290~1463251,360049192~84791,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:24.229 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~9105991,12468100~14092349,27329013~980763,29778841~737427,31769017~13613313,45857929~8483438,58033882~7969874,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~16162276,154799337~3192780,160337176~7153146,168236274~8367316,182034135~10303497,196727604~2108791,202420372~1435645,210147535~53978881,267179572~28414772,298036890~6960630,309308954~1620369,314393585~15610181,332944411~11089603,345811290~1311192,350891716~1451525,355806290~1463251,360049192~84791,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:24.237 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~26560449,27329013~980763,29360128~24981239,56705995~9297761,66861224~16545045,92274688~11289903,106500403~5285199,112275487~21702107,134217728~18215256,154799337~3192780,159383552~8106770,167772160~8831430,182034135~10303497,196727604~2108791,202420372~1435645,210147535~53978881,264241152~31353192,297795584~7201936,306184192~23819574,331350016~15772466,350891716~1451525,355806290~4327693,366341667~25704170,394880591~2187459,398418332~12623422] 2024-04-07T17:37:24.237 INFO:tasks.workunit.client.0.smithi100.stdout:from 3 to 19 diff [2329752~24230697,27329013~980763,29778841~737427,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~17685068,154799337~3192780,160337176~7153146,168236274~8367316,182034135~10303497,196727604~2108791,202420372~1435645,210147535~53978881,267179572~28414772,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350891716~1451525,355806290~1463251,360049192~84791,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:24.237 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~24230697,27329013~980763,29778841~737427,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~17685068,154799337~3192780,160337176~7153146,168236274~8367316,182034135~10303497,196727604~2108791,202420372~1435645,210147535~53978881,267179572~28414772,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350891716~1451525,355806290~1463251,360049192~84791,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:24.246 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~26560449,27329013~980763,29360128~24981239,55380210~10623546,66861224~16545045,92274688~11289903,106500403~5285199,112275487~21702107,134217728~19738048,154799337~3192780,159383552~8106770,167772160~8831430,182034135~10303497,196727604~2108791,202420372~1435645,210147535~53978881,264241152~31353192,297795584~7201936,306184192~23819574,331350016~15772466,350891716~1451525,355806290~4327693,366341667~25704170,394880591~2187459,398418332~12623422] 2024-04-07T17:37:24.246 INFO:tasks.workunit.client.0.smithi100.stdout:from 4 to 5 diff [140416111~6992252,148109105~141733,160337176~6458877,216361490~3081245,244611976~6205573,301491055~2008066,326964255~3039511,350891716~1451525,374607324~4374722] 2024-04-07T17:37:24.246 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [140416111~6992252,148109105~141733,160337176~6458877,216361490~3081245,244611976~6205573,301491055~2008066,326964255~3039511,350891716~1451525,374607324~4374722] 2024-04-07T17:37:24.253 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [138412032~8996331,148109105~141733,159383552~7412501,216361490~3081245,244611976~6205573,297795584~5703537,326506747~3497019,350891716~1451525,373293056~5688990] 2024-04-07T17:37:24.253 INFO:tasks.workunit.client.0.smithi100.stdout:from 4 to 6 diff [19480873~7079576,140416111~6992252,148109105~141733,160337176~7153146,182034135~1518423,196727604~2108791,210325964~283338,216361490~3081245,244611976~6306185,256332667~7793749,273160535~7546831,292398584~2871045,301491055~2008066,326964255~3039511,337086692~6947322,350891716~1451525,374607324~4374722] 2024-04-07T17:37:24.253 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [19480873~7079576,140416111~6992252,148109105~141733,160337176~7153146,182034135~1518423,196727604~2108791,210325964~283338,216361490~3081245,244611976~6306185,256332667~7793749,273160535~7546831,292398584~2871045,301491055~2008066,326964255~3039511,337086692~6947322,350891716~1451525,374607324~4374722] 2024-04-07T17:37:24.260 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [19480873~7079576,138412032~8996331,148109105~141733,159383552~8106770,182034135~1518423,196727604~2108791,210325964~283338,216361490~3081245,244611976~6306185,256332667~7793749,273160535~7546831,289406976~5862653,297795584~5703537,326506747~3497019,337086692~6947322,350891716~1451525,373293056~5688990] 2024-04-07T17:37:24.261 INFO:tasks.workunit.client.0.smithi100.stdout:from 4 to 7 diff [19480873~7079576,49566880~3318481,66861224~7774919,140416111~6992252,148109105~141733,160337176~7153146,182034135~5872430,196727604~2108791,202420372~1435645,210325964~283338,210854655~8588080,240886088~1607107,244611976~6306185,256332667~7793749,273160535~7546831,292398584~2871045,301491055~2008066,326964255~3039511,337086692~6947322,345811290~1311192,350891716~1451525,367170792~4497984,374580519~4455211] 2024-04-07T17:37:24.261 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [19480873~7079576,49566880~3318481,66861224~7774919,140416111~6992252,148109105~141733,160337176~7153146,182034135~5872430,196727604~2108791,202420372~1435645,210325964~283338,210854655~8588080,240886088~1607107,244611976~6306185,256332667~7793749,273160535~7546831,292398584~2871045,301491055~2008066,326964255~3039511,337086692~6947322,345811290~1311192,350891716~1451525,367170792~4497984,374580519~4455211] 2024-04-07T17:37:24.268 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [19480873~7079576,46137344~6748017,66861224~7774919,138412032~8996331,148109105~141733,159383552~8106770,182034135~5872430,196727604~2108791,202420372~1435645,210325964~283338,210854655~8588080,240886088~1607107,244611976~6306185,256332667~7793749,273160535~7546831,289406976~5862653,297795584~5703537,326506747~3497019,337086692~10035790,350891716~1451525,367170792~4497984,373293056~5742674] 2024-04-07T17:37:24.268 INFO:tasks.workunit.client.0.smithi100.stdout:from 4 to 8 diff [19480873~7079576,27909341~266890,49566880~3318481,66861224~7774919,92999623~7613429,128030792~5946802,140416111~6992252,148109105~141733,149233764~3199220,154895619~3096498,160337176~7153146,182034135~5872430,196727604~2108791,202420372~1435645,210325964~283338,210854655~8588080,235153608~5013034,240886088~1607107,244611976~6306185,253228895~825520,256332667~7793749,273160535~7546831,292398584~2871045,300063155~4934365,326964255~3039511,337086692~6947322,345811290~1311192,350891716~1451525,367170792~4497984,374580519~4455211] 2024-04-07T17:37:24.268 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [19480873~7079576,27909341~266890,49566880~3318481,66861224~7774919,92999623~7613429,128030792~5946802,140416111~6992252,148109105~141733,149233764~3199220,154895619~3096498,160337176~7153146,182034135~5872430,196727604~2108791,202420372~1435645,210325964~283338,210854655~8588080,235153608~5013034,240886088~1607107,244611976~6306185,253228895~825520,256332667~7793749,273160535~7546831,292398584~2871045,300063155~4934365,326964255~3039511,337086692~6947322,345811290~1311192,350891716~1451525,367170792~4497984,374580519~4455211] 2024-04-07T17:37:24.276 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [19480873~7079576,27909341~266890,46137344~6748017,66861224~7774919,92274688~8338364,128030792~5946802,138412032~8996331,148109105~141733,149233764~3199220,154895619~3096498,159383552~8106770,182034135~5872430,196727604~2108791,202420372~1435645,210325964~283338,210854655~8588080,234881024~5285618,240886088~1607107,244611976~6306185,253228895~825520,256332667~7793749,273160535~7546831,289406976~5862653,297795584~7201936,326506747~3497019,337086692~10035790,350891716~1451525,367170792~4497984,373293056~5742674] 2024-04-07T17:37:24.276 INFO:tasks.workunit.client.0.smithi100.stdout:from 4 to 9 diff [7275769~186849,19095670~7464779,27909341~266890,31769017~7139161,49566880~3318481,66861224~7774919,92999623~8522076,116025937~3770559,128030792~5946802,140416111~6992252,148109105~141733,149233764~3199220,154895619~3096498,160337176~7153146,182034135~7778892,196727604~2108791,202420372~1435645,210325964~283338,210854655~8588080,230813446~9353196,240886088~1607107,244611976~19514440,273160535~7546831,281234113~5242252,292398584~2871045,300063155~4934365,326964255~3039511,337086692~6947322,345811290~1311192,350891716~1451525,367170792~4497984,374580519~4455211] 2024-04-07T17:37:24.276 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [7275769~186849,19095670~7464779,27909341~266890,31769017~7139161,49566880~3318481,66861224~7774919,92999623~8522076,116025937~3770559,128030792~5946802,140416111~6992252,148109105~141733,149233764~3199220,154895619~3096498,160337176~7153146,182034135~7778892,196727604~2108791,202420372~1435645,210325964~283338,210854655~8588080,230813446~9353196,240886088~1607107,244611976~19514440,273160535~7546831,281234113~5242252,292398584~2871045,300063155~4934365,326964255~3039511,337086692~6947322,345811290~1311192,350891716~1451525,367170792~4497984,374580519~4455211] 2024-04-07T17:37:24.286 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [4194304~3268314,19095670~7464779,27909341~266890,30516268~8391910,46137344~6748017,66861224~7774919,92274688~9247011,113977505~5818991,128030792~5946802,138412032~8996331,148109105~141733,149233764~3199220,154895619~3096498,159383552~8106770,182034135~7778892,196727604~2108791,202420372~1435645,210325964~283338,210854655~8588080,230686720~9479922,240886088~1607107,244611976~19514440,273160535~7546831,281234113~5242252,289406976~5862653,297795584~7201936,326506747~3497019,337086692~10035790,350891716~1451525,367170792~4497984,373293056~5742674] 2024-04-07T17:37:24.286 INFO:tasks.workunit.client.0.smithi100.stdout:from 4 to 10 diff [2974197~5131400,19095670~7464779,27431447~878329,31769017~7139161,49566880~3318481,66861224~7774919,92999623~8522076,106500403~5285199,116025937~3770559,121501610~12475984,140416111~6992252,148109105~141733,149233764~3199220,154895619~3096498,160337176~7153146,182034135~7778892,196727604~2108791,202420372~1435645,210325964~283338,210854655~8588080,230813446~13138536,244611976~19514440,273160535~7546831,281234113~5242252,287123044~8146585,300063155~4934365,323691380~1672817,326964255~3039511,337086692~6947322,345811290~1311192,350891716~1451525,366341667~7420413,374580519~4455211] 2024-04-07T17:37:24.286 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2974197~5131400,19095670~7464779,27431447~878329,31769017~7139161,49566880~3318481,66861224~7774919,92999623~8522076,106500403~5285199,116025937~3770559,121501610~12475984,140416111~6992252,148109105~141733,149233764~3199220,154895619~3096498,160337176~7153146,182034135~7778892,196727604~2108791,202420372~1435645,210325964~283338,210854655~8588080,230813446~13138536,244611976~19514440,273160535~7546831,281234113~5242252,287123044~8146585,300063155~4934365,323691380~1672817,326964255~3039511,337086692~6947322,345811290~1311192,350891716~1451525,366341667~7420413,374580519~4455211] 2024-04-07T17:37:24.294 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~8105597,19095670~7464779,27431447~878329,30516268~8391910,46137344~6748017,66861224~7774919,92274688~9247011,106500403~5285199,113977505~20000089,138412032~8996331,148109105~141733,149233764~3199220,154895619~3096498,159383552~8106770,182034135~7778892,196727604~2108791,202420372~1435645,210325964~283338,210854655~8588080,230686720~13265262,244611976~19514440,273160535~7546831,281234113~14035516,297795584~7201936,323691380~1672817,326506747~3497019,337086692~10035790,350891716~1451525,366341667~12694063] 2024-04-07T17:37:24.295 INFO:tasks.workunit.client.0.smithi100.stdout:from 4 to 11 diff [2974197~5131400,19095670~7464779,27431447~878329,31769017~10175249,49566880~3318481,62264039~1975360,66861224~7774919,78193873~4609655,92999623~8522076,106500403~5285199,116025937~3770559,121501610~12475984,140416111~6992252,148109105~141733,149233764~3199220,154895619~3096498,160337176~7153146,182034135~7778892,196727604~2108791,202420372~1435645,210325964~283338,210854655~9339103,222539835~1385676,225244127~18707855,244611976~19514440,272896915~7810451,281234113~5242252,287123044~8146585,300063155~4934365,323691380~1672817,326964255~3039511,337086692~6947322,345811290~1311192,350891716~1451525,366341667~7420413,374580519~10318590,388479465~3566372,398418332~7708261] 2024-04-07T17:37:24.295 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2974197~5131400,19095670~7464779,27431447~878329,31769017~10175249,49566880~3318481,62264039~1975360,66861224~7774919,78193873~4609655,92999623~8522076,106500403~5285199,116025937~3770559,121501610~12475984,140416111~6992252,148109105~141733,149233764~3199220,154895619~3096498,160337176~7153146,182034135~7778892,196727604~2108791,202420372~1435645,210325964~283338,210854655~9339103,222539835~1385676,225244127~18707855,244611976~19514440,272896915~7810451,281234113~5242252,287123044~8146585,300063155~4934365,323691380~1672817,326964255~3039511,337086692~6947322,345811290~1311192,350891716~1451525,366341667~7420413,374580519~10318590,388479465~3566372,398418332~7708261] 2024-04-07T17:37:24.302 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~8105597,19095670~7464779,27431447~878329,30516268~11427998,46137344~6748017,58720256~5519143,66861224~7774919,75497472~7306056,92274688~9247011,106500403~5285199,113977505~20000089,138412032~8996331,148109105~141733,149233764~3199220,154895619~3096498,159383552~8106770,182034135~7778892,196727604~2108791,202420372~1435645,210325964~283338,210854655~9339103,222298112~21653870,244611976~19514440,272896915~7810451,281234113~14035516,297795584~7201936,323691380~1672817,326506747~3497019,337086692~10035790,350891716~1451525,366341667~18557442,388479465~3566372,398418332~7708261] 2024-04-07T17:37:24.302 INFO:tasks.workunit.client.0.smithi100.stdout:from 4 to 12 diff [2974197~5131400,19095670~7464779,27431447~878329,31769017~13613313,45857929~8483438,62264039~1975360,66861224~7774919,78193873~4609655,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,140416111~6992252,148109105~141733,149233764~3199220,154895619~3096498,160337176~7153146,182034135~7778892,196727604~2108791,202420372~1435645,210325964~283338,210854655~9339103,222539835~1385676,225244127~18707855,244611976~19514440,272896915~7810451,281234113~5242252,287123044~8471300,298036890~6960630,323691380~6312386,337086692~6947322,345811290~1311192,350891716~1451525,366341667~7420413,374580519~10318590,388479465~3566372,398418332~7708261] 2024-04-07T17:37:24.302 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2974197~5131400,19095670~7464779,27431447~878329,31769017~13613313,45857929~8483438,62264039~1975360,66861224~7774919,78193873~4609655,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,140416111~6992252,148109105~141733,149233764~3199220,154895619~3096498,160337176~7153146,182034135~7778892,196727604~2108791,202420372~1435645,210325964~283338,210854655~9339103,222539835~1385676,225244127~18707855,244611976~19514440,272896915~7810451,281234113~5242252,287123044~8471300,298036890~6960630,323691380~6312386,337086692~6947322,345811290~1311192,350891716~1451525,366341667~7420413,374580519~10318590,388479465~3566372,398418332~7708261] 2024-04-07T17:37:24.311 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~8105597,19095670~7464779,27431447~878329,30516268~23825099,58720256~5519143,66861224~7774919,75497472~7306056,92274688~11289903,106500403~5285199,112275487~21702107,138412032~8996331,148109105~141733,149233764~3199220,154895619~3096498,159383552~8106770,182034135~7778892,196727604~2108791,202420372~1435645,210325964~283338,210854655~9339103,222298112~21653870,244611976~19514440,272896915~7810451,281234113~14360231,297795584~7201936,323691380~6312386,337086692~10035790,350891716~1451525,366341667~18557442,388479465~3566372,398418332~7708261] 2024-04-07T17:37:24.311 INFO:tasks.workunit.client.0.smithi100.stdout:from 4 to 13 diff [2974197~5131400,19095670~7464779,27329013~980763,31769017~13613313,45857929~8483438,62264039~1975360,66861224~10262977,78193873~4609655,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~2853668,140416111~6992252,148109105~141733,149233764~3199220,154895619~3096498,160337176~7153146,182034135~7778892,196727604~2108791,202420372~1435645,210325964~283338,210854655~9339103,222539835~1385676,225244127~18707855,244611976~19514440,267179572~13527794,281234113~5242252,287123044~8471300,298036890~6960630,323691380~6312386,337086692~6947322,345811290~1311192,350891716~1451525,366341667~7420413,374580519~10318590,388479465~3566372,394880591~2187459,398418332~7708261,408516200~2525554] 2024-04-07T17:37:24.311 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2974197~5131400,19095670~7464779,27329013~980763,31769017~13613313,45857929~8483438,62264039~1975360,66861224~10262977,78193873~4609655,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~2853668,140416111~6992252,148109105~141733,149233764~3199220,154895619~3096498,160337176~7153146,182034135~7778892,196727604~2108791,202420372~1435645,210325964~283338,210854655~9339103,222539835~1385676,225244127~18707855,244611976~19514440,267179572~13527794,281234113~5242252,287123044~8471300,298036890~6960630,323691380~6312386,337086692~6947322,345811290~1311192,350891716~1451525,366341667~7420413,374580519~10318590,388479465~3566372,394880591~2187459,398418332~7708261,408516200~2525554] 2024-04-07T17:37:24.318 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~8105597,19095670~7464779,27329013~980763,30516268~23825099,58720256~5519143,66861224~15942304,92274688~11289903,106500403~5285199,112275487~21702107,134217728~13190635,148109105~141733,149233764~3199220,154895619~3096498,159383552~8106770,182034135~7778892,196727604~2108791,202420372~1435645,210325964~283338,210854655~9339103,222298112~21653870,244611976~19514440,264241152~16466214,281234113~14360231,297795584~7201936,323691380~6312386,337086692~10035790,350891716~1451525,366341667~18557442,388479465~3566372,394880591~2187459,398418332~7708261,406847488~4194266] 2024-04-07T17:37:24.319 INFO:tasks.workunit.client.0.smithi100.stdout:from 4 to 14 diff [2329752~5775845,18121228~8439221,27329013~980763,31769017~13613313,45857929~8483438,62264039~1975360,66861224~10262977,78193873~4609655,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~2853668,140416111~6992252,148109105~141733,149233764~3199220,154895619~3096498,160337176~7153146,182034135~10303497,196727604~2108791,202420372~1435645,210325964~283338,210854655~13070856,225244127~18707855,244611976~19514440,267179572~13527794,281234113~5242252,287123044~8471300,298036890~6960630,318962927~11040839,337086692~6947322,345811290~1311192,350891716~1451525,366341667~7420413,374580519~10318590,388479465~3566372,394880591~2187459,398418332~7708261,408516200~2525554] 2024-04-07T17:37:24.319 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~5775845,18121228~8439221,27329013~980763,31769017~13613313,45857929~8483438,62264039~1975360,66861224~10262977,78193873~4609655,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~2853668,140416111~6992252,148109105~141733,149233764~3199220,154895619~3096498,160337176~7153146,182034135~10303497,196727604~2108791,202420372~1435645,210325964~283338,210854655~13070856,225244127~18707855,244611976~19514440,267179572~13527794,281234113~5242252,287123044~8471300,298036890~6960630,318962927~11040839,337086692~6947322,345811290~1311192,350891716~1451525,366341667~7420413,374580519~10318590,388479465~3566372,394880591~2187459,398418332~7708261,408516200~2525554] 2024-04-07T17:37:24.327 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~8105597,18121228~8439221,27329013~980763,30516268~23825099,58720256~5519143,66861224~15942304,92274688~11289903,106500403~5285199,112275487~21702107,134217728~13190635,148109105~141733,149233764~3199220,154895619~3096498,159383552~8106770,182034135~10303497,196727604~2108791,202420372~1435645,210325964~283338,210854655~33097327,244611976~19514440,264241152~16466214,281234113~14360231,297795584~7201936,318962927~11040839,337086692~10035790,350891716~1451525,366341667~18557442,388479465~3566372,394880591~2187459,398418332~7708261,406847488~4194266] 2024-04-07T17:37:24.327 INFO:tasks.workunit.client.0.smithi100.stdout:from 4 to 15 diff [2329752~5775845,18121228~8439221,27329013~980763,31769017~13613313,45857929~8483438,62264039~1975360,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~2853668,140416111~6992252,148109105~141733,149233764~3199220,154895619~3096498,160337176~7153146,168236274~8367316,182034135~10303497,196727604~2108791,202420372~1435645,210325964~283338,210854655~13756087,225244127~38882289,267179572~13527794,281234113~5242252,287123044~8471300,298036890~6960630,314393585~15610181,337086692~6947322,345811290~1311192,350891716~1451525,366341667~7420413,374580519~10318590,388479465~3566372,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:24.327 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~5775845,18121228~8439221,27329013~980763,31769017~13613313,45857929~8483438,62264039~1975360,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~2853668,140416111~6992252,148109105~141733,149233764~3199220,154895619~3096498,160337176~7153146,168236274~8367316,182034135~10303497,196727604~2108791,202420372~1435645,210325964~283338,210854655~13756087,225244127~38882289,267179572~13527794,281234113~5242252,287123044~8471300,298036890~6960630,314393585~15610181,337086692~6947322,345811290~1311192,350891716~1451525,366341667~7420413,374580519~10318590,388479465~3566372,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:24.335 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~8105597,18121228~8439221,27329013~980763,30516268~23825099,58720256~5519143,66861224~16545045,92274688~11289903,106500403~5285199,112275487~21702107,134217728~13190635,148109105~141733,149233764~3199220,154895619~3096498,159383552~8106770,167772160~8831430,182034135~10303497,196727604~2108791,202420372~1435645,210325964~283338,210854655~53271761,264241152~16466214,281234113~14360231,297795584~7201936,310929323~19074443,337086692~10035790,350891716~1451525,366341667~18557442,388479465~3566372,394880591~2187459,398418332~12623422] 2024-04-07T17:37:24.335 INFO:tasks.workunit.client.0.smithi100.stdout:from 4 to 16 diff [2329752~5775845,14933611~2822456,18121228~8439221,27329013~980763,31769017~13613313,45857929~8483438,62264039~1975360,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~2853668,140416111~6992252,148109105~141733,149233764~3199220,154895619~3096498,160337176~7153146,168236274~8367316,182034135~10303497,196727604~2108791,202420372~1435645,210325964~283338,210854655~13756087,225244127~38882289,267179572~13527794,281234113~14360231,298036890~6960630,314393585~15610181,332944411~11089603,345811290~1311192,350891716~1451525,366341667~7420413,374580519~13389528,388479465~3566372,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:24.335 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~5775845,14933611~2822456,18121228~8439221,27329013~980763,31769017~13613313,45857929~8483438,62264039~1975360,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~2853668,140416111~6992252,148109105~141733,149233764~3199220,154895619~3096498,160337176~7153146,168236274~8367316,182034135~10303497,196727604~2108791,202420372~1435645,210325964~283338,210854655~13756087,225244127~38882289,267179572~13527794,281234113~14360231,298036890~6960630,314393585~15610181,332944411~11089603,345811290~1311192,350891716~1451525,366341667~7420413,374580519~13389528,388479465~3566372,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:24.343 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~8105597,12582912~5173155,18121228~8439221,27329013~980763,30516268~23825099,58720256~5519143,66861224~16545045,92274688~11289903,106500403~5285199,112275487~21702107,134217728~13190635,148109105~141733,149233764~3199220,154895619~3096498,159383552~8106770,167772160~8831430,182034135~10303497,196727604~2108791,202420372~1435645,210325964~283338,210854655~53271761,264241152~16466214,281234113~14360231,297795584~7201936,310929323~19074443,331350016~15772466,350891716~1451525,366341667~21628380,388479465~3566372,394880591~2187459,398418332~12623422] 2024-04-07T17:37:24.343 INFO:tasks.workunit.client.0.smithi100.stdout:from 4 to 17 diff [2329752~5775845,12468100~14092349,27329013~980763,31769017~13613313,45857929~8483438,62264039~1975360,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~11137655,148109105~141733,149233764~3199220,154895619~3096498,160337176~7153146,168236274~8367316,182034135~10303497,196727604~2108791,202420372~1435645,210325964~283338,210854655~53271761,267179572~13527794,281234113~14360231,298036890~6960630,314393585~15610181,332944411~11089603,345811290~1311192,350891716~1451525,355806290~1463251,366341667~7420413,374580519~13389528,388479465~3566372,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:24.343 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~5775845,12468100~14092349,27329013~980763,31769017~13613313,45857929~8483438,62264039~1975360,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~11137655,148109105~141733,149233764~3199220,154895619~3096498,160337176~7153146,168236274~8367316,182034135~10303497,196727604~2108791,202420372~1435645,210325964~283338,210854655~53271761,267179572~13527794,281234113~14360231,298036890~6960630,314393585~15610181,332944411~11089603,345811290~1311192,350891716~1451525,355806290~1463251,366341667~7420413,374580519~13389528,388479465~3566372,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:24.350 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~8105597,8388608~18171841,27329013~980763,30516268~23825099,58720256~5519143,66861224~16545045,92274688~11289903,106500403~5285199,112275487~21702107,134217728~13190635,148109105~141733,149233764~3199220,154895619~3096498,159383552~8106770,167772160~8831430,182034135~10303497,196727604~2108791,202420372~1435645,210325964~283338,210854655~53271761,264241152~16466214,281234113~14360231,297795584~7201936,310929323~19074443,331350016~15772466,350891716~1451525,355806290~1463251,366341667~21628380,388479465~3566372,394880591~2187459,398418332~12623422] 2024-04-07T17:37:24.351 INFO:tasks.workunit.client.0.smithi100.stdout:from 4 to 18 diff [2329752~9105991,12468100~14092349,27329013~980763,31769017~13613313,45857929~8483438,58033882~7969874,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~11137655,148109105~141733,149233764~3199220,154895619~3096498,160337176~7153146,168236274~8367316,182034135~10303497,196727604~2108791,202420372~1435645,210147535~53978881,267179572~13527794,281234113~14360231,298036890~6960630,314393585~15610181,332944411~11089603,345811290~1311192,350891716~1451525,355806290~1463251,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:24.351 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~9105991,12468100~14092349,27329013~980763,31769017~13613313,45857929~8483438,58033882~7969874,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~11137655,148109105~141733,149233764~3199220,154895619~3096498,160337176~7153146,168236274~8367316,182034135~10303497,196727604~2108791,202420372~1435645,210147535~53978881,267179572~13527794,281234113~14360231,298036890~6960630,314393585~15610181,332944411~11089603,345811290~1311192,350891716~1451525,355806290~1463251,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:24.359 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~26560449,27329013~980763,30516268~23825099,56705995~9297761,66861224~16545045,92274688~11289903,106500403~5285199,112275487~21702107,134217728~13190635,148109105~141733,149233764~3199220,154895619~3096498,159383552~8106770,167772160~8831430,182034135~10303497,196727604~2108791,202420372~1435645,210147535~53978881,264241152~16466214,281234113~14360231,297795584~7201936,310929323~19074443,331350016~15772466,350891716~1451525,355806290~1463251,366341667~25704170,394880591~2187459,398418332~12623422] 2024-04-07T17:37:24.359 INFO:tasks.workunit.client.0.smithi100.stdout:from 4 to 19 diff [2329752~24230697,27329013~980763,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~12098856,149233764~4722012,154895619~3096498,160337176~7153146,168236274~8367316,182034135~10303497,196727604~2108791,202420372~1435645,210147535~53978881,267179572~13527794,281234113~14360231,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350891716~1451525,355806290~1463251,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:24.359 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~24230697,27329013~980763,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~12098856,149233764~4722012,154895619~3096498,160337176~7153146,168236274~8367316,182034135~10303497,196727604~2108791,202420372~1435645,210147535~53978881,267179572~13527794,281234113~14360231,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350891716~1451525,355806290~1463251,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:24.366 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~26560449,27329013~980763,30516268~23825099,55380210~10623546,66861224~16545045,92274688~11289903,106500403~5285199,112275487~21702107,134217728~14151836,149233764~4722012,154895619~3096498,159383552~8106770,167772160~8831430,182034135~10303497,196727604~2108791,202420372~1435645,210147535~53978881,264241152~16466214,281234113~14360231,297795584~7201936,306903613~23100153,331350016~15772466,350891716~1451525,355806290~1463251,366341667~25704170,394880591~2187459,398418332~12623422] 2024-04-07T17:37:24.366 INFO:tasks.workunit.client.0.smithi100.stdout:from 5 to 6 diff [19480873~7079576,166133159~1357163,182034135~1518423,196727604~2108791,210325964~283338,248419159~2499002,256332667~7793749,273160535~7546831,292398584~2871045,337086692~6947322] 2024-04-07T17:37:24.366 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [19480873~7079576,166133159~1357163,182034135~1518423,196727604~2108791,210325964~283338,248419159~2499002,256332667~7793749,273160535~7546831,292398584~2871045,337086692~6947322] 2024-04-07T17:37:24.374 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [19480873~7079576,166133159~1357163,182034135~1518423,196727604~2108791,210325964~283338,248419159~2499002,256332667~7793749,273160535~7546831,289406976~5862653,337086692~6947322] 2024-04-07T17:37:24.374 INFO:tasks.workunit.client.0.smithi100.stdout:from 5 to 7 diff [19480873~7079576,49566880~3318481,66861224~7774919,141097128~3081966,166133159~1357163,182034135~5872430,196727604~2108791,202420372~1435645,210325964~283338,210854655~6060499,240886088~1607107,248419159~2499002,256332667~7793749,273160535~7546831,292398584~2871045,337086692~6947322,345811290~1311192,367170792~4497984,374580519~4455211] 2024-04-07T17:37:24.375 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [19480873~7079576,49566880~3318481,66861224~7774919,141097128~3081966,166133159~1357163,182034135~5872430,196727604~2108791,202420372~1435645,210325964~283338,210854655~6060499,240886088~1607107,248419159~2499002,256332667~7793749,273160535~7546831,292398584~2871045,337086692~6947322,345811290~1311192,367170792~4497984,374580519~4455211] 2024-04-07T17:37:24.383 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [19480873~7079576,46137344~6748017,66861224~7774919,141097128~3081966,166133159~1357163,182034135~5872430,196727604~2108791,202420372~1435645,210325964~283338,210854655~6060499,240886088~1607107,248419159~2499002,256332667~7793749,273160535~7546831,289406976~5862653,337086692~10035790,367170792~4497984,374580519~4455211] 2024-04-07T17:37:24.383 INFO:tasks.workunit.client.0.smithi100.stdout:from 5 to 8 diff [19480873~7079576,27909341~266890,49566880~3318481,66861224~7774919,92999623~7613429,128030792~5946802,141097128~3081966,149233764~3199220,154895619~3096498,166133159~1357163,182034135~5872430,196727604~2108791,202420372~1435645,210325964~283338,210854655~6060499,235153608~5013034,240886088~1607107,248419159~2499002,253228895~825520,256332667~7793749,273160535~7546831,292398584~2871045,300063155~4934365,337086692~6947322,345811290~1311192,367170792~4497984,374580519~4455211] 2024-04-07T17:37:24.383 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [19480873~7079576,27909341~266890,49566880~3318481,66861224~7774919,92999623~7613429,128030792~5946802,141097128~3081966,149233764~3199220,154895619~3096498,166133159~1357163,182034135~5872430,196727604~2108791,202420372~1435645,210325964~283338,210854655~6060499,235153608~5013034,240886088~1607107,248419159~2499002,253228895~825520,256332667~7793749,273160535~7546831,292398584~2871045,300063155~4934365,337086692~6947322,345811290~1311192,367170792~4497984,374580519~4455211] 2024-04-07T17:37:24.390 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [19480873~7079576,27909341~266890,46137344~6748017,66861224~7774919,92274688~8338364,128030792~5946802,141097128~3081966,149233764~3199220,154895619~3096498,166133159~1357163,182034135~5872430,196727604~2108791,202420372~1435645,210325964~283338,210854655~6060499,234881024~5285618,240886088~1607107,248419159~2499002,253228895~825520,256332667~7793749,273160535~7546831,289406976~5862653,300063155~4934365,337086692~10035790,367170792~4497984,374580519~4455211] 2024-04-07T17:37:24.390 INFO:tasks.workunit.client.0.smithi100.stdout:from 5 to 9 diff [7275769~186849,19095670~7464779,27909341~266890,31769017~7139161,49566880~3318481,66861224~7774919,92999623~8522076,116025937~3770559,128030792~5946802,141097128~3081966,149233764~3199220,154895619~3096498,166133159~1357163,182034135~7778892,196727604~2108791,202420372~1435645,210325964~283338,210854655~6060499,230813446~9353196,240886088~1607107,248419159~15707257,273160535~7546831,281234113~5242252,292398584~2871045,300063155~4934365,337086692~6947322,345811290~1311192,367170792~4497984,374580519~4455211] 2024-04-07T17:37:24.391 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [7275769~186849,19095670~7464779,27909341~266890,31769017~7139161,49566880~3318481,66861224~7774919,92999623~8522076,116025937~3770559,128030792~5946802,141097128~3081966,149233764~3199220,154895619~3096498,166133159~1357163,182034135~7778892,196727604~2108791,202420372~1435645,210325964~283338,210854655~6060499,230813446~9353196,240886088~1607107,248419159~15707257,273160535~7546831,281234113~5242252,292398584~2871045,300063155~4934365,337086692~6947322,345811290~1311192,367170792~4497984,374580519~4455211] 2024-04-07T17:37:24.397 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [4194304~3268314,19095670~7464779,27909341~266890,30516268~8391910,46137344~6748017,66861224~7774919,92274688~9247011,113977505~5818991,128030792~5946802,141097128~3081966,149233764~3199220,154895619~3096498,166133159~1357163,182034135~7778892,196727604~2108791,202420372~1435645,210325964~283338,210854655~6060499,230686720~9479922,240886088~1607107,248419159~15707257,273160535~7546831,281234113~5242252,289406976~5862653,300063155~4934365,337086692~10035790,367170792~4497984,374580519~4455211] 2024-04-07T17:37:24.397 INFO:tasks.workunit.client.0.smithi100.stdout:from 5 to 10 diff [2974197~5131400,19095670~7464779,27431447~878329,31769017~7139161,49566880~3318481,66861224~7774919,92999623~8522076,106500403~5285199,116025937~3770559,121501610~12475984,141097128~3081966,149233764~3199220,154895619~3096498,166133159~1357163,182034135~7778892,196727604~2108791,202420372~1435645,210325964~283338,210854655~6060499,230813446~13138536,248419159~15707257,273160535~7546831,281234113~5242252,287123044~8146585,300063155~4934365,323691380~1672817,337086692~6947322,345811290~1311192,366341667~7420413,374580519~4455211] 2024-04-07T17:37:24.397 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2974197~5131400,19095670~7464779,27431447~878329,31769017~7139161,49566880~3318481,66861224~7774919,92999623~8522076,106500403~5285199,116025937~3770559,121501610~12475984,141097128~3081966,149233764~3199220,154895619~3096498,166133159~1357163,182034135~7778892,196727604~2108791,202420372~1435645,210325964~283338,210854655~6060499,230813446~13138536,248419159~15707257,273160535~7546831,281234113~5242252,287123044~8146585,300063155~4934365,323691380~1672817,337086692~6947322,345811290~1311192,366341667~7420413,374580519~4455211] 2024-04-07T17:37:24.404 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~8105597,19095670~7464779,27431447~878329,30516268~8391910,46137344~6748017,66861224~7774919,92274688~9247011,106500403~5285199,113977505~20000089,141097128~3081966,149233764~3199220,154895619~3096498,166133159~1357163,182034135~7778892,196727604~2108791,202420372~1435645,210325964~283338,210854655~6060499,230686720~13265262,248419159~15707257,273160535~7546831,281234113~14035516,300063155~4934365,323691380~1672817,337086692~10035790,366341667~7420413,374580519~4455211] 2024-04-07T17:37:24.405 INFO:tasks.workunit.client.0.smithi100.stdout:from 5 to 11 diff [2974197~5131400,19095670~7464779,27431447~878329,31769017~10175249,49566880~3318481,62264039~1975360,66861224~7774919,78193873~4609655,92999623~8522076,106500403~5285199,116025937~3770559,121501610~12475984,141097128~3081966,149233764~3199220,154895619~3096498,166133159~1357163,182034135~7778892,196727604~2108791,202420372~1435645,210325964~283338,210854655~6060499,218521286~1672472,222539835~1385676,225244127~18707855,248419159~15707257,272896915~7810451,281234113~5242252,287123044~8146585,300063155~4934365,323691380~1672817,337086692~6947322,345811290~1311192,366341667~7420413,374580519~10318590,388479465~3566372,398418332~7708261] 2024-04-07T17:37:24.405 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2974197~5131400,19095670~7464779,27431447~878329,31769017~10175249,49566880~3318481,62264039~1975360,66861224~7774919,78193873~4609655,92999623~8522076,106500403~5285199,116025937~3770559,121501610~12475984,141097128~3081966,149233764~3199220,154895619~3096498,166133159~1357163,182034135~7778892,196727604~2108791,202420372~1435645,210325964~283338,210854655~6060499,218521286~1672472,222539835~1385676,225244127~18707855,248419159~15707257,272896915~7810451,281234113~5242252,287123044~8146585,300063155~4934365,323691380~1672817,337086692~6947322,345811290~1311192,366341667~7420413,374580519~10318590,388479465~3566372,398418332~7708261] 2024-04-07T17:37:24.413 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~8105597,19095670~7464779,27431447~878329,30516268~11427998,46137344~6748017,58720256~5519143,66861224~7774919,75497472~7306056,92274688~9247011,106500403~5285199,113977505~20000089,141097128~3081966,149233764~3199220,154895619~3096498,166133159~1357163,182034135~7778892,196727604~2108791,202420372~1435645,210325964~283338,210854655~6060499,218521286~1672472,222298112~21653870,248419159~15707257,272896915~7810451,281234113~14035516,300063155~4934365,323691380~1672817,337086692~10035790,366341667~7420413,374580519~10318590,388479465~3566372,398418332~7708261] 2024-04-07T17:37:24.413 INFO:tasks.workunit.client.0.smithi100.stdout:from 5 to 12 diff [2974197~5131400,19095670~7464779,27431447~878329,31769017~13613313,45857929~8483438,62264039~1975360,66861224~7774919,78193873~4609655,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,141097128~3081966,149233764~3199220,154895619~3096498,166133159~1357163,182034135~7778892,196727604~2108791,202420372~1435645,210325964~283338,210854655~6060499,218521286~1672472,222539835~1385676,225244127~18707855,248419159~15707257,272896915~7810451,281234113~5242252,287123044~8471300,298036890~6960630,323691380~4417991,337086692~6947322,345811290~1311192,366341667~7420413,374580519~10318590,388479465~3566372,398418332~7708261] 2024-04-07T17:37:24.413 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2974197~5131400,19095670~7464779,27431447~878329,31769017~13613313,45857929~8483438,62264039~1975360,66861224~7774919,78193873~4609655,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,141097128~3081966,149233764~3199220,154895619~3096498,166133159~1357163,182034135~7778892,196727604~2108791,202420372~1435645,210325964~283338,210854655~6060499,218521286~1672472,222539835~1385676,225244127~18707855,248419159~15707257,272896915~7810451,281234113~5242252,287123044~8471300,298036890~6960630,323691380~4417991,337086692~6947322,345811290~1311192,366341667~7420413,374580519~10318590,388479465~3566372,398418332~7708261] 2024-04-07T17:37:24.421 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~8105597,19095670~7464779,27431447~878329,30516268~23825099,58720256~5519143,66861224~7774919,75497472~7306056,92274688~11289903,106500403~5285199,112275487~21702107,141097128~3081966,149233764~3199220,154895619~3096498,166133159~1357163,182034135~7778892,196727604~2108791,202420372~1435645,210325964~283338,210854655~6060499,218521286~1672472,222298112~21653870,248419159~15707257,272896915~7810451,281234113~14360231,298036890~6960630,323691380~4417991,337086692~10035790,366341667~7420413,374580519~10318590,388479465~3566372,398418332~7708261] 2024-04-07T17:37:24.421 INFO:tasks.workunit.client.0.smithi100.stdout:from 5 to 13 diff [2974197~5131400,19095670~7464779,27329013~980763,31769017~13613313,45857929~8483438,62264039~1975360,66861224~10262977,78193873~4609655,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~2853668,141097128~3081966,144181083~2188757,149233764~3199220,154895619~3096498,166133159~1357163,182034135~7778892,196727604~2108791,202420372~1435645,210325964~283338,210854655~6060499,218521286~1672472,222539835~1385676,225244127~18707855,248419159~15707257,267179572~13527794,281234113~5242252,287123044~8471300,298036890~6960630,323691380~4417991,337086692~6947322,345811290~1311192,366341667~7420413,374580519~10318590,388479465~3566372,394880591~2187459,398418332~7708261,408516200~2525554] 2024-04-07T17:37:24.421 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2974197~5131400,19095670~7464779,27329013~980763,31769017~13613313,45857929~8483438,62264039~1975360,66861224~10262977,78193873~4609655,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~2853668,141097128~3081966,144181083~2188757,149233764~3199220,154895619~3096498,166133159~1357163,182034135~7778892,196727604~2108791,202420372~1435645,210325964~283338,210854655~6060499,218521286~1672472,222539835~1385676,225244127~18707855,248419159~15707257,267179572~13527794,281234113~5242252,287123044~8471300,298036890~6960630,323691380~4417991,337086692~6947322,345811290~1311192,366341667~7420413,374580519~10318590,388479465~3566372,394880591~2187459,398418332~7708261,408516200~2525554] 2024-04-07T17:37:24.429 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~8105597,19095670~7464779,27329013~980763,30516268~23825099,58720256~5519143,66861224~15942304,92274688~11289903,106500403~5285199,112275487~21702107,134217728~4906648,141097128~3081966,144181083~2188757,149233764~3199220,154895619~3096498,166133159~1357163,182034135~7778892,196727604~2108791,202420372~1435645,210325964~283338,210854655~6060499,218521286~1672472,222298112~21653870,248419159~15707257,264241152~16466214,281234113~14360231,298036890~6960630,323691380~4417991,337086692~10035790,366341667~7420413,374580519~10318590,388479465~3566372,394880591~2187459,398418332~7708261,406847488~4194266] 2024-04-07T17:37:24.429 INFO:tasks.workunit.client.0.smithi100.stdout:from 5 to 14 diff [2329752~5775845,18121228~8439221,27329013~980763,31769017~13613313,45857929~8483438,62264039~1975360,66861224~10262977,78193873~4609655,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~2853668,141097128~3081966,144181083~2188757,149233764~3199220,154895619~3096498,166133159~1357163,182034135~10303497,196727604~2108791,202420372~1435645,210325964~283338,210854655~13070856,225244127~18707855,248419159~15707257,267179572~13527794,281234113~5242252,287123044~8471300,298036890~6960630,318962927~9146444,337086692~6947322,345811290~1311192,366341667~7420413,374580519~10318590,388479465~3566372,394880591~2187459,398418332~7708261,408516200~2525554] 2024-04-07T17:37:24.429 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~5775845,18121228~8439221,27329013~980763,31769017~13613313,45857929~8483438,62264039~1975360,66861224~10262977,78193873~4609655,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~2853668,141097128~3081966,144181083~2188757,149233764~3199220,154895619~3096498,166133159~1357163,182034135~10303497,196727604~2108791,202420372~1435645,210325964~283338,210854655~13070856,225244127~18707855,248419159~15707257,267179572~13527794,281234113~5242252,287123044~8471300,298036890~6960630,318962927~9146444,337086692~6947322,345811290~1311192,366341667~7420413,374580519~10318590,388479465~3566372,394880591~2187459,398418332~7708261,408516200~2525554] 2024-04-07T17:37:24.437 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~8105597,18121228~8439221,27329013~980763,30516268~23825099,58720256~5519143,66861224~15942304,92274688~11289903,106500403~5285199,112275487~21702107,134217728~4906648,141097128~3081966,144181083~2188757,149233764~3199220,154895619~3096498,166133159~1357163,182034135~10303497,196727604~2108791,202420372~1435645,210325964~283338,210854655~33097327,248419159~15707257,264241152~16466214,281234113~14360231,298036890~6960630,318962927~9146444,337086692~10035790,366341667~7420413,374580519~10318590,388479465~3566372,394880591~2187459,398418332~7708261,406847488~4194266] 2024-04-07T17:37:24.437 INFO:tasks.workunit.client.0.smithi100.stdout:from 5 to 15 diff [2329752~5775845,18121228~8439221,27329013~980763,31769017~13613313,45857929~8483438,62264039~1975360,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~2853668,141097128~3081966,144181083~2188757,149233764~3199220,154895619~3096498,166133159~1357163,168236274~8367316,182034135~10303497,196727604~2108791,202420372~1435645,210325964~283338,210854655~13756087,225244127~22982789,248419159~15707257,267179572~13527794,281234113~5242252,287123044~8471300,298036890~6960630,314393585~13715786,337086692~6947322,345811290~1311192,366341667~7420413,374580519~10318590,388479465~3566372,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:24.437 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~5775845,18121228~8439221,27329013~980763,31769017~13613313,45857929~8483438,62264039~1975360,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~2853668,141097128~3081966,144181083~2188757,149233764~3199220,154895619~3096498,166133159~1357163,168236274~8367316,182034135~10303497,196727604~2108791,202420372~1435645,210325964~283338,210854655~13756087,225244127~22982789,248419159~15707257,267179572~13527794,281234113~5242252,287123044~8471300,298036890~6960630,314393585~13715786,337086692~6947322,345811290~1311192,366341667~7420413,374580519~10318590,388479465~3566372,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:24.444 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~8105597,18121228~8439221,27329013~980763,30516268~23825099,58720256~5519143,66861224~16545045,92274688~11289903,106500403~5285199,112275487~21702107,134217728~4906648,141097128~3081966,144181083~2188757,149233764~3199220,154895619~3096498,166133159~1357163,167772160~8831430,182034135~10303497,196727604~2108791,202420372~1435645,210325964~283338,210854655~37372261,248419159~15707257,264241152~16466214,281234113~14360231,298036890~6960630,310929323~17180048,337086692~10035790,366341667~7420413,374580519~10318590,388479465~3566372,394880591~2187459,398418332~12623422] 2024-04-07T17:37:24.444 INFO:tasks.workunit.client.0.smithi100.stdout:from 5 to 16 diff [2329752~5775845,14933611~2822456,18121228~8439221,27329013~980763,31769017~13613313,45857929~8483438,62264039~1975360,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~2853668,141097128~3081966,144181083~2188757,149233764~3199220,154895619~3096498,166133159~1357163,168236274~8367316,182034135~10303497,196727604~2108791,202420372~1435645,210325964~283338,210854655~13756087,225244127~22982789,248419159~15707257,267179572~13527794,281234113~14360231,298036890~6960630,314393585~13715786,332944411~11089603,345811290~1311192,366341667~7420413,374580519~13389528,388479465~3566372,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:24.444 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~5775845,14933611~2822456,18121228~8439221,27329013~980763,31769017~13613313,45857929~8483438,62264039~1975360,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~2853668,141097128~3081966,144181083~2188757,149233764~3199220,154895619~3096498,166133159~1357163,168236274~8367316,182034135~10303497,196727604~2108791,202420372~1435645,210325964~283338,210854655~13756087,225244127~22982789,248419159~15707257,267179572~13527794,281234113~14360231,298036890~6960630,314393585~13715786,332944411~11089603,345811290~1311192,366341667~7420413,374580519~13389528,388479465~3566372,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:24.451 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~8105597,12582912~5173155,18121228~8439221,27329013~980763,30516268~23825099,58720256~5519143,66861224~16545045,92274688~11289903,106500403~5285199,112275487~21702107,134217728~4906648,141097128~3081966,144181083~2188757,149233764~3199220,154895619~3096498,166133159~1357163,167772160~8831430,182034135~10303497,196727604~2108791,202420372~1435645,210325964~283338,210854655~37372261,248419159~15707257,264241152~16466214,281234113~14360231,298036890~6960630,310929323~17180048,331350016~15772466,366341667~7420413,374580519~13389528,388479465~3566372,394880591~2187459,398418332~12623422] 2024-04-07T17:37:24.451 INFO:tasks.workunit.client.0.smithi100.stdout:from 5 to 17 diff [2329752~5775845,12468100~14092349,27329013~980763,31769017~13613313,45857929~8483438,62264039~1975360,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~7908386,144181083~2188757,149233764~3199220,154895619~3096498,166133159~1357163,168236274~8367316,182034135~10303497,196727604~2108791,202420372~1435645,210325964~283338,210854655~37372261,248419159~15707257,267179572~13527794,281234113~14360231,298036890~6960630,314393585~13715786,332944411~11089603,345811290~1311192,355806290~1463251,366341667~7420413,374580519~13389528,388479465~3566372,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:24.451 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~5775845,12468100~14092349,27329013~980763,31769017~13613313,45857929~8483438,62264039~1975360,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~7908386,144181083~2188757,149233764~3199220,154895619~3096498,166133159~1357163,168236274~8367316,182034135~10303497,196727604~2108791,202420372~1435645,210325964~283338,210854655~37372261,248419159~15707257,267179572~13527794,281234113~14360231,298036890~6960630,314393585~13715786,332944411~11089603,345811290~1311192,355806290~1463251,366341667~7420413,374580519~13389528,388479465~3566372,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:24.458 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~8105597,8388608~18171841,27329013~980763,30516268~23825099,58720256~5519143,66861224~16545045,92274688~11289903,106500403~5285199,112275487~21702107,134217728~9961366,144181083~2188757,149233764~3199220,154895619~3096498,166133159~1357163,167772160~8831430,182034135~10303497,196727604~2108791,202420372~1435645,210325964~283338,210854655~37372261,248419159~15707257,264241152~16466214,281234113~14360231,298036890~6960630,310929323~17180048,331350016~15772466,355806290~1463251,366341667~7420413,374580519~13389528,388479465~3566372,394880591~2187459,398418332~12623422] 2024-04-07T17:37:24.459 INFO:tasks.workunit.client.0.smithi100.stdout:from 5 to 18 diff [2329752~9105991,12468100~14092349,27329013~980763,31769017~13613313,45857929~8483438,58033882~7969874,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~7908386,144181083~2188757,149233764~3199220,154895619~3096498,166133159~1357163,168236274~8367316,182034135~10303497,196727604~2108791,202420372~1435645,210147535~38079381,248419159~15707257,267179572~13527794,281234113~14360231,298036890~6960630,314393585~13715786,332944411~11089603,345811290~1311192,355806290~1463251,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:24.459 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~9105991,12468100~14092349,27329013~980763,31769017~13613313,45857929~8483438,58033882~7969874,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~7908386,144181083~2188757,149233764~3199220,154895619~3096498,166133159~1357163,168236274~8367316,182034135~10303497,196727604~2108791,202420372~1435645,210147535~38079381,248419159~15707257,267179572~13527794,281234113~14360231,298036890~6960630,314393585~13715786,332944411~11089603,345811290~1311192,355806290~1463251,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:24.465 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~26560449,27329013~980763,30516268~23825099,56705995~9297761,66861224~16545045,92274688~11289903,106500403~5285199,112275487~21702107,134217728~9961366,144181083~2188757,149233764~3199220,154895619~3096498,166133159~1357163,167772160~8831430,182034135~10303497,196727604~2108791,202420372~1435645,210147535~38079381,248419159~15707257,264241152~16466214,281234113~14360231,298036890~6960630,310929323~17180048,331350016~15772466,355806290~1463251,366341667~7420413,374580519~17465318,394880591~2187459,398418332~12623422] 2024-04-07T17:37:24.465 INFO:tasks.workunit.client.0.smithi100.stdout:from 5 to 19 diff [2329752~24230697,27329013~980763,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~12098856,149233764~4722012,154895619~3096498,166133159~1357163,168236274~8367316,182034135~10303497,196727604~2108791,202420372~1435645,210147535~38079381,248419159~15707257,267179572~13527794,281234113~14360231,298036890~6960630,306903613~5021411,314393585~13715786,332944411~11089603,345811290~1311192,355806290~1463251,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:24.465 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~24230697,27329013~980763,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~12098856,149233764~4722012,154895619~3096498,166133159~1357163,168236274~8367316,182034135~10303497,196727604~2108791,202420372~1435645,210147535~38079381,248419159~15707257,267179572~13527794,281234113~14360231,298036890~6960630,306903613~5021411,314393585~13715786,332944411~11089603,345811290~1311192,355806290~1463251,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:24.472 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~26560449,27329013~980763,30516268~23825099,55380210~10623546,66861224~16545045,92274688~11289903,106500403~5285199,112275487~21702107,134217728~14151836,149233764~4722012,154895619~3096498,166133159~1357163,167772160~8831430,182034135~10303497,196727604~2108791,202420372~1435645,210147535~38079381,248419159~15707257,264241152~16466214,281234113~14360231,298036890~6960630,306903613~21205758,331350016~15772466,355806290~1463251,366341667~7420413,374580519~17465318,394880591~2187459,398418332~12623422] 2024-04-07T17:37:24.472 INFO:tasks.workunit.client.0.smithi100.stdout:from 6 to 7 diff [49566880~3318481,66861224~7774919,141097128~3081966,183213271~4693294,202420372~1435645,210854655~6060499,240886088~1607107,345811290~1311192,367170792~4497984,374580519~4455211] 2024-04-07T17:37:24.472 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [49566880~3318481,66861224~7774919,141097128~3081966,183213271~4693294,202420372~1435645,210854655~6060499,240886088~1607107,345811290~1311192,367170792~4497984,374580519~4455211] 2024-04-07T17:37:24.480 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [46137344~6748017,66861224~7774919,141097128~3081966,183213271~4693294,202420372~1435645,210854655~6060499,240886088~1607107,344034014~3088468,367170792~4497984,374580519~4455211] 2024-04-07T17:37:24.480 INFO:tasks.workunit.client.0.smithi100.stdout:from 6 to 8 diff [27909341~266890,49566880~3318481,66861224~7774919,92999623~7613429,128030792~5946802,141097128~3081966,149233764~3199220,154895619~3096498,183213271~4693294,202420372~1435645,210854655~6060499,235153608~5013034,240886088~1607107,253228895~825520,300063155~4934365,345811290~1311192,367170792~4497984,374580519~4455211] 2024-04-07T17:37:24.480 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [27909341~266890,49566880~3318481,66861224~7774919,92999623~7613429,128030792~5946802,141097128~3081966,149233764~3199220,154895619~3096498,183213271~4693294,202420372~1435645,210854655~6060499,235153608~5013034,240886088~1607107,253228895~825520,300063155~4934365,345811290~1311192,367170792~4497984,374580519~4455211] 2024-04-07T17:37:24.488 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [27909341~266890,46137344~6748017,66861224~7774919,92274688~8338364,128030792~5946802,141097128~3081966,149233764~3199220,154895619~3096498,183213271~4693294,202420372~1435645,210854655~6060499,234881024~5285618,240886088~1607107,253228895~825520,300063155~4934365,344034014~3088468,367170792~4497984,374580519~4455211] 2024-04-07T17:37:24.488 INFO:tasks.workunit.client.0.smithi100.stdout:from 6 to 9 diff [7275769~186849,19095670~6652777,27909341~266890,31769017~7139161,49566880~3318481,66861224~7774919,92999623~8522076,116025937~3770559,128030792~5946802,141097128~3081966,149233764~3199220,154895619~3096498,182747004~7066023,202420372~1435645,210854655~6060499,230813446~9353196,240886088~1607107,250447165~7466990,281234113~5242252,300063155~4934365,345811290~1311192,367170792~4497984,374580519~4455211] 2024-04-07T17:37:24.488 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [7275769~186849,19095670~6652777,27909341~266890,31769017~7139161,49566880~3318481,66861224~7774919,92999623~8522076,116025937~3770559,128030792~5946802,141097128~3081966,149233764~3199220,154895619~3096498,182747004~7066023,202420372~1435645,210854655~6060499,230813446~9353196,240886088~1607107,250447165~7466990,281234113~5242252,300063155~4934365,345811290~1311192,367170792~4497984,374580519~4455211] 2024-04-07T17:37:24.495 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [4194304~3268314,19095670~6652777,27909341~266890,30516268~8391910,46137344~6748017,66861224~7774919,92274688~9247011,113977505~5818991,128030792~5946802,141097128~3081966,149233764~3199220,154895619~3096498,182747004~7066023,202420372~1435645,210854655~6060499,230686720~9479922,240886088~1607107,250447165~7466990,281234113~5242252,300063155~4934365,344034014~3088468,367170792~4497984,374580519~4455211] 2024-04-07T17:37:24.495 INFO:tasks.workunit.client.0.smithi100.stdout:from 6 to 10 diff [2974197~5131400,19095670~6652777,27431447~878329,31769017~7139161,49566880~3318481,66861224~7774919,92999623~8522076,106500403~5285199,116025937~3770559,121501610~12475984,141097128~3081966,149233764~3199220,154895619~3096498,182747004~7066023,202420372~1435645,210854655~6060499,230813446~13138536,250447165~7466990,281234113~5242252,287123044~5448918,300063155~4934365,323691380~1672817,345811290~1311192,366341667~7420413,374580519~4455211] 2024-04-07T17:37:24.495 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2974197~5131400,19095670~6652777,27431447~878329,31769017~7139161,49566880~3318481,66861224~7774919,92999623~8522076,106500403~5285199,116025937~3770559,121501610~12475984,141097128~3081966,149233764~3199220,154895619~3096498,182747004~7066023,202420372~1435645,210854655~6060499,230813446~13138536,250447165~7466990,281234113~5242252,287123044~5448918,300063155~4934365,323691380~1672817,345811290~1311192,366341667~7420413,374580519~4455211] 2024-04-07T17:37:24.501 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~8105597,19095670~6652777,27431447~878329,30516268~8391910,46137344~6748017,66861224~7774919,92274688~9247011,106500403~5285199,113977505~20000089,141097128~3081966,149233764~3199220,154895619~3096498,182747004~7066023,202420372~1435645,210854655~6060499,230686720~13265262,250447165~7466990,281234113~11337849,300063155~4934365,323691380~1672817,344034014~3088468,366341667~7420413,374580519~4455211] 2024-04-07T17:37:24.501 INFO:tasks.workunit.client.0.smithi100.stdout:from 6 to 11 diff [2974197~5131400,19095670~6652777,27431447~878329,31769017~10175249,49566880~3318481,62264039~1975360,66861224~7774919,78193873~4609655,92999623~8522076,106500403~5285199,116025937~3770559,121501610~12475984,141097128~3081966,149233764~3199220,154895619~3096498,182747004~7066023,202420372~1435645,210854655~6060499,218521286~1672472,222539835~1385676,225244127~18707855,250447165~7466990,272896915~6011986,281234113~5242252,287123044~5448918,300063155~4934365,323691380~1672817,345811290~1311192,366341667~7420413,374580519~10318590,388479465~3566372,398418332~7708261] 2024-04-07T17:37:24.501 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2974197~5131400,19095670~6652777,27431447~878329,31769017~10175249,49566880~3318481,62264039~1975360,66861224~7774919,78193873~4609655,92999623~8522076,106500403~5285199,116025937~3770559,121501610~12475984,141097128~3081966,149233764~3199220,154895619~3096498,182747004~7066023,202420372~1435645,210854655~6060499,218521286~1672472,222539835~1385676,225244127~18707855,250447165~7466990,272896915~6011986,281234113~5242252,287123044~5448918,300063155~4934365,323691380~1672817,345811290~1311192,366341667~7420413,374580519~10318590,388479465~3566372,398418332~7708261] 2024-04-07T17:37:24.510 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~8105597,19095670~6652777,27431447~878329,30516268~11427998,46137344~6748017,58720256~5519143,66861224~7774919,75497472~7306056,92274688~9247011,106500403~5285199,113977505~20000089,141097128~3081966,149233764~3199220,154895619~3096498,182747004~7066023,202420372~1435645,210854655~6060499,218521286~1672472,222298112~21653870,250447165~7466990,272896915~6011986,281234113~11337849,300063155~4934365,323691380~1672817,344034014~3088468,366341667~7420413,374580519~10318590,388479465~3566372,398418332~7708261] 2024-04-07T17:37:24.510 INFO:tasks.workunit.client.0.smithi100.stdout:from 6 to 12 diff [2974197~5131400,19095670~6652777,27431447~878329,31769017~13613313,45857929~8483438,62264039~1975360,66861224~7774919,78193873~4609655,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,141097128~3081966,149233764~3199220,154895619~3096498,182747004~7066023,202420372~1435645,210854655~6060499,218521286~1672472,222539835~1385676,225244127~18707855,250447165~7466990,272896915~6011986,281234113~5242252,287123044~5448918,294114638~1479706,298036890~6960630,323691380~4417991,345811290~1311192,366341667~7420413,374580519~10318590,388479465~3566372,398418332~7708261] 2024-04-07T17:37:24.510 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2974197~5131400,19095670~6652777,27431447~878329,31769017~13613313,45857929~8483438,62264039~1975360,66861224~7774919,78193873~4609655,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,141097128~3081966,149233764~3199220,154895619~3096498,182747004~7066023,202420372~1435645,210854655~6060499,218521286~1672472,222539835~1385676,225244127~18707855,250447165~7466990,272896915~6011986,281234113~5242252,287123044~5448918,294114638~1479706,298036890~6960630,323691380~4417991,345811290~1311192,366341667~7420413,374580519~10318590,388479465~3566372,398418332~7708261] 2024-04-07T17:37:24.518 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~8105597,19095670~6652777,27431447~878329,30516268~23825099,58720256~5519143,66861224~7774919,75497472~7306056,92274688~11289903,106500403~5285199,112275487~21702107,141097128~3081966,149233764~3199220,154895619~3096498,182747004~7066023,202420372~1435645,210854655~6060499,218521286~1672472,222298112~21653870,250447165~7466990,272896915~6011986,281234113~11337849,294114638~1479706,298036890~6960630,323691380~4417991,344034014~3088468,366341667~7420413,374580519~10318590,388479465~3566372,398418332~7708261] 2024-04-07T17:37:24.518 INFO:tasks.workunit.client.0.smithi100.stdout:from 6 to 13 diff [2974197~5131400,19095670~6652777,27329013~980763,31769017~13613313,45857929~8483438,62264039~1975360,66861224~10262977,78193873~4609655,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~2853668,141097128~3081966,144181083~2188757,149233764~3199220,154895619~3096498,182747004~7066023,202420372~1435645,210854655~6060499,218521286~1672472,222539835~1385676,225244127~18707855,250447165~7466990,267179572~11729329,281234113~5242252,287123044~5448918,294114638~1479706,298036890~6960630,323691380~4417991,345811290~1311192,366341667~7420413,374580519~10318590,388479465~3566372,394880591~2187459,398418332~7708261,408516200~2525554] 2024-04-07T17:37:24.518 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2974197~5131400,19095670~6652777,27329013~980763,31769017~13613313,45857929~8483438,62264039~1975360,66861224~10262977,78193873~4609655,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~2853668,141097128~3081966,144181083~2188757,149233764~3199220,154895619~3096498,182747004~7066023,202420372~1435645,210854655~6060499,218521286~1672472,222539835~1385676,225244127~18707855,250447165~7466990,267179572~11729329,281234113~5242252,287123044~5448918,294114638~1479706,298036890~6960630,323691380~4417991,345811290~1311192,366341667~7420413,374580519~10318590,388479465~3566372,394880591~2187459,398418332~7708261,408516200~2525554] 2024-04-07T17:37:24.524 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~8105597,19095670~6652777,27329013~980763,30516268~23825099,58720256~5519143,66861224~15942304,92274688~11289903,106500403~5285199,112275487~21702107,134217728~4906648,141097128~3081966,144181083~2188757,149233764~3199220,154895619~3096498,182747004~7066023,202420372~1435645,210854655~6060499,218521286~1672472,222298112~21653870,250447165~7466990,264241152~14667749,281234113~11337849,294114638~1479706,298036890~6960630,323691380~4417991,344034014~3088468,366341667~7420413,374580519~10318590,388479465~3566372,394880591~2187459,398418332~7708261,406847488~4194266] 2024-04-07T17:37:24.525 INFO:tasks.workunit.client.0.smithi100.stdout:from 6 to 14 diff [2329752~5775845,18121228~7627219,27329013~980763,31769017~13613313,45857929~8483438,62264039~1975360,66861224~10262977,78193873~4609655,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~2853668,141097128~3081966,144181083~2188757,149233764~3199220,154895619~3096498,182747004~9590628,202420372~1435645,210854655~13070856,225244127~18707855,250447165~7466990,267179572~11729329,281234113~5242252,287123044~8471300,298036890~6960630,318962927~9146444,345811290~1311192,366341667~7420413,374580519~10318590,388479465~3566372,394880591~2187459,398418332~7708261,408516200~2525554] 2024-04-07T17:37:24.525 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~5775845,18121228~7627219,27329013~980763,31769017~13613313,45857929~8483438,62264039~1975360,66861224~10262977,78193873~4609655,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~2853668,141097128~3081966,144181083~2188757,149233764~3199220,154895619~3096498,182747004~9590628,202420372~1435645,210854655~13070856,225244127~18707855,250447165~7466990,267179572~11729329,281234113~5242252,287123044~8471300,298036890~6960630,318962927~9146444,345811290~1311192,366341667~7420413,374580519~10318590,388479465~3566372,394880591~2187459,398418332~7708261,408516200~2525554] 2024-04-07T17:37:24.532 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~8105597,18121228~7627219,27329013~980763,30516268~23825099,58720256~5519143,66861224~15942304,92274688~11289903,106500403~5285199,112275487~21702107,134217728~4906648,141097128~3081966,144181083~2188757,149233764~3199220,154895619~3096498,182747004~9590628,202420372~1435645,210854655~33097327,250447165~7466990,264241152~14667749,281234113~14360231,298036890~6960630,318962927~9146444,344034014~3088468,366341667~7420413,374580519~10318590,388479465~3566372,394880591~2187459,398418332~7708261,406847488~4194266] 2024-04-07T17:37:24.532 INFO:tasks.workunit.client.0.smithi100.stdout:from 6 to 15 diff [2329752~5775845,18121228~7627219,27329013~980763,31769017~13613313,45857929~8483438,62264039~1975360,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~2853668,141097128~3081966,144181083~2188757,149233764~3199220,154895619~3096498,168236274~8367316,182747004~9590628,202420372~1435645,210854655~13756087,225244127~22982789,250447165~8059589,267179572~11729329,281234113~5242252,287123044~8471300,298036890~6960630,314393585~13715786,345811290~1311192,366341667~7420413,374580519~10318590,388479465~3566372,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:24.532 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~5775845,18121228~7627219,27329013~980763,31769017~13613313,45857929~8483438,62264039~1975360,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~2853668,141097128~3081966,144181083~2188757,149233764~3199220,154895619~3096498,168236274~8367316,182747004~9590628,202420372~1435645,210854655~13756087,225244127~22982789,250447165~8059589,267179572~11729329,281234113~5242252,287123044~8471300,298036890~6960630,314393585~13715786,345811290~1311192,366341667~7420413,374580519~10318590,388479465~3566372,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:24.540 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~8105597,18121228~7627219,27329013~980763,30516268~23825099,58720256~5519143,66861224~16545045,92274688~11289903,106500403~5285199,112275487~21702107,134217728~4906648,141097128~3081966,144181083~2188757,149233764~3199220,154895619~3096498,167772160~8831430,182747004~9590628,202420372~1435645,210854655~37372261,250447165~8059589,264241152~14667749,281234113~14360231,298036890~6960630,310929323~17180048,344034014~3088468,366341667~7420413,374580519~10318590,388479465~3566372,394880591~2187459,398418332~12623422] 2024-04-07T17:37:24.540 INFO:tasks.workunit.client.0.smithi100.stdout:from 6 to 16 diff [2329752~5775845,14933611~2822456,18121228~7627219,27329013~980763,31769017~13613313,45857929~8483438,62264039~1975360,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~2853668,141097128~3081966,144181083~2188757,149233764~3199220,154895619~3096498,168236274~8367316,182747004~9590628,202420372~1435645,210854655~13756087,225244127~22982789,250447165~8059589,267179572~11729329,281234113~14360231,298036890~6960630,314393585~13715786,332944411~6919544,345811290~1311192,366341667~7420413,374580519~13389528,388479465~3566372,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:24.540 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~5775845,14933611~2822456,18121228~7627219,27329013~980763,31769017~13613313,45857929~8483438,62264039~1975360,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~2853668,141097128~3081966,144181083~2188757,149233764~3199220,154895619~3096498,168236274~8367316,182747004~9590628,202420372~1435645,210854655~13756087,225244127~22982789,250447165~8059589,267179572~11729329,281234113~14360231,298036890~6960630,314393585~13715786,332944411~6919544,345811290~1311192,366341667~7420413,374580519~13389528,388479465~3566372,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:24.546 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~8105597,12582912~5173155,18121228~7627219,27329013~980763,30516268~23825099,58720256~5519143,66861224~16545045,92274688~11289903,106500403~5285199,112275487~21702107,134217728~4906648,141097128~3081966,144181083~2188757,149233764~3199220,154895619~3096498,167772160~8831430,182747004~9590628,202420372~1435645,210854655~37372261,250447165~8059589,264241152~14667749,281234113~14360231,298036890~6960630,310929323~17180048,331350016~8513939,344034014~3088468,366341667~7420413,374580519~13389528,388479465~3566372,394880591~2187459,398418332~12623422] 2024-04-07T17:37:24.546 INFO:tasks.workunit.client.0.smithi100.stdout:from 6 to 17 diff [2329752~5775845,12468100~13280347,27329013~980763,31769017~13613313,45857929~8483438,62264039~1975360,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~7908386,144181083~2188757,149233764~3199220,154895619~3096498,168236274~8367316,182747004~9590628,202420372~1435645,210854655~37372261,250447165~8059589,267179572~11729329,281234113~14360231,298036890~6960630,314393585~13715786,332944411~6919544,345811290~1311192,355806290~1463251,366341667~7420413,374580519~13389528,388479465~3566372,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:24.547 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~5775845,12468100~13280347,27329013~980763,31769017~13613313,45857929~8483438,62264039~1975360,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~7908386,144181083~2188757,149233764~3199220,154895619~3096498,168236274~8367316,182747004~9590628,202420372~1435645,210854655~37372261,250447165~8059589,267179572~11729329,281234113~14360231,298036890~6960630,314393585~13715786,332944411~6919544,345811290~1311192,355806290~1463251,366341667~7420413,374580519~13389528,388479465~3566372,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:24.554 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~8105597,8388608~17359839,27329013~980763,30516268~23825099,58720256~5519143,66861224~16545045,92274688~11289903,106500403~5285199,112275487~21702107,134217728~9961366,144181083~2188757,149233764~3199220,154895619~3096498,167772160~8831430,182747004~9590628,202420372~1435645,210854655~37372261,250447165~8059589,264241152~14667749,281234113~14360231,298036890~6960630,310929323~17180048,331350016~8513939,344034014~3088468,355806290~1463251,366341667~7420413,374580519~13389528,388479465~3566372,394880591~2187459,398418332~12623422] 2024-04-07T17:37:24.555 INFO:tasks.workunit.client.0.smithi100.stdout:from 6 to 18 diff [2329752~9105991,12468100~13280347,27329013~980763,31769017~13613313,45857929~8483438,58033882~7969874,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~7908386,144181083~2188757,149233764~3199220,154895619~3096498,168236274~8367316,182747004~9590628,202420372~1435645,210147535~38079381,250447165~8059589,267179572~11729329,281234113~14360231,298036890~6960630,314393585~13715786,332944411~6919544,345811290~1311192,355806290~1463251,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:24.555 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~9105991,12468100~13280347,27329013~980763,31769017~13613313,45857929~8483438,58033882~7969874,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~7908386,144181083~2188757,149233764~3199220,154895619~3096498,168236274~8367316,182747004~9590628,202420372~1435645,210147535~38079381,250447165~8059589,267179572~11729329,281234113~14360231,298036890~6960630,314393585~13715786,332944411~6919544,345811290~1311192,355806290~1463251,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:24.561 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~25748447,27329013~980763,30516268~23825099,56705995~9297761,66861224~16545045,92274688~11289903,106500403~5285199,112275487~21702107,134217728~9961366,144181083~2188757,149233764~3199220,154895619~3096498,167772160~8831430,182747004~9590628,202420372~1435645,210147535~38079381,250447165~8059589,264241152~14667749,281234113~14360231,298036890~6960630,310929323~17180048,331350016~8513939,344034014~3088468,355806290~1463251,366341667~7420413,374580519~17465318,394880591~2187459,398418332~12623422] 2024-04-07T17:37:24.561 INFO:tasks.workunit.client.0.smithi100.stdout:from 6 to 19 diff [2329752~23418695,27329013~980763,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~12098856,149233764~4722012,154895619~3096498,168236274~8367316,182747004~9590628,202420372~1435645,210147535~38079381,250447165~8059589,267179572~11729329,281234113~14360231,298036890~6960630,306903613~5021411,314393585~13715786,332944411~6919544,345811290~1311192,355806290~1463251,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:24.561 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~23418695,27329013~980763,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~12098856,149233764~4722012,154895619~3096498,168236274~8367316,182747004~9590628,202420372~1435645,210147535~38079381,250447165~8059589,267179572~11729329,281234113~14360231,298036890~6960630,306903613~5021411,314393585~13715786,332944411~6919544,345811290~1311192,355806290~1463251,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:24.568 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~25748447,27329013~980763,30516268~23825099,55380210~10623546,66861224~16545045,92274688~11289903,106500403~5285199,112275487~21702107,134217728~14151836,149233764~4722012,154895619~3096498,167772160~8831430,182747004~9590628,202420372~1435645,210147535~38079381,250447165~8059589,264241152~14667749,281234113~14360231,298036890~6960630,306903613~21205758,331350016~8513939,344034014~3088468,355806290~1463251,366341667~7420413,374580519~17465318,394880591~2187459,398418332~12623422] 2024-04-07T17:37:24.569 INFO:tasks.workunit.client.0.smithi100.stdout:from 7 to 8 diff [27909341~266890,92999623~7613429,128030792~5946802,149233764~3199220,154895619~3096498,235153608~5013034,253228895~825520,300063155~4934365,378193032~444908] 2024-04-07T17:37:24.569 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [27909341~266890,92999623~7613429,128030792~5946802,149233764~3199220,154895619~3096498,235153608~5013034,253228895~825520,300063155~4934365,378193032~444908] 2024-04-07T17:37:24.576 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [27909341~266890,92274688~8338364,128030792~5946802,149233764~3199220,154895619~3096498,234881024~5285618,253228895~825520,300063155~4934365,378193032~444908] 2024-04-07T17:37:24.576 INFO:tasks.workunit.client.0.smithi100.stdout:from 7 to 9 diff [7275769~186849,19095670~6652777,27909341~266890,31769017~7139161,92999623~8522076,116025937~3770559,128030792~5946802,149233764~3199220,154895619~3096498,182747004~7066023,230813446~9353196,250447165~7466990,281234113~5242252,300063155~4934365,378193032~444908] 2024-04-07T17:37:24.576 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [7275769~186849,19095670~6652777,27909341~266890,31769017~7139161,92999623~8522076,116025937~3770559,128030792~5946802,149233764~3199220,154895619~3096498,182747004~7066023,230813446~9353196,250447165~7466990,281234113~5242252,300063155~4934365,378193032~444908] 2024-04-07T17:37:24.586 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [4194304~3268314,19095670~6652777,27909341~266890,30516268~8391910,92274688~9247011,113977505~5818991,128030792~5946802,149233764~3199220,154895619~3096498,182747004~7066023,230686720~9479922,250447165~7466990,281234113~5242252,300063155~4934365,378193032~444908] 2024-04-07T17:37:24.587 INFO:tasks.workunit.client.0.smithi100.stdout:from 7 to 10 diff [2974197~5131400,19095670~6652777,27431447~878329,31769017~7139161,92999623~8522076,106500403~5285199,116025937~3770559,121501610~12475984,149233764~3199220,154895619~3096498,182747004~7066023,230813446~13138536,250447165~7466990,281234113~5242252,287123044~5448918,300063155~4934365,323691380~1672817,366341667~7420413,378193032~444908] 2024-04-07T17:37:24.587 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2974197~5131400,19095670~6652777,27431447~878329,31769017~7139161,92999623~8522076,106500403~5285199,116025937~3770559,121501610~12475984,149233764~3199220,154895619~3096498,182747004~7066023,230813446~13138536,250447165~7466990,281234113~5242252,287123044~5448918,300063155~4934365,323691380~1672817,366341667~7420413,378193032~444908] 2024-04-07T17:37:24.595 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~8105597,19095670~6652777,27431447~878329,30516268~8391910,92274688~9247011,106500403~5285199,113977505~20000089,149233764~3199220,154895619~3096498,182747004~7066023,230686720~13265262,250447165~7466990,281234113~11337849,300063155~4934365,323691380~1672817,366341667~7420413,378193032~444908] 2024-04-07T17:37:24.595 INFO:tasks.workunit.client.0.smithi100.stdout:from 7 to 11 diff [2974197~5131400,19095670~6652777,27431447~878329,31769017~10175249,62264039~1975360,78193873~4609655,92999623~8522076,106500403~5285199,116025937~3770559,121501610~12475984,149233764~3199220,154895619~3096498,182747004~7066023,218521286~1672472,222539835~1385676,225244127~18707855,250447165~7466990,272896915~6011986,281234113~5242252,287123044~5448918,300063155~4934365,323691380~1672817,366341667~7420413,377322524~7576585,388479465~3566372,398418332~7708261] 2024-04-07T17:37:24.595 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2974197~5131400,19095670~6652777,27431447~878329,31769017~10175249,62264039~1975360,78193873~4609655,92999623~8522076,106500403~5285199,116025937~3770559,121501610~12475984,149233764~3199220,154895619~3096498,182747004~7066023,218521286~1672472,222539835~1385676,225244127~18707855,250447165~7466990,272896915~6011986,281234113~5242252,287123044~5448918,300063155~4934365,323691380~1672817,366341667~7420413,377322524~7576585,388479465~3566372,398418332~7708261] 2024-04-07T17:37:24.602 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~8105597,19095670~6652777,27431447~878329,30516268~11427998,58720256~5519143,75497472~7306056,92274688~9247011,106500403~5285199,113977505~20000089,149233764~3199220,154895619~3096498,182747004~7066023,218521286~1672472,222298112~21653870,250447165~7466990,272896915~6011986,281234113~11337849,300063155~4934365,323691380~1672817,366341667~7420413,377322524~7576585,388479465~3566372,398418332~7708261] 2024-04-07T17:37:24.602 INFO:tasks.workunit.client.0.smithi100.stdout:from 7 to 12 diff [2974197~5131400,19095670~6652777,27431447~878329,31769017~13613313,45857929~8483438,62264039~1975360,78193873~4609655,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,149233764~3199220,154895619~3096498,182747004~7066023,218521286~1672472,222539835~1385676,225244127~18707855,250447165~7466990,272896915~6011986,281234113~5242252,287123044~5448918,294114638~1479706,298036890~6960630,323691380~4417991,366341667~7420413,377322524~7576585,388479465~3566372,398418332~7708261] 2024-04-07T17:37:24.602 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2974197~5131400,19095670~6652777,27431447~878329,31769017~13613313,45857929~8483438,62264039~1975360,78193873~4609655,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,149233764~3199220,154895619~3096498,182747004~7066023,218521286~1672472,222539835~1385676,225244127~18707855,250447165~7466990,272896915~6011986,281234113~5242252,287123044~5448918,294114638~1479706,298036890~6960630,323691380~4417991,366341667~7420413,377322524~7576585,388479465~3566372,398418332~7708261] 2024-04-07T17:37:24.608 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~8105597,19095670~6652777,27431447~878329,30516268~23825099,58720256~5519143,75497472~7306056,92274688~11289903,106500403~5285199,112275487~21702107,149233764~3199220,154895619~3096498,182747004~7066023,218521286~1672472,222298112~21653870,250447165~7466990,272896915~6011986,281234113~11337849,294114638~1479706,298036890~6960630,323691380~4417991,366341667~7420413,377322524~7576585,388479465~3566372,398418332~7708261] 2024-04-07T17:37:24.608 INFO:tasks.workunit.client.0.smithi100.stdout:from 7 to 13 diff [2974197~5131400,19095670~6652777,27329013~980763,31769017~13613313,45857929~8483438,62264039~1975360,70298794~6825407,78193873~4609655,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~2853668,144181083~2188757,149233764~3199220,154895619~3096498,182747004~7066023,218521286~1672472,222539835~1385676,225244127~18707855,250447165~7466990,267179572~11729329,281234113~5242252,287123044~5448918,294114638~1479706,298036890~6960630,323691380~4417991,366341667~7420413,377322524~7576585,388479465~3566372,394880591~2187459,398418332~7708261,408516200~2525554] 2024-04-07T17:37:24.608 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2974197~5131400,19095670~6652777,27329013~980763,31769017~13613313,45857929~8483438,62264039~1975360,70298794~6825407,78193873~4609655,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~2853668,144181083~2188757,149233764~3199220,154895619~3096498,182747004~7066023,218521286~1672472,222539835~1385676,225244127~18707855,250447165~7466990,267179572~11729329,281234113~5242252,287123044~5448918,294114638~1479706,298036890~6960630,323691380~4417991,366341667~7420413,377322524~7576585,388479465~3566372,394880591~2187459,398418332~7708261,408516200~2525554] 2024-04-07T17:37:24.616 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~8105597,19095670~6652777,27329013~980763,30516268~23825099,58720256~5519143,70298794~12504734,92274688~11289903,106500403~5285199,112275487~21702107,134217728~4906648,144181083~2188757,149233764~3199220,154895619~3096498,182747004~7066023,218521286~1672472,222298112~21653870,250447165~7466990,264241152~14667749,281234113~11337849,294114638~1479706,298036890~6960630,323691380~4417991,366341667~7420413,377322524~7576585,388479465~3566372,394880591~2187459,398418332~7708261,406847488~4194266] 2024-04-07T17:37:24.616 INFO:tasks.workunit.client.0.smithi100.stdout:from 7 to 14 diff [2329752~5775845,18121228~7627219,27329013~980763,31769017~13613313,45857929~8483438,62264039~1975360,70298794~6825407,78193873~4609655,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~2853668,144181083~2188757,149233764~3199220,154895619~3096498,182747004~9590628,213644600~10280911,225244127~18707855,250447165~7466990,267179572~11729329,281234113~5242252,287123044~8471300,298036890~6960630,318962927~9146444,366341667~7420413,377322524~7576585,388479465~3566372,394880591~2187459,398418332~7708261,408516200~2525554] 2024-04-07T17:37:24.616 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~5775845,18121228~7627219,27329013~980763,31769017~13613313,45857929~8483438,62264039~1975360,70298794~6825407,78193873~4609655,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~2853668,144181083~2188757,149233764~3199220,154895619~3096498,182747004~9590628,213644600~10280911,225244127~18707855,250447165~7466990,267179572~11729329,281234113~5242252,287123044~8471300,298036890~6960630,318962927~9146444,366341667~7420413,377322524~7576585,388479465~3566372,394880591~2187459,398418332~7708261,408516200~2525554] 2024-04-07T17:37:24.624 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~8105597,18121228~7627219,27329013~980763,30516268~23825099,58720256~5519143,70298794~12504734,92274688~11289903,106500403~5285199,112275487~21702107,134217728~4906648,144181083~2188757,149233764~3199220,154895619~3096498,182747004~9590628,213644600~30307382,250447165~7466990,264241152~14667749,281234113~14360231,298036890~6960630,318962927~9146444,366341667~7420413,377322524~7576585,388479465~3566372,394880591~2187459,398418332~7708261,406847488~4194266] 2024-04-07T17:37:24.624 INFO:tasks.workunit.client.0.smithi100.stdout:from 7 to 15 diff [2329752~5775845,18121228~7627219,27329013~980763,31769017~13613313,45857929~8483438,62264039~1975360,70298794~13107475,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~2853668,144181083~2188757,149233764~3199220,154895619~3096498,168236274~8367316,182747004~9590628,213644600~10966142,225244127~22982789,250447165~8059589,267179572~11729329,281234113~5242252,287123044~8471300,298036890~6960630,314393585~13715786,366341667~7420413,377322524~7576585,388479465~3566372,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:24.624 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~5775845,18121228~7627219,27329013~980763,31769017~13613313,45857929~8483438,62264039~1975360,70298794~13107475,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~2853668,144181083~2188757,149233764~3199220,154895619~3096498,168236274~8367316,182747004~9590628,213644600~10966142,225244127~22982789,250447165~8059589,267179572~11729329,281234113~5242252,287123044~8471300,298036890~6960630,314393585~13715786,366341667~7420413,377322524~7576585,388479465~3566372,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:24.632 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~8105597,18121228~7627219,27329013~980763,30516268~23825099,58720256~5519143,70298794~13107475,92274688~11289903,106500403~5285199,112275487~21702107,134217728~4906648,144181083~2188757,149233764~3199220,154895619~3096498,167772160~8831430,182747004~9590628,213644600~34582316,250447165~8059589,264241152~14667749,281234113~14360231,298036890~6960630,310929323~17180048,366341667~7420413,377322524~7576585,388479465~3566372,394880591~2187459,398418332~12623422] 2024-04-07T17:37:24.632 INFO:tasks.workunit.client.0.smithi100.stdout:from 7 to 16 diff [2329752~5775845,14933611~2822456,18121228~7627219,27329013~980763,31769017~13613313,45857929~8483438,62264039~1975360,70298794~13107475,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~2853668,144181083~2188757,149233764~3199220,154895619~3096498,168236274~8367316,182747004~9590628,213644600~10966142,225244127~22982789,250447165~8059589,267179572~11729329,281234113~14360231,298036890~6960630,314393585~13715786,332944411~6919544,366341667~7420413,377322524~10647523,388479465~3566372,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:24.632 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~5775845,14933611~2822456,18121228~7627219,27329013~980763,31769017~13613313,45857929~8483438,62264039~1975360,70298794~13107475,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~2853668,144181083~2188757,149233764~3199220,154895619~3096498,168236274~8367316,182747004~9590628,213644600~10966142,225244127~22982789,250447165~8059589,267179572~11729329,281234113~14360231,298036890~6960630,314393585~13715786,332944411~6919544,366341667~7420413,377322524~10647523,388479465~3566372,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:24.638 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~8105597,12582912~5173155,18121228~7627219,27329013~980763,30516268~23825099,58720256~5519143,70298794~13107475,92274688~11289903,106500403~5285199,112275487~21702107,134217728~4906648,144181083~2188757,149233764~3199220,154895619~3096498,167772160~8831430,182747004~9590628,213644600~34582316,250447165~8059589,264241152~14667749,281234113~14360231,298036890~6960630,310929323~17180048,331350016~8513939,366341667~7420413,377322524~10647523,388479465~3566372,394880591~2187459,398418332~12623422] 2024-04-07T17:37:24.638 INFO:tasks.workunit.client.0.smithi100.stdout:from 7 to 17 diff [2329752~5775845,12468100~13280347,27329013~980763,31769017~13613313,45857929~8483438,62264039~1975360,68206575~15199694,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~5240179,144181083~2188757,149233764~3199220,154895619~3096498,168236274~8367316,182747004~9590628,211678512~920259,213644600~34582316,250447165~8059589,267179572~11729329,281234113~14360231,298036890~6960630,314393585~13715786,332944411~6919544,355806290~1463251,366341667~7420413,377322524~10647523,388479465~3566372,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:24.639 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~5775845,12468100~13280347,27329013~980763,31769017~13613313,45857929~8483438,62264039~1975360,68206575~15199694,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~5240179,144181083~2188757,149233764~3199220,154895619~3096498,168236274~8367316,182747004~9590628,211678512~920259,213644600~34582316,250447165~8059589,267179572~11729329,281234113~14360231,298036890~6960630,314393585~13715786,332944411~6919544,355806290~1463251,366341667~7420413,377322524~10647523,388479465~3566372,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:24.646 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~8105597,8388608~17359839,27329013~980763,30516268~23825099,58720256~5519143,68206575~15199694,92274688~11289903,106500403~5285199,112275487~21702107,134217728~7293159,144181083~2188757,149233764~3199220,154895619~3096498,167772160~8831430,182747004~9590628,211678512~920259,213644600~34582316,250447165~8059589,264241152~14667749,281234113~14360231,298036890~6960630,310929323~17180048,331350016~8513939,355806290~1463251,366341667~7420413,377322524~10647523,388479465~3566372,394880591~2187459,398418332~12623422] 2024-04-07T17:37:24.646 INFO:tasks.workunit.client.0.smithi100.stdout:from 7 to 18 diff [2329752~9105991,12468100~13280347,27329013~980763,31769017~13613313,45857929~8483438,58033882~7969874,68206575~15199694,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~5240179,144181083~2188757,149233764~3199220,154895619~3096498,168236274~8367316,182747004~9590628,210147535~38079381,250447165~8059589,267179572~11729329,281234113~14360231,298036890~6960630,314393585~13715786,332944411~6919544,355806290~1463251,366341667~7420413,377322524~14723313,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:24.646 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~9105991,12468100~13280347,27329013~980763,31769017~13613313,45857929~8483438,58033882~7969874,68206575~15199694,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~5240179,144181083~2188757,149233764~3199220,154895619~3096498,168236274~8367316,182747004~9590628,210147535~38079381,250447165~8059589,267179572~11729329,281234113~14360231,298036890~6960630,314393585~13715786,332944411~6919544,355806290~1463251,366341667~7420413,377322524~14723313,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:24.652 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~25748447,27329013~980763,30516268~23825099,56705995~9297761,68206575~15199694,92274688~11289903,106500403~5285199,112275487~21702107,134217728~7293159,144181083~2188757,149233764~3199220,154895619~3096498,167772160~8831430,182747004~9590628,210147535~38079381,250447165~8059589,264241152~14667749,281234113~14360231,298036890~6960630,310929323~17180048,331350016~8513939,355806290~1463251,366341667~7420413,377322524~14723313,394880591~2187459,398418332~12623422] 2024-04-07T17:37:24.652 INFO:tasks.workunit.client.0.smithi100.stdout:from 7 to 19 diff [2329752~23418695,27329013~980763,31769017~13613313,45857929~8483438,55380210~10623546,68206575~15199694,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~5240179,141885014~6484550,149233764~4722012,154895619~3096498,168236274~8367316,182747004~9590628,210147535~38079381,250447165~8059589,267179572~11729329,281234113~14360231,298036890~6960630,306903613~5021411,314393585~13715786,332944411~6919544,355806290~1463251,366341667~7420413,377322524~14723313,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:24.652 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~23418695,27329013~980763,31769017~13613313,45857929~8483438,55380210~10623546,68206575~15199694,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~5240179,141885014~6484550,149233764~4722012,154895619~3096498,168236274~8367316,182747004~9590628,210147535~38079381,250447165~8059589,267179572~11729329,281234113~14360231,298036890~6960630,306903613~5021411,314393585~13715786,332944411~6919544,355806290~1463251,366341667~7420413,377322524~14723313,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:24.660 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~25748447,27329013~980763,30516268~23825099,55380210~10623546,68206575~15199694,92274688~11289903,106500403~5285199,112275487~21702107,134217728~7293159,141885014~6484550,149233764~4722012,154895619~3096498,167772160~8831430,182747004~9590628,210147535~38079381,250447165~8059589,264241152~14667749,281234113~14360231,298036890~6960630,306903613~21205758,331350016~8513939,355806290~1463251,366341667~7420413,377322524~14723313,394880591~2187459,398418332~12623422] 2024-04-07T17:37:24.660 INFO:tasks.workunit.client.0.smithi100.stdout:from 8 to 9 diff [7275769~186849,19095670~6652777,31769017~7139161,95470042~6051657,116025937~3770559,182747004~7066023,230813446~6715104,250447165~7466990,281234113~5242252] 2024-04-07T17:37:24.660 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [7275769~186849,19095670~6652777,31769017~7139161,95470042~6051657,116025937~3770559,182747004~7066023,230813446~6715104,250447165~7466990,281234113~5242252] 2024-04-07T17:37:24.667 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [4194304~3268314,19095670~6652777,30516268~8391910,95470042~6051657,113977505~5818991,182747004~7066023,230686720~6841830,250447165~7466990,281234113~5242252] 2024-04-07T17:37:24.668 INFO:tasks.workunit.client.0.smithi100.stdout:from 8 to 10 diff [2974197~5131400,19095670~6652777,27431447~878329,31769017~7139161,95470042~6051657,106500403~5285199,116025937~3770559,121501610~8078304,182747004~7066023,230813446~6715104,239191876~4760106,250447165~7466990,281234113~5242252,287123044~5448918,323691380~1672817,366341667~7420413] 2024-04-07T17:37:24.668 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2974197~5131400,19095670~6652777,27431447~878329,31769017~7139161,95470042~6051657,106500403~5285199,116025937~3770559,121501610~8078304,182747004~7066023,230813446~6715104,239191876~4760106,250447165~7466990,281234113~5242252,287123044~5448918,323691380~1672817,366341667~7420413] 2024-04-07T17:37:24.675 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~8105597,19095670~6652777,27431447~878329,30516268~8391910,95470042~6051657,106500403~5285199,113977505~15602409,182747004~7066023,230686720~6841830,239191876~4760106,250447165~7466990,281234113~11337849,323691380~1672817,366341667~7420413] 2024-04-07T17:37:24.675 INFO:tasks.workunit.client.0.smithi100.stdout:from 8 to 11 diff [2974197~5131400,19095670~6652777,27431447~878329,31769017~10175249,62264039~1975360,78193873~4609655,95470042~6051657,106500403~5285199,116025937~3770559,121501610~8078304,182747004~7066023,218521286~1672472,222539835~1385676,225244127~12284423,239191876~4760106,250447165~7466990,272896915~6011986,281234113~5242252,287123044~5448918,323691380~1672817,366341667~7420413,377322524~7576585,388479465~3566372,398418332~7708261] 2024-04-07T17:37:24.675 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2974197~5131400,19095670~6652777,27431447~878329,31769017~10175249,62264039~1975360,78193873~4609655,95470042~6051657,106500403~5285199,116025937~3770559,121501610~8078304,182747004~7066023,218521286~1672472,222539835~1385676,225244127~12284423,239191876~4760106,250447165~7466990,272896915~6011986,281234113~5242252,287123044~5448918,323691380~1672817,366341667~7420413,377322524~7576585,388479465~3566372,398418332~7708261] 2024-04-07T17:37:24.683 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~8105597,19095670~6652777,27431447~878329,30516268~11427998,58720256~5519143,75497472~7306056,95470042~6051657,106500403~5285199,113977505~15602409,182747004~7066023,218521286~1672472,222298112~15230438,239191876~4760106,250447165~7466990,272896915~6011986,281234113~11337849,323691380~1672817,366341667~7420413,377322524~7576585,388479465~3566372,398418332~7708261] 2024-04-07T17:37:24.683 INFO:tasks.workunit.client.0.smithi100.stdout:from 8 to 12 diff [2974197~5131400,19095670~6652777,27431447~878329,31769017~13613313,45857929~8483438,62264039~1975360,78193873~4609655,95470042~8094549,106500403~5285199,112275487~7841113,121501610~8078304,182747004~7066023,218521286~1672472,222539835~1385676,225244127~12284423,239191876~4760106,250447165~7466990,272896915~6011986,281234113~5242252,287123044~5448918,294114638~1479706,298036890~6918207,323691380~4417991,366341667~7420413,377322524~7576585,388479465~3566372,398418332~7708261] 2024-04-07T17:37:24.683 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2974197~5131400,19095670~6652777,27431447~878329,31769017~13613313,45857929~8483438,62264039~1975360,78193873~4609655,95470042~8094549,106500403~5285199,112275487~7841113,121501610~8078304,182747004~7066023,218521286~1672472,222539835~1385676,225244127~12284423,239191876~4760106,250447165~7466990,272896915~6011986,281234113~5242252,287123044~5448918,294114638~1479706,298036890~6918207,323691380~4417991,366341667~7420413,377322524~7576585,388479465~3566372,398418332~7708261] 2024-04-07T17:37:24.690 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~8105597,19095670~6652777,27431447~878329,30516268~23825099,58720256~5519143,75497472~7306056,95470042~8094549,106500403~5285199,112275487~17304427,182747004~7066023,218521286~1672472,222298112~15230438,239191876~4760106,250447165~7466990,272896915~6011986,281234113~11337849,294114638~1479706,298036890~6918207,323691380~4417991,366341667~7420413,377322524~7576585,388479465~3566372,398418332~7708261] 2024-04-07T17:37:24.690 INFO:tasks.workunit.client.0.smithi100.stdout:from 8 to 13 diff [2974197~5131400,19095670~6652777,27329013~980763,31769017~13613313,45857929~8483438,62264039~1975360,70298794~6825407,78193873~4609655,95470042~8094549,106500403~5285199,112275487~7841113,121501610~8078304,136270708~2853668,144181083~2188757,182747004~7066023,218521286~1672472,222539835~1385676,225244127~12284423,239191876~4760106,250447165~7466990,267179572~11729329,281234113~5242252,287123044~5448918,294114638~1479706,298036890~6918207,323691380~4417991,366341667~7420413,377322524~7576585,388479465~3566372,394880591~2187459,398418332~7708261,408516200~2525554] 2024-04-07T17:37:24.690 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2974197~5131400,19095670~6652777,27329013~980763,31769017~13613313,45857929~8483438,62264039~1975360,70298794~6825407,78193873~4609655,95470042~8094549,106500403~5285199,112275487~7841113,121501610~8078304,136270708~2853668,144181083~2188757,182747004~7066023,218521286~1672472,222539835~1385676,225244127~12284423,239191876~4760106,250447165~7466990,267179572~11729329,281234113~5242252,287123044~5448918,294114638~1479706,298036890~6918207,323691380~4417991,366341667~7420413,377322524~7576585,388479465~3566372,394880591~2187459,398418332~7708261,408516200~2525554] 2024-04-07T17:37:24.701 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~8105597,19095670~6652777,27329013~980763,30516268~23825099,58720256~5519143,70298794~12504734,95470042~8094549,106500403~5285199,112275487~17304427,134217728~4906648,144181083~2188757,182747004~7066023,218521286~1672472,222298112~15230438,239191876~4760106,250447165~7466990,264241152~14667749,281234113~11337849,294114638~1479706,298036890~6918207,323691380~4417991,366341667~7420413,377322524~7576585,388479465~3566372,394880591~2187459,398418332~7708261,406847488~4194266] 2024-04-07T17:37:24.701 INFO:tasks.workunit.client.0.smithi100.stdout:from 8 to 14 diff [2329752~5775845,18121228~7627219,27329013~980763,31769017~13613313,45857929~8483438,62264039~1975360,70298794~6825407,78193873~4609655,95470042~8094549,106500403~5285199,112275487~7841113,121501610~8758063,136270708~2853668,144181083~2188757,182747004~9590628,213644600~10280911,225244127~12284423,239191876~4760106,250447165~7466990,267179572~11729329,281234113~5242252,287123044~8471300,298036890~6918207,318962927~9146444,366341667~7420413,377322524~7576585,388479465~3566372,394880591~2187459,398418332~7708261,408516200~2525554] 2024-04-07T17:37:24.701 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~5775845,18121228~7627219,27329013~980763,31769017~13613313,45857929~8483438,62264039~1975360,70298794~6825407,78193873~4609655,95470042~8094549,106500403~5285199,112275487~7841113,121501610~8758063,136270708~2853668,144181083~2188757,182747004~9590628,213644600~10280911,225244127~12284423,239191876~4760106,250447165~7466990,267179572~11729329,281234113~5242252,287123044~8471300,298036890~6918207,318962927~9146444,366341667~7420413,377322524~7576585,388479465~3566372,394880591~2187459,398418332~7708261,408516200~2525554] 2024-04-07T17:37:24.707 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~8105597,18121228~7627219,27329013~980763,30516268~23825099,58720256~5519143,70298794~12504734,95470042~8094549,106500403~5285199,112275487~17984186,134217728~4906648,144181083~2188757,182747004~9590628,213644600~23883950,239191876~4760106,250447165~7466990,264241152~14667749,281234113~14360231,298036890~6918207,318962927~9146444,366341667~7420413,377322524~7576585,388479465~3566372,394880591~2187459,398418332~7708261,406847488~4194266] 2024-04-07T17:37:24.707 INFO:tasks.workunit.client.0.smithi100.stdout:from 8 to 15 diff [2329752~5775845,18121228~7627219,27329013~980763,31769017~13613313,45857929~8483438,62264039~1975360,70298794~13107475,95470042~8094549,106500403~5285199,112275487~7841113,121501610~8758063,136270708~2853668,144181083~2188757,168236274~8367316,182747004~9590628,213644600~10966142,225244127~12284423,239191876~9035040,250447165~8059589,267179572~11729329,281234113~5242252,287123044~8471300,298036890~6918207,314393585~13715786,366341667~7420413,377322524~7576585,388479465~3566372,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:24.708 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~5775845,18121228~7627219,27329013~980763,31769017~13613313,45857929~8483438,62264039~1975360,70298794~13107475,95470042~8094549,106500403~5285199,112275487~7841113,121501610~8758063,136270708~2853668,144181083~2188757,168236274~8367316,182747004~9590628,213644600~10966142,225244127~12284423,239191876~9035040,250447165~8059589,267179572~11729329,281234113~5242252,287123044~8471300,298036890~6918207,314393585~13715786,366341667~7420413,377322524~7576585,388479465~3566372,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:24.715 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~8105597,18121228~7627219,27329013~980763,30516268~23825099,58720256~5519143,70298794~13107475,95470042~8094549,106500403~5285199,112275487~17984186,134217728~4906648,144181083~2188757,167772160~8831430,182747004~9590628,213644600~23883950,239191876~9035040,250447165~8059589,264241152~14667749,281234113~14360231,298036890~6918207,310929323~17180048,366341667~7420413,377322524~7576585,388479465~3566372,394880591~2187459,398418332~12623422] 2024-04-07T17:37:24.715 INFO:tasks.workunit.client.0.smithi100.stdout:from 8 to 16 diff [2329752~5775845,14933611~2822456,18121228~7627219,27329013~980763,31769017~13613313,45857929~8483438,62264039~1975360,70298794~13107475,95470042~8094549,106500403~5285199,112275487~7841113,121501610~8758063,136270708~2853668,144181083~2188757,168236274~8367316,182747004~9590628,213644600~10966142,225244127~22982789,250447165~8059589,267179572~11729329,281234113~14360231,298036890~6918207,314393585~13715786,332944411~6919544,366341667~7420413,377322524~10647523,388479465~3566372,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:24.715 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~5775845,14933611~2822456,18121228~7627219,27329013~980763,31769017~13613313,45857929~8483438,62264039~1975360,70298794~13107475,95470042~8094549,106500403~5285199,112275487~7841113,121501610~8758063,136270708~2853668,144181083~2188757,168236274~8367316,182747004~9590628,213644600~10966142,225244127~22982789,250447165~8059589,267179572~11729329,281234113~14360231,298036890~6918207,314393585~13715786,332944411~6919544,366341667~7420413,377322524~10647523,388479465~3566372,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:24.722 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~8105597,12582912~5173155,18121228~7627219,27329013~980763,30516268~23825099,58720256~5519143,70298794~13107475,95470042~8094549,106500403~5285199,112275487~17984186,134217728~4906648,144181083~2188757,167772160~8831430,182747004~9590628,213644600~34582316,250447165~8059589,264241152~14667749,281234113~14360231,298036890~6918207,310929323~17180048,331350016~8513939,366341667~7420413,377322524~10647523,388479465~3566372,394880591~2187459,398418332~12623422] 2024-04-07T17:37:24.722 INFO:tasks.workunit.client.0.smithi100.stdout:from 8 to 17 diff [2329752~5775845,12468100~13280347,27329013~980763,31769017~13613313,45857929~8483438,62264039~1975360,68206575~15199694,95470042~8094549,106500403~5285199,112275487~7841113,121501610~8758063,136270708~5240179,144181083~2188757,168236274~8367316,182747004~9590628,211678512~920259,213644600~34582316,250447165~8059589,267179572~11729329,281234113~14360231,298036890~6918207,314393585~13715786,332944411~6919544,355806290~1463251,366341667~7420413,377322524~10647523,388479465~3566372,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:24.722 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~5775845,12468100~13280347,27329013~980763,31769017~13613313,45857929~8483438,62264039~1975360,68206575~15199694,95470042~8094549,106500403~5285199,112275487~7841113,121501610~8758063,136270708~5240179,144181083~2188757,168236274~8367316,182747004~9590628,211678512~920259,213644600~34582316,250447165~8059589,267179572~11729329,281234113~14360231,298036890~6918207,314393585~13715786,332944411~6919544,355806290~1463251,366341667~7420413,377322524~10647523,388479465~3566372,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:24.729 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~8105597,8388608~17359839,27329013~980763,30516268~23825099,58720256~5519143,68206575~15199694,95470042~8094549,106500403~5285199,112275487~17984186,134217728~7293159,144181083~2188757,167772160~8831430,182747004~9590628,211678512~920259,213644600~34582316,250447165~8059589,264241152~14667749,281234113~14360231,298036890~6918207,310929323~17180048,331350016~8513939,355806290~1463251,366341667~7420413,377322524~10647523,388479465~3566372,394880591~2187459,398418332~12623422] 2024-04-07T17:37:24.729 INFO:tasks.workunit.client.0.smithi100.stdout:from 8 to 18 diff [2329752~9105991,12468100~13280347,27329013~980763,31769017~13613313,45857929~8483438,58033882~7969874,68206575~15199694,95470042~8094549,106500403~5285199,112275487~7841113,121501610~10322918,136270708~5240179,144181083~2188757,168236274~8367316,182747004~9590628,210147535~38079381,250447165~8059589,267179572~11729329,281234113~14360231,298036890~6918207,314393585~13715786,332944411~6919544,355806290~1463251,366341667~7420413,377322524~14723313,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:24.729 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~9105991,12468100~13280347,27329013~980763,31769017~13613313,45857929~8483438,58033882~7969874,68206575~15199694,95470042~8094549,106500403~5285199,112275487~7841113,121501610~10322918,136270708~5240179,144181083~2188757,168236274~8367316,182747004~9590628,210147535~38079381,250447165~8059589,267179572~11729329,281234113~14360231,298036890~6918207,314393585~13715786,332944411~6919544,355806290~1463251,366341667~7420413,377322524~14723313,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:24.735 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~25748447,27329013~980763,30516268~23825099,56705995~9297761,68206575~15199694,95470042~8094549,106500403~5285199,112275487~19549041,134217728~7293159,144181083~2188757,167772160~8831430,182747004~9590628,210147535~38079381,250447165~8059589,264241152~14667749,281234113~14360231,298036890~6918207,310929323~17180048,331350016~8513939,355806290~1463251,366341667~7420413,377322524~14723313,394880591~2187459,398418332~12623422] 2024-04-07T17:37:24.735 INFO:tasks.workunit.client.0.smithi100.stdout:from 8 to 19 diff [2329752~23418695,27329013~980763,31769017~13613313,45857929~8483438,55380210~10623546,68206575~15199694,95470042~8094549,106500403~5285199,112275487~7841113,121501610~10322918,136270708~5240179,141885014~6484550,149552729~4403047,168236274~8367316,182747004~9590628,210147535~38079381,250447165~8059589,267179572~11729329,281234113~14360231,298036890~6918207,306903613~5021411,314393585~13715786,332944411~6919544,355806290~1463251,366341667~7420413,377322524~14723313,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:24.735 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~23418695,27329013~980763,31769017~13613313,45857929~8483438,55380210~10623546,68206575~15199694,95470042~8094549,106500403~5285199,112275487~7841113,121501610~10322918,136270708~5240179,141885014~6484550,149552729~4403047,168236274~8367316,182747004~9590628,210147535~38079381,250447165~8059589,267179572~11729329,281234113~14360231,298036890~6918207,306903613~5021411,314393585~13715786,332944411~6919544,355806290~1463251,366341667~7420413,377322524~14723313,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:24.742 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~25748447,27329013~980763,30516268~23825099,55380210~10623546,68206575~15199694,95470042~8094549,106500403~5285199,112275487~19549041,134217728~7293159,141885014~6484550,149552729~4403047,167772160~8831430,182747004~9590628,210147535~38079381,250447165~8059589,264241152~14667749,281234113~14360231,298036890~6918207,306903613~21205758,331350016~8513939,355806290~1463251,366341667~7420413,377322524~14723313,394880591~2187459,398418332~12623422] 2024-04-07T17:37:24.742 INFO:tasks.workunit.client.0.smithi100.stdout:from 9 to 10 diff [2974197~5131400,27431447~878329,106500403~5285199,118278253~711581,121501610~8078304,186734432~153638,239191876~4760106,287123044~5448918,323691380~1672817,366341667~7420413] 2024-04-07T17:37:24.742 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2974197~5131400,27431447~878329,106500403~5285199,118278253~711581,121501610~8078304,186734432~153638,239191876~4760106,287123044~5448918,323691380~1672817,366341667~7420413] 2024-04-07T17:37:24.750 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~8105597,27431447~878329,106500403~5285199,118278253~711581,119796496~9783418,186734432~153638,239191876~4760106,286476365~6095597,323691380~1672817,366341667~7420413] 2024-04-07T17:37:24.750 INFO:tasks.workunit.client.0.smithi100.stdout:from 9 to 11 diff [2974197~5131400,27431447~878329,37362822~4581444,62264039~1975360,78193873~4609655,106500403~5285199,118278253~711581,121501610~8078304,186734432~153638,218521286~1672472,222539835~1385676,225244127~7197860,239191876~4760106,272896915~6011986,287123044~5448918,323691380~1672817,366341667~7420413,377322524~7576585,388479465~3566372,398418332~7708261] 2024-04-07T17:37:24.750 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2974197~5131400,27431447~878329,37362822~4581444,62264039~1975360,78193873~4609655,106500403~5285199,118278253~711581,121501610~8078304,186734432~153638,218521286~1672472,222539835~1385676,225244127~7197860,239191876~4760106,272896915~6011986,287123044~5448918,323691380~1672817,366341667~7420413,377322524~7576585,388479465~3566372,398418332~7708261] 2024-04-07T17:37:24.758 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~8105597,27431447~878329,37362822~4581444,58720256~5519143,75497472~7306056,106500403~5285199,118278253~711581,119796496~9783418,186734432~153638,218521286~1672472,222298112~10143875,239191876~4760106,272896915~6011986,286476365~6095597,323691380~1672817,366341667~7420413,377322524~7576585,388479465~3566372,398418332~7708261] 2024-04-07T17:37:24.758 INFO:tasks.workunit.client.0.smithi100.stdout:from 9 to 12 diff [2974197~5131400,27431447~878329,37362822~8019508,45857929~8483438,62264039~1975360,78193873~4609655,98683126~4881465,106500403~5285199,112275487~7841113,121501610~8078304,186734432~153638,218521286~1672472,222539835~1385676,225244127~7197860,239191876~4760106,272896915~6011986,287123044~5448918,294114638~1479706,298036890~6918207,323691380~4417991,366341667~7420413,377322524~7576585,388479465~3566372,398418332~7708261] 2024-04-07T17:37:24.758 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2974197~5131400,27431447~878329,37362822~8019508,45857929~8483438,62264039~1975360,78193873~4609655,98683126~4881465,106500403~5285199,112275487~7841113,121501610~8078304,186734432~153638,218521286~1672472,222539835~1385676,225244127~7197860,239191876~4760106,272896915~6011986,287123044~5448918,294114638~1479706,298036890~6918207,323691380~4417991,366341667~7420413,377322524~7576585,388479465~3566372,398418332~7708261] 2024-04-07T17:37:24.766 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~8105597,27431447~878329,37362822~16978545,58720256~5519143,75497472~7306056,98683126~4881465,106500403~5285199,112275487~17304427,186734432~153638,218521286~1672472,222298112~10143875,239191876~4760106,272896915~6011986,286476365~6095597,294114638~1479706,298036890~6918207,323691380~4417991,366341667~7420413,377322524~7576585,388479465~3566372,398418332~7708261] 2024-04-07T17:37:24.766 INFO:tasks.workunit.client.0.smithi100.stdout:from 9 to 13 diff [2974197~5131400,20802751~3327980,27329013~980763,37362822~8019508,45857929~8483438,62264039~1975360,70298794~6825407,78193873~4609655,98683126~4881465,106500403~5285199,112275487~7841113,121501610~8078304,136270708~2853668,144181083~2188757,186734432~153638,218521286~1672472,222539835~1385676,225244127~7197860,239191876~4760106,267179572~11729329,287123044~5448918,294114638~1479706,298036890~6918207,323691380~4417991,366341667~7420413,377322524~7576585,388479465~3566372,394880591~2187459,398418332~7708261,408516200~2525554] 2024-04-07T17:37:24.766 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2974197~5131400,20802751~3327980,27329013~980763,37362822~8019508,45857929~8483438,62264039~1975360,70298794~6825407,78193873~4609655,98683126~4881465,106500403~5285199,112275487~7841113,121501610~8078304,136270708~2853668,144181083~2188757,186734432~153638,218521286~1672472,222539835~1385676,225244127~7197860,239191876~4760106,267179572~11729329,287123044~5448918,294114638~1479706,298036890~6918207,323691380~4417991,366341667~7420413,377322524~7576585,388479465~3566372,394880591~2187459,398418332~7708261,408516200~2525554] 2024-04-07T17:37:24.772 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~8105597,20802751~3327980,27329013~980763,37362822~16978545,58720256~5519143,70298794~12504734,98683126~4881465,106500403~5285199,112275487~17304427,134217728~4906648,144181083~2188757,186734432~153638,218521286~1672472,222298112~10143875,239191876~4760106,264241152~14667749,286476365~6095597,294114638~1479706,298036890~6918207,323691380~4417991,366341667~7420413,377322524~7576585,388479465~3566372,394880591~2187459,398418332~7708261,406847488~4194266] 2024-04-07T17:37:24.772 INFO:tasks.workunit.client.0.smithi100.stdout:from 9 to 14 diff [2329752~5775845,18121228~6009503,27329013~980763,37362822~8019508,45857929~8483438,62264039~1975360,70298794~6825407,78193873~4609655,98683126~4881465,106500403~5285199,112275487~7841113,121501610~8758063,136270708~2853668,144181083~2188757,186734432~153638,187506017~4831615,213644600~10280911,225244127~7197860,239191876~4760106,267179572~11729329,287123044~8471300,298036890~6918207,318962927~9146444,366341667~7420413,377322524~7576585,388479465~3566372,394880591~2187459,398418332~7708261,408516200~2525554] 2024-04-07T17:37:24.772 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~5775845,18121228~6009503,27329013~980763,37362822~8019508,45857929~8483438,62264039~1975360,70298794~6825407,78193873~4609655,98683126~4881465,106500403~5285199,112275487~7841113,121501610~8758063,136270708~2853668,144181083~2188757,186734432~153638,187506017~4831615,213644600~10280911,225244127~7197860,239191876~4760106,267179572~11729329,287123044~8471300,298036890~6918207,318962927~9146444,366341667~7420413,377322524~7576585,388479465~3566372,394880591~2187459,398418332~7708261,408516200~2525554] 2024-04-07T17:37:24.779 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~8105597,18121228~6009503,27329013~980763,37362822~16978545,58720256~5519143,70298794~12504734,98683126~4881465,106500403~5285199,112275487~17984186,134217728~4906648,144181083~2188757,186734432~153638,187506017~4831615,213644600~18797387,239191876~4760106,264241152~14667749,286476365~9117979,298036890~6918207,318962927~9146444,366341667~7420413,377322524~7576585,388479465~3566372,394880591~2187459,398418332~7708261,406847488~4194266] 2024-04-07T17:37:24.779 INFO:tasks.workunit.client.0.smithi100.stdout:from 9 to 15 diff [2329752~5775845,18121228~6009503,27329013~980763,37362822~8019508,45857929~8483438,62264039~1975360,70298794~13107475,98683126~4881465,106500403~5285199,112275487~7841113,121501610~8758063,136270708~2853668,144181083~2188757,168236274~8367316,186734432~153638,187506017~4831615,213644600~10966142,225244127~7197860,239191876~9035040,255043666~3463088,267179572~11729329,287123044~8471300,298036890~6918207,314393585~13715786,366341667~7420413,377322524~7576585,388479465~3566372,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:24.780 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~5775845,18121228~6009503,27329013~980763,37362822~8019508,45857929~8483438,62264039~1975360,70298794~13107475,98683126~4881465,106500403~5285199,112275487~7841113,121501610~8758063,136270708~2853668,144181083~2188757,168236274~8367316,186734432~153638,187506017~4831615,213644600~10966142,225244127~7197860,239191876~9035040,255043666~3463088,267179572~11729329,287123044~8471300,298036890~6918207,314393585~13715786,366341667~7420413,377322524~7576585,388479465~3566372,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:24.787 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~8105597,18121228~6009503,27329013~980763,37362822~16978545,58720256~5519143,70298794~13107475,98683126~4881465,106500403~5285199,112275487~17984186,134217728~4906648,144181083~2188757,167772160~8831430,186734432~153638,187506017~4831615,213644600~18797387,239191876~9035040,255043666~3463088,264241152~14667749,286476365~9117979,298036890~6918207,310929323~17180048,366341667~7420413,377322524~7576585,388479465~3566372,394880591~2187459,398418332~12623422] 2024-04-07T17:37:24.787 INFO:tasks.workunit.client.0.smithi100.stdout:from 9 to 16 diff [2329752~5775845,14933611~2822456,18121228~6009503,27329013~980763,37362822~8019508,45857929~8483438,62264039~1975360,70298794~13107475,95781519~7783072,106500403~5285199,112275487~7841113,121501610~8758063,136270708~2853668,144181083~2188757,168236274~8367316,186734432~153638,187506017~4831615,213644600~10966142,225244127~7197860,235932092~12294824,254075762~4430992,267179572~11729329,286319045~9275299,298036890~6918207,314393585~13715786,332944411~6919544,366341667~7420413,377322524~10647523,388479465~3566372,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:24.787 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~5775845,14933611~2822456,18121228~6009503,27329013~980763,37362822~8019508,45857929~8483438,62264039~1975360,70298794~13107475,95781519~7783072,106500403~5285199,112275487~7841113,121501610~8758063,136270708~2853668,144181083~2188757,168236274~8367316,186734432~153638,187506017~4831615,213644600~10966142,225244127~7197860,235932092~12294824,254075762~4430992,267179572~11729329,286319045~9275299,298036890~6918207,314393585~13715786,332944411~6919544,366341667~7420413,377322524~10647523,388479465~3566372,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:24.794 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~8105597,12582912~5173155,18121228~6009503,27329013~980763,37362822~16978545,58720256~5519143,70298794~13107475,95781519~7783072,106500403~5285199,112275487~17984186,134217728~4906648,144181083~2188757,167772160~8831430,186734432~153638,187506017~4831615,213644600~18797387,235932092~12294824,254075762~4430992,264241152~14667749,286319045~9275299,298036890~6918207,310929323~17180048,331350016~8513939,366341667~7420413,377322524~10647523,388479465~3566372,394880591~2187459,398418332~12623422] 2024-04-07T17:37:24.794 INFO:tasks.workunit.client.0.smithi100.stdout:from 9 to 17 diff [2329752~5775845,12468100~11662631,27329013~980763,37362822~8019508,45857929~8483438,62264039~1975360,68206575~15199694,95781519~7783072,106500403~5285199,112275487~7841113,121501610~8758063,136270708~5240179,144181083~2188757,168236274~8367316,186734432~153638,187506017~4831615,211678512~920259,213644600~18797387,235932092~12294824,254075762~4430992,267179572~11729329,282485483~13108861,298036890~6918207,314393585~13715786,332944411~6919544,355806290~1463251,366341667~7420413,377322524~10647523,388479465~3566372,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:24.794 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~5775845,12468100~11662631,27329013~980763,37362822~8019508,45857929~8483438,62264039~1975360,68206575~15199694,95781519~7783072,106500403~5285199,112275487~7841113,121501610~8758063,136270708~5240179,144181083~2188757,168236274~8367316,186734432~153638,187506017~4831615,211678512~920259,213644600~18797387,235932092~12294824,254075762~4430992,267179572~11729329,282485483~13108861,298036890~6918207,314393585~13715786,332944411~6919544,355806290~1463251,366341667~7420413,377322524~10647523,388479465~3566372,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:24.801 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~8105597,8388608~15742123,27329013~980763,37362822~16978545,58720256~5519143,68206575~15199694,95781519~7783072,106500403~5285199,112275487~17984186,134217728~7293159,144181083~2188757,167772160~8831430,186734432~153638,187506017~4831615,211678512~920259,213644600~18797387,235932092~12294824,254075762~4430992,264241152~14667749,282485483~13108861,298036890~6918207,310929323~17180048,331350016~8513939,355806290~1463251,366341667~7420413,377322524~10647523,388479465~3566372,394880591~2187459,398418332~12623422] 2024-04-07T17:37:24.801 INFO:tasks.workunit.client.0.smithi100.stdout:from 9 to 18 diff [2329752~9105991,12468100~11662631,27329013~980763,37362822~8019508,45857929~8483438,58033882~7969874,68206575~15199694,95781519~7783072,106500403~5285199,112275487~7841113,121501610~10322918,136270708~5240179,144181083~2188757,168236274~8367316,183274874~3283813,186734432~153638,187506017~4831615,210147535~38079381,254075762~4430992,267179572~11729329,282485483~13108861,298036890~6918207,314393585~13715786,332944411~6919544,355806290~1463251,366341667~7420413,377322524~14723313,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:24.801 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~9105991,12468100~11662631,27329013~980763,37362822~8019508,45857929~8483438,58033882~7969874,68206575~15199694,95781519~7783072,106500403~5285199,112275487~7841113,121501610~10322918,136270708~5240179,144181083~2188757,168236274~8367316,183274874~3283813,186734432~153638,187506017~4831615,210147535~38079381,254075762~4430992,267179572~11729329,282485483~13108861,298036890~6918207,314393585~13715786,332944411~6919544,355806290~1463251,366341667~7420413,377322524~14723313,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:24.807 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~24130731,27329013~980763,37362822~16978545,56705995~9297761,68206575~15199694,95781519~7783072,106500403~5285199,112275487~19549041,134217728~7293159,144181083~2188757,167772160~8831430,183274874~3283813,186734432~153638,187506017~4831615,210147535~38079381,254075762~4430992,264241152~14667749,282485483~13108861,298036890~6918207,310929323~17180048,331350016~8513939,355806290~1463251,366341667~7420413,377322524~14723313,394880591~2187459,398418332~12623422] 2024-04-07T17:37:24.807 INFO:tasks.workunit.client.0.smithi100.stdout:from 9 to 19 diff [2329752~21800979,27329013~980763,37362822~8019508,45857929~8483438,55380210~10623546,68206575~15199694,95781519~7783072,106500403~5285199,112275487~7841113,121501610~10322918,136270708~5240179,141885014~6484550,149552729~4403047,168236274~8367316,183274874~3283813,186734432~153638,187506017~4831615,210147535~38079381,254075762~4430992,267179572~11729329,282485483~13108861,298036890~6918207,306903613~5021411,314393585~13715786,332944411~6919544,355806290~1463251,366341667~7420413,377322524~14723313,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:24.807 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~21800979,27329013~980763,37362822~8019508,45857929~8483438,55380210~10623546,68206575~15199694,95781519~7783072,106500403~5285199,112275487~7841113,121501610~10322918,136270708~5240179,141885014~6484550,149552729~4403047,168236274~8367316,183274874~3283813,186734432~153638,187506017~4831615,210147535~38079381,254075762~4430992,267179572~11729329,282485483~13108861,298036890~6918207,306903613~5021411,314393585~13715786,332944411~6919544,355806290~1463251,366341667~7420413,377322524~14723313,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:24.814 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~24130731,27329013~980763,37362822~16978545,55380210~10623546,68206575~15199694,95781519~7783072,106500403~5285199,112275487~19549041,134217728~7293159,141885014~6484550,149552729~4403047,167772160~8831430,183274874~3283813,186734432~153638,187506017~4831615,210147535~38079381,254075762~4430992,264241152~14667749,282485483~13108861,298036890~6918207,306903613~21205758,331350016~8513939,355806290~1463251,366341667~7420413,377322524~14723313,394880591~2187459,398418332~12623422] 2024-04-07T17:37:24.814 INFO:tasks.workunit.client.0.smithi100.stdout:from 10 to 11 diff [37362822~4581444,62264039~1975360,78193873~4609655,218521286~1672472,222539835~1385676,225244127~7197860,272896915~6011986,377322524~7576585,388479465~3566372,398418332~7708261] 2024-04-07T17:37:24.814 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [37362822~4581444,62264039~1975360,78193873~4609655,218521286~1672472,222539835~1385676,225244127~7197860,272896915~6011986,377322524~7576585,388479465~3566372,398418332~7708261] 2024-04-07T17:37:24.822 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [37362822~4581444,58720256~5519143,75497472~7306056,218521286~1672472,222298112~10143875,272896915~6011986,377322524~7576585,388479465~3566372,398418332~7708261] 2024-04-07T17:37:24.822 INFO:tasks.workunit.client.0.smithi100.stdout:from 10 to 12 diff [37362822~8019508,45857929~8483438,62264039~1975360,78193873~4609655,98683126~4881465,112275487~7841113,218521286~1672472,222539835~1385676,225244127~7197860,272896915~6011986,294114638~1479706,298036890~6918207,324984001~3125370,377322524~7576585,388479465~3566372,398418332~7708261] 2024-04-07T17:37:24.822 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [37362822~8019508,45857929~8483438,62264039~1975360,78193873~4609655,98683126~4881465,112275487~7841113,218521286~1672472,222539835~1385676,225244127~7197860,272896915~6011986,294114638~1479706,298036890~6918207,324984001~3125370,377322524~7576585,388479465~3566372,398418332~7708261] 2024-04-07T17:37:24.829 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [37362822~16978545,58720256~5519143,75497472~7306056,98683126~4881465,112275487~7841113,218521286~1672472,222298112~10143875,272896915~6011986,294114638~1479706,298036890~6918207,324984001~3125370,377322524~7576585,388479465~3566372,398418332~7708261] 2024-04-07T17:37:24.829 INFO:tasks.workunit.client.0.smithi100.stdout:from 10 to 13 diff [20802751~3327980,27329013~249441,37362822~8019508,45857929~8483438,62264039~1975360,70298794~6825407,78193873~4609655,98683126~4881465,112275487~7841113,136270708~2853668,144181083~2188757,218521286~1672472,222539835~1385676,225244127~7197860,267179572~11729329,294114638~1479706,298036890~6918207,324984001~3125370,377322524~7576585,388479465~3566372,394880591~2187459,398418332~7708261,408516200~2525554] 2024-04-07T17:37:24.830 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [20802751~3327980,27329013~249441,37362822~8019508,45857929~8483438,62264039~1975360,70298794~6825407,78193873~4609655,98683126~4881465,112275487~7841113,136270708~2853668,144181083~2188757,218521286~1672472,222539835~1385676,225244127~7197860,267179572~11729329,294114638~1479706,298036890~6918207,324984001~3125370,377322524~7576585,388479465~3566372,394880591~2187459,398418332~7708261,408516200~2525554] 2024-04-07T17:37:24.836 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [20802751~3327980,27329013~249441,37362822~16978545,58720256~5519143,70298794~12504734,98683126~4881465,112275487~7841113,134217728~4906648,144181083~2188757,218521286~1672472,222298112~10143875,264241152~14667749,294114638~1479706,298036890~6918207,324984001~3125370,377322524~7576585,388479465~3566372,394880591~2187459,398418332~7708261,406847488~4194266] 2024-04-07T17:37:24.837 INFO:tasks.workunit.client.0.smithi100.stdout:from 10 to 14 diff [2329752~961414,18121228~6009503,27329013~249441,37362822~8019508,45857929~8483438,62264039~1975360,70298794~6825407,78193873~4609655,98683126~4881465,112275487~7841113,124240937~6018736,136270708~2853668,144181083~2188757,187506017~4831615,213644600~10280911,225244127~7197860,267179572~11729329,288216273~7378071,298036890~6918207,318962927~9146444,377322524~7576585,388479465~3566372,394880591~2187459,398418332~7708261,408516200~2525554] 2024-04-07T17:37:24.837 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~961414,18121228~6009503,27329013~249441,37362822~8019508,45857929~8483438,62264039~1975360,70298794~6825407,78193873~4609655,98683126~4881465,112275487~7841113,124240937~6018736,136270708~2853668,144181083~2188757,187506017~4831615,213644600~10280911,225244127~7197860,267179572~11729329,288216273~7378071,298036890~6918207,318962927~9146444,377322524~7576585,388479465~3566372,394880591~2187459,398418332~7708261,408516200~2525554] 2024-04-07T17:37:24.843 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [2329752~961414,18121228~6009503,27329013~249441,37362822~16978545,58720256~5519143,70298794~12504734,98683126~4881465,112275487~7841113,124240937~6018736,134217728~4906648,144181083~2188757,187506017~4831615,213644600~18797387,264241152~14667749,288216273~7378071,298036890~6918207,318962927~9146444,377322524~7576585,388479465~3566372,394880591~2187459,398418332~7708261,406847488~4194266] 2024-04-07T17:37:24.843 INFO:tasks.workunit.client.0.smithi100.stdout:from 10 to 15 diff [2329752~961414,18121228~6009503,27329013~249441,37362822~8019508,45857929~8483438,62264039~1975360,70298794~13107475,98683126~4881465,112275487~7841113,124240937~6018736,136270708~2853668,144181083~2188757,168236274~8367316,187506017~4831615,213644600~10966142,225244127~7197860,242402883~5824033,255043666~3463088,267179572~11729329,288216273~7378071,298036890~6918207,314393585~13715786,377322524~7576585,388479465~3566372,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:24.843 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~961414,18121228~6009503,27329013~249441,37362822~8019508,45857929~8483438,62264039~1975360,70298794~13107475,98683126~4881465,112275487~7841113,124240937~6018736,136270708~2853668,144181083~2188757,168236274~8367316,187506017~4831615,213644600~10966142,225244127~7197860,242402883~5824033,255043666~3463088,267179572~11729329,288216273~7378071,298036890~6918207,314393585~13715786,377322524~7576585,388479465~3566372,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:24.852 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [2329752~961414,18121228~6009503,27329013~249441,37362822~16978545,58720256~5519143,70298794~13107475,98683126~4881465,112275487~7841113,124240937~6018736,134217728~4906648,144181083~2188757,167772160~8831430,187506017~4831615,213644600~18797387,242402883~5824033,255043666~3463088,264241152~14667749,288216273~7378071,298036890~6918207,310929323~17180048,377322524~7576585,388479465~3566372,394880591~2187459,398418332~12623422] 2024-04-07T17:37:24.852 INFO:tasks.workunit.client.0.smithi100.stdout:from 10 to 16 diff [2329752~4625179,14933611~2822456,18121228~6009503,27329013~249441,37362822~8019508,45857929~8483438,62264039~1975360,70298794~13107475,95781519~7783072,112275487~7841113,124240937~6018736,136270708~2853668,144181083~2188757,168236274~8367316,187506017~4831615,213644600~10966142,225244127~7197860,235932092~5747954,242402883~5824033,254075762~4430992,267179572~11729329,286319045~9275299,298036890~6918207,314393585~13715786,332944411~6919544,377322524~10647523,388479465~3566372,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:24.852 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~4625179,14933611~2822456,18121228~6009503,27329013~249441,37362822~8019508,45857929~8483438,62264039~1975360,70298794~13107475,95781519~7783072,112275487~7841113,124240937~6018736,136270708~2853668,144181083~2188757,168236274~8367316,187506017~4831615,213644600~10966142,225244127~7197860,235932092~5747954,242402883~5824033,254075762~4430992,267179572~11729329,286319045~9275299,298036890~6918207,314393585~13715786,332944411~6919544,377322524~10647523,388479465~3566372,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:24.860 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [2329752~4625179,12582912~5173155,18121228~6009503,27329013~249441,37362822~16978545,58720256~5519143,70298794~13107475,95781519~7783072,112275487~7841113,124240937~6018736,134217728~4906648,144181083~2188757,167772160~8831430,187506017~4831615,213644600~18797387,235932092~5747954,242402883~5824033,254075762~4430992,264241152~14667749,286319045~9275299,298036890~6918207,310929323~17180048,331350016~8513939,377322524~10647523,388479465~3566372,394880591~2187459,398418332~12623422] 2024-04-07T17:37:24.860 INFO:tasks.workunit.client.0.smithi100.stdout:from 10 to 17 diff [2329752~4625179,12468100~11662631,27329013~249441,37362822~8019508,45857929~8483438,62264039~1975360,68206575~15199694,95781519~7783072,112275487~7841113,124240937~6018736,136270708~5240179,144181083~2188757,168236274~8367316,187506017~4831615,211678512~920259,213644600~18797387,235932092~5747954,242402883~5824033,254075762~4430992,267179572~11729329,282485483~13108861,298036890~6918207,314393585~13715786,332944411~6919544,355806290~1463251,377322524~10647523,388479465~3566372,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:24.860 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~4625179,12468100~11662631,27329013~249441,37362822~8019508,45857929~8483438,62264039~1975360,68206575~15199694,95781519~7783072,112275487~7841113,124240937~6018736,136270708~5240179,144181083~2188757,168236274~8367316,187506017~4831615,211678512~920259,213644600~18797387,235932092~5747954,242402883~5824033,254075762~4430992,267179572~11729329,282485483~13108861,298036890~6918207,314393585~13715786,332944411~6919544,355806290~1463251,377322524~10647523,388479465~3566372,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:24.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:37:24 smithi144 bash[66654]: cluster 2024-04-07T17:37:23.572708+0000 mon.a (mon.0) 3885 : cluster [DBG] osdmap e2012: 8 total, 8 up, 8 in 2024-04-07T17:37:24.868 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [2329752~4625179,8388608~15742123,27329013~249441,37362822~16978545,58720256~5519143,68206575~15199694,95781519~7783072,112275487~7841113,124240937~6018736,134217728~7293159,144181083~2188757,167772160~8831430,187506017~4831615,211678512~920259,213644600~18797387,235932092~5747954,242402883~5824033,254075762~4430992,264241152~14667749,282485483~13108861,298036890~6918207,310929323~17180048,331350016~8513939,355806290~1463251,377322524~10647523,388479465~3566372,394880591~2187459,398418332~12623422] 2024-04-07T17:37:24.868 INFO:tasks.workunit.client.0.smithi100.stdout:from 10 to 18 diff [2329752~9105991,12468100~11662631,27329013~249441,37362822~8019508,45857929~8483438,58033882~7969874,68206575~15199694,95781519~7783072,112275487~7841113,124240937~7583591,136270708~5240179,144181083~2188757,168236274~8367316,183274874~3283813,187506017~4831615,210147535~38079381,254075762~4430992,267179572~11729329,282485483~13108861,298036890~6918207,314393585~13715786,332944411~6919544,355806290~1463251,377322524~14723313,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:24.868 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~9105991,12468100~11662631,27329013~249441,37362822~8019508,45857929~8483438,58033882~7969874,68206575~15199694,95781519~7783072,112275487~7841113,124240937~7583591,136270708~5240179,144181083~2188757,168236274~8367316,183274874~3283813,187506017~4831615,210147535~38079381,254075762~4430992,267179572~11729329,282485483~13108861,298036890~6918207,314393585~13715786,332944411~6919544,355806290~1463251,377322524~14723313,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:24.875 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [2329752~21800979,27329013~249441,37362822~16978545,56705995~9297761,68206575~15199694,95781519~7783072,112275487~7841113,124240937~7583591,134217728~7293159,144181083~2188757,167772160~8831430,183274874~3283813,187506017~4831615,210147535~38079381,254075762~4430992,264241152~14667749,282485483~13108861,298036890~6918207,310929323~17180048,331350016~8513939,355806290~1463251,377322524~14723313,394880591~2187459,398418332~12623422] 2024-04-07T17:37:24.875 INFO:tasks.workunit.client.0.smithi100.stdout:from 10 to 19 diff [2329752~21800979,27329013~249441,37362822~8019508,45857929~8483438,55380210~10623546,68206575~15199694,95781519~7783072,112275487~7841113,124240937~7583591,136270708~5240179,141885014~6484550,149552729~4403047,168236274~8367316,183274874~3283813,187506017~4831615,210147535~38079381,254075762~4430992,267179572~11729329,282485483~13108861,298036890~6918207,306903613~5021411,314393585~13715786,332944411~6919544,355806290~1463251,377322524~14723313,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:24.875 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~21800979,27329013~249441,37362822~8019508,45857929~8483438,55380210~10623546,68206575~15199694,95781519~7783072,112275487~7841113,124240937~7583591,136270708~5240179,141885014~6484550,149552729~4403047,168236274~8367316,183274874~3283813,187506017~4831615,210147535~38079381,254075762~4430992,267179572~11729329,282485483~13108861,298036890~6918207,306903613~5021411,314393585~13715786,332944411~6919544,355806290~1463251,377322524~14723313,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:24.882 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [2329752~21800979,27329013~249441,37362822~16978545,55380210~10623546,68206575~15199694,95781519~7783072,112275487~7841113,124240937~7583591,134217728~7293159,141885014~6484550,149552729~4403047,167772160~8831430,183274874~3283813,187506017~4831615,210147535~38079381,254075762~4430992,264241152~14667749,282485483~13108861,298036890~6918207,306903613~21205758,331350016~8513939,355806290~1463251,377322524~14723313,394880591~2187459,398418332~12623422] 2024-04-07T17:37:24.882 INFO:tasks.workunit.client.0.smithi100.stdout:from 11 to 12 diff [37386125~7996205,45857929~8483438,98683126~4881465,112275487~7841113,294114638~1479706,298036890~6918207,324984001~3125370,380094867~2863217,400878444~1331564] 2024-04-07T17:37:24.882 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [37386125~7996205,45857929~8483438,98683126~4881465,112275487~7841113,294114638~1479706,298036890~6918207,324984001~3125370,380094867~2863217,400878444~1331564] 2024-04-07T17:37:24.889 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [37386125~16955242,98683126~4881465,112275487~7841113,294114638~1479706,298036890~6918207,324984001~3125370,380094867~2863217,400878444~1331564] 2024-04-07T17:37:24.889 INFO:tasks.workunit.client.0.smithi100.stdout:from 11 to 13 diff [20802751~3327980,27329013~249441,37386125~7996205,45857929~8483438,70298794~6825407,98683126~4881465,112275487~7841113,136270708~2853668,144181083~2188757,267179572~6811128,294114638~1479706,298036890~6918207,324984001~3125370,380094867~2863217,394880591~2187459,400878444~1331564,403289909~2301039,408516200~2525554] 2024-04-07T17:37:24.889 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [20802751~3327980,27329013~249441,37386125~7996205,45857929~8483438,70298794~6825407,98683126~4881465,112275487~7841113,136270708~2853668,144181083~2188757,267179572~6811128,294114638~1479706,298036890~6918207,324984001~3125370,380094867~2863217,394880591~2187459,400878444~1331564,403289909~2301039,408516200~2525554] 2024-04-07T17:37:24.896 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [20802751~3327980,27329013~249441,37386125~16955242,70298794~6825407,98683126~4881465,112275487~7841113,134217728~4906648,144181083~2188757,264241152~9749548,294114638~1479706,298036890~6918207,324984001~3125370,380094867~2863217,394880591~2187459,400878444~1331564,403289909~2301039,406847488~4194266] 2024-04-07T17:37:24.897 INFO:tasks.workunit.client.0.smithi100.stdout:from 11 to 14 diff [2329752~961414,18121228~6009503,27329013~249441,37386125~7996205,45857929~8483438,70298794~6825407,98683126~4881465,112275487~7841113,124240937~6018736,136270708~2853668,144181083~2188757,187506017~4831615,213644600~9382452,267179572~6811128,288216273~7378071,298036890~6918207,318962927~9146444,380094867~2863217,394880591~2187459,400878444~1331564,403289909~2301039,408516200~2525554] 2024-04-07T17:37:24.897 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~961414,18121228~6009503,27329013~249441,37386125~7996205,45857929~8483438,70298794~6825407,98683126~4881465,112275487~7841113,124240937~6018736,136270708~2853668,144181083~2188757,187506017~4831615,213644600~9382452,267179572~6811128,288216273~7378071,298036890~6918207,318962927~9146444,380094867~2863217,394880591~2187459,400878444~1331564,403289909~2301039,408516200~2525554] 2024-04-07T17:37:24.903 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [2329752~961414,18121228~6009503,27329013~249441,37386125~16955242,70298794~6825407,98683126~4881465,112275487~7841113,124240937~6018736,134217728~4906648,144181083~2188757,187506017~4831615,213644600~9382452,264241152~9749548,288216273~7378071,298036890~6918207,318962927~9146444,380094867~2863217,394880591~2187459,400878444~1331564,403289909~2301039,406847488~4194266] 2024-04-07T17:37:24.903 INFO:tasks.workunit.client.0.smithi100.stdout:from 11 to 15 diff [2329752~961414,18121228~6009503,27329013~249441,37386125~7996205,45857929~8483438,70298794~13107475,98683126~4881465,112275487~7841113,124240937~6018736,136270708~2853668,144181083~2188757,168236274~8367316,187506017~4831615,213644600~10966142,242402883~5824033,255043666~3463088,267179572~6811128,288216273~7378071,298036890~6918207,314393585~13715786,380094867~2863217,394880591~2187459,400878444~6009875,408516200~2525554] 2024-04-07T17:37:24.903 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~961414,18121228~6009503,27329013~249441,37386125~7996205,45857929~8483438,70298794~13107475,98683126~4881465,112275487~7841113,124240937~6018736,136270708~2853668,144181083~2188757,168236274~8367316,187506017~4831615,213644600~10966142,242402883~5824033,255043666~3463088,267179572~6811128,288216273~7378071,298036890~6918207,314393585~13715786,380094867~2863217,394880591~2187459,400878444~6009875,408516200~2525554] 2024-04-07T17:37:24.909 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [2329752~961414,18121228~6009503,27329013~249441,37386125~16955242,70298794~13107475,98683126~4881465,112275487~7841113,124240937~6018736,134217728~4906648,144181083~2188757,167772160~8831430,187506017~4831615,213644600~10966142,242402883~5824033,255043666~3463088,264241152~9749548,288216273~7378071,298036890~6918207,310929323~17180048,380094867~2863217,394880591~2187459,400878444~10163310] 2024-04-07T17:37:24.910 INFO:tasks.workunit.client.0.smithi100.stdout:from 11 to 16 diff [2329752~4625179,14933611~2822456,18121228~6009503,27329013~249441,37386125~7996205,45857929~8483438,70298794~13107475,95781519~7783072,112275487~7841113,124240937~6018736,136270708~2853668,144181083~2188757,168236274~8367316,187506017~4831615,213644600~10966142,235932092~5747954,242402883~5824033,254075762~4430992,267179572~6811128,286319045~9275299,298036890~6918207,314393585~13715786,332944411~6919544,380094867~7875180,394880591~2187459,400878444~6009875,408516200~2525554] 2024-04-07T17:37:24.910 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~4625179,14933611~2822456,18121228~6009503,27329013~249441,37386125~7996205,45857929~8483438,70298794~13107475,95781519~7783072,112275487~7841113,124240937~6018736,136270708~2853668,144181083~2188757,168236274~8367316,187506017~4831615,213644600~10966142,235932092~5747954,242402883~5824033,254075762~4430992,267179572~6811128,286319045~9275299,298036890~6918207,314393585~13715786,332944411~6919544,380094867~7875180,394880591~2187459,400878444~6009875,408516200~2525554] 2024-04-07T17:37:24.918 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [2329752~4625179,12582912~5173155,18121228~6009503,27329013~249441,37386125~16955242,70298794~13107475,95781519~7783072,112275487~7841113,124240937~6018736,134217728~4906648,144181083~2188757,167772160~8831430,187506017~4831615,213644600~10966142,235932092~5747954,242402883~5824033,254075762~4430992,264241152~9749548,286319045~9275299,298036890~6918207,310929323~17180048,331350016~8513939,380094867~7875180,394880591~2187459,400878444~10163310] 2024-04-07T17:37:24.918 INFO:tasks.workunit.client.0.smithi100.stdout:from 11 to 17 diff [2329752~4625179,12468100~11662631,27329013~249441,37386125~7996205,45857929~8483438,68206575~15199694,95781519~7783072,112275487~7841113,124240937~6018736,136270708~5240179,144181083~2188757,168236274~8367316,187506017~4831615,211678512~920259,213644600~15909515,235932092~5747954,242402883~5824033,254075762~4430992,267179572~6811128,282485483~13108861,298036890~6918207,314393585~13715786,332944411~6919544,355806290~1463251,380094867~7875180,394880591~2187459,400878444~6009875,408516200~2525554] 2024-04-07T17:37:24.919 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~4625179,12468100~11662631,27329013~249441,37386125~7996205,45857929~8483438,68206575~15199694,95781519~7783072,112275487~7841113,124240937~6018736,136270708~5240179,144181083~2188757,168236274~8367316,187506017~4831615,211678512~920259,213644600~15909515,235932092~5747954,242402883~5824033,254075762~4430992,267179572~6811128,282485483~13108861,298036890~6918207,314393585~13715786,332944411~6919544,355806290~1463251,380094867~7875180,394880591~2187459,400878444~6009875,408516200~2525554] 2024-04-07T17:37:24.925 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [2329752~4625179,8388608~15742123,27329013~249441,37386125~16955242,68206575~15199694,95781519~7783072,112275487~7841113,124240937~6018736,134217728~7293159,144181083~2188757,167772160~8831430,187506017~4831615,211678512~920259,213644600~15909515,235932092~5747954,242402883~5824033,254075762~4430992,264241152~9749548,282485483~13108861,298036890~6918207,310929323~17180048,331350016~8513939,355806290~1463251,380094867~7875180,394880591~2187459,400878444~10163310] 2024-04-07T17:37:24.925 INFO:tasks.workunit.client.0.smithi100.stdout:from 11 to 18 diff [2329752~9105991,12468100~11662631,27329013~249441,37386125~7996205,45857929~8483438,58033882~7969874,68206575~15199694,95781519~7783072,112275487~7841113,124240937~7583591,136270708~5240179,144181083~2188757,168236274~8367316,183274874~3283813,187506017~4831615,210147535~19406580,229742904~18484012,254075762~4430992,267179572~6811128,282485483~13108861,298036890~6918207,314393585~13715786,332944411~6919544,355806290~1463251,380094867~8701052,394880591~2187459,400878444~6009875,408516200~2525554] 2024-04-07T17:37:24.925 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~9105991,12468100~11662631,27329013~249441,37386125~7996205,45857929~8483438,58033882~7969874,68206575~15199694,95781519~7783072,112275487~7841113,124240937~7583591,136270708~5240179,144181083~2188757,168236274~8367316,183274874~3283813,187506017~4831615,210147535~19406580,229742904~18484012,254075762~4430992,267179572~6811128,282485483~13108861,298036890~6918207,314393585~13715786,332944411~6919544,355806290~1463251,380094867~8701052,394880591~2187459,400878444~6009875,408516200~2525554] 2024-04-07T17:37:24.931 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [2329752~21800979,27329013~249441,37386125~16955242,56705995~9297761,68206575~15199694,95781519~7783072,112275487~7841113,124240937~7583591,134217728~7293159,144181083~2188757,167772160~8831430,183274874~3283813,187506017~4831615,210147535~19406580,229742904~18484012,254075762~4430992,264241152~9749548,282485483~13108861,298036890~6918207,310929323~17180048,331350016~8513939,355806290~1463251,380094867~8701052,394880591~2187459,400878444~10163310] 2024-04-07T17:37:24.931 INFO:tasks.workunit.client.0.smithi100.stdout:from 11 to 19 diff [2329752~21800979,27329013~249441,37386125~7996205,45857929~8483438,55380210~10623546,68206575~15199694,95781519~7783072,112275487~7841113,124240937~7583591,136270708~5240179,141885014~6484550,149552729~4403047,168236274~8367316,183274874~3283813,187506017~4831615,210147535~19406580,229742904~18484012,254075762~4430992,267179572~6811128,282485483~13108861,298036890~6918207,306903613~5021411,314393585~13715786,332944411~6919544,355806290~1463251,380094867~8701052,394880591~2187459,400878444~6009875,408516200~2525554] 2024-04-07T17:37:24.931 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~21800979,27329013~249441,37386125~7996205,45857929~8483438,55380210~10623546,68206575~15199694,95781519~7783072,112275487~7841113,124240937~7583591,136270708~5240179,141885014~6484550,149552729~4403047,168236274~8367316,183274874~3283813,187506017~4831615,210147535~19406580,229742904~18484012,254075762~4430992,267179572~6811128,282485483~13108861,298036890~6918207,306903613~5021411,314393585~13715786,332944411~6919544,355806290~1463251,380094867~8701052,394880591~2187459,400878444~6009875,408516200~2525554] 2024-04-07T17:37:24.939 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [2329752~21800979,27329013~249441,37386125~16955242,55380210~10623546,68206575~15199694,95781519~7783072,112275487~7841113,124240937~7583591,134217728~7293159,141885014~6484550,149552729~4403047,167772160~8831430,183274874~3283813,187506017~4831615,210147535~19406580,229742904~18484012,254075762~4430992,264241152~9749548,282485483~13108861,298036890~6918207,306903613~21205758,331350016~8513939,355806290~1463251,380094867~8701052,394880591~2187459,400878444~10163310] 2024-04-07T17:37:24.940 INFO:tasks.workunit.client.0.smithi100.stdout:from 12 to 13 diff [20802751~3327980,27329013~249441,70298794~6825407,136270708~2853668,144181083~2188757,267179572~6811128,394880591~2187459,403289909~2301039,408516200~2525554] 2024-04-07T17:37:24.940 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [20802751~3327980,27329013~249441,70298794~6825407,136270708~2853668,144181083~2188757,267179572~6811128,394880591~2187459,403289909~2301039,408516200~2525554] 2024-04-07T17:37:24.947 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [20802751~3327980,27329013~249441,70298794~6825407,134217728~4906648,144181083~2188757,264241152~9749548,394880591~2187459,403289909~2301039,406847488~4194266] 2024-04-07T17:37:24.947 INFO:tasks.workunit.client.0.smithi100.stdout:from 12 to 14 diff [2329752~961414,18121228~6009503,27329013~249441,42101226~986344,70298794~6825407,124240937~6018736,136270708~2853668,144181083~2188757,187506017~4831615,213644600~9382452,267179572~6811128,288216273~6211110,299541775~4799266,318962927~7975475,394880591~2187459,403289909~2301039,408516200~2525554] 2024-04-07T17:37:24.948 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~961414,18121228~6009503,27329013~249441,42101226~986344,70298794~6825407,124240937~6018736,136270708~2853668,144181083~2188757,187506017~4831615,213644600~9382452,267179572~6811128,288216273~6211110,299541775~4799266,318962927~7975475,394880591~2187459,403289909~2301039,408516200~2525554] 2024-04-07T17:37:24.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:37:24 smithi100 bash[74942]: cluster 2024-04-07T17:37:23.572708+0000 mon.a (mon.0) 3885 : cluster [DBG] osdmap e2012: 8 total, 8 up, 8 in 2024-04-07T17:37:24.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:37:24 smithi100 bash[76202]: cluster 2024-04-07T17:37:23.572708+0000 mon.a (mon.0) 3885 : cluster [DBG] osdmap e2012: 8 total, 8 up, 8 in 2024-04-07T17:37:24.955 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [2329752~961414,18121228~6009503,27329013~249441,42101226~986344,70298794~6825407,124240937~6018736,134217728~4906648,144181083~2188757,187506017~4831615,213644600~9382452,264241152~9749548,288216273~6211110,299541775~4799266,318962927~7975475,394880591~2187459,403289909~2301039,406847488~4194266] 2024-04-07T17:37:24.955 INFO:tasks.workunit.client.0.smithi100.stdout:from 12 to 15 diff [2329752~961414,18121228~6009503,27329013~249441,39996176~4049302,70298794~13107475,124240937~6018736,136270708~2853668,144181083~2188757,168236274~8367316,187506017~4831615,213644600~10966142,242402883~5824033,255043666~3463088,267179572~6811128,288216273~6211110,299541775~4799266,314393585~12544817,394880591~2187459,401962264~4926055,408516200~2525554] 2024-04-07T17:37:24.955 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~961414,18121228~6009503,27329013~249441,39996176~4049302,70298794~13107475,124240937~6018736,136270708~2853668,144181083~2188757,168236274~8367316,187506017~4831615,213644600~10966142,242402883~5824033,255043666~3463088,267179572~6811128,288216273~6211110,299541775~4799266,314393585~12544817,394880591~2187459,401962264~4926055,408516200~2525554] 2024-04-07T17:37:24.963 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [2329752~961414,18121228~6009503,27329013~249441,39996176~4049302,70298794~13107475,124240937~6018736,134217728~4906648,144181083~2188757,167772160~8831430,187506017~4831615,213644600~10966142,242402883~5824033,255043666~3463088,264241152~9749548,288216273~6211110,299541775~4799266,310929323~16009079,394880591~2187459,401962264~9079490] 2024-04-07T17:37:24.963 INFO:tasks.workunit.client.0.smithi100.stdout:from 12 to 16 diff [2329752~4625179,14933611~2822456,18121228~6009503,27329013~249441,39996176~4049302,48881748~4683249,70298794~13107475,95781519~3680106,124240937~6018736,136270708~2853668,144181083~2188757,168236274~8367316,187506017~4831615,213644600~10966142,235932092~5747954,242402883~5824033,254075762~4430992,267179572~6811128,286319045~8108338,299541775~4799266,314393585~12544817,332944411~6919544,380563114~7406933,394880591~2187459,401962264~4926055,408516200~2525554] 2024-04-07T17:37:24.963 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~4625179,14933611~2822456,18121228~6009503,27329013~249441,39996176~4049302,48881748~4683249,70298794~13107475,95781519~3680106,124240937~6018736,136270708~2853668,144181083~2188757,168236274~8367316,187506017~4831615,213644600~10966142,235932092~5747954,242402883~5824033,254075762~4430992,267179572~6811128,286319045~8108338,299541775~4799266,314393585~12544817,332944411~6919544,380563114~7406933,394880591~2187459,401962264~4926055,408516200~2525554] 2024-04-07T17:37:24.970 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [2329752~4625179,12582912~5173155,18121228~6009503,27329013~249441,39996176~4049302,48881748~4683249,70298794~13107475,95781519~3680106,124240937~6018736,134217728~4906648,144181083~2188757,167772160~8831430,187506017~4831615,213644600~10966142,235932092~5747954,242402883~5824033,254075762~4430992,264241152~9749548,286319045~8108338,299541775~4799266,310929323~16009079,331350016~8513939,380563114~7406933,394880591~2187459,401962264~9079490] 2024-04-07T17:37:24.970 INFO:tasks.workunit.client.0.smithi100.stdout:from 12 to 17 diff [2329752~4625179,12468100~11662631,27329013~249441,39912210~4133268,48881748~4683249,68206575~15199694,95781519~3680106,124240937~6018736,136270708~5240179,144181083~2188757,168236274~8367316,187506017~4831615,211678512~920259,213644600~15909515,235932092~5747954,242402883~5824033,254075762~4430992,267179572~6811128,282485483~11941900,299541775~4799266,314393585~12544817,332944411~6919544,355806290~1463251,380563114~7406933,394880591~2187459,401962264~4926055,408516200~2525554] 2024-04-07T17:37:24.970 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~4625179,12468100~11662631,27329013~249441,39912210~4133268,48881748~4683249,68206575~15199694,95781519~3680106,124240937~6018736,136270708~5240179,144181083~2188757,168236274~8367316,187506017~4831615,211678512~920259,213644600~15909515,235932092~5747954,242402883~5824033,254075762~4430992,267179572~6811128,282485483~11941900,299541775~4799266,314393585~12544817,332944411~6919544,355806290~1463251,380563114~7406933,394880591~2187459,401962264~4926055,408516200~2525554] 2024-04-07T17:37:24.976 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [2329752~4625179,8388608~15742123,27329013~249441,39912210~4133268,48881748~4683249,68206575~15199694,95781519~3680106,124240937~6018736,134217728~7293159,144181083~2188757,167772160~8831430,187506017~4831615,211678512~920259,213644600~15909515,235932092~5747954,242402883~5824033,254075762~4430992,264241152~9749548,282485483~11941900,299541775~4799266,310929323~16009079,331350016~8513939,355806290~1463251,380563114~7406933,394880591~2187459,401962264~9079490] 2024-04-07T17:37:24.976 INFO:tasks.workunit.client.0.smithi100.stdout:from 12 to 18 diff [2329752~9105991,12468100~11662631,27329013~249441,39912210~4133268,48881748~4683249,58033882~7969874,68206575~15199694,95781519~3680106,124240937~7583591,136270708~5240179,144181083~2188757,168236274~8367316,183274874~3283813,187506017~4831615,210147535~19406580,229742904~18484012,254075762~4430992,267179572~6811128,282485483~11941900,298643235~5745703,314393585~12544817,332944411~6919544,355806290~1463251,380563114~8232805,394880591~2187459,401962264~4926055,408516200~2525554] 2024-04-07T17:37:24.976 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~9105991,12468100~11662631,27329013~249441,39912210~4133268,48881748~4683249,58033882~7969874,68206575~15199694,95781519~3680106,124240937~7583591,136270708~5240179,144181083~2188757,168236274~8367316,183274874~3283813,187506017~4831615,210147535~19406580,229742904~18484012,254075762~4430992,267179572~6811128,282485483~11941900,298643235~5745703,314393585~12544817,332944411~6919544,355806290~1463251,380563114~8232805,394880591~2187459,401962264~4926055,408516200~2525554] 2024-04-07T17:37:24.983 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [2329752~21800979,27329013~249441,39912210~4133268,48881748~4683249,56705995~9297761,68206575~15199694,95781519~3680106,124240937~7583591,134217728~7293159,144181083~2188757,167772160~8831430,183274874~3283813,187506017~4831615,210147535~19406580,229742904~18484012,254075762~4430992,264241152~9749548,282485483~11941900,298643235~5745703,310929323~16009079,331350016~8513939,355806290~1463251,380563114~8232805,394880591~2187459,401962264~9079490] 2024-04-07T17:37:24.983 INFO:tasks.workunit.client.0.smithi100.stdout:from 12 to 19 diff [2329752~21800979,27329013~249441,39912210~4133268,48881748~4683249,55380210~10623546,68206575~15199694,95781519~3680106,124240937~7583591,136270708~5240179,141885014~6484550,149552729~4403047,168236274~8367316,183274874~3283813,187506017~4831615,210147535~19406580,229742904~18484012,254075762~4430992,267179572~6811128,282485483~11941900,298643235~5745703,306903613~5021411,314393585~12544817,332944411~6919544,355806290~1463251,380563114~8232805,394880591~2187459,401962264~4926055,408516200~2525554] 2024-04-07T17:37:24.984 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~21800979,27329013~249441,39912210~4133268,48881748~4683249,55380210~10623546,68206575~15199694,95781519~3680106,124240937~7583591,136270708~5240179,141885014~6484550,149552729~4403047,168236274~8367316,183274874~3283813,187506017~4831615,210147535~19406580,229742904~18484012,254075762~4430992,267179572~6811128,282485483~11941900,298643235~5745703,306903613~5021411,314393585~12544817,332944411~6919544,355806290~1463251,380563114~8232805,394880591~2187459,401962264~4926055,408516200~2525554] 2024-04-07T17:37:24.991 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [2329752~21800979,27329013~249441,39912210~4133268,48881748~4683249,55380210~10623546,68206575~15199694,95781519~3680106,124240937~7583591,134217728~7293159,141885014~6484550,149552729~4403047,167772160~8831430,183274874~3283813,187506017~4831615,210147535~19406580,229742904~18484012,254075762~4430992,264241152~9749548,282485483~11941900,298643235~5745703,306903613~20034789,331350016~8513939,355806290~1463251,380563114~8232805,394880591~2187459,401962264~9079490] 2024-04-07T17:37:24.991 INFO:tasks.workunit.client.0.smithi100.stdout:from 13 to 14 diff [2329752~961414,18121228~5535757,42101226~986344,124240937~6018736,187506017~4831615,213644600~9382452,288216273~6211110,299541775~4799266,318962927~7975475] 2024-04-07T17:37:24.991 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~961414,18121228~5535757,42101226~986344,124240937~6018736,187506017~4831615,213644600~9382452,288216273~6211110,299541775~4799266,318962927~7975475] 2024-04-07T17:37:24.998 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [2329752~961414,18121228~5535757,42101226~986344,124240937~6018736,187506017~4831615,213644600~9382452,288216273~6211110,299541775~4799266,318962927~7975475] 2024-04-07T17:37:24.998 INFO:tasks.workunit.client.0.smithi100.stdout:from 13 to 15 diff [2329752~961414,18121228~5535757,39996176~4049302,76018526~7387743,124240937~6018736,168236274~8367316,187506017~4831615,213644600~10966142,242402883~5824033,255043666~3463088,288216273~6211110,299541775~4799266,314393585~12544817,401962264~4926055] 2024-04-07T17:37:24.998 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~961414,18121228~5535757,39996176~4049302,76018526~7387743,124240937~6018736,168236274~8367316,187506017~4831615,213644600~10966142,242402883~5824033,255043666~3463088,288216273~6211110,299541775~4799266,314393585~12544817,401962264~4926055] 2024-04-07T17:37:25.005 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [2329752~961414,18121228~5535757,39996176~4049302,76018526~7387743,124240937~6018736,167772160~8831430,187506017~4831615,213644600~10966142,242402883~5824033,255043666~3463088,288216273~6211110,299541775~4799266,310929323~16009079,401962264~4926055] 2024-04-07T17:37:25.005 INFO:tasks.workunit.client.0.smithi100.stdout:from 13 to 16 diff [2329752~4625179,14933611~2822456,18121228~5535757,39996176~4049302,48881748~4683249,76018526~7387743,95781519~3680106,124240937~6018736,168236274~8367316,187506017~4831615,213644600~10966142,235932092~5747954,242402883~5824033,254075762~4430992,286319045~8108338,299541775~4799266,314393585~12544817,332944411~6919544,380563114~7406933,401962264~4926055] 2024-04-07T17:37:25.005 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~4625179,14933611~2822456,18121228~5535757,39996176~4049302,48881748~4683249,76018526~7387743,95781519~3680106,124240937~6018736,168236274~8367316,187506017~4831615,213644600~10966142,235932092~5747954,242402883~5824033,254075762~4430992,286319045~8108338,299541775~4799266,314393585~12544817,332944411~6919544,380563114~7406933,401962264~4926055] 2024-04-07T17:37:25.017 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [2329752~4625179,12582912~5173155,18121228~5535757,39996176~4049302,48881748~4683249,76018526~7387743,95781519~3680106,124240937~6018736,167772160~8831430,187506017~4831615,213644600~10966142,235932092~5747954,242402883~5824033,254075762~4430992,286319045~8108338,299541775~4799266,310929323~16009079,331350016~8513939,380563114~7406933,401962264~4926055] 2024-04-07T17:37:25.017 INFO:tasks.workunit.client.0.smithi100.stdout:from 13 to 17 diff [2329752~4625179,12468100~11188885,39912210~4133268,48881748~4683249,68206575~6181429,76018526~7387743,95781519~3680106,124240937~6018736,136996101~4514786,168236274~8367316,187506017~4831615,211678512~920259,213644600~15909515,235932092~5747954,242402883~5824033,254075762~4430992,282485483~11941900,299541775~4799266,314393585~12544817,332944411~6919544,355806290~1463251,380563114~7406933,401962264~4926055] 2024-04-07T17:37:25.017 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~4625179,12468100~11188885,39912210~4133268,48881748~4683249,68206575~6181429,76018526~7387743,95781519~3680106,124240937~6018736,136996101~4514786,168236274~8367316,187506017~4831615,211678512~920259,213644600~15909515,235932092~5747954,242402883~5824033,254075762~4430992,282485483~11941900,299541775~4799266,314393585~12544817,332944411~6919544,355806290~1463251,380563114~7406933,401962264~4926055] 2024-04-07T17:37:25.023 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [2329752~4625179,8388608~15268377,39912210~4133268,48881748~4683249,68206575~6181429,76018526~7387743,95781519~3680106,124240937~6018736,136996101~4514786,167772160~8831430,187506017~4831615,211678512~920259,213644600~15909515,235932092~5747954,242402883~5824033,254075762~4430992,282485483~11941900,299541775~4799266,310929323~16009079,331350016~8513939,355806290~1463251,380563114~7406933,401962264~4926055] 2024-04-07T17:37:25.024 INFO:tasks.workunit.client.0.smithi100.stdout:from 13 to 18 diff [2329752~9105991,12468100~11188885,39912210~4133268,48881748~4683249,58033882~7969874,68206575~6181429,76018526~7387743,95781519~3680106,124240937~7583591,136996101~4514786,168236274~8367316,183274874~3283813,187506017~4831615,210147535~19406580,229742904~18484012,254075762~4430992,282485483~11941900,298643235~5745703,314393585~12544817,332944411~6919544,355806290~1463251,380563114~8232805,401962264~4926055] 2024-04-07T17:37:25.024 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~9105991,12468100~11188885,39912210~4133268,48881748~4683249,58033882~7969874,68206575~6181429,76018526~7387743,95781519~3680106,124240937~7583591,136996101~4514786,168236274~8367316,183274874~3283813,187506017~4831615,210147535~19406580,229742904~18484012,254075762~4430992,282485483~11941900,298643235~5745703,314393585~12544817,332944411~6919544,355806290~1463251,380563114~8232805,401962264~4926055] 2024-04-07T17:37:25.030 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [2329752~21327233,39912210~4133268,48881748~4683249,56705995~9297761,68206575~6181429,76018526~7387743,95781519~3680106,124240937~7583591,136996101~4514786,167772160~8831430,183274874~3283813,187506017~4831615,210147535~19406580,229742904~18484012,254075762~4430992,282485483~11941900,298643235~5745703,310929323~16009079,331350016~8513939,355806290~1463251,380563114~8232805,401962264~4926055] 2024-04-07T17:37:25.030 INFO:tasks.workunit.client.0.smithi100.stdout:from 13 to 19 diff [2329752~21327233,39912210~4133268,48881748~4683249,55380210~10623546,68206575~15199694,95781519~3680106,124240937~7583591,136996101~4514786,141885014~6484550,149552729~4403047,168236274~8367316,183274874~3283813,187506017~4831615,210147535~19406580,229742904~18484012,254075762~4430992,282485483~11941900,298643235~5745703,306903613~5021411,314393585~12544817,332944411~6919544,355806290~1463251,380563114~8232805,401962264~4926055] 2024-04-07T17:37:25.030 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~21327233,39912210~4133268,48881748~4683249,55380210~10623546,68206575~15199694,95781519~3680106,124240937~7583591,136996101~4514786,141885014~6484550,149552729~4403047,168236274~8367316,183274874~3283813,187506017~4831615,210147535~19406580,229742904~18484012,254075762~4430992,282485483~11941900,298643235~5745703,306903613~5021411,314393585~12544817,332944411~6919544,355806290~1463251,380563114~8232805,401962264~4926055] 2024-04-07T17:37:25.036 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [2329752~21327233,39912210~4133268,48881748~4683249,55380210~10623546,68206575~15199694,95781519~3680106,124240937~7583591,136996101~4514786,141885014~6484550,149552729~4403047,167772160~8831430,183274874~3283813,187506017~4831615,210147535~19406580,229742904~18484012,254075762~4430992,282485483~11941900,298643235~5745703,306903613~20034789,331350016~8513939,355806290~1463251,380563114~8232805,401962264~4926055] 2024-04-07T17:37:25.036 INFO:tasks.workunit.client.0.smithi100.stdout:from 14 to 15 diff [39996176~4049302,76018526~7387743,168236274~8367316,220978325~3632417,242402883~5824033,255043666~3463088,314393585~7979426,401962264~4926055] 2024-04-07T17:37:25.037 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [39996176~4049302,76018526~7387743,168236274~8367316,220978325~3632417,242402883~5824033,255043666~3463088,314393585~7979426,401962264~4926055] 2024-04-07T17:37:25.044 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [39996176~4049302,76018526~7387743,167772160~8831430,220978325~3632417,242402883~5824033,255043666~3463088,310929323~11443688,401962264~4926055] 2024-04-07T17:37:25.044 INFO:tasks.workunit.client.0.smithi100.stdout:from 14 to 16 diff [3107878~3847053,14933611~2822456,39996176~4049302,48881748~4683249,76018526~7387743,95781519~3680106,168236274~8367316,220978325~3632417,235932092~5747954,242402883~5824033,254075762~4430992,286319045~3681129,314393585~7979426,323018422~3348552,332944411~6919544,380563114~7406933,401962264~4926055] 2024-04-07T17:37:25.044 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [3107878~3847053,14933611~2822456,39996176~4049302,48881748~4683249,76018526~7387743,95781519~3680106,168236274~8367316,220978325~3632417,235932092~5747954,242402883~5824033,254075762~4430992,286319045~3681129,314393585~7979426,323018422~3348552,332944411~6919544,380563114~7406933,401962264~4926055] 2024-04-07T17:37:25.051 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [3107878~3847053,12582912~5173155,39996176~4049302,48881748~4683249,76018526~7387743,95781519~3680106,167772160~8831430,220978325~3632417,235932092~5747954,242402883~5824033,254075762~4430992,286319045~3681129,310929323~11443688,323018422~3348552,331350016~8513939,380563114~7406933,401962264~4926055] 2024-04-07T17:37:25.051 INFO:tasks.workunit.client.0.smithi100.stdout:from 14 to 17 diff [3107878~3847053,12468100~7703070,39912210~4133268,48881748~4683249,68206575~6181429,76018526~7387743,95781519~3680106,136996101~4514786,168236274~8367316,211678512~920259,220705920~8848195,235932092~5747954,242402883~5824033,254075762~4430992,282485483~7514691,314393585~7979426,323018422~3348552,332944411~6919544,355806290~1463251,380563114~7406933,401962264~4926055] 2024-04-07T17:37:25.051 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [3107878~3847053,12468100~7703070,39912210~4133268,48881748~4683249,68206575~6181429,76018526~7387743,95781519~3680106,136996101~4514786,168236274~8367316,211678512~920259,220705920~8848195,235932092~5747954,242402883~5824033,254075762~4430992,282485483~7514691,314393585~7979426,323018422~3348552,332944411~6919544,355806290~1463251,380563114~7406933,401962264~4926055] 2024-04-07T17:37:25.062 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [3107878~3847053,8388608~11782562,39912210~4133268,48881748~4683249,68206575~6181429,76018526~7387743,95781519~3680106,136996101~4514786,167772160~8831430,211678512~920259,220705920~8848195,235932092~5747954,242402883~5824033,254075762~4430992,282485483~7514691,310929323~11443688,323018422~3348552,331350016~8513939,355806290~1463251,380563114~7406933,401962264~4926055] 2024-04-07T17:37:25.062 INFO:tasks.workunit.client.0.smithi100.stdout:from 14 to 18 diff [3107878~8327865,12468100~7703070,39912210~4133268,48881748~4683249,58033882~7969874,68206575~6181429,76018526~7387743,95781519~3680106,126602360~5222168,136996101~4514786,168236274~8367316,183274874~3283813,210147535~5174157,219514234~10039881,229742904~18484012,254075762~4430992,282485483~7514691,298643235~5745703,314393585~7979426,323018422~3348552,332944411~6919544,355806290~1463251,380563114~8232805,401962264~4926055] 2024-04-07T17:37:25.062 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [3107878~8327865,12468100~7703070,39912210~4133268,48881748~4683249,58033882~7969874,68206575~6181429,76018526~7387743,95781519~3680106,126602360~5222168,136996101~4514786,168236274~8367316,183274874~3283813,210147535~5174157,219514234~10039881,229742904~18484012,254075762~4430992,282485483~7514691,298643235~5745703,314393585~7979426,323018422~3348552,332944411~6919544,355806290~1463251,380563114~8232805,401962264~4926055] 2024-04-07T17:37:25.066 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [3107878~17063292,39912210~4133268,48881748~4683249,56705995~9297761,68206575~6181429,76018526~7387743,95781519~3680106,126602360~5222168,136996101~4514786,167772160~8831430,183274874~3283813,210147535~5174157,219514234~10039881,229742904~18484012,254075762~4430992,282485483~7514691,298643235~5745703,310929323~11443688,323018422~3348552,331350016~8513939,355806290~1463251,380563114~8232805,401962264~4926055] 2024-04-07T17:37:25.067 INFO:tasks.workunit.client.0.smithi100.stdout:from 14 to 19 diff [3107878~17063292,39912210~4133268,48881748~4683249,55380210~10623546,68206575~15199694,95781519~3680106,126602360~5222168,136996101~4514786,141885014~6484550,149552729~4403047,168236274~8367316,183274874~3283813,210147535~5174157,219514234~10039881,229742904~18484012,254075762~4430992,282485483~7514691,298643235~5745703,306903613~5021411,314393585~7979426,323018422~3348552,332944411~6919544,355806290~1463251,380563114~8232805,401962264~4926055] 2024-04-07T17:37:25.067 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [3107878~17063292,39912210~4133268,48881748~4683249,55380210~10623546,68206575~15199694,95781519~3680106,126602360~5222168,136996101~4514786,141885014~6484550,149552729~4403047,168236274~8367316,183274874~3283813,210147535~5174157,219514234~10039881,229742904~18484012,254075762~4430992,282485483~7514691,298643235~5745703,306903613~5021411,314393585~7979426,323018422~3348552,332944411~6919544,355806290~1463251,380563114~8232805,401962264~4926055] 2024-04-07T17:37:25.073 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [3107878~17063292,39912210~4133268,48881748~4683249,55380210~10623546,68206575~15199694,95781519~3680106,126602360~5222168,136996101~4514786,141885014~6484550,149552729~4403047,167772160~8831430,183274874~3283813,210147535~5174157,219514234~10039881,229742904~18484012,254075762~4430992,282485483~7514691,298643235~5745703,306903613~15469398,323018422~3348552,331350016~8513939,355806290~1463251,380563114~8232805,401962264~4926055] 2024-04-07T17:37:25.073 INFO:tasks.workunit.client.0.smithi100.stdout:from 15 to 16 diff [3107878~3847053,14933611~2822456,48881748~4683249,95781519~3680106,235932092~5747954,254075762~1580725,286319045~3681129,323018422~3348552,332944411~6919544,380563114~7406933] 2024-04-07T17:37:25.073 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [3107878~3847053,14933611~2822456,48881748~4683249,95781519~3680106,235932092~5747954,254075762~1580725,286319045~3681129,323018422~3348552,332944411~6919544,380563114~7406933] 2024-04-07T17:37:25.079 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [3107878~3847053,12582912~5173155,48881748~4683249,95781519~3680106,235932092~5747954,254075762~1580725,286319045~3681129,323018422~3348552,331350016~8513939,380563114~7406933] 2024-04-07T17:37:25.079 INFO:tasks.workunit.client.0.smithi100.stdout:from 15 to 17 diff [3107878~3847053,12468100~7703070,39912210~3191969,48881748~4683249,68206575~6181429,95781519~3680106,136996101~4514786,211678512~920259,220705920~8848195,235932092~5747954,242476971~1393065,254075762~1580725,282485483~7514691,323018422~3348552,332944411~6919544,355806290~1463251,380563114~7406933] 2024-04-07T17:37:25.079 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [3107878~3847053,12468100~7703070,39912210~3191969,48881748~4683249,68206575~6181429,95781519~3680106,136996101~4514786,211678512~920259,220705920~8848195,235932092~5747954,242476971~1393065,254075762~1580725,282485483~7514691,323018422~3348552,332944411~6919544,355806290~1463251,380563114~7406933] 2024-04-07T17:37:25.086 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [3107878~3847053,8388608~11782562,39912210~3191969,48881748~4683249,68206575~6181429,95781519~3680106,136996101~4514786,211678512~920259,220705920~8848195,235932092~5747954,242476971~1393065,254075762~1580725,282485483~7514691,323018422~3348552,331350016~8513939,355806290~1463251,380563114~7406933] 2024-04-07T17:37:25.086 INFO:tasks.workunit.client.0.smithi100.stdout:from 15 to 18 diff [3107878~8327865,12468100~7703070,39912210~3191969,48881748~4683249,58033882~7969874,68206575~6181429,95781519~3680106,126602360~5222168,136996101~4514786,183274874~3283813,210147535~5174157,219514234~10039881,229742904~14127132,254075762~1580725,282485483~7514691,298643235~5745703,323018422~3348552,332944411~6919544,355806290~1463251,380563114~8232805] 2024-04-07T17:37:25.086 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [3107878~8327865,12468100~7703070,39912210~3191969,48881748~4683249,58033882~7969874,68206575~6181429,95781519~3680106,126602360~5222168,136996101~4514786,183274874~3283813,210147535~5174157,219514234~10039881,229742904~14127132,254075762~1580725,282485483~7514691,298643235~5745703,323018422~3348552,332944411~6919544,355806290~1463251,380563114~8232805] 2024-04-07T17:37:25.095 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [3107878~17063292,39912210~3191969,48881748~4683249,56705995~9297761,68206575~6181429,95781519~3680106,126602360~5222168,136996101~4514786,183274874~3283813,210147535~5174157,219514234~10039881,229742904~14127132,254075762~1580725,282485483~7514691,298643235~5745703,323018422~3348552,331350016~8513939,355806290~1463251,380563114~8232805] 2024-04-07T17:37:25.095 INFO:tasks.workunit.client.0.smithi100.stdout:from 15 to 19 diff [3107878~17063292,39912210~3191969,48881748~4683249,55380210~10623546,68206575~10344727,95781519~3680106,126602360~5222168,136996101~4514786,141885014~6484550,149552729~4403047,174179109~452103,183274874~3283813,210147535~5174157,219514234~10039881,229742904~14127132,254075762~1580725,282485483~7514691,298643235~5745703,306903613~5021411,323018422~3348552,332944411~6919544,355806290~1463251,380563114~8232805] 2024-04-07T17:37:25.096 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [3107878~17063292,39912210~3191969,48881748~4683249,55380210~10623546,68206575~10344727,95781519~3680106,126602360~5222168,136996101~4514786,141885014~6484550,149552729~4403047,174179109~452103,183274874~3283813,210147535~5174157,219514234~10039881,229742904~14127132,254075762~1580725,282485483~7514691,298643235~5745703,306903613~5021411,323018422~3348552,332944411~6919544,355806290~1463251,380563114~8232805] 2024-04-07T17:37:25.103 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [3107878~17063292,39912210~3191969,48881748~4683249,55380210~10623546,68206575~10344727,95781519~3680106,126602360~5222168,136996101~4514786,141885014~6484550,149552729~4403047,174179109~452103,183274874~3283813,210147535~5174157,219514234~10039881,229742904~14127132,254075762~1580725,282485483~7514691,298643235~5745703,306903613~5021411,323018422~3348552,331350016~8513939,355806290~1463251,380563114~8232805] 2024-04-07T17:37:25.103 INFO:tasks.workunit.client.0.smithi100.stdout:from 16 to 17 diff [12468100~7703070,39912210~3191969,68206575~6181429,136996101~4514786,211678512~920259,220705920~8848195,242476971~1393065,282485483~6755344,355806290~1463251] 2024-04-07T17:37:25.103 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [12468100~7703070,39912210~3191969,68206575~6181429,136996101~4514786,211678512~920259,220705920~8848195,242476971~1393065,282485483~6755344,355806290~1463251] 2024-04-07T17:37:25.110 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [8388608~11782562,39912210~3191969,68206575~6181429,136996101~4514786,211678512~920259,220705920~8848195,242476971~1393065,282485483~6755344,355806290~1463251] 2024-04-07T17:37:25.110 INFO:tasks.workunit.client.0.smithi100.stdout:from 16 to 18 diff [6688790~4746953,12468100~7703070,39912210~3191969,58033882~7969874,68206575~6181429,126602360~5222168,136996101~4514786,183274874~3283813,210147535~5174157,219514234~10039881,229742904~14127132,282485483~6755344,298643235~5745703,355806290~1463251,386437457~2358462] 2024-04-07T17:37:25.110 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [6688790~4746953,12468100~7703070,39912210~3191969,58033882~7969874,68206575~6181429,126602360~5222168,136996101~4514786,183274874~3283813,210147535~5174157,219514234~10039881,229742904~14127132,282485483~6755344,298643235~5745703,355806290~1463251,386437457~2358462] 2024-04-07T17:37:25.118 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [6688790~13482380,39912210~3191969,56705995~9297761,68206575~6181429,126602360~5222168,136996101~4514786,183274874~3283813,210147535~5174157,219514234~10039881,229742904~14127132,282485483~6755344,298643235~5745703,355806290~1463251,386437457~2358462] 2024-04-07T17:37:25.118 INFO:tasks.workunit.client.0.smithi100.stdout:from 16 to 19 diff [6688790~13482380,39912210~3191969,55380210~10623546,68206575~10344727,126602360~5222168,136996101~4514786,141885014~6484550,149552729~4403047,174179109~452103,183274874~3283813,210147535~5174157,219514234~10039881,229742904~14127132,282485483~6755344,298643235~5745703,306903613~5021411,355806290~1463251,386437457~2358462] 2024-04-07T17:37:25.118 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [6688790~13482380,39912210~3191969,55380210~10623546,68206575~10344727,126602360~5222168,136996101~4514786,141885014~6484550,149552729~4403047,174179109~452103,183274874~3283813,210147535~5174157,219514234~10039881,229742904~14127132,282485483~6755344,298643235~5745703,306903613~5021411,355806290~1463251,386437457~2358462] 2024-04-07T17:37:25.124 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [6688790~13482380,39912210~3191969,55380210~10623546,68206575~10344727,126602360~5222168,136996101~4514786,141885014~6484550,149552729~4403047,174179109~452103,183274874~3283813,210147535~5174157,219514234~10039881,229742904~14127132,282485483~6755344,298643235~5745703,306903613~5021411,355806290~1463251,386437457~2358462] 2024-04-07T17:37:25.124 INFO:tasks.workunit.client.0.smithi100.stdout:from 17 to 18 diff [6688790~4746953,58033882~7969874,126602360~5222168,183274874~3283813,210147535~5174157,219514234~5127006,229742904~13365134,298643235~5745703,386437457~2358462] 2024-04-07T17:37:25.124 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [6688790~4746953,58033882~7969874,126602360~5222168,183274874~3283813,210147535~5174157,219514234~5127006,229742904~13365134,298643235~5745703,386437457~2358462] 2024-04-07T17:37:25.131 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [6688790~4746953,56705995~9297761,126602360~5222168,183274874~3283813,210147535~5174157,219514234~5127006,229742904~13365134,298643235~5745703,386437457~2358462] 2024-04-07T17:37:25.132 INFO:tasks.workunit.client.0.smithi100.stdout:from 17 to 19 diff [6688790~7360627,55380210~10623546,71826911~6724391,126602360~5222168,141885014~6484550,149552729~4403047,174179109~452103,183274874~3283813,210147535~5174157,219514234~5689083,226376149~484931,229742904~13365134,298643235~5745703,306903613~5021411,386437457~2358462] 2024-04-07T17:37:25.132 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [6688790~7360627,55380210~10623546,71826911~6724391,126602360~5222168,141885014~6484550,149552729~4403047,174179109~452103,183274874~3283813,210147535~5174157,219514234~5689083,226376149~484931,229742904~13365134,298643235~5745703,306903613~5021411,386437457~2358462] 2024-04-07T17:37:25.139 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [6688790~7360627,55380210~10623546,71826911~6724391,126602360~5222168,141885014~6484550,149552729~4403047,174179109~452103,183274874~3283813,210147535~5174157,219514234~5689083,226376149~484931,229742904~13365134,298643235~5745703,306903613~5021411,386437457~2358462] 2024-04-07T17:37:25.139 INFO:tasks.workunit.client.0.smithi100.stdout:from 18 to 19 diff [9531660~4517757,55380210~7770032,71826911~6724391,141885014~6484550,149552729~4403047,174179109~452103,219630111~5573206,226376149~484931,306903613~5021411] 2024-04-07T17:37:25.139 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [9531660~4517757,55380210~7770032,71826911~6724391,141885014~6484550,149552729~4403047,174179109~452103,219630111~5573206,226376149~484931,306903613~5021411] 2024-04-07T17:37:25.146 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [9531660~4517757,55380210~7770032,71826911~6724391,141885014~6484550,149552729~4403047,174179109~452103,219630111~5573206,226376149~484931,306903613~5021411] 2024-04-07T17:37:25.623 INFO:tasks.workunit.client.0.smithi100.stdout:from 0 to HEAD diff [2329752~26214847,29778841~737427,31769017~13613313,45857929~20145827,66861224~16545045,85466334~5069656,92999623~10564968,106500403~13616197,121501610~12475984,136270708~22564048,160337176~7153146,168236274~8367316,180237750~12590714,194102759~7364680,202420372~5254601,210147535~53978881,267179572~28414772,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350406459~6863082,360049192~84791,366341667~7420413,374580519~17465318,392232408~14655911,408516200~2525554] 2024-04-07T17:37:25.623 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~26214847,29778841~737427,31769017~13613313,45857929~20145827,66861224~16545045,85466334~5069656,92999623~10564968,106500403~13616197,121501610~12475984,136270708~22564048,160337176~7153146,168236274~8367316,180237750~12590714,194102759~7364680,202420372~5254601,210147535~53978881,267179572~28414772,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350406459~6863082,360049192~84791,366341667~7420413,374580519~17465318,392232408~14655911,408516200~2525554] 2024-04-07T17:37:25.682 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~28544599,29360128~54046141,83886080~6649910,92274688~11289903,104857600~29119994,134217728~24617028,159383552~8106770,167772160~25056304,192937984~14736989,210147535~53978881,264241152~31353192,297795584~7201936,306184192~23819574,331350016~15772466,348127232~12006751,366341667~44700087] 2024-04-07T17:37:25.682 INFO:tasks.workunit.client.0.smithi100.stdout:from 1 to HEAD diff [2329752~26214847,29778841~737427,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~13616197,121501610~12475984,136270708~22564048,160337176~7153146,168236274~8367316,180237750~12590714,194866296~6601143,202420372~5254601,210147535~53978881,267179572~28414772,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350891716~1451525,355806290~1463251,360049192~84791,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:25.683 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~26214847,29778841~737427,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~13616197,121501610~12475984,136270708~22564048,160337176~7153146,168236274~8367316,180237750~12590714,194866296~6601143,202420372~5254601,210147535~53978881,267179572~28414772,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350891716~1451525,355806290~1463251,360049192~84791,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:25.695 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~28544599,29360128~24981239,55380210~10623546,66861224~16545045,92274688~11289903,104857600~29119994,134217728~24617028,159383552~8106770,167772160~25056304,194866296~12808677,210147535~53978881,264241152~31353192,297795584~7201936,306184192~23819574,331350016~15772466,350891716~1451525,355806290~4327693,366341667~25704170,394880591~2187459,398418332~12623422] 2024-04-07T17:37:25.695 INFO:tasks.workunit.client.0.smithi100.stdout:from 2 to HEAD diff [2329752~24230697,27329013~980763,29778841~737427,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~17685068,154799337~4035419,160337176~7153146,168236274~8367316,182034135~10794329,196727604~2108791,202420372~5254601,210147535~53978881,267179572~28414772,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350891716~1451525,355806290~1463251,360049192~84791,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:25.696 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~24230697,27329013~980763,29778841~737427,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~17685068,154799337~4035419,160337176~7153146,168236274~8367316,182034135~10794329,196727604~2108791,202420372~5254601,210147535~53978881,267179572~28414772,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350891716~1451525,355806290~1463251,360049192~84791,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:25.707 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~26560449,27329013~980763,29360128~24981239,55380210~10623546,66861224~16545045,92274688~11289903,106500403~5285199,112275487~21702107,134217728~19738048,154799337~4035419,159383552~8106770,167772160~8831430,182034135~10794329,196727604~2108791,201467439~6207534,210147535~53978881,264241152~31353192,297795584~7201936,306184192~23819574,331350016~15772466,350891716~1451525,355806290~4327693,366341667~25704170,394880591~2187459,398418332~12623422] 2024-04-07T17:37:25.707 INFO:tasks.workunit.client.0.smithi100.stdout:from 3 to HEAD diff [2329752~24230697,27329013~980763,29778841~737427,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~17685068,154799337~3192780,160337176~7153146,168236274~8367316,182034135~10303497,196727604~2108791,202420372~1435645,210147535~53978881,267179572~28414772,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350891716~1451525,355806290~1463251,360049192~84791,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:25.707 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~24230697,27329013~980763,29778841~737427,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~17685068,154799337~3192780,160337176~7153146,168236274~8367316,182034135~10303497,196727604~2108791,202420372~1435645,210147535~53978881,267179572~28414772,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350891716~1451525,355806290~1463251,360049192~84791,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:25.717 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~26560449,27329013~980763,29360128~24981239,55380210~10623546,66861224~16545045,92274688~11289903,106500403~5285199,112275487~21702107,134217728~19738048,154799337~3192780,159383552~8106770,167772160~8831430,182034135~10303497,196727604~2108791,202420372~1435645,210147535~53978881,264241152~31353192,297795584~7201936,306184192~23819574,331350016~15772466,350891716~1451525,355806290~4327693,366341667~25704170,394880591~2187459,398418332~12623422] 2024-04-07T17:37:25.717 INFO:tasks.workunit.client.0.smithi100.stdout:from 4 to HEAD diff [2329752~24230697,27329013~980763,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~12098856,149233764~4722012,154895619~3096498,160337176~7153146,168236274~8367316,182034135~10303497,196727604~2108791,202420372~1435645,210147535~53978881,267179572~13527794,281234113~14360231,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350891716~1451525,355806290~1463251,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:25.717 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~24230697,27329013~980763,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~12098856,149233764~4722012,154895619~3096498,160337176~7153146,168236274~8367316,182034135~10303497,196727604~2108791,202420372~1435645,210147535~53978881,267179572~13527794,281234113~14360231,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350891716~1451525,355806290~1463251,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:25.725 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~26560449,27329013~980763,30516268~23825099,55380210~10623546,66861224~16545045,92274688~11289903,106500403~5285199,112275487~21702107,134217728~14151836,149233764~4722012,154895619~3096498,159383552~8106770,167772160~8831430,182034135~10303497,196727604~2108791,202420372~1435645,210147535~53978881,264241152~16466214,281234113~14360231,297795584~7201936,306903613~23100153,331350016~15772466,350891716~1451525,355806290~1463251,366341667~25704170,394880591~2187459,398418332~12623422] 2024-04-07T17:37:25.726 INFO:tasks.workunit.client.0.smithi100.stdout:from 5 to HEAD diff [2329752~24230697,27329013~980763,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~12098856,149233764~4722012,154895619~3096498,166133159~1357163,168236274~8367316,182034135~10303497,196727604~2108791,202420372~1435645,210147535~38079381,248419159~15707257,267179572~13527794,281234113~14360231,298036890~6960630,306903613~5021411,314393585~13715786,332944411~11089603,345811290~1311192,355806290~1463251,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:25.726 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~24230697,27329013~980763,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~12098856,149233764~4722012,154895619~3096498,166133159~1357163,168236274~8367316,182034135~10303497,196727604~2108791,202420372~1435645,210147535~38079381,248419159~15707257,267179572~13527794,281234113~14360231,298036890~6960630,306903613~5021411,314393585~13715786,332944411~11089603,345811290~1311192,355806290~1463251,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:25.734 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~26560449,27329013~980763,30516268~23825099,55380210~10623546,66861224~16545045,92274688~11289903,106500403~5285199,112275487~21702107,134217728~14151836,149233764~4722012,154895619~3096498,166133159~1357163,167772160~8831430,182034135~10303497,196727604~2108791,202420372~1435645,210147535~38079381,248419159~15707257,264241152~16466214,281234113~14360231,298036890~6960630,306903613~21205758,331350016~15772466,355806290~1463251,366341667~7420413,374580519~17465318,394880591~2187459,398418332~12623422] 2024-04-07T17:37:25.734 INFO:tasks.workunit.client.0.smithi100.stdout:from 6 to HEAD diff [2329752~23418695,27329013~980763,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~12098856,149233764~4722012,154895619~3096498,168236274~8367316,182747004~9590628,202420372~1435645,210147535~38079381,250447165~8059589,267179572~11729329,281234113~14360231,298036890~6960630,306903613~5021411,314393585~13715786,332944411~6919544,345811290~1311192,355806290~1463251,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:25.734 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~23418695,27329013~980763,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~12098856,149233764~4722012,154895619~3096498,168236274~8367316,182747004~9590628,202420372~1435645,210147535~38079381,250447165~8059589,267179572~11729329,281234113~14360231,298036890~6960630,306903613~5021411,314393585~13715786,332944411~6919544,345811290~1311192,355806290~1463251,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:25.742 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~25748447,27329013~980763,30516268~23825099,55380210~10623546,66861224~16545045,92274688~11289903,106500403~5285199,112275487~21702107,134217728~14151836,149233764~4722012,154895619~3096498,167772160~8831430,182747004~9590628,202420372~1435645,210147535~38079381,250447165~8059589,264241152~14667749,281234113~14360231,298036890~6960630,306903613~21205758,331350016~8513939,344034014~3088468,355806290~1463251,366341667~7420413,374580519~17465318,394880591~2187459,398418332~12623422] 2024-04-07T17:37:25.742 INFO:tasks.workunit.client.0.smithi100.stdout:from 7 to HEAD diff [2329752~23418695,27329013~980763,31769017~13613313,45857929~8483438,55380210~10623546,68206575~15199694,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~5240179,141885014~6484550,149233764~4722012,154895619~3096498,168236274~8367316,182747004~9590628,210147535~38079381,250447165~8059589,267179572~11729329,281234113~14360231,298036890~6960630,306903613~5021411,314393585~13715786,332944411~6919544,355806290~1463251,366341667~7420413,377322524~14723313,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:25.742 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~23418695,27329013~980763,31769017~13613313,45857929~8483438,55380210~10623546,68206575~15199694,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~5240179,141885014~6484550,149233764~4722012,154895619~3096498,168236274~8367316,182747004~9590628,210147535~38079381,250447165~8059589,267179572~11729329,281234113~14360231,298036890~6960630,306903613~5021411,314393585~13715786,332944411~6919544,355806290~1463251,366341667~7420413,377322524~14723313,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:25.750 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~25748447,27329013~980763,30516268~23825099,55380210~10623546,68206575~15199694,92274688~11289903,106500403~5285199,112275487~21702107,134217728~7293159,141885014~6484550,149233764~4722012,154895619~3096498,167772160~8831430,182747004~9590628,210147535~38079381,250447165~8059589,264241152~14667749,281234113~14360231,298036890~6960630,306903613~21205758,331350016~8513939,355806290~1463251,366341667~7420413,377322524~14723313,394880591~2187459,398418332~12623422] 2024-04-07T17:37:25.750 INFO:tasks.workunit.client.0.smithi100.stdout:from 8 to HEAD diff [2329752~23418695,27329013~980763,31769017~13613313,45857929~8483438,55380210~10623546,68206575~15199694,95470042~8094549,106500403~5285199,112275487~7841113,121501610~10322918,136270708~5240179,141885014~6484550,149552729~4403047,168236274~8367316,182747004~9590628,210147535~38079381,250447165~8059589,267179572~11729329,281234113~14360231,298036890~6918207,306903613~5021411,314393585~13715786,332944411~6919544,355806290~1463251,366341667~7420413,377322524~14723313,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:25.751 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~23418695,27329013~980763,31769017~13613313,45857929~8483438,55380210~10623546,68206575~15199694,95470042~8094549,106500403~5285199,112275487~7841113,121501610~10322918,136270708~5240179,141885014~6484550,149552729~4403047,168236274~8367316,182747004~9590628,210147535~38079381,250447165~8059589,267179572~11729329,281234113~14360231,298036890~6918207,306903613~5021411,314393585~13715786,332944411~6919544,355806290~1463251,366341667~7420413,377322524~14723313,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:25.758 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~25748447,27329013~980763,30516268~23825099,55380210~10623546,68206575~15199694,95470042~8094549,106500403~5285199,112275487~19549041,134217728~7293159,141885014~6484550,149552729~4403047,167772160~8831430,182747004~9590628,210147535~38079381,250447165~8059589,264241152~14667749,281234113~14360231,298036890~6918207,306903613~21205758,331350016~8513939,355806290~1463251,366341667~7420413,377322524~14723313,394880591~2187459,398418332~12623422] 2024-04-07T17:37:25.758 INFO:tasks.workunit.client.0.smithi100.stdout:from 9 to HEAD diff [2329752~21800979,27329013~980763,37362822~8019508,45857929~8483438,55380210~10623546,68206575~15199694,95781519~7783072,106500403~5285199,112275487~7841113,121501610~10322918,136270708~5240179,141885014~6484550,149552729~4403047,168236274~8367316,183274874~3283813,186734432~153638,187506017~4831615,210147535~38079381,254075762~4430992,267179572~11729329,282485483~13108861,298036890~6918207,306903613~5021411,314393585~13715786,332944411~6919544,355806290~1463251,366341667~7420413,377322524~14723313,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:25.758 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~21800979,27329013~980763,37362822~8019508,45857929~8483438,55380210~10623546,68206575~15199694,95781519~7783072,106500403~5285199,112275487~7841113,121501610~10322918,136270708~5240179,141885014~6484550,149552729~4403047,168236274~8367316,183274874~3283813,186734432~153638,187506017~4831615,210147535~38079381,254075762~4430992,267179572~11729329,282485483~13108861,298036890~6918207,306903613~5021411,314393585~13715786,332944411~6919544,355806290~1463251,366341667~7420413,377322524~14723313,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:25.766 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~24130731,27329013~980763,37362822~16978545,55380210~10623546,68206575~15199694,95781519~7783072,106500403~5285199,112275487~19549041,134217728~7293159,141885014~6484550,149552729~4403047,167772160~8831430,183274874~3283813,186734432~153638,187506017~4831615,210147535~38079381,254075762~4430992,264241152~14667749,282485483~13108861,298036890~6918207,306903613~21205758,331350016~8513939,355806290~1463251,366341667~7420413,377322524~14723313,394880591~2187459,398418332~12623422] 2024-04-07T17:37:25.766 INFO:tasks.workunit.client.0.smithi100.stdout:from 10 to HEAD diff [2329752~21800979,27329013~249441,37362822~8019508,45857929~8483438,55380210~10623546,68206575~15199694,95781519~7783072,112275487~7841113,124240937~7583591,136270708~5240179,141885014~6484550,149552729~4403047,168236274~8367316,183274874~3283813,187506017~4831615,210147535~38079381,254075762~4430992,267179572~11729329,282485483~13108861,298036890~6918207,306903613~5021411,314393585~13715786,332944411~6919544,355806290~1463251,377322524~14723313,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:25.766 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~21800979,27329013~249441,37362822~8019508,45857929~8483438,55380210~10623546,68206575~15199694,95781519~7783072,112275487~7841113,124240937~7583591,136270708~5240179,141885014~6484550,149552729~4403047,168236274~8367316,183274874~3283813,187506017~4831615,210147535~38079381,254075762~4430992,267179572~11729329,282485483~13108861,298036890~6918207,306903613~5021411,314393585~13715786,332944411~6919544,355806290~1463251,377322524~14723313,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:25.774 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [2329752~21800979,27329013~249441,37362822~16978545,55380210~10623546,68206575~15199694,95781519~7783072,112275487~7841113,124240937~7583591,134217728~7293159,141885014~6484550,149552729~4403047,167772160~8831430,183274874~3283813,187506017~4831615,210147535~38079381,254075762~4430992,264241152~14667749,282485483~13108861,298036890~6918207,306903613~21205758,331350016~8513939,355806290~1463251,377322524~14723313,394880591~2187459,398418332~12623422] 2024-04-07T17:37:25.774 INFO:tasks.workunit.client.0.smithi100.stdout:from 11 to HEAD diff [2329752~21800979,27329013~249441,37386125~7996205,45857929~8483438,55380210~10623546,68206575~15199694,95781519~7783072,112275487~7841113,124240937~7583591,136270708~5240179,141885014~6484550,149552729~4403047,168236274~8367316,183274874~3283813,187506017~4831615,210147535~19406580,229742904~18484012,254075762~4430992,267179572~6811128,282485483~13108861,298036890~6918207,306903613~5021411,314393585~13715786,332944411~6919544,355806290~1463251,380094867~8701052,394880591~2187459,400878444~6009875,408516200~2525554] 2024-04-07T17:37:25.774 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~21800979,27329013~249441,37386125~7996205,45857929~8483438,55380210~10623546,68206575~15199694,95781519~7783072,112275487~7841113,124240937~7583591,136270708~5240179,141885014~6484550,149552729~4403047,168236274~8367316,183274874~3283813,187506017~4831615,210147535~19406580,229742904~18484012,254075762~4430992,267179572~6811128,282485483~13108861,298036890~6918207,306903613~5021411,314393585~13715786,332944411~6919544,355806290~1463251,380094867~8701052,394880591~2187459,400878444~6009875,408516200~2525554] 2024-04-07T17:37:25.781 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [2329752~21800979,27329013~249441,37386125~16955242,55380210~10623546,68206575~15199694,95781519~7783072,112275487~7841113,124240937~7583591,134217728~7293159,141885014~6484550,149552729~4403047,167772160~8831430,183274874~3283813,187506017~4831615,210147535~19406580,229742904~18484012,254075762~4430992,264241152~9749548,282485483~13108861,298036890~6918207,306903613~21205758,331350016~8513939,355806290~1463251,380094867~8701052,394880591~2187459,400878444~10163310] 2024-04-07T17:37:25.781 INFO:tasks.workunit.client.0.smithi100.stdout:from 12 to HEAD diff [2329752~21800979,27329013~249441,39912210~4133268,48881748~4683249,55380210~10623546,68206575~15199694,95781519~3680106,124240937~7583591,136270708~5240179,141885014~6484550,149552729~4403047,168236274~8367316,183274874~3283813,187506017~4831615,210147535~19406580,229742904~18484012,254075762~4430992,267179572~6811128,282485483~11941900,298643235~5745703,306903613~5021411,314393585~12544817,332944411~6919544,355806290~1463251,380563114~8232805,394880591~2187459,401962264~4926055,408516200~2525554] 2024-04-07T17:37:25.781 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~21800979,27329013~249441,39912210~4133268,48881748~4683249,55380210~10623546,68206575~15199694,95781519~3680106,124240937~7583591,136270708~5240179,141885014~6484550,149552729~4403047,168236274~8367316,183274874~3283813,187506017~4831615,210147535~19406580,229742904~18484012,254075762~4430992,267179572~6811128,282485483~11941900,298643235~5745703,306903613~5021411,314393585~12544817,332944411~6919544,355806290~1463251,380563114~8232805,394880591~2187459,401962264~4926055,408516200~2525554] 2024-04-07T17:37:25.788 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [2329752~21800979,27329013~249441,39912210~4133268,48881748~4683249,55380210~10623546,68206575~15199694,95781519~3680106,124240937~7583591,134217728~7293159,141885014~6484550,149552729~4403047,167772160~8831430,183274874~3283813,187506017~4831615,210147535~19406580,229742904~18484012,254075762~4430992,264241152~9749548,282485483~11941900,298643235~5745703,306903613~20034789,331350016~8513939,355806290~1463251,380563114~8232805,394880591~2187459,401962264~9079490] 2024-04-07T17:37:25.789 INFO:tasks.workunit.client.0.smithi100.stdout:from 13 to HEAD diff [2329752~21327233,39912210~4133268,48881748~4683249,55380210~10623546,68206575~15199694,95781519~3680106,124240937~7583591,136996101~4514786,141885014~6484550,149552729~4403047,168236274~8367316,183274874~3283813,187506017~4831615,210147535~19406580,229742904~18484012,254075762~4430992,282485483~11941900,298643235~5745703,306903613~5021411,314393585~12544817,332944411~6919544,355806290~1463251,380563114~8232805,401962264~4926055] 2024-04-07T17:37:25.789 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~21327233,39912210~4133268,48881748~4683249,55380210~10623546,68206575~15199694,95781519~3680106,124240937~7583591,136996101~4514786,141885014~6484550,149552729~4403047,168236274~8367316,183274874~3283813,187506017~4831615,210147535~19406580,229742904~18484012,254075762~4430992,282485483~11941900,298643235~5745703,306903613~5021411,314393585~12544817,332944411~6919544,355806290~1463251,380563114~8232805,401962264~4926055] 2024-04-07T17:37:25.797 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [2329752~21327233,39912210~4133268,48881748~4683249,55380210~10623546,68206575~15199694,95781519~3680106,124240937~7583591,136996101~4514786,141885014~6484550,149552729~4403047,167772160~8831430,183274874~3283813,187506017~4831615,210147535~19406580,229742904~18484012,254075762~4430992,282485483~11941900,298643235~5745703,306903613~20034789,331350016~8513939,355806290~1463251,380563114~8232805,401962264~4926055] 2024-04-07T17:37:25.797 INFO:tasks.workunit.client.0.smithi100.stdout:from 14 to HEAD diff [3107878~17063292,39912210~4133268,48881748~4683249,55380210~10623546,68206575~15199694,95781519~3680106,126602360~5222168,136996101~4514786,141885014~6484550,149552729~4403047,168236274~8367316,183274874~3283813,210147535~5174157,219514234~10039881,229742904~18484012,254075762~4430992,282485483~7514691,298643235~5745703,306903613~5021411,314393585~7979426,323018422~3348552,332944411~6919544,355806290~1463251,380563114~8232805,401962264~4926055] 2024-04-07T17:37:25.797 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [3107878~17063292,39912210~4133268,48881748~4683249,55380210~10623546,68206575~15199694,95781519~3680106,126602360~5222168,136996101~4514786,141885014~6484550,149552729~4403047,168236274~8367316,183274874~3283813,210147535~5174157,219514234~10039881,229742904~18484012,254075762~4430992,282485483~7514691,298643235~5745703,306903613~5021411,314393585~7979426,323018422~3348552,332944411~6919544,355806290~1463251,380563114~8232805,401962264~4926055] 2024-04-07T17:37:25.805 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [3107878~17063292,39912210~4133268,48881748~4683249,55380210~10623546,68206575~15199694,95781519~3680106,126602360~5222168,136996101~4514786,141885014~6484550,149552729~4403047,167772160~8831430,183274874~3283813,210147535~5174157,219514234~10039881,229742904~18484012,254075762~4430992,282485483~7514691,298643235~5745703,306903613~15469398,323018422~3348552,331350016~8513939,355806290~1463251,380563114~8232805,401962264~4926055] 2024-04-07T17:37:25.805 INFO:tasks.workunit.client.0.smithi100.stdout:from 15 to HEAD diff [3107878~17063292,39912210~3191969,48881748~4683249,55380210~10623546,68206575~10344727,95781519~3680106,126602360~5222168,136996101~4514786,141885014~6484550,149552729~4403047,174179109~452103,183274874~3283813,210147535~5174157,219514234~10039881,229742904~14127132,254075762~1580725,282485483~7514691,298643235~5745703,306903613~5021411,323018422~3348552,332944411~6919544,355806290~1463251,380563114~8232805] 2024-04-07T17:37:25.805 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [3107878~17063292,39912210~3191969,48881748~4683249,55380210~10623546,68206575~10344727,95781519~3680106,126602360~5222168,136996101~4514786,141885014~6484550,149552729~4403047,174179109~452103,183274874~3283813,210147535~5174157,219514234~10039881,229742904~14127132,254075762~1580725,282485483~7514691,298643235~5745703,306903613~5021411,323018422~3348552,332944411~6919544,355806290~1463251,380563114~8232805] 2024-04-07T17:37:25.813 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [3107878~17063292,39912210~3191969,48881748~4683249,55380210~10623546,68206575~10344727,95781519~3680106,126602360~5222168,136996101~4514786,141885014~6484550,149552729~4403047,174179109~452103,183274874~3283813,210147535~5174157,219514234~10039881,229742904~14127132,254075762~1580725,282485483~7514691,298643235~5745703,306903613~5021411,323018422~3348552,331350016~8513939,355806290~1463251,380563114~8232805] 2024-04-07T17:37:25.813 INFO:tasks.workunit.client.0.smithi100.stdout:from 16 to HEAD diff [6688790~13482380,39912210~3191969,55380210~10623546,68206575~10344727,126602360~5222168,136996101~4514786,141885014~6484550,149552729~4403047,174179109~452103,183274874~3283813,210147535~5174157,219514234~10039881,229742904~14127132,282485483~6755344,298643235~5745703,306903613~5021411,355806290~1463251,386437457~2358462] 2024-04-07T17:37:25.813 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [6688790~13482380,39912210~3191969,55380210~10623546,68206575~10344727,126602360~5222168,136996101~4514786,141885014~6484550,149552729~4403047,174179109~452103,183274874~3283813,210147535~5174157,219514234~10039881,229742904~14127132,282485483~6755344,298643235~5745703,306903613~5021411,355806290~1463251,386437457~2358462] 2024-04-07T17:37:25.820 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [6688790~13482380,39912210~3191969,55380210~10623546,68206575~10344727,126602360~5222168,136996101~4514786,141885014~6484550,149552729~4403047,174179109~452103,183274874~3283813,210147535~5174157,219514234~10039881,229742904~14127132,282485483~6755344,298643235~5745703,306903613~5021411,355806290~1463251,386437457~2358462] 2024-04-07T17:37:25.820 INFO:tasks.workunit.client.0.smithi100.stdout:from 17 to HEAD diff [6688790~7360627,55380210~10623546,71826911~6724391,126602360~5222168,141885014~6484550,149552729~4403047,174179109~452103,183274874~3283813,210147535~5174157,219514234~5689083,226376149~484931,229742904~13365134,298643235~5745703,306903613~5021411,386437457~2358462] 2024-04-07T17:37:25.820 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [6688790~7360627,55380210~10623546,71826911~6724391,126602360~5222168,141885014~6484550,149552729~4403047,174179109~452103,183274874~3283813,210147535~5174157,219514234~5689083,226376149~484931,229742904~13365134,298643235~5745703,306903613~5021411,386437457~2358462] 2024-04-07T17:37:25.826 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [6688790~7360627,55380210~10623546,71826911~6724391,126602360~5222168,141885014~6484550,149552729~4403047,174179109~452103,183274874~3283813,210147535~5174157,219514234~5689083,226376149~484931,229742904~13365134,298643235~5745703,306903613~5021411,386437457~2358462] 2024-04-07T17:37:25.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:37:25 smithi144 bash[66654]: cluster 2024-04-07T17:37:24.478925+0000 mgr.y (mgr.25175) 2461 : cluster [DBG] pgmap v4048: 137 pgs: 137 active+clean; 856 MiB data, 5.7 GiB used, 709 GiB / 715 GiB avail; 131 KiB/s rd, 43 MiB/s wr, 510 op/s 2024-04-07T17:37:25.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:37:25 smithi100 bash[74942]: cluster 2024-04-07T17:37:24.478925+0000 mgr.y (mgr.25175) 2461 : cluster [DBG] pgmap v4048: 137 pgs: 137 active+clean; 856 MiB data, 5.7 GiB used, 709 GiB / 715 GiB avail; 131 KiB/s rd, 43 MiB/s wr, 510 op/s 2024-04-07T17:37:25.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:37:25 smithi100 bash[76202]: cluster 2024-04-07T17:37:24.478925+0000 mgr.y (mgr.25175) 2461 : cluster [DBG] pgmap v4048: 137 pgs: 137 active+clean; 856 MiB data, 5.7 GiB used, 709 GiB / 715 GiB avail; 131 KiB/s rd, 43 MiB/s wr, 510 op/s 2024-04-07T17:37:26.652 INFO:tasks.workunit.client.0.smithi100.stdout:from 0 to HEAD diff [2329752~26214847,29778841~737427,31769017~13613313,45857929~20145827,66861224~16545045,85466334~5069656,92999623~10564968,106500403~13616197,121501610~12475984,136270708~22564048,160337176~7153146,168236274~8367316,180237750~12590714,194102759~7364680,202420372~5254601,210147535~53978881,267179572~28414772,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350406459~6863082,360049192~84791,366341667~7420413,374580519~17465318,392232408~14655911,408516200~2525554] 2024-04-07T17:37:26.652 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~26214847,29778841~737427,31769017~13613313,45857929~20145827,66861224~16545045,85466334~5069656,92999623~10564968,106500403~13616197,121501610~12475984,136270708~22564048,160337176~7153146,168236274~8367316,180237750~12590714,194102759~7364680,202420372~5254601,210147535~53978881,267179572~28414772,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350406459~6863082,360049192~84791,366341667~7420413,374580519~17465318,392232408~14655911,408516200~2525554] 2024-04-07T17:37:26.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:37:26 smithi144 bash[66654]: cluster 2024-04-07T17:37:25.603817+0000 mon.a (mon.0) 3886 : cluster [DBG] osdmap e2013: 8 total, 8 up, 8 in 2024-04-07T17:37:26.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:37:26 smithi144 bash[66654]: audit 2024-04-07T17:37:25.642034+0000 mon.a (mon.0) 3887 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "120.1", "id": [3, 1]}]: dispatch 2024-04-07T17:37:26.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:37:26 smithi144 bash[66654]: audit 2024-04-07T17:37:25.642582+0000 mon.a (mon.0) 3888 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "120.3", "id": [4, 1]}]: dispatch 2024-04-07T17:37:26.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:37:26 smithi144 bash[66654]: audit 2024-04-07T17:37:25.643176+0000 mon.a (mon.0) 3889 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "120.5", "id": [4, 6]}]: dispatch 2024-04-07T17:37:26.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:37:26 smithi144 bash[66654]: audit 2024-04-07T17:37:25.643730+0000 mon.a (mon.0) 3890 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "120.8", "id": [2, 6]}]: dispatch 2024-04-07T17:37:26.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:37:26 smithi100 bash[74942]: cluster 2024-04-07T17:37:25.603817+0000 mon.a (mon.0) 3886 : cluster [DBG] osdmap e2013: 8 total, 8 up, 8 in 2024-04-07T17:37:26.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:37:26 smithi100 bash[74942]: audit 2024-04-07T17:37:25.642034+0000 mon.a (mon.0) 3887 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "120.1", "id": [3, 1]}]: dispatch 2024-04-07T17:37:26.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:37:26 smithi100 bash[74942]: audit 2024-04-07T17:37:25.642582+0000 mon.a (mon.0) 3888 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "120.3", "id": [4, 1]}]: dispatch 2024-04-07T17:37:26.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:37:26 smithi100 bash[74942]: audit 2024-04-07T17:37:25.643176+0000 mon.a (mon.0) 3889 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "120.5", "id": [4, 6]}]: dispatch 2024-04-07T17:37:26.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:37:26 smithi100 bash[74942]: audit 2024-04-07T17:37:25.643730+0000 mon.a (mon.0) 3890 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "120.8", "id": [2, 6]}]: dispatch 2024-04-07T17:37:26.955 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:37:26 smithi100 bash[76202]: cluster 2024-04-07T17:37:25.603817+0000 mon.a (mon.0) 3886 : cluster [DBG] osdmap e2013: 8 total, 8 up, 8 in 2024-04-07T17:37:26.955 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:37:26 smithi100 bash[76202]: audit 2024-04-07T17:37:25.642034+0000 mon.a (mon.0) 3887 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "120.1", "id": [3, 1]}]: dispatch 2024-04-07T17:37:26.955 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:37:26 smithi100 bash[76202]: audit 2024-04-07T17:37:25.642582+0000 mon.a (mon.0) 3888 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "120.3", "id": [4, 1]}]: dispatch 2024-04-07T17:37:26.955 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:37:26 smithi100 bash[76202]: audit 2024-04-07T17:37:25.643176+0000 mon.a (mon.0) 3889 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "120.5", "id": [4, 6]}]: dispatch 2024-04-07T17:37:26.955 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:37:26 smithi100 bash[76202]: audit 2024-04-07T17:37:25.643730+0000 mon.a (mon.0) 3890 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "120.8", "id": [2, 6]}]: dispatch 2024-04-07T17:37:27.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:37:27 smithi100 bash[74942]: cluster 2024-04-07T17:37:26.479635+0000 mgr.y (mgr.25175) 2462 : cluster [DBG] pgmap v4050: 137 pgs: 137 active+clean; 876 MiB data, 5.8 GiB used, 709 GiB / 715 GiB avail; 170 KiB/s rd, 39 MiB/s wr, 922 op/s 2024-04-07T17:37:27.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:37:27 smithi100 bash[74942]: audit 2024-04-07T17:37:26.604999+0000 mon.a (mon.0) 3891 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "120.1", "id": [3, 1]}]': finished 2024-04-07T17:37:27.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:37:27 smithi100 bash[74942]: audit 2024-04-07T17:37:26.605140+0000 mon.a (mon.0) 3892 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "120.3", "id": [4, 1]}]': finished 2024-04-07T17:37:27.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:37:27 smithi100 bash[74942]: audit 2024-04-07T17:37:26.605287+0000 mon.a (mon.0) 3893 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "120.5", "id": [4, 6]}]': finished 2024-04-07T17:37:27.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:37:27 smithi100 bash[74942]: audit 2024-04-07T17:37:26.605391+0000 mon.a (mon.0) 3894 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "120.8", "id": [2, 6]}]': finished 2024-04-07T17:37:27.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:37:27 smithi100 bash[74942]: cluster 2024-04-07T17:37:26.622263+0000 mon.a (mon.0) 3895 : cluster [DBG] osdmap e2014: 8 total, 8 up, 8 in 2024-04-07T17:37:27.955 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:37:27 smithi100 bash[76202]: cluster 2024-04-07T17:37:26.479635+0000 mgr.y (mgr.25175) 2462 : cluster [DBG] pgmap v4050: 137 pgs: 137 active+clean; 876 MiB data, 5.8 GiB used, 709 GiB / 715 GiB avail; 170 KiB/s rd, 39 MiB/s wr, 922 op/s 2024-04-07T17:37:27.955 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:37:27 smithi100 bash[76202]: audit 2024-04-07T17:37:26.604999+0000 mon.a (mon.0) 3891 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "120.1", "id": [3, 1]}]': finished 2024-04-07T17:37:27.955 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:37:27 smithi100 bash[76202]: audit 2024-04-07T17:37:26.605140+0000 mon.a (mon.0) 3892 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "120.3", "id": [4, 1]}]': finished 2024-04-07T17:37:27.955 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:37:27 smithi100 bash[76202]: audit 2024-04-07T17:37:26.605287+0000 mon.a (mon.0) 3893 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "120.5", "id": [4, 6]}]': finished 2024-04-07T17:37:27.955 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:37:27 smithi100 bash[76202]: audit 2024-04-07T17:37:26.605391+0000 mon.a (mon.0) 3894 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "120.8", "id": [2, 6]}]': finished 2024-04-07T17:37:27.955 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:37:27 smithi100 bash[76202]: cluster 2024-04-07T17:37:26.622263+0000 mon.a (mon.0) 3895 : cluster [DBG] osdmap e2014: 8 total, 8 up, 8 in 2024-04-07T17:37:28.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:37:27 smithi144 bash[66654]: cluster 2024-04-07T17:37:26.479635+0000 mgr.y (mgr.25175) 2462 : cluster [DBG] pgmap v4050: 137 pgs: 137 active+clean; 876 MiB data, 5.8 GiB used, 709 GiB / 715 GiB avail; 170 KiB/s rd, 39 MiB/s wr, 922 op/s 2024-04-07T17:37:28.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:37:27 smithi144 bash[66654]: audit 2024-04-07T17:37:26.604999+0000 mon.a (mon.0) 3891 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "120.1", "id": [3, 1]}]': finished 2024-04-07T17:37:28.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:37:27 smithi144 bash[66654]: audit 2024-04-07T17:37:26.605140+0000 mon.a (mon.0) 3892 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "120.3", "id": [4, 1]}]': finished 2024-04-07T17:37:28.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:37:27 smithi144 bash[66654]: audit 2024-04-07T17:37:26.605287+0000 mon.a (mon.0) 3893 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "120.5", "id": [4, 6]}]': finished 2024-04-07T17:37:28.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:37:27 smithi144 bash[66654]: audit 2024-04-07T17:37:26.605391+0000 mon.a (mon.0) 3894 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "120.8", "id": [2, 6]}]': finished 2024-04-07T17:37:28.116 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:37:27 smithi144 bash[66654]: cluster 2024-04-07T17:37:26.622263+0000 mon.a (mon.0) 3895 : cluster [DBG] osdmap e2014: 8 total, 8 up, 8 in 2024-04-07T17:37:28.704 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:37:28 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:37:28] "GET /metrics HTTP/1.1" 200 36760 "" "Prometheus/2.43.0" 2024-04-07T17:37:28.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:37:28 smithi100 bash[76202]: cluster 2024-04-07T17:37:27.615880+0000 mon.a (mon.0) 3896 : cluster [DBG] osdmap e2015: 8 total, 8 up, 8 in 2024-04-07T17:37:28.705 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:37:28 smithi100 bash[74942]: cluster 2024-04-07T17:37:27.615880+0000 mon.a (mon.0) 3896 : cluster [DBG] osdmap e2015: 8 total, 8 up, 8 in 2024-04-07T17:37:28.775 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~28544599,29360128~54046141,83886080~6649910,92274688~11289903,104857600~29119994,134217728~24617028,159383552~8106770,167772160~25056304,192937984~14736989,210147535~53978881,264241152~31353192,297795584~7201936,306184192~23819574,331350016~15772466,348127232~12006751,366341667~44700087] 2024-04-07T17:37:28.775 INFO:tasks.workunit.client.0.smithi100.stdout:from 1 to HEAD diff [2329752~26214847,29778841~737427,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~13616197,121501610~12475984,136270708~22564048,160337176~7153146,168236274~8367316,180237750~12590714,194866296~6601143,202420372~5254601,210147535~53978881,267179572~28414772,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350891716~1451525,355806290~1463251,360049192~84791,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:28.775 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~26214847,29778841~737427,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~13616197,121501610~12475984,136270708~22564048,160337176~7153146,168236274~8367316,180237750~12590714,194866296~6601143,202420372~5254601,210147535~53978881,267179572~28414772,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350891716~1451525,355806290~1463251,360049192~84791,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:28.795 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~28544599,29360128~24981239,55380210~10623546,66861224~16545045,92274688~11289903,104857600~29119994,134217728~24617028,159383552~8106770,167772160~25056304,194866296~12808677,210147535~53978881,264241152~31353192,297795584~7201936,306184192~23819574,331350016~15772466,350891716~1451525,355806290~4327693,366341667~25704170,394880591~2187459,398418332~12623422] 2024-04-07T17:37:28.795 INFO:tasks.workunit.client.0.smithi100.stdout:from 2 to HEAD diff [2329752~24230697,27329013~980763,29778841~737427,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~17685068,154799337~4035419,160337176~7153146,168236274~8367316,182034135~10794329,196727604~2108791,202420372~5254601,210147535~53978881,267179572~28414772,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350891716~1451525,355806290~1463251,360049192~84791,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:28.795 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~24230697,27329013~980763,29778841~737427,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~17685068,154799337~4035419,160337176~7153146,168236274~8367316,182034135~10794329,196727604~2108791,202420372~5254601,210147535~53978881,267179572~28414772,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350891716~1451525,355806290~1463251,360049192~84791,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:28.812 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~26560449,27329013~980763,29360128~24981239,55380210~10623546,66861224~16545045,92274688~11289903,106500403~5285199,112275487~21702107,134217728~19738048,154799337~4035419,159383552~8106770,167772160~8831430,182034135~10794329,196727604~2108791,201467439~6207534,210147535~53978881,264241152~31353192,297795584~7201936,306184192~23819574,331350016~15772466,350891716~1451525,355806290~4327693,366341667~25704170,394880591~2187459,398418332~12623422] 2024-04-07T17:37:28.812 INFO:tasks.workunit.client.0.smithi100.stdout:from 3 to HEAD diff [2329752~24230697,27329013~980763,29778841~737427,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~17685068,154799337~3192780,160337176~7153146,168236274~8367316,182034135~10303497,196727604~2108791,202420372~1435645,210147535~53978881,267179572~28414772,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350891716~1451525,355806290~1463251,360049192~84791,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:28.812 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~24230697,27329013~980763,29778841~737427,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~17685068,154799337~3192780,160337176~7153146,168236274~8367316,182034135~10303497,196727604~2108791,202420372~1435645,210147535~53978881,267179572~28414772,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350891716~1451525,355806290~1463251,360049192~84791,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:28.827 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~26560449,27329013~980763,29360128~24981239,55380210~10623546,66861224~16545045,92274688~11289903,106500403~5285199,112275487~21702107,134217728~19738048,154799337~3192780,159383552~8106770,167772160~8831430,182034135~10303497,196727604~2108791,202420372~1435645,210147535~53978881,264241152~31353192,297795584~7201936,306184192~23819574,331350016~15772466,350891716~1451525,355806290~4327693,366341667~25704170,394880591~2187459,398418332~12623422] 2024-04-07T17:37:28.828 INFO:tasks.workunit.client.0.smithi100.stdout:from 4 to HEAD diff [2329752~24230697,27329013~980763,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~12098856,149233764~4722012,154895619~3096498,160337176~7153146,168236274~8367316,182034135~10303497,196727604~2108791,202420372~1435645,210147535~53978881,267179572~13527794,281234113~14360231,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350891716~1451525,355806290~1463251,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:28.828 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~24230697,27329013~980763,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~12098856,149233764~4722012,154895619~3096498,160337176~7153146,168236274~8367316,182034135~10303497,196727604~2108791,202420372~1435645,210147535~53978881,267179572~13527794,281234113~14360231,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350891716~1451525,355806290~1463251,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:28.840 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~26560449,27329013~980763,30516268~23825099,55380210~10623546,66861224~16545045,92274688~11289903,106500403~5285199,112275487~21702107,134217728~14151836,149233764~4722012,154895619~3096498,159383552~8106770,167772160~8831430,182034135~10303497,196727604~2108791,202420372~1435645,210147535~53978881,264241152~16466214,281234113~14360231,297795584~7201936,306903613~23100153,331350016~15772466,350891716~1451525,355806290~1463251,366341667~25704170,394880591~2187459,398418332~12623422] 2024-04-07T17:37:28.840 INFO:tasks.workunit.client.0.smithi100.stdout:from 5 to HEAD diff [2329752~24230697,27329013~980763,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~12098856,149233764~4722012,154895619~3096498,166133159~1357163,168236274~8367316,182034135~10303497,196727604~2108791,202420372~1435645,210147535~38079381,248419159~15707257,267179572~13527794,281234113~14360231,298036890~6960630,306903613~5021411,314393585~13715786,332944411~11089603,345811290~1311192,355806290~1463251,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:28.840 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~24230697,27329013~980763,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~12098856,149233764~4722012,154895619~3096498,166133159~1357163,168236274~8367316,182034135~10303497,196727604~2108791,202420372~1435645,210147535~38079381,248419159~15707257,267179572~13527794,281234113~14360231,298036890~6960630,306903613~5021411,314393585~13715786,332944411~11089603,345811290~1311192,355806290~1463251,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:28.852 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~26560449,27329013~980763,30516268~23825099,55380210~10623546,66861224~16545045,92274688~11289903,106500403~5285199,112275487~21702107,134217728~14151836,149233764~4722012,154895619~3096498,166133159~1357163,167772160~8831430,182034135~10303497,196727604~2108791,202420372~1435645,210147535~38079381,248419159~15707257,264241152~16466214,281234113~14360231,298036890~6960630,306903613~21205758,331350016~15772466,355806290~1463251,366341667~7420413,374580519~17465318,394880591~2187459,398418332~12623422] 2024-04-07T17:37:28.852 INFO:tasks.workunit.client.0.smithi100.stdout:from 6 to HEAD diff [2329752~23418695,27329013~980763,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~12098856,149233764~4722012,154895619~3096498,168236274~8367316,182747004~9590628,202420372~1435645,210147535~38079381,250447165~8059589,267179572~11729329,281234113~14360231,298036890~6960630,306903613~5021411,314393585~13715786,332944411~6919544,345811290~1311192,355806290~1463251,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:28.852 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~23418695,27329013~980763,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~12098856,149233764~4722012,154895619~3096498,168236274~8367316,182747004~9590628,202420372~1435645,210147535~38079381,250447165~8059589,267179572~11729329,281234113~14360231,298036890~6960630,306903613~5021411,314393585~13715786,332944411~6919544,345811290~1311192,355806290~1463251,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:28.863 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~25748447,27329013~980763,30516268~23825099,55380210~10623546,66861224~16545045,92274688~11289903,106500403~5285199,112275487~21702107,134217728~14151836,149233764~4722012,154895619~3096498,167772160~8831430,182747004~9590628,202420372~1435645,210147535~38079381,250447165~8059589,264241152~14667749,281234113~14360231,298036890~6960630,306903613~21205758,331350016~8513939,344034014~3088468,355806290~1463251,366341667~7420413,374580519~17465318,394880591~2187459,398418332~12623422] 2024-04-07T17:37:28.863 INFO:tasks.workunit.client.0.smithi100.stdout:from 7 to HEAD diff [2329752~23418695,27329013~980763,31769017~13613313,45857929~8483438,55380210~10623546,68206575~15199694,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~5240179,141885014~6484550,149233764~4722012,154895619~3096498,168236274~8367316,182747004~9590628,210147535~38079381,250447165~8059589,267179572~11729329,281234113~14360231,298036890~6960630,306903613~5021411,314393585~13715786,332944411~6919544,355806290~1463251,366341667~7420413,377322524~14723313,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:28.864 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~23418695,27329013~980763,31769017~13613313,45857929~8483438,55380210~10623546,68206575~15199694,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~5240179,141885014~6484550,149233764~4722012,154895619~3096498,168236274~8367316,182747004~9590628,210147535~38079381,250447165~8059589,267179572~11729329,281234113~14360231,298036890~6960630,306903613~5021411,314393585~13715786,332944411~6919544,355806290~1463251,366341667~7420413,377322524~14723313,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:28.873 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~25748447,27329013~980763,30516268~23825099,55380210~10623546,68206575~15199694,92274688~11289903,106500403~5285199,112275487~21702107,134217728~7293159,141885014~6484550,149233764~4722012,154895619~3096498,167772160~8831430,182747004~9590628,210147535~38079381,250447165~8059589,264241152~14667749,281234113~14360231,298036890~6960630,306903613~21205758,331350016~8513939,355806290~1463251,366341667~7420413,377322524~14723313,394880591~2187459,398418332~12623422] 2024-04-07T17:37:28.873 INFO:tasks.workunit.client.0.smithi100.stdout:from 8 to HEAD diff [2329752~23418695,27329013~980763,31769017~13613313,45857929~8483438,55380210~10623546,68206575~15199694,95470042~8094549,106500403~5285199,112275487~7841113,121501610~10322918,136270708~5240179,141885014~6484550,149552729~4403047,168236274~8367316,182747004~9590628,210147535~38079381,250447165~8059589,267179572~11729329,281234113~14360231,298036890~6918207,306903613~5021411,314393585~13715786,332944411~6919544,355806290~1463251,366341667~7420413,377322524~14723313,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:28.874 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~23418695,27329013~980763,31769017~13613313,45857929~8483438,55380210~10623546,68206575~15199694,95470042~8094549,106500403~5285199,112275487~7841113,121501610~10322918,136270708~5240179,141885014~6484550,149552729~4403047,168236274~8367316,182747004~9590628,210147535~38079381,250447165~8059589,267179572~11729329,281234113~14360231,298036890~6918207,306903613~5021411,314393585~13715786,332944411~6919544,355806290~1463251,366341667~7420413,377322524~14723313,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:28.882 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~25748447,27329013~980763,30516268~23825099,55380210~10623546,68206575~15199694,95470042~8094549,106500403~5285199,112275487~19549041,134217728~7293159,141885014~6484550,149552729~4403047,167772160~8831430,182747004~9590628,210147535~38079381,250447165~8059589,264241152~14667749,281234113~14360231,298036890~6918207,306903613~21205758,331350016~8513939,355806290~1463251,366341667~7420413,377322524~14723313,394880591~2187459,398418332~12623422] 2024-04-07T17:37:28.882 INFO:tasks.workunit.client.0.smithi100.stdout:from 9 to HEAD diff [2329752~21800979,27329013~980763,37362822~8019508,45857929~8483438,55380210~10623546,68206575~15199694,95781519~7783072,106500403~5285199,112275487~7841113,121501610~10322918,136270708~5240179,141885014~6484550,149552729~4403047,168236274~8367316,183274874~3283813,186734432~153638,187506017~4831615,210147535~38079381,254075762~4430992,267179572~11729329,282485483~13108861,298036890~6918207,306903613~5021411,314393585~13715786,332944411~6919544,355806290~1463251,366341667~7420413,377322524~14723313,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:28.883 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~21800979,27329013~980763,37362822~8019508,45857929~8483438,55380210~10623546,68206575~15199694,95781519~7783072,106500403~5285199,112275487~7841113,121501610~10322918,136270708~5240179,141885014~6484550,149552729~4403047,168236274~8367316,183274874~3283813,186734432~153638,187506017~4831615,210147535~38079381,254075762~4430992,267179572~11729329,282485483~13108861,298036890~6918207,306903613~5021411,314393585~13715786,332944411~6919544,355806290~1463251,366341667~7420413,377322524~14723313,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:28.891 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~24130731,27329013~980763,37362822~16978545,55380210~10623546,68206575~15199694,95781519~7783072,106500403~5285199,112275487~19549041,134217728~7293159,141885014~6484550,149552729~4403047,167772160~8831430,183274874~3283813,186734432~153638,187506017~4831615,210147535~38079381,254075762~4430992,264241152~14667749,282485483~13108861,298036890~6918207,306903613~21205758,331350016~8513939,355806290~1463251,366341667~7420413,377322524~14723313,394880591~2187459,398418332~12623422] 2024-04-07T17:37:28.891 INFO:tasks.workunit.client.0.smithi100.stdout:from 10 to HEAD diff [2329752~21800979,27329013~249441,37362822~8019508,45857929~8483438,55380210~10623546,68206575~15199694,95781519~7783072,112275487~7841113,124240937~7583591,136270708~5240179,141885014~6484550,149552729~4403047,168236274~8367316,183274874~3283813,187506017~4831615,210147535~38079381,254075762~4430992,267179572~11729329,282485483~13108861,298036890~6918207,306903613~5021411,314393585~13715786,332944411~6919544,355806290~1463251,377322524~14723313,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:28.891 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~21800979,27329013~249441,37362822~8019508,45857929~8483438,55380210~10623546,68206575~15199694,95781519~7783072,112275487~7841113,124240937~7583591,136270708~5240179,141885014~6484550,149552729~4403047,168236274~8367316,183274874~3283813,187506017~4831615,210147535~38079381,254075762~4430992,267179572~11729329,282485483~13108861,298036890~6918207,306903613~5021411,314393585~13715786,332944411~6919544,355806290~1463251,377322524~14723313,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:28.899 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [2329752~21800979,27329013~249441,37362822~16978545,55380210~10623546,68206575~15199694,95781519~7783072,112275487~7841113,124240937~7583591,134217728~7293159,141885014~6484550,149552729~4403047,167772160~8831430,183274874~3283813,187506017~4831615,210147535~38079381,254075762~4430992,264241152~14667749,282485483~13108861,298036890~6918207,306903613~21205758,331350016~8513939,355806290~1463251,377322524~14723313,394880591~2187459,398418332~12623422] 2024-04-07T17:37:28.900 INFO:tasks.workunit.client.0.smithi100.stdout:from 11 to HEAD diff [2329752~21800979,27329013~249441,37386125~7996205,45857929~8483438,55380210~10623546,68206575~15199694,95781519~7783072,112275487~7841113,124240937~7583591,136270708~5240179,141885014~6484550,149552729~4403047,168236274~8367316,183274874~3283813,187506017~4831615,210147535~19406580,229742904~18484012,254075762~4430992,267179572~6811128,282485483~13108861,298036890~6918207,306903613~5021411,314393585~13715786,332944411~6919544,355806290~1463251,380094867~8701052,394880591~2187459,400878444~6009875,408516200~2525554] 2024-04-07T17:37:28.900 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~21800979,27329013~249441,37386125~7996205,45857929~8483438,55380210~10623546,68206575~15199694,95781519~7783072,112275487~7841113,124240937~7583591,136270708~5240179,141885014~6484550,149552729~4403047,168236274~8367316,183274874~3283813,187506017~4831615,210147535~19406580,229742904~18484012,254075762~4430992,267179572~6811128,282485483~13108861,298036890~6918207,306903613~5021411,314393585~13715786,332944411~6919544,355806290~1463251,380094867~8701052,394880591~2187459,400878444~6009875,408516200~2525554] 2024-04-07T17:37:28.908 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [2329752~21800979,27329013~249441,37386125~16955242,55380210~10623546,68206575~15199694,95781519~7783072,112275487~7841113,124240937~7583591,134217728~7293159,141885014~6484550,149552729~4403047,167772160~8831430,183274874~3283813,187506017~4831615,210147535~19406580,229742904~18484012,254075762~4430992,264241152~9749548,282485483~13108861,298036890~6918207,306903613~21205758,331350016~8513939,355806290~1463251,380094867~8701052,394880591~2187459,400878444~10163310] 2024-04-07T17:37:28.909 INFO:tasks.workunit.client.0.smithi100.stdout:from 12 to HEAD diff [2329752~21800979,27329013~249441,39912210~4133268,48881748~4683249,55380210~10623546,68206575~15199694,95781519~3680106,124240937~7583591,136270708~5240179,141885014~6484550,149552729~4403047,168236274~8367316,183274874~3283813,187506017~4831615,210147535~19406580,229742904~18484012,254075762~4430992,267179572~6811128,282485483~11941900,298643235~5745703,306903613~5021411,314393585~12544817,332944411~6919544,355806290~1463251,380563114~8232805,394880591~2187459,401962264~4926055,408516200~2525554] 2024-04-07T17:37:28.909 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~21800979,27329013~249441,39912210~4133268,48881748~4683249,55380210~10623546,68206575~15199694,95781519~3680106,124240937~7583591,136270708~5240179,141885014~6484550,149552729~4403047,168236274~8367316,183274874~3283813,187506017~4831615,210147535~19406580,229742904~18484012,254075762~4430992,267179572~6811128,282485483~11941900,298643235~5745703,306903613~5021411,314393585~12544817,332944411~6919544,355806290~1463251,380563114~8232805,394880591~2187459,401962264~4926055,408516200~2525554] 2024-04-07T17:37:28.917 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [2329752~21800979,27329013~249441,39912210~4133268,48881748~4683249,55380210~10623546,68206575~15199694,95781519~3680106,124240937~7583591,134217728~7293159,141885014~6484550,149552729~4403047,167772160~8831430,183274874~3283813,187506017~4831615,210147535~19406580,229742904~18484012,254075762~4430992,264241152~9749548,282485483~11941900,298643235~5745703,306903613~20034789,331350016~8513939,355806290~1463251,380563114~8232805,394880591~2187459,401962264~9079490] 2024-04-07T17:37:28.917 INFO:tasks.workunit.client.0.smithi100.stdout:from 13 to HEAD diff [2329752~21327233,39912210~4133268,48881748~4683249,55380210~10623546,68206575~15199694,95781519~3680106,124240937~7583591,136996101~4514786,141885014~6484550,149552729~4403047,168236274~8367316,183274874~3283813,187506017~4831615,210147535~19406580,229742904~18484012,254075762~4430992,282485483~11941900,298643235~5745703,306903613~5021411,314393585~12544817,332944411~6919544,355806290~1463251,380563114~8232805,401962264~4926055] 2024-04-07T17:37:28.917 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~21327233,39912210~4133268,48881748~4683249,55380210~10623546,68206575~15199694,95781519~3680106,124240937~7583591,136996101~4514786,141885014~6484550,149552729~4403047,168236274~8367316,183274874~3283813,187506017~4831615,210147535~19406580,229742904~18484012,254075762~4430992,282485483~11941900,298643235~5745703,306903613~5021411,314393585~12544817,332944411~6919544,355806290~1463251,380563114~8232805,401962264~4926055] 2024-04-07T17:37:28.925 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [2329752~21327233,39912210~4133268,48881748~4683249,55380210~10623546,68206575~15199694,95781519~3680106,124240937~7583591,136996101~4514786,141885014~6484550,149552729~4403047,167772160~8831430,183274874~3283813,187506017~4831615,210147535~19406580,229742904~18484012,254075762~4430992,282485483~11941900,298643235~5745703,306903613~20034789,331350016~8513939,355806290~1463251,380563114~8232805,401962264~4926055] 2024-04-07T17:37:28.926 INFO:tasks.workunit.client.0.smithi100.stdout:from 14 to HEAD diff [3107878~17063292,39912210~4133268,48881748~4683249,55380210~10623546,68206575~15199694,95781519~3680106,126602360~5222168,136996101~4514786,141885014~6484550,149552729~4403047,168236274~8367316,183274874~3283813,210147535~5174157,219514234~10039881,229742904~18484012,254075762~4430992,282485483~7514691,298643235~5745703,306903613~5021411,314393585~7979426,323018422~3348552,332944411~6919544,355806290~1463251,380563114~8232805,401962264~4926055] 2024-04-07T17:37:28.926 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [3107878~17063292,39912210~4133268,48881748~4683249,55380210~10623546,68206575~15199694,95781519~3680106,126602360~5222168,136996101~4514786,141885014~6484550,149552729~4403047,168236274~8367316,183274874~3283813,210147535~5174157,219514234~10039881,229742904~18484012,254075762~4430992,282485483~7514691,298643235~5745703,306903613~5021411,314393585~7979426,323018422~3348552,332944411~6919544,355806290~1463251,380563114~8232805,401962264~4926055] 2024-04-07T17:37:28.934 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [3107878~17063292,39912210~4133268,48881748~4683249,55380210~10623546,68206575~15199694,95781519~3680106,126602360~5222168,136996101~4514786,141885014~6484550,149552729~4403047,167772160~8831430,183274874~3283813,210147535~5174157,219514234~10039881,229742904~18484012,254075762~4430992,282485483~7514691,298643235~5745703,306903613~15469398,323018422~3348552,331350016~8513939,355806290~1463251,380563114~8232805,401962264~4926055] 2024-04-07T17:37:28.934 INFO:tasks.workunit.client.0.smithi100.stdout:from 15 to HEAD diff [3107878~17063292,39912210~3191969,48881748~4683249,55380210~10623546,68206575~10344727,95781519~3680106,126602360~5222168,136996101~4514786,141885014~6484550,149552729~4403047,174179109~452103,183274874~3283813,210147535~5174157,219514234~10039881,229742904~14127132,254075762~1580725,282485483~7514691,298643235~5745703,306903613~5021411,323018422~3348552,332944411~6919544,355806290~1463251,380563114~8232805] 2024-04-07T17:37:28.934 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [3107878~17063292,39912210~3191969,48881748~4683249,55380210~10623546,68206575~10344727,95781519~3680106,126602360~5222168,136996101~4514786,141885014~6484550,149552729~4403047,174179109~452103,183274874~3283813,210147535~5174157,219514234~10039881,229742904~14127132,254075762~1580725,282485483~7514691,298643235~5745703,306903613~5021411,323018422~3348552,332944411~6919544,355806290~1463251,380563114~8232805] 2024-04-07T17:37:28.942 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [3107878~17063292,39912210~3191969,48881748~4683249,55380210~10623546,68206575~10344727,95781519~3680106,126602360~5222168,136996101~4514786,141885014~6484550,149552729~4403047,174179109~452103,183274874~3283813,210147535~5174157,219514234~10039881,229742904~14127132,254075762~1580725,282485483~7514691,298643235~5745703,306903613~5021411,323018422~3348552,331350016~8513939,355806290~1463251,380563114~8232805] 2024-04-07T17:37:28.942 INFO:tasks.workunit.client.0.smithi100.stdout:from 16 to HEAD diff [6688790~13482380,39912210~3191969,55380210~10623546,68206575~10344727,126602360~5222168,136996101~4514786,141885014~6484550,149552729~4403047,174179109~452103,183274874~3283813,210147535~5174157,219514234~10039881,229742904~14127132,282485483~6755344,298643235~5745703,306903613~5021411,355806290~1463251,386437457~2358462] 2024-04-07T17:37:28.942 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [6688790~13482380,39912210~3191969,55380210~10623546,68206575~10344727,126602360~5222168,136996101~4514786,141885014~6484550,149552729~4403047,174179109~452103,183274874~3283813,210147535~5174157,219514234~10039881,229742904~14127132,282485483~6755344,298643235~5745703,306903613~5021411,355806290~1463251,386437457~2358462] 2024-04-07T17:37:28.950 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [6688790~13482380,39912210~3191969,55380210~10623546,68206575~10344727,126602360~5222168,136996101~4514786,141885014~6484550,149552729~4403047,174179109~452103,183274874~3283813,210147535~5174157,219514234~10039881,229742904~14127132,282485483~6755344,298643235~5745703,306903613~5021411,355806290~1463251,386437457~2358462] 2024-04-07T17:37:29.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:37:28 smithi144 bash[66654]: cluster 2024-04-07T17:37:27.615880+0000 mon.a (mon.0) 3896 : cluster [DBG] osdmap e2015: 8 total, 8 up, 8 in 2024-04-07T17:37:29.681 INFO:tasks.workunit.client.0.smithi100.stdout:from 0 to HEAD diff [2329752~26214847,29778841~737427,31769017~13613313,45857929~20145827,66861224~16545045,85466334~5069656,92999623~10564968,106500403~13616197,121501610~12475984,136270708~22564048,160337176~7153146,168236274~8367316,180237750~12590714,194102759~7364680,202420372~5254601,210147535~53978881,267179572~28414772,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350406459~6863082,360049192~84791,366341667~7420413,374580519~17465318,392232408~14655911,408516200~2525554] 2024-04-07T17:37:29.681 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~26214847,29778841~737427,31769017~13613313,45857929~20145827,66861224~16545045,85466334~5069656,92999623~10564968,106500403~13616197,121501610~12475984,136270708~22564048,160337176~7153146,168236274~8367316,180237750~12590714,194102759~7364680,202420372~5254601,210147535~53978881,267179572~28414772,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350406459~6863082,360049192~84791,366341667~7420413,374580519~17465318,392232408~14655911,408516200~2525554] 2024-04-07T17:37:29.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:37:29 smithi100 bash[74942]: cluster 2024-04-07T17:37:28.480490+0000 mgr.y (mgr.25175) 2463 : cluster [DBG] pgmap v4053: 137 pgs: 2 peering, 135 active+clean; 893 MiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 183 KiB/s rd, 17 MiB/s wr, 3.36k op/s 2024-04-07T17:37:29.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:37:29 smithi100 bash[74942]: cluster 2024-04-07T17:37:28.650802+0000 mon.a (mon.0) 3897 : cluster [DBG] osdmap e2016: 8 total, 8 up, 8 in 2024-04-07T17:37:29.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:37:29 smithi100 bash[76202]: cluster 2024-04-07T17:37:28.480490+0000 mgr.y (mgr.25175) 2463 : cluster [DBG] pgmap v4053: 137 pgs: 2 peering, 135 active+clean; 893 MiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 183 KiB/s rd, 17 MiB/s wr, 3.36k op/s 2024-04-07T17:37:29.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:37:29 smithi100 bash[76202]: cluster 2024-04-07T17:37:28.650802+0000 mon.a (mon.0) 3897 : cluster [DBG] osdmap e2016: 8 total, 8 up, 8 in 2024-04-07T17:37:30.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:37:29 smithi144 bash[66654]: cluster 2024-04-07T17:37:28.480490+0000 mgr.y (mgr.25175) 2463 : cluster [DBG] pgmap v4053: 137 pgs: 2 peering, 135 active+clean; 893 MiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 183 KiB/s rd, 17 MiB/s wr, 3.36k op/s 2024-04-07T17:37:30.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:37:29 smithi144 bash[66654]: cluster 2024-04-07T17:37:28.650802+0000 mon.a (mon.0) 3897 : cluster [DBG] osdmap e2016: 8 total, 8 up, 8 in 2024-04-07T17:37:30.711 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~28544599,29360128~54046141,83886080~6649910,92274688~11289903,104857600~29119994,134217728~24617028,159383552~8106770,167772160~25056304,192937984~14736989,210147535~53978881,264241152~31353192,297795584~7201936,306184192~23819574,331350016~15772466,348127232~12006751,366341667~44700087] 2024-04-07T17:37:30.711 INFO:tasks.workunit.client.0.smithi100.stdout:from 1 to HEAD diff [2329752~26214847,29778841~737427,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~13616197,121501610~12475984,136270708~22564048,160337176~7153146,168236274~8367316,180237750~12590714,194866296~6601143,202420372~5254601,210147535~53978881,267179572~28414772,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350891716~1451525,355806290~1463251,360049192~84791,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:30.711 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~26214847,29778841~737427,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~13616197,121501610~12475984,136270708~22564048,160337176~7153146,168236274~8367316,180237750~12590714,194866296~6601143,202420372~5254601,210147535~53978881,267179572~28414772,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350891716~1451525,355806290~1463251,360049192~84791,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:30.726 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~28544599,29360128~24981239,55380210~10623546,66861224~16545045,92274688~11289903,104857600~29119994,134217728~24617028,159383552~8106770,167772160~25056304,194866296~12808677,210147535~53978881,264241152~31353192,297795584~7201936,306184192~23819574,331350016~15772466,350891716~1451525,355806290~4327693,366341667~25704170,394880591~2187459,398418332~12623422] 2024-04-07T17:37:30.726 INFO:tasks.workunit.client.0.smithi100.stdout:from 2 to HEAD diff [2329752~24230697,27329013~980763,29778841~737427,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~17685068,154799337~4035419,160337176~7153146,168236274~8367316,182034135~10794329,196727604~2108791,202420372~5254601,210147535~53978881,267179572~28414772,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350891716~1451525,355806290~1463251,360049192~84791,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:30.726 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~24230697,27329013~980763,29778841~737427,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~17685068,154799337~4035419,160337176~7153146,168236274~8367316,182034135~10794329,196727604~2108791,202420372~5254601,210147535~53978881,267179572~28414772,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350891716~1451525,355806290~1463251,360049192~84791,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:30.739 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~26560449,27329013~980763,29360128~24981239,55380210~10623546,66861224~16545045,92274688~11289903,106500403~5285199,112275487~21702107,134217728~19738048,154799337~4035419,159383552~8106770,167772160~8831430,182034135~10794329,196727604~2108791,201467439~6207534,210147535~53978881,264241152~31353192,297795584~7201936,306184192~23819574,331350016~15772466,350891716~1451525,355806290~4327693,366341667~25704170,394880591~2187459,398418332~12623422] 2024-04-07T17:37:30.739 INFO:tasks.workunit.client.0.smithi100.stdout:from 3 to HEAD diff [2329752~24230697,27329013~980763,29778841~737427,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~17685068,154799337~3192780,160337176~7153146,168236274~8367316,182034135~10303497,196727604~2108791,202420372~1435645,210147535~53978881,267179572~28414772,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350891716~1451525,355806290~1463251,360049192~84791,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:30.739 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~24230697,27329013~980763,29778841~737427,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~17685068,154799337~3192780,160337176~7153146,168236274~8367316,182034135~10303497,196727604~2108791,202420372~1435645,210147535~53978881,267179572~28414772,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350891716~1451525,355806290~1463251,360049192~84791,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:30.751 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~26560449,27329013~980763,29360128~24981239,55380210~10623546,66861224~16545045,92274688~11289903,106500403~5285199,112275487~21702107,134217728~19738048,154799337~3192780,159383552~8106770,167772160~8831430,182034135~10303497,196727604~2108791,202420372~1435645,210147535~53978881,264241152~31353192,297795584~7201936,306184192~23819574,331350016~15772466,350891716~1451525,355806290~4327693,366341667~25704170,394880591~2187459,398418332~12623422] 2024-04-07T17:37:30.751 INFO:tasks.workunit.client.0.smithi100.stdout:from 4 to HEAD diff [2329752~24230697,27329013~980763,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~12098856,149233764~4722012,154895619~3096498,160337176~7153146,168236274~8367316,182034135~10303497,196727604~2108791,202420372~1435645,210147535~53978881,267179572~13527794,281234113~14360231,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350891716~1451525,355806290~1463251,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:30.751 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~24230697,27329013~980763,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~12098856,149233764~4722012,154895619~3096498,160337176~7153146,168236274~8367316,182034135~10303497,196727604~2108791,202420372~1435645,210147535~53978881,267179572~13527794,281234113~14360231,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350891716~1451525,355806290~1463251,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:30.761 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~26560449,27329013~980763,30516268~23825099,55380210~10623546,66861224~16545045,92274688~11289903,106500403~5285199,112275487~21702107,134217728~14151836,149233764~4722012,154895619~3096498,159383552~8106770,167772160~8831430,182034135~10303497,196727604~2108791,202420372~1435645,210147535~53978881,264241152~16466214,281234113~14360231,297795584~7201936,306903613~23100153,331350016~15772466,350891716~1451525,355806290~1463251,366341667~25704170,394880591~2187459,398418332~12623422] 2024-04-07T17:37:30.762 INFO:tasks.workunit.client.0.smithi100.stdout:from 5 to HEAD diff [2329752~24230697,27329013~980763,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~12098856,149233764~4722012,154895619~3096498,166133159~1357163,168236274~8367316,182034135~10303497,196727604~2108791,202420372~1435645,210147535~38079381,248419159~15707257,267179572~13527794,281234113~14360231,298036890~6960630,306903613~5021411,314393585~13715786,332944411~11089603,345811290~1311192,355806290~1463251,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:30.762 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~24230697,27329013~980763,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~12098856,149233764~4722012,154895619~3096498,166133159~1357163,168236274~8367316,182034135~10303497,196727604~2108791,202420372~1435645,210147535~38079381,248419159~15707257,267179572~13527794,281234113~14360231,298036890~6960630,306903613~5021411,314393585~13715786,332944411~11089603,345811290~1311192,355806290~1463251,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:30.772 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~26560449,27329013~980763,30516268~23825099,55380210~10623546,66861224~16545045,92274688~11289903,106500403~5285199,112275487~21702107,134217728~14151836,149233764~4722012,154895619~3096498,166133159~1357163,167772160~8831430,182034135~10303497,196727604~2108791,202420372~1435645,210147535~38079381,248419159~15707257,264241152~16466214,281234113~14360231,298036890~6960630,306903613~21205758,331350016~15772466,355806290~1463251,366341667~7420413,374580519~17465318,394880591~2187459,398418332~12623422] 2024-04-07T17:37:30.772 INFO:tasks.workunit.client.0.smithi100.stdout:from 6 to HEAD diff [2329752~23418695,27329013~980763,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~12098856,149233764~4722012,154895619~3096498,168236274~8367316,182747004~9590628,202420372~1435645,210147535~38079381,250447165~8059589,267179572~11729329,281234113~14360231,298036890~6960630,306903613~5021411,314393585~13715786,332944411~6919544,345811290~1311192,355806290~1463251,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:30.772 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~23418695,27329013~980763,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~12098856,149233764~4722012,154895619~3096498,168236274~8367316,182747004~9590628,202420372~1435645,210147535~38079381,250447165~8059589,267179572~11729329,281234113~14360231,298036890~6960630,306903613~5021411,314393585~13715786,332944411~6919544,345811290~1311192,355806290~1463251,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:30.781 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~25748447,27329013~980763,30516268~23825099,55380210~10623546,66861224~16545045,92274688~11289903,106500403~5285199,112275487~21702107,134217728~14151836,149233764~4722012,154895619~3096498,167772160~8831430,182747004~9590628,202420372~1435645,210147535~38079381,250447165~8059589,264241152~14667749,281234113~14360231,298036890~6960630,306903613~21205758,331350016~8513939,344034014~3088468,355806290~1463251,366341667~7420413,374580519~17465318,394880591~2187459,398418332~12623422] 2024-04-07T17:37:30.781 INFO:tasks.workunit.client.0.smithi100.stdout:from 7 to HEAD diff [2329752~23418695,27329013~980763,31769017~13613313,45857929~8483438,55380210~10623546,68206575~15199694,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~5240179,141885014~6484550,149233764~4722012,154895619~3096498,168236274~8367316,182747004~9590628,210147535~38079381,250447165~8059589,267179572~11729329,281234113~14360231,298036890~6960630,306903613~5021411,314393585~13715786,332944411~6919544,355806290~1463251,366341667~7420413,377322524~14723313,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:30.781 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~23418695,27329013~980763,31769017~13613313,45857929~8483438,55380210~10623546,68206575~15199694,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~5240179,141885014~6484550,149233764~4722012,154895619~3096498,168236274~8367316,182747004~9590628,210147535~38079381,250447165~8059589,267179572~11729329,281234113~14360231,298036890~6960630,306903613~5021411,314393585~13715786,332944411~6919544,355806290~1463251,366341667~7420413,377322524~14723313,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:30.789 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~25748447,27329013~980763,30516268~23825099,55380210~10623546,68206575~15199694,92274688~11289903,106500403~5285199,112275487~21702107,134217728~7293159,141885014~6484550,149233764~4722012,154895619~3096498,167772160~8831430,182747004~9590628,210147535~38079381,250447165~8059589,264241152~14667749,281234113~14360231,298036890~6960630,306903613~21205758,331350016~8513939,355806290~1463251,366341667~7420413,377322524~14723313,394880591~2187459,398418332~12623422] 2024-04-07T17:37:30.789 INFO:tasks.workunit.client.0.smithi100.stdout:from 8 to HEAD diff [2329752~23418695,27329013~980763,31769017~13613313,45857929~8483438,55380210~10623546,68206575~15199694,95470042~8094549,106500403~5285199,112275487~7841113,121501610~10322918,136270708~5240179,141885014~6484550,149552729~4403047,168236274~8367316,182747004~9590628,210147535~38079381,250447165~8059589,267179572~11729329,281234113~14360231,298036890~6918207,306903613~5021411,314393585~13715786,332944411~6919544,355806290~1463251,366341667~7420413,377322524~14723313,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:30.789 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~23418695,27329013~980763,31769017~13613313,45857929~8483438,55380210~10623546,68206575~15199694,95470042~8094549,106500403~5285199,112275487~7841113,121501610~10322918,136270708~5240179,141885014~6484550,149552729~4403047,168236274~8367316,182747004~9590628,210147535~38079381,250447165~8059589,267179572~11729329,281234113~14360231,298036890~6918207,306903613~5021411,314393585~13715786,332944411~6919544,355806290~1463251,366341667~7420413,377322524~14723313,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:30.797 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~25748447,27329013~980763,30516268~23825099,55380210~10623546,68206575~15199694,95470042~8094549,106500403~5285199,112275487~19549041,134217728~7293159,141885014~6484550,149552729~4403047,167772160~8831430,182747004~9590628,210147535~38079381,250447165~8059589,264241152~14667749,281234113~14360231,298036890~6918207,306903613~21205758,331350016~8513939,355806290~1463251,366341667~7420413,377322524~14723313,394880591~2187459,398418332~12623422] 2024-04-07T17:37:30.797 INFO:tasks.workunit.client.0.smithi100.stdout:from 9 to HEAD diff [2329752~21800979,27329013~980763,37362822~8019508,45857929~8483438,55380210~10623546,68206575~15199694,95781519~7783072,106500403~5285199,112275487~7841113,121501610~10322918,136270708~5240179,141885014~6484550,149552729~4403047,168236274~8367316,183274874~3283813,186734432~153638,187506017~4831615,210147535~38079381,254075762~4430992,267179572~11729329,282485483~13108861,298036890~6918207,306903613~5021411,314393585~13715786,332944411~6919544,355806290~1463251,366341667~7420413,377322524~14723313,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:30.797 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~21800979,27329013~980763,37362822~8019508,45857929~8483438,55380210~10623546,68206575~15199694,95781519~7783072,106500403~5285199,112275487~7841113,121501610~10322918,136270708~5240179,141885014~6484550,149552729~4403047,168236274~8367316,183274874~3283813,186734432~153638,187506017~4831615,210147535~38079381,254075762~4430992,267179572~11729329,282485483~13108861,298036890~6918207,306903613~5021411,314393585~13715786,332944411~6919544,355806290~1463251,366341667~7420413,377322524~14723313,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:30.804 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~24130731,27329013~980763,37362822~16978545,55380210~10623546,68206575~15199694,95781519~7783072,106500403~5285199,112275487~19549041,134217728~7293159,141885014~6484550,149552729~4403047,167772160~8831430,183274874~3283813,186734432~153638,187506017~4831615,210147535~38079381,254075762~4430992,264241152~14667749,282485483~13108861,298036890~6918207,306903613~21205758,331350016~8513939,355806290~1463251,366341667~7420413,377322524~14723313,394880591~2187459,398418332~12623422] 2024-04-07T17:37:30.805 INFO:tasks.workunit.client.0.smithi100.stdout:from 10 to HEAD diff [2329752~21800979,27329013~249441,37362822~8019508,45857929~8483438,55380210~10623546,68206575~15199694,95781519~7783072,112275487~7841113,124240937~7583591,136270708~5240179,141885014~6484550,149552729~4403047,168236274~8367316,183274874~3283813,187506017~4831615,210147535~38079381,254075762~4430992,267179572~11729329,282485483~13108861,298036890~6918207,306903613~5021411,314393585~13715786,332944411~6919544,355806290~1463251,377322524~14723313,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:30.805 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~21800979,27329013~249441,37362822~8019508,45857929~8483438,55380210~10623546,68206575~15199694,95781519~7783072,112275487~7841113,124240937~7583591,136270708~5240179,141885014~6484550,149552729~4403047,168236274~8367316,183274874~3283813,187506017~4831615,210147535~38079381,254075762~4430992,267179572~11729329,282485483~13108861,298036890~6918207,306903613~5021411,314393585~13715786,332944411~6919544,355806290~1463251,377322524~14723313,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:30.812 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [2329752~21800979,27329013~249441,37362822~16978545,55380210~10623546,68206575~15199694,95781519~7783072,112275487~7841113,124240937~7583591,134217728~7293159,141885014~6484550,149552729~4403047,167772160~8831430,183274874~3283813,187506017~4831615,210147535~38079381,254075762~4430992,264241152~14667749,282485483~13108861,298036890~6918207,306903613~21205758,331350016~8513939,355806290~1463251,377322524~14723313,394880591~2187459,398418332~12623422] 2024-04-07T17:37:30.812 INFO:tasks.workunit.client.0.smithi100.stdout:from 11 to HEAD diff [2329752~21800979,27329013~249441,37386125~7996205,45857929~8483438,55380210~10623546,68206575~15199694,95781519~7783072,112275487~7841113,124240937~7583591,136270708~5240179,141885014~6484550,149552729~4403047,168236274~8367316,183274874~3283813,187506017~4831615,210147535~19406580,229742904~18484012,254075762~4430992,267179572~6811128,282485483~13108861,298036890~6918207,306903613~5021411,314393585~13715786,332944411~6919544,355806290~1463251,380094867~8701052,394880591~2187459,400878444~6009875,408516200~2525554] 2024-04-07T17:37:30.812 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~21800979,27329013~249441,37386125~7996205,45857929~8483438,55380210~10623546,68206575~15199694,95781519~7783072,112275487~7841113,124240937~7583591,136270708~5240179,141885014~6484550,149552729~4403047,168236274~8367316,183274874~3283813,187506017~4831615,210147535~19406580,229742904~18484012,254075762~4430992,267179572~6811128,282485483~13108861,298036890~6918207,306903613~5021411,314393585~13715786,332944411~6919544,355806290~1463251,380094867~8701052,394880591~2187459,400878444~6009875,408516200~2525554] 2024-04-07T17:37:30.819 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [2329752~21800979,27329013~249441,37386125~16955242,55380210~10623546,68206575~15199694,95781519~7783072,112275487~7841113,124240937~7583591,134217728~7293159,141885014~6484550,149552729~4403047,167772160~8831430,183274874~3283813,187506017~4831615,210147535~19406580,229742904~18484012,254075762~4430992,264241152~9749548,282485483~13108861,298036890~6918207,306903613~21205758,331350016~8513939,355806290~1463251,380094867~8701052,394880591~2187459,400878444~10163310] 2024-04-07T17:37:30.819 INFO:tasks.workunit.client.0.smithi100.stdout:from 12 to HEAD diff [2329752~21800979,27329013~249441,39912210~4133268,48881748~4683249,55380210~10623546,68206575~15199694,95781519~3680106,124240937~7583591,136270708~5240179,141885014~6484550,149552729~4403047,168236274~8367316,183274874~3283813,187506017~4831615,210147535~19406580,229742904~18484012,254075762~4430992,267179572~6811128,282485483~11941900,298643235~5745703,306903613~5021411,314393585~12544817,332944411~6919544,355806290~1463251,380563114~8232805,394880591~2187459,401962264~4926055,408516200~2525554] 2024-04-07T17:37:30.819 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~21800979,27329013~249441,39912210~4133268,48881748~4683249,55380210~10623546,68206575~15199694,95781519~3680106,124240937~7583591,136270708~5240179,141885014~6484550,149552729~4403047,168236274~8367316,183274874~3283813,187506017~4831615,210147535~19406580,229742904~18484012,254075762~4430992,267179572~6811128,282485483~11941900,298643235~5745703,306903613~5021411,314393585~12544817,332944411~6919544,355806290~1463251,380563114~8232805,394880591~2187459,401962264~4926055,408516200~2525554] 2024-04-07T17:37:30.828 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [2329752~21800979,27329013~249441,39912210~4133268,48881748~4683249,55380210~10623546,68206575~15199694,95781519~3680106,124240937~7583591,134217728~7293159,141885014~6484550,149552729~4403047,167772160~8831430,183274874~3283813,187506017~4831615,210147535~19406580,229742904~18484012,254075762~4430992,264241152~9749548,282485483~11941900,298643235~5745703,306903613~20034789,331350016~8513939,355806290~1463251,380563114~8232805,394880591~2187459,401962264~9079490] 2024-04-07T17:37:30.828 INFO:tasks.workunit.client.0.smithi100.stdout:from 13 to HEAD diff [2329752~21327233,39912210~4133268,48881748~4683249,55380210~10623546,68206575~15199694,95781519~3680106,124240937~7583591,136996101~4514786,141885014~6484550,149552729~4403047,168236274~8367316,183274874~3283813,187506017~4831615,210147535~19406580,229742904~18484012,254075762~4430992,282485483~11941900,298643235~5745703,306903613~5021411,314393585~12544817,332944411~6919544,355806290~1463251,380563114~8232805,401962264~4926055] 2024-04-07T17:37:30.828 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~21327233,39912210~4133268,48881748~4683249,55380210~10623546,68206575~15199694,95781519~3680106,124240937~7583591,136996101~4514786,141885014~6484550,149552729~4403047,168236274~8367316,183274874~3283813,187506017~4831615,210147535~19406580,229742904~18484012,254075762~4430992,282485483~11941900,298643235~5745703,306903613~5021411,314393585~12544817,332944411~6919544,355806290~1463251,380563114~8232805,401962264~4926055] 2024-04-07T17:37:30.835 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [2329752~21327233,39912210~4133268,48881748~4683249,55380210~10623546,68206575~15199694,95781519~3680106,124240937~7583591,136996101~4514786,141885014~6484550,149552729~4403047,167772160~8831430,183274874~3283813,187506017~4831615,210147535~19406580,229742904~18484012,254075762~4430992,282485483~11941900,298643235~5745703,306903613~20034789,331350016~8513939,355806290~1463251,380563114~8232805,401962264~4926055] 2024-04-07T17:37:30.836 INFO:tasks.workunit.client.0.smithi100.stdout:from 14 to HEAD diff [3107878~17063292,39912210~4133268,48881748~4683249,55380210~10623546,68206575~15199694,95781519~3680106,126602360~5222168,136996101~4514786,141885014~6484550,149552729~4403047,168236274~8367316,183274874~3283813,210147535~5174157,219514234~10039881,229742904~18484012,254075762~4430992,282485483~7514691,298643235~5745703,306903613~5021411,314393585~7979426,323018422~3348552,332944411~6919544,355806290~1463251,380563114~8232805,401962264~4926055] 2024-04-07T17:37:30.836 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [3107878~17063292,39912210~4133268,48881748~4683249,55380210~10623546,68206575~15199694,95781519~3680106,126602360~5222168,136996101~4514786,141885014~6484550,149552729~4403047,168236274~8367316,183274874~3283813,210147535~5174157,219514234~10039881,229742904~18484012,254075762~4430992,282485483~7514691,298643235~5745703,306903613~5021411,314393585~7979426,323018422~3348552,332944411~6919544,355806290~1463251,380563114~8232805,401962264~4926055] 2024-04-07T17:37:30.842 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [3107878~17063292,39912210~4133268,48881748~4683249,55380210~10623546,68206575~15199694,95781519~3680106,126602360~5222168,136996101~4514786,141885014~6484550,149552729~4403047,167772160~8831430,183274874~3283813,210147535~5174157,219514234~10039881,229742904~18484012,254075762~4430992,282485483~7514691,298643235~5745703,306903613~15469398,323018422~3348552,331350016~8513939,355806290~1463251,380563114~8232805,401962264~4926055] 2024-04-07T17:37:30.842 INFO:tasks.workunit.client.0.smithi100.stdout:from 15 to HEAD diff [3107878~17063292,39912210~3191969,48881748~4683249,55380210~10623546,68206575~10344727,95781519~3680106,126602360~5222168,136996101~4514786,141885014~6484550,149552729~4403047,174179109~452103,183274874~3283813,210147535~5174157,219514234~10039881,229742904~14127132,254075762~1580725,282485483~7514691,298643235~5745703,306903613~5021411,323018422~3348552,332944411~6919544,355806290~1463251,380563114~8232805] 2024-04-07T17:37:30.843 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [3107878~17063292,39912210~3191969,48881748~4683249,55380210~10623546,68206575~10344727,95781519~3680106,126602360~5222168,136996101~4514786,141885014~6484550,149552729~4403047,174179109~452103,183274874~3283813,210147535~5174157,219514234~10039881,229742904~14127132,254075762~1580725,282485483~7514691,298643235~5745703,306903613~5021411,323018422~3348552,332944411~6919544,355806290~1463251,380563114~8232805] 2024-04-07T17:37:30.849 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [3107878~17063292,39912210~3191969,48881748~4683249,55380210~10623546,68206575~10344727,95781519~3680106,126602360~5222168,136996101~4514786,141885014~6484550,149552729~4403047,174179109~452103,183274874~3283813,210147535~5174157,219514234~10039881,229742904~14127132,254075762~1580725,282485483~7514691,298643235~5745703,306903613~5021411,323018422~3348552,331350016~8513939,355806290~1463251,380563114~8232805] 2024-04-07T17:37:30.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:37:30 smithi100 bash[74942]: cluster 2024-04-07T17:37:29.647071+0000 mon.a (mon.0) 3898 : cluster [DBG] osdmap e2017: 8 total, 8 up, 8 in 2024-04-07T17:37:30.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:37:30 smithi100 bash[76202]: cluster 2024-04-07T17:37:29.647071+0000 mon.a (mon.0) 3898 : cluster [DBG] osdmap e2017: 8 total, 8 up, 8 in 2024-04-07T17:37:31.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:37:30 smithi144 bash[66654]: cluster 2024-04-07T17:37:29.647071+0000 mon.a (mon.0) 3898 : cluster [DBG] osdmap e2017: 8 total, 8 up, 8 in 2024-04-07T17:37:31.712 INFO:tasks.workunit.client.0.smithi100.stdout:from 0 to HEAD diff [2329752~26214847,29778841~737427,31769017~13613313,45857929~20145827,66861224~16545045,85466334~5069656,92999623~10564968,106500403~13616197,121501610~12475984,136270708~22564048,160337176~7153146,168236274~8367316,180237750~12590714,194102759~7364680,202420372~5254601,210147535~53978881,267179572~28414772,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350406459~6863082,360049192~84791,366341667~7420413,374580519~17465318,392232408~14655911,408516200~2525554] 2024-04-07T17:37:31.712 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~26214847,29778841~737427,31769017~13613313,45857929~20145827,66861224~16545045,85466334~5069656,92999623~10564968,106500403~13616197,121501610~12475984,136270708~22564048,160337176~7153146,168236274~8367316,180237750~12590714,194102759~7364680,202420372~5254601,210147535~53978881,267179572~28414772,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350406459~6863082,360049192~84791,366341667~7420413,374580519~17465318,392232408~14655911,408516200~2525554] 2024-04-07T17:37:31.765 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~28544599,29360128~54046141,83886080~6649910,92274688~11289903,104857600~29119994,134217728~24617028,159383552~8106770,167772160~25056304,192937984~14736989,210147535~53978881,264241152~31353192,297795584~7201936,306184192~23819574,331350016~15772466,348127232~12006751,366341667~44700087] 2024-04-07T17:37:31.765 INFO:tasks.workunit.client.0.smithi100.stdout:from 1 to HEAD diff [2329752~26214847,29778841~737427,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~13616197,121501610~12475984,136270708~22564048,160337176~7153146,168236274~8367316,180237750~12590714,194866296~6601143,202420372~5254601,210147535~53978881,267179572~28414772,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350891716~1451525,355806290~1463251,360049192~84791,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:31.765 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~26214847,29778841~737427,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~13616197,121501610~12475984,136270708~22564048,160337176~7153146,168236274~8367316,180237750~12590714,194866296~6601143,202420372~5254601,210147535~53978881,267179572~28414772,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350891716~1451525,355806290~1463251,360049192~84791,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:31.779 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~28544599,29360128~24981239,55380210~10623546,66861224~16545045,92274688~11289903,104857600~29119994,134217728~24617028,159383552~8106770,167772160~25056304,194866296~12808677,210147535~53978881,264241152~31353192,297795584~7201936,306184192~23819574,331350016~15772466,350891716~1451525,355806290~4327693,366341667~25704170,394880591~2187459,398418332~12623422] 2024-04-07T17:37:31.779 INFO:tasks.workunit.client.0.smithi100.stdout:from 2 to HEAD diff [2329752~24230697,27329013~980763,29778841~737427,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~17685068,154799337~4035419,160337176~7153146,168236274~8367316,182034135~10794329,196727604~2108791,202420372~5254601,210147535~53978881,267179572~28414772,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350891716~1451525,355806290~1463251,360049192~84791,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:31.780 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~24230697,27329013~980763,29778841~737427,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~17685068,154799337~4035419,160337176~7153146,168236274~8367316,182034135~10794329,196727604~2108791,202420372~5254601,210147535~53978881,267179572~28414772,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350891716~1451525,355806290~1463251,360049192~84791,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:31.788 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~26560449,27329013~980763,29360128~24981239,55380210~10623546,66861224~16545045,92274688~11289903,106500403~5285199,112275487~21702107,134217728~19738048,154799337~4035419,159383552~8106770,167772160~8831430,182034135~10794329,196727604~2108791,201467439~6207534,210147535~53978881,264241152~31353192,297795584~7201936,306184192~23819574,331350016~15772466,350891716~1451525,355806290~4327693,366341667~25704170,394880591~2187459,398418332~12623422] 2024-04-07T17:37:31.788 INFO:tasks.workunit.client.0.smithi100.stdout:from 3 to HEAD diff [2329752~24230697,27329013~980763,29778841~737427,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~17685068,154799337~3192780,160337176~7153146,168236274~8367316,182034135~10303497,196727604~2108791,202420372~1435645,210147535~53978881,267179572~28414772,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350891716~1451525,355806290~1463251,360049192~84791,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:31.788 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~24230697,27329013~980763,29778841~737427,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~17685068,154799337~3192780,160337176~7153146,168236274~8367316,182034135~10303497,196727604~2108791,202420372~1435645,210147535~53978881,267179572~28414772,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350891716~1451525,355806290~1463251,360049192~84791,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:31.801 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~26560449,27329013~980763,29360128~24981239,55380210~10623546,66861224~16545045,92274688~11289903,106500403~5285199,112275487~21702107,134217728~19738048,154799337~3192780,159383552~8106770,167772160~8831430,182034135~10303497,196727604~2108791,202420372~1435645,210147535~53978881,264241152~31353192,297795584~7201936,306184192~23819574,331350016~15772466,350891716~1451525,355806290~4327693,366341667~25704170,394880591~2187459,398418332~12623422] 2024-04-07T17:37:31.801 INFO:tasks.workunit.client.0.smithi100.stdout:from 4 to HEAD diff [2329752~24230697,27329013~980763,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~12098856,149233764~4722012,154895619~3096498,160337176~7153146,168236274~8367316,182034135~10303497,196727604~2108791,202420372~1435645,210147535~53978881,267179572~13527794,281234113~14360231,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350891716~1451525,355806290~1463251,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:31.801 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~24230697,27329013~980763,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~12098856,149233764~4722012,154895619~3096498,160337176~7153146,168236274~8367316,182034135~10303497,196727604~2108791,202420372~1435645,210147535~53978881,267179572~13527794,281234113~14360231,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350891716~1451525,355806290~1463251,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:31.809 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~26560449,27329013~980763,30516268~23825099,55380210~10623546,66861224~16545045,92274688~11289903,106500403~5285199,112275487~21702107,134217728~14151836,149233764~4722012,154895619~3096498,159383552~8106770,167772160~8831430,182034135~10303497,196727604~2108791,202420372~1435645,210147535~53978881,264241152~16466214,281234113~14360231,297795584~7201936,306903613~23100153,331350016~15772466,350891716~1451525,355806290~1463251,366341667~25704170,394880591~2187459,398418332~12623422] 2024-04-07T17:37:31.809 INFO:tasks.workunit.client.0.smithi100.stdout:from 5 to HEAD diff [2329752~24230697,27329013~980763,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~12098856,149233764~4722012,154895619~3096498,166133159~1357163,168236274~8367316,182034135~10303497,196727604~2108791,202420372~1435645,210147535~38079381,248419159~15707257,267179572~13527794,281234113~14360231,298036890~6960630,306903613~5021411,314393585~13715786,332944411~11089603,345811290~1311192,355806290~1463251,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:31.809 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~24230697,27329013~980763,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~12098856,149233764~4722012,154895619~3096498,166133159~1357163,168236274~8367316,182034135~10303497,196727604~2108791,202420372~1435645,210147535~38079381,248419159~15707257,267179572~13527794,281234113~14360231,298036890~6960630,306903613~5021411,314393585~13715786,332944411~11089603,345811290~1311192,355806290~1463251,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:31.818 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~26560449,27329013~980763,30516268~23825099,55380210~10623546,66861224~16545045,92274688~11289903,106500403~5285199,112275487~21702107,134217728~14151836,149233764~4722012,154895619~3096498,166133159~1357163,167772160~8831430,182034135~10303497,196727604~2108791,202420372~1435645,210147535~38079381,248419159~15707257,264241152~16466214,281234113~14360231,298036890~6960630,306903613~21205758,331350016~15772466,355806290~1463251,366341667~7420413,374580519~17465318,394880591~2187459,398418332~12623422] 2024-04-07T17:37:31.818 INFO:tasks.workunit.client.0.smithi100.stdout:from 6 to HEAD diff [2329752~23418695,27329013~980763,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~12098856,149233764~4722012,154895619~3096498,168236274~8367316,182747004~9590628,202420372~1435645,210147535~38079381,250447165~8059589,267179572~11729329,281234113~14360231,298036890~6960630,306903613~5021411,314393585~13715786,332944411~6919544,345811290~1311192,355806290~1463251,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:31.818 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~23418695,27329013~980763,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~12098856,149233764~4722012,154895619~3096498,168236274~8367316,182747004~9590628,202420372~1435645,210147535~38079381,250447165~8059589,267179572~11729329,281234113~14360231,298036890~6960630,306903613~5021411,314393585~13715786,332944411~6919544,345811290~1311192,355806290~1463251,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:31.823 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~25748447,27329013~980763,30516268~23825099,55380210~10623546,66861224~16545045,92274688~11289903,106500403~5285199,112275487~21702107,134217728~14151836,149233764~4722012,154895619~3096498,167772160~8831430,182747004~9590628,202420372~1435645,210147535~38079381,250447165~8059589,264241152~14667749,281234113~14360231,298036890~6960630,306903613~21205758,331350016~8513939,344034014~3088468,355806290~1463251,366341667~7420413,374580519~17465318,394880591~2187459,398418332~12623422] 2024-04-07T17:37:31.823 INFO:tasks.workunit.client.0.smithi100.stdout:from 7 to HEAD diff [2329752~23418695,27329013~980763,31769017~13613313,45857929~8483438,55380210~10623546,68206575~15199694,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~5240179,141885014~6484550,149233764~4722012,154895619~3096498,168236274~8367316,182747004~9590628,210147535~38079381,250447165~8059589,267179572~11729329,281234113~14360231,298036890~6960630,306903613~5021411,314393585~13715786,332944411~6919544,355806290~1463251,366341667~7420413,377322524~14723313,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:31.824 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~23418695,27329013~980763,31769017~13613313,45857929~8483438,55380210~10623546,68206575~15199694,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~5240179,141885014~6484550,149233764~4722012,154895619~3096498,168236274~8367316,182747004~9590628,210147535~38079381,250447165~8059589,267179572~11729329,281234113~14360231,298036890~6960630,306903613~5021411,314393585~13715786,332944411~6919544,355806290~1463251,366341667~7420413,377322524~14723313,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:31.831 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~25748447,27329013~980763,30516268~23825099,55380210~10623546,68206575~15199694,92274688~11289903,106500403~5285199,112275487~21702107,134217728~7293159,141885014~6484550,149233764~4722012,154895619~3096498,167772160~8831430,182747004~9590628,210147535~38079381,250447165~8059589,264241152~14667749,281234113~14360231,298036890~6960630,306903613~21205758,331350016~8513939,355806290~1463251,366341667~7420413,377322524~14723313,394880591~2187459,398418332~12623422] 2024-04-07T17:37:31.831 INFO:tasks.workunit.client.0.smithi100.stdout:from 8 to HEAD diff [2329752~23418695,27329013~980763,31769017~13613313,45857929~8483438,55380210~10623546,68206575~15199694,95470042~8094549,106500403~5285199,112275487~7841113,121501610~10322918,136270708~5240179,141885014~6484550,149552729~4403047,168236274~8367316,182747004~9590628,210147535~38079381,250447165~8059589,267179572~11729329,281234113~14360231,298036890~6918207,306903613~5021411,314393585~13715786,332944411~6919544,355806290~1463251,366341667~7420413,377322524~14723313,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:31.831 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~23418695,27329013~980763,31769017~13613313,45857929~8483438,55380210~10623546,68206575~15199694,95470042~8094549,106500403~5285199,112275487~7841113,121501610~10322918,136270708~5240179,141885014~6484550,149552729~4403047,168236274~8367316,182747004~9590628,210147535~38079381,250447165~8059589,267179572~11729329,281234113~14360231,298036890~6918207,306903613~5021411,314393585~13715786,332944411~6919544,355806290~1463251,366341667~7420413,377322524~14723313,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:31.837 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~25748447,27329013~980763,30516268~23825099,55380210~10623546,68206575~15199694,95470042~8094549,106500403~5285199,112275487~19549041,134217728~7293159,141885014~6484550,149552729~4403047,167772160~8831430,182747004~9590628,210147535~38079381,250447165~8059589,264241152~14667749,281234113~14360231,298036890~6918207,306903613~21205758,331350016~8513939,355806290~1463251,366341667~7420413,377322524~14723313,394880591~2187459,398418332~12623422] 2024-04-07T17:37:31.838 INFO:tasks.workunit.client.0.smithi100.stdout:from 9 to HEAD diff [2329752~21800979,27329013~980763,37362822~8019508,45857929~8483438,55380210~10623546,68206575~15199694,95781519~7783072,106500403~5285199,112275487~7841113,121501610~10322918,136270708~5240179,141885014~6484550,149552729~4403047,168236274~8367316,183274874~3283813,186734432~153638,187506017~4831615,210147535~38079381,254075762~4430992,267179572~11729329,282485483~13108861,298036890~6918207,306903613~5021411,314393585~13715786,332944411~6919544,355806290~1463251,366341667~7420413,377322524~14723313,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:31.838 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~21800979,27329013~980763,37362822~8019508,45857929~8483438,55380210~10623546,68206575~15199694,95781519~7783072,106500403~5285199,112275487~7841113,121501610~10322918,136270708~5240179,141885014~6484550,149552729~4403047,168236274~8367316,183274874~3283813,186734432~153638,187506017~4831615,210147535~38079381,254075762~4430992,267179572~11729329,282485483~13108861,298036890~6918207,306903613~5021411,314393585~13715786,332944411~6919544,355806290~1463251,366341667~7420413,377322524~14723313,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:31.844 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~24130731,27329013~980763,37362822~16978545,55380210~10623546,68206575~15199694,95781519~7783072,106500403~5285199,112275487~19549041,134217728~7293159,141885014~6484550,149552729~4403047,167772160~8831430,183274874~3283813,186734432~153638,187506017~4831615,210147535~38079381,254075762~4430992,264241152~14667749,282485483~13108861,298036890~6918207,306903613~21205758,331350016~8513939,355806290~1463251,366341667~7420413,377322524~14723313,394880591~2187459,398418332~12623422] 2024-04-07T17:37:31.845 INFO:tasks.workunit.client.0.smithi100.stdout:from 10 to HEAD diff [2329752~21800979,27329013~249441,37362822~8019508,45857929~8483438,55380210~10623546,68206575~15199694,95781519~7783072,112275487~7841113,124240937~7583591,136270708~5240179,141885014~6484550,149552729~4403047,168236274~8367316,183274874~3283813,187506017~4831615,210147535~38079381,254075762~4430992,267179572~11729329,282485483~13108861,298036890~6918207,306903613~5021411,314393585~13715786,332944411~6919544,355806290~1463251,377322524~14723313,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:31.845 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~21800979,27329013~249441,37362822~8019508,45857929~8483438,55380210~10623546,68206575~15199694,95781519~7783072,112275487~7841113,124240937~7583591,136270708~5240179,141885014~6484550,149552729~4403047,168236274~8367316,183274874~3283813,187506017~4831615,210147535~38079381,254075762~4430992,267179572~11729329,282485483~13108861,298036890~6918207,306903613~5021411,314393585~13715786,332944411~6919544,355806290~1463251,377322524~14723313,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:31.851 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [2329752~21800979,27329013~249441,37362822~16978545,55380210~10623546,68206575~15199694,95781519~7783072,112275487~7841113,124240937~7583591,134217728~7293159,141885014~6484550,149552729~4403047,167772160~8831430,183274874~3283813,187506017~4831615,210147535~38079381,254075762~4430992,264241152~14667749,282485483~13108861,298036890~6918207,306903613~21205758,331350016~8513939,355806290~1463251,377322524~14723313,394880591~2187459,398418332~12623422] 2024-04-07T17:37:31.852 INFO:tasks.workunit.client.0.smithi100.stdout:from 11 to HEAD diff [2329752~21800979,27329013~249441,37386125~7996205,45857929~8483438,55380210~10623546,68206575~15199694,95781519~7783072,112275487~7841113,124240937~7583591,136270708~5240179,141885014~6484550,149552729~4403047,168236274~8367316,183274874~3283813,187506017~4831615,210147535~19406580,229742904~18484012,254075762~4430992,267179572~6811128,282485483~13108861,298036890~6918207,306903613~5021411,314393585~13715786,332944411~6919544,355806290~1463251,380094867~8701052,394880591~2187459,400878444~6009875,408516200~2525554] 2024-04-07T17:37:31.852 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~21800979,27329013~249441,37386125~7996205,45857929~8483438,55380210~10623546,68206575~15199694,95781519~7783072,112275487~7841113,124240937~7583591,136270708~5240179,141885014~6484550,149552729~4403047,168236274~8367316,183274874~3283813,187506017~4831615,210147535~19406580,229742904~18484012,254075762~4430992,267179572~6811128,282485483~13108861,298036890~6918207,306903613~5021411,314393585~13715786,332944411~6919544,355806290~1463251,380094867~8701052,394880591~2187459,400878444~6009875,408516200~2525554] 2024-04-07T17:37:31.859 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [2329752~21800979,27329013~249441,37386125~16955242,55380210~10623546,68206575~15199694,95781519~7783072,112275487~7841113,124240937~7583591,134217728~7293159,141885014~6484550,149552729~4403047,167772160~8831430,183274874~3283813,187506017~4831615,210147535~19406580,229742904~18484012,254075762~4430992,264241152~9749548,282485483~13108861,298036890~6918207,306903613~21205758,331350016~8513939,355806290~1463251,380094867~8701052,394880591~2187459,400878444~10163310] 2024-04-07T17:37:31.859 INFO:tasks.workunit.client.0.smithi100.stdout:from 12 to HEAD diff [2329752~21800979,27329013~249441,39912210~4133268,48881748~4683249,55380210~10623546,68206575~15199694,95781519~3680106,124240937~7583591,136270708~5240179,141885014~6484550,149552729~4403047,168236274~8367316,183274874~3283813,187506017~4831615,210147535~19406580,229742904~18484012,254075762~4430992,267179572~6811128,282485483~11941900,298643235~5745703,306903613~5021411,314393585~12544817,332944411~6919544,355806290~1463251,380563114~8232805,394880591~2187459,401962264~4926055,408516200~2525554] 2024-04-07T17:37:31.859 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~21800979,27329013~249441,39912210~4133268,48881748~4683249,55380210~10623546,68206575~15199694,95781519~3680106,124240937~7583591,136270708~5240179,141885014~6484550,149552729~4403047,168236274~8367316,183274874~3283813,187506017~4831615,210147535~19406580,229742904~18484012,254075762~4430992,267179572~6811128,282485483~11941900,298643235~5745703,306903613~5021411,314393585~12544817,332944411~6919544,355806290~1463251,380563114~8232805,394880591~2187459,401962264~4926055,408516200~2525554] 2024-04-07T17:37:31.868 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [2329752~21800979,27329013~249441,39912210~4133268,48881748~4683249,55380210~10623546,68206575~15199694,95781519~3680106,124240937~7583591,134217728~7293159,141885014~6484550,149552729~4403047,167772160~8831430,183274874~3283813,187506017~4831615,210147535~19406580,229742904~18484012,254075762~4430992,264241152~9749548,282485483~11941900,298643235~5745703,306903613~20034789,331350016~8513939,355806290~1463251,380563114~8232805,394880591~2187459,401962264~9079490] 2024-04-07T17:37:31.868 INFO:tasks.workunit.client.0.smithi100.stdout:from 13 to HEAD diff [2329752~21327233,39912210~4133268,48881748~4683249,55380210~10623546,68206575~15199694,95781519~3680106,124240937~7583591,136996101~4514786,141885014~6484550,149552729~4403047,168236274~8367316,183274874~3283813,187506017~4831615,210147535~19406580,229742904~18484012,254075762~4430992,282485483~11941900,298643235~5745703,306903613~5021411,314393585~12544817,332944411~6919544,355806290~1463251,380563114~8232805,401962264~4926055] 2024-04-07T17:37:31.868 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~21327233,39912210~4133268,48881748~4683249,55380210~10623546,68206575~15199694,95781519~3680106,124240937~7583591,136996101~4514786,141885014~6484550,149552729~4403047,168236274~8367316,183274874~3283813,187506017~4831615,210147535~19406580,229742904~18484012,254075762~4430992,282485483~11941900,298643235~5745703,306903613~5021411,314393585~12544817,332944411~6919544,355806290~1463251,380563114~8232805,401962264~4926055] 2024-04-07T17:37:31.877 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [2329752~21327233,39912210~4133268,48881748~4683249,55380210~10623546,68206575~15199694,95781519~3680106,124240937~7583591,136996101~4514786,141885014~6484550,149552729~4403047,167772160~8831430,183274874~3283813,187506017~4831615,210147535~19406580,229742904~18484012,254075762~4430992,282485483~11941900,298643235~5745703,306903613~20034789,331350016~8513939,355806290~1463251,380563114~8232805,401962264~4926055] 2024-04-07T17:37:31.877 INFO:tasks.workunit.client.0.smithi100.stdout:from 14 to HEAD diff [3107878~17063292,39912210~4133268,48881748~4683249,55380210~10623546,68206575~15199694,95781519~3680106,126602360~5222168,136996101~4514786,141885014~6484550,149552729~4403047,168236274~8367316,183274874~3283813,210147535~5174157,219514234~10039881,229742904~18484012,254075762~4430992,282485483~7514691,298643235~5745703,306903613~5021411,314393585~7979426,323018422~3348552,332944411~6919544,355806290~1463251,380563114~8232805,401962264~4926055] 2024-04-07T17:37:31.877 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [3107878~17063292,39912210~4133268,48881748~4683249,55380210~10623546,68206575~15199694,95781519~3680106,126602360~5222168,136996101~4514786,141885014~6484550,149552729~4403047,168236274~8367316,183274874~3283813,210147535~5174157,219514234~10039881,229742904~18484012,254075762~4430992,282485483~7514691,298643235~5745703,306903613~5021411,314393585~7979426,323018422~3348552,332944411~6919544,355806290~1463251,380563114~8232805,401962264~4926055] 2024-04-07T17:37:31.887 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [3107878~17063292,39912210~4133268,48881748~4683249,55380210~10623546,68206575~15199694,95781519~3680106,126602360~5222168,136996101~4514786,141885014~6484550,149552729~4403047,167772160~8831430,183274874~3283813,210147535~5174157,219514234~10039881,229742904~18484012,254075762~4430992,282485483~7514691,298643235~5745703,306903613~15469398,323018422~3348552,331350016~8513939,355806290~1463251,380563114~8232805,401962264~4926055] 2024-04-07T17:37:31.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:37:31 smithi100 bash[74942]: cluster 2024-04-07T17:37:30.481410+0000 mgr.y (mgr.25175) 2464 : cluster [DBG] pgmap v4056: 137 pgs: 3 peering, 134 active+clean; 888 MiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 157 KiB/s rd, 6.5 MiB/s wr, 4.34k op/s; 12 MiB/s, 8 objects/s recovering 2024-04-07T17:37:31.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:37:31 smithi100 bash[74942]: cluster 2024-04-07T17:37:30.673106+0000 mon.a (mon.0) 3899 : cluster [DBG] osdmap e2018: 8 total, 8 up, 8 in 2024-04-07T17:37:31.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:37:31 smithi100 bash[76202]: cluster 2024-04-07T17:37:30.481410+0000 mgr.y (mgr.25175) 2464 : cluster [DBG] pgmap v4056: 137 pgs: 3 peering, 134 active+clean; 888 MiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 157 KiB/s rd, 6.5 MiB/s wr, 4.34k op/s; 12 MiB/s, 8 objects/s recovering 2024-04-07T17:37:31.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:37:31 smithi100 bash[76202]: cluster 2024-04-07T17:37:30.673106+0000 mon.a (mon.0) 3899 : cluster [DBG] osdmap e2018: 8 total, 8 up, 8 in 2024-04-07T17:37:32.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:37:31 smithi144 bash[66654]: cluster 2024-04-07T17:37:30.481410+0000 mgr.y (mgr.25175) 2464 : cluster [DBG] pgmap v4056: 137 pgs: 3 peering, 134 active+clean; 888 MiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 157 KiB/s rd, 6.5 MiB/s wr, 4.34k op/s; 12 MiB/s, 8 objects/s recovering 2024-04-07T17:37:32.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:37:31 smithi144 bash[66654]: cluster 2024-04-07T17:37:30.673106+0000 mon.a (mon.0) 3899 : cluster [DBG] osdmap e2018: 8 total, 8 up, 8 in 2024-04-07T17:37:32.615 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:37:32 smithi144 bash[95233]: ts=2024-04-07T17:37:32.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:37:32.719 INFO:tasks.workunit.client.0.smithi100.stdout:from 0 to HEAD diff [2329752~26214847,29778841~737427,31769017~13613313,45857929~20145827,66861224~16545045,85466334~5069656,92999623~10564968,106500403~13616197,121501610~12475984,136270708~22564048,160337176~7153146,168236274~8367316,180237750~12590714,194102759~7364680,202420372~5254601,210147535~53978881,267179572~28414772,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350406459~6863082,360049192~84791,366341667~7420413,374580519~17465318,392232408~14655911,408516200~2525554] 2024-04-07T17:37:32.719 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~26214847,29778841~737427,31769017~13613313,45857929~20145827,66861224~16545045,85466334~5069656,92999623~10564968,106500403~13616197,121501610~12475984,136270708~22564048,160337176~7153146,168236274~8367316,180237750~12590714,194102759~7364680,202420372~5254601,210147535~53978881,267179572~28414772,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350406459~6863082,360049192~84791,366341667~7420413,374580519~17465318,392232408~14655911,408516200~2525554] 2024-04-07T17:37:32.775 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~28544599,29360128~54046141,83886080~6649910,92274688~11289903,104857600~29119994,134217728~24617028,159383552~8106770,167772160~25056304,192937984~14736989,210147535~53978881,264241152~31353192,297795584~7201936,306184192~23819574,331350016~15772466,348127232~12006751,366341667~44700087] 2024-04-07T17:37:32.775 INFO:tasks.workunit.client.0.smithi100.stdout:from 1 to HEAD diff [2329752~26214847,29778841~737427,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~13616197,121501610~12475984,136270708~22564048,160337176~7153146,168236274~8367316,180237750~12590714,194866296~6601143,202420372~5254601,210147535~53978881,267179572~28414772,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350891716~1451525,355806290~1463251,360049192~84791,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:32.775 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~26214847,29778841~737427,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~13616197,121501610~12475984,136270708~22564048,160337176~7153146,168236274~8367316,180237750~12590714,194866296~6601143,202420372~5254601,210147535~53978881,267179572~28414772,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350891716~1451525,355806290~1463251,360049192~84791,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:32.783 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~28544599,29360128~24981239,55380210~10623546,66861224~16545045,92274688~11289903,104857600~29119994,134217728~24617028,159383552~8106770,167772160~25056304,194866296~12808677,210147535~53978881,264241152~31353192,297795584~7201936,306184192~23819574,331350016~15772466,350891716~1451525,355806290~4327693,366341667~25704170,394880591~2187459,398418332~12623422] 2024-04-07T17:37:32.783 INFO:tasks.workunit.client.0.smithi100.stdout:from 2 to HEAD diff [2329752~24230697,27329013~980763,29778841~737427,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~17685068,154799337~4035419,160337176~7153146,168236274~8367316,182034135~10794329,196727604~2108791,202420372~5254601,210147535~53978881,267179572~28414772,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350891716~1451525,355806290~1463251,360049192~84791,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:32.783 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~24230697,27329013~980763,29778841~737427,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~17685068,154799337~4035419,160337176~7153146,168236274~8367316,182034135~10794329,196727604~2108791,202420372~5254601,210147535~53978881,267179572~28414772,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350891716~1451525,355806290~1463251,360049192~84791,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:32.790 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~26560449,27329013~980763,29360128~24981239,55380210~10623546,66861224~16545045,92274688~11289903,106500403~5285199,112275487~21702107,134217728~19738048,154799337~4035419,159383552~8106770,167772160~8831430,182034135~10794329,196727604~2108791,201467439~6207534,210147535~53978881,264241152~31353192,297795584~7201936,306184192~23819574,331350016~15772466,350891716~1451525,355806290~4327693,366341667~25704170,394880591~2187459,398418332~12623422] 2024-04-07T17:37:32.790 INFO:tasks.workunit.client.0.smithi100.stdout:from 3 to HEAD diff [2329752~24230697,27329013~980763,29778841~737427,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~17685068,154799337~3192780,160337176~7153146,168236274~8367316,182034135~10303497,196727604~2108791,202420372~1435645,210147535~53978881,267179572~28414772,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350891716~1451525,355806290~1463251,360049192~84791,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:32.790 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~24230697,27329013~980763,29778841~737427,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~17685068,154799337~3192780,160337176~7153146,168236274~8367316,182034135~10303497,196727604~2108791,202420372~1435645,210147535~53978881,267179572~28414772,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350891716~1451525,355806290~1463251,360049192~84791,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:32.797 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~26560449,27329013~980763,29360128~24981239,55380210~10623546,66861224~16545045,92274688~11289903,106500403~5285199,112275487~21702107,134217728~19738048,154799337~3192780,159383552~8106770,167772160~8831430,182034135~10303497,196727604~2108791,202420372~1435645,210147535~53978881,264241152~31353192,297795584~7201936,306184192~23819574,331350016~15772466,350891716~1451525,355806290~4327693,366341667~25704170,394880591~2187459,398418332~12623422] 2024-04-07T17:37:32.797 INFO:tasks.workunit.client.0.smithi100.stdout:from 4 to HEAD diff [2329752~24230697,27329013~980763,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~12098856,149233764~4722012,154895619~3096498,160337176~7153146,168236274~8367316,182034135~10303497,196727604~2108791,202420372~1435645,210147535~53978881,267179572~13527794,281234113~14360231,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350891716~1451525,355806290~1463251,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:32.797 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~24230697,27329013~980763,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~12098856,149233764~4722012,154895619~3096498,160337176~7153146,168236274~8367316,182034135~10303497,196727604~2108791,202420372~1435645,210147535~53978881,267179572~13527794,281234113~14360231,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350891716~1451525,355806290~1463251,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:32.804 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~26560449,27329013~980763,30516268~23825099,55380210~10623546,66861224~16545045,92274688~11289903,106500403~5285199,112275487~21702107,134217728~14151836,149233764~4722012,154895619~3096498,159383552~8106770,167772160~8831430,182034135~10303497,196727604~2108791,202420372~1435645,210147535~53978881,264241152~16466214,281234113~14360231,297795584~7201936,306903613~23100153,331350016~15772466,350891716~1451525,355806290~1463251,366341667~25704170,394880591~2187459,398418332~12623422] 2024-04-07T17:37:32.804 INFO:tasks.workunit.client.0.smithi100.stdout:from 5 to HEAD diff [2329752~24230697,27329013~980763,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~12098856,149233764~4722012,154895619~3096498,166133159~1357163,168236274~8367316,182034135~10303497,196727604~2108791,202420372~1435645,210147535~38079381,248419159~15707257,267179572~13527794,281234113~14360231,298036890~6960630,306903613~5021411,314393585~13715786,332944411~11089603,345811290~1311192,355806290~1463251,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:32.804 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~24230697,27329013~980763,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~12098856,149233764~4722012,154895619~3096498,166133159~1357163,168236274~8367316,182034135~10303497,196727604~2108791,202420372~1435645,210147535~38079381,248419159~15707257,267179572~13527794,281234113~14360231,298036890~6960630,306903613~5021411,314393585~13715786,332944411~11089603,345811290~1311192,355806290~1463251,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:32.811 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~26560449,27329013~980763,30516268~23825099,55380210~10623546,66861224~16545045,92274688~11289903,106500403~5285199,112275487~21702107,134217728~14151836,149233764~4722012,154895619~3096498,166133159~1357163,167772160~8831430,182034135~10303497,196727604~2108791,202420372~1435645,210147535~38079381,248419159~15707257,264241152~16466214,281234113~14360231,298036890~6960630,306903613~21205758,331350016~15772466,355806290~1463251,366341667~7420413,374580519~17465318,394880591~2187459,398418332~12623422] 2024-04-07T17:37:32.811 INFO:tasks.workunit.client.0.smithi100.stdout:from 6 to HEAD diff [2329752~23418695,27329013~980763,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~12098856,149233764~4722012,154895619~3096498,168236274~8367316,182747004~9590628,202420372~1435645,210147535~38079381,250447165~8059589,267179572~11729329,281234113~14360231,298036890~6960630,306903613~5021411,314393585~13715786,332944411~6919544,345811290~1311192,355806290~1463251,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:32.811 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~23418695,27329013~980763,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~12098856,149233764~4722012,154895619~3096498,168236274~8367316,182747004~9590628,202420372~1435645,210147535~38079381,250447165~8059589,267179572~11729329,281234113~14360231,298036890~6960630,306903613~5021411,314393585~13715786,332944411~6919544,345811290~1311192,355806290~1463251,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:32.818 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~25748447,27329013~980763,30516268~23825099,55380210~10623546,66861224~16545045,92274688~11289903,106500403~5285199,112275487~21702107,134217728~14151836,149233764~4722012,154895619~3096498,167772160~8831430,182747004~9590628,202420372~1435645,210147535~38079381,250447165~8059589,264241152~14667749,281234113~14360231,298036890~6960630,306903613~21205758,331350016~8513939,344034014~3088468,355806290~1463251,366341667~7420413,374580519~17465318,394880591~2187459,398418332~12623422] 2024-04-07T17:37:32.818 INFO:tasks.workunit.client.0.smithi100.stdout:from 7 to HEAD diff [2329752~23418695,27329013~980763,31769017~13613313,45857929~8483438,55380210~10623546,68206575~15199694,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~5240179,141885014~6484550,149233764~4722012,154895619~3096498,168236274~8367316,182747004~9590628,210147535~38079381,250447165~8059589,267179572~11729329,281234113~14360231,298036890~6960630,306903613~5021411,314393585~13715786,332944411~6919544,355806290~1463251,366341667~7420413,377322524~14723313,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:32.818 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~23418695,27329013~980763,31769017~13613313,45857929~8483438,55380210~10623546,68206575~15199694,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~5240179,141885014~6484550,149233764~4722012,154895619~3096498,168236274~8367316,182747004~9590628,210147535~38079381,250447165~8059589,267179572~11729329,281234113~14360231,298036890~6960630,306903613~5021411,314393585~13715786,332944411~6919544,355806290~1463251,366341667~7420413,377322524~14723313,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:32.824 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~25748447,27329013~980763,30516268~23825099,55380210~10623546,68206575~15199694,92274688~11289903,106500403~5285199,112275487~21702107,134217728~7293159,141885014~6484550,149233764~4722012,154895619~3096498,167772160~8831430,182747004~9590628,210147535~38079381,250447165~8059589,264241152~14667749,281234113~14360231,298036890~6960630,306903613~21205758,331350016~8513939,355806290~1463251,366341667~7420413,377322524~14723313,394880591~2187459,398418332~12623422] 2024-04-07T17:37:32.824 INFO:tasks.workunit.client.0.smithi100.stdout:from 8 to HEAD diff [2329752~23418695,27329013~980763,31769017~13613313,45857929~8483438,55380210~10623546,68206575~15199694,95470042~8094549,106500403~5285199,112275487~7841113,121501610~10322918,136270708~5240179,141885014~6484550,149552729~4403047,168236274~8367316,182747004~9590628,210147535~38079381,250447165~8059589,267179572~11729329,281234113~14360231,298036890~6918207,306903613~5021411,314393585~13715786,332944411~6919544,355806290~1463251,366341667~7420413,377322524~14723313,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:32.824 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~23418695,27329013~980763,31769017~13613313,45857929~8483438,55380210~10623546,68206575~15199694,95470042~8094549,106500403~5285199,112275487~7841113,121501610~10322918,136270708~5240179,141885014~6484550,149552729~4403047,168236274~8367316,182747004~9590628,210147535~38079381,250447165~8059589,267179572~11729329,281234113~14360231,298036890~6918207,306903613~5021411,314393585~13715786,332944411~6919544,355806290~1463251,366341667~7420413,377322524~14723313,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:32.831 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~25748447,27329013~980763,30516268~23825099,55380210~10623546,68206575~15199694,95470042~8094549,106500403~5285199,112275487~19549041,134217728~7293159,141885014~6484550,149552729~4403047,167772160~8831430,182747004~9590628,210147535~38079381,250447165~8059589,264241152~14667749,281234113~14360231,298036890~6918207,306903613~21205758,331350016~8513939,355806290~1463251,366341667~7420413,377322524~14723313,394880591~2187459,398418332~12623422] 2024-04-07T17:37:32.831 INFO:tasks.workunit.client.0.smithi100.stdout:from 9 to HEAD diff [2329752~21800979,27329013~980763,37362822~8019508,45857929~8483438,55380210~10623546,68206575~15199694,95781519~7783072,106500403~5285199,112275487~7841113,121501610~10322918,136270708~5240179,141885014~6484550,149552729~4403047,168236274~8367316,183274874~3283813,186734432~153638,187506017~4831615,210147535~38079381,254075762~4430992,267179572~11729329,282485483~13108861,298036890~6918207,306903613~5021411,314393585~13715786,332944411~6919544,355806290~1463251,366341667~7420413,377322524~14723313,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:32.831 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~21800979,27329013~980763,37362822~8019508,45857929~8483438,55380210~10623546,68206575~15199694,95781519~7783072,106500403~5285199,112275487~7841113,121501610~10322918,136270708~5240179,141885014~6484550,149552729~4403047,168236274~8367316,183274874~3283813,186734432~153638,187506017~4831615,210147535~38079381,254075762~4430992,267179572~11729329,282485483~13108861,298036890~6918207,306903613~5021411,314393585~13715786,332944411~6919544,355806290~1463251,366341667~7420413,377322524~14723313,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:32.838 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~24130731,27329013~980763,37362822~16978545,55380210~10623546,68206575~15199694,95781519~7783072,106500403~5285199,112275487~19549041,134217728~7293159,141885014~6484550,149552729~4403047,167772160~8831430,183274874~3283813,186734432~153638,187506017~4831615,210147535~38079381,254075762~4430992,264241152~14667749,282485483~13108861,298036890~6918207,306903613~21205758,331350016~8513939,355806290~1463251,366341667~7420413,377322524~14723313,394880591~2187459,398418332~12623422] 2024-04-07T17:37:32.838 INFO:tasks.workunit.client.0.smithi100.stdout:from 10 to HEAD diff [2329752~21800979,27329013~249441,37362822~8019508,45857929~8483438,55380210~10623546,68206575~15199694,95781519~7783072,112275487~7841113,124240937~7583591,136270708~5240179,141885014~6484550,149552729~4403047,168236274~8367316,183274874~3283813,187506017~4831615,210147535~38079381,254075762~4430992,267179572~11729329,282485483~13108861,298036890~6918207,306903613~5021411,314393585~13715786,332944411~6919544,355806290~1463251,377322524~14723313,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:32.838 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~21800979,27329013~249441,37362822~8019508,45857929~8483438,55380210~10623546,68206575~15199694,95781519~7783072,112275487~7841113,124240937~7583591,136270708~5240179,141885014~6484550,149552729~4403047,168236274~8367316,183274874~3283813,187506017~4831615,210147535~38079381,254075762~4430992,267179572~11729329,282485483~13108861,298036890~6918207,306903613~5021411,314393585~13715786,332944411~6919544,355806290~1463251,377322524~14723313,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:32.846 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [2329752~21800979,27329013~249441,37362822~16978545,55380210~10623546,68206575~15199694,95781519~7783072,112275487~7841113,124240937~7583591,134217728~7293159,141885014~6484550,149552729~4403047,167772160~8831430,183274874~3283813,187506017~4831615,210147535~38079381,254075762~4430992,264241152~14667749,282485483~13108861,298036890~6918207,306903613~21205758,331350016~8513939,355806290~1463251,377322524~14723313,394880591~2187459,398418332~12623422] 2024-04-07T17:37:32.846 INFO:tasks.workunit.client.0.smithi100.stdout:from 11 to HEAD diff [2329752~21800979,27329013~249441,37386125~7996205,45857929~8483438,55380210~10623546,68206575~15199694,95781519~7783072,112275487~7841113,124240937~7583591,136270708~5240179,141885014~6484550,149552729~4403047,168236274~8367316,183274874~3283813,187506017~4831615,210147535~19406580,229742904~18484012,254075762~4430992,267179572~6811128,282485483~13108861,298036890~6918207,306903613~5021411,314393585~13715786,332944411~6919544,355806290~1463251,380094867~8701052,394880591~2187459,400878444~6009875,408516200~2525554] 2024-04-07T17:37:32.847 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~21800979,27329013~249441,37386125~7996205,45857929~8483438,55380210~10623546,68206575~15199694,95781519~7783072,112275487~7841113,124240937~7583591,136270708~5240179,141885014~6484550,149552729~4403047,168236274~8367316,183274874~3283813,187506017~4831615,210147535~19406580,229742904~18484012,254075762~4430992,267179572~6811128,282485483~13108861,298036890~6918207,306903613~5021411,314393585~13715786,332944411~6919544,355806290~1463251,380094867~8701052,394880591~2187459,400878444~6009875,408516200~2525554] 2024-04-07T17:37:32.853 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [2329752~21800979,27329013~249441,37386125~16955242,55380210~10623546,68206575~15199694,95781519~7783072,112275487~7841113,124240937~7583591,134217728~7293159,141885014~6484550,149552729~4403047,167772160~8831430,183274874~3283813,187506017~4831615,210147535~19406580,229742904~18484012,254075762~4430992,264241152~9749548,282485483~13108861,298036890~6918207,306903613~21205758,331350016~8513939,355806290~1463251,380094867~8701052,394880591~2187459,400878444~10163310] 2024-04-07T17:37:32.853 INFO:tasks.workunit.client.0.smithi100.stdout:from 12 to HEAD diff [2329752~21800979,27329013~249441,39912210~4133268,48881748~4683249,55380210~10623546,68206575~15199694,95781519~3680106,124240937~7583591,136270708~5240179,141885014~6484550,149552729~4403047,168236274~8367316,183274874~3283813,187506017~4831615,210147535~19406580,229742904~18484012,254075762~4430992,267179572~6811128,282485483~11941900,298643235~5745703,306903613~5021411,314393585~12544817,332944411~6919544,355806290~1463251,380563114~8232805,394880591~2187459,401962264~4926055,408516200~2525554] 2024-04-07T17:37:32.853 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~21800979,27329013~249441,39912210~4133268,48881748~4683249,55380210~10623546,68206575~15199694,95781519~3680106,124240937~7583591,136270708~5240179,141885014~6484550,149552729~4403047,168236274~8367316,183274874~3283813,187506017~4831615,210147535~19406580,229742904~18484012,254075762~4430992,267179572~6811128,282485483~11941900,298643235~5745703,306903613~5021411,314393585~12544817,332944411~6919544,355806290~1463251,380563114~8232805,394880591~2187459,401962264~4926055,408516200~2525554] 2024-04-07T17:37:32.860 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [2329752~21800979,27329013~249441,39912210~4133268,48881748~4683249,55380210~10623546,68206575~15199694,95781519~3680106,124240937~7583591,134217728~7293159,141885014~6484550,149552729~4403047,167772160~8831430,183274874~3283813,187506017~4831615,210147535~19406580,229742904~18484012,254075762~4430992,264241152~9749548,282485483~11941900,298643235~5745703,306903613~20034789,331350016~8513939,355806290~1463251,380563114~8232805,394880591~2187459,401962264~9079490] 2024-04-07T17:37:32.860 INFO:tasks.workunit.client.0.smithi100.stdout:from 13 to HEAD diff [2329752~21327233,39912210~4133268,48881748~4683249,55380210~10623546,68206575~15199694,95781519~3680106,124240937~7583591,136996101~4514786,141885014~6484550,149552729~4403047,168236274~8367316,183274874~3283813,187506017~4831615,210147535~19406580,229742904~18484012,254075762~4430992,282485483~11941900,298643235~5745703,306903613~5021411,314393585~12544817,332944411~6919544,355806290~1463251,380563114~8232805,401962264~4926055] 2024-04-07T17:37:32.860 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~21327233,39912210~4133268,48881748~4683249,55380210~10623546,68206575~15199694,95781519~3680106,124240937~7583591,136996101~4514786,141885014~6484550,149552729~4403047,168236274~8367316,183274874~3283813,187506017~4831615,210147535~19406580,229742904~18484012,254075762~4430992,282485483~11941900,298643235~5745703,306903613~5021411,314393585~12544817,332944411~6919544,355806290~1463251,380563114~8232805,401962264~4926055] 2024-04-07T17:37:32.867 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [2329752~21327233,39912210~4133268,48881748~4683249,55380210~10623546,68206575~15199694,95781519~3680106,124240937~7583591,136996101~4514786,141885014~6484550,149552729~4403047,167772160~8831430,183274874~3283813,187506017~4831615,210147535~19406580,229742904~18484012,254075762~4430992,282485483~11941900,298643235~5745703,306903613~20034789,331350016~8513939,355806290~1463251,380563114~8232805,401962264~4926055] 2024-04-07T17:37:32.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:37:32 smithi100 bash[74942]: cluster 2024-04-07T17:37:31.673769+0000 mon.a (mon.0) 3900 : cluster [DBG] osdmap e2019: 8 total, 8 up, 8 in 2024-04-07T17:37:32.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:37:32 smithi100 bash[74942]: audit 2024-04-07T17:37:31.857278+0000 mon.a (mon.0) 3901 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:37:32.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:37:32 smithi100 bash[76202]: cluster 2024-04-07T17:37:31.673769+0000 mon.a (mon.0) 3900 : cluster [DBG] osdmap e2019: 8 total, 8 up, 8 in 2024-04-07T17:37:32.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:37:32 smithi100 bash[76202]: audit 2024-04-07T17:37:31.857278+0000 mon.a (mon.0) 3901 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:37:33.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:37:32 smithi144 bash[66654]: cluster 2024-04-07T17:37:31.673769+0000 mon.a (mon.0) 3900 : cluster [DBG] osdmap e2019: 8 total, 8 up, 8 in 2024-04-07T17:37:33.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:37:32 smithi144 bash[66654]: audit 2024-04-07T17:37:31.857278+0000 mon.a (mon.0) 3901 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:37:33.240 INFO:tasks.workunit.client.0.smithi100.stdout:from 0 to HEAD diff [2329752~26214847,29778841~737427,31769017~13613313,45857929~20145827,66861224~16545045,85466334~5069656,92999623~10564968,106500403~13616197,121501610~12475984,136270708~22564048,160337176~7153146,168236274~8367316,180237750~12590714,194102759~7364680,202420372~5254601,210147535~53978881,267179572~28414772,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350406459~6863082,360049192~84791,366341667~7420413,374580519~17465318,392232408~14655911,408516200~2525554] 2024-04-07T17:37:33.241 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~26214847,29778841~737427,31769017~13613313,45857929~20145827,66861224~16545045,85466334~5069656,92999623~10564968,106500403~13616197,121501610~12475984,136270708~22564048,160337176~7153146,168236274~8367316,180237750~12590714,194102759~7364680,202420372~5254601,210147535~53978881,267179572~28414772,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350406459~6863082,360049192~84791,366341667~7420413,374580519~17465318,392232408~14655911,408516200~2525554] 2024-04-07T17:37:33.298 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~28544599,29360128~54046141,83886080~6649910,92274688~11289903,104857600~29119994,134217728~24617028,159383552~8106770,167772160~25056304,192937984~14736989,210147535~53978881,264241152~31353192,297795584~7201936,306184192~23819574,331350016~15772466,348127232~12006751,366341667~44700087] 2024-04-07T17:37:33.299 INFO:tasks.workunit.client.0.smithi100.stdout:from 1 to HEAD diff [2329752~26214847,29778841~737427,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~13616197,121501610~12475984,136270708~22564048,160337176~7153146,168236274~8367316,180237750~12590714,194866296~6601143,202420372~5254601,210147535~53978881,267179572~28414772,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350891716~1451525,355806290~1463251,360049192~84791,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:33.299 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~26214847,29778841~737427,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~13616197,121501610~12475984,136270708~22564048,160337176~7153146,168236274~8367316,180237750~12590714,194866296~6601143,202420372~5254601,210147535~53978881,267179572~28414772,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350891716~1451525,355806290~1463251,360049192~84791,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:33.310 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~28544599,29360128~24981239,55380210~10623546,66861224~16545045,92274688~11289903,104857600~29119994,134217728~24617028,159383552~8106770,167772160~25056304,194866296~12808677,210147535~53978881,264241152~31353192,297795584~7201936,306184192~23819574,331350016~15772466,350891716~1451525,355806290~4327693,366341667~25704170,394880591~2187459,398418332~12623422] 2024-04-07T17:37:33.310 INFO:tasks.workunit.client.0.smithi100.stdout:from 2 to HEAD diff [2329752~24230697,27329013~980763,29778841~737427,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~17685068,154799337~4035419,160337176~7153146,168236274~8367316,182034135~10794329,196727604~2108791,202420372~5254601,210147535~53978881,267179572~28414772,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350891716~1451525,355806290~1463251,360049192~84791,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:33.310 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~24230697,27329013~980763,29778841~737427,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~17685068,154799337~4035419,160337176~7153146,168236274~8367316,182034135~10794329,196727604~2108791,202420372~5254601,210147535~53978881,267179572~28414772,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350891716~1451525,355806290~1463251,360049192~84791,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:33.317 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~26560449,27329013~980763,29360128~24981239,55380210~10623546,66861224~16545045,92274688~11289903,106500403~5285199,112275487~21702107,134217728~19738048,154799337~4035419,159383552~8106770,167772160~8831430,182034135~10794329,196727604~2108791,201467439~6207534,210147535~53978881,264241152~31353192,297795584~7201936,306184192~23819574,331350016~15772466,350891716~1451525,355806290~4327693,366341667~25704170,394880591~2187459,398418332~12623422] 2024-04-07T17:37:33.317 INFO:tasks.workunit.client.0.smithi100.stdout:from 3 to HEAD diff [2329752~24230697,27329013~980763,29778841~737427,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~17685068,154799337~3192780,160337176~7153146,168236274~8367316,182034135~10303497,196727604~2108791,202420372~1435645,210147535~53978881,267179572~28414772,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350891716~1451525,355806290~1463251,360049192~84791,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:33.317 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~24230697,27329013~980763,29778841~737427,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~17685068,154799337~3192780,160337176~7153146,168236274~8367316,182034135~10303497,196727604~2108791,202420372~1435645,210147535~53978881,267179572~28414772,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350891716~1451525,355806290~1463251,360049192~84791,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:33.324 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~26560449,27329013~980763,29360128~24981239,55380210~10623546,66861224~16545045,92274688~11289903,106500403~5285199,112275487~21702107,134217728~19738048,154799337~3192780,159383552~8106770,167772160~8831430,182034135~10303497,196727604~2108791,202420372~1435645,210147535~53978881,264241152~31353192,297795584~7201936,306184192~23819574,331350016~15772466,350891716~1451525,355806290~4327693,366341667~25704170,394880591~2187459,398418332~12623422] 2024-04-07T17:37:33.324 INFO:tasks.workunit.client.0.smithi100.stdout:from 4 to HEAD diff [2329752~24230697,27329013~980763,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~12098856,149233764~4722012,154895619~3096498,160337176~7153146,168236274~8367316,182034135~10303497,196727604~2108791,202420372~1435645,210147535~53978881,267179572~13527794,281234113~14360231,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350891716~1451525,355806290~1463251,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:33.324 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~24230697,27329013~980763,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~12098856,149233764~4722012,154895619~3096498,160337176~7153146,168236274~8367316,182034135~10303497,196727604~2108791,202420372~1435645,210147535~53978881,267179572~13527794,281234113~14360231,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350891716~1451525,355806290~1463251,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:33.331 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~26560449,27329013~980763,30516268~23825099,55380210~10623546,66861224~16545045,92274688~11289903,106500403~5285199,112275487~21702107,134217728~14151836,149233764~4722012,154895619~3096498,159383552~8106770,167772160~8831430,182034135~10303497,196727604~2108791,202420372~1435645,210147535~53978881,264241152~16466214,281234113~14360231,297795584~7201936,306903613~23100153,331350016~15772466,350891716~1451525,355806290~1463251,366341667~25704170,394880591~2187459,398418332~12623422] 2024-04-07T17:37:33.331 INFO:tasks.workunit.client.0.smithi100.stdout:from 5 to HEAD diff [2329752~24230697,27329013~980763,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~12098856,149233764~4722012,154895619~3096498,166133159~1357163,168236274~8367316,182034135~10303497,196727604~2108791,202420372~1435645,210147535~38079381,248419159~15707257,267179572~13527794,281234113~14360231,298036890~6960630,306903613~5021411,314393585~13715786,332944411~11089603,345811290~1311192,355806290~1463251,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:33.331 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~24230697,27329013~980763,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~12098856,149233764~4722012,154895619~3096498,166133159~1357163,168236274~8367316,182034135~10303497,196727604~2108791,202420372~1435645,210147535~38079381,248419159~15707257,267179572~13527794,281234113~14360231,298036890~6960630,306903613~5021411,314393585~13715786,332944411~11089603,345811290~1311192,355806290~1463251,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:33.338 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~26560449,27329013~980763,30516268~23825099,55380210~10623546,66861224~16545045,92274688~11289903,106500403~5285199,112275487~21702107,134217728~14151836,149233764~4722012,154895619~3096498,166133159~1357163,167772160~8831430,182034135~10303497,196727604~2108791,202420372~1435645,210147535~38079381,248419159~15707257,264241152~16466214,281234113~14360231,298036890~6960630,306903613~21205758,331350016~15772466,355806290~1463251,366341667~7420413,374580519~17465318,394880591~2187459,398418332~12623422] 2024-04-07T17:37:33.338 INFO:tasks.workunit.client.0.smithi100.stdout:from 6 to HEAD diff [2329752~23418695,27329013~980763,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~12098856,149233764~4722012,154895619~3096498,168236274~8367316,182747004~9590628,202420372~1435645,210147535~38079381,250447165~8059589,267179572~11729329,281234113~14360231,298036890~6960630,306903613~5021411,314393585~13715786,332944411~6919544,345811290~1311192,355806290~1463251,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:33.338 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~23418695,27329013~980763,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~12098856,149233764~4722012,154895619~3096498,168236274~8367316,182747004~9590628,202420372~1435645,210147535~38079381,250447165~8059589,267179572~11729329,281234113~14360231,298036890~6960630,306903613~5021411,314393585~13715786,332944411~6919544,345811290~1311192,355806290~1463251,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:33.345 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~25748447,27329013~980763,30516268~23825099,55380210~10623546,66861224~16545045,92274688~11289903,106500403~5285199,112275487~21702107,134217728~14151836,149233764~4722012,154895619~3096498,167772160~8831430,182747004~9590628,202420372~1435645,210147535~38079381,250447165~8059589,264241152~14667749,281234113~14360231,298036890~6960630,306903613~21205758,331350016~8513939,344034014~3088468,355806290~1463251,366341667~7420413,374580519~17465318,394880591~2187459,398418332~12623422] 2024-04-07T17:37:33.345 INFO:tasks.workunit.client.0.smithi100.stdout:from 7 to HEAD diff [2329752~23418695,27329013~980763,31769017~13613313,45857929~8483438,55380210~10623546,68206575~15199694,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~5240179,141885014~6484550,149233764~4722012,154895619~3096498,168236274~8367316,182747004~9590628,210147535~38079381,250447165~8059589,267179572~11729329,281234113~14360231,298036890~6960630,306903613~5021411,314393585~13715786,332944411~6919544,355806290~1463251,366341667~7420413,377322524~14723313,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:33.345 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~23418695,27329013~980763,31769017~13613313,45857929~8483438,55380210~10623546,68206575~15199694,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~5240179,141885014~6484550,149233764~4722012,154895619~3096498,168236274~8367316,182747004~9590628,210147535~38079381,250447165~8059589,267179572~11729329,281234113~14360231,298036890~6960630,306903613~5021411,314393585~13715786,332944411~6919544,355806290~1463251,366341667~7420413,377322524~14723313,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:33.352 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~25748447,27329013~980763,30516268~23825099,55380210~10623546,68206575~15199694,92274688~11289903,106500403~5285199,112275487~21702107,134217728~7293159,141885014~6484550,149233764~4722012,154895619~3096498,167772160~8831430,182747004~9590628,210147535~38079381,250447165~8059589,264241152~14667749,281234113~14360231,298036890~6960630,306903613~21205758,331350016~8513939,355806290~1463251,366341667~7420413,377322524~14723313,394880591~2187459,398418332~12623422] 2024-04-07T17:37:33.352 INFO:tasks.workunit.client.0.smithi100.stdout:from 8 to HEAD diff [2329752~23418695,27329013~980763,31769017~13613313,45857929~8483438,55380210~10623546,68206575~15199694,95470042~8094549,106500403~5285199,112275487~7841113,121501610~10322918,136270708~5240179,141885014~6484550,149552729~4403047,168236274~8367316,182747004~9590628,210147535~38079381,250447165~8059589,267179572~11729329,281234113~14360231,298036890~6918207,306903613~5021411,314393585~13715786,332944411~6919544,355806290~1463251,366341667~7420413,377322524~14723313,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:33.352 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~23418695,27329013~980763,31769017~13613313,45857929~8483438,55380210~10623546,68206575~15199694,95470042~8094549,106500403~5285199,112275487~7841113,121501610~10322918,136270708~5240179,141885014~6484550,149552729~4403047,168236274~8367316,182747004~9590628,210147535~38079381,250447165~8059589,267179572~11729329,281234113~14360231,298036890~6918207,306903613~5021411,314393585~13715786,332944411~6919544,355806290~1463251,366341667~7420413,377322524~14723313,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:33.359 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~25748447,27329013~980763,30516268~23825099,55380210~10623546,68206575~15199694,95470042~8094549,106500403~5285199,112275487~19549041,134217728~7293159,141885014~6484550,149552729~4403047,167772160~8831430,182747004~9590628,210147535~38079381,250447165~8059589,264241152~14667749,281234113~14360231,298036890~6918207,306903613~21205758,331350016~8513939,355806290~1463251,366341667~7420413,377322524~14723313,394880591~2187459,398418332~12623422] 2024-04-07T17:37:33.359 INFO:tasks.workunit.client.0.smithi100.stdout:from 9 to HEAD diff [2329752~21800979,27329013~980763,37362822~8019508,45857929~8483438,55380210~10623546,68206575~15199694,95781519~7783072,106500403~5285199,112275487~7841113,121501610~10322918,136270708~5240179,141885014~6484550,149552729~4403047,168236274~8367316,183274874~3283813,186734432~153638,187506017~4831615,210147535~38079381,254075762~4430992,267179572~11729329,282485483~13108861,298036890~6918207,306903613~5021411,314393585~13715786,332944411~6919544,355806290~1463251,366341667~7420413,377322524~14723313,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:33.359 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~21800979,27329013~980763,37362822~8019508,45857929~8483438,55380210~10623546,68206575~15199694,95781519~7783072,106500403~5285199,112275487~7841113,121501610~10322918,136270708~5240179,141885014~6484550,149552729~4403047,168236274~8367316,183274874~3283813,186734432~153638,187506017~4831615,210147535~38079381,254075762~4430992,267179572~11729329,282485483~13108861,298036890~6918207,306903613~5021411,314393585~13715786,332944411~6919544,355806290~1463251,366341667~7420413,377322524~14723313,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:33.366 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~24130731,27329013~980763,37362822~16978545,55380210~10623546,68206575~15199694,95781519~7783072,106500403~5285199,112275487~19549041,134217728~7293159,141885014~6484550,149552729~4403047,167772160~8831430,183274874~3283813,186734432~153638,187506017~4831615,210147535~38079381,254075762~4430992,264241152~14667749,282485483~13108861,298036890~6918207,306903613~21205758,331350016~8513939,355806290~1463251,366341667~7420413,377322524~14723313,394880591~2187459,398418332~12623422] 2024-04-07T17:37:33.366 INFO:tasks.workunit.client.0.smithi100.stdout:from 10 to HEAD diff [2329752~21800979,27329013~249441,37362822~8019508,45857929~8483438,55380210~10623546,68206575~15199694,95781519~7783072,112275487~7841113,124240937~7583591,136270708~5240179,141885014~6484550,149552729~4403047,168236274~8367316,183274874~3283813,187506017~4831615,210147535~38079381,254075762~4430992,267179572~11729329,282485483~13108861,298036890~6918207,306903613~5021411,314393585~13715786,332944411~6919544,355806290~1463251,377322524~14723313,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:33.366 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~21800979,27329013~249441,37362822~8019508,45857929~8483438,55380210~10623546,68206575~15199694,95781519~7783072,112275487~7841113,124240937~7583591,136270708~5240179,141885014~6484550,149552729~4403047,168236274~8367316,183274874~3283813,187506017~4831615,210147535~38079381,254075762~4430992,267179572~11729329,282485483~13108861,298036890~6918207,306903613~5021411,314393585~13715786,332944411~6919544,355806290~1463251,377322524~14723313,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:33.373 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [2329752~21800979,27329013~249441,37362822~16978545,55380210~10623546,68206575~15199694,95781519~7783072,112275487~7841113,124240937~7583591,134217728~7293159,141885014~6484550,149552729~4403047,167772160~8831430,183274874~3283813,187506017~4831615,210147535~38079381,254075762~4430992,264241152~14667749,282485483~13108861,298036890~6918207,306903613~21205758,331350016~8513939,355806290~1463251,377322524~14723313,394880591~2187459,398418332~12623422] 2024-04-07T17:37:33.373 INFO:tasks.workunit.client.0.smithi100.stdout:from 11 to HEAD diff [2329752~21800979,27329013~249441,37386125~7996205,45857929~8483438,55380210~10623546,68206575~15199694,95781519~7783072,112275487~7841113,124240937~7583591,136270708~5240179,141885014~6484550,149552729~4403047,168236274~8367316,183274874~3283813,187506017~4831615,210147535~19406580,229742904~18484012,254075762~4430992,267179572~6811128,282485483~13108861,298036890~6918207,306903613~5021411,314393585~13715786,332944411~6919544,355806290~1463251,380094867~8701052,394880591~2187459,400878444~6009875,408516200~2525554] 2024-04-07T17:37:33.373 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~21800979,27329013~249441,37386125~7996205,45857929~8483438,55380210~10623546,68206575~15199694,95781519~7783072,112275487~7841113,124240937~7583591,136270708~5240179,141885014~6484550,149552729~4403047,168236274~8367316,183274874~3283813,187506017~4831615,210147535~19406580,229742904~18484012,254075762~4430992,267179572~6811128,282485483~13108861,298036890~6918207,306903613~5021411,314393585~13715786,332944411~6919544,355806290~1463251,380094867~8701052,394880591~2187459,400878444~6009875,408516200~2525554] 2024-04-07T17:37:33.381 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [2329752~21800979,27329013~249441,37386125~16955242,55380210~10623546,68206575~15199694,95781519~7783072,112275487~7841113,124240937~7583591,134217728~7293159,141885014~6484550,149552729~4403047,167772160~8831430,183274874~3283813,187506017~4831615,210147535~19406580,229742904~18484012,254075762~4430992,264241152~9749548,282485483~13108861,298036890~6918207,306903613~21205758,331350016~8513939,355806290~1463251,380094867~8701052,394880591~2187459,400878444~10163310] 2024-04-07T17:37:33.381 INFO:tasks.workunit.client.0.smithi100.stdout:from 12 to HEAD diff [2329752~21800979,27329013~249441,39912210~4133268,48881748~4683249,55380210~10623546,68206575~15199694,95781519~3680106,124240937~7583591,136270708~5240179,141885014~6484550,149552729~4403047,168236274~8367316,183274874~3283813,187506017~4831615,210147535~19406580,229742904~18484012,254075762~4430992,267179572~6811128,282485483~11941900,298643235~5745703,306903613~5021411,314393585~12544817,332944411~6919544,355806290~1463251,380563114~8232805,394880591~2187459,401962264~4926055,408516200~2525554] 2024-04-07T17:37:33.381 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~21800979,27329013~249441,39912210~4133268,48881748~4683249,55380210~10623546,68206575~15199694,95781519~3680106,124240937~7583591,136270708~5240179,141885014~6484550,149552729~4403047,168236274~8367316,183274874~3283813,187506017~4831615,210147535~19406580,229742904~18484012,254075762~4430992,267179572~6811128,282485483~11941900,298643235~5745703,306903613~5021411,314393585~12544817,332944411~6919544,355806290~1463251,380563114~8232805,394880591~2187459,401962264~4926055,408516200~2525554] 2024-04-07T17:37:33.388 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [2329752~21800979,27329013~249441,39912210~4133268,48881748~4683249,55380210~10623546,68206575~15199694,95781519~3680106,124240937~7583591,134217728~7293159,141885014~6484550,149552729~4403047,167772160~8831430,183274874~3283813,187506017~4831615,210147535~19406580,229742904~18484012,254075762~4430992,264241152~9749548,282485483~11941900,298643235~5745703,306903613~20034789,331350016~8513939,355806290~1463251,380563114~8232805,394880591~2187459,401962264~9079490] 2024-04-07T17:37:33.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:37:33 smithi100 bash[74942]: cluster 2024-04-07T17:37:32.482692+0000 mgr.y (mgr.25175) 2465 : cluster [DBG] pgmap v4059: 137 pgs: 1 peering, 136 active+clean; 876 MiB data, 5.8 GiB used, 709 GiB / 715 GiB avail; 70 KiB/s rd, 3.5 KiB/s wr, 1.98k op/s; 30 MiB/s, 2 keys/s, 16 objects/s recovering 2024-04-07T17:37:33.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:37:33 smithi100 bash[74942]: cluster 2024-04-07T17:37:32.690602+0000 mon.a (mon.0) 3902 : cluster [DBG] osdmap e2020: 8 total, 8 up, 8 in 2024-04-07T17:37:33.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:37:33 smithi100 bash[74942]: cluster 2024-04-07T17:37:33.204724+0000 mon.a (mon.0) 3903 : cluster [DBG] osdmap e2021: 8 total, 8 up, 8 in 2024-04-07T17:37:33.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:37:33 smithi100 bash[76202]: cluster 2024-04-07T17:37:32.482692+0000 mgr.y (mgr.25175) 2465 : cluster [DBG] pgmap v4059: 137 pgs: 1 peering, 136 active+clean; 876 MiB data, 5.8 GiB used, 709 GiB / 715 GiB avail; 70 KiB/s rd, 3.5 KiB/s wr, 1.98k op/s; 30 MiB/s, 2 keys/s, 16 objects/s recovering 2024-04-07T17:37:33.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:37:33 smithi100 bash[76202]: cluster 2024-04-07T17:37:32.690602+0000 mon.a (mon.0) 3902 : cluster [DBG] osdmap e2020: 8 total, 8 up, 8 in 2024-04-07T17:37:33.955 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:37:33 smithi100 bash[76202]: cluster 2024-04-07T17:37:33.204724+0000 mon.a (mon.0) 3903 : cluster [DBG] osdmap e2021: 8 total, 8 up, 8 in 2024-04-07T17:37:34.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:37:33 smithi144 bash[66654]: cluster 2024-04-07T17:37:32.482692+0000 mgr.y (mgr.25175) 2465 : cluster [DBG] pgmap v4059: 137 pgs: 1 peering, 136 active+clean; 876 MiB data, 5.8 GiB used, 709 GiB / 715 GiB avail; 70 KiB/s rd, 3.5 KiB/s wr, 1.98k op/s; 30 MiB/s, 2 keys/s, 16 objects/s recovering 2024-04-07T17:37:34.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:37:33 smithi144 bash[66654]: cluster 2024-04-07T17:37:32.690602+0000 mon.a (mon.0) 3902 : cluster [DBG] osdmap e2020: 8 total, 8 up, 8 in 2024-04-07T17:37:34.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:37:33 smithi144 bash[66654]: cluster 2024-04-07T17:37:33.204724+0000 mon.a (mon.0) 3903 : cluster [DBG] osdmap e2021: 8 total, 8 up, 8 in 2024-04-07T17:37:34.229 INFO:tasks.workunit.client.0.smithi100.stdout:from 0 to HEAD diff [2329752~26214847,29778841~737427,31769017~13613313,45857929~20145827,66861224~16545045,85466334~5069656,92999623~10564968,106500403~13616197,121501610~12475984,136270708~22564048,160337176~7153146,168236274~8367316,180237750~12590714,194102759~7364680,202420372~5254601,210147535~53978881,267179572~28414772,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350406459~6863082,360049192~84791,366341667~7420413,374580519~17465318,392232408~14655911,408516200~2525554] 2024-04-07T17:37:34.229 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~26214847,29778841~737427,31769017~13613313,45857929~20145827,66861224~16545045,85466334~5069656,92999623~10564968,106500403~13616197,121501610~12475984,136270708~22564048,160337176~7153146,168236274~8367316,180237750~12590714,194102759~7364680,202420372~5254601,210147535~53978881,267179572~28414772,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350406459~6863082,360049192~84791,366341667~7420413,374580519~17465318,392232408~14655911,408516200~2525554] 2024-04-07T17:37:34.296 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~28544599,29360128~54046141,83886080~6649910,92274688~11289903,104857600~29119994,134217728~24617028,159383552~8106770,167772160~25056304,192937984~14736989,210147535~53978881,264241152~31353192,297795584~7201936,306184192~23819574,331350016~15772466,348127232~12006751,366341667~44700087] 2024-04-07T17:37:34.296 INFO:tasks.workunit.client.0.smithi100.stdout:from 1 to HEAD diff [2329752~26214847,29778841~737427,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~13616197,121501610~12475984,136270708~22564048,160337176~7153146,168236274~8367316,180237750~12590714,194866296~6601143,202420372~5254601,210147535~53978881,267179572~28414772,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350891716~1451525,355806290~1463251,360049192~84791,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:34.296 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~26214847,29778841~737427,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~13616197,121501610~12475984,136270708~22564048,160337176~7153146,168236274~8367316,180237750~12590714,194866296~6601143,202420372~5254601,210147535~53978881,267179572~28414772,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350891716~1451525,355806290~1463251,360049192~84791,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:34.311 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~28544599,29360128~24981239,55380210~10623546,66861224~16545045,92274688~11289903,104857600~29119994,134217728~24617028,159383552~8106770,167772160~25056304,194866296~12808677,210147535~53978881,264241152~31353192,297795584~7201936,306184192~23819574,331350016~15772466,350891716~1451525,355806290~4327693,366341667~25704170,394880591~2187459,398418332~12623422] 2024-04-07T17:37:34.312 INFO:tasks.workunit.client.0.smithi100.stdout:from 2 to HEAD diff [2329752~24230697,27329013~980763,29778841~737427,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~17685068,154799337~4035419,160337176~7153146,168236274~8367316,182034135~10794329,196727604~2108791,202420372~5254601,210147535~53978881,267179572~28414772,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350891716~1451525,355806290~1463251,360049192~84791,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:34.312 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~24230697,27329013~980763,29778841~737427,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~17685068,154799337~4035419,160337176~7153146,168236274~8367316,182034135~10794329,196727604~2108791,202420372~5254601,210147535~53978881,267179572~28414772,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350891716~1451525,355806290~1463251,360049192~84791,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:34.321 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~26560449,27329013~980763,29360128~24981239,55380210~10623546,66861224~16545045,92274688~11289903,106500403~5285199,112275487~21702107,134217728~19738048,154799337~4035419,159383552~8106770,167772160~8831430,182034135~10794329,196727604~2108791,201467439~6207534,210147535~53978881,264241152~31353192,297795584~7201936,306184192~23819574,331350016~15772466,350891716~1451525,355806290~4327693,366341667~25704170,394880591~2187459,398418332~12623422] 2024-04-07T17:37:34.321 INFO:tasks.workunit.client.0.smithi100.stdout:from 3 to HEAD diff [2329752~24230697,27329013~980763,29778841~737427,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~17685068,154799337~3192780,160337176~7153146,168236274~8367316,182034135~10303497,196727604~2108791,202420372~1435645,210147535~53978881,267179572~28414772,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350891716~1451525,355806290~1463251,360049192~84791,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:34.322 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~24230697,27329013~980763,29778841~737427,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~17685068,154799337~3192780,160337176~7153146,168236274~8367316,182034135~10303497,196727604~2108791,202420372~1435645,210147535~53978881,267179572~28414772,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350891716~1451525,355806290~1463251,360049192~84791,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:34.328 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~26560449,27329013~980763,29360128~24981239,55380210~10623546,66861224~16545045,92274688~11289903,106500403~5285199,112275487~21702107,134217728~19738048,154799337~3192780,159383552~8106770,167772160~8831430,182034135~10303497,196727604~2108791,202420372~1435645,210147535~53978881,264241152~31353192,297795584~7201936,306184192~23819574,331350016~15772466,350891716~1451525,355806290~4327693,366341667~25704170,394880591~2187459,398418332~12623422] 2024-04-07T17:37:34.328 INFO:tasks.workunit.client.0.smithi100.stdout:from 4 to HEAD diff [2329752~24230697,27329013~980763,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~12098856,149233764~4722012,154895619~3096498,160337176~7153146,168236274~8367316,182034135~10303497,196727604~2108791,202420372~1435645,210147535~53978881,267179572~13527794,281234113~14360231,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350891716~1451525,355806290~1463251,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:34.328 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~24230697,27329013~980763,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~12098856,149233764~4722012,154895619~3096498,160337176~7153146,168236274~8367316,182034135~10303497,196727604~2108791,202420372~1435645,210147535~53978881,267179572~13527794,281234113~14360231,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350891716~1451525,355806290~1463251,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:34.336 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~26560449,27329013~980763,30516268~23825099,55380210~10623546,66861224~16545045,92274688~11289903,106500403~5285199,112275487~21702107,134217728~14151836,149233764~4722012,154895619~3096498,159383552~8106770,167772160~8831430,182034135~10303497,196727604~2108791,202420372~1435645,210147535~53978881,264241152~16466214,281234113~14360231,297795584~7201936,306903613~23100153,331350016~15772466,350891716~1451525,355806290~1463251,366341667~25704170,394880591~2187459,398418332~12623422] 2024-04-07T17:37:34.336 INFO:tasks.workunit.client.0.smithi100.stdout:from 5 to HEAD diff [2329752~24230697,27329013~980763,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~12098856,149233764~4722012,154895619~3096498,166133159~1357163,168236274~8367316,182034135~10303497,196727604~2108791,202420372~1435645,210147535~38079381,248419159~15707257,267179572~13527794,281234113~14360231,298036890~6960630,306903613~5021411,314393585~13715786,332944411~11089603,345811290~1311192,355806290~1463251,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:34.336 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~24230697,27329013~980763,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~12098856,149233764~4722012,154895619~3096498,166133159~1357163,168236274~8367316,182034135~10303497,196727604~2108791,202420372~1435645,210147535~38079381,248419159~15707257,267179572~13527794,281234113~14360231,298036890~6960630,306903613~5021411,314393585~13715786,332944411~11089603,345811290~1311192,355806290~1463251,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:34.343 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~26560449,27329013~980763,30516268~23825099,55380210~10623546,66861224~16545045,92274688~11289903,106500403~5285199,112275487~21702107,134217728~14151836,149233764~4722012,154895619~3096498,166133159~1357163,167772160~8831430,182034135~10303497,196727604~2108791,202420372~1435645,210147535~38079381,248419159~15707257,264241152~16466214,281234113~14360231,298036890~6960630,306903613~21205758,331350016~15772466,355806290~1463251,366341667~7420413,374580519~17465318,394880591~2187459,398418332~12623422] 2024-04-07T17:37:34.343 INFO:tasks.workunit.client.0.smithi100.stdout:from 6 to HEAD diff [2329752~23418695,27329013~980763,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~12098856,149233764~4722012,154895619~3096498,168236274~8367316,182747004~9590628,202420372~1435645,210147535~38079381,250447165~8059589,267179572~11729329,281234113~14360231,298036890~6960630,306903613~5021411,314393585~13715786,332944411~6919544,345811290~1311192,355806290~1463251,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:34.343 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~23418695,27329013~980763,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~12098856,149233764~4722012,154895619~3096498,168236274~8367316,182747004~9590628,202420372~1435645,210147535~38079381,250447165~8059589,267179572~11729329,281234113~14360231,298036890~6960630,306903613~5021411,314393585~13715786,332944411~6919544,345811290~1311192,355806290~1463251,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:34.350 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~25748447,27329013~980763,30516268~23825099,55380210~10623546,66861224~16545045,92274688~11289903,106500403~5285199,112275487~21702107,134217728~14151836,149233764~4722012,154895619~3096498,167772160~8831430,182747004~9590628,202420372~1435645,210147535~38079381,250447165~8059589,264241152~14667749,281234113~14360231,298036890~6960630,306903613~21205758,331350016~8513939,344034014~3088468,355806290~1463251,366341667~7420413,374580519~17465318,394880591~2187459,398418332~12623422] 2024-04-07T17:37:34.350 INFO:tasks.workunit.client.0.smithi100.stdout:from 7 to HEAD diff [2329752~23418695,27329013~980763,31769017~13613313,45857929~8483438,55380210~10623546,68206575~15199694,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~5240179,141885014~6484550,149233764~4722012,154895619~3096498,168236274~8367316,182747004~9590628,210147535~38079381,250447165~8059589,267179572~11729329,281234113~14360231,298036890~6960630,306903613~5021411,314393585~13715786,332944411~6919544,355806290~1463251,366341667~7420413,377322524~14723313,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:34.350 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~23418695,27329013~980763,31769017~13613313,45857929~8483438,55380210~10623546,68206575~15199694,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~5240179,141885014~6484550,149233764~4722012,154895619~3096498,168236274~8367316,182747004~9590628,210147535~38079381,250447165~8059589,267179572~11729329,281234113~14360231,298036890~6960630,306903613~5021411,314393585~13715786,332944411~6919544,355806290~1463251,366341667~7420413,377322524~14723313,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:34.357 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~25748447,27329013~980763,30516268~23825099,55380210~10623546,68206575~15199694,92274688~11289903,106500403~5285199,112275487~21702107,134217728~7293159,141885014~6484550,149233764~4722012,154895619~3096498,167772160~8831430,182747004~9590628,210147535~38079381,250447165~8059589,264241152~14667749,281234113~14360231,298036890~6960630,306903613~21205758,331350016~8513939,355806290~1463251,366341667~7420413,377322524~14723313,394880591~2187459,398418332~12623422] 2024-04-07T17:37:34.357 INFO:tasks.workunit.client.0.smithi100.stdout:from 8 to HEAD diff [2329752~23418695,27329013~980763,31769017~13613313,45857929~8483438,55380210~10623546,68206575~15199694,95470042~8094549,106500403~5285199,112275487~7841113,121501610~10322918,136270708~5240179,141885014~6484550,149552729~4403047,168236274~8367316,182747004~9590628,210147535~38079381,250447165~8059589,267179572~11729329,281234113~14360231,298036890~6918207,306903613~5021411,314393585~13715786,332944411~6919544,355806290~1463251,366341667~7420413,377322524~14723313,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:34.357 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~23418695,27329013~980763,31769017~13613313,45857929~8483438,55380210~10623546,68206575~15199694,95470042~8094549,106500403~5285199,112275487~7841113,121501610~10322918,136270708~5240179,141885014~6484550,149552729~4403047,168236274~8367316,182747004~9590628,210147535~38079381,250447165~8059589,267179572~11729329,281234113~14360231,298036890~6918207,306903613~5021411,314393585~13715786,332944411~6919544,355806290~1463251,366341667~7420413,377322524~14723313,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:34.364 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~25748447,27329013~980763,30516268~23825099,55380210~10623546,68206575~15199694,95470042~8094549,106500403~5285199,112275487~19549041,134217728~7293159,141885014~6484550,149552729~4403047,167772160~8831430,182747004~9590628,210147535~38079381,250447165~8059589,264241152~14667749,281234113~14360231,298036890~6918207,306903613~21205758,331350016~8513939,355806290~1463251,366341667~7420413,377322524~14723313,394880591~2187459,398418332~12623422] 2024-04-07T17:37:34.364 INFO:tasks.workunit.client.0.smithi100.stdout:from 9 to HEAD diff [2329752~21800979,27329013~980763,37362822~8019508,45857929~8483438,55380210~10623546,68206575~15199694,95781519~7783072,106500403~5285199,112275487~7841113,121501610~10322918,136270708~5240179,141885014~6484550,149552729~4403047,168236274~8367316,183274874~3283813,186734432~153638,187506017~4831615,210147535~38079381,254075762~4430992,267179572~11729329,282485483~13108861,298036890~6918207,306903613~5021411,314393585~13715786,332944411~6919544,355806290~1463251,366341667~7420413,377322524~14723313,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:34.364 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~21800979,27329013~980763,37362822~8019508,45857929~8483438,55380210~10623546,68206575~15199694,95781519~7783072,106500403~5285199,112275487~7841113,121501610~10322918,136270708~5240179,141885014~6484550,149552729~4403047,168236274~8367316,183274874~3283813,186734432~153638,187506017~4831615,210147535~38079381,254075762~4430992,267179572~11729329,282485483~13108861,298036890~6918207,306903613~5021411,314393585~13715786,332944411~6919544,355806290~1463251,366341667~7420413,377322524~14723313,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:34.371 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~24130731,27329013~980763,37362822~16978545,55380210~10623546,68206575~15199694,95781519~7783072,106500403~5285199,112275487~19549041,134217728~7293159,141885014~6484550,149552729~4403047,167772160~8831430,183274874~3283813,186734432~153638,187506017~4831615,210147535~38079381,254075762~4430992,264241152~14667749,282485483~13108861,298036890~6918207,306903613~21205758,331350016~8513939,355806290~1463251,366341667~7420413,377322524~14723313,394880591~2187459,398418332~12623422] 2024-04-07T17:37:34.371 INFO:tasks.workunit.client.0.smithi100.stdout:from 10 to HEAD diff [2329752~21800979,27329013~249441,37362822~8019508,45857929~8483438,55380210~10623546,68206575~15199694,95781519~7783072,112275487~7841113,124240937~7583591,136270708~5240179,141885014~6484550,149552729~4403047,168236274~8367316,183274874~3283813,187506017~4831615,210147535~38079381,254075762~4430992,267179572~11729329,282485483~13108861,298036890~6918207,306903613~5021411,314393585~13715786,332944411~6919544,355806290~1463251,377322524~14723313,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:34.371 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~21800979,27329013~249441,37362822~8019508,45857929~8483438,55380210~10623546,68206575~15199694,95781519~7783072,112275487~7841113,124240937~7583591,136270708~5240179,141885014~6484550,149552729~4403047,168236274~8367316,183274874~3283813,187506017~4831615,210147535~38079381,254075762~4430992,267179572~11729329,282485483~13108861,298036890~6918207,306903613~5021411,314393585~13715786,332944411~6919544,355806290~1463251,377322524~14723313,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:34.378 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [2329752~21800979,27329013~249441,37362822~16978545,55380210~10623546,68206575~15199694,95781519~7783072,112275487~7841113,124240937~7583591,134217728~7293159,141885014~6484550,149552729~4403047,167772160~8831430,183274874~3283813,187506017~4831615,210147535~38079381,254075762~4430992,264241152~14667749,282485483~13108861,298036890~6918207,306903613~21205758,331350016~8513939,355806290~1463251,377322524~14723313,394880591~2187459,398418332~12623422] 2024-04-07T17:37:34.378 INFO:tasks.workunit.client.0.smithi100.stdout:from 11 to HEAD diff [2329752~21800979,27329013~249441,37386125~7996205,45857929~8483438,55380210~10623546,68206575~15199694,95781519~7783072,112275487~7841113,124240937~7583591,136270708~5240179,141885014~6484550,149552729~4403047,168236274~8367316,183274874~3283813,187506017~4831615,210147535~19406580,229742904~18484012,254075762~4430992,267179572~6811128,282485483~13108861,298036890~6918207,306903613~5021411,314393585~13715786,332944411~6919544,355806290~1463251,380094867~8701052,394880591~2187459,400878444~6009875,408516200~2525554] 2024-04-07T17:37:34.378 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~21800979,27329013~249441,37386125~7996205,45857929~8483438,55380210~10623546,68206575~15199694,95781519~7783072,112275487~7841113,124240937~7583591,136270708~5240179,141885014~6484550,149552729~4403047,168236274~8367316,183274874~3283813,187506017~4831615,210147535~19406580,229742904~18484012,254075762~4430992,267179572~6811128,282485483~13108861,298036890~6918207,306903613~5021411,314393585~13715786,332944411~6919544,355806290~1463251,380094867~8701052,394880591~2187459,400878444~6009875,408516200~2525554] 2024-04-07T17:37:34.384 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [2329752~21800979,27329013~249441,37386125~16955242,55380210~10623546,68206575~15199694,95781519~7783072,112275487~7841113,124240937~7583591,134217728~7293159,141885014~6484550,149552729~4403047,167772160~8831430,183274874~3283813,187506017~4831615,210147535~19406580,229742904~18484012,254075762~4430992,264241152~9749548,282485483~13108861,298036890~6918207,306903613~21205758,331350016~8513939,355806290~1463251,380094867~8701052,394880591~2187459,400878444~10163310] 2024-04-07T17:37:35.238 INFO:tasks.workunit.client.0.smithi100.stdout:from 0 to HEAD diff [2329752~26214847,29778841~737427,31769017~13613313,45857929~20145827,66861224~16545045,85466334~5069656,92999623~10564968,106500403~13616197,121501610~12475984,136270708~22564048,160337176~7153146,168236274~8367316,180237750~12590714,194102759~7364680,202420372~5254601,210147535~53978881,267179572~28414772,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350406459~6863082,360049192~84791,366341667~7420413,374580519~17465318,392232408~14655911,408516200~2525554] 2024-04-07T17:37:35.238 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~26214847,29778841~737427,31769017~13613313,45857929~20145827,66861224~16545045,85466334~5069656,92999623~10564968,106500403~13616197,121501610~12475984,136270708~22564048,160337176~7153146,168236274~8367316,180237750~12590714,194102759~7364680,202420372~5254601,210147535~53978881,267179572~28414772,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350406459~6863082,360049192~84791,366341667~7420413,374580519~17465318,392232408~14655911,408516200~2525554] 2024-04-07T17:37:35.306 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~28544599,29360128~54046141,83886080~6649910,92274688~11289903,104857600~29119994,134217728~24617028,159383552~8106770,167772160~25056304,192937984~14736989,210147535~53978881,264241152~31353192,297795584~7201936,306184192~23819574,331350016~15772466,348127232~12006751,366341667~44700087] 2024-04-07T17:37:35.306 INFO:tasks.workunit.client.0.smithi100.stdout:from 1 to HEAD diff [2329752~26214847,29778841~737427,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~13616197,121501610~12475984,136270708~22564048,160337176~7153146,168236274~8367316,180237750~12590714,194866296~6601143,202420372~5254601,210147535~53978881,267179572~28414772,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350891716~1451525,355806290~1463251,360049192~84791,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:35.306 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~26214847,29778841~737427,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~13616197,121501610~12475984,136270708~22564048,160337176~7153146,168236274~8367316,180237750~12590714,194866296~6601143,202420372~5254601,210147535~53978881,267179572~28414772,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350891716~1451525,355806290~1463251,360049192~84791,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:35.322 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~28544599,29360128~24981239,55380210~10623546,66861224~16545045,92274688~11289903,104857600~29119994,134217728~24617028,159383552~8106770,167772160~25056304,194866296~12808677,210147535~53978881,264241152~31353192,297795584~7201936,306184192~23819574,331350016~15772466,350891716~1451525,355806290~4327693,366341667~25704170,394880591~2187459,398418332~12623422] 2024-04-07T17:37:35.322 INFO:tasks.workunit.client.0.smithi100.stdout:from 2 to HEAD diff [2329752~24230697,27329013~980763,29778841~737427,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~17685068,154799337~4035419,160337176~7153146,168236274~8367316,182034135~10794329,196727604~2108791,202420372~5254601,210147535~53978881,267179572~28414772,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350891716~1451525,355806290~1463251,360049192~84791,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:35.322 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~24230697,27329013~980763,29778841~737427,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~17685068,154799337~4035419,160337176~7153146,168236274~8367316,182034135~10794329,196727604~2108791,202420372~5254601,210147535~53978881,267179572~28414772,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350891716~1451525,355806290~1463251,360049192~84791,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:35.330 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~26560449,27329013~980763,29360128~24981239,55380210~10623546,66861224~16545045,92274688~11289903,106500403~5285199,112275487~21702107,134217728~19738048,154799337~4035419,159383552~8106770,167772160~8831430,182034135~10794329,196727604~2108791,201467439~6207534,210147535~53978881,264241152~31353192,297795584~7201936,306184192~23819574,331350016~15772466,350891716~1451525,355806290~4327693,366341667~25704170,394880591~2187459,398418332~12623422] 2024-04-07T17:37:35.331 INFO:tasks.workunit.client.0.smithi100.stdout:from 3 to HEAD diff [2329752~24230697,27329013~980763,29778841~737427,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~17685068,154799337~3192780,160337176~7153146,168236274~8367316,182034135~10303497,196727604~2108791,202420372~1435645,210147535~53978881,267179572~28414772,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350891716~1451525,355806290~1463251,360049192~84791,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:35.331 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~24230697,27329013~980763,29778841~737427,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~17685068,154799337~3192780,160337176~7153146,168236274~8367316,182034135~10303497,196727604~2108791,202420372~1435645,210147535~53978881,267179572~28414772,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350891716~1451525,355806290~1463251,360049192~84791,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:35.337 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~26560449,27329013~980763,29360128~24981239,55380210~10623546,66861224~16545045,92274688~11289903,106500403~5285199,112275487~21702107,134217728~19738048,154799337~3192780,159383552~8106770,167772160~8831430,182034135~10303497,196727604~2108791,202420372~1435645,210147535~53978881,264241152~31353192,297795584~7201936,306184192~23819574,331350016~15772466,350891716~1451525,355806290~4327693,366341667~25704170,394880591~2187459,398418332~12623422] 2024-04-07T17:37:35.337 INFO:tasks.workunit.client.0.smithi100.stdout:from 4 to HEAD diff [2329752~24230697,27329013~980763,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~12098856,149233764~4722012,154895619~3096498,160337176~7153146,168236274~8367316,182034135~10303497,196727604~2108791,202420372~1435645,210147535~53978881,267179572~13527794,281234113~14360231,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350891716~1451525,355806290~1463251,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:35.337 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~24230697,27329013~980763,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~12098856,149233764~4722012,154895619~3096498,160337176~7153146,168236274~8367316,182034135~10303497,196727604~2108791,202420372~1435645,210147535~53978881,267179572~13527794,281234113~14360231,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350891716~1451525,355806290~1463251,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:35.344 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~26560449,27329013~980763,30516268~23825099,55380210~10623546,66861224~16545045,92274688~11289903,106500403~5285199,112275487~21702107,134217728~14151836,149233764~4722012,154895619~3096498,159383552~8106770,167772160~8831430,182034135~10303497,196727604~2108791,202420372~1435645,210147535~53978881,264241152~16466214,281234113~14360231,297795584~7201936,306903613~23100153,331350016~15772466,350891716~1451525,355806290~1463251,366341667~25704170,394880591~2187459,398418332~12623422] 2024-04-07T17:37:35.344 INFO:tasks.workunit.client.0.smithi100.stdout:from 5 to HEAD diff [2329752~24230697,27329013~980763,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~12098856,149233764~4722012,154895619~3096498,166133159~1357163,168236274~8367316,182034135~10303497,196727604~2108791,202420372~1435645,210147535~38079381,248419159~15707257,267179572~13527794,281234113~14360231,298036890~6960630,306903613~5021411,314393585~13715786,332944411~11089603,345811290~1311192,355806290~1463251,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:35.345 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~24230697,27329013~980763,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~12098856,149233764~4722012,154895619~3096498,166133159~1357163,168236274~8367316,182034135~10303497,196727604~2108791,202420372~1435645,210147535~38079381,248419159~15707257,267179572~13527794,281234113~14360231,298036890~6960630,306903613~5021411,314393585~13715786,332944411~11089603,345811290~1311192,355806290~1463251,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:35.352 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~26560449,27329013~980763,30516268~23825099,55380210~10623546,66861224~16545045,92274688~11289903,106500403~5285199,112275487~21702107,134217728~14151836,149233764~4722012,154895619~3096498,166133159~1357163,167772160~8831430,182034135~10303497,196727604~2108791,202420372~1435645,210147535~38079381,248419159~15707257,264241152~16466214,281234113~14360231,298036890~6960630,306903613~21205758,331350016~15772466,355806290~1463251,366341667~7420413,374580519~17465318,394880591~2187459,398418332~12623422] 2024-04-07T17:37:35.352 INFO:tasks.workunit.client.0.smithi100.stdout:from 6 to HEAD diff [2329752~23418695,27329013~980763,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~12098856,149233764~4722012,154895619~3096498,168236274~8367316,182747004~9590628,202420372~1435645,210147535~38079381,250447165~8059589,267179572~11729329,281234113~14360231,298036890~6960630,306903613~5021411,314393585~13715786,332944411~6919544,345811290~1311192,355806290~1463251,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:35.352 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~23418695,27329013~980763,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~12098856,149233764~4722012,154895619~3096498,168236274~8367316,182747004~9590628,202420372~1435645,210147535~38079381,250447165~8059589,267179572~11729329,281234113~14360231,298036890~6960630,306903613~5021411,314393585~13715786,332944411~6919544,345811290~1311192,355806290~1463251,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:35.359 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~25748447,27329013~980763,30516268~23825099,55380210~10623546,66861224~16545045,92274688~11289903,106500403~5285199,112275487~21702107,134217728~14151836,149233764~4722012,154895619~3096498,167772160~8831430,182747004~9590628,202420372~1435645,210147535~38079381,250447165~8059589,264241152~14667749,281234113~14360231,298036890~6960630,306903613~21205758,331350016~8513939,344034014~3088468,355806290~1463251,366341667~7420413,374580519~17465318,394880591~2187459,398418332~12623422] 2024-04-07T17:37:35.359 INFO:tasks.workunit.client.0.smithi100.stdout:from 7 to HEAD diff [2329752~23418695,27329013~980763,31769017~13613313,45857929~8483438,55380210~10623546,68206575~15199694,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~5240179,141885014~6484550,149233764~4722012,154895619~3096498,168236274~8367316,182747004~9590628,210147535~38079381,250447165~8059589,267179572~11729329,281234113~14360231,298036890~6960630,306903613~5021411,314393585~13715786,332944411~6919544,355806290~1463251,366341667~7420413,377322524~14723313,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:35.359 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~23418695,27329013~980763,31769017~13613313,45857929~8483438,55380210~10623546,68206575~15199694,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~5240179,141885014~6484550,149233764~4722012,154895619~3096498,168236274~8367316,182747004~9590628,210147535~38079381,250447165~8059589,267179572~11729329,281234113~14360231,298036890~6960630,306903613~5021411,314393585~13715786,332944411~6919544,355806290~1463251,366341667~7420413,377322524~14723313,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:35.365 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~25748447,27329013~980763,30516268~23825099,55380210~10623546,68206575~15199694,92274688~11289903,106500403~5285199,112275487~21702107,134217728~7293159,141885014~6484550,149233764~4722012,154895619~3096498,167772160~8831430,182747004~9590628,210147535~38079381,250447165~8059589,264241152~14667749,281234113~14360231,298036890~6960630,306903613~21205758,331350016~8513939,355806290~1463251,366341667~7420413,377322524~14723313,394880591~2187459,398418332~12623422] 2024-04-07T17:37:35.365 INFO:tasks.workunit.client.0.smithi100.stdout:from 8 to HEAD diff [2329752~23418695,27329013~980763,31769017~13613313,45857929~8483438,55380210~10623546,68206575~15199694,95470042~8094549,106500403~5285199,112275487~7841113,121501610~10322918,136270708~5240179,141885014~6484550,149552729~4403047,168236274~8367316,182747004~9590628,210147535~38079381,250447165~8059589,267179572~11729329,281234113~14360231,298036890~6918207,306903613~5021411,314393585~13715786,332944411~6919544,355806290~1463251,366341667~7420413,377322524~14723313,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:35.365 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~23418695,27329013~980763,31769017~13613313,45857929~8483438,55380210~10623546,68206575~15199694,95470042~8094549,106500403~5285199,112275487~7841113,121501610~10322918,136270708~5240179,141885014~6484550,149552729~4403047,168236274~8367316,182747004~9590628,210147535~38079381,250447165~8059589,267179572~11729329,281234113~14360231,298036890~6918207,306903613~5021411,314393585~13715786,332944411~6919544,355806290~1463251,366341667~7420413,377322524~14723313,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:35.372 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~25748447,27329013~980763,30516268~23825099,55380210~10623546,68206575~15199694,95470042~8094549,106500403~5285199,112275487~19549041,134217728~7293159,141885014~6484550,149552729~4403047,167772160~8831430,182747004~9590628,210147535~38079381,250447165~8059589,264241152~14667749,281234113~14360231,298036890~6918207,306903613~21205758,331350016~8513939,355806290~1463251,366341667~7420413,377322524~14723313,394880591~2187459,398418332~12623422] 2024-04-07T17:37:35.372 INFO:tasks.workunit.client.0.smithi100.stdout:from 9 to HEAD diff [2329752~21800979,27329013~980763,37362822~8019508,45857929~8483438,55380210~10623546,68206575~15199694,95781519~7783072,106500403~5285199,112275487~7841113,121501610~10322918,136270708~5240179,141885014~6484550,149552729~4403047,168236274~8367316,183274874~3283813,186734432~153638,187506017~4831615,210147535~38079381,254075762~4430992,267179572~11729329,282485483~13108861,298036890~6918207,306903613~5021411,314393585~13715786,332944411~6919544,355806290~1463251,366341667~7420413,377322524~14723313,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:35.372 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~21800979,27329013~980763,37362822~8019508,45857929~8483438,55380210~10623546,68206575~15199694,95781519~7783072,106500403~5285199,112275487~7841113,121501610~10322918,136270708~5240179,141885014~6484550,149552729~4403047,168236274~8367316,183274874~3283813,186734432~153638,187506017~4831615,210147535~38079381,254075762~4430992,267179572~11729329,282485483~13108861,298036890~6918207,306903613~5021411,314393585~13715786,332944411~6919544,355806290~1463251,366341667~7420413,377322524~14723313,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:35.379 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~24130731,27329013~980763,37362822~16978545,55380210~10623546,68206575~15199694,95781519~7783072,106500403~5285199,112275487~19549041,134217728~7293159,141885014~6484550,149552729~4403047,167772160~8831430,183274874~3283813,186734432~153638,187506017~4831615,210147535~38079381,254075762~4430992,264241152~14667749,282485483~13108861,298036890~6918207,306903613~21205758,331350016~8513939,355806290~1463251,366341667~7420413,377322524~14723313,394880591~2187459,398418332~12623422] 2024-04-07T17:37:35.379 INFO:tasks.workunit.client.0.smithi100.stdout:from 10 to HEAD diff [2329752~21800979,27329013~249441,37362822~8019508,45857929~8483438,55380210~10623546,68206575~15199694,95781519~7783072,112275487~7841113,124240937~7583591,136270708~5240179,141885014~6484550,149552729~4403047,168236274~8367316,183274874~3283813,187506017~4831615,210147535~38079381,254075762~4430992,267179572~11729329,282485483~13108861,298036890~6918207,306903613~5021411,314393585~13715786,332944411~6919544,355806290~1463251,377322524~14723313,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:35.379 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~21800979,27329013~249441,37362822~8019508,45857929~8483438,55380210~10623546,68206575~15199694,95781519~7783072,112275487~7841113,124240937~7583591,136270708~5240179,141885014~6484550,149552729~4403047,168236274~8367316,183274874~3283813,187506017~4831615,210147535~38079381,254075762~4430992,267179572~11729329,282485483~13108861,298036890~6918207,306903613~5021411,314393585~13715786,332944411~6919544,355806290~1463251,377322524~14723313,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:35.385 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [2329752~21800979,27329013~249441,37362822~16978545,55380210~10623546,68206575~15199694,95781519~7783072,112275487~7841113,124240937~7583591,134217728~7293159,141885014~6484550,149552729~4403047,167772160~8831430,183274874~3283813,187506017~4831615,210147535~38079381,254075762~4430992,264241152~14667749,282485483~13108861,298036890~6918207,306903613~21205758,331350016~8513939,355806290~1463251,377322524~14723313,394880591~2187459,398418332~12623422] 2024-04-07T17:37:35.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:37:35 smithi144 bash[66654]: cluster 2024-04-07T17:37:34.204032+0000 mon.a (mon.0) 3904 : cluster [DBG] osdmap e2022: 8 total, 8 up, 8 in 2024-04-07T17:37:35.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:37:35 smithi144 bash[66654]: cluster 2024-04-07T17:37:34.483476+0000 mgr.y (mgr.25175) 2466 : cluster [DBG] pgmap v4063: 137 pgs: 1 peering, 136 active+clean; 857 MiB data, 5.8 GiB used, 709 GiB / 715 GiB avail; 59 KiB/s rd, 2.9 KiB/s wr, 1.21k op/s; 18 MiB/s, 2 keys/s, 8 objects/s recovering 2024-04-07T17:37:35.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:37:35 smithi100 bash[74942]: cluster 2024-04-07T17:37:34.204032+0000 mon.a (mon.0) 3904 : cluster [DBG] osdmap e2022: 8 total, 8 up, 8 in 2024-04-07T17:37:35.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:37:35 smithi100 bash[74942]: cluster 2024-04-07T17:37:34.483476+0000 mgr.y (mgr.25175) 2466 : cluster [DBG] pgmap v4063: 137 pgs: 1 peering, 136 active+clean; 857 MiB data, 5.8 GiB used, 709 GiB / 715 GiB avail; 59 KiB/s rd, 2.9 KiB/s wr, 1.21k op/s; 18 MiB/s, 2 keys/s, 8 objects/s recovering 2024-04-07T17:37:35.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:37:35 smithi100 bash[76202]: cluster 2024-04-07T17:37:34.204032+0000 mon.a (mon.0) 3904 : cluster [DBG] osdmap e2022: 8 total, 8 up, 8 in 2024-04-07T17:37:35.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:37:35 smithi100 bash[76202]: cluster 2024-04-07T17:37:34.483476+0000 mgr.y (mgr.25175) 2466 : cluster [DBG] pgmap v4063: 137 pgs: 1 peering, 136 active+clean; 857 MiB data, 5.8 GiB used, 709 GiB / 715 GiB avail; 59 KiB/s rd, 2.9 KiB/s wr, 1.21k op/s; 18 MiB/s, 2 keys/s, 8 objects/s recovering 2024-04-07T17:37:36.275 INFO:tasks.workunit.client.0.smithi100.stdout:from 0 to HEAD diff [2329752~26214847,29778841~737427,31769017~13613313,45857929~20145827,66861224~16545045,85466334~5069656,92999623~10564968,106500403~13616197,121501610~12475984,136270708~22564048,160337176~7153146,168236274~8367316,180237750~12590714,194102759~7364680,202420372~5254601,210147535~53978881,267179572~28414772,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350406459~6863082,360049192~84791,366341667~7420413,374580519~17465318,392232408~14655911,408516200~2525554] 2024-04-07T17:37:36.275 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~26214847,29778841~737427,31769017~13613313,45857929~20145827,66861224~16545045,85466334~5069656,92999623~10564968,106500403~13616197,121501610~12475984,136270708~22564048,160337176~7153146,168236274~8367316,180237750~12590714,194102759~7364680,202420372~5254601,210147535~53978881,267179572~28414772,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350406459~6863082,360049192~84791,366341667~7420413,374580519~17465318,392232408~14655911,408516200~2525554] 2024-04-07T17:37:36.328 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~28544599,29360128~54046141,83886080~6649910,92274688~11289903,104857600~29119994,134217728~24617028,159383552~8106770,167772160~25056304,192937984~14736989,210147535~53978881,264241152~31353192,297795584~7201936,306184192~23819574,331350016~15772466,348127232~12006751,366341667~44700087] 2024-04-07T17:37:36.329 INFO:tasks.workunit.client.0.smithi100.stdout:from 1 to HEAD diff [2329752~26214847,29778841~737427,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~13616197,121501610~12475984,136270708~22564048,160337176~7153146,168236274~8367316,180237750~12590714,194866296~6601143,202420372~5254601,210147535~53978881,267179572~28414772,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350891716~1451525,355806290~1463251,360049192~84791,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:36.329 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~26214847,29778841~737427,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~13616197,121501610~12475984,136270708~22564048,160337176~7153146,168236274~8367316,180237750~12590714,194866296~6601143,202420372~5254601,210147535~53978881,267179572~28414772,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350891716~1451525,355806290~1463251,360049192~84791,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:36.345 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~28544599,29360128~24981239,55380210~10623546,66861224~16545045,92274688~11289903,104857600~29119994,134217728~24617028,159383552~8106770,167772160~25056304,194866296~12808677,210147535~53978881,264241152~31353192,297795584~7201936,306184192~23819574,331350016~15772466,350891716~1451525,355806290~4327693,366341667~25704170,394880591~2187459,398418332~12623422] 2024-04-07T17:37:36.345 INFO:tasks.workunit.client.0.smithi100.stdout:from 2 to HEAD diff [2329752~24230697,27329013~980763,29778841~737427,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~17685068,154799337~4035419,160337176~7153146,168236274~8367316,182034135~10794329,196727604~2108791,202420372~5254601,210147535~53978881,267179572~28414772,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350891716~1451525,355806290~1463251,360049192~84791,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:36.345 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~24230697,27329013~980763,29778841~737427,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~17685068,154799337~4035419,160337176~7153146,168236274~8367316,182034135~10794329,196727604~2108791,202420372~5254601,210147535~53978881,267179572~28414772,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350891716~1451525,355806290~1463251,360049192~84791,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:36.351 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~26560449,27329013~980763,29360128~24981239,55380210~10623546,66861224~16545045,92274688~11289903,106500403~5285199,112275487~21702107,134217728~19738048,154799337~4035419,159383552~8106770,167772160~8831430,182034135~10794329,196727604~2108791,201467439~6207534,210147535~53978881,264241152~31353192,297795584~7201936,306184192~23819574,331350016~15772466,350891716~1451525,355806290~4327693,366341667~25704170,394880591~2187459,398418332~12623422] 2024-04-07T17:37:36.352 INFO:tasks.workunit.client.0.smithi100.stdout:from 3 to HEAD diff [2329752~24230697,27329013~980763,29778841~737427,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~17685068,154799337~3192780,160337176~7153146,168236274~8367316,182034135~10303497,196727604~2108791,202420372~1435645,210147535~53978881,267179572~28414772,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350891716~1451525,355806290~1463251,360049192~84791,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:36.352 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~24230697,27329013~980763,29778841~737427,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~17685068,154799337~3192780,160337176~7153146,168236274~8367316,182034135~10303497,196727604~2108791,202420372~1435645,210147535~53978881,267179572~28414772,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350891716~1451525,355806290~1463251,360049192~84791,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:36.361 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~26560449,27329013~980763,29360128~24981239,55380210~10623546,66861224~16545045,92274688~11289903,106500403~5285199,112275487~21702107,134217728~19738048,154799337~3192780,159383552~8106770,167772160~8831430,182034135~10303497,196727604~2108791,202420372~1435645,210147535~53978881,264241152~31353192,297795584~7201936,306184192~23819574,331350016~15772466,350891716~1451525,355806290~4327693,366341667~25704170,394880591~2187459,398418332~12623422] 2024-04-07T17:37:36.361 INFO:tasks.workunit.client.0.smithi100.stdout:from 4 to HEAD diff [2329752~24230697,27329013~980763,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~12098856,149233764~4722012,154895619~3096498,160337176~7153146,168236274~8367316,182034135~10303497,196727604~2108791,202420372~1435645,210147535~53978881,267179572~13527794,281234113~14360231,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350891716~1451525,355806290~1463251,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:36.361 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~24230697,27329013~980763,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~12098856,149233764~4722012,154895619~3096498,160337176~7153146,168236274~8367316,182034135~10303497,196727604~2108791,202420372~1435645,210147535~53978881,267179572~13527794,281234113~14360231,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350891716~1451525,355806290~1463251,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:36.368 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~26560449,27329013~980763,30516268~23825099,55380210~10623546,66861224~16545045,92274688~11289903,106500403~5285199,112275487~21702107,134217728~14151836,149233764~4722012,154895619~3096498,159383552~8106770,167772160~8831430,182034135~10303497,196727604~2108791,202420372~1435645,210147535~53978881,264241152~16466214,281234113~14360231,297795584~7201936,306903613~23100153,331350016~15772466,350891716~1451525,355806290~1463251,366341667~25704170,394880591~2187459,398418332~12623422] 2024-04-07T17:37:36.368 INFO:tasks.workunit.client.0.smithi100.stdout:from 5 to HEAD diff [2329752~24230697,27329013~980763,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~12098856,149233764~4722012,154895619~3096498,166133159~1357163,168236274~8367316,182034135~10303497,196727604~2108791,202420372~1435645,210147535~38079381,248419159~15707257,267179572~13527794,281234113~14360231,298036890~6960630,306903613~5021411,314393585~13715786,332944411~11089603,345811290~1311192,355806290~1463251,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:36.368 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~24230697,27329013~980763,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~12098856,149233764~4722012,154895619~3096498,166133159~1357163,168236274~8367316,182034135~10303497,196727604~2108791,202420372~1435645,210147535~38079381,248419159~15707257,267179572~13527794,281234113~14360231,298036890~6960630,306903613~5021411,314393585~13715786,332944411~11089603,345811290~1311192,355806290~1463251,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:36.375 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~26560449,27329013~980763,30516268~23825099,55380210~10623546,66861224~16545045,92274688~11289903,106500403~5285199,112275487~21702107,134217728~14151836,149233764~4722012,154895619~3096498,166133159~1357163,167772160~8831430,182034135~10303497,196727604~2108791,202420372~1435645,210147535~38079381,248419159~15707257,264241152~16466214,281234113~14360231,298036890~6960630,306903613~21205758,331350016~15772466,355806290~1463251,366341667~7420413,374580519~17465318,394880591~2187459,398418332~12623422] 2024-04-07T17:37:36.375 INFO:tasks.workunit.client.0.smithi100.stdout:from 6 to HEAD diff [2329752~23418695,27329013~980763,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~12098856,149233764~4722012,154895619~3096498,168236274~8367316,182747004~9590628,202420372~1435645,210147535~38079381,250447165~8059589,267179572~11729329,281234113~14360231,298036890~6960630,306903613~5021411,314393585~13715786,332944411~6919544,345811290~1311192,355806290~1463251,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:36.375 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~23418695,27329013~980763,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~12098856,149233764~4722012,154895619~3096498,168236274~8367316,182747004~9590628,202420372~1435645,210147535~38079381,250447165~8059589,267179572~11729329,281234113~14360231,298036890~6960630,306903613~5021411,314393585~13715786,332944411~6919544,345811290~1311192,355806290~1463251,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:36.382 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~25748447,27329013~980763,30516268~23825099,55380210~10623546,66861224~16545045,92274688~11289903,106500403~5285199,112275487~21702107,134217728~14151836,149233764~4722012,154895619~3096498,167772160~8831430,182747004~9590628,202420372~1435645,210147535~38079381,250447165~8059589,264241152~14667749,281234113~14360231,298036890~6960630,306903613~21205758,331350016~8513939,344034014~3088468,355806290~1463251,366341667~7420413,374580519~17465318,394880591~2187459,398418332~12623422] 2024-04-07T17:37:36.383 INFO:tasks.workunit.client.0.smithi100.stdout:from 7 to HEAD diff [2329752~23418695,27329013~980763,31769017~13613313,45857929~8483438,55380210~10623546,68206575~15199694,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~5240179,141885014~6484550,149233764~4722012,154895619~3096498,168236274~8367316,182747004~9590628,210147535~38079381,250447165~8059589,267179572~11729329,281234113~14360231,298036890~6960630,306903613~5021411,314393585~13715786,332944411~6919544,355806290~1463251,366341667~7420413,377322524~14723313,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:36.383 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~23418695,27329013~980763,31769017~13613313,45857929~8483438,55380210~10623546,68206575~15199694,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~5240179,141885014~6484550,149233764~4722012,154895619~3096498,168236274~8367316,182747004~9590628,210147535~38079381,250447165~8059589,267179572~11729329,281234113~14360231,298036890~6960630,306903613~5021411,314393585~13715786,332944411~6919544,355806290~1463251,366341667~7420413,377322524~14723313,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:36.389 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~25748447,27329013~980763,30516268~23825099,55380210~10623546,68206575~15199694,92274688~11289903,106500403~5285199,112275487~21702107,134217728~7293159,141885014~6484550,149233764~4722012,154895619~3096498,167772160~8831430,182747004~9590628,210147535~38079381,250447165~8059589,264241152~14667749,281234113~14360231,298036890~6960630,306903613~21205758,331350016~8513939,355806290~1463251,366341667~7420413,377322524~14723313,394880591~2187459,398418332~12623422] 2024-04-07T17:37:36.390 INFO:tasks.workunit.client.0.smithi100.stdout:from 8 to HEAD diff [2329752~23418695,27329013~980763,31769017~13613313,45857929~8483438,55380210~10623546,68206575~15199694,95470042~8094549,106500403~5285199,112275487~7841113,121501610~10322918,136270708~5240179,141885014~6484550,149552729~4403047,168236274~8367316,182747004~9590628,210147535~38079381,250447165~8059589,267179572~11729329,281234113~14360231,298036890~6918207,306903613~5021411,314393585~13715786,332944411~6919544,355806290~1463251,366341667~7420413,377322524~14723313,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:36.390 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~23418695,27329013~980763,31769017~13613313,45857929~8483438,55380210~10623546,68206575~15199694,95470042~8094549,106500403~5285199,112275487~7841113,121501610~10322918,136270708~5240179,141885014~6484550,149552729~4403047,168236274~8367316,182747004~9590628,210147535~38079381,250447165~8059589,267179572~11729329,281234113~14360231,298036890~6918207,306903613~5021411,314393585~13715786,332944411~6919544,355806290~1463251,366341667~7420413,377322524~14723313,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:36.396 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~25748447,27329013~980763,30516268~23825099,55380210~10623546,68206575~15199694,95470042~8094549,106500403~5285199,112275487~19549041,134217728~7293159,141885014~6484550,149552729~4403047,167772160~8831430,182747004~9590628,210147535~38079381,250447165~8059589,264241152~14667749,281234113~14360231,298036890~6918207,306903613~21205758,331350016~8513939,355806290~1463251,366341667~7420413,377322524~14723313,394880591~2187459,398418332~12623422] 2024-04-07T17:37:36.396 INFO:tasks.workunit.client.0.smithi100.stdout:from 9 to HEAD diff [2329752~21800979,27329013~980763,37362822~8019508,45857929~8483438,55380210~10623546,68206575~15199694,95781519~7783072,106500403~5285199,112275487~7841113,121501610~10322918,136270708~5240179,141885014~6484550,149552729~4403047,168236274~8367316,183274874~3283813,186734432~153638,187506017~4831615,210147535~38079381,254075762~4430992,267179572~11729329,282485483~13108861,298036890~6918207,306903613~5021411,314393585~13715786,332944411~6919544,355806290~1463251,366341667~7420413,377322524~14723313,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:36.396 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~21800979,27329013~980763,37362822~8019508,45857929~8483438,55380210~10623546,68206575~15199694,95781519~7783072,106500403~5285199,112275487~7841113,121501610~10322918,136270708~5240179,141885014~6484550,149552729~4403047,168236274~8367316,183274874~3283813,186734432~153638,187506017~4831615,210147535~38079381,254075762~4430992,267179572~11729329,282485483~13108861,298036890~6918207,306903613~5021411,314393585~13715786,332944411~6919544,355806290~1463251,366341667~7420413,377322524~14723313,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:36.402 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~24130731,27329013~980763,37362822~16978545,55380210~10623546,68206575~15199694,95781519~7783072,106500403~5285199,112275487~19549041,134217728~7293159,141885014~6484550,149552729~4403047,167772160~8831430,183274874~3283813,186734432~153638,187506017~4831615,210147535~38079381,254075762~4430992,264241152~14667749,282485483~13108861,298036890~6918207,306903613~21205758,331350016~8513939,355806290~1463251,366341667~7420413,377322524~14723313,394880591~2187459,398418332~12623422] 2024-04-07T17:37:36.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:37:36 smithi144 bash[66654]: cluster 2024-04-07T17:37:35.209537+0000 mon.a (mon.0) 3905 : cluster [DBG] osdmap e2023: 8 total, 8 up, 8 in 2024-04-07T17:37:36.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:37:36 smithi100 bash[74942]: cluster 2024-04-07T17:37:35.209537+0000 mon.a (mon.0) 3905 : cluster [DBG] osdmap e2023: 8 total, 8 up, 8 in 2024-04-07T17:37:36.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:37:36 smithi100 bash[76202]: cluster 2024-04-07T17:37:35.209537+0000 mon.a (mon.0) 3905 : cluster [DBG] osdmap e2023: 8 total, 8 up, 8 in 2024-04-07T17:37:37.288 INFO:tasks.workunit.client.0.smithi100.stdout:from 0 to HEAD diff [2329752~26214847,29778841~737427,31769017~13613313,45857929~20145827,66861224~16545045,85466334~5069656,92999623~10564968,106500403~13616197,121501610~12475984,136270708~22564048,160337176~7153146,168236274~8367316,180237750~12590714,194102759~7364680,202420372~5254601,210147535~53978881,267179572~28414772,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350406459~6863082,360049192~84791,366341667~7420413,374580519~17465318,392232408~14655911,408516200~2525554] 2024-04-07T17:37:37.288 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~26214847,29778841~737427,31769017~13613313,45857929~20145827,66861224~16545045,85466334~5069656,92999623~10564968,106500403~13616197,121501610~12475984,136270708~22564048,160337176~7153146,168236274~8367316,180237750~12590714,194102759~7364680,202420372~5254601,210147535~53978881,267179572~28414772,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350406459~6863082,360049192~84791,366341667~7420413,374580519~17465318,392232408~14655911,408516200~2525554] 2024-04-07T17:37:37.375 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~28544599,29360128~54046141,83886080~6649910,92274688~11289903,104857600~29119994,134217728~24617028,159383552~8106770,167772160~25056304,192937984~14736989,210147535~53978881,264241152~31353192,297795584~7201936,306184192~23819574,331350016~15772466,348127232~12006751,366341667~44700087] 2024-04-07T17:37:37.375 INFO:tasks.workunit.client.0.smithi100.stdout:from 1 to HEAD diff [2329752~26214847,29778841~737427,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~13616197,121501610~12475984,136270708~22564048,160337176~7153146,168236274~8367316,180237750~12590714,194866296~6601143,202420372~5254601,210147535~53978881,267179572~28414772,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350891716~1451525,355806290~1463251,360049192~84791,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:37.376 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~26214847,29778841~737427,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~13616197,121501610~12475984,136270708~22564048,160337176~7153146,168236274~8367316,180237750~12590714,194866296~6601143,202420372~5254601,210147535~53978881,267179572~28414772,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350891716~1451525,355806290~1463251,360049192~84791,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:37.385 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~28544599,29360128~24981239,55380210~10623546,66861224~16545045,92274688~11289903,104857600~29119994,134217728~24617028,159383552~8106770,167772160~25056304,194866296~12808677,210147535~53978881,264241152~31353192,297795584~7201936,306184192~23819574,331350016~15772466,350891716~1451525,355806290~4327693,366341667~25704170,394880591~2187459,398418332~12623422] 2024-04-07T17:37:37.385 INFO:tasks.workunit.client.0.smithi100.stdout:from 2 to HEAD diff [2329752~24230697,27329013~980763,29778841~737427,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~17685068,154799337~4035419,160337176~7153146,168236274~8367316,182034135~10794329,196727604~2108791,202420372~5254601,210147535~53978881,267179572~28414772,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350891716~1451525,355806290~1463251,360049192~84791,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:37.385 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~24230697,27329013~980763,29778841~737427,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~17685068,154799337~4035419,160337176~7153146,168236274~8367316,182034135~10794329,196727604~2108791,202420372~5254601,210147535~53978881,267179572~28414772,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350891716~1451525,355806290~1463251,360049192~84791,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:37.393 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~26560449,27329013~980763,29360128~24981239,55380210~10623546,66861224~16545045,92274688~11289903,106500403~5285199,112275487~21702107,134217728~19738048,154799337~4035419,159383552~8106770,167772160~8831430,182034135~10794329,196727604~2108791,201467439~6207534,210147535~53978881,264241152~31353192,297795584~7201936,306184192~23819574,331350016~15772466,350891716~1451525,355806290~4327693,366341667~25704170,394880591~2187459,398418332~12623422] 2024-04-07T17:37:37.393 INFO:tasks.workunit.client.0.smithi100.stdout:from 3 to HEAD diff [2329752~24230697,27329013~980763,29778841~737427,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~17685068,154799337~3192780,160337176~7153146,168236274~8367316,182034135~10303497,196727604~2108791,202420372~1435645,210147535~53978881,267179572~28414772,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350891716~1451525,355806290~1463251,360049192~84791,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:37.393 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~24230697,27329013~980763,29778841~737427,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~17685068,154799337~3192780,160337176~7153146,168236274~8367316,182034135~10303497,196727604~2108791,202420372~1435645,210147535~53978881,267179572~28414772,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350891716~1451525,355806290~1463251,360049192~84791,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:37.435 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~26560449,27329013~980763,29360128~24981239,55380210~10623546,66861224~16545045,92274688~11289903,106500403~5285199,112275487~21702107,134217728~19738048,154799337~3192780,159383552~8106770,167772160~8831430,182034135~10303497,196727604~2108791,202420372~1435645,210147535~53978881,264241152~31353192,297795584~7201936,306184192~23819574,331350016~15772466,350891716~1451525,355806290~4327693,366341667~25704170,394880591~2187459,398418332~12623422] 2024-04-07T17:37:37.435 INFO:tasks.workunit.client.0.smithi100.stdout:from 4 to HEAD diff [2329752~24230697,27329013~980763,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~12098856,149233764~4722012,154895619~3096498,160337176~7153146,168236274~8367316,182034135~10303497,196727604~2108791,202420372~1435645,210147535~53978881,267179572~13527794,281234113~14360231,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350891716~1451525,355806290~1463251,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:37.435 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~24230697,27329013~980763,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~12098856,149233764~4722012,154895619~3096498,160337176~7153146,168236274~8367316,182034135~10303497,196727604~2108791,202420372~1435645,210147535~53978881,267179572~13527794,281234113~14360231,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350891716~1451525,355806290~1463251,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:37.445 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~26560449,27329013~980763,30516268~23825099,55380210~10623546,66861224~16545045,92274688~11289903,106500403~5285199,112275487~21702107,134217728~14151836,149233764~4722012,154895619~3096498,159383552~8106770,167772160~8831430,182034135~10303497,196727604~2108791,202420372~1435645,210147535~53978881,264241152~16466214,281234113~14360231,297795584~7201936,306903613~23100153,331350016~15772466,350891716~1451525,355806290~1463251,366341667~25704170,394880591~2187459,398418332~12623422] 2024-04-07T17:37:37.445 INFO:tasks.workunit.client.0.smithi100.stdout:from 5 to HEAD diff [2329752~24230697,27329013~980763,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~12098856,149233764~4722012,154895619~3096498,166133159~1357163,168236274~8367316,182034135~10303497,196727604~2108791,202420372~1435645,210147535~38079381,248419159~15707257,267179572~13527794,281234113~14360231,298036890~6960630,306903613~5021411,314393585~13715786,332944411~11089603,345811290~1311192,355806290~1463251,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:37.445 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~24230697,27329013~980763,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~12098856,149233764~4722012,154895619~3096498,166133159~1357163,168236274~8367316,182034135~10303497,196727604~2108791,202420372~1435645,210147535~38079381,248419159~15707257,267179572~13527794,281234113~14360231,298036890~6960630,306903613~5021411,314393585~13715786,332944411~11089603,345811290~1311192,355806290~1463251,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:37.455 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~26560449,27329013~980763,30516268~23825099,55380210~10623546,66861224~16545045,92274688~11289903,106500403~5285199,112275487~21702107,134217728~14151836,149233764~4722012,154895619~3096498,166133159~1357163,167772160~8831430,182034135~10303497,196727604~2108791,202420372~1435645,210147535~38079381,248419159~15707257,264241152~16466214,281234113~14360231,298036890~6960630,306903613~21205758,331350016~15772466,355806290~1463251,366341667~7420413,374580519~17465318,394880591~2187459,398418332~12623422] 2024-04-07T17:37:37.455 INFO:tasks.workunit.client.0.smithi100.stdout:from 6 to HEAD diff [2329752~23418695,27329013~980763,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~12098856,149233764~4722012,154895619~3096498,168236274~8367316,182747004~9590628,202420372~1435645,210147535~38079381,250447165~8059589,267179572~11729329,281234113~14360231,298036890~6960630,306903613~5021411,314393585~13715786,332944411~6919544,345811290~1311192,355806290~1463251,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:37.455 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~23418695,27329013~980763,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~12098856,149233764~4722012,154895619~3096498,168236274~8367316,182747004~9590628,202420372~1435645,210147535~38079381,250447165~8059589,267179572~11729329,281234113~14360231,298036890~6960630,306903613~5021411,314393585~13715786,332944411~6919544,345811290~1311192,355806290~1463251,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:37.464 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~25748447,27329013~980763,30516268~23825099,55380210~10623546,66861224~16545045,92274688~11289903,106500403~5285199,112275487~21702107,134217728~14151836,149233764~4722012,154895619~3096498,167772160~8831430,182747004~9590628,202420372~1435645,210147535~38079381,250447165~8059589,264241152~14667749,281234113~14360231,298036890~6960630,306903613~21205758,331350016~8513939,344034014~3088468,355806290~1463251,366341667~7420413,374580519~17465318,394880591~2187459,398418332~12623422] 2024-04-07T17:37:37.464 INFO:tasks.workunit.client.0.smithi100.stdout:from 7 to HEAD diff [2329752~23418695,27329013~980763,31769017~13613313,45857929~8483438,55380210~10623546,68206575~15199694,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~5240179,141885014~6484550,149233764~4722012,154895619~3096498,168236274~8367316,182747004~9590628,210147535~38079381,250447165~8059589,267179572~11729329,281234113~14360231,298036890~6960630,306903613~5021411,314393585~13715786,332944411~6919544,355806290~1463251,366341667~7420413,377322524~14723313,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:37.464 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~23418695,27329013~980763,31769017~13613313,45857929~8483438,55380210~10623546,68206575~15199694,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~5240179,141885014~6484550,149233764~4722012,154895619~3096498,168236274~8367316,182747004~9590628,210147535~38079381,250447165~8059589,267179572~11729329,281234113~14360231,298036890~6960630,306903613~5021411,314393585~13715786,332944411~6919544,355806290~1463251,366341667~7420413,377322524~14723313,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:37.473 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~25748447,27329013~980763,30516268~23825099,55380210~10623546,68206575~15199694,92274688~11289903,106500403~5285199,112275487~21702107,134217728~7293159,141885014~6484550,149233764~4722012,154895619~3096498,167772160~8831430,182747004~9590628,210147535~38079381,250447165~8059589,264241152~14667749,281234113~14360231,298036890~6960630,306903613~21205758,331350016~8513939,355806290~1463251,366341667~7420413,377322524~14723313,394880591~2187459,398418332~12623422] 2024-04-07T17:37:37.473 INFO:tasks.workunit.client.0.smithi100.stdout:from 8 to HEAD diff [2329752~23418695,27329013~980763,31769017~13613313,45857929~8483438,55380210~10623546,68206575~15199694,95470042~8094549,106500403~5285199,112275487~7841113,121501610~10322918,136270708~5240179,141885014~6484550,149552729~4403047,168236274~8367316,182747004~9590628,210147535~38079381,250447165~8059589,267179572~11729329,281234113~14360231,298036890~6918207,306903613~5021411,314393585~13715786,332944411~6919544,355806290~1463251,366341667~7420413,377322524~14723313,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:37.473 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~23418695,27329013~980763,31769017~13613313,45857929~8483438,55380210~10623546,68206575~15199694,95470042~8094549,106500403~5285199,112275487~7841113,121501610~10322918,136270708~5240179,141885014~6484550,149552729~4403047,168236274~8367316,182747004~9590628,210147535~38079381,250447165~8059589,267179572~11729329,281234113~14360231,298036890~6918207,306903613~5021411,314393585~13715786,332944411~6919544,355806290~1463251,366341667~7420413,377322524~14723313,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:37.481 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~25748447,27329013~980763,30516268~23825099,55380210~10623546,68206575~15199694,95470042~8094549,106500403~5285199,112275487~19549041,134217728~7293159,141885014~6484550,149552729~4403047,167772160~8831430,182747004~9590628,210147535~38079381,250447165~8059589,264241152~14667749,281234113~14360231,298036890~6918207,306903613~21205758,331350016~8513939,355806290~1463251,366341667~7420413,377322524~14723313,394880591~2187459,398418332~12623422] 2024-04-07T17:37:37.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:37:37 smithi144 bash[66654]: cluster 2024-04-07T17:37:36.230983+0000 mon.a (mon.0) 3906 : cluster [DBG] osdmap e2024: 8 total, 8 up, 8 in 2024-04-07T17:37:37.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:37:37 smithi144 bash[66654]: cluster 2024-04-07T17:37:36.484214+0000 mgr.y (mgr.25175) 2467 : cluster [DBG] pgmap v4066: 137 pgs: 137 active+clean; 843 MiB data, 5.8 GiB used, 709 GiB / 715 GiB avail; 4.2 KiB/s rd, 1.3 KiB/s wr, 581 op/s 2024-04-07T17:37:37.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:37:37 smithi100 bash[74942]: cluster 2024-04-07T17:37:36.230983+0000 mon.a (mon.0) 3906 : cluster [DBG] osdmap e2024: 8 total, 8 up, 8 in 2024-04-07T17:37:37.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:37:37 smithi100 bash[74942]: cluster 2024-04-07T17:37:36.484214+0000 mgr.y (mgr.25175) 2467 : cluster [DBG] pgmap v4066: 137 pgs: 137 active+clean; 843 MiB data, 5.8 GiB used, 709 GiB / 715 GiB avail; 4.2 KiB/s rd, 1.3 KiB/s wr, 581 op/s 2024-04-07T17:37:37.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:37:37 smithi100 bash[76202]: cluster 2024-04-07T17:37:36.230983+0000 mon.a (mon.0) 3906 : cluster [DBG] osdmap e2024: 8 total, 8 up, 8 in 2024-04-07T17:37:37.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:37:37 smithi100 bash[76202]: cluster 2024-04-07T17:37:36.484214+0000 mgr.y (mgr.25175) 2467 : cluster [DBG] pgmap v4066: 137 pgs: 137 active+clean; 843 MiB data, 5.8 GiB used, 709 GiB / 715 GiB avail; 4.2 KiB/s rd, 1.3 KiB/s wr, 581 op/s 2024-04-07T17:37:38.258 INFO:tasks.workunit.client.0.smithi100.stdout:from 0 to HEAD diff [2329752~26214847,29778841~737427,31769017~13613313,45857929~20145827,66861224~16545045,85466334~5069656,92999623~10564968,106500403~13616197,121501610~12475984,136270708~22564048,160337176~7153146,168236274~8367316,180237750~12590714,194102759~7364680,202420372~5254601,210147535~53978881,267179572~28414772,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350406459~6863082,360049192~84791,366341667~7420413,374580519~17465318,392232408~14655911,408516200~2525554] 2024-04-07T17:37:38.258 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~26214847,29778841~737427,31769017~13613313,45857929~20145827,66861224~16545045,85466334~5069656,92999623~10564968,106500403~13616197,121501610~12475984,136270708~22564048,160337176~7153146,168236274~8367316,180237750~12590714,194102759~7364680,202420372~5254601,210147535~53978881,267179572~28414772,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350406459~6863082,360049192~84791,366341667~7420413,374580519~17465318,392232408~14655911,408516200~2525554] 2024-04-07T17:37:38.310 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~28544599,29360128~54046141,83886080~6649910,92274688~11289903,104857600~29119994,134217728~24617028,159383552~8106770,167772160~25056304,192937984~14736989,210147535~53978881,264241152~31353192,297795584~7201936,306184192~23819574,331350016~15772466,348127232~12006751,366341667~44700087] 2024-04-07T17:37:38.310 INFO:tasks.workunit.client.0.smithi100.stdout:from 1 to HEAD diff [2329752~26214847,29778841~737427,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~13616197,121501610~12475984,136270708~22564048,160337176~7153146,168236274~8367316,180237750~12590714,194866296~6601143,202420372~5254601,210147535~53978881,267179572~28414772,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350891716~1451525,355806290~1463251,360049192~84791,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:38.310 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~26214847,29778841~737427,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~13616197,121501610~12475984,136270708~22564048,160337176~7153146,168236274~8367316,180237750~12590714,194866296~6601143,202420372~5254601,210147535~53978881,267179572~28414772,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350891716~1451525,355806290~1463251,360049192~84791,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:38.320 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~28544599,29360128~24981239,55380210~10623546,66861224~16545045,92274688~11289903,104857600~29119994,134217728~24617028,159383552~8106770,167772160~25056304,194866296~12808677,210147535~53978881,264241152~31353192,297795584~7201936,306184192~23819574,331350016~15772466,350891716~1451525,355806290~4327693,366341667~25704170,394880591~2187459,398418332~12623422] 2024-04-07T17:37:38.321 INFO:tasks.workunit.client.0.smithi100.stdout:from 2 to HEAD diff [2329752~24230697,27329013~980763,29778841~737427,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~17685068,154799337~4035419,160337176~7153146,168236274~8367316,182034135~10794329,196727604~2108791,202420372~5254601,210147535~53978881,267179572~28414772,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350891716~1451525,355806290~1463251,360049192~84791,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:38.321 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~24230697,27329013~980763,29778841~737427,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~17685068,154799337~4035419,160337176~7153146,168236274~8367316,182034135~10794329,196727604~2108791,202420372~5254601,210147535~53978881,267179572~28414772,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350891716~1451525,355806290~1463251,360049192~84791,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:38.327 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~26560449,27329013~980763,29360128~24981239,55380210~10623546,66861224~16545045,92274688~11289903,106500403~5285199,112275487~21702107,134217728~19738048,154799337~4035419,159383552~8106770,167772160~8831430,182034135~10794329,196727604~2108791,201467439~6207534,210147535~53978881,264241152~31353192,297795584~7201936,306184192~23819574,331350016~15772466,350891716~1451525,355806290~4327693,366341667~25704170,394880591~2187459,398418332~12623422] 2024-04-07T17:37:38.327 INFO:tasks.workunit.client.0.smithi100.stdout:from 3 to HEAD diff [2329752~24230697,27329013~980763,29778841~737427,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~17685068,154799337~3192780,160337176~7153146,168236274~8367316,182034135~10303497,196727604~2108791,202420372~1435645,210147535~53978881,267179572~28414772,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350891716~1451525,355806290~1463251,360049192~84791,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:38.327 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~24230697,27329013~980763,29778841~737427,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~17685068,154799337~3192780,160337176~7153146,168236274~8367316,182034135~10303497,196727604~2108791,202420372~1435645,210147535~53978881,267179572~28414772,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350891716~1451525,355806290~1463251,360049192~84791,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:38.334 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~26560449,27329013~980763,29360128~24981239,55380210~10623546,66861224~16545045,92274688~11289903,106500403~5285199,112275487~21702107,134217728~19738048,154799337~3192780,159383552~8106770,167772160~8831430,182034135~10303497,196727604~2108791,202420372~1435645,210147535~53978881,264241152~31353192,297795584~7201936,306184192~23819574,331350016~15772466,350891716~1451525,355806290~4327693,366341667~25704170,394880591~2187459,398418332~12623422] 2024-04-07T17:37:38.334 INFO:tasks.workunit.client.0.smithi100.stdout:from 4 to HEAD diff [2329752~24230697,27329013~980763,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~12098856,149233764~4722012,154895619~3096498,160337176~7153146,168236274~8367316,182034135~10303497,196727604~2108791,202420372~1435645,210147535~53978881,267179572~13527794,281234113~14360231,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350891716~1451525,355806290~1463251,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:38.334 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~24230697,27329013~980763,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~12098856,149233764~4722012,154895619~3096498,160337176~7153146,168236274~8367316,182034135~10303497,196727604~2108791,202420372~1435645,210147535~53978881,267179572~13527794,281234113~14360231,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350891716~1451525,355806290~1463251,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:38.340 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~26560449,27329013~980763,30516268~23825099,55380210~10623546,66861224~16545045,92274688~11289903,106500403~5285199,112275487~21702107,134217728~14151836,149233764~4722012,154895619~3096498,159383552~8106770,167772160~8831430,182034135~10303497,196727604~2108791,202420372~1435645,210147535~53978881,264241152~16466214,281234113~14360231,297795584~7201936,306903613~23100153,331350016~15772466,350891716~1451525,355806290~1463251,366341667~25704170,394880591~2187459,398418332~12623422] 2024-04-07T17:37:38.340 INFO:tasks.workunit.client.0.smithi100.stdout:from 5 to HEAD diff [2329752~24230697,27329013~980763,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~12098856,149233764~4722012,154895619~3096498,166133159~1357163,168236274~8367316,182034135~10303497,196727604~2108791,202420372~1435645,210147535~38079381,248419159~15707257,267179572~13527794,281234113~14360231,298036890~6960630,306903613~5021411,314393585~13715786,332944411~11089603,345811290~1311192,355806290~1463251,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:38.341 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~24230697,27329013~980763,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~12098856,149233764~4722012,154895619~3096498,166133159~1357163,168236274~8367316,182034135~10303497,196727604~2108791,202420372~1435645,210147535~38079381,248419159~15707257,267179572~13527794,281234113~14360231,298036890~6960630,306903613~5021411,314393585~13715786,332944411~11089603,345811290~1311192,355806290~1463251,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:38.347 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~26560449,27329013~980763,30516268~23825099,55380210~10623546,66861224~16545045,92274688~11289903,106500403~5285199,112275487~21702107,134217728~14151836,149233764~4722012,154895619~3096498,166133159~1357163,167772160~8831430,182034135~10303497,196727604~2108791,202420372~1435645,210147535~38079381,248419159~15707257,264241152~16466214,281234113~14360231,298036890~6960630,306903613~21205758,331350016~15772466,355806290~1463251,366341667~7420413,374580519~17465318,394880591~2187459,398418332~12623422] 2024-04-07T17:37:38.347 INFO:tasks.workunit.client.0.smithi100.stdout:from 6 to HEAD diff [2329752~23418695,27329013~980763,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~12098856,149233764~4722012,154895619~3096498,168236274~8367316,182747004~9590628,202420372~1435645,210147535~38079381,250447165~8059589,267179572~11729329,281234113~14360231,298036890~6960630,306903613~5021411,314393585~13715786,332944411~6919544,345811290~1311192,355806290~1463251,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:38.348 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~23418695,27329013~980763,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~12098856,149233764~4722012,154895619~3096498,168236274~8367316,182747004~9590628,202420372~1435645,210147535~38079381,250447165~8059589,267179572~11729329,281234113~14360231,298036890~6960630,306903613~5021411,314393585~13715786,332944411~6919544,345811290~1311192,355806290~1463251,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:38.354 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~25748447,27329013~980763,30516268~23825099,55380210~10623546,66861224~16545045,92274688~11289903,106500403~5285199,112275487~21702107,134217728~14151836,149233764~4722012,154895619~3096498,167772160~8831430,182747004~9590628,202420372~1435645,210147535~38079381,250447165~8059589,264241152~14667749,281234113~14360231,298036890~6960630,306903613~21205758,331350016~8513939,344034014~3088468,355806290~1463251,366341667~7420413,374580519~17465318,394880591~2187459,398418332~12623422] 2024-04-07T17:37:38.354 INFO:tasks.workunit.client.0.smithi100.stdout:from 7 to HEAD diff [2329752~23418695,27329013~980763,31769017~13613313,45857929~8483438,55380210~10623546,68206575~15199694,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~5240179,141885014~6484550,149233764~4722012,154895619~3096498,168236274~8367316,182747004~9590628,210147535~38079381,250447165~8059589,267179572~11729329,281234113~14360231,298036890~6960630,306903613~5021411,314393585~13715786,332944411~6919544,355806290~1463251,366341667~7420413,377322524~14723313,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:38.354 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~23418695,27329013~980763,31769017~13613313,45857929~8483438,55380210~10623546,68206575~15199694,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~5240179,141885014~6484550,149233764~4722012,154895619~3096498,168236274~8367316,182747004~9590628,210147535~38079381,250447165~8059589,267179572~11729329,281234113~14360231,298036890~6960630,306903613~5021411,314393585~13715786,332944411~6919544,355806290~1463251,366341667~7420413,377322524~14723313,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:38.360 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~25748447,27329013~980763,30516268~23825099,55380210~10623546,68206575~15199694,92274688~11289903,106500403~5285199,112275487~21702107,134217728~7293159,141885014~6484550,149233764~4722012,154895619~3096498,167772160~8831430,182747004~9590628,210147535~38079381,250447165~8059589,264241152~14667749,281234113~14360231,298036890~6960630,306903613~21205758,331350016~8513939,355806290~1463251,366341667~7420413,377322524~14723313,394880591~2187459,398418332~12623422] 2024-04-07T17:37:38.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:37:38 smithi144 bash[66654]: cluster 2024-04-07T17:37:37.260925+0000 mon.a (mon.0) 3907 : cluster [DBG] osdmap e2025: 8 total, 8 up, 8 in 2024-04-07T17:37:38.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:37:38 smithi144 bash[66654]: cluster 2024-04-07T17:37:38.203906+0000 mon.a (mon.0) 3908 : cluster [DBG] osdmap e2026: 8 total, 8 up, 8 in 2024-04-07T17:37:38.704 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:37:38 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:37:38] "GET /metrics HTTP/1.1" 200 36771 "" "Prometheus/2.43.0" 2024-04-07T17:37:38.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:37:38 smithi100 bash[74942]: cluster 2024-04-07T17:37:37.260925+0000 mon.a (mon.0) 3907 : cluster [DBG] osdmap e2025: 8 total, 8 up, 8 in 2024-04-07T17:37:38.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:37:38 smithi100 bash[74942]: cluster 2024-04-07T17:37:38.203906+0000 mon.a (mon.0) 3908 : cluster [DBG] osdmap e2026: 8 total, 8 up, 8 in 2024-04-07T17:37:38.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:37:38 smithi100 bash[76202]: cluster 2024-04-07T17:37:37.260925+0000 mon.a (mon.0) 3907 : cluster [DBG] osdmap e2025: 8 total, 8 up, 8 in 2024-04-07T17:37:38.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:37:38 smithi100 bash[76202]: cluster 2024-04-07T17:37:38.203906+0000 mon.a (mon.0) 3908 : cluster [DBG] osdmap e2026: 8 total, 8 up, 8 in 2024-04-07T17:37:39.299 INFO:tasks.workunit.client.0.smithi100.stdout:from 0 to HEAD diff [2329752~26214847,29778841~737427,31769017~13613313,45857929~20145827,66861224~16545045,85466334~5069656,92999623~10564968,106500403~13616197,121501610~12475984,136270708~22564048,160337176~7153146,168236274~8367316,180237750~12590714,194102759~7364680,202420372~5254601,210147535~53978881,267179572~28414772,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350406459~6863082,360049192~84791,366341667~7420413,374580519~17465318,392232408~14655911,408516200~2525554] 2024-04-07T17:37:39.299 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~26214847,29778841~737427,31769017~13613313,45857929~20145827,66861224~16545045,85466334~5069656,92999623~10564968,106500403~13616197,121501610~12475984,136270708~22564048,160337176~7153146,168236274~8367316,180237750~12590714,194102759~7364680,202420372~5254601,210147535~53978881,267179572~28414772,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350406459~6863082,360049192~84791,366341667~7420413,374580519~17465318,392232408~14655911,408516200~2525554] 2024-04-07T17:37:39.363 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~28544599,29360128~54046141,83886080~6649910,92274688~11289903,104857600~29119994,134217728~24617028,159383552~8106770,167772160~25056304,192937984~14736989,210147535~53978881,264241152~31353192,297795584~7201936,306184192~23819574,331350016~15772466,348127232~12006751,366341667~44700087] 2024-04-07T17:37:39.363 INFO:tasks.workunit.client.0.smithi100.stdout:from 1 to HEAD diff [2329752~26214847,29778841~737427,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~13616197,121501610~12475984,136270708~22564048,160337176~7153146,168236274~8367316,180237750~12590714,194866296~6601143,202420372~5254601,210147535~53978881,267179572~28414772,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350891716~1451525,355806290~1463251,360049192~84791,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:39.363 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~26214847,29778841~737427,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~13616197,121501610~12475984,136270708~22564048,160337176~7153146,168236274~8367316,180237750~12590714,194866296~6601143,202420372~5254601,210147535~53978881,267179572~28414772,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350891716~1451525,355806290~1463251,360049192~84791,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:39.374 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~28544599,29360128~24981239,55380210~10623546,66861224~16545045,92274688~11289903,104857600~29119994,134217728~24617028,159383552~8106770,167772160~25056304,194866296~12808677,210147535~53978881,264241152~31353192,297795584~7201936,306184192~23819574,331350016~15772466,350891716~1451525,355806290~4327693,366341667~25704170,394880591~2187459,398418332~12623422] 2024-04-07T17:37:39.374 INFO:tasks.workunit.client.0.smithi100.stdout:from 2 to HEAD diff [2329752~24230697,27329013~980763,29778841~737427,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~17685068,154799337~4035419,160337176~7153146,168236274~8367316,182034135~10794329,196727604~2108791,202420372~5254601,210147535~53978881,267179572~28414772,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350891716~1451525,355806290~1463251,360049192~84791,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:39.374 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~24230697,27329013~980763,29778841~737427,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~17685068,154799337~4035419,160337176~7153146,168236274~8367316,182034135~10794329,196727604~2108791,202420372~5254601,210147535~53978881,267179572~28414772,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350891716~1451525,355806290~1463251,360049192~84791,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:39.381 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~26560449,27329013~980763,29360128~24981239,55380210~10623546,66861224~16545045,92274688~11289903,106500403~5285199,112275487~21702107,134217728~19738048,154799337~4035419,159383552~8106770,167772160~8831430,182034135~10794329,196727604~2108791,201467439~6207534,210147535~53978881,264241152~31353192,297795584~7201936,306184192~23819574,331350016~15772466,350891716~1451525,355806290~4327693,366341667~25704170,394880591~2187459,398418332~12623422] 2024-04-07T17:37:39.381 INFO:tasks.workunit.client.0.smithi100.stdout:from 3 to HEAD diff [2329752~24230697,27329013~980763,29778841~737427,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~17685068,154799337~3192780,160337176~7153146,168236274~8367316,182034135~10303497,196727604~2108791,202420372~1435645,210147535~53978881,267179572~28414772,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350891716~1451525,355806290~1463251,360049192~84791,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:39.381 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~24230697,27329013~980763,29778841~737427,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~17685068,154799337~3192780,160337176~7153146,168236274~8367316,182034135~10303497,196727604~2108791,202420372~1435645,210147535~53978881,267179572~28414772,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350891716~1451525,355806290~1463251,360049192~84791,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:39.387 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~26560449,27329013~980763,29360128~24981239,55380210~10623546,66861224~16545045,92274688~11289903,106500403~5285199,112275487~21702107,134217728~19738048,154799337~3192780,159383552~8106770,167772160~8831430,182034135~10303497,196727604~2108791,202420372~1435645,210147535~53978881,264241152~31353192,297795584~7201936,306184192~23819574,331350016~15772466,350891716~1451525,355806290~4327693,366341667~25704170,394880591~2187459,398418332~12623422] 2024-04-07T17:37:39.388 INFO:tasks.workunit.client.0.smithi100.stdout:from 4 to HEAD diff [2329752~24230697,27329013~980763,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~12098856,149233764~4722012,154895619~3096498,160337176~7153146,168236274~8367316,182034135~10303497,196727604~2108791,202420372~1435645,210147535~53978881,267179572~13527794,281234113~14360231,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350891716~1451525,355806290~1463251,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:39.388 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~24230697,27329013~980763,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~12098856,149233764~4722012,154895619~3096498,160337176~7153146,168236274~8367316,182034135~10303497,196727604~2108791,202420372~1435645,210147535~53978881,267179572~13527794,281234113~14360231,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350891716~1451525,355806290~1463251,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:39.395 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~26560449,27329013~980763,30516268~23825099,55380210~10623546,66861224~16545045,92274688~11289903,106500403~5285199,112275487~21702107,134217728~14151836,149233764~4722012,154895619~3096498,159383552~8106770,167772160~8831430,182034135~10303497,196727604~2108791,202420372~1435645,210147535~53978881,264241152~16466214,281234113~14360231,297795584~7201936,306903613~23100153,331350016~15772466,350891716~1451525,355806290~1463251,366341667~25704170,394880591~2187459,398418332~12623422] 2024-04-07T17:37:39.395 INFO:tasks.workunit.client.0.smithi100.stdout:from 5 to HEAD diff [2329752~24230697,27329013~980763,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~12098856,149233764~4722012,154895619~3096498,166133159~1357163,168236274~8367316,182034135~10303497,196727604~2108791,202420372~1435645,210147535~38079381,248419159~15707257,267179572~13527794,281234113~14360231,298036890~6960630,306903613~5021411,314393585~13715786,332944411~11089603,345811290~1311192,355806290~1463251,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:39.395 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~24230697,27329013~980763,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~12098856,149233764~4722012,154895619~3096498,166133159~1357163,168236274~8367316,182034135~10303497,196727604~2108791,202420372~1435645,210147535~38079381,248419159~15707257,267179572~13527794,281234113~14360231,298036890~6960630,306903613~5021411,314393585~13715786,332944411~11089603,345811290~1311192,355806290~1463251,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:39.402 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~26560449,27329013~980763,30516268~23825099,55380210~10623546,66861224~16545045,92274688~11289903,106500403~5285199,112275487~21702107,134217728~14151836,149233764~4722012,154895619~3096498,166133159~1357163,167772160~8831430,182034135~10303497,196727604~2108791,202420372~1435645,210147535~38079381,248419159~15707257,264241152~16466214,281234113~14360231,298036890~6960630,306903613~21205758,331350016~15772466,355806290~1463251,366341667~7420413,374580519~17465318,394880591~2187459,398418332~12623422] 2024-04-07T17:37:39.402 INFO:tasks.workunit.client.0.smithi100.stdout:from 6 to HEAD diff [2329752~23418695,27329013~980763,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~12098856,149233764~4722012,154895619~3096498,168236274~8367316,182747004~9590628,202420372~1435645,210147535~38079381,250447165~8059589,267179572~11729329,281234113~14360231,298036890~6960630,306903613~5021411,314393585~13715786,332944411~6919544,345811290~1311192,355806290~1463251,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:39.402 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~23418695,27329013~980763,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~12098856,149233764~4722012,154895619~3096498,168236274~8367316,182747004~9590628,202420372~1435645,210147535~38079381,250447165~8059589,267179572~11729329,281234113~14360231,298036890~6960630,306903613~5021411,314393585~13715786,332944411~6919544,345811290~1311192,355806290~1463251,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:39.409 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~25748447,27329013~980763,30516268~23825099,55380210~10623546,66861224~16545045,92274688~11289903,106500403~5285199,112275487~21702107,134217728~14151836,149233764~4722012,154895619~3096498,167772160~8831430,182747004~9590628,202420372~1435645,210147535~38079381,250447165~8059589,264241152~14667749,281234113~14360231,298036890~6960630,306903613~21205758,331350016~8513939,344034014~3088468,355806290~1463251,366341667~7420413,374580519~17465318,394880591~2187459,398418332~12623422] 2024-04-07T17:37:39.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:37:39 smithi144 bash[66654]: cluster 2024-04-07T17:37:38.484966+0000 mgr.y (mgr.25175) 2468 : cluster [DBG] pgmap v4069: 137 pgs: 137 active+clean; 766 MiB data, 5.6 GiB used, 710 GiB / 715 GiB avail; 116 KiB/s rd, 6.2 KiB/s wr, 1.31k op/s 2024-04-07T17:37:39.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:37:39 smithi100 bash[74942]: cluster 2024-04-07T17:37:38.484966+0000 mgr.y (mgr.25175) 2468 : cluster [DBG] pgmap v4069: 137 pgs: 137 active+clean; 766 MiB data, 5.6 GiB used, 710 GiB / 715 GiB avail; 116 KiB/s rd, 6.2 KiB/s wr, 1.31k op/s 2024-04-07T17:37:39.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:37:39 smithi100 bash[76202]: cluster 2024-04-07T17:37:38.484966+0000 mgr.y (mgr.25175) 2468 : cluster [DBG] pgmap v4069: 137 pgs: 137 active+clean; 766 MiB data, 5.6 GiB used, 710 GiB / 715 GiB avail; 116 KiB/s rd, 6.2 KiB/s wr, 1.31k op/s 2024-04-07T17:37:40.304 INFO:tasks.workunit.client.0.smithi100.stdout:from 0 to HEAD diff [2329752~26214847,29778841~737427,31769017~13613313,45857929~20145827,66861224~16545045,85466334~5069656,92999623~10564968,106500403~13616197,121501610~12475984,136270708~22564048,160337176~7153146,168236274~8367316,180237750~12590714,194102759~7364680,202420372~5254601,210147535~53978881,267179572~28414772,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350406459~6863082,360049192~84791,366341667~7420413,374580519~17465318,392232408~14655911,408516200~2525554] 2024-04-07T17:37:40.304 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~26214847,29778841~737427,31769017~13613313,45857929~20145827,66861224~16545045,85466334~5069656,92999623~10564968,106500403~13616197,121501610~12475984,136270708~22564048,160337176~7153146,168236274~8367316,180237750~12590714,194102759~7364680,202420372~5254601,210147535~53978881,267179572~28414772,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350406459~6863082,360049192~84791,366341667~7420413,374580519~17465318,392232408~14655911,408516200~2525554] 2024-04-07T17:37:40.364 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~28544599,29360128~54046141,83886080~6649910,92274688~11289903,104857600~29119994,134217728~24617028,159383552~8106770,167772160~25056304,192937984~14736989,210147535~53978881,264241152~31353192,297795584~7201936,306184192~23819574,331350016~15772466,348127232~12006751,366341667~44700087] 2024-04-07T17:37:40.365 INFO:tasks.workunit.client.0.smithi100.stdout:from 1 to HEAD diff [2329752~26214847,29778841~737427,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~13616197,121501610~12475984,136270708~22564048,160337176~7153146,168236274~8367316,180237750~12590714,194866296~6601143,202420372~5254601,210147535~53978881,267179572~28414772,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350891716~1451525,355806290~1463251,360049192~84791,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:40.365 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~26214847,29778841~737427,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~13616197,121501610~12475984,136270708~22564048,160337176~7153146,168236274~8367316,180237750~12590714,194866296~6601143,202420372~5254601,210147535~53978881,267179572~28414772,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350891716~1451525,355806290~1463251,360049192~84791,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:40.374 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~28544599,29360128~24981239,55380210~10623546,66861224~16545045,92274688~11289903,104857600~29119994,134217728~24617028,159383552~8106770,167772160~25056304,194866296~12808677,210147535~53978881,264241152~31353192,297795584~7201936,306184192~23819574,331350016~15772466,350891716~1451525,355806290~4327693,366341667~25704170,394880591~2187459,398418332~12623422] 2024-04-07T17:37:40.374 INFO:tasks.workunit.client.0.smithi100.stdout:from 2 to HEAD diff [2329752~24230697,27329013~980763,29778841~737427,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~17685068,154799337~4035419,160337176~7153146,168236274~8367316,182034135~10794329,196727604~2108791,202420372~5254601,210147535~53978881,267179572~28414772,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350891716~1451525,355806290~1463251,360049192~84791,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:40.374 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~24230697,27329013~980763,29778841~737427,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~17685068,154799337~4035419,160337176~7153146,168236274~8367316,182034135~10794329,196727604~2108791,202420372~5254601,210147535~53978881,267179572~28414772,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350891716~1451525,355806290~1463251,360049192~84791,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:40.381 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~26560449,27329013~980763,29360128~24981239,55380210~10623546,66861224~16545045,92274688~11289903,106500403~5285199,112275487~21702107,134217728~19738048,154799337~4035419,159383552~8106770,167772160~8831430,182034135~10794329,196727604~2108791,201467439~6207534,210147535~53978881,264241152~31353192,297795584~7201936,306184192~23819574,331350016~15772466,350891716~1451525,355806290~4327693,366341667~25704170,394880591~2187459,398418332~12623422] 2024-04-07T17:37:40.381 INFO:tasks.workunit.client.0.smithi100.stdout:from 3 to HEAD diff [2329752~24230697,27329013~980763,29778841~737427,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~17685068,154799337~3192780,160337176~7153146,168236274~8367316,182034135~10303497,196727604~2108791,202420372~1435645,210147535~53978881,267179572~28414772,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350891716~1451525,355806290~1463251,360049192~84791,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:40.381 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~24230697,27329013~980763,29778841~737427,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~17685068,154799337~3192780,160337176~7153146,168236274~8367316,182034135~10303497,196727604~2108791,202420372~1435645,210147535~53978881,267179572~28414772,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350891716~1451525,355806290~1463251,360049192~84791,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:40.388 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~26560449,27329013~980763,29360128~24981239,55380210~10623546,66861224~16545045,92274688~11289903,106500403~5285199,112275487~21702107,134217728~19738048,154799337~3192780,159383552~8106770,167772160~8831430,182034135~10303497,196727604~2108791,202420372~1435645,210147535~53978881,264241152~31353192,297795584~7201936,306184192~23819574,331350016~15772466,350891716~1451525,355806290~4327693,366341667~25704170,394880591~2187459,398418332~12623422] 2024-04-07T17:37:40.388 INFO:tasks.workunit.client.0.smithi100.stdout:from 4 to HEAD diff [2329752~24230697,27329013~980763,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~12098856,149233764~4722012,154895619~3096498,160337176~7153146,168236274~8367316,182034135~10303497,196727604~2108791,202420372~1435645,210147535~53978881,267179572~13527794,281234113~14360231,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350891716~1451525,355806290~1463251,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:40.388 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~24230697,27329013~980763,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~12098856,149233764~4722012,154895619~3096498,160337176~7153146,168236274~8367316,182034135~10303497,196727604~2108791,202420372~1435645,210147535~53978881,267179572~13527794,281234113~14360231,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350891716~1451525,355806290~1463251,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:40.394 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~26560449,27329013~980763,30516268~23825099,55380210~10623546,66861224~16545045,92274688~11289903,106500403~5285199,112275487~21702107,134217728~14151836,149233764~4722012,154895619~3096498,159383552~8106770,167772160~8831430,182034135~10303497,196727604~2108791,202420372~1435645,210147535~53978881,264241152~16466214,281234113~14360231,297795584~7201936,306903613~23100153,331350016~15772466,350891716~1451525,355806290~1463251,366341667~25704170,394880591~2187459,398418332~12623422] 2024-04-07T17:37:40.394 INFO:tasks.workunit.client.0.smithi100.stdout:from 5 to HEAD diff [2329752~24230697,27329013~980763,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~12098856,149233764~4722012,154895619~3096498,166133159~1357163,168236274~8367316,182034135~10303497,196727604~2108791,202420372~1435645,210147535~38079381,248419159~15707257,267179572~13527794,281234113~14360231,298036890~6960630,306903613~5021411,314393585~13715786,332944411~11089603,345811290~1311192,355806290~1463251,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:40.394 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~24230697,27329013~980763,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~12098856,149233764~4722012,154895619~3096498,166133159~1357163,168236274~8367316,182034135~10303497,196727604~2108791,202420372~1435645,210147535~38079381,248419159~15707257,267179572~13527794,281234113~14360231,298036890~6960630,306903613~5021411,314393585~13715786,332944411~11089603,345811290~1311192,355806290~1463251,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:40.400 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~26560449,27329013~980763,30516268~23825099,55380210~10623546,66861224~16545045,92274688~11289903,106500403~5285199,112275487~21702107,134217728~14151836,149233764~4722012,154895619~3096498,166133159~1357163,167772160~8831430,182034135~10303497,196727604~2108791,202420372~1435645,210147535~38079381,248419159~15707257,264241152~16466214,281234113~14360231,298036890~6960630,306903613~21205758,331350016~15772466,355806290~1463251,366341667~7420413,374580519~17465318,394880591~2187459,398418332~12623422] 2024-04-07T17:37:40.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:37:40 smithi144 bash[66654]: cluster 2024-04-07T17:37:39.266120+0000 mon.a (mon.0) 3909 : cluster [DBG] osdmap e2027: 8 total, 8 up, 8 in 2024-04-07T17:37:40.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:37:40 smithi100 bash[74942]: cluster 2024-04-07T17:37:39.266120+0000 mon.a (mon.0) 3909 : cluster [DBG] osdmap e2027: 8 total, 8 up, 8 in 2024-04-07T17:37:40.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:37:40 smithi100 bash[76202]: cluster 2024-04-07T17:37:39.266120+0000 mon.a (mon.0) 3909 : cluster [DBG] osdmap e2027: 8 total, 8 up, 8 in 2024-04-07T17:37:41.327 INFO:tasks.workunit.client.0.smithi100.stdout:from 0 to HEAD diff [2329752~26214847,29778841~737427,31769017~13613313,45857929~20145827,66861224~16545045,85466334~5069656,92999623~10564968,106500403~13616197,121501610~12475984,136270708~22564048,160337176~7153146,168236274~8367316,180237750~12590714,194102759~7364680,202420372~5254601,210147535~53978881,267179572~28414772,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350406459~6863082,360049192~84791,366341667~7420413,374580519~17465318,392232408~14655911,408516200~2525554] 2024-04-07T17:37:41.328 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~26214847,29778841~737427,31769017~13613313,45857929~20145827,66861224~16545045,85466334~5069656,92999623~10564968,106500403~13616197,121501610~12475984,136270708~22564048,160337176~7153146,168236274~8367316,180237750~12590714,194102759~7364680,202420372~5254601,210147535~53978881,267179572~28414772,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350406459~6863082,360049192~84791,366341667~7420413,374580519~17465318,392232408~14655911,408516200~2525554] 2024-04-07T17:37:41.396 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~28544599,29360128~54046141,83886080~6649910,92274688~11289903,104857600~29119994,134217728~24617028,159383552~8106770,167772160~25056304,192937984~14736989,210147535~53978881,264241152~31353192,297795584~7201936,306184192~23819574,331350016~15772466,348127232~12006751,366341667~44700087] 2024-04-07T17:37:41.396 INFO:tasks.workunit.client.0.smithi100.stdout:from 1 to HEAD diff [2329752~26214847,29778841~737427,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~13616197,121501610~12475984,136270708~22564048,160337176~7153146,168236274~8367316,180237750~12590714,194866296~6601143,202420372~5254601,210147535~53978881,267179572~28414772,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350891716~1451525,355806290~1463251,360049192~84791,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:41.396 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~26214847,29778841~737427,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~13616197,121501610~12475984,136270708~22564048,160337176~7153146,168236274~8367316,180237750~12590714,194866296~6601143,202420372~5254601,210147535~53978881,267179572~28414772,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350891716~1451525,355806290~1463251,360049192~84791,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:41.408 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~28544599,29360128~24981239,55380210~10623546,66861224~16545045,92274688~11289903,104857600~29119994,134217728~24617028,159383552~8106770,167772160~25056304,194866296~12808677,210147535~53978881,264241152~31353192,297795584~7201936,306184192~23819574,331350016~15772466,350891716~1451525,355806290~4327693,366341667~25704170,394880591~2187459,398418332~12623422] 2024-04-07T17:37:41.408 INFO:tasks.workunit.client.0.smithi100.stdout:from 2 to HEAD diff [2329752~24230697,27329013~980763,29778841~737427,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~17685068,154799337~4035419,160337176~7153146,168236274~8367316,182034135~10794329,196727604~2108791,202420372~5254601,210147535~53978881,267179572~28414772,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350891716~1451525,355806290~1463251,360049192~84791,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:41.408 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~24230697,27329013~980763,29778841~737427,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~17685068,154799337~4035419,160337176~7153146,168236274~8367316,182034135~10794329,196727604~2108791,202420372~5254601,210147535~53978881,267179572~28414772,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350891716~1451525,355806290~1463251,360049192~84791,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:41.415 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~26560449,27329013~980763,29360128~24981239,55380210~10623546,66861224~16545045,92274688~11289903,106500403~5285199,112275487~21702107,134217728~19738048,154799337~4035419,159383552~8106770,167772160~8831430,182034135~10794329,196727604~2108791,201467439~6207534,210147535~53978881,264241152~31353192,297795584~7201936,306184192~23819574,331350016~15772466,350891716~1451525,355806290~4327693,366341667~25704170,394880591~2187459,398418332~12623422] 2024-04-07T17:37:41.415 INFO:tasks.workunit.client.0.smithi100.stdout:from 3 to HEAD diff [2329752~24230697,27329013~980763,29778841~737427,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~17685068,154799337~3192780,160337176~7153146,168236274~8367316,182034135~10303497,196727604~2108791,202420372~1435645,210147535~53978881,267179572~28414772,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350891716~1451525,355806290~1463251,360049192~84791,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:41.415 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~24230697,27329013~980763,29778841~737427,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~17685068,154799337~3192780,160337176~7153146,168236274~8367316,182034135~10303497,196727604~2108791,202420372~1435645,210147535~53978881,267179572~28414772,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350891716~1451525,355806290~1463251,360049192~84791,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:41.422 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~26560449,27329013~980763,29360128~24981239,55380210~10623546,66861224~16545045,92274688~11289903,106500403~5285199,112275487~21702107,134217728~19738048,154799337~3192780,159383552~8106770,167772160~8831430,182034135~10303497,196727604~2108791,202420372~1435645,210147535~53978881,264241152~31353192,297795584~7201936,306184192~23819574,331350016~15772466,350891716~1451525,355806290~4327693,366341667~25704170,394880591~2187459,398418332~12623422] 2024-04-07T17:37:41.422 INFO:tasks.workunit.client.0.smithi100.stdout:from 4 to HEAD diff [2329752~24230697,27329013~980763,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~12098856,149233764~4722012,154895619~3096498,160337176~7153146,168236274~8367316,182034135~10303497,196727604~2108791,202420372~1435645,210147535~53978881,267179572~13527794,281234113~14360231,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350891716~1451525,355806290~1463251,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:41.422 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~24230697,27329013~980763,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~12098856,149233764~4722012,154895619~3096498,160337176~7153146,168236274~8367316,182034135~10303497,196727604~2108791,202420372~1435645,210147535~53978881,267179572~13527794,281234113~14360231,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350891716~1451525,355806290~1463251,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:41.428 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~26560449,27329013~980763,30516268~23825099,55380210~10623546,66861224~16545045,92274688~11289903,106500403~5285199,112275487~21702107,134217728~14151836,149233764~4722012,154895619~3096498,159383552~8106770,167772160~8831430,182034135~10303497,196727604~2108791,202420372~1435645,210147535~53978881,264241152~16466214,281234113~14360231,297795584~7201936,306903613~23100153,331350016~15772466,350891716~1451525,355806290~1463251,366341667~25704170,394880591~2187459,398418332~12623422] 2024-04-07T17:37:41.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:37:41 smithi144 bash[66654]: cluster 2024-04-07T17:37:40.279707+0000 mon.a (mon.0) 3910 : cluster [DBG] osdmap e2028: 8 total, 8 up, 8 in 2024-04-07T17:37:41.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:37:41 smithi144 bash[66654]: cluster 2024-04-07T17:37:40.485825+0000 mgr.y (mgr.25175) 2469 : cluster [DBG] pgmap v4072: 137 pgs: 137 active+clean; 721 MiB data, 5.4 GiB used, 710 GiB / 715 GiB avail; 118 KiB/s rd, 6.2 KiB/s wr, 1.52k op/s 2024-04-07T17:37:41.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:37:41 smithi100 bash[74942]: cluster 2024-04-07T17:37:40.279707+0000 mon.a (mon.0) 3910 : cluster [DBG] osdmap e2028: 8 total, 8 up, 8 in 2024-04-07T17:37:41.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:37:41 smithi100 bash[74942]: cluster 2024-04-07T17:37:40.485825+0000 mgr.y (mgr.25175) 2469 : cluster [DBG] pgmap v4072: 137 pgs: 137 active+clean; 721 MiB data, 5.4 GiB used, 710 GiB / 715 GiB avail; 118 KiB/s rd, 6.2 KiB/s wr, 1.52k op/s 2024-04-07T17:37:41.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:37:41 smithi100 bash[76202]: cluster 2024-04-07T17:37:40.279707+0000 mon.a (mon.0) 3910 : cluster [DBG] osdmap e2028: 8 total, 8 up, 8 in 2024-04-07T17:37:41.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:37:41 smithi100 bash[76202]: cluster 2024-04-07T17:37:40.485825+0000 mgr.y (mgr.25175) 2469 : cluster [DBG] pgmap v4072: 137 pgs: 137 active+clean; 721 MiB data, 5.4 GiB used, 710 GiB / 715 GiB avail; 118 KiB/s rd, 6.2 KiB/s wr, 1.52k op/s 2024-04-07T17:37:42.345 INFO:tasks.workunit.client.0.smithi100.stdout:from 0 to HEAD diff [2329752~26214847,29778841~737427,31769017~13613313,45857929~20145827,66861224~16545045,85466334~5069656,92999623~10564968,106500403~13616197,121501610~12475984,136270708~22564048,160337176~7153146,168236274~8367316,180237750~12590714,194102759~7364680,202420372~5254601,210147535~53978881,267179572~28414772,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350406459~6863082,360049192~84791,366341667~7420413,374580519~17465318,392232408~14655911,408516200~2525554] 2024-04-07T17:37:42.345 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~26214847,29778841~737427,31769017~13613313,45857929~20145827,66861224~16545045,85466334~5069656,92999623~10564968,106500403~13616197,121501610~12475984,136270708~22564048,160337176~7153146,168236274~8367316,180237750~12590714,194102759~7364680,202420372~5254601,210147535~53978881,267179572~28414772,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350406459~6863082,360049192~84791,366341667~7420413,374580519~17465318,392232408~14655911,408516200~2525554] 2024-04-07T17:37:42.409 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~28544599,29360128~54046141,83886080~6649910,92274688~11289903,104857600~29119994,134217728~24617028,159383552~8106770,167772160~25056304,192937984~14736989,210147535~53978881,264241152~31353192,297795584~7201936,306184192~23819574,331350016~15772466,348127232~12006751,366341667~44700087] 2024-04-07T17:37:42.409 INFO:tasks.workunit.client.0.smithi100.stdout:from 1 to HEAD diff [2329752~26214847,29778841~737427,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~13616197,121501610~12475984,136270708~22564048,160337176~7153146,168236274~8367316,180237750~12590714,194866296~6601143,202420372~5254601,210147535~53978881,267179572~28414772,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350891716~1451525,355806290~1463251,360049192~84791,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:42.409 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~26214847,29778841~737427,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~13616197,121501610~12475984,136270708~22564048,160337176~7153146,168236274~8367316,180237750~12590714,194866296~6601143,202420372~5254601,210147535~53978881,267179572~28414772,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350891716~1451525,355806290~1463251,360049192~84791,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:42.419 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~28544599,29360128~24981239,55380210~10623546,66861224~16545045,92274688~11289903,104857600~29119994,134217728~24617028,159383552~8106770,167772160~25056304,194866296~12808677,210147535~53978881,264241152~31353192,297795584~7201936,306184192~23819574,331350016~15772466,350891716~1451525,355806290~4327693,366341667~25704170,394880591~2187459,398418332~12623422] 2024-04-07T17:37:42.419 INFO:tasks.workunit.client.0.smithi100.stdout:from 2 to HEAD diff [2329752~24230697,27329013~980763,29778841~737427,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~17685068,154799337~4035419,160337176~7153146,168236274~8367316,182034135~10794329,196727604~2108791,202420372~5254601,210147535~53978881,267179572~28414772,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350891716~1451525,355806290~1463251,360049192~84791,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:42.419 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~24230697,27329013~980763,29778841~737427,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~17685068,154799337~4035419,160337176~7153146,168236274~8367316,182034135~10794329,196727604~2108791,202420372~5254601,210147535~53978881,267179572~28414772,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350891716~1451525,355806290~1463251,360049192~84791,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:42.427 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~26560449,27329013~980763,29360128~24981239,55380210~10623546,66861224~16545045,92274688~11289903,106500403~5285199,112275487~21702107,134217728~19738048,154799337~4035419,159383552~8106770,167772160~8831430,182034135~10794329,196727604~2108791,201467439~6207534,210147535~53978881,264241152~31353192,297795584~7201936,306184192~23819574,331350016~15772466,350891716~1451525,355806290~4327693,366341667~25704170,394880591~2187459,398418332~12623422] 2024-04-07T17:37:42.427 INFO:tasks.workunit.client.0.smithi100.stdout:from 3 to HEAD diff [2329752~24230697,27329013~980763,29778841~737427,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~17685068,154799337~3192780,160337176~7153146,168236274~8367316,182034135~10303497,196727604~2108791,202420372~1435645,210147535~53978881,267179572~28414772,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350891716~1451525,355806290~1463251,360049192~84791,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:42.427 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~24230697,27329013~980763,29778841~737427,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~17685068,154799337~3192780,160337176~7153146,168236274~8367316,182034135~10303497,196727604~2108791,202420372~1435645,210147535~53978881,267179572~28414772,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350891716~1451525,355806290~1463251,360049192~84791,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:42.434 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~26560449,27329013~980763,29360128~24981239,55380210~10623546,66861224~16545045,92274688~11289903,106500403~5285199,112275487~21702107,134217728~19738048,154799337~3192780,159383552~8106770,167772160~8831430,182034135~10303497,196727604~2108791,202420372~1435645,210147535~53978881,264241152~31353192,297795584~7201936,306184192~23819574,331350016~15772466,350891716~1451525,355806290~4327693,366341667~25704170,394880591~2187459,398418332~12623422] 2024-04-07T17:37:42.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:37:42 smithi144 bash[66654]: cluster 2024-04-07T17:37:41.302775+0000 mon.a (mon.0) 3911 : cluster [DBG] osdmap e2029: 8 total, 8 up, 8 in 2024-04-07T17:37:42.616 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:37:42 smithi144 bash[95233]: ts=2024-04-07T17:37:42.317Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:37:42.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:37:42 smithi100 bash[74942]: cluster 2024-04-07T17:37:41.302775+0000 mon.a (mon.0) 3911 : cluster [DBG] osdmap e2029: 8 total, 8 up, 8 in 2024-04-07T17:37:42.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:37:42 smithi100 bash[76202]: cluster 2024-04-07T17:37:41.302775+0000 mon.a (mon.0) 3911 : cluster [DBG] osdmap e2029: 8 total, 8 up, 8 in 2024-04-07T17:37:43.234 INFO:tasks.workunit.client.0.smithi100.stdout:from 0 to HEAD diff [2329752~26214847,29778841~737427,31769017~13613313,45857929~20145827,66861224~16545045,85466334~5069656,92999623~10564968,106500403~13616197,121501610~12475984,136270708~22564048,160337176~7153146,168236274~8367316,180237750~12590714,194102759~7364680,202420372~5254601,210147535~53978881,267179572~28414772,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350406459~6863082,360049192~84791,366341667~7420413,374580519~17465318,392232408~14655911,408516200~2525554] 2024-04-07T17:37:43.234 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~26214847,29778841~737427,31769017~13613313,45857929~20145827,66861224~16545045,85466334~5069656,92999623~10564968,106500403~13616197,121501610~12475984,136270708~22564048,160337176~7153146,168236274~8367316,180237750~12590714,194102759~7364680,202420372~5254601,210147535~53978881,267179572~28414772,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350406459~6863082,360049192~84791,366341667~7420413,374580519~17465318,392232408~14655911,408516200~2525554] 2024-04-07T17:37:43.311 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~28544599,29360128~54046141,83886080~6649910,92274688~11289903,104857600~29119994,134217728~24617028,159383552~8106770,167772160~25056304,192937984~14736989,210147535~53978881,264241152~31353192,297795584~7201936,306184192~23819574,331350016~15772466,348127232~12006751,366341667~44700087] 2024-04-07T17:37:43.311 INFO:tasks.workunit.client.0.smithi100.stdout:from 1 to HEAD diff [2329752~26214847,29778841~737427,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~13616197,121501610~12475984,136270708~22564048,160337176~7153146,168236274~8367316,180237750~12590714,194866296~6601143,202420372~5254601,210147535~53978881,267179572~28414772,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350891716~1451525,355806290~1463251,360049192~84791,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:43.311 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~26214847,29778841~737427,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~13616197,121501610~12475984,136270708~22564048,160337176~7153146,168236274~8367316,180237750~12590714,194866296~6601143,202420372~5254601,210147535~53978881,267179572~28414772,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350891716~1451525,355806290~1463251,360049192~84791,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:43.318 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~28544599,29360128~24981239,55380210~10623546,66861224~16545045,92274688~11289903,104857600~29119994,134217728~24617028,159383552~8106770,167772160~25056304,194866296~12808677,210147535~53978881,264241152~31353192,297795584~7201936,306184192~23819574,331350016~15772466,350891716~1451525,355806290~4327693,366341667~25704170,394880591~2187459,398418332~12623422] 2024-04-07T17:37:43.318 INFO:tasks.workunit.client.0.smithi100.stdout:from 2 to HEAD diff [2329752~24230697,27329013~980763,29778841~737427,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~17685068,154799337~4035419,160337176~7153146,168236274~8367316,182034135~10794329,196727604~2108791,202420372~5254601,210147535~53978881,267179572~28414772,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350891716~1451525,355806290~1463251,360049192~84791,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:43.318 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~24230697,27329013~980763,29778841~737427,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~5285199,112275487~7841113,121501610~12475984,136270708~17685068,154799337~4035419,160337176~7153146,168236274~8367316,182034135~10794329,196727604~2108791,202420372~5254601,210147535~53978881,267179572~28414772,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350891716~1451525,355806290~1463251,360049192~84791,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:43.326 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~26560449,27329013~980763,29360128~24981239,55380210~10623546,66861224~16545045,92274688~11289903,106500403~5285199,112275487~21702107,134217728~19738048,154799337~4035419,159383552~8106770,167772160~8831430,182034135~10794329,196727604~2108791,201467439~6207534,210147535~53978881,264241152~31353192,297795584~7201936,306184192~23819574,331350016~15772466,350891716~1451525,355806290~4327693,366341667~25704170,394880591~2187459,398418332~12623422] 2024-04-07T17:37:43.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:37:43 smithi144 bash[66654]: cluster 2024-04-07T17:37:42.306435+0000 mon.a (mon.0) 3912 : cluster [DBG] osdmap e2030: 8 total, 8 up, 8 in 2024-04-07T17:37:43.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:37:43 smithi144 bash[66654]: cluster 2024-04-07T17:37:42.486620+0000 mgr.y (mgr.25175) 2470 : cluster [DBG] pgmap v4075: 137 pgs: 137 active+clean; 633 MiB data, 5.2 GiB used, 710 GiB / 715 GiB avail; 84 KiB/s rd, 6.2 KiB/s wr, 1.07k op/s 2024-04-07T17:37:43.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:37:43 smithi144 bash[66654]: cluster 2024-04-07T17:37:43.206181+0000 mon.a (mon.0) 3913 : cluster [DBG] osdmap e2031: 8 total, 8 up, 8 in 2024-04-07T17:37:43.705 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:37:43 smithi100 bash[74942]: cluster 2024-04-07T17:37:42.306435+0000 mon.a (mon.0) 3912 : cluster [DBG] osdmap e2030: 8 total, 8 up, 8 in 2024-04-07T17:37:43.705 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:37:43 smithi100 bash[74942]: cluster 2024-04-07T17:37:42.486620+0000 mgr.y (mgr.25175) 2470 : cluster [DBG] pgmap v4075: 137 pgs: 137 active+clean; 633 MiB data, 5.2 GiB used, 710 GiB / 715 GiB avail; 84 KiB/s rd, 6.2 KiB/s wr, 1.07k op/s 2024-04-07T17:37:43.706 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:37:43 smithi100 bash[74942]: cluster 2024-04-07T17:37:43.206181+0000 mon.a (mon.0) 3913 : cluster [DBG] osdmap e2031: 8 total, 8 up, 8 in 2024-04-07T17:37:43.706 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:37:43 smithi100 bash[76202]: cluster 2024-04-07T17:37:42.306435+0000 mon.a (mon.0) 3912 : cluster [DBG] osdmap e2030: 8 total, 8 up, 8 in 2024-04-07T17:37:43.706 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:37:43 smithi100 bash[76202]: cluster 2024-04-07T17:37:42.486620+0000 mgr.y (mgr.25175) 2470 : cluster [DBG] pgmap v4075: 137 pgs: 137 active+clean; 633 MiB data, 5.2 GiB used, 710 GiB / 715 GiB avail; 84 KiB/s rd, 6.2 KiB/s wr, 1.07k op/s 2024-04-07T17:37:43.706 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:37:43 smithi100 bash[76202]: cluster 2024-04-07T17:37:43.206181+0000 mon.a (mon.0) 3913 : cluster [DBG] osdmap e2031: 8 total, 8 up, 8 in 2024-04-07T17:37:44.374 INFO:tasks.workunit.client.0.smithi100.stdout:from 0 to HEAD diff [2329752~26214847,29778841~737427,31769017~13613313,45857929~20145827,66861224~16545045,85466334~5069656,92999623~10564968,106500403~13616197,121501610~12475984,136270708~22564048,160337176~7153146,168236274~8367316,180237750~12590714,194102759~7364680,202420372~5254601,210147535~53978881,267179572~28414772,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350406459~6863082,360049192~84791,366341667~7420413,374580519~17465318,392232408~14655911,408516200~2525554] 2024-04-07T17:37:44.375 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~26214847,29778841~737427,31769017~13613313,45857929~20145827,66861224~16545045,85466334~5069656,92999623~10564968,106500403~13616197,121501610~12475984,136270708~22564048,160337176~7153146,168236274~8367316,180237750~12590714,194102759~7364680,202420372~5254601,210147535~53978881,267179572~28414772,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350406459~6863082,360049192~84791,366341667~7420413,374580519~17465318,392232408~14655911,408516200~2525554] 2024-04-07T17:37:44.423 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~28544599,29360128~54046141,83886080~6649910,92274688~11289903,104857600~29119994,134217728~24617028,159383552~8106770,167772160~25056304,192937984~14736989,210147535~53978881,264241152~31353192,297795584~7201936,306184192~23819574,331350016~15772466,348127232~12006751,366341667~44700087] 2024-04-07T17:37:44.423 INFO:tasks.workunit.client.0.smithi100.stdout:from 1 to HEAD diff [2329752~26214847,29778841~737427,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~13616197,121501610~12475984,136270708~22564048,160337176~7153146,168236274~8367316,180237750~12590714,194866296~6601143,202420372~5254601,210147535~53978881,267179572~28414772,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350891716~1451525,355806290~1463251,360049192~84791,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:44.423 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~26214847,29778841~737427,31769017~13613313,45857929~8483438,55380210~10623546,66861224~16545045,92999623~10564968,106500403~13616197,121501610~12475984,136270708~22564048,160337176~7153146,168236274~8367316,180237750~12590714,194866296~6601143,202420372~5254601,210147535~53978881,267179572~28414772,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350891716~1451525,355806290~1463251,360049192~84791,366341667~7420413,374580519~17465318,394880591~2187459,398418332~8469987,408516200~2525554] 2024-04-07T17:37:44.431 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~28544599,29360128~24981239,55380210~10623546,66861224~16545045,92274688~11289903,104857600~29119994,134217728~24617028,159383552~8106770,167772160~25056304,194866296~12808677,210147535~53978881,264241152~31353192,297795584~7201936,306184192~23819574,331350016~15772466,350891716~1451525,355806290~4327693,366341667~25704170,394880591~2187459,398418332~12623422] 2024-04-07T17:37:45.403 INFO:tasks.workunit.client.0.smithi100.stdout:from 0 to HEAD diff [2329752~26214847,29778841~737427,31769017~13613313,45857929~20145827,66861224~16545045,85466334~5069656,92999623~10564968,106500403~13616197,121501610~12475984,136270708~22564048,160337176~7153146,168236274~8367316,180237750~12590714,194102759~7364680,202420372~5254601,210147535~53978881,267179572~28414772,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350406459~6863082,360049192~84791,366341667~7420413,374580519~17465318,392232408~14655911,408516200~2525554] 2024-04-07T17:37:45.403 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [2329752~26214847,29778841~737427,31769017~13613313,45857929~20145827,66861224~16545045,85466334~5069656,92999623~10564968,106500403~13616197,121501610~12475984,136270708~22564048,160337176~7153146,168236274~8367316,180237750~12590714,194102759~7364680,202420372~5254601,210147535~53978881,267179572~28414772,298036890~6960630,306903613~5021411,314393585~15610181,332944411~11089603,345811290~1311192,350406459~6863082,360049192~84791,366341667~7420413,374580519~17465318,392232408~14655911,408516200~2525554] 2024-04-07T17:37:45.437 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~28544599,29360128~54046141,83886080~6649910,92274688~11289903,104857600~29119994,134217728~24617028,159383552~8106770,167772160~25056304,192937984~14736989,210147535~53978881,264241152~31353192,297795584~7201936,306184192~23819574,331350016~15772466,348127232~12006751,366341667~44700087] 2024-04-07T17:37:45.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:37:45 smithi144 bash[66654]: cluster 2024-04-07T17:37:44.336653+0000 mon.a (mon.0) 3914 : cluster [DBG] osdmap e2032: 8 total, 8 up, 8 in 2024-04-07T17:37:45.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:37:45 smithi144 bash[66654]: cluster 2024-04-07T17:37:44.487396+0000 mgr.y (mgr.25175) 2471 : cluster [DBG] pgmap v4078: 137 pgs: 137 active+clean; 598 MiB data, 5.1 GiB used, 710 GiB / 715 GiB avail; 81 KiB/s rd, 5.0 KiB/s wr, 798 op/s 2024-04-07T17:37:45.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:37:45 smithi100 bash[74942]: cluster 2024-04-07T17:37:44.336653+0000 mon.a (mon.0) 3914 : cluster [DBG] osdmap e2032: 8 total, 8 up, 8 in 2024-04-07T17:37:45.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:37:45 smithi100 bash[74942]: cluster 2024-04-07T17:37:44.487396+0000 mgr.y (mgr.25175) 2471 : cluster [DBG] pgmap v4078: 137 pgs: 137 active+clean; 598 MiB data, 5.1 GiB used, 710 GiB / 715 GiB avail; 81 KiB/s rd, 5.0 KiB/s wr, 798 op/s 2024-04-07T17:37:45.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:37:45 smithi100 bash[76202]: cluster 2024-04-07T17:37:44.336653+0000 mon.a (mon.0) 3914 : cluster [DBG] osdmap e2032: 8 total, 8 up, 8 in 2024-04-07T17:37:45.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:37:45 smithi100 bash[76202]: cluster 2024-04-07T17:37:44.487396+0000 mgr.y (mgr.25175) 2471 : cluster [DBG] pgmap v4078: 137 pgs: 137 active+clean; 598 MiB data, 5.1 GiB used, 710 GiB / 715 GiB avail; 81 KiB/s rd, 5.0 KiB/s wr, 798 op/s 2024-04-07T17:37:46.423 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] DiffIterateTest/0.DiffIterateStress (43036 ms) 2024-04-07T17:37:46.423 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] DiffIterateTest/0.DiffIterateParent 2024-04-07T17:37:46.423 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:37:46.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:37:46 smithi144 bash[66654]: cluster 2024-04-07T17:37:45.346344+0000 mon.a (mon.0) 3915 : cluster [DBG] osdmap e2033: 8 total, 8 up, 8 in 2024-04-07T17:37:46.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:37:46 smithi100 bash[74942]: cluster 2024-04-07T17:37:45.346344+0000 mon.a (mon.0) 3915 : cluster [DBG] osdmap e2033: 8 total, 8 up, 8 in 2024-04-07T17:37:46.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:37:46 smithi100 bash[76202]: cluster 2024-04-07T17:37:45.346344+0000 mon.a (mon.0) 3915 : cluster [DBG] osdmap e2033: 8 total, 8 up, 8 in 2024-04-07T17:37:47.644 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] DiffIterateTest/0.DiffIterateParent (1221 ms) 2024-04-07T17:37:47.644 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] DiffIterateTest/0.DiffIterateIgnoreParent 2024-04-07T17:37:47.644 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:37:47.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:37:47 smithi100 bash[74942]: cluster 2024-04-07T17:37:46.351229+0000 mon.a (mon.0) 3916 : cluster [DBG] osdmap e2034: 8 total, 8 up, 8 in 2024-04-07T17:37:47.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:37:47 smithi100 bash[74942]: cluster 2024-04-07T17:37:46.487965+0000 mgr.y (mgr.25175) 2472 : cluster [DBG] pgmap v4081: 137 pgs: 137 active+clean; 579 MiB data, 5.1 GiB used, 710 GiB / 715 GiB avail; 4.0 KiB/s rd, 767 B/s wr, 211 op/s 2024-04-07T17:37:47.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:37:47 smithi100 bash[74942]: audit 2024-04-07T17:37:46.858204+0000 mon.a (mon.0) 3917 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:37:47.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:37:47 smithi100 bash[76202]: cluster 2024-04-07T17:37:46.351229+0000 mon.a (mon.0) 3916 : cluster [DBG] osdmap e2034: 8 total, 8 up, 8 in 2024-04-07T17:37:47.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:37:47 smithi100 bash[76202]: cluster 2024-04-07T17:37:46.487965+0000 mgr.y (mgr.25175) 2472 : cluster [DBG] pgmap v4081: 137 pgs: 137 active+clean; 579 MiB data, 5.1 GiB used, 710 GiB / 715 GiB avail; 4.0 KiB/s rd, 767 B/s wr, 211 op/s 2024-04-07T17:37:47.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:37:47 smithi100 bash[76202]: audit 2024-04-07T17:37:46.858204+0000 mon.a (mon.0) 3917 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:37:47.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:37:47 smithi144 bash[66654]: cluster 2024-04-07T17:37:46.351229+0000 mon.a (mon.0) 3916 : cluster [DBG] osdmap e2034: 8 total, 8 up, 8 in 2024-04-07T17:37:47.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:37:47 smithi144 bash[66654]: cluster 2024-04-07T17:37:46.487965+0000 mgr.y (mgr.25175) 2472 : cluster [DBG] pgmap v4081: 137 pgs: 137 active+clean; 579 MiB data, 5.1 GiB used, 710 GiB / 715 GiB avail; 4.0 KiB/s rd, 767 B/s wr, 211 op/s 2024-04-07T17:37:47.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:37:47 smithi144 bash[66654]: audit 2024-04-07T17:37:46.858204+0000 mon.a (mon.0) 3917 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:37:48.466 INFO:tasks.workunit.client.0.smithi100.stdout: wrote [1001079~9400,1115485~90674,1825454~84382,5131849~9734,6022048~61412,8456112~15783,11758780~69767,12268018~54340,12554367~64623,13283117~28774] to clone 2024-04-07T17:37:48.467 INFO:tasks.workunit.client.0.smithi100.stdout: diff was [0~1909836,4194304~1889156,8388608~4923283] 2024-04-07T17:37:48.478 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] DiffIterateTest/0.DiffIterateIgnoreParent (834 ms) 2024-04-07T17:37:48.478 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] DiffIterateTest/0.DiffIterateCallbackError 2024-04-07T17:37:48.478 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:37:48.536 INFO:tasks.workunit.client.0.smithi100.stdout: wrote [767437~41024,4624207~28375,5134305~75372,5421059~16853,7478385~74178,7785081~65807,10826478~50853,11652895~13597,11721871~21625,12178383~68850] 2024-04-07T17:37:48.563 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] DiffIterateTest/0.DiffIterateCallbackError (85 ms) 2024-04-07T17:37:48.563 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] DiffIterateTest/0.DiffIterateParentDiscard 2024-04-07T17:37:48.563 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:37:48.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:37:48 smithi100 bash[74942]: cluster 2024-04-07T17:37:47.373632+0000 mon.a (mon.0) 3918 : cluster [DBG] osdmap e2035: 8 total, 8 up, 8 in 2024-04-07T17:37:48.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:37:48 smithi100 bash[74942]: cluster 2024-04-07T17:37:48.207286+0000 mon.a (mon.0) 3919 : cluster [DBG] osdmap e2036: 8 total, 8 up, 8 in 2024-04-07T17:37:48.704 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:37:48 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:37:48] "GET /metrics HTTP/1.1" 200 36754 "" "Prometheus/2.43.0" 2024-04-07T17:37:48.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:37:48 smithi100 bash[76202]: cluster 2024-04-07T17:37:47.373632+0000 mon.a (mon.0) 3918 : cluster [DBG] osdmap e2035: 8 total, 8 up, 8 in 2024-04-07T17:37:48.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:37:48 smithi100 bash[76202]: cluster 2024-04-07T17:37:48.207286+0000 mon.a (mon.0) 3919 : cluster [DBG] osdmap e2036: 8 total, 8 up, 8 in 2024-04-07T17:37:48.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:37:48 smithi144 bash[66654]: cluster 2024-04-07T17:37:47.373632+0000 mon.a (mon.0) 3918 : cluster [DBG] osdmap e2035: 8 total, 8 up, 8 in 2024-04-07T17:37:48.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:37:48 smithi144 bash[66654]: cluster 2024-04-07T17:37:48.207286+0000 mon.a (mon.0) 3919 : cluster [DBG] osdmap e2036: 8 total, 8 up, 8 in 2024-04-07T17:37:49.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:37:49 smithi100 bash[74942]: cluster 2024-04-07T17:37:48.488678+0000 mgr.y (mgr.25175) 2473 : cluster [DBG] pgmap v4084: 137 pgs: 137 active+clean; 491 MiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 64 KiB/s rd, 4.0 MiB/s wr, 313 op/s 2024-04-07T17:37:49.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:37:49 smithi100 bash[76202]: cluster 2024-04-07T17:37:48.488678+0000 mgr.y (mgr.25175) 2473 : cluster [DBG] pgmap v4084: 137 pgs: 137 active+clean; 491 MiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 64 KiB/s rd, 4.0 MiB/s wr, 313 op/s 2024-04-07T17:37:49.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:37:49 smithi144 bash[66654]: cluster 2024-04-07T17:37:48.488678+0000 mgr.y (mgr.25175) 2473 : cluster [DBG] pgmap v4084: 137 pgs: 137 active+clean; 491 MiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 64 KiB/s rd, 4.0 MiB/s wr, 313 op/s 2024-04-07T17:37:50.658 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] DiffIterateTest/0.DiffIterateParentDiscard (2095 ms) 2024-04-07T17:37:50.658 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] DiffIterateTest/0.DiffIterateUnalignedSmall 2024-04-07T17:37:50.658 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:37:50.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:37:50 smithi100 bash[74942]: cluster 2024-04-07T17:37:49.385832+0000 mon.a (mon.0) 3920 : cluster [DBG] osdmap e2037: 8 total, 8 up, 8 in 2024-04-07T17:37:50.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:37:50 smithi100 bash[76202]: cluster 2024-04-07T17:37:49.385832+0000 mon.a (mon.0) 3920 : cluster [DBG] osdmap e2037: 8 total, 8 up, 8 in 2024-04-07T17:37:50.773 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] DiffIterateTest/0.DiffIterateUnalignedSmall (115 ms) 2024-04-07T17:37:50.773 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] DiffIterateTest/0.DiffIterateUnaligned 2024-04-07T17:37:50.773 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:37:50.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:37:50 smithi144 bash[66654]: cluster 2024-04-07T17:37:49.385832+0000 mon.a (mon.0) 3920 : cluster [DBG] osdmap e2037: 8 total, 8 up, 8 in 2024-04-07T17:37:50.901 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] DiffIterateTest/0.DiffIterateUnaligned (128 ms) 2024-04-07T17:37:50.902 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] DiffIterateTest/0.DiffIterateTryAcquireLock 2024-04-07T17:37:50.902 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:37:51.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:37:51 smithi100 bash[76202]: cluster 2024-04-07T17:37:50.395251+0000 mon.a (mon.0) 3921 : cluster [DBG] osdmap e2038: 8 total, 8 up, 8 in 2024-04-07T17:37:51.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:37:51 smithi100 bash[76202]: cluster 2024-04-07T17:37:50.489461+0000 mgr.y (mgr.25175) 2474 : cluster [DBG] pgmap v4087: 137 pgs: 137 active+clean; 485 MiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 4.1 MiB/s rd, 9.2 MiB/s wr, 356 op/s 2024-04-07T17:37:51.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:37:51 smithi100 bash[74942]: cluster 2024-04-07T17:37:50.395251+0000 mon.a (mon.0) 3921 : cluster [DBG] osdmap e2038: 8 total, 8 up, 8 in 2024-04-07T17:37:51.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:37:51 smithi100 bash[74942]: cluster 2024-04-07T17:37:50.489461+0000 mgr.y (mgr.25175) 2474 : cluster [DBG] pgmap v4087: 137 pgs: 137 active+clean; 485 MiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 4.1 MiB/s rd, 9.2 MiB/s wr, 356 op/s 2024-04-07T17:37:51.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:37:51 smithi144 bash[66654]: cluster 2024-04-07T17:37:50.395251+0000 mon.a (mon.0) 3921 : cluster [DBG] osdmap e2038: 8 total, 8 up, 8 in 2024-04-07T17:37:51.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:37:51 smithi144 bash[66654]: cluster 2024-04-07T17:37:50.489461+0000 mgr.y (mgr.25175) 2474 : cluster [DBG] pgmap v4087: 137 pgs: 137 active+clean; 485 MiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 4.1 MiB/s rd, 9.2 MiB/s wr, 356 op/s 2024-04-07T17:37:52.615 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:37:52 smithi144 bash[95233]: ts=2024-04-07T17:37:52.317Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:37:53.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:37:53 smithi144 bash[66654]: cluster 2024-04-07T17:37:52.490667+0000 mgr.y (mgr.25175) 2475 : cluster [DBG] pgmap v4088: 137 pgs: 137 active+clean; 546 MiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 4.1 MiB/s rd, 10 MiB/s wr, 284 op/s 2024-04-07T17:37:53.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:37:53 smithi144 bash[66654]: cluster 2024-04-07T17:37:53.216067+0000 mon.a (mon.0) 3922 : cluster [DBG] osdmap e2039: 8 total, 8 up, 8 in 2024-04-07T17:37:53.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:37:53 smithi100 bash[74942]: cluster 2024-04-07T17:37:52.490667+0000 mgr.y (mgr.25175) 2475 : cluster [DBG] pgmap v4088: 137 pgs: 137 active+clean; 546 MiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 4.1 MiB/s rd, 10 MiB/s wr, 284 op/s 2024-04-07T17:37:53.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:37:53 smithi100 bash[74942]: cluster 2024-04-07T17:37:53.216067+0000 mon.a (mon.0) 3922 : cluster [DBG] osdmap e2039: 8 total, 8 up, 8 in 2024-04-07T17:37:53.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:37:53 smithi100 bash[76202]: cluster 2024-04-07T17:37:52.490667+0000 mgr.y (mgr.25175) 2475 : cluster [DBG] pgmap v4088: 137 pgs: 137 active+clean; 546 MiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 4.1 MiB/s rd, 10 MiB/s wr, 284 op/s 2024-04-07T17:37:53.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:37:53 smithi100 bash[76202]: cluster 2024-04-07T17:37:53.216067+0000 mon.a (mon.0) 3922 : cluster [DBG] osdmap e2039: 8 total, 8 up, 8 in 2024-04-07T17:37:55.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:37:55 smithi144 bash[66654]: cluster 2024-04-07T17:37:54.491789+0000 mgr.y (mgr.25175) 2476 : cluster [DBG] pgmap v4090: 137 pgs: 137 active+clean; 554 MiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 3.6 MiB/s rd, 9.6 MiB/s wr, 264 op/s 2024-04-07T17:37:55.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:37:55 smithi100 bash[74942]: cluster 2024-04-07T17:37:54.491789+0000 mgr.y (mgr.25175) 2476 : cluster [DBG] pgmap v4090: 137 pgs: 137 active+clean; 554 MiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 3.6 MiB/s rd, 9.6 MiB/s wr, 264 op/s 2024-04-07T17:37:55.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:37:55 smithi100 bash[76202]: cluster 2024-04-07T17:37:54.491789+0000 mgr.y (mgr.25175) 2476 : cluster [DBG] pgmap v4090: 137 pgs: 137 active+clean; 554 MiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 3.6 MiB/s rd, 9.6 MiB/s wr, 264 op/s 2024-04-07T17:37:56.003 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] DiffIterateTest/0.DiffIterateTryAcquireLock (5101 ms) 2024-04-07T17:37:56.003 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] DiffIterateTest/0.DiffIterateStriping 2024-04-07T17:37:56.003 INFO:tasks.workunit.client.0.smithi100.stdout:./src/test/librbd/test_librbd.cc:8604: Skipped 2024-04-07T17:37:56.003 INFO:tasks.workunit.client.0.smithi100.stdout:Skipping due to unmet REQUIRE 2024-04-07T17:37:56.003 INFO:tasks.workunit.client.0.smithi100.stdout:[ SKIPPED ] DiffIterateTest/0.DiffIterateStriping (0 ms) 2024-04-07T17:37:56.572 INFO:tasks.workunit.client.0.smithi100.stdout:[----------] 15 tests from DiffIterateTest/0 (72367 ms total) 2024-04-07T17:37:56.572 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:37:56.572 INFO:tasks.workunit.client.0.smithi100.stdout:[----------] 15 tests from DiffIterateTest/1, where TypeParam = DiffIterateParams 2024-04-07T17:37:56.676 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] DiffIterateTest/1.DiffIterate 2024-04-07T17:37:57.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:37:57 smithi144 bash[66654]: cluster 2024-04-07T17:37:56.492721+0000 mgr.y (mgr.25175) 2477 : cluster [DBG] pgmap v4091: 137 pgs: 137 active+clean; 554 MiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 1.9 MiB/s rd, 6.4 MiB/s wr, 172 op/s 2024-04-07T17:37:57.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:37:57 smithi144 bash[66654]: cluster 2024-04-07T17:37:56.559602+0000 mon.a (mon.0) 3923 : cluster [DBG] osdmap e2040: 8 total, 8 up, 8 in 2024-04-07T17:37:57.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:37:57 smithi100 bash[74942]: cluster 2024-04-07T17:37:56.492721+0000 mgr.y (mgr.25175) 2477 : cluster [DBG] pgmap v4091: 137 pgs: 137 active+clean; 554 MiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 1.9 MiB/s rd, 6.4 MiB/s wr, 172 op/s 2024-04-07T17:37:57.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:37:57 smithi100 bash[74942]: cluster 2024-04-07T17:37:56.559602+0000 mon.a (mon.0) 3923 : cluster [DBG] osdmap e2040: 8 total, 8 up, 8 in 2024-04-07T17:37:57.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:37:57 smithi100 bash[76202]: cluster 2024-04-07T17:37:56.492721+0000 mgr.y (mgr.25175) 2477 : cluster [DBG] pgmap v4091: 137 pgs: 137 active+clean; 554 MiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 1.9 MiB/s rd, 6.4 MiB/s wr, 172 op/s 2024-04-07T17:37:57.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:37:57 smithi100 bash[76202]: cluster 2024-04-07T17:37:56.559602+0000 mon.a (mon.0) 3923 : cluster [DBG] osdmap e2040: 8 total, 8 up, 8 in 2024-04-07T17:37:58.616 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:37:58.704 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:37:58 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:37:58] "GET /metrics HTTP/1.1" 200 36754 "" "Prometheus/2.43.0" 2024-04-07T17:37:58.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:37:58 smithi100 bash[74942]: cluster 2024-04-07T17:37:57.580264+0000 mon.a (mon.0) 3924 : cluster [DBG] osdmap e2041: 8 total, 8 up, 8 in 2024-04-07T17:37:58.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:37:58 smithi100 bash[74942]: audit 2024-04-07T17:37:57.593610+0000 mon.c (mon.2) 100 : audit [INF] from='client.? 172.21.15.100:0/1632297930' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:37:58.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:37:58 smithi100 bash[74942]: audit 2024-04-07T17:37:57.594359+0000 mon.a (mon.0) 3925 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:37:58.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:37:58 smithi100 bash[76202]: cluster 2024-04-07T17:37:57.580264+0000 mon.a (mon.0) 3924 : cluster [DBG] osdmap e2041: 8 total, 8 up, 8 in 2024-04-07T17:37:58.705 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:37:58 smithi100 bash[76202]: audit 2024-04-07T17:37:57.593610+0000 mon.c (mon.2) 100 : audit [INF] from='client.? 172.21.15.100:0/1632297930' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:37:58.705 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:37:58 smithi100 bash[76202]: audit 2024-04-07T17:37:57.594359+0000 mon.a (mon.0) 3925 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:37:58.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:37:58 smithi144 bash[66654]: cluster 2024-04-07T17:37:57.580264+0000 mon.a (mon.0) 3924 : cluster [DBG] osdmap e2041: 8 total, 8 up, 8 in 2024-04-07T17:37:58.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:37:58 smithi144 bash[66654]: audit 2024-04-07T17:37:57.593610+0000 mon.c (mon.2) 100 : audit [INF] from='client.? 172.21.15.100:0/1632297930' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:37:58.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:37:58 smithi144 bash[66654]: audit 2024-04-07T17:37:57.594359+0000 mon.a (mon.0) 3925 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:37:59.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:37:59 smithi144 bash[66654]: cluster 2024-04-07T17:37:58.493616+0000 mgr.y (mgr.25175) 2478 : cluster [DBG] pgmap v4094: 137 pgs: 4 creating+peering, 28 unknown, 105 active+clean; 4.6 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail 2024-04-07T17:37:59.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:37:59 smithi144 bash[66654]: cluster 2024-04-07T17:37:58.573126+0000 mon.a (mon.0) 3926 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T17:37:59.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:37:59 smithi144 bash[66654]: audit 2024-04-07T17:37:58.583814+0000 mon.a (mon.0) 3927 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-45","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T17:37:59.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:37:59 smithi144 bash[66654]: cluster 2024-04-07T17:37:58.596363+0000 mon.a (mon.0) 3928 : cluster [DBG] osdmap e2042: 8 total, 8 up, 8 in 2024-04-07T17:37:59.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:37:59 smithi100 bash[74942]: cluster 2024-04-07T17:37:58.493616+0000 mgr.y (mgr.25175) 2478 : cluster [DBG] pgmap v4094: 137 pgs: 4 creating+peering, 28 unknown, 105 active+clean; 4.6 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail 2024-04-07T17:37:59.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:37:59 smithi100 bash[74942]: cluster 2024-04-07T17:37:58.573126+0000 mon.a (mon.0) 3926 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T17:37:59.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:37:59 smithi100 bash[74942]: audit 2024-04-07T17:37:58.583814+0000 mon.a (mon.0) 3927 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-45","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T17:37:59.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:37:59 smithi100 bash[74942]: cluster 2024-04-07T17:37:58.596363+0000 mon.a (mon.0) 3928 : cluster [DBG] osdmap e2042: 8 total, 8 up, 8 in 2024-04-07T17:37:59.955 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:37:59 smithi100 bash[76202]: cluster 2024-04-07T17:37:58.493616+0000 mgr.y (mgr.25175) 2478 : cluster [DBG] pgmap v4094: 137 pgs: 4 creating+peering, 28 unknown, 105 active+clean; 4.6 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail 2024-04-07T17:37:59.955 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:37:59 smithi100 bash[76202]: cluster 2024-04-07T17:37:58.573126+0000 mon.a (mon.0) 3926 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T17:37:59.955 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:37:59 smithi100 bash[76202]: audit 2024-04-07T17:37:58.583814+0000 mon.a (mon.0) 3927 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-librbd-smithi100-115303-45","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-07T17:37:59.955 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:37:59 smithi100 bash[76202]: cluster 2024-04-07T17:37:58.596363+0000 mon.a (mon.0) 3928 : cluster [DBG] osdmap e2042: 8 total, 8 up, 8 in 2024-04-07T17:38:00.717 INFO:tasks.workunit.client.0.smithi100.stdout: wrote [4637142~16868,5589428~29052,8435950~22699,11396600~70477,11491570~23134,12631615~90332,13553845~54658,15448340~38811,17695556~52334,19951964~97180] 2024-04-07T17:38:00.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:38:00 smithi100 bash[74942]: cluster 2024-04-07T17:37:59.595710+0000 mon.a (mon.0) 3929 : cluster [DBG] osdmap e2043: 8 total, 8 up, 8 in 2024-04-07T17:38:00.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:38:00 smithi100 bash[76202]: cluster 2024-04-07T17:37:59.595710+0000 mon.a (mon.0) 3929 : cluster [DBG] osdmap e2043: 8 total, 8 up, 8 in 2024-04-07T17:38:01.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:38:00 smithi144 bash[66654]: cluster 2024-04-07T17:37:59.595710+0000 mon.a (mon.0) 3929 : cluster [DBG] osdmap e2043: 8 total, 8 up, 8 in 2024-04-07T17:38:01.678 INFO:tasks.workunit.client.0.smithi100.stdout: wrote [4194304~16777216] 2024-04-07T17:38:01.689 INFO:tasks.workunit.client.0.smithi100.stdout: diff was [4194304~16777216] 2024-04-07T17:38:01.696 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] DiffIterateTest/1.DiffIterate (5020 ms) 2024-04-07T17:38:01.696 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] DiffIterateTest/1.DiffIterateDeterministic 2024-04-07T17:38:01.697 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:38:01.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:38:01 smithi100 bash[74942]: cluster 2024-04-07T17:38:00.494476+0000 mgr.y (mgr.25175) 2479 : cluster [DBG] pgmap v4097: 137 pgs: 4 creating+peering, 11 unknown, 122 active+clean; 4.6 MiB data, 3.7 GiB used, 712 GiB / 715 GiB avail 2024-04-07T17:38:01.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:38:01 smithi100 bash[74942]: cluster 2024-04-07T17:38:00.599994+0000 mon.a (mon.0) 3930 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T17:38:01.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:38:01 smithi100 bash[74942]: cluster 2024-04-07T17:38:00.624390+0000 mon.a (mon.0) 3931 : cluster [DBG] osdmap e2044: 8 total, 8 up, 8 in 2024-04-07T17:38:01.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:38:01 smithi100 bash[76202]: cluster 2024-04-07T17:38:00.494476+0000 mgr.y (mgr.25175) 2479 : cluster [DBG] pgmap v4097: 137 pgs: 4 creating+peering, 11 unknown, 122 active+clean; 4.6 MiB data, 3.7 GiB used, 712 GiB / 715 GiB avail 2024-04-07T17:38:01.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:38:01 smithi100 bash[76202]: cluster 2024-04-07T17:38:00.599994+0000 mon.a (mon.0) 3930 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T17:38:01.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:38:01 smithi100 bash[76202]: cluster 2024-04-07T17:38:00.624390+0000 mon.a (mon.0) 3931 : cluster [DBG] osdmap e2044: 8 total, 8 up, 8 in 2024-04-07T17:38:02.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:38:01 smithi144 bash[66654]: cluster 2024-04-07T17:38:00.494476+0000 mgr.y (mgr.25175) 2479 : cluster [DBG] pgmap v4097: 137 pgs: 4 creating+peering, 11 unknown, 122 active+clean; 4.6 MiB data, 3.7 GiB used, 712 GiB / 715 GiB avail 2024-04-07T17:38:02.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:38:01 smithi144 bash[66654]: cluster 2024-04-07T17:38:00.599994+0000 mon.a (mon.0) 3930 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T17:38:02.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:38:01 smithi144 bash[66654]: cluster 2024-04-07T17:38:00.624390+0000 mon.a (mon.0) 3931 : cluster [DBG] osdmap e2044: 8 total, 8 up, 8 in 2024-04-07T17:38:02.615 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:38:02 smithi144 bash[95233]: ts=2024-04-07T17:38:02.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:38:02.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:38:02 smithi100 bash[74942]: cluster 2024-04-07T17:38:01.627443+0000 mon.a (mon.0) 3932 : cluster [DBG] osdmap e2045: 8 total, 8 up, 8 in 2024-04-07T17:38:02.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:38:02 smithi100 bash[74942]: audit 2024-04-07T17:38:01.858695+0000 mon.a (mon.0) 3933 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:38:02.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:38:02 smithi100 bash[76202]: cluster 2024-04-07T17:38:01.627443+0000 mon.a (mon.0) 3932 : cluster [DBG] osdmap e2045: 8 total, 8 up, 8 in 2024-04-07T17:38:02.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:38:02 smithi100 bash[76202]: audit 2024-04-07T17:38:01.858695+0000 mon.a (mon.0) 3933 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:38:03.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:38:02 smithi144 bash[66654]: cluster 2024-04-07T17:38:01.627443+0000 mon.a (mon.0) 3932 : cluster [DBG] osdmap e2045: 8 total, 8 up, 8 in 2024-04-07T17:38:03.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:38:02 smithi144 bash[66654]: audit 2024-04-07T17:38:01.858695+0000 mon.a (mon.0) 3933 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:38:03.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:38:03 smithi100 bash[74942]: cluster 2024-04-07T17:38:02.495743+0000 mgr.y (mgr.25175) 2480 : cluster [DBG] pgmap v4100: 137 pgs: 4 creating+peering, 133 active+clean; 4.6 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 5.5 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-07T17:38:03.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:38:03 smithi100 bash[74942]: cluster 2024-04-07T17:38:02.655308+0000 mon.a (mon.0) 3934 : cluster [DBG] osdmap e2046: 8 total, 8 up, 8 in 2024-04-07T17:38:03.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:38:03 smithi100 bash[76202]: cluster 2024-04-07T17:38:02.495743+0000 mgr.y (mgr.25175) 2480 : cluster [DBG] pgmap v4100: 137 pgs: 4 creating+peering, 133 active+clean; 4.6 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 5.5 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-07T17:38:03.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:38:03 smithi100 bash[76202]: cluster 2024-04-07T17:38:02.655308+0000 mon.a (mon.0) 3934 : cluster [DBG] osdmap e2046: 8 total, 8 up, 8 in 2024-04-07T17:38:04.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:38:03 smithi144 bash[66654]: cluster 2024-04-07T17:38:02.495743+0000 mgr.y (mgr.25175) 2480 : cluster [DBG] pgmap v4100: 137 pgs: 4 creating+peering, 133 active+clean; 4.6 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 5.5 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-07T17:38:04.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:38:03 smithi144 bash[66654]: cluster 2024-04-07T17:38:02.655308+0000 mon.a (mon.0) 3934 : cluster [DBG] osdmap e2046: 8 total, 8 up, 8 in 2024-04-07T17:38:04.754 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] DiffIterateTest/1.DiffIterateDeterministic (3058 ms) 2024-04-07T17:38:04.754 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] DiffIterateTest/1.DiffIterateDeterministicPP 2024-04-07T17:38:04.754 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:38:04.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:38:04 smithi100 bash[74942]: cluster 2024-04-07T17:38:03.655209+0000 mon.a (mon.0) 3935 : cluster [DBG] osdmap e2047: 8 total, 8 up, 8 in 2024-04-07T17:38:04.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:38:04 smithi100 bash[76202]: cluster 2024-04-07T17:38:03.655209+0000 mon.a (mon.0) 3935 : cluster [DBG] osdmap e2047: 8 total, 8 up, 8 in 2024-04-07T17:38:05.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:38:04 smithi144 bash[66654]: cluster 2024-04-07T17:38:03.655209+0000 mon.a (mon.0) 3935 : cluster [DBG] osdmap e2047: 8 total, 8 up, 8 in 2024-04-07T17:38:05.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:38:05 smithi100 bash[74942]: cluster 2024-04-07T17:38:04.496771+0000 mgr.y (mgr.25175) 2481 : cluster [DBG] pgmap v4103: 137 pgs: 137 active+clean; 15 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 20 KiB/s rd, 185 KiB/s wr, 37 op/s 2024-04-07T17:38:05.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:38:05 smithi100 bash[74942]: cluster 2024-04-07T17:38:04.684496+0000 mon.a (mon.0) 3936 : cluster [DBG] osdmap e2048: 8 total, 8 up, 8 in 2024-04-07T17:38:05.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:38:05 smithi100 bash[76202]: cluster 2024-04-07T17:38:04.496771+0000 mgr.y (mgr.25175) 2481 : cluster [DBG] pgmap v4103: 137 pgs: 137 active+clean; 15 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 20 KiB/s rd, 185 KiB/s wr, 37 op/s 2024-04-07T17:38:05.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:38:05 smithi100 bash[76202]: cluster 2024-04-07T17:38:04.684496+0000 mon.a (mon.0) 3936 : cluster [DBG] osdmap e2048: 8 total, 8 up, 8 in 2024-04-07T17:38:06.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:38:05 smithi144 bash[66654]: cluster 2024-04-07T17:38:04.496771+0000 mgr.y (mgr.25175) 2481 : cluster [DBG] pgmap v4103: 137 pgs: 137 active+clean; 15 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 20 KiB/s rd, 185 KiB/s wr, 37 op/s 2024-04-07T17:38:06.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:38:05 smithi144 bash[66654]: cluster 2024-04-07T17:38:04.684496+0000 mon.a (mon.0) 3936 : cluster [DBG] osdmap e2048: 8 total, 8 up, 8 in 2024-04-07T17:38:06.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:38:06 smithi100 bash[74942]: cluster 2024-04-07T17:38:05.684130+0000 mon.a (mon.0) 3937 : cluster [DBG] osdmap e2049: 8 total, 8 up, 8 in 2024-04-07T17:38:06.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:38:06 smithi100 bash[76202]: cluster 2024-04-07T17:38:05.684130+0000 mon.a (mon.0) 3937 : cluster [DBG] osdmap e2049: 8 total, 8 up, 8 in 2024-04-07T17:38:07.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:38:06 smithi144 bash[66654]: cluster 2024-04-07T17:38:05.684130+0000 mon.a (mon.0) 3937 : cluster [DBG] osdmap e2049: 8 total, 8 up, 8 in 2024-04-07T17:38:07.787 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] DiffIterateTest/1.DiffIterateDeterministicPP (3033 ms) 2024-04-07T17:38:07.787 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] DiffIterateTest/1.DiffIterateDiscard 2024-04-07T17:38:07.787 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:38:07.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:38:07 smithi100 bash[74942]: cluster 2024-04-07T17:38:06.497496+0000 mgr.y (mgr.25175) 2482 : cluster [DBG] pgmap v4106: 137 pgs: 137 active+clean; 18 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 40 KiB/s rd, 264 KiB/s wr, 69 op/s 2024-04-07T17:38:07.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:38:07 smithi100 bash[74942]: cluster 2024-04-07T17:38:06.705213+0000 mon.a (mon.0) 3938 : cluster [DBG] osdmap e2050: 8 total, 8 up, 8 in 2024-04-07T17:38:07.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:38:07 smithi100 bash[76202]: cluster 2024-04-07T17:38:06.497496+0000 mgr.y (mgr.25175) 2482 : cluster [DBG] pgmap v4106: 137 pgs: 137 active+clean; 18 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 40 KiB/s rd, 264 KiB/s wr, 69 op/s 2024-04-07T17:38:07.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:38:07 smithi100 bash[76202]: cluster 2024-04-07T17:38:06.705213+0000 mon.a (mon.0) 3938 : cluster [DBG] osdmap e2050: 8 total, 8 up, 8 in 2024-04-07T17:38:08.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:38:07 smithi144 bash[66654]: cluster 2024-04-07T17:38:06.497496+0000 mgr.y (mgr.25175) 2482 : cluster [DBG] pgmap v4106: 137 pgs: 137 active+clean; 18 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 40 KiB/s rd, 264 KiB/s wr, 69 op/s 2024-04-07T17:38:08.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:38:07 smithi144 bash[66654]: cluster 2024-04-07T17:38:06.705213+0000 mon.a (mon.0) 3938 : cluster [DBG] osdmap e2050: 8 total, 8 up, 8 in 2024-04-07T17:38:08.704 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:38:08 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:38:08] "GET /metrics HTTP/1.1" 200 36715 "" "Prometheus/2.43.0" 2024-04-07T17:38:09.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:38:08 smithi144 bash[66654]: cluster 2024-04-07T17:38:07.712700+0000 mon.a (mon.0) 3939 : cluster [DBG] osdmap e2051: 8 total, 8 up, 8 in 2024-04-07T17:38:09.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:38:08 smithi144 bash[66654]: cluster 2024-04-07T17:38:08.217368+0000 mon.a (mon.0) 3940 : cluster [DBG] osdmap e2052: 8 total, 8 up, 8 in 2024-04-07T17:38:09.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:38:08 smithi100 bash[74942]: cluster 2024-04-07T17:38:07.712700+0000 mon.a (mon.0) 3939 : cluster [DBG] osdmap e2051: 8 total, 8 up, 8 in 2024-04-07T17:38:09.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:38:08 smithi100 bash[74942]: cluster 2024-04-07T17:38:08.217368+0000 mon.a (mon.0) 3940 : cluster [DBG] osdmap e2052: 8 total, 8 up, 8 in 2024-04-07T17:38:09.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:38:08 smithi100 bash[76202]: cluster 2024-04-07T17:38:07.712700+0000 mon.a (mon.0) 3939 : cluster [DBG] osdmap e2051: 8 total, 8 up, 8 in 2024-04-07T17:38:09.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:38:08 smithi100 bash[76202]: cluster 2024-04-07T17:38:08.217368+0000 mon.a (mon.0) 3940 : cluster [DBG] osdmap e2052: 8 total, 8 up, 8 in 2024-04-07T17:38:10.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:38:09 smithi144 bash[66654]: cluster 2024-04-07T17:38:08.498245+0000 mgr.y (mgr.25175) 2483 : cluster [DBG] pgmap v4110: 137 pgs: 137 active+clean; 18 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 64 KiB/s rd, 87 KiB/s wr, 96 op/s 2024-04-07T17:38:10.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:38:09 smithi144 bash[66654]: cluster 2024-04-07T17:38:09.226159+0000 mon.a (mon.0) 3941 : cluster [DBG] osdmap e2053: 8 total, 8 up, 8 in 2024-04-07T17:38:10.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:38:09 smithi100 bash[74942]: cluster 2024-04-07T17:38:08.498245+0000 mgr.y (mgr.25175) 2483 : cluster [DBG] pgmap v4110: 137 pgs: 137 active+clean; 18 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 64 KiB/s rd, 87 KiB/s wr, 96 op/s 2024-04-07T17:38:10.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:38:09 smithi100 bash[74942]: cluster 2024-04-07T17:38:09.226159+0000 mon.a (mon.0) 3941 : cluster [DBG] osdmap e2053: 8 total, 8 up, 8 in 2024-04-07T17:38:10.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:38:09 smithi100 bash[76202]: cluster 2024-04-07T17:38:08.498245+0000 mgr.y (mgr.25175) 2483 : cluster [DBG] pgmap v4110: 137 pgs: 137 active+clean; 18 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 64 KiB/s rd, 87 KiB/s wr, 96 op/s 2024-04-07T17:38:10.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:38:09 smithi100 bash[76202]: cluster 2024-04-07T17:38:09.226159+0000 mon.a (mon.0) 3941 : cluster [DBG] osdmap e2053: 8 total, 8 up, 8 in 2024-04-07T17:38:10.287 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] DiffIterateTest/1.DiffIterateDiscard (2500 ms) 2024-04-07T17:38:10.287 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] DiffIterateTest/1.DiffIterateTruncate 2024-04-07T17:38:10.287 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:38:11.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:38:11 smithi144 bash[66654]: cluster 2024-04-07T17:38:10.230645+0000 mon.a (mon.0) 3942 : cluster [DBG] osdmap e2054: 8 total, 8 up, 8 in 2024-04-07T17:38:11.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:38:11 smithi144 bash[66654]: cluster 2024-04-07T17:38:10.498965+0000 mgr.y (mgr.25175) 2484 : cluster [DBG] pgmap v4113: 137 pgs: 137 active+clean; 18 MiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 78 KiB/s rd, 13 KiB/s wr, 114 op/s 2024-04-07T17:38:11.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:38:11 smithi100 bash[74942]: cluster 2024-04-07T17:38:10.230645+0000 mon.a (mon.0) 3942 : cluster [DBG] osdmap e2054: 8 total, 8 up, 8 in 2024-04-07T17:38:11.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:38:11 smithi100 bash[74942]: cluster 2024-04-07T17:38:10.498965+0000 mgr.y (mgr.25175) 2484 : cluster [DBG] pgmap v4113: 137 pgs: 137 active+clean; 18 MiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 78 KiB/s rd, 13 KiB/s wr, 114 op/s 2024-04-07T17:38:11.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:38:11 smithi100 bash[76202]: cluster 2024-04-07T17:38:10.230645+0000 mon.a (mon.0) 3942 : cluster [DBG] osdmap e2054: 8 total, 8 up, 8 in 2024-04-07T17:38:11.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:38:11 smithi100 bash[76202]: cluster 2024-04-07T17:38:10.498965+0000 mgr.y (mgr.25175) 2484 : cluster [DBG] pgmap v4113: 137 pgs: 137 active+clean; 18 MiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 78 KiB/s rd, 13 KiB/s wr, 114 op/s 2024-04-07T17:38:12.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:38:12 smithi144 bash[66654]: cluster 2024-04-07T17:38:11.242001+0000 mon.a (mon.0) 3943 : cluster [DBG] osdmap e2055: 8 total, 8 up, 8 in 2024-04-07T17:38:12.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:38:12 smithi144 bash[66654]: audit 2024-04-07T17:38:12.171910+0000 mon.a (mon.0) 3944 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:38:12.616 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:38:12 smithi144 bash[95233]: ts=2024-04-07T17:38:12.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:38:12.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:38:12 smithi100 bash[74942]: cluster 2024-04-07T17:38:11.242001+0000 mon.a (mon.0) 3943 : cluster [DBG] osdmap e2055: 8 total, 8 up, 8 in 2024-04-07T17:38:12.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:38:12 smithi100 bash[74942]: audit 2024-04-07T17:38:12.171910+0000 mon.a (mon.0) 3944 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:38:12.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:38:12 smithi100 bash[76202]: cluster 2024-04-07T17:38:11.242001+0000 mon.a (mon.0) 3943 : cluster [DBG] osdmap e2055: 8 total, 8 up, 8 in 2024-04-07T17:38:12.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:38:12 smithi100 bash[76202]: audit 2024-04-07T17:38:12.171910+0000 mon.a (mon.0) 3944 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:38:13.378 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] DiffIterateTest/1.DiffIterateTruncate (3091 ms) 2024-04-07T17:38:13.378 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] DiffIterateTest/1.DiffIterateWriteAndTruncate 2024-04-07T17:38:13.378 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:38:13.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:38:13 smithi144 bash[66654]: cluster 2024-04-07T17:38:12.243910+0000 mon.a (mon.0) 3945 : cluster [DBG] osdmap e2056: 8 total, 8 up, 8 in 2024-04-07T17:38:13.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:38:13 smithi144 bash[66654]: cluster 2024-04-07T17:38:12.499663+0000 mgr.y (mgr.25175) 2485 : cluster [DBG] pgmap v4116: 137 pgs: 137 active+clean; 18 MiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 43 KiB/s rd, 137 KiB/s wr, 66 op/s 2024-04-07T17:38:13.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:38:13 smithi144 bash[66654]: audit 2024-04-07T17:38:12.962655+0000 mon.a (mon.0) 3946 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:38:13.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:38:13 smithi144 bash[66654]: audit 2024-04-07T17:38:12.964599+0000 mon.a (mon.0) 3947 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:38:13.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:38:13 smithi144 bash[66654]: audit 2024-04-07T17:38:12.975407+0000 mon.a (mon.0) 3948 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:38:13.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:38:13 smithi100 bash[74942]: cluster 2024-04-07T17:38:12.243910+0000 mon.a (mon.0) 3945 : cluster [DBG] osdmap e2056: 8 total, 8 up, 8 in 2024-04-07T17:38:13.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:38:13 smithi100 bash[74942]: cluster 2024-04-07T17:38:12.499663+0000 mgr.y (mgr.25175) 2485 : cluster [DBG] pgmap v4116: 137 pgs: 137 active+clean; 18 MiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 43 KiB/s rd, 137 KiB/s wr, 66 op/s 2024-04-07T17:38:13.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:38:13 smithi100 bash[74942]: audit 2024-04-07T17:38:12.962655+0000 mon.a (mon.0) 3946 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:38:13.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:38:13 smithi100 bash[74942]: audit 2024-04-07T17:38:12.964599+0000 mon.a (mon.0) 3947 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:38:13.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:38:13 smithi100 bash[74942]: audit 2024-04-07T17:38:12.975407+0000 mon.a (mon.0) 3948 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:38:13.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:38:13 smithi100 bash[76202]: cluster 2024-04-07T17:38:12.243910+0000 mon.a (mon.0) 3945 : cluster [DBG] osdmap e2056: 8 total, 8 up, 8 in 2024-04-07T17:38:13.705 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:38:13 smithi100 bash[76202]: cluster 2024-04-07T17:38:12.499663+0000 mgr.y (mgr.25175) 2485 : cluster [DBG] pgmap v4116: 137 pgs: 137 active+clean; 18 MiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 43 KiB/s rd, 137 KiB/s wr, 66 op/s 2024-04-07T17:38:13.705 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:38:13 smithi100 bash[76202]: audit 2024-04-07T17:38:12.962655+0000 mon.a (mon.0) 3946 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:38:13.705 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:38:13 smithi100 bash[76202]: audit 2024-04-07T17:38:12.964599+0000 mon.a (mon.0) 3947 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:38:13.705 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:38:13 smithi100 bash[76202]: audit 2024-04-07T17:38:12.975407+0000 mon.a (mon.0) 3948 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:38:14.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:38:14 smithi144 bash[66654]: cluster 2024-04-07T17:38:13.266867+0000 mon.a (mon.0) 3949 : cluster [DBG] osdmap e2057: 8 total, 8 up, 8 in 2024-04-07T17:38:14.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:38:14 smithi100 bash[74942]: cluster 2024-04-07T17:38:13.266867+0000 mon.a (mon.0) 3949 : cluster [DBG] osdmap e2057: 8 total, 8 up, 8 in 2024-04-07T17:38:14.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:38:14 smithi100 bash[76202]: cluster 2024-04-07T17:38:13.266867+0000 mon.a (mon.0) 3949 : cluster [DBG] osdmap e2057: 8 total, 8 up, 8 in 2024-04-07T17:38:15.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:38:15 smithi144 bash[66654]: cluster 2024-04-07T17:38:14.279872+0000 mon.a (mon.0) 3950 : cluster [DBG] osdmap e2058: 8 total, 8 up, 8 in 2024-04-07T17:38:15.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:38:15 smithi144 bash[66654]: cluster 2024-04-07T17:38:14.500355+0000 mgr.y (mgr.25175) 2486 : cluster [DBG] pgmap v4119: 137 pgs: 137 active+clean; 18 MiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 67 KiB/s rd, 135 KiB/s wr, 95 op/s 2024-04-07T17:38:15.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:38:15 smithi100 bash[74942]: cluster 2024-04-07T17:38:14.279872+0000 mon.a (mon.0) 3950 : cluster [DBG] osdmap e2058: 8 total, 8 up, 8 in 2024-04-07T17:38:15.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:38:15 smithi100 bash[74942]: cluster 2024-04-07T17:38:14.500355+0000 mgr.y (mgr.25175) 2486 : cluster [DBG] pgmap v4119: 137 pgs: 137 active+clean; 18 MiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 67 KiB/s rd, 135 KiB/s wr, 95 op/s 2024-04-07T17:38:15.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:38:15 smithi100 bash[76202]: cluster 2024-04-07T17:38:14.279872+0000 mon.a (mon.0) 3950 : cluster [DBG] osdmap e2058: 8 total, 8 up, 8 in 2024-04-07T17:38:15.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:38:15 smithi100 bash[76202]: cluster 2024-04-07T17:38:14.500355+0000 mgr.y (mgr.25175) 2486 : cluster [DBG] pgmap v4119: 137 pgs: 137 active+clean; 18 MiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 67 KiB/s rd, 135 KiB/s wr, 95 op/s 2024-04-07T17:38:16.396 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] DiffIterateTest/1.DiffIterateWriteAndTruncate (3018 ms) 2024-04-07T17:38:16.397 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] DiffIterateTest/1.DiffIterateStress 2024-04-07T17:38:16.397 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:38:16.534 INFO:tasks.workunit.client.0.smithi100.stdout: i=0 exists [33781672~6108467,53857509~1127803,63382715~8163650,130769017~6957866,259124039~7568658,337088926~3384051,346923665~8917941,375360913~1040682] wrote [33781672~6108467,53857509~1127803,63382715~8163650,130769017~6957866,259124039~7568658,337088926~3384051,346923665~8917941,375360913~1040682] 2024-04-07T17:38:16.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:38:16 smithi144 bash[66654]: cluster 2024-04-07T17:38:15.289083+0000 mon.a (mon.0) 3951 : cluster [DBG] osdmap e2059: 8 total, 8 up, 8 in 2024-04-07T17:38:16.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:38:16 smithi100 bash[74942]: cluster 2024-04-07T17:38:15.289083+0000 mon.a (mon.0) 3951 : cluster [DBG] osdmap e2059: 8 total, 8 up, 8 in 2024-04-07T17:38:16.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:38:16 smithi100 bash[76202]: cluster 2024-04-07T17:38:15.289083+0000 mon.a (mon.0) 3951 : cluster [DBG] osdmap e2059: 8 total, 8 up, 8 in 2024-04-07T17:38:17.443 INFO:tasks.workunit.client.0.smithi100.stdout: i=1 exists [135970~3600283,20851609~5239642,33781672~6108467,53857509~1127803,59522208~12024157,130769017~6957866,139897475~6992307,156672877~7813147,228319985~8038844,259124039~7568658,278800011~2587375,300606881~2031214,329883097~5193465,337088926~3384051,346923665~8917941,375360913~1040682] wrote [135970~3600283,20851609~5239642,59522208~6680313,139897475~6992307,156672877~7813147,228319985~8038844,278800011~2587375,300606881~2031214,329883097~5193465] 2024-04-07T17:38:17.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:38:17 smithi144 bash[66654]: cluster 2024-04-07T17:38:16.296764+0000 mon.a (mon.0) 3952 : cluster [DBG] osdmap e2060: 8 total, 8 up, 8 in 2024-04-07T17:38:17.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:38:17 smithi144 bash[66654]: cluster 2024-04-07T17:38:16.501139+0000 mgr.y (mgr.25175) 2487 : cluster [DBG] pgmap v4122: 137 pgs: 137 active+clean; 18 MiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 88 KiB/s rd, 10 KiB/s wr, 127 op/s 2024-04-07T17:38:17.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:38:17 smithi144 bash[66654]: audit 2024-04-07T17:38:16.859142+0000 mon.a (mon.0) 3953 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:38:17.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:38:17 smithi100 bash[74942]: cluster 2024-04-07T17:38:16.296764+0000 mon.a (mon.0) 3952 : cluster [DBG] osdmap e2060: 8 total, 8 up, 8 in 2024-04-07T17:38:17.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:38:17 smithi100 bash[74942]: cluster 2024-04-07T17:38:16.501139+0000 mgr.y (mgr.25175) 2487 : cluster [DBG] pgmap v4122: 137 pgs: 137 active+clean; 18 MiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 88 KiB/s rd, 10 KiB/s wr, 127 op/s 2024-04-07T17:38:17.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:38:17 smithi100 bash[74942]: audit 2024-04-07T17:38:16.859142+0000 mon.a (mon.0) 3953 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:38:17.705 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:38:17 smithi100 bash[76202]: cluster 2024-04-07T17:38:16.296764+0000 mon.a (mon.0) 3952 : cluster [DBG] osdmap e2060: 8 total, 8 up, 8 in 2024-04-07T17:38:17.705 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:38:17 smithi100 bash[76202]: cluster 2024-04-07T17:38:16.501139+0000 mgr.y (mgr.25175) 2487 : cluster [DBG] pgmap v4122: 137 pgs: 137 active+clean; 18 MiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 88 KiB/s rd, 10 KiB/s wr, 127 op/s 2024-04-07T17:38:17.705 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:38:17 smithi100 bash[76202]: audit 2024-04-07T17:38:16.859142+0000 mon.a (mon.0) 3953 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:38:18.506 INFO:tasks.workunit.client.0.smithi100.stdout: i=2 exists [135970~3600283,4241959~4424634,20851609~5239642,33781672~6108467,53857509~1127803,55635196~15911169,76665584~828837,121765212~15961671,139897475~6992307,156672877~13899884,228319985~8038844,259124039~7568658,278800011~2587375,296656267~560897,300606881~2031214,323372821~11703741,337088926~3384051,342677306~13164300,375360913~1040682] wrote [4241959~4424634,55635196~6903815,76665584~828837,121765212~9419574,162448049~8124712,296656267~560897,323372821~7994889,342677306~7170069] 2024-04-07T17:38:18.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:38:18 smithi144 bash[66654]: cluster 2024-04-07T17:38:17.307952+0000 mon.a (mon.0) 3954 : cluster [DBG] osdmap e2061: 8 total, 8 up, 8 in 2024-04-07T17:38:18.615 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 17:38:18 smithi144 bash[98866]: logger=cleanup t=2024-04-07T17:38:18.479041432Z level=info msg="Completed cleanup jobs" duration=60.336129ms 2024-04-07T17:38:18.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:38:18 smithi100 bash[74942]: cluster 2024-04-07T17:38:17.307952+0000 mon.a (mon.0) 3954 : cluster [DBG] osdmap e2061: 8 total, 8 up, 8 in 2024-04-07T17:38:18.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:38:18 smithi100 bash[76202]: cluster 2024-04-07T17:38:17.307952+0000 mon.a (mon.0) 3954 : cluster [DBG] osdmap e2061: 8 total, 8 up, 8 in 2024-04-07T17:38:18.704 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:38:18 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:38:18] "GET /metrics HTTP/1.1" 200 36747 "" "Prometheus/2.43.0" 2024-04-07T17:38:19.466 INFO:tasks.workunit.client.0.smithi100.stdout: i=3 exists [135970~8530623,10863249~6739751,20851609~5239642,33781672~6108467,53857509~1127803,55635196~15911169,76665584~828837,121765212~15961671,139897475~6992307,156672877~13899884,190801444~9111211,204003780~6195382,228319985~8038844,255564028~11128669,278800011~2587375,296656267~560897,300606881~2031214,323372821~11837557,337088926~3384051,342677306~13164300,375360913~1040682,395424119~4872830] wrote [354834~6427035,10863249~6739751,129576153~2777121,190801444~9111211,204003780~6195382,255564028~7196780,329805355~5405023,395424119~4872830] 2024-04-07T17:38:19.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:38:19 smithi144 bash[66654]: cluster 2024-04-07T17:38:18.309126+0000 mon.a (mon.0) 3955 : cluster [DBG] osdmap e2062: 8 total, 8 up, 8 in 2024-04-07T17:38:19.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:38:19 smithi144 bash[66654]: cluster 2024-04-07T17:38:18.501712+0000 mgr.y (mgr.25175) 2488 : cluster [DBG] pgmap v4125: 137 pgs: 137 active+clean; 51 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 41 KiB/s rd, 8.2 MiB/s wr, 77 op/s 2024-04-07T17:38:19.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:38:19 smithi100 bash[74942]: cluster 2024-04-07T17:38:18.309126+0000 mon.a (mon.0) 3955 : cluster [DBG] osdmap e2062: 8 total, 8 up, 8 in 2024-04-07T17:38:19.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:38:19 smithi100 bash[74942]: cluster 2024-04-07T17:38:18.501712+0000 mgr.y (mgr.25175) 2488 : cluster [DBG] pgmap v4125: 137 pgs: 137 active+clean; 51 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 41 KiB/s rd, 8.2 MiB/s wr, 77 op/s 2024-04-07T17:38:19.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:38:19 smithi100 bash[76202]: cluster 2024-04-07T17:38:18.309126+0000 mon.a (mon.0) 3955 : cluster [DBG] osdmap e2062: 8 total, 8 up, 8 in 2024-04-07T17:38:19.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:38:19 smithi100 bash[76202]: cluster 2024-04-07T17:38:18.501712+0000 mgr.y (mgr.25175) 2488 : cluster [DBG] pgmap v4125: 137 pgs: 137 active+clean; 51 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 41 KiB/s rd, 8.2 MiB/s wr, 77 op/s 2024-04-07T17:38:20.448 INFO:tasks.workunit.client.0.smithi100.stdout: i=4 exists [135970~8530623,10863249~6739751,20851609~5239642,33781672~6108467,53857509~1127803,55635196~15911169,72496287~4998134,99383206~1096517,121765212~15961671,139897475~6992307,156672877~13899884,178187370~6502412,190801444~9111211,204003780~6195382,228319985~9542533,242022317~6703915,255564028~11128669,278800011~2587375,292708356~3415041,296656267~560897,300606881~2031214,323372821~11837557,337088926~18752680,364286689~6481364,375360913~1040682,395424119~4872830] wrote [72496287~4652282,99383206~1096517,178187370~6502412,232414757~5447761,242022317~6703915,292708356~3415041,338184666~4782341,346585174~1423971,348610231~4279116,364286689~6481364] 2024-04-07T17:38:20.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:38:20 smithi144 bash[66654]: cluster 2024-04-07T17:38:19.332639+0000 mon.a (mon.0) 3956 : cluster [DBG] osdmap e2063: 8 total, 8 up, 8 in 2024-04-07T17:38:20.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:38:20 smithi100 bash[74942]: cluster 2024-04-07T17:38:19.332639+0000 mon.a (mon.0) 3956 : cluster [DBG] osdmap e2063: 8 total, 8 up, 8 in 2024-04-07T17:38:20.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:38:20 smithi100 bash[76202]: cluster 2024-04-07T17:38:19.332639+0000 mon.a (mon.0) 3956 : cluster [DBG] osdmap e2063: 8 total, 8 up, 8 in 2024-04-07T17:38:21.500 INFO:tasks.workunit.client.0.smithi100.stdout: i=5 exists [135970~9751143,10863249~6739751,20851609~5239642,33781672~6108467,53857509~1127803,55635196~15911169,72496287~4998134,99383206~1096517,121765212~15961671,139897475~6992307,156672877~13899884,178187370~6502412,189665184~10247471,200961744~9237418,228319985~11883315,242022317~8014907,255564028~11128669,278800011~2587375,292708356~3415041,296656267~560897,300606881~2031214,307713907~7655734,323372821~11837557,337088926~18752680,364286689~10237892,375360913~1040682,395424119~4872830,408416885~7938494] wrote [1898349~7988764,123566629~4132484,189665184~5822546,200961744~4004361,232402639~7800661,242336180~7701044,307713907~7655734,349087836~1678255,370156410~4368171,408416885~7938494] 2024-04-07T17:38:21.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:38:21 smithi144 bash[66654]: cluster 2024-04-07T17:38:20.332259+0000 mon.a (mon.0) 3957 : cluster [DBG] osdmap e2064: 8 total, 8 up, 8 in 2024-04-07T17:38:21.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:38:21 smithi144 bash[66654]: cluster 2024-04-07T17:38:20.502223+0000 mgr.y (mgr.25175) 2489 : cluster [DBG] pgmap v4128: 137 pgs: 137 active+clean; 157 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 118 KiB/s rd, 27 MiB/s wr, 207 op/s 2024-04-07T17:38:21.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:38:21 smithi100 bash[74942]: cluster 2024-04-07T17:38:20.332259+0000 mon.a (mon.0) 3957 : cluster [DBG] osdmap e2064: 8 total, 8 up, 8 in 2024-04-07T17:38:21.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:38:21 smithi100 bash[74942]: cluster 2024-04-07T17:38:20.502223+0000 mgr.y (mgr.25175) 2489 : cluster [DBG] pgmap v4128: 137 pgs: 137 active+clean; 157 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 118 KiB/s rd, 27 MiB/s wr, 207 op/s 2024-04-07T17:38:21.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:38:21 smithi100 bash[76202]: cluster 2024-04-07T17:38:20.332259+0000 mon.a (mon.0) 3957 : cluster [DBG] osdmap e2064: 8 total, 8 up, 8 in 2024-04-07T17:38:21.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:38:21 smithi100 bash[76202]: cluster 2024-04-07T17:38:20.502223+0000 mgr.y (mgr.25175) 2489 : cluster [DBG] pgmap v4128: 137 pgs: 137 active+clean; 157 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 118 KiB/s rd, 27 MiB/s wr, 207 op/s 2024-04-07T17:38:22.471 INFO:tasks.workunit.client.0.smithi100.stdout: i=6 exists [135970~9751143,10863249~6739751,20851609~5239642,33781672~6108467,42885279~809907,53857509~1127803,55635196~15911169,71854876~5639545,78256833~2984608,96290273~4196163,121140662~16586221,139897475~6992307,156672877~13899884,178187370~6502412,189665184~20533978,228319985~11883315,242022317~8014907,255564028~11128669,273511517~7875869,292708356~3415041,296656267~560897,300606881~2031214,307713907~9025756,319594034~15616344,337088926~18752680,364286689~10237892,375360913~1040682,395424119~4872830,408416885~8094065] wrote [42885279~809907,71854876~1044452,78256833~2984608,96290273~4196163,121140662~3008828,195590200~7043720,273511517~7246441,312099035~4640628,319594034~8141909,408618043~7892907] 2024-04-07T17:38:22.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:38:22 smithi144 bash[66654]: cluster 2024-04-07T17:38:21.352863+0000 mon.a (mon.0) 3958 : cluster [DBG] osdmap e2065: 8 total, 8 up, 8 in 2024-04-07T17:38:22.615 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:38:22 smithi144 bash[95233]: ts=2024-04-07T17:38:22.317Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:38:22.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:38:22 smithi100 bash[74942]: cluster 2024-04-07T17:38:21.352863+0000 mon.a (mon.0) 3958 : cluster [DBG] osdmap e2065: 8 total, 8 up, 8 in 2024-04-07T17:38:22.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:38:22 smithi100 bash[76202]: cluster 2024-04-07T17:38:21.352863+0000 mon.a (mon.0) 3958 : cluster [DBG] osdmap e2065: 8 total, 8 up, 8 in 2024-04-07T17:38:23.498 INFO:tasks.workunit.client.0.smithi100.stdout: i=7 exists [135970~17467030,20851609~5239642,33781672~7960490,42885279~809907,53857509~1127803,55635196~15911169,71854876~5639545,78256833~2984608,85628593~8024294,96290273~4196163,121140662~16586221,139897475~6992307,149846055~1346293,156672877~13899884,178187370~6635944,189665184~20533978,228319985~11883315,242022317~8881446,255564028~11128669,273511517~7875869,282944906~723991,287307706~4750023,292708356~3415041,296656267~560897,300606881~2031214,307713907~9025756,319594034~15616344,337088926~18752680,364286689~10237892,375360913~1040682,395424119~4872830,408416885~8094065] wrote [2346428~3032018,6866991~7349907,36311735~5430427,85628593~8024294,149846055~1346293,182021599~2801715,243596419~7307344,282944906~723991,287307706~4750023] 2024-04-07T17:38:23.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:38:23 smithi100 bash[74942]: cluster 2024-04-07T17:38:22.354783+0000 mon.a (mon.0) 3959 : cluster [DBG] osdmap e2066: 8 total, 8 up, 8 in 2024-04-07T17:38:23.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:38:23 smithi100 bash[74942]: cluster 2024-04-07T17:38:22.502645+0000 mgr.y (mgr.25175) 2490 : cluster [DBG] pgmap v4131: 137 pgs: 137 active+clean; 251 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 147 KiB/s rd, 40 MiB/s wr, 240 op/s 2024-04-07T17:38:23.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:38:23 smithi100 bash[76202]: cluster 2024-04-07T17:38:22.354783+0000 mon.a (mon.0) 3959 : cluster [DBG] osdmap e2066: 8 total, 8 up, 8 in 2024-04-07T17:38:23.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:38:23 smithi100 bash[76202]: cluster 2024-04-07T17:38:22.502645+0000 mgr.y (mgr.25175) 2490 : cluster [DBG] pgmap v4131: 137 pgs: 137 active+clean; 251 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 147 KiB/s rd, 40 MiB/s wr, 240 op/s 2024-04-07T17:38:23.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:38:23 smithi144 bash[66654]: cluster 2024-04-07T17:38:22.354783+0000 mon.a (mon.0) 3959 : cluster [DBG] osdmap e2066: 8 total, 8 up, 8 in 2024-04-07T17:38:23.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:38:23 smithi144 bash[66654]: cluster 2024-04-07T17:38:22.502645+0000 mgr.y (mgr.25175) 2490 : cluster [DBG] pgmap v4131: 137 pgs: 137 active+clean; 251 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 147 KiB/s rd, 40 MiB/s wr, 240 op/s 2024-04-07T17:38:24.489 INFO:tasks.workunit.client.0.smithi100.stdout: i=8 exists [135970~17571613,20851609~5239642,33781672~7960490,41762228~990328,42885279~809907,53857509~1127803,55635196~15911169,71854876~9660522,85628593~8256466,96290273~4196163,121140662~16586221,139897475~6992307,149846055~1346293,156672877~13899884,178187370~6635944,189665184~20533978,215033017~2159193,228319985~11883315,242022317~8881446,255564028~11128669,272271157~937426,273511517~7875869,282944906~723991,287307706~4750023,292708356~3415041,296656267~560897,300606881~2031214,307713907~9025756,319594034~36247572,361027603~13496978,375360913~1040682,395424119~4872830,408416885~8094065] wrote [16990981~716602,41762228~990328,76830497~4684901,87765767~6119292,215033017~2159193,243069335~1315015,246807198~1750980,272271157~937426,331771165~7669225,361027603~7224190] 2024-04-07T17:38:24.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:38:24 smithi100 bash[74942]: cluster 2024-04-07T17:38:23.381874+0000 mon.a (mon.0) 3960 : cluster [DBG] osdmap e2067: 8 total, 8 up, 8 in 2024-04-07T17:38:24.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:38:24 smithi100 bash[76202]: cluster 2024-04-07T17:38:23.381874+0000 mon.a (mon.0) 3960 : cluster [DBG] osdmap e2067: 8 total, 8 up, 8 in 2024-04-07T17:38:24.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:38:24 smithi144 bash[66654]: cluster 2024-04-07T17:38:23.381874+0000 mon.a (mon.0) 3960 : cluster [DBG] osdmap e2067: 8 total, 8 up, 8 in 2024-04-07T17:38:25.504 INFO:tasks.workunit.client.0.smithi100.stdout: i=9 exists [135970~17571613,20851609~5239642,33781672~7960490,41762228~990328,42885279~809907,53857509~1127803,55635196~15911169,71854876~9660522,85628593~8256466,94453348~7285664,111865283~985864,121140662~16586221,138090725~8799057,149846055~1346293,156672877~13899884,178187370~6635944,189665184~23417353,215033017~2159193,228319985~11883315,242022317~8881446,255564028~11128669,272271157~937426,273511517~7875869,282944906~723991,287307706~4750023,292708356~3415041,296656267~560897,300606881~2031214,307713907~9025756,319594034~36247572,361027603~13496978,375360913~1040682,391525100~1487154,395424119~4872830,408416885~8094065] wrote [6610653~177805,94453348~7285664,111865283~985864,138090725~2838875,206957739~6124798,315012752~1521156,332884881~5688704,338809080~4167833,391525100~1487154] 2024-04-07T17:38:25.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:38:25 smithi100 bash[74942]: cluster 2024-04-07T17:38:24.392485+0000 mon.a (mon.0) 3961 : cluster [DBG] osdmap e2068: 8 total, 8 up, 8 in 2024-04-07T17:38:25.705 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:38:25 smithi100 bash[74942]: cluster 2024-04-07T17:38:24.504395+0000 mgr.y (mgr.25175) 2491 : cluster [DBG] pgmap v4134: 137 pgs: 137 active+clean; 365 MiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 177 KiB/s rd, 47 MiB/s wr, 292 op/s 2024-04-07T17:38:25.705 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:38:25 smithi100 bash[76202]: cluster 2024-04-07T17:38:24.392485+0000 mon.a (mon.0) 3961 : cluster [DBG] osdmap e2068: 8 total, 8 up, 8 in 2024-04-07T17:38:25.705 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:38:25 smithi100 bash[76202]: cluster 2024-04-07T17:38:24.504395+0000 mgr.y (mgr.25175) 2491 : cluster [DBG] pgmap v4134: 137 pgs: 137 active+clean; 365 MiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 177 KiB/s rd, 47 MiB/s wr, 292 op/s 2024-04-07T17:38:25.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:38:25 smithi144 bash[66654]: cluster 2024-04-07T17:38:24.392485+0000 mon.a (mon.0) 3961 : cluster [DBG] osdmap e2068: 8 total, 8 up, 8 in 2024-04-07T17:38:25.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:38:25 smithi144 bash[66654]: cluster 2024-04-07T17:38:24.504395+0000 mgr.y (mgr.25175) 2491 : cluster [DBG] pgmap v4134: 137 pgs: 137 active+clean; 365 MiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 177 KiB/s rd, 47 MiB/s wr, 292 op/s 2024-04-07T17:38:26.531 INFO:tasks.workunit.client.0.smithi100.stdout: i=10 exists [135970~17571613,20851609~5239642,33781672~7960490,41762228~990328,42885279~809907,49437536~3890828,53857509~1127803,55635196~15911169,71854876~9660522,85628593~8256466,94453348~7285664,111865283~985864,121140662~16586221,138090725~8799057,149846055~1346293,156672877~13911289,178187370~6635944,189665184~23417353,215033017~2159193,228319985~11883315,242022317~8881446,255564028~11128669,272271157~937426,273511517~8782856,282944906~3695314,287307706~4750023,292708356~3415041,296656267~560897,300606881~2031214,303099269~13640394,319594034~36247572,361027603~13496978,375360913~1040682,389403602~801349,391525100~1487154,395424119~4872830,408416885~8094065] wrote [10563034~5821557,49437536~3890828,134440787~1545869,137469043~159978,168202059~2382107,275546541~6747832,282992676~3647544,303099269~7431219,389403602~801349] 2024-04-07T17:38:26.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:38:26 smithi100 bash[74942]: cluster 2024-04-07T17:38:25.403082+0000 mon.a (mon.0) 3962 : cluster [DBG] osdmap e2069: 8 total, 8 up, 8 in 2024-04-07T17:38:26.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:38:26 smithi100 bash[76202]: cluster 2024-04-07T17:38:25.403082+0000 mon.a (mon.0) 3962 : cluster [DBG] osdmap e2069: 8 total, 8 up, 8 in 2024-04-07T17:38:26.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:38:26 smithi144 bash[66654]: cluster 2024-04-07T17:38:25.403082+0000 mon.a (mon.0) 3962 : cluster [DBG] osdmap e2069: 8 total, 8 up, 8 in 2024-04-07T17:38:27.497 INFO:tasks.workunit.client.0.smithi100.stdout: i=11 exists [135970~17571613,20851609~5239642,33781672~7960490,41762228~990328,42885279~809907,49437536~3890828,53857509~1127803,55635196~15911169,71854876~9660522,85628593~8256466,94453348~7285664,111865283~985864,121140662~16586221,138090725~8799057,149846055~1346293,156672877~13911289,178187370~6635944,189665184~23417353,215033017~2159193,228319985~11883315,242022317~8881446,255564028~11128669,270324491~109544,272271157~937426,273511517~8782856,282944906~3695314,287307706~4750023,292708356~3415041,296303025~80482,296656267~560897,300606881~2031214,303099269~13640394,319594034~36247572,361027603~15373992,389403602~801349,391525100~1487154,395424119~4872830,408416885~8094065] wrote [60545147~709252,74203763~4220819,180972784~3496491,210507027~2001515,258233101~1648715,261176217~3235343,270324491~109544,296303025~80482,368144454~7751689] 2024-04-07T17:38:27.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:38:27 smithi100 bash[74942]: cluster 2024-04-07T17:38:26.403476+0000 mon.a (mon.0) 3963 : cluster [DBG] osdmap e2070: 8 total, 8 up, 8 in 2024-04-07T17:38:27.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:38:27 smithi100 bash[74942]: cluster 2024-04-07T17:38:26.504962+0000 mgr.y (mgr.25175) 2492 : cluster [DBG] pgmap v4137: 137 pgs: 137 active+clean; 422 MiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 134 KiB/s rd, 38 MiB/s wr, 231 op/s 2024-04-07T17:38:27.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:38:27 smithi100 bash[76202]: cluster 2024-04-07T17:38:26.403476+0000 mon.a (mon.0) 3963 : cluster [DBG] osdmap e2070: 8 total, 8 up, 8 in 2024-04-07T17:38:27.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:38:27 smithi100 bash[76202]: cluster 2024-04-07T17:38:26.504962+0000 mgr.y (mgr.25175) 2492 : cluster [DBG] pgmap v4137: 137 pgs: 137 active+clean; 422 MiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 134 KiB/s rd, 38 MiB/s wr, 231 op/s 2024-04-07T17:38:27.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:38:27 smithi144 bash[66654]: cluster 2024-04-07T17:38:26.403476+0000 mon.a (mon.0) 3963 : cluster [DBG] osdmap e2070: 8 total, 8 up, 8 in 2024-04-07T17:38:27.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:38:27 smithi144 bash[66654]: cluster 2024-04-07T17:38:26.504962+0000 mgr.y (mgr.25175) 2492 : cluster [DBG] pgmap v4137: 137 pgs: 137 active+clean; 422 MiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 134 KiB/s rd, 38 MiB/s wr, 231 op/s 2024-04-07T17:38:28.552 INFO:tasks.workunit.client.0.smithi100.stdout: i=12 exists [135970~17571613,20851609~5239642,33781672~7960490,41762228~990328,42885279~809907,49437536~3890828,53857509~1127803,55635196~15911169,71854876~9660522,85628593~8256466,94453348~7285664,111865283~985864,114996116~5098745,121140662~16586221,138090725~8799057,149846055~1346293,156672877~13911289,178187370~6635944,189665184~23417353,215033017~2159193,228319985~11883315,242022317~8881446,255564028~11128669,270324491~109544,272271157~937426,273511517~8782856,282944906~3695314,287307706~4750023,292708356~3415041,296303025~80482,296656267~560897,300606881~2031214,303099269~13640394,319594034~38171014,361027603~15373992,383522705~252850,389403602~801349,391525100~1487154,395424119~4872830,408416885~8094065] wrote [2066589~7969438,36678864~809099,37898556~2506907,58247479~2841214,71877350~3402400,114996116~5098745,194587522~6018201,347627264~1358215,350159037~7606011,383522705~252850] 2024-04-07T17:38:28.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:38:28 smithi100 bash[74942]: cluster 2024-04-07T17:38:27.431349+0000 mon.a (mon.0) 3964 : cluster [DBG] osdmap e2071: 8 total, 8 up, 8 in 2024-04-07T17:38:28.704 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:38:28 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:38:28] "GET /metrics HTTP/1.1" 200 36747 "" "Prometheus/2.43.0" 2024-04-07T17:38:28.705 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:38:28 smithi100 bash[76202]: cluster 2024-04-07T17:38:27.431349+0000 mon.a (mon.0) 3964 : cluster [DBG] osdmap e2071: 8 total, 8 up, 8 in 2024-04-07T17:38:28.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:38:28 smithi144 bash[66654]: cluster 2024-04-07T17:38:27.431349+0000 mon.a (mon.0) 3964 : cluster [DBG] osdmap e2071: 8 total, 8 up, 8 in 2024-04-07T17:38:29.600 INFO:tasks.workunit.client.0.smithi100.stdout: i=13 exists [135970~17571613,20851609~5239642,33781672~7960490,41762228~990328,42885279~809907,49437536~3890828,53857509~1127803,55635196~15911169,71854876~22030183,94453348~7285664,105788147~7063000,114996116~5098745,121140662~16586221,138090725~14603813,156672877~13911289,178187370~8138708,189665184~23417353,215033017~2159193,228319985~11883315,242022317~8881446,255564028~11128669,270324491~109544,272271157~937426,273511517~8782856,282944906~3695314,287307706~4750023,292708356~3415041,296303025~80482,296656267~560897,300606881~2031214,303099269~13640394,319594034~38171014,361027603~15373992,383522705~252850,389403602~801349,391525100~1487154,395424119~4872830,408416885~8094065] wrote [8941663~3359652,63080343~8183501,81224748~5845976,105788147~6896671,144844030~7850508,180225622~6100456,191047711~4793341,331681945~7547106,348956913~2505896] 2024-04-07T17:38:29.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:38:29 smithi144 bash[66654]: cluster 2024-04-07T17:38:28.432775+0000 mon.a (mon.0) 3965 : cluster [DBG] osdmap e2072: 8 total, 8 up, 8 in 2024-04-07T17:38:29.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:38:29 smithi144 bash[66654]: cluster 2024-04-07T17:38:28.506042+0000 mgr.y (mgr.25175) 2493 : cluster [DBG] pgmap v4140: 137 pgs: 137 active+clean; 534 MiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 75 KiB/s rd, 36 MiB/s wr, 119 op/s 2024-04-07T17:38:29.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:38:29 smithi100 bash[74942]: cluster 2024-04-07T17:38:28.432775+0000 mon.a (mon.0) 3965 : cluster [DBG] osdmap e2072: 8 total, 8 up, 8 in 2024-04-07T17:38:29.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:38:29 smithi100 bash[74942]: cluster 2024-04-07T17:38:28.506042+0000 mgr.y (mgr.25175) 2493 : cluster [DBG] pgmap v4140: 137 pgs: 137 active+clean; 534 MiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 75 KiB/s rd, 36 MiB/s wr, 119 op/s 2024-04-07T17:38:29.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:38:29 smithi100 bash[76202]: cluster 2024-04-07T17:38:28.432775+0000 mon.a (mon.0) 3965 : cluster [DBG] osdmap e2072: 8 total, 8 up, 8 in 2024-04-07T17:38:29.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:38:29 smithi100 bash[76202]: cluster 2024-04-07T17:38:28.506042+0000 mgr.y (mgr.25175) 2493 : cluster [DBG] pgmap v4140: 137 pgs: 137 active+clean; 534 MiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 75 KiB/s rd, 36 MiB/s wr, 119 op/s 2024-04-07T17:38:30.585 INFO:tasks.workunit.client.0.smithi100.stdout: i=14 exists [135970~20301325,20851609~5239642,33781672~7960490,41762228~990328,42885279~809907,47199670~2098524,49437536~3890828,53857509~1127803,55635196~15911169,71854876~22030183,94453348~7285664,105788147~7063000,114996116~5098745,121140662~16586221,138090725~14603813,156672877~13911289,178187370~9473348,189665184~23417353,215033017~2159193,227079958~13123342,242022317~8881446,255564028~11128669,270324491~109544,272271157~937426,273511517~8782856,282944906~3695314,287307706~4750023,292708356~3415041,296303025~80482,296656267~560897,300606881~2031214,303099269~13640394,319594034~38171014,361027603~15373992,383522705~252850,387602283~5409971,395424119~4872830,408416885~8094065] wrote [6086922~2865666,17595571~2841724,47199670~2098524,79509484~354470,143823044~7003044,164607639~1989872,184417657~3243061,206451372~6625993,227079958~6835535,387602283~5019425] 2024-04-07T17:38:30.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:38:30 smithi144 bash[66654]: cluster 2024-04-07T17:38:29.450925+0000 mon.a (mon.0) 3966 : cluster [DBG] osdmap e2073: 8 total, 8 up, 8 in 2024-04-07T17:38:30.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:38:30 smithi100 bash[74942]: cluster 2024-04-07T17:38:29.450925+0000 mon.a (mon.0) 3966 : cluster [DBG] osdmap e2073: 8 total, 8 up, 8 in 2024-04-07T17:38:30.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:38:30 smithi100 bash[76202]: cluster 2024-04-07T17:38:29.450925+0000 mon.a (mon.0) 3966 : cluster [DBG] osdmap e2073: 8 total, 8 up, 8 in 2024-04-07T17:38:31.640 INFO:tasks.workunit.client.0.smithi100.stdout: i=15 exists [135970~20301325,20851609~5239642,33781672~7960490,41762228~990328,42885279~809907,47199670~2098524,49437536~3890828,53857509~1127803,55635196~15911169,71854876~22030183,94453348~7285664,105788147~7063000,114996116~5098745,121140662~16586221,138090725~14603813,156672877~13911289,178187370~9473348,189665184~23417353,215033017~2159193,227079958~13123342,242022317~8881446,255564028~11128669,270324491~109544,272271157~937426,273511517~8782856,282944906~3695314,287307706~4750023,292708356~3415041,296303025~80482,296656267~560897,300606881~16132782,319127194~38637854,361027603~18475175,379938482~4481520,387602283~5409971,395424119~4872830,408416885~8094065] wrote [6173974~4586326,80277895~6428652,107132853~4110562,301512734~6676926,319127194~8107314,350520810~7029376,374944227~4558551,379938482~4481520] 2024-04-07T17:38:31.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:38:31 smithi144 bash[66654]: cluster 2024-04-07T17:38:30.472306+0000 mon.a (mon.0) 3967 : cluster [DBG] osdmap e2074: 8 total, 8 up, 8 in 2024-04-07T17:38:31.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:38:31 smithi144 bash[66654]: cluster 2024-04-07T17:38:30.506924+0000 mgr.y (mgr.25175) 2494 : cluster [DBG] pgmap v4143: 137 pgs: 137 active+clean; 628 MiB data, 5.1 GiB used, 710 GiB / 715 GiB avail; 176 KiB/s rd, 46 MiB/s wr, 287 op/s 2024-04-07T17:38:31.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:38:31 smithi100 bash[74942]: cluster 2024-04-07T17:38:30.472306+0000 mon.a (mon.0) 3967 : cluster [DBG] osdmap e2074: 8 total, 8 up, 8 in 2024-04-07T17:38:31.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:38:31 smithi100 bash[74942]: cluster 2024-04-07T17:38:30.506924+0000 mgr.y (mgr.25175) 2494 : cluster [DBG] pgmap v4143: 137 pgs: 137 active+clean; 628 MiB data, 5.1 GiB used, 710 GiB / 715 GiB avail; 176 KiB/s rd, 46 MiB/s wr, 287 op/s 2024-04-07T17:38:31.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:38:31 smithi100 bash[76202]: cluster 2024-04-07T17:38:30.472306+0000 mon.a (mon.0) 3967 : cluster [DBG] osdmap e2074: 8 total, 8 up, 8 in 2024-04-07T17:38:31.955 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:38:31 smithi100 bash[76202]: cluster 2024-04-07T17:38:30.506924+0000 mgr.y (mgr.25175) 2494 : cluster [DBG] pgmap v4143: 137 pgs: 137 active+clean; 628 MiB data, 5.1 GiB used, 710 GiB / 715 GiB avail; 176 KiB/s rd, 46 MiB/s wr, 287 op/s 2024-04-07T17:38:32.591 INFO:tasks.workunit.client.0.smithi100.stdout: i=16 exists [135970~20301325,20851609~5239642,33781672~7960490,41762228~990328,42885279~809907,47199670~2098524,49437536~3890828,53857509~1127803,55635196~15911169,71854876~22030183,94453348~7285664,102845688~10005459,114996116~5098745,121140662~16586221,138090725~14603813,156672877~13911289,178187370~9473348,189665184~23417353,215033017~2159193,227079958~13123342,242022317~8881446,255564028~11128669,270324491~109544,272271157~937426,273511517~8782856,282944906~3695314,287307706~4750023,292708356~3415041,296303025~80482,296656267~560897,300606881~16132782,317905370~39859678,361027603~18475175,379938482~4481520,387602283~7759799,395424119~4872830,408416885~8094065] wrote [66472543~5011811,74468923~5775911,102845688~4685075,130133038~6359999,161190951~1367491,234510514~2026394,317905370~6675353,333998699~2241053,344572610~913422,388134112~7227970] 2024-04-07T17:38:32.615 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:38:32 smithi144 bash[95233]: ts=2024-04-07T17:38:32.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:38:32.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:38:32 smithi144 bash[66654]: cluster 2024-04-07T17:38:31.482181+0000 mon.a (mon.0) 3968 : cluster [DBG] osdmap e2075: 8 total, 8 up, 8 in 2024-04-07T17:38:32.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:38:32 smithi144 bash[66654]: audit 2024-04-07T17:38:31.859651+0000 mon.a (mon.0) 3969 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:38:32.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:38:32 smithi100 bash[74942]: cluster 2024-04-07T17:38:31.482181+0000 mon.a (mon.0) 3968 : cluster [DBG] osdmap e2075: 8 total, 8 up, 8 in 2024-04-07T17:38:32.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:38:32 smithi100 bash[74942]: audit 2024-04-07T17:38:31.859651+0000 mon.a (mon.0) 3969 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:38:32.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:38:32 smithi100 bash[76202]: cluster 2024-04-07T17:38:31.482181+0000 mon.a (mon.0) 3968 : cluster [DBG] osdmap e2075: 8 total, 8 up, 8 in 2024-04-07T17:38:32.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:38:32 smithi100 bash[76202]: audit 2024-04-07T17:38:31.859651+0000 mon.a (mon.0) 3969 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:38:33.662 INFO:tasks.workunit.client.0.smithi100.stdout: i=17 exists [135970~20301325,20851609~9735227,33781672~7960490,41762228~990328,42885279~809907,47199670~2098524,49437536~3890828,53857509~1127803,55635196~15911169,71854876~22030183,94453348~7285664,102845688~10005459,114996116~5098745,121140662~31553876,156672877~13911289,171455245~1897728,178187370~34895167,215033017~2159193,227079958~13123342,242022317~8881446,255564028~11128669,270324491~109544,272271157~937426,273511517~13128703,287307706~4750023,292708356~3415041,296303025~80482,296656267~560897,300606881~16132782,317905370~42390563,361027603~18475175,379938482~4481520,387602283~7759799,395424119~4872830,402688738~13822212] wrote [23194361~7392475,125020827~7976080,136497196~7268612,171455245~1897728,185594310~4604676,230152020~3153940,281834000~4702748,338386379~707273,354233840~6062093,402688738~8112353] 2024-04-07T17:38:33.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:38:33 smithi144 bash[66654]: cluster 2024-04-07T17:38:32.483612+0000 mon.a (mon.0) 3970 : cluster [DBG] osdmap e2076: 8 total, 8 up, 8 in 2024-04-07T17:38:33.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:38:33 smithi144 bash[66654]: cluster 2024-04-07T17:38:32.507696+0000 mgr.y (mgr.25175) 2495 : cluster [DBG] pgmap v4146: 137 pgs: 137 active+clean; 694 MiB data, 5.3 GiB used, 710 GiB / 715 GiB avail; 210 KiB/s rd, 39 MiB/s wr, 325 op/s 2024-04-07T17:38:33.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:38:33 smithi100 bash[74942]: cluster 2024-04-07T17:38:32.483612+0000 mon.a (mon.0) 3970 : cluster [DBG] osdmap e2076: 8 total, 8 up, 8 in 2024-04-07T17:38:33.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:38:33 smithi100 bash[74942]: cluster 2024-04-07T17:38:32.507696+0000 mgr.y (mgr.25175) 2495 : cluster [DBG] pgmap v4146: 137 pgs: 137 active+clean; 694 MiB data, 5.3 GiB used, 710 GiB / 715 GiB avail; 210 KiB/s rd, 39 MiB/s wr, 325 op/s 2024-04-07T17:38:33.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:38:33 smithi100 bash[76202]: cluster 2024-04-07T17:38:32.483612+0000 mon.a (mon.0) 3970 : cluster [DBG] osdmap e2076: 8 total, 8 up, 8 in 2024-04-07T17:38:33.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:38:33 smithi100 bash[76202]: cluster 2024-04-07T17:38:32.507696+0000 mgr.y (mgr.25175) 2495 : cluster [DBG] pgmap v4146: 137 pgs: 137 active+clean; 694 MiB data, 5.3 GiB used, 710 GiB / 715 GiB avail; 210 KiB/s rd, 39 MiB/s wr, 325 op/s 2024-04-07T17:38:34.679 INFO:tasks.workunit.client.0.smithi100.stdout: i=18 exists [135970~20301325,20851609~9735227,33781672~7960490,41762228~990328,42885279~809907,47199670~2098524,49437536~3890828,53857509~1127803,55635196~15911169,71854876~22030183,94453348~7285664,102845688~10005459,114996116~5098745,121140662~31553876,156144461~14439705,171455245~1897728,178187370~34895167,215033017~8474688,227079958~13123342,242022317~8881446,254276436~12416261,270324491~109544,272271157~937426,273511517~13128703,287307706~4750023,292708356~3415041,296303025~80482,296656267~560897,300606881~16132782,317905370~42390563,361027603~18475175,379938482~4481520,387602283~7759799,395424119~4872830,402688738~13822212] wrote [35912503~4294987,59157938~4444849,78294597~901434,121679458~6885941,146019593~5146512,156144461~4790456,203308918~6876418,215547490~7960215,254276436~11281882] 2024-04-07T17:38:34.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:38:34 smithi144 bash[66654]: cluster 2024-04-07T17:38:33.510533+0000 mon.a (mon.0) 3971 : cluster [DBG] osdmap e2077: 8 total, 8 up, 8 in 2024-04-07T17:38:34.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:38:34 smithi100 bash[74942]: cluster 2024-04-07T17:38:33.510533+0000 mon.a (mon.0) 3971 : cluster [DBG] osdmap e2077: 8 total, 8 up, 8 in 2024-04-07T17:38:34.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:38:34 smithi100 bash[76202]: cluster 2024-04-07T17:38:33.510533+0000 mon.a (mon.0) 3971 : cluster [DBG] osdmap e2077: 8 total, 8 up, 8 in 2024-04-07T17:38:35.632 INFO:tasks.workunit.client.0.smithi100.stdout: i=19 exists [135970~20301325,20851609~9735227,33781672~7960490,41762228~990328,42885279~809907,47199670~2098524,49437536~3890828,53857509~1127803,55635196~15911169,71854876~22030183,94453348~7285664,102845688~10005459,113742233~6806840,121140662~31553876,156144461~17208512,178187370~34895167,215033017~9582213,227079958~13123342,242022317~8881446,254276436~12416261,270324491~109544,272271157~937426,273511517~13128703,287307706~4750023,292708356~3415041,296303025~80482,296656267~560897,300606881~16132782,317377612~42918321,361027603~18475175,379938482~4481520,387602283~7759799,395424119~4872830,402688738~13822212] wrote [17293072~905091,113742233~6806840,125925092~3990918,141292080~875057,157852046~3866462,169312103~3035430,221649899~2965331,229158886~2306295,317377612~3926549,367380194~4808242] 2024-04-07T17:38:35.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:38:35 smithi144 bash[66654]: cluster 2024-04-07T17:38:34.508708+0000 mgr.y (mgr.25175) 2496 : cluster [DBG] pgmap v4149: 137 pgs: 137 active+clean; 775 MiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 233 KiB/s rd, 37 MiB/s wr, 360 op/s 2024-04-07T17:38:35.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:38:35 smithi144 bash[66654]: cluster 2024-04-07T17:38:34.510532+0000 mon.a (mon.0) 3972 : cluster [DBG] osdmap e2078: 8 total, 8 up, 8 in 2024-04-07T17:38:35.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:38:35 smithi100 bash[74942]: cluster 2024-04-07T17:38:34.508708+0000 mgr.y (mgr.25175) 2496 : cluster [DBG] pgmap v4149: 137 pgs: 137 active+clean; 775 MiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 233 KiB/s rd, 37 MiB/s wr, 360 op/s 2024-04-07T17:38:35.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:38:35 smithi100 bash[74942]: cluster 2024-04-07T17:38:34.510532+0000 mon.a (mon.0) 3972 : cluster [DBG] osdmap e2078: 8 total, 8 up, 8 in 2024-04-07T17:38:35.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:38:35 smithi100 bash[76202]: cluster 2024-04-07T17:38:34.508708+0000 mgr.y (mgr.25175) 2496 : cluster [DBG] pgmap v4149: 137 pgs: 137 active+clean; 775 MiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 233 KiB/s rd, 37 MiB/s wr, 360 op/s 2024-04-07T17:38:35.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:38:35 smithi100 bash[76202]: cluster 2024-04-07T17:38:34.510532+0000 mon.a (mon.0) 3972 : cluster [DBG] osdmap e2078: 8 total, 8 up, 8 in 2024-04-07T17:38:36.582 INFO:tasks.workunit.client.0.smithi100.stdout:from 0 to 1 diff [0~4194304,16777216~8388608,58720256~8388608,138412032~8388608,155189248~8388608,226492416~8388608,276824064~4194304,297795584~4194304,327155712~8388608] 2024-04-07T17:38:36.582 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~4194304,16777216~8388608,58720256~8388608,138412032~8388608,155189248~8388608,226492416~8388608,276824064~4194304,297795584~4194304,327155712~8388608] 2024-04-07T17:38:36.614 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~4194304,16777216~12582912,58720256~8388608,138412032~12582912,155189248~12582912,226492416~12582912,276824064~8388608,297795584~8388608,327155712~8388608] 2024-04-07T17:38:36.614 INFO:tasks.workunit.client.0.smithi100.stdout:from 0 to 2 diff [0~12582912,16777216~8388608,54525952~12582912,75497472~4194304,121634816~12582912,138412032~8388608,155189248~16777216,226492416~8388608,276824064~4194304,293601280~8388608,322961408~12582912,339738624~8388608] 2024-04-07T17:38:36.614 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~12582912,16777216~8388608,54525952~12582912,75497472~4194304,121634816~12582912,138412032~8388608,155189248~16777216,226492416~8388608,276824064~4194304,293601280~8388608,322961408~12582912,339738624~8388608] 2024-04-07T17:38:36.617 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~12582912,16777216~12582912,54525952~12582912,75497472~4194304,121634816~12582912,138412032~12582912,155189248~16777216,226492416~12582912,276824064~8388608,293601280~12582912,322961408~12582912,339738624~12582912] 2024-04-07T17:38:36.617 INFO:tasks.workunit.client.0.smithi100.stdout:from 0 to 3 diff [0~25165824,54525952~12582912,75497472~4194304,121634816~12582912,138412032~8388608,155189248~16777216,188743680~20971520,226492416~8388608,251658240~8388608,276824064~4194304,293601280~8388608,322961408~12582912,339738624~8388608,394264576~8388608] 2024-04-07T17:38:36.617 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~25165824,54525952~12582912,75497472~4194304,121634816~12582912,138412032~8388608,155189248~16777216,188743680~20971520,226492416~8388608,251658240~8388608,276824064~4194304,293601280~8388608,322961408~12582912,339738624~8388608,394264576~8388608] 2024-04-07T17:38:36.622 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~29360128,54525952~12582912,75497472~4194304,121634816~12582912,138412032~12582912,155189248~16777216,188743680~25165824,226492416~12582912,251658240~12582912,276824064~8388608,293601280~12582912,322961408~12582912,339738624~12582912,394264576~8388608] 2024-04-07T17:38:36.622 INFO:tasks.workunit.client.0.smithi100.stdout:from 0 to 4 diff [0~25165824,54525952~12582912,71303168~8388608,96468992~4194304,121634816~12582912,138412032~8388608,155189248~16777216,176160768~8388608,188743680~20971520,226492416~20971520,251658240~8388608,276824064~4194304,289406976~12582912,322961408~29360128,360710144~8388608,394264576~8388608] 2024-04-07T17:38:36.623 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~25165824,54525952~12582912,71303168~8388608,96468992~4194304,121634816~12582912,138412032~8388608,155189248~16777216,176160768~8388608,188743680~20971520,226492416~20971520,251658240~8388608,276824064~4194304,289406976~12582912,322961408~29360128,360710144~8388608,394264576~8388608] 2024-04-07T17:38:36.628 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~29360128,54525952~12582912,71303168~8388608,96468992~4194304,121634816~12582912,138412032~12582912,155189248~16777216,176160768~37748736,226492416~37748736,276824064~8388608,289406976~16777216,322961408~33554432,360710144~12582912,394264576~8388608] 2024-04-07T17:38:36.628 INFO:tasks.workunit.client.0.smithi100.stdout:from 0 to 5 diff [0~25165824,54525952~12582912,71303168~8388608,96468992~4194304,121634816~12582912,138412032~8388608,155189248~16777216,176160768~8388608,188743680~20971520,226492416~20971520,251658240~8388608,276824064~4194304,289406976~12582912,306184192~8388608,322961408~29360128,360710144~12582912,394264576~8388608,406847488~8388608] 2024-04-07T17:38:36.629 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~25165824,54525952~12582912,71303168~8388608,96468992~4194304,121634816~12582912,138412032~8388608,155189248~16777216,176160768~8388608,188743680~20971520,226492416~20971520,251658240~8388608,276824064~4194304,289406976~12582912,306184192~8388608,322961408~29360128,360710144~12582912,394264576~8388608,406847488~8388608] 2024-04-07T17:38:36.636 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~29360128,54525952~12582912,71303168~8388608,96468992~4194304,121634816~12582912,138412032~12582912,155189248~16777216,176160768~37748736,226492416~37748736,276824064~8388608,289406976~29360128,322961408~33554432,360710144~16777216,394264576~8388608,406847488~12582912] 2024-04-07T17:38:36.636 INFO:tasks.workunit.client.0.smithi100.stdout:from 0 to 6 diff [0~25165824,41943040~4194304,54525952~12582912,71303168~8388608,92274688~8388608,117440512~12582912,138412032~8388608,155189248~16777216,176160768~8388608,188743680~20971520,226492416~20971520,251658240~8388608,272629760~8388608,289406976~12582912,306184192~46137344,360710144~12582912,394264576~8388608,406847488~8388608] 2024-04-07T17:38:36.636 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~25165824,41943040~4194304,54525952~12582912,71303168~8388608,92274688~8388608,117440512~12582912,138412032~8388608,155189248~16777216,176160768~8388608,188743680~20971520,226492416~20971520,251658240~8388608,272629760~8388608,289406976~12582912,306184192~46137344,360710144~12582912,394264576~8388608,406847488~8388608] 2024-04-07T17:38:36.645 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~29360128,41943040~4194304,54525952~12582912,71303168~12582912,92274688~8388608,117440512~16777216,138412032~12582912,155189248~16777216,176160768~37748736,226492416~37748736,272629760~12582912,289406976~67108864,360710144~16777216,394264576~8388608,406847488~12582912] 2024-04-07T17:38:36.645 INFO:tasks.workunit.client.0.smithi100.stdout:from 0 to 7 diff [0~25165824,33554432~12582912,54525952~12582912,71303168~8388608,83886080~16777216,117440512~12582912,138412032~12582912,155189248~16777216,176160768~8388608,188743680~20971520,226492416~33554432,272629760~29360128,306184192~46137344,360710144~12582912,394264576~8388608,406847488~8388608] 2024-04-07T17:38:36.645 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~25165824,33554432~12582912,54525952~12582912,71303168~8388608,83886080~16777216,117440512~12582912,138412032~12582912,155189248~16777216,176160768~8388608,188743680~20971520,226492416~33554432,272629760~29360128,306184192~46137344,360710144~12582912,394264576~8388608,406847488~8388608] 2024-04-07T17:38:36.654 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~29360128,33554432~12582912,54525952~12582912,71303168~29360128,117440512~16777216,138412032~33554432,176160768~37748736,226492416~37748736,272629760~83886080,360710144~16777216,394264576~8388608,406847488~12582912] 2024-04-07T17:38:36.655 INFO:tasks.workunit.client.0.smithi100.stdout:from 0 to 8 diff [0~25165824,33554432~12582912,54525952~12582912,71303168~29360128,117440512~12582912,138412032~12582912,155189248~16777216,176160768~8388608,188743680~20971520,213909504~4194304,226492416~33554432,268435456~33554432,306184192~46137344,360710144~16777216,394264576~8388608,406847488~8388608] 2024-04-07T17:38:36.655 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~25165824,33554432~12582912,54525952~12582912,71303168~29360128,117440512~12582912,138412032~12582912,155189248~16777216,176160768~8388608,188743680~20971520,213909504~4194304,226492416~33554432,268435456~33554432,306184192~46137344,360710144~16777216,394264576~8388608,406847488~8388608] 2024-04-07T17:38:36.665 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~29360128,33554432~12582912,54525952~12582912,71303168~29360128,117440512~16777216,138412032~33554432,176160768~41943040,226492416~37748736,268435456~88080384,360710144~16777216,394264576~8388608,406847488~12582912] 2024-04-07T17:38:36.665 INFO:tasks.workunit.client.0.smithi100.stdout:from 0 to 9 diff [0~25165824,33554432~12582912,54525952~12582912,71303168~29360128,109051904~4194304,117440512~12582912,134217728~16777216,155189248~16777216,176160768~8388608,188743680~29360128,226492416~33554432,268435456~33554432,306184192~46137344,360710144~16777216,390070272~12582912,406847488~8388608] 2024-04-07T17:38:36.665 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~25165824,33554432~12582912,54525952~12582912,71303168~29360128,109051904~4194304,117440512~12582912,134217728~16777216,155189248~16777216,176160768~8388608,188743680~29360128,226492416~33554432,268435456~33554432,306184192~46137344,360710144~16777216,390070272~12582912,406847488~8388608] 2024-04-07T17:38:36.676 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~29360128,33554432~12582912,54525952~12582912,71303168~33554432,109051904~4194304,117440512~54525952,176160768~41943040,226492416~37748736,268435456~88080384,360710144~16777216,390070272~12582912,406847488~12582912] 2024-04-07T17:38:36.676 INFO:tasks.workunit.client.0.smithi100.stdout:from 0 to 10 diff [0~25165824,33554432~16777216,54525952~12582912,71303168~29360128,109051904~4194304,117440512~12582912,134217728~16777216,155189248~16777216,176160768~8388608,188743680~29360128,226492416~33554432,268435456~83886080,360710144~16777216,385875968~16777216,406847488~8388608] 2024-04-07T17:38:36.676 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~25165824,33554432~16777216,54525952~12582912,71303168~29360128,109051904~4194304,117440512~12582912,134217728~16777216,155189248~16777216,176160768~8388608,188743680~29360128,226492416~33554432,268435456~83886080,360710144~16777216,385875968~16777216,406847488~8388608] 2024-04-07T17:38:36.688 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~29360128,33554432~33554432,71303168~33554432,109051904~4194304,117440512~54525952,176160768~41943040,226492416~37748736,268435456~88080384,360710144~16777216,385875968~16777216,406847488~12582912] 2024-04-07T17:38:36.688 INFO:tasks.workunit.client.0.smithi100.stdout:from 0 to 11 diff [0~25165824,33554432~16777216,54525952~12582912,71303168~29360128,109051904~4194304,117440512~12582912,134217728~16777216,155189248~16777216,176160768~8388608,188743680~29360128,226492416~37748736,268435456~83886080,360710144~16777216,385875968~16777216,406847488~8388608] 2024-04-07T17:38:36.688 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~25165824,33554432~16777216,54525952~12582912,71303168~29360128,109051904~4194304,117440512~12582912,134217728~16777216,155189248~16777216,176160768~8388608,188743680~29360128,226492416~37748736,268435456~83886080,360710144~16777216,385875968~16777216,406847488~8388608] 2024-04-07T17:38:36.702 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~29360128,33554432~33554432,71303168~33554432,109051904~4194304,117440512~54525952,176160768~41943040,226492416~130023424,360710144~16777216,385875968~16777216,406847488~12582912] 2024-04-07T17:38:36.702 INFO:tasks.workunit.client.0.smithi100.stdout:from 0 to 12 diff [0~25165824,33554432~16777216,54525952~12582912,71303168~29360128,109051904~20971520,134217728~16777216,155189248~16777216,176160768~8388608,188743680~29360128,226492416~37748736,268435456~109051904,381681664~20971520,406847488~8388608] 2024-04-07T17:38:36.702 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~25165824,33554432~16777216,54525952~12582912,71303168~29360128,109051904~20971520,134217728~16777216,155189248~16777216,176160768~8388608,188743680~29360128,226492416~37748736,268435456~109051904,381681664~20971520,406847488~8388608] 2024-04-07T17:38:36.716 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~29360128,33554432~33554432,71303168~33554432,109051904~62914560,176160768~41943040,226492416~150994944,381681664~20971520,406847488~12582912] 2024-04-07T17:38:36.716 INFO:tasks.workunit.client.0.smithi100.stdout:from 0 to 13 diff [0~25165824,33554432~16777216,54525952~46137344,104857600~25165824,134217728~16777216,155189248~16777216,176160768~8388608,188743680~29360128,226492416~37748736,268435456~109051904,381681664~20971520,406847488~8388608] 2024-04-07T17:38:36.716 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~25165824,33554432~16777216,54525952~46137344,104857600~25165824,134217728~16777216,155189248~16777216,176160768~8388608,188743680~29360128,226492416~37748736,268435456~109051904,381681664~20971520,406847488~8388608] 2024-04-07T17:38:36.731 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~29360128,33554432~138412032,176160768~41943040,226492416~150994944,381681664~20971520,406847488~12582912] 2024-04-07T17:38:36.731 INFO:tasks.workunit.client.0.smithi100.stdout:from 0 to 14 diff [0~25165824,33554432~16777216,54525952~46137344,104857600~25165824,134217728~16777216,155189248~16777216,176160768~41943040,226492416~37748736,268435456~109051904,381681664~20971520,406847488~8388608] 2024-04-07T17:38:36.731 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~25165824,33554432~16777216,54525952~46137344,104857600~25165824,134217728~16777216,155189248~16777216,176160768~41943040,226492416~37748736,268435456~109051904,381681664~20971520,406847488~8388608] 2024-04-07T17:38:36.747 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~29360128,33554432~138412032,176160768~41943040,226492416~150994944,381681664~20971520,406847488~12582912] 2024-04-07T17:38:36.747 INFO:tasks.workunit.client.0.smithi100.stdout:from 0 to 15 diff [0~25165824,33554432~16777216,54525952~46137344,104857600~25165824,134217728~16777216,155189248~16777216,176160768~41943040,226492416~37748736,268435456~46137344,318767104~83886080,406847488~8388608] 2024-04-07T17:38:36.747 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~25165824,33554432~16777216,54525952~46137344,104857600~25165824,134217728~16777216,155189248~16777216,176160768~41943040,226492416~37748736,268435456~46137344,318767104~83886080,406847488~8388608] 2024-04-07T17:38:36.763 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~29360128,33554432~138412032,176160768~41943040,226492416~176160768,406847488~12582912] 2024-04-07T17:38:36.763 INFO:tasks.workunit.client.0.smithi100.stdout:from 0 to 16 diff [0~25165824,33554432~16777216,54525952~96468992,155189248~16777216,176160768~41943040,226492416~37748736,268435456~88080384,360710144~41943040,406847488~8388608] 2024-04-07T17:38:36.763 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~25165824,33554432~16777216,54525952~96468992,155189248~16777216,176160768~41943040,226492416~37748736,268435456~88080384,360710144~41943040,406847488~8388608] 2024-04-07T17:38:36.781 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~29360128,33554432~138412032,176160768~41943040,226492416~176160768,406847488~12582912] 2024-04-07T17:38:36.781 INFO:tasks.workunit.client.0.smithi100.stdout:from 0 to 17 diff [0~29360128,33554432~16777216,54525952~96468992,155189248~16777216,176160768~41943040,226492416~37748736,268435456~150994944] 2024-04-07T17:38:36.781 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~16777216,54525952~96468992,155189248~16777216,176160768~41943040,226492416~37748736,268435456~150994944] 2024-04-07T17:38:36.799 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~218103808,226492416~192937984] 2024-04-07T17:38:36.799 INFO:tasks.workunit.client.0.smithi100.stdout:from 0 to 18 diff [0~29360128,33554432~16777216,54525952~96468992,155189248~16777216,176160768~88080384,268435456~150994944] 2024-04-07T17:38:36.799 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~16777216,54525952~96468992,155189248~16777216,176160768~88080384,268435456~150994944] 2024-04-07T17:38:36.819 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~419430400] 2024-04-07T17:38:36.819 INFO:tasks.workunit.client.0.smithi100.stdout:from 0 to 19 diff [0~29360128,33554432~16777216,54525952~96468992,155189248~109051904,268435456~150994944] 2024-04-07T17:38:36.819 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~16777216,54525952~96468992,155189248~109051904,268435456~150994944] 2024-04-07T17:38:36.839 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~419430400] 2024-04-07T17:38:36.840 INFO:tasks.workunit.client.0.smithi100.stdout:from 1 to 2 diff [4194304~8388608,54525952~8388608,75497472~4194304,121634816~12582912,159383552~8388608,293601280~4194304,322961408~8388608,339738624~8388608] 2024-04-07T17:38:36.840 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [4194304~8388608,54525952~8388608,75497472~4194304,121634816~12582912,159383552~8388608,293601280~4194304,322961408~8388608,339738624~8388608] 2024-04-07T17:38:36.843 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [4194304~8388608,54525952~8388608,75497472~4194304,121634816~12582912,159383552~12582912,293601280~4194304,322961408~12582912,339738624~12582912] 2024-04-07T17:38:36.843 INFO:tasks.workunit.client.0.smithi100.stdout:from 1 to 3 diff [0~16777216,54525952~8388608,75497472~4194304,121634816~12582912,159383552~8388608,188743680~20971520,251658240~8388608,293601280~4194304,322961408~12582912,339738624~8388608,394264576~8388608] 2024-04-07T17:38:36.843 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~16777216,54525952~8388608,75497472~4194304,121634816~12582912,159383552~8388608,188743680~20971520,251658240~8388608,293601280~4194304,322961408~12582912,339738624~8388608,394264576~8388608] 2024-04-07T17:38:36.847 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~20971520,54525952~8388608,75497472~4194304,121634816~12582912,159383552~12582912,188743680~25165824,251658240~12582912,293601280~4194304,322961408~12582912,339738624~12582912,394264576~8388608] 2024-04-07T17:38:36.847 INFO:tasks.workunit.client.0.smithi100.stdout:from 1 to 4 diff [0~16777216,54525952~8388608,71303168~8388608,96468992~4194304,121634816~12582912,159383552~8388608,176160768~8388608,188743680~20971520,230686720~16777216,251658240~8388608,289406976~8388608,322961408~29360128,360710144~8388608,394264576~8388608] 2024-04-07T17:38:36.847 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~16777216,54525952~8388608,71303168~8388608,96468992~4194304,121634816~12582912,159383552~8388608,176160768~8388608,188743680~20971520,230686720~16777216,251658240~8388608,289406976~8388608,322961408~29360128,360710144~8388608,394264576~8388608] 2024-04-07T17:38:36.852 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~20971520,54525952~8388608,71303168~8388608,96468992~4194304,121634816~12582912,159383552~12582912,176160768~37748736,230686720~33554432,289406976~8388608,322961408~33554432,360710144~12582912,394264576~8388608] 2024-04-07T17:38:36.852 INFO:tasks.workunit.client.0.smithi100.stdout:from 1 to 5 diff [0~16777216,54525952~8388608,71303168~8388608,96468992~4194304,121634816~12582912,159383552~8388608,176160768~8388608,188743680~20971520,230686720~16777216,251658240~8388608,289406976~8388608,306184192~8388608,322961408~29360128,360710144~12582912,394264576~8388608,406847488~8388608] 2024-04-07T17:38:36.852 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~16777216,54525952~8388608,71303168~8388608,96468992~4194304,121634816~12582912,159383552~8388608,176160768~8388608,188743680~20971520,230686720~16777216,251658240~8388608,289406976~8388608,306184192~8388608,322961408~29360128,360710144~12582912,394264576~8388608,406847488~8388608] 2024-04-07T17:38:36.858 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~20971520,54525952~8388608,71303168~8388608,96468992~4194304,121634816~12582912,159383552~12582912,176160768~37748736,230686720~33554432,289406976~8388608,306184192~12582912,322961408~33554432,360710144~16777216,394264576~8388608,406847488~12582912] 2024-04-07T17:38:36.858 INFO:tasks.workunit.client.0.smithi100.stdout:from 1 to 6 diff [0~16777216,41943040~4194304,54525952~8388608,71303168~8388608,92274688~8388608,117440512~12582912,159383552~8388608,176160768~8388608,188743680~20971520,230686720~16777216,251658240~8388608,272629760~8388608,289406976~8388608,306184192~46137344,360710144~12582912,394264576~8388608,406847488~8388608] 2024-04-07T17:38:36.858 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~16777216,41943040~4194304,54525952~8388608,71303168~8388608,92274688~8388608,117440512~12582912,159383552~8388608,176160768~8388608,188743680~20971520,230686720~16777216,251658240~8388608,272629760~8388608,289406976~8388608,306184192~46137344,360710144~12582912,394264576~8388608,406847488~8388608] 2024-04-07T17:38:36.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:38:36 smithi144 bash[66654]: cluster 2024-04-07T17:38:35.538642+0000 mon.a (mon.0) 3973 : cluster [DBG] osdmap e2079: 8 total, 8 up, 8 in 2024-04-07T17:38:36.865 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~20971520,41943040~4194304,54525952~8388608,71303168~12582912,92274688~8388608,117440512~16777216,159383552~12582912,176160768~37748736,230686720~33554432,272629760~8388608,289406976~8388608,306184192~50331648,360710144~16777216,394264576~8388608,406847488~12582912] 2024-04-07T17:38:36.865 INFO:tasks.workunit.client.0.smithi100.stdout:from 1 to 7 diff [0~20971520,33554432~12582912,54525952~8388608,71303168~8388608,83886080~16777216,117440512~12582912,146800640~4194304,159383552~8388608,176160768~8388608,188743680~20971520,230686720~29360128,272629760~25165824,306184192~46137344,360710144~12582912,394264576~8388608,406847488~8388608] 2024-04-07T17:38:36.866 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~20971520,33554432~12582912,54525952~8388608,71303168~8388608,83886080~16777216,117440512~12582912,146800640~4194304,159383552~8388608,176160768~8388608,188743680~20971520,230686720~29360128,272629760~25165824,306184192~46137344,360710144~12582912,394264576~8388608,406847488~8388608] 2024-04-07T17:38:36.873 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~20971520,33554432~12582912,54525952~8388608,71303168~29360128,117440512~16777216,146800640~8388608,159383552~12582912,176160768~37748736,230686720~33554432,272629760~25165824,306184192~50331648,360710144~16777216,394264576~8388608,406847488~12582912] 2024-04-07T17:38:36.873 INFO:tasks.workunit.client.0.smithi100.stdout:from 1 to 8 diff [0~20971520,33554432~12582912,54525952~8388608,71303168~29360128,117440512~12582912,146800640~4194304,159383552~8388608,176160768~8388608,188743680~20971520,213909504~4194304,230686720~29360128,268435456~29360128,306184192~46137344,360710144~16777216,394264576~8388608,406847488~8388608] 2024-04-07T17:38:36.873 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~20971520,33554432~12582912,54525952~8388608,71303168~29360128,117440512~12582912,146800640~4194304,159383552~8388608,176160768~8388608,188743680~20971520,213909504~4194304,230686720~29360128,268435456~29360128,306184192~46137344,360710144~16777216,394264576~8388608,406847488~8388608] 2024-04-07T17:38:36.882 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~20971520,33554432~12582912,54525952~8388608,71303168~29360128,117440512~16777216,146800640~8388608,159383552~12582912,176160768~41943040,230686720~33554432,268435456~29360128,306184192~50331648,360710144~16777216,394264576~8388608,406847488~12582912] 2024-04-07T17:38:36.882 INFO:tasks.workunit.client.0.smithi100.stdout:from 1 to 9 diff [0~20971520,33554432~12582912,54525952~8388608,71303168~29360128,109051904~4194304,117440512~12582912,134217728~4194304,146800640~4194304,159383552~8388608,176160768~8388608,188743680~29360128,230686720~29360128,268435456~29360128,306184192~46137344,360710144~16777216,390070272~12582912,406847488~8388608] 2024-04-07T17:38:36.882 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~20971520,33554432~12582912,54525952~8388608,71303168~29360128,109051904~4194304,117440512~12582912,134217728~4194304,146800640~4194304,159383552~8388608,176160768~8388608,188743680~29360128,230686720~29360128,268435456~29360128,306184192~46137344,360710144~16777216,390070272~12582912,406847488~8388608] 2024-04-07T17:38:36.892 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~20971520,33554432~12582912,54525952~8388608,71303168~33554432,109051904~4194304,117440512~25165824,146800640~8388608,159383552~12582912,176160768~41943040,230686720~33554432,268435456~29360128,306184192~50331648,360710144~16777216,390070272~12582912,406847488~12582912] 2024-04-07T17:38:36.892 INFO:tasks.workunit.client.0.smithi100.stdout:from 1 to 10 diff [0~20971520,33554432~16777216,54525952~8388608,71303168~29360128,109051904~4194304,117440512~12582912,134217728~4194304,146800640~4194304,159383552~8388608,176160768~8388608,188743680~29360128,230686720~29360128,268435456~29360128,301989888~50331648,360710144~16777216,385875968~16777216,406847488~8388608] 2024-04-07T17:38:36.892 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~20971520,33554432~16777216,54525952~8388608,71303168~29360128,109051904~4194304,117440512~12582912,134217728~4194304,146800640~4194304,159383552~8388608,176160768~8388608,188743680~29360128,230686720~29360128,268435456~29360128,301989888~50331648,360710144~16777216,385875968~16777216,406847488~8388608] 2024-04-07T17:38:36.903 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~20971520,33554432~29360128,71303168~33554432,109051904~4194304,117440512~25165824,146800640~8388608,159383552~12582912,176160768~41943040,230686720~33554432,268435456~29360128,301989888~54525952,360710144~16777216,385875968~16777216,406847488~12582912] 2024-04-07T17:38:36.903 INFO:tasks.workunit.client.0.smithi100.stdout:from 1 to 11 diff [0~20971520,33554432~16777216,54525952~8388608,71303168~29360128,109051904~4194304,117440512~12582912,134217728~4194304,146800640~4194304,159383552~8388608,176160768~8388608,188743680~29360128,230686720~33554432,268435456~29360128,301989888~50331648,360710144~16777216,385875968~16777216,406847488~8388608] 2024-04-07T17:38:36.903 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~20971520,33554432~16777216,54525952~8388608,71303168~29360128,109051904~4194304,117440512~12582912,134217728~4194304,146800640~4194304,159383552~8388608,176160768~8388608,188743680~29360128,230686720~33554432,268435456~29360128,301989888~50331648,360710144~16777216,385875968~16777216,406847488~8388608] 2024-04-07T17:38:36.915 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~20971520,33554432~29360128,71303168~33554432,109051904~4194304,117440512~25165824,146800640~8388608,159383552~12582912,176160768~41943040,230686720~67108864,301989888~54525952,360710144~16777216,385875968~16777216,406847488~12582912] 2024-04-07T17:38:36.915 INFO:tasks.workunit.client.0.smithi100.stdout:from 1 to 12 diff [0~20971520,33554432~16777216,54525952~8388608,71303168~29360128,109051904~20971520,134217728~4194304,146800640~4194304,159383552~8388608,176160768~8388608,188743680~29360128,230686720~33554432,268435456~29360128,301989888~75497472,381681664~20971520,406847488~8388608] 2024-04-07T17:38:36.916 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~20971520,33554432~16777216,54525952~8388608,71303168~29360128,109051904~20971520,134217728~4194304,146800640~4194304,159383552~8388608,176160768~8388608,188743680~29360128,230686720~33554432,268435456~29360128,301989888~75497472,381681664~20971520,406847488~8388608] 2024-04-07T17:38:36.929 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~20971520,33554432~29360128,71303168~33554432,109051904~33554432,146800640~8388608,159383552~12582912,176160768~41943040,230686720~67108864,301989888~75497472,381681664~20971520,406847488~12582912] 2024-04-07T17:38:36.929 INFO:tasks.workunit.client.0.smithi100.stdout:from 1 to 13 diff [0~20971520,33554432~16777216,54525952~46137344,104857600~25165824,134217728~4194304,142606336~8388608,159383552~8388608,176160768~8388608,188743680~29360128,230686720~33554432,268435456~29360128,301989888~75497472,381681664~20971520,406847488~8388608] 2024-04-07T17:38:36.929 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~20971520,33554432~16777216,54525952~46137344,104857600~25165824,134217728~4194304,142606336~8388608,159383552~8388608,176160768~8388608,188743680~29360128,230686720~33554432,268435456~29360128,301989888~75497472,381681664~20971520,406847488~8388608] 2024-04-07T17:38:36.942 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~20971520,33554432~121634816,159383552~12582912,176160768~41943040,230686720~67108864,301989888~75497472,381681664~20971520,406847488~12582912] 2024-04-07T17:38:36.942 INFO:tasks.workunit.client.0.smithi100.stdout:from 1 to 14 diff [0~20971520,33554432~16777216,54525952~46137344,104857600~25165824,134217728~4194304,142606336~12582912,159383552~8388608,176160768~41943040,226492416~37748736,268435456~29360128,301989888~75497472,381681664~20971520,406847488~8388608] 2024-04-07T17:38:36.942 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~20971520,33554432~16777216,54525952~46137344,104857600~25165824,134217728~4194304,142606336~12582912,159383552~8388608,176160768~41943040,226492416~37748736,268435456~29360128,301989888~75497472,381681664~20971520,406847488~8388608] 2024-04-07T17:38:36.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:38:36 smithi100 bash[74942]: cluster 2024-04-07T17:38:35.538642+0000 mon.a (mon.0) 3973 : cluster [DBG] osdmap e2079: 8 total, 8 up, 8 in 2024-04-07T17:38:36.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:38:36 smithi100 bash[76202]: cluster 2024-04-07T17:38:35.538642+0000 mon.a (mon.0) 3973 : cluster [DBG] osdmap e2079: 8 total, 8 up, 8 in 2024-04-07T17:38:36.957 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~20971520,33554432~121634816,159383552~12582912,176160768~41943040,226492416~71303168,301989888~75497472,381681664~20971520,406847488~12582912] 2024-04-07T17:38:36.957 INFO:tasks.workunit.client.0.smithi100.stdout:from 1 to 15 diff [0~20971520,33554432~16777216,54525952~46137344,104857600~25165824,134217728~4194304,142606336~12582912,159383552~8388608,176160768~41943040,226492416~37748736,268435456~46137344,318767104~83886080,406847488~8388608] 2024-04-07T17:38:36.957 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~20971520,33554432~16777216,54525952~46137344,104857600~25165824,134217728~4194304,142606336~12582912,159383552~8388608,176160768~41943040,226492416~37748736,268435456~46137344,318767104~83886080,406847488~8388608] 2024-04-07T17:38:36.972 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~20971520,33554432~121634816,159383552~12582912,176160768~41943040,226492416~176160768,406847488~12582912] 2024-04-07T17:38:36.972 INFO:tasks.workunit.client.0.smithi100.stdout:from 1 to 16 diff [0~20971520,33554432~16777216,54525952~83886080,142606336~12582912,159383552~12582912,176160768~41943040,226492416~37748736,268435456~88080384,360710144~41943040,406847488~8388608] 2024-04-07T17:38:36.972 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~20971520,33554432~16777216,54525952~83886080,142606336~12582912,159383552~12582912,176160768~41943040,226492416~37748736,268435456~88080384,360710144~41943040,406847488~8388608] 2024-04-07T17:38:36.988 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~20971520,33554432~121634816,159383552~12582912,176160768~41943040,226492416~176160768,406847488~12582912] 2024-04-07T17:38:36.988 INFO:tasks.workunit.client.0.smithi100.stdout:from 1 to 17 diff [0~29360128,33554432~16777216,54525952~100663296,159383552~12582912,176160768~41943040,226492416~37748736,268435456~150994944] 2024-04-07T17:38:36.988 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~16777216,54525952~100663296,159383552~12582912,176160768~41943040,226492416~37748736,268435456~150994944] 2024-04-07T17:38:37.007 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~155189248,159383552~58720256,226492416~192937984] 2024-04-07T17:38:37.007 INFO:tasks.workunit.client.0.smithi100.stdout:from 1 to 18 diff [0~29360128,33554432~16777216,54525952~117440512,176160768~88080384,268435456~150994944] 2024-04-07T17:38:37.007 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~16777216,54525952~117440512,176160768~88080384,268435456~150994944] 2024-04-07T17:38:37.027 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~419430400] 2024-04-07T17:38:37.027 INFO:tasks.workunit.client.0.smithi100.stdout:from 1 to 19 diff [0~29360128,33554432~16777216,54525952~209715200,268435456~150994944] 2024-04-07T17:38:37.028 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~16777216,54525952~209715200,268435456~150994944] 2024-04-07T17:38:37.046 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~419430400] 2024-04-07T17:38:37.046 INFO:tasks.workunit.client.0.smithi100.stdout:from 2 to 3 diff [0~16777216,125829120~4194304,188743680~20971520,251658240~8388608,327155712~8388608,394264576~8388608] 2024-04-07T17:38:37.046 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~16777216,125829120~4194304,188743680~20971520,251658240~8388608,327155712~8388608,394264576~8388608] 2024-04-07T17:38:37.051 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~20971520,125829120~8388608,188743680~25165824,251658240~12582912,327155712~8388608,394264576~8388608] 2024-04-07T17:38:37.051 INFO:tasks.workunit.client.0.smithi100.stdout:from 2 to 4 diff [0~16777216,71303168~8388608,96468992~4194304,125829120~4194304,176160768~8388608,188743680~20971520,230686720~16777216,251658240~8388608,289406976~4194304,327155712~29360128,360710144~8388608,394264576~8388608] 2024-04-07T17:38:37.051 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~16777216,71303168~8388608,96468992~4194304,125829120~4194304,176160768~8388608,188743680~20971520,230686720~16777216,251658240~8388608,289406976~4194304,327155712~29360128,360710144~8388608,394264576~8388608] 2024-04-07T17:38:37.054 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~20971520,71303168~8388608,96468992~4194304,125829120~8388608,176160768~37748736,230686720~33554432,289406976~8388608,327155712~29360128,360710144~12582912,394264576~8388608] 2024-04-07T17:38:37.054 INFO:tasks.workunit.client.0.smithi100.stdout:from 2 to 5 diff [0~16777216,71303168~8388608,96468992~4194304,121634816~8388608,176160768~8388608,188743680~20971520,230686720~16777216,251658240~8388608,289406976~4194304,306184192~8388608,327155712~29360128,360710144~12582912,394264576~8388608,406847488~8388608] 2024-04-07T17:38:37.054 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~16777216,71303168~8388608,96468992~4194304,121634816~8388608,176160768~8388608,188743680~20971520,230686720~16777216,251658240~8388608,289406976~4194304,306184192~8388608,327155712~29360128,360710144~12582912,394264576~8388608,406847488~8388608] 2024-04-07T17:38:37.059 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~20971520,71303168~8388608,96468992~4194304,121634816~12582912,176160768~37748736,230686720~33554432,289406976~8388608,306184192~12582912,327155712~29360128,360710144~16777216,394264576~8388608,406847488~12582912] 2024-04-07T17:38:37.059 INFO:tasks.workunit.client.0.smithi100.stdout:from 2 to 6 diff [0~16777216,41943040~4194304,71303168~8388608,92274688~8388608,117440512~12582912,176160768~8388608,188743680~20971520,230686720~16777216,251658240~8388608,272629760~8388608,289406976~4194304,306184192~50331648,360710144~12582912,394264576~8388608,406847488~8388608] 2024-04-07T17:38:37.059 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~16777216,41943040~4194304,71303168~8388608,92274688~8388608,117440512~12582912,176160768~8388608,188743680~20971520,230686720~16777216,251658240~8388608,272629760~8388608,289406976~4194304,306184192~50331648,360710144~12582912,394264576~8388608,406847488~8388608] 2024-04-07T17:38:37.065 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~20971520,41943040~4194304,71303168~12582912,92274688~8388608,117440512~16777216,176160768~37748736,230686720~33554432,272629760~8388608,289406976~8388608,306184192~50331648,360710144~16777216,394264576~8388608,406847488~12582912] 2024-04-07T17:38:37.065 INFO:tasks.workunit.client.0.smithi100.stdout:from 2 to 7 diff [0~20971520,33554432~12582912,71303168~8388608,83886080~16777216,117440512~12582912,146800640~4194304,176160768~8388608,188743680~20971520,230686720~29360128,272629760~20971520,306184192~50331648,360710144~12582912,394264576~8388608,406847488~8388608] 2024-04-07T17:38:37.065 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~20971520,33554432~12582912,71303168~8388608,83886080~16777216,117440512~12582912,146800640~4194304,176160768~8388608,188743680~20971520,230686720~29360128,272629760~20971520,306184192~50331648,360710144~12582912,394264576~8388608,406847488~8388608] 2024-04-07T17:38:37.072 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~20971520,33554432~12582912,71303168~29360128,117440512~16777216,146800640~8388608,176160768~37748736,230686720~33554432,272629760~25165824,306184192~50331648,360710144~16777216,394264576~8388608,406847488~12582912] 2024-04-07T17:38:37.072 INFO:tasks.workunit.client.0.smithi100.stdout:from 2 to 8 diff [0~20971520,33554432~12582912,71303168~29360128,117440512~12582912,146800640~4194304,176160768~8388608,188743680~20971520,213909504~4194304,230686720~29360128,268435456~25165824,306184192~50331648,360710144~16777216,394264576~8388608,406847488~8388608] 2024-04-07T17:38:37.072 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~20971520,33554432~12582912,71303168~29360128,117440512~12582912,146800640~4194304,176160768~8388608,188743680~20971520,213909504~4194304,230686720~29360128,268435456~25165824,306184192~50331648,360710144~16777216,394264576~8388608,406847488~8388608] 2024-04-07T17:38:37.081 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~20971520,33554432~12582912,71303168~29360128,117440512~16777216,146800640~8388608,176160768~41943040,230686720~33554432,268435456~29360128,306184192~50331648,360710144~16777216,394264576~8388608,406847488~12582912] 2024-04-07T17:38:37.081 INFO:tasks.workunit.client.0.smithi100.stdout:from 2 to 9 diff [0~20971520,33554432~12582912,71303168~29360128,109051904~4194304,117440512~12582912,134217728~4194304,146800640~4194304,176160768~8388608,188743680~29360128,230686720~29360128,268435456~25165824,306184192~50331648,360710144~16777216,390070272~12582912,406847488~8388608] 2024-04-07T17:38:37.081 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~20971520,33554432~12582912,71303168~29360128,109051904~4194304,117440512~12582912,134217728~4194304,146800640~4194304,176160768~8388608,188743680~29360128,230686720~29360128,268435456~25165824,306184192~50331648,360710144~16777216,390070272~12582912,406847488~8388608] 2024-04-07T17:38:37.091 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~20971520,33554432~12582912,71303168~33554432,109051904~4194304,117440512~25165824,146800640~8388608,176160768~41943040,230686720~33554432,268435456~29360128,306184192~50331648,360710144~16777216,390070272~12582912,406847488~12582912] 2024-04-07T17:38:37.091 INFO:tasks.workunit.client.0.smithi100.stdout:from 2 to 10 diff [0~20971520,33554432~16777216,71303168~29360128,109051904~4194304,117440512~12582912,134217728~4194304,146800640~4194304,167772160~4194304,176160768~8388608,188743680~29360128,230686720~29360128,268435456~25165824,301989888~54525952,360710144~16777216,385875968~16777216,406847488~8388608] 2024-04-07T17:38:37.091 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~20971520,33554432~16777216,71303168~29360128,109051904~4194304,117440512~12582912,134217728~4194304,146800640~4194304,167772160~4194304,176160768~8388608,188743680~29360128,230686720~29360128,268435456~25165824,301989888~54525952,360710144~16777216,385875968~16777216,406847488~8388608] 2024-04-07T17:38:37.100 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~20971520,33554432~20971520,71303168~33554432,109051904~4194304,117440512~25165824,146800640~8388608,167772160~4194304,176160768~41943040,230686720~33554432,268435456~29360128,301989888~54525952,360710144~16777216,385875968~16777216,406847488~12582912] 2024-04-07T17:38:37.100 INFO:tasks.workunit.client.0.smithi100.stdout:from 2 to 11 diff [0~20971520,33554432~16777216,58720256~4194304,71303168~29360128,109051904~4194304,117440512~12582912,134217728~4194304,146800640~4194304,167772160~4194304,176160768~8388608,188743680~29360128,230686720~33554432,268435456~29360128,301989888~54525952,360710144~16777216,385875968~16777216,406847488~8388608] 2024-04-07T17:38:37.100 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~20971520,33554432~16777216,58720256~4194304,71303168~29360128,109051904~4194304,117440512~12582912,134217728~4194304,146800640~4194304,167772160~4194304,176160768~8388608,188743680~29360128,230686720~33554432,268435456~29360128,301989888~54525952,360710144~16777216,385875968~16777216,406847488~8388608] 2024-04-07T17:38:37.112 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~20971520,33554432~20971520,58720256~4194304,71303168~33554432,109051904~4194304,117440512~25165824,146800640~8388608,167772160~4194304,176160768~41943040,230686720~67108864,301989888~54525952,360710144~16777216,385875968~16777216,406847488~12582912] 2024-04-07T17:38:37.112 INFO:tasks.workunit.client.0.smithi100.stdout:from 2 to 12 diff [0~20971520,33554432~16777216,54525952~4194304,71303168~29360128,109051904~20971520,134217728~4194304,146800640~4194304,167772160~4194304,176160768~8388608,188743680~29360128,230686720~33554432,268435456~29360128,301989888~54525952,360710144~16777216,381681664~20971520,406847488~8388608] 2024-04-07T17:38:37.112 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~20971520,33554432~16777216,54525952~4194304,71303168~29360128,109051904~20971520,134217728~4194304,146800640~4194304,167772160~4194304,176160768~8388608,188743680~29360128,230686720~33554432,268435456~29360128,301989888~54525952,360710144~16777216,381681664~20971520,406847488~8388608] 2024-04-07T17:38:37.124 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~20971520,33554432~29360128,71303168~33554432,109051904~33554432,146800640~8388608,167772160~4194304,176160768~41943040,230686720~67108864,301989888~75497472,381681664~20971520,406847488~12582912] 2024-04-07T17:38:37.124 INFO:tasks.workunit.client.0.smithi100.stdout:from 2 to 13 diff [0~20971520,33554432~16777216,54525952~4194304,62914560~37748736,104857600~25165824,134217728~4194304,142606336~8388608,167772160~4194304,176160768~8388608,188743680~29360128,230686720~33554432,268435456~29360128,301989888~54525952,360710144~16777216,381681664~20971520,406847488~8388608] 2024-04-07T17:38:37.124 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~20971520,33554432~16777216,54525952~4194304,62914560~37748736,104857600~25165824,134217728~4194304,142606336~8388608,167772160~4194304,176160768~8388608,188743680~29360128,230686720~33554432,268435456~29360128,301989888~54525952,360710144~16777216,381681664~20971520,406847488~8388608] 2024-04-07T17:38:37.137 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~20971520,33554432~121634816,167772160~4194304,176160768~41943040,230686720~67108864,301989888~75497472,381681664~20971520,406847488~12582912] 2024-04-07T17:38:37.137 INFO:tasks.workunit.client.0.smithi100.stdout:from 2 to 14 diff [0~20971520,33554432~16777216,54525952~4194304,62914560~37748736,104857600~25165824,134217728~4194304,142606336~12582912,163577856~8388608,176160768~41943040,226492416~37748736,268435456~29360128,301989888~54525952,360710144~16777216,381681664~20971520,406847488~8388608] 2024-04-07T17:38:37.137 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~20971520,33554432~16777216,54525952~4194304,62914560~37748736,104857600~25165824,134217728~4194304,142606336~12582912,163577856~8388608,176160768~41943040,226492416~37748736,268435456~29360128,301989888~54525952,360710144~16777216,381681664~20971520,406847488~8388608] 2024-04-07T17:38:37.152 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~20971520,33554432~121634816,163577856~8388608,176160768~41943040,226492416~71303168,301989888~75497472,381681664~20971520,406847488~12582912] 2024-04-07T17:38:37.152 INFO:tasks.workunit.client.0.smithi100.stdout:from 2 to 15 diff [0~20971520,33554432~16777216,54525952~4194304,62914560~37748736,104857600~25165824,134217728~4194304,142606336~12582912,163577856~8388608,176160768~41943040,226492416~37748736,268435456~46137344,318767104~37748736,360710144~41943040,406847488~8388608] 2024-04-07T17:38:37.152 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~20971520,33554432~16777216,54525952~4194304,62914560~37748736,104857600~25165824,134217728~4194304,142606336~12582912,163577856~8388608,176160768~41943040,226492416~37748736,268435456~46137344,318767104~37748736,360710144~41943040,406847488~8388608] 2024-04-07T17:38:37.166 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~20971520,33554432~121634816,163577856~8388608,176160768~41943040,226492416~176160768,406847488~12582912] 2024-04-07T17:38:37.166 INFO:tasks.workunit.client.0.smithi100.stdout:from 2 to 16 diff [0~20971520,33554432~16777216,54525952~4194304,62914560~75497472,142606336~12582912,159383552~12582912,176160768~41943040,226492416~37748736,268435456~88080384,360710144~41943040,406847488~8388608] 2024-04-07T17:38:37.167 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~20971520,33554432~16777216,54525952~4194304,62914560~75497472,142606336~12582912,159383552~12582912,176160768~41943040,226492416~37748736,268435456~88080384,360710144~41943040,406847488~8388608] 2024-04-07T17:38:37.184 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~20971520,33554432~121634816,159383552~12582912,176160768~41943040,226492416~176160768,406847488~12582912] 2024-04-07T17:38:37.184 INFO:tasks.workunit.client.0.smithi100.stdout:from 2 to 17 diff [0~29360128,33554432~16777216,54525952~4194304,62914560~92274688,159383552~12582912,176160768~41943040,226492416~37748736,268435456~150994944] 2024-04-07T17:38:37.185 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~16777216,54525952~4194304,62914560~92274688,159383552~12582912,176160768~41943040,226492416~37748736,268435456~150994944] 2024-04-07T17:38:37.201 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~155189248,159383552~58720256,226492416~192937984] 2024-04-07T17:38:37.201 INFO:tasks.workunit.client.0.smithi100.stdout:from 2 to 18 diff [0~29360128,33554432~16777216,54525952~117440512,176160768~88080384,268435456~150994944] 2024-04-07T17:38:37.202 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~16777216,54525952~117440512,176160768~88080384,268435456~150994944] 2024-04-07T17:38:37.219 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~419430400] 2024-04-07T17:38:37.219 INFO:tasks.workunit.client.0.smithi100.stdout:from 2 to 19 diff [0~29360128,33554432~16777216,54525952~209715200,268435456~150994944] 2024-04-07T17:38:37.219 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~16777216,54525952~209715200,268435456~150994944] 2024-04-07T17:38:37.238 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~419430400] 2024-04-07T17:38:37.238 INFO:tasks.workunit.client.0.smithi100.stdout:from 3 to 4 diff [71303168~8388608,96468992~4194304,176160768~8388608,230686720~16777216,289406976~4194304,335544320~20971520,360710144~8388608] 2024-04-07T17:38:37.239 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [71303168~8388608,96468992~4194304,176160768~8388608,230686720~16777216,289406976~4194304,335544320~20971520,360710144~8388608] 2024-04-07T17:38:37.241 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [71303168~8388608,96468992~4194304,176160768~12582912,230686720~20971520,289406976~8388608,335544320~20971520,360710144~12582912] 2024-04-07T17:38:37.241 INFO:tasks.workunit.client.0.smithi100.stdout:from 3 to 5 diff [0~8388608,71303168~8388608,96468992~4194304,121634816~4194304,176160768~8388608,188743680~12582912,230686720~16777216,289406976~4194304,306184192~8388608,335544320~20971520,360710144~12582912,406847488~8388608] 2024-04-07T17:38:37.241 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~8388608,71303168~8388608,96468992~4194304,121634816~4194304,176160768~8388608,188743680~12582912,230686720~16777216,289406976~4194304,306184192~8388608,335544320~20971520,360710144~12582912,406847488~8388608] 2024-04-07T17:38:37.245 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~12582912,71303168~8388608,96468992~4194304,121634816~8388608,176160768~29360128,230686720~20971520,289406976~8388608,306184192~12582912,335544320~20971520,360710144~16777216,406847488~12582912] 2024-04-07T17:38:37.246 INFO:tasks.workunit.client.0.smithi100.stdout:from 3 to 6 diff [0~8388608,41943040~4194304,71303168~8388608,92274688~8388608,117440512~8388608,176160768~8388608,188743680~16777216,230686720~16777216,272629760~8388608,289406976~4194304,306184192~20971520,335544320~20971520,360710144~12582912,406847488~8388608] 2024-04-07T17:38:37.246 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~8388608,41943040~4194304,71303168~8388608,92274688~8388608,117440512~8388608,176160768~8388608,188743680~16777216,230686720~16777216,272629760~8388608,289406976~4194304,306184192~20971520,335544320~20971520,360710144~12582912,406847488~8388608] 2024-04-07T17:38:37.251 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~12582912,41943040~4194304,71303168~12582912,92274688~8388608,117440512~12582912,176160768~29360128,230686720~20971520,272629760~8388608,289406976~8388608,306184192~25165824,335544320~20971520,360710144~16777216,406847488~12582912] 2024-04-07T17:38:37.251 INFO:tasks.workunit.client.0.smithi100.stdout:from 3 to 7 diff [0~12582912,33554432~12582912,71303168~8388608,83886080~16777216,117440512~8388608,146800640~4194304,176160768~8388608,188743680~16777216,230686720~20971520,272629760~20971520,306184192~20971520,335544320~20971520,360710144~12582912,406847488~8388608] 2024-04-07T17:38:37.251 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~12582912,33554432~12582912,71303168~8388608,83886080~16777216,117440512~8388608,146800640~4194304,176160768~8388608,188743680~16777216,230686720~20971520,272629760~20971520,306184192~20971520,335544320~20971520,360710144~12582912,406847488~8388608] 2024-04-07T17:38:37.257 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~16777216,33554432~12582912,71303168~29360128,117440512~12582912,146800640~8388608,176160768~29360128,230686720~20971520,272629760~25165824,306184192~25165824,335544320~20971520,360710144~16777216,406847488~12582912] 2024-04-07T17:38:37.257 INFO:tasks.workunit.client.0.smithi100.stdout:from 3 to 8 diff [0~12582912,16777216~4194304,33554432~12582912,71303168~29360128,117440512~8388608,146800640~4194304,176160768~8388608,188743680~16777216,213909504~4194304,230686720~20971520,268435456~25165824,306184192~20971520,331350016~25165824,360710144~16777216,406847488~8388608] 2024-04-07T17:38:37.257 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~12582912,16777216~4194304,33554432~12582912,71303168~29360128,117440512~8388608,146800640~4194304,176160768~8388608,188743680~16777216,213909504~4194304,230686720~20971520,268435456~25165824,306184192~20971520,331350016~25165824,360710144~16777216,406847488~8388608] 2024-04-07T17:38:37.265 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~20971520,33554432~12582912,71303168~29360128,117440512~12582912,146800640~8388608,176160768~29360128,213909504~4194304,230686720~20971520,268435456~29360128,306184192~50331648,360710144~16777216,406847488~12582912] 2024-04-07T17:38:37.265 INFO:tasks.workunit.client.0.smithi100.stdout:from 3 to 9 diff [0~12582912,16777216~4194304,33554432~12582912,71303168~29360128,109051904~4194304,117440512~8388608,134217728~4194304,146800640~4194304,176160768~8388608,188743680~29360128,230686720~20971520,268435456~25165824,306184192~20971520,331350016~25165824,360710144~16777216,390070272~4194304,406847488~8388608] 2024-04-07T17:38:37.265 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~12582912,16777216~4194304,33554432~12582912,71303168~29360128,109051904~4194304,117440512~8388608,134217728~4194304,146800640~4194304,176160768~8388608,188743680~29360128,230686720~20971520,268435456~25165824,306184192~20971520,331350016~25165824,360710144~16777216,390070272~4194304,406847488~8388608] 2024-04-07T17:38:37.274 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~20971520,33554432~12582912,71303168~33554432,109051904~4194304,117440512~12582912,134217728~8388608,146800640~8388608,176160768~41943040,230686720~20971520,268435456~29360128,306184192~50331648,360710144~16777216,390070272~4194304,406847488~12582912] 2024-04-07T17:38:37.274 INFO:tasks.workunit.client.0.smithi100.stdout:from 3 to 10 diff [0~20971520,33554432~16777216,71303168~29360128,109051904~4194304,117440512~8388608,134217728~4194304,146800640~4194304,167772160~4194304,176160768~8388608,188743680~29360128,230686720~20971520,268435456~25165824,301989888~25165824,331350016~25165824,360710144~16777216,385875968~8388608,406847488~8388608] 2024-04-07T17:38:37.274 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~20971520,33554432~16777216,71303168~29360128,109051904~4194304,117440512~8388608,134217728~4194304,146800640~4194304,167772160~4194304,176160768~8388608,188743680~29360128,230686720~20971520,268435456~25165824,301989888~25165824,331350016~25165824,360710144~16777216,385875968~8388608,406847488~8388608] 2024-04-07T17:38:37.284 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~20971520,33554432~20971520,71303168~33554432,109051904~4194304,117440512~12582912,134217728~8388608,146800640~8388608,167772160~4194304,176160768~41943040,230686720~20971520,268435456~29360128,301989888~54525952,360710144~16777216,385875968~8388608,406847488~12582912] 2024-04-07T17:38:37.284 INFO:tasks.workunit.client.0.smithi100.stdout:from 3 to 11 diff [0~20971520,33554432~16777216,58720256~4194304,71303168~29360128,109051904~4194304,117440512~8388608,134217728~4194304,146800640~4194304,167772160~4194304,176160768~8388608,188743680~29360128,230686720~20971520,255852544~8388608,268435456~29360128,301989888~25165824,331350016~25165824,360710144~16777216,385875968~8388608,406847488~8388608] 2024-04-07T17:38:37.284 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~20971520,33554432~16777216,58720256~4194304,71303168~29360128,109051904~4194304,117440512~8388608,134217728~4194304,146800640~4194304,167772160~4194304,176160768~8388608,188743680~29360128,230686720~20971520,255852544~8388608,268435456~29360128,301989888~25165824,331350016~25165824,360710144~16777216,385875968~8388608,406847488~8388608] 2024-04-07T17:38:37.294 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~20971520,33554432~20971520,58720256~4194304,71303168~33554432,109051904~4194304,117440512~12582912,134217728~8388608,146800640~8388608,167772160~4194304,176160768~41943040,230686720~20971520,255852544~41943040,301989888~54525952,360710144~16777216,385875968~8388608,406847488~12582912] 2024-04-07T17:38:37.295 INFO:tasks.workunit.client.0.smithi100.stdout:from 3 to 12 diff [0~20971520,33554432~16777216,54525952~4194304,71303168~29360128,109051904~16777216,134217728~4194304,146800640~4194304,167772160~4194304,176160768~8388608,188743680~29360128,230686720~20971520,255852544~8388608,268435456~29360128,301989888~25165824,331350016~25165824,360710144~16777216,381681664~12582912,406847488~8388608] 2024-04-07T17:38:37.295 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~20971520,33554432~16777216,54525952~4194304,71303168~29360128,109051904~16777216,134217728~4194304,146800640~4194304,167772160~4194304,176160768~8388608,188743680~29360128,230686720~20971520,255852544~8388608,268435456~29360128,301989888~25165824,331350016~25165824,360710144~16777216,381681664~12582912,406847488~8388608] 2024-04-07T17:38:37.306 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~20971520,33554432~29360128,71303168~33554432,109051904~20971520,134217728~8388608,146800640~8388608,167772160~4194304,176160768~41943040,230686720~20971520,255852544~41943040,301989888~75497472,381681664~12582912,406847488~12582912] 2024-04-07T17:38:37.306 INFO:tasks.workunit.client.0.smithi100.stdout:from 3 to 13 diff [0~20971520,33554432~16777216,54525952~4194304,62914560~37748736,104857600~20971520,134217728~4194304,142606336~8388608,167772160~4194304,176160768~8388608,188743680~29360128,230686720~20971520,255852544~8388608,268435456~29360128,301989888~25165824,331350016~25165824,360710144~16777216,381681664~12582912,406847488~8388608] 2024-04-07T17:38:37.306 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~20971520,33554432~16777216,54525952~4194304,62914560~37748736,104857600~20971520,134217728~4194304,142606336~8388608,167772160~4194304,176160768~8388608,188743680~29360128,230686720~20971520,255852544~8388608,268435456~29360128,301989888~25165824,331350016~25165824,360710144~16777216,381681664~12582912,406847488~8388608] 2024-04-07T17:38:37.317 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~20971520,33554432~96468992,134217728~20971520,167772160~4194304,176160768~41943040,230686720~20971520,255852544~41943040,301989888~75497472,381681664~12582912,406847488~12582912] 2024-04-07T17:38:37.319 INFO:tasks.workunit.client.0.smithi100.stdout:from 3 to 14 diff [0~20971520,33554432~16777216,54525952~4194304,62914560~37748736,104857600~20971520,134217728~4194304,142606336~12582912,163577856~8388608,176160768~41943040,226492416~25165824,255852544~8388608,268435456~29360128,301989888~25165824,331350016~25165824,360710144~16777216,381681664~12582912,406847488~8388608] 2024-04-07T17:38:37.319 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~20971520,33554432~16777216,54525952~4194304,62914560~37748736,104857600~20971520,134217728~4194304,142606336~12582912,163577856~8388608,176160768~41943040,226492416~25165824,255852544~8388608,268435456~29360128,301989888~25165824,331350016~25165824,360710144~16777216,381681664~12582912,406847488~8388608] 2024-04-07T17:38:37.331 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~20971520,33554432~96468992,134217728~20971520,163577856~8388608,176160768~41943040,226492416~25165824,255852544~41943040,301989888~75497472,381681664~12582912,406847488~12582912] 2024-04-07T17:38:37.331 INFO:tasks.workunit.client.0.smithi100.stdout:from 3 to 15 diff [0~20971520,33554432~16777216,54525952~4194304,62914560~37748736,104857600~20971520,134217728~4194304,142606336~12582912,163577856~8388608,176160768~41943040,226492416~25165824,255852544~8388608,268435456~46137344,318767104~37748736,360710144~33554432,406847488~8388608] 2024-04-07T17:38:37.332 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~20971520,33554432~16777216,54525952~4194304,62914560~37748736,104857600~20971520,134217728~4194304,142606336~12582912,163577856~8388608,176160768~41943040,226492416~25165824,255852544~8388608,268435456~46137344,318767104~37748736,360710144~33554432,406847488~8388608] 2024-04-07T17:38:37.346 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~20971520,33554432~96468992,134217728~20971520,163577856~8388608,176160768~41943040,226492416~25165824,255852544~138412032,406847488~12582912] 2024-04-07T17:38:37.346 INFO:tasks.workunit.client.0.smithi100.stdout:from 3 to 16 diff [0~20971520,33554432~16777216,54525952~4194304,62914560~62914560,130023424~8388608,142606336~12582912,159383552~12582912,176160768~41943040,226492416~25165824,255852544~8388608,268435456~58720256,331350016~25165824,360710144~33554432,406847488~8388608] 2024-04-07T17:38:37.346 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~20971520,33554432~16777216,54525952~4194304,62914560~62914560,130023424~8388608,142606336~12582912,159383552~12582912,176160768~41943040,226492416~25165824,255852544~8388608,268435456~58720256,331350016~25165824,360710144~33554432,406847488~8388608] 2024-04-07T17:38:37.360 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~20971520,33554432~121634816,159383552~12582912,176160768~41943040,226492416~25165824,255852544~142606336,406847488~12582912] 2024-04-07T17:38:37.360 INFO:tasks.workunit.client.0.smithi100.stdout:from 3 to 17 diff [0~29360128,33554432~16777216,54525952~4194304,62914560~92274688,159383552~12582912,176160768~41943040,226492416~25165824,255852544~8388608,268435456~58720256,331350016~62914560,402653184~16777216] 2024-04-07T17:38:37.360 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~16777216,54525952~4194304,62914560~92274688,159383552~12582912,176160768~41943040,226492416~25165824,255852544~8388608,268435456~58720256,331350016~62914560,402653184~16777216] 2024-04-07T17:38:37.375 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~155189248,159383552~58720256,226492416~25165824,255852544~142606336,402653184~16777216] 2024-04-07T17:38:37.375 INFO:tasks.workunit.client.0.smithi100.stdout:from 3 to 18 diff [0~29360128,33554432~16777216,54525952~117440512,176160768~88080384,268435456~58720256,331350016~62914560,402653184~16777216] 2024-04-07T17:38:37.375 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~16777216,54525952~117440512,176160768~88080384,268435456~58720256,331350016~62914560,402653184~16777216] 2024-04-07T17:38:37.391 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~398458880,402653184~16777216] 2024-04-07T17:38:37.391 INFO:tasks.workunit.client.0.smithi100.stdout:from 3 to 19 diff [0~29360128,33554432~16777216,54525952~209715200,268435456~58720256,331350016~62914560,402653184~16777216] 2024-04-07T17:38:37.392 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~16777216,54525952~209715200,268435456~58720256,331350016~62914560,402653184~16777216] 2024-04-07T17:38:37.408 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~398458880,402653184~16777216] 2024-04-07T17:38:37.409 INFO:tasks.workunit.client.0.smithi100.stdout:from 4 to 5 diff [0~8388608,121634816~4194304,188743680~12582912,230686720~16777216,306184192~8388608,348127232~4194304,369098752~8388608,406847488~8388608] 2024-04-07T17:38:37.409 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~8388608,121634816~4194304,188743680~12582912,230686720~16777216,306184192~8388608,348127232~4194304,369098752~8388608,406847488~8388608] 2024-04-07T17:38:37.412 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~12582912,121634816~8388608,188743680~16777216,230686720~20971520,306184192~12582912,348127232~4194304,369098752~8388608,406847488~12582912] 2024-04-07T17:38:37.412 INFO:tasks.workunit.client.0.smithi100.stdout:from 4 to 6 diff [0~8388608,41943040~4194304,71303168~8388608,92274688~8388608,117440512~8388608,188743680~16777216,230686720~16777216,272629760~8388608,306184192~20971520,348127232~4194304,369098752~8388608,406847488~8388608] 2024-04-07T17:38:37.412 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~8388608,41943040~4194304,71303168~8388608,92274688~8388608,117440512~8388608,188743680~16777216,230686720~16777216,272629760~8388608,306184192~20971520,348127232~4194304,369098752~8388608,406847488~8388608] 2024-04-07T17:38:37.416 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~12582912,41943040~4194304,71303168~12582912,92274688~8388608,117440512~12582912,188743680~16777216,230686720~20971520,272629760~8388608,306184192~25165824,348127232~4194304,369098752~8388608,406847488~12582912] 2024-04-07T17:38:37.416 INFO:tasks.workunit.client.0.smithi100.stdout:from 4 to 7 diff [0~12582912,33554432~12582912,71303168~8388608,83886080~16777216,117440512~8388608,146800640~4194304,180355072~4194304,188743680~16777216,230686720~20971520,272629760~20971520,306184192~20971520,348127232~4194304,369098752~8388608,406847488~8388608] 2024-04-07T17:38:37.416 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~12582912,33554432~12582912,71303168~8388608,83886080~16777216,117440512~8388608,146800640~4194304,180355072~4194304,188743680~16777216,230686720~20971520,272629760~20971520,306184192~20971520,348127232~4194304,369098752~8388608,406847488~8388608] 2024-04-07T17:38:37.421 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~16777216,33554432~12582912,71303168~29360128,117440512~12582912,146800640~8388608,180355072~25165824,230686720~20971520,272629760~20971520,306184192~25165824,348127232~4194304,369098752~8388608,406847488~12582912] 2024-04-07T17:38:37.421 INFO:tasks.workunit.client.0.smithi100.stdout:from 4 to 8 diff [0~12582912,16777216~4194304,33554432~12582912,71303168~29360128,117440512~8388608,146800640~4194304,180355072~4194304,188743680~16777216,213909504~4194304,230686720~20971520,268435456~25165824,306184192~20971520,331350016~8388608,348127232~4194304,360710144~16777216,406847488~8388608] 2024-04-07T17:38:37.422 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~12582912,16777216~4194304,33554432~12582912,71303168~29360128,117440512~8388608,146800640~4194304,180355072~4194304,188743680~16777216,213909504~4194304,230686720~20971520,268435456~25165824,306184192~20971520,331350016~8388608,348127232~4194304,360710144~16777216,406847488~8388608] 2024-04-07T17:38:37.428 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~20971520,33554432~12582912,71303168~29360128,117440512~12582912,146800640~8388608,180355072~25165824,213909504~4194304,230686720~20971520,268435456~25165824,306184192~33554432,348127232~4194304,360710144~16777216,406847488~12582912] 2024-04-07T17:38:37.428 INFO:tasks.workunit.client.0.smithi100.stdout:from 4 to 9 diff [0~12582912,16777216~4194304,33554432~12582912,71303168~29360128,109051904~4194304,117440512~8388608,134217728~4194304,146800640~4194304,180355072~4194304,188743680~29360128,230686720~20971520,268435456~25165824,306184192~20971520,331350016~12582912,348127232~4194304,360710144~16777216,390070272~4194304,406847488~8388608] 2024-04-07T17:38:37.428 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~12582912,16777216~4194304,33554432~12582912,71303168~29360128,109051904~4194304,117440512~8388608,134217728~4194304,146800640~4194304,180355072~4194304,188743680~29360128,230686720~20971520,268435456~25165824,306184192~20971520,331350016~12582912,348127232~4194304,360710144~16777216,390070272~4194304,406847488~8388608] 2024-04-07T17:38:37.435 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~20971520,33554432~12582912,71303168~33554432,109051904~4194304,117440512~12582912,134217728~8388608,146800640~8388608,180355072~37748736,230686720~20971520,268435456~25165824,306184192~37748736,348127232~4194304,360710144~16777216,390070272~4194304,406847488~12582912] 2024-04-07T17:38:37.435 INFO:tasks.workunit.client.0.smithi100.stdout:from 4 to 10 diff [0~20971520,33554432~16777216,71303168~29360128,109051904~4194304,117440512~8388608,134217728~4194304,146800640~4194304,167772160~4194304,180355072~4194304,188743680~29360128,230686720~20971520,268435456~25165824,301989888~25165824,331350016~12582912,348127232~4194304,360710144~16777216,385875968~8388608,406847488~8388608] 2024-04-07T17:38:37.435 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~20971520,33554432~16777216,71303168~29360128,109051904~4194304,117440512~8388608,134217728~4194304,146800640~4194304,167772160~4194304,180355072~4194304,188743680~29360128,230686720~20971520,268435456~25165824,301989888~25165824,331350016~12582912,348127232~4194304,360710144~16777216,385875968~8388608,406847488~8388608] 2024-04-07T17:38:37.443 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~20971520,33554432~20971520,71303168~33554432,109051904~4194304,117440512~12582912,134217728~8388608,146800640~8388608,167772160~4194304,180355072~37748736,230686720~20971520,268435456~25165824,301989888~41943040,348127232~4194304,360710144~16777216,385875968~8388608,406847488~12582912] 2024-04-07T17:38:37.443 INFO:tasks.workunit.client.0.smithi100.stdout:from 4 to 11 diff [0~20971520,33554432~16777216,58720256~4194304,71303168~8388608,83886080~16777216,109051904~4194304,117440512~8388608,134217728~4194304,146800640~4194304,167772160~4194304,180355072~4194304,188743680~29360128,230686720~20971520,255852544~8388608,268435456~29360128,301989888~25165824,331350016~12582912,348127232~4194304,360710144~16777216,385875968~8388608,406847488~8388608] 2024-04-07T17:38:37.444 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~20971520,33554432~16777216,58720256~4194304,71303168~8388608,83886080~16777216,109051904~4194304,117440512~8388608,134217728~4194304,146800640~4194304,167772160~4194304,180355072~4194304,188743680~29360128,230686720~20971520,255852544~8388608,268435456~29360128,301989888~25165824,331350016~12582912,348127232~4194304,360710144~16777216,385875968~8388608,406847488~8388608] 2024-04-07T17:38:37.452 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~20971520,33554432~20971520,58720256~4194304,71303168~33554432,109051904~4194304,117440512~12582912,134217728~8388608,146800640~8388608,167772160~4194304,180355072~37748736,230686720~20971520,255852544~41943040,301989888~41943040,348127232~4194304,360710144~16777216,385875968~8388608,406847488~12582912] 2024-04-07T17:38:37.453 INFO:tasks.workunit.client.0.smithi100.stdout:from 4 to 12 diff [0~20971520,33554432~16777216,54525952~4194304,71303168~29360128,109051904~16777216,134217728~4194304,146800640~4194304,167772160~4194304,180355072~4194304,188743680~29360128,230686720~20971520,255852544~8388608,268435456~29360128,301989888~25165824,331350016~46137344,381681664~12582912,406847488~8388608] 2024-04-07T17:38:37.453 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~20971520,33554432~16777216,54525952~4194304,71303168~29360128,109051904~16777216,134217728~4194304,146800640~4194304,167772160~4194304,180355072~4194304,188743680~29360128,230686720~20971520,255852544~8388608,268435456~29360128,301989888~25165824,331350016~46137344,381681664~12582912,406847488~8388608] 2024-04-07T17:38:37.463 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~20971520,33554432~29360128,71303168~33554432,109051904~20971520,134217728~8388608,146800640~8388608,167772160~4194304,180355072~37748736,230686720~20971520,255852544~41943040,301989888~75497472,381681664~12582912,406847488~12582912] 2024-04-07T17:38:37.463 INFO:tasks.workunit.client.0.smithi100.stdout:from 4 to 13 diff [0~20971520,33554432~16777216,54525952~4194304,62914560~37748736,104857600~20971520,134217728~4194304,142606336~8388608,167772160~4194304,176160768~8388608,188743680~29360128,230686720~20971520,255852544~8388608,268435456~29360128,301989888~25165824,331350016~25165824,360710144~16777216,381681664~12582912,406847488~8388608] 2024-04-07T17:38:37.463 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~20971520,33554432~16777216,54525952~4194304,62914560~37748736,104857600~20971520,134217728~4194304,142606336~8388608,167772160~4194304,176160768~8388608,188743680~29360128,230686720~20971520,255852544~8388608,268435456~29360128,301989888~25165824,331350016~25165824,360710144~16777216,381681664~12582912,406847488~8388608] 2024-04-07T17:38:37.474 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~20971520,33554432~96468992,134217728~20971520,167772160~4194304,176160768~41943040,230686720~20971520,255852544~41943040,301989888~75497472,381681664~12582912,406847488~12582912] 2024-04-07T17:38:37.474 INFO:tasks.workunit.client.0.smithi100.stdout:from 4 to 14 diff [0~20971520,33554432~16777216,54525952~4194304,62914560~37748736,104857600~20971520,134217728~4194304,142606336~12582912,163577856~8388608,176160768~8388608,188743680~29360128,226492416~25165824,255852544~8388608,268435456~29360128,301989888~25165824,331350016~25165824,360710144~16777216,381681664~12582912,406847488~8388608] 2024-04-07T17:38:37.474 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~20971520,33554432~16777216,54525952~4194304,62914560~37748736,104857600~20971520,134217728~4194304,142606336~12582912,163577856~8388608,176160768~8388608,188743680~29360128,226492416~25165824,255852544~8388608,268435456~29360128,301989888~25165824,331350016~25165824,360710144~16777216,381681664~12582912,406847488~8388608] 2024-04-07T17:38:37.486 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~20971520,33554432~96468992,134217728~20971520,163577856~8388608,176160768~41943040,226492416~25165824,255852544~41943040,301989888~75497472,381681664~12582912,406847488~12582912] 2024-04-07T17:38:37.486 INFO:tasks.workunit.client.0.smithi100.stdout:from 4 to 15 diff [0~20971520,33554432~16777216,54525952~4194304,62914560~37748736,104857600~20971520,134217728~4194304,142606336~12582912,163577856~8388608,176160768~8388608,188743680~29360128,226492416~25165824,255852544~8388608,268435456~46137344,318767104~37748736,360710144~33554432,406847488~8388608] 2024-04-07T17:38:37.486 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~20971520,33554432~16777216,54525952~4194304,62914560~37748736,104857600~20971520,134217728~4194304,142606336~12582912,163577856~8388608,176160768~8388608,188743680~29360128,226492416~25165824,255852544~8388608,268435456~46137344,318767104~37748736,360710144~33554432,406847488~8388608] 2024-04-07T17:38:37.498 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~20971520,33554432~96468992,134217728~20971520,163577856~8388608,176160768~41943040,226492416~25165824,255852544~138412032,406847488~12582912] 2024-04-07T17:38:37.499 INFO:tasks.workunit.client.0.smithi100.stdout:from 4 to 16 diff [0~20971520,33554432~16777216,54525952~4194304,62914560~62914560,130023424~8388608,142606336~12582912,159383552~12582912,176160768~8388608,188743680~29360128,226492416~25165824,255852544~8388608,268435456~58720256,331350016~25165824,360710144~33554432,406847488~8388608] 2024-04-07T17:38:37.499 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~20971520,33554432~16777216,54525952~4194304,62914560~62914560,130023424~8388608,142606336~12582912,159383552~12582912,176160768~8388608,188743680~29360128,226492416~25165824,255852544~8388608,268435456~58720256,331350016~25165824,360710144~33554432,406847488~8388608] 2024-04-07T17:38:37.512 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~20971520,33554432~121634816,159383552~12582912,176160768~41943040,226492416~25165824,255852544~142606336,406847488~12582912] 2024-04-07T17:38:37.512 INFO:tasks.workunit.client.0.smithi100.stdout:from 4 to 17 diff [0~29360128,33554432~16777216,54525952~4194304,62914560~92274688,159383552~12582912,176160768~25165824,205520896~12582912,226492416~25165824,255852544~8388608,268435456~58720256,331350016~62914560,402653184~16777216] 2024-04-07T17:38:37.512 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~16777216,54525952~4194304,62914560~92274688,159383552~12582912,176160768~25165824,205520896~12582912,226492416~25165824,255852544~8388608,268435456~58720256,331350016~62914560,402653184~16777216] 2024-04-07T17:38:37.527 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~155189248,159383552~58720256,226492416~25165824,255852544~142606336,402653184~16777216] 2024-04-07T17:38:37.528 INFO:tasks.workunit.client.0.smithi100.stdout:from 4 to 18 diff [0~29360128,33554432~16777216,54525952~117440512,176160768~33554432,213909504~50331648,268435456~58720256,331350016~62914560,402653184~16777216] 2024-04-07T17:38:37.528 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~16777216,54525952~117440512,176160768~33554432,213909504~50331648,268435456~58720256,331350016~62914560,402653184~16777216] 2024-04-07T17:38:37.545 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~398458880,402653184~16777216] 2024-04-07T17:38:37.545 INFO:tasks.workunit.client.0.smithi100.stdout:from 4 to 19 diff [0~29360128,33554432~16777216,54525952~155189248,213909504~50331648,268435456~58720256,331350016~62914560,402653184~16777216] 2024-04-07T17:38:37.546 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~16777216,54525952~155189248,213909504~50331648,268435456~58720256,331350016~62914560,402653184~16777216] 2024-04-07T17:38:37.563 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~398458880,402653184~16777216] 2024-04-07T17:38:37.563 INFO:tasks.workunit.client.0.smithi100.stdout:from 5 to 6 diff [41943040~4194304,71303168~8388608,92274688~8388608,117440512~4194304,192937984~8388608,272629760~8388608,310378496~16777216,406847488~8388608] 2024-04-07T17:38:37.563 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [41943040~4194304,71303168~8388608,92274688~8388608,117440512~4194304,192937984~8388608,272629760~8388608,310378496~16777216,406847488~8388608] 2024-04-07T17:38:37.566 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [41943040~4194304,71303168~12582912,92274688~8388608,117440512~8388608,192937984~12582912,272629760~8388608,310378496~20971520,406847488~12582912] 2024-04-07T17:38:37.567 INFO:tasks.workunit.client.0.smithi100.stdout:from 5 to 7 diff [0~12582912,33554432~12582912,71303168~8388608,83886080~16777216,117440512~4194304,146800640~4194304,180355072~4194304,192937984~8388608,243269632~8388608,272629760~20971520,310378496~16777216,406847488~8388608] 2024-04-07T17:38:37.567 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~12582912,33554432~12582912,71303168~8388608,83886080~16777216,117440512~4194304,146800640~4194304,180355072~4194304,192937984~8388608,243269632~8388608,272629760~20971520,310378496~16777216,406847488~8388608] 2024-04-07T17:38:37.572 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~16777216,33554432~12582912,71303168~29360128,117440512~8388608,146800640~8388608,180355072~8388608,192937984~12582912,243269632~8388608,272629760~20971520,310378496~20971520,406847488~12582912] 2024-04-07T17:38:37.572 INFO:tasks.workunit.client.0.smithi100.stdout:from 5 to 8 diff [0~12582912,16777216~4194304,33554432~12582912,71303168~29360128,117440512~4194304,146800640~4194304,180355072~4194304,192937984~8388608,213909504~4194304,239075328~8388608,268435456~25165824,310378496~16777216,331350016~8388608,360710144~8388608,406847488~8388608] 2024-04-07T17:38:37.572 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~12582912,16777216~4194304,33554432~12582912,71303168~29360128,117440512~4194304,146800640~4194304,180355072~4194304,192937984~8388608,213909504~4194304,239075328~8388608,268435456~25165824,310378496~16777216,331350016~8388608,360710144~8388608,406847488~8388608] 2024-04-07T17:38:37.577 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~20971520,33554432~12582912,71303168~29360128,117440512~8388608,146800640~8388608,180355072~8388608,192937984~12582912,213909504~4194304,239075328~12582912,268435456~25165824,310378496~29360128,360710144~8388608,406847488~12582912] 2024-04-07T17:38:37.577 INFO:tasks.workunit.client.0.smithi100.stdout:from 5 to 9 diff [0~12582912,16777216~4194304,33554432~12582912,71303168~29360128,109051904~4194304,117440512~4194304,134217728~4194304,146800640~4194304,180355072~4194304,192937984~8388608,205520896~12582912,239075328~8388608,268435456~25165824,310378496~16777216,331350016~12582912,360710144~8388608,390070272~4194304,406847488~8388608] 2024-04-07T17:38:37.577 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~12582912,16777216~4194304,33554432~12582912,71303168~29360128,109051904~4194304,117440512~4194304,134217728~4194304,146800640~4194304,180355072~4194304,192937984~8388608,205520896~12582912,239075328~8388608,268435456~25165824,310378496~16777216,331350016~12582912,360710144~8388608,390070272~4194304,406847488~8388608] 2024-04-07T17:38:37.584 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~20971520,33554432~12582912,71303168~33554432,109051904~4194304,117440512~8388608,134217728~8388608,146800640~8388608,180355072~8388608,192937984~25165824,239075328~12582912,268435456~25165824,310378496~33554432,360710144~8388608,390070272~4194304,406847488~12582912] 2024-04-07T17:38:37.584 INFO:tasks.workunit.client.0.smithi100.stdout:from 5 to 10 diff [0~20971520,33554432~16777216,71303168~29360128,109051904~4194304,117440512~4194304,134217728~4194304,146800640~4194304,167772160~4194304,180355072~4194304,192937984~8388608,205520896~12582912,239075328~8388608,268435456~25165824,301989888~25165824,331350016~12582912,360710144~8388608,385875968~8388608,406847488~8388608] 2024-04-07T17:38:37.584 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~20971520,33554432~16777216,71303168~29360128,109051904~4194304,117440512~4194304,134217728~4194304,146800640~4194304,167772160~4194304,180355072~4194304,192937984~8388608,205520896~12582912,239075328~8388608,268435456~25165824,301989888~25165824,331350016~12582912,360710144~8388608,385875968~8388608,406847488~8388608] 2024-04-07T17:38:37.593 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~20971520,33554432~20971520,71303168~33554432,109051904~4194304,117440512~8388608,134217728~8388608,146800640~8388608,167772160~4194304,180355072~8388608,192937984~25165824,239075328~12582912,268435456~25165824,301989888~41943040,360710144~8388608,385875968~8388608,406847488~12582912] 2024-04-07T17:38:37.593 INFO:tasks.workunit.client.0.smithi100.stdout:from 5 to 11 diff [0~20971520,33554432~16777216,58720256~4194304,71303168~8388608,83886080~16777216,109051904~4194304,117440512~4194304,134217728~4194304,146800640~4194304,167772160~4194304,180355072~4194304,192937984~8388608,205520896~12582912,239075328~8388608,255852544~8388608,268435456~29360128,301989888~25165824,331350016~12582912,360710144~16777216,385875968~8388608,406847488~8388608] 2024-04-07T17:38:37.593 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~20971520,33554432~16777216,58720256~4194304,71303168~8388608,83886080~16777216,109051904~4194304,117440512~4194304,134217728~4194304,146800640~4194304,167772160~4194304,180355072~4194304,192937984~8388608,205520896~12582912,239075328~8388608,255852544~8388608,268435456~29360128,301989888~25165824,331350016~12582912,360710144~16777216,385875968~8388608,406847488~8388608] 2024-04-07T17:38:37.601 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~20971520,33554432~20971520,58720256~4194304,71303168~33554432,109051904~4194304,117440512~8388608,134217728~8388608,146800640~8388608,167772160~4194304,180355072~8388608,192937984~25165824,239075328~12582912,255852544~41943040,301989888~41943040,360710144~16777216,385875968~8388608,406847488~12582912] 2024-04-07T17:38:37.601 INFO:tasks.workunit.client.0.smithi100.stdout:from 5 to 12 diff [0~20971520,33554432~16777216,54525952~4194304,71303168~29360128,109051904~12582912,134217728~4194304,146800640~4194304,167772160~4194304,180355072~4194304,192937984~8388608,205520896~12582912,239075328~8388608,255852544~8388608,268435456~29360128,301989888~25165824,331350016~25165824,360710144~16777216,381681664~12582912,406847488~8388608] 2024-04-07T17:38:37.601 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~20971520,33554432~16777216,54525952~4194304,71303168~29360128,109051904~12582912,134217728~4194304,146800640~4194304,167772160~4194304,180355072~4194304,192937984~8388608,205520896~12582912,239075328~8388608,255852544~8388608,268435456~29360128,301989888~25165824,331350016~25165824,360710144~16777216,381681664~12582912,406847488~8388608] 2024-04-07T17:38:37.611 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~20971520,33554432~29360128,71303168~33554432,109051904~16777216,134217728~8388608,146800640~8388608,167772160~4194304,180355072~8388608,192937984~25165824,239075328~12582912,255852544~41943040,301989888~75497472,381681664~12582912,406847488~12582912] 2024-04-07T17:38:37.611 INFO:tasks.workunit.client.0.smithi100.stdout:from 5 to 13 diff [0~20971520,33554432~16777216,54525952~4194304,62914560~37748736,104857600~16777216,134217728~4194304,142606336~8388608,167772160~4194304,176160768~8388608,188743680~12582912,205520896~12582912,239075328~8388608,255852544~8388608,268435456~29360128,301989888~25165824,331350016~25165824,360710144~16777216,381681664~12582912,406847488~8388608] 2024-04-07T17:38:37.611 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~20971520,33554432~16777216,54525952~4194304,62914560~37748736,104857600~16777216,134217728~4194304,142606336~8388608,167772160~4194304,176160768~8388608,188743680~12582912,205520896~12582912,239075328~8388608,255852544~8388608,268435456~29360128,301989888~25165824,331350016~25165824,360710144~16777216,381681664~12582912,406847488~8388608] 2024-04-07T17:38:37.621 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~20971520,33554432~92274688,134217728~20971520,167772160~4194304,176160768~41943040,239075328~12582912,255852544~41943040,301989888~75497472,381681664~12582912,406847488~12582912] 2024-04-07T17:38:37.622 INFO:tasks.workunit.client.0.smithi100.stdout:from 5 to 14 diff [0~20971520,33554432~16777216,54525952~4194304,62914560~37748736,104857600~16777216,134217728~4194304,142606336~12582912,163577856~8388608,176160768~8388608,188743680~12582912,205520896~12582912,226492416~8388608,239075328~8388608,255852544~8388608,268435456~29360128,301989888~25165824,331350016~25165824,360710144~16777216,381681664~12582912,406847488~8388608] 2024-04-07T17:38:37.622 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~20971520,33554432~16777216,54525952~4194304,62914560~37748736,104857600~16777216,134217728~4194304,142606336~12582912,163577856~8388608,176160768~8388608,188743680~12582912,205520896~12582912,226492416~8388608,239075328~8388608,255852544~8388608,268435456~29360128,301989888~25165824,331350016~25165824,360710144~16777216,381681664~12582912,406847488~8388608] 2024-04-07T17:38:37.633 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~20971520,33554432~92274688,134217728~20971520,163577856~8388608,176160768~41943040,226492416~8388608,239075328~12582912,255852544~41943040,301989888~75497472,381681664~12582912,406847488~12582912] 2024-04-07T17:38:37.633 INFO:tasks.workunit.client.0.smithi100.stdout:from 5 to 15 diff [0~20971520,33554432~16777216,54525952~4194304,62914560~37748736,104857600~16777216,134217728~4194304,142606336~12582912,163577856~8388608,176160768~8388608,188743680~12582912,205520896~12582912,226492416~8388608,239075328~8388608,255852544~8388608,268435456~88080384,360710144~33554432,406847488~8388608] 2024-04-07T17:38:37.633 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~20971520,33554432~16777216,54525952~4194304,62914560~37748736,104857600~16777216,134217728~4194304,142606336~12582912,163577856~8388608,176160768~8388608,188743680~12582912,205520896~12582912,226492416~8388608,239075328~8388608,255852544~8388608,268435456~88080384,360710144~33554432,406847488~8388608] 2024-04-07T17:38:37.645 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~20971520,33554432~92274688,134217728~20971520,163577856~8388608,176160768~41943040,226492416~8388608,239075328~12582912,255852544~138412032,406847488~12582912] 2024-04-07T17:38:37.645 INFO:tasks.workunit.client.0.smithi100.stdout:from 5 to 16 diff [0~20971520,33554432~16777216,54525952~4194304,62914560~58720256,130023424~8388608,142606336~12582912,159383552~12582912,176160768~8388608,188743680~12582912,205520896~12582912,226492416~8388608,239075328~8388608,255852544~8388608,268435456~58720256,331350016~25165824,360710144~33554432,406847488~8388608] 2024-04-07T17:38:37.645 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~20971520,33554432~16777216,54525952~4194304,62914560~58720256,130023424~8388608,142606336~12582912,159383552~12582912,176160768~8388608,188743680~12582912,205520896~12582912,226492416~8388608,239075328~8388608,255852544~8388608,268435456~58720256,331350016~25165824,360710144~33554432,406847488~8388608] 2024-04-07T17:38:37.659 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~20971520,33554432~92274688,130023424~25165824,159383552~12582912,176160768~41943040,226492416~25165824,255852544~142606336,406847488~12582912] 2024-04-07T17:38:37.659 INFO:tasks.workunit.client.0.smithi100.stdout:from 5 to 17 diff [0~29360128,33554432~16777216,54525952~4194304,62914560~92274688,159383552~12582912,176160768~25165824,205520896~12582912,226492416~8388608,239075328~8388608,255852544~8388608,268435456~58720256,331350016~62914560,402653184~16777216] 2024-04-07T17:38:37.659 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~16777216,54525952~4194304,62914560~92274688,159383552~12582912,176160768~25165824,205520896~12582912,226492416~8388608,239075328~8388608,255852544~8388608,268435456~58720256,331350016~62914560,402653184~16777216] 2024-04-07T17:38:37.673 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~155189248,159383552~58720256,226492416~25165824,255852544~142606336,402653184~16777216] 2024-04-07T17:38:37.673 INFO:tasks.workunit.client.0.smithi100.stdout:from 5 to 18 diff [0~29360128,33554432~16777216,54525952~117440512,176160768~58720256,239075328~8388608,251658240~12582912,268435456~58720256,331350016~62914560,402653184~16777216] 2024-04-07T17:38:37.673 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~16777216,54525952~117440512,176160768~58720256,239075328~8388608,251658240~12582912,268435456~58720256,331350016~62914560,402653184~16777216] 2024-04-07T17:38:37.688 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~398458880,402653184~16777216] 2024-04-07T17:38:37.688 INFO:tasks.workunit.client.0.smithi100.stdout:from 5 to 19 diff [0~29360128,33554432~16777216,54525952~180355072,239075328~8388608,251658240~12582912,268435456~58720256,331350016~62914560,402653184~16777216] 2024-04-07T17:38:37.688 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~16777216,54525952~180355072,239075328~8388608,251658240~12582912,268435456~58720256,331350016~62914560,402653184~16777216] 2024-04-07T17:38:37.703 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~398458880,402653184~16777216] 2024-04-07T17:38:37.703 INFO:tasks.workunit.client.0.smithi100.stdout:from 6 to 7 diff [0~12582912,33554432~8388608,83886080~8388608,146800640~4194304,180355072~4194304,243269632~8388608,281018368~12582912] 2024-04-07T17:38:37.703 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~12582912,33554432~8388608,83886080~8388608,146800640~4194304,180355072~4194304,243269632~8388608,281018368~12582912] 2024-04-07T17:38:37.706 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~16777216,33554432~8388608,83886080~12582912,146800640~8388608,180355072~8388608,243269632~8388608,281018368~12582912] 2024-04-07T17:38:37.707 INFO:tasks.workunit.client.0.smithi100.stdout:from 6 to 8 diff [0~12582912,16777216~4194304,33554432~8388608,75497472~16777216,146800640~4194304,180355072~4194304,213909504~4194304,239075328~8388608,268435456~4194304,281018368~12582912,331350016~8388608,360710144~8388608] 2024-04-07T17:38:37.707 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~12582912,16777216~4194304,33554432~8388608,75497472~16777216,146800640~4194304,180355072~4194304,213909504~4194304,239075328~8388608,268435456~4194304,281018368~12582912,331350016~8388608,360710144~8388608] 2024-04-07T17:38:37.711 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~20971520,33554432~12582912,75497472~20971520,146800640~8388608,180355072~8388608,213909504~4194304,239075328~12582912,268435456~8388608,281018368~12582912,331350016~8388608,360710144~8388608] 2024-04-07T17:38:37.711 INFO:tasks.workunit.client.0.smithi100.stdout:from 6 to 9 diff [0~12582912,16777216~4194304,33554432~8388608,75497472~25165824,109051904~4194304,134217728~4194304,146800640~4194304,180355072~4194304,205520896~12582912,239075328~8388608,268435456~4194304,281018368~12582912,314572800~4194304,331350016~12582912,360710144~8388608,390070272~4194304] 2024-04-07T17:38:37.711 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~12582912,16777216~4194304,33554432~8388608,75497472~25165824,109051904~4194304,134217728~4194304,146800640~4194304,180355072~4194304,205520896~12582912,239075328~8388608,268435456~4194304,281018368~12582912,314572800~4194304,331350016~12582912,360710144~8388608,390070272~4194304] 2024-04-07T17:38:37.716 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~20971520,33554432~12582912,75497472~29360128,109051904~4194304,134217728~8388608,146800640~8388608,180355072~8388608,205520896~12582912,239075328~12582912,268435456~8388608,281018368~12582912,314572800~4194304,331350016~12582912,360710144~8388608,390070272~4194304] 2024-04-07T17:38:37.716 INFO:tasks.workunit.client.0.smithi100.stdout:from 6 to 10 diff [0~20971520,33554432~8388608,46137344~4194304,75497472~25165824,109051904~4194304,134217728~4194304,146800640~4194304,167772160~4194304,180355072~4194304,205520896~12582912,239075328~8388608,268435456~25165824,301989888~8388608,314572800~4194304,331350016~12582912,360710144~8388608,385875968~8388608] 2024-04-07T17:38:37.716 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~20971520,33554432~8388608,46137344~4194304,75497472~25165824,109051904~4194304,134217728~4194304,146800640~4194304,167772160~4194304,180355072~4194304,205520896~12582912,239075328~8388608,268435456~25165824,301989888~8388608,314572800~4194304,331350016~12582912,360710144~8388608,385875968~8388608] 2024-04-07T17:38:37.723 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~20971520,33554432~20971520,75497472~29360128,109051904~4194304,134217728~8388608,146800640~8388608,167772160~4194304,180355072~8388608,205520896~12582912,239075328~12582912,268435456~25165824,301989888~16777216,331350016~12582912,360710144~8388608,385875968~8388608] 2024-04-07T17:38:37.723 INFO:tasks.workunit.client.0.smithi100.stdout:from 6 to 11 diff [0~20971520,33554432~8388608,46137344~4194304,58720256~4194304,71303168~8388608,83886080~16777216,109051904~4194304,134217728~4194304,146800640~4194304,167772160~4194304,180355072~4194304,205520896~12582912,239075328~8388608,255852544~8388608,268435456~29360128,301989888~8388608,314572800~4194304,331350016~12582912,360710144~16777216,385875968~8388608] 2024-04-07T17:38:37.723 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~20971520,33554432~8388608,46137344~4194304,58720256~4194304,71303168~8388608,83886080~16777216,109051904~4194304,134217728~4194304,146800640~4194304,167772160~4194304,180355072~4194304,205520896~12582912,239075328~8388608,255852544~8388608,268435456~29360128,301989888~8388608,314572800~4194304,331350016~12582912,360710144~16777216,385875968~8388608] 2024-04-07T17:38:37.732 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~20971520,33554432~20971520,58720256~4194304,71303168~33554432,109051904~4194304,134217728~8388608,146800640~8388608,167772160~4194304,180355072~8388608,205520896~12582912,239075328~12582912,255852544~41943040,301989888~16777216,331350016~12582912,360710144~16777216,385875968~8388608] 2024-04-07T17:38:37.732 INFO:tasks.workunit.client.0.smithi100.stdout:from 6 to 12 diff [0~20971520,33554432~8388608,46137344~4194304,54525952~4194304,71303168~29360128,109051904~12582912,134217728~4194304,146800640~4194304,167772160~4194304,180355072~4194304,192937984~8388608,205520896~12582912,239075328~8388608,255852544~8388608,268435456~29360128,301989888~8388608,314572800~4194304,331350016~25165824,360710144~16777216,381681664~12582912] 2024-04-07T17:38:37.732 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~20971520,33554432~8388608,46137344~4194304,54525952~4194304,71303168~29360128,109051904~12582912,134217728~4194304,146800640~4194304,167772160~4194304,180355072~4194304,192937984~8388608,205520896~12582912,239075328~8388608,255852544~8388608,268435456~29360128,301989888~8388608,314572800~4194304,331350016~25165824,360710144~16777216,381681664~12582912] 2024-04-07T17:38:37.740 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~20971520,33554432~29360128,71303168~33554432,109051904~12582912,134217728~8388608,146800640~8388608,167772160~4194304,180355072~8388608,192937984~8388608,205520896~12582912,239075328~12582912,255852544~41943040,301989888~16777216,331350016~46137344,381681664~12582912] 2024-04-07T17:38:37.740 INFO:tasks.workunit.client.0.smithi100.stdout:from 6 to 13 diff [0~20971520,33554432~8388608,46137344~4194304,54525952~4194304,62914560~37748736,104857600~16777216,134217728~4194304,142606336~8388608,167772160~4194304,176160768~8388608,188743680~12582912,205520896~12582912,239075328~8388608,255852544~8388608,268435456~29360128,301989888~8388608,314572800~4194304,331350016~25165824,360710144~16777216,381681664~12582912] 2024-04-07T17:38:37.740 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~20971520,33554432~8388608,46137344~4194304,54525952~4194304,62914560~37748736,104857600~16777216,134217728~4194304,142606336~8388608,167772160~4194304,176160768~8388608,188743680~12582912,205520896~12582912,239075328~8388608,255852544~8388608,268435456~29360128,301989888~8388608,314572800~4194304,331350016~25165824,360710144~16777216,381681664~12582912] 2024-04-07T17:38:37.750 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~20971520,33554432~88080384,134217728~20971520,167772160~4194304,176160768~25165824,205520896~12582912,239075328~12582912,255852544~41943040,301989888~16777216,331350016~46137344,381681664~12582912] 2024-04-07T17:38:37.750 INFO:tasks.workunit.client.0.smithi100.stdout:from 6 to 14 diff [0~20971520,33554432~8388608,46137344~4194304,54525952~4194304,62914560~37748736,104857600~16777216,134217728~4194304,142606336~12582912,163577856~8388608,176160768~8388608,188743680~12582912,205520896~12582912,226492416~8388608,239075328~8388608,255852544~8388608,268435456~29360128,301989888~8388608,314572800~4194304,331350016~25165824,360710144~16777216,381681664~12582912] 2024-04-07T17:38:37.750 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~20971520,33554432~8388608,46137344~4194304,54525952~4194304,62914560~37748736,104857600~16777216,134217728~4194304,142606336~12582912,163577856~8388608,176160768~8388608,188743680~12582912,205520896~12582912,226492416~8388608,239075328~8388608,255852544~8388608,268435456~29360128,301989888~8388608,314572800~4194304,331350016~25165824,360710144~16777216,381681664~12582912] 2024-04-07T17:38:37.760 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~20971520,33554432~88080384,134217728~20971520,163577856~8388608,176160768~25165824,205520896~12582912,226492416~8388608,239075328~12582912,255852544~41943040,301989888~16777216,331350016~46137344,381681664~12582912] 2024-04-07T17:38:37.760 INFO:tasks.workunit.client.0.smithi100.stdout:from 6 to 15 diff [0~20971520,33554432~8388608,46137344~4194304,54525952~4194304,62914560~37748736,104857600~16777216,134217728~4194304,142606336~12582912,163577856~8388608,176160768~8388608,188743680~12582912,205520896~12582912,226492416~8388608,239075328~8388608,255852544~8388608,268435456~41943040,314572800~12582912,331350016~25165824,360710144~33554432] 2024-04-07T17:38:37.761 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~20971520,33554432~8388608,46137344~4194304,54525952~4194304,62914560~37748736,104857600~16777216,134217728~4194304,142606336~12582912,163577856~8388608,176160768~8388608,188743680~12582912,205520896~12582912,226492416~8388608,239075328~8388608,255852544~8388608,268435456~41943040,314572800~12582912,331350016~25165824,360710144~33554432] 2024-04-07T17:38:37.772 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~20971520,33554432~88080384,134217728~20971520,163577856~8388608,176160768~25165824,205520896~12582912,226492416~8388608,239075328~12582912,255852544~138412032] 2024-04-07T17:38:37.772 INFO:tasks.workunit.client.0.smithi100.stdout:from 6 to 16 diff [0~20971520,33554432~8388608,46137344~4194304,54525952~4194304,62914560~58720256,130023424~8388608,142606336~12582912,159383552~12582912,176160768~8388608,188743680~12582912,205520896~12582912,226492416~8388608,239075328~8388608,255852544~8388608,268435456~41943040,314572800~12582912,331350016~25165824,360710144~33554432] 2024-04-07T17:38:37.772 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~20971520,33554432~8388608,46137344~4194304,54525952~4194304,62914560~58720256,130023424~8388608,142606336~12582912,159383552~12582912,176160768~8388608,188743680~12582912,205520896~12582912,226492416~8388608,239075328~8388608,255852544~8388608,268435456~41943040,314572800~12582912,331350016~25165824,360710144~33554432] 2024-04-07T17:38:37.784 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~20971520,33554432~88080384,130023424~25165824,159383552~12582912,176160768~25165824,205520896~12582912,226492416~25165824,255852544~142606336] 2024-04-07T17:38:37.784 INFO:tasks.workunit.client.0.smithi100.stdout:from 6 to 17 diff [0~29360128,33554432~8388608,46137344~4194304,54525952~4194304,62914560~92274688,159383552~12582912,176160768~25165824,205520896~12582912,226492416~8388608,239075328~8388608,255852544~8388608,268435456~41943040,314572800~12582912,331350016~62914560,402653184~8388608] 2024-04-07T17:38:37.784 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~8388608,46137344~4194304,54525952~4194304,62914560~92274688,159383552~12582912,176160768~25165824,205520896~12582912,226492416~8388608,239075328~8388608,255852544~8388608,268435456~41943040,314572800~12582912,331350016~62914560,402653184~8388608] 2024-04-07T17:38:37.797 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~155189248,159383552~41943040,205520896~12582912,226492416~25165824,255852544~142606336,402653184~8388608] 2024-04-07T17:38:37.797 INFO:tasks.workunit.client.0.smithi100.stdout:from 6 to 18 diff [0~29360128,33554432~8388608,46137344~4194304,54525952~117440512,176160768~58720256,239075328~8388608,251658240~12582912,268435456~41943040,314572800~12582912,331350016~62914560,402653184~8388608] 2024-04-07T17:38:37.798 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~8388608,46137344~4194304,54525952~117440512,176160768~58720256,239075328~8388608,251658240~12582912,268435456~41943040,314572800~12582912,331350016~62914560,402653184~8388608] 2024-04-07T17:38:37.812 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~398458880,402653184~8388608] 2024-04-07T17:38:37.812 INFO:tasks.workunit.client.0.smithi100.stdout:from 6 to 19 diff [0~29360128,33554432~8388608,46137344~4194304,54525952~180355072,239075328~8388608,251658240~12582912,268435456~41943040,314572800~12582912,331350016~62914560,402653184~8388608] 2024-04-07T17:38:37.812 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~8388608,46137344~4194304,54525952~180355072,239075328~8388608,251658240~12582912,268435456~41943040,314572800~12582912,331350016~62914560,402653184~8388608] 2024-04-07T17:38:37.827 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~398458880,402653184~8388608] 2024-04-07T17:38:37.827 INFO:tasks.workunit.client.0.smithi100.stdout:from 7 to 8 diff [16777216~4194304,37748736~4194304,75497472~16777216,213909504~4194304,239075328~8388608,268435456~4194304,331350016~8388608,360710144~8388608] 2024-04-07T17:38:37.827 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [16777216~4194304,37748736~4194304,75497472~16777216,213909504~4194304,239075328~8388608,268435456~4194304,331350016~8388608,360710144~8388608] 2024-04-07T17:38:37.831 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [16777216~4194304,37748736~8388608,75497472~20971520,213909504~4194304,239075328~12582912,268435456~8388608,331350016~8388608,360710144~8388608] 2024-04-07T17:38:37.831 INFO:tasks.workunit.client.0.smithi100.stdout:from 7 to 9 diff [4194304~4194304,16777216~4194304,37748736~4194304,75497472~25165824,109051904~4194304,134217728~4194304,205520896~12582912,239075328~8388608,268435456~4194304,314572800~4194304,331350016~12582912,360710144~8388608,390070272~4194304] 2024-04-07T17:38:37.831 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [4194304~4194304,16777216~4194304,37748736~4194304,75497472~25165824,109051904~4194304,134217728~4194304,205520896~12582912,239075328~8388608,268435456~4194304,314572800~4194304,331350016~12582912,360710144~8388608,390070272~4194304] 2024-04-07T17:38:37.836 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [4194304~4194304,16777216~4194304,37748736~8388608,75497472~29360128,109051904~4194304,134217728~8388608,205520896~12582912,239075328~12582912,268435456~8388608,314572800~4194304,331350016~12582912,360710144~8388608,390070272~4194304] 2024-04-07T17:38:37.836 INFO:tasks.workunit.client.0.smithi100.stdout:from 7 to 10 diff [4194304~16777216,37748736~4194304,46137344~4194304,75497472~25165824,109051904~4194304,134217728~4194304,167772160~4194304,205520896~12582912,239075328~8388608,268435456~16777216,301989888~8388608,314572800~4194304,331350016~12582912,360710144~8388608,385875968~8388608] 2024-04-07T17:38:37.836 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [4194304~16777216,37748736~4194304,46137344~4194304,75497472~25165824,109051904~4194304,134217728~4194304,167772160~4194304,205520896~12582912,239075328~8388608,268435456~16777216,301989888~8388608,314572800~4194304,331350016~12582912,360710144~8388608,385875968~8388608] 2024-04-07T17:38:37.841 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [4194304~16777216,37748736~16777216,75497472~29360128,109051904~4194304,134217728~8388608,167772160~4194304,205520896~12582912,239075328~12582912,268435456~20971520,301989888~16777216,331350016~12582912,360710144~8388608,385875968~8388608] 2024-04-07T17:38:37.841 INFO:tasks.workunit.client.0.smithi100.stdout:from 7 to 11 diff [4194304~16777216,37748736~4194304,46137344~4194304,58720256~4194304,71303168~8388608,83886080~16777216,109051904~4194304,134217728~4194304,167772160~4194304,180355072~4194304,205520896~12582912,239075328~8388608,255852544~8388608,268435456~16777216,293601280~4194304,301989888~8388608,314572800~4194304,331350016~12582912,360710144~16777216,385875968~8388608] 2024-04-07T17:38:37.841 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [4194304~16777216,37748736~4194304,46137344~4194304,58720256~4194304,71303168~8388608,83886080~16777216,109051904~4194304,134217728~4194304,167772160~4194304,180355072~4194304,205520896~12582912,239075328~8388608,255852544~8388608,268435456~16777216,293601280~4194304,301989888~8388608,314572800~4194304,331350016~12582912,360710144~16777216,385875968~8388608] 2024-04-07T17:38:37.848 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [4194304~16777216,37748736~16777216,58720256~4194304,71303168~33554432,109051904~4194304,134217728~8388608,167772160~4194304,180355072~4194304,205520896~12582912,239075328~12582912,255852544~33554432,293601280~4194304,301989888~16777216,331350016~12582912,360710144~16777216,385875968~8388608] 2024-04-07T17:38:37.848 INFO:tasks.workunit.client.0.smithi100.stdout:from 7 to 12 diff [0~20971520,33554432~8388608,46137344~4194304,54525952~4194304,71303168~29360128,109051904~12582912,134217728~4194304,167772160~4194304,180355072~4194304,192937984~8388608,205520896~12582912,239075328~8388608,255852544~8388608,268435456~16777216,293601280~4194304,301989888~8388608,314572800~4194304,331350016~25165824,360710144~16777216,381681664~12582912] 2024-04-07T17:38:37.848 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~20971520,33554432~8388608,46137344~4194304,54525952~4194304,71303168~29360128,109051904~12582912,134217728~4194304,167772160~4194304,180355072~4194304,192937984~8388608,205520896~12582912,239075328~8388608,255852544~8388608,268435456~16777216,293601280~4194304,301989888~8388608,314572800~4194304,331350016~25165824,360710144~16777216,381681664~12582912] 2024-04-07T17:38:37.855 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~20971520,33554432~29360128,71303168~33554432,109051904~12582912,134217728~8388608,167772160~4194304,180355072~4194304,192937984~8388608,205520896~12582912,239075328~12582912,255852544~33554432,293601280~4194304,301989888~16777216,331350016~46137344,381681664~12582912] 2024-04-07T17:38:37.855 INFO:tasks.workunit.client.0.smithi100.stdout:from 7 to 13 diff [0~20971520,33554432~8388608,46137344~4194304,54525952~4194304,62914560~37748736,104857600~16777216,134217728~4194304,142606336~8388608,167772160~4194304,176160768~8388608,188743680~12582912,205520896~12582912,239075328~8388608,255852544~8388608,268435456~16777216,293601280~4194304,301989888~8388608,314572800~4194304,331350016~25165824,360710144~16777216,381681664~12582912] 2024-04-07T17:38:37.855 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~20971520,33554432~8388608,46137344~4194304,54525952~4194304,62914560~37748736,104857600~16777216,134217728~4194304,142606336~8388608,167772160~4194304,176160768~8388608,188743680~12582912,205520896~12582912,239075328~8388608,255852544~8388608,268435456~16777216,293601280~4194304,301989888~8388608,314572800~4194304,331350016~25165824,360710144~16777216,381681664~12582912] 2024-04-07T17:38:37.864 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~20971520,33554432~88080384,134217728~20971520,167772160~4194304,176160768~25165824,205520896~12582912,239075328~12582912,255852544~33554432,293601280~4194304,301989888~16777216,331350016~46137344,381681664~12582912] 2024-04-07T17:38:37.864 INFO:tasks.workunit.client.0.smithi100.stdout:from 7 to 14 diff [0~20971520,33554432~8388608,46137344~4194304,54525952~4194304,62914560~37748736,104857600~16777216,134217728~4194304,142606336~12582912,163577856~8388608,176160768~8388608,188743680~12582912,205520896~12582912,226492416~8388608,239075328~8388608,255852544~8388608,268435456~16777216,293601280~4194304,301989888~8388608,314572800~4194304,331350016~25165824,360710144~16777216,381681664~12582912] 2024-04-07T17:38:37.864 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~20971520,33554432~8388608,46137344~4194304,54525952~4194304,62914560~37748736,104857600~16777216,134217728~4194304,142606336~12582912,163577856~8388608,176160768~8388608,188743680~12582912,205520896~12582912,226492416~8388608,239075328~8388608,255852544~8388608,268435456~16777216,293601280~4194304,301989888~8388608,314572800~4194304,331350016~25165824,360710144~16777216,381681664~12582912] 2024-04-07T17:38:37.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:38:37 smithi144 bash[66654]: cluster 2024-04-07T17:38:36.509470+0000 mgr.y (mgr.25175) 2497 : cluster [DBG] pgmap v4151: 137 pgs: 137 active+clean; 829 MiB data, 5.6 GiB used, 710 GiB / 715 GiB avail; 124 KiB/s rd, 33 MiB/s wr, 214 op/s 2024-04-07T17:38:37.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:38:37 smithi144 bash[66654]: cluster 2024-04-07T17:38:36.551390+0000 mon.a (mon.0) 3974 : cluster [DBG] osdmap e2080: 8 total, 8 up, 8 in 2024-04-07T17:38:37.873 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~20971520,33554432~88080384,134217728~20971520,163577856~8388608,176160768~25165824,205520896~12582912,226492416~8388608,239075328~12582912,255852544~33554432,293601280~4194304,301989888~16777216,331350016~46137344,381681664~12582912] 2024-04-07T17:38:37.873 INFO:tasks.workunit.client.0.smithi100.stdout:from 7 to 15 diff [0~20971520,33554432~8388608,46137344~4194304,54525952~4194304,62914560~37748736,104857600~16777216,134217728~4194304,142606336~12582912,163577856~8388608,176160768~8388608,188743680~12582912,205520896~12582912,226492416~8388608,239075328~8388608,255852544~8388608,268435456~16777216,293601280~16777216,314572800~12582912,331350016~25165824,360710144~33554432] 2024-04-07T17:38:37.873 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~20971520,33554432~8388608,46137344~4194304,54525952~4194304,62914560~37748736,104857600~16777216,134217728~4194304,142606336~12582912,163577856~8388608,176160768~8388608,188743680~12582912,205520896~12582912,226492416~8388608,239075328~8388608,255852544~8388608,268435456~16777216,293601280~16777216,314572800~12582912,331350016~25165824,360710144~33554432] 2024-04-07T17:38:37.884 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~20971520,33554432~88080384,134217728~20971520,163577856~8388608,176160768~25165824,205520896~12582912,226492416~8388608,239075328~12582912,255852544~33554432,293601280~100663296] 2024-04-07T17:38:37.884 INFO:tasks.workunit.client.0.smithi100.stdout:from 7 to 16 diff [0~20971520,33554432~8388608,46137344~4194304,54525952~4194304,62914560~58720256,130023424~8388608,142606336~12582912,159383552~12582912,176160768~8388608,188743680~12582912,205520896~12582912,226492416~8388608,239075328~8388608,255852544~8388608,268435456~16777216,293601280~16777216,314572800~12582912,331350016~25165824,360710144~33554432] 2024-04-07T17:38:37.885 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~20971520,33554432~8388608,46137344~4194304,54525952~4194304,62914560~58720256,130023424~8388608,142606336~12582912,159383552~12582912,176160768~8388608,188743680~12582912,205520896~12582912,226492416~8388608,239075328~8388608,255852544~8388608,268435456~16777216,293601280~16777216,314572800~12582912,331350016~25165824,360710144~33554432] 2024-04-07T17:38:37.896 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~20971520,33554432~88080384,130023424~25165824,159383552~12582912,176160768~25165824,205520896~12582912,226492416~25165824,255852544~33554432,293601280~104857600] 2024-04-07T17:38:37.896 INFO:tasks.workunit.client.0.smithi100.stdout:from 7 to 17 diff [0~29360128,33554432~8388608,46137344~4194304,54525952~4194304,62914560~92274688,159383552~12582912,176160768~25165824,205520896~12582912,226492416~8388608,239075328~8388608,255852544~8388608,268435456~16777216,293601280~16777216,314572800~12582912,331350016~62914560,402653184~8388608] 2024-04-07T17:38:37.896 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~8388608,46137344~4194304,54525952~4194304,62914560~92274688,159383552~12582912,176160768~25165824,205520896~12582912,226492416~8388608,239075328~8388608,255852544~8388608,268435456~16777216,293601280~16777216,314572800~12582912,331350016~62914560,402653184~8388608] 2024-04-07T17:38:37.907 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~155189248,159383552~41943040,205520896~12582912,226492416~25165824,255852544~33554432,293601280~104857600,402653184~8388608] 2024-04-07T17:38:37.907 INFO:tasks.workunit.client.0.smithi100.stdout:from 7 to 18 diff [0~29360128,33554432~8388608,46137344~4194304,54525952~117440512,176160768~58720256,239075328~8388608,251658240~12582912,268435456~16777216,293601280~16777216,314572800~12582912,331350016~62914560,402653184~8388608] 2024-04-07T17:38:37.908 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~8388608,46137344~4194304,54525952~117440512,176160768~58720256,239075328~8388608,251658240~12582912,268435456~16777216,293601280~16777216,314572800~12582912,331350016~62914560,402653184~8388608] 2024-04-07T17:38:37.922 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~289406976,293601280~104857600,402653184~8388608] 2024-04-07T17:38:37.922 INFO:tasks.workunit.client.0.smithi100.stdout:from 7 to 19 diff [0~29360128,33554432~8388608,46137344~4194304,54525952~180355072,239075328~8388608,251658240~12582912,268435456~16777216,293601280~16777216,314572800~12582912,331350016~62914560,402653184~8388608] 2024-04-07T17:38:37.922 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~8388608,46137344~4194304,54525952~180355072,239075328~8388608,251658240~12582912,268435456~16777216,293601280~16777216,314572800~12582912,331350016~62914560,402653184~8388608] 2024-04-07T17:38:37.936 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~289406976,293601280~104857600,402653184~8388608] 2024-04-07T17:38:37.936 INFO:tasks.workunit.client.0.smithi100.stdout:from 8 to 9 diff [4194304~4194304,92274688~8388608,109051904~4194304,134217728~4194304,205520896~8388608,314572800~4194304,331350016~8388608,390070272~4194304] 2024-04-07T17:38:37.936 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [4194304~4194304,92274688~8388608,109051904~4194304,134217728~4194304,205520896~8388608,314572800~4194304,331350016~8388608,390070272~4194304] 2024-04-07T17:38:37.940 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [4194304~4194304,92274688~12582912,109051904~4194304,134217728~8388608,205520896~8388608,314572800~4194304,331350016~12582912,390070272~4194304] 2024-04-07T17:38:37.940 INFO:tasks.workunit.client.0.smithi100.stdout:from 8 to 10 diff [4194304~12582912,46137344~4194304,92274688~8388608,109051904~4194304,134217728~4194304,167772160~4194304,205520896~8388608,272629760~12582912,301989888~8388608,314572800~4194304,331350016~8388608,385875968~8388608] 2024-04-07T17:38:37.940 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [4194304~12582912,46137344~4194304,92274688~8388608,109051904~4194304,134217728~4194304,167772160~4194304,205520896~8388608,272629760~12582912,301989888~8388608,314572800~4194304,331350016~8388608,385875968~8388608] 2024-04-07T17:38:37.944 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [4194304~12582912,46137344~8388608,92274688~12582912,109051904~4194304,134217728~8388608,167772160~4194304,205520896~8388608,272629760~16777216,301989888~16777216,331350016~12582912,385875968~8388608] 2024-04-07T17:38:37.944 INFO:tasks.workunit.client.0.smithi100.stdout:from 8 to 11 diff [4194304~12582912,46137344~4194304,58720256~4194304,71303168~8388608,92274688~8388608,109051904~4194304,134217728~4194304,167772160~4194304,180355072~4194304,205520896~8388608,255852544~8388608,268435456~16777216,293601280~4194304,301989888~8388608,314572800~4194304,331350016~8388608,364904448~8388608,385875968~8388608] 2024-04-07T17:38:37.944 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [4194304~12582912,46137344~4194304,58720256~4194304,71303168~8388608,92274688~8388608,109051904~4194304,134217728~4194304,167772160~4194304,180355072~4194304,205520896~8388608,255852544~8388608,268435456~16777216,293601280~4194304,301989888~8388608,314572800~4194304,331350016~8388608,364904448~8388608,385875968~8388608] 2024-04-07T17:38:37.949 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [4194304~12582912,46137344~8388608,58720256~4194304,71303168~8388608,92274688~12582912,109051904~4194304,134217728~8388608,167772160~4194304,180355072~4194304,205520896~8388608,255852544~33554432,293601280~4194304,301989888~16777216,331350016~12582912,364904448~12582912,385875968~8388608] 2024-04-07T17:38:37.949 INFO:tasks.workunit.client.0.smithi100.stdout:from 8 to 12 diff [0~16777216,33554432~8388608,46137344~4194304,54525952~4194304,71303168~8388608,92274688~8388608,109051904~12582912,134217728~4194304,167772160~4194304,180355072~4194304,192937984~8388608,205520896~8388608,255852544~8388608,268435456~16777216,293601280~4194304,301989888~8388608,314572800~4194304,331350016~8388608,343932928~12582912,364904448~8388608,381681664~12582912] 2024-04-07T17:38:37.949 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~16777216,33554432~8388608,46137344~4194304,54525952~4194304,71303168~8388608,92274688~8388608,109051904~12582912,134217728~4194304,167772160~4194304,180355072~4194304,192937984~8388608,205520896~8388608,255852544~8388608,268435456~16777216,293601280~4194304,301989888~8388608,314572800~4194304,331350016~8388608,343932928~12582912,364904448~8388608,381681664~12582912] 2024-04-07T17:38:37.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:38:37 smithi100 bash[74942]: cluster 2024-04-07T17:38:36.509470+0000 mgr.y (mgr.25175) 2497 : cluster [DBG] pgmap v4151: 137 pgs: 137 active+clean; 829 MiB data, 5.6 GiB used, 710 GiB / 715 GiB avail; 124 KiB/s rd, 33 MiB/s wr, 214 op/s 2024-04-07T17:38:37.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:38:37 smithi100 bash[74942]: cluster 2024-04-07T17:38:36.551390+0000 mon.a (mon.0) 3974 : cluster [DBG] osdmap e2080: 8 total, 8 up, 8 in 2024-04-07T17:38:37.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:38:37 smithi100 bash[76202]: cluster 2024-04-07T17:38:36.509470+0000 mgr.y (mgr.25175) 2497 : cluster [DBG] pgmap v4151: 137 pgs: 137 active+clean; 829 MiB data, 5.6 GiB used, 710 GiB / 715 GiB avail; 124 KiB/s rd, 33 MiB/s wr, 214 op/s 2024-04-07T17:38:37.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:38:37 smithi100 bash[76202]: cluster 2024-04-07T17:38:36.551390+0000 mon.a (mon.0) 3974 : cluster [DBG] osdmap e2080: 8 total, 8 up, 8 in 2024-04-07T17:38:37.955 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~16777216,33554432~8388608,46137344~16777216,71303168~8388608,92274688~12582912,109051904~12582912,134217728~8388608,167772160~4194304,180355072~4194304,192937984~8388608,205520896~8388608,255852544~33554432,293601280~4194304,301989888~16777216,331350016~29360128,364904448~12582912,381681664~12582912] 2024-04-07T17:38:37.955 INFO:tasks.workunit.client.0.smithi100.stdout:from 8 to 13 diff [0~16777216,33554432~8388608,46137344~4194304,54525952~4194304,62914560~25165824,92274688~8388608,104857600~16777216,134217728~4194304,142606336~8388608,167772160~4194304,176160768~8388608,188743680~12582912,205520896~8388608,255852544~8388608,268435456~16777216,293601280~4194304,301989888~8388608,314572800~4194304,331350016~25165824,364904448~8388608,381681664~12582912] 2024-04-07T17:38:37.955 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~16777216,33554432~8388608,46137344~4194304,54525952~4194304,62914560~25165824,92274688~8388608,104857600~16777216,134217728~4194304,142606336~8388608,167772160~4194304,176160768~8388608,188743680~12582912,205520896~8388608,255852544~8388608,268435456~16777216,293601280~4194304,301989888~8388608,314572800~4194304,331350016~25165824,364904448~8388608,381681664~12582912] 2024-04-07T17:38:37.962 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~16777216,33554432~8388608,46137344~41943040,92274688~29360128,134217728~20971520,167772160~4194304,176160768~25165824,205520896~8388608,255852544~33554432,293601280~4194304,301989888~16777216,331350016~29360128,364904448~12582912,381681664~12582912] 2024-04-07T17:38:37.962 INFO:tasks.workunit.client.0.smithi100.stdout:from 8 to 14 diff [0~20971520,33554432~8388608,46137344~4194304,54525952~4194304,62914560~25165824,92274688~8388608,104857600~16777216,134217728~4194304,142606336~12582912,163577856~8388608,176160768~8388608,188743680~12582912,205520896~8388608,226492416~8388608,255852544~8388608,268435456~16777216,293601280~4194304,301989888~8388608,314572800~4194304,331350016~25165824,364904448~8388608,381681664~12582912] 2024-04-07T17:38:37.962 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~20971520,33554432~8388608,46137344~4194304,54525952~4194304,62914560~25165824,92274688~8388608,104857600~16777216,134217728~4194304,142606336~12582912,163577856~8388608,176160768~8388608,188743680~12582912,205520896~8388608,226492416~8388608,255852544~8388608,268435456~16777216,293601280~4194304,301989888~8388608,314572800~4194304,331350016~25165824,364904448~8388608,381681664~12582912] 2024-04-07T17:38:37.970 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~20971520,33554432~8388608,46137344~41943040,92274688~29360128,134217728~20971520,163577856~8388608,176160768~25165824,205520896~8388608,226492416~8388608,255852544~33554432,293601280~4194304,301989888~16777216,331350016~29360128,364904448~12582912,381681664~12582912] 2024-04-07T17:38:37.970 INFO:tasks.workunit.client.0.smithi100.stdout:from 8 to 15 diff [0~20971520,33554432~8388608,46137344~4194304,54525952~4194304,62914560~25165824,92274688~8388608,104857600~16777216,134217728~4194304,142606336~12582912,163577856~8388608,176160768~8388608,188743680~12582912,205520896~8388608,226492416~8388608,255852544~8388608,268435456~16777216,293601280~16777216,314572800~12582912,331350016~25165824,364904448~29360128] 2024-04-07T17:38:37.970 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~20971520,33554432~8388608,46137344~4194304,54525952~4194304,62914560~25165824,92274688~8388608,104857600~16777216,134217728~4194304,142606336~12582912,163577856~8388608,176160768~8388608,188743680~12582912,205520896~8388608,226492416~8388608,255852544~8388608,268435456~16777216,293601280~16777216,314572800~12582912,331350016~25165824,364904448~29360128] 2024-04-07T17:38:37.978 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~20971520,33554432~8388608,46137344~41943040,92274688~29360128,134217728~20971520,163577856~8388608,176160768~25165824,205520896~8388608,226492416~8388608,255852544~33554432,293601280~67108864,364904448~29360128] 2024-04-07T17:38:37.978 INFO:tasks.workunit.client.0.smithi100.stdout:from 8 to 16 diff [0~20971520,33554432~8388608,46137344~4194304,54525952~4194304,62914560~25165824,92274688~29360128,130023424~8388608,142606336~12582912,159383552~12582912,176160768~8388608,188743680~12582912,205520896~8388608,226492416~8388608,255852544~8388608,268435456~16777216,293601280~16777216,314572800~12582912,331350016~25165824,364904448~29360128] 2024-04-07T17:38:37.978 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~20971520,33554432~8388608,46137344~4194304,54525952~4194304,62914560~25165824,92274688~29360128,130023424~8388608,142606336~12582912,159383552~12582912,176160768~8388608,188743680~12582912,205520896~8388608,226492416~8388608,255852544~8388608,268435456~16777216,293601280~16777216,314572800~12582912,331350016~25165824,364904448~29360128] 2024-04-07T17:38:37.988 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~20971520,33554432~8388608,46137344~41943040,92274688~29360128,130023424~25165824,159383552~12582912,176160768~25165824,205520896~8388608,226492416~12582912,255852544~33554432,293601280~67108864,364904448~33554432] 2024-04-07T17:38:37.988 INFO:tasks.workunit.client.0.smithi100.stdout:from 8 to 17 diff [0~29360128,33554432~8388608,46137344~4194304,54525952~4194304,62914560~25165824,92274688~62914560,159383552~12582912,176160768~25165824,205520896~8388608,226492416~8388608,255852544~8388608,268435456~16777216,293601280~16777216,314572800~12582912,331350016~29360128,364904448~29360128,402653184~8388608] 2024-04-07T17:38:37.988 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~8388608,46137344~4194304,54525952~4194304,62914560~25165824,92274688~62914560,159383552~12582912,176160768~25165824,205520896~8388608,226492416~8388608,255852544~8388608,268435456~16777216,293601280~16777216,314572800~12582912,331350016~29360128,364904448~29360128,402653184~8388608] 2024-04-07T17:38:38.000 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~41943040,46137344~41943040,92274688~62914560,159383552~41943040,205520896~8388608,226492416~12582912,255852544~33554432,293601280~67108864,364904448~33554432,402653184~8388608] 2024-04-07T17:38:38.000 INFO:tasks.workunit.client.0.smithi100.stdout:from 8 to 18 diff [0~29360128,33554432~8388608,46137344~4194304,54525952~33554432,92274688~79691776,176160768~46137344,226492416~8388608,251658240~12582912,268435456~16777216,293601280~16777216,314572800~12582912,331350016~29360128,364904448~29360128,402653184~8388608] 2024-04-07T17:38:38.000 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~8388608,46137344~4194304,54525952~33554432,92274688~79691776,176160768~46137344,226492416~8388608,251658240~12582912,268435456~16777216,293601280~16777216,314572800~12582912,331350016~29360128,364904448~29360128,402653184~8388608] 2024-04-07T17:38:38.013 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~41943040,46137344~41943040,92274688~146800640,251658240~37748736,293601280~67108864,364904448~33554432,402653184~8388608] 2024-04-07T17:38:38.013 INFO:tasks.workunit.client.0.smithi100.stdout:from 8 to 19 diff [0~29360128,33554432~8388608,46137344~4194304,54525952~33554432,92274688~142606336,251658240~12582912,268435456~16777216,293601280~16777216,314572800~12582912,331350016~29360128,364904448~29360128,402653184~8388608] 2024-04-07T17:38:38.013 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~8388608,46137344~4194304,54525952~33554432,92274688~142606336,251658240~12582912,268435456~16777216,293601280~16777216,314572800~12582912,331350016~29360128,364904448~29360128,402653184~8388608] 2024-04-07T17:38:38.027 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~41943040,46137344~41943040,92274688~146800640,251658240~37748736,293601280~67108864,364904448~33554432,402653184~8388608] 2024-04-07T17:38:38.027 INFO:tasks.workunit.client.0.smithi100.stdout:from 9 to 10 diff [8388608~8388608,46137344~4194304,134217728~4194304,167772160~4194304,272629760~12582912,301989888~8388608,385875968~4194304] 2024-04-07T17:38:38.027 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [8388608~8388608,46137344~4194304,134217728~4194304,167772160~4194304,272629760~12582912,301989888~8388608,385875968~4194304] 2024-04-07T17:38:38.030 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [8388608~8388608,46137344~8388608,134217728~4194304,167772160~4194304,272629760~16777216,301989888~12582912,385875968~8388608] 2024-04-07T17:38:38.030 INFO:tasks.workunit.client.0.smithi100.stdout:from 9 to 11 diff [8388608~8388608,46137344~4194304,58720256~4194304,71303168~8388608,134217728~4194304,167772160~4194304,180355072~4194304,209715200~4194304,255852544~8388608,268435456~16777216,293601280~4194304,301989888~8388608,364904448~8388608,385875968~4194304] 2024-04-07T17:38:38.030 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [8388608~8388608,46137344~4194304,58720256~4194304,71303168~8388608,134217728~4194304,167772160~4194304,180355072~4194304,209715200~4194304,255852544~8388608,268435456~16777216,293601280~4194304,301989888~8388608,364904448~8388608,385875968~4194304] 2024-04-07T17:38:38.035 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [8388608~8388608,46137344~8388608,58720256~4194304,71303168~8388608,134217728~4194304,167772160~4194304,180355072~4194304,209715200~4194304,255852544~33554432,293601280~4194304,301989888~12582912,364904448~12582912,385875968~8388608] 2024-04-07T17:38:38.035 INFO:tasks.workunit.client.0.smithi100.stdout:from 9 to 12 diff [0~16777216,33554432~8388608,46137344~4194304,54525952~4194304,71303168~8388608,113246208~8388608,134217728~4194304,167772160~4194304,180355072~4194304,192937984~8388608,209715200~4194304,255852544~8388608,268435456~16777216,293601280~4194304,301989888~8388608,343932928~12582912,364904448~8388608,381681664~8388608] 2024-04-07T17:38:38.035 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~16777216,33554432~8388608,46137344~4194304,54525952~4194304,71303168~8388608,113246208~8388608,134217728~4194304,167772160~4194304,180355072~4194304,192937984~8388608,209715200~4194304,255852544~8388608,268435456~16777216,293601280~4194304,301989888~8388608,343932928~12582912,364904448~8388608,381681664~8388608] 2024-04-07T17:38:38.042 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~16777216,33554432~8388608,46137344~16777216,71303168~8388608,113246208~8388608,134217728~4194304,167772160~4194304,180355072~4194304,192937984~8388608,209715200~4194304,255852544~33554432,293601280~4194304,301989888~12582912,343932928~16777216,364904448~12582912,381681664~12582912] 2024-04-07T17:38:38.042 INFO:tasks.workunit.client.0.smithi100.stdout:from 9 to 13 diff [0~16777216,33554432~8388608,46137344~4194304,54525952~4194304,62914560~25165824,104857600~16777216,134217728~4194304,142606336~8388608,167772160~4194304,176160768~8388608,188743680~12582912,209715200~4194304,255852544~8388608,268435456~16777216,293601280~4194304,301989888~8388608,331350016~8388608,343932928~12582912,364904448~8388608,381681664~8388608] 2024-04-07T17:38:38.042 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~16777216,33554432~8388608,46137344~4194304,54525952~4194304,62914560~25165824,104857600~16777216,134217728~4194304,142606336~8388608,167772160~4194304,176160768~8388608,188743680~12582912,209715200~4194304,255852544~8388608,268435456~16777216,293601280~4194304,301989888~8388608,331350016~8388608,343932928~12582912,364904448~8388608,381681664~8388608] 2024-04-07T17:38:38.048 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~16777216,33554432~8388608,46137344~41943040,104857600~16777216,134217728~4194304,142606336~12582912,167772160~4194304,176160768~25165824,209715200~4194304,255852544~33554432,293601280~4194304,301989888~12582912,331350016~8388608,343932928~16777216,364904448~12582912,381681664~12582912] 2024-04-07T17:38:38.048 INFO:tasks.workunit.client.0.smithi100.stdout:from 9 to 14 diff [0~20971520,33554432~8388608,46137344~4194304,54525952~4194304,62914560~25165824,104857600~16777216,134217728~4194304,142606336~12582912,163577856~8388608,176160768~8388608,188743680~12582912,205520896~8388608,226492416~8388608,255852544~8388608,268435456~16777216,293601280~4194304,301989888~8388608,331350016~8388608,343932928~12582912,364904448~8388608,381681664~12582912] 2024-04-07T17:38:38.048 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~20971520,33554432~8388608,46137344~4194304,54525952~4194304,62914560~25165824,104857600~16777216,134217728~4194304,142606336~12582912,163577856~8388608,176160768~8388608,188743680~12582912,205520896~8388608,226492416~8388608,255852544~8388608,268435456~16777216,293601280~4194304,301989888~8388608,331350016~8388608,343932928~12582912,364904448~8388608,381681664~12582912] 2024-04-07T17:38:38.056 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~20971520,33554432~8388608,46137344~41943040,104857600~16777216,134217728~4194304,142606336~12582912,163577856~8388608,176160768~25165824,205520896~8388608,226492416~8388608,255852544~33554432,293601280~4194304,301989888~12582912,331350016~8388608,343932928~16777216,364904448~12582912,381681664~12582912] 2024-04-07T17:38:38.056 INFO:tasks.workunit.client.0.smithi100.stdout:from 9 to 15 diff [0~20971520,33554432~8388608,46137344~4194304,54525952~4194304,62914560~25165824,104857600~16777216,134217728~4194304,142606336~12582912,163577856~8388608,176160768~8388608,188743680~12582912,205520896~8388608,226492416~8388608,255852544~8388608,268435456~16777216,293601280~16777216,318767104~8388608,331350016~8388608,343932928~12582912,364904448~29360128] 2024-04-07T17:38:38.056 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~20971520,33554432~8388608,46137344~4194304,54525952~4194304,62914560~25165824,104857600~16777216,134217728~4194304,142606336~12582912,163577856~8388608,176160768~8388608,188743680~12582912,205520896~8388608,226492416~8388608,255852544~8388608,268435456~16777216,293601280~16777216,318767104~8388608,331350016~8388608,343932928~12582912,364904448~29360128] 2024-04-07T17:38:38.063 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~20971520,33554432~8388608,46137344~41943040,104857600~16777216,134217728~4194304,142606336~12582912,163577856~8388608,176160768~25165824,205520896~8388608,226492416~8388608,255852544~33554432,293601280~20971520,318767104~20971520,343932928~16777216,364904448~29360128] 2024-04-07T17:38:38.064 INFO:tasks.workunit.client.0.smithi100.stdout:from 9 to 16 diff [0~20971520,33554432~8388608,46137344~4194304,54525952~4194304,62914560~25165824,100663296~20971520,130023424~8388608,142606336~12582912,159383552~12582912,176160768~8388608,188743680~12582912,205520896~8388608,226492416~8388608,255852544~8388608,268435456~16777216,293601280~16777216,314572800~12582912,331350016~8388608,343932928~12582912,364904448~29360128] 2024-04-07T17:38:38.064 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~20971520,33554432~8388608,46137344~4194304,54525952~4194304,62914560~25165824,100663296~20971520,130023424~8388608,142606336~12582912,159383552~12582912,176160768~8388608,188743680~12582912,205520896~8388608,226492416~8388608,255852544~8388608,268435456~16777216,293601280~16777216,314572800~12582912,331350016~8388608,343932928~12582912,364904448~29360128] 2024-04-07T17:38:38.072 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~20971520,33554432~8388608,46137344~41943040,100663296~20971520,130023424~8388608,142606336~12582912,159383552~12582912,176160768~25165824,205520896~8388608,226492416~12582912,255852544~33554432,293601280~46137344,343932928~16777216,364904448~33554432] 2024-04-07T17:38:38.072 INFO:tasks.workunit.client.0.smithi100.stdout:from 9 to 17 diff [0~29360128,33554432~8388608,46137344~4194304,54525952~4194304,62914560~25165824,100663296~54525952,159383552~12582912,176160768~25165824,205520896~8388608,226492416~8388608,255852544~8388608,268435456~16777216,293601280~16777216,314572800~12582912,331350016~8388608,343932928~16777216,364904448~29360128,402653184~8388608] 2024-04-07T17:38:38.072 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~8388608,46137344~4194304,54525952~4194304,62914560~25165824,100663296~54525952,159383552~12582912,176160768~25165824,205520896~8388608,226492416~8388608,255852544~8388608,268435456~16777216,293601280~16777216,314572800~12582912,331350016~8388608,343932928~16777216,364904448~29360128,402653184~8388608] 2024-04-07T17:38:38.082 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~41943040,46137344~41943040,100663296~54525952,159383552~41943040,205520896~8388608,226492416~12582912,255852544~33554432,293601280~46137344,343932928~16777216,364904448~33554432,402653184~8388608] 2024-04-07T17:38:38.082 INFO:tasks.workunit.client.0.smithi100.stdout:from 9 to 18 diff [0~29360128,33554432~8388608,46137344~4194304,54525952~33554432,100663296~71303168,176160768~46137344,226492416~8388608,251658240~12582912,268435456~16777216,293601280~16777216,314572800~12582912,331350016~8388608,343932928~16777216,364904448~29360128,402653184~8388608] 2024-04-07T17:38:38.082 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~8388608,46137344~4194304,54525952~33554432,100663296~71303168,176160768~46137344,226492416~8388608,251658240~12582912,268435456~16777216,293601280~16777216,314572800~12582912,331350016~8388608,343932928~16777216,364904448~29360128,402653184~8388608] 2024-04-07T17:38:38.093 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~41943040,46137344~41943040,100663296~138412032,251658240~37748736,293601280~46137344,343932928~16777216,364904448~33554432,402653184~8388608] 2024-04-07T17:38:38.093 INFO:tasks.workunit.client.0.smithi100.stdout:from 9 to 19 diff [0~29360128,33554432~8388608,46137344~4194304,54525952~33554432,100663296~134217728,251658240~12582912,268435456~16777216,293601280~16777216,314572800~12582912,331350016~8388608,343932928~16777216,364904448~29360128,402653184~8388608] 2024-04-07T17:38:38.093 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~8388608,46137344~4194304,54525952~33554432,100663296~134217728,251658240~12582912,268435456~16777216,293601280~16777216,314572800~12582912,331350016~8388608,343932928~16777216,364904448~29360128,402653184~8388608] 2024-04-07T17:38:38.105 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~41943040,46137344~41943040,100663296~138412032,251658240~37748736,293601280~46137344,343932928~16777216,364904448~33554432,402653184~8388608] 2024-04-07T17:38:38.105 INFO:tasks.workunit.client.0.smithi100.stdout:from 10 to 11 diff [58720256~4194304,71303168~8388608,180355072~4194304,209715200~4194304,255852544~8388608,268435456~4194304,293601280~4194304,364904448~8388608] 2024-04-07T17:38:38.105 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [58720256~4194304,71303168~8388608,180355072~4194304,209715200~4194304,255852544~8388608,268435456~4194304,293601280~4194304,364904448~8388608] 2024-04-07T17:38:38.108 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [58720256~4194304,71303168~8388608,180355072~4194304,209715200~4194304,255852544~16777216,293601280~4194304,364904448~12582912] 2024-04-07T17:38:38.108 INFO:tasks.workunit.client.0.smithi100.stdout:from 10 to 12 diff [0~8388608,33554432~8388608,54525952~4194304,71303168~8388608,113246208~8388608,180355072~4194304,192937984~8388608,209715200~4194304,255852544~8388608,268435456~4194304,293601280~4194304,343932928~12582912,364904448~8388608,381681664~4194304] 2024-04-07T17:38:38.108 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~8388608,33554432~8388608,54525952~4194304,71303168~8388608,113246208~8388608,180355072~4194304,192937984~8388608,209715200~4194304,255852544~8388608,268435456~4194304,293601280~4194304,343932928~12582912,364904448~8388608,381681664~4194304] 2024-04-07T17:38:38.112 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~12582912,33554432~8388608,54525952~8388608,71303168~8388608,113246208~8388608,180355072~4194304,192937984~8388608,209715200~4194304,255852544~16777216,293601280~4194304,343932928~16777216,364904448~12582912,381681664~4194304] 2024-04-07T17:38:38.112 INFO:tasks.workunit.client.0.smithi100.stdout:from 10 to 13 diff [0~12582912,33554432~8388608,54525952~4194304,62914560~25165824,104857600~16777216,142606336~8388608,176160768~8388608,188743680~12582912,209715200~4194304,255852544~8388608,268435456~4194304,293601280~4194304,331350016~8388608,343932928~12582912,364904448~8388608,381681664~4194304] 2024-04-07T17:38:38.113 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~12582912,33554432~8388608,54525952~4194304,62914560~25165824,104857600~16777216,142606336~8388608,176160768~8388608,188743680~12582912,209715200~4194304,255852544~8388608,268435456~4194304,293601280~4194304,331350016~8388608,343932928~12582912,364904448~8388608,381681664~4194304] 2024-04-07T17:38:38.117 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~12582912,33554432~8388608,54525952~33554432,104857600~16777216,142606336~12582912,176160768~25165824,209715200~4194304,255852544~16777216,293601280~4194304,331350016~8388608,343932928~16777216,364904448~12582912,381681664~4194304] 2024-04-07T17:38:38.117 INFO:tasks.workunit.client.0.smithi100.stdout:from 10 to 14 diff [0~12582912,16777216~4194304,33554432~8388608,46137344~4194304,54525952~4194304,62914560~25165824,104857600~16777216,142606336~12582912,163577856~4194304,176160768~8388608,188743680~12582912,205520896~8388608,226492416~8388608,255852544~8388608,268435456~4194304,293601280~4194304,331350016~8388608,343932928~12582912,364904448~8388608,381681664~12582912] 2024-04-07T17:38:38.117 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~12582912,16777216~4194304,33554432~8388608,46137344~4194304,54525952~4194304,62914560~25165824,104857600~16777216,142606336~12582912,163577856~4194304,176160768~8388608,188743680~12582912,205520896~8388608,226492416~8388608,255852544~8388608,268435456~4194304,293601280~4194304,331350016~8388608,343932928~12582912,364904448~8388608,381681664~12582912] 2024-04-07T17:38:38.123 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~12582912,16777216~4194304,33554432~8388608,46137344~4194304,54525952~33554432,104857600~16777216,142606336~12582912,163577856~4194304,176160768~25165824,205520896~8388608,226492416~8388608,255852544~16777216,293601280~4194304,331350016~8388608,343932928~16777216,364904448~12582912,381681664~12582912] 2024-04-07T17:38:38.123 INFO:tasks.workunit.client.0.smithi100.stdout:from 10 to 15 diff [0~12582912,16777216~4194304,33554432~8388608,46137344~4194304,54525952~4194304,62914560~25165824,104857600~16777216,142606336~12582912,163577856~4194304,176160768~8388608,188743680~12582912,205520896~8388608,226492416~8388608,255852544~8388608,268435456~4194304,293601280~12582912,318767104~8388608,331350016~8388608,343932928~12582912,364904448~29360128] 2024-04-07T17:38:38.123 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~12582912,16777216~4194304,33554432~8388608,46137344~4194304,54525952~4194304,62914560~25165824,104857600~16777216,142606336~12582912,163577856~4194304,176160768~8388608,188743680~12582912,205520896~8388608,226492416~8388608,255852544~8388608,268435456~4194304,293601280~12582912,318767104~8388608,331350016~8388608,343932928~12582912,364904448~29360128] 2024-04-07T17:38:38.130 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~12582912,16777216~4194304,33554432~8388608,46137344~4194304,54525952~33554432,104857600~16777216,142606336~12582912,163577856~4194304,176160768~25165824,205520896~8388608,226492416~8388608,255852544~16777216,293601280~16777216,318767104~20971520,343932928~16777216,364904448~29360128] 2024-04-07T17:38:38.130 INFO:tasks.workunit.client.0.smithi100.stdout:from 10 to 16 diff [0~12582912,16777216~4194304,33554432~8388608,46137344~4194304,54525952~4194304,62914560~25165824,100663296~20971520,130023424~8388608,142606336~12582912,159383552~8388608,176160768~8388608,188743680~12582912,205520896~8388608,226492416~8388608,255852544~8388608,268435456~4194304,293601280~12582912,314572800~12582912,331350016~8388608,343932928~12582912,364904448~29360128] 2024-04-07T17:38:38.130 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~12582912,16777216~4194304,33554432~8388608,46137344~4194304,54525952~4194304,62914560~25165824,100663296~20971520,130023424~8388608,142606336~12582912,159383552~8388608,176160768~8388608,188743680~12582912,205520896~8388608,226492416~8388608,255852544~8388608,268435456~4194304,293601280~12582912,314572800~12582912,331350016~8388608,343932928~12582912,364904448~29360128] 2024-04-07T17:38:38.138 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~12582912,16777216~4194304,33554432~8388608,46137344~4194304,54525952~33554432,100663296~20971520,130023424~8388608,142606336~12582912,159383552~8388608,176160768~25165824,205520896~8388608,226492416~12582912,255852544~16777216,293601280~16777216,314572800~25165824,343932928~16777216,364904448~33554432] 2024-04-07T17:38:38.138 INFO:tasks.workunit.client.0.smithi100.stdout:from 10 to 17 diff [0~12582912,16777216~12582912,33554432~8388608,46137344~4194304,54525952~4194304,62914560~25165824,100663296~54525952,159383552~12582912,176160768~25165824,205520896~8388608,226492416~8388608,255852544~8388608,268435456~4194304,281018368~8388608,293601280~12582912,314572800~12582912,331350016~8388608,343932928~16777216,364904448~29360128,402653184~8388608] 2024-04-07T17:38:38.138 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~12582912,16777216~12582912,33554432~8388608,46137344~4194304,54525952~4194304,62914560~25165824,100663296~54525952,159383552~12582912,176160768~25165824,205520896~8388608,226492416~8388608,255852544~8388608,268435456~4194304,281018368~8388608,293601280~12582912,314572800~12582912,331350016~8388608,343932928~16777216,364904448~29360128,402653184~8388608] 2024-04-07T17:38:38.148 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~12582912,16777216~25165824,46137344~4194304,54525952~33554432,100663296~54525952,159383552~41943040,205520896~8388608,226492416~12582912,255852544~16777216,281018368~8388608,293601280~16777216,314572800~25165824,343932928~16777216,364904448~33554432,402653184~8388608] 2024-04-07T17:38:38.148 INFO:tasks.workunit.client.0.smithi100.stdout:from 10 to 18 diff [0~12582912,16777216~12582912,33554432~8388608,46137344~4194304,54525952~33554432,100663296~71303168,176160768~46137344,226492416~8388608,251658240~12582912,268435456~4194304,281018368~8388608,293601280~12582912,314572800~12582912,331350016~8388608,343932928~16777216,364904448~29360128,402653184~8388608] 2024-04-07T17:38:38.148 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~12582912,16777216~12582912,33554432~8388608,46137344~4194304,54525952~33554432,100663296~71303168,176160768~46137344,226492416~8388608,251658240~12582912,268435456~4194304,281018368~8388608,293601280~12582912,314572800~12582912,331350016~8388608,343932928~16777216,364904448~29360128,402653184~8388608] 2024-04-07T17:38:38.158 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~12582912,16777216~25165824,46137344~4194304,54525952~33554432,100663296~138412032,251658240~20971520,281018368~8388608,293601280~16777216,314572800~25165824,343932928~16777216,364904448~33554432,402653184~8388608] 2024-04-07T17:38:38.158 INFO:tasks.workunit.client.0.smithi100.stdout:from 10 to 19 diff [0~12582912,16777216~12582912,33554432~8388608,46137344~4194304,54525952~33554432,100663296~71303168,176160768~58720256,251658240~12582912,268435456~4194304,281018368~8388608,293601280~12582912,314572800~12582912,331350016~8388608,343932928~16777216,364904448~29360128,402653184~8388608] 2024-04-07T17:38:38.158 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~12582912,16777216~12582912,33554432~8388608,46137344~4194304,54525952~33554432,100663296~71303168,176160768~58720256,251658240~12582912,268435456~4194304,281018368~8388608,293601280~12582912,314572800~12582912,331350016~8388608,343932928~16777216,364904448~29360128,402653184~8388608] 2024-04-07T17:38:38.169 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~12582912,16777216~25165824,46137344~4194304,54525952~33554432,100663296~138412032,251658240~20971520,281018368~8388608,293601280~16777216,314572800~25165824,343932928~16777216,364904448~33554432,402653184~8388608] 2024-04-07T17:38:38.170 INFO:tasks.workunit.client.0.smithi100.stdout:from 11 to 12 diff [0~8388608,33554432~8388608,54525952~4194304,71303168~4194304,113246208~8388608,192937984~8388608,343932928~12582912,381681664~4194304] 2024-04-07T17:38:38.170 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~8388608,33554432~8388608,54525952~4194304,71303168~4194304,113246208~8388608,192937984~8388608,343932928~12582912,381681664~4194304] 2024-04-07T17:38:38.172 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~12582912,33554432~8388608,54525952~8388608,71303168~4194304,113246208~8388608,192937984~8388608,343932928~16777216,381681664~4194304] 2024-04-07T17:38:38.172 INFO:tasks.workunit.client.0.smithi100.stdout:from 11 to 13 diff [0~12582912,33554432~8388608,54525952~4194304,62914560~12582912,79691776~8388608,104857600~16777216,142606336~8388608,176160768~8388608,188743680~12582912,331350016~8388608,343932928~12582912,381681664~4194304] 2024-04-07T17:38:38.173 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~12582912,33554432~8388608,54525952~4194304,62914560~12582912,79691776~8388608,104857600~16777216,142606336~8388608,176160768~8388608,188743680~12582912,331350016~8388608,343932928~12582912,381681664~4194304] 2024-04-07T17:38:38.177 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~12582912,33554432~8388608,54525952~20971520,79691776~8388608,104857600~16777216,142606336~12582912,176160768~25165824,331350016~8388608,343932928~16777216,381681664~4194304] 2024-04-07T17:38:38.177 INFO:tasks.workunit.client.0.smithi100.stdout:from 11 to 14 diff [0~12582912,16777216~4194304,33554432~8388608,46137344~4194304,54525952~4194304,62914560~25165824,104857600~16777216,142606336~12582912,163577856~4194304,176160768~8388608,188743680~12582912,205520896~8388608,226492416~8388608,331350016~8388608,343932928~12582912,381681664~12582912] 2024-04-07T17:38:38.177 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~12582912,16777216~4194304,33554432~8388608,46137344~4194304,54525952~4194304,62914560~25165824,104857600~16777216,142606336~12582912,163577856~4194304,176160768~8388608,188743680~12582912,205520896~8388608,226492416~8388608,331350016~8388608,343932928~12582912,381681664~12582912] 2024-04-07T17:38:38.182 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~12582912,16777216~4194304,33554432~8388608,46137344~4194304,54525952~33554432,104857600~16777216,142606336~12582912,163577856~4194304,176160768~25165824,205520896~8388608,226492416~8388608,331350016~8388608,343932928~16777216,381681664~12582912] 2024-04-07T17:38:38.182 INFO:tasks.workunit.client.0.smithi100.stdout:from 11 to 15 diff [0~12582912,16777216~4194304,33554432~8388608,46137344~4194304,54525952~4194304,62914560~25165824,104857600~16777216,142606336~12582912,163577856~4194304,176160768~8388608,188743680~12582912,205520896~8388608,226492416~8388608,297795584~8388608,318767104~8388608,331350016~8388608,343932928~12582912,373293056~20971520] 2024-04-07T17:38:38.182 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~12582912,16777216~4194304,33554432~8388608,46137344~4194304,54525952~4194304,62914560~25165824,104857600~16777216,142606336~12582912,163577856~4194304,176160768~8388608,188743680~12582912,205520896~8388608,226492416~8388608,297795584~8388608,318767104~8388608,331350016~8388608,343932928~12582912,373293056~20971520] 2024-04-07T17:38:38.188 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~12582912,16777216~4194304,33554432~8388608,46137344~4194304,54525952~33554432,104857600~16777216,142606336~12582912,163577856~4194304,176160768~25165824,205520896~8388608,226492416~8388608,297795584~12582912,318767104~20971520,343932928~16777216,373293056~20971520] 2024-04-07T17:38:38.188 INFO:tasks.workunit.client.0.smithi100.stdout:from 11 to 16 diff [0~12582912,16777216~4194304,33554432~8388608,46137344~4194304,54525952~4194304,62914560~25165824,100663296~20971520,130023424~8388608,142606336~12582912,159383552~8388608,176160768~8388608,188743680~12582912,205520896~8388608,226492416~8388608,297795584~8388608,314572800~12582912,331350016~8388608,343932928~12582912,373293056~20971520] 2024-04-07T17:38:38.188 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~12582912,16777216~4194304,33554432~8388608,46137344~4194304,54525952~4194304,62914560~25165824,100663296~20971520,130023424~8388608,142606336~12582912,159383552~8388608,176160768~8388608,188743680~12582912,205520896~8388608,226492416~8388608,297795584~8388608,314572800~12582912,331350016~8388608,343932928~12582912,373293056~20971520] 2024-04-07T17:38:38.195 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~12582912,16777216~4194304,33554432~8388608,46137344~4194304,54525952~33554432,100663296~20971520,130023424~8388608,142606336~12582912,159383552~8388608,176160768~25165824,205520896~8388608,226492416~12582912,297795584~12582912,314572800~25165824,343932928~16777216,373293056~25165824] 2024-04-07T17:38:38.195 INFO:tasks.workunit.client.0.smithi100.stdout:from 11 to 17 diff [0~12582912,16777216~12582912,33554432~8388608,46137344~4194304,54525952~4194304,62914560~25165824,100663296~54525952,159383552~12582912,176160768~25165824,205520896~8388608,226492416~8388608,281018368~8388608,297795584~8388608,314572800~12582912,331350016~8388608,343932928~16777216,373293056~20971520,402653184~8388608] 2024-04-07T17:38:38.195 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~12582912,16777216~12582912,33554432~8388608,46137344~4194304,54525952~4194304,62914560~25165824,100663296~54525952,159383552~12582912,176160768~25165824,205520896~8388608,226492416~8388608,281018368~8388608,297795584~8388608,314572800~12582912,331350016~8388608,343932928~16777216,373293056~20971520,402653184~8388608] 2024-04-07T17:38:38.204 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~12582912,16777216~25165824,46137344~4194304,54525952~33554432,100663296~54525952,159383552~41943040,205520896~8388608,226492416~12582912,281018368~8388608,297795584~12582912,314572800~25165824,343932928~16777216,373293056~25165824,402653184~8388608] 2024-04-07T17:38:38.204 INFO:tasks.workunit.client.0.smithi100.stdout:from 11 to 18 diff [0~12582912,16777216~12582912,33554432~8388608,46137344~4194304,54525952~33554432,100663296~71303168,176160768~46137344,226492416~8388608,251658240~12582912,281018368~8388608,297795584~8388608,314572800~12582912,331350016~8388608,343932928~16777216,373293056~20971520,402653184~8388608] 2024-04-07T17:38:38.204 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~12582912,16777216~12582912,33554432~8388608,46137344~4194304,54525952~33554432,100663296~71303168,176160768~46137344,226492416~8388608,251658240~12582912,281018368~8388608,297795584~8388608,314572800~12582912,331350016~8388608,343932928~16777216,373293056~20971520,402653184~8388608] 2024-04-07T17:38:38.212 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~12582912,16777216~25165824,46137344~4194304,54525952~33554432,100663296~138412032,251658240~16777216,281018368~8388608,297795584~12582912,314572800~25165824,343932928~16777216,373293056~25165824,402653184~8388608] 2024-04-07T17:38:38.212 INFO:tasks.workunit.client.0.smithi100.stdout:from 11 to 19 diff [0~12582912,16777216~12582912,33554432~8388608,46137344~4194304,54525952~33554432,100663296~71303168,176160768~58720256,251658240~12582912,281018368~8388608,297795584~8388608,314572800~12582912,331350016~8388608,343932928~16777216,364904448~29360128,402653184~8388608] 2024-04-07T17:38:38.212 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~12582912,16777216~12582912,33554432~8388608,46137344~4194304,54525952~33554432,100663296~71303168,176160768~58720256,251658240~12582912,281018368~8388608,297795584~8388608,314572800~12582912,331350016~8388608,343932928~16777216,364904448~29360128,402653184~8388608] 2024-04-07T17:38:38.222 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~12582912,16777216~25165824,46137344~4194304,54525952~33554432,100663296~138412032,251658240~16777216,281018368~8388608,297795584~12582912,314572800~25165824,343932928~16777216,364904448~33554432,402653184~8388608] 2024-04-07T17:38:38.222 INFO:tasks.workunit.client.0.smithi100.stdout:from 12 to 13 diff [8388608~4194304,62914560~8388608,79691776~8388608,104857600~8388608,142606336~8388608,176160768~8388608,188743680~8388608,331350016~8388608,348127232~4194304] 2024-04-07T17:38:38.222 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [8388608~4194304,62914560~8388608,79691776~8388608,104857600~8388608,142606336~8388608,176160768~8388608,188743680~8388608,331350016~8388608,348127232~4194304] 2024-04-07T17:38:38.225 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [8388608~4194304,62914560~8388608,79691776~8388608,104857600~8388608,142606336~12582912,176160768~20971520,331350016~8388608,348127232~4194304] 2024-04-07T17:38:38.225 INFO:tasks.workunit.client.0.smithi100.stdout:from 12 to 14 diff [4194304~8388608,16777216~4194304,46137344~4194304,62914560~8388608,75497472~12582912,104857600~8388608,142606336~12582912,163577856~4194304,176160768~8388608,188743680~8388608,205520896~8388608,226492416~8388608,331350016~8388608,348127232~4194304,385875968~8388608] 2024-04-07T17:38:38.225 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [4194304~8388608,16777216~4194304,46137344~4194304,62914560~8388608,75497472~12582912,104857600~8388608,142606336~12582912,163577856~4194304,176160768~8388608,188743680~8388608,205520896~8388608,226492416~8388608,331350016~8388608,348127232~4194304,385875968~8388608] 2024-04-07T17:38:38.229 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [4194304~8388608,16777216~4194304,46137344~4194304,62914560~8388608,75497472~12582912,104857600~8388608,142606336~12582912,163577856~4194304,176160768~20971520,205520896~8388608,226492416~8388608,331350016~8388608,348127232~4194304,385875968~8388608] 2024-04-07T17:38:38.229 INFO:tasks.workunit.client.0.smithi100.stdout:from 12 to 15 diff [4194304~8388608,16777216~4194304,46137344~4194304,62914560~8388608,75497472~12582912,104857600~8388608,142606336~12582912,163577856~4194304,176160768~8388608,188743680~8388608,205520896~8388608,226492416~8388608,297795584~8388608,318767104~8388608,331350016~8388608,348127232~12582912,373293056~20971520] 2024-04-07T17:38:38.229 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [4194304~8388608,16777216~4194304,46137344~4194304,62914560~8388608,75497472~12582912,104857600~8388608,142606336~12582912,163577856~4194304,176160768~8388608,188743680~8388608,205520896~8388608,226492416~8388608,297795584~8388608,318767104~8388608,331350016~8388608,348127232~12582912,373293056~20971520] 2024-04-07T17:38:38.234 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [4194304~8388608,16777216~4194304,46137344~4194304,62914560~8388608,75497472~12582912,104857600~8388608,142606336~12582912,163577856~4194304,176160768~20971520,205520896~8388608,226492416~8388608,297795584~12582912,318767104~20971520,348127232~12582912,373293056~20971520] 2024-04-07T17:38:38.234 INFO:tasks.workunit.client.0.smithi100.stdout:from 12 to 16 diff [4194304~8388608,16777216~4194304,46137344~4194304,62914560~25165824,100663296~12582912,130023424~8388608,142606336~12582912,159383552~8388608,176160768~8388608,188743680~8388608,205520896~8388608,226492416~8388608,297795584~8388608,314572800~12582912,331350016~8388608,343932928~16777216,373293056~20971520] 2024-04-07T17:38:38.234 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [4194304~8388608,16777216~4194304,46137344~4194304,62914560~25165824,100663296~12582912,130023424~8388608,142606336~12582912,159383552~8388608,176160768~8388608,188743680~8388608,205520896~8388608,226492416~8388608,297795584~8388608,314572800~12582912,331350016~8388608,343932928~16777216,373293056~20971520] 2024-04-07T17:38:38.240 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [4194304~8388608,16777216~4194304,46137344~4194304,62914560~25165824,100663296~12582912,130023424~8388608,142606336~12582912,159383552~8388608,176160768~20971520,205520896~8388608,226492416~12582912,297795584~12582912,314572800~25165824,343932928~16777216,373293056~25165824] 2024-04-07T17:38:38.240 INFO:tasks.workunit.client.0.smithi100.stdout:from 12 to 17 diff [4194304~8388608,16777216~12582912,46137344~4194304,62914560~25165824,100663296~12582912,121634816~33554432,159383552~12582912,176160768~20971520,205520896~8388608,226492416~8388608,281018368~8388608,297795584~8388608,314572800~12582912,331350016~8388608,343932928~16777216,373293056~20971520,402653184~8388608] 2024-04-07T17:38:38.240 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [4194304~8388608,16777216~12582912,46137344~4194304,62914560~25165824,100663296~12582912,121634816~33554432,159383552~12582912,176160768~20971520,205520896~8388608,226492416~8388608,281018368~8388608,297795584~8388608,314572800~12582912,331350016~8388608,343932928~16777216,373293056~20971520,402653184~8388608] 2024-04-07T17:38:38.246 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [4194304~8388608,16777216~16777216,46137344~4194304,62914560~25165824,100663296~12582912,121634816~33554432,159383552~37748736,205520896~8388608,226492416~12582912,281018368~8388608,297795584~12582912,314572800~25165824,343932928~16777216,373293056~25165824,402653184~8388608] 2024-04-07T17:38:38.247 INFO:tasks.workunit.client.0.smithi100.stdout:from 12 to 18 diff [4194304~8388608,16777216~12582912,33554432~8388608,46137344~4194304,58720256~29360128,100663296~12582912,121634816~50331648,176160768~20971520,201326592~20971520,226492416~8388608,251658240~12582912,281018368~8388608,297795584~8388608,314572800~12582912,331350016~8388608,343932928~16777216,373293056~20971520,402653184~8388608] 2024-04-07T17:38:38.247 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [4194304~8388608,16777216~12582912,33554432~8388608,46137344~4194304,58720256~29360128,100663296~12582912,121634816~50331648,176160768~20971520,201326592~20971520,226492416~8388608,251658240~12582912,281018368~8388608,297795584~8388608,314572800~12582912,331350016~8388608,343932928~16777216,373293056~20971520,402653184~8388608] 2024-04-07T17:38:38.255 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [4194304~8388608,16777216~25165824,46137344~4194304,58720256~29360128,100663296~12582912,121634816~75497472,201326592~37748736,251658240~16777216,281018368~8388608,297795584~12582912,314572800~25165824,343932928~16777216,373293056~25165824,402653184~8388608] 2024-04-07T17:38:38.255 INFO:tasks.workunit.client.0.smithi100.stdout:from 12 to 19 diff [4194304~8388608,16777216~12582912,33554432~8388608,46137344~4194304,58720256~29360128,100663296~71303168,176160768~20971520,201326592~33554432,251658240~12582912,281018368~8388608,297795584~8388608,314572800~12582912,331350016~8388608,343932928~16777216,364904448~29360128,402653184~8388608] 2024-04-07T17:38:38.255 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [4194304~8388608,16777216~12582912,33554432~8388608,46137344~4194304,58720256~29360128,100663296~71303168,176160768~20971520,201326592~33554432,251658240~12582912,281018368~8388608,297795584~8388608,314572800~12582912,331350016~8388608,343932928~16777216,364904448~29360128,402653184~8388608] 2024-04-07T17:38:38.264 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [4194304~8388608,16777216~25165824,46137344~4194304,58720256~29360128,100663296~96468992,201326592~37748736,251658240~16777216,281018368~8388608,297795584~12582912,314572800~25165824,343932928~16777216,364904448~33554432,402653184~8388608] 2024-04-07T17:38:38.264 INFO:tasks.workunit.client.0.smithi100.stdout:from 13 to 14 diff [4194304~4194304,16777216~4194304,46137344~4194304,75497472~4194304,142606336~8388608,163577856~4194304,180355072~4194304,205520896~8388608,226492416~8388608,385875968~8388608] 2024-04-07T17:38:38.264 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [4194304~4194304,16777216~4194304,46137344~4194304,75497472~4194304,142606336~8388608,163577856~4194304,180355072~4194304,205520896~8388608,226492416~8388608,385875968~8388608] 2024-04-07T17:38:38.267 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [4194304~8388608,16777216~4194304,46137344~4194304,75497472~8388608,142606336~8388608,163577856~4194304,180355072~8388608,205520896~8388608,226492416~8388608,385875968~8388608] 2024-04-07T17:38:38.267 INFO:tasks.workunit.client.0.smithi100.stdout:from 13 to 15 diff [4194304~8388608,16777216~4194304,46137344~4194304,75497472~12582912,104857600~4194304,142606336~8388608,163577856~4194304,180355072~4194304,205520896~8388608,226492416~8388608,297795584~8388608,318767104~8388608,348127232~8388608,373293056~20971520] 2024-04-07T17:38:38.267 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [4194304~8388608,16777216~4194304,46137344~4194304,75497472~12582912,104857600~4194304,142606336~8388608,163577856~4194304,180355072~4194304,205520896~8388608,226492416~8388608,297795584~8388608,318767104~8388608,348127232~8388608,373293056~20971520] 2024-04-07T17:38:38.271 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [4194304~8388608,16777216~4194304,46137344~4194304,75497472~12582912,104857600~8388608,142606336~8388608,163577856~4194304,180355072~8388608,205520896~8388608,226492416~8388608,297795584~12582912,318767104~12582912,348127232~12582912,373293056~20971520] 2024-04-07T17:38:38.271 INFO:tasks.workunit.client.0.smithi100.stdout:from 13 to 16 diff [4194304~8388608,16777216~4194304,46137344~4194304,62914560~25165824,100663296~12582912,130023424~8388608,142606336~8388608,159383552~8388608,180355072~4194304,205520896~8388608,226492416~8388608,297795584~8388608,314572800~12582912,331350016~4194304,343932928~12582912,373293056~20971520] 2024-04-07T17:38:38.271 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [4194304~8388608,16777216~4194304,46137344~4194304,62914560~25165824,100663296~12582912,130023424~8388608,142606336~8388608,159383552~8388608,180355072~4194304,205520896~8388608,226492416~8388608,297795584~8388608,314572800~12582912,331350016~4194304,343932928~12582912,373293056~20971520] 2024-04-07T17:38:38.275 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [4194304~8388608,16777216~4194304,46137344~4194304,62914560~25165824,100663296~12582912,130023424~8388608,142606336~8388608,159383552~8388608,180355072~8388608,205520896~8388608,226492416~12582912,297795584~12582912,314572800~25165824,343932928~16777216,373293056~25165824] 2024-04-07T17:38:38.275 INFO:tasks.workunit.client.0.smithi100.stdout:from 13 to 17 diff [4194304~8388608,16777216~12582912,46137344~4194304,62914560~25165824,100663296~12582912,121634816~29360128,159383552~12582912,180355072~8388608,205520896~8388608,226492416~8388608,281018368~8388608,297795584~8388608,314572800~12582912,331350016~8388608,343932928~16777216,373293056~20971520,402653184~8388608] 2024-04-07T17:38:38.276 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [4194304~8388608,16777216~12582912,46137344~4194304,62914560~25165824,100663296~12582912,121634816~29360128,159383552~12582912,180355072~8388608,205520896~8388608,226492416~8388608,281018368~8388608,297795584~8388608,314572800~12582912,331350016~8388608,343932928~16777216,373293056~20971520,402653184~8388608] 2024-04-07T17:38:38.281 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [4194304~8388608,16777216~16777216,46137344~4194304,62914560~25165824,100663296~12582912,121634816~29360128,159383552~16777216,180355072~12582912,205520896~8388608,226492416~12582912,281018368~8388608,297795584~12582912,314572800~25165824,343932928~16777216,373293056~25165824,402653184~8388608] 2024-04-07T17:38:38.281 INFO:tasks.workunit.client.0.smithi100.stdout:from 13 to 18 diff [4194304~8388608,16777216~12582912,33554432~8388608,46137344~4194304,58720256~29360128,100663296~12582912,121634816~29360128,155189248~16777216,180355072~8388608,201326592~20971520,226492416~8388608,251658240~12582912,281018368~8388608,297795584~8388608,314572800~12582912,331350016~8388608,343932928~16777216,373293056~20971520,402653184~8388608] 2024-04-07T17:38:38.281 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [4194304~8388608,16777216~12582912,33554432~8388608,46137344~4194304,58720256~29360128,100663296~12582912,121634816~29360128,155189248~16777216,180355072~8388608,201326592~20971520,226492416~8388608,251658240~12582912,281018368~8388608,297795584~8388608,314572800~12582912,331350016~8388608,343932928~16777216,373293056~20971520,402653184~8388608] 2024-04-07T17:38:38.288 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [4194304~8388608,16777216~25165824,46137344~4194304,58720256~29360128,100663296~12582912,121634816~54525952,180355072~12582912,201326592~37748736,251658240~16777216,281018368~8388608,297795584~12582912,314572800~25165824,343932928~16777216,373293056~25165824,402653184~8388608] 2024-04-07T17:38:38.288 INFO:tasks.workunit.client.0.smithi100.stdout:from 13 to 19 diff [4194304~8388608,16777216~12582912,33554432~8388608,46137344~4194304,58720256~29360128,100663296~50331648,155189248~16777216,180355072~8388608,201326592~33554432,251658240~12582912,281018368~8388608,297795584~8388608,314572800~12582912,331350016~8388608,343932928~16777216,364904448~29360128,402653184~8388608] 2024-04-07T17:38:38.288 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [4194304~8388608,16777216~12582912,33554432~8388608,46137344~4194304,58720256~29360128,100663296~50331648,155189248~16777216,180355072~8388608,201326592~33554432,251658240~12582912,281018368~8388608,297795584~8388608,314572800~12582912,331350016~8388608,343932928~16777216,364904448~29360128,402653184~8388608] 2024-04-07T17:38:38.296 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [4194304~8388608,16777216~25165824,46137344~4194304,58720256~29360128,100663296~75497472,180355072~12582912,201326592~37748736,251658240~16777216,281018368~8388608,297795584~12582912,314572800~25165824,343932928~16777216,364904448~33554432,402653184~8388608] 2024-04-07T17:38:38.296 INFO:tasks.workunit.client.0.smithi100.stdout:from 14 to 15 diff [4194304~8388608,79691776~8388608,104857600~4194304,297795584~8388608,318767104~8388608,348127232~8388608,373293056~12582912] 2024-04-07T17:38:38.296 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [4194304~8388608,79691776~8388608,104857600~4194304,297795584~8388608,318767104~8388608,348127232~8388608,373293056~12582912] 2024-04-07T17:38:38.299 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [4194304~8388608,79691776~8388608,104857600~8388608,297795584~12582912,318767104~12582912,348127232~12582912,373293056~12582912] 2024-04-07T17:38:38.299 INFO:tasks.workunit.client.0.smithi100.stdout:from 14 to 16 diff [4194304~8388608,62914560~25165824,100663296~12582912,130023424~8388608,159383552~4194304,230686720~4194304,297795584~8388608,314572800~12582912,331350016~4194304,343932928~12582912,373293056~20971520] 2024-04-07T17:38:38.299 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [4194304~8388608,62914560~25165824,100663296~12582912,130023424~8388608,159383552~4194304,230686720~4194304,297795584~8388608,314572800~12582912,331350016~4194304,343932928~12582912,373293056~20971520] 2024-04-07T17:38:38.303 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [4194304~8388608,62914560~25165824,100663296~12582912,130023424~8388608,159383552~4194304,230686720~8388608,297795584~12582912,314572800~25165824,343932928~16777216,373293056~25165824] 2024-04-07T17:38:38.303 INFO:tasks.workunit.client.0.smithi100.stdout:from 14 to 17 diff [4194304~8388608,20971520~8388608,62914560~25165824,100663296~12582912,121634816~20971520,159383552~4194304,167772160~4194304,184549376~8388608,226492416~8388608,281018368~8388608,297795584~8388608,314572800~12582912,331350016~8388608,343932928~16777216,373293056~20971520,402653184~8388608] 2024-04-07T17:38:38.303 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [4194304~8388608,20971520~8388608,62914560~25165824,100663296~12582912,121634816~20971520,159383552~4194304,167772160~4194304,184549376~8388608,226492416~8388608,281018368~8388608,297795584~8388608,314572800~12582912,331350016~8388608,343932928~16777216,373293056~20971520,402653184~8388608] 2024-04-07T17:38:38.308 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [4194304~8388608,20971520~12582912,62914560~25165824,100663296~12582912,121634816~25165824,159383552~4194304,167772160~8388608,184549376~8388608,226492416~12582912,281018368~8388608,297795584~12582912,314572800~25165824,343932928~16777216,373293056~25165824,402653184~8388608] 2024-04-07T17:38:38.308 INFO:tasks.workunit.client.0.smithi100.stdout:from 14 to 18 diff [4194304~8388608,20971520~8388608,33554432~8388608,58720256~29360128,100663296~12582912,121634816~29360128,155189248~8388608,167772160~4194304,184549376~8388608,201326592~8388608,213909504~8388608,226492416~8388608,251658240~12582912,281018368~8388608,297795584~8388608,314572800~12582912,331350016~8388608,343932928~16777216,373293056~20971520,402653184~8388608] 2024-04-07T17:38:38.308 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [4194304~8388608,20971520~8388608,33554432~8388608,58720256~29360128,100663296~12582912,121634816~29360128,155189248~8388608,167772160~4194304,184549376~8388608,201326592~8388608,213909504~8388608,226492416~8388608,251658240~12582912,281018368~8388608,297795584~8388608,314572800~12582912,331350016~8388608,343932928~16777216,373293056~20971520,402653184~8388608] 2024-04-07T17:38:38.314 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [4194304~8388608,20971520~20971520,58720256~29360128,100663296~12582912,121634816~41943040,167772160~8388608,184549376~8388608,201326592~37748736,251658240~16777216,281018368~8388608,297795584~12582912,314572800~25165824,343932928~16777216,373293056~25165824,402653184~8388608] 2024-04-07T17:38:38.314 INFO:tasks.workunit.client.0.smithi100.stdout:from 14 to 19 diff [4194304~8388608,16777216~12582912,33554432~8388608,58720256~29360128,100663296~50331648,155189248~8388608,167772160~4194304,184549376~8388608,201326592~8388608,213909504~20971520,251658240~12582912,281018368~8388608,297795584~8388608,314572800~12582912,331350016~8388608,343932928~16777216,364904448~29360128,402653184~8388608] 2024-04-07T17:38:38.314 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [4194304~8388608,16777216~12582912,33554432~8388608,58720256~29360128,100663296~50331648,155189248~8388608,167772160~4194304,184549376~8388608,201326592~8388608,213909504~20971520,251658240~12582912,281018368~8388608,297795584~8388608,314572800~12582912,331350016~8388608,343932928~16777216,364904448~29360128,402653184~8388608] 2024-04-07T17:38:38.321 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [4194304~8388608,16777216~25165824,58720256~29360128,100663296~62914560,167772160~8388608,184549376~8388608,201326592~37748736,251658240~16777216,281018368~8388608,297795584~12582912,314572800~25165824,343932928~16777216,364904448~33554432,402653184~8388608] 2024-04-07T17:38:38.321 INFO:tasks.workunit.client.0.smithi100.stdout:from 15 to 16 diff [62914560~16777216,100663296~8388608,130023424~8388608,159383552~4194304,230686720~4194304,314572800~8388608,331350016~4194304,343932928~4194304,385875968~8388608] 2024-04-07T17:38:38.321 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [62914560~16777216,100663296~8388608,130023424~8388608,159383552~4194304,230686720~4194304,314572800~8388608,331350016~4194304,343932928~4194304,385875968~8388608] 2024-04-07T17:38:38.324 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [62914560~20971520,100663296~8388608,130023424~8388608,159383552~4194304,230686720~8388608,314572800~12582912,331350016~8388608,343932928~4194304,385875968~12582912] 2024-04-07T17:38:38.324 INFO:tasks.workunit.client.0.smithi100.stdout:from 15 to 17 diff [20971520~8388608,62914560~16777216,100663296~8388608,121634816~20971520,159383552~4194304,167772160~4194304,184549376~8388608,226492416~8388608,281018368~8388608,314572800~8388608,331350016~8388608,343932928~4194304,352321536~8388608,385875968~8388608,402653184~8388608] 2024-04-07T17:38:38.324 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [20971520~8388608,62914560~16777216,100663296~8388608,121634816~20971520,159383552~4194304,167772160~4194304,184549376~8388608,226492416~8388608,281018368~8388608,314572800~8388608,331350016~8388608,343932928~4194304,352321536~8388608,385875968~8388608,402653184~8388608] 2024-04-07T17:38:38.328 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [20971520~12582912,62914560~20971520,100663296~8388608,121634816~25165824,159383552~4194304,167772160~8388608,184549376~8388608,226492416~12582912,281018368~8388608,314572800~12582912,331350016~8388608,343932928~4194304,352321536~8388608,385875968~12582912,402653184~8388608] 2024-04-07T17:38:38.328 INFO:tasks.workunit.client.0.smithi100.stdout:from 15 to 18 diff [20971520~8388608,33554432~8388608,58720256~20971520,100663296~8388608,121634816~29360128,155189248~8388608,167772160~4194304,184549376~8388608,201326592~8388608,213909504~8388608,226492416~8388608,251658240~12582912,281018368~8388608,314572800~8388608,331350016~8388608,343932928~4194304,352321536~8388608,385875968~8388608,402653184~8388608] 2024-04-07T17:38:38.328 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [20971520~8388608,33554432~8388608,58720256~20971520,100663296~8388608,121634816~29360128,155189248~8388608,167772160~4194304,184549376~8388608,201326592~8388608,213909504~8388608,226492416~8388608,251658240~12582912,281018368~8388608,314572800~8388608,331350016~8388608,343932928~4194304,352321536~8388608,385875968~8388608,402653184~8388608] 2024-04-07T17:38:38.333 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [20971520~20971520,58720256~25165824,100663296~8388608,121634816~41943040,167772160~8388608,184549376~8388608,201326592~37748736,251658240~16777216,281018368~8388608,314572800~12582912,331350016~8388608,343932928~4194304,352321536~8388608,385875968~12582912,402653184~8388608] 2024-04-07T17:38:38.333 INFO:tasks.workunit.client.0.smithi100.stdout:from 15 to 19 diff [16777216~12582912,33554432~8388608,58720256~20971520,100663296~8388608,113246208~37748736,155189248~8388608,167772160~4194304,184549376~8388608,201326592~8388608,213909504~20971520,251658240~12582912,281018368~8388608,314572800~8388608,331350016~8388608,343932928~4194304,352321536~8388608,364904448~8388608,385875968~8388608,402653184~8388608] 2024-04-07T17:38:38.333 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [16777216~12582912,33554432~8388608,58720256~20971520,100663296~8388608,113246208~37748736,155189248~8388608,167772160~4194304,184549376~8388608,201326592~8388608,213909504~20971520,251658240~12582912,281018368~8388608,314572800~8388608,331350016~8388608,343932928~4194304,352321536~8388608,364904448~8388608,385875968~8388608,402653184~8388608] 2024-04-07T17:38:38.339 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [16777216~25165824,58720256~25165824,100663296~8388608,113246208~50331648,167772160~8388608,184549376~8388608,201326592~37748736,251658240~16777216,281018368~8388608,314572800~12582912,331350016~8388608,343932928~4194304,352321536~8388608,364904448~8388608,385875968~12582912,402653184~8388608] 2024-04-07T17:38:38.339 INFO:tasks.workunit.client.0.smithi100.stdout:from 16 to 17 diff [20971520~8388608,121634816~8388608,134217728~8388608,167772160~4194304,184549376~8388608,226492416~4194304,281018368~8388608,335544320~4194304,352321536~8388608,402653184~8388608] 2024-04-07T17:38:38.340 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [20971520~8388608,121634816~8388608,134217728~8388608,167772160~4194304,184549376~8388608,226492416~4194304,281018368~8388608,335544320~4194304,352321536~8388608,402653184~8388608] 2024-04-07T17:38:38.343 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [20971520~12582912,121634816~25165824,167772160~8388608,184549376~8388608,226492416~8388608,281018368~8388608,335544320~4194304,352321536~8388608,402653184~8388608] 2024-04-07T17:38:38.343 INFO:tasks.workunit.client.0.smithi100.stdout:from 16 to 18 diff [20971520~8388608,33554432~8388608,58720256~8388608,75497472~4194304,121634816~29360128,155189248~8388608,167772160~4194304,184549376~8388608,201326592~8388608,213909504~8388608,226492416~4194304,251658240~12582912,281018368~8388608,335544320~4194304,352321536~8388608,402653184~8388608] 2024-04-07T17:38:38.343 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [20971520~8388608,33554432~8388608,58720256~8388608,75497472~4194304,121634816~29360128,155189248~8388608,167772160~4194304,184549376~8388608,201326592~8388608,213909504~8388608,226492416~4194304,251658240~12582912,281018368~8388608,335544320~4194304,352321536~8388608,402653184~8388608] 2024-04-07T17:38:38.347 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [20971520~20971520,58720256~8388608,75497472~4194304,121634816~41943040,167772160~8388608,184549376~8388608,201326592~33554432,251658240~16777216,281018368~8388608,335544320~4194304,352321536~8388608,402653184~8388608] 2024-04-07T17:38:38.347 INFO:tasks.workunit.client.0.smithi100.stdout:from 16 to 19 diff [16777216~12582912,33554432~8388608,58720256~8388608,75497472~4194304,113246208~37748736,155189248~8388608,167772160~4194304,184549376~8388608,201326592~8388608,213909504~16777216,251658240~12582912,281018368~8388608,314572800~4194304,335544320~4194304,352321536~8388608,364904448~8388608,402653184~8388608] 2024-04-07T17:38:38.347 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [16777216~12582912,33554432~8388608,58720256~8388608,75497472~4194304,113246208~37748736,155189248~8388608,167772160~4194304,184549376~8388608,201326592~8388608,213909504~16777216,251658240~12582912,281018368~8388608,314572800~4194304,335544320~4194304,352321536~8388608,364904448~8388608,402653184~8388608] 2024-04-07T17:38:38.352 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [16777216~25165824,58720256~8388608,75497472~4194304,113246208~50331648,167772160~8388608,184549376~8388608,201326592~33554432,251658240~16777216,281018368~8388608,314572800~8388608,335544320~4194304,352321536~8388608,364904448~8388608,402653184~8388608] 2024-04-07T17:38:38.352 INFO:tasks.workunit.client.0.smithi100.stdout:from 17 to 18 diff [33554432~8388608,58720256~8388608,75497472~4194304,121634816~8388608,142606336~8388608,155189248~8388608,201326592~8388608,213909504~8388608,251658240~12582912] 2024-04-07T17:38:38.353 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [33554432~8388608,58720256~8388608,75497472~4194304,121634816~8388608,142606336~8388608,155189248~8388608,201326592~8388608,213909504~8388608,251658240~12582912] 2024-04-07T17:38:38.355 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [33554432~8388608,58720256~8388608,75497472~4194304,121634816~8388608,142606336~20971520,201326592~25165824,251658240~16777216] 2024-04-07T17:38:38.356 INFO:tasks.workunit.client.0.smithi100.stdout:from 17 to 19 diff [16777216~4194304,33554432~8388608,58720256~8388608,75497472~4194304,113246208~16777216,138412032~12582912,155189248~8388608,167772160~4194304,201326592~8388608,213909504~16777216,251658240~12582912,314572800~4194304,364904448~8388608] 2024-04-07T17:38:38.356 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [16777216~4194304,33554432~8388608,58720256~8388608,75497472~4194304,113246208~16777216,138412032~12582912,155189248~8388608,167772160~4194304,201326592~8388608,213909504~16777216,251658240~12582912,314572800~4194304,364904448~8388608] 2024-04-07T17:38:38.360 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [16777216~4194304,33554432~8388608,58720256~8388608,75497472~4194304,113246208~16777216,138412032~25165824,167772160~8388608,201326592~33554432,251658240~16777216,314572800~8388608,364904448~8388608] 2024-04-07T17:38:38.360 INFO:tasks.workunit.client.0.smithi100.stdout:from 18 to 19 diff [16777216~4194304,113246208~8388608,125829120~4194304,138412032~4194304,155189248~4194304,167772160~4194304,218103808~4194304,226492416~4194304,314572800~4194304,364904448~8388608] 2024-04-07T17:38:38.361 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [16777216~4194304,113246208~8388608,125829120~4194304,138412032~4194304,155189248~4194304,167772160~4194304,218103808~4194304,226492416~4194304,314572800~4194304,364904448~8388608] 2024-04-07T17:38:38.364 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [16777216~4194304,113246208~8388608,125829120~4194304,138412032~4194304,155189248~8388608,167772160~8388608,218103808~16777216,314572800~8388608,364904448~8388608] 2024-04-07T17:38:38.584 INFO:tasks.workunit.client.0.smithi100.stdout:from 0 to HEAD diff [0~29360128,33554432~16777216,54525952~96468992,155189248~109051904,268435456~150994944] 2024-04-07T17:38:38.584 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~16777216,54525952~96468992,155189248~109051904,268435456~150994944] 2024-04-07T17:38:38.638 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~419430400] 2024-04-07T17:38:38.638 INFO:tasks.workunit.client.0.smithi100.stdout:from 1 to HEAD diff [0~29360128,33554432~16777216,54525952~209715200,268435456~150994944] 2024-04-07T17:38:38.638 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~16777216,54525952~209715200,268435456~150994944] 2024-04-07T17:38:38.653 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~419430400] 2024-04-07T17:38:38.653 INFO:tasks.workunit.client.0.smithi100.stdout:from 2 to HEAD diff [0~29360128,33554432~16777216,54525952~209715200,268435456~150994944] 2024-04-07T17:38:38.653 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~16777216,54525952~209715200,268435456~150994944] 2024-04-07T17:38:38.668 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~419430400] 2024-04-07T17:38:38.668 INFO:tasks.workunit.client.0.smithi100.stdout:from 3 to HEAD diff [0~29360128,33554432~16777216,54525952~209715200,268435456~58720256,331350016~62914560,402653184~16777216] 2024-04-07T17:38:38.668 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~16777216,54525952~209715200,268435456~58720256,331350016~62914560,402653184~16777216] 2024-04-07T17:38:38.681 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~398458880,402653184~16777216] 2024-04-07T17:38:38.682 INFO:tasks.workunit.client.0.smithi100.stdout:from 4 to HEAD diff [0~29360128,33554432~16777216,54525952~155189248,213909504~50331648,268435456~58720256,331350016~62914560,402653184~16777216] 2024-04-07T17:38:38.682 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~16777216,54525952~155189248,213909504~50331648,268435456~58720256,331350016~62914560,402653184~16777216] 2024-04-07T17:38:38.695 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~398458880,402653184~16777216] 2024-04-07T17:38:38.695 INFO:tasks.workunit.client.0.smithi100.stdout:from 5 to HEAD diff [0~29360128,33554432~16777216,54525952~180355072,239075328~8388608,251658240~12582912,268435456~58720256,331350016~62914560,402653184~16777216] 2024-04-07T17:38:38.695 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~16777216,54525952~180355072,239075328~8388608,251658240~12582912,268435456~58720256,331350016~62914560,402653184~16777216] 2024-04-07T17:38:38.703 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:38:38 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:38:38] "GET /metrics HTTP/1.1" 200 36754 "" "Prometheus/2.43.0" 2024-04-07T17:38:38.709 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~398458880,402653184~16777216] 2024-04-07T17:38:38.709 INFO:tasks.workunit.client.0.smithi100.stdout:from 6 to HEAD diff [0~29360128,33554432~8388608,46137344~4194304,54525952~180355072,239075328~8388608,251658240~12582912,268435456~41943040,314572800~12582912,331350016~62914560,402653184~8388608] 2024-04-07T17:38:38.709 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~8388608,46137344~4194304,54525952~180355072,239075328~8388608,251658240~12582912,268435456~41943040,314572800~12582912,331350016~62914560,402653184~8388608] 2024-04-07T17:38:38.720 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~398458880,402653184~8388608] 2024-04-07T17:38:38.721 INFO:tasks.workunit.client.0.smithi100.stdout:from 7 to HEAD diff [0~29360128,33554432~8388608,46137344~4194304,54525952~180355072,239075328~8388608,251658240~12582912,268435456~16777216,293601280~16777216,314572800~12582912,331350016~62914560,402653184~8388608] 2024-04-07T17:38:38.721 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~8388608,46137344~4194304,54525952~180355072,239075328~8388608,251658240~12582912,268435456~16777216,293601280~16777216,314572800~12582912,331350016~62914560,402653184~8388608] 2024-04-07T17:38:38.731 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~289406976,293601280~104857600,402653184~8388608] 2024-04-07T17:38:38.731 INFO:tasks.workunit.client.0.smithi100.stdout:from 8 to HEAD diff [0~29360128,33554432~8388608,46137344~4194304,54525952~33554432,92274688~142606336,251658240~12582912,268435456~16777216,293601280~16777216,314572800~12582912,331350016~29360128,364904448~29360128,402653184~8388608] 2024-04-07T17:38:38.731 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~8388608,46137344~4194304,54525952~33554432,92274688~142606336,251658240~12582912,268435456~16777216,293601280~16777216,314572800~12582912,331350016~29360128,364904448~29360128,402653184~8388608] 2024-04-07T17:38:38.741 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~41943040,46137344~41943040,92274688~146800640,251658240~37748736,293601280~67108864,364904448~33554432,402653184~8388608] 2024-04-07T17:38:38.741 INFO:tasks.workunit.client.0.smithi100.stdout:from 9 to HEAD diff [0~29360128,33554432~8388608,46137344~4194304,54525952~33554432,100663296~134217728,251658240~12582912,268435456~16777216,293601280~16777216,314572800~12582912,331350016~8388608,343932928~16777216,364904448~29360128,402653184~8388608] 2024-04-07T17:38:38.741 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~8388608,46137344~4194304,54525952~33554432,100663296~134217728,251658240~12582912,268435456~16777216,293601280~16777216,314572800~12582912,331350016~8388608,343932928~16777216,364904448~29360128,402653184~8388608] 2024-04-07T17:38:38.750 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~41943040,46137344~41943040,100663296~138412032,251658240~37748736,293601280~46137344,343932928~16777216,364904448~33554432,402653184~8388608] 2024-04-07T17:38:38.750 INFO:tasks.workunit.client.0.smithi100.stdout:from 10 to HEAD diff [0~12582912,16777216~12582912,33554432~8388608,46137344~4194304,54525952~33554432,100663296~71303168,176160768~58720256,251658240~12582912,268435456~4194304,281018368~8388608,293601280~12582912,314572800~12582912,331350016~8388608,343932928~16777216,364904448~29360128,402653184~8388608] 2024-04-07T17:38:38.750 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~12582912,16777216~12582912,33554432~8388608,46137344~4194304,54525952~33554432,100663296~71303168,176160768~58720256,251658240~12582912,268435456~4194304,281018368~8388608,293601280~12582912,314572800~12582912,331350016~8388608,343932928~16777216,364904448~29360128,402653184~8388608] 2024-04-07T17:38:38.758 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~12582912,16777216~25165824,46137344~4194304,54525952~33554432,100663296~138412032,251658240~20971520,281018368~8388608,293601280~16777216,314572800~25165824,343932928~16777216,364904448~33554432,402653184~8388608] 2024-04-07T17:38:38.758 INFO:tasks.workunit.client.0.smithi100.stdout:from 11 to HEAD diff [0~12582912,16777216~12582912,33554432~8388608,46137344~4194304,54525952~33554432,100663296~71303168,176160768~58720256,251658240~12582912,281018368~8388608,297795584~8388608,314572800~12582912,331350016~8388608,343932928~16777216,364904448~29360128,402653184~8388608] 2024-04-07T17:38:38.758 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~12582912,16777216~12582912,33554432~8388608,46137344~4194304,54525952~33554432,100663296~71303168,176160768~58720256,251658240~12582912,281018368~8388608,297795584~8388608,314572800~12582912,331350016~8388608,343932928~16777216,364904448~29360128,402653184~8388608] 2024-04-07T17:38:38.766 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~12582912,16777216~25165824,46137344~4194304,54525952~33554432,100663296~138412032,251658240~16777216,281018368~8388608,297795584~12582912,314572800~25165824,343932928~16777216,364904448~33554432,402653184~8388608] 2024-04-07T17:38:38.766 INFO:tasks.workunit.client.0.smithi100.stdout:from 12 to HEAD diff [4194304~8388608,16777216~12582912,33554432~8388608,46137344~4194304,58720256~29360128,100663296~71303168,176160768~20971520,201326592~33554432,251658240~12582912,281018368~8388608,297795584~8388608,314572800~12582912,331350016~8388608,343932928~16777216,364904448~29360128,402653184~8388608] 2024-04-07T17:38:38.766 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [4194304~8388608,16777216~12582912,33554432~8388608,46137344~4194304,58720256~29360128,100663296~71303168,176160768~20971520,201326592~33554432,251658240~12582912,281018368~8388608,297795584~8388608,314572800~12582912,331350016~8388608,343932928~16777216,364904448~29360128,402653184~8388608] 2024-04-07T17:38:38.773 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [4194304~8388608,16777216~25165824,46137344~4194304,58720256~29360128,100663296~96468992,201326592~37748736,251658240~16777216,281018368~8388608,297795584~12582912,314572800~25165824,343932928~16777216,364904448~33554432,402653184~8388608] 2024-04-07T17:38:38.773 INFO:tasks.workunit.client.0.smithi100.stdout:from 13 to HEAD diff [4194304~8388608,16777216~12582912,33554432~8388608,46137344~4194304,58720256~29360128,100663296~50331648,155189248~16777216,180355072~8388608,201326592~33554432,251658240~12582912,281018368~8388608,297795584~8388608,314572800~12582912,331350016~8388608,343932928~16777216,364904448~29360128,402653184~8388608] 2024-04-07T17:38:38.773 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [4194304~8388608,16777216~12582912,33554432~8388608,46137344~4194304,58720256~29360128,100663296~50331648,155189248~16777216,180355072~8388608,201326592~33554432,251658240~12582912,281018368~8388608,297795584~8388608,314572800~12582912,331350016~8388608,343932928~16777216,364904448~29360128,402653184~8388608] 2024-04-07T17:38:38.779 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [4194304~8388608,16777216~25165824,46137344~4194304,58720256~29360128,100663296~75497472,180355072~12582912,201326592~37748736,251658240~16777216,281018368~8388608,297795584~12582912,314572800~25165824,343932928~16777216,364904448~33554432,402653184~8388608] 2024-04-07T17:38:38.779 INFO:tasks.workunit.client.0.smithi100.stdout:from 14 to HEAD diff [4194304~8388608,16777216~12582912,33554432~8388608,58720256~29360128,100663296~50331648,155189248~8388608,167772160~4194304,184549376~8388608,201326592~8388608,213909504~20971520,251658240~12582912,281018368~8388608,297795584~8388608,314572800~12582912,331350016~8388608,343932928~16777216,364904448~29360128,402653184~8388608] 2024-04-07T17:38:38.779 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [4194304~8388608,16777216~12582912,33554432~8388608,58720256~29360128,100663296~50331648,155189248~8388608,167772160~4194304,184549376~8388608,201326592~8388608,213909504~20971520,251658240~12582912,281018368~8388608,297795584~8388608,314572800~12582912,331350016~8388608,343932928~16777216,364904448~29360128,402653184~8388608] 2024-04-07T17:38:38.784 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [4194304~8388608,16777216~25165824,58720256~29360128,100663296~62914560,167772160~8388608,184549376~8388608,201326592~37748736,251658240~16777216,281018368~8388608,297795584~12582912,314572800~25165824,343932928~16777216,364904448~33554432,402653184~8388608] 2024-04-07T17:38:38.784 INFO:tasks.workunit.client.0.smithi100.stdout:from 15 to HEAD diff [16777216~12582912,33554432~8388608,58720256~20971520,100663296~8388608,113246208~37748736,155189248~8388608,167772160~4194304,184549376~8388608,201326592~8388608,213909504~20971520,251658240~12582912,281018368~8388608,314572800~8388608,331350016~8388608,343932928~4194304,352321536~8388608,364904448~8388608,385875968~8388608,402653184~8388608] 2024-04-07T17:38:38.784 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [16777216~12582912,33554432~8388608,58720256~20971520,100663296~8388608,113246208~37748736,155189248~8388608,167772160~4194304,184549376~8388608,201326592~8388608,213909504~20971520,251658240~12582912,281018368~8388608,314572800~8388608,331350016~8388608,343932928~4194304,352321536~8388608,364904448~8388608,385875968~8388608,402653184~8388608] 2024-04-07T17:38:38.789 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [16777216~25165824,58720256~25165824,100663296~8388608,113246208~50331648,167772160~8388608,184549376~8388608,201326592~37748736,251658240~16777216,281018368~8388608,314572800~12582912,331350016~8388608,343932928~4194304,352321536~8388608,364904448~8388608,385875968~12582912,402653184~8388608] 2024-04-07T17:38:38.789 INFO:tasks.workunit.client.0.smithi100.stdout:from 16 to HEAD diff [16777216~12582912,33554432~8388608,58720256~8388608,75497472~4194304,113246208~37748736,155189248~8388608,167772160~4194304,184549376~8388608,201326592~8388608,213909504~16777216,251658240~12582912,281018368~8388608,314572800~4194304,335544320~4194304,352321536~8388608,364904448~8388608,402653184~8388608] 2024-04-07T17:38:38.789 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [16777216~12582912,33554432~8388608,58720256~8388608,75497472~4194304,113246208~37748736,155189248~8388608,167772160~4194304,184549376~8388608,201326592~8388608,213909504~16777216,251658240~12582912,281018368~8388608,314572800~4194304,335544320~4194304,352321536~8388608,364904448~8388608,402653184~8388608] 2024-04-07T17:38:38.793 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [16777216~25165824,58720256~8388608,75497472~4194304,113246208~50331648,167772160~8388608,184549376~8388608,201326592~33554432,251658240~16777216,281018368~8388608,314572800~8388608,335544320~4194304,352321536~8388608,364904448~8388608,402653184~8388608] 2024-04-07T17:38:38.793 INFO:tasks.workunit.client.0.smithi100.stdout:from 17 to HEAD diff [16777216~4194304,33554432~8388608,58720256~8388608,75497472~4194304,113246208~16777216,138412032~12582912,155189248~8388608,167772160~4194304,201326592~8388608,213909504~16777216,251658240~12582912,314572800~4194304,364904448~8388608] 2024-04-07T17:38:38.793 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [16777216~4194304,33554432~8388608,58720256~8388608,75497472~4194304,113246208~16777216,138412032~12582912,155189248~8388608,167772160~4194304,201326592~8388608,213909504~16777216,251658240~12582912,314572800~4194304,364904448~8388608] 2024-04-07T17:38:38.796 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [16777216~4194304,33554432~8388608,58720256~8388608,75497472~4194304,113246208~16777216,138412032~25165824,167772160~8388608,201326592~33554432,251658240~16777216,314572800~8388608,364904448~8388608] 2024-04-07T17:38:39.598 INFO:tasks.workunit.client.0.smithi100.stdout:from 0 to HEAD diff [0~29360128,33554432~16777216,54525952~96468992,155189248~109051904,268435456~150994944] 2024-04-07T17:38:39.598 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~16777216,54525952~96468992,155189248~109051904,268435456~150994944] 2024-04-07T17:38:39.664 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~419430400] 2024-04-07T17:38:39.664 INFO:tasks.workunit.client.0.smithi100.stdout:from 1 to HEAD diff [0~29360128,33554432~16777216,54525952~209715200,268435456~150994944] 2024-04-07T17:38:39.664 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~16777216,54525952~209715200,268435456~150994944] 2024-04-07T17:38:39.677 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~419430400] 2024-04-07T17:38:39.677 INFO:tasks.workunit.client.0.smithi100.stdout:from 2 to HEAD diff [0~29360128,33554432~16777216,54525952~209715200,268435456~150994944] 2024-04-07T17:38:39.677 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~16777216,54525952~209715200,268435456~150994944] 2024-04-07T17:38:39.688 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~419430400] 2024-04-07T17:38:39.688 INFO:tasks.workunit.client.0.smithi100.stdout:from 3 to HEAD diff [0~29360128,33554432~16777216,54525952~209715200,268435456~58720256,331350016~62914560,402653184~16777216] 2024-04-07T17:38:39.688 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~16777216,54525952~209715200,268435456~58720256,331350016~62914560,402653184~16777216] 2024-04-07T17:38:39.700 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~398458880,402653184~16777216] 2024-04-07T17:38:39.700 INFO:tasks.workunit.client.0.smithi100.stdout:from 4 to HEAD diff [0~29360128,33554432~16777216,54525952~155189248,213909504~50331648,268435456~58720256,331350016~62914560,402653184~16777216] 2024-04-07T17:38:39.700 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~16777216,54525952~155189248,213909504~50331648,268435456~58720256,331350016~62914560,402653184~16777216] 2024-04-07T17:38:39.710 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~398458880,402653184~16777216] 2024-04-07T17:38:39.710 INFO:tasks.workunit.client.0.smithi100.stdout:from 5 to HEAD diff [0~29360128,33554432~16777216,54525952~180355072,239075328~8388608,251658240~12582912,268435456~58720256,331350016~62914560,402653184~16777216] 2024-04-07T17:38:39.710 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~16777216,54525952~180355072,239075328~8388608,251658240~12582912,268435456~58720256,331350016~62914560,402653184~16777216] 2024-04-07T17:38:39.721 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~398458880,402653184~16777216] 2024-04-07T17:38:39.721 INFO:tasks.workunit.client.0.smithi100.stdout:from 6 to HEAD diff [0~29360128,33554432~8388608,46137344~4194304,54525952~180355072,239075328~8388608,251658240~12582912,268435456~41943040,314572800~12582912,331350016~62914560,402653184~8388608] 2024-04-07T17:38:39.721 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~8388608,46137344~4194304,54525952~180355072,239075328~8388608,251658240~12582912,268435456~41943040,314572800~12582912,331350016~62914560,402653184~8388608] 2024-04-07T17:38:39.730 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~398458880,402653184~8388608] 2024-04-07T17:38:39.730 INFO:tasks.workunit.client.0.smithi100.stdout:from 7 to HEAD diff [0~29360128,33554432~8388608,46137344~4194304,54525952~180355072,239075328~8388608,251658240~12582912,268435456~16777216,293601280~16777216,314572800~12582912,331350016~62914560,402653184~8388608] 2024-04-07T17:38:39.730 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~8388608,46137344~4194304,54525952~180355072,239075328~8388608,251658240~12582912,268435456~16777216,293601280~16777216,314572800~12582912,331350016~62914560,402653184~8388608] 2024-04-07T17:38:39.738 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~289406976,293601280~104857600,402653184~8388608] 2024-04-07T17:38:39.738 INFO:tasks.workunit.client.0.smithi100.stdout:from 8 to HEAD diff [0~29360128,33554432~8388608,46137344~4194304,54525952~33554432,92274688~142606336,251658240~12582912,268435456~16777216,293601280~16777216,314572800~12582912,331350016~29360128,364904448~29360128,402653184~8388608] 2024-04-07T17:38:39.738 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~8388608,46137344~4194304,54525952~33554432,92274688~142606336,251658240~12582912,268435456~16777216,293601280~16777216,314572800~12582912,331350016~29360128,364904448~29360128,402653184~8388608] 2024-04-07T17:38:39.745 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~41943040,46137344~41943040,92274688~146800640,251658240~37748736,293601280~67108864,364904448~33554432,402653184~8388608] 2024-04-07T17:38:39.745 INFO:tasks.workunit.client.0.smithi100.stdout:from 9 to HEAD diff [0~29360128,33554432~8388608,46137344~4194304,54525952~33554432,100663296~134217728,251658240~12582912,268435456~16777216,293601280~16777216,314572800~12582912,331350016~8388608,343932928~16777216,364904448~29360128,402653184~8388608] 2024-04-07T17:38:39.745 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~8388608,46137344~4194304,54525952~33554432,100663296~134217728,251658240~12582912,268435456~16777216,293601280~16777216,314572800~12582912,331350016~8388608,343932928~16777216,364904448~29360128,402653184~8388608] 2024-04-07T17:38:39.751 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~41943040,46137344~41943040,100663296~138412032,251658240~37748736,293601280~46137344,343932928~16777216,364904448~33554432,402653184~8388608] 2024-04-07T17:38:39.751 INFO:tasks.workunit.client.0.smithi100.stdout:from 10 to HEAD diff [0~12582912,16777216~12582912,33554432~8388608,46137344~4194304,54525952~33554432,100663296~71303168,176160768~58720256,251658240~12582912,268435456~4194304,281018368~8388608,293601280~12582912,314572800~12582912,331350016~8388608,343932928~16777216,364904448~29360128,402653184~8388608] 2024-04-07T17:38:39.751 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~12582912,16777216~12582912,33554432~8388608,46137344~4194304,54525952~33554432,100663296~71303168,176160768~58720256,251658240~12582912,268435456~4194304,281018368~8388608,293601280~12582912,314572800~12582912,331350016~8388608,343932928~16777216,364904448~29360128,402653184~8388608] 2024-04-07T17:38:39.756 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~12582912,16777216~25165824,46137344~4194304,54525952~33554432,100663296~138412032,251658240~20971520,281018368~8388608,293601280~16777216,314572800~25165824,343932928~16777216,364904448~33554432,402653184~8388608] 2024-04-07T17:38:39.756 INFO:tasks.workunit.client.0.smithi100.stdout:from 11 to HEAD diff [0~12582912,16777216~12582912,33554432~8388608,46137344~4194304,54525952~33554432,100663296~71303168,176160768~58720256,251658240~12582912,281018368~8388608,297795584~8388608,314572800~12582912,331350016~8388608,343932928~16777216,364904448~29360128,402653184~8388608] 2024-04-07T17:38:39.756 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~12582912,16777216~12582912,33554432~8388608,46137344~4194304,54525952~33554432,100663296~71303168,176160768~58720256,251658240~12582912,281018368~8388608,297795584~8388608,314572800~12582912,331350016~8388608,343932928~16777216,364904448~29360128,402653184~8388608] 2024-04-07T17:38:39.761 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~12582912,16777216~25165824,46137344~4194304,54525952~33554432,100663296~138412032,251658240~16777216,281018368~8388608,297795584~12582912,314572800~25165824,343932928~16777216,364904448~33554432,402653184~8388608] 2024-04-07T17:38:39.761 INFO:tasks.workunit.client.0.smithi100.stdout:from 12 to HEAD diff [4194304~8388608,16777216~12582912,33554432~8388608,46137344~4194304,58720256~29360128,100663296~71303168,176160768~20971520,201326592~33554432,251658240~12582912,281018368~8388608,297795584~8388608,314572800~12582912,331350016~8388608,343932928~16777216,364904448~29360128,402653184~8388608] 2024-04-07T17:38:39.761 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [4194304~8388608,16777216~12582912,33554432~8388608,46137344~4194304,58720256~29360128,100663296~71303168,176160768~20971520,201326592~33554432,251658240~12582912,281018368~8388608,297795584~8388608,314572800~12582912,331350016~8388608,343932928~16777216,364904448~29360128,402653184~8388608] 2024-04-07T17:38:39.766 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [4194304~8388608,16777216~25165824,46137344~4194304,58720256~29360128,100663296~96468992,201326592~37748736,251658240~16777216,281018368~8388608,297795584~12582912,314572800~25165824,343932928~16777216,364904448~33554432,402653184~8388608] 2024-04-07T17:38:39.766 INFO:tasks.workunit.client.0.smithi100.stdout:from 13 to HEAD diff [4194304~8388608,16777216~12582912,33554432~8388608,46137344~4194304,58720256~29360128,100663296~50331648,155189248~16777216,180355072~8388608,201326592~33554432,251658240~12582912,281018368~8388608,297795584~8388608,314572800~12582912,331350016~8388608,343932928~16777216,364904448~29360128,402653184~8388608] 2024-04-07T17:38:39.766 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [4194304~8388608,16777216~12582912,33554432~8388608,46137344~4194304,58720256~29360128,100663296~50331648,155189248~16777216,180355072~8388608,201326592~33554432,251658240~12582912,281018368~8388608,297795584~8388608,314572800~12582912,331350016~8388608,343932928~16777216,364904448~29360128,402653184~8388608] 2024-04-07T17:38:39.769 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [4194304~8388608,16777216~25165824,46137344~4194304,58720256~29360128,100663296~75497472,180355072~12582912,201326592~37748736,251658240~16777216,281018368~8388608,297795584~12582912,314572800~25165824,343932928~16777216,364904448~33554432,402653184~8388608] 2024-04-07T17:38:39.770 INFO:tasks.workunit.client.0.smithi100.stdout:from 14 to HEAD diff [4194304~8388608,16777216~12582912,33554432~8388608,58720256~29360128,100663296~50331648,155189248~8388608,167772160~4194304,184549376~8388608,201326592~8388608,213909504~20971520,251658240~12582912,281018368~8388608,297795584~8388608,314572800~12582912,331350016~8388608,343932928~16777216,364904448~29360128,402653184~8388608] 2024-04-07T17:38:39.770 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [4194304~8388608,16777216~12582912,33554432~8388608,58720256~29360128,100663296~50331648,155189248~8388608,167772160~4194304,184549376~8388608,201326592~8388608,213909504~20971520,251658240~12582912,281018368~8388608,297795584~8388608,314572800~12582912,331350016~8388608,343932928~16777216,364904448~29360128,402653184~8388608] 2024-04-07T17:38:39.773 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [4194304~8388608,16777216~25165824,58720256~29360128,100663296~62914560,167772160~8388608,184549376~8388608,201326592~37748736,251658240~16777216,281018368~8388608,297795584~12582912,314572800~25165824,343932928~16777216,364904448~33554432,402653184~8388608] 2024-04-07T17:38:39.773 INFO:tasks.workunit.client.0.smithi100.stdout:from 15 to HEAD diff [16777216~12582912,33554432~8388608,58720256~20971520,100663296~8388608,113246208~37748736,155189248~8388608,167772160~4194304,184549376~8388608,201326592~8388608,213909504~20971520,251658240~12582912,281018368~8388608,314572800~8388608,331350016~8388608,343932928~4194304,352321536~8388608,364904448~8388608,385875968~8388608,402653184~8388608] 2024-04-07T17:38:39.773 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [16777216~12582912,33554432~8388608,58720256~20971520,100663296~8388608,113246208~37748736,155189248~8388608,167772160~4194304,184549376~8388608,201326592~8388608,213909504~20971520,251658240~12582912,281018368~8388608,314572800~8388608,331350016~8388608,343932928~4194304,352321536~8388608,364904448~8388608,385875968~8388608,402653184~8388608] 2024-04-07T17:38:39.775 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [16777216~25165824,58720256~25165824,100663296~8388608,113246208~50331648,167772160~8388608,184549376~8388608,201326592~37748736,251658240~16777216,281018368~8388608,314572800~12582912,331350016~8388608,343932928~4194304,352321536~8388608,364904448~8388608,385875968~12582912,402653184~8388608] 2024-04-07T17:38:39.775 INFO:tasks.workunit.client.0.smithi100.stdout:from 16 to HEAD diff [16777216~12582912,33554432~8388608,58720256~8388608,75497472~4194304,113246208~37748736,155189248~8388608,167772160~4194304,184549376~8388608,201326592~8388608,213909504~16777216,251658240~12582912,281018368~8388608,314572800~4194304,335544320~4194304,352321536~8388608,364904448~8388608,402653184~8388608] 2024-04-07T17:38:39.775 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [16777216~12582912,33554432~8388608,58720256~8388608,75497472~4194304,113246208~37748736,155189248~8388608,167772160~4194304,184549376~8388608,201326592~8388608,213909504~16777216,251658240~12582912,281018368~8388608,314572800~4194304,335544320~4194304,352321536~8388608,364904448~8388608,402653184~8388608] 2024-04-07T17:38:39.777 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [16777216~25165824,58720256~8388608,75497472~4194304,113246208~50331648,167772160~8388608,184549376~8388608,201326592~33554432,251658240~16777216,281018368~8388608,314572800~8388608,335544320~4194304,352321536~8388608,364904448~8388608,402653184~8388608] 2024-04-07T17:38:39.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:38:39 smithi144 bash[66654]: cluster 2024-04-07T17:38:38.510847+0000 mgr.y (mgr.25175) 2498 : cluster [DBG] pgmap v4153: 137 pgs: 137 active+clean; 929 MiB data, 6.0 GiB used, 709 GiB / 715 GiB avail; 291 KiB/s rd, 42 MiB/s wr, 488 op/s 2024-04-07T17:38:39.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:38:39 smithi144 bash[66654]: cluster 2024-04-07T17:38:38.559137+0000 mon.a (mon.0) 3975 : cluster [DBG] osdmap e2081: 8 total, 8 up, 8 in 2024-04-07T17:38:39.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:38:39 smithi100 bash[74942]: cluster 2024-04-07T17:38:38.510847+0000 mgr.y (mgr.25175) 2498 : cluster [DBG] pgmap v4153: 137 pgs: 137 active+clean; 929 MiB data, 6.0 GiB used, 709 GiB / 715 GiB avail; 291 KiB/s rd, 42 MiB/s wr, 488 op/s 2024-04-07T17:38:39.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:38:39 smithi100 bash[74942]: cluster 2024-04-07T17:38:38.559137+0000 mon.a (mon.0) 3975 : cluster [DBG] osdmap e2081: 8 total, 8 up, 8 in 2024-04-07T17:38:39.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:38:39 smithi100 bash[76202]: cluster 2024-04-07T17:38:38.510847+0000 mgr.y (mgr.25175) 2498 : cluster [DBG] pgmap v4153: 137 pgs: 137 active+clean; 929 MiB data, 6.0 GiB used, 709 GiB / 715 GiB avail; 291 KiB/s rd, 42 MiB/s wr, 488 op/s 2024-04-07T17:38:39.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:38:39 smithi100 bash[76202]: cluster 2024-04-07T17:38:38.559137+0000 mon.a (mon.0) 3975 : cluster [DBG] osdmap e2081: 8 total, 8 up, 8 in 2024-04-07T17:38:40.614 INFO:tasks.workunit.client.0.smithi100.stdout:from 0 to HEAD diff [0~29360128,33554432~16777216,54525952~96468992,155189248~109051904,268435456~150994944] 2024-04-07T17:38:40.615 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~16777216,54525952~96468992,155189248~109051904,268435456~150994944] 2024-04-07T17:38:40.691 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~419430400] 2024-04-07T17:38:40.691 INFO:tasks.workunit.client.0.smithi100.stdout:from 1 to HEAD diff [0~29360128,33554432~16777216,54525952~209715200,268435456~150994944] 2024-04-07T17:38:40.691 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~16777216,54525952~209715200,268435456~150994944] 2024-04-07T17:38:40.700 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~419430400] 2024-04-07T17:38:40.700 INFO:tasks.workunit.client.0.smithi100.stdout:from 2 to HEAD diff [0~29360128,33554432~16777216,54525952~209715200,268435456~150994944] 2024-04-07T17:38:40.701 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~16777216,54525952~209715200,268435456~150994944] 2024-04-07T17:38:40.708 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~419430400] 2024-04-07T17:38:40.708 INFO:tasks.workunit.client.0.smithi100.stdout:from 3 to HEAD diff [0~29360128,33554432~16777216,54525952~209715200,268435456~58720256,331350016~62914560,402653184~16777216] 2024-04-07T17:38:40.708 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~16777216,54525952~209715200,268435456~58720256,331350016~62914560,402653184~16777216] 2024-04-07T17:38:40.716 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~398458880,402653184~16777216] 2024-04-07T17:38:40.717 INFO:tasks.workunit.client.0.smithi100.stdout:from 4 to HEAD diff [0~29360128,33554432~16777216,54525952~155189248,213909504~50331648,268435456~58720256,331350016~62914560,402653184~16777216] 2024-04-07T17:38:40.717 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~16777216,54525952~155189248,213909504~50331648,268435456~58720256,331350016~62914560,402653184~16777216] 2024-04-07T17:38:40.725 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~398458880,402653184~16777216] 2024-04-07T17:38:40.726 INFO:tasks.workunit.client.0.smithi100.stdout:from 5 to HEAD diff [0~29360128,33554432~16777216,54525952~180355072,239075328~8388608,251658240~12582912,268435456~58720256,331350016~62914560,402653184~16777216] 2024-04-07T17:38:40.726 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~16777216,54525952~180355072,239075328~8388608,251658240~12582912,268435456~58720256,331350016~62914560,402653184~16777216] 2024-04-07T17:38:40.733 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~398458880,402653184~16777216] 2024-04-07T17:38:40.733 INFO:tasks.workunit.client.0.smithi100.stdout:from 6 to HEAD diff [0~29360128,33554432~8388608,46137344~4194304,54525952~180355072,239075328~8388608,251658240~12582912,268435456~41943040,314572800~12582912,331350016~62914560,402653184~8388608] 2024-04-07T17:38:40.733 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~8388608,46137344~4194304,54525952~180355072,239075328~8388608,251658240~12582912,268435456~41943040,314572800~12582912,331350016~62914560,402653184~8388608] 2024-04-07T17:38:40.740 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~398458880,402653184~8388608] 2024-04-07T17:38:40.740 INFO:tasks.workunit.client.0.smithi100.stdout:from 7 to HEAD diff [0~29360128,33554432~8388608,46137344~4194304,54525952~180355072,239075328~8388608,251658240~12582912,268435456~16777216,293601280~16777216,314572800~12582912,331350016~62914560,402653184~8388608] 2024-04-07T17:38:40.740 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~8388608,46137344~4194304,54525952~180355072,239075328~8388608,251658240~12582912,268435456~16777216,293601280~16777216,314572800~12582912,331350016~62914560,402653184~8388608] 2024-04-07T17:38:40.746 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~289406976,293601280~104857600,402653184~8388608] 2024-04-07T17:38:40.746 INFO:tasks.workunit.client.0.smithi100.stdout:from 8 to HEAD diff [0~29360128,33554432~8388608,46137344~4194304,54525952~33554432,92274688~142606336,251658240~12582912,268435456~16777216,293601280~16777216,314572800~12582912,331350016~29360128,364904448~29360128,402653184~8388608] 2024-04-07T17:38:40.746 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~8388608,46137344~4194304,54525952~33554432,92274688~142606336,251658240~12582912,268435456~16777216,293601280~16777216,314572800~12582912,331350016~29360128,364904448~29360128,402653184~8388608] 2024-04-07T17:38:40.752 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~41943040,46137344~41943040,92274688~146800640,251658240~37748736,293601280~67108864,364904448~33554432,402653184~8388608] 2024-04-07T17:38:40.752 INFO:tasks.workunit.client.0.smithi100.stdout:from 9 to HEAD diff [0~29360128,33554432~8388608,46137344~4194304,54525952~33554432,100663296~134217728,251658240~12582912,268435456~16777216,293601280~16777216,314572800~12582912,331350016~8388608,343932928~16777216,364904448~29360128,402653184~8388608] 2024-04-07T17:38:40.752 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~8388608,46137344~4194304,54525952~33554432,100663296~134217728,251658240~12582912,268435456~16777216,293601280~16777216,314572800~12582912,331350016~8388608,343932928~16777216,364904448~29360128,402653184~8388608] 2024-04-07T17:38:40.757 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~41943040,46137344~41943040,100663296~138412032,251658240~37748736,293601280~46137344,343932928~16777216,364904448~33554432,402653184~8388608] 2024-04-07T17:38:40.757 INFO:tasks.workunit.client.0.smithi100.stdout:from 10 to HEAD diff [0~12582912,16777216~12582912,33554432~8388608,46137344~4194304,54525952~33554432,100663296~71303168,176160768~58720256,251658240~12582912,268435456~4194304,281018368~8388608,293601280~12582912,314572800~12582912,331350016~8388608,343932928~16777216,364904448~29360128,402653184~8388608] 2024-04-07T17:38:40.757 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~12582912,16777216~12582912,33554432~8388608,46137344~4194304,54525952~33554432,100663296~71303168,176160768~58720256,251658240~12582912,268435456~4194304,281018368~8388608,293601280~12582912,314572800~12582912,331350016~8388608,343932928~16777216,364904448~29360128,402653184~8388608] 2024-04-07T17:38:40.760 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~12582912,16777216~25165824,46137344~4194304,54525952~33554432,100663296~138412032,251658240~20971520,281018368~8388608,293601280~16777216,314572800~25165824,343932928~16777216,364904448~33554432,402653184~8388608] 2024-04-07T17:38:40.760 INFO:tasks.workunit.client.0.smithi100.stdout:from 11 to HEAD diff [0~12582912,16777216~12582912,33554432~8388608,46137344~4194304,54525952~33554432,100663296~71303168,176160768~58720256,251658240~12582912,281018368~8388608,297795584~8388608,314572800~12582912,331350016~8388608,343932928~16777216,364904448~29360128,402653184~8388608] 2024-04-07T17:38:40.760 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~12582912,16777216~12582912,33554432~8388608,46137344~4194304,54525952~33554432,100663296~71303168,176160768~58720256,251658240~12582912,281018368~8388608,297795584~8388608,314572800~12582912,331350016~8388608,343932928~16777216,364904448~29360128,402653184~8388608] 2024-04-07T17:38:40.764 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~12582912,16777216~25165824,46137344~4194304,54525952~33554432,100663296~138412032,251658240~16777216,281018368~8388608,297795584~12582912,314572800~25165824,343932928~16777216,364904448~33554432,402653184~8388608] 2024-04-07T17:38:40.764 INFO:tasks.workunit.client.0.smithi100.stdout:from 12 to HEAD diff [4194304~8388608,16777216~12582912,33554432~8388608,46137344~4194304,58720256~29360128,100663296~71303168,176160768~20971520,201326592~33554432,251658240~12582912,281018368~8388608,297795584~8388608,314572800~12582912,331350016~8388608,343932928~16777216,364904448~29360128,402653184~8388608] 2024-04-07T17:38:40.764 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [4194304~8388608,16777216~12582912,33554432~8388608,46137344~4194304,58720256~29360128,100663296~71303168,176160768~20971520,201326592~33554432,251658240~12582912,281018368~8388608,297795584~8388608,314572800~12582912,331350016~8388608,343932928~16777216,364904448~29360128,402653184~8388608] 2024-04-07T17:38:40.767 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [4194304~8388608,16777216~25165824,46137344~4194304,58720256~29360128,100663296~96468992,201326592~37748736,251658240~16777216,281018368~8388608,297795584~12582912,314572800~25165824,343932928~16777216,364904448~33554432,402653184~8388608] 2024-04-07T17:38:40.767 INFO:tasks.workunit.client.0.smithi100.stdout:from 13 to HEAD diff [4194304~8388608,16777216~12582912,33554432~8388608,46137344~4194304,58720256~29360128,100663296~50331648,155189248~16777216,180355072~8388608,201326592~33554432,251658240~12582912,281018368~8388608,297795584~8388608,314572800~12582912,331350016~8388608,343932928~16777216,364904448~29360128,402653184~8388608] 2024-04-07T17:38:40.767 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [4194304~8388608,16777216~12582912,33554432~8388608,46137344~4194304,58720256~29360128,100663296~50331648,155189248~16777216,180355072~8388608,201326592~33554432,251658240~12582912,281018368~8388608,297795584~8388608,314572800~12582912,331350016~8388608,343932928~16777216,364904448~29360128,402653184~8388608] 2024-04-07T17:38:40.769 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [4194304~8388608,16777216~25165824,46137344~4194304,58720256~29360128,100663296~75497472,180355072~12582912,201326592~37748736,251658240~16777216,281018368~8388608,297795584~12582912,314572800~25165824,343932928~16777216,364904448~33554432,402653184~8388608] 2024-04-07T17:38:40.769 INFO:tasks.workunit.client.0.smithi100.stdout:from 14 to HEAD diff [4194304~8388608,16777216~12582912,33554432~8388608,58720256~29360128,100663296~50331648,155189248~8388608,167772160~4194304,184549376~8388608,201326592~8388608,213909504~20971520,251658240~12582912,281018368~8388608,297795584~8388608,314572800~12582912,331350016~8388608,343932928~16777216,364904448~29360128,402653184~8388608] 2024-04-07T17:38:40.770 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [4194304~8388608,16777216~12582912,33554432~8388608,58720256~29360128,100663296~50331648,155189248~8388608,167772160~4194304,184549376~8388608,201326592~8388608,213909504~20971520,251658240~12582912,281018368~8388608,297795584~8388608,314572800~12582912,331350016~8388608,343932928~16777216,364904448~29360128,402653184~8388608] 2024-04-07T17:38:40.772 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [4194304~8388608,16777216~25165824,58720256~29360128,100663296~62914560,167772160~8388608,184549376~8388608,201326592~37748736,251658240~16777216,281018368~8388608,297795584~12582912,314572800~25165824,343932928~16777216,364904448~33554432,402653184~8388608] 2024-04-07T17:38:40.772 INFO:tasks.workunit.client.0.smithi100.stdout:from 15 to HEAD diff [16777216~12582912,33554432~8388608,58720256~20971520,100663296~8388608,113246208~37748736,155189248~8388608,167772160~4194304,184549376~8388608,201326592~8388608,213909504~20971520,251658240~12582912,281018368~8388608,314572800~8388608,331350016~8388608,343932928~4194304,352321536~8388608,364904448~8388608,385875968~8388608,402653184~8388608] 2024-04-07T17:38:40.772 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [16777216~12582912,33554432~8388608,58720256~20971520,100663296~8388608,113246208~37748736,155189248~8388608,167772160~4194304,184549376~8388608,201326592~8388608,213909504~20971520,251658240~12582912,281018368~8388608,314572800~8388608,331350016~8388608,343932928~4194304,352321536~8388608,364904448~8388608,385875968~8388608,402653184~8388608] 2024-04-07T17:38:40.773 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [16777216~25165824,58720256~25165824,100663296~8388608,113246208~50331648,167772160~8388608,184549376~8388608,201326592~37748736,251658240~16777216,281018368~8388608,314572800~12582912,331350016~8388608,343932928~4194304,352321536~8388608,364904448~8388608,385875968~12582912,402653184~8388608] 2024-04-07T17:38:40.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:38:40 smithi144 bash[66654]: cluster 2024-04-07T17:38:39.563035+0000 mon.a (mon.0) 3976 : cluster [DBG] osdmap e2082: 8 total, 8 up, 8 in 2024-04-07T17:38:40.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:38:40 smithi100 bash[74942]: cluster 2024-04-07T17:38:39.563035+0000 mon.a (mon.0) 3976 : cluster [DBG] osdmap e2082: 8 total, 8 up, 8 in 2024-04-07T17:38:40.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:38:40 smithi100 bash[76202]: cluster 2024-04-07T17:38:39.563035+0000 mon.a (mon.0) 3976 : cluster [DBG] osdmap e2082: 8 total, 8 up, 8 in 2024-04-07T17:38:41.624 INFO:tasks.workunit.client.0.smithi100.stdout:from 0 to HEAD diff [0~29360128,33554432~16777216,54525952~96468992,155189248~109051904,268435456~150994944] 2024-04-07T17:38:41.624 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~16777216,54525952~96468992,155189248~109051904,268435456~150994944] 2024-04-07T17:38:41.694 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~419430400] 2024-04-07T17:38:41.694 INFO:tasks.workunit.client.0.smithi100.stdout:from 1 to HEAD diff [0~29360128,33554432~16777216,54525952~209715200,268435456~150994944] 2024-04-07T17:38:41.694 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~16777216,54525952~209715200,268435456~150994944] 2024-04-07T17:38:41.702 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~419430400] 2024-04-07T17:38:41.702 INFO:tasks.workunit.client.0.smithi100.stdout:from 2 to HEAD diff [0~29360128,33554432~16777216,54525952~209715200,268435456~150994944] 2024-04-07T17:38:41.702 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~16777216,54525952~209715200,268435456~150994944] 2024-04-07T17:38:41.709 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~419430400] 2024-04-07T17:38:41.709 INFO:tasks.workunit.client.0.smithi100.stdout:from 3 to HEAD diff [0~29360128,33554432~16777216,54525952~209715200,268435456~58720256,331350016~62914560,402653184~16777216] 2024-04-07T17:38:41.709 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~16777216,54525952~209715200,268435456~58720256,331350016~62914560,402653184~16777216] 2024-04-07T17:38:41.715 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~398458880,402653184~16777216] 2024-04-07T17:38:41.715 INFO:tasks.workunit.client.0.smithi100.stdout:from 4 to HEAD diff [0~29360128,33554432~16777216,54525952~155189248,213909504~50331648,268435456~58720256,331350016~62914560,402653184~16777216] 2024-04-07T17:38:41.715 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~16777216,54525952~155189248,213909504~50331648,268435456~58720256,331350016~62914560,402653184~16777216] 2024-04-07T17:38:41.721 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~398458880,402653184~16777216] 2024-04-07T17:38:41.721 INFO:tasks.workunit.client.0.smithi100.stdout:from 5 to HEAD diff [0~29360128,33554432~16777216,54525952~180355072,239075328~8388608,251658240~12582912,268435456~58720256,331350016~62914560,402653184~16777216] 2024-04-07T17:38:41.721 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~16777216,54525952~180355072,239075328~8388608,251658240~12582912,268435456~58720256,331350016~62914560,402653184~16777216] 2024-04-07T17:38:41.727 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~398458880,402653184~16777216] 2024-04-07T17:38:41.727 INFO:tasks.workunit.client.0.smithi100.stdout:from 6 to HEAD diff [0~29360128,33554432~8388608,46137344~4194304,54525952~180355072,239075328~8388608,251658240~12582912,268435456~41943040,314572800~12582912,331350016~62914560,402653184~8388608] 2024-04-07T17:38:41.727 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~8388608,46137344~4194304,54525952~180355072,239075328~8388608,251658240~12582912,268435456~41943040,314572800~12582912,331350016~62914560,402653184~8388608] 2024-04-07T17:38:41.732 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~398458880,402653184~8388608] 2024-04-07T17:38:41.732 INFO:tasks.workunit.client.0.smithi100.stdout:from 7 to HEAD diff [0~29360128,33554432~8388608,46137344~4194304,54525952~180355072,239075328~8388608,251658240~12582912,268435456~16777216,293601280~16777216,314572800~12582912,331350016~62914560,402653184~8388608] 2024-04-07T17:38:41.732 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~8388608,46137344~4194304,54525952~180355072,239075328~8388608,251658240~12582912,268435456~16777216,293601280~16777216,314572800~12582912,331350016~62914560,402653184~8388608] 2024-04-07T17:38:41.737 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~289406976,293601280~104857600,402653184~8388608] 2024-04-07T17:38:41.737 INFO:tasks.workunit.client.0.smithi100.stdout:from 8 to HEAD diff [0~29360128,33554432~8388608,46137344~4194304,54525952~33554432,92274688~142606336,251658240~12582912,268435456~16777216,293601280~16777216,314572800~12582912,331350016~29360128,364904448~29360128,402653184~8388608] 2024-04-07T17:38:41.737 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~8388608,46137344~4194304,54525952~33554432,92274688~142606336,251658240~12582912,268435456~16777216,293601280~16777216,314572800~12582912,331350016~29360128,364904448~29360128,402653184~8388608] 2024-04-07T17:38:41.741 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~41943040,46137344~41943040,92274688~146800640,251658240~37748736,293601280~67108864,364904448~33554432,402653184~8388608] 2024-04-07T17:38:41.741 INFO:tasks.workunit.client.0.smithi100.stdout:from 9 to HEAD diff [0~29360128,33554432~8388608,46137344~4194304,54525952~33554432,100663296~134217728,251658240~12582912,268435456~16777216,293601280~16777216,314572800~12582912,331350016~8388608,343932928~16777216,364904448~29360128,402653184~8388608] 2024-04-07T17:38:41.741 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~8388608,46137344~4194304,54525952~33554432,100663296~134217728,251658240~12582912,268435456~16777216,293601280~16777216,314572800~12582912,331350016~8388608,343932928~16777216,364904448~29360128,402653184~8388608] 2024-04-07T17:38:41.744 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~41943040,46137344~41943040,100663296~138412032,251658240~37748736,293601280~46137344,343932928~16777216,364904448~33554432,402653184~8388608] 2024-04-07T17:38:41.744 INFO:tasks.workunit.client.0.smithi100.stdout:from 10 to HEAD diff [0~12582912,16777216~12582912,33554432~8388608,46137344~4194304,54525952~33554432,100663296~71303168,176160768~58720256,251658240~12582912,268435456~4194304,281018368~8388608,293601280~12582912,314572800~12582912,331350016~8388608,343932928~16777216,364904448~29360128,402653184~8388608] 2024-04-07T17:38:41.744 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~12582912,16777216~12582912,33554432~8388608,46137344~4194304,54525952~33554432,100663296~71303168,176160768~58720256,251658240~12582912,268435456~4194304,281018368~8388608,293601280~12582912,314572800~12582912,331350016~8388608,343932928~16777216,364904448~29360128,402653184~8388608] 2024-04-07T17:38:41.747 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~12582912,16777216~25165824,46137344~4194304,54525952~33554432,100663296~138412032,251658240~20971520,281018368~8388608,293601280~16777216,314572800~25165824,343932928~16777216,364904448~33554432,402653184~8388608] 2024-04-07T17:38:41.747 INFO:tasks.workunit.client.0.smithi100.stdout:from 11 to HEAD diff [0~12582912,16777216~12582912,33554432~8388608,46137344~4194304,54525952~33554432,100663296~71303168,176160768~58720256,251658240~12582912,281018368~8388608,297795584~8388608,314572800~12582912,331350016~8388608,343932928~16777216,364904448~29360128,402653184~8388608] 2024-04-07T17:38:41.747 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~12582912,16777216~12582912,33554432~8388608,46137344~4194304,54525952~33554432,100663296~71303168,176160768~58720256,251658240~12582912,281018368~8388608,297795584~8388608,314572800~12582912,331350016~8388608,343932928~16777216,364904448~29360128,402653184~8388608] 2024-04-07T17:38:41.749 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~12582912,16777216~25165824,46137344~4194304,54525952~33554432,100663296~138412032,251658240~16777216,281018368~8388608,297795584~12582912,314572800~25165824,343932928~16777216,364904448~33554432,402653184~8388608] 2024-04-07T17:38:41.749 INFO:tasks.workunit.client.0.smithi100.stdout:from 12 to HEAD diff [4194304~8388608,16777216~12582912,33554432~8388608,46137344~4194304,58720256~29360128,100663296~71303168,176160768~20971520,201326592~33554432,251658240~12582912,281018368~8388608,297795584~8388608,314572800~12582912,331350016~8388608,343932928~16777216,364904448~29360128,402653184~8388608] 2024-04-07T17:38:41.749 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [4194304~8388608,16777216~12582912,33554432~8388608,46137344~4194304,58720256~29360128,100663296~71303168,176160768~20971520,201326592~33554432,251658240~12582912,281018368~8388608,297795584~8388608,314572800~12582912,331350016~8388608,343932928~16777216,364904448~29360128,402653184~8388608] 2024-04-07T17:38:41.751 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [4194304~8388608,16777216~25165824,46137344~4194304,58720256~29360128,100663296~96468992,201326592~37748736,251658240~16777216,281018368~8388608,297795584~12582912,314572800~25165824,343932928~16777216,364904448~33554432,402653184~8388608] 2024-04-07T17:38:41.751 INFO:tasks.workunit.client.0.smithi100.stdout:from 13 to HEAD diff [4194304~8388608,16777216~12582912,33554432~8388608,46137344~4194304,58720256~29360128,100663296~50331648,155189248~16777216,180355072~8388608,201326592~33554432,251658240~12582912,281018368~8388608,297795584~8388608,314572800~12582912,331350016~8388608,343932928~16777216,364904448~29360128,402653184~8388608] 2024-04-07T17:38:41.751 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [4194304~8388608,16777216~12582912,33554432~8388608,46137344~4194304,58720256~29360128,100663296~50331648,155189248~16777216,180355072~8388608,201326592~33554432,251658240~12582912,281018368~8388608,297795584~8388608,314572800~12582912,331350016~8388608,343932928~16777216,364904448~29360128,402653184~8388608] 2024-04-07T17:38:41.753 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [4194304~8388608,16777216~25165824,46137344~4194304,58720256~29360128,100663296~75497472,180355072~12582912,201326592~37748736,251658240~16777216,281018368~8388608,297795584~12582912,314572800~25165824,343932928~16777216,364904448~33554432,402653184~8388608] 2024-04-07T17:38:41.753 INFO:tasks.workunit.client.0.smithi100.stdout:from 14 to HEAD diff [4194304~8388608,16777216~12582912,33554432~8388608,58720256~29360128,100663296~50331648,155189248~8388608,167772160~4194304,184549376~8388608,201326592~8388608,213909504~20971520,251658240~12582912,281018368~8388608,297795584~8388608,314572800~12582912,331350016~8388608,343932928~16777216,364904448~29360128,402653184~8388608] 2024-04-07T17:38:41.753 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [4194304~8388608,16777216~12582912,33554432~8388608,58720256~29360128,100663296~50331648,155189248~8388608,167772160~4194304,184549376~8388608,201326592~8388608,213909504~20971520,251658240~12582912,281018368~8388608,297795584~8388608,314572800~12582912,331350016~8388608,343932928~16777216,364904448~29360128,402653184~8388608] 2024-04-07T17:38:41.754 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [4194304~8388608,16777216~25165824,58720256~29360128,100663296~62914560,167772160~8388608,184549376~8388608,201326592~37748736,251658240~16777216,281018368~8388608,297795584~12582912,314572800~25165824,343932928~16777216,364904448~33554432,402653184~8388608] 2024-04-07T17:38:41.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:38:41 smithi144 bash[66654]: cluster 2024-04-07T17:38:40.511846+0000 mgr.y (mgr.25175) 2499 : cluster [DBG] pgmap v4156: 137 pgs: 137 active+clean; 941 MiB data, 6.1 GiB used, 709 GiB / 715 GiB avail; 437 KiB/s rd, 23 MiB/s wr, 795 op/s 2024-04-07T17:38:41.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:38:41 smithi144 bash[66654]: cluster 2024-04-07T17:38:40.590927+0000 mon.a (mon.0) 3977 : cluster [DBG] osdmap e2083: 8 total, 8 up, 8 in 2024-04-07T17:38:41.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:38:41 smithi100 bash[74942]: cluster 2024-04-07T17:38:40.511846+0000 mgr.y (mgr.25175) 2499 : cluster [DBG] pgmap v4156: 137 pgs: 137 active+clean; 941 MiB data, 6.1 GiB used, 709 GiB / 715 GiB avail; 437 KiB/s rd, 23 MiB/s wr, 795 op/s 2024-04-07T17:38:41.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:38:41 smithi100 bash[74942]: cluster 2024-04-07T17:38:40.590927+0000 mon.a (mon.0) 3977 : cluster [DBG] osdmap e2083: 8 total, 8 up, 8 in 2024-04-07T17:38:41.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:38:41 smithi100 bash[76202]: cluster 2024-04-07T17:38:40.511846+0000 mgr.y (mgr.25175) 2499 : cluster [DBG] pgmap v4156: 137 pgs: 137 active+clean; 941 MiB data, 6.1 GiB used, 709 GiB / 715 GiB avail; 437 KiB/s rd, 23 MiB/s wr, 795 op/s 2024-04-07T17:38:41.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:38:41 smithi100 bash[76202]: cluster 2024-04-07T17:38:40.590927+0000 mon.a (mon.0) 3977 : cluster [DBG] osdmap e2083: 8 total, 8 up, 8 in 2024-04-07T17:38:42.615 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:38:42 smithi144 bash[95233]: ts=2024-04-07T17:38:42.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:38:42.636 INFO:tasks.workunit.client.0.smithi100.stdout:from 0 to HEAD diff [0~29360128,33554432~16777216,54525952~96468992,155189248~109051904,268435456~150994944] 2024-04-07T17:38:42.636 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~16777216,54525952~96468992,155189248~109051904,268435456~150994944] 2024-04-07T17:38:42.701 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~419430400] 2024-04-07T17:38:42.701 INFO:tasks.workunit.client.0.smithi100.stdout:from 1 to HEAD diff [0~29360128,33554432~16777216,54525952~209715200,268435456~150994944] 2024-04-07T17:38:42.701 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~16777216,54525952~209715200,268435456~150994944] 2024-04-07T17:38:42.709 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~419430400] 2024-04-07T17:38:42.709 INFO:tasks.workunit.client.0.smithi100.stdout:from 2 to HEAD diff [0~29360128,33554432~16777216,54525952~209715200,268435456~150994944] 2024-04-07T17:38:42.709 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~16777216,54525952~209715200,268435456~150994944] 2024-04-07T17:38:42.717 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~419430400] 2024-04-07T17:38:42.717 INFO:tasks.workunit.client.0.smithi100.stdout:from 3 to HEAD diff [0~29360128,33554432~16777216,54525952~209715200,268435456~58720256,331350016~62914560,402653184~16777216] 2024-04-07T17:38:42.718 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~16777216,54525952~209715200,268435456~58720256,331350016~62914560,402653184~16777216] 2024-04-07T17:38:42.722 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~398458880,402653184~16777216] 2024-04-07T17:38:42.722 INFO:tasks.workunit.client.0.smithi100.stdout:from 4 to HEAD diff [0~29360128,33554432~16777216,54525952~155189248,213909504~50331648,268435456~58720256,331350016~62914560,402653184~16777216] 2024-04-07T17:38:42.722 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~16777216,54525952~155189248,213909504~50331648,268435456~58720256,331350016~62914560,402653184~16777216] 2024-04-07T17:38:42.728 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~398458880,402653184~16777216] 2024-04-07T17:38:42.728 INFO:tasks.workunit.client.0.smithi100.stdout:from 5 to HEAD diff [0~29360128,33554432~16777216,54525952~180355072,239075328~8388608,251658240~12582912,268435456~58720256,331350016~62914560,402653184~16777216] 2024-04-07T17:38:42.728 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~16777216,54525952~180355072,239075328~8388608,251658240~12582912,268435456~58720256,331350016~62914560,402653184~16777216] 2024-04-07T17:38:42.734 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~398458880,402653184~16777216] 2024-04-07T17:38:42.734 INFO:tasks.workunit.client.0.smithi100.stdout:from 6 to HEAD diff [0~29360128,33554432~8388608,46137344~4194304,54525952~180355072,239075328~8388608,251658240~12582912,268435456~41943040,314572800~12582912,331350016~62914560,402653184~8388608] 2024-04-07T17:38:42.734 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~8388608,46137344~4194304,54525952~180355072,239075328~8388608,251658240~12582912,268435456~41943040,314572800~12582912,331350016~62914560,402653184~8388608] 2024-04-07T17:38:42.738 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~398458880,402653184~8388608] 2024-04-07T17:38:42.739 INFO:tasks.workunit.client.0.smithi100.stdout:from 7 to HEAD diff [0~29360128,33554432~8388608,46137344~4194304,54525952~180355072,239075328~8388608,251658240~12582912,268435456~16777216,293601280~16777216,314572800~12582912,331350016~62914560,402653184~8388608] 2024-04-07T17:38:42.739 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~8388608,46137344~4194304,54525952~180355072,239075328~8388608,251658240~12582912,268435456~16777216,293601280~16777216,314572800~12582912,331350016~62914560,402653184~8388608] 2024-04-07T17:38:42.743 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~289406976,293601280~104857600,402653184~8388608] 2024-04-07T17:38:42.743 INFO:tasks.workunit.client.0.smithi100.stdout:from 8 to HEAD diff [0~29360128,33554432~8388608,46137344~4194304,54525952~33554432,92274688~142606336,251658240~12582912,268435456~16777216,293601280~16777216,314572800~12582912,331350016~29360128,364904448~29360128,402653184~8388608] 2024-04-07T17:38:42.743 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~8388608,46137344~4194304,54525952~33554432,92274688~142606336,251658240~12582912,268435456~16777216,293601280~16777216,314572800~12582912,331350016~29360128,364904448~29360128,402653184~8388608] 2024-04-07T17:38:42.748 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~41943040,46137344~41943040,92274688~146800640,251658240~37748736,293601280~67108864,364904448~33554432,402653184~8388608] 2024-04-07T17:38:42.748 INFO:tasks.workunit.client.0.smithi100.stdout:from 9 to HEAD diff [0~29360128,33554432~8388608,46137344~4194304,54525952~33554432,100663296~134217728,251658240~12582912,268435456~16777216,293601280~16777216,314572800~12582912,331350016~8388608,343932928~16777216,364904448~29360128,402653184~8388608] 2024-04-07T17:38:42.748 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~8388608,46137344~4194304,54525952~33554432,100663296~134217728,251658240~12582912,268435456~16777216,293601280~16777216,314572800~12582912,331350016~8388608,343932928~16777216,364904448~29360128,402653184~8388608] 2024-04-07T17:38:42.751 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~41943040,46137344~41943040,100663296~138412032,251658240~37748736,293601280~46137344,343932928~16777216,364904448~33554432,402653184~8388608] 2024-04-07T17:38:42.751 INFO:tasks.workunit.client.0.smithi100.stdout:from 10 to HEAD diff [0~12582912,16777216~12582912,33554432~8388608,46137344~4194304,54525952~33554432,100663296~71303168,176160768~58720256,251658240~12582912,268435456~4194304,281018368~8388608,293601280~12582912,314572800~12582912,331350016~8388608,343932928~16777216,364904448~29360128,402653184~8388608] 2024-04-07T17:38:42.751 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~12582912,16777216~12582912,33554432~8388608,46137344~4194304,54525952~33554432,100663296~71303168,176160768~58720256,251658240~12582912,268435456~4194304,281018368~8388608,293601280~12582912,314572800~12582912,331350016~8388608,343932928~16777216,364904448~29360128,402653184~8388608] 2024-04-07T17:38:42.754 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~12582912,16777216~25165824,46137344~4194304,54525952~33554432,100663296~138412032,251658240~20971520,281018368~8388608,293601280~16777216,314572800~25165824,343932928~16777216,364904448~33554432,402653184~8388608] 2024-04-07T17:38:42.754 INFO:tasks.workunit.client.0.smithi100.stdout:from 11 to HEAD diff [0~12582912,16777216~12582912,33554432~8388608,46137344~4194304,54525952~33554432,100663296~71303168,176160768~58720256,251658240~12582912,281018368~8388608,297795584~8388608,314572800~12582912,331350016~8388608,343932928~16777216,364904448~29360128,402653184~8388608] 2024-04-07T17:38:42.754 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~12582912,16777216~12582912,33554432~8388608,46137344~4194304,54525952~33554432,100663296~71303168,176160768~58720256,251658240~12582912,281018368~8388608,297795584~8388608,314572800~12582912,331350016~8388608,343932928~16777216,364904448~29360128,402653184~8388608] 2024-04-07T17:38:42.756 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~12582912,16777216~25165824,46137344~4194304,54525952~33554432,100663296~138412032,251658240~16777216,281018368~8388608,297795584~12582912,314572800~25165824,343932928~16777216,364904448~33554432,402653184~8388608] 2024-04-07T17:38:42.757 INFO:tasks.workunit.client.0.smithi100.stdout:from 12 to HEAD diff [4194304~8388608,16777216~12582912,33554432~8388608,46137344~4194304,58720256~29360128,100663296~71303168,176160768~20971520,201326592~33554432,251658240~12582912,281018368~8388608,297795584~8388608,314572800~12582912,331350016~8388608,343932928~16777216,364904448~29360128,402653184~8388608] 2024-04-07T17:38:42.757 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [4194304~8388608,16777216~12582912,33554432~8388608,46137344~4194304,58720256~29360128,100663296~71303168,176160768~20971520,201326592~33554432,251658240~12582912,281018368~8388608,297795584~8388608,314572800~12582912,331350016~8388608,343932928~16777216,364904448~29360128,402653184~8388608] 2024-04-07T17:38:42.758 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [4194304~8388608,16777216~25165824,46137344~4194304,58720256~29360128,100663296~96468992,201326592~37748736,251658240~16777216,281018368~8388608,297795584~12582912,314572800~25165824,343932928~16777216,364904448~33554432,402653184~8388608] 2024-04-07T17:38:42.758 INFO:tasks.workunit.client.0.smithi100.stdout:from 13 to HEAD diff [4194304~8388608,16777216~12582912,33554432~8388608,46137344~4194304,58720256~29360128,100663296~50331648,155189248~16777216,180355072~8388608,201326592~33554432,251658240~12582912,281018368~8388608,297795584~8388608,314572800~12582912,331350016~8388608,343932928~16777216,364904448~29360128,402653184~8388608] 2024-04-07T17:38:42.758 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [4194304~8388608,16777216~12582912,33554432~8388608,46137344~4194304,58720256~29360128,100663296~50331648,155189248~16777216,180355072~8388608,201326592~33554432,251658240~12582912,281018368~8388608,297795584~8388608,314572800~12582912,331350016~8388608,343932928~16777216,364904448~29360128,402653184~8388608] 2024-04-07T17:38:42.760 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [4194304~8388608,16777216~25165824,46137344~4194304,58720256~29360128,100663296~75497472,180355072~12582912,201326592~37748736,251658240~16777216,281018368~8388608,297795584~12582912,314572800~25165824,343932928~16777216,364904448~33554432,402653184~8388608] 2024-04-07T17:38:42.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:38:42 smithi100 bash[74942]: cluster 2024-04-07T17:38:41.588973+0000 mon.a (mon.0) 3978 : cluster [DBG] osdmap e2084: 8 total, 8 up, 8 in 2024-04-07T17:38:42.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:38:42 smithi100 bash[76202]: cluster 2024-04-07T17:38:41.588973+0000 mon.a (mon.0) 3978 : cluster [DBG] osdmap e2084: 8 total, 8 up, 8 in 2024-04-07T17:38:43.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:38:42 smithi144 bash[66654]: cluster 2024-04-07T17:38:41.588973+0000 mon.a (mon.0) 3978 : cluster [DBG] osdmap e2084: 8 total, 8 up, 8 in 2024-04-07T17:38:43.670 INFO:tasks.workunit.client.0.smithi100.stdout:from 0 to HEAD diff [0~29360128,33554432~16777216,54525952~96468992,155189248~109051904,268435456~150994944] 2024-04-07T17:38:43.670 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~16777216,54525952~96468992,155189248~109051904,268435456~150994944] 2024-04-07T17:38:43.711 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~419430400] 2024-04-07T17:38:43.711 INFO:tasks.workunit.client.0.smithi100.stdout:from 1 to HEAD diff [0~29360128,33554432~16777216,54525952~209715200,268435456~150994944] 2024-04-07T17:38:43.711 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~16777216,54525952~209715200,268435456~150994944] 2024-04-07T17:38:43.727 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~419430400] 2024-04-07T17:38:43.727 INFO:tasks.workunit.client.0.smithi100.stdout:from 2 to HEAD diff [0~29360128,33554432~16777216,54525952~209715200,268435456~150994944] 2024-04-07T17:38:43.727 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~16777216,54525952~209715200,268435456~150994944] 2024-04-07T17:38:43.746 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~419430400] 2024-04-07T17:38:43.746 INFO:tasks.workunit.client.0.smithi100.stdout:from 3 to HEAD diff [0~29360128,33554432~16777216,54525952~209715200,268435456~58720256,331350016~62914560,402653184~16777216] 2024-04-07T17:38:43.746 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~16777216,54525952~209715200,268435456~58720256,331350016~62914560,402653184~16777216] 2024-04-07T17:38:43.755 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~398458880,402653184~16777216] 2024-04-07T17:38:43.755 INFO:tasks.workunit.client.0.smithi100.stdout:from 4 to HEAD diff [0~29360128,33554432~16777216,54525952~155189248,213909504~50331648,268435456~58720256,331350016~62914560,402653184~16777216] 2024-04-07T17:38:43.755 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~16777216,54525952~155189248,213909504~50331648,268435456~58720256,331350016~62914560,402653184~16777216] 2024-04-07T17:38:43.760 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~398458880,402653184~16777216] 2024-04-07T17:38:43.761 INFO:tasks.workunit.client.0.smithi100.stdout:from 5 to HEAD diff [0~29360128,33554432~16777216,54525952~180355072,239075328~8388608,251658240~12582912,268435456~58720256,331350016~62914560,402653184~16777216] 2024-04-07T17:38:43.761 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~16777216,54525952~180355072,239075328~8388608,251658240~12582912,268435456~58720256,331350016~62914560,402653184~16777216] 2024-04-07T17:38:43.765 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~398458880,402653184~16777216] 2024-04-07T17:38:43.765 INFO:tasks.workunit.client.0.smithi100.stdout:from 6 to HEAD diff [0~29360128,33554432~8388608,46137344~4194304,54525952~180355072,239075328~8388608,251658240~12582912,268435456~41943040,314572800~12582912,331350016~62914560,402653184~8388608] 2024-04-07T17:38:43.765 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~8388608,46137344~4194304,54525952~180355072,239075328~8388608,251658240~12582912,268435456~41943040,314572800~12582912,331350016~62914560,402653184~8388608] 2024-04-07T17:38:43.769 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~398458880,402653184~8388608] 2024-04-07T17:38:43.769 INFO:tasks.workunit.client.0.smithi100.stdout:from 7 to HEAD diff [0~29360128,33554432~8388608,46137344~4194304,54525952~180355072,239075328~8388608,251658240~12582912,268435456~16777216,293601280~16777216,314572800~12582912,331350016~62914560,402653184~8388608] 2024-04-07T17:38:43.769 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~8388608,46137344~4194304,54525952~180355072,239075328~8388608,251658240~12582912,268435456~16777216,293601280~16777216,314572800~12582912,331350016~62914560,402653184~8388608] 2024-04-07T17:38:43.773 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~289406976,293601280~104857600,402653184~8388608] 2024-04-07T17:38:43.773 INFO:tasks.workunit.client.0.smithi100.stdout:from 8 to HEAD diff [0~29360128,33554432~8388608,46137344~4194304,54525952~33554432,92274688~142606336,251658240~12582912,268435456~16777216,293601280~16777216,314572800~12582912,331350016~29360128,364904448~29360128,402653184~8388608] 2024-04-07T17:38:43.773 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~8388608,46137344~4194304,54525952~33554432,92274688~142606336,251658240~12582912,268435456~16777216,293601280~16777216,314572800~12582912,331350016~29360128,364904448~29360128,402653184~8388608] 2024-04-07T17:38:43.776 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~41943040,46137344~41943040,92274688~146800640,251658240~37748736,293601280~67108864,364904448~33554432,402653184~8388608] 2024-04-07T17:38:43.777 INFO:tasks.workunit.client.0.smithi100.stdout:from 9 to HEAD diff [0~29360128,33554432~8388608,46137344~4194304,54525952~33554432,100663296~134217728,251658240~12582912,268435456~16777216,293601280~16777216,314572800~12582912,331350016~8388608,343932928~16777216,364904448~29360128,402653184~8388608] 2024-04-07T17:38:43.777 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~8388608,46137344~4194304,54525952~33554432,100663296~134217728,251658240~12582912,268435456~16777216,293601280~16777216,314572800~12582912,331350016~8388608,343932928~16777216,364904448~29360128,402653184~8388608] 2024-04-07T17:38:43.779 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~41943040,46137344~41943040,100663296~138412032,251658240~37748736,293601280~46137344,343932928~16777216,364904448~33554432,402653184~8388608] 2024-04-07T17:38:43.779 INFO:tasks.workunit.client.0.smithi100.stdout:from 10 to HEAD diff [0~12582912,16777216~12582912,33554432~8388608,46137344~4194304,54525952~33554432,100663296~71303168,176160768~58720256,251658240~12582912,268435456~4194304,281018368~8388608,293601280~12582912,314572800~12582912,331350016~8388608,343932928~16777216,364904448~29360128,402653184~8388608] 2024-04-07T17:38:43.779 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~12582912,16777216~12582912,33554432~8388608,46137344~4194304,54525952~33554432,100663296~71303168,176160768~58720256,251658240~12582912,268435456~4194304,281018368~8388608,293601280~12582912,314572800~12582912,331350016~8388608,343932928~16777216,364904448~29360128,402653184~8388608] 2024-04-07T17:38:43.781 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~12582912,16777216~25165824,46137344~4194304,54525952~33554432,100663296~138412032,251658240~20971520,281018368~8388608,293601280~16777216,314572800~25165824,343932928~16777216,364904448~33554432,402653184~8388608] 2024-04-07T17:38:43.781 INFO:tasks.workunit.client.0.smithi100.stdout:from 11 to HEAD diff [0~12582912,16777216~12582912,33554432~8388608,46137344~4194304,54525952~33554432,100663296~71303168,176160768~58720256,251658240~12582912,281018368~8388608,297795584~8388608,314572800~12582912,331350016~8388608,343932928~16777216,364904448~29360128,402653184~8388608] 2024-04-07T17:38:43.781 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~12582912,16777216~12582912,33554432~8388608,46137344~4194304,54525952~33554432,100663296~71303168,176160768~58720256,251658240~12582912,281018368~8388608,297795584~8388608,314572800~12582912,331350016~8388608,343932928~16777216,364904448~29360128,402653184~8388608] 2024-04-07T17:38:43.783 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~12582912,16777216~25165824,46137344~4194304,54525952~33554432,100663296~138412032,251658240~16777216,281018368~8388608,297795584~12582912,314572800~25165824,343932928~16777216,364904448~33554432,402653184~8388608] 2024-04-07T17:38:43.783 INFO:tasks.workunit.client.0.smithi100.stdout:from 12 to HEAD diff [4194304~8388608,16777216~12582912,33554432~8388608,46137344~4194304,58720256~29360128,100663296~71303168,176160768~20971520,201326592~33554432,251658240~12582912,281018368~8388608,297795584~8388608,314572800~12582912,331350016~8388608,343932928~16777216,364904448~29360128,402653184~8388608] 2024-04-07T17:38:43.783 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [4194304~8388608,16777216~12582912,33554432~8388608,46137344~4194304,58720256~29360128,100663296~71303168,176160768~20971520,201326592~33554432,251658240~12582912,281018368~8388608,297795584~8388608,314572800~12582912,331350016~8388608,343932928~16777216,364904448~29360128,402653184~8388608] 2024-04-07T17:38:43.784 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [4194304~8388608,16777216~25165824,46137344~4194304,58720256~29360128,100663296~96468992,201326592~37748736,251658240~16777216,281018368~8388608,297795584~12582912,314572800~25165824,343932928~16777216,364904448~33554432,402653184~8388608] 2024-04-07T17:38:43.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:38:43 smithi100 bash[74942]: cluster 2024-04-07T17:38:42.513035+0000 mgr.y (mgr.25175) 2500 : cluster [DBG] pgmap v4159: 137 pgs: 137 active+clean; 931 MiB data, 6.1 GiB used, 709 GiB / 715 GiB avail; 585 KiB/s rd, 3.8 MiB/s wr, 1.08k op/s 2024-04-07T17:38:43.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:38:43 smithi100 bash[74942]: cluster 2024-04-07T17:38:42.607096+0000 mon.a (mon.0) 3979 : cluster [DBG] osdmap e2085: 8 total, 8 up, 8 in 2024-04-07T17:38:43.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:38:43 smithi100 bash[76202]: cluster 2024-04-07T17:38:42.513035+0000 mgr.y (mgr.25175) 2500 : cluster [DBG] pgmap v4159: 137 pgs: 137 active+clean; 931 MiB data, 6.1 GiB used, 709 GiB / 715 GiB avail; 585 KiB/s rd, 3.8 MiB/s wr, 1.08k op/s 2024-04-07T17:38:43.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:38:43 smithi100 bash[76202]: cluster 2024-04-07T17:38:42.607096+0000 mon.a (mon.0) 3979 : cluster [DBG] osdmap e2085: 8 total, 8 up, 8 in 2024-04-07T17:38:44.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:38:43 smithi144 bash[66654]: cluster 2024-04-07T17:38:42.513035+0000 mgr.y (mgr.25175) 2500 : cluster [DBG] pgmap v4159: 137 pgs: 137 active+clean; 931 MiB data, 6.1 GiB used, 709 GiB / 715 GiB avail; 585 KiB/s rd, 3.8 MiB/s wr, 1.08k op/s 2024-04-07T17:38:44.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:38:43 smithi144 bash[66654]: cluster 2024-04-07T17:38:42.607096+0000 mon.a (mon.0) 3979 : cluster [DBG] osdmap e2085: 8 total, 8 up, 8 in 2024-04-07T17:38:44.674 INFO:tasks.workunit.client.0.smithi100.stdout:from 0 to HEAD diff [0~29360128,33554432~16777216,54525952~96468992,155189248~109051904,268435456~150994944] 2024-04-07T17:38:44.674 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~16777216,54525952~96468992,155189248~109051904,268435456~150994944] 2024-04-07T17:38:44.727 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~419430400] 2024-04-07T17:38:44.727 INFO:tasks.workunit.client.0.smithi100.stdout:from 1 to HEAD diff [0~29360128,33554432~16777216,54525952~209715200,268435456~150994944] 2024-04-07T17:38:44.727 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~16777216,54525952~209715200,268435456~150994944] 2024-04-07T17:38:44.740 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~419430400] 2024-04-07T17:38:44.740 INFO:tasks.workunit.client.0.smithi100.stdout:from 2 to HEAD diff [0~29360128,33554432~16777216,54525952~209715200,268435456~150994944] 2024-04-07T17:38:44.740 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~16777216,54525952~209715200,268435456~150994944] 2024-04-07T17:38:44.746 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~419430400] 2024-04-07T17:38:44.746 INFO:tasks.workunit.client.0.smithi100.stdout:from 3 to HEAD diff [0~29360128,33554432~16777216,54525952~209715200,268435456~58720256,331350016~62914560,402653184~16777216] 2024-04-07T17:38:44.746 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~16777216,54525952~209715200,268435456~58720256,331350016~62914560,402653184~16777216] 2024-04-07T17:38:44.751 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~398458880,402653184~16777216] 2024-04-07T17:38:44.751 INFO:tasks.workunit.client.0.smithi100.stdout:from 4 to HEAD diff [0~29360128,33554432~16777216,54525952~155189248,213909504~50331648,268435456~58720256,331350016~62914560,402653184~16777216] 2024-04-07T17:38:44.751 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~16777216,54525952~155189248,213909504~50331648,268435456~58720256,331350016~62914560,402653184~16777216] 2024-04-07T17:38:44.755 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~398458880,402653184~16777216] 2024-04-07T17:38:44.755 INFO:tasks.workunit.client.0.smithi100.stdout:from 5 to HEAD diff [0~29360128,33554432~16777216,54525952~180355072,239075328~8388608,251658240~12582912,268435456~58720256,331350016~62914560,402653184~16777216] 2024-04-07T17:38:44.755 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~16777216,54525952~180355072,239075328~8388608,251658240~12582912,268435456~58720256,331350016~62914560,402653184~16777216] 2024-04-07T17:38:44.759 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~398458880,402653184~16777216] 2024-04-07T17:38:44.759 INFO:tasks.workunit.client.0.smithi100.stdout:from 6 to HEAD diff [0~29360128,33554432~8388608,46137344~4194304,54525952~180355072,239075328~8388608,251658240~12582912,268435456~41943040,314572800~12582912,331350016~62914560,402653184~8388608] 2024-04-07T17:38:44.759 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~8388608,46137344~4194304,54525952~180355072,239075328~8388608,251658240~12582912,268435456~41943040,314572800~12582912,331350016~62914560,402653184~8388608] 2024-04-07T17:38:44.762 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~398458880,402653184~8388608] 2024-04-07T17:38:44.762 INFO:tasks.workunit.client.0.smithi100.stdout:from 7 to HEAD diff [0~29360128,33554432~8388608,46137344~4194304,54525952~180355072,239075328~8388608,251658240~12582912,268435456~16777216,293601280~16777216,314572800~12582912,331350016~62914560,402653184~8388608] 2024-04-07T17:38:44.763 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~8388608,46137344~4194304,54525952~180355072,239075328~8388608,251658240~12582912,268435456~16777216,293601280~16777216,314572800~12582912,331350016~62914560,402653184~8388608] 2024-04-07T17:38:44.766 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~289406976,293601280~104857600,402653184~8388608] 2024-04-07T17:38:44.766 INFO:tasks.workunit.client.0.smithi100.stdout:from 8 to HEAD diff [0~29360128,33554432~8388608,46137344~4194304,54525952~33554432,92274688~142606336,251658240~12582912,268435456~16777216,293601280~16777216,314572800~12582912,331350016~29360128,364904448~29360128,402653184~8388608] 2024-04-07T17:38:44.766 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~8388608,46137344~4194304,54525952~33554432,92274688~142606336,251658240~12582912,268435456~16777216,293601280~16777216,314572800~12582912,331350016~29360128,364904448~29360128,402653184~8388608] 2024-04-07T17:38:44.768 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~41943040,46137344~41943040,92274688~146800640,251658240~37748736,293601280~67108864,364904448~33554432,402653184~8388608] 2024-04-07T17:38:44.768 INFO:tasks.workunit.client.0.smithi100.stdout:from 9 to HEAD diff [0~29360128,33554432~8388608,46137344~4194304,54525952~33554432,100663296~134217728,251658240~12582912,268435456~16777216,293601280~16777216,314572800~12582912,331350016~8388608,343932928~16777216,364904448~29360128,402653184~8388608] 2024-04-07T17:38:44.768 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~8388608,46137344~4194304,54525952~33554432,100663296~134217728,251658240~12582912,268435456~16777216,293601280~16777216,314572800~12582912,331350016~8388608,343932928~16777216,364904448~29360128,402653184~8388608] 2024-04-07T17:38:44.770 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~41943040,46137344~41943040,100663296~138412032,251658240~37748736,293601280~46137344,343932928~16777216,364904448~33554432,402653184~8388608] 2024-04-07T17:38:44.770 INFO:tasks.workunit.client.0.smithi100.stdout:from 10 to HEAD diff [0~12582912,16777216~12582912,33554432~8388608,46137344~4194304,54525952~33554432,100663296~71303168,176160768~58720256,251658240~12582912,268435456~4194304,281018368~8388608,293601280~12582912,314572800~12582912,331350016~8388608,343932928~16777216,364904448~29360128,402653184~8388608] 2024-04-07T17:38:44.770 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~12582912,16777216~12582912,33554432~8388608,46137344~4194304,54525952~33554432,100663296~71303168,176160768~58720256,251658240~12582912,268435456~4194304,281018368~8388608,293601280~12582912,314572800~12582912,331350016~8388608,343932928~16777216,364904448~29360128,402653184~8388608] 2024-04-07T17:38:44.772 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~12582912,16777216~25165824,46137344~4194304,54525952~33554432,100663296~138412032,251658240~20971520,281018368~8388608,293601280~16777216,314572800~25165824,343932928~16777216,364904448~33554432,402653184~8388608] 2024-04-07T17:38:44.772 INFO:tasks.workunit.client.0.smithi100.stdout:from 11 to HEAD diff [0~12582912,16777216~12582912,33554432~8388608,46137344~4194304,54525952~33554432,100663296~71303168,176160768~58720256,251658240~12582912,281018368~8388608,297795584~8388608,314572800~12582912,331350016~8388608,343932928~16777216,364904448~29360128,402653184~8388608] 2024-04-07T17:38:44.772 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~12582912,16777216~12582912,33554432~8388608,46137344~4194304,54525952~33554432,100663296~71303168,176160768~58720256,251658240~12582912,281018368~8388608,297795584~8388608,314572800~12582912,331350016~8388608,343932928~16777216,364904448~29360128,402653184~8388608] 2024-04-07T17:38:44.773 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~12582912,16777216~25165824,46137344~4194304,54525952~33554432,100663296~138412032,251658240~16777216,281018368~8388608,297795584~12582912,314572800~25165824,343932928~16777216,364904448~33554432,402653184~8388608] 2024-04-07T17:38:44.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:38:44 smithi100 bash[74942]: cluster 2024-04-07T17:38:43.634491+0000 mon.a (mon.0) 3980 : cluster [DBG] osdmap e2086: 8 total, 8 up, 8 in 2024-04-07T17:38:44.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:38:44 smithi100 bash[76202]: cluster 2024-04-07T17:38:43.634491+0000 mon.a (mon.0) 3980 : cluster [DBG] osdmap e2086: 8 total, 8 up, 8 in 2024-04-07T17:38:45.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:38:44 smithi144 bash[66654]: cluster 2024-04-07T17:38:43.634491+0000 mon.a (mon.0) 3980 : cluster [DBG] osdmap e2086: 8 total, 8 up, 8 in 2024-04-07T17:38:45.691 INFO:tasks.workunit.client.0.smithi100.stdout:from 0 to HEAD diff [0~29360128,33554432~16777216,54525952~96468992,155189248~109051904,268435456~150994944] 2024-04-07T17:38:45.691 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~16777216,54525952~96468992,155189248~109051904,268435456~150994944] 2024-04-07T17:38:45.748 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~419430400] 2024-04-07T17:38:45.748 INFO:tasks.workunit.client.0.smithi100.stdout:from 1 to HEAD diff [0~29360128,33554432~16777216,54525952~209715200,268435456~150994944] 2024-04-07T17:38:45.748 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~16777216,54525952~209715200,268435456~150994944] 2024-04-07T17:38:45.760 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~419430400] 2024-04-07T17:38:45.760 INFO:tasks.workunit.client.0.smithi100.stdout:from 2 to HEAD diff [0~29360128,33554432~16777216,54525952~209715200,268435456~150994944] 2024-04-07T17:38:45.760 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~16777216,54525952~209715200,268435456~150994944] 2024-04-07T17:38:45.765 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~419430400] 2024-04-07T17:38:45.765 INFO:tasks.workunit.client.0.smithi100.stdout:from 3 to HEAD diff [0~29360128,33554432~16777216,54525952~209715200,268435456~58720256,331350016~62914560,402653184~16777216] 2024-04-07T17:38:45.766 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~16777216,54525952~209715200,268435456~58720256,331350016~62914560,402653184~16777216] 2024-04-07T17:38:45.770 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~398458880,402653184~16777216] 2024-04-07T17:38:45.770 INFO:tasks.workunit.client.0.smithi100.stdout:from 4 to HEAD diff [0~29360128,33554432~16777216,54525952~155189248,213909504~50331648,268435456~58720256,331350016~62914560,402653184~16777216] 2024-04-07T17:38:45.770 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~16777216,54525952~155189248,213909504~50331648,268435456~58720256,331350016~62914560,402653184~16777216] 2024-04-07T17:38:45.774 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~398458880,402653184~16777216] 2024-04-07T17:38:45.774 INFO:tasks.workunit.client.0.smithi100.stdout:from 5 to HEAD diff [0~29360128,33554432~16777216,54525952~180355072,239075328~8388608,251658240~12582912,268435456~58720256,331350016~62914560,402653184~16777216] 2024-04-07T17:38:45.774 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~16777216,54525952~180355072,239075328~8388608,251658240~12582912,268435456~58720256,331350016~62914560,402653184~16777216] 2024-04-07T17:38:45.778 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~398458880,402653184~16777216] 2024-04-07T17:38:45.778 INFO:tasks.workunit.client.0.smithi100.stdout:from 6 to HEAD diff [0~29360128,33554432~8388608,46137344~4194304,54525952~180355072,239075328~8388608,251658240~12582912,268435456~41943040,314572800~12582912,331350016~62914560,402653184~8388608] 2024-04-07T17:38:45.778 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~8388608,46137344~4194304,54525952~180355072,239075328~8388608,251658240~12582912,268435456~41943040,314572800~12582912,331350016~62914560,402653184~8388608] 2024-04-07T17:38:45.781 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~398458880,402653184~8388608] 2024-04-07T17:38:45.781 INFO:tasks.workunit.client.0.smithi100.stdout:from 7 to HEAD diff [0~29360128,33554432~8388608,46137344~4194304,54525952~180355072,239075328~8388608,251658240~12582912,268435456~16777216,293601280~16777216,314572800~12582912,331350016~62914560,402653184~8388608] 2024-04-07T17:38:45.781 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~8388608,46137344~4194304,54525952~180355072,239075328~8388608,251658240~12582912,268435456~16777216,293601280~16777216,314572800~12582912,331350016~62914560,402653184~8388608] 2024-04-07T17:38:45.784 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~289406976,293601280~104857600,402653184~8388608] 2024-04-07T17:38:45.784 INFO:tasks.workunit.client.0.smithi100.stdout:from 8 to HEAD diff [0~29360128,33554432~8388608,46137344~4194304,54525952~33554432,92274688~142606336,251658240~12582912,268435456~16777216,293601280~16777216,314572800~12582912,331350016~29360128,364904448~29360128,402653184~8388608] 2024-04-07T17:38:45.784 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~8388608,46137344~4194304,54525952~33554432,92274688~142606336,251658240~12582912,268435456~16777216,293601280~16777216,314572800~12582912,331350016~29360128,364904448~29360128,402653184~8388608] 2024-04-07T17:38:45.786 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~41943040,46137344~41943040,92274688~146800640,251658240~37748736,293601280~67108864,364904448~33554432,402653184~8388608] 2024-04-07T17:38:45.786 INFO:tasks.workunit.client.0.smithi100.stdout:from 9 to HEAD diff [0~29360128,33554432~8388608,46137344~4194304,54525952~33554432,100663296~134217728,251658240~12582912,268435456~16777216,293601280~16777216,314572800~12582912,331350016~8388608,343932928~16777216,364904448~29360128,402653184~8388608] 2024-04-07T17:38:45.786 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~8388608,46137344~4194304,54525952~33554432,100663296~134217728,251658240~12582912,268435456~16777216,293601280~16777216,314572800~12582912,331350016~8388608,343932928~16777216,364904448~29360128,402653184~8388608] 2024-04-07T17:38:45.788 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~41943040,46137344~41943040,100663296~138412032,251658240~37748736,293601280~46137344,343932928~16777216,364904448~33554432,402653184~8388608] 2024-04-07T17:38:45.788 INFO:tasks.workunit.client.0.smithi100.stdout:from 10 to HEAD diff [0~12582912,16777216~12582912,33554432~8388608,46137344~4194304,54525952~33554432,100663296~71303168,176160768~58720256,251658240~12582912,268435456~4194304,281018368~8388608,293601280~12582912,314572800~12582912,331350016~8388608,343932928~16777216,364904448~29360128,402653184~8388608] 2024-04-07T17:38:45.788 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~12582912,16777216~12582912,33554432~8388608,46137344~4194304,54525952~33554432,100663296~71303168,176160768~58720256,251658240~12582912,268435456~4194304,281018368~8388608,293601280~12582912,314572800~12582912,331350016~8388608,343932928~16777216,364904448~29360128,402653184~8388608] 2024-04-07T17:38:45.789 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~12582912,16777216~25165824,46137344~4194304,54525952~33554432,100663296~138412032,251658240~20971520,281018368~8388608,293601280~16777216,314572800~25165824,343932928~16777216,364904448~33554432,402653184~8388608] 2024-04-07T17:38:45.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:38:45 smithi100 bash[74942]: cluster 2024-04-07T17:38:44.514065+0000 mgr.y (mgr.25175) 2501 : cluster [DBG] pgmap v4162: 137 pgs: 137 active+clean; 912 MiB data, 6.0 GiB used, 709 GiB / 715 GiB avail; 372 KiB/s rd, 6.0 KiB/s wr, 671 op/s 2024-04-07T17:38:45.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:38:45 smithi100 bash[74942]: cluster 2024-04-07T17:38:44.637247+0000 mon.a (mon.0) 3981 : cluster [DBG] osdmap e2087: 8 total, 8 up, 8 in 2024-04-07T17:38:45.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:38:45 smithi100 bash[76202]: cluster 2024-04-07T17:38:44.514065+0000 mgr.y (mgr.25175) 2501 : cluster [DBG] pgmap v4162: 137 pgs: 137 active+clean; 912 MiB data, 6.0 GiB used, 709 GiB / 715 GiB avail; 372 KiB/s rd, 6.0 KiB/s wr, 671 op/s 2024-04-07T17:38:45.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:38:45 smithi100 bash[76202]: cluster 2024-04-07T17:38:44.637247+0000 mon.a (mon.0) 3981 : cluster [DBG] osdmap e2087: 8 total, 8 up, 8 in 2024-04-07T17:38:46.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:38:45 smithi144 bash[66654]: cluster 2024-04-07T17:38:44.514065+0000 mgr.y (mgr.25175) 2501 : cluster [DBG] pgmap v4162: 137 pgs: 137 active+clean; 912 MiB data, 6.0 GiB used, 709 GiB / 715 GiB avail; 372 KiB/s rd, 6.0 KiB/s wr, 671 op/s 2024-04-07T17:38:46.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:38:45 smithi144 bash[66654]: cluster 2024-04-07T17:38:44.637247+0000 mon.a (mon.0) 3981 : cluster [DBG] osdmap e2087: 8 total, 8 up, 8 in 2024-04-07T17:38:46.712 INFO:tasks.workunit.client.0.smithi100.stdout:from 0 to HEAD diff [0~29360128,33554432~16777216,54525952~96468992,155189248~109051904,268435456~150994944] 2024-04-07T17:38:46.712 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~16777216,54525952~96468992,155189248~109051904,268435456~150994944] 2024-04-07T17:38:46.755 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~419430400] 2024-04-07T17:38:46.755 INFO:tasks.workunit.client.0.smithi100.stdout:from 1 to HEAD diff [0~29360128,33554432~16777216,54525952~209715200,268435456~150994944] 2024-04-07T17:38:46.755 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~16777216,54525952~209715200,268435456~150994944] 2024-04-07T17:38:46.769 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~419430400] 2024-04-07T17:38:46.769 INFO:tasks.workunit.client.0.smithi100.stdout:from 2 to HEAD diff [0~29360128,33554432~16777216,54525952~209715200,268435456~150994944] 2024-04-07T17:38:46.769 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~16777216,54525952~209715200,268435456~150994944] 2024-04-07T17:38:46.774 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~419430400] 2024-04-07T17:38:46.775 INFO:tasks.workunit.client.0.smithi100.stdout:from 3 to HEAD diff [0~29360128,33554432~16777216,54525952~209715200,268435456~58720256,331350016~62914560,402653184~16777216] 2024-04-07T17:38:46.775 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~16777216,54525952~209715200,268435456~58720256,331350016~62914560,402653184~16777216] 2024-04-07T17:38:46.779 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~398458880,402653184~16777216] 2024-04-07T17:38:46.779 INFO:tasks.workunit.client.0.smithi100.stdout:from 4 to HEAD diff [0~29360128,33554432~16777216,54525952~155189248,213909504~50331648,268435456~58720256,331350016~62914560,402653184~16777216] 2024-04-07T17:38:46.779 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~16777216,54525952~155189248,213909504~50331648,268435456~58720256,331350016~62914560,402653184~16777216] 2024-04-07T17:38:46.783 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~398458880,402653184~16777216] 2024-04-07T17:38:46.783 INFO:tasks.workunit.client.0.smithi100.stdout:from 5 to HEAD diff [0~29360128,33554432~16777216,54525952~180355072,239075328~8388608,251658240~12582912,268435456~58720256,331350016~62914560,402653184~16777216] 2024-04-07T17:38:46.783 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~16777216,54525952~180355072,239075328~8388608,251658240~12582912,268435456~58720256,331350016~62914560,402653184~16777216] 2024-04-07T17:38:46.786 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~398458880,402653184~16777216] 2024-04-07T17:38:46.786 INFO:tasks.workunit.client.0.smithi100.stdout:from 6 to HEAD diff [0~29360128,33554432~8388608,46137344~4194304,54525952~180355072,239075328~8388608,251658240~12582912,268435456~41943040,314572800~12582912,331350016~62914560,402653184~8388608] 2024-04-07T17:38:46.786 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~8388608,46137344~4194304,54525952~180355072,239075328~8388608,251658240~12582912,268435456~41943040,314572800~12582912,331350016~62914560,402653184~8388608] 2024-04-07T17:38:46.789 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~398458880,402653184~8388608] 2024-04-07T17:38:46.789 INFO:tasks.workunit.client.0.smithi100.stdout:from 7 to HEAD diff [0~29360128,33554432~8388608,46137344~4194304,54525952~180355072,239075328~8388608,251658240~12582912,268435456~16777216,293601280~16777216,314572800~12582912,331350016~62914560,402653184~8388608] 2024-04-07T17:38:46.789 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~8388608,46137344~4194304,54525952~180355072,239075328~8388608,251658240~12582912,268435456~16777216,293601280~16777216,314572800~12582912,331350016~62914560,402653184~8388608] 2024-04-07T17:38:46.791 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~289406976,293601280~104857600,402653184~8388608] 2024-04-07T17:38:46.791 INFO:tasks.workunit.client.0.smithi100.stdout:from 8 to HEAD diff [0~29360128,33554432~8388608,46137344~4194304,54525952~33554432,92274688~142606336,251658240~12582912,268435456~16777216,293601280~16777216,314572800~12582912,331350016~29360128,364904448~29360128,402653184~8388608] 2024-04-07T17:38:46.791 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~8388608,46137344~4194304,54525952~33554432,92274688~142606336,251658240~12582912,268435456~16777216,293601280~16777216,314572800~12582912,331350016~29360128,364904448~29360128,402653184~8388608] 2024-04-07T17:38:46.793 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~41943040,46137344~41943040,92274688~146800640,251658240~37748736,293601280~67108864,364904448~33554432,402653184~8388608] 2024-04-07T17:38:46.793 INFO:tasks.workunit.client.0.smithi100.stdout:from 9 to HEAD diff [0~29360128,33554432~8388608,46137344~4194304,54525952~33554432,100663296~134217728,251658240~12582912,268435456~16777216,293601280~16777216,314572800~12582912,331350016~8388608,343932928~16777216,364904448~29360128,402653184~8388608] 2024-04-07T17:38:46.793 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~8388608,46137344~4194304,54525952~33554432,100663296~134217728,251658240~12582912,268435456~16777216,293601280~16777216,314572800~12582912,331350016~8388608,343932928~16777216,364904448~29360128,402653184~8388608] 2024-04-07T17:38:46.794 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~41943040,46137344~41943040,100663296~138412032,251658240~37748736,293601280~46137344,343932928~16777216,364904448~33554432,402653184~8388608] 2024-04-07T17:38:46.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:38:46 smithi100 bash[74942]: cluster 2024-04-07T17:38:45.655475+0000 mon.a (mon.0) 3982 : cluster [DBG] osdmap e2088: 8 total, 8 up, 8 in 2024-04-07T17:38:46.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:38:46 smithi100 bash[76202]: cluster 2024-04-07T17:38:45.655475+0000 mon.a (mon.0) 3982 : cluster [DBG] osdmap e2088: 8 total, 8 up, 8 in 2024-04-07T17:38:47.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:38:46 smithi144 bash[66654]: cluster 2024-04-07T17:38:45.655475+0000 mon.a (mon.0) 3982 : cluster [DBG] osdmap e2088: 8 total, 8 up, 8 in 2024-04-07T17:38:47.713 INFO:tasks.workunit.client.0.smithi100.stdout:from 0 to HEAD diff [0~29360128,33554432~16777216,54525952~96468992,155189248~109051904,268435456~150994944] 2024-04-07T17:38:47.714 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~16777216,54525952~96468992,155189248~109051904,268435456~150994944] 2024-04-07T17:38:47.762 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~419430400] 2024-04-07T17:38:47.762 INFO:tasks.workunit.client.0.smithi100.stdout:from 1 to HEAD diff [0~29360128,33554432~16777216,54525952~209715200,268435456~150994944] 2024-04-07T17:38:47.762 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~16777216,54525952~209715200,268435456~150994944] 2024-04-07T17:38:47.772 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~419430400] 2024-04-07T17:38:47.773 INFO:tasks.workunit.client.0.smithi100.stdout:from 2 to HEAD diff [0~29360128,33554432~16777216,54525952~209715200,268435456~150994944] 2024-04-07T17:38:47.773 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~16777216,54525952~209715200,268435456~150994944] 2024-04-07T17:38:47.779 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~419430400] 2024-04-07T17:38:47.779 INFO:tasks.workunit.client.0.smithi100.stdout:from 3 to HEAD diff [0~29360128,33554432~16777216,54525952~209715200,268435456~58720256,331350016~62914560,402653184~16777216] 2024-04-07T17:38:47.779 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~16777216,54525952~209715200,268435456~58720256,331350016~62914560,402653184~16777216] 2024-04-07T17:38:47.783 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~398458880,402653184~16777216] 2024-04-07T17:38:47.783 INFO:tasks.workunit.client.0.smithi100.stdout:from 4 to HEAD diff [0~29360128,33554432~16777216,54525952~155189248,213909504~50331648,268435456~58720256,331350016~62914560,402653184~16777216] 2024-04-07T17:38:47.783 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~16777216,54525952~155189248,213909504~50331648,268435456~58720256,331350016~62914560,402653184~16777216] 2024-04-07T17:38:47.786 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~398458880,402653184~16777216] 2024-04-07T17:38:47.786 INFO:tasks.workunit.client.0.smithi100.stdout:from 5 to HEAD diff [0~29360128,33554432~16777216,54525952~180355072,239075328~8388608,251658240~12582912,268435456~58720256,331350016~62914560,402653184~16777216] 2024-04-07T17:38:47.786 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~16777216,54525952~180355072,239075328~8388608,251658240~12582912,268435456~58720256,331350016~62914560,402653184~16777216] 2024-04-07T17:38:47.789 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~398458880,402653184~16777216] 2024-04-07T17:38:47.790 INFO:tasks.workunit.client.0.smithi100.stdout:from 6 to HEAD diff [0~29360128,33554432~8388608,46137344~4194304,54525952~180355072,239075328~8388608,251658240~12582912,268435456~41943040,314572800~12582912,331350016~62914560,402653184~8388608] 2024-04-07T17:38:47.790 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~8388608,46137344~4194304,54525952~180355072,239075328~8388608,251658240~12582912,268435456~41943040,314572800~12582912,331350016~62914560,402653184~8388608] 2024-04-07T17:38:47.792 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~398458880,402653184~8388608] 2024-04-07T17:38:47.792 INFO:tasks.workunit.client.0.smithi100.stdout:from 7 to HEAD diff [0~29360128,33554432~8388608,46137344~4194304,54525952~180355072,239075328~8388608,251658240~12582912,268435456~16777216,293601280~16777216,314572800~12582912,331350016~62914560,402653184~8388608] 2024-04-07T17:38:47.792 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~8388608,46137344~4194304,54525952~180355072,239075328~8388608,251658240~12582912,268435456~16777216,293601280~16777216,314572800~12582912,331350016~62914560,402653184~8388608] 2024-04-07T17:38:47.793 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~289406976,293601280~104857600,402653184~8388608] 2024-04-07T17:38:47.794 INFO:tasks.workunit.client.0.smithi100.stdout:from 8 to HEAD diff [0~29360128,33554432~8388608,46137344~4194304,54525952~33554432,92274688~142606336,251658240~12582912,268435456~16777216,293601280~16777216,314572800~12582912,331350016~29360128,364904448~29360128,402653184~8388608] 2024-04-07T17:38:47.794 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~8388608,46137344~4194304,54525952~33554432,92274688~142606336,251658240~12582912,268435456~16777216,293601280~16777216,314572800~12582912,331350016~29360128,364904448~29360128,402653184~8388608] 2024-04-07T17:38:47.795 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~41943040,46137344~41943040,92274688~146800640,251658240~37748736,293601280~67108864,364904448~33554432,402653184~8388608] 2024-04-07T17:38:47.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:38:47 smithi100 bash[74942]: cluster 2024-04-07T17:38:46.514836+0000 mgr.y (mgr.25175) 2502 : cluster [DBG] pgmap v4165: 137 pgs: 137 active+clean; 896 MiB data, 6.0 GiB used, 709 GiB / 715 GiB avail; 133 KiB/s rd, 1.2 KiB/s wr, 265 op/s 2024-04-07T17:38:47.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:38:47 smithi100 bash[74942]: cluster 2024-04-07T17:38:46.678678+0000 mon.a (mon.0) 3983 : cluster [DBG] osdmap e2089: 8 total, 8 up, 8 in 2024-04-07T17:38:47.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:38:47 smithi100 bash[74942]: audit 2024-04-07T17:38:46.860012+0000 mon.a (mon.0) 3984 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:38:47.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:38:47 smithi100 bash[76202]: cluster 2024-04-07T17:38:46.514836+0000 mgr.y (mgr.25175) 2502 : cluster [DBG] pgmap v4165: 137 pgs: 137 active+clean; 896 MiB data, 6.0 GiB used, 709 GiB / 715 GiB avail; 133 KiB/s rd, 1.2 KiB/s wr, 265 op/s 2024-04-07T17:38:47.955 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:38:47 smithi100 bash[76202]: cluster 2024-04-07T17:38:46.678678+0000 mon.a (mon.0) 3983 : cluster [DBG] osdmap e2089: 8 total, 8 up, 8 in 2024-04-07T17:38:47.955 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:38:47 smithi100 bash[76202]: audit 2024-04-07T17:38:46.860012+0000 mon.a (mon.0) 3984 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:38:48.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:38:47 smithi144 bash[66654]: cluster 2024-04-07T17:38:46.514836+0000 mgr.y (mgr.25175) 2502 : cluster [DBG] pgmap v4165: 137 pgs: 137 active+clean; 896 MiB data, 6.0 GiB used, 709 GiB / 715 GiB avail; 133 KiB/s rd, 1.2 KiB/s wr, 265 op/s 2024-04-07T17:38:48.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:38:47 smithi144 bash[66654]: cluster 2024-04-07T17:38:46.678678+0000 mon.a (mon.0) 3983 : cluster [DBG] osdmap e2089: 8 total, 8 up, 8 in 2024-04-07T17:38:48.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:38:47 smithi144 bash[66654]: audit 2024-04-07T17:38:46.860012+0000 mon.a (mon.0) 3984 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:38:48.263 INFO:tasks.workunit.client.0.smithi100.stdout:from 0 to HEAD diff [0~29360128,33554432~16777216,54525952~96468992,155189248~109051904,268435456~150994944] 2024-04-07T17:38:48.263 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~16777216,54525952~96468992,155189248~109051904,268435456~150994944] 2024-04-07T17:38:48.312 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~419430400] 2024-04-07T17:38:48.312 INFO:tasks.workunit.client.0.smithi100.stdout:from 1 to HEAD diff [0~29360128,33554432~16777216,54525952~209715200,268435456~150994944] 2024-04-07T17:38:48.312 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~16777216,54525952~209715200,268435456~150994944] 2024-04-07T17:38:48.331 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~419430400] 2024-04-07T17:38:48.332 INFO:tasks.workunit.client.0.smithi100.stdout:from 2 to HEAD diff [0~29360128,33554432~16777216,54525952~209715200,268435456~150994944] 2024-04-07T17:38:48.332 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~16777216,54525952~209715200,268435456~150994944] 2024-04-07T17:38:48.336 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~419430400] 2024-04-07T17:38:48.336 INFO:tasks.workunit.client.0.smithi100.stdout:from 3 to HEAD diff [0~29360128,33554432~16777216,54525952~209715200,268435456~58720256,331350016~62914560,402653184~16777216] 2024-04-07T17:38:48.337 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~16777216,54525952~209715200,268435456~58720256,331350016~62914560,402653184~16777216] 2024-04-07T17:38:48.340 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~398458880,402653184~16777216] 2024-04-07T17:38:48.340 INFO:tasks.workunit.client.0.smithi100.stdout:from 4 to HEAD diff [0~29360128,33554432~16777216,54525952~155189248,213909504~50331648,268435456~58720256,331350016~62914560,402653184~16777216] 2024-04-07T17:38:48.340 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~16777216,54525952~155189248,213909504~50331648,268435456~58720256,331350016~62914560,402653184~16777216] 2024-04-07T17:38:48.343 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~398458880,402653184~16777216] 2024-04-07T17:38:48.343 INFO:tasks.workunit.client.0.smithi100.stdout:from 5 to HEAD diff [0~29360128,33554432~16777216,54525952~180355072,239075328~8388608,251658240~12582912,268435456~58720256,331350016~62914560,402653184~16777216] 2024-04-07T17:38:48.343 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~16777216,54525952~180355072,239075328~8388608,251658240~12582912,268435456~58720256,331350016~62914560,402653184~16777216] 2024-04-07T17:38:48.346 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~398458880,402653184~16777216] 2024-04-07T17:38:48.346 INFO:tasks.workunit.client.0.smithi100.stdout:from 6 to HEAD diff [0~29360128,33554432~8388608,46137344~4194304,54525952~180355072,239075328~8388608,251658240~12582912,268435456~41943040,314572800~12582912,331350016~62914560,402653184~8388608] 2024-04-07T17:38:48.346 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~8388608,46137344~4194304,54525952~180355072,239075328~8388608,251658240~12582912,268435456~41943040,314572800~12582912,331350016~62914560,402653184~8388608] 2024-04-07T17:38:48.348 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~398458880,402653184~8388608] 2024-04-07T17:38:48.348 INFO:tasks.workunit.client.0.smithi100.stdout:from 7 to HEAD diff [0~29360128,33554432~8388608,46137344~4194304,54525952~180355072,239075328~8388608,251658240~12582912,268435456~16777216,293601280~16777216,314572800~12582912,331350016~62914560,402653184~8388608] 2024-04-07T17:38:48.348 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~8388608,46137344~4194304,54525952~180355072,239075328~8388608,251658240~12582912,268435456~16777216,293601280~16777216,314572800~12582912,331350016~62914560,402653184~8388608] 2024-04-07T17:38:48.349 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~289406976,293601280~104857600,402653184~8388608] 2024-04-07T17:38:48.688 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:38:48 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:38:48] "GET /metrics HTTP/1.1" 200 36759 "" "Prometheus/2.43.0" 2024-04-07T17:38:48.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:38:48 smithi100 bash[76202]: cluster 2024-04-07T17:38:47.678614+0000 mon.a (mon.0) 3985 : cluster [DBG] osdmap e2090: 8 total, 8 up, 8 in 2024-04-07T17:38:48.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:38:48 smithi100 bash[76202]: cluster 2024-04-07T17:38:48.225848+0000 mon.a (mon.0) 3986 : cluster [DBG] osdmap e2091: 8 total, 8 up, 8 in 2024-04-07T17:38:48.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:38:48 smithi100 bash[74942]: cluster 2024-04-07T17:38:47.678614+0000 mon.a (mon.0) 3985 : cluster [DBG] osdmap e2090: 8 total, 8 up, 8 in 2024-04-07T17:38:48.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:38:48 smithi100 bash[74942]: cluster 2024-04-07T17:38:48.225848+0000 mon.a (mon.0) 3986 : cluster [DBG] osdmap e2091: 8 total, 8 up, 8 in 2024-04-07T17:38:49.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:38:48 smithi144 bash[66654]: cluster 2024-04-07T17:38:47.678614+0000 mon.a (mon.0) 3985 : cluster [DBG] osdmap e2090: 8 total, 8 up, 8 in 2024-04-07T17:38:49.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:38:48 smithi144 bash[66654]: cluster 2024-04-07T17:38:48.225848+0000 mon.a (mon.0) 3986 : cluster [DBG] osdmap e2091: 8 total, 8 up, 8 in 2024-04-07T17:38:49.267 INFO:tasks.workunit.client.0.smithi100.stdout:from 0 to HEAD diff [0~29360128,33554432~16777216,54525952~96468992,155189248~109051904,268435456~150994944] 2024-04-07T17:38:49.267 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~16777216,54525952~96468992,155189248~109051904,268435456~150994944] 2024-04-07T17:38:49.293 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~419430400] 2024-04-07T17:38:49.294 INFO:tasks.workunit.client.0.smithi100.stdout:from 1 to HEAD diff [0~29360128,33554432~16777216,54525952~209715200,268435456~150994944] 2024-04-07T17:38:49.294 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~16777216,54525952~209715200,268435456~150994944] 2024-04-07T17:38:49.327 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~419430400] 2024-04-07T17:38:49.327 INFO:tasks.workunit.client.0.smithi100.stdout:from 2 to HEAD diff [0~29360128,33554432~16777216,54525952~209715200,268435456~150994944] 2024-04-07T17:38:49.327 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~16777216,54525952~209715200,268435456~150994944] 2024-04-07T17:38:49.333 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~419430400] 2024-04-07T17:38:49.333 INFO:tasks.workunit.client.0.smithi100.stdout:from 3 to HEAD diff [0~29360128,33554432~16777216,54525952~209715200,268435456~58720256,331350016~62914560,402653184~16777216] 2024-04-07T17:38:49.333 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~16777216,54525952~209715200,268435456~58720256,331350016~62914560,402653184~16777216] 2024-04-07T17:38:49.337 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~398458880,402653184~16777216] 2024-04-07T17:38:49.337 INFO:tasks.workunit.client.0.smithi100.stdout:from 4 to HEAD diff [0~29360128,33554432~16777216,54525952~155189248,213909504~50331648,268435456~58720256,331350016~62914560,402653184~16777216] 2024-04-07T17:38:49.337 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~16777216,54525952~155189248,213909504~50331648,268435456~58720256,331350016~62914560,402653184~16777216] 2024-04-07T17:38:49.340 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~398458880,402653184~16777216] 2024-04-07T17:38:49.340 INFO:tasks.workunit.client.0.smithi100.stdout:from 5 to HEAD diff [0~29360128,33554432~16777216,54525952~180355072,239075328~8388608,251658240~12582912,268435456~58720256,331350016~62914560,402653184~16777216] 2024-04-07T17:38:49.340 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~16777216,54525952~180355072,239075328~8388608,251658240~12582912,268435456~58720256,331350016~62914560,402653184~16777216] 2024-04-07T17:38:49.342 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~398458880,402653184~16777216] 2024-04-07T17:38:49.342 INFO:tasks.workunit.client.0.smithi100.stdout:from 6 to HEAD diff [0~29360128,33554432~8388608,46137344~4194304,54525952~180355072,239075328~8388608,251658240~12582912,268435456~41943040,314572800~12582912,331350016~62914560,402653184~8388608] 2024-04-07T17:38:49.342 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~8388608,46137344~4194304,54525952~180355072,239075328~8388608,251658240~12582912,268435456~41943040,314572800~12582912,331350016~62914560,402653184~8388608] 2024-04-07T17:38:49.343 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~398458880,402653184~8388608] 2024-04-07T17:38:49.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:38:49 smithi100 bash[74942]: cluster 2024-04-07T17:38:48.515625+0000 mgr.y (mgr.25175) 2503 : cluster [DBG] pgmap v4169: 137 pgs: 137 active+clean; 843 MiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 234 KiB/s rd, 5.1 KiB/s wr, 393 op/s 2024-04-07T17:38:49.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:38:49 smithi100 bash[74942]: cluster 2024-04-07T17:38:49.241161+0000 mon.a (mon.0) 3987 : cluster [DBG] osdmap e2092: 8 total, 8 up, 8 in 2024-04-07T17:38:49.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:38:49 smithi100 bash[76202]: cluster 2024-04-07T17:38:48.515625+0000 mgr.y (mgr.25175) 2503 : cluster [DBG] pgmap v4169: 137 pgs: 137 active+clean; 843 MiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 234 KiB/s rd, 5.1 KiB/s wr, 393 op/s 2024-04-07T17:38:49.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:38:49 smithi100 bash[76202]: cluster 2024-04-07T17:38:49.241161+0000 mon.a (mon.0) 3987 : cluster [DBG] osdmap e2092: 8 total, 8 up, 8 in 2024-04-07T17:38:50.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:38:49 smithi144 bash[66654]: cluster 2024-04-07T17:38:48.515625+0000 mgr.y (mgr.25175) 2503 : cluster [DBG] pgmap v4169: 137 pgs: 137 active+clean; 843 MiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 234 KiB/s rd, 5.1 KiB/s wr, 393 op/s 2024-04-07T17:38:50.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:38:49 smithi144 bash[66654]: cluster 2024-04-07T17:38:49.241161+0000 mon.a (mon.0) 3987 : cluster [DBG] osdmap e2092: 8 total, 8 up, 8 in 2024-04-07T17:38:50.271 INFO:tasks.workunit.client.0.smithi100.stdout:from 0 to HEAD diff [0~29360128,33554432~16777216,54525952~96468992,155189248~109051904,268435456~150994944] 2024-04-07T17:38:50.271 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~16777216,54525952~96468992,155189248~109051904,268435456~150994944] 2024-04-07T17:38:50.308 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~419430400] 2024-04-07T17:38:50.309 INFO:tasks.workunit.client.0.smithi100.stdout:from 1 to HEAD diff [0~29360128,33554432~16777216,54525952~209715200,268435456~150994944] 2024-04-07T17:38:50.309 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~16777216,54525952~209715200,268435456~150994944] 2024-04-07T17:38:50.316 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~419430400] 2024-04-07T17:38:50.316 INFO:tasks.workunit.client.0.smithi100.stdout:from 2 to HEAD diff [0~29360128,33554432~16777216,54525952~209715200,268435456~150994944] 2024-04-07T17:38:50.316 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~16777216,54525952~209715200,268435456~150994944] 2024-04-07T17:38:50.322 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~419430400] 2024-04-07T17:38:50.322 INFO:tasks.workunit.client.0.smithi100.stdout:from 3 to HEAD diff [0~29360128,33554432~16777216,54525952~209715200,268435456~58720256,331350016~62914560,402653184~16777216] 2024-04-07T17:38:50.322 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~16777216,54525952~209715200,268435456~58720256,331350016~62914560,402653184~16777216] 2024-04-07T17:38:50.345 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~398458880,402653184~16777216] 2024-04-07T17:38:50.345 INFO:tasks.workunit.client.0.smithi100.stdout:from 4 to HEAD diff [0~29360128,33554432~16777216,54525952~155189248,213909504~50331648,268435456~58720256,331350016~62914560,402653184~16777216] 2024-04-07T17:38:50.345 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~16777216,54525952~155189248,213909504~50331648,268435456~58720256,331350016~62914560,402653184~16777216] 2024-04-07T17:38:50.347 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~398458880,402653184~16777216] 2024-04-07T17:38:50.348 INFO:tasks.workunit.client.0.smithi100.stdout:from 5 to HEAD diff [0~29360128,33554432~16777216,54525952~180355072,239075328~8388608,251658240~12582912,268435456~58720256,331350016~62914560,402653184~16777216] 2024-04-07T17:38:50.348 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~16777216,54525952~180355072,239075328~8388608,251658240~12582912,268435456~58720256,331350016~62914560,402653184~16777216] 2024-04-07T17:38:50.350 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~398458880,402653184~16777216] 2024-04-07T17:38:51.285 INFO:tasks.workunit.client.0.smithi100.stdout:from 0 to HEAD diff [0~29360128,33554432~16777216,54525952~96468992,155189248~109051904,268435456~150994944] 2024-04-07T17:38:51.285 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~16777216,54525952~96468992,155189248~109051904,268435456~150994944] 2024-04-07T17:38:51.314 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~419430400] 2024-04-07T17:38:51.314 INFO:tasks.workunit.client.0.smithi100.stdout:from 1 to HEAD diff [0~29360128,33554432~16777216,54525952~209715200,268435456~150994944] 2024-04-07T17:38:51.314 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~16777216,54525952~209715200,268435456~150994944] 2024-04-07T17:38:51.324 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~419430400] 2024-04-07T17:38:51.324 INFO:tasks.workunit.client.0.smithi100.stdout:from 2 to HEAD diff [0~29360128,33554432~16777216,54525952~209715200,268435456~150994944] 2024-04-07T17:38:51.325 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~16777216,54525952~209715200,268435456~150994944] 2024-04-07T17:38:51.332 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~419430400] 2024-04-07T17:38:51.332 INFO:tasks.workunit.client.0.smithi100.stdout:from 3 to HEAD diff [0~29360128,33554432~16777216,54525952~209715200,268435456~58720256,331350016~62914560,402653184~16777216] 2024-04-07T17:38:51.332 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~16777216,54525952~209715200,268435456~58720256,331350016~62914560,402653184~16777216] 2024-04-07T17:38:51.338 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~398458880,402653184~16777216] 2024-04-07T17:38:51.338 INFO:tasks.workunit.client.0.smithi100.stdout:from 4 to HEAD diff [0~29360128,33554432~16777216,54525952~155189248,213909504~50331648,268435456~58720256,331350016~62914560,402653184~16777216] 2024-04-07T17:38:51.338 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~16777216,54525952~155189248,213909504~50331648,268435456~58720256,331350016~62914560,402653184~16777216] 2024-04-07T17:38:51.340 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~398458880,402653184~16777216] 2024-04-07T17:38:51.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:38:51 smithi144 bash[66654]: cluster 2024-04-07T17:38:50.239841+0000 mon.a (mon.0) 3988 : cluster [DBG] osdmap e2093: 8 total, 8 up, 8 in 2024-04-07T17:38:51.709 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:38:51 smithi144 bash[66654]: cluster 2024-04-07T17:38:50.516390+0000 mgr.y (mgr.25175) 2504 : cluster [DBG] pgmap v4172: 137 pgs: 137 active+clean; 795 MiB data, 5.7 GiB used, 709 GiB / 715 GiB avail; 230 KiB/s rd, 6.5 KiB/s wr, 384 op/s 2024-04-07T17:38:51.710 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:38:51 smithi100 bash[74942]: cluster 2024-04-07T17:38:50.239841+0000 mon.a (mon.0) 3988 : cluster [DBG] osdmap e2093: 8 total, 8 up, 8 in 2024-04-07T17:38:51.710 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:38:51 smithi100 bash[74942]: cluster 2024-04-07T17:38:50.516390+0000 mgr.y (mgr.25175) 2504 : cluster [DBG] pgmap v4172: 137 pgs: 137 active+clean; 795 MiB data, 5.7 GiB used, 709 GiB / 715 GiB avail; 230 KiB/s rd, 6.5 KiB/s wr, 384 op/s 2024-04-07T17:38:51.710 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:38:51 smithi100 bash[76202]: cluster 2024-04-07T17:38:50.239841+0000 mon.a (mon.0) 3988 : cluster [DBG] osdmap e2093: 8 total, 8 up, 8 in 2024-04-07T17:38:51.710 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:38:51 smithi100 bash[76202]: cluster 2024-04-07T17:38:50.516390+0000 mgr.y (mgr.25175) 2504 : cluster [DBG] pgmap v4172: 137 pgs: 137 active+clean; 795 MiB data, 5.7 GiB used, 709 GiB / 715 GiB avail; 230 KiB/s rd, 6.5 KiB/s wr, 384 op/s 2024-04-07T17:38:52.280 INFO:tasks.workunit.client.0.smithi100.stdout:from 0 to HEAD diff [0~29360128,33554432~16777216,54525952~96468992,155189248~109051904,268435456~150994944] 2024-04-07T17:38:52.280 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~16777216,54525952~96468992,155189248~109051904,268435456~150994944] 2024-04-07T17:38:52.321 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~419430400] 2024-04-07T17:38:52.321 INFO:tasks.workunit.client.0.smithi100.stdout:from 1 to HEAD diff [0~29360128,33554432~16777216,54525952~209715200,268435456~150994944] 2024-04-07T17:38:52.321 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~16777216,54525952~209715200,268435456~150994944] 2024-04-07T17:38:52.325 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~419430400] 2024-04-07T17:38:52.326 INFO:tasks.workunit.client.0.smithi100.stdout:from 2 to HEAD diff [0~29360128,33554432~16777216,54525952~209715200,268435456~150994944] 2024-04-07T17:38:52.326 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~16777216,54525952~209715200,268435456~150994944] 2024-04-07T17:38:52.328 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~419430400] 2024-04-07T17:38:52.328 INFO:tasks.workunit.client.0.smithi100.stdout:from 3 to HEAD diff [0~29360128,33554432~16777216,54525952~209715200,268435456~58720256,331350016~62914560,402653184~16777216] 2024-04-07T17:38:52.328 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~16777216,54525952~209715200,268435456~58720256,331350016~62914560,402653184~16777216] 2024-04-07T17:38:52.329 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~398458880,402653184~16777216] 2024-04-07T17:38:52.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:38:52 smithi144 bash[66654]: cluster 2024-04-07T17:38:51.252819+0000 mon.a (mon.0) 3989 : cluster [DBG] osdmap e2094: 8 total, 8 up, 8 in 2024-04-07T17:38:52.615 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:38:52 smithi144 bash[95233]: ts=2024-04-07T17:38:52.317Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:38:52.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:38:52 smithi100 bash[74942]: cluster 2024-04-07T17:38:51.252819+0000 mon.a (mon.0) 3989 : cluster [DBG] osdmap e2094: 8 total, 8 up, 8 in 2024-04-07T17:38:52.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:38:52 smithi100 bash[76202]: cluster 2024-04-07T17:38:51.252819+0000 mon.a (mon.0) 3989 : cluster [DBG] osdmap e2094: 8 total, 8 up, 8 in 2024-04-07T17:38:53.256 INFO:tasks.workunit.client.0.smithi100.stdout:from 0 to HEAD diff [0~29360128,33554432~16777216,54525952~96468992,155189248~109051904,268435456~150994944] 2024-04-07T17:38:53.256 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~16777216,54525952~96468992,155189248~109051904,268435456~150994944] 2024-04-07T17:38:53.306 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~419430400] 2024-04-07T17:38:53.306 INFO:tasks.workunit.client.0.smithi100.stdout:from 1 to HEAD diff [0~29360128,33554432~16777216,54525952~209715200,268435456~150994944] 2024-04-07T17:38:53.306 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~16777216,54525952~209715200,268435456~150994944] 2024-04-07T17:38:53.318 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~419430400] 2024-04-07T17:38:53.318 INFO:tasks.workunit.client.0.smithi100.stdout:from 2 to HEAD diff [0~29360128,33554432~16777216,54525952~209715200,268435456~150994944] 2024-04-07T17:38:53.318 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~16777216,54525952~209715200,268435456~150994944] 2024-04-07T17:38:53.320 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~419430400] 2024-04-07T17:38:53.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:38:53 smithi144 bash[66654]: cluster 2024-04-07T17:38:52.254769+0000 mon.a (mon.0) 3990 : cluster [DBG] osdmap e2095: 8 total, 8 up, 8 in 2024-04-07T17:38:53.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:38:53 smithi144 bash[66654]: cluster 2024-04-07T17:38:52.517168+0000 mgr.y (mgr.25175) 2505 : cluster [DBG] pgmap v4175: 137 pgs: 137 active+clean; 739 MiB data, 5.6 GiB used, 710 GiB / 715 GiB avail; 137 KiB/s rd, 6.2 KiB/s wr, 225 op/s 2024-04-07T17:38:53.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:38:53 smithi144 bash[66654]: cluster 2024-04-07T17:38:53.228631+0000 mon.a (mon.0) 3991 : cluster [DBG] osdmap e2096: 8 total, 8 up, 8 in 2024-04-07T17:38:53.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:38:53 smithi100 bash[74942]: cluster 2024-04-07T17:38:52.254769+0000 mon.a (mon.0) 3990 : cluster [DBG] osdmap e2095: 8 total, 8 up, 8 in 2024-04-07T17:38:53.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:38:53 smithi100 bash[74942]: cluster 2024-04-07T17:38:52.517168+0000 mgr.y (mgr.25175) 2505 : cluster [DBG] pgmap v4175: 137 pgs: 137 active+clean; 739 MiB data, 5.6 GiB used, 710 GiB / 715 GiB avail; 137 KiB/s rd, 6.2 KiB/s wr, 225 op/s 2024-04-07T17:38:53.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:38:53 smithi100 bash[74942]: cluster 2024-04-07T17:38:53.228631+0000 mon.a (mon.0) 3991 : cluster [DBG] osdmap e2096: 8 total, 8 up, 8 in 2024-04-07T17:38:53.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:38:53 smithi100 bash[76202]: cluster 2024-04-07T17:38:52.254769+0000 mon.a (mon.0) 3990 : cluster [DBG] osdmap e2095: 8 total, 8 up, 8 in 2024-04-07T17:38:53.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:38:53 smithi100 bash[76202]: cluster 2024-04-07T17:38:52.517168+0000 mgr.y (mgr.25175) 2505 : cluster [DBG] pgmap v4175: 137 pgs: 137 active+clean; 739 MiB data, 5.6 GiB used, 710 GiB / 715 GiB avail; 137 KiB/s rd, 6.2 KiB/s wr, 225 op/s 2024-04-07T17:38:53.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:38:53 smithi100 bash[76202]: cluster 2024-04-07T17:38:53.228631+0000 mon.a (mon.0) 3991 : cluster [DBG] osdmap e2096: 8 total, 8 up, 8 in 2024-04-07T17:38:54.333 INFO:tasks.workunit.client.0.smithi100.stdout:from 0 to HEAD diff [0~29360128,33554432~16777216,54525952~96468992,155189248~109051904,268435456~150994944] 2024-04-07T17:38:54.333 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~16777216,54525952~96468992,155189248~109051904,268435456~150994944] 2024-04-07T17:38:54.349 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~419430400] 2024-04-07T17:38:54.349 INFO:tasks.workunit.client.0.smithi100.stdout:from 1 to HEAD diff [0~29360128,33554432~16777216,54525952~209715200,268435456~150994944] 2024-04-07T17:38:54.349 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~16777216,54525952~209715200,268435456~150994944] 2024-04-07T17:38:54.351 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~419430400] 2024-04-07T17:38:55.339 INFO:tasks.workunit.client.0.smithi100.stdout:from 0 to HEAD diff [0~29360128,33554432~16777216,54525952~96468992,155189248~109051904,268435456~150994944] 2024-04-07T17:38:55.339 INFO:tasks.workunit.client.0.smithi100.stdout: limited diff [0~29360128,33554432~16777216,54525952~96468992,155189248~109051904,268435456~150994944] 2024-04-07T17:38:55.373 INFO:tasks.workunit.client.0.smithi100.stdout: actual was [0~419430400] 2024-04-07T17:38:55.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:38:55 smithi144 bash[66654]: cluster 2024-04-07T17:38:54.297556+0000 mon.a (mon.0) 3992 : cluster [DBG] osdmap e2097: 8 total, 8 up, 8 in 2024-04-07T17:38:55.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:38:55 smithi144 bash[66654]: cluster 2024-04-07T17:38:54.517887+0000 mgr.y (mgr.25175) 2506 : cluster [DBG] pgmap v4178: 137 pgs: 137 active+clean; 657 MiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 129 KiB/s rd, 5.7 KiB/s wr, 207 op/s 2024-04-07T17:38:55.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:38:55 smithi100 bash[74942]: cluster 2024-04-07T17:38:54.297556+0000 mon.a (mon.0) 3992 : cluster [DBG] osdmap e2097: 8 total, 8 up, 8 in 2024-04-07T17:38:55.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:38:55 smithi100 bash[74942]: cluster 2024-04-07T17:38:54.517887+0000 mgr.y (mgr.25175) 2506 : cluster [DBG] pgmap v4178: 137 pgs: 137 active+clean; 657 MiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 129 KiB/s rd, 5.7 KiB/s wr, 207 op/s 2024-04-07T17:38:55.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:38:55 smithi100 bash[76202]: cluster 2024-04-07T17:38:54.297556+0000 mon.a (mon.0) 3992 : cluster [DBG] osdmap e2097: 8 total, 8 up, 8 in 2024-04-07T17:38:55.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:38:55 smithi100 bash[76202]: cluster 2024-04-07T17:38:54.517887+0000 mgr.y (mgr.25175) 2506 : cluster [DBG] pgmap v4178: 137 pgs: 137 active+clean; 657 MiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 129 KiB/s rd, 5.7 KiB/s wr, 207 op/s 2024-04-07T17:38:56.387 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] DiffIterateTest/1.DiffIterateStress (39991 ms) 2024-04-07T17:38:56.388 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] DiffIterateTest/1.DiffIterateParent 2024-04-07T17:38:56.388 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:38:56.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:38:56 smithi144 bash[66654]: cluster 2024-04-07T17:38:55.301645+0000 mon.a (mon.0) 3993 : cluster [DBG] osdmap e2098: 8 total, 8 up, 8 in 2024-04-07T17:38:56.703 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:38:56 smithi100 bash[74942]: cluster 2024-04-07T17:38:55.301645+0000 mon.a (mon.0) 3993 : cluster [DBG] osdmap e2098: 8 total, 8 up, 8 in 2024-04-07T17:38:56.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:38:56 smithi100 bash[76202]: cluster 2024-04-07T17:38:55.301645+0000 mon.a (mon.0) 3993 : cluster [DBG] osdmap e2098: 8 total, 8 up, 8 in 2024-04-07T17:38:57.589 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] DiffIterateTest/1.DiffIterateParent (1201 ms) 2024-04-07T17:38:57.589 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] DiffIterateTest/1.DiffIterateIgnoreParent 2024-04-07T17:38:57.589 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:38:57.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:38:57 smithi144 bash[66654]: cluster 2024-04-07T17:38:56.329221+0000 mon.a (mon.0) 3994 : cluster [DBG] osdmap e2099: 8 total, 8 up, 8 in 2024-04-07T17:38:57.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:38:57 smithi144 bash[66654]: cluster 2024-04-07T17:38:56.518573+0000 mgr.y (mgr.25175) 2507 : cluster [DBG] pgmap v4181: 137 pgs: 137 active+clean; 604 MiB data, 5.4 GiB used, 710 GiB / 715 GiB avail; 29 KiB/s rd, 767 B/s wr, 57 op/s 2024-04-07T17:38:57.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:38:57 smithi100 bash[74942]: cluster 2024-04-07T17:38:56.329221+0000 mon.a (mon.0) 3994 : cluster [DBG] osdmap e2099: 8 total, 8 up, 8 in 2024-04-07T17:38:57.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:38:57 smithi100 bash[74942]: cluster 2024-04-07T17:38:56.518573+0000 mgr.y (mgr.25175) 2507 : cluster [DBG] pgmap v4181: 137 pgs: 137 active+clean; 604 MiB data, 5.4 GiB used, 710 GiB / 715 GiB avail; 29 KiB/s rd, 767 B/s wr, 57 op/s 2024-04-07T17:38:57.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:38:57 smithi100 bash[76202]: cluster 2024-04-07T17:38:56.329221+0000 mon.a (mon.0) 3994 : cluster [DBG] osdmap e2099: 8 total, 8 up, 8 in 2024-04-07T17:38:57.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:38:57 smithi100 bash[76202]: cluster 2024-04-07T17:38:56.518573+0000 mgr.y (mgr.25175) 2507 : cluster [DBG] pgmap v4181: 137 pgs: 137 active+clean; 604 MiB data, 5.4 GiB used, 710 GiB / 715 GiB avail; 29 KiB/s rd, 767 B/s wr, 57 op/s 2024-04-07T17:38:58.538 INFO:tasks.workunit.client.0.smithi100.stdout: wrote [0~20971520] to clone 2024-04-07T17:38:58.538 INFO:tasks.workunit.client.0.smithi100.stdout: diff was [0~20971520] 2024-04-07T17:38:58.549 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] DiffIterateTest/1.DiffIterateIgnoreParent (960 ms) 2024-04-07T17:38:58.549 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] DiffIterateTest/1.DiffIterateCallbackError 2024-04-07T17:38:58.549 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:38:58.600 INFO:tasks.workunit.client.0.smithi100.stdout: wrote [99610~83421,2991382~61080,3568055~22928,5840703~6360,10342125~35588,12895112~42601,13257283~33518,16416261~91070,18271600~90294,20186879~21785] 2024-04-07T17:38:58.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:38:58 smithi144 bash[66654]: cluster 2024-04-07T17:38:57.340142+0000 mon.a (mon.0) 3995 : cluster [DBG] osdmap e2100: 8 total, 8 up, 8 in 2024-04-07T17:38:58.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:38:58 smithi144 bash[66654]: cluster 2024-04-07T17:38:58.231621+0000 mon.a (mon.0) 3996 : cluster [DBG] osdmap e2101: 8 total, 8 up, 8 in 2024-04-07T17:38:58.627 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] DiffIterateTest/1.DiffIterateCallbackError (78 ms) 2024-04-07T17:38:58.627 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] DiffIterateTest/1.DiffIterateParentDiscard 2024-04-07T17:38:58.627 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:38:58.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:38:58 smithi100 bash[74942]: cluster 2024-04-07T17:38:57.340142+0000 mon.a (mon.0) 3995 : cluster [DBG] osdmap e2100: 8 total, 8 up, 8 in 2024-04-07T17:38:58.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:38:58 smithi100 bash[74942]: cluster 2024-04-07T17:38:58.231621+0000 mon.a (mon.0) 3996 : cluster [DBG] osdmap e2101: 8 total, 8 up, 8 in 2024-04-07T17:38:58.704 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:38:58 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:38:58] "GET /metrics HTTP/1.1" 200 36759 "" "Prometheus/2.43.0" 2024-04-07T17:38:58.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:38:58 smithi100 bash[76202]: cluster 2024-04-07T17:38:57.340142+0000 mon.a (mon.0) 3995 : cluster [DBG] osdmap e2100: 8 total, 8 up, 8 in 2024-04-07T17:38:58.705 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:38:58 smithi100 bash[76202]: cluster 2024-04-07T17:38:58.231621+0000 mon.a (mon.0) 3996 : cluster [DBG] osdmap e2101: 8 total, 8 up, 8 in 2024-04-07T17:38:59.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:38:59 smithi144 bash[66654]: cluster 2024-04-07T17:38:58.519393+0000 mgr.y (mgr.25175) 2508 : cluster [DBG] pgmap v4184: 137 pgs: 137 active+clean; 520 MiB data, 5.1 GiB used, 710 GiB / 715 GiB avail; 68 KiB/s rd, 2.0 MiB/s wr, 105 op/s 2024-04-07T17:38:59.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:38:59 smithi100 bash[74942]: cluster 2024-04-07T17:38:58.519393+0000 mgr.y (mgr.25175) 2508 : cluster [DBG] pgmap v4184: 137 pgs: 137 active+clean; 520 MiB data, 5.1 GiB used, 710 GiB / 715 GiB avail; 68 KiB/s rd, 2.0 MiB/s wr, 105 op/s 2024-04-07T17:38:59.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:38:59 smithi100 bash[76202]: cluster 2024-04-07T17:38:58.519393+0000 mgr.y (mgr.25175) 2508 : cluster [DBG] pgmap v4184: 137 pgs: 137 active+clean; 520 MiB data, 5.1 GiB used, 710 GiB / 715 GiB avail; 68 KiB/s rd, 2.0 MiB/s wr, 105 op/s 2024-04-07T17:39:00.601 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] DiffIterateTest/1.DiffIterateParentDiscard (1974 ms) 2024-04-07T17:39:00.601 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] DiffIterateTest/1.DiffIterateUnalignedSmall 2024-04-07T17:39:00.601 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:39:00.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:39:00 smithi144 bash[66654]: cluster 2024-04-07T17:38:59.352213+0000 mon.a (mon.0) 3997 : cluster [DBG] osdmap e2102: 8 total, 8 up, 8 in 2024-04-07T17:39:00.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:39:00 smithi100 bash[74942]: cluster 2024-04-07T17:38:59.352213+0000 mon.a (mon.0) 3997 : cluster [DBG] osdmap e2102: 8 total, 8 up, 8 in 2024-04-07T17:39:00.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:39:00 smithi100 bash[76202]: cluster 2024-04-07T17:38:59.352213+0000 mon.a (mon.0) 3997 : cluster [DBG] osdmap e2102: 8 total, 8 up, 8 in 2024-04-07T17:39:00.704 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] DiffIterateTest/1.DiffIterateUnalignedSmall (103 ms) 2024-04-07T17:39:00.704 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] DiffIterateTest/1.DiffIterateUnaligned 2024-04-07T17:39:00.704 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:39:00.840 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] DiffIterateTest/1.DiffIterateUnaligned (136 ms) 2024-04-07T17:39:00.840 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] DiffIterateTest/1.DiffIterateTryAcquireLock 2024-04-07T17:39:00.841 INFO:tasks.workunit.client.0.smithi100.stdout:using new format! 2024-04-07T17:39:01.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:39:01 smithi144 bash[66654]: cluster 2024-04-07T17:39:00.359959+0000 mon.a (mon.0) 3998 : cluster [DBG] osdmap e2103: 8 total, 8 up, 8 in 2024-04-07T17:39:01.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:39:01 smithi144 bash[66654]: cluster 2024-04-07T17:39:00.520223+0000 mgr.y (mgr.25175) 2509 : cluster [DBG] pgmap v4187: 137 pgs: 137 active+clean; 497 MiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 2.1 MiB/s rd, 7.1 MiB/s wr, 206 op/s 2024-04-07T17:39:01.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:39:01 smithi100 bash[74942]: cluster 2024-04-07T17:39:00.359959+0000 mon.a (mon.0) 3998 : cluster [DBG] osdmap e2103: 8 total, 8 up, 8 in 2024-04-07T17:39:01.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:39:01 smithi100 bash[74942]: cluster 2024-04-07T17:39:00.520223+0000 mgr.y (mgr.25175) 2509 : cluster [DBG] pgmap v4187: 137 pgs: 137 active+clean; 497 MiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 2.1 MiB/s rd, 7.1 MiB/s wr, 206 op/s 2024-04-07T17:39:01.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:39:01 smithi100 bash[76202]: cluster 2024-04-07T17:39:00.359959+0000 mon.a (mon.0) 3998 : cluster [DBG] osdmap e2103: 8 total, 8 up, 8 in 2024-04-07T17:39:01.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:39:01 smithi100 bash[76202]: cluster 2024-04-07T17:39:00.520223+0000 mgr.y (mgr.25175) 2509 : cluster [DBG] pgmap v4187: 137 pgs: 137 active+clean; 497 MiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 2.1 MiB/s rd, 7.1 MiB/s wr, 206 op/s 2024-04-07T17:39:02.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:39:02 smithi144 bash[66654]: audit 2024-04-07T17:39:01.860521+0000 mon.a (mon.0) 3999 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:39:02.615 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:39:02 smithi144 bash[95233]: ts=2024-04-07T17:39:02.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:39:02.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:39:02 smithi100 bash[74942]: audit 2024-04-07T17:39:01.860521+0000 mon.a (mon.0) 3999 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:39:02.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:39:02 smithi100 bash[76202]: audit 2024-04-07T17:39:01.860521+0000 mon.a (mon.0) 3999 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:39:03.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:39:03 smithi100 bash[74942]: cluster 2024-04-07T17:39:02.521449+0000 mgr.y (mgr.25175) 2510 : cluster [DBG] pgmap v4188: 137 pgs: 137 active+clean; 535 MiB data, 5.1 GiB used, 710 GiB / 715 GiB avail; 4.8 MiB/s rd, 8.4 MiB/s wr, 210 op/s 2024-04-07T17:39:03.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:39:03 smithi100 bash[74942]: cluster 2024-04-07T17:39:03.233929+0000 mon.a (mon.0) 4000 : cluster [DBG] osdmap e2104: 8 total, 8 up, 8 in 2024-04-07T17:39:03.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:39:03 smithi100 bash[76202]: cluster 2024-04-07T17:39:02.521449+0000 mgr.y (mgr.25175) 2510 : cluster [DBG] pgmap v4188: 137 pgs: 137 active+clean; 535 MiB data, 5.1 GiB used, 710 GiB / 715 GiB avail; 4.8 MiB/s rd, 8.4 MiB/s wr, 210 op/s 2024-04-07T17:39:03.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:39:03 smithi100 bash[76202]: cluster 2024-04-07T17:39:03.233929+0000 mon.a (mon.0) 4000 : cluster [DBG] osdmap e2104: 8 total, 8 up, 8 in 2024-04-07T17:39:03.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:39:03 smithi144 bash[66654]: cluster 2024-04-07T17:39:02.521449+0000 mgr.y (mgr.25175) 2510 : cluster [DBG] pgmap v4188: 137 pgs: 137 active+clean; 535 MiB data, 5.1 GiB used, 710 GiB / 715 GiB avail; 4.8 MiB/s rd, 8.4 MiB/s wr, 210 op/s 2024-04-07T17:39:03.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:39:03 smithi144 bash[66654]: cluster 2024-04-07T17:39:03.233929+0000 mon.a (mon.0) 4000 : cluster [DBG] osdmap e2104: 8 total, 8 up, 8 in 2024-04-07T17:39:05.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:39:05 smithi144 bash[66654]: cluster 2024-04-07T17:39:04.522458+0000 mgr.y (mgr.25175) 2511 : cluster [DBG] pgmap v4190: 137 pgs: 137 active+clean; 555 MiB data, 5.1 GiB used, 710 GiB / 715 GiB avail; 4.2 MiB/s rd, 9.9 MiB/s wr, 246 op/s 2024-04-07T17:39:05.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:39:05 smithi100 bash[74942]: cluster 2024-04-07T17:39:04.522458+0000 mgr.y (mgr.25175) 2511 : cluster [DBG] pgmap v4190: 137 pgs: 137 active+clean; 555 MiB data, 5.1 GiB used, 710 GiB / 715 GiB avail; 4.2 MiB/s rd, 9.9 MiB/s wr, 246 op/s 2024-04-07T17:39:05.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:39:05 smithi100 bash[76202]: cluster 2024-04-07T17:39:04.522458+0000 mgr.y (mgr.25175) 2511 : cluster [DBG] pgmap v4190: 137 pgs: 137 active+clean; 555 MiB data, 5.1 GiB used, 710 GiB / 715 GiB avail; 4.2 MiB/s rd, 9.9 MiB/s wr, 246 op/s 2024-04-07T17:39:05.962 INFO:tasks.workunit.client.0.smithi100.stdout:[ OK ] DiffIterateTest/1.DiffIterateTryAcquireLock (5122 ms) 2024-04-07T17:39:05.962 INFO:tasks.workunit.client.0.smithi100.stdout:[ RUN ] DiffIterateTest/1.DiffIterateStriping 2024-04-07T17:39:05.962 INFO:tasks.workunit.client.0.smithi100.stdout:./src/test/librbd/test_librbd.cc:8604: Skipped 2024-04-07T17:39:05.962 INFO:tasks.workunit.client.0.smithi100.stdout:Skipping due to unmet REQUIRE 2024-04-07T17:39:05.962 INFO:tasks.workunit.client.0.smithi100.stdout:[ SKIPPED ] DiffIterateTest/1.DiffIterateStriping (0 ms) 2024-04-07T17:39:06.614 INFO:tasks.workunit.client.0.smithi100.stdout:[----------] 15 tests from DiffIterateTest/1 (69286 ms total) 2024-04-07T17:39:06.614 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:39:06.614 INFO:tasks.workunit.client.0.smithi100.stdout:[----------] Global test environment tear-down 2024-04-07T17:39:06.614 INFO:tasks.workunit.client.0.smithi100.stdout:[==========] 480 tests from 24 test suites ran. (2336836 ms total) 2024-04-07T17:39:06.614 INFO:tasks.workunit.client.0.smithi100.stdout:[ PASSED ] 434 tests. 2024-04-07T17:39:06.614 INFO:tasks.workunit.client.0.smithi100.stdout:[ SKIPPED ] 46 tests, listed below: 2024-04-07T17:39:06.614 INFO:tasks.workunit.client.0.smithi100.stdout:[ SKIPPED ] TestJournalStress.DiscardWithPruneWriteOverlap 2024-04-07T17:39:06.614 INFO:tasks.workunit.client.0.smithi100.stdout:[ SKIPPED ] TestJournalReplay.AioDiscardEvent 2024-04-07T17:39:06.614 INFO:tasks.workunit.client.0.smithi100.stdout:[ SKIPPED ] TestJournalReplay.AioWriteEvent 2024-04-07T17:39:06.614 INFO:tasks.workunit.client.0.smithi100.stdout:[ SKIPPED ] TestJournalReplay.AioFlushEvent 2024-04-07T17:39:06.615 INFO:tasks.workunit.client.0.smithi100.stdout:[ SKIPPED ] TestJournalReplay.SnapCreate 2024-04-07T17:39:06.615 INFO:tasks.workunit.client.0.smithi100.stdout:[ SKIPPED ] TestJournalReplay.SnapProtect 2024-04-07T17:39:06.615 INFO:tasks.workunit.client.0.smithi100.stdout:[ SKIPPED ] TestJournalReplay.SnapUnprotect 2024-04-07T17:39:06.615 INFO:tasks.workunit.client.0.smithi100.stdout:[ SKIPPED ] TestJournalReplay.SnapRename 2024-04-07T17:39:06.615 INFO:tasks.workunit.client.0.smithi100.stdout:[ SKIPPED ] TestJournalReplay.SnapRollback 2024-04-07T17:39:06.615 INFO:tasks.workunit.client.0.smithi100.stdout:[ SKIPPED ] TestJournalReplay.SnapRemove 2024-04-07T17:39:06.615 INFO:tasks.workunit.client.0.smithi100.stdout:[ SKIPPED ] TestJournalReplay.Rename 2024-04-07T17:39:06.615 INFO:tasks.workunit.client.0.smithi100.stdout:[ SKIPPED ] TestJournalReplay.Resize 2024-04-07T17:39:06.615 INFO:tasks.workunit.client.0.smithi100.stdout:[ SKIPPED ] TestJournalReplay.Flatten 2024-04-07T17:39:06.615 INFO:tasks.workunit.client.0.smithi100.stdout:[ SKIPPED ] TestJournalReplay.UpdateFeatures 2024-04-07T17:39:06.615 INFO:tasks.workunit.client.0.smithi100.stdout:[ SKIPPED ] TestJournalReplay.MetadataSet 2024-04-07T17:39:06.615 INFO:tasks.workunit.client.0.smithi100.stdout:[ SKIPPED ] TestJournalReplay.MetadataRemove 2024-04-07T17:39:06.615 INFO:tasks.workunit.client.0.smithi100.stdout:[ SKIPPED ] TestJournalReplay.ObjectPosition 2024-04-07T17:39:06.615 INFO:tasks.workunit.client.0.smithi100.stdout:[ SKIPPED ] TestJournalEntries.AioWrite 2024-04-07T17:39:06.616 INFO:tasks.workunit.client.0.smithi100.stdout:[ SKIPPED ] TestJournalEntries.AioDiscard 2024-04-07T17:39:06.616 INFO:tasks.workunit.client.0.smithi100.stdout:[ SKIPPED ] TestJournalEntries.AioDiscardWithPrune 2024-04-07T17:39:06.616 INFO:tasks.workunit.client.0.smithi100.stdout:[ SKIPPED ] TestJournalEntries.AioFlush 2024-04-07T17:39:06.616 INFO:tasks.workunit.client.0.smithi100.stdout:[ SKIPPED ] TestOperations.DisableJournalingCorrupt 2024-04-07T17:39:06.616 INFO:tasks.workunit.client.0.smithi100.stdout:[ SKIPPED ] TestMigration.MirroringSamePool 2024-04-07T17:39:06.616 INFO:tasks.workunit.client.0.smithi100.stdout:[ SKIPPED ] TestMigration.MirroringAbort 2024-04-07T17:39:06.616 INFO:tasks.workunit.client.0.smithi100.stdout:[ SKIPPED ] TestMigration.MirroringOtherPoolDisabled 2024-04-07T17:39:06.616 INFO:tasks.workunit.client.0.smithi100.stdout:[ SKIPPED ] TestMigration.MirroringOtherPoolEnabled 2024-04-07T17:39:06.616 INFO:tasks.workunit.client.0.smithi100.stdout:[ SKIPPED ] TestMigration.MirroringPool 2024-04-07T17:39:06.616 INFO:tasks.workunit.client.0.smithi100.stdout:[ SKIPPED ] TestDeepCopy.NoSnaps_StrippingLargerDstObjSize 2024-04-07T17:39:06.616 INFO:tasks.workunit.client.0.smithi100.stdout:[ SKIPPED ] TestDeepCopy.Snaps_StrippingLargerDstObjSize 2024-04-07T17:39:06.616 INFO:tasks.workunit.client.0.smithi100.stdout:[ SKIPPED ] TestDeepCopy.Clone_StrippingLargerDstObjSize 2024-04-07T17:39:06.616 INFO:tasks.workunit.client.0.smithi100.stdout:[ SKIPPED ] TestDeepCopy.CloneFlatten_StrippingLargerDstObjSize 2024-04-07T17:39:06.617 INFO:tasks.workunit.client.0.smithi100.stdout:[ SKIPPED ] TestDeepCopy.Stress_StrippingLargerDstObjSize 2024-04-07T17:39:06.617 INFO:tasks.workunit.client.0.smithi100.stdout:[ SKIPPED ] TestDeepCopy.NoSnaps_StrippingSmallerDstObjSize 2024-04-07T17:39:06.617 INFO:tasks.workunit.client.0.smithi100.stdout:[ SKIPPED ] TestDeepCopy.Snaps_StrippingSmallerDstObjSize 2024-04-07T17:39:06.617 INFO:tasks.workunit.client.0.smithi100.stdout:[ SKIPPED ] TestDeepCopy.Clone_StrippingSmallerDstObjSize 2024-04-07T17:39:06.617 INFO:tasks.workunit.client.0.smithi100.stdout:[ SKIPPED ] TestDeepCopy.CloneFlatten_StrippingSmallerDstObjSize 2024-04-07T17:39:06.617 INFO:tasks.workunit.client.0.smithi100.stdout:[ SKIPPED ] TestDeepCopy.Stress_StrippingSmallerDstObjSize 2024-04-07T17:39:06.617 INFO:tasks.workunit.client.0.smithi100.stdout:[ SKIPPED ] TestMirroring.DisableImageMirrorWithPeer 2024-04-07T17:39:06.617 INFO:tasks.workunit.client.0.smithi100.stdout:[ SKIPPED ] TestMirroring.DisableJournalingWithPeer 2024-04-07T17:39:06.617 INFO:tasks.workunit.client.0.smithi100.stdout:[ SKIPPED ] TestInternal.DiscardCopyup 2024-04-07T17:39:06.617 INFO:tasks.workunit.client.0.smithi100.stdout:[ SKIPPED ] TestInternal.TestCoR 2024-04-07T17:39:06.617 INFO:tasks.workunit.client.0.smithi100.stdout:[ SKIPPED ] TestLibRBD.EncryptionLoadBadStripePattern 2024-04-07T17:39:06.617 INFO:tasks.workunit.client.0.smithi100.stdout:[ SKIPPED ] TestLibRBD.EnableJournalingViaLockOwner 2024-04-07T17:39:06.617 INFO:tasks.workunit.client.0.smithi100.stdout:[ SKIPPED ] TestLibRBD.ExclusiveLockReadTransition 2024-04-07T17:39:06.617 INFO:tasks.workunit.client.0.smithi100.stdout:[ SKIPPED ] DiffIterateTest/0.DiffIterateStriping 2024-04-07T17:39:06.618 INFO:tasks.workunit.client.0.smithi100.stdout:[ SKIPPED ] DiffIterateTest/1.DiffIterateStriping 2024-04-07T17:39:06.618 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:39:06.618 INFO:tasks.workunit.client.0.smithi100.stdout: YOU HAVE 2 DISABLED TESTS 2024-04-07T17:39:06.618 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:39:06.696 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-07T17:39:06.696 DEBUG:teuthology.orchestra.run.smithi100:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-07T17:39:06.706 INFO:tasks.workunit:Stopping ['rbd/test_librbd.sh'] on client.0... 2024-04-07T17:39:06.706 DEBUG:teuthology.orchestra.run.smithi100:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-04-07T17:39:06.970 DEBUG:teuthology.parallel:result is None 2024-04-07T17:39:06.971 DEBUG:teuthology.orchestra.run.smithi100:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-04-07T17:39:06.983 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-04-07T17:39:06.984 DEBUG:teuthology.orchestra.run.smithi100:> rmdir -- /home/ubuntu/cephtest/mnt.0 2024-04-07T17:39:07.032 INFO:tasks.workunit:Deleted artificial mount point /home/ubuntu/cephtest/mnt.0/client.0 2024-04-07T17:39:07.032 INFO:teuthology.task.full_sequential:In full_sequential, running task print... 2024-04-07T17:39:07.032 INFO:teuthology.task.print:**** done end test_rbd_api.yaml 2024-04-07T17:39:07.032 INFO:teuthology.task.full_sequential:In full_sequential, running task print... 2024-04-07T17:39:07.032 INFO:teuthology.task.print:**** done start test_rbd_python.yaml 2024-04-07T17:39:07.032 INFO:teuthology.task.full_sequential:In full_sequential, running task workunit... 2024-04-07T17:39:07.033 INFO:tasks.workunit:Pulling workunits from ref reef 2024-04-07T17:39:07.033 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-04-07T17:39:07.033 DEBUG:teuthology.orchestra.run.smithi100:> stat -- /home/ubuntu/cephtest/mnt.0 2024-04-07T17:39:07.085 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-07T17:39:07.085 INFO:teuthology.orchestra.run.smithi100.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-04-07T17:39:07.085 DEBUG:teuthology.orchestra.run.smithi100:> mkdir -- /home/ubuntu/cephtest/mnt.0 2024-04-07T17:39:07.133 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2024-04-07T17:39:07.133 DEBUG:teuthology.orchestra.run.smithi100:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2024-04-07T17:39:07.185 INFO:tasks.workunit:timeout=3h 2024-04-07T17:39:07.185 INFO:tasks.workunit:cleanup=True 2024-04-07T17:39:07.185 DEBUG:teuthology.orchestra.run.smithi100:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone --depth 1 --branch reef https://github.com/ceph/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 2024-04-07T17:39:07.238 INFO:tasks.workunit.client.0.smithi100.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-04-07T17:39:07.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:39:07 smithi144 bash[66654]: cluster 2024-04-07T17:39:06.523412+0000 mgr.y (mgr.25175) 2512 : cluster [DBG] pgmap v4191: 137 pgs: 137 active+clean; 571 MiB data, 5.1 GiB used, 710 GiB / 715 GiB avail; 2.4 MiB/s rd, 6.9 MiB/s wr, 154 op/s 2024-04-07T17:39:07.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:39:07 smithi144 bash[66654]: cluster 2024-04-07T17:39:06.592918+0000 mon.a (mon.0) 4001 : cluster [DBG] osdmap e2105: 8 total, 8 up, 8 in 2024-04-07T17:39:07.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:39:07 smithi100 bash[74942]: cluster 2024-04-07T17:39:06.523412+0000 mgr.y (mgr.25175) 2512 : cluster [DBG] pgmap v4191: 137 pgs: 137 active+clean; 571 MiB data, 5.1 GiB used, 710 GiB / 715 GiB avail; 2.4 MiB/s rd, 6.9 MiB/s wr, 154 op/s 2024-04-07T17:39:07.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:39:07 smithi100 bash[74942]: cluster 2024-04-07T17:39:06.592918+0000 mon.a (mon.0) 4001 : cluster [DBG] osdmap e2105: 8 total, 8 up, 8 in 2024-04-07T17:39:07.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:39:07 smithi100 bash[76202]: cluster 2024-04-07T17:39:06.523412+0000 mgr.y (mgr.25175) 2512 : cluster [DBG] pgmap v4191: 137 pgs: 137 active+clean; 571 MiB data, 5.1 GiB used, 710 GiB / 715 GiB avail; 2.4 MiB/s rd, 6.9 MiB/s wr, 154 op/s 2024-04-07T17:39:07.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:39:07 smithi100 bash[76202]: cluster 2024-04-07T17:39:06.592918+0000 mon.a (mon.0) 4001 : cluster [DBG] osdmap e2105: 8 total, 8 up, 8 in 2024-04-07T17:39:08.704 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:39:08 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:39:08] "GET /metrics HTTP/1.1" 200 36758 "" "Prometheus/2.43.0" 2024-04-07T17:39:09.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:39:09 smithi144 bash[66654]: cluster 2024-04-07T17:39:08.524597+0000 mgr.y (mgr.25175) 2513 : cluster [DBG] pgmap v4193: 105 pgs: 105 active+clean; 4.6 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail 2024-04-07T17:39:09.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:39:09 smithi100 bash[74942]: cluster 2024-04-07T17:39:08.524597+0000 mgr.y (mgr.25175) 2513 : cluster [DBG] pgmap v4193: 105 pgs: 105 active+clean; 4.6 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail 2024-04-07T17:39:09.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:39:09 smithi100 bash[76202]: cluster 2024-04-07T17:39:08.524597+0000 mgr.y (mgr.25175) 2513 : cluster [DBG] pgmap v4193: 105 pgs: 105 active+clean; 4.6 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail 2024-04-07T17:39:11.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:39:11 smithi144 bash[66654]: cluster 2024-04-07T17:39:10.525609+0000 mgr.y (mgr.25175) 2514 : cluster [DBG] pgmap v4194: 105 pgs: 105 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail 2024-04-07T17:39:11.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:39:11 smithi100 bash[74942]: cluster 2024-04-07T17:39:10.525609+0000 mgr.y (mgr.25175) 2514 : cluster [DBG] pgmap v4194: 105 pgs: 105 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail 2024-04-07T17:39:11.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:39:11 smithi100 bash[76202]: cluster 2024-04-07T17:39:10.525609+0000 mgr.y (mgr.25175) 2514 : cluster [DBG] pgmap v4194: 105 pgs: 105 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail 2024-04-07T17:39:12.615 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:39:12 smithi144 bash[95233]: ts=2024-04-07T17:39:12.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:39:13.834 DEBUG:teuthology.orchestra.run.smithi100:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-04-07T17:39:13.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:39:13 smithi144 bash[66654]: cluster 2024-04-07T17:39:12.526741+0000 mgr.y (mgr.25175) 2515 : cluster [DBG] pgmap v4195: 105 pgs: 105 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail 2024-04-07T17:39:13.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:39:13 smithi144 bash[66654]: audit 2024-04-07T17:39:13.066900+0000 mon.a (mon.0) 4002 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:39:13.882 INFO:tasks.workunit.client.0.smithi100.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-04-07T17:39:13.884 INFO:tasks.workunit.client.0.smithi100.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-07T17:39:13.884 INFO:tasks.workunit.client.0.smithi100.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-04-07T17:39:13.937 INFO:tasks.workunit.client.0.smithi100.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-04-07T17:39:13.953 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:39:13 smithi100 bash[76202]: cluster 2024-04-07T17:39:12.526741+0000 mgr.y (mgr.25175) 2515 : cluster [DBG] pgmap v4195: 105 pgs: 105 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail 2024-04-07T17:39:13.953 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:39:13 smithi100 bash[76202]: audit 2024-04-07T17:39:13.066900+0000 mon.a (mon.0) 4002 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:39:13.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:39:13 smithi100 bash[74942]: cluster 2024-04-07T17:39:12.526741+0000 mgr.y (mgr.25175) 2515 : cluster [DBG] pgmap v4195: 105 pgs: 105 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail 2024-04-07T17:39:13.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:39:13 smithi100 bash[74942]: audit 2024-04-07T17:39:13.066900+0000 mon.a (mon.0) 4002 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:39:13.993 INFO:tasks.workunit.client.0.smithi100.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-04-07T17:39:14.037 INFO:tasks.workunit.client.0.smithi100.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-07T17:39:14.038 INFO:tasks.workunit.client.0.smithi100.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-07T17:39:14.038 INFO:tasks.workunit.client.0.smithi100.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-04-07T17:39:14.082 INFO:tasks.workunit.client.0.smithi100.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-07T17:39:14.086 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-07T17:39:14.086 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-04-07T17:39:14.132 INFO:tasks.workunit:Running workunits matching rbd/test_librbd_python.sh on client.0... 2024-04-07T17:39:14.134 INFO:tasks.workunit:Running workunit rbd/test_librbd_python.sh... 2024-04-07T17:39:14.134 DEBUG:teuthology.orchestra.run.smithi100:workunit test rbd/test_librbd_python.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=reef TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 RBD_FEATURES=61 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rbd/test_librbd_python.sh 2024-04-07T17:39:14.190 INFO:tasks.workunit.client.0.smithi100.stderr:+ dirname /home/ubuntu/cephtest/clone.client.0/qa/workunits/rbd/test_librbd_python.sh 2024-04-07T17:39:14.190 INFO:tasks.workunit.client.0.smithi100.stderr:+ relpath=/home/ubuntu/cephtest/clone.client.0/qa/workunits/rbd/../../../src/test/pybind 2024-04-07T17:39:14.190 INFO:tasks.workunit.client.0.smithi100.stderr:+ [ -n ] 2024-04-07T17:39:14.190 INFO:tasks.workunit.client.0.smithi100.stderr:+ python3 -m pytest -v /home/ubuntu/cephtest/clone.client.0/qa/workunits/rbd/../../../src/test/pybind/test_rbd.py 2024-04-07T17:39:14.344 INFO:tasks.workunit.client.0.smithi100.stdout:============================= test session starts ============================== 2024-04-07T17:39:14.344 INFO:tasks.workunit.client.0.smithi100.stdout:platform linux -- Python 3.10.6, pytest-6.2.5, py-1.10.0, pluggy-0.13.0 -- /usr/bin/python3 2024-04-07T17:39:14.345 INFO:tasks.workunit.client.0.smithi100.stdout:cachedir: .pytest_cache 2024-04-07T17:39:14.345 INFO:tasks.workunit.client.0.smithi100.stdout:rootdir: /home/ubuntu/cephtest/clone.client.0/src/test/pybind, configfile: pytest.ini 2024-04-07T17:39:14.739 INFO:tasks.workunit.client.0.smithi100.stdout:collecting ... collected 175 items 2024-04-07T17:39:14.739 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:39:14.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:39:14 smithi144 bash[66654]: audit 2024-04-07T17:39:13.798881+0000 mon.a (mon.0) 4003 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:39:14.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:39:14 smithi144 bash[66654]: audit 2024-04-07T17:39:13.800506+0000 mon.a (mon.0) 4004 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:39:14.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:39:14 smithi144 bash[66654]: audit 2024-04-07T17:39:13.812015+0000 mon.a (mon.0) 4005 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:39:14.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:39:14 smithi100 bash[74942]: audit 2024-04-07T17:39:13.798881+0000 mon.a (mon.0) 4003 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:39:14.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:39:14 smithi100 bash[74942]: audit 2024-04-07T17:39:13.800506+0000 mon.a (mon.0) 4004 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:39:14.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:39:14 smithi100 bash[74942]: audit 2024-04-07T17:39:13.812015+0000 mon.a (mon.0) 4005 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:39:14.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:39:14 smithi100 bash[76202]: audit 2024-04-07T17:39:13.798881+0000 mon.a (mon.0) 4003 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:39:14.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:39:14 smithi100 bash[76202]: audit 2024-04-07T17:39:13.800506+0000 mon.a (mon.0) 4004 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:39:14.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:39:14 smithi100 bash[76202]: audit 2024-04-07T17:39:13.812015+0000 mon.a (mon.0) 4005 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:39:16.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:39:15 smithi144 bash[66654]: cluster 2024-04-07T17:39:14.527716+0000 mgr.y (mgr.25175) 2516 : cluster [DBG] pgmap v4196: 105 pgs: 105 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail 2024-04-07T17:39:16.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:39:15 smithi144 bash[66654]: cluster 2024-04-07T17:39:15.620210+0000 mon.a (mon.0) 4006 : cluster [DBG] osdmap e2106: 8 total, 8 up, 8 in 2024-04-07T17:39:16.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:39:15 smithi144 bash[66654]: audit 2024-04-07T17:39:15.625809+0000 mon.c (mon.2) 101 : audit [INF] from='client.? 172.21.15.100:0/60836617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rbd-api-smithi100-170558-1","app": "rbd","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:39:16.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:39:15 smithi144 bash[66654]: audit 2024-04-07T17:39:15.630533+0000 mon.a (mon.0) 4007 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rbd-api-smithi100-170558-1","app": "rbd","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:39:16.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:39:15 smithi100 bash[76202]: cluster 2024-04-07T17:39:14.527716+0000 mgr.y (mgr.25175) 2516 : cluster [DBG] pgmap v4196: 105 pgs: 105 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail 2024-04-07T17:39:16.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:39:15 smithi100 bash[76202]: cluster 2024-04-07T17:39:15.620210+0000 mon.a (mon.0) 4006 : cluster [DBG] osdmap e2106: 8 total, 8 up, 8 in 2024-04-07T17:39:16.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:39:15 smithi100 bash[76202]: audit 2024-04-07T17:39:15.625809+0000 mon.c (mon.2) 101 : audit [INF] from='client.? 172.21.15.100:0/60836617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rbd-api-smithi100-170558-1","app": "rbd","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:39:16.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:39:15 smithi100 bash[76202]: audit 2024-04-07T17:39:15.630533+0000 mon.a (mon.0) 4007 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rbd-api-smithi100-170558-1","app": "rbd","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:39:16.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:39:15 smithi100 bash[74942]: cluster 2024-04-07T17:39:14.527716+0000 mgr.y (mgr.25175) 2516 : cluster [DBG] pgmap v4196: 105 pgs: 105 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail 2024-04-07T17:39:16.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:39:15 smithi100 bash[74942]: cluster 2024-04-07T17:39:15.620210+0000 mon.a (mon.0) 4006 : cluster [DBG] osdmap e2106: 8 total, 8 up, 8 in 2024-04-07T17:39:16.205 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:39:15 smithi100 bash[74942]: audit 2024-04-07T17:39:15.625809+0000 mon.c (mon.2) 101 : audit [INF] from='client.? 172.21.15.100:0/60836617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rbd-api-smithi100-170558-1","app": "rbd","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:39:16.205 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:39:15 smithi100 bash[74942]: audit 2024-04-07T17:39:15.630533+0000 mon.a (mon.0) 4007 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rbd-api-smithi100-170558-1","app": "rbd","yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:39:17.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:39:17 smithi100 bash[74942]: cluster 2024-04-07T17:39:16.528410+0000 mgr.y (mgr.25175) 2517 : cluster [DBG] pgmap v4198: 137 pgs: 32 unknown, 105 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail 2024-04-07T17:39:17.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:39:17 smithi100 bash[74942]: audit 2024-04-07T17:39:16.612616+0000 mon.a (mon.0) 4008 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rbd-api-smithi100-170558-1","app": "rbd","yes_i_really_mean_it": true}]': finished 2024-04-07T17:39:17.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:39:17 smithi100 bash[74942]: cluster 2024-04-07T17:39:16.619556+0000 mon.a (mon.0) 4009 : cluster [DBG] osdmap e2107: 8 total, 8 up, 8 in 2024-04-07T17:39:17.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:39:17 smithi100 bash[74942]: cluster 2024-04-07T17:39:16.812474+0000 mon.a (mon.0) 4010 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T17:39:17.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:39:17 smithi100 bash[74942]: audit 2024-04-07T17:39:16.867551+0000 mon.a (mon.0) 4011 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:39:17.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:39:17 smithi100 bash[74942]: audit 2024-04-07T17:39:16.869884+0000 mon.a (mon.0) 4012 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:39:17.955 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:39:17 smithi100 bash[76202]: cluster 2024-04-07T17:39:16.528410+0000 mgr.y (mgr.25175) 2517 : cluster [DBG] pgmap v4198: 137 pgs: 32 unknown, 105 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail 2024-04-07T17:39:17.955 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:39:17 smithi100 bash[76202]: audit 2024-04-07T17:39:16.612616+0000 mon.a (mon.0) 4008 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rbd-api-smithi100-170558-1","app": "rbd","yes_i_really_mean_it": true}]': finished 2024-04-07T17:39:17.955 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:39:17 smithi100 bash[76202]: cluster 2024-04-07T17:39:16.619556+0000 mon.a (mon.0) 4009 : cluster [DBG] osdmap e2107: 8 total, 8 up, 8 in 2024-04-07T17:39:17.955 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:39:17 smithi100 bash[76202]: cluster 2024-04-07T17:39:16.812474+0000 mon.a (mon.0) 4010 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T17:39:17.955 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:39:17 smithi100 bash[76202]: audit 2024-04-07T17:39:16.867551+0000 mon.a (mon.0) 4011 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:39:17.955 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:39:17 smithi100 bash[76202]: audit 2024-04-07T17:39:16.869884+0000 mon.a (mon.0) 4012 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:39:18.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:39:17 smithi144 bash[66654]: cluster 2024-04-07T17:39:16.528410+0000 mgr.y (mgr.25175) 2517 : cluster [DBG] pgmap v4198: 137 pgs: 32 unknown, 105 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail 2024-04-07T17:39:18.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:39:17 smithi144 bash[66654]: audit 2024-04-07T17:39:16.612616+0000 mon.a (mon.0) 4008 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rbd-api-smithi100-170558-1","app": "rbd","yes_i_really_mean_it": true}]': finished 2024-04-07T17:39:18.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:39:17 smithi144 bash[66654]: cluster 2024-04-07T17:39:16.619556+0000 mon.a (mon.0) 4009 : cluster [DBG] osdmap e2107: 8 total, 8 up, 8 in 2024-04-07T17:39:18.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:39:17 smithi144 bash[66654]: cluster 2024-04-07T17:39:16.812474+0000 mon.a (mon.0) 4010 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-07T17:39:18.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:39:17 smithi144 bash[66654]: audit 2024-04-07T17:39:16.867551+0000 mon.a (mon.0) 4011 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:39:18.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:39:17 smithi144 bash[66654]: audit 2024-04-07T17:39:16.869884+0000 mon.a (mon.0) 4012 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:39:18.666 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::test_version PASSED [ 0%] 2024-04-07T17:39:18.705 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:39:18 smithi100 bash[74942]: cluster 2024-04-07T17:39:17.623704+0000 mon.a (mon.0) 4013 : cluster [DBG] osdmap e2108: 8 total, 8 up, 8 in 2024-04-07T17:39:18.706 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:39:18 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:39:18] "GET /metrics HTTP/1.1" 200 36695 "" "Prometheus/2.43.0" 2024-04-07T17:39:18.706 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:39:18 smithi100 bash[76202]: cluster 2024-04-07T17:39:17.623704+0000 mon.a (mon.0) 4013 : cluster [DBG] osdmap e2108: 8 total, 8 up, 8 in 2024-04-07T17:39:18.839 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::test_create PASSED [ 1%] 2024-04-07T17:39:19.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:39:18 smithi144 bash[66654]: cluster 2024-04-07T17:39:17.623704+0000 mon.a (mon.0) 4013 : cluster [DBG] osdmap e2108: 8 total, 8 up, 8 in 2024-04-07T17:39:19.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:39:19 smithi100 bash[76202]: cluster 2024-04-07T17:39:18.529275+0000 mgr.y (mgr.25175) 2518 : cluster [DBG] pgmap v4201: 137 pgs: 12 unknown, 125 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail 2024-04-07T17:39:19.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:39:19 smithi100 bash[76202]: cluster 2024-04-07T17:39:18.626100+0000 mon.a (mon.0) 4014 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T17:39:19.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:39:19 smithi100 bash[76202]: cluster 2024-04-07T17:39:18.653456+0000 mon.a (mon.0) 4015 : cluster [DBG] osdmap e2109: 8 total, 8 up, 8 in 2024-04-07T17:39:19.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:39:19 smithi100 bash[74942]: cluster 2024-04-07T17:39:18.529275+0000 mgr.y (mgr.25175) 2518 : cluster [DBG] pgmap v4201: 137 pgs: 12 unknown, 125 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail 2024-04-07T17:39:19.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:39:19 smithi100 bash[74942]: cluster 2024-04-07T17:39:18.626100+0000 mon.a (mon.0) 4014 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T17:39:19.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:39:19 smithi100 bash[74942]: cluster 2024-04-07T17:39:18.653456+0000 mon.a (mon.0) 4015 : cluster [DBG] osdmap e2109: 8 total, 8 up, 8 in 2024-04-07T17:39:20.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:39:19 smithi144 bash[66654]: cluster 2024-04-07T17:39:18.529275+0000 mgr.y (mgr.25175) 2518 : cluster [DBG] pgmap v4201: 137 pgs: 12 unknown, 125 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail 2024-04-07T17:39:20.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:39:19 smithi144 bash[66654]: cluster 2024-04-07T17:39:18.626100+0000 mon.a (mon.0) 4014 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-07T17:39:20.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:39:19 smithi144 bash[66654]: cluster 2024-04-07T17:39:18.653456+0000 mon.a (mon.0) 4015 : cluster [DBG] osdmap e2109: 8 total, 8 up, 8 in 2024-04-07T17:39:20.297 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::test_create_defaults PASSED [ 1%] 2024-04-07T17:39:20.517 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::test_context_manager PASSED [ 2%] 2024-04-07T17:39:21.919 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::test_open_read_only PASSED [ 2%] 2024-04-07T17:39:21.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:39:21 smithi100 bash[74942]: cluster 2024-04-07T17:39:20.530236+0000 mgr.y (mgr.25175) 2519 : cluster [DBG] pgmap v4203: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 6.7 KiB/s rd, 2.2 KiB/s wr, 15 op/s 2024-04-07T17:39:21.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:39:21 smithi100 bash[74942]: cluster 2024-04-07T17:39:20.658808+0000 mon.a (mon.0) 4016 : cluster [DBG] osdmap e2110: 8 total, 8 up, 8 in 2024-04-07T17:39:21.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:39:21 smithi100 bash[76202]: cluster 2024-04-07T17:39:20.530236+0000 mgr.y (mgr.25175) 2519 : cluster [DBG] pgmap v4203: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 6.7 KiB/s rd, 2.2 KiB/s wr, 15 op/s 2024-04-07T17:39:21.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:39:21 smithi100 bash[76202]: cluster 2024-04-07T17:39:20.658808+0000 mon.a (mon.0) 4016 : cluster [DBG] osdmap e2110: 8 total, 8 up, 8 in 2024-04-07T17:39:22.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:39:21 smithi144 bash[66654]: cluster 2024-04-07T17:39:20.530236+0000 mgr.y (mgr.25175) 2519 : cluster [DBG] pgmap v4203: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 6.7 KiB/s rd, 2.2 KiB/s wr, 15 op/s 2024-04-07T17:39:22.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:39:21 smithi144 bash[66654]: cluster 2024-04-07T17:39:20.658808+0000 mon.a (mon.0) 4016 : cluster [DBG] osdmap e2110: 8 total, 8 up, 8 in 2024-04-07T17:39:22.615 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:39:22 smithi144 bash[95233]: ts=2024-04-07T17:39:22.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:39:22.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:39:22 smithi100 bash[74942]: cluster 2024-04-07T17:39:21.670530+0000 mon.a (mon.0) 4017 : cluster [DBG] osdmap e2111: 8 total, 8 up, 8 in 2024-04-07T17:39:22.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:39:22 smithi100 bash[76202]: cluster 2024-04-07T17:39:21.670530+0000 mon.a (mon.0) 4017 : cluster [DBG] osdmap e2111: 8 total, 8 up, 8 in 2024-04-07T17:39:23.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:39:22 smithi144 bash[66654]: cluster 2024-04-07T17:39:21.670530+0000 mon.a (mon.0) 4017 : cluster [DBG] osdmap e2111: 8 total, 8 up, 8 in 2024-04-07T17:39:23.306 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::test_open_dne PASSED [ 3%] 2024-04-07T17:39:23.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:39:23 smithi100 bash[74942]: cluster 2024-04-07T17:39:22.531609+0000 mgr.y (mgr.25175) 2520 : cluster [DBG] pgmap v4206: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 167 KiB/s rd, 10 KiB/s wr, 226 op/s 2024-04-07T17:39:23.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:39:23 smithi100 bash[76202]: cluster 2024-04-07T17:39:22.531609+0000 mgr.y (mgr.25175) 2520 : cluster [DBG] pgmap v4206: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 167 KiB/s rd, 10 KiB/s wr, 226 op/s 2024-04-07T17:39:24.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:39:23 smithi144 bash[66654]: cluster 2024-04-07T17:39:22.531609+0000 mgr.y (mgr.25175) 2520 : cluster [DBG] pgmap v4206: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 167 KiB/s rd, 10 KiB/s wr, 226 op/s 2024-04-07T17:39:24.660 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::test_open_readonly_dne PASSED [ 4%] 2024-04-07T17:39:24.898 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::test_open_by_id PASSED [ 4%] 2024-04-07T17:39:25.060 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::test_aio_open PASSED [ 5%] 2024-04-07T17:39:25.082 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::test_remove_dne PASSED [ 5%] 2024-04-07T17:39:25.085 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::test_list_empty PASSED [ 6%] 2024-04-07T17:39:25.276 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::test_list PASSED [ 6%] 2024-04-07T17:39:25.447 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::test_remove_with_progress PASSED [ 7%] 2024-04-07T17:39:25.681 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::test_remove_canceled PASSED [ 8%] 2024-04-07T17:39:25.869 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::test_remove_with_progress_except PASSED [ 8%] 2024-04-07T17:39:25.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:39:25 smithi100 bash[74942]: cluster 2024-04-07T17:39:24.532848+0000 mgr.y (mgr.25175) 2521 : cluster [DBG] pgmap v4207: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 243 KiB/s rd, 20 KiB/s wr, 333 op/s 2024-04-07T17:39:25.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:39:25 smithi100 bash[76202]: cluster 2024-04-07T17:39:24.532848+0000 mgr.y (mgr.25175) 2521 : cluster [DBG] pgmap v4207: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 243 KiB/s rd, 20 KiB/s wr, 333 op/s 2024-04-07T17:39:26.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:39:25 smithi144 bash[66654]: cluster 2024-04-07T17:39:24.532848+0000 mgr.y (mgr.25175) 2521 : cluster [DBG] pgmap v4207: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 243 KiB/s rd, 20 KiB/s wr, 333 op/s 2024-04-07T17:39:26.178 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::test_rename PASSED [ 9%] 2024-04-07T17:39:26.937 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::test_pool_metadata PASSED [ 9%] 2024-04-07T17:39:26.979 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::test_config_list PASSED [ 10%] 2024-04-07T17:39:27.005 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::test_pool_config_set_and_get_and_remove PASSED [ 10%] 2024-04-07T17:39:27.058 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::test_namespaces PASSED [ 11%] 2024-04-07T17:39:27.454 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:39:26 smithi100 bash[68137]: [07/Apr/2024:17:39:26] ENGINE Bus STOPPING 2024-04-07T17:39:27.454 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:39:26 smithi100 bash[68137]: [07/Apr/2024:17:39:26] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-04-07T17:39:27.454 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:39:26 smithi100 bash[68137]: [07/Apr/2024:17:39:26] ENGINE Bus STOPPED 2024-04-07T17:39:27.454 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:39:26 smithi100 bash[68137]: [07/Apr/2024:17:39:26] ENGINE Bus STARTING 2024-04-07T17:39:27.454 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:39:27 smithi100 bash[68137]: [07/Apr/2024:17:39:27] ENGINE Serving on http://:::9283 2024-04-07T17:39:27.454 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:39:27 smithi100 bash[68137]: [07/Apr/2024:17:39:27] ENGINE Bus STARTED 2024-04-07T17:39:28.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:39:27 smithi144 bash[66654]: cluster 2024-04-07T17:39:26.533911+0000 mgr.y (mgr.25175) 2522 : cluster [DBG] pgmap v4208: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 185 KiB/s rd, 15 KiB/s wr, 254 op/s 2024-04-07T17:39:28.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:39:27 smithi144 bash[66654]: audit 2024-04-07T17:39:26.958860+0000 mon.a (mon.0) 4018 : audit [INF] from='client.? ' entity='client.admin' 2024-04-07T17:39:28.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:39:27 smithi144 bash[66654]: audit 2024-04-07T17:39:27.007662+0000 mon.a (mon.0) 4019 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:39:28.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:39:27 smithi144 bash[66654]: audit 2024-04-07T17:39:27.029826+0000 mon.a (mon.0) 4020 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config-key dump", "format": "json"}]: dispatch 2024-04-07T17:39:28.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:39:27 smithi144 bash[66654]: audit 2024-04-07T17:39:27.035258+0000 mon.a (mon.0) 4021 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:39:28.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:39:27 smithi144 bash[66654]: audit 2024-04-07T17:39:27.038248+0000 mon.a (mon.0) 4022 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:39:28.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:39:27 smithi144 bash[66654]: audit 2024-04-07T17:39:27.039947+0000 mon.a (mon.0) 4023 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:39:28.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:39:27 smithi144 bash[66654]: audit 2024-04-07T17:39:27.048120+0000 mon.a (mon.0) 4024 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:39:28.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:39:27 smithi144 bash[66654]: audit 2024-04-07T17:39:27.434373+0000 mon.a (mon.0) 4025 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:39:28.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:39:27 smithi144 bash[66654]: audit 2024-04-07T17:39:27.434637+0000 mon.a (mon.0) 4026 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:39:28.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:39:27 smithi144 bash[66654]: audit 2024-04-07T17:39:27.434879+0000 mon.a (mon.0) 4027 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:39:28.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:39:27 smithi144 bash[66654]: audit 2024-04-07T17:39:27.437565+0000 mon.a (mon.0) 4028 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:39:28.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:39:27 smithi144 bash[66654]: audit 2024-04-07T17:39:27.437749+0000 mon.a (mon.0) 4029 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:39:28.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:39:27 smithi144 bash[66654]: audit 2024-04-07T17:39:27.438004+0000 mon.a (mon.0) 4030 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:39:28.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:39:27 smithi144 bash[66654]: audit 2024-04-07T17:39:27.439859+0000 mon.a (mon.0) 4031 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:39:28.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:39:27 smithi144 bash[66654]: audit 2024-04-07T17:39:27.440063+0000 mon.a (mon.0) 4032 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:39:28.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:39:27 smithi144 bash[66654]: audit 2024-04-07T17:39:27.440305+0000 mon.a (mon.0) 4033 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:39:28.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:39:27 smithi144 bash[66654]: audit 2024-04-07T17:39:27.451773+0000 mon.a (mon.0) 4034 : audit [INF] from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T17:39:28.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:39:27 smithi144 bash[66654]: audit 2024-04-07T17:39:27.451929+0000 mon.a (mon.0) 4035 : audit [INF] from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T17:39:28.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:39:27 smithi144 bash[66654]: audit 2024-04-07T17:39:27.452174+0000 mon.a (mon.0) 4036 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:39:28.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:39:27 smithi144 bash[66654]: audit 2024-04-07T17:39:27.452372+0000 mon.a (mon.0) 4037 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:39:28.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:39:27 smithi144 bash[66654]: audit 2024-04-07T17:39:27.452590+0000 mon.a (mon.0) 4038 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:39:28.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:39:27 smithi144 bash[66654]: audit 2024-04-07T17:39:27.455042+0000 mon.a (mon.0) 4039 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:39:28.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:39:27 smithi144 bash[66654]: audit 2024-04-07T17:39:27.455207+0000 mon.a (mon.0) 4040 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:39:28.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:39:27 smithi144 bash[66654]: audit 2024-04-07T17:39:27.455619+0000 mon.a (mon.0) 4041 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:39:28.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:39:27 smithi144 bash[66654]: audit 2024-04-07T17:39:27.457196+0000 mon.c (mon.2) 102 : audit [INF] from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T17:39:28.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:39:27 smithi144 bash[66654]: audit 2024-04-07T17:39:27.457267+0000 mon.c (mon.2) 103 : audit [INF] from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T17:39:28.367 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:39:27 smithi144 bash[66654]: audit 2024-04-07T17:39:27.457327+0000 mon.b (mon.1) 21 : audit [INF] from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T17:39:28.368 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:39:27 smithi144 bash[66654]: audit 2024-04-07T17:39:27.457487+0000 mon.b (mon.1) 22 : audit [INF] from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T17:39:28.368 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:39:27 smithi144 bash[66654]: audit 2024-04-07T17:39:27.457558+0000 mon.a (mon.0) 4042 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:39:28.368 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:39:27 smithi144 bash[66654]: audit 2024-04-07T17:39:27.457952+0000 mon.a (mon.0) 4043 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:39:28.368 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:39:27 smithi144 bash[66654]: audit 2024-04-07T17:39:27.458168+0000 mon.a (mon.0) 4044 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:39:28.429 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:39:27 smithi100 bash[76202]: cluster 2024-04-07T17:39:26.533911+0000 mgr.y (mgr.25175) 2522 : cluster [DBG] pgmap v4208: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 185 KiB/s rd, 15 KiB/s wr, 254 op/s 2024-04-07T17:39:28.429 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:39:27 smithi100 bash[76202]: audit 2024-04-07T17:39:26.958860+0000 mon.a (mon.0) 4018 : audit [INF] from='client.? ' entity='client.admin' 2024-04-07T17:39:28.429 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:39:27 smithi100 bash[76202]: audit 2024-04-07T17:39:27.007662+0000 mon.a (mon.0) 4019 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:39:28.429 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:39:27 smithi100 bash[76202]: audit 2024-04-07T17:39:27.029826+0000 mon.a (mon.0) 4020 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config-key dump", "format": "json"}]: dispatch 2024-04-07T17:39:28.429 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:39:27 smithi100 bash[76202]: audit 2024-04-07T17:39:27.035258+0000 mon.a (mon.0) 4021 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:39:28.429 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:39:27 smithi100 bash[76202]: audit 2024-04-07T17:39:27.038248+0000 mon.a (mon.0) 4022 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:39:28.429 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:39:27 smithi100 bash[76202]: audit 2024-04-07T17:39:27.039947+0000 mon.a (mon.0) 4023 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:39:28.429 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:39:27 smithi100 bash[76202]: audit 2024-04-07T17:39:27.048120+0000 mon.a (mon.0) 4024 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:39:28.429 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:39:27 smithi100 bash[76202]: audit 2024-04-07T17:39:27.434373+0000 mon.a (mon.0) 4025 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:39:28.429 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:39:27 smithi100 bash[76202]: audit 2024-04-07T17:39:27.434637+0000 mon.a (mon.0) 4026 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:39:28.429 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:39:27 smithi100 bash[76202]: audit 2024-04-07T17:39:27.434879+0000 mon.a (mon.0) 4027 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:39:28.429 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:39:27 smithi100 bash[76202]: audit 2024-04-07T17:39:27.437565+0000 mon.a (mon.0) 4028 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:39:28.429 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:39:27 smithi100 bash[76202]: audit 2024-04-07T17:39:27.437749+0000 mon.a (mon.0) 4029 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:39:28.430 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:39:27 smithi100 bash[76202]: audit 2024-04-07T17:39:27.438004+0000 mon.a (mon.0) 4030 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:39:28.430 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:39:27 smithi100 bash[76202]: audit 2024-04-07T17:39:27.439859+0000 mon.a (mon.0) 4031 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:39:28.430 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:39:27 smithi100 bash[76202]: audit 2024-04-07T17:39:27.440063+0000 mon.a (mon.0) 4032 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:39:28.430 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:39:27 smithi100 bash[76202]: audit 2024-04-07T17:39:27.440305+0000 mon.a (mon.0) 4033 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:39:28.430 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:39:27 smithi100 bash[76202]: audit 2024-04-07T17:39:27.451773+0000 mon.a (mon.0) 4034 : audit [INF] from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T17:39:28.430 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:39:27 smithi100 bash[76202]: audit 2024-04-07T17:39:27.451929+0000 mon.a (mon.0) 4035 : audit [INF] from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T17:39:28.430 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:39:27 smithi100 bash[74942]: cluster 2024-04-07T17:39:26.533911+0000 mgr.y (mgr.25175) 2522 : cluster [DBG] pgmap v4208: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 185 KiB/s rd, 15 KiB/s wr, 254 op/s 2024-04-07T17:39:28.430 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:39:27 smithi100 bash[74942]: audit 2024-04-07T17:39:26.958860+0000 mon.a (mon.0) 4018 : audit [INF] from='client.? ' entity='client.admin' 2024-04-07T17:39:28.431 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:39:27 smithi100 bash[74942]: audit 2024-04-07T17:39:27.007662+0000 mon.a (mon.0) 4019 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:39:28.431 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:39:27 smithi100 bash[74942]: audit 2024-04-07T17:39:27.029826+0000 mon.a (mon.0) 4020 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config-key dump", "format": "json"}]: dispatch 2024-04-07T17:39:28.431 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:39:27 smithi100 bash[74942]: audit 2024-04-07T17:39:27.035258+0000 mon.a (mon.0) 4021 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:39:28.431 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:39:27 smithi100 bash[74942]: audit 2024-04-07T17:39:27.038248+0000 mon.a (mon.0) 4022 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:39:28.431 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:39:27 smithi100 bash[74942]: audit 2024-04-07T17:39:27.039947+0000 mon.a (mon.0) 4023 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:39:28.431 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:39:27 smithi100 bash[74942]: audit 2024-04-07T17:39:27.048120+0000 mon.a (mon.0) 4024 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:39:28.431 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:39:27 smithi100 bash[74942]: audit 2024-04-07T17:39:27.434373+0000 mon.a (mon.0) 4025 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:39:28.431 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:39:27 smithi100 bash[74942]: audit 2024-04-07T17:39:27.434637+0000 mon.a (mon.0) 4026 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:39:28.431 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:39:27 smithi100 bash[74942]: audit 2024-04-07T17:39:27.434879+0000 mon.a (mon.0) 4027 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:39:28.431 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:39:27 smithi100 bash[74942]: audit 2024-04-07T17:39:27.437565+0000 mon.a (mon.0) 4028 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:39:28.431 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:39:27 smithi100 bash[74942]: audit 2024-04-07T17:39:27.437749+0000 mon.a (mon.0) 4029 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:39:28.431 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:39:27 smithi100 bash[74942]: audit 2024-04-07T17:39:27.438004+0000 mon.a (mon.0) 4030 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:39:28.431 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:39:27 smithi100 bash[74942]: audit 2024-04-07T17:39:27.439859+0000 mon.a (mon.0) 4031 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:39:28.431 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:39:27 smithi100 bash[74942]: audit 2024-04-07T17:39:27.440063+0000 mon.a (mon.0) 4032 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:39:28.431 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:39:27 smithi100 bash[74942]: audit 2024-04-07T17:39:27.440305+0000 mon.a (mon.0) 4033 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:39:28.432 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:39:27 smithi100 bash[74942]: audit 2024-04-07T17:39:27.451773+0000 mon.a (mon.0) 4034 : audit [INF] from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T17:39:28.432 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:39:27 smithi100 bash[74942]: audit 2024-04-07T17:39:27.451929+0000 mon.a (mon.0) 4035 : audit [INF] from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T17:39:28.432 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:39:27 smithi100 bash[74942]: audit 2024-04-07T17:39:27.452174+0000 mon.a (mon.0) 4036 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:39:28.432 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:39:27 smithi100 bash[74942]: audit 2024-04-07T17:39:27.452372+0000 mon.a (mon.0) 4037 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:39:28.432 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:39:27 smithi100 bash[74942]: audit 2024-04-07T17:39:27.452590+0000 mon.a (mon.0) 4038 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:39:28.432 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:39:27 smithi100 bash[74942]: audit 2024-04-07T17:39:27.455042+0000 mon.a (mon.0) 4039 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:39:28.432 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:39:27 smithi100 bash[74942]: audit 2024-04-07T17:39:27.455207+0000 mon.a (mon.0) 4040 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:39:28.432 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:39:27 smithi100 bash[74942]: audit 2024-04-07T17:39:27.455619+0000 mon.a (mon.0) 4041 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:39:28.432 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:39:27 smithi100 bash[74942]: audit 2024-04-07T17:39:27.457196+0000 mon.c (mon.2) 102 : audit [INF] from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T17:39:28.432 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:39:27 smithi100 bash[74942]: audit 2024-04-07T17:39:27.457267+0000 mon.c (mon.2) 103 : audit [INF] from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T17:39:28.432 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:39:27 smithi100 bash[74942]: audit 2024-04-07T17:39:27.457327+0000 mon.b (mon.1) 21 : audit [INF] from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T17:39:28.432 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:39:27 smithi100 bash[74942]: audit 2024-04-07T17:39:27.457487+0000 mon.b (mon.1) 22 : audit [INF] from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T17:39:28.432 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:39:27 smithi100 bash[74942]: audit 2024-04-07T17:39:27.457558+0000 mon.a (mon.0) 4042 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:39:28.432 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:39:27 smithi100 bash[74942]: audit 2024-04-07T17:39:27.457952+0000 mon.a (mon.0) 4043 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:39:28.432 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:39:27 smithi100 bash[74942]: audit 2024-04-07T17:39:27.458168+0000 mon.a (mon.0) 4044 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:39:28.433 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:39:27 smithi100 bash[76202]: audit 2024-04-07T17:39:27.452174+0000 mon.a (mon.0) 4036 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:39:28.433 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:39:27 smithi100 bash[76202]: audit 2024-04-07T17:39:27.452372+0000 mon.a (mon.0) 4037 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:39:28.433 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:39:27 smithi100 bash[76202]: audit 2024-04-07T17:39:27.452590+0000 mon.a (mon.0) 4038 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:39:28.433 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:39:27 smithi100 bash[76202]: audit 2024-04-07T17:39:27.455042+0000 mon.a (mon.0) 4039 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:39:28.433 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:39:27 smithi100 bash[76202]: audit 2024-04-07T17:39:27.455207+0000 mon.a (mon.0) 4040 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:39:28.433 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:39:27 smithi100 bash[76202]: audit 2024-04-07T17:39:27.455619+0000 mon.a (mon.0) 4041 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:39:28.433 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:39:27 smithi100 bash[76202]: audit 2024-04-07T17:39:27.457196+0000 mon.c (mon.2) 102 : audit [INF] from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T17:39:28.433 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:39:27 smithi100 bash[76202]: audit 2024-04-07T17:39:27.457267+0000 mon.c (mon.2) 103 : audit [INF] from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T17:39:28.433 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:39:27 smithi100 bash[76202]: audit 2024-04-07T17:39:27.457327+0000 mon.b (mon.1) 21 : audit [INF] from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T17:39:28.433 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:39:27 smithi100 bash[76202]: audit 2024-04-07T17:39:27.457487+0000 mon.b (mon.1) 22 : audit [INF] from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T17:39:28.433 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:39:27 smithi100 bash[76202]: audit 2024-04-07T17:39:27.457558+0000 mon.a (mon.0) 4042 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:39:28.433 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:39:27 smithi100 bash[76202]: audit 2024-04-07T17:39:27.457952+0000 mon.a (mon.0) 4043 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:39:28.433 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:39:27 smithi100 bash[76202]: audit 2024-04-07T17:39:27.458168+0000 mon.a (mon.0) 4044 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:39:28.704 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:39:28 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:39:28] "GET /metrics HTTP/1.1" 200 36695 "" "Prometheus/2.43.0" 2024-04-07T17:39:29.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:39:29 smithi144 bash[66654]: cluster 2024-04-07T17:39:28.055723+0000 mon.a (mon.0) 4045 : cluster [DBG] osdmap e2112: 8 total, 8 up, 8 in 2024-04-07T17:39:29.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:39:29 smithi144 bash[66654]: cluster 2024-04-07T17:39:28.241033+0000 mon.a (mon.0) 4046 : cluster [DBG] osdmap e2113: 8 total, 8 up, 8 in 2024-04-07T17:39:29.448 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::test_pool_stats PASSED [ 12%] 2024-04-07T17:39:29.451 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::test_features_to_string PASSED [ 12%] 2024-04-07T17:39:29.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:39:29 smithi100 bash[76202]: cluster 2024-04-07T17:39:28.055723+0000 mon.a (mon.0) 4045 : cluster [DBG] osdmap e2112: 8 total, 8 up, 8 in 2024-04-07T17:39:29.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:39:29 smithi100 bash[76202]: cluster 2024-04-07T17:39:28.241033+0000 mon.a (mon.0) 4046 : cluster [DBG] osdmap e2113: 8 total, 8 up, 8 in 2024-04-07T17:39:29.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:39:29 smithi100 bash[74942]: cluster 2024-04-07T17:39:28.055723+0000 mon.a (mon.0) 4045 : cluster [DBG] osdmap e2112: 8 total, 8 up, 8 in 2024-04-07T17:39:29.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:39:29 smithi100 bash[74942]: cluster 2024-04-07T17:39:28.241033+0000 mon.a (mon.0) 4046 : cluster [DBG] osdmap e2113: 8 total, 8 up, 8 in 2024-04-07T17:39:29.455 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::test_features_from_string PASSED [ 13%] 2024-04-07T17:39:29.662 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_update_features SKIPPED [ 13%] 2024-04-07T17:39:29.865 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_create_with_params SKIPPED [ 14%] 2024-04-07T17:39:30.083 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_id PASSED [ 14%] 2024-04-07T17:39:30.299 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_block_name_prefix PASSED [ 15%] 2024-04-07T17:39:30.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:39:30 smithi144 bash[66654]: cluster 2024-04-07T17:39:28.534836+0000 mgr.y (mgr.25175) 2523 : cluster [DBG] pgmap v4211: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 980 KiB/s rd, 39 KiB/s wr, 480 op/s 2024-04-07T17:39:30.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:39:30 smithi144 bash[66654]: cluster 2024-04-07T17:39:29.250906+0000 mon.a (mon.0) 4047 : cluster [DBG] osdmap e2114: 8 total, 8 up, 8 in 2024-04-07T17:39:30.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:39:30 smithi100 bash[74942]: cluster 2024-04-07T17:39:28.534836+0000 mgr.y (mgr.25175) 2523 : cluster [DBG] pgmap v4211: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 980 KiB/s rd, 39 KiB/s wr, 480 op/s 2024-04-07T17:39:30.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:39:30 smithi100 bash[74942]: cluster 2024-04-07T17:39:29.250906+0000 mon.a (mon.0) 4047 : cluster [DBG] osdmap e2114: 8 total, 8 up, 8 in 2024-04-07T17:39:30.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:39:30 smithi100 bash[76202]: cluster 2024-04-07T17:39:28.534836+0000 mgr.y (mgr.25175) 2523 : cluster [DBG] pgmap v4211: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 980 KiB/s rd, 39 KiB/s wr, 480 op/s 2024-04-07T17:39:30.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:39:30 smithi100 bash[76202]: cluster 2024-04-07T17:39:29.250906+0000 mon.a (mon.0) 4047 : cluster [DBG] osdmap e2114: 8 total, 8 up, 8 in 2024-04-07T17:39:30.508 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_data_pool_id PASSED [ 16%] 2024-04-07T17:39:30.714 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_create_timestamp PASSED [ 16%] 2024-04-07T17:39:30.916 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_access_timestamp PASSED [ 17%] 2024-04-07T17:39:31.127 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_modify_timestamp PASSED [ 17%] 2024-04-07T17:39:31.371 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_invalidate_cache PASSED [ 18%] 2024-04-07T17:39:31.578 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_stat PASSED [ 18%] 2024-04-07T17:39:31.802 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_flags PASSED [ 19%] 2024-04-07T17:39:32.043 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_image_auto_close PASSED [ 20%] 2024-04-07T17:39:32.258 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_use_after_close PASSED [ 20%] 2024-04-07T17:39:32.317 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:39:32 smithi144 bash[66654]: cluster 2024-04-07T17:39:30.536058+0000 mgr.y (mgr.25175) 2524 : cluster [DBG] pgmap v4213: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 966 KiB/s rd, 37 KiB/s wr, 354 op/s 2024-04-07T17:39:32.317 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:39:32 smithi144 bash[66654]: audit 2024-04-07T17:39:31.869443+0000 mon.a (mon.0) 4048 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:39:32.317 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:39:32 smithi144 bash[66654]: audit 2024-04-07T17:39:31.872683+0000 mon.a (mon.0) 4049 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:39:32.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:39:32 smithi100 bash[74942]: cluster 2024-04-07T17:39:30.536058+0000 mgr.y (mgr.25175) 2524 : cluster [DBG] pgmap v4213: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 966 KiB/s rd, 37 KiB/s wr, 354 op/s 2024-04-07T17:39:32.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:39:32 smithi100 bash[74942]: audit 2024-04-07T17:39:31.869443+0000 mon.a (mon.0) 4048 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:39:32.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:39:32 smithi100 bash[74942]: audit 2024-04-07T17:39:31.872683+0000 mon.a (mon.0) 4049 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:39:32.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:39:32 smithi100 bash[76202]: cluster 2024-04-07T17:39:30.536058+0000 mgr.y (mgr.25175) 2524 : cluster [DBG] pgmap v4213: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 966 KiB/s rd, 37 KiB/s wr, 354 op/s 2024-04-07T17:39:32.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:39:32 smithi100 bash[76202]: audit 2024-04-07T17:39:31.869443+0000 mon.a (mon.0) 4048 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:39:32.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:39:32 smithi100 bash[76202]: audit 2024-04-07T17:39:31.872683+0000 mon.a (mon.0) 4049 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:39:32.503 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_write PASSED [ 21%] 2024-04-07T17:39:32.615 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:39:32 smithi144 bash[95233]: ts=2024-04-07T17:39:32.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:39:32.755 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_write_with_fadvise_flags PASSED [ 21%] 2024-04-07T17:39:32.999 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_write_zeroes PASSED [ 22%] 2024-04-07T17:39:33.249 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_write_zeroes_thick_provision PASSED [ 22%] 2024-04-07T17:39:33.464 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_read PASSED [ 23%] 2024-04-07T17:39:33.668 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_read_with_fadvise_flags PASSED [ 24%] 2024-04-07T17:39:33.992 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_large_write PASSED [ 24%] 2024-04-07T17:39:34.210 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_large_read PASSED [ 25%] 2024-04-07T17:39:34.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:39:34 smithi144 bash[66654]: cluster 2024-04-07T17:39:32.537408+0000 mgr.y (mgr.25175) 2525 : cluster [DBG] pgmap v4214: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 1.2 MiB/s rd, 42 KiB/s wr, 639 op/s 2024-04-07T17:39:34.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:39:34 smithi100 bash[74942]: cluster 2024-04-07T17:39:32.537408+0000 mgr.y (mgr.25175) 2525 : cluster [DBG] pgmap v4214: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 1.2 MiB/s rd, 42 KiB/s wr, 639 op/s 2024-04-07T17:39:34.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:39:34 smithi100 bash[76202]: cluster 2024-04-07T17:39:32.537408+0000 mgr.y (mgr.25175) 2525 : cluster [DBG] pgmap v4214: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 1.2 MiB/s rd, 42 KiB/s wr, 639 op/s 2024-04-07T17:39:34.465 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_write_read PASSED [ 25%] 2024-04-07T17:39:34.674 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_read_bad_offset PASSED [ 26%] 2024-04-07T17:39:34.923 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_resize PASSED [ 26%] 2024-04-07T17:39:35.176 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_resize_allow_shrink_False PASSED [ 27%] 2024-04-07T17:39:36.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:39:36 smithi144 bash[66654]: cluster 2024-04-07T17:39:34.538558+0000 mgr.y (mgr.25175) 2526 : cluster [DBG] pgmap v4215: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 443 KiB/s rd, 32 KiB/s wr, 613 op/s 2024-04-07T17:39:36.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:39:36 smithi100 bash[74942]: cluster 2024-04-07T17:39:34.538558+0000 mgr.y (mgr.25175) 2526 : cluster [DBG] pgmap v4215: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 443 KiB/s rd, 32 KiB/s wr, 613 op/s 2024-04-07T17:39:36.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:39:36 smithi100 bash[76202]: cluster 2024-04-07T17:39:34.538558+0000 mgr.y (mgr.25175) 2526 : cluster [DBG] pgmap v4215: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 443 KiB/s rd, 32 KiB/s wr, 613 op/s 2024-04-07T17:39:37.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:39:37 smithi144 bash[66654]: cluster 2024-04-07T17:39:36.073424+0000 mon.a (mon.0) 4050 : cluster [DBG] osdmap e2115: 8 total, 8 up, 8 in 2024-04-07T17:39:37.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:39:37 smithi100 bash[74942]: cluster 2024-04-07T17:39:36.073424+0000 mon.a (mon.0) 4050 : cluster [DBG] osdmap e2115: 8 total, 8 up, 8 in 2024-04-07T17:39:37.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:39:37 smithi100 bash[76202]: cluster 2024-04-07T17:39:36.073424+0000 mon.a (mon.0) 4050 : cluster [DBG] osdmap e2115: 8 total, 8 up, 8 in 2024-04-07T17:39:38.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:39:38 smithi144 bash[66654]: cluster 2024-04-07T17:39:36.539337+0000 mgr.y (mgr.25175) 2527 : cluster [DBG] pgmap v4217: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 364 KiB/s rd, 27 KiB/s wr, 508 op/s 2024-04-07T17:39:38.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:39:38 smithi144 bash[66654]: cluster 2024-04-07T17:39:37.094378+0000 mon.a (mon.0) 4051 : cluster [DBG] osdmap e2116: 8 total, 8 up, 8 in 2024-04-07T17:39:38.425 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:39:38 smithi100 bash[76202]: cluster 2024-04-07T17:39:36.539337+0000 mgr.y (mgr.25175) 2527 : cluster [DBG] pgmap v4217: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 364 KiB/s rd, 27 KiB/s wr, 508 op/s 2024-04-07T17:39:38.425 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:39:38 smithi100 bash[76202]: cluster 2024-04-07T17:39:37.094378+0000 mon.a (mon.0) 4051 : cluster [DBG] osdmap e2116: 8 total, 8 up, 8 in 2024-04-07T17:39:38.426 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:39:38 smithi100 bash[74942]: cluster 2024-04-07T17:39:36.539337+0000 mgr.y (mgr.25175) 2527 : cluster [DBG] pgmap v4217: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 364 KiB/s rd, 27 KiB/s wr, 508 op/s 2024-04-07T17:39:38.426 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:39:38 smithi100 bash[74942]: cluster 2024-04-07T17:39:37.094378+0000 mon.a (mon.0) 4051 : cluster [DBG] osdmap e2116: 8 total, 8 up, 8 in 2024-04-07T17:39:38.444 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_size PASSED [ 28%] 2024-04-07T17:39:38.611 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_resize_down PASSED [ 28%] 2024-04-07T17:39:38.703 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:39:38 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:39:38] "GET /metrics HTTP/1.1" 200 36735 "" "Prometheus/2.43.0" 2024-04-07T17:39:38.855 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_resize_bytes PASSED [ 29%] 2024-04-07T17:39:39.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:39:39 smithi144 bash[66654]: cluster 2024-04-07T17:39:38.098241+0000 mon.a (mon.0) 4052 : cluster [DBG] osdmap e2117: 8 total, 8 up, 8 in 2024-04-07T17:39:39.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:39:39 smithi144 bash[66654]: cluster 2024-04-07T17:39:38.254787+0000 mon.a (mon.0) 4053 : cluster [DBG] osdmap e2118: 8 total, 8 up, 8 in 2024-04-07T17:39:39.380 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_copy PASSED [ 29%] 2024-04-07T17:39:39.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:39:39 smithi100 bash[76202]: cluster 2024-04-07T17:39:38.098241+0000 mon.a (mon.0) 4052 : cluster [DBG] osdmap e2117: 8 total, 8 up, 8 in 2024-04-07T17:39:39.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:39:39 smithi100 bash[76202]: cluster 2024-04-07T17:39:38.254787+0000 mon.a (mon.0) 4053 : cluster [DBG] osdmap e2118: 8 total, 8 up, 8 in 2024-04-07T17:39:39.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:39:39 smithi100 bash[74942]: cluster 2024-04-07T17:39:38.098241+0000 mon.a (mon.0) 4052 : cluster [DBG] osdmap e2117: 8 total, 8 up, 8 in 2024-04-07T17:39:39.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:39:39 smithi100 bash[74942]: cluster 2024-04-07T17:39:38.254787+0000 mon.a (mon.0) 4053 : cluster [DBG] osdmap e2118: 8 total, 8 up, 8 in 2024-04-07T17:39:39.926 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_copy2 PASSED [ 30%] 2024-04-07T17:39:40.138 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_copy3 SKIPPED [ 30%] 2024-04-07T17:39:40.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:39:40 smithi144 bash[66654]: cluster 2024-04-07T17:39:38.539861+0000 mgr.y (mgr.25175) 2528 : cluster [DBG] pgmap v4221: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 303 KiB/s rd, 2.0 MiB/s wr, 425 op/s 2024-04-07T17:39:40.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:39:40 smithi100 bash[74942]: cluster 2024-04-07T17:39:38.539861+0000 mgr.y (mgr.25175) 2528 : cluster [DBG] pgmap v4221: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 303 KiB/s rd, 2.0 MiB/s wr, 425 op/s 2024-04-07T17:39:40.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:39:40 smithi100 bash[76202]: cluster 2024-04-07T17:39:38.539861+0000 mgr.y (mgr.25175) 2528 : cluster [DBG] pgmap v4221: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 303 KiB/s rd, 2.0 MiB/s wr, 425 op/s 2024-04-07T17:39:42.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:39:42 smithi100 bash[74942]: cluster 2024-04-07T17:39:40.541037+0000 mgr.y (mgr.25175) 2529 : cluster [DBG] pgmap v4222: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 349 KiB/s rd, 1.8 MiB/s wr, 492 op/s 2024-04-07T17:39:42.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:39:42 smithi100 bash[74942]: cluster 2024-04-07T17:39:41.122289+0000 mon.a (mon.0) 4054 : cluster [DBG] osdmap e2119: 8 total, 8 up, 8 in 2024-04-07T17:39:42.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:39:42 smithi100 bash[76202]: cluster 2024-04-07T17:39:40.541037+0000 mgr.y (mgr.25175) 2529 : cluster [DBG] pgmap v4222: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 349 KiB/s rd, 1.8 MiB/s wr, 492 op/s 2024-04-07T17:39:42.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:39:42 smithi100 bash[76202]: cluster 2024-04-07T17:39:41.122289+0000 mon.a (mon.0) 4054 : cluster [DBG] osdmap e2119: 8 total, 8 up, 8 in 2024-04-07T17:39:42.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:39:42 smithi144 bash[66654]: cluster 2024-04-07T17:39:40.541037+0000 mgr.y (mgr.25175) 2529 : cluster [DBG] pgmap v4222: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 349 KiB/s rd, 1.8 MiB/s wr, 492 op/s 2024-04-07T17:39:42.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:39:42 smithi144 bash[66654]: cluster 2024-04-07T17:39:41.122289+0000 mon.a (mon.0) 4054 : cluster [DBG] osdmap e2119: 8 total, 8 up, 8 in 2024-04-07T17:39:42.615 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:39:42 smithi144 bash[95233]: ts=2024-04-07T17:39:42.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:39:43.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:39:43 smithi100 bash[74942]: cluster 2024-04-07T17:39:42.135226+0000 mon.a (mon.0) 4055 : cluster [DBG] osdmap e2120: 8 total, 8 up, 8 in 2024-04-07T17:39:43.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:39:43 smithi100 bash[76202]: cluster 2024-04-07T17:39:42.135226+0000 mon.a (mon.0) 4055 : cluster [DBG] osdmap e2120: 8 total, 8 up, 8 in 2024-04-07T17:39:43.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:39:43 smithi144 bash[66654]: cluster 2024-04-07T17:39:42.135226+0000 mon.a (mon.0) 4055 : cluster [DBG] osdmap e2120: 8 total, 8 up, 8 in 2024-04-07T17:39:44.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:39:44 smithi100 bash[74942]: cluster 2024-04-07T17:39:42.541832+0000 mgr.y (mgr.25175) 2530 : cluster [DBG] pgmap v4225: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 282 KiB/s rd, 27 KiB/s wr, 400 op/s 2024-04-07T17:39:44.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:39:44 smithi100 bash[74942]: cluster 2024-04-07T17:39:43.146598+0000 mon.a (mon.0) 4056 : cluster [DBG] osdmap e2121: 8 total, 8 up, 8 in 2024-04-07T17:39:44.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:39:44 smithi100 bash[76202]: cluster 2024-04-07T17:39:42.541832+0000 mgr.y (mgr.25175) 2530 : cluster [DBG] pgmap v4225: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 282 KiB/s rd, 27 KiB/s wr, 400 op/s 2024-04-07T17:39:44.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:39:44 smithi100 bash[76202]: cluster 2024-04-07T17:39:43.146598+0000 mon.a (mon.0) 4056 : cluster [DBG] osdmap e2121: 8 total, 8 up, 8 in 2024-04-07T17:39:44.575 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_deep_copy PASSED [ 31%] 2024-04-07T17:39:44.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:39:44 smithi144 bash[66654]: cluster 2024-04-07T17:39:42.541832+0000 mgr.y (mgr.25175) 2530 : cluster [DBG] pgmap v4225: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 282 KiB/s rd, 27 KiB/s wr, 400 op/s 2024-04-07T17:39:44.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:39:44 smithi144 bash[66654]: cluster 2024-04-07T17:39:43.146598+0000 mon.a (mon.0) 4056 : cluster [DBG] osdmap e2121: 8 total, 8 up, 8 in 2024-04-07T17:39:45.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:39:45 smithi100 bash[74942]: cluster 2024-04-07T17:39:44.153359+0000 mon.a (mon.0) 4057 : cluster [DBG] osdmap e2122: 8 total, 8 up, 8 in 2024-04-07T17:39:45.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:39:45 smithi100 bash[76202]: cluster 2024-04-07T17:39:44.153359+0000 mon.a (mon.0) 4057 : cluster [DBG] osdmap e2122: 8 total, 8 up, 8 in 2024-04-07T17:39:45.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:39:45 smithi144 bash[66654]: cluster 2024-04-07T17:39:44.153359+0000 mon.a (mon.0) 4057 : cluster [DBG] osdmap e2122: 8 total, 8 up, 8 in 2024-04-07T17:39:46.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:39:46 smithi100 bash[74942]: cluster 2024-04-07T17:39:44.542575+0000 mgr.y (mgr.25175) 2531 : cluster [DBG] pgmap v4228: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 304 KiB/s rd, 2.0 MiB/s wr, 427 op/s 2024-04-07T17:39:46.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:39:46 smithi100 bash[74942]: cluster 2024-04-07T17:39:45.169353+0000 mon.a (mon.0) 4058 : cluster [DBG] osdmap e2123: 8 total, 8 up, 8 in 2024-04-07T17:39:46.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:39:46 smithi100 bash[76202]: cluster 2024-04-07T17:39:44.542575+0000 mgr.y (mgr.25175) 2531 : cluster [DBG] pgmap v4228: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 304 KiB/s rd, 2.0 MiB/s wr, 427 op/s 2024-04-07T17:39:46.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:39:46 smithi100 bash[76202]: cluster 2024-04-07T17:39:45.169353+0000 mon.a (mon.0) 4058 : cluster [DBG] osdmap e2123: 8 total, 8 up, 8 in 2024-04-07T17:39:46.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:39:46 smithi144 bash[66654]: cluster 2024-04-07T17:39:44.542575+0000 mgr.y (mgr.25175) 2531 : cluster [DBG] pgmap v4228: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 304 KiB/s rd, 2.0 MiB/s wr, 427 op/s 2024-04-07T17:39:46.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:39:46 smithi144 bash[66654]: cluster 2024-04-07T17:39:45.169353+0000 mon.a (mon.0) 4058 : cluster [DBG] osdmap e2123: 8 total, 8 up, 8 in 2024-04-07T17:39:47.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:39:47 smithi100 bash[74942]: cluster 2024-04-07T17:39:46.174550+0000 mon.a (mon.0) 4059 : cluster [DBG] osdmap e2124: 8 total, 8 up, 8 in 2024-04-07T17:39:47.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:39:47 smithi100 bash[74942]: audit 2024-04-07T17:39:46.861821+0000 mon.a (mon.0) 4060 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:39:47.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:39:47 smithi100 bash[76202]: cluster 2024-04-07T17:39:46.174550+0000 mon.a (mon.0) 4059 : cluster [DBG] osdmap e2124: 8 total, 8 up, 8 in 2024-04-07T17:39:47.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:39:47 smithi100 bash[76202]: audit 2024-04-07T17:39:46.861821+0000 mon.a (mon.0) 4060 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:39:47.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:39:47 smithi144 bash[66654]: cluster 2024-04-07T17:39:46.174550+0000 mon.a (mon.0) 4059 : cluster [DBG] osdmap e2124: 8 total, 8 up, 8 in 2024-04-07T17:39:47.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:39:47 smithi144 bash[66654]: audit 2024-04-07T17:39:46.861821+0000 mon.a (mon.0) 4060 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:39:48.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:39:48 smithi100 bash[74942]: cluster 2024-04-07T17:39:46.543327+0000 mgr.y (mgr.25175) 2532 : cluster [DBG] pgmap v4231: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 116 KiB/s rd, 2.0 MiB/s wr, 165 op/s 2024-04-07T17:39:48.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:39:48 smithi100 bash[74942]: cluster 2024-04-07T17:39:47.193836+0000 mon.a (mon.0) 4061 : cluster [DBG] osdmap e2125: 8 total, 8 up, 8 in 2024-04-07T17:39:48.454 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:39:48 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:39:48] "GET /metrics HTTP/1.1" 200 36739 "" "Prometheus/2.43.0" 2024-04-07T17:39:48.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:39:48 smithi100 bash[76202]: cluster 2024-04-07T17:39:46.543327+0000 mgr.y (mgr.25175) 2532 : cluster [DBG] pgmap v4231: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 116 KiB/s rd, 2.0 MiB/s wr, 165 op/s 2024-04-07T17:39:48.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:39:48 smithi100 bash[76202]: cluster 2024-04-07T17:39:47.193836+0000 mon.a (mon.0) 4061 : cluster [DBG] osdmap e2125: 8 total, 8 up, 8 in 2024-04-07T17:39:48.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:39:48 smithi144 bash[66654]: cluster 2024-04-07T17:39:46.543327+0000 mgr.y (mgr.25175) 2532 : cluster [DBG] pgmap v4231: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 116 KiB/s rd, 2.0 MiB/s wr, 165 op/s 2024-04-07T17:39:48.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:39:48 smithi144 bash[66654]: cluster 2024-04-07T17:39:47.193836+0000 mon.a (mon.0) 4061 : cluster [DBG] osdmap e2125: 8 total, 8 up, 8 in 2024-04-07T17:39:49.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:39:49 smithi100 bash[74942]: cluster 2024-04-07T17:39:48.200753+0000 mon.a (mon.0) 4062 : cluster [DBG] osdmap e2126: 8 total, 8 up, 8 in 2024-04-07T17:39:49.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:39:49 smithi100 bash[74942]: cluster 2024-04-07T17:39:48.257270+0000 mon.a (mon.0) 4063 : cluster [DBG] osdmap e2127: 8 total, 8 up, 8 in 2024-04-07T17:39:49.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:39:49 smithi100 bash[76202]: cluster 2024-04-07T17:39:48.200753+0000 mon.a (mon.0) 4062 : cluster [DBG] osdmap e2126: 8 total, 8 up, 8 in 2024-04-07T17:39:49.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:39:49 smithi100 bash[76202]: cluster 2024-04-07T17:39:48.257270+0000 mon.a (mon.0) 4063 : cluster [DBG] osdmap e2127: 8 total, 8 up, 8 in 2024-04-07T17:39:49.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:39:49 smithi144 bash[66654]: cluster 2024-04-07T17:39:48.200753+0000 mon.a (mon.0) 4062 : cluster [DBG] osdmap e2126: 8 total, 8 up, 8 in 2024-04-07T17:39:49.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:39:49 smithi144 bash[66654]: cluster 2024-04-07T17:39:48.257270+0000 mon.a (mon.0) 4063 : cluster [DBG] osdmap e2127: 8 total, 8 up, 8 in 2024-04-07T17:39:50.453 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:39:50 smithi100 bash[76202]: cluster 2024-04-07T17:39:48.544147+0000 mgr.y (mgr.25175) 2533 : cluster [DBG] pgmap v4235: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 65 KiB/s rd, 11 KiB/s wr, 103 op/s 2024-04-07T17:39:50.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:39:50 smithi100 bash[76202]: cluster 2024-04-07T17:39:49.266177+0000 mon.a (mon.0) 4064 : cluster [DBG] osdmap e2128: 8 total, 8 up, 8 in 2024-04-07T17:39:50.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:39:50 smithi100 bash[74942]: cluster 2024-04-07T17:39:48.544147+0000 mgr.y (mgr.25175) 2533 : cluster [DBG] pgmap v4235: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 65 KiB/s rd, 11 KiB/s wr, 103 op/s 2024-04-07T17:39:50.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:39:50 smithi100 bash[74942]: cluster 2024-04-07T17:39:49.266177+0000 mon.a (mon.0) 4064 : cluster [DBG] osdmap e2128: 8 total, 8 up, 8 in 2024-04-07T17:39:50.477 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_deep_copy_clone PASSED [ 32%] 2024-04-07T17:39:50.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:39:50 smithi144 bash[66654]: cluster 2024-04-07T17:39:48.544147+0000 mgr.y (mgr.25175) 2533 : cluster [DBG] pgmap v4235: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 65 KiB/s rd, 11 KiB/s wr, 103 op/s 2024-04-07T17:39:50.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:39:50 smithi144 bash[66654]: cluster 2024-04-07T17:39:49.266177+0000 mon.a (mon.0) 4064 : cluster [DBG] osdmap e2128: 8 total, 8 up, 8 in 2024-04-07T17:39:51.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:39:51 smithi144 bash[66654]: cluster 2024-04-07T17:39:50.267283+0000 mon.a (mon.0) 4065 : cluster [DBG] osdmap e2129: 8 total, 8 up, 8 in 2024-04-07T17:39:51.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:39:51 smithi144 bash[66654]: cluster 2024-04-07T17:39:50.544841+0000 mgr.y (mgr.25175) 2534 : cluster [DBG] pgmap v4238: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 172 KiB/s rd, 14 KiB/s wr, 227 op/s 2024-04-07T17:39:51.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:39:51 smithi100 bash[74942]: cluster 2024-04-07T17:39:50.267283+0000 mon.a (mon.0) 4065 : cluster [DBG] osdmap e2129: 8 total, 8 up, 8 in 2024-04-07T17:39:51.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:39:51 smithi100 bash[74942]: cluster 2024-04-07T17:39:50.544841+0000 mgr.y (mgr.25175) 2534 : cluster [DBG] pgmap v4238: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 172 KiB/s rd, 14 KiB/s wr, 227 op/s 2024-04-07T17:39:51.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:39:51 smithi100 bash[76202]: cluster 2024-04-07T17:39:50.267283+0000 mon.a (mon.0) 4065 : cluster [DBG] osdmap e2129: 8 total, 8 up, 8 in 2024-04-07T17:39:51.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:39:51 smithi100 bash[76202]: cluster 2024-04-07T17:39:50.544841+0000 mgr.y (mgr.25175) 2534 : cluster [DBG] pgmap v4238: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 172 KiB/s rd, 14 KiB/s wr, 227 op/s 2024-04-07T17:39:52.526 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_create_snap PASSED [ 32%] 2024-04-07T17:39:52.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:39:52 smithi144 bash[66654]: cluster 2024-04-07T17:39:51.280701+0000 mon.a (mon.0) 4066 : cluster [DBG] osdmap e2130: 8 total, 8 up, 8 in 2024-04-07T17:39:52.615 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:39:52 smithi144 bash[95233]: ts=2024-04-07T17:39:52.318Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:39:52.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:39:52 smithi100 bash[74942]: cluster 2024-04-07T17:39:51.280701+0000 mon.a (mon.0) 4066 : cluster [DBG] osdmap e2130: 8 total, 8 up, 8 in 2024-04-07T17:39:52.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:39:52 smithi100 bash[76202]: cluster 2024-04-07T17:39:51.280701+0000 mon.a (mon.0) 4066 : cluster [DBG] osdmap e2130: 8 total, 8 up, 8 in 2024-04-07T17:39:53.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:39:53 smithi144 bash[66654]: cluster 2024-04-07T17:39:52.288603+0000 mon.a (mon.0) 4067 : cluster [DBG] osdmap e2131: 8 total, 8 up, 8 in 2024-04-07T17:39:53.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:39:53 smithi144 bash[66654]: cluster 2024-04-07T17:39:52.545533+0000 mgr.y (mgr.25175) 2535 : cluster [DBG] pgmap v4241: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 145 KiB/s rd, 9.0 KiB/s wr, 191 op/s 2024-04-07T17:39:53.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:39:53 smithi144 bash[66654]: cluster 2024-04-07T17:39:53.258407+0000 mon.a (mon.0) 4068 : cluster [DBG] osdmap e2132: 8 total, 8 up, 8 in 2024-04-07T17:39:53.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:39:53 smithi100 bash[74942]: cluster 2024-04-07T17:39:52.288603+0000 mon.a (mon.0) 4067 : cluster [DBG] osdmap e2131: 8 total, 8 up, 8 in 2024-04-07T17:39:53.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:39:53 smithi100 bash[74942]: cluster 2024-04-07T17:39:52.545533+0000 mgr.y (mgr.25175) 2535 : cluster [DBG] pgmap v4241: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 145 KiB/s rd, 9.0 KiB/s wr, 191 op/s 2024-04-07T17:39:53.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:39:53 smithi100 bash[74942]: cluster 2024-04-07T17:39:53.258407+0000 mon.a (mon.0) 4068 : cluster [DBG] osdmap e2132: 8 total, 8 up, 8 in 2024-04-07T17:39:53.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:39:53 smithi100 bash[76202]: cluster 2024-04-07T17:39:52.288603+0000 mon.a (mon.0) 4067 : cluster [DBG] osdmap e2131: 8 total, 8 up, 8 in 2024-04-07T17:39:53.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:39:53 smithi100 bash[76202]: cluster 2024-04-07T17:39:52.545533+0000 mgr.y (mgr.25175) 2535 : cluster [DBG] pgmap v4241: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 145 KiB/s rd, 9.0 KiB/s wr, 191 op/s 2024-04-07T17:39:53.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:39:53 smithi100 bash[76202]: cluster 2024-04-07T17:39:53.258407+0000 mon.a (mon.0) 4068 : cluster [DBG] osdmap e2132: 8 total, 8 up, 8 in 2024-04-07T17:39:54.537 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_create_snap_exists PASSED [ 33%] 2024-04-07T17:39:55.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:39:55 smithi144 bash[66654]: cluster 2024-04-07T17:39:54.302113+0000 mon.a (mon.0) 4069 : cluster [DBG] osdmap e2133: 8 total, 8 up, 8 in 2024-04-07T17:39:55.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:39:55 smithi144 bash[66654]: cluster 2024-04-07T17:39:54.546334+0000 mgr.y (mgr.25175) 2536 : cluster [DBG] pgmap v4244: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 115 KiB/s rd, 8.7 KiB/s wr, 163 op/s 2024-04-07T17:39:55.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:39:55 smithi100 bash[74942]: cluster 2024-04-07T17:39:54.302113+0000 mon.a (mon.0) 4069 : cluster [DBG] osdmap e2133: 8 total, 8 up, 8 in 2024-04-07T17:39:55.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:39:55 smithi100 bash[74942]: cluster 2024-04-07T17:39:54.546334+0000 mgr.y (mgr.25175) 2536 : cluster [DBG] pgmap v4244: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 115 KiB/s rd, 8.7 KiB/s wr, 163 op/s 2024-04-07T17:39:55.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:39:55 smithi100 bash[76202]: cluster 2024-04-07T17:39:54.302113+0000 mon.a (mon.0) 4069 : cluster [DBG] osdmap e2133: 8 total, 8 up, 8 in 2024-04-07T17:39:55.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:39:55 smithi100 bash[76202]: cluster 2024-04-07T17:39:54.546334+0000 mgr.y (mgr.25175) 2536 : cluster [DBG] pgmap v4244: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 115 KiB/s rd, 8.7 KiB/s wr, 163 op/s 2024-04-07T17:39:56.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:39:56 smithi144 bash[66654]: cluster 2024-04-07T17:39:55.314431+0000 mon.a (mon.0) 4070 : cluster [DBG] osdmap e2134: 8 total, 8 up, 8 in 2024-04-07T17:39:56.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:39:56 smithi100 bash[74942]: cluster 2024-04-07T17:39:55.314431+0000 mon.a (mon.0) 4070 : cluster [DBG] osdmap e2134: 8 total, 8 up, 8 in 2024-04-07T17:39:56.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:39:56 smithi100 bash[76202]: cluster 2024-04-07T17:39:55.314431+0000 mon.a (mon.0) 4070 : cluster [DBG] osdmap e2134: 8 total, 8 up, 8 in 2024-04-07T17:39:57.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:39:57 smithi144 bash[66654]: cluster 2024-04-07T17:39:56.315285+0000 mon.a (mon.0) 4071 : cluster [DBG] osdmap e2135: 8 total, 8 up, 8 in 2024-04-07T17:39:57.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:39:57 smithi144 bash[66654]: cluster 2024-04-07T17:39:56.547121+0000 mgr.y (mgr.25175) 2537 : cluster [DBG] pgmap v4247: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 120 KiB/s rd, 8.0 KiB/s wr, 161 op/s 2024-04-07T17:39:57.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:39:57 smithi100 bash[74942]: cluster 2024-04-07T17:39:56.315285+0000 mon.a (mon.0) 4071 : cluster [DBG] osdmap e2135: 8 total, 8 up, 8 in 2024-04-07T17:39:57.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:39:57 smithi100 bash[74942]: cluster 2024-04-07T17:39:56.547121+0000 mgr.y (mgr.25175) 2537 : cluster [DBG] pgmap v4247: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 120 KiB/s rd, 8.0 KiB/s wr, 161 op/s 2024-04-07T17:39:57.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:39:57 smithi100 bash[76202]: cluster 2024-04-07T17:39:56.315285+0000 mon.a (mon.0) 4071 : cluster [DBG] osdmap e2135: 8 total, 8 up, 8 in 2024-04-07T17:39:57.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:39:57 smithi100 bash[76202]: cluster 2024-04-07T17:39:56.547121+0000 mgr.y (mgr.25175) 2537 : cluster [DBG] pgmap v4247: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 120 KiB/s rd, 8.0 KiB/s wr, 161 op/s 2024-04-07T17:39:58.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:39:58 smithi144 bash[66654]: cluster 2024-04-07T17:39:57.342591+0000 mon.a (mon.0) 4072 : cluster [DBG] osdmap e2136: 8 total, 8 up, 8 in 2024-04-07T17:39:58.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:39:58 smithi144 bash[66654]: cluster 2024-04-07T17:39:58.259995+0000 mon.a (mon.0) 4073 : cluster [DBG] osdmap e2137: 8 total, 8 up, 8 in 2024-04-07T17:39:58.704 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:39:58 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:39:58] "GET /metrics HTTP/1.1" 200 36739 "" "Prometheus/2.43.0" 2024-04-07T17:39:58.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:39:58 smithi100 bash[76202]: cluster 2024-04-07T17:39:57.342591+0000 mon.a (mon.0) 4072 : cluster [DBG] osdmap e2136: 8 total, 8 up, 8 in 2024-04-07T17:39:58.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:39:58 smithi100 bash[76202]: cluster 2024-04-07T17:39:58.259995+0000 mon.a (mon.0) 4073 : cluster [DBG] osdmap e2137: 8 total, 8 up, 8 in 2024-04-07T17:39:58.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:39:58 smithi100 bash[74942]: cluster 2024-04-07T17:39:57.342591+0000 mon.a (mon.0) 4072 : cluster [DBG] osdmap e2136: 8 total, 8 up, 8 in 2024-04-07T17:39:58.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:39:58 smithi100 bash[74942]: cluster 2024-04-07T17:39:58.259995+0000 mon.a (mon.0) 4073 : cluster [DBG] osdmap e2137: 8 total, 8 up, 8 in 2024-04-07T17:39:59.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:39:59 smithi144 bash[66654]: cluster 2024-04-07T17:39:58.547903+0000 mgr.y (mgr.25175) 2538 : cluster [DBG] pgmap v4250: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 71 KiB/s rd, 4.0 KiB/s wr, 96 op/s 2024-04-07T17:39:59.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:39:59 smithi144 bash[66654]: cluster 2024-04-07T17:39:59.264172+0000 mon.a (mon.0) 4074 : cluster [DBG] osdmap e2138: 8 total, 8 up, 8 in 2024-04-07T17:39:59.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:39:59 smithi100 bash[74942]: cluster 2024-04-07T17:39:58.547903+0000 mgr.y (mgr.25175) 2538 : cluster [DBG] pgmap v4250: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 71 KiB/s rd, 4.0 KiB/s wr, 96 op/s 2024-04-07T17:39:59.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:39:59 smithi100 bash[74942]: cluster 2024-04-07T17:39:59.264172+0000 mon.a (mon.0) 4074 : cluster [DBG] osdmap e2138: 8 total, 8 up, 8 in 2024-04-07T17:39:59.705 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:39:59 smithi100 bash[76202]: cluster 2024-04-07T17:39:58.547903+0000 mgr.y (mgr.25175) 2538 : cluster [DBG] pgmap v4250: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 71 KiB/s rd, 4.0 KiB/s wr, 96 op/s 2024-04-07T17:39:59.705 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:39:59 smithi100 bash[76202]: cluster 2024-04-07T17:39:59.264172+0000 mon.a (mon.0) 4074 : cluster [DBG] osdmap e2138: 8 total, 8 up, 8 in 2024-04-07T17:40:00.487 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_create_snap_flags PASSED [ 33%] 2024-04-07T17:40:00.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:40:00 smithi144 bash[66654]: cluster 2024-04-07T17:40:00.000164+0000 mon.a (mon.0) 4075 : cluster [WRN] overall HEALTH_WARN Telemetry requires re-opt-in 2024-04-07T17:40:00.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:40:00 smithi144 bash[66654]: cluster 2024-04-07T17:40:00.270865+0000 mon.a (mon.0) 4076 : cluster [DBG] osdmap e2139: 8 total, 8 up, 8 in 2024-04-07T17:40:00.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:40:00 smithi100 bash[74942]: cluster 2024-04-07T17:40:00.000164+0000 mon.a (mon.0) 4075 : cluster [WRN] overall HEALTH_WARN Telemetry requires re-opt-in 2024-04-07T17:40:00.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:40:00 smithi100 bash[74942]: cluster 2024-04-07T17:40:00.270865+0000 mon.a (mon.0) 4076 : cluster [DBG] osdmap e2139: 8 total, 8 up, 8 in 2024-04-07T17:40:00.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:40:00 smithi100 bash[76202]: cluster 2024-04-07T17:40:00.000164+0000 mon.a (mon.0) 4075 : cluster [WRN] overall HEALTH_WARN Telemetry requires re-opt-in 2024-04-07T17:40:00.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:40:00 smithi100 bash[76202]: cluster 2024-04-07T17:40:00.270865+0000 mon.a (mon.0) 4076 : cluster [DBG] osdmap e2139: 8 total, 8 up, 8 in 2024-04-07T17:40:01.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:40:01 smithi144 bash[66654]: cluster 2024-04-07T17:40:00.548611+0000 mgr.y (mgr.25175) 2539 : cluster [DBG] pgmap v4253: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 81 KiB/s rd, 8.0 KiB/s wr, 113 op/s 2024-04-07T17:40:01.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:40:01 smithi100 bash[74942]: cluster 2024-04-07T17:40:00.548611+0000 mgr.y (mgr.25175) 2539 : cluster [DBG] pgmap v4253: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 81 KiB/s rd, 8.0 KiB/s wr, 113 op/s 2024-04-07T17:40:01.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:40:01 smithi100 bash[76202]: cluster 2024-04-07T17:40:00.548611+0000 mgr.y (mgr.25175) 2539 : cluster [DBG] pgmap v4253: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 81 KiB/s rd, 8.0 KiB/s wr, 113 op/s 2024-04-07T17:40:02.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:40:02 smithi144 bash[66654]: cluster 2024-04-07T17:40:01.362705+0000 mon.a (mon.0) 4077 : cluster [DBG] osdmap e2140: 8 total, 8 up, 8 in 2024-04-07T17:40:02.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:40:02 smithi144 bash[66654]: audit 2024-04-07T17:40:01.862409+0000 mon.a (mon.0) 4078 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:40:02.615 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:40:02 smithi144 bash[95233]: ts=2024-04-07T17:40:02.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:40:02.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:40:02 smithi100 bash[74942]: cluster 2024-04-07T17:40:01.362705+0000 mon.a (mon.0) 4077 : cluster [DBG] osdmap e2140: 8 total, 8 up, 8 in 2024-04-07T17:40:02.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:40:02 smithi100 bash[74942]: audit 2024-04-07T17:40:01.862409+0000 mon.a (mon.0) 4078 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:40:02.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:40:02 smithi100 bash[76202]: cluster 2024-04-07T17:40:01.362705+0000 mon.a (mon.0) 4077 : cluster [DBG] osdmap e2140: 8 total, 8 up, 8 in 2024-04-07T17:40:02.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:40:02 smithi100 bash[76202]: audit 2024-04-07T17:40:01.862409+0000 mon.a (mon.0) 4078 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:40:03.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:40:03 smithi100 bash[74942]: cluster 2024-04-07T17:40:02.377203+0000 mon.a (mon.0) 4079 : cluster [DBG] osdmap e2141: 8 total, 8 up, 8 in 2024-04-07T17:40:03.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:40:03 smithi100 bash[74942]: cluster 2024-04-07T17:40:02.549383+0000 mgr.y (mgr.25175) 2540 : cluster [DBG] pgmap v4256: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 52 KiB/s rd, 5.7 KiB/s wr, 72 op/s 2024-04-07T17:40:03.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:40:03 smithi100 bash[74942]: cluster 2024-04-07T17:40:03.266797+0000 mon.a (mon.0) 4080 : cluster [DBG] osdmap e2142: 8 total, 8 up, 8 in 2024-04-07T17:40:03.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:40:03 smithi100 bash[76202]: cluster 2024-04-07T17:40:02.377203+0000 mon.a (mon.0) 4079 : cluster [DBG] osdmap e2141: 8 total, 8 up, 8 in 2024-04-07T17:40:03.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:40:03 smithi100 bash[76202]: cluster 2024-04-07T17:40:02.549383+0000 mgr.y (mgr.25175) 2540 : cluster [DBG] pgmap v4256: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 52 KiB/s rd, 5.7 KiB/s wr, 72 op/s 2024-04-07T17:40:03.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:40:03 smithi100 bash[76202]: cluster 2024-04-07T17:40:03.266797+0000 mon.a (mon.0) 4080 : cluster [DBG] osdmap e2142: 8 total, 8 up, 8 in 2024-04-07T17:40:03.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:40:03 smithi144 bash[66654]: cluster 2024-04-07T17:40:02.377203+0000 mon.a (mon.0) 4079 : cluster [DBG] osdmap e2141: 8 total, 8 up, 8 in 2024-04-07T17:40:03.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:40:03 smithi144 bash[66654]: cluster 2024-04-07T17:40:02.549383+0000 mgr.y (mgr.25175) 2540 : cluster [DBG] pgmap v4256: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 52 KiB/s rd, 5.7 KiB/s wr, 72 op/s 2024-04-07T17:40:03.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:40:03 smithi144 bash[66654]: cluster 2024-04-07T17:40:03.266797+0000 mon.a (mon.0) 4080 : cluster [DBG] osdmap e2142: 8 total, 8 up, 8 in 2024-04-07T17:40:04.476 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_list_snaps PASSED [ 34%] 2024-04-07T17:40:05.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:40:05 smithi144 bash[66654]: cluster 2024-04-07T17:40:04.265065+0000 mon.a (mon.0) 4081 : cluster [DBG] osdmap e2143: 8 total, 8 up, 8 in 2024-04-07T17:40:05.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:40:05 smithi144 bash[66654]: cluster 2024-04-07T17:40:04.550139+0000 mgr.y (mgr.25175) 2541 : cluster [DBG] pgmap v4259: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 53 KiB/s rd, 4.7 KiB/s wr, 75 op/s 2024-04-07T17:40:05.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:40:05 smithi100 bash[74942]: cluster 2024-04-07T17:40:04.265065+0000 mon.a (mon.0) 4081 : cluster [DBG] osdmap e2143: 8 total, 8 up, 8 in 2024-04-07T17:40:05.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:40:05 smithi100 bash[74942]: cluster 2024-04-07T17:40:04.550139+0000 mgr.y (mgr.25175) 2541 : cluster [DBG] pgmap v4259: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 53 KiB/s rd, 4.7 KiB/s wr, 75 op/s 2024-04-07T17:40:05.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:40:05 smithi100 bash[76202]: cluster 2024-04-07T17:40:04.265065+0000 mon.a (mon.0) 4081 : cluster [DBG] osdmap e2143: 8 total, 8 up, 8 in 2024-04-07T17:40:05.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:40:05 smithi100 bash[76202]: cluster 2024-04-07T17:40:04.550139+0000 mgr.y (mgr.25175) 2541 : cluster [DBG] pgmap v4259: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 53 KiB/s rd, 4.7 KiB/s wr, 75 op/s 2024-04-07T17:40:06.492 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_list_snaps_iterator_auto_close PASSED [ 34%] 2024-04-07T17:40:06.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:40:06 smithi144 bash[66654]: cluster 2024-04-07T17:40:05.284530+0000 mon.a (mon.0) 4082 : cluster [DBG] osdmap e2144: 8 total, 8 up, 8 in 2024-04-07T17:40:06.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:40:06 smithi100 bash[74942]: cluster 2024-04-07T17:40:05.284530+0000 mon.a (mon.0) 4082 : cluster [DBG] osdmap e2144: 8 total, 8 up, 8 in 2024-04-07T17:40:06.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:40:06 smithi100 bash[76202]: cluster 2024-04-07T17:40:05.284530+0000 mon.a (mon.0) 4082 : cluster [DBG] osdmap e2144: 8 total, 8 up, 8 in 2024-04-07T17:40:07.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:40:07 smithi144 bash[66654]: cluster 2024-04-07T17:40:06.293345+0000 mon.a (mon.0) 4083 : cluster [DBG] osdmap e2145: 8 total, 8 up, 8 in 2024-04-07T17:40:07.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:40:07 smithi144 bash[66654]: cluster 2024-04-07T17:40:06.550892+0000 mgr.y (mgr.25175) 2542 : cluster [DBG] pgmap v4262: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 32 KiB/s rd, 3.2 KiB/s wr, 44 op/s 2024-04-07T17:40:07.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:40:07 smithi100 bash[74942]: cluster 2024-04-07T17:40:06.293345+0000 mon.a (mon.0) 4083 : cluster [DBG] osdmap e2145: 8 total, 8 up, 8 in 2024-04-07T17:40:07.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:40:07 smithi100 bash[74942]: cluster 2024-04-07T17:40:06.550892+0000 mgr.y (mgr.25175) 2542 : cluster [DBG] pgmap v4262: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 32 KiB/s rd, 3.2 KiB/s wr, 44 op/s 2024-04-07T17:40:07.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:40:07 smithi100 bash[76202]: cluster 2024-04-07T17:40:06.293345+0000 mon.a (mon.0) 4083 : cluster [DBG] osdmap e2145: 8 total, 8 up, 8 in 2024-04-07T17:40:07.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:40:07 smithi100 bash[76202]: cluster 2024-04-07T17:40:06.550892+0000 mgr.y (mgr.25175) 2542 : cluster [DBG] pgmap v4262: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 32 KiB/s rd, 3.2 KiB/s wr, 44 op/s 2024-04-07T17:40:08.503 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_remove_snap PASSED [ 35%] 2024-04-07T17:40:08.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:40:08 smithi144 bash[66654]: cluster 2024-04-07T17:40:07.302797+0000 mon.a (mon.0) 4084 : cluster [DBG] osdmap e2146: 8 total, 8 up, 8 in 2024-04-07T17:40:08.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:40:08 smithi144 bash[66654]: cluster 2024-04-07T17:40:08.264800+0000 mon.a (mon.0) 4085 : cluster [DBG] osdmap e2147: 8 total, 8 up, 8 in 2024-04-07T17:40:08.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:40:08 smithi100 bash[74942]: cluster 2024-04-07T17:40:07.302797+0000 mon.a (mon.0) 4084 : cluster [DBG] osdmap e2146: 8 total, 8 up, 8 in 2024-04-07T17:40:08.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:40:08 smithi100 bash[74942]: cluster 2024-04-07T17:40:08.264800+0000 mon.a (mon.0) 4085 : cluster [DBG] osdmap e2147: 8 total, 8 up, 8 in 2024-04-07T17:40:08.704 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:40:08 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:40:08] "GET /metrics HTTP/1.1" 200 36740 "" "Prometheus/2.43.0" 2024-04-07T17:40:08.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:40:08 smithi100 bash[76202]: cluster 2024-04-07T17:40:07.302797+0000 mon.a (mon.0) 4084 : cluster [DBG] osdmap e2146: 8 total, 8 up, 8 in 2024-04-07T17:40:08.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:40:08 smithi100 bash[76202]: cluster 2024-04-07T17:40:08.264800+0000 mon.a (mon.0) 4085 : cluster [DBG] osdmap e2147: 8 total, 8 up, 8 in 2024-04-07T17:40:08.714 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_remove_snap_not_found PASSED [ 36%] 2024-04-07T17:40:09.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:40:09 smithi144 bash[66654]: cluster 2024-04-07T17:40:08.551677+0000 mgr.y (mgr.25175) 2543 : cluster [DBG] pgmap v4265: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 86 KiB/s rd, 5.7 KiB/s wr, 117 op/s 2024-04-07T17:40:09.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:40:09 smithi100 bash[74942]: cluster 2024-04-07T17:40:08.551677+0000 mgr.y (mgr.25175) 2543 : cluster [DBG] pgmap v4265: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 86 KiB/s rd, 5.7 KiB/s wr, 117 op/s 2024-04-07T17:40:09.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:40:09 smithi100 bash[76202]: cluster 2024-04-07T17:40:08.551677+0000 mgr.y (mgr.25175) 2543 : cluster [DBG] pgmap v4265: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 86 KiB/s rd, 5.7 KiB/s wr, 117 op/s 2024-04-07T17:40:10.550 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_remove_snap2 PASSED [ 36%] 2024-04-07T17:40:10.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:40:10 smithi144 bash[66654]: cluster 2024-04-07T17:40:09.314777+0000 mon.a (mon.0) 4086 : cluster [DBG] osdmap e2148: 8 total, 8 up, 8 in 2024-04-07T17:40:10.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:40:10 smithi100 bash[74942]: cluster 2024-04-07T17:40:09.314777+0000 mon.a (mon.0) 4086 : cluster [DBG] osdmap e2148: 8 total, 8 up, 8 in 2024-04-07T17:40:10.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:40:10 smithi100 bash[76202]: cluster 2024-04-07T17:40:09.314777+0000 mon.a (mon.0) 4086 : cluster [DBG] osdmap e2148: 8 total, 8 up, 8 in 2024-04-07T17:40:11.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:40:11 smithi144 bash[66654]: cluster 2024-04-07T17:40:10.323025+0000 mon.a (mon.0) 4087 : cluster [DBG] osdmap e2149: 8 total, 8 up, 8 in 2024-04-07T17:40:11.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:40:11 smithi144 bash[66654]: cluster 2024-04-07T17:40:10.552567+0000 mgr.y (mgr.25175) 2544 : cluster [DBG] pgmap v4268: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 183 KiB/s rd, 13 KiB/s wr, 248 op/s 2024-04-07T17:40:11.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:40:11 smithi100 bash[74942]: cluster 2024-04-07T17:40:10.323025+0000 mon.a (mon.0) 4087 : cluster [DBG] osdmap e2149: 8 total, 8 up, 8 in 2024-04-07T17:40:11.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:40:11 smithi100 bash[74942]: cluster 2024-04-07T17:40:10.552567+0000 mgr.y (mgr.25175) 2544 : cluster [DBG] pgmap v4268: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 183 KiB/s rd, 13 KiB/s wr, 248 op/s 2024-04-07T17:40:11.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:40:11 smithi100 bash[76202]: cluster 2024-04-07T17:40:10.323025+0000 mon.a (mon.0) 4087 : cluster [DBG] osdmap e2149: 8 total, 8 up, 8 in 2024-04-07T17:40:11.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:40:11 smithi100 bash[76202]: cluster 2024-04-07T17:40:10.552567+0000 mgr.y (mgr.25175) 2544 : cluster [DBG] pgmap v4268: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 183 KiB/s rd, 13 KiB/s wr, 248 op/s 2024-04-07T17:40:12.587 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_remove_snap_by_id PASSED [ 37%] 2024-04-07T17:40:12.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:40:12 smithi144 bash[66654]: cluster 2024-04-07T17:40:11.334531+0000 mon.a (mon.0) 4088 : cluster [DBG] osdmap e2150: 8 total, 8 up, 8 in 2024-04-07T17:40:12.615 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:40:12 smithi144 bash[95233]: ts=2024-04-07T17:40:12.317Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:40:12.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:40:12 smithi100 bash[74942]: cluster 2024-04-07T17:40:11.334531+0000 mon.a (mon.0) 4088 : cluster [DBG] osdmap e2150: 8 total, 8 up, 8 in 2024-04-07T17:40:12.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:40:12 smithi100 bash[76202]: cluster 2024-04-07T17:40:11.334531+0000 mon.a (mon.0) 4088 : cluster [DBG] osdmap e2150: 8 total, 8 up, 8 in 2024-04-07T17:40:13.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:40:13 smithi144 bash[66654]: cluster 2024-04-07T17:40:12.342764+0000 mon.a (mon.0) 4089 : cluster [DBG] osdmap e2151: 8 total, 8 up, 8 in 2024-04-07T17:40:13.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:40:13 smithi144 bash[66654]: cluster 2024-04-07T17:40:12.553332+0000 mgr.y (mgr.25175) 2545 : cluster [DBG] pgmap v4271: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 118 KiB/s rd, 10 KiB/s wr, 161 op/s 2024-04-07T17:40:13.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:40:13 smithi144 bash[66654]: cluster 2024-04-07T17:40:13.266865+0000 mon.a (mon.0) 4090 : cluster [DBG] osdmap e2152: 8 total, 8 up, 8 in 2024-04-07T17:40:13.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:40:13 smithi100 bash[74942]: cluster 2024-04-07T17:40:12.342764+0000 mon.a (mon.0) 4089 : cluster [DBG] osdmap e2151: 8 total, 8 up, 8 in 2024-04-07T17:40:13.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:40:13 smithi100 bash[74942]: cluster 2024-04-07T17:40:12.553332+0000 mgr.y (mgr.25175) 2545 : cluster [DBG] pgmap v4271: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 118 KiB/s rd, 10 KiB/s wr, 161 op/s 2024-04-07T17:40:13.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:40:13 smithi100 bash[74942]: cluster 2024-04-07T17:40:13.266865+0000 mon.a (mon.0) 4090 : cluster [DBG] osdmap e2152: 8 total, 8 up, 8 in 2024-04-07T17:40:13.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:40:13 smithi100 bash[76202]: cluster 2024-04-07T17:40:12.342764+0000 mon.a (mon.0) 4089 : cluster [DBG] osdmap e2151: 8 total, 8 up, 8 in 2024-04-07T17:40:13.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:40:13 smithi100 bash[76202]: cluster 2024-04-07T17:40:12.553332+0000 mgr.y (mgr.25175) 2545 : cluster [DBG] pgmap v4271: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 118 KiB/s rd, 10 KiB/s wr, 161 op/s 2024-04-07T17:40:13.705 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:40:13 smithi100 bash[76202]: cluster 2024-04-07T17:40:13.266865+0000 mon.a (mon.0) 4090 : cluster [DBG] osdmap e2152: 8 total, 8 up, 8 in 2024-04-07T17:40:14.494 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_rename_snap PASSED [ 37%] 2024-04-07T17:40:15.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:40:15 smithi144 bash[66654]: cluster 2024-04-07T17:40:14.272623+0000 mon.a (mon.0) 4091 : cluster [DBG] osdmap e2153: 8 total, 8 up, 8 in 2024-04-07T17:40:15.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:40:15 smithi144 bash[66654]: cluster 2024-04-07T17:40:14.554103+0000 mgr.y (mgr.25175) 2546 : cluster [DBG] pgmap v4274: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 39 KiB/s rd, 6.2 KiB/s wr, 65 op/s 2024-04-07T17:40:15.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:40:15 smithi100 bash[74942]: cluster 2024-04-07T17:40:14.272623+0000 mon.a (mon.0) 4091 : cluster [DBG] osdmap e2153: 8 total, 8 up, 8 in 2024-04-07T17:40:15.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:40:15 smithi100 bash[74942]: cluster 2024-04-07T17:40:14.554103+0000 mgr.y (mgr.25175) 2546 : cluster [DBG] pgmap v4274: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 39 KiB/s rd, 6.2 KiB/s wr, 65 op/s 2024-04-07T17:40:15.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:40:15 smithi100 bash[76202]: cluster 2024-04-07T17:40:14.272623+0000 mon.a (mon.0) 4091 : cluster [DBG] osdmap e2153: 8 total, 8 up, 8 in 2024-04-07T17:40:15.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:40:15 smithi100 bash[76202]: cluster 2024-04-07T17:40:14.554103+0000 mgr.y (mgr.25175) 2546 : cluster [DBG] pgmap v4274: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 39 KiB/s rd, 6.2 KiB/s wr, 65 op/s 2024-04-07T17:40:16.524 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_protect_snap PASSED [ 38%] 2024-04-07T17:40:16.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:40:16 smithi144 bash[66654]: cluster 2024-04-07T17:40:15.284574+0000 mon.a (mon.0) 4092 : cluster [DBG] osdmap e2154: 8 total, 8 up, 8 in 2024-04-07T17:40:16.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:40:16 smithi100 bash[74942]: cluster 2024-04-07T17:40:15.284574+0000 mon.a (mon.0) 4092 : cluster [DBG] osdmap e2154: 8 total, 8 up, 8 in 2024-04-07T17:40:16.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:40:16 smithi100 bash[76202]: cluster 2024-04-07T17:40:15.284574+0000 mon.a (mon.0) 4092 : cluster [DBG] osdmap e2154: 8 total, 8 up, 8 in 2024-04-07T17:40:17.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:40:17 smithi144 bash[66654]: cluster 2024-04-07T17:40:16.295555+0000 mon.a (mon.0) 4093 : cluster [DBG] osdmap e2155: 8 total, 8 up, 8 in 2024-04-07T17:40:17.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:40:17 smithi144 bash[66654]: cluster 2024-04-07T17:40:16.554844+0000 mgr.y (mgr.25175) 2547 : cluster [DBG] pgmap v4277: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 43 KiB/s rd, 3.7 KiB/s wr, 63 op/s 2024-04-07T17:40:17.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:40:17 smithi144 bash[66654]: audit 2024-04-07T17:40:16.862752+0000 mon.a (mon.0) 4094 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:40:17.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:40:17 smithi100 bash[74942]: cluster 2024-04-07T17:40:16.295555+0000 mon.a (mon.0) 4093 : cluster [DBG] osdmap e2155: 8 total, 8 up, 8 in 2024-04-07T17:40:17.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:40:17 smithi100 bash[74942]: cluster 2024-04-07T17:40:16.554844+0000 mgr.y (mgr.25175) 2547 : cluster [DBG] pgmap v4277: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 43 KiB/s rd, 3.7 KiB/s wr, 63 op/s 2024-04-07T17:40:17.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:40:17 smithi100 bash[74942]: audit 2024-04-07T17:40:16.862752+0000 mon.a (mon.0) 4094 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:40:17.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:40:17 smithi100 bash[76202]: cluster 2024-04-07T17:40:16.295555+0000 mon.a (mon.0) 4093 : cluster [DBG] osdmap e2155: 8 total, 8 up, 8 in 2024-04-07T17:40:17.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:40:17 smithi100 bash[76202]: cluster 2024-04-07T17:40:16.554844+0000 mgr.y (mgr.25175) 2547 : cluster [DBG] pgmap v4277: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 43 KiB/s rd, 3.7 KiB/s wr, 63 op/s 2024-04-07T17:40:17.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:40:17 smithi100 bash[76202]: audit 2024-04-07T17:40:16.862752+0000 mon.a (mon.0) 4094 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:40:18.488 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_snap_exists PASSED [ 38%] 2024-04-07T17:40:18.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:40:18 smithi144 bash[66654]: cluster 2024-04-07T17:40:17.305113+0000 mon.a (mon.0) 4095 : cluster [DBG] osdmap e2156: 8 total, 8 up, 8 in 2024-04-07T17:40:18.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:40:18 smithi144 bash[66654]: cluster 2024-04-07T17:40:18.269014+0000 mon.a (mon.0) 4096 : cluster [DBG] osdmap e2157: 8 total, 8 up, 8 in 2024-04-07T17:40:18.704 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:40:18 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:40:18] "GET /metrics HTTP/1.1" 200 36732 "" "Prometheus/2.43.0" 2024-04-07T17:40:18.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:40:18 smithi100 bash[74942]: cluster 2024-04-07T17:40:17.305113+0000 mon.a (mon.0) 4095 : cluster [DBG] osdmap e2156: 8 total, 8 up, 8 in 2024-04-07T17:40:18.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:40:18 smithi100 bash[74942]: cluster 2024-04-07T17:40:18.269014+0000 mon.a (mon.0) 4096 : cluster [DBG] osdmap e2157: 8 total, 8 up, 8 in 2024-04-07T17:40:18.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:40:18 smithi100 bash[76202]: cluster 2024-04-07T17:40:17.305113+0000 mon.a (mon.0) 4095 : cluster [DBG] osdmap e2156: 8 total, 8 up, 8 in 2024-04-07T17:40:18.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:40:18 smithi100 bash[76202]: cluster 2024-04-07T17:40:18.269014+0000 mon.a (mon.0) 4096 : cluster [DBG] osdmap e2157: 8 total, 8 up, 8 in 2024-04-07T17:40:19.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:40:19 smithi144 bash[66654]: cluster 2024-04-07T17:40:18.555662+0000 mgr.y (mgr.25175) 2548 : cluster [DBG] pgmap v4280: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 160 KiB/s rd, 7.0 KiB/s wr, 208 op/s 2024-04-07T17:40:19.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:40:19 smithi100 bash[74942]: cluster 2024-04-07T17:40:18.555662+0000 mgr.y (mgr.25175) 2548 : cluster [DBG] pgmap v4280: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 160 KiB/s rd, 7.0 KiB/s wr, 208 op/s 2024-04-07T17:40:19.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:40:19 smithi100 bash[76202]: cluster 2024-04-07T17:40:18.555662+0000 mgr.y (mgr.25175) 2548 : cluster [DBG] pgmap v4280: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 160 KiB/s rd, 7.0 KiB/s wr, 208 op/s 2024-04-07T17:40:20.525 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_snap_timestamp PASSED [ 39%] 2024-04-07T17:40:20.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:40:20 smithi144 bash[66654]: cluster 2024-04-07T17:40:19.319074+0000 mon.a (mon.0) 4097 : cluster [DBG] osdmap e2158: 8 total, 8 up, 8 in 2024-04-07T17:40:20.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:40:20 smithi100 bash[74942]: cluster 2024-04-07T17:40:19.319074+0000 mon.a (mon.0) 4097 : cluster [DBG] osdmap e2158: 8 total, 8 up, 8 in 2024-04-07T17:40:20.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:40:20 smithi100 bash[76202]: cluster 2024-04-07T17:40:19.319074+0000 mon.a (mon.0) 4097 : cluster [DBG] osdmap e2158: 8 total, 8 up, 8 in 2024-04-07T17:40:21.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:40:21 smithi144 bash[66654]: cluster 2024-04-07T17:40:20.330691+0000 mon.a (mon.0) 4098 : cluster [DBG] osdmap e2159: 8 total, 8 up, 8 in 2024-04-07T17:40:21.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:40:21 smithi144 bash[66654]: cluster 2024-04-07T17:40:20.556497+0000 mgr.y (mgr.25175) 2549 : cluster [DBG] pgmap v4283: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 178 KiB/s rd, 11 KiB/s wr, 237 op/s 2024-04-07T17:40:21.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:40:21 smithi100 bash[74942]: cluster 2024-04-07T17:40:20.330691+0000 mon.a (mon.0) 4098 : cluster [DBG] osdmap e2159: 8 total, 8 up, 8 in 2024-04-07T17:40:21.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:40:21 smithi100 bash[74942]: cluster 2024-04-07T17:40:20.556497+0000 mgr.y (mgr.25175) 2549 : cluster [DBG] pgmap v4283: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 178 KiB/s rd, 11 KiB/s wr, 237 op/s 2024-04-07T17:40:21.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:40:21 smithi100 bash[76202]: cluster 2024-04-07T17:40:20.330691+0000 mon.a (mon.0) 4098 : cluster [DBG] osdmap e2159: 8 total, 8 up, 8 in 2024-04-07T17:40:21.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:40:21 smithi100 bash[76202]: cluster 2024-04-07T17:40:20.556497+0000 mgr.y (mgr.25175) 2549 : cluster [DBG] pgmap v4283: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 178 KiB/s rd, 11 KiB/s wr, 237 op/s 2024-04-07T17:40:22.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:40:22 smithi144 bash[66654]: cluster 2024-04-07T17:40:21.339512+0000 mon.a (mon.0) 4099 : cluster [DBG] osdmap e2160: 8 total, 8 up, 8 in 2024-04-07T17:40:22.615 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:40:22 smithi144 bash[95233]: ts=2024-04-07T17:40:22.317Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:40:22.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:40:22 smithi100 bash[74942]: cluster 2024-04-07T17:40:21.339512+0000 mon.a (mon.0) 4099 : cluster [DBG] osdmap e2160: 8 total, 8 up, 8 in 2024-04-07T17:40:22.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:40:22 smithi100 bash[76202]: cluster 2024-04-07T17:40:21.339512+0000 mon.a (mon.0) 4099 : cluster [DBG] osdmap e2160: 8 total, 8 up, 8 in 2024-04-07T17:40:23.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:40:23 smithi144 bash[66654]: cluster 2024-04-07T17:40:22.341487+0000 mon.a (mon.0) 4100 : cluster [DBG] osdmap e2161: 8 total, 8 up, 8 in 2024-04-07T17:40:23.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:40:23 smithi144 bash[66654]: cluster 2024-04-07T17:40:22.557300+0000 mgr.y (mgr.25175) 2550 : cluster [DBG] pgmap v4286: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 96 KiB/s rd, 8.0 KiB/s wr, 129 op/s 2024-04-07T17:40:23.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:40:23 smithi144 bash[66654]: cluster 2024-04-07T17:40:23.270665+0000 mon.a (mon.0) 4101 : cluster [DBG] osdmap e2162: 8 total, 8 up, 8 in 2024-04-07T17:40:23.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:40:23 smithi100 bash[74942]: cluster 2024-04-07T17:40:22.341487+0000 mon.a (mon.0) 4100 : cluster [DBG] osdmap e2161: 8 total, 8 up, 8 in 2024-04-07T17:40:23.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:40:23 smithi100 bash[74942]: cluster 2024-04-07T17:40:22.557300+0000 mgr.y (mgr.25175) 2550 : cluster [DBG] pgmap v4286: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 96 KiB/s rd, 8.0 KiB/s wr, 129 op/s 2024-04-07T17:40:23.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:40:23 smithi100 bash[74942]: cluster 2024-04-07T17:40:23.270665+0000 mon.a (mon.0) 4101 : cluster [DBG] osdmap e2162: 8 total, 8 up, 8 in 2024-04-07T17:40:23.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:40:23 smithi100 bash[76202]: cluster 2024-04-07T17:40:22.341487+0000 mon.a (mon.0) 4100 : cluster [DBG] osdmap e2161: 8 total, 8 up, 8 in 2024-04-07T17:40:23.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:40:23 smithi100 bash[76202]: cluster 2024-04-07T17:40:22.557300+0000 mgr.y (mgr.25175) 2550 : cluster [DBG] pgmap v4286: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 96 KiB/s rd, 8.0 KiB/s wr, 129 op/s 2024-04-07T17:40:23.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:40:23 smithi100 bash[76202]: cluster 2024-04-07T17:40:23.270665+0000 mon.a (mon.0) 4101 : cluster [DBG] osdmap e2162: 8 total, 8 up, 8 in 2024-04-07T17:40:25.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:40:25 smithi144 bash[66654]: cluster 2024-04-07T17:40:24.273768+0000 mon.a (mon.0) 4102 : cluster [DBG] osdmap e2163: 8 total, 8 up, 8 in 2024-04-07T17:40:25.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:40:25 smithi144 bash[66654]: cluster 2024-04-07T17:40:24.558061+0000 mgr.y (mgr.25175) 2551 : cluster [DBG] pgmap v4289: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 64 KiB/s rd, 5.2 KiB/s wr, 90 op/s 2024-04-07T17:40:25.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:40:25 smithi100 bash[74942]: cluster 2024-04-07T17:40:24.273768+0000 mon.a (mon.0) 4102 : cluster [DBG] osdmap e2163: 8 total, 8 up, 8 in 2024-04-07T17:40:25.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:40:25 smithi100 bash[74942]: cluster 2024-04-07T17:40:24.558061+0000 mgr.y (mgr.25175) 2551 : cluster [DBG] pgmap v4289: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 64 KiB/s rd, 5.2 KiB/s wr, 90 op/s 2024-04-07T17:40:25.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:40:25 smithi100 bash[76202]: cluster 2024-04-07T17:40:24.273768+0000 mon.a (mon.0) 4102 : cluster [DBG] osdmap e2163: 8 total, 8 up, 8 in 2024-04-07T17:40:25.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:40:25 smithi100 bash[76202]: cluster 2024-04-07T17:40:24.558061+0000 mgr.y (mgr.25175) 2551 : cluster [DBG] pgmap v4289: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 64 KiB/s rd, 5.2 KiB/s wr, 90 op/s 2024-04-07T17:40:26.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:40:26 smithi144 bash[66654]: cluster 2024-04-07T17:40:25.286756+0000 mon.a (mon.0) 4103 : cluster [DBG] osdmap e2164: 8 total, 8 up, 8 in 2024-04-07T17:40:26.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:40:26 smithi100 bash[74942]: cluster 2024-04-07T17:40:25.286756+0000 mon.a (mon.0) 4103 : cluster [DBG] osdmap e2164: 8 total, 8 up, 8 in 2024-04-07T17:40:26.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:40:26 smithi100 bash[76202]: cluster 2024-04-07T17:40:25.286756+0000 mon.a (mon.0) 4103 : cluster [DBG] osdmap e2164: 8 total, 8 up, 8 in 2024-04-07T17:40:27.614 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:40:27 smithi144 bash[66654]: cluster 2024-04-07T17:40:26.287873+0000 mon.a (mon.0) 4104 : cluster [DBG] osdmap e2165: 8 total, 8 up, 8 in 2024-04-07T17:40:27.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:40:27 smithi144 bash[66654]: cluster 2024-04-07T17:40:26.558803+0000 mgr.y (mgr.25175) 2552 : cluster [DBG] pgmap v4292: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 9.7 KiB/s rd, 2.0 KiB/s wr, 18 op/s 2024-04-07T17:40:27.706 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:40:27 smithi100 bash[74942]: cluster 2024-04-07T17:40:26.287873+0000 mon.a (mon.0) 4104 : cluster [DBG] osdmap e2165: 8 total, 8 up, 8 in 2024-04-07T17:40:27.706 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:40:27 smithi100 bash[74942]: cluster 2024-04-07T17:40:26.558803+0000 mgr.y (mgr.25175) 2552 : cluster [DBG] pgmap v4292: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 9.7 KiB/s rd, 2.0 KiB/s wr, 18 op/s 2024-04-07T17:40:27.706 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:40:27 smithi100 bash[76202]: cluster 2024-04-07T17:40:26.287873+0000 mon.a (mon.0) 4104 : cluster [DBG] osdmap e2165: 8 total, 8 up, 8 in 2024-04-07T17:40:27.706 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:40:27 smithi100 bash[76202]: cluster 2024-04-07T17:40:26.558803+0000 mgr.y (mgr.25175) 2552 : cluster [DBG] pgmap v4292: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 9.7 KiB/s rd, 2.0 KiB/s wr, 18 op/s 2024-04-07T17:40:28.474 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_limit_snaps PASSED [ 40%] 2024-04-07T17:40:28.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:40:28 smithi144 bash[66654]: cluster 2024-04-07T17:40:27.312128+0000 mon.a (mon.0) 4105 : cluster [DBG] osdmap e2166: 8 total, 8 up, 8 in 2024-04-07T17:40:28.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:40:28 smithi144 bash[66654]: audit 2024-04-07T17:40:27.434590+0000 mon.a (mon.0) 4106 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:40:28.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:40:28 smithi144 bash[66654]: audit 2024-04-07T17:40:28.165711+0000 mon.a (mon.0) 4107 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:40:28.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:40:28 smithi144 bash[66654]: audit 2024-04-07T17:40:28.167388+0000 mon.a (mon.0) 4108 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:40:28.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:40:28 smithi144 bash[66654]: audit 2024-04-07T17:40:28.175221+0000 mon.a (mon.0) 4109 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:40:28.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:40:28 smithi144 bash[66654]: cluster 2024-04-07T17:40:28.269531+0000 mon.a (mon.0) 4110 : cluster [DBG] osdmap e2167: 8 total, 8 up, 8 in 2024-04-07T17:40:28.703 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:40:28 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:40:28] "GET /metrics HTTP/1.1" 200 36732 "" "Prometheus/2.43.0" 2024-04-07T17:40:28.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:40:28 smithi100 bash[74942]: cluster 2024-04-07T17:40:27.312128+0000 mon.a (mon.0) 4105 : cluster [DBG] osdmap e2166: 8 total, 8 up, 8 in 2024-04-07T17:40:28.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:40:28 smithi100 bash[74942]: audit 2024-04-07T17:40:27.434590+0000 mon.a (mon.0) 4106 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:40:28.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:40:28 smithi100 bash[74942]: audit 2024-04-07T17:40:28.165711+0000 mon.a (mon.0) 4107 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:40:28.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:40:28 smithi100 bash[74942]: audit 2024-04-07T17:40:28.167388+0000 mon.a (mon.0) 4108 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:40:28.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:40:28 smithi100 bash[74942]: audit 2024-04-07T17:40:28.175221+0000 mon.a (mon.0) 4109 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:40:28.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:40:28 smithi100 bash[74942]: cluster 2024-04-07T17:40:28.269531+0000 mon.a (mon.0) 4110 : cluster [DBG] osdmap e2167: 8 total, 8 up, 8 in 2024-04-07T17:40:28.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:40:28 smithi100 bash[76202]: cluster 2024-04-07T17:40:27.312128+0000 mon.a (mon.0) 4105 : cluster [DBG] osdmap e2166: 8 total, 8 up, 8 in 2024-04-07T17:40:28.705 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:40:28 smithi100 bash[76202]: audit 2024-04-07T17:40:27.434590+0000 mon.a (mon.0) 4106 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:40:28.705 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:40:28 smithi100 bash[76202]: audit 2024-04-07T17:40:28.165711+0000 mon.a (mon.0) 4107 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:40:28.705 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:40:28 smithi100 bash[76202]: audit 2024-04-07T17:40:28.167388+0000 mon.a (mon.0) 4108 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:40:28.705 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:40:28 smithi100 bash[76202]: audit 2024-04-07T17:40:28.175221+0000 mon.a (mon.0) 4109 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:40:28.705 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:40:28 smithi100 bash[76202]: cluster 2024-04-07T17:40:28.269531+0000 mon.a (mon.0) 4110 : cluster [DBG] osdmap e2167: 8 total, 8 up, 8 in 2024-04-07T17:40:28.738 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_remove_with_exclusive_lock PASSED [ 40%] 2024-04-07T17:40:28.956 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_remove_with_snap SKIPPED [ 41%] 2024-04-07T17:40:29.185 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_remove_with_watcher SKIPPED [ 41%] 2024-04-07T17:40:29.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:40:29 smithi144 bash[66654]: cluster 2024-04-07T17:40:28.559556+0000 mgr.y (mgr.25175) 2553 : cluster [DBG] pgmap v4295: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 27 KiB/s rd, 3.0 KiB/s wr, 36 op/s 2024-04-07T17:40:29.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:40:29 smithi100 bash[74942]: cluster 2024-04-07T17:40:28.559556+0000 mgr.y (mgr.25175) 2553 : cluster [DBG] pgmap v4295: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 27 KiB/s rd, 3.0 KiB/s wr, 36 op/s 2024-04-07T17:40:29.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:40:29 smithi100 bash[76202]: cluster 2024-04-07T17:40:28.559556+0000 mgr.y (mgr.25175) 2553 : cluster [DBG] pgmap v4295: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 27 KiB/s rd, 3.0 KiB/s wr, 36 op/s 2024-04-07T17:40:30.544 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_rollback_to_snap PASSED [ 42%] 2024-04-07T17:40:30.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:40:30 smithi144 bash[66654]: cluster 2024-04-07T17:40:29.323459+0000 mon.a (mon.0) 4111 : cluster [DBG] osdmap e2168: 8 total, 8 up, 8 in 2024-04-07T17:40:30.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:40:30 smithi100 bash[74942]: cluster 2024-04-07T17:40:29.323459+0000 mon.a (mon.0) 4111 : cluster [DBG] osdmap e2168: 8 total, 8 up, 8 in 2024-04-07T17:40:30.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:40:30 smithi100 bash[76202]: cluster 2024-04-07T17:40:29.323459+0000 mon.a (mon.0) 4111 : cluster [DBG] osdmap e2168: 8 total, 8 up, 8 in 2024-04-07T17:40:31.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:40:31 smithi144 bash[66654]: cluster 2024-04-07T17:40:30.332456+0000 mon.a (mon.0) 4112 : cluster [DBG] osdmap e2169: 8 total, 8 up, 8 in 2024-04-07T17:40:31.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:40:31 smithi144 bash[66654]: cluster 2024-04-07T17:40:30.560327+0000 mgr.y (mgr.25175) 2554 : cluster [DBG] pgmap v4298: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 68 KiB/s rd, 6.5 KiB/s wr, 98 op/s 2024-04-07T17:40:31.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:40:31 smithi100 bash[74942]: cluster 2024-04-07T17:40:30.332456+0000 mon.a (mon.0) 4112 : cluster [DBG] osdmap e2169: 8 total, 8 up, 8 in 2024-04-07T17:40:31.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:40:31 smithi100 bash[74942]: cluster 2024-04-07T17:40:30.560327+0000 mgr.y (mgr.25175) 2554 : cluster [DBG] pgmap v4298: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 68 KiB/s rd, 6.5 KiB/s wr, 98 op/s 2024-04-07T17:40:31.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:40:31 smithi100 bash[76202]: cluster 2024-04-07T17:40:30.332456+0000 mon.a (mon.0) 4112 : cluster [DBG] osdmap e2169: 8 total, 8 up, 8 in 2024-04-07T17:40:31.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:40:31 smithi100 bash[76202]: cluster 2024-04-07T17:40:30.560327+0000 mgr.y (mgr.25175) 2554 : cluster [DBG] pgmap v4298: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 68 KiB/s rd, 6.5 KiB/s wr, 98 op/s 2024-04-07T17:40:32.566 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_rollback_to_snap_sparse PASSED [ 42%] 2024-04-07T17:40:32.615 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:40:32 smithi144 bash[95233]: ts=2024-04-07T17:40:32.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:40:32.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:40:32 smithi144 bash[66654]: cluster 2024-04-07T17:40:31.348872+0000 mon.a (mon.0) 4113 : cluster [DBG] osdmap e2170: 8 total, 8 up, 8 in 2024-04-07T17:40:32.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:40:32 smithi144 bash[66654]: audit 2024-04-07T17:40:31.863183+0000 mon.a (mon.0) 4114 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:40:32.705 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:40:32 smithi100 bash[74942]: cluster 2024-04-07T17:40:31.348872+0000 mon.a (mon.0) 4113 : cluster [DBG] osdmap e2170: 8 total, 8 up, 8 in 2024-04-07T17:40:32.705 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:40:32 smithi100 bash[74942]: audit 2024-04-07T17:40:31.863183+0000 mon.a (mon.0) 4114 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:40:32.706 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:40:32 smithi100 bash[76202]: cluster 2024-04-07T17:40:31.348872+0000 mon.a (mon.0) 4113 : cluster [DBG] osdmap e2170: 8 total, 8 up, 8 in 2024-04-07T17:40:32.706 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:40:32 smithi100 bash[76202]: audit 2024-04-07T17:40:31.863183+0000 mon.a (mon.0) 4114 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:40:33.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:40:33 smithi144 bash[66654]: cluster 2024-04-07T17:40:32.345526+0000 mon.a (mon.0) 4115 : cluster [DBG] osdmap e2171: 8 total, 8 up, 8 in 2024-04-07T17:40:33.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:40:33 smithi144 bash[66654]: cluster 2024-04-07T17:40:32.561085+0000 mgr.y (mgr.25175) 2555 : cluster [DBG] pgmap v4301: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 183 KiB/s rd, 12 KiB/s wr, 253 op/s 2024-04-07T17:40:33.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:40:33 smithi144 bash[66654]: cluster 2024-04-07T17:40:33.276498+0000 mon.a (mon.0) 4116 : cluster [DBG] osdmap e2172: 8 total, 8 up, 8 in 2024-04-07T17:40:33.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:40:33 smithi100 bash[74942]: cluster 2024-04-07T17:40:32.345526+0000 mon.a (mon.0) 4115 : cluster [DBG] osdmap e2171: 8 total, 8 up, 8 in 2024-04-07T17:40:33.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:40:33 smithi100 bash[74942]: cluster 2024-04-07T17:40:32.561085+0000 mgr.y (mgr.25175) 2555 : cluster [DBG] pgmap v4301: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 183 KiB/s rd, 12 KiB/s wr, 253 op/s 2024-04-07T17:40:33.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:40:33 smithi100 bash[74942]: cluster 2024-04-07T17:40:33.276498+0000 mon.a (mon.0) 4116 : cluster [DBG] osdmap e2172: 8 total, 8 up, 8 in 2024-04-07T17:40:33.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:40:33 smithi100 bash[76202]: cluster 2024-04-07T17:40:32.345526+0000 mon.a (mon.0) 4115 : cluster [DBG] osdmap e2171: 8 total, 8 up, 8 in 2024-04-07T17:40:33.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:40:33 smithi100 bash[76202]: cluster 2024-04-07T17:40:32.561085+0000 mgr.y (mgr.25175) 2555 : cluster [DBG] pgmap v4301: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 183 KiB/s rd, 12 KiB/s wr, 253 op/s 2024-04-07T17:40:33.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:40:33 smithi100 bash[76202]: cluster 2024-04-07T17:40:33.276498+0000 mon.a (mon.0) 4116 : cluster [DBG] osdmap e2172: 8 total, 8 up, 8 in 2024-04-07T17:40:35.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:40:35 smithi144 bash[66654]: cluster 2024-04-07T17:40:34.278351+0000 mon.a (mon.0) 4117 : cluster [DBG] osdmap e2173: 8 total, 8 up, 8 in 2024-04-07T17:40:35.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:40:35 smithi144 bash[66654]: cluster 2024-04-07T17:40:34.561891+0000 mgr.y (mgr.25175) 2556 : cluster [DBG] pgmap v4304: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 199 KiB/s rd, 16 KiB/s wr, 272 op/s 2024-04-07T17:40:35.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:40:35 smithi100 bash[74942]: cluster 2024-04-07T17:40:34.278351+0000 mon.a (mon.0) 4117 : cluster [DBG] osdmap e2173: 8 total, 8 up, 8 in 2024-04-07T17:40:35.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:40:35 smithi100 bash[74942]: cluster 2024-04-07T17:40:34.561891+0000 mgr.y (mgr.25175) 2556 : cluster [DBG] pgmap v4304: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 199 KiB/s rd, 16 KiB/s wr, 272 op/s 2024-04-07T17:40:35.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:40:35 smithi100 bash[76202]: cluster 2024-04-07T17:40:34.278351+0000 mon.a (mon.0) 4117 : cluster [DBG] osdmap e2173: 8 total, 8 up, 8 in 2024-04-07T17:40:35.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:40:35 smithi100 bash[76202]: cluster 2024-04-07T17:40:34.561891+0000 mgr.y (mgr.25175) 2556 : cluster [DBG] pgmap v4304: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 199 KiB/s rd, 16 KiB/s wr, 272 op/s 2024-04-07T17:40:36.518 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_rollback_with_resize PASSED [ 43%] 2024-04-07T17:40:36.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:40:36 smithi144 bash[66654]: cluster 2024-04-07T17:40:35.290407+0000 mon.a (mon.0) 4118 : cluster [DBG] osdmap e2174: 8 total, 8 up, 8 in 2024-04-07T17:40:36.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:40:36 smithi100 bash[74942]: cluster 2024-04-07T17:40:35.290407+0000 mon.a (mon.0) 4118 : cluster [DBG] osdmap e2174: 8 total, 8 up, 8 in 2024-04-07T17:40:36.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:40:36 smithi100 bash[76202]: cluster 2024-04-07T17:40:35.290407+0000 mon.a (mon.0) 4118 : cluster [DBG] osdmap e2174: 8 total, 8 up, 8 in 2024-04-07T17:40:37.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:40:37 smithi144 bash[66654]: cluster 2024-04-07T17:40:36.300808+0000 mon.a (mon.0) 4119 : cluster [DBG] osdmap e2175: 8 total, 8 up, 8 in 2024-04-07T17:40:37.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:40:37 smithi144 bash[66654]: cluster 2024-04-07T17:40:36.562880+0000 mgr.y (mgr.25175) 2557 : cluster [DBG] pgmap v4307: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 95 KiB/s rd, 9.5 KiB/s wr, 129 op/s 2024-04-07T17:40:37.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:40:37 smithi100 bash[74942]: cluster 2024-04-07T17:40:36.300808+0000 mon.a (mon.0) 4119 : cluster [DBG] osdmap e2175: 8 total, 8 up, 8 in 2024-04-07T17:40:37.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:40:37 smithi100 bash[74942]: cluster 2024-04-07T17:40:36.562880+0000 mgr.y (mgr.25175) 2557 : cluster [DBG] pgmap v4307: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 95 KiB/s rd, 9.5 KiB/s wr, 129 op/s 2024-04-07T17:40:37.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:40:37 smithi100 bash[76202]: cluster 2024-04-07T17:40:36.300808+0000 mon.a (mon.0) 4119 : cluster [DBG] osdmap e2175: 8 total, 8 up, 8 in 2024-04-07T17:40:37.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:40:37 smithi100 bash[76202]: cluster 2024-04-07T17:40:36.562880+0000 mgr.y (mgr.25175) 2557 : cluster [DBG] pgmap v4307: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 95 KiB/s rd, 9.5 KiB/s wr, 129 op/s 2024-04-07T17:40:38.521 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_set_snap PASSED [ 44%] 2024-04-07T17:40:38.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:40:38 smithi144 bash[66654]: cluster 2024-04-07T17:40:37.313383+0000 mon.a (mon.0) 4120 : cluster [DBG] osdmap e2176: 8 total, 8 up, 8 in 2024-04-07T17:40:38.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:40:38 smithi144 bash[66654]: cluster 2024-04-07T17:40:38.279969+0000 mon.a (mon.0) 4121 : cluster [DBG] osdmap e2177: 8 total, 8 up, 8 in 2024-04-07T17:40:38.704 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:40:38 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:40:38] "GET /metrics HTTP/1.1" 200 36737 "" "Prometheus/2.43.0" 2024-04-07T17:40:38.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:40:38 smithi100 bash[74942]: cluster 2024-04-07T17:40:37.313383+0000 mon.a (mon.0) 4120 : cluster [DBG] osdmap e2176: 8 total, 8 up, 8 in 2024-04-07T17:40:38.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:40:38 smithi100 bash[74942]: cluster 2024-04-07T17:40:38.279969+0000 mon.a (mon.0) 4121 : cluster [DBG] osdmap e2177: 8 total, 8 up, 8 in 2024-04-07T17:40:38.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:40:38 smithi100 bash[76202]: cluster 2024-04-07T17:40:37.313383+0000 mon.a (mon.0) 4120 : cluster [DBG] osdmap e2176: 8 total, 8 up, 8 in 2024-04-07T17:40:38.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:40:38 smithi100 bash[76202]: cluster 2024-04-07T17:40:38.279969+0000 mon.a (mon.0) 4121 : cluster [DBG] osdmap e2177: 8 total, 8 up, 8 in 2024-04-07T17:40:39.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:40:39 smithi144 bash[66654]: cluster 2024-04-07T17:40:38.563884+0000 mgr.y (mgr.25175) 2558 : cluster [DBG] pgmap v4310: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 68 KiB/s rd, 9.0 KiB/s wr, 98 op/s 2024-04-07T17:40:39.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:40:39 smithi100 bash[74942]: cluster 2024-04-07T17:40:38.563884+0000 mgr.y (mgr.25175) 2558 : cluster [DBG] pgmap v4310: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 68 KiB/s rd, 9.0 KiB/s wr, 98 op/s 2024-04-07T17:40:39.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:40:39 smithi100 bash[76202]: cluster 2024-04-07T17:40:38.563884+0000 mgr.y (mgr.25175) 2558 : cluster [DBG] pgmap v4310: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 68 KiB/s rd, 9.0 KiB/s wr, 98 op/s 2024-04-07T17:40:40.550 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_set_no_snap PASSED [ 44%] 2024-04-07T17:40:40.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:40:40 smithi144 bash[66654]: cluster 2024-04-07T17:40:39.323661+0000 mon.a (mon.0) 4122 : cluster [DBG] osdmap e2178: 8 total, 8 up, 8 in 2024-04-07T17:40:40.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:40:40 smithi100 bash[74942]: cluster 2024-04-07T17:40:39.323661+0000 mon.a (mon.0) 4122 : cluster [DBG] osdmap e2178: 8 total, 8 up, 8 in 2024-04-07T17:40:40.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:40:40 smithi100 bash[76202]: cluster 2024-04-07T17:40:39.323661+0000 mon.a (mon.0) 4122 : cluster [DBG] osdmap e2178: 8 total, 8 up, 8 in 2024-04-07T17:40:41.618 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:40:41 smithi144 bash[66654]: cluster 2024-04-07T17:40:40.330920+0000 mon.a (mon.0) 4123 : cluster [DBG] osdmap e2179: 8 total, 8 up, 8 in 2024-04-07T17:40:41.618 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:40:41 smithi144 bash[66654]: cluster 2024-04-07T17:40:40.564713+0000 mgr.y (mgr.25175) 2559 : cluster [DBG] pgmap v4313: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 78 KiB/s rd, 13 KiB/s wr, 121 op/s 2024-04-07T17:40:41.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:40:41 smithi100 bash[76202]: cluster 2024-04-07T17:40:40.330920+0000 mon.a (mon.0) 4123 : cluster [DBG] osdmap e2179: 8 total, 8 up, 8 in 2024-04-07T17:40:41.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:40:41 smithi100 bash[76202]: cluster 2024-04-07T17:40:40.564713+0000 mgr.y (mgr.25175) 2559 : cluster [DBG] pgmap v4313: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 78 KiB/s rd, 13 KiB/s wr, 121 op/s 2024-04-07T17:40:41.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:40:41 smithi100 bash[74942]: cluster 2024-04-07T17:40:40.330920+0000 mon.a (mon.0) 4123 : cluster [DBG] osdmap e2179: 8 total, 8 up, 8 in 2024-04-07T17:40:41.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:40:41 smithi100 bash[74942]: cluster 2024-04-07T17:40:40.564713+0000 mgr.y (mgr.25175) 2559 : cluster [DBG] pgmap v4313: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 78 KiB/s rd, 13 KiB/s wr, 121 op/s 2024-04-07T17:40:42.575 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_set_snap_by_id PASSED [ 45%] 2024-04-07T17:40:42.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:40:42 smithi144 bash[66654]: cluster 2024-04-07T17:40:41.346767+0000 mon.a (mon.0) 4124 : cluster [DBG] osdmap e2180: 8 total, 8 up, 8 in 2024-04-07T17:40:42.615 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:40:42 smithi144 bash[95233]: ts=2024-04-07T17:40:42.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:40:42.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:40:42 smithi100 bash[74942]: cluster 2024-04-07T17:40:41.346767+0000 mon.a (mon.0) 4124 : cluster [DBG] osdmap e2180: 8 total, 8 up, 8 in 2024-04-07T17:40:42.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:40:42 smithi100 bash[76202]: cluster 2024-04-07T17:40:41.346767+0000 mon.a (mon.0) 4124 : cluster [DBG] osdmap e2180: 8 total, 8 up, 8 in 2024-04-07T17:40:43.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:40:43 smithi144 bash[66654]: cluster 2024-04-07T17:40:42.350984+0000 mon.a (mon.0) 4125 : cluster [DBG] osdmap e2181: 8 total, 8 up, 8 in 2024-04-07T17:40:43.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:40:43 smithi144 bash[66654]: cluster 2024-04-07T17:40:42.565468+0000 mgr.y (mgr.25175) 2560 : cluster [DBG] pgmap v4316: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 110 KiB/s rd, 15 KiB/s wr, 163 op/s 2024-04-07T17:40:43.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:40:43 smithi144 bash[66654]: cluster 2024-04-07T17:40:43.281489+0000 mon.a (mon.0) 4126 : cluster [DBG] osdmap e2182: 8 total, 8 up, 8 in 2024-04-07T17:40:43.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:40:43 smithi100 bash[74942]: cluster 2024-04-07T17:40:42.350984+0000 mon.a (mon.0) 4125 : cluster [DBG] osdmap e2181: 8 total, 8 up, 8 in 2024-04-07T17:40:43.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:40:43 smithi100 bash[74942]: cluster 2024-04-07T17:40:42.565468+0000 mgr.y (mgr.25175) 2560 : cluster [DBG] pgmap v4316: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 110 KiB/s rd, 15 KiB/s wr, 163 op/s 2024-04-07T17:40:43.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:40:43 smithi100 bash[74942]: cluster 2024-04-07T17:40:43.281489+0000 mon.a (mon.0) 4126 : cluster [DBG] osdmap e2182: 8 total, 8 up, 8 in 2024-04-07T17:40:43.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:40:43 smithi100 bash[76202]: cluster 2024-04-07T17:40:42.350984+0000 mon.a (mon.0) 4125 : cluster [DBG] osdmap e2181: 8 total, 8 up, 8 in 2024-04-07T17:40:43.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:40:43 smithi100 bash[76202]: cluster 2024-04-07T17:40:42.565468+0000 mgr.y (mgr.25175) 2560 : cluster [DBG] pgmap v4316: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 110 KiB/s rd, 15 KiB/s wr, 163 op/s 2024-04-07T17:40:43.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:40:43 smithi100 bash[76202]: cluster 2024-04-07T17:40:43.281489+0000 mon.a (mon.0) 4126 : cluster [DBG] osdmap e2182: 8 total, 8 up, 8 in 2024-04-07T17:40:45.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:40:45 smithi144 bash[66654]: cluster 2024-04-07T17:40:44.283760+0000 mon.a (mon.0) 4127 : cluster [DBG] osdmap e2183: 8 total, 8 up, 8 in 2024-04-07T17:40:45.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:40:45 smithi144 bash[66654]: cluster 2024-04-07T17:40:44.566234+0000 mgr.y (mgr.25175) 2561 : cluster [DBG] pgmap v4319: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 101 KiB/s rd, 13 KiB/s wr, 146 op/s 2024-04-07T17:40:45.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:40:45 smithi100 bash[74942]: cluster 2024-04-07T17:40:44.283760+0000 mon.a (mon.0) 4127 : cluster [DBG] osdmap e2183: 8 total, 8 up, 8 in 2024-04-07T17:40:45.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:40:45 smithi100 bash[74942]: cluster 2024-04-07T17:40:44.566234+0000 mgr.y (mgr.25175) 2561 : cluster [DBG] pgmap v4319: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 101 KiB/s rd, 13 KiB/s wr, 146 op/s 2024-04-07T17:40:45.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:40:45 smithi100 bash[76202]: cluster 2024-04-07T17:40:44.283760+0000 mon.a (mon.0) 4127 : cluster [DBG] osdmap e2183: 8 total, 8 up, 8 in 2024-04-07T17:40:45.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:40:45 smithi100 bash[76202]: cluster 2024-04-07T17:40:44.566234+0000 mgr.y (mgr.25175) 2561 : cluster [DBG] pgmap v4319: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 101 KiB/s rd, 13 KiB/s wr, 146 op/s 2024-04-07T17:40:46.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:40:46 smithi144 bash[66654]: cluster 2024-04-07T17:40:45.296447+0000 mon.a (mon.0) 4128 : cluster [DBG] osdmap e2184: 8 total, 8 up, 8 in 2024-04-07T17:40:46.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:40:46 smithi100 bash[74942]: cluster 2024-04-07T17:40:45.296447+0000 mon.a (mon.0) 4128 : cluster [DBG] osdmap e2184: 8 total, 8 up, 8 in 2024-04-07T17:40:46.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:40:46 smithi100 bash[76202]: cluster 2024-04-07T17:40:45.296447+0000 mon.a (mon.0) 4128 : cluster [DBG] osdmap e2184: 8 total, 8 up, 8 in 2024-04-07T17:40:47.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:40:47 smithi144 bash[66654]: cluster 2024-04-07T17:40:46.302325+0000 mon.a (mon.0) 4129 : cluster [DBG] osdmap e2185: 8 total, 8 up, 8 in 2024-04-07T17:40:47.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:40:47 smithi144 bash[66654]: cluster 2024-04-07T17:40:46.566993+0000 mgr.y (mgr.25175) 2562 : cluster [DBG] pgmap v4322: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 39 KiB/s rd, 5.0 KiB/s wr, 54 op/s 2024-04-07T17:40:47.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:40:47 smithi144 bash[66654]: audit 2024-04-07T17:40:46.863603+0000 mon.a (mon.0) 4130 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:40:47.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:40:47 smithi100 bash[74942]: cluster 2024-04-07T17:40:46.302325+0000 mon.a (mon.0) 4129 : cluster [DBG] osdmap e2185: 8 total, 8 up, 8 in 2024-04-07T17:40:47.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:40:47 smithi100 bash[74942]: cluster 2024-04-07T17:40:46.566993+0000 mgr.y (mgr.25175) 2562 : cluster [DBG] pgmap v4322: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 39 KiB/s rd, 5.0 KiB/s wr, 54 op/s 2024-04-07T17:40:47.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:40:47 smithi100 bash[74942]: audit 2024-04-07T17:40:46.863603+0000 mon.a (mon.0) 4130 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:40:47.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:40:47 smithi100 bash[76202]: cluster 2024-04-07T17:40:46.302325+0000 mon.a (mon.0) 4129 : cluster [DBG] osdmap e2185: 8 total, 8 up, 8 in 2024-04-07T17:40:47.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:40:47 smithi100 bash[76202]: cluster 2024-04-07T17:40:46.566993+0000 mgr.y (mgr.25175) 2562 : cluster [DBG] pgmap v4322: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 39 KiB/s rd, 5.0 KiB/s wr, 54 op/s 2024-04-07T17:40:47.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:40:47 smithi100 bash[76202]: audit 2024-04-07T17:40:46.863603+0000 mon.a (mon.0) 4130 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:40:48.480 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_snap_get_name PASSED [ 45%] 2024-04-07T17:40:48.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:40:48 smithi144 bash[66654]: cluster 2024-04-07T17:40:47.319745+0000 mon.a (mon.0) 4131 : cluster [DBG] osdmap e2186: 8 total, 8 up, 8 in 2024-04-07T17:40:48.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:40:48 smithi144 bash[66654]: cluster 2024-04-07T17:40:48.285392+0000 mon.a (mon.0) 4132 : cluster [DBG] osdmap e2187: 8 total, 8 up, 8 in 2024-04-07T17:40:48.704 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:40:48 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:40:48] "GET /metrics HTTP/1.1" 200 36737 "" "Prometheus/2.43.0" 2024-04-07T17:40:48.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:40:48 smithi100 bash[74942]: cluster 2024-04-07T17:40:47.319745+0000 mon.a (mon.0) 4131 : cluster [DBG] osdmap e2186: 8 total, 8 up, 8 in 2024-04-07T17:40:48.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:40:48 smithi100 bash[74942]: cluster 2024-04-07T17:40:48.285392+0000 mon.a (mon.0) 4132 : cluster [DBG] osdmap e2187: 8 total, 8 up, 8 in 2024-04-07T17:40:48.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:40:48 smithi100 bash[76202]: cluster 2024-04-07T17:40:47.319745+0000 mon.a (mon.0) 4131 : cluster [DBG] osdmap e2186: 8 total, 8 up, 8 in 2024-04-07T17:40:48.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:40:48 smithi100 bash[76202]: cluster 2024-04-07T17:40:48.285392+0000 mon.a (mon.0) 4132 : cluster [DBG] osdmap e2187: 8 total, 8 up, 8 in 2024-04-07T17:40:49.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:40:49 smithi144 bash[66654]: cluster 2024-04-07T17:40:48.567783+0000 mgr.y (mgr.25175) 2563 : cluster [DBG] pgmap v4325: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 69 KiB/s rd, 5.5 KiB/s wr, 93 op/s 2024-04-07T17:40:49.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:40:49 smithi100 bash[74942]: cluster 2024-04-07T17:40:48.567783+0000 mgr.y (mgr.25175) 2563 : cluster [DBG] pgmap v4325: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 69 KiB/s rd, 5.5 KiB/s wr, 93 op/s 2024-04-07T17:40:49.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:40:49 smithi100 bash[76202]: cluster 2024-04-07T17:40:48.567783+0000 mgr.y (mgr.25175) 2563 : cluster [DBG] pgmap v4325: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 69 KiB/s rd, 5.5 KiB/s wr, 93 op/s 2024-04-07T17:40:50.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:40:50 smithi144 bash[66654]: cluster 2024-04-07T17:40:49.331072+0000 mon.a (mon.0) 4133 : cluster [DBG] osdmap e2188: 8 total, 8 up, 8 in 2024-04-07T17:40:50.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:40:50 smithi100 bash[74942]: cluster 2024-04-07T17:40:49.331072+0000 mon.a (mon.0) 4133 : cluster [DBG] osdmap e2188: 8 total, 8 up, 8 in 2024-04-07T17:40:50.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:40:50 smithi100 bash[76202]: cluster 2024-04-07T17:40:49.331072+0000 mon.a (mon.0) 4133 : cluster [DBG] osdmap e2188: 8 total, 8 up, 8 in 2024-04-07T17:40:51.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:40:51 smithi144 bash[66654]: cluster 2024-04-07T17:40:50.332778+0000 mon.a (mon.0) 4134 : cluster [DBG] osdmap e2189: 8 total, 8 up, 8 in 2024-04-07T17:40:51.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:40:51 smithi144 bash[66654]: cluster 2024-04-07T17:40:50.568564+0000 mgr.y (mgr.25175) 2564 : cluster [DBG] pgmap v4328: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 71 KiB/s rd, 6.7 KiB/s wr, 99 op/s 2024-04-07T17:40:51.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:40:51 smithi100 bash[74942]: cluster 2024-04-07T17:40:50.332778+0000 mon.a (mon.0) 4134 : cluster [DBG] osdmap e2189: 8 total, 8 up, 8 in 2024-04-07T17:40:51.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:40:51 smithi100 bash[74942]: cluster 2024-04-07T17:40:50.568564+0000 mgr.y (mgr.25175) 2564 : cluster [DBG] pgmap v4328: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 71 KiB/s rd, 6.7 KiB/s wr, 99 op/s 2024-04-07T17:40:51.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:40:51 smithi100 bash[76202]: cluster 2024-04-07T17:40:50.332778+0000 mon.a (mon.0) 4134 : cluster [DBG] osdmap e2189: 8 total, 8 up, 8 in 2024-04-07T17:40:51.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:40:51 smithi100 bash[76202]: cluster 2024-04-07T17:40:50.568564+0000 mgr.y (mgr.25175) 2564 : cluster [DBG] pgmap v4328: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 71 KiB/s rd, 6.7 KiB/s wr, 99 op/s 2024-04-07T17:40:52.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:40:52 smithi144 bash[66654]: cluster 2024-04-07T17:40:51.361142+0000 mon.a (mon.0) 4135 : cluster [DBG] osdmap e2190: 8 total, 8 up, 8 in 2024-04-07T17:40:52.615 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:40:52 smithi144 bash[95233]: ts=2024-04-07T17:40:52.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:40:52.703 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:40:52 smithi100 bash[74942]: cluster 2024-04-07T17:40:51.361142+0000 mon.a (mon.0) 4135 : cluster [DBG] osdmap e2190: 8 total, 8 up, 8 in 2024-04-07T17:40:52.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:40:52 smithi100 bash[76202]: cluster 2024-04-07T17:40:51.361142+0000 mon.a (mon.0) 4135 : cluster [DBG] osdmap e2190: 8 total, 8 up, 8 in 2024-04-07T17:40:53.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:40:53 smithi100 bash[74942]: cluster 2024-04-07T17:40:52.361581+0000 mon.a (mon.0) 4136 : cluster [DBG] osdmap e2191: 8 total, 8 up, 8 in 2024-04-07T17:40:53.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:40:53 smithi100 bash[74942]: cluster 2024-04-07T17:40:52.569299+0000 mgr.y (mgr.25175) 2565 : cluster [DBG] pgmap v4331: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 37 KiB/s rd, 3.0 KiB/s wr, 50 op/s 2024-04-07T17:40:53.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:40:53 smithi100 bash[74942]: cluster 2024-04-07T17:40:53.286746+0000 mon.a (mon.0) 4137 : cluster [DBG] osdmap e2192: 8 total, 8 up, 8 in 2024-04-07T17:40:53.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:40:53 smithi100 bash[76202]: cluster 2024-04-07T17:40:52.361581+0000 mon.a (mon.0) 4136 : cluster [DBG] osdmap e2191: 8 total, 8 up, 8 in 2024-04-07T17:40:53.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:40:53 smithi100 bash[76202]: cluster 2024-04-07T17:40:52.569299+0000 mgr.y (mgr.25175) 2565 : cluster [DBG] pgmap v4331: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 37 KiB/s rd, 3.0 KiB/s wr, 50 op/s 2024-04-07T17:40:53.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:40:53 smithi100 bash[76202]: cluster 2024-04-07T17:40:53.286746+0000 mon.a (mon.0) 4137 : cluster [DBG] osdmap e2192: 8 total, 8 up, 8 in 2024-04-07T17:40:53.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:40:53 smithi144 bash[66654]: cluster 2024-04-07T17:40:52.361581+0000 mon.a (mon.0) 4136 : cluster [DBG] osdmap e2191: 8 total, 8 up, 8 in 2024-04-07T17:40:53.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:40:53 smithi144 bash[66654]: cluster 2024-04-07T17:40:52.569299+0000 mgr.y (mgr.25175) 2565 : cluster [DBG] pgmap v4331: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 37 KiB/s rd, 3.0 KiB/s wr, 50 op/s 2024-04-07T17:40:53.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:40:53 smithi144 bash[66654]: cluster 2024-04-07T17:40:53.286746+0000 mon.a (mon.0) 4137 : cluster [DBG] osdmap e2192: 8 total, 8 up, 8 in 2024-04-07T17:40:54.496 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_snap_get_id PASSED [ 46%] 2024-04-07T17:40:55.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:40:55 smithi144 bash[66654]: cluster 2024-04-07T17:40:54.290658+0000 mon.a (mon.0) 4138 : cluster [DBG] osdmap e2193: 8 total, 8 up, 8 in 2024-04-07T17:40:55.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:40:55 smithi144 bash[66654]: cluster 2024-04-07T17:40:54.570049+0000 mgr.y (mgr.25175) 2566 : cluster [DBG] pgmap v4334: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 80 KiB/s rd, 5.7 KiB/s wr, 106 op/s 2024-04-07T17:40:55.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:40:55 smithi100 bash[74942]: cluster 2024-04-07T17:40:54.290658+0000 mon.a (mon.0) 4138 : cluster [DBG] osdmap e2193: 8 total, 8 up, 8 in 2024-04-07T17:40:55.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:40:55 smithi100 bash[74942]: cluster 2024-04-07T17:40:54.570049+0000 mgr.y (mgr.25175) 2566 : cluster [DBG] pgmap v4334: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 80 KiB/s rd, 5.7 KiB/s wr, 106 op/s 2024-04-07T17:40:55.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:40:55 smithi100 bash[76202]: cluster 2024-04-07T17:40:54.290658+0000 mon.a (mon.0) 4138 : cluster [DBG] osdmap e2193: 8 total, 8 up, 8 in 2024-04-07T17:40:55.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:40:55 smithi100 bash[76202]: cluster 2024-04-07T17:40:54.570049+0000 mgr.y (mgr.25175) 2566 : cluster [DBG] pgmap v4334: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 80 KiB/s rd, 5.7 KiB/s wr, 106 op/s 2024-04-07T17:40:56.517 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_set_snap_sparse PASSED [ 46%] 2024-04-07T17:40:56.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:40:56 smithi144 bash[66654]: cluster 2024-04-07T17:40:55.299364+0000 mon.a (mon.0) 4139 : cluster [DBG] osdmap e2194: 8 total, 8 up, 8 in 2024-04-07T17:40:56.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:40:56 smithi100 bash[74942]: cluster 2024-04-07T17:40:55.299364+0000 mon.a (mon.0) 4139 : cluster [DBG] osdmap e2194: 8 total, 8 up, 8 in 2024-04-07T17:40:56.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:40:56 smithi100 bash[76202]: cluster 2024-04-07T17:40:55.299364+0000 mon.a (mon.0) 4139 : cluster [DBG] osdmap e2194: 8 total, 8 up, 8 in 2024-04-07T17:40:57.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:40:57 smithi144 bash[66654]: cluster 2024-04-07T17:40:56.310435+0000 mon.a (mon.0) 4140 : cluster [DBG] osdmap e2195: 8 total, 8 up, 8 in 2024-04-07T17:40:57.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:40:57 smithi144 bash[66654]: cluster 2024-04-07T17:40:56.570797+0000 mgr.y (mgr.25175) 2567 : cluster [DBG] pgmap v4337: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 45 KiB/s rd, 4.2 KiB/s wr, 63 op/s 2024-04-07T17:40:57.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:40:57 smithi100 bash[74942]: cluster 2024-04-07T17:40:56.310435+0000 mon.a (mon.0) 4140 : cluster [DBG] osdmap e2195: 8 total, 8 up, 8 in 2024-04-07T17:40:57.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:40:57 smithi100 bash[74942]: cluster 2024-04-07T17:40:56.570797+0000 mgr.y (mgr.25175) 2567 : cluster [DBG] pgmap v4337: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 45 KiB/s rd, 4.2 KiB/s wr, 63 op/s 2024-04-07T17:40:57.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:40:57 smithi100 bash[76202]: cluster 2024-04-07T17:40:56.310435+0000 mon.a (mon.0) 4140 : cluster [DBG] osdmap e2195: 8 total, 8 up, 8 in 2024-04-07T17:40:57.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:40:57 smithi100 bash[76202]: cluster 2024-04-07T17:40:56.570797+0000 mgr.y (mgr.25175) 2567 : cluster [DBG] pgmap v4337: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 45 KiB/s rd, 4.2 KiB/s wr, 63 op/s 2024-04-07T17:40:58.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:40:58 smithi144 bash[66654]: cluster 2024-04-07T17:40:57.315864+0000 mon.a (mon.0) 4141 : cluster [DBG] osdmap e2196: 8 total, 8 up, 8 in 2024-04-07T17:40:58.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:40:58 smithi144 bash[66654]: cluster 2024-04-07T17:40:58.291562+0000 mon.a (mon.0) 4142 : cluster [DBG] osdmap e2197: 8 total, 8 up, 8 in 2024-04-07T17:40:58.704 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:40:58 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:40:58] "GET /metrics HTTP/1.1" 200 36737 "" "Prometheus/2.43.0" 2024-04-07T17:40:58.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:40:58 smithi100 bash[74942]: cluster 2024-04-07T17:40:57.315864+0000 mon.a (mon.0) 4141 : cluster [DBG] osdmap e2196: 8 total, 8 up, 8 in 2024-04-07T17:40:58.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:40:58 smithi100 bash[74942]: cluster 2024-04-07T17:40:58.291562+0000 mon.a (mon.0) 4142 : cluster [DBG] osdmap e2197: 8 total, 8 up, 8 in 2024-04-07T17:40:58.705 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:40:58 smithi100 bash[76202]: cluster 2024-04-07T17:40:57.315864+0000 mon.a (mon.0) 4141 : cluster [DBG] osdmap e2196: 8 total, 8 up, 8 in 2024-04-07T17:40:58.705 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:40:58 smithi100 bash[76202]: cluster 2024-04-07T17:40:58.291562+0000 mon.a (mon.0) 4142 : cluster [DBG] osdmap e2197: 8 total, 8 up, 8 in 2024-04-07T17:40:59.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:40:59 smithi144 bash[66654]: cluster 2024-04-07T17:40:58.571584+0000 mgr.y (mgr.25175) 2568 : cluster [DBG] pgmap v4340: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 81 KiB/s rd, 6.2 KiB/s wr, 109 op/s 2024-04-07T17:40:59.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:40:59 smithi100 bash[74942]: cluster 2024-04-07T17:40:58.571584+0000 mgr.y (mgr.25175) 2568 : cluster [DBG] pgmap v4340: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 81 KiB/s rd, 6.2 KiB/s wr, 109 op/s 2024-04-07T17:40:59.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:40:59 smithi100 bash[76202]: cluster 2024-04-07T17:40:58.571584+0000 mgr.y (mgr.25175) 2568 : cluster [DBG] pgmap v4340: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 81 KiB/s rd, 6.2 KiB/s wr, 109 op/s 2024-04-07T17:41:00.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:41:00 smithi144 bash[66654]: cluster 2024-04-07T17:40:59.343150+0000 mon.a (mon.0) 4143 : cluster [DBG] osdmap e2198: 8 total, 8 up, 8 in 2024-04-07T17:41:00.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:41:00 smithi100 bash[74942]: cluster 2024-04-07T17:40:59.343150+0000 mon.a (mon.0) 4143 : cluster [DBG] osdmap e2198: 8 total, 8 up, 8 in 2024-04-07T17:41:00.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:41:00 smithi100 bash[76202]: cluster 2024-04-07T17:40:59.343150+0000 mon.a (mon.0) 4143 : cluster [DBG] osdmap e2198: 8 total, 8 up, 8 in 2024-04-07T17:41:01.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:41:01 smithi144 bash[66654]: cluster 2024-04-07T17:41:00.343144+0000 mon.a (mon.0) 4144 : cluster [DBG] osdmap e2199: 8 total, 8 up, 8 in 2024-04-07T17:41:01.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:41:01 smithi144 bash[66654]: cluster 2024-04-07T17:41:00.572281+0000 mgr.y (mgr.25175) 2569 : cluster [DBG] pgmap v4343: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 106 KiB/s rd, 9.2 KiB/s wr, 146 op/s 2024-04-07T17:41:01.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:41:01 smithi100 bash[74942]: cluster 2024-04-07T17:41:00.343144+0000 mon.a (mon.0) 4144 : cluster [DBG] osdmap e2199: 8 total, 8 up, 8 in 2024-04-07T17:41:01.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:41:01 smithi100 bash[74942]: cluster 2024-04-07T17:41:00.572281+0000 mgr.y (mgr.25175) 2569 : cluster [DBG] pgmap v4343: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 106 KiB/s rd, 9.2 KiB/s wr, 146 op/s 2024-04-07T17:41:01.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:41:01 smithi100 bash[76202]: cluster 2024-04-07T17:41:00.343144+0000 mon.a (mon.0) 4144 : cluster [DBG] osdmap e2199: 8 total, 8 up, 8 in 2024-04-07T17:41:01.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:41:01 smithi100 bash[76202]: cluster 2024-04-07T17:41:00.572281+0000 mgr.y (mgr.25175) 2569 : cluster [DBG] pgmap v4343: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 106 KiB/s rd, 9.2 KiB/s wr, 146 op/s 2024-04-07T17:41:02.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:41:02 smithi144 bash[66654]: cluster 2024-04-07T17:41:01.366502+0000 mon.a (mon.0) 4145 : cluster [DBG] osdmap e2200: 8 total, 8 up, 8 in 2024-04-07T17:41:02.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:41:02 smithi144 bash[66654]: audit 2024-04-07T17:41:01.864155+0000 mon.a (mon.0) 4146 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:41:02.615 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:41:02 smithi144 bash[95233]: ts=2024-04-07T17:41:02.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:41:02.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:41:02 smithi100 bash[74942]: cluster 2024-04-07T17:41:01.366502+0000 mon.a (mon.0) 4145 : cluster [DBG] osdmap e2200: 8 total, 8 up, 8 in 2024-04-07T17:41:02.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:41:02 smithi100 bash[74942]: audit 2024-04-07T17:41:01.864155+0000 mon.a (mon.0) 4146 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:41:02.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:41:02 smithi100 bash[76202]: cluster 2024-04-07T17:41:01.366502+0000 mon.a (mon.0) 4145 : cluster [DBG] osdmap e2200: 8 total, 8 up, 8 in 2024-04-07T17:41:02.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:41:02 smithi100 bash[76202]: audit 2024-04-07T17:41:01.864155+0000 mon.a (mon.0) 4146 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:41:03.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:41:03 smithi100 bash[74942]: cluster 2024-04-07T17:41:02.374652+0000 mon.a (mon.0) 4147 : cluster [DBG] osdmap e2201: 8 total, 8 up, 8 in 2024-04-07T17:41:03.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:41:03 smithi100 bash[74942]: cluster 2024-04-07T17:41:02.573035+0000 mgr.y (mgr.25175) 2570 : cluster [DBG] pgmap v4346: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 30 KiB/s rd, 4.0 KiB/s wr, 45 op/s 2024-04-07T17:41:03.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:41:03 smithi100 bash[74942]: cluster 2024-04-07T17:41:03.290773+0000 mon.a (mon.0) 4148 : cluster [DBG] osdmap e2202: 8 total, 8 up, 8 in 2024-04-07T17:41:03.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:41:03 smithi100 bash[76202]: cluster 2024-04-07T17:41:02.374652+0000 mon.a (mon.0) 4147 : cluster [DBG] osdmap e2201: 8 total, 8 up, 8 in 2024-04-07T17:41:03.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:41:03 smithi100 bash[76202]: cluster 2024-04-07T17:41:02.573035+0000 mgr.y (mgr.25175) 2570 : cluster [DBG] pgmap v4346: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 30 KiB/s rd, 4.0 KiB/s wr, 45 op/s 2024-04-07T17:41:03.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:41:03 smithi100 bash[76202]: cluster 2024-04-07T17:41:03.290773+0000 mon.a (mon.0) 4148 : cluster [DBG] osdmap e2202: 8 total, 8 up, 8 in 2024-04-07T17:41:03.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:41:03 smithi144 bash[66654]: cluster 2024-04-07T17:41:02.374652+0000 mon.a (mon.0) 4147 : cluster [DBG] osdmap e2201: 8 total, 8 up, 8 in 2024-04-07T17:41:03.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:41:03 smithi144 bash[66654]: cluster 2024-04-07T17:41:02.573035+0000 mgr.y (mgr.25175) 2570 : cluster [DBG] pgmap v4346: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 30 KiB/s rd, 4.0 KiB/s wr, 45 op/s 2024-04-07T17:41:03.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:41:03 smithi144 bash[66654]: cluster 2024-04-07T17:41:03.290773+0000 mon.a (mon.0) 4148 : cluster [DBG] osdmap e2202: 8 total, 8 up, 8 in 2024-04-07T17:41:05.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:41:05 smithi144 bash[66654]: cluster 2024-04-07T17:41:04.293777+0000 mon.a (mon.0) 4149 : cluster [DBG] osdmap e2203: 8 total, 8 up, 8 in 2024-04-07T17:41:05.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:41:05 smithi144 bash[66654]: cluster 2024-04-07T17:41:04.573749+0000 mgr.y (mgr.25175) 2571 : cluster [DBG] pgmap v4349: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 61 KiB/s rd, 4.0 KiB/s wr, 81 op/s 2024-04-07T17:41:05.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:41:05 smithi100 bash[74942]: cluster 2024-04-07T17:41:04.293777+0000 mon.a (mon.0) 4149 : cluster [DBG] osdmap e2203: 8 total, 8 up, 8 in 2024-04-07T17:41:05.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:41:05 smithi100 bash[74942]: cluster 2024-04-07T17:41:04.573749+0000 mgr.y (mgr.25175) 2571 : cluster [DBG] pgmap v4349: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 61 KiB/s rd, 4.0 KiB/s wr, 81 op/s 2024-04-07T17:41:05.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:41:05 smithi100 bash[76202]: cluster 2024-04-07T17:41:04.293777+0000 mon.a (mon.0) 4149 : cluster [DBG] osdmap e2203: 8 total, 8 up, 8 in 2024-04-07T17:41:05.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:41:05 smithi100 bash[76202]: cluster 2024-04-07T17:41:04.573749+0000 mgr.y (mgr.25175) 2571 : cluster [DBG] pgmap v4349: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 61 KiB/s rd, 4.0 KiB/s wr, 81 op/s 2024-04-07T17:41:06.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:41:06 smithi144 bash[66654]: cluster 2024-04-07T17:41:05.306607+0000 mon.a (mon.0) 4150 : cluster [DBG] osdmap e2204: 8 total, 8 up, 8 in 2024-04-07T17:41:06.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:41:06 smithi100 bash[74942]: cluster 2024-04-07T17:41:05.306607+0000 mon.a (mon.0) 4150 : cluster [DBG] osdmap e2204: 8 total, 8 up, 8 in 2024-04-07T17:41:06.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:41:06 smithi100 bash[76202]: cluster 2024-04-07T17:41:05.306607+0000 mon.a (mon.0) 4150 : cluster [DBG] osdmap e2204: 8 total, 8 up, 8 in 2024-04-07T17:41:07.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:41:07 smithi144 bash[66654]: cluster 2024-04-07T17:41:06.321106+0000 mon.a (mon.0) 4151 : cluster [DBG] osdmap e2205: 8 total, 8 up, 8 in 2024-04-07T17:41:07.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:41:07 smithi144 bash[66654]: cluster 2024-04-07T17:41:06.574455+0000 mgr.y (mgr.25175) 2572 : cluster [DBG] pgmap v4352: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 56 KiB/s rd, 3.2 KiB/s wr, 74 op/s 2024-04-07T17:41:07.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:41:07 smithi100 bash[74942]: cluster 2024-04-07T17:41:06.321106+0000 mon.a (mon.0) 4151 : cluster [DBG] osdmap e2205: 8 total, 8 up, 8 in 2024-04-07T17:41:07.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:41:07 smithi100 bash[74942]: cluster 2024-04-07T17:41:06.574455+0000 mgr.y (mgr.25175) 2572 : cluster [DBG] pgmap v4352: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 56 KiB/s rd, 3.2 KiB/s wr, 74 op/s 2024-04-07T17:41:07.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:41:07 smithi100 bash[76202]: cluster 2024-04-07T17:41:06.321106+0000 mon.a (mon.0) 4151 : cluster [DBG] osdmap e2205: 8 total, 8 up, 8 in 2024-04-07T17:41:07.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:41:07 smithi100 bash[76202]: cluster 2024-04-07T17:41:06.574455+0000 mgr.y (mgr.25175) 2572 : cluster [DBG] pgmap v4352: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 56 KiB/s rd, 3.2 KiB/s wr, 74 op/s 2024-04-07T17:41:08.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:41:08 smithi144 bash[66654]: cluster 2024-04-07T17:41:07.341127+0000 mon.a (mon.0) 4152 : cluster [DBG] osdmap e2206: 8 total, 8 up, 8 in 2024-04-07T17:41:08.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:41:08 smithi100 bash[74942]: cluster 2024-04-07T17:41:07.341127+0000 mon.a (mon.0) 4152 : cluster [DBG] osdmap e2206: 8 total, 8 up, 8 in 2024-04-07T17:41:08.704 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:41:08 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:41:08] "GET /metrics HTTP/1.1" 200 36735 "" "Prometheus/2.43.0" 2024-04-07T17:41:08.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:41:08 smithi100 bash[76202]: cluster 2024-04-07T17:41:07.341127+0000 mon.a (mon.0) 4152 : cluster [DBG] osdmap e2206: 8 total, 8 up, 8 in 2024-04-07T17:41:09.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:41:09 smithi144 bash[66654]: cluster 2024-04-07T17:41:08.343803+0000 mon.a (mon.0) 4153 : cluster [DBG] osdmap e2207: 8 total, 8 up, 8 in 2024-04-07T17:41:09.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:41:09 smithi144 bash[66654]: cluster 2024-04-07T17:41:08.575205+0000 mgr.y (mgr.25175) 2573 : cluster [DBG] pgmap v4355: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 5.0 KiB/s rd, 1023 B/s wr, 7 op/s 2024-04-07T17:41:09.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:41:09 smithi100 bash[74942]: cluster 2024-04-07T17:41:08.343803+0000 mon.a (mon.0) 4153 : cluster [DBG] osdmap e2207: 8 total, 8 up, 8 in 2024-04-07T17:41:09.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:41:09 smithi100 bash[74942]: cluster 2024-04-07T17:41:08.575205+0000 mgr.y (mgr.25175) 2573 : cluster [DBG] pgmap v4355: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 5.0 KiB/s rd, 1023 B/s wr, 7 op/s 2024-04-07T17:41:09.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:41:09 smithi100 bash[76202]: cluster 2024-04-07T17:41:08.343803+0000 mon.a (mon.0) 4153 : cluster [DBG] osdmap e2207: 8 total, 8 up, 8 in 2024-04-07T17:41:09.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:41:09 smithi100 bash[76202]: cluster 2024-04-07T17:41:08.575205+0000 mgr.y (mgr.25175) 2573 : cluster [DBG] pgmap v4355: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 5.0 KiB/s rd, 1023 B/s wr, 7 op/s 2024-04-07T17:41:10.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:41:10 smithi144 bash[66654]: cluster 2024-04-07T17:41:09.355094+0000 mon.a (mon.0) 4154 : cluster [DBG] osdmap e2208: 8 total, 8 up, 8 in 2024-04-07T17:41:10.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:41:10 smithi100 bash[74942]: cluster 2024-04-07T17:41:09.355094+0000 mon.a (mon.0) 4154 : cluster [DBG] osdmap e2208: 8 total, 8 up, 8 in 2024-04-07T17:41:10.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:41:10 smithi100 bash[76202]: cluster 2024-04-07T17:41:09.355094+0000 mon.a (mon.0) 4154 : cluster [DBG] osdmap e2208: 8 total, 8 up, 8 in 2024-04-07T17:41:11.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:41:11 smithi100 bash[74942]: cluster 2024-04-07T17:41:10.356122+0000 mon.a (mon.0) 4155 : cluster [DBG] osdmap e2209: 8 total, 8 up, 8 in 2024-04-07T17:41:11.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:41:11 smithi100 bash[74942]: cluster 2024-04-07T17:41:10.575937+0000 mgr.y (mgr.25175) 2574 : cluster [DBG] pgmap v4358: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 93 KiB/s rd, 3.7 KiB/s wr, 124 op/s 2024-04-07T17:41:11.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:41:11 smithi100 bash[76202]: cluster 2024-04-07T17:41:10.356122+0000 mon.a (mon.0) 4155 : cluster [DBG] osdmap e2209: 8 total, 8 up, 8 in 2024-04-07T17:41:11.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:41:11 smithi100 bash[76202]: cluster 2024-04-07T17:41:10.575937+0000 mgr.y (mgr.25175) 2574 : cluster [DBG] pgmap v4358: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 93 KiB/s rd, 3.7 KiB/s wr, 124 op/s 2024-04-07T17:41:11.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:41:11 smithi144 bash[66654]: cluster 2024-04-07T17:41:10.356122+0000 mon.a (mon.0) 4155 : cluster [DBG] osdmap e2209: 8 total, 8 up, 8 in 2024-04-07T17:41:11.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:41:11 smithi144 bash[66654]: cluster 2024-04-07T17:41:10.575937+0000 mgr.y (mgr.25175) 2574 : cluster [DBG] pgmap v4358: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 93 KiB/s rd, 3.7 KiB/s wr, 124 op/s 2024-04-07T17:41:12.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:41:12 smithi144 bash[66654]: cluster 2024-04-07T17:41:11.377153+0000 mon.a (mon.0) 4156 : cluster [DBG] osdmap e2210: 8 total, 8 up, 8 in 2024-04-07T17:41:12.615 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:41:12 smithi144 bash[95233]: ts=2024-04-07T17:41:12.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:41:12.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:41:12 smithi100 bash[74942]: cluster 2024-04-07T17:41:11.377153+0000 mon.a (mon.0) 4156 : cluster [DBG] osdmap e2210: 8 total, 8 up, 8 in 2024-04-07T17:41:12.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:41:12 smithi100 bash[76202]: cluster 2024-04-07T17:41:11.377153+0000 mon.a (mon.0) 4156 : cluster [DBG] osdmap e2210: 8 total, 8 up, 8 in 2024-04-07T17:41:13.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:41:13 smithi100 bash[74942]: cluster 2024-04-07T17:41:12.377779+0000 mon.a (mon.0) 4157 : cluster [DBG] osdmap e2211: 8 total, 8 up, 8 in 2024-04-07T17:41:13.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:41:13 smithi100 bash[74942]: cluster 2024-04-07T17:41:12.576686+0000 mgr.y (mgr.25175) 2575 : cluster [DBG] pgmap v4361: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 93 KiB/s rd, 3.2 KiB/s wr, 124 op/s 2024-04-07T17:41:13.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:41:13 smithi100 bash[76202]: cluster 2024-04-07T17:41:12.377779+0000 mon.a (mon.0) 4157 : cluster [DBG] osdmap e2211: 8 total, 8 up, 8 in 2024-04-07T17:41:13.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:41:13 smithi100 bash[76202]: cluster 2024-04-07T17:41:12.576686+0000 mgr.y (mgr.25175) 2575 : cluster [DBG] pgmap v4361: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 93 KiB/s rd, 3.2 KiB/s wr, 124 op/s 2024-04-07T17:41:13.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:41:13 smithi144 bash[66654]: cluster 2024-04-07T17:41:12.377779+0000 mon.a (mon.0) 4157 : cluster [DBG] osdmap e2211: 8 total, 8 up, 8 in 2024-04-07T17:41:13.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:41:13 smithi144 bash[66654]: cluster 2024-04-07T17:41:12.576686+0000 mgr.y (mgr.25175) 2575 : cluster [DBG] pgmap v4361: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 93 KiB/s rd, 3.2 KiB/s wr, 124 op/s 2024-04-07T17:41:14.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:41:14 smithi100 bash[74942]: cluster 2024-04-07T17:41:13.399637+0000 mon.a (mon.0) 4158 : cluster [DBG] osdmap e2212: 8 total, 8 up, 8 in 2024-04-07T17:41:14.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:41:14 smithi100 bash[76202]: cluster 2024-04-07T17:41:13.399637+0000 mon.a (mon.0) 4158 : cluster [DBG] osdmap e2212: 8 total, 8 up, 8 in 2024-04-07T17:41:14.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:41:14 smithi144 bash[66654]: cluster 2024-04-07T17:41:13.399637+0000 mon.a (mon.0) 4158 : cluster [DBG] osdmap e2212: 8 total, 8 up, 8 in 2024-04-07T17:41:15.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:41:15 smithi100 bash[74942]: cluster 2024-04-07T17:41:14.407763+0000 mon.a (mon.0) 4159 : cluster [DBG] osdmap e2213: 8 total, 8 up, 8 in 2024-04-07T17:41:15.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:41:15 smithi100 bash[74942]: cluster 2024-04-07T17:41:14.577415+0000 mgr.y (mgr.25175) 2576 : cluster [DBG] pgmap v4364: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 126 KiB/s rd, 3.7 KiB/s wr, 168 op/s 2024-04-07T17:41:15.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:41:15 smithi100 bash[76202]: cluster 2024-04-07T17:41:14.407763+0000 mon.a (mon.0) 4159 : cluster [DBG] osdmap e2213: 8 total, 8 up, 8 in 2024-04-07T17:41:15.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:41:15 smithi100 bash[76202]: cluster 2024-04-07T17:41:14.577415+0000 mgr.y (mgr.25175) 2576 : cluster [DBG] pgmap v4364: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 126 KiB/s rd, 3.7 KiB/s wr, 168 op/s 2024-04-07T17:41:15.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:41:15 smithi144 bash[66654]: cluster 2024-04-07T17:41:14.407763+0000 mon.a (mon.0) 4159 : cluster [DBG] osdmap e2213: 8 total, 8 up, 8 in 2024-04-07T17:41:15.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:41:15 smithi144 bash[66654]: cluster 2024-04-07T17:41:14.577415+0000 mgr.y (mgr.25175) 2576 : cluster [DBG] pgmap v4364: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 126 KiB/s rd, 3.7 KiB/s wr, 168 op/s 2024-04-07T17:41:16.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:41:16 smithi100 bash[74942]: cluster 2024-04-07T17:41:15.420466+0000 mon.a (mon.0) 4160 : cluster [DBG] osdmap e2214: 8 total, 8 up, 8 in 2024-04-07T17:41:16.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:41:16 smithi100 bash[76202]: cluster 2024-04-07T17:41:15.420466+0000 mon.a (mon.0) 4160 : cluster [DBG] osdmap e2214: 8 total, 8 up, 8 in 2024-04-07T17:41:16.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:41:16 smithi144 bash[66654]: cluster 2024-04-07T17:41:15.420466+0000 mon.a (mon.0) 4160 : cluster [DBG] osdmap e2214: 8 total, 8 up, 8 in 2024-04-07T17:41:17.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:41:17 smithi100 bash[74942]: cluster 2024-04-07T17:41:16.425550+0000 mon.a (mon.0) 4161 : cluster [DBG] osdmap e2215: 8 total, 8 up, 8 in 2024-04-07T17:41:17.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:41:17 smithi100 bash[74942]: cluster 2024-04-07T17:41:16.578147+0000 mgr.y (mgr.25175) 2577 : cluster [DBG] pgmap v4367: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 121 KiB/s rd, 3.5 KiB/s wr, 162 op/s 2024-04-07T17:41:17.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:41:17 smithi100 bash[74942]: audit 2024-04-07T17:41:16.864692+0000 mon.a (mon.0) 4162 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:41:17.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:41:17 smithi100 bash[76202]: cluster 2024-04-07T17:41:16.425550+0000 mon.a (mon.0) 4161 : cluster [DBG] osdmap e2215: 8 total, 8 up, 8 in 2024-04-07T17:41:17.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:41:17 smithi100 bash[76202]: cluster 2024-04-07T17:41:16.578147+0000 mgr.y (mgr.25175) 2577 : cluster [DBG] pgmap v4367: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 121 KiB/s rd, 3.5 KiB/s wr, 162 op/s 2024-04-07T17:41:17.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:41:17 smithi100 bash[76202]: audit 2024-04-07T17:41:16.864692+0000 mon.a (mon.0) 4162 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:41:17.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:41:17 smithi144 bash[66654]: cluster 2024-04-07T17:41:16.425550+0000 mon.a (mon.0) 4161 : cluster [DBG] osdmap e2215: 8 total, 8 up, 8 in 2024-04-07T17:41:17.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:41:17 smithi144 bash[66654]: cluster 2024-04-07T17:41:16.578147+0000 mgr.y (mgr.25175) 2577 : cluster [DBG] pgmap v4367: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 121 KiB/s rd, 3.5 KiB/s wr, 162 op/s 2024-04-07T17:41:17.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:41:17 smithi144 bash[66654]: audit 2024-04-07T17:41:16.864692+0000 mon.a (mon.0) 4162 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:41:18.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:41:18 smithi100 bash[74942]: cluster 2024-04-07T17:41:17.439664+0000 mon.a (mon.0) 4163 : cluster [DBG] osdmap e2216: 8 total, 8 up, 8 in 2024-04-07T17:41:18.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:41:18 smithi100 bash[76202]: cluster 2024-04-07T17:41:17.439664+0000 mon.a (mon.0) 4163 : cluster [DBG] osdmap e2216: 8 total, 8 up, 8 in 2024-04-07T17:41:18.704 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:41:18 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:41:18] "GET /metrics HTTP/1.1" 200 36744 "" "Prometheus/2.43.0" 2024-04-07T17:41:18.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:41:18 smithi144 bash[66654]: cluster 2024-04-07T17:41:17.439664+0000 mon.a (mon.0) 4163 : cluster [DBG] osdmap e2216: 8 total, 8 up, 8 in 2024-04-07T17:41:19.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:41:19 smithi100 bash[74942]: cluster 2024-04-07T17:41:18.438095+0000 mon.a (mon.0) 4164 : cluster [DBG] osdmap e2217: 8 total, 8 up, 8 in 2024-04-07T17:41:19.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:41:19 smithi100 bash[74942]: cluster 2024-04-07T17:41:18.578926+0000 mgr.y (mgr.25175) 2578 : cluster [DBG] pgmap v4370: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 5.0 KiB/s rd, 1023 B/s wr, 7 op/s 2024-04-07T17:41:19.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:41:19 smithi100 bash[76202]: cluster 2024-04-07T17:41:18.438095+0000 mon.a (mon.0) 4164 : cluster [DBG] osdmap e2217: 8 total, 8 up, 8 in 2024-04-07T17:41:19.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:41:19 smithi100 bash[76202]: cluster 2024-04-07T17:41:18.578926+0000 mgr.y (mgr.25175) 2578 : cluster [DBG] pgmap v4370: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 5.0 KiB/s rd, 1023 B/s wr, 7 op/s 2024-04-07T17:41:19.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:41:19 smithi144 bash[66654]: cluster 2024-04-07T17:41:18.438095+0000 mon.a (mon.0) 4164 : cluster [DBG] osdmap e2217: 8 total, 8 up, 8 in 2024-04-07T17:41:19.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:41:19 smithi144 bash[66654]: cluster 2024-04-07T17:41:18.578926+0000 mgr.y (mgr.25175) 2578 : cluster [DBG] pgmap v4370: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 5.0 KiB/s rd, 1023 B/s wr, 7 op/s 2024-04-07T17:41:20.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:41:20 smithi144 bash[66654]: cluster 2024-04-07T17:41:19.465836+0000 mon.a (mon.0) 4165 : cluster [DBG] osdmap e2218: 8 total, 8 up, 8 in 2024-04-07T17:41:20.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:41:20 smithi100 bash[74942]: cluster 2024-04-07T17:41:19.465836+0000 mon.a (mon.0) 4165 : cluster [DBG] osdmap e2218: 8 total, 8 up, 8 in 2024-04-07T17:41:20.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:41:20 smithi100 bash[76202]: cluster 2024-04-07T17:41:19.465836+0000 mon.a (mon.0) 4165 : cluster [DBG] osdmap e2218: 8 total, 8 up, 8 in 2024-04-07T17:41:21.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:41:21 smithi144 bash[66654]: cluster 2024-04-07T17:41:20.469299+0000 mon.a (mon.0) 4166 : cluster [DBG] osdmap e2219: 8 total, 8 up, 8 in 2024-04-07T17:41:21.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:41:21 smithi144 bash[66654]: cluster 2024-04-07T17:41:20.579645+0000 mgr.y (mgr.25175) 2579 : cluster [DBG] pgmap v4373: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 157 KiB/s rd, 3.2 KiB/s wr, 211 op/s 2024-04-07T17:41:21.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:41:21 smithi100 bash[74942]: cluster 2024-04-07T17:41:20.469299+0000 mon.a (mon.0) 4166 : cluster [DBG] osdmap e2219: 8 total, 8 up, 8 in 2024-04-07T17:41:21.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:41:21 smithi100 bash[74942]: cluster 2024-04-07T17:41:20.579645+0000 mgr.y (mgr.25175) 2579 : cluster [DBG] pgmap v4373: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 157 KiB/s rd, 3.2 KiB/s wr, 211 op/s 2024-04-07T17:41:21.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:41:21 smithi100 bash[76202]: cluster 2024-04-07T17:41:20.469299+0000 mon.a (mon.0) 4166 : cluster [DBG] osdmap e2219: 8 total, 8 up, 8 in 2024-04-07T17:41:21.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:41:21 smithi100 bash[76202]: cluster 2024-04-07T17:41:20.579645+0000 mgr.y (mgr.25175) 2579 : cluster [DBG] pgmap v4373: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 157 KiB/s rd, 3.2 KiB/s wr, 211 op/s 2024-04-07T17:41:22.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:41:22 smithi144 bash[66654]: cluster 2024-04-07T17:41:21.491754+0000 mon.a (mon.0) 4167 : cluster [DBG] osdmap e2220: 8 total, 8 up, 8 in 2024-04-07T17:41:22.615 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:41:22 smithi144 bash[95233]: ts=2024-04-07T17:41:22.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:41:22.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:41:22 smithi100 bash[74942]: cluster 2024-04-07T17:41:21.491754+0000 mon.a (mon.0) 4167 : cluster [DBG] osdmap e2220: 8 total, 8 up, 8 in 2024-04-07T17:41:22.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:41:22 smithi100 bash[76202]: cluster 2024-04-07T17:41:21.491754+0000 mon.a (mon.0) 4167 : cluster [DBG] osdmap e2220: 8 total, 8 up, 8 in 2024-04-07T17:41:23.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:41:23 smithi144 bash[66654]: cluster 2024-04-07T17:41:22.500160+0000 mon.a (mon.0) 4168 : cluster [DBG] osdmap e2221: 8 total, 8 up, 8 in 2024-04-07T17:41:23.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:41:23 smithi144 bash[66654]: cluster 2024-04-07T17:41:22.580346+0000 mgr.y (mgr.25175) 2580 : cluster [DBG] pgmap v4376: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 157 KiB/s rd, 3.7 KiB/s wr, 212 op/s 2024-04-07T17:41:23.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:41:23 smithi100 bash[74942]: cluster 2024-04-07T17:41:22.500160+0000 mon.a (mon.0) 4168 : cluster [DBG] osdmap e2221: 8 total, 8 up, 8 in 2024-04-07T17:41:23.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:41:23 smithi100 bash[74942]: cluster 2024-04-07T17:41:22.580346+0000 mgr.y (mgr.25175) 2580 : cluster [DBG] pgmap v4376: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 157 KiB/s rd, 3.7 KiB/s wr, 212 op/s 2024-04-07T17:41:23.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:41:23 smithi100 bash[76202]: cluster 2024-04-07T17:41:22.500160+0000 mon.a (mon.0) 4168 : cluster [DBG] osdmap e2221: 8 total, 8 up, 8 in 2024-04-07T17:41:23.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:41:23 smithi100 bash[76202]: cluster 2024-04-07T17:41:22.580346+0000 mgr.y (mgr.25175) 2580 : cluster [DBG] pgmap v4376: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 157 KiB/s rd, 3.7 KiB/s wr, 212 op/s 2024-04-07T17:41:24.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:41:24 smithi144 bash[66654]: cluster 2024-04-07T17:41:23.510926+0000 mon.a (mon.0) 4169 : cluster [DBG] osdmap e2222: 8 total, 8 up, 8 in 2024-04-07T17:41:24.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:41:24 smithi100 bash[74942]: cluster 2024-04-07T17:41:23.510926+0000 mon.a (mon.0) 4169 : cluster [DBG] osdmap e2222: 8 total, 8 up, 8 in 2024-04-07T17:41:24.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:41:24 smithi100 bash[76202]: cluster 2024-04-07T17:41:23.510926+0000 mon.a (mon.0) 4169 : cluster [DBG] osdmap e2222: 8 total, 8 up, 8 in 2024-04-07T17:41:25.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:41:25 smithi144 bash[66654]: cluster 2024-04-07T17:41:24.511908+0000 mon.a (mon.0) 4170 : cluster [DBG] osdmap e2223: 8 total, 8 up, 8 in 2024-04-07T17:41:25.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:41:25 smithi144 bash[66654]: cluster 2024-04-07T17:41:24.581102+0000 mgr.y (mgr.25175) 2581 : cluster [DBG] pgmap v4379: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 191 KiB/s rd, 4.0 KiB/s wr, 256 op/s 2024-04-07T17:41:25.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:41:25 smithi100 bash[74942]: cluster 2024-04-07T17:41:24.511908+0000 mon.a (mon.0) 4170 : cluster [DBG] osdmap e2223: 8 total, 8 up, 8 in 2024-04-07T17:41:25.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:41:25 smithi100 bash[74942]: cluster 2024-04-07T17:41:24.581102+0000 mgr.y (mgr.25175) 2581 : cluster [DBG] pgmap v4379: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 191 KiB/s rd, 4.0 KiB/s wr, 256 op/s 2024-04-07T17:41:25.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:41:25 smithi100 bash[76202]: cluster 2024-04-07T17:41:24.511908+0000 mon.a (mon.0) 4170 : cluster [DBG] osdmap e2223: 8 total, 8 up, 8 in 2024-04-07T17:41:25.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:41:25 smithi100 bash[76202]: cluster 2024-04-07T17:41:24.581102+0000 mgr.y (mgr.25175) 2581 : cluster [DBG] pgmap v4379: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 191 KiB/s rd, 4.0 KiB/s wr, 256 op/s 2024-04-07T17:41:26.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:41:26 smithi144 bash[66654]: cluster 2024-04-07T17:41:25.537936+0000 mon.a (mon.0) 4171 : cluster [DBG] osdmap e2224: 8 total, 8 up, 8 in 2024-04-07T17:41:26.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:41:26 smithi100 bash[74942]: cluster 2024-04-07T17:41:25.537936+0000 mon.a (mon.0) 4171 : cluster [DBG] osdmap e2224: 8 total, 8 up, 8 in 2024-04-07T17:41:26.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:41:26 smithi100 bash[76202]: cluster 2024-04-07T17:41:25.537936+0000 mon.a (mon.0) 4171 : cluster [DBG] osdmap e2224: 8 total, 8 up, 8 in 2024-04-07T17:41:27.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:41:27 smithi144 bash[66654]: cluster 2024-04-07T17:41:26.537656+0000 mon.a (mon.0) 4172 : cluster [DBG] osdmap e2225: 8 total, 8 up, 8 in 2024-04-07T17:41:27.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:41:27 smithi144 bash[66654]: cluster 2024-04-07T17:41:26.581848+0000 mgr.y (mgr.25175) 2582 : cluster [DBG] pgmap v4382: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 186 KiB/s rd, 2.7 KiB/s wr, 248 op/s 2024-04-07T17:41:27.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:41:27 smithi100 bash[74942]: cluster 2024-04-07T17:41:26.537656+0000 mon.a (mon.0) 4172 : cluster [DBG] osdmap e2225: 8 total, 8 up, 8 in 2024-04-07T17:41:27.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:41:27 smithi100 bash[74942]: cluster 2024-04-07T17:41:26.581848+0000 mgr.y (mgr.25175) 2582 : cluster [DBG] pgmap v4382: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 186 KiB/s rd, 2.7 KiB/s wr, 248 op/s 2024-04-07T17:41:27.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:41:27 smithi100 bash[76202]: cluster 2024-04-07T17:41:26.537656+0000 mon.a (mon.0) 4172 : cluster [DBG] osdmap e2225: 8 total, 8 up, 8 in 2024-04-07T17:41:27.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:41:27 smithi100 bash[76202]: cluster 2024-04-07T17:41:26.581848+0000 mgr.y (mgr.25175) 2582 : cluster [DBG] pgmap v4382: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 186 KiB/s rd, 2.7 KiB/s wr, 248 op/s 2024-04-07T17:41:28.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:41:28 smithi100 bash[74942]: cluster 2024-04-07T17:41:27.551545+0000 mon.a (mon.0) 4173 : cluster [DBG] osdmap e2226: 8 total, 8 up, 8 in 2024-04-07T17:41:28.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:41:28 smithi100 bash[74942]: audit 2024-04-07T17:41:28.253720+0000 mon.a (mon.0) 4174 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:41:28.704 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:41:28 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:41:28] "GET /metrics HTTP/1.1" 200 36744 "" "Prometheus/2.43.0" 2024-04-07T17:41:28.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:41:28 smithi100 bash[76202]: cluster 2024-04-07T17:41:27.551545+0000 mon.a (mon.0) 4173 : cluster [DBG] osdmap e2226: 8 total, 8 up, 8 in 2024-04-07T17:41:28.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:41:28 smithi100 bash[76202]: audit 2024-04-07T17:41:28.253720+0000 mon.a (mon.0) 4174 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:41:28.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:41:28 smithi144 bash[66654]: cluster 2024-04-07T17:41:27.551545+0000 mon.a (mon.0) 4173 : cluster [DBG] osdmap e2226: 8 total, 8 up, 8 in 2024-04-07T17:41:28.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:41:28 smithi144 bash[66654]: audit 2024-04-07T17:41:28.253720+0000 mon.a (mon.0) 4174 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:41:29.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:41:29 smithi144 bash[66654]: cluster 2024-04-07T17:41:28.570526+0000 mon.a (mon.0) 4175 : cluster [DBG] osdmap e2227: 8 total, 8 up, 8 in 2024-04-07T17:41:29.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:41:29 smithi144 bash[66654]: cluster 2024-04-07T17:41:28.582712+0000 mgr.y (mgr.25175) 2583 : cluster [DBG] pgmap v4385: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 5.0 KiB/s rd, 511 B/s wr, 6 op/s 2024-04-07T17:41:29.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:41:29 smithi100 bash[74942]: cluster 2024-04-07T17:41:28.570526+0000 mon.a (mon.0) 4175 : cluster [DBG] osdmap e2227: 8 total, 8 up, 8 in 2024-04-07T17:41:29.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:41:29 smithi100 bash[74942]: cluster 2024-04-07T17:41:28.582712+0000 mgr.y (mgr.25175) 2583 : cluster [DBG] pgmap v4385: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 5.0 KiB/s rd, 511 B/s wr, 6 op/s 2024-04-07T17:41:29.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:41:29 smithi100 bash[76202]: cluster 2024-04-07T17:41:28.570526+0000 mon.a (mon.0) 4175 : cluster [DBG] osdmap e2227: 8 total, 8 up, 8 in 2024-04-07T17:41:29.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:41:29 smithi100 bash[76202]: cluster 2024-04-07T17:41:28.582712+0000 mgr.y (mgr.25175) 2583 : cluster [DBG] pgmap v4385: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 5.0 KiB/s rd, 511 B/s wr, 6 op/s 2024-04-07T17:41:30.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:41:30 smithi144 bash[66654]: cluster 2024-04-07T17:41:29.574450+0000 mon.a (mon.0) 4176 : cluster [DBG] osdmap e2228: 8 total, 8 up, 8 in 2024-04-07T17:41:30.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:41:30 smithi100 bash[74942]: cluster 2024-04-07T17:41:29.574450+0000 mon.a (mon.0) 4176 : cluster [DBG] osdmap e2228: 8 total, 8 up, 8 in 2024-04-07T17:41:30.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:41:30 smithi100 bash[76202]: cluster 2024-04-07T17:41:29.574450+0000 mon.a (mon.0) 4176 : cluster [DBG] osdmap e2228: 8 total, 8 up, 8 in 2024-04-07T17:41:31.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:41:31 smithi144 bash[66654]: cluster 2024-04-07T17:41:30.583653+0000 mgr.y (mgr.25175) 2584 : cluster [DBG] pgmap v4387: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 220 KiB/s rd, 3.7 KiB/s wr, 296 op/s 2024-04-07T17:41:31.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:41:31 smithi144 bash[66654]: cluster 2024-04-07T17:41:30.595401+0000 mon.a (mon.0) 4177 : cluster [DBG] osdmap e2229: 8 total, 8 up, 8 in 2024-04-07T17:41:31.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:41:31 smithi100 bash[74942]: cluster 2024-04-07T17:41:30.583653+0000 mgr.y (mgr.25175) 2584 : cluster [DBG] pgmap v4387: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 220 KiB/s rd, 3.7 KiB/s wr, 296 op/s 2024-04-07T17:41:31.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:41:31 smithi100 bash[74942]: cluster 2024-04-07T17:41:30.595401+0000 mon.a (mon.0) 4177 : cluster [DBG] osdmap e2229: 8 total, 8 up, 8 in 2024-04-07T17:41:31.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:41:31 smithi100 bash[76202]: cluster 2024-04-07T17:41:30.583653+0000 mgr.y (mgr.25175) 2584 : cluster [DBG] pgmap v4387: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 220 KiB/s rd, 3.7 KiB/s wr, 296 op/s 2024-04-07T17:41:31.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:41:31 smithi100 bash[76202]: cluster 2024-04-07T17:41:30.595401+0000 mon.a (mon.0) 4177 : cluster [DBG] osdmap e2229: 8 total, 8 up, 8 in 2024-04-07T17:41:32.607 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:41:32 smithi144 bash[95233]: ts=2024-04-07T17:41:32.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:41:32.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:41:32 smithi144 bash[66654]: cluster 2024-04-07T17:41:31.598299+0000 mon.a (mon.0) 4178 : cluster [DBG] osdmap e2230: 8 total, 8 up, 8 in 2024-04-07T17:41:32.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:41:32 smithi144 bash[66654]: audit 2024-04-07T17:41:31.865164+0000 mon.a (mon.0) 4179 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:41:32.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:41:32 smithi100 bash[74942]: cluster 2024-04-07T17:41:31.598299+0000 mon.a (mon.0) 4178 : cluster [DBG] osdmap e2230: 8 total, 8 up, 8 in 2024-04-07T17:41:32.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:41:32 smithi100 bash[74942]: audit 2024-04-07T17:41:31.865164+0000 mon.a (mon.0) 4179 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:41:32.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:41:32 smithi100 bash[76202]: cluster 2024-04-07T17:41:31.598299+0000 mon.a (mon.0) 4178 : cluster [DBG] osdmap e2230: 8 total, 8 up, 8 in 2024-04-07T17:41:32.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:41:32 smithi100 bash[76202]: audit 2024-04-07T17:41:31.865164+0000 mon.a (mon.0) 4179 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:41:33.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:41:33 smithi100 bash[74942]: cluster 2024-04-07T17:41:32.584880+0000 mgr.y (mgr.25175) 2585 : cluster [DBG] pgmap v4390: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 221 KiB/s rd, 3.7 KiB/s wr, 298 op/s 2024-04-07T17:41:33.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:41:33 smithi100 bash[74942]: cluster 2024-04-07T17:41:32.621676+0000 mon.a (mon.0) 4180 : cluster [DBG] osdmap e2231: 8 total, 8 up, 8 in 2024-04-07T17:41:33.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:41:33 smithi100 bash[76202]: cluster 2024-04-07T17:41:32.584880+0000 mgr.y (mgr.25175) 2585 : cluster [DBG] pgmap v4390: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 221 KiB/s rd, 3.7 KiB/s wr, 298 op/s 2024-04-07T17:41:33.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:41:33 smithi100 bash[76202]: cluster 2024-04-07T17:41:32.621676+0000 mon.a (mon.0) 4180 : cluster [DBG] osdmap e2231: 8 total, 8 up, 8 in 2024-04-07T17:41:34.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:41:33 smithi144 bash[66654]: cluster 2024-04-07T17:41:32.584880+0000 mgr.y (mgr.25175) 2585 : cluster [DBG] pgmap v4390: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 221 KiB/s rd, 3.7 KiB/s wr, 298 op/s 2024-04-07T17:41:34.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:41:33 smithi144 bash[66654]: cluster 2024-04-07T17:41:32.621676+0000 mon.a (mon.0) 4180 : cluster [DBG] osdmap e2231: 8 total, 8 up, 8 in 2024-04-07T17:41:34.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:41:34 smithi100 bash[74942]: cluster 2024-04-07T17:41:33.632001+0000 mon.a (mon.0) 4181 : cluster [DBG] osdmap e2232: 8 total, 8 up, 8 in 2024-04-07T17:41:34.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:41:34 smithi100 bash[74942]: audit 2024-04-07T17:41:33.944846+0000 mon.a (mon.0) 4182 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:41:34.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:41:34 smithi100 bash[74942]: audit 2024-04-07T17:41:33.953376+0000 mon.a (mon.0) 4183 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:41:34.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:41:34 smithi100 bash[74942]: audit 2024-04-07T17:41:34.206966+0000 mon.a (mon.0) 4184 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:41:34.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:41:34 smithi100 bash[74942]: audit 2024-04-07T17:41:34.216342+0000 mon.a (mon.0) 4185 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:41:34.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:41:34 smithi100 bash[76202]: cluster 2024-04-07T17:41:33.632001+0000 mon.a (mon.0) 4181 : cluster [DBG] osdmap e2232: 8 total, 8 up, 8 in 2024-04-07T17:41:34.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:41:34 smithi100 bash[76202]: audit 2024-04-07T17:41:33.944846+0000 mon.a (mon.0) 4182 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:41:34.955 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:41:34 smithi100 bash[76202]: audit 2024-04-07T17:41:33.953376+0000 mon.a (mon.0) 4183 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:41:34.955 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:41:34 smithi100 bash[76202]: audit 2024-04-07T17:41:34.206966+0000 mon.a (mon.0) 4184 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:41:34.955 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:41:34 smithi100 bash[76202]: audit 2024-04-07T17:41:34.216342+0000 mon.a (mon.0) 4185 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:41:35.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:41:34 smithi144 bash[66654]: cluster 2024-04-07T17:41:33.632001+0000 mon.a (mon.0) 4181 : cluster [DBG] osdmap e2232: 8 total, 8 up, 8 in 2024-04-07T17:41:35.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:41:34 smithi144 bash[66654]: audit 2024-04-07T17:41:33.944846+0000 mon.a (mon.0) 4182 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:41:35.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:41:34 smithi144 bash[66654]: audit 2024-04-07T17:41:33.953376+0000 mon.a (mon.0) 4183 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:41:35.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:41:34 smithi144 bash[66654]: audit 2024-04-07T17:41:34.206966+0000 mon.a (mon.0) 4184 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:41:35.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:41:34 smithi144 bash[66654]: audit 2024-04-07T17:41:34.216342+0000 mon.a (mon.0) 4185 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:41:35.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:41:35 smithi100 bash[74942]: cluster 2024-04-07T17:41:34.585791+0000 mgr.y (mgr.25175) 2586 : cluster [DBG] pgmap v4393: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 256 KiB/s rd, 3.7 KiB/s wr, 343 op/s 2024-04-07T17:41:35.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:41:35 smithi100 bash[74942]: cluster 2024-04-07T17:41:34.639078+0000 mon.a (mon.0) 4186 : cluster [DBG] osdmap e2233: 8 total, 8 up, 8 in 2024-04-07T17:41:35.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:41:35 smithi100 bash[74942]: audit 2024-04-07T17:41:34.843284+0000 mon.a (mon.0) 4187 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:41:35.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:41:35 smithi100 bash[74942]: audit 2024-04-07T17:41:34.845285+0000 mon.a (mon.0) 4188 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:41:35.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:41:35 smithi100 bash[74942]: audit 2024-04-07T17:41:34.856212+0000 mon.a (mon.0) 4189 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:41:35.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:41:35 smithi100 bash[76202]: cluster 2024-04-07T17:41:34.585791+0000 mgr.y (mgr.25175) 2586 : cluster [DBG] pgmap v4393: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 256 KiB/s rd, 3.7 KiB/s wr, 343 op/s 2024-04-07T17:41:35.955 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:41:35 smithi100 bash[76202]: cluster 2024-04-07T17:41:34.639078+0000 mon.a (mon.0) 4186 : cluster [DBG] osdmap e2233: 8 total, 8 up, 8 in 2024-04-07T17:41:35.955 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:41:35 smithi100 bash[76202]: audit 2024-04-07T17:41:34.843284+0000 mon.a (mon.0) 4187 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:41:35.955 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:41:35 smithi100 bash[76202]: audit 2024-04-07T17:41:34.845285+0000 mon.a (mon.0) 4188 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:41:35.955 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:41:35 smithi100 bash[76202]: audit 2024-04-07T17:41:34.856212+0000 mon.a (mon.0) 4189 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:41:36.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:41:35 smithi144 bash[66654]: cluster 2024-04-07T17:41:34.585791+0000 mgr.y (mgr.25175) 2586 : cluster [DBG] pgmap v4393: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 256 KiB/s rd, 3.7 KiB/s wr, 343 op/s 2024-04-07T17:41:36.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:41:35 smithi144 bash[66654]: cluster 2024-04-07T17:41:34.639078+0000 mon.a (mon.0) 4186 : cluster [DBG] osdmap e2233: 8 total, 8 up, 8 in 2024-04-07T17:41:36.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:41:35 smithi144 bash[66654]: audit 2024-04-07T17:41:34.843284+0000 mon.a (mon.0) 4187 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:41:36.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:41:35 smithi144 bash[66654]: audit 2024-04-07T17:41:34.845285+0000 mon.a (mon.0) 4188 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:41:36.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:41:35 smithi144 bash[66654]: audit 2024-04-07T17:41:34.856212+0000 mon.a (mon.0) 4189 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:41:36.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:41:36 smithi100 bash[74942]: cluster 2024-04-07T17:41:35.643414+0000 mon.a (mon.0) 4190 : cluster [DBG] osdmap e2234: 8 total, 8 up, 8 in 2024-04-07T17:41:36.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:41:36 smithi100 bash[76202]: cluster 2024-04-07T17:41:35.643414+0000 mon.a (mon.0) 4190 : cluster [DBG] osdmap e2234: 8 total, 8 up, 8 in 2024-04-07T17:41:37.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:41:36 smithi144 bash[66654]: cluster 2024-04-07T17:41:35.643414+0000 mon.a (mon.0) 4190 : cluster [DBG] osdmap e2234: 8 total, 8 up, 8 in 2024-04-07T17:41:37.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:41:37 smithi100 bash[74942]: cluster 2024-04-07T17:41:36.586537+0000 mgr.y (mgr.25175) 2587 : cluster [DBG] pgmap v4396: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 251 KiB/s rd, 3.2 KiB/s wr, 336 op/s 2024-04-07T17:41:37.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:41:37 smithi100 bash[74942]: cluster 2024-04-07T17:41:36.657359+0000 mon.a (mon.0) 4191 : cluster [DBG] osdmap e2235: 8 total, 8 up, 8 in 2024-04-07T17:41:37.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:41:37 smithi100 bash[76202]: cluster 2024-04-07T17:41:36.586537+0000 mgr.y (mgr.25175) 2587 : cluster [DBG] pgmap v4396: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 251 KiB/s rd, 3.2 KiB/s wr, 336 op/s 2024-04-07T17:41:37.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:41:37 smithi100 bash[76202]: cluster 2024-04-07T17:41:36.657359+0000 mon.a (mon.0) 4191 : cluster [DBG] osdmap e2235: 8 total, 8 up, 8 in 2024-04-07T17:41:38.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:41:37 smithi144 bash[66654]: cluster 2024-04-07T17:41:36.586537+0000 mgr.y (mgr.25175) 2587 : cluster [DBG] pgmap v4396: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 251 KiB/s rd, 3.2 KiB/s wr, 336 op/s 2024-04-07T17:41:38.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:41:37 smithi144 bash[66654]: cluster 2024-04-07T17:41:36.657359+0000 mon.a (mon.0) 4191 : cluster [DBG] osdmap e2235: 8 total, 8 up, 8 in 2024-04-07T17:41:38.686 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:41:38 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:41:38] "GET /metrics HTTP/1.1" 200 36741 "" "Prometheus/2.43.0" 2024-04-07T17:41:38.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:41:38 smithi100 bash[74942]: cluster 2024-04-07T17:41:37.670805+0000 mon.a (mon.0) 4192 : cluster [DBG] osdmap e2236: 8 total, 8 up, 8 in 2024-04-07T17:41:38.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:41:38 smithi100 bash[76202]: cluster 2024-04-07T17:41:37.670805+0000 mon.a (mon.0) 4192 : cluster [DBG] osdmap e2236: 8 total, 8 up, 8 in 2024-04-07T17:41:39.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:41:38 smithi144 bash[66654]: cluster 2024-04-07T17:41:37.670805+0000 mon.a (mon.0) 4192 : cluster [DBG] osdmap e2236: 8 total, 8 up, 8 in 2024-04-07T17:41:39.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:41:39 smithi100 bash[76202]: cluster 2024-04-07T17:41:38.587595+0000 mgr.y (mgr.25175) 2588 : cluster [DBG] pgmap v4399: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 5.0 KiB/s rd, 1023 B/s wr, 7 op/s 2024-04-07T17:41:39.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:41:39 smithi100 bash[76202]: cluster 2024-04-07T17:41:38.692629+0000 mon.a (mon.0) 4193 : cluster [DBG] osdmap e2237: 8 total, 8 up, 8 in 2024-04-07T17:41:39.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:41:39 smithi100 bash[74942]: cluster 2024-04-07T17:41:38.587595+0000 mgr.y (mgr.25175) 2588 : cluster [DBG] pgmap v4399: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 5.0 KiB/s rd, 1023 B/s wr, 7 op/s 2024-04-07T17:41:39.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:41:39 smithi100 bash[74942]: cluster 2024-04-07T17:41:38.692629+0000 mon.a (mon.0) 4193 : cluster [DBG] osdmap e2237: 8 total, 8 up, 8 in 2024-04-07T17:41:40.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:41:39 smithi144 bash[66654]: cluster 2024-04-07T17:41:38.587595+0000 mgr.y (mgr.25175) 2588 : cluster [DBG] pgmap v4399: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 5.0 KiB/s rd, 1023 B/s wr, 7 op/s 2024-04-07T17:41:40.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:41:39 smithi144 bash[66654]: cluster 2024-04-07T17:41:38.692629+0000 mon.a (mon.0) 4193 : cluster [DBG] osdmap e2237: 8 total, 8 up, 8 in 2024-04-07T17:41:41.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:41:40 smithi144 bash[66654]: cluster 2024-04-07T17:41:39.702117+0000 mon.a (mon.0) 4194 : cluster [DBG] osdmap e2238: 8 total, 8 up, 8 in 2024-04-07T17:41:41.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:41:40 smithi100 bash[74942]: cluster 2024-04-07T17:41:39.702117+0000 mon.a (mon.0) 4194 : cluster [DBG] osdmap e2238: 8 total, 8 up, 8 in 2024-04-07T17:41:41.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:41:40 smithi100 bash[76202]: cluster 2024-04-07T17:41:39.702117+0000 mon.a (mon.0) 4194 : cluster [DBG] osdmap e2238: 8 total, 8 up, 8 in 2024-04-07T17:41:42.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:41:41 smithi144 bash[66654]: cluster 2024-04-07T17:41:40.588544+0000 mgr.y (mgr.25175) 2589 : cluster [DBG] pgmap v4402: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 289 KiB/s rd, 4.0 KiB/s wr, 387 op/s 2024-04-07T17:41:42.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:41:41 smithi144 bash[66654]: cluster 2024-04-07T17:41:40.712490+0000 mon.a (mon.0) 4195 : cluster [DBG] osdmap e2239: 8 total, 8 up, 8 in 2024-04-07T17:41:42.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:41:41 smithi100 bash[74942]: cluster 2024-04-07T17:41:40.588544+0000 mgr.y (mgr.25175) 2589 : cluster [DBG] pgmap v4402: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 289 KiB/s rd, 4.0 KiB/s wr, 387 op/s 2024-04-07T17:41:42.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:41:41 smithi100 bash[74942]: cluster 2024-04-07T17:41:40.712490+0000 mon.a (mon.0) 4195 : cluster [DBG] osdmap e2239: 8 total, 8 up, 8 in 2024-04-07T17:41:42.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:41:41 smithi100 bash[76202]: cluster 2024-04-07T17:41:40.588544+0000 mgr.y (mgr.25175) 2589 : cluster [DBG] pgmap v4402: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 289 KiB/s rd, 4.0 KiB/s wr, 387 op/s 2024-04-07T17:41:42.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:41:41 smithi100 bash[76202]: cluster 2024-04-07T17:41:40.712490+0000 mon.a (mon.0) 4195 : cluster [DBG] osdmap e2239: 8 total, 8 up, 8 in 2024-04-07T17:41:42.614 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:41:42 smithi144 bash[95233]: ts=2024-04-07T17:41:42.317Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:41:43.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:41:42 smithi144 bash[66654]: cluster 2024-04-07T17:41:41.720676+0000 mon.a (mon.0) 4196 : cluster [DBG] osdmap e2240: 8 total, 8 up, 8 in 2024-04-07T17:41:43.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:41:42 smithi100 bash[74942]: cluster 2024-04-07T17:41:41.720676+0000 mon.a (mon.0) 4196 : cluster [DBG] osdmap e2240: 8 total, 8 up, 8 in 2024-04-07T17:41:43.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:41:42 smithi100 bash[76202]: cluster 2024-04-07T17:41:41.720676+0000 mon.a (mon.0) 4196 : cluster [DBG] osdmap e2240: 8 total, 8 up, 8 in 2024-04-07T17:41:44.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:41:43 smithi144 bash[66654]: cluster 2024-04-07T17:41:42.589769+0000 mgr.y (mgr.25175) 2590 : cluster [DBG] pgmap v4405: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 289 KiB/s rd, 3.2 KiB/s wr, 386 op/s 2024-04-07T17:41:44.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:41:43 smithi144 bash[66654]: cluster 2024-04-07T17:41:42.731322+0000 mon.a (mon.0) 4197 : cluster [DBG] osdmap e2241: 8 total, 8 up, 8 in 2024-04-07T17:41:44.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:41:43 smithi100 bash[74942]: cluster 2024-04-07T17:41:42.589769+0000 mgr.y (mgr.25175) 2590 : cluster [DBG] pgmap v4405: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 289 KiB/s rd, 3.2 KiB/s wr, 386 op/s 2024-04-07T17:41:44.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:41:43 smithi100 bash[74942]: cluster 2024-04-07T17:41:42.731322+0000 mon.a (mon.0) 4197 : cluster [DBG] osdmap e2241: 8 total, 8 up, 8 in 2024-04-07T17:41:44.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:41:43 smithi100 bash[76202]: cluster 2024-04-07T17:41:42.589769+0000 mgr.y (mgr.25175) 2590 : cluster [DBG] pgmap v4405: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 289 KiB/s rd, 3.2 KiB/s wr, 386 op/s 2024-04-07T17:41:44.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:41:43 smithi100 bash[76202]: cluster 2024-04-07T17:41:42.731322+0000 mon.a (mon.0) 4197 : cluster [DBG] osdmap e2241: 8 total, 8 up, 8 in 2024-04-07T17:41:45.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:41:44 smithi144 bash[66654]: cluster 2024-04-07T17:41:43.733086+0000 mon.a (mon.0) 4198 : cluster [DBG] osdmap e2242: 8 total, 8 up, 8 in 2024-04-07T17:41:45.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:41:44 smithi100 bash[74942]: cluster 2024-04-07T17:41:43.733086+0000 mon.a (mon.0) 4198 : cluster [DBG] osdmap e2242: 8 total, 8 up, 8 in 2024-04-07T17:41:45.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:41:44 smithi100 bash[76202]: cluster 2024-04-07T17:41:43.733086+0000 mon.a (mon.0) 4198 : cluster [DBG] osdmap e2242: 8 total, 8 up, 8 in 2024-04-07T17:41:46.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:41:45 smithi144 bash[66654]: cluster 2024-04-07T17:41:44.590780+0000 mgr.y (mgr.25175) 2591 : cluster [DBG] pgmap v4408: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 321 KiB/s rd, 3.2 KiB/s wr, 430 op/s 2024-04-07T17:41:46.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:41:45 smithi144 bash[66654]: cluster 2024-04-07T17:41:44.765113+0000 mon.a (mon.0) 4199 : cluster [DBG] osdmap e2243: 8 total, 8 up, 8 in 2024-04-07T17:41:46.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:41:45 smithi100 bash[74942]: cluster 2024-04-07T17:41:44.590780+0000 mgr.y (mgr.25175) 2591 : cluster [DBG] pgmap v4408: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 321 KiB/s rd, 3.2 KiB/s wr, 430 op/s 2024-04-07T17:41:46.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:41:45 smithi100 bash[74942]: cluster 2024-04-07T17:41:44.765113+0000 mon.a (mon.0) 4199 : cluster [DBG] osdmap e2243: 8 total, 8 up, 8 in 2024-04-07T17:41:46.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:41:45 smithi100 bash[76202]: cluster 2024-04-07T17:41:44.590780+0000 mgr.y (mgr.25175) 2591 : cluster [DBG] pgmap v4408: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 321 KiB/s rd, 3.2 KiB/s wr, 430 op/s 2024-04-07T17:41:46.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:41:45 smithi100 bash[76202]: cluster 2024-04-07T17:41:44.765113+0000 mon.a (mon.0) 4199 : cluster [DBG] osdmap e2243: 8 total, 8 up, 8 in 2024-04-07T17:41:47.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:41:46 smithi144 bash[66654]: cluster 2024-04-07T17:41:45.762661+0000 mon.a (mon.0) 4200 : cluster [DBG] osdmap e2244: 8 total, 8 up, 8 in 2024-04-07T17:41:47.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:41:46 smithi100 bash[74942]: cluster 2024-04-07T17:41:45.762661+0000 mon.a (mon.0) 4200 : cluster [DBG] osdmap e2244: 8 total, 8 up, 8 in 2024-04-07T17:41:47.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:41:46 smithi100 bash[76202]: cluster 2024-04-07T17:41:45.762661+0000 mon.a (mon.0) 4200 : cluster [DBG] osdmap e2244: 8 total, 8 up, 8 in 2024-04-07T17:41:48.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:41:47 smithi144 bash[66654]: cluster 2024-04-07T17:41:46.591525+0000 mgr.y (mgr.25175) 2592 : cluster [DBG] pgmap v4411: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 316 KiB/s rd, 3.0 KiB/s wr, 424 op/s 2024-04-07T17:41:48.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:41:47 smithi144 bash[66654]: cluster 2024-04-07T17:41:46.785527+0000 mon.a (mon.0) 4201 : cluster [DBG] osdmap e2245: 8 total, 8 up, 8 in 2024-04-07T17:41:48.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:41:47 smithi144 bash[66654]: audit 2024-04-07T17:41:46.865567+0000 mon.a (mon.0) 4202 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:41:48.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:41:47 smithi100 bash[74942]: cluster 2024-04-07T17:41:46.591525+0000 mgr.y (mgr.25175) 2592 : cluster [DBG] pgmap v4411: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 316 KiB/s rd, 3.0 KiB/s wr, 424 op/s 2024-04-07T17:41:48.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:41:47 smithi100 bash[74942]: cluster 2024-04-07T17:41:46.785527+0000 mon.a (mon.0) 4201 : cluster [DBG] osdmap e2245: 8 total, 8 up, 8 in 2024-04-07T17:41:48.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:41:47 smithi100 bash[74942]: audit 2024-04-07T17:41:46.865567+0000 mon.a (mon.0) 4202 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:41:48.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:41:47 smithi100 bash[76202]: cluster 2024-04-07T17:41:46.591525+0000 mgr.y (mgr.25175) 2592 : cluster [DBG] pgmap v4411: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 316 KiB/s rd, 3.0 KiB/s wr, 424 op/s 2024-04-07T17:41:48.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:41:47 smithi100 bash[76202]: cluster 2024-04-07T17:41:46.785527+0000 mon.a (mon.0) 4201 : cluster [DBG] osdmap e2245: 8 total, 8 up, 8 in 2024-04-07T17:41:48.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:41:47 smithi100 bash[76202]: audit 2024-04-07T17:41:46.865567+0000 mon.a (mon.0) 4202 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:41:48.704 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:41:48 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:41:48] "GET /metrics HTTP/1.1" 200 36745 "" "Prometheus/2.43.0" 2024-04-07T17:41:49.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:41:48 smithi144 bash[66654]: cluster 2024-04-07T17:41:47.797622+0000 mon.a (mon.0) 4203 : cluster [DBG] osdmap e2246: 8 total, 8 up, 8 in 2024-04-07T17:41:49.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:41:48 smithi100 bash[74942]: cluster 2024-04-07T17:41:47.797622+0000 mon.a (mon.0) 4203 : cluster [DBG] osdmap e2246: 8 total, 8 up, 8 in 2024-04-07T17:41:49.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:41:48 smithi100 bash[76202]: cluster 2024-04-07T17:41:47.797622+0000 mon.a (mon.0) 4203 : cluster [DBG] osdmap e2246: 8 total, 8 up, 8 in 2024-04-07T17:41:50.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:41:49 smithi144 bash[66654]: cluster 2024-04-07T17:41:48.592373+0000 mgr.y (mgr.25175) 2593 : cluster [DBG] pgmap v4414: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 5.0 KiB/s rd, 1023 B/s wr, 7 op/s 2024-04-07T17:41:50.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:41:49 smithi144 bash[66654]: cluster 2024-04-07T17:41:48.812697+0000 mon.a (mon.0) 4204 : cluster [DBG] osdmap e2247: 8 total, 8 up, 8 in 2024-04-07T17:41:50.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:41:49 smithi100 bash[74942]: cluster 2024-04-07T17:41:48.592373+0000 mgr.y (mgr.25175) 2593 : cluster [DBG] pgmap v4414: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 5.0 KiB/s rd, 1023 B/s wr, 7 op/s 2024-04-07T17:41:50.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:41:49 smithi100 bash[74942]: cluster 2024-04-07T17:41:48.812697+0000 mon.a (mon.0) 4204 : cluster [DBG] osdmap e2247: 8 total, 8 up, 8 in 2024-04-07T17:41:50.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:41:49 smithi100 bash[76202]: cluster 2024-04-07T17:41:48.592373+0000 mgr.y (mgr.25175) 2593 : cluster [DBG] pgmap v4414: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 5.0 KiB/s rd, 1023 B/s wr, 7 op/s 2024-04-07T17:41:50.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:41:49 smithi100 bash[76202]: cluster 2024-04-07T17:41:48.812697+0000 mon.a (mon.0) 4204 : cluster [DBG] osdmap e2247: 8 total, 8 up, 8 in 2024-04-07T17:41:51.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:41:50 smithi144 bash[66654]: cluster 2024-04-07T17:41:49.828549+0000 mon.a (mon.0) 4205 : cluster [DBG] osdmap e2248: 8 total, 8 up, 8 in 2024-04-07T17:41:51.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:41:50 smithi100 bash[74942]: cluster 2024-04-07T17:41:49.828549+0000 mon.a (mon.0) 4205 : cluster [DBG] osdmap e2248: 8 total, 8 up, 8 in 2024-04-07T17:41:51.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:41:50 smithi100 bash[76202]: cluster 2024-04-07T17:41:49.828549+0000 mon.a (mon.0) 4205 : cluster [DBG] osdmap e2248: 8 total, 8 up, 8 in 2024-04-07T17:41:52.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:41:51 smithi144 bash[66654]: cluster 2024-04-07T17:41:50.593297+0000 mgr.y (mgr.25175) 2594 : cluster [DBG] pgmap v4417: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 354 KiB/s rd, 4.0 KiB/s wr, 475 op/s 2024-04-07T17:41:52.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:41:51 smithi144 bash[66654]: cluster 2024-04-07T17:41:50.841901+0000 mon.a (mon.0) 4206 : cluster [DBG] osdmap e2249: 8 total, 8 up, 8 in 2024-04-07T17:41:52.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:41:51 smithi100 bash[74942]: cluster 2024-04-07T17:41:50.593297+0000 mgr.y (mgr.25175) 2594 : cluster [DBG] pgmap v4417: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 354 KiB/s rd, 4.0 KiB/s wr, 475 op/s 2024-04-07T17:41:52.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:41:51 smithi100 bash[74942]: cluster 2024-04-07T17:41:50.841901+0000 mon.a (mon.0) 4206 : cluster [DBG] osdmap e2249: 8 total, 8 up, 8 in 2024-04-07T17:41:52.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:41:51 smithi100 bash[76202]: cluster 2024-04-07T17:41:50.593297+0000 mgr.y (mgr.25175) 2594 : cluster [DBG] pgmap v4417: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 354 KiB/s rd, 4.0 KiB/s wr, 475 op/s 2024-04-07T17:41:52.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:41:51 smithi100 bash[76202]: cluster 2024-04-07T17:41:50.841901+0000 mon.a (mon.0) 4206 : cluster [DBG] osdmap e2249: 8 total, 8 up, 8 in 2024-04-07T17:41:52.618 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:41:52 smithi144 bash[95233]: ts=2024-04-07T17:41:52.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:41:53.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:41:52 smithi144 bash[66654]: cluster 2024-04-07T17:41:51.848885+0000 mon.a (mon.0) 4207 : cluster [DBG] osdmap e2250: 8 total, 8 up, 8 in 2024-04-07T17:41:53.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:41:52 smithi100 bash[74942]: cluster 2024-04-07T17:41:51.848885+0000 mon.a (mon.0) 4207 : cluster [DBG] osdmap e2250: 8 total, 8 up, 8 in 2024-04-07T17:41:53.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:41:52 smithi100 bash[76202]: cluster 2024-04-07T17:41:51.848885+0000 mon.a (mon.0) 4207 : cluster [DBG] osdmap e2250: 8 total, 8 up, 8 in 2024-04-07T17:41:54.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:41:53 smithi100 bash[74942]: cluster 2024-04-07T17:41:52.594618+0000 mgr.y (mgr.25175) 2595 : cluster [DBG] pgmap v4420: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 354 KiB/s rd, 3.7 KiB/s wr, 474 op/s 2024-04-07T17:41:54.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:41:53 smithi100 bash[74942]: cluster 2024-04-07T17:41:52.860182+0000 mon.a (mon.0) 4208 : cluster [DBG] osdmap e2251: 8 total, 8 up, 8 in 2024-04-07T17:41:54.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:41:53 smithi100 bash[76202]: cluster 2024-04-07T17:41:52.594618+0000 mgr.y (mgr.25175) 2595 : cluster [DBG] pgmap v4420: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 354 KiB/s rd, 3.7 KiB/s wr, 474 op/s 2024-04-07T17:41:54.205 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:41:53 smithi100 bash[76202]: cluster 2024-04-07T17:41:52.860182+0000 mon.a (mon.0) 4208 : cluster [DBG] osdmap e2251: 8 total, 8 up, 8 in 2024-04-07T17:41:54.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:41:53 smithi144 bash[66654]: cluster 2024-04-07T17:41:52.594618+0000 mgr.y (mgr.25175) 2595 : cluster [DBG] pgmap v4420: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 354 KiB/s rd, 3.7 KiB/s wr, 474 op/s 2024-04-07T17:41:54.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:41:53 smithi144 bash[66654]: cluster 2024-04-07T17:41:52.860182+0000 mon.a (mon.0) 4208 : cluster [DBG] osdmap e2251: 8 total, 8 up, 8 in 2024-04-07T17:41:55.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:41:54 smithi100 bash[74942]: cluster 2024-04-07T17:41:53.861248+0000 mon.a (mon.0) 4209 : cluster [DBG] osdmap e2252: 8 total, 8 up, 8 in 2024-04-07T17:41:55.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:41:54 smithi100 bash[76202]: cluster 2024-04-07T17:41:53.861248+0000 mon.a (mon.0) 4209 : cluster [DBG] osdmap e2252: 8 total, 8 up, 8 in 2024-04-07T17:41:55.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:41:54 smithi144 bash[66654]: cluster 2024-04-07T17:41:53.861248+0000 mon.a (mon.0) 4209 : cluster [DBG] osdmap e2252: 8 total, 8 up, 8 in 2024-04-07T17:41:56.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:41:55 smithi100 bash[74942]: cluster 2024-04-07T17:41:54.595692+0000 mgr.y (mgr.25175) 2596 : cluster [DBG] pgmap v4423: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 386 KiB/s rd, 3.7 KiB/s wr, 518 op/s 2024-04-07T17:41:56.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:41:55 smithi100 bash[74942]: cluster 2024-04-07T17:41:54.885751+0000 mon.a (mon.0) 4210 : cluster [DBG] osdmap e2253: 8 total, 8 up, 8 in 2024-04-07T17:41:56.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:41:55 smithi100 bash[76202]: cluster 2024-04-07T17:41:54.595692+0000 mgr.y (mgr.25175) 2596 : cluster [DBG] pgmap v4423: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 386 KiB/s rd, 3.7 KiB/s wr, 518 op/s 2024-04-07T17:41:56.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:41:55 smithi100 bash[76202]: cluster 2024-04-07T17:41:54.885751+0000 mon.a (mon.0) 4210 : cluster [DBG] osdmap e2253: 8 total, 8 up, 8 in 2024-04-07T17:41:56.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:41:55 smithi144 bash[66654]: cluster 2024-04-07T17:41:54.595692+0000 mgr.y (mgr.25175) 2596 : cluster [DBG] pgmap v4423: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 386 KiB/s rd, 3.7 KiB/s wr, 518 op/s 2024-04-07T17:41:56.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:41:55 smithi144 bash[66654]: cluster 2024-04-07T17:41:54.885751+0000 mon.a (mon.0) 4210 : cluster [DBG] osdmap e2253: 8 total, 8 up, 8 in 2024-04-07T17:41:57.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:41:56 smithi100 bash[74942]: cluster 2024-04-07T17:41:55.886439+0000 mon.a (mon.0) 4211 : cluster [DBG] osdmap e2254: 8 total, 8 up, 8 in 2024-04-07T17:41:57.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:41:56 smithi100 bash[76202]: cluster 2024-04-07T17:41:55.886439+0000 mon.a (mon.0) 4211 : cluster [DBG] osdmap e2254: 8 total, 8 up, 8 in 2024-04-07T17:41:57.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:41:56 smithi144 bash[66654]: cluster 2024-04-07T17:41:55.886439+0000 mon.a (mon.0) 4211 : cluster [DBG] osdmap e2254: 8 total, 8 up, 8 in 2024-04-07T17:41:58.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:41:57 smithi100 bash[74942]: cluster 2024-04-07T17:41:56.596408+0000 mgr.y (mgr.25175) 2597 : cluster [DBG] pgmap v4426: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 381 KiB/s rd, 3.0 KiB/s wr, 511 op/s 2024-04-07T17:41:58.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:41:57 smithi100 bash[74942]: cluster 2024-04-07T17:41:56.908187+0000 mon.a (mon.0) 4212 : cluster [DBG] osdmap e2255: 8 total, 8 up, 8 in 2024-04-07T17:41:58.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:41:57 smithi100 bash[76202]: cluster 2024-04-07T17:41:56.596408+0000 mgr.y (mgr.25175) 2597 : cluster [DBG] pgmap v4426: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 381 KiB/s rd, 3.0 KiB/s wr, 511 op/s 2024-04-07T17:41:58.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:41:57 smithi100 bash[76202]: cluster 2024-04-07T17:41:56.908187+0000 mon.a (mon.0) 4212 : cluster [DBG] osdmap e2255: 8 total, 8 up, 8 in 2024-04-07T17:41:58.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:41:57 smithi144 bash[66654]: cluster 2024-04-07T17:41:56.596408+0000 mgr.y (mgr.25175) 2597 : cluster [DBG] pgmap v4426: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 381 KiB/s rd, 3.0 KiB/s wr, 511 op/s 2024-04-07T17:41:58.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:41:57 smithi144 bash[66654]: cluster 2024-04-07T17:41:56.908187+0000 mon.a (mon.0) 4212 : cluster [DBG] osdmap e2255: 8 total, 8 up, 8 in 2024-04-07T17:41:58.704 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:41:58 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:41:58] "GET /metrics HTTP/1.1" 200 36745 "" "Prometheus/2.43.0" 2024-04-07T17:41:59.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:41:58 smithi100 bash[74942]: cluster 2024-04-07T17:41:57.911230+0000 mon.a (mon.0) 4213 : cluster [DBG] osdmap e2256: 8 total, 8 up, 8 in 2024-04-07T17:41:59.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:41:58 smithi100 bash[76202]: cluster 2024-04-07T17:41:57.911230+0000 mon.a (mon.0) 4213 : cluster [DBG] osdmap e2256: 8 total, 8 up, 8 in 2024-04-07T17:41:59.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:41:58 smithi144 bash[66654]: cluster 2024-04-07T17:41:57.911230+0000 mon.a (mon.0) 4213 : cluster [DBG] osdmap e2256: 8 total, 8 up, 8 in 2024-04-07T17:42:00.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:41:59 smithi100 bash[76202]: cluster 2024-04-07T17:41:58.597276+0000 mgr.y (mgr.25175) 2598 : cluster [DBG] pgmap v4429: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 4.7 KiB/s rd, 511 B/s wr, 6 op/s 2024-04-07T17:42:00.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:41:59 smithi100 bash[76202]: cluster 2024-04-07T17:41:58.939049+0000 mon.a (mon.0) 4214 : cluster [DBG] osdmap e2257: 8 total, 8 up, 8 in 2024-04-07T17:42:00.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:41:59 smithi100 bash[74942]: cluster 2024-04-07T17:41:58.597276+0000 mgr.y (mgr.25175) 2598 : cluster [DBG] pgmap v4429: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 4.7 KiB/s rd, 511 B/s wr, 6 op/s 2024-04-07T17:42:00.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:41:59 smithi100 bash[74942]: cluster 2024-04-07T17:41:58.939049+0000 mon.a (mon.0) 4214 : cluster [DBG] osdmap e2257: 8 total, 8 up, 8 in 2024-04-07T17:42:00.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:41:59 smithi144 bash[66654]: cluster 2024-04-07T17:41:58.597276+0000 mgr.y (mgr.25175) 2598 : cluster [DBG] pgmap v4429: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 4.7 KiB/s rd, 511 B/s wr, 6 op/s 2024-04-07T17:42:00.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:41:59 smithi144 bash[66654]: cluster 2024-04-07T17:41:58.939049+0000 mon.a (mon.0) 4214 : cluster [DBG] osdmap e2257: 8 total, 8 up, 8 in 2024-04-07T17:42:01.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:42:00 smithi144 bash[66654]: cluster 2024-04-07T17:41:59.937777+0000 mon.a (mon.0) 4215 : cluster [DBG] osdmap e2258: 8 total, 8 up, 8 in 2024-04-07T17:42:01.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:42:00 smithi100 bash[74942]: cluster 2024-04-07T17:41:59.937777+0000 mon.a (mon.0) 4215 : cluster [DBG] osdmap e2258: 8 total, 8 up, 8 in 2024-04-07T17:42:01.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:42:00 smithi100 bash[76202]: cluster 2024-04-07T17:41:59.937777+0000 mon.a (mon.0) 4215 : cluster [DBG] osdmap e2258: 8 total, 8 up, 8 in 2024-04-07T17:42:02.315 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:42:01 smithi144 bash[66654]: cluster 2024-04-07T17:42:00.597983+0000 mgr.y (mgr.25175) 2599 : cluster [DBG] pgmap v4432: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 418 KiB/s rd, 3.5 KiB/s wr, 561 op/s 2024-04-07T17:42:02.315 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:42:01 smithi144 bash[66654]: cluster 2024-04-07T17:42:00.962317+0000 mon.a (mon.0) 4216 : cluster [DBG] osdmap e2259: 8 total, 8 up, 8 in 2024-04-07T17:42:02.315 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:42:01 smithi144 bash[66654]: audit 2024-04-07T17:42:01.865999+0000 mon.a (mon.0) 4217 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:42:02.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:42:01 smithi100 bash[74942]: cluster 2024-04-07T17:42:00.597983+0000 mgr.y (mgr.25175) 2599 : cluster [DBG] pgmap v4432: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 418 KiB/s rd, 3.5 KiB/s wr, 561 op/s 2024-04-07T17:42:02.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:42:01 smithi100 bash[74942]: cluster 2024-04-07T17:42:00.962317+0000 mon.a (mon.0) 4216 : cluster [DBG] osdmap e2259: 8 total, 8 up, 8 in 2024-04-07T17:42:02.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:42:01 smithi100 bash[74942]: audit 2024-04-07T17:42:01.865999+0000 mon.a (mon.0) 4217 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:42:02.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:42:01 smithi100 bash[76202]: cluster 2024-04-07T17:42:00.597983+0000 mgr.y (mgr.25175) 2599 : cluster [DBG] pgmap v4432: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 418 KiB/s rd, 3.5 KiB/s wr, 561 op/s 2024-04-07T17:42:02.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:42:01 smithi100 bash[76202]: cluster 2024-04-07T17:42:00.962317+0000 mon.a (mon.0) 4216 : cluster [DBG] osdmap e2259: 8 total, 8 up, 8 in 2024-04-07T17:42:02.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:42:01 smithi100 bash[76202]: audit 2024-04-07T17:42:01.865999+0000 mon.a (mon.0) 4217 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:42:02.615 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:42:02 smithi144 bash[95233]: ts=2024-04-07T17:42:02.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:42:03.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:42:02 smithi144 bash[66654]: cluster 2024-04-07T17:42:01.961943+0000 mon.a (mon.0) 4218 : cluster [DBG] osdmap e2260: 8 total, 8 up, 8 in 2024-04-07T17:42:03.453 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:42:02 smithi100 bash[74942]: cluster 2024-04-07T17:42:01.961943+0000 mon.a (mon.0) 4218 : cluster [DBG] osdmap e2260: 8 total, 8 up, 8 in 2024-04-07T17:42:03.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:42:02 smithi100 bash[76202]: cluster 2024-04-07T17:42:01.961943+0000 mon.a (mon.0) 4218 : cluster [DBG] osdmap e2260: 8 total, 8 up, 8 in 2024-04-07T17:42:04.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:42:03 smithi144 bash[66654]: cluster 2024-04-07T17:42:02.598969+0000 mgr.y (mgr.25175) 2600 : cluster [DBG] pgmap v4435: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 418 KiB/s rd, 3.7 KiB/s wr, 561 op/s 2024-04-07T17:42:04.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:42:03 smithi144 bash[66654]: cluster 2024-04-07T17:42:02.985542+0000 mon.a (mon.0) 4219 : cluster [DBG] osdmap e2261: 8 total, 8 up, 8 in 2024-04-07T17:42:04.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:42:03 smithi100 bash[74942]: cluster 2024-04-07T17:42:02.598969+0000 mgr.y (mgr.25175) 2600 : cluster [DBG] pgmap v4435: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 418 KiB/s rd, 3.7 KiB/s wr, 561 op/s 2024-04-07T17:42:04.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:42:03 smithi100 bash[74942]: cluster 2024-04-07T17:42:02.985542+0000 mon.a (mon.0) 4219 : cluster [DBG] osdmap e2261: 8 total, 8 up, 8 in 2024-04-07T17:42:04.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:42:03 smithi100 bash[76202]: cluster 2024-04-07T17:42:02.598969+0000 mgr.y (mgr.25175) 2600 : cluster [DBG] pgmap v4435: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 418 KiB/s rd, 3.7 KiB/s wr, 561 op/s 2024-04-07T17:42:04.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:42:03 smithi100 bash[76202]: cluster 2024-04-07T17:42:02.985542+0000 mon.a (mon.0) 4219 : cluster [DBG] osdmap e2261: 8 total, 8 up, 8 in 2024-04-07T17:42:05.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:42:04 smithi144 bash[66654]: cluster 2024-04-07T17:42:03.987580+0000 mon.a (mon.0) 4220 : cluster [DBG] osdmap e2262: 8 total, 8 up, 8 in 2024-04-07T17:42:05.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:42:04 smithi100 bash[74942]: cluster 2024-04-07T17:42:03.987580+0000 mon.a (mon.0) 4220 : cluster [DBG] osdmap e2262: 8 total, 8 up, 8 in 2024-04-07T17:42:05.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:42:04 smithi100 bash[76202]: cluster 2024-04-07T17:42:03.987580+0000 mon.a (mon.0) 4220 : cluster [DBG] osdmap e2262: 8 total, 8 up, 8 in 2024-04-07T17:42:06.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:42:06 smithi144 bash[66654]: cluster 2024-04-07T17:42:04.599995+0000 mgr.y (mgr.25175) 2601 : cluster [DBG] pgmap v4438: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 451 KiB/s rd, 3.7 KiB/s wr, 606 op/s 2024-04-07T17:42:06.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:42:06 smithi144 bash[66654]: cluster 2024-04-07T17:42:05.007855+0000 mon.a (mon.0) 4221 : cluster [DBG] osdmap e2263: 8 total, 8 up, 8 in 2024-04-07T17:42:06.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:42:06 smithi100 bash[74942]: cluster 2024-04-07T17:42:04.599995+0000 mgr.y (mgr.25175) 2601 : cluster [DBG] pgmap v4438: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 451 KiB/s rd, 3.7 KiB/s wr, 606 op/s 2024-04-07T17:42:06.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:42:06 smithi100 bash[74942]: cluster 2024-04-07T17:42:05.007855+0000 mon.a (mon.0) 4221 : cluster [DBG] osdmap e2263: 8 total, 8 up, 8 in 2024-04-07T17:42:06.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:42:06 smithi100 bash[76202]: cluster 2024-04-07T17:42:04.599995+0000 mgr.y (mgr.25175) 2601 : cluster [DBG] pgmap v4438: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 451 KiB/s rd, 3.7 KiB/s wr, 606 op/s 2024-04-07T17:42:06.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:42:06 smithi100 bash[76202]: cluster 2024-04-07T17:42:05.007855+0000 mon.a (mon.0) 4221 : cluster [DBG] osdmap e2263: 8 total, 8 up, 8 in 2024-04-07T17:42:07.364 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:42:07 smithi144 bash[66654]: cluster 2024-04-07T17:42:06.010550+0000 mon.a (mon.0) 4222 : cluster [DBG] osdmap e2264: 8 total, 8 up, 8 in 2024-04-07T17:42:07.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:42:07 smithi100 bash[74942]: cluster 2024-04-07T17:42:06.010550+0000 mon.a (mon.0) 4222 : cluster [DBG] osdmap e2264: 8 total, 8 up, 8 in 2024-04-07T17:42:07.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:42:07 smithi100 bash[76202]: cluster 2024-04-07T17:42:06.010550+0000 mon.a (mon.0) 4222 : cluster [DBG] osdmap e2264: 8 total, 8 up, 8 in 2024-04-07T17:42:08.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:42:08 smithi144 bash[66654]: cluster 2024-04-07T17:42:06.600767+0000 mgr.y (mgr.25175) 2602 : cluster [DBG] pgmap v4441: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 447 KiB/s rd, 3.0 KiB/s wr, 600 op/s 2024-04-07T17:42:08.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:42:08 smithi144 bash[66654]: cluster 2024-04-07T17:42:07.036514+0000 mon.a (mon.0) 4223 : cluster [DBG] osdmap e2265: 8 total, 8 up, 8 in 2024-04-07T17:42:08.426 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:42:08 smithi100 bash[74942]: cluster 2024-04-07T17:42:06.600767+0000 mgr.y (mgr.25175) 2602 : cluster [DBG] pgmap v4441: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 447 KiB/s rd, 3.0 KiB/s wr, 600 op/s 2024-04-07T17:42:08.426 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:42:08 smithi100 bash[74942]: cluster 2024-04-07T17:42:07.036514+0000 mon.a (mon.0) 4223 : cluster [DBG] osdmap e2265: 8 total, 8 up, 8 in 2024-04-07T17:42:08.427 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:42:08 smithi100 bash[76202]: cluster 2024-04-07T17:42:06.600767+0000 mgr.y (mgr.25175) 2602 : cluster [DBG] pgmap v4441: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 447 KiB/s rd, 3.0 KiB/s wr, 600 op/s 2024-04-07T17:42:08.427 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:42:08 smithi100 bash[76202]: cluster 2024-04-07T17:42:07.036514+0000 mon.a (mon.0) 4223 : cluster [DBG] osdmap e2265: 8 total, 8 up, 8 in 2024-04-07T17:42:08.704 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:42:08 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:42:08] "GET /metrics HTTP/1.1" 200 36745 "" "Prometheus/2.43.0" 2024-04-07T17:42:09.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:42:09 smithi144 bash[66654]: cluster 2024-04-07T17:42:08.032784+0000 mon.a (mon.0) 4224 : cluster [DBG] osdmap e2266: 8 total, 8 up, 8 in 2024-04-07T17:42:09.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:42:09 smithi100 bash[74942]: cluster 2024-04-07T17:42:08.032784+0000 mon.a (mon.0) 4224 : cluster [DBG] osdmap e2266: 8 total, 8 up, 8 in 2024-04-07T17:42:09.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:42:09 smithi100 bash[76202]: cluster 2024-04-07T17:42:08.032784+0000 mon.a (mon.0) 4224 : cluster [DBG] osdmap e2266: 8 total, 8 up, 8 in 2024-04-07T17:42:10.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:42:10 smithi144 bash[66654]: cluster 2024-04-07T17:42:08.601481+0000 mgr.y (mgr.25175) 2603 : cluster [DBG] pgmap v4444: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 5.0 KiB/s rd, 1023 B/s wr, 7 op/s 2024-04-07T17:42:10.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:42:10 smithi144 bash[66654]: cluster 2024-04-07T17:42:09.059340+0000 mon.a (mon.0) 4225 : cluster [DBG] osdmap e2267: 8 total, 8 up, 8 in 2024-04-07T17:42:10.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:42:10 smithi100 bash[74942]: cluster 2024-04-07T17:42:08.601481+0000 mgr.y (mgr.25175) 2603 : cluster [DBG] pgmap v4444: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 5.0 KiB/s rd, 1023 B/s wr, 7 op/s 2024-04-07T17:42:10.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:42:10 smithi100 bash[74942]: cluster 2024-04-07T17:42:09.059340+0000 mon.a (mon.0) 4225 : cluster [DBG] osdmap e2267: 8 total, 8 up, 8 in 2024-04-07T17:42:10.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:42:10 smithi100 bash[76202]: cluster 2024-04-07T17:42:08.601481+0000 mgr.y (mgr.25175) 2603 : cluster [DBG] pgmap v4444: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 5.0 KiB/s rd, 1023 B/s wr, 7 op/s 2024-04-07T17:42:10.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:42:10 smithi100 bash[76202]: cluster 2024-04-07T17:42:09.059340+0000 mon.a (mon.0) 4225 : cluster [DBG] osdmap e2267: 8 total, 8 up, 8 in 2024-04-07T17:42:11.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:42:11 smithi144 bash[66654]: cluster 2024-04-07T17:42:10.058259+0000 mon.a (mon.0) 4226 : cluster [DBG] osdmap e2268: 8 total, 8 up, 8 in 2024-04-07T17:42:11.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:42:11 smithi100 bash[74942]: cluster 2024-04-07T17:42:10.058259+0000 mon.a (mon.0) 4226 : cluster [DBG] osdmap e2268: 8 total, 8 up, 8 in 2024-04-07T17:42:11.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:42:11 smithi100 bash[76202]: cluster 2024-04-07T17:42:10.058259+0000 mon.a (mon.0) 4226 : cluster [DBG] osdmap e2268: 8 total, 8 up, 8 in 2024-04-07T17:42:12.365 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:42:12 smithi144 bash[95233]: ts=2024-04-07T17:42:12.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:42:12.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:42:12 smithi144 bash[66654]: cluster 2024-04-07T17:42:10.602148+0000 mgr.y (mgr.25175) 2604 : cluster [DBG] pgmap v4447: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 386 KiB/s rd, 3.2 KiB/s wr, 520 op/s 2024-04-07T17:42:12.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:42:12 smithi144 bash[66654]: cluster 2024-04-07T17:42:11.080011+0000 mon.a (mon.0) 4227 : cluster [DBG] osdmap e2269: 8 total, 8 up, 8 in 2024-04-07T17:42:12.453 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:42:12 smithi100 bash[74942]: cluster 2024-04-07T17:42:10.602148+0000 mgr.y (mgr.25175) 2604 : cluster [DBG] pgmap v4447: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 386 KiB/s rd, 3.2 KiB/s wr, 520 op/s 2024-04-07T17:42:12.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:42:12 smithi100 bash[74942]: cluster 2024-04-07T17:42:11.080011+0000 mon.a (mon.0) 4227 : cluster [DBG] osdmap e2269: 8 total, 8 up, 8 in 2024-04-07T17:42:12.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:42:12 smithi100 bash[76202]: cluster 2024-04-07T17:42:10.602148+0000 mgr.y (mgr.25175) 2604 : cluster [DBG] pgmap v4447: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 386 KiB/s rd, 3.2 KiB/s wr, 520 op/s 2024-04-07T17:42:12.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:42:12 smithi100 bash[76202]: cluster 2024-04-07T17:42:11.080011+0000 mon.a (mon.0) 4227 : cluster [DBG] osdmap e2269: 8 total, 8 up, 8 in 2024-04-07T17:42:13.364 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:42:13 smithi144 bash[66654]: cluster 2024-04-07T17:42:12.082975+0000 mon.a (mon.0) 4228 : cluster [DBG] osdmap e2270: 8 total, 8 up, 8 in 2024-04-07T17:42:13.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:42:13 smithi100 bash[74942]: cluster 2024-04-07T17:42:12.082975+0000 mon.a (mon.0) 4228 : cluster [DBG] osdmap e2270: 8 total, 8 up, 8 in 2024-04-07T17:42:13.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:42:13 smithi100 bash[76202]: cluster 2024-04-07T17:42:12.082975+0000 mon.a (mon.0) 4228 : cluster [DBG] osdmap e2270: 8 total, 8 up, 8 in 2024-04-07T17:42:14.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:42:14 smithi144 bash[66654]: cluster 2024-04-07T17:42:12.602886+0000 mgr.y (mgr.25175) 2605 : cluster [DBG] pgmap v4450: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 386 KiB/s rd, 2.5 KiB/s wr, 520 op/s 2024-04-07T17:42:14.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:42:14 smithi144 bash[66654]: cluster 2024-04-07T17:42:13.106724+0000 mon.a (mon.0) 4229 : cluster [DBG] osdmap e2271: 8 total, 8 up, 8 in 2024-04-07T17:42:14.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:42:14 smithi100 bash[74942]: cluster 2024-04-07T17:42:12.602886+0000 mgr.y (mgr.25175) 2605 : cluster [DBG] pgmap v4450: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 386 KiB/s rd, 2.5 KiB/s wr, 520 op/s 2024-04-07T17:42:14.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:42:14 smithi100 bash[74942]: cluster 2024-04-07T17:42:13.106724+0000 mon.a (mon.0) 4229 : cluster [DBG] osdmap e2271: 8 total, 8 up, 8 in 2024-04-07T17:42:14.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:42:14 smithi100 bash[76202]: cluster 2024-04-07T17:42:12.602886+0000 mgr.y (mgr.25175) 2605 : cluster [DBG] pgmap v4450: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 386 KiB/s rd, 2.5 KiB/s wr, 520 op/s 2024-04-07T17:42:14.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:42:14 smithi100 bash[76202]: cluster 2024-04-07T17:42:13.106724+0000 mon.a (mon.0) 4229 : cluster [DBG] osdmap e2271: 8 total, 8 up, 8 in 2024-04-07T17:42:15.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:42:15 smithi100 bash[74942]: cluster 2024-04-07T17:42:14.114818+0000 mon.a (mon.0) 4230 : cluster [DBG] osdmap e2272: 8 total, 8 up, 8 in 2024-04-07T17:42:15.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:42:15 smithi100 bash[76202]: cluster 2024-04-07T17:42:14.114818+0000 mon.a (mon.0) 4230 : cluster [DBG] osdmap e2272: 8 total, 8 up, 8 in 2024-04-07T17:42:15.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:42:15 smithi144 bash[66654]: cluster 2024-04-07T17:42:14.114818+0000 mon.a (mon.0) 4230 : cluster [DBG] osdmap e2272: 8 total, 8 up, 8 in 2024-04-07T17:42:16.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:42:16 smithi100 bash[74942]: cluster 2024-04-07T17:42:14.603574+0000 mgr.y (mgr.25175) 2606 : cluster [DBG] pgmap v4453: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 511 KiB/s rd, 3.2 KiB/s wr, 688 op/s 2024-04-07T17:42:16.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:42:16 smithi100 bash[74942]: cluster 2024-04-07T17:42:15.128123+0000 mon.a (mon.0) 4231 : cluster [DBG] osdmap e2273: 8 total, 8 up, 8 in 2024-04-07T17:42:16.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:42:16 smithi100 bash[76202]: cluster 2024-04-07T17:42:14.603574+0000 mgr.y (mgr.25175) 2606 : cluster [DBG] pgmap v4453: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 511 KiB/s rd, 3.2 KiB/s wr, 688 op/s 2024-04-07T17:42:16.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:42:16 smithi100 bash[76202]: cluster 2024-04-07T17:42:15.128123+0000 mon.a (mon.0) 4231 : cluster [DBG] osdmap e2273: 8 total, 8 up, 8 in 2024-04-07T17:42:16.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:42:16 smithi144 bash[66654]: cluster 2024-04-07T17:42:14.603574+0000 mgr.y (mgr.25175) 2606 : cluster [DBG] pgmap v4453: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 511 KiB/s rd, 3.2 KiB/s wr, 688 op/s 2024-04-07T17:42:16.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:42:16 smithi144 bash[66654]: cluster 2024-04-07T17:42:15.128123+0000 mon.a (mon.0) 4231 : cluster [DBG] osdmap e2273: 8 total, 8 up, 8 in 2024-04-07T17:42:17.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:42:17 smithi100 bash[74942]: cluster 2024-04-07T17:42:16.127020+0000 mon.a (mon.0) 4232 : cluster [DBG] osdmap e2274: 8 total, 8 up, 8 in 2024-04-07T17:42:17.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:42:17 smithi100 bash[74942]: audit 2024-04-07T17:42:16.866611+0000 mon.a (mon.0) 4233 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:42:17.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:42:17 smithi100 bash[76202]: cluster 2024-04-07T17:42:16.127020+0000 mon.a (mon.0) 4232 : cluster [DBG] osdmap e2274: 8 total, 8 up, 8 in 2024-04-07T17:42:17.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:42:17 smithi100 bash[76202]: audit 2024-04-07T17:42:16.866611+0000 mon.a (mon.0) 4233 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:42:17.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:42:17 smithi144 bash[66654]: cluster 2024-04-07T17:42:16.127020+0000 mon.a (mon.0) 4232 : cluster [DBG] osdmap e2274: 8 total, 8 up, 8 in 2024-04-07T17:42:17.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:42:17 smithi144 bash[66654]: audit 2024-04-07T17:42:16.866611+0000 mon.a (mon.0) 4233 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:42:18.427 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:42:18 smithi100 bash[74942]: cluster 2024-04-07T17:42:16.604221+0000 mgr.y (mgr.25175) 2607 : cluster [DBG] pgmap v4456: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 506 KiB/s rd, 3.5 KiB/s wr, 682 op/s 2024-04-07T17:42:18.427 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:42:18 smithi100 bash[74942]: cluster 2024-04-07T17:42:17.148652+0000 mon.a (mon.0) 4234 : cluster [DBG] osdmap e2275: 8 total, 8 up, 8 in 2024-04-07T17:42:18.427 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:42:18 smithi100 bash[76202]: cluster 2024-04-07T17:42:16.604221+0000 mgr.y (mgr.25175) 2607 : cluster [DBG] pgmap v4456: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 506 KiB/s rd, 3.5 KiB/s wr, 682 op/s 2024-04-07T17:42:18.427 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:42:18 smithi100 bash[76202]: cluster 2024-04-07T17:42:17.148652+0000 mon.a (mon.0) 4234 : cluster [DBG] osdmap e2275: 8 total, 8 up, 8 in 2024-04-07T17:42:18.614 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:42:18 smithi144 bash[66654]: cluster 2024-04-07T17:42:16.604221+0000 mgr.y (mgr.25175) 2607 : cluster [DBG] pgmap v4456: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 506 KiB/s rd, 3.5 KiB/s wr, 682 op/s 2024-04-07T17:42:18.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:42:18 smithi144 bash[66654]: cluster 2024-04-07T17:42:17.148652+0000 mon.a (mon.0) 4234 : cluster [DBG] osdmap e2275: 8 total, 8 up, 8 in 2024-04-07T17:42:18.703 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:42:18 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:42:18] "GET /metrics HTTP/1.1" 200 36742 "" "Prometheus/2.43.0" 2024-04-07T17:42:19.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:42:19 smithi100 bash[74942]: cluster 2024-04-07T17:42:18.156443+0000 mon.a (mon.0) 4235 : cluster [DBG] osdmap e2276: 8 total, 8 up, 8 in 2024-04-07T17:42:19.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:42:19 smithi100 bash[76202]: cluster 2024-04-07T17:42:18.156443+0000 mon.a (mon.0) 4235 : cluster [DBG] osdmap e2276: 8 total, 8 up, 8 in 2024-04-07T17:42:19.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:42:19 smithi144 bash[66654]: cluster 2024-04-07T17:42:18.156443+0000 mon.a (mon.0) 4235 : cluster [DBG] osdmap e2276: 8 total, 8 up, 8 in 2024-04-07T17:42:20.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:42:20 smithi100 bash[74942]: cluster 2024-04-07T17:42:18.604960+0000 mgr.y (mgr.25175) 2608 : cluster [DBG] pgmap v4459: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 5.0 KiB/s rd, 1023 B/s wr, 7 op/s 2024-04-07T17:42:20.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:42:20 smithi100 bash[74942]: cluster 2024-04-07T17:42:19.167337+0000 mon.a (mon.0) 4236 : cluster [DBG] osdmap e2277: 8 total, 8 up, 8 in 2024-04-07T17:42:20.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:42:20 smithi100 bash[76202]: cluster 2024-04-07T17:42:18.604960+0000 mgr.y (mgr.25175) 2608 : cluster [DBG] pgmap v4459: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 5.0 KiB/s rd, 1023 B/s wr, 7 op/s 2024-04-07T17:42:20.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:42:20 smithi100 bash[76202]: cluster 2024-04-07T17:42:19.167337+0000 mon.a (mon.0) 4236 : cluster [DBG] osdmap e2277: 8 total, 8 up, 8 in 2024-04-07T17:42:20.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:42:20 smithi144 bash[66654]: cluster 2024-04-07T17:42:18.604960+0000 mgr.y (mgr.25175) 2608 : cluster [DBG] pgmap v4459: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 5.0 KiB/s rd, 1023 B/s wr, 7 op/s 2024-04-07T17:42:20.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:42:20 smithi144 bash[66654]: cluster 2024-04-07T17:42:19.167337+0000 mon.a (mon.0) 4236 : cluster [DBG] osdmap e2277: 8 total, 8 up, 8 in 2024-04-07T17:42:21.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:42:21 smithi100 bash[74942]: cluster 2024-04-07T17:42:20.170099+0000 mon.a (mon.0) 4237 : cluster [DBG] osdmap e2278: 8 total, 8 up, 8 in 2024-04-07T17:42:21.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:42:21 smithi100 bash[76202]: cluster 2024-04-07T17:42:20.170099+0000 mon.a (mon.0) 4237 : cluster [DBG] osdmap e2278: 8 total, 8 up, 8 in 2024-04-07T17:42:21.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:42:21 smithi144 bash[66654]: cluster 2024-04-07T17:42:20.170099+0000 mon.a (mon.0) 4237 : cluster [DBG] osdmap e2278: 8 total, 8 up, 8 in 2024-04-07T17:42:22.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:42:22 smithi100 bash[74942]: cluster 2024-04-07T17:42:20.605673+0000 mgr.y (mgr.25175) 2609 : cluster [DBG] pgmap v4462: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 544 KiB/s rd, 3.7 KiB/s wr, 732 op/s 2024-04-07T17:42:22.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:42:22 smithi100 bash[74942]: cluster 2024-04-07T17:42:21.191901+0000 mon.a (mon.0) 4238 : cluster [DBG] osdmap e2279: 8 total, 8 up, 8 in 2024-04-07T17:42:22.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:42:22 smithi100 bash[76202]: cluster 2024-04-07T17:42:20.605673+0000 mgr.y (mgr.25175) 2609 : cluster [DBG] pgmap v4462: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 544 KiB/s rd, 3.7 KiB/s wr, 732 op/s 2024-04-07T17:42:22.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:42:22 smithi100 bash[76202]: cluster 2024-04-07T17:42:21.191901+0000 mon.a (mon.0) 4238 : cluster [DBG] osdmap e2279: 8 total, 8 up, 8 in 2024-04-07T17:42:22.615 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:42:22 smithi144 bash[95233]: ts=2024-04-07T17:42:22.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:42:22.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:42:22 smithi144 bash[66654]: cluster 2024-04-07T17:42:20.605673+0000 mgr.y (mgr.25175) 2609 : cluster [DBG] pgmap v4462: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 544 KiB/s rd, 3.7 KiB/s wr, 732 op/s 2024-04-07T17:42:22.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:42:22 smithi144 bash[66654]: cluster 2024-04-07T17:42:21.191901+0000 mon.a (mon.0) 4238 : cluster [DBG] osdmap e2279: 8 total, 8 up, 8 in 2024-04-07T17:42:23.614 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:42:23 smithi144 bash[66654]: cluster 2024-04-07T17:42:22.193720+0000 mon.a (mon.0) 4239 : cluster [DBG] osdmap e2280: 8 total, 8 up, 8 in 2024-04-07T17:42:23.703 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:42:23 smithi100 bash[74942]: cluster 2024-04-07T17:42:22.193720+0000 mon.a (mon.0) 4239 : cluster [DBG] osdmap e2280: 8 total, 8 up, 8 in 2024-04-07T17:42:23.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:42:23 smithi100 bash[76202]: cluster 2024-04-07T17:42:22.193720+0000 mon.a (mon.0) 4239 : cluster [DBG] osdmap e2280: 8 total, 8 up, 8 in 2024-04-07T17:42:24.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:42:24 smithi144 bash[66654]: cluster 2024-04-07T17:42:22.606363+0000 mgr.y (mgr.25175) 2610 : cluster [DBG] pgmap v4465: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 544 KiB/s rd, 3.7 KiB/s wr, 732 op/s 2024-04-07T17:42:24.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:42:24 smithi144 bash[66654]: cluster 2024-04-07T17:42:23.218119+0000 mon.a (mon.0) 4240 : cluster [DBG] osdmap e2281: 8 total, 8 up, 8 in 2024-04-07T17:42:24.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:42:24 smithi100 bash[74942]: cluster 2024-04-07T17:42:22.606363+0000 mgr.y (mgr.25175) 2610 : cluster [DBG] pgmap v4465: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 544 KiB/s rd, 3.7 KiB/s wr, 732 op/s 2024-04-07T17:42:24.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:42:24 smithi100 bash[74942]: cluster 2024-04-07T17:42:23.218119+0000 mon.a (mon.0) 4240 : cluster [DBG] osdmap e2281: 8 total, 8 up, 8 in 2024-04-07T17:42:24.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:42:24 smithi100 bash[76202]: cluster 2024-04-07T17:42:22.606363+0000 mgr.y (mgr.25175) 2610 : cluster [DBG] pgmap v4465: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 544 KiB/s rd, 3.7 KiB/s wr, 732 op/s 2024-04-07T17:42:24.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:42:24 smithi100 bash[76202]: cluster 2024-04-07T17:42:23.218119+0000 mon.a (mon.0) 4240 : cluster [DBG] osdmap e2281: 8 total, 8 up, 8 in 2024-04-07T17:42:25.614 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:42:25 smithi144 bash[66654]: cluster 2024-04-07T17:42:24.219987+0000 mon.a (mon.0) 4241 : cluster [DBG] osdmap e2282: 8 total, 8 up, 8 in 2024-04-07T17:42:25.703 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:42:25 smithi100 bash[74942]: cluster 2024-04-07T17:42:24.219987+0000 mon.a (mon.0) 4241 : cluster [DBG] osdmap e2282: 8 total, 8 up, 8 in 2024-04-07T17:42:25.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:42:25 smithi100 bash[76202]: cluster 2024-04-07T17:42:24.219987+0000 mon.a (mon.0) 4241 : cluster [DBG] osdmap e2282: 8 total, 8 up, 8 in 2024-04-07T17:42:26.614 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:42:26 smithi144 bash[66654]: cluster 2024-04-07T17:42:24.607023+0000 mgr.y (mgr.25175) 2611 : cluster [DBG] pgmap v4468: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 575 KiB/s rd, 3.5 KiB/s wr, 775 op/s 2024-04-07T17:42:26.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:42:26 smithi144 bash[66654]: cluster 2024-04-07T17:42:25.244656+0000 mon.a (mon.0) 4242 : cluster [DBG] osdmap e2283: 8 total, 8 up, 8 in 2024-04-07T17:42:26.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:42:26 smithi100 bash[76202]: cluster 2024-04-07T17:42:24.607023+0000 mgr.y (mgr.25175) 2611 : cluster [DBG] pgmap v4468: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 575 KiB/s rd, 3.5 KiB/s wr, 775 op/s 2024-04-07T17:42:26.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:42:26 smithi100 bash[76202]: cluster 2024-04-07T17:42:25.244656+0000 mon.a (mon.0) 4242 : cluster [DBG] osdmap e2283: 8 total, 8 up, 8 in 2024-04-07T17:42:26.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:42:26 smithi100 bash[74942]: cluster 2024-04-07T17:42:24.607023+0000 mgr.y (mgr.25175) 2611 : cluster [DBG] pgmap v4468: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 575 KiB/s rd, 3.5 KiB/s wr, 775 op/s 2024-04-07T17:42:26.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:42:26 smithi100 bash[74942]: cluster 2024-04-07T17:42:25.244656+0000 mon.a (mon.0) 4242 : cluster [DBG] osdmap e2283: 8 total, 8 up, 8 in 2024-04-07T17:42:27.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:42:27 smithi144 bash[66654]: cluster 2024-04-07T17:42:26.258258+0000 mon.a (mon.0) 4243 : cluster [DBG] osdmap e2284: 8 total, 8 up, 8 in 2024-04-07T17:42:27.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:42:27 smithi144 bash[66654]: cluster 2024-04-07T17:42:26.607691+0000 mgr.y (mgr.25175) 2612 : cluster [DBG] pgmap v4471: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 570 KiB/s rd, 3.5 KiB/s wr, 769 op/s 2024-04-07T17:42:27.703 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:42:27 smithi100 bash[74942]: cluster 2024-04-07T17:42:26.258258+0000 mon.a (mon.0) 4243 : cluster [DBG] osdmap e2284: 8 total, 8 up, 8 in 2024-04-07T17:42:27.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:42:27 smithi100 bash[74942]: cluster 2024-04-07T17:42:26.607691+0000 mgr.y (mgr.25175) 2612 : cluster [DBG] pgmap v4471: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 570 KiB/s rd, 3.5 KiB/s wr, 769 op/s 2024-04-07T17:42:27.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:42:27 smithi100 bash[76202]: cluster 2024-04-07T17:42:26.258258+0000 mon.a (mon.0) 4243 : cluster [DBG] osdmap e2284: 8 total, 8 up, 8 in 2024-04-07T17:42:27.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:42:27 smithi100 bash[76202]: cluster 2024-04-07T17:42:26.607691+0000 mgr.y (mgr.25175) 2612 : cluster [DBG] pgmap v4471: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 570 KiB/s rd, 3.5 KiB/s wr, 769 op/s 2024-04-07T17:42:28.614 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:42:28 smithi144 bash[66654]: cluster 2024-04-07T17:42:27.268638+0000 mon.a (mon.0) 4244 : cluster [DBG] osdmap e2285: 8 total, 8 up, 8 in 2024-04-07T17:42:28.703 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:42:28 smithi100 bash[74942]: cluster 2024-04-07T17:42:27.268638+0000 mon.a (mon.0) 4244 : cluster [DBG] osdmap e2285: 8 total, 8 up, 8 in 2024-04-07T17:42:28.704 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:42:28 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:42:28] "GET /metrics HTTP/1.1" 200 36742 "" "Prometheus/2.43.0" 2024-04-07T17:42:28.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:42:28 smithi100 bash[76202]: cluster 2024-04-07T17:42:27.268638+0000 mon.a (mon.0) 4244 : cluster [DBG] osdmap e2285: 8 total, 8 up, 8 in 2024-04-07T17:42:29.614 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:42:29 smithi144 bash[66654]: cluster 2024-04-07T17:42:28.271996+0000 mon.a (mon.0) 4245 : cluster [DBG] osdmap e2286: 8 total, 8 up, 8 in 2024-04-07T17:42:29.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:42:29 smithi144 bash[66654]: cluster 2024-04-07T17:42:28.608411+0000 mgr.y (mgr.25175) 2613 : cluster [DBG] pgmap v4474: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 5.0 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-04-07T17:42:29.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:42:29 smithi100 bash[74942]: cluster 2024-04-07T17:42:28.271996+0000 mon.a (mon.0) 4245 : cluster [DBG] osdmap e2286: 8 total, 8 up, 8 in 2024-04-07T17:42:29.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:42:29 smithi100 bash[74942]: cluster 2024-04-07T17:42:28.608411+0000 mgr.y (mgr.25175) 2613 : cluster [DBG] pgmap v4474: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 5.0 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-04-07T17:42:29.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:42:29 smithi100 bash[76202]: cluster 2024-04-07T17:42:28.271996+0000 mon.a (mon.0) 4245 : cluster [DBG] osdmap e2286: 8 total, 8 up, 8 in 2024-04-07T17:42:29.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:42:29 smithi100 bash[76202]: cluster 2024-04-07T17:42:28.608411+0000 mgr.y (mgr.25175) 2613 : cluster [DBG] pgmap v4474: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 5.0 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-04-07T17:42:30.614 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:42:30 smithi144 bash[66654]: cluster 2024-04-07T17:42:29.315124+0000 mon.a (mon.0) 4246 : cluster [DBG] osdmap e2287: 8 total, 8 up, 8 in 2024-04-07T17:42:30.705 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:42:30 smithi100 bash[74942]: cluster 2024-04-07T17:42:29.315124+0000 mon.a (mon.0) 4246 : cluster [DBG] osdmap e2287: 8 total, 8 up, 8 in 2024-04-07T17:42:30.705 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:42:30 smithi100 bash[76202]: cluster 2024-04-07T17:42:29.315124+0000 mon.a (mon.0) 4246 : cluster [DBG] osdmap e2287: 8 total, 8 up, 8 in 2024-04-07T17:42:31.614 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:42:31 smithi144 bash[66654]: cluster 2024-04-07T17:42:30.329644+0000 mon.a (mon.0) 4247 : cluster [DBG] osdmap e2288: 8 total, 8 up, 8 in 2024-04-07T17:42:31.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:42:31 smithi144 bash[66654]: cluster 2024-04-07T17:42:30.609142+0000 mgr.y (mgr.25175) 2614 : cluster [DBG] pgmap v4477: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 609 KiB/s rd, 3.2 KiB/s wr, 819 op/s 2024-04-07T17:42:31.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:42:31 smithi100 bash[74942]: cluster 2024-04-07T17:42:30.329644+0000 mon.a (mon.0) 4247 : cluster [DBG] osdmap e2288: 8 total, 8 up, 8 in 2024-04-07T17:42:31.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:42:31 smithi100 bash[74942]: cluster 2024-04-07T17:42:30.609142+0000 mgr.y (mgr.25175) 2614 : cluster [DBG] pgmap v4477: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 609 KiB/s rd, 3.2 KiB/s wr, 819 op/s 2024-04-07T17:42:31.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:42:31 smithi100 bash[76202]: cluster 2024-04-07T17:42:30.329644+0000 mon.a (mon.0) 4247 : cluster [DBG] osdmap e2288: 8 total, 8 up, 8 in 2024-04-07T17:42:31.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:42:31 smithi100 bash[76202]: cluster 2024-04-07T17:42:30.609142+0000 mgr.y (mgr.25175) 2614 : cluster [DBG] pgmap v4477: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 609 KiB/s rd, 3.2 KiB/s wr, 819 op/s 2024-04-07T17:42:32.615 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:42:32 smithi144 bash[95233]: ts=2024-04-07T17:42:32.317Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:42:32.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:42:32 smithi144 bash[66654]: cluster 2024-04-07T17:42:31.340293+0000 mon.a (mon.0) 4248 : cluster [DBG] osdmap e2289: 8 total, 8 up, 8 in 2024-04-07T17:42:32.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:42:32 smithi144 bash[66654]: audit 2024-04-07T17:42:31.866902+0000 mon.a (mon.0) 4249 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:42:32.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:42:32 smithi100 bash[74942]: cluster 2024-04-07T17:42:31.340293+0000 mon.a (mon.0) 4248 : cluster [DBG] osdmap e2289: 8 total, 8 up, 8 in 2024-04-07T17:42:32.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:42:32 smithi100 bash[74942]: audit 2024-04-07T17:42:31.866902+0000 mon.a (mon.0) 4249 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:42:32.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:42:32 smithi100 bash[76202]: cluster 2024-04-07T17:42:31.340293+0000 mon.a (mon.0) 4248 : cluster [DBG] osdmap e2289: 8 total, 8 up, 8 in 2024-04-07T17:42:32.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:42:32 smithi100 bash[76202]: audit 2024-04-07T17:42:31.866902+0000 mon.a (mon.0) 4249 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:42:33.614 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:42:33 smithi144 bash[66654]: cluster 2024-04-07T17:42:32.342096+0000 mon.a (mon.0) 4250 : cluster [DBG] osdmap e2290: 8 total, 8 up, 8 in 2024-04-07T17:42:33.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:42:33 smithi144 bash[66654]: cluster 2024-04-07T17:42:32.609836+0000 mgr.y (mgr.25175) 2615 : cluster [DBG] pgmap v4480: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 609 KiB/s rd, 3.5 KiB/s wr, 819 op/s 2024-04-07T17:42:33.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:42:33 smithi100 bash[74942]: cluster 2024-04-07T17:42:32.342096+0000 mon.a (mon.0) 4250 : cluster [DBG] osdmap e2290: 8 total, 8 up, 8 in 2024-04-07T17:42:33.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:42:33 smithi100 bash[74942]: cluster 2024-04-07T17:42:32.609836+0000 mgr.y (mgr.25175) 2615 : cluster [DBG] pgmap v4480: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 609 KiB/s rd, 3.5 KiB/s wr, 819 op/s 2024-04-07T17:42:33.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:42:33 smithi100 bash[76202]: cluster 2024-04-07T17:42:32.342096+0000 mon.a (mon.0) 4250 : cluster [DBG] osdmap e2290: 8 total, 8 up, 8 in 2024-04-07T17:42:33.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:42:33 smithi100 bash[76202]: cluster 2024-04-07T17:42:32.609836+0000 mgr.y (mgr.25175) 2615 : cluster [DBG] pgmap v4480: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 609 KiB/s rd, 3.5 KiB/s wr, 819 op/s 2024-04-07T17:42:34.703 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:42:34 smithi100 bash[74942]: cluster 2024-04-07T17:42:33.365449+0000 mon.a (mon.0) 4251 : cluster [DBG] osdmap e2291: 8 total, 8 up, 8 in 2024-04-07T17:42:34.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:42:34 smithi100 bash[76202]: cluster 2024-04-07T17:42:33.365449+0000 mon.a (mon.0) 4251 : cluster [DBG] osdmap e2291: 8 total, 8 up, 8 in 2024-04-07T17:42:34.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:42:34 smithi144 bash[66654]: cluster 2024-04-07T17:42:33.365449+0000 mon.a (mon.0) 4251 : cluster [DBG] osdmap e2291: 8 total, 8 up, 8 in 2024-04-07T17:42:35.703 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:42:35 smithi100 bash[74942]: cluster 2024-04-07T17:42:34.373130+0000 mon.a (mon.0) 4252 : cluster [DBG] osdmap e2292: 8 total, 8 up, 8 in 2024-04-07T17:42:35.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:42:35 smithi100 bash[74942]: cluster 2024-04-07T17:42:34.610498+0000 mgr.y (mgr.25175) 2616 : cluster [DBG] pgmap v4483: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 640 KiB/s rd, 3.7 KiB/s wr, 863 op/s 2024-04-07T17:42:35.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:42:35 smithi100 bash[74942]: audit 2024-04-07T17:42:34.942422+0000 mon.a (mon.0) 4253 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:42:35.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:42:35 smithi100 bash[76202]: cluster 2024-04-07T17:42:34.373130+0000 mon.a (mon.0) 4252 : cluster [DBG] osdmap e2292: 8 total, 8 up, 8 in 2024-04-07T17:42:35.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:42:35 smithi100 bash[76202]: cluster 2024-04-07T17:42:34.610498+0000 mgr.y (mgr.25175) 2616 : cluster [DBG] pgmap v4483: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 640 KiB/s rd, 3.7 KiB/s wr, 863 op/s 2024-04-07T17:42:35.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:42:35 smithi100 bash[76202]: audit 2024-04-07T17:42:34.942422+0000 mon.a (mon.0) 4253 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:42:35.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:42:35 smithi144 bash[66654]: cluster 2024-04-07T17:42:34.373130+0000 mon.a (mon.0) 4252 : cluster [DBG] osdmap e2292: 8 total, 8 up, 8 in 2024-04-07T17:42:35.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:42:35 smithi144 bash[66654]: cluster 2024-04-07T17:42:34.610498+0000 mgr.y (mgr.25175) 2616 : cluster [DBG] pgmap v4483: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 640 KiB/s rd, 3.7 KiB/s wr, 863 op/s 2024-04-07T17:42:35.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:42:35 smithi144 bash[66654]: audit 2024-04-07T17:42:34.942422+0000 mon.a (mon.0) 4253 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:42:36.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:42:36 smithi100 bash[74942]: cluster 2024-04-07T17:42:35.395688+0000 mon.a (mon.0) 4254 : cluster [DBG] osdmap e2293: 8 total, 8 up, 8 in 2024-04-07T17:42:36.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:42:36 smithi100 bash[74942]: audit 2024-04-07T17:42:35.674757+0000 mon.a (mon.0) 4255 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:42:36.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:42:36 smithi100 bash[74942]: audit 2024-04-07T17:42:35.676323+0000 mon.a (mon.0) 4256 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:42:36.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:42:36 smithi100 bash[74942]: audit 2024-04-07T17:42:35.685502+0000 mon.a (mon.0) 4257 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:42:36.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:42:36 smithi100 bash[76202]: cluster 2024-04-07T17:42:35.395688+0000 mon.a (mon.0) 4254 : cluster [DBG] osdmap e2293: 8 total, 8 up, 8 in 2024-04-07T17:42:36.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:42:36 smithi100 bash[76202]: audit 2024-04-07T17:42:35.674757+0000 mon.a (mon.0) 4255 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:42:36.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:42:36 smithi100 bash[76202]: audit 2024-04-07T17:42:35.676323+0000 mon.a (mon.0) 4256 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:42:36.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:42:36 smithi100 bash[76202]: audit 2024-04-07T17:42:35.685502+0000 mon.a (mon.0) 4257 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:42:36.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:42:36 smithi144 bash[66654]: cluster 2024-04-07T17:42:35.395688+0000 mon.a (mon.0) 4254 : cluster [DBG] osdmap e2293: 8 total, 8 up, 8 in 2024-04-07T17:42:36.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:42:36 smithi144 bash[66654]: audit 2024-04-07T17:42:35.674757+0000 mon.a (mon.0) 4255 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:42:36.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:42:36 smithi144 bash[66654]: audit 2024-04-07T17:42:35.676323+0000 mon.a (mon.0) 4256 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:42:36.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:42:36 smithi144 bash[66654]: audit 2024-04-07T17:42:35.685502+0000 mon.a (mon.0) 4257 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:42:37.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:42:37 smithi100 bash[74942]: cluster 2024-04-07T17:42:36.397563+0000 mon.a (mon.0) 4258 : cluster [DBG] osdmap e2294: 8 total, 8 up, 8 in 2024-04-07T17:42:37.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:42:37 smithi100 bash[74942]: cluster 2024-04-07T17:42:36.611302+0000 mgr.y (mgr.25175) 2617 : cluster [DBG] pgmap v4486: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 635 KiB/s rd, 3.0 KiB/s wr, 856 op/s 2024-04-07T17:42:37.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:42:37 smithi100 bash[76202]: cluster 2024-04-07T17:42:36.397563+0000 mon.a (mon.0) 4258 : cluster [DBG] osdmap e2294: 8 total, 8 up, 8 in 2024-04-07T17:42:37.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:42:37 smithi100 bash[76202]: cluster 2024-04-07T17:42:36.611302+0000 mgr.y (mgr.25175) 2617 : cluster [DBG] pgmap v4486: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 635 KiB/s rd, 3.0 KiB/s wr, 856 op/s 2024-04-07T17:42:37.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:42:37 smithi144 bash[66654]: cluster 2024-04-07T17:42:36.397563+0000 mon.a (mon.0) 4258 : cluster [DBG] osdmap e2294: 8 total, 8 up, 8 in 2024-04-07T17:42:37.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:42:37 smithi144 bash[66654]: cluster 2024-04-07T17:42:36.611302+0000 mgr.y (mgr.25175) 2617 : cluster [DBG] pgmap v4486: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 635 KiB/s rd, 3.0 KiB/s wr, 856 op/s 2024-04-07T17:42:38.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:42:38 smithi100 bash[74942]: cluster 2024-04-07T17:42:37.419346+0000 mon.a (mon.0) 4259 : cluster [DBG] osdmap e2295: 8 total, 8 up, 8 in 2024-04-07T17:42:38.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:42:38 smithi100 bash[76202]: cluster 2024-04-07T17:42:37.419346+0000 mon.a (mon.0) 4259 : cluster [DBG] osdmap e2295: 8 total, 8 up, 8 in 2024-04-07T17:42:38.704 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:42:38 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:42:38] "GET /metrics HTTP/1.1" 200 36747 "" "Prometheus/2.43.0" 2024-04-07T17:42:38.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:42:38 smithi144 bash[66654]: cluster 2024-04-07T17:42:37.419346+0000 mon.a (mon.0) 4259 : cluster [DBG] osdmap e2295: 8 total, 8 up, 8 in 2024-04-07T17:42:39.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:42:39 smithi100 bash[74942]: cluster 2024-04-07T17:42:38.430363+0000 mon.a (mon.0) 4260 : cluster [DBG] osdmap e2296: 8 total, 8 up, 8 in 2024-04-07T17:42:39.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:42:39 smithi100 bash[74942]: cluster 2024-04-07T17:42:38.612029+0000 mgr.y (mgr.25175) 2618 : cluster [DBG] pgmap v4489: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 5.0 KiB/s rd, 767 B/s wr, 6 op/s 2024-04-07T17:42:39.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:42:39 smithi100 bash[76202]: cluster 2024-04-07T17:42:38.430363+0000 mon.a (mon.0) 4260 : cluster [DBG] osdmap e2296: 8 total, 8 up, 8 in 2024-04-07T17:42:39.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:42:39 smithi100 bash[76202]: cluster 2024-04-07T17:42:38.612029+0000 mgr.y (mgr.25175) 2618 : cluster [DBG] pgmap v4489: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 5.0 KiB/s rd, 767 B/s wr, 6 op/s 2024-04-07T17:42:39.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:42:39 smithi144 bash[66654]: cluster 2024-04-07T17:42:38.430363+0000 mon.a (mon.0) 4260 : cluster [DBG] osdmap e2296: 8 total, 8 up, 8 in 2024-04-07T17:42:39.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:42:39 smithi144 bash[66654]: cluster 2024-04-07T17:42:38.612029+0000 mgr.y (mgr.25175) 2618 : cluster [DBG] pgmap v4489: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 5.0 KiB/s rd, 767 B/s wr, 6 op/s 2024-04-07T17:42:40.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:42:40 smithi100 bash[74942]: cluster 2024-04-07T17:42:39.442320+0000 mon.a (mon.0) 4261 : cluster [DBG] osdmap e2297: 8 total, 8 up, 8 in 2024-04-07T17:42:40.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:42:40 smithi100 bash[76202]: cluster 2024-04-07T17:42:39.442320+0000 mon.a (mon.0) 4261 : cluster [DBG] osdmap e2297: 8 total, 8 up, 8 in 2024-04-07T17:42:40.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:42:40 smithi144 bash[66654]: cluster 2024-04-07T17:42:39.442320+0000 mon.a (mon.0) 4261 : cluster [DBG] osdmap e2297: 8 total, 8 up, 8 in 2024-04-07T17:42:41.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:42:41 smithi100 bash[74942]: cluster 2024-04-07T17:42:40.444060+0000 mon.a (mon.0) 4262 : cluster [DBG] osdmap e2298: 8 total, 8 up, 8 in 2024-04-07T17:42:41.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:42:41 smithi100 bash[74942]: cluster 2024-04-07T17:42:40.612747+0000 mgr.y (mgr.25175) 2619 : cluster [DBG] pgmap v4492: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 674 KiB/s rd, 3.5 KiB/s wr, 906 op/s 2024-04-07T17:42:41.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:42:41 smithi100 bash[76202]: cluster 2024-04-07T17:42:40.444060+0000 mon.a (mon.0) 4262 : cluster [DBG] osdmap e2298: 8 total, 8 up, 8 in 2024-04-07T17:42:41.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:42:41 smithi100 bash[76202]: cluster 2024-04-07T17:42:40.612747+0000 mgr.y (mgr.25175) 2619 : cluster [DBG] pgmap v4492: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 674 KiB/s rd, 3.5 KiB/s wr, 906 op/s 2024-04-07T17:42:41.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:42:41 smithi144 bash[66654]: cluster 2024-04-07T17:42:40.444060+0000 mon.a (mon.0) 4262 : cluster [DBG] osdmap e2298: 8 total, 8 up, 8 in 2024-04-07T17:42:41.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:42:41 smithi144 bash[66654]: cluster 2024-04-07T17:42:40.612747+0000 mgr.y (mgr.25175) 2619 : cluster [DBG] pgmap v4492: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 674 KiB/s rd, 3.5 KiB/s wr, 906 op/s 2024-04-07T17:42:42.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:42:42 smithi144 bash[66654]: cluster 2024-04-07T17:42:41.464647+0000 mon.a (mon.0) 4263 : cluster [DBG] osdmap e2299: 8 total, 8 up, 8 in 2024-04-07T17:42:42.615 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:42:42 smithi144 bash[95233]: ts=2024-04-07T17:42:42.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:42:42.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:42:42 smithi100 bash[74942]: cluster 2024-04-07T17:42:41.464647+0000 mon.a (mon.0) 4263 : cluster [DBG] osdmap e2299: 8 total, 8 up, 8 in 2024-04-07T17:42:42.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:42:42 smithi100 bash[76202]: cluster 2024-04-07T17:42:41.464647+0000 mon.a (mon.0) 4263 : cluster [DBG] osdmap e2299: 8 total, 8 up, 8 in 2024-04-07T17:42:43.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:42:43 smithi144 bash[66654]: cluster 2024-04-07T17:42:42.468006+0000 mon.a (mon.0) 4264 : cluster [DBG] osdmap e2300: 8 total, 8 up, 8 in 2024-04-07T17:42:43.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:42:43 smithi144 bash[66654]: cluster 2024-04-07T17:42:42.613539+0000 mgr.y (mgr.25175) 2620 : cluster [DBG] pgmap v4495: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 674 KiB/s rd, 3.7 KiB/s wr, 907 op/s 2024-04-07T17:42:43.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:42:43 smithi100 bash[74942]: cluster 2024-04-07T17:42:42.468006+0000 mon.a (mon.0) 4264 : cluster [DBG] osdmap e2300: 8 total, 8 up, 8 in 2024-04-07T17:42:43.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:42:43 smithi100 bash[74942]: cluster 2024-04-07T17:42:42.613539+0000 mgr.y (mgr.25175) 2620 : cluster [DBG] pgmap v4495: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 674 KiB/s rd, 3.7 KiB/s wr, 907 op/s 2024-04-07T17:42:43.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:42:43 smithi100 bash[76202]: cluster 2024-04-07T17:42:42.468006+0000 mon.a (mon.0) 4264 : cluster [DBG] osdmap e2300: 8 total, 8 up, 8 in 2024-04-07T17:42:43.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:42:43 smithi100 bash[76202]: cluster 2024-04-07T17:42:42.613539+0000 mgr.y (mgr.25175) 2620 : cluster [DBG] pgmap v4495: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 674 KiB/s rd, 3.7 KiB/s wr, 907 op/s 2024-04-07T17:42:44.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:42:44 smithi144 bash[66654]: cluster 2024-04-07T17:42:43.496127+0000 mon.a (mon.0) 4265 : cluster [DBG] osdmap e2301: 8 total, 8 up, 8 in 2024-04-07T17:42:44.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:42:44 smithi100 bash[74942]: cluster 2024-04-07T17:42:43.496127+0000 mon.a (mon.0) 4265 : cluster [DBG] osdmap e2301: 8 total, 8 up, 8 in 2024-04-07T17:42:44.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:42:44 smithi100 bash[76202]: cluster 2024-04-07T17:42:43.496127+0000 mon.a (mon.0) 4265 : cluster [DBG] osdmap e2301: 8 total, 8 up, 8 in 2024-04-07T17:42:45.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:42:45 smithi144 bash[66654]: cluster 2024-04-07T17:42:44.494701+0000 mon.a (mon.0) 4266 : cluster [DBG] osdmap e2302: 8 total, 8 up, 8 in 2024-04-07T17:42:45.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:42:45 smithi144 bash[66654]: cluster 2024-04-07T17:42:44.614279+0000 mgr.y (mgr.25175) 2621 : cluster [DBG] pgmap v4498: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 706 KiB/s rd, 3.7 KiB/s wr, 950 op/s 2024-04-07T17:42:45.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:42:45 smithi100 bash[74942]: cluster 2024-04-07T17:42:44.494701+0000 mon.a (mon.0) 4266 : cluster [DBG] osdmap e2302: 8 total, 8 up, 8 in 2024-04-07T17:42:45.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:42:45 smithi100 bash[74942]: cluster 2024-04-07T17:42:44.614279+0000 mgr.y (mgr.25175) 2621 : cluster [DBG] pgmap v4498: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 706 KiB/s rd, 3.7 KiB/s wr, 950 op/s 2024-04-07T17:42:45.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:42:45 smithi100 bash[76202]: cluster 2024-04-07T17:42:44.494701+0000 mon.a (mon.0) 4266 : cluster [DBG] osdmap e2302: 8 total, 8 up, 8 in 2024-04-07T17:42:45.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:42:45 smithi100 bash[76202]: cluster 2024-04-07T17:42:44.614279+0000 mgr.y (mgr.25175) 2621 : cluster [DBG] pgmap v4498: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 706 KiB/s rd, 3.7 KiB/s wr, 950 op/s 2024-04-07T17:42:46.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:42:46 smithi144 bash[66654]: cluster 2024-04-07T17:42:45.524317+0000 mon.a (mon.0) 4267 : cluster [DBG] osdmap e2303: 8 total, 8 up, 8 in 2024-04-07T17:42:46.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:42:46 smithi100 bash[74942]: cluster 2024-04-07T17:42:45.524317+0000 mon.a (mon.0) 4267 : cluster [DBG] osdmap e2303: 8 total, 8 up, 8 in 2024-04-07T17:42:46.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:42:46 smithi100 bash[76202]: cluster 2024-04-07T17:42:45.524317+0000 mon.a (mon.0) 4267 : cluster [DBG] osdmap e2303: 8 total, 8 up, 8 in 2024-04-07T17:42:47.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:42:47 smithi144 bash[66654]: cluster 2024-04-07T17:42:46.531476+0000 mon.a (mon.0) 4268 : cluster [DBG] osdmap e2304: 8 total, 8 up, 8 in 2024-04-07T17:42:47.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:42:47 smithi144 bash[66654]: cluster 2024-04-07T17:42:46.615012+0000 mgr.y (mgr.25175) 2622 : cluster [DBG] pgmap v4501: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 701 KiB/s rd, 3.0 KiB/s wr, 943 op/s 2024-04-07T17:42:47.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:42:47 smithi144 bash[66654]: audit 2024-04-07T17:42:46.867391+0000 mon.a (mon.0) 4269 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:42:47.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:42:47 smithi100 bash[74942]: cluster 2024-04-07T17:42:46.531476+0000 mon.a (mon.0) 4268 : cluster [DBG] osdmap e2304: 8 total, 8 up, 8 in 2024-04-07T17:42:47.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:42:47 smithi100 bash[74942]: cluster 2024-04-07T17:42:46.615012+0000 mgr.y (mgr.25175) 2622 : cluster [DBG] pgmap v4501: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 701 KiB/s rd, 3.0 KiB/s wr, 943 op/s 2024-04-07T17:42:47.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:42:47 smithi100 bash[74942]: audit 2024-04-07T17:42:46.867391+0000 mon.a (mon.0) 4269 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:42:47.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:42:47 smithi100 bash[76202]: cluster 2024-04-07T17:42:46.531476+0000 mon.a (mon.0) 4268 : cluster [DBG] osdmap e2304: 8 total, 8 up, 8 in 2024-04-07T17:42:47.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:42:47 smithi100 bash[76202]: cluster 2024-04-07T17:42:46.615012+0000 mgr.y (mgr.25175) 2622 : cluster [DBG] pgmap v4501: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 701 KiB/s rd, 3.0 KiB/s wr, 943 op/s 2024-04-07T17:42:47.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:42:47 smithi100 bash[76202]: audit 2024-04-07T17:42:46.867391+0000 mon.a (mon.0) 4269 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:42:48.703 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:42:48 smithi100 bash[74942]: cluster 2024-04-07T17:42:47.541703+0000 mon.a (mon.0) 4270 : cluster [DBG] osdmap e2305: 8 total, 8 up, 8 in 2024-04-07T17:42:48.704 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:42:48 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:42:48] "GET /metrics HTTP/1.1" 200 36749 "" "Prometheus/2.43.0" 2024-04-07T17:42:48.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:42:48 smithi100 bash[76202]: cluster 2024-04-07T17:42:47.541703+0000 mon.a (mon.0) 4270 : cluster [DBG] osdmap e2305: 8 total, 8 up, 8 in 2024-04-07T17:42:48.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:42:48 smithi144 bash[66654]: cluster 2024-04-07T17:42:47.541703+0000 mon.a (mon.0) 4270 : cluster [DBG] osdmap e2305: 8 total, 8 up, 8 in 2024-04-07T17:42:49.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:42:49 smithi144 bash[66654]: cluster 2024-04-07T17:42:48.542286+0000 mon.a (mon.0) 4271 : cluster [DBG] osdmap e2306: 8 total, 8 up, 8 in 2024-04-07T17:42:49.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:42:49 smithi144 bash[66654]: cluster 2024-04-07T17:42:48.615774+0000 mgr.y (mgr.25175) 2623 : cluster [DBG] pgmap v4504: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 5.0 KiB/s rd, 1023 B/s wr, 7 op/s 2024-04-07T17:42:49.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:42:49 smithi100 bash[74942]: cluster 2024-04-07T17:42:48.542286+0000 mon.a (mon.0) 4271 : cluster [DBG] osdmap e2306: 8 total, 8 up, 8 in 2024-04-07T17:42:49.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:42:49 smithi100 bash[74942]: cluster 2024-04-07T17:42:48.615774+0000 mgr.y (mgr.25175) 2623 : cluster [DBG] pgmap v4504: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 5.0 KiB/s rd, 1023 B/s wr, 7 op/s 2024-04-07T17:42:49.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:42:49 smithi100 bash[76202]: cluster 2024-04-07T17:42:48.542286+0000 mon.a (mon.0) 4271 : cluster [DBG] osdmap e2306: 8 total, 8 up, 8 in 2024-04-07T17:42:49.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:42:49 smithi100 bash[76202]: cluster 2024-04-07T17:42:48.615774+0000 mgr.y (mgr.25175) 2623 : cluster [DBG] pgmap v4504: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 5.0 KiB/s rd, 1023 B/s wr, 7 op/s 2024-04-07T17:42:50.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:42:50 smithi144 bash[66654]: cluster 2024-04-07T17:42:49.562406+0000 mon.a (mon.0) 4272 : cluster [DBG] osdmap e2307: 8 total, 8 up, 8 in 2024-04-07T17:42:50.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:42:50 smithi100 bash[74942]: cluster 2024-04-07T17:42:49.562406+0000 mon.a (mon.0) 4272 : cluster [DBG] osdmap e2307: 8 total, 8 up, 8 in 2024-04-07T17:42:50.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:42:50 smithi100 bash[76202]: cluster 2024-04-07T17:42:49.562406+0000 mon.a (mon.0) 4272 : cluster [DBG] osdmap e2307: 8 total, 8 up, 8 in 2024-04-07T17:42:51.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:42:51 smithi144 bash[66654]: cluster 2024-04-07T17:42:50.568095+0000 mon.a (mon.0) 4273 : cluster [DBG] osdmap e2308: 8 total, 8 up, 8 in 2024-04-07T17:42:51.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:42:51 smithi144 bash[66654]: cluster 2024-04-07T17:42:50.616460+0000 mgr.y (mgr.25175) 2624 : cluster [DBG] pgmap v4507: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 739 KiB/s rd, 4.0 KiB/s wr, 994 op/s 2024-04-07T17:42:51.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:42:51 smithi100 bash[74942]: cluster 2024-04-07T17:42:50.568095+0000 mon.a (mon.0) 4273 : cluster [DBG] osdmap e2308: 8 total, 8 up, 8 in 2024-04-07T17:42:51.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:42:51 smithi100 bash[74942]: cluster 2024-04-07T17:42:50.616460+0000 mgr.y (mgr.25175) 2624 : cluster [DBG] pgmap v4507: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 739 KiB/s rd, 4.0 KiB/s wr, 994 op/s 2024-04-07T17:42:51.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:42:51 smithi100 bash[76202]: cluster 2024-04-07T17:42:50.568095+0000 mon.a (mon.0) 4273 : cluster [DBG] osdmap e2308: 8 total, 8 up, 8 in 2024-04-07T17:42:51.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:42:51 smithi100 bash[76202]: cluster 2024-04-07T17:42:50.616460+0000 mgr.y (mgr.25175) 2624 : cluster [DBG] pgmap v4507: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 739 KiB/s rd, 4.0 KiB/s wr, 994 op/s 2024-04-07T17:42:52.599 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:42:52 smithi144 bash[95233]: ts=2024-04-07T17:42:52.317Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:42:52.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:42:52 smithi144 bash[66654]: cluster 2024-04-07T17:42:51.591369+0000 mon.a (mon.0) 4274 : cluster [DBG] osdmap e2309: 8 total, 8 up, 8 in 2024-04-07T17:42:52.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:42:52 smithi100 bash[74942]: cluster 2024-04-07T17:42:51.591369+0000 mon.a (mon.0) 4274 : cluster [DBG] osdmap e2309: 8 total, 8 up, 8 in 2024-04-07T17:42:52.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:42:52 smithi100 bash[76202]: cluster 2024-04-07T17:42:51.591369+0000 mon.a (mon.0) 4274 : cluster [DBG] osdmap e2309: 8 total, 8 up, 8 in 2024-04-07T17:42:53.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:42:53 smithi144 bash[66654]: cluster 2024-04-07T17:42:52.592618+0000 mon.a (mon.0) 4275 : cluster [DBG] osdmap e2310: 8 total, 8 up, 8 in 2024-04-07T17:42:53.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:42:53 smithi144 bash[66654]: cluster 2024-04-07T17:42:52.617234+0000 mgr.y (mgr.25175) 2625 : cluster [DBG] pgmap v4510: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 739 KiB/s rd, 3.5 KiB/s wr, 994 op/s 2024-04-07T17:42:53.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:42:53 smithi100 bash[74942]: cluster 2024-04-07T17:42:52.592618+0000 mon.a (mon.0) 4275 : cluster [DBG] osdmap e2310: 8 total, 8 up, 8 in 2024-04-07T17:42:53.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:42:53 smithi100 bash[74942]: cluster 2024-04-07T17:42:52.617234+0000 mgr.y (mgr.25175) 2625 : cluster [DBG] pgmap v4510: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 739 KiB/s rd, 3.5 KiB/s wr, 994 op/s 2024-04-07T17:42:53.955 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:42:53 smithi100 bash[76202]: cluster 2024-04-07T17:42:52.592618+0000 mon.a (mon.0) 4275 : cluster [DBG] osdmap e2310: 8 total, 8 up, 8 in 2024-04-07T17:42:53.955 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:42:53 smithi100 bash[76202]: cluster 2024-04-07T17:42:52.617234+0000 mgr.y (mgr.25175) 2625 : cluster [DBG] pgmap v4510: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 739 KiB/s rd, 3.5 KiB/s wr, 994 op/s 2024-04-07T17:42:54.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:42:54 smithi100 bash[74942]: cluster 2024-04-07T17:42:53.616313+0000 mon.a (mon.0) 4276 : cluster [DBG] osdmap e2311: 8 total, 8 up, 8 in 2024-04-07T17:42:54.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:42:54 smithi100 bash[76202]: cluster 2024-04-07T17:42:53.616313+0000 mon.a (mon.0) 4276 : cluster [DBG] osdmap e2311: 8 total, 8 up, 8 in 2024-04-07T17:42:55.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:42:54 smithi144 bash[66654]: cluster 2024-04-07T17:42:53.616313+0000 mon.a (mon.0) 4276 : cluster [DBG] osdmap e2311: 8 total, 8 up, 8 in 2024-04-07T17:42:55.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:42:55 smithi100 bash[74942]: cluster 2024-04-07T17:42:54.616981+0000 mon.a (mon.0) 4277 : cluster [DBG] osdmap e2312: 8 total, 8 up, 8 in 2024-04-07T17:42:55.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:42:55 smithi100 bash[74942]: cluster 2024-04-07T17:42:54.618349+0000 mgr.y (mgr.25175) 2626 : cluster [DBG] pgmap v4513: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 771 KiB/s rd, 3.7 KiB/s wr, 1.04k op/s 2024-04-07T17:42:55.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:42:55 smithi100 bash[76202]: cluster 2024-04-07T17:42:54.616981+0000 mon.a (mon.0) 4277 : cluster [DBG] osdmap e2312: 8 total, 8 up, 8 in 2024-04-07T17:42:55.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:42:55 smithi100 bash[76202]: cluster 2024-04-07T17:42:54.618349+0000 mgr.y (mgr.25175) 2626 : cluster [DBG] pgmap v4513: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 771 KiB/s rd, 3.7 KiB/s wr, 1.04k op/s 2024-04-07T17:42:56.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:42:55 smithi144 bash[66654]: cluster 2024-04-07T17:42:54.616981+0000 mon.a (mon.0) 4277 : cluster [DBG] osdmap e2312: 8 total, 8 up, 8 in 2024-04-07T17:42:56.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:42:55 smithi144 bash[66654]: cluster 2024-04-07T17:42:54.618349+0000 mgr.y (mgr.25175) 2626 : cluster [DBG] pgmap v4513: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 771 KiB/s rd, 3.7 KiB/s wr, 1.04k op/s 2024-04-07T17:42:56.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:42:56 smithi100 bash[74942]: cluster 2024-04-07T17:42:55.637225+0000 mon.a (mon.0) 4278 : cluster [DBG] osdmap e2313: 8 total, 8 up, 8 in 2024-04-07T17:42:56.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:42:56 smithi100 bash[76202]: cluster 2024-04-07T17:42:55.637225+0000 mon.a (mon.0) 4278 : cluster [DBG] osdmap e2313: 8 total, 8 up, 8 in 2024-04-07T17:42:57.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:42:56 smithi144 bash[66654]: cluster 2024-04-07T17:42:55.637225+0000 mon.a (mon.0) 4278 : cluster [DBG] osdmap e2313: 8 total, 8 up, 8 in 2024-04-07T17:42:57.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:42:57 smithi100 bash[74942]: cluster 2024-04-07T17:42:56.619094+0000 mgr.y (mgr.25175) 2627 : cluster [DBG] pgmap v4515: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 760 KiB/s rd, 3.5 KiB/s wr, 1.02k op/s 2024-04-07T17:42:57.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:42:57 smithi100 bash[74942]: cluster 2024-04-07T17:42:56.660898+0000 mon.a (mon.0) 4279 : cluster [DBG] osdmap e2314: 8 total, 8 up, 8 in 2024-04-07T17:42:57.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:42:57 smithi100 bash[76202]: cluster 2024-04-07T17:42:56.619094+0000 mgr.y (mgr.25175) 2627 : cluster [DBG] pgmap v4515: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 760 KiB/s rd, 3.5 KiB/s wr, 1.02k op/s 2024-04-07T17:42:57.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:42:57 smithi100 bash[76202]: cluster 2024-04-07T17:42:56.660898+0000 mon.a (mon.0) 4279 : cluster [DBG] osdmap e2314: 8 total, 8 up, 8 in 2024-04-07T17:42:58.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:42:57 smithi144 bash[66654]: cluster 2024-04-07T17:42:56.619094+0000 mgr.y (mgr.25175) 2627 : cluster [DBG] pgmap v4515: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 760 KiB/s rd, 3.5 KiB/s wr, 1.02k op/s 2024-04-07T17:42:58.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:42:57 smithi144 bash[66654]: cluster 2024-04-07T17:42:56.660898+0000 mon.a (mon.0) 4279 : cluster [DBG] osdmap e2314: 8 total, 8 up, 8 in 2024-04-07T17:42:58.678 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:42:58 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:42:58] "GET /metrics HTTP/1.1" 200 36749 "" "Prometheus/2.43.0" 2024-04-07T17:42:58.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:42:58 smithi100 bash[74942]: cluster 2024-04-07T17:42:57.669170+0000 mon.a (mon.0) 4280 : cluster [DBG] osdmap e2315: 8 total, 8 up, 8 in 2024-04-07T17:42:58.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:42:58 smithi100 bash[76202]: cluster 2024-04-07T17:42:57.669170+0000 mon.a (mon.0) 4280 : cluster [DBG] osdmap e2315: 8 total, 8 up, 8 in 2024-04-07T17:42:59.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:42:58 smithi144 bash[66654]: cluster 2024-04-07T17:42:57.669170+0000 mon.a (mon.0) 4280 : cluster [DBG] osdmap e2315: 8 total, 8 up, 8 in 2024-04-07T17:42:59.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:42:59 smithi100 bash[74942]: cluster 2024-04-07T17:42:58.619990+0000 mgr.y (mgr.25175) 2628 : cluster [DBG] pgmap v4518: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 5.0 KiB/s rd, 766 B/s wr, 6 op/s 2024-04-07T17:42:59.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:42:59 smithi100 bash[74942]: cluster 2024-04-07T17:42:58.672483+0000 mon.a (mon.0) 4281 : cluster [DBG] osdmap e2316: 8 total, 8 up, 8 in 2024-04-07T17:42:59.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:42:59 smithi100 bash[76202]: cluster 2024-04-07T17:42:58.619990+0000 mgr.y (mgr.25175) 2628 : cluster [DBG] pgmap v4518: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 5.0 KiB/s rd, 766 B/s wr, 6 op/s 2024-04-07T17:42:59.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:42:59 smithi100 bash[76202]: cluster 2024-04-07T17:42:58.672483+0000 mon.a (mon.0) 4281 : cluster [DBG] osdmap e2316: 8 total, 8 up, 8 in 2024-04-07T17:43:00.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:42:59 smithi144 bash[66654]: cluster 2024-04-07T17:42:58.619990+0000 mgr.y (mgr.25175) 2628 : cluster [DBG] pgmap v4518: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 5.0 KiB/s rd, 766 B/s wr, 6 op/s 2024-04-07T17:43:00.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:42:59 smithi144 bash[66654]: cluster 2024-04-07T17:42:58.672483+0000 mon.a (mon.0) 4281 : cluster [DBG] osdmap e2316: 8 total, 8 up, 8 in 2024-04-07T17:43:01.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:43:00 smithi144 bash[66654]: cluster 2024-04-07T17:42:59.686908+0000 mon.a (mon.0) 4282 : cluster [DBG] osdmap e2317: 8 total, 8 up, 8 in 2024-04-07T17:43:01.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:43:00 smithi100 bash[74942]: cluster 2024-04-07T17:42:59.686908+0000 mon.a (mon.0) 4282 : cluster [DBG] osdmap e2317: 8 total, 8 up, 8 in 2024-04-07T17:43:01.205 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:43:00 smithi100 bash[76202]: cluster 2024-04-07T17:42:59.686908+0000 mon.a (mon.0) 4282 : cluster [DBG] osdmap e2317: 8 total, 8 up, 8 in 2024-04-07T17:43:02.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:43:01 smithi144 bash[66654]: cluster 2024-04-07T17:43:00.620867+0000 mgr.y (mgr.25175) 2629 : cluster [DBG] pgmap v4521: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 803 KiB/s rd, 3.7 KiB/s wr, 1.08k op/s 2024-04-07T17:43:02.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:43:01 smithi144 bash[66654]: cluster 2024-04-07T17:43:00.713684+0000 mon.a (mon.0) 4283 : cluster [DBG] osdmap e2318: 8 total, 8 up, 8 in 2024-04-07T17:43:02.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:43:01 smithi100 bash[74942]: cluster 2024-04-07T17:43:00.620867+0000 mgr.y (mgr.25175) 2629 : cluster [DBG] pgmap v4521: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 803 KiB/s rd, 3.7 KiB/s wr, 1.08k op/s 2024-04-07T17:43:02.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:43:01 smithi100 bash[74942]: cluster 2024-04-07T17:43:00.713684+0000 mon.a (mon.0) 4283 : cluster [DBG] osdmap e2318: 8 total, 8 up, 8 in 2024-04-07T17:43:02.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:43:01 smithi100 bash[76202]: cluster 2024-04-07T17:43:00.620867+0000 mgr.y (mgr.25175) 2629 : cluster [DBG] pgmap v4521: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 803 KiB/s rd, 3.7 KiB/s wr, 1.08k op/s 2024-04-07T17:43:02.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:43:01 smithi100 bash[76202]: cluster 2024-04-07T17:43:00.713684+0000 mon.a (mon.0) 4283 : cluster [DBG] osdmap e2318: 8 total, 8 up, 8 in 2024-04-07T17:43:02.614 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:43:02 smithi144 bash[95233]: ts=2024-04-07T17:43:02.317Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:43:03.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:43:02 smithi144 bash[66654]: cluster 2024-04-07T17:43:01.725814+0000 mon.a (mon.0) 4284 : cluster [DBG] osdmap e2319: 8 total, 8 up, 8 in 2024-04-07T17:43:03.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:43:02 smithi144 bash[66654]: audit 2024-04-07T17:43:01.868034+0000 mon.a (mon.0) 4285 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:43:03.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:43:02 smithi100 bash[74942]: cluster 2024-04-07T17:43:01.725814+0000 mon.a (mon.0) 4284 : cluster [DBG] osdmap e2319: 8 total, 8 up, 8 in 2024-04-07T17:43:03.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:43:02 smithi100 bash[74942]: audit 2024-04-07T17:43:01.868034+0000 mon.a (mon.0) 4285 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:43:03.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:43:02 smithi100 bash[76202]: cluster 2024-04-07T17:43:01.725814+0000 mon.a (mon.0) 4284 : cluster [DBG] osdmap e2319: 8 total, 8 up, 8 in 2024-04-07T17:43:03.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:43:02 smithi100 bash[76202]: audit 2024-04-07T17:43:01.868034+0000 mon.a (mon.0) 4285 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:43:04.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:43:03 smithi144 bash[66654]: cluster 2024-04-07T17:43:02.622100+0000 mgr.y (mgr.25175) 2630 : cluster [DBG] pgmap v4524: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 803 KiB/s rd, 3.5 KiB/s wr, 1.08k op/s 2024-04-07T17:43:04.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:43:03 smithi144 bash[66654]: cluster 2024-04-07T17:43:02.734710+0000 mon.a (mon.0) 4286 : cluster [DBG] osdmap e2320: 8 total, 8 up, 8 in 2024-04-07T17:43:04.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:43:03 smithi100 bash[74942]: cluster 2024-04-07T17:43:02.622100+0000 mgr.y (mgr.25175) 2630 : cluster [DBG] pgmap v4524: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 803 KiB/s rd, 3.5 KiB/s wr, 1.08k op/s 2024-04-07T17:43:04.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:43:03 smithi100 bash[74942]: cluster 2024-04-07T17:43:02.734710+0000 mon.a (mon.0) 4286 : cluster [DBG] osdmap e2320: 8 total, 8 up, 8 in 2024-04-07T17:43:04.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:43:03 smithi100 bash[76202]: cluster 2024-04-07T17:43:02.622100+0000 mgr.y (mgr.25175) 2630 : cluster [DBG] pgmap v4524: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 803 KiB/s rd, 3.5 KiB/s wr, 1.08k op/s 2024-04-07T17:43:04.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:43:03 smithi100 bash[76202]: cluster 2024-04-07T17:43:02.734710+0000 mon.a (mon.0) 4286 : cluster [DBG] osdmap e2320: 8 total, 8 up, 8 in 2024-04-07T17:43:05.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:43:04 smithi144 bash[66654]: cluster 2024-04-07T17:43:03.737874+0000 mon.a (mon.0) 4287 : cluster [DBG] osdmap e2321: 8 total, 8 up, 8 in 2024-04-07T17:43:05.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:43:04 smithi100 bash[74942]: cluster 2024-04-07T17:43:03.737874+0000 mon.a (mon.0) 4287 : cluster [DBG] osdmap e2321: 8 total, 8 up, 8 in 2024-04-07T17:43:05.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:43:04 smithi100 bash[76202]: cluster 2024-04-07T17:43:03.737874+0000 mon.a (mon.0) 4287 : cluster [DBG] osdmap e2321: 8 total, 8 up, 8 in 2024-04-07T17:43:06.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:43:05 smithi144 bash[66654]: cluster 2024-04-07T17:43:04.623112+0000 mgr.y (mgr.25175) 2631 : cluster [DBG] pgmap v4527: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 836 KiB/s rd, 3.2 KiB/s wr, 1.12k op/s 2024-04-07T17:43:06.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:43:05 smithi144 bash[66654]: cluster 2024-04-07T17:43:04.752754+0000 mon.a (mon.0) 4288 : cluster [DBG] osdmap e2322: 8 total, 8 up, 8 in 2024-04-07T17:43:06.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:43:05 smithi100 bash[74942]: cluster 2024-04-07T17:43:04.623112+0000 mgr.y (mgr.25175) 2631 : cluster [DBG] pgmap v4527: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 836 KiB/s rd, 3.2 KiB/s wr, 1.12k op/s 2024-04-07T17:43:06.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:43:05 smithi100 bash[74942]: cluster 2024-04-07T17:43:04.752754+0000 mon.a (mon.0) 4288 : cluster [DBG] osdmap e2322: 8 total, 8 up, 8 in 2024-04-07T17:43:06.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:43:05 smithi100 bash[76202]: cluster 2024-04-07T17:43:04.623112+0000 mgr.y (mgr.25175) 2631 : cluster [DBG] pgmap v4527: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 836 KiB/s rd, 3.2 KiB/s wr, 1.12k op/s 2024-04-07T17:43:06.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:43:05 smithi100 bash[76202]: cluster 2024-04-07T17:43:04.752754+0000 mon.a (mon.0) 4288 : cluster [DBG] osdmap e2322: 8 total, 8 up, 8 in 2024-04-07T17:43:07.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:43:06 smithi144 bash[66654]: cluster 2024-04-07T17:43:05.774797+0000 mon.a (mon.0) 4289 : cluster [DBG] osdmap e2323: 8 total, 8 up, 8 in 2024-04-07T17:43:07.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:43:06 smithi100 bash[74942]: cluster 2024-04-07T17:43:05.774797+0000 mon.a (mon.0) 4289 : cluster [DBG] osdmap e2323: 8 total, 8 up, 8 in 2024-04-07T17:43:07.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:43:06 smithi100 bash[76202]: cluster 2024-04-07T17:43:05.774797+0000 mon.a (mon.0) 4289 : cluster [DBG] osdmap e2323: 8 total, 8 up, 8 in 2024-04-07T17:43:08.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:43:07 smithi144 bash[66654]: cluster 2024-04-07T17:43:06.623877+0000 mgr.y (mgr.25175) 2632 : cluster [DBG] pgmap v4530: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 831 KiB/s rd, 3.5 KiB/s wr, 1.12k op/s 2024-04-07T17:43:08.117 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:43:07 smithi144 bash[66654]: cluster 2024-04-07T17:43:06.798868+0000 mon.a (mon.0) 4290 : cluster [DBG] osdmap e2324: 8 total, 8 up, 8 in 2024-04-07T17:43:08.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:43:07 smithi100 bash[74942]: cluster 2024-04-07T17:43:06.623877+0000 mgr.y (mgr.25175) 2632 : cluster [DBG] pgmap v4530: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 831 KiB/s rd, 3.5 KiB/s wr, 1.12k op/s 2024-04-07T17:43:08.205 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:43:07 smithi100 bash[74942]: cluster 2024-04-07T17:43:06.798868+0000 mon.a (mon.0) 4290 : cluster [DBG] osdmap e2324: 8 total, 8 up, 8 in 2024-04-07T17:43:08.205 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:43:07 smithi100 bash[76202]: cluster 2024-04-07T17:43:06.623877+0000 mgr.y (mgr.25175) 2632 : cluster [DBG] pgmap v4530: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 831 KiB/s rd, 3.5 KiB/s wr, 1.12k op/s 2024-04-07T17:43:08.205 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:43:07 smithi100 bash[76202]: cluster 2024-04-07T17:43:06.798868+0000 mon.a (mon.0) 4290 : cluster [DBG] osdmap e2324: 8 total, 8 up, 8 in 2024-04-07T17:43:08.703 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:43:08 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:43:08] "GET /metrics HTTP/1.1" 200 36751 "" "Prometheus/2.43.0" 2024-04-07T17:43:09.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:43:08 smithi144 bash[66654]: cluster 2024-04-07T17:43:07.802312+0000 mon.a (mon.0) 4291 : cluster [DBG] osdmap e2325: 8 total, 8 up, 8 in 2024-04-07T17:43:09.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:43:08 smithi100 bash[74942]: cluster 2024-04-07T17:43:07.802312+0000 mon.a (mon.0) 4291 : cluster [DBG] osdmap e2325: 8 total, 8 up, 8 in 2024-04-07T17:43:09.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:43:08 smithi100 bash[76202]: cluster 2024-04-07T17:43:07.802312+0000 mon.a (mon.0) 4291 : cluster [DBG] osdmap e2325: 8 total, 8 up, 8 in 2024-04-07T17:43:10.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:43:09 smithi144 bash[66654]: cluster 2024-04-07T17:43:08.624869+0000 mgr.y (mgr.25175) 2633 : cluster [DBG] pgmap v4533: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 5.0 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-04-07T17:43:10.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:43:09 smithi144 bash[66654]: cluster 2024-04-07T17:43:08.826520+0000 mon.a (mon.0) 4292 : cluster [DBG] osdmap e2326: 8 total, 8 up, 8 in 2024-04-07T17:43:10.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:43:09 smithi100 bash[74942]: cluster 2024-04-07T17:43:08.624869+0000 mgr.y (mgr.25175) 2633 : cluster [DBG] pgmap v4533: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 5.0 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-04-07T17:43:10.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:43:09 smithi100 bash[74942]: cluster 2024-04-07T17:43:08.826520+0000 mon.a (mon.0) 4292 : cluster [DBG] osdmap e2326: 8 total, 8 up, 8 in 2024-04-07T17:43:10.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:43:09 smithi100 bash[76202]: cluster 2024-04-07T17:43:08.624869+0000 mgr.y (mgr.25175) 2633 : cluster [DBG] pgmap v4533: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 5.0 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-04-07T17:43:10.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:43:09 smithi100 bash[76202]: cluster 2024-04-07T17:43:08.826520+0000 mon.a (mon.0) 4292 : cluster [DBG] osdmap e2326: 8 total, 8 up, 8 in 2024-04-07T17:43:11.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:43:10 smithi144 bash[66654]: cluster 2024-04-07T17:43:09.833614+0000 mon.a (mon.0) 4293 : cluster [DBG] osdmap e2327: 8 total, 8 up, 8 in 2024-04-07T17:43:11.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:43:10 smithi100 bash[74942]: cluster 2024-04-07T17:43:09.833614+0000 mon.a (mon.0) 4293 : cluster [DBG] osdmap e2327: 8 total, 8 up, 8 in 2024-04-07T17:43:11.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:43:10 smithi100 bash[76202]: cluster 2024-04-07T17:43:09.833614+0000 mon.a (mon.0) 4293 : cluster [DBG] osdmap e2327: 8 total, 8 up, 8 in 2024-04-07T17:43:12.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:43:11 smithi144 bash[66654]: cluster 2024-04-07T17:43:10.625806+0000 mgr.y (mgr.25175) 2634 : cluster [DBG] pgmap v4536: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 869 KiB/s rd, 3.2 KiB/s wr, 1.17k op/s 2024-04-07T17:43:12.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:43:11 smithi144 bash[66654]: cluster 2024-04-07T17:43:10.849900+0000 mon.a (mon.0) 4294 : cluster [DBG] osdmap e2328: 8 total, 8 up, 8 in 2024-04-07T17:43:12.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:43:11 smithi100 bash[74942]: cluster 2024-04-07T17:43:10.625806+0000 mgr.y (mgr.25175) 2634 : cluster [DBG] pgmap v4536: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 869 KiB/s rd, 3.2 KiB/s wr, 1.17k op/s 2024-04-07T17:43:12.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:43:11 smithi100 bash[74942]: cluster 2024-04-07T17:43:10.849900+0000 mon.a (mon.0) 4294 : cluster [DBG] osdmap e2328: 8 total, 8 up, 8 in 2024-04-07T17:43:12.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:43:11 smithi100 bash[76202]: cluster 2024-04-07T17:43:10.625806+0000 mgr.y (mgr.25175) 2634 : cluster [DBG] pgmap v4536: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 869 KiB/s rd, 3.2 KiB/s wr, 1.17k op/s 2024-04-07T17:43:12.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:43:11 smithi100 bash[76202]: cluster 2024-04-07T17:43:10.849900+0000 mon.a (mon.0) 4294 : cluster [DBG] osdmap e2328: 8 total, 8 up, 8 in 2024-04-07T17:43:12.615 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:43:12 smithi144 bash[95233]: ts=2024-04-07T17:43:12.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:43:13.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:43:12 smithi144 bash[66654]: cluster 2024-04-07T17:43:11.847754+0000 mon.a (mon.0) 4295 : cluster [DBG] osdmap e2329: 8 total, 8 up, 8 in 2024-04-07T17:43:13.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:43:12 smithi100 bash[74942]: cluster 2024-04-07T17:43:11.847754+0000 mon.a (mon.0) 4295 : cluster [DBG] osdmap e2329: 8 total, 8 up, 8 in 2024-04-07T17:43:13.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:43:12 smithi100 bash[76202]: cluster 2024-04-07T17:43:11.847754+0000 mon.a (mon.0) 4295 : cluster [DBG] osdmap e2329: 8 total, 8 up, 8 in 2024-04-07T17:43:14.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:43:13 smithi100 bash[74942]: cluster 2024-04-07T17:43:12.627013+0000 mgr.y (mgr.25175) 2635 : cluster [DBG] pgmap v4539: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 869 KiB/s rd, 3.2 KiB/s wr, 1.17k op/s 2024-04-07T17:43:14.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:43:13 smithi100 bash[74942]: cluster 2024-04-07T17:43:12.870491+0000 mon.a (mon.0) 4296 : cluster [DBG] osdmap e2330: 8 total, 8 up, 8 in 2024-04-07T17:43:14.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:43:13 smithi100 bash[76202]: cluster 2024-04-07T17:43:12.627013+0000 mgr.y (mgr.25175) 2635 : cluster [DBG] pgmap v4539: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 869 KiB/s rd, 3.2 KiB/s wr, 1.17k op/s 2024-04-07T17:43:14.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:43:13 smithi100 bash[76202]: cluster 2024-04-07T17:43:12.870491+0000 mon.a (mon.0) 4296 : cluster [DBG] osdmap e2330: 8 total, 8 up, 8 in 2024-04-07T17:43:14.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:43:13 smithi144 bash[66654]: cluster 2024-04-07T17:43:12.627013+0000 mgr.y (mgr.25175) 2635 : cluster [DBG] pgmap v4539: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 869 KiB/s rd, 3.2 KiB/s wr, 1.17k op/s 2024-04-07T17:43:14.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:43:13 smithi144 bash[66654]: cluster 2024-04-07T17:43:12.870491+0000 mon.a (mon.0) 4296 : cluster [DBG] osdmap e2330: 8 total, 8 up, 8 in 2024-04-07T17:43:15.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:43:14 smithi100 bash[74942]: cluster 2024-04-07T17:43:13.872720+0000 mon.a (mon.0) 4297 : cluster [DBG] osdmap e2331: 8 total, 8 up, 8 in 2024-04-07T17:43:15.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:43:14 smithi100 bash[76202]: cluster 2024-04-07T17:43:13.872720+0000 mon.a (mon.0) 4297 : cluster [DBG] osdmap e2331: 8 total, 8 up, 8 in 2024-04-07T17:43:15.364 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:43:14 smithi144 bash[66654]: cluster 2024-04-07T17:43:13.872720+0000 mon.a (mon.0) 4297 : cluster [DBG] osdmap e2331: 8 total, 8 up, 8 in 2024-04-07T17:43:16.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:43:15 smithi100 bash[74942]: cluster 2024-04-07T17:43:14.628220+0000 mgr.y (mgr.25175) 2636 : cluster [DBG] pgmap v4542: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 902 KiB/s rd, 4.0 KiB/s wr, 1.22k op/s 2024-04-07T17:43:16.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:43:15 smithi100 bash[74942]: cluster 2024-04-07T17:43:14.893470+0000 mon.a (mon.0) 4298 : cluster [DBG] osdmap e2332: 8 total, 8 up, 8 in 2024-04-07T17:43:16.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:43:15 smithi100 bash[76202]: cluster 2024-04-07T17:43:14.628220+0000 mgr.y (mgr.25175) 2636 : cluster [DBG] pgmap v4542: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 902 KiB/s rd, 4.0 KiB/s wr, 1.22k op/s 2024-04-07T17:43:16.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:43:15 smithi100 bash[76202]: cluster 2024-04-07T17:43:14.893470+0000 mon.a (mon.0) 4298 : cluster [DBG] osdmap e2332: 8 total, 8 up, 8 in 2024-04-07T17:43:16.364 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:43:15 smithi144 bash[66654]: cluster 2024-04-07T17:43:14.628220+0000 mgr.y (mgr.25175) 2636 : cluster [DBG] pgmap v4542: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 902 KiB/s rd, 4.0 KiB/s wr, 1.22k op/s 2024-04-07T17:43:16.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:43:15 smithi144 bash[66654]: cluster 2024-04-07T17:43:14.893470+0000 mon.a (mon.0) 4298 : cluster [DBG] osdmap e2332: 8 total, 8 up, 8 in 2024-04-07T17:43:17.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:43:16 smithi100 bash[74942]: cluster 2024-04-07T17:43:15.895801+0000 mon.a (mon.0) 4299 : cluster [DBG] osdmap e2333: 8 total, 8 up, 8 in 2024-04-07T17:43:17.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:43:16 smithi100 bash[74942]: audit 2024-04-07T17:43:16.868749+0000 mon.a (mon.0) 4300 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:43:17.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:43:16 smithi100 bash[76202]: cluster 2024-04-07T17:43:15.895801+0000 mon.a (mon.0) 4299 : cluster [DBG] osdmap e2333: 8 total, 8 up, 8 in 2024-04-07T17:43:17.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:43:16 smithi100 bash[76202]: audit 2024-04-07T17:43:16.868749+0000 mon.a (mon.0) 4300 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:43:17.364 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:43:16 smithi144 bash[66654]: cluster 2024-04-07T17:43:15.895801+0000 mon.a (mon.0) 4299 : cluster [DBG] osdmap e2333: 8 total, 8 up, 8 in 2024-04-07T17:43:17.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:43:16 smithi144 bash[66654]: audit 2024-04-07T17:43:16.868749+0000 mon.a (mon.0) 4300 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:43:18.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:43:17 smithi100 bash[74942]: cluster 2024-04-07T17:43:16.628993+0000 mgr.y (mgr.25175) 2637 : cluster [DBG] pgmap v4545: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 897 KiB/s rd, 3.5 KiB/s wr, 1.21k op/s 2024-04-07T17:43:18.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:43:17 smithi100 bash[74942]: cluster 2024-04-07T17:43:16.920256+0000 mon.a (mon.0) 4301 : cluster [DBG] osdmap e2334: 8 total, 8 up, 8 in 2024-04-07T17:43:18.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:43:17 smithi100 bash[76202]: cluster 2024-04-07T17:43:16.628993+0000 mgr.y (mgr.25175) 2637 : cluster [DBG] pgmap v4545: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 897 KiB/s rd, 3.5 KiB/s wr, 1.21k op/s 2024-04-07T17:43:18.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:43:17 smithi100 bash[76202]: cluster 2024-04-07T17:43:16.920256+0000 mon.a (mon.0) 4301 : cluster [DBG] osdmap e2334: 8 total, 8 up, 8 in 2024-04-07T17:43:18.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:43:17 smithi144 bash[66654]: cluster 2024-04-07T17:43:16.628993+0000 mgr.y (mgr.25175) 2637 : cluster [DBG] pgmap v4545: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 897 KiB/s rd, 3.5 KiB/s wr, 1.21k op/s 2024-04-07T17:43:18.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:43:17 smithi144 bash[66654]: cluster 2024-04-07T17:43:16.920256+0000 mon.a (mon.0) 4301 : cluster [DBG] osdmap e2334: 8 total, 8 up, 8 in 2024-04-07T17:43:18.703 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:43:18 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:43:18] "GET /metrics HTTP/1.1" 200 36747 "" "Prometheus/2.43.0" 2024-04-07T17:43:19.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:43:18 smithi100 bash[74942]: cluster 2024-04-07T17:43:17.928705+0000 mon.a (mon.0) 4302 : cluster [DBG] osdmap e2335: 8 total, 8 up, 8 in 2024-04-07T17:43:19.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:43:18 smithi100 bash[76202]: cluster 2024-04-07T17:43:17.928705+0000 mon.a (mon.0) 4302 : cluster [DBG] osdmap e2335: 8 total, 8 up, 8 in 2024-04-07T17:43:19.364 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:43:18 smithi144 bash[66654]: cluster 2024-04-07T17:43:17.928705+0000 mon.a (mon.0) 4302 : cluster [DBG] osdmap e2335: 8 total, 8 up, 8 in 2024-04-07T17:43:20.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:43:19 smithi100 bash[74942]: cluster 2024-04-07T17:43:18.629885+0000 mgr.y (mgr.25175) 2638 : cluster [DBG] pgmap v4548: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 4.7 KiB/s rd, 511 B/s wr, 6 op/s 2024-04-07T17:43:20.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:43:19 smithi100 bash[74942]: cluster 2024-04-07T17:43:18.940298+0000 mon.a (mon.0) 4303 : cluster [DBG] osdmap e2336: 8 total, 8 up, 8 in 2024-04-07T17:43:20.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:43:19 smithi100 bash[76202]: cluster 2024-04-07T17:43:18.629885+0000 mgr.y (mgr.25175) 2638 : cluster [DBG] pgmap v4548: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 4.7 KiB/s rd, 511 B/s wr, 6 op/s 2024-04-07T17:43:20.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:43:19 smithi100 bash[76202]: cluster 2024-04-07T17:43:18.940298+0000 mon.a (mon.0) 4303 : cluster [DBG] osdmap e2336: 8 total, 8 up, 8 in 2024-04-07T17:43:20.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:43:19 smithi144 bash[66654]: cluster 2024-04-07T17:43:18.629885+0000 mgr.y (mgr.25175) 2638 : cluster [DBG] pgmap v4548: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 4.7 KiB/s rd, 511 B/s wr, 6 op/s 2024-04-07T17:43:20.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:43:19 smithi144 bash[66654]: cluster 2024-04-07T17:43:18.940298+0000 mon.a (mon.0) 4303 : cluster [DBG] osdmap e2336: 8 total, 8 up, 8 in 2024-04-07T17:43:21.364 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:43:20 smithi144 bash[66654]: cluster 2024-04-07T17:43:19.945428+0000 mon.a (mon.0) 4304 : cluster [DBG] osdmap e2337: 8 total, 8 up, 8 in 2024-04-07T17:43:21.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:43:20 smithi100 bash[74942]: cluster 2024-04-07T17:43:19.945428+0000 mon.a (mon.0) 4304 : cluster [DBG] osdmap e2337: 8 total, 8 up, 8 in 2024-04-07T17:43:21.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:43:20 smithi100 bash[76202]: cluster 2024-04-07T17:43:19.945428+0000 mon.a (mon.0) 4304 : cluster [DBG] osdmap e2337: 8 total, 8 up, 8 in 2024-04-07T17:43:22.316 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:43:21 smithi144 bash[66654]: cluster 2024-04-07T17:43:20.630731+0000 mgr.y (mgr.25175) 2639 : cluster [DBG] pgmap v4551: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 934 KiB/s rd, 3.7 KiB/s wr, 1.26k op/s 2024-04-07T17:43:22.316 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:43:21 smithi144 bash[66654]: cluster 2024-04-07T17:43:20.973569+0000 mon.a (mon.0) 4305 : cluster [DBG] osdmap e2338: 8 total, 8 up, 8 in 2024-04-07T17:43:22.453 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:43:21 smithi100 bash[74942]: cluster 2024-04-07T17:43:20.630731+0000 mgr.y (mgr.25175) 2639 : cluster [DBG] pgmap v4551: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 934 KiB/s rd, 3.7 KiB/s wr, 1.26k op/s 2024-04-07T17:43:22.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:43:21 smithi100 bash[74942]: cluster 2024-04-07T17:43:20.973569+0000 mon.a (mon.0) 4305 : cluster [DBG] osdmap e2338: 8 total, 8 up, 8 in 2024-04-07T17:43:22.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:43:21 smithi100 bash[76202]: cluster 2024-04-07T17:43:20.630731+0000 mgr.y (mgr.25175) 2639 : cluster [DBG] pgmap v4551: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 934 KiB/s rd, 3.7 KiB/s wr, 1.26k op/s 2024-04-07T17:43:22.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:43:21 smithi100 bash[76202]: cluster 2024-04-07T17:43:20.973569+0000 mon.a (mon.0) 4305 : cluster [DBG] osdmap e2338: 8 total, 8 up, 8 in 2024-04-07T17:43:22.614 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:43:22 smithi144 bash[95233]: ts=2024-04-07T17:43:22.317Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:43:23.364 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:43:22 smithi144 bash[66654]: cluster 2024-04-07T17:43:21.975537+0000 mon.a (mon.0) 4306 : cluster [DBG] osdmap e2339: 8 total, 8 up, 8 in 2024-04-07T17:43:23.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:43:22 smithi100 bash[74942]: cluster 2024-04-07T17:43:21.975537+0000 mon.a (mon.0) 4306 : cluster [DBG] osdmap e2339: 8 total, 8 up, 8 in 2024-04-07T17:43:23.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:43:22 smithi100 bash[76202]: cluster 2024-04-07T17:43:21.975537+0000 mon.a (mon.0) 4306 : cluster [DBG] osdmap e2339: 8 total, 8 up, 8 in 2024-04-07T17:43:24.364 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:43:24 smithi144 bash[66654]: cluster 2024-04-07T17:43:22.631669+0000 mgr.y (mgr.25175) 2640 : cluster [DBG] pgmap v4554: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 934 KiB/s rd, 4.0 KiB/s wr, 1.26k op/s 2024-04-07T17:43:24.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:43:24 smithi144 bash[66654]: cluster 2024-04-07T17:43:23.000518+0000 mon.a (mon.0) 4307 : cluster [DBG] osdmap e2340: 8 total, 8 up, 8 in 2024-04-07T17:43:24.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:43:24 smithi100 bash[74942]: cluster 2024-04-07T17:43:22.631669+0000 mgr.y (mgr.25175) 2640 : cluster [DBG] pgmap v4554: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 934 KiB/s rd, 4.0 KiB/s wr, 1.26k op/s 2024-04-07T17:43:24.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:43:24 smithi100 bash[74942]: cluster 2024-04-07T17:43:23.000518+0000 mon.a (mon.0) 4307 : cluster [DBG] osdmap e2340: 8 total, 8 up, 8 in 2024-04-07T17:43:24.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:43:24 smithi100 bash[76202]: cluster 2024-04-07T17:43:22.631669+0000 mgr.y (mgr.25175) 2640 : cluster [DBG] pgmap v4554: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 934 KiB/s rd, 4.0 KiB/s wr, 1.26k op/s 2024-04-07T17:43:24.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:43:24 smithi100 bash[76202]: cluster 2024-04-07T17:43:23.000518+0000 mon.a (mon.0) 4307 : cluster [DBG] osdmap e2340: 8 total, 8 up, 8 in 2024-04-07T17:43:25.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:43:25 smithi144 bash[66654]: cluster 2024-04-07T17:43:24.000935+0000 mon.a (mon.0) 4308 : cluster [DBG] osdmap e2341: 8 total, 8 up, 8 in 2024-04-07T17:43:25.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:43:25 smithi100 bash[74942]: cluster 2024-04-07T17:43:24.000935+0000 mon.a (mon.0) 4308 : cluster [DBG] osdmap e2341: 8 total, 8 up, 8 in 2024-04-07T17:43:25.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:43:25 smithi100 bash[76202]: cluster 2024-04-07T17:43:24.000935+0000 mon.a (mon.0) 4308 : cluster [DBG] osdmap e2341: 8 total, 8 up, 8 in 2024-04-07T17:43:26.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:43:26 smithi144 bash[66654]: cluster 2024-04-07T17:43:24.632667+0000 mgr.y (mgr.25175) 2641 : cluster [DBG] pgmap v4557: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 966 KiB/s rd, 3.5 KiB/s wr, 1.30k op/s 2024-04-07T17:43:26.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:43:26 smithi144 bash[66654]: cluster 2024-04-07T17:43:25.021658+0000 mon.a (mon.0) 4309 : cluster [DBG] osdmap e2342: 8 total, 8 up, 8 in 2024-04-07T17:43:26.453 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:43:26 smithi100 bash[74942]: cluster 2024-04-07T17:43:24.632667+0000 mgr.y (mgr.25175) 2641 : cluster [DBG] pgmap v4557: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 966 KiB/s rd, 3.5 KiB/s wr, 1.30k op/s 2024-04-07T17:43:26.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:43:26 smithi100 bash[74942]: cluster 2024-04-07T17:43:25.021658+0000 mon.a (mon.0) 4309 : cluster [DBG] osdmap e2342: 8 total, 8 up, 8 in 2024-04-07T17:43:26.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:43:26 smithi100 bash[76202]: cluster 2024-04-07T17:43:24.632667+0000 mgr.y (mgr.25175) 2641 : cluster [DBG] pgmap v4557: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 966 KiB/s rd, 3.5 KiB/s wr, 1.30k op/s 2024-04-07T17:43:26.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:43:26 smithi100 bash[76202]: cluster 2024-04-07T17:43:25.021658+0000 mon.a (mon.0) 4309 : cluster [DBG] osdmap e2342: 8 total, 8 up, 8 in 2024-04-07T17:43:27.364 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:43:27 smithi144 bash[66654]: cluster 2024-04-07T17:43:26.024072+0000 mon.a (mon.0) 4310 : cluster [DBG] osdmap e2343: 8 total, 8 up, 8 in 2024-04-07T17:43:27.453 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:43:27 smithi100 bash[76202]: cluster 2024-04-07T17:43:26.024072+0000 mon.a (mon.0) 4310 : cluster [DBG] osdmap e2343: 8 total, 8 up, 8 in 2024-04-07T17:43:27.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:43:27 smithi100 bash[74942]: cluster 2024-04-07T17:43:26.024072+0000 mon.a (mon.0) 4310 : cluster [DBG] osdmap e2343: 8 total, 8 up, 8 in 2024-04-07T17:43:28.364 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:43:28 smithi144 bash[66654]: cluster 2024-04-07T17:43:26.633545+0000 mgr.y (mgr.25175) 2642 : cluster [DBG] pgmap v4560: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 962 KiB/s rd, 3.0 KiB/s wr, 1.30k op/s 2024-04-07T17:43:28.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:43:28 smithi144 bash[66654]: cluster 2024-04-07T17:43:27.047600+0000 mon.a (mon.0) 4311 : cluster [DBG] osdmap e2344: 8 total, 8 up, 8 in 2024-04-07T17:43:28.426 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:43:28 smithi100 bash[74942]: cluster 2024-04-07T17:43:26.633545+0000 mgr.y (mgr.25175) 2642 : cluster [DBG] pgmap v4560: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 962 KiB/s rd, 3.0 KiB/s wr, 1.30k op/s 2024-04-07T17:43:28.426 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:43:28 smithi100 bash[74942]: cluster 2024-04-07T17:43:27.047600+0000 mon.a (mon.0) 4311 : cluster [DBG] osdmap e2344: 8 total, 8 up, 8 in 2024-04-07T17:43:28.427 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:43:28 smithi100 bash[76202]: cluster 2024-04-07T17:43:26.633545+0000 mgr.y (mgr.25175) 2642 : cluster [DBG] pgmap v4560: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 962 KiB/s rd, 3.0 KiB/s wr, 1.30k op/s 2024-04-07T17:43:28.427 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:43:28 smithi100 bash[76202]: cluster 2024-04-07T17:43:27.047600+0000 mon.a (mon.0) 4311 : cluster [DBG] osdmap e2344: 8 total, 8 up, 8 in 2024-04-07T17:43:28.703 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:43:28 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:43:28] "GET /metrics HTTP/1.1" 200 36747 "" "Prometheus/2.43.0" 2024-04-07T17:43:29.364 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:43:29 smithi144 bash[66654]: cluster 2024-04-07T17:43:28.049701+0000 mon.a (mon.0) 4312 : cluster [DBG] osdmap e2345: 8 total, 8 up, 8 in 2024-04-07T17:43:29.453 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:43:29 smithi100 bash[74942]: cluster 2024-04-07T17:43:28.049701+0000 mon.a (mon.0) 4312 : cluster [DBG] osdmap e2345: 8 total, 8 up, 8 in 2024-04-07T17:43:29.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:43:29 smithi100 bash[76202]: cluster 2024-04-07T17:43:28.049701+0000 mon.a (mon.0) 4312 : cluster [DBG] osdmap e2345: 8 total, 8 up, 8 in 2024-04-07T17:43:30.364 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:43:30 smithi144 bash[66654]: cluster 2024-04-07T17:43:28.634430+0000 mgr.y (mgr.25175) 2643 : cluster [DBG] pgmap v4563: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 5.0 KiB/s rd, 511 B/s wr, 6 op/s 2024-04-07T17:43:30.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:43:30 smithi144 bash[66654]: cluster 2024-04-07T17:43:29.070511+0000 mon.a (mon.0) 4313 : cluster [DBG] osdmap e2346: 8 total, 8 up, 8 in 2024-04-07T17:43:30.453 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:43:30 smithi100 bash[76202]: cluster 2024-04-07T17:43:28.634430+0000 mgr.y (mgr.25175) 2643 : cluster [DBG] pgmap v4563: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 5.0 KiB/s rd, 511 B/s wr, 6 op/s 2024-04-07T17:43:30.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:43:30 smithi100 bash[76202]: cluster 2024-04-07T17:43:29.070511+0000 mon.a (mon.0) 4313 : cluster [DBG] osdmap e2346: 8 total, 8 up, 8 in 2024-04-07T17:43:30.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:43:30 smithi100 bash[74942]: cluster 2024-04-07T17:43:28.634430+0000 mgr.y (mgr.25175) 2643 : cluster [DBG] pgmap v4563: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 5.0 KiB/s rd, 511 B/s wr, 6 op/s 2024-04-07T17:43:30.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:43:30 smithi100 bash[74942]: cluster 2024-04-07T17:43:29.070511+0000 mon.a (mon.0) 4313 : cluster [DBG] osdmap e2346: 8 total, 8 up, 8 in 2024-04-07T17:43:31.364 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:43:31 smithi144 bash[66654]: cluster 2024-04-07T17:43:30.082857+0000 mon.a (mon.0) 4314 : cluster [DBG] osdmap e2347: 8 total, 8 up, 8 in 2024-04-07T17:43:31.453 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:43:31 smithi100 bash[74942]: cluster 2024-04-07T17:43:30.082857+0000 mon.a (mon.0) 4314 : cluster [DBG] osdmap e2347: 8 total, 8 up, 8 in 2024-04-07T17:43:31.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:43:31 smithi100 bash[76202]: cluster 2024-04-07T17:43:30.082857+0000 mon.a (mon.0) 4314 : cluster [DBG] osdmap e2347: 8 total, 8 up, 8 in 2024-04-07T17:43:32.365 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:43:32 smithi144 bash[95233]: ts=2024-04-07T17:43:32.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:43:32.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:43:32 smithi144 bash[66654]: cluster 2024-04-07T17:43:30.635188+0000 mgr.y (mgr.25175) 2644 : cluster [DBG] pgmap v4566: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 802 KiB/s rd, 3.5 KiB/s wr, 1.08k op/s 2024-04-07T17:43:32.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:43:32 smithi144 bash[66654]: cluster 2024-04-07T17:43:31.099645+0000 mon.a (mon.0) 4315 : cluster [DBG] osdmap e2348: 8 total, 8 up, 8 in 2024-04-07T17:43:32.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:43:32 smithi144 bash[66654]: audit 2024-04-07T17:43:31.869130+0000 mon.a (mon.0) 4316 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:43:32.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:43:32 smithi100 bash[74942]: cluster 2024-04-07T17:43:30.635188+0000 mgr.y (mgr.25175) 2644 : cluster [DBG] pgmap v4566: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 802 KiB/s rd, 3.5 KiB/s wr, 1.08k op/s 2024-04-07T17:43:32.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:43:32 smithi100 bash[74942]: cluster 2024-04-07T17:43:31.099645+0000 mon.a (mon.0) 4315 : cluster [DBG] osdmap e2348: 8 total, 8 up, 8 in 2024-04-07T17:43:32.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:43:32 smithi100 bash[74942]: audit 2024-04-07T17:43:31.869130+0000 mon.a (mon.0) 4316 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:43:32.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:43:32 smithi100 bash[76202]: cluster 2024-04-07T17:43:30.635188+0000 mgr.y (mgr.25175) 2644 : cluster [DBG] pgmap v4566: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 802 KiB/s rd, 3.5 KiB/s wr, 1.08k op/s 2024-04-07T17:43:32.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:43:32 smithi100 bash[76202]: cluster 2024-04-07T17:43:31.099645+0000 mon.a (mon.0) 4315 : cluster [DBG] osdmap e2348: 8 total, 8 up, 8 in 2024-04-07T17:43:32.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:43:32 smithi100 bash[76202]: audit 2024-04-07T17:43:31.869130+0000 mon.a (mon.0) 4316 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:43:33.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:43:33 smithi144 bash[66654]: cluster 2024-04-07T17:43:32.097255+0000 mon.a (mon.0) 4317 : cluster [DBG] osdmap e2349: 8 total, 8 up, 8 in 2024-04-07T17:43:33.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:43:33 smithi100 bash[74942]: cluster 2024-04-07T17:43:32.097255+0000 mon.a (mon.0) 4317 : cluster [DBG] osdmap e2349: 8 total, 8 up, 8 in 2024-04-07T17:43:33.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:43:33 smithi100 bash[76202]: cluster 2024-04-07T17:43:32.097255+0000 mon.a (mon.0) 4317 : cluster [DBG] osdmap e2349: 8 total, 8 up, 8 in 2024-04-07T17:43:34.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:43:34 smithi100 bash[74942]: cluster 2024-04-07T17:43:32.635949+0000 mgr.y (mgr.25175) 2645 : cluster [DBG] pgmap v4569: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 802 KiB/s rd, 3.2 KiB/s wr, 1.08k op/s 2024-04-07T17:43:34.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:43:34 smithi100 bash[74942]: cluster 2024-04-07T17:43:33.121834+0000 mon.a (mon.0) 4318 : cluster [DBG] osdmap e2350: 8 total, 8 up, 8 in 2024-04-07T17:43:34.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:43:34 smithi100 bash[76202]: cluster 2024-04-07T17:43:32.635949+0000 mgr.y (mgr.25175) 2645 : cluster [DBG] pgmap v4569: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 802 KiB/s rd, 3.2 KiB/s wr, 1.08k op/s 2024-04-07T17:43:34.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:43:34 smithi100 bash[76202]: cluster 2024-04-07T17:43:33.121834+0000 mon.a (mon.0) 4318 : cluster [DBG] osdmap e2350: 8 total, 8 up, 8 in 2024-04-07T17:43:34.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:43:34 smithi144 bash[66654]: cluster 2024-04-07T17:43:32.635949+0000 mgr.y (mgr.25175) 2645 : cluster [DBG] pgmap v4569: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 802 KiB/s rd, 3.2 KiB/s wr, 1.08k op/s 2024-04-07T17:43:34.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:43:34 smithi144 bash[66654]: cluster 2024-04-07T17:43:33.121834+0000 mon.a (mon.0) 4318 : cluster [DBG] osdmap e2350: 8 total, 8 up, 8 in 2024-04-07T17:43:35.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:43:35 smithi100 bash[74942]: cluster 2024-04-07T17:43:34.120531+0000 mon.a (mon.0) 4319 : cluster [DBG] osdmap e2351: 8 total, 8 up, 8 in 2024-04-07T17:43:35.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:43:35 smithi100 bash[76202]: cluster 2024-04-07T17:43:34.120531+0000 mon.a (mon.0) 4319 : cluster [DBG] osdmap e2351: 8 total, 8 up, 8 in 2024-04-07T17:43:35.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:43:35 smithi144 bash[66654]: cluster 2024-04-07T17:43:34.120531+0000 mon.a (mon.0) 4319 : cluster [DBG] osdmap e2351: 8 total, 8 up, 8 in 2024-04-07T17:43:36.453 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:43:36 smithi100 bash[76202]: cluster 2024-04-07T17:43:34.636616+0000 mgr.y (mgr.25175) 2646 : cluster [DBG] pgmap v4572: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 1021 KiB/s rd, 3.5 KiB/s wr, 1.38k op/s 2024-04-07T17:43:36.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:43:36 smithi100 bash[76202]: cluster 2024-04-07T17:43:35.141392+0000 mon.a (mon.0) 4320 : cluster [DBG] osdmap e2352: 8 total, 8 up, 8 in 2024-04-07T17:43:36.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:43:36 smithi100 bash[76202]: audit 2024-04-07T17:43:35.777147+0000 mon.a (mon.0) 4321 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:43:36.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:43:36 smithi100 bash[74942]: cluster 2024-04-07T17:43:34.636616+0000 mgr.y (mgr.25175) 2646 : cluster [DBG] pgmap v4572: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 1021 KiB/s rd, 3.5 KiB/s wr, 1.38k op/s 2024-04-07T17:43:36.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:43:36 smithi100 bash[74942]: cluster 2024-04-07T17:43:35.141392+0000 mon.a (mon.0) 4320 : cluster [DBG] osdmap e2352: 8 total, 8 up, 8 in 2024-04-07T17:43:36.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:43:36 smithi100 bash[74942]: audit 2024-04-07T17:43:35.777147+0000 mon.a (mon.0) 4321 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:43:36.614 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:43:36 smithi144 bash[66654]: cluster 2024-04-07T17:43:34.636616+0000 mgr.y (mgr.25175) 2646 : cluster [DBG] pgmap v4572: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 1021 KiB/s rd, 3.5 KiB/s wr, 1.38k op/s 2024-04-07T17:43:36.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:43:36 smithi144 bash[66654]: cluster 2024-04-07T17:43:35.141392+0000 mon.a (mon.0) 4320 : cluster [DBG] osdmap e2352: 8 total, 8 up, 8 in 2024-04-07T17:43:36.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:43:36 smithi144 bash[66654]: audit 2024-04-07T17:43:35.777147+0000 mon.a (mon.0) 4321 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:43:37.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:43:37 smithi100 bash[74942]: cluster 2024-04-07T17:43:36.146894+0000 mon.a (mon.0) 4322 : cluster [DBG] osdmap e2353: 8 total, 8 up, 8 in 2024-04-07T17:43:37.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:43:37 smithi100 bash[74942]: audit 2024-04-07T17:43:36.508464+0000 mon.a (mon.0) 4323 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:43:37.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:43:37 smithi100 bash[74942]: audit 2024-04-07T17:43:36.509924+0000 mon.a (mon.0) 4324 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:43:37.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:43:37 smithi100 bash[74942]: audit 2024-04-07T17:43:36.519536+0000 mon.a (mon.0) 4325 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:43:37.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:43:37 smithi100 bash[76202]: cluster 2024-04-07T17:43:36.146894+0000 mon.a (mon.0) 4322 : cluster [DBG] osdmap e2353: 8 total, 8 up, 8 in 2024-04-07T17:43:37.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:43:37 smithi100 bash[76202]: audit 2024-04-07T17:43:36.508464+0000 mon.a (mon.0) 4323 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:43:37.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:43:37 smithi100 bash[76202]: audit 2024-04-07T17:43:36.509924+0000 mon.a (mon.0) 4324 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:43:37.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:43:37 smithi100 bash[76202]: audit 2024-04-07T17:43:36.519536+0000 mon.a (mon.0) 4325 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:43:37.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:43:37 smithi144 bash[66654]: cluster 2024-04-07T17:43:36.146894+0000 mon.a (mon.0) 4322 : cluster [DBG] osdmap e2353: 8 total, 8 up, 8 in 2024-04-07T17:43:37.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:43:37 smithi144 bash[66654]: audit 2024-04-07T17:43:36.508464+0000 mon.a (mon.0) 4323 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:43:37.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:43:37 smithi144 bash[66654]: audit 2024-04-07T17:43:36.509924+0000 mon.a (mon.0) 4324 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:43:37.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:43:37 smithi144 bash[66654]: audit 2024-04-07T17:43:36.519536+0000 mon.a (mon.0) 4325 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:43:38.454 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:43:38 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:43:38] "GET /metrics HTTP/1.1" 200 36746 "" "Prometheus/2.43.0" 2024-04-07T17:43:38.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:43:38 smithi100 bash[74942]: cluster 2024-04-07T17:43:36.637354+0000 mgr.y (mgr.25175) 2647 : cluster [DBG] pgmap v4575: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 1016 KiB/s rd, 3.2 KiB/s wr, 1.37k op/s 2024-04-07T17:43:38.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:43:38 smithi100 bash[74942]: cluster 2024-04-07T17:43:37.170868+0000 mon.a (mon.0) 4326 : cluster [DBG] osdmap e2354: 8 total, 8 up, 8 in 2024-04-07T17:43:38.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:43:38 smithi100 bash[76202]: cluster 2024-04-07T17:43:36.637354+0000 mgr.y (mgr.25175) 2647 : cluster [DBG] pgmap v4575: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 1016 KiB/s rd, 3.2 KiB/s wr, 1.37k op/s 2024-04-07T17:43:38.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:43:38 smithi100 bash[76202]: cluster 2024-04-07T17:43:37.170868+0000 mon.a (mon.0) 4326 : cluster [DBG] osdmap e2354: 8 total, 8 up, 8 in 2024-04-07T17:43:38.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:43:38 smithi144 bash[66654]: cluster 2024-04-07T17:43:36.637354+0000 mgr.y (mgr.25175) 2647 : cluster [DBG] pgmap v4575: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 1016 KiB/s rd, 3.2 KiB/s wr, 1.37k op/s 2024-04-07T17:43:38.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:43:38 smithi144 bash[66654]: cluster 2024-04-07T17:43:37.170868+0000 mon.a (mon.0) 4326 : cluster [DBG] osdmap e2354: 8 total, 8 up, 8 in 2024-04-07T17:43:39.453 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:43:39 smithi100 bash[74942]: cluster 2024-04-07T17:43:38.174197+0000 mon.a (mon.0) 4327 : cluster [DBG] osdmap e2355: 8 total, 8 up, 8 in 2024-04-07T17:43:39.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:43:39 smithi100 bash[76202]: cluster 2024-04-07T17:43:38.174197+0000 mon.a (mon.0) 4327 : cluster [DBG] osdmap e2355: 8 total, 8 up, 8 in 2024-04-07T17:43:39.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:43:39 smithi144 bash[66654]: cluster 2024-04-07T17:43:38.174197+0000 mon.a (mon.0) 4327 : cluster [DBG] osdmap e2355: 8 total, 8 up, 8 in 2024-04-07T17:43:40.453 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:43:40 smithi100 bash[74942]: cluster 2024-04-07T17:43:38.638089+0000 mgr.y (mgr.25175) 2648 : cluster [DBG] pgmap v4578: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 5.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-07T17:43:40.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:43:40 smithi100 bash[74942]: cluster 2024-04-07T17:43:39.197476+0000 mon.a (mon.0) 4328 : cluster [DBG] osdmap e2356: 8 total, 8 up, 8 in 2024-04-07T17:43:40.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:43:40 smithi100 bash[76202]: cluster 2024-04-07T17:43:38.638089+0000 mgr.y (mgr.25175) 2648 : cluster [DBG] pgmap v4578: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 5.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-07T17:43:40.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:43:40 smithi100 bash[76202]: cluster 2024-04-07T17:43:39.197476+0000 mon.a (mon.0) 4328 : cluster [DBG] osdmap e2356: 8 total, 8 up, 8 in 2024-04-07T17:43:40.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:43:40 smithi144 bash[66654]: cluster 2024-04-07T17:43:38.638089+0000 mgr.y (mgr.25175) 2648 : cluster [DBG] pgmap v4578: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 5.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-07T17:43:40.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:43:40 smithi144 bash[66654]: cluster 2024-04-07T17:43:39.197476+0000 mon.a (mon.0) 4328 : cluster [DBG] osdmap e2356: 8 total, 8 up, 8 in 2024-04-07T17:43:41.614 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:43:41 smithi144 bash[66654]: cluster 2024-04-07T17:43:40.195347+0000 mon.a (mon.0) 4329 : cluster [DBG] osdmap e2357: 8 total, 8 up, 8 in 2024-04-07T17:43:41.703 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:43:41 smithi100 bash[74942]: cluster 2024-04-07T17:43:40.195347+0000 mon.a (mon.0) 4329 : cluster [DBG] osdmap e2357: 8 total, 8 up, 8 in 2024-04-07T17:43:41.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:43:41 smithi100 bash[76202]: cluster 2024-04-07T17:43:40.195347+0000 mon.a (mon.0) 4329 : cluster [DBG] osdmap e2357: 8 total, 8 up, 8 in 2024-04-07T17:43:42.615 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:43:42 smithi144 bash[95233]: ts=2024-04-07T17:43:42.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:43:42.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:43:42 smithi144 bash[66654]: cluster 2024-04-07T17:43:40.638791+0000 mgr.y (mgr.25175) 2649 : cluster [DBG] pgmap v4581: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 1.0 MiB/s rd, 3.7 KiB/s wr, 1.43k op/s 2024-04-07T17:43:42.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:43:42 smithi144 bash[66654]: cluster 2024-04-07T17:43:41.220837+0000 mon.a (mon.0) 4330 : cluster [DBG] osdmap e2358: 8 total, 8 up, 8 in 2024-04-07T17:43:42.703 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:43:42 smithi100 bash[74942]: cluster 2024-04-07T17:43:40.638791+0000 mgr.y (mgr.25175) 2649 : cluster [DBG] pgmap v4581: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 1.0 MiB/s rd, 3.7 KiB/s wr, 1.43k op/s 2024-04-07T17:43:42.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:43:42 smithi100 bash[74942]: cluster 2024-04-07T17:43:41.220837+0000 mon.a (mon.0) 4330 : cluster [DBG] osdmap e2358: 8 total, 8 up, 8 in 2024-04-07T17:43:42.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:43:42 smithi100 bash[76202]: cluster 2024-04-07T17:43:40.638791+0000 mgr.y (mgr.25175) 2649 : cluster [DBG] pgmap v4581: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 1.0 MiB/s rd, 3.7 KiB/s wr, 1.43k op/s 2024-04-07T17:43:42.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:43:42 smithi100 bash[76202]: cluster 2024-04-07T17:43:41.220837+0000 mon.a (mon.0) 4330 : cluster [DBG] osdmap e2358: 8 total, 8 up, 8 in 2024-04-07T17:43:43.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:43:43 smithi144 bash[66654]: cluster 2024-04-07T17:43:42.220785+0000 mon.a (mon.0) 4331 : cluster [DBG] osdmap e2359: 8 total, 8 up, 8 in 2024-04-07T17:43:43.703 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:43:43 smithi100 bash[74942]: cluster 2024-04-07T17:43:42.220785+0000 mon.a (mon.0) 4331 : cluster [DBG] osdmap e2359: 8 total, 8 up, 8 in 2024-04-07T17:43:43.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:43:43 smithi100 bash[76202]: cluster 2024-04-07T17:43:42.220785+0000 mon.a (mon.0) 4331 : cluster [DBG] osdmap e2359: 8 total, 8 up, 8 in 2024-04-07T17:43:44.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:43:44 smithi144 bash[66654]: cluster 2024-04-07T17:43:42.639509+0000 mgr.y (mgr.25175) 2650 : cluster [DBG] pgmap v4584: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 1.0 MiB/s rd, 4.2 KiB/s wr, 1.43k op/s 2024-04-07T17:43:44.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:43:44 smithi144 bash[66654]: cluster 2024-04-07T17:43:43.245125+0000 mon.a (mon.0) 4332 : cluster [DBG] osdmap e2360: 8 total, 8 up, 8 in 2024-04-07T17:43:44.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:43:44 smithi100 bash[74942]: cluster 2024-04-07T17:43:42.639509+0000 mgr.y (mgr.25175) 2650 : cluster [DBG] pgmap v4584: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 1.0 MiB/s rd, 4.2 KiB/s wr, 1.43k op/s 2024-04-07T17:43:44.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:43:44 smithi100 bash[74942]: cluster 2024-04-07T17:43:43.245125+0000 mon.a (mon.0) 4332 : cluster [DBG] osdmap e2360: 8 total, 8 up, 8 in 2024-04-07T17:43:44.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:43:44 smithi100 bash[76202]: cluster 2024-04-07T17:43:42.639509+0000 mgr.y (mgr.25175) 2650 : cluster [DBG] pgmap v4584: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 1.0 MiB/s rd, 4.2 KiB/s wr, 1.43k op/s 2024-04-07T17:43:44.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:43:44 smithi100 bash[76202]: cluster 2024-04-07T17:43:43.245125+0000 mon.a (mon.0) 4332 : cluster [DBG] osdmap e2360: 8 total, 8 up, 8 in 2024-04-07T17:43:45.614 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:43:45 smithi144 bash[66654]: cluster 2024-04-07T17:43:44.247214+0000 mon.a (mon.0) 4333 : cluster [DBG] osdmap e2361: 8 total, 8 up, 8 in 2024-04-07T17:43:45.703 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:43:45 smithi100 bash[74942]: cluster 2024-04-07T17:43:44.247214+0000 mon.a (mon.0) 4333 : cluster [DBG] osdmap e2361: 8 total, 8 up, 8 in 2024-04-07T17:43:45.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:43:45 smithi100 bash[76202]: cluster 2024-04-07T17:43:44.247214+0000 mon.a (mon.0) 4333 : cluster [DBG] osdmap e2361: 8 total, 8 up, 8 in 2024-04-07T17:43:46.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:43:46 smithi144 bash[66654]: cluster 2024-04-07T17:43:44.640306+0000 mgr.y (mgr.25175) 2651 : cluster [DBG] pgmap v4587: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 1.1 MiB/s rd, 3.5 KiB/s wr, 1.47k op/s 2024-04-07T17:43:46.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:43:46 smithi144 bash[66654]: cluster 2024-04-07T17:43:45.270219+0000 mon.a (mon.0) 4334 : cluster [DBG] osdmap e2362: 8 total, 8 up, 8 in 2024-04-07T17:43:46.703 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:43:46 smithi100 bash[74942]: cluster 2024-04-07T17:43:44.640306+0000 mgr.y (mgr.25175) 2651 : cluster [DBG] pgmap v4587: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 1.1 MiB/s rd, 3.5 KiB/s wr, 1.47k op/s 2024-04-07T17:43:46.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:43:46 smithi100 bash[74942]: cluster 2024-04-07T17:43:45.270219+0000 mon.a (mon.0) 4334 : cluster [DBG] osdmap e2362: 8 total, 8 up, 8 in 2024-04-07T17:43:46.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:43:46 smithi100 bash[76202]: cluster 2024-04-07T17:43:44.640306+0000 mgr.y (mgr.25175) 2651 : cluster [DBG] pgmap v4587: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 1.1 MiB/s rd, 3.5 KiB/s wr, 1.47k op/s 2024-04-07T17:43:46.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:43:46 smithi100 bash[76202]: cluster 2024-04-07T17:43:45.270219+0000 mon.a (mon.0) 4334 : cluster [DBG] osdmap e2362: 8 total, 8 up, 8 in 2024-04-07T17:43:47.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:43:47 smithi144 bash[66654]: cluster 2024-04-07T17:43:46.279868+0000 mon.a (mon.0) 4335 : cluster [DBG] osdmap e2363: 8 total, 8 up, 8 in 2024-04-07T17:43:47.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:43:47 smithi144 bash[66654]: cluster 2024-04-07T17:43:46.641182+0000 mgr.y (mgr.25175) 2652 : cluster [DBG] pgmap v4590: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 1.1 MiB/s rd, 2.7 KiB/s wr, 1.46k op/s 2024-04-07T17:43:47.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:43:47 smithi144 bash[66654]: audit 2024-04-07T17:43:46.869627+0000 mon.a (mon.0) 4336 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:43:47.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:43:47 smithi100 bash[74942]: cluster 2024-04-07T17:43:46.279868+0000 mon.a (mon.0) 4335 : cluster [DBG] osdmap e2363: 8 total, 8 up, 8 in 2024-04-07T17:43:47.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:43:47 smithi100 bash[74942]: cluster 2024-04-07T17:43:46.641182+0000 mgr.y (mgr.25175) 2652 : cluster [DBG] pgmap v4590: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 1.1 MiB/s rd, 2.7 KiB/s wr, 1.46k op/s 2024-04-07T17:43:47.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:43:47 smithi100 bash[74942]: audit 2024-04-07T17:43:46.869627+0000 mon.a (mon.0) 4336 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:43:47.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:43:47 smithi100 bash[76202]: cluster 2024-04-07T17:43:46.279868+0000 mon.a (mon.0) 4335 : cluster [DBG] osdmap e2363: 8 total, 8 up, 8 in 2024-04-07T17:43:47.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:43:47 smithi100 bash[76202]: cluster 2024-04-07T17:43:46.641182+0000 mgr.y (mgr.25175) 2652 : cluster [DBG] pgmap v4590: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 1.1 MiB/s rd, 2.7 KiB/s wr, 1.46k op/s 2024-04-07T17:43:47.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:43:47 smithi100 bash[76202]: audit 2024-04-07T17:43:46.869627+0000 mon.a (mon.0) 4336 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:43:48.614 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:43:48 smithi144 bash[66654]: cluster 2024-04-07T17:43:47.293171+0000 mon.a (mon.0) 4337 : cluster [DBG] osdmap e2364: 8 total, 8 up, 8 in 2024-04-07T17:43:48.703 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:43:48 smithi100 bash[74942]: cluster 2024-04-07T17:43:47.293171+0000 mon.a (mon.0) 4337 : cluster [DBG] osdmap e2364: 8 total, 8 up, 8 in 2024-04-07T17:43:48.704 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:43:48 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:43:48] "GET /metrics HTTP/1.1" 200 36747 "" "Prometheus/2.43.0" 2024-04-07T17:43:48.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:43:48 smithi100 bash[76202]: cluster 2024-04-07T17:43:47.293171+0000 mon.a (mon.0) 4337 : cluster [DBG] osdmap e2364: 8 total, 8 up, 8 in 2024-04-07T17:43:49.614 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:43:49 smithi144 bash[66654]: cluster 2024-04-07T17:43:48.292133+0000 mon.a (mon.0) 4338 : cluster [DBG] osdmap e2365: 8 total, 8 up, 8 in 2024-04-07T17:43:49.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:43:49 smithi144 bash[66654]: cluster 2024-04-07T17:43:48.641951+0000 mgr.y (mgr.25175) 2653 : cluster [DBG] pgmap v4593: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 5.0 KiB/s rd, 1023 B/s wr, 7 op/s 2024-04-07T17:43:49.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:43:49 smithi100 bash[74942]: cluster 2024-04-07T17:43:48.292133+0000 mon.a (mon.0) 4338 : cluster [DBG] osdmap e2365: 8 total, 8 up, 8 in 2024-04-07T17:43:49.705 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:43:49 smithi100 bash[74942]: cluster 2024-04-07T17:43:48.641951+0000 mgr.y (mgr.25175) 2653 : cluster [DBG] pgmap v4593: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 5.0 KiB/s rd, 1023 B/s wr, 7 op/s 2024-04-07T17:43:49.705 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:43:49 smithi100 bash[76202]: cluster 2024-04-07T17:43:48.292133+0000 mon.a (mon.0) 4338 : cluster [DBG] osdmap e2365: 8 total, 8 up, 8 in 2024-04-07T17:43:49.705 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:43:49 smithi100 bash[76202]: cluster 2024-04-07T17:43:48.641951+0000 mgr.y (mgr.25175) 2653 : cluster [DBG] pgmap v4593: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 5.0 KiB/s rd, 1023 B/s wr, 7 op/s 2024-04-07T17:43:50.614 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:43:50 smithi144 bash[66654]: cluster 2024-04-07T17:43:49.312934+0000 mon.a (mon.0) 4339 : cluster [DBG] osdmap e2366: 8 total, 8 up, 8 in 2024-04-07T17:43:50.703 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:43:50 smithi100 bash[74942]: cluster 2024-04-07T17:43:49.312934+0000 mon.a (mon.0) 4339 : cluster [DBG] osdmap e2366: 8 total, 8 up, 8 in 2024-04-07T17:43:50.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:43:50 smithi100 bash[76202]: cluster 2024-04-07T17:43:49.312934+0000 mon.a (mon.0) 4339 : cluster [DBG] osdmap e2366: 8 total, 8 up, 8 in 2024-04-07T17:43:51.614 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:43:51 smithi144 bash[66654]: cluster 2024-04-07T17:43:50.314855+0000 mon.a (mon.0) 4340 : cluster [DBG] osdmap e2367: 8 total, 8 up, 8 in 2024-04-07T17:43:51.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:43:51 smithi144 bash[66654]: cluster 2024-04-07T17:43:50.642667+0000 mgr.y (mgr.25175) 2654 : cluster [DBG] pgmap v4596: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 1.1 MiB/s rd, 3.7 KiB/s wr, 1.51k op/s 2024-04-07T17:43:51.703 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:43:51 smithi100 bash[74942]: cluster 2024-04-07T17:43:50.314855+0000 mon.a (mon.0) 4340 : cluster [DBG] osdmap e2367: 8 total, 8 up, 8 in 2024-04-07T17:43:51.703 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:43:51 smithi100 bash[74942]: cluster 2024-04-07T17:43:50.642667+0000 mgr.y (mgr.25175) 2654 : cluster [DBG] pgmap v4596: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 1.1 MiB/s rd, 3.7 KiB/s wr, 1.51k op/s 2024-04-07T17:43:51.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:43:51 smithi100 bash[76202]: cluster 2024-04-07T17:43:50.314855+0000 mon.a (mon.0) 4340 : cluster [DBG] osdmap e2367: 8 total, 8 up, 8 in 2024-04-07T17:43:51.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:43:51 smithi100 bash[76202]: cluster 2024-04-07T17:43:50.642667+0000 mgr.y (mgr.25175) 2654 : cluster [DBG] pgmap v4596: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 1.1 MiB/s rd, 3.7 KiB/s wr, 1.51k op/s 2024-04-07T17:43:52.614 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:43:52 smithi144 bash[66654]: cluster 2024-04-07T17:43:51.336595+0000 mon.a (mon.0) 4341 : cluster [DBG] osdmap e2368: 8 total, 8 up, 8 in 2024-04-07T17:43:52.615 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:43:52 smithi144 bash[95233]: ts=2024-04-07T17:43:52.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:43:52.703 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:43:52 smithi100 bash[74942]: cluster 2024-04-07T17:43:51.336595+0000 mon.a (mon.0) 4341 : cluster [DBG] osdmap e2368: 8 total, 8 up, 8 in 2024-04-07T17:43:52.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:43:52 smithi100 bash[76202]: cluster 2024-04-07T17:43:51.336595+0000 mon.a (mon.0) 4341 : cluster [DBG] osdmap e2368: 8 total, 8 up, 8 in 2024-04-07T17:43:53.614 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:43:53 smithi144 bash[66654]: cluster 2024-04-07T17:43:52.347220+0000 mon.a (mon.0) 4342 : cluster [DBG] osdmap e2369: 8 total, 8 up, 8 in 2024-04-07T17:43:53.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:43:53 smithi144 bash[66654]: cluster 2024-04-07T17:43:52.643363+0000 mgr.y (mgr.25175) 2655 : cluster [DBG] pgmap v4599: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 1.1 MiB/s rd, 3.5 KiB/s wr, 1.51k op/s 2024-04-07T17:43:53.703 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:43:53 smithi100 bash[74942]: cluster 2024-04-07T17:43:52.347220+0000 mon.a (mon.0) 4342 : cluster [DBG] osdmap e2369: 8 total, 8 up, 8 in 2024-04-07T17:43:53.703 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:43:53 smithi100 bash[74942]: cluster 2024-04-07T17:43:52.643363+0000 mgr.y (mgr.25175) 2655 : cluster [DBG] pgmap v4599: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 1.1 MiB/s rd, 3.5 KiB/s wr, 1.51k op/s 2024-04-07T17:43:53.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:43:53 smithi100 bash[76202]: cluster 2024-04-07T17:43:52.347220+0000 mon.a (mon.0) 4342 : cluster [DBG] osdmap e2369: 8 total, 8 up, 8 in 2024-04-07T17:43:53.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:43:53 smithi100 bash[76202]: cluster 2024-04-07T17:43:52.643363+0000 mgr.y (mgr.25175) 2655 : cluster [DBG] pgmap v4599: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 1.1 MiB/s rd, 3.5 KiB/s wr, 1.51k op/s 2024-04-07T17:43:54.614 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:43:54 smithi144 bash[66654]: cluster 2024-04-07T17:43:53.358396+0000 mon.a (mon.0) 4343 : cluster [DBG] osdmap e2370: 8 total, 8 up, 8 in 2024-04-07T17:43:54.703 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:43:54 smithi100 bash[74942]: cluster 2024-04-07T17:43:53.358396+0000 mon.a (mon.0) 4343 : cluster [DBG] osdmap e2370: 8 total, 8 up, 8 in 2024-04-07T17:43:54.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:43:54 smithi100 bash[76202]: cluster 2024-04-07T17:43:53.358396+0000 mon.a (mon.0) 4343 : cluster [DBG] osdmap e2370: 8 total, 8 up, 8 in 2024-04-07T17:43:55.703 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:43:55 smithi100 bash[74942]: cluster 2024-04-07T17:43:54.365167+0000 mon.a (mon.0) 4344 : cluster [DBG] osdmap e2371: 8 total, 8 up, 8 in 2024-04-07T17:43:55.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:43:55 smithi100 bash[74942]: cluster 2024-04-07T17:43:54.644062+0000 mgr.y (mgr.25175) 2656 : cluster [DBG] pgmap v4602: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 1.1 MiB/s rd, 3.7 KiB/s wr, 1.56k op/s 2024-04-07T17:43:55.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:43:55 smithi100 bash[76202]: cluster 2024-04-07T17:43:54.365167+0000 mon.a (mon.0) 4344 : cluster [DBG] osdmap e2371: 8 total, 8 up, 8 in 2024-04-07T17:43:55.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:43:55 smithi100 bash[76202]: cluster 2024-04-07T17:43:54.644062+0000 mgr.y (mgr.25175) 2656 : cluster [DBG] pgmap v4602: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 1.1 MiB/s rd, 3.7 KiB/s wr, 1.56k op/s 2024-04-07T17:43:55.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:43:55 smithi144 bash[66654]: cluster 2024-04-07T17:43:54.365167+0000 mon.a (mon.0) 4344 : cluster [DBG] osdmap e2371: 8 total, 8 up, 8 in 2024-04-07T17:43:55.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:43:55 smithi144 bash[66654]: cluster 2024-04-07T17:43:54.644062+0000 mgr.y (mgr.25175) 2656 : cluster [DBG] pgmap v4602: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 1.1 MiB/s rd, 3.7 KiB/s wr, 1.56k op/s 2024-04-07T17:43:56.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:43:56 smithi100 bash[76202]: cluster 2024-04-07T17:43:55.377405+0000 mon.a (mon.0) 4345 : cluster [DBG] osdmap e2372: 8 total, 8 up, 8 in 2024-04-07T17:43:56.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:43:56 smithi100 bash[74942]: cluster 2024-04-07T17:43:55.377405+0000 mon.a (mon.0) 4345 : cluster [DBG] osdmap e2372: 8 total, 8 up, 8 in 2024-04-07T17:43:56.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:43:56 smithi144 bash[66654]: cluster 2024-04-07T17:43:55.377405+0000 mon.a (mon.0) 4345 : cluster [DBG] osdmap e2372: 8 total, 8 up, 8 in 2024-04-07T17:43:57.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:43:57 smithi100 bash[74942]: cluster 2024-04-07T17:43:56.378488+0000 mon.a (mon.0) 4346 : cluster [DBG] osdmap e2373: 8 total, 8 up, 8 in 2024-04-07T17:43:57.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:43:57 smithi100 bash[74942]: cluster 2024-04-07T17:43:56.644811+0000 mgr.y (mgr.25175) 2657 : cluster [DBG] pgmap v4605: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 1.1 MiB/s rd, 3.0 KiB/s wr, 1.55k op/s 2024-04-07T17:43:57.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:43:57 smithi100 bash[76202]: cluster 2024-04-07T17:43:56.378488+0000 mon.a (mon.0) 4346 : cluster [DBG] osdmap e2373: 8 total, 8 up, 8 in 2024-04-07T17:43:57.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:43:57 smithi100 bash[76202]: cluster 2024-04-07T17:43:56.644811+0000 mgr.y (mgr.25175) 2657 : cluster [DBG] pgmap v4605: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 1.1 MiB/s rd, 3.0 KiB/s wr, 1.55k op/s 2024-04-07T17:43:57.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:43:57 smithi144 bash[66654]: cluster 2024-04-07T17:43:56.378488+0000 mon.a (mon.0) 4346 : cluster [DBG] osdmap e2373: 8 total, 8 up, 8 in 2024-04-07T17:43:57.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:43:57 smithi144 bash[66654]: cluster 2024-04-07T17:43:56.644811+0000 mgr.y (mgr.25175) 2657 : cluster [DBG] pgmap v4605: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 1.1 MiB/s rd, 3.0 KiB/s wr, 1.55k op/s 2024-04-07T17:43:58.703 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:43:58 smithi100 bash[74942]: cluster 2024-04-07T17:43:57.404975+0000 mon.a (mon.0) 4347 : cluster [DBG] osdmap e2374: 8 total, 8 up, 8 in 2024-04-07T17:43:58.704 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:43:58 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:43:58] "GET /metrics HTTP/1.1" 200 36747 "" "Prometheus/2.43.0" 2024-04-07T17:43:58.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:43:58 smithi100 bash[76202]: cluster 2024-04-07T17:43:57.404975+0000 mon.a (mon.0) 4347 : cluster [DBG] osdmap e2374: 8 total, 8 up, 8 in 2024-04-07T17:43:58.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:43:58 smithi144 bash[66654]: cluster 2024-04-07T17:43:57.404975+0000 mon.a (mon.0) 4347 : cluster [DBG] osdmap e2374: 8 total, 8 up, 8 in 2024-04-07T17:43:59.703 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:43:59 smithi100 bash[74942]: cluster 2024-04-07T17:43:58.407006+0000 mon.a (mon.0) 4348 : cluster [DBG] osdmap e2375: 8 total, 8 up, 8 in 2024-04-07T17:43:59.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:43:59 smithi100 bash[74942]: cluster 2024-04-07T17:43:58.645584+0000 mgr.y (mgr.25175) 2658 : cluster [DBG] pgmap v4608: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 5.0 KiB/s rd, 767 B/s wr, 6 op/s 2024-04-07T17:43:59.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:43:59 smithi100 bash[76202]: cluster 2024-04-07T17:43:58.407006+0000 mon.a (mon.0) 4348 : cluster [DBG] osdmap e2375: 8 total, 8 up, 8 in 2024-04-07T17:43:59.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:43:59 smithi100 bash[76202]: cluster 2024-04-07T17:43:58.645584+0000 mgr.y (mgr.25175) 2658 : cluster [DBG] pgmap v4608: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 5.0 KiB/s rd, 767 B/s wr, 6 op/s 2024-04-07T17:43:59.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:43:59 smithi144 bash[66654]: cluster 2024-04-07T17:43:58.407006+0000 mon.a (mon.0) 4348 : cluster [DBG] osdmap e2375: 8 total, 8 up, 8 in 2024-04-07T17:43:59.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:43:59 smithi144 bash[66654]: cluster 2024-04-07T17:43:58.645584+0000 mgr.y (mgr.25175) 2658 : cluster [DBG] pgmap v4608: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 5.0 KiB/s rd, 767 B/s wr, 6 op/s 2024-04-07T17:44:00.703 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:44:00 smithi100 bash[74942]: cluster 2024-04-07T17:43:59.430636+0000 mon.a (mon.0) 4349 : cluster [DBG] osdmap e2376: 8 total, 8 up, 8 in 2024-04-07T17:44:00.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:44:00 smithi100 bash[76202]: cluster 2024-04-07T17:43:59.430636+0000 mon.a (mon.0) 4349 : cluster [DBG] osdmap e2376: 8 total, 8 up, 8 in 2024-04-07T17:44:00.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:44:00 smithi144 bash[66654]: cluster 2024-04-07T17:43:59.430636+0000 mon.a (mon.0) 4349 : cluster [DBG] osdmap e2376: 8 total, 8 up, 8 in 2024-04-07T17:44:01.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:44:01 smithi100 bash[74942]: cluster 2024-04-07T17:44:00.431861+0000 mon.a (mon.0) 4350 : cluster [DBG] osdmap e2377: 8 total, 8 up, 8 in 2024-04-07T17:44:01.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:44:01 smithi100 bash[74942]: cluster 2024-04-07T17:44:00.646371+0000 mgr.y (mgr.25175) 2659 : cluster [DBG] pgmap v4611: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 1.2 MiB/s rd, 3.5 KiB/s wr, 1.60k op/s 2024-04-07T17:44:01.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:44:01 smithi100 bash[76202]: cluster 2024-04-07T17:44:00.431861+0000 mon.a (mon.0) 4350 : cluster [DBG] osdmap e2377: 8 total, 8 up, 8 in 2024-04-07T17:44:01.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:44:01 smithi100 bash[76202]: cluster 2024-04-07T17:44:00.646371+0000 mgr.y (mgr.25175) 2659 : cluster [DBG] pgmap v4611: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 1.2 MiB/s rd, 3.5 KiB/s wr, 1.60k op/s 2024-04-07T17:44:01.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:44:01 smithi144 bash[66654]: cluster 2024-04-07T17:44:00.431861+0000 mon.a (mon.0) 4350 : cluster [DBG] osdmap e2377: 8 total, 8 up, 8 in 2024-04-07T17:44:01.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:44:01 smithi144 bash[66654]: cluster 2024-04-07T17:44:00.646371+0000 mgr.y (mgr.25175) 2659 : cluster [DBG] pgmap v4611: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 1.2 MiB/s rd, 3.5 KiB/s wr, 1.60k op/s 2024-04-07T17:44:02.615 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:44:02 smithi144 bash[95233]: ts=2024-04-07T17:44:02.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:44:02.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:44:02 smithi144 bash[66654]: cluster 2024-04-07T17:44:01.454633+0000 mon.a (mon.0) 4351 : cluster [DBG] osdmap e2378: 8 total, 8 up, 8 in 2024-04-07T17:44:02.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:44:02 smithi144 bash[66654]: audit 2024-04-07T17:44:01.870264+0000 mon.a (mon.0) 4352 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:44:02.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:44:02 smithi100 bash[74942]: cluster 2024-04-07T17:44:01.454633+0000 mon.a (mon.0) 4351 : cluster [DBG] osdmap e2378: 8 total, 8 up, 8 in 2024-04-07T17:44:02.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:44:02 smithi100 bash[74942]: audit 2024-04-07T17:44:01.870264+0000 mon.a (mon.0) 4352 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:44:02.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:44:02 smithi100 bash[76202]: cluster 2024-04-07T17:44:01.454633+0000 mon.a (mon.0) 4351 : cluster [DBG] osdmap e2378: 8 total, 8 up, 8 in 2024-04-07T17:44:02.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:44:02 smithi100 bash[76202]: audit 2024-04-07T17:44:01.870264+0000 mon.a (mon.0) 4352 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:44:03.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:44:03 smithi144 bash[66654]: cluster 2024-04-07T17:44:02.454934+0000 mon.a (mon.0) 4353 : cluster [DBG] osdmap e2379: 8 total, 8 up, 8 in 2024-04-07T17:44:03.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:44:03 smithi144 bash[66654]: cluster 2024-04-07T17:44:02.647026+0000 mgr.y (mgr.25175) 2660 : cluster [DBG] pgmap v4614: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 1.2 MiB/s rd, 3.5 KiB/s wr, 1.60k op/s 2024-04-07T17:44:03.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:44:03 smithi100 bash[74942]: cluster 2024-04-07T17:44:02.454934+0000 mon.a (mon.0) 4353 : cluster [DBG] osdmap e2379: 8 total, 8 up, 8 in 2024-04-07T17:44:03.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:44:03 smithi100 bash[74942]: cluster 2024-04-07T17:44:02.647026+0000 mgr.y (mgr.25175) 2660 : cluster [DBG] pgmap v4614: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 1.2 MiB/s rd, 3.5 KiB/s wr, 1.60k op/s 2024-04-07T17:44:03.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:44:03 smithi100 bash[76202]: cluster 2024-04-07T17:44:02.454934+0000 mon.a (mon.0) 4353 : cluster [DBG] osdmap e2379: 8 total, 8 up, 8 in 2024-04-07T17:44:03.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:44:03 smithi100 bash[76202]: cluster 2024-04-07T17:44:02.647026+0000 mgr.y (mgr.25175) 2660 : cluster [DBG] pgmap v4614: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 1.2 MiB/s rd, 3.5 KiB/s wr, 1.60k op/s 2024-04-07T17:44:04.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:44:04 smithi144 bash[66654]: cluster 2024-04-07T17:44:03.480874+0000 mon.a (mon.0) 4354 : cluster [DBG] osdmap e2380: 8 total, 8 up, 8 in 2024-04-07T17:44:04.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:44:04 smithi100 bash[74942]: cluster 2024-04-07T17:44:03.480874+0000 mon.a (mon.0) 4354 : cluster [DBG] osdmap e2380: 8 total, 8 up, 8 in 2024-04-07T17:44:04.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:44:04 smithi100 bash[76202]: cluster 2024-04-07T17:44:03.480874+0000 mon.a (mon.0) 4354 : cluster [DBG] osdmap e2380: 8 total, 8 up, 8 in 2024-04-07T17:44:05.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:44:05 smithi144 bash[66654]: cluster 2024-04-07T17:44:04.488666+0000 mon.a (mon.0) 4355 : cluster [DBG] osdmap e2381: 8 total, 8 up, 8 in 2024-04-07T17:44:05.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:44:05 smithi144 bash[66654]: cluster 2024-04-07T17:44:04.647786+0000 mgr.y (mgr.25175) 2661 : cluster [DBG] pgmap v4617: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 1.2 MiB/s rd, 3.7 KiB/s wr, 1.65k op/s 2024-04-07T17:44:05.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:44:05 smithi100 bash[74942]: cluster 2024-04-07T17:44:04.488666+0000 mon.a (mon.0) 4355 : cluster [DBG] osdmap e2381: 8 total, 8 up, 8 in 2024-04-07T17:44:05.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:44:05 smithi100 bash[74942]: cluster 2024-04-07T17:44:04.647786+0000 mgr.y (mgr.25175) 2661 : cluster [DBG] pgmap v4617: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 1.2 MiB/s rd, 3.7 KiB/s wr, 1.65k op/s 2024-04-07T17:44:05.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:44:05 smithi100 bash[76202]: cluster 2024-04-07T17:44:04.488666+0000 mon.a (mon.0) 4355 : cluster [DBG] osdmap e2381: 8 total, 8 up, 8 in 2024-04-07T17:44:05.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:44:05 smithi100 bash[76202]: cluster 2024-04-07T17:44:04.647786+0000 mgr.y (mgr.25175) 2661 : cluster [DBG] pgmap v4617: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 1.2 MiB/s rd, 3.7 KiB/s wr, 1.65k op/s 2024-04-07T17:44:06.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:44:06 smithi144 bash[66654]: cluster 2024-04-07T17:44:05.498566+0000 mon.a (mon.0) 4356 : cluster [DBG] osdmap e2382: 8 total, 8 up, 8 in 2024-04-07T17:44:06.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:44:06 smithi100 bash[74942]: cluster 2024-04-07T17:44:05.498566+0000 mon.a (mon.0) 4356 : cluster [DBG] osdmap e2382: 8 total, 8 up, 8 in 2024-04-07T17:44:06.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:44:06 smithi100 bash[76202]: cluster 2024-04-07T17:44:05.498566+0000 mon.a (mon.0) 4356 : cluster [DBG] osdmap e2382: 8 total, 8 up, 8 in 2024-04-07T17:44:07.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:44:07 smithi144 bash[66654]: cluster 2024-04-07T17:44:06.506337+0000 mon.a (mon.0) 4357 : cluster [DBG] osdmap e2383: 8 total, 8 up, 8 in 2024-04-07T17:44:07.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:44:07 smithi144 bash[66654]: cluster 2024-04-07T17:44:06.648468+0000 mgr.y (mgr.25175) 2662 : cluster [DBG] pgmap v4620: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 1.2 MiB/s rd, 3.2 KiB/s wr, 1.64k op/s 2024-04-07T17:44:07.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:44:07 smithi100 bash[74942]: cluster 2024-04-07T17:44:06.506337+0000 mon.a (mon.0) 4357 : cluster [DBG] osdmap e2383: 8 total, 8 up, 8 in 2024-04-07T17:44:07.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:44:07 smithi100 bash[74942]: cluster 2024-04-07T17:44:06.648468+0000 mgr.y (mgr.25175) 2662 : cluster [DBG] pgmap v4620: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 1.2 MiB/s rd, 3.2 KiB/s wr, 1.64k op/s 2024-04-07T17:44:07.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:44:07 smithi100 bash[76202]: cluster 2024-04-07T17:44:06.506337+0000 mon.a (mon.0) 4357 : cluster [DBG] osdmap e2383: 8 total, 8 up, 8 in 2024-04-07T17:44:07.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:44:07 smithi100 bash[76202]: cluster 2024-04-07T17:44:06.648468+0000 mgr.y (mgr.25175) 2662 : cluster [DBG] pgmap v4620: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 1.2 MiB/s rd, 3.2 KiB/s wr, 1.64k op/s 2024-04-07T17:44:08.703 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:44:08 smithi100 bash[74942]: cluster 2024-04-07T17:44:07.517517+0000 mon.a (mon.0) 4358 : cluster [DBG] osdmap e2384: 8 total, 8 up, 8 in 2024-04-07T17:44:08.704 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:44:08 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:44:08] "GET /metrics HTTP/1.1" 200 36750 "" "Prometheus/2.43.0" 2024-04-07T17:44:08.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:44:08 smithi100 bash[76202]: cluster 2024-04-07T17:44:07.517517+0000 mon.a (mon.0) 4358 : cluster [DBG] osdmap e2384: 8 total, 8 up, 8 in 2024-04-07T17:44:08.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:44:08 smithi144 bash[66654]: cluster 2024-04-07T17:44:07.517517+0000 mon.a (mon.0) 4358 : cluster [DBG] osdmap e2384: 8 total, 8 up, 8 in 2024-04-07T17:44:09.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:44:09 smithi144 bash[66654]: cluster 2024-04-07T17:44:08.522811+0000 mon.a (mon.0) 4359 : cluster [DBG] osdmap e2385: 8 total, 8 up, 8 in 2024-04-07T17:44:09.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:44:09 smithi144 bash[66654]: cluster 2024-04-07T17:44:08.649308+0000 mgr.y (mgr.25175) 2663 : cluster [DBG] pgmap v4623: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 5.0 KiB/s rd, 1023 B/s wr, 7 op/s 2024-04-07T17:44:09.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:44:09 smithi100 bash[74942]: cluster 2024-04-07T17:44:08.522811+0000 mon.a (mon.0) 4359 : cluster [DBG] osdmap e2385: 8 total, 8 up, 8 in 2024-04-07T17:44:09.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:44:09 smithi100 bash[74942]: cluster 2024-04-07T17:44:08.649308+0000 mgr.y (mgr.25175) 2663 : cluster [DBG] pgmap v4623: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 5.0 KiB/s rd, 1023 B/s wr, 7 op/s 2024-04-07T17:44:09.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:44:09 smithi100 bash[76202]: cluster 2024-04-07T17:44:08.522811+0000 mon.a (mon.0) 4359 : cluster [DBG] osdmap e2385: 8 total, 8 up, 8 in 2024-04-07T17:44:09.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:44:09 smithi100 bash[76202]: cluster 2024-04-07T17:44:08.649308+0000 mgr.y (mgr.25175) 2663 : cluster [DBG] pgmap v4623: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 5.0 KiB/s rd, 1023 B/s wr, 7 op/s 2024-04-07T17:44:10.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:44:10 smithi144 bash[66654]: cluster 2024-04-07T17:44:09.546351+0000 mon.a (mon.0) 4360 : cluster [DBG] osdmap e2386: 8 total, 8 up, 8 in 2024-04-07T17:44:10.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:44:10 smithi100 bash[74942]: cluster 2024-04-07T17:44:09.546351+0000 mon.a (mon.0) 4360 : cluster [DBG] osdmap e2386: 8 total, 8 up, 8 in 2024-04-07T17:44:10.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:44:10 smithi100 bash[76202]: cluster 2024-04-07T17:44:09.546351+0000 mon.a (mon.0) 4360 : cluster [DBG] osdmap e2386: 8 total, 8 up, 8 in 2024-04-07T17:44:11.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:44:11 smithi144 bash[66654]: cluster 2024-04-07T17:44:10.549448+0000 mon.a (mon.0) 4361 : cluster [DBG] osdmap e2387: 8 total, 8 up, 8 in 2024-04-07T17:44:11.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:44:11 smithi144 bash[66654]: cluster 2024-04-07T17:44:10.650073+0000 mgr.y (mgr.25175) 2664 : cluster [DBG] pgmap v4626: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 1.2 MiB/s rd, 4.0 KiB/s wr, 1.69k op/s 2024-04-07T17:44:11.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:44:11 smithi100 bash[74942]: cluster 2024-04-07T17:44:10.549448+0000 mon.a (mon.0) 4361 : cluster [DBG] osdmap e2387: 8 total, 8 up, 8 in 2024-04-07T17:44:11.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:44:11 smithi100 bash[74942]: cluster 2024-04-07T17:44:10.650073+0000 mgr.y (mgr.25175) 2664 : cluster [DBG] pgmap v4626: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 1.2 MiB/s rd, 4.0 KiB/s wr, 1.69k op/s 2024-04-07T17:44:11.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:44:11 smithi100 bash[76202]: cluster 2024-04-07T17:44:10.549448+0000 mon.a (mon.0) 4361 : cluster [DBG] osdmap e2387: 8 total, 8 up, 8 in 2024-04-07T17:44:11.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:44:11 smithi100 bash[76202]: cluster 2024-04-07T17:44:10.650073+0000 mgr.y (mgr.25175) 2664 : cluster [DBG] pgmap v4626: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 1.2 MiB/s rd, 4.0 KiB/s wr, 1.69k op/s 2024-04-07T17:44:12.577 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:44:12 smithi144 bash[95233]: ts=2024-04-07T17:44:12.317Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:44:12.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:44:12 smithi144 bash[66654]: cluster 2024-04-07T17:44:11.573403+0000 mon.a (mon.0) 4362 : cluster [DBG] osdmap e2388: 8 total, 8 up, 8 in 2024-04-07T17:44:12.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:44:12 smithi100 bash[74942]: cluster 2024-04-07T17:44:11.573403+0000 mon.a (mon.0) 4362 : cluster [DBG] osdmap e2388: 8 total, 8 up, 8 in 2024-04-07T17:44:12.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:44:12 smithi100 bash[76202]: cluster 2024-04-07T17:44:11.573403+0000 mon.a (mon.0) 4362 : cluster [DBG] osdmap e2388: 8 total, 8 up, 8 in 2024-04-07T17:44:13.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:44:13 smithi144 bash[66654]: cluster 2024-04-07T17:44:12.572107+0000 mon.a (mon.0) 4363 : cluster [DBG] osdmap e2389: 8 total, 8 up, 8 in 2024-04-07T17:44:13.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:44:13 smithi144 bash[66654]: cluster 2024-04-07T17:44:12.650835+0000 mgr.y (mgr.25175) 2665 : cluster [DBG] pgmap v4629: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 1.2 MiB/s rd, 3.5 KiB/s wr, 1.69k op/s 2024-04-07T17:44:13.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:44:13 smithi100 bash[74942]: cluster 2024-04-07T17:44:12.572107+0000 mon.a (mon.0) 4363 : cluster [DBG] osdmap e2389: 8 total, 8 up, 8 in 2024-04-07T17:44:13.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:44:13 smithi100 bash[74942]: cluster 2024-04-07T17:44:12.650835+0000 mgr.y (mgr.25175) 2665 : cluster [DBG] pgmap v4629: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 1.2 MiB/s rd, 3.5 KiB/s wr, 1.69k op/s 2024-04-07T17:44:13.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:44:13 smithi100 bash[76202]: cluster 2024-04-07T17:44:12.572107+0000 mon.a (mon.0) 4363 : cluster [DBG] osdmap e2389: 8 total, 8 up, 8 in 2024-04-07T17:44:13.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:44:13 smithi100 bash[76202]: cluster 2024-04-07T17:44:12.650835+0000 mgr.y (mgr.25175) 2665 : cluster [DBG] pgmap v4629: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 1.2 MiB/s rd, 3.5 KiB/s wr, 1.69k op/s 2024-04-07T17:44:14.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:44:14 smithi144 bash[66654]: cluster 2024-04-07T17:44:13.603701+0000 mon.a (mon.0) 4364 : cluster [DBG] osdmap e2390: 8 total, 8 up, 8 in 2024-04-07T17:44:14.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:44:14 smithi100 bash[74942]: cluster 2024-04-07T17:44:13.603701+0000 mon.a (mon.0) 4364 : cluster [DBG] osdmap e2390: 8 total, 8 up, 8 in 2024-04-07T17:44:14.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:44:14 smithi100 bash[76202]: cluster 2024-04-07T17:44:13.603701+0000 mon.a (mon.0) 4364 : cluster [DBG] osdmap e2390: 8 total, 8 up, 8 in 2024-04-07T17:44:15.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:44:15 smithi100 bash[74942]: cluster 2024-04-07T17:44:14.598840+0000 mon.a (mon.0) 4365 : cluster [DBG] osdmap e2391: 8 total, 8 up, 8 in 2024-04-07T17:44:15.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:44:15 smithi100 bash[74942]: cluster 2024-04-07T17:44:14.651599+0000 mgr.y (mgr.25175) 2666 : cluster [DBG] pgmap v4632: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 1.3 MiB/s rd, 3.2 KiB/s wr, 1.73k op/s 2024-04-07T17:44:15.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:44:15 smithi100 bash[76202]: cluster 2024-04-07T17:44:14.598840+0000 mon.a (mon.0) 4365 : cluster [DBG] osdmap e2391: 8 total, 8 up, 8 in 2024-04-07T17:44:15.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:44:15 smithi100 bash[76202]: cluster 2024-04-07T17:44:14.651599+0000 mgr.y (mgr.25175) 2666 : cluster [DBG] pgmap v4632: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 1.3 MiB/s rd, 3.2 KiB/s wr, 1.73k op/s 2024-04-07T17:44:16.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:44:15 smithi144 bash[66654]: cluster 2024-04-07T17:44:14.598840+0000 mon.a (mon.0) 4365 : cluster [DBG] osdmap e2391: 8 total, 8 up, 8 in 2024-04-07T17:44:16.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:44:15 smithi144 bash[66654]: cluster 2024-04-07T17:44:14.651599+0000 mgr.y (mgr.25175) 2666 : cluster [DBG] pgmap v4632: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 1.3 MiB/s rd, 3.2 KiB/s wr, 1.73k op/s 2024-04-07T17:44:16.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:44:16 smithi100 bash[74942]: cluster 2024-04-07T17:44:15.625028+0000 mon.a (mon.0) 4366 : cluster [DBG] osdmap e2392: 8 total, 8 up, 8 in 2024-04-07T17:44:16.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:44:16 smithi100 bash[76202]: cluster 2024-04-07T17:44:15.625028+0000 mon.a (mon.0) 4366 : cluster [DBG] osdmap e2392: 8 total, 8 up, 8 in 2024-04-07T17:44:17.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:44:16 smithi144 bash[66654]: cluster 2024-04-07T17:44:15.625028+0000 mon.a (mon.0) 4366 : cluster [DBG] osdmap e2392: 8 total, 8 up, 8 in 2024-04-07T17:44:17.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:44:17 smithi100 bash[74942]: cluster 2024-04-07T17:44:16.624578+0000 mon.a (mon.0) 4367 : cluster [DBG] osdmap e2393: 8 total, 8 up, 8 in 2024-04-07T17:44:17.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:44:17 smithi100 bash[74942]: cluster 2024-04-07T17:44:16.652437+0000 mgr.y (mgr.25175) 2667 : cluster [DBG] pgmap v4635: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 1.3 MiB/s rd, 3.5 KiB/s wr, 1.73k op/s 2024-04-07T17:44:17.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:44:17 smithi100 bash[74942]: audit 2024-04-07T17:44:16.870545+0000 mon.a (mon.0) 4368 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:44:17.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:44:17 smithi100 bash[76202]: cluster 2024-04-07T17:44:16.624578+0000 mon.a (mon.0) 4367 : cluster [DBG] osdmap e2393: 8 total, 8 up, 8 in 2024-04-07T17:44:17.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:44:17 smithi100 bash[76202]: cluster 2024-04-07T17:44:16.652437+0000 mgr.y (mgr.25175) 2667 : cluster [DBG] pgmap v4635: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 1.3 MiB/s rd, 3.5 KiB/s wr, 1.73k op/s 2024-04-07T17:44:17.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:44:17 smithi100 bash[76202]: audit 2024-04-07T17:44:16.870545+0000 mon.a (mon.0) 4368 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:44:18.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:44:17 smithi144 bash[66654]: cluster 2024-04-07T17:44:16.624578+0000 mon.a (mon.0) 4367 : cluster [DBG] osdmap e2393: 8 total, 8 up, 8 in 2024-04-07T17:44:18.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:44:17 smithi144 bash[66654]: cluster 2024-04-07T17:44:16.652437+0000 mgr.y (mgr.25175) 2667 : cluster [DBG] pgmap v4635: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 1.3 MiB/s rd, 3.5 KiB/s wr, 1.73k op/s 2024-04-07T17:44:18.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:44:17 smithi144 bash[66654]: audit 2024-04-07T17:44:16.870545+0000 mon.a (mon.0) 4368 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:44:18.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:44:18 smithi100 bash[74942]: cluster 2024-04-07T17:44:17.652875+0000 mon.a (mon.0) 4369 : cluster [DBG] osdmap e2394: 8 total, 8 up, 8 in 2024-04-07T17:44:18.704 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:44:18 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:44:18] "GET /metrics HTTP/1.1" 200 36748 "" "Prometheus/2.43.0" 2024-04-07T17:44:18.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:44:18 smithi100 bash[76202]: cluster 2024-04-07T17:44:17.652875+0000 mon.a (mon.0) 4369 : cluster [DBG] osdmap e2394: 8 total, 8 up, 8 in 2024-04-07T17:44:19.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:44:18 smithi144 bash[66654]: cluster 2024-04-07T17:44:17.652875+0000 mon.a (mon.0) 4369 : cluster [DBG] osdmap e2394: 8 total, 8 up, 8 in 2024-04-07T17:44:19.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:44:19 smithi100 bash[74942]: cluster 2024-04-07T17:44:18.651271+0000 mon.a (mon.0) 4370 : cluster [DBG] osdmap e2395: 8 total, 8 up, 8 in 2024-04-07T17:44:19.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:44:19 smithi100 bash[74942]: cluster 2024-04-07T17:44:18.653676+0000 mgr.y (mgr.25175) 2668 : cluster [DBG] pgmap v4638: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 5.0 KiB/s rd, 1023 B/s wr, 7 op/s 2024-04-07T17:44:19.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:44:19 smithi100 bash[76202]: cluster 2024-04-07T17:44:18.651271+0000 mon.a (mon.0) 4370 : cluster [DBG] osdmap e2395: 8 total, 8 up, 8 in 2024-04-07T17:44:19.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:44:19 smithi100 bash[76202]: cluster 2024-04-07T17:44:18.653676+0000 mgr.y (mgr.25175) 2668 : cluster [DBG] pgmap v4638: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 5.0 KiB/s rd, 1023 B/s wr, 7 op/s 2024-04-07T17:44:20.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:44:19 smithi144 bash[66654]: cluster 2024-04-07T17:44:18.651271+0000 mon.a (mon.0) 4370 : cluster [DBG] osdmap e2395: 8 total, 8 up, 8 in 2024-04-07T17:44:20.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:44:19 smithi144 bash[66654]: cluster 2024-04-07T17:44:18.653676+0000 mgr.y (mgr.25175) 2668 : cluster [DBG] pgmap v4638: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 5.0 KiB/s rd, 1023 B/s wr, 7 op/s 2024-04-07T17:44:20.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:44:20 smithi100 bash[74942]: cluster 2024-04-07T17:44:19.673765+0000 mon.a (mon.0) 4371 : cluster [DBG] osdmap e2396: 8 total, 8 up, 8 in 2024-04-07T17:44:20.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:44:20 smithi100 bash[76202]: cluster 2024-04-07T17:44:19.673765+0000 mon.a (mon.0) 4371 : cluster [DBG] osdmap e2396: 8 total, 8 up, 8 in 2024-04-07T17:44:21.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:44:20 smithi144 bash[66654]: cluster 2024-04-07T17:44:19.673765+0000 mon.a (mon.0) 4371 : cluster [DBG] osdmap e2396: 8 total, 8 up, 8 in 2024-04-07T17:44:21.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:44:21 smithi100 bash[74942]: cluster 2024-04-07T17:44:20.655126+0000 mgr.y (mgr.25175) 2669 : cluster [DBG] pgmap v4640: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 1.3 MiB/s rd, 4.2 KiB/s wr, 1.77k op/s 2024-04-07T17:44:21.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:44:21 smithi100 bash[74942]: cluster 2024-04-07T17:44:20.686742+0000 mon.a (mon.0) 4372 : cluster [DBG] osdmap e2397: 8 total, 8 up, 8 in 2024-04-07T17:44:21.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:44:21 smithi100 bash[76202]: cluster 2024-04-07T17:44:20.655126+0000 mgr.y (mgr.25175) 2669 : cluster [DBG] pgmap v4640: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 1.3 MiB/s rd, 4.2 KiB/s wr, 1.77k op/s 2024-04-07T17:44:21.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:44:21 smithi100 bash[76202]: cluster 2024-04-07T17:44:20.686742+0000 mon.a (mon.0) 4372 : cluster [DBG] osdmap e2397: 8 total, 8 up, 8 in 2024-04-07T17:44:22.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:44:21 smithi144 bash[66654]: cluster 2024-04-07T17:44:20.655126+0000 mgr.y (mgr.25175) 2669 : cluster [DBG] pgmap v4640: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 1.3 MiB/s rd, 4.2 KiB/s wr, 1.77k op/s 2024-04-07T17:44:22.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:44:21 smithi144 bash[66654]: cluster 2024-04-07T17:44:20.686742+0000 mon.a (mon.0) 4372 : cluster [DBG] osdmap e2397: 8 total, 8 up, 8 in 2024-04-07T17:44:22.614 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:44:22 smithi144 bash[95233]: ts=2024-04-07T17:44:22.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:44:22.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:44:22 smithi100 bash[74942]: cluster 2024-04-07T17:44:21.691480+0000 mon.a (mon.0) 4373 : cluster [DBG] osdmap e2398: 8 total, 8 up, 8 in 2024-04-07T17:44:22.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:44:22 smithi100 bash[76202]: cluster 2024-04-07T17:44:21.691480+0000 mon.a (mon.0) 4373 : cluster [DBG] osdmap e2398: 8 total, 8 up, 8 in 2024-04-07T17:44:23.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:44:22 smithi144 bash[66654]: cluster 2024-04-07T17:44:21.691480+0000 mon.a (mon.0) 4373 : cluster [DBG] osdmap e2398: 8 total, 8 up, 8 in 2024-04-07T17:44:23.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:44:23 smithi100 bash[74942]: cluster 2024-04-07T17:44:22.656508+0000 mgr.y (mgr.25175) 2670 : cluster [DBG] pgmap v4643: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 1.3 MiB/s rd, 4.0 KiB/s wr, 1.78k op/s 2024-04-07T17:44:23.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:44:23 smithi100 bash[74942]: cluster 2024-04-07T17:44:22.703256+0000 mon.a (mon.0) 4374 : cluster [DBG] osdmap e2399: 8 total, 8 up, 8 in 2024-04-07T17:44:23.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:44:23 smithi100 bash[76202]: cluster 2024-04-07T17:44:22.656508+0000 mgr.y (mgr.25175) 2670 : cluster [DBG] pgmap v4643: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 1.3 MiB/s rd, 4.0 KiB/s wr, 1.78k op/s 2024-04-07T17:44:23.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:44:23 smithi100 bash[76202]: cluster 2024-04-07T17:44:22.703256+0000 mon.a (mon.0) 4374 : cluster [DBG] osdmap e2399: 8 total, 8 up, 8 in 2024-04-07T17:44:24.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:44:23 smithi144 bash[66654]: cluster 2024-04-07T17:44:22.656508+0000 mgr.y (mgr.25175) 2670 : cluster [DBG] pgmap v4643: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 1.3 MiB/s rd, 4.0 KiB/s wr, 1.78k op/s 2024-04-07T17:44:24.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:44:23 smithi144 bash[66654]: cluster 2024-04-07T17:44:22.703256+0000 mon.a (mon.0) 4374 : cluster [DBG] osdmap e2399: 8 total, 8 up, 8 in 2024-04-07T17:44:25.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:44:24 smithi144 bash[66654]: cluster 2024-04-07T17:44:23.716948+0000 mon.a (mon.0) 4375 : cluster [DBG] osdmap e2400: 8 total, 8 up, 8 in 2024-04-07T17:44:25.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:44:24 smithi100 bash[74942]: cluster 2024-04-07T17:44:23.716948+0000 mon.a (mon.0) 4375 : cluster [DBG] osdmap e2400: 8 total, 8 up, 8 in 2024-04-07T17:44:25.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:44:24 smithi100 bash[76202]: cluster 2024-04-07T17:44:23.716948+0000 mon.a (mon.0) 4375 : cluster [DBG] osdmap e2400: 8 total, 8 up, 8 in 2024-04-07T17:44:26.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:44:25 smithi144 bash[66654]: cluster 2024-04-07T17:44:24.657557+0000 mgr.y (mgr.25175) 2671 : cluster [DBG] pgmap v4646: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 1.3 MiB/s rd, 5.2 KiB/s wr, 1.78k op/s 2024-04-07T17:44:26.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:44:25 smithi144 bash[66654]: cluster 2024-04-07T17:44:24.723129+0000 mon.a (mon.0) 4376 : cluster [DBG] osdmap e2401: 8 total, 8 up, 8 in 2024-04-07T17:44:26.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:44:25 smithi100 bash[74942]: cluster 2024-04-07T17:44:24.657557+0000 mgr.y (mgr.25175) 2671 : cluster [DBG] pgmap v4646: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 1.3 MiB/s rd, 5.2 KiB/s wr, 1.78k op/s 2024-04-07T17:44:26.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:44:25 smithi100 bash[74942]: cluster 2024-04-07T17:44:24.723129+0000 mon.a (mon.0) 4376 : cluster [DBG] osdmap e2401: 8 total, 8 up, 8 in 2024-04-07T17:44:26.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:44:25 smithi100 bash[76202]: cluster 2024-04-07T17:44:24.657557+0000 mgr.y (mgr.25175) 2671 : cluster [DBG] pgmap v4646: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 1.3 MiB/s rd, 5.2 KiB/s wr, 1.78k op/s 2024-04-07T17:44:26.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:44:25 smithi100 bash[76202]: cluster 2024-04-07T17:44:24.723129+0000 mon.a (mon.0) 4376 : cluster [DBG] osdmap e2401: 8 total, 8 up, 8 in 2024-04-07T17:44:27.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:44:26 smithi144 bash[66654]: cluster 2024-04-07T17:44:25.736031+0000 mon.a (mon.0) 4377 : cluster [DBG] osdmap e2402: 8 total, 8 up, 8 in 2024-04-07T17:44:27.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:44:26 smithi100 bash[76202]: cluster 2024-04-07T17:44:25.736031+0000 mon.a (mon.0) 4377 : cluster [DBG] osdmap e2402: 8 total, 8 up, 8 in 2024-04-07T17:44:27.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:44:26 smithi100 bash[74942]: cluster 2024-04-07T17:44:25.736031+0000 mon.a (mon.0) 4377 : cluster [DBG] osdmap e2402: 8 total, 8 up, 8 in 2024-04-07T17:44:28.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:44:27 smithi144 bash[66654]: cluster 2024-04-07T17:44:26.658283+0000 mgr.y (mgr.25175) 2672 : cluster [DBG] pgmap v4649: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 1.3 MiB/s rd, 5.0 KiB/s wr, 1.77k op/s 2024-04-07T17:44:28.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:44:27 smithi144 bash[66654]: cluster 2024-04-07T17:44:26.744599+0000 mon.a (mon.0) 4378 : cluster [DBG] osdmap e2403: 8 total, 8 up, 8 in 2024-04-07T17:44:28.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:44:27 smithi100 bash[74942]: cluster 2024-04-07T17:44:26.658283+0000 mgr.y (mgr.25175) 2672 : cluster [DBG] pgmap v4649: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 1.3 MiB/s rd, 5.0 KiB/s wr, 1.77k op/s 2024-04-07T17:44:28.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:44:27 smithi100 bash[74942]: cluster 2024-04-07T17:44:26.744599+0000 mon.a (mon.0) 4378 : cluster [DBG] osdmap e2403: 8 total, 8 up, 8 in 2024-04-07T17:44:28.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:44:27 smithi100 bash[76202]: cluster 2024-04-07T17:44:26.658283+0000 mgr.y (mgr.25175) 2672 : cluster [DBG] pgmap v4649: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 1.3 MiB/s rd, 5.0 KiB/s wr, 1.77k op/s 2024-04-07T17:44:28.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:44:27 smithi100 bash[76202]: cluster 2024-04-07T17:44:26.744599+0000 mon.a (mon.0) 4378 : cluster [DBG] osdmap e2403: 8 total, 8 up, 8 in 2024-04-07T17:44:28.703 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:44:28 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:44:28] "GET /metrics HTTP/1.1" 200 36748 "" "Prometheus/2.43.0" 2024-04-07T17:44:29.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:44:28 smithi144 bash[66654]: cluster 2024-04-07T17:44:27.747683+0000 mon.a (mon.0) 4379 : cluster [DBG] osdmap e2404: 8 total, 8 up, 8 in 2024-04-07T17:44:29.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:44:28 smithi144 bash[66654]: cluster 2024-04-07T17:44:28.377989+0000 mon.a (mon.0) 4380 : cluster [DBG] osdmap e2405: 8 total, 8 up, 8 in 2024-04-07T17:44:29.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:44:28 smithi100 bash[74942]: cluster 2024-04-07T17:44:27.747683+0000 mon.a (mon.0) 4379 : cluster [DBG] osdmap e2404: 8 total, 8 up, 8 in 2024-04-07T17:44:29.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:44:28 smithi100 bash[74942]: cluster 2024-04-07T17:44:28.377989+0000 mon.a (mon.0) 4380 : cluster [DBG] osdmap e2405: 8 total, 8 up, 8 in 2024-04-07T17:44:29.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:44:28 smithi100 bash[76202]: cluster 2024-04-07T17:44:27.747683+0000 mon.a (mon.0) 4379 : cluster [DBG] osdmap e2404: 8 total, 8 up, 8 in 2024-04-07T17:44:29.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:44:28 smithi100 bash[76202]: cluster 2024-04-07T17:44:28.377989+0000 mon.a (mon.0) 4380 : cluster [DBG] osdmap e2405: 8 total, 8 up, 8 in 2024-04-07T17:44:30.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:44:29 smithi144 bash[66654]: cluster 2024-04-07T17:44:28.659113+0000 mgr.y (mgr.25175) 2673 : cluster [DBG] pgmap v4653: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 3.0 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-07T17:44:30.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:44:29 smithi144 bash[66654]: cluster 2024-04-07T17:44:29.393062+0000 mon.a (mon.0) 4381 : cluster [DBG] osdmap e2406: 8 total, 8 up, 8 in 2024-04-07T17:44:30.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:44:29 smithi100 bash[74942]: cluster 2024-04-07T17:44:28.659113+0000 mgr.y (mgr.25175) 2673 : cluster [DBG] pgmap v4653: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 3.0 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-07T17:44:30.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:44:29 smithi100 bash[74942]: cluster 2024-04-07T17:44:29.393062+0000 mon.a (mon.0) 4381 : cluster [DBG] osdmap e2406: 8 total, 8 up, 8 in 2024-04-07T17:44:30.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:44:29 smithi100 bash[76202]: cluster 2024-04-07T17:44:28.659113+0000 mgr.y (mgr.25175) 2673 : cluster [DBG] pgmap v4653: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 3.0 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-07T17:44:30.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:44:29 smithi100 bash[76202]: cluster 2024-04-07T17:44:29.393062+0000 mon.a (mon.0) 4381 : cluster [DBG] osdmap e2406: 8 total, 8 up, 8 in 2024-04-07T17:44:31.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:44:31 smithi100 bash[74942]: cluster 2024-04-07T17:44:30.409210+0000 mon.a (mon.0) 4382 : cluster [DBG] osdmap e2407: 8 total, 8 up, 8 in 2024-04-07T17:44:31.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:44:31 smithi100 bash[74942]: cluster 2024-04-07T17:44:30.659858+0000 mgr.y (mgr.25175) 2674 : cluster [DBG] pgmap v4656: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 1.3 MiB/s rd, 5.1 KiB/s wr, 1.77k op/s 2024-04-07T17:44:31.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:44:31 smithi100 bash[76202]: cluster 2024-04-07T17:44:30.409210+0000 mon.a (mon.0) 4382 : cluster [DBG] osdmap e2407: 8 total, 8 up, 8 in 2024-04-07T17:44:31.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:44:31 smithi100 bash[76202]: cluster 2024-04-07T17:44:30.659858+0000 mgr.y (mgr.25175) 2674 : cluster [DBG] pgmap v4656: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 1.3 MiB/s rd, 5.1 KiB/s wr, 1.77k op/s 2024-04-07T17:44:31.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:44:31 smithi144 bash[66654]: cluster 2024-04-07T17:44:30.409210+0000 mon.a (mon.0) 4382 : cluster [DBG] osdmap e2407: 8 total, 8 up, 8 in 2024-04-07T17:44:31.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:44:31 smithi144 bash[66654]: cluster 2024-04-07T17:44:30.659858+0000 mgr.y (mgr.25175) 2674 : cluster [DBG] pgmap v4656: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 1.3 MiB/s rd, 5.1 KiB/s wr, 1.77k op/s 2024-04-07T17:44:32.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:44:32 smithi144 bash[66654]: cluster 2024-04-07T17:44:31.423068+0000 mon.a (mon.0) 4383 : cluster [DBG] osdmap e2408: 8 total, 8 up, 8 in 2024-04-07T17:44:32.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:44:32 smithi144 bash[66654]: audit 2024-04-07T17:44:31.871054+0000 mon.a (mon.0) 4384 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:44:32.615 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:44:32 smithi144 bash[95233]: ts=2024-04-07T17:44:32.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:44:32.703 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:44:32 smithi100 bash[74942]: cluster 2024-04-07T17:44:31.423068+0000 mon.a (mon.0) 4383 : cluster [DBG] osdmap e2408: 8 total, 8 up, 8 in 2024-04-07T17:44:32.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:44:32 smithi100 bash[74942]: audit 2024-04-07T17:44:31.871054+0000 mon.a (mon.0) 4384 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:44:32.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:44:32 smithi100 bash[76202]: cluster 2024-04-07T17:44:31.423068+0000 mon.a (mon.0) 4383 : cluster [DBG] osdmap e2408: 8 total, 8 up, 8 in 2024-04-07T17:44:32.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:44:32 smithi100 bash[76202]: audit 2024-04-07T17:44:31.871054+0000 mon.a (mon.0) 4384 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:44:33.703 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:44:33 smithi100 bash[74942]: cluster 2024-04-07T17:44:32.431470+0000 mon.a (mon.0) 4385 : cluster [DBG] osdmap e2409: 8 total, 8 up, 8 in 2024-04-07T17:44:33.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:44:33 smithi100 bash[74942]: cluster 2024-04-07T17:44:32.660589+0000 mgr.y (mgr.25175) 2675 : cluster [DBG] pgmap v4659: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 1.3 MiB/s rd, 5.0 KiB/s wr, 1.73k op/s 2024-04-07T17:44:33.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:44:33 smithi100 bash[74942]: cluster 2024-04-07T17:44:33.378130+0000 mon.a (mon.0) 4386 : cluster [DBG] osdmap e2410: 8 total, 8 up, 8 in 2024-04-07T17:44:33.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:44:33 smithi100 bash[76202]: cluster 2024-04-07T17:44:32.431470+0000 mon.a (mon.0) 4385 : cluster [DBG] osdmap e2409: 8 total, 8 up, 8 in 2024-04-07T17:44:33.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:44:33 smithi100 bash[76202]: cluster 2024-04-07T17:44:32.660589+0000 mgr.y (mgr.25175) 2675 : cluster [DBG] pgmap v4659: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 1.3 MiB/s rd, 5.0 KiB/s wr, 1.73k op/s 2024-04-07T17:44:33.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:44:33 smithi100 bash[76202]: cluster 2024-04-07T17:44:33.378130+0000 mon.a (mon.0) 4386 : cluster [DBG] osdmap e2410: 8 total, 8 up, 8 in 2024-04-07T17:44:33.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:44:33 smithi144 bash[66654]: cluster 2024-04-07T17:44:32.431470+0000 mon.a (mon.0) 4385 : cluster [DBG] osdmap e2409: 8 total, 8 up, 8 in 2024-04-07T17:44:33.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:44:33 smithi144 bash[66654]: cluster 2024-04-07T17:44:32.660589+0000 mgr.y (mgr.25175) 2675 : cluster [DBG] pgmap v4659: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 1.3 MiB/s rd, 5.0 KiB/s wr, 1.73k op/s 2024-04-07T17:44:33.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:44:33 smithi144 bash[66654]: cluster 2024-04-07T17:44:33.378130+0000 mon.a (mon.0) 4386 : cluster [DBG] osdmap e2410: 8 total, 8 up, 8 in 2024-04-07T17:44:35.703 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:44:35 smithi100 bash[74942]: cluster 2024-04-07T17:44:34.440338+0000 mon.a (mon.0) 4387 : cluster [DBG] osdmap e2411: 8 total, 8 up, 8 in 2024-04-07T17:44:35.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:44:35 smithi100 bash[74942]: cluster 2024-04-07T17:44:34.661348+0000 mgr.y (mgr.25175) 2676 : cluster [DBG] pgmap v4662: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 1.2 MiB/s rd, 5.0 KiB/s wr, 1.69k op/s 2024-04-07T17:44:35.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:44:35 smithi100 bash[76202]: cluster 2024-04-07T17:44:34.440338+0000 mon.a (mon.0) 4387 : cluster [DBG] osdmap e2411: 8 total, 8 up, 8 in 2024-04-07T17:44:35.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:44:35 smithi100 bash[76202]: cluster 2024-04-07T17:44:34.661348+0000 mgr.y (mgr.25175) 2676 : cluster [DBG] pgmap v4662: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 1.2 MiB/s rd, 5.0 KiB/s wr, 1.69k op/s 2024-04-07T17:44:35.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:44:35 smithi144 bash[66654]: cluster 2024-04-07T17:44:34.440338+0000 mon.a (mon.0) 4387 : cluster [DBG] osdmap e2411: 8 total, 8 up, 8 in 2024-04-07T17:44:35.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:44:35 smithi144 bash[66654]: cluster 2024-04-07T17:44:34.661348+0000 mgr.y (mgr.25175) 2676 : cluster [DBG] pgmap v4662: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 1.2 MiB/s rd, 5.0 KiB/s wr, 1.69k op/s 2024-04-07T17:44:36.703 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:44:36 smithi100 bash[76202]: cluster 2024-04-07T17:44:35.453751+0000 mon.a (mon.0) 4388 : cluster [DBG] osdmap e2412: 8 total, 8 up, 8 in 2024-04-07T17:44:36.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:44:36 smithi100 bash[74942]: cluster 2024-04-07T17:44:35.453751+0000 mon.a (mon.0) 4388 : cluster [DBG] osdmap e2412: 8 total, 8 up, 8 in 2024-04-07T17:44:36.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:44:36 smithi144 bash[66654]: cluster 2024-04-07T17:44:35.453751+0000 mon.a (mon.0) 4388 : cluster [DBG] osdmap e2412: 8 total, 8 up, 8 in 2024-04-07T17:44:37.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:44:37 smithi144 bash[66654]: cluster 2024-04-07T17:44:36.464864+0000 mon.a (mon.0) 4389 : cluster [DBG] osdmap e2413: 8 total, 8 up, 8 in 2024-04-07T17:44:37.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:44:37 smithi144 bash[66654]: audit 2024-04-07T17:44:36.611579+0000 mon.a (mon.0) 4390 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:44:37.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:44:37 smithi144 bash[66654]: cluster 2024-04-07T17:44:36.662316+0000 mgr.y (mgr.25175) 2677 : cluster [DBG] pgmap v4665: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 1.2 MiB/s rd, 5.0 KiB/s wr, 1.69k op/s 2024-04-07T17:44:37.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:44:37 smithi144 bash[66654]: audit 2024-04-07T17:44:37.345489+0000 mon.a (mon.0) 4391 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:44:37.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:44:37 smithi144 bash[66654]: audit 2024-04-07T17:44:37.347077+0000 mon.a (mon.0) 4392 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:44:37.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:44:37 smithi144 bash[66654]: audit 2024-04-07T17:44:37.356785+0000 mon.a (mon.0) 4393 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:44:37.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:44:37 smithi100 bash[76202]: cluster 2024-04-07T17:44:36.464864+0000 mon.a (mon.0) 4389 : cluster [DBG] osdmap e2413: 8 total, 8 up, 8 in 2024-04-07T17:44:37.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:44:37 smithi100 bash[76202]: audit 2024-04-07T17:44:36.611579+0000 mon.a (mon.0) 4390 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:44:37.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:44:37 smithi100 bash[76202]: cluster 2024-04-07T17:44:36.662316+0000 mgr.y (mgr.25175) 2677 : cluster [DBG] pgmap v4665: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 1.2 MiB/s rd, 5.0 KiB/s wr, 1.69k op/s 2024-04-07T17:44:37.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:44:37 smithi100 bash[76202]: audit 2024-04-07T17:44:37.345489+0000 mon.a (mon.0) 4391 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:44:37.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:44:37 smithi100 bash[76202]: audit 2024-04-07T17:44:37.347077+0000 mon.a (mon.0) 4392 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:44:37.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:44:37 smithi100 bash[76202]: audit 2024-04-07T17:44:37.356785+0000 mon.a (mon.0) 4393 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:44:37.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:44:37 smithi100 bash[74942]: cluster 2024-04-07T17:44:36.464864+0000 mon.a (mon.0) 4389 : cluster [DBG] osdmap e2413: 8 total, 8 up, 8 in 2024-04-07T17:44:37.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:44:37 smithi100 bash[74942]: audit 2024-04-07T17:44:36.611579+0000 mon.a (mon.0) 4390 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:44:37.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:44:37 smithi100 bash[74942]: cluster 2024-04-07T17:44:36.662316+0000 mgr.y (mgr.25175) 2677 : cluster [DBG] pgmap v4665: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 1.2 MiB/s rd, 5.0 KiB/s wr, 1.69k op/s 2024-04-07T17:44:37.955 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:44:37 smithi100 bash[74942]: audit 2024-04-07T17:44:37.345489+0000 mon.a (mon.0) 4391 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:44:37.955 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:44:37 smithi100 bash[74942]: audit 2024-04-07T17:44:37.347077+0000 mon.a (mon.0) 4392 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:44:37.955 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:44:37 smithi100 bash[74942]: audit 2024-04-07T17:44:37.356785+0000 mon.a (mon.0) 4393 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:44:38.703 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:44:38 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:44:38] "GET /metrics HTTP/1.1" 200 36751 "" "Prometheus/2.43.0" 2024-04-07T17:44:38.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:44:38 smithi100 bash[74942]: cluster 2024-04-07T17:44:37.478134+0000 mon.a (mon.0) 4394 : cluster [DBG] osdmap e2414: 8 total, 8 up, 8 in 2024-04-07T17:44:38.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:44:38 smithi100 bash[74942]: cluster 2024-04-07T17:44:38.380559+0000 mon.a (mon.0) 4395 : cluster [DBG] osdmap e2415: 8 total, 8 up, 8 in 2024-04-07T17:44:38.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:44:38 smithi100 bash[76202]: cluster 2024-04-07T17:44:37.478134+0000 mon.a (mon.0) 4394 : cluster [DBG] osdmap e2414: 8 total, 8 up, 8 in 2024-04-07T17:44:38.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:44:38 smithi100 bash[76202]: cluster 2024-04-07T17:44:38.380559+0000 mon.a (mon.0) 4395 : cluster [DBG] osdmap e2415: 8 total, 8 up, 8 in 2024-04-07T17:44:38.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:44:38 smithi144 bash[66654]: cluster 2024-04-07T17:44:37.478134+0000 mon.a (mon.0) 4394 : cluster [DBG] osdmap e2414: 8 total, 8 up, 8 in 2024-04-07T17:44:38.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:44:38 smithi144 bash[66654]: cluster 2024-04-07T17:44:38.380559+0000 mon.a (mon.0) 4395 : cluster [DBG] osdmap e2415: 8 total, 8 up, 8 in 2024-04-07T17:44:39.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:44:39 smithi144 bash[66654]: cluster 2024-04-07T17:44:38.663086+0000 mgr.y (mgr.25175) 2678 : cluster [DBG] pgmap v4668: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 3.0 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-07T17:44:39.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:44:39 smithi100 bash[74942]: cluster 2024-04-07T17:44:38.663086+0000 mgr.y (mgr.25175) 2678 : cluster [DBG] pgmap v4668: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 3.0 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-07T17:44:39.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:44:39 smithi100 bash[76202]: cluster 2024-04-07T17:44:38.663086+0000 mgr.y (mgr.25175) 2678 : cluster [DBG] pgmap v4668: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 3.0 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-07T17:44:40.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:44:40 smithi144 bash[66654]: cluster 2024-04-07T17:44:39.492548+0000 mon.a (mon.0) 4396 : cluster [DBG] osdmap e2416: 8 total, 8 up, 8 in 2024-04-07T17:44:40.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:44:40 smithi100 bash[74942]: cluster 2024-04-07T17:44:39.492548+0000 mon.a (mon.0) 4396 : cluster [DBG] osdmap e2416: 8 total, 8 up, 8 in 2024-04-07T17:44:40.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:44:40 smithi100 bash[76202]: cluster 2024-04-07T17:44:39.492548+0000 mon.a (mon.0) 4396 : cluster [DBG] osdmap e2416: 8 total, 8 up, 8 in 2024-04-07T17:44:41.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:44:41 smithi144 bash[66654]: cluster 2024-04-07T17:44:40.498303+0000 mon.a (mon.0) 4397 : cluster [DBG] osdmap e2417: 8 total, 8 up, 8 in 2024-04-07T17:44:41.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:44:41 smithi144 bash[66654]: cluster 2024-04-07T17:44:40.663818+0000 mgr.y (mgr.25175) 2679 : cluster [DBG] pgmap v4671: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 1.2 MiB/s rd, 5.0 KiB/s wr, 1.64k op/s 2024-04-07T17:44:41.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:44:41 smithi100 bash[74942]: cluster 2024-04-07T17:44:40.498303+0000 mon.a (mon.0) 4397 : cluster [DBG] osdmap e2417: 8 total, 8 up, 8 in 2024-04-07T17:44:41.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:44:41 smithi100 bash[74942]: cluster 2024-04-07T17:44:40.663818+0000 mgr.y (mgr.25175) 2679 : cluster [DBG] pgmap v4671: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 1.2 MiB/s rd, 5.0 KiB/s wr, 1.64k op/s 2024-04-07T17:44:41.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:44:41 smithi100 bash[76202]: cluster 2024-04-07T17:44:40.498303+0000 mon.a (mon.0) 4397 : cluster [DBG] osdmap e2417: 8 total, 8 up, 8 in 2024-04-07T17:44:41.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:44:41 smithi100 bash[76202]: cluster 2024-04-07T17:44:40.663818+0000 mgr.y (mgr.25175) 2679 : cluster [DBG] pgmap v4671: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 1.2 MiB/s rd, 5.0 KiB/s wr, 1.64k op/s 2024-04-07T17:44:42.614 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:44:42 smithi144 bash[66654]: cluster 2024-04-07T17:44:41.509581+0000 mon.a (mon.0) 4398 : cluster [DBG] osdmap e2418: 8 total, 8 up, 8 in 2024-04-07T17:44:42.615 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:44:42 smithi144 bash[95233]: ts=2024-04-07T17:44:42.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:44:42.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:44:42 smithi100 bash[74942]: cluster 2024-04-07T17:44:41.509581+0000 mon.a (mon.0) 4398 : cluster [DBG] osdmap e2418: 8 total, 8 up, 8 in 2024-04-07T17:44:42.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:44:42 smithi100 bash[76202]: cluster 2024-04-07T17:44:41.509581+0000 mon.a (mon.0) 4398 : cluster [DBG] osdmap e2418: 8 total, 8 up, 8 in 2024-04-07T17:44:43.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:44:43 smithi144 bash[66654]: cluster 2024-04-07T17:44:42.510181+0000 mon.a (mon.0) 4399 : cluster [DBG] osdmap e2419: 8 total, 8 up, 8 in 2024-04-07T17:44:43.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:44:43 smithi144 bash[66654]: cluster 2024-04-07T17:44:42.664505+0000 mgr.y (mgr.25175) 2680 : cluster [DBG] pgmap v4674: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 1.2 MiB/s rd, 5.0 KiB/s wr, 1.65k op/s 2024-04-07T17:44:43.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:44:43 smithi144 bash[66654]: cluster 2024-04-07T17:44:43.385493+0000 mon.a (mon.0) 4400 : cluster [DBG] osdmap e2420: 8 total, 8 up, 8 in 2024-04-07T17:44:43.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:44:43 smithi100 bash[74942]: cluster 2024-04-07T17:44:42.510181+0000 mon.a (mon.0) 4399 : cluster [DBG] osdmap e2419: 8 total, 8 up, 8 in 2024-04-07T17:44:43.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:44:43 smithi100 bash[74942]: cluster 2024-04-07T17:44:42.664505+0000 mgr.y (mgr.25175) 2680 : cluster [DBG] pgmap v4674: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 1.2 MiB/s rd, 5.0 KiB/s wr, 1.65k op/s 2024-04-07T17:44:43.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:44:43 smithi100 bash[74942]: cluster 2024-04-07T17:44:43.385493+0000 mon.a (mon.0) 4400 : cluster [DBG] osdmap e2420: 8 total, 8 up, 8 in 2024-04-07T17:44:43.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:44:43 smithi100 bash[76202]: cluster 2024-04-07T17:44:42.510181+0000 mon.a (mon.0) 4399 : cluster [DBG] osdmap e2419: 8 total, 8 up, 8 in 2024-04-07T17:44:43.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:44:43 smithi100 bash[76202]: cluster 2024-04-07T17:44:42.664505+0000 mgr.y (mgr.25175) 2680 : cluster [DBG] pgmap v4674: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 1.2 MiB/s rd, 5.0 KiB/s wr, 1.65k op/s 2024-04-07T17:44:43.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:44:43 smithi100 bash[76202]: cluster 2024-04-07T17:44:43.385493+0000 mon.a (mon.0) 4400 : cluster [DBG] osdmap e2420: 8 total, 8 up, 8 in 2024-04-07T17:44:45.703 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:44:45 smithi100 bash[74942]: cluster 2024-04-07T17:44:44.384698+0000 mon.a (mon.0) 4401 : cluster [DBG] osdmap e2421: 8 total, 8 up, 8 in 2024-04-07T17:44:45.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:44:45 smithi100 bash[74942]: cluster 2024-04-07T17:44:44.665366+0000 mgr.y (mgr.25175) 2681 : cluster [DBG] pgmap v4677: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 1.2 MiB/s rd, 5.0 KiB/s wr, 1.60k op/s 2024-04-07T17:44:45.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:44:45 smithi100 bash[76202]: cluster 2024-04-07T17:44:44.384698+0000 mon.a (mon.0) 4401 : cluster [DBG] osdmap e2421: 8 total, 8 up, 8 in 2024-04-07T17:44:45.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:44:45 smithi100 bash[76202]: cluster 2024-04-07T17:44:44.665366+0000 mgr.y (mgr.25175) 2681 : cluster [DBG] pgmap v4677: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 1.2 MiB/s rd, 5.0 KiB/s wr, 1.60k op/s 2024-04-07T17:44:45.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:44:45 smithi144 bash[66654]: cluster 2024-04-07T17:44:44.384698+0000 mon.a (mon.0) 4401 : cluster [DBG] osdmap e2421: 8 total, 8 up, 8 in 2024-04-07T17:44:45.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:44:45 smithi144 bash[66654]: cluster 2024-04-07T17:44:44.665366+0000 mgr.y (mgr.25175) 2681 : cluster [DBG] pgmap v4677: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 1.2 MiB/s rd, 5.0 KiB/s wr, 1.60k op/s 2024-04-07T17:44:46.703 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:44:46 smithi100 bash[74942]: cluster 2024-04-07T17:44:45.395012+0000 mon.a (mon.0) 4402 : cluster [DBG] osdmap e2422: 8 total, 8 up, 8 in 2024-04-07T17:44:46.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:44:46 smithi100 bash[76202]: cluster 2024-04-07T17:44:45.395012+0000 mon.a (mon.0) 4402 : cluster [DBG] osdmap e2422: 8 total, 8 up, 8 in 2024-04-07T17:44:46.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:44:46 smithi144 bash[66654]: cluster 2024-04-07T17:44:45.395012+0000 mon.a (mon.0) 4402 : cluster [DBG] osdmap e2422: 8 total, 8 up, 8 in 2024-04-07T17:44:47.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:44:47 smithi100 bash[74942]: cluster 2024-04-07T17:44:46.405450+0000 mon.a (mon.0) 4403 : cluster [DBG] osdmap e2423: 8 total, 8 up, 8 in 2024-04-07T17:44:47.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:44:47 smithi100 bash[74942]: cluster 2024-04-07T17:44:46.666283+0000 mgr.y (mgr.25175) 2682 : cluster [DBG] pgmap v4680: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 1.2 MiB/s rd, 5.0 KiB/s wr, 1.59k op/s 2024-04-07T17:44:47.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:44:47 smithi100 bash[74942]: audit 2024-04-07T17:44:46.871507+0000 mon.a (mon.0) 4404 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:44:47.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:44:47 smithi100 bash[76202]: cluster 2024-04-07T17:44:46.405450+0000 mon.a (mon.0) 4403 : cluster [DBG] osdmap e2423: 8 total, 8 up, 8 in 2024-04-07T17:44:47.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:44:47 smithi100 bash[76202]: cluster 2024-04-07T17:44:46.666283+0000 mgr.y (mgr.25175) 2682 : cluster [DBG] pgmap v4680: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 1.2 MiB/s rd, 5.0 KiB/s wr, 1.59k op/s 2024-04-07T17:44:47.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:44:47 smithi100 bash[76202]: audit 2024-04-07T17:44:46.871507+0000 mon.a (mon.0) 4404 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:44:47.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:44:47 smithi144 bash[66654]: cluster 2024-04-07T17:44:46.405450+0000 mon.a (mon.0) 4403 : cluster [DBG] osdmap e2423: 8 total, 8 up, 8 in 2024-04-07T17:44:47.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:44:47 smithi144 bash[66654]: cluster 2024-04-07T17:44:46.666283+0000 mgr.y (mgr.25175) 2682 : cluster [DBG] pgmap v4680: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 1.2 MiB/s rd, 5.0 KiB/s wr, 1.59k op/s 2024-04-07T17:44:47.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:44:47 smithi144 bash[66654]: audit 2024-04-07T17:44:46.871507+0000 mon.a (mon.0) 4404 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:44:48.703 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:44:48 smithi100 bash[74942]: cluster 2024-04-07T17:44:47.417152+0000 mon.a (mon.0) 4405 : cluster [DBG] osdmap e2424: 8 total, 8 up, 8 in 2024-04-07T17:44:48.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:44:48 smithi100 bash[74942]: cluster 2024-04-07T17:44:48.387041+0000 mon.a (mon.0) 4406 : cluster [DBG] osdmap e2425: 8 total, 8 up, 8 in 2024-04-07T17:44:48.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:44:48 smithi100 bash[76202]: cluster 2024-04-07T17:44:47.417152+0000 mon.a (mon.0) 4405 : cluster [DBG] osdmap e2424: 8 total, 8 up, 8 in 2024-04-07T17:44:48.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:44:48 smithi100 bash[76202]: cluster 2024-04-07T17:44:48.387041+0000 mon.a (mon.0) 4406 : cluster [DBG] osdmap e2425: 8 total, 8 up, 8 in 2024-04-07T17:44:48.704 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:44:48 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:44:48] "GET /metrics HTTP/1.1" 200 36745 "" "Prometheus/2.43.0" 2024-04-07T17:44:48.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:44:48 smithi144 bash[66654]: cluster 2024-04-07T17:44:47.417152+0000 mon.a (mon.0) 4405 : cluster [DBG] osdmap e2424: 8 total, 8 up, 8 in 2024-04-07T17:44:48.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:44:48 smithi144 bash[66654]: cluster 2024-04-07T17:44:48.387041+0000 mon.a (mon.0) 4406 : cluster [DBG] osdmap e2425: 8 total, 8 up, 8 in 2024-04-07T17:44:49.703 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:44:49 smithi100 bash[74942]: cluster 2024-04-07T17:44:48.667033+0000 mgr.y (mgr.25175) 2683 : cluster [DBG] pgmap v4683: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 2.7 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-07T17:44:49.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:44:49 smithi100 bash[76202]: cluster 2024-04-07T17:44:48.667033+0000 mgr.y (mgr.25175) 2683 : cluster [DBG] pgmap v4683: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 2.7 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-07T17:44:49.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:44:49 smithi144 bash[66654]: cluster 2024-04-07T17:44:48.667033+0000 mgr.y (mgr.25175) 2683 : cluster [DBG] pgmap v4683: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 2.7 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-07T17:44:50.703 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:44:50 smithi100 bash[74942]: cluster 2024-04-07T17:44:49.430028+0000 mon.a (mon.0) 4407 : cluster [DBG] osdmap e2426: 8 total, 8 up, 8 in 2024-04-07T17:44:50.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:44:50 smithi100 bash[76202]: cluster 2024-04-07T17:44:49.430028+0000 mon.a (mon.0) 4407 : cluster [DBG] osdmap e2426: 8 total, 8 up, 8 in 2024-04-07T17:44:50.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:44:50 smithi144 bash[66654]: cluster 2024-04-07T17:44:49.430028+0000 mon.a (mon.0) 4407 : cluster [DBG] osdmap e2426: 8 total, 8 up, 8 in 2024-04-07T17:44:51.703 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:44:51 smithi100 bash[74942]: cluster 2024-04-07T17:44:50.441710+0000 mon.a (mon.0) 4408 : cluster [DBG] osdmap e2427: 8 total, 8 up, 8 in 2024-04-07T17:44:51.703 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:44:51 smithi100 bash[74942]: cluster 2024-04-07T17:44:50.667773+0000 mgr.y (mgr.25175) 2684 : cluster [DBG] pgmap v4686: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 1.1 MiB/s rd, 5.0 KiB/s wr, 1.56k op/s 2024-04-07T17:44:51.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:44:51 smithi100 bash[76202]: cluster 2024-04-07T17:44:50.441710+0000 mon.a (mon.0) 4408 : cluster [DBG] osdmap e2427: 8 total, 8 up, 8 in 2024-04-07T17:44:51.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:44:51 smithi100 bash[76202]: cluster 2024-04-07T17:44:50.667773+0000 mgr.y (mgr.25175) 2684 : cluster [DBG] pgmap v4686: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 1.1 MiB/s rd, 5.0 KiB/s wr, 1.56k op/s 2024-04-07T17:44:51.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:44:51 smithi144 bash[66654]: cluster 2024-04-07T17:44:50.441710+0000 mon.a (mon.0) 4408 : cluster [DBG] osdmap e2427: 8 total, 8 up, 8 in 2024-04-07T17:44:51.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:44:51 smithi144 bash[66654]: cluster 2024-04-07T17:44:50.667773+0000 mgr.y (mgr.25175) 2684 : cluster [DBG] pgmap v4686: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 1.1 MiB/s rd, 5.0 KiB/s wr, 1.56k op/s 2024-04-07T17:44:52.614 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:44:52 smithi144 bash[66654]: cluster 2024-04-07T17:44:51.453659+0000 mon.a (mon.0) 4409 : cluster [DBG] osdmap e2428: 8 total, 8 up, 8 in 2024-04-07T17:44:52.615 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:44:52 smithi144 bash[95233]: ts=2024-04-07T17:44:52.317Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:44:52.703 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:44:52 smithi100 bash[74942]: cluster 2024-04-07T17:44:51.453659+0000 mon.a (mon.0) 4409 : cluster [DBG] osdmap e2428: 8 total, 8 up, 8 in 2024-04-07T17:44:52.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:44:52 smithi100 bash[76202]: cluster 2024-04-07T17:44:51.453659+0000 mon.a (mon.0) 4409 : cluster [DBG] osdmap e2428: 8 total, 8 up, 8 in 2024-04-07T17:44:53.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:44:53 smithi144 bash[66654]: cluster 2024-04-07T17:44:52.466782+0000 mon.a (mon.0) 4410 : cluster [DBG] osdmap e2429: 8 total, 8 up, 8 in 2024-04-07T17:44:53.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:44:53 smithi144 bash[66654]: cluster 2024-04-07T17:44:52.668599+0000 mgr.y (mgr.25175) 2685 : cluster [DBG] pgmap v4689: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 1.1 MiB/s rd, 5.0 KiB/s wr, 1.56k op/s 2024-04-07T17:44:53.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:44:53 smithi144 bash[66654]: cluster 2024-04-07T17:44:53.387350+0000 mon.a (mon.0) 4411 : cluster [DBG] osdmap e2430: 8 total, 8 up, 8 in 2024-04-07T17:44:53.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:44:53 smithi100 bash[74942]: cluster 2024-04-07T17:44:52.466782+0000 mon.a (mon.0) 4410 : cluster [DBG] osdmap e2429: 8 total, 8 up, 8 in 2024-04-07T17:44:53.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:44:53 smithi100 bash[74942]: cluster 2024-04-07T17:44:52.668599+0000 mgr.y (mgr.25175) 2685 : cluster [DBG] pgmap v4689: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 1.1 MiB/s rd, 5.0 KiB/s wr, 1.56k op/s 2024-04-07T17:44:53.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:44:53 smithi100 bash[74942]: cluster 2024-04-07T17:44:53.387350+0000 mon.a (mon.0) 4411 : cluster [DBG] osdmap e2430: 8 total, 8 up, 8 in 2024-04-07T17:44:53.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:44:53 smithi100 bash[76202]: cluster 2024-04-07T17:44:52.466782+0000 mon.a (mon.0) 4410 : cluster [DBG] osdmap e2429: 8 total, 8 up, 8 in 2024-04-07T17:44:53.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:44:53 smithi100 bash[76202]: cluster 2024-04-07T17:44:52.668599+0000 mgr.y (mgr.25175) 2685 : cluster [DBG] pgmap v4689: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 1.1 MiB/s rd, 5.0 KiB/s wr, 1.56k op/s 2024-04-07T17:44:53.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:44:53 smithi100 bash[76202]: cluster 2024-04-07T17:44:53.387350+0000 mon.a (mon.0) 4411 : cluster [DBG] osdmap e2430: 8 total, 8 up, 8 in 2024-04-07T17:44:55.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:44:55 smithi144 bash[66654]: cluster 2024-04-07T17:44:54.478682+0000 mon.a (mon.0) 4412 : cluster [DBG] osdmap e2431: 8 total, 8 up, 8 in 2024-04-07T17:44:55.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:44:55 smithi144 bash[66654]: cluster 2024-04-07T17:44:54.669410+0000 mgr.y (mgr.25175) 2686 : cluster [DBG] pgmap v4692: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 1.1 MiB/s rd, 5.0 KiB/s wr, 1.51k op/s 2024-04-07T17:44:55.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:44:55 smithi100 bash[74942]: cluster 2024-04-07T17:44:54.478682+0000 mon.a (mon.0) 4412 : cluster [DBG] osdmap e2431: 8 total, 8 up, 8 in 2024-04-07T17:44:55.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:44:55 smithi100 bash[74942]: cluster 2024-04-07T17:44:54.669410+0000 mgr.y (mgr.25175) 2686 : cluster [DBG] pgmap v4692: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 1.1 MiB/s rd, 5.0 KiB/s wr, 1.51k op/s 2024-04-07T17:44:55.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:44:55 smithi100 bash[76202]: cluster 2024-04-07T17:44:54.478682+0000 mon.a (mon.0) 4412 : cluster [DBG] osdmap e2431: 8 total, 8 up, 8 in 2024-04-07T17:44:55.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:44:55 smithi100 bash[76202]: cluster 2024-04-07T17:44:54.669410+0000 mgr.y (mgr.25175) 2686 : cluster [DBG] pgmap v4692: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 1.1 MiB/s rd, 5.0 KiB/s wr, 1.51k op/s 2024-04-07T17:44:56.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:44:56 smithi144 bash[66654]: cluster 2024-04-07T17:44:55.487957+0000 mon.a (mon.0) 4413 : cluster [DBG] osdmap e2432: 8 total, 8 up, 8 in 2024-04-07T17:44:56.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:44:56 smithi100 bash[74942]: cluster 2024-04-07T17:44:55.487957+0000 mon.a (mon.0) 4413 : cluster [DBG] osdmap e2432: 8 total, 8 up, 8 in 2024-04-07T17:44:56.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:44:56 smithi100 bash[76202]: cluster 2024-04-07T17:44:55.487957+0000 mon.a (mon.0) 4413 : cluster [DBG] osdmap e2432: 8 total, 8 up, 8 in 2024-04-07T17:44:57.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:44:57 smithi144 bash[66654]: cluster 2024-04-07T17:44:56.496223+0000 mon.a (mon.0) 4414 : cluster [DBG] osdmap e2433: 8 total, 8 up, 8 in 2024-04-07T17:44:57.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:44:57 smithi144 bash[66654]: cluster 2024-04-07T17:44:56.670169+0000 mgr.y (mgr.25175) 2687 : cluster [DBG] pgmap v4695: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 1.1 MiB/s rd, 5.0 KiB/s wr, 1.51k op/s 2024-04-07T17:44:57.953 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:44:57 smithi100 bash[76202]: cluster 2024-04-07T17:44:56.496223+0000 mon.a (mon.0) 4414 : cluster [DBG] osdmap e2433: 8 total, 8 up, 8 in 2024-04-07T17:44:57.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:44:57 smithi100 bash[76202]: cluster 2024-04-07T17:44:56.670169+0000 mgr.y (mgr.25175) 2687 : cluster [DBG] pgmap v4695: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 1.1 MiB/s rd, 5.0 KiB/s wr, 1.51k op/s 2024-04-07T17:44:57.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:44:57 smithi100 bash[74942]: cluster 2024-04-07T17:44:56.496223+0000 mon.a (mon.0) 4414 : cluster [DBG] osdmap e2433: 8 total, 8 up, 8 in 2024-04-07T17:44:57.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:44:57 smithi100 bash[74942]: cluster 2024-04-07T17:44:56.670169+0000 mgr.y (mgr.25175) 2687 : cluster [DBG] pgmap v4695: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 1.1 MiB/s rd, 5.0 KiB/s wr, 1.51k op/s 2024-04-07T17:44:58.703 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:44:58 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:44:58] "GET /metrics HTTP/1.1" 200 36745 "" "Prometheus/2.43.0" 2024-04-07T17:44:58.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:44:58 smithi100 bash[74942]: cluster 2024-04-07T17:44:57.512871+0000 mon.a (mon.0) 4415 : cluster [DBG] osdmap e2434: 8 total, 8 up, 8 in 2024-04-07T17:44:58.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:44:58 smithi100 bash[74942]: cluster 2024-04-07T17:44:58.394810+0000 mon.a (mon.0) 4416 : cluster [DBG] osdmap e2435: 8 total, 8 up, 8 in 2024-04-07T17:44:58.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:44:58 smithi100 bash[76202]: cluster 2024-04-07T17:44:57.512871+0000 mon.a (mon.0) 4415 : cluster [DBG] osdmap e2434: 8 total, 8 up, 8 in 2024-04-07T17:44:58.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:44:58 smithi100 bash[76202]: cluster 2024-04-07T17:44:58.394810+0000 mon.a (mon.0) 4416 : cluster [DBG] osdmap e2435: 8 total, 8 up, 8 in 2024-04-07T17:44:58.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:44:58 smithi144 bash[66654]: cluster 2024-04-07T17:44:57.512871+0000 mon.a (mon.0) 4415 : cluster [DBG] osdmap e2434: 8 total, 8 up, 8 in 2024-04-07T17:44:58.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:44:58 smithi144 bash[66654]: cluster 2024-04-07T17:44:58.394810+0000 mon.a (mon.0) 4416 : cluster [DBG] osdmap e2435: 8 total, 8 up, 8 in 2024-04-07T17:44:59.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:44:59 smithi144 bash[66654]: cluster 2024-04-07T17:44:58.670932+0000 mgr.y (mgr.25175) 2688 : cluster [DBG] pgmap v4698: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 3.2 KiB/s rd, 0 B/s wr, 6 op/s 2024-04-07T17:44:59.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:44:59 smithi100 bash[74942]: cluster 2024-04-07T17:44:58.670932+0000 mgr.y (mgr.25175) 2688 : cluster [DBG] pgmap v4698: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 3.2 KiB/s rd, 0 B/s wr, 6 op/s 2024-04-07T17:44:59.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:44:59 smithi100 bash[76202]: cluster 2024-04-07T17:44:58.670932+0000 mgr.y (mgr.25175) 2688 : cluster [DBG] pgmap v4698: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 3.2 KiB/s rd, 0 B/s wr, 6 op/s 2024-04-07T17:45:00.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:45:00 smithi144 bash[66654]: cluster 2024-04-07T17:44:59.520119+0000 mon.a (mon.0) 4417 : cluster [DBG] osdmap e2436: 8 total, 8 up, 8 in 2024-04-07T17:45:00.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:45:00 smithi100 bash[74942]: cluster 2024-04-07T17:44:59.520119+0000 mon.a (mon.0) 4417 : cluster [DBG] osdmap e2436: 8 total, 8 up, 8 in 2024-04-07T17:45:00.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:45:00 smithi100 bash[76202]: cluster 2024-04-07T17:44:59.520119+0000 mon.a (mon.0) 4417 : cluster [DBG] osdmap e2436: 8 total, 8 up, 8 in 2024-04-07T17:45:01.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:45:01 smithi144 bash[66654]: cluster 2024-04-07T17:45:00.527887+0000 mon.a (mon.0) 4418 : cluster [DBG] osdmap e2437: 8 total, 8 up, 8 in 2024-04-07T17:45:01.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:45:01 smithi144 bash[66654]: cluster 2024-04-07T17:45:00.671671+0000 mgr.y (mgr.25175) 2689 : cluster [DBG] pgmap v4701: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 1.1 MiB/s rd, 5.0 KiB/s wr, 1.47k op/s 2024-04-07T17:45:01.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:45:01 smithi100 bash[74942]: cluster 2024-04-07T17:45:00.527887+0000 mon.a (mon.0) 4418 : cluster [DBG] osdmap e2437: 8 total, 8 up, 8 in 2024-04-07T17:45:01.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:45:01 smithi100 bash[74942]: cluster 2024-04-07T17:45:00.671671+0000 mgr.y (mgr.25175) 2689 : cluster [DBG] pgmap v4701: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 1.1 MiB/s rd, 5.0 KiB/s wr, 1.47k op/s 2024-04-07T17:45:01.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:45:01 smithi100 bash[76202]: cluster 2024-04-07T17:45:00.527887+0000 mon.a (mon.0) 4418 : cluster [DBG] osdmap e2437: 8 total, 8 up, 8 in 2024-04-07T17:45:01.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:45:01 smithi100 bash[76202]: cluster 2024-04-07T17:45:00.671671+0000 mgr.y (mgr.25175) 2689 : cluster [DBG] pgmap v4701: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 1.1 MiB/s rd, 5.0 KiB/s wr, 1.47k op/s 2024-04-07T17:45:02.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:45:02 smithi144 bash[66654]: cluster 2024-04-07T17:45:01.538229+0000 mon.a (mon.0) 4419 : cluster [DBG] osdmap e2438: 8 total, 8 up, 8 in 2024-04-07T17:45:02.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:45:02 smithi144 bash[66654]: audit 2024-04-07T17:45:01.871900+0000 mon.a (mon.0) 4420 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:45:02.615 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:45:02 smithi144 bash[95233]: ts=2024-04-07T17:45:02.317Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:45:02.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:45:02 smithi100 bash[74942]: cluster 2024-04-07T17:45:01.538229+0000 mon.a (mon.0) 4419 : cluster [DBG] osdmap e2438: 8 total, 8 up, 8 in 2024-04-07T17:45:02.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:45:02 smithi100 bash[74942]: audit 2024-04-07T17:45:01.871900+0000 mon.a (mon.0) 4420 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:45:02.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:45:02 smithi100 bash[76202]: cluster 2024-04-07T17:45:01.538229+0000 mon.a (mon.0) 4419 : cluster [DBG] osdmap e2438: 8 total, 8 up, 8 in 2024-04-07T17:45:02.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:45:02 smithi100 bash[76202]: audit 2024-04-07T17:45:01.871900+0000 mon.a (mon.0) 4420 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:45:03.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:45:03 smithi144 bash[66654]: cluster 2024-04-07T17:45:02.554491+0000 mon.a (mon.0) 4421 : cluster [DBG] osdmap e2439: 8 total, 8 up, 8 in 2024-04-07T17:45:03.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:45:03 smithi144 bash[66654]: cluster 2024-04-07T17:45:02.672622+0000 mgr.y (mgr.25175) 2690 : cluster [DBG] pgmap v4704: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 1.1 MiB/s rd, 5.0 KiB/s wr, 1.47k op/s 2024-04-07T17:45:03.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:45:03 smithi144 bash[66654]: cluster 2024-04-07T17:45:03.391682+0000 mon.a (mon.0) 4422 : cluster [DBG] osdmap e2440: 8 total, 8 up, 8 in 2024-04-07T17:45:03.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:45:03 smithi100 bash[74942]: cluster 2024-04-07T17:45:02.554491+0000 mon.a (mon.0) 4421 : cluster [DBG] osdmap e2439: 8 total, 8 up, 8 in 2024-04-07T17:45:03.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:45:03 smithi100 bash[74942]: cluster 2024-04-07T17:45:02.672622+0000 mgr.y (mgr.25175) 2690 : cluster [DBG] pgmap v4704: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 1.1 MiB/s rd, 5.0 KiB/s wr, 1.47k op/s 2024-04-07T17:45:03.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:45:03 smithi100 bash[74942]: cluster 2024-04-07T17:45:03.391682+0000 mon.a (mon.0) 4422 : cluster [DBG] osdmap e2440: 8 total, 8 up, 8 in 2024-04-07T17:45:03.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:45:03 smithi100 bash[76202]: cluster 2024-04-07T17:45:02.554491+0000 mon.a (mon.0) 4421 : cluster [DBG] osdmap e2439: 8 total, 8 up, 8 in 2024-04-07T17:45:03.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:45:03 smithi100 bash[76202]: cluster 2024-04-07T17:45:02.672622+0000 mgr.y (mgr.25175) 2690 : cluster [DBG] pgmap v4704: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 1.1 MiB/s rd, 5.0 KiB/s wr, 1.47k op/s 2024-04-07T17:45:03.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:45:03 smithi100 bash[76202]: cluster 2024-04-07T17:45:03.391682+0000 mon.a (mon.0) 4422 : cluster [DBG] osdmap e2440: 8 total, 8 up, 8 in 2024-04-07T17:45:05.703 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:45:05 smithi100 bash[76202]: cluster 2024-04-07T17:45:04.394634+0000 mon.a (mon.0) 4423 : cluster [DBG] osdmap e2441: 8 total, 8 up, 8 in 2024-04-07T17:45:05.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:45:05 smithi100 bash[76202]: cluster 2024-04-07T17:45:04.673434+0000 mgr.y (mgr.25175) 2691 : cluster [DBG] pgmap v4707: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 1.0 MiB/s rd, 5.0 KiB/s wr, 1.43k op/s 2024-04-07T17:45:05.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:45:05 smithi100 bash[74942]: cluster 2024-04-07T17:45:04.394634+0000 mon.a (mon.0) 4423 : cluster [DBG] osdmap e2441: 8 total, 8 up, 8 in 2024-04-07T17:45:05.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:45:05 smithi100 bash[74942]: cluster 2024-04-07T17:45:04.673434+0000 mgr.y (mgr.25175) 2691 : cluster [DBG] pgmap v4707: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 1.0 MiB/s rd, 5.0 KiB/s wr, 1.43k op/s 2024-04-07T17:45:05.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:45:05 smithi144 bash[66654]: cluster 2024-04-07T17:45:04.394634+0000 mon.a (mon.0) 4423 : cluster [DBG] osdmap e2441: 8 total, 8 up, 8 in 2024-04-07T17:45:05.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:45:05 smithi144 bash[66654]: cluster 2024-04-07T17:45:04.673434+0000 mgr.y (mgr.25175) 2691 : cluster [DBG] pgmap v4707: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 1.0 MiB/s rd, 5.0 KiB/s wr, 1.43k op/s 2024-04-07T17:45:06.703 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:45:06 smithi100 bash[74942]: cluster 2024-04-07T17:45:05.404917+0000 mon.a (mon.0) 4424 : cluster [DBG] osdmap e2442: 8 total, 8 up, 8 in 2024-04-07T17:45:06.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:45:06 smithi100 bash[76202]: cluster 2024-04-07T17:45:05.404917+0000 mon.a (mon.0) 4424 : cluster [DBG] osdmap e2442: 8 total, 8 up, 8 in 2024-04-07T17:45:06.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:45:06 smithi144 bash[66654]: cluster 2024-04-07T17:45:05.404917+0000 mon.a (mon.0) 4424 : cluster [DBG] osdmap e2442: 8 total, 8 up, 8 in 2024-04-07T17:45:07.703 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:45:07 smithi100 bash[76202]: cluster 2024-04-07T17:45:06.405908+0000 mon.a (mon.0) 4425 : cluster [DBG] osdmap e2443: 8 total, 8 up, 8 in 2024-04-07T17:45:07.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:45:07 smithi100 bash[76202]: cluster 2024-04-07T17:45:06.674220+0000 mgr.y (mgr.25175) 2692 : cluster [DBG] pgmap v4710: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 1.0 MiB/s rd, 5.0 KiB/s wr, 1.42k op/s 2024-04-07T17:45:07.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:45:07 smithi100 bash[74942]: cluster 2024-04-07T17:45:06.405908+0000 mon.a (mon.0) 4425 : cluster [DBG] osdmap e2443: 8 total, 8 up, 8 in 2024-04-07T17:45:07.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:45:07 smithi100 bash[74942]: cluster 2024-04-07T17:45:06.674220+0000 mgr.y (mgr.25175) 2692 : cluster [DBG] pgmap v4710: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 1.0 MiB/s rd, 5.0 KiB/s wr, 1.42k op/s 2024-04-07T17:45:07.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:45:07 smithi144 bash[66654]: cluster 2024-04-07T17:45:06.405908+0000 mon.a (mon.0) 4425 : cluster [DBG] osdmap e2443: 8 total, 8 up, 8 in 2024-04-07T17:45:07.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:45:07 smithi144 bash[66654]: cluster 2024-04-07T17:45:06.674220+0000 mgr.y (mgr.25175) 2692 : cluster [DBG] pgmap v4710: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 1.0 MiB/s rd, 5.0 KiB/s wr, 1.42k op/s 2024-04-07T17:45:08.703 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:45:08 smithi100 bash[74942]: cluster 2024-04-07T17:45:07.431279+0000 mon.a (mon.0) 4426 : cluster [DBG] osdmap e2444: 8 total, 8 up, 8 in 2024-04-07T17:45:08.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:45:08 smithi100 bash[74942]: cluster 2024-04-07T17:45:08.398395+0000 mon.a (mon.0) 4427 : cluster [DBG] osdmap e2445: 8 total, 8 up, 8 in 2024-04-07T17:45:08.704 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:45:08 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:45:08] "GET /metrics HTTP/1.1" 200 36747 "" "Prometheus/2.43.0" 2024-04-07T17:45:08.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:45:08 smithi100 bash[76202]: cluster 2024-04-07T17:45:07.431279+0000 mon.a (mon.0) 4426 : cluster [DBG] osdmap e2444: 8 total, 8 up, 8 in 2024-04-07T17:45:08.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:45:08 smithi100 bash[76202]: cluster 2024-04-07T17:45:08.398395+0000 mon.a (mon.0) 4427 : cluster [DBG] osdmap e2445: 8 total, 8 up, 8 in 2024-04-07T17:45:08.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:45:08 smithi144 bash[66654]: cluster 2024-04-07T17:45:07.431279+0000 mon.a (mon.0) 4426 : cluster [DBG] osdmap e2444: 8 total, 8 up, 8 in 2024-04-07T17:45:08.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:45:08 smithi144 bash[66654]: cluster 2024-04-07T17:45:08.398395+0000 mon.a (mon.0) 4427 : cluster [DBG] osdmap e2445: 8 total, 8 up, 8 in 2024-04-07T17:45:09.703 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:45:09 smithi100 bash[74942]: cluster 2024-04-07T17:45:08.674899+0000 mgr.y (mgr.25175) 2693 : cluster [DBG] pgmap v4713: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 3.2 KiB/s rd, 0 B/s wr, 6 op/s 2024-04-07T17:45:09.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:45:09 smithi100 bash[76202]: cluster 2024-04-07T17:45:08.674899+0000 mgr.y (mgr.25175) 2693 : cluster [DBG] pgmap v4713: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 3.2 KiB/s rd, 0 B/s wr, 6 op/s 2024-04-07T17:45:09.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:45:09 smithi144 bash[66654]: cluster 2024-04-07T17:45:08.674899+0000 mgr.y (mgr.25175) 2693 : cluster [DBG] pgmap v4713: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 3.2 KiB/s rd, 0 B/s wr, 6 op/s 2024-04-07T17:45:10.703 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:45:10 smithi100 bash[74942]: cluster 2024-04-07T17:45:09.447267+0000 mon.a (mon.0) 4428 : cluster [DBG] osdmap e2446: 8 total, 8 up, 8 in 2024-04-07T17:45:10.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:45:10 smithi100 bash[76202]: cluster 2024-04-07T17:45:09.447267+0000 mon.a (mon.0) 4428 : cluster [DBG] osdmap e2446: 8 total, 8 up, 8 in 2024-04-07T17:45:10.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:45:10 smithi144 bash[66654]: cluster 2024-04-07T17:45:09.447267+0000 mon.a (mon.0) 4428 : cluster [DBG] osdmap e2446: 8 total, 8 up, 8 in 2024-04-07T17:45:11.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:45:11 smithi144 bash[66654]: cluster 2024-04-07T17:45:10.466605+0000 mon.a (mon.0) 4429 : cluster [DBG] osdmap e2447: 8 total, 8 up, 8 in 2024-04-07T17:45:11.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:45:11 smithi144 bash[66654]: cluster 2024-04-07T17:45:10.675686+0000 mgr.y (mgr.25175) 2694 : cluster [DBG] pgmap v4716: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 1022 KiB/s rd, 5.0 KiB/s wr, 1.38k op/s 2024-04-07T17:45:11.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:45:11 smithi100 bash[74942]: cluster 2024-04-07T17:45:10.466605+0000 mon.a (mon.0) 4429 : cluster [DBG] osdmap e2447: 8 total, 8 up, 8 in 2024-04-07T17:45:11.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:45:11 smithi100 bash[74942]: cluster 2024-04-07T17:45:10.675686+0000 mgr.y (mgr.25175) 2694 : cluster [DBG] pgmap v4716: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 1022 KiB/s rd, 5.0 KiB/s wr, 1.38k op/s 2024-04-07T17:45:11.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:45:11 smithi100 bash[76202]: cluster 2024-04-07T17:45:10.466605+0000 mon.a (mon.0) 4429 : cluster [DBG] osdmap e2447: 8 total, 8 up, 8 in 2024-04-07T17:45:11.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:45:11 smithi100 bash[76202]: cluster 2024-04-07T17:45:10.675686+0000 mgr.y (mgr.25175) 2694 : cluster [DBG] pgmap v4716: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 1022 KiB/s rd, 5.0 KiB/s wr, 1.38k op/s 2024-04-07T17:45:12.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:45:12 smithi144 bash[66654]: cluster 2024-04-07T17:45:11.483024+0000 mon.a (mon.0) 4430 : cluster [DBG] osdmap e2448: 8 total, 8 up, 8 in 2024-04-07T17:45:12.615 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:45:12 smithi144 bash[95233]: ts=2024-04-07T17:45:12.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:45:12.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:45:12 smithi100 bash[74942]: cluster 2024-04-07T17:45:11.483024+0000 mon.a (mon.0) 4430 : cluster [DBG] osdmap e2448: 8 total, 8 up, 8 in 2024-04-07T17:45:12.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:45:12 smithi100 bash[76202]: cluster 2024-04-07T17:45:11.483024+0000 mon.a (mon.0) 4430 : cluster [DBG] osdmap e2448: 8 total, 8 up, 8 in 2024-04-07T17:45:13.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:45:13 smithi144 bash[66654]: cluster 2024-04-07T17:45:12.485088+0000 mon.a (mon.0) 4431 : cluster [DBG] osdmap e2449: 8 total, 8 up, 8 in 2024-04-07T17:45:13.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:45:13 smithi144 bash[66654]: cluster 2024-04-07T17:45:12.676374+0000 mgr.y (mgr.25175) 2695 : cluster [DBG] pgmap v4719: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 1022 KiB/s rd, 5.0 KiB/s wr, 1.38k op/s 2024-04-07T17:45:13.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:45:13 smithi144 bash[66654]: cluster 2024-04-07T17:45:13.396665+0000 mon.a (mon.0) 4432 : cluster [DBG] osdmap e2450: 8 total, 8 up, 8 in 2024-04-07T17:45:13.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:45:13 smithi100 bash[74942]: cluster 2024-04-07T17:45:12.485088+0000 mon.a (mon.0) 4431 : cluster [DBG] osdmap e2449: 8 total, 8 up, 8 in 2024-04-07T17:45:13.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:45:13 smithi100 bash[74942]: cluster 2024-04-07T17:45:12.676374+0000 mgr.y (mgr.25175) 2695 : cluster [DBG] pgmap v4719: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 1022 KiB/s rd, 5.0 KiB/s wr, 1.38k op/s 2024-04-07T17:45:13.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:45:13 smithi100 bash[74942]: cluster 2024-04-07T17:45:13.396665+0000 mon.a (mon.0) 4432 : cluster [DBG] osdmap e2450: 8 total, 8 up, 8 in 2024-04-07T17:45:13.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:45:13 smithi100 bash[76202]: cluster 2024-04-07T17:45:12.485088+0000 mon.a (mon.0) 4431 : cluster [DBG] osdmap e2449: 8 total, 8 up, 8 in 2024-04-07T17:45:13.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:45:13 smithi100 bash[76202]: cluster 2024-04-07T17:45:12.676374+0000 mgr.y (mgr.25175) 2695 : cluster [DBG] pgmap v4719: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 1022 KiB/s rd, 5.0 KiB/s wr, 1.38k op/s 2024-04-07T17:45:13.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:45:13 smithi100 bash[76202]: cluster 2024-04-07T17:45:13.396665+0000 mon.a (mon.0) 4432 : cluster [DBG] osdmap e2450: 8 total, 8 up, 8 in 2024-04-07T17:45:15.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:45:15 smithi144 bash[66654]: cluster 2024-04-07T17:45:14.495654+0000 mon.a (mon.0) 4433 : cluster [DBG] osdmap e2451: 8 total, 8 up, 8 in 2024-04-07T17:45:15.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:45:15 smithi144 bash[66654]: cluster 2024-04-07T17:45:14.677082+0000 mgr.y (mgr.25175) 2696 : cluster [DBG] pgmap v4722: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 988 KiB/s rd, 5.0 KiB/s wr, 1.34k op/s 2024-04-07T17:45:15.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:45:15 smithi100 bash[74942]: cluster 2024-04-07T17:45:14.495654+0000 mon.a (mon.0) 4433 : cluster [DBG] osdmap e2451: 8 total, 8 up, 8 in 2024-04-07T17:45:15.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:45:15 smithi100 bash[74942]: cluster 2024-04-07T17:45:14.677082+0000 mgr.y (mgr.25175) 2696 : cluster [DBG] pgmap v4722: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 988 KiB/s rd, 5.0 KiB/s wr, 1.34k op/s 2024-04-07T17:45:15.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:45:15 smithi100 bash[76202]: cluster 2024-04-07T17:45:14.495654+0000 mon.a (mon.0) 4433 : cluster [DBG] osdmap e2451: 8 total, 8 up, 8 in 2024-04-07T17:45:15.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:45:15 smithi100 bash[76202]: cluster 2024-04-07T17:45:14.677082+0000 mgr.y (mgr.25175) 2696 : cluster [DBG] pgmap v4722: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 988 KiB/s rd, 5.0 KiB/s wr, 1.34k op/s 2024-04-07T17:45:16.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:45:16 smithi144 bash[66654]: cluster 2024-04-07T17:45:15.514948+0000 mon.a (mon.0) 4434 : cluster [DBG] osdmap e2452: 8 total, 8 up, 8 in 2024-04-07T17:45:16.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:45:16 smithi100 bash[74942]: cluster 2024-04-07T17:45:15.514948+0000 mon.a (mon.0) 4434 : cluster [DBG] osdmap e2452: 8 total, 8 up, 8 in 2024-04-07T17:45:16.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:45:16 smithi100 bash[76202]: cluster 2024-04-07T17:45:15.514948+0000 mon.a (mon.0) 4434 : cluster [DBG] osdmap e2452: 8 total, 8 up, 8 in 2024-04-07T17:45:17.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:45:17 smithi144 bash[66654]: cluster 2024-04-07T17:45:16.519344+0000 mon.a (mon.0) 4435 : cluster [DBG] osdmap e2453: 8 total, 8 up, 8 in 2024-04-07T17:45:17.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:45:17 smithi144 bash[66654]: cluster 2024-04-07T17:45:16.678053+0000 mgr.y (mgr.25175) 2697 : cluster [DBG] pgmap v4725: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 986 KiB/s rd, 5.0 KiB/s wr, 1.33k op/s 2024-04-07T17:45:17.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:45:17 smithi144 bash[66654]: audit 2024-04-07T17:45:16.874609+0000 mon.a (mon.0) 4436 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:45:17.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:45:17 smithi100 bash[74942]: cluster 2024-04-07T17:45:16.519344+0000 mon.a (mon.0) 4435 : cluster [DBG] osdmap e2453: 8 total, 8 up, 8 in 2024-04-07T17:45:17.955 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:45:17 smithi100 bash[74942]: cluster 2024-04-07T17:45:16.678053+0000 mgr.y (mgr.25175) 2697 : cluster [DBG] pgmap v4725: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 986 KiB/s rd, 5.0 KiB/s wr, 1.33k op/s 2024-04-07T17:45:17.955 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:45:17 smithi100 bash[74942]: audit 2024-04-07T17:45:16.874609+0000 mon.a (mon.0) 4436 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:45:17.955 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:45:17 smithi100 bash[76202]: cluster 2024-04-07T17:45:16.519344+0000 mon.a (mon.0) 4435 : cluster [DBG] osdmap e2453: 8 total, 8 up, 8 in 2024-04-07T17:45:17.955 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:45:17 smithi100 bash[76202]: cluster 2024-04-07T17:45:16.678053+0000 mgr.y (mgr.25175) 2697 : cluster [DBG] pgmap v4725: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 986 KiB/s rd, 5.0 KiB/s wr, 1.33k op/s 2024-04-07T17:45:17.955 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:45:17 smithi100 bash[76202]: audit 2024-04-07T17:45:16.874609+0000 mon.a (mon.0) 4436 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:45:18.703 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:45:18 smithi100 bash[74942]: cluster 2024-04-07T17:45:17.532118+0000 mon.a (mon.0) 4437 : cluster [DBG] osdmap e2454: 8 total, 8 up, 8 in 2024-04-07T17:45:18.703 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:45:18 smithi100 bash[74942]: cluster 2024-04-07T17:45:18.399183+0000 mon.a (mon.0) 4438 : cluster [DBG] osdmap e2455: 8 total, 8 up, 8 in 2024-04-07T17:45:18.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:45:18 smithi100 bash[76202]: cluster 2024-04-07T17:45:17.532118+0000 mon.a (mon.0) 4437 : cluster [DBG] osdmap e2454: 8 total, 8 up, 8 in 2024-04-07T17:45:18.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:45:18 smithi100 bash[76202]: cluster 2024-04-07T17:45:18.399183+0000 mon.a (mon.0) 4438 : cluster [DBG] osdmap e2455: 8 total, 8 up, 8 in 2024-04-07T17:45:18.704 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:45:18 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:45:18] "GET /metrics HTTP/1.1" 200 36746 "" "Prometheus/2.43.0" 2024-04-07T17:45:18.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:45:18 smithi144 bash[66654]: cluster 2024-04-07T17:45:17.532118+0000 mon.a (mon.0) 4437 : cluster [DBG] osdmap e2454: 8 total, 8 up, 8 in 2024-04-07T17:45:18.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:45:18 smithi144 bash[66654]: cluster 2024-04-07T17:45:18.399183+0000 mon.a (mon.0) 4438 : cluster [DBG] osdmap e2455: 8 total, 8 up, 8 in 2024-04-07T17:45:19.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:45:19 smithi144 bash[66654]: cluster 2024-04-07T17:45:18.678760+0000 mgr.y (mgr.25175) 2698 : cluster [DBG] pgmap v4728: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 2.5 KiB/s rd, 0 B/s wr, 4 op/s 2024-04-07T17:45:19.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:45:19 smithi144 bash[66654]: cluster 2024-04-07T17:45:19.401965+0000 mon.a (mon.0) 4439 : cluster [DBG] osdmap e2456: 8 total, 8 up, 8 in 2024-04-07T17:45:19.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:45:19 smithi100 bash[74942]: cluster 2024-04-07T17:45:18.678760+0000 mgr.y (mgr.25175) 2698 : cluster [DBG] pgmap v4728: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 2.5 KiB/s rd, 0 B/s wr, 4 op/s 2024-04-07T17:45:19.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:45:19 smithi100 bash[74942]: cluster 2024-04-07T17:45:19.401965+0000 mon.a (mon.0) 4439 : cluster [DBG] osdmap e2456: 8 total, 8 up, 8 in 2024-04-07T17:45:19.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:45:19 smithi100 bash[76202]: cluster 2024-04-07T17:45:18.678760+0000 mgr.y (mgr.25175) 2698 : cluster [DBG] pgmap v4728: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 2.5 KiB/s rd, 0 B/s wr, 4 op/s 2024-04-07T17:45:19.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:45:19 smithi100 bash[76202]: cluster 2024-04-07T17:45:19.401965+0000 mon.a (mon.0) 4439 : cluster [DBG] osdmap e2456: 8 total, 8 up, 8 in 2024-04-07T17:45:21.703 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:45:21 smithi100 bash[76202]: cluster 2024-04-07T17:45:20.405087+0000 mon.a (mon.0) 4440 : cluster [DBG] osdmap e2457: 8 total, 8 up, 8 in 2024-04-07T17:45:21.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:45:21 smithi100 bash[76202]: cluster 2024-04-07T17:45:20.679538+0000 mgr.y (mgr.25175) 2699 : cluster [DBG] pgmap v4731: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 957 KiB/s rd, 5.0 KiB/s wr, 1.29k op/s 2024-04-07T17:45:21.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:45:21 smithi100 bash[74942]: cluster 2024-04-07T17:45:20.405087+0000 mon.a (mon.0) 4440 : cluster [DBG] osdmap e2457: 8 total, 8 up, 8 in 2024-04-07T17:45:21.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:45:21 smithi100 bash[74942]: cluster 2024-04-07T17:45:20.679538+0000 mgr.y (mgr.25175) 2699 : cluster [DBG] pgmap v4731: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 957 KiB/s rd, 5.0 KiB/s wr, 1.29k op/s 2024-04-07T17:45:21.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:45:21 smithi144 bash[66654]: cluster 2024-04-07T17:45:20.405087+0000 mon.a (mon.0) 4440 : cluster [DBG] osdmap e2457: 8 total, 8 up, 8 in 2024-04-07T17:45:21.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:45:21 smithi144 bash[66654]: cluster 2024-04-07T17:45:20.679538+0000 mgr.y (mgr.25175) 2699 : cluster [DBG] pgmap v4731: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 957 KiB/s rd, 5.0 KiB/s wr, 1.29k op/s 2024-04-07T17:45:22.614 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:45:22 smithi144 bash[66654]: cluster 2024-04-07T17:45:21.421594+0000 mon.a (mon.0) 4441 : cluster [DBG] osdmap e2458: 8 total, 8 up, 8 in 2024-04-07T17:45:22.615 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:45:22 smithi144 bash[95233]: ts=2024-04-07T17:45:22.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:45:22.703 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:45:22 smithi100 bash[74942]: cluster 2024-04-07T17:45:21.421594+0000 mon.a (mon.0) 4441 : cluster [DBG] osdmap e2458: 8 total, 8 up, 8 in 2024-04-07T17:45:22.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:45:22 smithi100 bash[76202]: cluster 2024-04-07T17:45:21.421594+0000 mon.a (mon.0) 4441 : cluster [DBG] osdmap e2458: 8 total, 8 up, 8 in 2024-04-07T17:45:23.703 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:45:23 smithi100 bash[74942]: cluster 2024-04-07T17:45:22.418129+0000 mon.a (mon.0) 4442 : cluster [DBG] osdmap e2459: 8 total, 8 up, 8 in 2024-04-07T17:45:23.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:45:23 smithi100 bash[74942]: cluster 2024-04-07T17:45:22.680216+0000 mgr.y (mgr.25175) 2700 : cluster [DBG] pgmap v4734: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 957 KiB/s rd, 5.0 KiB/s wr, 1.30k op/s 2024-04-07T17:45:23.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:45:23 smithi100 bash[74942]: cluster 2024-04-07T17:45:23.403068+0000 mon.a (mon.0) 4443 : cluster [DBG] osdmap e2460: 8 total, 8 up, 8 in 2024-04-07T17:45:23.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:45:23 smithi100 bash[76202]: cluster 2024-04-07T17:45:22.418129+0000 mon.a (mon.0) 4442 : cluster [DBG] osdmap e2459: 8 total, 8 up, 8 in 2024-04-07T17:45:23.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:45:23 smithi100 bash[76202]: cluster 2024-04-07T17:45:22.680216+0000 mgr.y (mgr.25175) 2700 : cluster [DBG] pgmap v4734: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 957 KiB/s rd, 5.0 KiB/s wr, 1.30k op/s 2024-04-07T17:45:23.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:45:23 smithi100 bash[76202]: cluster 2024-04-07T17:45:23.403068+0000 mon.a (mon.0) 4443 : cluster [DBG] osdmap e2460: 8 total, 8 up, 8 in 2024-04-07T17:45:23.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:45:23 smithi144 bash[66654]: cluster 2024-04-07T17:45:22.418129+0000 mon.a (mon.0) 4442 : cluster [DBG] osdmap e2459: 8 total, 8 up, 8 in 2024-04-07T17:45:23.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:45:23 smithi144 bash[66654]: cluster 2024-04-07T17:45:22.680216+0000 mgr.y (mgr.25175) 2700 : cluster [DBG] pgmap v4734: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 957 KiB/s rd, 5.0 KiB/s wr, 1.30k op/s 2024-04-07T17:45:23.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:45:23 smithi144 bash[66654]: cluster 2024-04-07T17:45:23.403068+0000 mon.a (mon.0) 4443 : cluster [DBG] osdmap e2460: 8 total, 8 up, 8 in 2024-04-07T17:45:25.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:45:25 smithi144 bash[66654]: cluster 2024-04-07T17:45:24.458025+0000 mon.a (mon.0) 4444 : cluster [DBG] osdmap e2461: 8 total, 8 up, 8 in 2024-04-07T17:45:25.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:45:25 smithi144 bash[66654]: cluster 2024-04-07T17:45:24.681006+0000 mgr.y (mgr.25175) 2701 : cluster [DBG] pgmap v4737: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 924 KiB/s rd, 5.0 KiB/s wr, 1.25k op/s 2024-04-07T17:45:25.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:45:25 smithi100 bash[74942]: cluster 2024-04-07T17:45:24.458025+0000 mon.a (mon.0) 4444 : cluster [DBG] osdmap e2461: 8 total, 8 up, 8 in 2024-04-07T17:45:25.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:45:25 smithi100 bash[74942]: cluster 2024-04-07T17:45:24.681006+0000 mgr.y (mgr.25175) 2701 : cluster [DBG] pgmap v4737: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 924 KiB/s rd, 5.0 KiB/s wr, 1.25k op/s 2024-04-07T17:45:25.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:45:25 smithi100 bash[76202]: cluster 2024-04-07T17:45:24.458025+0000 mon.a (mon.0) 4444 : cluster [DBG] osdmap e2461: 8 total, 8 up, 8 in 2024-04-07T17:45:25.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:45:25 smithi100 bash[76202]: cluster 2024-04-07T17:45:24.681006+0000 mgr.y (mgr.25175) 2701 : cluster [DBG] pgmap v4737: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 924 KiB/s rd, 5.0 KiB/s wr, 1.25k op/s 2024-04-07T17:45:26.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:45:26 smithi144 bash[66654]: cluster 2024-04-07T17:45:25.472370+0000 mon.a (mon.0) 4445 : cluster [DBG] osdmap e2462: 8 total, 8 up, 8 in 2024-04-07T17:45:26.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:45:26 smithi100 bash[74942]: cluster 2024-04-07T17:45:25.472370+0000 mon.a (mon.0) 4445 : cluster [DBG] osdmap e2462: 8 total, 8 up, 8 in 2024-04-07T17:45:26.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:45:26 smithi100 bash[76202]: cluster 2024-04-07T17:45:25.472370+0000 mon.a (mon.0) 4445 : cluster [DBG] osdmap e2462: 8 total, 8 up, 8 in 2024-04-07T17:45:27.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:45:27 smithi144 bash[66654]: cluster 2024-04-07T17:45:26.477196+0000 mon.a (mon.0) 4446 : cluster [DBG] osdmap e2463: 8 total, 8 up, 8 in 2024-04-07T17:45:27.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:45:27 smithi144 bash[66654]: cluster 2024-04-07T17:45:26.681831+0000 mgr.y (mgr.25175) 2702 : cluster [DBG] pgmap v4740: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 921 KiB/s rd, 5.0 KiB/s wr, 1.25k op/s 2024-04-07T17:45:27.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:45:27 smithi100 bash[74942]: cluster 2024-04-07T17:45:26.477196+0000 mon.a (mon.0) 4446 : cluster [DBG] osdmap e2463: 8 total, 8 up, 8 in 2024-04-07T17:45:27.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:45:27 smithi100 bash[74942]: cluster 2024-04-07T17:45:26.681831+0000 mgr.y (mgr.25175) 2702 : cluster [DBG] pgmap v4740: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 921 KiB/s rd, 5.0 KiB/s wr, 1.25k op/s 2024-04-07T17:45:27.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:45:27 smithi100 bash[76202]: cluster 2024-04-07T17:45:26.477196+0000 mon.a (mon.0) 4446 : cluster [DBG] osdmap e2463: 8 total, 8 up, 8 in 2024-04-07T17:45:27.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:45:27 smithi100 bash[76202]: cluster 2024-04-07T17:45:26.681831+0000 mgr.y (mgr.25175) 2702 : cluster [DBG] pgmap v4740: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 921 KiB/s rd, 5.0 KiB/s wr, 1.25k op/s 2024-04-07T17:45:28.704 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:45:28 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:45:28] "GET /metrics HTTP/1.1" 200 36746 "" "Prometheus/2.43.0" 2024-04-07T17:45:28.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:45:28 smithi100 bash[74942]: cluster 2024-04-07T17:45:27.489045+0000 mon.a (mon.0) 4447 : cluster [DBG] osdmap e2464: 8 total, 8 up, 8 in 2024-04-07T17:45:28.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:45:28 smithi100 bash[74942]: cluster 2024-04-07T17:45:28.403962+0000 mon.a (mon.0) 4448 : cluster [DBG] osdmap e2465: 8 total, 8 up, 8 in 2024-04-07T17:45:28.705 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:45:28 smithi100 bash[76202]: cluster 2024-04-07T17:45:27.489045+0000 mon.a (mon.0) 4447 : cluster [DBG] osdmap e2464: 8 total, 8 up, 8 in 2024-04-07T17:45:28.705 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:45:28 smithi100 bash[76202]: cluster 2024-04-07T17:45:28.403962+0000 mon.a (mon.0) 4448 : cluster [DBG] osdmap e2465: 8 total, 8 up, 8 in 2024-04-07T17:45:28.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:45:28 smithi144 bash[66654]: cluster 2024-04-07T17:45:27.489045+0000 mon.a (mon.0) 4447 : cluster [DBG] osdmap e2464: 8 total, 8 up, 8 in 2024-04-07T17:45:28.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:45:28 smithi144 bash[66654]: cluster 2024-04-07T17:45:28.403962+0000 mon.a (mon.0) 4448 : cluster [DBG] osdmap e2465: 8 total, 8 up, 8 in 2024-04-07T17:45:29.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:45:29 smithi144 bash[66654]: cluster 2024-04-07T17:45:28.682704+0000 mgr.y (mgr.25175) 2703 : cluster [DBG] pgmap v4743: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 3.5 KiB/s rd, 0 B/s wr, 6 op/s 2024-04-07T17:45:29.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:45:29 smithi100 bash[74942]: cluster 2024-04-07T17:45:28.682704+0000 mgr.y (mgr.25175) 2703 : cluster [DBG] pgmap v4743: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 3.5 KiB/s rd, 0 B/s wr, 6 op/s 2024-04-07T17:45:29.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:45:29 smithi100 bash[76202]: cluster 2024-04-07T17:45:28.682704+0000 mgr.y (mgr.25175) 2703 : cluster [DBG] pgmap v4743: 137 pgs: 137 active+clean; 4.6 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 3.5 KiB/s rd, 0 B/s wr, 6 op/s 2024-04-07T17:45:30.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:45:30 smithi144 bash[66654]: cluster 2024-04-07T17:45:29.498065+0000 mon.a (mon.0) 4449 : cluster [DBG] osdmap e2466: 8 total, 8 up, 8 in 2024-04-07T17:45:30.953 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:45:30 smithi100 bash[76202]: cluster 2024-04-07T17:45:29.498065+0000 mon.a (mon.0) 4449 : cluster [DBG] osdmap e2466: 8 total, 8 up, 8 in 2024-04-07T17:45:30.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:45:30 smithi100 bash[74942]: cluster 2024-04-07T17:45:29.498065+0000 mon.a (mon.0) 4449 : cluster [DBG] osdmap e2466: 8 total, 8 up, 8 in 2024-04-07T17:45:31.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:45:31 smithi144 bash[66654]: cluster 2024-04-07T17:45:30.506881+0000 mon.a (mon.0) 4450 : cluster [DBG] osdmap e2467: 8 total, 8 up, 8 in 2024-04-07T17:45:31.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:45:31 smithi144 bash[66654]: cluster 2024-04-07T17:45:30.683469+0000 mgr.y (mgr.25175) 2704 : cluster [DBG] pgmap v4746: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 892 KiB/s rd, 5.0 KiB/s wr, 1.21k op/s 2024-04-07T17:45:31.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:45:31 smithi100 bash[74942]: cluster 2024-04-07T17:45:30.506881+0000 mon.a (mon.0) 4450 : cluster [DBG] osdmap e2467: 8 total, 8 up, 8 in 2024-04-07T17:45:31.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:45:31 smithi100 bash[74942]: cluster 2024-04-07T17:45:30.683469+0000 mgr.y (mgr.25175) 2704 : cluster [DBG] pgmap v4746: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 892 KiB/s rd, 5.0 KiB/s wr, 1.21k op/s 2024-04-07T17:45:31.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:45:31 smithi100 bash[76202]: cluster 2024-04-07T17:45:30.506881+0000 mon.a (mon.0) 4450 : cluster [DBG] osdmap e2467: 8 total, 8 up, 8 in 2024-04-07T17:45:31.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:45:31 smithi100 bash[76202]: cluster 2024-04-07T17:45:30.683469+0000 mgr.y (mgr.25175) 2704 : cluster [DBG] pgmap v4746: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 892 KiB/s rd, 5.0 KiB/s wr, 1.21k op/s 2024-04-07T17:45:32.614 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:45:32 smithi144 bash[66654]: cluster 2024-04-07T17:45:31.517289+0000 mon.a (mon.0) 4451 : cluster [DBG] osdmap e2468: 8 total, 8 up, 8 in 2024-04-07T17:45:32.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:45:32 smithi144 bash[66654]: audit 2024-04-07T17:45:31.875049+0000 mon.a (mon.0) 4452 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:45:32.615 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:45:32 smithi144 bash[95233]: ts=2024-04-07T17:45:32.317Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:45:32.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:45:32 smithi100 bash[74942]: cluster 2024-04-07T17:45:31.517289+0000 mon.a (mon.0) 4451 : cluster [DBG] osdmap e2468: 8 total, 8 up, 8 in 2024-04-07T17:45:32.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:45:32 smithi100 bash[74942]: audit 2024-04-07T17:45:31.875049+0000 mon.a (mon.0) 4452 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:45:32.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:45:32 smithi100 bash[76202]: cluster 2024-04-07T17:45:31.517289+0000 mon.a (mon.0) 4451 : cluster [DBG] osdmap e2468: 8 total, 8 up, 8 in 2024-04-07T17:45:32.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:45:32 smithi100 bash[76202]: audit 2024-04-07T17:45:31.875049+0000 mon.a (mon.0) 4452 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:45:33.866 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:45:33 smithi144 bash[66654]: cluster 2024-04-07T17:45:32.525078+0000 mon.a (mon.0) 4453 : cluster [DBG] osdmap e2469: 8 total, 8 up, 8 in 2024-04-07T17:45:33.867 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:45:33 smithi144 bash[66654]: cluster 2024-04-07T17:45:32.684164+0000 mgr.y (mgr.25175) 2705 : cluster [DBG] pgmap v4749: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 891 KiB/s rd, 5.0 KiB/s wr, 1.20k op/s 2024-04-07T17:45:33.867 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:45:33 smithi144 bash[66654]: cluster 2024-04-07T17:45:33.406745+0000 mon.a (mon.0) 4454 : cluster [DBG] osdmap e2470: 8 total, 8 up, 8 in 2024-04-07T17:45:33.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:45:33 smithi100 bash[74942]: cluster 2024-04-07T17:45:32.525078+0000 mon.a (mon.0) 4453 : cluster [DBG] osdmap e2469: 8 total, 8 up, 8 in 2024-04-07T17:45:33.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:45:33 smithi100 bash[74942]: cluster 2024-04-07T17:45:32.684164+0000 mgr.y (mgr.25175) 2705 : cluster [DBG] pgmap v4749: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 891 KiB/s rd, 5.0 KiB/s wr, 1.20k op/s 2024-04-07T17:45:33.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:45:33 smithi100 bash[74942]: cluster 2024-04-07T17:45:33.406745+0000 mon.a (mon.0) 4454 : cluster [DBG] osdmap e2470: 8 total, 8 up, 8 in 2024-04-07T17:45:33.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:45:33 smithi100 bash[76202]: cluster 2024-04-07T17:45:32.525078+0000 mon.a (mon.0) 4453 : cluster [DBG] osdmap e2469: 8 total, 8 up, 8 in 2024-04-07T17:45:33.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:45:33 smithi100 bash[76202]: cluster 2024-04-07T17:45:32.684164+0000 mgr.y (mgr.25175) 2705 : cluster [DBG] pgmap v4749: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 891 KiB/s rd, 5.0 KiB/s wr, 1.20k op/s 2024-04-07T17:45:33.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:45:33 smithi100 bash[76202]: cluster 2024-04-07T17:45:33.406745+0000 mon.a (mon.0) 4454 : cluster [DBG] osdmap e2470: 8 total, 8 up, 8 in 2024-04-07T17:45:35.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:45:35 smithi144 bash[66654]: cluster 2024-04-07T17:45:34.539455+0000 mon.a (mon.0) 4455 : cluster [DBG] osdmap e2471: 8 total, 8 up, 8 in 2024-04-07T17:45:35.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:45:35 smithi144 bash[66654]: cluster 2024-04-07T17:45:34.684861+0000 mgr.y (mgr.25175) 2706 : cluster [DBG] pgmap v4752: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 857 KiB/s rd, 5.0 KiB/s wr, 1.16k op/s 2024-04-07T17:45:35.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:45:35 smithi100 bash[74942]: cluster 2024-04-07T17:45:34.539455+0000 mon.a (mon.0) 4455 : cluster [DBG] osdmap e2471: 8 total, 8 up, 8 in 2024-04-07T17:45:35.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:45:35 smithi100 bash[74942]: cluster 2024-04-07T17:45:34.684861+0000 mgr.y (mgr.25175) 2706 : cluster [DBG] pgmap v4752: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 857 KiB/s rd, 5.0 KiB/s wr, 1.16k op/s 2024-04-07T17:45:35.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:45:35 smithi100 bash[76202]: cluster 2024-04-07T17:45:34.539455+0000 mon.a (mon.0) 4455 : cluster [DBG] osdmap e2471: 8 total, 8 up, 8 in 2024-04-07T17:45:35.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:45:35 smithi100 bash[76202]: cluster 2024-04-07T17:45:34.684861+0000 mgr.y (mgr.25175) 2706 : cluster [DBG] pgmap v4752: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 857 KiB/s rd, 5.0 KiB/s wr, 1.16k op/s 2024-04-07T17:45:36.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:45:36 smithi144 bash[66654]: cluster 2024-04-07T17:45:35.553673+0000 mon.a (mon.0) 4456 : cluster [DBG] osdmap e2472: 8 total, 8 up, 8 in 2024-04-07T17:45:36.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:45:36 smithi100 bash[74942]: cluster 2024-04-07T17:45:35.553673+0000 mon.a (mon.0) 4456 : cluster [DBG] osdmap e2472: 8 total, 8 up, 8 in 2024-04-07T17:45:36.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:45:36 smithi100 bash[76202]: cluster 2024-04-07T17:45:35.553673+0000 mon.a (mon.0) 4456 : cluster [DBG] osdmap e2472: 8 total, 8 up, 8 in 2024-04-07T17:45:37.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:45:37 smithi144 bash[66654]: cluster 2024-04-07T17:45:36.557801+0000 mon.a (mon.0) 4457 : cluster [DBG] osdmap e2473: 8 total, 8 up, 8 in 2024-04-07T17:45:37.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:45:37 smithi144 bash[66654]: cluster 2024-04-07T17:45:36.685630+0000 mgr.y (mgr.25175) 2707 : cluster [DBG] pgmap v4755: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 856 KiB/s rd, 5.0 KiB/s wr, 1.16k op/s 2024-04-07T17:45:37.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:45:37 smithi144 bash[66654]: audit 2024-04-07T17:45:37.449367+0000 mon.a (mon.0) 4458 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:45:37.953 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:45:37 smithi100 bash[76202]: cluster 2024-04-07T17:45:36.557801+0000 mon.a (mon.0) 4457 : cluster [DBG] osdmap e2473: 8 total, 8 up, 8 in 2024-04-07T17:45:37.953 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:45:37 smithi100 bash[76202]: cluster 2024-04-07T17:45:36.685630+0000 mgr.y (mgr.25175) 2707 : cluster [DBG] pgmap v4755: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 856 KiB/s rd, 5.0 KiB/s wr, 1.16k op/s 2024-04-07T17:45:37.953 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:45:37 smithi100 bash[76202]: audit 2024-04-07T17:45:37.449367+0000 mon.a (mon.0) 4458 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:45:37.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:45:37 smithi100 bash[74942]: cluster 2024-04-07T17:45:36.557801+0000 mon.a (mon.0) 4457 : cluster [DBG] osdmap e2473: 8 total, 8 up, 8 in 2024-04-07T17:45:37.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:45:37 smithi100 bash[74942]: cluster 2024-04-07T17:45:36.685630+0000 mgr.y (mgr.25175) 2707 : cluster [DBG] pgmap v4755: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 856 KiB/s rd, 5.0 KiB/s wr, 1.16k op/s 2024-04-07T17:45:37.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:45:37 smithi100 bash[74942]: audit 2024-04-07T17:45:37.449367+0000 mon.a (mon.0) 4458 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:45:38.703 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:45:38 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:45:38] "GET /metrics HTTP/1.1" 200 36747 "" "Prometheus/2.43.0" 2024-04-07T17:45:38.990 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:45:38 smithi100 bash[76202]: cluster 2024-04-07T17:45:37.569709+0000 mon.a (mon.0) 4459 : cluster [DBG] osdmap e2474: 8 total, 8 up, 8 in 2024-04-07T17:45:38.990 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:45:38 smithi100 bash[76202]: audit 2024-04-07T17:45:38.179608+0000 mon.a (mon.0) 4460 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:45:38.990 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:45:38 smithi100 bash[76202]: audit 2024-04-07T17:45:38.181004+0000 mon.a (mon.0) 4461 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:45:38.991 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:45:38 smithi100 bash[76202]: audit 2024-04-07T17:45:38.189662+0000 mon.a (mon.0) 4462 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:45:38.991 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:45:38 smithi100 bash[76202]: cluster 2024-04-07T17:45:38.409943+0000 mon.a (mon.0) 4463 : cluster [DBG] osdmap e2475: 8 total, 8 up, 8 in 2024-04-07T17:45:38.991 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:45:38 smithi100 bash[74942]: cluster 2024-04-07T17:45:37.569709+0000 mon.a (mon.0) 4459 : cluster [DBG] osdmap e2474: 8 total, 8 up, 8 in 2024-04-07T17:45:38.991 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:45:38 smithi100 bash[74942]: audit 2024-04-07T17:45:38.179608+0000 mon.a (mon.0) 4460 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:45:38.991 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:45:38 smithi100 bash[74942]: audit 2024-04-07T17:45:38.181004+0000 mon.a (mon.0) 4461 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:45:38.991 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:45:38 smithi100 bash[74942]: audit 2024-04-07T17:45:38.189662+0000 mon.a (mon.0) 4462 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:45:38.991 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:45:38 smithi100 bash[74942]: cluster 2024-04-07T17:45:38.409943+0000 mon.a (mon.0) 4463 : cluster [DBG] osdmap e2475: 8 total, 8 up, 8 in 2024-04-07T17:45:38.992 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:45:38 smithi144 bash[66654]: cluster 2024-04-07T17:45:37.569709+0000 mon.a (mon.0) 4459 : cluster [DBG] osdmap e2474: 8 total, 8 up, 8 in 2024-04-07T17:45:38.992 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:45:38 smithi144 bash[66654]: audit 2024-04-07T17:45:38.179608+0000 mon.a (mon.0) 4460 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:45:38.992 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:45:38 smithi144 bash[66654]: audit 2024-04-07T17:45:38.181004+0000 mon.a (mon.0) 4461 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:45:38.992 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:45:38 smithi144 bash[66654]: audit 2024-04-07T17:45:38.189662+0000 mon.a (mon.0) 4462 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:45:38.992 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:45:38 smithi144 bash[66654]: cluster 2024-04-07T17:45:38.409943+0000 mon.a (mon.0) 4463 : cluster [DBG] osdmap e2475: 8 total, 8 up, 8 in 2024-04-07T17:45:39.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:45:39 smithi144 bash[66654]: cluster 2024-04-07T17:45:38.686350+0000 mgr.y (mgr.25175) 2708 : cluster [DBG] pgmap v4758: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 3.0 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-07T17:45:39.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:45:39 smithi144 bash[66654]: cluster 2024-04-07T17:45:39.413047+0000 mon.a (mon.0) 4464 : cluster [DBG] osdmap e2476: 8 total, 8 up, 8 in 2024-04-07T17:45:39.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:45:39 smithi100 bash[74942]: cluster 2024-04-07T17:45:38.686350+0000 mgr.y (mgr.25175) 2708 : cluster [DBG] pgmap v4758: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 3.0 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-07T17:45:39.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:45:39 smithi100 bash[74942]: cluster 2024-04-07T17:45:39.413047+0000 mon.a (mon.0) 4464 : cluster [DBG] osdmap e2476: 8 total, 8 up, 8 in 2024-04-07T17:45:39.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:45:39 smithi100 bash[76202]: cluster 2024-04-07T17:45:38.686350+0000 mgr.y (mgr.25175) 2708 : cluster [DBG] pgmap v4758: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 3.0 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-07T17:45:39.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:45:39 smithi100 bash[76202]: cluster 2024-04-07T17:45:39.413047+0000 mon.a (mon.0) 4464 : cluster [DBG] osdmap e2476: 8 total, 8 up, 8 in 2024-04-07T17:45:41.703 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:45:41 smithi100 bash[74942]: cluster 2024-04-07T17:45:40.416334+0000 mon.a (mon.0) 4465 : cluster [DBG] osdmap e2477: 8 total, 8 up, 8 in 2024-04-07T17:45:41.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:45:41 smithi100 bash[74942]: cluster 2024-04-07T17:45:40.687225+0000 mgr.y (mgr.25175) 2709 : cluster [DBG] pgmap v4761: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 826 KiB/s rd, 5.0 KiB/s wr, 1.12k op/s 2024-04-07T17:45:41.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:45:41 smithi100 bash[76202]: cluster 2024-04-07T17:45:40.416334+0000 mon.a (mon.0) 4465 : cluster [DBG] osdmap e2477: 8 total, 8 up, 8 in 2024-04-07T17:45:41.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:45:41 smithi100 bash[76202]: cluster 2024-04-07T17:45:40.687225+0000 mgr.y (mgr.25175) 2709 : cluster [DBG] pgmap v4761: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 826 KiB/s rd, 5.0 KiB/s wr, 1.12k op/s 2024-04-07T17:45:41.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:45:41 smithi144 bash[66654]: cluster 2024-04-07T17:45:40.416334+0000 mon.a (mon.0) 4465 : cluster [DBG] osdmap e2477: 8 total, 8 up, 8 in 2024-04-07T17:45:41.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:45:41 smithi144 bash[66654]: cluster 2024-04-07T17:45:40.687225+0000 mgr.y (mgr.25175) 2709 : cluster [DBG] pgmap v4761: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 826 KiB/s rd, 5.0 KiB/s wr, 1.12k op/s 2024-04-07T17:45:42.614 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:45:42 smithi144 bash[66654]: cluster 2024-04-07T17:45:41.432872+0000 mon.a (mon.0) 4466 : cluster [DBG] osdmap e2478: 8 total, 8 up, 8 in 2024-04-07T17:45:42.615 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:45:42 smithi144 bash[95233]: ts=2024-04-07T17:45:42.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:45:42.703 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:45:42 smithi100 bash[74942]: cluster 2024-04-07T17:45:41.432872+0000 mon.a (mon.0) 4466 : cluster [DBG] osdmap e2478: 8 total, 8 up, 8 in 2024-04-07T17:45:42.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:45:42 smithi100 bash[76202]: cluster 2024-04-07T17:45:41.432872+0000 mon.a (mon.0) 4466 : cluster [DBG] osdmap e2478: 8 total, 8 up, 8 in 2024-04-07T17:45:43.703 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:45:43 smithi100 bash[74942]: cluster 2024-04-07T17:45:42.438771+0000 mon.a (mon.0) 4467 : cluster [DBG] osdmap e2479: 8 total, 8 up, 8 in 2024-04-07T17:45:43.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:45:43 smithi100 bash[74942]: cluster 2024-04-07T17:45:42.687999+0000 mgr.y (mgr.25175) 2710 : cluster [DBG] pgmap v4764: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 826 KiB/s rd, 5.0 KiB/s wr, 1.12k op/s 2024-04-07T17:45:43.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:45:43 smithi100 bash[74942]: cluster 2024-04-07T17:45:43.410863+0000 mon.a (mon.0) 4468 : cluster [DBG] osdmap e2480: 8 total, 8 up, 8 in 2024-04-07T17:45:43.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:45:43 smithi100 bash[76202]: cluster 2024-04-07T17:45:42.438771+0000 mon.a (mon.0) 4467 : cluster [DBG] osdmap e2479: 8 total, 8 up, 8 in 2024-04-07T17:45:43.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:45:43 smithi100 bash[76202]: cluster 2024-04-07T17:45:42.687999+0000 mgr.y (mgr.25175) 2710 : cluster [DBG] pgmap v4764: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 826 KiB/s rd, 5.0 KiB/s wr, 1.12k op/s 2024-04-07T17:45:43.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:45:43 smithi100 bash[76202]: cluster 2024-04-07T17:45:43.410863+0000 mon.a (mon.0) 4468 : cluster [DBG] osdmap e2480: 8 total, 8 up, 8 in 2024-04-07T17:45:43.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:45:43 smithi144 bash[66654]: cluster 2024-04-07T17:45:42.438771+0000 mon.a (mon.0) 4467 : cluster [DBG] osdmap e2479: 8 total, 8 up, 8 in 2024-04-07T17:45:43.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:45:43 smithi144 bash[66654]: cluster 2024-04-07T17:45:42.687999+0000 mgr.y (mgr.25175) 2710 : cluster [DBG] pgmap v4764: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 826 KiB/s rd, 5.0 KiB/s wr, 1.12k op/s 2024-04-07T17:45:43.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:45:43 smithi144 bash[66654]: cluster 2024-04-07T17:45:43.410863+0000 mon.a (mon.0) 4468 : cluster [DBG] osdmap e2480: 8 total, 8 up, 8 in 2024-04-07T17:45:45.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:45:45 smithi144 bash[66654]: cluster 2024-04-07T17:45:44.456840+0000 mon.a (mon.0) 4469 : cluster [DBG] osdmap e2481: 8 total, 8 up, 8 in 2024-04-07T17:45:45.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:45:45 smithi144 bash[66654]: cluster 2024-04-07T17:45:44.688690+0000 mgr.y (mgr.25175) 2711 : cluster [DBG] pgmap v4767: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 793 KiB/s rd, 5.0 KiB/s wr, 1.07k op/s 2024-04-07T17:45:45.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:45:45 smithi100 bash[74942]: cluster 2024-04-07T17:45:44.456840+0000 mon.a (mon.0) 4469 : cluster [DBG] osdmap e2481: 8 total, 8 up, 8 in 2024-04-07T17:45:45.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:45:45 smithi100 bash[74942]: cluster 2024-04-07T17:45:44.688690+0000 mgr.y (mgr.25175) 2711 : cluster [DBG] pgmap v4767: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 793 KiB/s rd, 5.0 KiB/s wr, 1.07k op/s 2024-04-07T17:45:45.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:45:45 smithi100 bash[76202]: cluster 2024-04-07T17:45:44.456840+0000 mon.a (mon.0) 4469 : cluster [DBG] osdmap e2481: 8 total, 8 up, 8 in 2024-04-07T17:45:45.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:45:45 smithi100 bash[76202]: cluster 2024-04-07T17:45:44.688690+0000 mgr.y (mgr.25175) 2711 : cluster [DBG] pgmap v4767: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 793 KiB/s rd, 5.0 KiB/s wr, 1.07k op/s 2024-04-07T17:45:46.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:45:46 smithi144 bash[66654]: cluster 2024-04-07T17:45:45.465772+0000 mon.a (mon.0) 4470 : cluster [DBG] osdmap e2482: 8 total, 8 up, 8 in 2024-04-07T17:45:46.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:45:46 smithi100 bash[74942]: cluster 2024-04-07T17:45:45.465772+0000 mon.a (mon.0) 4470 : cluster [DBG] osdmap e2482: 8 total, 8 up, 8 in 2024-04-07T17:45:46.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:45:46 smithi100 bash[76202]: cluster 2024-04-07T17:45:45.465772+0000 mon.a (mon.0) 4470 : cluster [DBG] osdmap e2482: 8 total, 8 up, 8 in 2024-04-07T17:45:47.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:45:47 smithi144 bash[66654]: cluster 2024-04-07T17:45:46.480042+0000 mon.a (mon.0) 4471 : cluster [DBG] osdmap e2483: 8 total, 8 up, 8 in 2024-04-07T17:45:47.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:45:47 smithi144 bash[66654]: cluster 2024-04-07T17:45:46.689393+0000 mgr.y (mgr.25175) 2712 : cluster [DBG] pgmap v4770: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 791 KiB/s rd, 5.0 KiB/s wr, 1.07k op/s 2024-04-07T17:45:47.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:45:47 smithi144 bash[66654]: audit 2024-04-07T17:45:46.875519+0000 mon.a (mon.0) 4472 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:45:47.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:45:47 smithi100 bash[74942]: cluster 2024-04-07T17:45:46.480042+0000 mon.a (mon.0) 4471 : cluster [DBG] osdmap e2483: 8 total, 8 up, 8 in 2024-04-07T17:45:47.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:45:47 smithi100 bash[74942]: cluster 2024-04-07T17:45:46.689393+0000 mgr.y (mgr.25175) 2712 : cluster [DBG] pgmap v4770: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 791 KiB/s rd, 5.0 KiB/s wr, 1.07k op/s 2024-04-07T17:45:47.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:45:47 smithi100 bash[74942]: audit 2024-04-07T17:45:46.875519+0000 mon.a (mon.0) 4472 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:45:47.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:45:47 smithi100 bash[76202]: cluster 2024-04-07T17:45:46.480042+0000 mon.a (mon.0) 4471 : cluster [DBG] osdmap e2483: 8 total, 8 up, 8 in 2024-04-07T17:45:47.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:45:47 smithi100 bash[76202]: cluster 2024-04-07T17:45:46.689393+0000 mgr.y (mgr.25175) 2712 : cluster [DBG] pgmap v4770: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 791 KiB/s rd, 5.0 KiB/s wr, 1.07k op/s 2024-04-07T17:45:47.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:45:47 smithi100 bash[76202]: audit 2024-04-07T17:45:46.875519+0000 mon.a (mon.0) 4472 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:45:48.703 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:45:48 smithi100 bash[74942]: cluster 2024-04-07T17:45:47.491107+0000 mon.a (mon.0) 4473 : cluster [DBG] osdmap e2484: 8 total, 8 up, 8 in 2024-04-07T17:45:48.703 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:45:48 smithi100 bash[74942]: cluster 2024-04-07T17:45:48.413724+0000 mon.a (mon.0) 4474 : cluster [DBG] osdmap e2485: 8 total, 8 up, 8 in 2024-04-07T17:45:48.704 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:45:48 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:45:48] "GET /metrics HTTP/1.1" 200 36752 "" "Prometheus/2.43.0" 2024-04-07T17:45:48.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:45:48 smithi100 bash[76202]: cluster 2024-04-07T17:45:47.491107+0000 mon.a (mon.0) 4473 : cluster [DBG] osdmap e2484: 8 total, 8 up, 8 in 2024-04-07T17:45:48.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:45:48 smithi100 bash[76202]: cluster 2024-04-07T17:45:48.413724+0000 mon.a (mon.0) 4474 : cluster [DBG] osdmap e2485: 8 total, 8 up, 8 in 2024-04-07T17:45:48.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:45:48 smithi144 bash[66654]: cluster 2024-04-07T17:45:47.491107+0000 mon.a (mon.0) 4473 : cluster [DBG] osdmap e2484: 8 total, 8 up, 8 in 2024-04-07T17:45:48.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:45:48 smithi144 bash[66654]: cluster 2024-04-07T17:45:48.413724+0000 mon.a (mon.0) 4474 : cluster [DBG] osdmap e2485: 8 total, 8 up, 8 in 2024-04-07T17:45:49.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:45:49 smithi144 bash[66654]: cluster 2024-04-07T17:45:48.690146+0000 mgr.y (mgr.25175) 2713 : cluster [DBG] pgmap v4773: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 4.0 KiB/s rd, 0 B/s wr, 7 op/s 2024-04-07T17:45:49.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:45:49 smithi100 bash[74942]: cluster 2024-04-07T17:45:48.690146+0000 mgr.y (mgr.25175) 2713 : cluster [DBG] pgmap v4773: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 4.0 KiB/s rd, 0 B/s wr, 7 op/s 2024-04-07T17:45:49.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:45:49 smithi100 bash[76202]: cluster 2024-04-07T17:45:48.690146+0000 mgr.y (mgr.25175) 2713 : cluster [DBG] pgmap v4773: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 4.0 KiB/s rd, 0 B/s wr, 7 op/s 2024-04-07T17:45:50.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:45:50 smithi144 bash[66654]: cluster 2024-04-07T17:45:49.500045+0000 mon.a (mon.0) 4475 : cluster [DBG] osdmap e2486: 8 total, 8 up, 8 in 2024-04-07T17:45:50.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:45:50 smithi100 bash[74942]: cluster 2024-04-07T17:45:49.500045+0000 mon.a (mon.0) 4475 : cluster [DBG] osdmap e2486: 8 total, 8 up, 8 in 2024-04-07T17:45:50.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:45:50 smithi100 bash[76202]: cluster 2024-04-07T17:45:49.500045+0000 mon.a (mon.0) 4475 : cluster [DBG] osdmap e2486: 8 total, 8 up, 8 in 2024-04-07T17:45:51.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:45:51 smithi144 bash[66654]: cluster 2024-04-07T17:45:50.501252+0000 mon.a (mon.0) 4476 : cluster [DBG] osdmap e2487: 8 total, 8 up, 8 in 2024-04-07T17:45:51.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:45:51 smithi144 bash[66654]: cluster 2024-04-07T17:45:50.690880+0000 mgr.y (mgr.25175) 2714 : cluster [DBG] pgmap v4776: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 760 KiB/s rd, 5.0 KiB/s wr, 1.03k op/s 2024-04-07T17:45:51.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:45:51 smithi100 bash[74942]: cluster 2024-04-07T17:45:50.501252+0000 mon.a (mon.0) 4476 : cluster [DBG] osdmap e2487: 8 total, 8 up, 8 in 2024-04-07T17:45:51.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:45:51 smithi100 bash[74942]: cluster 2024-04-07T17:45:50.690880+0000 mgr.y (mgr.25175) 2714 : cluster [DBG] pgmap v4776: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 760 KiB/s rd, 5.0 KiB/s wr, 1.03k op/s 2024-04-07T17:45:51.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:45:51 smithi100 bash[76202]: cluster 2024-04-07T17:45:50.501252+0000 mon.a (mon.0) 4476 : cluster [DBG] osdmap e2487: 8 total, 8 up, 8 in 2024-04-07T17:45:51.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:45:51 smithi100 bash[76202]: cluster 2024-04-07T17:45:50.690880+0000 mgr.y (mgr.25175) 2714 : cluster [DBG] pgmap v4776: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 760 KiB/s rd, 5.0 KiB/s wr, 1.03k op/s 2024-04-07T17:45:52.614 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:45:52 smithi144 bash[66654]: cluster 2024-04-07T17:45:51.523913+0000 mon.a (mon.0) 4477 : cluster [DBG] osdmap e2488: 8 total, 8 up, 8 in 2024-04-07T17:45:52.615 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:45:52 smithi144 bash[95233]: ts=2024-04-07T17:45:52.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:45:52.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:45:52 smithi100 bash[74942]: cluster 2024-04-07T17:45:51.523913+0000 mon.a (mon.0) 4477 : cluster [DBG] osdmap e2488: 8 total, 8 up, 8 in 2024-04-07T17:45:52.953 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:45:52 smithi100 bash[76202]: cluster 2024-04-07T17:45:51.523913+0000 mon.a (mon.0) 4477 : cluster [DBG] osdmap e2488: 8 total, 8 up, 8 in 2024-04-07T17:45:53.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:45:53 smithi144 bash[66654]: cluster 2024-04-07T17:45:52.533657+0000 mon.a (mon.0) 4478 : cluster [DBG] osdmap e2489: 8 total, 8 up, 8 in 2024-04-07T17:45:53.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:45:53 smithi144 bash[66654]: cluster 2024-04-07T17:45:52.691584+0000 mgr.y (mgr.25175) 2715 : cluster [DBG] pgmap v4779: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 760 KiB/s rd, 5.0 KiB/s wr, 1.03k op/s 2024-04-07T17:45:53.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:45:53 smithi144 bash[66654]: cluster 2024-04-07T17:45:53.415833+0000 mon.a (mon.0) 4479 : cluster [DBG] osdmap e2490: 8 total, 8 up, 8 in 2024-04-07T17:45:53.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:45:53 smithi100 bash[74942]: cluster 2024-04-07T17:45:52.533657+0000 mon.a (mon.0) 4478 : cluster [DBG] osdmap e2489: 8 total, 8 up, 8 in 2024-04-07T17:45:53.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:45:53 smithi100 bash[74942]: cluster 2024-04-07T17:45:52.691584+0000 mgr.y (mgr.25175) 2715 : cluster [DBG] pgmap v4779: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 760 KiB/s rd, 5.0 KiB/s wr, 1.03k op/s 2024-04-07T17:45:53.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:45:53 smithi100 bash[74942]: cluster 2024-04-07T17:45:53.415833+0000 mon.a (mon.0) 4479 : cluster [DBG] osdmap e2490: 8 total, 8 up, 8 in 2024-04-07T17:45:53.955 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:45:53 smithi100 bash[76202]: cluster 2024-04-07T17:45:52.533657+0000 mon.a (mon.0) 4478 : cluster [DBG] osdmap e2489: 8 total, 8 up, 8 in 2024-04-07T17:45:53.955 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:45:53 smithi100 bash[76202]: cluster 2024-04-07T17:45:52.691584+0000 mgr.y (mgr.25175) 2715 : cluster [DBG] pgmap v4779: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 760 KiB/s rd, 5.0 KiB/s wr, 1.03k op/s 2024-04-07T17:45:53.955 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:45:53 smithi100 bash[76202]: cluster 2024-04-07T17:45:53.415833+0000 mon.a (mon.0) 4479 : cluster [DBG] osdmap e2490: 8 total, 8 up, 8 in 2024-04-07T17:45:55.703 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:45:55 smithi100 bash[74942]: cluster 2024-04-07T17:45:54.418296+0000 mon.a (mon.0) 4480 : cluster [DBG] osdmap e2491: 8 total, 8 up, 8 in 2024-04-07T17:45:55.703 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:45:55 smithi100 bash[74942]: cluster 2024-04-07T17:45:54.692280+0000 mgr.y (mgr.25175) 2716 : cluster [DBG] pgmap v4782: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 728 KiB/s rd, 5.0 KiB/s wr, 985 op/s 2024-04-07T17:45:55.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:45:55 smithi100 bash[76202]: cluster 2024-04-07T17:45:54.418296+0000 mon.a (mon.0) 4480 : cluster [DBG] osdmap e2491: 8 total, 8 up, 8 in 2024-04-07T17:45:55.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:45:55 smithi100 bash[76202]: cluster 2024-04-07T17:45:54.692280+0000 mgr.y (mgr.25175) 2716 : cluster [DBG] pgmap v4782: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 728 KiB/s rd, 5.0 KiB/s wr, 985 op/s 2024-04-07T17:45:55.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:45:55 smithi144 bash[66654]: cluster 2024-04-07T17:45:54.418296+0000 mon.a (mon.0) 4480 : cluster [DBG] osdmap e2491: 8 total, 8 up, 8 in 2024-04-07T17:45:55.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:45:55 smithi144 bash[66654]: cluster 2024-04-07T17:45:54.692280+0000 mgr.y (mgr.25175) 2716 : cluster [DBG] pgmap v4782: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 728 KiB/s rd, 5.0 KiB/s wr, 985 op/s 2024-04-07T17:45:56.703 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:45:56 smithi100 bash[74942]: cluster 2024-04-07T17:45:55.432150+0000 mon.a (mon.0) 4481 : cluster [DBG] osdmap e2492: 8 total, 8 up, 8 in 2024-04-07T17:45:56.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:45:56 smithi100 bash[76202]: cluster 2024-04-07T17:45:55.432150+0000 mon.a (mon.0) 4481 : cluster [DBG] osdmap e2492: 8 total, 8 up, 8 in 2024-04-07T17:45:56.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:45:56 smithi144 bash[66654]: cluster 2024-04-07T17:45:55.432150+0000 mon.a (mon.0) 4481 : cluster [DBG] osdmap e2492: 8 total, 8 up, 8 in 2024-04-07T17:45:57.703 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:45:57 smithi100 bash[74942]: cluster 2024-04-07T17:45:56.442334+0000 mon.a (mon.0) 4482 : cluster [DBG] osdmap e2493: 8 total, 8 up, 8 in 2024-04-07T17:45:57.703 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:45:57 smithi100 bash[74942]: cluster 2024-04-07T17:45:56.693031+0000 mgr.y (mgr.25175) 2717 : cluster [DBG] pgmap v4785: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 726 KiB/s rd, 5.0 KiB/s wr, 980 op/s 2024-04-07T17:45:57.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:45:57 smithi100 bash[76202]: cluster 2024-04-07T17:45:56.442334+0000 mon.a (mon.0) 4482 : cluster [DBG] osdmap e2493: 8 total, 8 up, 8 in 2024-04-07T17:45:57.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:45:57 smithi100 bash[76202]: cluster 2024-04-07T17:45:56.693031+0000 mgr.y (mgr.25175) 2717 : cluster [DBG] pgmap v4785: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 726 KiB/s rd, 5.0 KiB/s wr, 980 op/s 2024-04-07T17:45:57.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:45:57 smithi144 bash[66654]: cluster 2024-04-07T17:45:56.442334+0000 mon.a (mon.0) 4482 : cluster [DBG] osdmap e2493: 8 total, 8 up, 8 in 2024-04-07T17:45:57.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:45:57 smithi144 bash[66654]: cluster 2024-04-07T17:45:56.693031+0000 mgr.y (mgr.25175) 2717 : cluster [DBG] pgmap v4785: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 726 KiB/s rd, 5.0 KiB/s wr, 980 op/s 2024-04-07T17:45:58.703 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:45:58 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:45:58] "GET /metrics HTTP/1.1" 200 36752 "" "Prometheus/2.43.0" 2024-04-07T17:45:58.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:45:58 smithi100 bash[74942]: cluster 2024-04-07T17:45:57.451397+0000 mon.a (mon.0) 4483 : cluster [DBG] osdmap e2494: 8 total, 8 up, 8 in 2024-04-07T17:45:58.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:45:58 smithi100 bash[74942]: cluster 2024-04-07T17:45:58.418219+0000 mon.a (mon.0) 4484 : cluster [DBG] osdmap e2495: 8 total, 8 up, 8 in 2024-04-07T17:45:58.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:45:58 smithi100 bash[76202]: cluster 2024-04-07T17:45:57.451397+0000 mon.a (mon.0) 4483 : cluster [DBG] osdmap e2494: 8 total, 8 up, 8 in 2024-04-07T17:45:58.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:45:58 smithi100 bash[76202]: cluster 2024-04-07T17:45:58.418219+0000 mon.a (mon.0) 4484 : cluster [DBG] osdmap e2495: 8 total, 8 up, 8 in 2024-04-07T17:45:58.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:45:58 smithi144 bash[66654]: cluster 2024-04-07T17:45:57.451397+0000 mon.a (mon.0) 4483 : cluster [DBG] osdmap e2494: 8 total, 8 up, 8 in 2024-04-07T17:45:58.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:45:58 smithi144 bash[66654]: cluster 2024-04-07T17:45:58.418219+0000 mon.a (mon.0) 4484 : cluster [DBG] osdmap e2495: 8 total, 8 up, 8 in 2024-04-07T17:45:59.872 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:45:59 smithi144 bash[66654]: cluster 2024-04-07T17:45:58.693786+0000 mgr.y (mgr.25175) 2718 : cluster [DBG] pgmap v4788: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 2.7 KiB/s rd, 0 B/s wr, 4 op/s 2024-04-07T17:45:59.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:45:59 smithi100 bash[74942]: cluster 2024-04-07T17:45:58.693786+0000 mgr.y (mgr.25175) 2718 : cluster [DBG] pgmap v4788: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 2.7 KiB/s rd, 0 B/s wr, 4 op/s 2024-04-07T17:45:59.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:45:59 smithi100 bash[76202]: cluster 2024-04-07T17:45:58.693786+0000 mgr.y (mgr.25175) 2718 : cluster [DBG] pgmap v4788: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 2.7 KiB/s rd, 0 B/s wr, 4 op/s 2024-04-07T17:46:00.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:46:00 smithi144 bash[66654]: cluster 2024-04-07T17:45:59.466927+0000 mon.a (mon.0) 4485 : cluster [DBG] osdmap e2496: 8 total, 8 up, 8 in 2024-04-07T17:46:00.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:46:00 smithi100 bash[74942]: cluster 2024-04-07T17:45:59.466927+0000 mon.a (mon.0) 4485 : cluster [DBG] osdmap e2496: 8 total, 8 up, 8 in 2024-04-07T17:46:00.953 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:46:00 smithi100 bash[76202]: cluster 2024-04-07T17:45:59.466927+0000 mon.a (mon.0) 4485 : cluster [DBG] osdmap e2496: 8 total, 8 up, 8 in 2024-04-07T17:46:01.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:46:01 smithi144 bash[66654]: cluster 2024-04-07T17:46:00.473188+0000 mon.a (mon.0) 4486 : cluster [DBG] osdmap e2497: 8 total, 8 up, 8 in 2024-04-07T17:46:01.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:46:01 smithi144 bash[66654]: cluster 2024-04-07T17:46:00.694485+0000 mgr.y (mgr.25175) 2719 : cluster [DBG] pgmap v4791: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 695 KiB/s rd, 5.0 KiB/s wr, 940 op/s 2024-04-07T17:46:01.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:46:01 smithi100 bash[74942]: cluster 2024-04-07T17:46:00.473188+0000 mon.a (mon.0) 4486 : cluster [DBG] osdmap e2497: 8 total, 8 up, 8 in 2024-04-07T17:46:01.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:46:01 smithi100 bash[74942]: cluster 2024-04-07T17:46:00.694485+0000 mgr.y (mgr.25175) 2719 : cluster [DBG] pgmap v4791: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 695 KiB/s rd, 5.0 KiB/s wr, 940 op/s 2024-04-07T17:46:01.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:46:01 smithi100 bash[76202]: cluster 2024-04-07T17:46:00.473188+0000 mon.a (mon.0) 4486 : cluster [DBG] osdmap e2497: 8 total, 8 up, 8 in 2024-04-07T17:46:01.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:46:01 smithi100 bash[76202]: cluster 2024-04-07T17:46:00.694485+0000 mgr.y (mgr.25175) 2719 : cluster [DBG] pgmap v4791: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 695 KiB/s rd, 5.0 KiB/s wr, 940 op/s 2024-04-07T17:46:02.614 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:46:02 smithi144 bash[95233]: ts=2024-04-07T17:46:02.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:46:02.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:46:02 smithi144 bash[66654]: cluster 2024-04-07T17:46:01.487833+0000 mon.a (mon.0) 4487 : cluster [DBG] osdmap e2498: 8 total, 8 up, 8 in 2024-04-07T17:46:02.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:46:02 smithi144 bash[66654]: audit 2024-04-07T17:46:01.875943+0000 mon.a (mon.0) 4488 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:46:02.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:46:02 smithi100 bash[74942]: cluster 2024-04-07T17:46:01.487833+0000 mon.a (mon.0) 4487 : cluster [DBG] osdmap e2498: 8 total, 8 up, 8 in 2024-04-07T17:46:02.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:46:02 smithi100 bash[74942]: audit 2024-04-07T17:46:01.875943+0000 mon.a (mon.0) 4488 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:46:02.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:46:02 smithi100 bash[76202]: cluster 2024-04-07T17:46:01.487833+0000 mon.a (mon.0) 4487 : cluster [DBG] osdmap e2498: 8 total, 8 up, 8 in 2024-04-07T17:46:02.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:46:02 smithi100 bash[76202]: audit 2024-04-07T17:46:01.875943+0000 mon.a (mon.0) 4488 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:46:03.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:46:03 smithi144 bash[66654]: cluster 2024-04-07T17:46:02.491290+0000 mon.a (mon.0) 4489 : cluster [DBG] osdmap e2499: 8 total, 8 up, 8 in 2024-04-07T17:46:03.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:46:03 smithi144 bash[66654]: cluster 2024-04-07T17:46:02.695208+0000 mgr.y (mgr.25175) 2720 : cluster [DBG] pgmap v4794: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 696 KiB/s rd, 5.0 KiB/s wr, 942 op/s 2024-04-07T17:46:03.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:46:03 smithi144 bash[66654]: cluster 2024-04-07T17:46:03.420801+0000 mon.a (mon.0) 4490 : cluster [DBG] osdmap e2500: 8 total, 8 up, 8 in 2024-04-07T17:46:03.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:46:03 smithi100 bash[74942]: cluster 2024-04-07T17:46:02.491290+0000 mon.a (mon.0) 4489 : cluster [DBG] osdmap e2499: 8 total, 8 up, 8 in 2024-04-07T17:46:03.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:46:03 smithi100 bash[74942]: cluster 2024-04-07T17:46:02.695208+0000 mgr.y (mgr.25175) 2720 : cluster [DBG] pgmap v4794: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 696 KiB/s rd, 5.0 KiB/s wr, 942 op/s 2024-04-07T17:46:03.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:46:03 smithi100 bash[74942]: cluster 2024-04-07T17:46:03.420801+0000 mon.a (mon.0) 4490 : cluster [DBG] osdmap e2500: 8 total, 8 up, 8 in 2024-04-07T17:46:03.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:46:03 smithi100 bash[76202]: cluster 2024-04-07T17:46:02.491290+0000 mon.a (mon.0) 4489 : cluster [DBG] osdmap e2499: 8 total, 8 up, 8 in 2024-04-07T17:46:03.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:46:03 smithi100 bash[76202]: cluster 2024-04-07T17:46:02.695208+0000 mgr.y (mgr.25175) 2720 : cluster [DBG] pgmap v4794: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 696 KiB/s rd, 5.0 KiB/s wr, 942 op/s 2024-04-07T17:46:03.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:46:03 smithi100 bash[76202]: cluster 2024-04-07T17:46:03.420801+0000 mon.a (mon.0) 4490 : cluster [DBG] osdmap e2500: 8 total, 8 up, 8 in 2024-04-07T17:46:05.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:46:05 smithi144 bash[66654]: cluster 2024-04-07T17:46:04.504794+0000 mon.a (mon.0) 4491 : cluster [DBG] osdmap e2501: 8 total, 8 up, 8 in 2024-04-07T17:46:05.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:46:05 smithi144 bash[66654]: cluster 2024-04-07T17:46:04.695934+0000 mgr.y (mgr.25175) 2721 : cluster [DBG] pgmap v4797: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 663 KiB/s rd, 5.0 KiB/s wr, 898 op/s 2024-04-07T17:46:05.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:46:05 smithi100 bash[74942]: cluster 2024-04-07T17:46:04.504794+0000 mon.a (mon.0) 4491 : cluster [DBG] osdmap e2501: 8 total, 8 up, 8 in 2024-04-07T17:46:05.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:46:05 smithi100 bash[74942]: cluster 2024-04-07T17:46:04.695934+0000 mgr.y (mgr.25175) 2721 : cluster [DBG] pgmap v4797: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 663 KiB/s rd, 5.0 KiB/s wr, 898 op/s 2024-04-07T17:46:05.955 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:46:05 smithi100 bash[76202]: cluster 2024-04-07T17:46:04.504794+0000 mon.a (mon.0) 4491 : cluster [DBG] osdmap e2501: 8 total, 8 up, 8 in 2024-04-07T17:46:05.955 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:46:05 smithi100 bash[76202]: cluster 2024-04-07T17:46:04.695934+0000 mgr.y (mgr.25175) 2721 : cluster [DBG] pgmap v4797: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 663 KiB/s rd, 5.0 KiB/s wr, 898 op/s 2024-04-07T17:46:06.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:46:06 smithi144 bash[66654]: cluster 2024-04-07T17:46:05.515170+0000 mon.a (mon.0) 4492 : cluster [DBG] osdmap e2502: 8 total, 8 up, 8 in 2024-04-07T17:46:06.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:46:06 smithi100 bash[74942]: cluster 2024-04-07T17:46:05.515170+0000 mon.a (mon.0) 4492 : cluster [DBG] osdmap e2502: 8 total, 8 up, 8 in 2024-04-07T17:46:06.953 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:46:06 smithi100 bash[76202]: cluster 2024-04-07T17:46:05.515170+0000 mon.a (mon.0) 4492 : cluster [DBG] osdmap e2502: 8 total, 8 up, 8 in 2024-04-07T17:46:07.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:46:07 smithi144 bash[66654]: cluster 2024-04-07T17:46:06.527004+0000 mon.a (mon.0) 4493 : cluster [DBG] osdmap e2503: 8 total, 8 up, 8 in 2024-04-07T17:46:07.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:46:07 smithi144 bash[66654]: cluster 2024-04-07T17:46:06.696634+0000 mgr.y (mgr.25175) 2722 : cluster [DBG] pgmap v4800: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 660 KiB/s rd, 5.0 KiB/s wr, 893 op/s 2024-04-07T17:46:07.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:46:07 smithi100 bash[74942]: cluster 2024-04-07T17:46:06.527004+0000 mon.a (mon.0) 4493 : cluster [DBG] osdmap e2503: 8 total, 8 up, 8 in 2024-04-07T17:46:07.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:46:07 smithi100 bash[74942]: cluster 2024-04-07T17:46:06.696634+0000 mgr.y (mgr.25175) 2722 : cluster [DBG] pgmap v4800: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 660 KiB/s rd, 5.0 KiB/s wr, 893 op/s 2024-04-07T17:46:07.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:46:07 smithi100 bash[76202]: cluster 2024-04-07T17:46:06.527004+0000 mon.a (mon.0) 4493 : cluster [DBG] osdmap e2503: 8 total, 8 up, 8 in 2024-04-07T17:46:07.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:46:07 smithi100 bash[76202]: cluster 2024-04-07T17:46:06.696634+0000 mgr.y (mgr.25175) 2722 : cluster [DBG] pgmap v4800: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 660 KiB/s rd, 5.0 KiB/s wr, 893 op/s 2024-04-07T17:46:08.703 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:46:08 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:46:08] "GET /metrics HTTP/1.1" 200 36749 "" "Prometheus/2.43.0" 2024-04-07T17:46:08.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:46:08 smithi100 bash[74942]: cluster 2024-04-07T17:46:07.543195+0000 mon.a (mon.0) 4494 : cluster [DBG] osdmap e2504: 8 total, 8 up, 8 in 2024-04-07T17:46:08.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:46:08 smithi100 bash[74942]: cluster 2024-04-07T17:46:08.425057+0000 mon.a (mon.0) 4495 : cluster [DBG] osdmap e2505: 8 total, 8 up, 8 in 2024-04-07T17:46:08.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:46:08 smithi100 bash[76202]: cluster 2024-04-07T17:46:07.543195+0000 mon.a (mon.0) 4494 : cluster [DBG] osdmap e2504: 8 total, 8 up, 8 in 2024-04-07T17:46:08.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:46:08 smithi100 bash[76202]: cluster 2024-04-07T17:46:08.425057+0000 mon.a (mon.0) 4495 : cluster [DBG] osdmap e2505: 8 total, 8 up, 8 in 2024-04-07T17:46:08.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:46:08 smithi144 bash[66654]: cluster 2024-04-07T17:46:07.543195+0000 mon.a (mon.0) 4494 : cluster [DBG] osdmap e2504: 8 total, 8 up, 8 in 2024-04-07T17:46:08.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:46:08 smithi144 bash[66654]: cluster 2024-04-07T17:46:08.425057+0000 mon.a (mon.0) 4495 : cluster [DBG] osdmap e2505: 8 total, 8 up, 8 in 2024-04-07T17:46:09.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:46:09 smithi144 bash[66654]: cluster 2024-04-07T17:46:08.697469+0000 mgr.y (mgr.25175) 2723 : cluster [DBG] pgmap v4803: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 3.0 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-07T17:46:09.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:46:09 smithi144 bash[66654]: cluster 2024-04-07T17:46:09.430825+0000 mon.a (mon.0) 4496 : cluster [DBG] osdmap e2506: 8 total, 8 up, 8 in 2024-04-07T17:46:09.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:46:09 smithi100 bash[74942]: cluster 2024-04-07T17:46:08.697469+0000 mgr.y (mgr.25175) 2723 : cluster [DBG] pgmap v4803: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 3.0 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-07T17:46:09.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:46:09 smithi100 bash[74942]: cluster 2024-04-07T17:46:09.430825+0000 mon.a (mon.0) 4496 : cluster [DBG] osdmap e2506: 8 total, 8 up, 8 in 2024-04-07T17:46:09.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:46:09 smithi100 bash[76202]: cluster 2024-04-07T17:46:08.697469+0000 mgr.y (mgr.25175) 2723 : cluster [DBG] pgmap v4803: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 3.0 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-07T17:46:09.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:46:09 smithi100 bash[76202]: cluster 2024-04-07T17:46:09.430825+0000 mon.a (mon.0) 4496 : cluster [DBG] osdmap e2506: 8 total, 8 up, 8 in 2024-04-07T17:46:11.703 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:46:11 smithi100 bash[74942]: cluster 2024-04-07T17:46:10.429892+0000 mon.a (mon.0) 4497 : cluster [DBG] osdmap e2507: 8 total, 8 up, 8 in 2024-04-07T17:46:11.703 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:46:11 smithi100 bash[74942]: cluster 2024-04-07T17:46:10.698227+0000 mgr.y (mgr.25175) 2724 : cluster [DBG] pgmap v4806: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 630 KiB/s rd, 5.0 KiB/s wr, 855 op/s 2024-04-07T17:46:11.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:46:11 smithi100 bash[76202]: cluster 2024-04-07T17:46:10.429892+0000 mon.a (mon.0) 4497 : cluster [DBG] osdmap e2507: 8 total, 8 up, 8 in 2024-04-07T17:46:11.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:46:11 smithi100 bash[76202]: cluster 2024-04-07T17:46:10.698227+0000 mgr.y (mgr.25175) 2724 : cluster [DBG] pgmap v4806: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 630 KiB/s rd, 5.0 KiB/s wr, 855 op/s 2024-04-07T17:46:11.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:46:11 smithi144 bash[66654]: cluster 2024-04-07T17:46:10.429892+0000 mon.a (mon.0) 4497 : cluster [DBG] osdmap e2507: 8 total, 8 up, 8 in 2024-04-07T17:46:11.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:46:11 smithi144 bash[66654]: cluster 2024-04-07T17:46:10.698227+0000 mgr.y (mgr.25175) 2724 : cluster [DBG] pgmap v4806: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 630 KiB/s rd, 5.0 KiB/s wr, 855 op/s 2024-04-07T17:46:12.614 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:46:12 smithi144 bash[66654]: cluster 2024-04-07T17:46:11.444422+0000 mon.a (mon.0) 4498 : cluster [DBG] osdmap e2508: 8 total, 8 up, 8 in 2024-04-07T17:46:12.615 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:46:12 smithi144 bash[95233]: ts=2024-04-07T17:46:12.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:46:12.703 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:46:12 smithi100 bash[74942]: cluster 2024-04-07T17:46:11.444422+0000 mon.a (mon.0) 4498 : cluster [DBG] osdmap e2508: 8 total, 8 up, 8 in 2024-04-07T17:46:12.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:46:12 smithi100 bash[76202]: cluster 2024-04-07T17:46:11.444422+0000 mon.a (mon.0) 4498 : cluster [DBG] osdmap e2508: 8 total, 8 up, 8 in 2024-04-07T17:46:13.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:46:13 smithi144 bash[66654]: cluster 2024-04-07T17:46:12.451336+0000 mon.a (mon.0) 4499 : cluster [DBG] osdmap e2509: 8 total, 8 up, 8 in 2024-04-07T17:46:13.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:46:13 smithi144 bash[66654]: cluster 2024-04-07T17:46:12.698950+0000 mgr.y (mgr.25175) 2725 : cluster [DBG] pgmap v4809: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 630 KiB/s rd, 5.0 KiB/s wr, 853 op/s 2024-04-07T17:46:13.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:46:13 smithi144 bash[66654]: cluster 2024-04-07T17:46:13.426159+0000 mon.a (mon.0) 4500 : cluster [DBG] osdmap e2510: 8 total, 8 up, 8 in 2024-04-07T17:46:13.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:46:13 smithi100 bash[74942]: cluster 2024-04-07T17:46:12.451336+0000 mon.a (mon.0) 4499 : cluster [DBG] osdmap e2509: 8 total, 8 up, 8 in 2024-04-07T17:46:13.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:46:13 smithi100 bash[74942]: cluster 2024-04-07T17:46:12.698950+0000 mgr.y (mgr.25175) 2725 : cluster [DBG] pgmap v4809: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 630 KiB/s rd, 5.0 KiB/s wr, 853 op/s 2024-04-07T17:46:13.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:46:13 smithi100 bash[74942]: cluster 2024-04-07T17:46:13.426159+0000 mon.a (mon.0) 4500 : cluster [DBG] osdmap e2510: 8 total, 8 up, 8 in 2024-04-07T17:46:13.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:46:13 smithi100 bash[76202]: cluster 2024-04-07T17:46:12.451336+0000 mon.a (mon.0) 4499 : cluster [DBG] osdmap e2509: 8 total, 8 up, 8 in 2024-04-07T17:46:13.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:46:13 smithi100 bash[76202]: cluster 2024-04-07T17:46:12.698950+0000 mgr.y (mgr.25175) 2725 : cluster [DBG] pgmap v4809: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 630 KiB/s rd, 5.0 KiB/s wr, 853 op/s 2024-04-07T17:46:13.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:46:13 smithi100 bash[76202]: cluster 2024-04-07T17:46:13.426159+0000 mon.a (mon.0) 4500 : cluster [DBG] osdmap e2510: 8 total, 8 up, 8 in 2024-04-07T17:46:15.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:46:15 smithi144 bash[66654]: cluster 2024-04-07T17:46:14.466895+0000 mon.a (mon.0) 4501 : cluster [DBG] osdmap e2511: 8 total, 8 up, 8 in 2024-04-07T17:46:15.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:46:15 smithi144 bash[66654]: cluster 2024-04-07T17:46:14.699788+0000 mgr.y (mgr.25175) 2726 : cluster [DBG] pgmap v4812: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 598 KiB/s rd, 5.0 KiB/s wr, 810 op/s 2024-04-07T17:46:15.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:46:15 smithi100 bash[74942]: cluster 2024-04-07T17:46:14.466895+0000 mon.a (mon.0) 4501 : cluster [DBG] osdmap e2511: 8 total, 8 up, 8 in 2024-04-07T17:46:15.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:46:15 smithi100 bash[74942]: cluster 2024-04-07T17:46:14.699788+0000 mgr.y (mgr.25175) 2726 : cluster [DBG] pgmap v4812: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 598 KiB/s rd, 5.0 KiB/s wr, 810 op/s 2024-04-07T17:46:15.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:46:15 smithi100 bash[76202]: cluster 2024-04-07T17:46:14.466895+0000 mon.a (mon.0) 4501 : cluster [DBG] osdmap e2511: 8 total, 8 up, 8 in 2024-04-07T17:46:15.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:46:15 smithi100 bash[76202]: cluster 2024-04-07T17:46:14.699788+0000 mgr.y (mgr.25175) 2726 : cluster [DBG] pgmap v4812: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 598 KiB/s rd, 5.0 KiB/s wr, 810 op/s 2024-04-07T17:46:16.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:46:16 smithi144 bash[66654]: cluster 2024-04-07T17:46:15.476654+0000 mon.a (mon.0) 4502 : cluster [DBG] osdmap e2512: 8 total, 8 up, 8 in 2024-04-07T17:46:16.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:46:16 smithi100 bash[74942]: cluster 2024-04-07T17:46:15.476654+0000 mon.a (mon.0) 4502 : cluster [DBG] osdmap e2512: 8 total, 8 up, 8 in 2024-04-07T17:46:16.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:46:16 smithi100 bash[76202]: cluster 2024-04-07T17:46:15.476654+0000 mon.a (mon.0) 4502 : cluster [DBG] osdmap e2512: 8 total, 8 up, 8 in 2024-04-07T17:46:17.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:46:17 smithi144 bash[66654]: cluster 2024-04-07T17:46:16.478067+0000 mon.a (mon.0) 4503 : cluster [DBG] osdmap e2513: 8 total, 8 up, 8 in 2024-04-07T17:46:17.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:46:17 smithi144 bash[66654]: cluster 2024-04-07T17:46:16.700486+0000 mgr.y (mgr.25175) 2727 : cluster [DBG] pgmap v4815: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 596 KiB/s rd, 5.0 KiB/s wr, 807 op/s 2024-04-07T17:46:17.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:46:17 smithi144 bash[66654]: audit 2024-04-07T17:46:16.876509+0000 mon.a (mon.0) 4504 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:46:17.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:46:17 smithi100 bash[74942]: cluster 2024-04-07T17:46:16.478067+0000 mon.a (mon.0) 4503 : cluster [DBG] osdmap e2513: 8 total, 8 up, 8 in 2024-04-07T17:46:17.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:46:17 smithi100 bash[74942]: cluster 2024-04-07T17:46:16.700486+0000 mgr.y (mgr.25175) 2727 : cluster [DBG] pgmap v4815: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 596 KiB/s rd, 5.0 KiB/s wr, 807 op/s 2024-04-07T17:46:17.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:46:17 smithi100 bash[74942]: audit 2024-04-07T17:46:16.876509+0000 mon.a (mon.0) 4504 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:46:17.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:46:17 smithi100 bash[76202]: cluster 2024-04-07T17:46:16.478067+0000 mon.a (mon.0) 4503 : cluster [DBG] osdmap e2513: 8 total, 8 up, 8 in 2024-04-07T17:46:17.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:46:17 smithi100 bash[76202]: cluster 2024-04-07T17:46:16.700486+0000 mgr.y (mgr.25175) 2727 : cluster [DBG] pgmap v4815: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 596 KiB/s rd, 5.0 KiB/s wr, 807 op/s 2024-04-07T17:46:17.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:46:17 smithi100 bash[76202]: audit 2024-04-07T17:46:16.876509+0000 mon.a (mon.0) 4504 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:46:18.703 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:46:18 smithi100 bash[74942]: cluster 2024-04-07T17:46:17.504906+0000 mon.a (mon.0) 4505 : cluster [DBG] osdmap e2514: 8 total, 8 up, 8 in 2024-04-07T17:46:18.703 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:46:18 smithi100 bash[74942]: cluster 2024-04-07T17:46:18.430059+0000 mon.a (mon.0) 4506 : cluster [DBG] osdmap e2515: 8 total, 8 up, 8 in 2024-04-07T17:46:18.704 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:46:18 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:46:18] "GET /metrics HTTP/1.1" 200 36746 "" "Prometheus/2.43.0" 2024-04-07T17:46:18.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:46:18 smithi100 bash[76202]: cluster 2024-04-07T17:46:17.504906+0000 mon.a (mon.0) 4505 : cluster [DBG] osdmap e2514: 8 total, 8 up, 8 in 2024-04-07T17:46:18.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:46:18 smithi100 bash[76202]: cluster 2024-04-07T17:46:18.430059+0000 mon.a (mon.0) 4506 : cluster [DBG] osdmap e2515: 8 total, 8 up, 8 in 2024-04-07T17:46:18.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:46:18 smithi144 bash[66654]: cluster 2024-04-07T17:46:17.504906+0000 mon.a (mon.0) 4505 : cluster [DBG] osdmap e2514: 8 total, 8 up, 8 in 2024-04-07T17:46:18.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:46:18 smithi144 bash[66654]: cluster 2024-04-07T17:46:18.430059+0000 mon.a (mon.0) 4506 : cluster [DBG] osdmap e2515: 8 total, 8 up, 8 in 2024-04-07T17:46:19.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:46:19 smithi144 bash[66654]: cluster 2024-04-07T17:46:18.701135+0000 mgr.y (mgr.25175) 2728 : cluster [DBG] pgmap v4818: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 2.7 KiB/s rd, 0 B/s wr, 4 op/s 2024-04-07T17:46:19.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:46:19 smithi100 bash[74942]: cluster 2024-04-07T17:46:18.701135+0000 mgr.y (mgr.25175) 2728 : cluster [DBG] pgmap v4818: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 2.7 KiB/s rd, 0 B/s wr, 4 op/s 2024-04-07T17:46:19.953 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:46:19 smithi100 bash[76202]: cluster 2024-04-07T17:46:18.701135+0000 mgr.y (mgr.25175) 2728 : cluster [DBG] pgmap v4818: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 2.7 KiB/s rd, 0 B/s wr, 4 op/s 2024-04-07T17:46:20.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:46:20 smithi144 bash[66654]: cluster 2024-04-07T17:46:19.514266+0000 mon.a (mon.0) 4507 : cluster [DBG] osdmap e2516: 8 total, 8 up, 8 in 2024-04-07T17:46:20.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:46:20 smithi100 bash[74942]: cluster 2024-04-07T17:46:19.514266+0000 mon.a (mon.0) 4507 : cluster [DBG] osdmap e2516: 8 total, 8 up, 8 in 2024-04-07T17:46:20.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:46:20 smithi100 bash[76202]: cluster 2024-04-07T17:46:19.514266+0000 mon.a (mon.0) 4507 : cluster [DBG] osdmap e2516: 8 total, 8 up, 8 in 2024-04-07T17:46:21.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:46:21 smithi144 bash[66654]: cluster 2024-04-07T17:46:20.515368+0000 mon.a (mon.0) 4508 : cluster [DBG] osdmap e2517: 8 total, 8 up, 8 in 2024-04-07T17:46:21.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:46:21 smithi144 bash[66654]: cluster 2024-04-07T17:46:20.701857+0000 mgr.y (mgr.25175) 2729 : cluster [DBG] pgmap v4821: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 565 KiB/s rd, 5.0 KiB/s wr, 767 op/s 2024-04-07T17:46:21.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:46:21 smithi100 bash[74942]: cluster 2024-04-07T17:46:20.515368+0000 mon.a (mon.0) 4508 : cluster [DBG] osdmap e2517: 8 total, 8 up, 8 in 2024-04-07T17:46:21.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:46:21 smithi100 bash[74942]: cluster 2024-04-07T17:46:20.701857+0000 mgr.y (mgr.25175) 2729 : cluster [DBG] pgmap v4821: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 565 KiB/s rd, 5.0 KiB/s wr, 767 op/s 2024-04-07T17:46:21.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:46:21 smithi100 bash[76202]: cluster 2024-04-07T17:46:20.515368+0000 mon.a (mon.0) 4508 : cluster [DBG] osdmap e2517: 8 total, 8 up, 8 in 2024-04-07T17:46:21.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:46:21 smithi100 bash[76202]: cluster 2024-04-07T17:46:20.701857+0000 mgr.y (mgr.25175) 2729 : cluster [DBG] pgmap v4821: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 565 KiB/s rd, 5.0 KiB/s wr, 767 op/s 2024-04-07T17:46:22.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:46:22 smithi144 bash[66654]: cluster 2024-04-07T17:46:21.537886+0000 mon.a (mon.0) 4509 : cluster [DBG] osdmap e2518: 8 total, 8 up, 8 in 2024-04-07T17:46:22.615 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:46:22 smithi144 bash[95233]: ts=2024-04-07T17:46:22.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:46:22.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:46:22 smithi100 bash[74942]: cluster 2024-04-07T17:46:21.537886+0000 mon.a (mon.0) 4509 : cluster [DBG] osdmap e2518: 8 total, 8 up, 8 in 2024-04-07T17:46:22.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:46:22 smithi100 bash[76202]: cluster 2024-04-07T17:46:21.537886+0000 mon.a (mon.0) 4509 : cluster [DBG] osdmap e2518: 8 total, 8 up, 8 in 2024-04-07T17:46:23.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:46:23 smithi144 bash[66654]: cluster 2024-04-07T17:46:22.538329+0000 mon.a (mon.0) 4510 : cluster [DBG] osdmap e2519: 8 total, 8 up, 8 in 2024-04-07T17:46:23.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:46:23 smithi144 bash[66654]: cluster 2024-04-07T17:46:22.702609+0000 mgr.y (mgr.25175) 2730 : cluster [DBG] pgmap v4824: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 565 KiB/s rd, 5.0 KiB/s wr, 766 op/s 2024-04-07T17:46:23.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:46:23 smithi144 bash[66654]: cluster 2024-04-07T17:46:23.432737+0000 mon.a (mon.0) 4511 : cluster [DBG] osdmap e2520: 8 total, 8 up, 8 in 2024-04-07T17:46:23.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:46:23 smithi100 bash[74942]: cluster 2024-04-07T17:46:22.538329+0000 mon.a (mon.0) 4510 : cluster [DBG] osdmap e2519: 8 total, 8 up, 8 in 2024-04-07T17:46:23.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:46:23 smithi100 bash[74942]: cluster 2024-04-07T17:46:22.702609+0000 mgr.y (mgr.25175) 2730 : cluster [DBG] pgmap v4824: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 565 KiB/s rd, 5.0 KiB/s wr, 766 op/s 2024-04-07T17:46:23.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:46:23 smithi100 bash[74942]: cluster 2024-04-07T17:46:23.432737+0000 mon.a (mon.0) 4511 : cluster [DBG] osdmap e2520: 8 total, 8 up, 8 in 2024-04-07T17:46:23.955 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:46:23 smithi100 bash[76202]: cluster 2024-04-07T17:46:22.538329+0000 mon.a (mon.0) 4510 : cluster [DBG] osdmap e2519: 8 total, 8 up, 8 in 2024-04-07T17:46:23.955 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:46:23 smithi100 bash[76202]: cluster 2024-04-07T17:46:22.702609+0000 mgr.y (mgr.25175) 2730 : cluster [DBG] pgmap v4824: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 565 KiB/s rd, 5.0 KiB/s wr, 766 op/s 2024-04-07T17:46:23.955 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:46:23 smithi100 bash[76202]: cluster 2024-04-07T17:46:23.432737+0000 mon.a (mon.0) 4511 : cluster [DBG] osdmap e2520: 8 total, 8 up, 8 in 2024-04-07T17:46:25.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:46:25 smithi144 bash[66654]: cluster 2024-04-07T17:46:24.559827+0000 mon.a (mon.0) 4512 : cluster [DBG] osdmap e2521: 8 total, 8 up, 8 in 2024-04-07T17:46:25.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:46:25 smithi144 bash[66654]: cluster 2024-04-07T17:46:24.703333+0000 mgr.y (mgr.25175) 2731 : cluster [DBG] pgmap v4827: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 533 KiB/s rd, 5.0 KiB/s wr, 721 op/s 2024-04-07T17:46:25.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:46:25 smithi100 bash[74942]: cluster 2024-04-07T17:46:24.559827+0000 mon.a (mon.0) 4512 : cluster [DBG] osdmap e2521: 8 total, 8 up, 8 in 2024-04-07T17:46:25.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:46:25 smithi100 bash[74942]: cluster 2024-04-07T17:46:24.703333+0000 mgr.y (mgr.25175) 2731 : cluster [DBG] pgmap v4827: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 533 KiB/s rd, 5.0 KiB/s wr, 721 op/s 2024-04-07T17:46:25.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:46:25 smithi100 bash[76202]: cluster 2024-04-07T17:46:24.559827+0000 mon.a (mon.0) 4512 : cluster [DBG] osdmap e2521: 8 total, 8 up, 8 in 2024-04-07T17:46:25.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:46:25 smithi100 bash[76202]: cluster 2024-04-07T17:46:24.703333+0000 mgr.y (mgr.25175) 2731 : cluster [DBG] pgmap v4827: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 533 KiB/s rd, 5.0 KiB/s wr, 721 op/s 2024-04-07T17:46:26.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:46:26 smithi144 bash[66654]: cluster 2024-04-07T17:46:25.570236+0000 mon.a (mon.0) 4513 : cluster [DBG] osdmap e2522: 8 total, 8 up, 8 in 2024-04-07T17:46:26.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:46:26 smithi100 bash[74942]: cluster 2024-04-07T17:46:25.570236+0000 mon.a (mon.0) 4513 : cluster [DBG] osdmap e2522: 8 total, 8 up, 8 in 2024-04-07T17:46:26.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:46:26 smithi100 bash[76202]: cluster 2024-04-07T17:46:25.570236+0000 mon.a (mon.0) 4513 : cluster [DBG] osdmap e2522: 8 total, 8 up, 8 in 2024-04-07T17:46:27.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:46:27 smithi144 bash[66654]: cluster 2024-04-07T17:46:26.571463+0000 mon.a (mon.0) 4514 : cluster [DBG] osdmap e2523: 8 total, 8 up, 8 in 2024-04-07T17:46:27.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:46:27 smithi144 bash[66654]: cluster 2024-04-07T17:46:26.704092+0000 mgr.y (mgr.25175) 2732 : cluster [DBG] pgmap v4830: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 532 KiB/s rd, 5.0 KiB/s wr, 720 op/s 2024-04-07T17:46:27.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:46:27 smithi100 bash[74942]: cluster 2024-04-07T17:46:26.571463+0000 mon.a (mon.0) 4514 : cluster [DBG] osdmap e2523: 8 total, 8 up, 8 in 2024-04-07T17:46:27.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:46:27 smithi100 bash[74942]: cluster 2024-04-07T17:46:26.704092+0000 mgr.y (mgr.25175) 2732 : cluster [DBG] pgmap v4830: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 532 KiB/s rd, 5.0 KiB/s wr, 720 op/s 2024-04-07T17:46:27.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:46:27 smithi100 bash[76202]: cluster 2024-04-07T17:46:26.571463+0000 mon.a (mon.0) 4514 : cluster [DBG] osdmap e2523: 8 total, 8 up, 8 in 2024-04-07T17:46:27.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:46:27 smithi100 bash[76202]: cluster 2024-04-07T17:46:26.704092+0000 mgr.y (mgr.25175) 2732 : cluster [DBG] pgmap v4830: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 532 KiB/s rd, 5.0 KiB/s wr, 720 op/s 2024-04-07T17:46:28.703 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:46:28 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:46:28] "GET /metrics HTTP/1.1" 200 36746 "" "Prometheus/2.43.0" 2024-04-07T17:46:28.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:46:28 smithi100 bash[74942]: cluster 2024-04-07T17:46:27.598276+0000 mon.a (mon.0) 4515 : cluster [DBG] osdmap e2524: 8 total, 8 up, 8 in 2024-04-07T17:46:28.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:46:28 smithi100 bash[74942]: cluster 2024-04-07T17:46:28.435282+0000 mon.a (mon.0) 4516 : cluster [DBG] osdmap e2525: 8 total, 8 up, 8 in 2024-04-07T17:46:28.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:46:28 smithi100 bash[76202]: cluster 2024-04-07T17:46:27.598276+0000 mon.a (mon.0) 4515 : cluster [DBG] osdmap e2524: 8 total, 8 up, 8 in 2024-04-07T17:46:28.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:46:28 smithi100 bash[76202]: cluster 2024-04-07T17:46:28.435282+0000 mon.a (mon.0) 4516 : cluster [DBG] osdmap e2525: 8 total, 8 up, 8 in 2024-04-07T17:46:28.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:46:28 smithi144 bash[66654]: cluster 2024-04-07T17:46:27.598276+0000 mon.a (mon.0) 4515 : cluster [DBG] osdmap e2524: 8 total, 8 up, 8 in 2024-04-07T17:46:28.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:46:28 smithi144 bash[66654]: cluster 2024-04-07T17:46:28.435282+0000 mon.a (mon.0) 4516 : cluster [DBG] osdmap e2525: 8 total, 8 up, 8 in 2024-04-07T17:46:29.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:46:29 smithi144 bash[66654]: cluster 2024-04-07T17:46:28.704823+0000 mgr.y (mgr.25175) 2733 : cluster [DBG] pgmap v4833: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 3.5 KiB/s rd, 0 B/s wr, 6 op/s 2024-04-07T17:46:29.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:46:29 smithi144 bash[66654]: cluster 2024-04-07T17:46:29.453475+0000 mon.a (mon.0) 4517 : cluster [DBG] osdmap e2526: 8 total, 8 up, 8 in 2024-04-07T17:46:29.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:46:29 smithi100 bash[74942]: cluster 2024-04-07T17:46:28.704823+0000 mgr.y (mgr.25175) 2733 : cluster [DBG] pgmap v4833: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 3.5 KiB/s rd, 0 B/s wr, 6 op/s 2024-04-07T17:46:29.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:46:29 smithi100 bash[74942]: cluster 2024-04-07T17:46:29.453475+0000 mon.a (mon.0) 4517 : cluster [DBG] osdmap e2526: 8 total, 8 up, 8 in 2024-04-07T17:46:29.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:46:29 smithi100 bash[76202]: cluster 2024-04-07T17:46:28.704823+0000 mgr.y (mgr.25175) 2733 : cluster [DBG] pgmap v4833: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 3.5 KiB/s rd, 0 B/s wr, 6 op/s 2024-04-07T17:46:29.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:46:29 smithi100 bash[76202]: cluster 2024-04-07T17:46:29.453475+0000 mon.a (mon.0) 4517 : cluster [DBG] osdmap e2526: 8 total, 8 up, 8 in 2024-04-07T17:46:31.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:46:31 smithi144 bash[66654]: cluster 2024-04-07T17:46:30.455403+0000 mon.a (mon.0) 4518 : cluster [DBG] osdmap e2527: 8 total, 8 up, 8 in 2024-04-07T17:46:31.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:46:31 smithi144 bash[66654]: cluster 2024-04-07T17:46:30.705652+0000 mgr.y (mgr.25175) 2734 : cluster [DBG] pgmap v4836: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 500 KiB/s rd, 5.0 KiB/s wr, 678 op/s 2024-04-07T17:46:31.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:46:31 smithi100 bash[74942]: cluster 2024-04-07T17:46:30.455403+0000 mon.a (mon.0) 4518 : cluster [DBG] osdmap e2527: 8 total, 8 up, 8 in 2024-04-07T17:46:31.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:46:31 smithi100 bash[74942]: cluster 2024-04-07T17:46:30.705652+0000 mgr.y (mgr.25175) 2734 : cluster [DBG] pgmap v4836: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 500 KiB/s rd, 5.0 KiB/s wr, 678 op/s 2024-04-07T17:46:31.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:46:31 smithi100 bash[76202]: cluster 2024-04-07T17:46:30.455403+0000 mon.a (mon.0) 4518 : cluster [DBG] osdmap e2527: 8 total, 8 up, 8 in 2024-04-07T17:46:31.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:46:31 smithi100 bash[76202]: cluster 2024-04-07T17:46:30.705652+0000 mgr.y (mgr.25175) 2734 : cluster [DBG] pgmap v4836: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 500 KiB/s rd, 5.0 KiB/s wr, 678 op/s 2024-04-07T17:46:32.614 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:46:32 smithi144 bash[66654]: cluster 2024-04-07T17:46:31.473558+0000 mon.a (mon.0) 4519 : cluster [DBG] osdmap e2528: 8 total, 8 up, 8 in 2024-04-07T17:46:32.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:46:32 smithi144 bash[66654]: audit 2024-04-07T17:46:31.877131+0000 mon.a (mon.0) 4520 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:46:32.615 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:46:32 smithi144 bash[95233]: ts=2024-04-07T17:46:32.317Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:46:32.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:46:32 smithi100 bash[74942]: cluster 2024-04-07T17:46:31.473558+0000 mon.a (mon.0) 4519 : cluster [DBG] osdmap e2528: 8 total, 8 up, 8 in 2024-04-07T17:46:32.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:46:32 smithi100 bash[74942]: audit 2024-04-07T17:46:31.877131+0000 mon.a (mon.0) 4520 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:46:32.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:46:32 smithi100 bash[76202]: cluster 2024-04-07T17:46:31.473558+0000 mon.a (mon.0) 4519 : cluster [DBG] osdmap e2528: 8 total, 8 up, 8 in 2024-04-07T17:46:32.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:46:32 smithi100 bash[76202]: audit 2024-04-07T17:46:31.877131+0000 mon.a (mon.0) 4520 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:46:33.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:46:33 smithi144 bash[66654]: cluster 2024-04-07T17:46:32.480669+0000 mon.a (mon.0) 4521 : cluster [DBG] osdmap e2529: 8 total, 8 up, 8 in 2024-04-07T17:46:33.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:46:33 smithi144 bash[66654]: cluster 2024-04-07T17:46:32.706412+0000 mgr.y (mgr.25175) 2735 : cluster [DBG] pgmap v4839: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 499 KiB/s rd, 5.0 KiB/s wr, 677 op/s 2024-04-07T17:46:33.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:46:33 smithi144 bash[66654]: cluster 2024-04-07T17:46:33.437517+0000 mon.a (mon.0) 4522 : cluster [DBG] osdmap e2530: 8 total, 8 up, 8 in 2024-04-07T17:46:33.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:46:33 smithi100 bash[74942]: cluster 2024-04-07T17:46:32.480669+0000 mon.a (mon.0) 4521 : cluster [DBG] osdmap e2529: 8 total, 8 up, 8 in 2024-04-07T17:46:33.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:46:33 smithi100 bash[74942]: cluster 2024-04-07T17:46:32.706412+0000 mgr.y (mgr.25175) 2735 : cluster [DBG] pgmap v4839: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 499 KiB/s rd, 5.0 KiB/s wr, 677 op/s 2024-04-07T17:46:33.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:46:33 smithi100 bash[74942]: cluster 2024-04-07T17:46:33.437517+0000 mon.a (mon.0) 4522 : cluster [DBG] osdmap e2530: 8 total, 8 up, 8 in 2024-04-07T17:46:33.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:46:33 smithi100 bash[76202]: cluster 2024-04-07T17:46:32.480669+0000 mon.a (mon.0) 4521 : cluster [DBG] osdmap e2529: 8 total, 8 up, 8 in 2024-04-07T17:46:33.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:46:33 smithi100 bash[76202]: cluster 2024-04-07T17:46:32.706412+0000 mgr.y (mgr.25175) 2735 : cluster [DBG] pgmap v4839: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 499 KiB/s rd, 5.0 KiB/s wr, 677 op/s 2024-04-07T17:46:33.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:46:33 smithi100 bash[76202]: cluster 2024-04-07T17:46:33.437517+0000 mon.a (mon.0) 4522 : cluster [DBG] osdmap e2530: 8 total, 8 up, 8 in 2024-04-07T17:46:35.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:46:35 smithi144 bash[66654]: cluster 2024-04-07T17:46:34.494044+0000 mon.a (mon.0) 4523 : cluster [DBG] osdmap e2531: 8 total, 8 up, 8 in 2024-04-07T17:46:35.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:46:35 smithi144 bash[66654]: cluster 2024-04-07T17:46:34.707181+0000 mgr.y (mgr.25175) 2736 : cluster [DBG] pgmap v4842: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 469 KiB/s rd, 5.0 KiB/s wr, 636 op/s 2024-04-07T17:46:35.955 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:46:35 smithi100 bash[74942]: cluster 2024-04-07T17:46:34.494044+0000 mon.a (mon.0) 4523 : cluster [DBG] osdmap e2531: 8 total, 8 up, 8 in 2024-04-07T17:46:35.955 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:46:35 smithi100 bash[74942]: cluster 2024-04-07T17:46:34.707181+0000 mgr.y (mgr.25175) 2736 : cluster [DBG] pgmap v4842: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 469 KiB/s rd, 5.0 KiB/s wr, 636 op/s 2024-04-07T17:46:35.956 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:46:35 smithi100 bash[76202]: cluster 2024-04-07T17:46:34.494044+0000 mon.a (mon.0) 4523 : cluster [DBG] osdmap e2531: 8 total, 8 up, 8 in 2024-04-07T17:46:35.956 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:46:35 smithi100 bash[76202]: cluster 2024-04-07T17:46:34.707181+0000 mgr.y (mgr.25175) 2736 : cluster [DBG] pgmap v4842: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 469 KiB/s rd, 5.0 KiB/s wr, 636 op/s 2024-04-07T17:46:36.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:46:36 smithi144 bash[66654]: cluster 2024-04-07T17:46:35.506395+0000 mon.a (mon.0) 4524 : cluster [DBG] osdmap e2532: 8 total, 8 up, 8 in 2024-04-07T17:46:36.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:46:36 smithi100 bash[74942]: cluster 2024-04-07T17:46:35.506395+0000 mon.a (mon.0) 4524 : cluster [DBG] osdmap e2532: 8 total, 8 up, 8 in 2024-04-07T17:46:36.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:46:36 smithi100 bash[76202]: cluster 2024-04-07T17:46:35.506395+0000 mon.a (mon.0) 4524 : cluster [DBG] osdmap e2532: 8 total, 8 up, 8 in 2024-04-07T17:46:37.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:46:37 smithi144 bash[66654]: cluster 2024-04-07T17:46:36.513200+0000 mon.a (mon.0) 4525 : cluster [DBG] osdmap e2533: 8 total, 8 up, 8 in 2024-04-07T17:46:37.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:46:37 smithi144 bash[66654]: cluster 2024-04-07T17:46:36.707945+0000 mgr.y (mgr.25175) 2737 : cluster [DBG] pgmap v4845: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 466 KiB/s rd, 5.0 KiB/s wr, 632 op/s 2024-04-07T17:46:37.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:46:37 smithi100 bash[74942]: cluster 2024-04-07T17:46:36.513200+0000 mon.a (mon.0) 4525 : cluster [DBG] osdmap e2533: 8 total, 8 up, 8 in 2024-04-07T17:46:37.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:46:37 smithi100 bash[74942]: cluster 2024-04-07T17:46:36.707945+0000 mgr.y (mgr.25175) 2737 : cluster [DBG] pgmap v4845: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 466 KiB/s rd, 5.0 KiB/s wr, 632 op/s 2024-04-07T17:46:37.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:46:37 smithi100 bash[76202]: cluster 2024-04-07T17:46:36.513200+0000 mon.a (mon.0) 4525 : cluster [DBG] osdmap e2533: 8 total, 8 up, 8 in 2024-04-07T17:46:37.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:46:37 smithi100 bash[76202]: cluster 2024-04-07T17:46:36.707945+0000 mgr.y (mgr.25175) 2737 : cluster [DBG] pgmap v4845: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 466 KiB/s rd, 5.0 KiB/s wr, 632 op/s 2024-04-07T17:46:38.703 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:46:38 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:46:38] "GET /metrics HTTP/1.1" 200 36744 "" "Prometheus/2.43.0" 2024-04-07T17:46:38.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:46:38 smithi100 bash[76202]: cluster 2024-04-07T17:46:37.522769+0000 mon.a (mon.0) 4526 : cluster [DBG] osdmap e2534: 8 total, 8 up, 8 in 2024-04-07T17:46:38.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:46:38 smithi100 bash[76202]: audit 2024-04-07T17:46:38.257277+0000 mon.a (mon.0) 4527 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:46:38.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:46:38 smithi100 bash[76202]: cluster 2024-04-07T17:46:38.438977+0000 mon.a (mon.0) 4528 : cluster [DBG] osdmap e2535: 8 total, 8 up, 8 in 2024-04-07T17:46:38.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:46:38 smithi100 bash[74942]: cluster 2024-04-07T17:46:37.522769+0000 mon.a (mon.0) 4526 : cluster [DBG] osdmap e2534: 8 total, 8 up, 8 in 2024-04-07T17:46:38.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:46:38 smithi100 bash[74942]: audit 2024-04-07T17:46:38.257277+0000 mon.a (mon.0) 4527 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:46:38.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:46:38 smithi100 bash[74942]: cluster 2024-04-07T17:46:38.438977+0000 mon.a (mon.0) 4528 : cluster [DBG] osdmap e2535: 8 total, 8 up, 8 in 2024-04-07T17:46:38.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:46:38 smithi144 bash[66654]: cluster 2024-04-07T17:46:37.522769+0000 mon.a (mon.0) 4526 : cluster [DBG] osdmap e2534: 8 total, 8 up, 8 in 2024-04-07T17:46:38.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:46:38 smithi144 bash[66654]: audit 2024-04-07T17:46:38.257277+0000 mon.a (mon.0) 4527 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:46:38.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:46:38 smithi144 bash[66654]: cluster 2024-04-07T17:46:38.438977+0000 mon.a (mon.0) 4528 : cluster [DBG] osdmap e2535: 8 total, 8 up, 8 in 2024-04-07T17:46:40.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:46:39 smithi144 bash[66654]: cluster 2024-04-07T17:46:38.708724+0000 mgr.y (mgr.25175) 2738 : cluster [DBG] pgmap v4848: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 2.0 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-07T17:46:40.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:46:39 smithi144 bash[66654]: audit 2024-04-07T17:46:38.851985+0000 mon.a (mon.0) 4529 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:46:40.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:46:39 smithi144 bash[66654]: audit 2024-04-07T17:46:38.869520+0000 mon.a (mon.0) 4530 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:46:40.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:46:39 smithi144 bash[66654]: cluster 2024-04-07T17:46:39.527528+0000 mon.a (mon.0) 4531 : cluster [DBG] osdmap e2536: 8 total, 8 up, 8 in 2024-04-07T17:46:40.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:46:39 smithi144 bash[66654]: audit 2024-04-07T17:46:39.573531+0000 mon.a (mon.0) 4532 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:46:40.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:46:39 smithi144 bash[66654]: audit 2024-04-07T17:46:39.575408+0000 mon.a (mon.0) 4533 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:46:40.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:46:39 smithi144 bash[66654]: audit 2024-04-07T17:46:39.584432+0000 mon.a (mon.0) 4534 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:46:40.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:46:39 smithi100 bash[74942]: cluster 2024-04-07T17:46:38.708724+0000 mgr.y (mgr.25175) 2738 : cluster [DBG] pgmap v4848: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 2.0 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-07T17:46:40.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:46:39 smithi100 bash[74942]: audit 2024-04-07T17:46:38.851985+0000 mon.a (mon.0) 4529 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:46:40.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:46:39 smithi100 bash[74942]: audit 2024-04-07T17:46:38.869520+0000 mon.a (mon.0) 4530 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:46:40.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:46:39 smithi100 bash[74942]: cluster 2024-04-07T17:46:39.527528+0000 mon.a (mon.0) 4531 : cluster [DBG] osdmap e2536: 8 total, 8 up, 8 in 2024-04-07T17:46:40.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:46:39 smithi100 bash[74942]: audit 2024-04-07T17:46:39.573531+0000 mon.a (mon.0) 4532 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:46:40.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:46:39 smithi100 bash[74942]: audit 2024-04-07T17:46:39.575408+0000 mon.a (mon.0) 4533 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:46:40.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:46:39 smithi100 bash[74942]: audit 2024-04-07T17:46:39.584432+0000 mon.a (mon.0) 4534 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:46:40.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:46:39 smithi100 bash[76202]: cluster 2024-04-07T17:46:38.708724+0000 mgr.y (mgr.25175) 2738 : cluster [DBG] pgmap v4848: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 2.0 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-07T17:46:40.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:46:39 smithi100 bash[76202]: audit 2024-04-07T17:46:38.851985+0000 mon.a (mon.0) 4529 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:46:40.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:46:39 smithi100 bash[76202]: audit 2024-04-07T17:46:38.869520+0000 mon.a (mon.0) 4530 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:46:40.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:46:39 smithi100 bash[76202]: cluster 2024-04-07T17:46:39.527528+0000 mon.a (mon.0) 4531 : cluster [DBG] osdmap e2536: 8 total, 8 up, 8 in 2024-04-07T17:46:40.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:46:39 smithi100 bash[76202]: audit 2024-04-07T17:46:39.573531+0000 mon.a (mon.0) 4532 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:46:40.205 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:46:39 smithi100 bash[76202]: audit 2024-04-07T17:46:39.575408+0000 mon.a (mon.0) 4533 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:46:40.205 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:46:39 smithi100 bash[76202]: audit 2024-04-07T17:46:39.584432+0000 mon.a (mon.0) 4534 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:46:41.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:46:41 smithi144 bash[66654]: cluster 2024-04-07T17:46:40.591112+0000 mon.a (mon.0) 4535 : cluster [DBG] osdmap e2537: 8 total, 8 up, 8 in 2024-04-07T17:46:41.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:46:41 smithi144 bash[66654]: cluster 2024-04-07T17:46:40.709465+0000 mgr.y (mgr.25175) 2739 : cluster [DBG] pgmap v4851: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 434 KiB/s rd, 5.0 KiB/s wr, 588 op/s 2024-04-07T17:46:41.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:46:41 smithi100 bash[74942]: cluster 2024-04-07T17:46:40.591112+0000 mon.a (mon.0) 4535 : cluster [DBG] osdmap e2537: 8 total, 8 up, 8 in 2024-04-07T17:46:41.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:46:41 smithi100 bash[74942]: cluster 2024-04-07T17:46:40.709465+0000 mgr.y (mgr.25175) 2739 : cluster [DBG] pgmap v4851: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 434 KiB/s rd, 5.0 KiB/s wr, 588 op/s 2024-04-07T17:46:41.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:46:41 smithi100 bash[76202]: cluster 2024-04-07T17:46:40.591112+0000 mon.a (mon.0) 4535 : cluster [DBG] osdmap e2537: 8 total, 8 up, 8 in 2024-04-07T17:46:41.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:46:41 smithi100 bash[76202]: cluster 2024-04-07T17:46:40.709465+0000 mgr.y (mgr.25175) 2739 : cluster [DBG] pgmap v4851: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 434 KiB/s rd, 5.0 KiB/s wr, 588 op/s 2024-04-07T17:46:42.598 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:46:42 smithi144 bash[95233]: ts=2024-04-07T17:46:42.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:46:42.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:46:42 smithi144 bash[66654]: cluster 2024-04-07T17:46:41.606346+0000 mon.a (mon.0) 4536 : cluster [DBG] osdmap e2538: 8 total, 8 up, 8 in 2024-04-07T17:46:42.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:46:42 smithi100 bash[74942]: cluster 2024-04-07T17:46:41.606346+0000 mon.a (mon.0) 4536 : cluster [DBG] osdmap e2538: 8 total, 8 up, 8 in 2024-04-07T17:46:42.953 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:46:42 smithi100 bash[76202]: cluster 2024-04-07T17:46:41.606346+0000 mon.a (mon.0) 4536 : cluster [DBG] osdmap e2538: 8 total, 8 up, 8 in 2024-04-07T17:46:43.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:46:43 smithi144 bash[66654]: cluster 2024-04-07T17:46:42.610648+0000 mon.a (mon.0) 4537 : cluster [DBG] osdmap e2539: 8 total, 8 up, 8 in 2024-04-07T17:46:43.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:46:43 smithi144 bash[66654]: cluster 2024-04-07T17:46:42.710206+0000 mgr.y (mgr.25175) 2740 : cluster [DBG] pgmap v4854: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 435 KiB/s rd, 5.0 KiB/s wr, 591 op/s 2024-04-07T17:46:43.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:46:43 smithi144 bash[66654]: cluster 2024-04-07T17:46:43.441257+0000 mon.a (mon.0) 4538 : cluster [DBG] osdmap e2540: 8 total, 8 up, 8 in 2024-04-07T17:46:43.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:46:43 smithi100 bash[74942]: cluster 2024-04-07T17:46:42.610648+0000 mon.a (mon.0) 4537 : cluster [DBG] osdmap e2539: 8 total, 8 up, 8 in 2024-04-07T17:46:43.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:46:43 smithi100 bash[74942]: cluster 2024-04-07T17:46:42.710206+0000 mgr.y (mgr.25175) 2740 : cluster [DBG] pgmap v4854: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 435 KiB/s rd, 5.0 KiB/s wr, 591 op/s 2024-04-07T17:46:43.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:46:43 smithi100 bash[74942]: cluster 2024-04-07T17:46:43.441257+0000 mon.a (mon.0) 4538 : cluster [DBG] osdmap e2540: 8 total, 8 up, 8 in 2024-04-07T17:46:43.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:46:43 smithi100 bash[76202]: cluster 2024-04-07T17:46:42.610648+0000 mon.a (mon.0) 4537 : cluster [DBG] osdmap e2539: 8 total, 8 up, 8 in 2024-04-07T17:46:43.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:46:43 smithi100 bash[76202]: cluster 2024-04-07T17:46:42.710206+0000 mgr.y (mgr.25175) 2740 : cluster [DBG] pgmap v4854: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 435 KiB/s rd, 5.0 KiB/s wr, 591 op/s 2024-04-07T17:46:43.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:46:43 smithi100 bash[76202]: cluster 2024-04-07T17:46:43.441257+0000 mon.a (mon.0) 4538 : cluster [DBG] osdmap e2540: 8 total, 8 up, 8 in 2024-04-07T17:46:45.703 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:46:45 smithi100 bash[74942]: cluster 2024-04-07T17:46:44.445000+0000 mon.a (mon.0) 4539 : cluster [DBG] osdmap e2541: 8 total, 8 up, 8 in 2024-04-07T17:46:45.703 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:46:45 smithi100 bash[74942]: cluster 2024-04-07T17:46:44.710987+0000 mgr.y (mgr.25175) 2741 : cluster [DBG] pgmap v4857: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 403 KiB/s rd, 5.0 KiB/s wr, 545 op/s 2024-04-07T17:46:45.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:46:45 smithi100 bash[76202]: cluster 2024-04-07T17:46:44.445000+0000 mon.a (mon.0) 4539 : cluster [DBG] osdmap e2541: 8 total, 8 up, 8 in 2024-04-07T17:46:45.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:46:45 smithi100 bash[76202]: cluster 2024-04-07T17:46:44.710987+0000 mgr.y (mgr.25175) 2741 : cluster [DBG] pgmap v4857: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 403 KiB/s rd, 5.0 KiB/s wr, 545 op/s 2024-04-07T17:46:45.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:46:45 smithi144 bash[66654]: cluster 2024-04-07T17:46:44.445000+0000 mon.a (mon.0) 4539 : cluster [DBG] osdmap e2541: 8 total, 8 up, 8 in 2024-04-07T17:46:45.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:46:45 smithi144 bash[66654]: cluster 2024-04-07T17:46:44.710987+0000 mgr.y (mgr.25175) 2741 : cluster [DBG] pgmap v4857: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 403 KiB/s rd, 5.0 KiB/s wr, 545 op/s 2024-04-07T17:46:46.703 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:46:46 smithi100 bash[74942]: cluster 2024-04-07T17:46:45.457181+0000 mon.a (mon.0) 4540 : cluster [DBG] osdmap e2542: 8 total, 8 up, 8 in 2024-04-07T17:46:46.703 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:46:46 smithi100 bash[76202]: cluster 2024-04-07T17:46:45.457181+0000 mon.a (mon.0) 4540 : cluster [DBG] osdmap e2542: 8 total, 8 up, 8 in 2024-04-07T17:46:46.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:46:46 smithi144 bash[66654]: cluster 2024-04-07T17:46:45.457181+0000 mon.a (mon.0) 4540 : cluster [DBG] osdmap e2542: 8 total, 8 up, 8 in 2024-04-07T17:46:47.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:46:47 smithi144 bash[66654]: cluster 2024-04-07T17:46:46.463229+0000 mon.a (mon.0) 4541 : cluster [DBG] osdmap e2543: 8 total, 8 up, 8 in 2024-04-07T17:46:47.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:46:47 smithi144 bash[66654]: cluster 2024-04-07T17:46:46.711767+0000 mgr.y (mgr.25175) 2742 : cluster [DBG] pgmap v4860: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 400 KiB/s rd, 5.0 KiB/s wr, 540 op/s 2024-04-07T17:46:47.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:46:47 smithi144 bash[66654]: audit 2024-04-07T17:46:46.877332+0000 mon.a (mon.0) 4542 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:46:47.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:46:47 smithi100 bash[74942]: cluster 2024-04-07T17:46:46.463229+0000 mon.a (mon.0) 4541 : cluster [DBG] osdmap e2543: 8 total, 8 up, 8 in 2024-04-07T17:46:47.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:46:47 smithi100 bash[74942]: cluster 2024-04-07T17:46:46.711767+0000 mgr.y (mgr.25175) 2742 : cluster [DBG] pgmap v4860: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 400 KiB/s rd, 5.0 KiB/s wr, 540 op/s 2024-04-07T17:46:47.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:46:47 smithi100 bash[74942]: audit 2024-04-07T17:46:46.877332+0000 mon.a (mon.0) 4542 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:46:47.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:46:47 smithi100 bash[76202]: cluster 2024-04-07T17:46:46.463229+0000 mon.a (mon.0) 4541 : cluster [DBG] osdmap e2543: 8 total, 8 up, 8 in 2024-04-07T17:46:47.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:46:47 smithi100 bash[76202]: cluster 2024-04-07T17:46:46.711767+0000 mgr.y (mgr.25175) 2742 : cluster [DBG] pgmap v4860: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 400 KiB/s rd, 5.0 KiB/s wr, 540 op/s 2024-04-07T17:46:47.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:46:47 smithi100 bash[76202]: audit 2024-04-07T17:46:46.877332+0000 mon.a (mon.0) 4542 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:46:48.703 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:46:48 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:46:48] "GET /metrics HTTP/1.1" 200 36745 "" "Prometheus/2.43.0" 2024-04-07T17:46:48.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:46:48 smithi100 bash[74942]: cluster 2024-04-07T17:46:47.475176+0000 mon.a (mon.0) 4543 : cluster [DBG] osdmap e2544: 8 total, 8 up, 8 in 2024-04-07T17:46:48.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:46:48 smithi100 bash[74942]: cluster 2024-04-07T17:46:48.443905+0000 mon.a (mon.0) 4544 : cluster [DBG] osdmap e2545: 8 total, 8 up, 8 in 2024-04-07T17:46:48.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:46:48 smithi100 bash[76202]: cluster 2024-04-07T17:46:47.475176+0000 mon.a (mon.0) 4543 : cluster [DBG] osdmap e2544: 8 total, 8 up, 8 in 2024-04-07T17:46:48.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:46:48 smithi100 bash[76202]: cluster 2024-04-07T17:46:48.443905+0000 mon.a (mon.0) 4544 : cluster [DBG] osdmap e2545: 8 total, 8 up, 8 in 2024-04-07T17:46:48.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:46:48 smithi144 bash[66654]: cluster 2024-04-07T17:46:47.475176+0000 mon.a (mon.0) 4543 : cluster [DBG] osdmap e2544: 8 total, 8 up, 8 in 2024-04-07T17:46:48.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:46:48 smithi144 bash[66654]: cluster 2024-04-07T17:46:48.443905+0000 mon.a (mon.0) 4544 : cluster [DBG] osdmap e2545: 8 total, 8 up, 8 in 2024-04-07T17:46:49.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:46:49 smithi144 bash[66654]: cluster 2024-04-07T17:46:48.712444+0000 mgr.y (mgr.25175) 2743 : cluster [DBG] pgmap v4863: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-07T17:46:49.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:46:49 smithi100 bash[74942]: cluster 2024-04-07T17:46:48.712444+0000 mgr.y (mgr.25175) 2743 : cluster [DBG] pgmap v4863: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-07T17:46:49.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:46:49 smithi100 bash[76202]: cluster 2024-04-07T17:46:48.712444+0000 mgr.y (mgr.25175) 2743 : cluster [DBG] pgmap v4863: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-07T17:46:50.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:46:50 smithi144 bash[66654]: cluster 2024-04-07T17:46:49.488071+0000 mon.a (mon.0) 4545 : cluster [DBG] osdmap e2546: 8 total, 8 up, 8 in 2024-04-07T17:46:50.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:46:50 smithi100 bash[74942]: cluster 2024-04-07T17:46:49.488071+0000 mon.a (mon.0) 4545 : cluster [DBG] osdmap e2546: 8 total, 8 up, 8 in 2024-04-07T17:46:50.953 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:46:50 smithi100 bash[76202]: cluster 2024-04-07T17:46:49.488071+0000 mon.a (mon.0) 4545 : cluster [DBG] osdmap e2546: 8 total, 8 up, 8 in 2024-04-07T17:46:51.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:46:51 smithi144 bash[66654]: cluster 2024-04-07T17:46:50.497295+0000 mon.a (mon.0) 4546 : cluster [DBG] osdmap e2547: 8 total, 8 up, 8 in 2024-04-07T17:46:51.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:46:51 smithi144 bash[66654]: cluster 2024-04-07T17:46:50.713229+0000 mgr.y (mgr.25175) 2744 : cluster [DBG] pgmap v4866: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 368 KiB/s rd, 5.0 KiB/s wr, 499 op/s 2024-04-07T17:46:51.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:46:51 smithi100 bash[74942]: cluster 2024-04-07T17:46:50.497295+0000 mon.a (mon.0) 4546 : cluster [DBG] osdmap e2547: 8 total, 8 up, 8 in 2024-04-07T17:46:51.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:46:51 smithi100 bash[74942]: cluster 2024-04-07T17:46:50.713229+0000 mgr.y (mgr.25175) 2744 : cluster [DBG] pgmap v4866: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 368 KiB/s rd, 5.0 KiB/s wr, 499 op/s 2024-04-07T17:46:51.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:46:51 smithi100 bash[76202]: cluster 2024-04-07T17:46:50.497295+0000 mon.a (mon.0) 4546 : cluster [DBG] osdmap e2547: 8 total, 8 up, 8 in 2024-04-07T17:46:51.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:46:51 smithi100 bash[76202]: cluster 2024-04-07T17:46:50.713229+0000 mgr.y (mgr.25175) 2744 : cluster [DBG] pgmap v4866: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 368 KiB/s rd, 5.0 KiB/s wr, 499 op/s 2024-04-07T17:46:52.614 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:46:52 smithi144 bash[66654]: cluster 2024-04-07T17:46:51.510732+0000 mon.a (mon.0) 4547 : cluster [DBG] osdmap e2548: 8 total, 8 up, 8 in 2024-04-07T17:46:52.615 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:46:52 smithi144 bash[95233]: ts=2024-04-07T17:46:52.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:46:52.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:46:52 smithi100 bash[74942]: cluster 2024-04-07T17:46:51.510732+0000 mon.a (mon.0) 4547 : cluster [DBG] osdmap e2548: 8 total, 8 up, 8 in 2024-04-07T17:46:52.953 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:46:52 smithi100 bash[76202]: cluster 2024-04-07T17:46:51.510732+0000 mon.a (mon.0) 4547 : cluster [DBG] osdmap e2548: 8 total, 8 up, 8 in 2024-04-07T17:46:53.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:46:53 smithi144 bash[66654]: cluster 2024-04-07T17:46:52.518296+0000 mon.a (mon.0) 4548 : cluster [DBG] osdmap e2549: 8 total, 8 up, 8 in 2024-04-07T17:46:53.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:46:53 smithi144 bash[66654]: cluster 2024-04-07T17:46:52.714002+0000 mgr.y (mgr.25175) 2745 : cluster [DBG] pgmap v4869: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 368 KiB/s rd, 5.0 KiB/s wr, 497 op/s 2024-04-07T17:46:53.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:46:53 smithi144 bash[66654]: cluster 2024-04-07T17:46:53.446605+0000 mon.a (mon.0) 4549 : cluster [DBG] osdmap e2550: 8 total, 8 up, 8 in 2024-04-07T17:46:53.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:46:53 smithi100 bash[74942]: cluster 2024-04-07T17:46:52.518296+0000 mon.a (mon.0) 4548 : cluster [DBG] osdmap e2549: 8 total, 8 up, 8 in 2024-04-07T17:46:53.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:46:53 smithi100 bash[74942]: cluster 2024-04-07T17:46:52.714002+0000 mgr.y (mgr.25175) 2745 : cluster [DBG] pgmap v4869: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 368 KiB/s rd, 5.0 KiB/s wr, 497 op/s 2024-04-07T17:46:53.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:46:53 smithi100 bash[74942]: cluster 2024-04-07T17:46:53.446605+0000 mon.a (mon.0) 4549 : cluster [DBG] osdmap e2550: 8 total, 8 up, 8 in 2024-04-07T17:46:53.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:46:53 smithi100 bash[76202]: cluster 2024-04-07T17:46:52.518296+0000 mon.a (mon.0) 4548 : cluster [DBG] osdmap e2549: 8 total, 8 up, 8 in 2024-04-07T17:46:53.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:46:53 smithi100 bash[76202]: cluster 2024-04-07T17:46:52.714002+0000 mgr.y (mgr.25175) 2745 : cluster [DBG] pgmap v4869: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 368 KiB/s rd, 5.0 KiB/s wr, 497 op/s 2024-04-07T17:46:53.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:46:53 smithi100 bash[76202]: cluster 2024-04-07T17:46:53.446605+0000 mon.a (mon.0) 4549 : cluster [DBG] osdmap e2550: 8 total, 8 up, 8 in 2024-04-07T17:46:55.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:46:55 smithi144 bash[66654]: cluster 2024-04-07T17:46:54.531415+0000 mon.a (mon.0) 4550 : cluster [DBG] osdmap e2551: 8 total, 8 up, 8 in 2024-04-07T17:46:55.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:46:55 smithi144 bash[66654]: cluster 2024-04-07T17:46:54.714744+0000 mgr.y (mgr.25175) 2746 : cluster [DBG] pgmap v4872: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 338 KiB/s rd, 5.0 KiB/s wr, 458 op/s 2024-04-07T17:46:55.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:46:55 smithi100 bash[74942]: cluster 2024-04-07T17:46:54.531415+0000 mon.a (mon.0) 4550 : cluster [DBG] osdmap e2551: 8 total, 8 up, 8 in 2024-04-07T17:46:55.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:46:55 smithi100 bash[74942]: cluster 2024-04-07T17:46:54.714744+0000 mgr.y (mgr.25175) 2746 : cluster [DBG] pgmap v4872: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 338 KiB/s rd, 5.0 KiB/s wr, 458 op/s 2024-04-07T17:46:55.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:46:55 smithi100 bash[76202]: cluster 2024-04-07T17:46:54.531415+0000 mon.a (mon.0) 4550 : cluster [DBG] osdmap e2551: 8 total, 8 up, 8 in 2024-04-07T17:46:55.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:46:55 smithi100 bash[76202]: cluster 2024-04-07T17:46:54.714744+0000 mgr.y (mgr.25175) 2746 : cluster [DBG] pgmap v4872: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 338 KiB/s rd, 5.0 KiB/s wr, 458 op/s 2024-04-07T17:46:56.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:46:56 smithi144 bash[66654]: cluster 2024-04-07T17:46:55.546089+0000 mon.a (mon.0) 4551 : cluster [DBG] osdmap e2552: 8 total, 8 up, 8 in 2024-04-07T17:46:56.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:46:56 smithi100 bash[74942]: cluster 2024-04-07T17:46:55.546089+0000 mon.a (mon.0) 4551 : cluster [DBG] osdmap e2552: 8 total, 8 up, 8 in 2024-04-07T17:46:56.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:46:56 smithi100 bash[76202]: cluster 2024-04-07T17:46:55.546089+0000 mon.a (mon.0) 4551 : cluster [DBG] osdmap e2552: 8 total, 8 up, 8 in 2024-04-07T17:46:57.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:46:57 smithi144 bash[66654]: cluster 2024-04-07T17:46:56.545213+0000 mon.a (mon.0) 4552 : cluster [DBG] osdmap e2553: 8 total, 8 up, 8 in 2024-04-07T17:46:57.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:46:57 smithi144 bash[66654]: cluster 2024-04-07T17:46:56.715529+0000 mgr.y (mgr.25175) 2747 : cluster [DBG] pgmap v4875: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 336 KiB/s rd, 5.0 KiB/s wr, 456 op/s 2024-04-07T17:46:57.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:46:57 smithi100 bash[74942]: cluster 2024-04-07T17:46:56.545213+0000 mon.a (mon.0) 4552 : cluster [DBG] osdmap e2553: 8 total, 8 up, 8 in 2024-04-07T17:46:57.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:46:57 smithi100 bash[74942]: cluster 2024-04-07T17:46:56.715529+0000 mgr.y (mgr.25175) 2747 : cluster [DBG] pgmap v4875: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 336 KiB/s rd, 5.0 KiB/s wr, 456 op/s 2024-04-07T17:46:57.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:46:57 smithi100 bash[76202]: cluster 2024-04-07T17:46:56.545213+0000 mon.a (mon.0) 4552 : cluster [DBG] osdmap e2553: 8 total, 8 up, 8 in 2024-04-07T17:46:57.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:46:57 smithi100 bash[76202]: cluster 2024-04-07T17:46:56.715529+0000 mgr.y (mgr.25175) 2747 : cluster [DBG] pgmap v4875: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 336 KiB/s rd, 5.0 KiB/s wr, 456 op/s 2024-04-07T17:46:58.703 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:46:58 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:46:58] "GET /metrics HTTP/1.1" 200 36745 "" "Prometheus/2.43.0" 2024-04-07T17:46:58.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:46:58 smithi100 bash[74942]: cluster 2024-04-07T17:46:57.570589+0000 mon.a (mon.0) 4553 : cluster [DBG] osdmap e2554: 8 total, 8 up, 8 in 2024-04-07T17:46:58.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:46:58 smithi100 bash[74942]: cluster 2024-04-07T17:46:58.448551+0000 mon.a (mon.0) 4554 : cluster [DBG] osdmap e2555: 8 total, 8 up, 8 in 2024-04-07T17:46:58.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:46:58 smithi100 bash[76202]: cluster 2024-04-07T17:46:57.570589+0000 mon.a (mon.0) 4553 : cluster [DBG] osdmap e2554: 8 total, 8 up, 8 in 2024-04-07T17:46:58.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:46:58 smithi100 bash[76202]: cluster 2024-04-07T17:46:58.448551+0000 mon.a (mon.0) 4554 : cluster [DBG] osdmap e2555: 8 total, 8 up, 8 in 2024-04-07T17:46:58.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:46:58 smithi144 bash[66654]: cluster 2024-04-07T17:46:57.570589+0000 mon.a (mon.0) 4553 : cluster [DBG] osdmap e2554: 8 total, 8 up, 8 in 2024-04-07T17:46:58.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:46:58 smithi144 bash[66654]: cluster 2024-04-07T17:46:58.448551+0000 mon.a (mon.0) 4554 : cluster [DBG] osdmap e2555: 8 total, 8 up, 8 in 2024-04-07T17:46:59.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:46:59 smithi144 bash[66654]: cluster 2024-04-07T17:46:58.716218+0000 mgr.y (mgr.25175) 2748 : cluster [DBG] pgmap v4878: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-07T17:46:59.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:46:59 smithi144 bash[66654]: cluster 2024-04-07T17:46:59.453356+0000 mon.a (mon.0) 4555 : cluster [DBG] osdmap e2556: 8 total, 8 up, 8 in 2024-04-07T17:46:59.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:46:59 smithi100 bash[74942]: cluster 2024-04-07T17:46:58.716218+0000 mgr.y (mgr.25175) 2748 : cluster [DBG] pgmap v4878: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-07T17:46:59.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:46:59 smithi100 bash[74942]: cluster 2024-04-07T17:46:59.453356+0000 mon.a (mon.0) 4555 : cluster [DBG] osdmap e2556: 8 total, 8 up, 8 in 2024-04-07T17:46:59.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:46:59 smithi100 bash[76202]: cluster 2024-04-07T17:46:58.716218+0000 mgr.y (mgr.25175) 2748 : cluster [DBG] pgmap v4878: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-07T17:46:59.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:46:59 smithi100 bash[76202]: cluster 2024-04-07T17:46:59.453356+0000 mon.a (mon.0) 4555 : cluster [DBG] osdmap e2556: 8 total, 8 up, 8 in 2024-04-07T17:47:01.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:47:01 smithi144 bash[66654]: cluster 2024-04-07T17:47:00.454311+0000 mon.a (mon.0) 4556 : cluster [DBG] osdmap e2557: 8 total, 8 up, 8 in 2024-04-07T17:47:01.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:47:01 smithi144 bash[66654]: cluster 2024-04-07T17:47:00.717013+0000 mgr.y (mgr.25175) 2749 : cluster [DBG] pgmap v4881: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 304 KiB/s rd, 5.0 KiB/s wr, 412 op/s 2024-04-07T17:47:01.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:47:01 smithi100 bash[74942]: cluster 2024-04-07T17:47:00.454311+0000 mon.a (mon.0) 4556 : cluster [DBG] osdmap e2557: 8 total, 8 up, 8 in 2024-04-07T17:47:01.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:47:01 smithi100 bash[74942]: cluster 2024-04-07T17:47:00.717013+0000 mgr.y (mgr.25175) 2749 : cluster [DBG] pgmap v4881: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 304 KiB/s rd, 5.0 KiB/s wr, 412 op/s 2024-04-07T17:47:01.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:47:01 smithi100 bash[76202]: cluster 2024-04-07T17:47:00.454311+0000 mon.a (mon.0) 4556 : cluster [DBG] osdmap e2557: 8 total, 8 up, 8 in 2024-04-07T17:47:01.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:47:01 smithi100 bash[76202]: cluster 2024-04-07T17:47:00.717013+0000 mgr.y (mgr.25175) 2749 : cluster [DBG] pgmap v4881: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 304 KiB/s rd, 5.0 KiB/s wr, 412 op/s 2024-04-07T17:47:02.614 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:47:02 smithi144 bash[95233]: ts=2024-04-07T17:47:02.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:47:02.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:47:02 smithi144 bash[66654]: cluster 2024-04-07T17:47:01.466557+0000 mon.a (mon.0) 4557 : cluster [DBG] osdmap e2558: 8 total, 8 up, 8 in 2024-04-07T17:47:02.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:47:02 smithi144 bash[66654]: audit 2024-04-07T17:47:01.877724+0000 mon.a (mon.0) 4558 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:47:02.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:47:02 smithi100 bash[74942]: cluster 2024-04-07T17:47:01.466557+0000 mon.a (mon.0) 4557 : cluster [DBG] osdmap e2558: 8 total, 8 up, 8 in 2024-04-07T17:47:02.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:47:02 smithi100 bash[74942]: audit 2024-04-07T17:47:01.877724+0000 mon.a (mon.0) 4558 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:47:02.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:47:02 smithi100 bash[76202]: cluster 2024-04-07T17:47:01.466557+0000 mon.a (mon.0) 4557 : cluster [DBG] osdmap e2558: 8 total, 8 up, 8 in 2024-04-07T17:47:02.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:47:02 smithi100 bash[76202]: audit 2024-04-07T17:47:01.877724+0000 mon.a (mon.0) 4558 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:47:03.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:47:03 smithi144 bash[66654]: cluster 2024-04-07T17:47:02.469253+0000 mon.a (mon.0) 4559 : cluster [DBG] osdmap e2559: 8 total, 8 up, 8 in 2024-04-07T17:47:03.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:47:03 smithi144 bash[66654]: cluster 2024-04-07T17:47:02.717749+0000 mgr.y (mgr.25175) 2750 : cluster [DBG] pgmap v4884: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 305 KiB/s rd, 5.0 KiB/s wr, 416 op/s 2024-04-07T17:47:03.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:47:03 smithi144 bash[66654]: cluster 2024-04-07T17:47:03.451041+0000 mon.a (mon.0) 4560 : cluster [DBG] osdmap e2560: 8 total, 8 up, 8 in 2024-04-07T17:47:03.953 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:47:03 smithi100 bash[76202]: cluster 2024-04-07T17:47:02.469253+0000 mon.a (mon.0) 4559 : cluster [DBG] osdmap e2559: 8 total, 8 up, 8 in 2024-04-07T17:47:03.953 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:47:03 smithi100 bash[76202]: cluster 2024-04-07T17:47:02.717749+0000 mgr.y (mgr.25175) 2750 : cluster [DBG] pgmap v4884: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 305 KiB/s rd, 5.0 KiB/s wr, 416 op/s 2024-04-07T17:47:03.953 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:47:03 smithi100 bash[76202]: cluster 2024-04-07T17:47:03.451041+0000 mon.a (mon.0) 4560 : cluster [DBG] osdmap e2560: 8 total, 8 up, 8 in 2024-04-07T17:47:03.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:47:03 smithi100 bash[74942]: cluster 2024-04-07T17:47:02.469253+0000 mon.a (mon.0) 4559 : cluster [DBG] osdmap e2559: 8 total, 8 up, 8 in 2024-04-07T17:47:03.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:47:03 smithi100 bash[74942]: cluster 2024-04-07T17:47:02.717749+0000 mgr.y (mgr.25175) 2750 : cluster [DBG] pgmap v4884: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 305 KiB/s rd, 5.0 KiB/s wr, 416 op/s 2024-04-07T17:47:03.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:47:03 smithi100 bash[74942]: cluster 2024-04-07T17:47:03.451041+0000 mon.a (mon.0) 4560 : cluster [DBG] osdmap e2560: 8 total, 8 up, 8 in 2024-04-07T17:47:05.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:47:05 smithi144 bash[66654]: cluster 2024-04-07T17:47:04.500790+0000 mon.a (mon.0) 4561 : cluster [DBG] osdmap e2561: 8 total, 8 up, 8 in 2024-04-07T17:47:05.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:47:05 smithi144 bash[66654]: cluster 2024-04-07T17:47:04.718485+0000 mgr.y (mgr.25175) 2751 : cluster [DBG] pgmap v4887: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 272 KiB/s rd, 5.0 KiB/s wr, 370 op/s 2024-04-07T17:47:05.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:47:05 smithi100 bash[74942]: cluster 2024-04-07T17:47:04.500790+0000 mon.a (mon.0) 4561 : cluster [DBG] osdmap e2561: 8 total, 8 up, 8 in 2024-04-07T17:47:05.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:47:05 smithi100 bash[74942]: cluster 2024-04-07T17:47:04.718485+0000 mgr.y (mgr.25175) 2751 : cluster [DBG] pgmap v4887: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 272 KiB/s rd, 5.0 KiB/s wr, 370 op/s 2024-04-07T17:47:05.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:47:05 smithi100 bash[76202]: cluster 2024-04-07T17:47:04.500790+0000 mon.a (mon.0) 4561 : cluster [DBG] osdmap e2561: 8 total, 8 up, 8 in 2024-04-07T17:47:05.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:47:05 smithi100 bash[76202]: cluster 2024-04-07T17:47:04.718485+0000 mgr.y (mgr.25175) 2751 : cluster [DBG] pgmap v4887: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 272 KiB/s rd, 5.0 KiB/s wr, 370 op/s 2024-04-07T17:47:06.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:47:06 smithi144 bash[66654]: cluster 2024-04-07T17:47:05.509971+0000 mon.a (mon.0) 4562 : cluster [DBG] osdmap e2562: 8 total, 8 up, 8 in 2024-04-07T17:47:06.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:47:06 smithi100 bash[74942]: cluster 2024-04-07T17:47:05.509971+0000 mon.a (mon.0) 4562 : cluster [DBG] osdmap e2562: 8 total, 8 up, 8 in 2024-04-07T17:47:06.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:47:06 smithi100 bash[76202]: cluster 2024-04-07T17:47:05.509971+0000 mon.a (mon.0) 4562 : cluster [DBG] osdmap e2562: 8 total, 8 up, 8 in 2024-04-07T17:47:07.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:47:07 smithi144 bash[66654]: cluster 2024-04-07T17:47:06.511202+0000 mon.a (mon.0) 4563 : cluster [DBG] osdmap e2563: 8 total, 8 up, 8 in 2024-04-07T17:47:07.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:47:07 smithi144 bash[66654]: cluster 2024-04-07T17:47:06.719239+0000 mgr.y (mgr.25175) 2752 : cluster [DBG] pgmap v4890: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 269 KiB/s rd, 5.0 KiB/s wr, 364 op/s 2024-04-07T17:47:07.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:47:07 smithi100 bash[74942]: cluster 2024-04-07T17:47:06.511202+0000 mon.a (mon.0) 4563 : cluster [DBG] osdmap e2563: 8 total, 8 up, 8 in 2024-04-07T17:47:07.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:47:07 smithi100 bash[74942]: cluster 2024-04-07T17:47:06.719239+0000 mgr.y (mgr.25175) 2752 : cluster [DBG] pgmap v4890: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 269 KiB/s rd, 5.0 KiB/s wr, 364 op/s 2024-04-07T17:47:07.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:47:07 smithi100 bash[76202]: cluster 2024-04-07T17:47:06.511202+0000 mon.a (mon.0) 4563 : cluster [DBG] osdmap e2563: 8 total, 8 up, 8 in 2024-04-07T17:47:07.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:47:07 smithi100 bash[76202]: cluster 2024-04-07T17:47:06.719239+0000 mgr.y (mgr.25175) 2752 : cluster [DBG] pgmap v4890: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 269 KiB/s rd, 5.0 KiB/s wr, 364 op/s 2024-04-07T17:47:08.703 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:47:08 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:47:08] "GET /metrics HTTP/1.1" 200 36744 "" "Prometheus/2.43.0" 2024-04-07T17:47:08.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:47:08 smithi100 bash[74942]: cluster 2024-04-07T17:47:07.537445+0000 mon.a (mon.0) 4564 : cluster [DBG] osdmap e2564: 8 total, 8 up, 8 in 2024-04-07T17:47:08.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:47:08 smithi100 bash[74942]: cluster 2024-04-07T17:47:08.462867+0000 mon.a (mon.0) 4565 : cluster [DBG] osdmap e2565: 8 total, 8 up, 8 in 2024-04-07T17:47:08.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:47:08 smithi100 bash[76202]: cluster 2024-04-07T17:47:07.537445+0000 mon.a (mon.0) 4564 : cluster [DBG] osdmap e2564: 8 total, 8 up, 8 in 2024-04-07T17:47:08.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:47:08 smithi100 bash[76202]: cluster 2024-04-07T17:47:08.462867+0000 mon.a (mon.0) 4565 : cluster [DBG] osdmap e2565: 8 total, 8 up, 8 in 2024-04-07T17:47:08.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:47:08 smithi144 bash[66654]: cluster 2024-04-07T17:47:07.537445+0000 mon.a (mon.0) 4564 : cluster [DBG] osdmap e2564: 8 total, 8 up, 8 in 2024-04-07T17:47:08.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:47:08 smithi144 bash[66654]: cluster 2024-04-07T17:47:08.462867+0000 mon.a (mon.0) 4565 : cluster [DBG] osdmap e2565: 8 total, 8 up, 8 in 2024-04-07T17:47:09.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:47:09 smithi144 bash[66654]: cluster 2024-04-07T17:47:08.720025+0000 mgr.y (mgr.25175) 2753 : cluster [DBG] pgmap v4893: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 3.0 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-07T17:47:09.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:47:09 smithi100 bash[74942]: cluster 2024-04-07T17:47:08.720025+0000 mgr.y (mgr.25175) 2753 : cluster [DBG] pgmap v4893: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 3.0 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-07T17:47:09.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:47:09 smithi100 bash[76202]: cluster 2024-04-07T17:47:08.720025+0000 mgr.y (mgr.25175) 2753 : cluster [DBG] pgmap v4893: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 3.0 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-07T17:47:10.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:47:10 smithi144 bash[66654]: cluster 2024-04-07T17:47:09.537312+0000 mon.a (mon.0) 4566 : cluster [DBG] osdmap e2566: 8 total, 8 up, 8 in 2024-04-07T17:47:10.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:47:10 smithi100 bash[74942]: cluster 2024-04-07T17:47:09.537312+0000 mon.a (mon.0) 4566 : cluster [DBG] osdmap e2566: 8 total, 8 up, 8 in 2024-04-07T17:47:10.953 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:47:10 smithi100 bash[76202]: cluster 2024-04-07T17:47:09.537312+0000 mon.a (mon.0) 4566 : cluster [DBG] osdmap e2566: 8 total, 8 up, 8 in 2024-04-07T17:47:11.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:47:11 smithi144 bash[66654]: cluster 2024-04-07T17:47:10.565189+0000 mon.a (mon.0) 4567 : cluster [DBG] osdmap e2567: 8 total, 8 up, 8 in 2024-04-07T17:47:11.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:47:11 smithi144 bash[66654]: cluster 2024-04-07T17:47:10.720737+0000 mgr.y (mgr.25175) 2754 : cluster [DBG] pgmap v4896: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 239 KiB/s rd, 5.0 KiB/s wr, 326 op/s 2024-04-07T17:47:11.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:47:11 smithi100 bash[74942]: cluster 2024-04-07T17:47:10.565189+0000 mon.a (mon.0) 4567 : cluster [DBG] osdmap e2567: 8 total, 8 up, 8 in 2024-04-07T17:47:11.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:47:11 smithi100 bash[74942]: cluster 2024-04-07T17:47:10.720737+0000 mgr.y (mgr.25175) 2754 : cluster [DBG] pgmap v4896: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 239 KiB/s rd, 5.0 KiB/s wr, 326 op/s 2024-04-07T17:47:11.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:47:11 smithi100 bash[76202]: cluster 2024-04-07T17:47:10.565189+0000 mon.a (mon.0) 4567 : cluster [DBG] osdmap e2567: 8 total, 8 up, 8 in 2024-04-07T17:47:11.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:47:11 smithi100 bash[76202]: cluster 2024-04-07T17:47:10.720737+0000 mgr.y (mgr.25175) 2754 : cluster [DBG] pgmap v4896: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 239 KiB/s rd, 5.0 KiB/s wr, 326 op/s 2024-04-07T17:47:12.570 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:47:12 smithi144 bash[95233]: ts=2024-04-07T17:47:12.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:47:12.866 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:47:12 smithi144 bash[66654]: cluster 2024-04-07T17:47:11.590511+0000 mon.a (mon.0) 4568 : cluster [DBG] osdmap e2568: 8 total, 8 up, 8 in 2024-04-07T17:47:12.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:47:12 smithi100 bash[74942]: cluster 2024-04-07T17:47:11.590511+0000 mon.a (mon.0) 4568 : cluster [DBG] osdmap e2568: 8 total, 8 up, 8 in 2024-04-07T17:47:12.953 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:47:12 smithi100 bash[76202]: cluster 2024-04-07T17:47:11.590511+0000 mon.a (mon.0) 4568 : cluster [DBG] osdmap e2568: 8 total, 8 up, 8 in 2024-04-07T17:47:13.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:47:13 smithi144 bash[66654]: cluster 2024-04-07T17:47:12.587004+0000 mon.a (mon.0) 4569 : cluster [DBG] osdmap e2569: 8 total, 8 up, 8 in 2024-04-07T17:47:13.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:47:13 smithi144 bash[66654]: cluster 2024-04-07T17:47:12.721501+0000 mgr.y (mgr.25175) 2755 : cluster [DBG] pgmap v4899: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 238 KiB/s rd, 5.0 KiB/s wr, 325 op/s 2024-04-07T17:47:13.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:47:13 smithi144 bash[66654]: cluster 2024-04-07T17:47:13.458817+0000 mon.a (mon.0) 4570 : cluster [DBG] osdmap e2570: 8 total, 8 up, 8 in 2024-04-07T17:47:13.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:47:13 smithi100 bash[74942]: cluster 2024-04-07T17:47:12.587004+0000 mon.a (mon.0) 4569 : cluster [DBG] osdmap e2569: 8 total, 8 up, 8 in 2024-04-07T17:47:13.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:47:13 smithi100 bash[74942]: cluster 2024-04-07T17:47:12.721501+0000 mgr.y (mgr.25175) 2755 : cluster [DBG] pgmap v4899: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 238 KiB/s rd, 5.0 KiB/s wr, 325 op/s 2024-04-07T17:47:13.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:47:13 smithi100 bash[74942]: cluster 2024-04-07T17:47:13.458817+0000 mon.a (mon.0) 4570 : cluster [DBG] osdmap e2570: 8 total, 8 up, 8 in 2024-04-07T17:47:13.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:47:13 smithi100 bash[76202]: cluster 2024-04-07T17:47:12.587004+0000 mon.a (mon.0) 4569 : cluster [DBG] osdmap e2569: 8 total, 8 up, 8 in 2024-04-07T17:47:13.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:47:13 smithi100 bash[76202]: cluster 2024-04-07T17:47:12.721501+0000 mgr.y (mgr.25175) 2755 : cluster [DBG] pgmap v4899: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 238 KiB/s rd, 5.0 KiB/s wr, 325 op/s 2024-04-07T17:47:13.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:47:13 smithi100 bash[76202]: cluster 2024-04-07T17:47:13.458817+0000 mon.a (mon.0) 4570 : cluster [DBG] osdmap e2570: 8 total, 8 up, 8 in 2024-04-07T17:47:15.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:47:15 smithi144 bash[66654]: cluster 2024-04-07T17:47:14.459728+0000 mon.a (mon.0) 4571 : cluster [DBG] osdmap e2571: 8 total, 8 up, 8 in 2024-04-07T17:47:15.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:47:15 smithi144 bash[66654]: cluster 2024-04-07T17:47:14.722249+0000 mgr.y (mgr.25175) 2756 : cluster [DBG] pgmap v4902: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 206 KiB/s rd, 5.0 KiB/s wr, 283 op/s 2024-04-07T17:47:15.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:47:15 smithi100 bash[74942]: cluster 2024-04-07T17:47:14.459728+0000 mon.a (mon.0) 4571 : cluster [DBG] osdmap e2571: 8 total, 8 up, 8 in 2024-04-07T17:47:15.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:47:15 smithi100 bash[74942]: cluster 2024-04-07T17:47:14.722249+0000 mgr.y (mgr.25175) 2756 : cluster [DBG] pgmap v4902: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 206 KiB/s rd, 5.0 KiB/s wr, 283 op/s 2024-04-07T17:47:15.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:47:15 smithi100 bash[76202]: cluster 2024-04-07T17:47:14.459728+0000 mon.a (mon.0) 4571 : cluster [DBG] osdmap e2571: 8 total, 8 up, 8 in 2024-04-07T17:47:15.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:47:15 smithi100 bash[76202]: cluster 2024-04-07T17:47:14.722249+0000 mgr.y (mgr.25175) 2756 : cluster [DBG] pgmap v4902: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 206 KiB/s rd, 5.0 KiB/s wr, 283 op/s 2024-04-07T17:47:16.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:47:16 smithi144 bash[66654]: cluster 2024-04-07T17:47:15.472994+0000 mon.a (mon.0) 4572 : cluster [DBG] osdmap e2572: 8 total, 8 up, 8 in 2024-04-07T17:47:16.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:47:16 smithi100 bash[74942]: cluster 2024-04-07T17:47:15.472994+0000 mon.a (mon.0) 4572 : cluster [DBG] osdmap e2572: 8 total, 8 up, 8 in 2024-04-07T17:47:16.953 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:47:16 smithi100 bash[76202]: cluster 2024-04-07T17:47:15.472994+0000 mon.a (mon.0) 4572 : cluster [DBG] osdmap e2572: 8 total, 8 up, 8 in 2024-04-07T17:47:17.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:47:17 smithi144 bash[66654]: cluster 2024-04-07T17:47:16.484412+0000 mon.a (mon.0) 4573 : cluster [DBG] osdmap e2573: 8 total, 8 up, 8 in 2024-04-07T17:47:17.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:47:17 smithi144 bash[66654]: cluster 2024-04-07T17:47:16.722970+0000 mgr.y (mgr.25175) 2757 : cluster [DBG] pgmap v4905: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 204 KiB/s rd, 5.0 KiB/s wr, 278 op/s 2024-04-07T17:47:17.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:47:17 smithi144 bash[66654]: audit 2024-04-07T17:47:16.878330+0000 mon.a (mon.0) 4574 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:47:17.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:47:17 smithi100 bash[74942]: cluster 2024-04-07T17:47:16.484412+0000 mon.a (mon.0) 4573 : cluster [DBG] osdmap e2573: 8 total, 8 up, 8 in 2024-04-07T17:47:17.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:47:17 smithi100 bash[74942]: cluster 2024-04-07T17:47:16.722970+0000 mgr.y (mgr.25175) 2757 : cluster [DBG] pgmap v4905: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 204 KiB/s rd, 5.0 KiB/s wr, 278 op/s 2024-04-07T17:47:17.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:47:17 smithi100 bash[74942]: audit 2024-04-07T17:47:16.878330+0000 mon.a (mon.0) 4574 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:47:17.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:47:17 smithi100 bash[76202]: cluster 2024-04-07T17:47:16.484412+0000 mon.a (mon.0) 4573 : cluster [DBG] osdmap e2573: 8 total, 8 up, 8 in 2024-04-07T17:47:17.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:47:17 smithi100 bash[76202]: cluster 2024-04-07T17:47:16.722970+0000 mgr.y (mgr.25175) 2757 : cluster [DBG] pgmap v4905: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 204 KiB/s rd, 5.0 KiB/s wr, 278 op/s 2024-04-07T17:47:17.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:47:17 smithi100 bash[76202]: audit 2024-04-07T17:47:16.878330+0000 mon.a (mon.0) 4574 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:47:18.703 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:47:18 smithi100 bash[74942]: cluster 2024-04-07T17:47:17.494716+0000 mon.a (mon.0) 4575 : cluster [DBG] osdmap e2574: 8 total, 8 up, 8 in 2024-04-07T17:47:18.703 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:47:18 smithi100 bash[74942]: cluster 2024-04-07T17:47:18.459699+0000 mon.a (mon.0) 4576 : cluster [DBG] osdmap e2575: 8 total, 8 up, 8 in 2024-04-07T17:47:18.704 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:47:18 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:47:18] "GET /metrics HTTP/1.1" 200 36742 "" "Prometheus/2.43.0" 2024-04-07T17:47:18.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:47:18 smithi100 bash[76202]: cluster 2024-04-07T17:47:17.494716+0000 mon.a (mon.0) 4575 : cluster [DBG] osdmap e2574: 8 total, 8 up, 8 in 2024-04-07T17:47:18.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:47:18 smithi100 bash[76202]: cluster 2024-04-07T17:47:18.459699+0000 mon.a (mon.0) 4576 : cluster [DBG] osdmap e2575: 8 total, 8 up, 8 in 2024-04-07T17:47:18.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:47:18 smithi144 bash[66654]: cluster 2024-04-07T17:47:17.494716+0000 mon.a (mon.0) 4575 : cluster [DBG] osdmap e2574: 8 total, 8 up, 8 in 2024-04-07T17:47:18.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:47:18 smithi144 bash[66654]: cluster 2024-04-07T17:47:18.459699+0000 mon.a (mon.0) 4576 : cluster [DBG] osdmap e2575: 8 total, 8 up, 8 in 2024-04-07T17:47:19.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:47:19 smithi144 bash[66654]: cluster 2024-04-07T17:47:18.723889+0000 mgr.y (mgr.25175) 2758 : cluster [DBG] pgmap v4908: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 3.0 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-07T17:47:19.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:47:19 smithi100 bash[74942]: cluster 2024-04-07T17:47:18.723889+0000 mgr.y (mgr.25175) 2758 : cluster [DBG] pgmap v4908: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 3.0 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-07T17:47:19.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:47:19 smithi100 bash[76202]: cluster 2024-04-07T17:47:18.723889+0000 mgr.y (mgr.25175) 2758 : cluster [DBG] pgmap v4908: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 3.0 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-07T17:47:20.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:47:20 smithi144 bash[66654]: cluster 2024-04-07T17:47:19.506175+0000 mon.a (mon.0) 4577 : cluster [DBG] osdmap e2576: 8 total, 8 up, 8 in 2024-04-07T17:47:20.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:47:20 smithi100 bash[74942]: cluster 2024-04-07T17:47:19.506175+0000 mon.a (mon.0) 4577 : cluster [DBG] osdmap e2576: 8 total, 8 up, 8 in 2024-04-07T17:47:20.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:47:20 smithi100 bash[76202]: cluster 2024-04-07T17:47:19.506175+0000 mon.a (mon.0) 4577 : cluster [DBG] osdmap e2576: 8 total, 8 up, 8 in 2024-04-07T17:47:21.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:47:21 smithi144 bash[66654]: cluster 2024-04-07T17:47:20.515824+0000 mon.a (mon.0) 4578 : cluster [DBG] osdmap e2577: 8 total, 8 up, 8 in 2024-04-07T17:47:21.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:47:21 smithi144 bash[66654]: cluster 2024-04-07T17:47:20.724682+0000 mgr.y (mgr.25175) 2759 : cluster [DBG] pgmap v4911: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 174 KiB/s rd, 5.0 KiB/s wr, 237 op/s 2024-04-07T17:47:21.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:47:21 smithi100 bash[74942]: cluster 2024-04-07T17:47:20.515824+0000 mon.a (mon.0) 4578 : cluster [DBG] osdmap e2577: 8 total, 8 up, 8 in 2024-04-07T17:47:21.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:47:21 smithi100 bash[74942]: cluster 2024-04-07T17:47:20.724682+0000 mgr.y (mgr.25175) 2759 : cluster [DBG] pgmap v4911: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 174 KiB/s rd, 5.0 KiB/s wr, 237 op/s 2024-04-07T17:47:21.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:47:21 smithi100 bash[76202]: cluster 2024-04-07T17:47:20.515824+0000 mon.a (mon.0) 4578 : cluster [DBG] osdmap e2577: 8 total, 8 up, 8 in 2024-04-07T17:47:21.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:47:21 smithi100 bash[76202]: cluster 2024-04-07T17:47:20.724682+0000 mgr.y (mgr.25175) 2759 : cluster [DBG] pgmap v4911: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 174 KiB/s rd, 5.0 KiB/s wr, 237 op/s 2024-04-07T17:47:22.614 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:47:22 smithi144 bash[66654]: cluster 2024-04-07T17:47:21.526438+0000 mon.a (mon.0) 4579 : cluster [DBG] osdmap e2578: 8 total, 8 up, 8 in 2024-04-07T17:47:22.615 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:47:22 smithi144 bash[95233]: ts=2024-04-07T17:47:22.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:47:22.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:47:22 smithi100 bash[74942]: cluster 2024-04-07T17:47:21.526438+0000 mon.a (mon.0) 4579 : cluster [DBG] osdmap e2578: 8 total, 8 up, 8 in 2024-04-07T17:47:22.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:47:22 smithi100 bash[76202]: cluster 2024-04-07T17:47:21.526438+0000 mon.a (mon.0) 4579 : cluster [DBG] osdmap e2578: 8 total, 8 up, 8 in 2024-04-07T17:47:23.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:47:23 smithi144 bash[66654]: cluster 2024-04-07T17:47:22.540885+0000 mon.a (mon.0) 4580 : cluster [DBG] osdmap e2579: 8 total, 8 up, 8 in 2024-04-07T17:47:23.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:47:23 smithi144 bash[66654]: cluster 2024-04-07T17:47:22.725374+0000 mgr.y (mgr.25175) 2760 : cluster [DBG] pgmap v4914: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 173 KiB/s rd, 5.0 KiB/s wr, 234 op/s 2024-04-07T17:47:23.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:47:23 smithi144 bash[66654]: cluster 2024-04-07T17:47:23.462921+0000 mon.a (mon.0) 4581 : cluster [DBG] osdmap e2580: 8 total, 8 up, 8 in 2024-04-07T17:47:23.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:47:23 smithi100 bash[74942]: cluster 2024-04-07T17:47:22.540885+0000 mon.a (mon.0) 4580 : cluster [DBG] osdmap e2579: 8 total, 8 up, 8 in 2024-04-07T17:47:23.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:47:23 smithi100 bash[74942]: cluster 2024-04-07T17:47:22.725374+0000 mgr.y (mgr.25175) 2760 : cluster [DBG] pgmap v4914: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 173 KiB/s rd, 5.0 KiB/s wr, 234 op/s 2024-04-07T17:47:23.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:47:23 smithi100 bash[74942]: cluster 2024-04-07T17:47:23.462921+0000 mon.a (mon.0) 4581 : cluster [DBG] osdmap e2580: 8 total, 8 up, 8 in 2024-04-07T17:47:23.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:47:23 smithi100 bash[76202]: cluster 2024-04-07T17:47:22.540885+0000 mon.a (mon.0) 4580 : cluster [DBG] osdmap e2579: 8 total, 8 up, 8 in 2024-04-07T17:47:23.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:47:23 smithi100 bash[76202]: cluster 2024-04-07T17:47:22.725374+0000 mgr.y (mgr.25175) 2760 : cluster [DBG] pgmap v4914: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 173 KiB/s rd, 5.0 KiB/s wr, 234 op/s 2024-04-07T17:47:23.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:47:23 smithi100 bash[76202]: cluster 2024-04-07T17:47:23.462921+0000 mon.a (mon.0) 4581 : cluster [DBG] osdmap e2580: 8 total, 8 up, 8 in 2024-04-07T17:47:25.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:47:25 smithi144 bash[66654]: cluster 2024-04-07T17:47:24.552746+0000 mon.a (mon.0) 4582 : cluster [DBG] osdmap e2581: 8 total, 8 up, 8 in 2024-04-07T17:47:25.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:47:25 smithi144 bash[66654]: cluster 2024-04-07T17:47:24.726153+0000 mgr.y (mgr.25175) 2761 : cluster [DBG] pgmap v4917: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 141 KiB/s rd, 5.0 KiB/s wr, 195 op/s 2024-04-07T17:47:25.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:47:25 smithi100 bash[74942]: cluster 2024-04-07T17:47:24.552746+0000 mon.a (mon.0) 4582 : cluster [DBG] osdmap e2581: 8 total, 8 up, 8 in 2024-04-07T17:47:25.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:47:25 smithi100 bash[74942]: cluster 2024-04-07T17:47:24.726153+0000 mgr.y (mgr.25175) 2761 : cluster [DBG] pgmap v4917: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 141 KiB/s rd, 5.0 KiB/s wr, 195 op/s 2024-04-07T17:47:25.955 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:47:25 smithi100 bash[76202]: cluster 2024-04-07T17:47:24.552746+0000 mon.a (mon.0) 4582 : cluster [DBG] osdmap e2581: 8 total, 8 up, 8 in 2024-04-07T17:47:25.955 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:47:25 smithi100 bash[76202]: cluster 2024-04-07T17:47:24.726153+0000 mgr.y (mgr.25175) 2761 : cluster [DBG] pgmap v4917: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 141 KiB/s rd, 5.0 KiB/s wr, 195 op/s 2024-04-07T17:47:26.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:47:26 smithi144 bash[66654]: cluster 2024-04-07T17:47:25.566885+0000 mon.a (mon.0) 4583 : cluster [DBG] osdmap e2582: 8 total, 8 up, 8 in 2024-04-07T17:47:26.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:47:26 smithi100 bash[74942]: cluster 2024-04-07T17:47:25.566885+0000 mon.a (mon.0) 4583 : cluster [DBG] osdmap e2582: 8 total, 8 up, 8 in 2024-04-07T17:47:26.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:47:26 smithi100 bash[76202]: cluster 2024-04-07T17:47:25.566885+0000 mon.a (mon.0) 4583 : cluster [DBG] osdmap e2582: 8 total, 8 up, 8 in 2024-04-07T17:47:27.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:47:27 smithi144 bash[66654]: cluster 2024-04-07T17:47:26.566064+0000 mon.a (mon.0) 4584 : cluster [DBG] osdmap e2583: 8 total, 8 up, 8 in 2024-04-07T17:47:27.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:47:27 smithi144 bash[66654]: cluster 2024-04-07T17:47:26.726935+0000 mgr.y (mgr.25175) 2762 : cluster [DBG] pgmap v4920: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 140 KiB/s rd, 5.0 KiB/s wr, 193 op/s 2024-04-07T17:47:27.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:47:27 smithi100 bash[74942]: cluster 2024-04-07T17:47:26.566064+0000 mon.a (mon.0) 4584 : cluster [DBG] osdmap e2583: 8 total, 8 up, 8 in 2024-04-07T17:47:27.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:47:27 smithi100 bash[74942]: cluster 2024-04-07T17:47:26.726935+0000 mgr.y (mgr.25175) 2762 : cluster [DBG] pgmap v4920: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 140 KiB/s rd, 5.0 KiB/s wr, 193 op/s 2024-04-07T17:47:27.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:47:27 smithi100 bash[76202]: cluster 2024-04-07T17:47:26.566064+0000 mon.a (mon.0) 4584 : cluster [DBG] osdmap e2583: 8 total, 8 up, 8 in 2024-04-07T17:47:27.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:47:27 smithi100 bash[76202]: cluster 2024-04-07T17:47:26.726935+0000 mgr.y (mgr.25175) 2762 : cluster [DBG] pgmap v4920: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 140 KiB/s rd, 5.0 KiB/s wr, 193 op/s 2024-04-07T17:47:28.703 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:47:28 smithi100 bash[74942]: cluster 2024-04-07T17:47:27.587698+0000 mon.a (mon.0) 4585 : cluster [DBG] osdmap e2584: 8 total, 8 up, 8 in 2024-04-07T17:47:28.703 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:47:28 smithi100 bash[74942]: cluster 2024-04-07T17:47:28.469307+0000 mon.a (mon.0) 4586 : cluster [DBG] osdmap e2585: 8 total, 8 up, 8 in 2024-04-07T17:47:28.704 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:47:28 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:47:28] "GET /metrics HTTP/1.1" 200 36742 "" "Prometheus/2.43.0" 2024-04-07T17:47:28.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:47:28 smithi100 bash[76202]: cluster 2024-04-07T17:47:27.587698+0000 mon.a (mon.0) 4585 : cluster [DBG] osdmap e2584: 8 total, 8 up, 8 in 2024-04-07T17:47:28.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:47:28 smithi100 bash[76202]: cluster 2024-04-07T17:47:28.469307+0000 mon.a (mon.0) 4586 : cluster [DBG] osdmap e2585: 8 total, 8 up, 8 in 2024-04-07T17:47:28.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:47:28 smithi144 bash[66654]: cluster 2024-04-07T17:47:27.587698+0000 mon.a (mon.0) 4585 : cluster [DBG] osdmap e2584: 8 total, 8 up, 8 in 2024-04-07T17:47:28.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:47:28 smithi144 bash[66654]: cluster 2024-04-07T17:47:28.469307+0000 mon.a (mon.0) 4586 : cluster [DBG] osdmap e2585: 8 total, 8 up, 8 in 2024-04-07T17:47:29.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:47:29 smithi144 bash[66654]: cluster 2024-04-07T17:47:28.727647+0000 mgr.y (mgr.25175) 2763 : cluster [DBG] pgmap v4923: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 2.7 KiB/s rd, 0 B/s wr, 4 op/s 2024-04-07T17:47:29.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:47:29 smithi144 bash[66654]: cluster 2024-04-07T17:47:29.467904+0000 mon.a (mon.0) 4587 : cluster [DBG] osdmap e2586: 8 total, 8 up, 8 in 2024-04-07T17:47:29.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:47:29 smithi100 bash[74942]: cluster 2024-04-07T17:47:28.727647+0000 mgr.y (mgr.25175) 2763 : cluster [DBG] pgmap v4923: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 2.7 KiB/s rd, 0 B/s wr, 4 op/s 2024-04-07T17:47:29.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:47:29 smithi100 bash[74942]: cluster 2024-04-07T17:47:29.467904+0000 mon.a (mon.0) 4587 : cluster [DBG] osdmap e2586: 8 total, 8 up, 8 in 2024-04-07T17:47:29.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:47:29 smithi100 bash[76202]: cluster 2024-04-07T17:47:28.727647+0000 mgr.y (mgr.25175) 2763 : cluster [DBG] pgmap v4923: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 2.7 KiB/s rd, 0 B/s wr, 4 op/s 2024-04-07T17:47:29.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:47:29 smithi100 bash[76202]: cluster 2024-04-07T17:47:29.467904+0000 mon.a (mon.0) 4587 : cluster [DBG] osdmap e2586: 8 total, 8 up, 8 in 2024-04-07T17:47:31.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:47:31 smithi144 bash[66654]: cluster 2024-04-07T17:47:30.471197+0000 mon.a (mon.0) 4588 : cluster [DBG] osdmap e2587: 8 total, 8 up, 8 in 2024-04-07T17:47:31.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:47:31 smithi144 bash[66654]: cluster 2024-04-07T17:47:30.728408+0000 mgr.y (mgr.25175) 2764 : cluster [DBG] pgmap v4926: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 110 KiB/s rd, 5.0 KiB/s wr, 151 op/s 2024-04-07T17:47:31.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:47:31 smithi100 bash[74942]: cluster 2024-04-07T17:47:30.471197+0000 mon.a (mon.0) 4588 : cluster [DBG] osdmap e2587: 8 total, 8 up, 8 in 2024-04-07T17:47:31.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:47:31 smithi100 bash[74942]: cluster 2024-04-07T17:47:30.728408+0000 mgr.y (mgr.25175) 2764 : cluster [DBG] pgmap v4926: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 110 KiB/s rd, 5.0 KiB/s wr, 151 op/s 2024-04-07T17:47:31.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:47:31 smithi100 bash[76202]: cluster 2024-04-07T17:47:30.471197+0000 mon.a (mon.0) 4588 : cluster [DBG] osdmap e2587: 8 total, 8 up, 8 in 2024-04-07T17:47:31.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:47:31 smithi100 bash[76202]: cluster 2024-04-07T17:47:30.728408+0000 mgr.y (mgr.25175) 2764 : cluster [DBG] pgmap v4926: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 110 KiB/s rd, 5.0 KiB/s wr, 151 op/s 2024-04-07T17:47:32.614 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:47:32 smithi144 bash[95233]: ts=2024-04-07T17:47:32.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:47:32.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:47:32 smithi144 bash[66654]: cluster 2024-04-07T17:47:31.487010+0000 mon.a (mon.0) 4589 : cluster [DBG] osdmap e2588: 8 total, 8 up, 8 in 2024-04-07T17:47:32.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:47:32 smithi144 bash[66654]: audit 2024-04-07T17:47:31.878852+0000 mon.a (mon.0) 4590 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:47:32.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:47:32 smithi100 bash[74942]: cluster 2024-04-07T17:47:31.487010+0000 mon.a (mon.0) 4589 : cluster [DBG] osdmap e2588: 8 total, 8 up, 8 in 2024-04-07T17:47:32.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:47:32 smithi100 bash[74942]: audit 2024-04-07T17:47:31.878852+0000 mon.a (mon.0) 4590 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:47:32.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:47:32 smithi100 bash[76202]: cluster 2024-04-07T17:47:31.487010+0000 mon.a (mon.0) 4589 : cluster [DBG] osdmap e2588: 8 total, 8 up, 8 in 2024-04-07T17:47:32.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:47:32 smithi100 bash[76202]: audit 2024-04-07T17:47:31.878852+0000 mon.a (mon.0) 4590 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:47:33.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:47:33 smithi144 bash[66654]: cluster 2024-04-07T17:47:32.496405+0000 mon.a (mon.0) 4591 : cluster [DBG] osdmap e2589: 8 total, 8 up, 8 in 2024-04-07T17:47:33.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:47:33 smithi144 bash[66654]: cluster 2024-04-07T17:47:32.729240+0000 mgr.y (mgr.25175) 2765 : cluster [DBG] pgmap v4929: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 110 KiB/s rd, 5.0 KiB/s wr, 150 op/s 2024-04-07T17:47:33.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:47:33 smithi144 bash[66654]: cluster 2024-04-07T17:47:33.466737+0000 mon.a (mon.0) 4592 : cluster [DBG] osdmap e2590: 8 total, 8 up, 8 in 2024-04-07T17:47:33.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:47:33 smithi100 bash[74942]: cluster 2024-04-07T17:47:32.496405+0000 mon.a (mon.0) 4591 : cluster [DBG] osdmap e2589: 8 total, 8 up, 8 in 2024-04-07T17:47:33.955 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:47:33 smithi100 bash[74942]: cluster 2024-04-07T17:47:32.729240+0000 mgr.y (mgr.25175) 2765 : cluster [DBG] pgmap v4929: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 110 KiB/s rd, 5.0 KiB/s wr, 150 op/s 2024-04-07T17:47:33.955 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:47:33 smithi100 bash[74942]: cluster 2024-04-07T17:47:33.466737+0000 mon.a (mon.0) 4592 : cluster [DBG] osdmap e2590: 8 total, 8 up, 8 in 2024-04-07T17:47:33.955 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:47:33 smithi100 bash[76202]: cluster 2024-04-07T17:47:32.496405+0000 mon.a (mon.0) 4591 : cluster [DBG] osdmap e2589: 8 total, 8 up, 8 in 2024-04-07T17:47:33.955 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:47:33 smithi100 bash[76202]: cluster 2024-04-07T17:47:32.729240+0000 mgr.y (mgr.25175) 2765 : cluster [DBG] pgmap v4929: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 110 KiB/s rd, 5.0 KiB/s wr, 150 op/s 2024-04-07T17:47:33.955 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:47:33 smithi100 bash[76202]: cluster 2024-04-07T17:47:33.466737+0000 mon.a (mon.0) 4592 : cluster [DBG] osdmap e2590: 8 total, 8 up, 8 in 2024-04-07T17:47:35.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:47:35 smithi144 bash[66654]: cluster 2024-04-07T17:47:34.518466+0000 mon.a (mon.0) 4593 : cluster [DBG] osdmap e2591: 8 total, 8 up, 8 in 2024-04-07T17:47:35.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:47:35 smithi144 bash[66654]: cluster 2024-04-07T17:47:34.730028+0000 mgr.y (mgr.25175) 2766 : cluster [DBG] pgmap v4932: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 75 KiB/s rd, 5.0 KiB/s wr, 104 op/s 2024-04-07T17:47:35.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:47:35 smithi100 bash[74942]: cluster 2024-04-07T17:47:34.518466+0000 mon.a (mon.0) 4593 : cluster [DBG] osdmap e2591: 8 total, 8 up, 8 in 2024-04-07T17:47:35.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:47:35 smithi100 bash[74942]: cluster 2024-04-07T17:47:34.730028+0000 mgr.y (mgr.25175) 2766 : cluster [DBG] pgmap v4932: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 75 KiB/s rd, 5.0 KiB/s wr, 104 op/s 2024-04-07T17:47:35.953 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:47:35 smithi100 bash[76202]: cluster 2024-04-07T17:47:34.518466+0000 mon.a (mon.0) 4593 : cluster [DBG] osdmap e2591: 8 total, 8 up, 8 in 2024-04-07T17:47:35.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:47:35 smithi100 bash[76202]: cluster 2024-04-07T17:47:34.730028+0000 mgr.y (mgr.25175) 2766 : cluster [DBG] pgmap v4932: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 75 KiB/s rd, 5.0 KiB/s wr, 104 op/s 2024-04-07T17:47:36.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:47:36 smithi144 bash[66654]: cluster 2024-04-07T17:47:35.526905+0000 mon.a (mon.0) 4594 : cluster [DBG] osdmap e2592: 8 total, 8 up, 8 in 2024-04-07T17:47:36.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:47:36 smithi100 bash[74942]: cluster 2024-04-07T17:47:35.526905+0000 mon.a (mon.0) 4594 : cluster [DBG] osdmap e2592: 8 total, 8 up, 8 in 2024-04-07T17:47:36.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:47:36 smithi100 bash[76202]: cluster 2024-04-07T17:47:35.526905+0000 mon.a (mon.0) 4594 : cluster [DBG] osdmap e2592: 8 total, 8 up, 8 in 2024-04-07T17:47:37.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:47:37 smithi144 bash[66654]: cluster 2024-04-07T17:47:36.528545+0000 mon.a (mon.0) 4595 : cluster [DBG] osdmap e2593: 8 total, 8 up, 8 in 2024-04-07T17:47:37.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:47:37 smithi144 bash[66654]: cluster 2024-04-07T17:47:36.730781+0000 mgr.y (mgr.25175) 2767 : cluster [DBG] pgmap v4935: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 74 KiB/s rd, 5.0 KiB/s wr, 103 op/s 2024-04-07T17:47:37.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:47:37 smithi100 bash[74942]: cluster 2024-04-07T17:47:36.528545+0000 mon.a (mon.0) 4595 : cluster [DBG] osdmap e2593: 8 total, 8 up, 8 in 2024-04-07T17:47:37.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:47:37 smithi100 bash[74942]: cluster 2024-04-07T17:47:36.730781+0000 mgr.y (mgr.25175) 2767 : cluster [DBG] pgmap v4935: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 74 KiB/s rd, 5.0 KiB/s wr, 103 op/s 2024-04-07T17:47:37.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:47:37 smithi100 bash[76202]: cluster 2024-04-07T17:47:36.528545+0000 mon.a (mon.0) 4595 : cluster [DBG] osdmap e2593: 8 total, 8 up, 8 in 2024-04-07T17:47:37.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:47:37 smithi100 bash[76202]: cluster 2024-04-07T17:47:36.730781+0000 mgr.y (mgr.25175) 2767 : cluster [DBG] pgmap v4935: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 74 KiB/s rd, 5.0 KiB/s wr, 103 op/s 2024-04-07T17:47:38.701 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_many_snaps PASSED [ 47%] 2024-04-07T17:47:38.703 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:47:38 smithi100 bash[74942]: cluster 2024-04-07T17:47:37.557433+0000 mon.a (mon.0) 4596 : cluster [DBG] osdmap e2594: 8 total, 8 up, 8 in 2024-04-07T17:47:38.703 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:47:38 smithi100 bash[74942]: cluster 2024-04-07T17:47:38.470429+0000 mon.a (mon.0) 4597 : cluster [DBG] osdmap e2595: 8 total, 8 up, 8 in 2024-04-07T17:47:38.703 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:47:38 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:47:38] "GET /metrics HTTP/1.1" 200 36742 "" "Prometheus/2.43.0" 2024-04-07T17:47:38.703 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:47:38 smithi100 bash[76202]: cluster 2024-04-07T17:47:37.557433+0000 mon.a (mon.0) 4596 : cluster [DBG] osdmap e2594: 8 total, 8 up, 8 in 2024-04-07T17:47:38.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:47:38 smithi100 bash[76202]: cluster 2024-04-07T17:47:38.470429+0000 mon.a (mon.0) 4597 : cluster [DBG] osdmap e2595: 8 total, 8 up, 8 in 2024-04-07T17:47:38.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:47:38 smithi144 bash[66654]: cluster 2024-04-07T17:47:37.557433+0000 mon.a (mon.0) 4596 : cluster [DBG] osdmap e2594: 8 total, 8 up, 8 in 2024-04-07T17:47:38.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:47:38 smithi144 bash[66654]: cluster 2024-04-07T17:47:38.470429+0000 mon.a (mon.0) 4597 : cluster [DBG] osdmap e2595: 8 total, 8 up, 8 in 2024-04-07T17:47:39.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:47:39 smithi144 bash[66654]: cluster 2024-04-07T17:47:38.731527+0000 mgr.y (mgr.25175) 2768 : cluster [DBG] pgmap v4938: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 3.5 KiB/s rd, 0 B/s wr, 6 op/s 2024-04-07T17:47:39.953 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:47:39 smithi100 bash[76202]: cluster 2024-04-07T17:47:38.731527+0000 mgr.y (mgr.25175) 2768 : cluster [DBG] pgmap v4938: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 3.5 KiB/s rd, 0 B/s wr, 6 op/s 2024-04-07T17:47:39.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:47:39 smithi100 bash[74942]: cluster 2024-04-07T17:47:38.731527+0000 mgr.y (mgr.25175) 2768 : cluster [DBG] pgmap v4938: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 3.5 KiB/s rd, 0 B/s wr, 6 op/s 2024-04-07T17:47:40.739 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_set_snap_deleted PASSED [ 48%] 2024-04-07T17:47:40.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:47:40 smithi144 bash[66654]: cluster 2024-04-07T17:47:39.563993+0000 mon.a (mon.0) 4598 : cluster [DBG] osdmap e2596: 8 total, 8 up, 8 in 2024-04-07T17:47:40.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:47:40 smithi144 bash[66654]: audit 2024-04-07T17:47:39.662968+0000 mon.a (mon.0) 4599 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:47:40.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:47:40 smithi144 bash[66654]: audit 2024-04-07T17:47:40.189435+0000 mon.a (mon.0) 4600 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:47:40.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:47:40 smithi144 bash[66654]: audit 2024-04-07T17:47:40.195461+0000 mon.a (mon.0) 4601 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:47:40.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:47:40 smithi100 bash[74942]: cluster 2024-04-07T17:47:39.563993+0000 mon.a (mon.0) 4598 : cluster [DBG] osdmap e2596: 8 total, 8 up, 8 in 2024-04-07T17:47:40.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:47:40 smithi100 bash[74942]: audit 2024-04-07T17:47:39.662968+0000 mon.a (mon.0) 4599 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:47:40.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:47:40 smithi100 bash[74942]: audit 2024-04-07T17:47:40.189435+0000 mon.a (mon.0) 4600 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:47:40.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:47:40 smithi100 bash[74942]: audit 2024-04-07T17:47:40.195461+0000 mon.a (mon.0) 4601 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:47:40.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:47:40 smithi100 bash[76202]: cluster 2024-04-07T17:47:39.563993+0000 mon.a (mon.0) 4598 : cluster [DBG] osdmap e2596: 8 total, 8 up, 8 in 2024-04-07T17:47:40.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:47:40 smithi100 bash[76202]: audit 2024-04-07T17:47:39.662968+0000 mon.a (mon.0) 4599 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:47:40.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:47:40 smithi100 bash[76202]: audit 2024-04-07T17:47:40.189435+0000 mon.a (mon.0) 4600 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:47:40.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:47:40 smithi100 bash[76202]: audit 2024-04-07T17:47:40.195461+0000 mon.a (mon.0) 4601 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:47:41.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:47:41 smithi144 bash[66654]: cluster 2024-04-07T17:47:40.565347+0000 mon.a (mon.0) 4602 : cluster [DBG] osdmap e2597: 8 total, 8 up, 8 in 2024-04-07T17:47:41.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:47:41 smithi144 bash[66654]: cluster 2024-04-07T17:47:40.732179+0000 mgr.y (mgr.25175) 2769 : cluster [DBG] pgmap v4941: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 61 KiB/s rd, 5.0 KiB/s wr, 83 op/s 2024-04-07T17:47:41.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:47:41 smithi144 bash[66654]: audit 2024-04-07T17:47:40.830584+0000 mon.a (mon.0) 4603 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:47:41.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:47:41 smithi144 bash[66654]: audit 2024-04-07T17:47:40.832281+0000 mon.a (mon.0) 4604 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:47:41.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:47:41 smithi144 bash[66654]: audit 2024-04-07T17:47:40.842448+0000 mon.a (mon.0) 4605 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:47:41.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:47:41 smithi100 bash[74942]: cluster 2024-04-07T17:47:40.565347+0000 mon.a (mon.0) 4602 : cluster [DBG] osdmap e2597: 8 total, 8 up, 8 in 2024-04-07T17:47:41.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:47:41 smithi100 bash[74942]: cluster 2024-04-07T17:47:40.732179+0000 mgr.y (mgr.25175) 2769 : cluster [DBG] pgmap v4941: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 61 KiB/s rd, 5.0 KiB/s wr, 83 op/s 2024-04-07T17:47:41.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:47:41 smithi100 bash[74942]: audit 2024-04-07T17:47:40.830584+0000 mon.a (mon.0) 4603 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:47:41.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:47:41 smithi100 bash[74942]: audit 2024-04-07T17:47:40.832281+0000 mon.a (mon.0) 4604 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:47:41.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:47:41 smithi100 bash[74942]: audit 2024-04-07T17:47:40.842448+0000 mon.a (mon.0) 4605 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:47:41.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:47:41 smithi100 bash[76202]: cluster 2024-04-07T17:47:40.565347+0000 mon.a (mon.0) 4602 : cluster [DBG] osdmap e2597: 8 total, 8 up, 8 in 2024-04-07T17:47:41.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:47:41 smithi100 bash[76202]: cluster 2024-04-07T17:47:40.732179+0000 mgr.y (mgr.25175) 2769 : cluster [DBG] pgmap v4941: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 61 KiB/s rd, 5.0 KiB/s wr, 83 op/s 2024-04-07T17:47:41.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:47:41 smithi100 bash[76202]: audit 2024-04-07T17:47:40.830584+0000 mon.a (mon.0) 4603 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:47:41.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:47:41 smithi100 bash[76202]: audit 2024-04-07T17:47:40.832281+0000 mon.a (mon.0) 4604 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:47:41.955 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:47:41 smithi100 bash[76202]: audit 2024-04-07T17:47:40.842448+0000 mon.a (mon.0) 4605 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:47:42.584 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:47:42 smithi144 bash[95233]: ts=2024-04-07T17:47:42.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:47:42.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:47:42 smithi144 bash[66654]: cluster 2024-04-07T17:47:41.591080+0000 mon.a (mon.0) 4606 : cluster [DBG] osdmap e2598: 8 total, 8 up, 8 in 2024-04-07T17:47:42.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:47:42 smithi100 bash[74942]: cluster 2024-04-07T17:47:41.591080+0000 mon.a (mon.0) 4606 : cluster [DBG] osdmap e2598: 8 total, 8 up, 8 in 2024-04-07T17:47:42.953 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:47:42 smithi100 bash[76202]: cluster 2024-04-07T17:47:41.591080+0000 mon.a (mon.0) 4606 : cluster [DBG] osdmap e2598: 8 total, 8 up, 8 in 2024-04-07T17:47:43.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:47:43 smithi144 bash[66654]: cluster 2024-04-07T17:47:42.600022+0000 mon.a (mon.0) 4607 : cluster [DBG] osdmap e2599: 8 total, 8 up, 8 in 2024-04-07T17:47:43.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:47:43 smithi144 bash[66654]: cluster 2024-04-07T17:47:42.732874+0000 mgr.y (mgr.25175) 2770 : cluster [DBG] pgmap v4944: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 114 KiB/s rd, 11 KiB/s wr, 157 op/s 2024-04-07T17:47:43.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:47:43 smithi144 bash[66654]: cluster 2024-04-07T17:47:43.471770+0000 mon.a (mon.0) 4608 : cluster [DBG] osdmap e2600: 8 total, 8 up, 8 in 2024-04-07T17:47:43.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:47:43 smithi100 bash[74942]: cluster 2024-04-07T17:47:42.600022+0000 mon.a (mon.0) 4607 : cluster [DBG] osdmap e2599: 8 total, 8 up, 8 in 2024-04-07T17:47:43.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:47:43 smithi100 bash[74942]: cluster 2024-04-07T17:47:42.732874+0000 mgr.y (mgr.25175) 2770 : cluster [DBG] pgmap v4944: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 114 KiB/s rd, 11 KiB/s wr, 157 op/s 2024-04-07T17:47:43.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:47:43 smithi100 bash[74942]: cluster 2024-04-07T17:47:43.471770+0000 mon.a (mon.0) 4608 : cluster [DBG] osdmap e2600: 8 total, 8 up, 8 in 2024-04-07T17:47:43.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:47:43 smithi100 bash[76202]: cluster 2024-04-07T17:47:42.600022+0000 mon.a (mon.0) 4607 : cluster [DBG] osdmap e2599: 8 total, 8 up, 8 in 2024-04-07T17:47:43.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:47:43 smithi100 bash[76202]: cluster 2024-04-07T17:47:42.732874+0000 mgr.y (mgr.25175) 2770 : cluster [DBG] pgmap v4944: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 114 KiB/s rd, 11 KiB/s wr, 157 op/s 2024-04-07T17:47:43.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:47:43 smithi100 bash[76202]: cluster 2024-04-07T17:47:43.471770+0000 mon.a (mon.0) 4608 : cluster [DBG] osdmap e2600: 8 total, 8 up, 8 in 2024-04-07T17:47:44.694 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_set_snap_recreated PASSED [ 48%] 2024-04-07T17:47:44.941 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_lock_unlock PASSED [ 49%] 2024-04-07T17:47:45.381 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_list_lockers PASSED [ 49%] 2024-04-07T17:47:45.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:47:45 smithi144 bash[66654]: cluster 2024-04-07T17:47:44.473804+0000 mon.a (mon.0) 4609 : cluster [DBG] osdmap e2601: 8 total, 8 up, 8 in 2024-04-07T17:47:45.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:47:45 smithi144 bash[66654]: cluster 2024-04-07T17:47:44.733688+0000 mgr.y (mgr.25175) 2771 : cluster [DBG] pgmap v4947: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 92 KiB/s rd, 13 KiB/s wr, 133 op/s 2024-04-07T17:47:45.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:47:45 smithi100 bash[74942]: cluster 2024-04-07T17:47:44.473804+0000 mon.a (mon.0) 4609 : cluster [DBG] osdmap e2601: 8 total, 8 up, 8 in 2024-04-07T17:47:45.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:47:45 smithi100 bash[74942]: cluster 2024-04-07T17:47:44.733688+0000 mgr.y (mgr.25175) 2771 : cluster [DBG] pgmap v4947: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 92 KiB/s rd, 13 KiB/s wr, 133 op/s 2024-04-07T17:47:45.953 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:47:45 smithi100 bash[76202]: cluster 2024-04-07T17:47:44.473804+0000 mon.a (mon.0) 4609 : cluster [DBG] osdmap e2601: 8 total, 8 up, 8 in 2024-04-07T17:47:45.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:47:45 smithi100 bash[76202]: cluster 2024-04-07T17:47:44.733688+0000 mgr.y (mgr.25175) 2771 : cluster [DBG] pgmap v4947: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 92 KiB/s rd, 13 KiB/s wr, 133 op/s 2024-04-07T17:47:46.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:47:46 smithi144 bash[66654]: cluster 2024-04-07T17:47:45.486987+0000 mon.a (mon.0) 4610 : cluster [DBG] osdmap e2602: 8 total, 8 up, 8 in 2024-04-07T17:47:46.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:47:46 smithi100 bash[74942]: cluster 2024-04-07T17:47:45.486987+0000 mon.a (mon.0) 4610 : cluster [DBG] osdmap e2602: 8 total, 8 up, 8 in 2024-04-07T17:47:46.953 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:47:46 smithi100 bash[76202]: cluster 2024-04-07T17:47:45.486987+0000 mon.a (mon.0) 4610 : cluster [DBG] osdmap e2602: 8 total, 8 up, 8 in 2024-04-07T17:47:47.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:47:47 smithi144 bash[66654]: cluster 2024-04-07T17:47:46.488204+0000 mon.a (mon.0) 4611 : cluster [DBG] osdmap e2603: 8 total, 8 up, 8 in 2024-04-07T17:47:47.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:47:47 smithi144 bash[66654]: cluster 2024-04-07T17:47:46.734507+0000 mgr.y (mgr.25175) 2772 : cluster [DBG] pgmap v4950: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 36 KiB/s rd, 7.0 KiB/s wr, 53 op/s 2024-04-07T17:47:47.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:47:47 smithi144 bash[66654]: audit 2024-04-07T17:47:46.879387+0000 mon.a (mon.0) 4612 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:47:47.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:47:47 smithi100 bash[74942]: cluster 2024-04-07T17:47:46.488204+0000 mon.a (mon.0) 4611 : cluster [DBG] osdmap e2603: 8 total, 8 up, 8 in 2024-04-07T17:47:47.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:47:47 smithi100 bash[74942]: cluster 2024-04-07T17:47:46.734507+0000 mgr.y (mgr.25175) 2772 : cluster [DBG] pgmap v4950: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 36 KiB/s rd, 7.0 KiB/s wr, 53 op/s 2024-04-07T17:47:47.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:47:47 smithi100 bash[74942]: audit 2024-04-07T17:47:46.879387+0000 mon.a (mon.0) 4612 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:47:47.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:47:47 smithi100 bash[76202]: cluster 2024-04-07T17:47:46.488204+0000 mon.a (mon.0) 4611 : cluster [DBG] osdmap e2603: 8 total, 8 up, 8 in 2024-04-07T17:47:47.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:47:47 smithi100 bash[76202]: cluster 2024-04-07T17:47:46.734507+0000 mgr.y (mgr.25175) 2772 : cluster [DBG] pgmap v4950: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 36 KiB/s rd, 7.0 KiB/s wr, 53 op/s 2024-04-07T17:47:47.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:47:47 smithi100 bash[76202]: audit 2024-04-07T17:47:46.879387+0000 mon.a (mon.0) 4612 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:47:48.673 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_diff_iterate PASSED [ 50%] 2024-04-07T17:47:48.703 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:47:48 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:47:48] "GET /metrics HTTP/1.1" 200 36739 "" "Prometheus/2.43.0" 2024-04-07T17:47:48.703 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:47:48 smithi100 bash[74942]: cluster 2024-04-07T17:47:47.509504+0000 mon.a (mon.0) 4613 : cluster [DBG] osdmap e2604: 8 total, 8 up, 8 in 2024-04-07T17:47:48.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:47:48 smithi100 bash[74942]: cluster 2024-04-07T17:47:48.475844+0000 mon.a (mon.0) 4614 : cluster [DBG] osdmap e2605: 8 total, 8 up, 8 in 2024-04-07T17:47:48.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:47:48 smithi100 bash[76202]: cluster 2024-04-07T17:47:47.509504+0000 mon.a (mon.0) 4613 : cluster [DBG] osdmap e2604: 8 total, 8 up, 8 in 2024-04-07T17:47:48.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:47:48 smithi100 bash[76202]: cluster 2024-04-07T17:47:48.475844+0000 mon.a (mon.0) 4614 : cluster [DBG] osdmap e2605: 8 total, 8 up, 8 in 2024-04-07T17:47:48.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:47:48 smithi144 bash[66654]: cluster 2024-04-07T17:47:47.509504+0000 mon.a (mon.0) 4613 : cluster [DBG] osdmap e2604: 8 total, 8 up, 8 in 2024-04-07T17:47:48.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:47:48 smithi144 bash[66654]: cluster 2024-04-07T17:47:48.475844+0000 mon.a (mon.0) 4614 : cluster [DBG] osdmap e2605: 8 total, 8 up, 8 in 2024-04-07T17:47:48.889 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_aio_read PASSED [ 50%] 2024-04-07T17:47:49.128 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_aio_write PASSED [ 51%] 2024-04-07T17:47:49.406 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_aio_discard PASSED [ 52%] 2024-04-07T17:47:49.663 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_aio_write_zeroes PASSED [ 52%] 2024-04-07T17:47:49.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:47:49 smithi144 bash[66654]: cluster 2024-04-07T17:47:48.735359+0000 mgr.y (mgr.25175) 2773 : cluster [DBG] pgmap v4953: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 76 KiB/s rd, 6.0 KiB/s wr, 105 op/s 2024-04-07T17:47:49.895 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_aio_flush PASSED [ 53%] 2024-04-07T17:47:49.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:47:49 smithi100 bash[74942]: cluster 2024-04-07T17:47:48.735359+0000 mgr.y (mgr.25175) 2773 : cluster [DBG] pgmap v4953: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 76 KiB/s rd, 6.0 KiB/s wr, 105 op/s 2024-04-07T17:47:49.953 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:47:49 smithi100 bash[76202]: cluster 2024-04-07T17:47:48.735359+0000 mgr.y (mgr.25175) 2773 : cluster [DBG] pgmap v4953: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 76 KiB/s rd, 6.0 KiB/s wr, 105 op/s 2024-04-07T17:47:51.848 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_metadata PASSED [ 53%] 2024-04-07T17:47:52.064 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_watchers_list PASSED [ 54%] 2024-04-07T17:47:52.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:47:51 smithi144 bash[66654]: cluster 2024-04-07T17:47:50.736529+0000 mgr.y (mgr.25175) 2774 : cluster [DBG] pgmap v4954: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 192 KiB/s rd, 11 KiB/s wr, 253 op/s 2024-04-07T17:47:52.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:47:51 smithi100 bash[74942]: cluster 2024-04-07T17:47:50.736529+0000 mgr.y (mgr.25175) 2774 : cluster [DBG] pgmap v4954: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 192 KiB/s rd, 11 KiB/s wr, 253 op/s 2024-04-07T17:47:52.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:47:51 smithi100 bash[76202]: cluster 2024-04-07T17:47:50.736529+0000 mgr.y (mgr.25175) 2774 : cluster [DBG] pgmap v4954: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 192 KiB/s rd, 11 KiB/s wr, 253 op/s 2024-04-07T17:47:52.363 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_config_list PASSED [ 54%] 2024-04-07T17:47:52.614 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:47:52 smithi144 bash[95233]: ts=2024-04-07T17:47:52.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:47:52.652 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_image_config_set_and_get_and_remove PASSED [ 55%] 2024-04-07T17:47:52.888 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_sparsify PASSED [ 56%] 2024-04-07T17:47:54.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:47:53 smithi144 bash[66654]: cluster 2024-04-07T17:47:52.737992+0000 mgr.y (mgr.25175) 2775 : cluster [DBG] pgmap v4955: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 255 KiB/s rd, 14 KiB/s wr, 336 op/s 2024-04-07T17:47:54.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:47:53 smithi144 bash[66654]: cluster 2024-04-07T17:47:53.476007+0000 mon.a (mon.0) 4615 : cluster [DBG] osdmap e2606: 8 total, 8 up, 8 in 2024-04-07T17:47:54.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:47:53 smithi100 bash[74942]: cluster 2024-04-07T17:47:52.737992+0000 mgr.y (mgr.25175) 2775 : cluster [DBG] pgmap v4955: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 255 KiB/s rd, 14 KiB/s wr, 336 op/s 2024-04-07T17:47:54.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:47:53 smithi100 bash[74942]: cluster 2024-04-07T17:47:53.476007+0000 mon.a (mon.0) 4615 : cluster [DBG] osdmap e2606: 8 total, 8 up, 8 in 2024-04-07T17:47:54.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:47:53 smithi100 bash[76202]: cluster 2024-04-07T17:47:52.737992+0000 mgr.y (mgr.25175) 2775 : cluster [DBG] pgmap v4955: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 255 KiB/s rd, 14 KiB/s wr, 336 op/s 2024-04-07T17:47:54.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:47:53 smithi100 bash[76202]: cluster 2024-04-07T17:47:53.476007+0000 mon.a (mon.0) 4615 : cluster [DBG] osdmap e2606: 8 total, 8 up, 8 in 2024-04-07T17:47:56.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:47:55 smithi144 bash[66654]: cluster 2024-04-07T17:47:54.739085+0000 mgr.y (mgr.25175) 2776 : cluster [DBG] pgmap v4957: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 293 KiB/s rd, 17 KiB/s wr, 394 op/s 2024-04-07T17:47:56.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:47:55 smithi100 bash[76202]: cluster 2024-04-07T17:47:54.739085+0000 mgr.y (mgr.25175) 2776 : cluster [DBG] pgmap v4957: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 293 KiB/s rd, 17 KiB/s wr, 394 op/s 2024-04-07T17:47:56.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:47:55 smithi100 bash[74942]: cluster 2024-04-07T17:47:54.739085+0000 mgr.y (mgr.25175) 2776 : cluster [DBG] pgmap v4957: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 293 KiB/s rd, 17 KiB/s wr, 394 op/s 2024-04-07T17:47:58.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:47:57 smithi144 bash[66654]: cluster 2024-04-07T17:47:56.740094+0000 mgr.y (mgr.25175) 2777 : cluster [DBG] pgmap v4958: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 1.5 MiB/s rd, 31 KiB/s wr, 656 op/s 2024-04-07T17:47:58.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:47:57 smithi100 bash[74942]: cluster 2024-04-07T17:47:56.740094+0000 mgr.y (mgr.25175) 2777 : cluster [DBG] pgmap v4958: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 1.5 MiB/s rd, 31 KiB/s wr, 656 op/s 2024-04-07T17:47:58.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:47:57 smithi100 bash[76202]: cluster 2024-04-07T17:47:56.740094+0000 mgr.y (mgr.25175) 2777 : cluster [DBG] pgmap v4958: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 1.5 MiB/s rd, 31 KiB/s wr, 656 op/s 2024-04-07T17:47:58.703 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:47:58 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:47:58] "GET /metrics HTTP/1.1" 200 36739 "" "Prometheus/2.43.0" 2024-04-07T17:47:59.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:47:59 smithi144 bash[66654]: cluster 2024-04-07T17:47:58.476835+0000 mon.a (mon.0) 4616 : cluster [DBG] osdmap e2607: 8 total, 8 up, 8 in 2024-04-07T17:47:59.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:47:59 smithi144 bash[66654]: cluster 2024-04-07T17:47:58.740799+0000 mgr.y (mgr.25175) 2778 : cluster [DBG] pgmap v4960: 137 pgs: 137 active+clean; 8.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 1.6 MiB/s rd, 541 KiB/s wr, 604 op/s 2024-04-07T17:47:59.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:47:59 smithi100 bash[76202]: cluster 2024-04-07T17:47:58.476835+0000 mon.a (mon.0) 4616 : cluster [DBG] osdmap e2607: 8 total, 8 up, 8 in 2024-04-07T17:47:59.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:47:59 smithi100 bash[76202]: cluster 2024-04-07T17:47:58.740799+0000 mgr.y (mgr.25175) 2778 : cluster [DBG] pgmap v4960: 137 pgs: 137 active+clean; 8.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 1.6 MiB/s rd, 541 KiB/s wr, 604 op/s 2024-04-07T17:47:59.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:47:59 smithi100 bash[74942]: cluster 2024-04-07T17:47:58.476835+0000 mon.a (mon.0) 4616 : cluster [DBG] osdmap e2607: 8 total, 8 up, 8 in 2024-04-07T17:47:59.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:47:59 smithi100 bash[74942]: cluster 2024-04-07T17:47:58.740799+0000 mgr.y (mgr.25175) 2778 : cluster [DBG] pgmap v4960: 137 pgs: 137 active+clean; 8.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 1.6 MiB/s rd, 541 KiB/s wr, 604 op/s 2024-04-07T17:48:01.404 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_encryption_luks1 PASSED [ 56%] 2024-04-07T17:48:02.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:48:01 smithi144 bash[66654]: cluster 2024-04-07T17:48:00.741913+0000 mgr.y (mgr.25175) 2779 : cluster [DBG] pgmap v4961: 137 pgs: 137 active+clean; 8.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 1.5 MiB/s rd, 537 KiB/s wr, 514 op/s 2024-04-07T17:48:02.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:48:01 smithi100 bash[74942]: cluster 2024-04-07T17:48:00.741913+0000 mgr.y (mgr.25175) 2779 : cluster [DBG] pgmap v4961: 137 pgs: 137 active+clean; 8.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 1.5 MiB/s rd, 537 KiB/s wr, 514 op/s 2024-04-07T17:48:02.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:48:01 smithi100 bash[76202]: cluster 2024-04-07T17:48:00.741913+0000 mgr.y (mgr.25175) 2779 : cluster [DBG] pgmap v4961: 137 pgs: 137 active+clean; 8.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 1.5 MiB/s rd, 537 KiB/s wr, 514 op/s 2024-04-07T17:48:02.614 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:48:02 smithi144 bash[95233]: ts=2024-04-07T17:48:02.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:48:03.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:48:02 smithi144 bash[66654]: audit 2024-04-07T17:48:01.879569+0000 mon.a (mon.0) 4617 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:48:03.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:48:02 smithi100 bash[74942]: audit 2024-04-07T17:48:01.879569+0000 mon.a (mon.0) 4617 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:48:03.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:48:02 smithi100 bash[76202]: audit 2024-04-07T17:48:01.879569+0000 mon.a (mon.0) 4617 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:48:04.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:48:03 smithi144 bash[66654]: cluster 2024-04-07T17:48:02.742803+0000 mgr.y (mgr.25175) 2780 : cluster [DBG] pgmap v4962: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 1.3 MiB/s rd, 463 KiB/s wr, 406 op/s 2024-04-07T17:48:04.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:48:03 smithi100 bash[74942]: cluster 2024-04-07T17:48:02.742803+0000 mgr.y (mgr.25175) 2780 : cluster [DBG] pgmap v4962: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 1.3 MiB/s rd, 463 KiB/s wr, 406 op/s 2024-04-07T17:48:04.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:48:03 smithi100 bash[76202]: cluster 2024-04-07T17:48:02.742803+0000 mgr.y (mgr.25175) 2780 : cluster [DBG] pgmap v4962: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 1.3 MiB/s rd, 463 KiB/s wr, 406 op/s 2024-04-07T17:48:06.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:48:05 smithi144 bash[66654]: cluster 2024-04-07T17:48:04.743438+0000 mgr.y (mgr.25175) 2781 : cluster [DBG] pgmap v4963: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 1.2 MiB/s rd, 426 KiB/s wr, 327 op/s 2024-04-07T17:48:06.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:48:05 smithi100 bash[74942]: cluster 2024-04-07T17:48:04.743438+0000 mgr.y (mgr.25175) 2781 : cluster [DBG] pgmap v4963: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 1.2 MiB/s rd, 426 KiB/s wr, 327 op/s 2024-04-07T17:48:06.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:48:05 smithi100 bash[76202]: cluster 2024-04-07T17:48:04.743438+0000 mgr.y (mgr.25175) 2781 : cluster [DBG] pgmap v4963: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 1.2 MiB/s rd, 426 KiB/s wr, 327 op/s 2024-04-07T17:48:08.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:48:07 smithi144 bash[66654]: cluster 2024-04-07T17:48:06.744005+0000 mgr.y (mgr.25175) 2782 : cluster [DBG] pgmap v4964: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 105 KiB/s rd, 413 KiB/s wr, 65 op/s 2024-04-07T17:48:08.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:48:07 smithi100 bash[74942]: cluster 2024-04-07T17:48:06.744005+0000 mgr.y (mgr.25175) 2782 : cluster [DBG] pgmap v4964: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 105 KiB/s rd, 413 KiB/s wr, 65 op/s 2024-04-07T17:48:08.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:48:07 smithi100 bash[76202]: cluster 2024-04-07T17:48:06.744005+0000 mgr.y (mgr.25175) 2782 : cluster [DBG] pgmap v4964: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 105 KiB/s rd, 413 KiB/s wr, 65 op/s 2024-04-07T17:48:08.703 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:48:08 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:48:08] "GET /metrics HTTP/1.1" 200 36755 "" "Prometheus/2.43.0" 2024-04-07T17:48:10.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:48:09 smithi144 bash[66654]: cluster 2024-04-07T17:48:08.745364+0000 mgr.y (mgr.25175) 2783 : cluster [DBG] pgmap v4965: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 52 KiB/s rd, 1.8 KiB/s wr, 32 op/s 2024-04-07T17:48:10.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:48:09 smithi100 bash[74942]: cluster 2024-04-07T17:48:08.745364+0000 mgr.y (mgr.25175) 2783 : cluster [DBG] pgmap v4965: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 52 KiB/s rd, 1.8 KiB/s wr, 32 op/s 2024-04-07T17:48:10.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:48:09 smithi100 bash[76202]: cluster 2024-04-07T17:48:08.745364+0000 mgr.y (mgr.25175) 2783 : cluster [DBG] pgmap v4965: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 52 KiB/s rd, 1.8 KiB/s wr, 32 op/s 2024-04-07T17:48:12.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:48:11 smithi144 bash[66654]: cluster 2024-04-07T17:48:10.746197+0000 mgr.y (mgr.25175) 2784 : cluster [DBG] pgmap v4966: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 44 KiB/s rd, 1.6 KiB/s wr, 28 op/s 2024-04-07T17:48:12.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:48:11 smithi100 bash[74942]: cluster 2024-04-07T17:48:10.746197+0000 mgr.y (mgr.25175) 2784 : cluster [DBG] pgmap v4966: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 44 KiB/s rd, 1.6 KiB/s wr, 28 op/s 2024-04-07T17:48:12.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:48:11 smithi100 bash[76202]: cluster 2024-04-07T17:48:10.746197+0000 mgr.y (mgr.25175) 2784 : cluster [DBG] pgmap v4966: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 44 KiB/s rd, 1.6 KiB/s wr, 28 op/s 2024-04-07T17:48:12.614 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:48:12 smithi144 bash[95233]: ts=2024-04-07T17:48:12.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:48:14.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:48:13 smithi144 bash[66654]: cluster 2024-04-07T17:48:12.747408+0000 mgr.y (mgr.25175) 2785 : cluster [DBG] pgmap v4967: 137 pgs: 137 active+clean; 13 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 44 KiB/s rd, 685 KiB/s wr, 28 op/s 2024-04-07T17:48:14.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:48:13 smithi100 bash[74942]: cluster 2024-04-07T17:48:12.747408+0000 mgr.y (mgr.25175) 2785 : cluster [DBG] pgmap v4967: 137 pgs: 137 active+clean; 13 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 44 KiB/s rd, 685 KiB/s wr, 28 op/s 2024-04-07T17:48:14.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:48:13 smithi100 bash[76202]: cluster 2024-04-07T17:48:12.747408+0000 mgr.y (mgr.25175) 2785 : cluster [DBG] pgmap v4967: 137 pgs: 137 active+clean; 13 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 44 KiB/s rd, 685 KiB/s wr, 28 op/s 2024-04-07T17:48:15.024 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_encryption_luks2 PASSED [ 57%] 2024-04-07T17:48:15.263 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImageId::test_read PASSED [ 57%] 2024-04-07T17:48:15.534 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImageId::test_write PASSED [ 58%] 2024-04-07T17:48:15.793 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImageId::test_resize PASSED [ 58%] 2024-04-07T17:48:16.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:48:15 smithi144 bash[66654]: cluster 2024-04-07T17:48:14.748559+0000 mgr.y (mgr.25175) 2786 : cluster [DBG] pgmap v4968: 137 pgs: 137 active+clean; 21 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 59 KiB/s rd, 1.3 MiB/s wr, 15 op/s 2024-04-07T17:48:16.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:48:15 smithi100 bash[74942]: cluster 2024-04-07T17:48:14.748559+0000 mgr.y (mgr.25175) 2786 : cluster [DBG] pgmap v4968: 137 pgs: 137 active+clean; 21 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 59 KiB/s rd, 1.3 MiB/s wr, 15 op/s 2024-04-07T17:48:16.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:48:15 smithi100 bash[76202]: cluster 2024-04-07T17:48:14.748559+0000 mgr.y (mgr.25175) 2786 : cluster [DBG] pgmap v4968: 137 pgs: 137 active+clean; 21 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 59 KiB/s rd, 1.3 MiB/s wr, 15 op/s 2024-04-07T17:48:18.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:48:17 smithi144 bash[66654]: cluster 2024-04-07T17:48:16.749442+0000 mgr.y (mgr.25175) 2787 : cluster [DBG] pgmap v4969: 137 pgs: 137 active+clean; 21 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 52 KiB/s rd, 1.3 MiB/s wr, 5 op/s 2024-04-07T17:48:18.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:48:17 smithi144 bash[66654]: cluster 2024-04-07T17:48:16.855393+0000 mon.a (mon.0) 4618 : cluster [DBG] osdmap e2608: 8 total, 8 up, 8 in 2024-04-07T17:48:18.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:48:17 smithi144 bash[66654]: audit 2024-04-07T17:48:16.883024+0000 mon.a (mon.0) 4619 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:48:18.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:48:17 smithi100 bash[74942]: cluster 2024-04-07T17:48:16.749442+0000 mgr.y (mgr.25175) 2787 : cluster [DBG] pgmap v4969: 137 pgs: 137 active+clean; 21 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 52 KiB/s rd, 1.3 MiB/s wr, 5 op/s 2024-04-07T17:48:18.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:48:17 smithi100 bash[74942]: cluster 2024-04-07T17:48:16.855393+0000 mon.a (mon.0) 4618 : cluster [DBG] osdmap e2608: 8 total, 8 up, 8 in 2024-04-07T17:48:18.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:48:17 smithi100 bash[74942]: audit 2024-04-07T17:48:16.883024+0000 mon.a (mon.0) 4619 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:48:18.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:48:17 smithi100 bash[76202]: cluster 2024-04-07T17:48:16.749442+0000 mgr.y (mgr.25175) 2787 : cluster [DBG] pgmap v4969: 137 pgs: 137 active+clean; 21 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 52 KiB/s rd, 1.3 MiB/s wr, 5 op/s 2024-04-07T17:48:18.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:48:17 smithi100 bash[76202]: cluster 2024-04-07T17:48:16.855393+0000 mon.a (mon.0) 4618 : cluster [DBG] osdmap e2608: 8 total, 8 up, 8 in 2024-04-07T17:48:18.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:48:17 smithi100 bash[76202]: audit 2024-04-07T17:48:16.883024+0000 mon.a (mon.0) 4619 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:48:18.703 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:48:18 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:48:18] "GET /metrics HTTP/1.1" 200 36762 "" "Prometheus/2.43.0" 2024-04-07T17:48:18.864 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 17:48:18 smithi144 bash[98866]: logger=cleanup t=2024-04-07T17:48:18.602160473Z level=info msg="Completed cleanup jobs" duration=182.9539ms 2024-04-07T17:48:19.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:48:18 smithi100 bash[74942]: cluster 2024-04-07T17:48:17.865176+0000 mon.a (mon.0) 4620 : cluster [DBG] osdmap e2609: 8 total, 8 up, 8 in 2024-04-07T17:48:19.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:48:18 smithi100 bash[76202]: cluster 2024-04-07T17:48:17.865176+0000 mon.a (mon.0) 4620 : cluster [DBG] osdmap e2609: 8 total, 8 up, 8 in 2024-04-07T17:48:19.364 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:48:18 smithi144 bash[66654]: cluster 2024-04-07T17:48:17.865176+0000 mon.a (mon.0) 4620 : cluster [DBG] osdmap e2609: 8 total, 8 up, 8 in 2024-04-07T17:48:20.133 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestClone::test_with_params PASSED [ 59%] 2024-04-07T17:48:20.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:48:19 smithi100 bash[74942]: cluster 2024-04-07T17:48:18.750311+0000 mgr.y (mgr.25175) 2788 : cluster [DBG] pgmap v4972: 137 pgs: 137 active+clean; 13 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 167 KiB/s rd, 2.0 MiB/s wr, 128 op/s 2024-04-07T17:48:20.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:48:19 smithi100 bash[74942]: cluster 2024-04-07T17:48:18.874757+0000 mon.a (mon.0) 4621 : cluster [DBG] osdmap e2610: 8 total, 8 up, 8 in 2024-04-07T17:48:20.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:48:19 smithi100 bash[76202]: cluster 2024-04-07T17:48:18.750311+0000 mgr.y (mgr.25175) 2788 : cluster [DBG] pgmap v4972: 137 pgs: 137 active+clean; 13 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 167 KiB/s rd, 2.0 MiB/s wr, 128 op/s 2024-04-07T17:48:20.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:48:19 smithi100 bash[76202]: cluster 2024-04-07T17:48:18.874757+0000 mon.a (mon.0) 4621 : cluster [DBG] osdmap e2610: 8 total, 8 up, 8 in 2024-04-07T17:48:20.364 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:48:19 smithi144 bash[66654]: cluster 2024-04-07T17:48:18.750311+0000 mgr.y (mgr.25175) 2788 : cluster [DBG] pgmap v4972: 137 pgs: 137 active+clean; 13 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 167 KiB/s rd, 2.0 MiB/s wr, 128 op/s 2024-04-07T17:48:20.364 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:48:19 smithi144 bash[66654]: cluster 2024-04-07T17:48:18.874757+0000 mon.a (mon.0) 4621 : cluster [DBG] osdmap e2610: 8 total, 8 up, 8 in 2024-04-07T17:48:21.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:48:20 smithi100 bash[74942]: cluster 2024-04-07T17:48:19.886534+0000 mon.a (mon.0) 4622 : cluster [DBG] osdmap e2611: 8 total, 8 up, 8 in 2024-04-07T17:48:21.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:48:20 smithi100 bash[76202]: cluster 2024-04-07T17:48:19.886534+0000 mon.a (mon.0) 4622 : cluster [DBG] osdmap e2611: 8 total, 8 up, 8 in 2024-04-07T17:48:21.364 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:48:20 smithi144 bash[66654]: cluster 2024-04-07T17:48:19.886534+0000 mon.a (mon.0) 4622 : cluster [DBG] osdmap e2611: 8 total, 8 up, 8 in 2024-04-07T17:48:22.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:48:21 smithi100 bash[74942]: cluster 2024-04-07T17:48:20.751211+0000 mgr.y (mgr.25175) 2789 : cluster [DBG] pgmap v4975: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 242 KiB/s rd, 17 KiB/s wr, 327 op/s 2024-04-07T17:48:22.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:48:21 smithi100 bash[74942]: cluster 2024-04-07T17:48:20.894260+0000 mon.a (mon.0) 4623 : cluster [DBG] osdmap e2612: 8 total, 8 up, 8 in 2024-04-07T17:48:22.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:48:21 smithi100 bash[76202]: cluster 2024-04-07T17:48:20.751211+0000 mgr.y (mgr.25175) 2789 : cluster [DBG] pgmap v4975: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 242 KiB/s rd, 17 KiB/s wr, 327 op/s 2024-04-07T17:48:22.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:48:21 smithi100 bash[76202]: cluster 2024-04-07T17:48:20.894260+0000 mon.a (mon.0) 4623 : cluster [DBG] osdmap e2612: 8 total, 8 up, 8 in 2024-04-07T17:48:22.315 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:48:21 smithi144 bash[66654]: cluster 2024-04-07T17:48:20.751211+0000 mgr.y (mgr.25175) 2789 : cluster [DBG] pgmap v4975: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 242 KiB/s rd, 17 KiB/s wr, 327 op/s 2024-04-07T17:48:22.315 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:48:21 smithi144 bash[66654]: cluster 2024-04-07T17:48:20.894260+0000 mon.a (mon.0) 4623 : cluster [DBG] osdmap e2612: 8 total, 8 up, 8 in 2024-04-07T17:48:22.615 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:48:22 smithi144 bash[95233]: ts=2024-04-07T17:48:22.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:48:23.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:48:22 smithi100 bash[74942]: cluster 2024-04-07T17:48:21.905974+0000 mon.a (mon.0) 4624 : cluster [DBG] osdmap e2613: 8 total, 8 up, 8 in 2024-04-07T17:48:23.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:48:22 smithi100 bash[76202]: cluster 2024-04-07T17:48:21.905974+0000 mon.a (mon.0) 4624 : cluster [DBG] osdmap e2613: 8 total, 8 up, 8 in 2024-04-07T17:48:23.364 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:48:22 smithi144 bash[66654]: cluster 2024-04-07T17:48:21.905974+0000 mon.a (mon.0) 4624 : cluster [DBG] osdmap e2613: 8 total, 8 up, 8 in 2024-04-07T17:48:24.156 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestClone::test_with_params2 PASSED [ 60%] 2024-04-07T17:48:24.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:48:23 smithi100 bash[76202]: cluster 2024-04-07T17:48:22.752482+0000 mgr.y (mgr.25175) 2790 : cluster [DBG] pgmap v4978: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 214 KiB/s rd, 16 KiB/s wr, 291 op/s 2024-04-07T17:48:24.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:48:23 smithi100 bash[76202]: cluster 2024-04-07T17:48:22.922868+0000 mon.a (mon.0) 4625 : cluster [DBG] osdmap e2614: 8 total, 8 up, 8 in 2024-04-07T17:48:24.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:48:23 smithi100 bash[74942]: cluster 2024-04-07T17:48:22.752482+0000 mgr.y (mgr.25175) 2790 : cluster [DBG] pgmap v4978: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 214 KiB/s rd, 16 KiB/s wr, 291 op/s 2024-04-07T17:48:24.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:48:23 smithi100 bash[74942]: cluster 2024-04-07T17:48:22.922868+0000 mon.a (mon.0) 4625 : cluster [DBG] osdmap e2614: 8 total, 8 up, 8 in 2024-04-07T17:48:24.364 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:48:23 smithi144 bash[66654]: cluster 2024-04-07T17:48:22.752482+0000 mgr.y (mgr.25175) 2790 : cluster [DBG] pgmap v4978: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 214 KiB/s rd, 16 KiB/s wr, 291 op/s 2024-04-07T17:48:24.364 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:48:23 smithi144 bash[66654]: cluster 2024-04-07T17:48:22.922868+0000 mon.a (mon.0) 4625 : cluster [DBG] osdmap e2614: 8 total, 8 up, 8 in 2024-04-07T17:48:25.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:48:24 smithi100 bash[74942]: cluster 2024-04-07T17:48:23.926296+0000 mon.a (mon.0) 4626 : cluster [DBG] osdmap e2615: 8 total, 8 up, 8 in 2024-04-07T17:48:25.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:48:24 smithi100 bash[76202]: cluster 2024-04-07T17:48:23.926296+0000 mon.a (mon.0) 4626 : cluster [DBG] osdmap e2615: 8 total, 8 up, 8 in 2024-04-07T17:48:25.364 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:48:24 smithi144 bash[66654]: cluster 2024-04-07T17:48:23.926296+0000 mon.a (mon.0) 4626 : cluster [DBG] osdmap e2615: 8 total, 8 up, 8 in 2024-04-07T17:48:26.198 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestClone::test_with_params3 SKIPPED [ 60%] 2024-04-07T17:48:26.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:48:25 smithi100 bash[74942]: cluster 2024-04-07T17:48:24.753541+0000 mgr.y (mgr.25175) 2791 : cluster [DBG] pgmap v4981: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 215 KiB/s rd, 15 KiB/s wr, 295 op/s 2024-04-07T17:48:26.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:48:25 smithi100 bash[74942]: cluster 2024-04-07T17:48:24.941823+0000 mon.a (mon.0) 4627 : cluster [DBG] osdmap e2616: 8 total, 8 up, 8 in 2024-04-07T17:48:26.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:48:25 smithi100 bash[76202]: cluster 2024-04-07T17:48:24.753541+0000 mgr.y (mgr.25175) 2791 : cluster [DBG] pgmap v4981: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 215 KiB/s rd, 15 KiB/s wr, 295 op/s 2024-04-07T17:48:26.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:48:25 smithi100 bash[76202]: cluster 2024-04-07T17:48:24.941823+0000 mon.a (mon.0) 4627 : cluster [DBG] osdmap e2616: 8 total, 8 up, 8 in 2024-04-07T17:48:26.364 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:48:25 smithi144 bash[66654]: cluster 2024-04-07T17:48:24.753541+0000 mgr.y (mgr.25175) 2791 : cluster [DBG] pgmap v4981: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 215 KiB/s rd, 15 KiB/s wr, 295 op/s 2024-04-07T17:48:26.364 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:48:25 smithi144 bash[66654]: cluster 2024-04-07T17:48:24.941823+0000 mon.a (mon.0) 4627 : cluster [DBG] osdmap e2616: 8 total, 8 up, 8 in 2024-04-07T17:48:27.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:48:26 smithi100 bash[74942]: cluster 2024-04-07T17:48:25.946549+0000 mon.a (mon.0) 4628 : cluster [DBG] osdmap e2617: 8 total, 8 up, 8 in 2024-04-07T17:48:27.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:48:26 smithi100 bash[76202]: cluster 2024-04-07T17:48:25.946549+0000 mon.a (mon.0) 4628 : cluster [DBG] osdmap e2617: 8 total, 8 up, 8 in 2024-04-07T17:48:27.364 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:48:26 smithi144 bash[66654]: cluster 2024-04-07T17:48:25.946549+0000 mon.a (mon.0) 4628 : cluster [DBG] osdmap e2617: 8 total, 8 up, 8 in 2024-04-07T17:48:28.364 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:48:27 smithi144 bash[66654]: cluster 2024-04-07T17:48:26.754408+0000 mgr.y (mgr.25175) 2792 : cluster [DBG] pgmap v4984: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 65 KiB/s rd, 5.2 KiB/s wr, 91 op/s 2024-04-07T17:48:28.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:48:27 smithi144 bash[66654]: cluster 2024-04-07T17:48:26.956876+0000 mon.a (mon.0) 4629 : cluster [DBG] osdmap e2618: 8 total, 8 up, 8 in 2024-04-07T17:48:28.425 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:48:27 smithi100 bash[74942]: cluster 2024-04-07T17:48:26.754408+0000 mgr.y (mgr.25175) 2792 : cluster [DBG] pgmap v4984: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 65 KiB/s rd, 5.2 KiB/s wr, 91 op/s 2024-04-07T17:48:28.425 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:48:27 smithi100 bash[74942]: cluster 2024-04-07T17:48:26.956876+0000 mon.a (mon.0) 4629 : cluster [DBG] osdmap e2618: 8 total, 8 up, 8 in 2024-04-07T17:48:28.426 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:48:27 smithi100 bash[76202]: cluster 2024-04-07T17:48:26.754408+0000 mgr.y (mgr.25175) 2792 : cluster [DBG] pgmap v4984: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 65 KiB/s rd, 5.2 KiB/s wr, 91 op/s 2024-04-07T17:48:28.426 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:48:27 smithi100 bash[76202]: cluster 2024-04-07T17:48:26.956876+0000 mon.a (mon.0) 4629 : cluster [DBG] osdmap e2618: 8 total, 8 up, 8 in 2024-04-07T17:48:28.703 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:48:28 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:48:28] "GET /metrics HTTP/1.1" 200 36762 "" "Prometheus/2.43.0" 2024-04-07T17:48:29.364 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:48:28 smithi144 bash[66654]: cluster 2024-04-07T17:48:27.956591+0000 mon.a (mon.0) 4630 : cluster [DBG] osdmap e2619: 8 total, 8 up, 8 in 2024-04-07T17:48:29.364 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:48:28 smithi144 bash[66654]: cluster 2024-04-07T17:48:28.486032+0000 mon.a (mon.0) 4631 : cluster [DBG] osdmap e2620: 8 total, 8 up, 8 in 2024-04-07T17:48:29.453 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:48:28 smithi100 bash[74942]: cluster 2024-04-07T17:48:27.956591+0000 mon.a (mon.0) 4630 : cluster [DBG] osdmap e2619: 8 total, 8 up, 8 in 2024-04-07T17:48:29.453 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:48:28 smithi100 bash[74942]: cluster 2024-04-07T17:48:28.486032+0000 mon.a (mon.0) 4631 : cluster [DBG] osdmap e2620: 8 total, 8 up, 8 in 2024-04-07T17:48:29.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:48:28 smithi100 bash[76202]: cluster 2024-04-07T17:48:27.956591+0000 mon.a (mon.0) 4630 : cluster [DBG] osdmap e2619: 8 total, 8 up, 8 in 2024-04-07T17:48:29.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:48:28 smithi100 bash[76202]: cluster 2024-04-07T17:48:28.486032+0000 mon.a (mon.0) 4631 : cluster [DBG] osdmap e2620: 8 total, 8 up, 8 in 2024-04-07T17:48:29.754 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestClone::test_stripe_unit_and_count PASSED [ 61%] 2024-04-07T17:48:30.364 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:48:29 smithi144 bash[66654]: cluster 2024-04-07T17:48:28.755211+0000 mgr.y (mgr.25175) 2793 : cluster [DBG] pgmap v4988: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 151 KiB/s rd, 12 KiB/s wr, 206 op/s 2024-04-07T17:48:30.364 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:48:29 smithi144 bash[66654]: cluster 2024-04-07T17:48:29.494061+0000 mon.a (mon.0) 4632 : cluster [DBG] osdmap e2621: 8 total, 8 up, 8 in 2024-04-07T17:48:30.453 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:48:29 smithi100 bash[74942]: cluster 2024-04-07T17:48:28.755211+0000 mgr.y (mgr.25175) 2793 : cluster [DBG] pgmap v4988: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 151 KiB/s rd, 12 KiB/s wr, 206 op/s 2024-04-07T17:48:30.453 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:48:29 smithi100 bash[74942]: cluster 2024-04-07T17:48:29.494061+0000 mon.a (mon.0) 4632 : cluster [DBG] osdmap e2621: 8 total, 8 up, 8 in 2024-04-07T17:48:30.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:48:29 smithi100 bash[76202]: cluster 2024-04-07T17:48:28.755211+0000 mgr.y (mgr.25175) 2793 : cluster [DBG] pgmap v4988: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 151 KiB/s rd, 12 KiB/s wr, 206 op/s 2024-04-07T17:48:30.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:48:29 smithi100 bash[76202]: cluster 2024-04-07T17:48:29.494061+0000 mon.a (mon.0) 4632 : cluster [DBG] osdmap e2621: 8 total, 8 up, 8 in 2024-04-07T17:48:31.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:48:31 smithi144 bash[66654]: cluster 2024-04-07T17:48:30.496425+0000 mon.a (mon.0) 4633 : cluster [DBG] osdmap e2622: 8 total, 8 up, 8 in 2024-04-07T17:48:31.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:48:31 smithi144 bash[66654]: cluster 2024-04-07T17:48:30.756002+0000 mgr.y (mgr.25175) 2794 : cluster [DBG] pgmap v4991: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 391 KiB/s rd, 26 KiB/s wr, 525 op/s 2024-04-07T17:48:31.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:48:31 smithi100 bash[74942]: cluster 2024-04-07T17:48:30.496425+0000 mon.a (mon.0) 4633 : cluster [DBG] osdmap e2622: 8 total, 8 up, 8 in 2024-04-07T17:48:31.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:48:31 smithi100 bash[74942]: cluster 2024-04-07T17:48:30.756002+0000 mgr.y (mgr.25175) 2794 : cluster [DBG] pgmap v4991: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 391 KiB/s rd, 26 KiB/s wr, 525 op/s 2024-04-07T17:48:31.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:48:31 smithi100 bash[76202]: cluster 2024-04-07T17:48:30.496425+0000 mon.a (mon.0) 4633 : cluster [DBG] osdmap e2622: 8 total, 8 up, 8 in 2024-04-07T17:48:31.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:48:31 smithi100 bash[76202]: cluster 2024-04-07T17:48:30.756002+0000 mgr.y (mgr.25175) 2794 : cluster [DBG] pgmap v4991: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 391 KiB/s rd, 26 KiB/s wr, 525 op/s 2024-04-07T17:48:32.614 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:48:32 smithi144 bash[66654]: cluster 2024-04-07T17:48:31.506676+0000 mon.a (mon.0) 4634 : cluster [DBG] osdmap e2623: 8 total, 8 up, 8 in 2024-04-07T17:48:32.614 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:48:32 smithi144 bash[66654]: audit 2024-04-07T17:48:31.882592+0000 mon.a (mon.0) 4635 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:48:32.615 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:48:32 smithi144 bash[95233]: ts=2024-04-07T17:48:32.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:48:32.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:48:32 smithi100 bash[74942]: cluster 2024-04-07T17:48:31.506676+0000 mon.a (mon.0) 4634 : cluster [DBG] osdmap e2623: 8 total, 8 up, 8 in 2024-04-07T17:48:32.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:48:32 smithi100 bash[74942]: audit 2024-04-07T17:48:31.882592+0000 mon.a (mon.0) 4635 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:48:32.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:48:32 smithi100 bash[76202]: cluster 2024-04-07T17:48:31.506676+0000 mon.a (mon.0) 4634 : cluster [DBG] osdmap e2623: 8 total, 8 up, 8 in 2024-04-07T17:48:32.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:48:32 smithi100 bash[76202]: audit 2024-04-07T17:48:31.882592+0000 mon.a (mon.0) 4635 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:48:33.718 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestClone::test_unprotected PASSED [ 61%] 2024-04-07T17:48:33.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:48:33 smithi144 bash[66654]: cluster 2024-04-07T17:48:32.517214+0000 mon.a (mon.0) 4636 : cluster [DBG] osdmap e2624: 8 total, 8 up, 8 in 2024-04-07T17:48:33.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:48:33 smithi144 bash[66654]: cluster 2024-04-07T17:48:32.756695+0000 mgr.y (mgr.25175) 2795 : cluster [DBG] pgmap v4994: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 283 KiB/s rd, 18 KiB/s wr, 384 op/s 2024-04-07T17:48:33.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:48:33 smithi144 bash[66654]: cluster 2024-04-07T17:48:33.488236+0000 mon.a (mon.0) 4637 : cluster [DBG] osdmap e2625: 8 total, 8 up, 8 in 2024-04-07T17:48:33.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:48:33 smithi100 bash[74942]: cluster 2024-04-07T17:48:32.517214+0000 mon.a (mon.0) 4636 : cluster [DBG] osdmap e2624: 8 total, 8 up, 8 in 2024-04-07T17:48:33.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:48:33 smithi100 bash[74942]: cluster 2024-04-07T17:48:32.756695+0000 mgr.y (mgr.25175) 2795 : cluster [DBG] pgmap v4994: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 283 KiB/s rd, 18 KiB/s wr, 384 op/s 2024-04-07T17:48:33.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:48:33 smithi100 bash[74942]: cluster 2024-04-07T17:48:33.488236+0000 mon.a (mon.0) 4637 : cluster [DBG] osdmap e2625: 8 total, 8 up, 8 in 2024-04-07T17:48:33.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:48:33 smithi100 bash[76202]: cluster 2024-04-07T17:48:32.517214+0000 mon.a (mon.0) 4636 : cluster [DBG] osdmap e2624: 8 total, 8 up, 8 in 2024-04-07T17:48:33.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:48:33 smithi100 bash[76202]: cluster 2024-04-07T17:48:32.756695+0000 mgr.y (mgr.25175) 2795 : cluster [DBG] pgmap v4994: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 283 KiB/s rd, 18 KiB/s wr, 384 op/s 2024-04-07T17:48:33.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:48:33 smithi100 bash[76202]: cluster 2024-04-07T17:48:33.488236+0000 mon.a (mon.0) 4637 : cluster [DBG] osdmap e2625: 8 total, 8 up, 8 in 2024-04-07T17:48:35.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:48:35 smithi144 bash[66654]: cluster 2024-04-07T17:48:34.535265+0000 mon.a (mon.0) 4638 : cluster [DBG] osdmap e2626: 8 total, 8 up, 8 in 2024-04-07T17:48:35.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:48:35 smithi144 bash[66654]: cluster 2024-04-07T17:48:34.757217+0000 mgr.y (mgr.25175) 2796 : cluster [DBG] pgmap v4997: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 132 KiB/s rd, 11 KiB/s wr, 187 op/s 2024-04-07T17:48:35.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:48:35 smithi100 bash[74942]: cluster 2024-04-07T17:48:34.535265+0000 mon.a (mon.0) 4638 : cluster [DBG] osdmap e2626: 8 total, 8 up, 8 in 2024-04-07T17:48:35.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:48:35 smithi100 bash[74942]: cluster 2024-04-07T17:48:34.757217+0000 mgr.y (mgr.25175) 2796 : cluster [DBG] pgmap v4997: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 132 KiB/s rd, 11 KiB/s wr, 187 op/s 2024-04-07T17:48:35.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:48:35 smithi100 bash[76202]: cluster 2024-04-07T17:48:34.535265+0000 mon.a (mon.0) 4638 : cluster [DBG] osdmap e2626: 8 total, 8 up, 8 in 2024-04-07T17:48:35.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:48:35 smithi100 bash[76202]: cluster 2024-04-07T17:48:34.757217+0000 mgr.y (mgr.25175) 2796 : cluster [DBG] pgmap v4997: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 132 KiB/s rd, 11 KiB/s wr, 187 op/s 2024-04-07T17:48:36.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:48:36 smithi144 bash[66654]: cluster 2024-04-07T17:48:35.550862+0000 mon.a (mon.0) 4639 : cluster [DBG] osdmap e2627: 8 total, 8 up, 8 in 2024-04-07T17:48:36.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:48:36 smithi144 bash[66654]: audit 2024-04-07T17:48:35.554547+0000 mon.c (mon.2) 104 : audit [INF] from='client.? 172.21.15.100:0/60836617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rbd-api-smithi100-170558-2","app": "rbd"}]: dispatch 2024-04-07T17:48:36.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:48:36 smithi144 bash[66654]: audit 2024-04-07T17:48:35.564248+0000 mon.a (mon.0) 4640 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rbd-api-smithi100-170558-2","app": "rbd"}]: dispatch 2024-04-07T17:48:36.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:48:36 smithi100 bash[74942]: cluster 2024-04-07T17:48:35.550862+0000 mon.a (mon.0) 4639 : cluster [DBG] osdmap e2627: 8 total, 8 up, 8 in 2024-04-07T17:48:36.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:48:36 smithi100 bash[74942]: audit 2024-04-07T17:48:35.554547+0000 mon.c (mon.2) 104 : audit [INF] from='client.? 172.21.15.100:0/60836617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rbd-api-smithi100-170558-2","app": "rbd"}]: dispatch 2024-04-07T17:48:36.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:48:36 smithi100 bash[74942]: audit 2024-04-07T17:48:35.564248+0000 mon.a (mon.0) 4640 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rbd-api-smithi100-170558-2","app": "rbd"}]: dispatch 2024-04-07T17:48:36.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:48:36 smithi100 bash[76202]: cluster 2024-04-07T17:48:35.550862+0000 mon.a (mon.0) 4639 : cluster [DBG] osdmap e2627: 8 total, 8 up, 8 in 2024-04-07T17:48:36.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:48:36 smithi100 bash[76202]: audit 2024-04-07T17:48:35.554547+0000 mon.c (mon.2) 104 : audit [INF] from='client.? 172.21.15.100:0/60836617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rbd-api-smithi100-170558-2","app": "rbd"}]: dispatch 2024-04-07T17:48:36.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:48:36 smithi100 bash[76202]: audit 2024-04-07T17:48:35.564248+0000 mon.a (mon.0) 4640 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rbd-api-smithi100-170558-2","app": "rbd"}]: dispatch 2024-04-07T17:48:37.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:48:37 smithi144 bash[66654]: audit 2024-04-07T17:48:36.542780+0000 mon.a (mon.0) 4641 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rbd-api-smithi100-170558-2","app": "rbd"}]': finished 2024-04-07T17:48:37.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:48:37 smithi144 bash[66654]: cluster 2024-04-07T17:48:36.553550+0000 mon.a (mon.0) 4642 : cluster [DBG] osdmap e2628: 8 total, 8 up, 8 in 2024-04-07T17:48:37.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:48:37 smithi144 bash[66654]: cluster 2024-04-07T17:48:36.757923+0000 mgr.y (mgr.25175) 2797 : cluster [DBG] pgmap v5000: 169 pgs: 32 unknown, 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 157 KiB/s rd, 11 KiB/s wr, 208 op/s 2024-04-07T17:48:37.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:48:37 smithi100 bash[74942]: audit 2024-04-07T17:48:36.542780+0000 mon.a (mon.0) 4641 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rbd-api-smithi100-170558-2","app": "rbd"}]': finished 2024-04-07T17:48:37.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:48:37 smithi100 bash[74942]: cluster 2024-04-07T17:48:36.553550+0000 mon.a (mon.0) 4642 : cluster [DBG] osdmap e2628: 8 total, 8 up, 8 in 2024-04-07T17:48:37.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:48:37 smithi100 bash[74942]: cluster 2024-04-07T17:48:36.757923+0000 mgr.y (mgr.25175) 2797 : cluster [DBG] pgmap v5000: 169 pgs: 32 unknown, 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 157 KiB/s rd, 11 KiB/s wr, 208 op/s 2024-04-07T17:48:37.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:48:37 smithi100 bash[76202]: audit 2024-04-07T17:48:36.542780+0000 mon.a (mon.0) 4641 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rbd-api-smithi100-170558-2","app": "rbd"}]': finished 2024-04-07T17:48:37.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:48:37 smithi100 bash[76202]: cluster 2024-04-07T17:48:36.553550+0000 mon.a (mon.0) 4642 : cluster [DBG] osdmap e2628: 8 total, 8 up, 8 in 2024-04-07T17:48:37.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:48:37 smithi100 bash[76202]: cluster 2024-04-07T17:48:36.757923+0000 mgr.y (mgr.25175) 2797 : cluster [DBG] pgmap v5000: 169 pgs: 32 unknown, 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 157 KiB/s rd, 11 KiB/s wr, 208 op/s 2024-04-07T17:48:38.703 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:48:38 smithi100 bash[74942]: cluster 2024-04-07T17:48:37.590116+0000 mon.a (mon.0) 4643 : cluster [DBG] osdmap e2629: 8 total, 8 up, 8 in 2024-04-07T17:48:38.703 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:48:38 smithi100 bash[74942]: cluster 2024-04-07T17:48:38.490504+0000 mon.a (mon.0) 4644 : cluster [DBG] osdmap e2630: 8 total, 8 up, 8 in 2024-04-07T17:48:38.704 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:48:38 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:48:38] "GET /metrics HTTP/1.1" 200 36741 "" "Prometheus/2.43.0" 2024-04-07T17:48:38.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:48:38 smithi100 bash[76202]: cluster 2024-04-07T17:48:37.590116+0000 mon.a (mon.0) 4643 : cluster [DBG] osdmap e2629: 8 total, 8 up, 8 in 2024-04-07T17:48:38.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:48:38 smithi100 bash[76202]: cluster 2024-04-07T17:48:38.490504+0000 mon.a (mon.0) 4644 : cluster [DBG] osdmap e2630: 8 total, 8 up, 8 in 2024-04-07T17:48:38.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:48:38 smithi144 bash[66654]: cluster 2024-04-07T17:48:37.590116+0000 mon.a (mon.0) 4643 : cluster [DBG] osdmap e2629: 8 total, 8 up, 8 in 2024-04-07T17:48:38.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:48:38 smithi144 bash[66654]: cluster 2024-04-07T17:48:38.490504+0000 mon.a (mon.0) 4644 : cluster [DBG] osdmap e2630: 8 total, 8 up, 8 in 2024-04-07T17:48:39.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:48:39 smithi144 bash[66654]: cluster 2024-04-07T17:48:38.758704+0000 mgr.y (mgr.25175) 2798 : cluster [DBG] pgmap v5003: 169 pgs: 11 unknown, 158 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 137 KiB/s rd, 8.0 KiB/s wr, 176 op/s 2024-04-07T17:48:39.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:48:39 smithi100 bash[74942]: cluster 2024-04-07T17:48:38.758704+0000 mgr.y (mgr.25175) 2798 : cluster [DBG] pgmap v5003: 169 pgs: 11 unknown, 158 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 137 KiB/s rd, 8.0 KiB/s wr, 176 op/s 2024-04-07T17:48:39.953 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:48:39 smithi100 bash[76202]: cluster 2024-04-07T17:48:38.758704+0000 mgr.y (mgr.25175) 2798 : cluster [DBG] pgmap v5003: 169 pgs: 11 unknown, 158 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 137 KiB/s rd, 8.0 KiB/s wr, 176 op/s 2024-04-07T17:48:40.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:48:40 smithi144 bash[66654]: cluster 2024-04-07T17:48:39.600750+0000 mon.a (mon.0) 4645 : cluster [DBG] osdmap e2631: 8 total, 8 up, 8 in 2024-04-07T17:48:40.873 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestClone::test_unprotect_with_children PASSED [ 62%] 2024-04-07T17:48:40.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:48:40 smithi100 bash[74942]: cluster 2024-04-07T17:48:39.600750+0000 mon.a (mon.0) 4645 : cluster [DBG] osdmap e2631: 8 total, 8 up, 8 in 2024-04-07T17:48:40.953 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:48:40 smithi100 bash[76202]: cluster 2024-04-07T17:48:39.600750+0000 mon.a (mon.0) 4645 : cluster [DBG] osdmap e2631: 8 total, 8 up, 8 in 2024-04-07T17:48:41.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:48:41 smithi100 bash[74942]: cluster 2024-04-07T17:48:40.609985+0000 mon.a (mon.0) 4646 : cluster [DBG] osdmap e2632: 8 total, 8 up, 8 in 2024-04-07T17:48:41.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:48:41 smithi100 bash[74942]: cluster 2024-04-07T17:48:40.759455+0000 mgr.y (mgr.25175) 2799 : cluster [DBG] pgmap v5006: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 67 KiB/s rd, 4.5 KiB/s wr, 88 op/s 2024-04-07T17:48:41.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:48:41 smithi100 bash[74942]: audit 2024-04-07T17:48:40.925952+0000 mon.a (mon.0) 4647 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:48:41.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:48:41 smithi100 bash[76202]: cluster 2024-04-07T17:48:40.609985+0000 mon.a (mon.0) 4646 : cluster [DBG] osdmap e2632: 8 total, 8 up, 8 in 2024-04-07T17:48:41.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:48:41 smithi100 bash[76202]: cluster 2024-04-07T17:48:40.759455+0000 mgr.y (mgr.25175) 2799 : cluster [DBG] pgmap v5006: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 67 KiB/s rd, 4.5 KiB/s wr, 88 op/s 2024-04-07T17:48:41.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:48:41 smithi100 bash[76202]: audit 2024-04-07T17:48:40.925952+0000 mon.a (mon.0) 4647 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:48:42.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:48:41 smithi144 bash[66654]: cluster 2024-04-07T17:48:40.609985+0000 mon.a (mon.0) 4646 : cluster [DBG] osdmap e2632: 8 total, 8 up, 8 in 2024-04-07T17:48:42.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:48:41 smithi144 bash[66654]: cluster 2024-04-07T17:48:40.759455+0000 mgr.y (mgr.25175) 2799 : cluster [DBG] pgmap v5006: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 67 KiB/s rd, 4.5 KiB/s wr, 88 op/s 2024-04-07T17:48:42.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:48:41 smithi144 bash[66654]: audit 2024-04-07T17:48:40.925952+0000 mon.a (mon.0) 4647 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:48:42.614 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:48:42 smithi144 bash[95233]: ts=2024-04-07T17:48:42.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:48:42.848 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestClone::test_stat PASSED [ 62%] 2024-04-07T17:48:42.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:48:42 smithi100 bash[74942]: audit 2024-04-07T17:48:41.604004+0000 mon.a (mon.0) 4648 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:48:42.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:48:42 smithi100 bash[74942]: audit 2024-04-07T17:48:41.607956+0000 mon.a (mon.0) 4649 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:48:42.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:48:42 smithi100 bash[74942]: cluster 2024-04-07T17:48:41.615153+0000 mon.a (mon.0) 4650 : cluster [DBG] osdmap e2633: 8 total, 8 up, 8 in 2024-04-07T17:48:42.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:48:42 smithi100 bash[74942]: audit 2024-04-07T17:48:41.626019+0000 mon.a (mon.0) 4651 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:48:42.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:48:42 smithi100 bash[76202]: audit 2024-04-07T17:48:41.604004+0000 mon.a (mon.0) 4648 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:48:42.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:48:42 smithi100 bash[76202]: audit 2024-04-07T17:48:41.607956+0000 mon.a (mon.0) 4649 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:48:42.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:48:42 smithi100 bash[76202]: cluster 2024-04-07T17:48:41.615153+0000 mon.a (mon.0) 4650 : cluster [DBG] osdmap e2633: 8 total, 8 up, 8 in 2024-04-07T17:48:42.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:48:42 smithi100 bash[76202]: audit 2024-04-07T17:48:41.626019+0000 mon.a (mon.0) 4651 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:48:43.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:48:42 smithi144 bash[66654]: audit 2024-04-07T17:48:41.604004+0000 mon.a (mon.0) 4648 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:48:43.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:48:42 smithi144 bash[66654]: audit 2024-04-07T17:48:41.607956+0000 mon.a (mon.0) 4649 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:48:43.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:48:42 smithi144 bash[66654]: cluster 2024-04-07T17:48:41.615153+0000 mon.a (mon.0) 4650 : cluster [DBG] osdmap e2633: 8 total, 8 up, 8 in 2024-04-07T17:48:43.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:48:42 smithi144 bash[66654]: audit 2024-04-07T17:48:41.626019+0000 mon.a (mon.0) 4651 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:48:43.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:48:43 smithi100 bash[74942]: cluster 2024-04-07T17:48:42.633733+0000 mon.a (mon.0) 4652 : cluster [DBG] osdmap e2634: 8 total, 8 up, 8 in 2024-04-07T17:48:43.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:48:43 smithi100 bash[74942]: cluster 2024-04-07T17:48:42.760314+0000 mgr.y (mgr.25175) 2800 : cluster [DBG] pgmap v5009: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 118 KiB/s rd, 8.0 KiB/s wr, 156 op/s 2024-04-07T17:48:43.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:48:43 smithi100 bash[74942]: cluster 2024-04-07T17:48:43.497310+0000 mon.a (mon.0) 4653 : cluster [DBG] osdmap e2635: 8 total, 8 up, 8 in 2024-04-07T17:48:43.953 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:48:43 smithi100 bash[76202]: cluster 2024-04-07T17:48:42.633733+0000 mon.a (mon.0) 4652 : cluster [DBG] osdmap e2634: 8 total, 8 up, 8 in 2024-04-07T17:48:43.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:48:43 smithi100 bash[76202]: cluster 2024-04-07T17:48:42.760314+0000 mgr.y (mgr.25175) 2800 : cluster [DBG] pgmap v5009: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 118 KiB/s rd, 8.0 KiB/s wr, 156 op/s 2024-04-07T17:48:43.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:48:43 smithi100 bash[76202]: cluster 2024-04-07T17:48:43.497310+0000 mon.a (mon.0) 4653 : cluster [DBG] osdmap e2635: 8 total, 8 up, 8 in 2024-04-07T17:48:44.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:48:43 smithi144 bash[66654]: cluster 2024-04-07T17:48:42.633733+0000 mon.a (mon.0) 4652 : cluster [DBG] osdmap e2634: 8 total, 8 up, 8 in 2024-04-07T17:48:44.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:48:43 smithi144 bash[66654]: cluster 2024-04-07T17:48:42.760314+0000 mgr.y (mgr.25175) 2800 : cluster [DBG] pgmap v5009: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 118 KiB/s rd, 8.0 KiB/s wr, 156 op/s 2024-04-07T17:48:44.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:48:43 smithi144 bash[66654]: cluster 2024-04-07T17:48:43.497310+0000 mon.a (mon.0) 4653 : cluster [DBG] osdmap e2635: 8 total, 8 up, 8 in 2024-04-07T17:48:44.897 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestClone::test_resize_stat PASSED [ 63%] 2024-04-07T17:48:45.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:48:45 smithi100 bash[74942]: cluster 2024-04-07T17:48:44.648860+0000 mon.a (mon.0) 4654 : cluster [DBG] osdmap e2636: 8 total, 8 up, 8 in 2024-04-07T17:48:45.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:48:45 smithi100 bash[74942]: cluster 2024-04-07T17:48:44.761108+0000 mgr.y (mgr.25175) 2801 : cluster [DBG] pgmap v5012: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 185 KiB/s rd, 15 KiB/s wr, 253 op/s 2024-04-07T17:48:45.953 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:48:45 smithi100 bash[76202]: cluster 2024-04-07T17:48:44.648860+0000 mon.a (mon.0) 4654 : cluster [DBG] osdmap e2636: 8 total, 8 up, 8 in 2024-04-07T17:48:45.953 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:48:45 smithi100 bash[76202]: cluster 2024-04-07T17:48:44.761108+0000 mgr.y (mgr.25175) 2801 : cluster [DBG] pgmap v5012: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 185 KiB/s rd, 15 KiB/s wr, 253 op/s 2024-04-07T17:48:46.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:48:45 smithi144 bash[66654]: cluster 2024-04-07T17:48:44.648860+0000 mon.a (mon.0) 4654 : cluster [DBG] osdmap e2636: 8 total, 8 up, 8 in 2024-04-07T17:48:46.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:48:45 smithi144 bash[66654]: cluster 2024-04-07T17:48:44.761108+0000 mgr.y (mgr.25175) 2801 : cluster [DBG] pgmap v5012: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 185 KiB/s rd, 15 KiB/s wr, 253 op/s 2024-04-07T17:48:46.906 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestClone::test_resize_io PASSED [ 64%] 2024-04-07T17:48:46.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:48:46 smithi100 bash[74942]: cluster 2024-04-07T17:48:45.661373+0000 mon.a (mon.0) 4655 : cluster [DBG] osdmap e2637: 8 total, 8 up, 8 in 2024-04-07T17:48:46.953 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:48:46 smithi100 bash[76202]: cluster 2024-04-07T17:48:45.661373+0000 mon.a (mon.0) 4655 : cluster [DBG] osdmap e2637: 8 total, 8 up, 8 in 2024-04-07T17:48:47.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:48:46 smithi144 bash[66654]: cluster 2024-04-07T17:48:45.661373+0000 mon.a (mon.0) 4655 : cluster [DBG] osdmap e2637: 8 total, 8 up, 8 in 2024-04-07T17:48:47.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:48:47 smithi100 bash[74942]: cluster 2024-04-07T17:48:46.668889+0000 mon.a (mon.0) 4656 : cluster [DBG] osdmap e2638: 8 total, 8 up, 8 in 2024-04-07T17:48:47.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:48:47 smithi100 bash[74942]: cluster 2024-04-07T17:48:46.761858+0000 mgr.y (mgr.25175) 2802 : cluster [DBG] pgmap v5015: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 107 KiB/s rd, 8.5 KiB/s wr, 146 op/s 2024-04-07T17:48:47.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:48:47 smithi100 bash[74942]: audit 2024-04-07T17:48:46.883413+0000 mon.a (mon.0) 4657 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:48:47.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:48:47 smithi100 bash[76202]: cluster 2024-04-07T17:48:46.668889+0000 mon.a (mon.0) 4656 : cluster [DBG] osdmap e2638: 8 total, 8 up, 8 in 2024-04-07T17:48:47.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:48:47 smithi100 bash[76202]: cluster 2024-04-07T17:48:46.761858+0000 mgr.y (mgr.25175) 2802 : cluster [DBG] pgmap v5015: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 107 KiB/s rd, 8.5 KiB/s wr, 146 op/s 2024-04-07T17:48:47.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:48:47 smithi100 bash[76202]: audit 2024-04-07T17:48:46.883413+0000 mon.a (mon.0) 4657 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:48:48.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:48:47 smithi144 bash[66654]: cluster 2024-04-07T17:48:46.668889+0000 mon.a (mon.0) 4656 : cluster [DBG] osdmap e2638: 8 total, 8 up, 8 in 2024-04-07T17:48:48.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:48:47 smithi144 bash[66654]: cluster 2024-04-07T17:48:46.761858+0000 mgr.y (mgr.25175) 2802 : cluster [DBG] pgmap v5015: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 107 KiB/s rd, 8.5 KiB/s wr, 146 op/s 2024-04-07T17:48:48.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:48:47 smithi144 bash[66654]: audit 2024-04-07T17:48:46.883413+0000 mon.a (mon.0) 4657 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:48:48.677 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:48:48 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:48:48] "GET /metrics HTTP/1.1" 200 36744 "" "Prometheus/2.43.0" 2024-04-07T17:48:48.731 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestClone::test_read PASSED [ 64%] 2024-04-07T17:48:48.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:48:48 smithi100 bash[74942]: cluster 2024-04-07T17:48:47.680295+0000 mon.a (mon.0) 4658 : cluster [DBG] osdmap e2639: 8 total, 8 up, 8 in 2024-04-07T17:48:48.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:48:48 smithi100 bash[74942]: cluster 2024-04-07T17:48:48.494405+0000 mon.a (mon.0) 4659 : cluster [DBG] osdmap e2640: 8 total, 8 up, 8 in 2024-04-07T17:48:48.953 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:48:48 smithi100 bash[76202]: cluster 2024-04-07T17:48:47.680295+0000 mon.a (mon.0) 4658 : cluster [DBG] osdmap e2639: 8 total, 8 up, 8 in 2024-04-07T17:48:48.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:48:48 smithi100 bash[76202]: cluster 2024-04-07T17:48:48.494405+0000 mon.a (mon.0) 4659 : cluster [DBG] osdmap e2640: 8 total, 8 up, 8 in 2024-04-07T17:48:49.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:48:48 smithi144 bash[66654]: cluster 2024-04-07T17:48:47.680295+0000 mon.a (mon.0) 4658 : cluster [DBG] osdmap e2639: 8 total, 8 up, 8 in 2024-04-07T17:48:49.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:48:48 smithi144 bash[66654]: cluster 2024-04-07T17:48:48.494405+0000 mon.a (mon.0) 4659 : cluster [DBG] osdmap e2640: 8 total, 8 up, 8 in 2024-04-07T17:48:49.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:48:49 smithi100 bash[74942]: cluster 2024-04-07T17:48:48.762595+0000 mgr.y (mgr.25175) 2803 : cluster [DBG] pgmap v5018: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 253 KiB/s rd, 18 KiB/s wr, 345 op/s 2024-04-07T17:48:49.953 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:48:49 smithi100 bash[76202]: cluster 2024-04-07T17:48:48.762595+0000 mgr.y (mgr.25175) 2803 : cluster [DBG] pgmap v5018: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 253 KiB/s rd, 18 KiB/s wr, 345 op/s 2024-04-07T17:48:50.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:48:49 smithi144 bash[66654]: cluster 2024-04-07T17:48:48.762595+0000 mgr.y (mgr.25175) 2803 : cluster [DBG] pgmap v5018: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 253 KiB/s rd, 18 KiB/s wr, 345 op/s 2024-04-07T17:48:50.944 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestClone::test_write PASSED [ 65%] 2024-04-07T17:48:50.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:48:50 smithi100 bash[74942]: cluster 2024-04-07T17:48:49.694640+0000 mon.a (mon.0) 4660 : cluster [DBG] osdmap e2641: 8 total, 8 up, 8 in 2024-04-07T17:48:50.953 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:48:50 smithi100 bash[76202]: cluster 2024-04-07T17:48:49.694640+0000 mon.a (mon.0) 4660 : cluster [DBG] osdmap e2641: 8 total, 8 up, 8 in 2024-04-07T17:48:51.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:48:50 smithi144 bash[66654]: cluster 2024-04-07T17:48:49.694640+0000 mon.a (mon.0) 4660 : cluster [DBG] osdmap e2641: 8 total, 8 up, 8 in 2024-04-07T17:48:52.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:48:51 smithi144 bash[66654]: cluster 2024-04-07T17:48:50.705449+0000 mon.a (mon.0) 4661 : cluster [DBG] osdmap e2642: 8 total, 8 up, 8 in 2024-04-07T17:48:52.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:48:51 smithi144 bash[66654]: cluster 2024-04-07T17:48:50.763227+0000 mgr.y (mgr.25175) 2804 : cluster [DBG] pgmap v5021: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 311 KiB/s rd, 27 KiB/s wr, 430 op/s 2024-04-07T17:48:52.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:48:51 smithi100 bash[74942]: cluster 2024-04-07T17:48:50.705449+0000 mon.a (mon.0) 4661 : cluster [DBG] osdmap e2642: 8 total, 8 up, 8 in 2024-04-07T17:48:52.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:48:51 smithi100 bash[74942]: cluster 2024-04-07T17:48:50.763227+0000 mgr.y (mgr.25175) 2804 : cluster [DBG] pgmap v5021: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 311 KiB/s rd, 27 KiB/s wr, 430 op/s 2024-04-07T17:48:52.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:48:51 smithi100 bash[76202]: cluster 2024-04-07T17:48:50.705449+0000 mon.a (mon.0) 4661 : cluster [DBG] osdmap e2642: 8 total, 8 up, 8 in 2024-04-07T17:48:52.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:48:51 smithi100 bash[76202]: cluster 2024-04-07T17:48:50.763227+0000 mgr.y (mgr.25175) 2804 : cluster [DBG] pgmap v5021: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 311 KiB/s rd, 27 KiB/s wr, 430 op/s 2024-04-07T17:48:52.614 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:48:52 smithi144 bash[95233]: ts=2024-04-07T17:48:52.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:48:53.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:48:52 smithi144 bash[66654]: cluster 2024-04-07T17:48:51.716883+0000 mon.a (mon.0) 4662 : cluster [DBG] osdmap e2643: 8 total, 8 up, 8 in 2024-04-07T17:48:53.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:48:52 smithi100 bash[74942]: cluster 2024-04-07T17:48:51.716883+0000 mon.a (mon.0) 4662 : cluster [DBG] osdmap e2643: 8 total, 8 up, 8 in 2024-04-07T17:48:53.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:48:52 smithi100 bash[76202]: cluster 2024-04-07T17:48:51.716883+0000 mon.a (mon.0) 4662 : cluster [DBG] osdmap e2643: 8 total, 8 up, 8 in 2024-04-07T17:48:54.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:48:53 smithi144 bash[66654]: cluster 2024-04-07T17:48:52.764513+0000 mgr.y (mgr.25175) 2805 : cluster [DBG] pgmap v5023: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 243 KiB/s rd, 18 KiB/s wr, 322 op/s 2024-04-07T17:48:54.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:48:53 smithi144 bash[66654]: cluster 2024-04-07T17:48:53.497178+0000 mon.a (mon.0) 4663 : cluster [DBG] osdmap e2644: 8 total, 8 up, 8 in 2024-04-07T17:48:54.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:48:53 smithi100 bash[74942]: cluster 2024-04-07T17:48:52.764513+0000 mgr.y (mgr.25175) 2805 : cluster [DBG] pgmap v5023: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 243 KiB/s rd, 18 KiB/s wr, 322 op/s 2024-04-07T17:48:54.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:48:53 smithi100 bash[74942]: cluster 2024-04-07T17:48:53.497178+0000 mon.a (mon.0) 4663 : cluster [DBG] osdmap e2644: 8 total, 8 up, 8 in 2024-04-07T17:48:54.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:48:53 smithi100 bash[76202]: cluster 2024-04-07T17:48:52.764513+0000 mgr.y (mgr.25175) 2805 : cluster [DBG] pgmap v5023: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 243 KiB/s rd, 18 KiB/s wr, 322 op/s 2024-04-07T17:48:54.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:48:53 smithi100 bash[76202]: cluster 2024-04-07T17:48:53.497178+0000 mon.a (mon.0) 4663 : cluster [DBG] osdmap e2644: 8 total, 8 up, 8 in 2024-04-07T17:48:56.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:48:55 smithi144 bash[66654]: cluster 2024-04-07T17:48:54.765505+0000 mgr.y (mgr.25175) 2806 : cluster [DBG] pgmap v5025: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 367 KiB/s rd, 20 KiB/s wr, 475 op/s 2024-04-07T17:48:56.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:48:55 smithi100 bash[74942]: cluster 2024-04-07T17:48:54.765505+0000 mgr.y (mgr.25175) 2806 : cluster [DBG] pgmap v5025: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 367 KiB/s rd, 20 KiB/s wr, 475 op/s 2024-04-07T17:48:56.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:48:55 smithi100 bash[76202]: cluster 2024-04-07T17:48:54.765505+0000 mgr.y (mgr.25175) 2806 : cluster [DBG] pgmap v5025: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 367 KiB/s rd, 20 KiB/s wr, 475 op/s 2024-04-07T17:48:58.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:48:57 smithi144 bash[66654]: cluster 2024-04-07T17:48:56.766410+0000 mgr.y (mgr.25175) 2807 : cluster [DBG] pgmap v5026: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 419 KiB/s rd, 15 KiB/s wr, 523 op/s 2024-04-07T17:48:58.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:48:57 smithi100 bash[74942]: cluster 2024-04-07T17:48:56.766410+0000 mgr.y (mgr.25175) 2807 : cluster [DBG] pgmap v5026: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 419 KiB/s rd, 15 KiB/s wr, 523 op/s 2024-04-07T17:48:58.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:48:57 smithi100 bash[76202]: cluster 2024-04-07T17:48:56.766410+0000 mgr.y (mgr.25175) 2807 : cluster [DBG] pgmap v5026: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 419 KiB/s rd, 15 KiB/s wr, 523 op/s 2024-04-07T17:48:58.703 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:48:58 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:48:58] "GET /metrics HTTP/1.1" 200 36744 "" "Prometheus/2.43.0" 2024-04-07T17:48:59.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:48:59 smithi144 bash[66654]: cluster 2024-04-07T17:48:58.504748+0000 mon.a (mon.0) 4664 : cluster [DBG] osdmap e2645: 8 total, 8 up, 8 in 2024-04-07T17:48:59.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:48:59 smithi144 bash[66654]: cluster 2024-04-07T17:48:58.767149+0000 mgr.y (mgr.25175) 2808 : cluster [DBG] pgmap v5028: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 807 KiB/s rd, 19 KiB/s wr, 968 op/s 2024-04-07T17:48:59.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:48:59 smithi100 bash[74942]: cluster 2024-04-07T17:48:58.504748+0000 mon.a (mon.0) 4664 : cluster [DBG] osdmap e2645: 8 total, 8 up, 8 in 2024-04-07T17:48:59.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:48:59 smithi100 bash[74942]: cluster 2024-04-07T17:48:58.767149+0000 mgr.y (mgr.25175) 2808 : cluster [DBG] pgmap v5028: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 807 KiB/s rd, 19 KiB/s wr, 968 op/s 2024-04-07T17:48:59.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:48:59 smithi100 bash[76202]: cluster 2024-04-07T17:48:58.504748+0000 mon.a (mon.0) 4664 : cluster [DBG] osdmap e2645: 8 total, 8 up, 8 in 2024-04-07T17:48:59.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:48:59 smithi100 bash[76202]: cluster 2024-04-07T17:48:58.767149+0000 mgr.y (mgr.25175) 2808 : cluster [DBG] pgmap v5028: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 807 KiB/s rd, 19 KiB/s wr, 968 op/s 2024-04-07T17:49:02.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:49:01 smithi144 bash[66654]: cluster 2024-04-07T17:49:00.768322+0000 mgr.y (mgr.25175) 2809 : cluster [DBG] pgmap v5029: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 908 KiB/s rd, 14 KiB/s wr, 1.07k op/s 2024-04-07T17:49:02.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:49:01 smithi100 bash[74942]: cluster 2024-04-07T17:49:00.768322+0000 mgr.y (mgr.25175) 2809 : cluster [DBG] pgmap v5029: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 908 KiB/s rd, 14 KiB/s wr, 1.07k op/s 2024-04-07T17:49:02.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:49:01 smithi100 bash[76202]: cluster 2024-04-07T17:49:00.768322+0000 mgr.y (mgr.25175) 2809 : cluster [DBG] pgmap v5029: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 908 KiB/s rd, 14 KiB/s wr, 1.07k op/s 2024-04-07T17:49:02.614 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:49:02 smithi144 bash[95233]: ts=2024-04-07T17:49:02.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:49:03.089 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestClone::test_list_children PASSED [ 65%] 2024-04-07T17:49:03.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:49:02 smithi144 bash[66654]: audit 2024-04-07T17:49:01.884022+0000 mon.a (mon.0) 4665 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:49:03.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:49:02 smithi100 bash[74942]: audit 2024-04-07T17:49:01.884022+0000 mon.a (mon.0) 4665 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:49:03.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:49:02 smithi100 bash[76202]: audit 2024-04-07T17:49:01.884022+0000 mon.a (mon.0) 4665 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:49:04.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:49:03 smithi144 bash[66654]: cluster 2024-04-07T17:49:02.769582+0000 mgr.y (mgr.25175) 2810 : cluster [DBG] pgmap v5030: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 1011 KiB/s rd, 11 KiB/s wr, 1.19k op/s 2024-04-07T17:49:04.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:49:03 smithi144 bash[66654]: cluster 2024-04-07T17:49:02.846242+0000 mon.a (mon.0) 4666 : cluster [DBG] osdmap e2646: 8 total, 8 up, 8 in 2024-04-07T17:49:04.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:49:03 smithi100 bash[74942]: cluster 2024-04-07T17:49:02.769582+0000 mgr.y (mgr.25175) 2810 : cluster [DBG] pgmap v5030: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 1011 KiB/s rd, 11 KiB/s wr, 1.19k op/s 2024-04-07T17:49:04.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:49:03 smithi100 bash[74942]: cluster 2024-04-07T17:49:02.846242+0000 mon.a (mon.0) 4666 : cluster [DBG] osdmap e2646: 8 total, 8 up, 8 in 2024-04-07T17:49:04.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:49:03 smithi100 bash[76202]: cluster 2024-04-07T17:49:02.769582+0000 mgr.y (mgr.25175) 2810 : cluster [DBG] pgmap v5030: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 1011 KiB/s rd, 11 KiB/s wr, 1.19k op/s 2024-04-07T17:49:04.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:49:03 smithi100 bash[76202]: cluster 2024-04-07T17:49:02.846242+0000 mon.a (mon.0) 4666 : cluster [DBG] osdmap e2646: 8 total, 8 up, 8 in 2024-04-07T17:49:05.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:49:04 smithi144 bash[66654]: cluster 2024-04-07T17:49:03.845236+0000 mon.a (mon.0) 4667 : cluster [DBG] osdmap e2647: 8 total, 8 up, 8 in 2024-04-07T17:49:05.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:49:04 smithi100 bash[74942]: cluster 2024-04-07T17:49:03.845236+0000 mon.a (mon.0) 4667 : cluster [DBG] osdmap e2647: 8 total, 8 up, 8 in 2024-04-07T17:49:05.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:49:04 smithi100 bash[76202]: cluster 2024-04-07T17:49:03.845236+0000 mon.a (mon.0) 4667 : cluster [DBG] osdmap e2647: 8 total, 8 up, 8 in 2024-04-07T17:49:06.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:49:05 smithi100 bash[74942]: cluster 2024-04-07T17:49:04.770721+0000 mgr.y (mgr.25175) 2811 : cluster [DBG] pgmap v5033: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 997 KiB/s rd, 11 KiB/s wr, 1.18k op/s 2024-04-07T17:49:06.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:49:05 smithi100 bash[74942]: cluster 2024-04-07T17:49:04.869331+0000 mon.a (mon.0) 4668 : cluster [DBG] osdmap e2648: 8 total, 8 up, 8 in 2024-04-07T17:49:06.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:49:05 smithi100 bash[76202]: cluster 2024-04-07T17:49:04.770721+0000 mgr.y (mgr.25175) 2811 : cluster [DBG] pgmap v5033: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 997 KiB/s rd, 11 KiB/s wr, 1.18k op/s 2024-04-07T17:49:06.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:49:05 smithi100 bash[76202]: cluster 2024-04-07T17:49:04.869331+0000 mon.a (mon.0) 4668 : cluster [DBG] osdmap e2648: 8 total, 8 up, 8 in 2024-04-07T17:49:06.364 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:49:05 smithi144 bash[66654]: cluster 2024-04-07T17:49:04.770721+0000 mgr.y (mgr.25175) 2811 : cluster [DBG] pgmap v5033: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 997 KiB/s rd, 11 KiB/s wr, 1.18k op/s 2024-04-07T17:49:06.364 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:49:05 smithi144 bash[66654]: cluster 2024-04-07T17:49:04.869331+0000 mon.a (mon.0) 4668 : cluster [DBG] osdmap e2648: 8 total, 8 up, 8 in 2024-04-07T17:49:07.124 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestClone::test_flatten_errors PASSED [ 66%] 2024-04-07T17:49:07.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:49:06 smithi100 bash[74942]: cluster 2024-04-07T17:49:05.878569+0000 mon.a (mon.0) 4669 : cluster [DBG] osdmap e2649: 8 total, 8 up, 8 in 2024-04-07T17:49:07.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:49:06 smithi100 bash[76202]: cluster 2024-04-07T17:49:05.878569+0000 mon.a (mon.0) 4669 : cluster [DBG] osdmap e2649: 8 total, 8 up, 8 in 2024-04-07T17:49:07.364 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:49:06 smithi144 bash[66654]: cluster 2024-04-07T17:49:05.878569+0000 mon.a (mon.0) 4669 : cluster [DBG] osdmap e2649: 8 total, 8 up, 8 in 2024-04-07T17:49:08.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:49:07 smithi100 bash[74942]: cluster 2024-04-07T17:49:06.771528+0000 mgr.y (mgr.25175) 2812 : cluster [DBG] pgmap v5036: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 477 KiB/s rd, 12 KiB/s wr, 578 op/s 2024-04-07T17:49:08.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:49:07 smithi100 bash[74942]: cluster 2024-04-07T17:49:06.888754+0000 mon.a (mon.0) 4670 : cluster [DBG] osdmap e2650: 8 total, 8 up, 8 in 2024-04-07T17:49:08.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:49:07 smithi100 bash[76202]: cluster 2024-04-07T17:49:06.771528+0000 mgr.y (mgr.25175) 2812 : cluster [DBG] pgmap v5036: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 477 KiB/s rd, 12 KiB/s wr, 578 op/s 2024-04-07T17:49:08.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:49:07 smithi100 bash[76202]: cluster 2024-04-07T17:49:06.888754+0000 mon.a (mon.0) 4670 : cluster [DBG] osdmap e2650: 8 total, 8 up, 8 in 2024-04-07T17:49:08.364 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:49:07 smithi144 bash[66654]: cluster 2024-04-07T17:49:06.771528+0000 mgr.y (mgr.25175) 2812 : cluster [DBG] pgmap v5036: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 477 KiB/s rd, 12 KiB/s wr, 578 op/s 2024-04-07T17:49:08.364 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:49:07 smithi144 bash[66654]: cluster 2024-04-07T17:49:06.888754+0000 mon.a (mon.0) 4670 : cluster [DBG] osdmap e2650: 8 total, 8 up, 8 in 2024-04-07T17:49:08.703 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:49:08 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:49:08] "GET /metrics HTTP/1.1" 200 36744 "" "Prometheus/2.43.0" 2024-04-07T17:49:09.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:49:08 smithi100 bash[74942]: cluster 2024-04-07T17:49:07.896989+0000 mon.a (mon.0) 4671 : cluster [DBG] osdmap e2651: 8 total, 8 up, 8 in 2024-04-07T17:49:09.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:49:08 smithi100 bash[76202]: cluster 2024-04-07T17:49:07.896989+0000 mon.a (mon.0) 4671 : cluster [DBG] osdmap e2651: 8 total, 8 up, 8 in 2024-04-07T17:49:09.364 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:49:08 smithi144 bash[66654]: cluster 2024-04-07T17:49:07.896989+0000 mon.a (mon.0) 4671 : cluster [DBG] osdmap e2651: 8 total, 8 up, 8 in 2024-04-07T17:49:10.162 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestClone::test_flatten_basic PASSED [ 66%] 2024-04-07T17:49:10.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:49:09 smithi100 bash[74942]: cluster 2024-04-07T17:49:08.772417+0000 mgr.y (mgr.25175) 2813 : cluster [DBG] pgmap v5039: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 155 KiB/s rd, 5.2 KiB/s wr, 201 op/s 2024-04-07T17:49:10.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:49:09 smithi100 bash[76202]: cluster 2024-04-07T17:49:08.772417+0000 mgr.y (mgr.25175) 2813 : cluster [DBG] pgmap v5039: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 155 KiB/s rd, 5.2 KiB/s wr, 201 op/s 2024-04-07T17:49:10.364 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:49:09 smithi144 bash[66654]: cluster 2024-04-07T17:49:08.772417+0000 mgr.y (mgr.25175) 2813 : cluster [DBG] pgmap v5039: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 155 KiB/s rd, 5.2 KiB/s wr, 201 op/s 2024-04-07T17:49:11.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:49:10 smithi100 bash[76202]: cluster 2024-04-07T17:49:09.921111+0000 mon.a (mon.0) 4672 : cluster [DBG] osdmap e2652: 8 total, 8 up, 8 in 2024-04-07T17:49:11.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:49:10 smithi100 bash[74942]: cluster 2024-04-07T17:49:09.921111+0000 mon.a (mon.0) 4672 : cluster [DBG] osdmap e2652: 8 total, 8 up, 8 in 2024-04-07T17:49:11.364 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:49:10 smithi144 bash[66654]: cluster 2024-04-07T17:49:09.921111+0000 mon.a (mon.0) 4672 : cluster [DBG] osdmap e2652: 8 total, 8 up, 8 in 2024-04-07T17:49:12.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:49:11 smithi100 bash[74942]: cluster 2024-04-07T17:49:10.773441+0000 mgr.y (mgr.25175) 2814 : cluster [DBG] pgmap v5041: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 207 KiB/s rd, 12 KiB/s wr, 270 op/s 2024-04-07T17:49:12.204 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:49:11 smithi100 bash[74942]: cluster 2024-04-07T17:49:10.931228+0000 mon.a (mon.0) 4673 : cluster [DBG] osdmap e2653: 8 total, 8 up, 8 in 2024-04-07T17:49:12.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:49:11 smithi100 bash[76202]: cluster 2024-04-07T17:49:10.773441+0000 mgr.y (mgr.25175) 2814 : cluster [DBG] pgmap v5041: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 207 KiB/s rd, 12 KiB/s wr, 270 op/s 2024-04-07T17:49:12.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:49:11 smithi100 bash[76202]: cluster 2024-04-07T17:49:10.931228+0000 mon.a (mon.0) 4673 : cluster [DBG] osdmap e2653: 8 total, 8 up, 8 in 2024-04-07T17:49:12.315 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:49:11 smithi144 bash[66654]: cluster 2024-04-07T17:49:10.773441+0000 mgr.y (mgr.25175) 2814 : cluster [DBG] pgmap v5041: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 207 KiB/s rd, 12 KiB/s wr, 270 op/s 2024-04-07T17:49:12.315 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:49:11 smithi144 bash[66654]: cluster 2024-04-07T17:49:10.931228+0000 mon.a (mon.0) 4673 : cluster [DBG] osdmap e2653: 8 total, 8 up, 8 in 2024-04-07T17:49:12.614 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:49:12 smithi144 bash[95233]: ts=2024-04-07T17:49:12.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:49:13.164 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestClone::test_flatten_smaller_order PASSED [ 67%] 2024-04-07T17:49:14.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:49:13 smithi100 bash[74942]: cluster 2024-04-07T17:49:12.774733+0000 mgr.y (mgr.25175) 2815 : cluster [DBG] pgmap v5043: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 337 KiB/s rd, 26 KiB/s wr, 461 op/s 2024-04-07T17:49:14.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:49:13 smithi100 bash[74942]: cluster 2024-04-07T17:49:12.940873+0000 mon.a (mon.0) 4674 : cluster [DBG] osdmap e2654: 8 total, 8 up, 8 in 2024-04-07T17:49:14.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:49:13 smithi100 bash[74942]: cluster 2024-04-07T17:49:13.529572+0000 mon.a (mon.0) 4675 : cluster [DBG] osdmap e2655: 8 total, 8 up, 8 in 2024-04-07T17:49:14.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:49:13 smithi100 bash[76202]: cluster 2024-04-07T17:49:12.774733+0000 mgr.y (mgr.25175) 2815 : cluster [DBG] pgmap v5043: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 337 KiB/s rd, 26 KiB/s wr, 461 op/s 2024-04-07T17:49:14.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:49:13 smithi100 bash[76202]: cluster 2024-04-07T17:49:12.940873+0000 mon.a (mon.0) 4674 : cluster [DBG] osdmap e2654: 8 total, 8 up, 8 in 2024-04-07T17:49:14.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:49:13 smithi100 bash[76202]: cluster 2024-04-07T17:49:13.529572+0000 mon.a (mon.0) 4675 : cluster [DBG] osdmap e2655: 8 total, 8 up, 8 in 2024-04-07T17:49:14.364 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:49:13 smithi144 bash[66654]: cluster 2024-04-07T17:49:12.774733+0000 mgr.y (mgr.25175) 2815 : cluster [DBG] pgmap v5043: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 337 KiB/s rd, 26 KiB/s wr, 461 op/s 2024-04-07T17:49:14.364 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:49:13 smithi144 bash[66654]: cluster 2024-04-07T17:49:12.940873+0000 mon.a (mon.0) 4674 : cluster [DBG] osdmap e2654: 8 total, 8 up, 8 in 2024-04-07T17:49:14.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:49:13 smithi144 bash[66654]: cluster 2024-04-07T17:49:13.529572+0000 mon.a (mon.0) 4675 : cluster [DBG] osdmap e2655: 8 total, 8 up, 8 in 2024-04-07T17:49:16.189 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestClone::test_flatten_larger_order PASSED [ 68%] 2024-04-07T17:49:16.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:49:15 smithi100 bash[74942]: cluster 2024-04-07T17:49:14.775884+0000 mgr.y (mgr.25175) 2816 : cluster [DBG] pgmap v5046: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 456 KiB/s rd, 36 KiB/s wr, 624 op/s 2024-04-07T17:49:16.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:49:15 smithi100 bash[76202]: cluster 2024-04-07T17:49:14.775884+0000 mgr.y (mgr.25175) 2816 : cluster [DBG] pgmap v5046: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 456 KiB/s rd, 36 KiB/s wr, 624 op/s 2024-04-07T17:49:16.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:49:15 smithi144 bash[66654]: cluster 2024-04-07T17:49:14.775884+0000 mgr.y (mgr.25175) 2816 : cluster [DBG] pgmap v5046: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 456 KiB/s rd, 36 KiB/s wr, 624 op/s 2024-04-07T17:49:17.364 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:49:16 smithi144 bash[66654]: cluster 2024-04-07T17:49:15.957352+0000 mon.a (mon.0) 4676 : cluster [DBG] osdmap e2656: 8 total, 8 up, 8 in 2024-04-07T17:49:17.364 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:49:16 smithi144 bash[66654]: audit 2024-04-07T17:49:16.886020+0000 mon.a (mon.0) 4677 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:49:17.453 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:49:16 smithi100 bash[74942]: cluster 2024-04-07T17:49:15.957352+0000 mon.a (mon.0) 4676 : cluster [DBG] osdmap e2656: 8 total, 8 up, 8 in 2024-04-07T17:49:17.453 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:49:16 smithi100 bash[74942]: audit 2024-04-07T17:49:16.886020+0000 mon.a (mon.0) 4677 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:49:17.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:49:16 smithi100 bash[76202]: cluster 2024-04-07T17:49:15.957352+0000 mon.a (mon.0) 4676 : cluster [DBG] osdmap e2656: 8 total, 8 up, 8 in 2024-04-07T17:49:17.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:49:16 smithi100 bash[76202]: audit 2024-04-07T17:49:16.886020+0000 mon.a (mon.0) 4677 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:49:18.226 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestClone::test_flatten_drops_cache PASSED [ 68%] 2024-04-07T17:49:18.364 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:49:17 smithi144 bash[66654]: cluster 2024-04-07T17:49:16.776745+0000 mgr.y (mgr.25175) 2817 : cluster [DBG] pgmap v5048: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 378 KiB/s rd, 30 KiB/s wr, 519 op/s 2024-04-07T17:49:18.364 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:49:17 smithi144 bash[66654]: cluster 2024-04-07T17:49:16.968347+0000 mon.a (mon.0) 4678 : cluster [DBG] osdmap e2657: 8 total, 8 up, 8 in 2024-04-07T17:49:18.426 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:49:17 smithi100 bash[74942]: cluster 2024-04-07T17:49:16.776745+0000 mgr.y (mgr.25175) 2817 : cluster [DBG] pgmap v5048: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 378 KiB/s rd, 30 KiB/s wr, 519 op/s 2024-04-07T17:49:18.426 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:49:17 smithi100 bash[74942]: cluster 2024-04-07T17:49:16.968347+0000 mon.a (mon.0) 4678 : cluster [DBG] osdmap e2657: 8 total, 8 up, 8 in 2024-04-07T17:49:18.426 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:49:17 smithi100 bash[76202]: cluster 2024-04-07T17:49:16.776745+0000 mgr.y (mgr.25175) 2817 : cluster [DBG] pgmap v5048: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 378 KiB/s rd, 30 KiB/s wr, 519 op/s 2024-04-07T17:49:18.426 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:49:17 smithi100 bash[76202]: cluster 2024-04-07T17:49:16.968347+0000 mon.a (mon.0) 4678 : cluster [DBG] osdmap e2657: 8 total, 8 up, 8 in 2024-04-07T17:49:18.703 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:49:18 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:49:18] "GET /metrics HTTP/1.1" 200 36745 "" "Prometheus/2.43.0" 2024-04-07T17:49:19.364 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:49:18 smithi144 bash[66654]: cluster 2024-04-07T17:49:17.978493+0000 mon.a (mon.0) 4679 : cluster [DBG] osdmap e2658: 8 total, 8 up, 8 in 2024-04-07T17:49:19.364 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:49:18 smithi144 bash[66654]: cluster 2024-04-07T17:49:18.525844+0000 mon.a (mon.0) 4680 : cluster [DBG] osdmap e2659: 8 total, 8 up, 8 in 2024-04-07T17:49:19.453 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:49:18 smithi100 bash[74942]: cluster 2024-04-07T17:49:17.978493+0000 mon.a (mon.0) 4679 : cluster [DBG] osdmap e2658: 8 total, 8 up, 8 in 2024-04-07T17:49:19.453 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:49:18 smithi100 bash[74942]: cluster 2024-04-07T17:49:18.525844+0000 mon.a (mon.0) 4680 : cluster [DBG] osdmap e2659: 8 total, 8 up, 8 in 2024-04-07T17:49:19.453 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:49:18 smithi100 bash[76202]: cluster 2024-04-07T17:49:17.978493+0000 mon.a (mon.0) 4679 : cluster [DBG] osdmap e2658: 8 total, 8 up, 8 in 2024-04-07T17:49:19.453 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:49:18 smithi100 bash[76202]: cluster 2024-04-07T17:49:18.525844+0000 mon.a (mon.0) 4680 : cluster [DBG] osdmap e2659: 8 total, 8 up, 8 in 2024-04-07T17:49:20.364 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:49:19 smithi144 bash[66654]: cluster 2024-04-07T17:49:18.777543+0000 mgr.y (mgr.25175) 2818 : cluster [DBG] pgmap v5052: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 357 KiB/s rd, 17 KiB/s wr, 464 op/s 2024-04-07T17:49:20.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:49:19 smithi144 bash[66654]: cluster 2024-04-07T17:49:19.533275+0000 mon.a (mon.0) 4681 : cluster [DBG] osdmap e2660: 8 total, 8 up, 8 in 2024-04-07T17:49:20.453 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:49:19 smithi100 bash[74942]: cluster 2024-04-07T17:49:18.777543+0000 mgr.y (mgr.25175) 2818 : cluster [DBG] pgmap v5052: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 357 KiB/s rd, 17 KiB/s wr, 464 op/s 2024-04-07T17:49:20.453 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:49:19 smithi100 bash[74942]: cluster 2024-04-07T17:49:19.533275+0000 mon.a (mon.0) 4681 : cluster [DBG] osdmap e2660: 8 total, 8 up, 8 in 2024-04-07T17:49:20.453 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:49:19 smithi100 bash[76202]: cluster 2024-04-07T17:49:18.777543+0000 mgr.y (mgr.25175) 2818 : cluster [DBG] pgmap v5052: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 357 KiB/s rd, 17 KiB/s wr, 464 op/s 2024-04-07T17:49:20.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:49:19 smithi100 bash[76202]: cluster 2024-04-07T17:49:19.533275+0000 mon.a (mon.0) 4681 : cluster [DBG] osdmap e2660: 8 total, 8 up, 8 in 2024-04-07T17:49:21.773 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestClone::test_flatten_multi_level PASSED [ 69%] 2024-04-07T17:49:21.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:49:21 smithi144 bash[66654]: cluster 2024-04-07T17:49:20.541490+0000 mon.a (mon.0) 4682 : cluster [DBG] osdmap e2661: 8 total, 8 up, 8 in 2024-04-07T17:49:21.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:49:21 smithi144 bash[66654]: cluster 2024-04-07T17:49:20.778281+0000 mgr.y (mgr.25175) 2819 : cluster [DBG] pgmap v5055: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 552 KiB/s rd, 34 KiB/s wr, 731 op/s 2024-04-07T17:49:21.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:49:21 smithi100 bash[74942]: cluster 2024-04-07T17:49:20.541490+0000 mon.a (mon.0) 4682 : cluster [DBG] osdmap e2661: 8 total, 8 up, 8 in 2024-04-07T17:49:21.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:49:21 smithi100 bash[74942]: cluster 2024-04-07T17:49:20.778281+0000 mgr.y (mgr.25175) 2819 : cluster [DBG] pgmap v5055: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 552 KiB/s rd, 34 KiB/s wr, 731 op/s 2024-04-07T17:49:21.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:49:21 smithi100 bash[76202]: cluster 2024-04-07T17:49:20.541490+0000 mon.a (mon.0) 4682 : cluster [DBG] osdmap e2661: 8 total, 8 up, 8 in 2024-04-07T17:49:21.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:49:21 smithi100 bash[76202]: cluster 2024-04-07T17:49:20.778281+0000 mgr.y (mgr.25175) 2819 : cluster [DBG] pgmap v5055: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 552 KiB/s rd, 34 KiB/s wr, 731 op/s 2024-04-07T17:49:22.614 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:49:22 smithi144 bash[66654]: cluster 2024-04-07T17:49:21.554799+0000 mon.a (mon.0) 4683 : cluster [DBG] osdmap e2662: 8 total, 8 up, 8 in 2024-04-07T17:49:22.615 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:49:22 smithi144 bash[95233]: ts=2024-04-07T17:49:22.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:49:22.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:49:22 smithi100 bash[74942]: cluster 2024-04-07T17:49:21.554799+0000 mon.a (mon.0) 4683 : cluster [DBG] osdmap e2662: 8 total, 8 up, 8 in 2024-04-07T17:49:22.953 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:49:22 smithi100 bash[76202]: cluster 2024-04-07T17:49:21.554799+0000 mon.a (mon.0) 4683 : cluster [DBG] osdmap e2662: 8 total, 8 up, 8 in 2024-04-07T17:49:23.765 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestClone::test_flatten_with_progress PASSED [ 69%] 2024-04-07T17:49:23.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:49:23 smithi144 bash[66654]: cluster 2024-04-07T17:49:22.562231+0000 mon.a (mon.0) 4684 : cluster [DBG] osdmap e2663: 8 total, 8 up, 8 in 2024-04-07T17:49:23.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:49:23 smithi144 bash[66654]: cluster 2024-04-07T17:49:22.779175+0000 mgr.y (mgr.25175) 2820 : cluster [DBG] pgmap v5058: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 326 KiB/s rd, 29 KiB/s wr, 455 op/s 2024-04-07T17:49:23.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:49:23 smithi144 bash[66654]: cluster 2024-04-07T17:49:23.529098+0000 mon.a (mon.0) 4685 : cluster [DBG] osdmap e2664: 8 total, 8 up, 8 in 2024-04-07T17:49:23.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:49:23 smithi100 bash[74942]: cluster 2024-04-07T17:49:22.562231+0000 mon.a (mon.0) 4684 : cluster [DBG] osdmap e2663: 8 total, 8 up, 8 in 2024-04-07T17:49:23.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:49:23 smithi100 bash[74942]: cluster 2024-04-07T17:49:22.779175+0000 mgr.y (mgr.25175) 2820 : cluster [DBG] pgmap v5058: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 326 KiB/s rd, 29 KiB/s wr, 455 op/s 2024-04-07T17:49:23.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:49:23 smithi100 bash[74942]: cluster 2024-04-07T17:49:23.529098+0000 mon.a (mon.0) 4685 : cluster [DBG] osdmap e2664: 8 total, 8 up, 8 in 2024-04-07T17:49:23.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:49:23 smithi100 bash[76202]: cluster 2024-04-07T17:49:22.562231+0000 mon.a (mon.0) 4684 : cluster [DBG] osdmap e2663: 8 total, 8 up, 8 in 2024-04-07T17:49:23.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:49:23 smithi100 bash[76202]: cluster 2024-04-07T17:49:22.779175+0000 mgr.y (mgr.25175) 2820 : cluster [DBG] pgmap v5058: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 326 KiB/s rd, 29 KiB/s wr, 455 op/s 2024-04-07T17:49:23.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:49:23 smithi100 bash[76202]: cluster 2024-04-07T17:49:23.529098+0000 mon.a (mon.0) 4685 : cluster [DBG] osdmap e2664: 8 total, 8 up, 8 in 2024-04-07T17:49:25.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:49:25 smithi144 bash[66654]: cluster 2024-04-07T17:49:24.580854+0000 mon.a (mon.0) 4686 : cluster [DBG] osdmap e2665: 8 total, 8 up, 8 in 2024-04-07T17:49:25.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:49:25 smithi144 bash[66654]: cluster 2024-04-07T17:49:24.780101+0000 mgr.y (mgr.25175) 2821 : cluster [DBG] pgmap v5061: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 325 KiB/s rd, 28 KiB/s wr, 450 op/s 2024-04-07T17:49:25.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:49:25 smithi100 bash[74942]: cluster 2024-04-07T17:49:24.580854+0000 mon.a (mon.0) 4686 : cluster [DBG] osdmap e2665: 8 total, 8 up, 8 in 2024-04-07T17:49:25.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:49:25 smithi100 bash[74942]: cluster 2024-04-07T17:49:24.780101+0000 mgr.y (mgr.25175) 2821 : cluster [DBG] pgmap v5061: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 325 KiB/s rd, 28 KiB/s wr, 450 op/s 2024-04-07T17:49:25.953 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:49:25 smithi100 bash[76202]: cluster 2024-04-07T17:49:24.580854+0000 mon.a (mon.0) 4686 : cluster [DBG] osdmap e2665: 8 total, 8 up, 8 in 2024-04-07T17:49:25.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:49:25 smithi100 bash[76202]: cluster 2024-04-07T17:49:24.780101+0000 mgr.y (mgr.25175) 2821 : cluster [DBG] pgmap v5061: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 325 KiB/s rd, 28 KiB/s wr, 450 op/s 2024-04-07T17:49:26.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:49:26 smithi144 bash[66654]: cluster 2024-04-07T17:49:25.591251+0000 mon.a (mon.0) 4687 : cluster [DBG] osdmap e2666: 8 total, 8 up, 8 in 2024-04-07T17:49:26.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:49:26 smithi100 bash[74942]: cluster 2024-04-07T17:49:25.591251+0000 mon.a (mon.0) 4687 : cluster [DBG] osdmap e2666: 8 total, 8 up, 8 in 2024-04-07T17:49:26.953 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:49:26 smithi100 bash[76202]: cluster 2024-04-07T17:49:25.591251+0000 mon.a (mon.0) 4687 : cluster [DBG] osdmap e2666: 8 total, 8 up, 8 in 2024-04-07T17:49:27.857 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestClone::test_resize_flatten_multi_level PASSED [ 70%] 2024-04-07T17:49:27.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:49:27 smithi144 bash[66654]: cluster 2024-04-07T17:49:26.592366+0000 mon.a (mon.0) 4688 : cluster [DBG] osdmap e2667: 8 total, 8 up, 8 in 2024-04-07T17:49:27.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:49:27 smithi144 bash[66654]: cluster 2024-04-07T17:49:26.780857+0000 mgr.y (mgr.25175) 2822 : cluster [DBG] pgmap v5064: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 168 KiB/s rd, 14 KiB/s wr, 230 op/s 2024-04-07T17:49:27.953 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:49:27 smithi100 bash[76202]: cluster 2024-04-07T17:49:26.592366+0000 mon.a (mon.0) 4688 : cluster [DBG] osdmap e2667: 8 total, 8 up, 8 in 2024-04-07T17:49:27.953 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:49:27 smithi100 bash[76202]: cluster 2024-04-07T17:49:26.780857+0000 mgr.y (mgr.25175) 2822 : cluster [DBG] pgmap v5064: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 168 KiB/s rd, 14 KiB/s wr, 230 op/s 2024-04-07T17:49:27.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:49:27 smithi100 bash[74942]: cluster 2024-04-07T17:49:26.592366+0000 mon.a (mon.0) 4688 : cluster [DBG] osdmap e2667: 8 total, 8 up, 8 in 2024-04-07T17:49:27.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:49:27 smithi100 bash[74942]: cluster 2024-04-07T17:49:26.780857+0000 mgr.y (mgr.25175) 2822 : cluster [DBG] pgmap v5064: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 168 KiB/s rd, 14 KiB/s wr, 230 op/s 2024-04-07T17:49:28.703 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:49:28 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:49:28] "GET /metrics HTTP/1.1" 200 36745 "" "Prometheus/2.43.0" 2024-04-07T17:49:28.703 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:49:28 smithi100 bash[74942]: cluster 2024-04-07T17:49:27.620204+0000 mon.a (mon.0) 4689 : cluster [DBG] osdmap e2668: 8 total, 8 up, 8 in 2024-04-07T17:49:28.703 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:49:28 smithi100 bash[74942]: cluster 2024-04-07T17:49:28.529615+0000 mon.a (mon.0) 4690 : cluster [DBG] osdmap e2669: 8 total, 8 up, 8 in 2024-04-07T17:49:28.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:49:28 smithi100 bash[76202]: cluster 2024-04-07T17:49:27.620204+0000 mon.a (mon.0) 4689 : cluster [DBG] osdmap e2668: 8 total, 8 up, 8 in 2024-04-07T17:49:28.704 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:49:28 smithi100 bash[76202]: cluster 2024-04-07T17:49:28.529615+0000 mon.a (mon.0) 4690 : cluster [DBG] osdmap e2669: 8 total, 8 up, 8 in 2024-04-07T17:49:28.866 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:49:28 smithi144 bash[66654]: cluster 2024-04-07T17:49:27.620204+0000 mon.a (mon.0) 4689 : cluster [DBG] osdmap e2668: 8 total, 8 up, 8 in 2024-04-07T17:49:28.867 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:49:28 smithi144 bash[66654]: cluster 2024-04-07T17:49:28.529615+0000 mon.a (mon.0) 4690 : cluster [DBG] osdmap e2669: 8 total, 8 up, 8 in 2024-04-07T17:49:29.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:49:29 smithi100 bash[74942]: cluster 2024-04-07T17:49:28.781595+0000 mgr.y (mgr.25175) 2823 : cluster [DBG] pgmap v5067: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 310 KiB/s rd, 18 KiB/s wr, 407 op/s 2024-04-07T17:49:29.953 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:49:29 smithi100 bash[76202]: cluster 2024-04-07T17:49:28.781595+0000 mgr.y (mgr.25175) 2823 : cluster [DBG] pgmap v5067: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 310 KiB/s rd, 18 KiB/s wr, 407 op/s 2024-04-07T17:49:30.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:49:29 smithi144 bash[66654]: cluster 2024-04-07T17:49:28.781595+0000 mgr.y (mgr.25175) 2823 : cluster [DBG] pgmap v5067: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 310 KiB/s rd, 18 KiB/s wr, 407 op/s 2024-04-07T17:49:30.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:49:30 smithi100 bash[74942]: cluster 2024-04-07T17:49:29.638403+0000 mon.a (mon.0) 4691 : cluster [DBG] osdmap e2670: 8 total, 8 up, 8 in 2024-04-07T17:49:30.953 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:49:30 smithi100 bash[76202]: cluster 2024-04-07T17:49:29.638403+0000 mon.a (mon.0) 4691 : cluster [DBG] osdmap e2670: 8 total, 8 up, 8 in 2024-04-07T17:49:31.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:49:30 smithi144 bash[66654]: cluster 2024-04-07T17:49:29.638403+0000 mon.a (mon.0) 4691 : cluster [DBG] osdmap e2670: 8 total, 8 up, 8 in 2024-04-07T17:49:31.894 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestClone::test_trash_snapshot PASSED [ 70%] 2024-04-07T17:49:31.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:49:31 smithi100 bash[74942]: cluster 2024-04-07T17:49:30.652547+0000 mon.a (mon.0) 4692 : cluster [DBG] osdmap e2671: 8 total, 8 up, 8 in 2024-04-07T17:49:31.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:49:31 smithi100 bash[74942]: cluster 2024-04-07T17:49:30.782501+0000 mgr.y (mgr.25175) 2824 : cluster [DBG] pgmap v5070: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 360 KiB/s rd, 24 KiB/s wr, 481 op/s 2024-04-07T17:49:31.953 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:49:31 smithi100 bash[76202]: cluster 2024-04-07T17:49:30.652547+0000 mon.a (mon.0) 4692 : cluster [DBG] osdmap e2671: 8 total, 8 up, 8 in 2024-04-07T17:49:31.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:49:31 smithi100 bash[76202]: cluster 2024-04-07T17:49:30.782501+0000 mgr.y (mgr.25175) 2824 : cluster [DBG] pgmap v5070: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 360 KiB/s rd, 24 KiB/s wr, 481 op/s 2024-04-07T17:49:32.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:49:31 smithi144 bash[66654]: cluster 2024-04-07T17:49:30.652547+0000 mon.a (mon.0) 4692 : cluster [DBG] osdmap e2671: 8 total, 8 up, 8 in 2024-04-07T17:49:32.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:49:31 smithi144 bash[66654]: cluster 2024-04-07T17:49:30.782501+0000 mgr.y (mgr.25175) 2824 : cluster [DBG] pgmap v5070: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 360 KiB/s rd, 24 KiB/s wr, 481 op/s 2024-04-07T17:49:32.614 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:49:32 smithi144 bash[95233]: ts=2024-04-07T17:49:32.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:49:32.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:49:32 smithi100 bash[74942]: cluster 2024-04-07T17:49:31.667881+0000 mon.a (mon.0) 4693 : cluster [DBG] osdmap e2672: 8 total, 8 up, 8 in 2024-04-07T17:49:32.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:49:32 smithi100 bash[74942]: audit 2024-04-07T17:49:31.886497+0000 mon.a (mon.0) 4694 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:49:32.953 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:49:32 smithi100 bash[76202]: cluster 2024-04-07T17:49:31.667881+0000 mon.a (mon.0) 4693 : cluster [DBG] osdmap e2672: 8 total, 8 up, 8 in 2024-04-07T17:49:32.953 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:49:32 smithi100 bash[76202]: audit 2024-04-07T17:49:31.886497+0000 mon.a (mon.0) 4694 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:49:33.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:49:32 smithi144 bash[66654]: cluster 2024-04-07T17:49:31.667881+0000 mon.a (mon.0) 4693 : cluster [DBG] osdmap e2672: 8 total, 8 up, 8 in 2024-04-07T17:49:33.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:49:32 smithi144 bash[66654]: audit 2024-04-07T17:49:31.886497+0000 mon.a (mon.0) 4694 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:49:33.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:49:33 smithi100 bash[74942]: cluster 2024-04-07T17:49:32.663260+0000 mon.a (mon.0) 4695 : cluster [DBG] osdmap e2673: 8 total, 8 up, 8 in 2024-04-07T17:49:33.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:49:33 smithi100 bash[74942]: cluster 2024-04-07T17:49:32.783366+0000 mgr.y (mgr.25175) 2825 : cluster [DBG] pgmap v5073: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 91 KiB/s rd, 9.7 KiB/s wr, 135 op/s 2024-04-07T17:49:33.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:49:33 smithi100 bash[74942]: cluster 2024-04-07T17:49:33.534970+0000 mon.a (mon.0) 4696 : cluster [DBG] osdmap e2674: 8 total, 8 up, 8 in 2024-04-07T17:49:33.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:49:33 smithi100 bash[76202]: cluster 2024-04-07T17:49:32.663260+0000 mon.a (mon.0) 4695 : cluster [DBG] osdmap e2673: 8 total, 8 up, 8 in 2024-04-07T17:49:33.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:49:33 smithi100 bash[76202]: cluster 2024-04-07T17:49:32.783366+0000 mgr.y (mgr.25175) 2825 : cluster [DBG] pgmap v5073: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 91 KiB/s rd, 9.7 KiB/s wr, 135 op/s 2024-04-07T17:49:33.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:49:33 smithi100 bash[76202]: cluster 2024-04-07T17:49:33.534970+0000 mon.a (mon.0) 4696 : cluster [DBG] osdmap e2674: 8 total, 8 up, 8 in 2024-04-07T17:49:34.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:49:33 smithi144 bash[66654]: cluster 2024-04-07T17:49:32.663260+0000 mon.a (mon.0) 4695 : cluster [DBG] osdmap e2673: 8 total, 8 up, 8 in 2024-04-07T17:49:34.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:49:33 smithi144 bash[66654]: cluster 2024-04-07T17:49:32.783366+0000 mgr.y (mgr.25175) 2825 : cluster [DBG] pgmap v5073: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 91 KiB/s rd, 9.7 KiB/s wr, 135 op/s 2024-04-07T17:49:34.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:49:33 smithi144 bash[66654]: cluster 2024-04-07T17:49:33.534970+0000 mon.a (mon.0) 4696 : cluster [DBG] osdmap e2674: 8 total, 8 up, 8 in 2024-04-07T17:49:36.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:49:35 smithi144 bash[66654]: cluster 2024-04-07T17:49:34.784445+0000 mgr.y (mgr.25175) 2826 : cluster [DBG] pgmap v5075: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 244 KiB/s rd, 16 KiB/s wr, 317 op/s 2024-04-07T17:49:36.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:49:35 smithi100 bash[74942]: cluster 2024-04-07T17:49:34.784445+0000 mgr.y (mgr.25175) 2826 : cluster [DBG] pgmap v5075: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 244 KiB/s rd, 16 KiB/s wr, 317 op/s 2024-04-07T17:49:36.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:49:35 smithi100 bash[76202]: cluster 2024-04-07T17:49:34.784445+0000 mgr.y (mgr.25175) 2826 : cluster [DBG] pgmap v5075: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 244 KiB/s rd, 16 KiB/s wr, 317 op/s 2024-04-07T17:49:38.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:49:37 smithi144 bash[66654]: cluster 2024-04-07T17:49:36.785282+0000 mgr.y (mgr.25175) 2827 : cluster [DBG] pgmap v5076: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 169 KiB/s rd, 11 KiB/s wr, 222 op/s 2024-04-07T17:49:38.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:49:37 smithi100 bash[74942]: cluster 2024-04-07T17:49:36.785282+0000 mgr.y (mgr.25175) 2827 : cluster [DBG] pgmap v5076: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 169 KiB/s rd, 11 KiB/s wr, 222 op/s 2024-04-07T17:49:38.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:49:37 smithi100 bash[76202]: cluster 2024-04-07T17:49:36.785282+0000 mgr.y (mgr.25175) 2827 : cluster [DBG] pgmap v5076: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 169 KiB/s rd, 11 KiB/s wr, 222 op/s 2024-04-07T17:49:38.703 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:49:38 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:49:38] "GET /metrics HTTP/1.1" 200 36742 "" "Prometheus/2.43.0" 2024-04-07T17:49:39.794 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestClone::test_encryption_luks1 PASSED [ 71%] 2024-04-07T17:49:39.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:49:39 smithi144 bash[66654]: cluster 2024-04-07T17:49:38.534108+0000 mon.a (mon.0) 4697 : cluster [DBG] osdmap e2675: 8 total, 8 up, 8 in 2024-04-07T17:49:39.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:49:39 smithi144 bash[66654]: cluster 2024-04-07T17:49:38.785971+0000 mgr.y (mgr.25175) 2828 : cluster [DBG] pgmap v5078: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 142 KiB/s rd, 9.5 KiB/s wr, 185 op/s 2024-04-07T17:49:39.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:49:39 smithi100 bash[74942]: cluster 2024-04-07T17:49:38.534108+0000 mon.a (mon.0) 4697 : cluster [DBG] osdmap e2675: 8 total, 8 up, 8 in 2024-04-07T17:49:39.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:49:39 smithi100 bash[74942]: cluster 2024-04-07T17:49:38.785971+0000 mgr.y (mgr.25175) 2828 : cluster [DBG] pgmap v5078: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 142 KiB/s rd, 9.5 KiB/s wr, 185 op/s 2024-04-07T17:49:39.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:49:39 smithi100 bash[76202]: cluster 2024-04-07T17:49:38.534108+0000 mon.a (mon.0) 4697 : cluster [DBG] osdmap e2675: 8 total, 8 up, 8 in 2024-04-07T17:49:39.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:49:39 smithi100 bash[76202]: cluster 2024-04-07T17:49:38.785971+0000 mgr.y (mgr.25175) 2828 : cluster [DBG] pgmap v5078: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 142 KiB/s rd, 9.5 KiB/s wr, 185 op/s 2024-04-07T17:49:40.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:49:40 smithi144 bash[66654]: cluster 2024-04-07T17:49:39.546059+0000 mon.a (mon.0) 4698 : cluster [DBG] osdmap e2676: 8 total, 8 up, 8 in 2024-04-07T17:49:40.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:49:40 smithi100 bash[74942]: cluster 2024-04-07T17:49:39.546059+0000 mon.a (mon.0) 4698 : cluster [DBG] osdmap e2676: 8 total, 8 up, 8 in 2024-04-07T17:49:40.953 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:49:40 smithi100 bash[76202]: cluster 2024-04-07T17:49:39.546059+0000 mon.a (mon.0) 4698 : cluster [DBG] osdmap e2676: 8 total, 8 up, 8 in 2024-04-07T17:49:41.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:49:41 smithi144 bash[66654]: cluster 2024-04-07T17:49:40.555967+0000 mon.a (mon.0) 4699 : cluster [DBG] osdmap e2677: 8 total, 8 up, 8 in 2024-04-07T17:49:41.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:49:41 smithi144 bash[66654]: cluster 2024-04-07T17:49:40.786725+0000 mgr.y (mgr.25175) 2829 : cluster [DBG] pgmap v5081: 137 pgs: 137 active+clean; 8.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 77 KiB/s rd, 685 KiB/s wr, 42 op/s 2024-04-07T17:49:41.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:49:41 smithi100 bash[74942]: cluster 2024-04-07T17:49:40.555967+0000 mon.a (mon.0) 4699 : cluster [DBG] osdmap e2677: 8 total, 8 up, 8 in 2024-04-07T17:49:41.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:49:41 smithi100 bash[74942]: cluster 2024-04-07T17:49:40.786725+0000 mgr.y (mgr.25175) 2829 : cluster [DBG] pgmap v5081: 137 pgs: 137 active+clean; 8.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 77 KiB/s rd, 685 KiB/s wr, 42 op/s 2024-04-07T17:49:41.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:49:41 smithi100 bash[76202]: cluster 2024-04-07T17:49:40.555967+0000 mon.a (mon.0) 4699 : cluster [DBG] osdmap e2677: 8 total, 8 up, 8 in 2024-04-07T17:49:41.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:49:41 smithi100 bash[76202]: cluster 2024-04-07T17:49:40.786725+0000 mgr.y (mgr.25175) 2829 : cluster [DBG] pgmap v5081: 137 pgs: 137 active+clean; 8.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 77 KiB/s rd, 685 KiB/s wr, 42 op/s 2024-04-07T17:49:42.614 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:49:42 smithi144 bash[95233]: ts=2024-04-07T17:49:42.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:49:42.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:49:42 smithi144 bash[66654]: audit 2024-04-07T17:49:41.704390+0000 mon.a (mon.0) 4700 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:49:42.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:49:42 smithi144 bash[66654]: audit 2024-04-07T17:49:42.378728+0000 mon.a (mon.0) 4701 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:49:42.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:49:42 smithi144 bash[66654]: audit 2024-04-07T17:49:42.379897+0000 mon.a (mon.0) 4702 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:49:42.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:49:42 smithi144 bash[66654]: audit 2024-04-07T17:49:42.387255+0000 mon.a (mon.0) 4703 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:49:42.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:49:42 smithi100 bash[74942]: audit 2024-04-07T17:49:41.704390+0000 mon.a (mon.0) 4700 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:49:42.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:49:42 smithi100 bash[74942]: audit 2024-04-07T17:49:42.378728+0000 mon.a (mon.0) 4701 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:49:42.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:49:42 smithi100 bash[74942]: audit 2024-04-07T17:49:42.379897+0000 mon.a (mon.0) 4702 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:49:42.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:49:42 smithi100 bash[74942]: audit 2024-04-07T17:49:42.387255+0000 mon.a (mon.0) 4703 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:49:42.953 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:49:42 smithi100 bash[76202]: audit 2024-04-07T17:49:41.704390+0000 mon.a (mon.0) 4700 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:49:42.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:49:42 smithi100 bash[76202]: audit 2024-04-07T17:49:42.378728+0000 mon.a (mon.0) 4701 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:49:42.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:49:42 smithi100 bash[76202]: audit 2024-04-07T17:49:42.379897+0000 mon.a (mon.0) 4702 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:49:42.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:49:42 smithi100 bash[76202]: audit 2024-04-07T17:49:42.387255+0000 mon.a (mon.0) 4703 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:49:43.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:49:43 smithi144 bash[66654]: cluster 2024-04-07T17:49:42.787610+0000 mgr.y (mgr.25175) 2830 : cluster [DBG] pgmap v5082: 137 pgs: 137 active+clean; 8.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 93 KiB/s rd, 688 KiB/s wr, 65 op/s 2024-04-07T17:49:43.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:49:43 smithi100 bash[74942]: cluster 2024-04-07T17:49:42.787610+0000 mgr.y (mgr.25175) 2830 : cluster [DBG] pgmap v5082: 137 pgs: 137 active+clean; 8.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 93 KiB/s rd, 688 KiB/s wr, 65 op/s 2024-04-07T17:49:43.953 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:49:43 smithi100 bash[76202]: cluster 2024-04-07T17:49:42.787610+0000 mgr.y (mgr.25175) 2830 : cluster [DBG] pgmap v5082: 137 pgs: 137 active+clean; 8.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 93 KiB/s rd, 688 KiB/s wr, 65 op/s 2024-04-07T17:49:46.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:49:45 smithi144 bash[66654]: cluster 2024-04-07T17:49:44.788336+0000 mgr.y (mgr.25175) 2831 : cluster [DBG] pgmap v5083: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 127 KiB/s rd, 661 KiB/s wr, 111 op/s 2024-04-07T17:49:46.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:49:45 smithi100 bash[74942]: cluster 2024-04-07T17:49:44.788336+0000 mgr.y (mgr.25175) 2831 : cluster [DBG] pgmap v5083: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 127 KiB/s rd, 661 KiB/s wr, 111 op/s 2024-04-07T17:49:46.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:49:45 smithi100 bash[76202]: cluster 2024-04-07T17:49:44.788336+0000 mgr.y (mgr.25175) 2831 : cluster [DBG] pgmap v5083: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 127 KiB/s rd, 661 KiB/s wr, 111 op/s 2024-04-07T17:49:48.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:49:47 smithi144 bash[66654]: cluster 2024-04-07T17:49:46.788938+0000 mgr.y (mgr.25175) 2832 : cluster [DBG] pgmap v5084: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 99 KiB/s rd, 517 KiB/s wr, 86 op/s 2024-04-07T17:49:48.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:49:47 smithi144 bash[66654]: audit 2024-04-07T17:49:46.886191+0000 mon.a (mon.0) 4704 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:49:48.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:49:47 smithi100 bash[74942]: cluster 2024-04-07T17:49:46.788938+0000 mgr.y (mgr.25175) 2832 : cluster [DBG] pgmap v5084: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 99 KiB/s rd, 517 KiB/s wr, 86 op/s 2024-04-07T17:49:48.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:49:47 smithi100 bash[74942]: audit 2024-04-07T17:49:46.886191+0000 mon.a (mon.0) 4704 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:49:48.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:49:47 smithi100 bash[76202]: cluster 2024-04-07T17:49:46.788938+0000 mgr.y (mgr.25175) 2832 : cluster [DBG] pgmap v5084: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 99 KiB/s rd, 517 KiB/s wr, 86 op/s 2024-04-07T17:49:48.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:49:47 smithi100 bash[76202]: audit 2024-04-07T17:49:46.886191+0000 mon.a (mon.0) 4704 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:49:48.705 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:49:48 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:49:48] "GET /metrics HTTP/1.1" 200 36740 "" "Prometheus/2.43.0" 2024-04-07T17:49:49.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:49:49 smithi144 bash[66654]: cluster 2024-04-07T17:49:48.538919+0000 mon.a (mon.0) 4705 : cluster [DBG] osdmap e2678: 8 total, 8 up, 8 in 2024-04-07T17:49:49.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:49:49 smithi144 bash[66654]: cluster 2024-04-07T17:49:48.789424+0000 mgr.y (mgr.25175) 2833 : cluster [DBG] pgmap v5086: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 44 KiB/s rd, 4.0 KiB/s wr, 61 op/s 2024-04-07T17:49:49.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:49:49 smithi100 bash[74942]: cluster 2024-04-07T17:49:48.538919+0000 mon.a (mon.0) 4705 : cluster [DBG] osdmap e2678: 8 total, 8 up, 8 in 2024-04-07T17:49:49.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:49:49 smithi100 bash[74942]: cluster 2024-04-07T17:49:48.789424+0000 mgr.y (mgr.25175) 2833 : cluster [DBG] pgmap v5086: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 44 KiB/s rd, 4.0 KiB/s wr, 61 op/s 2024-04-07T17:49:49.953 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:49:49 smithi100 bash[76202]: cluster 2024-04-07T17:49:48.538919+0000 mon.a (mon.0) 4705 : cluster [DBG] osdmap e2678: 8 total, 8 up, 8 in 2024-04-07T17:49:49.953 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:49:49 smithi100 bash[76202]: cluster 2024-04-07T17:49:48.789424+0000 mgr.y (mgr.25175) 2833 : cluster [DBG] pgmap v5086: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 44 KiB/s rd, 4.0 KiB/s wr, 61 op/s 2024-04-07T17:49:52.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:49:51 smithi144 bash[66654]: cluster 2024-04-07T17:49:50.790139+0000 mgr.y (mgr.25175) 2834 : cluster [DBG] pgmap v5087: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 36 KiB/s rd, 3.3 KiB/s wr, 50 op/s 2024-04-07T17:49:52.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:49:51 smithi100 bash[74942]: cluster 2024-04-07T17:49:50.790139+0000 mgr.y (mgr.25175) 2834 : cluster [DBG] pgmap v5087: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 36 KiB/s rd, 3.3 KiB/s wr, 50 op/s 2024-04-07T17:49:52.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:49:51 smithi100 bash[76202]: cluster 2024-04-07T17:49:50.790139+0000 mgr.y (mgr.25175) 2834 : cluster [DBG] pgmap v5087: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 36 KiB/s rd, 3.3 KiB/s wr, 50 op/s 2024-04-07T17:49:52.614 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:49:52 smithi144 bash[95233]: ts=2024-04-07T17:49:52.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:49:53.092 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestClone::test_encryption_luks2 PASSED [ 72%] 2024-04-07T17:49:53.310 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestExclusiveLock::test_ownership PASSED [ 72%] 2024-04-07T17:49:54.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:49:53 smithi144 bash[66654]: cluster 2024-04-07T17:49:52.791336+0000 mgr.y (mgr.25175) 2835 : cluster [DBG] pgmap v5088: 137 pgs: 137 active+clean; 17 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 58 KiB/s rd, 1.2 MiB/s wr, 39 op/s 2024-04-07T17:49:54.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:49:53 smithi144 bash[66654]: cluster 2024-04-07T17:49:52.860506+0000 mon.a (mon.0) 4706 : cluster [DBG] osdmap e2679: 8 total, 8 up, 8 in 2024-04-07T17:49:54.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:49:53 smithi100 bash[74942]: cluster 2024-04-07T17:49:52.791336+0000 mgr.y (mgr.25175) 2835 : cluster [DBG] pgmap v5088: 137 pgs: 137 active+clean; 17 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 58 KiB/s rd, 1.2 MiB/s wr, 39 op/s 2024-04-07T17:49:54.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:49:53 smithi100 bash[74942]: cluster 2024-04-07T17:49:52.860506+0000 mon.a (mon.0) 4706 : cluster [DBG] osdmap e2679: 8 total, 8 up, 8 in 2024-04-07T17:49:54.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:49:53 smithi100 bash[76202]: cluster 2024-04-07T17:49:52.791336+0000 mgr.y (mgr.25175) 2835 : cluster [DBG] pgmap v5088: 137 pgs: 137 active+clean; 17 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 58 KiB/s rd, 1.2 MiB/s wr, 39 op/s 2024-04-07T17:49:54.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:49:53 smithi100 bash[76202]: cluster 2024-04-07T17:49:52.860506+0000 mon.a (mon.0) 4706 : cluster [DBG] osdmap e2679: 8 total, 8 up, 8 in 2024-04-07T17:49:55.133 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestExclusiveLock::test_snapshot_leadership PASSED [ 73%] 2024-04-07T17:49:55.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:49:54 smithi100 bash[74942]: cluster 2024-04-07T17:49:53.874749+0000 mon.a (mon.0) 4707 : cluster [DBG] osdmap e2680: 8 total, 8 up, 8 in 2024-04-07T17:49:55.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:49:54 smithi100 bash[76202]: cluster 2024-04-07T17:49:53.874749+0000 mon.a (mon.0) 4707 : cluster [DBG] osdmap e2680: 8 total, 8 up, 8 in 2024-04-07T17:49:55.327 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestExclusiveLock::test_read_only_leadership PASSED [ 73%] 2024-04-07T17:49:55.364 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:49:54 smithi144 bash[66654]: cluster 2024-04-07T17:49:53.874749+0000 mon.a (mon.0) 4707 : cluster [DBG] osdmap e2680: 8 total, 8 up, 8 in 2024-04-07T17:49:56.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:49:55 smithi100 bash[76202]: cluster 2024-04-07T17:49:54.792378+0000 mgr.y (mgr.25175) 2836 : cluster [DBG] pgmap v5091: 137 pgs: 137 active+clean; 17 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 97 KiB/s rd, 2.6 MiB/s wr, 72 op/s 2024-04-07T17:49:56.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:49:55 smithi100 bash[76202]: cluster 2024-04-07T17:49:54.875414+0000 mon.a (mon.0) 4708 : cluster [DBG] osdmap e2681: 8 total, 8 up, 8 in 2024-04-07T17:49:56.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:49:55 smithi100 bash[74942]: cluster 2024-04-07T17:49:54.792378+0000 mgr.y (mgr.25175) 2836 : cluster [DBG] pgmap v5091: 137 pgs: 137 active+clean; 17 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 97 KiB/s rd, 2.6 MiB/s wr, 72 op/s 2024-04-07T17:49:56.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:49:55 smithi100 bash[74942]: cluster 2024-04-07T17:49:54.875414+0000 mon.a (mon.0) 4708 : cluster [DBG] osdmap e2681: 8 total, 8 up, 8 in 2024-04-07T17:49:56.364 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:49:55 smithi144 bash[66654]: cluster 2024-04-07T17:49:54.792378+0000 mgr.y (mgr.25175) 2836 : cluster [DBG] pgmap v5091: 137 pgs: 137 active+clean; 17 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 97 KiB/s rd, 2.6 MiB/s wr, 72 op/s 2024-04-07T17:49:56.364 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:49:55 smithi144 bash[66654]: cluster 2024-04-07T17:49:54.875414+0000 mon.a (mon.0) 4708 : cluster [DBG] osdmap e2681: 8 total, 8 up, 8 in 2024-04-07T17:49:57.148 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestExclusiveLock::test_follower_flatten PASSED [ 74%] 2024-04-07T17:49:57.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:49:56 smithi100 bash[74942]: cluster 2024-04-07T17:49:55.893911+0000 mon.a (mon.0) 4709 : cluster [DBG] osdmap e2682: 8 total, 8 up, 8 in 2024-04-07T17:49:57.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:49:56 smithi100 bash[76202]: cluster 2024-04-07T17:49:55.893911+0000 mon.a (mon.0) 4709 : cluster [DBG] osdmap e2682: 8 total, 8 up, 8 in 2024-04-07T17:49:57.364 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:49:56 smithi144 bash[66654]: cluster 2024-04-07T17:49:55.893911+0000 mon.a (mon.0) 4709 : cluster [DBG] osdmap e2682: 8 total, 8 up, 8 in 2024-04-07T17:49:57.496 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestExclusiveLock::test_follower_resize PASSED [ 74%] 2024-04-07T17:49:58.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:49:57 smithi100 bash[74942]: cluster 2024-04-07T17:49:56.793078+0000 mgr.y (mgr.25175) 2837 : cluster [DBG] pgmap v5094: 137 pgs: 137 active+clean; 17 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 72 KiB/s rd, 1.0 MiB/s wr, 104 op/s 2024-04-07T17:49:58.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:49:57 smithi100 bash[74942]: cluster 2024-04-07T17:49:56.914136+0000 mon.a (mon.0) 4710 : cluster [DBG] osdmap e2683: 8 total, 8 up, 8 in 2024-04-07T17:49:58.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:49:57 smithi100 bash[76202]: cluster 2024-04-07T17:49:56.793078+0000 mgr.y (mgr.25175) 2837 : cluster [DBG] pgmap v5094: 137 pgs: 137 active+clean; 17 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 72 KiB/s rd, 1.0 MiB/s wr, 104 op/s 2024-04-07T17:49:58.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:49:57 smithi100 bash[76202]: cluster 2024-04-07T17:49:56.914136+0000 mon.a (mon.0) 4710 : cluster [DBG] osdmap e2683: 8 total, 8 up, 8 in 2024-04-07T17:49:58.364 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:49:57 smithi144 bash[66654]: cluster 2024-04-07T17:49:56.793078+0000 mgr.y (mgr.25175) 2837 : cluster [DBG] pgmap v5094: 137 pgs: 137 active+clean; 17 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 72 KiB/s rd, 1.0 MiB/s wr, 104 op/s 2024-04-07T17:49:58.364 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:49:57 smithi144 bash[66654]: cluster 2024-04-07T17:49:56.914136+0000 mon.a (mon.0) 4710 : cluster [DBG] osdmap e2683: 8 total, 8 up, 8 in 2024-04-07T17:49:58.703 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:49:58 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:49:58] "GET /metrics HTTP/1.1" 200 36740 "" "Prometheus/2.43.0" 2024-04-07T17:49:59.161 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestExclusiveLock::test_follower_snap_create PASSED [ 75%] 2024-04-07T17:49:59.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:49:58 smithi100 bash[74942]: cluster 2024-04-07T17:49:57.915821+0000 mon.a (mon.0) 4711 : cluster [DBG] osdmap e2684: 8 total, 8 up, 8 in 2024-04-07T17:49:59.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:49:58 smithi100 bash[76202]: cluster 2024-04-07T17:49:57.915821+0000 mon.a (mon.0) 4711 : cluster [DBG] osdmap e2684: 8 total, 8 up, 8 in 2024-04-07T17:49:59.364 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:49:58 smithi144 bash[66654]: cluster 2024-04-07T17:49:57.915821+0000 mon.a (mon.0) 4711 : cluster [DBG] osdmap e2684: 8 total, 8 up, 8 in 2024-04-07T17:50:00.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:49:59 smithi100 bash[74942]: cluster 2024-04-07T17:49:58.794055+0000 mgr.y (mgr.25175) 2838 : cluster [DBG] pgmap v5097: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 210 KiB/s rd, 14 KiB/s wr, 293 op/s 2024-04-07T17:50:00.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:49:59 smithi100 bash[74942]: cluster 2024-04-07T17:49:58.939795+0000 mon.a (mon.0) 4712 : cluster [DBG] osdmap e2685: 8 total, 8 up, 8 in 2024-04-07T17:50:00.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:49:59 smithi100 bash[76202]: cluster 2024-04-07T17:49:58.794055+0000 mgr.y (mgr.25175) 2838 : cluster [DBG] pgmap v5097: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 210 KiB/s rd, 14 KiB/s wr, 293 op/s 2024-04-07T17:50:00.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:49:59 smithi100 bash[76202]: cluster 2024-04-07T17:49:58.939795+0000 mon.a (mon.0) 4712 : cluster [DBG] osdmap e2685: 8 total, 8 up, 8 in 2024-04-07T17:50:00.364 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:49:59 smithi144 bash[66654]: cluster 2024-04-07T17:49:58.794055+0000 mgr.y (mgr.25175) 2838 : cluster [DBG] pgmap v5097: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 210 KiB/s rd, 14 KiB/s wr, 293 op/s 2024-04-07T17:50:00.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:49:59 smithi144 bash[66654]: cluster 2024-04-07T17:49:58.939795+0000 mon.a (mon.0) 4712 : cluster [DBG] osdmap e2685: 8 total, 8 up, 8 in 2024-04-07T17:50:01.190 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestExclusiveLock::test_follower_snap_rollback PASSED [ 76%] 2024-04-07T17:50:01.364 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:50:00 smithi144 bash[66654]: cluster 2024-04-07T17:49:59.939490+0000 mon.a (mon.0) 4713 : cluster [DBG] osdmap e2686: 8 total, 8 up, 8 in 2024-04-07T17:50:01.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:50:00 smithi144 bash[66654]: cluster 2024-04-07T17:50:00.000130+0000 mon.a (mon.0) 4714 : cluster [WRN] overall HEALTH_WARN Telemetry requires re-opt-in 2024-04-07T17:50:01.394 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestExclusiveLock::test_follower_discard PASSED [ 76%] 2024-04-07T17:50:01.453 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:50:00 smithi100 bash[74942]: cluster 2024-04-07T17:49:59.939490+0000 mon.a (mon.0) 4713 : cluster [DBG] osdmap e2686: 8 total, 8 up, 8 in 2024-04-07T17:50:01.453 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:50:00 smithi100 bash[74942]: cluster 2024-04-07T17:50:00.000130+0000 mon.a (mon.0) 4714 : cluster [WRN] overall HEALTH_WARN Telemetry requires re-opt-in 2024-04-07T17:50:01.453 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:50:00 smithi100 bash[76202]: cluster 2024-04-07T17:49:59.939490+0000 mon.a (mon.0) 4713 : cluster [DBG] osdmap e2686: 8 total, 8 up, 8 in 2024-04-07T17:50:01.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:50:00 smithi100 bash[76202]: cluster 2024-04-07T17:50:00.000130+0000 mon.a (mon.0) 4714 : cluster [WRN] overall HEALTH_WARN Telemetry requires re-opt-in 2024-04-07T17:50:01.623 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestExclusiveLock::test_follower_write PASSED [ 77%] 2024-04-07T17:50:01.799 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestExclusiveLock::test_acquire_release_lock PASSED [ 77%] 2024-04-07T17:50:02.215 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestExclusiveLock::test_break_lock PASSED [ 78%] 2024-04-07T17:50:02.316 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:50:01 smithi144 bash[66654]: cluster 2024-04-07T17:50:00.795013+0000 mgr.y (mgr.25175) 2839 : cluster [DBG] pgmap v5100: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 335 KiB/s rd, 24 KiB/s wr, 459 op/s 2024-04-07T17:50:02.316 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:50:01 smithi144 bash[66654]: cluster 2024-04-07T17:50:00.969991+0000 mon.a (mon.0) 4715 : cluster [DBG] osdmap e2687: 8 total, 8 up, 8 in 2024-04-07T17:50:02.316 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:50:01 smithi144 bash[66654]: audit 2024-04-07T17:50:01.886553+0000 mon.a (mon.0) 4716 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:50:02.316 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:50:01 smithi144 bash[66654]: audit 2024-04-07T17:50:01.905154+0000 mon.c (mon.2) 105 : audit [INF] from='client.? 172.21.15.100:0/332012019' entity='client.admin' cmd=[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.100:0/200721231"}]: dispatch 2024-04-07T17:50:02.316 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:50:01 smithi144 bash[66654]: audit 2024-04-07T17:50:01.905623+0000 mon.a (mon.0) 4717 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.100:0/200721231"}]: dispatch 2024-04-07T17:50:02.406 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestMirroring::test_uuid PASSED [ 78%] 2024-04-07T17:50:02.453 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:50:01 smithi100 bash[74942]: cluster 2024-04-07T17:50:00.795013+0000 mgr.y (mgr.25175) 2839 : cluster [DBG] pgmap v5100: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 335 KiB/s rd, 24 KiB/s wr, 459 op/s 2024-04-07T17:50:02.453 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:50:01 smithi100 bash[74942]: cluster 2024-04-07T17:50:00.969991+0000 mon.a (mon.0) 4715 : cluster [DBG] osdmap e2687: 8 total, 8 up, 8 in 2024-04-07T17:50:02.453 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:50:01 smithi100 bash[74942]: audit 2024-04-07T17:50:01.886553+0000 mon.a (mon.0) 4716 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:50:02.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:50:01 smithi100 bash[74942]: audit 2024-04-07T17:50:01.905154+0000 mon.c (mon.2) 105 : audit [INF] from='client.? 172.21.15.100:0/332012019' entity='client.admin' cmd=[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.100:0/200721231"}]: dispatch 2024-04-07T17:50:02.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:50:01 smithi100 bash[74942]: audit 2024-04-07T17:50:01.905623+0000 mon.a (mon.0) 4717 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.100:0/200721231"}]: dispatch 2024-04-07T17:50:02.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:50:01 smithi100 bash[76202]: cluster 2024-04-07T17:50:00.795013+0000 mgr.y (mgr.25175) 2839 : cluster [DBG] pgmap v5100: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 335 KiB/s rd, 24 KiB/s wr, 459 op/s 2024-04-07T17:50:02.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:50:01 smithi100 bash[76202]: cluster 2024-04-07T17:50:00.969991+0000 mon.a (mon.0) 4715 : cluster [DBG] osdmap e2687: 8 total, 8 up, 8 in 2024-04-07T17:50:02.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:50:01 smithi100 bash[76202]: audit 2024-04-07T17:50:01.886553+0000 mon.a (mon.0) 4716 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:50:02.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:50:01 smithi100 bash[76202]: audit 2024-04-07T17:50:01.905154+0000 mon.c (mon.2) 105 : audit [INF] from='client.? 172.21.15.100:0/332012019' entity='client.admin' cmd=[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.100:0/200721231"}]: dispatch 2024-04-07T17:50:02.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:50:01 smithi100 bash[76202]: audit 2024-04-07T17:50:01.905623+0000 mon.a (mon.0) 4717 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.100:0/200721231"}]: dispatch 2024-04-07T17:50:02.614 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:50:02 smithi144 bash[95233]: ts=2024-04-07T17:50:02.317Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:50:02.657 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestMirroring::test_site_name PASSED [ 79%] 2024-04-07T17:50:02.890 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestMirroring::test_mirror_peer_bootstrap PASSED [ 80%] 2024-04-07T17:50:03.120 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestMirroring::test_mirror_peer PASSED [ 80%] 2024-04-07T17:50:03.321 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestMirroring::test_mirror_image SKIPPED [ 81%] 2024-04-07T17:50:03.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:50:02 smithi144 bash[66654]: audit 2024-04-07T17:50:01.960907+0000 mon.a (mon.0) 4718 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.100:0/200721231"}]': finished 2024-04-07T17:50:03.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:50:02 smithi144 bash[66654]: cluster 2024-04-07T17:50:01.969909+0000 mon.a (mon.0) 4719 : cluster [DBG] osdmap e2688: 8 total, 8 up, 8 in 2024-04-07T17:50:03.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:50:02 smithi144 bash[66654]: audit 2024-04-07T17:50:02.464929+0000 mon.a (mon.0) 4720 : audit [INF] from='client.? ' entity='client.admin' 2024-04-07T17:50:03.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:50:02 smithi144 bash[66654]: audit 2024-04-07T17:50:02.466097+0000 mon.c (mon.2) 106 : audit [DBG] from='client.? 172.21.15.100:0/60836617' entity='client.admin' cmd=[{"prefix": "config-key get", "key": "rbd/mirror/site_name"}]: dispatch 2024-04-07T17:50:03.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:50:02 smithi144 bash[66654]: audit 2024-04-07T17:50:02.466724+0000 mon.c (mon.2) 107 : audit [INF] from='client.? 172.21.15.100:0/60836617' entity='client.admin' cmd=[{"prefix": "config-key rm", "key": "rbd/mirror/site_name"}]: dispatch 2024-04-07T17:50:03.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:50:02 smithi144 bash[66654]: audit 2024-04-07T17:50:02.467092+0000 mon.a (mon.0) 4721 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "config-key rm", "key": "rbd/mirror/site_name"}]: dispatch 2024-04-07T17:50:03.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:50:02 smithi144 bash[66654]: audit 2024-04-07T17:50:02.472939+0000 mon.a (mon.0) 4722 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "config-key rm", "key": "rbd/mirror/site_name"}]': finished 2024-04-07T17:50:03.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:50:02 smithi144 bash[66654]: audit 2024-04-07T17:50:02.473991+0000 mon.c (mon.2) 108 : audit [DBG] from='client.? 172.21.15.100:0/60836617' entity='client.admin' cmd=[{"prefix": "config-key get", "key": "rbd/mirror/site_name"}]: dispatch 2024-04-07T17:50:03.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:50:02 smithi144 bash[66654]: audit 2024-04-07T17:50:02.733620+0000 mon.c (mon.2) 109 : audit [DBG] from='client.? 172.21.15.100:0/60836617' entity='client.admin' cmd=[{"prefix": "config-key get", "key": "rbd/mirror/peer_client_id"}]: dispatch 2024-04-07T17:50:03.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:50:02 smithi144 bash[66654]: audit 2024-04-07T17:50:02.734320+0000 mon.c (mon.2) 110 : audit [INF] from='client.? 172.21.15.100:0/60836617' entity='client.admin' cmd=[{ "prefix": "auth get-or-create", "entity": "client.rbd-mirror-peer", "caps": [ "mon", "profile rbd-mirror-peer", "osd", "profile rbd"], "format": "json"}]: dispatch 2024-04-07T17:50:03.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:50:02 smithi144 bash[66654]: audit 2024-04-07T17:50:02.734950+0000 mon.a (mon.0) 4723 : audit [INF] from='client.? ' entity='client.admin' cmd=[{ "prefix": "auth get-or-create", "entity": "client.rbd-mirror-peer", "caps": [ "mon", "profile rbd-mirror-peer", "osd", "profile rbd"], "format": "json"}]: dispatch 2024-04-07T17:50:03.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:50:02 smithi144 bash[66654]: audit 2024-04-07T17:50:02.736251+0000 mon.c (mon.2) 111 : audit [DBG] from='client.? 172.21.15.100:0/60836617' entity='client.admin' cmd=[{"prefix": "config-key get", "key": "rbd/mirror/site_name"}]: dispatch 2024-04-07T17:50:03.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:50:02 smithi144 bash[66654]: audit 2024-04-07T17:50:02.950073+0000 mon.a (mon.0) 4724 : audit [INF] from='client.? ' entity='client.admin' 2024-04-07T17:50:03.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:50:02 smithi144 bash[66654]: audit 2024-04-07T17:50:02.951130+0000 mon.c (mon.2) 112 : audit [DBG] from='client.? 172.21.15.100:0/60836617' entity='client.admin' cmd=[{"prefix": "config-key get", "key": "rbd/mirror/peer/122/8e377bcd-183c-4043-b88a-8bea58d9a313"}]: dispatch 2024-04-07T17:50:03.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:50:02 smithi144 bash[66654]: audit 2024-04-07T17:50:02.951680+0000 mon.c (mon.2) 113 : audit [INF] from='client.? 172.21.15.100:0/60836617' entity='client.admin' cmd=[{"prefix": "config-key rm", "key": "rbd/mirror/peer/122/8e377bcd-183c-4043-b88a-8bea58d9a313"}]: dispatch 2024-04-07T17:50:03.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:50:02 smithi144 bash[66654]: audit 2024-04-07T17:50:02.951999+0000 mon.a (mon.0) 4725 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "config-key rm", "key": "rbd/mirror/peer/122/8e377bcd-183c-4043-b88a-8bea58d9a313"}]: dispatch 2024-04-07T17:50:03.366 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:50:02 smithi144 bash[66654]: audit 2024-04-07T17:50:02.958040+0000 mon.a (mon.0) 4726 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "config-key rm", "key": "rbd/mirror/peer/122/8e377bcd-183c-4043-b88a-8bea58d9a313"}]': finished 2024-04-07T17:50:03.453 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:50:02 smithi100 bash[74942]: audit 2024-04-07T17:50:01.960907+0000 mon.a (mon.0) 4718 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.100:0/200721231"}]': finished 2024-04-07T17:50:03.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:50:02 smithi100 bash[74942]: cluster 2024-04-07T17:50:01.969909+0000 mon.a (mon.0) 4719 : cluster [DBG] osdmap e2688: 8 total, 8 up, 8 in 2024-04-07T17:50:03.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:50:02 smithi100 bash[74942]: audit 2024-04-07T17:50:02.464929+0000 mon.a (mon.0) 4720 : audit [INF] from='client.? ' entity='client.admin' 2024-04-07T17:50:03.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:50:02 smithi100 bash[74942]: audit 2024-04-07T17:50:02.466097+0000 mon.c (mon.2) 106 : audit [DBG] from='client.? 172.21.15.100:0/60836617' entity='client.admin' cmd=[{"prefix": "config-key get", "key": "rbd/mirror/site_name"}]: dispatch 2024-04-07T17:50:03.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:50:02 smithi100 bash[74942]: audit 2024-04-07T17:50:02.466724+0000 mon.c (mon.2) 107 : audit [INF] from='client.? 172.21.15.100:0/60836617' entity='client.admin' cmd=[{"prefix": "config-key rm", "key": "rbd/mirror/site_name"}]: dispatch 2024-04-07T17:50:03.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:50:02 smithi100 bash[74942]: audit 2024-04-07T17:50:02.467092+0000 mon.a (mon.0) 4721 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "config-key rm", "key": "rbd/mirror/site_name"}]: dispatch 2024-04-07T17:50:03.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:50:02 smithi100 bash[74942]: audit 2024-04-07T17:50:02.472939+0000 mon.a (mon.0) 4722 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "config-key rm", "key": "rbd/mirror/site_name"}]': finished 2024-04-07T17:50:03.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:50:02 smithi100 bash[74942]: audit 2024-04-07T17:50:02.473991+0000 mon.c (mon.2) 108 : audit [DBG] from='client.? 172.21.15.100:0/60836617' entity='client.admin' cmd=[{"prefix": "config-key get", "key": "rbd/mirror/site_name"}]: dispatch 2024-04-07T17:50:03.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:50:02 smithi100 bash[74942]: audit 2024-04-07T17:50:02.733620+0000 mon.c (mon.2) 109 : audit [DBG] from='client.? 172.21.15.100:0/60836617' entity='client.admin' cmd=[{"prefix": "config-key get", "key": "rbd/mirror/peer_client_id"}]: dispatch 2024-04-07T17:50:03.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:50:02 smithi100 bash[74942]: audit 2024-04-07T17:50:02.734320+0000 mon.c (mon.2) 110 : audit [INF] from='client.? 172.21.15.100:0/60836617' entity='client.admin' cmd=[{ "prefix": "auth get-or-create", "entity": "client.rbd-mirror-peer", "caps": [ "mon", "profile rbd-mirror-peer", "osd", "profile rbd"], "format": "json"}]: dispatch 2024-04-07T17:50:03.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:50:02 smithi100 bash[74942]: audit 2024-04-07T17:50:02.734950+0000 mon.a (mon.0) 4723 : audit [INF] from='client.? ' entity='client.admin' cmd=[{ "prefix": "auth get-or-create", "entity": "client.rbd-mirror-peer", "caps": [ "mon", "profile rbd-mirror-peer", "osd", "profile rbd"], "format": "json"}]: dispatch 2024-04-07T17:50:03.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:50:02 smithi100 bash[74942]: audit 2024-04-07T17:50:02.736251+0000 mon.c (mon.2) 111 : audit [DBG] from='client.? 172.21.15.100:0/60836617' entity='client.admin' cmd=[{"prefix": "config-key get", "key": "rbd/mirror/site_name"}]: dispatch 2024-04-07T17:50:03.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:50:02 smithi100 bash[74942]: audit 2024-04-07T17:50:02.950073+0000 mon.a (mon.0) 4724 : audit [INF] from='client.? ' entity='client.admin' 2024-04-07T17:50:03.455 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:50:02 smithi100 bash[74942]: audit 2024-04-07T17:50:02.951130+0000 mon.c (mon.2) 112 : audit [DBG] from='client.? 172.21.15.100:0/60836617' entity='client.admin' cmd=[{"prefix": "config-key get", "key": "rbd/mirror/peer/122/8e377bcd-183c-4043-b88a-8bea58d9a313"}]: dispatch 2024-04-07T17:50:03.455 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:50:02 smithi100 bash[74942]: audit 2024-04-07T17:50:02.951680+0000 mon.c (mon.2) 113 : audit [INF] from='client.? 172.21.15.100:0/60836617' entity='client.admin' cmd=[{"prefix": "config-key rm", "key": "rbd/mirror/peer/122/8e377bcd-183c-4043-b88a-8bea58d9a313"}]: dispatch 2024-04-07T17:50:03.455 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:50:02 smithi100 bash[74942]: audit 2024-04-07T17:50:02.951999+0000 mon.a (mon.0) 4725 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "config-key rm", "key": "rbd/mirror/peer/122/8e377bcd-183c-4043-b88a-8bea58d9a313"}]: dispatch 2024-04-07T17:50:03.455 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:50:02 smithi100 bash[74942]: audit 2024-04-07T17:50:02.958040+0000 mon.a (mon.0) 4726 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "config-key rm", "key": "rbd/mirror/peer/122/8e377bcd-183c-4043-b88a-8bea58d9a313"}]': finished 2024-04-07T17:50:03.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:50:02 smithi100 bash[76202]: audit 2024-04-07T17:50:01.960907+0000 mon.a (mon.0) 4718 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.100:0/200721231"}]': finished 2024-04-07T17:50:03.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:50:02 smithi100 bash[76202]: cluster 2024-04-07T17:50:01.969909+0000 mon.a (mon.0) 4719 : cluster [DBG] osdmap e2688: 8 total, 8 up, 8 in 2024-04-07T17:50:03.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:50:02 smithi100 bash[76202]: audit 2024-04-07T17:50:02.464929+0000 mon.a (mon.0) 4720 : audit [INF] from='client.? ' entity='client.admin' 2024-04-07T17:50:03.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:50:02 smithi100 bash[76202]: audit 2024-04-07T17:50:02.466097+0000 mon.c (mon.2) 106 : audit [DBG] from='client.? 172.21.15.100:0/60836617' entity='client.admin' cmd=[{"prefix": "config-key get", "key": "rbd/mirror/site_name"}]: dispatch 2024-04-07T17:50:03.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:50:02 smithi100 bash[76202]: audit 2024-04-07T17:50:02.466724+0000 mon.c (mon.2) 107 : audit [INF] from='client.? 172.21.15.100:0/60836617' entity='client.admin' cmd=[{"prefix": "config-key rm", "key": "rbd/mirror/site_name"}]: dispatch 2024-04-07T17:50:03.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:50:02 smithi100 bash[76202]: audit 2024-04-07T17:50:02.467092+0000 mon.a (mon.0) 4721 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "config-key rm", "key": "rbd/mirror/site_name"}]: dispatch 2024-04-07T17:50:03.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:50:02 smithi100 bash[76202]: audit 2024-04-07T17:50:02.472939+0000 mon.a (mon.0) 4722 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "config-key rm", "key": "rbd/mirror/site_name"}]': finished 2024-04-07T17:50:03.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:50:02 smithi100 bash[76202]: audit 2024-04-07T17:50:02.473991+0000 mon.c (mon.2) 108 : audit [DBG] from='client.? 172.21.15.100:0/60836617' entity='client.admin' cmd=[{"prefix": "config-key get", "key": "rbd/mirror/site_name"}]: dispatch 2024-04-07T17:50:03.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:50:02 smithi100 bash[76202]: audit 2024-04-07T17:50:02.733620+0000 mon.c (mon.2) 109 : audit [DBG] from='client.? 172.21.15.100:0/60836617' entity='client.admin' cmd=[{"prefix": "config-key get", "key": "rbd/mirror/peer_client_id"}]: dispatch 2024-04-07T17:50:03.456 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:50:02 smithi100 bash[76202]: audit 2024-04-07T17:50:02.734320+0000 mon.c (mon.2) 110 : audit [INF] from='client.? 172.21.15.100:0/60836617' entity='client.admin' cmd=[{ "prefix": "auth get-or-create", "entity": "client.rbd-mirror-peer", "caps": [ "mon", "profile rbd-mirror-peer", "osd", "profile rbd"], "format": "json"}]: dispatch 2024-04-07T17:50:03.456 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:50:02 smithi100 bash[76202]: audit 2024-04-07T17:50:02.734950+0000 mon.a (mon.0) 4723 : audit [INF] from='client.? ' entity='client.admin' cmd=[{ "prefix": "auth get-or-create", "entity": "client.rbd-mirror-peer", "caps": [ "mon", "profile rbd-mirror-peer", "osd", "profile rbd"], "format": "json"}]: dispatch 2024-04-07T17:50:03.456 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:50:02 smithi100 bash[76202]: audit 2024-04-07T17:50:02.736251+0000 mon.c (mon.2) 111 : audit [DBG] from='client.? 172.21.15.100:0/60836617' entity='client.admin' cmd=[{"prefix": "config-key get", "key": "rbd/mirror/site_name"}]: dispatch 2024-04-07T17:50:03.456 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:50:02 smithi100 bash[76202]: audit 2024-04-07T17:50:02.950073+0000 mon.a (mon.0) 4724 : audit [INF] from='client.? ' entity='client.admin' 2024-04-07T17:50:03.456 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:50:02 smithi100 bash[76202]: audit 2024-04-07T17:50:02.951130+0000 mon.c (mon.2) 112 : audit [DBG] from='client.? 172.21.15.100:0/60836617' entity='client.admin' cmd=[{"prefix": "config-key get", "key": "rbd/mirror/peer/122/8e377bcd-183c-4043-b88a-8bea58d9a313"}]: dispatch 2024-04-07T17:50:03.456 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:50:02 smithi100 bash[76202]: audit 2024-04-07T17:50:02.951680+0000 mon.c (mon.2) 113 : audit [INF] from='client.? 172.21.15.100:0/60836617' entity='client.admin' cmd=[{"prefix": "config-key rm", "key": "rbd/mirror/peer/122/8e377bcd-183c-4043-b88a-8bea58d9a313"}]: dispatch 2024-04-07T17:50:03.456 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:50:02 smithi100 bash[76202]: audit 2024-04-07T17:50:02.951999+0000 mon.a (mon.0) 4725 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "config-key rm", "key": "rbd/mirror/peer/122/8e377bcd-183c-4043-b88a-8bea58d9a313"}]: dispatch 2024-04-07T17:50:03.456 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:50:02 smithi100 bash[76202]: audit 2024-04-07T17:50:02.958040+0000 mon.a (mon.0) 4726 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "config-key rm", "key": "rbd/mirror/peer/122/8e377bcd-183c-4043-b88a-8bea58d9a313"}]': finished 2024-04-07T17:50:03.573 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestMirroring::test_mirror_image_status SKIPPED [ 81%] 2024-04-07T17:50:04.364 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:50:03 smithi144 bash[66654]: cluster 2024-04-07T17:50:02.795793+0000 mgr.y (mgr.25175) 2840 : cluster [DBG] pgmap v5103: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 328 KiB/s rd, 20 KiB/s wr, 442 op/s 2024-04-07T17:50:04.364 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:50:03 smithi144 bash[66654]: cluster 2024-04-07T17:50:03.551117+0000 mon.a (mon.0) 4727 : cluster [DBG] osdmap e2689: 8 total, 8 up, 8 in 2024-04-07T17:50:04.453 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:50:03 smithi100 bash[74942]: cluster 2024-04-07T17:50:02.795793+0000 mgr.y (mgr.25175) 2840 : cluster [DBG] pgmap v5103: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 328 KiB/s rd, 20 KiB/s wr, 442 op/s 2024-04-07T17:50:04.453 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:50:03 smithi100 bash[74942]: cluster 2024-04-07T17:50:03.551117+0000 mon.a (mon.0) 4727 : cluster [DBG] osdmap e2689: 8 total, 8 up, 8 in 2024-04-07T17:50:04.453 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:50:03 smithi100 bash[76202]: cluster 2024-04-07T17:50:02.795793+0000 mgr.y (mgr.25175) 2840 : cluster [DBG] pgmap v5103: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 328 KiB/s rd, 20 KiB/s wr, 442 op/s 2024-04-07T17:50:04.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:50:03 smithi100 bash[76202]: cluster 2024-04-07T17:50:03.551117+0000 mon.a (mon.0) 4727 : cluster [DBG] osdmap e2689: 8 total, 8 up, 8 in 2024-04-07T17:50:05.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:50:05 smithi144 bash[66654]: cluster 2024-04-07T17:50:04.550473+0000 mon.a (mon.0) 4728 : cluster [DBG] osdmap e2690: 8 total, 8 up, 8 in 2024-04-07T17:50:05.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:50:05 smithi144 bash[66654]: cluster 2024-04-07T17:50:04.796509+0000 mgr.y (mgr.25175) 2841 : cluster [DBG] pgmap v5106: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 333 KiB/s rd, 23 KiB/s wr, 463 op/s 2024-04-07T17:50:05.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:50:05 smithi100 bash[74942]: cluster 2024-04-07T17:50:04.550473+0000 mon.a (mon.0) 4728 : cluster [DBG] osdmap e2690: 8 total, 8 up, 8 in 2024-04-07T17:50:05.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:50:05 smithi100 bash[74942]: cluster 2024-04-07T17:50:04.796509+0000 mgr.y (mgr.25175) 2841 : cluster [DBG] pgmap v5106: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 333 KiB/s rd, 23 KiB/s wr, 463 op/s 2024-04-07T17:50:05.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:50:05 smithi100 bash[76202]: cluster 2024-04-07T17:50:04.550473+0000 mon.a (mon.0) 4728 : cluster [DBG] osdmap e2690: 8 total, 8 up, 8 in 2024-04-07T17:50:05.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:50:05 smithi100 bash[76202]: cluster 2024-04-07T17:50:04.796509+0000 mgr.y (mgr.25175) 2841 : cluster [DBG] pgmap v5106: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 333 KiB/s rd, 23 KiB/s wr, 463 op/s 2024-04-07T17:50:06.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:50:06 smithi144 bash[66654]: cluster 2024-04-07T17:50:05.563001+0000 mon.a (mon.0) 4729 : cluster [DBG] osdmap e2691: 8 total, 8 up, 8 in 2024-04-07T17:50:06.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:50:06 smithi100 bash[74942]: cluster 2024-04-07T17:50:05.563001+0000 mon.a (mon.0) 4729 : cluster [DBG] osdmap e2691: 8 total, 8 up, 8 in 2024-04-07T17:50:06.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:50:06 smithi100 bash[76202]: cluster 2024-04-07T17:50:05.563001+0000 mon.a (mon.0) 4729 : cluster [DBG] osdmap e2691: 8 total, 8 up, 8 in 2024-04-07T17:50:07.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:50:07 smithi144 bash[66654]: cluster 2024-04-07T17:50:06.567346+0000 mon.a (mon.0) 4730 : cluster [DBG] osdmap e2692: 8 total, 8 up, 8 in 2024-04-07T17:50:07.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:50:07 smithi144 bash[66654]: audit 2024-04-07T17:50:06.633895+0000 mon.c (mon.2) 114 : audit [INF] from='client.? 172.21.15.100:0/60836617' entity='client.admin' cmd=[{"prefix": "config-key rm", "key": "rbd/mirror/peer/122/0d41eeb4-94de-472f-9a0a-eafc16604453"}]: dispatch 2024-04-07T17:50:07.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:50:07 smithi144 bash[66654]: audit 2024-04-07T17:50:06.634698+0000 mon.a (mon.0) 4731 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "config-key rm", "key": "rbd/mirror/peer/122/0d41eeb4-94de-472f-9a0a-eafc16604453"}]: dispatch 2024-04-07T17:50:07.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:50:07 smithi144 bash[66654]: audit 2024-04-07T17:50:06.641523+0000 mon.a (mon.0) 4732 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "config-key rm", "key": "rbd/mirror/peer/122/0d41eeb4-94de-472f-9a0a-eafc16604453"}]': finished 2024-04-07T17:50:07.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:50:07 smithi144 bash[66654]: audit 2024-04-07T17:50:06.723323+0000 mon.c (mon.2) 115 : audit [INF] from='client.? 172.21.15.100:0/60836617' entity='client.admin' cmd=[{"prefix": "config-key rm", "key": "rbd/mirror/peer/122/6e925d05-4757-474c-a0cc-bc3a19604d0b"}]: dispatch 2024-04-07T17:50:07.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:50:07 smithi144 bash[66654]: audit 2024-04-07T17:50:06.724023+0000 mon.a (mon.0) 4733 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "config-key rm", "key": "rbd/mirror/peer/122/6e925d05-4757-474c-a0cc-bc3a19604d0b"}]: dispatch 2024-04-07T17:50:07.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:50:07 smithi144 bash[66654]: audit 2024-04-07T17:50:06.730524+0000 mon.a (mon.0) 4734 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "config-key rm", "key": "rbd/mirror/peer/122/6e925d05-4757-474c-a0cc-bc3a19604d0b"}]': finished 2024-04-07T17:50:07.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:50:07 smithi144 bash[66654]: cluster 2024-04-07T17:50:06.797271+0000 mgr.y (mgr.25175) 2842 : cluster [DBG] pgmap v5109: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 160 KiB/s rd, 14 KiB/s wr, 230 op/s 2024-04-07T17:50:07.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:50:07 smithi100 bash[74942]: cluster 2024-04-07T17:50:06.567346+0000 mon.a (mon.0) 4730 : cluster [DBG] osdmap e2692: 8 total, 8 up, 8 in 2024-04-07T17:50:07.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:50:07 smithi100 bash[74942]: audit 2024-04-07T17:50:06.633895+0000 mon.c (mon.2) 114 : audit [INF] from='client.? 172.21.15.100:0/60836617' entity='client.admin' cmd=[{"prefix": "config-key rm", "key": "rbd/mirror/peer/122/0d41eeb4-94de-472f-9a0a-eafc16604453"}]: dispatch 2024-04-07T17:50:07.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:50:07 smithi100 bash[74942]: audit 2024-04-07T17:50:06.634698+0000 mon.a (mon.0) 4731 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "config-key rm", "key": "rbd/mirror/peer/122/0d41eeb4-94de-472f-9a0a-eafc16604453"}]: dispatch 2024-04-07T17:50:07.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:50:07 smithi100 bash[74942]: audit 2024-04-07T17:50:06.641523+0000 mon.a (mon.0) 4732 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "config-key rm", "key": "rbd/mirror/peer/122/0d41eeb4-94de-472f-9a0a-eafc16604453"}]': finished 2024-04-07T17:50:07.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:50:07 smithi100 bash[74942]: audit 2024-04-07T17:50:06.723323+0000 mon.c (mon.2) 115 : audit [INF] from='client.? 172.21.15.100:0/60836617' entity='client.admin' cmd=[{"prefix": "config-key rm", "key": "rbd/mirror/peer/122/6e925d05-4757-474c-a0cc-bc3a19604d0b"}]: dispatch 2024-04-07T17:50:07.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:50:07 smithi100 bash[74942]: audit 2024-04-07T17:50:06.724023+0000 mon.a (mon.0) 4733 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "config-key rm", "key": "rbd/mirror/peer/122/6e925d05-4757-474c-a0cc-bc3a19604d0b"}]: dispatch 2024-04-07T17:50:07.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:50:07 smithi100 bash[74942]: audit 2024-04-07T17:50:06.730524+0000 mon.a (mon.0) 4734 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "config-key rm", "key": "rbd/mirror/peer/122/6e925d05-4757-474c-a0cc-bc3a19604d0b"}]': finished 2024-04-07T17:50:07.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:50:07 smithi100 bash[74942]: cluster 2024-04-07T17:50:06.797271+0000 mgr.y (mgr.25175) 2842 : cluster [DBG] pgmap v5109: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 160 KiB/s rd, 14 KiB/s wr, 230 op/s 2024-04-07T17:50:07.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:50:07 smithi100 bash[76202]: cluster 2024-04-07T17:50:06.567346+0000 mon.a (mon.0) 4730 : cluster [DBG] osdmap e2692: 8 total, 8 up, 8 in 2024-04-07T17:50:07.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:50:07 smithi100 bash[76202]: audit 2024-04-07T17:50:06.633895+0000 mon.c (mon.2) 114 : audit [INF] from='client.? 172.21.15.100:0/60836617' entity='client.admin' cmd=[{"prefix": "config-key rm", "key": "rbd/mirror/peer/122/0d41eeb4-94de-472f-9a0a-eafc16604453"}]: dispatch 2024-04-07T17:50:07.955 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:50:07 smithi100 bash[76202]: audit 2024-04-07T17:50:06.634698+0000 mon.a (mon.0) 4731 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "config-key rm", "key": "rbd/mirror/peer/122/0d41eeb4-94de-472f-9a0a-eafc16604453"}]: dispatch 2024-04-07T17:50:07.955 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:50:07 smithi100 bash[76202]: audit 2024-04-07T17:50:06.641523+0000 mon.a (mon.0) 4732 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "config-key rm", "key": "rbd/mirror/peer/122/0d41eeb4-94de-472f-9a0a-eafc16604453"}]': finished 2024-04-07T17:50:07.955 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:50:07 smithi100 bash[76202]: audit 2024-04-07T17:50:06.723323+0000 mon.c (mon.2) 115 : audit [INF] from='client.? 172.21.15.100:0/60836617' entity='client.admin' cmd=[{"prefix": "config-key rm", "key": "rbd/mirror/peer/122/6e925d05-4757-474c-a0cc-bc3a19604d0b"}]: dispatch 2024-04-07T17:50:07.955 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:50:07 smithi100 bash[76202]: audit 2024-04-07T17:50:06.724023+0000 mon.a (mon.0) 4733 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "config-key rm", "key": "rbd/mirror/peer/122/6e925d05-4757-474c-a0cc-bc3a19604d0b"}]: dispatch 2024-04-07T17:50:07.955 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:50:07 smithi100 bash[76202]: audit 2024-04-07T17:50:06.730524+0000 mon.a (mon.0) 4734 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "config-key rm", "key": "rbd/mirror/peer/122/6e925d05-4757-474c-a0cc-bc3a19604d0b"}]': finished 2024-04-07T17:50:07.955 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:50:07 smithi100 bash[76202]: cluster 2024-04-07T17:50:06.797271+0000 mgr.y (mgr.25175) 2842 : cluster [DBG] pgmap v5109: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 160 KiB/s rd, 14 KiB/s wr, 230 op/s 2024-04-07T17:50:08.703 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:50:08 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:50:08] "GET /metrics HTTP/1.1" 200 36744 "" "Prometheus/2.43.0" 2024-04-07T17:50:09.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:50:08 smithi144 bash[66654]: cluster 2024-04-07T17:50:07.740889+0000 mon.a (mon.0) 4735 : cluster [DBG] osdmap e2693: 8 total, 8 up, 8 in 2024-04-07T17:50:09.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:50:08 smithi144 bash[66654]: cluster 2024-04-07T17:50:08.547247+0000 mon.a (mon.0) 4736 : cluster [DBG] osdmap e2694: 8 total, 8 up, 8 in 2024-04-07T17:50:09.206 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:50:08 smithi100 bash[74942]: cluster 2024-04-07T17:50:07.740889+0000 mon.a (mon.0) 4735 : cluster [DBG] osdmap e2693: 8 total, 8 up, 8 in 2024-04-07T17:50:09.206 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:50:08 smithi100 bash[74942]: cluster 2024-04-07T17:50:08.547247+0000 mon.a (mon.0) 4736 : cluster [DBG] osdmap e2694: 8 total, 8 up, 8 in 2024-04-07T17:50:09.206 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:50:08 smithi100 bash[76202]: cluster 2024-04-07T17:50:07.740889+0000 mon.a (mon.0) 4735 : cluster [DBG] osdmap e2693: 8 total, 8 up, 8 in 2024-04-07T17:50:09.206 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:50:08 smithi100 bash[76202]: cluster 2024-04-07T17:50:08.547247+0000 mon.a (mon.0) 4736 : cluster [DBG] osdmap e2694: 8 total, 8 up, 8 in 2024-04-07T17:50:10.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:50:09 smithi144 bash[66654]: cluster 2024-04-07T17:50:08.798020+0000 mgr.y (mgr.25175) 2843 : cluster [DBG] pgmap v5112: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 265 KiB/s rd, 19 KiB/s wr, 360 op/s 2024-04-07T17:50:10.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:50:09 smithi144 bash[66654]: cluster 2024-04-07T17:50:09.549134+0000 mon.a (mon.0) 4737 : cluster [DBG] osdmap e2695: 8 total, 8 up, 8 in 2024-04-07T17:50:10.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:50:09 smithi100 bash[74942]: cluster 2024-04-07T17:50:08.798020+0000 mgr.y (mgr.25175) 2843 : cluster [DBG] pgmap v5112: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 265 KiB/s rd, 19 KiB/s wr, 360 op/s 2024-04-07T17:50:10.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:50:09 smithi100 bash[74942]: cluster 2024-04-07T17:50:09.549134+0000 mon.a (mon.0) 4737 : cluster [DBG] osdmap e2695: 8 total, 8 up, 8 in 2024-04-07T17:50:10.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:50:09 smithi100 bash[76202]: cluster 2024-04-07T17:50:08.798020+0000 mgr.y (mgr.25175) 2843 : cluster [DBG] pgmap v5112: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 265 KiB/s rd, 19 KiB/s wr, 360 op/s 2024-04-07T17:50:10.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:50:09 smithi100 bash[76202]: cluster 2024-04-07T17:50:09.549134+0000 mon.a (mon.0) 4737 : cluster [DBG] osdmap e2695: 8 total, 8 up, 8 in 2024-04-07T17:50:11.714 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestMirroring::test_mirror_image_create_snapshot PASSED [ 82%] 2024-04-07T17:50:11.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:50:11 smithi144 bash[66654]: cluster 2024-04-07T17:50:10.552129+0000 mon.a (mon.0) 4738 : cluster [DBG] osdmap e2696: 8 total, 8 up, 8 in 2024-04-07T17:50:11.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:50:11 smithi144 bash[66654]: cluster 2024-04-07T17:50:10.798786+0000 mgr.y (mgr.25175) 2844 : cluster [DBG] pgmap v5115: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 242 KiB/s rd, 19 KiB/s wr, 326 op/s 2024-04-07T17:50:11.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:50:11 smithi100 bash[74942]: cluster 2024-04-07T17:50:10.552129+0000 mon.a (mon.0) 4738 : cluster [DBG] osdmap e2696: 8 total, 8 up, 8 in 2024-04-07T17:50:11.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:50:11 smithi100 bash[74942]: cluster 2024-04-07T17:50:10.798786+0000 mgr.y (mgr.25175) 2844 : cluster [DBG] pgmap v5115: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 242 KiB/s rd, 19 KiB/s wr, 326 op/s 2024-04-07T17:50:11.953 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:50:11 smithi100 bash[76202]: cluster 2024-04-07T17:50:10.552129+0000 mon.a (mon.0) 4738 : cluster [DBG] osdmap e2696: 8 total, 8 up, 8 in 2024-04-07T17:50:11.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:50:11 smithi100 bash[76202]: cluster 2024-04-07T17:50:10.798786+0000 mgr.y (mgr.25175) 2844 : cluster [DBG] pgmap v5115: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 242 KiB/s rd, 19 KiB/s wr, 326 op/s 2024-04-07T17:50:12.614 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:50:12 smithi144 bash[66654]: cluster 2024-04-07T17:50:11.568610+0000 mon.a (mon.0) 4739 : cluster [DBG] osdmap e2697: 8 total, 8 up, 8 in 2024-04-07T17:50:12.615 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:50:12 smithi144 bash[95233]: ts=2024-04-07T17:50:12.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:50:12.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:50:12 smithi100 bash[74942]: cluster 2024-04-07T17:50:11.568610+0000 mon.a (mon.0) 4739 : cluster [DBG] osdmap e2697: 8 total, 8 up, 8 in 2024-04-07T17:50:12.953 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:50:12 smithi100 bash[76202]: cluster 2024-04-07T17:50:11.568610+0000 mon.a (mon.0) 4739 : cluster [DBG] osdmap e2697: 8 total, 8 up, 8 in 2024-04-07T17:50:13.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:50:13 smithi144 bash[66654]: cluster 2024-04-07T17:50:12.581448+0000 mon.a (mon.0) 4740 : cluster [DBG] osdmap e2698: 8 total, 8 up, 8 in 2024-04-07T17:50:13.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:50:13 smithi144 bash[66654]: cluster 2024-04-07T17:50:12.799485+0000 mgr.y (mgr.25175) 2845 : cluster [DBG] pgmap v5118: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 77 KiB/s rd, 7.0 KiB/s wr, 106 op/s 2024-04-07T17:50:13.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:50:13 smithi100 bash[74942]: cluster 2024-04-07T17:50:12.581448+0000 mon.a (mon.0) 4740 : cluster [DBG] osdmap e2698: 8 total, 8 up, 8 in 2024-04-07T17:50:13.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:50:13 smithi100 bash[74942]: cluster 2024-04-07T17:50:12.799485+0000 mgr.y (mgr.25175) 2845 : cluster [DBG] pgmap v5118: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 77 KiB/s rd, 7.0 KiB/s wr, 106 op/s 2024-04-07T17:50:13.953 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:50:13 smithi100 bash[76202]: cluster 2024-04-07T17:50:12.581448+0000 mon.a (mon.0) 4740 : cluster [DBG] osdmap e2698: 8 total, 8 up, 8 in 2024-04-07T17:50:13.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:50:13 smithi100 bash[76202]: cluster 2024-04-07T17:50:12.799485+0000 mgr.y (mgr.25175) 2845 : cluster [DBG] pgmap v5118: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 77 KiB/s rd, 7.0 KiB/s wr, 106 op/s 2024-04-07T17:50:14.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:50:14 smithi144 bash[66654]: cluster 2024-04-07T17:50:13.595008+0000 mon.a (mon.0) 4741 : cluster [DBG] osdmap e2699: 8 total, 8 up, 8 in 2024-04-07T17:50:14.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:50:14 smithi144 bash[66654]: audit 2024-04-07T17:50:13.639242+0000 mon.c (mon.2) 116 : audit [INF] from='client.? 172.21.15.100:0/60836617' entity='client.admin' cmd=[{"prefix": "config-key rm", "key": "rbd/mirror/peer/122/6af23cb7-3bdc-46ed-acf3-b3832653cd81"}]: dispatch 2024-04-07T17:50:14.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:50:14 smithi144 bash[66654]: audit 2024-04-07T17:50:13.639995+0000 mon.a (mon.0) 4742 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "config-key rm", "key": "rbd/mirror/peer/122/6af23cb7-3bdc-46ed-acf3-b3832653cd81"}]: dispatch 2024-04-07T17:50:14.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:50:14 smithi144 bash[66654]: audit 2024-04-07T17:50:13.648718+0000 mon.a (mon.0) 4743 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "config-key rm", "key": "rbd/mirror/peer/122/6af23cb7-3bdc-46ed-acf3-b3832653cd81"}]': finished 2024-04-07T17:50:14.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:50:14 smithi100 bash[74942]: cluster 2024-04-07T17:50:13.595008+0000 mon.a (mon.0) 4741 : cluster [DBG] osdmap e2699: 8 total, 8 up, 8 in 2024-04-07T17:50:14.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:50:14 smithi100 bash[74942]: audit 2024-04-07T17:50:13.639242+0000 mon.c (mon.2) 116 : audit [INF] from='client.? 172.21.15.100:0/60836617' entity='client.admin' cmd=[{"prefix": "config-key rm", "key": "rbd/mirror/peer/122/6af23cb7-3bdc-46ed-acf3-b3832653cd81"}]: dispatch 2024-04-07T17:50:14.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:50:14 smithi100 bash[74942]: audit 2024-04-07T17:50:13.639995+0000 mon.a (mon.0) 4742 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "config-key rm", "key": "rbd/mirror/peer/122/6af23cb7-3bdc-46ed-acf3-b3832653cd81"}]: dispatch 2024-04-07T17:50:14.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:50:14 smithi100 bash[74942]: audit 2024-04-07T17:50:13.648718+0000 mon.a (mon.0) 4743 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "config-key rm", "key": "rbd/mirror/peer/122/6af23cb7-3bdc-46ed-acf3-b3832653cd81"}]': finished 2024-04-07T17:50:14.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:50:14 smithi100 bash[76202]: cluster 2024-04-07T17:50:13.595008+0000 mon.a (mon.0) 4741 : cluster [DBG] osdmap e2699: 8 total, 8 up, 8 in 2024-04-07T17:50:14.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:50:14 smithi100 bash[76202]: audit 2024-04-07T17:50:13.639242+0000 mon.c (mon.2) 116 : audit [INF] from='client.? 172.21.15.100:0/60836617' entity='client.admin' cmd=[{"prefix": "config-key rm", "key": "rbd/mirror/peer/122/6af23cb7-3bdc-46ed-acf3-b3832653cd81"}]: dispatch 2024-04-07T17:50:14.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:50:14 smithi100 bash[76202]: audit 2024-04-07T17:50:13.639995+0000 mon.a (mon.0) 4742 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "config-key rm", "key": "rbd/mirror/peer/122/6af23cb7-3bdc-46ed-acf3-b3832653cd81"}]: dispatch 2024-04-07T17:50:14.954 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:50:14 smithi100 bash[76202]: audit 2024-04-07T17:50:13.648718+0000 mon.a (mon.0) 4743 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "config-key rm", "key": "rbd/mirror/peer/122/6af23cb7-3bdc-46ed-acf3-b3832653cd81"}]': finished 2024-04-07T17:50:15.824 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestMirroring::test_aio_mirror_image_create_snapshot PASSED [ 82%] 2024-04-07T17:50:15.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:50:15 smithi100 bash[74942]: cluster 2024-04-07T17:50:14.653909+0000 mon.a (mon.0) 4744 : cluster [DBG] osdmap e2700: 8 total, 8 up, 8 in 2024-04-07T17:50:15.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:50:15 smithi100 bash[74942]: cluster 2024-04-07T17:50:14.800217+0000 mgr.y (mgr.25175) 2846 : cluster [DBG] pgmap v5121: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 111 KiB/s rd, 11 KiB/s wr, 153 op/s 2024-04-07T17:50:15.953 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:50:15 smithi100 bash[76202]: cluster 2024-04-07T17:50:14.653909+0000 mon.a (mon.0) 4744 : cluster [DBG] osdmap e2700: 8 total, 8 up, 8 in 2024-04-07T17:50:15.953 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:50:15 smithi100 bash[76202]: cluster 2024-04-07T17:50:14.800217+0000 mgr.y (mgr.25175) 2846 : cluster [DBG] pgmap v5121: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 111 KiB/s rd, 11 KiB/s wr, 153 op/s 2024-04-07T17:50:15.998 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestTrash::test_move PASSED [ 83%] 2024-04-07T17:50:16.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:50:15 smithi144 bash[66654]: cluster 2024-04-07T17:50:14.653909+0000 mon.a (mon.0) 4744 : cluster [DBG] osdmap e2700: 8 total, 8 up, 8 in 2024-04-07T17:50:16.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:50:15 smithi144 bash[66654]: cluster 2024-04-07T17:50:14.800217+0000 mgr.y (mgr.25175) 2846 : cluster [DBG] pgmap v5121: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 111 KiB/s rd, 11 KiB/s wr, 153 op/s 2024-04-07T17:50:16.305 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestTrash::test_purge PASSED [ 84%] 2024-04-07T17:50:16.481 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestTrash::test_remove_denied PASSED [ 84%] 2024-04-07T17:50:16.612 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestTrash::test_remove PASSED [ 85%] 2024-04-07T17:50:16.772 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestTrash::test_remove_with_progress PASSED [ 85%] 2024-04-07T17:50:16.928 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestTrash::test_get PASSED [ 86%] 2024-04-07T17:50:16.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:50:16 smithi100 bash[74942]: cluster 2024-04-07T17:50:15.664236+0000 mon.a (mon.0) 4745 : cluster [DBG] osdmap e2701: 8 total, 8 up, 8 in 2024-04-07T17:50:16.953 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:50:16 smithi100 bash[76202]: cluster 2024-04-07T17:50:15.664236+0000 mon.a (mon.0) 4745 : cluster [DBG] osdmap e2701: 8 total, 8 up, 8 in 2024-04-07T17:50:17.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:50:16 smithi144 bash[66654]: cluster 2024-04-07T17:50:15.664236+0000 mon.a (mon.0) 4745 : cluster [DBG] osdmap e2701: 8 total, 8 up, 8 in 2024-04-07T17:50:17.259 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestTrash::test_list PASSED [ 86%] 2024-04-07T17:50:17.541 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestTrash::test_restore PASSED [ 87%] 2024-04-07T17:50:17.558 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::test_create_group PASSED [ 88%] 2024-04-07T17:50:17.582 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::test_rename_group PASSED [ 88%] 2024-04-07T17:50:17.586 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::test_list_groups_empty PASSED [ 89%] 2024-04-07T17:50:17.604 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::test_list_groups PASSED [ 89%] 2024-04-07T17:50:17.623 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::test_list_groups_after_removed PASSED [ 90%] 2024-04-07T17:50:17.882 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestGroups::test_group_image_add PASSED [ 90%] 2024-04-07T17:50:17.953 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:50:17 smithi100 bash[76202]: cluster 2024-04-07T17:50:16.800735+0000 mgr.y (mgr.25175) 2847 : cluster [DBG] pgmap v5123: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 36 KiB/s rd, 4.0 KiB/s wr, 50 op/s 2024-04-07T17:50:17.953 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:50:17 smithi100 bash[76202]: audit 2024-04-07T17:50:16.886782+0000 mon.a (mon.0) 4746 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:50:17.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:50:17 smithi100 bash[74942]: cluster 2024-04-07T17:50:16.800735+0000 mgr.y (mgr.25175) 2847 : cluster [DBG] pgmap v5123: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 36 KiB/s rd, 4.0 KiB/s wr, 50 op/s 2024-04-07T17:50:17.954 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:50:17 smithi100 bash[74942]: audit 2024-04-07T17:50:16.886782+0000 mon.a (mon.0) 4746 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:50:18.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:50:17 smithi144 bash[66654]: cluster 2024-04-07T17:50:16.800735+0000 mgr.y (mgr.25175) 2847 : cluster [DBG] pgmap v5123: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 36 KiB/s rd, 4.0 KiB/s wr, 50 op/s 2024-04-07T17:50:18.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:50:17 smithi144 bash[66654]: audit 2024-04-07T17:50:16.886782+0000 mon.a (mon.0) 4746 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:50:18.128 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestGroups::test_group_image_list_empty PASSED [ 91%] 2024-04-07T17:50:18.413 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestGroups::test_group_image_list PASSED [ 92%] 2024-04-07T17:50:18.703 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:50:18 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:50:18] "GET /metrics HTTP/1.1" 200 36746 "" "Prometheus/2.43.0" 2024-04-07T17:50:18.810 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestGroups::test_group_image_list_move_to_trash PASSED [ 92%] 2024-04-07T17:50:19.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:50:19 smithi144 bash[66654]: cluster 2024-04-07T17:50:18.550453+0000 mon.a (mon.0) 4747 : cluster [DBG] osdmap e2702: 8 total, 8 up, 8 in 2024-04-07T17:50:19.865 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:50:19 smithi144 bash[66654]: cluster 2024-04-07T17:50:18.801396+0000 mgr.y (mgr.25175) 2848 : cluster [DBG] pgmap v5125: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 165 KiB/s rd, 9.8 KiB/s wr, 225 op/s 2024-04-07T17:50:19.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:50:19 smithi100 bash[74942]: cluster 2024-04-07T17:50:18.550453+0000 mon.a (mon.0) 4747 : cluster [DBG] osdmap e2702: 8 total, 8 up, 8 in 2024-04-07T17:50:19.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:50:19 smithi100 bash[74942]: cluster 2024-04-07T17:50:18.801396+0000 mgr.y (mgr.25175) 2848 : cluster [DBG] pgmap v5125: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 165 KiB/s rd, 9.8 KiB/s wr, 225 op/s 2024-04-07T17:50:19.953 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:50:19 smithi100 bash[76202]: cluster 2024-04-07T17:50:18.550453+0000 mon.a (mon.0) 4747 : cluster [DBG] osdmap e2702: 8 total, 8 up, 8 in 2024-04-07T17:50:19.953 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:50:19 smithi100 bash[76202]: cluster 2024-04-07T17:50:18.801396+0000 mgr.y (mgr.25175) 2848 : cluster [DBG] pgmap v5125: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 165 KiB/s rd, 9.8 KiB/s wr, 225 op/s 2024-04-07T17:50:22.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:50:21 smithi144 bash[66654]: cluster 2024-04-07T17:50:20.802800+0000 mgr.y (mgr.25175) 2849 : cluster [DBG] pgmap v5126: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 222 KiB/s rd, 25 KiB/s wr, 327 op/s 2024-04-07T17:50:22.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:50:21 smithi100 bash[74942]: cluster 2024-04-07T17:50:20.802800+0000 mgr.y (mgr.25175) 2849 : cluster [DBG] pgmap v5126: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 222 KiB/s rd, 25 KiB/s wr, 327 op/s 2024-04-07T17:50:22.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:50:21 smithi100 bash[76202]: cluster 2024-04-07T17:50:20.802800+0000 mgr.y (mgr.25175) 2849 : cluster [DBG] pgmap v5126: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 222 KiB/s rd, 25 KiB/s wr, 327 op/s 2024-04-07T17:50:22.614 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:50:22 smithi144 bash[95233]: ts=2024-04-07T17:50:22.317Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:50:23.386 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestGroups::test_group_image_many_images PASSED [ 93%] 2024-04-07T17:50:23.726 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestGroups::test_group_image_remove PASSED [ 93%] 2024-04-07T17:50:23.980 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestGroups::test_group_snap PASSED [ 94%] 2024-04-07T17:50:24.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:50:23 smithi144 bash[66654]: cluster 2024-04-07T17:50:22.804250+0000 mgr.y (mgr.25175) 2850 : cluster [DBG] pgmap v5127: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 325 KiB/s rd, 39 KiB/s wr, 481 op/s 2024-04-07T17:50:24.114 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:50:23 smithi144 bash[66654]: cluster 2024-04-07T17:50:23.555147+0000 mon.a (mon.0) 4748 : cluster [DBG] osdmap e2703: 8 total, 8 up, 8 in 2024-04-07T17:50:24.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:50:23 smithi100 bash[74942]: cluster 2024-04-07T17:50:22.804250+0000 mgr.y (mgr.25175) 2850 : cluster [DBG] pgmap v5127: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 325 KiB/s rd, 39 KiB/s wr, 481 op/s 2024-04-07T17:50:24.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:50:23 smithi100 bash[74942]: cluster 2024-04-07T17:50:23.555147+0000 mon.a (mon.0) 4748 : cluster [DBG] osdmap e2703: 8 total, 8 up, 8 in 2024-04-07T17:50:24.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:50:23 smithi100 bash[76202]: cluster 2024-04-07T17:50:22.804250+0000 mgr.y (mgr.25175) 2850 : cluster [DBG] pgmap v5127: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 325 KiB/s rd, 39 KiB/s wr, 481 op/s 2024-04-07T17:50:24.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:50:23 smithi100 bash[76202]: cluster 2024-04-07T17:50:23.555147+0000 mon.a (mon.0) 4748 : cluster [DBG] osdmap e2703: 8 total, 8 up, 8 in 2024-04-07T17:50:24.279 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestGroups::test_group_snap_flags PASSED [ 94%] 2024-04-07T17:50:24.721 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestGroups::test_group_snap_list_many PASSED [ 95%] 2024-04-07T17:50:26.142 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestGroups::test_group_snap_namespace PASSED [ 96%] 2024-04-07T17:50:26.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:50:25 smithi100 bash[74942]: cluster 2024-04-07T17:50:24.805241+0000 mgr.y (mgr.25175) 2851 : cluster [DBG] pgmap v5129: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 481 KiB/s rd, 58 KiB/s wr, 704 op/s 2024-04-07T17:50:26.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:50:25 smithi100 bash[74942]: cluster 2024-04-07T17:50:24.881157+0000 mon.a (mon.0) 4749 : cluster [DBG] osdmap e2704: 8 total, 8 up, 8 in 2024-04-07T17:50:26.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:50:25 smithi100 bash[76202]: cluster 2024-04-07T17:50:24.805241+0000 mgr.y (mgr.25175) 2851 : cluster [DBG] pgmap v5129: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 481 KiB/s rd, 58 KiB/s wr, 704 op/s 2024-04-07T17:50:26.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:50:25 smithi100 bash[76202]: cluster 2024-04-07T17:50:24.881157+0000 mon.a (mon.0) 4749 : cluster [DBG] osdmap e2704: 8 total, 8 up, 8 in 2024-04-07T17:50:26.364 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:50:25 smithi144 bash[66654]: cluster 2024-04-07T17:50:24.805241+0000 mgr.y (mgr.25175) 2851 : cluster [DBG] pgmap v5129: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 481 KiB/s rd, 58 KiB/s wr, 704 op/s 2024-04-07T17:50:26.364 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:50:25 smithi144 bash[66654]: cluster 2024-04-07T17:50:24.881157+0000 mon.a (mon.0) 4749 : cluster [DBG] osdmap e2704: 8 total, 8 up, 8 in 2024-04-07T17:50:26.419 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestGroups::test_group_snap_rename PASSED [ 96%] 2024-04-07T17:50:27.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:50:26 smithi100 bash[74942]: cluster 2024-04-07T17:50:25.891745+0000 mon.a (mon.0) 4750 : cluster [DBG] osdmap e2705: 8 total, 8 up, 8 in 2024-04-07T17:50:27.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:50:26 smithi100 bash[76202]: cluster 2024-04-07T17:50:25.891745+0000 mon.a (mon.0) 4750 : cluster [DBG] osdmap e2705: 8 total, 8 up, 8 in 2024-04-07T17:50:27.364 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:50:26 smithi144 bash[66654]: cluster 2024-04-07T17:50:25.891745+0000 mon.a (mon.0) 4750 : cluster [DBG] osdmap e2705: 8 total, 8 up, 8 in 2024-04-07T17:50:28.190 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestGroups::test_group_snap_rollback PASSED [ 97%] 2024-04-07T17:50:28.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:50:27 smithi100 bash[74942]: cluster 2024-04-07T17:50:26.805984+0000 mgr.y (mgr.25175) 2852 : cluster [DBG] pgmap v5132: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 464 KiB/s rd, 59 KiB/s wr, 679 op/s 2024-04-07T17:50:28.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:50:27 smithi100 bash[74942]: cluster 2024-04-07T17:50:26.905455+0000 mon.a (mon.0) 4751 : cluster [DBG] osdmap e2706: 8 total, 8 up, 8 in 2024-04-07T17:50:28.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:50:27 smithi100 bash[76202]: cluster 2024-04-07T17:50:26.805984+0000 mgr.y (mgr.25175) 2852 : cluster [DBG] pgmap v5132: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 464 KiB/s rd, 59 KiB/s wr, 679 op/s 2024-04-07T17:50:28.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:50:27 smithi100 bash[76202]: cluster 2024-04-07T17:50:26.905455+0000 mon.a (mon.0) 4751 : cluster [DBG] osdmap e2706: 8 total, 8 up, 8 in 2024-04-07T17:50:28.364 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:50:27 smithi144 bash[66654]: cluster 2024-04-07T17:50:26.805984+0000 mgr.y (mgr.25175) 2852 : cluster [DBG] pgmap v5132: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 464 KiB/s rd, 59 KiB/s wr, 679 op/s 2024-04-07T17:50:28.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:50:27 smithi144 bash[66654]: cluster 2024-04-07T17:50:26.905455+0000 mon.a (mon.0) 4751 : cluster [DBG] osdmap e2706: 8 total, 8 up, 8 in 2024-04-07T17:50:28.703 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:50:28 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:50:28] "GET /metrics HTTP/1.1" 200 36746 "" "Prometheus/2.43.0" 2024-04-07T17:50:28.961 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestMigration::test_migration PASSED [ 97%] 2024-04-07T17:50:29.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:50:28 smithi100 bash[74942]: cluster 2024-04-07T17:50:27.916687+0000 mon.a (mon.0) 4752 : cluster [DBG] osdmap e2707: 8 total, 8 up, 8 in 2024-04-07T17:50:29.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:50:28 smithi100 bash[76202]: cluster 2024-04-07T17:50:27.916687+0000 mon.a (mon.0) 4752 : cluster [DBG] osdmap e2707: 8 total, 8 up, 8 in 2024-04-07T17:50:29.364 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:50:28 smithi144 bash[66654]: cluster 2024-04-07T17:50:27.916687+0000 mon.a (mon.0) 4752 : cluster [DBG] osdmap e2707: 8 total, 8 up, 8 in 2024-04-07T17:50:30.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:50:29 smithi100 bash[74942]: cluster 2024-04-07T17:50:28.806933+0000 mgr.y (mgr.25175) 2853 : cluster [DBG] pgmap v5135: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 394 KiB/s rd, 26 KiB/s wr, 545 op/s 2024-04-07T17:50:30.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:50:29 smithi100 bash[76202]: cluster 2024-04-07T17:50:28.806933+0000 mgr.y (mgr.25175) 2853 : cluster [DBG] pgmap v5135: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 394 KiB/s rd, 26 KiB/s wr, 545 op/s 2024-04-07T17:50:30.364 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:50:29 smithi144 bash[66654]: cluster 2024-04-07T17:50:28.806933+0000 mgr.y (mgr.25175) 2853 : cluster [DBG] pgmap v5135: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 394 KiB/s rd, 26 KiB/s wr, 545 op/s 2024-04-07T17:50:31.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:50:30 smithi100 bash[74942]: cluster 2024-04-07T17:50:29.933439+0000 mon.a (mon.0) 4753 : cluster [DBG] osdmap e2708: 8 total, 8 up, 8 in 2024-04-07T17:50:31.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:50:30 smithi100 bash[76202]: cluster 2024-04-07T17:50:29.933439+0000 mon.a (mon.0) 4753 : cluster [DBG] osdmap e2708: 8 total, 8 up, 8 in 2024-04-07T17:50:31.364 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:50:30 smithi144 bash[66654]: cluster 2024-04-07T17:50:29.933439+0000 mon.a (mon.0) 4753 : cluster [DBG] osdmap e2708: 8 total, 8 up, 8 in 2024-04-07T17:50:32.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:50:31 smithi100 bash[74942]: cluster 2024-04-07T17:50:30.807870+0000 mgr.y (mgr.25175) 2854 : cluster [DBG] pgmap v5137: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 360 KiB/s rd, 21 KiB/s wr, 487 op/s 2024-04-07T17:50:32.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:50:31 smithi100 bash[74942]: cluster 2024-04-07T17:50:30.945818+0000 mon.a (mon.0) 4754 : cluster [DBG] osdmap e2709: 8 total, 8 up, 8 in 2024-04-07T17:50:32.203 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:50:31 smithi100 bash[74942]: audit 2024-04-07T17:50:31.887980+0000 mon.a (mon.0) 4755 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:50:32.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:50:31 smithi100 bash[76202]: cluster 2024-04-07T17:50:30.807870+0000 mgr.y (mgr.25175) 2854 : cluster [DBG] pgmap v5137: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 360 KiB/s rd, 21 KiB/s wr, 487 op/s 2024-04-07T17:50:32.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:50:31 smithi100 bash[76202]: cluster 2024-04-07T17:50:30.945818+0000 mon.a (mon.0) 4754 : cluster [DBG] osdmap e2709: 8 total, 8 up, 8 in 2024-04-07T17:50:32.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:50:31 smithi100 bash[76202]: audit 2024-04-07T17:50:31.887980+0000 mon.a (mon.0) 4755 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:50:32.316 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:50:31 smithi144 bash[66654]: cluster 2024-04-07T17:50:30.807870+0000 mgr.y (mgr.25175) 2854 : cluster [DBG] pgmap v5137: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 360 KiB/s rd, 21 KiB/s wr, 487 op/s 2024-04-07T17:50:32.316 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:50:31 smithi144 bash[66654]: cluster 2024-04-07T17:50:30.945818+0000 mon.a (mon.0) 4754 : cluster [DBG] osdmap e2709: 8 total, 8 up, 8 in 2024-04-07T17:50:32.316 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:50:31 smithi144 bash[66654]: audit 2024-04-07T17:50:31.887980+0000 mon.a (mon.0) 4755 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:50:32.614 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:50:32 smithi144 bash[95233]: ts=2024-04-07T17:50:32.317Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:50:33.357 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestMigration::test_migration_import PASSED [ 98%] 2024-04-07T17:50:33.364 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:50:32 smithi144 bash[66654]: cluster 2024-04-07T17:50:31.953946+0000 mon.a (mon.0) 4756 : cluster [DBG] osdmap e2710: 8 total, 8 up, 8 in 2024-04-07T17:50:33.453 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:50:32 smithi100 bash[74942]: cluster 2024-04-07T17:50:31.953946+0000 mon.a (mon.0) 4756 : cluster [DBG] osdmap e2710: 8 total, 8 up, 8 in 2024-04-07T17:50:33.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:50:32 smithi100 bash[76202]: cluster 2024-04-07T17:50:31.953946+0000 mon.a (mon.0) 4756 : cluster [DBG] osdmap e2710: 8 total, 8 up, 8 in 2024-04-07T17:50:34.005 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestMigration::test_migration_with_progress PASSED [ 98%] 2024-04-07T17:50:34.364 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:50:33 smithi144 bash[66654]: cluster 2024-04-07T17:50:32.809100+0000 mgr.y (mgr.25175) 2855 : cluster [DBG] pgmap v5140: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 278 KiB/s rd, 17 KiB/s wr, 365 op/s 2024-04-07T17:50:34.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:50:33 smithi144 bash[66654]: cluster 2024-04-07T17:50:32.963897+0000 mon.a (mon.0) 4757 : cluster [DBG] osdmap e2711: 8 total, 8 up, 8 in 2024-04-07T17:50:34.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:50:33 smithi144 bash[66654]: cluster 2024-04-07T17:50:33.557989+0000 mon.a (mon.0) 4758 : cluster [DBG] osdmap e2712: 8 total, 8 up, 8 in 2024-04-07T17:50:34.453 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:50:33 smithi100 bash[74942]: cluster 2024-04-07T17:50:32.809100+0000 mgr.y (mgr.25175) 2855 : cluster [DBG] pgmap v5140: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 278 KiB/s rd, 17 KiB/s wr, 365 op/s 2024-04-07T17:50:34.453 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:50:33 smithi100 bash[74942]: cluster 2024-04-07T17:50:32.963897+0000 mon.a (mon.0) 4757 : cluster [DBG] osdmap e2711: 8 total, 8 up, 8 in 2024-04-07T17:50:34.453 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:50:33 smithi100 bash[74942]: cluster 2024-04-07T17:50:33.557989+0000 mon.a (mon.0) 4758 : cluster [DBG] osdmap e2712: 8 total, 8 up, 8 in 2024-04-07T17:50:34.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:50:33 smithi100 bash[76202]: cluster 2024-04-07T17:50:32.809100+0000 mgr.y (mgr.25175) 2855 : cluster [DBG] pgmap v5140: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 278 KiB/s rd, 17 KiB/s wr, 365 op/s 2024-04-07T17:50:34.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:50:33 smithi100 bash[76202]: cluster 2024-04-07T17:50:32.963897+0000 mon.a (mon.0) 4757 : cluster [DBG] osdmap e2711: 8 total, 8 up, 8 in 2024-04-07T17:50:34.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:50:33 smithi100 bash[76202]: cluster 2024-04-07T17:50:33.557989+0000 mon.a (mon.0) 4758 : cluster [DBG] osdmap e2712: 8 total, 8 up, 8 in 2024-04-07T17:50:34.536 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestMigration::test_migrate_abort PASSED [ 99%] 2024-04-07T17:50:36.005 INFO:tasks.workunit.client.0.smithi100.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestMigration::test_migrate_abort_with_progress PASSED [100%] 2024-04-07T17:50:36.005 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:50:36.005 INFO:tasks.workunit.client.0.smithi100.stdout:=============================== warnings summary =============================== 2024-04-07T17:50:36.006 INFO:tasks.workunit.client.0.smithi100.stdout:test_rbd.py::test_remove_with_progress_except 2024-04-07T17:50:36.006 INFO:tasks.workunit.client.0.smithi100.stdout: /usr/lib/python3/dist-packages/_pytest/unraisableexception.py:78: PytestUnraisableExceptionWarning: Exception ignored in: 'rbd.progress_callback' 2024-04-07T17:50:36.006 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:50:36.006 INFO:tasks.workunit.client.0.smithi100.stdout: Traceback (most recent call last): 2024-04-07T17:50:36.006 INFO:tasks.workunit.client.0.smithi100.stdout: File "/home/ubuntu/cephtest/clone.client.0/src/test/pybind/test_rbd.py", line 423, in progress_cb 2024-04-07T17:50:36.006 INFO:tasks.workunit.client.0.smithi100.stdout: raise Exception() 2024-04-07T17:50:36.006 INFO:tasks.workunit.client.0.smithi100.stdout: Exception 2024-04-07T17:50:36.006 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:50:36.006 INFO:tasks.workunit.client.0.smithi100.stdout: warnings.warn(pytest.PytestUnraisableExceptionWarning(msg)) 2024-04-07T17:50:36.006 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:50:36.006 INFO:tasks.workunit.client.0.smithi100.stdout:-- Docs: https://docs.pytest.org/en/stable/warnings.html 2024-04-07T17:50:36.006 INFO:tasks.workunit.client.0.smithi100.stdout:============ 167 passed, 8 skipped, 1 warning in 681.66s (0:11:21) ============= 2024-04-07T17:50:36.080 INFO:tasks.workunit.client.0.smithi100.stderr:+ exit 0 2024-04-07T17:50:36.081 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-07T17:50:36.081 DEBUG:teuthology.orchestra.run.smithi100:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-07T17:50:36.093 INFO:tasks.workunit:Stopping ['rbd/test_librbd_python.sh'] on client.0... 2024-04-07T17:50:36.093 DEBUG:teuthology.orchestra.run.smithi100:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-04-07T17:50:36.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:50:35 smithi144 bash[66654]: cluster 2024-04-07T17:50:34.810143+0000 mgr.y (mgr.25175) 2856 : cluster [DBG] pgmap v5143: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 356 KiB/s rd, 17 KiB/s wr, 450 op/s 2024-04-07T17:50:36.375 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:50:35 smithi100 bash[74942]: cluster 2024-04-07T17:50:34.810143+0000 mgr.y (mgr.25175) 2856 : cluster [DBG] pgmap v5143: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 356 KiB/s rd, 17 KiB/s wr, 450 op/s 2024-04-07T17:50:36.376 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:50:35 smithi100 bash[76202]: cluster 2024-04-07T17:50:34.810143+0000 mgr.y (mgr.25175) 2856 : cluster [DBG] pgmap v5143: 137 pgs: 137 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 356 KiB/s rd, 17 KiB/s wr, 450 op/s 2024-04-07T17:50:36.377 DEBUG:teuthology.parallel:result is None 2024-04-07T17:50:36.377 DEBUG:teuthology.orchestra.run.smithi100:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-04-07T17:50:36.390 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-04-07T17:50:36.390 DEBUG:teuthology.orchestra.run.smithi100:> rmdir -- /home/ubuntu/cephtest/mnt.0 2024-04-07T17:50:36.435 INFO:tasks.workunit:Deleted artificial mount point /home/ubuntu/cephtest/mnt.0/client.0 2024-04-07T17:50:36.436 INFO:teuthology.task.full_sequential:In full_sequential, running task print... 2024-04-07T17:50:36.436 INFO:teuthology.task.print:**** done end test_rbd_python.yaml 2024-04-07T17:50:36.436 DEBUG:teuthology.parallel:result is None 2024-04-07T17:50:36.436 INFO:teuthology.run_tasks:Running task print... 2024-04-07T17:50:36.446 INFO:teuthology.task.print:**** done end parallel 2024-04-07T17:50:36.446 INFO:teuthology.run_tasks:Running task print... 2024-04-07T17:50:36.455 INFO:teuthology.task.print:**** done start telemetry x... 2024-04-07T17:50:36.456 INFO:teuthology.run_tasks:Running task workunit... 2024-04-07T17:50:36.466 INFO:tasks.workunit:Pulling workunits from ref adce92d056533f22dc1ee37e3141d098b046ec1b 2024-04-07T17:50:36.466 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-04-07T17:50:36.467 DEBUG:teuthology.orchestra.run.smithi100:> stat -- /home/ubuntu/cephtest/mnt.0 2024-04-07T17:50:36.483 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-07T17:50:36.483 INFO:teuthology.orchestra.run.smithi100.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-04-07T17:50:36.484 DEBUG:teuthology.orchestra.run.smithi100:> mkdir -- /home/ubuntu/cephtest/mnt.0 2024-04-07T17:50:36.532 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2024-04-07T17:50:36.532 DEBUG:teuthology.orchestra.run.smithi100:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2024-04-07T17:50:36.580 INFO:tasks.workunit:timeout=3h 2024-04-07T17:50:36.580 INFO:tasks.workunit:cleanup=True 2024-04-07T17:50:36.580 DEBUG:teuthology.orchestra.run.smithi100:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/ceph/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout adce92d056533f22dc1ee37e3141d098b046ec1b 2024-04-07T17:50:36.630 INFO:tasks.workunit.client.0.smithi100.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-04-07T17:50:37.364 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:50:36 smithi144 bash[66654]: cluster 2024-04-07T17:50:35.978935+0000 mon.a (mon.0) 4759 : cluster [DBG] osdmap e2713: 8 total, 8 up, 8 in 2024-04-07T17:50:37.453 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:50:36 smithi100 bash[74942]: cluster 2024-04-07T17:50:35.978935+0000 mon.a (mon.0) 4759 : cluster [DBG] osdmap e2713: 8 total, 8 up, 8 in 2024-04-07T17:50:37.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:50:36 smithi100 bash[76202]: cluster 2024-04-07T17:50:35.978935+0000 mon.a (mon.0) 4759 : cluster [DBG] osdmap e2713: 8 total, 8 up, 8 in 2024-04-07T17:50:38.364 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:50:37 smithi144 bash[66654]: cluster 2024-04-07T17:50:36.810836+0000 mgr.y (mgr.25175) 2857 : cluster [DBG] pgmap v5145: 105 pgs: 105 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail 2024-04-07T17:50:38.426 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:50:37 smithi100 bash[74942]: cluster 2024-04-07T17:50:36.810836+0000 mgr.y (mgr.25175) 2857 : cluster [DBG] pgmap v5145: 105 pgs: 105 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail 2024-04-07T17:50:38.426 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:50:37 smithi100 bash[76202]: cluster 2024-04-07T17:50:36.810836+0000 mgr.y (mgr.25175) 2857 : cluster [DBG] pgmap v5145: 105 pgs: 105 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail 2024-04-07T17:50:38.703 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:50:38 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:50:38] "GET /metrics HTTP/1.1" 200 36742 "" "Prometheus/2.43.0" 2024-04-07T17:50:40.364 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:50:39 smithi144 bash[66654]: cluster 2024-04-07T17:50:38.812008+0000 mgr.y (mgr.25175) 2858 : cluster [DBG] pgmap v5146: 105 pgs: 105 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail 2024-04-07T17:50:40.453 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:50:39 smithi100 bash[74942]: cluster 2024-04-07T17:50:38.812008+0000 mgr.y (mgr.25175) 2858 : cluster [DBG] pgmap v5146: 105 pgs: 105 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail 2024-04-07T17:50:40.453 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:50:39 smithi100 bash[76202]: cluster 2024-04-07T17:50:38.812008+0000 mgr.y (mgr.25175) 2858 : cluster [DBG] pgmap v5146: 105 pgs: 105 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail 2024-04-07T17:50:42.315 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:50:41 smithi144 bash[66654]: cluster 2024-04-07T17:50:40.813061+0000 mgr.y (mgr.25175) 2859 : cluster [DBG] pgmap v5147: 105 pgs: 105 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail 2024-04-07T17:50:42.453 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:50:41 smithi100 bash[76202]: cluster 2024-04-07T17:50:40.813061+0000 mgr.y (mgr.25175) 2859 : cluster [DBG] pgmap v5147: 105 pgs: 105 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail 2024-04-07T17:50:42.453 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:50:41 smithi100 bash[74942]: cluster 2024-04-07T17:50:40.813061+0000 mgr.y (mgr.25175) 2859 : cluster [DBG] pgmap v5147: 105 pgs: 105 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail 2024-04-07T17:50:42.614 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:50:42 smithi144 bash[95233]: ts=2024-04-07T17:50:42.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:50:43.364 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:50:42 smithi144 bash[66654]: audit 2024-04-07T17:50:42.468644+0000 mon.a (mon.0) 4760 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:50:43.453 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:50:42 smithi100 bash[74942]: audit 2024-04-07T17:50:42.468644+0000 mon.a (mon.0) 4760 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:50:43.453 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:50:42 smithi100 bash[76202]: audit 2024-04-07T17:50:42.468644+0000 mon.a (mon.0) 4760 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:50:44.364 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:50:44 smithi144 bash[66654]: cluster 2024-04-07T17:50:42.814273+0000 mgr.y (mgr.25175) 2860 : cluster [DBG] pgmap v5148: 105 pgs: 105 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail 2024-04-07T17:50:44.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:50:44 smithi144 bash[66654]: audit 2024-04-07T17:50:43.145937+0000 mon.a (mon.0) 4761 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:50:44.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:50:44 smithi144 bash[66654]: audit 2024-04-07T17:50:43.147187+0000 mon.a (mon.0) 4762 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:50:44.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:50:44 smithi144 bash[66654]: audit 2024-04-07T17:50:43.155372+0000 mon.a (mon.0) 4763 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:50:44.453 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:50:44 smithi100 bash[76202]: cluster 2024-04-07T17:50:42.814273+0000 mgr.y (mgr.25175) 2860 : cluster [DBG] pgmap v5148: 105 pgs: 105 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail 2024-04-07T17:50:44.453 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:50:44 smithi100 bash[76202]: audit 2024-04-07T17:50:43.145937+0000 mon.a (mon.0) 4761 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:50:44.453 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:50:44 smithi100 bash[76202]: audit 2024-04-07T17:50:43.147187+0000 mon.a (mon.0) 4762 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:50:44.453 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:50:44 smithi100 bash[76202]: audit 2024-04-07T17:50:43.155372+0000 mon.a (mon.0) 4763 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:50:44.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:50:43 smithi100 bash[74942]: cluster 2024-04-07T17:50:42.814273+0000 mgr.y (mgr.25175) 2860 : cluster [DBG] pgmap v5148: 105 pgs: 105 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail 2024-04-07T17:50:44.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:50:43 smithi100 bash[74942]: audit 2024-04-07T17:50:43.145937+0000 mon.a (mon.0) 4761 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:50:44.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:50:44 smithi100 bash[74942]: audit 2024-04-07T17:50:43.147187+0000 mon.a (mon.0) 4762 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:50:44.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:50:44 smithi100 bash[74942]: audit 2024-04-07T17:50:43.155372+0000 mon.a (mon.0) 4763 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:50:46.364 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:50:46 smithi144 bash[66654]: cluster 2024-04-07T17:50:44.814915+0000 mgr.y (mgr.25175) 2861 : cluster [DBG] pgmap v5149: 105 pgs: 105 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail 2024-04-07T17:50:46.453 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:50:46 smithi100 bash[74942]: cluster 2024-04-07T17:50:44.814915+0000 mgr.y (mgr.25175) 2861 : cluster [DBG] pgmap v5149: 105 pgs: 105 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail 2024-04-07T17:50:46.453 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:50:46 smithi100 bash[76202]: cluster 2024-04-07T17:50:44.814915+0000 mgr.y (mgr.25175) 2861 : cluster [DBG] pgmap v5149: 105 pgs: 105 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail 2024-04-07T17:50:47.364 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:50:47 smithi144 bash[66654]: audit 2024-04-07T17:50:46.888437+0000 mon.a (mon.0) 4764 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:50:47.453 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:50:47 smithi100 bash[74942]: audit 2024-04-07T17:50:46.888437+0000 mon.a (mon.0) 4764 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:50:47.453 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:50:47 smithi100 bash[76202]: audit 2024-04-07T17:50:46.888437+0000 mon.a (mon.0) 4764 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:50:48.364 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:50:48 smithi144 bash[66654]: cluster 2024-04-07T17:50:46.815726+0000 mgr.y (mgr.25175) 2862 : cluster [DBG] pgmap v5150: 105 pgs: 105 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail 2024-04-07T17:50:48.425 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:50:48 smithi100 bash[74942]: cluster 2024-04-07T17:50:46.815726+0000 mgr.y (mgr.25175) 2862 : cluster [DBG] pgmap v5150: 105 pgs: 105 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail 2024-04-07T17:50:48.425 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:50:48 smithi100 bash[76202]: cluster 2024-04-07T17:50:46.815726+0000 mgr.y (mgr.25175) 2862 : cluster [DBG] pgmap v5150: 105 pgs: 105 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail 2024-04-07T17:50:48.703 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:50:48 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:50:48] "GET /metrics HTTP/1.1" 200 34265 "" "Prometheus/2.43.0" 2024-04-07T17:50:50.364 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:50:50 smithi144 bash[66654]: cluster 2024-04-07T17:50:48.816872+0000 mgr.y (mgr.25175) 2863 : cluster [DBG] pgmap v5151: 105 pgs: 105 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail 2024-04-07T17:50:50.453 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:50:50 smithi100 bash[76202]: cluster 2024-04-07T17:50:48.816872+0000 mgr.y (mgr.25175) 2863 : cluster [DBG] pgmap v5151: 105 pgs: 105 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail 2024-04-07T17:50:50.453 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:50:50 smithi100 bash[74942]: cluster 2024-04-07T17:50:48.816872+0000 mgr.y (mgr.25175) 2863 : cluster [DBG] pgmap v5151: 105 pgs: 105 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail 2024-04-07T17:50:52.315 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:50:52 smithi144 bash[66654]: cluster 2024-04-07T17:50:50.817889+0000 mgr.y (mgr.25175) 2864 : cluster [DBG] pgmap v5152: 105 pgs: 105 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail 2024-04-07T17:50:52.453 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:50:52 smithi100 bash[74942]: cluster 2024-04-07T17:50:50.817889+0000 mgr.y (mgr.25175) 2864 : cluster [DBG] pgmap v5152: 105 pgs: 105 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail 2024-04-07T17:50:52.453 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:50:52 smithi100 bash[76202]: cluster 2024-04-07T17:50:50.817889+0000 mgr.y (mgr.25175) 2864 : cluster [DBG] pgmap v5152: 105 pgs: 105 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail 2024-04-07T17:50:52.614 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:50:52 smithi144 bash[95233]: ts=2024-04-07T17:50:52.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:50:54.364 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:50:54 smithi144 bash[66654]: cluster 2024-04-07T17:50:52.818561+0000 mgr.y (mgr.25175) 2865 : cluster [DBG] pgmap v5153: 105 pgs: 105 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail 2024-04-07T17:50:54.453 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:50:54 smithi100 bash[74942]: cluster 2024-04-07T17:50:52.818561+0000 mgr.y (mgr.25175) 2865 : cluster [DBG] pgmap v5153: 105 pgs: 105 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail 2024-04-07T17:50:54.453 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:50:54 smithi100 bash[76202]: cluster 2024-04-07T17:50:52.818561+0000 mgr.y (mgr.25175) 2865 : cluster [DBG] pgmap v5153: 105 pgs: 105 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail 2024-04-07T17:50:56.364 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:50:56 smithi144 bash[66654]: cluster 2024-04-07T17:50:54.819131+0000 mgr.y (mgr.25175) 2866 : cluster [DBG] pgmap v5154: 105 pgs: 105 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail 2024-04-07T17:50:56.453 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:50:56 smithi100 bash[74942]: cluster 2024-04-07T17:50:54.819131+0000 mgr.y (mgr.25175) 2866 : cluster [DBG] pgmap v5154: 105 pgs: 105 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail 2024-04-07T17:50:56.453 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:50:56 smithi100 bash[76202]: cluster 2024-04-07T17:50:54.819131+0000 mgr.y (mgr.25175) 2866 : cluster [DBG] pgmap v5154: 105 pgs: 105 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail 2024-04-07T17:50:58.364 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:50:58 smithi144 bash[66654]: cluster 2024-04-07T17:50:56.819632+0000 mgr.y (mgr.25175) 2867 : cluster [DBG] pgmap v5155: 105 pgs: 105 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail 2024-04-07T17:50:58.424 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:50:58 smithi100 bash[74942]: cluster 2024-04-07T17:50:56.819632+0000 mgr.y (mgr.25175) 2867 : cluster [DBG] pgmap v5155: 105 pgs: 105 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail 2024-04-07T17:50:58.424 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:50:58 smithi100 bash[76202]: cluster 2024-04-07T17:50:56.819632+0000 mgr.y (mgr.25175) 2867 : cluster [DBG] pgmap v5155: 105 pgs: 105 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail 2024-04-07T17:50:58.703 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:50:58 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:50:58] "GET /metrics HTTP/1.1" 200 34265 "" "Prometheus/2.43.0" 2024-04-07T17:51:00.364 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:00 smithi144 bash[66654]: cluster 2024-04-07T17:50:58.820288+0000 mgr.y (mgr.25175) 2868 : cluster [DBG] pgmap v5156: 105 pgs: 105 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail 2024-04-07T17:51:00.453 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:00 smithi100 bash[74942]: cluster 2024-04-07T17:50:58.820288+0000 mgr.y (mgr.25175) 2868 : cluster [DBG] pgmap v5156: 105 pgs: 105 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail 2024-04-07T17:51:00.453 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:00 smithi100 bash[76202]: cluster 2024-04-07T17:50:58.820288+0000 mgr.y (mgr.25175) 2868 : cluster [DBG] pgmap v5156: 105 pgs: 105 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail 2024-04-07T17:51:02.316 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:02 smithi144 bash[66654]: cluster 2024-04-07T17:51:00.820870+0000 mgr.y (mgr.25175) 2869 : cluster [DBG] pgmap v5157: 105 pgs: 105 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail 2024-04-07T17:51:02.316 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:02 smithi144 bash[66654]: audit 2024-04-07T17:51:01.887958+0000 mon.a (mon.0) 4765 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:51:02.453 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:02 smithi100 bash[74942]: cluster 2024-04-07T17:51:00.820870+0000 mgr.y (mgr.25175) 2869 : cluster [DBG] pgmap v5157: 105 pgs: 105 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail 2024-04-07T17:51:02.453 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:02 smithi100 bash[74942]: audit 2024-04-07T17:51:01.887958+0000 mon.a (mon.0) 4765 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:51:02.453 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:02 smithi100 bash[76202]: cluster 2024-04-07T17:51:00.820870+0000 mgr.y (mgr.25175) 2869 : cluster [DBG] pgmap v5157: 105 pgs: 105 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail 2024-04-07T17:51:02.453 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:02 smithi100 bash[76202]: audit 2024-04-07T17:51:01.887958+0000 mon.a (mon.0) 4765 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:51:02.614 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:51:02 smithi144 bash[95233]: ts=2024-04-07T17:51:02.317Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:51:04.364 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:04 smithi144 bash[66654]: cluster 2024-04-07T17:51:02.821489+0000 mgr.y (mgr.25175) 2870 : cluster [DBG] pgmap v5158: 105 pgs: 105 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail 2024-04-07T17:51:04.453 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:04 smithi100 bash[74942]: cluster 2024-04-07T17:51:02.821489+0000 mgr.y (mgr.25175) 2870 : cluster [DBG] pgmap v5158: 105 pgs: 105 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail 2024-04-07T17:51:04.453 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:04 smithi100 bash[76202]: cluster 2024-04-07T17:51:02.821489+0000 mgr.y (mgr.25175) 2870 : cluster [DBG] pgmap v5158: 105 pgs: 105 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail 2024-04-07T17:51:06.364 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:06 smithi144 bash[66654]: cluster 2024-04-07T17:51:04.822074+0000 mgr.y (mgr.25175) 2871 : cluster [DBG] pgmap v5159: 105 pgs: 105 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail 2024-04-07T17:51:06.453 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:06 smithi100 bash[74942]: cluster 2024-04-07T17:51:04.822074+0000 mgr.y (mgr.25175) 2871 : cluster [DBG] pgmap v5159: 105 pgs: 105 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail 2024-04-07T17:51:06.453 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:06 smithi100 bash[76202]: cluster 2024-04-07T17:51:04.822074+0000 mgr.y (mgr.25175) 2871 : cluster [DBG] pgmap v5159: 105 pgs: 105 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail 2024-04-07T17:51:08.364 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:08 smithi144 bash[66654]: cluster 2024-04-07T17:51:06.822554+0000 mgr.y (mgr.25175) 2872 : cluster [DBG] pgmap v5160: 105 pgs: 105 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail 2024-04-07T17:51:08.424 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:08 smithi100 bash[74942]: cluster 2024-04-07T17:51:06.822554+0000 mgr.y (mgr.25175) 2872 : cluster [DBG] pgmap v5160: 105 pgs: 105 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail 2024-04-07T17:51:08.424 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:08 smithi100 bash[76202]: cluster 2024-04-07T17:51:06.822554+0000 mgr.y (mgr.25175) 2872 : cluster [DBG] pgmap v5160: 105 pgs: 105 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail 2024-04-07T17:51:08.703 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:51:08 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:51:08] "GET /metrics HTTP/1.1" 200 34264 "" "Prometheus/2.43.0" 2024-04-07T17:51:10.364 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:10 smithi144 bash[66654]: cluster 2024-04-07T17:51:08.823187+0000 mgr.y (mgr.25175) 2873 : cluster [DBG] pgmap v5161: 105 pgs: 105 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail 2024-04-07T17:51:10.453 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:10 smithi100 bash[74942]: cluster 2024-04-07T17:51:08.823187+0000 mgr.y (mgr.25175) 2873 : cluster [DBG] pgmap v5161: 105 pgs: 105 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail 2024-04-07T17:51:10.453 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:10 smithi100 bash[76202]: cluster 2024-04-07T17:51:08.823187+0000 mgr.y (mgr.25175) 2873 : cluster [DBG] pgmap v5161: 105 pgs: 105 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail 2024-04-07T17:51:12.314 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:12 smithi144 bash[66654]: cluster 2024-04-07T17:51:10.823698+0000 mgr.y (mgr.25175) 2874 : cluster [DBG] pgmap v5162: 105 pgs: 105 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail 2024-04-07T17:51:12.453 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:12 smithi100 bash[74942]: cluster 2024-04-07T17:51:10.823698+0000 mgr.y (mgr.25175) 2874 : cluster [DBG] pgmap v5162: 105 pgs: 105 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail 2024-04-07T17:51:12.453 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:12 smithi100 bash[76202]: cluster 2024-04-07T17:51:10.823698+0000 mgr.y (mgr.25175) 2874 : cluster [DBG] pgmap v5162: 105 pgs: 105 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail 2024-04-07T17:51:12.614 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:51:12 smithi144 bash[95233]: ts=2024-04-07T17:51:12.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:51:14.364 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:14 smithi144 bash[66654]: cluster 2024-04-07T17:51:12.824307+0000 mgr.y (mgr.25175) 2875 : cluster [DBG] pgmap v5163: 105 pgs: 105 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail 2024-04-07T17:51:14.453 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:14 smithi100 bash[74942]: cluster 2024-04-07T17:51:12.824307+0000 mgr.y (mgr.25175) 2875 : cluster [DBG] pgmap v5163: 105 pgs: 105 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail 2024-04-07T17:51:14.453 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:14 smithi100 bash[76202]: cluster 2024-04-07T17:51:12.824307+0000 mgr.y (mgr.25175) 2875 : cluster [DBG] pgmap v5163: 105 pgs: 105 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail 2024-04-07T17:51:16.364 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:16 smithi144 bash[66654]: cluster 2024-04-07T17:51:14.825268+0000 mgr.y (mgr.25175) 2876 : cluster [DBG] pgmap v5164: 105 pgs: 105 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail 2024-04-07T17:51:16.453 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:16 smithi100 bash[74942]: cluster 2024-04-07T17:51:14.825268+0000 mgr.y (mgr.25175) 2876 : cluster [DBG] pgmap v5164: 105 pgs: 105 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail 2024-04-07T17:51:16.453 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:16 smithi100 bash[76202]: cluster 2024-04-07T17:51:14.825268+0000 mgr.y (mgr.25175) 2876 : cluster [DBG] pgmap v5164: 105 pgs: 105 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail 2024-04-07T17:51:17.364 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:17 smithi144 bash[66654]: audit 2024-04-07T17:51:16.888013+0000 mon.a (mon.0) 4766 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:51:17.453 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:17 smithi100 bash[74942]: audit 2024-04-07T17:51:16.888013+0000 mon.a (mon.0) 4766 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:51:17.453 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:17 smithi100 bash[76202]: audit 2024-04-07T17:51:16.888013+0000 mon.a (mon.0) 4766 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:51:18.364 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:18 smithi144 bash[66654]: cluster 2024-04-07T17:51:16.825923+0000 mgr.y (mgr.25175) 2877 : cluster [DBG] pgmap v5165: 105 pgs: 105 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail 2024-04-07T17:51:18.426 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:18 smithi100 bash[74942]: cluster 2024-04-07T17:51:16.825923+0000 mgr.y (mgr.25175) 2877 : cluster [DBG] pgmap v5165: 105 pgs: 105 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail 2024-04-07T17:51:18.426 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:18 smithi100 bash[76202]: cluster 2024-04-07T17:51:16.825923+0000 mgr.y (mgr.25175) 2877 : cluster [DBG] pgmap v5165: 105 pgs: 105 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail 2024-04-07T17:51:18.703 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:51:18 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:51:18] "GET /metrics HTTP/1.1" 200 34269 "" "Prometheus/2.43.0" 2024-04-07T17:51:20.260 INFO:tasks.workunit.client.0.smithi100.stderr:Note: switching to 'adce92d056533f22dc1ee37e3141d098b046ec1b'. 2024-04-07T17:51:20.260 INFO:tasks.workunit.client.0.smithi100.stderr: 2024-04-07T17:51:20.260 INFO:tasks.workunit.client.0.smithi100.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-04-07T17:51:20.261 INFO:tasks.workunit.client.0.smithi100.stderr:changes and commit them, and you can discard any commits you make in this 2024-04-07T17:51:20.261 INFO:tasks.workunit.client.0.smithi100.stderr:state without impacting any branches by switching back to a branch. 2024-04-07T17:51:20.261 INFO:tasks.workunit.client.0.smithi100.stderr: 2024-04-07T17:51:20.261 INFO:tasks.workunit.client.0.smithi100.stderr:If you want to create a new branch to retain commits you create, you may 2024-04-07T17:51:20.261 INFO:tasks.workunit.client.0.smithi100.stderr:do so (now or later) by using -c with the switch command. Example: 2024-04-07T17:51:20.261 INFO:tasks.workunit.client.0.smithi100.stderr: 2024-04-07T17:51:20.261 INFO:tasks.workunit.client.0.smithi100.stderr: git switch -c 2024-04-07T17:51:20.261 INFO:tasks.workunit.client.0.smithi100.stderr: 2024-04-07T17:51:20.261 INFO:tasks.workunit.client.0.smithi100.stderr:Or undo this operation with: 2024-04-07T17:51:20.261 INFO:tasks.workunit.client.0.smithi100.stderr: 2024-04-07T17:51:20.261 INFO:tasks.workunit.client.0.smithi100.stderr: git switch - 2024-04-07T17:51:20.261 INFO:tasks.workunit.client.0.smithi100.stderr: 2024-04-07T17:51:20.261 INFO:tasks.workunit.client.0.smithi100.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-04-07T17:51:20.262 INFO:tasks.workunit.client.0.smithi100.stderr: 2024-04-07T17:51:20.262 INFO:tasks.workunit.client.0.smithi100.stderr:HEAD is now at adce92d0565 Merge branch 'wlk-dev' of https://github.com/wanglinke521/ceph into wip-yuri3-testing-2024-04-05-0825 2024-04-07T17:51:20.268 DEBUG:teuthology.orchestra.run.smithi100:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-04-07T17:51:20.317 INFO:tasks.workunit.client.0.smithi100.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-04-07T17:51:20.320 INFO:tasks.workunit.client.0.smithi100.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-07T17:51:20.320 INFO:tasks.workunit.client.0.smithi100.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-04-07T17:51:20.364 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:20 smithi144 bash[66654]: cluster 2024-04-07T17:51:18.827012+0000 mgr.y (mgr.25175) 2878 : cluster [DBG] pgmap v5166: 105 pgs: 105 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail 2024-04-07T17:51:20.382 INFO:tasks.workunit.client.0.smithi100.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-04-07T17:51:20.438 INFO:tasks.workunit.client.0.smithi100.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-04-07T17:51:20.453 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:20 smithi100 bash[76202]: cluster 2024-04-07T17:51:18.827012+0000 mgr.y (mgr.25175) 2878 : cluster [DBG] pgmap v5166: 105 pgs: 105 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail 2024-04-07T17:51:20.453 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:20 smithi100 bash[74942]: cluster 2024-04-07T17:51:18.827012+0000 mgr.y (mgr.25175) 2878 : cluster [DBG] pgmap v5166: 105 pgs: 105 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail 2024-04-07T17:51:20.483 INFO:tasks.workunit.client.0.smithi100.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-07T17:51:20.484 INFO:tasks.workunit.client.0.smithi100.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-07T17:51:20.484 INFO:tasks.workunit.client.0.smithi100.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-04-07T17:51:20.527 INFO:tasks.workunit.client.0.smithi100.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-07T17:51:20.530 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-07T17:51:20.531 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-04-07T17:51:20.576 INFO:tasks.workunit:Running workunits matching test_telemetry_reef_x.sh on client.0... 2024-04-07T17:51:20.577 INFO:tasks.workunit:Running workunit test_telemetry_reef_x.sh... 2024-04-07T17:51:20.577 DEBUG:teuthology.orchestra.run.smithi100:workunit test test_telemetry_reef_x.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=adce92d056533f22dc1ee37e3141d098b046ec1b TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/test_telemetry_reef_x.sh 2024-04-07T17:51:20.628 INFO:tasks.workunit.client.0.smithi100.stderr:++ ceph config get mgr mgr/telemetry/last_opt_revision 2024-04-07T17:51:21.053 INFO:tasks.workunit.client.0.smithi100.stderr:+ LAST_OPT_REVISION=1 2024-04-07T17:51:21.053 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' 1 -ne 1 ']' 2024-04-07T17:51:21.053 INFO:tasks.workunit.client.0.smithi100.stderr:+ ceph -s 2024-04-07T17:51:21.364 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:21 smithi144 bash[66654]: audit 2024-04-07T17:51:20.936967+0000 mon.c (mon.2) 117 : audit [DBG] from='client.? 172.21.15.100:0/1366160804' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/telemetry/last_opt_revision"}]: dispatch 2024-04-07T17:51:21.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:21 smithi144 bash[66654]: audit 2024-04-07T17:51:21.036722+0000 mon.c (mon.2) 118 : audit [DBG] from='client.? 172.21.15.100:0/1366160804' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/telemetry/last_opt_revision"}]: dispatch 2024-04-07T17:51:21.453 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:21 smithi100 bash[76202]: audit 2024-04-07T17:51:20.936967+0000 mon.c (mon.2) 117 : audit [DBG] from='client.? 172.21.15.100:0/1366160804' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/telemetry/last_opt_revision"}]: dispatch 2024-04-07T17:51:21.453 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:21 smithi100 bash[76202]: audit 2024-04-07T17:51:21.036722+0000 mon.c (mon.2) 118 : audit [DBG] from='client.? 172.21.15.100:0/1366160804' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/telemetry/last_opt_revision"}]: dispatch 2024-04-07T17:51:21.453 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:21 smithi100 bash[74942]: audit 2024-04-07T17:51:20.936967+0000 mon.c (mon.2) 117 : audit [DBG] from='client.? 172.21.15.100:0/1366160804' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/telemetry/last_opt_revision"}]: dispatch 2024-04-07T17:51:21.453 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:21 smithi100 bash[74942]: audit 2024-04-07T17:51:21.036722+0000 mon.c (mon.2) 118 : audit [DBG] from='client.? 172.21.15.100:0/1366160804' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/telemetry/last_opt_revision"}]: dispatch 2024-04-07T17:51:21.566 INFO:tasks.workunit.client.0.smithi100.stdout: cluster: 2024-04-07T17:51:21.566 INFO:tasks.workunit.client.0.smithi100.stdout: id: 0702e3a8-f4f8-11ee-b649-cb9ed24678a4 2024-04-07T17:51:21.566 INFO:tasks.workunit.client.0.smithi100.stdout: health: HEALTH_WARN 2024-04-07T17:51:21.566 INFO:tasks.workunit.client.0.smithi100.stdout: Telemetry requires re-opt-in 2024-04-07T17:51:21.566 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:51:21.566 INFO:tasks.workunit.client.0.smithi100.stdout: services: 2024-04-07T17:51:21.567 INFO:tasks.workunit.client.0.smithi100.stdout: mon: 3 daemons, quorum a,b,c (age 90m) 2024-04-07T17:51:21.567 INFO:tasks.workunit.client.0.smithi100.stdout: mgr: y(active, since 91m), standbys: x 2024-04-07T17:51:21.567 INFO:tasks.workunit.client.0.smithi100.stdout: mds: 1/1 daemons up, 1 standby 2024-04-07T17:51:21.567 INFO:tasks.workunit.client.0.smithi100.stdout: osd: 8 osds: 8 up (since 85m), 8 in (since 100m) 2024-04-07T17:51:21.567 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:51:21.567 INFO:tasks.workunit.client.0.smithi100.stdout: data: 2024-04-07T17:51:21.567 INFO:tasks.workunit.client.0.smithi100.stdout: volumes: 1/1 healthy 2024-04-07T17:51:21.567 INFO:tasks.workunit.client.0.smithi100.stdout: pools: 4 pools, 105 pgs 2024-04-07T17:51:21.567 INFO:tasks.workunit.client.0.smithi100.stdout: objects: 28 objects, 4.6 MiB 2024-04-07T17:51:21.567 INFO:tasks.workunit.client.0.smithi100.stdout: usage: 3.9 GiB used, 711 GiB / 715 GiB avail 2024-04-07T17:51:21.567 INFO:tasks.workunit.client.0.smithi100.stdout: pgs: 105 active+clean 2024-04-07T17:51:21.567 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-04-07T17:51:21.585 INFO:tasks.workunit.client.0.smithi100.stderr:++ ceph telemetry collection ls 2024-04-07T17:51:22.095 INFO:tasks.workunit.client.0.smithi100.stderr:+ COLLECTIONS='New collections are available: 2024-04-07T17:51:22.095 INFO:tasks.workunit.client.0.smithi100.stderr:['\''basic_pool_flags'\''] 2024-04-07T17:51:22.095 INFO:tasks.workunit.client.0.smithi100.stderr:Run `ceph telemetry on` to opt-in to these collections. 2024-04-07T17:51:22.095 INFO:tasks.workunit.client.0.smithi100.stderr: 2024-04-07T17:51:22.095 INFO:tasks.workunit.client.0.smithi100.stderr:NAME STATUS DESC 2024-04-07T17:51:22.095 INFO:tasks.workunit.client.0.smithi100.stderr:basic_base REPORTING Basic information about the cluster (capacity, number and type of daemons, version, etc.) 2024-04-07T17:51:22.095 INFO:tasks.workunit.client.0.smithi100.stderr:basic_mds_metadata REPORTING MDS metadata 2024-04-07T17:51:22.095 INFO:tasks.workunit.client.0.smithi100.stderr:basic_pool_flags NOT REPORTING: NOT OPTED-IN Per-pool flags 2024-04-07T17:51:22.095 INFO:tasks.workunit.client.0.smithi100.stderr:basic_pool_options_bluestore REPORTING Per-pool bluestore config options 2024-04-07T17:51:22.096 INFO:tasks.workunit.client.0.smithi100.stderr:basic_pool_usage REPORTING Default pool application and usage statistics 2024-04-07T17:51:22.096 INFO:tasks.workunit.client.0.smithi100.stderr:basic_rook_v01 REPORTING Basic Rook deployment data 2024-04-07T17:51:22.096 INFO:tasks.workunit.client.0.smithi100.stderr:basic_usage_by_class REPORTING Default device class usage statistics 2024-04-07T17:51:22.096 INFO:tasks.workunit.client.0.smithi100.stderr:crash_base REPORTING Information about daemon crashes (daemon type and version, backtrace, etc.) 2024-04-07T17:51:22.096 INFO:tasks.workunit.client.0.smithi100.stderr:device_base REPORTING Information about device health metrics 2024-04-07T17:51:22.096 INFO:tasks.workunit.client.0.smithi100.stderr:ident_base REPORTING User-provided identifying information about the cluster 2024-04-07T17:51:22.096 INFO:tasks.workunit.client.0.smithi100.stderr:perf_memory_metrics REPORTING Heap stats and mempools for mon and mds 2024-04-07T17:51:22.096 INFO:tasks.workunit.client.0.smithi100.stderr:perf_perf REPORTING Information about performance counters of the cluster ' 2024-04-07T17:51:22.096 INFO:tasks.workunit.client.0.smithi100.stderr:+ NEW_COLLECTIONS=("perf_perf" "basic_mds_metadata" "basic_pool_usage" "basic_rook_v01" "perf_memory_metrics" "basic_pool_flags") 2024-04-07T17:51:22.096 INFO:tasks.workunit.client.0.smithi100.stderr:+ for col in ${NEW_COLLECTIONS[@]} 2024-04-07T17:51:22.096 INFO:tasks.workunit.client.0.smithi100.stderr:+ [[ New collections are available: 2024-04-07T17:51:22.096 INFO:tasks.workunit.client.0.smithi100.stderr:['basic_pool_flags'] 2024-04-07T17:51:22.096 INFO:tasks.workunit.client.0.smithi100.stderr:Run `ceph telemetry on` to opt-in to these collections. 2024-04-07T17:51:22.096 INFO:tasks.workunit.client.0.smithi100.stderr: 2024-04-07T17:51:22.096 INFO:tasks.workunit.client.0.smithi100.stderr:NAME STATUS DESC 2024-04-07T17:51:22.097 INFO:tasks.workunit.client.0.smithi100.stderr:basic_base REPORTING Basic information about the cluster (capacity, number and type of daemons, version, etc.) 2024-04-07T17:51:22.097 INFO:tasks.workunit.client.0.smithi100.stderr:basic_mds_metadata REPORTING MDS metadata 2024-04-07T17:51:22.097 INFO:tasks.workunit.client.0.smithi100.stderr:basic_pool_flags NOT REPORTING: NOT OPTED-IN Per-pool flags 2024-04-07T17:51:22.097 INFO:tasks.workunit.client.0.smithi100.stderr:basic_pool_options_bluestore REPORTING Per-pool bluestore config options 2024-04-07T17:51:22.097 INFO:tasks.workunit.client.0.smithi100.stderr:basic_pool_usage REPORTING Default pool application and usage statistics 2024-04-07T17:51:22.097 INFO:tasks.workunit.client.0.smithi100.stderr:basic_rook_v01 REPORTING Basic Rook deployment data 2024-04-07T17:51:22.097 INFO:tasks.workunit.client.0.smithi100.stderr:basic_usage_by_class REPORTING Default device class usage statistics 2024-04-07T17:51:22.097 INFO:tasks.workunit.client.0.smithi100.stderr:crash_base REPORTING Information about daemon crashes (daemon type and version, backtrace, etc.) 2024-04-07T17:51:22.097 INFO:tasks.workunit.client.0.smithi100.stderr:device_base REPORTING Information about device health metrics 2024-04-07T17:51:22.097 INFO:tasks.workunit.client.0.smithi100.stderr:ident_base REPORTING User-provided identifying information about the cluster 2024-04-07T17:51:22.097 INFO:tasks.workunit.client.0.smithi100.stderr:perf_memory_metrics REPORTING Heap stats and mempools for mon and mds 2024-04-07T17:51:22.097 INFO:tasks.workunit.client.0.smithi100.stderr:perf_perf REPORTING Information about performance counters of the cluster == *perf_perf* ]] 2024-04-07T17:51:22.097 INFO:tasks.workunit.client.0.smithi100.stderr:+ for col in ${NEW_COLLECTIONS[@]} 2024-04-07T17:51:22.097 INFO:tasks.workunit.client.0.smithi100.stderr:+ [[ New collections are available: 2024-04-07T17:51:22.098 INFO:tasks.workunit.client.0.smithi100.stderr:['basic_pool_flags'] 2024-04-07T17:51:22.098 INFO:tasks.workunit.client.0.smithi100.stderr:Run `ceph telemetry on` to opt-in to these collections. 2024-04-07T17:51:22.098 INFO:tasks.workunit.client.0.smithi100.stderr: 2024-04-07T17:51:22.098 INFO:tasks.workunit.client.0.smithi100.stderr:NAME STATUS DESC 2024-04-07T17:51:22.098 INFO:tasks.workunit.client.0.smithi100.stderr:basic_base REPORTING Basic information about the cluster (capacity, number and type of daemons, version, etc.) 2024-04-07T17:51:22.098 INFO:tasks.workunit.client.0.smithi100.stderr:basic_mds_metadata REPORTING MDS metadata 2024-04-07T17:51:22.098 INFO:tasks.workunit.client.0.smithi100.stderr:basic_pool_flags NOT REPORTING: NOT OPTED-IN Per-pool flags 2024-04-07T17:51:22.098 INFO:tasks.workunit.client.0.smithi100.stderr:basic_pool_options_bluestore REPORTING Per-pool bluestore config options 2024-04-07T17:51:22.098 INFO:tasks.workunit.client.0.smithi100.stderr:basic_pool_usage REPORTING Default pool application and usage statistics 2024-04-07T17:51:22.098 INFO:tasks.workunit.client.0.smithi100.stderr:basic_rook_v01 REPORTING Basic Rook deployment data 2024-04-07T17:51:22.098 INFO:tasks.workunit.client.0.smithi100.stderr:basic_usage_by_class REPORTING Default device class usage statistics 2024-04-07T17:51:22.098 INFO:tasks.workunit.client.0.smithi100.stderr:crash_base REPORTING Information about daemon crashes (daemon type and version, backtrace, etc.) 2024-04-07T17:51:22.098 INFO:tasks.workunit.client.0.smithi100.stderr:device_base REPORTING Information about device health metrics 2024-04-07T17:51:22.098 INFO:tasks.workunit.client.0.smithi100.stderr:ident_base REPORTING User-provided identifying information about the cluster 2024-04-07T17:51:22.098 INFO:tasks.workunit.client.0.smithi100.stderr:perf_memory_metrics REPORTING Heap stats and mempools for mon and mds 2024-04-07T17:51:22.099 INFO:tasks.workunit.client.0.smithi100.stderr:perf_perf REPORTING Information about performance counters of the cluster == *basic_mds_metadata* ]] 2024-04-07T17:51:22.099 INFO:tasks.workunit.client.0.smithi100.stderr:+ for col in ${NEW_COLLECTIONS[@]} 2024-04-07T17:51:22.099 INFO:tasks.workunit.client.0.smithi100.stderr:+ [[ New collections are available: 2024-04-07T17:51:22.099 INFO:tasks.workunit.client.0.smithi100.stderr:['basic_pool_flags'] 2024-04-07T17:51:22.099 INFO:tasks.workunit.client.0.smithi100.stderr:Run `ceph telemetry on` to opt-in to these collections. 2024-04-07T17:51:22.099 INFO:tasks.workunit.client.0.smithi100.stderr: 2024-04-07T17:51:22.099 INFO:tasks.workunit.client.0.smithi100.stderr:NAME STATUS DESC 2024-04-07T17:51:22.099 INFO:tasks.workunit.client.0.smithi100.stderr:basic_base REPORTING Basic information about the cluster (capacity, number and type of daemons, version, etc.) 2024-04-07T17:51:22.099 INFO:tasks.workunit.client.0.smithi100.stderr:basic_mds_metadata REPORTING MDS metadata 2024-04-07T17:51:22.099 INFO:tasks.workunit.client.0.smithi100.stderr:basic_pool_flags NOT REPORTING: NOT OPTED-IN Per-pool flags 2024-04-07T17:51:22.099 INFO:tasks.workunit.client.0.smithi100.stderr:basic_pool_options_bluestore REPORTING Per-pool bluestore config options 2024-04-07T17:51:22.099 INFO:tasks.workunit.client.0.smithi100.stderr:basic_pool_usage REPORTING Default pool application and usage statistics 2024-04-07T17:51:22.099 INFO:tasks.workunit.client.0.smithi100.stderr:basic_rook_v01 REPORTING Basic Rook deployment data 2024-04-07T17:51:22.099 INFO:tasks.workunit.client.0.smithi100.stderr:basic_usage_by_class REPORTING Default device class usage statistics 2024-04-07T17:51:22.099 INFO:tasks.workunit.client.0.smithi100.stderr:crash_base REPORTING Information about daemon crashes (daemon type and version, backtrace, etc.) 2024-04-07T17:51:22.100 INFO:tasks.workunit.client.0.smithi100.stderr:device_base REPORTING Information about device health metrics 2024-04-07T17:51:22.100 INFO:tasks.workunit.client.0.smithi100.stderr:ident_base REPORTING User-provided identifying information about the cluster 2024-04-07T17:51:22.100 INFO:tasks.workunit.client.0.smithi100.stderr:perf_memory_metrics REPORTING Heap stats and mempools for mon and mds 2024-04-07T17:51:22.100 INFO:tasks.workunit.client.0.smithi100.stderr:perf_perf REPORTING Information about performance counters of the cluster == *basic_pool_usage* ]] 2024-04-07T17:51:22.100 INFO:tasks.workunit.client.0.smithi100.stderr:+ for col in ${NEW_COLLECTIONS[@]} 2024-04-07T17:51:22.100 INFO:tasks.workunit.client.0.smithi100.stderr:+ [[ New collections are available: 2024-04-07T17:51:22.100 INFO:tasks.workunit.client.0.smithi100.stderr:['basic_pool_flags'] 2024-04-07T17:51:22.100 INFO:tasks.workunit.client.0.smithi100.stderr:Run `ceph telemetry on` to opt-in to these collections. 2024-04-07T17:51:22.100 INFO:tasks.workunit.client.0.smithi100.stderr: 2024-04-07T17:51:22.100 INFO:tasks.workunit.client.0.smithi100.stderr:NAME STATUS DESC 2024-04-07T17:51:22.100 INFO:tasks.workunit.client.0.smithi100.stderr:basic_base REPORTING Basic information about the cluster (capacity, number and type of daemons, version, etc.) 2024-04-07T17:51:22.100 INFO:tasks.workunit.client.0.smithi100.stderr:basic_mds_metadata REPORTING MDS metadata 2024-04-07T17:51:22.100 INFO:tasks.workunit.client.0.smithi100.stderr:basic_pool_flags NOT REPORTING: NOT OPTED-IN Per-pool flags 2024-04-07T17:51:22.100 INFO:tasks.workunit.client.0.smithi100.stderr:basic_pool_options_bluestore REPORTING Per-pool bluestore config options 2024-04-07T17:51:22.100 INFO:tasks.workunit.client.0.smithi100.stderr:basic_pool_usage REPORTING Default pool application and usage statistics 2024-04-07T17:51:22.100 INFO:tasks.workunit.client.0.smithi100.stderr:basic_rook_v01 REPORTING Basic Rook deployment data 2024-04-07T17:51:22.101 INFO:tasks.workunit.client.0.smithi100.stderr:basic_usage_by_class REPORTING Default device class usage statistics 2024-04-07T17:51:22.101 INFO:tasks.workunit.client.0.smithi100.stderr:crash_base REPORTING Information about daemon crashes (daemon type and version, backtrace, etc.) 2024-04-07T17:51:22.101 INFO:tasks.workunit.client.0.smithi100.stderr:device_base REPORTING Information about device health metrics 2024-04-07T17:51:22.101 INFO:tasks.workunit.client.0.smithi100.stderr:ident_base REPORTING User-provided identifying information about the cluster 2024-04-07T17:51:22.101 INFO:tasks.workunit.client.0.smithi100.stderr:perf_memory_metrics REPORTING Heap stats and mempools for mon and mds 2024-04-07T17:51:22.101 INFO:tasks.workunit.client.0.smithi100.stderr:perf_perf REPORTING Information about performance counters of the cluster == *basic_rook_v01* ]] 2024-04-07T17:51:22.101 INFO:tasks.workunit.client.0.smithi100.stderr:+ for col in ${NEW_COLLECTIONS[@]} 2024-04-07T17:51:22.101 INFO:tasks.workunit.client.0.smithi100.stderr:+ [[ New collections are available: 2024-04-07T17:51:22.101 INFO:tasks.workunit.client.0.smithi100.stderr:['basic_pool_flags'] 2024-04-07T17:51:22.101 INFO:tasks.workunit.client.0.smithi100.stderr:Run `ceph telemetry on` to opt-in to these collections. 2024-04-07T17:51:22.101 INFO:tasks.workunit.client.0.smithi100.stderr: 2024-04-07T17:51:22.101 INFO:tasks.workunit.client.0.smithi100.stderr:NAME STATUS DESC 2024-04-07T17:51:22.101 INFO:tasks.workunit.client.0.smithi100.stderr:basic_base REPORTING Basic information about the cluster (capacity, number and type of daemons, version, etc.) 2024-04-07T17:51:22.101 INFO:tasks.workunit.client.0.smithi100.stderr:basic_mds_metadata REPORTING MDS metadata 2024-04-07T17:51:22.101 INFO:tasks.workunit.client.0.smithi100.stderr:basic_pool_flags NOT REPORTING: NOT OPTED-IN Per-pool flags 2024-04-07T17:51:22.101 INFO:tasks.workunit.client.0.smithi100.stderr:basic_pool_options_bluestore REPORTING Per-pool bluestore config options 2024-04-07T17:51:22.101 INFO:tasks.workunit.client.0.smithi100.stderr:basic_pool_usage REPORTING Default pool application and usage statistics 2024-04-07T17:51:22.102 INFO:tasks.workunit.client.0.smithi100.stderr:basic_rook_v01 REPORTING Basic Rook deployment data 2024-04-07T17:51:22.102 INFO:tasks.workunit.client.0.smithi100.stderr:basic_usage_by_class REPORTING Default device class usage statistics 2024-04-07T17:51:22.102 INFO:tasks.workunit.client.0.smithi100.stderr:crash_base REPORTING Information about daemon crashes (daemon type and version, backtrace, etc.) 2024-04-07T17:51:22.102 INFO:tasks.workunit.client.0.smithi100.stderr:device_base REPORTING Information about device health metrics 2024-04-07T17:51:22.102 INFO:tasks.workunit.client.0.smithi100.stderr:ident_base REPORTING User-provided identifying information about the cluster 2024-04-07T17:51:22.102 INFO:tasks.workunit.client.0.smithi100.stderr:perf_memory_metrics REPORTING Heap stats and mempools for mon and mds 2024-04-07T17:51:22.102 INFO:tasks.workunit.client.0.smithi100.stderr:perf_perf REPORTING Information about performance counters of the cluster == *perf_memory_metrics* ]] 2024-04-07T17:51:22.102 INFO:tasks.workunit.client.0.smithi100.stderr:+ for col in ${NEW_COLLECTIONS[@]} 2024-04-07T17:51:22.102 INFO:tasks.workunit.client.0.smithi100.stderr:+ [[ New collections are available: 2024-04-07T17:51:22.102 INFO:tasks.workunit.client.0.smithi100.stderr:['basic_pool_flags'] 2024-04-07T17:51:22.102 INFO:tasks.workunit.client.0.smithi100.stderr:Run `ceph telemetry on` to opt-in to these collections. 2024-04-07T17:51:22.102 INFO:tasks.workunit.client.0.smithi100.stderr: 2024-04-07T17:51:22.102 INFO:tasks.workunit.client.0.smithi100.stderr:NAME STATUS DESC 2024-04-07T17:51:22.102 INFO:tasks.workunit.client.0.smithi100.stderr:basic_base REPORTING Basic information about the cluster (capacity, number and type of daemons, version, etc.) 2024-04-07T17:51:22.103 INFO:tasks.workunit.client.0.smithi100.stderr:basic_mds_metadata REPORTING MDS metadata 2024-04-07T17:51:22.103 INFO:tasks.workunit.client.0.smithi100.stderr:basic_pool_flags NOT REPORTING: NOT OPTED-IN Per-pool flags 2024-04-07T17:51:22.103 INFO:tasks.workunit.client.0.smithi100.stderr:basic_pool_options_bluestore REPORTING Per-pool bluestore config options 2024-04-07T17:51:22.103 INFO:tasks.workunit.client.0.smithi100.stderr:basic_pool_usage REPORTING Default pool application and usage statistics 2024-04-07T17:51:22.103 INFO:tasks.workunit.client.0.smithi100.stderr:basic_rook_v01 REPORTING Basic Rook deployment data 2024-04-07T17:51:22.103 INFO:tasks.workunit.client.0.smithi100.stderr:basic_usage_by_class REPORTING Default device class usage statistics 2024-04-07T17:51:22.103 INFO:tasks.workunit.client.0.smithi100.stderr:crash_base REPORTING Information about daemon crashes (daemon type and version, backtrace, etc.) 2024-04-07T17:51:22.103 INFO:tasks.workunit.client.0.smithi100.stderr:device_base REPORTING Information about device health metrics 2024-04-07T17:51:22.103 INFO:tasks.workunit.client.0.smithi100.stderr:ident_base REPORTING User-provided identifying information about the cluster 2024-04-07T17:51:22.103 INFO:tasks.workunit.client.0.smithi100.stderr:perf_memory_metrics REPORTING Heap stats and mempools for mon and mds 2024-04-07T17:51:22.103 INFO:tasks.workunit.client.0.smithi100.stderr:perf_perf REPORTING Information about performance counters of the cluster == *basic_pool_flags* ]] 2024-04-07T17:51:22.103 INFO:tasks.workunit.client.0.smithi100.stderr:+ ceph telemetry preview 2024-04-07T17:51:22.364 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:51:22 smithi144 bash[95233]: ts=2024-04-07T17:51:22.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:51:22.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:22 smithi144 bash[66654]: cluster 2024-04-07T17:51:20.827670+0000 mgr.y (mgr.25175) 2879 : cluster [DBG] pgmap v5167: 105 pgs: 105 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail 2024-04-07T17:51:22.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:22 smithi144 bash[66654]: audit 2024-04-07T17:51:21.397616+0000 mon.c (mon.2) 119 : audit [DBG] from='client.? 172.21.15.100:0/112895494' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-04-07T17:51:22.365 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:22 smithi144 bash[66654]: audit 2024-04-07T17:51:21.565862+0000 mon.c (mon.2) 120 : audit [DBG] from='client.? 172.21.15.100:0/112895494' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-04-07T17:51:22.453 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:22 smithi100 bash[74942]: cluster 2024-04-07T17:51:20.827670+0000 mgr.y (mgr.25175) 2879 : cluster [DBG] pgmap v5167: 105 pgs: 105 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail 2024-04-07T17:51:22.453 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:22 smithi100 bash[74942]: audit 2024-04-07T17:51:21.397616+0000 mon.c (mon.2) 119 : audit [DBG] from='client.? 172.21.15.100:0/112895494' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-04-07T17:51:22.453 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:22 smithi100 bash[74942]: audit 2024-04-07T17:51:21.565862+0000 mon.c (mon.2) 120 : audit [DBG] from='client.? 172.21.15.100:0/112895494' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-04-07T17:51:22.453 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:22 smithi100 bash[76202]: cluster 2024-04-07T17:51:20.827670+0000 mgr.y (mgr.25175) 2879 : cluster [DBG] pgmap v5167: 105 pgs: 105 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail 2024-04-07T17:51:22.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:22 smithi100 bash[76202]: audit 2024-04-07T17:51:21.397616+0000 mon.c (mon.2) 119 : audit [DBG] from='client.? 172.21.15.100:0/112895494' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-04-07T17:51:22.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:22 smithi100 bash[76202]: audit 2024-04-07T17:51:21.565862+0000 mon.c (mon.2) 120 : audit [DBG] from='client.? 172.21.15.100:0/112895494' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-04-07T17:51:23.352 INFO:tasks.workunit.client.0.smithi100.stdout:{ 2024-04-07T17:51:23.352 INFO:tasks.workunit.client.0.smithi100.stdout: "balancer": { 2024-04-07T17:51:23.352 INFO:tasks.workunit.client.0.smithi100.stdout: "active": true, 2024-04-07T17:51:23.352 INFO:tasks.workunit.client.0.smithi100.stdout: "mode": "" 2024-04-07T17:51:23.352 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.352 INFO:tasks.workunit.client.0.smithi100.stdout: "channels": [ 2024-04-07T17:51:23.352 INFO:tasks.workunit.client.0.smithi100.stdout: "basic", 2024-04-07T17:51:23.352 INFO:tasks.workunit.client.0.smithi100.stdout: "crash", 2024-04-07T17:51:23.352 INFO:tasks.workunit.client.0.smithi100.stdout: "device", 2024-04-07T17:51:23.352 INFO:tasks.workunit.client.0.smithi100.stdout: "ident", 2024-04-07T17:51:23.352 INFO:tasks.workunit.client.0.smithi100.stdout: "perf" 2024-04-07T17:51:23.353 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:23.353 INFO:tasks.workunit.client.0.smithi100.stdout: "channels_available": [ 2024-04-07T17:51:23.353 INFO:tasks.workunit.client.0.smithi100.stdout: "basic", 2024-04-07T17:51:23.353 INFO:tasks.workunit.client.0.smithi100.stdout: "ident", 2024-04-07T17:51:23.353 INFO:tasks.workunit.client.0.smithi100.stdout: "crash", 2024-04-07T17:51:23.353 INFO:tasks.workunit.client.0.smithi100.stdout: "device", 2024-04-07T17:51:23.353 INFO:tasks.workunit.client.0.smithi100.stdout: "perf" 2024-04-07T17:51:23.353 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:23.353 INFO:tasks.workunit.client.0.smithi100.stdout: "collections_available": [ 2024-04-07T17:51:23.353 INFO:tasks.workunit.client.0.smithi100.stdout: "basic_base", 2024-04-07T17:51:23.353 INFO:tasks.workunit.client.0.smithi100.stdout: "device_base", 2024-04-07T17:51:23.353 INFO:tasks.workunit.client.0.smithi100.stdout: "crash_base", 2024-04-07T17:51:23.353 INFO:tasks.workunit.client.0.smithi100.stdout: "ident_base", 2024-04-07T17:51:23.353 INFO:tasks.workunit.client.0.smithi100.stdout: "perf_perf", 2024-04-07T17:51:23.353 INFO:tasks.workunit.client.0.smithi100.stdout: "basic_mds_metadata", 2024-04-07T17:51:23.354 INFO:tasks.workunit.client.0.smithi100.stdout: "basic_pool_usage", 2024-04-07T17:51:23.354 INFO:tasks.workunit.client.0.smithi100.stdout: "basic_usage_by_class", 2024-04-07T17:51:23.354 INFO:tasks.workunit.client.0.smithi100.stdout: "basic_rook_v01", 2024-04-07T17:51:23.354 INFO:tasks.workunit.client.0.smithi100.stdout: "perf_memory_metrics", 2024-04-07T17:51:23.354 INFO:tasks.workunit.client.0.smithi100.stdout: "basic_pool_options_bluestore", 2024-04-07T17:51:23.354 INFO:tasks.workunit.client.0.smithi100.stdout: "basic_pool_flags" 2024-04-07T17:51:23.354 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:23.354 INFO:tasks.workunit.client.0.smithi100.stdout: "collections_opted_in": [ 2024-04-07T17:51:23.354 INFO:tasks.workunit.client.0.smithi100.stdout: "basic_base", 2024-04-07T17:51:23.354 INFO:tasks.workunit.client.0.smithi100.stdout: "device_base", 2024-04-07T17:51:23.354 INFO:tasks.workunit.client.0.smithi100.stdout: "crash_base", 2024-04-07T17:51:23.354 INFO:tasks.workunit.client.0.smithi100.stdout: "ident_base", 2024-04-07T17:51:23.354 INFO:tasks.workunit.client.0.smithi100.stdout: "perf_perf", 2024-04-07T17:51:23.355 INFO:tasks.workunit.client.0.smithi100.stdout: "basic_mds_metadata", 2024-04-07T17:51:23.355 INFO:tasks.workunit.client.0.smithi100.stdout: "basic_pool_usage", 2024-04-07T17:51:23.355 INFO:tasks.workunit.client.0.smithi100.stdout: "basic_usage_by_class", 2024-04-07T17:51:23.355 INFO:tasks.workunit.client.0.smithi100.stdout: "basic_rook_v01", 2024-04-07T17:51:23.355 INFO:tasks.workunit.client.0.smithi100.stdout: "perf_memory_metrics", 2024-04-07T17:51:23.355 INFO:tasks.workunit.client.0.smithi100.stdout: "basic_pool_options_bluestore", 2024-04-07T17:51:23.355 INFO:tasks.workunit.client.0.smithi100.stdout: "basic_pool_flags" 2024-04-07T17:51:23.355 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:23.355 INFO:tasks.workunit.client.0.smithi100.stdout: "config": { 2024-04-07T17:51:23.355 INFO:tasks.workunit.client.0.smithi100.stdout: "active_changed": [ 2024-04-07T17:51:23.355 INFO:tasks.workunit.client.0.smithi100.stdout: "auth_allow_insecure_global_id_reclaim", 2024-04-07T17:51:23.355 INFO:tasks.workunit.client.0.smithi100.stdout: "auth_debug", 2024-04-07T17:51:23.355 INFO:tasks.workunit.client.0.smithi100.stdout: "auth_mon_ticket_ttl", 2024-04-07T17:51:23.355 INFO:tasks.workunit.client.0.smithi100.stdout: "auth_service_ticket_ttl", 2024-04-07T17:51:23.356 INFO:tasks.workunit.client.0.smithi100.stdout: "bdev_debug_aio", 2024-04-07T17:51:23.356 INFO:tasks.workunit.client.0.smithi100.stdout: "client_check_pool_perm", 2024-04-07T17:51:23.356 INFO:tasks.workunit.client.0.smithi100.stdout: "client_mount_gid", 2024-04-07T17:51:23.356 INFO:tasks.workunit.client.0.smithi100.stdout: "client_mount_uid", 2024-04-07T17:51:23.356 INFO:tasks.workunit.client.0.smithi100.stdout: "client_quota", 2024-04-07T17:51:23.356 INFO:tasks.workunit.client.0.smithi100.stdout: "container_image", 2024-04-07T17:51:23.356 INFO:tasks.workunit.client.0.smithi100.stdout: "daemonize", 2024-04-07T17:51:23.356 INFO:tasks.workunit.client.0.smithi100.stdout: "debug_asserts_on_shutdown", 2024-04-07T17:51:23.356 INFO:tasks.workunit.client.0.smithi100.stdout: "debug_mgr", 2024-04-07T17:51:23.356 INFO:tasks.workunit.client.0.smithi100.stdout: "debug_mon", 2024-04-07T17:51:23.356 INFO:tasks.workunit.client.0.smithi100.stdout: "debug_ms", 2024-04-07T17:51:23.356 INFO:tasks.workunit.client.0.smithi100.stdout: "debug_osd", 2024-04-07T17:51:23.356 INFO:tasks.workunit.client.0.smithi100.stdout: "debug_paxos", 2024-04-07T17:51:23.356 INFO:tasks.workunit.client.0.smithi100.stdout: "keyring", 2024-04-07T17:51:23.356 INFO:tasks.workunit.client.0.smithi100.stdout: "log_stderr_prefix", 2024-04-07T17:51:23.356 INFO:tasks.workunit.client.0.smithi100.stdout: "log_to_file", 2024-04-07T17:51:23.357 INFO:tasks.workunit.client.0.smithi100.stdout: "log_to_journald", 2024-04-07T17:51:23.357 INFO:tasks.workunit.client.0.smithi100.stdout: "log_to_stderr", 2024-04-07T17:51:23.357 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_allow_pool_delete", 2024-04-07T17:51:23.357 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_clock_drift_allowed", 2024-04-07T17:51:23.357 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_cluster_log_file_level", 2024-04-07T17:51:23.357 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_cluster_log_to_file", 2024-04-07T17:51:23.357 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_cluster_log_to_stderr", 2024-04-07T17:51:23.357 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_data_avail_warn", 2024-04-07T17:51:23.357 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_election_default_strategy", 2024-04-07T17:51:23.357 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_host", 2024-04-07T17:51:23.357 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_max_pg_per_osd", 2024-04-07T17:51:23.357 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_mds_skip_sanity", 2024-04-07T17:51:23.357 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_mgr_mkfs_grace", 2024-04-07T17:51:23.358 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_osd_prime_pg_temp", 2024-04-07T17:51:23.358 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_osd_reporter_subtree_level", 2024-04-07T17:51:23.358 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_pg_warn_max_object_skew", 2024-04-07T17:51:23.358 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_reweight_min_bytes_per_osd", 2024-04-07T17:51:23.358 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_reweight_min_pgs_per_osd", 2024-04-07T17:51:23.358 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_warn_on_crush_straw_calc_version_zero", 2024-04-07T17:51:23.358 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_warn_on_insecure_global_id_reclaim", 2024-04-07T17:51:23.358 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_warn_on_insecure_global_id_reclaim_allowed", 2024-04-07T17:51:23.358 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_warn_on_legacy_crush_tunables", 2024-04-07T17:51:23.358 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_warn_on_osd_down_out_interval_zero", 2024-04-07T17:51:23.358 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_warn_on_pool_pg_num_not_power_of_two", 2024-04-07T17:51:23.358 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_warn_on_too_few_osds", 2024-04-07T17:51:23.359 INFO:tasks.workunit.client.0.smithi100.stdout: "ms_die_on_bug", 2024-04-07T17:51:23.359 INFO:tasks.workunit.client.0.smithi100.stdout: "ms_die_on_old_message", 2024-04-07T17:51:23.359 INFO:tasks.workunit.client.0.smithi100.stdout: "no_config_file", 2024-04-07T17:51:23.359 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_class_default_list", 2024-04-07T17:51:23.359 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_class_load_list", 2024-04-07T17:51:23.359 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_debug_misdirected_ops", 2024-04-07T17:51:23.359 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_debug_op_order", 2024-04-07T17:51:23.359 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_debug_pg_log_writeout", 2024-04-07T17:51:23.359 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_debug_shutdown", 2024-04-07T17:51:23.359 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_debug_verify_cached_snaps", 2024-04-07T17:51:23.359 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_debug_verify_missing_on_start", 2024-04-07T17:51:23.359 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_debug_verify_stray_on_activate", 2024-04-07T17:51:23.359 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_deep_scrub_update_digest_min_age", 2024-04-07T17:51:23.359 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_delete_sleep", 2024-04-07T17:51:23.359 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_delete_sleep_hdd", 2024-04-07T17:51:23.360 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_delete_sleep_hybrid", 2024-04-07T17:51:23.360 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_delete_sleep_ssd", 2024-04-07T17:51:23.360 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_max_backfills", 2024-04-07T17:51:23.360 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mclock_max_capacity_iops_ssd", 2024-04-07T17:51:23.360 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mclock_profile", 2024-04-07T17:51:23.360 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mclock_scheduler_background_best_effort_lim", 2024-04-07T17:51:23.360 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mclock_scheduler_background_best_effort_res", 2024-04-07T17:51:23.360 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mclock_scheduler_background_best_effort_wgt", 2024-04-07T17:51:23.360 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mclock_scheduler_background_recovery_lim", 2024-04-07T17:51:23.360 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mclock_scheduler_background_recovery_res", 2024-04-07T17:51:23.360 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mclock_scheduler_background_recovery_wgt", 2024-04-07T17:51:23.360 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mclock_scheduler_client_lim", 2024-04-07T17:51:23.360 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mclock_scheduler_client_res", 2024-04-07T17:51:23.361 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mclock_scheduler_client_wgt", 2024-04-07T17:51:23.361 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_memory_target", 2024-04-07T17:51:23.361 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_memory_target_autotune", 2024-04-07T17:51:23.361 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_op_queue", 2024-04-07T17:51:23.361 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_op_queue_cut_off", 2024-04-07T17:51:23.361 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_pool_default_erasure_code_profile", 2024-04-07T17:51:23.361 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_pool_default_pg_autoscale_mode", 2024-04-07T17:51:23.361 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_recover_clone_overlap", 2024-04-07T17:51:23.361 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_recovery_max_active", 2024-04-07T17:51:23.361 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_recovery_max_active_hdd", 2024-04-07T17:51:23.361 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_recovery_max_active_ssd", 2024-04-07T17:51:23.361 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_recovery_max_chunk", 2024-04-07T17:51:23.361 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_recovery_sleep", 2024-04-07T17:51:23.361 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_recovery_sleep_hdd", 2024-04-07T17:51:23.362 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_recovery_sleep_hybrid", 2024-04-07T17:51:23.362 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_recovery_sleep_ssd", 2024-04-07T17:51:23.362 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_scrub_load_threshold", 2024-04-07T17:51:23.362 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_scrub_max_interval", 2024-04-07T17:51:23.362 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_scrub_sleep", 2024-04-07T17:51:23.362 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_shutdown_pgref_assert", 2024-04-07T17:51:23.362 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_snap_trim_sleep", 2024-04-07T17:51:23.362 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_snap_trim_sleep_hdd", 2024-04-07T17:51:23.362 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_snap_trim_sleep_hybrid", 2024-04-07T17:51:23.362 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_snap_trim_sleep_ssd", 2024-04-07T17:51:23.362 INFO:tasks.workunit.client.0.smithi100.stdout: "public_network", 2024-04-07T17:51:23.362 INFO:tasks.workunit.client.0.smithi100.stdout: "rbd_config_pool_override_update_timestamp", 2024-04-07T17:51:23.362 INFO:tasks.workunit.client.0.smithi100.stdout: "rbd_default_features", 2024-04-07T17:51:23.362 INFO:tasks.workunit.client.0.smithi100.stdout: "rbd_qos_exclude_ops", 2024-04-07T17:51:23.363 INFO:tasks.workunit.client.0.smithi100.stdout: "setgroup", 2024-04-07T17:51:23.363 INFO:tasks.workunit.client.0.smithi100.stdout: "setuser" 2024-04-07T17:51:23.363 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:23.363 INFO:tasks.workunit.client.0.smithi100.stdout: "cluster_changed": [ 2024-04-07T17:51:23.363 INFO:tasks.workunit.client.0.smithi100.stdout: "auth_allow_insecure_global_id_reclaim", 2024-04-07T17:51:23.363 INFO:tasks.workunit.client.0.smithi100.stdout: "auth_debug", 2024-04-07T17:51:23.363 INFO:tasks.workunit.client.0.smithi100.stdout: "auth_mon_ticket_ttl", 2024-04-07T17:51:23.363 INFO:tasks.workunit.client.0.smithi100.stdout: "auth_service_ticket_ttl", 2024-04-07T17:51:23.363 INFO:tasks.workunit.client.0.smithi100.stdout: "bdev_debug_aio", 2024-04-07T17:51:23.363 INFO:tasks.workunit.client.0.smithi100.stdout: "container_image", 2024-04-07T17:51:23.363 INFO:tasks.workunit.client.0.smithi100.stdout: "debug_asserts_on_shutdown", 2024-04-07T17:51:23.363 INFO:tasks.workunit.client.0.smithi100.stdout: "debug_mgr", 2024-04-07T17:51:23.363 INFO:tasks.workunit.client.0.smithi100.stdout: "debug_mon", 2024-04-07T17:51:23.363 INFO:tasks.workunit.client.0.smithi100.stdout: "debug_ms", 2024-04-07T17:51:23.363 INFO:tasks.workunit.client.0.smithi100.stdout: "debug_osd", 2024-04-07T17:51:23.364 INFO:tasks.workunit.client.0.smithi100.stdout: "debug_paxos", 2024-04-07T17:51:23.364 INFO:tasks.workunit.client.0.smithi100.stdout: "log_to_file", 2024-04-07T17:51:23.364 INFO:tasks.workunit.client.0.smithi100.stdout: "log_to_journald", 2024-04-07T17:51:23.364 INFO:tasks.workunit.client.0.smithi100.stdout: "log_to_stderr", 2024-04-07T17:51:23.364 INFO:tasks.workunit.client.0.smithi100.stdout: "mds_join_fs", 2024-04-07T17:51:23.364 INFO:tasks.workunit.client.0.smithi100.stdout: "mgr/cephadm/allow_ptrace", 2024-04-07T17:51:23.364 INFO:tasks.workunit.client.0.smithi100.stdout: "mgr/cephadm/container_init", 2024-04-07T17:51:23.364 INFO:tasks.workunit.client.0.smithi100.stdout: "mgr/cephadm/migration_current", 2024-04-07T17:51:23.364 INFO:tasks.workunit.client.0.smithi100.stdout: "mgr/cephadm/use_repo_digest", 2024-04-07T17:51:23.364 INFO:tasks.workunit.client.0.smithi100.stdout: "mgr/dashboard/ALERTMANAGER_API_HOST", 2024-04-07T17:51:23.364 INFO:tasks.workunit.client.0.smithi100.stdout: "mgr/dashboard/GRAFANA_API_SSL_VERIFY", 2024-04-07T17:51:23.364 INFO:tasks.workunit.client.0.smithi100.stdout: "mgr/dashboard/GRAFANA_API_URL", 2024-04-07T17:51:23.364 INFO:tasks.workunit.client.0.smithi100.stdout: "mgr/dashboard/PROMETHEUS_API_HOST", 2024-04-07T17:51:23.364 INFO:tasks.workunit.client.0.smithi100.stdout: "mgr/dashboard/ssl_server_port", 2024-04-07T17:51:23.365 INFO:tasks.workunit.client.0.smithi100.stdout: "mgr/orchestrator/orchestrator", 2024-04-07T17:51:23.365 INFO:tasks.workunit.client.0.smithi100.stdout: "mgr/telemetry/channel_ident", 2024-04-07T17:51:23.365 INFO:tasks.workunit.client.0.smithi100.stdout: "mgr/telemetry/channel_perf", 2024-04-07T17:51:23.365 INFO:tasks.workunit.client.0.smithi100.stdout: "mgr/telemetry/description", 2024-04-07T17:51:23.365 INFO:tasks.workunit.client.0.smithi100.stdout: "mgr/telemetry/enabled", 2024-04-07T17:51:23.365 INFO:tasks.workunit.client.0.smithi100.stdout: "mgr/telemetry/organization", 2024-04-07T17:51:23.365 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_allow_pool_delete", 2024-04-07T17:51:23.365 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_clock_drift_allowed", 2024-04-07T17:51:23.365 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_cluster_log_file_level", 2024-04-07T17:51:23.365 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_cluster_log_to_file", 2024-04-07T17:51:23.365 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_data_avail_warn", 2024-04-07T17:51:23.365 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_election_default_strategy", 2024-04-07T17:51:23.365 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_max_pg_per_osd", 2024-04-07T17:51:23.365 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_mds_skip_sanity", 2024-04-07T17:51:23.365 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_mgr_mkfs_grace", 2024-04-07T17:51:23.366 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_osd_prime_pg_temp", 2024-04-07T17:51:23.366 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_osd_reporter_subtree_level", 2024-04-07T17:51:23.366 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_pg_warn_max_object_skew", 2024-04-07T17:51:23.366 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_reweight_min_bytes_per_osd", 2024-04-07T17:51:23.366 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_reweight_min_pgs_per_osd", 2024-04-07T17:51:23.366 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_warn_on_crush_straw_calc_version_zero", 2024-04-07T17:51:23.366 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_warn_on_insecure_global_id_reclaim", 2024-04-07T17:51:23.366 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_warn_on_insecure_global_id_reclaim_allowed", 2024-04-07T17:51:23.366 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_warn_on_legacy_crush_tunables", 2024-04-07T17:51:23.366 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_warn_on_osd_down_out_interval_zero", 2024-04-07T17:51:23.366 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_warn_on_pool_pg_num_not_power_of_two", 2024-04-07T17:51:23.366 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_warn_on_too_few_osds", 2024-04-07T17:51:23.367 INFO:tasks.workunit.client.0.smithi100.stdout: "ms_die_on_bug", 2024-04-07T17:51:23.367 INFO:tasks.workunit.client.0.smithi100.stdout: "ms_die_on_old_message", 2024-04-07T17:51:23.367 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_class_default_list", 2024-04-07T17:51:23.367 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_class_load_list", 2024-04-07T17:51:23.367 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_debug_misdirected_ops", 2024-04-07T17:51:23.367 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_debug_op_order", 2024-04-07T17:51:23.367 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_debug_pg_log_writeout", 2024-04-07T17:51:23.367 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_debug_shutdown", 2024-04-07T17:51:23.367 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_debug_verify_cached_snaps", 2024-04-07T17:51:23.367 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_debug_verify_missing_on_start", 2024-04-07T17:51:23.367 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_debug_verify_stray_on_activate", 2024-04-07T17:51:23.367 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_deep_scrub_update_digest_min_age", 2024-04-07T17:51:23.367 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mclock_max_capacity_iops_ssd", 2024-04-07T17:51:23.367 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mclock_profile", 2024-04-07T17:51:23.367 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_memory_target", 2024-04-07T17:51:23.368 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_memory_target_autotune", 2024-04-07T17:51:23.368 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_op_queue", 2024-04-07T17:51:23.368 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_op_queue_cut_off", 2024-04-07T17:51:23.368 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_pool_default_erasure_code_profile", 2024-04-07T17:51:23.368 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_pool_default_pg_autoscale_mode", 2024-04-07T17:51:23.368 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_recover_clone_overlap", 2024-04-07T17:51:23.368 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_recovery_max_chunk", 2024-04-07T17:51:23.368 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_scrub_load_threshold", 2024-04-07T17:51:23.368 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_scrub_max_interval", 2024-04-07T17:51:23.368 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_shutdown_pgref_assert", 2024-04-07T17:51:23.368 INFO:tasks.workunit.client.0.smithi100.stdout: "public_network", 2024-04-07T17:51:23.368 INFO:tasks.workunit.client.0.smithi100.stdout: "rbd_config_pool_override_update_timestamp", 2024-04-07T17:51:23.368 INFO:tasks.workunit.client.0.smithi100.stdout: "rgw_cache_enabled", 2024-04-07T17:51:23.368 INFO:tasks.workunit.client.0.smithi100.stdout: "rgw_enable_ops_log", 2024-04-07T17:51:23.369 INFO:tasks.workunit.client.0.smithi100.stdout: "rgw_enable_usage_log" 2024-04-07T17:51:23.369 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:23.369 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.369 INFO:tasks.workunit.client.0.smithi100.stdout: "contact": null, 2024-04-07T17:51:23.369 INFO:tasks.workunit.client.0.smithi100.stdout: "crashes": [], 2024-04-07T17:51:23.369 INFO:tasks.workunit.client.0.smithi100.stdout: "created": "2024-04-07T16:02:17.280630Z", 2024-04-07T17:51:23.369 INFO:tasks.workunit.client.0.smithi100.stdout: "crush": { 2024-04-07T17:51:23.369 INFO:tasks.workunit.client.0.smithi100.stdout: "bucket_algs": { 2024-04-07T17:51:23.369 INFO:tasks.workunit.client.0.smithi100.stdout: "straw2": 3 2024-04-07T17:51:23.369 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.369 INFO:tasks.workunit.client.0.smithi100.stdout: "bucket_sizes": { 2024-04-07T17:51:23.369 INFO:tasks.workunit.client.0.smithi100.stdout: "2": 1, 2024-04-07T17:51:23.369 INFO:tasks.workunit.client.0.smithi100.stdout: "4": 2 2024-04-07T17:51:23.369 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.369 INFO:tasks.workunit.client.0.smithi100.stdout: "bucket_types": { 2024-04-07T17:51:23.370 INFO:tasks.workunit.client.0.smithi100.stdout: "1": 2, 2024-04-07T17:51:23.370 INFO:tasks.workunit.client.0.smithi100.stdout: "11": 1 2024-04-07T17:51:23.370 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.370 INFO:tasks.workunit.client.0.smithi100.stdout: "compat_weight_set": false, 2024-04-07T17:51:23.370 INFO:tasks.workunit.client.0.smithi100.stdout: "device_classes": [ 2024-04-07T17:51:23.370 INFO:tasks.workunit.client.0.smithi100.stdout: 8 2024-04-07T17:51:23.370 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:23.370 INFO:tasks.workunit.client.0.smithi100.stdout: "num_buckets": 6, 2024-04-07T17:51:23.370 INFO:tasks.workunit.client.0.smithi100.stdout: "num_devices": 8, 2024-04-07T17:51:23.370 INFO:tasks.workunit.client.0.smithi100.stdout: "num_rules": 2, 2024-04-07T17:51:23.370 INFO:tasks.workunit.client.0.smithi100.stdout: "num_types": 12, 2024-04-07T17:51:23.370 INFO:tasks.workunit.client.0.smithi100.stdout: "num_weight_sets": 0, 2024-04-07T17:51:23.370 INFO:tasks.workunit.client.0.smithi100.stdout: "tunables": { 2024-04-07T17:51:23.371 INFO:tasks.workunit.client.0.smithi100.stdout: "allowed_bucket_algs": 54, 2024-04-07T17:51:23.371 INFO:tasks.workunit.client.0.smithi100.stdout: "choose_local_fallback_tries": 0, 2024-04-07T17:51:23.371 INFO:tasks.workunit.client.0.smithi100.stdout: "choose_local_tries": 0, 2024-04-07T17:51:23.371 INFO:tasks.workunit.client.0.smithi100.stdout: "choose_total_tries": 50, 2024-04-07T17:51:23.371 INFO:tasks.workunit.client.0.smithi100.stdout: "chooseleaf_descend_once": 1, 2024-04-07T17:51:23.371 INFO:tasks.workunit.client.0.smithi100.stdout: "chooseleaf_stable": 1, 2024-04-07T17:51:23.371 INFO:tasks.workunit.client.0.smithi100.stdout: "chooseleaf_vary_r": 1, 2024-04-07T17:51:23.371 INFO:tasks.workunit.client.0.smithi100.stdout: "has_msr_rules": 0, 2024-04-07T17:51:23.371 INFO:tasks.workunit.client.0.smithi100.stdout: "has_v2_rules": 1, 2024-04-07T17:51:23.371 INFO:tasks.workunit.client.0.smithi100.stdout: "has_v3_rules": 0, 2024-04-07T17:51:23.371 INFO:tasks.workunit.client.0.smithi100.stdout: "has_v4_buckets": 1, 2024-04-07T17:51:23.371 INFO:tasks.workunit.client.0.smithi100.stdout: "has_v5_rules": 0, 2024-04-07T17:51:23.371 INFO:tasks.workunit.client.0.smithi100.stdout: "legacy_tunables": 0, 2024-04-07T17:51:23.371 INFO:tasks.workunit.client.0.smithi100.stdout: "minimum_required_version": "jewel", 2024-04-07T17:51:23.371 INFO:tasks.workunit.client.0.smithi100.stdout: "msr_collision_tries": 100, 2024-04-07T17:51:23.372 INFO:tasks.workunit.client.0.smithi100.stdout: "msr_descents": 100, 2024-04-07T17:51:23.372 INFO:tasks.workunit.client.0.smithi100.stdout: "optimal_tunables": 1, 2024-04-07T17:51:23.372 INFO:tasks.workunit.client.0.smithi100.stdout: "profile": "jewel", 2024-04-07T17:51:23.372 INFO:tasks.workunit.client.0.smithi100.stdout: "require_feature_tunables": 1, 2024-04-07T17:51:23.372 INFO:tasks.workunit.client.0.smithi100.stdout: "require_feature_tunables2": 1, 2024-04-07T17:51:23.372 INFO:tasks.workunit.client.0.smithi100.stdout: "require_feature_tunables3": 1, 2024-04-07T17:51:23.372 INFO:tasks.workunit.client.0.smithi100.stdout: "require_feature_tunables5": 1, 2024-04-07T17:51:23.372 INFO:tasks.workunit.client.0.smithi100.stdout: "straw_calc_version": 1 2024-04-07T17:51:23.372 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:23.372 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.372 INFO:tasks.workunit.client.0.smithi100.stdout: "description": "upgrade test cluster", 2024-04-07T17:51:23.372 INFO:tasks.workunit.client.0.smithi100.stdout: "fs": { 2024-04-07T17:51:23.373 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1, 2024-04-07T17:51:23.373 INFO:tasks.workunit.client.0.smithi100.stdout: "feature_flags": { 2024-04-07T17:51:23.373 INFO:tasks.workunit.client.0.smithi100.stdout: "enable_multiple": true, 2024-04-07T17:51:23.373 INFO:tasks.workunit.client.0.smithi100.stdout: "ever_enabled_multiple": true 2024-04-07T17:51:23.373 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.373 INFO:tasks.workunit.client.0.smithi100.stdout: "filesystems": [ 2024-04-07T17:51:23.373 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:23.373 INFO:tasks.workunit.client.0.smithi100.stdout: "approx_ctime": "2024-04", 2024-04-07T17:51:23.373 INFO:tasks.workunit.client.0.smithi100.stdout: "balancer_enabled": false, 2024-04-07T17:51:23.373 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.373 INFO:tasks.workunit.client.0.smithi100.stdout: "cached_caps": 0, 2024-04-07T17:51:23.373 INFO:tasks.workunit.client.0.smithi100.stdout: "cached_dns": 10, 2024-04-07T17:51:23.373 INFO:tasks.workunit.client.0.smithi100.stdout: "cached_inos": 13, 2024-04-07T17:51:23.374 INFO:tasks.workunit.client.0.smithi100.stdout: "cached_subtrees": 2, 2024-04-07T17:51:23.374 INFO:tasks.workunit.client.0.smithi100.stdout: "ever_allowed_features": 0, 2024-04-07T17:51:23.374 INFO:tasks.workunit.client.0.smithi100.stdout: "explicitly_allowed_features": 0, 2024-04-07T17:51:23.374 INFO:tasks.workunit.client.0.smithi100.stdout: "files": 0, 2024-04-07T17:51:23.374 INFO:tasks.workunit.client.0.smithi100.stdout: "max_mds": 1, 2024-04-07T17:51:23.374 INFO:tasks.workunit.client.0.smithi100.stdout: "num_data_pools": 1, 2024-04-07T17:51:23.374 INFO:tasks.workunit.client.0.smithi100.stdout: "num_in": 1, 2024-04-07T17:51:23.374 INFO:tasks.workunit.client.0.smithi100.stdout: "num_mds": 1, 2024-04-07T17:51:23.374 INFO:tasks.workunit.client.0.smithi100.stdout: "num_sessions": 0, 2024-04-07T17:51:23.374 INFO:tasks.workunit.client.0.smithi100.stdout: "num_standby_replay": 0, 2024-04-07T17:51:23.374 INFO:tasks.workunit.client.0.smithi100.stdout: "num_up": 1, 2024-04-07T17:51:23.374 INFO:tasks.workunit.client.0.smithi100.stdout: "snaps": 0, 2024-04-07T17:51:23.374 INFO:tasks.workunit.client.0.smithi100.stdout: "standby_count_wanted": 0 2024-04-07T17:51:23.374 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:23.375 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:23.375 INFO:tasks.workunit.client.0.smithi100.stdout: "num_standby_mds": 1, 2024-04-07T17:51:23.375 INFO:tasks.workunit.client.0.smithi100.stdout: "total_num_mds": 2 2024-04-07T17:51:23.375 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.375 INFO:tasks.workunit.client.0.smithi100.stdout: "heap_stats": { 2024-04-07T17:51:23.375 INFO:tasks.workunit.client.0.smithi100.stdout: "mds": { 2024-04-07T17:51:23.375 INFO:tasks.workunit.client.0.smithi100.stdout: "mds.a52fc0d8acb515431879057649ff294cfe14c620": { 2024-04-07T17:51:23.375 INFO:tasks.workunit.client.0.smithi100.stdout: "actual_memory_used": 20258816, 2024-04-07T17:51:23.375 INFO:tasks.workunit.client.0.smithi100.stdout: "central_cache_freelist": 392176, 2024-04-07T17:51:23.375 INFO:tasks.workunit.client.0.smithi100.stdout: "malloc_metadata": 2752512, 2024-04-07T17:51:23.375 INFO:tasks.workunit.client.0.smithi100.stdout: "page_heap_freelist": 884736, 2024-04-07T17:51:23.375 INFO:tasks.workunit.client.0.smithi100.stdout: "released_to_os": 0, 2024-04-07T17:51:23.375 INFO:tasks.workunit.client.0.smithi100.stdout: "spans_in_use": 339, 2024-04-07T17:51:23.375 INFO:tasks.workunit.client.0.smithi100.stdout: "tcmalloc_page_size": 8192, 2024-04-07T17:51:23.375 INFO:tasks.workunit.client.0.smithi100.stdout: "thread_cache_freelists": 1114120, 2024-04-07T17:51:23.376 INFO:tasks.workunit.client.0.smithi100.stdout: "thread_heaps_in_use": 13, 2024-04-07T17:51:23.376 INFO:tasks.workunit.client.0.smithi100.stdout: "transfer_cache_freelist": 632832, 2024-04-07T17:51:23.376 INFO:tasks.workunit.client.0.smithi100.stdout: "use_by_application": 14482440, 2024-04-07T17:51:23.376 INFO:tasks.workunit.client.0.smithi100.stdout: "virtual_address_space_used": 20258816 2024-04-07T17:51:23.376 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.376 INFO:tasks.workunit.client.0.smithi100.stdout: "mds.f47a4ff15a45950a63160265564a123fc5da5e5e": { 2024-04-07T17:51:23.376 INFO:tasks.workunit.client.0.smithi100.stdout: "actual_memory_used": 20398080, 2024-04-07T17:51:23.376 INFO:tasks.workunit.client.0.smithi100.stdout: "central_cache_freelist": 439448, 2024-04-07T17:51:23.376 INFO:tasks.workunit.client.0.smithi100.stdout: "malloc_metadata": 2752512, 2024-04-07T17:51:23.376 INFO:tasks.workunit.client.0.smithi100.stdout: "page_heap_freelist": 0, 2024-04-07T17:51:23.376 INFO:tasks.workunit.client.0.smithi100.stdout: "released_to_os": 909312, 2024-04-07T17:51:23.376 INFO:tasks.workunit.client.0.smithi100.stdout: "spans_in_use": 426, 2024-04-07T17:51:23.376 INFO:tasks.workunit.client.0.smithi100.stdout: "tcmalloc_page_size": 8192, 2024-04-07T17:51:23.376 INFO:tasks.workunit.client.0.smithi100.stdout: "thread_cache_freelists": 1285152, 2024-04-07T17:51:23.376 INFO:tasks.workunit.client.0.smithi100.stdout: "thread_heaps_in_use": 22, 2024-04-07T17:51:23.377 INFO:tasks.workunit.client.0.smithi100.stdout: "transfer_cache_freelist": 1038336, 2024-04-07T17:51:23.377 INFO:tasks.workunit.client.0.smithi100.stdout: "use_by_application": 14882632, 2024-04-07T17:51:23.377 INFO:tasks.workunit.client.0.smithi100.stdout: "virtual_address_space_used": 21307392 2024-04-07T17:51:23.377 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:23.377 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.377 INFO:tasks.workunit.client.0.smithi100.stdout: "mon": { 2024-04-07T17:51:23.377 INFO:tasks.workunit.client.0.smithi100.stdout: "mon.8dd0baa036f0a7e6d1f3bdce86136fe527836c3d": { 2024-04-07T17:51:23.377 INFO:tasks.workunit.client.0.smithi100.stdout: "actual_memory_used": 170909696, 2024-04-07T17:51:23.377 INFO:tasks.workunit.client.0.smithi100.stdout: "central_cache_freelist": 1905680, 2024-04-07T17:51:23.377 INFO:tasks.workunit.client.0.smithi100.stdout: "malloc_metadata": 5111808, 2024-04-07T17:51:23.377 INFO:tasks.workunit.client.0.smithi100.stdout: "page_heap_freelist": 0, 2024-04-07T17:51:23.377 INFO:tasks.workunit.client.0.smithi100.stdout: "released_to_os": 884736, 2024-04-07T17:51:23.377 INFO:tasks.workunit.client.0.smithi100.stdout: "spans_in_use": 6323, 2024-04-07T17:51:23.377 INFO:tasks.workunit.client.0.smithi100.stdout: "tcmalloc_page_size": 8192, 2024-04-07T17:51:23.377 INFO:tasks.workunit.client.0.smithi100.stdout: "thread_cache_freelists": 11300328, 2024-04-07T17:51:23.377 INFO:tasks.workunit.client.0.smithi100.stdout: "thread_heaps_in_use": 21, 2024-04-07T17:51:23.378 INFO:tasks.workunit.client.0.smithi100.stdout: "transfer_cache_freelist": 3245312, 2024-04-07T17:51:23.378 INFO:tasks.workunit.client.0.smithi100.stdout: "use_by_application": 149346568, 2024-04-07T17:51:23.378 INFO:tasks.workunit.client.0.smithi100.stdout: "virtual_address_space_used": 171794432 2024-04-07T17:51:23.378 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.378 INFO:tasks.workunit.client.0.smithi100.stdout: "mon.a52fc0d8acb515431879057649ff294cfe14c620": { 2024-04-07T17:51:23.378 INFO:tasks.workunit.client.0.smithi100.stdout: "actual_memory_used": 165142528, 2024-04-07T17:51:23.378 INFO:tasks.workunit.client.0.smithi100.stdout: "central_cache_freelist": 2430200, 2024-04-07T17:51:23.378 INFO:tasks.workunit.client.0.smithi100.stdout: "malloc_metadata": 3014656, 2024-04-07T17:51:23.378 INFO:tasks.workunit.client.0.smithi100.stdout: "page_heap_freelist": 0, 2024-04-07T17:51:23.378 INFO:tasks.workunit.client.0.smithi100.stdout: "released_to_os": 2457600, 2024-04-07T17:51:23.378 INFO:tasks.workunit.client.0.smithi100.stdout: "spans_in_use": 6222, 2024-04-07T17:51:23.378 INFO:tasks.workunit.client.0.smithi100.stdout: "tcmalloc_page_size": 8192, 2024-04-07T17:51:23.378 INFO:tasks.workunit.client.0.smithi100.stdout: "thread_cache_freelists": 9224784, 2024-04-07T17:51:23.378 INFO:tasks.workunit.client.0.smithi100.stdout: "thread_heaps_in_use": 21, 2024-04-07T17:51:23.379 INFO:tasks.workunit.client.0.smithi100.stdout: "transfer_cache_freelist": 2607360, 2024-04-07T17:51:23.379 INFO:tasks.workunit.client.0.smithi100.stdout: "use_by_application": 147865528, 2024-04-07T17:51:23.379 INFO:tasks.workunit.client.0.smithi100.stdout: "virtual_address_space_used": 167600128 2024-04-07T17:51:23.379 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.379 INFO:tasks.workunit.client.0.smithi100.stdout: "mon.f47a4ff15a45950a63160265564a123fc5da5e5e": { 2024-04-07T17:51:23.379 INFO:tasks.workunit.client.0.smithi100.stdout: "actual_memory_used": 262201344, 2024-04-07T17:51:23.379 INFO:tasks.workunit.client.0.smithi100.stdout: "central_cache_freelist": 6494440, 2024-04-07T17:51:23.379 INFO:tasks.workunit.client.0.smithi100.stdout: "malloc_metadata": 3145728, 2024-04-07T17:51:23.379 INFO:tasks.workunit.client.0.smithi100.stdout: "page_heap_freelist": 868352, 2024-04-07T17:51:23.379 INFO:tasks.workunit.client.0.smithi100.stdout: "released_to_os": 106496, 2024-04-07T17:51:23.379 INFO:tasks.workunit.client.0.smithi100.stdout: "spans_in_use": 9404, 2024-04-07T17:51:23.379 INFO:tasks.workunit.client.0.smithi100.stdout: "tcmalloc_page_size": 8192, 2024-04-07T17:51:23.379 INFO:tasks.workunit.client.0.smithi100.stdout: "thread_cache_freelists": 15151432, 2024-04-07T17:51:23.379 INFO:tasks.workunit.client.0.smithi100.stdout: "thread_heaps_in_use": 23, 2024-04-07T17:51:23.379 INFO:tasks.workunit.client.0.smithi100.stdout: "transfer_cache_freelist": 4899584, 2024-04-07T17:51:23.379 INFO:tasks.workunit.client.0.smithi100.stdout: "use_by_application": 231641808, 2024-04-07T17:51:23.380 INFO:tasks.workunit.client.0.smithi100.stdout: "virtual_address_space_used": 262307840 2024-04-07T17:51:23.380 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:23.380 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.380 INFO:tasks.workunit.client.0.smithi100.stdout: "osd": { 2024-04-07T17:51:23.380 INFO:tasks.workunit.client.0.smithi100.stdout: "osd.0": { 2024-04-07T17:51:23.380 INFO:tasks.workunit.client.0.smithi100.stdout: "actual_memory_used": 473178112, 2024-04-07T17:51:23.380 INFO:tasks.workunit.client.0.smithi100.stdout: "central_cache_freelist": 27184880, 2024-04-07T17:51:23.380 INFO:tasks.workunit.client.0.smithi100.stdout: "malloc_metadata": 3407872, 2024-04-07T17:51:23.380 INFO:tasks.workunit.client.0.smithi100.stdout: "page_heap_freelist": 0, 2024-04-07T17:51:23.380 INFO:tasks.workunit.client.0.smithi100.stdout: "released_to_os": 38920192, 2024-04-07T17:51:23.380 INFO:tasks.workunit.client.0.smithi100.stdout: "spans_in_use": 12519, 2024-04-07T17:51:23.380 INFO:tasks.workunit.client.0.smithi100.stdout: "tcmalloc_page_size": 8192, 2024-04-07T17:51:23.380 INFO:tasks.workunit.client.0.smithi100.stdout: "thread_cache_freelists": 94717872, 2024-04-07T17:51:23.380 INFO:tasks.workunit.client.0.smithi100.stdout: "thread_heaps_in_use": 60, 2024-04-07T17:51:23.380 INFO:tasks.workunit.client.0.smithi100.stdout: "transfer_cache_freelist": 8591872, 2024-04-07T17:51:23.381 INFO:tasks.workunit.client.0.smithi100.stdout: "use_by_application": 339275616, 2024-04-07T17:51:23.381 INFO:tasks.workunit.client.0.smithi100.stdout: "virtual_address_space_used": 512098304 2024-04-07T17:51:23.381 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.381 INFO:tasks.workunit.client.0.smithi100.stdout: "osd.1": { 2024-04-07T17:51:23.381 INFO:tasks.workunit.client.0.smithi100.stdout: "actual_memory_used": 664215552, 2024-04-07T17:51:23.381 INFO:tasks.workunit.client.0.smithi100.stdout: "central_cache_freelist": 27350496, 2024-04-07T17:51:23.381 INFO:tasks.workunit.client.0.smithi100.stdout: "malloc_metadata": 3407872, 2024-04-07T17:51:23.381 INFO:tasks.workunit.client.0.smithi100.stdout: "page_heap_freelist": 0, 2024-04-07T17:51:23.381 INFO:tasks.workunit.client.0.smithi100.stdout: "released_to_os": 35110912, 2024-04-07T17:51:23.381 INFO:tasks.workunit.client.0.smithi100.stdout: "spans_in_use": 13060, 2024-04-07T17:51:23.381 INFO:tasks.workunit.client.0.smithi100.stdout: "tcmalloc_page_size": 8192, 2024-04-07T17:51:23.381 INFO:tasks.workunit.client.0.smithi100.stdout: "thread_cache_freelists": 85554896, 2024-04-07T17:51:23.381 INFO:tasks.workunit.client.0.smithi100.stdout: "thread_heaps_in_use": 53, 2024-04-07T17:51:23.381 INFO:tasks.workunit.client.0.smithi100.stdout: "transfer_cache_freelist": 8306432, 2024-04-07T17:51:23.381 INFO:tasks.workunit.client.0.smithi100.stdout: "use_by_application": 539595856, 2024-04-07T17:51:23.381 INFO:tasks.workunit.client.0.smithi100.stdout: "virtual_address_space_used": 699326464 2024-04-07T17:51:23.382 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.382 INFO:tasks.workunit.client.0.smithi100.stdout: "osd.2": { 2024-04-07T17:51:23.382 INFO:tasks.workunit.client.0.smithi100.stdout: "actual_memory_used": 517251072, 2024-04-07T17:51:23.382 INFO:tasks.workunit.client.0.smithi100.stdout: "central_cache_freelist": 26325688, 2024-04-07T17:51:23.382 INFO:tasks.workunit.client.0.smithi100.stdout: "malloc_metadata": 3276800, 2024-04-07T17:51:23.382 INFO:tasks.workunit.client.0.smithi100.stdout: "page_heap_freelist": 0, 2024-04-07T17:51:23.382 INFO:tasks.workunit.client.0.smithi100.stdout: "released_to_os": 49307648, 2024-04-07T17:51:23.382 INFO:tasks.workunit.client.0.smithi100.stdout: "spans_in_use": 10732, 2024-04-07T17:51:23.382 INFO:tasks.workunit.client.0.smithi100.stdout: "tcmalloc_page_size": 8192, 2024-04-07T17:51:23.382 INFO:tasks.workunit.client.0.smithi100.stdout: "thread_cache_freelists": 85437008, 2024-04-07T17:51:23.382 INFO:tasks.workunit.client.0.smithi100.stdout: "thread_heaps_in_use": 60, 2024-04-07T17:51:23.382 INFO:tasks.workunit.client.0.smithi100.stdout: "transfer_cache_freelist": 9232384, 2024-04-07T17:51:23.382 INFO:tasks.workunit.client.0.smithi100.stdout: "use_by_application": 392979192, 2024-04-07T17:51:23.382 INFO:tasks.workunit.client.0.smithi100.stdout: "virtual_address_space_used": 566558720 2024-04-07T17:51:23.382 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.383 INFO:tasks.workunit.client.0.smithi100.stdout: "osd.3": { 2024-04-07T17:51:23.383 INFO:tasks.workunit.client.0.smithi100.stdout: "actual_memory_used": 473825280, 2024-04-07T17:51:23.383 INFO:tasks.workunit.client.0.smithi100.stdout: "central_cache_freelist": 20092392, 2024-04-07T17:51:23.383 INFO:tasks.workunit.client.0.smithi100.stdout: "malloc_metadata": 3276800, 2024-04-07T17:51:23.383 INFO:tasks.workunit.client.0.smithi100.stdout: "page_heap_freelist": 0, 2024-04-07T17:51:23.383 INFO:tasks.workunit.client.0.smithi100.stdout: "released_to_os": 33226752, 2024-04-07T17:51:23.383 INFO:tasks.workunit.client.0.smithi100.stdout: "spans_in_use": 10246, 2024-04-07T17:51:23.383 INFO:tasks.workunit.client.0.smithi100.stdout: "tcmalloc_page_size": 8192, 2024-04-07T17:51:23.383 INFO:tasks.workunit.client.0.smithi100.stdout: "thread_cache_freelists": 80027944, 2024-04-07T17:51:23.383 INFO:tasks.workunit.client.0.smithi100.stdout: "thread_heaps_in_use": 60, 2024-04-07T17:51:23.383 INFO:tasks.workunit.client.0.smithi100.stdout: "transfer_cache_freelist": 10035200, 2024-04-07T17:51:23.383 INFO:tasks.workunit.client.0.smithi100.stdout: "use_by_application": 360392944, 2024-04-07T17:51:23.383 INFO:tasks.workunit.client.0.smithi100.stdout: "virtual_address_space_used": 507052032 2024-04-07T17:51:23.384 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.384 INFO:tasks.workunit.client.0.smithi100.stdout: "osd.4": { 2024-04-07T17:51:23.384 INFO:tasks.workunit.client.0.smithi100.stdout: "actual_memory_used": 596606976, 2024-04-07T17:51:23.384 INFO:tasks.workunit.client.0.smithi100.stdout: "central_cache_freelist": 24773520, 2024-04-07T17:51:23.384 INFO:tasks.workunit.client.0.smithi100.stdout: "malloc_metadata": 3407872, 2024-04-07T17:51:23.384 INFO:tasks.workunit.client.0.smithi100.stdout: "page_heap_freelist": 0, 2024-04-07T17:51:23.384 INFO:tasks.workunit.client.0.smithi100.stdout: "released_to_os": 30064640, 2024-04-07T17:51:23.384 INFO:tasks.workunit.client.0.smithi100.stdout: "spans_in_use": 11402, 2024-04-07T17:51:23.384 INFO:tasks.workunit.client.0.smithi100.stdout: "tcmalloc_page_size": 8192, 2024-04-07T17:51:23.384 INFO:tasks.workunit.client.0.smithi100.stdout: "thread_cache_freelists": 98951696, 2024-04-07T17:51:23.384 INFO:tasks.workunit.client.0.smithi100.stdout: "thread_heaps_in_use": 53, 2024-04-07T17:51:23.384 INFO:tasks.workunit.client.0.smithi100.stdout: "transfer_cache_freelist": 12066816, 2024-04-07T17:51:23.384 INFO:tasks.workunit.client.0.smithi100.stdout: "use_by_application": 457407072, 2024-04-07T17:51:23.384 INFO:tasks.workunit.client.0.smithi100.stdout: "virtual_address_space_used": 626671616 2024-04-07T17:51:23.384 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.384 INFO:tasks.workunit.client.0.smithi100.stdout: "osd.5": { 2024-04-07T17:51:23.385 INFO:tasks.workunit.client.0.smithi100.stdout: "actual_memory_used": 562798592, 2024-04-07T17:51:23.385 INFO:tasks.workunit.client.0.smithi100.stdout: "central_cache_freelist": 27189648, 2024-04-07T17:51:23.385 INFO:tasks.workunit.client.0.smithi100.stdout: "malloc_metadata": 5636096, 2024-04-07T17:51:23.385 INFO:tasks.workunit.client.0.smithi100.stdout: "page_heap_freelist": 0, 2024-04-07T17:51:23.385 INFO:tasks.workunit.client.0.smithi100.stdout: "released_to_os": 13197312, 2024-04-07T17:51:23.385 INFO:tasks.workunit.client.0.smithi100.stdout: "spans_in_use": 13822, 2024-04-07T17:51:23.385 INFO:tasks.workunit.client.0.smithi100.stdout: "tcmalloc_page_size": 8192, 2024-04-07T17:51:23.385 INFO:tasks.workunit.client.0.smithi100.stdout: "thread_cache_freelists": 86718672, 2024-04-07T17:51:23.385 INFO:tasks.workunit.client.0.smithi100.stdout: "thread_heaps_in_use": 60, 2024-04-07T17:51:23.385 INFO:tasks.workunit.client.0.smithi100.stdout: "transfer_cache_freelist": 10719232, 2024-04-07T17:51:23.385 INFO:tasks.workunit.client.0.smithi100.stdout: "use_by_application": 432534944, 2024-04-07T17:51:23.385 INFO:tasks.workunit.client.0.smithi100.stdout: "virtual_address_space_used": 575995904 2024-04-07T17:51:23.385 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.385 INFO:tasks.workunit.client.0.smithi100.stdout: "osd.6": { 2024-04-07T17:51:23.385 INFO:tasks.workunit.client.0.smithi100.stdout: "actual_memory_used": 362029056, 2024-04-07T17:51:23.386 INFO:tasks.workunit.client.0.smithi100.stdout: "central_cache_freelist": 16280240, 2024-04-07T17:51:23.386 INFO:tasks.workunit.client.0.smithi100.stdout: "malloc_metadata": 3276800, 2024-04-07T17:51:23.386 INFO:tasks.workunit.client.0.smithi100.stdout: "page_heap_freelist": 0, 2024-04-07T17:51:23.386 INFO:tasks.workunit.client.0.smithi100.stdout: "released_to_os": 64888832, 2024-04-07T17:51:23.386 INFO:tasks.workunit.client.0.smithi100.stdout: "spans_in_use": 9372, 2024-04-07T17:51:23.386 INFO:tasks.workunit.client.0.smithi100.stdout: "tcmalloc_page_size": 8192, 2024-04-07T17:51:23.386 INFO:tasks.workunit.client.0.smithi100.stdout: "thread_cache_freelists": 49844352, 2024-04-07T17:51:23.386 INFO:tasks.workunit.client.0.smithi100.stdout: "thread_heaps_in_use": 60, 2024-04-07T17:51:23.386 INFO:tasks.workunit.client.0.smithi100.stdout: "transfer_cache_freelist": 10153472, 2024-04-07T17:51:23.386 INFO:tasks.workunit.client.0.smithi100.stdout: "use_by_application": 282474192, 2024-04-07T17:51:23.386 INFO:tasks.workunit.client.0.smithi100.stdout: "virtual_address_space_used": 426917888 2024-04-07T17:51:23.386 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.386 INFO:tasks.workunit.client.0.smithi100.stdout: "osd.7": { 2024-04-07T17:51:23.386 INFO:tasks.workunit.client.0.smithi100.stdout: "actual_memory_used": 434135040, 2024-04-07T17:51:23.387 INFO:tasks.workunit.client.0.smithi100.stdout: "central_cache_freelist": 20838416, 2024-04-07T17:51:23.387 INFO:tasks.workunit.client.0.smithi100.stdout: "malloc_metadata": 3276800, 2024-04-07T17:51:23.387 INFO:tasks.workunit.client.0.smithi100.stdout: "page_heap_freelist": 0, 2024-04-07T17:51:23.387 INFO:tasks.workunit.client.0.smithi100.stdout: "released_to_os": 32473088, 2024-04-07T17:51:23.387 INFO:tasks.workunit.client.0.smithi100.stdout: "spans_in_use": 10455, 2024-04-07T17:51:23.387 INFO:tasks.workunit.client.0.smithi100.stdout: "tcmalloc_page_size": 8192, 2024-04-07T17:51:23.387 INFO:tasks.workunit.client.0.smithi100.stdout: "thread_cache_freelists": 72478304, 2024-04-07T17:51:23.387 INFO:tasks.workunit.client.0.smithi100.stdout: "thread_heaps_in_use": 59, 2024-04-07T17:51:23.387 INFO:tasks.workunit.client.0.smithi100.stdout: "transfer_cache_freelist": 10252800, 2024-04-07T17:51:23.387 INFO:tasks.workunit.client.0.smithi100.stdout: "use_by_application": 327288720, 2024-04-07T17:51:23.387 INFO:tasks.workunit.client.0.smithi100.stdout: "virtual_address_space_used": 466608128 2024-04-07T17:51:23.387 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:23.387 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:23.387 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.388 INFO:tasks.workunit.client.0.smithi100.stdout: "hosts": { 2024-04-07T17:51:23.388 INFO:tasks.workunit.client.0.smithi100.stdout: "num": 2, 2024-04-07T17:51:23.388 INFO:tasks.workunit.client.0.smithi100.stdout: "num_with_mds": 2, 2024-04-07T17:51:23.388 INFO:tasks.workunit.client.0.smithi100.stdout: "num_with_mgr": 2, 2024-04-07T17:51:23.388 INFO:tasks.workunit.client.0.smithi100.stdout: "num_with_mon": 2, 2024-04-07T17:51:23.388 INFO:tasks.workunit.client.0.smithi100.stdout: "num_with_osd": 2 2024-04-07T17:51:23.388 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.388 INFO:tasks.workunit.client.0.smithi100.stdout: "io_rate": { 2024-04-07T17:51:23.388 INFO:tasks.workunit.client.0.smithi100.stdout: "pg_stats_delta": { 2024-04-07T17:51:23.388 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": 0, 2024-04-07T17:51:23.388 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:23.388 INFO:tasks.workunit.client.0.smithi100.stdout: "num_store_stats": 0, 2024-04-07T17:51:23.388 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:23.388 INFO:tasks.workunit.client.0.smithi100.stdout: "stamp_delta": "12.004568", 2024-04-07T17:51:23.388 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:23.389 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:23.389 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:23.389 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:23.389 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:23.389 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:23.389 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:23.389 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:23.389 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:23.389 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:23.389 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:23.389 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:23.389 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:23.389 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:23.389 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:23.389 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:23.389 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:23.390 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:23.390 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:23.390 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:23.390 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:23.390 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:23.390 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:23.390 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:23.390 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:23.390 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:23.390 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:23.390 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:23.390 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:23.390 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:23.390 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:23.390 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:23.391 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:23.391 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:23.391 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:23.391 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:23.391 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:23.391 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:23.391 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:23.391 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:23.391 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:23.391 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.391 INFO:tasks.workunit.client.0.smithi100.stdout: "store_stats": { 2024-04-07T17:51:23.391 INFO:tasks.workunit.client.0.smithi100.stdout: "allocated": 0, 2024-04-07T17:51:23.391 INFO:tasks.workunit.client.0.smithi100.stdout: "available": 0, 2024-04-07T17:51:23.391 INFO:tasks.workunit.client.0.smithi100.stdout: "data_compressed": 0, 2024-04-07T17:51:23.391 INFO:tasks.workunit.client.0.smithi100.stdout: "data_compressed_allocated": 0, 2024-04-07T17:51:23.391 INFO:tasks.workunit.client.0.smithi100.stdout: "data_compressed_original": 0, 2024-04-07T17:51:23.392 INFO:tasks.workunit.client.0.smithi100.stdout: "data_stored": 0, 2024-04-07T17:51:23.392 INFO:tasks.workunit.client.0.smithi100.stdout: "internal_metadata": 0, 2024-04-07T17:51:23.392 INFO:tasks.workunit.client.0.smithi100.stdout: "internally_reserved": 0, 2024-04-07T17:51:23.392 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_allocated": 0, 2024-04-07T17:51:23.392 INFO:tasks.workunit.client.0.smithi100.stdout: "total": 0 2024-04-07T17:51:23.392 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.392 INFO:tasks.workunit.client.0.smithi100.stdout: "up": 0 2024-04-07T17:51:23.392 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:23.392 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.392 INFO:tasks.workunit.client.0.smithi100.stdout: "leaderboard": false, 2024-04-07T17:51:23.392 INFO:tasks.workunit.client.0.smithi100.stdout: "leaderboard_description": null, 2024-04-07T17:51:23.392 INFO:tasks.workunit.client.0.smithi100.stdout: "license": "sharing-1-0", 2024-04-07T17:51:23.392 INFO:tasks.workunit.client.0.smithi100.stdout: "mempool": { 2024-04-07T17:51:23.392 INFO:tasks.workunit.client.0.smithi100.stdout: "mds": { 2024-04-07T17:51:23.392 INFO:tasks.workunit.client.0.smithi100.stdout: "mds.a52fc0d8acb515431879057649ff294cfe14c620": { 2024-04-07T17:51:23.392 INFO:tasks.workunit.client.0.smithi100.stdout: "bloom_filter": { 2024-04-07T17:51:23.393 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.393 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.393 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.393 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs": { 2024-04-07T17:51:23.393 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.393 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.393 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.393 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs_file_reader": { 2024-04-07T17:51:23.393 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.393 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.393 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.393 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs_file_writer": { 2024-04-07T17:51:23.393 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.393 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.393 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.393 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_alloc": { 2024-04-07T17:51:23.394 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.394 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.394 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.394 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_blob": { 2024-04-07T17:51:23.394 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.394 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.394 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.394 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_buffer": { 2024-04-07T17:51:23.394 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.394 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.394 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.394 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_data": { 2024-04-07T17:51:23.394 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.394 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.395 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.395 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_meta": { 2024-04-07T17:51:23.395 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.395 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.395 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.395 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_onode": { 2024-04-07T17:51:23.395 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.395 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.395 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.395 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_other": { 2024-04-07T17:51:23.395 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.395 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.395 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.395 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_extent": { 2024-04-07T17:51:23.395 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.396 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.396 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.396 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_fsck": { 2024-04-07T17:51:23.396 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.396 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.396 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.396 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_inline_bl": { 2024-04-07T17:51:23.396 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.396 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.396 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.396 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_shared_blob": { 2024-04-07T17:51:23.396 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.396 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.396 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.396 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_txc": { 2024-04-07T17:51:23.396 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.397 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.397 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.397 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_writing": { 2024-04-07T17:51:23.397 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.397 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.397 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.397 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_writing_deferred": { 2024-04-07T17:51:23.397 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.397 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.397 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.397 INFO:tasks.workunit.client.0.smithi100.stdout: "buffer_anon": { 2024-04-07T17:51:23.397 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 149048, 2024-04-07T17:51:23.397 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 34 2024-04-07T17:51:23.397 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.397 INFO:tasks.workunit.client.0.smithi100.stdout: "buffer_meta": { 2024-04-07T17:51:23.397 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.398 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.398 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.398 INFO:tasks.workunit.client.0.smithi100.stdout: "mds_co": { 2024-04-07T17:51:23.398 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 928, 2024-04-07T17:51:23.398 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 2 2024-04-07T17:51:23.398 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.398 INFO:tasks.workunit.client.0.smithi100.stdout: "osd": { 2024-04-07T17:51:23.398 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.398 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.398 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.398 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mapbl": { 2024-04-07T17:51:23.398 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.398 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.398 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.399 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_pglog": { 2024-04-07T17:51:23.399 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.399 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.399 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.399 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap": { 2024-04-07T17:51:23.399 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.399 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.399 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.399 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap_mapping": { 2024-04-07T17:51:23.399 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.399 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.399 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.399 INFO:tasks.workunit.client.0.smithi100.stdout: "pgmap": { 2024-04-07T17:51:23.399 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.400 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.400 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.400 INFO:tasks.workunit.client.0.smithi100.stdout: "unittest_1": { 2024-04-07T17:51:23.400 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.400 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.400 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.400 INFO:tasks.workunit.client.0.smithi100.stdout: "unittest_2": { 2024-04-07T17:51:23.400 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.400 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.400 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:23.400 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.400 INFO:tasks.workunit.client.0.smithi100.stdout: "mds.f47a4ff15a45950a63160265564a123fc5da5e5e": { 2024-04-07T17:51:23.400 INFO:tasks.workunit.client.0.smithi100.stdout: "bloom_filter": { 2024-04-07T17:51:23.400 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.401 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.401 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.401 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs": { 2024-04-07T17:51:23.401 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.401 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.401 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.401 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs_file_reader": { 2024-04-07T17:51:23.401 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.401 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.401 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.401 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs_file_writer": { 2024-04-07T17:51:23.401 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.401 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.401 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.401 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_alloc": { 2024-04-07T17:51:23.401 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.402 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.402 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.402 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_blob": { 2024-04-07T17:51:23.402 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.402 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.402 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.402 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_buffer": { 2024-04-07T17:51:23.402 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.402 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.402 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.402 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_data": { 2024-04-07T17:51:23.402 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.402 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.403 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.403 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_meta": { 2024-04-07T17:51:23.403 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.403 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.403 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.403 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_onode": { 2024-04-07T17:51:23.403 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.403 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.403 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.403 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_other": { 2024-04-07T17:51:23.403 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.403 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.403 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.403 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_extent": { 2024-04-07T17:51:23.403 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.403 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.404 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.404 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_fsck": { 2024-04-07T17:51:23.404 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.404 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.404 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.404 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_inline_bl": { 2024-04-07T17:51:23.404 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.404 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.404 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.404 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_shared_blob": { 2024-04-07T17:51:23.404 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.404 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.404 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.404 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_txc": { 2024-04-07T17:51:23.404 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.405 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.405 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.405 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_writing": { 2024-04-07T17:51:23.405 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.405 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.405 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.405 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_writing_deferred": { 2024-04-07T17:51:23.405 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.405 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.405 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.405 INFO:tasks.workunit.client.0.smithi100.stdout: "buffer_anon": { 2024-04-07T17:51:23.405 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 221552, 2024-04-07T17:51:23.405 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 64 2024-04-07T17:51:23.405 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.405 INFO:tasks.workunit.client.0.smithi100.stdout: "buffer_meta": { 2024-04-07T17:51:23.406 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.406 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.406 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.406 INFO:tasks.workunit.client.0.smithi100.stdout: "mds_co": { 2024-04-07T17:51:23.406 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 52112, 2024-04-07T17:51:23.406 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 181 2024-04-07T17:51:23.406 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.406 INFO:tasks.workunit.client.0.smithi100.stdout: "osd": { 2024-04-07T17:51:23.406 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.406 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.406 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.406 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mapbl": { 2024-04-07T17:51:23.406 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.406 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.407 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.407 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_pglog": { 2024-04-07T17:51:23.407 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.407 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.407 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.407 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap": { 2024-04-07T17:51:23.407 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 12216, 2024-04-07T17:51:23.407 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 252 2024-04-07T17:51:23.407 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.407 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap_mapping": { 2024-04-07T17:51:23.407 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.407 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.407 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.407 INFO:tasks.workunit.client.0.smithi100.stdout: "pgmap": { 2024-04-07T17:51:23.407 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.408 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.408 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.408 INFO:tasks.workunit.client.0.smithi100.stdout: "unittest_1": { 2024-04-07T17:51:23.408 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.408 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.408 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.408 INFO:tasks.workunit.client.0.smithi100.stdout: "unittest_2": { 2024-04-07T17:51:23.408 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.409 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.409 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:23.409 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:23.409 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.409 INFO:tasks.workunit.client.0.smithi100.stdout: "mon": { 2024-04-07T17:51:23.409 INFO:tasks.workunit.client.0.smithi100.stdout: "mon.8dd0baa036f0a7e6d1f3bdce86136fe527836c3d": { 2024-04-07T17:51:23.409 INFO:tasks.workunit.client.0.smithi100.stdout: "bloom_filter": { 2024-04-07T17:51:23.409 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.409 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.409 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.409 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs": { 2024-04-07T17:51:23.409 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.409 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.409 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.409 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs_file_reader": { 2024-04-07T17:51:23.409 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.410 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.410 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.410 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs_file_writer": { 2024-04-07T17:51:23.410 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.410 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.410 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.410 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_alloc": { 2024-04-07T17:51:23.410 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.410 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.410 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.410 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_blob": { 2024-04-07T17:51:23.410 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.410 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.410 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.410 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_buffer": { 2024-04-07T17:51:23.411 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.411 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.411 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.411 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_data": { 2024-04-07T17:51:23.411 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.411 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.411 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.411 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_meta": { 2024-04-07T17:51:23.411 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.411 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.411 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.411 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_onode": { 2024-04-07T17:51:23.411 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.411 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.412 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.412 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_other": { 2024-04-07T17:51:23.412 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.412 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.412 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.412 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_extent": { 2024-04-07T17:51:23.412 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.412 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.412 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.412 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_fsck": { 2024-04-07T17:51:23.412 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.412 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.412 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.412 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_inline_bl": { 2024-04-07T17:51:23.412 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.412 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.413 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.413 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_shared_blob": { 2024-04-07T17:51:23.413 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.413 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.413 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.413 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_txc": { 2024-04-07T17:51:23.413 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.413 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.413 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.413 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_writing": { 2024-04-07T17:51:23.413 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.413 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.413 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.413 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_writing_deferred": { 2024-04-07T17:51:23.413 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.414 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.414 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.414 INFO:tasks.workunit.client.0.smithi100.stdout: "buffer_anon": { 2024-04-07T17:51:23.414 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 40701650, 2024-04-07T17:51:23.414 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 5027 2024-04-07T17:51:23.414 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.414 INFO:tasks.workunit.client.0.smithi100.stdout: "buffer_meta": { 2024-04-07T17:51:23.414 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.414 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.414 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.414 INFO:tasks.workunit.client.0.smithi100.stdout: "mds_co": { 2024-04-07T17:51:23.414 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.414 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.414 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.414 INFO:tasks.workunit.client.0.smithi100.stdout: "osd": { 2024-04-07T17:51:23.415 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.415 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.415 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.415 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mapbl": { 2024-04-07T17:51:23.415 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.415 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.415 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.415 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_pglog": { 2024-04-07T17:51:23.415 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.415 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.415 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.415 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap": { 2024-04-07T17:51:23.415 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 10792, 2024-04-07T17:51:23.415 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 251 2024-04-07T17:51:23.415 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.416 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap_mapping": { 2024-04-07T17:51:23.416 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 70440, 2024-04-07T17:51:23.416 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 5158 2024-04-07T17:51:23.416 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.416 INFO:tasks.workunit.client.0.smithi100.stdout: "pgmap": { 2024-04-07T17:51:23.416 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 5896, 2024-04-07T17:51:23.416 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 174 2024-04-07T17:51:23.416 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.416 INFO:tasks.workunit.client.0.smithi100.stdout: "unittest_1": { 2024-04-07T17:51:23.416 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.416 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.416 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.416 INFO:tasks.workunit.client.0.smithi100.stdout: "unittest_2": { 2024-04-07T17:51:23.416 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.416 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.417 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:23.417 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.417 INFO:tasks.workunit.client.0.smithi100.stdout: "mon.a52fc0d8acb515431879057649ff294cfe14c620": { 2024-04-07T17:51:23.417 INFO:tasks.workunit.client.0.smithi100.stdout: "bloom_filter": { 2024-04-07T17:51:23.417 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.417 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.417 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.417 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs": { 2024-04-07T17:51:23.417 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.417 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.417 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.417 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs_file_reader": { 2024-04-07T17:51:23.417 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.417 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.417 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.417 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs_file_writer": { 2024-04-07T17:51:23.418 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.418 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.418 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.418 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_alloc": { 2024-04-07T17:51:23.418 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.418 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.418 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.418 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_blob": { 2024-04-07T17:51:23.418 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.418 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.418 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.418 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_buffer": { 2024-04-07T17:51:23.418 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.419 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.419 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.419 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_data": { 2024-04-07T17:51:23.419 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.419 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.419 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.419 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_meta": { 2024-04-07T17:51:23.419 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.419 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.419 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.419 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_onode": { 2024-04-07T17:51:23.419 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.419 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.419 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.419 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_other": { 2024-04-07T17:51:23.419 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.420 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.420 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.420 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_extent": { 2024-04-07T17:51:23.420 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.420 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.420 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.420 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_fsck": { 2024-04-07T17:51:23.420 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.420 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.420 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.420 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_inline_bl": { 2024-04-07T17:51:23.420 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.420 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.420 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.420 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_shared_blob": { 2024-04-07T17:51:23.421 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.421 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.421 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.421 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_txc": { 2024-04-07T17:51:23.421 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.421 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.421 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.421 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_writing": { 2024-04-07T17:51:23.421 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.421 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.421 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.421 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_writing_deferred": { 2024-04-07T17:51:23.421 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.421 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.421 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.422 INFO:tasks.workunit.client.0.smithi100.stdout: "buffer_anon": { 2024-04-07T17:51:23.422 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 40463250, 2024-04-07T17:51:23.422 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 4988 2024-04-07T17:51:23.422 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.422 INFO:tasks.workunit.client.0.smithi100.stdout: "buffer_meta": { 2024-04-07T17:51:23.422 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 1408, 2024-04-07T17:51:23.422 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 16 2024-04-07T17:51:23.422 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.422 INFO:tasks.workunit.client.0.smithi100.stdout: "mds_co": { 2024-04-07T17:51:23.422 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.422 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.422 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.422 INFO:tasks.workunit.client.0.smithi100.stdout: "osd": { 2024-04-07T17:51:23.422 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.423 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.423 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.423 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mapbl": { 2024-04-07T17:51:23.423 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.423 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.423 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.423 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_pglog": { 2024-04-07T17:51:23.423 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.423 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.423 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.423 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap": { 2024-04-07T17:51:23.423 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 10792, 2024-04-07T17:51:23.423 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 251 2024-04-07T17:51:23.423 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.423 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap_mapping": { 2024-04-07T17:51:23.423 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 70440, 2024-04-07T17:51:23.424 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 5158 2024-04-07T17:51:23.424 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.424 INFO:tasks.workunit.client.0.smithi100.stdout: "pgmap": { 2024-04-07T17:51:23.424 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 5896, 2024-04-07T17:51:23.424 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 174 2024-04-07T17:51:23.424 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.424 INFO:tasks.workunit.client.0.smithi100.stdout: "unittest_1": { 2024-04-07T17:51:23.424 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.424 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.424 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.424 INFO:tasks.workunit.client.0.smithi100.stdout: "unittest_2": { 2024-04-07T17:51:23.424 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.424 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.424 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:23.424 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.425 INFO:tasks.workunit.client.0.smithi100.stdout: "mon.f47a4ff15a45950a63160265564a123fc5da5e5e": { 2024-04-07T17:51:23.425 INFO:tasks.workunit.client.0.smithi100.stdout: "bloom_filter": { 2024-04-07T17:51:23.425 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.425 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.425 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.425 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs": { 2024-04-07T17:51:23.425 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.425 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.425 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.425 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs_file_reader": { 2024-04-07T17:51:23.425 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.425 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.425 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.425 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs_file_writer": { 2024-04-07T17:51:23.425 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.426 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.426 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.426 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_alloc": { 2024-04-07T17:51:23.426 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.426 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.426 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.426 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_blob": { 2024-04-07T17:51:23.426 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.426 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.426 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.426 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_buffer": { 2024-04-07T17:51:23.426 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.426 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.426 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.426 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_data": { 2024-04-07T17:51:23.427 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.427 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.427 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.427 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_meta": { 2024-04-07T17:51:23.427 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.427 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.427 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.427 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_onode": { 2024-04-07T17:51:23.427 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.427 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.427 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.427 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_other": { 2024-04-07T17:51:23.427 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.427 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.427 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.428 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_extent": { 2024-04-07T17:51:23.428 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.428 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.428 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.428 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_fsck": { 2024-04-07T17:51:23.428 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.428 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.428 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.428 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_inline_bl": { 2024-04-07T17:51:23.428 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.428 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.428 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.428 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_shared_blob": { 2024-04-07T17:51:23.428 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.428 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.428 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.429 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_txc": { 2024-04-07T17:51:23.429 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.429 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.429 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.429 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_writing": { 2024-04-07T17:51:23.429 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.429 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.429 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.429 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_writing_deferred": { 2024-04-07T17:51:23.429 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.429 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.429 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.429 INFO:tasks.workunit.client.0.smithi100.stdout: "buffer_anon": { 2024-04-07T17:51:23.429 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 41173338, 2024-04-07T17:51:23.429 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 5106 2024-04-07T17:51:23.429 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.430 INFO:tasks.workunit.client.0.smithi100.stdout: "buffer_meta": { 2024-04-07T17:51:23.430 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.430 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.430 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.430 INFO:tasks.workunit.client.0.smithi100.stdout: "mds_co": { 2024-04-07T17:51:23.430 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.430 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.430 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.430 INFO:tasks.workunit.client.0.smithi100.stdout: "osd": { 2024-04-07T17:51:23.430 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.430 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.430 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.430 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mapbl": { 2024-04-07T17:51:23.430 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.430 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.431 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.431 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_pglog": { 2024-04-07T17:51:23.431 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.431 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.431 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.431 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap": { 2024-04-07T17:51:23.431 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 10792, 2024-04-07T17:51:23.431 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 251 2024-04-07T17:51:23.431 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.431 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap_mapping": { 2024-04-07T17:51:23.431 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 70440, 2024-04-07T17:51:23.431 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 5158 2024-04-07T17:51:23.431 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.431 INFO:tasks.workunit.client.0.smithi100.stdout: "pgmap": { 2024-04-07T17:51:23.431 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 11792, 2024-04-07T17:51:23.431 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 348 2024-04-07T17:51:23.432 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.432 INFO:tasks.workunit.client.0.smithi100.stdout: "unittest_1": { 2024-04-07T17:51:23.432 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.432 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.432 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.432 INFO:tasks.workunit.client.0.smithi100.stdout: "unittest_2": { 2024-04-07T17:51:23.432 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.432 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.432 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:23.432 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:23.432 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.432 INFO:tasks.workunit.client.0.smithi100.stdout: "osd": { 2024-04-07T17:51:23.432 INFO:tasks.workunit.client.0.smithi100.stdout: "osd.0": { 2024-04-07T17:51:23.432 INFO:tasks.workunit.client.0.smithi100.stdout: "bloom_filter": { 2024-04-07T17:51:23.432 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.432 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.433 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.433 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs": { 2024-04-07T17:51:23.433 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 22768, 2024-04-07T17:51:23.433 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 424 2024-04-07T17:51:23.433 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.433 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs_file_reader": { 2024-04-07T17:51:23.433 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 31360, 2024-04-07T17:51:23.433 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 28 2024-04-07T17:51:23.433 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.433 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs_file_writer": { 2024-04-07T17:51:23.433 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 2496, 2024-04-07T17:51:23.433 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 13 2024-04-07T17:51:23.433 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.433 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_alloc": { 2024-04-07T17:51:23.433 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 59760, 2024-04-07T17:51:23.434 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 747 2024-04-07T17:51:23.434 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.434 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_blob": { 2024-04-07T17:51:23.434 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 382512, 2024-04-07T17:51:23.434 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 1839 2024-04-07T17:51:23.434 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.434 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_buffer": { 2024-04-07T17:51:23.434 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 256, 2024-04-07T17:51:23.434 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 32 2024-04-07T17:51:23.434 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.434 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_data": { 2024-04-07T17:51:23.434 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 774695, 2024-04-07T17:51:23.434 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 215 2024-04-07T17:51:23.434 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.434 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_meta": { 2024-04-07T17:51:23.435 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 2427203, 2024-04-07T17:51:23.435 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 537865 2024-04-07T17:51:23.435 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.435 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_onode": { 2024-04-07T17:51:23.435 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 6943736, 2024-04-07T17:51:23.435 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 10243 2024-04-07T17:51:23.435 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.435 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_other": { 2024-04-07T17:51:23.435 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 72976, 2024-04-07T17:51:23.435 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 7617 2024-04-07T17:51:23.435 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.435 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_extent": { 2024-04-07T17:51:23.435 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 88272, 2024-04-07T17:51:23.435 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 1839 2024-04-07T17:51:23.435 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.436 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_fsck": { 2024-04-07T17:51:23.436 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.436 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.436 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.436 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_inline_bl": { 2024-04-07T17:51:23.436 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 175624, 2024-04-07T17:51:23.436 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 1258 2024-04-07T17:51:23.436 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.436 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_shared_blob": { 2024-04-07T17:51:23.436 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.436 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.437 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.437 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_txc": { 2024-04-07T17:51:23.437 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 736, 2024-04-07T17:51:23.437 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 1 2024-04-07T17:51:23.437 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.437 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_writing": { 2024-04-07T17:51:23.437 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 53513, 2024-04-07T17:51:23.437 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 40 2024-04-07T17:51:23.437 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.437 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_writing_deferred": { 2024-04-07T17:51:23.437 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 12112, 2024-04-07T17:51:23.437 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 3 2024-04-07T17:51:23.437 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.437 INFO:tasks.workunit.client.0.smithi100.stdout: "buffer_anon": { 2024-04-07T17:51:23.437 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 15040092, 2024-04-07T17:51:23.438 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 287 2024-04-07T17:51:23.438 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.438 INFO:tasks.workunit.client.0.smithi100.stdout: "buffer_meta": { 2024-04-07T17:51:23.438 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 19536, 2024-04-07T17:51:23.438 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 222 2024-04-07T17:51:23.438 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.438 INFO:tasks.workunit.client.0.smithi100.stdout: "mds_co": { 2024-04-07T17:51:23.438 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.438 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.438 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.438 INFO:tasks.workunit.client.0.smithi100.stdout: "osd": { 2024-04-07T17:51:23.438 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 513216, 2024-04-07T17:51:23.438 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 44 2024-04-07T17:51:23.438 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.439 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mapbl": { 2024-04-07T17:51:23.439 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.439 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.439 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.439 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_pglog": { 2024-04-07T17:51:23.439 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 7159360, 2024-04-07T17:51:23.439 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 18568 2024-04-07T17:51:23.439 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.439 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap": { 2024-04-07T17:51:23.439 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 635376, 2024-04-07T17:51:23.439 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 11076 2024-04-07T17:51:23.439 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.439 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap_mapping": { 2024-04-07T17:51:23.439 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.439 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.440 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.440 INFO:tasks.workunit.client.0.smithi100.stdout: "pgmap": { 2024-04-07T17:51:23.440 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.440 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.440 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.440 INFO:tasks.workunit.client.0.smithi100.stdout: "unittest_1": { 2024-04-07T17:51:23.440 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.440 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.440 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.440 INFO:tasks.workunit.client.0.smithi100.stdout: "unittest_2": { 2024-04-07T17:51:23.440 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.440 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.440 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:23.440 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.440 INFO:tasks.workunit.client.0.smithi100.stdout: "osd.1": { 2024-04-07T17:51:23.441 INFO:tasks.workunit.client.0.smithi100.stdout: "bloom_filter": { 2024-04-07T17:51:23.441 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.441 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.441 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.441 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs": { 2024-04-07T17:51:23.441 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 20720, 2024-04-07T17:51:23.441 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 385 2024-04-07T17:51:23.441 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.441 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs_file_reader": { 2024-04-07T17:51:23.441 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 35072, 2024-04-07T17:51:23.441 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 26 2024-04-07T17:51:23.441 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.441 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs_file_writer": { 2024-04-07T17:51:23.441 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 4800, 2024-04-07T17:51:23.441 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 25 2024-04-07T17:51:23.441 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.442 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_alloc": { 2024-04-07T17:51:23.442 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 58160, 2024-04-07T17:51:23.442 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 727 2024-04-07T17:51:23.442 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.442 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_blob": { 2024-04-07T17:51:23.442 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 380224, 2024-04-07T17:51:23.442 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 1828 2024-04-07T17:51:23.442 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.442 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_buffer": { 2024-04-07T17:51:23.442 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 256, 2024-04-07T17:51:23.442 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 32 2024-04-07T17:51:23.442 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.442 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_data": { 2024-04-07T17:51:23.442 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 1045046, 2024-04-07T17:51:23.443 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 285 2024-04-07T17:51:23.443 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.443 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_meta": { 2024-04-07T17:51:23.443 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 2355008, 2024-04-07T17:51:23.443 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 521245 2024-04-07T17:51:23.443 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.443 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_onode": { 2024-04-07T17:51:23.443 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 6739736, 2024-04-07T17:51:23.443 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 9943 2024-04-07T17:51:23.443 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.443 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_other": { 2024-04-07T17:51:23.443 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 70888, 2024-04-07T17:51:23.443 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 7440 2024-04-07T17:51:23.443 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.443 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_extent": { 2024-04-07T17:51:23.444 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 87744, 2024-04-07T17:51:23.444 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 1828 2024-04-07T17:51:23.444 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.444 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_fsck": { 2024-04-07T17:51:23.444 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.444 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.444 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.444 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_inline_bl": { 2024-04-07T17:51:23.444 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 175712, 2024-04-07T17:51:23.444 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 1259 2024-04-07T17:51:23.444 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.444 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_shared_blob": { 2024-04-07T17:51:23.444 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.444 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.444 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.444 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_txc": { 2024-04-07T17:51:23.445 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.445 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.445 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.445 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_writing": { 2024-04-07T17:51:23.445 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 34652, 2024-04-07T17:51:23.445 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 29 2024-04-07T17:51:23.445 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.445 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_writing_deferred": { 2024-04-07T17:51:23.445 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.445 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.445 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.445 INFO:tasks.workunit.client.0.smithi100.stdout: "buffer_anon": { 2024-04-07T17:51:23.445 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 26499475, 2024-04-07T17:51:23.445 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 286 2024-04-07T17:51:23.445 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.445 INFO:tasks.workunit.client.0.smithi100.stdout: "buffer_meta": { 2024-04-07T17:51:23.446 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 26928, 2024-04-07T17:51:23.446 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 306 2024-04-07T17:51:23.446 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.446 INFO:tasks.workunit.client.0.smithi100.stdout: "mds_co": { 2024-04-07T17:51:23.446 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.446 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.446 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.446 INFO:tasks.workunit.client.0.smithi100.stdout: "osd": { 2024-04-07T17:51:23.446 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 443232, 2024-04-07T17:51:23.446 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 38 2024-04-07T17:51:23.446 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.446 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mapbl": { 2024-04-07T17:51:23.446 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.446 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.447 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.447 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_pglog": { 2024-04-07T17:51:23.447 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 7182344, 2024-04-07T17:51:23.447 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 20929 2024-04-07T17:51:23.447 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.447 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap": { 2024-04-07T17:51:23.447 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 635376, 2024-04-07T17:51:23.447 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 11076 2024-04-07T17:51:23.447 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.447 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap_mapping": { 2024-04-07T17:51:23.447 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.447 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.447 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.447 INFO:tasks.workunit.client.0.smithi100.stdout: "pgmap": { 2024-04-07T17:51:23.447 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.447 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.448 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.448 INFO:tasks.workunit.client.0.smithi100.stdout: "unittest_1": { 2024-04-07T17:51:23.448 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.448 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.448 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.448 INFO:tasks.workunit.client.0.smithi100.stdout: "unittest_2": { 2024-04-07T17:51:23.448 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.448 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.448 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:23.448 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.448 INFO:tasks.workunit.client.0.smithi100.stdout: "osd.2": { 2024-04-07T17:51:23.448 INFO:tasks.workunit.client.0.smithi100.stdout: "bloom_filter": { 2024-04-07T17:51:23.448 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.448 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.448 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.448 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs": { 2024-04-07T17:51:23.449 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 19496, 2024-04-07T17:51:23.449 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 362 2024-04-07T17:51:23.449 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.449 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs_file_reader": { 2024-04-07T17:51:23.449 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 31360, 2024-04-07T17:51:23.449 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 28 2024-04-07T17:51:23.449 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.449 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs_file_writer": { 2024-04-07T17:51:23.449 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 3648, 2024-04-07T17:51:23.449 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 19 2024-04-07T17:51:23.449 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.449 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_alloc": { 2024-04-07T17:51:23.449 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 58080, 2024-04-07T17:51:23.449 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 726 2024-04-07T17:51:23.449 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.449 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_blob": { 2024-04-07T17:51:23.450 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 379600, 2024-04-07T17:51:23.450 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 1825 2024-04-07T17:51:23.450 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.450 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_buffer": { 2024-04-07T17:51:23.450 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 256, 2024-04-07T17:51:23.450 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 32 2024-04-07T17:51:23.450 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.450 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_data": { 2024-04-07T17:51:23.450 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 434802, 2024-04-07T17:51:23.450 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 100 2024-04-07T17:51:23.450 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.450 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_meta": { 2024-04-07T17:51:23.451 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 1711873, 2024-04-07T17:51:23.451 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 379494 2024-04-07T17:51:23.451 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.451 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_onode": { 2024-04-07T17:51:23.451 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 5016616, 2024-04-07T17:51:23.451 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 7409 2024-04-07T17:51:23.451 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.451 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_other": { 2024-04-07T17:51:23.451 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 69948, 2024-04-07T17:51:23.451 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 7427 2024-04-07T17:51:23.451 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.451 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_extent": { 2024-04-07T17:51:23.451 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 87600, 2024-04-07T17:51:23.451 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 1825 2024-04-07T17:51:23.451 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.451 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_fsck": { 2024-04-07T17:51:23.452 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.452 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.452 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.452 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_inline_bl": { 2024-04-07T17:51:23.452 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 175624, 2024-04-07T17:51:23.452 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 1258 2024-04-07T17:51:23.452 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.452 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_shared_blob": { 2024-04-07T17:51:23.452 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.452 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.452 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.452 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_txc": { 2024-04-07T17:51:23.452 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 736, 2024-04-07T17:51:23.452 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 1 2024-04-07T17:51:23.452 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.452 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_writing": { 2024-04-07T17:51:23.453 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 69418, 2024-04-07T17:51:23.453 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 53 2024-04-07T17:51:23.453 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.453 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_writing_deferred": { 2024-04-07T17:51:23.453 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 12112, 2024-04-07T17:51:23.453 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 3 2024-04-07T17:51:23.453 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.453 INFO:tasks.workunit.client.0.smithi100.stdout: "buffer_anon": { 2024-04-07T17:51:23.453 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 20645063, 2024-04-07T17:51:23.453 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 260 2024-04-07T17:51:23.453 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.453 INFO:tasks.workunit.client.0.smithi100.stdout: "buffer_meta": { 2024-04-07T17:51:23.453 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 12760, 2024-04-07T17:51:23.453 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 145 2024-04-07T17:51:23.454 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.454 INFO:tasks.workunit.client.0.smithi100.stdout: "mds_co": { 2024-04-07T17:51:23.454 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.454 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.454 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.454 INFO:tasks.workunit.client.0.smithi100.stdout: "osd": { 2024-04-07T17:51:23.454 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 349920, 2024-04-07T17:51:23.454 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 30 2024-04-07T17:51:23.454 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.454 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mapbl": { 2024-04-07T17:51:23.454 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.454 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.455 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.455 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_pglog": { 2024-04-07T17:51:23.455 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 3170440, 2024-04-07T17:51:23.455 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 6097 2024-04-07T17:51:23.455 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.455 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap": { 2024-04-07T17:51:23.455 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 635376, 2024-04-07T17:51:23.455 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 11076 2024-04-07T17:51:23.455 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.455 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap_mapping": { 2024-04-07T17:51:23.455 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.455 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.455 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.455 INFO:tasks.workunit.client.0.smithi100.stdout: "pgmap": { 2024-04-07T17:51:23.455 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.456 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.456 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.456 INFO:tasks.workunit.client.0.smithi100.stdout: "unittest_1": { 2024-04-07T17:51:23.456 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.456 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.456 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.456 INFO:tasks.workunit.client.0.smithi100.stdout: "unittest_2": { 2024-04-07T17:51:23.456 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.456 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.456 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:23.456 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.456 INFO:tasks.workunit.client.0.smithi100.stdout: "osd.3": { 2024-04-07T17:51:23.456 INFO:tasks.workunit.client.0.smithi100.stdout: "bloom_filter": { 2024-04-07T17:51:23.456 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.457 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.457 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.457 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs": { 2024-04-07T17:51:23.457 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 17048, 2024-04-07T17:51:23.457 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 316 2024-04-07T17:51:23.457 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.457 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs_file_reader": { 2024-04-07T17:51:23.457 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 30976, 2024-04-07T17:51:23.457 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 25 2024-04-07T17:51:23.457 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.457 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs_file_writer": { 2024-04-07T17:51:23.457 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 3072, 2024-04-07T17:51:23.457 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 16 2024-04-07T17:51:23.457 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.457 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_alloc": { 2024-04-07T17:51:23.457 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 56960, 2024-04-07T17:51:23.458 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 712 2024-04-07T17:51:23.458 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.458 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_blob": { 2024-04-07T17:51:23.458 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 380640, 2024-04-07T17:51:23.458 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 1830 2024-04-07T17:51:23.458 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.458 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_buffer": { 2024-04-07T17:51:23.458 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 256, 2024-04-07T17:51:23.458 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 32 2024-04-07T17:51:23.458 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.458 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_data": { 2024-04-07T17:51:23.458 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 381771, 2024-04-07T17:51:23.458 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 104 2024-04-07T17:51:23.458 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.459 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_meta": { 2024-04-07T17:51:23.459 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 1733357, 2024-04-07T17:51:23.459 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 381544 2024-04-07T17:51:23.459 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.459 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_onode": { 2024-04-07T17:51:23.459 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 5029536, 2024-04-07T17:51:23.459 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 7428 2024-04-07T17:51:23.459 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.459 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_other": { 2024-04-07T17:51:23.459 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 70748, 2024-04-07T17:51:23.459 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 7446 2024-04-07T17:51:23.459 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.459 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_extent": { 2024-04-07T17:51:23.459 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 87840, 2024-04-07T17:51:23.459 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 1830 2024-04-07T17:51:23.460 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.460 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_fsck": { 2024-04-07T17:51:23.460 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.460 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.460 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.460 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_inline_bl": { 2024-04-07T17:51:23.460 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 175624, 2024-04-07T17:51:23.460 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 1258 2024-04-07T17:51:23.460 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.460 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_shared_blob": { 2024-04-07T17:51:23.460 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.460 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.460 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.460 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_txc": { 2024-04-07T17:51:23.460 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 736, 2024-04-07T17:51:23.461 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 1 2024-04-07T17:51:23.461 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.461 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_writing": { 2024-04-07T17:51:23.461 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 83741, 2024-04-07T17:51:23.461 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 55 2024-04-07T17:51:23.461 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.461 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_writing_deferred": { 2024-04-07T17:51:23.461 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 12112, 2024-04-07T17:51:23.461 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 3 2024-04-07T17:51:23.461 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.461 INFO:tasks.workunit.client.0.smithi100.stdout: "buffer_anon": { 2024-04-07T17:51:23.461 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 18443066, 2024-04-07T17:51:23.461 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 280 2024-04-07T17:51:23.461 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.461 INFO:tasks.workunit.client.0.smithi100.stdout: "buffer_meta": { 2024-04-07T17:51:23.462 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 11880, 2024-04-07T17:51:23.462 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 135 2024-04-07T17:51:23.462 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.462 INFO:tasks.workunit.client.0.smithi100.stdout: "mds_co": { 2024-04-07T17:51:23.462 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.462 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.462 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.462 INFO:tasks.workunit.client.0.smithi100.stdout: "osd": { 2024-04-07T17:51:23.462 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 419904, 2024-04-07T17:51:23.462 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 36 2024-04-07T17:51:23.462 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.462 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mapbl": { 2024-04-07T17:51:23.462 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.462 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.463 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.463 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_pglog": { 2024-04-07T17:51:23.463 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 3485560, 2024-04-07T17:51:23.463 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 9103 2024-04-07T17:51:23.463 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.463 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap": { 2024-04-07T17:51:23.463 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 635376, 2024-04-07T17:51:23.463 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 11076 2024-04-07T17:51:23.463 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.463 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap_mapping": { 2024-04-07T17:51:23.463 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.463 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.463 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.463 INFO:tasks.workunit.client.0.smithi100.stdout: "pgmap": { 2024-04-07T17:51:23.463 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.464 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.464 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.464 INFO:tasks.workunit.client.0.smithi100.stdout: "unittest_1": { 2024-04-07T17:51:23.464 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.464 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.464 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.464 INFO:tasks.workunit.client.0.smithi100.stdout: "unittest_2": { 2024-04-07T17:51:23.464 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.464 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.464 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:23.464 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.464 INFO:tasks.workunit.client.0.smithi100.stdout: "osd.4": { 2024-04-07T17:51:23.464 INFO:tasks.workunit.client.0.smithi100.stdout: "bloom_filter": { 2024-04-07T17:51:23.464 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.464 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.464 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.465 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs": { 2024-04-07T17:51:23.465 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 20816, 2024-04-07T17:51:23.465 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 393 2024-04-07T17:51:23.465 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.465 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs_file_reader": { 2024-04-07T17:51:23.465 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 31488, 2024-04-07T17:51:23.465 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 29 2024-04-07T17:51:23.465 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.465 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs_file_writer": { 2024-04-07T17:51:23.465 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 4032, 2024-04-07T17:51:23.465 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 21 2024-04-07T17:51:23.465 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.465 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_alloc": { 2024-04-07T17:51:23.465 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 58240, 2024-04-07T17:51:23.466 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 728 2024-04-07T17:51:23.466 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.466 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_blob": { 2024-04-07T17:51:23.466 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 393952, 2024-04-07T17:51:23.466 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 1894 2024-04-07T17:51:23.466 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.466 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_buffer": { 2024-04-07T17:51:23.466 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 256, 2024-04-07T17:51:23.466 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 32 2024-04-07T17:51:23.466 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.466 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_data": { 2024-04-07T17:51:23.466 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 1050407, 2024-04-07T17:51:23.466 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 273 2024-04-07T17:51:23.466 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.467 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_meta": { 2024-04-07T17:51:23.467 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 2027647, 2024-04-07T17:51:23.467 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 446306 2024-04-07T17:51:23.467 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.467 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_onode": { 2024-04-07T17:51:23.467 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 5812216, 2024-04-07T17:51:23.467 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 8579 2024-04-07T17:51:23.467 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.467 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_other": { 2024-04-07T17:51:23.467 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 80384, 2024-04-07T17:51:23.467 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 8599 2024-04-07T17:51:23.467 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.467 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_extent": { 2024-04-07T17:51:23.467 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 90912, 2024-04-07T17:51:23.467 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 1894 2024-04-07T17:51:23.468 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.468 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_fsck": { 2024-04-07T17:51:23.468 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.468 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.468 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.468 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_inline_bl": { 2024-04-07T17:51:23.468 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 175456, 2024-04-07T17:51:23.468 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 1258 2024-04-07T17:51:23.468 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.468 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_shared_blob": { 2024-04-07T17:51:23.468 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.468 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.468 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.468 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_txc": { 2024-04-07T17:51:23.468 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 736, 2024-04-07T17:51:23.469 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 1 2024-04-07T17:51:23.469 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.469 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_writing": { 2024-04-07T17:51:23.469 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 66705, 2024-04-07T17:51:23.469 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 51 2024-04-07T17:51:23.469 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.469 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_writing_deferred": { 2024-04-07T17:51:23.469 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 12112, 2024-04-07T17:51:23.469 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 3 2024-04-07T17:51:23.469 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.469 INFO:tasks.workunit.client.0.smithi100.stdout: "buffer_anon": { 2024-04-07T17:51:23.469 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 23146638, 2024-04-07T17:51:23.469 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 273 2024-04-07T17:51:23.469 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.469 INFO:tasks.workunit.client.0.smithi100.stdout: "buffer_meta": { 2024-04-07T17:51:23.470 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 26928, 2024-04-07T17:51:23.470 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 306 2024-04-07T17:51:23.470 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.470 INFO:tasks.workunit.client.0.smithi100.stdout: "mds_co": { 2024-04-07T17:51:23.470 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.470 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.470 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.470 INFO:tasks.workunit.client.0.smithi100.stdout: "osd": { 2024-04-07T17:51:23.470 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 466560, 2024-04-07T17:51:23.470 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 40 2024-04-07T17:51:23.470 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.470 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mapbl": { 2024-04-07T17:51:23.470 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.470 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.470 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.471 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_pglog": { 2024-04-07T17:51:23.471 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 5065320, 2024-04-07T17:51:23.471 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 12141 2024-04-07T17:51:23.471 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.471 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap": { 2024-04-07T17:51:23.471 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 635376, 2024-04-07T17:51:23.471 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 11076 2024-04-07T17:51:23.471 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.471 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap_mapping": { 2024-04-07T17:51:23.471 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.471 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.471 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.471 INFO:tasks.workunit.client.0.smithi100.stdout: "pgmap": { 2024-04-07T17:51:23.471 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.471 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.472 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.472 INFO:tasks.workunit.client.0.smithi100.stdout: "unittest_1": { 2024-04-07T17:51:23.472 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.472 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.472 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.472 INFO:tasks.workunit.client.0.smithi100.stdout: "unittest_2": { 2024-04-07T17:51:23.472 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.472 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.472 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:23.472 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.472 INFO:tasks.workunit.client.0.smithi100.stdout: "osd.5": { 2024-04-07T17:51:23.472 INFO:tasks.workunit.client.0.smithi100.stdout: "bloom_filter": { 2024-04-07T17:51:23.472 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.472 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.472 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.473 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs": { 2024-04-07T17:51:23.473 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 19216, 2024-04-07T17:51:23.473 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 365 2024-04-07T17:51:23.473 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.473 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs_file_reader": { 2024-04-07T17:51:23.473 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 31616, 2024-04-07T17:51:23.473 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 30 2024-04-07T17:51:23.473 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.473 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs_file_writer": { 2024-04-07T17:51:23.473 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 3072, 2024-04-07T17:51:23.473 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 16 2024-04-07T17:51:23.474 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.474 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_alloc": { 2024-04-07T17:51:23.474 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 58400, 2024-04-07T17:51:23.474 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 730 2024-04-07T17:51:23.474 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.474 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_blob": { 2024-04-07T17:51:23.474 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 393952, 2024-04-07T17:51:23.474 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 1894 2024-04-07T17:51:23.474 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.474 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_buffer": { 2024-04-07T17:51:23.474 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 256, 2024-04-07T17:51:23.474 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 32 2024-04-07T17:51:23.474 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.475 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_data": { 2024-04-07T17:51:23.475 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 274397, 2024-04-07T17:51:23.475 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 87 2024-04-07T17:51:23.475 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.475 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_meta": { 2024-04-07T17:51:23.475 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 2323821, 2024-04-07T17:51:23.475 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 518276 2024-04-07T17:51:23.475 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.475 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_onode": { 2024-04-07T17:51:23.475 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 6677176, 2024-04-07T17:51:23.475 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 9851 2024-04-07T17:51:23.475 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.475 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_other": { 2024-04-07T17:51:23.475 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 80636, 2024-04-07T17:51:23.475 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 8604 2024-04-07T17:51:23.476 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.476 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_extent": { 2024-04-07T17:51:23.476 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 90912, 2024-04-07T17:51:23.476 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 1894 2024-04-07T17:51:23.476 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.476 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_fsck": { 2024-04-07T17:51:23.476 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.476 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.476 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.476 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_inline_bl": { 2024-04-07T17:51:23.476 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 175712, 2024-04-07T17:51:23.476 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 1259 2024-04-07T17:51:23.476 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.476 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_shared_blob": { 2024-04-07T17:51:23.477 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.477 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.477 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.477 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_txc": { 2024-04-07T17:51:23.477 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 736, 2024-04-07T17:51:23.477 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 1 2024-04-07T17:51:23.477 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.477 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_writing": { 2024-04-07T17:51:23.477 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 87571, 2024-04-07T17:51:23.477 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 56 2024-04-07T17:51:23.477 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.477 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_writing_deferred": { 2024-04-07T17:51:23.477 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 12112, 2024-04-07T17:51:23.477 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 3 2024-04-07T17:51:23.477 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.478 INFO:tasks.workunit.client.0.smithi100.stdout: "buffer_anon": { 2024-04-07T17:51:23.478 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 18319894, 2024-04-07T17:51:23.478 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 12576 2024-04-07T17:51:23.478 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.478 INFO:tasks.workunit.client.0.smithi100.stdout: "buffer_meta": { 2024-04-07T17:51:23.478 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 10208, 2024-04-07T17:51:23.478 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 116 2024-04-07T17:51:23.478 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.478 INFO:tasks.workunit.client.0.smithi100.stdout: "mds_co": { 2024-04-07T17:51:23.478 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.478 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.478 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.478 INFO:tasks.workunit.client.0.smithi100.stdout: "osd": { 2024-04-07T17:51:23.479 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 489888, 2024-04-07T17:51:23.479 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 42 2024-04-07T17:51:23.479 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.479 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mapbl": { 2024-04-07T17:51:23.479 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.479 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.479 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.479 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_pglog": { 2024-04-07T17:51:23.479 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 6315920, 2024-04-07T17:51:23.479 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 12146 2024-04-07T17:51:23.479 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.479 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap": { 2024-04-07T17:51:23.479 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 635376, 2024-04-07T17:51:23.479 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 11076 2024-04-07T17:51:23.479 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.480 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap_mapping": { 2024-04-07T17:51:23.480 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.480 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.480 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.480 INFO:tasks.workunit.client.0.smithi100.stdout: "pgmap": { 2024-04-07T17:51:23.480 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.480 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.480 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.480 INFO:tasks.workunit.client.0.smithi100.stdout: "unittest_1": { 2024-04-07T17:51:23.480 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.480 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.480 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.480 INFO:tasks.workunit.client.0.smithi100.stdout: "unittest_2": { 2024-04-07T17:51:23.480 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.481 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.481 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:23.481 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.481 INFO:tasks.workunit.client.0.smithi100.stdout: "osd.6": { 2024-04-07T17:51:23.481 INFO:tasks.workunit.client.0.smithi100.stdout: "bloom_filter": { 2024-04-07T17:51:23.481 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.481 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.481 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.481 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs": { 2024-04-07T17:51:23.481 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 13656, 2024-04-07T17:51:23.481 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 230 2024-04-07T17:51:23.481 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.481 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs_file_reader": { 2024-04-07T17:51:23.481 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 30336, 2024-04-07T17:51:23.481 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 20 2024-04-07T17:51:23.482 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.482 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs_file_writer": { 2024-04-07T17:51:23.482 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 2496, 2024-04-07T17:51:23.482 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 13 2024-04-07T17:51:23.482 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.482 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_alloc": { 2024-04-07T17:51:23.482 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 56880, 2024-04-07T17:51:23.482 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 711 2024-04-07T17:51:23.482 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.482 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_blob": { 2024-04-07T17:51:23.482 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 395616, 2024-04-07T17:51:23.482 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 1902 2024-04-07T17:51:23.482 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.482 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_buffer": { 2024-04-07T17:51:23.483 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 256, 2024-04-07T17:51:23.483 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 32 2024-04-07T17:51:23.483 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.483 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_data": { 2024-04-07T17:51:23.483 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 463979, 2024-04-07T17:51:23.483 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 128 2024-04-07T17:51:23.483 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.483 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_meta": { 2024-04-07T17:51:23.483 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 1719133, 2024-04-07T17:51:23.483 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 377364 2024-04-07T17:51:23.483 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.483 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_onode": { 2024-04-07T17:51:23.483 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 4954056, 2024-04-07T17:51:23.483 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 7317 2024-04-07T17:51:23.484 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.484 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_other": { 2024-04-07T17:51:23.484 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 81576, 2024-04-07T17:51:23.484 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 8761 2024-04-07T17:51:23.484 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.484 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_extent": { 2024-04-07T17:51:23.484 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 91296, 2024-04-07T17:51:23.484 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 1902 2024-04-07T17:51:23.484 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.484 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_fsck": { 2024-04-07T17:51:23.484 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.484 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.484 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.484 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_inline_bl": { 2024-04-07T17:51:23.484 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 175368, 2024-04-07T17:51:23.485 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 1257 2024-04-07T17:51:23.485 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.485 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_shared_blob": { 2024-04-07T17:51:23.485 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.485 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.485 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.485 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_txc": { 2024-04-07T17:51:23.485 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 736, 2024-04-07T17:51:23.485 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 1 2024-04-07T17:51:23.485 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.485 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_writing": { 2024-04-07T17:51:23.485 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 101893, 2024-04-07T17:51:23.485 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 56 2024-04-07T17:51:23.485 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.485 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_writing_deferred": { 2024-04-07T17:51:23.486 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 12112, 2024-04-07T17:51:23.486 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 3 2024-04-07T17:51:23.486 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.486 INFO:tasks.workunit.client.0.smithi100.stdout: "buffer_anon": { 2024-04-07T17:51:23.486 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 13267228, 2024-04-07T17:51:23.486 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 243 2024-04-07T17:51:23.486 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.486 INFO:tasks.workunit.client.0.smithi100.stdout: "buffer_meta": { 2024-04-07T17:51:23.486 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 12408, 2024-04-07T17:51:23.486 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 141 2024-04-07T17:51:23.486 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.486 INFO:tasks.workunit.client.0.smithi100.stdout: "mds_co": { 2024-04-07T17:51:23.486 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.487 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.487 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.487 INFO:tasks.workunit.client.0.smithi100.stdout: "osd": { 2024-04-07T17:51:23.487 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 454896, 2024-04-07T17:51:23.487 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 39 2024-04-07T17:51:23.487 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.487 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mapbl": { 2024-04-07T17:51:23.487 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.487 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.487 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.487 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_pglog": { 2024-04-07T17:51:23.487 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 3660800, 2024-04-07T17:51:23.487 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 9440 2024-04-07T17:51:23.487 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.487 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap": { 2024-04-07T17:51:23.487 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 635376, 2024-04-07T17:51:23.488 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 11076 2024-04-07T17:51:23.488 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.488 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap_mapping": { 2024-04-07T17:51:23.488 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.488 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.488 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.488 INFO:tasks.workunit.client.0.smithi100.stdout: "pgmap": { 2024-04-07T17:51:23.488 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.488 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.488 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.488 INFO:tasks.workunit.client.0.smithi100.stdout: "unittest_1": { 2024-04-07T17:51:23.488 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.488 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.488 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.488 INFO:tasks.workunit.client.0.smithi100.stdout: "unittest_2": { 2024-04-07T17:51:23.489 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.489 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.489 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:23.489 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.489 INFO:tasks.workunit.client.0.smithi100.stdout: "osd.7": { 2024-04-07T17:51:23.489 INFO:tasks.workunit.client.0.smithi100.stdout: "bloom_filter": { 2024-04-07T17:51:23.489 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.489 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.489 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.489 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs": { 2024-04-07T17:51:23.489 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 15992, 2024-04-07T17:51:23.489 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 307 2024-04-07T17:51:23.489 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.489 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs_file_reader": { 2024-04-07T17:51:23.489 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 39040, 2024-04-07T17:51:23.490 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 25 2024-04-07T17:51:23.490 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.490 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs_file_writer": { 2024-04-07T17:51:23.490 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 2688, 2024-04-07T17:51:23.490 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 14 2024-04-07T17:51:23.490 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.490 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_alloc": { 2024-04-07T17:51:23.490 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 57200, 2024-04-07T17:51:23.490 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 715 2024-04-07T17:51:23.490 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.490 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_blob": { 2024-04-07T17:51:23.490 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 382096, 2024-04-07T17:51:23.490 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 1837 2024-04-07T17:51:23.490 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.490 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_buffer": { 2024-04-07T17:51:23.491 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 256, 2024-04-07T17:51:23.491 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 32 2024-04-07T17:51:23.491 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.491 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_data": { 2024-04-07T17:51:23.491 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 358315, 2024-04-07T17:51:23.491 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 110 2024-04-07T17:51:23.491 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.491 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_meta": { 2024-04-07T17:51:23.491 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 1986086, 2024-04-07T17:51:23.491 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 443610 2024-04-07T17:51:23.492 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.492 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_onode": { 2024-04-07T17:51:23.492 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 5763936, 2024-04-07T17:51:23.492 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 8508 2024-04-07T17:51:23.492 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.492 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_other": { 2024-04-07T17:51:23.492 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 73096, 2024-04-07T17:51:23.492 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 7611 2024-04-07T17:51:23.492 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.492 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_extent": { 2024-04-07T17:51:23.492 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 88176, 2024-04-07T17:51:23.492 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 1837 2024-04-07T17:51:23.492 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.492 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_fsck": { 2024-04-07T17:51:23.492 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.493 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.493 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.493 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_inline_bl": { 2024-04-07T17:51:23.493 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 175624, 2024-04-07T17:51:23.493 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 1258 2024-04-07T17:51:23.493 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.493 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_shared_blob": { 2024-04-07T17:51:23.493 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.493 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.493 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.493 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_txc": { 2024-04-07T17:51:23.493 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 736, 2024-04-07T17:51:23.493 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 1 2024-04-07T17:51:23.493 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.493 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_writing": { 2024-04-07T17:51:23.494 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 69646, 2024-04-07T17:51:23.494 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 48 2024-04-07T17:51:23.494 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.494 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_writing_deferred": { 2024-04-07T17:51:23.494 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 12112, 2024-04-07T17:51:23.494 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 3 2024-04-07T17:51:23.494 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.494 INFO:tasks.workunit.client.0.smithi100.stdout: "buffer_anon": { 2024-04-07T17:51:23.494 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 14946714, 2024-04-07T17:51:23.494 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 270 2024-04-07T17:51:23.494 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.494 INFO:tasks.workunit.client.0.smithi100.stdout: "buffer_meta": { 2024-04-07T17:51:23.494 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 10824, 2024-04-07T17:51:23.494 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 123 2024-04-07T17:51:23.494 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.495 INFO:tasks.workunit.client.0.smithi100.stdout: "mds_co": { 2024-04-07T17:51:23.495 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.495 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.495 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.495 INFO:tasks.workunit.client.0.smithi100.stdout: "osd": { 2024-04-07T17:51:23.495 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 536544, 2024-04-07T17:51:23.495 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 46 2024-04-07T17:51:23.495 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.495 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mapbl": { 2024-04-07T17:51:23.495 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.495 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.495 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.495 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_pglog": { 2024-04-07T17:51:23.495 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 4998240, 2024-04-07T17:51:23.495 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 9628 2024-04-07T17:51:23.496 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.496 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap": { 2024-04-07T17:51:23.496 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 635376, 2024-04-07T17:51:23.496 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 11076 2024-04-07T17:51:23.496 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.496 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap_mapping": { 2024-04-07T17:51:23.496 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.496 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.496 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.496 INFO:tasks.workunit.client.0.smithi100.stdout: "pgmap": { 2024-04-07T17:51:23.496 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.496 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.496 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.496 INFO:tasks.workunit.client.0.smithi100.stdout: "unittest_1": { 2024-04-07T17:51:23.496 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.496 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.497 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.497 INFO:tasks.workunit.client.0.smithi100.stdout: "unittest_2": { 2024-04-07T17:51:23.497 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:23.497 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:23.497 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:23.497 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:23.497 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:23.497 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.497 INFO:tasks.workunit.client.0.smithi100.stdout: "metadata": { 2024-04-07T17:51:23.497 INFO:tasks.workunit.client.0.smithi100.stdout: "mds": { 2024-04-07T17:51:23.497 INFO:tasks.workunit.client.0.smithi100.stdout: "arch": { 2024-04-07T17:51:23.497 INFO:tasks.workunit.client.0.smithi100.stdout: "x86_64": 2 2024-04-07T17:51:23.497 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.497 INFO:tasks.workunit.client.0.smithi100.stdout: "ceph_version": { 2024-04-07T17:51:23.497 INFO:tasks.workunit.client.0.smithi100.stdout: "ceph version 19.0.0-2785-gadce92d0 (adce92d056533f22dc1ee37e3141d098b046ec1b) squid (dev)": 2 2024-04-07T17:51:23.498 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.498 INFO:tasks.workunit.client.0.smithi100.stdout: "cpu": { 2024-04-07T17:51:23.498 INFO:tasks.workunit.client.0.smithi100.stdout: "Intel(R) Xeon(R) CPU E5-1620 v4 @ 3.50GHz": 2 2024-04-07T17:51:23.498 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.498 INFO:tasks.workunit.client.0.smithi100.stdout: "distro": { 2024-04-07T17:51:23.498 INFO:tasks.workunit.client.0.smithi100.stdout: "centos": 2 2024-04-07T17:51:23.498 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.498 INFO:tasks.workunit.client.0.smithi100.stdout: "distro_description": { 2024-04-07T17:51:23.498 INFO:tasks.workunit.client.0.smithi100.stdout: "CentOS Stream 8": 2 2024-04-07T17:51:23.498 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.498 INFO:tasks.workunit.client.0.smithi100.stdout: "kernel_description": { 2024-04-07T17:51:23.498 INFO:tasks.workunit.client.0.smithi100.stdout: "#111-Ubuntu SMP Tue Mar 5 20:16:58 UTC 2024": 2 2024-04-07T17:51:23.498 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.499 INFO:tasks.workunit.client.0.smithi100.stdout: "kernel_version": { 2024-04-07T17:51:23.499 INFO:tasks.workunit.client.0.smithi100.stdout: "5.15.0-101-generic": 2 2024-04-07T17:51:23.499 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.499 INFO:tasks.workunit.client.0.smithi100.stdout: "os": { 2024-04-07T17:51:23.499 INFO:tasks.workunit.client.0.smithi100.stdout: "Linux": 2 2024-04-07T17:51:23.499 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:23.499 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.499 INFO:tasks.workunit.client.0.smithi100.stdout: "mon": { 2024-04-07T17:51:23.499 INFO:tasks.workunit.client.0.smithi100.stdout: "arch": { 2024-04-07T17:51:23.499 INFO:tasks.workunit.client.0.smithi100.stdout: "x86_64": 3 2024-04-07T17:51:23.499 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.499 INFO:tasks.workunit.client.0.smithi100.stdout: "ceph_version": { 2024-04-07T17:51:23.499 INFO:tasks.workunit.client.0.smithi100.stdout: "ceph version 19.0.0-2785-gadce92d0 (adce92d056533f22dc1ee37e3141d098b046ec1b) squid (dev)": 3 2024-04-07T17:51:23.499 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.499 INFO:tasks.workunit.client.0.smithi100.stdout: "cpu": { 2024-04-07T17:51:23.500 INFO:tasks.workunit.client.0.smithi100.stdout: "Intel(R) Xeon(R) CPU E5-1620 v4 @ 3.50GHz": 3 2024-04-07T17:51:23.500 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.500 INFO:tasks.workunit.client.0.smithi100.stdout: "distro": { 2024-04-07T17:51:23.500 INFO:tasks.workunit.client.0.smithi100.stdout: "centos": 3 2024-04-07T17:51:23.500 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.500 INFO:tasks.workunit.client.0.smithi100.stdout: "distro_description": { 2024-04-07T17:51:23.500 INFO:tasks.workunit.client.0.smithi100.stdout: "CentOS Stream 8": 3 2024-04-07T17:51:23.500 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.500 INFO:tasks.workunit.client.0.smithi100.stdout: "kernel_description": { 2024-04-07T17:51:23.500 INFO:tasks.workunit.client.0.smithi100.stdout: "#111-Ubuntu SMP Tue Mar 5 20:16:58 UTC 2024": 3 2024-04-07T17:51:23.500 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.500 INFO:tasks.workunit.client.0.smithi100.stdout: "kernel_version": { 2024-04-07T17:51:23.500 INFO:tasks.workunit.client.0.smithi100.stdout: "5.15.0-101-generic": 3 2024-04-07T17:51:23.500 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.501 INFO:tasks.workunit.client.0.smithi100.stdout: "os": { 2024-04-07T17:51:23.501 INFO:tasks.workunit.client.0.smithi100.stdout: "Linux": 3 2024-04-07T17:51:23.501 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:23.501 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.501 INFO:tasks.workunit.client.0.smithi100.stdout: "osd": { 2024-04-07T17:51:23.501 INFO:tasks.workunit.client.0.smithi100.stdout: "arch": { 2024-04-07T17:51:23.501 INFO:tasks.workunit.client.0.smithi100.stdout: "x86_64": 8 2024-04-07T17:51:23.501 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.501 INFO:tasks.workunit.client.0.smithi100.stdout: "ceph_version": { 2024-04-07T17:51:23.501 INFO:tasks.workunit.client.0.smithi100.stdout: "ceph version 19.0.0-2785-gadce92d0 (adce92d056533f22dc1ee37e3141d098b046ec1b) squid (dev)": 8 2024-04-07T17:51:23.501 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.501 INFO:tasks.workunit.client.0.smithi100.stdout: "cpu": { 2024-04-07T17:51:23.501 INFO:tasks.workunit.client.0.smithi100.stdout: "Intel(R) Xeon(R) CPU E5-1620 v4 @ 3.50GHz": 8 2024-04-07T17:51:23.501 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.501 INFO:tasks.workunit.client.0.smithi100.stdout: "distro": { 2024-04-07T17:51:23.502 INFO:tasks.workunit.client.0.smithi100.stdout: "centos": 8 2024-04-07T17:51:23.502 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.502 INFO:tasks.workunit.client.0.smithi100.stdout: "distro_description": { 2024-04-07T17:51:23.502 INFO:tasks.workunit.client.0.smithi100.stdout: "CentOS Stream 8": 8 2024-04-07T17:51:23.502 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.502 INFO:tasks.workunit.client.0.smithi100.stdout: "kernel_description": { 2024-04-07T17:51:23.502 INFO:tasks.workunit.client.0.smithi100.stdout: "#111-Ubuntu SMP Tue Mar 5 20:16:58 UTC 2024": 8 2024-04-07T17:51:23.502 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.502 INFO:tasks.workunit.client.0.smithi100.stdout: "kernel_version": { 2024-04-07T17:51:23.502 INFO:tasks.workunit.client.0.smithi100.stdout: "5.15.0-101-generic": 8 2024-04-07T17:51:23.502 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.502 INFO:tasks.workunit.client.0.smithi100.stdout: "os": { 2024-04-07T17:51:23.502 INFO:tasks.workunit.client.0.smithi100.stdout: "Linux": 8 2024-04-07T17:51:23.502 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.503 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_objectstore": { 2024-04-07T17:51:23.503 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore": 8 2024-04-07T17:51:23.503 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.503 INFO:tasks.workunit.client.0.smithi100.stdout: "rotational": { 2024-04-07T17:51:23.503 INFO:tasks.workunit.client.0.smithi100.stdout: "0": 8 2024-04-07T17:51:23.503 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:23.503 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:23.503 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.503 INFO:tasks.workunit.client.0.smithi100.stdout: "mon": { 2024-04-07T17:51:23.503 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 3, 2024-04-07T17:51:23.503 INFO:tasks.workunit.client.0.smithi100.stdout: "features": { 2024-04-07T17:51:23.503 INFO:tasks.workunit.client.0.smithi100.stdout: "optional": [], 2024-04-07T17:51:23.503 INFO:tasks.workunit.client.0.smithi100.stdout: "persistent": [ 2024-04-07T17:51:23.503 INFO:tasks.workunit.client.0.smithi100.stdout: "kraken", 2024-04-07T17:51:23.504 INFO:tasks.workunit.client.0.smithi100.stdout: "luminous", 2024-04-07T17:51:23.504 INFO:tasks.workunit.client.0.smithi100.stdout: "mimic", 2024-04-07T17:51:23.504 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap-prune", 2024-04-07T17:51:23.504 INFO:tasks.workunit.client.0.smithi100.stdout: "nautilus", 2024-04-07T17:51:23.504 INFO:tasks.workunit.client.0.smithi100.stdout: "octopus", 2024-04-07T17:51:23.504 INFO:tasks.workunit.client.0.smithi100.stdout: "pacific", 2024-04-07T17:51:23.504 INFO:tasks.workunit.client.0.smithi100.stdout: "elector-pinging", 2024-04-07T17:51:23.504 INFO:tasks.workunit.client.0.smithi100.stdout: "quincy", 2024-04-07T17:51:23.504 INFO:tasks.workunit.client.0.smithi100.stdout: "reef", 2024-04-07T17:51:23.504 INFO:tasks.workunit.client.0.smithi100.stdout: "squid" 2024-04-07T17:51:23.504 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:23.504 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.504 INFO:tasks.workunit.client.0.smithi100.stdout: "ipv4_addr_mons": 6, 2024-04-07T17:51:23.504 INFO:tasks.workunit.client.0.smithi100.stdout: "ipv6_addr_mons": 0, 2024-04-07T17:51:23.504 INFO:tasks.workunit.client.0.smithi100.stdout: "min_mon_release": 19, 2024-04-07T17:51:23.504 INFO:tasks.workunit.client.0.smithi100.stdout: "v1_addr_mons": 3, 2024-04-07T17:51:23.505 INFO:tasks.workunit.client.0.smithi100.stdout: "v2_addr_mons": 3 2024-04-07T17:51:23.505 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.505 INFO:tasks.workunit.client.0.smithi100.stdout: "organization": "ceph-qa", 2024-04-07T17:51:23.505 INFO:tasks.workunit.client.0.smithi100.stdout: "osd": { 2024-04-07T17:51:23.505 INFO:tasks.workunit.client.0.smithi100.stdout: "cluster_network": false, 2024-04-07T17:51:23.505 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 8, 2024-04-07T17:51:23.505 INFO:tasks.workunit.client.0.smithi100.stdout: "require_min_compat_client": "luminous", 2024-04-07T17:51:23.505 INFO:tasks.workunit.client.0.smithi100.stdout: "require_osd_release": "squid" 2024-04-07T17:51:23.505 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.505 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_perf_histograms": [ 2024-04-07T17:51:23.505 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:23.505 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_latency_out_bytes_histogram": { 2024-04-07T17:51:23.505 INFO:tasks.workunit.client.0.smithi100.stdout: "axes": [ 2024-04-07T17:51:23.505 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:23.505 INFO:tasks.workunit.client.0.smithi100.stdout: "buckets": 32, 2024-04-07T17:51:23.505 INFO:tasks.workunit.client.0.smithi100.stdout: "min": 0, 2024-04-07T17:51:23.506 INFO:tasks.workunit.client.0.smithi100.stdout: "name": "Latency (usec)", 2024-04-07T17:51:23.506 INFO:tasks.workunit.client.0.smithi100.stdout: "quant_size": 100000, 2024-04-07T17:51:23.506 INFO:tasks.workunit.client.0.smithi100.stdout: "ranges": [ 2024-04-07T17:51:23.506 INFO:tasks.workunit.client.0.smithi100.stdout: "[None, -1]", 2024-04-07T17:51:23.506 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 99999]", 2024-04-07T17:51:23.506 INFO:tasks.workunit.client.0.smithi100.stdout: "[100000, 199999]", 2024-04-07T17:51:23.506 INFO:tasks.workunit.client.0.smithi100.stdout: "[200000, 399999]", 2024-04-07T17:51:23.506 INFO:tasks.workunit.client.0.smithi100.stdout: "[400000, 799999]", 2024-04-07T17:51:23.506 INFO:tasks.workunit.client.0.smithi100.stdout: "[800000, 1599999]", 2024-04-07T17:51:23.506 INFO:tasks.workunit.client.0.smithi100.stdout: "[1600000, 3199999]", 2024-04-07T17:51:23.506 INFO:tasks.workunit.client.0.smithi100.stdout: "[3200000, 6399999]", 2024-04-07T17:51:23.506 INFO:tasks.workunit.client.0.smithi100.stdout: "[6400000, 12799999]", 2024-04-07T17:51:23.506 INFO:tasks.workunit.client.0.smithi100.stdout: "[12800000, 25599999]", 2024-04-07T17:51:23.506 INFO:tasks.workunit.client.0.smithi100.stdout: "[25600000, 51199999]", 2024-04-07T17:51:23.506 INFO:tasks.workunit.client.0.smithi100.stdout: "[51200000, 102399999]", 2024-04-07T17:51:23.507 INFO:tasks.workunit.client.0.smithi100.stdout: "[102400000, 204799999]", 2024-04-07T17:51:23.507 INFO:tasks.workunit.client.0.smithi100.stdout: "[204800000, 409599999]", 2024-04-07T17:51:23.507 INFO:tasks.workunit.client.0.smithi100.stdout: "[409600000, 819199999]", 2024-04-07T17:51:23.507 INFO:tasks.workunit.client.0.smithi100.stdout: "[819200000, 1638399999]", 2024-04-07T17:51:23.507 INFO:tasks.workunit.client.0.smithi100.stdout: "[1638400000, 3276799999]", 2024-04-07T17:51:23.507 INFO:tasks.workunit.client.0.smithi100.stdout: "[3276800000, 6553599999]", 2024-04-07T17:51:23.507 INFO:tasks.workunit.client.0.smithi100.stdout: "[6553600000, 13107199999]", 2024-04-07T17:51:23.507 INFO:tasks.workunit.client.0.smithi100.stdout: "[13107200000, 26214399999]", 2024-04-07T17:51:23.507 INFO:tasks.workunit.client.0.smithi100.stdout: "[26214400000, 52428799999]", 2024-04-07T17:51:23.507 INFO:tasks.workunit.client.0.smithi100.stdout: "[52428800000, 104857599999]", 2024-04-07T17:51:23.507 INFO:tasks.workunit.client.0.smithi100.stdout: "[104857600000, 209715199999]", 2024-04-07T17:51:23.507 INFO:tasks.workunit.client.0.smithi100.stdout: "[209715200000, 419430399999]", 2024-04-07T17:51:23.507 INFO:tasks.workunit.client.0.smithi100.stdout: "[419430400000, 838860799999]", 2024-04-07T17:51:23.508 INFO:tasks.workunit.client.0.smithi100.stdout: "[838860800000, 1677721599999]", 2024-04-07T17:51:23.508 INFO:tasks.workunit.client.0.smithi100.stdout: "[1677721600000, 3355443199999]", 2024-04-07T17:51:23.508 INFO:tasks.workunit.client.0.smithi100.stdout: "[3355443200000, 6710886399999]", 2024-04-07T17:51:23.508 INFO:tasks.workunit.client.0.smithi100.stdout: "[6710886400000, 13421772799999]", 2024-04-07T17:51:23.508 INFO:tasks.workunit.client.0.smithi100.stdout: "[13421772800000, 26843545599999]", 2024-04-07T17:51:23.508 INFO:tasks.workunit.client.0.smithi100.stdout: "[26843545600000, 53687091199999]", 2024-04-07T17:51:23.508 INFO:tasks.workunit.client.0.smithi100.stdout: "[53687091200000, None]" 2024-04-07T17:51:23.508 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:23.508 INFO:tasks.workunit.client.0.smithi100.stdout: "scale_type": "log2" 2024-04-07T17:51:23.508 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.508 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:23.508 INFO:tasks.workunit.client.0.smithi100.stdout: "buckets": 32, 2024-04-07T17:51:23.508 INFO:tasks.workunit.client.0.smithi100.stdout: "min": 0, 2024-04-07T17:51:23.508 INFO:tasks.workunit.client.0.smithi100.stdout: "name": "Request size (bytes)", 2024-04-07T17:51:23.508 INFO:tasks.workunit.client.0.smithi100.stdout: "quant_size": 512, 2024-04-07T17:51:23.509 INFO:tasks.workunit.client.0.smithi100.stdout: "ranges": [ 2024-04-07T17:51:23.509 INFO:tasks.workunit.client.0.smithi100.stdout: "[None, -1]", 2024-04-07T17:51:23.509 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 511]", 2024-04-07T17:51:23.509 INFO:tasks.workunit.client.0.smithi100.stdout: "[512, 1023]", 2024-04-07T17:51:23.509 INFO:tasks.workunit.client.0.smithi100.stdout: "[1024, 2047]", 2024-04-07T17:51:23.509 INFO:tasks.workunit.client.0.smithi100.stdout: "[2048, 4095]", 2024-04-07T17:51:23.509 INFO:tasks.workunit.client.0.smithi100.stdout: "[4096, 8191]", 2024-04-07T17:51:23.509 INFO:tasks.workunit.client.0.smithi100.stdout: "[8192, 16383]", 2024-04-07T17:51:23.509 INFO:tasks.workunit.client.0.smithi100.stdout: "[16384, 32767]", 2024-04-07T17:51:23.509 INFO:tasks.workunit.client.0.smithi100.stdout: "[32768, 65535]", 2024-04-07T17:51:23.509 INFO:tasks.workunit.client.0.smithi100.stdout: "[65536, 131071]", 2024-04-07T17:51:23.509 INFO:tasks.workunit.client.0.smithi100.stdout: "[131072, 262143]", 2024-04-07T17:51:23.509 INFO:tasks.workunit.client.0.smithi100.stdout: "[262144, 524287]", 2024-04-07T17:51:23.509 INFO:tasks.workunit.client.0.smithi100.stdout: "[524288, 1048575]", 2024-04-07T17:51:23.510 INFO:tasks.workunit.client.0.smithi100.stdout: "[1048576, 2097151]", 2024-04-07T17:51:23.510 INFO:tasks.workunit.client.0.smithi100.stdout: "[2097152, 4194303]", 2024-04-07T17:51:23.510 INFO:tasks.workunit.client.0.smithi100.stdout: "[4194304, 8388607]", 2024-04-07T17:51:23.510 INFO:tasks.workunit.client.0.smithi100.stdout: "[8388608, 16777215]", 2024-04-07T17:51:23.510 INFO:tasks.workunit.client.0.smithi100.stdout: "[16777216, 33554431]", 2024-04-07T17:51:23.510 INFO:tasks.workunit.client.0.smithi100.stdout: "[33554432, 67108863]", 2024-04-07T17:51:23.510 INFO:tasks.workunit.client.0.smithi100.stdout: "[67108864, 134217727]", 2024-04-07T17:51:23.510 INFO:tasks.workunit.client.0.smithi100.stdout: "[134217728, 268435455]", 2024-04-07T17:51:23.510 INFO:tasks.workunit.client.0.smithi100.stdout: "[268435456, 536870911]", 2024-04-07T17:51:23.510 INFO:tasks.workunit.client.0.smithi100.stdout: "[536870912, 1073741823]", 2024-04-07T17:51:23.510 INFO:tasks.workunit.client.0.smithi100.stdout: "[1073741824, 2147483647]", 2024-04-07T17:51:23.510 INFO:tasks.workunit.client.0.smithi100.stdout: "[2147483648, 4294967295]", 2024-04-07T17:51:23.511 INFO:tasks.workunit.client.0.smithi100.stdout: "[4294967296, 8589934591]", 2024-04-07T17:51:23.511 INFO:tasks.workunit.client.0.smithi100.stdout: "[8589934592, 17179869183]", 2024-04-07T17:51:23.511 INFO:tasks.workunit.client.0.smithi100.stdout: "[17179869184, 34359738367]", 2024-04-07T17:51:23.511 INFO:tasks.workunit.client.0.smithi100.stdout: "[34359738368, 68719476735]", 2024-04-07T17:51:23.511 INFO:tasks.workunit.client.0.smithi100.stdout: "[68719476736, 137438953471]", 2024-04-07T17:51:23.511 INFO:tasks.workunit.client.0.smithi100.stdout: "[137438953472, 274877906943]", 2024-04-07T17:51:23.511 INFO:tasks.workunit.client.0.smithi100.stdout: "[274877906944, None]" 2024-04-07T17:51:23.511 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:23.511 INFO:tasks.workunit.client.0.smithi100.stdout: "scale_type": "log2" 2024-04-07T17:51:23.511 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:23.511 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:23.511 INFO:tasks.workunit.client.0.smithi100.stdout: "osds": [ 2024-04-07T17:51:23.511 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:23.511 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 0, 2024-04-07T17:51:23.512 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:23.512 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.512 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.512 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 997, 13, 0, 0, 648, 16, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.512 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 7211, 3165, 0, 1, 4231, 80, 0, 4, 1, 0, 3, 2, 4, 0, 12, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.512 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 11863, 1050, 12, 2, 2904, 503, 11, 11, 26, 39, 32, 36, 59, 21, 77, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.512 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1610, 30, 63, 0, 19, 15, 2, 5, 5, 13, 105, 163, 32, 5, 39, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.512 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 417, 0, 5, 0, 3, 9, 0, 1, 0, 1, 3, 37, 361, 22, 36, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.512 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 141, 1, 0, 0, 5, 11, 0, 0, 0, 1, 2, 13, 25, 31, 205, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.512 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 120, 1, 0, 0, 0, 0, 1, 2, 2, 0, 0, 7, 21, 15, 56, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.512 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 192, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 4, 14, 26, 33, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.512 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 32, 0, 0, 0, 0, 0, 0, 0, 0, 0, 2, 0, 4, 19, 20, 5, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.512 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 6, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 4, 3, 9, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.512 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.512 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.512 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.513 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.513 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.513 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.513 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.513 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.513 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.513 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.513 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.513 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.513 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.513 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.513 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.513 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.513 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.513 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.514 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.514 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:23.514 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:23.514 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.514 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:23.514 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 1, 2024-04-07T17:51:23.514 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:23.514 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.514 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.514 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1446, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.514 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 5787, 4, 145, 0, 8, 0, 0, 0, 1, 1, 2, 1, 2, 0, 9, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.514 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 9598, 33, 88, 8, 28, 51, 80, 25, 39, 53, 38, 35, 64, 23, 44, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.514 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1252, 11, 1178, 4, 3, 3, 38, 118, 6, 17, 129, 228, 29, 7, 16, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.514 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 284, 1, 144, 24, 0, 0, 0, 14, 2, 1, 4, 45, 487, 18, 5, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.514 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 188, 0, 9, 6, 0, 0, 0, 0, 0, 1, 8, 2, 4, 12, 161, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.515 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 174, 0, 76, 0, 0, 0, 0, 0, 0, 0, 9, 0, 4, 5, 36, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.515 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 95, 0, 62, 0, 0, 0, 0, 0, 0, 0, 3, 0, 3, 11, 17, 6, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.515 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 7, 0, 153, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 5, 2, 10, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.515 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 3, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.515 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.515 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.515 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.515 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.515 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.515 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.515 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.515 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.515 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.516 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.516 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.516 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.516 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.516 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.516 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.516 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.516 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.516 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.516 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.516 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:23.516 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:23.516 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.516 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:23.516 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 2, 2024-04-07T17:51:23.517 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:23.517 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.517 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.517 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 2187, 6, 0, 0, 0, 0, 0, 0, 0, 0, 2, 0, 0, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.517 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 6047, 2629, 2809, 0, 8, 0, 0, 1, 2, 0, 23, 0, 2, 7, 5, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.517 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 9066, 1269, 1692, 0, 3, 1, 0, 2, 6, 6, 32, 2, 20, 30, 68, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.517 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1309, 32, 27, 1, 2, 0, 0, 1, 9, 9, 29, 7, 17, 16, 33, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.517 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 335, 7, 6, 1, 0, 0, 0, 0, 0, 0, 3, 7, 47, 45, 17, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.517 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 152, 2, 0, 0, 0, 0, 0, 3, 0, 1, 3, 25, 11, 60, 181, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.517 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 149, 1, 0, 0, 0, 0, 0, 1, 3, 0, 3, 18, 37, 45, 52, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.517 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 208, 0, 0, 1, 0, 1, 2, 0, 0, 0, 4, 20, 43, 66, 44, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.517 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 52, 0, 0, 0, 0, 0, 0, 3, 0, 0, 0, 2, 3, 18, 51, 7, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.517 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 10, 0, 0, 0, 1, 0, 0, 1, 0, 0, 0, 0, 0, 8, 3, 4, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.517 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.517 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.517 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.518 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.518 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.518 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.518 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.518 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.518 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.518 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.518 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.518 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.518 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.518 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.518 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.518 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.518 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.519 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.519 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.519 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:23.519 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:23.519 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.519 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:23.519 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 3, 2024-04-07T17:51:23.519 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:23.519 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.519 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.519 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 3687, 36, 0, 1, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.519 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 12530, 2576, 0, 0, 13, 0, 0, 0, 0, 0, 1, 1, 1, 0, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.519 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 13127, 1790, 0, 0, 8, 6, 2, 8, 12, 16, 12, 7, 12, 5, 27, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.519 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1412, 42, 24, 0, 26, 0, 0, 2, 3, 5, 41, 74, 13, 6, 10, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.519 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 416, 3, 13, 0, 157, 0, 0, 0, 0, 0, 2, 12, 225, 42, 4, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.519 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 125, 1, 0, 0, 306, 1, 0, 0, 0, 0, 0, 0, 6, 21, 84, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.520 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 27, 0, 2, 2, 334, 0, 0, 0, 0, 0, 0, 2, 6, 7, 22, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.520 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 10, 0, 0, 0, 1487, 0, 0, 0, 0, 0, 0, 0, 4, 9, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.520 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 8, 0, 0, 0, 374, 0, 0, 0, 0, 0, 0, 0, 0, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.520 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.520 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.520 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.520 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.520 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.520 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.520 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.520 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.520 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.520 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.520 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.521 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.521 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.521 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.521 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.521 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.521 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.521 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.521 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.521 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.521 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:23.521 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:23.521 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.521 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:23.521 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 4, 2024-04-07T17:51:23.521 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:23.522 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.522 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.522 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 609, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.522 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 3885, 8, 15, 0, 0, 0, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.522 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 10736, 14, 53, 5, 22, 32, 30, 14, 13, 15, 11, 13, 30, 4, 36, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.522 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1540, 27, 25, 0, 9, 3, 41, 70, 1, 5, 58, 81, 10, 3, 28, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.522 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 375, 10, 14, 0, 0, 0, 0, 0, 0, 0, 1, 16, 198, 13, 3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.522 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 187, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 3, 9, 9, 56, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.522 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 61, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 5, 13, 3, 46, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.522 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 34, 0, 0, 0, 0, 0, 0, 0, 0, 2, 2, 3, 6, 3, 18, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.522 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 8, 0, 0, 0, 0, 0, 0, 1, 0, 0, 1, 0, 0, 0, 2, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.522 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.522 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.522 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.523 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.523 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.523 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.523 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.523 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.523 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.523 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.523 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.523 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.523 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.523 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.523 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.523 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.523 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.523 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.523 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.524 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.524 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:23.524 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:23.524 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.524 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:23.524 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 5, 2024-04-07T17:51:23.524 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:23.524 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.524 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.524 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 946, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.524 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 5643, 1, 0, 1, 0, 0, 1, 0, 0, 1, 3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.524 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 14942, 20, 3, 0, 21, 3, 0, 4, 1, 9, 44, 5, 10, 29, 54, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.524 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 2337, 103, 114, 1, 1, 1, 0, 2, 3, 4, 28, 12, 3, 5, 47, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.524 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 783, 40, 72, 4, 6, 0, 0, 0, 0, 0, 2, 0, 27, 26, 14, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.524 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 237, 9, 6, 4, 0, 0, 0, 2, 1, 0, 0, 3, 4, 39, 200, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.525 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 158, 10, 24, 5, 1, 0, 0, 1, 0, 0, 0, 2, 1, 6, 66, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.525 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 156, 1, 10, 45, 24, 0, 0, 0, 0, 0, 0, 0, 4, 6, 14, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.525 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 10, 0, 0, 16, 104, 86, 0, 0, 0, 0, 0, 0, 3, 6, 14, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.525 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 3, 0, 0, 0, 6, 43, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.525 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.525 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.525 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.525 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.525 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.525 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.525 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.525 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.525 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.525 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.526 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.526 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.526 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.526 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.526 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.526 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.526 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.526 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.526 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.526 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:23.526 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:23.526 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.526 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:23.526 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 6, 2024-04-07T17:51:23.527 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:23.527 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.527 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.527 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 830, 5, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.527 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 6114, 1983, 0, 0, 2, 1, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.527 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 10059, 2215, 1, 1, 22, 3, 1, 0, 18, 18, 10, 10, 26, 17, 43, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.527 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1593, 28, 21, 0, 2, 1, 0, 0, 5, 13, 58, 83, 12, 5, 27, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.527 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 625, 3, 7, 0, 0, 0, 0, 0, 0, 1, 7, 26, 211, 19, 7, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.527 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 531, 11, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 6, 22, 90, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.527 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 196, 8, 0, 0, 0, 1, 0, 0, 0, 1, 0, 0, 2, 1, 53, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.527 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 155, 8, 0, 0, 0, 0, 0, 1, 0, 3, 0, 0, 0, 1, 6, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.527 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 8, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.527 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.527 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.528 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.528 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.528 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.528 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.528 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.528 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.528 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.528 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.528 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.528 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.528 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.528 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.528 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.528 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.528 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.529 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.529 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.529 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.529 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:23.529 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:23.529 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.529 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:23.529 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 7, 2024-04-07T17:51:23.529 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:23.529 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.529 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.529 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 385, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.529 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 6666, 2090, 1, 0, 0, 0, 0, 0, 0, 0, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.529 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 13763, 2103, 0, 0, 7, 2, 0, 1, 0, 1, 1, 13, 10, 17, 55, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.529 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1294, 17, 4, 1, 0, 0, 0, 1, 0, 2, 17, 6, 3, 7, 31, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.530 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 308, 17, 0, 0, 0, 0, 0, 0, 0, 0, 0, 4, 20, 40, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.530 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 152, 63, 8, 0, 0, 0, 0, 2, 0, 0, 3, 1, 6, 32, 113, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.530 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 98, 138, 5, 0, 0, 0, 0, 1, 0, 0, 0, 2, 2, 7, 52, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.530 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 223, 58, 1, 0, 0, 0, 0, 0, 0, 0, 3, 0, 0, 8, 11, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.530 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 49, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 6, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.530 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.530 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.530 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.530 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.530 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.530 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.530 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.530 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.530 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.531 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.531 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.531 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.531 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.531 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.531 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.531 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.531 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.531 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.531 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.531 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.531 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:23.531 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:23.531 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:23.531 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:23.531 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.532 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_latency_in_bytes_histogram": { 2024-04-07T17:51:23.532 INFO:tasks.workunit.client.0.smithi100.stdout: "axes": [ 2024-04-07T17:51:23.532 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:23.532 INFO:tasks.workunit.client.0.smithi100.stdout: "buckets": 32, 2024-04-07T17:51:23.532 INFO:tasks.workunit.client.0.smithi100.stdout: "min": 0, 2024-04-07T17:51:23.532 INFO:tasks.workunit.client.0.smithi100.stdout: "name": "Latency (usec)", 2024-04-07T17:51:23.532 INFO:tasks.workunit.client.0.smithi100.stdout: "quant_size": 100000, 2024-04-07T17:51:23.532 INFO:tasks.workunit.client.0.smithi100.stdout: "ranges": [ 2024-04-07T17:51:23.532 INFO:tasks.workunit.client.0.smithi100.stdout: "[None, -1]", 2024-04-07T17:51:23.532 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 99999]", 2024-04-07T17:51:23.532 INFO:tasks.workunit.client.0.smithi100.stdout: "[100000, 199999]", 2024-04-07T17:51:23.532 INFO:tasks.workunit.client.0.smithi100.stdout: "[200000, 399999]", 2024-04-07T17:51:23.532 INFO:tasks.workunit.client.0.smithi100.stdout: "[400000, 799999]", 2024-04-07T17:51:23.532 INFO:tasks.workunit.client.0.smithi100.stdout: "[800000, 1599999]", 2024-04-07T17:51:23.532 INFO:tasks.workunit.client.0.smithi100.stdout: "[1600000, 3199999]", 2024-04-07T17:51:23.533 INFO:tasks.workunit.client.0.smithi100.stdout: "[3200000, 6399999]", 2024-04-07T17:51:23.533 INFO:tasks.workunit.client.0.smithi100.stdout: "[6400000, 12799999]", 2024-04-07T17:51:23.533 INFO:tasks.workunit.client.0.smithi100.stdout: "[12800000, 25599999]", 2024-04-07T17:51:23.533 INFO:tasks.workunit.client.0.smithi100.stdout: "[25600000, 51199999]", 2024-04-07T17:51:23.533 INFO:tasks.workunit.client.0.smithi100.stdout: "[51200000, 102399999]", 2024-04-07T17:51:23.533 INFO:tasks.workunit.client.0.smithi100.stdout: "[102400000, 204799999]", 2024-04-07T17:51:23.533 INFO:tasks.workunit.client.0.smithi100.stdout: "[204800000, 409599999]", 2024-04-07T17:51:23.533 INFO:tasks.workunit.client.0.smithi100.stdout: "[409600000, 819199999]", 2024-04-07T17:51:23.533 INFO:tasks.workunit.client.0.smithi100.stdout: "[819200000, 1638399999]", 2024-04-07T17:51:23.533 INFO:tasks.workunit.client.0.smithi100.stdout: "[1638400000, 3276799999]", 2024-04-07T17:51:23.533 INFO:tasks.workunit.client.0.smithi100.stdout: "[3276800000, 6553599999]", 2024-04-07T17:51:23.533 INFO:tasks.workunit.client.0.smithi100.stdout: "[6553600000, 13107199999]", 2024-04-07T17:51:23.533 INFO:tasks.workunit.client.0.smithi100.stdout: "[13107200000, 26214399999]", 2024-04-07T17:51:23.533 INFO:tasks.workunit.client.0.smithi100.stdout: "[26214400000, 52428799999]", 2024-04-07T17:51:23.533 INFO:tasks.workunit.client.0.smithi100.stdout: "[52428800000, 104857599999]", 2024-04-07T17:51:23.533 INFO:tasks.workunit.client.0.smithi100.stdout: "[104857600000, 209715199999]", 2024-04-07T17:51:23.534 INFO:tasks.workunit.client.0.smithi100.stdout: "[209715200000, 419430399999]", 2024-04-07T17:51:23.534 INFO:tasks.workunit.client.0.smithi100.stdout: "[419430400000, 838860799999]", 2024-04-07T17:51:23.534 INFO:tasks.workunit.client.0.smithi100.stdout: "[838860800000, 1677721599999]", 2024-04-07T17:51:23.534 INFO:tasks.workunit.client.0.smithi100.stdout: "[1677721600000, 3355443199999]", 2024-04-07T17:51:23.534 INFO:tasks.workunit.client.0.smithi100.stdout: "[3355443200000, 6710886399999]", 2024-04-07T17:51:23.534 INFO:tasks.workunit.client.0.smithi100.stdout: "[6710886400000, 13421772799999]", 2024-04-07T17:51:23.534 INFO:tasks.workunit.client.0.smithi100.stdout: "[13421772800000, 26843545599999]", 2024-04-07T17:51:23.534 INFO:tasks.workunit.client.0.smithi100.stdout: "[26843545600000, 53687091199999]", 2024-04-07T17:51:23.534 INFO:tasks.workunit.client.0.smithi100.stdout: "[53687091200000, None]" 2024-04-07T17:51:23.534 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:23.534 INFO:tasks.workunit.client.0.smithi100.stdout: "scale_type": "log2" 2024-04-07T17:51:23.534 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.534 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:23.534 INFO:tasks.workunit.client.0.smithi100.stdout: "buckets": 32, 2024-04-07T17:51:23.534 INFO:tasks.workunit.client.0.smithi100.stdout: "min": 0, 2024-04-07T17:51:23.535 INFO:tasks.workunit.client.0.smithi100.stdout: "name": "Request size (bytes)", 2024-04-07T17:51:23.535 INFO:tasks.workunit.client.0.smithi100.stdout: "quant_size": 512, 2024-04-07T17:51:23.535 INFO:tasks.workunit.client.0.smithi100.stdout: "ranges": [ 2024-04-07T17:51:23.535 INFO:tasks.workunit.client.0.smithi100.stdout: "[None, -1]", 2024-04-07T17:51:23.535 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 511]", 2024-04-07T17:51:23.535 INFO:tasks.workunit.client.0.smithi100.stdout: "[512, 1023]", 2024-04-07T17:51:23.535 INFO:tasks.workunit.client.0.smithi100.stdout: "[1024, 2047]", 2024-04-07T17:51:23.535 INFO:tasks.workunit.client.0.smithi100.stdout: "[2048, 4095]", 2024-04-07T17:51:23.535 INFO:tasks.workunit.client.0.smithi100.stdout: "[4096, 8191]", 2024-04-07T17:51:23.535 INFO:tasks.workunit.client.0.smithi100.stdout: "[8192, 16383]", 2024-04-07T17:51:23.535 INFO:tasks.workunit.client.0.smithi100.stdout: "[16384, 32767]", 2024-04-07T17:51:23.535 INFO:tasks.workunit.client.0.smithi100.stdout: "[32768, 65535]", 2024-04-07T17:51:23.535 INFO:tasks.workunit.client.0.smithi100.stdout: "[65536, 131071]", 2024-04-07T17:51:23.535 INFO:tasks.workunit.client.0.smithi100.stdout: "[131072, 262143]", 2024-04-07T17:51:23.535 INFO:tasks.workunit.client.0.smithi100.stdout: "[262144, 524287]", 2024-04-07T17:51:23.535 INFO:tasks.workunit.client.0.smithi100.stdout: "[524288, 1048575]", 2024-04-07T17:51:23.536 INFO:tasks.workunit.client.0.smithi100.stdout: "[1048576, 2097151]", 2024-04-07T17:51:23.536 INFO:tasks.workunit.client.0.smithi100.stdout: "[2097152, 4194303]", 2024-04-07T17:51:23.536 INFO:tasks.workunit.client.0.smithi100.stdout: "[4194304, 8388607]", 2024-04-07T17:51:23.536 INFO:tasks.workunit.client.0.smithi100.stdout: "[8388608, 16777215]", 2024-04-07T17:51:23.536 INFO:tasks.workunit.client.0.smithi100.stdout: "[16777216, 33554431]", 2024-04-07T17:51:23.536 INFO:tasks.workunit.client.0.smithi100.stdout: "[33554432, 67108863]", 2024-04-07T17:51:23.536 INFO:tasks.workunit.client.0.smithi100.stdout: "[67108864, 134217727]", 2024-04-07T17:51:23.536 INFO:tasks.workunit.client.0.smithi100.stdout: "[134217728, 268435455]", 2024-04-07T17:51:23.536 INFO:tasks.workunit.client.0.smithi100.stdout: "[268435456, 536870911]", 2024-04-07T17:51:23.536 INFO:tasks.workunit.client.0.smithi100.stdout: "[536870912, 1073741823]", 2024-04-07T17:51:23.536 INFO:tasks.workunit.client.0.smithi100.stdout: "[1073741824, 2147483647]", 2024-04-07T17:51:23.536 INFO:tasks.workunit.client.0.smithi100.stdout: "[2147483648, 4294967295]", 2024-04-07T17:51:23.536 INFO:tasks.workunit.client.0.smithi100.stdout: "[4294967296, 8589934591]", 2024-04-07T17:51:23.536 INFO:tasks.workunit.client.0.smithi100.stdout: "[8589934592, 17179869183]", 2024-04-07T17:51:23.536 INFO:tasks.workunit.client.0.smithi100.stdout: "[17179869184, 34359738367]", 2024-04-07T17:51:23.537 INFO:tasks.workunit.client.0.smithi100.stdout: "[34359738368, 68719476735]", 2024-04-07T17:51:23.537 INFO:tasks.workunit.client.0.smithi100.stdout: "[68719476736, 137438953471]", 2024-04-07T17:51:23.537 INFO:tasks.workunit.client.0.smithi100.stdout: "[137438953472, 274877906943]", 2024-04-07T17:51:23.537 INFO:tasks.workunit.client.0.smithi100.stdout: "[274877906944, None]" 2024-04-07T17:51:23.537 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:23.537 INFO:tasks.workunit.client.0.smithi100.stdout: "scale_type": "log2" 2024-04-07T17:51:23.537 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:23.537 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:23.537 INFO:tasks.workunit.client.0.smithi100.stdout: "osds": [ 2024-04-07T17:51:23.537 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:23.537 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 0, 2024-04-07T17:51:23.537 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:23.537 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.537 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.537 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.538 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.538 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.538 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 416, 12, 5, 6, 1, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.538 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 4275, 54, 74, 123, 377, 516, 419, 4, 11, 5, 0, 0, 3, 3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.538 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 775, 16, 26, 72, 112, 96, 307, 0, 3, 12, 2, 3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.538 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 369, 1, 0, 4, 2, 1, 2, 0, 1, 1, 0, 3, 13, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.538 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 201, 1, 0, 0, 0, 0, 1, 0, 0, 1, 0, 1, 10, 14, 29, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.538 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 88, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 1, 1, 7, 41, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.538 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 6, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 25, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.538 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.538 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.538 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.538 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.539 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.539 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.539 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.539 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.539 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.539 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.539 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.539 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.539 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.539 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.539 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.539 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.539 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.539 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.539 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.540 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:23.540 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:23.540 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.540 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:23.540 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 1, 2024-04-07T17:51:23.540 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:23.540 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.540 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.540 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.540 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.540 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.540 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 463, 210, 9, 10, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.540 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 2623, 76, 25, 59, 122, 1, 2, 2, 0, 0, 0, 1, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.540 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 819, 8, 815, 0, 1, 0, 1, 0, 2, 1, 0, 1, 0, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.540 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 994, 507, 544, 0, 0, 0, 0, 0, 0, 0, 1, 1, 5, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.541 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 844, 647, 73, 0, 0, 0, 0, 1, 0, 0, 1, 0, 5, 7, 15, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.541 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1618, 1450, 103, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 4, 34, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.541 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 291, 394, 108, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 12, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.541 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.541 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.541 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.541 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.541 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.541 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.541 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.541 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.541 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.541 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.541 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.541 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.542 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.542 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.542 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.542 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.542 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.542 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.542 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.542 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:23.542 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:23.542 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.542 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:23.542 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 2, 2024-04-07T17:51:23.542 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:23.542 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.543 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.543 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.543 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.543 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.543 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 248, 5, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.543 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 7869, 12, 0, 0, 0, 2, 5, 3, 9, 2, 0, 2, 3, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.543 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 17697, 0, 1, 0, 1, 0, 0, 1, 2, 11, 0, 2, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.543 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 887, 0, 1, 0, 0, 0, 0, 0, 1, 0, 0, 2, 9, 3, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.543 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 267, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 7, 21, 28, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.543 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 103, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 6, 6, 37, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.543 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 2, 23, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.543 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.543 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.543 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.543 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.544 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.544 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.544 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.544 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.544 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.544 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.544 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.544 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.544 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.544 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.544 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.544 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.544 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.544 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.545 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.545 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:23.545 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:23.545 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.545 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:23.545 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 3, 2024-04-07T17:51:23.545 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:23.545 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.545 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.545 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.545 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.545 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.545 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 280, 1, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.545 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 3748, 33, 1, 0, 5, 1, 5, 4, 1, 0, 0, 0, 3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.545 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 519, 29, 31, 40, 1, 0, 1, 0, 0, 0, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.545 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 290, 227, 3, 37, 0, 0, 0, 0, 0, 0, 0, 2, 3, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.546 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 2130, 1325, 1, 172, 0, 0, 0, 0, 0, 0, 0, 0, 6, 6, 16, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.546 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 2779, 1327, 0, 285, 0, 0, 0, 0, 0, 0, 0, 1, 3, 4, 24, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.546 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 233, 115, 0, 51, 0, 0, 0, 0, 0, 0, 0, 0, 0, 2, 15, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.546 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.546 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.546 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.546 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.546 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.546 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.546 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.546 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.546 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.546 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.547 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.547 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.547 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.547 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.547 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.547 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.547 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.547 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.547 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.547 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:23.547 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:23.547 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.547 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:23.548 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 4, 2024-04-07T17:51:23.548 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:23.548 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.548 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.548 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.548 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.548 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.548 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 113, 11, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.548 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 3576, 127, 33, 67, 137, 233, 86, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.548 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 303, 84, 0, 0, 1, 40, 395, 0, 1, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.548 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 124, 2, 0, 0, 1, 2, 1, 0, 0, 0, 1, 2, 7, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.548 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 73, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 4, 4, 6, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.548 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 25, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 29, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.548 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 16, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.549 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.549 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.549 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.549 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.549 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.549 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.549 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.549 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.549 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.549 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.549 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.549 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.549 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.549 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.549 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.550 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.550 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.550 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.550 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.550 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:23.550 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:23.550 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.550 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:23.550 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 5, 2024-04-07T17:51:23.550 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:23.550 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.550 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.550 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.550 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.550 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.551 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 109, 21, 86, 200, 454, 576, 651, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.551 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 5257, 50, 38, 54, 17, 36, 74, 1, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.551 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 689, 12, 17, 5, 0, 1, 0, 1, 0, 1, 2, 3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.551 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 457, 32, 66, 127, 29, 1, 0, 2, 0, 0, 2, 4, 8, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.551 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 302, 8, 16, 1, 0, 0, 0, 1, 0, 2, 2, 1, 9, 18, 14, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.551 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 96, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 4, 8, 35, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.551 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 18, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.551 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.551 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.551 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.551 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.551 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.551 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.551 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.552 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.552 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.552 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.552 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.552 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.552 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.552 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.552 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.552 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.552 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.552 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.552 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.552 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:23.553 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:23.553 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.553 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:23.553 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 6, 2024-04-07T17:51:23.553 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:23.553 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.553 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.553 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.553 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.553 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.553 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 51, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.553 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 2959, 10, 0, 0, 1, 1, 0, 1, 0, 0, 0, 1, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.553 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 347, 9, 0, 0, 1, 1, 0, 0, 0, 1, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.553 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 267, 64, 0, 0, 0, 0, 1, 0, 1, 2, 0, 3, 7, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.553 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 150, 40, 0, 0, 0, 0, 0, 0, 0, 0, 1, 2, 3, 4, 6, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.554 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 58, 5, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 6, 27, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.554 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 6, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 2, 20, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.554 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.554 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.554 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.554 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.554 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.554 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.554 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.554 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.554 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.554 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.554 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.554 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.554 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.555 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.555 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.555 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.555 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.555 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.555 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.555 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:23.555 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:23.555 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.555 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:23.555 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 7, 2024-04-07T17:51:23.555 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:23.555 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.555 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.555 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.556 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.556 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.556 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 267, 14, 33, 41, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.556 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 4195, 22, 41, 115, 0, 1, 1, 1, 2, 0, 0, 0, 2, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.556 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 6037, 25, 32, 5, 0, 1, 0, 1, 2, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.556 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 292, 5, 2, 0, 0, 0, 0, 0, 0, 1, 0, 2, 4, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.556 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 85, 1, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 2, 13, 6, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.556 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 43, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 2, 5, 28, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.556 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 16, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.556 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.556 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.556 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.556 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.556 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.556 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.557 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.557 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.557 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.557 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.557 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.557 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.557 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.557 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.557 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.557 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.557 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.557 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.557 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.557 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:23.558 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:23.558 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:23.558 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:23.558 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.558 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_latency_out_bytes_histogram": { 2024-04-07T17:51:23.558 INFO:tasks.workunit.client.0.smithi100.stdout: "axes": [ 2024-04-07T17:51:23.558 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:23.558 INFO:tasks.workunit.client.0.smithi100.stdout: "buckets": 32, 2024-04-07T17:51:23.558 INFO:tasks.workunit.client.0.smithi100.stdout: "min": 0, 2024-04-07T17:51:23.558 INFO:tasks.workunit.client.0.smithi100.stdout: "name": "Latency (usec)", 2024-04-07T17:51:23.558 INFO:tasks.workunit.client.0.smithi100.stdout: "quant_size": 100000, 2024-04-07T17:51:23.558 INFO:tasks.workunit.client.0.smithi100.stdout: "ranges": [ 2024-04-07T17:51:23.558 INFO:tasks.workunit.client.0.smithi100.stdout: "[None, -1]", 2024-04-07T17:51:23.558 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 99999]", 2024-04-07T17:51:23.559 INFO:tasks.workunit.client.0.smithi100.stdout: "[100000, 199999]", 2024-04-07T17:51:23.559 INFO:tasks.workunit.client.0.smithi100.stdout: "[200000, 399999]", 2024-04-07T17:51:23.559 INFO:tasks.workunit.client.0.smithi100.stdout: "[400000, 799999]", 2024-04-07T17:51:23.559 INFO:tasks.workunit.client.0.smithi100.stdout: "[800000, 1599999]", 2024-04-07T17:51:23.559 INFO:tasks.workunit.client.0.smithi100.stdout: "[1600000, 3199999]", 2024-04-07T17:51:23.559 INFO:tasks.workunit.client.0.smithi100.stdout: "[3200000, 6399999]", 2024-04-07T17:51:23.559 INFO:tasks.workunit.client.0.smithi100.stdout: "[6400000, 12799999]", 2024-04-07T17:51:23.559 INFO:tasks.workunit.client.0.smithi100.stdout: "[12800000, 25599999]", 2024-04-07T17:51:23.559 INFO:tasks.workunit.client.0.smithi100.stdout: "[25600000, 51199999]", 2024-04-07T17:51:23.559 INFO:tasks.workunit.client.0.smithi100.stdout: "[51200000, 102399999]", 2024-04-07T17:51:23.559 INFO:tasks.workunit.client.0.smithi100.stdout: "[102400000, 204799999]", 2024-04-07T17:51:23.559 INFO:tasks.workunit.client.0.smithi100.stdout: "[204800000, 409599999]", 2024-04-07T17:51:23.559 INFO:tasks.workunit.client.0.smithi100.stdout: "[409600000, 819199999]", 2024-04-07T17:51:23.559 INFO:tasks.workunit.client.0.smithi100.stdout: "[819200000, 1638399999]", 2024-04-07T17:51:23.559 INFO:tasks.workunit.client.0.smithi100.stdout: "[1638400000, 3276799999]", 2024-04-07T17:51:23.560 INFO:tasks.workunit.client.0.smithi100.stdout: "[3276800000, 6553599999]", 2024-04-07T17:51:23.560 INFO:tasks.workunit.client.0.smithi100.stdout: "[6553600000, 13107199999]", 2024-04-07T17:51:23.560 INFO:tasks.workunit.client.0.smithi100.stdout: "[13107200000, 26214399999]", 2024-04-07T17:51:23.560 INFO:tasks.workunit.client.0.smithi100.stdout: "[26214400000, 52428799999]", 2024-04-07T17:51:23.560 INFO:tasks.workunit.client.0.smithi100.stdout: "[52428800000, 104857599999]", 2024-04-07T17:51:23.560 INFO:tasks.workunit.client.0.smithi100.stdout: "[104857600000, 209715199999]", 2024-04-07T17:51:23.560 INFO:tasks.workunit.client.0.smithi100.stdout: "[209715200000, 419430399999]", 2024-04-07T17:51:23.560 INFO:tasks.workunit.client.0.smithi100.stdout: "[419430400000, 838860799999]", 2024-04-07T17:51:23.560 INFO:tasks.workunit.client.0.smithi100.stdout: "[838860800000, 1677721599999]", 2024-04-07T17:51:23.560 INFO:tasks.workunit.client.0.smithi100.stdout: "[1677721600000, 3355443199999]", 2024-04-07T17:51:23.560 INFO:tasks.workunit.client.0.smithi100.stdout: "[3355443200000, 6710886399999]", 2024-04-07T17:51:23.560 INFO:tasks.workunit.client.0.smithi100.stdout: "[6710886400000, 13421772799999]", 2024-04-07T17:51:23.560 INFO:tasks.workunit.client.0.smithi100.stdout: "[13421772800000, 26843545599999]", 2024-04-07T17:51:23.560 INFO:tasks.workunit.client.0.smithi100.stdout: "[26843545600000, 53687091199999]", 2024-04-07T17:51:23.560 INFO:tasks.workunit.client.0.smithi100.stdout: "[53687091200000, None]" 2024-04-07T17:51:23.560 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:23.561 INFO:tasks.workunit.client.0.smithi100.stdout: "scale_type": "log2" 2024-04-07T17:51:23.561 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.561 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:23.561 INFO:tasks.workunit.client.0.smithi100.stdout: "buckets": 32, 2024-04-07T17:51:23.561 INFO:tasks.workunit.client.0.smithi100.stdout: "min": 0, 2024-04-07T17:51:23.561 INFO:tasks.workunit.client.0.smithi100.stdout: "name": "Request size (bytes)", 2024-04-07T17:51:23.561 INFO:tasks.workunit.client.0.smithi100.stdout: "quant_size": 512, 2024-04-07T17:51:23.561 INFO:tasks.workunit.client.0.smithi100.stdout: "ranges": [ 2024-04-07T17:51:23.561 INFO:tasks.workunit.client.0.smithi100.stdout: "[None, -1]", 2024-04-07T17:51:23.561 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 511]", 2024-04-07T17:51:23.561 INFO:tasks.workunit.client.0.smithi100.stdout: "[512, 1023]", 2024-04-07T17:51:23.561 INFO:tasks.workunit.client.0.smithi100.stdout: "[1024, 2047]", 2024-04-07T17:51:23.561 INFO:tasks.workunit.client.0.smithi100.stdout: "[2048, 4095]", 2024-04-07T17:51:23.561 INFO:tasks.workunit.client.0.smithi100.stdout: "[4096, 8191]", 2024-04-07T17:51:23.561 INFO:tasks.workunit.client.0.smithi100.stdout: "[8192, 16383]", 2024-04-07T17:51:23.562 INFO:tasks.workunit.client.0.smithi100.stdout: "[16384, 32767]", 2024-04-07T17:51:23.562 INFO:tasks.workunit.client.0.smithi100.stdout: "[32768, 65535]", 2024-04-07T17:51:23.562 INFO:tasks.workunit.client.0.smithi100.stdout: "[65536, 131071]", 2024-04-07T17:51:23.562 INFO:tasks.workunit.client.0.smithi100.stdout: "[131072, 262143]", 2024-04-07T17:51:23.562 INFO:tasks.workunit.client.0.smithi100.stdout: "[262144, 524287]", 2024-04-07T17:51:23.562 INFO:tasks.workunit.client.0.smithi100.stdout: "[524288, 1048575]", 2024-04-07T17:51:23.562 INFO:tasks.workunit.client.0.smithi100.stdout: "[1048576, 2097151]", 2024-04-07T17:51:23.562 INFO:tasks.workunit.client.0.smithi100.stdout: "[2097152, 4194303]", 2024-04-07T17:51:23.562 INFO:tasks.workunit.client.0.smithi100.stdout: "[4194304, 8388607]", 2024-04-07T17:51:23.562 INFO:tasks.workunit.client.0.smithi100.stdout: "[8388608, 16777215]", 2024-04-07T17:51:23.562 INFO:tasks.workunit.client.0.smithi100.stdout: "[16777216, 33554431]", 2024-04-07T17:51:23.562 INFO:tasks.workunit.client.0.smithi100.stdout: "[33554432, 67108863]", 2024-04-07T17:51:23.562 INFO:tasks.workunit.client.0.smithi100.stdout: "[67108864, 134217727]", 2024-04-07T17:51:23.562 INFO:tasks.workunit.client.0.smithi100.stdout: "[134217728, 268435455]", 2024-04-07T17:51:23.563 INFO:tasks.workunit.client.0.smithi100.stdout: "[268435456, 536870911]", 2024-04-07T17:51:23.563 INFO:tasks.workunit.client.0.smithi100.stdout: "[536870912, 1073741823]", 2024-04-07T17:51:23.563 INFO:tasks.workunit.client.0.smithi100.stdout: "[1073741824, 2147483647]", 2024-04-07T17:51:23.563 INFO:tasks.workunit.client.0.smithi100.stdout: "[2147483648, 4294967295]", 2024-04-07T17:51:23.563 INFO:tasks.workunit.client.0.smithi100.stdout: "[4294967296, 8589934591]", 2024-04-07T17:51:23.563 INFO:tasks.workunit.client.0.smithi100.stdout: "[8589934592, 17179869183]", 2024-04-07T17:51:23.563 INFO:tasks.workunit.client.0.smithi100.stdout: "[17179869184, 34359738367]", 2024-04-07T17:51:23.563 INFO:tasks.workunit.client.0.smithi100.stdout: "[34359738368, 68719476735]", 2024-04-07T17:51:23.563 INFO:tasks.workunit.client.0.smithi100.stdout: "[68719476736, 137438953471]", 2024-04-07T17:51:23.563 INFO:tasks.workunit.client.0.smithi100.stdout: "[137438953472, 274877906943]", 2024-04-07T17:51:23.563 INFO:tasks.workunit.client.0.smithi100.stdout: "[274877906944, None]" 2024-04-07T17:51:23.563 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:23.563 INFO:tasks.workunit.client.0.smithi100.stdout: "scale_type": "log2" 2024-04-07T17:51:23.563 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:23.563 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:23.564 INFO:tasks.workunit.client.0.smithi100.stdout: "osds": [ 2024-04-07T17:51:23.564 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:23.564 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 0, 2024-04-07T17:51:23.564 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:23.564 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.564 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.564 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.564 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.564 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.564 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 441, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.564 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 5864, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.564 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1424, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.564 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 398, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.564 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 258, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.564 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 139, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.564 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 32, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.565 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.565 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.565 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.565 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.565 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.565 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.565 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.565 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.565 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.565 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.565 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.565 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.565 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.565 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.566 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.566 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.566 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.566 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.566 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.566 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:23.566 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:23.566 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.566 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:23.566 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 1, 2024-04-07T17:51:23.566 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:23.566 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.566 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.567 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.567 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.567 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.567 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 692, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.567 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 2912, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.567 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1650, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.567 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 2053, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.567 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1593, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.567 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 3210, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.567 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 807, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.567 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.567 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.567 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.567 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.567 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.568 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.568 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.568 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.568 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.568 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.568 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.568 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.568 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.568 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.568 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.568 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.568 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.568 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.569 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.569 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:23.569 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:23.569 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.569 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:23.569 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 2, 2024-04-07T17:51:23.569 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:23.569 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.569 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.569 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.569 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.569 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.569 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 253, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.569 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 7908, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.569 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 17717, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.570 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 904, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.570 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 325, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.570 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 152, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.570 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 28, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.570 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.570 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.570 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.570 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.570 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.570 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.570 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.570 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.570 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.570 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.571 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.571 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.571 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.571 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.571 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.571 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.571 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.571 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.571 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.571 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:23.571 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:23.571 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.571 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:23.571 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 3, 2024-04-07T17:51:23.572 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:23.572 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.572 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.572 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.572 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.572 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.572 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 282, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.572 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 3801, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.572 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 624, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.572 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 564, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.572 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 3656, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.572 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 4423, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.572 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 416, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.572 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.572 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.573 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.573 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.573 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.573 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.573 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.573 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.573 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.573 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.573 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.573 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.573 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.573 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.573 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.573 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.573 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.574 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.574 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.574 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:23.574 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:23.574 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.574 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:23.574 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 4, 2024-04-07T17:51:23.574 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:23.574 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.574 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.574 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.574 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.574 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.574 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 126, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.575 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 4261, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.575 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 825, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.575 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 140, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.575 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 88, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.575 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 55, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.575 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 16, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.575 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.575 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.575 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.575 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.575 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.575 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.575 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.575 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.575 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.576 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.576 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.576 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.576 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.576 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.576 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.576 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.576 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.576 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.576 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.576 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:23.576 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:23.576 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.576 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:23.577 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 5, 2024-04-07T17:51:23.577 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:23.577 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.577 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.577 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.577 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.577 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.577 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 2097, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.577 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 5529, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.577 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 731, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.577 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 730, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.577 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 374, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.577 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 143, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.577 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 20, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.577 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.578 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.578 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.578 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.578 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.578 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.578 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.578 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.578 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.578 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.578 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.578 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.578 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.578 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.578 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.578 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.579 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.579 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.579 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.579 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:23.579 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:23.579 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.579 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:23.579 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 6, 2024-04-07T17:51:23.579 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:23.579 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.579 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.579 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.579 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.579 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.579 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 52, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.580 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 2975, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.580 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 360, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.580 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 346, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.580 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 206, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.580 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 97, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.580 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 28, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.580 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.580 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.580 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.580 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.580 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.580 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.580 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.580 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.581 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.581 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.581 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.581 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.581 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.581 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.581 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.581 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.581 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.581 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.581 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.581 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:23.581 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:23.581 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.581 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:23.582 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 7, 2024-04-07T17:51:23.582 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:23.582 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.582 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.582 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.582 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.582 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.582 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 355, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.582 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 4382, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.582 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 6106, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.582 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 307, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.582 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 109, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.582 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 79, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.582 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 19, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.583 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.583 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.583 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.583 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.583 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.583 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.583 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.583 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.583 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.583 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.583 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.583 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.583 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.583 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.583 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.583 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.584 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.584 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.584 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.584 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:23.584 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:23.584 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:23.584 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:23.584 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.584 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_latency_in_bytes_histogram": { 2024-04-07T17:51:23.584 INFO:tasks.workunit.client.0.smithi100.stdout: "axes": [ 2024-04-07T17:51:23.584 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:23.584 INFO:tasks.workunit.client.0.smithi100.stdout: "buckets": 32, 2024-04-07T17:51:23.584 INFO:tasks.workunit.client.0.smithi100.stdout: "min": 0, 2024-04-07T17:51:23.584 INFO:tasks.workunit.client.0.smithi100.stdout: "name": "Latency (usec)", 2024-04-07T17:51:23.584 INFO:tasks.workunit.client.0.smithi100.stdout: "quant_size": 100000, 2024-04-07T17:51:23.584 INFO:tasks.workunit.client.0.smithi100.stdout: "ranges": [ 2024-04-07T17:51:23.585 INFO:tasks.workunit.client.0.smithi100.stdout: "[None, -1]", 2024-04-07T17:51:23.585 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 99999]", 2024-04-07T17:51:23.585 INFO:tasks.workunit.client.0.smithi100.stdout: "[100000, 199999]", 2024-04-07T17:51:23.585 INFO:tasks.workunit.client.0.smithi100.stdout: "[200000, 399999]", 2024-04-07T17:51:23.585 INFO:tasks.workunit.client.0.smithi100.stdout: "[400000, 799999]", 2024-04-07T17:51:23.585 INFO:tasks.workunit.client.0.smithi100.stdout: "[800000, 1599999]", 2024-04-07T17:51:23.585 INFO:tasks.workunit.client.0.smithi100.stdout: "[1600000, 3199999]", 2024-04-07T17:51:23.585 INFO:tasks.workunit.client.0.smithi100.stdout: "[3200000, 6399999]", 2024-04-07T17:51:23.585 INFO:tasks.workunit.client.0.smithi100.stdout: "[6400000, 12799999]", 2024-04-07T17:51:23.585 INFO:tasks.workunit.client.0.smithi100.stdout: "[12800000, 25599999]", 2024-04-07T17:51:23.585 INFO:tasks.workunit.client.0.smithi100.stdout: "[25600000, 51199999]", 2024-04-07T17:51:23.585 INFO:tasks.workunit.client.0.smithi100.stdout: "[51200000, 102399999]", 2024-04-07T17:51:23.585 INFO:tasks.workunit.client.0.smithi100.stdout: "[102400000, 204799999]", 2024-04-07T17:51:23.585 INFO:tasks.workunit.client.0.smithi100.stdout: "[204800000, 409599999]", 2024-04-07T17:51:23.585 INFO:tasks.workunit.client.0.smithi100.stdout: "[409600000, 819199999]", 2024-04-07T17:51:23.586 INFO:tasks.workunit.client.0.smithi100.stdout: "[819200000, 1638399999]", 2024-04-07T17:51:23.586 INFO:tasks.workunit.client.0.smithi100.stdout: "[1638400000, 3276799999]", 2024-04-07T17:51:23.586 INFO:tasks.workunit.client.0.smithi100.stdout: "[3276800000, 6553599999]", 2024-04-07T17:51:23.586 INFO:tasks.workunit.client.0.smithi100.stdout: "[6553600000, 13107199999]", 2024-04-07T17:51:23.586 INFO:tasks.workunit.client.0.smithi100.stdout: "[13107200000, 26214399999]", 2024-04-07T17:51:23.586 INFO:tasks.workunit.client.0.smithi100.stdout: "[26214400000, 52428799999]", 2024-04-07T17:51:23.586 INFO:tasks.workunit.client.0.smithi100.stdout: "[52428800000, 104857599999]", 2024-04-07T17:51:23.586 INFO:tasks.workunit.client.0.smithi100.stdout: "[104857600000, 209715199999]", 2024-04-07T17:51:23.586 INFO:tasks.workunit.client.0.smithi100.stdout: "[209715200000, 419430399999]", 2024-04-07T17:51:23.586 INFO:tasks.workunit.client.0.smithi100.stdout: "[419430400000, 838860799999]", 2024-04-07T17:51:23.586 INFO:tasks.workunit.client.0.smithi100.stdout: "[838860800000, 1677721599999]", 2024-04-07T17:51:23.586 INFO:tasks.workunit.client.0.smithi100.stdout: "[1677721600000, 3355443199999]", 2024-04-07T17:51:23.586 INFO:tasks.workunit.client.0.smithi100.stdout: "[3355443200000, 6710886399999]", 2024-04-07T17:51:23.586 INFO:tasks.workunit.client.0.smithi100.stdout: "[6710886400000, 13421772799999]", 2024-04-07T17:51:23.587 INFO:tasks.workunit.client.0.smithi100.stdout: "[13421772800000, 26843545599999]", 2024-04-07T17:51:23.587 INFO:tasks.workunit.client.0.smithi100.stdout: "[26843545600000, 53687091199999]", 2024-04-07T17:51:23.587 INFO:tasks.workunit.client.0.smithi100.stdout: "[53687091200000, None]" 2024-04-07T17:51:23.587 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:23.587 INFO:tasks.workunit.client.0.smithi100.stdout: "scale_type": "log2" 2024-04-07T17:51:23.587 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.587 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:23.587 INFO:tasks.workunit.client.0.smithi100.stdout: "buckets": 32, 2024-04-07T17:51:23.587 INFO:tasks.workunit.client.0.smithi100.stdout: "min": 0, 2024-04-07T17:51:23.587 INFO:tasks.workunit.client.0.smithi100.stdout: "name": "Request size (bytes)", 2024-04-07T17:51:23.587 INFO:tasks.workunit.client.0.smithi100.stdout: "quant_size": 512, 2024-04-07T17:51:23.588 INFO:tasks.workunit.client.0.smithi100.stdout: "ranges": [ 2024-04-07T17:51:23.588 INFO:tasks.workunit.client.0.smithi100.stdout: "[None, -1]", 2024-04-07T17:51:23.588 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 511]", 2024-04-07T17:51:23.588 INFO:tasks.workunit.client.0.smithi100.stdout: "[512, 1023]", 2024-04-07T17:51:23.588 INFO:tasks.workunit.client.0.smithi100.stdout: "[1024, 2047]", 2024-04-07T17:51:23.588 INFO:tasks.workunit.client.0.smithi100.stdout: "[2048, 4095]", 2024-04-07T17:51:23.588 INFO:tasks.workunit.client.0.smithi100.stdout: "[4096, 8191]", 2024-04-07T17:51:23.588 INFO:tasks.workunit.client.0.smithi100.stdout: "[8192, 16383]", 2024-04-07T17:51:23.588 INFO:tasks.workunit.client.0.smithi100.stdout: "[16384, 32767]", 2024-04-07T17:51:23.588 INFO:tasks.workunit.client.0.smithi100.stdout: "[32768, 65535]", 2024-04-07T17:51:23.588 INFO:tasks.workunit.client.0.smithi100.stdout: "[65536, 131071]", 2024-04-07T17:51:23.588 INFO:tasks.workunit.client.0.smithi100.stdout: "[131072, 262143]", 2024-04-07T17:51:23.588 INFO:tasks.workunit.client.0.smithi100.stdout: "[262144, 524287]", 2024-04-07T17:51:23.588 INFO:tasks.workunit.client.0.smithi100.stdout: "[524288, 1048575]", 2024-04-07T17:51:23.588 INFO:tasks.workunit.client.0.smithi100.stdout: "[1048576, 2097151]", 2024-04-07T17:51:23.588 INFO:tasks.workunit.client.0.smithi100.stdout: "[2097152, 4194303]", 2024-04-07T17:51:23.588 INFO:tasks.workunit.client.0.smithi100.stdout: "[4194304, 8388607]", 2024-04-07T17:51:23.589 INFO:tasks.workunit.client.0.smithi100.stdout: "[8388608, 16777215]", 2024-04-07T17:51:23.589 INFO:tasks.workunit.client.0.smithi100.stdout: "[16777216, 33554431]", 2024-04-07T17:51:23.589 INFO:tasks.workunit.client.0.smithi100.stdout: "[33554432, 67108863]", 2024-04-07T17:51:23.589 INFO:tasks.workunit.client.0.smithi100.stdout: "[67108864, 134217727]", 2024-04-07T17:51:23.589 INFO:tasks.workunit.client.0.smithi100.stdout: "[134217728, 268435455]", 2024-04-07T17:51:23.589 INFO:tasks.workunit.client.0.smithi100.stdout: "[268435456, 536870911]", 2024-04-07T17:51:23.589 INFO:tasks.workunit.client.0.smithi100.stdout: "[536870912, 1073741823]", 2024-04-07T17:51:23.589 INFO:tasks.workunit.client.0.smithi100.stdout: "[1073741824, 2147483647]", 2024-04-07T17:51:23.589 INFO:tasks.workunit.client.0.smithi100.stdout: "[2147483648, 4294967295]", 2024-04-07T17:51:23.589 INFO:tasks.workunit.client.0.smithi100.stdout: "[4294967296, 8589934591]", 2024-04-07T17:51:23.589 INFO:tasks.workunit.client.0.smithi100.stdout: "[8589934592, 17179869183]", 2024-04-07T17:51:23.589 INFO:tasks.workunit.client.0.smithi100.stdout: "[17179869184, 34359738367]", 2024-04-07T17:51:23.589 INFO:tasks.workunit.client.0.smithi100.stdout: "[34359738368, 68719476735]", 2024-04-07T17:51:23.589 INFO:tasks.workunit.client.0.smithi100.stdout: "[68719476736, 137438953471]", 2024-04-07T17:51:23.589 INFO:tasks.workunit.client.0.smithi100.stdout: "[137438953472, 274877906943]", 2024-04-07T17:51:23.590 INFO:tasks.workunit.client.0.smithi100.stdout: "[274877906944, None]" 2024-04-07T17:51:23.590 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:23.590 INFO:tasks.workunit.client.0.smithi100.stdout: "scale_type": "log2" 2024-04-07T17:51:23.590 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:23.590 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:23.590 INFO:tasks.workunit.client.0.smithi100.stdout: "osds": [ 2024-04-07T17:51:23.590 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:23.590 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 0, 2024-04-07T17:51:23.590 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:23.590 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.590 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.590 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.590 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.590 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.590 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 45, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.591 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 604, 18, 1, 2, 6, 0, 3, 3, 6, 3, 4, 1, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.591 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 2693, 8, 3, 0, 4, 3, 9, 15, 24, 57, 124, 51, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.591 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 270, 5, 0, 0, 1, 1, 0, 4, 3, 10, 38, 191, 289, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.591 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 410, 5, 0, 0, 0, 0, 1, 1, 5, 7, 14, 34, 121, 20, 38, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.591 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 566, 0, 0, 0, 0, 0, 0, 0, 0, 2, 3, 6, 14, 38, 152, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.591 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 778, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 4, 29, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.591 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 481, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.591 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 388, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.591 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.591 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 4, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.591 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.591 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.591 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.591 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.592 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.592 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.592 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.592 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.592 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.592 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.592 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.592 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.592 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.592 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.592 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.592 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:23.592 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:23.592 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.593 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:23.593 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 1, 2024-04-07T17:51:23.593 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:23.593 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.593 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.593 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.593 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.593 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.593 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 21, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.593 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 641, 34, 4, 2, 6, 0, 0, 1, 0, 0, 2, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.593 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 3452, 9, 3, 2, 2, 7, 13, 17, 33, 73, 134, 69, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.593 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 133, 5, 0, 0, 2, 1, 2, 1, 3, 6, 19, 211, 422, 3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.593 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 66, 11, 0, 0, 0, 0, 0, 0, 3, 1, 6, 9, 158, 22, 32, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.593 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 191, 32, 0, 0, 0, 0, 0, 0, 0, 0, 6, 9, 24, 39, 105, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.594 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1198, 50, 0, 0, 0, 0, 0, 0, 0, 1, 0, 1, 0, 3, 18, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.594 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1626, 138, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.594 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 662, 17, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.594 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.594 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.594 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.594 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.594 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.594 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.594 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.595 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.595 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.595 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.595 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.595 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.595 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.595 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.595 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.595 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.595 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.595 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:23.595 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:23.595 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.595 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:23.596 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 2, 2024-04-07T17:51:23.596 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:23.596 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.596 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.596 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.596 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.596 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.596 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 22, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.596 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 335, 6, 1, 0, 4, 0, 1, 5, 6, 2, 5, 5, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.596 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 207, 9, 1, 0, 3, 0, 4, 3, 10, 17, 51, 63, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.596 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 199, 14, 0, 0, 0, 1, 4, 6, 13, 13, 79, 92, 13, 6, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.596 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 155, 8, 0, 0, 0, 0, 2, 2, 4, 11, 32, 52, 31, 36, 46, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.596 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 44, 0, 0, 0, 0, 0, 0, 0, 0, 3, 12, 14, 17, 59, 136, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.596 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 6, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 2, 4, 4, 27, 3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.597 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.597 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.597 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.597 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.597 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.597 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.597 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.597 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.597 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.597 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.597 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.597 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.597 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.597 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.598 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.598 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.598 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.598 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.598 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.598 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:23.598 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:23.598 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.598 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:23.598 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 3, 2024-04-07T17:51:23.598 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:23.598 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.598 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.598 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.599 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.599 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.599 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 26, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.599 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 494, 15, 0, 0, 11, 2, 2, 3, 5, 0, 3, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.599 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1353, 2, 2, 0, 3, 3, 5, 7, 20, 29, 59, 33, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.599 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 366, 7, 0, 0, 1, 0, 0, 3, 4, 6, 16, 89, 157, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.599 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 404, 5, 0, 0, 0, 0, 0, 1, 3, 1, 4, 17, 61, 29, 16, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.599 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 330, 0, 0, 0, 0, 0, 1, 0, 2, 1, 0, 4, 21, 46, 72, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.599 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 124, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 5, 3, 13, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.599 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 46, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.599 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.599 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.599 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.600 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.600 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.600 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.600 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.600 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.600 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.600 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.600 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.600 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.600 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.600 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.600 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.600 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.600 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.600 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.600 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:23.601 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:23.601 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.601 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:23.601 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 4, 2024-04-07T17:51:23.601 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:23.601 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.601 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.601 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.601 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.601 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.601 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 16, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.601 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 441, 22, 0, 0, 7, 0, 1, 0, 0, 0, 1, 2, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.601 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1429, 3, 1, 1, 4, 5, 1, 8, 16, 28, 51, 23, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.601 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 545, 0, 0, 0, 0, 2, 3, 3, 3, 3, 26, 99, 129, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.601 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 331, 0, 0, 0, 0, 0, 0, 0, 1, 3, 19, 19, 107, 9, 19, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.602 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 162, 0, 0, 0, 0, 0, 0, 0, 0, 1, 2, 3, 7, 23, 85, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.602 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 151, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 2, 19, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.602 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 71, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.602 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.602 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.602 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.602 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.602 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.602 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.602 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.602 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.602 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.602 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.602 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.603 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.603 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.603 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.603 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.603 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.603 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.603 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.603 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:23.603 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:23.603 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.603 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:23.603 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 5, 2024-04-07T17:51:23.603 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:23.604 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.604 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.604 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.604 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.604 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.604 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 12, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.604 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 550, 9, 0, 0, 5, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.604 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 111, 8, 0, 0, 8, 0, 0, 2, 3, 4, 9, 9, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.604 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 90, 5, 0, 0, 0, 0, 3, 4, 5, 5, 13, 12, 14, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.604 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 41, 4, 0, 0, 0, 0, 0, 0, 0, 3, 4, 15, 16, 15, 20, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.604 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 13, 0, 0, 0, 0, 0, 0, 1, 0, 0, 6, 4, 6, 42, 128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.604 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 3, 26, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.604 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.605 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.605 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.605 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.605 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.605 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.605 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.605 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.605 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.605 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.605 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.605 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.605 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.605 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.605 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.605 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.605 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.605 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.606 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.606 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:23.606 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:23.606 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.606 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:23.606 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 6, 2024-04-07T17:51:23.606 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:23.606 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.606 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.606 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.606 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.606 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.606 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 11, 0, 0, 1, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.606 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 423, 33, 0, 0, 16, 0, 2, 2, 1, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.607 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1503, 6, 1, 2, 5, 0, 4, 10, 23, 27, 44, 18, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.607 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 492, 4, 0, 0, 1, 1, 1, 1, 1, 2, 19, 86, 117, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.607 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 382, 9, 0, 0, 0, 0, 0, 0, 1, 1, 4, 7, 121, 9, 15, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.607 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 160, 28, 0, 0, 0, 0, 0, 0, 1, 1, 1, 3, 11, 32, 99, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.607 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 129, 75, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 2, 20, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.607 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 75, 153, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.607 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.607 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.607 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.607 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.607 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.607 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.607 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.607 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.607 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.607 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.608 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.608 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.608 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.608 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.608 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.608 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.608 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.608 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.608 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:23.608 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:23.608 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.608 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:23.608 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 7, 2024-04-07T17:51:23.608 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:23.608 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.608 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.609 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.609 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.609 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.609 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 12, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.609 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 239, 5, 0, 0, 6, 0, 2, 3, 5, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.609 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 61, 4, 0, 0, 2, 1, 1, 2, 3, 12, 8, 13, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.609 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 75, 12, 0, 0, 0, 0, 2, 2, 2, 9, 9, 19, 6, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.609 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 49, 15, 0, 0, 0, 0, 1, 2, 2, 2, 8, 21, 29, 29, 10, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.609 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 13, 33, 0, 0, 0, 0, 1, 0, 0, 2, 2, 4, 15, 67, 122, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.609 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 64, 0, 0, 0, 0, 0, 0, 0, 1, 0, 1, 0, 4, 26, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.609 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 149, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.609 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.609 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.609 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.609 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.610 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.610 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.610 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.610 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.610 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.610 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.610 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.610 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.610 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.610 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.610 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.610 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.610 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.610 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.610 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:23.611 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:23.611 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:23.611 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:23.611 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:23.611 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.611 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:23.611 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_resrv_repnum_vs_duration": { 2024-04-07T17:51:23.611 INFO:tasks.workunit.client.0.smithi100.stdout: "axes": [ 2024-04-07T17:51:23.611 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:23.611 INFO:tasks.workunit.client.0.smithi100.stdout: "buckets": 8, 2024-04-07T17:51:23.611 INFO:tasks.workunit.client.0.smithi100.stdout: "min": 0, 2024-04-07T17:51:23.611 INFO:tasks.workunit.client.0.smithi100.stdout: "name": "number of replicas", 2024-04-07T17:51:23.611 INFO:tasks.workunit.client.0.smithi100.stdout: "quant_size": 1, 2024-04-07T17:51:23.611 INFO:tasks.workunit.client.0.smithi100.stdout: "ranges": [ 2024-04-07T17:51:23.611 INFO:tasks.workunit.client.0.smithi100.stdout: "[None, -1]", 2024-04-07T17:51:23.611 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0]", 2024-04-07T17:51:23.612 INFO:tasks.workunit.client.0.smithi100.stdout: "[1, 1]", 2024-04-07T17:51:23.612 INFO:tasks.workunit.client.0.smithi100.stdout: "[2, 2]", 2024-04-07T17:51:23.612 INFO:tasks.workunit.client.0.smithi100.stdout: "[3, 3]", 2024-04-07T17:51:23.612 INFO:tasks.workunit.client.0.smithi100.stdout: "[4, 4]", 2024-04-07T17:51:23.612 INFO:tasks.workunit.client.0.smithi100.stdout: "[5, 5]", 2024-04-07T17:51:23.612 INFO:tasks.workunit.client.0.smithi100.stdout: "[6, None]" 2024-04-07T17:51:23.612 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:23.612 INFO:tasks.workunit.client.0.smithi100.stdout: "scale_type": "linear" 2024-04-07T17:51:23.612 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.612 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:23.612 INFO:tasks.workunit.client.0.smithi100.stdout: "buckets": 10, 2024-04-07T17:51:23.612 INFO:tasks.workunit.client.0.smithi100.stdout: "min": 0, 2024-04-07T17:51:23.612 INFO:tasks.workunit.client.0.smithi100.stdout: "name": "duration", 2024-04-07T17:51:23.612 INFO:tasks.workunit.client.0.smithi100.stdout: "quant_size": 250000, 2024-04-07T17:51:23.612 INFO:tasks.workunit.client.0.smithi100.stdout: "ranges": [ 2024-04-07T17:51:23.612 INFO:tasks.workunit.client.0.smithi100.stdout: "[None, -1]", 2024-04-07T17:51:23.613 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 249999]", 2024-04-07T17:51:23.613 INFO:tasks.workunit.client.0.smithi100.stdout: "[250000, 499999]", 2024-04-07T17:51:23.613 INFO:tasks.workunit.client.0.smithi100.stdout: "[500000, 999999]", 2024-04-07T17:51:23.613 INFO:tasks.workunit.client.0.smithi100.stdout: "[1000000, 1999999]", 2024-04-07T17:51:23.613 INFO:tasks.workunit.client.0.smithi100.stdout: "[2000000, 3999999]", 2024-04-07T17:51:23.613 INFO:tasks.workunit.client.0.smithi100.stdout: "[4000000, 7999999]", 2024-04-07T17:51:23.613 INFO:tasks.workunit.client.0.smithi100.stdout: "[8000000, 15999999]", 2024-04-07T17:51:23.613 INFO:tasks.workunit.client.0.smithi100.stdout: "[16000000, 31999999]", 2024-04-07T17:51:23.613 INFO:tasks.workunit.client.0.smithi100.stdout: "[32000000, None]" 2024-04-07T17:51:23.613 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:23.613 INFO:tasks.workunit.client.0.smithi100.stdout: "scale_type": "log2" 2024-04-07T17:51:23.613 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:23.613 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:23.613 INFO:tasks.workunit.client.0.smithi100.stdout: "osds": [ 2024-04-07T17:51:23.613 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:23.614 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 0, 2024-04-07T17:51:23.614 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:23.614 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.614 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.614 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.614 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.614 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.614 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.614 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.614 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:23.614 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:23.614 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.614 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:23.614 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 1, 2024-04-07T17:51:23.614 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:23.615 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.615 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.615 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.615 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.615 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.615 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.615 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.615 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:23.615 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:23.615 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.615 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:23.615 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 2, 2024-04-07T17:51:23.615 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:23.615 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.615 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.615 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.616 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.616 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.616 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.616 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.616 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:23.616 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:23.616 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.616 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:23.616 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 3, 2024-04-07T17:51:23.616 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:23.616 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.616 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.616 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.616 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.617 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.617 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.617 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.617 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:23.617 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:23.617 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.617 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:23.617 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 4, 2024-04-07T17:51:23.617 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:23.617 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.617 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.617 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.617 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.617 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.617 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.618 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.618 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:23.618 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:23.618 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.618 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:23.618 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 5, 2024-04-07T17:51:23.618 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:23.618 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.618 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.618 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.618 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.618 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.618 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.619 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.619 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:23.619 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:23.619 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.619 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:23.619 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 6, 2024-04-07T17:51:23.619 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:23.619 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.619 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.619 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.619 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.619 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.619 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.619 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.619 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:23.620 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:23.620 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.620 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:23.620 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 7, 2024-04-07T17:51:23.620 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:23.620 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.620 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.620 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.620 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.620 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.620 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.620 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:23.620 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:23.620 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:23.620 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:23.620 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:23.621 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:23.621 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:23.621 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:23.621 INFO:tasks.workunit.client.0.smithi100.stdout: "perf_counters": { 2024-04-07T17:51:23.621 INFO:tasks.workunit.client.0.smithi100.stdout: "mds.f47a4ff15a45950a63160265564a123fc5da5e5e": { 2024-04-07T17:51:23.621 INFO:tasks.workunit.client.0.smithi100.stdout: "mds": { 2024-04-07T17:51:23.621 INFO:tasks.workunit.client.0.smithi100.stdout: "caps": { 2024-04-07T17:51:23.621 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.621 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.621 INFO:tasks.workunit.client.0.smithi100.stdout: "ceph_cap_op_flush_ack": { 2024-04-07T17:51:23.621 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.621 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.621 INFO:tasks.workunit.client.0.smithi100.stdout: "ceph_cap_op_flushsnap_ack": { 2024-04-07T17:51:23.621 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.621 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.621 INFO:tasks.workunit.client.0.smithi100.stdout: "ceph_cap_op_grant": { 2024-04-07T17:51:23.622 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.622 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.622 INFO:tasks.workunit.client.0.smithi100.stdout: "ceph_cap_op_revoke": { 2024-04-07T17:51:23.622 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.622 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.622 INFO:tasks.workunit.client.0.smithi100.stdout: "ceph_cap_op_trunc": { 2024-04-07T17:51:23.622 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.622 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.622 INFO:tasks.workunit.client.0.smithi100.stdout: "dir_commit": { 2024-04-07T17:51:23.622 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.622 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.622 INFO:tasks.workunit.client.0.smithi100.stdout: "dir_fetch_complete": { 2024-04-07T17:51:23.622 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 12 2024-04-07T17:51:23.622 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.622 INFO:tasks.workunit.client.0.smithi100.stdout: "dir_fetch_keys": { 2024-04-07T17:51:23.623 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 10 2024-04-07T17:51:23.623 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.623 INFO:tasks.workunit.client.0.smithi100.stdout: "dir_merge": { 2024-04-07T17:51:23.623 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.623 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.623 INFO:tasks.workunit.client.0.smithi100.stdout: "dir_split": { 2024-04-07T17:51:23.623 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.623 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.623 INFO:tasks.workunit.client.0.smithi100.stdout: "exported_inodes": { 2024-04-07T17:51:23.623 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.623 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.623 INFO:tasks.workunit.client.0.smithi100.stdout: "forward": { 2024-04-07T17:51:23.623 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.623 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.623 INFO:tasks.workunit.client.0.smithi100.stdout: "handle_client_cap_release": { 2024-04-07T17:51:23.624 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.624 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.624 INFO:tasks.workunit.client.0.smithi100.stdout: "handle_client_caps": { 2024-04-07T17:51:23.624 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.624 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.624 INFO:tasks.workunit.client.0.smithi100.stdout: "handle_client_caps_dirty": { 2024-04-07T17:51:23.624 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.624 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.624 INFO:tasks.workunit.client.0.smithi100.stdout: "handle_inode_file_caps": { 2024-04-07T17:51:23.624 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.624 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.624 INFO:tasks.workunit.client.0.smithi100.stdout: "imported_inodes": { 2024-04-07T17:51:23.624 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.624 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.625 INFO:tasks.workunit.client.0.smithi100.stdout: "inodes": { 2024-04-07T17:51:23.625 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 10 2024-04-07T17:51:23.625 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.625 INFO:tasks.workunit.client.0.smithi100.stdout: "inodes_expired": { 2024-04-07T17:51:23.625 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.625 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.625 INFO:tasks.workunit.client.0.smithi100.stdout: "inodes_pinned": { 2024-04-07T17:51:23.625 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 10 2024-04-07T17:51:23.625 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.625 INFO:tasks.workunit.client.0.smithi100.stdout: "inodes_with_caps": { 2024-04-07T17:51:23.625 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.625 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.625 INFO:tasks.workunit.client.0.smithi100.stdout: "load_cent": { 2024-04-07T17:51:23.625 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.625 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.626 INFO:tasks.workunit.client.0.smithi100.stdout: "openino_dir_fetch": { 2024-04-07T17:51:23.626 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.626 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.626 INFO:tasks.workunit.client.0.smithi100.stdout: "process_request_cap_release": { 2024-04-07T17:51:23.626 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.626 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.626 INFO:tasks.workunit.client.0.smithi100.stdout: "reply_latency": { 2024-04-07T17:51:23.626 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:23.626 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.626 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.626 INFO:tasks.workunit.client.0.smithi100.stdout: "request": { 2024-04-07T17:51:23.626 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.626 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.626 INFO:tasks.workunit.client.0.smithi100.stdout: "root_rbytes": { 2024-04-07T17:51:23.626 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.627 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.627 INFO:tasks.workunit.client.0.smithi100.stdout: "root_rfiles": { 2024-04-07T17:51:23.627 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.627 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.627 INFO:tasks.workunit.client.0.smithi100.stdout: "root_rsnaps": { 2024-04-07T17:51:23.627 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.627 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.627 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_reply": { 2024-04-07T17:51:23.627 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.627 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.627 INFO:tasks.workunit.client.0.smithi100.stdout: "subtrees": { 2024-04-07T17:51:23.627 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2 2024-04-07T17:51:23.627 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:23.627 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.627 INFO:tasks.workunit.client.0.smithi100.stdout: "mds_cache": { 2024-04-07T17:51:23.627 INFO:tasks.workunit.client.0.smithi100.stdout: "ireq_enqueue_scrub": { 2024-04-07T17:51:23.628 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.628 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.628 INFO:tasks.workunit.client.0.smithi100.stdout: "ireq_exportdir": { 2024-04-07T17:51:23.628 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.628 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.628 INFO:tasks.workunit.client.0.smithi100.stdout: "ireq_flush": { 2024-04-07T17:51:23.628 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.628 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.628 INFO:tasks.workunit.client.0.smithi100.stdout: "ireq_fragmentdir": { 2024-04-07T17:51:23.628 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.628 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.628 INFO:tasks.workunit.client.0.smithi100.stdout: "ireq_fragstats": { 2024-04-07T17:51:23.628 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.628 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.628 INFO:tasks.workunit.client.0.smithi100.stdout: "ireq_inodestats": { 2024-04-07T17:51:23.629 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.629 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.629 INFO:tasks.workunit.client.0.smithi100.stdout: "ireq_quiesce_inode": { 2024-04-07T17:51:23.629 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.629 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.629 INFO:tasks.workunit.client.0.smithi100.stdout: "ireq_quiesce_path": { 2024-04-07T17:51:23.629 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.629 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.629 INFO:tasks.workunit.client.0.smithi100.stdout: "num_recovering_enqueued": { 2024-04-07T17:51:23.629 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.629 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.629 INFO:tasks.workunit.client.0.smithi100.stdout: "num_recovering_prioritized": { 2024-04-07T17:51:23.629 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.629 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.629 INFO:tasks.workunit.client.0.smithi100.stdout: "num_recovering_processing": { 2024-04-07T17:51:23.630 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.630 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.630 INFO:tasks.workunit.client.0.smithi100.stdout: "num_strays": { 2024-04-07T17:51:23.630 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.630 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.630 INFO:tasks.workunit.client.0.smithi100.stdout: "num_strays_delayed": { 2024-04-07T17:51:23.630 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.630 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.630 INFO:tasks.workunit.client.0.smithi100.stdout: "num_strays_enqueuing": { 2024-04-07T17:51:23.630 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.630 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.630 INFO:tasks.workunit.client.0.smithi100.stdout: "recovery_completed": { 2024-04-07T17:51:23.630 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.630 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.631 INFO:tasks.workunit.client.0.smithi100.stdout: "recovery_started": { 2024-04-07T17:51:23.631 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.631 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.631 INFO:tasks.workunit.client.0.smithi100.stdout: "strays_created": { 2024-04-07T17:51:23.631 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.631 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.631 INFO:tasks.workunit.client.0.smithi100.stdout: "strays_enqueued": { 2024-04-07T17:51:23.631 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.631 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.631 INFO:tasks.workunit.client.0.smithi100.stdout: "strays_migrated": { 2024-04-07T17:51:23.631 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.631 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.631 INFO:tasks.workunit.client.0.smithi100.stdout: "strays_reintegrated": { 2024-04-07T17:51:23.631 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.631 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:23.632 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.632 INFO:tasks.workunit.client.0.smithi100.stdout: "mds_log": { 2024-04-07T17:51:23.632 INFO:tasks.workunit.client.0.smithi100.stdout: "ev": { 2024-04-07T17:51:23.632 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 17 2024-04-07T17:51:23.632 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.632 INFO:tasks.workunit.client.0.smithi100.stdout: "evadd": { 2024-04-07T17:51:23.632 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.632 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.632 INFO:tasks.workunit.client.0.smithi100.stdout: "evex": { 2024-04-07T17:51:23.632 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.632 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.632 INFO:tasks.workunit.client.0.smithi100.stdout: "evexd": { 2024-04-07T17:51:23.632 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.632 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.632 INFO:tasks.workunit.client.0.smithi100.stdout: "evexg": { 2024-04-07T17:51:23.632 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.632 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.633 INFO:tasks.workunit.client.0.smithi100.stdout: "evlrg": { 2024-04-07T17:51:23.633 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.633 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.633 INFO:tasks.workunit.client.0.smithi100.stdout: "evtrm": { 2024-04-07T17:51:23.633 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.633 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.633 INFO:tasks.workunit.client.0.smithi100.stdout: "jlat": { 2024-04-07T17:51:23.633 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:23.633 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.633 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.633 INFO:tasks.workunit.client.0.smithi100.stdout: "replayed": { 2024-04-07T17:51:23.633 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 17 2024-04-07T17:51:23.633 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.633 INFO:tasks.workunit.client.0.smithi100.stdout: "seg": { 2024-04-07T17:51:23.633 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1 2024-04-07T17:51:23.634 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.634 INFO:tasks.workunit.client.0.smithi100.stdout: "segadd": { 2024-04-07T17:51:23.634 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.634 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.634 INFO:tasks.workunit.client.0.smithi100.stdout: "segex": { 2024-04-07T17:51:23.634 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.634 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.634 INFO:tasks.workunit.client.0.smithi100.stdout: "segexd": { 2024-04-07T17:51:23.634 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.634 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.634 INFO:tasks.workunit.client.0.smithi100.stdout: "segexg": { 2024-04-07T17:51:23.634 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.634 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.634 INFO:tasks.workunit.client.0.smithi100.stdout: "segmjr": { 2024-04-07T17:51:23.634 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1 2024-04-07T17:51:23.635 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.635 INFO:tasks.workunit.client.0.smithi100.stdout: "segtrm": { 2024-04-07T17:51:23.635 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.635 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:23.635 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.635 INFO:tasks.workunit.client.0.smithi100.stdout: "mds_mem": { 2024-04-07T17:51:23.635 INFO:tasks.workunit.client.0.smithi100.stdout: "cap": { 2024-04-07T17:51:23.635 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.635 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.635 INFO:tasks.workunit.client.0.smithi100.stdout: "cap+": { 2024-04-07T17:51:23.635 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.635 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.635 INFO:tasks.workunit.client.0.smithi100.stdout: "cap-": { 2024-04-07T17:51:23.635 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.635 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.636 INFO:tasks.workunit.client.0.smithi100.stdout: "dir": { 2024-04-07T17:51:23.636 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 12 2024-04-07T17:51:23.636 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.636 INFO:tasks.workunit.client.0.smithi100.stdout: "dir+": { 2024-04-07T17:51:23.636 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 12 2024-04-07T17:51:23.636 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.636 INFO:tasks.workunit.client.0.smithi100.stdout: "dir-": { 2024-04-07T17:51:23.636 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.636 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.636 INFO:tasks.workunit.client.0.smithi100.stdout: "dn": { 2024-04-07T17:51:23.636 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 10 2024-04-07T17:51:23.636 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.636 INFO:tasks.workunit.client.0.smithi100.stdout: "dn+": { 2024-04-07T17:51:23.636 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 10 2024-04-07T17:51:23.636 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.637 INFO:tasks.workunit.client.0.smithi100.stdout: "dn-": { 2024-04-07T17:51:23.637 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.637 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.637 INFO:tasks.workunit.client.0.smithi100.stdout: "heap": { 2024-04-07T17:51:23.637 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 248184 2024-04-07T17:51:23.637 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.637 INFO:tasks.workunit.client.0.smithi100.stdout: "ino": { 2024-04-07T17:51:23.637 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 13 2024-04-07T17:51:23.637 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.637 INFO:tasks.workunit.client.0.smithi100.stdout: "ino+": { 2024-04-07T17:51:23.637 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 13 2024-04-07T17:51:23.637 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.637 INFO:tasks.workunit.client.0.smithi100.stdout: "ino-": { 2024-04-07T17:51:23.637 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.637 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.638 INFO:tasks.workunit.client.0.smithi100.stdout: "rss": { 2024-04-07T17:51:23.638 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 46296 2024-04-07T17:51:23.638 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:23.638 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.638 INFO:tasks.workunit.client.0.smithi100.stdout: "mds_server": { 2024-04-07T17:51:23.638 INFO:tasks.workunit.client.0.smithi100.stdout: "cap_acquisition_throttle": { 2024-04-07T17:51:23.638 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.638 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.638 INFO:tasks.workunit.client.0.smithi100.stdout: "cap_revoke_eviction": { 2024-04-07T17:51:23.638 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.638 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.638 INFO:tasks.workunit.client.0.smithi100.stdout: "handle_client_request": { 2024-04-07T17:51:23.638 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.638 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.639 INFO:tasks.workunit.client.0.smithi100.stdout: "handle_client_session": { 2024-04-07T17:51:23.639 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.639 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.639 INFO:tasks.workunit.client.0.smithi100.stdout: "handle_peer_request": { 2024-04-07T17:51:23.639 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.639 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.639 INFO:tasks.workunit.client.0.smithi100.stdout: "req_create_latency": { 2024-04-07T17:51:23.639 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:23.639 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.639 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.639 INFO:tasks.workunit.client.0.smithi100.stdout: "req_getattr_latency": { 2024-04-07T17:51:23.639 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:23.639 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.639 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.639 INFO:tasks.workunit.client.0.smithi100.stdout: "req_getfilelock_latency": { 2024-04-07T17:51:23.639 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:23.639 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.640 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.640 INFO:tasks.workunit.client.0.smithi100.stdout: "req_getvxattr_latency": { 2024-04-07T17:51:23.640 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:23.640 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.640 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.640 INFO:tasks.workunit.client.0.smithi100.stdout: "req_link_latency": { 2024-04-07T17:51:23.640 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:23.640 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.640 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.640 INFO:tasks.workunit.client.0.smithi100.stdout: "req_lookup_latency": { 2024-04-07T17:51:23.640 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:23.640 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.640 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.640 INFO:tasks.workunit.client.0.smithi100.stdout: "req_lookuphash_latency": { 2024-04-07T17:51:23.640 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:23.641 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.641 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.641 INFO:tasks.workunit.client.0.smithi100.stdout: "req_lookupino_latency": { 2024-04-07T17:51:23.641 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:23.641 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.641 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.641 INFO:tasks.workunit.client.0.smithi100.stdout: "req_lookupname_latency": { 2024-04-07T17:51:23.641 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:23.641 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.641 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.641 INFO:tasks.workunit.client.0.smithi100.stdout: "req_lookupparent_latency": { 2024-04-07T17:51:23.641 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:23.641 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.641 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.641 INFO:tasks.workunit.client.0.smithi100.stdout: "req_lookupsnap_latency": { 2024-04-07T17:51:23.641 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:23.642 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.642 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.642 INFO:tasks.workunit.client.0.smithi100.stdout: "req_lssnap_latency": { 2024-04-07T17:51:23.642 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:23.642 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.642 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.642 INFO:tasks.workunit.client.0.smithi100.stdout: "req_mkdir_latency": { 2024-04-07T17:51:23.642 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:23.642 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.642 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.642 INFO:tasks.workunit.client.0.smithi100.stdout: "req_mknod_latency": { 2024-04-07T17:51:23.642 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:23.642 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.642 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.643 INFO:tasks.workunit.client.0.smithi100.stdout: "req_mksnap_latency": { 2024-04-07T17:51:23.643 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:23.643 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.643 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.643 INFO:tasks.workunit.client.0.smithi100.stdout: "req_open_latency": { 2024-04-07T17:51:23.643 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:23.643 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.643 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.643 INFO:tasks.workunit.client.0.smithi100.stdout: "req_readdir_latency": { 2024-04-07T17:51:23.643 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:23.643 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.643 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.643 INFO:tasks.workunit.client.0.smithi100.stdout: "req_rename_latency": { 2024-04-07T17:51:23.643 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:23.643 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.643 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.644 INFO:tasks.workunit.client.0.smithi100.stdout: "req_renamesnap_latency": { 2024-04-07T17:51:23.644 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:23.644 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.644 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.644 INFO:tasks.workunit.client.0.smithi100.stdout: "req_rmdir_latency": { 2024-04-07T17:51:23.644 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:23.644 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.644 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.644 INFO:tasks.workunit.client.0.smithi100.stdout: "req_rmsnap_latency": { 2024-04-07T17:51:23.644 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:23.644 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.644 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.644 INFO:tasks.workunit.client.0.smithi100.stdout: "req_rmxattr_latency": { 2024-04-07T17:51:23.644 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:23.644 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.645 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.645 INFO:tasks.workunit.client.0.smithi100.stdout: "req_setattr_latency": { 2024-04-07T17:51:23.645 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:23.645 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.645 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.645 INFO:tasks.workunit.client.0.smithi100.stdout: "req_setdirlayout_latency": { 2024-04-07T17:51:23.645 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:23.645 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.645 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.645 INFO:tasks.workunit.client.0.smithi100.stdout: "req_setfilelock_latency": { 2024-04-07T17:51:23.645 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:23.645 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.645 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.645 INFO:tasks.workunit.client.0.smithi100.stdout: "req_setlayout_latency": { 2024-04-07T17:51:23.645 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:23.645 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.645 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.646 INFO:tasks.workunit.client.0.smithi100.stdout: "req_setxattr_latency": { 2024-04-07T17:51:23.646 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:23.646 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.646 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.646 INFO:tasks.workunit.client.0.smithi100.stdout: "req_snapdiff_latency": { 2024-04-07T17:51:23.646 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:23.646 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.646 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.646 INFO:tasks.workunit.client.0.smithi100.stdout: "req_symlink_latency": { 2024-04-07T17:51:23.646 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:23.646 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.646 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.646 INFO:tasks.workunit.client.0.smithi100.stdout: "req_unlink_latency": { 2024-04-07T17:51:23.647 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:23.647 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.647 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:23.647 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.647 INFO:tasks.workunit.client.0.smithi100.stdout: "mds_sessions": { 2024-04-07T17:51:23.647 INFO:tasks.workunit.client.0.smithi100.stdout: "average_load": { 2024-04-07T17:51:23.647 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.647 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.647 INFO:tasks.workunit.client.0.smithi100.stdout: "avg_session_uptime": { 2024-04-07T17:51:23.647 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.647 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.647 INFO:tasks.workunit.client.0.smithi100.stdout: "mdthresh_evicted": { 2024-04-07T17:51:23.647 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.647 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.647 INFO:tasks.workunit.client.0.smithi100.stdout: "session_add": { 2024-04-07T17:51:23.648 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 8 2024-04-07T17:51:23.648 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.648 INFO:tasks.workunit.client.0.smithi100.stdout: "session_count": { 2024-04-07T17:51:23.648 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.648 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.648 INFO:tasks.workunit.client.0.smithi100.stdout: "session_remove": { 2024-04-07T17:51:23.648 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 8 2024-04-07T17:51:23.648 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.648 INFO:tasks.workunit.client.0.smithi100.stdout: "sessions_open": { 2024-04-07T17:51:23.648 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2 2024-04-07T17:51:23.648 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.648 INFO:tasks.workunit.client.0.smithi100.stdout: "sessions_stale": { 2024-04-07T17:51:23.648 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.648 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.648 INFO:tasks.workunit.client.0.smithi100.stdout: "total_load": { 2024-04-07T17:51:23.649 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.649 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:23.649 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.649 INFO:tasks.workunit.client.0.smithi100.stdout: "objecter": { 2024-04-07T17:51:23.649 INFO:tasks.workunit.client.0.smithi100.stdout: "op_active": { 2024-04-07T17:51:23.649 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.649 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.649 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r": { 2024-04-07T17:51:23.649 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 37 2024-04-07T17:51:23.649 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.649 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rmw": { 2024-04-07T17:51:23.649 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.649 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.649 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w": { 2024-04-07T17:51:23.649 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 7 2024-04-07T17:51:23.650 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:23.650 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.650 INFO:tasks.workunit.client.0.smithi100.stdout: "purge_queue": { 2024-04-07T17:51:23.650 INFO:tasks.workunit.client.0.smithi100.stdout: "pq_executed": { 2024-04-07T17:51:23.650 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.650 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.650 INFO:tasks.workunit.client.0.smithi100.stdout: "pq_executed_ops": { 2024-04-07T17:51:23.650 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.650 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.650 INFO:tasks.workunit.client.0.smithi100.stdout: "pq_executing": { 2024-04-07T17:51:23.650 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.650 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.650 INFO:tasks.workunit.client.0.smithi100.stdout: "pq_executing_high_water": { 2024-04-07T17:51:23.650 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.651 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.651 INFO:tasks.workunit.client.0.smithi100.stdout: "pq_executing_ops": { 2024-04-07T17:51:23.651 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.651 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.651 INFO:tasks.workunit.client.0.smithi100.stdout: "pq_executing_ops_high_water": { 2024-04-07T17:51:23.651 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.651 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.651 INFO:tasks.workunit.client.0.smithi100.stdout: "pq_item_in_journal": { 2024-04-07T17:51:23.651 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.651 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:23.651 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:23.651 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.651 INFO:tasks.workunit.client.0.smithi100.stdout: "mon.8dd0baa036f0a7e6d1f3bdce86136fe527836c3d": { 2024-04-07T17:51:23.651 INFO:tasks.workunit.client.0.smithi100.stdout: "mon": { 2024-04-07T17:51:23.651 INFO:tasks.workunit.client.0.smithi100.stdout: "election_call": { 2024-04-07T17:51:23.652 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2 2024-04-07T17:51:23.652 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.652 INFO:tasks.workunit.client.0.smithi100.stdout: "election_lose": { 2024-04-07T17:51:23.652 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3 2024-04-07T17:51:23.652 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.652 INFO:tasks.workunit.client.0.smithi100.stdout: "election_win": { 2024-04-07T17:51:23.652 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.652 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.652 INFO:tasks.workunit.client.0.smithi100.stdout: "num_elections": { 2024-04-07T17:51:23.652 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 12 2024-04-07T17:51:23.652 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.652 INFO:tasks.workunit.client.0.smithi100.stdout: "num_sessions": { 2024-04-07T17:51:23.652 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 7 2024-04-07T17:51:23.652 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.652 INFO:tasks.workunit.client.0.smithi100.stdout: "session_add": { 2024-04-07T17:51:23.653 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4 2024-04-07T17:51:23.653 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.653 INFO:tasks.workunit.client.0.smithi100.stdout: "session_rm": { 2024-04-07T17:51:23.653 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 253 2024-04-07T17:51:23.653 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.653 INFO:tasks.workunit.client.0.smithi100.stdout: "session_trim": { 2024-04-07T17:51:23.653 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.653 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:23.653 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.653 INFO:tasks.workunit.client.0.smithi100.stdout: "paxos": { 2024-04-07T17:51:23.653 INFO:tasks.workunit.client.0.smithi100.stdout: "accept_timeout": { 2024-04-07T17:51:23.653 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.653 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.653 INFO:tasks.workunit.client.0.smithi100.stdout: "begin": { 2024-04-07T17:51:23.653 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 8797 2024-04-07T17:51:23.654 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.654 INFO:tasks.workunit.client.0.smithi100.stdout: "begin_bytes": { 2024-04-07T17:51:23.654 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 8797, 2024-04-07T17:51:23.654 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 59268356 2024-04-07T17:51:23.654 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.654 INFO:tasks.workunit.client.0.smithi100.stdout: "begin_keys": { 2024-04-07T17:51:23.654 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:23.654 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.654 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.654 INFO:tasks.workunit.client.0.smithi100.stdout: "begin_latency": { 2024-04-07T17:51:23.654 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 8797, 2024-04-07T17:51:23.654 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3095913350 2024-04-07T17:51:23.655 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.655 INFO:tasks.workunit.client.0.smithi100.stdout: "collect": { 2024-04-07T17:51:23.655 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3 2024-04-07T17:51:23.655 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.655 INFO:tasks.workunit.client.0.smithi100.stdout: "collect_bytes": { 2024-04-07T17:51:23.655 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 3, 2024-04-07T17:51:23.655 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 219 2024-04-07T17:51:23.655 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.655 INFO:tasks.workunit.client.0.smithi100.stdout: "collect_keys": { 2024-04-07T17:51:23.655 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 3, 2024-04-07T17:51:23.655 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3 2024-04-07T17:51:23.655 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.655 INFO:tasks.workunit.client.0.smithi100.stdout: "collect_latency": { 2024-04-07T17:51:23.655 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 3, 2024-04-07T17:51:23.655 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.656 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.656 INFO:tasks.workunit.client.0.smithi100.stdout: "collect_timeout": { 2024-04-07T17:51:23.656 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.656 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.656 INFO:tasks.workunit.client.0.smithi100.stdout: "collect_uncommitted": { 2024-04-07T17:51:23.656 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1 2024-04-07T17:51:23.656 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.656 INFO:tasks.workunit.client.0.smithi100.stdout: "commit": { 2024-04-07T17:51:23.656 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 8798 2024-04-07T17:51:23.656 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.656 INFO:tasks.workunit.client.0.smithi100.stdout: "commit_bytes": { 2024-04-07T17:51:23.656 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:23.656 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.656 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.656 INFO:tasks.workunit.client.0.smithi100.stdout: "commit_keys": { 2024-04-07T17:51:23.657 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:23.657 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.657 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.657 INFO:tasks.workunit.client.0.smithi100.stdout: "commit_latency": { 2024-04-07T17:51:23.657 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:23.657 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.657 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.657 INFO:tasks.workunit.client.0.smithi100.stdout: "lease_ack_timeout": { 2024-04-07T17:51:23.657 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.657 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.657 INFO:tasks.workunit.client.0.smithi100.stdout: "lease_timeout": { 2024-04-07T17:51:23.657 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.657 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.658 INFO:tasks.workunit.client.0.smithi100.stdout: "new_pn": { 2024-04-07T17:51:23.658 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.658 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.658 INFO:tasks.workunit.client.0.smithi100.stdout: "new_pn_latency": { 2024-04-07T17:51:23.658 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:23.658 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.658 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.658 INFO:tasks.workunit.client.0.smithi100.stdout: "refresh": { 2024-04-07T17:51:23.658 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 8798 2024-04-07T17:51:23.658 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.658 INFO:tasks.workunit.client.0.smithi100.stdout: "refresh_latency": { 2024-04-07T17:51:23.658 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 8798, 2024-04-07T17:51:23.658 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 52838506301 2024-04-07T17:51:23.658 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.658 INFO:tasks.workunit.client.0.smithi100.stdout: "restart": { 2024-04-07T17:51:23.659 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 14 2024-04-07T17:51:23.659 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.659 INFO:tasks.workunit.client.0.smithi100.stdout: "share_state": { 2024-04-07T17:51:23.659 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.659 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.659 INFO:tasks.workunit.client.0.smithi100.stdout: "share_state_bytes": { 2024-04-07T17:51:23.659 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:23.659 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.659 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.659 INFO:tasks.workunit.client.0.smithi100.stdout: "share_state_keys": { 2024-04-07T17:51:23.659 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:23.659 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.659 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.659 INFO:tasks.workunit.client.0.smithi100.stdout: "start_leader": { 2024-04-07T17:51:23.660 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.660 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.660 INFO:tasks.workunit.client.0.smithi100.stdout: "start_peon": { 2024-04-07T17:51:23.660 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3 2024-04-07T17:51:23.660 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.660 INFO:tasks.workunit.client.0.smithi100.stdout: "store_state": { 2024-04-07T17:51:23.660 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 8797 2024-04-07T17:51:23.660 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.660 INFO:tasks.workunit.client.0.smithi100.stdout: "store_state_bytes": { 2024-04-07T17:51:23.660 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 8797, 2024-04-07T17:51:23.660 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 116970414 2024-04-07T17:51:23.660 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.660 INFO:tasks.workunit.client.0.smithi100.stdout: "store_state_keys": { 2024-04-07T17:51:23.660 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 8797, 2024-04-07T17:51:23.660 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 76589 2024-04-07T17:51:23.661 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.661 INFO:tasks.workunit.client.0.smithi100.stdout: "store_state_latency": { 2024-04-07T17:51:23.661 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 8797, 2024-04-07T17:51:23.661 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5995831139 2024-04-07T17:51:23.661 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:23.661 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.661 INFO:tasks.workunit.client.0.smithi100.stdout: "prioritycache": { 2024-04-07T17:51:23.661 INFO:tasks.workunit.client.0.smithi100.stdout: "cache_bytes": { 2024-04-07T17:51:23.661 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1020054731 2024-04-07T17:51:23.661 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.661 INFO:tasks.workunit.client.0.smithi100.stdout: "heap_bytes": { 2024-04-07T17:51:23.661 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 166682624 2024-04-07T17:51:23.661 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.661 INFO:tasks.workunit.client.0.smithi100.stdout: "mapped_bytes": { 2024-04-07T17:51:23.662 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 165036032 2024-04-07T17:51:23.662 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.662 INFO:tasks.workunit.client.0.smithi100.stdout: "target_bytes": { 2024-04-07T17:51:23.662 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2147483648 2024-04-07T17:51:23.662 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.662 INFO:tasks.workunit.client.0.smithi100.stdout: "unmapped_bytes": { 2024-04-07T17:51:23.662 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1646592 2024-04-07T17:51:23.662 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:23.662 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.662 INFO:tasks.workunit.client.0.smithi100.stdout: "prioritycache:full": { 2024-04-07T17:51:23.662 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:23.662 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 360710144 2024-04-07T17:51:23.662 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.662 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:23.662 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.663 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.663 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:23.663 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.663 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.663 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:23.663 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 267782252 2024-04-07T17:51:23.663 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.663 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:23.663 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 23512176 2024-04-07T17:51:23.663 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.663 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:23.663 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.663 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.663 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:23.663 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.664 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.664 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:23.664 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.664 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.664 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:23.664 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.664 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.664 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:23.664 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.664 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.664 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:23.664 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.664 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.664 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:23.664 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.664 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.665 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:23.665 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.665 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.665 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:23.665 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 69415716 2024-04-07T17:51:23.665 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:23.665 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.665 INFO:tasks.workunit.client.0.smithi100.stdout: "prioritycache:inc": { 2024-04-07T17:51:23.665 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:23.665 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 339738624 2024-04-07T17:51:23.665 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.665 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:23.665 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.665 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.665 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:23.665 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.666 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.666 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:23.666 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 267782252 2024-04-07T17:51:23.666 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.666 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:23.666 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1438180 2024-04-07T17:51:23.666 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.666 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:23.666 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.666 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.666 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:23.666 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.666 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.666 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:23.667 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.667 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.667 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:23.667 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.667 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.667 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:23.667 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.667 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.667 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:23.667 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.667 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.667 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:23.667 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.667 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.667 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:23.667 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.667 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.668 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:23.668 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 70518192 2024-04-07T17:51:23.668 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:23.668 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.668 INFO:tasks.workunit.client.0.smithi100.stdout: "prioritycache:kv": { 2024-04-07T17:51:23.668 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:23.668 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 314572800 2024-04-07T17:51:23.668 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.668 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:23.668 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 67108864 2024-04-07T17:51:23.668 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.668 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:23.668 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.669 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.669 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:23.669 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 178521501 2024-04-07T17:51:23.669 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.669 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:23.669 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.669 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.669 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:23.669 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.669 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.669 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:23.669 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.669 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.669 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:23.669 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.669 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.670 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:23.670 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.670 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.670 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:23.670 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.670 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.670 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:23.670 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.670 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.670 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:23.670 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.670 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.670 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:23.670 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.670 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.670 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:23.671 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 68942435 2024-04-07T17:51:23.671 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:23.671 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.671 INFO:tasks.workunit.client.0.smithi100.stdout: "rocksdb": { 2024-04-07T17:51:23.671 INFO:tasks.workunit.client.0.smithi100.stdout: "compact": { 2024-04-07T17:51:23.671 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.671 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.671 INFO:tasks.workunit.client.0.smithi100.stdout: "compact_queue_len": { 2024-04-07T17:51:23.671 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.671 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.671 INFO:tasks.workunit.client.0.smithi100.stdout: "compact_queue_merge": { 2024-04-07T17:51:23.671 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.671 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.671 INFO:tasks.workunit.client.0.smithi100.stdout: "compact_range": { 2024-04-07T17:51:23.671 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 105 2024-04-07T17:51:23.671 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.672 INFO:tasks.workunit.client.0.smithi100.stdout: "get_latency": { 2024-04-07T17:51:23.672 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 380867, 2024-04-07T17:51:23.672 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4241508762 2024-04-07T17:51:23.672 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.672 INFO:tasks.workunit.client.0.smithi100.stdout: "rocksdb_write_delay_time": { 2024-04-07T17:51:23.672 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:23.672 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.672 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.672 INFO:tasks.workunit.client.0.smithi100.stdout: "rocksdb_write_memtable_time": { 2024-04-07T17:51:23.672 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:23.672 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.672 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.672 INFO:tasks.workunit.client.0.smithi100.stdout: "rocksdb_write_pre_and_post_time": { 2024-04-07T17:51:23.672 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:23.672 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.672 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.673 INFO:tasks.workunit.client.0.smithi100.stdout: "rocksdb_write_wal_time": { 2024-04-07T17:51:23.673 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:23.673 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.673 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.673 INFO:tasks.workunit.client.0.smithi100.stdout: "submit_latency": { 2024-04-07T17:51:23.673 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:23.673 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.673 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.673 INFO:tasks.workunit.client.0.smithi100.stdout: "submit_sync_latency": { 2024-04-07T17:51:23.673 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 21106, 2024-04-07T17:51:23.673 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 10897184086 2024-04-07T17:51:23.673 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:23.673 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:23.673 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.674 INFO:tasks.workunit.client.0.smithi100.stdout: "mon.a52fc0d8acb515431879057649ff294cfe14c620": { 2024-04-07T17:51:23.674 INFO:tasks.workunit.client.0.smithi100.stdout: "mon": { 2024-04-07T17:51:23.674 INFO:tasks.workunit.client.0.smithi100.stdout: "election_call": { 2024-04-07T17:51:23.674 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2 2024-04-07T17:51:23.674 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.674 INFO:tasks.workunit.client.0.smithi100.stdout: "election_lose": { 2024-04-07T17:51:23.674 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2 2024-04-07T17:51:23.674 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.674 INFO:tasks.workunit.client.0.smithi100.stdout: "election_win": { 2024-04-07T17:51:23.674 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.674 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.674 INFO:tasks.workunit.client.0.smithi100.stdout: "num_elections": { 2024-04-07T17:51:23.674 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 7 2024-04-07T17:51:23.674 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.674 INFO:tasks.workunit.client.0.smithi100.stdout: "num_sessions": { 2024-04-07T17:51:23.675 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 8 2024-04-07T17:51:23.675 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.675 INFO:tasks.workunit.client.0.smithi100.stdout: "session_add": { 2024-04-07T17:51:23.675 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3 2024-04-07T17:51:23.675 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.675 INFO:tasks.workunit.client.0.smithi100.stdout: "session_rm": { 2024-04-07T17:51:23.675 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 245 2024-04-07T17:51:23.675 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.675 INFO:tasks.workunit.client.0.smithi100.stdout: "session_trim": { 2024-04-07T17:51:23.675 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.675 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:23.675 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.675 INFO:tasks.workunit.client.0.smithi100.stdout: "paxos": { 2024-04-07T17:51:23.675 INFO:tasks.workunit.client.0.smithi100.stdout: "accept_timeout": { 2024-04-07T17:51:23.675 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.676 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.676 INFO:tasks.workunit.client.0.smithi100.stdout: "begin": { 2024-04-07T17:51:23.676 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 8738 2024-04-07T17:51:23.676 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.676 INFO:tasks.workunit.client.0.smithi100.stdout: "begin_bytes": { 2024-04-07T17:51:23.676 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 8738, 2024-04-07T17:51:23.676 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 58850172 2024-04-07T17:51:23.676 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.676 INFO:tasks.workunit.client.0.smithi100.stdout: "begin_keys": { 2024-04-07T17:51:23.676 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:23.676 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.676 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.676 INFO:tasks.workunit.client.0.smithi100.stdout: "begin_latency": { 2024-04-07T17:51:23.676 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 8738, 2024-04-07T17:51:23.676 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2543954542 2024-04-07T17:51:23.677 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.677 INFO:tasks.workunit.client.0.smithi100.stdout: "collect": { 2024-04-07T17:51:23.677 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2 2024-04-07T17:51:23.677 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.677 INFO:tasks.workunit.client.0.smithi100.stdout: "collect_bytes": { 2024-04-07T17:51:23.677 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 2, 2024-04-07T17:51:23.677 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 146 2024-04-07T17:51:23.677 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.677 INFO:tasks.workunit.client.0.smithi100.stdout: "collect_keys": { 2024-04-07T17:51:23.677 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 2, 2024-04-07T17:51:23.677 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2 2024-04-07T17:51:23.677 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.677 INFO:tasks.workunit.client.0.smithi100.stdout: "collect_latency": { 2024-04-07T17:51:23.677 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 2, 2024-04-07T17:51:23.677 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3999909 2024-04-07T17:51:23.678 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.678 INFO:tasks.workunit.client.0.smithi100.stdout: "collect_timeout": { 2024-04-07T17:51:23.678 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.678 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.678 INFO:tasks.workunit.client.0.smithi100.stdout: "collect_uncommitted": { 2024-04-07T17:51:23.678 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.678 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.678 INFO:tasks.workunit.client.0.smithi100.stdout: "commit": { 2024-04-07T17:51:23.678 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 8738 2024-04-07T17:51:23.678 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.678 INFO:tasks.workunit.client.0.smithi100.stdout: "commit_bytes": { 2024-04-07T17:51:23.678 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:23.678 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.678 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.678 INFO:tasks.workunit.client.0.smithi100.stdout: "commit_keys": { 2024-04-07T17:51:23.679 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:23.679 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.679 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.679 INFO:tasks.workunit.client.0.smithi100.stdout: "commit_latency": { 2024-04-07T17:51:23.679 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:23.679 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.679 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.679 INFO:tasks.workunit.client.0.smithi100.stdout: "lease_ack_timeout": { 2024-04-07T17:51:23.679 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.679 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.679 INFO:tasks.workunit.client.0.smithi100.stdout: "lease_timeout": { 2024-04-07T17:51:23.679 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.679 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.679 INFO:tasks.workunit.client.0.smithi100.stdout: "new_pn": { 2024-04-07T17:51:23.679 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.679 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.680 INFO:tasks.workunit.client.0.smithi100.stdout: "new_pn_latency": { 2024-04-07T17:51:23.680 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:23.680 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.680 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.680 INFO:tasks.workunit.client.0.smithi100.stdout: "refresh": { 2024-04-07T17:51:23.680 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 8738 2024-04-07T17:51:23.680 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.680 INFO:tasks.workunit.client.0.smithi100.stdout: "refresh_latency": { 2024-04-07T17:51:23.680 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 8738, 2024-04-07T17:51:23.680 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 48943127324 2024-04-07T17:51:23.680 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.680 INFO:tasks.workunit.client.0.smithi100.stdout: "restart": { 2024-04-07T17:51:23.680 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 9 2024-04-07T17:51:23.680 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.680 INFO:tasks.workunit.client.0.smithi100.stdout: "share_state": { 2024-04-07T17:51:23.680 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.681 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.681 INFO:tasks.workunit.client.0.smithi100.stdout: "share_state_bytes": { 2024-04-07T17:51:23.681 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:23.681 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.681 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.681 INFO:tasks.workunit.client.0.smithi100.stdout: "share_state_keys": { 2024-04-07T17:51:23.681 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:23.681 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.681 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.681 INFO:tasks.workunit.client.0.smithi100.stdout: "start_leader": { 2024-04-07T17:51:23.681 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.681 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.681 INFO:tasks.workunit.client.0.smithi100.stdout: "start_peon": { 2024-04-07T17:51:23.682 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2 2024-04-07T17:51:23.682 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.682 INFO:tasks.workunit.client.0.smithi100.stdout: "store_state": { 2024-04-07T17:51:23.682 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 8738 2024-04-07T17:51:23.682 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.682 INFO:tasks.workunit.client.0.smithi100.stdout: "store_state_bytes": { 2024-04-07T17:51:23.682 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 8738, 2024-04-07T17:51:23.682 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 116035468 2024-04-07T17:51:23.682 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.682 INFO:tasks.workunit.client.0.smithi100.stdout: "store_state_keys": { 2024-04-07T17:51:23.682 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 8738, 2024-04-07T17:51:23.682 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 75603 2024-04-07T17:51:23.682 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.683 INFO:tasks.workunit.client.0.smithi100.stdout: "store_state_latency": { 2024-04-07T17:51:23.683 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 8738, 2024-04-07T17:51:23.683 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3183943365 2024-04-07T17:51:23.683 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:23.683 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.683 INFO:tasks.workunit.client.0.smithi100.stdout: "prioritycache": { 2024-04-07T17:51:23.683 INFO:tasks.workunit.client.0.smithi100.stdout: "cache_bytes": { 2024-04-07T17:51:23.683 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1020054731 2024-04-07T17:51:23.683 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.683 INFO:tasks.workunit.client.0.smithi100.stdout: "heap_bytes": { 2024-04-07T17:51:23.683 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 164585472 2024-04-07T17:51:23.683 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.683 INFO:tasks.workunit.client.0.smithi100.stdout: "mapped_bytes": { 2024-04-07T17:51:23.683 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 162013184 2024-04-07T17:51:23.683 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.683 INFO:tasks.workunit.client.0.smithi100.stdout: "target_bytes": { 2024-04-07T17:51:23.684 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2147483648 2024-04-07T17:51:23.684 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.684 INFO:tasks.workunit.client.0.smithi100.stdout: "unmapped_bytes": { 2024-04-07T17:51:23.684 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2572288 2024-04-07T17:51:23.684 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:23.684 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.684 INFO:tasks.workunit.client.0.smithi100.stdout: "prioritycache:full": { 2024-04-07T17:51:23.684 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:23.684 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 360710144 2024-04-07T17:51:23.684 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.684 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:23.684 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.684 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.684 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:23.684 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.685 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.685 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:23.685 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 267855893 2024-04-07T17:51:23.685 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.685 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:23.685 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 23328861 2024-04-07T17:51:23.685 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.685 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:23.685 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.685 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.685 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:23.685 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.685 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.685 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:23.685 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.686 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.686 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:23.686 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.686 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.686 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:23.686 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.686 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.686 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:23.686 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.686 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.686 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:23.686 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.686 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.686 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:23.687 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.687 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.687 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:23.687 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 69525390 2024-04-07T17:51:23.687 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:23.687 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.687 INFO:tasks.workunit.client.0.smithi100.stdout: "prioritycache:inc": { 2024-04-07T17:51:23.687 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:23.687 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 339738624 2024-04-07T17:51:23.687 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.687 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:23.687 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.687 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.687 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:23.687 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.688 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.688 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:23.688 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 267855893 2024-04-07T17:51:23.688 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.688 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:23.688 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1425119 2024-04-07T17:51:23.688 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.688 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:23.688 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.688 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.688 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:23.688 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.688 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.688 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:23.688 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.689 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.689 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:23.689 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.689 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.689 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:23.689 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.689 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.689 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:23.689 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.689 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.689 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:23.689 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.689 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.689 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:23.689 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.689 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.690 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:23.690 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 70457612 2024-04-07T17:51:23.690 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:23.690 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.690 INFO:tasks.workunit.client.0.smithi100.stdout: "prioritycache:kv": { 2024-04-07T17:51:23.690 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:23.690 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 314572800 2024-04-07T17:51:23.690 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.690 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:23.690 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 67108864 2024-04-07T17:51:23.690 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.690 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:23.690 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.690 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.690 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:23.691 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 178570595 2024-04-07T17:51:23.691 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.691 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:23.691 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.691 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.691 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:23.691 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.691 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.691 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:23.691 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.691 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.691 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:23.691 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.691 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.691 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:23.692 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.692 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.692 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:23.692 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.692 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.692 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:23.692 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.692 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.692 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:23.692 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.692 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.692 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:23.692 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.692 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.692 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:23.693 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 68893341 2024-04-07T17:51:23.693 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:23.693 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.693 INFO:tasks.workunit.client.0.smithi100.stdout: "rocksdb": { 2024-04-07T17:51:23.693 INFO:tasks.workunit.client.0.smithi100.stdout: "compact": { 2024-04-07T17:51:23.693 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.693 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.693 INFO:tasks.workunit.client.0.smithi100.stdout: "compact_queue_len": { 2024-04-07T17:51:23.693 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.693 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.693 INFO:tasks.workunit.client.0.smithi100.stdout: "compact_queue_merge": { 2024-04-07T17:51:23.693 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.693 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.693 INFO:tasks.workunit.client.0.smithi100.stdout: "compact_range": { 2024-04-07T17:51:23.694 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 101 2024-04-07T17:51:23.694 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.694 INFO:tasks.workunit.client.0.smithi100.stdout: "get_latency": { 2024-04-07T17:51:23.694 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 378157, 2024-04-07T17:51:23.694 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3964885955 2024-04-07T17:51:23.694 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.694 INFO:tasks.workunit.client.0.smithi100.stdout: "rocksdb_write_delay_time": { 2024-04-07T17:51:23.694 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:23.694 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.694 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.694 INFO:tasks.workunit.client.0.smithi100.stdout: "rocksdb_write_memtable_time": { 2024-04-07T17:51:23.694 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:23.694 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.695 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.695 INFO:tasks.workunit.client.0.smithi100.stdout: "rocksdb_write_pre_and_post_time": { 2024-04-07T17:51:23.695 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:23.695 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.695 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.695 INFO:tasks.workunit.client.0.smithi100.stdout: "rocksdb_write_wal_time": { 2024-04-07T17:51:23.695 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:23.695 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.695 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.695 INFO:tasks.workunit.client.0.smithi100.stdout: "submit_latency": { 2024-04-07T17:51:23.695 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:23.695 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.695 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.695 INFO:tasks.workunit.client.0.smithi100.stdout: "submit_sync_latency": { 2024-04-07T17:51:23.696 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 20955, 2024-04-07T17:51:23.696 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 8051722523 2024-04-07T17:51:23.696 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:23.696 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:23.696 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.696 INFO:tasks.workunit.client.0.smithi100.stdout: "mon.f47a4ff15a45950a63160265564a123fc5da5e5e": { 2024-04-07T17:51:23.696 INFO:tasks.workunit.client.0.smithi100.stdout: "mon": { 2024-04-07T17:51:23.696 INFO:tasks.workunit.client.0.smithi100.stdout: "election_call": { 2024-04-07T17:51:23.696 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5 2024-04-07T17:51:23.696 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.696 INFO:tasks.workunit.client.0.smithi100.stdout: "election_lose": { 2024-04-07T17:51:23.696 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.696 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.696 INFO:tasks.workunit.client.0.smithi100.stdout: "election_win": { 2024-04-07T17:51:23.697 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5 2024-04-07T17:51:23.697 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.697 INFO:tasks.workunit.client.0.smithi100.stdout: "num_elections": { 2024-04-07T17:51:23.697 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 17 2024-04-07T17:51:23.697 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.697 INFO:tasks.workunit.client.0.smithi100.stdout: "num_sessions": { 2024-04-07T17:51:23.697 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 11 2024-04-07T17:51:23.697 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.697 INFO:tasks.workunit.client.0.smithi100.stdout: "session_add": { 2024-04-07T17:51:23.697 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5 2024-04-07T17:51:23.697 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.697 INFO:tasks.workunit.client.0.smithi100.stdout: "session_rm": { 2024-04-07T17:51:23.697 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 278 2024-04-07T17:51:23.697 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.697 INFO:tasks.workunit.client.0.smithi100.stdout: "session_trim": { 2024-04-07T17:51:23.698 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.698 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:23.698 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.698 INFO:tasks.workunit.client.0.smithi100.stdout: "paxos": { 2024-04-07T17:51:23.698 INFO:tasks.workunit.client.0.smithi100.stdout: "accept_timeout": { 2024-04-07T17:51:23.698 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.698 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.698 INFO:tasks.workunit.client.0.smithi100.stdout: "begin": { 2024-04-07T17:51:23.698 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 8891 2024-04-07T17:51:23.698 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.698 INFO:tasks.workunit.client.0.smithi100.stdout: "begin_bytes": { 2024-04-07T17:51:23.698 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 8891, 2024-04-07T17:51:23.698 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 60665253 2024-04-07T17:51:23.699 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.699 INFO:tasks.workunit.client.0.smithi100.stdout: "begin_keys": { 2024-04-07T17:51:23.699 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 8891, 2024-04-07T17:51:23.699 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 26673 2024-04-07T17:51:23.699 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.699 INFO:tasks.workunit.client.0.smithi100.stdout: "begin_latency": { 2024-04-07T17:51:23.699 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 8891, 2024-04-07T17:51:23.699 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2895918242 2024-04-07T17:51:23.699 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.699 INFO:tasks.workunit.client.0.smithi100.stdout: "collect": { 2024-04-07T17:51:23.699 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.699 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.700 INFO:tasks.workunit.client.0.smithi100.stdout: "collect_bytes": { 2024-04-07T17:51:23.700 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:23.700 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.700 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.700 INFO:tasks.workunit.client.0.smithi100.stdout: "collect_keys": { 2024-04-07T17:51:23.700 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:23.700 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.700 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.700 INFO:tasks.workunit.client.0.smithi100.stdout: "collect_latency": { 2024-04-07T17:51:23.700 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:23.700 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.700 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.700 INFO:tasks.workunit.client.0.smithi100.stdout: "collect_timeout": { 2024-04-07T17:51:23.701 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.701 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.701 INFO:tasks.workunit.client.0.smithi100.stdout: "collect_uncommitted": { 2024-04-07T17:51:23.701 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2 2024-04-07T17:51:23.701 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.701 INFO:tasks.workunit.client.0.smithi100.stdout: "commit": { 2024-04-07T17:51:23.701 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 8889 2024-04-07T17:51:23.701 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.701 INFO:tasks.workunit.client.0.smithi100.stdout: "commit_bytes": { 2024-04-07T17:51:23.701 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 8889, 2024-04-07T17:51:23.702 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 59997923 2024-04-07T17:51:23.702 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.702 INFO:tasks.workunit.client.0.smithi100.stdout: "commit_keys": { 2024-04-07T17:51:23.702 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 8889, 2024-04-07T17:51:23.702 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 68421 2024-04-07T17:51:23.702 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.702 INFO:tasks.workunit.client.0.smithi100.stdout: "commit_latency": { 2024-04-07T17:51:23.702 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 8889, 2024-04-07T17:51:23.702 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3946227492 2024-04-07T17:51:23.702 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.702 INFO:tasks.workunit.client.0.smithi100.stdout: "lease_ack_timeout": { 2024-04-07T17:51:23.702 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.703 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.703 INFO:tasks.workunit.client.0.smithi100.stdout: "lease_timeout": { 2024-04-07T17:51:23.703 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.703 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.703 INFO:tasks.workunit.client.0.smithi100.stdout: "new_pn": { 2024-04-07T17:51:23.703 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5 2024-04-07T17:51:23.703 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.703 INFO:tasks.workunit.client.0.smithi100.stdout: "new_pn_latency": { 2024-04-07T17:51:23.703 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 5, 2024-04-07T17:51:23.703 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.703 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.703 INFO:tasks.workunit.client.0.smithi100.stdout: "refresh": { 2024-04-07T17:51:23.704 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 8892 2024-04-07T17:51:23.704 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.704 INFO:tasks.workunit.client.0.smithi100.stdout: "refresh_latency": { 2024-04-07T17:51:23.704 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 8892, 2024-04-07T17:51:23.704 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 56214415654 2024-04-07T17:51:23.704 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.704 INFO:tasks.workunit.client.0.smithi100.stdout: "restart": { 2024-04-07T17:51:23.704 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 19 2024-04-07T17:51:23.704 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.704 INFO:tasks.workunit.client.0.smithi100.stdout: "share_state": { 2024-04-07T17:51:23.705 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2 2024-04-07T17:51:23.705 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.705 INFO:tasks.workunit.client.0.smithi100.stdout: "share_state_bytes": { 2024-04-07T17:51:23.705 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 2, 2024-04-07T17:51:23.705 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 129358 2024-04-07T17:51:23.705 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.705 INFO:tasks.workunit.client.0.smithi100.stdout: "share_state_keys": { 2024-04-07T17:51:23.705 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 2, 2024-04-07T17:51:23.705 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 10 2024-04-07T17:51:23.705 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.706 INFO:tasks.workunit.client.0.smithi100.stdout: "start_leader": { 2024-04-07T17:51:23.706 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5 2024-04-07T17:51:23.706 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.706 INFO:tasks.workunit.client.0.smithi100.stdout: "start_peon": { 2024-04-07T17:51:23.706 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.706 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.706 INFO:tasks.workunit.client.0.smithi100.stdout: "store_state": { 2024-04-07T17:51:23.706 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.707 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.707 INFO:tasks.workunit.client.0.smithi100.stdout: "store_state_bytes": { 2024-04-07T17:51:23.707 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:23.707 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.707 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.707 INFO:tasks.workunit.client.0.smithi100.stdout: "store_state_keys": { 2024-04-07T17:51:23.708 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:23.708 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.708 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.708 INFO:tasks.workunit.client.0.smithi100.stdout: "store_state_latency": { 2024-04-07T17:51:23.708 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:23.708 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.708 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:23.708 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.709 INFO:tasks.workunit.client.0.smithi100.stdout: "prioritycache": { 2024-04-07T17:51:23.709 INFO:tasks.workunit.client.0.smithi100.stdout: "cache_bytes": { 2024-04-07T17:51:23.709 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1020054731 2024-04-07T17:51:23.709 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.709 INFO:tasks.workunit.client.0.smithi100.stdout: "heap_bytes": { 2024-04-07T17:51:23.710 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 258113536 2024-04-07T17:51:23.710 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.710 INFO:tasks.workunit.client.0.smithi100.stdout: "mapped_bytes": { 2024-04-07T17:51:23.711 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 257466368 2024-04-07T17:51:23.711 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.711 INFO:tasks.workunit.client.0.smithi100.stdout: "target_bytes": { 2024-04-07T17:51:23.712 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2147483648 2024-04-07T17:51:23.712 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.712 INFO:tasks.workunit.client.0.smithi100.stdout: "unmapped_bytes": { 2024-04-07T17:51:23.712 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 647168 2024-04-07T17:51:23.712 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:23.715 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.715 INFO:tasks.workunit.client.0.smithi100.stdout: "prioritycache:full": { 2024-04-07T17:51:23.715 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:23.716 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 360710144 2024-04-07T17:51:23.718 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.718 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:23.718 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.718 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.719 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:23.719 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.719 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.719 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:23.719 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 267670953 2024-04-07T17:51:23.719 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.719 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:23.719 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 23788534 2024-04-07T17:51:23.719 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.719 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:23.719 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.720 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.720 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:23.720 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.720 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.720 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:23.720 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.720 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.720 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:23.720 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.720 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.720 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:23.720 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.720 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.720 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:23.721 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.721 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.721 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:23.721 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.721 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.721 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:23.721 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.721 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.721 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:23.721 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 69250657 2024-04-07T17:51:23.721 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:23.721 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.721 INFO:tasks.workunit.client.0.smithi100.stdout: "prioritycache:inc": { 2024-04-07T17:51:23.722 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:23.722 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 339738624 2024-04-07T17:51:23.722 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.722 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:23.722 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.722 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.722 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:23.722 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.723 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.723 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:23.723 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 267670953 2024-04-07T17:51:23.723 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.723 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:23.723 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1458619 2024-04-07T17:51:23.723 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.723 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:23.723 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.723 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.724 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:23.724 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.724 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.724 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:23.724 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.724 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.724 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:23.724 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.724 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.725 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:23.725 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.725 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.725 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:23.725 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.725 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.725 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:23.726 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.726 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.726 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:23.726 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.726 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.726 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:23.726 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 70609052 2024-04-07T17:51:23.726 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:23.727 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.727 INFO:tasks.workunit.client.0.smithi100.stdout: "prioritycache:kv": { 2024-04-07T17:51:23.727 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:23.727 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 314572800 2024-04-07T17:51:23.727 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.727 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:23.727 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 67108864 2024-04-07T17:51:23.728 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.728 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:23.728 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.728 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.728 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:23.728 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 178447302 2024-04-07T17:51:23.728 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.728 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:23.728 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.728 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.729 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:23.729 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.729 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.729 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:23.729 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.729 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.729 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:23.729 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.729 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.729 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:23.729 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.730 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.730 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:23.730 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.730 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.730 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:23.730 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.730 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.731 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:23.731 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.731 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.731 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:23.734 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.734 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.734 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:23.734 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 69016634 2024-04-07T17:51:23.734 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:23.734 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.734 INFO:tasks.workunit.client.0.smithi100.stdout: "rocksdb": { 2024-04-07T17:51:23.734 INFO:tasks.workunit.client.0.smithi100.stdout: "compact": { 2024-04-07T17:51:23.734 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.735 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.735 INFO:tasks.workunit.client.0.smithi100.stdout: "compact_queue_len": { 2024-04-07T17:51:23.735 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.735 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.735 INFO:tasks.workunit.client.0.smithi100.stdout: "compact_queue_merge": { 2024-04-07T17:51:23.735 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.735 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.735 INFO:tasks.workunit.client.0.smithi100.stdout: "compact_range": { 2024-04-07T17:51:23.735 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 105 2024-04-07T17:51:23.735 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.735 INFO:tasks.workunit.client.0.smithi100.stdout: "get_latency": { 2024-04-07T17:51:23.735 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 398669, 2024-04-07T17:51:23.735 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4722518373 2024-04-07T17:51:23.736 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.736 INFO:tasks.workunit.client.0.smithi100.stdout: "rocksdb_write_delay_time": { 2024-04-07T17:51:23.736 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:23.736 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.736 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.736 INFO:tasks.workunit.client.0.smithi100.stdout: "rocksdb_write_memtable_time": { 2024-04-07T17:51:23.736 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:23.736 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.736 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.736 INFO:tasks.workunit.client.0.smithi100.stdout: "rocksdb_write_pre_and_post_time": { 2024-04-07T17:51:23.736 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:23.737 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.737 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.737 INFO:tasks.workunit.client.0.smithi100.stdout: "rocksdb_write_wal_time": { 2024-04-07T17:51:23.737 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:23.737 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.737 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.737 INFO:tasks.workunit.client.0.smithi100.stdout: "submit_latency": { 2024-04-07T17:51:23.737 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:23.737 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.737 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.737 INFO:tasks.workunit.client.0.smithi100.stdout: "submit_sync_latency": { 2024-04-07T17:51:23.737 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 21335, 2024-04-07T17:51:23.738 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 8939358365 2024-04-07T17:51:23.738 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:23.738 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:23.738 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.738 INFO:tasks.workunit.client.0.smithi100.stdout: "osd.0": { 2024-04-07T17:51:23.738 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs": { 2024-04-07T17:51:23.738 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_db_max_lat": { 2024-04-07T17:51:23.738 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 21463 2024-04-07T17:51:23.738 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.738 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_slow_fallback": { 2024-04-07T17:51:23.739 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.739 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.739 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_slow_max_lat": { 2024-04-07T17:51:23.739 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.739 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.739 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_slow_size_fallback": { 2024-04-07T17:51:23.739 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.739 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.739 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit_db": { 2024-04-07T17:51:23.739 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 65536 2024-04-07T17:51:23.739 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.739 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit_slow": { 2024-04-07T17:51:23.739 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.740 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.740 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit_wal": { 2024-04-07T17:51:23.740 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.740 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.740 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_wal_max_lat": { 2024-04-07T17:51:23.740 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.740 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.740 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_written_slow": { 2024-04-07T17:51:23.740 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.740 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.741 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_written_sst": { 2024-04-07T17:51:23.741 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 14692352 2024-04-07T17:51:23.741 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.741 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_written_wal": { 2024-04-07T17:51:23.741 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 742674432 2024-04-07T17:51:23.741 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.741 INFO:tasks.workunit.client.0.smithi100.stdout: "compact_lat": { 2024-04-07T17:51:23.741 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 19, 2024-04-07T17:51:23.741 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 7885793 2024-04-07T17:51:23.741 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.742 INFO:tasks.workunit.client.0.smithi100.stdout: "compact_lock_lat": { 2024-04-07T17:51:23.742 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 19, 2024-04-07T17:51:23.742 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2646042 2024-04-07T17:51:23.742 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.742 INFO:tasks.workunit.client.0.smithi100.stdout: "db_alloc_lat": { 2024-04-07T17:51:23.742 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 211, 2024-04-07T17:51:23.742 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1477946 2024-04-07T17:51:23.742 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.742 INFO:tasks.workunit.client.0.smithi100.stdout: "db_total_bytes": { 2024-04-07T17:51:23.742 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 95995027456 2024-04-07T17:51:23.742 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.742 INFO:tasks.workunit.client.0.smithi100.stdout: "db_used_bytes": { 2024-04-07T17:51:23.742 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 735182848 2024-04-07T17:51:23.743 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.743 INFO:tasks.workunit.client.0.smithi100.stdout: "flush_lat": { 2024-04-07T17:51:23.743 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 79477, 2024-04-07T17:51:23.743 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 14955834078 2024-04-07T17:51:23.743 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.743 INFO:tasks.workunit.client.0.smithi100.stdout: "fsync_lat": { 2024-04-07T17:51:23.743 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 79485, 2024-04-07T17:51:23.743 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 25864611168 2024-04-07T17:51:23.743 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.743 INFO:tasks.workunit.client.0.smithi100.stdout: "log_bytes": { 2024-04-07T17:51:23.744 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 9666560 2024-04-07T17:51:23.744 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.744 INFO:tasks.workunit.client.0.smithi100.stdout: "logged_bytes": { 2024-04-07T17:51:23.744 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 325562368 2024-04-07T17:51:23.744 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.744 INFO:tasks.workunit.client.0.smithi100.stdout: "max_bytes_db": { 2024-04-07T17:51:23.744 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 747962368 2024-04-07T17:51:23.745 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.745 INFO:tasks.workunit.client.0.smithi100.stdout: "max_bytes_slow": { 2024-04-07T17:51:23.745 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.745 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.745 INFO:tasks.workunit.client.0.smithi100.stdout: "max_bytes_wal": { 2024-04-07T17:51:23.745 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.746 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.746 INFO:tasks.workunit.client.0.smithi100.stdout: "num_files": { 2024-04-07T17:51:23.746 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 55 2024-04-07T17:51:23.746 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.746 INFO:tasks.workunit.client.0.smithi100.stdout: "read_bytes": { 2024-04-07T17:51:23.746 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 13112107291 2024-04-07T17:51:23.746 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.746 INFO:tasks.workunit.client.0.smithi100.stdout: "read_count": { 2024-04-07T17:51:23.747 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 86948 2024-04-07T17:51:23.747 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.747 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes": { 2024-04-07T17:51:23.748 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 11361755136 2024-04-07T17:51:23.748 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.748 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes_db": { 2024-04-07T17:51:23.748 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 11361759232 2024-04-07T17:51:23.748 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.748 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes_slow": { 2024-04-07T17:51:23.748 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.749 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.749 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes_wal": { 2024-04-07T17:51:23.749 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.749 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.749 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_count": { 2024-04-07T17:51:23.749 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 88640 2024-04-07T17:51:23.749 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.750 INFO:tasks.workunit.client.0.smithi100.stdout: "read_lat": { 2024-04-07T17:51:23.750 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 86948, 2024-04-07T17:51:23.750 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5555540195 2024-04-07T17:51:23.750 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.750 INFO:tasks.workunit.client.0.smithi100.stdout: "read_prefetch_bytes": { 2024-04-07T17:51:23.750 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 12971063490 2024-04-07T17:51:23.750 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.750 INFO:tasks.workunit.client.0.smithi100.stdout: "read_prefetch_count": { 2024-04-07T17:51:23.750 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 82627 2024-04-07T17:51:23.751 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.751 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_buffer_bytes": { 2024-04-07T17:51:23.751 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5154118 2024-04-07T17:51:23.751 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.751 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_buffer_count": { 2024-04-07T17:51:23.751 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 723 2024-04-07T17:51:23.751 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.751 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_bytes": { 2024-04-07T17:51:23.751 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 14619250 2024-04-07T17:51:23.751 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.751 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_count": { 2024-04-07T17:51:23.751 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1029 2024-04-07T17:51:23.752 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.752 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes": { 2024-04-07T17:51:23.752 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 9465132 2024-04-07T17:51:23.752 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.752 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes_db": { 2024-04-07T17:51:23.752 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 9465132 2024-04-07T17:51:23.752 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.752 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes_slow": { 2024-04-07T17:51:23.752 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.753 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.753 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes_wal": { 2024-04-07T17:51:23.753 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.753 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.753 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_count": { 2024-04-07T17:51:23.753 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 321 2024-04-07T17:51:23.753 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.754 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_lat": { 2024-04-07T17:51:23.754 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1029, 2024-04-07T17:51:23.754 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 7138854 2024-04-07T17:51:23.754 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.754 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_alloc_lat": { 2024-04-07T17:51:23.754 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:23.754 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.754 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.755 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_total_bytes": { 2024-04-07T17:51:23.755 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.755 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.755 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_used_bytes": { 2024-04-07T17:51:23.755 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.755 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.755 INFO:tasks.workunit.client.0.smithi100.stdout: "truncate_lat": { 2024-04-07T17:51:23.756 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1, 2024-04-07T17:51:23.756 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 593 2024-04-07T17:51:23.756 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.756 INFO:tasks.workunit.client.0.smithi100.stdout: "unlink_lat": { 2024-04-07T17:51:23.756 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 11, 2024-04-07T17:51:23.756 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 14997 2024-04-07T17:51:23.756 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.756 INFO:tasks.workunit.client.0.smithi100.stdout: "wal_alloc_lat": { 2024-04-07T17:51:23.757 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:23.757 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.757 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.757 INFO:tasks.workunit.client.0.smithi100.stdout: "wal_total_bytes": { 2024-04-07T17:51:23.757 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.757 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.757 INFO:tasks.workunit.client.0.smithi100.stdout: "wal_used_bytes": { 2024-04-07T17:51:23.757 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.757 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.758 INFO:tasks.workunit.client.0.smithi100.stdout: "write_bytes": { 2024-04-07T17:51:23.758 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1085816832 2024-04-07T17:51:23.758 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:23.758 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.758 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore": { 2024-04-07T17:51:23.759 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit": { 2024-04-07T17:51:23.759 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4096 2024-04-07T17:51:23.759 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.759 INFO:tasks.workunit.client.0.smithi100.stdout: "allocated": { 2024-04-07T17:51:23.759 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 10412032 2024-04-07T17:51:23.759 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.759 INFO:tasks.workunit.client.0.smithi100.stdout: "allocator_lat": { 2024-04-07T17:51:23.760 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 23588, 2024-04-07T17:51:23.760 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 122707682 2024-04-07T17:51:23.760 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.760 INFO:tasks.workunit.client.0.smithi100.stdout: "clist_lat": { 2024-04-07T17:51:23.760 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 3594, 2024-04-07T17:51:23.760 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 12445185285 2024-04-07T17:51:23.760 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.760 INFO:tasks.workunit.client.0.smithi100.stdout: "compress_lat": { 2024-04-07T17:51:23.760 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:23.760 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.761 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.761 INFO:tasks.workunit.client.0.smithi100.stdout: "compressed": { 2024-04-07T17:51:23.761 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.761 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.761 INFO:tasks.workunit.client.0.smithi100.stdout: "compressed_allocated": { 2024-04-07T17:51:23.761 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.761 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.762 INFO:tasks.workunit.client.0.smithi100.stdout: "compressed_original": { 2024-04-07T17:51:23.762 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.762 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.762 INFO:tasks.workunit.client.0.smithi100.stdout: "csum_lat": { 2024-04-07T17:51:23.762 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 29723, 2024-04-07T17:51:23.762 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 345925142 2024-04-07T17:51:23.763 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.763 INFO:tasks.workunit.client.0.smithi100.stdout: "decompress_lat": { 2024-04-07T17:51:23.763 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:23.763 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.763 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.763 INFO:tasks.workunit.client.0.smithi100.stdout: "fragmentation_micros": { 2024-04-07T17:51:23.763 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.763 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.763 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_commit_lat": { 2024-04-07T17:51:23.763 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 79403, 2024-04-07T17:51:23.763 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 33855770677 2024-04-07T17:51:23.764 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.764 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_final_lat": { 2024-04-07T17:51:23.764 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 78455, 2024-04-07T17:51:23.764 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3723540743 2024-04-07T17:51:23.764 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.764 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_flush_lat": { 2024-04-07T17:51:23.764 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 79403, 2024-04-07T17:51:23.764 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 104760188 2024-04-07T17:51:23.764 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.765 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_sync_lat": { 2024-04-07T17:51:23.765 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 79403, 2024-04-07T17:51:23.765 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 33960530865 2024-04-07T17:51:23.765 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.765 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_get_keys_lat": { 2024-04-07T17:51:23.765 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:23.765 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.765 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.765 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_get_values_lat": { 2024-04-07T17:51:23.765 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 35913, 2024-04-07T17:51:23.765 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1906667723 2024-04-07T17:51:23.766 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.766 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_lower_bound_lat": { 2024-04-07T17:51:23.766 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 941, 2024-04-07T17:51:23.766 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5755773 2024-04-07T17:51:23.766 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.766 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_next_lat": { 2024-04-07T17:51:23.766 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 24954, 2024-04-07T17:51:23.766 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 21515897 2024-04-07T17:51:23.766 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.766 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_seek_to_first_lat": { 2024-04-07T17:51:23.766 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 52, 2024-04-07T17:51:23.767 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 102213 2024-04-07T17:51:23.767 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.767 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_upper_bound_lat": { 2024-04-07T17:51:23.767 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 16665, 2024-04-07T17:51:23.767 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5746422318 2024-04-07T17:51:23.767 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.767 INFO:tasks.workunit.client.0.smithi100.stdout: "onode_hits": { 2024-04-07T17:51:23.767 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 260474 2024-04-07T17:51:23.767 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.767 INFO:tasks.workunit.client.0.smithi100.stdout: "onode_misses": { 2024-04-07T17:51:23.767 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 24792 2024-04-07T17:51:23.767 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.767 INFO:tasks.workunit.client.0.smithi100.stdout: "read_lat": { 2024-04-07T17:51:23.768 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 26968, 2024-04-07T17:51:23.768 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3799445403 2024-04-07T17:51:23.768 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.768 INFO:tasks.workunit.client.0.smithi100.stdout: "read_onode_meta_lat": { 2024-04-07T17:51:23.768 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 62381, 2024-04-07T17:51:23.768 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 46963984 2024-04-07T17:51:23.769 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.769 INFO:tasks.workunit.client.0.smithi100.stdout: "read_wait_aio_lat": { 2024-04-07T17:51:23.769 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 35413, 2024-04-07T17:51:23.769 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2436705902 2024-04-07T17:51:23.769 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.769 INFO:tasks.workunit.client.0.smithi100.stdout: "reads_with_retries": { 2024-04-07T17:51:23.769 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.770 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.770 INFO:tasks.workunit.client.0.smithi100.stdout: "remove_lat": { 2024-04-07T17:51:23.770 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 15389, 2024-04-07T17:51:23.770 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 819542013 2024-04-07T17:51:23.770 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.770 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_aio_wait_count": { 2024-04-07T17:51:23.770 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.771 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.771 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_committed_kv_count": { 2024-04-07T17:51:23.771 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.771 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.771 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_read_onode_meta_count": { 2024-04-07T17:51:23.771 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.772 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.772 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_read_wait_aio_count": { 2024-04-07T17:51:23.772 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.772 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.772 INFO:tasks.workunit.client.0.smithi100.stdout: "state_aio_wait_lat": { 2024-04-07T17:51:23.772 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 93393, 2024-04-07T17:51:23.772 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 12329349719 2024-04-07T17:51:23.772 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.773 INFO:tasks.workunit.client.0.smithi100.stdout: "state_deferred_aio_wait_lat": { 2024-04-07T17:51:23.773 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 9754, 2024-04-07T17:51:23.773 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1209751248 2024-04-07T17:51:23.773 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.773 INFO:tasks.workunit.client.0.smithi100.stdout: "state_deferred_cleanup_lat": { 2024-04-07T17:51:23.773 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 9753, 2024-04-07T17:51:23.773 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4213165502302 2024-04-07T17:51:23.773 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.773 INFO:tasks.workunit.client.0.smithi100.stdout: "state_deferred_queued_lat": { 2024-04-07T17:51:23.773 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 9754, 2024-04-07T17:51:23.773 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 6800803572839 2024-04-07T17:51:23.773 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.774 INFO:tasks.workunit.client.0.smithi100.stdout: "state_done_lat": { 2024-04-07T17:51:23.774 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 93392, 2024-04-07T17:51:23.774 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2914483870678 2024-04-07T17:51:23.774 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.774 INFO:tasks.workunit.client.0.smithi100.stdout: "state_finishing_lat": { 2024-04-07T17:51:23.774 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 93392, 2024-04-07T17:51:23.774 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 20544100 2024-04-07T17:51:23.774 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.775 INFO:tasks.workunit.client.0.smithi100.stdout: "state_io_done_lat": { 2024-04-07T17:51:23.775 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 93393, 2024-04-07T17:51:23.775 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 589384870 2024-04-07T17:51:23.775 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.775 INFO:tasks.workunit.client.0.smithi100.stdout: "state_kv_commiting_lat": { 2024-04-07T17:51:23.775 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 93393, 2024-04-07T17:51:23.775 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 60184889904 2024-04-07T17:51:23.775 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.775 INFO:tasks.workunit.client.0.smithi100.stdout: "state_kv_done_lat": { 2024-04-07T17:51:23.775 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 93393, 2024-04-07T17:51:23.775 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 69530872 2024-04-07T17:51:23.776 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.776 INFO:tasks.workunit.client.0.smithi100.stdout: "state_kv_queued_lat": { 2024-04-07T17:51:23.776 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 93393, 2024-04-07T17:51:23.776 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 31858207932 2024-04-07T17:51:23.776 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.776 INFO:tasks.workunit.client.0.smithi100.stdout: "state_prepare_lat": { 2024-04-07T17:51:23.776 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 93393, 2024-04-07T17:51:23.777 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 26654236951 2024-04-07T17:51:23.777 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.777 INFO:tasks.workunit.client.0.smithi100.stdout: "stored": { 2024-04-07T17:51:23.777 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 6290778 2024-04-07T17:51:23.777 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.777 INFO:tasks.workunit.client.0.smithi100.stdout: "truncate_lat": { 2024-04-07T17:51:23.778 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 2482, 2024-04-07T17:51:23.778 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 25806717 2024-04-07T17:51:23.778 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.778 INFO:tasks.workunit.client.0.smithi100.stdout: "txc_commit_lat": { 2024-04-07T17:51:23.778 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 93393, 2024-04-07T17:51:23.778 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 131646585667 2024-04-07T17:51:23.779 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.779 INFO:tasks.workunit.client.0.smithi100.stdout: "txc_submit_lat": { 2024-04-07T17:51:23.779 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 93393, 2024-04-07T17:51:23.779 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 29197915299 2024-04-07T17:51:23.779 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.779 INFO:tasks.workunit.client.0.smithi100.stdout: "txc_throttle_lat": { 2024-04-07T17:51:23.779 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 93393, 2024-04-07T17:51:23.779 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1435528401 2024-04-07T17:51:23.779 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:23.779 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.779 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache": { 2024-04-07T17:51:23.780 INFO:tasks.workunit.client.0.smithi100.stdout: "cache_bytes": { 2024-04-07T17:51:23.780 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1444614092 2024-04-07T17:51:23.780 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.780 INFO:tasks.workunit.client.0.smithi100.stdout: "heap_bytes": { 2024-04-07T17:51:23.780 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 508690432 2024-04-07T17:51:23.780 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.780 INFO:tasks.workunit.client.0.smithi100.stdout: "mapped_bytes": { 2024-04-07T17:51:23.780 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 469770240 2024-04-07T17:51:23.781 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.781 INFO:tasks.workunit.client.0.smithi100.stdout: "target_bytes": { 2024-04-07T17:51:23.781 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2646965248 2024-04-07T17:51:23.781 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.781 INFO:tasks.workunit.client.0.smithi100.stdout: "unmapped_bytes": { 2024-04-07T17:51:23.781 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 38920192 2024-04-07T17:51:23.781 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:23.781 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.782 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:data": { 2024-04-07T17:51:23.782 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:23.782 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 134217728 2024-04-07T17:51:23.782 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.782 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:23.782 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.783 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.783 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:23.783 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.783 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.783 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:23.783 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 58795598 2024-04-07T17:51:23.783 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.783 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:23.784 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.784 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.784 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:23.784 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.784 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.784 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:23.784 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.784 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.784 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:23.784 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.785 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.785 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:23.785 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 12288 2024-04-07T17:51:23.785 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.785 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:23.785 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 430080 2024-04-07T17:51:23.785 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.785 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:23.786 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.786 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.786 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:23.786 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.786 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.786 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:23.786 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.786 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.786 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:23.786 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 74979762 2024-04-07T17:51:23.786 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:23.786 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.786 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:kv": { 2024-04-07T17:51:23.786 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:23.786 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 578813952 2024-04-07T17:51:23.787 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.787 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:23.787 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 67108864 2024-04-07T17:51:23.787 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.787 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:23.787 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.787 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.787 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:23.787 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 440966985 2024-04-07T17:51:23.787 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.787 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:23.787 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.787 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.787 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:23.787 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.788 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.788 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:23.788 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1776 2024-04-07T17:51:23.788 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.788 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:23.788 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.788 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.788 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:23.788 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 121696 2024-04-07T17:51:23.788 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.788 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:23.788 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.789 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.789 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:23.789 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.789 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.789 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:23.789 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.789 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.789 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:23.789 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.789 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.789 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:23.789 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 70614631 2024-04-07T17:51:23.789 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:23.789 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.789 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:kv_onode": { 2024-04-07T17:51:23.790 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:23.790 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 192937984 2024-04-07T17:51:23.790 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.790 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:23.790 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 67108864 2024-04-07T17:51:23.790 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.790 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:23.790 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 14485360 2024-04-07T17:51:23.790 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.790 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:23.790 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 39197065 2024-04-07T17:51:23.790 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.790 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:23.790 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.791 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.791 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:23.791 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.791 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.791 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:23.791 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.791 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.791 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:23.791 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.791 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.791 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:23.791 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.791 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.791 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:23.791 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.792 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.792 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:23.792 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.792 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.792 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:23.792 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.792 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.792 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:23.792 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.792 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.792 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:23.792 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 72146695 2024-04-07T17:51:23.792 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:23.792 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.793 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:meta": { 2024-04-07T17:51:23.793 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:23.793 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 511705088 2024-04-07T17:51:23.793 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.793 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:23.793 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.793 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.793 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:23.793 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.793 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.793 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:23.793 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 440966985 2024-04-07T17:51:23.793 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.793 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:23.793 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.793 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.794 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:23.794 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.794 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.794 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:23.794 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.794 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.794 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:23.794 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.794 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.794 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:23.794 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 161520 2024-04-07T17:51:23.794 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.794 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:23.794 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1089279 2024-04-07T17:51:23.795 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.795 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:23.795 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.795 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.795 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:23.795 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.795 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.795 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:23.795 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.795 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.795 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:23.795 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 69487304 2024-04-07T17:51:23.795 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:23.795 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.795 INFO:tasks.workunit.client.0.smithi100.stdout: "objecter": { 2024-04-07T17:51:23.796 INFO:tasks.workunit.client.0.smithi100.stdout: "op_active": { 2024-04-07T17:51:23.796 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.796 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.796 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r": { 2024-04-07T17:51:23.796 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 56 2024-04-07T17:51:23.796 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.796 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rmw": { 2024-04-07T17:51:23.796 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.796 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.796 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w": { 2024-04-07T17:51:23.796 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.796 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:23.796 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.796 INFO:tasks.workunit.client.0.smithi100.stdout: "osd": { 2024-04-07T17:51:23.796 INFO:tasks.workunit.client.0.smithi100.stdout: "numpg": { 2024-04-07T17:51:23.797 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 55 2024-04-07T17:51:23.797 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.797 INFO:tasks.workunit.client.0.smithi100.stdout: "numpg_removing": { 2024-04-07T17:51:23.797 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.797 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.797 INFO:tasks.workunit.client.0.smithi100.stdout: "op": { 2024-04-07T17:51:23.797 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 53260 2024-04-07T17:51:23.797 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.797 INFO:tasks.workunit.client.0.smithi100.stdout: "op_before_queue_op_lat": { 2024-04-07T17:51:23.797 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 138139, 2024-04-07T17:51:23.797 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 14238870516 2024-04-07T17:51:23.797 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.797 INFO:tasks.workunit.client.0.smithi100.stdout: "op_delayed_degraded": { 2024-04-07T17:51:23.797 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4 2024-04-07T17:51:23.797 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.798 INFO:tasks.workunit.client.0.smithi100.stdout: "op_delayed_unreadable": { 2024-04-07T17:51:23.798 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.798 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.798 INFO:tasks.workunit.client.0.smithi100.stdout: "op_in_bytes": { 2024-04-07T17:51:23.798 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2665625896 2024-04-07T17:51:23.798 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.798 INFO:tasks.workunit.client.0.smithi100.stdout: "op_latency": { 2024-04-07T17:51:23.798 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 53260, 2024-04-07T17:51:23.798 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 365460763820 2024-04-07T17:51:23.798 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.798 INFO:tasks.workunit.client.0.smithi100.stdout: "op_out_bytes": { 2024-04-07T17:51:23.798 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3752394298 2024-04-07T17:51:23.798 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.798 INFO:tasks.workunit.client.0.smithi100.stdout: "op_prepare_latency": { 2024-04-07T17:51:23.798 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 54135, 2024-04-07T17:51:23.799 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 47144202978 2024-04-07T17:51:23.799 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.799 INFO:tasks.workunit.client.0.smithi100.stdout: "op_process_latency": { 2024-04-07T17:51:23.799 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 53260, 2024-04-07T17:51:23.799 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 125632474360 2024-04-07T17:51:23.799 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.799 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r": { 2024-04-07T17:51:23.799 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 37072 2024-04-07T17:51:23.799 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.799 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_latency": { 2024-04-07T17:51:23.799 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 37072, 2024-04-07T17:51:23.799 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 32580468629 2024-04-07T17:51:23.799 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.799 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_latency_out_bytes_histogram": { 2024-04-07T17:51:23.799 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.800 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.800 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_out_bytes": { 2024-04-07T17:51:23.800 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3752394298 2024-04-07T17:51:23.800 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.800 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_prepare_latency": { 2024-04-07T17:51:23.800 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 37432, 2024-04-07T17:51:23.800 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 17128799481 2024-04-07T17:51:23.800 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.800 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_process_latency": { 2024-04-07T17:51:23.800 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 37072, 2024-04-07T17:51:23.800 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 17450603396 2024-04-07T17:51:23.800 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.800 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw": { 2024-04-07T17:51:23.800 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 8557 2024-04-07T17:51:23.800 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.801 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_in_bytes": { 2024-04-07T17:51:23.801 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 574680166 2024-04-07T17:51:23.801 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.801 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_latency": { 2024-04-07T17:51:23.801 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 8557, 2024-04-07T17:51:23.801 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 36381436318 2024-04-07T17:51:23.801 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.801 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_latency_in_bytes_histogram": { 2024-04-07T17:51:23.801 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.801 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.801 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_latency_out_bytes_histogram": { 2024-04-07T17:51:23.801 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.801 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.801 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_out_bytes": { 2024-04-07T17:51:23.801 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.801 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.802 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_prepare_latency": { 2024-04-07T17:51:23.802 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 8868, 2024-04-07T17:51:23.802 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 12409822934 2024-04-07T17:51:23.802 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.802 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_process_latency": { 2024-04-07T17:51:23.802 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 8557, 2024-04-07T17:51:23.802 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 31269983898 2024-04-07T17:51:23.802 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.802 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w": { 2024-04-07T17:51:23.802 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 7631 2024-04-07T17:51:23.802 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.802 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_in_bytes": { 2024-04-07T17:51:23.802 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2090945730 2024-04-07T17:51:23.802 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.802 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_latency": { 2024-04-07T17:51:23.803 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 7631, 2024-04-07T17:51:23.803 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 296498858873 2024-04-07T17:51:23.803 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.803 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_latency_in_bytes_histogram": { 2024-04-07T17:51:23.803 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.803 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.803 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_prepare_latency": { 2024-04-07T17:51:23.803 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 7835, 2024-04-07T17:51:23.803 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 17605580563 2024-04-07T17:51:23.803 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.803 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_process_latency": { 2024-04-07T17:51:23.803 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 7631, 2024-04-07T17:51:23.803 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 76911887066 2024-04-07T17:51:23.803 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.804 INFO:tasks.workunit.client.0.smithi100.stdout: "op_wip": { 2024-04-07T17:51:23.804 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.804 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.804 INFO:tasks.workunit.client.0.smithi100.stdout: "recovery_bytes": { 2024-04-07T17:51:23.804 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.804 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.804 INFO:tasks.workunit.client.0.smithi100.stdout: "recovery_ops": { 2024-04-07T17:51:23.804 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 26 2024-04-07T17:51:23.804 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.804 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_bytes": { 2024-04-07T17:51:23.804 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 95995035648 2024-04-07T17:51:23.804 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.804 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_bytes_used": { 2024-04-07T17:51:23.804 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 745607168 2024-04-07T17:51:23.804 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:23.804 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:23.805 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.805 INFO:tasks.workunit.client.0.smithi100.stdout: "osd.1": { 2024-04-07T17:51:23.805 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs": { 2024-04-07T17:51:23.805 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_db_max_lat": { 2024-04-07T17:51:23.805 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 30825 2024-04-07T17:51:23.805 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.805 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_slow_fallback": { 2024-04-07T17:51:23.805 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.805 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.805 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_slow_max_lat": { 2024-04-07T17:51:23.805 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.805 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.805 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_slow_size_fallback": { 2024-04-07T17:51:23.805 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.805 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.806 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit_db": { 2024-04-07T17:51:23.806 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 65536 2024-04-07T17:51:23.806 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.806 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit_slow": { 2024-04-07T17:51:23.806 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.806 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.806 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit_wal": { 2024-04-07T17:51:23.806 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.806 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.806 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_wal_max_lat": { 2024-04-07T17:51:23.806 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.806 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.806 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_written_slow": { 2024-04-07T17:51:23.806 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.806 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.807 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_written_sst": { 2024-04-07T17:51:23.807 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5402624 2024-04-07T17:51:23.807 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.807 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_written_wal": { 2024-04-07T17:51:23.807 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 723148800 2024-04-07T17:51:23.807 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.807 INFO:tasks.workunit.client.0.smithi100.stdout: "compact_lat": { 2024-04-07T17:51:23.807 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 18, 2024-04-07T17:51:23.807 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5817020 2024-04-07T17:51:23.807 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.807 INFO:tasks.workunit.client.0.smithi100.stdout: "compact_lock_lat": { 2024-04-07T17:51:23.807 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 18, 2024-04-07T17:51:23.807 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2179935 2024-04-07T17:51:23.807 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.808 INFO:tasks.workunit.client.0.smithi100.stdout: "db_alloc_lat": { 2024-04-07T17:51:23.808 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 187, 2024-04-07T17:51:23.808 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1295182 2024-04-07T17:51:23.808 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.808 INFO:tasks.workunit.client.0.smithi100.stdout: "db_total_bytes": { 2024-04-07T17:51:23.808 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 95995027456 2024-04-07T17:51:23.808 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.808 INFO:tasks.workunit.client.0.smithi100.stdout: "db_used_bytes": { 2024-04-07T17:51:23.808 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 721354752 2024-04-07T17:51:23.808 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.808 INFO:tasks.workunit.client.0.smithi100.stdout: "flush_lat": { 2024-04-07T17:51:23.808 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 74047, 2024-04-07T17:51:23.808 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 13624180905 2024-04-07T17:51:23.808 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.808 INFO:tasks.workunit.client.0.smithi100.stdout: "fsync_lat": { 2024-04-07T17:51:23.809 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 74056, 2024-04-07T17:51:23.809 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 23366921692 2024-04-07T17:51:23.809 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.809 INFO:tasks.workunit.client.0.smithi100.stdout: "log_bytes": { 2024-04-07T17:51:23.809 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4120576 2024-04-07T17:51:23.809 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.809 INFO:tasks.workunit.client.0.smithi100.stdout: "logged_bytes": { 2024-04-07T17:51:23.809 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 303386624 2024-04-07T17:51:23.809 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.809 INFO:tasks.workunit.client.0.smithi100.stdout: "max_bytes_db": { 2024-04-07T17:51:23.809 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 738394112 2024-04-07T17:51:23.809 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.809 INFO:tasks.workunit.client.0.smithi100.stdout: "max_bytes_slow": { 2024-04-07T17:51:23.809 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.809 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.809 INFO:tasks.workunit.client.0.smithi100.stdout: "max_bytes_wal": { 2024-04-07T17:51:23.810 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.810 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.810 INFO:tasks.workunit.client.0.smithi100.stdout: "num_files": { 2024-04-07T17:51:23.810 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 50 2024-04-07T17:51:23.810 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.810 INFO:tasks.workunit.client.0.smithi100.stdout: "read_bytes": { 2024-04-07T17:51:23.810 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3171092523 2024-04-07T17:51:23.810 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.810 INFO:tasks.workunit.client.0.smithi100.stdout: "read_count": { 2024-04-07T17:51:23.810 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 42792 2024-04-07T17:51:23.810 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.810 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes": { 2024-04-07T17:51:23.810 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2761277440 2024-04-07T17:51:23.810 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.811 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes_db": { 2024-04-07T17:51:23.811 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2761281536 2024-04-07T17:51:23.811 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.811 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes_slow": { 2024-04-07T17:51:23.811 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.811 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.811 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes_wal": { 2024-04-07T17:51:23.811 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.811 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.811 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_count": { 2024-04-07T17:51:23.811 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 35333 2024-04-07T17:51:23.811 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.811 INFO:tasks.workunit.client.0.smithi100.stdout: "read_lat": { 2024-04-07T17:51:23.811 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 42792, 2024-04-07T17:51:23.811 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2006369197 2024-04-07T17:51:23.812 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.812 INFO:tasks.workunit.client.0.smithi100.stdout: "read_prefetch_bytes": { 2024-04-07T17:51:23.812 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3054473279 2024-04-07T17:51:23.812 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.812 INFO:tasks.workunit.client.0.smithi100.stdout: "read_prefetch_count": { 2024-04-07T17:51:23.812 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 39218 2024-04-07T17:51:23.812 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.812 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_buffer_bytes": { 2024-04-07T17:51:23.812 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1733730 2024-04-07T17:51:23.812 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.812 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_buffer_count": { 2024-04-07T17:51:23.812 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 618 2024-04-07T17:51:23.812 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.812 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_bytes": { 2024-04-07T17:51:23.812 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5349743 2024-04-07T17:51:23.813 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.813 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_count": { 2024-04-07T17:51:23.813 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 831 2024-04-07T17:51:23.813 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.813 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes": { 2024-04-07T17:51:23.813 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3616013 2024-04-07T17:51:23.813 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.813 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes_db": { 2024-04-07T17:51:23.813 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3616013 2024-04-07T17:51:23.813 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.813 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes_slow": { 2024-04-07T17:51:23.813 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.813 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.813 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes_wal": { 2024-04-07T17:51:23.813 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.813 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.814 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_count": { 2024-04-07T17:51:23.814 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 217 2024-04-07T17:51:23.814 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.814 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_lat": { 2024-04-07T17:51:23.814 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 831, 2024-04-07T17:51:23.814 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2170610 2024-04-07T17:51:23.814 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.814 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_alloc_lat": { 2024-04-07T17:51:23.814 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:23.814 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.814 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.814 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_total_bytes": { 2024-04-07T17:51:23.814 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.814 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.815 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_used_bytes": { 2024-04-07T17:51:23.815 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.815 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.815 INFO:tasks.workunit.client.0.smithi100.stdout: "truncate_lat": { 2024-04-07T17:51:23.815 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1, 2024-04-07T17:51:23.815 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1014 2024-04-07T17:51:23.815 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.815 INFO:tasks.workunit.client.0.smithi100.stdout: "unlink_lat": { 2024-04-07T17:51:23.815 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 10, 2024-04-07T17:51:23.815 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 14895 2024-04-07T17:51:23.815 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.815 INFO:tasks.workunit.client.0.smithi100.stdout: "wal_alloc_lat": { 2024-04-07T17:51:23.815 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:23.815 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.815 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.815 INFO:tasks.workunit.client.0.smithi100.stdout: "wal_total_bytes": { 2024-04-07T17:51:23.816 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.816 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.816 INFO:tasks.workunit.client.0.smithi100.stdout: "wal_used_bytes": { 2024-04-07T17:51:23.816 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.816 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.816 INFO:tasks.workunit.client.0.smithi100.stdout: "write_bytes": { 2024-04-07T17:51:23.816 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1034661888 2024-04-07T17:51:23.816 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:23.816 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.816 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore": { 2024-04-07T17:51:23.816 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit": { 2024-04-07T17:51:23.816 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4096 2024-04-07T17:51:23.816 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.816 INFO:tasks.workunit.client.0.smithi100.stdout: "allocated": { 2024-04-07T17:51:23.816 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 9814016 2024-04-07T17:51:23.816 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.817 INFO:tasks.workunit.client.0.smithi100.stdout: "allocator_lat": { 2024-04-07T17:51:23.817 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 18804, 2024-04-07T17:51:23.817 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 100612374 2024-04-07T17:51:23.817 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.817 INFO:tasks.workunit.client.0.smithi100.stdout: "clist_lat": { 2024-04-07T17:51:23.817 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 3554, 2024-04-07T17:51:23.817 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 9702602885 2024-04-07T17:51:23.817 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.817 INFO:tasks.workunit.client.0.smithi100.stdout: "compress_lat": { 2024-04-07T17:51:23.817 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:23.817 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.817 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.817 INFO:tasks.workunit.client.0.smithi100.stdout: "compressed": { 2024-04-07T17:51:23.817 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.817 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.818 INFO:tasks.workunit.client.0.smithi100.stdout: "compressed_allocated": { 2024-04-07T17:51:23.818 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.818 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.818 INFO:tasks.workunit.client.0.smithi100.stdout: "compressed_original": { 2024-04-07T17:51:23.818 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.818 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.818 INFO:tasks.workunit.client.0.smithi100.stdout: "csum_lat": { 2024-04-07T17:51:23.818 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 20948, 2024-04-07T17:51:23.818 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 250530103 2024-04-07T17:51:23.818 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.818 INFO:tasks.workunit.client.0.smithi100.stdout: "decompress_lat": { 2024-04-07T17:51:23.818 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:23.818 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.818 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.818 INFO:tasks.workunit.client.0.smithi100.stdout: "fragmentation_micros": { 2024-04-07T17:51:23.819 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.819 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.819 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_commit_lat": { 2024-04-07T17:51:23.819 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 73998, 2024-04-07T17:51:23.819 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 30592293335 2024-04-07T17:51:23.819 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.819 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_final_lat": { 2024-04-07T17:51:23.819 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 73127, 2024-04-07T17:51:23.819 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3885020813 2024-04-07T17:51:23.819 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.819 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_flush_lat": { 2024-04-07T17:51:23.819 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 73998, 2024-04-07T17:51:23.819 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 99101609 2024-04-07T17:51:23.819 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.819 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_sync_lat": { 2024-04-07T17:51:23.820 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 73998, 2024-04-07T17:51:23.820 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 30691394944 2024-04-07T17:51:23.820 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.820 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_get_keys_lat": { 2024-04-07T17:51:23.820 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:23.820 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.820 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.820 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_get_values_lat": { 2024-04-07T17:51:23.820 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 29479, 2024-04-07T17:51:23.820 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1535907762 2024-04-07T17:51:23.820 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.820 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_lower_bound_lat": { 2024-04-07T17:51:23.820 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 847, 2024-04-07T17:51:23.820 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5353667 2024-04-07T17:51:23.820 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.820 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_next_lat": { 2024-04-07T17:51:23.821 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 17576, 2024-04-07T17:51:23.821 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 23428630 2024-04-07T17:51:23.821 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.821 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_seek_to_first_lat": { 2024-04-07T17:51:23.821 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 45, 2024-04-07T17:51:23.821 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 74604 2024-04-07T17:51:23.821 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.821 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_upper_bound_lat": { 2024-04-07T17:51:23.821 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 11749, 2024-04-07T17:51:23.821 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3050678508 2024-04-07T17:51:23.821 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.821 INFO:tasks.workunit.client.0.smithi100.stdout: "onode_hits": { 2024-04-07T17:51:23.821 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 227653 2024-04-07T17:51:23.821 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.821 INFO:tasks.workunit.client.0.smithi100.stdout: "onode_misses": { 2024-04-07T17:51:23.821 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 26792 2024-04-07T17:51:23.822 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.822 INFO:tasks.workunit.client.0.smithi100.stdout: "read_lat": { 2024-04-07T17:51:23.822 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 23747, 2024-04-07T17:51:23.822 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3478797990 2024-04-07T17:51:23.822 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.822 INFO:tasks.workunit.client.0.smithi100.stdout: "read_onode_meta_lat": { 2024-04-07T17:51:23.822 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 76353, 2024-04-07T17:51:23.822 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 39167256 2024-04-07T17:51:23.822 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.822 INFO:tasks.workunit.client.0.smithi100.stdout: "read_wait_aio_lat": { 2024-04-07T17:51:23.822 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 52606, 2024-04-07T17:51:23.822 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1660372202 2024-04-07T17:51:23.822 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.822 INFO:tasks.workunit.client.0.smithi100.stdout: "reads_with_retries": { 2024-04-07T17:51:23.822 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.823 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.823 INFO:tasks.workunit.client.0.smithi100.stdout: "remove_lat": { 2024-04-07T17:51:23.823 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 15062, 2024-04-07T17:51:23.823 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 718857059 2024-04-07T17:51:23.823 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.823 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_aio_wait_count": { 2024-04-07T17:51:23.823 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.823 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.823 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_committed_kv_count": { 2024-04-07T17:51:23.823 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.823 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.823 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_read_onode_meta_count": { 2024-04-07T17:51:23.823 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.823 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.823 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_read_wait_aio_count": { 2024-04-07T17:51:23.823 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.824 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.824 INFO:tasks.workunit.client.0.smithi100.stdout: "state_aio_wait_lat": { 2024-04-07T17:51:23.824 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 85696, 2024-04-07T17:51:23.824 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 8515002567 2024-04-07T17:51:23.824 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.824 INFO:tasks.workunit.client.0.smithi100.stdout: "state_deferred_aio_wait_lat": { 2024-04-07T17:51:23.824 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 22706, 2024-04-07T17:51:23.824 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2143104911 2024-04-07T17:51:23.824 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.824 INFO:tasks.workunit.client.0.smithi100.stdout: "state_deferred_cleanup_lat": { 2024-04-07T17:51:23.824 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 22706, 2024-04-07T17:51:23.824 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4386813655909 2024-04-07T17:51:23.824 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.824 INFO:tasks.workunit.client.0.smithi100.stdout: "state_deferred_queued_lat": { 2024-04-07T17:51:23.825 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 22706, 2024-04-07T17:51:23.825 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 7212579664352 2024-04-07T17:51:23.825 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.825 INFO:tasks.workunit.client.0.smithi100.stdout: "state_done_lat": { 2024-04-07T17:51:23.825 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 85696, 2024-04-07T17:51:23.825 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2950395751529 2024-04-07T17:51:23.825 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.825 INFO:tasks.workunit.client.0.smithi100.stdout: "state_finishing_lat": { 2024-04-07T17:51:23.825 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 85696, 2024-04-07T17:51:23.825 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 17842298 2024-04-07T17:51:23.825 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.825 INFO:tasks.workunit.client.0.smithi100.stdout: "state_io_done_lat": { 2024-04-07T17:51:23.825 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 85696, 2024-04-07T17:51:23.825 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 318013774 2024-04-07T17:51:23.825 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.826 INFO:tasks.workunit.client.0.smithi100.stdout: "state_kv_commiting_lat": { 2024-04-07T17:51:23.826 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 85696, 2024-04-07T17:51:23.826 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 50196560708 2024-04-07T17:51:23.826 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.826 INFO:tasks.workunit.client.0.smithi100.stdout: "state_kv_done_lat": { 2024-04-07T17:51:23.826 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 85696, 2024-04-07T17:51:23.826 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 64451217 2024-04-07T17:51:23.826 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.826 INFO:tasks.workunit.client.0.smithi100.stdout: "state_kv_queued_lat": { 2024-04-07T17:51:23.826 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 85696, 2024-04-07T17:51:23.826 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 24917952642 2024-04-07T17:51:23.826 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.826 INFO:tasks.workunit.client.0.smithi100.stdout: "state_prepare_lat": { 2024-04-07T17:51:23.826 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 85696, 2024-04-07T17:51:23.826 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 23724441505 2024-04-07T17:51:23.826 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.827 INFO:tasks.workunit.client.0.smithi100.stdout: "stored": { 2024-04-07T17:51:23.827 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5699861 2024-04-07T17:51:23.827 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.827 INFO:tasks.workunit.client.0.smithi100.stdout: "truncate_lat": { 2024-04-07T17:51:23.827 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 2507, 2024-04-07T17:51:23.827 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 25130678 2024-04-07T17:51:23.827 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.827 INFO:tasks.workunit.client.0.smithi100.stdout: "txc_commit_lat": { 2024-04-07T17:51:23.827 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 85696, 2024-04-07T17:51:23.827 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 107702564402 2024-04-07T17:51:23.827 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.827 INFO:tasks.workunit.client.0.smithi100.stdout: "txc_submit_lat": { 2024-04-07T17:51:23.827 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 85696, 2024-04-07T17:51:23.827 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 25902403453 2024-04-07T17:51:23.827 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.828 INFO:tasks.workunit.client.0.smithi100.stdout: "txc_throttle_lat": { 2024-04-07T17:51:23.828 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 85696, 2024-04-07T17:51:23.828 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 991891115 2024-04-07T17:51:23.828 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:23.828 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.828 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache": { 2024-04-07T17:51:23.828 INFO:tasks.workunit.client.0.smithi100.stdout: "cache_bytes": { 2024-04-07T17:51:23.828 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1444614092 2024-04-07T17:51:23.828 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.828 INFO:tasks.workunit.client.0.smithi100.stdout: "heap_bytes": { 2024-04-07T17:51:23.828 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 695918592 2024-04-07T17:51:23.828 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.828 INFO:tasks.workunit.client.0.smithi100.stdout: "mapped_bytes": { 2024-04-07T17:51:23.828 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 660807680 2024-04-07T17:51:23.828 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.829 INFO:tasks.workunit.client.0.smithi100.stdout: "target_bytes": { 2024-04-07T17:51:23.829 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2646965248 2024-04-07T17:51:23.829 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.829 INFO:tasks.workunit.client.0.smithi100.stdout: "unmapped_bytes": { 2024-04-07T17:51:23.829 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 35110912 2024-04-07T17:51:23.829 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:23.829 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.829 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:data": { 2024-04-07T17:51:23.829 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:23.829 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 134217728 2024-04-07T17:51:23.829 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.829 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:23.829 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.829 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.829 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:23.830 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.830 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.830 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:23.830 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 59401471 2024-04-07T17:51:23.830 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.830 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:23.830 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.830 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.830 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:23.830 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.830 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.831 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:23.831 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.831 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.831 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:23.831 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.831 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.831 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:23.831 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 114688 2024-04-07T17:51:23.831 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.831 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:23.831 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 733184 2024-04-07T17:51:23.831 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.831 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:23.831 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.831 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.832 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:23.832 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.832 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.832 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:23.832 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.832 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.832 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:23.832 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 73968385 2024-04-07T17:51:23.832 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:23.832 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.832 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:kv": { 2024-04-07T17:51:23.832 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:23.832 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 587202560 2024-04-07T17:51:23.832 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.832 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:23.832 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 67108864 2024-04-07T17:51:23.833 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.833 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:23.833 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.833 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.833 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:23.833 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 445511035 2024-04-07T17:51:23.833 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.833 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:23.833 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.833 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.833 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:23.833 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 256 2024-04-07T17:51:23.833 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.833 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:23.833 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.834 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.834 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:23.834 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.834 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.834 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:23.834 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 112208 2024-04-07T17:51:23.834 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.834 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:23.834 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.834 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.834 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:23.834 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.834 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.834 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:23.834 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.835 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.835 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:23.835 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.835 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.835 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:23.835 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 74470197 2024-04-07T17:51:23.835 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:23.835 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.835 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:kv_onode": { 2024-04-07T17:51:23.835 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:23.835 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 184549376 2024-04-07T17:51:23.835 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.835 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:23.835 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 67108864 2024-04-07T17:51:23.835 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.836 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:23.836 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4603888 2024-04-07T17:51:23.836 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.836 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:23.836 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 39600980 2024-04-07T17:51:23.836 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.836 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:23.836 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.836 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.836 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:23.836 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.836 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.836 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:23.836 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.836 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.836 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:23.837 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.837 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.837 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:23.837 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.837 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.837 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:23.837 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.837 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.837 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:23.837 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.837 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.837 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:23.837 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.837 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.837 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:23.837 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.838 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.838 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:23.838 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 73235644 2024-04-07T17:51:23.838 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:23.838 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.838 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:meta": { 2024-04-07T17:51:23.838 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:23.838 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 520093696 2024-04-07T17:51:23.838 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.838 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:23.838 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.838 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.838 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:23.838 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.838 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.839 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:23.839 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 445511035 2024-04-07T17:51:23.839 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.839 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:23.839 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.839 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.839 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:23.839 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.839 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.839 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:23.839 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.839 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.839 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:23.839 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.839 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.840 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:23.840 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 176559 2024-04-07T17:51:23.840 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.840 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:23.840 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1067248 2024-04-07T17:51:23.840 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.840 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:23.840 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.840 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.840 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:23.840 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.840 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.840 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:23.840 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.840 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.840 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:23.841 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 73338854 2024-04-07T17:51:23.841 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:23.841 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.841 INFO:tasks.workunit.client.0.smithi100.stdout: "objecter": { 2024-04-07T17:51:23.841 INFO:tasks.workunit.client.0.smithi100.stdout: "op_active": { 2024-04-07T17:51:23.841 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.841 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.841 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r": { 2024-04-07T17:51:23.841 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 21 2024-04-07T17:51:23.841 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.841 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rmw": { 2024-04-07T17:51:23.841 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.841 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.841 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w": { 2024-04-07T17:51:23.841 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1 2024-04-07T17:51:23.841 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:23.842 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.842 INFO:tasks.workunit.client.0.smithi100.stdout: "osd": { 2024-04-07T17:51:23.842 INFO:tasks.workunit.client.0.smithi100.stdout: "numpg": { 2024-04-07T17:51:23.842 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 52 2024-04-07T17:51:23.842 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.842 INFO:tasks.workunit.client.0.smithi100.stdout: "numpg_removing": { 2024-04-07T17:51:23.842 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.842 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.842 INFO:tasks.workunit.client.0.smithi100.stdout: "op": { 2024-04-07T17:51:23.842 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 45437 2024-04-07T17:51:23.842 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.842 INFO:tasks.workunit.client.0.smithi100.stdout: "op_before_queue_op_lat": { 2024-04-07T17:51:23.842 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 141249, 2024-04-07T17:51:23.843 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 14856125226 2024-04-07T17:51:23.843 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.843 INFO:tasks.workunit.client.0.smithi100.stdout: "op_delayed_degraded": { 2024-04-07T17:51:23.843 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.843 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.843 INFO:tasks.workunit.client.0.smithi100.stdout: "op_delayed_unreadable": { 2024-04-07T17:51:23.843 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1 2024-04-07T17:51:23.843 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.843 INFO:tasks.workunit.client.0.smithi100.stdout: "op_in_bytes": { 2024-04-07T17:51:23.843 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2465060363 2024-04-07T17:51:23.843 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.843 INFO:tasks.workunit.client.0.smithi100.stdout: "op_latency": { 2024-04-07T17:51:23.843 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 45437, 2024-04-07T17:51:23.843 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 845704656846 2024-04-07T17:51:23.843 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.844 INFO:tasks.workunit.client.0.smithi100.stdout: "op_out_bytes": { 2024-04-07T17:51:23.844 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2916484455 2024-04-07T17:51:23.844 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.844 INFO:tasks.workunit.client.0.smithi100.stdout: "op_prepare_latency": { 2024-04-07T17:51:23.844 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 47101, 2024-04-07T17:51:23.844 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 66781908488 2024-04-07T17:51:23.844 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.844 INFO:tasks.workunit.client.0.smithi100.stdout: "op_process_latency": { 2024-04-07T17:51:23.844 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 45437, 2024-04-07T17:51:23.844 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 194958987564 2024-04-07T17:51:23.844 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.844 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r": { 2024-04-07T17:51:23.844 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 22754 2024-04-07T17:51:23.844 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.844 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_latency": { 2024-04-07T17:51:23.845 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 22754, 2024-04-07T17:51:23.845 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 30613921114 2024-04-07T17:51:23.845 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.845 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_latency_out_bytes_histogram": { 2024-04-07T17:51:23.845 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.845 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.845 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_out_bytes": { 2024-04-07T17:51:23.845 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2916484455 2024-04-07T17:51:23.845 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.845 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_prepare_latency": { 2024-04-07T17:51:23.845 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 23171, 2024-04-07T17:51:23.845 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 12769117700 2024-04-07T17:51:23.845 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.845 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_process_latency": { 2024-04-07T17:51:23.845 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 22754, 2024-04-07T17:51:23.845 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 12464549208 2024-04-07T17:51:23.846 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.846 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw": { 2024-04-07T17:51:23.846 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 12919 2024-04-07T17:51:23.846 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.846 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_in_bytes": { 2024-04-07T17:51:23.846 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 335492738 2024-04-07T17:51:23.846 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.846 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_latency": { 2024-04-07T17:51:23.846 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 12919, 2024-04-07T17:51:23.846 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 239996971130 2024-04-07T17:51:23.846 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.846 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_latency_in_bytes_histogram": { 2024-04-07T17:51:23.846 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.846 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.847 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_latency_out_bytes_histogram": { 2024-04-07T17:51:23.847 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.847 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.847 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_out_bytes": { 2024-04-07T17:51:23.847 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.847 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.847 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_prepare_latency": { 2024-04-07T17:51:23.847 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 13070, 2024-04-07T17:51:23.847 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 30679482792 2024-04-07T17:51:23.847 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.847 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_process_latency": { 2024-04-07T17:51:23.847 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 12919, 2024-04-07T17:51:23.847 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 75646162399 2024-04-07T17:51:23.847 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.847 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w": { 2024-04-07T17:51:23.847 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 9764 2024-04-07T17:51:23.848 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.848 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_in_bytes": { 2024-04-07T17:51:23.848 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2129567625 2024-04-07T17:51:23.848 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.848 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_latency": { 2024-04-07T17:51:23.848 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 9764, 2024-04-07T17:51:23.848 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 575093764602 2024-04-07T17:51:23.848 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.848 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_latency_in_bytes_histogram": { 2024-04-07T17:51:23.848 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.848 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.848 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_prepare_latency": { 2024-04-07T17:51:23.848 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 10860, 2024-04-07T17:51:23.848 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 23333307996 2024-04-07T17:51:23.848 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.849 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_process_latency": { 2024-04-07T17:51:23.849 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 9764, 2024-04-07T17:51:23.849 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 106848275957 2024-04-07T17:51:23.849 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.849 INFO:tasks.workunit.client.0.smithi100.stdout: "op_wip": { 2024-04-07T17:51:23.849 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.849 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.849 INFO:tasks.workunit.client.0.smithi100.stdout: "recovery_bytes": { 2024-04-07T17:51:23.849 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 40249613 2024-04-07T17:51:23.849 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.849 INFO:tasks.workunit.client.0.smithi100.stdout: "recovery_ops": { 2024-04-07T17:51:23.849 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 28 2024-04-07T17:51:23.849 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.849 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_bytes": { 2024-04-07T17:51:23.849 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 95995035648 2024-04-07T17:51:23.849 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.850 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_bytes_used": { 2024-04-07T17:51:23.850 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 731176960 2024-04-07T17:51:23.850 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:23.850 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:23.850 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.850 INFO:tasks.workunit.client.0.smithi100.stdout: "osd.2": { 2024-04-07T17:51:23.850 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs": { 2024-04-07T17:51:23.850 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_db_max_lat": { 2024-04-07T17:51:23.850 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 33529 2024-04-07T17:51:23.850 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.850 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_slow_fallback": { 2024-04-07T17:51:23.850 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.850 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.850 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_slow_max_lat": { 2024-04-07T17:51:23.851 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.851 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.851 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_slow_size_fallback": { 2024-04-07T17:51:23.851 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.851 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.851 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit_db": { 2024-04-07T17:51:23.851 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 65536 2024-04-07T17:51:23.851 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.851 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit_slow": { 2024-04-07T17:51:23.851 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.851 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.851 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit_wal": { 2024-04-07T17:51:23.851 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.851 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.851 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_wal_max_lat": { 2024-04-07T17:51:23.851 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.852 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.852 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_written_slow": { 2024-04-07T17:51:23.852 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.852 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.852 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_written_sst": { 2024-04-07T17:51:23.852 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 11583488 2024-04-07T17:51:23.852 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.852 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_written_wal": { 2024-04-07T17:51:23.852 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 567971840 2024-04-07T17:51:23.852 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.852 INFO:tasks.workunit.client.0.smithi100.stdout: "compact_lat": { 2024-04-07T17:51:23.852 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 17, 2024-04-07T17:51:23.852 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 10549451 2024-04-07T17:51:23.852 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.852 INFO:tasks.workunit.client.0.smithi100.stdout: "compact_lock_lat": { 2024-04-07T17:51:23.853 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 17, 2024-04-07T17:51:23.853 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3589378 2024-04-07T17:51:23.853 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.853 INFO:tasks.workunit.client.0.smithi100.stdout: "db_alloc_lat": { 2024-04-07T17:51:23.853 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 176, 2024-04-07T17:51:23.853 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1295988 2024-04-07T17:51:23.853 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.853 INFO:tasks.workunit.client.0.smithi100.stdout: "db_total_bytes": { 2024-04-07T17:51:23.853 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 95995027456 2024-04-07T17:51:23.853 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.853 INFO:tasks.workunit.client.0.smithi100.stdout: "db_used_bytes": { 2024-04-07T17:51:23.853 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 446365696 2024-04-07T17:51:23.853 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.853 INFO:tasks.workunit.client.0.smithi100.stdout: "flush_lat": { 2024-04-07T17:51:23.853 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 69116, 2024-04-07T17:51:23.853 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 12568150083 2024-04-07T17:51:23.854 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.854 INFO:tasks.workunit.client.0.smithi100.stdout: "fsync_lat": { 2024-04-07T17:51:23.854 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 69111, 2024-04-07T17:51:23.854 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 21920914517 2024-04-07T17:51:23.854 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.854 INFO:tasks.workunit.client.0.smithi100.stdout: "log_bytes": { 2024-04-07T17:51:23.854 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 491520 2024-04-07T17:51:23.854 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.854 INFO:tasks.workunit.client.0.smithi100.stdout: "logged_bytes": { 2024-04-07T17:51:23.854 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 283127808 2024-04-07T17:51:23.854 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.854 INFO:tasks.workunit.client.0.smithi100.stdout: "max_bytes_db": { 2024-04-07T17:51:23.854 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 449183744 2024-04-07T17:51:23.854 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.854 INFO:tasks.workunit.client.0.smithi100.stdout: "max_bytes_slow": { 2024-04-07T17:51:23.855 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.855 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.855 INFO:tasks.workunit.client.0.smithi100.stdout: "max_bytes_wal": { 2024-04-07T17:51:23.855 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.855 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.855 INFO:tasks.workunit.client.0.smithi100.stdout: "num_files": { 2024-04-07T17:51:23.855 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 47 2024-04-07T17:51:23.855 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.855 INFO:tasks.workunit.client.0.smithi100.stdout: "read_bytes": { 2024-04-07T17:51:23.855 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 8760902516 2024-04-07T17:51:23.855 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.855 INFO:tasks.workunit.client.0.smithi100.stdout: "read_count": { 2024-04-07T17:51:23.855 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 74144 2024-04-07T17:51:23.855 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.855 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes": { 2024-04-07T17:51:23.856 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 7434956800 2024-04-07T17:51:23.856 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.856 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes_db": { 2024-04-07T17:51:23.856 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 7434960896 2024-04-07T17:51:23.856 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.856 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes_slow": { 2024-04-07T17:51:23.856 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.856 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.856 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes_wal": { 2024-04-07T17:51:23.856 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.856 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.856 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_count": { 2024-04-07T17:51:23.856 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 64202 2024-04-07T17:51:23.856 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.856 INFO:tasks.workunit.client.0.smithi100.stdout: "read_lat": { 2024-04-07T17:51:23.856 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 74144, 2024-04-07T17:51:23.857 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4199208949 2024-04-07T17:51:23.857 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.857 INFO:tasks.workunit.client.0.smithi100.stdout: "read_prefetch_bytes": { 2024-04-07T17:51:23.857 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 8601015701 2024-04-07T17:51:23.857 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.857 INFO:tasks.workunit.client.0.smithi100.stdout: "read_prefetch_count": { 2024-04-07T17:51:23.857 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 69247 2024-04-07T17:51:23.857 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.857 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_buffer_bytes": { 2024-04-07T17:51:23.857 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3233145 2024-04-07T17:51:23.857 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.857 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_buffer_count": { 2024-04-07T17:51:23.857 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 709 2024-04-07T17:51:23.857 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.857 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_bytes": { 2024-04-07T17:51:23.858 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 11496373 2024-04-07T17:51:23.858 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.858 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_count": { 2024-04-07T17:51:23.858 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1006 2024-04-07T17:51:23.858 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.858 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes": { 2024-04-07T17:51:23.858 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 8263228 2024-04-07T17:51:23.858 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.858 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes_db": { 2024-04-07T17:51:23.858 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 8263228 2024-04-07T17:51:23.858 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.858 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes_slow": { 2024-04-07T17:51:23.858 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.858 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.858 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes_wal": { 2024-04-07T17:51:23.859 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.859 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.859 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_count": { 2024-04-07T17:51:23.859 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 312 2024-04-07T17:51:23.859 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.859 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_lat": { 2024-04-07T17:51:23.859 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1006, 2024-04-07T17:51:23.859 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4007550 2024-04-07T17:51:23.859 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.859 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_alloc_lat": { 2024-04-07T17:51:23.859 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:23.859 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.859 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.859 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_total_bytes": { 2024-04-07T17:51:23.859 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.859 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.860 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_used_bytes": { 2024-04-07T17:51:23.860 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.860 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.860 INFO:tasks.workunit.client.0.smithi100.stdout: "truncate_lat": { 2024-04-07T17:51:23.860 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1, 2024-04-07T17:51:23.860 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 636 2024-04-07T17:51:23.860 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.860 INFO:tasks.workunit.client.0.smithi100.stdout: "unlink_lat": { 2024-04-07T17:51:23.860 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 13, 2024-04-07T17:51:23.860 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 14014 2024-04-07T17:51:23.860 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.860 INFO:tasks.workunit.client.0.smithi100.stdout: "wal_alloc_lat": { 2024-04-07T17:51:23.860 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:23.860 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.861 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.861 INFO:tasks.workunit.client.0.smithi100.stdout: "wal_total_bytes": { 2024-04-07T17:51:23.861 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.861 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.861 INFO:tasks.workunit.client.0.smithi100.stdout: "wal_used_bytes": { 2024-04-07T17:51:23.861 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.861 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.861 INFO:tasks.workunit.client.0.smithi100.stdout: "write_bytes": { 2024-04-07T17:51:23.861 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 865255424 2024-04-07T17:51:23.861 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:23.861 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.861 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore": { 2024-04-07T17:51:23.861 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit": { 2024-04-07T17:51:23.861 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4096 2024-04-07T17:51:23.861 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.861 INFO:tasks.workunit.client.0.smithi100.stdout: "allocated": { 2024-04-07T17:51:23.862 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 9801728 2024-04-07T17:51:23.862 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.862 INFO:tasks.workunit.client.0.smithi100.stdout: "allocator_lat": { 2024-04-07T17:51:23.862 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 13808, 2024-04-07T17:51:23.862 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 73423246 2024-04-07T17:51:23.862 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.862 INFO:tasks.workunit.client.0.smithi100.stdout: "clist_lat": { 2024-04-07T17:51:23.862 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 3623, 2024-04-07T17:51:23.862 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 12377775846 2024-04-07T17:51:23.862 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.862 INFO:tasks.workunit.client.0.smithi100.stdout: "compress_lat": { 2024-04-07T17:51:23.862 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:23.862 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.862 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.863 INFO:tasks.workunit.client.0.smithi100.stdout: "compressed": { 2024-04-07T17:51:23.863 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.863 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.863 INFO:tasks.workunit.client.0.smithi100.stdout: "compressed_allocated": { 2024-04-07T17:51:23.863 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.863 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.863 INFO:tasks.workunit.client.0.smithi100.stdout: "compressed_original": { 2024-04-07T17:51:23.863 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.863 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.863 INFO:tasks.workunit.client.0.smithi100.stdout: "csum_lat": { 2024-04-07T17:51:23.863 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 26449, 2024-04-07T17:51:23.863 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 333837013 2024-04-07T17:51:23.863 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.863 INFO:tasks.workunit.client.0.smithi100.stdout: "decompress_lat": { 2024-04-07T17:51:23.863 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:23.863 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.864 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.864 INFO:tasks.workunit.client.0.smithi100.stdout: "fragmentation_micros": { 2024-04-07T17:51:23.864 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.864 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.864 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_commit_lat": { 2024-04-07T17:51:23.864 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 69053, 2024-04-07T17:51:23.864 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 28692593210 2024-04-07T17:51:23.864 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.864 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_final_lat": { 2024-04-07T17:51:23.864 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 68316, 2024-04-07T17:51:23.864 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3308256168 2024-04-07T17:51:23.864 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.864 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_flush_lat": { 2024-04-07T17:51:23.864 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 69053, 2024-04-07T17:51:23.864 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 77146286 2024-04-07T17:51:23.864 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.865 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_sync_lat": { 2024-04-07T17:51:23.865 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 69053, 2024-04-07T17:51:23.865 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 28769739496 2024-04-07T17:51:23.865 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.865 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_get_keys_lat": { 2024-04-07T17:51:23.865 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:23.865 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.865 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.865 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_get_values_lat": { 2024-04-07T17:51:23.865 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 54368, 2024-04-07T17:51:23.865 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1618898953 2024-04-07T17:51:23.865 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.865 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_lower_bound_lat": { 2024-04-07T17:51:23.865 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 911, 2024-04-07T17:51:23.866 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5592855 2024-04-07T17:51:23.866 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.866 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_next_lat": { 2024-04-07T17:51:23.866 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 22756, 2024-04-07T17:51:23.866 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 25939924 2024-04-07T17:51:23.866 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.866 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_seek_to_first_lat": { 2024-04-07T17:51:23.866 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 36, 2024-04-07T17:51:23.866 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 71387 2024-04-07T17:51:23.866 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.866 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_upper_bound_lat": { 2024-04-07T17:51:23.866 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 19162, 2024-04-07T17:51:23.866 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2934148688 2024-04-07T17:51:23.866 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.866 INFO:tasks.workunit.client.0.smithi100.stdout: "onode_hits": { 2024-04-07T17:51:23.867 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 251164 2024-04-07T17:51:23.867 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.867 INFO:tasks.workunit.client.0.smithi100.stdout: "onode_misses": { 2024-04-07T17:51:23.867 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 19790 2024-04-07T17:51:23.867 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.867 INFO:tasks.workunit.client.0.smithi100.stdout: "read_lat": { 2024-04-07T17:51:23.867 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 17396, 2024-04-07T17:51:23.867 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2437128824 2024-04-07T17:51:23.867 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.867 INFO:tasks.workunit.client.0.smithi100.stdout: "read_onode_meta_lat": { 2024-04-07T17:51:23.867 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 41161, 2024-04-07T17:51:23.867 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 40062786 2024-04-07T17:51:23.867 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.867 INFO:tasks.workunit.client.0.smithi100.stdout: "read_wait_aio_lat": { 2024-04-07T17:51:23.867 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 23765, 2024-04-07T17:51:23.868 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1924044806 2024-04-07T17:51:23.868 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.868 INFO:tasks.workunit.client.0.smithi100.stdout: "reads_with_retries": { 2024-04-07T17:51:23.868 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.868 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.868 INFO:tasks.workunit.client.0.smithi100.stdout: "remove_lat": { 2024-04-07T17:51:23.868 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 12440, 2024-04-07T17:51:23.868 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 700556520 2024-04-07T17:51:23.868 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.868 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_aio_wait_count": { 2024-04-07T17:51:23.868 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.868 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.868 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_committed_kv_count": { 2024-04-07T17:51:23.868 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.868 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.869 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_read_onode_meta_count": { 2024-04-07T17:51:23.869 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.869 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.869 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_read_wait_aio_count": { 2024-04-07T17:51:23.869 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.869 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.869 INFO:tasks.workunit.client.0.smithi100.stdout: "state_aio_wait_lat": { 2024-04-07T17:51:23.869 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 77648, 2024-04-07T17:51:23.869 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 7605898743 2024-04-07T17:51:23.869 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.869 INFO:tasks.workunit.client.0.smithi100.stdout: "state_deferred_aio_wait_lat": { 2024-04-07T17:51:23.869 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 5880, 2024-04-07T17:51:23.869 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 870604827 2024-04-07T17:51:23.869 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.869 INFO:tasks.workunit.client.0.smithi100.stdout: "state_deferred_cleanup_lat": { 2024-04-07T17:51:23.870 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 5879, 2024-04-07T17:51:23.870 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4540688258752 2024-04-07T17:51:23.870 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.870 INFO:tasks.workunit.client.0.smithi100.stdout: "state_deferred_queued_lat": { 2024-04-07T17:51:23.870 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 5880, 2024-04-07T17:51:23.870 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 6337522548440 2024-04-07T17:51:23.870 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.870 INFO:tasks.workunit.client.0.smithi100.stdout: "state_done_lat": { 2024-04-07T17:51:23.870 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 77647, 2024-04-07T17:51:23.870 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2768612828005 2024-04-07T17:51:23.870 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.870 INFO:tasks.workunit.client.0.smithi100.stdout: "state_finishing_lat": { 2024-04-07T17:51:23.870 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 77647, 2024-04-07T17:51:23.870 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 18125716 2024-04-07T17:51:23.870 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.871 INFO:tasks.workunit.client.0.smithi100.stdout: "state_io_done_lat": { 2024-04-07T17:51:23.871 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 77648, 2024-04-07T17:51:23.871 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 245246928 2024-04-07T17:51:23.871 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.871 INFO:tasks.workunit.client.0.smithi100.stdout: "state_kv_commiting_lat": { 2024-04-07T17:51:23.871 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 77648, 2024-04-07T17:51:23.871 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 45419960556 2024-04-07T17:51:23.871 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.871 INFO:tasks.workunit.client.0.smithi100.stdout: "state_kv_done_lat": { 2024-04-07T17:51:23.871 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 77648, 2024-04-07T17:51:23.873 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 60433024 2024-04-07T17:51:23.873 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.873 INFO:tasks.workunit.client.0.smithi100.stdout: "state_kv_queued_lat": { 2024-04-07T17:51:23.873 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 77648, 2024-04-07T17:51:23.873 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 21195606198 2024-04-07T17:51:23.873 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.873 INFO:tasks.workunit.client.0.smithi100.stdout: "state_prepare_lat": { 2024-04-07T17:51:23.873 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 77648, 2024-04-07T17:51:23.873 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 23621700730 2024-04-07T17:51:23.873 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.874 INFO:tasks.workunit.client.0.smithi100.stdout: "stored": { 2024-04-07T17:51:23.874 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5699682 2024-04-07T17:51:23.874 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.874 INFO:tasks.workunit.client.0.smithi100.stdout: "truncate_lat": { 2024-04-07T17:51:23.874 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 2411, 2024-04-07T17:51:23.874 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 23697978 2024-04-07T17:51:23.874 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.874 INFO:tasks.workunit.client.0.smithi100.stdout: "txc_commit_lat": { 2024-04-07T17:51:23.874 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 77648, 2024-04-07T17:51:23.874 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 98115535290 2024-04-07T17:51:23.874 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.874 INFO:tasks.workunit.client.0.smithi100.stdout: "txc_submit_lat": { 2024-04-07T17:51:23.874 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 77648, 2024-04-07T17:51:23.874 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 25610928464 2024-04-07T17:51:23.874 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.875 INFO:tasks.workunit.client.0.smithi100.stdout: "txc_throttle_lat": { 2024-04-07T17:51:23.875 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 77648, 2024-04-07T17:51:23.875 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 903590085 2024-04-07T17:51:23.875 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:23.875 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.875 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache": { 2024-04-07T17:51:23.875 INFO:tasks.workunit.client.0.smithi100.stdout: "cache_bytes": { 2024-04-07T17:51:23.875 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1444614092 2024-04-07T17:51:23.875 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.875 INFO:tasks.workunit.client.0.smithi100.stdout: "heap_bytes": { 2024-04-07T17:51:23.875 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 563281920 2024-04-07T17:51:23.875 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.875 INFO:tasks.workunit.client.0.smithi100.stdout: "mapped_bytes": { 2024-04-07T17:51:23.875 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 513925120 2024-04-07T17:51:23.875 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.876 INFO:tasks.workunit.client.0.smithi100.stdout: "target_bytes": { 2024-04-07T17:51:23.876 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2646965248 2024-04-07T17:51:23.876 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.876 INFO:tasks.workunit.client.0.smithi100.stdout: "unmapped_bytes": { 2024-04-07T17:51:23.876 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 49356800 2024-04-07T17:51:23.876 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:23.876 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.876 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:data": { 2024-04-07T17:51:23.876 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:23.876 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 134217728 2024-04-07T17:51:23.876 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.876 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:23.876 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.876 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.876 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:23.877 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.877 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.877 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:23.877 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 59173493 2024-04-07T17:51:23.877 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.877 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:23.877 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.877 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.877 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:23.877 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.877 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.877 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:23.877 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.877 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.877 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:23.877 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.878 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.878 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:23.878 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 8192 2024-04-07T17:51:23.878 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.878 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:23.878 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 77824 2024-04-07T17:51:23.878 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.878 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:23.878 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.878 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.878 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:23.878 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.878 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.878 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:23.879 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.879 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.879 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:23.879 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 74958219 2024-04-07T17:51:23.879 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:23.879 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.879 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:kv": { 2024-04-07T17:51:23.879 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:23.879 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 578813952 2024-04-07T17:51:23.879 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.879 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:23.879 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 67108864 2024-04-07T17:51:23.879 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.879 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:23.879 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.880 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.880 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:23.880 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 443801202 2024-04-07T17:51:23.880 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.880 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:23.880 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.880 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.880 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:23.880 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.880 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.880 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:23.880 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2384 2024-04-07T17:51:23.880 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.880 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:23.880 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.881 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.881 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:23.881 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 150592 2024-04-07T17:51:23.881 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.881 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:23.881 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 384 2024-04-07T17:51:23.881 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.881 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:23.881 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.881 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.881 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:23.881 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.881 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.881 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:23.881 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.881 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.882 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:23.882 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 67750526 2024-04-07T17:51:23.882 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:23.882 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.882 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:kv_onode": { 2024-04-07T17:51:23.882 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:23.882 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 184549376 2024-04-07T17:51:23.882 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.882 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:23.882 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 67108864 2024-04-07T17:51:23.882 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.882 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:23.882 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 10790880 2024-04-07T17:51:23.882 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.883 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:23.883 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 39448995 2024-04-07T17:51:23.883 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.883 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:23.883 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.883 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.883 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:23.883 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.883 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.883 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:23.883 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.883 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.883 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:23.883 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.883 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.884 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:23.884 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.884 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.884 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:23.884 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.884 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.884 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:23.884 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.884 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.884 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:23.884 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.884 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.884 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:23.884 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.884 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.884 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:23.885 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 67200637 2024-04-07T17:51:23.885 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:23.885 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.885 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:meta": { 2024-04-07T17:51:23.885 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:23.885 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 520093696 2024-04-07T17:51:23.885 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.885 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:23.885 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.885 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.885 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:23.885 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.885 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.885 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:23.885 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 443801202 2024-04-07T17:51:23.886 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.886 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:23.886 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.886 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.886 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:23.886 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.886 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.886 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:23.886 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.886 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.886 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:23.886 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.886 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.886 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:23.886 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 148649 2024-04-07T17:51:23.887 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.887 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:23.887 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1108845 2024-04-07T17:51:23.887 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.887 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:23.887 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.887 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.887 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:23.887 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.887 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.887 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:23.887 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.887 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.887 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:23.887 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 75035000 2024-04-07T17:51:23.888 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:23.888 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.888 INFO:tasks.workunit.client.0.smithi100.stdout: "objecter": { 2024-04-07T17:51:23.888 INFO:tasks.workunit.client.0.smithi100.stdout: "op_active": { 2024-04-07T17:51:23.888 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.888 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.888 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r": { 2024-04-07T17:51:23.888 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 91 2024-04-07T17:51:23.888 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.888 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rmw": { 2024-04-07T17:51:23.888 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.888 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.888 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w": { 2024-04-07T17:51:23.888 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.888 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:23.888 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.889 INFO:tasks.workunit.client.0.smithi100.stdout: "osd": { 2024-04-07T17:51:23.889 INFO:tasks.workunit.client.0.smithi100.stdout: "numpg": { 2024-04-07T17:51:23.889 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 40 2024-04-07T17:51:23.889 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.889 INFO:tasks.workunit.client.0.smithi100.stdout: "numpg_removing": { 2024-04-07T17:51:23.889 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.889 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.889 INFO:tasks.workunit.client.0.smithi100.stdout: "op": { 2024-04-07T17:51:23.889 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 58391 2024-04-07T17:51:23.889 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.889 INFO:tasks.workunit.client.0.smithi100.stdout: "op_before_queue_op_lat": { 2024-04-07T17:51:23.889 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 150392, 2024-04-07T17:51:23.889 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 13814312300 2024-04-07T17:51:23.889 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.889 INFO:tasks.workunit.client.0.smithi100.stdout: "op_delayed_degraded": { 2024-04-07T17:51:23.889 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5 2024-04-07T17:51:23.890 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.890 INFO:tasks.workunit.client.0.smithi100.stdout: "op_delayed_unreadable": { 2024-04-07T17:51:23.890 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.890 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.890 INFO:tasks.workunit.client.0.smithi100.stdout: "op_in_bytes": { 2024-04-07T17:51:23.890 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2120776034 2024-04-07T17:51:23.890 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.890 INFO:tasks.workunit.client.0.smithi100.stdout: "op_latency": { 2024-04-07T17:51:23.890 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 58391, 2024-04-07T17:51:23.890 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 187069135567 2024-04-07T17:51:23.890 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.890 INFO:tasks.workunit.client.0.smithi100.stdout: "op_out_bytes": { 2024-04-07T17:51:23.890 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3533359549 2024-04-07T17:51:23.890 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.891 INFO:tasks.workunit.client.0.smithi100.stdout: "op_prepare_latency": { 2024-04-07T17:51:23.891 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 59237, 2024-04-07T17:51:23.891 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 89091892647 2024-04-07T17:51:23.891 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.891 INFO:tasks.workunit.client.0.smithi100.stdout: "op_process_latency": { 2024-04-07T17:51:23.891 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 58391, 2024-04-07T17:51:23.891 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 156357104214 2024-04-07T17:51:23.891 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.891 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r": { 2024-04-07T17:51:23.891 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 29188 2024-04-07T17:51:23.891 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.891 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_latency": { 2024-04-07T17:51:23.891 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 29188, 2024-04-07T17:51:23.891 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 31668185766 2024-04-07T17:51:23.891 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.891 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_latency_out_bytes_histogram": { 2024-04-07T17:51:23.892 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.892 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.892 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_out_bytes": { 2024-04-07T17:51:23.892 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3533359549 2024-04-07T17:51:23.892 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.892 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_prepare_latency": { 2024-04-07T17:51:23.892 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 29608, 2024-04-07T17:51:23.892 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 13047146829 2024-04-07T17:51:23.892 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.892 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_process_latency": { 2024-04-07T17:51:23.892 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 29188, 2024-04-07T17:51:23.892 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 17316757778 2024-04-07T17:51:23.892 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.892 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw": { 2024-04-07T17:51:23.892 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 27288 2024-04-07T17:51:23.893 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.893 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_in_bytes": { 2024-04-07T17:51:23.893 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 535101355 2024-04-07T17:51:23.893 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.893 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_latency": { 2024-04-07T17:51:23.893 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 27288, 2024-04-07T17:51:23.893 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 125896770692 2024-04-07T17:51:23.893 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.893 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_latency_in_bytes_histogram": { 2024-04-07T17:51:23.893 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.893 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.893 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_latency_out_bytes_histogram": { 2024-04-07T17:51:23.893 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.893 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.893 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_out_bytes": { 2024-04-07T17:51:23.893 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.894 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.894 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_prepare_latency": { 2024-04-07T17:51:23.894 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 27539, 2024-04-07T17:51:23.894 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 69930450013 2024-04-07T17:51:23.894 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.894 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_process_latency": { 2024-04-07T17:51:23.894 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 27288, 2024-04-07T17:51:23.894 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 117911423955 2024-04-07T17:51:23.894 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.894 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w": { 2024-04-07T17:51:23.894 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1915 2024-04-07T17:51:23.894 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.894 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_in_bytes": { 2024-04-07T17:51:23.894 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1585674679 2024-04-07T17:51:23.894 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.895 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_latency": { 2024-04-07T17:51:23.895 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1915, 2024-04-07T17:51:23.895 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 29504179109 2024-04-07T17:51:23.895 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.895 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_latency_in_bytes_histogram": { 2024-04-07T17:51:23.895 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.895 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.895 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_prepare_latency": { 2024-04-07T17:51:23.895 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 2090, 2024-04-07T17:51:23.895 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 6114295805 2024-04-07T17:51:23.895 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.895 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_process_latency": { 2024-04-07T17:51:23.895 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1915, 2024-04-07T17:51:23.895 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 21128922481 2024-04-07T17:51:23.895 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.896 INFO:tasks.workunit.client.0.smithi100.stdout: "op_wip": { 2024-04-07T17:51:23.896 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.896 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.896 INFO:tasks.workunit.client.0.smithi100.stdout: "recovery_bytes": { 2024-04-07T17:51:23.896 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4194321 2024-04-07T17:51:23.896 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.896 INFO:tasks.workunit.client.0.smithi100.stdout: "recovery_ops": { 2024-04-07T17:51:23.896 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 38 2024-04-07T17:51:23.896 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.896 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_bytes": { 2024-04-07T17:51:23.896 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 95995035648 2024-04-07T17:51:23.896 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.896 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_bytes_used": { 2024-04-07T17:51:23.896 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 456179712 2024-04-07T17:51:23.896 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:23.897 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:23.897 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.897 INFO:tasks.workunit.client.0.smithi100.stdout: "osd.3": { 2024-04-07T17:51:23.897 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs": { 2024-04-07T17:51:23.897 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_db_max_lat": { 2024-04-07T17:51:23.897 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 29319 2024-04-07T17:51:23.897 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.897 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_slow_fallback": { 2024-04-07T17:51:23.897 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.897 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.897 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_slow_max_lat": { 2024-04-07T17:51:23.897 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.897 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.897 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_slow_size_fallback": { 2024-04-07T17:51:23.898 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.898 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.898 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit_db": { 2024-04-07T17:51:23.898 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 65536 2024-04-07T17:51:23.898 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.898 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit_slow": { 2024-04-07T17:51:23.898 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.898 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.898 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit_wal": { 2024-04-07T17:51:23.898 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.898 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.898 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_wal_max_lat": { 2024-04-07T17:51:23.898 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.898 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.898 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_written_slow": { 2024-04-07T17:51:23.899 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.899 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.899 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_written_sst": { 2024-04-07T17:51:23.899 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 6107136 2024-04-07T17:51:23.899 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.899 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_written_wal": { 2024-04-07T17:51:23.899 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 546639872 2024-04-07T17:51:23.899 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.899 INFO:tasks.workunit.client.0.smithi100.stdout: "compact_lat": { 2024-04-07T17:51:23.899 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 16, 2024-04-07T17:51:23.899 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5676850 2024-04-07T17:51:23.899 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.899 INFO:tasks.workunit.client.0.smithi100.stdout: "compact_lock_lat": { 2024-04-07T17:51:23.900 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 16, 2024-04-07T17:51:23.900 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1688739 2024-04-07T17:51:23.900 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.900 INFO:tasks.workunit.client.0.smithi100.stdout: "db_alloc_lat": { 2024-04-07T17:51:23.900 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 158, 2024-04-07T17:51:23.900 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1098247 2024-04-07T17:51:23.900 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.900 INFO:tasks.workunit.client.0.smithi100.stdout: "db_total_bytes": { 2024-04-07T17:51:23.901 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 95995027456 2024-04-07T17:51:23.901 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.901 INFO:tasks.workunit.client.0.smithi100.stdout: "db_used_bytes": { 2024-04-07T17:51:23.901 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 449314816 2024-04-07T17:51:23.901 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.901 INFO:tasks.workunit.client.0.smithi100.stdout: "flush_lat": { 2024-04-07T17:51:23.901 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 67796, 2024-04-07T17:51:23.901 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 12755738535 2024-04-07T17:51:23.901 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.901 INFO:tasks.workunit.client.0.smithi100.stdout: "fsync_lat": { 2024-04-07T17:51:23.901 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 67808, 2024-04-07T17:51:23.901 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 22453593571 2024-04-07T17:51:23.901 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.901 INFO:tasks.workunit.client.0.smithi100.stdout: "log_bytes": { 2024-04-07T17:51:23.901 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 11792384 2024-04-07T17:51:23.902 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.902 INFO:tasks.workunit.client.0.smithi100.stdout: "logged_bytes": { 2024-04-07T17:51:23.902 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 277786624 2024-04-07T17:51:23.902 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.902 INFO:tasks.workunit.client.0.smithi100.stdout: "max_bytes_db": { 2024-04-07T17:51:23.902 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 462094336 2024-04-07T17:51:23.902 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.902 INFO:tasks.workunit.client.0.smithi100.stdout: "max_bytes_slow": { 2024-04-07T17:51:23.902 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.902 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.902 INFO:tasks.workunit.client.0.smithi100.stdout: "max_bytes_wal": { 2024-04-07T17:51:23.902 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.902 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.902 INFO:tasks.workunit.client.0.smithi100.stdout: "num_files": { 2024-04-07T17:51:23.902 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 41 2024-04-07T17:51:23.903 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.903 INFO:tasks.workunit.client.0.smithi100.stdout: "read_bytes": { 2024-04-07T17:51:23.903 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5081425364 2024-04-07T17:51:23.903 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.903 INFO:tasks.workunit.client.0.smithi100.stdout: "read_count": { 2024-04-07T17:51:23.903 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 55136 2024-04-07T17:51:23.903 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.903 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes": { 2024-04-07T17:51:23.903 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4273045504 2024-04-07T17:51:23.903 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.903 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes_db": { 2024-04-07T17:51:23.903 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4273049600 2024-04-07T17:51:23.903 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.903 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes_slow": { 2024-04-07T17:51:23.904 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.904 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.904 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes_wal": { 2024-04-07T17:51:23.904 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.904 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.904 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_count": { 2024-04-07T17:51:23.904 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 47715 2024-04-07T17:51:23.904 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.904 INFO:tasks.workunit.client.0.smithi100.stdout: "read_lat": { 2024-04-07T17:51:23.905 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 55136, 2024-04-07T17:51:23.905 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2526773312 2024-04-07T17:51:23.905 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.905 INFO:tasks.workunit.client.0.smithi100.stdout: "read_prefetch_bytes": { 2024-04-07T17:51:23.905 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4957544695 2024-04-07T17:51:23.905 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.905 INFO:tasks.workunit.client.0.smithi100.stdout: "read_prefetch_count": { 2024-04-07T17:51:23.905 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 51339 2024-04-07T17:51:23.905 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.905 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_buffer_bytes": { 2024-04-07T17:51:23.905 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1547577 2024-04-07T17:51:23.905 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.905 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_buffer_count": { 2024-04-07T17:51:23.905 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 561 2024-04-07T17:51:23.905 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.906 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_bytes": { 2024-04-07T17:51:23.906 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 6055753 2024-04-07T17:51:23.906 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.906 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_count": { 2024-04-07T17:51:23.906 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 720 2024-04-07T17:51:23.906 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.906 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes": { 2024-04-07T17:51:23.906 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4508176 2024-04-07T17:51:23.906 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.906 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes_db": { 2024-04-07T17:51:23.906 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4508176 2024-04-07T17:51:23.906 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.906 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes_slow": { 2024-04-07T17:51:23.906 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.906 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.907 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes_wal": { 2024-04-07T17:51:23.907 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.907 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.907 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_count": { 2024-04-07T17:51:23.907 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 161 2024-04-07T17:51:23.907 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.907 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_lat": { 2024-04-07T17:51:23.907 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 720, 2024-04-07T17:51:23.907 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2153712 2024-04-07T17:51:23.907 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.907 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_alloc_lat": { 2024-04-07T17:51:23.907 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:23.907 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.907 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.908 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_total_bytes": { 2024-04-07T17:51:23.908 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.908 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.908 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_used_bytes": { 2024-04-07T17:51:23.908 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.908 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.908 INFO:tasks.workunit.client.0.smithi100.stdout: "truncate_lat": { 2024-04-07T17:51:23.908 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1, 2024-04-07T17:51:23.909 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 872 2024-04-07T17:51:23.909 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.909 INFO:tasks.workunit.client.0.smithi100.stdout: "unlink_lat": { 2024-04-07T17:51:23.909 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 10, 2024-04-07T17:51:23.909 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 15209 2024-04-07T17:51:23.909 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.909 INFO:tasks.workunit.client.0.smithi100.stdout: "wal_alloc_lat": { 2024-04-07T17:51:23.909 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:23.910 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.910 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.910 INFO:tasks.workunit.client.0.smithi100.stdout: "wal_total_bytes": { 2024-04-07T17:51:23.910 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.910 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.910 INFO:tasks.workunit.client.0.smithi100.stdout: "wal_used_bytes": { 2024-04-07T17:51:23.910 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.910 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.910 INFO:tasks.workunit.client.0.smithi100.stdout: "write_bytes": { 2024-04-07T17:51:23.910 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 832966656 2024-04-07T17:51:23.910 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:23.911 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.911 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore": { 2024-04-07T17:51:23.911 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit": { 2024-04-07T17:51:23.911 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4096 2024-04-07T17:51:23.911 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.911 INFO:tasks.workunit.client.0.smithi100.stdout: "allocated": { 2024-04-07T17:51:23.911 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 9826304 2024-04-07T17:51:23.911 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.911 INFO:tasks.workunit.client.0.smithi100.stdout: "allocator_lat": { 2024-04-07T17:51:23.911 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 19863, 2024-04-07T17:51:23.911 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 103164491 2024-04-07T17:51:23.911 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.911 INFO:tasks.workunit.client.0.smithi100.stdout: "clist_lat": { 2024-04-07T17:51:23.912 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 3584, 2024-04-07T17:51:23.912 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 10742765419 2024-04-07T17:51:23.912 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.912 INFO:tasks.workunit.client.0.smithi100.stdout: "compress_lat": { 2024-04-07T17:51:23.912 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:23.912 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.912 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.912 INFO:tasks.workunit.client.0.smithi100.stdout: "compressed": { 2024-04-07T17:51:23.912 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.912 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.912 INFO:tasks.workunit.client.0.smithi100.stdout: "compressed_allocated": { 2024-04-07T17:51:23.913 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.913 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.913 INFO:tasks.workunit.client.0.smithi100.stdout: "compressed_original": { 2024-04-07T17:51:23.913 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.913 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.913 INFO:tasks.workunit.client.0.smithi100.stdout: "csum_lat": { 2024-04-07T17:51:23.913 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 13277, 2024-04-07T17:51:23.913 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 158291808 2024-04-07T17:51:23.913 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.913 INFO:tasks.workunit.client.0.smithi100.stdout: "decompress_lat": { 2024-04-07T17:51:23.913 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:23.913 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.913 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.913 INFO:tasks.workunit.client.0.smithi100.stdout: "fragmentation_micros": { 2024-04-07T17:51:23.913 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.914 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.914 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_commit_lat": { 2024-04-07T17:51:23.914 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 67759, 2024-04-07T17:51:23.914 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 28639225606 2024-04-07T17:51:23.914 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.914 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_final_lat": { 2024-04-07T17:51:23.914 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 66833, 2024-04-07T17:51:23.914 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3952581972 2024-04-07T17:51:23.914 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.914 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_flush_lat": { 2024-04-07T17:51:23.914 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 67759, 2024-04-07T17:51:23.914 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 98642276 2024-04-07T17:51:23.914 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.914 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_sync_lat": { 2024-04-07T17:51:23.914 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 67759, 2024-04-07T17:51:23.915 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 28737867882 2024-04-07T17:51:23.915 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.915 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_get_keys_lat": { 2024-04-07T17:51:23.915 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:23.915 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.915 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.915 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_get_values_lat": { 2024-04-07T17:51:23.915 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 26953, 2024-04-07T17:51:23.915 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1128418885 2024-04-07T17:51:23.915 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.915 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_lower_bound_lat": { 2024-04-07T17:51:23.915 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1271, 2024-04-07T17:51:23.915 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 7079072 2024-04-07T17:51:23.915 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.915 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_next_lat": { 2024-04-07T17:51:23.915 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 8457, 2024-04-07T17:51:23.916 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 12008607 2024-04-07T17:51:23.916 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.916 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_seek_to_first_lat": { 2024-04-07T17:51:23.916 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 40, 2024-04-07T17:51:23.916 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 80474 2024-04-07T17:51:23.916 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.916 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_upper_bound_lat": { 2024-04-07T17:51:23.916 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 11692, 2024-04-07T17:51:23.916 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4518542429 2024-04-07T17:51:23.916 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.916 INFO:tasks.workunit.client.0.smithi100.stdout: "onode_hits": { 2024-04-07T17:51:23.916 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 208992 2024-04-07T17:51:23.916 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.916 INFO:tasks.workunit.client.0.smithi100.stdout: "onode_misses": { 2024-04-07T17:51:23.916 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 18294 2024-04-07T17:51:23.916 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.917 INFO:tasks.workunit.client.0.smithi100.stdout: "read_lat": { 2024-04-07T17:51:23.917 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 30350, 2024-04-07T17:51:23.917 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2036962015 2024-04-07T17:51:23.917 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.917 INFO:tasks.workunit.client.0.smithi100.stdout: "read_onode_meta_lat": { 2024-04-07T17:51:23.917 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 80371, 2024-04-07T17:51:23.917 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 45758452 2024-04-07T17:51:23.917 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.917 INFO:tasks.workunit.client.0.smithi100.stdout: "read_wait_aio_lat": { 2024-04-07T17:51:23.917 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 50021, 2024-04-07T17:51:23.917 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1142513586 2024-04-07T17:51:23.917 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.917 INFO:tasks.workunit.client.0.smithi100.stdout: "reads_with_retries": { 2024-04-07T17:51:23.917 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.918 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.918 INFO:tasks.workunit.client.0.smithi100.stdout: "remove_lat": { 2024-04-07T17:51:23.918 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 12382, 2024-04-07T17:51:23.918 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 706656351 2024-04-07T17:51:23.918 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.918 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_aio_wait_count": { 2024-04-07T17:51:23.918 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.918 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.918 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_committed_kv_count": { 2024-04-07T17:51:23.918 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.918 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.918 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_read_onode_meta_count": { 2024-04-07T17:51:23.918 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.918 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.918 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_read_wait_aio_count": { 2024-04-07T17:51:23.919 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.919 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.919 INFO:tasks.workunit.client.0.smithi100.stdout: "state_aio_wait_lat": { 2024-04-07T17:51:23.919 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 77291, 2024-04-07T17:51:23.919 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 12556920270 2024-04-07T17:51:23.919 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.919 INFO:tasks.workunit.client.0.smithi100.stdout: "state_deferred_aio_wait_lat": { 2024-04-07T17:51:23.919 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 17849, 2024-04-07T17:51:23.919 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1593523661 2024-04-07T17:51:23.919 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.919 INFO:tasks.workunit.client.0.smithi100.stdout: "state_deferred_cleanup_lat": { 2024-04-07T17:51:23.919 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 17848, 2024-04-07T17:51:23.919 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3943299065015 2024-04-07T17:51:23.919 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.919 INFO:tasks.workunit.client.0.smithi100.stdout: "state_deferred_queued_lat": { 2024-04-07T17:51:23.920 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 17849, 2024-04-07T17:51:23.920 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 6510467867246 2024-04-07T17:51:23.920 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.920 INFO:tasks.workunit.client.0.smithi100.stdout: "state_done_lat": { 2024-04-07T17:51:23.920 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 77290, 2024-04-07T17:51:23.920 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2750364291611 2024-04-07T17:51:23.920 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.920 INFO:tasks.workunit.client.0.smithi100.stdout: "state_finishing_lat": { 2024-04-07T17:51:23.920 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 77290, 2024-04-07T17:51:23.920 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 16788531 2024-04-07T17:51:23.920 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.920 INFO:tasks.workunit.client.0.smithi100.stdout: "state_io_done_lat": { 2024-04-07T17:51:23.920 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 77291, 2024-04-07T17:51:23.920 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 461265229 2024-04-07T17:51:23.920 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.921 INFO:tasks.workunit.client.0.smithi100.stdout: "state_kv_commiting_lat": { 2024-04-07T17:51:23.921 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 77291, 2024-04-07T17:51:23.921 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 50377334346 2024-04-07T17:51:23.921 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.921 INFO:tasks.workunit.client.0.smithi100.stdout: "state_kv_done_lat": { 2024-04-07T17:51:23.921 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 77291, 2024-04-07T17:51:23.921 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 62405628 2024-04-07T17:51:23.921 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.921 INFO:tasks.workunit.client.0.smithi100.stdout: "state_kv_queued_lat": { 2024-04-07T17:51:23.921 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 77291, 2024-04-07T17:51:23.921 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 24119810266 2024-04-07T17:51:23.921 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.921 INFO:tasks.workunit.client.0.smithi100.stdout: "state_prepare_lat": { 2024-04-07T17:51:23.921 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 77291, 2024-04-07T17:51:23.921 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 26664283551 2024-04-07T17:51:23.922 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.922 INFO:tasks.workunit.client.0.smithi100.stdout: "stored": { 2024-04-07T17:51:23.922 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5705610 2024-04-07T17:51:23.922 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.922 INFO:tasks.workunit.client.0.smithi100.stdout: "truncate_lat": { 2024-04-07T17:51:23.922 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 2398, 2024-04-07T17:51:23.922 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 27539717 2024-04-07T17:51:23.922 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.922 INFO:tasks.workunit.client.0.smithi100.stdout: "txc_commit_lat": { 2024-04-07T17:51:23.922 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 77291, 2024-04-07T17:51:23.922 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 114204848603 2024-04-07T17:51:23.922 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.922 INFO:tasks.workunit.client.0.smithi100.stdout: "txc_submit_lat": { 2024-04-07T17:51:23.922 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 77291, 2024-04-07T17:51:23.923 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 28995862182 2024-04-07T17:51:23.923 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.923 INFO:tasks.workunit.client.0.smithi100.stdout: "txc_throttle_lat": { 2024-04-07T17:51:23.923 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 77291, 2024-04-07T17:51:23.923 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1079109304 2024-04-07T17:51:23.923 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:23.923 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.923 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache": { 2024-04-07T17:51:23.923 INFO:tasks.workunit.client.0.smithi100.stdout: "cache_bytes": { 2024-04-07T17:51:23.923 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1444614092 2024-04-07T17:51:23.923 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.923 INFO:tasks.workunit.client.0.smithi100.stdout: "heap_bytes": { 2024-04-07T17:51:23.923 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 503775232 2024-04-07T17:51:23.923 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.923 INFO:tasks.workunit.client.0.smithi100.stdout: "mapped_bytes": { 2024-04-07T17:51:23.924 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 470474752 2024-04-07T17:51:23.924 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.924 INFO:tasks.workunit.client.0.smithi100.stdout: "target_bytes": { 2024-04-07T17:51:23.924 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2646965248 2024-04-07T17:51:23.924 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.924 INFO:tasks.workunit.client.0.smithi100.stdout: "unmapped_bytes": { 2024-04-07T17:51:23.924 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 33300480 2024-04-07T17:51:23.924 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:23.924 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.924 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:data": { 2024-04-07T17:51:23.924 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:23.924 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 134217728 2024-04-07T17:51:23.924 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.924 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:23.925 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.925 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.925 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:23.925 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.925 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.925 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:23.925 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 59559460 2024-04-07T17:51:23.925 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.925 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:23.925 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.925 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.925 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:23.925 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.925 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.925 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:23.925 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.926 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.926 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:23.926 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.926 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.926 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:23.926 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 8192 2024-04-07T17:51:23.926 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.926 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:23.926 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 102400 2024-04-07T17:51:23.926 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.926 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:23.926 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.926 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.926 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:23.927 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.927 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.927 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:23.927 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.927 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.927 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:23.927 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 74547676 2024-04-07T17:51:23.927 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:23.927 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.927 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:kv": { 2024-04-07T17:51:23.927 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:23.927 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 587202560 2024-04-07T17:51:23.927 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.927 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:23.927 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 67108864 2024-04-07T17:51:23.928 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.928 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:23.928 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.928 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.928 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:23.928 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 446695953 2024-04-07T17:51:23.928 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.928 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:23.928 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.928 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.928 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:23.928 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1776 2024-04-07T17:51:23.928 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.928 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:23.928 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.929 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.929 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:23.929 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.929 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.929 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:23.929 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 110304 2024-04-07T17:51:23.929 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.929 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:23.929 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.929 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.929 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:23.929 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.929 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.929 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:23.929 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.929 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.930 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:23.930 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.930 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.930 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:23.930 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 73285663 2024-04-07T17:51:23.930 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:23.930 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.930 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:kv_onode": { 2024-04-07T17:51:23.930 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:23.930 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 184549376 2024-04-07T17:51:23.930 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.930 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:23.930 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 67108864 2024-04-07T17:51:23.930 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.931 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:23.931 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5620784 2024-04-07T17:51:23.931 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.931 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:23.931 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 39706307 2024-04-07T17:51:23.931 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.931 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:23.931 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.931 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.931 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:23.931 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.931 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.931 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:23.931 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.931 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.932 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:23.932 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.932 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.932 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:23.932 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.932 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.932 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:23.932 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.932 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.932 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:23.932 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.932 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.932 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:23.932 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.932 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.932 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:23.933 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.933 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.933 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:23.933 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 72113421 2024-04-07T17:51:23.933 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:23.933 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.933 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:meta": { 2024-04-07T17:51:23.933 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:23.933 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 520093696 2024-04-07T17:51:23.933 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.933 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:23.933 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.933 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.933 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:23.933 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.933 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.934 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:23.934 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 446695953 2024-04-07T17:51:23.934 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.934 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:23.934 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.934 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.934 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:23.934 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.934 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.934 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:23.934 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.934 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.934 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:23.934 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.935 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.935 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:23.935 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 151009 2024-04-07T17:51:23.935 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.935 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:23.935 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1112438 2024-04-07T17:51:23.935 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.935 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:23.935 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.935 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.935 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:23.935 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.935 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.935 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:23.935 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.935 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.936 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:23.936 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 72134296 2024-04-07T17:51:23.936 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:23.936 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.936 INFO:tasks.workunit.client.0.smithi100.stdout: "objecter": { 2024-04-07T17:51:23.936 INFO:tasks.workunit.client.0.smithi100.stdout: "op_active": { 2024-04-07T17:51:23.936 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.936 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.936 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r": { 2024-04-07T17:51:23.936 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 18 2024-04-07T17:51:23.936 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.936 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rmw": { 2024-04-07T17:51:23.936 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.936 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.936 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w": { 2024-04-07T17:51:23.937 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.937 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:23.937 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.937 INFO:tasks.workunit.client.0.smithi100.stdout: "osd": { 2024-04-07T17:51:23.937 INFO:tasks.workunit.client.0.smithi100.stdout: "numpg": { 2024-04-07T17:51:23.937 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 47 2024-04-07T17:51:23.937 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.937 INFO:tasks.workunit.client.0.smithi100.stdout: "numpg_removing": { 2024-04-07T17:51:23.937 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.937 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.937 INFO:tasks.workunit.client.0.smithi100.stdout: "op": { 2024-04-07T17:51:23.937 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 56964 2024-04-07T17:51:23.937 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.937 INFO:tasks.workunit.client.0.smithi100.stdout: "op_before_queue_op_lat": { 2024-04-07T17:51:23.937 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 133884, 2024-04-07T17:51:23.937 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 12292525237 2024-04-07T17:51:23.938 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.938 INFO:tasks.workunit.client.0.smithi100.stdout: "op_delayed_degraded": { 2024-04-07T17:51:23.938 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.938 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.938 INFO:tasks.workunit.client.0.smithi100.stdout: "op_delayed_unreadable": { 2024-04-07T17:51:23.938 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.938 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.938 INFO:tasks.workunit.client.0.smithi100.stdout: "op_in_bytes": { 2024-04-07T17:51:23.938 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1485365521 2024-04-07T17:51:23.938 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.938 INFO:tasks.workunit.client.0.smithi100.stdout: "op_latency": { 2024-04-07T17:51:23.938 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 56964, 2024-04-07T17:51:23.938 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 394646376159 2024-04-07T17:51:23.938 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.938 INFO:tasks.workunit.client.0.smithi100.stdout: "op_out_bytes": { 2024-04-07T17:51:23.939 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1430791176 2024-04-07T17:51:23.939 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.939 INFO:tasks.workunit.client.0.smithi100.stdout: "op_prepare_latency": { 2024-04-07T17:51:23.939 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 59216, 2024-04-07T17:51:23.939 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 48205385292 2024-04-07T17:51:23.939 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.939 INFO:tasks.workunit.client.0.smithi100.stdout: "op_process_latency": { 2024-04-07T17:51:23.939 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 56964, 2024-04-07T17:51:23.939 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 169980516116 2024-04-07T17:51:23.939 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.939 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r": { 2024-04-07T17:51:23.939 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 39255 2024-04-07T17:51:23.939 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.939 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_latency": { 2024-04-07T17:51:23.939 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 39255, 2024-04-07T17:51:23.940 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 64075850096 2024-04-07T17:51:23.940 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.940 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_latency_out_bytes_histogram": { 2024-04-07T17:51:23.940 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.940 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.940 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_out_bytes": { 2024-04-07T17:51:23.940 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1430791176 2024-04-07T17:51:23.940 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.940 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_prepare_latency": { 2024-04-07T17:51:23.940 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 39480, 2024-04-07T17:51:23.940 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 16088819423 2024-04-07T17:51:23.940 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.940 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_process_latency": { 2024-04-07T17:51:23.940 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 39255, 2024-04-07T17:51:23.940 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 13918274599 2024-04-07T17:51:23.940 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.941 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw": { 2024-04-07T17:51:23.941 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 13767 2024-04-07T17:51:23.941 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.941 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_in_bytes": { 2024-04-07T17:51:23.941 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 309949184 2024-04-07T17:51:23.941 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.941 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_latency": { 2024-04-07T17:51:23.941 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 13767, 2024-04-07T17:51:23.941 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 275992837370 2024-04-07T17:51:23.941 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.941 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_latency_in_bytes_histogram": { 2024-04-07T17:51:23.941 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.941 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.941 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_latency_out_bytes_histogram": { 2024-04-07T17:51:23.941 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.941 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.941 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_out_bytes": { 2024-04-07T17:51:23.942 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.942 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.942 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_prepare_latency": { 2024-04-07T17:51:23.942 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 15694, 2024-04-07T17:51:23.942 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 23263942693 2024-04-07T17:51:23.942 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.942 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_process_latency": { 2024-04-07T17:51:23.942 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 13767, 2024-04-07T17:51:23.942 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 118411226197 2024-04-07T17:51:23.942 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.942 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w": { 2024-04-07T17:51:23.942 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3942 2024-04-07T17:51:23.942 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.942 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_in_bytes": { 2024-04-07T17:51:23.942 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1175416337 2024-04-07T17:51:23.943 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.943 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_latency": { 2024-04-07T17:51:23.943 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 3942, 2024-04-07T17:51:23.943 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 54577688693 2024-04-07T17:51:23.943 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.943 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_latency_in_bytes_histogram": { 2024-04-07T17:51:23.943 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.943 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.943 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_prepare_latency": { 2024-04-07T17:51:23.943 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 4042, 2024-04-07T17:51:23.943 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 8852623176 2024-04-07T17:51:23.943 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.943 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_process_latency": { 2024-04-07T17:51:23.943 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 3942, 2024-04-07T17:51:23.943 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 37651015320 2024-04-07T17:51:23.944 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.944 INFO:tasks.workunit.client.0.smithi100.stdout: "op_wip": { 2024-04-07T17:51:23.944 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.944 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.944 INFO:tasks.workunit.client.0.smithi100.stdout: "recovery_bytes": { 2024-04-07T17:51:23.944 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 35 2024-04-07T17:51:23.944 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.944 INFO:tasks.workunit.client.0.smithi100.stdout: "recovery_ops": { 2024-04-07T17:51:23.944 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3 2024-04-07T17:51:23.944 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.944 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_bytes": { 2024-04-07T17:51:23.944 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 95995035648 2024-04-07T17:51:23.944 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.944 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_bytes_used": { 2024-04-07T17:51:23.944 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 459153408 2024-04-07T17:51:23.944 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:23.945 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:23.945 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.945 INFO:tasks.workunit.client.0.smithi100.stdout: "osd.4": { 2024-04-07T17:51:23.945 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs": { 2024-04-07T17:51:23.945 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_db_max_lat": { 2024-04-07T17:51:23.945 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 53233 2024-04-07T17:51:23.945 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.945 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_slow_fallback": { 2024-04-07T17:51:23.945 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.945 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.945 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_slow_max_lat": { 2024-04-07T17:51:23.945 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.945 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.945 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_slow_size_fallback": { 2024-04-07T17:51:23.945 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.945 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.946 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit_db": { 2024-04-07T17:51:23.946 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 65536 2024-04-07T17:51:23.946 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.946 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit_slow": { 2024-04-07T17:51:23.946 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.946 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.946 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit_wal": { 2024-04-07T17:51:23.946 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.946 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.946 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_wal_max_lat": { 2024-04-07T17:51:23.946 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.946 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.946 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_written_slow": { 2024-04-07T17:51:23.946 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.946 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.947 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_written_sst": { 2024-04-07T17:51:23.947 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 14217216 2024-04-07T17:51:23.947 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.947 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_written_wal": { 2024-04-07T17:51:23.947 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 588619776 2024-04-07T17:51:23.947 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.947 INFO:tasks.workunit.client.0.smithi100.stdout: "compact_lat": { 2024-04-07T17:51:23.947 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 14, 2024-04-07T17:51:23.947 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5250958 2024-04-07T17:51:23.947 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.947 INFO:tasks.workunit.client.0.smithi100.stdout: "compact_lock_lat": { 2024-04-07T17:51:23.947 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 14, 2024-04-07T17:51:23.947 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1869555 2024-04-07T17:51:23.947 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.947 INFO:tasks.workunit.client.0.smithi100.stdout: "db_alloc_lat": { 2024-04-07T17:51:23.947 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 167, 2024-04-07T17:51:23.948 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1195870 2024-04-07T17:51:23.948 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.948 INFO:tasks.workunit.client.0.smithi100.stdout: "db_total_bytes": { 2024-04-07T17:51:23.948 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 95995027456 2024-04-07T17:51:23.948 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.948 INFO:tasks.workunit.client.0.smithi100.stdout: "db_used_bytes": { 2024-04-07T17:51:23.948 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 572653568 2024-04-07T17:51:23.948 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.948 INFO:tasks.workunit.client.0.smithi100.stdout: "flush_lat": { 2024-04-07T17:51:23.948 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 60244, 2024-04-07T17:51:23.948 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 12173027260 2024-04-07T17:51:23.948 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.948 INFO:tasks.workunit.client.0.smithi100.stdout: "fsync_lat": { 2024-04-07T17:51:23.948 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 60248, 2024-04-07T17:51:23.948 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 21497842884 2024-04-07T17:51:23.948 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.949 INFO:tasks.workunit.client.0.smithi100.stdout: "log_bytes": { 2024-04-07T17:51:23.949 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 14090240 2024-04-07T17:51:23.949 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.949 INFO:tasks.workunit.client.0.smithi100.stdout: "logged_bytes": { 2024-04-07T17:51:23.949 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 246812672 2024-04-07T17:51:23.949 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.949 INFO:tasks.workunit.client.0.smithi100.stdout: "max_bytes_db": { 2024-04-07T17:51:23.949 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 568393728 2024-04-07T17:51:23.949 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.949 INFO:tasks.workunit.client.0.smithi100.stdout: "max_bytes_slow": { 2024-04-07T17:51:23.949 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.949 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.949 INFO:tasks.workunit.client.0.smithi100.stdout: "max_bytes_wal": { 2024-04-07T17:51:23.949 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.949 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.950 INFO:tasks.workunit.client.0.smithi100.stdout: "num_files": { 2024-04-07T17:51:23.950 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 50 2024-04-07T17:51:23.950 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.950 INFO:tasks.workunit.client.0.smithi100.stdout: "read_bytes": { 2024-04-07T17:51:23.950 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 10796585872 2024-04-07T17:51:23.950 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.950 INFO:tasks.workunit.client.0.smithi100.stdout: "read_count": { 2024-04-07T17:51:23.950 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 87890 2024-04-07T17:51:23.950 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.950 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes": { 2024-04-07T17:51:23.950 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 9426481152 2024-04-07T17:51:23.950 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.950 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes_db": { 2024-04-07T17:51:23.950 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 9426485248 2024-04-07T17:51:23.950 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.951 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes_slow": { 2024-04-07T17:51:23.951 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.951 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.951 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes_wal": { 2024-04-07T17:51:23.951 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.951 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.951 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_count": { 2024-04-07T17:51:23.951 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 80816 2024-04-07T17:51:23.951 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.951 INFO:tasks.workunit.client.0.smithi100.stdout: "read_lat": { 2024-04-07T17:51:23.951 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 87890, 2024-04-07T17:51:23.951 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5954312472 2024-04-07T17:51:23.951 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.951 INFO:tasks.workunit.client.0.smithi100.stdout: "read_prefetch_bytes": { 2024-04-07T17:51:23.951 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 10604152323 2024-04-07T17:51:23.951 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.952 INFO:tasks.workunit.client.0.smithi100.stdout: "read_prefetch_count": { 2024-04-07T17:51:23.952 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 82000 2024-04-07T17:51:23.952 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.952 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_buffer_bytes": { 2024-04-07T17:51:23.952 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4778070 2024-04-07T17:51:23.952 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.952 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_buffer_count": { 2024-04-07T17:51:23.952 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 805 2024-04-07T17:51:23.952 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.952 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_bytes": { 2024-04-07T17:51:23.952 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 14103486 2024-04-07T17:51:23.952 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.952 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_count": { 2024-04-07T17:51:23.952 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1149 2024-04-07T17:51:23.952 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.952 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes": { 2024-04-07T17:51:23.953 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 9325416 2024-04-07T17:51:23.953 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.953 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes_db": { 2024-04-07T17:51:23.953 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 9325416 2024-04-07T17:51:23.953 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.953 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes_slow": { 2024-04-07T17:51:23.953 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.953 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.953 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes_wal": { 2024-04-07T17:51:23.953 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.953 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.953 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_count": { 2024-04-07T17:51:23.953 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 361 2024-04-07T17:51:23.953 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.953 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_lat": { 2024-04-07T17:51:23.953 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1149, 2024-04-07T17:51:23.954 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4714569 2024-04-07T17:51:23.954 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.954 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_alloc_lat": { 2024-04-07T17:51:23.954 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:23.954 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.954 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.954 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_total_bytes": { 2024-04-07T17:51:23.954 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.954 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.954 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_used_bytes": { 2024-04-07T17:51:23.954 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.954 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.954 INFO:tasks.workunit.client.0.smithi100.stdout: "truncate_lat": { 2024-04-07T17:51:23.954 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1, 2024-04-07T17:51:23.954 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 731 2024-04-07T17:51:23.955 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.955 INFO:tasks.workunit.client.0.smithi100.stdout: "unlink_lat": { 2024-04-07T17:51:23.955 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 14, 2024-04-07T17:51:23.955 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 19309 2024-04-07T17:51:23.955 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.955 INFO:tasks.workunit.client.0.smithi100.stdout: "wal_alloc_lat": { 2024-04-07T17:51:23.955 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:23.955 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.955 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.955 INFO:tasks.workunit.client.0.smithi100.stdout: "wal_total_bytes": { 2024-04-07T17:51:23.955 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.955 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.955 INFO:tasks.workunit.client.0.smithi100.stdout: "wal_used_bytes": { 2024-04-07T17:51:23.955 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.955 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.956 INFO:tasks.workunit.client.0.smithi100.stdout: "write_bytes": { 2024-04-07T17:51:23.956 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 851795968 2024-04-07T17:51:23.956 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:23.956 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.956 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore": { 2024-04-07T17:51:23.956 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit": { 2024-04-07T17:51:23.956 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4096 2024-04-07T17:51:23.956 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.956 INFO:tasks.workunit.client.0.smithi100.stdout: "allocated": { 2024-04-07T17:51:23.956 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 14016512 2024-04-07T17:51:23.956 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.956 INFO:tasks.workunit.client.0.smithi100.stdout: "allocator_lat": { 2024-04-07T17:51:23.956 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 20261, 2024-04-07T17:51:23.956 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 99191054 2024-04-07T17:51:23.956 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.957 INFO:tasks.workunit.client.0.smithi100.stdout: "clist_lat": { 2024-04-07T17:51:23.957 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 3532, 2024-04-07T17:51:23.957 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 14554851866 2024-04-07T17:51:23.957 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.957 INFO:tasks.workunit.client.0.smithi100.stdout: "compress_lat": { 2024-04-07T17:51:23.957 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:23.957 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.957 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.957 INFO:tasks.workunit.client.0.smithi100.stdout: "compressed": { 2024-04-07T17:51:23.957 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.957 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.957 INFO:tasks.workunit.client.0.smithi100.stdout: "compressed_allocated": { 2024-04-07T17:51:23.957 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.957 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.957 INFO:tasks.workunit.client.0.smithi100.stdout: "compressed_original": { 2024-04-07T17:51:23.957 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.958 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.958 INFO:tasks.workunit.client.0.smithi100.stdout: "csum_lat": { 2024-04-07T17:51:23.958 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 12243, 2024-04-07T17:51:23.958 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 157875076 2024-04-07T17:51:23.958 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.958 INFO:tasks.workunit.client.0.smithi100.stdout: "decompress_lat": { 2024-04-07T17:51:23.958 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:23.958 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.958 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.958 INFO:tasks.workunit.client.0.smithi100.stdout: "fragmentation_micros": { 2024-04-07T17:51:23.958 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.958 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.958 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_commit_lat": { 2024-04-07T17:51:23.958 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 60186, 2024-04-07T17:51:23.959 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 27783488280 2024-04-07T17:51:23.959 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.959 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_final_lat": { 2024-04-07T17:51:23.959 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 59468, 2024-04-07T17:51:23.959 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3239794550 2024-04-07T17:51:23.959 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.959 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_flush_lat": { 2024-04-07T17:51:23.959 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 60186, 2024-04-07T17:51:23.959 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 106527146 2024-04-07T17:51:23.959 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.959 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_sync_lat": { 2024-04-07T17:51:23.959 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 60186, 2024-04-07T17:51:23.959 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 27890015426 2024-04-07T17:51:23.959 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.959 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_get_keys_lat": { 2024-04-07T17:51:23.959 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:23.960 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.960 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.960 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_get_values_lat": { 2024-04-07T17:51:23.960 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 22645, 2024-04-07T17:51:23.960 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1026880183 2024-04-07T17:51:23.960 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.960 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_lower_bound_lat": { 2024-04-07T17:51:23.960 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 697, 2024-04-07T17:51:23.960 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4156713 2024-04-07T17:51:23.960 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.960 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_next_lat": { 2024-04-07T17:51:23.960 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 13851, 2024-04-07T17:51:23.960 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 16294854 2024-04-07T17:51:23.960 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.960 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_seek_to_first_lat": { 2024-04-07T17:51:23.961 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 47, 2024-04-07T17:51:23.961 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 58460 2024-04-07T17:51:23.961 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.961 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_upper_bound_lat": { 2024-04-07T17:51:23.961 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 14286, 2024-04-07T17:51:23.961 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 6817372199 2024-04-07T17:51:23.961 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.961 INFO:tasks.workunit.client.0.smithi100.stdout: "onode_hits": { 2024-04-07T17:51:23.961 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 175006 2024-04-07T17:51:23.961 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.961 INFO:tasks.workunit.client.0.smithi100.stdout: "onode_misses": { 2024-04-07T17:51:23.961 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 19274 2024-04-07T17:51:23.961 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.961 INFO:tasks.workunit.client.0.smithi100.stdout: "read_lat": { 2024-04-07T17:51:23.961 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 8345, 2024-04-07T17:51:23.961 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1757977707 2024-04-07T17:51:23.962 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.962 INFO:tasks.workunit.client.0.smithi100.stdout: "read_onode_meta_lat": { 2024-04-07T17:51:23.962 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 35241, 2024-04-07T17:51:23.962 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 18704077 2024-04-07T17:51:23.962 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.962 INFO:tasks.workunit.client.0.smithi100.stdout: "read_wait_aio_lat": { 2024-04-07T17:51:23.962 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 26896, 2024-04-07T17:51:23.962 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1180234012 2024-04-07T17:51:23.962 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.962 INFO:tasks.workunit.client.0.smithi100.stdout: "reads_with_retries": { 2024-04-07T17:51:23.962 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.962 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.962 INFO:tasks.workunit.client.0.smithi100.stdout: "remove_lat": { 2024-04-07T17:51:23.962 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 13430, 2024-04-07T17:51:23.963 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 730792101 2024-04-07T17:51:23.963 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.963 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_aio_wait_count": { 2024-04-07T17:51:23.963 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.963 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.963 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_committed_kv_count": { 2024-04-07T17:51:23.963 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.963 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.963 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_read_onode_meta_count": { 2024-04-07T17:51:23.963 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.963 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.963 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_read_wait_aio_count": { 2024-04-07T17:51:23.963 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.963 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.963 INFO:tasks.workunit.client.0.smithi100.stdout: "state_aio_wait_lat": { 2024-04-07T17:51:23.963 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 69678, 2024-04-07T17:51:23.964 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 8516743959 2024-04-07T17:51:23.964 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.964 INFO:tasks.workunit.client.0.smithi100.stdout: "state_deferred_aio_wait_lat": { 2024-04-07T17:51:23.964 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 18006, 2024-04-07T17:51:23.964 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1525120051 2024-04-07T17:51:23.964 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.964 INFO:tasks.workunit.client.0.smithi100.stdout: "state_deferred_cleanup_lat": { 2024-04-07T17:51:23.964 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 18005, 2024-04-07T17:51:23.964 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4347507639061 2024-04-07T17:51:23.964 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.964 INFO:tasks.workunit.client.0.smithi100.stdout: "state_deferred_queued_lat": { 2024-04-07T17:51:23.964 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 18006, 2024-04-07T17:51:23.964 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 6594884467617 2024-04-07T17:51:23.964 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.964 INFO:tasks.workunit.client.0.smithi100.stdout: "state_done_lat": { 2024-04-07T17:51:23.964 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 69677, 2024-04-07T17:51:23.965 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3006179254631 2024-04-07T17:51:23.965 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.965 INFO:tasks.workunit.client.0.smithi100.stdout: "state_finishing_lat": { 2024-04-07T17:51:23.965 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 69677, 2024-04-07T17:51:23.965 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 16535015 2024-04-07T17:51:23.965 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.965 INFO:tasks.workunit.client.0.smithi100.stdout: "state_io_done_lat": { 2024-04-07T17:51:23.965 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 69678, 2024-04-07T17:51:23.965 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 283438924 2024-04-07T17:51:23.965 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.965 INFO:tasks.workunit.client.0.smithi100.stdout: "state_kv_commiting_lat": { 2024-04-07T17:51:23.965 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 69678, 2024-04-07T17:51:23.965 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 46634763201 2024-04-07T17:51:23.965 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.965 INFO:tasks.workunit.client.0.smithi100.stdout: "state_kv_done_lat": { 2024-04-07T17:51:23.965 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 69678, 2024-04-07T17:51:23.966 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 54811413 2024-04-07T17:51:23.966 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.966 INFO:tasks.workunit.client.0.smithi100.stdout: "state_kv_queued_lat": { 2024-04-07T17:51:23.966 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 69678, 2024-04-07T17:51:23.966 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 21669658103 2024-04-07T17:51:23.966 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.966 INFO:tasks.workunit.client.0.smithi100.stdout: "state_prepare_lat": { 2024-04-07T17:51:23.966 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 69678, 2024-04-07T17:51:23.966 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 25344251701 2024-04-07T17:51:23.966 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.966 INFO:tasks.workunit.client.0.smithi100.stdout: "stored": { 2024-04-07T17:51:23.966 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 9894713 2024-04-07T17:51:23.966 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.966 INFO:tasks.workunit.client.0.smithi100.stdout: "truncate_lat": { 2024-04-07T17:51:23.966 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 2371, 2024-04-07T17:51:23.967 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 24012279 2024-04-07T17:51:23.967 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.967 INFO:tasks.workunit.client.0.smithi100.stdout: "txc_commit_lat": { 2024-04-07T17:51:23.967 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 69678, 2024-04-07T17:51:23.967 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 102473756098 2024-04-07T17:51:23.967 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.967 INFO:tasks.workunit.client.0.smithi100.stdout: "txc_submit_lat": { 2024-04-07T17:51:23.967 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 69678, 2024-04-07T17:51:23.967 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 27569044264 2024-04-07T17:51:23.967 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.967 INFO:tasks.workunit.client.0.smithi100.stdout: "txc_throttle_lat": { 2024-04-07T17:51:23.967 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 69678, 2024-04-07T17:51:23.967 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 877546314 2024-04-07T17:51:23.967 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:23.967 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.967 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache": { 2024-04-07T17:51:23.968 INFO:tasks.workunit.client.0.smithi100.stdout: "cache_bytes": { 2024-04-07T17:51:23.968 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1444650649 2024-04-07T17:51:23.968 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.968 INFO:tasks.workunit.client.0.smithi100.stdout: "heap_bytes": { 2024-04-07T17:51:23.968 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 623263744 2024-04-07T17:51:23.968 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.968 INFO:tasks.workunit.client.0.smithi100.stdout: "mapped_bytes": { 2024-04-07T17:51:23.968 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 593174528 2024-04-07T17:51:23.968 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.968 INFO:tasks.workunit.client.0.smithi100.stdout: "target_bytes": { 2024-04-07T17:51:23.968 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2647008256 2024-04-07T17:51:23.968 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.968 INFO:tasks.workunit.client.0.smithi100.stdout: "unmapped_bytes": { 2024-04-07T17:51:23.968 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 30089216 2024-04-07T17:51:23.968 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:23.969 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.969 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:data": { 2024-04-07T17:51:23.969 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:23.969 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 134217728 2024-04-07T17:51:23.969 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.969 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:23.969 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.969 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.969 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:23.969 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.969 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.969 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:23.969 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 58883623 2024-04-07T17:51:23.969 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.969 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:23.970 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.970 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.970 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:23.970 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.970 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.970 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:23.970 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.970 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.970 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:23.970 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.970 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.970 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:23.970 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 28672 2024-04-07T17:51:23.970 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.971 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:23.971 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 757760 2024-04-07T17:51:23.971 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.971 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:23.971 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.971 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.971 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:23.971 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.971 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.971 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:23.971 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.971 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.971 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:23.971 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 74547673 2024-04-07T17:51:23.971 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:23.972 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.972 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:kv": { 2024-04-07T17:51:23.972 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:23.972 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 578813952 2024-04-07T17:51:23.972 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.972 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:23.972 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 67108864 2024-04-07T17:51:23.972 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.972 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:23.972 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.972 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.972 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:23.972 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 441627178 2024-04-07T17:51:23.972 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.973 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:23.973 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.973 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.973 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:23.973 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.973 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.973 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:23.973 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2416 2024-04-07T17:51:23.973 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.973 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:23.973 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.973 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.973 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:23.973 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 153616 2024-04-07T17:51:23.973 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.974 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:23.974 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.974 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.974 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:23.974 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.974 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.974 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:23.974 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.974 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.974 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:23.974 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.974 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.974 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:23.974 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 69921878 2024-04-07T17:51:23.974 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:23.975 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.975 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:kv_onode": { 2024-04-07T17:51:23.975 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:23.975 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 192937984 2024-04-07T17:51:23.975 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.975 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:23.975 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 67108864 2024-04-07T17:51:23.975 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.975 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:23.975 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 13844688 2024-04-07T17:51:23.975 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.975 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:23.975 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 39255749 2024-04-07T17:51:23.975 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.975 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:23.976 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.976 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.976 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:23.976 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.976 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.976 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:23.976 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.976 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.976 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:23.976 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.977 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.977 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:23.977 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.977 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.977 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:23.977 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.977 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.977 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:23.977 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.977 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.977 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:23.977 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.977 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.978 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:23.978 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.978 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.978 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:23.978 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 72728683 2024-04-07T17:51:23.978 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:23.978 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.979 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:meta": { 2024-04-07T17:51:23.979 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:23.979 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 511705088 2024-04-07T17:51:23.979 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.979 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:23.979 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.979 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.979 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:23.979 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.979 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.979 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:23.979 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 441627178 2024-04-07T17:51:23.979 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.980 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:23.980 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.980 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.980 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:23.980 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.980 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.980 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:23.980 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.980 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.980 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:23.980 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.981 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.981 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:23.981 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 158033 2024-04-07T17:51:23.981 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.981 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:23.981 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1110235 2024-04-07T17:51:23.981 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.981 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:23.981 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.981 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.981 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:23.981 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.982 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.982 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:23.982 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.982 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.982 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:23.982 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 68809642 2024-04-07T17:51:23.982 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:23.982 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.982 INFO:tasks.workunit.client.0.smithi100.stdout: "objecter": { 2024-04-07T17:51:23.982 INFO:tasks.workunit.client.0.smithi100.stdout: "op_active": { 2024-04-07T17:51:23.983 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.983 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.983 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r": { 2024-04-07T17:51:23.983 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 22 2024-04-07T17:51:23.983 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.983 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rmw": { 2024-04-07T17:51:23.983 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.983 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.983 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w": { 2024-04-07T17:51:23.983 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.983 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:23.983 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.983 INFO:tasks.workunit.client.0.smithi100.stdout: "osd": { 2024-04-07T17:51:23.983 INFO:tasks.workunit.client.0.smithi100.stdout: "numpg": { 2024-04-07T17:51:23.983 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 51 2024-04-07T17:51:23.983 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.984 INFO:tasks.workunit.client.0.smithi100.stdout: "numpg_removing": { 2024-04-07T17:51:23.984 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.984 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.984 INFO:tasks.workunit.client.0.smithi100.stdout: "op": { 2024-04-07T17:51:23.984 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 27984 2024-04-07T17:51:23.984 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.984 INFO:tasks.workunit.client.0.smithi100.stdout: "op_before_queue_op_lat": { 2024-04-07T17:51:23.984 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 87803, 2024-04-07T17:51:23.984 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 14036212222 2024-04-07T17:51:23.984 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.984 INFO:tasks.workunit.client.0.smithi100.stdout: "op_delayed_degraded": { 2024-04-07T17:51:23.984 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1 2024-04-07T17:51:23.984 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.984 INFO:tasks.workunit.client.0.smithi100.stdout: "op_delayed_unreadable": { 2024-04-07T17:51:23.984 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.985 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.985 INFO:tasks.workunit.client.0.smithi100.stdout: "op_in_bytes": { 2024-04-07T17:51:23.985 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1438594914 2024-04-07T17:51:23.985 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.985 INFO:tasks.workunit.client.0.smithi100.stdout: "op_latency": { 2024-04-07T17:51:23.985 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 27984, 2024-04-07T17:51:23.985 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 91475787538 2024-04-07T17:51:23.986 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.986 INFO:tasks.workunit.client.0.smithi100.stdout: "op_out_bytes": { 2024-04-07T17:51:23.986 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1486727151 2024-04-07T17:51:23.986 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.986 INFO:tasks.workunit.client.0.smithi100.stdout: "op_prepare_latency": { 2024-04-07T17:51:23.986 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 28568, 2024-04-07T17:51:23.986 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 27069205821 2024-04-07T17:51:23.986 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.987 INFO:tasks.workunit.client.0.smithi100.stdout: "op_process_latency": { 2024-04-07T17:51:23.987 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 27984, 2024-04-07T17:51:23.987 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 54395342610 2024-04-07T17:51:23.987 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.987 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r": { 2024-04-07T17:51:23.987 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 18559 2024-04-07T17:51:23.987 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.987 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_latency": { 2024-04-07T17:51:23.987 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 18559, 2024-04-07T17:51:23.987 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 15330456442 2024-04-07T17:51:23.987 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.987 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_latency_out_bytes_histogram": { 2024-04-07T17:51:23.987 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.987 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.987 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_out_bytes": { 2024-04-07T17:51:23.988 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1486727151 2024-04-07T17:51:23.988 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.988 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_prepare_latency": { 2024-04-07T17:51:23.988 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 18752, 2024-04-07T17:51:23.988 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 10042264838 2024-04-07T17:51:23.988 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.988 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_process_latency": { 2024-04-07T17:51:23.988 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 18559, 2024-04-07T17:51:23.988 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 9329326658 2024-04-07T17:51:23.988 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.988 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw": { 2024-04-07T17:51:23.988 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5511 2024-04-07T17:51:23.988 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.988 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_in_bytes": { 2024-04-07T17:51:23.988 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 266802303 2024-04-07T17:51:23.988 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.989 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_latency": { 2024-04-07T17:51:23.989 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 5511, 2024-04-07T17:51:23.989 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 19608757265 2024-04-07T17:51:23.989 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.989 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_latency_in_bytes_histogram": { 2024-04-07T17:51:23.989 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.989 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.989 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_latency_out_bytes_histogram": { 2024-04-07T17:51:23.989 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.989 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.989 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_out_bytes": { 2024-04-07T17:51:23.989 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.989 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.989 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_prepare_latency": { 2024-04-07T17:51:23.989 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 5777, 2024-04-07T17:51:23.989 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 7872110857 2024-04-07T17:51:23.990 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.990 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_process_latency": { 2024-04-07T17:51:23.990 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 5511, 2024-04-07T17:51:23.990 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 17096965462 2024-04-07T17:51:23.990 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.990 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w": { 2024-04-07T17:51:23.990 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3914 2024-04-07T17:51:23.990 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.990 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_in_bytes": { 2024-04-07T17:51:23.990 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1171792611 2024-04-07T17:51:23.990 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.990 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_latency": { 2024-04-07T17:51:23.990 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 3914, 2024-04-07T17:51:23.990 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 56536573831 2024-04-07T17:51:23.991 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.991 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_latency_in_bytes_histogram": { 2024-04-07T17:51:23.991 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.991 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.991 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_prepare_latency": { 2024-04-07T17:51:23.991 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 4039, 2024-04-07T17:51:23.991 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 9154830126 2024-04-07T17:51:23.991 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.991 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_process_latency": { 2024-04-07T17:51:23.991 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 3914, 2024-04-07T17:51:23.991 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 27969050490 2024-04-07T17:51:23.991 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.991 INFO:tasks.workunit.client.0.smithi100.stdout: "op_wip": { 2024-04-07T17:51:23.991 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.992 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.992 INFO:tasks.workunit.client.0.smithi100.stdout: "recovery_bytes": { 2024-04-07T17:51:23.992 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 8388656 2024-04-07T17:51:23.992 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.992 INFO:tasks.workunit.client.0.smithi100.stdout: "recovery_ops": { 2024-04-07T17:51:23.992 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 11 2024-04-07T17:51:23.992 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.992 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_bytes": { 2024-04-07T17:51:23.992 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 95995035648 2024-04-07T17:51:23.992 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.992 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_bytes_used": { 2024-04-07T17:51:23.992 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 586682368 2024-04-07T17:51:23.992 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:23.992 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:23.992 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.993 INFO:tasks.workunit.client.0.smithi100.stdout: "osd.5": { 2024-04-07T17:51:23.993 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs": { 2024-04-07T17:51:23.993 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_db_max_lat": { 2024-04-07T17:51:23.993 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 26003 2024-04-07T17:51:23.993 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.993 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_slow_fallback": { 2024-04-07T17:51:23.993 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.993 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.993 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_slow_max_lat": { 2024-04-07T17:51:23.993 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.993 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.993 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_slow_size_fallback": { 2024-04-07T17:51:23.993 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.993 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.993 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit_db": { 2024-04-07T17:51:23.994 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 65536 2024-04-07T17:51:23.994 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.994 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit_slow": { 2024-04-07T17:51:23.994 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.994 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.994 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit_wal": { 2024-04-07T17:51:23.994 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.994 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.994 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_wal_max_lat": { 2024-04-07T17:51:23.994 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.994 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.994 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_written_slow": { 2024-04-07T17:51:23.994 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.994 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.995 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_written_sst": { 2024-04-07T17:51:23.995 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 15654912 2024-04-07T17:51:23.995 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.995 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_written_wal": { 2024-04-07T17:51:23.995 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 603410432 2024-04-07T17:51:23.995 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.995 INFO:tasks.workunit.client.0.smithi100.stdout: "compact_lat": { 2024-04-07T17:51:23.995 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 19, 2024-04-07T17:51:23.995 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 8231783 2024-04-07T17:51:23.995 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.995 INFO:tasks.workunit.client.0.smithi100.stdout: "compact_lock_lat": { 2024-04-07T17:51:23.995 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 19, 2024-04-07T17:51:23.995 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2455108 2024-04-07T17:51:23.995 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.995 INFO:tasks.workunit.client.0.smithi100.stdout: "db_alloc_lat": { 2024-04-07T17:51:23.995 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 196, 2024-04-07T17:51:23.996 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1310582 2024-04-07T17:51:23.996 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.996 INFO:tasks.workunit.client.0.smithi100.stdout: "db_total_bytes": { 2024-04-07T17:51:23.996 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 95995027456 2024-04-07T17:51:23.996 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.996 INFO:tasks.workunit.client.0.smithi100.stdout: "db_used_bytes": { 2024-04-07T17:51:23.996 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 444465152 2024-04-07T17:51:23.996 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.996 INFO:tasks.workunit.client.0.smithi100.stdout: "flush_lat": { 2024-04-07T17:51:23.996 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 80665, 2024-04-07T17:51:23.996 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 15235152455 2024-04-07T17:51:23.996 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.996 INFO:tasks.workunit.client.0.smithi100.stdout: "fsync_lat": { 2024-04-07T17:51:23.996 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 80656, 2024-04-07T17:51:23.997 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 26858743324 2024-04-07T17:51:23.997 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.997 INFO:tasks.workunit.client.0.smithi100.stdout: "log_bytes": { 2024-04-07T17:51:23.997 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 14532608 2024-04-07T17:51:23.997 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.997 INFO:tasks.workunit.client.0.smithi100.stdout: "logged_bytes": { 2024-04-07T17:51:23.997 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 330424320 2024-04-07T17:51:23.998 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.998 INFO:tasks.workunit.client.0.smithi100.stdout: "max_bytes_db": { 2024-04-07T17:51:23.998 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 452984832 2024-04-07T17:51:23.998 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.998 INFO:tasks.workunit.client.0.smithi100.stdout: "max_bytes_slow": { 2024-04-07T17:51:23.998 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.998 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.998 INFO:tasks.workunit.client.0.smithi100.stdout: "max_bytes_wal": { 2024-04-07T17:51:23.998 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.998 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.998 INFO:tasks.workunit.client.0.smithi100.stdout: "num_files": { 2024-04-07T17:51:23.998 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 46 2024-04-07T17:51:23.998 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.998 INFO:tasks.workunit.client.0.smithi100.stdout: "read_bytes": { 2024-04-07T17:51:23.999 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 12789730188 2024-04-07T17:51:23.999 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.999 INFO:tasks.workunit.client.0.smithi100.stdout: "read_count": { 2024-04-07T17:51:23.999 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 100535 2024-04-07T17:51:23.999 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.999 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes": { 2024-04-07T17:51:23.999 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 10848047104 2024-04-07T17:51:23.999 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.999 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes_db": { 2024-04-07T17:51:23.999 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 10848051200 2024-04-07T17:51:23.999 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.999 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes_slow": { 2024-04-07T17:51:23.999 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:23.999 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:23.999 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes_wal": { 2024-04-07T17:51:23.999 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.000 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.000 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_count": { 2024-04-07T17:51:24.000 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 94858 2024-04-07T17:51:24.000 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.000 INFO:tasks.workunit.client.0.smithi100.stdout: "read_lat": { 2024-04-07T17:51:24.000 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 100535, 2024-04-07T17:51:24.000 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5637691717 2024-04-07T17:51:24.000 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.000 INFO:tasks.workunit.client.0.smithi100.stdout: "read_prefetch_bytes": { 2024-04-07T17:51:24.000 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 12578820194 2024-04-07T17:51:24.000 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.000 INFO:tasks.workunit.client.0.smithi100.stdout: "read_prefetch_count": { 2024-04-07T17:51:24.000 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 94081 2024-04-07T17:51:24.000 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.000 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_buffer_bytes": { 2024-04-07T17:51:24.001 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4494109 2024-04-07T17:51:24.001 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.001 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_buffer_count": { 2024-04-07T17:51:24.001 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 924 2024-04-07T17:51:24.001 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.001 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_bytes": { 2024-04-07T17:51:24.001 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 15530604 2024-04-07T17:51:24.001 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.001 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_count": { 2024-04-07T17:51:24.001 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1234 2024-04-07T17:51:24.001 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.001 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes": { 2024-04-07T17:51:24.001 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 11036495 2024-04-07T17:51:24.001 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.001 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes_db": { 2024-04-07T17:51:24.002 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 11036495 2024-04-07T17:51:24.002 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.002 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes_slow": { 2024-04-07T17:51:24.002 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.002 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.002 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes_wal": { 2024-04-07T17:51:24.002 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.002 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.002 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_count": { 2024-04-07T17:51:24.002 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 324 2024-04-07T17:51:24.002 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.002 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_lat": { 2024-04-07T17:51:24.002 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1234, 2024-04-07T17:51:24.002 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4942084 2024-04-07T17:51:24.003 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.003 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_alloc_lat": { 2024-04-07T17:51:24.003 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:24.003 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.003 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.003 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_total_bytes": { 2024-04-07T17:51:24.003 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.003 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.003 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_used_bytes": { 2024-04-07T17:51:24.003 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.003 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.003 INFO:tasks.workunit.client.0.smithi100.stdout: "truncate_lat": { 2024-04-07T17:51:24.003 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1, 2024-04-07T17:51:24.004 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 682 2024-04-07T17:51:24.004 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.004 INFO:tasks.workunit.client.0.smithi100.stdout: "unlink_lat": { 2024-04-07T17:51:24.004 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 15, 2024-04-07T17:51:24.004 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 19508 2024-04-07T17:51:24.004 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.004 INFO:tasks.workunit.client.0.smithi100.stdout: "wal_alloc_lat": { 2024-04-07T17:51:24.004 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:24.004 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.004 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.004 INFO:tasks.workunit.client.0.smithi100.stdout: "wal_total_bytes": { 2024-04-07T17:51:24.004 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.004 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.004 INFO:tasks.workunit.client.0.smithi100.stdout: "wal_used_bytes": { 2024-04-07T17:51:24.004 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.004 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.005 INFO:tasks.workunit.client.0.smithi100.stdout: "write_bytes": { 2024-04-07T17:51:24.005 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 952373248 2024-04-07T17:51:24.005 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:24.005 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.005 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore": { 2024-04-07T17:51:24.005 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit": { 2024-04-07T17:51:24.005 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4096 2024-04-07T17:51:24.005 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.005 INFO:tasks.workunit.client.0.smithi100.stdout: "allocated": { 2024-04-07T17:51:24.005 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 14020608 2024-04-07T17:51:24.005 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.005 INFO:tasks.workunit.client.0.smithi100.stdout: "allocator_lat": { 2024-04-07T17:51:24.005 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 22611, 2024-04-07T17:51:24.005 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 116632904 2024-04-07T17:51:24.005 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.006 INFO:tasks.workunit.client.0.smithi100.stdout: "clist_lat": { 2024-04-07T17:51:24.006 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 3546, 2024-04-07T17:51:24.006 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 17766845093 2024-04-07T17:51:24.006 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.006 INFO:tasks.workunit.client.0.smithi100.stdout: "compress_lat": { 2024-04-07T17:51:24.006 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:24.006 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.006 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.006 INFO:tasks.workunit.client.0.smithi100.stdout: "compressed": { 2024-04-07T17:51:24.006 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.006 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.006 INFO:tasks.workunit.client.0.smithi100.stdout: "compressed_allocated": { 2024-04-07T17:51:24.006 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.006 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.007 INFO:tasks.workunit.client.0.smithi100.stdout: "compressed_original": { 2024-04-07T17:51:24.007 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.007 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.007 INFO:tasks.workunit.client.0.smithi100.stdout: "csum_lat": { 2024-04-07T17:51:24.007 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 24579, 2024-04-07T17:51:24.007 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 325075173 2024-04-07T17:51:24.007 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.007 INFO:tasks.workunit.client.0.smithi100.stdout: "decompress_lat": { 2024-04-07T17:51:24.007 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:24.007 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.007 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.008 INFO:tasks.workunit.client.0.smithi100.stdout: "fragmentation_micros": { 2024-04-07T17:51:24.008 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.008 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.008 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_commit_lat": { 2024-04-07T17:51:24.008 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 80597, 2024-04-07T17:51:24.008 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 34623882580 2024-04-07T17:51:24.008 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.008 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_final_lat": { 2024-04-07T17:51:24.008 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 79698, 2024-04-07T17:51:24.008 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3958472053 2024-04-07T17:51:24.008 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.008 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_flush_lat": { 2024-04-07T17:51:24.008 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 80597, 2024-04-07T17:51:24.008 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 99120687 2024-04-07T17:51:24.008 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.009 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_sync_lat": { 2024-04-07T17:51:24.009 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 80597, 2024-04-07T17:51:24.009 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 34723003267 2024-04-07T17:51:24.009 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.009 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_get_keys_lat": { 2024-04-07T17:51:24.009 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:24.009 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.009 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.009 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_get_values_lat": { 2024-04-07T17:51:24.009 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 239680, 2024-04-07T17:51:24.009 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2166046359 2024-04-07T17:51:24.009 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.009 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_lower_bound_lat": { 2024-04-07T17:51:24.009 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 808, 2024-04-07T17:51:24.009 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5289872 2024-04-07T17:51:24.010 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.010 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_next_lat": { 2024-04-07T17:51:24.010 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 128881, 2024-04-07T17:51:24.010 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 97129343 2024-04-07T17:51:24.010 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.010 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_seek_to_first_lat": { 2024-04-07T17:51:24.010 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 50, 2024-04-07T17:51:24.010 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 74621 2024-04-07T17:51:24.010 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.010 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_upper_bound_lat": { 2024-04-07T17:51:24.010 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 19469, 2024-04-07T17:51:24.010 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 8325495929 2024-04-07T17:51:24.011 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.011 INFO:tasks.workunit.client.0.smithi100.stdout: "onode_hits": { 2024-04-07T17:51:24.011 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 436725 2024-04-07T17:51:24.011 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.011 INFO:tasks.workunit.client.0.smithi100.stdout: "onode_misses": { 2024-04-07T17:51:24.011 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 20982 2024-04-07T17:51:24.011 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.011 INFO:tasks.workunit.client.0.smithi100.stdout: "read_lat": { 2024-04-07T17:51:24.011 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 10659, 2024-04-07T17:51:24.011 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2220392001 2024-04-07T17:51:24.011 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.011 INFO:tasks.workunit.client.0.smithi100.stdout: "read_onode_meta_lat": { 2024-04-07T17:51:24.011 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 30132, 2024-04-07T17:51:24.011 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 15167976 2024-04-07T17:51:24.011 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.012 INFO:tasks.workunit.client.0.smithi100.stdout: "read_wait_aio_lat": { 2024-04-07T17:51:24.012 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 19473, 2024-04-07T17:51:24.012 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1807973266 2024-04-07T17:51:24.012 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.012 INFO:tasks.workunit.client.0.smithi100.stdout: "reads_with_retries": { 2024-04-07T17:51:24.012 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.012 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.012 INFO:tasks.workunit.client.0.smithi100.stdout: "remove_lat": { 2024-04-07T17:51:24.012 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 15467, 2024-04-07T17:51:24.012 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 823899553 2024-04-07T17:51:24.012 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.012 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_aio_wait_count": { 2024-04-07T17:51:24.012 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.012 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.012 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_committed_kv_count": { 2024-04-07T17:51:24.013 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.013 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.013 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_read_onode_meta_count": { 2024-04-07T17:51:24.013 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.013 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.013 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_read_wait_aio_count": { 2024-04-07T17:51:24.013 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.013 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.013 INFO:tasks.workunit.client.0.smithi100.stdout: "state_aio_wait_lat": { 2024-04-07T17:51:24.013 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 93751, 2024-04-07T17:51:24.013 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 13111865981 2024-04-07T17:51:24.013 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.013 INFO:tasks.workunit.client.0.smithi100.stdout: "state_deferred_aio_wait_lat": { 2024-04-07T17:51:24.013 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 8297, 2024-04-07T17:51:24.013 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1069282503 2024-04-07T17:51:24.014 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.014 INFO:tasks.workunit.client.0.smithi100.stdout: "state_deferred_cleanup_lat": { 2024-04-07T17:51:24.014 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 8296, 2024-04-07T17:51:24.014 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4488212413467 2024-04-07T17:51:24.014 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.014 INFO:tasks.workunit.client.0.smithi100.stdout: "state_deferred_queued_lat": { 2024-04-07T17:51:24.014 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 8297, 2024-04-07T17:51:24.014 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 6666966893111 2024-04-07T17:51:24.014 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.014 INFO:tasks.workunit.client.0.smithi100.stdout: "state_done_lat": { 2024-04-07T17:51:24.014 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 93750, 2024-04-07T17:51:24.014 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3192176271911 2024-04-07T17:51:24.014 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.015 INFO:tasks.workunit.client.0.smithi100.stdout: "state_finishing_lat": { 2024-04-07T17:51:24.015 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 93750, 2024-04-07T17:51:24.015 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 23533677 2024-04-07T17:51:24.015 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.015 INFO:tasks.workunit.client.0.smithi100.stdout: "state_io_done_lat": { 2024-04-07T17:51:24.015 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 93751, 2024-04-07T17:51:24.015 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 354231200 2024-04-07T17:51:24.015 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.015 INFO:tasks.workunit.client.0.smithi100.stdout: "state_kv_commiting_lat": { 2024-04-07T17:51:24.015 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 93751, 2024-04-07T17:51:24.015 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 60032684809 2024-04-07T17:51:24.015 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.015 INFO:tasks.workunit.client.0.smithi100.stdout: "state_kv_done_lat": { 2024-04-07T17:51:24.015 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 93751, 2024-04-07T17:51:24.015 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 73338408 2024-04-07T17:51:24.016 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.016 INFO:tasks.workunit.client.0.smithi100.stdout: "state_kv_queued_lat": { 2024-04-07T17:51:24.016 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 93751, 2024-04-07T17:51:24.016 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 29192592705 2024-04-07T17:51:24.016 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.016 INFO:tasks.workunit.client.0.smithi100.stdout: "state_prepare_lat": { 2024-04-07T17:51:24.016 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 93751, 2024-04-07T17:51:24.016 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 35861066911 2024-04-07T17:51:24.016 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.016 INFO:tasks.workunit.client.0.smithi100.stdout: "stored": { 2024-04-07T17:51:24.016 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 9899919 2024-04-07T17:51:24.016 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.016 INFO:tasks.workunit.client.0.smithi100.stdout: "truncate_lat": { 2024-04-07T17:51:24.016 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 2320, 2024-04-07T17:51:24.016 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 23026482 2024-04-07T17:51:24.017 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.017 INFO:tasks.workunit.client.0.smithi100.stdout: "txc_commit_lat": { 2024-04-07T17:51:24.017 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 93751, 2024-04-07T17:51:24.017 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 138586597593 2024-04-07T17:51:24.017 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.017 INFO:tasks.workunit.client.0.smithi100.stdout: "txc_submit_lat": { 2024-04-07T17:51:24.017 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 93751, 2024-04-07T17:51:24.017 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 38590847980 2024-04-07T17:51:24.017 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.017 INFO:tasks.workunit.client.0.smithi100.stdout: "txc_throttle_lat": { 2024-04-07T17:51:24.017 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 93751, 2024-04-07T17:51:24.017 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1304542608 2024-04-07T17:51:24.017 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:24.017 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.018 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache": { 2024-04-07T17:51:24.018 INFO:tasks.workunit.client.0.smithi100.stdout: "cache_bytes": { 2024-04-07T17:51:24.018 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1444650649 2024-04-07T17:51:24.018 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.018 INFO:tasks.workunit.client.0.smithi100.stdout: "heap_bytes": { 2024-04-07T17:51:24.018 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 570359808 2024-04-07T17:51:24.018 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.018 INFO:tasks.workunit.client.0.smithi100.stdout: "mapped_bytes": { 2024-04-07T17:51:24.019 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 557162496 2024-04-07T17:51:24.019 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.019 INFO:tasks.workunit.client.0.smithi100.stdout: "target_bytes": { 2024-04-07T17:51:24.019 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2647008256 2024-04-07T17:51:24.019 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.019 INFO:tasks.workunit.client.0.smithi100.stdout: "unmapped_bytes": { 2024-04-07T17:51:24.019 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 13197312 2024-04-07T17:51:24.019 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:24.019 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.019 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:data": { 2024-04-07T17:51:24.019 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:24.019 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 134217728 2024-04-07T17:51:24.019 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.019 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:24.019 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.020 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.020 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:24.020 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.020 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.020 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:24.020 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 58739848 2024-04-07T17:51:24.020 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.020 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:24.020 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.020 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.020 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:24.020 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.020 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.020 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:24.020 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.021 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.021 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:24.021 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.021 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.021 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:24.021 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4096 2024-04-07T17:51:24.021 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.021 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:24.021 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 24576 2024-04-07T17:51:24.021 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.021 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:24.021 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.021 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.021 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:24.021 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.022 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.022 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:24.022 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.022 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.022 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:24.022 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 75449208 2024-04-07T17:51:24.022 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:24.022 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.022 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:kv": { 2024-04-07T17:51:24.022 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:24.022 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 578813952 2024-04-07T17:51:24.022 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.022 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:24.022 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 67108864 2024-04-07T17:51:24.023 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.023 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:24.023 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.023 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.023 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:24.023 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 440548866 2024-04-07T17:51:24.023 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.023 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:24.023 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.023 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.023 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:24.023 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.023 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.023 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:24.023 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2128 2024-04-07T17:51:24.024 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.024 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:24.024 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.024 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.024 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:24.024 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 163344 2024-04-07T17:51:24.024 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.024 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:24.024 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.024 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.024 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:24.024 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.024 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.024 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:24.024 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.024 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.025 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:24.025 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.025 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.025 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:24.025 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 70990750 2024-04-07T17:51:24.025 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:24.025 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.025 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:kv_onode": { 2024-04-07T17:51:24.025 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:24.025 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 192937984 2024-04-07T17:51:24.025 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.025 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:24.025 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 67108864 2024-04-07T17:51:24.025 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.025 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:24.026 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 15205920 2024-04-07T17:51:24.026 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.026 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:24.026 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 39159899 2024-04-07T17:51:24.026 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.026 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:24.026 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.026 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.026 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:24.026 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.026 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.026 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:24.026 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.026 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.026 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:24.027 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.027 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.027 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:24.027 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.027 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.027 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:24.027 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.027 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.027 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:24.027 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.027 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.027 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:24.027 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.027 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.027 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:24.028 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.028 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.028 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:24.028 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 71463301 2024-04-07T17:51:24.028 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:24.028 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.028 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:meta": { 2024-04-07T17:51:24.028 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:24.028 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 511705088 2024-04-07T17:51:24.028 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.028 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:24.028 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.028 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.029 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:24.029 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.029 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.029 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:24.029 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 440548866 2024-04-07T17:51:24.029 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.029 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:24.030 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.030 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.030 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:24.030 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.030 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.030 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:24.030 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.031 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.031 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:24.031 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.031 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.031 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:24.032 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 156259 2024-04-07T17:51:24.032 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.032 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:24.032 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1093814 2024-04-07T17:51:24.032 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.032 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:24.032 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.032 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.033 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:24.033 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.033 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.033 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:24.033 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.033 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.033 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:24.033 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 69906149 2024-04-07T17:51:24.033 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:24.033 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.033 INFO:tasks.workunit.client.0.smithi100.stdout: "objecter": { 2024-04-07T17:51:24.033 INFO:tasks.workunit.client.0.smithi100.stdout: "op_active": { 2024-04-07T17:51:24.033 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.033 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.034 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r": { 2024-04-07T17:51:24.034 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 24 2024-04-07T17:51:24.034 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.034 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rmw": { 2024-04-07T17:51:24.034 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.034 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.034 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w": { 2024-04-07T17:51:24.034 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.034 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:24.034 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.034 INFO:tasks.workunit.client.0.smithi100.stdout: "osd": { 2024-04-07T17:51:24.034 INFO:tasks.workunit.client.0.smithi100.stdout: "numpg": { 2024-04-07T17:51:24.034 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 54 2024-04-07T17:51:24.034 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.035 INFO:tasks.workunit.client.0.smithi100.stdout: "numpg_removing": { 2024-04-07T17:51:24.035 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.035 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.035 INFO:tasks.workunit.client.0.smithi100.stdout: "op": { 2024-04-07T17:51:24.035 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 37568 2024-04-07T17:51:24.035 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.035 INFO:tasks.workunit.client.0.smithi100.stdout: "op_before_queue_op_lat": { 2024-04-07T17:51:24.035 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 118839, 2024-04-07T17:51:24.035 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 18250742844 2024-04-07T17:51:24.035 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.035 INFO:tasks.workunit.client.0.smithi100.stdout: "op_delayed_degraded": { 2024-04-07T17:51:24.035 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.035 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.035 INFO:tasks.workunit.client.0.smithi100.stdout: "op_delayed_unreadable": { 2024-04-07T17:51:24.035 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.036 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.036 INFO:tasks.workunit.client.0.smithi100.stdout: "op_in_bytes": { 2024-04-07T17:51:24.036 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1478194555 2024-04-07T17:51:24.036 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.036 INFO:tasks.workunit.client.0.smithi100.stdout: "op_latency": { 2024-04-07T17:51:24.036 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 37568, 2024-04-07T17:51:24.036 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 92458218161 2024-04-07T17:51:24.036 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.036 INFO:tasks.workunit.client.0.smithi100.stdout: "op_out_bytes": { 2024-04-07T17:51:24.036 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2285559046 2024-04-07T17:51:24.036 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.036 INFO:tasks.workunit.client.0.smithi100.stdout: "op_prepare_latency": { 2024-04-07T17:51:24.036 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 38251, 2024-04-07T17:51:24.036 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 45291105721 2024-04-07T17:51:24.037 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.037 INFO:tasks.workunit.client.0.smithi100.stdout: "op_process_latency": { 2024-04-07T17:51:24.037 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 37568, 2024-04-07T17:51:24.037 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 67926473334 2024-04-07T17:51:24.037 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.037 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r": { 2024-04-07T17:51:24.037 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 26708 2024-04-07T17:51:24.037 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.037 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_latency": { 2024-04-07T17:51:24.037 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 26708, 2024-04-07T17:51:24.037 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 37896778616 2024-04-07T17:51:24.037 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.037 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_latency_out_bytes_histogram": { 2024-04-07T17:51:24.037 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.038 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.038 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_out_bytes": { 2024-04-07T17:51:24.038 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2285559046 2024-04-07T17:51:24.038 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.038 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_prepare_latency": { 2024-04-07T17:51:24.038 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 26937, 2024-04-07T17:51:24.038 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 27634387557 2024-04-07T17:51:24.038 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.038 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_process_latency": { 2024-04-07T17:51:24.038 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 26708, 2024-04-07T17:51:24.038 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 26262052230 2024-04-07T17:51:24.038 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.038 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw": { 2024-04-07T17:51:24.038 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 9627 2024-04-07T17:51:24.038 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.039 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_in_bytes": { 2024-04-07T17:51:24.039 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 446562633 2024-04-07T17:51:24.039 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.039 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_latency": { 2024-04-07T17:51:24.039 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 9627, 2024-04-07T17:51:24.039 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 39696616362 2024-04-07T17:51:24.039 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.039 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_latency_in_bytes_histogram": { 2024-04-07T17:51:24.039 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.039 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.039 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_latency_out_bytes_histogram": { 2024-04-07T17:51:24.039 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.039 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.040 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_out_bytes": { 2024-04-07T17:51:24.040 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.040 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.040 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_prepare_latency": { 2024-04-07T17:51:24.040 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 9890, 2024-04-07T17:51:24.040 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 13660567957 2024-04-07T17:51:24.040 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.040 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_process_latency": { 2024-04-07T17:51:24.040 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 9627, 2024-04-07T17:51:24.040 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 30252117533 2024-04-07T17:51:24.040 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.040 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w": { 2024-04-07T17:51:24.041 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1233 2024-04-07T17:51:24.041 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.041 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_in_bytes": { 2024-04-07T17:51:24.041 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1031631922 2024-04-07T17:51:24.041 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.041 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_latency": { 2024-04-07T17:51:24.041 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1233, 2024-04-07T17:51:24.041 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 14864823183 2024-04-07T17:51:24.041 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.041 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_latency_in_bytes_histogram": { 2024-04-07T17:51:24.041 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.041 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.041 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_prepare_latency": { 2024-04-07T17:51:24.041 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1424, 2024-04-07T17:51:24.041 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3996150207 2024-04-07T17:51:24.041 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.042 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_process_latency": { 2024-04-07T17:51:24.042 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1233, 2024-04-07T17:51:24.042 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 11412303571 2024-04-07T17:51:24.042 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.042 INFO:tasks.workunit.client.0.smithi100.stdout: "op_wip": { 2024-04-07T17:51:24.042 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.042 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.042 INFO:tasks.workunit.client.0.smithi100.stdout: "recovery_bytes": { 2024-04-07T17:51:24.042 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 11394690 2024-04-07T17:51:24.042 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.042 INFO:tasks.workunit.client.0.smithi100.stdout: "recovery_ops": { 2024-04-07T17:51:24.043 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 18 2024-04-07T17:51:24.043 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.043 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_bytes": { 2024-04-07T17:51:24.043 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 95995035648 2024-04-07T17:51:24.043 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.043 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_bytes_used": { 2024-04-07T17:51:24.043 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 458498048 2024-04-07T17:51:24.043 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:24.043 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:24.043 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.043 INFO:tasks.workunit.client.0.smithi100.stdout: "osd.6": { 2024-04-07T17:51:24.043 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs": { 2024-04-07T17:51:24.044 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_db_max_lat": { 2024-04-07T17:51:24.044 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 42976 2024-04-07T17:51:24.044 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.044 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_slow_fallback": { 2024-04-07T17:51:24.044 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.044 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.044 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_slow_max_lat": { 2024-04-07T17:51:24.044 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.044 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.044 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_slow_size_fallback": { 2024-04-07T17:51:24.044 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.044 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.044 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit_db": { 2024-04-07T17:51:24.044 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 65536 2024-04-07T17:51:24.044 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.045 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit_slow": { 2024-04-07T17:51:24.045 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.045 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.045 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit_wal": { 2024-04-07T17:51:24.045 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.045 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.045 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_wal_max_lat": { 2024-04-07T17:51:24.045 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.045 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.045 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_written_slow": { 2024-04-07T17:51:24.045 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.045 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.045 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_written_sst": { 2024-04-07T17:51:24.045 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1253376 2024-04-07T17:51:24.045 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.045 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_written_wal": { 2024-04-07T17:51:24.046 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 458960896 2024-04-07T17:51:24.046 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.046 INFO:tasks.workunit.client.0.smithi100.stdout: "compact_lat": { 2024-04-07T17:51:24.046 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 15, 2024-04-07T17:51:24.046 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5459458 2024-04-07T17:51:24.046 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.046 INFO:tasks.workunit.client.0.smithi100.stdout: "compact_lock_lat": { 2024-04-07T17:51:24.046 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 15, 2024-04-07T17:51:24.046 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1804720 2024-04-07T17:51:24.046 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.046 INFO:tasks.workunit.client.0.smithi100.stdout: "db_alloc_lat": { 2024-04-07T17:51:24.046 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 135, 2024-04-07T17:51:24.046 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 985275 2024-04-07T17:51:24.046 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.047 INFO:tasks.workunit.client.0.smithi100.stdout: "db_total_bytes": { 2024-04-07T17:51:24.047 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 95995027456 2024-04-07T17:51:24.047 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.047 INFO:tasks.workunit.client.0.smithi100.stdout: "db_used_bytes": { 2024-04-07T17:51:24.047 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 343408640 2024-04-07T17:51:24.047 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.047 INFO:tasks.workunit.client.0.smithi100.stdout: "flush_lat": { 2024-04-07T17:51:24.047 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 61485, 2024-04-07T17:51:24.047 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 11982013100 2024-04-07T17:51:24.047 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.047 INFO:tasks.workunit.client.0.smithi100.stdout: "fsync_lat": { 2024-04-07T17:51:24.047 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 61497, 2024-04-07T17:51:24.047 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 21138088477 2024-04-07T17:51:24.047 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.047 INFO:tasks.workunit.client.0.smithi100.stdout: "log_bytes": { 2024-04-07T17:51:24.047 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2564096 2024-04-07T17:51:24.048 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.048 INFO:tasks.workunit.client.0.smithi100.stdout: "logged_bytes": { 2024-04-07T17:51:24.048 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 251932672 2024-04-07T17:51:24.048 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.048 INFO:tasks.workunit.client.0.smithi100.stdout: "max_bytes_db": { 2024-04-07T17:51:24.048 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 364707840 2024-04-07T17:51:24.048 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.048 INFO:tasks.workunit.client.0.smithi100.stdout: "max_bytes_slow": { 2024-04-07T17:51:24.048 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.048 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.048 INFO:tasks.workunit.client.0.smithi100.stdout: "max_bytes_wal": { 2024-04-07T17:51:24.048 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.048 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.048 INFO:tasks.workunit.client.0.smithi100.stdout: "num_files": { 2024-04-07T17:51:24.048 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 33 2024-04-07T17:51:24.049 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.049 INFO:tasks.workunit.client.0.smithi100.stdout: "read_bytes": { 2024-04-07T17:51:24.049 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 611394701 2024-04-07T17:51:24.049 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.049 INFO:tasks.workunit.client.0.smithi100.stdout: "read_count": { 2024-04-07T17:51:24.049 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 18243 2024-04-07T17:51:24.049 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.049 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes": { 2024-04-07T17:51:24.049 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 531197952 2024-04-07T17:51:24.049 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.049 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes_db": { 2024-04-07T17:51:24.049 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 531202048 2024-04-07T17:51:24.049 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.049 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes_slow": { 2024-04-07T17:51:24.049 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.049 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.050 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes_wal": { 2024-04-07T17:51:24.050 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.050 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.050 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_count": { 2024-04-07T17:51:24.050 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 16144 2024-04-07T17:51:24.050 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.050 INFO:tasks.workunit.client.0.smithi100.stdout: "read_lat": { 2024-04-07T17:51:24.050 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 18243, 2024-04-07T17:51:24.050 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 442981158 2024-04-07T17:51:24.050 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.050 INFO:tasks.workunit.client.0.smithi100.stdout: "read_prefetch_bytes": { 2024-04-07T17:51:24.050 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 565009679 2024-04-07T17:51:24.050 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.050 INFO:tasks.workunit.client.0.smithi100.stdout: "read_prefetch_count": { 2024-04-07T17:51:24.051 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 16814 2024-04-07T17:51:24.051 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.051 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_buffer_bytes": { 2024-04-07T17:51:24.051 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 563551 2024-04-07T17:51:24.051 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.051 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_buffer_count": { 2024-04-07T17:51:24.051 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 339 2024-04-07T17:51:24.051 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.051 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_bytes": { 2024-04-07T17:51:24.051 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1239831 2024-04-07T17:51:24.052 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.052 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_count": { 2024-04-07T17:51:24.052 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 451 2024-04-07T17:51:24.052 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.052 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes": { 2024-04-07T17:51:24.052 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 676280 2024-04-07T17:51:24.052 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.052 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes_db": { 2024-04-07T17:51:24.052 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 676280 2024-04-07T17:51:24.052 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.052 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes_slow": { 2024-04-07T17:51:24.052 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.052 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.052 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes_wal": { 2024-04-07T17:51:24.052 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.053 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.053 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_count": { 2024-04-07T17:51:24.053 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 112 2024-04-07T17:51:24.053 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.053 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_lat": { 2024-04-07T17:51:24.053 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 451, 2024-04-07T17:51:24.053 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 827027 2024-04-07T17:51:24.053 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.053 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_alloc_lat": { 2024-04-07T17:51:24.053 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:24.054 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.054 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.054 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_total_bytes": { 2024-04-07T17:51:24.054 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.054 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.054 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_used_bytes": { 2024-04-07T17:51:24.054 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.054 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.054 INFO:tasks.workunit.client.0.smithi100.stdout: "truncate_lat": { 2024-04-07T17:51:24.054 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1, 2024-04-07T17:51:24.054 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 646 2024-04-07T17:51:24.054 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.054 INFO:tasks.workunit.client.0.smithi100.stdout: "unlink_lat": { 2024-04-07T17:51:24.055 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 5, 2024-04-07T17:51:24.055 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 8184 2024-04-07T17:51:24.055 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.055 INFO:tasks.workunit.client.0.smithi100.stdout: "wal_alloc_lat": { 2024-04-07T17:51:24.055 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:24.055 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.055 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.055 INFO:tasks.workunit.client.0.smithi100.stdout: "wal_total_bytes": { 2024-04-07T17:51:24.055 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.055 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.055 INFO:tasks.workunit.client.0.smithi100.stdout: "wal_used_bytes": { 2024-04-07T17:51:24.055 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.055 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.056 INFO:tasks.workunit.client.0.smithi100.stdout: "write_bytes": { 2024-04-07T17:51:24.056 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 714424320 2024-04-07T17:51:24.056 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:24.056 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.056 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore": { 2024-04-07T17:51:24.056 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit": { 2024-04-07T17:51:24.056 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4096 2024-04-07T17:51:24.056 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.056 INFO:tasks.workunit.client.0.smithi100.stdout: "allocated": { 2024-04-07T17:51:24.056 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 14606336 2024-04-07T17:51:24.056 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.056 INFO:tasks.workunit.client.0.smithi100.stdout: "allocator_lat": { 2024-04-07T17:51:24.056 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 16890, 2024-04-07T17:51:24.057 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 74243402 2024-04-07T17:51:24.057 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.057 INFO:tasks.workunit.client.0.smithi100.stdout: "clist_lat": { 2024-04-07T17:51:24.057 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 3441, 2024-04-07T17:51:24.057 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 6473693390 2024-04-07T17:51:24.057 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.057 INFO:tasks.workunit.client.0.smithi100.stdout: "compress_lat": { 2024-04-07T17:51:24.057 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:24.057 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.057 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.057 INFO:tasks.workunit.client.0.smithi100.stdout: "compressed": { 2024-04-07T17:51:24.057 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.057 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.057 INFO:tasks.workunit.client.0.smithi100.stdout: "compressed_allocated": { 2024-04-07T17:51:24.058 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.058 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.058 INFO:tasks.workunit.client.0.smithi100.stdout: "compressed_original": { 2024-04-07T17:51:24.058 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.058 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.058 INFO:tasks.workunit.client.0.smithi100.stdout: "csum_lat": { 2024-04-07T17:51:24.058 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 14729, 2024-04-07T17:51:24.058 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 190648102 2024-04-07T17:51:24.058 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.058 INFO:tasks.workunit.client.0.smithi100.stdout: "decompress_lat": { 2024-04-07T17:51:24.058 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:24.058 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.058 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.058 INFO:tasks.workunit.client.0.smithi100.stdout: "fragmentation_micros": { 2024-04-07T17:51:24.059 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.059 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.059 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_commit_lat": { 2024-04-07T17:51:24.059 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 61461, 2024-04-07T17:51:24.059 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 27465827150 2024-04-07T17:51:24.059 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.059 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_final_lat": { 2024-04-07T17:51:24.059 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 60716, 2024-04-07T17:51:24.059 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3226568509 2024-04-07T17:51:24.059 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.059 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_flush_lat": { 2024-04-07T17:51:24.059 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 61461, 2024-04-07T17:51:24.059 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 74420054 2024-04-07T17:51:24.059 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.059 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_sync_lat": { 2024-04-07T17:51:24.060 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 61461, 2024-04-07T17:51:24.060 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 27540247204 2024-04-07T17:51:24.060 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.060 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_get_keys_lat": { 2024-04-07T17:51:24.060 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:24.060 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.060 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.060 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_get_values_lat": { 2024-04-07T17:51:24.060 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 22581, 2024-04-07T17:51:24.060 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 685980505 2024-04-07T17:51:24.060 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.060 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_lower_bound_lat": { 2024-04-07T17:51:24.061 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 700, 2024-04-07T17:51:24.061 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5026445 2024-04-07T17:51:24.061 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.061 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_next_lat": { 2024-04-07T17:51:24.061 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 10108, 2024-04-07T17:51:24.061 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 12802315 2024-04-07T17:51:24.061 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.061 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_seek_to_first_lat": { 2024-04-07T17:51:24.061 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 43, 2024-04-07T17:51:24.061 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 77153 2024-04-07T17:51:24.061 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.061 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_upper_bound_lat": { 2024-04-07T17:51:24.061 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 10657, 2024-04-07T17:51:24.061 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1970969784 2024-04-07T17:51:24.062 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.062 INFO:tasks.workunit.client.0.smithi100.stdout: "onode_hits": { 2024-04-07T17:51:24.062 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 175572 2024-04-07T17:51:24.062 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.062 INFO:tasks.workunit.client.0.smithi100.stdout: "onode_misses": { 2024-04-07T17:51:24.062 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 18139 2024-04-07T17:51:24.062 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.062 INFO:tasks.workunit.client.0.smithi100.stdout: "read_lat": { 2024-04-07T17:51:24.062 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 11039, 2024-04-07T17:51:24.062 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1962439157 2024-04-07T17:51:24.062 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.062 INFO:tasks.workunit.client.0.smithi100.stdout: "read_onode_meta_lat": { 2024-04-07T17:51:24.062 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 38635, 2024-04-07T17:51:24.063 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 20820077 2024-04-07T17:51:24.063 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.063 INFO:tasks.workunit.client.0.smithi100.stdout: "read_wait_aio_lat": { 2024-04-07T17:51:24.063 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 27596, 2024-04-07T17:51:24.063 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1296711458 2024-04-07T17:51:24.063 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.063 INFO:tasks.workunit.client.0.smithi100.stdout: "reads_with_retries": { 2024-04-07T17:51:24.063 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.063 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.063 INFO:tasks.workunit.client.0.smithi100.stdout: "remove_lat": { 2024-04-07T17:51:24.063 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 11796, 2024-04-07T17:51:24.063 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 656690853 2024-04-07T17:51:24.063 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.063 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_aio_wait_count": { 2024-04-07T17:51:24.064 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.064 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.064 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_committed_kv_count": { 2024-04-07T17:51:24.064 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.064 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.064 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_read_onode_meta_count": { 2024-04-07T17:51:24.064 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.064 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.064 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_read_wait_aio_count": { 2024-04-07T17:51:24.064 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.064 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.065 INFO:tasks.workunit.client.0.smithi100.stdout: "state_aio_wait_lat": { 2024-04-07T17:51:24.065 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 74635, 2024-04-07T17:51:24.065 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 10364611458 2024-04-07T17:51:24.065 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.065 INFO:tasks.workunit.client.0.smithi100.stdout: "state_deferred_aio_wait_lat": { 2024-04-07T17:51:24.065 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 13939, 2024-04-07T17:51:24.065 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1372751186 2024-04-07T17:51:24.065 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.065 INFO:tasks.workunit.client.0.smithi100.stdout: "state_deferred_cleanup_lat": { 2024-04-07T17:51:24.065 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 13938, 2024-04-07T17:51:24.065 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3655450811587 2024-04-07T17:51:24.065 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.065 INFO:tasks.workunit.client.0.smithi100.stdout: "state_deferred_queued_lat": { 2024-04-07T17:51:24.065 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 13939, 2024-04-07T17:51:24.065 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5923221764357 2024-04-07T17:51:24.066 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.066 INFO:tasks.workunit.client.0.smithi100.stdout: "state_done_lat": { 2024-04-07T17:51:24.066 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 74634, 2024-04-07T17:51:24.066 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2803609494529 2024-04-07T17:51:24.066 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.066 INFO:tasks.workunit.client.0.smithi100.stdout: "state_finishing_lat": { 2024-04-07T17:51:24.066 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 74634, 2024-04-07T17:51:24.066 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 20557962 2024-04-07T17:51:24.066 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.066 INFO:tasks.workunit.client.0.smithi100.stdout: "state_io_done_lat": { 2024-04-07T17:51:24.066 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 74635, 2024-04-07T17:51:24.066 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 593824141 2024-04-07T17:51:24.066 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.066 INFO:tasks.workunit.client.0.smithi100.stdout: "state_kv_commiting_lat": { 2024-04-07T17:51:24.067 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 74635, 2024-04-07T17:51:24.067 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 51948333590 2024-04-07T17:51:24.067 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.067 INFO:tasks.workunit.client.0.smithi100.stdout: "state_kv_done_lat": { 2024-04-07T17:51:24.067 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 74635, 2024-04-07T17:51:24.067 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 60647317 2024-04-07T17:51:24.067 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.067 INFO:tasks.workunit.client.0.smithi100.stdout: "state_kv_queued_lat": { 2024-04-07T17:51:24.067 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 74635, 2024-04-07T17:51:24.067 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 28224012224 2024-04-07T17:51:24.067 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.067 INFO:tasks.workunit.client.0.smithi100.stdout: "state_prepare_lat": { 2024-04-07T17:51:24.067 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 74635, 2024-04-07T17:51:24.067 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 21897965540 2024-04-07T17:51:24.067 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.067 INFO:tasks.workunit.client.0.smithi100.stdout: "stored": { 2024-04-07T17:51:24.068 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 10490178 2024-04-07T17:51:24.068 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.068 INFO:tasks.workunit.client.0.smithi100.stdout: "truncate_lat": { 2024-04-07T17:51:24.068 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 2295, 2024-04-07T17:51:24.068 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 25153482 2024-04-07T17:51:24.068 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.068 INFO:tasks.workunit.client.0.smithi100.stdout: "txc_commit_lat": { 2024-04-07T17:51:24.068 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 74635, 2024-04-07T17:51:24.068 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 113055819629 2024-04-07T17:51:24.068 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.068 INFO:tasks.workunit.client.0.smithi100.stdout: "txc_submit_lat": { 2024-04-07T17:51:24.068 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 74635, 2024-04-07T17:51:24.068 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 23968070054 2024-04-07T17:51:24.068 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.068 INFO:tasks.workunit.client.0.smithi100.stdout: "txc_throttle_lat": { 2024-04-07T17:51:24.069 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 74635, 2024-04-07T17:51:24.069 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1130773563 2024-04-07T17:51:24.069 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:24.069 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.069 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache": { 2024-04-07T17:51:24.069 INFO:tasks.workunit.client.0.smithi100.stdout: "cache_bytes": { 2024-04-07T17:51:24.069 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1444650649 2024-04-07T17:51:24.069 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.069 INFO:tasks.workunit.client.0.smithi100.stdout: "heap_bytes": { 2024-04-07T17:51:24.069 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 423641088 2024-04-07T17:51:24.069 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.069 INFO:tasks.workunit.client.0.smithi100.stdout: "mapped_bytes": { 2024-04-07T17:51:24.069 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 358727680 2024-04-07T17:51:24.069 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.069 INFO:tasks.workunit.client.0.smithi100.stdout: "target_bytes": { 2024-04-07T17:51:24.070 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2647008256 2024-04-07T17:51:24.070 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.070 INFO:tasks.workunit.client.0.smithi100.stdout: "unmapped_bytes": { 2024-04-07T17:51:24.070 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 64913408 2024-04-07T17:51:24.070 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:24.070 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.070 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:data": { 2024-04-07T17:51:24.070 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:24.070 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 134217728 2024-04-07T17:51:24.070 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.071 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:24.071 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.071 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.071 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:24.071 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.071 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.071 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:24.071 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 59908063 2024-04-07T17:51:24.071 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.071 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:24.071 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.071 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.071 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:24.071 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.071 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.072 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:24.072 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.072 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.072 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:24.072 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.072 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.072 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:24.072 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.072 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.072 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:24.072 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 184320 2024-04-07T17:51:24.073 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.073 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:24.073 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.073 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.073 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:24.073 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.073 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.073 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:24.073 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.073 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.073 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:24.073 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 74125345 2024-04-07T17:51:24.073 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:24.073 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.074 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:kv": { 2024-04-07T17:51:24.074 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:24.074 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 587202560 2024-04-07T17:51:24.074 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.074 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:24.074 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 67108864 2024-04-07T17:51:24.074 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.074 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:24.074 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.075 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.075 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:24.075 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 449310472 2024-04-07T17:51:24.075 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.075 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:24.075 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.075 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.075 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:24.075 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.075 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.075 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:24.075 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1040 2024-04-07T17:51:24.075 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.076 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:24.076 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.076 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.076 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:24.076 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 66384 2024-04-07T17:51:24.076 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.076 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:24.076 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.076 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.076 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:24.076 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.076 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.076 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:24.076 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.077 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.077 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:24.077 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.077 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.077 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:24.077 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 70715800 2024-04-07T17:51:24.077 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:24.077 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.077 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:kv_onode": { 2024-04-07T17:51:24.077 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:24.077 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 176160768 2024-04-07T17:51:24.077 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.077 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:24.077 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 67108864 2024-04-07T17:51:24.077 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.078 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:24.078 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 691824 2024-04-07T17:51:24.078 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.078 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:24.078 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 39938708 2024-04-07T17:51:24.078 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.078 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:24.078 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.078 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.078 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:24.078 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.078 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.078 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:24.078 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.079 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.079 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:24.079 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.079 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.079 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:24.079 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.079 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.079 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:24.079 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.079 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.079 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:24.079 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.079 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.079 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:24.079 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.079 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.080 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:24.080 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.080 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.080 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:24.080 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 68421372 2024-04-07T17:51:24.080 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:24.080 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.080 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:meta": { 2024-04-07T17:51:24.080 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:24.080 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 520093696 2024-04-07T17:51:24.081 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.081 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:24.081 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.081 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.081 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:24.081 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.081 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.081 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:24.081 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 449310472 2024-04-07T17:51:24.081 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.081 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:24.081 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.081 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.081 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:24.081 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.082 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.082 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:24.082 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.082 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.082 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:24.082 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.082 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.082 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:24.082 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 157124 2024-04-07T17:51:24.082 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.082 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:24.082 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1119133 2024-04-07T17:51:24.082 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.082 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:24.083 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.083 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.083 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:24.083 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.083 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.083 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:24.083 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.083 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.083 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:24.083 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 69506967 2024-04-07T17:51:24.083 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:24.083 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.083 INFO:tasks.workunit.client.0.smithi100.stdout: "objecter": { 2024-04-07T17:51:24.083 INFO:tasks.workunit.client.0.smithi100.stdout: "op_active": { 2024-04-07T17:51:24.083 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.083 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.084 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r": { 2024-04-07T17:51:24.084 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5 2024-04-07T17:51:24.084 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.084 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rmw": { 2024-04-07T17:51:24.084 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.084 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.084 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w": { 2024-04-07T17:51:24.084 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.084 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:24.084 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.084 INFO:tasks.workunit.client.0.smithi100.stdout: "osd": { 2024-04-07T17:51:24.084 INFO:tasks.workunit.client.0.smithi100.stdout: "numpg": { 2024-04-07T17:51:24.084 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 54 2024-04-07T17:51:24.084 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.084 INFO:tasks.workunit.client.0.smithi100.stdout: "numpg_removing": { 2024-04-07T17:51:24.084 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.085 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.085 INFO:tasks.workunit.client.0.smithi100.stdout: "op": { 2024-04-07T17:51:24.085 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 33509 2024-04-07T17:51:24.085 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.085 INFO:tasks.workunit.client.0.smithi100.stdout: "op_before_queue_op_lat": { 2024-04-07T17:51:24.085 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 91089, 2024-04-07T17:51:24.085 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 12791557788 2024-04-07T17:51:24.085 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.085 INFO:tasks.workunit.client.0.smithi100.stdout: "op_delayed_degraded": { 2024-04-07T17:51:24.085 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.086 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.086 INFO:tasks.workunit.client.0.smithi100.stdout: "op_delayed_unreadable": { 2024-04-07T17:51:24.086 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.086 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.086 INFO:tasks.workunit.client.0.smithi100.stdout: "op_in_bytes": { 2024-04-07T17:51:24.086 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1535388602 2024-04-07T17:51:24.086 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.086 INFO:tasks.workunit.client.0.smithi100.stdout: "op_latency": { 2024-04-07T17:51:24.086 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 33509, 2024-04-07T17:51:24.086 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 131541624216 2024-04-07T17:51:24.086 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.086 INFO:tasks.workunit.client.0.smithi100.stdout: "op_out_bytes": { 2024-04-07T17:51:24.086 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1703112431 2024-04-07T17:51:24.086 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.087 INFO:tasks.workunit.client.0.smithi100.stdout: "op_prepare_latency": { 2024-04-07T17:51:24.087 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 34219, 2024-04-07T17:51:24.087 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 27830378329 2024-04-07T17:51:24.087 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.087 INFO:tasks.workunit.client.0.smithi100.stdout: "op_process_latency": { 2024-04-07T17:51:24.087 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 33509, 2024-04-07T17:51:24.087 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 53861705532 2024-04-07T17:51:24.087 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.087 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r": { 2024-04-07T17:51:24.087 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 25245 2024-04-07T17:51:24.087 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.087 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_latency": { 2024-04-07T17:51:24.087 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 25245, 2024-04-07T17:51:24.087 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 23376821581 2024-04-07T17:51:24.087 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.087 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_latency_out_bytes_histogram": { 2024-04-07T17:51:24.088 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.088 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.088 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_out_bytes": { 2024-04-07T17:51:24.088 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1703112431 2024-04-07T17:51:24.088 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.088 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_prepare_latency": { 2024-04-07T17:51:24.088 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 25399, 2024-04-07T17:51:24.088 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 11923589716 2024-04-07T17:51:24.088 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.088 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_process_latency": { 2024-04-07T17:51:24.088 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 25245, 2024-04-07T17:51:24.088 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 10146174967 2024-04-07T17:51:24.088 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.088 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw": { 2024-04-07T17:51:24.088 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4064 2024-04-07T17:51:24.089 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.089 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_in_bytes": { 2024-04-07T17:51:24.089 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 290840421 2024-04-07T17:51:24.089 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.089 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_latency": { 2024-04-07T17:51:24.089 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 4064, 2024-04-07T17:51:24.089 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 20965538275 2024-04-07T17:51:24.089 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.089 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_latency_in_bytes_histogram": { 2024-04-07T17:51:24.089 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.089 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.089 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_latency_out_bytes_histogram": { 2024-04-07T17:51:24.089 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.089 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.089 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_out_bytes": { 2024-04-07T17:51:24.089 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.090 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.090 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_prepare_latency": { 2024-04-07T17:51:24.090 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 4504, 2024-04-07T17:51:24.090 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 6305172918 2024-04-07T17:51:24.090 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.090 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_process_latency": { 2024-04-07T17:51:24.090 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 4064, 2024-04-07T17:51:24.090 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 14864893688 2024-04-07T17:51:24.090 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.090 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w": { 2024-04-07T17:51:24.090 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4200 2024-04-07T17:51:24.090 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.090 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_in_bytes": { 2024-04-07T17:51:24.091 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1244548181 2024-04-07T17:51:24.091 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.091 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_latency": { 2024-04-07T17:51:24.091 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 4200, 2024-04-07T17:51:24.091 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 87199264360 2024-04-07T17:51:24.091 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.091 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_latency_in_bytes_histogram": { 2024-04-07T17:51:24.091 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.091 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.091 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_prepare_latency": { 2024-04-07T17:51:24.091 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 4316, 2024-04-07T17:51:24.091 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 9601615695 2024-04-07T17:51:24.091 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.092 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_process_latency": { 2024-04-07T17:51:24.092 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 4200, 2024-04-07T17:51:24.092 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 28850636877 2024-04-07T17:51:24.092 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.092 INFO:tasks.workunit.client.0.smithi100.stdout: "op_wip": { 2024-04-07T17:51:24.092 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.092 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.092 INFO:tasks.workunit.client.0.smithi100.stdout: "recovery_bytes": { 2024-04-07T17:51:24.092 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 73766334 2024-04-07T17:51:24.092 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.092 INFO:tasks.workunit.client.0.smithi100.stdout: "recovery_ops": { 2024-04-07T17:51:24.092 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 34 2024-04-07T17:51:24.092 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.092 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_bytes": { 2024-04-07T17:51:24.092 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 95995035648 2024-04-07T17:51:24.093 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.093 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_bytes_used": { 2024-04-07T17:51:24.093 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 358027264 2024-04-07T17:51:24.093 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:24.093 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:24.093 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.093 INFO:tasks.workunit.client.0.smithi100.stdout: "osd.7": { 2024-04-07T17:51:24.093 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs": { 2024-04-07T17:51:24.093 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_db_max_lat": { 2024-04-07T17:51:24.093 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1425185 2024-04-07T17:51:24.093 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.093 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_slow_fallback": { 2024-04-07T17:51:24.093 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.093 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.093 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_slow_max_lat": { 2024-04-07T17:51:24.094 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.094 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.094 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_slow_size_fallback": { 2024-04-07T17:51:24.094 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.094 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.094 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit_db": { 2024-04-07T17:51:24.094 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 65536 2024-04-07T17:51:24.094 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.094 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit_slow": { 2024-04-07T17:51:24.094 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.094 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.094 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit_wal": { 2024-04-07T17:51:24.094 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.094 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.095 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_wal_max_lat": { 2024-04-07T17:51:24.095 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.095 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.095 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_written_slow": { 2024-04-07T17:51:24.095 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.095 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.095 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_written_sst": { 2024-04-07T17:51:24.095 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5050368 2024-04-07T17:51:24.095 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.095 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_written_wal": { 2024-04-07T17:51:24.095 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 495513600 2024-04-07T17:51:24.095 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.096 INFO:tasks.workunit.client.0.smithi100.stdout: "compact_lat": { 2024-04-07T17:51:24.096 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 15, 2024-04-07T17:51:24.096 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 6850825 2024-04-07T17:51:24.096 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.096 INFO:tasks.workunit.client.0.smithi100.stdout: "compact_lock_lat": { 2024-04-07T17:51:24.096 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 15, 2024-04-07T17:51:24.096 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1769094 2024-04-07T17:51:24.096 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.096 INFO:tasks.workunit.client.0.smithi100.stdout: "db_alloc_lat": { 2024-04-07T17:51:24.096 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 148, 2024-04-07T17:51:24.096 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2568628 2024-04-07T17:51:24.096 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.096 INFO:tasks.workunit.client.0.smithi100.stdout: "db_total_bytes": { 2024-04-07T17:51:24.097 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 95995027456 2024-04-07T17:51:24.097 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.097 INFO:tasks.workunit.client.0.smithi100.stdout: "db_used_bytes": { 2024-04-07T17:51:24.097 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 411107328 2024-04-07T17:51:24.097 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.097 INFO:tasks.workunit.client.0.smithi100.stdout: "flush_lat": { 2024-04-07T17:51:24.097 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 63017, 2024-04-07T17:51:24.097 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 12122188924 2024-04-07T17:51:24.097 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.097 INFO:tasks.workunit.client.0.smithi100.stdout: "fsync_lat": { 2024-04-07T17:51:24.097 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 63028, 2024-04-07T17:51:24.097 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 21247379624 2024-04-07T17:51:24.097 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.097 INFO:tasks.workunit.client.0.smithi100.stdout: "log_bytes": { 2024-04-07T17:51:24.097 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 8843264 2024-04-07T17:51:24.097 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.098 INFO:tasks.workunit.client.0.smithi100.stdout: "logged_bytes": { 2024-04-07T17:51:24.098 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 258203648 2024-04-07T17:51:24.098 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.098 INFO:tasks.workunit.client.0.smithi100.stdout: "max_bytes_db": { 2024-04-07T17:51:24.098 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 423886848 2024-04-07T17:51:24.098 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.098 INFO:tasks.workunit.client.0.smithi100.stdout: "max_bytes_slow": { 2024-04-07T17:51:24.098 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.098 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.098 INFO:tasks.workunit.client.0.smithi100.stdout: "max_bytes_wal": { 2024-04-07T17:51:24.098 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.098 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.098 INFO:tasks.workunit.client.0.smithi100.stdout: "num_files": { 2024-04-07T17:51:24.099 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 38 2024-04-07T17:51:24.099 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.099 INFO:tasks.workunit.client.0.smithi100.stdout: "read_bytes": { 2024-04-07T17:51:24.099 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3167491823 2024-04-07T17:51:24.099 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.099 INFO:tasks.workunit.client.0.smithi100.stdout: "read_count": { 2024-04-07T17:51:24.099 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 40027 2024-04-07T17:51:24.099 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.099 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes": { 2024-04-07T17:51:24.099 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2790236160 2024-04-07T17:51:24.099 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.099 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes_db": { 2024-04-07T17:51:24.099 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2790240256 2024-04-07T17:51:24.099 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.099 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes_slow": { 2024-04-07T17:51:24.099 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.100 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.100 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes_wal": { 2024-04-07T17:51:24.100 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.100 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.100 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_count": { 2024-04-07T17:51:24.100 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 25974 2024-04-07T17:51:24.100 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.100 INFO:tasks.workunit.client.0.smithi100.stdout: "read_lat": { 2024-04-07T17:51:24.100 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 40027, 2024-04-07T17:51:24.100 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1975651191 2024-04-07T17:51:24.100 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.100 INFO:tasks.workunit.client.0.smithi100.stdout: "read_prefetch_bytes": { 2024-04-07T17:51:24.100 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3056697503 2024-04-07T17:51:24.100 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.100 INFO:tasks.workunit.client.0.smithi100.stdout: "read_prefetch_count": { 2024-04-07T17:51:24.100 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 36631 2024-04-07T17:51:24.101 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.101 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_buffer_bytes": { 2024-04-07T17:51:24.101 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1782285 2024-04-07T17:51:24.101 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.101 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_buffer_count": { 2024-04-07T17:51:24.101 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 538 2024-04-07T17:51:24.101 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.101 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_bytes": { 2024-04-07T17:51:24.101 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5008896 2024-04-07T17:51:24.101 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.101 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_count": { 2024-04-07T17:51:24.101 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 703 2024-04-07T17:51:24.102 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.102 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes": { 2024-04-07T17:51:24.102 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3226611 2024-04-07T17:51:24.102 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.102 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes_db": { 2024-04-07T17:51:24.102 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3226611 2024-04-07T17:51:24.102 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.102 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes_slow": { 2024-04-07T17:51:24.102 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.102 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.102 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes_wal": { 2024-04-07T17:51:24.102 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.102 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.102 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_count": { 2024-04-07T17:51:24.103 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 167 2024-04-07T17:51:24.103 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.103 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_lat": { 2024-04-07T17:51:24.103 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 703, 2024-04-07T17:51:24.103 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1945572 2024-04-07T17:51:24.103 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.103 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_alloc_lat": { 2024-04-07T17:51:24.103 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:24.103 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.103 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.103 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_total_bytes": { 2024-04-07T17:51:24.103 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.103 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.103 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_used_bytes": { 2024-04-07T17:51:24.103 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.104 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.104 INFO:tasks.workunit.client.0.smithi100.stdout: "truncate_lat": { 2024-04-07T17:51:24.104 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1, 2024-04-07T17:51:24.104 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 858 2024-04-07T17:51:24.104 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.104 INFO:tasks.workunit.client.0.smithi100.stdout: "unlink_lat": { 2024-04-07T17:51:24.104 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 9, 2024-04-07T17:51:24.104 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 12676 2024-04-07T17:51:24.104 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.104 INFO:tasks.workunit.client.0.smithi100.stdout: "wal_alloc_lat": { 2024-04-07T17:51:24.104 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:24.104 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.104 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.104 INFO:tasks.workunit.client.0.smithi100.stdout: "wal_total_bytes": { 2024-04-07T17:51:24.104 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.105 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.105 INFO:tasks.workunit.client.0.smithi100.stdout: "wal_used_bytes": { 2024-04-07T17:51:24.105 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.105 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.105 INFO:tasks.workunit.client.0.smithi100.stdout: "write_bytes": { 2024-04-07T17:51:24.105 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 761053184 2024-04-07T17:51:24.105 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:24.105 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.105 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore": { 2024-04-07T17:51:24.105 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit": { 2024-04-07T17:51:24.105 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4096 2024-04-07T17:51:24.105 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.105 INFO:tasks.workunit.client.0.smithi100.stdout: "allocated": { 2024-04-07T17:51:24.105 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 10403840 2024-04-07T17:51:24.105 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.105 INFO:tasks.workunit.client.0.smithi100.stdout: "allocator_lat": { 2024-04-07T17:51:24.105 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 17672, 2024-04-07T17:51:24.106 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 84277754 2024-04-07T17:51:24.106 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.106 INFO:tasks.workunit.client.0.smithi100.stdout: "clist_lat": { 2024-04-07T17:51:24.106 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 3466, 2024-04-07T17:51:24.106 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 6069941459 2024-04-07T17:51:24.106 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.106 INFO:tasks.workunit.client.0.smithi100.stdout: "compress_lat": { 2024-04-07T17:51:24.106 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:24.106 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.107 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.107 INFO:tasks.workunit.client.0.smithi100.stdout: "compressed": { 2024-04-07T17:51:24.107 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.107 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.107 INFO:tasks.workunit.client.0.smithi100.stdout: "compressed_allocated": { 2024-04-07T17:51:24.107 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.107 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.107 INFO:tasks.workunit.client.0.smithi100.stdout: "compressed_original": { 2024-04-07T17:51:24.107 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.107 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.107 INFO:tasks.workunit.client.0.smithi100.stdout: "csum_lat": { 2024-04-07T17:51:24.107 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 16446, 2024-04-07T17:51:24.107 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 220970389 2024-04-07T17:51:24.107 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.107 INFO:tasks.workunit.client.0.smithi100.stdout: "decompress_lat": { 2024-04-07T17:51:24.108 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:24.108 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.108 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.108 INFO:tasks.workunit.client.0.smithi100.stdout: "fragmentation_micros": { 2024-04-07T17:51:24.108 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.108 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.108 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_commit_lat": { 2024-04-07T17:51:24.108 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 62983, 2024-04-07T17:51:24.108 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 27504738088 2024-04-07T17:51:24.108 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.108 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_final_lat": { 2024-04-07T17:51:24.108 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 62244, 2024-04-07T17:51:24.108 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3517645071 2024-04-07T17:51:24.108 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.108 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_flush_lat": { 2024-04-07T17:51:24.108 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 62983, 2024-04-07T17:51:24.109 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 96259530 2024-04-07T17:51:24.109 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.109 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_sync_lat": { 2024-04-07T17:51:24.109 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 62983, 2024-04-07T17:51:24.109 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 27600997618 2024-04-07T17:51:24.109 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.109 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_get_keys_lat": { 2024-04-07T17:51:24.109 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:24.109 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.109 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.109 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_get_values_lat": { 2024-04-07T17:51:24.109 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 33493, 2024-04-07T17:51:24.109 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 950805853 2024-04-07T17:51:24.109 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.109 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_lower_bound_lat": { 2024-04-07T17:51:24.110 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 632, 2024-04-07T17:51:24.110 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4331297 2024-04-07T17:51:24.110 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.110 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_next_lat": { 2024-04-07T17:51:24.110 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 7000, 2024-04-07T17:51:24.110 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 13697854 2024-04-07T17:51:24.110 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.110 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_seek_to_first_lat": { 2024-04-07T17:51:24.110 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 50, 2024-04-07T17:51:24.110 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 102727 2024-04-07T17:51:24.110 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.110 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_upper_bound_lat": { 2024-04-07T17:51:24.110 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 11201, 2024-04-07T17:51:24.110 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1557562272 2024-04-07T17:51:24.111 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.111 INFO:tasks.workunit.client.0.smithi100.stdout: "onode_hits": { 2024-04-07T17:51:24.111 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 194378 2024-04-07T17:51:24.111 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.111 INFO:tasks.workunit.client.0.smithi100.stdout: "onode_misses": { 2024-04-07T17:51:24.111 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 17489 2024-04-07T17:51:24.111 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.111 INFO:tasks.workunit.client.0.smithi100.stdout: "read_lat": { 2024-04-07T17:51:24.111 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 9939, 2024-04-07T17:51:24.111 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1593794096 2024-04-07T17:51:24.111 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.111 INFO:tasks.workunit.client.0.smithi100.stdout: "read_onode_meta_lat": { 2024-04-07T17:51:24.111 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 39993, 2024-04-07T17:51:24.112 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 19793062 2024-04-07T17:51:24.112 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.112 INFO:tasks.workunit.client.0.smithi100.stdout: "read_wait_aio_lat": { 2024-04-07T17:51:24.112 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 30054, 2024-04-07T17:51:24.112 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1325463873 2024-04-07T17:51:24.112 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.112 INFO:tasks.workunit.client.0.smithi100.stdout: "reads_with_retries": { 2024-04-07T17:51:24.112 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.112 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.112 INFO:tasks.workunit.client.0.smithi100.stdout: "remove_lat": { 2024-04-07T17:51:24.112 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 13264, 2024-04-07T17:51:24.112 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 667186707 2024-04-07T17:51:24.112 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.112 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_aio_wait_count": { 2024-04-07T17:51:24.113 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.113 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.113 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_committed_kv_count": { 2024-04-07T17:51:24.113 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.113 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.113 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_read_onode_meta_count": { 2024-04-07T17:51:24.113 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.113 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.113 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_read_wait_aio_count": { 2024-04-07T17:51:24.113 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.113 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.113 INFO:tasks.workunit.client.0.smithi100.stdout: "state_aio_wait_lat": { 2024-04-07T17:51:24.113 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 74429, 2024-04-07T17:51:24.113 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 8392301592 2024-04-07T17:51:24.113 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.113 INFO:tasks.workunit.client.0.smithi100.stdout: "state_deferred_aio_wait_lat": { 2024-04-07T17:51:24.114 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 16391, 2024-04-07T17:51:24.114 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1329233809 2024-04-07T17:51:24.114 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.114 INFO:tasks.workunit.client.0.smithi100.stdout: "state_deferred_cleanup_lat": { 2024-04-07T17:51:24.114 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 16390, 2024-04-07T17:51:24.114 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4040344878498 2024-04-07T17:51:24.114 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.114 INFO:tasks.workunit.client.0.smithi100.stdout: "state_deferred_queued_lat": { 2024-04-07T17:51:24.114 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 16391, 2024-04-07T17:51:24.114 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 6763629312230 2024-04-07T17:51:24.114 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.114 INFO:tasks.workunit.client.0.smithi100.stdout: "state_done_lat": { 2024-04-07T17:51:24.114 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 74428, 2024-04-07T17:51:24.114 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2916516169130 2024-04-07T17:51:24.114 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.115 INFO:tasks.workunit.client.0.smithi100.stdout: "state_finishing_lat": { 2024-04-07T17:51:24.115 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 74428, 2024-04-07T17:51:24.115 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 17625368 2024-04-07T17:51:24.115 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.115 INFO:tasks.workunit.client.0.smithi100.stdout: "state_io_done_lat": { 2024-04-07T17:51:24.115 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 74429, 2024-04-07T17:51:24.115 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 362782490 2024-04-07T17:51:24.115 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.115 INFO:tasks.workunit.client.0.smithi100.stdout: "state_kv_commiting_lat": { 2024-04-07T17:51:24.115 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 74429, 2024-04-07T17:51:24.115 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 49654142030 2024-04-07T17:51:24.115 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.115 INFO:tasks.workunit.client.0.smithi100.stdout: "state_kv_done_lat": { 2024-04-07T17:51:24.115 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 74429, 2024-04-07T17:51:24.115 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 57857521 2024-04-07T17:51:24.116 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.116 INFO:tasks.workunit.client.0.smithi100.stdout: "state_kv_queued_lat": { 2024-04-07T17:51:24.116 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 74429, 2024-04-07T17:51:24.116 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 25576588902 2024-04-07T17:51:24.116 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.116 INFO:tasks.workunit.client.0.smithi100.stdout: "state_prepare_lat": { 2024-04-07T17:51:24.116 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 74429, 2024-04-07T17:51:24.116 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 18974977018 2024-04-07T17:51:24.116 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.116 INFO:tasks.workunit.client.0.smithi100.stdout: "stored": { 2024-04-07T17:51:24.116 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 6290654 2024-04-07T17:51:24.116 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.116 INFO:tasks.workunit.client.0.smithi100.stdout: "truncate_lat": { 2024-04-07T17:51:24.117 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 2242, 2024-04-07T17:51:24.117 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 22293375 2024-04-07T17:51:24.117 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.117 INFO:tasks.workunit.client.0.smithi100.stdout: "txc_commit_lat": { 2024-04-07T17:51:24.117 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 74429, 2024-04-07T17:51:24.117 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 102987546391 2024-04-07T17:51:24.117 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.117 INFO:tasks.workunit.client.0.smithi100.stdout: "txc_submit_lat": { 2024-04-07T17:51:24.117 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 74429, 2024-04-07T17:51:24.117 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 21153024583 2024-04-07T17:51:24.117 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.117 INFO:tasks.workunit.client.0.smithi100.stdout: "txc_throttle_lat": { 2024-04-07T17:51:24.117 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 74429, 2024-04-07T17:51:24.117 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 946122269 2024-04-07T17:51:24.118 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:24.118 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.118 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache": { 2024-04-07T17:51:24.118 INFO:tasks.workunit.client.0.smithi100.stdout: "cache_bytes": { 2024-04-07T17:51:24.118 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1444650649 2024-04-07T17:51:24.118 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.118 INFO:tasks.workunit.client.0.smithi100.stdout: "heap_bytes": { 2024-04-07T17:51:24.118 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 463331328 2024-04-07T17:51:24.118 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.118 INFO:tasks.workunit.client.0.smithi100.stdout: "mapped_bytes": { 2024-04-07T17:51:24.118 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 430809088 2024-04-07T17:51:24.118 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.118 INFO:tasks.workunit.client.0.smithi100.stdout: "target_bytes": { 2024-04-07T17:51:24.119 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2647008256 2024-04-07T17:51:24.119 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.119 INFO:tasks.workunit.client.0.smithi100.stdout: "unmapped_bytes": { 2024-04-07T17:51:24.119 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 32522240 2024-04-07T17:51:24.119 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:24.119 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.119 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:data": { 2024-04-07T17:51:24.119 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:24.119 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 134217728 2024-04-07T17:51:24.119 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.119 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:24.119 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.119 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.119 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:24.119 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.119 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.120 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:24.120 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 59553061 2024-04-07T17:51:24.120 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.120 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:24.120 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.120 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.120 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:24.120 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.120 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.120 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:24.120 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.120 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.120 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:24.120 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.120 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.121 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:24.121 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 32768 2024-04-07T17:51:24.121 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.121 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:24.121 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 61440 2024-04-07T17:51:24.121 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.121 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:24.121 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.121 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.121 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:24.121 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.121 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.121 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:24.121 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.121 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.121 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:24.122 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 74570459 2024-04-07T17:51:24.122 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:24.122 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.122 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:kv": { 2024-04-07T17:51:24.122 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:24.122 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 587202560 2024-04-07T17:51:24.122 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.122 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:24.122 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 67108864 2024-04-07T17:51:24.122 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.122 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:24.122 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.123 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.123 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:24.123 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 446647962 2024-04-07T17:51:24.123 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.123 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:24.123 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.123 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.123 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:24.123 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.123 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.123 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:24.124 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1504 2024-04-07T17:51:24.124 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.124 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:24.124 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.124 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.124 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:24.124 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 95456 2024-04-07T17:51:24.125 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.125 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:24.125 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.125 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.125 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:24.125 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.125 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.126 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:24.126 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.126 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.126 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:24.126 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.126 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.126 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:24.126 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 73348774 2024-04-07T17:51:24.126 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:24.126 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.126 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:kv_onode": { 2024-04-07T17:51:24.126 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:24.126 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 184549376 2024-04-07T17:51:24.126 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.127 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:24.127 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 67108864 2024-04-07T17:51:24.127 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.127 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:24.127 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4539488 2024-04-07T17:51:24.127 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.127 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:24.127 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 39702041 2024-04-07T17:51:24.127 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.127 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:24.128 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.128 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.128 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:24.128 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.128 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.128 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:24.128 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.128 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.128 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:24.128 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.128 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.128 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:24.128 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.128 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.128 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:24.129 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.129 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.129 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:24.129 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.129 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.129 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:24.129 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.129 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.129 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:24.129 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.129 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.130 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:24.130 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 73198983 2024-04-07T17:51:24.130 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:24.130 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.130 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:meta": { 2024-04-07T17:51:24.130 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:24.131 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 520093696 2024-04-07T17:51:24.131 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.131 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:24.131 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.131 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.131 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:24.131 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.131 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.131 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:24.131 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 446647962 2024-04-07T17:51:24.131 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.131 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:24.131 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.132 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.132 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:24.132 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.132 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.132 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:24.132 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.132 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.132 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:24.132 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.132 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.132 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:24.132 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 155289 2024-04-07T17:51:24.133 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.133 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:24.133 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1107933 2024-04-07T17:51:24.133 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.133 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:24.133 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.133 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.133 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:24.133 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.133 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.133 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:24.133 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.133 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.133 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:24.133 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 72182512 2024-04-07T17:51:24.134 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:24.134 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.134 INFO:tasks.workunit.client.0.smithi100.stdout: "objecter": { 2024-04-07T17:51:24.134 INFO:tasks.workunit.client.0.smithi100.stdout: "op_active": { 2024-04-07T17:51:24.134 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.134 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.134 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r": { 2024-04-07T17:51:24.134 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 23 2024-04-07T17:51:24.134 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.134 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rmw": { 2024-04-07T17:51:24.134 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.134 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.134 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w": { 2024-04-07T17:51:24.134 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.135 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:24.135 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.135 INFO:tasks.workunit.client.0.smithi100.stdout: "osd": { 2024-04-07T17:51:24.135 INFO:tasks.workunit.client.0.smithi100.stdout: "numpg": { 2024-04-07T17:51:24.135 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 58 2024-04-07T17:51:24.135 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.135 INFO:tasks.workunit.client.0.smithi100.stdout: "numpg_removing": { 2024-04-07T17:51:24.135 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.135 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.135 INFO:tasks.workunit.client.0.smithi100.stdout: "op": { 2024-04-07T17:51:24.135 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 40489 2024-04-07T17:51:24.135 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.135 INFO:tasks.workunit.client.0.smithi100.stdout: "op_before_queue_op_lat": { 2024-04-07T17:51:24.135 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 108065, 2024-04-07T17:51:24.135 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 15066892416 2024-04-07T17:51:24.136 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.136 INFO:tasks.workunit.client.0.smithi100.stdout: "op_delayed_degraded": { 2024-04-07T17:51:24.136 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.136 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.136 INFO:tasks.workunit.client.0.smithi100.stdout: "op_delayed_unreadable": { 2024-04-07T17:51:24.136 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.136 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.136 INFO:tasks.workunit.client.0.smithi100.stdout: "op_in_bytes": { 2024-04-07T17:51:24.136 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1416321031 2024-04-07T17:51:24.136 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.136 INFO:tasks.workunit.client.0.smithi100.stdout: "op_latency": { 2024-04-07T17:51:24.136 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 40489, 2024-04-07T17:51:24.136 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 116806238602 2024-04-07T17:51:24.136 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.137 INFO:tasks.workunit.client.0.smithi100.stdout: "op_out_bytes": { 2024-04-07T17:51:24.137 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1572109991 2024-04-07T17:51:24.137 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.137 INFO:tasks.workunit.client.0.smithi100.stdout: "op_prepare_latency": { 2024-04-07T17:51:24.137 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 40866, 2024-04-07T17:51:24.137 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 41965414839 2024-04-07T17:51:24.137 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.137 INFO:tasks.workunit.client.0.smithi100.stdout: "op_process_latency": { 2024-04-07T17:51:24.137 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 40489, 2024-04-07T17:51:24.137 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 66194385741 2024-04-07T17:51:24.137 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.137 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r": { 2024-04-07T17:51:24.137 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 27940 2024-04-07T17:51:24.138 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.138 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_latency": { 2024-04-07T17:51:24.138 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 27940, 2024-04-07T17:51:24.138 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 25174152653 2024-04-07T17:51:24.138 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.138 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_latency_out_bytes_histogram": { 2024-04-07T17:51:24.139 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.139 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.139 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_out_bytes": { 2024-04-07T17:51:24.139 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1572109991 2024-04-07T17:51:24.139 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.139 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_prepare_latency": { 2024-04-07T17:51:24.139 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 28058, 2024-04-07T17:51:24.139 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 13058211260 2024-04-07T17:51:24.139 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.139 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_process_latency": { 2024-04-07T17:51:24.139 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 27940, 2024-04-07T17:51:24.139 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 11404588197 2024-04-07T17:51:24.139 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.139 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw": { 2024-04-07T17:51:24.139 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 11358 2024-04-07T17:51:24.140 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.140 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_in_bytes": { 2024-04-07T17:51:24.140 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 298732687 2024-04-07T17:51:24.140 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.140 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_latency": { 2024-04-07T17:51:24.140 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 11358, 2024-04-07T17:51:24.140 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 46852373505 2024-04-07T17:51:24.140 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.140 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_latency_in_bytes_histogram": { 2024-04-07T17:51:24.140 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.140 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.140 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_latency_out_bytes_histogram": { 2024-04-07T17:51:24.141 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.141 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.141 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_out_bytes": { 2024-04-07T17:51:24.141 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.141 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.141 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_prepare_latency": { 2024-04-07T17:51:24.141 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 11546, 2024-04-07T17:51:24.141 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 24931969765 2024-04-07T17:51:24.141 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.142 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_process_latency": { 2024-04-07T17:51:24.142 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 11358, 2024-04-07T17:51:24.142 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 41982228701 2024-04-07T17:51:24.142 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.142 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w": { 2024-04-07T17:51:24.142 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1191 2024-04-07T17:51:24.142 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.142 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_in_bytes": { 2024-04-07T17:51:24.142 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1117588344 2024-04-07T17:51:24.142 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.142 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_latency": { 2024-04-07T17:51:24.143 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1191, 2024-04-07T17:51:24.143 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 44779712444 2024-04-07T17:51:24.143 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.143 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_latency_in_bytes_histogram": { 2024-04-07T17:51:24.143 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.143 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.143 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_prepare_latency": { 2024-04-07T17:51:24.143 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1262, 2024-04-07T17:51:24.143 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3975233814 2024-04-07T17:51:24.143 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.143 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_process_latency": { 2024-04-07T17:51:24.143 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1191, 2024-04-07T17:51:24.143 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 12807568843 2024-04-07T17:51:24.143 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.144 INFO:tasks.workunit.client.0.smithi100.stdout: "op_wip": { 2024-04-07T17:51:24.144 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.144 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.144 INFO:tasks.workunit.client.0.smithi100.stdout: "recovery_bytes": { 2024-04-07T17:51:24.144 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:24.144 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.144 INFO:tasks.workunit.client.0.smithi100.stdout: "recovery_ops": { 2024-04-07T17:51:24.144 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2 2024-04-07T17:51:24.144 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.144 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_bytes": { 2024-04-07T17:51:24.144 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 95995035648 2024-04-07T17:51:24.144 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.144 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_bytes_used": { 2024-04-07T17:51:24.144 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 421523456 2024-04-07T17:51:24.144 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:24.145 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:24.145 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:24.145 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.145 INFO:tasks.workunit.client.0.smithi100.stdout: "pools": [ 2024-04-07T17:51:24.145 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:24.145 INFO:tasks.workunit.client.0.smithi100.stdout: "application": [ 2024-04-07T17:51:24.145 INFO:tasks.workunit.client.0.smithi100.stdout: "mgr" 2024-04-07T17:51:24.145 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.145 INFO:tasks.workunit.client.0.smithi100.stdout: "cache_mode": "none", 2024-04-07T17:51:24.145 INFO:tasks.workunit.client.0.smithi100.stdout: "erasure_code_profile": {}, 2024-04-07T17:51:24.145 INFO:tasks.workunit.client.0.smithi100.stdout: "flags_names": [ 2024-04-07T17:51:24.145 INFO:tasks.workunit.client.0.smithi100.stdout: "hashpspool" 2024-04-07T17:51:24.145 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.145 INFO:tasks.workunit.client.0.smithi100.stdout: "min_size": 2, 2024-04-07T17:51:24.146 INFO:tasks.workunit.client.0.smithi100.stdout: "options": {}, 2024-04-07T17:51:24.146 INFO:tasks.workunit.client.0.smithi100.stdout: "pg_autoscale_mode": "off", 2024-04-07T17:51:24.146 INFO:tasks.workunit.client.0.smithi100.stdout: "pg_num": 1, 2024-04-07T17:51:24.146 INFO:tasks.workunit.client.0.smithi100.stdout: "pgp_num": 1, 2024-04-07T17:51:24.146 INFO:tasks.workunit.client.0.smithi100.stdout: "pool": 1, 2024-04-07T17:51:24.146 INFO:tasks.workunit.client.0.smithi100.stdout: "size": 3, 2024-04-07T17:51:24.146 INFO:tasks.workunit.client.0.smithi100.stdout: "stats": { 2024-04-07T17:51:24.146 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_raw": 723597404426, 2024-04-07T17:51:24.146 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_used": 1781760, 2024-04-07T17:51:24.146 INFO:tasks.workunit.client.0.smithi100.stdout: "compress_bytes_used": 0, 2024-04-07T17:51:24.146 INFO:tasks.workunit.client.0.smithi100.stdout: "compress_under_bytes": 0, 2024-04-07T17:51:24.146 INFO:tasks.workunit.client.0.smithi100.stdout: "data_bytes_used": 1781760, 2024-04-07T17:51:24.146 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty": 0, 2024-04-07T17:51:24.146 INFO:tasks.workunit.client.0.smithi100.stdout: "max_avail": 241199136768, 2024-04-07T17:51:24.147 INFO:tasks.workunit.client.0.smithi100.stdout: "objects": 2, 2024-04-07T17:51:24.147 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_bytes_used": 0, 2024-04-07T17:51:24.147 INFO:tasks.workunit.client.0.smithi100.stdout: "percent_used": 2.46235777012771e-06, 2024-04-07T17:51:24.147 INFO:tasks.workunit.client.0.smithi100.stdout: "quota_bytes": 0, 2024-04-07T17:51:24.147 INFO:tasks.workunit.client.0.smithi100.stdout: "quota_objects": 0, 2024-04-07T17:51:24.147 INFO:tasks.workunit.client.0.smithi100.stdout: "rd": 1214, 2024-04-07T17:51:24.147 INFO:tasks.workunit.client.0.smithi100.stdout: "rd_bytes": 2072576, 2024-04-07T17:51:24.147 INFO:tasks.workunit.client.0.smithi100.stdout: "stored": 590368, 2024-04-07T17:51:24.147 INFO:tasks.workunit.client.0.smithi100.stdout: "stored_data": 590368, 2024-04-07T17:51:24.147 INFO:tasks.workunit.client.0.smithi100.stdout: "stored_omap": 0, 2024-04-07T17:51:24.147 INFO:tasks.workunit.client.0.smithi100.stdout: "stored_raw": 1771104, 2024-04-07T17:51:24.147 INFO:tasks.workunit.client.0.smithi100.stdout: "wr": 399, 2024-04-07T17:51:24.147 INFO:tasks.workunit.client.0.smithi100.stdout: "wr_bytes": 2473984 2024-04-07T17:51:24.147 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.148 INFO:tasks.workunit.client.0.smithi100.stdout: "target_max_bytes": 0, 2024-04-07T17:51:24.148 INFO:tasks.workunit.client.0.smithi100.stdout: "target_max_objects": 0, 2024-04-07T17:51:24.148 INFO:tasks.workunit.client.0.smithi100.stdout: "type": "replicated" 2024-04-07T17:51:24.148 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.148 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:24.148 INFO:tasks.workunit.client.0.smithi100.stdout: "application": [ 2024-04-07T17:51:24.148 INFO:tasks.workunit.client.0.smithi100.stdout: "cephfs" 2024-04-07T17:51:24.148 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.149 INFO:tasks.workunit.client.0.smithi100.stdout: "cache_mode": "none", 2024-04-07T17:51:24.149 INFO:tasks.workunit.client.0.smithi100.stdout: "erasure_code_profile": {}, 2024-04-07T17:51:24.149 INFO:tasks.workunit.client.0.smithi100.stdout: "flags_names": [ 2024-04-07T17:51:24.149 INFO:tasks.workunit.client.0.smithi100.stdout: "hashpspool" 2024-04-07T17:51:24.149 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.149 INFO:tasks.workunit.client.0.smithi100.stdout: "min_size": 2, 2024-04-07T17:51:24.149 INFO:tasks.workunit.client.0.smithi100.stdout: "options": {}, 2024-04-07T17:51:24.149 INFO:tasks.workunit.client.0.smithi100.stdout: "pg_autoscale_mode": "off", 2024-04-07T17:51:24.149 INFO:tasks.workunit.client.0.smithi100.stdout: "pg_num": 32, 2024-04-07T17:51:24.149 INFO:tasks.workunit.client.0.smithi100.stdout: "pgp_num": 32, 2024-04-07T17:51:24.149 INFO:tasks.workunit.client.0.smithi100.stdout: "pool": 2, 2024-04-07T17:51:24.149 INFO:tasks.workunit.client.0.smithi100.stdout: "size": 3, 2024-04-07T17:51:24.149 INFO:tasks.workunit.client.0.smithi100.stdout: "stats": { 2024-04-07T17:51:24.149 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_raw": 723597404426, 2024-04-07T17:51:24.149 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_used": 159173, 2024-04-07T17:51:24.150 INFO:tasks.workunit.client.0.smithi100.stdout: "compress_bytes_used": 0, 2024-04-07T17:51:24.150 INFO:tasks.workunit.client.0.smithi100.stdout: "compress_under_bytes": 0, 2024-04-07T17:51:24.150 INFO:tasks.workunit.client.0.smithi100.stdout: "data_bytes_used": 110592, 2024-04-07T17:51:24.150 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty": 0, 2024-04-07T17:51:24.150 INFO:tasks.workunit.client.0.smithi100.stdout: "max_avail": 241199136768, 2024-04-07T17:51:24.150 INFO:tasks.workunit.client.0.smithi100.stdout: "objects": 22, 2024-04-07T17:51:24.150 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_bytes_used": 48581, 2024-04-07T17:51:24.150 INFO:tasks.workunit.client.0.smithi100.stdout: "percent_used": 2.1997449550781312e-07, 2024-04-07T17:51:24.150 INFO:tasks.workunit.client.0.smithi100.stdout: "quota_bytes": 0, 2024-04-07T17:51:24.150 INFO:tasks.workunit.client.0.smithi100.stdout: "quota_objects": 0, 2024-04-07T17:51:24.150 INFO:tasks.workunit.client.0.smithi100.stdout: "rd": 79, 2024-04-07T17:51:24.150 INFO:tasks.workunit.client.0.smithi100.stdout: "rd_bytes": 89088, 2024-04-07T17:51:24.150 INFO:tasks.workunit.client.0.smithi100.stdout: "stored": 22815, 2024-04-07T17:51:24.151 INFO:tasks.workunit.client.0.smithi100.stdout: "stored_data": 6622, 2024-04-07T17:51:24.151 INFO:tasks.workunit.client.0.smithi100.stdout: "stored_omap": 16193, 2024-04-07T17:51:24.151 INFO:tasks.workunit.client.0.smithi100.stdout: "stored_raw": 68445, 2024-04-07T17:51:24.151 INFO:tasks.workunit.client.0.smithi100.stdout: "wr": 68, 2024-04-07T17:51:24.151 INFO:tasks.workunit.client.0.smithi100.stdout: "wr_bytes": 36864 2024-04-07T17:51:24.151 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.151 INFO:tasks.workunit.client.0.smithi100.stdout: "target_max_bytes": 0, 2024-04-07T17:51:24.151 INFO:tasks.workunit.client.0.smithi100.stdout: "target_max_objects": 0, 2024-04-07T17:51:24.151 INFO:tasks.workunit.client.0.smithi100.stdout: "type": "replicated" 2024-04-07T17:51:24.151 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.151 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:24.151 INFO:tasks.workunit.client.0.smithi100.stdout: "application": [ 2024-04-07T17:51:24.151 INFO:tasks.workunit.client.0.smithi100.stdout: "cephfs" 2024-04-07T17:51:24.151 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.152 INFO:tasks.workunit.client.0.smithi100.stdout: "cache_mode": "none", 2024-04-07T17:51:24.152 INFO:tasks.workunit.client.0.smithi100.stdout: "erasure_code_profile": {}, 2024-04-07T17:51:24.152 INFO:tasks.workunit.client.0.smithi100.stdout: "flags_names": [ 2024-04-07T17:51:24.152 INFO:tasks.workunit.client.0.smithi100.stdout: "hashpspool" 2024-04-07T17:51:24.152 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.152 INFO:tasks.workunit.client.0.smithi100.stdout: "min_size": 2, 2024-04-07T17:51:24.152 INFO:tasks.workunit.client.0.smithi100.stdout: "options": {}, 2024-04-07T17:51:24.152 INFO:tasks.workunit.client.0.smithi100.stdout: "pg_autoscale_mode": "off", 2024-04-07T17:51:24.152 INFO:tasks.workunit.client.0.smithi100.stdout: "pg_num": 64, 2024-04-07T17:51:24.152 INFO:tasks.workunit.client.0.smithi100.stdout: "pgp_num": 64, 2024-04-07T17:51:24.152 INFO:tasks.workunit.client.0.smithi100.stdout: "pool": 3, 2024-04-07T17:51:24.152 INFO:tasks.workunit.client.0.smithi100.stdout: "size": 3, 2024-04-07T17:51:24.152 INFO:tasks.workunit.client.0.smithi100.stdout: "stats": { 2024-04-07T17:51:24.152 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_raw": 723597404426, 2024-04-07T17:51:24.153 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_used": 0, 2024-04-07T17:51:24.153 INFO:tasks.workunit.client.0.smithi100.stdout: "compress_bytes_used": 0, 2024-04-07T17:51:24.153 INFO:tasks.workunit.client.0.smithi100.stdout: "compress_under_bytes": 0, 2024-04-07T17:51:24.153 INFO:tasks.workunit.client.0.smithi100.stdout: "data_bytes_used": 0, 2024-04-07T17:51:24.153 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty": 0, 2024-04-07T17:51:24.153 INFO:tasks.workunit.client.0.smithi100.stdout: "max_avail": 241199136768, 2024-04-07T17:51:24.153 INFO:tasks.workunit.client.0.smithi100.stdout: "objects": 0, 2024-04-07T17:51:24.153 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_bytes_used": 0, 2024-04-07T17:51:24.153 INFO:tasks.workunit.client.0.smithi100.stdout: "percent_used": 0.0, 2024-04-07T17:51:24.153 INFO:tasks.workunit.client.0.smithi100.stdout: "quota_bytes": 0, 2024-04-07T17:51:24.154 INFO:tasks.workunit.client.0.smithi100.stdout: "quota_objects": 0, 2024-04-07T17:51:24.154 INFO:tasks.workunit.client.0.smithi100.stdout: "rd": 0, 2024-04-07T17:51:24.154 INFO:tasks.workunit.client.0.smithi100.stdout: "rd_bytes": 0, 2024-04-07T17:51:24.154 INFO:tasks.workunit.client.0.smithi100.stdout: "stored": 0, 2024-04-07T17:51:24.154 INFO:tasks.workunit.client.0.smithi100.stdout: "stored_data": 0, 2024-04-07T17:51:24.154 INFO:tasks.workunit.client.0.smithi100.stdout: "stored_omap": 0, 2024-04-07T17:51:24.154 INFO:tasks.workunit.client.0.smithi100.stdout: "stored_raw": 0, 2024-04-07T17:51:24.154 INFO:tasks.workunit.client.0.smithi100.stdout: "wr": 0, 2024-04-07T17:51:24.154 INFO:tasks.workunit.client.0.smithi100.stdout: "wr_bytes": 0 2024-04-07T17:51:24.154 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.154 INFO:tasks.workunit.client.0.smithi100.stdout: "target_max_bytes": 0, 2024-04-07T17:51:24.154 INFO:tasks.workunit.client.0.smithi100.stdout: "target_max_objects": 0, 2024-04-07T17:51:24.154 INFO:tasks.workunit.client.0.smithi100.stdout: "type": "replicated" 2024-04-07T17:51:24.155 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.155 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:24.155 INFO:tasks.workunit.client.0.smithi100.stdout: "application": [ 2024-04-07T17:51:24.155 INFO:tasks.workunit.client.0.smithi100.stdout: "rbd" 2024-04-07T17:51:24.155 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.155 INFO:tasks.workunit.client.0.smithi100.stdout: "cache_mode": "none", 2024-04-07T17:51:24.155 INFO:tasks.workunit.client.0.smithi100.stdout: "erasure_code_profile": {}, 2024-04-07T17:51:24.155 INFO:tasks.workunit.client.0.smithi100.stdout: "flags_names": [ 2024-04-07T17:51:24.155 INFO:tasks.workunit.client.0.smithi100.stdout: "hashpspool", 2024-04-07T17:51:24.155 INFO:tasks.workunit.client.0.smithi100.stdout: "selfmanaged_snaps" 2024-04-07T17:51:24.155 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.155 INFO:tasks.workunit.client.0.smithi100.stdout: "min_size": 2, 2024-04-07T17:51:24.155 INFO:tasks.workunit.client.0.smithi100.stdout: "options": {}, 2024-04-07T17:51:24.155 INFO:tasks.workunit.client.0.smithi100.stdout: "pg_autoscale_mode": "off", 2024-04-07T17:51:24.156 INFO:tasks.workunit.client.0.smithi100.stdout: "pg_num": 8, 2024-04-07T17:51:24.156 INFO:tasks.workunit.client.0.smithi100.stdout: "pgp_num": 8, 2024-04-07T17:51:24.156 INFO:tasks.workunit.client.0.smithi100.stdout: "pool": 4, 2024-04-07T17:51:24.156 INFO:tasks.workunit.client.0.smithi100.stdout: "size": 3, 2024-04-07T17:51:24.156 INFO:tasks.workunit.client.0.smithi100.stdout: "stats": { 2024-04-07T17:51:24.156 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_raw": 723597404426, 2024-04-07T17:51:24.156 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_used": 12595200, 2024-04-07T17:51:24.156 INFO:tasks.workunit.client.0.smithi100.stdout: "compress_bytes_used": 0, 2024-04-07T17:51:24.156 INFO:tasks.workunit.client.0.smithi100.stdout: "compress_under_bytes": 0, 2024-04-07T17:51:24.156 INFO:tasks.workunit.client.0.smithi100.stdout: "data_bytes_used": 12595200, 2024-04-07T17:51:24.156 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty": 0, 2024-04-07T17:51:24.156 INFO:tasks.workunit.client.0.smithi100.stdout: "max_avail": 241199136768, 2024-04-07T17:51:24.156 INFO:tasks.workunit.client.0.smithi100.stdout: "objects": 4, 2024-04-07T17:51:24.156 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_bytes_used": 0, 2024-04-07T17:51:24.157 INFO:tasks.workunit.client.0.smithi100.stdout: "percent_used": 1.74060623976402e-05, 2024-04-07T17:51:24.157 INFO:tasks.workunit.client.0.smithi100.stdout: "quota_bytes": 0, 2024-04-07T17:51:24.157 INFO:tasks.workunit.client.0.smithi100.stdout: "quota_objects": 0, 2024-04-07T17:51:24.157 INFO:tasks.workunit.client.0.smithi100.stdout: "rd": 9087, 2024-04-07T17:51:24.157 INFO:tasks.workunit.client.0.smithi100.stdout: "rd_bytes": 3222810624, 2024-04-07T17:51:24.157 INFO:tasks.workunit.client.0.smithi100.stdout: "stored": 4194323, 2024-04-07T17:51:24.157 INFO:tasks.workunit.client.0.smithi100.stdout: "stored_data": 4194323, 2024-04-07T17:51:24.157 INFO:tasks.workunit.client.0.smithi100.stdout: "stored_omap": 0, 2024-04-07T17:51:24.157 INFO:tasks.workunit.client.0.smithi100.stdout: "stored_raw": 12582969, 2024-04-07T17:51:24.157 INFO:tasks.workunit.client.0.smithi100.stdout: "wr": 24273, 2024-04-07T17:51:24.157 INFO:tasks.workunit.client.0.smithi100.stdout: "wr_bytes": 3103659008 2024-04-07T17:51:24.157 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.157 INFO:tasks.workunit.client.0.smithi100.stdout: "target_max_bytes": 0, 2024-04-07T17:51:24.157 INFO:tasks.workunit.client.0.smithi100.stdout: "target_max_objects": 0, 2024-04-07T17:51:24.157 INFO:tasks.workunit.client.0.smithi100.stdout: "type": "replicated" 2024-04-07T17:51:24.158 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:24.158 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.158 INFO:tasks.workunit.client.0.smithi100.stdout: "rbd": { 2024-04-07T17:51:24.158 INFO:tasks.workunit.client.0.smithi100.stdout: "mirroring_by_pool": [ 2024-04-07T17:51:24.158 INFO:tasks.workunit.client.0.smithi100.stdout: false 2024-04-07T17:51:24.158 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.158 INFO:tasks.workunit.client.0.smithi100.stdout: "num_images_by_pool": [ 2024-04-07T17:51:24.158 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T17:51:24.158 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.158 INFO:tasks.workunit.client.0.smithi100.stdout: "num_pools": 1 2024-04-07T17:51:24.159 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.159 INFO:tasks.workunit.client.0.smithi100.stdout: "report_id": "ee983b25-10fa-4633-b6cf-1c3036b91b52", 2024-04-07T17:51:24.159 INFO:tasks.workunit.client.0.smithi100.stdout: "report_timestamp": "2024-04-07T17:51:22.937359", 2024-04-07T17:51:24.159 INFO:tasks.workunit.client.0.smithi100.stdout: "report_version": 1, 2024-04-07T17:51:24.159 INFO:tasks.workunit.client.0.smithi100.stdout: "rocksdb_stats": { 2024-04-07T17:51:24.159 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "7.9.2" 2024-04-07T17:51:24.159 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.159 INFO:tasks.workunit.client.0.smithi100.stdout: "rook": { 2024-04-07T17:51:24.159 INFO:tasks.workunit.client.0.smithi100.stdout: "cluster": { 2024-04-07T17:51:24.160 INFO:tasks.workunit.client.0.smithi100.stdout: "external-mode": null, 2024-04-07T17:51:24.160 INFO:tasks.workunit.client.0.smithi100.stdout: "mon": { 2024-04-07T17:51:24.160 INFO:tasks.workunit.client.0.smithi100.stdout: "allow-multiple-per-node": null, 2024-04-07T17:51:24.160 INFO:tasks.workunit.client.0.smithi100.stdout: "count": null, 2024-04-07T17:51:24.160 INFO:tasks.workunit.client.0.smithi100.stdout: "max-id": null, 2024-04-07T17:51:24.160 INFO:tasks.workunit.client.0.smithi100.stdout: "pvc": { 2024-04-07T17:51:24.160 INFO:tasks.workunit.client.0.smithi100.stdout: "enabled": null 2024-04-07T17:51:24.160 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.160 INFO:tasks.workunit.client.0.smithi100.stdout: "stretch": { 2024-04-07T17:51:24.160 INFO:tasks.workunit.client.0.smithi100.stdout: "enabled": null 2024-04-07T17:51:24.160 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:24.160 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.160 INFO:tasks.workunit.client.0.smithi100.stdout: "network": { 2024-04-07T17:51:24.160 INFO:tasks.workunit.client.0.smithi100.stdout: "provider": null 2024-04-07T17:51:24.160 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.161 INFO:tasks.workunit.client.0.smithi100.stdout: "storage": { 2024-04-07T17:51:24.161 INFO:tasks.workunit.client.0.smithi100.stdout: "device-set": { 2024-04-07T17:51:24.161 INFO:tasks.workunit.client.0.smithi100.stdout: "count": { 2024-04-07T17:51:24.161 INFO:tasks.workunit.client.0.smithi100.stdout: "non-portable": null, 2024-04-07T17:51:24.161 INFO:tasks.workunit.client.0.smithi100.stdout: "portable": null, 2024-04-07T17:51:24.161 INFO:tasks.workunit.client.0.smithi100.stdout: "total": null 2024-04-07T17:51:24.161 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:24.161 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:24.161 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:24.161 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.161 INFO:tasks.workunit.client.0.smithi100.stdout: "csi": { 2024-04-07T17:51:24.161 INFO:tasks.workunit.client.0.smithi100.stdout: "version": null 2024-04-07T17:51:24.161 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.162 INFO:tasks.workunit.client.0.smithi100.stdout: "kubernetes": { 2024-04-07T17:51:24.162 INFO:tasks.workunit.client.0.smithi100.stdout: "version": null 2024-04-07T17:51:24.162 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.162 INFO:tasks.workunit.client.0.smithi100.stdout: "node": { 2024-04-07T17:51:24.162 INFO:tasks.workunit.client.0.smithi100.stdout: "count": { 2024-04-07T17:51:24.162 INFO:tasks.workunit.client.0.smithi100.stdout: "kubernetes-total": null, 2024-04-07T17:51:24.162 INFO:tasks.workunit.client.0.smithi100.stdout: "with-ceph-daemons": null, 2024-04-07T17:51:24.162 INFO:tasks.workunit.client.0.smithi100.stdout: "with-csi-cephfs-plugin": null, 2024-04-07T17:51:24.162 INFO:tasks.workunit.client.0.smithi100.stdout: "with-csi-nfs-plugin": null, 2024-04-07T17:51:24.162 INFO:tasks.workunit.client.0.smithi100.stdout: "with-csi-rbd-plugin": null 2024-04-07T17:51:24.162 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:24.162 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.162 INFO:tasks.workunit.client.0.smithi100.stdout: "usage": { 2024-04-07T17:51:24.162 INFO:tasks.workunit.client.0.smithi100.stdout: "storage-class": { 2024-04-07T17:51:24.163 INFO:tasks.workunit.client.0.smithi100.stdout: "count": { 2024-04-07T17:51:24.163 INFO:tasks.workunit.client.0.smithi100.stdout: "bucket": null, 2024-04-07T17:51:24.163 INFO:tasks.workunit.client.0.smithi100.stdout: "cephfs": null, 2024-04-07T17:51:24.163 INFO:tasks.workunit.client.0.smithi100.stdout: "nfs": null, 2024-04-07T17:51:24.163 INFO:tasks.workunit.client.0.smithi100.stdout: "rbd": null, 2024-04-07T17:51:24.163 INFO:tasks.workunit.client.0.smithi100.stdout: "total": null 2024-04-07T17:51:24.163 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:24.163 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:24.163 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.164 INFO:tasks.workunit.client.0.smithi100.stdout: "version": null 2024-04-07T17:51:24.164 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.164 INFO:tasks.workunit.client.0.smithi100.stdout: "services": { 2024-04-07T17:51:24.164 INFO:tasks.workunit.client.0.smithi100.stdout: "mon": 1 2024-04-07T17:51:24.164 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.164 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_per_pg": [ 2024-04-07T17:51:24.164 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:24.164 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:24.164 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:24.164 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:24.164 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T17:51:24.164 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.164 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 6, 2024-04-07T17:51:24.164 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:24.165 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:24.165 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:24.165 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:24.165 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:24.165 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:24.165 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.371282+0000", 2024-04-07T17:51:24.165 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:08.962622+0000", 2024-04-07T17:51:24.165 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:08.962622+0000", 2024-04-07T17:51:24.165 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:08.963061+0000", 2024-04-07T17:51:24.165 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.371282+0000", 2024-04-07T17:51:24.165 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.165 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:24.165 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.165 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 554, 2024-04-07T17:51:24.166 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.371282+0000", 2024-04-07T17:51:24.166 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.371282+0000", 2024-04-07T17:51:24.166 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.371282+0000", 2024-04-07T17:51:24.166 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:24.166 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:24.166 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.166 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.371282+0000", 2024-04-07T17:51:24.166 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.371282+0000", 2024-04-07T17:51:24.166 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:24.166 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:24.166 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:24.166 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:24.166 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 553, 2024-04-07T17:51:24.166 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:24.167 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:24.167 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:24.167 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:24.167 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:24.167 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:24.167 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:24.167 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:24.167 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.2c", 2024-04-07T17:51:24.167 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:24.167 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:24.167 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:24.167 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4846, 2024-04-07T17:51:24.167 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:24.167 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T02:53:55.780425+0000", 2024-04-07T17:51:24.167 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:24.167 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:24.168 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:24.168 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:24.168 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:24.168 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:24.168 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:24.168 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:24.168 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:24.168 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:24.168 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:24.168 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:24.168 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:24.168 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:24.168 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:24.168 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:24.168 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:24.169 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:24.169 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:24.169 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:24.169 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:24.169 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:24.169 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:24.169 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:24.169 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:24.169 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:24.169 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:24.169 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:24.170 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:24.170 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:24.170 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:24.170 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:24.170 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:24.170 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:24.170 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:24.170 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:24.170 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:24.170 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:24.170 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:24.171 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:24.171 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:24.171 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:24.171 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:24.171 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.171 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:24.171 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:24.171 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:24.171 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:24.171 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:24.171 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T17:51:24.171 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.171 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 6, 2024-04-07T17:51:24.171 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:24.172 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.172 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:24.172 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:24.172 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:24.172 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:24.172 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T17:51:24.172 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.172 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 0, 2024-04-07T17:51:24.172 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:24.172 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:24.172 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:24.172 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:24.172 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:24.172 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:24.172 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:35.989606+0000", 2024-04-07T17:51:24.172 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:39.003945+0000", 2024-04-07T17:51:24.173 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:39.003945+0000", 2024-04-07T17:51:24.173 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:39.004542+0000", 2024-04-07T17:51:24.173 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:35.989606+0000", 2024-04-07T17:51:24.173 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.173 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:24.173 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.173 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:24.173 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:35.989606+0000", 2024-04-07T17:51:24.173 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:35.989606+0000", 2024-04-07T17:51:24.173 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:35.989606+0000", 2024-04-07T17:51:24.173 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:24.173 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:24.173 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.174 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:35.989606+0000", 2024-04-07T17:51:24.174 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:35.989606+0000", 2024-04-07T17:51:24.174 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:24.174 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:24.174 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:24.174 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:24.174 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:24.174 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:24.174 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:24.174 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:24.174 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:24.174 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:24.175 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:24.175 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:24.175 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:24.175 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.2b", 2024-04-07T17:51:24.175 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:24.175 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:24.175 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:24.175 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4997, 2024-04-07T17:51:24.175 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:24.175 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T18:56:04.984002+0000", 2024-04-07T17:51:24.175 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:24.175 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:24.175 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:24.175 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:24.176 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:24.176 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:24.176 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:24.176 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:24.176 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:24.176 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:24.176 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:24.176 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:24.176 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:24.176 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:24.176 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:24.176 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:24.176 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:24.176 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:24.176 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:24.177 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:24.177 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:24.177 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:24.177 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:24.177 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:24.177 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:24.177 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:24.177 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:24.177 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:24.177 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:24.177 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:24.177 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:24.177 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:24.177 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:24.177 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:24.178 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:24.178 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:24.178 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:24.178 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:24.178 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:24.178 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:24.178 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:24.178 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:24.178 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:24.178 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.178 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:24.178 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:24.178 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:24.179 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:24.179 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:24.179 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T17:51:24.179 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.179 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 0, 2024-04-07T17:51:24.179 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:24.179 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.179 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:24.179 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:24.179 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:24.179 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:24.179 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T17:51:24.179 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.179 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 5, 2024-04-07T17:51:24.179 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:24.180 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:24.180 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:24.180 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:24.180 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:24.180 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:24.180 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.369807+0000", 2024-04-07T17:51:24.180 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:38.977861+0000", 2024-04-07T17:51:24.180 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:38.977861+0000", 2024-04-07T17:51:24.180 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:38.978928+0000", 2024-04-07T17:51:24.181 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.369807+0000", 2024-04-07T17:51:24.181 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.181 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:24.181 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.181 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:24.181 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.369807+0000", 2024-04-07T17:51:24.181 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.369807+0000", 2024-04-07T17:51:24.181 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.369807+0000", 2024-04-07T17:51:24.181 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:24.181 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:24.181 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.181 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.369807+0000", 2024-04-07T17:51:24.181 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.369807+0000", 2024-04-07T17:51:24.182 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:24.182 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:24.182 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:24.182 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:24.182 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:24.182 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:24.182 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:24.182 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:24.182 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:24.182 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:24.182 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:24.182 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:24.182 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:24.182 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.2a", 2024-04-07T17:51:24.182 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:24.183 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:24.183 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:24.183 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4871, 2024-04-07T17:51:24.183 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:24.183 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T01:59:28.849179+0000", 2024-04-07T17:51:24.183 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:24.183 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:24.183 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:24.183 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:24.183 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:24.183 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:24.183 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:24.183 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:24.183 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:24.183 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:24.184 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:24.184 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:24.184 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:24.184 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:24.184 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:24.184 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:24.184 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:24.184 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:24.184 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:24.184 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:24.184 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:24.184 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:24.185 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:24.185 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:24.185 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:24.185 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:24.185 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:24.185 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:24.185 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:24.185 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:24.185 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:24.185 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:24.185 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:24.185 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:24.185 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:24.185 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:24.185 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:24.185 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:24.186 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:24.186 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:24.186 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:24.186 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:24.186 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:24.186 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.186 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:24.186 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:24.186 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:24.186 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:24.186 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:24.186 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T17:51:24.186 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.187 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 5, 2024-04-07T17:51:24.187 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:24.187 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.187 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:24.187 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:24.187 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:24.187 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:24.187 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T17:51:24.187 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.187 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 3, 2024-04-07T17:51:24.187 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:24.187 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:24.187 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:24.187 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:24.187 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:24.188 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:24.188 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:35.976785+0000", 2024-04-07T17:51:24.188 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:24:13.933027+0000", 2024-04-07T17:51:24.188 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:24:13.933027+0000", 2024-04-07T17:51:24.188 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:24:13.933359+0000", 2024-04-07T17:51:24.188 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:35.976785+0000", 2024-04-07T17:51:24.188 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.188 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:24.188 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.188 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 500, 2024-04-07T17:51:24.188 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:35.976785+0000", 2024-04-07T17:51:24.188 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:35.976785+0000", 2024-04-07T17:51:24.188 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:35.976785+0000", 2024-04-07T17:51:24.188 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:24.188 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:24.188 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.188 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:35.976785+0000", 2024-04-07T17:51:24.189 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:35.976785+0000", 2024-04-07T17:51:24.189 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:24.189 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:24.189 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:24.189 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:24.189 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 499, 2024-04-07T17:51:24.189 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:24.189 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:24.189 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:24.189 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:24.189 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:24.189 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:24.189 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:24.189 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:24.189 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.29", 2024-04-07T17:51:24.190 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:24.190 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:24.190 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:24.190 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4966, 2024-04-07T17:51:24.190 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:24.190 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T00:08:14.265846+0000", 2024-04-07T17:51:24.190 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:24.190 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:24.190 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:24.190 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:24.190 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:24.191 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:24.191 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:24.191 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:24.191 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:24.191 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:24.191 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:24.191 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:24.191 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:24.191 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:24.191 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:24.191 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:24.191 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:24.191 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:24.192 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:24.192 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:24.192 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:24.192 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:24.192 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:24.192 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:24.192 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:24.192 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:24.192 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:24.192 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:24.192 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:24.192 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:24.192 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:24.192 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:24.192 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:24.193 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:24.193 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:24.193 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:24.193 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:24.193 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:24.193 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:24.193 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:24.193 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:24.193 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:24.193 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:24.194 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.194 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:24.194 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:24.194 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:24.194 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:24.195 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:24.195 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T17:51:24.195 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.195 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 3, 2024-04-07T17:51:24.195 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:24.195 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.195 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:24.195 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:24.195 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:24.195 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:24.195 INFO:tasks.workunit.client.0.smithi100.stdout: 2 2024-04-07T17:51:24.195 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.195 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 5, 2024-04-07T17:51:24.195 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:24.195 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:24.196 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:24.196 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:24.196 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:24.196 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:24.196 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.363975+0000", 2024-04-07T17:51:24.196 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:24:41.553450+0000", 2024-04-07T17:51:24.196 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:24:41.553450+0000", 2024-04-07T17:51:24.196 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:24:41.557281+0000", 2024-04-07T17:51:24.196 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.363975+0000", 2024-04-07T17:51:24.196 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.196 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:24.196 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.196 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 527, 2024-04-07T17:51:24.197 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.363975+0000", 2024-04-07T17:51:24.197 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.363975+0000", 2024-04-07T17:51:24.197 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.363975+0000", 2024-04-07T17:51:24.197 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:24.197 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:24.197 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.197 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.363975+0000", 2024-04-07T17:51:24.197 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.363975+0000", 2024-04-07T17:51:24.197 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:24.197 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:24.197 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:24.197 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:24.197 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 526, 2024-04-07T17:51:24.197 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:24.197 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:24.197 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:24.198 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:24.198 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:24.198 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:24.198 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:24.198 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:24.198 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.28", 2024-04-07T17:51:24.198 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:24.198 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:24.199 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:24.199 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4884, 2024-04-07T17:51:24.199 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:24.199 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T19:49:37.795766+0000", 2024-04-07T17:51:24.199 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:24.199 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:24.199 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:24.199 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:24.199 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:24.199 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:24.199 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:24.199 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:24.199 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:24.199 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:24.200 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:24.200 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:24.200 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:24.200 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:24.200 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:24.200 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:24.200 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:24.200 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:24.200 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:24.200 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:24.200 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:24.200 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:24.200 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:24.200 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:24.200 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:24.200 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:24.201 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:24.201 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:24.201 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:24.201 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:24.201 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:24.201 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:24.201 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:24.201 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:24.201 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:24.201 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:24.201 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:24.202 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:24.202 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:24.202 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:24.202 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:24.202 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:24.202 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:24.202 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.202 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:24.202 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:24.202 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:24.202 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:24.203 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:24.203 INFO:tasks.workunit.client.0.smithi100.stdout: 2 2024-04-07T17:51:24.203 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.203 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 5, 2024-04-07T17:51:24.203 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:24.203 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.203 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:24.203 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:24.203 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:24.203 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:24.203 INFO:tasks.workunit.client.0.smithi100.stdout: 2 2024-04-07T17:51:24.203 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.203 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 3, 2024-04-07T17:51:24.203 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:24.203 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:24.203 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:24.204 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:24.204 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:24.204 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:24.204 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:35.982188+0000", 2024-04-07T17:51:24.204 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:24:13.933691+0000", 2024-04-07T17:51:24.204 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:24:13.933691+0000", 2024-04-07T17:51:24.204 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:24:13.934646+0000", 2024-04-07T17:51:24.204 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:35.982188+0000", 2024-04-07T17:51:24.204 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.204 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:24.204 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.204 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 500, 2024-04-07T17:51:24.204 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:35.982188+0000", 2024-04-07T17:51:24.204 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:35.982188+0000", 2024-04-07T17:51:24.204 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:35.982188+0000", 2024-04-07T17:51:24.205 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:24.205 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:24.205 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.205 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:35.982188+0000", 2024-04-07T17:51:24.205 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:35.982188+0000", 2024-04-07T17:51:24.205 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:24.205 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:24.205 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:24.205 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:24.205 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 499, 2024-04-07T17:51:24.205 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:24.205 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:24.205 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:24.206 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:24.206 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:24.206 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:24.206 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:24.206 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:24.206 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.27", 2024-04-07T17:51:24.206 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:24.206 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:24.206 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:24.206 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4954, 2024-04-07T17:51:24.206 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:24.206 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T19:33:33.960559+0000", 2024-04-07T17:51:24.206 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:24.206 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:24.207 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:24.207 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:24.207 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:24.207 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:24.207 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:24.207 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:24.207 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:24.207 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:24.207 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:24.207 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:24.207 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:24.207 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:24.207 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:24.207 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:24.207 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:24.207 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:24.208 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:24.208 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:24.208 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:24.208 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:24.208 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:24.208 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:24.208 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:24.208 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:24.208 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:24.208 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:24.208 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:24.208 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:24.208 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:24.208 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:24.208 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:24.208 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:24.209 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:24.209 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:24.209 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:24.209 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:24.209 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:24.209 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:24.209 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:24.209 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:24.209 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:24.209 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.209 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:24.209 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:24.209 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:24.209 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:24.209 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:24.209 INFO:tasks.workunit.client.0.smithi100.stdout: 2 2024-04-07T17:51:24.209 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.210 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 3, 2024-04-07T17:51:24.210 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:24.210 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.210 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:24.210 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:24.210 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:24.210 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:24.210 INFO:tasks.workunit.client.0.smithi100.stdout: 4 2024-04-07T17:51:24.210 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.210 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 6, 2024-04-07T17:51:24.210 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:24.210 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:24.210 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:24.210 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:24.211 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:24.211 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:24.211 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.368356+0000", 2024-04-07T17:51:24.211 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:38.960595+0000", 2024-04-07T17:51:24.211 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:38.960595+0000", 2024-04-07T17:51:24.211 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:38.962091+0000", 2024-04-07T17:51:24.211 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.368356+0000", 2024-04-07T17:51:24.211 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.211 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:24.211 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.211 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:24.211 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.368356+0000", 2024-04-07T17:51:24.211 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.368356+0000", 2024-04-07T17:51:24.212 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.368356+0000", 2024-04-07T17:51:24.212 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:24.212 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:24.212 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.212 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.368356+0000", 2024-04-07T17:51:24.212 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.368356+0000", 2024-04-07T17:51:24.212 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:24.212 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:24.212 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:24.212 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:24.212 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:24.212 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:24.212 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:24.212 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:24.213 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:24.213 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:24.213 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:24.213 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:24.213 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:24.213 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.26", 2024-04-07T17:51:24.213 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:24.213 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:24.213 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:24.213 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4841, 2024-04-07T17:51:24.213 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:24.213 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T19:11:01.462021+0000", 2024-04-07T17:51:24.213 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:24.213 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:24.213 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:24.213 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:24.214 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:24.214 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:24.214 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:24.214 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:24.214 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:24.214 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:24.214 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:24.214 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:24.214 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:24.214 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:24.214 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:24.214 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:24.214 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:24.214 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:24.214 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:24.215 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:24.215 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:24.215 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:24.215 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:24.215 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:24.215 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:24.215 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:24.215 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:24.215 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:24.215 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:24.215 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:24.215 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:24.215 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:24.215 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:24.216 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:24.216 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:24.216 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:24.216 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:24.216 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:24.216 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:24.216 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:24.216 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:24.216 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:24.216 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:24.216 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.216 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:24.216 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:24.216 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:24.216 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:24.216 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:24.217 INFO:tasks.workunit.client.0.smithi100.stdout: 4 2024-04-07T17:51:24.217 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.217 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 6, 2024-04-07T17:51:24.217 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:24.217 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.217 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:24.217 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:24.217 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:24.217 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:24.217 INFO:tasks.workunit.client.0.smithi100.stdout: 2 2024-04-07T17:51:24.217 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.217 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 1, 2024-04-07T17:51:24.217 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:24.217 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:24.217 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:24.218 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:24.218 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:24.218 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:24.218 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.334473+0000", 2024-04-07T17:51:24.218 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:23:43.924498+0000", 2024-04-07T17:51:24.218 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:23:43.924498+0000", 2024-04-07T17:51:24.218 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:23:43.926295+0000", 2024-04-07T17:51:24.218 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.334473+0000", 2024-04-07T17:51:24.218 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.218 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:24.218 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.218 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 471, 2024-04-07T17:51:24.218 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.334473+0000", 2024-04-07T17:51:24.219 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.334473+0000", 2024-04-07T17:51:24.219 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.334473+0000", 2024-04-07T17:51:24.219 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:24.219 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:24.219 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.219 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.334473+0000", 2024-04-07T17:51:24.219 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.334473+0000", 2024-04-07T17:51:24.219 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:24.219 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:24.219 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:24.219 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:24.219 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 470, 2024-04-07T17:51:24.219 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:24.219 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:24.219 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:24.220 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:24.220 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:24.220 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:24.220 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:24.220 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:24.220 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.25", 2024-04-07T17:51:24.220 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:24.220 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:24.220 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:24.220 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4981, 2024-04-07T17:51:24.220 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:24.220 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T17:58:42.938076+0000", 2024-04-07T17:51:24.220 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:24.220 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:24.220 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:24.220 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:24.221 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:24.221 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:24.221 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:24.221 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:24.221 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:24.221 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:24.221 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:24.221 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:24.221 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:24.221 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:24.221 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:24.221 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:24.221 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:24.221 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:24.221 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:24.222 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:24.222 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:24.222 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:24.222 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:24.222 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:24.222 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:24.222 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:24.222 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:24.222 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:24.223 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:24.223 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:24.223 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:24.223 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:24.223 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:24.223 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:24.223 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:24.223 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:24.223 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:24.223 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:24.223 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:24.223 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:24.223 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:24.223 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:24.223 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:24.224 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.224 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:24.224 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:24.224 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:24.224 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:24.224 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:24.224 INFO:tasks.workunit.client.0.smithi100.stdout: 2 2024-04-07T17:51:24.224 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.224 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 1, 2024-04-07T17:51:24.224 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:24.225 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.225 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:24.225 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:24.225 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:24.225 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:24.225 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T17:51:24.226 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.226 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 3, 2024-04-07T17:51:24.226 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:24.226 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:24.226 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:24.226 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:24.226 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:24.226 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:24.226 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:35.977918+0000", 2024-04-07T17:51:24.227 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:24:13.930420+0000", 2024-04-07T17:51:24.227 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:24:13.930420+0000", 2024-04-07T17:51:24.227 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:24:13.931275+0000", 2024-04-07T17:51:24.227 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:35.977918+0000", 2024-04-07T17:51:24.227 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.227 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:24.227 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.227 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 500, 2024-04-07T17:51:24.228 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:35.977918+0000", 2024-04-07T17:51:24.228 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:35.977918+0000", 2024-04-07T17:51:24.228 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:35.977918+0000", 2024-04-07T17:51:24.228 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:24.228 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:24.228 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.228 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:35.977918+0000", 2024-04-07T17:51:24.228 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:35.977918+0000", 2024-04-07T17:51:24.228 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:24.229 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:24.229 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:24.229 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:24.229 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 499, 2024-04-07T17:51:24.229 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:24.229 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:24.229 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:24.229 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:24.229 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:24.230 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:24.230 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:24.230 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:24.230 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.24", 2024-04-07T17:51:24.230 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:24.230 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:24.230 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:24.230 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4940, 2024-04-07T17:51:24.231 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:24.231 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T00:26:09.028937+0000", 2024-04-07T17:51:24.231 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:24.231 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:24.231 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:24.231 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:24.231 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:24.231 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:24.231 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:24.231 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:24.232 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:24.232 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:24.232 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:24.232 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:24.232 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:24.232 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:24.232 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:24.232 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:24.233 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:24.233 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:24.233 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:24.233 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:24.233 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:24.233 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:24.233 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:24.233 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:24.233 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:24.234 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:24.234 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:24.234 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:24.234 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:24.234 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:24.234 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:24.234 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:24.234 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:24.235 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:24.235 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:24.235 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:24.235 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:24.235 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:24.235 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:24.235 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:24.235 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:24.235 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:24.236 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:24.236 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.236 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:24.236 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:24.236 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:24.236 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:24.236 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:24.236 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T17:51:24.236 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.237 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 3, 2024-04-07T17:51:24.237 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:24.237 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.237 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:24.237 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:24.237 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:24.237 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:24.237 INFO:tasks.workunit.client.0.smithi100.stdout: 4 2024-04-07T17:51:24.238 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.238 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 0, 2024-04-07T17:51:24.238 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:24.238 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:24.238 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:24.238 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:24.238 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:24.238 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:24.239 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:35.989350+0000", 2024-04-07T17:51:24.239 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:24:13.962893+0000", 2024-04-07T17:51:24.239 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:24:13.962893+0000", 2024-04-07T17:51:24.239 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:24:13.964515+0000", 2024-04-07T17:51:24.239 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:35.989350+0000", 2024-04-07T17:51:24.239 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.239 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:24.239 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.239 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 500, 2024-04-07T17:51:24.240 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:35.989350+0000", 2024-04-07T17:51:24.240 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:35.989350+0000", 2024-04-07T17:51:24.240 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:35.989350+0000", 2024-04-07T17:51:24.240 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:24.240 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:24.240 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.240 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:35.989350+0000", 2024-04-07T17:51:24.240 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:35.989350+0000", 2024-04-07T17:51:24.240 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:24.240 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:24.240 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:24.240 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:24.240 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 499, 2024-04-07T17:51:24.240 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:24.240 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:24.241 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:24.241 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:24.241 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:24.241 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:24.241 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:24.241 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:24.241 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.23", 2024-04-07T17:51:24.241 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:24.241 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:24.241 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:24.241 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 5000, 2024-04-07T17:51:24.241 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:24.241 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T17:30:33.589121+0000", 2024-04-07T17:51:24.241 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:24.241 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:24.242 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:24.242 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:24.242 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:24.242 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:24.242 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:24.242 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:24.242 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:24.242 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:24.242 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:24.242 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:24.242 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:24.243 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:24.243 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:24.243 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:24.243 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:24.243 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:24.243 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:24.243 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:24.243 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:24.243 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:24.243 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:24.243 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:24.243 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:24.243 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:24.243 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:24.244 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:24.244 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:24.244 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:24.244 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:24.244 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:24.244 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:24.244 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:24.244 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:24.244 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:24.244 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:24.244 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:24.244 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:24.244 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:24.244 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:24.244 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:24.244 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:24.245 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.245 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:24.245 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:24.245 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:24.245 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:24.245 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:24.245 INFO:tasks.workunit.client.0.smithi100.stdout: 4 2024-04-07T17:51:24.245 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.245 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 0, 2024-04-07T17:51:24.245 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:24.245 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.245 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:24.245 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:24.245 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:24.245 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:24.246 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T17:51:24.246 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.246 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 5, 2024-04-07T17:51:24.246 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:24.246 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:24.246 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:24.246 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:24.246 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:24.246 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:24.246 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.369012+0000", 2024-04-07T17:51:24.246 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:24:41.561766+0000", 2024-04-07T17:51:24.246 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:24:41.561766+0000", 2024-04-07T17:51:24.246 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:24:41.564435+0000", 2024-04-07T17:51:24.246 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.369012+0000", 2024-04-07T17:51:24.247 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.247 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:24.247 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.247 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 527, 2024-04-07T17:51:24.247 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.369012+0000", 2024-04-07T17:51:24.247 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.369012+0000", 2024-04-07T17:51:24.247 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.369012+0000", 2024-04-07T17:51:24.247 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:24.247 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:24.247 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.247 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.369012+0000", 2024-04-07T17:51:24.247 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.369012+0000", 2024-04-07T17:51:24.247 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:24.247 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:24.248 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:24.248 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:24.248 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 526, 2024-04-07T17:51:24.248 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:24.248 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:24.248 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:24.248 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:24.248 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:24.248 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:24.248 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:24.248 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:24.248 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.22", 2024-04-07T17:51:24.248 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:24.248 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:24.249 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:24.249 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4886, 2024-04-07T17:51:24.249 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:24.249 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T17:08:34.046549+0000", 2024-04-07T17:51:24.249 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:24.249 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:24.249 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:24.249 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:24.249 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:24.249 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:24.249 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:24.249 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:24.249 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:24.249 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:24.250 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:24.250 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:24.250 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:24.250 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:24.250 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:24.250 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:24.250 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:24.250 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:24.250 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:24.250 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:24.250 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:24.250 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:24.250 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:24.250 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:24.251 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:24.251 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:24.251 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:24.251 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:24.251 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:24.251 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:24.251 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:24.251 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:24.251 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:24.251 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:24.251 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:24.251 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:24.251 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:24.251 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:24.251 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:24.252 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:24.252 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:24.252 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:24.252 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:24.252 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.252 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:24.252 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:24.252 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:24.252 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:24.252 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:24.252 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T17:51:24.252 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.252 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 5, 2024-04-07T17:51:24.252 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:24.252 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.252 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:24.253 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:24.253 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T17:51:24.253 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:24.253 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T17:51:24.253 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.253 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 2, 2024-04-07T17:51:24.253 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:24.253 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:24.253 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:24.253 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:24.253 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:24.253 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:24.253 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.354974+0000", 2024-04-07T17:51:24.254 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:08.976322+0000", 2024-04-07T17:51:24.254 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:08.976322+0000", 2024-04-07T17:51:24.254 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:08.976897+0000", 2024-04-07T17:51:24.254 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.354974+0000", 2024-04-07T17:51:24.254 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.254 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:24.254 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.254 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 554, 2024-04-07T17:51:24.254 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.354974+0000", 2024-04-07T17:51:24.254 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.354974+0000", 2024-04-07T17:51:24.254 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.354974+0000", 2024-04-07T17:51:24.254 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:24.254 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:24.255 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.255 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.354974+0000", 2024-04-07T17:51:24.255 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.354974+0000", 2024-04-07T17:51:24.255 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:24.255 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:24.255 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:24.255 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:24.255 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 553, 2024-04-07T17:51:24.255 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:24.255 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:24.255 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:24.255 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:24.255 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:24.255 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:24.255 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:24.256 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:24.256 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.21", 2024-04-07T17:51:24.256 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:24.256 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:24.256 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:24.256 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4970, 2024-04-07T17:51:24.256 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:24.256 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T18:02:01.757959+0000", 2024-04-07T17:51:24.256 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:24.256 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:24.256 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:24.256 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:24.256 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:24.257 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:24.257 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:24.257 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:24.257 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:24.257 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:24.257 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:24.257 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:24.257 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:24.257 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:24.257 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:24.257 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:24.257 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:24.257 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:24.257 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:24.257 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:24.257 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:24.257 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:24.258 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:24.258 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:24.258 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:24.258 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:24.258 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:24.258 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:24.258 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:24.258 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:24.258 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:24.258 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:24.258 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:24.258 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:24.258 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:24.258 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:24.259 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:24.259 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:24.259 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:24.259 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:24.259 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:24.259 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:24.259 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:24.259 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.259 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:24.259 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:24.259 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:24.259 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T17:51:24.259 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:24.259 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T17:51:24.259 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.259 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 2, 2024-04-07T17:51:24.260 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:24.260 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.260 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:24.260 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:24.260 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:24.260 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:24.260 INFO:tasks.workunit.client.0.smithi100.stdout: 5 2024-04-07T17:51:24.260 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.260 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 6, 2024-04-07T17:51:24.260 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:24.260 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:24.260 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:24.260 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:24.260 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:24.260 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:24.260 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.370975+0000", 2024-04-07T17:51:24.260 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:08.975767+0000", 2024-04-07T17:51:24.261 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:08.975767+0000", 2024-04-07T17:51:24.261 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:08.976755+0000", 2024-04-07T17:51:24.261 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.370975+0000", 2024-04-07T17:51:24.261 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.261 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:24.261 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.261 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 554, 2024-04-07T17:51:24.261 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.370975+0000", 2024-04-07T17:51:24.261 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.370975+0000", 2024-04-07T17:51:24.261 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.370975+0000", 2024-04-07T17:51:24.261 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:24.261 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:24.261 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.261 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.370975+0000", 2024-04-07T17:51:24.261 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.370975+0000", 2024-04-07T17:51:24.261 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:24.261 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:24.262 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:24.262 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:24.262 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 553, 2024-04-07T17:51:24.262 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:24.262 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:24.262 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:24.262 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:24.262 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:24.262 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:24.262 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:24.262 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:24.262 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.20", 2024-04-07T17:51:24.262 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:24.262 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:24.263 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:24.263 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4834, 2024-04-07T17:51:24.263 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:24.263 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T16:33:00.288075+0000", 2024-04-07T17:51:24.263 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:24.263 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:24.263 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:24.263 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:24.263 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:24.263 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:24.263 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:24.263 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:24.263 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:24.263 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:24.264 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:24.264 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:24.264 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:24.264 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:24.264 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:24.264 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:24.264 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:24.264 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:24.264 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:24.264 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:24.264 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:24.264 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:24.264 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:24.264 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:24.265 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:24.265 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:24.265 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:24.265 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:24.265 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:24.265 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:24.265 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:24.265 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:24.265 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:24.265 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:24.265 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:24.265 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:24.265 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:24.265 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:24.265 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:24.265 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:24.266 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:24.266 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:24.266 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:24.266 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.266 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:24.266 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:24.266 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:24.266 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:24.266 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:24.266 INFO:tasks.workunit.client.0.smithi100.stdout: 5 2024-04-07T17:51:24.266 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.266 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 6, 2024-04-07T17:51:24.266 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:24.266 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.267 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:24.267 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:24.267 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:24.267 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:24.267 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T17:51:24.267 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.267 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 7, 2024-04-07T17:51:24.267 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:24.267 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:24.267 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:24.267 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:24.267 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:24.267 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:24.267 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.367504+0000", 2024-04-07T17:51:24.268 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:38.976388+0000", 2024-04-07T17:51:24.268 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:38.976388+0000", 2024-04-07T17:51:24.268 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:38.994496+0000", 2024-04-07T17:51:24.268 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.367504+0000", 2024-04-07T17:51:24.268 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.268 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:24.268 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.268 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:24.268 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.367504+0000", 2024-04-07T17:51:24.268 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.367504+0000", 2024-04-07T17:51:24.268 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.367504+0000", 2024-04-07T17:51:24.268 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:24.268 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:24.268 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.269 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.367504+0000", 2024-04-07T17:51:24.269 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.367504+0000", 2024-04-07T17:51:24.269 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:24.269 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:24.269 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:24.269 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:24.269 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:24.269 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:24.269 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:24.269 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:24.269 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:24.269 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:24.269 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:24.269 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:24.269 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:24.270 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.f", 2024-04-07T17:51:24.270 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:24.270 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:24.270 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:24.270 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4790, 2024-04-07T17:51:24.270 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:24.270 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T17:59:57.662112+0000", 2024-04-07T17:51:24.270 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:24.270 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:24.270 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:24.270 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:24.270 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:24.271 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:24.271 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:24.271 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:24.271 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:24.271 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:24.271 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:24.271 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:24.271 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:24.271 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:24.271 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:24.271 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:24.271 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:24.271 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:24.271 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:24.272 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:24.272 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:24.272 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:24.272 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:24.272 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:24.272 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:24.272 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:24.272 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:24.272 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:24.272 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:24.272 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:24.272 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:24.272 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:24.272 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:24.272 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:24.272 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:24.273 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:24.273 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:24.273 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:24.273 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:24.273 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:24.273 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:24.273 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:24.273 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:24.273 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.273 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:24.273 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:24.273 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:24.273 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:24.273 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:24.273 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T17:51:24.273 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.274 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 7, 2024-04-07T17:51:24.274 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:24.274 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.274 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:24.274 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:24.274 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T17:51:24.274 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:24.274 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T17:51:24.274 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.274 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 2, 2024-04-07T17:51:24.274 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:24.274 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:24.274 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T17:51:24.274 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:24.275 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:24.275 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:24.275 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.359582+0000", 2024-04-07T17:51:24.275 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:38.967914+0000", 2024-04-07T17:51:24.275 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:38.967914+0000", 2024-04-07T17:51:24.275 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:38.968390+0000", 2024-04-07T17:51:24.275 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.359582+0000", 2024-04-07T17:51:24.275 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:24.275 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:24.275 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:24.275 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:24.275 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.359582+0000", 2024-04-07T17:51:24.275 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.359582+0000", 2024-04-07T17:51:24.275 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.359582+0000", 2024-04-07T17:51:24.275 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:24.276 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:24.276 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:24.276 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.359582+0000", 2024-04-07T17:51:24.276 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.359582+0000", 2024-04-07T17:51:24.276 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:24.276 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 2, 2024-04-07T17:51:24.276 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:24.276 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:24.276 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:24.276 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:24.276 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:24.276 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:24.276 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:24.276 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 2, 2024-04-07T17:51:24.277 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:24.277 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:24.277 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:24.277 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.e", 2024-04-07T17:51:24.277 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:24.277 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:24.277 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:24.277 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4971, 2024-04-07T17:51:24.277 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:24.277 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T21:28:08.591091+0000", 2024-04-07T17:51:24.277 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:24.277 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:24.277 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:24.277 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:24.278 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:24.278 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:24.278 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:24.278 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:24.278 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:24.278 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:24.278 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:24.278 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:24.278 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:24.278 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:24.278 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:24.278 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:24.278 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:24.278 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:24.279 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:24.279 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 3, 2024-04-07T17:51:24.279 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 1, 2024-04-07T17:51:24.279 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:24.279 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 1, 2024-04-07T17:51:24.279 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:24.279 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:24.279 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:24.279 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:24.279 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:24.279 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 1, 2024-04-07T17:51:24.279 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:24.279 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:24.279 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:24.279 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:24.280 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:24.280 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:24.280 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:24.280 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 6, 2024-04-07T17:51:24.280 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 6, 2024-04-07T17:51:24.280 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:24.280 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:24.280 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:24.280 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 3, 2024-04-07T17:51:24.280 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:24.280 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.280 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:24.280 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:24.280 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:24.280 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T17:51:24.281 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:24.281 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T17:51:24.281 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.281 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 2, 2024-04-07T17:51:24.281 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "61'2" 2024-04-07T17:51:24.281 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.281 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:24.281 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:24.281 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:24.281 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:24.281 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T17:51:24.281 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.282 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 5, 2024-04-07T17:51:24.282 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:24.282 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:24.282 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:24.282 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:24.282 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:24.282 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:24.282 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.366524+0000", 2024-04-07T17:51:24.282 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:08.971755+0000", 2024-04-07T17:51:24.282 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:08.971755+0000", 2024-04-07T17:51:24.282 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:08.973166+0000", 2024-04-07T17:51:24.282 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.366524+0000", 2024-04-07T17:51:24.282 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.283 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:24.283 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.283 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 554, 2024-04-07T17:51:24.283 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.366524+0000", 2024-04-07T17:51:24.283 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.366524+0000", 2024-04-07T17:51:24.283 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.366524+0000", 2024-04-07T17:51:24.283 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:24.283 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:24.283 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.283 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.366524+0000", 2024-04-07T17:51:24.283 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.366524+0000", 2024-04-07T17:51:24.283 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:24.283 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:24.283 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:24.283 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:24.284 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 553, 2024-04-07T17:51:24.284 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:24.284 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:24.284 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:24.284 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:24.284 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:24.284 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:24.284 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:24.284 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:24.284 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.c", 2024-04-07T17:51:24.284 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:24.284 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:24.284 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:24.284 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4877, 2024-04-07T17:51:24.284 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:24.285 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T02:59:49.583756+0000", 2024-04-07T17:51:24.285 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:24.285 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:24.285 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:24.285 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:24.285 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:24.285 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:24.285 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:24.285 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:24.285 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:24.285 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:24.285 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:24.285 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:24.285 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:24.286 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:24.286 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:24.286 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:24.286 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:24.286 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:24.286 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:24.286 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:24.286 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:24.286 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:24.286 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:24.286 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:24.286 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:24.286 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:24.286 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:24.286 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:24.287 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:24.287 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:24.287 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:24.287 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:24.287 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:24.287 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:24.287 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:24.287 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:24.287 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:24.287 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:24.287 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:24.287 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:24.287 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:24.287 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:24.287 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:24.287 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.288 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:24.288 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:24.288 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:24.288 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:24.288 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:24.288 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T17:51:24.288 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.288 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 5, 2024-04-07T17:51:24.288 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:24.288 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.288 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:24.288 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:24.288 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:24.288 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:24.288 INFO:tasks.workunit.client.0.smithi100.stdout: 3 2024-04-07T17:51:24.288 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.289 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 1, 2024-04-07T17:51:24.289 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:24.289 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:24.289 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T17:51:24.289 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:24.289 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:24.289 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:24.289 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.333944+0000", 2024-04-07T17:51:24.289 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:24:13.932662+0000", 2024-04-07T17:51:24.289 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:24:13.932662+0000", 2024-04-07T17:51:24.289 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:24:13.933216+0000", 2024-04-07T17:51:24.289 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.333944+0000", 2024-04-07T17:51:24.289 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:24.289 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:24.289 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:24.290 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 500, 2024-04-07T17:51:24.290 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.333944+0000", 2024-04-07T17:51:24.290 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.333944+0000", 2024-04-07T17:51:24.290 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.333944+0000", 2024-04-07T17:51:24.290 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:24.290 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:24.290 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:24.290 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.333944+0000", 2024-04-07T17:51:24.290 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.333944+0000", 2024-04-07T17:51:24.290 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:24.290 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 1, 2024-04-07T17:51:24.290 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:24.290 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:24.290 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 499, 2024-04-07T17:51:24.291 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:24.291 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:24.291 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:24.291 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:24.291 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 1, 2024-04-07T17:51:24.291 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:24.291 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:24.291 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:24.291 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.d", 2024-04-07T17:51:24.291 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:24.291 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:24.291 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:24.291 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4983, 2024-04-07T17:51:24.291 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:24.291 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T17:17:11.134012+0000", 2024-04-07T17:51:24.292 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:24.292 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:24.292 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:24.292 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 70, 2024-04-07T17:51:24.292 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:24.292 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:24.292 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:24.292 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:24.292 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:24.292 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:24.292 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:24.292 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:24.292 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:24.292 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:24.292 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:24.293 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:24.293 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:24.293 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:24.293 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:24.293 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 3, 2024-04-07T17:51:24.293 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 1, 2024-04-07T17:51:24.293 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:24.293 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 1, 2024-04-07T17:51:24.293 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:24.293 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:24.293 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:24.293 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:24.293 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:24.293 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:24.293 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:24.294 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:24.294 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:24.294 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:24.294 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:24.294 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:24.294 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:24.294 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 1, 2024-04-07T17:51:24.294 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 1, 2024-04-07T17:51:24.294 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:24.294 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:24.294 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:24.294 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 1, 2024-04-07T17:51:24.294 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 1 2024-04-07T17:51:24.294 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.295 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:24.295 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:24.295 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:24.295 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:24.295 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:24.295 INFO:tasks.workunit.client.0.smithi100.stdout: 3 2024-04-07T17:51:24.295 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.295 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 1, 2024-04-07T17:51:24.295 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "61'1" 2024-04-07T17:51:24.295 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.295 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:24.295 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:24.295 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:24.295 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:24.296 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T17:51:24.296 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.296 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 7, 2024-04-07T17:51:24.296 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:24.296 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:24.296 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:24.296 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:24.296 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:24.296 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:24.296 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.373148+0000", 2024-04-07T17:51:24.296 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:38.989051+0000", 2024-04-07T17:51:24.296 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:38.989051+0000", 2024-04-07T17:51:24.296 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:39.000662+0000", 2024-04-07T17:51:24.296 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.373148+0000", 2024-04-07T17:51:24.296 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.297 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:24.297 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.297 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:24.297 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.373148+0000", 2024-04-07T17:51:24.297 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.373148+0000", 2024-04-07T17:51:24.297 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.373148+0000", 2024-04-07T17:51:24.297 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:24.297 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:24.297 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.297 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.373148+0000", 2024-04-07T17:51:24.297 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.373148+0000", 2024-04-07T17:51:24.297 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:24.297 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:24.297 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:24.297 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:24.297 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:24.298 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:24.298 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:24.298 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:24.298 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:24.298 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:24.298 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:24.298 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:24.298 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:24.298 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.d", 2024-04-07T17:51:24.298 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:24.298 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:24.298 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:24.298 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4803, 2024-04-07T17:51:24.298 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:24.298 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T23:45:21.166051+0000", 2024-04-07T17:51:24.299 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:24.299 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:24.299 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:24.299 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:24.299 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:24.299 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:24.299 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:24.299 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:24.299 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:24.299 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:24.299 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:24.299 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:24.299 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:24.299 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:24.299 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:24.299 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:24.300 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:24.300 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:24.300 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:24.300 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:24.300 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:24.300 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:24.300 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:24.300 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:24.300 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:24.300 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:24.300 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:24.300 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:24.300 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:24.300 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:24.300 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:24.300 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:24.300 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:24.301 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:24.301 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:24.301 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:24.301 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:24.301 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:24.301 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:24.301 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:24.301 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:24.301 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:24.301 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:24.301 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.301 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:24.301 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:24.301 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:24.301 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:24.301 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:24.301 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T17:51:24.302 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.302 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 7, 2024-04-07T17:51:24.302 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:24.302 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.302 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:24.302 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:24.302 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T17:51:24.302 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:24.302 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T17:51:24.302 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.302 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 2, 2024-04-07T17:51:24.302 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:24.302 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:24.302 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T17:51:24.302 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:24.302 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:24.303 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:24.303 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.355658+0000", 2024-04-07T17:51:24.303 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:24:41.549188+0000", 2024-04-07T17:51:24.303 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:24:41.549188+0000", 2024-04-07T17:51:24.303 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:24:41.550837+0000", 2024-04-07T17:51:24.303 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.355658+0000", 2024-04-07T17:51:24.303 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:24.303 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:24.303 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:24.303 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 527, 2024-04-07T17:51:24.303 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.355658+0000", 2024-04-07T17:51:24.303 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.355658+0000", 2024-04-07T17:51:24.303 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.355658+0000", 2024-04-07T17:51:24.303 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:24.303 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:24.303 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:24.303 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.355658+0000", 2024-04-07T17:51:24.304 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.355658+0000", 2024-04-07T17:51:24.304 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:24.304 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:24.304 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:24.304 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:24.304 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 526, 2024-04-07T17:51:24.304 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:24.304 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:24.304 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:24.304 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:24.304 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:24.304 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:24.304 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:24.304 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:24.304 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.c", 2024-04-07T17:51:24.304 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:24.304 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:24.305 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:24.305 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4973, 2024-04-07T17:51:24.305 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:24.305 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T18:56:44.543940+0000", 2024-04-07T17:51:24.305 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:24.305 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:24.305 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:24.305 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:24.305 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:24.305 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:24.305 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:24.305 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:24.305 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:24.305 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:24.305 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:24.305 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:24.306 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:24.306 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:24.306 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:24.306 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:24.306 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:24.306 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:24.306 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:24.306 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:24.306 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:24.306 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:24.306 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:24.306 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:24.306 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:24.306 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:24.306 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:24.306 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:24.307 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:24.307 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:24.307 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:24.307 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:24.307 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:24.307 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:24.307 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:24.307 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:24.307 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:24.307 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:24.307 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:24.307 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:24.307 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:24.307 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:24.307 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:24.307 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.308 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:24.308 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:24.308 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:24.308 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T17:51:24.308 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:24.308 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T17:51:24.308 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.308 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 2, 2024-04-07T17:51:24.308 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:24.308 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.308 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:24.308 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:24.308 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:24.308 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:24.308 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T17:51:24.308 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.308 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 6, 2024-04-07T17:51:24.309 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:24.309 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:24.309 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:24.309 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:24.309 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:24.309 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:24.309 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.386750+0000", 2024-04-07T17:51:24.309 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:08.966124+0000", 2024-04-07T17:51:24.309 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:08.966124+0000", 2024-04-07T17:51:24.309 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:08.966628+0000", 2024-04-07T17:51:24.309 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.386750+0000", 2024-04-07T17:51:24.309 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.309 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:24.309 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.309 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 554, 2024-04-07T17:51:24.309 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.386750+0000", 2024-04-07T17:51:24.310 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.386750+0000", 2024-04-07T17:51:24.310 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.386750+0000", 2024-04-07T17:51:24.310 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:24.310 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:24.310 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.310 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.386750+0000", 2024-04-07T17:51:24.310 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.386750+0000", 2024-04-07T17:51:24.310 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:24.310 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:24.310 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:24.310 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:24.310 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 553, 2024-04-07T17:51:24.310 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:24.310 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:24.310 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:24.310 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:24.311 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:24.311 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:24.311 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:24.311 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:24.311 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.a", 2024-04-07T17:51:24.311 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:24.311 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:24.311 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:24.311 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4847, 2024-04-07T17:51:24.311 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:24.311 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T19:57:28.644470+0000", 2024-04-07T17:51:24.311 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:24.311 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:24.311 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:24.311 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:24.311 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:24.311 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:24.312 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:24.312 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:24.312 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:24.312 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:24.312 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:24.312 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:24.312 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:24.312 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:24.312 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:24.312 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:24.312 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:24.312 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:24.312 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:24.312 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:24.312 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:24.312 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:24.313 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:24.313 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:24.313 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:24.313 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:24.313 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:24.313 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:24.313 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:24.313 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:24.313 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:24.313 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:24.313 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:24.313 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:24.313 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:24.313 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:24.313 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:24.313 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:24.313 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:24.314 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:24.314 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:24.314 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:24.314 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:24.314 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.314 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:24.314 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:24.314 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:24.314 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:24.314 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:24.314 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T17:51:24.314 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.314 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 6, 2024-04-07T17:51:24.314 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:24.314 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.315 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:24.315 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:24.315 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:24.315 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:24.315 INFO:tasks.workunit.client.0.smithi100.stdout: 5 2024-04-07T17:51:24.315 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.315 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 7, 2024-04-07T17:51:24.315 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:24.315 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:24.315 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T17:51:24.315 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:24.315 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:24.315 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:24.315 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.369146+0000", 2024-04-07T17:51:24.316 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:38.990309+0000", 2024-04-07T17:51:24.316 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:38.990309+0000", 2024-04-07T17:51:24.316 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:38.992642+0000", 2024-04-07T17:51:24.316 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.369146+0000", 2024-04-07T17:51:24.316 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:24.316 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:24.316 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:24.316 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:24.316 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.369146+0000", 2024-04-07T17:51:24.316 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.369146+0000", 2024-04-07T17:51:24.316 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.369146+0000", 2024-04-07T17:51:24.316 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:24.316 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:24.316 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:24.316 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.369146+0000", 2024-04-07T17:51:24.317 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.369146+0000", 2024-04-07T17:51:24.317 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:24.317 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 3, 2024-04-07T17:51:24.317 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:24.317 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:24.317 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:24.317 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:24.317 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:24.317 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:24.317 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:24.317 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 3, 2024-04-07T17:51:24.317 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:24.317 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:24.318 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:24.318 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.b", 2024-04-07T17:51:24.318 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:24.318 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:24.318 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:24.318 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4823, 2024-04-07T17:51:24.318 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:24.318 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T17:09:27.535561+0000", 2024-04-07T17:51:24.318 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:24.318 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:24.318 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:24.318 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:24.318 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:24.318 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:24.318 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:24.319 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:24.319 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:24.319 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:24.319 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:24.319 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:24.319 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:24.319 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:24.319 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:24.319 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:24.319 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:24.319 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:24.319 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:24.319 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 3, 2024-04-07T17:51:24.319 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 1, 2024-04-07T17:51:24.319 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:24.319 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 1, 2024-04-07T17:51:24.320 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:24.320 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:24.320 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:24.320 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:24.320 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:24.320 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 1, 2024-04-07T17:51:24.320 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:24.320 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:24.320 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:24.320 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:24.320 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:24.320 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:24.320 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:24.320 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 2, 2024-04-07T17:51:24.320 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 2, 2024-04-07T17:51:24.320 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:24.320 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:24.321 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:24.321 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 1, 2024-04-07T17:51:24.321 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:24.321 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.321 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:24.321 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:24.321 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:24.321 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:24.321 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:24.321 INFO:tasks.workunit.client.0.smithi100.stdout: 5 2024-04-07T17:51:24.321 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.321 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 7, 2024-04-07T17:51:24.321 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "623'3" 2024-04-07T17:51:24.321 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.321 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:24.321 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:24.321 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:24.322 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:24.322 INFO:tasks.workunit.client.0.smithi100.stdout: 4 2024-04-07T17:51:24.322 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.322 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 3, 2024-04-07T17:51:24.322 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:24.322 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:24.322 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:24.322 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:24.322 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:24.322 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:24.322 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:35.976901+0000", 2024-04-07T17:51:24.322 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:24:13.933754+0000", 2024-04-07T17:51:24.322 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:24:13.933754+0000", 2024-04-07T17:51:24.322 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:24:13.934742+0000", 2024-04-07T17:51:24.322 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:35.976901+0000", 2024-04-07T17:51:24.323 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.323 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:24.323 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.323 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 500, 2024-04-07T17:51:24.323 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:35.976901+0000", 2024-04-07T17:51:24.323 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:35.976901+0000", 2024-04-07T17:51:24.323 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:35.976901+0000", 2024-04-07T17:51:24.323 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:24.323 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:24.323 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.323 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:35.976901+0000", 2024-04-07T17:51:24.323 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:35.976901+0000", 2024-04-07T17:51:24.323 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:24.323 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:24.323 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:24.323 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:24.324 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 499, 2024-04-07T17:51:24.324 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:24.324 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:24.324 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:24.324 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:24.324 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:24.324 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:24.324 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:24.324 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:24.324 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.b", 2024-04-07T17:51:24.324 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:24.324 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:24.324 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:24.324 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4938, 2024-04-07T17:51:24.325 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:24.325 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T02:43:13.536050+0000", 2024-04-07T17:51:24.325 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:24.325 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:24.325 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:24.325 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:24.325 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:24.325 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:24.325 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:24.325 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:24.325 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:24.325 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:24.325 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:24.325 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:24.325 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:24.325 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:24.325 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:24.326 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:24.326 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:24.326 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:24.326 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:24.326 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:24.326 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:24.326 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:24.326 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:24.326 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:24.326 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:24.326 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:24.326 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:24.326 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:24.326 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:24.327 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:24.327 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:24.327 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:24.327 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:24.327 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:24.327 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:24.327 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:24.327 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:24.327 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:24.327 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:24.327 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:24.327 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:24.327 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:24.327 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:24.327 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.327 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:24.327 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:24.328 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:24.328 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:24.328 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:24.328 INFO:tasks.workunit.client.0.smithi100.stdout: 4 2024-04-07T17:51:24.328 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.328 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 3, 2024-04-07T17:51:24.328 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:24.328 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.328 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:24.328 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:24.328 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:24.328 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:24.328 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T17:51:24.328 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.328 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 1, 2024-04-07T17:51:24.328 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:24.328 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:24.329 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T17:51:24.329 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:24.329 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:24.329 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:24.329 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.335142+0000", 2024-04-07T17:51:24.329 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:38.975734+0000", 2024-04-07T17:51:24.329 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:38.975734+0000", 2024-04-07T17:51:24.329 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:38.976248+0000", 2024-04-07T17:51:24.329 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.335142+0000", 2024-04-07T17:51:24.329 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:24.329 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:24.329 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:24.329 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:24.329 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.335142+0000", 2024-04-07T17:51:24.329 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.335142+0000", 2024-04-07T17:51:24.329 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.335142+0000", 2024-04-07T17:51:24.329 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:24.330 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:24.330 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:24.330 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.335142+0000", 2024-04-07T17:51:24.330 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.335142+0000", 2024-04-07T17:51:24.330 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:24.330 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:24.330 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:24.330 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:24.330 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:24.330 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:24.330 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:24.330 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:24.330 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:24.330 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:24.330 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:24.331 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:24.331 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:24.331 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.a", 2024-04-07T17:51:24.331 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:24.331 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:24.331 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:24.331 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4981, 2024-04-07T17:51:24.331 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:24.331 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T19:45:35.140357+0000", 2024-04-07T17:51:24.331 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:24.331 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:24.331 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:24.331 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:24.331 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:24.331 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:24.332 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:24.332 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:24.332 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:24.332 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:24.332 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:24.332 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:24.332 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:24.332 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:24.332 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:24.332 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:24.332 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:24.332 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:24.332 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:24.332 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:24.332 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:24.333 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:24.333 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:24.333 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:24.333 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:24.333 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:24.333 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:24.333 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:24.333 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:24.333 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:24.333 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:24.333 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:24.333 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:24.333 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:24.334 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:24.334 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:24.334 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:24.334 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:24.334 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:24.334 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:24.334 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:24.334 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:24.335 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:24.335 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.335 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:24.335 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:24.335 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:24.335 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:24.335 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:24.335 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T17:51:24.335 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.335 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 1, 2024-04-07T17:51:24.335 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:24.335 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.336 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:24.336 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:24.336 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:24.336 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:24.336 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T17:51:24.336 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.336 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 3, 2024-04-07T17:51:24.336 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:24.336 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:24.336 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:24.336 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:24.336 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:24.336 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:24.336 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:35.975502+0000", 2024-04-07T17:51:24.336 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:38.997582+0000", 2024-04-07T17:51:24.336 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:38.997582+0000", 2024-04-07T17:51:24.337 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:38.998502+0000", 2024-04-07T17:51:24.337 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:35.975502+0000", 2024-04-07T17:51:24.337 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.337 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:24.337 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.337 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:24.337 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:35.975502+0000", 2024-04-07T17:51:24.337 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:35.975502+0000", 2024-04-07T17:51:24.337 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:35.975502+0000", 2024-04-07T17:51:24.337 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:24.337 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:24.337 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.337 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:35.975502+0000", 2024-04-07T17:51:24.337 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:35.975502+0000", 2024-04-07T17:51:24.338 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:24.338 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:24.338 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:24.338 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:24.338 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:24.338 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:24.338 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:24.338 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:24.338 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:24.338 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:24.338 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:24.338 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:24.338 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:24.338 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.8", 2024-04-07T17:51:24.338 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:24.339 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:24.339 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:24.339 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4968, 2024-04-07T17:51:24.339 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:24.339 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T23:09:31.581874+0000", 2024-04-07T17:51:24.339 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:24.339 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:24.339 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:24.339 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:24.339 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:24.339 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:24.339 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:24.339 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:24.339 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:24.339 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:24.339 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:24.339 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:24.340 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:24.340 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:24.340 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:24.340 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:24.340 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:24.340 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:24.340 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:24.340 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:24.340 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:24.340 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:24.340 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:24.340 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:24.340 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:24.340 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:24.340 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:24.340 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:24.341 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:24.341 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:24.341 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:24.341 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:24.341 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:24.341 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:24.341 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:24.341 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:24.341 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:24.341 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:24.341 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:24.341 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:24.341 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:24.341 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:24.341 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:24.341 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.341 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:24.342 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:24.342 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:24.342 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:24.342 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:24.342 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T17:51:24.342 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.342 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 3, 2024-04-07T17:51:24.342 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:24.342 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.342 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:24.342 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:24.343 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:24.343 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:24.343 INFO:tasks.workunit.client.0.smithi100.stdout: 3 2024-04-07T17:51:24.343 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.343 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 1, 2024-04-07T17:51:24.343 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:24.343 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:24.343 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T17:51:24.343 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:24.343 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:24.344 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:24.344 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.331976+0000", 2024-04-07T17:51:24.344 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:38.992340+0000", 2024-04-07T17:51:24.344 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:38.992340+0000", 2024-04-07T17:51:24.344 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:38.992779+0000", 2024-04-07T17:51:24.344 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.331976+0000", 2024-04-07T17:51:24.344 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:24.344 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:24.344 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:24.344 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:24.344 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.331976+0000", 2024-04-07T17:51:24.344 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.331976+0000", 2024-04-07T17:51:24.345 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.331976+0000", 2024-04-07T17:51:24.345 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:24.345 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:24.345 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:24.345 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.331976+0000", 2024-04-07T17:51:24.345 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.331976+0000", 2024-04-07T17:51:24.345 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:24.345 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:24.345 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:24.346 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:24.346 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:24.346 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:24.346 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:24.346 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:24.346 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:24.346 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:24.346 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:24.346 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:24.346 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:24.346 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.9", 2024-04-07T17:51:24.346 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:24.346 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:24.347 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:24.347 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4981, 2024-04-07T17:51:24.347 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:24.347 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T21:25:43.908090+0000", 2024-04-07T17:51:24.347 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:24.347 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:24.347 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:24.347 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:24.347 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:24.347 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:24.347 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:24.347 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:24.347 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:24.347 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:24.347 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:24.348 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:24.348 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:24.348 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:24.348 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:24.348 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:24.348 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:24.348 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:24.348 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:24.348 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:24.348 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:24.348 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:24.348 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:24.348 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:24.348 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:24.348 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:24.349 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:24.349 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:24.349 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:24.349 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:24.349 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:24.349 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:24.349 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:24.349 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:24.349 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:24.349 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:24.349 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:24.349 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:24.349 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:24.349 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:24.349 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:24.349 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:24.350 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:24.350 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.350 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:24.350 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:24.350 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:24.350 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:24.350 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:24.350 INFO:tasks.workunit.client.0.smithi100.stdout: 3 2024-04-07T17:51:24.350 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.350 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 1, 2024-04-07T17:51:24.350 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:24.350 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.350 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:24.350 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:24.351 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:24.351 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T17:51:24.351 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T17:51:24.351 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.351 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 4, 2024-04-07T17:51:24.351 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:24.351 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:24.351 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:24.351 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:24.351 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:24.351 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:24.351 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:35.977778+0000", 2024-04-07T17:51:24.351 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:38.985206+0000", 2024-04-07T17:51:24.351 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:38.985206+0000", 2024-04-07T17:51:24.351 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:38.985745+0000", 2024-04-07T17:51:24.352 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:35.977778+0000", 2024-04-07T17:51:24.352 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.352 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:24.352 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.352 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:24.352 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:35.977778+0000", 2024-04-07T17:51:24.352 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:35.977778+0000", 2024-04-07T17:51:24.352 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:35.977778+0000", 2024-04-07T17:51:24.352 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:24.352 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:24.352 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.352 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:35.977778+0000", 2024-04-07T17:51:24.352 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:35.977778+0000", 2024-04-07T17:51:24.352 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:24.353 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:24.353 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:24.353 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:24.353 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:24.353 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:24.353 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:24.353 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:24.353 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:24.353 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:24.353 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:24.353 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:24.353 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:24.353 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.9", 2024-04-07T17:51:24.353 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:24.354 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:24.354 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:24.354 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4931, 2024-04-07T17:51:24.354 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:24.354 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T00:51:54.981164+0000", 2024-04-07T17:51:24.354 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:24.354 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:24.354 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:24.354 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:24.354 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:24.354 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:24.354 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:24.354 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:24.354 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:24.355 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:24.355 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:24.355 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:24.355 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:24.355 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:24.355 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:24.355 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:24.355 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:24.355 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:24.355 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:24.355 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:24.355 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:24.355 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:24.355 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:24.355 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:24.356 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:24.356 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:24.356 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:24.356 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:24.356 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:24.356 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:24.356 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:24.356 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:24.356 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:24.356 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:24.356 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:24.356 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:24.356 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:24.356 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:24.356 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:24.357 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:24.357 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:24.357 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:24.357 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:24.357 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.357 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:24.357 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:24.357 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:24.357 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:24.357 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T17:51:24.357 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T17:51:24.357 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.357 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 4, 2024-04-07T17:51:24.357 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:24.358 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.358 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:24.358 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:24.358 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:24.358 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:24.358 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T17:51:24.358 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.358 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 7, 2024-04-07T17:51:24.358 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:24.358 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:24.358 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T17:51:24.358 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:24.358 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:24.358 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:24.359 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.364254+0000", 2024-04-07T17:51:24.359 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:38.996008+0000", 2024-04-07T17:51:24.359 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:38.996008+0000", 2024-04-07T17:51:24.359 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:39.001597+0000", 2024-04-07T17:51:24.359 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.364254+0000", 2024-04-07T17:51:24.359 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:24.359 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:24.359 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:24.359 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:24.359 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.364254+0000", 2024-04-07T17:51:24.359 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.364254+0000", 2024-04-07T17:51:24.359 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.364254+0000", 2024-04-07T17:51:24.359 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:24.359 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:24.360 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:24.360 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.364254+0000", 2024-04-07T17:51:24.360 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.364254+0000", 2024-04-07T17:51:24.360 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:24.360 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:24.360 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:24.360 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:24.360 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:24.360 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:24.360 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:24.360 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:24.360 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:24.360 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:24.360 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:24.360 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:24.360 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:24.361 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.8", 2024-04-07T17:51:24.361 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:24.361 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:24.361 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:24.361 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4792, 2024-04-07T17:51:24.361 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:24.361 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T19:53:15.918325+0000", 2024-04-07T17:51:24.361 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:24.361 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:24.361 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:24.361 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:24.361 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:24.361 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:24.361 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:24.362 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:24.362 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:24.362 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:24.362 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:24.362 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:24.362 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:24.362 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:24.362 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:24.362 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:24.362 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:24.362 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:24.362 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:24.362 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:24.362 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:24.362 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:24.363 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:24.363 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:24.363 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:24.363 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:24.363 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:24.363 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:24.363 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:24.363 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:24.363 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:24.363 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:24.363 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:24.363 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:24.363 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:24.363 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:24.363 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:24.364 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:24.364 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:24.364 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:24.364 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:24.364 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:24.364 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:24.364 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.364 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:24.364 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:24.364 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:24.364 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:24.364 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:24.364 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T17:51:24.364 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.364 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 7, 2024-04-07T17:51:24.365 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:24.365 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.365 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:24.365 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:24.365 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:24.365 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:24.365 INFO:tasks.workunit.client.0.smithi100.stdout: 4 2024-04-07T17:51:24.365 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.365 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 0, 2024-04-07T17:51:24.365 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:24.365 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:24.365 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:24.365 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:24.365 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:24.365 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:24.366 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:35.981688+0000", 2024-04-07T17:51:24.366 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:24:13.962953+0000", 2024-04-07T17:51:24.366 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:24:13.962953+0000", 2024-04-07T17:51:24.366 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:24:13.963599+0000", 2024-04-07T17:51:24.366 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:35.981688+0000", 2024-04-07T17:51:24.366 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.366 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:24.366 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.366 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 500, 2024-04-07T17:51:24.366 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:35.981688+0000", 2024-04-07T17:51:24.366 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:35.981688+0000", 2024-04-07T17:51:24.366 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:35.981688+0000", 2024-04-07T17:51:24.366 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:24.367 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:24.367 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.367 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:35.981688+0000", 2024-04-07T17:51:24.367 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:35.981688+0000", 2024-04-07T17:51:24.367 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:24.367 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:24.367 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:24.367 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:24.367 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 499, 2024-04-07T17:51:24.367 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:24.367 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:24.367 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:24.367 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:24.367 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:24.368 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:24.368 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:24.368 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:24.368 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.6", 2024-04-07T17:51:24.368 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:24.368 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:24.368 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:24.368 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4999, 2024-04-07T17:51:24.368 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:24.368 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T22:29:41.083754+0000", 2024-04-07T17:51:24.368 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:24.368 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:24.368 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:24.368 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:24.368 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:24.369 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:24.369 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:24.369 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:24.369 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:24.369 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:24.369 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:24.369 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:24.369 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:24.369 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:24.369 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:24.369 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:24.369 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:24.369 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:24.369 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:24.369 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:24.369 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:24.370 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:24.370 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:24.370 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:24.370 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:24.370 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:24.370 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:24.370 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:24.370 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:24.370 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:24.370 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:24.370 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:24.370 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:24.370 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:24.370 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:24.371 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:24.371 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:24.371 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:24.371 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:24.371 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:24.371 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:24.371 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:24.371 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:24.371 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.371 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:24.371 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:24.371 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:24.371 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:24.371 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:24.371 INFO:tasks.workunit.client.0.smithi100.stdout: 4 2024-04-07T17:51:24.371 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.372 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 0, 2024-04-07T17:51:24.372 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:24.372 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.372 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:24.372 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:24.372 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:24.372 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:24.372 INFO:tasks.workunit.client.0.smithi100.stdout: 2 2024-04-07T17:51:24.372 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.372 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 6, 2024-04-07T17:51:24.372 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:24.372 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:24.372 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T17:51:24.372 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:24.372 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:24.372 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:24.373 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.369806+0000", 2024-04-07T17:51:24.373 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:08.974754+0000", 2024-04-07T17:51:24.373 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:08.974754+0000", 2024-04-07T17:51:24.373 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:08.975487+0000", 2024-04-07T17:51:24.373 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.369806+0000", 2024-04-07T17:51:24.373 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:24.373 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:24.373 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:24.373 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 554, 2024-04-07T17:51:24.373 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.369806+0000", 2024-04-07T17:51:24.373 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.369806+0000", 2024-04-07T17:51:24.373 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.369806+0000", 2024-04-07T17:51:24.373 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:24.373 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:24.373 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:24.374 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.369806+0000", 2024-04-07T17:51:24.374 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.369806+0000", 2024-04-07T17:51:24.374 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:24.374 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 2, 2024-04-07T17:51:24.374 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:24.374 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:24.374 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 553, 2024-04-07T17:51:24.374 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:24.374 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:24.374 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:24.374 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:24.374 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 2, 2024-04-07T17:51:24.374 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:24.374 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:24.375 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:24.375 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.7", 2024-04-07T17:51:24.375 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:24.375 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:24.375 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:24.375 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4865, 2024-04-07T17:51:24.375 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:24.375 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T01:37:46.529471+0000", 2024-04-07T17:51:24.375 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:24.375 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:24.375 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:24.375 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:24.375 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:24.375 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:24.375 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:24.376 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:24.376 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:24.376 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:24.376 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:24.376 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:24.376 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:24.376 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:24.376 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:24.376 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:24.376 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:24.376 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:24.376 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:24.376 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 3, 2024-04-07T17:51:24.376 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 1, 2024-04-07T17:51:24.376 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:24.377 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 1, 2024-04-07T17:51:24.377 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:24.377 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:24.377 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:24.377 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:24.377 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:24.377 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 1, 2024-04-07T17:51:24.377 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:24.377 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:24.377 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:24.377 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:24.377 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:24.377 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:24.377 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:24.377 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 6, 2024-04-07T17:51:24.377 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 6, 2024-04-07T17:51:24.378 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:24.378 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:24.378 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:24.378 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 3, 2024-04-07T17:51:24.378 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:24.378 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.378 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:24.378 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:24.378 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:24.378 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:24.378 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:24.378 INFO:tasks.workunit.client.0.smithi100.stdout: 2 2024-04-07T17:51:24.378 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.378 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 6, 2024-04-07T17:51:24.379 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "61'2" 2024-04-07T17:51:24.379 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.379 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:24.379 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:24.379 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:24.379 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:24.379 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T17:51:24.379 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.379 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 4, 2024-04-07T17:51:24.379 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:24.379 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:24.379 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 66, 2024-04-07T17:51:24.379 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:24.379 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:24.379 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:24.380 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:35.977631+0000", 2024-04-07T17:51:24.380 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:08.952433+0000", 2024-04-07T17:51:24.380 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:08.952433+0000", 2024-04-07T17:51:24.380 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:59:56.534054+0000", 2024-04-07T17:51:24.380 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:35.977631+0000", 2024-04-07T17:51:24.380 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:13:28.963534+0000", 2024-04-07T17:51:24.380 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:24.380 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:13:28.963534+0000", 2024-04-07T17:51:24.380 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 554, 2024-04-07T17:51:24.380 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:35.977631+0000", 2024-04-07T17:51:24.380 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:35.977631+0000", 2024-04-07T17:51:24.380 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:35.977631+0000", 2024-04-07T17:51:24.380 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:24.380 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:24.380 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:13:28.963534+0000", 2024-04-07T17:51:24.381 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:35.977631+0000", 2024-04-07T17:51:24.381 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:35.977631+0000", 2024-04-07T17:51:24.381 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:24.381 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 3036, 2024-04-07T17:51:24.381 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:24.381 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:24.381 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 553, 2024-04-07T17:51:24.381 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:24.381 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:24.381 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:24.381 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:24.381 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 3036, 2024-04-07T17:51:24.381 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:24.381 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:24.381 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:24.382 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "4.1", 2024-04-07T17:51:24.382 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:24.382 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:24.382 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:24.382 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 8679, 2024-04-07T17:51:24.382 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:24.382 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T00:59:44.742269+0000", 2024-04-07T17:51:24.382 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.007050569, 2024-04-07T17:51:24.382 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:24.382 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:24.382 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 4194304, 2024-04-07T17:51:24.382 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:24.382 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:24.382 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:24.383 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:24.383 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:24.383 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:24.383 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:24.383 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:24.383 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:24.383 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:24.383 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:24.383 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:24.383 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:24.383 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:24.383 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:24.383 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 3, 2024-04-07T17:51:24.383 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 1, 2024-04-07T17:51:24.383 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:24.383 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 1, 2024-04-07T17:51:24.384 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:24.384 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:24.384 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:24.384 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:24.384 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:24.384 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:24.384 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:24.384 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:24.384 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:24.384 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:24.384 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:24.384 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:24.384 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:24.384 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 1087, 2024-04-07T17:51:24.384 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 402424, 2024-04-07T17:51:24.384 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:24.385 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:24.385 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:24.385 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 3017, 2024-04-07T17:51:24.385 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 407157 2024-04-07T17:51:24.385 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.385 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:24.385 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:24.385 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:24.385 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:24.385 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:24.385 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T17:51:24.385 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.385 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 4, 2024-04-07T17:51:24.385 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "1083'3036" 2024-04-07T17:51:24.385 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.386 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:24.386 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:24.386 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:24.386 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:24.386 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T17:51:24.386 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.386 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 3, 2024-04-07T17:51:24.386 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:24.386 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:24.386 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:24.386 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:24.386 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:24.386 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:24.386 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:35.981722+0000", 2024-04-07T17:51:24.387 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:38.974388+0000", 2024-04-07T17:51:24.387 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:38.974388+0000", 2024-04-07T17:51:24.387 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:38.974763+0000", 2024-04-07T17:51:24.387 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:35.981722+0000", 2024-04-07T17:51:24.387 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.387 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:24.387 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.387 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:24.387 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:35.981722+0000", 2024-04-07T17:51:24.387 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:35.981722+0000", 2024-04-07T17:51:24.387 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:35.981722+0000", 2024-04-07T17:51:24.387 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:24.387 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:24.387 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.387 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:35.981722+0000", 2024-04-07T17:51:24.388 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:35.981722+0000", 2024-04-07T17:51:24.388 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:24.388 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:24.388 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:24.388 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:24.388 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:24.388 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:24.388 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:24.388 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:24.388 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:24.388 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:24.388 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:24.388 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:24.388 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:24.388 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.7", 2024-04-07T17:51:24.388 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:24.389 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:24.389 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:24.389 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4940, 2024-04-07T17:51:24.389 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:24.389 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T03:43:59.412072+0000", 2024-04-07T17:51:24.389 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:24.389 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:24.389 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:24.389 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:24.389 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:24.389 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:24.389 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:24.389 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:24.389 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:24.389 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:24.390 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:24.390 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:24.390 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:24.390 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:24.390 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:24.390 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:24.390 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:24.390 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:24.390 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:24.390 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:24.390 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:24.390 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:24.390 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:24.390 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:24.390 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:24.391 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:24.391 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:24.391 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:24.391 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:24.391 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:24.391 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:24.391 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:24.391 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:24.391 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:24.391 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:24.391 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:24.391 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:24.391 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:24.391 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:24.391 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:24.392 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:24.392 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:24.392 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:24.392 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.392 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:24.392 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:24.392 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:24.392 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:24.392 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:24.392 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T17:51:24.392 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.392 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 3, 2024-04-07T17:51:24.392 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:24.392 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.392 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:24.392 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:24.393 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:24.393 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:24.393 INFO:tasks.workunit.client.0.smithi100.stdout: 4 2024-04-07T17:51:24.393 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.393 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 1, 2024-04-07T17:51:24.393 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:24.393 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:24.393 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T17:51:24.393 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:24.393 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:24.393 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:24.393 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.335258+0000", 2024-04-07T17:51:24.393 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:08.962773+0000", 2024-04-07T17:51:24.393 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:08.962773+0000", 2024-04-07T17:51:24.393 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:08.963320+0000", 2024-04-07T17:51:24.393 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.335258+0000", 2024-04-07T17:51:24.394 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:24.394 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:24.394 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:24.394 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 554, 2024-04-07T17:51:24.394 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.335258+0000", 2024-04-07T17:51:24.394 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.335258+0000", 2024-04-07T17:51:24.394 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.335258+0000", 2024-04-07T17:51:24.394 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:24.394 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:24.394 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:24.394 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.335258+0000", 2024-04-07T17:51:24.394 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.335258+0000", 2024-04-07T17:51:24.394 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:24.394 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 2, 2024-04-07T17:51:24.395 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:24.395 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:24.395 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 553, 2024-04-07T17:51:24.395 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:24.395 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:24.395 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:24.395 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:24.395 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 2, 2024-04-07T17:51:24.395 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:24.395 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:24.395 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:24.395 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.6", 2024-04-07T17:51:24.395 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:24.395 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:24.395 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:24.396 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4986, 2024-04-07T17:51:24.396 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:24.396 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T18:43:13.545482+0000", 2024-04-07T17:51:24.396 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:24.396 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:24.396 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:24.396 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:24.396 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:24.396 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:24.396 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:24.396 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:24.396 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:24.396 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:24.396 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:24.396 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:24.397 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:24.397 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:24.397 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:24.397 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:24.397 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:24.397 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:24.397 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:24.397 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 3, 2024-04-07T17:51:24.397 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 1, 2024-04-07T17:51:24.397 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:24.397 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 1, 2024-04-07T17:51:24.397 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:24.397 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:24.397 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:24.397 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:24.398 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:24.398 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 1, 2024-04-07T17:51:24.398 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:24.398 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:24.398 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:24.398 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:24.398 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:24.398 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:24.398 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:24.398 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 2, 2024-04-07T17:51:24.398 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 2, 2024-04-07T17:51:24.398 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:24.398 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:24.398 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:24.398 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 2, 2024-04-07T17:51:24.399 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:24.399 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.399 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:24.399 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:24.399 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:24.399 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:24.399 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:24.399 INFO:tasks.workunit.client.0.smithi100.stdout: 4 2024-04-07T17:51:24.399 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.399 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 1, 2024-04-07T17:51:24.399 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "627'2" 2024-04-07T17:51:24.399 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.399 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:24.399 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:24.399 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:24.400 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:24.400 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T17:51:24.400 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.400 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 3, 2024-04-07T17:51:24.400 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:24.400 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:24.400 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 66, 2024-04-07T17:51:24.400 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:24.400 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:24.400 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:24.400 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:35.975295+0000", 2024-04-07T17:51:24.400 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:38.997956+0000", 2024-04-07T17:51:24.400 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:38.997956+0000", 2024-04-07T17:51:24.400 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:59:56.537779+0000", 2024-04-07T17:51:24.400 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:35.975295+0000", 2024-04-07T17:51:24.400 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:13:28.963534+0000", 2024-04-07T17:51:24.401 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:24.401 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:13:28.963534+0000", 2024-04-07T17:51:24.401 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:24.401 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:35.975295+0000", 2024-04-07T17:51:24.401 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:35.975295+0000", 2024-04-07T17:51:24.401 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:35.975295+0000", 2024-04-07T17:51:24.401 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:24.401 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:24.401 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:13:28.963534+0000", 2024-04-07T17:51:24.401 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:35.975295+0000", 2024-04-07T17:51:24.401 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:35.975295+0000", 2024-04-07T17:51:24.401 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:24.401 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 3089, 2024-04-07T17:51:24.401 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:24.401 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:24.401 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:24.402 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:24.402 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:24.402 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:24.402 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:24.402 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 3089, 2024-04-07T17:51:24.402 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:24.402 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:24.402 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:24.402 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "4.0", 2024-04-07T17:51:24.402 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:24.402 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:24.402 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:24.402 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 8748, 2024-04-07T17:51:24.402 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:24.402 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T16:44:21.465234+0000", 2024-04-07T17:51:24.403 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.013008564, 2024-04-07T17:51:24.403 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:24.403 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:24.403 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:24.403 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:24.403 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:24.403 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:24.403 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:24.403 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:24.403 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:24.403 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:24.403 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:24.403 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:24.403 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:24.403 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:24.403 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:24.404 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:24.404 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:24.404 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:24.404 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:24.404 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:24.404 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:24.404 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:24.404 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:24.404 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:24.404 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:24.404 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:24.404 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:24.404 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:24.404 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:24.404 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:24.404 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:24.404 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:24.405 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:24.405 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:24.405 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:24.405 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 1133, 2024-04-07T17:51:24.405 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 401354, 2024-04-07T17:51:24.405 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:24.405 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:24.405 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:24.405 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 3064, 2024-04-07T17:51:24.405 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 366365 2024-04-07T17:51:24.405 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.405 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:24.405 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:24.405 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:24.405 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:24.406 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:24.406 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T17:51:24.406 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.406 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 3, 2024-04-07T17:51:24.406 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "1083'3089" 2024-04-07T17:51:24.406 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.406 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:24.406 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:24.406 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:24.406 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T17:51:24.406 INFO:tasks.workunit.client.0.smithi100.stdout: 5 2024-04-07T17:51:24.406 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.406 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 1, 2024-04-07T17:51:24.406 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:24.407 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:24.407 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:24.407 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:24.407 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:24.407 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:24.407 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.333938+0000", 2024-04-07T17:51:24.407 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:24:41.552588+0000", 2024-04-07T17:51:24.407 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:24:41.552588+0000", 2024-04-07T17:51:24.407 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:24:41.553129+0000", 2024-04-07T17:51:24.407 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.333938+0000", 2024-04-07T17:51:24.407 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.407 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:24.407 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.407 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 527, 2024-04-07T17:51:24.407 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.333938+0000", 2024-04-07T17:51:24.408 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.333938+0000", 2024-04-07T17:51:24.408 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.333938+0000", 2024-04-07T17:51:24.408 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:24.408 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:24.408 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.408 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.333938+0000", 2024-04-07T17:51:24.408 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.333938+0000", 2024-04-07T17:51:24.408 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:24.408 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:24.408 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:24.408 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:24.408 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 526, 2024-04-07T17:51:24.408 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:24.408 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:24.409 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:24.409 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:24.409 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:24.409 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:24.409 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:24.409 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:24.409 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.4", 2024-04-07T17:51:24.409 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:24.409 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:24.409 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:24.409 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4983, 2024-04-07T17:51:24.409 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:24.409 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T18:17:15.023877+0000", 2024-04-07T17:51:24.409 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:24.409 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:24.409 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:24.410 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:24.410 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:24.410 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:24.410 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:24.410 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:24.410 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:24.410 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:24.410 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:24.410 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:24.410 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:24.410 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:24.410 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:24.410 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:24.411 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:24.411 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:24.411 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:24.411 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:24.411 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:24.411 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:24.411 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:24.411 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:24.411 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:24.411 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:24.411 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:24.411 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:24.411 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:24.411 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:24.411 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:24.411 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:24.412 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:24.412 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:24.412 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:24.412 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:24.412 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:24.412 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:24.412 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:24.412 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:24.412 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:24.412 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:24.412 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:24.412 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.412 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:24.412 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:24.412 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:24.413 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:24.413 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T17:51:24.413 INFO:tasks.workunit.client.0.smithi100.stdout: 5 2024-04-07T17:51:24.413 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.413 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 1, 2024-04-07T17:51:24.413 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:24.413 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.413 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:24.413 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:24.413 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:24.413 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:24.413 INFO:tasks.workunit.client.0.smithi100.stdout: 4 2024-04-07T17:51:24.413 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.414 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 7, 2024-04-07T17:51:24.414 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:24.414 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:24.414 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T17:51:24.414 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:24.414 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:24.414 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:24.414 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.372971+0000", 2024-04-07T17:51:24.414 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:39.001835+0000", 2024-04-07T17:51:24.414 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:39.001835+0000", 2024-04-07T17:51:24.414 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:39.003607+0000", 2024-04-07T17:51:24.414 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.372971+0000", 2024-04-07T17:51:24.414 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:24.414 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:24.415 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:24.415 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:24.415 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.372971+0000", 2024-04-07T17:51:24.415 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.372971+0000", 2024-04-07T17:51:24.415 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.372971+0000", 2024-04-07T17:51:24.415 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:24.415 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:24.415 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:24.415 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.372971+0000", 2024-04-07T17:51:24.415 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.372971+0000", 2024-04-07T17:51:24.415 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:24.415 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:24.415 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:24.415 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:24.415 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:24.416 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:24.416 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:24.416 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:24.416 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:24.416 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:24.416 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:24.416 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:24.416 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:24.416 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.5", 2024-04-07T17:51:24.416 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:24.416 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:24.416 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:24.416 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4795, 2024-04-07T17:51:24.416 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:24.416 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T21:54:36.255484+0000", 2024-04-07T17:51:24.417 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:24.417 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:24.417 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:24.417 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:24.417 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:24.417 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:24.417 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:24.417 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:24.417 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:24.417 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:24.417 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:24.417 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:24.417 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:24.417 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:24.417 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:24.417 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:24.418 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:24.418 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:24.418 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:24.418 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:24.418 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:24.418 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:24.418 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:24.418 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:24.418 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:24.418 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:24.418 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:24.418 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:24.418 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:24.419 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:24.419 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:24.419 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:24.419 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:24.419 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:24.419 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:24.419 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:24.419 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:24.419 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:24.419 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:24.419 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:24.419 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:24.419 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:24.419 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:24.419 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.420 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:24.420 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:24.420 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:24.420 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:24.420 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:24.420 INFO:tasks.workunit.client.0.smithi100.stdout: 4 2024-04-07T17:51:24.420 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.420 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 7, 2024-04-07T17:51:24.420 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:24.420 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.420 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:24.420 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:24.420 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:24.420 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:24.420 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T17:51:24.420 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.421 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 0, 2024-04-07T17:51:24.421 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:24.421 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:24.421 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 66, 2024-04-07T17:51:24.421 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:24.421 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:24.421 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:24.421 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:35.988907+0000", 2024-04-07T17:51:24.421 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:39.003851+0000", 2024-04-07T17:51:24.421 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:39.003851+0000", 2024-04-07T17:51:24.421 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:59:56.571405+0000", 2024-04-07T17:51:24.421 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:35.988907+0000", 2024-04-07T17:51:24.421 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:13:28.963534+0000", 2024-04-07T17:51:24.421 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:24.421 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:13:28.963534+0000", 2024-04-07T17:51:24.422 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:24.422 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:35.988907+0000", 2024-04-07T17:51:24.422 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:35.988907+0000", 2024-04-07T17:51:24.422 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:35.988907+0000", 2024-04-07T17:51:24.422 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:24.422 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:24.422 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:13:28.963534+0000", 2024-04-07T17:51:24.422 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:35.988907+0000", 2024-04-07T17:51:24.422 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:35.988907+0000", 2024-04-07T17:51:24.422 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:24.422 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 3135, 2024-04-07T17:51:24.422 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:24.422 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:24.423 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:24.423 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:24.423 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:24.423 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:24.423 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:24.423 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 3135, 2024-04-07T17:51:24.423 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:24.423 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:24.423 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:24.423 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "4.3", 2024-04-07T17:51:24.423 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:24.423 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:24.423 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:24.423 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 8776, 2024-04-07T17:51:24.423 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:24.423 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T23:06:05.133511+0000", 2024-04-07T17:51:24.424 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.019028691, 2024-04-07T17:51:24.424 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:24.424 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:24.424 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:24.424 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:24.424 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:24.424 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:24.424 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:24.424 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:24.424 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:24.424 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:24.424 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:24.424 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:24.424 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:24.424 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:24.425 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:24.425 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:24.425 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:24.425 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:24.425 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 3, 2024-04-07T17:51:24.425 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 1, 2024-04-07T17:51:24.425 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:24.425 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 1, 2024-04-07T17:51:24.425 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:24.425 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:24.425 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:24.425 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:24.425 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:24.425 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 1, 2024-04-07T17:51:24.425 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:24.425 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:24.426 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:24.426 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:24.426 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:24.426 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:24.426 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:24.426 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 871, 2024-04-07T17:51:24.426 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 391391, 2024-04-07T17:51:24.426 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:24.426 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:24.426 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:24.426 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 3127, 2024-04-07T17:51:24.426 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 378194 2024-04-07T17:51:24.426 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.426 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:24.427 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:24.427 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:24.427 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:24.427 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:24.427 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T17:51:24.427 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.427 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 0, 2024-04-07T17:51:24.427 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "1084'3135" 2024-04-07T17:51:24.427 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.427 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:24.427 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:24.427 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:24.427 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:24.427 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T17:51:24.427 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.428 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 7, 2024-04-07T17:51:24.428 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:24.428 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:24.428 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T17:51:24.428 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:24.428 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:24.428 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:24.428 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.364517+0000", 2024-04-07T17:51:24.428 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:39.000134+0000", 2024-04-07T17:51:24.428 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:39.000134+0000", 2024-04-07T17:51:24.428 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:39.003944+0000", 2024-04-07T17:51:24.428 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.364517+0000", 2024-04-07T17:51:24.428 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:24.428 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:24.428 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:24.429 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:24.429 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.364517+0000", 2024-04-07T17:51:24.429 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.364517+0000", 2024-04-07T17:51:24.429 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.364517+0000", 2024-04-07T17:51:24.429 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:24.429 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:24.429 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:24.429 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.364517+0000", 2024-04-07T17:51:24.429 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.364517+0000", 2024-04-07T17:51:24.429 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:24.429 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:24.429 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:24.429 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:24.429 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:24.429 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:24.430 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:24.430 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:24.430 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:24.430 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:24.430 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:24.430 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:24.430 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:24.430 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.0", 2024-04-07T17:51:24.430 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:24.430 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:24.430 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:24.430 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4796, 2024-04-07T17:51:24.430 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:24.430 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T22:28:12.376172+0000", 2024-04-07T17:51:24.431 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:24.431 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:24.431 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:24.431 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:24.431 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:24.431 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:24.431 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:24.431 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:24.431 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:24.431 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:24.431 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:24.431 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:24.431 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:24.431 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:24.431 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:24.432 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:24.432 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:24.432 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:24.432 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:24.432 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:24.432 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:24.432 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:24.432 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:24.432 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:24.432 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:24.432 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:24.432 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:24.432 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:24.432 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:24.432 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:24.432 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:24.433 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:24.433 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:24.433 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:24.433 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:24.433 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:24.433 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:24.433 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:24.433 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:24.433 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:24.433 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:24.433 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:24.433 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:24.433 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.433 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:24.433 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:24.434 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:24.434 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:24.434 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:24.434 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T17:51:24.434 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.434 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 7, 2024-04-07T17:51:24.434 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:24.434 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.434 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:24.434 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:24.434 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T17:51:24.434 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:24.434 INFO:tasks.workunit.client.0.smithi100.stdout: 3 2024-04-07T17:51:24.434 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.434 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 2, 2024-04-07T17:51:24.435 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:24.435 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:24.435 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:24.435 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:24.435 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:24.435 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:24.435 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.354824+0000", 2024-04-07T17:51:24.435 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:24:13.930517+0000", 2024-04-07T17:51:24.435 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:24:13.930517+0000", 2024-04-07T17:51:24.435 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:24:13.931259+0000", 2024-04-07T17:51:24.435 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.354824+0000", 2024-04-07T17:51:24.435 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.435 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:24.435 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.435 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 500, 2024-04-07T17:51:24.436 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.354824+0000", 2024-04-07T17:51:24.436 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.354824+0000", 2024-04-07T17:51:24.436 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.354824+0000", 2024-04-07T17:51:24.436 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:24.436 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:24.436 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.436 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.354824+0000", 2024-04-07T17:51:24.436 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.354824+0000", 2024-04-07T17:51:24.436 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:24.436 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:24.436 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:24.436 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:24.436 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 499, 2024-04-07T17:51:24.436 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:24.437 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:24.437 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:24.437 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:24.437 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:24.437 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:24.437 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:24.437 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:24.437 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.1", 2024-04-07T17:51:24.437 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:24.437 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:24.437 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:24.437 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4929, 2024-04-07T17:51:24.437 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:24.437 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T01:43:11.885075+0000", 2024-04-07T17:51:24.437 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:24.438 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:24.438 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:24.438 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:24.438 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:24.438 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:24.438 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:24.438 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:24.438 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:24.438 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:24.438 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:24.438 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:24.438 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:24.438 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:24.439 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:24.439 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:24.439 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:24.439 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:24.439 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:24.439 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:24.439 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:24.439 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:24.439 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:24.439 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:24.439 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:24.439 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:24.439 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:24.439 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:24.439 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:24.440 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:24.440 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:24.440 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:24.440 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:24.440 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:24.440 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:24.440 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:24.440 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:24.440 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:24.440 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:24.440 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:24.440 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:24.440 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:24.440 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:24.441 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.441 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:24.441 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:24.441 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:24.441 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T17:51:24.441 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:24.441 INFO:tasks.workunit.client.0.smithi100.stdout: 3 2024-04-07T17:51:24.441 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.441 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 2, 2024-04-07T17:51:24.441 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:24.441 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.441 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:24.441 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:24.441 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:24.441 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:24.442 INFO:tasks.workunit.client.0.smithi100.stdout: 2 2024-04-07T17:51:24.442 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.442 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 0, 2024-04-07T17:51:24.442 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:24.442 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:24.442 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 66, 2024-04-07T17:51:24.442 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:24.442 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:24.442 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:24.442 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:35.982117+0000", 2024-04-07T17:51:24.442 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:23:15.882881+0000", 2024-04-07T17:51:24.442 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:23:15.882881+0000", 2024-04-07T17:51:24.442 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:59:56.569536+0000", 2024-04-07T17:51:24.442 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:35.982117+0000", 2024-04-07T17:51:24.443 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:13:28.963534+0000", 2024-04-07T17:51:24.443 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:24.443 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:13:28.963534+0000", 2024-04-07T17:51:24.443 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 445, 2024-04-07T17:51:24.443 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:35.982117+0000", 2024-04-07T17:51:24.443 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:35.982117+0000", 2024-04-07T17:51:24.443 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:35.982117+0000", 2024-04-07T17:51:24.443 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:24.443 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:24.443 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:13:28.963534+0000", 2024-04-07T17:51:24.443 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:35.982117+0000", 2024-04-07T17:51:24.443 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:35.982117+0000", 2024-04-07T17:51:24.443 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:24.443 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 3107, 2024-04-07T17:51:24.443 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:24.444 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:24.444 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 444, 2024-04-07T17:51:24.444 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:24.444 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:24.444 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:24.444 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:24.444 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 3107, 2024-04-07T17:51:24.444 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:24.444 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:24.444 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:24.444 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "4.6", 2024-04-07T17:51:24.444 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:24.444 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:24.444 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:24.445 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 8717, 2024-04-07T17:51:24.445 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:24.445 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T18:35:28.250440+0000", 2024-04-07T17:51:24.445 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.017233416, 2024-04-07T17:51:24.445 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:24.445 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:24.445 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:24.445 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:24.445 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:24.445 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:24.445 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:24.445 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:24.445 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:24.445 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:24.445 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:24.446 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:24.446 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:24.446 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:24.446 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:24.446 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:24.446 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:24.446 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:24.446 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:24.446 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:24.446 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:24.446 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:24.446 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:24.446 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:24.447 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:24.447 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:24.447 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:24.447 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:24.447 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:24.447 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:24.447 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:24.447 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:24.447 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:24.447 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:24.447 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:24.447 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 858, 2024-04-07T17:51:24.447 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 396543, 2024-04-07T17:51:24.447 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:24.447 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:24.448 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:24.448 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 3070, 2024-04-07T17:51:24.448 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 377559 2024-04-07T17:51:24.448 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.448 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:24.448 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:24.448 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:24.448 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:24.448 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:24.448 INFO:tasks.workunit.client.0.smithi100.stdout: 2 2024-04-07T17:51:24.448 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.448 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 0, 2024-04-07T17:51:24.448 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "1084'3107" 2024-04-07T17:51:24.448 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.448 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:24.449 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:24.449 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:24.449 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:24.449 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T17:51:24.449 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.449 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 7, 2024-04-07T17:51:24.449 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:24.449 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:24.449 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 19, 2024-04-07T17:51:24.449 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:24.449 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:24.449 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:24.449 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.366640+0000", 2024-04-07T17:51:24.449 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:39.000587+0000", 2024-04-07T17:51:24.449 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:39.000587+0000", 2024-04-07T17:51:24.450 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:39.004716+0000", 2024-04-07T17:51:24.450 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.366640+0000", 2024-04-07T17:51:24.450 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:07:26.866003+0000", 2024-04-07T17:51:24.450 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:24.450 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:07:26.866003+0000", 2024-04-07T17:51:24.450 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:24.450 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.366640+0000", 2024-04-07T17:51:24.450 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.366640+0000", 2024-04-07T17:51:24.450 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.366640+0000", 2024-04-07T17:51:24.450 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:24.450 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:24.450 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:07:26.866003+0000", 2024-04-07T17:51:24.450 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.366640+0000", 2024-04-07T17:51:24.451 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.366640+0000", 2024-04-07T17:51:24.451 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:24.451 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 241, 2024-04-07T17:51:24.451 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:24.451 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:24.451 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:24.451 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:24.451 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:24.451 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:24.451 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:24.451 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 241, 2024-04-07T17:51:24.451 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:24.451 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:24.451 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:24.451 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "1.0", 2024-04-07T17:51:24.452 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:24.452 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:24.452 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:24.452 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 5460, 2024-04-07T17:51:24.452 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:24.452 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T21:59:54.730545+0000", 2024-04-07T17:51:24.452 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:24.452 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:24.452 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:24.452 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 590368, 2024-04-07T17:51:24.452 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:24.452 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 1771104, 2024-04-07T17:51:24.452 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:24.452 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:24.453 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:24.453 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:24.453 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:24.453 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:24.453 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:24.453 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:24.453 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:24.453 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:24.453 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:24.453 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:24.453 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:24.453 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 6, 2024-04-07T17:51:24.453 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 2, 2024-04-07T17:51:24.453 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:24.453 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 2, 2024-04-07T17:51:24.454 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:24.454 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:24.454 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:24.454 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:24.454 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:24.454 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:24.454 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:24.454 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 6, 2024-04-07T17:51:24.454 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:24.454 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:24.454 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:24.454 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:24.454 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:24.455 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 1214, 2024-04-07T17:51:24.455 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 2024, 2024-04-07T17:51:24.455 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:24.455 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:24.455 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:24.455 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 399, 2024-04-07T17:51:24.455 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 2416 2024-04-07T17:51:24.455 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.455 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:24.455 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:24.455 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:24.455 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:24.455 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:24.455 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T17:51:24.455 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.456 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 7, 2024-04-07T17:51:24.456 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "2111'241" 2024-04-07T17:51:24.456 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.456 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:24.456 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:24.456 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:24.456 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T17:51:24.456 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T17:51:24.456 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.456 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 5, 2024-04-07T17:51:24.456 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:24.456 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:24.456 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T17:51:24.456 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:24.456 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:24.457 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:24.457 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.370078+0000", 2024-04-07T17:51:24.457 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:38.985921+0000", 2024-04-07T17:51:24.457 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:38.985921+0000", 2024-04-07T17:51:24.457 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:38.986647+0000", 2024-04-07T17:51:24.457 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.370078+0000", 2024-04-07T17:51:24.457 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:24.457 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:24.457 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:24.457 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:24.457 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.370078+0000", 2024-04-07T17:51:24.457 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.370078+0000", 2024-04-07T17:51:24.457 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.370078+0000", 2024-04-07T17:51:24.457 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:24.457 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:24.458 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:24.458 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.370078+0000", 2024-04-07T17:51:24.458 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.370078+0000", 2024-04-07T17:51:24.458 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:24.458 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:24.458 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:24.458 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:24.458 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:24.458 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:24.458 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:24.458 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:24.458 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:24.458 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:24.458 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:24.458 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:24.459 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:24.459 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.3", 2024-04-07T17:51:24.459 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:24.459 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:24.459 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:24.459 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4855, 2024-04-07T17:51:24.459 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:24.459 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T02:44:42.906581+0000", 2024-04-07T17:51:24.459 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:24.459 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:24.459 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:24.459 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:24.459 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:24.459 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:24.459 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:24.460 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:24.460 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:24.460 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:24.460 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:24.460 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:24.460 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:24.460 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:24.460 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:24.460 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:24.460 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:24.460 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:24.460 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:24.460 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:24.460 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:24.460 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:24.461 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:24.461 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:24.461 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:24.461 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:24.461 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:24.461 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:24.461 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:24.461 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:24.461 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:24.461 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:24.461 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:24.461 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:24.461 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:24.461 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:24.461 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:24.462 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:24.462 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:24.462 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:24.462 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:24.462 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:24.462 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:24.462 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.462 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:24.462 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:24.462 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:24.462 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:24.462 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T17:51:24.462 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T17:51:24.463 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.463 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 5, 2024-04-07T17:51:24.463 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:24.463 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.463 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:24.463 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:24.463 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:24.463 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:24.463 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T17:51:24.463 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.463 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 3, 2024-04-07T17:51:24.463 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:24.463 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:24.463 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:24.463 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:24.464 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:24.464 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:24.464 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:35.982147+0000", 2024-04-07T17:51:24.464 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:08.977825+0000", 2024-04-07T17:51:24.464 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:08.977825+0000", 2024-04-07T17:51:24.464 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:08.978231+0000", 2024-04-07T17:51:24.464 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:35.982147+0000", 2024-04-07T17:51:24.464 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.464 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:24.464 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.464 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 554, 2024-04-07T17:51:24.464 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:35.982147+0000", 2024-04-07T17:51:24.464 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:35.982147+0000", 2024-04-07T17:51:24.464 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:35.982147+0000", 2024-04-07T17:51:24.465 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:24.465 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:24.465 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.465 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:35.982147+0000", 2024-04-07T17:51:24.465 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:35.982147+0000", 2024-04-07T17:51:24.465 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:24.465 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:24.465 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:24.465 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:24.465 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 553, 2024-04-07T17:51:24.465 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:24.465 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:24.465 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:24.466 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:24.466 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:24.466 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:24.466 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:24.466 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:24.466 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.2", 2024-04-07T17:51:24.466 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:24.466 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:24.466 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:24.466 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4937, 2024-04-07T17:51:24.466 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:24.466 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T01:34:08.840626+0000", 2024-04-07T17:51:24.466 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:24.466 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:24.467 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:24.467 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:24.467 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:24.467 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:24.467 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:24.467 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:24.467 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:24.467 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:24.467 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:24.467 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:24.467 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:24.467 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:24.467 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:24.467 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:24.467 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:24.468 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:24.468 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:24.468 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:24.468 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:24.468 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:24.468 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:24.468 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:24.468 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:24.468 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:24.468 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:24.468 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:24.468 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:24.468 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:24.468 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:24.468 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:24.468 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:24.469 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:24.469 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:24.469 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:24.469 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:24.469 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:24.469 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:24.469 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:24.469 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:24.469 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:24.469 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:24.469 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.469 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:24.469 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:24.469 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:24.469 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:24.469 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:24.470 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T17:51:24.470 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.470 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 3, 2024-04-07T17:51:24.470 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:24.470 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.470 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:24.470 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:24.470 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:24.470 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:24.470 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T17:51:24.470 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.470 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 6, 2024-04-07T17:51:24.470 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:24.470 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:24.471 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 66, 2024-04-07T17:51:24.471 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:24.471 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:24.471 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:24.471 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.360030+0000", 2024-04-07T17:51:24.471 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:39.000185+0000", 2024-04-07T17:51:24.471 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:39.000185+0000", 2024-04-07T17:51:24.471 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:59:56.544598+0000", 2024-04-07T17:51:24.471 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.360030+0000", 2024-04-07T17:51:24.471 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:13:28.963534+0000", 2024-04-07T17:51:24.471 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:24.471 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:13:28.963534+0000", 2024-04-07T17:51:24.471 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:24.471 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.360030+0000", 2024-04-07T17:51:24.471 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.360030+0000", 2024-04-07T17:51:24.472 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.360030+0000", 2024-04-07T17:51:24.472 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:24.472 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:24.472 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:13:28.963534+0000", 2024-04-07T17:51:24.472 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.360030+0000", 2024-04-07T17:51:24.472 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.360030+0000", 2024-04-07T17:51:24.472 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:24.472 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 3124, 2024-04-07T17:51:24.472 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:24.472 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:24.472 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:24.472 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:24.472 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:24.472 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:24.472 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:24.472 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 3124, 2024-04-07T17:51:24.473 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:24.473 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:24.473 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:24.473 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "4.5", 2024-04-07T17:51:24.473 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:24.473 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:24.473 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:24.473 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 8600, 2024-04-07T17:51:24.473 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:24.473 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T01:53:54.315291+0000", 2024-04-07T17:51:24.473 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.008144472, 2024-04-07T17:51:24.473 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:24.473 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:24.473 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:24.473 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:24.473 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:24.474 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:24.474 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:24.474 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:24.474 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:24.474 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:24.474 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:24.474 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:24.474 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:24.474 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:24.474 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:24.474 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:24.474 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:24.474 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:24.474 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:24.474 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:24.475 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:24.475 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:24.475 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:24.475 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:24.475 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:24.475 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:24.475 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:24.475 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:24.475 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:24.475 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:24.475 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:24.475 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:24.475 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:24.475 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:24.475 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:24.475 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 979, 2024-04-07T17:51:24.476 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 424424, 2024-04-07T17:51:24.476 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:24.476 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:24.476 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:24.476 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 3125, 2024-04-07T17:51:24.476 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 420851 2024-04-07T17:51:24.476 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.476 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:24.476 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:24.476 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:24.476 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:24.476 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:24.476 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T17:51:24.476 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.476 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 6, 2024-04-07T17:51:24.477 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "1083'3124" 2024-04-07T17:51:24.477 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.477 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:24.477 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:24.477 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T17:51:24.477 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:24.477 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T17:51:24.477 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.477 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 2, 2024-04-07T17:51:24.477 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:24.477 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:24.477 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T17:51:24.477 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:24.477 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:24.477 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:24.477 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.355116+0000", 2024-04-07T17:51:24.478 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:23:43.929900+0000", 2024-04-07T17:51:24.478 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:23:43.929900+0000", 2024-04-07T17:51:24.478 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:23:43.930960+0000", 2024-04-07T17:51:24.478 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.355116+0000", 2024-04-07T17:51:24.478 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:24.478 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:24.478 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:24.478 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 471, 2024-04-07T17:51:24.478 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.355116+0000", 2024-04-07T17:51:24.478 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.355116+0000", 2024-04-07T17:51:24.478 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.355116+0000", 2024-04-07T17:51:24.478 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:24.478 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:24.478 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:24.479 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.355116+0000", 2024-04-07T17:51:24.479 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.355116+0000", 2024-04-07T17:51:24.479 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:24.479 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:24.479 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:24.479 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:24.479 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 470, 2024-04-07T17:51:24.479 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:24.479 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:24.479 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:24.479 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:24.479 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:24.479 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:24.479 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:24.479 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:24.480 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.1", 2024-04-07T17:51:24.480 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:24.480 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:24.480 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:24.480 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4977, 2024-04-07T17:51:24.480 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:24.480 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T20:05:04.268255+0000", 2024-04-07T17:51:24.480 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:24.480 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:24.480 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:24.480 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:24.480 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:24.480 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:24.480 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:24.480 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:24.481 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:24.481 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:24.481 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:24.481 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:24.481 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:24.481 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:24.481 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:24.481 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:24.481 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:24.481 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:24.481 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:24.481 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:24.481 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:24.481 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:24.481 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:24.481 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:24.482 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:24.482 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:24.482 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:24.482 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:24.482 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:24.482 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:24.482 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:24.482 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:24.482 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:24.482 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:24.482 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:24.482 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:24.482 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:24.482 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:24.483 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:24.483 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:24.483 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:24.483 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:24.483 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:24.483 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.483 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:24.483 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:24.483 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:24.483 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T17:51:24.483 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:24.483 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T17:51:24.483 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.483 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 2, 2024-04-07T17:51:24.483 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:24.483 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.484 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:24.484 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:24.484 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:24.484 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T17:51:24.484 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T17:51:24.484 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.484 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 1, 2024-04-07T17:51:24.484 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:24.484 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:24.484 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:24.484 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:24.484 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:24.484 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:24.484 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.331050+0000", 2024-04-07T17:51:24.484 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:08.961818+0000", 2024-04-07T17:51:24.485 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:08.961818+0000", 2024-04-07T17:51:24.485 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:08.962269+0000", 2024-04-07T17:51:24.485 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.331050+0000", 2024-04-07T17:51:24.485 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.485 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:24.485 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.485 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 554, 2024-04-07T17:51:24.485 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.331050+0000", 2024-04-07T17:51:24.485 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.331050+0000", 2024-04-07T17:51:24.485 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.331050+0000", 2024-04-07T17:51:24.485 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:24.485 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:24.485 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.485 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.331050+0000", 2024-04-07T17:51:24.485 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.331050+0000", 2024-04-07T17:51:24.486 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:24.486 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:24.486 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:24.486 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:24.486 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 553, 2024-04-07T17:51:24.486 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:24.486 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:24.486 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:24.486 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:24.486 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:24.486 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:24.486 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:24.486 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:24.486 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.0", 2024-04-07T17:51:24.487 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:24.487 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:24.487 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:24.487 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4985, 2024-04-07T17:51:24.487 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:24.487 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T01:33:47.318091+0000", 2024-04-07T17:51:24.487 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:24.487 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:24.487 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:24.487 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:24.487 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:24.487 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:24.487 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:24.487 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:24.487 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:24.487 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:24.488 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:24.488 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:24.488 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:24.488 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:24.488 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:24.488 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:24.488 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:24.488 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:24.488 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:24.488 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:24.488 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:24.488 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:24.488 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:24.488 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:24.488 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:24.488 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:24.489 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:24.489 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:24.489 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:24.489 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:24.489 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:24.489 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:24.489 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:24.489 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:24.489 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:24.489 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:24.489 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:24.489 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:24.489 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:24.489 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:24.489 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:24.489 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:24.490 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:24.490 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.490 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:24.490 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:24.490 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:24.490 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:24.490 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T17:51:24.490 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T17:51:24.490 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.490 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 1, 2024-04-07T17:51:24.490 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:24.490 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.490 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:24.490 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:24.491 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:24.491 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:24.491 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T17:51:24.491 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.491 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 1, 2024-04-07T17:51:24.491 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:24.491 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:24.491 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 66, 2024-04-07T17:51:24.491 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:24.491 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:24.491 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:24.491 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.332996+0000", 2024-04-07T17:51:24.491 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:24:41.570251+0000", 2024-04-07T17:51:24.491 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:24:41.570251+0000", 2024-04-07T17:51:24.491 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:59:56.540592+0000", 2024-04-07T17:51:24.491 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.332996+0000", 2024-04-07T17:51:24.492 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:13:28.963534+0000", 2024-04-07T17:51:24.492 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:24.492 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:13:28.963534+0000", 2024-04-07T17:51:24.492 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 527, 2024-04-07T17:51:24.492 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.332996+0000", 2024-04-07T17:51:24.492 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.332996+0000", 2024-04-07T17:51:24.492 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.332996+0000", 2024-04-07T17:51:24.492 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:24.492 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:24.492 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:13:28.963534+0000", 2024-04-07T17:51:24.492 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.332996+0000", 2024-04-07T17:51:24.492 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.332996+0000", 2024-04-07T17:51:24.492 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:24.493 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 2973, 2024-04-07T17:51:24.493 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:24.493 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:24.493 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 526, 2024-04-07T17:51:24.493 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:24.493 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:24.493 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:24.493 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:24.493 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 2973, 2024-04-07T17:51:24.493 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:24.493 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:24.493 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:24.493 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "4.7", 2024-04-07T17:51:24.493 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:24.493 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:24.493 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:24.494 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 8716, 2024-04-07T17:51:24.494 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:24.494 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T19:36:01.668815+0000", 2024-04-07T17:51:24.494 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.014750449, 2024-04-07T17:51:24.494 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:24.494 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:24.494 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:24.494 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:24.494 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:24.494 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:24.494 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:24.494 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:24.494 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:24.494 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:24.495 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:24.495 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:24.495 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:24.495 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:24.495 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:24.495 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:24.495 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:24.495 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:24.495 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:24.495 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:24.495 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:24.495 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:24.495 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:24.495 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:24.495 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:24.496 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:24.496 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:24.496 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:24.496 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:24.496 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:24.496 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:24.496 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:24.496 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:24.496 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:24.496 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:24.496 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 1318, 2024-04-07T17:51:24.496 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 368358, 2024-04-07T17:51:24.496 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:24.496 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:24.496 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:24.497 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 2957, 2024-04-07T17:51:24.497 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 380954 2024-04-07T17:51:24.497 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.497 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:24.497 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:24.497 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:24.497 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:24.497 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:24.497 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T17:51:24.497 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.497 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 1, 2024-04-07T17:51:24.497 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "1084'2973" 2024-04-07T17:51:24.497 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.497 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:24.497 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:24.497 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:24.498 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:24.498 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T17:51:24.498 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.498 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 4, 2024-04-07T17:51:24.498 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:24.498 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:24.498 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:24.498 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:24.498 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:24.498 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:24.498 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:35.979039+0000", 2024-04-07T17:51:24.498 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:08.952350+0000", 2024-04-07T17:51:24.498 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:08.952350+0000", 2024-04-07T17:51:24.498 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:08.952832+0000", 2024-04-07T17:51:24.498 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:35.979039+0000", 2024-04-07T17:51:24.498 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.499 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:24.499 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.499 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 554, 2024-04-07T17:51:24.499 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:35.979039+0000", 2024-04-07T17:51:24.499 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:35.979039+0000", 2024-04-07T17:51:24.499 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:35.979039+0000", 2024-04-07T17:51:24.499 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:24.499 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:24.499 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.499 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:35.979039+0000", 2024-04-07T17:51:24.499 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:35.979039+0000", 2024-04-07T17:51:24.499 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:24.499 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:24.499 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:24.499 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:24.500 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 553, 2024-04-07T17:51:24.500 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:24.500 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:24.500 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:24.500 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:24.500 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:24.500 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:24.500 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:24.500 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:24.500 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.3", 2024-04-07T17:51:24.500 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:24.500 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:24.500 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:24.500 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4919, 2024-04-07T17:51:24.500 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:24.501 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T22:57:40.507778+0000", 2024-04-07T17:51:24.501 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:24.501 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:24.501 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:24.501 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:24.501 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:24.501 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:24.501 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:24.501 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:24.501 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:24.501 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:24.501 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:24.501 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:24.501 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:24.501 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:24.501 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:24.502 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:24.502 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:24.502 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:24.502 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:24.502 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:24.502 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:24.502 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:24.502 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:24.502 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:24.502 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:24.502 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:24.502 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:24.502 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:24.502 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:24.503 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:24.503 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:24.503 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:24.503 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:24.503 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:24.503 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:24.503 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:24.503 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:24.503 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:24.503 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:24.503 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:24.503 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:24.503 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:24.503 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:24.503 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.503 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:24.504 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:24.504 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:24.504 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:24.504 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:24.504 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T17:51:24.504 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.504 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 4, 2024-04-07T17:51:24.504 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:24.504 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.504 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:24.504 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:24.504 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:24.504 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:24.504 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T17:51:24.504 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.505 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 5, 2024-04-07T17:51:24.505 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:24.505 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:24.505 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T17:51:24.505 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:24.505 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:24.505 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:24.505 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.369727+0000", 2024-04-07T17:51:24.505 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:08.972663+0000", 2024-04-07T17:51:24.505 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:08.972663+0000", 2024-04-07T17:51:24.505 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:08.973250+0000", 2024-04-07T17:51:24.505 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.369727+0000", 2024-04-07T17:51:24.505 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:24.505 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:24.505 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:24.506 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 554, 2024-04-07T17:51:24.506 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.369727+0000", 2024-04-07T17:51:24.506 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.369727+0000", 2024-04-07T17:51:24.506 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.369727+0000", 2024-04-07T17:51:24.506 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:24.506 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:24.506 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:24.506 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.369727+0000", 2024-04-07T17:51:24.506 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.369727+0000", 2024-04-07T17:51:24.506 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:24.506 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:24.506 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:24.506 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:24.507 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 553, 2024-04-07T17:51:24.507 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:24.507 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:24.507 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:24.507 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:24.507 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:24.507 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:24.507 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:24.507 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:24.507 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.2", 2024-04-07T17:51:24.507 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:24.507 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:24.507 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:24.507 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4868, 2024-04-07T17:51:24.507 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:24.507 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T22:42:44.603307+0000", 2024-04-07T17:51:24.508 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:24.508 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:24.508 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:24.508 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:24.508 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:24.508 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:24.508 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:24.508 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:24.508 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:24.508 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:24.508 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:24.508 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:24.508 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:24.508 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:24.508 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:24.508 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:24.509 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:24.509 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:24.509 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:24.509 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:24.509 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:24.509 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:24.509 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:24.509 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:24.509 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:24.509 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:24.509 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:24.509 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:24.509 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:24.509 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:24.509 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:24.510 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:24.510 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:24.510 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:24.510 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:24.510 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:24.510 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:24.510 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:24.510 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:24.510 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:24.510 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:24.510 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:24.510 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:24.510 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.510 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:24.511 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:24.511 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:24.511 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:24.511 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:24.511 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T17:51:24.511 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.511 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 5, 2024-04-07T17:51:24.511 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:24.511 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.511 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:24.511 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:24.511 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:24.511 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T17:51:24.511 INFO:tasks.workunit.client.0.smithi100.stdout: 3 2024-04-07T17:51:24.511 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.512 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 1, 2024-04-07T17:51:24.512 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:24.512 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:24.512 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 66, 2024-04-07T17:51:24.512 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:24.512 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:24.512 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:24.512 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.334401+0000", 2024-04-07T17:51:24.512 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:23:43.924443+0000", 2024-04-07T17:51:24.512 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:23:43.924443+0000", 2024-04-07T17:51:24.512 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:59:56.538443+0000", 2024-04-07T17:51:24.512 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.334401+0000", 2024-04-07T17:51:24.512 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:13:28.963534+0000", 2024-04-07T17:51:24.512 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:24.512 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:13:28.963534+0000", 2024-04-07T17:51:24.512 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 471, 2024-04-07T17:51:24.513 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.334401+0000", 2024-04-07T17:51:24.513 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.334401+0000", 2024-04-07T17:51:24.513 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.334401+0000", 2024-04-07T17:51:24.513 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:24.513 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:24.513 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:13:28.963534+0000", 2024-04-07T17:51:24.513 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.334401+0000", 2024-04-07T17:51:24.513 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.334401+0000", 2024-04-07T17:51:24.513 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:24.513 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 2976, 2024-04-07T17:51:24.513 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:24.513 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:24.513 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 470, 2024-04-07T17:51:24.513 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:24.513 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:24.513 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:24.514 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:24.514 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 2976, 2024-04-07T17:51:24.514 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:24.514 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:24.514 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:24.514 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "4.4", 2024-04-07T17:51:24.514 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:24.514 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:24.514 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:24.514 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 8713, 2024-04-07T17:51:24.514 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:24.514 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T16:43:02.513339+0000", 2024-04-07T17:51:24.514 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.010915499, 2024-04-07T17:51:24.514 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:24.515 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:24.515 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:24.515 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:24.515 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:24.515 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:24.515 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:24.515 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:24.515 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:24.515 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:24.515 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:24.515 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:24.515 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:24.515 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:24.515 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:24.515 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:24.516 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:24.516 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:24.516 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 3, 2024-04-07T17:51:24.516 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 1, 2024-04-07T17:51:24.516 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:24.516 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 1, 2024-04-07T17:51:24.516 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:24.516 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:24.516 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:24.516 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:24.516 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:24.516 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 1, 2024-04-07T17:51:24.516 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:24.516 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:24.516 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:24.517 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:24.517 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:24.517 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:24.517 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:24.517 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 1187, 2024-04-07T17:51:24.517 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 375700, 2024-04-07T17:51:24.517 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:24.517 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:24.517 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:24.517 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 2976, 2024-04-07T17:51:24.517 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 350581 2024-04-07T17:51:24.517 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.517 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:24.517 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:24.517 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:24.518 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:24.518 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T17:51:24.518 INFO:tasks.workunit.client.0.smithi100.stdout: 3 2024-04-07T17:51:24.518 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.518 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 1, 2024-04-07T17:51:24.518 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "1084'2976" 2024-04-07T17:51:24.518 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.518 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:24.518 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:24.518 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:24.518 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:24.518 INFO:tasks.workunit.client.0.smithi100.stdout: 2 2024-04-07T17:51:24.518 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.518 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 5, 2024-04-07T17:51:24.518 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:24.519 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:24.519 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:24.519 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:24.519 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:24.519 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:24.519 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.364565+0000", 2024-04-07T17:51:24.519 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:24:41.554485+0000", 2024-04-07T17:51:24.519 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:24:41.554485+0000", 2024-04-07T17:51:24.519 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:24:41.557859+0000", 2024-04-07T17:51:24.519 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.364565+0000", 2024-04-07T17:51:24.519 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.519 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:24.519 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.519 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 527, 2024-04-07T17:51:24.519 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.364565+0000", 2024-04-07T17:51:24.519 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.364565+0000", 2024-04-07T17:51:24.520 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.364565+0000", 2024-04-07T17:51:24.520 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:24.520 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:24.520 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.520 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.364565+0000", 2024-04-07T17:51:24.520 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.364565+0000", 2024-04-07T17:51:24.520 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:24.520 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:24.520 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:24.520 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:24.520 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 526, 2024-04-07T17:51:24.520 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:24.520 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:24.520 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:24.521 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:24.521 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:24.521 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:24.521 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:24.521 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:24.521 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.5", 2024-04-07T17:51:24.521 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:24.521 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:24.521 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:24.521 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4886, 2024-04-07T17:51:24.521 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:24.521 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T18:53:57.243053+0000", 2024-04-07T17:51:24.521 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:24.521 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:24.521 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:24.522 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:24.522 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:24.522 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:24.522 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:24.522 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:24.522 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:24.522 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:24.522 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:24.522 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:24.522 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:24.522 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:24.522 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:24.522 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:24.522 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:24.523 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:24.523 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:24.523 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:24.523 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:24.523 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:24.523 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:24.523 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:24.523 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:24.523 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:24.523 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:24.523 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:24.523 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:24.523 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:24.523 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:24.524 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:24.524 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:24.524 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:24.524 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:24.524 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:24.524 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:24.524 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:24.524 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:24.524 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:24.524 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:24.524 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:24.524 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:24.524 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.524 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:24.524 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:24.524 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:24.525 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:24.525 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:24.525 INFO:tasks.workunit.client.0.smithi100.stdout: 2 2024-04-07T17:51:24.525 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.525 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 5, 2024-04-07T17:51:24.525 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:24.525 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.525 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:24.525 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:24.525 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:24.525 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:24.525 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T17:51:24.525 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.525 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 1, 2024-04-07T17:51:24.525 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:24.526 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:24.526 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T17:51:24.526 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:24.526 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:24.526 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:24.526 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.333650+0000", 2024-04-07T17:51:24.526 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:38.999666+0000", 2024-04-07T17:51:24.526 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:38.999666+0000", 2024-04-07T17:51:24.526 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:39.000176+0000", 2024-04-07T17:51:24.526 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.333650+0000", 2024-04-07T17:51:24.526 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:24.526 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:24.526 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:24.526 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:24.527 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.333650+0000", 2024-04-07T17:51:24.527 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.333650+0000", 2024-04-07T17:51:24.527 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.333650+0000", 2024-04-07T17:51:24.527 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:24.527 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:24.527 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:24.527 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.333650+0000", 2024-04-07T17:51:24.527 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.333650+0000", 2024-04-07T17:51:24.527 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:24.527 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:24.527 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:24.527 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:24.527 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:24.527 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:24.527 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:24.528 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:24.528 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:24.528 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:24.528 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:24.528 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:24.528 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:24.528 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.4", 2024-04-07T17:51:24.528 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:24.528 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:24.528 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:24.528 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4990, 2024-04-07T17:51:24.528 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:24.528 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T19:36:37.701087+0000", 2024-04-07T17:51:24.528 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:24.528 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:24.528 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:24.529 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:24.529 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:24.529 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:24.529 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:24.529 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:24.529 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:24.529 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:24.529 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:24.529 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:24.529 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:24.529 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:24.529 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:24.529 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:24.529 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:24.530 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:24.530 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:24.530 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:24.530 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:24.530 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:24.530 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:24.530 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:24.530 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:24.530 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:24.530 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:24.530 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:24.530 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:24.530 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:24.530 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:24.531 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:24.531 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:24.531 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:24.531 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:24.531 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:24.531 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:24.531 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:24.531 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:24.531 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:24.531 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:24.531 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:24.531 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:24.531 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.531 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:24.531 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:24.532 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:24.532 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:24.532 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:24.532 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T17:51:24.532 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.532 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 1, 2024-04-07T17:51:24.532 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:24.532 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.532 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:24.532 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:24.532 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:24.532 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:24.532 INFO:tasks.workunit.client.0.smithi100.stdout: 4 2024-04-07T17:51:24.532 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.533 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 1, 2024-04-07T17:51:24.533 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:24.533 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:24.533 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 66, 2024-04-07T17:51:24.533 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:24.533 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:24.533 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:24.533 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.335612+0000", 2024-04-07T17:51:24.533 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:24:41.537435+0000", 2024-04-07T17:51:24.533 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:24:41.537435+0000", 2024-04-07T17:51:24.533 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:59:56.544979+0000", 2024-04-07T17:51:24.533 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.335612+0000", 2024-04-07T17:51:24.533 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:13:28.963534+0000", 2024-04-07T17:51:24.533 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:24.533 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:13:28.963534+0000", 2024-04-07T17:51:24.534 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 527, 2024-04-07T17:51:24.534 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.335612+0000", 2024-04-07T17:51:24.534 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.335612+0000", 2024-04-07T17:51:24.534 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.335612+0000", 2024-04-07T17:51:24.534 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:24.534 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:24.534 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:13:28.963534+0000", 2024-04-07T17:51:24.534 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.335612+0000", 2024-04-07T17:51:24.534 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.335612+0000", 2024-04-07T17:51:24.534 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:24.534 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 2963, 2024-04-07T17:51:24.534 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:24.534 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:24.534 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 526, 2024-04-07T17:51:24.535 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:24.535 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:24.535 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:24.535 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:24.535 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 2963, 2024-04-07T17:51:24.535 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:24.535 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:24.535 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:24.535 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "4.2", 2024-04-07T17:51:24.535 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:24.535 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:24.535 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:24.535 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 8984, 2024-04-07T17:51:24.536 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:24.536 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T23:06:36.963212+0000", 2024-04-07T17:51:24.536 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.015938325, 2024-04-07T17:51:24.536 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:24.536 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:24.536 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 19, 2024-04-07T17:51:24.536 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:24.536 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:24.536 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:24.536 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:24.536 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:24.536 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:24.536 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:24.536 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:24.536 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:24.537 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:24.537 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:24.537 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:24.537 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:24.537 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:24.537 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:24.537 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 3, 2024-04-07T17:51:24.537 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 1, 2024-04-07T17:51:24.537 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:24.537 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 1, 2024-04-07T17:51:24.537 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:24.537 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:24.537 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:24.537 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:24.537 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:24.538 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:24.538 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:24.538 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:24.538 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:24.538 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:24.538 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:24.538 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:24.538 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:24.538 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 1654, 2024-04-07T17:51:24.538 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 387082, 2024-04-07T17:51:24.538 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:24.538 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:24.538 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:24.538 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 2937, 2024-04-07T17:51:24.539 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 349256 2024-04-07T17:51:24.539 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.539 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:24.539 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:24.539 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:24.539 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:24.539 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:24.539 INFO:tasks.workunit.client.0.smithi100.stdout: 4 2024-04-07T17:51:24.539 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.539 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 1, 2024-04-07T17:51:24.539 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "1083'2963" 2024-04-07T17:51:24.539 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.539 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:24.539 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:24.539 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:24.539 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:24.540 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T17:51:24.540 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.540 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 7, 2024-04-07T17:51:24.540 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:24.540 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:24.540 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:24.540 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:24.540 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:24.540 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:24.540 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.372374+0000", 2024-04-07T17:51:24.540 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:38.968892+0000", 2024-04-07T17:51:24.540 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:38.968892+0000", 2024-04-07T17:51:24.540 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:38.970807+0000", 2024-04-07T17:51:24.540 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.372374+0000", 2024-04-07T17:51:24.540 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.541 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:24.541 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.541 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:24.541 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.372374+0000", 2024-04-07T17:51:24.541 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.372374+0000", 2024-04-07T17:51:24.541 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.372374+0000", 2024-04-07T17:51:24.541 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:24.541 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:24.541 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.541 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.372374+0000", 2024-04-07T17:51:24.541 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.372374+0000", 2024-04-07T17:51:24.541 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:24.541 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:24.541 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:24.541 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:24.542 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:24.542 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:24.542 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:24.542 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:24.542 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:24.542 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:24.542 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:24.542 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:24.542 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:24.542 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.e", 2024-04-07T17:51:24.542 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:24.542 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:24.542 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:24.542 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4791, 2024-04-07T17:51:24.543 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:24.543 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T21:17:23.974869+0000", 2024-04-07T17:51:24.543 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:24.543 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:24.543 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:24.543 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:24.543 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:24.543 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:24.543 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:24.543 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:24.543 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:24.543 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:24.543 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:24.543 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:24.544 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:24.544 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:24.544 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:24.544 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:24.544 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:24.544 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:24.544 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:24.544 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:24.544 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:24.544 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:24.544 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:24.544 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:24.544 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:24.544 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:24.544 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:24.544 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:24.545 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:24.545 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:24.545 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:24.545 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:24.545 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:24.545 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:24.545 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:24.545 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:24.545 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:24.545 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:24.545 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:24.545 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:24.545 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:24.545 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:24.545 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:24.546 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.546 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:24.546 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:24.546 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:24.546 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:24.546 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:24.546 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T17:51:24.546 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.546 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 7, 2024-04-07T17:51:24.546 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:24.546 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.546 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:24.546 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:24.546 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:24.547 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:24.547 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T17:51:24.547 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.547 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 4, 2024-04-07T17:51:24.547 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:24.547 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:24.547 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T17:51:24.547 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:24.547 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:24.547 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:24.547 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:35.981613+0000", 2024-04-07T17:51:24.547 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:38.967913+0000", 2024-04-07T17:51:24.547 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:38.967913+0000", 2024-04-07T17:51:24.547 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:38.968805+0000", 2024-04-07T17:51:24.547 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:35.981613+0000", 2024-04-07T17:51:24.548 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:24.548 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:24.548 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:24.548 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:24.548 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:35.981613+0000", 2024-04-07T17:51:24.548 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:35.981613+0000", 2024-04-07T17:51:24.548 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:35.981613+0000", 2024-04-07T17:51:24.548 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:24.548 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:24.548 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:24.548 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:35.981613+0000", 2024-04-07T17:51:24.548 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:35.981613+0000", 2024-04-07T17:51:24.548 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:24.548 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 1, 2024-04-07T17:51:24.548 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:24.549 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:24.549 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:24.549 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:24.549 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:24.549 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:24.549 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:24.549 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 1, 2024-04-07T17:51:24.549 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:24.549 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:24.549 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:24.549 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.f", 2024-04-07T17:51:24.549 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:24.549 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:24.549 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:24.549 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4919, 2024-04-07T17:51:24.550 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:24.550 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T19:43:23.875589+0000", 2024-04-07T17:51:24.550 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:24.550 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:24.550 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:24.550 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 22, 2024-04-07T17:51:24.550 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:24.550 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:24.550 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:24.550 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:24.550 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:24.550 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:24.550 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:24.550 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:24.551 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:24.551 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:24.551 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:24.551 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:24.551 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:24.551 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:24.551 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:24.551 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 3, 2024-04-07T17:51:24.551 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 1, 2024-04-07T17:51:24.551 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:24.551 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 1, 2024-04-07T17:51:24.551 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:24.551 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:24.551 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:24.553 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:24.553 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:24.553 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:24.553 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:24.553 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:24.553 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:24.553 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:24.553 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:24.554 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:24.554 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:24.554 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 1, 2024-04-07T17:51:24.554 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 1, 2024-04-07T17:51:24.554 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:24.554 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:24.554 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:24.554 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 1, 2024-04-07T17:51:24.554 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 1 2024-04-07T17:51:24.554 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.554 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:24.554 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:24.554 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:24.554 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:24.555 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:24.555 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T17:51:24.555 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.555 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 4, 2024-04-07T17:51:24.555 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "61'1" 2024-04-07T17:51:24.555 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.555 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:24.555 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:24.555 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:24.555 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:24.555 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T17:51:24.555 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.555 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 7, 2024-04-07T17:51:24.555 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:24.555 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:24.556 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:24.556 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:24.556 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:24.556 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:24.556 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.358184+0000", 2024-04-07T17:51:24.556 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:38.987996+0000", 2024-04-07T17:51:24.556 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:38.987996+0000", 2024-04-07T17:51:24.556 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:38.989241+0000", 2024-04-07T17:51:24.556 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.358184+0000", 2024-04-07T17:51:24.556 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.556 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:24.556 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.556 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:24.556 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.358184+0000", 2024-04-07T17:51:24.556 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.358184+0000", 2024-04-07T17:51:24.557 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.358184+0000", 2024-04-07T17:51:24.557 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:24.557 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:24.557 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.557 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.358184+0000", 2024-04-07T17:51:24.557 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.358184+0000", 2024-04-07T17:51:24.557 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:24.557 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:24.557 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:24.557 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:24.557 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:24.557 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:24.557 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:24.557 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:24.557 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:24.558 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:24.558 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:24.558 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:24.558 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:24.558 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.11", 2024-04-07T17:51:24.558 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:24.558 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:24.558 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:24.558 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4801, 2024-04-07T17:51:24.558 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:24.558 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T16:46:00.763122+0000", 2024-04-07T17:51:24.558 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:24.558 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:24.558 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:24.559 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:24.559 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:24.559 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:24.559 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:24.559 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:24.559 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:24.559 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:24.559 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:24.559 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:24.559 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:24.559 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:24.559 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:24.559 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:24.559 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:24.559 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:24.560 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:24.560 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:24.560 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:24.560 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:24.560 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:24.560 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:24.560 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:24.560 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:24.560 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:24.560 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:24.560 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:24.560 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:24.560 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:24.560 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:24.560 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:24.560 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:24.560 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:24.561 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:24.561 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:24.561 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:24.561 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:24.561 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:24.561 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:24.561 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:24.561 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:24.561 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.561 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:24.561 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:24.561 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:24.561 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:24.561 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:24.561 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T17:51:24.562 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.562 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 7, 2024-04-07T17:51:24.562 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:24.562 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.562 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:24.562 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:24.562 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T17:51:24.562 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:24.562 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T17:51:24.562 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.562 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 2, 2024-04-07T17:51:24.562 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:24.562 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:24.563 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T17:51:24.563 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:24.563 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:24.563 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:24.563 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.354070+0000", 2024-04-07T17:51:24.563 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:23:15.875351+0000", 2024-04-07T17:51:24.563 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:23:15.875351+0000", 2024-04-07T17:51:24.563 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:23:15.880710+0000", 2024-04-07T17:51:24.563 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.354070+0000", 2024-04-07T17:51:24.563 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:24.563 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:24.563 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:24.563 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 445, 2024-04-07T17:51:24.563 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.354070+0000", 2024-04-07T17:51:24.563 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.354070+0000", 2024-04-07T17:51:24.563 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.354070+0000", 2024-04-07T17:51:24.564 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:24.564 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:24.564 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:24.564 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.354070+0000", 2024-04-07T17:51:24.564 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.354070+0000", 2024-04-07T17:51:24.564 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:24.564 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 8, 2024-04-07T17:51:24.564 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:24.564 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:24.564 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 444, 2024-04-07T17:51:24.564 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:24.564 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:24.564 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:24.564 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:24.564 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 8, 2024-04-07T17:51:24.565 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:24.565 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:24.565 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:24.565 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.10", 2024-04-07T17:51:24.565 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:24.565 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:24.565 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:24.565 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4979, 2024-04-07T17:51:24.565 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:24.565 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T17:16:20.268788+0000", 2024-04-07T17:51:24.565 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:24.565 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:24.565 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:24.565 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:24.565 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:24.565 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:24.566 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:24.566 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:24.566 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:24.566 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:24.566 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:24.566 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:24.566 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:24.566 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:24.566 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:24.566 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:24.566 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:24.566 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:24.566 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:24.567 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 12, 2024-04-07T17:51:24.567 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 4, 2024-04-07T17:51:24.567 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:24.567 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 4, 2024-04-07T17:51:24.567 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:24.567 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:24.567 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:24.567 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:24.567 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:24.567 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 4, 2024-04-07T17:51:24.567 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:24.567 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:24.567 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:24.567 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:24.567 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:24.568 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:24.568 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:24.568 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 24, 2024-04-07T17:51:24.568 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 24, 2024-04-07T17:51:24.568 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:24.568 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:24.568 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:24.568 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 12, 2024-04-07T17:51:24.568 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:24.568 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.568 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:24.568 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:24.568 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:24.568 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T17:51:24.568 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:24.569 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T17:51:24.569 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.569 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 2, 2024-04-07T17:51:24.569 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "61'8" 2024-04-07T17:51:24.569 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.569 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:24.569 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:24.569 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:24.569 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:24.569 INFO:tasks.workunit.client.0.smithi100.stdout: 5 2024-04-07T17:51:24.569 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.569 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 6, 2024-04-07T17:51:24.569 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:24.569 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:24.569 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:24.570 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:24.570 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:24.570 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:24.570 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.370556+0000", 2024-04-07T17:51:24.570 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:08.963259+0000", 2024-04-07T17:51:24.570 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:08.963259+0000", 2024-04-07T17:51:24.570 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:08.964055+0000", 2024-04-07T17:51:24.570 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.370556+0000", 2024-04-07T17:51:24.570 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.570 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:24.570 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.570 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 554, 2024-04-07T17:51:24.570 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.370556+0000", 2024-04-07T17:51:24.570 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.370556+0000", 2024-04-07T17:51:24.571 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.370556+0000", 2024-04-07T17:51:24.571 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:24.571 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:24.571 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.571 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.370556+0000", 2024-04-07T17:51:24.571 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.370556+0000", 2024-04-07T17:51:24.571 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:24.571 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:24.571 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:24.571 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:24.571 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 553, 2024-04-07T17:51:24.571 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:24.571 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:24.571 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:24.572 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:24.572 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:24.572 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:24.572 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:24.572 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:24.572 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.10", 2024-04-07T17:51:24.572 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:24.572 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:24.572 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:24.572 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4829, 2024-04-07T17:51:24.572 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:24.572 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T23:13:34.886609+0000", 2024-04-07T17:51:24.572 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:24.572 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:24.573 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:24.573 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:24.573 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:24.573 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:24.573 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:24.573 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:24.573 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:24.573 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:24.573 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:24.573 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:24.573 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:24.573 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:24.573 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:24.573 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:24.573 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:24.574 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:24.574 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:24.574 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:24.574 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:24.574 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:24.574 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:24.574 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:24.574 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:24.574 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:24.574 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:24.574 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:24.574 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:24.574 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:24.575 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:24.575 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:24.575 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:24.575 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:24.575 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:24.575 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:24.575 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:24.575 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:24.575 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:24.575 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:24.575 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:24.575 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:24.575 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:24.575 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.575 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:24.576 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:24.576 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:24.576 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:24.576 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:24.576 INFO:tasks.workunit.client.0.smithi100.stdout: 5 2024-04-07T17:51:24.576 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.576 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 6, 2024-04-07T17:51:24.576 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:24.576 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.576 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:24.576 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:24.576 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:24.576 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:24.576 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T17:51:24.576 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.577 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 6, 2024-04-07T17:51:24.577 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:24.577 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:24.577 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T17:51:24.577 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:24.577 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:24.577 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:24.577 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.371043+0000", 2024-04-07T17:51:24.577 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:08.965799+0000", 2024-04-07T17:51:24.577 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:08.965799+0000", 2024-04-07T17:51:24.577 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:08.966301+0000", 2024-04-07T17:51:24.577 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.371043+0000", 2024-04-07T17:51:24.577 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:24.577 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:24.577 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:24.578 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 554, 2024-04-07T17:51:24.578 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.371043+0000", 2024-04-07T17:51:24.578 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.371043+0000", 2024-04-07T17:51:24.578 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.371043+0000", 2024-04-07T17:51:24.578 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:24.578 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:24.578 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:24.578 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.371043+0000", 2024-04-07T17:51:24.578 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.371043+0000", 2024-04-07T17:51:24.578 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:24.578 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:24.578 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:24.578 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:24.578 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 553, 2024-04-07T17:51:24.579 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:24.579 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:24.579 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:24.579 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:24.579 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:24.579 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:24.579 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:24.579 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:24.579 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.11", 2024-04-07T17:51:24.579 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:24.579 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:24.579 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:24.579 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4845, 2024-04-07T17:51:24.579 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:24.580 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T21:12:09.705565+0000", 2024-04-07T17:51:24.580 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:24.580 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:24.580 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:24.580 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:24.580 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:24.580 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:24.580 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:24.580 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:24.580 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:24.580 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:24.580 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:24.580 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:24.580 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:24.580 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:24.580 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:24.581 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:24.581 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:24.581 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:24.581 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:24.581 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:24.581 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:24.581 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:24.581 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:24.581 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:24.581 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:24.581 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:24.581 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:24.581 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:24.581 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:24.581 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:24.581 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:24.581 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:24.582 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:24.582 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:24.582 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:24.582 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:24.582 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:24.582 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:24.582 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:24.582 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:24.582 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:24.582 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:24.582 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:24.582 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.582 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:24.582 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:24.583 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:24.583 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:24.583 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:24.583 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T17:51:24.583 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.583 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 6, 2024-04-07T17:51:24.583 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:24.583 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.583 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:24.583 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:24.583 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:24.583 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:24.583 INFO:tasks.workunit.client.0.smithi100.stdout: 2 2024-04-07T17:51:24.583 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.583 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 7, 2024-04-07T17:51:24.583 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:24.584 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:24.584 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:24.584 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:24.584 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:24.584 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:24.584 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.368748+0000", 2024-04-07T17:51:24.584 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:38.987994+0000", 2024-04-07T17:51:24.584 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:38.987994+0000", 2024-04-07T17:51:24.584 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:38.991515+0000", 2024-04-07T17:51:24.584 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.368748+0000", 2024-04-07T17:51:24.584 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.584 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:24.584 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.584 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:24.585 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.368748+0000", 2024-04-07T17:51:24.585 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.368748+0000", 2024-04-07T17:51:24.585 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.368748+0000", 2024-04-07T17:51:24.585 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:24.585 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:24.585 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.585 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.368748+0000", 2024-04-07T17:51:24.585 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.368748+0000", 2024-04-07T17:51:24.585 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:24.585 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:24.585 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:24.585 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:24.585 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:24.585 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:24.585 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:24.586 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:24.586 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:24.586 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:24.586 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:24.586 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:24.586 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:24.586 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.13", 2024-04-07T17:51:24.586 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:24.586 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:24.586 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:24.586 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4789, 2024-04-07T17:51:24.586 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:24.586 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T20:51:08.365772+0000", 2024-04-07T17:51:24.586 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:24.587 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:24.587 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:24.587 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:24.587 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:24.587 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:24.587 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:24.587 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:24.587 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:24.587 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:24.587 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:24.587 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:24.587 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:24.587 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:24.587 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:24.588 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:24.588 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:24.588 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:24.588 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:24.588 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:24.588 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:24.588 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:24.588 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:24.588 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:24.588 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:24.588 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:24.588 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:24.588 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:24.588 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:24.588 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:24.589 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:24.589 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:24.589 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:24.589 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:24.589 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:24.589 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:24.589 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:24.589 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:24.589 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:24.589 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:24.589 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:24.589 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:24.589 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:24.589 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.589 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:24.589 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:24.590 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:24.590 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:24.590 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:24.590 INFO:tasks.workunit.client.0.smithi100.stdout: 2 2024-04-07T17:51:24.590 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.590 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 7, 2024-04-07T17:51:24.590 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:24.590 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.590 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:24.590 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:24.590 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:24.590 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:24.590 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T17:51:24.590 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.591 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 5, 2024-04-07T17:51:24.591 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:24.591 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:24.591 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T17:51:24.591 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:24.591 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:24.591 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:24.591 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.369416+0000", 2024-04-07T17:51:24.591 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:38.978180+0000", 2024-04-07T17:51:24.591 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:38.978180+0000", 2024-04-07T17:51:24.591 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:38.979026+0000", 2024-04-07T17:51:24.591 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.369416+0000", 2024-04-07T17:51:24.591 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:24.591 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:24.591 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:24.592 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:24.592 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.369416+0000", 2024-04-07T17:51:24.592 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.369416+0000", 2024-04-07T17:51:24.592 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.369416+0000", 2024-04-07T17:51:24.592 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:24.592 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:24.592 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:24.592 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.369416+0000", 2024-04-07T17:51:24.592 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.369416+0000", 2024-04-07T17:51:24.592 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:24.592 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:24.592 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:24.592 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:24.593 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:24.593 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:24.593 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:24.593 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:24.593 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:24.593 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:24.593 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:24.593 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:24.593 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:24.593 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.12", 2024-04-07T17:51:24.593 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:24.593 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:24.593 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:24.593 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4877, 2024-04-07T17:51:24.593 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:24.594 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T01:41:45.196090+0000", 2024-04-07T17:51:24.594 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:24.594 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:24.594 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:24.594 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:24.594 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:24.594 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:24.594 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:24.594 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:24.594 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:24.594 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:24.594 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:24.594 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:24.594 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:24.595 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:24.595 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:24.595 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:24.595 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:24.595 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:24.595 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:24.595 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:24.595 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:24.595 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:24.595 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:24.595 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:24.595 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:24.595 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:24.595 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:24.596 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:24.596 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:24.596 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:24.596 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:24.596 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:24.596 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:24.596 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:24.596 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:24.596 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:24.596 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:24.596 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:24.596 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:24.596 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:24.596 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:24.596 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:24.597 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:24.597 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.597 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:24.597 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:24.597 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:24.597 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:24.597 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:24.597 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T17:51:24.597 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.597 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 5, 2024-04-07T17:51:24.597 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:24.597 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.597 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:24.597 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:24.597 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T17:51:24.598 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:24.598 INFO:tasks.workunit.client.0.smithi100.stdout: 3 2024-04-07T17:51:24.598 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.598 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 2, 2024-04-07T17:51:24.598 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:24.598 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:24.598 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:24.598 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:24.598 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:24.598 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:24.598 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.354815+0000", 2024-04-07T17:51:24.598 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:38.969402+0000", 2024-04-07T17:51:24.598 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:38.969402+0000", 2024-04-07T17:51:24.598 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:38.970054+0000", 2024-04-07T17:51:24.599 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.354815+0000", 2024-04-07T17:51:24.599 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.599 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:24.599 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.599 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:24.599 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.354815+0000", 2024-04-07T17:51:24.599 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.354815+0000", 2024-04-07T17:51:24.599 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.354815+0000", 2024-04-07T17:51:24.599 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:24.599 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:24.599 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.599 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.354815+0000", 2024-04-07T17:51:24.599 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.354815+0000", 2024-04-07T17:51:24.599 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:24.599 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:24.600 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:24.600 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:24.600 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:24.600 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:24.600 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:24.600 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:24.600 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:24.600 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:24.600 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:24.600 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:24.600 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:24.600 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.12", 2024-04-07T17:51:24.600 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:24.600 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:24.600 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:24.600 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4924, 2024-04-07T17:51:24.601 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:24.601 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T23:11:21.401536+0000", 2024-04-07T17:51:24.601 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:24.601 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:24.601 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:24.601 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:24.601 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:24.601 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:24.601 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:24.601 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:24.601 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:24.601 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:24.601 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:24.601 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:24.601 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:24.601 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:24.602 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:24.602 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:24.602 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:24.602 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:24.602 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:24.602 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:24.602 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:24.602 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:24.602 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:24.602 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:24.602 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:24.602 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:24.602 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:24.602 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:24.603 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:24.603 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:24.603 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:24.603 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:24.603 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:24.603 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:24.603 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:24.603 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:24.603 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:24.603 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:24.603 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:24.603 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:24.603 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:24.603 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:24.603 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:24.603 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.604 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:24.604 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:24.604 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:24.604 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T17:51:24.604 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:24.604 INFO:tasks.workunit.client.0.smithi100.stdout: 3 2024-04-07T17:51:24.604 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.604 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 2, 2024-04-07T17:51:24.604 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:24.604 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.604 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:24.604 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:24.604 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:24.604 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:24.604 INFO:tasks.workunit.client.0.smithi100.stdout: 3 2024-04-07T17:51:24.605 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.605 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 0, 2024-04-07T17:51:24.605 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:24.605 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:24.605 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T17:51:24.605 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:24.605 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:24.605 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:24.605 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:35.988325+0000", 2024-04-07T17:51:24.605 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:24:13.963512+0000", 2024-04-07T17:51:24.605 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:24:13.963512+0000", 2024-04-07T17:51:24.605 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:24:13.964689+0000", 2024-04-07T17:51:24.605 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:35.988325+0000", 2024-04-07T17:51:24.605 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:24.605 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:24.606 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:24.606 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 500, 2024-04-07T17:51:24.606 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:35.988325+0000", 2024-04-07T17:51:24.606 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:35.988325+0000", 2024-04-07T17:51:24.606 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:35.988325+0000", 2024-04-07T17:51:24.606 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:24.606 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:24.606 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:24.606 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:35.988325+0000", 2024-04-07T17:51:24.606 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:35.988325+0000", 2024-04-07T17:51:24.606 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:24.606 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 2, 2024-04-07T17:51:24.606 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:24.606 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:24.607 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 499, 2024-04-07T17:51:24.607 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:24.607 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:24.607 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:24.607 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:24.607 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 2, 2024-04-07T17:51:24.607 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:24.607 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:24.607 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:24.607 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.13", 2024-04-07T17:51:24.607 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:24.607 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:24.607 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:24.607 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 5000, 2024-04-07T17:51:24.607 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:24.608 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T16:15:37.955732+0000", 2024-04-07T17:51:24.608 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:24.608 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:24.608 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:24.608 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 34, 2024-04-07T17:51:24.608 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:24.608 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:24.608 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:24.608 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:24.608 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:24.608 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:24.608 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:24.608 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:24.608 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:24.608 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:24.609 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:24.609 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:24.609 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:24.609 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:24.609 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:24.609 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 6, 2024-04-07T17:51:24.609 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 2, 2024-04-07T17:51:24.609 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:24.609 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 2, 2024-04-07T17:51:24.609 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:24.609 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:24.609 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:24.609 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:24.609 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:24.609 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 1, 2024-04-07T17:51:24.610 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:24.610 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:24.610 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:24.610 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:24.610 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:24.610 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:24.610 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:24.610 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 4, 2024-04-07T17:51:24.610 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 8, 2024-04-07T17:51:24.610 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:24.610 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:24.610 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:24.610 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 3, 2024-04-07T17:51:24.610 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 7 2024-04-07T17:51:24.610 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.611 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:24.611 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:24.611 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:24.611 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:24.611 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:24.611 INFO:tasks.workunit.client.0.smithi100.stdout: 3 2024-04-07T17:51:24.611 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.611 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 0, 2024-04-07T17:51:24.611 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "61'2" 2024-04-07T17:51:24.611 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.611 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:24.611 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:24.611 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:24.611 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:24.611 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T17:51:24.611 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.612 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 7, 2024-04-07T17:51:24.612 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:24.612 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:24.612 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:24.612 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:24.612 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:24.612 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:24.612 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.372482+0000", 2024-04-07T17:51:24.612 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:38.987197+0000", 2024-04-07T17:51:24.612 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:38.987197+0000", 2024-04-07T17:51:24.612 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:38.999803+0000", 2024-04-07T17:51:24.612 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.372482+0000", 2024-04-07T17:51:24.612 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.612 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:24.612 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.613 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:24.613 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.372482+0000", 2024-04-07T17:51:24.613 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.372482+0000", 2024-04-07T17:51:24.613 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.372482+0000", 2024-04-07T17:51:24.613 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:24.613 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:24.613 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.613 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.372482+0000", 2024-04-07T17:51:24.613 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.372482+0000", 2024-04-07T17:51:24.613 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:24.613 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:24.613 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:24.613 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:24.613 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:24.613 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:24.613 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:24.614 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:24.614 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:24.614 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:24.614 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:24.614 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:24.614 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:24.614 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.15", 2024-04-07T17:51:24.614 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:24.614 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:24.614 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:24.614 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4809, 2024-04-07T17:51:24.614 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:24.614 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T03:29:56.940037+0000", 2024-04-07T17:51:24.614 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:24.614 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:24.615 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:24.615 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:24.615 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:24.615 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:24.615 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:24.615 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:24.615 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:24.615 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:24.615 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:24.615 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:24.615 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:24.615 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:24.615 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:24.615 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:24.616 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:24.616 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:24.616 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:24.616 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:24.616 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:24.616 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:24.616 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:24.616 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:24.616 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:24.616 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:24.616 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:24.616 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:24.616 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:24.616 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:24.616 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:24.617 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:24.617 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:24.617 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:24.617 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:24.617 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:24.617 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:24.617 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:24.617 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:24.617 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:24.617 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:24.617 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:24.617 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:24.617 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.617 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:24.617 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:24.617 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:24.618 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:24.618 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:24.618 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T17:51:24.618 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.618 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 7, 2024-04-07T17:51:24.618 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:24.618 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.618 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:24.618 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:24.618 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:24.618 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:24.618 INFO:tasks.workunit.client.0.smithi100.stdout: 5 2024-04-07T17:51:24.618 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.618 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 6, 2024-04-07T17:51:24.618 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:24.619 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:24.619 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T17:51:24.619 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:24.619 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:24.619 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:24.619 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.370811+0000", 2024-04-07T17:51:24.619 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:08.975701+0000", 2024-04-07T17:51:24.619 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:08.975701+0000", 2024-04-07T17:51:24.619 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:08.977082+0000", 2024-04-07T17:51:24.619 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.370811+0000", 2024-04-07T17:51:24.619 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:24.619 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:24.619 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:24.619 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 554, 2024-04-07T17:51:24.619 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.370811+0000", 2024-04-07T17:51:24.619 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.370811+0000", 2024-04-07T17:51:24.620 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.370811+0000", 2024-04-07T17:51:24.620 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:24.620 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:24.620 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:24.620 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.370811+0000", 2024-04-07T17:51:24.620 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.370811+0000", 2024-04-07T17:51:24.620 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:24.620 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 29, 2024-04-07T17:51:24.620 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:24.620 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:24.620 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 553, 2024-04-07T17:51:24.620 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:24.620 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:24.620 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:24.620 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:24.620 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 29, 2024-04-07T17:51:24.621 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:24.621 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:24.621 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:24.621 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.14", 2024-04-07T17:51:24.621 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:24.621 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:24.621 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:24.621 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4865, 2024-04-07T17:51:24.621 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:24.621 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T20:48:03.443222+0000", 2024-04-07T17:51:24.621 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:24.621 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:24.621 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:24.621 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 5824, 2024-04-07T17:51:24.621 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:24.621 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:24.622 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:24.622 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:24.622 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:24.622 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:24.622 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:24.622 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:24.622 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:24.622 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:24.622 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:24.622 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:24.622 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:24.622 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:24.622 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:24.622 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 12, 2024-04-07T17:51:24.623 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 4, 2024-04-07T17:51:24.623 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:24.623 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 4, 2024-04-07T17:51:24.623 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:24.623 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:24.623 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:24.623 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:24.623 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:24.623 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 1, 2024-04-07T17:51:24.623 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:24.623 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:24.623 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:24.623 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:24.623 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:24.623 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:24.623 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:24.624 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 10, 2024-04-07T17:51:24.624 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 14, 2024-04-07T17:51:24.624 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:24.624 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:24.624 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:24.624 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 28, 2024-04-07T17:51:24.624 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 25 2024-04-07T17:51:24.624 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.624 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:24.624 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:24.624 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:24.624 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:24.624 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:24.624 INFO:tasks.workunit.client.0.smithi100.stdout: 5 2024-04-07T17:51:24.624 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.624 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 6, 2024-04-07T17:51:24.625 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "623'29" 2024-04-07T17:51:24.625 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.625 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:24.625 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:24.625 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:24.625 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:24.625 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T17:51:24.625 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.625 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 4, 2024-04-07T17:51:24.625 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:24.625 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:24.625 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:24.625 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:24.625 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:24.625 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:24.626 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:35.981022+0000", 2024-04-07T17:51:24.626 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:38.985283+0000", 2024-04-07T17:51:24.626 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:38.985283+0000", 2024-04-07T17:51:24.626 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:38.986153+0000", 2024-04-07T17:51:24.626 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:35.981022+0000", 2024-04-07T17:51:24.626 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.626 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:24.626 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.626 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:24.626 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:35.981022+0000", 2024-04-07T17:51:24.626 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:35.981022+0000", 2024-04-07T17:51:24.626 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:35.981022+0000", 2024-04-07T17:51:24.626 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:24.626 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:24.627 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.627 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:35.981022+0000", 2024-04-07T17:51:24.627 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:35.981022+0000", 2024-04-07T17:51:24.627 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:24.627 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:24.627 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:24.627 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:24.627 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:24.627 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:24.627 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:24.627 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:24.627 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:24.627 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:24.627 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:24.627 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:24.628 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:24.628 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.14", 2024-04-07T17:51:24.628 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:24.628 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:24.628 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:24.628 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4907, 2024-04-07T17:51:24.628 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:24.628 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T22:16:21.117705+0000", 2024-04-07T17:51:24.628 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:24.628 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:24.628 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:24.628 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:24.628 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:24.628 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:24.628 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:24.629 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:24.629 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:24.629 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:24.629 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:24.629 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:24.629 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:24.629 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:24.629 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:24.629 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:24.629 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:24.629 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:24.629 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:24.629 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:24.629 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:24.629 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:24.630 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:24.630 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:24.630 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:24.630 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:24.630 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:24.630 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:24.630 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:24.630 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:24.630 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:24.630 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:24.630 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:24.630 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:24.630 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:24.630 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:24.631 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:24.631 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:24.631 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:24.631 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:24.631 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:24.631 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:24.631 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:24.631 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.631 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:24.631 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:24.631 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:24.631 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:24.631 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:24.631 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T17:51:24.631 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.632 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 4, 2024-04-07T17:51:24.632 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:24.632 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.632 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:24.632 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:24.632 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:24.632 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:24.632 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T17:51:24.632 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.632 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 1, 2024-04-07T17:51:24.632 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:24.632 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:24.632 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T17:51:24.632 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:24.632 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:24.632 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:24.633 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.332631+0000", 2024-04-07T17:51:24.633 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:24:41.527553+0000", 2024-04-07T17:51:24.633 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:24:41.527553+0000", 2024-04-07T17:51:24.633 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:24:41.530429+0000", 2024-04-07T17:51:24.633 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.332631+0000", 2024-04-07T17:51:24.633 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:24.633 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:24.633 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:24.633 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 527, 2024-04-07T17:51:24.633 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.332631+0000", 2024-04-07T17:51:24.633 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.332631+0000", 2024-04-07T17:51:24.633 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.332631+0000", 2024-04-07T17:51:24.633 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:24.633 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:24.633 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:24.633 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.332631+0000", 2024-04-07T17:51:24.634 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.332631+0000", 2024-04-07T17:51:24.634 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:24.634 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 3, 2024-04-07T17:51:24.634 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:24.634 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:24.634 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 526, 2024-04-07T17:51:24.634 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:24.634 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:24.634 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:24.634 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:24.634 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 3, 2024-04-07T17:51:24.634 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:24.634 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:24.634 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:24.634 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.15", 2024-04-07T17:51:24.635 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:24.635 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:24.635 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:24.635 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4990, 2024-04-07T17:51:24.635 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:24.635 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T16:34:47.703047+0000", 2024-04-07T17:51:24.635 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:24.635 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:24.635 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:24.635 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 90, 2024-04-07T17:51:24.635 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:24.635 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:24.635 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:24.635 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:24.636 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:24.636 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:24.636 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:24.636 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:24.636 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:24.636 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:24.636 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:24.636 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:24.636 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:24.636 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:24.636 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:24.636 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 6, 2024-04-07T17:51:24.636 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 2, 2024-04-07T17:51:24.636 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:24.636 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 2, 2024-04-07T17:51:24.637 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:24.637 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:24.637 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:24.637 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:24.637 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:24.637 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 1, 2024-04-07T17:51:24.637 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:24.637 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:24.637 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:24.637 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:24.637 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:24.637 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:24.637 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:24.637 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 7, 2024-04-07T17:51:24.637 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 7, 2024-04-07T17:51:24.638 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:24.638 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:24.638 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:24.638 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 4, 2024-04-07T17:51:24.638 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 1 2024-04-07T17:51:24.638 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.638 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:24.638 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:24.638 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:24.638 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:24.638 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:24.638 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T17:51:24.638 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.638 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 1, 2024-04-07T17:51:24.638 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "61'3" 2024-04-07T17:51:24.639 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.639 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:24.639 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:24.639 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T17:51:24.639 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:24.639 INFO:tasks.workunit.client.0.smithi100.stdout: 3 2024-04-07T17:51:24.639 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.639 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 2, 2024-04-07T17:51:24.639 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:24.639 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:24.639 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:24.639 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:24.639 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:24.639 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:24.639 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.363660+0000", 2024-04-07T17:51:24.640 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:24:41.558300+0000", 2024-04-07T17:51:24.640 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:24:41.558300+0000", 2024-04-07T17:51:24.640 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:24:41.558648+0000", 2024-04-07T17:51:24.640 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.363660+0000", 2024-04-07T17:51:24.640 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.640 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:24.640 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.640 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 527, 2024-04-07T17:51:24.640 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.363660+0000", 2024-04-07T17:51:24.640 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.363660+0000", 2024-04-07T17:51:24.640 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.363660+0000", 2024-04-07T17:51:24.640 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:24.640 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:24.640 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.640 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.363660+0000", 2024-04-07T17:51:24.641 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.363660+0000", 2024-04-07T17:51:24.641 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:24.641 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:24.641 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:24.641 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:24.641 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 526, 2024-04-07T17:51:24.641 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:24.641 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:24.641 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:24.641 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:24.641 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:24.641 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:24.641 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:24.641 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:24.641 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.17", 2024-04-07T17:51:24.642 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:24.642 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:24.642 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:24.642 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4923, 2024-04-07T17:51:24.642 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:24.642 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T01:44:12.981988+0000", 2024-04-07T17:51:24.642 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:24.642 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:24.642 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:24.642 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:24.642 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:24.642 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:24.642 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:24.642 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:24.643 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:24.643 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:24.643 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:24.643 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:24.643 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:24.643 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:24.643 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:24.643 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:24.643 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:24.643 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:24.643 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:24.643 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:24.643 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:24.643 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:24.643 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:24.643 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:24.644 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:24.644 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:24.644 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:24.644 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:24.644 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:24.644 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:24.644 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:24.644 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:24.644 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:24.644 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:24.644 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:24.644 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:24.644 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:24.644 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:24.644 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:24.645 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:24.645 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:24.645 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:24.645 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:24.645 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.645 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:24.645 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:24.645 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:24.645 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T17:51:24.645 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:24.645 INFO:tasks.workunit.client.0.smithi100.stdout: 3 2024-04-07T17:51:24.645 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.645 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 2, 2024-04-07T17:51:24.645 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:24.645 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.646 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:24.646 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:24.646 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:24.646 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:24.646 INFO:tasks.workunit.client.0.smithi100.stdout: 2 2024-04-07T17:51:24.646 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.646 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 5, 2024-04-07T17:51:24.646 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:24.646 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:24.646 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T17:51:24.646 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:24.646 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:24.646 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:24.646 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.368702+0000", 2024-04-07T17:51:24.646 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:08.971659+0000", 2024-04-07T17:51:24.647 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:08.971659+0000", 2024-04-07T17:51:24.647 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:08.976930+0000", 2024-04-07T17:51:24.647 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.368702+0000", 2024-04-07T17:51:24.647 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:24.647 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:24.647 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:24.647 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 554, 2024-04-07T17:51:24.647 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.368702+0000", 2024-04-07T17:51:24.647 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.368702+0000", 2024-04-07T17:51:24.647 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.368702+0000", 2024-04-07T17:51:24.647 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:24.647 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:24.647 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:24.647 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.368702+0000", 2024-04-07T17:51:24.647 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.368702+0000", 2024-04-07T17:51:24.648 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:24.648 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:24.648 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:24.648 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:24.648 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 553, 2024-04-07T17:51:24.648 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:24.648 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:24.648 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:24.648 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:24.648 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:24.648 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:24.648 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:24.648 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:24.648 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.16", 2024-04-07T17:51:24.648 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:24.648 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:24.648 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:24.649 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4848, 2024-04-07T17:51:24.649 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:24.649 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T02:13:26.493523+0000", 2024-04-07T17:51:24.649 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:24.649 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:24.649 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:24.649 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:24.649 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:24.649 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:24.649 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:24.649 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:24.649 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:24.649 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:24.649 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:24.650 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:24.650 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:24.650 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:24.650 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:24.650 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:24.650 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:24.650 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:24.650 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:24.650 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:24.650 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:24.650 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:24.650 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:24.650 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:24.650 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:24.650 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:24.651 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:24.651 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:24.651 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:24.651 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:24.651 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:24.651 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:24.651 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:24.651 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:24.651 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:24.651 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:24.651 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:24.651 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:24.651 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:24.651 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:24.651 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:24.651 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:24.652 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:24.652 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.652 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:24.652 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:24.652 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:24.652 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:24.652 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:24.652 INFO:tasks.workunit.client.0.smithi100.stdout: 2 2024-04-07T17:51:24.652 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.652 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 5, 2024-04-07T17:51:24.652 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:24.652 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.652 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:24.652 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:24.652 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:24.653 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:24.653 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T17:51:24.653 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.653 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 5, 2024-04-07T17:51:24.653 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:24.653 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:24.653 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:24.653 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:24.653 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:24.653 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:24.653 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.369127+0000", 2024-04-07T17:51:24.653 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:38.977859+0000", 2024-04-07T17:51:24.653 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:38.977859+0000", 2024-04-07T17:51:24.653 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:38.978447+0000", 2024-04-07T17:51:24.653 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.369127+0000", 2024-04-07T17:51:24.653 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.654 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:24.654 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.654 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:24.654 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.369127+0000", 2024-04-07T17:51:24.654 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.369127+0000", 2024-04-07T17:51:24.654 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.369127+0000", 2024-04-07T17:51:24.654 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:24.654 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:24.654 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.654 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.369127+0000", 2024-04-07T17:51:24.654 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.369127+0000", 2024-04-07T17:51:24.654 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:24.654 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:24.655 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:24.655 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:24.655 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:24.655 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:24.655 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:24.655 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:24.655 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:24.655 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:24.655 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:24.655 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:24.655 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:24.655 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.16", 2024-04-07T17:51:24.655 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:24.655 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:24.655 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:24.656 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4860, 2024-04-07T17:51:24.656 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:24.656 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T16:22:51.136501+0000", 2024-04-07T17:51:24.656 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:24.656 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:24.656 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:24.656 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:24.656 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:24.656 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:24.656 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:24.656 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:24.656 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:24.656 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:24.656 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:24.657 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:24.657 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:24.657 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:24.657 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:24.657 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:24.657 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:24.657 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:24.657 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:24.657 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:24.657 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:24.657 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:24.657 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:24.657 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:24.657 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:24.657 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:24.658 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:24.658 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:24.658 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:24.658 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:24.658 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:24.658 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:24.658 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:24.658 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:24.658 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:24.658 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:24.658 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:24.658 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:24.658 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:24.658 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:24.658 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:24.659 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:24.659 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:24.659 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.659 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:24.659 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:24.659 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:24.659 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:24.659 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:24.659 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T17:51:24.659 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.659 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 5, 2024-04-07T17:51:24.659 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:24.659 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.659 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:24.659 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:24.659 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:24.660 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:24.660 INFO:tasks.workunit.client.0.smithi100.stdout: 2 2024-04-07T17:51:24.660 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.660 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 6, 2024-04-07T17:51:24.660 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:24.660 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:24.660 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T17:51:24.660 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:24.660 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:24.660 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:24.660 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.370115+0000", 2024-04-07T17:51:24.660 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:08.977704+0000", 2024-04-07T17:51:24.660 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:08.977704+0000", 2024-04-07T17:51:24.660 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:08.978095+0000", 2024-04-07T17:51:24.660 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.370115+0000", 2024-04-07T17:51:24.660 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:24.661 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:24.661 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:24.661 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 554, 2024-04-07T17:51:24.661 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.370115+0000", 2024-04-07T17:51:24.661 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.370115+0000", 2024-04-07T17:51:24.661 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.370115+0000", 2024-04-07T17:51:24.661 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:24.661 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:24.661 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:24.661 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.370115+0000", 2024-04-07T17:51:24.661 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.370115+0000", 2024-04-07T17:51:24.661 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:24.661 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:24.661 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:24.661 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:24.662 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 553, 2024-04-07T17:51:24.662 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:24.662 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:24.662 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:24.662 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:24.662 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:24.662 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:24.662 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:24.662 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:24.662 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.17", 2024-04-07T17:51:24.662 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:24.662 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:24.662 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:24.662 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4829, 2024-04-07T17:51:24.662 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:24.662 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T18:47:28.916356+0000", 2024-04-07T17:51:24.663 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:24.663 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:24.663 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:24.663 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:24.663 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:24.663 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:24.663 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:24.663 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:24.663 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:24.663 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:24.663 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:24.663 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:24.663 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:24.663 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:24.664 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:24.664 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:24.664 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:24.664 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:24.664 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:24.664 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:24.664 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:24.664 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:24.664 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:24.664 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:24.664 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:24.664 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:24.664 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:24.664 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:24.664 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:24.664 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:24.665 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:24.665 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:24.665 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:24.665 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:24.665 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:24.665 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:24.665 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:24.665 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:24.665 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:24.665 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:24.665 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:24.665 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:24.665 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:24.665 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.665 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:24.665 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:24.666 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:24.666 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:24.666 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:24.666 INFO:tasks.workunit.client.0.smithi100.stdout: 2 2024-04-07T17:51:24.666 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.666 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 6, 2024-04-07T17:51:24.666 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:24.666 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.666 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:24.666 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:24.666 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:24.666 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:24.666 INFO:tasks.workunit.client.0.smithi100.stdout: 4 2024-04-07T17:51:24.666 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.666 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 1, 2024-04-07T17:51:24.667 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:24.667 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:24.667 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:24.667 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:24.667 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:24.667 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:24.667 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.331404+0000", 2024-04-07T17:51:24.667 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:24:13.932694+0000", 2024-04-07T17:51:24.667 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:24:13.932694+0000", 2024-04-07T17:51:24.667 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:24:13.933594+0000", 2024-04-07T17:51:24.667 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.331404+0000", 2024-04-07T17:51:24.667 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.667 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:24.667 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.667 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 500, 2024-04-07T17:51:24.668 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.331404+0000", 2024-04-07T17:51:24.668 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.331404+0000", 2024-04-07T17:51:24.668 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.331404+0000", 2024-04-07T17:51:24.668 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:24.668 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:24.668 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.668 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.331404+0000", 2024-04-07T17:51:24.668 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.331404+0000", 2024-04-07T17:51:24.668 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:24.668 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:24.668 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:24.668 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:24.668 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 499, 2024-04-07T17:51:24.668 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:24.668 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:24.668 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:24.669 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:24.669 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:24.669 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:24.669 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:24.669 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:24.669 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.19", 2024-04-07T17:51:24.669 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:24.669 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:24.669 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:24.669 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4977, 2024-04-07T17:51:24.669 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:24.669 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T17:08:30.909773+0000", 2024-04-07T17:51:24.669 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:24.669 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:24.669 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:24.670 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:24.670 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:24.670 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:24.670 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:24.670 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:24.670 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:24.670 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:24.670 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:24.670 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:24.670 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:24.670 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:24.670 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:24.670 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:24.670 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:24.670 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:24.671 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:24.671 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:24.671 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:24.671 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:24.671 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:24.671 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:24.671 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:24.671 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:24.671 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:24.671 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:24.671 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:24.671 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:24.671 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:24.671 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:24.671 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:24.671 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:24.672 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:24.672 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:24.672 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:24.672 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:24.672 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:24.672 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:24.672 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:24.672 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:24.672 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:24.672 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.672 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:24.672 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:24.672 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:24.672 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:24.672 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:24.672 INFO:tasks.workunit.client.0.smithi100.stdout: 4 2024-04-07T17:51:24.673 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.673 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 1, 2024-04-07T17:51:24.673 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:24.673 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.673 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:24.673 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:24.673 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:24.673 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:24.673 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T17:51:24.673 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.673 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 5, 2024-04-07T17:51:24.673 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:24.673 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:24.673 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T17:51:24.673 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:24.674 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:24.674 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:24.674 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.361600+0000", 2024-04-07T17:51:24.674 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:38.990134+0000", 2024-04-07T17:51:24.674 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:38.990134+0000", 2024-04-07T17:51:24.674 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:38.990885+0000", 2024-04-07T17:51:24.674 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.361600+0000", 2024-04-07T17:51:24.674 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:24.674 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:24.674 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:24.674 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:24.674 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.361600+0000", 2024-04-07T17:51:24.674 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.361600+0000", 2024-04-07T17:51:24.674 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.361600+0000", 2024-04-07T17:51:24.674 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:24.675 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:24.675 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:24.675 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.361600+0000", 2024-04-07T17:51:24.675 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.361600+0000", 2024-04-07T17:51:24.675 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:24.675 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:24.675 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:24.675 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:24.675 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:24.675 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:24.675 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:24.675 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:24.675 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:24.675 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:24.675 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:24.675 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:24.676 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:24.676 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.18", 2024-04-07T17:51:24.676 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:24.676 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:24.676 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:24.676 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4861, 2024-04-07T17:51:24.676 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:24.676 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T19:33:03.967565+0000", 2024-04-07T17:51:24.676 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:24.676 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:24.676 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:24.676 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:24.676 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:24.676 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:24.677 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:24.677 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:24.677 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:24.677 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:24.677 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:24.677 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:24.677 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:24.677 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:24.677 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:24.677 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:24.677 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:24.677 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:24.677 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:24.677 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:24.677 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:24.678 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:24.678 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:24.678 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:24.678 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:24.678 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:24.678 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:24.678 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:24.678 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:24.678 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:24.678 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:24.678 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:24.678 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:24.678 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:24.678 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:24.678 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:24.679 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:24.679 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:24.679 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:24.679 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:24.679 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:24.679 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:24.679 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:24.679 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.679 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:24.679 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:24.679 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:24.679 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:24.679 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:24.679 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T17:51:24.679 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.680 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 5, 2024-04-07T17:51:24.680 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:24.680 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.680 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:24.680 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:24.680 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:24.680 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:24.680 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T17:51:24.680 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.680 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 3, 2024-04-07T17:51:24.680 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:24.680 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:24.680 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:24.680 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:24.680 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:24.681 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:24.681 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:35.975334+0000", 2024-04-07T17:51:24.681 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:23:43.903341+0000", 2024-04-07T17:51:24.681 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:23:43.903341+0000", 2024-04-07T17:51:24.681 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:23:43.904215+0000", 2024-04-07T17:51:24.681 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:35.975334+0000", 2024-04-07T17:51:24.681 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.681 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:24.681 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.681 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 471, 2024-04-07T17:51:24.681 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:35.975334+0000", 2024-04-07T17:51:24.681 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:35.975334+0000", 2024-04-07T17:51:24.681 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:35.975334+0000", 2024-04-07T17:51:24.681 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:24.681 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:24.681 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.682 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:35.975334+0000", 2024-04-07T17:51:24.682 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:35.975334+0000", 2024-04-07T17:51:24.682 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:24.682 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:24.682 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:24.682 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:24.682 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 470, 2024-04-07T17:51:24.682 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:24.682 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:24.682 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:24.682 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:24.682 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:24.682 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:24.682 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:24.682 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:24.682 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.18", 2024-04-07T17:51:24.683 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:24.683 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:24.683 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:24.683 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4969, 2024-04-07T17:51:24.683 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:24.683 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T23:54:23.660433+0000", 2024-04-07T17:51:24.683 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:24.683 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:24.683 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:24.683 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:24.683 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:24.683 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:24.683 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:24.683 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:24.683 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:24.683 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:24.684 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:24.684 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:24.684 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:24.684 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:24.684 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:24.684 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:24.684 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:24.684 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:24.684 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:24.684 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:24.684 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:24.684 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:24.684 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:24.684 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:24.684 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:24.684 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:24.685 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:24.685 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:24.685 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:24.685 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:24.685 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:24.685 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:24.685 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:24.685 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:24.685 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:24.685 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:24.685 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:24.685 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:24.685 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:24.685 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:24.685 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:24.685 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:24.686 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:24.686 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.686 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:24.686 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:24.686 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:24.686 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:24.686 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:24.686 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T17:51:24.686 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.686 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 3, 2024-04-07T17:51:24.686 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:24.686 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.686 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:24.686 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:24.686 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:24.687 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:24.687 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T17:51:24.687 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.687 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 3, 2024-04-07T17:51:24.687 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:24.687 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:24.687 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T17:51:24.687 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:24.687 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:24.687 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:24.687 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:35.977675+0000", 2024-04-07T17:51:24.687 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:08.976003+0000", 2024-04-07T17:51:24.687 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:08.976003+0000", 2024-04-07T17:51:24.687 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:08.976353+0000", 2024-04-07T17:51:24.687 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:35.977675+0000", 2024-04-07T17:51:24.687 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:24.688 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:24.688 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:24.688 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 554, 2024-04-07T17:51:24.688 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:35.977675+0000", 2024-04-07T17:51:24.688 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:35.977675+0000", 2024-04-07T17:51:24.688 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:35.977675+0000", 2024-04-07T17:51:24.688 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:24.688 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:24.688 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:24.688 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:35.977675+0000", 2024-04-07T17:51:24.688 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:35.977675+0000", 2024-04-07T17:51:24.688 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:24.688 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:24.688 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:24.688 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:24.688 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 553, 2024-04-07T17:51:24.689 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:24.689 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:24.689 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:24.689 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:24.689 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:24.689 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:24.689 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:24.689 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:24.689 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.19", 2024-04-07T17:51:24.689 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:24.689 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:24.689 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:24.689 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4937, 2024-04-07T17:51:24.689 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:24.689 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T01:23:38.086617+0000", 2024-04-07T17:51:24.689 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:24.689 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:24.690 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:24.690 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:24.690 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:24.690 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:24.690 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:24.690 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:24.690 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:24.690 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:24.690 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:24.690 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:24.690 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:24.690 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:24.690 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:24.690 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:24.690 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:24.691 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:24.691 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:24.691 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:24.691 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:24.691 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:24.691 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:24.691 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:24.691 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:24.691 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:24.691 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:24.691 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:24.691 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:24.691 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:24.691 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:24.691 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:24.691 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:24.691 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:24.692 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:24.692 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:24.692 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:24.692 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:24.692 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:24.692 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:24.692 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:24.692 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:24.692 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:24.692 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.692 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:24.692 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:24.692 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:24.692 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:24.692 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:24.692 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T17:51:24.693 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.693 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 3, 2024-04-07T17:51:24.693 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:24.693 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.693 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:24.693 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:24.693 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:24.693 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:24.693 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T17:51:24.693 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.693 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 0, 2024-04-07T17:51:24.693 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:24.693 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:24.693 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:24.693 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:24.694 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:24.694 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:24.694 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:35.988644+0000", 2024-04-07T17:51:24.694 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:39.004366+0000", 2024-04-07T17:51:24.694 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:39.004366+0000", 2024-04-07T17:51:24.694 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:39.004810+0000", 2024-04-07T17:51:24.694 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:35.988644+0000", 2024-04-07T17:51:24.694 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.694 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:24.694 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.694 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:24.694 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:35.988644+0000", 2024-04-07T17:51:24.694 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:35.988644+0000", 2024-04-07T17:51:24.694 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:35.988644+0000", 2024-04-07T17:51:24.694 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:24.694 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:24.695 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.695 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:35.988644+0000", 2024-04-07T17:51:24.695 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:35.988644+0000", 2024-04-07T17:51:24.695 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:24.695 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:24.695 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:24.695 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:24.695 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:24.695 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:24.695 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:24.695 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:24.695 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:24.695 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:24.695 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:24.695 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:24.696 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:24.696 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.1b", 2024-04-07T17:51:24.696 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:24.696 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:24.696 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:24.696 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4992, 2024-04-07T17:51:24.696 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:24.696 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T19:31:04.334052+0000", 2024-04-07T17:51:24.696 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:24.696 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:24.696 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:24.696 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:24.696 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:24.696 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:24.696 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:24.697 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:24.697 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:24.697 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:24.697 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:24.697 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:24.697 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:24.697 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:24.697 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:24.697 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:24.697 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:24.697 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:24.697 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:24.697 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:24.697 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:24.697 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:24.698 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:24.698 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:24.698 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:24.698 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:24.698 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:24.698 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:24.698 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:24.698 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:24.698 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:24.698 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:24.698 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:24.698 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:24.698 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:24.698 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:24.698 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:24.698 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:24.699 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:24.699 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:24.699 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:24.699 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:24.699 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:24.699 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.699 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:24.699 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:24.699 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:24.699 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:24.699 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:24.699 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T17:51:24.699 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.699 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 0, 2024-04-07T17:51:24.699 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:24.699 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.699 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:24.700 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:24.700 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:24.700 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:24.700 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T17:51:24.700 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.700 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 6, 2024-04-07T17:51:24.700 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:24.700 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:24.700 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T17:51:24.700 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:24.700 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:24.700 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:24.700 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.372918+0000", 2024-04-07T17:51:24.700 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:38.960251+0000", 2024-04-07T17:51:24.700 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:38.960251+0000", 2024-04-07T17:51:24.700 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:38.960896+0000", 2024-04-07T17:51:24.701 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.372918+0000", 2024-04-07T17:51:24.701 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:24.701 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:24.701 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:24.701 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:24.701 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.372918+0000", 2024-04-07T17:51:24.701 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.372918+0000", 2024-04-07T17:51:24.701 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.372918+0000", 2024-04-07T17:51:24.701 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:24.701 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:24.701 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:24.701 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.372918+0000", 2024-04-07T17:51:24.701 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.372918+0000", 2024-04-07T17:51:24.701 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:24.701 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 2, 2024-04-07T17:51:24.702 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:24.702 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:24.702 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:24.702 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:24.702 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:24.702 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:24.702 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:24.702 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 2, 2024-04-07T17:51:24.702 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:24.702 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:24.702 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:24.702 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.1a", 2024-04-07T17:51:24.702 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:24.702 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:24.702 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:24.702 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4839, 2024-04-07T17:51:24.703 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:24.703 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T18:24:54.929614+0000", 2024-04-07T17:51:24.703 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:24.703 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:24.703 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:24.703 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:24.703 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:24.703 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:24.703 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:24.703 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:24.703 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:24.703 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:24.703 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:24.703 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:24.704 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:24.704 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:24.704 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:24.704 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:24.704 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:24.704 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:24.704 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:24.704 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:24.704 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:24.704 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:24.704 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:24.704 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:24.704 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:24.704 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:24.704 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:24.704 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:24.705 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:24.705 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:24.705 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:24.705 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:24.705 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:24.705 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:24.705 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:24.705 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:24.705 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:24.705 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:24.705 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:24.705 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:24.705 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:24.705 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:24.706 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:24.706 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.706 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:24.706 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:24.706 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:24.706 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:24.706 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:24.706 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T17:51:24.706 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.706 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 6, 2024-04-07T17:51:24.706 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "623'2" 2024-04-07T17:51:24.706 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.706 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:24.706 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:24.706 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:24.707 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:24.707 INFO:tasks.workunit.client.0.smithi100.stdout: 2 2024-04-07T17:51:24.707 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.707 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 4, 2024-04-07T17:51:24.707 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:24.707 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:24.707 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:24.707 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:24.707 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:24.707 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:24.707 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:35.981965+0000", 2024-04-07T17:51:24.707 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:24:13.936610+0000", 2024-04-07T17:51:24.707 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:24:13.936610+0000", 2024-04-07T17:51:24.707 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:24:13.937948+0000", 2024-04-07T17:51:24.707 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:35.981965+0000", 2024-04-07T17:51:24.708 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.708 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:24.708 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.708 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 500, 2024-04-07T17:51:24.708 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:35.981965+0000", 2024-04-07T17:51:24.708 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:35.981965+0000", 2024-04-07T17:51:24.708 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:35.981965+0000", 2024-04-07T17:51:24.708 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:24.708 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:24.708 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.708 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:35.981965+0000", 2024-04-07T17:51:24.708 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:35.981965+0000", 2024-04-07T17:51:24.708 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:24.708 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:24.708 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:24.709 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:24.709 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 499, 2024-04-07T17:51:24.709 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:24.709 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:24.709 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:24.709 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:24.709 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:24.709 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:24.709 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:24.709 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:24.709 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.1a", 2024-04-07T17:51:24.709 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:24.709 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:24.709 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:24.709 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4930, 2024-04-07T17:51:24.709 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:24.710 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T02:25:15.032887+0000", 2024-04-07T17:51:24.710 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:24.710 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:24.710 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:24.710 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:24.710 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:24.710 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:24.710 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:24.710 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:24.710 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:24.710 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:24.710 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:24.710 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:24.710 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:24.710 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:24.710 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:24.711 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:24.711 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:24.711 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:24.711 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:24.711 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:24.711 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:24.711 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:24.711 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:24.711 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:24.711 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:24.711 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:24.711 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:24.711 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:24.711 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:24.711 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:24.711 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:24.711 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:24.712 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:24.712 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:24.712 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:24.712 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:24.712 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:24.712 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:24.712 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:24.712 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:24.712 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:24.712 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:24.712 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:24.712 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.712 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:24.712 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:24.712 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:24.712 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:24.713 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:24.713 INFO:tasks.workunit.client.0.smithi100.stdout: 2 2024-04-07T17:51:24.713 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.713 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 4, 2024-04-07T17:51:24.713 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:24.713 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.713 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:24.713 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:24.713 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:24.713 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:24.713 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T17:51:24.713 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.713 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 3, 2024-04-07T17:51:24.713 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:24.713 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:24.713 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T17:51:24.714 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:24.714 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:24.714 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:24.714 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:35.977526+0000", 2024-04-07T17:51:24.714 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:38.997910+0000", 2024-04-07T17:51:24.714 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:38.997910+0000", 2024-04-07T17:51:24.714 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:38.998434+0000", 2024-04-07T17:51:24.714 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:35.977526+0000", 2024-04-07T17:51:24.714 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:24.714 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:24.714 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:24.714 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:24.714 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:35.977526+0000", 2024-04-07T17:51:24.714 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:35.977526+0000", 2024-04-07T17:51:24.714 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:35.977526+0000", 2024-04-07T17:51:24.714 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:24.714 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:24.715 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:24.715 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:35.977526+0000", 2024-04-07T17:51:24.715 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:35.977526+0000", 2024-04-07T17:51:24.715 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:24.715 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 2, 2024-04-07T17:51:24.715 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:24.715 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:24.715 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:24.715 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:24.715 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:24.715 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:24.715 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:24.715 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 2, 2024-04-07T17:51:24.715 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:24.715 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:24.715 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:24.716 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.1b", 2024-04-07T17:51:24.716 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:24.716 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:24.716 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:24.716 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4932, 2024-04-07T17:51:24.716 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:24.716 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T02:18:46.594583+0000", 2024-04-07T17:51:24.716 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:24.716 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:24.716 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:24.716 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:24.716 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:24.716 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:24.717 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:24.717 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:24.717 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:24.717 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:24.717 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:24.717 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:24.717 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:24.717 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:24.717 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:24.717 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:24.717 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:24.717 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:24.717 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:24.717 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:24.717 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:24.718 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:24.718 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:24.718 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:24.718 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:24.718 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:24.718 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:24.718 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:24.718 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:24.718 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:24.718 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:24.718 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:24.718 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:24.718 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:24.718 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:24.719 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:24.719 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:24.719 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:24.719 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:24.719 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:24.719 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:24.719 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:24.719 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:24.719 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.719 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:24.719 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:24.719 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:24.719 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:24.719 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:24.719 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T17:51:24.719 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.720 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 3, 2024-04-07T17:51:24.720 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "623'2" 2024-04-07T17:51:24.720 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.720 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:24.720 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:24.720 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:24.720 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:24.720 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T17:51:24.720 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.720 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 5, 2024-04-07T17:51:24.720 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:24.720 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:24.720 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:24.720 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:24.720 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:24.720 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:24.721 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.365302+0000", 2024-04-07T17:51:24.721 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:08.974827+0000", 2024-04-07T17:51:24.721 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:08.974827+0000", 2024-04-07T17:51:24.721 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:08.975369+0000", 2024-04-07T17:51:24.721 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.365302+0000", 2024-04-07T17:51:24.721 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.721 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:24.721 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.721 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 554, 2024-04-07T17:51:24.721 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.365302+0000", 2024-04-07T17:51:24.721 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.365302+0000", 2024-04-07T17:51:24.721 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.365302+0000", 2024-04-07T17:51:24.721 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:24.721 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:24.721 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.722 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.365302+0000", 2024-04-07T17:51:24.722 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.365302+0000", 2024-04-07T17:51:24.722 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:24.722 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:24.722 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:24.722 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:24.722 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 553, 2024-04-07T17:51:24.722 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:24.722 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:24.722 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:24.722 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:24.722 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:24.722 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:24.722 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:24.722 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:24.723 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.1d", 2024-04-07T17:51:24.723 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:24.723 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:24.723 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:24.723 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4868, 2024-04-07T17:51:24.723 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:24.723 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T19:16:10.778146+0000", 2024-04-07T17:51:24.723 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:24.723 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:24.723 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:24.723 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:24.723 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:24.723 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:24.723 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:24.723 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:24.724 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:24.724 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:24.724 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:24.724 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:24.724 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:24.724 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:24.724 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:24.724 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:24.724 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:24.724 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:24.724 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:24.724 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:24.724 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:24.724 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:24.724 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:24.725 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:24.725 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:24.725 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:24.725 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:24.725 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:24.725 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:24.725 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:24.725 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:24.725 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:24.725 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:24.725 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:24.725 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:24.725 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:24.725 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:24.725 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:24.725 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:24.726 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:24.726 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:24.726 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:24.726 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:24.726 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.726 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:24.726 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:24.726 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:24.726 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:24.726 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:24.726 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T17:51:24.726 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.726 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 5, 2024-04-07T17:51:24.726 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:24.726 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.727 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:24.727 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:24.727 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:24.727 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:24.727 INFO:tasks.workunit.client.0.smithi100.stdout: 2 2024-04-07T17:51:24.727 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.727 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 7, 2024-04-07T17:51:24.727 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:24.727 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:24.727 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T17:51:24.727 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:24.727 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:24.727 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:24.727 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.364942+0000", 2024-04-07T17:51:24.727 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:38.975920+0000", 2024-04-07T17:51:24.728 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:38.975920+0000", 2024-04-07T17:51:24.728 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:38.991127+0000", 2024-04-07T17:51:24.728 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.364942+0000", 2024-04-07T17:51:24.728 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:24.728 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:24.728 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:24.728 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:24.728 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.364942+0000", 2024-04-07T17:51:24.728 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.364942+0000", 2024-04-07T17:51:24.728 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.364942+0000", 2024-04-07T17:51:24.728 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:24.728 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:24.728 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:24.728 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.364942+0000", 2024-04-07T17:51:24.728 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.364942+0000", 2024-04-07T17:51:24.729 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:24.729 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 2, 2024-04-07T17:51:24.729 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:24.729 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:24.729 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:24.729 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:24.729 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:24.729 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:24.729 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:24.729 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 2, 2024-04-07T17:51:24.729 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:24.729 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:24.729 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:24.729 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.1c", 2024-04-07T17:51:24.729 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:24.729 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:24.729 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:24.730 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4819, 2024-04-07T17:51:24.730 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:24.730 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T22:48:18.986526+0000", 2024-04-07T17:51:24.730 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:24.730 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:24.730 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:24.730 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:24.730 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:24.730 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:24.730 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:24.730 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:24.730 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:24.730 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:24.731 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:24.731 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:24.731 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:24.731 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:24.731 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:24.731 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:24.731 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:24.731 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:24.731 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:24.731 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 3, 2024-04-07T17:51:24.731 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 1, 2024-04-07T17:51:24.731 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:24.731 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 1, 2024-04-07T17:51:24.731 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:24.731 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:24.731 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:24.732 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:24.732 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:24.732 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 1, 2024-04-07T17:51:24.732 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:24.732 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:24.732 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:24.732 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:24.732 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:24.732 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:24.732 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:24.732 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 6, 2024-04-07T17:51:24.732 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 6, 2024-04-07T17:51:24.732 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:24.732 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:24.732 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:24.733 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 3, 2024-04-07T17:51:24.733 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:24.733 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.733 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:24.733 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:24.733 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:24.733 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:24.733 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:24.733 INFO:tasks.workunit.client.0.smithi100.stdout: 2 2024-04-07T17:51:24.733 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.733 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 7, 2024-04-07T17:51:24.733 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "61'2" 2024-04-07T17:51:24.733 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.733 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:24.733 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:24.734 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:24.734 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:24.734 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T17:51:24.734 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.734 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 5, 2024-04-07T17:51:24.734 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:24.734 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:24.734 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:24.734 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:24.734 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:24.734 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:24.734 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.367080+0000", 2024-04-07T17:51:24.734 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:24:41.542687+0000", 2024-04-07T17:51:24.734 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:24:41.542687+0000", 2024-04-07T17:51:24.734 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:24:41.543356+0000", 2024-04-07T17:51:24.735 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.367080+0000", 2024-04-07T17:51:24.735 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.735 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:24.735 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.735 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 527, 2024-04-07T17:51:24.735 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.367080+0000", 2024-04-07T17:51:24.735 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.367080+0000", 2024-04-07T17:51:24.735 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.367080+0000", 2024-04-07T17:51:24.735 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:24.735 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:24.735 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.735 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.367080+0000", 2024-04-07T17:51:24.735 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.367080+0000", 2024-04-07T17:51:24.735 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:24.735 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:24.735 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:24.736 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:24.736 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 526, 2024-04-07T17:51:24.736 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:24.736 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:24.736 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:24.736 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:24.736 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:24.736 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:24.736 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:24.736 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:24.736 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.1c", 2024-04-07T17:51:24.736 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:24.736 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:24.736 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:24.736 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4865, 2024-04-07T17:51:24.737 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:24.737 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T21:43:40.340691+0000", 2024-04-07T17:51:24.737 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:24.737 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:24.737 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:24.737 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:24.737 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:24.737 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:24.737 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:24.737 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:24.737 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:24.737 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:24.737 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:24.737 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:24.737 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:24.738 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:24.738 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:24.738 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:24.738 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:24.738 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:24.738 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:24.738 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:24.738 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:24.738 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:24.738 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:24.738 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:24.738 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:24.738 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:24.738 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:24.738 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:24.739 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:24.739 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:24.739 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:24.739 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:24.739 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:24.739 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:24.739 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:24.739 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:24.739 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:24.739 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:24.739 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:24.739 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:24.739 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:24.739 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:24.739 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:24.739 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.740 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:24.740 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:24.740 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:24.740 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:24.740 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:24.740 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T17:51:24.740 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.740 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 5, 2024-04-07T17:51:24.740 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:24.740 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.740 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:24.740 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:24.740 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:24.740 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:24.740 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T17:51:24.741 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.741 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 7, 2024-04-07T17:51:24.741 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:24.741 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:24.741 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T17:51:24.741 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:24.741 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:24.741 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:24.741 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.373509+0000", 2024-04-07T17:51:24.741 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:39.001898+0000", 2024-04-07T17:51:24.741 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:39.001898+0000", 2024-04-07T17:51:24.741 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:39.004508+0000", 2024-04-07T17:51:24.741 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.373509+0000", 2024-04-07T17:51:24.741 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:24.741 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:24.741 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:24.742 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:24.742 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.373509+0000", 2024-04-07T17:51:24.742 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.373509+0000", 2024-04-07T17:51:24.742 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.373509+0000", 2024-04-07T17:51:24.742 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:24.742 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:24.742 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:24.742 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.373509+0000", 2024-04-07T17:51:24.742 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.373509+0000", 2024-04-07T17:51:24.742 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:24.742 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 2, 2024-04-07T17:51:24.742 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:24.742 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:24.742 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:24.743 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:24.743 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:24.743 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:24.743 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:24.743 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 2, 2024-04-07T17:51:24.743 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:24.743 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:24.743 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:24.743 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.1d", 2024-04-07T17:51:24.743 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:24.743 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:24.743 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:24.743 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4804, 2024-04-07T17:51:24.743 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:24.743 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T01:47:40.701645+0000", 2024-04-07T17:51:24.743 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:24.743 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:24.744 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:24.744 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:24.744 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:24.744 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:24.744 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:24.744 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:24.744 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:24.744 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:24.744 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:24.744 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:24.744 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:24.744 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:24.744 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:24.744 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:24.744 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:24.745 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:24.745 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:24.745 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 3, 2024-04-07T17:51:24.745 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 1, 2024-04-07T17:51:24.745 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:24.745 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 1, 2024-04-07T17:51:24.745 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:24.745 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:24.745 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:24.745 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:24.745 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:24.745 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 1, 2024-04-07T17:51:24.745 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:24.745 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:24.745 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:24.745 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:24.746 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:24.746 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:24.746 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:24.746 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 6, 2024-04-07T17:51:24.746 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 6, 2024-04-07T17:51:24.746 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:24.746 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:24.746 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:24.746 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 3, 2024-04-07T17:51:24.746 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:24.746 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.746 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:24.746 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:24.746 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:24.746 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:24.747 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:24.747 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T17:51:24.747 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.747 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 7, 2024-04-07T17:51:24.747 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "61'2" 2024-04-07T17:51:24.747 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.747 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:24.747 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:24.747 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:24.747 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:24.747 INFO:tasks.workunit.client.0.smithi100.stdout: 2 2024-04-07T17:51:24.747 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.747 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 0, 2024-04-07T17:51:24.747 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:24.747 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:24.748 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:24.748 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:24.748 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:24.748 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:24.748 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:35.983257+0000", 2024-04-07T17:51:24.748 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:24:41.564700+0000", 2024-04-07T17:51:24.748 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:24:41.564700+0000", 2024-04-07T17:51:24.748 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:24:41.565093+0000", 2024-04-07T17:51:24.748 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:35.983257+0000", 2024-04-07T17:51:24.748 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.748 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:24.748 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.748 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 527, 2024-04-07T17:51:24.748 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:35.983257+0000", 2024-04-07T17:51:24.748 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:35.983257+0000", 2024-04-07T17:51:24.748 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:35.983257+0000", 2024-04-07T17:51:24.749 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:24.749 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:24.749 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.749 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:35.983257+0000", 2024-04-07T17:51:24.749 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:35.983257+0000", 2024-04-07T17:51:24.749 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:24.749 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:24.749 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:24.749 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:24.749 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 526, 2024-04-07T17:51:24.749 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:24.749 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:24.749 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:24.749 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:24.749 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:24.749 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:24.750 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:24.750 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:24.750 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.1f", 2024-04-07T17:51:24.750 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:24.750 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:24.750 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:24.750 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 5000, 2024-04-07T17:51:24.750 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:24.750 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T01:20:29.103128+0000", 2024-04-07T17:51:24.750 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:24.750 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:24.750 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:24.750 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:24.750 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:24.751 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:24.751 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:24.751 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:24.751 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:24.751 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:24.751 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:24.751 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:24.751 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:24.751 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:24.751 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:24.751 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:24.751 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:24.751 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:24.751 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:24.751 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:24.751 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:24.752 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:24.752 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:24.752 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:24.752 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:24.752 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:24.752 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:24.752 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:24.752 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:24.752 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:24.752 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:24.752 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:24.753 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:24.753 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:24.753 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:24.753 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:24.753 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:24.753 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:24.753 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:24.753 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:24.754 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:24.754 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:24.754 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:24.754 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.754 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:24.754 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:24.754 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:24.754 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:24.754 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:24.754 INFO:tasks.workunit.client.0.smithi100.stdout: 2 2024-04-07T17:51:24.754 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.754 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 0, 2024-04-07T17:51:24.754 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:24.754 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.754 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:24.755 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:24.755 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:24.755 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:24.755 INFO:tasks.workunit.client.0.smithi100.stdout: 5 2024-04-07T17:51:24.755 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.755 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 3, 2024-04-07T17:51:24.755 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:24.755 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:24.755 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T17:51:24.755 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:24.755 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:24.755 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:24.755 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:35.982794+0000", 2024-04-07T17:51:24.755 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:24:41.546310+0000", 2024-04-07T17:51:24.755 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:24:41.546310+0000", 2024-04-07T17:51:24.756 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:24:41.546952+0000", 2024-04-07T17:51:24.756 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:35.982794+0000", 2024-04-07T17:51:24.756 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:24.756 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:24.756 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:24.756 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 527, 2024-04-07T17:51:24.756 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:35.982794+0000", 2024-04-07T17:51:24.756 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:35.982794+0000", 2024-04-07T17:51:24.756 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:35.982794+0000", 2024-04-07T17:51:24.756 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:24.756 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:24.756 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:24.756 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:35.982794+0000", 2024-04-07T17:51:24.756 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:35.982794+0000", 2024-04-07T17:51:24.756 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:24.757 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 2, 2024-04-07T17:51:24.757 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:24.757 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:24.757 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 526, 2024-04-07T17:51:24.757 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:24.757 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:24.757 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:24.757 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:24.757 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 2, 2024-04-07T17:51:24.757 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:24.757 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:24.757 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:24.757 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.1e", 2024-04-07T17:51:24.757 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:24.758 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:24.758 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:24.758 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4947, 2024-04-07T17:51:24.758 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:24.758 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T21:45:55.182027+0000", 2024-04-07T17:51:24.758 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:24.758 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:24.758 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:24.758 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:24.758 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:24.758 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:24.758 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:24.758 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:24.758 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:24.758 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:24.759 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:24.759 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:24.759 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:24.759 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:24.759 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:24.759 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:24.759 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:24.759 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:24.759 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:24.759 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:24.759 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:24.759 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:24.759 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:24.759 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:24.759 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:24.760 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:24.760 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:24.760 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:24.760 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:24.760 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:24.760 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:24.760 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:24.760 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:24.760 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:24.760 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:24.760 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:24.760 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:24.760 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:24.760 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:24.760 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:24.761 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:24.761 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:24.761 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:24.761 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.761 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:24.761 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:24.761 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:24.761 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:24.761 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:24.761 INFO:tasks.workunit.client.0.smithi100.stdout: 5 2024-04-07T17:51:24.761 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.761 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 3, 2024-04-07T17:51:24.761 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "623'2" 2024-04-07T17:51:24.761 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.761 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:24.761 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:24.762 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:24.762 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:24.762 INFO:tasks.workunit.client.0.smithi100.stdout: 2 2024-04-07T17:51:24.762 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.762 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 3, 2024-04-07T17:51:24.762 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:24.762 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:24.762 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:24.762 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:24.762 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:24.762 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:24.762 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:35.982633+0000", 2024-04-07T17:51:24.762 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:08.977988+0000", 2024-04-07T17:51:24.762 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:08.977988+0000", 2024-04-07T17:51:24.763 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:08.978435+0000", 2024-04-07T17:51:24.763 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:35.982633+0000", 2024-04-07T17:51:24.763 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.763 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:24.763 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.763 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 554, 2024-04-07T17:51:24.763 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:35.982633+0000", 2024-04-07T17:51:24.763 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:35.982633+0000", 2024-04-07T17:51:24.763 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:35.982633+0000", 2024-04-07T17:51:24.763 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:24.763 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:24.763 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.763 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:35.982633+0000", 2024-04-07T17:51:24.763 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:35.982633+0000", 2024-04-07T17:51:24.764 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:24.764 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:24.764 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:24.764 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:24.764 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 553, 2024-04-07T17:51:24.764 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:24.764 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:24.764 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:24.764 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:24.764 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:24.764 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:24.764 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:24.764 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:24.764 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.1e", 2024-04-07T17:51:24.764 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:24.764 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:24.764 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:24.765 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4957, 2024-04-07T17:51:24.765 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:24.765 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T21:46:59.340658+0000", 2024-04-07T17:51:24.765 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:24.765 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:24.765 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:24.765 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:24.765 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:24.765 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:24.765 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:24.765 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:24.765 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:24.765 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:24.765 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:24.766 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:24.766 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:24.766 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:24.766 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:24.766 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:24.766 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:24.766 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:24.766 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:24.766 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:24.766 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:24.766 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:24.766 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:24.766 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:24.766 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:24.766 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:24.767 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:24.767 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:24.767 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:24.767 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:24.767 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:24.767 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:24.767 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:24.767 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:24.767 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:24.767 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:24.767 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:24.767 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:24.767 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:24.767 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:24.767 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:24.768 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:24.768 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:24.768 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.768 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:24.768 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:24.768 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:24.768 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:24.768 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:24.768 INFO:tasks.workunit.client.0.smithi100.stdout: 2 2024-04-07T17:51:24.768 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.768 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 3, 2024-04-07T17:51:24.768 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:24.768 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.768 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:24.769 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:24.769 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:24.769 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:24.769 INFO:tasks.workunit.client.0.smithi100.stdout: 4 2024-04-07T17:51:24.769 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.769 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 0, 2024-04-07T17:51:24.769 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:24.769 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:24.769 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T17:51:24.769 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:24.769 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:24.769 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:24.769 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:35.984435+0000", 2024-04-07T17:51:24.769 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:38.997317+0000", 2024-04-07T17:51:24.769 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:38.997317+0000", 2024-04-07T17:51:24.770 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:38.997712+0000", 2024-04-07T17:51:24.770 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:35.984435+0000", 2024-04-07T17:51:24.770 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:24.770 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:24.770 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:24.770 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:24.770 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:35.984435+0000", 2024-04-07T17:51:24.770 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:35.984435+0000", 2024-04-07T17:51:24.770 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:35.984435+0000", 2024-04-07T17:51:24.770 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:24.770 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:24.770 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:24.770 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:35.984435+0000", 2024-04-07T17:51:24.770 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:35.984435+0000", 2024-04-07T17:51:24.771 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:24.771 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 5, 2024-04-07T17:51:24.771 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:24.771 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:24.771 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:24.771 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:24.771 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:24.771 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:24.771 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:24.771 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 5, 2024-04-07T17:51:24.771 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:24.771 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:24.771 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:24.771 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.1f", 2024-04-07T17:51:24.771 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:24.771 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:24.772 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:24.772 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 5008, 2024-04-07T17:51:24.772 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:24.772 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T18:23:18.184011+0000", 2024-04-07T17:51:24.772 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:24.772 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:24.772 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:24.772 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 582, 2024-04-07T17:51:24.772 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:24.772 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:24.772 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:24.772 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:24.772 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:24.772 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:24.772 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:24.773 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:24.773 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:24.773 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:24.773 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:24.773 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:24.773 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:24.773 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:24.773 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:24.773 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 6, 2024-04-07T17:51:24.773 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 2, 2024-04-07T17:51:24.773 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:24.773 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 2, 2024-04-07T17:51:24.773 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:24.773 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:24.773 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:24.774 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:24.774 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:24.774 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 1, 2024-04-07T17:51:24.774 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:24.774 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:24.774 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:24.774 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:24.774 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:24.774 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:24.774 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:24.774 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 4, 2024-04-07T17:51:24.774 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 4, 2024-04-07T17:51:24.774 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:24.774 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:24.775 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:24.775 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 4, 2024-04-07T17:51:24.775 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 1 2024-04-07T17:51:24.775 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.775 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:24.775 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:24.775 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:24.775 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:24.775 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:24.775 INFO:tasks.workunit.client.0.smithi100.stdout: 4 2024-04-07T17:51:24.775 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.775 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 0, 2024-04-07T17:51:24.775 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "623'5" 2024-04-07T17:51:24.775 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.775 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:24.776 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:24.776 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:24.776 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T17:51:24.776 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T17:51:24.776 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.776 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 7, 2024-04-07T17:51:24.776 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:24.776 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:24.776 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:24.776 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:24.776 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:24.776 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:24.776 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.373284+0000", 2024-04-07T17:51:24.776 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:38.996053+0000", 2024-04-07T17:51:24.776 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:38.996053+0000", 2024-04-07T17:51:24.777 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:39.002704+0000", 2024-04-07T17:51:24.777 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.373284+0000", 2024-04-07T17:51:24.777 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.777 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:24.777 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.777 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:24.777 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.373284+0000", 2024-04-07T17:51:24.777 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.373284+0000", 2024-04-07T17:51:24.777 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.373284+0000", 2024-04-07T17:51:24.777 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:24.777 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:24.777 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.777 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.373284+0000", 2024-04-07T17:51:24.777 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.373284+0000", 2024-04-07T17:51:24.777 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:24.778 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:24.778 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:24.778 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:24.778 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:24.778 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:24.778 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:24.778 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:24.778 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:24.778 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:24.778 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:24.778 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:24.778 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:24.778 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.2d", 2024-04-07T17:51:24.778 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:24.779 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:24.779 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:24.779 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4802, 2024-04-07T17:51:24.779 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:24.779 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T17:45:19.934847+0000", 2024-04-07T17:51:24.779 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:24.779 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:24.779 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:24.779 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:24.779 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:24.779 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:24.779 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:24.779 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:24.779 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:24.779 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:24.780 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:24.780 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:24.780 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:24.780 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:24.780 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:24.780 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:24.780 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:24.780 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:24.780 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:24.780 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:24.780 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:24.780 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:24.780 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:24.780 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:24.780 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:24.780 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:24.781 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:24.781 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:24.781 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:24.781 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:24.781 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:24.781 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:24.781 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:24.781 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:24.781 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:24.781 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:24.781 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:24.781 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:24.781 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:24.781 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:24.781 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:24.781 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:24.782 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:24.782 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.782 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:24.782 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:24.782 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:24.782 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:24.782 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T17:51:24.782 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T17:51:24.782 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.782 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 7, 2024-04-07T17:51:24.782 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:24.782 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.782 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:24.782 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:24.782 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T17:51:24.782 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:24.783 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T17:51:24.783 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.783 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 2, 2024-04-07T17:51:24.783 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:24.783 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:24.783 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:24.783 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:24.783 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:24.783 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:24.783 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.359468+0000", 2024-04-07T17:51:24.783 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:38.967369+0000", 2024-04-07T17:51:24.783 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:38.967369+0000", 2024-04-07T17:51:24.783 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:38.967737+0000", 2024-04-07T17:51:24.783 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.359468+0000", 2024-04-07T17:51:24.784 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.784 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:24.784 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.784 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:24.784 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.359468+0000", 2024-04-07T17:51:24.784 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.359468+0000", 2024-04-07T17:51:24.784 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.359468+0000", 2024-04-07T17:51:24.784 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:24.784 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:24.784 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.784 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.359468+0000", 2024-04-07T17:51:24.784 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.359468+0000", 2024-04-07T17:51:24.784 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:24.784 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:24.784 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:24.784 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:24.785 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:24.785 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:24.785 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:24.785 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:24.785 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:24.785 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:24.785 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:24.785 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:24.785 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:24.785 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.2e", 2024-04-07T17:51:24.785 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:24.785 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:24.785 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:24.785 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4964, 2024-04-07T17:51:24.785 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:24.785 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T22:15:36.602436+0000", 2024-04-07T17:51:24.786 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:24.786 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:24.786 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:24.786 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:24.786 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:24.786 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:24.786 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:24.786 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:24.786 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:24.786 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:24.786 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:24.786 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:24.786 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:24.786 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:24.787 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:24.787 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:24.787 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:24.787 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:24.787 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:24.787 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:24.787 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:24.787 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:24.787 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:24.787 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:24.787 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:24.787 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:24.787 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:24.787 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:24.787 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:24.788 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:24.788 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:24.788 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:24.788 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:24.788 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:24.788 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:24.788 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:24.788 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:24.788 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:24.788 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:24.788 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:24.788 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:24.788 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:24.788 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:24.789 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.789 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:24.789 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:24.789 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:24.789 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T17:51:24.789 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:24.789 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T17:51:24.789 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.789 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 2, 2024-04-07T17:51:24.789 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:24.789 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.789 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:24.789 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:24.789 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:24.789 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:24.790 INFO:tasks.workunit.client.0.smithi100.stdout: 4 2024-04-07T17:51:24.790 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.790 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 5, 2024-04-07T17:51:24.790 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:24.790 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:24.790 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:24.790 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:24.790 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:24.790 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:24.790 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.361813+0000", 2024-04-07T17:51:24.790 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:38.971115+0000", 2024-04-07T17:51:24.790 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:38.971115+0000", 2024-04-07T17:51:24.790 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:38.971858+0000", 2024-04-07T17:51:24.790 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.361813+0000", 2024-04-07T17:51:24.791 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.791 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:24.791 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.791 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:24.791 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.361813+0000", 2024-04-07T17:51:24.791 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.361813+0000", 2024-04-07T17:51:24.791 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.361813+0000", 2024-04-07T17:51:24.791 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:24.791 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:24.791 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.791 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.361813+0000", 2024-04-07T17:51:24.791 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.361813+0000", 2024-04-07T17:51:24.791 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:24.791 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:24.791 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:24.792 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:24.792 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:24.792 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:24.792 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:24.792 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:24.792 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:24.792 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:24.792 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:24.792 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:24.792 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:24.792 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.2f", 2024-04-07T17:51:24.792 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:24.792 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:24.792 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:24.792 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4863, 2024-04-07T17:51:24.793 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:24.793 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T22:01:10.276598+0000", 2024-04-07T17:51:24.793 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:24.793 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:24.793 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:24.793 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:24.793 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:24.793 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:24.793 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:24.793 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:24.793 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:24.793 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:24.793 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:24.793 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:24.793 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:24.794 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:24.794 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:24.794 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:24.794 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:24.794 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:24.794 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:24.794 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:24.794 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:24.794 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:24.794 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:24.794 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:24.794 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:24.794 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:24.794 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:24.795 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:24.795 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:24.795 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:24.795 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:24.795 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:24.795 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:24.795 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:24.795 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:24.795 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:24.795 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:24.795 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:24.795 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:24.795 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:24.795 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:24.795 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:24.795 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:24.796 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.796 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:24.796 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:24.796 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:24.796 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:24.796 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:24.796 INFO:tasks.workunit.client.0.smithi100.stdout: 4 2024-04-07T17:51:24.796 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.796 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 5, 2024-04-07T17:51:24.796 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:24.796 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.796 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:24.796 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:24.796 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:24.796 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:24.797 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T17:51:24.797 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.797 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 5, 2024-04-07T17:51:24.797 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:24.797 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:24.797 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:24.797 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:24.797 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:24.797 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:24.797 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.364596+0000", 2024-04-07T17:51:24.797 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:39.005541+0000", 2024-04-07T17:51:24.797 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:39.005541+0000", 2024-04-07T17:51:24.797 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:39.006138+0000", 2024-04-07T17:51:24.797 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.364596+0000", 2024-04-07T17:51:24.797 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.798 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:24.798 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.798 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:24.798 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.364596+0000", 2024-04-07T17:51:24.798 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.364596+0000", 2024-04-07T17:51:24.798 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.364596+0000", 2024-04-07T17:51:24.798 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:24.798 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:24.798 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.798 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.364596+0000", 2024-04-07T17:51:24.798 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.364596+0000", 2024-04-07T17:51:24.798 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:24.798 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:24.798 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:24.798 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:24.799 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:24.799 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:24.799 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:24.799 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:24.799 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:24.799 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:24.799 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:24.799 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:24.799 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:24.799 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.30", 2024-04-07T17:51:24.799 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:24.799 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:24.799 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:24.799 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4876, 2024-04-07T17:51:24.799 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:24.799 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T20:40:58.093262+0000", 2024-04-07T17:51:24.800 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:24.800 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:24.800 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:24.800 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:24.800 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:24.800 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:24.800 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:24.800 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:24.800 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:24.800 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:24.800 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:24.800 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:24.800 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:24.800 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:24.800 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:24.801 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:24.801 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:24.801 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:24.801 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:24.801 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:24.801 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:24.801 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:24.801 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:24.801 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:24.801 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:24.801 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:24.801 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:24.801 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:24.801 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:24.801 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:24.802 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:24.802 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:24.802 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:24.802 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:24.802 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:24.802 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:24.802 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:24.802 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:24.802 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:24.802 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:24.802 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:24.802 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:24.802 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:24.802 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.802 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:24.803 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:24.803 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:24.803 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:24.803 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:24.803 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T17:51:24.803 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.803 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 5, 2024-04-07T17:51:24.803 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:24.803 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.803 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:24.803 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:24.803 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:24.803 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:24.803 INFO:tasks.workunit.client.0.smithi100.stdout: 5 2024-04-07T17:51:24.803 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.803 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 1, 2024-04-07T17:51:24.804 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:24.804 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:24.804 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:24.804 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:24.804 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:24.804 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:24.804 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.331123+0000", 2024-04-07T17:51:24.804 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:24:41.533626+0000", 2024-04-07T17:51:24.804 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:24:41.533626+0000", 2024-04-07T17:51:24.804 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:24:41.534090+0000", 2024-04-07T17:51:24.804 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.331123+0000", 2024-04-07T17:51:24.804 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.804 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:24.804 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.804 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 527, 2024-04-07T17:51:24.805 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.331123+0000", 2024-04-07T17:51:24.805 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.331123+0000", 2024-04-07T17:51:24.805 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.331123+0000", 2024-04-07T17:51:24.805 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:24.805 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:24.805 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.805 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.331123+0000", 2024-04-07T17:51:24.805 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.331123+0000", 2024-04-07T17:51:24.805 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:24.805 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:24.805 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:24.805 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:24.805 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 526, 2024-04-07T17:51:24.805 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:24.805 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:24.806 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:24.806 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:24.806 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:24.806 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:24.806 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:24.806 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:24.806 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.31", 2024-04-07T17:51:24.806 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:24.806 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:24.806 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:24.806 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4978, 2024-04-07T17:51:24.806 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:24.806 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T16:42:51.811301+0000", 2024-04-07T17:51:24.806 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:24.806 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:24.807 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:24.807 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:24.807 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:24.807 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:24.807 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:24.807 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:24.807 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:24.807 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:24.807 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:24.807 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:24.807 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:24.807 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:24.807 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:24.807 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:24.807 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:24.808 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:24.808 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:24.808 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:24.808 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:24.808 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:24.808 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:24.808 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:24.808 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:24.808 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:24.808 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:24.808 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:24.808 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:24.808 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:24.808 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:24.808 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:24.809 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:24.809 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:24.809 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:24.809 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:24.809 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:24.809 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:24.809 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:24.809 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:24.809 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:24.809 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:24.809 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:24.809 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.809 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:24.809 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:24.809 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:24.809 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:24.810 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:24.810 INFO:tasks.workunit.client.0.smithi100.stdout: 5 2024-04-07T17:51:24.810 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.810 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 1, 2024-04-07T17:51:24.810 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:24.810 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.810 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:24.810 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:24.810 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:24.810 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:24.810 INFO:tasks.workunit.client.0.smithi100.stdout: 5 2024-04-07T17:51:24.810 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.810 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 0, 2024-04-07T17:51:24.810 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:24.811 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:24.811 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:24.811 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:24.811 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:24.811 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:24.811 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:35.984291+0000", 2024-04-07T17:51:24.811 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:24:41.538233+0000", 2024-04-07T17:51:24.811 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:24:41.538233+0000", 2024-04-07T17:51:24.811 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:24:41.538718+0000", 2024-04-07T17:51:24.811 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:35.984291+0000", 2024-04-07T17:51:24.811 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.811 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:24.811 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.811 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 527, 2024-04-07T17:51:24.811 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:35.984291+0000", 2024-04-07T17:51:24.811 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:35.984291+0000", 2024-04-07T17:51:24.812 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:35.984291+0000", 2024-04-07T17:51:24.812 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:24.812 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:24.812 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.812 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:35.984291+0000", 2024-04-07T17:51:24.812 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:35.984291+0000", 2024-04-07T17:51:24.812 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:24.812 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:24.812 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:24.812 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:24.812 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 526, 2024-04-07T17:51:24.812 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:24.812 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:24.812 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:24.812 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:24.812 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:24.812 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:24.813 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:24.813 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:24.813 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.32", 2024-04-07T17:51:24.813 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:24.813 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:24.813 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:24.813 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4994, 2024-04-07T17:51:24.813 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:24.813 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T22:18:55.330487+0000", 2024-04-07T17:51:24.813 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:24.813 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:24.813 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:24.813 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:24.813 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:24.813 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:24.813 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:24.814 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:24.814 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:24.814 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:24.814 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:24.814 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:24.814 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:24.814 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:24.814 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:24.814 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:24.814 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:24.814 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:24.814 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:24.814 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:24.814 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:24.814 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:24.815 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:24.815 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:24.815 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:24.815 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:24.815 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:24.815 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:24.815 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:24.815 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:24.815 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:24.815 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:24.815 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:24.815 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:24.815 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:24.815 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:24.815 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:24.816 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:24.816 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:24.816 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:24.816 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:24.816 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:24.816 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:24.816 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.816 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:24.816 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:24.816 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:24.816 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:24.816 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:24.816 INFO:tasks.workunit.client.0.smithi100.stdout: 5 2024-04-07T17:51:24.816 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.816 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 0, 2024-04-07T17:51:24.817 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:24.817 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.817 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:24.817 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:24.817 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:24.817 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:24.817 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T17:51:24.817 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.817 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 3, 2024-04-07T17:51:24.817 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:24.817 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:24.817 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:24.817 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:24.817 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:24.817 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:24.817 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:35.977234+0000", 2024-04-07T17:51:24.818 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:08.976145+0000", 2024-04-07T17:51:24.818 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:08.976145+0000", 2024-04-07T17:51:24.818 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:08.976499+0000", 2024-04-07T17:51:24.818 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:35.977234+0000", 2024-04-07T17:51:24.818 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.818 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:24.818 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.818 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 554, 2024-04-07T17:51:24.818 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:35.977234+0000", 2024-04-07T17:51:24.818 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:35.977234+0000", 2024-04-07T17:51:24.818 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:35.977234+0000", 2024-04-07T17:51:24.818 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:24.818 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:24.818 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.819 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:35.977234+0000", 2024-04-07T17:51:24.819 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:35.977234+0000", 2024-04-07T17:51:24.819 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:24.819 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:24.819 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:24.819 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:24.819 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 553, 2024-04-07T17:51:24.819 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:24.819 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:24.819 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:24.819 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:24.819 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:24.819 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:24.819 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:24.819 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:24.819 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.33", 2024-04-07T17:51:24.820 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:24.820 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:24.820 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:24.820 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4963, 2024-04-07T17:51:24.820 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:24.820 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T04:02:25.054838+0000", 2024-04-07T17:51:24.820 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:24.820 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:24.820 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:24.820 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:24.820 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:24.820 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:24.820 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:24.820 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:24.820 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:24.821 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:24.821 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:24.821 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:24.821 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:24.821 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:24.821 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:24.821 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:24.821 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:24.821 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:24.821 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:24.821 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:24.821 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:24.821 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:24.821 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:24.821 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:24.822 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:24.822 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:24.822 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:24.822 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:24.822 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:24.822 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:24.822 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:24.822 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:24.822 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:24.822 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:24.822 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:24.822 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:24.822 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:24.822 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:24.822 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:24.823 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:24.823 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:24.823 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:24.823 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:24.823 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.823 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:24.823 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:24.823 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:24.823 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:24.823 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:24.823 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T17:51:24.823 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.823 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 3, 2024-04-07T17:51:24.823 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:24.823 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.823 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:24.824 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:24.824 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:24.824 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:24.824 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T17:51:24.824 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.824 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 4, 2024-04-07T17:51:24.824 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:24.824 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:24.824 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:24.824 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:24.824 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:24.824 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:24.824 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:35.980246+0000", 2024-04-07T17:51:24.824 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:08.955654+0000", 2024-04-07T17:51:24.824 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:08.955654+0000", 2024-04-07T17:51:24.824 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:08.956220+0000", 2024-04-07T17:51:24.824 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:35.980246+0000", 2024-04-07T17:51:24.825 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.825 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:24.825 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.825 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 554, 2024-04-07T17:51:24.825 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:35.980246+0000", 2024-04-07T17:51:24.825 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:35.980246+0000", 2024-04-07T17:51:24.825 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:35.980246+0000", 2024-04-07T17:51:24.825 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:24.825 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:24.825 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.825 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:35.980246+0000", 2024-04-07T17:51:24.825 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:35.980246+0000", 2024-04-07T17:51:24.825 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:24.825 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:24.825 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:24.826 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:24.826 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 553, 2024-04-07T17:51:24.826 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:24.826 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:24.826 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:24.826 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:24.826 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:24.826 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:24.826 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:24.826 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:24.826 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.34", 2024-04-07T17:51:24.826 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:24.826 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:24.826 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:24.827 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4913, 2024-04-07T17:51:24.827 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:24.827 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T00:20:14.081274+0000", 2024-04-07T17:51:24.827 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:24.827 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:24.827 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:24.827 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:24.827 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:24.827 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:24.827 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:24.827 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:24.827 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:24.827 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:24.827 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:24.827 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:24.827 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:24.828 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:24.828 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:24.828 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:24.828 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:24.828 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:24.828 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:24.828 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:24.828 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:24.828 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:24.828 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:24.828 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:24.828 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:24.829 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:24.829 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:24.829 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:24.829 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:24.829 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:24.829 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:24.829 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:24.829 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:24.829 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:24.829 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:24.829 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:24.829 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:24.829 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:24.829 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:24.830 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:24.830 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:24.830 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:24.830 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:24.830 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.830 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:24.830 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:24.830 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:24.830 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:24.830 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:24.830 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T17:51:24.830 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.830 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 4, 2024-04-07T17:51:24.831 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:24.831 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.831 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:24.831 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:24.831 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:24.831 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:24.831 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T17:51:24.831 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.831 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 6, 2024-04-07T17:51:24.831 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:24.831 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:24.831 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:24.831 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:24.831 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:24.831 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:24.831 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.359086+0000", 2024-04-07T17:51:24.832 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:08.978750+0000", 2024-04-07T17:51:24.832 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:08.978750+0000", 2024-04-07T17:51:24.832 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:08.979235+0000", 2024-04-07T17:51:24.832 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.359086+0000", 2024-04-07T17:51:24.832 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.832 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:24.832 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.832 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 554, 2024-04-07T17:51:24.832 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.359086+0000", 2024-04-07T17:51:24.832 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.359086+0000", 2024-04-07T17:51:24.832 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.359086+0000", 2024-04-07T17:51:24.832 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:24.832 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:24.832 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.832 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.359086+0000", 2024-04-07T17:51:24.833 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.359086+0000", 2024-04-07T17:51:24.833 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:24.833 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:24.833 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:24.833 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:24.833 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 553, 2024-04-07T17:51:24.833 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:24.833 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:24.833 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:24.833 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:24.833 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:24.833 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:24.833 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:24.833 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:24.833 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.35", 2024-04-07T17:51:24.834 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:24.834 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:24.834 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:24.834 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4828, 2024-04-07T17:51:24.834 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:24.834 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T03:27:45.790517+0000", 2024-04-07T17:51:24.834 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:24.834 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:24.834 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:24.834 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:24.834 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:24.834 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:24.834 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:24.834 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:24.834 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:24.835 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:24.835 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:24.835 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:24.835 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:24.835 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:24.835 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:24.835 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:24.835 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:24.835 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:24.835 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:24.835 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:24.835 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:24.835 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:24.835 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:24.835 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:24.836 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:24.836 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:24.836 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:24.836 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:24.836 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:24.836 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:24.836 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:24.836 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:24.836 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:24.836 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:24.836 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:24.836 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:24.836 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:24.836 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:24.836 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:24.836 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:24.837 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:24.837 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:24.837 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:24.837 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.837 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:24.837 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:24.837 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:24.837 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:24.837 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:24.837 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T17:51:24.837 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.837 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 6, 2024-04-07T17:51:24.837 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:24.837 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.837 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:24.837 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:24.838 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:24.838 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:24.838 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T17:51:24.838 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.838 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 6, 2024-04-07T17:51:24.838 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:24.838 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:24.838 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:24.838 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:24.838 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:24.838 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:24.838 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.368833+0000", 2024-04-07T17:51:24.838 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:08.977646+0000", 2024-04-07T17:51:24.838 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:08.977646+0000", 2024-04-07T17:51:24.839 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:08.978144+0000", 2024-04-07T17:51:24.839 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.368833+0000", 2024-04-07T17:51:24.839 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.839 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:24.839 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.839 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 554, 2024-04-07T17:51:24.839 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.368833+0000", 2024-04-07T17:51:24.839 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.368833+0000", 2024-04-07T17:51:24.839 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.368833+0000", 2024-04-07T17:51:24.839 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:24.839 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:24.839 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.839 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.368833+0000", 2024-04-07T17:51:24.839 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.368833+0000", 2024-04-07T17:51:24.839 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:24.840 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:24.840 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:24.840 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:24.840 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 553, 2024-04-07T17:51:24.840 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:24.840 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:24.840 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:24.840 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:24.840 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:24.840 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:24.840 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:24.840 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:24.840 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.36", 2024-04-07T17:51:24.840 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:24.840 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:24.840 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:24.841 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4828, 2024-04-07T17:51:24.841 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:24.841 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T01:10:43.702742+0000", 2024-04-07T17:51:24.841 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:24.841 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:24.841 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:24.841 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:24.841 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:24.841 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:24.841 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:24.841 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:24.841 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:24.841 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:24.841 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:24.841 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:24.842 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:24.842 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:24.842 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:24.842 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:24.842 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:24.842 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:24.842 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:24.842 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:24.842 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:24.842 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:24.842 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:24.842 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:24.842 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:24.842 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:24.843 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:24.843 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:24.843 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:24.843 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:24.843 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:24.843 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:24.843 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:24.843 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:24.843 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:24.843 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:24.843 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:24.843 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:24.843 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:24.843 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:24.843 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:24.843 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:24.844 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:24.844 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.844 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:24.844 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:24.844 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:24.844 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:24.844 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:24.844 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T17:51:24.844 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.844 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 6, 2024-04-07T17:51:24.844 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:24.844 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.844 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:24.844 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:24.844 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:24.844 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:24.845 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T17:51:24.845 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.845 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 7, 2024-04-07T17:51:24.845 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:24.845 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:24.845 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:24.845 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:24.845 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:24.845 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:24.845 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.366733+0000", 2024-04-07T17:51:24.845 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:38.968877+0000", 2024-04-07T17:51:24.845 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:38.968877+0000", 2024-04-07T17:51:24.845 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:38.976137+0000", 2024-04-07T17:51:24.845 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.366733+0000", 2024-04-07T17:51:24.845 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.846 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:24.846 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.846 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:24.846 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.366733+0000", 2024-04-07T17:51:24.846 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.366733+0000", 2024-04-07T17:51:24.846 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.366733+0000", 2024-04-07T17:51:24.846 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:24.846 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:24.846 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.846 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.366733+0000", 2024-04-07T17:51:24.846 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.366733+0000", 2024-04-07T17:51:24.846 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:24.846 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:24.846 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:24.847 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:24.847 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:24.847 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:24.847 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:24.847 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:24.847 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:24.847 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:24.847 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:24.847 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:24.847 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:24.847 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.37", 2024-04-07T17:51:24.847 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:24.847 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:24.847 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:24.847 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4815, 2024-04-07T17:51:24.848 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:24.848 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T17:38:48.549583+0000", 2024-04-07T17:51:24.848 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:24.848 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:24.848 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:24.848 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:24.848 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:24.848 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:24.848 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:24.848 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:24.848 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:24.848 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:24.848 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:24.848 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:24.848 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:24.848 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:24.849 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:24.849 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:24.849 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:24.849 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:24.849 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:24.849 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:24.849 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:24.849 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:24.849 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:24.849 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:24.849 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:24.849 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:24.849 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:24.849 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:24.849 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:24.850 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:24.850 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:24.850 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:24.850 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:24.850 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:24.850 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:24.850 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:24.850 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:24.850 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:24.850 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:24.850 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:24.850 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:24.850 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:24.850 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:24.851 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.851 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:24.851 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:24.851 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:24.851 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:24.851 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:24.851 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T17:51:24.851 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.851 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 7, 2024-04-07T17:51:24.851 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:24.851 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.851 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:24.851 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:24.851 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:24.851 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:24.851 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T17:51:24.852 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.852 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 3, 2024-04-07T17:51:24.852 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:24.852 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:24.852 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:24.852 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:24.852 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:24.852 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:24.852 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:35.975131+0000", 2024-04-07T17:51:24.852 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:08.977725+0000", 2024-04-07T17:51:24.852 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:08.977725+0000", 2024-04-07T17:51:24.852 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:08.978090+0000", 2024-04-07T17:51:24.852 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:35.975131+0000", 2024-04-07T17:51:24.852 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.852 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:24.853 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.853 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 554, 2024-04-07T17:51:24.853 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:35.975131+0000", 2024-04-07T17:51:24.853 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:35.975131+0000", 2024-04-07T17:51:24.853 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:35.975131+0000", 2024-04-07T17:51:24.853 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:24.853 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:24.853 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.853 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:35.975131+0000", 2024-04-07T17:51:24.853 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:35.975131+0000", 2024-04-07T17:51:24.853 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:24.853 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:24.853 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:24.853 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:24.853 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 553, 2024-04-07T17:51:24.853 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:24.854 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:24.854 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:24.854 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:24.854 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:24.854 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:24.854 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:24.854 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:24.854 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.38", 2024-04-07T17:51:24.854 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:24.854 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:24.854 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:24.854 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4929, 2024-04-07T17:51:24.854 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:24.854 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T18:02:53.604912+0000", 2024-04-07T17:51:24.854 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:24.855 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:24.855 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:24.855 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:24.855 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:24.855 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:24.855 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:24.855 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:24.855 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:24.855 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:24.855 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:24.855 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:24.855 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:24.855 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:24.855 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:24.855 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:24.856 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:24.856 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:24.856 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:24.856 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:24.856 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:24.856 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:24.856 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:24.856 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:24.856 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:24.856 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:24.856 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:24.856 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:24.856 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:24.856 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:24.857 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:24.857 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:24.857 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:24.857 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:24.857 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:24.857 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:24.857 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:24.857 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:24.857 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:24.857 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:24.857 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:24.857 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:24.857 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:24.857 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.857 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:24.858 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:24.858 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:24.858 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:24.858 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:24.858 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T17:51:24.858 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.858 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 3, 2024-04-07T17:51:24.858 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:24.858 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.858 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:24.858 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:24.858 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:24.858 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:24.858 INFO:tasks.workunit.client.0.smithi100.stdout: 4 2024-04-07T17:51:24.859 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.859 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 7, 2024-04-07T17:51:24.859 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:24.859 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:24.859 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:24.859 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:24.859 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:24.859 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:24.859 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.358816+0000", 2024-04-07T17:51:24.859 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:38.996215+0000", 2024-04-07T17:51:24.859 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:38.996215+0000", 2024-04-07T17:51:24.859 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:39.000120+0000", 2024-04-07T17:51:24.859 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.358816+0000", 2024-04-07T17:51:24.859 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.859 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:24.860 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.860 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:24.860 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.358816+0000", 2024-04-07T17:51:24.860 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.358816+0000", 2024-04-07T17:51:24.860 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.358816+0000", 2024-04-07T17:51:24.860 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:24.860 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:24.860 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.860 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.358816+0000", 2024-04-07T17:51:24.860 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.358816+0000", 2024-04-07T17:51:24.860 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:24.860 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:24.860 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:24.860 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:24.860 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:24.860 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:24.861 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:24.861 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:24.861 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:24.861 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:24.861 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:24.861 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:24.861 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:24.861 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.39", 2024-04-07T17:51:24.861 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:24.861 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:24.861 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:24.861 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4787, 2024-04-07T17:51:24.861 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:24.861 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T22:36:35.618662+0000", 2024-04-07T17:51:24.861 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:24.861 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:24.862 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:24.862 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:24.862 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:24.862 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:24.862 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:24.862 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:24.862 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:24.862 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:24.862 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:24.862 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:24.862 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:24.862 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:24.862 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:24.863 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:24.863 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:24.863 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:24.863 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:24.863 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:24.863 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:24.863 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:24.863 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:24.863 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:24.863 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:24.863 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:24.863 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:24.863 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:24.863 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:24.863 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:24.864 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:24.864 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:24.864 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:24.864 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:24.864 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:24.864 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:24.864 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:24.864 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:24.864 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:24.864 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:24.864 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:24.864 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:24.864 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:24.864 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.864 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:24.864 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:24.865 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:24.865 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:24.865 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:24.865 INFO:tasks.workunit.client.0.smithi100.stdout: 4 2024-04-07T17:51:24.865 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.865 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 7, 2024-04-07T17:51:24.865 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:24.865 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.865 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:24.865 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:24.865 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:24.865 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:24.865 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T17:51:24.865 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.865 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 5, 2024-04-07T17:51:24.866 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:24.866 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:24.866 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:24.866 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:24.866 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:24.866 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:24.866 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.368733+0000", 2024-04-07T17:51:24.866 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:24:41.548397+0000", 2024-04-07T17:51:24.866 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:24:41.548397+0000", 2024-04-07T17:51:24.866 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:24:41.551443+0000", 2024-04-07T17:51:24.866 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.368733+0000", 2024-04-07T17:51:24.866 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.866 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:24.866 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.867 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 527, 2024-04-07T17:51:24.867 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.368733+0000", 2024-04-07T17:51:24.867 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.368733+0000", 2024-04-07T17:51:24.867 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.368733+0000", 2024-04-07T17:51:24.867 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:24.867 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:24.867 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.867 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.368733+0000", 2024-04-07T17:51:24.867 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.368733+0000", 2024-04-07T17:51:24.867 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:24.867 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:24.867 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:24.867 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:24.867 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 526, 2024-04-07T17:51:24.867 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:24.867 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:24.868 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:24.868 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:24.868 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:24.868 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:24.868 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:24.868 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:24.868 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.3a", 2024-04-07T17:51:24.868 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:24.868 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:24.868 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:24.868 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4865, 2024-04-07T17:51:24.868 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:24.868 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T20:38:47.135272+0000", 2024-04-07T17:51:24.868 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:24.868 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:24.868 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:24.869 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:24.869 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:24.869 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:24.869 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:24.869 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:24.869 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:24.869 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:24.869 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:24.869 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:24.869 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:24.869 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:24.869 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:24.869 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:24.869 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:24.869 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:24.869 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:24.870 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:24.870 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:24.870 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:24.870 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:24.870 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:24.870 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:24.870 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:24.870 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:24.870 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:24.870 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:24.870 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:24.870 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:24.870 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:24.870 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:24.871 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:24.871 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:24.871 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:24.871 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:24.871 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:24.871 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:24.871 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:24.871 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:24.871 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:24.871 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:24.871 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.871 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:24.871 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:24.871 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:24.871 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:24.871 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:24.872 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T17:51:24.872 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.872 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 5, 2024-04-07T17:51:24.872 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:24.872 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.872 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:24.872 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:24.872 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:24.872 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T17:51:24.872 INFO:tasks.workunit.client.0.smithi100.stdout: 4 2024-04-07T17:51:24.872 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.872 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 7, 2024-04-07T17:51:24.872 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:24.872 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:24.872 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:24.872 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:24.873 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:24.873 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:24.873 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.369564+0000", 2024-04-07T17:51:24.873 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:38.985715+0000", 2024-04-07T17:51:24.873 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:38.985715+0000", 2024-04-07T17:51:24.873 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:38.990756+0000", 2024-04-07T17:51:24.873 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.369564+0000", 2024-04-07T17:51:24.873 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.873 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:24.873 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.873 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:24.873 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.369564+0000", 2024-04-07T17:51:24.873 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.369564+0000", 2024-04-07T17:51:24.873 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.369564+0000", 2024-04-07T17:51:24.873 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:24.874 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:24.874 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.874 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.369564+0000", 2024-04-07T17:51:24.874 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.369564+0000", 2024-04-07T17:51:24.874 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:24.874 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:24.874 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:24.874 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:24.874 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:24.874 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:24.874 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:24.874 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:24.874 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:24.874 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:24.874 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:24.875 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:24.875 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:24.875 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.3b", 2024-04-07T17:51:24.875 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:24.875 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:24.875 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:24.875 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4787, 2024-04-07T17:51:24.875 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:24.875 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T17:50:57.375885+0000", 2024-04-07T17:51:24.875 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:24.875 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:24.875 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:24.875 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:24.875 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:24.875 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:24.875 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:24.876 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:24.876 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:24.876 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:24.876 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:24.876 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:24.876 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:24.876 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:24.876 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:24.876 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:24.876 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:24.876 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:24.876 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:24.876 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:24.876 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:24.876 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:24.877 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:24.877 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:24.877 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:24.877 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:24.877 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:24.877 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:24.877 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:24.877 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:24.877 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:24.877 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:24.877 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:24.877 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:24.877 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:24.877 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:24.878 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:24.878 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:24.878 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:24.878 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:24.878 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:24.878 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:24.878 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:24.878 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.878 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:24.878 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:24.878 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:24.878 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:24.878 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T17:51:24.878 INFO:tasks.workunit.client.0.smithi100.stdout: 4 2024-04-07T17:51:24.878 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.879 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 7, 2024-04-07T17:51:24.879 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:24.879 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.879 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:24.879 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:24.879 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:24.879 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:24.879 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T17:51:24.879 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.879 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 5, 2024-04-07T17:51:24.879 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:24.879 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:24.879 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:24.879 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:24.879 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:24.880 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:24.880 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.365779+0000", 2024-04-07T17:51:24.880 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:38.986923+0000", 2024-04-07T17:51:24.880 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:38.986923+0000", 2024-04-07T17:51:24.880 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:38.987454+0000", 2024-04-07T17:51:24.880 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.365779+0000", 2024-04-07T17:51:24.880 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.880 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:24.880 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.880 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:24.880 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.365779+0000", 2024-04-07T17:51:24.880 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.365779+0000", 2024-04-07T17:51:24.880 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.365779+0000", 2024-04-07T17:51:24.880 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:24.880 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:24.881 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.881 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.365779+0000", 2024-04-07T17:51:24.881 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.365779+0000", 2024-04-07T17:51:24.881 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:24.881 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:24.881 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:24.881 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:24.881 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:24.881 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:24.881 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:24.881 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:24.881 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:24.881 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:24.881 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:24.881 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:24.881 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:24.881 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.3c", 2024-04-07T17:51:24.882 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:24.882 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:24.882 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:24.882 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4883, 2024-04-07T17:51:24.882 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:24.882 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T02:20:58.654636+0000", 2024-04-07T17:51:24.882 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:24.882 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:24.882 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:24.882 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:24.882 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:24.882 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:24.882 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:24.882 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:24.882 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:24.883 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:24.883 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:24.883 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:24.883 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:24.883 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:24.883 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:24.883 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:24.883 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:24.883 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:24.883 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:24.883 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:24.883 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:24.883 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:24.883 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:24.883 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:24.883 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:24.883 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:24.884 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:24.884 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:24.884 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:24.884 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:24.884 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:24.884 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:24.884 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:24.884 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:24.884 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:24.884 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:24.884 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:24.884 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:24.884 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:24.884 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:24.884 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:24.884 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:24.885 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:24.885 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.885 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:24.885 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:24.885 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:24.885 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:24.885 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:24.885 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T17:51:24.885 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.885 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 5, 2024-04-07T17:51:24.885 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:24.885 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.885 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:24.885 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:24.885 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:24.885 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T17:51:24.886 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T17:51:24.886 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.886 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 0, 2024-04-07T17:51:24.886 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:24.886 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:24.886 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:24.886 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:24.886 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:24.886 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:24.886 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:35.982778+0000", 2024-04-07T17:51:24.886 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:23:15.876439+0000", 2024-04-07T17:51:24.886 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:23:15.876439+0000", 2024-04-07T17:51:24.886 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:23:15.876856+0000", 2024-04-07T17:51:24.886 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:35.982778+0000", 2024-04-07T17:51:24.887 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.887 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:24.887 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.887 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 445, 2024-04-07T17:51:24.887 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:35.982778+0000", 2024-04-07T17:51:24.887 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:35.982778+0000", 2024-04-07T17:51:24.887 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:35.982778+0000", 2024-04-07T17:51:24.887 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:24.887 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:24.887 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.887 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:35.982778+0000", 2024-04-07T17:51:24.887 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:35.982778+0000", 2024-04-07T17:51:24.887 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:24.887 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:24.887 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:24.887 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:24.888 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 444, 2024-04-07T17:51:24.888 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:24.888 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:24.888 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:24.888 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:24.888 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:24.888 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:24.888 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:24.888 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:24.888 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.3d", 2024-04-07T17:51:24.888 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:24.888 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:24.888 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:24.888 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 5000, 2024-04-07T17:51:24.888 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:24.888 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T03:48:36.975141+0000", 2024-04-07T17:51:24.889 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:24.889 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:24.889 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:24.889 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:24.889 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:24.889 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:24.889 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:24.889 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:24.889 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:24.889 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:24.889 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:24.889 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:24.889 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:24.889 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:24.889 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:24.889 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:24.890 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:24.890 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:24.890 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:24.890 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:24.890 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:24.890 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:24.890 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:24.890 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:24.890 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:24.890 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:24.890 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:24.890 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:24.890 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:24.890 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:24.891 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:24.891 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:24.891 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:24.891 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:24.891 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:24.891 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:24.891 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:24.891 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:24.891 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:24.891 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:24.891 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:24.891 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:24.891 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:24.891 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.891 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:24.892 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:24.892 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:24.892 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:24.892 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T17:51:24.892 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T17:51:24.892 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.892 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 0, 2024-04-07T17:51:24.892 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:24.892 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.892 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:24.892 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:24.892 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:24.892 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:24.892 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T17:51:24.892 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.893 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 1, 2024-04-07T17:51:24.893 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:24.893 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:24.893 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:24.893 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:24.893 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:24.893 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:24.893 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.334821+0000", 2024-04-07T17:51:24.893 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:08.962145+0000", 2024-04-07T17:51:24.893 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:08.962145+0000", 2024-04-07T17:51:24.893 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:08.962703+0000", 2024-04-07T17:51:24.893 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.334821+0000", 2024-04-07T17:51:24.893 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.893 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:24.893 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.894 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 554, 2024-04-07T17:51:24.894 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.334821+0000", 2024-04-07T17:51:24.894 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.334821+0000", 2024-04-07T17:51:24.894 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.334821+0000", 2024-04-07T17:51:24.894 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:24.894 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:24.894 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.894 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.334821+0000", 2024-04-07T17:51:24.894 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.334821+0000", 2024-04-07T17:51:24.894 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:24.894 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:24.894 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:24.894 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:24.895 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 553, 2024-04-07T17:51:24.895 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:24.895 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:24.895 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:24.895 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:24.895 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:24.895 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:24.895 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:24.895 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:24.895 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.3e", 2024-04-07T17:51:24.895 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:24.895 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:24.895 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:24.895 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4989, 2024-04-07T17:51:24.895 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:24.896 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T23:09:45.491615+0000", 2024-04-07T17:51:24.896 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:24.896 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:24.896 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:24.896 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:24.896 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:24.896 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:24.896 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:24.896 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:24.896 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:24.896 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:24.896 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:24.896 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:24.896 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:24.896 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:24.896 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:24.897 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:24.897 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:24.897 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:24.897 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:24.897 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:24.897 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:24.897 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:24.897 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:24.897 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:24.897 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:24.897 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:24.897 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:24.897 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:24.897 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:24.897 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:24.898 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:24.898 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:24.898 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:24.898 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:24.898 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:24.898 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:24.898 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:24.898 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:24.898 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:24.898 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:24.898 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:24.898 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:24.898 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:24.898 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.898 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:24.899 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:24.899 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:24.899 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:24.899 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:24.899 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T17:51:24.899 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.899 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 1, 2024-04-07T17:51:24.899 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:24.899 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.899 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:24.899 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:24.899 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:24.899 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:24.899 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T17:51:24.899 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.899 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 7, 2024-04-07T17:51:24.899 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:24.900 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:24.900 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:24.900 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:24.900 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:24.900 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:24.900 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.367170+0000", 2024-04-07T17:51:24.900 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:38.966876+0000", 2024-04-07T17:51:24.900 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:38.966876+0000", 2024-04-07T17:51:24.900 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:38.969419+0000", 2024-04-07T17:51:24.900 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.367170+0000", 2024-04-07T17:51:24.900 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.900 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:24.900 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.900 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:24.900 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.367170+0000", 2024-04-07T17:51:24.901 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.367170+0000", 2024-04-07T17:51:24.901 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.367170+0000", 2024-04-07T17:51:24.901 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:24.901 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:24.901 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:24.901 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.367170+0000", 2024-04-07T17:51:24.901 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.367170+0000", 2024-04-07T17:51:24.901 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:24.901 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:24.901 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:24.901 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:24.901 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:24.901 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:24.901 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:24.901 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:24.901 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:24.901 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:24.902 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:24.902 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:24.902 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:24.902 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.3f", 2024-04-07T17:51:24.902 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:24.902 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:24.902 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:24.902 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4807, 2024-04-07T17:51:24.902 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:24.902 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T01:03:39.405808+0000", 2024-04-07T17:51:24.902 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:24.902 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:24.902 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:24.902 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:24.902 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:24.903 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:24.903 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:24.903 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:24.903 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:24.903 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:24.903 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:24.903 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:24.903 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:24.903 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:24.903 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:24.903 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:24.903 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:24.903 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:24.903 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:24.903 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:24.904 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:24.904 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:24.904 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:24.904 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:24.904 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:24.904 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:24.904 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:24.904 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:24.904 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:24.904 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:24.904 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:24.904 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:24.904 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:24.904 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:24.904 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:24.905 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:24.905 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:24.905 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:24.905 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:24.905 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:24.905 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:24.905 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:24.905 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:24.905 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.905 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:24.905 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:24.905 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:24.905 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:24.905 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:24.905 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T17:51:24.906 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.906 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 7, 2024-04-07T17:51:24.906 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:24.906 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:24.906 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.906 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_per_pool": [ 2024-04-07T17:51:24.906 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:24.906 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": 24, 2024-04-07T17:51:24.906 INFO:tasks.workunit.client.0.smithi100.stdout: "application": [ 2024-04-07T17:51:24.906 INFO:tasks.workunit.client.0.smithi100.stdout: "rbd" 2024-04-07T17:51:24.906 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.906 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 24403, 2024-04-07T17:51:24.906 INFO:tasks.workunit.client.0.smithi100.stdout: "num_pg": 8, 2024-04-07T17:51:24.906 INFO:tasks.workunit.client.0.smithi100.stdout: "num_store_stats": 8, 2024-04-07T17:51:24.906 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 24403, 2024-04-07T17:51:24.907 INFO:tasks.workunit.client.0.smithi100.stdout: "poolid": 4, 2024-04-07T17:51:24.907 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:24.907 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 4194323, 2024-04-07T17:51:24.907 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:24.907 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:24.907 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:24.907 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:24.907 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:24.907 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:24.907 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:24.907 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:24.907 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:24.907 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:24.907 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:24.907 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:24.907 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:24.908 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:24.908 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:24.908 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 12, 2024-04-07T17:51:24.908 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 4, 2024-04-07T17:51:24.908 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:24.908 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 4, 2024-04-07T17:51:24.908 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:24.908 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:24.908 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:24.908 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:24.908 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:24.908 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 2, 2024-04-07T17:51:24.908 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:24.908 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:24.908 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:24.908 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:24.909 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:24.909 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:24.909 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:24.909 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 9087, 2024-04-07T17:51:24.909 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 3147276, 2024-04-07T17:51:24.909 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:24.909 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:24.909 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:24.909 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 24273, 2024-04-07T17:51:24.909 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 3030917 2024-04-07T17:51:24.909 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.909 INFO:tasks.workunit.client.0.smithi100.stdout: "store_stats": { 2024-04-07T17:51:24.909 INFO:tasks.workunit.client.0.smithi100.stdout: "allocated": 12595200, 2024-04-07T17:51:24.909 INFO:tasks.workunit.client.0.smithi100.stdout: "available": 0, 2024-04-07T17:51:24.909 INFO:tasks.workunit.client.0.smithi100.stdout: "data_compressed": 0, 2024-04-07T17:51:24.909 INFO:tasks.workunit.client.0.smithi100.stdout: "data_compressed_allocated": 0, 2024-04-07T17:51:24.909 INFO:tasks.workunit.client.0.smithi100.stdout: "data_compressed_original": 0, 2024-04-07T17:51:24.910 INFO:tasks.workunit.client.0.smithi100.stdout: "data_stored": 12582969, 2024-04-07T17:51:24.910 INFO:tasks.workunit.client.0.smithi100.stdout: "internal_metadata": 0, 2024-04-07T17:51:24.910 INFO:tasks.workunit.client.0.smithi100.stdout: "internally_reserved": 0, 2024-04-07T17:51:24.910 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_allocated": 0, 2024-04-07T17:51:24.910 INFO:tasks.workunit.client.0.smithi100.stdout: "total": 0 2024-04-07T17:51:24.910 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.910 INFO:tasks.workunit.client.0.smithi100.stdout: "up": 24 2024-04-07T17:51:24.910 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.910 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:24.910 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": 192, 2024-04-07T17:51:24.910 INFO:tasks.workunit.client.0.smithi100.stdout: "application": [ 2024-04-07T17:51:24.910 INFO:tasks.workunit.client.0.smithi100.stdout: "cephfs" 2024-04-07T17:51:24.910 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.910 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:24.910 INFO:tasks.workunit.client.0.smithi100.stdout: "num_pg": 64, 2024-04-07T17:51:24.911 INFO:tasks.workunit.client.0.smithi100.stdout: "num_store_stats": 8, 2024-04-07T17:51:24.911 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:24.911 INFO:tasks.workunit.client.0.smithi100.stdout: "poolid": 3, 2024-04-07T17:51:24.911 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:24.911 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:24.911 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:24.911 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:24.911 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:24.911 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:24.911 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:24.911 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:24.911 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:24.911 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:24.911 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:24.911 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:24.911 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:24.911 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:24.912 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:24.912 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:24.912 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:24.912 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:24.912 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:24.912 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:24.912 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:24.912 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:24.912 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:24.912 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:24.912 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:24.912 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:24.912 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:24.912 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:24.912 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:24.913 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:24.913 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:24.913 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:24.913 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:24.913 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:24.913 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:24.913 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:24.913 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:24.913 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:24.913 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:24.913 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:24.913 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:24.913 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.913 INFO:tasks.workunit.client.0.smithi100.stdout: "store_stats": { 2024-04-07T17:51:24.913 INFO:tasks.workunit.client.0.smithi100.stdout: "allocated": 0, 2024-04-07T17:51:24.914 INFO:tasks.workunit.client.0.smithi100.stdout: "available": 0, 2024-04-07T17:51:24.914 INFO:tasks.workunit.client.0.smithi100.stdout: "data_compressed": 0, 2024-04-07T17:51:24.914 INFO:tasks.workunit.client.0.smithi100.stdout: "data_compressed_allocated": 0, 2024-04-07T17:51:24.914 INFO:tasks.workunit.client.0.smithi100.stdout: "data_compressed_original": 0, 2024-04-07T17:51:24.914 INFO:tasks.workunit.client.0.smithi100.stdout: "data_stored": 0, 2024-04-07T17:51:24.914 INFO:tasks.workunit.client.0.smithi100.stdout: "internal_metadata": 0, 2024-04-07T17:51:24.914 INFO:tasks.workunit.client.0.smithi100.stdout: "internally_reserved": 0, 2024-04-07T17:51:24.914 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_allocated": 0, 2024-04-07T17:51:24.914 INFO:tasks.workunit.client.0.smithi100.stdout: "total": 0 2024-04-07T17:51:24.914 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.914 INFO:tasks.workunit.client.0.smithi100.stdout: "up": 192 2024-04-07T17:51:24.914 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.914 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:24.914 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": 3, 2024-04-07T17:51:24.915 INFO:tasks.workunit.client.0.smithi100.stdout: "application": [ 2024-04-07T17:51:24.915 INFO:tasks.workunit.client.0.smithi100.stdout: "mgr" 2024-04-07T17:51:24.915 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.915 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 241, 2024-04-07T17:51:24.915 INFO:tasks.workunit.client.0.smithi100.stdout: "num_pg": 1, 2024-04-07T17:51:24.915 INFO:tasks.workunit.client.0.smithi100.stdout: "num_store_stats": 3, 2024-04-07T17:51:24.915 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 241, 2024-04-07T17:51:24.915 INFO:tasks.workunit.client.0.smithi100.stdout: "poolid": 1, 2024-04-07T17:51:24.915 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:24.915 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 590368, 2024-04-07T17:51:24.915 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:24.915 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 1771104, 2024-04-07T17:51:24.915 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:24.915 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:24.915 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:24.915 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:24.916 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:24.916 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:24.916 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:24.916 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:24.916 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:24.916 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:24.916 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:24.916 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:24.916 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:24.916 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 6, 2024-04-07T17:51:24.916 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 2, 2024-04-07T17:51:24.916 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:24.916 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 2, 2024-04-07T17:51:24.916 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:24.917 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:24.917 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:24.917 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:24.917 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:24.917 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:24.917 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:24.917 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 6, 2024-04-07T17:51:24.917 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:24.917 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:24.917 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:24.917 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:24.917 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:24.917 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 1214, 2024-04-07T17:51:24.917 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 2024, 2024-04-07T17:51:24.917 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:24.918 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:24.918 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:24.918 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 399, 2024-04-07T17:51:24.918 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 2416 2024-04-07T17:51:24.918 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.918 INFO:tasks.workunit.client.0.smithi100.stdout: "store_stats": { 2024-04-07T17:51:24.918 INFO:tasks.workunit.client.0.smithi100.stdout: "allocated": 1781760, 2024-04-07T17:51:24.918 INFO:tasks.workunit.client.0.smithi100.stdout: "available": 0, 2024-04-07T17:51:24.918 INFO:tasks.workunit.client.0.smithi100.stdout: "data_compressed": 0, 2024-04-07T17:51:24.918 INFO:tasks.workunit.client.0.smithi100.stdout: "data_compressed_allocated": 0, 2024-04-07T17:51:24.918 INFO:tasks.workunit.client.0.smithi100.stdout: "data_compressed_original": 0, 2024-04-07T17:51:24.918 INFO:tasks.workunit.client.0.smithi100.stdout: "data_stored": 1771104, 2024-04-07T17:51:24.918 INFO:tasks.workunit.client.0.smithi100.stdout: "internal_metadata": 0, 2024-04-07T17:51:24.918 INFO:tasks.workunit.client.0.smithi100.stdout: "internally_reserved": 0, 2024-04-07T17:51:24.919 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_allocated": 0, 2024-04-07T17:51:24.919 INFO:tasks.workunit.client.0.smithi100.stdout: "total": 0 2024-04-07T17:51:24.919 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.919 INFO:tasks.workunit.client.0.smithi100.stdout: "up": 3 2024-04-07T17:51:24.919 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.919 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:24.919 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": 96, 2024-04-07T17:51:24.919 INFO:tasks.workunit.client.0.smithi100.stdout: "application": [ 2024-04-07T17:51:24.919 INFO:tasks.workunit.client.0.smithi100.stdout: "cephfs" 2024-04-07T17:51:24.919 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.919 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 68, 2024-04-07T17:51:24.919 INFO:tasks.workunit.client.0.smithi100.stdout: "num_pg": 32, 2024-04-07T17:51:24.919 INFO:tasks.workunit.client.0.smithi100.stdout: "num_store_stats": 8, 2024-04-07T17:51:24.919 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 68, 2024-04-07T17:51:24.919 INFO:tasks.workunit.client.0.smithi100.stdout: "poolid": 2, 2024-04-07T17:51:24.919 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:24.920 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 6622, 2024-04-07T17:51:24.920 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:24.920 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:24.920 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:24.920 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:24.920 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:24.920 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:24.920 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:24.920 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:24.920 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:24.920 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:24.920 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:24.920 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:24.920 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:24.920 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:24.921 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:24.921 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 66, 2024-04-07T17:51:24.921 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 22, 2024-04-07T17:51:24.921 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:24.921 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 22, 2024-04-07T17:51:24.921 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:24.921 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:24.921 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:24.921 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:24.921 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:24.921 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 14, 2024-04-07T17:51:24.921 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:24.921 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:24.921 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:24.921 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:24.922 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:24.922 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:24.922 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:24.922 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 79, 2024-04-07T17:51:24.922 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 87, 2024-04-07T17:51:24.922 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:24.922 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:24.922 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:24.922 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 68, 2024-04-07T17:51:24.922 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 36 2024-04-07T17:51:24.922 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.922 INFO:tasks.workunit.client.0.smithi100.stdout: "store_stats": { 2024-04-07T17:51:24.922 INFO:tasks.workunit.client.0.smithi100.stdout: "allocated": 110592, 2024-04-07T17:51:24.922 INFO:tasks.workunit.client.0.smithi100.stdout: "available": 0, 2024-04-07T17:51:24.922 INFO:tasks.workunit.client.0.smithi100.stdout: "data_compressed": 0, 2024-04-07T17:51:24.923 INFO:tasks.workunit.client.0.smithi100.stdout: "data_compressed_allocated": 0, 2024-04-07T17:51:24.923 INFO:tasks.workunit.client.0.smithi100.stdout: "data_compressed_original": 0, 2024-04-07T17:51:24.923 INFO:tasks.workunit.client.0.smithi100.stdout: "data_stored": 19866, 2024-04-07T17:51:24.923 INFO:tasks.workunit.client.0.smithi100.stdout: "internal_metadata": 0, 2024-04-07T17:51:24.923 INFO:tasks.workunit.client.0.smithi100.stdout: "internally_reserved": 0, 2024-04-07T17:51:24.923 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_allocated": 48581, 2024-04-07T17:51:24.923 INFO:tasks.workunit.client.0.smithi100.stdout: "total": 0 2024-04-07T17:51:24.923 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.923 INFO:tasks.workunit.client.0.smithi100.stdout: "up": 96 2024-04-07T17:51:24.923 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:24.923 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:24.923 INFO:tasks.workunit.client.0.smithi100.stdout: "usage": { 2024-04-07T17:51:24.923 INFO:tasks.workunit.client.0.smithi100.stdout: "pg_num": 105, 2024-04-07T17:51:24.923 INFO:tasks.workunit.client.0.smithi100.stdout: "pools": 4, 2024-04-07T17:51:24.923 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_by_class": { 2024-04-07T17:51:24.923 INFO:tasks.workunit.client.0.smithi100.stdout: "ssd": { 2024-04-07T17:51:24.924 INFO:tasks.workunit.client.0.smithi100.stdout: "total_avail_bytes": 763743436800, 2024-04-07T17:51:24.924 INFO:tasks.workunit.client.0.smithi100.stdout: "total_bytes": 767960285184, 2024-04-07T17:51:24.924 INFO:tasks.workunit.client.0.smithi100.stdout: "total_used_bytes": 4216848384, 2024-04-07T17:51:24.924 INFO:tasks.workunit.client.0.smithi100.stdout: "total_used_raw_bytes": 4216848384, 2024-04-07T17:51:24.924 INFO:tasks.workunit.client.0.smithi100.stdout: "total_used_raw_ratio": 0.005490971729159355 2024-04-07T17:51:24.924 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:24.924 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:24.924 INFO:tasks.workunit.client.0.smithi100.stdout: "total_avail_bytes": 763743436800, 2024-04-07T17:51:24.924 INFO:tasks.workunit.client.0.smithi100.stdout: "total_bytes": 767960285184, 2024-04-07T17:51:24.924 INFO:tasks.workunit.client.0.smithi100.stdout: "total_used_bytes": 4216848384 2024-04-07T17:51:24.924 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:24.924 INFO:tasks.workunit.client.0.smithi100.stdout:} 2024-04-07T17:51:24.924 INFO:tasks.workunit.client.0.smithi100.stdout:Device report is generated separately. To see it run 'ceph telemetry preview-device'. 2024-04-07T17:51:24.925 INFO:tasks.workunit.client.0.smithi100.stderr:+ ceph telemetry preview-device 2024-04-07T17:51:24.926 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:23 smithi100 bash[76202]: audit 2024-04-07T17:51:21.919695+0000 mgr.y (mgr.25175) 2880 : audit [DBG] from='client.36057 -' entity='client.admin' cmd=[{"prefix": "telemetry collection ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T17:51:24.926 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:23 smithi100 bash[76202]: audit 2024-04-07T17:51:22.070853+0000 mgr.y (mgr.25175) 2881 : audit [DBG] from='client.36057 -' entity='client.admin' cmd=[{"prefix": "telemetry collection ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T17:51:24.926 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:23 smithi100 bash[76202]: audit 2024-04-07T17:51:22.422180+0000 mon.a (mon.0) 4767 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:51:24.926 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:23 smithi100 bash[76202]: audit 2024-04-07T17:51:22.434583+0000 mon.a (mon.0) 4768 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config-key dump", "format": "json"}]: dispatch 2024-04-07T17:51:24.926 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:23 smithi100 bash[76202]: audit 2024-04-07T17:51:22.615499+0000 mon.a (mon.0) 4769 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:24.926 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:23 smithi100 bash[76202]: audit 2024-04-07T17:51:22.616092+0000 mon.a (mon.0) 4770 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:24.926 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:23 smithi100 bash[76202]: audit 2024-04-07T17:51:22.616573+0000 mon.a (mon.0) 4771 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:24.926 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:23 smithi100 bash[76202]: audit 2024-04-07T17:51:22.620125+0000 mon.a (mon.0) 4772 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:24.926 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:23 smithi100 bash[76202]: audit 2024-04-07T17:51:22.620653+0000 mon.a (mon.0) 4773 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:24.926 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:23 smithi100 bash[76202]: audit 2024-04-07T17:51:22.621147+0000 mon.a (mon.0) 4774 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:24.926 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:23 smithi100 bash[76202]: audit 2024-04-07T17:51:22.624235+0000 mon.a (mon.0) 4775 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:24.926 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:23 smithi100 bash[76202]: audit 2024-04-07T17:51:22.624730+0000 mon.a (mon.0) 4776 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:24.927 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:23 smithi100 bash[76202]: audit 2024-04-07T17:51:22.625149+0000 mon.a (mon.0) 4777 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:24.927 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:23 smithi100 bash[76202]: audit 2024-04-07T17:51:22.644322+0000 mon.a (mon.0) 4778 : audit [INF] from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T17:51:24.927 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:23 smithi100 bash[76202]: audit 2024-04-07T17:51:22.644482+0000 mon.a (mon.0) 4779 : audit [INF] from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T17:51:24.927 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:23 smithi100 bash[76202]: audit 2024-04-07T17:51:22.644977+0000 mon.a (mon.0) 4780 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:24.927 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:23 smithi100 bash[76202]: audit 2024-04-07T17:51:22.645403+0000 mon.a (mon.0) 4781 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:24.927 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:23 smithi100 bash[76202]: audit 2024-04-07T17:51:22.645868+0000 mon.a (mon.0) 4782 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:24.927 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:23 smithi100 bash[74942]: audit 2024-04-07T17:51:21.919695+0000 mgr.y (mgr.25175) 2880 : audit [DBG] from='client.36057 -' entity='client.admin' cmd=[{"prefix": "telemetry collection ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T17:51:24.927 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:23 smithi100 bash[74942]: audit 2024-04-07T17:51:22.070853+0000 mgr.y (mgr.25175) 2881 : audit [DBG] from='client.36057 -' entity='client.admin' cmd=[{"prefix": "telemetry collection ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T17:51:24.927 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:23 smithi100 bash[74942]: audit 2024-04-07T17:51:22.422180+0000 mon.a (mon.0) 4767 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:51:24.927 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:23 smithi100 bash[74942]: audit 2024-04-07T17:51:22.434583+0000 mon.a (mon.0) 4768 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config-key dump", "format": "json"}]: dispatch 2024-04-07T17:51:24.928 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:23 smithi100 bash[74942]: audit 2024-04-07T17:51:22.615499+0000 mon.a (mon.0) 4769 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:24.928 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:23 smithi100 bash[74942]: audit 2024-04-07T17:51:22.616092+0000 mon.a (mon.0) 4770 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:24.928 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:23 smithi100 bash[74942]: audit 2024-04-07T17:51:22.616573+0000 mon.a (mon.0) 4771 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:24.928 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:23 smithi100 bash[74942]: audit 2024-04-07T17:51:22.620125+0000 mon.a (mon.0) 4772 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:24.928 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:23 smithi100 bash[74942]: audit 2024-04-07T17:51:22.620653+0000 mon.a (mon.0) 4773 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:24.928 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:23 smithi100 bash[74942]: audit 2024-04-07T17:51:22.621147+0000 mon.a (mon.0) 4774 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:24.928 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:23 smithi100 bash[74942]: audit 2024-04-07T17:51:22.624235+0000 mon.a (mon.0) 4775 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:24.928 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:23 smithi100 bash[74942]: audit 2024-04-07T17:51:22.624730+0000 mon.a (mon.0) 4776 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:24.928 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:23 smithi100 bash[74942]: audit 2024-04-07T17:51:22.625149+0000 mon.a (mon.0) 4777 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:24.928 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:23 smithi100 bash[74942]: audit 2024-04-07T17:51:22.644322+0000 mon.a (mon.0) 4778 : audit [INF] from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T17:51:24.928 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:23 smithi100 bash[74942]: audit 2024-04-07T17:51:22.644482+0000 mon.a (mon.0) 4779 : audit [INF] from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T17:51:24.928 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:23 smithi100 bash[74942]: audit 2024-04-07T17:51:22.644977+0000 mon.a (mon.0) 4780 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:24.928 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:23 smithi100 bash[74942]: audit 2024-04-07T17:51:22.645403+0000 mon.a (mon.0) 4781 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:24.928 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:23 smithi100 bash[74942]: audit 2024-04-07T17:51:22.645868+0000 mon.a (mon.0) 4782 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:24.929 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:23 smithi144 bash[66654]: audit 2024-04-07T17:51:21.919695+0000 mgr.y (mgr.25175) 2880 : audit [DBG] from='client.36057 -' entity='client.admin' cmd=[{"prefix": "telemetry collection ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T17:51:24.929 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:23 smithi144 bash[66654]: audit 2024-04-07T17:51:22.070853+0000 mgr.y (mgr.25175) 2881 : audit [DBG] from='client.36057 -' entity='client.admin' cmd=[{"prefix": "telemetry collection ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T17:51:24.929 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:23 smithi144 bash[66654]: audit 2024-04-07T17:51:22.422180+0000 mon.a (mon.0) 4767 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:51:24.929 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:23 smithi144 bash[66654]: audit 2024-04-07T17:51:22.434583+0000 mon.a (mon.0) 4768 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config-key dump", "format": "json"}]: dispatch 2024-04-07T17:51:24.929 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:23 smithi144 bash[66654]: audit 2024-04-07T17:51:22.615499+0000 mon.a (mon.0) 4769 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:24.929 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:23 smithi144 bash[66654]: audit 2024-04-07T17:51:22.616092+0000 mon.a (mon.0) 4770 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:24.929 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:23 smithi144 bash[66654]: audit 2024-04-07T17:51:22.616573+0000 mon.a (mon.0) 4771 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:24.929 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:23 smithi144 bash[66654]: audit 2024-04-07T17:51:22.620125+0000 mon.a (mon.0) 4772 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:24.929 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:23 smithi144 bash[66654]: audit 2024-04-07T17:51:22.620653+0000 mon.a (mon.0) 4773 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:24.929 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:23 smithi144 bash[66654]: audit 2024-04-07T17:51:22.621147+0000 mon.a (mon.0) 4774 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:24.929 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:23 smithi144 bash[66654]: audit 2024-04-07T17:51:22.624235+0000 mon.a (mon.0) 4775 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:24.929 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:23 smithi144 bash[66654]: audit 2024-04-07T17:51:22.624730+0000 mon.a (mon.0) 4776 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:24.930 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:23 smithi144 bash[66654]: audit 2024-04-07T17:51:22.625149+0000 mon.a (mon.0) 4777 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:24.930 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:23 smithi144 bash[66654]: audit 2024-04-07T17:51:22.644322+0000 mon.a (mon.0) 4778 : audit [INF] from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T17:51:24.930 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:23 smithi144 bash[66654]: audit 2024-04-07T17:51:22.644482+0000 mon.a (mon.0) 4779 : audit [INF] from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T17:51:24.930 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:23 smithi144 bash[66654]: audit 2024-04-07T17:51:22.644977+0000 mon.a (mon.0) 4780 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:24.930 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:23 smithi144 bash[66654]: audit 2024-04-07T17:51:22.645403+0000 mon.a (mon.0) 4781 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:24.930 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:23 smithi144 bash[66654]: audit 2024-04-07T17:51:22.645868+0000 mon.a (mon.0) 4782 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:24.930 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:23 smithi144 bash[66654]: audit 2024-04-07T17:51:22.649344+0000 mon.a (mon.0) 4783 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:24.930 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:23 smithi144 bash[66654]: audit 2024-04-07T17:51:22.650343+0000 mon.a (mon.0) 4784 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:24.930 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:23 smithi144 bash[66654]: audit 2024-04-07T17:51:22.650792+0000 mon.a (mon.0) 4785 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:24.930 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:23 smithi144 bash[66654]: audit 2024-04-07T17:51:22.651171+0000 mon.b (mon.1) 23 : audit [INF] from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T17:51:24.930 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:23 smithi144 bash[66654]: audit 2024-04-07T17:51:22.651304+0000 mon.b (mon.1) 24 : audit [INF] from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T17:51:24.930 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:23 smithi144 bash[66654]: audit 2024-04-07T17:51:22.653707+0000 mon.c (mon.2) 121 : audit [INF] from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T17:51:24.930 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:23 smithi144 bash[66654]: audit 2024-04-07T17:51:22.653842+0000 mon.c (mon.2) 122 : audit [INF] from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T17:51:24.930 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:23 smithi144 bash[66654]: audit 2024-04-07T17:51:22.654170+0000 mon.a (mon.0) 4786 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:24.931 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:23 smithi144 bash[66654]: audit 2024-04-07T17:51:22.655048+0000 mon.a (mon.0) 4787 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:24.931 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:23 smithi144 bash[66654]: audit 2024-04-07T17:51:22.655411+0000 mon.a (mon.0) 4788 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:24.931 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:23 smithi144 bash[66654]: audit 2024-04-07T17:51:22.940207+0000 mon.a (mon.0) 4789 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:51:24.931 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:23 smithi144 bash[66654]: audit 2024-04-07T17:51:22.947642+0000 mon.a (mon.0) 4790 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config-key dump", "format": "json"}]: dispatch 2024-04-07T17:51:24.931 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:24 smithi144 bash[66654]: audit 2024-04-07T17:51:22.417701+0000 mgr.y (mgr.25175) 2882 : audit [DBG] from='client.36063 -' entity='client.admin' cmd=[{"prefix": "telemetry preview", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T17:51:24.931 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:24 smithi144 bash[66654]: cluster 2024-04-07T17:51:22.828785+0000 mgr.y (mgr.25175) 2883 : cluster [DBG] pgmap v5168: 105 pgs: 105 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail 2024-04-07T17:51:24.931 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:24 smithi144 bash[66654]: audit 2024-04-07T17:51:22.937144+0000 mgr.y (mgr.25175) 2884 : audit [DBG] from='client.36063 -' entity='client.admin' cmd=[{"prefix": "telemetry preview", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T17:51:24.931 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:24 smithi144 bash[66654]: audit 2024-04-07T17:51:23.175495+0000 mon.a (mon.0) 4791 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:24.931 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:24 smithi144 bash[66654]: audit 2024-04-07T17:51:23.175974+0000 mon.a (mon.0) 4792 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:24.931 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:24 smithi144 bash[66654]: audit 2024-04-07T17:51:23.176376+0000 mon.a (mon.0) 4793 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:24.931 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:24 smithi144 bash[66654]: audit 2024-04-07T17:51:23.179807+0000 mon.a (mon.0) 4794 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:24.931 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:24 smithi144 bash[66654]: audit 2024-04-07T17:51:23.180280+0000 mon.a (mon.0) 4795 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:24.931 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:24 smithi144 bash[66654]: audit 2024-04-07T17:51:23.180694+0000 mon.a (mon.0) 4796 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:24.931 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:24 smithi144 bash[66654]: audit 2024-04-07T17:51:23.183755+0000 mon.a (mon.0) 4797 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:24.931 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:24 smithi144 bash[66654]: audit 2024-04-07T17:51:23.184294+0000 mon.a (mon.0) 4798 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:24.932 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:24 smithi144 bash[66654]: audit 2024-04-07T17:51:23.184797+0000 mon.a (mon.0) 4799 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:24.932 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:24 smithi144 bash[66654]: audit 2024-04-07T17:51:23.204303+0000 mon.a (mon.0) 4800 : audit [INF] from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T17:51:24.932 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:24 smithi144 bash[66654]: audit 2024-04-07T17:51:23.204409+0000 mon.a (mon.0) 4801 : audit [INF] from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T17:51:24.932 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:24 smithi144 bash[66654]: audit 2024-04-07T17:51:23.204752+0000 mon.a (mon.0) 4802 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:24.932 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:24 smithi144 bash[66654]: audit 2024-04-07T17:51:23.205338+0000 mon.a (mon.0) 4803 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:24.932 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:24 smithi144 bash[66654]: audit 2024-04-07T17:51:23.205726+0000 mon.a (mon.0) 4804 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:24.932 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:24 smithi144 bash[66654]: audit 2024-04-07T17:51:23.209158+0000 mon.a (mon.0) 4805 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:24.932 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:24 smithi144 bash[66654]: audit 2024-04-07T17:51:23.210008+0000 mon.a (mon.0) 4806 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:24.932 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:24 smithi144 bash[66654]: audit 2024-04-07T17:51:23.210437+0000 mon.a (mon.0) 4807 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:24.932 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:24 smithi144 bash[66654]: audit 2024-04-07T17:51:23.210938+0000 mon.b (mon.1) 25 : audit [INF] from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T17:51:24.932 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:24 smithi144 bash[66654]: audit 2024-04-07T17:51:23.211071+0000 mon.b (mon.1) 26 : audit [INF] from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T17:51:24.932 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:24 smithi144 bash[66654]: audit 2024-04-07T17:51:23.213065+0000 mon.c (mon.2) 123 : audit [INF] from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T17:51:24.932 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:24 smithi144 bash[66654]: audit 2024-04-07T17:51:23.213201+0000 mon.c (mon.2) 124 : audit [INF] from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T17:51:24.932 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:24 smithi144 bash[66654]: audit 2024-04-07T17:51:23.213521+0000 mon.a (mon.0) 4808 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:24.932 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:24 smithi144 bash[66654]: audit 2024-04-07T17:51:23.214270+0000 mon.a (mon.0) 4809 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:24.933 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:24 smithi144 bash[66654]: audit 2024-04-07T17:51:23.214610+0000 mon.a (mon.0) 4810 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:24.933 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:23 smithi100 bash[76202]: audit 2024-04-07T17:51:22.649344+0000 mon.a (mon.0) 4783 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:24.933 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:23 smithi100 bash[76202]: audit 2024-04-07T17:51:22.650343+0000 mon.a (mon.0) 4784 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:24.933 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:23 smithi100 bash[76202]: audit 2024-04-07T17:51:22.650792+0000 mon.a (mon.0) 4785 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:24.933 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:23 smithi100 bash[76202]: audit 2024-04-07T17:51:22.651171+0000 mon.b (mon.1) 23 : audit [INF] from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T17:51:24.933 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:23 smithi100 bash[76202]: audit 2024-04-07T17:51:22.651304+0000 mon.b (mon.1) 24 : audit [INF] from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T17:51:24.933 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:23 smithi100 bash[76202]: audit 2024-04-07T17:51:22.653707+0000 mon.c (mon.2) 121 : audit [INF] from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T17:51:24.933 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:23 smithi100 bash[76202]: audit 2024-04-07T17:51:22.653842+0000 mon.c (mon.2) 122 : audit [INF] from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T17:51:24.933 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:23 smithi100 bash[76202]: audit 2024-04-07T17:51:22.654170+0000 mon.a (mon.0) 4786 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:24.933 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:23 smithi100 bash[76202]: audit 2024-04-07T17:51:22.655048+0000 mon.a (mon.0) 4787 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:24.933 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:23 smithi100 bash[76202]: audit 2024-04-07T17:51:22.655411+0000 mon.a (mon.0) 4788 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:24.933 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:23 smithi100 bash[76202]: audit 2024-04-07T17:51:22.940207+0000 mon.a (mon.0) 4789 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:51:24.934 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:23 smithi100 bash[76202]: audit 2024-04-07T17:51:22.947642+0000 mon.a (mon.0) 4790 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config-key dump", "format": "json"}]: dispatch 2024-04-07T17:51:24.934 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:23 smithi100 bash[74942]: audit 2024-04-07T17:51:22.649344+0000 mon.a (mon.0) 4783 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:24.934 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:23 smithi100 bash[74942]: audit 2024-04-07T17:51:22.650343+0000 mon.a (mon.0) 4784 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:24.934 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:23 smithi100 bash[74942]: audit 2024-04-07T17:51:22.650792+0000 mon.a (mon.0) 4785 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:24.934 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:23 smithi100 bash[74942]: audit 2024-04-07T17:51:22.651171+0000 mon.b (mon.1) 23 : audit [INF] from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T17:51:24.934 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:23 smithi100 bash[74942]: audit 2024-04-07T17:51:22.651304+0000 mon.b (mon.1) 24 : audit [INF] from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T17:51:24.934 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:23 smithi100 bash[74942]: audit 2024-04-07T17:51:22.653707+0000 mon.c (mon.2) 121 : audit [INF] from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T17:51:24.934 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:23 smithi100 bash[74942]: audit 2024-04-07T17:51:22.653842+0000 mon.c (mon.2) 122 : audit [INF] from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T17:51:24.934 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:23 smithi100 bash[74942]: audit 2024-04-07T17:51:22.654170+0000 mon.a (mon.0) 4786 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:24.934 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:23 smithi100 bash[74942]: audit 2024-04-07T17:51:22.655048+0000 mon.a (mon.0) 4787 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:24.934 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:23 smithi100 bash[74942]: audit 2024-04-07T17:51:22.655411+0000 mon.a (mon.0) 4788 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:24.935 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:23 smithi100 bash[74942]: audit 2024-04-07T17:51:22.940207+0000 mon.a (mon.0) 4789 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:51:24.935 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:23 smithi100 bash[74942]: audit 2024-04-07T17:51:22.947642+0000 mon.a (mon.0) 4790 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config-key dump", "format": "json"}]: dispatch 2024-04-07T17:51:24.935 INFO:tasks.workunit.client.0.smithi100.stdout:device channel is on and up to date, see report with `ceph telemetry show-device`. 2024-04-07T17:51:24.935 INFO:tasks.workunit.client.0.smithi100.stderr:+ ceph telemetry preview-all 2024-04-07T17:51:24.935 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:24 smithi100 bash[76202]: audit 2024-04-07T17:51:22.417701+0000 mgr.y (mgr.25175) 2882 : audit [DBG] from='client.36063 -' entity='client.admin' cmd=[{"prefix": "telemetry preview", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T17:51:24.936 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:24 smithi100 bash[76202]: cluster 2024-04-07T17:51:22.828785+0000 mgr.y (mgr.25175) 2883 : cluster [DBG] pgmap v5168: 105 pgs: 105 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail 2024-04-07T17:51:24.936 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:24 smithi100 bash[76202]: audit 2024-04-07T17:51:22.937144+0000 mgr.y (mgr.25175) 2884 : audit [DBG] from='client.36063 -' entity='client.admin' cmd=[{"prefix": "telemetry preview", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T17:51:24.936 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:24 smithi100 bash[76202]: audit 2024-04-07T17:51:23.175495+0000 mon.a (mon.0) 4791 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:24.936 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:24 smithi100 bash[76202]: audit 2024-04-07T17:51:23.175974+0000 mon.a (mon.0) 4792 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:24.936 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:24 smithi100 bash[76202]: audit 2024-04-07T17:51:23.176376+0000 mon.a (mon.0) 4793 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:24.936 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:24 smithi100 bash[76202]: audit 2024-04-07T17:51:23.179807+0000 mon.a (mon.0) 4794 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:24.936 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:24 smithi100 bash[76202]: audit 2024-04-07T17:51:23.180280+0000 mon.a (mon.0) 4795 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:24.936 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:24 smithi100 bash[76202]: audit 2024-04-07T17:51:23.180694+0000 mon.a (mon.0) 4796 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:24.936 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:24 smithi100 bash[76202]: audit 2024-04-07T17:51:23.183755+0000 mon.a (mon.0) 4797 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:24.936 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:24 smithi100 bash[76202]: audit 2024-04-07T17:51:23.184294+0000 mon.a (mon.0) 4798 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:24.936 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:24 smithi100 bash[76202]: audit 2024-04-07T17:51:23.184797+0000 mon.a (mon.0) 4799 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:24.936 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:24 smithi100 bash[76202]: audit 2024-04-07T17:51:23.204303+0000 mon.a (mon.0) 4800 : audit [INF] from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T17:51:24.936 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:24 smithi100 bash[76202]: audit 2024-04-07T17:51:23.204409+0000 mon.a (mon.0) 4801 : audit [INF] from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T17:51:24.936 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:24 smithi100 bash[76202]: audit 2024-04-07T17:51:23.204752+0000 mon.a (mon.0) 4802 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:24.936 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:24 smithi100 bash[76202]: audit 2024-04-07T17:51:23.205338+0000 mon.a (mon.0) 4803 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:24.936 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:24 smithi100 bash[76202]: audit 2024-04-07T17:51:23.205726+0000 mon.a (mon.0) 4804 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:24.937 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:24 smithi100 bash[76202]: audit 2024-04-07T17:51:23.209158+0000 mon.a (mon.0) 4805 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:24.937 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:24 smithi100 bash[74942]: audit 2024-04-07T17:51:22.417701+0000 mgr.y (mgr.25175) 2882 : audit [DBG] from='client.36063 -' entity='client.admin' cmd=[{"prefix": "telemetry preview", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T17:51:24.937 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:24 smithi100 bash[74942]: cluster 2024-04-07T17:51:22.828785+0000 mgr.y (mgr.25175) 2883 : cluster [DBG] pgmap v5168: 105 pgs: 105 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail 2024-04-07T17:51:24.937 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:24 smithi100 bash[74942]: audit 2024-04-07T17:51:22.937144+0000 mgr.y (mgr.25175) 2884 : audit [DBG] from='client.36063 -' entity='client.admin' cmd=[{"prefix": "telemetry preview", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T17:51:24.937 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:24 smithi100 bash[74942]: audit 2024-04-07T17:51:23.175495+0000 mon.a (mon.0) 4791 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:24.937 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:24 smithi100 bash[74942]: audit 2024-04-07T17:51:23.175974+0000 mon.a (mon.0) 4792 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:24.937 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:24 smithi100 bash[74942]: audit 2024-04-07T17:51:23.176376+0000 mon.a (mon.0) 4793 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:24.937 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:24 smithi100 bash[74942]: audit 2024-04-07T17:51:23.179807+0000 mon.a (mon.0) 4794 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:24.937 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:24 smithi100 bash[74942]: audit 2024-04-07T17:51:23.180280+0000 mon.a (mon.0) 4795 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:24.937 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:24 smithi100 bash[74942]: audit 2024-04-07T17:51:23.180694+0000 mon.a (mon.0) 4796 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:24.937 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:24 smithi100 bash[74942]: audit 2024-04-07T17:51:23.183755+0000 mon.a (mon.0) 4797 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:24.937 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:24 smithi100 bash[74942]: audit 2024-04-07T17:51:23.184294+0000 mon.a (mon.0) 4798 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:24.938 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:24 smithi100 bash[74942]: audit 2024-04-07T17:51:23.184797+0000 mon.a (mon.0) 4799 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:24.938 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:24 smithi100 bash[74942]: audit 2024-04-07T17:51:23.204303+0000 mon.a (mon.0) 4800 : audit [INF] from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T17:51:24.938 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:24 smithi100 bash[74942]: audit 2024-04-07T17:51:23.204409+0000 mon.a (mon.0) 4801 : audit [INF] from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T17:51:24.938 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:24 smithi100 bash[74942]: audit 2024-04-07T17:51:23.204752+0000 mon.a (mon.0) 4802 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:24.938 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:24 smithi100 bash[74942]: audit 2024-04-07T17:51:23.205338+0000 mon.a (mon.0) 4803 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:24.938 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:24 smithi100 bash[74942]: audit 2024-04-07T17:51:23.205726+0000 mon.a (mon.0) 4804 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:24.938 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:24 smithi100 bash[74942]: audit 2024-04-07T17:51:23.209158+0000 mon.a (mon.0) 4805 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:24.938 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:24 smithi100 bash[76202]: audit 2024-04-07T17:51:23.210008+0000 mon.a (mon.0) 4806 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:24.938 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:24 smithi100 bash[76202]: audit 2024-04-07T17:51:23.210437+0000 mon.a (mon.0) 4807 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:24.939 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:24 smithi100 bash[76202]: audit 2024-04-07T17:51:23.210938+0000 mon.b (mon.1) 25 : audit [INF] from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T17:51:24.939 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:24 smithi100 bash[76202]: audit 2024-04-07T17:51:23.211071+0000 mon.b (mon.1) 26 : audit [INF] from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T17:51:24.939 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:24 smithi100 bash[76202]: audit 2024-04-07T17:51:23.213065+0000 mon.c (mon.2) 123 : audit [INF] from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T17:51:24.939 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:24 smithi100 bash[76202]: audit 2024-04-07T17:51:23.213201+0000 mon.c (mon.2) 124 : audit [INF] from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T17:51:24.939 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:24 smithi100 bash[76202]: audit 2024-04-07T17:51:23.213521+0000 mon.a (mon.0) 4808 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:24.939 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:24 smithi100 bash[76202]: audit 2024-04-07T17:51:23.214270+0000 mon.a (mon.0) 4809 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:24.939 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:24 smithi100 bash[76202]: audit 2024-04-07T17:51:23.214610+0000 mon.a (mon.0) 4810 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:24.939 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:24 smithi100 bash[74942]: audit 2024-04-07T17:51:23.210008+0000 mon.a (mon.0) 4806 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:24.939 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:24 smithi100 bash[74942]: audit 2024-04-07T17:51:23.210437+0000 mon.a (mon.0) 4807 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:24.939 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:24 smithi100 bash[74942]: audit 2024-04-07T17:51:23.210938+0000 mon.b (mon.1) 25 : audit [INF] from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T17:51:24.939 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:24 smithi100 bash[74942]: audit 2024-04-07T17:51:23.211071+0000 mon.b (mon.1) 26 : audit [INF] from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T17:51:24.939 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:24 smithi100 bash[74942]: audit 2024-04-07T17:51:23.213065+0000 mon.c (mon.2) 123 : audit [INF] from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T17:51:24.939 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:24 smithi100 bash[74942]: audit 2024-04-07T17:51:23.213201+0000 mon.c (mon.2) 124 : audit [INF] from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T17:51:24.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:24 smithi100 bash[74942]: audit 2024-04-07T17:51:23.213521+0000 mon.a (mon.0) 4808 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:24.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:24 smithi100 bash[74942]: audit 2024-04-07T17:51:23.214270+0000 mon.a (mon.0) 4809 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:24.940 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:24 smithi100 bash[74942]: audit 2024-04-07T17:51:23.214610+0000 mon.a (mon.0) 4810 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:25.258 INFO:tasks.workunit.client.0.smithi100.stdout:{ 2024-04-07T17:51:25.258 INFO:tasks.workunit.client.0.smithi100.stdout: "device_report": { 2024-04-07T17:51:25.259 INFO:tasks.workunit.client.0.smithi100.stdout: "251e7b66-f4fa-11ee-8447-0cc47a8fd733": { 2024-04-07T17:51:25.259 INFO:tasks.workunit.client.0.smithi100.stdout: "INTEL_SSDPEDMD400G4_251ffd38-f4fa-11ee-8447-0cc47a8fd733": { 2024-04-07T17:51:25.259 INFO:tasks.workunit.client.0.smithi100.stdout: "20240407-160729": { 2024-04-07T17:51:25.259 INFO:tasks.workunit.client.0.smithi100.stdout: "device": { 2024-04-07T17:51:25.259 INFO:tasks.workunit.client.0.smithi100.stdout: "info_name": "/dev/nvme0n1", 2024-04-07T17:51:25.259 INFO:tasks.workunit.client.0.smithi100.stdout: "name": "/dev/nvme0n1", 2024-04-07T17:51:25.259 INFO:tasks.workunit.client.0.smithi100.stdout: "protocol": "NVMe", 2024-04-07T17:51:25.259 INFO:tasks.workunit.client.0.smithi100.stdout: "type": "nvme" 2024-04-07T17:51:25.259 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.259 INFO:tasks.workunit.client.0.smithi100.stdout: "firmware_version": "8DV101H0", 2024-04-07T17:51:25.259 INFO:tasks.workunit.client.0.smithi100.stdout: "host_id": "251e7b66-f4fa-11ee-8447-0cc47a8fd733", 2024-04-07T17:51:25.259 INFO:tasks.workunit.client.0.smithi100.stdout: "json_format_version": [ 2024-04-07T17:51:25.259 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:25.259 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T17:51:25.259 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:25.260 INFO:tasks.workunit.client.0.smithi100.stdout: "local_time": { 2024-04-07T17:51:25.260 INFO:tasks.workunit.client.0.smithi100.stdout: "asctime": "Sun Apr 7 16:07:28 2024 UTC", 2024-04-07T17:51:25.260 INFO:tasks.workunit.client.0.smithi100.stdout: "time_t": 1712506048 2024-04-07T17:51:25.260 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.260 INFO:tasks.workunit.client.0.smithi100.stdout: "logical_block_size": 512, 2024-04-07T17:51:25.260 INFO:tasks.workunit.client.0.smithi100.stdout: "model_name": "INTEL SSDPEDMD400G4", 2024-04-07T17:51:25.260 INFO:tasks.workunit.client.0.smithi100.stdout: "nvme_controller_id": 0, 2024-04-07T17:51:25.260 INFO:tasks.workunit.client.0.smithi100.stdout: "nvme_ieee_oui_identifier": 6083300, 2024-04-07T17:51:25.260 INFO:tasks.workunit.client.0.smithi100.stdout: "nvme_namespaces": [ 2024-04-07T17:51:25.260 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:25.260 INFO:tasks.workunit.client.0.smithi100.stdout: "capacity": { 2024-04-07T17:51:25.260 INFO:tasks.workunit.client.0.smithi100.stdout: "blocks": 781422768, 2024-04-07T17:51:25.260 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 400088457216 2024-04-07T17:51:25.260 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.260 INFO:tasks.workunit.client.0.smithi100.stdout: "formatted_lba_size": 512, 2024-04-07T17:51:25.261 INFO:tasks.workunit.client.0.smithi100.stdout: "id": 1, 2024-04-07T17:51:25.261 INFO:tasks.workunit.client.0.smithi100.stdout: "size": { 2024-04-07T17:51:25.261 INFO:tasks.workunit.client.0.smithi100.stdout: "blocks": 781422768, 2024-04-07T17:51:25.261 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 400088457216 2024-04-07T17:51:25.261 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.261 INFO:tasks.workunit.client.0.smithi100.stdout: "utilization": { 2024-04-07T17:51:25.261 INFO:tasks.workunit.client.0.smithi100.stdout: "blocks": 781422768, 2024-04-07T17:51:25.261 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 400088457216 2024-04-07T17:51:25.261 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.261 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.261 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:25.261 INFO:tasks.workunit.client.0.smithi100.stdout: "nvme_number_of_namespaces": 1, 2024-04-07T17:51:25.261 INFO:tasks.workunit.client.0.smithi100.stdout: "nvme_pci_vendor": { 2024-04-07T17:51:25.261 INFO:tasks.workunit.client.0.smithi100.stdout: "id": 32902, 2024-04-07T17:51:25.261 INFO:tasks.workunit.client.0.smithi100.stdout: "subsystem_id": 32902 2024-04-07T17:51:25.262 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.262 INFO:tasks.workunit.client.0.smithi100.stdout: "nvme_smart_health_information_add_log": { 2024-04-07T17:51:25.262 INFO:tasks.workunit.client.0.smithi100.stdout: "Device stats": { 2024-04-07T17:51:25.262 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_fw_downgrades": { 2024-04-07T17:51:25.262 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 0, 2024-04-07T17:51:25.262 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 0 2024-04-07T17:51:25.262 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.262 INFO:tasks.workunit.client.0.smithi100.stdout: "crc_error_count": { 2024-04-07T17:51:25.262 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 100, 2024-04-07T17:51:25.262 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 0 2024-04-07T17:51:25.262 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.262 INFO:tasks.workunit.client.0.smithi100.stdout: "end_to_end_error_detection_count": { 2024-04-07T17:51:25.262 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 100, 2024-04-07T17:51:25.262 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 0 2024-04-07T17:51:25.262 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.263 INFO:tasks.workunit.client.0.smithi100.stdout: "erase_fail_count": { 2024-04-07T17:51:25.263 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 100, 2024-04-07T17:51:25.263 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 0 2024-04-07T17:51:25.263 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.263 INFO:tasks.workunit.client.0.smithi100.stdout: "host_bytes_written": { 2024-04-07T17:51:25.263 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 100, 2024-04-07T17:51:25.263 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 29568720 2024-04-07T17:51:25.263 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.263 INFO:tasks.workunit.client.0.smithi100.stdout: "host_ctx_wear_used": { 2024-04-07T17:51:25.263 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 100, 2024-04-07T17:51:25.263 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 1 2024-04-07T17:51:25.263 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.263 INFO:tasks.workunit.client.0.smithi100.stdout: "media_bytes_read": { 2024-04-07T17:51:25.263 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 0, 2024-04-07T17:51:25.264 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 0 2024-04-07T17:51:25.264 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.264 INFO:tasks.workunit.client.0.smithi100.stdout: "nand_bytes_written": { 2024-04-07T17:51:25.264 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 100, 2024-04-07T17:51:25.264 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 47802746 2024-04-07T17:51:25.264 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.264 INFO:tasks.workunit.client.0.smithi100.stdout: "perf_stat_indicator": { 2024-04-07T17:51:25.264 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 0, 2024-04-07T17:51:25.264 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 0 2024-04-07T17:51:25.264 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.264 INFO:tasks.workunit.client.0.smithi100.stdout: "pll_lock_loss_count": { 2024-04-07T17:51:25.264 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 100, 2024-04-07T17:51:25.264 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 0 2024-04-07T17:51:25.265 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.265 INFO:tasks.workunit.client.0.smithi100.stdout: "program_fail_count": { 2024-04-07T17:51:25.265 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 100, 2024-04-07T17:51:25.265 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 0 2024-04-07T17:51:25.265 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.265 INFO:tasks.workunit.client.0.smithi100.stdout: "re_alloc_sectr_cnt": { 2024-04-07T17:51:25.265 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 0, 2024-04-07T17:51:25.265 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 0 2024-04-07T17:51:25.265 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.265 INFO:tasks.workunit.client.0.smithi100.stdout: "retry_buffer_overflow_count": { 2024-04-07T17:51:25.265 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 100, 2024-04-07T17:51:25.265 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 0 2024-04-07T17:51:25.265 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.265 INFO:tasks.workunit.client.0.smithi100.stdout: "soft_ecc_err_rate": { 2024-04-07T17:51:25.265 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 0, 2024-04-07T17:51:25.266 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 0 2024-04-07T17:51:25.266 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.266 INFO:tasks.workunit.client.0.smithi100.stdout: "thermal_throttle_status": { 2024-04-07T17:51:25.266 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 100, 2024-04-07T17:51:25.266 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": { 2024-04-07T17:51:25.266 INFO:tasks.workunit.client.0.smithi100.stdout: "cnt": 0, 2024-04-07T17:51:25.266 INFO:tasks.workunit.client.0.smithi100.stdout: "pct": 0 2024-04-07T17:51:25.266 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.266 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.266 INFO:tasks.workunit.client.0.smithi100.stdout: "timed_workload_host_reads": { 2024-04-07T17:51:25.266 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 100, 2024-04-07T17:51:25.266 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 65535 2024-04-07T17:51:25.266 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.267 INFO:tasks.workunit.client.0.smithi100.stdout: "timed_workload_media_wear": { 2024-04-07T17:51:25.267 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 100, 2024-04-07T17:51:25.267 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 64 2024-04-07T17:51:25.267 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.267 INFO:tasks.workunit.client.0.smithi100.stdout: "timed_workload_timer": { 2024-04-07T17:51:25.267 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 100, 2024-04-07T17:51:25.267 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 65535 2024-04-07T17:51:25.267 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.267 INFO:tasks.workunit.client.0.smithi100.stdout: "unexp_power_loss": { 2024-04-07T17:51:25.267 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 0, 2024-04-07T17:51:25.267 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 0 2024-04-07T17:51:25.267 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.267 INFO:tasks.workunit.client.0.smithi100.stdout: "wear_leveling": { 2024-04-07T17:51:25.267 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 90, 2024-04-07T17:51:25.267 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": { 2024-04-07T17:51:25.268 INFO:tasks.workunit.client.0.smithi100.stdout: "avg": 3209, 2024-04-07T17:51:25.268 INFO:tasks.workunit.client.0.smithi100.stdout: "max": 3225, 2024-04-07T17:51:25.268 INFO:tasks.workunit.client.0.smithi100.stdout: "min": 3194 2024-04-07T17:51:25.268 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.268 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.268 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.268 INFO:tasks.workunit.client.0.smithi100.stdout: "Intel Smart log": "nvme0n1" 2024-04-07T17:51:25.268 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.268 INFO:tasks.workunit.client.0.smithi100.stdout: "nvme_smart_health_information_log": { 2024-04-07T17:51:25.268 INFO:tasks.workunit.client.0.smithi100.stdout: "available_spare": 100, 2024-04-07T17:51:25.268 INFO:tasks.workunit.client.0.smithi100.stdout: "available_spare_threshold": 10, 2024-04-07T17:51:25.268 INFO:tasks.workunit.client.0.smithi100.stdout: "controller_busy_time": 4, 2024-04-07T17:51:25.268 INFO:tasks.workunit.client.0.smithi100.stdout: "critical_warning": 0, 2024-04-07T17:51:25.268 INFO:tasks.workunit.client.0.smithi100.stdout: "data_units_read": 385784931, 2024-04-07T17:51:25.268 INFO:tasks.workunit.client.0.smithi100.stdout: "data_units_written": 1937815661, 2024-04-07T17:51:25.269 INFO:tasks.workunit.client.0.smithi100.stdout: "host_reads": 5771442418, 2024-04-07T17:51:25.269 INFO:tasks.workunit.client.0.smithi100.stdout: "host_writes": 31722089956, 2024-04-07T17:51:25.269 INFO:tasks.workunit.client.0.smithi100.stdout: "media_errors": 0, 2024-04-07T17:51:25.269 INFO:tasks.workunit.client.0.smithi100.stdout: "num_err_log_entries": 0, 2024-04-07T17:51:25.269 INFO:tasks.workunit.client.0.smithi100.stdout: "percentage_used": 10, 2024-04-07T17:51:25.269 INFO:tasks.workunit.client.0.smithi100.stdout: "power_cycles": 45774, 2024-04-07T17:51:25.269 INFO:tasks.workunit.client.0.smithi100.stdout: "power_on_hours": 57978, 2024-04-07T17:51:25.269 INFO:tasks.workunit.client.0.smithi100.stdout: "temperature": 24, 2024-04-07T17:51:25.269 INFO:tasks.workunit.client.0.smithi100.stdout: "unsafe_shutdowns": 210 2024-04-07T17:51:25.269 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.269 INFO:tasks.workunit.client.0.smithi100.stdout: "nvme_vendor": "intel", 2024-04-07T17:51:25.269 INFO:tasks.workunit.client.0.smithi100.stdout: "power_cycle_count": 45774, 2024-04-07T17:51:25.269 INFO:tasks.workunit.client.0.smithi100.stdout: "power_on_time": { 2024-04-07T17:51:25.269 INFO:tasks.workunit.client.0.smithi100.stdout: "hours": 57978 2024-04-07T17:51:25.269 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.270 INFO:tasks.workunit.client.0.smithi100.stdout: "serial_number": "deleted", 2024-04-07T17:51:25.270 INFO:tasks.workunit.client.0.smithi100.stdout: "smart_status": { 2024-04-07T17:51:25.270 INFO:tasks.workunit.client.0.smithi100.stdout: "nvme": { 2024-04-07T17:51:25.270 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.270 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.270 INFO:tasks.workunit.client.0.smithi100.stdout: "passed": true 2024-04-07T17:51:25.270 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.270 INFO:tasks.workunit.client.0.smithi100.stdout: "smartctl": { 2024-04-07T17:51:25.270 INFO:tasks.workunit.client.0.smithi100.stdout: "argv": [ 2024-04-07T17:51:25.270 INFO:tasks.workunit.client.0.smithi100.stdout: "smartctl", 2024-04-07T17:51:25.270 INFO:tasks.workunit.client.0.smithi100.stdout: "-x", 2024-04-07T17:51:25.270 INFO:tasks.workunit.client.0.smithi100.stdout: "--json=o", 2024-04-07T17:51:25.270 INFO:tasks.workunit.client.0.smithi100.stdout: "/dev/nvme0n1" 2024-04-07T17:51:25.270 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:25.270 INFO:tasks.workunit.client.0.smithi100.stdout: "build_info": "(local build)", 2024-04-07T17:51:25.271 INFO:tasks.workunit.client.0.smithi100.stdout: "exit_status": 0, 2024-04-07T17:51:25.271 INFO:tasks.workunit.client.0.smithi100.stdout: "output": [ 2024-04-07T17:51:25.271 INFO:tasks.workunit.client.0.smithi100.stdout: "smartctl 7.1 2019-12-30 r5022 [x86_64-linux-5.15.0-101-generic] (local build)", 2024-04-07T17:51:25.271 INFO:tasks.workunit.client.0.smithi100.stdout: "Copyright (C) 2002-19, Bruce Allen, Christian Franke, www.smartmontools.org", 2024-04-07T17:51:25.271 INFO:tasks.workunit.client.0.smithi100.stdout: "", 2024-04-07T17:51:25.271 INFO:tasks.workunit.client.0.smithi100.stdout: "=== START OF INFORMATION SECTION ===", 2024-04-07T17:51:25.271 INFO:tasks.workunit.client.0.smithi100.stdout: "Model Number: INTEL SSDPEDMD400G4", 2024-04-07T17:51:25.271 INFO:tasks.workunit.client.0.smithi100.stdout: "Serial Number: deleted", 2024-04-07T17:51:25.271 INFO:tasks.workunit.client.0.smithi100.stdout: "Firmware Version: 8DV101H0", 2024-04-07T17:51:25.271 INFO:tasks.workunit.client.0.smithi100.stdout: "PCI Vendor/Subsystem ID: 0x8086", 2024-04-07T17:51:25.271 INFO:tasks.workunit.client.0.smithi100.stdout: "IEEE OUI Identifier: 0x5cd2e4", 2024-04-07T17:51:25.271 INFO:tasks.workunit.client.0.smithi100.stdout: "Controller ID: 0", 2024-04-07T17:51:25.271 INFO:tasks.workunit.client.0.smithi100.stdout: "Number of Namespaces: 1", 2024-04-07T17:51:25.271 INFO:tasks.workunit.client.0.smithi100.stdout: "Namespace 1 Size/Capacity: 400,088,457,216 [400 GB]", 2024-04-07T17:51:25.272 INFO:tasks.workunit.client.0.smithi100.stdout: "Namespace 1 Formatted LBA Size: 512", 2024-04-07T17:51:25.272 INFO:tasks.workunit.client.0.smithi100.stdout: "Local Time is: Sun Apr 7 16:07:28 2024 UTC", 2024-04-07T17:51:25.272 INFO:tasks.workunit.client.0.smithi100.stdout: "Firmware Updates (0x02): 1 Slot", 2024-04-07T17:51:25.272 INFO:tasks.workunit.client.0.smithi100.stdout: "Optional Admin Commands (0x0006): Format Frmw_DL", 2024-04-07T17:51:25.272 INFO:tasks.workunit.client.0.smithi100.stdout: "Optional NVM Commands (0x0006): Wr_Unc DS_Mngmt", 2024-04-07T17:51:25.272 INFO:tasks.workunit.client.0.smithi100.stdout: "Maximum Data Transfer Size: 32 Pages", 2024-04-07T17:51:25.272 INFO:tasks.workunit.client.0.smithi100.stdout: "", 2024-04-07T17:51:25.272 INFO:tasks.workunit.client.0.smithi100.stdout: "Supported Power States", 2024-04-07T17:51:25.272 INFO:tasks.workunit.client.0.smithi100.stdout: "St Op Max Active Idle RL RT WL WT Ent_Lat Ex_Lat", 2024-04-07T17:51:25.272 INFO:tasks.workunit.client.0.smithi100.stdout: " 0 + 25.00W - - 0 0 0 0 0 0", 2024-04-07T17:51:25.272 INFO:tasks.workunit.client.0.smithi100.stdout: "", 2024-04-07T17:51:25.272 INFO:tasks.workunit.client.0.smithi100.stdout: "Supported LBA Sizes (NSID 0x1)", 2024-04-07T17:51:25.272 INFO:tasks.workunit.client.0.smithi100.stdout: "Id Fmt Data Metadt Rel_Perf", 2024-04-07T17:51:25.272 INFO:tasks.workunit.client.0.smithi100.stdout: " 0 + 512 0 2", 2024-04-07T17:51:25.272 INFO:tasks.workunit.client.0.smithi100.stdout: " 1 - 512 8 2", 2024-04-07T17:51:25.272 INFO:tasks.workunit.client.0.smithi100.stdout: " 2 - 512 16 2", 2024-04-07T17:51:25.273 INFO:tasks.workunit.client.0.smithi100.stdout: " 3 - 4096 0 0", 2024-04-07T17:51:25.273 INFO:tasks.workunit.client.0.smithi100.stdout: " 4 - 4096 8 0", 2024-04-07T17:51:25.273 INFO:tasks.workunit.client.0.smithi100.stdout: " 5 - 4096 64 0", 2024-04-07T17:51:25.273 INFO:tasks.workunit.client.0.smithi100.stdout: " 6 - 4096 128 0", 2024-04-07T17:51:25.273 INFO:tasks.workunit.client.0.smithi100.stdout: "", 2024-04-07T17:51:25.273 INFO:tasks.workunit.client.0.smithi100.stdout: "=== START OF SMART DATA SECTION ===", 2024-04-07T17:51:25.273 INFO:tasks.workunit.client.0.smithi100.stdout: "SMART overall-health self-assessment test result: PASSED", 2024-04-07T17:51:25.273 INFO:tasks.workunit.client.0.smithi100.stdout: "", 2024-04-07T17:51:25.273 INFO:tasks.workunit.client.0.smithi100.stdout: "SMART/Health Information (NVMe Log 0x02)", 2024-04-07T17:51:25.273 INFO:tasks.workunit.client.0.smithi100.stdout: "Critical Warning: 0x00", 2024-04-07T17:51:25.273 INFO:tasks.workunit.client.0.smithi100.stdout: "Temperature: 24 Celsius", 2024-04-07T17:51:25.273 INFO:tasks.workunit.client.0.smithi100.stdout: "Available Spare: 100%", 2024-04-07T17:51:25.273 INFO:tasks.workunit.client.0.smithi100.stdout: "Available Spare Threshold: 10%", 2024-04-07T17:51:25.273 INFO:tasks.workunit.client.0.smithi100.stdout: "Percentage Used: 10%", 2024-04-07T17:51:25.274 INFO:tasks.workunit.client.0.smithi100.stdout: "Data Units Read: 385,784,931 [197 TB]", 2024-04-07T17:51:25.274 INFO:tasks.workunit.client.0.smithi100.stdout: "Data Units Written: 1,937,815,661 [992 TB]", 2024-04-07T17:51:25.274 INFO:tasks.workunit.client.0.smithi100.stdout: "Host Read Commands: 5,771,442,418", 2024-04-07T17:51:25.274 INFO:tasks.workunit.client.0.smithi100.stdout: "Host Write Commands: 31,722,089,956", 2024-04-07T17:51:25.274 INFO:tasks.workunit.client.0.smithi100.stdout: "Controller Busy Time: 4", 2024-04-07T17:51:25.274 INFO:tasks.workunit.client.0.smithi100.stdout: "Power Cycles: 45,774", 2024-04-07T17:51:25.274 INFO:tasks.workunit.client.0.smithi100.stdout: "Power On Hours: 57,978", 2024-04-07T17:51:25.274 INFO:tasks.workunit.client.0.smithi100.stdout: "Unsafe Shutdowns: 210", 2024-04-07T17:51:25.274 INFO:tasks.workunit.client.0.smithi100.stdout: "Media and Data Integrity Errors: 0", 2024-04-07T17:51:25.274 INFO:tasks.workunit.client.0.smithi100.stdout: "Error Information Log Entries: 0", 2024-04-07T17:51:25.274 INFO:tasks.workunit.client.0.smithi100.stdout: "", 2024-04-07T17:51:25.274 INFO:tasks.workunit.client.0.smithi100.stdout: "Error Information (NVMe Log 0x01, max 64 entries)", 2024-04-07T17:51:25.275 INFO:tasks.workunit.client.0.smithi100.stdout: "No Errors Logged", 2024-04-07T17:51:25.275 INFO:tasks.workunit.client.0.smithi100.stdout: "" 2024-04-07T17:51:25.275 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:25.275 INFO:tasks.workunit.client.0.smithi100.stdout: "platform_info": "x86_64-linux-5.15.0-101-generic", 2024-04-07T17:51:25.275 INFO:tasks.workunit.client.0.smithi100.stdout: "svn_revision": "5022", 2024-04-07T17:51:25.275 INFO:tasks.workunit.client.0.smithi100.stdout: "version": [ 2024-04-07T17:51:25.275 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:25.275 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T17:51:25.275 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:25.275 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.275 INFO:tasks.workunit.client.0.smithi100.stdout: "temperature": { 2024-04-07T17:51:25.275 INFO:tasks.workunit.client.0.smithi100.stdout: "current": 24 2024-04-07T17:51:25.276 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.276 INFO:tasks.workunit.client.0.smithi100.stdout: "user_capacity": { 2024-04-07T17:51:25.276 INFO:tasks.workunit.client.0.smithi100.stdout: "blocks": 781422768, 2024-04-07T17:51:25.276 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 400088457216 2024-04-07T17:51:25.276 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.276 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.276 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.276 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.276 INFO:tasks.workunit.client.0.smithi100.stdout: "25233610-f4fa-11ee-8447-0cc47a8fd733": { 2024-04-07T17:51:25.277 INFO:tasks.workunit.client.0.smithi100.stdout: "INTEL_SSDPEDMD400G4_2524b6c0-f4fa-11ee-8447-0cc47a8fd733": { 2024-04-07T17:51:25.277 INFO:tasks.workunit.client.0.smithi100.stdout: "20240407-160732": { 2024-04-07T17:51:25.277 INFO:tasks.workunit.client.0.smithi100.stdout: "device": { 2024-04-07T17:51:25.277 INFO:tasks.workunit.client.0.smithi100.stdout: "info_name": "/dev/nvme0n1", 2024-04-07T17:51:25.277 INFO:tasks.workunit.client.0.smithi100.stdout: "name": "/dev/nvme0n1", 2024-04-07T17:51:25.277 INFO:tasks.workunit.client.0.smithi100.stdout: "protocol": "NVMe", 2024-04-07T17:51:25.277 INFO:tasks.workunit.client.0.smithi100.stdout: "type": "nvme" 2024-04-07T17:51:25.277 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.277 INFO:tasks.workunit.client.0.smithi100.stdout: "firmware_version": "8DV101H0", 2024-04-07T17:51:25.277 INFO:tasks.workunit.client.0.smithi100.stdout: "host_id": "25233610-f4fa-11ee-8447-0cc47a8fd733", 2024-04-07T17:51:25.277 INFO:tasks.workunit.client.0.smithi100.stdout: "json_format_version": [ 2024-04-07T17:51:25.277 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:25.277 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T17:51:25.278 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:25.278 INFO:tasks.workunit.client.0.smithi100.stdout: "local_time": { 2024-04-07T17:51:25.278 INFO:tasks.workunit.client.0.smithi100.stdout: "asctime": "Sun Apr 7 16:07:32 2024 UTC", 2024-04-07T17:51:25.278 INFO:tasks.workunit.client.0.smithi100.stdout: "time_t": 1712506052 2024-04-07T17:51:25.278 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.278 INFO:tasks.workunit.client.0.smithi100.stdout: "logical_block_size": 512, 2024-04-07T17:51:25.278 INFO:tasks.workunit.client.0.smithi100.stdout: "model_name": "INTEL SSDPEDMD400G4", 2024-04-07T17:51:25.278 INFO:tasks.workunit.client.0.smithi100.stdout: "nvme_controller_id": 0, 2024-04-07T17:51:25.278 INFO:tasks.workunit.client.0.smithi100.stdout: "nvme_ieee_oui_identifier": 6083300, 2024-04-07T17:51:25.278 INFO:tasks.workunit.client.0.smithi100.stdout: "nvme_namespaces": [ 2024-04-07T17:51:25.278 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:25.278 INFO:tasks.workunit.client.0.smithi100.stdout: "capacity": { 2024-04-07T17:51:25.278 INFO:tasks.workunit.client.0.smithi100.stdout: "blocks": 781422768, 2024-04-07T17:51:25.278 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 400088457216 2024-04-07T17:51:25.278 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.279 INFO:tasks.workunit.client.0.smithi100.stdout: "formatted_lba_size": 512, 2024-04-07T17:51:25.279 INFO:tasks.workunit.client.0.smithi100.stdout: "id": 1, 2024-04-07T17:51:25.279 INFO:tasks.workunit.client.0.smithi100.stdout: "size": { 2024-04-07T17:51:25.279 INFO:tasks.workunit.client.0.smithi100.stdout: "blocks": 781422768, 2024-04-07T17:51:25.279 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 400088457216 2024-04-07T17:51:25.279 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.279 INFO:tasks.workunit.client.0.smithi100.stdout: "utilization": { 2024-04-07T17:51:25.279 INFO:tasks.workunit.client.0.smithi100.stdout: "blocks": 781422768, 2024-04-07T17:51:25.279 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 400088457216 2024-04-07T17:51:25.279 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.279 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.279 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:25.279 INFO:tasks.workunit.client.0.smithi100.stdout: "nvme_number_of_namespaces": 1, 2024-04-07T17:51:25.279 INFO:tasks.workunit.client.0.smithi100.stdout: "nvme_pci_vendor": { 2024-04-07T17:51:25.280 INFO:tasks.workunit.client.0.smithi100.stdout: "id": 32902, 2024-04-07T17:51:25.280 INFO:tasks.workunit.client.0.smithi100.stdout: "subsystem_id": 32902 2024-04-07T17:51:25.280 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.280 INFO:tasks.workunit.client.0.smithi100.stdout: "nvme_smart_health_information_add_log": { 2024-04-07T17:51:25.280 INFO:tasks.workunit.client.0.smithi100.stdout: "Device stats": { 2024-04-07T17:51:25.280 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_fw_downgrades": { 2024-04-07T17:51:25.280 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 0, 2024-04-07T17:51:25.280 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 0 2024-04-07T17:51:25.280 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.280 INFO:tasks.workunit.client.0.smithi100.stdout: "crc_error_count": { 2024-04-07T17:51:25.280 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 100, 2024-04-07T17:51:25.280 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 0 2024-04-07T17:51:25.280 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.280 INFO:tasks.workunit.client.0.smithi100.stdout: "end_to_end_error_detection_count": { 2024-04-07T17:51:25.280 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 100, 2024-04-07T17:51:25.281 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 0 2024-04-07T17:51:25.281 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.281 INFO:tasks.workunit.client.0.smithi100.stdout: "erase_fail_count": { 2024-04-07T17:51:25.281 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 100, 2024-04-07T17:51:25.281 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 0 2024-04-07T17:51:25.281 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.281 INFO:tasks.workunit.client.0.smithi100.stdout: "host_bytes_written": { 2024-04-07T17:51:25.281 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 100, 2024-04-07T17:51:25.281 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 25060599 2024-04-07T17:51:25.281 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.281 INFO:tasks.workunit.client.0.smithi100.stdout: "host_ctx_wear_used": { 2024-04-07T17:51:25.281 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 100, 2024-04-07T17:51:25.281 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 1 2024-04-07T17:51:25.281 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.281 INFO:tasks.workunit.client.0.smithi100.stdout: "media_bytes_read": { 2024-04-07T17:51:25.281 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 0, 2024-04-07T17:51:25.282 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 0 2024-04-07T17:51:25.282 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.282 INFO:tasks.workunit.client.0.smithi100.stdout: "nand_bytes_written": { 2024-04-07T17:51:25.282 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 100, 2024-04-07T17:51:25.282 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 42751216 2024-04-07T17:51:25.282 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.282 INFO:tasks.workunit.client.0.smithi100.stdout: "perf_stat_indicator": { 2024-04-07T17:51:25.282 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 0, 2024-04-07T17:51:25.282 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 0 2024-04-07T17:51:25.282 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.282 INFO:tasks.workunit.client.0.smithi100.stdout: "pll_lock_loss_count": { 2024-04-07T17:51:25.282 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 100, 2024-04-07T17:51:25.282 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 0 2024-04-07T17:51:25.282 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.283 INFO:tasks.workunit.client.0.smithi100.stdout: "program_fail_count": { 2024-04-07T17:51:25.283 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 100, 2024-04-07T17:51:25.283 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 0 2024-04-07T17:51:25.283 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.283 INFO:tasks.workunit.client.0.smithi100.stdout: "re_alloc_sectr_cnt": { 2024-04-07T17:51:25.283 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 0, 2024-04-07T17:51:25.283 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 0 2024-04-07T17:51:25.283 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.283 INFO:tasks.workunit.client.0.smithi100.stdout: "retry_buffer_overflow_count": { 2024-04-07T17:51:25.283 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 100, 2024-04-07T17:51:25.283 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 0 2024-04-07T17:51:25.283 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.283 INFO:tasks.workunit.client.0.smithi100.stdout: "soft_ecc_err_rate": { 2024-04-07T17:51:25.283 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 0, 2024-04-07T17:51:25.283 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 0 2024-04-07T17:51:25.283 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.284 INFO:tasks.workunit.client.0.smithi100.stdout: "thermal_throttle_status": { 2024-04-07T17:51:25.284 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 100, 2024-04-07T17:51:25.284 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": { 2024-04-07T17:51:25.284 INFO:tasks.workunit.client.0.smithi100.stdout: "cnt": 0, 2024-04-07T17:51:25.284 INFO:tasks.workunit.client.0.smithi100.stdout: "pct": 0 2024-04-07T17:51:25.284 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.284 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.284 INFO:tasks.workunit.client.0.smithi100.stdout: "timed_workload_host_reads": { 2024-04-07T17:51:25.284 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 100, 2024-04-07T17:51:25.284 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 65535 2024-04-07T17:51:25.284 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.284 INFO:tasks.workunit.client.0.smithi100.stdout: "timed_workload_media_wear": { 2024-04-07T17:51:25.284 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 100, 2024-04-07T17:51:25.285 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 64 2024-04-07T17:51:25.285 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.285 INFO:tasks.workunit.client.0.smithi100.stdout: "timed_workload_timer": { 2024-04-07T17:51:25.285 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 100, 2024-04-07T17:51:25.285 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 65535 2024-04-07T17:51:25.285 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.285 INFO:tasks.workunit.client.0.smithi100.stdout: "unexp_power_loss": { 2024-04-07T17:51:25.285 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 0, 2024-04-07T17:51:25.285 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 0 2024-04-07T17:51:25.285 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.285 INFO:tasks.workunit.client.0.smithi100.stdout: "wear_leveling": { 2024-04-07T17:51:25.285 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 91, 2024-04-07T17:51:25.286 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": { 2024-04-07T17:51:25.286 INFO:tasks.workunit.client.0.smithi100.stdout: "avg": 2895, 2024-04-07T17:51:25.286 INFO:tasks.workunit.client.0.smithi100.stdout: "max": 2907, 2024-04-07T17:51:25.286 INFO:tasks.workunit.client.0.smithi100.stdout: "min": 2884 2024-04-07T17:51:25.286 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.286 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.286 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.286 INFO:tasks.workunit.client.0.smithi100.stdout: "Intel Smart log": "nvme0n1" 2024-04-07T17:51:25.286 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.286 INFO:tasks.workunit.client.0.smithi100.stdout: "nvme_smart_health_information_log": { 2024-04-07T17:51:25.286 INFO:tasks.workunit.client.0.smithi100.stdout: "available_spare": 100, 2024-04-07T17:51:25.287 INFO:tasks.workunit.client.0.smithi100.stdout: "available_spare_threshold": 10, 2024-04-07T17:51:25.287 INFO:tasks.workunit.client.0.smithi100.stdout: "controller_busy_time": 0, 2024-04-07T17:51:25.287 INFO:tasks.workunit.client.0.smithi100.stdout: "critical_warning": 0, 2024-04-07T17:51:25.287 INFO:tasks.workunit.client.0.smithi100.stdout: "data_units_read": 359011585, 2024-04-07T17:51:25.287 INFO:tasks.workunit.client.0.smithi100.stdout: "data_units_written": 1642371481, 2024-04-07T17:51:25.287 INFO:tasks.workunit.client.0.smithi100.stdout: "host_reads": 5750712954, 2024-04-07T17:51:25.287 INFO:tasks.workunit.client.0.smithi100.stdout: "host_writes": 28276536650, 2024-04-07T17:51:25.287 INFO:tasks.workunit.client.0.smithi100.stdout: "media_errors": 0, 2024-04-07T17:51:25.287 INFO:tasks.workunit.client.0.smithi100.stdout: "num_err_log_entries": 0, 2024-04-07T17:51:25.287 INFO:tasks.workunit.client.0.smithi100.stdout: "percentage_used": 9, 2024-04-07T17:51:25.287 INFO:tasks.workunit.client.0.smithi100.stdout: "power_cycles": 47059, 2024-04-07T17:51:25.287 INFO:tasks.workunit.client.0.smithi100.stdout: "power_on_hours": 54821, 2024-04-07T17:51:25.287 INFO:tasks.workunit.client.0.smithi100.stdout: "temperature": 25, 2024-04-07T17:51:25.287 INFO:tasks.workunit.client.0.smithi100.stdout: "unsafe_shutdowns": 12 2024-04-07T17:51:25.288 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.288 INFO:tasks.workunit.client.0.smithi100.stdout: "nvme_vendor": "intel", 2024-04-07T17:51:25.288 INFO:tasks.workunit.client.0.smithi100.stdout: "power_cycle_count": 47059, 2024-04-07T17:51:25.288 INFO:tasks.workunit.client.0.smithi100.stdout: "power_on_time": { 2024-04-07T17:51:25.288 INFO:tasks.workunit.client.0.smithi100.stdout: "hours": 54821 2024-04-07T17:51:25.288 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.288 INFO:tasks.workunit.client.0.smithi100.stdout: "serial_number": "deleted", 2024-04-07T17:51:25.288 INFO:tasks.workunit.client.0.smithi100.stdout: "smart_status": { 2024-04-07T17:51:25.288 INFO:tasks.workunit.client.0.smithi100.stdout: "nvme": { 2024-04-07T17:51:25.288 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.288 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.288 INFO:tasks.workunit.client.0.smithi100.stdout: "passed": true 2024-04-07T17:51:25.288 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.288 INFO:tasks.workunit.client.0.smithi100.stdout: "smartctl": { 2024-04-07T17:51:25.288 INFO:tasks.workunit.client.0.smithi100.stdout: "argv": [ 2024-04-07T17:51:25.288 INFO:tasks.workunit.client.0.smithi100.stdout: "smartctl", 2024-04-07T17:51:25.289 INFO:tasks.workunit.client.0.smithi100.stdout: "-x", 2024-04-07T17:51:25.289 INFO:tasks.workunit.client.0.smithi100.stdout: "--json=o", 2024-04-07T17:51:25.289 INFO:tasks.workunit.client.0.smithi100.stdout: "/dev/nvme0n1" 2024-04-07T17:51:25.289 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:25.289 INFO:tasks.workunit.client.0.smithi100.stdout: "build_info": "(local build)", 2024-04-07T17:51:25.289 INFO:tasks.workunit.client.0.smithi100.stdout: "exit_status": 0, 2024-04-07T17:51:25.289 INFO:tasks.workunit.client.0.smithi100.stdout: "output": [ 2024-04-07T17:51:25.289 INFO:tasks.workunit.client.0.smithi100.stdout: "smartctl 7.1 2019-12-30 r5022 [x86_64-linux-5.15.0-101-generic] (local build)", 2024-04-07T17:51:25.289 INFO:tasks.workunit.client.0.smithi100.stdout: "Copyright (C) 2002-19, Bruce Allen, Christian Franke, www.smartmontools.org", 2024-04-07T17:51:25.289 INFO:tasks.workunit.client.0.smithi100.stdout: "", 2024-04-07T17:51:25.289 INFO:tasks.workunit.client.0.smithi100.stdout: "=== START OF INFORMATION SECTION ===", 2024-04-07T17:51:25.289 INFO:tasks.workunit.client.0.smithi100.stdout: "Model Number: INTEL SSDPEDMD400G4", 2024-04-07T17:51:25.289 INFO:tasks.workunit.client.0.smithi100.stdout: "Serial Number: deleted", 2024-04-07T17:51:25.289 INFO:tasks.workunit.client.0.smithi100.stdout: "Firmware Version: 8DV101H0", 2024-04-07T17:51:25.289 INFO:tasks.workunit.client.0.smithi100.stdout: "PCI Vendor/Subsystem ID: 0x8086", 2024-04-07T17:51:25.290 INFO:tasks.workunit.client.0.smithi100.stdout: "IEEE OUI Identifier: 0x5cd2e4", 2024-04-07T17:51:25.290 INFO:tasks.workunit.client.0.smithi100.stdout: "Controller ID: 0", 2024-04-07T17:51:25.290 INFO:tasks.workunit.client.0.smithi100.stdout: "Number of Namespaces: 1", 2024-04-07T17:51:25.290 INFO:tasks.workunit.client.0.smithi100.stdout: "Namespace 1 Size/Capacity: 400,088,457,216 [400 GB]", 2024-04-07T17:51:25.290 INFO:tasks.workunit.client.0.smithi100.stdout: "Namespace 1 Formatted LBA Size: 512", 2024-04-07T17:51:25.290 INFO:tasks.workunit.client.0.smithi100.stdout: "Local Time is: Sun Apr 7 16:07:32 2024 UTC", 2024-04-07T17:51:25.290 INFO:tasks.workunit.client.0.smithi100.stdout: "Firmware Updates (0x02): 1 Slot", 2024-04-07T17:51:25.290 INFO:tasks.workunit.client.0.smithi100.stdout: "Optional Admin Commands (0x0006): Format Frmw_DL", 2024-04-07T17:51:25.290 INFO:tasks.workunit.client.0.smithi100.stdout: "Optional NVM Commands (0x0006): Wr_Unc DS_Mngmt", 2024-04-07T17:51:25.290 INFO:tasks.workunit.client.0.smithi100.stdout: "Maximum Data Transfer Size: 32 Pages", 2024-04-07T17:51:25.290 INFO:tasks.workunit.client.0.smithi100.stdout: "", 2024-04-07T17:51:25.290 INFO:tasks.workunit.client.0.smithi100.stdout: "Supported Power States", 2024-04-07T17:51:25.291 INFO:tasks.workunit.client.0.smithi100.stdout: "St Op Max Active Idle RL RT WL WT Ent_Lat Ex_Lat", 2024-04-07T17:51:25.291 INFO:tasks.workunit.client.0.smithi100.stdout: " 0 + 25.00W - - 0 0 0 0 0 0", 2024-04-07T17:51:25.291 INFO:tasks.workunit.client.0.smithi100.stdout: "", 2024-04-07T17:51:25.291 INFO:tasks.workunit.client.0.smithi100.stdout: "Supported LBA Sizes (NSID 0x1)", 2024-04-07T17:51:25.291 INFO:tasks.workunit.client.0.smithi100.stdout: "Id Fmt Data Metadt Rel_Perf", 2024-04-07T17:51:25.291 INFO:tasks.workunit.client.0.smithi100.stdout: " 0 + 512 0 2", 2024-04-07T17:51:25.291 INFO:tasks.workunit.client.0.smithi100.stdout: " 1 - 512 8 2", 2024-04-07T17:51:25.291 INFO:tasks.workunit.client.0.smithi100.stdout: " 2 - 512 16 2", 2024-04-07T17:51:25.291 INFO:tasks.workunit.client.0.smithi100.stdout: " 3 - 4096 0 0", 2024-04-07T17:51:25.291 INFO:tasks.workunit.client.0.smithi100.stdout: " 4 - 4096 8 0", 2024-04-07T17:51:25.291 INFO:tasks.workunit.client.0.smithi100.stdout: " 5 - 4096 64 0", 2024-04-07T17:51:25.291 INFO:tasks.workunit.client.0.smithi100.stdout: " 6 - 4096 128 0", 2024-04-07T17:51:25.291 INFO:tasks.workunit.client.0.smithi100.stdout: "", 2024-04-07T17:51:25.291 INFO:tasks.workunit.client.0.smithi100.stdout: "=== START OF SMART DATA SECTION ===", 2024-04-07T17:51:25.291 INFO:tasks.workunit.client.0.smithi100.stdout: "SMART overall-health self-assessment test result: PASSED", 2024-04-07T17:51:25.291 INFO:tasks.workunit.client.0.smithi100.stdout: "", 2024-04-07T17:51:25.292 INFO:tasks.workunit.client.0.smithi100.stdout: "SMART/Health Information (NVMe Log 0x02)", 2024-04-07T17:51:25.292 INFO:tasks.workunit.client.0.smithi100.stdout: "Critical Warning: 0x00", 2024-04-07T17:51:25.292 INFO:tasks.workunit.client.0.smithi100.stdout: "Temperature: 25 Celsius", 2024-04-07T17:51:25.292 INFO:tasks.workunit.client.0.smithi100.stdout: "Available Spare: 100%", 2024-04-07T17:51:25.292 INFO:tasks.workunit.client.0.smithi100.stdout: "Available Spare Threshold: 10%", 2024-04-07T17:51:25.292 INFO:tasks.workunit.client.0.smithi100.stdout: "Percentage Used: 9%", 2024-04-07T17:51:25.292 INFO:tasks.workunit.client.0.smithi100.stdout: "Data Units Read: 359,011,585 [183 TB]", 2024-04-07T17:51:25.292 INFO:tasks.workunit.client.0.smithi100.stdout: "Data Units Written: 1,642,371,481 [840 TB]", 2024-04-07T17:51:25.292 INFO:tasks.workunit.client.0.smithi100.stdout: "Host Read Commands: 5,750,712,954", 2024-04-07T17:51:25.292 INFO:tasks.workunit.client.0.smithi100.stdout: "Host Write Commands: 28,276,536,650", 2024-04-07T17:51:25.292 INFO:tasks.workunit.client.0.smithi100.stdout: "Controller Busy Time: 0", 2024-04-07T17:51:25.292 INFO:tasks.workunit.client.0.smithi100.stdout: "Power Cycles: 47,059", 2024-04-07T17:51:25.292 INFO:tasks.workunit.client.0.smithi100.stdout: "Power On Hours: 54,821", 2024-04-07T17:51:25.292 INFO:tasks.workunit.client.0.smithi100.stdout: "Unsafe Shutdowns: 12", 2024-04-07T17:51:25.292 INFO:tasks.workunit.client.0.smithi100.stdout: "Media and Data Integrity Errors: 0", 2024-04-07T17:51:25.293 INFO:tasks.workunit.client.0.smithi100.stdout: "Error Information Log Entries: 0", 2024-04-07T17:51:25.293 INFO:tasks.workunit.client.0.smithi100.stdout: "", 2024-04-07T17:51:25.293 INFO:tasks.workunit.client.0.smithi100.stdout: "Error Information (NVMe Log 0x01, max 64 entries)", 2024-04-07T17:51:25.293 INFO:tasks.workunit.client.0.smithi100.stdout: "No Errors Logged", 2024-04-07T17:51:25.293 INFO:tasks.workunit.client.0.smithi100.stdout: "" 2024-04-07T17:51:25.293 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:25.293 INFO:tasks.workunit.client.0.smithi100.stdout: "platform_info": "x86_64-linux-5.15.0-101-generic", 2024-04-07T17:51:25.293 INFO:tasks.workunit.client.0.smithi100.stdout: "svn_revision": "5022", 2024-04-07T17:51:25.293 INFO:tasks.workunit.client.0.smithi100.stdout: "version": [ 2024-04-07T17:51:25.293 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:25.293 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T17:51:25.294 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:25.294 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.294 INFO:tasks.workunit.client.0.smithi100.stdout: "temperature": { 2024-04-07T17:51:25.294 INFO:tasks.workunit.client.0.smithi100.stdout: "current": 25 2024-04-07T17:51:25.294 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.294 INFO:tasks.workunit.client.0.smithi100.stdout: "user_capacity": { 2024-04-07T17:51:25.294 INFO:tasks.workunit.client.0.smithi100.stdout: "blocks": 781422768, 2024-04-07T17:51:25.294 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 400088457216 2024-04-07T17:51:25.294 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.294 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.294 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.294 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.294 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.294 INFO:tasks.workunit.client.0.smithi100.stdout: "report": { 2024-04-07T17:51:25.294 INFO:tasks.workunit.client.0.smithi100.stdout: "balancer": { 2024-04-07T17:51:25.295 INFO:tasks.workunit.client.0.smithi100.stdout: "active": true, 2024-04-07T17:51:25.295 INFO:tasks.workunit.client.0.smithi100.stdout: "mode": "" 2024-04-07T17:51:25.295 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.295 INFO:tasks.workunit.client.0.smithi100.stdout: "channels": [ 2024-04-07T17:51:25.295 INFO:tasks.workunit.client.0.smithi100.stdout: "basic", 2024-04-07T17:51:25.295 INFO:tasks.workunit.client.0.smithi100.stdout: "crash", 2024-04-07T17:51:25.295 INFO:tasks.workunit.client.0.smithi100.stdout: "device", 2024-04-07T17:51:25.295 INFO:tasks.workunit.client.0.smithi100.stdout: "ident", 2024-04-07T17:51:25.295 INFO:tasks.workunit.client.0.smithi100.stdout: "perf" 2024-04-07T17:51:25.295 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:25.295 INFO:tasks.workunit.client.0.smithi100.stdout: "channels_available": [ 2024-04-07T17:51:25.295 INFO:tasks.workunit.client.0.smithi100.stdout: "basic", 2024-04-07T17:51:25.295 INFO:tasks.workunit.client.0.smithi100.stdout: "ident", 2024-04-07T17:51:25.295 INFO:tasks.workunit.client.0.smithi100.stdout: "crash", 2024-04-07T17:51:25.296 INFO:tasks.workunit.client.0.smithi100.stdout: "device", 2024-04-07T17:51:25.296 INFO:tasks.workunit.client.0.smithi100.stdout: "perf" 2024-04-07T17:51:25.296 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:25.296 INFO:tasks.workunit.client.0.smithi100.stdout: "collections_available": [ 2024-04-07T17:51:25.296 INFO:tasks.workunit.client.0.smithi100.stdout: "basic_base", 2024-04-07T17:51:25.296 INFO:tasks.workunit.client.0.smithi100.stdout: "device_base", 2024-04-07T17:51:25.296 INFO:tasks.workunit.client.0.smithi100.stdout: "crash_base", 2024-04-07T17:51:25.296 INFO:tasks.workunit.client.0.smithi100.stdout: "ident_base", 2024-04-07T17:51:25.296 INFO:tasks.workunit.client.0.smithi100.stdout: "perf_perf", 2024-04-07T17:51:25.296 INFO:tasks.workunit.client.0.smithi100.stdout: "basic_mds_metadata", 2024-04-07T17:51:25.296 INFO:tasks.workunit.client.0.smithi100.stdout: "basic_pool_usage", 2024-04-07T17:51:25.296 INFO:tasks.workunit.client.0.smithi100.stdout: "basic_usage_by_class", 2024-04-07T17:51:25.296 INFO:tasks.workunit.client.0.smithi100.stdout: "basic_rook_v01", 2024-04-07T17:51:25.296 INFO:tasks.workunit.client.0.smithi100.stdout: "perf_memory_metrics", 2024-04-07T17:51:25.297 INFO:tasks.workunit.client.0.smithi100.stdout: "basic_pool_options_bluestore", 2024-04-07T17:51:25.297 INFO:tasks.workunit.client.0.smithi100.stdout: "basic_pool_flags" 2024-04-07T17:51:25.297 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:25.297 INFO:tasks.workunit.client.0.smithi100.stdout: "collections_opted_in": [ 2024-04-07T17:51:25.297 INFO:tasks.workunit.client.0.smithi100.stdout: "basic_base", 2024-04-07T17:51:25.297 INFO:tasks.workunit.client.0.smithi100.stdout: "device_base", 2024-04-07T17:51:25.297 INFO:tasks.workunit.client.0.smithi100.stdout: "crash_base", 2024-04-07T17:51:25.297 INFO:tasks.workunit.client.0.smithi100.stdout: "ident_base", 2024-04-07T17:51:25.297 INFO:tasks.workunit.client.0.smithi100.stdout: "perf_perf", 2024-04-07T17:51:25.297 INFO:tasks.workunit.client.0.smithi100.stdout: "basic_mds_metadata", 2024-04-07T17:51:25.297 INFO:tasks.workunit.client.0.smithi100.stdout: "basic_pool_usage", 2024-04-07T17:51:25.297 INFO:tasks.workunit.client.0.smithi100.stdout: "basic_usage_by_class", 2024-04-07T17:51:25.298 INFO:tasks.workunit.client.0.smithi100.stdout: "basic_rook_v01", 2024-04-07T17:51:25.298 INFO:tasks.workunit.client.0.smithi100.stdout: "perf_memory_metrics", 2024-04-07T17:51:25.298 INFO:tasks.workunit.client.0.smithi100.stdout: "basic_pool_options_bluestore", 2024-04-07T17:51:25.298 INFO:tasks.workunit.client.0.smithi100.stdout: "basic_pool_flags" 2024-04-07T17:51:25.298 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:25.298 INFO:tasks.workunit.client.0.smithi100.stdout: "config": { 2024-04-07T17:51:25.298 INFO:tasks.workunit.client.0.smithi100.stdout: "active_changed": [ 2024-04-07T17:51:25.298 INFO:tasks.workunit.client.0.smithi100.stdout: "auth_allow_insecure_global_id_reclaim", 2024-04-07T17:51:25.298 INFO:tasks.workunit.client.0.smithi100.stdout: "auth_debug", 2024-04-07T17:51:25.298 INFO:tasks.workunit.client.0.smithi100.stdout: "auth_mon_ticket_ttl", 2024-04-07T17:51:25.298 INFO:tasks.workunit.client.0.smithi100.stdout: "auth_service_ticket_ttl", 2024-04-07T17:51:25.298 INFO:tasks.workunit.client.0.smithi100.stdout: "bdev_debug_aio", 2024-04-07T17:51:25.298 INFO:tasks.workunit.client.0.smithi100.stdout: "client_check_pool_perm", 2024-04-07T17:51:25.298 INFO:tasks.workunit.client.0.smithi100.stdout: "client_mount_gid", 2024-04-07T17:51:25.299 INFO:tasks.workunit.client.0.smithi100.stdout: "client_mount_uid", 2024-04-07T17:51:25.299 INFO:tasks.workunit.client.0.smithi100.stdout: "client_quota", 2024-04-07T17:51:25.299 INFO:tasks.workunit.client.0.smithi100.stdout: "container_image", 2024-04-07T17:51:25.299 INFO:tasks.workunit.client.0.smithi100.stdout: "daemonize", 2024-04-07T17:51:25.299 INFO:tasks.workunit.client.0.smithi100.stdout: "debug_asserts_on_shutdown", 2024-04-07T17:51:25.299 INFO:tasks.workunit.client.0.smithi100.stdout: "debug_mgr", 2024-04-07T17:51:25.299 INFO:tasks.workunit.client.0.smithi100.stdout: "debug_mon", 2024-04-07T17:51:25.299 INFO:tasks.workunit.client.0.smithi100.stdout: "debug_ms", 2024-04-07T17:51:25.299 INFO:tasks.workunit.client.0.smithi100.stdout: "debug_osd", 2024-04-07T17:51:25.299 INFO:tasks.workunit.client.0.smithi100.stdout: "debug_paxos", 2024-04-07T17:51:25.299 INFO:tasks.workunit.client.0.smithi100.stdout: "keyring", 2024-04-07T17:51:25.299 INFO:tasks.workunit.client.0.smithi100.stdout: "log_stderr_prefix", 2024-04-07T17:51:25.299 INFO:tasks.workunit.client.0.smithi100.stdout: "log_to_file", 2024-04-07T17:51:25.299 INFO:tasks.workunit.client.0.smithi100.stdout: "log_to_journald", 2024-04-07T17:51:25.299 INFO:tasks.workunit.client.0.smithi100.stdout: "log_to_stderr", 2024-04-07T17:51:25.300 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_allow_pool_delete", 2024-04-07T17:51:25.300 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_clock_drift_allowed", 2024-04-07T17:51:25.300 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_cluster_log_file_level", 2024-04-07T17:51:25.300 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_cluster_log_to_file", 2024-04-07T17:51:25.300 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_cluster_log_to_stderr", 2024-04-07T17:51:25.300 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_data_avail_warn", 2024-04-07T17:51:25.300 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_election_default_strategy", 2024-04-07T17:51:25.300 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_host", 2024-04-07T17:51:25.300 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_max_pg_per_osd", 2024-04-07T17:51:25.300 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_mds_skip_sanity", 2024-04-07T17:51:25.300 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_mgr_mkfs_grace", 2024-04-07T17:51:25.300 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_osd_prime_pg_temp", 2024-04-07T17:51:25.300 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_osd_reporter_subtree_level", 2024-04-07T17:51:25.300 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_pg_warn_max_object_skew", 2024-04-07T17:51:25.300 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_reweight_min_bytes_per_osd", 2024-04-07T17:51:25.300 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_reweight_min_pgs_per_osd", 2024-04-07T17:51:25.300 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_warn_on_crush_straw_calc_version_zero", 2024-04-07T17:51:25.301 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_warn_on_insecure_global_id_reclaim", 2024-04-07T17:51:25.301 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_warn_on_insecure_global_id_reclaim_allowed", 2024-04-07T17:51:25.301 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_warn_on_legacy_crush_tunables", 2024-04-07T17:51:25.301 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_warn_on_osd_down_out_interval_zero", 2024-04-07T17:51:25.301 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_warn_on_pool_pg_num_not_power_of_two", 2024-04-07T17:51:25.301 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_warn_on_too_few_osds", 2024-04-07T17:51:25.301 INFO:tasks.workunit.client.0.smithi100.stdout: "ms_die_on_bug", 2024-04-07T17:51:25.301 INFO:tasks.workunit.client.0.smithi100.stdout: "ms_die_on_old_message", 2024-04-07T17:51:25.301 INFO:tasks.workunit.client.0.smithi100.stdout: "no_config_file", 2024-04-07T17:51:25.301 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_class_default_list", 2024-04-07T17:51:25.301 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_class_load_list", 2024-04-07T17:51:25.302 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_debug_misdirected_ops", 2024-04-07T17:51:25.302 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_debug_op_order", 2024-04-07T17:51:25.302 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_debug_pg_log_writeout", 2024-04-07T17:51:25.302 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_debug_shutdown", 2024-04-07T17:51:25.302 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_debug_verify_cached_snaps", 2024-04-07T17:51:25.302 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_debug_verify_missing_on_start", 2024-04-07T17:51:25.302 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_debug_verify_stray_on_activate", 2024-04-07T17:51:25.302 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_deep_scrub_update_digest_min_age", 2024-04-07T17:51:25.302 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_delete_sleep", 2024-04-07T17:51:25.302 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_delete_sleep_hdd", 2024-04-07T17:51:25.302 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_delete_sleep_hybrid", 2024-04-07T17:51:25.302 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_delete_sleep_ssd", 2024-04-07T17:51:25.302 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_max_backfills", 2024-04-07T17:51:25.302 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mclock_max_capacity_iops_ssd", 2024-04-07T17:51:25.302 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mclock_profile", 2024-04-07T17:51:25.303 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mclock_scheduler_background_best_effort_lim", 2024-04-07T17:51:25.303 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mclock_scheduler_background_best_effort_res", 2024-04-07T17:51:25.303 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mclock_scheduler_background_best_effort_wgt", 2024-04-07T17:51:25.303 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mclock_scheduler_background_recovery_lim", 2024-04-07T17:51:25.303 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mclock_scheduler_background_recovery_res", 2024-04-07T17:51:25.303 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mclock_scheduler_background_recovery_wgt", 2024-04-07T17:51:25.303 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mclock_scheduler_client_lim", 2024-04-07T17:51:25.303 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mclock_scheduler_client_res", 2024-04-07T17:51:25.303 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mclock_scheduler_client_wgt", 2024-04-07T17:51:25.303 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_memory_target", 2024-04-07T17:51:25.303 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_memory_target_autotune", 2024-04-07T17:51:25.303 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_op_queue", 2024-04-07T17:51:25.303 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_op_queue_cut_off", 2024-04-07T17:51:25.303 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_pool_default_erasure_code_profile", 2024-04-07T17:51:25.303 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_pool_default_pg_autoscale_mode", 2024-04-07T17:51:25.303 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_recover_clone_overlap", 2024-04-07T17:51:25.304 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_recovery_max_active", 2024-04-07T17:51:25.304 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_recovery_max_active_hdd", 2024-04-07T17:51:25.304 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_recovery_max_active_ssd", 2024-04-07T17:51:25.304 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_recovery_max_chunk", 2024-04-07T17:51:25.304 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_recovery_sleep", 2024-04-07T17:51:25.304 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_recovery_sleep_hdd", 2024-04-07T17:51:25.304 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_recovery_sleep_hybrid", 2024-04-07T17:51:25.304 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_recovery_sleep_ssd", 2024-04-07T17:51:25.304 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_scrub_load_threshold", 2024-04-07T17:51:25.304 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_scrub_max_interval", 2024-04-07T17:51:25.304 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_scrub_sleep", 2024-04-07T17:51:25.304 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_shutdown_pgref_assert", 2024-04-07T17:51:25.304 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_snap_trim_sleep", 2024-04-07T17:51:25.304 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_snap_trim_sleep_hdd", 2024-04-07T17:51:25.304 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_snap_trim_sleep_hybrid", 2024-04-07T17:51:25.304 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_snap_trim_sleep_ssd", 2024-04-07T17:51:25.305 INFO:tasks.workunit.client.0.smithi100.stdout: "public_network", 2024-04-07T17:51:25.305 INFO:tasks.workunit.client.0.smithi100.stdout: "rbd_config_pool_override_update_timestamp", 2024-04-07T17:51:25.305 INFO:tasks.workunit.client.0.smithi100.stdout: "rbd_default_features", 2024-04-07T17:51:25.305 INFO:tasks.workunit.client.0.smithi100.stdout: "rbd_qos_exclude_ops", 2024-04-07T17:51:25.305 INFO:tasks.workunit.client.0.smithi100.stdout: "setgroup", 2024-04-07T17:51:25.305 INFO:tasks.workunit.client.0.smithi100.stdout: "setuser" 2024-04-07T17:51:25.305 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:25.305 INFO:tasks.workunit.client.0.smithi100.stdout: "cluster_changed": [ 2024-04-07T17:51:25.305 INFO:tasks.workunit.client.0.smithi100.stdout: "auth_allow_insecure_global_id_reclaim", 2024-04-07T17:51:25.305 INFO:tasks.workunit.client.0.smithi100.stdout: "auth_debug", 2024-04-07T17:51:25.305 INFO:tasks.workunit.client.0.smithi100.stdout: "auth_mon_ticket_ttl", 2024-04-07T17:51:25.305 INFO:tasks.workunit.client.0.smithi100.stdout: "auth_service_ticket_ttl", 2024-04-07T17:51:25.305 INFO:tasks.workunit.client.0.smithi100.stdout: "bdev_debug_aio", 2024-04-07T17:51:25.305 INFO:tasks.workunit.client.0.smithi100.stdout: "container_image", 2024-04-07T17:51:25.305 INFO:tasks.workunit.client.0.smithi100.stdout: "debug_asserts_on_shutdown", 2024-04-07T17:51:25.306 INFO:tasks.workunit.client.0.smithi100.stdout: "debug_mgr", 2024-04-07T17:51:25.306 INFO:tasks.workunit.client.0.smithi100.stdout: "debug_mon", 2024-04-07T17:51:25.306 INFO:tasks.workunit.client.0.smithi100.stdout: "debug_ms", 2024-04-07T17:51:25.306 INFO:tasks.workunit.client.0.smithi100.stdout: "debug_osd", 2024-04-07T17:51:25.306 INFO:tasks.workunit.client.0.smithi100.stdout: "debug_paxos", 2024-04-07T17:51:25.306 INFO:tasks.workunit.client.0.smithi100.stdout: "log_to_file", 2024-04-07T17:51:25.306 INFO:tasks.workunit.client.0.smithi100.stdout: "log_to_journald", 2024-04-07T17:51:25.306 INFO:tasks.workunit.client.0.smithi100.stdout: "log_to_stderr", 2024-04-07T17:51:25.306 INFO:tasks.workunit.client.0.smithi100.stdout: "mds_join_fs", 2024-04-07T17:51:25.306 INFO:tasks.workunit.client.0.smithi100.stdout: "mgr/cephadm/allow_ptrace", 2024-04-07T17:51:25.306 INFO:tasks.workunit.client.0.smithi100.stdout: "mgr/cephadm/container_init", 2024-04-07T17:51:25.306 INFO:tasks.workunit.client.0.smithi100.stdout: "mgr/cephadm/migration_current", 2024-04-07T17:51:25.307 INFO:tasks.workunit.client.0.smithi100.stdout: "mgr/cephadm/use_repo_digest", 2024-04-07T17:51:25.307 INFO:tasks.workunit.client.0.smithi100.stdout: "mgr/dashboard/ALERTMANAGER_API_HOST", 2024-04-07T17:51:25.307 INFO:tasks.workunit.client.0.smithi100.stdout: "mgr/dashboard/GRAFANA_API_SSL_VERIFY", 2024-04-07T17:51:25.307 INFO:tasks.workunit.client.0.smithi100.stdout: "mgr/dashboard/GRAFANA_API_URL", 2024-04-07T17:51:25.307 INFO:tasks.workunit.client.0.smithi100.stdout: "mgr/dashboard/PROMETHEUS_API_HOST", 2024-04-07T17:51:25.307 INFO:tasks.workunit.client.0.smithi100.stdout: "mgr/dashboard/ssl_server_port", 2024-04-07T17:51:25.307 INFO:tasks.workunit.client.0.smithi100.stdout: "mgr/orchestrator/orchestrator", 2024-04-07T17:51:25.307 INFO:tasks.workunit.client.0.smithi100.stdout: "mgr/telemetry/channel_ident", 2024-04-07T17:51:25.307 INFO:tasks.workunit.client.0.smithi100.stdout: "mgr/telemetry/channel_perf", 2024-04-07T17:51:25.307 INFO:tasks.workunit.client.0.smithi100.stdout: "mgr/telemetry/description", 2024-04-07T17:51:25.307 INFO:tasks.workunit.client.0.smithi100.stdout: "mgr/telemetry/enabled", 2024-04-07T17:51:25.307 INFO:tasks.workunit.client.0.smithi100.stdout: "mgr/telemetry/organization", 2024-04-07T17:51:25.307 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_allow_pool_delete", 2024-04-07T17:51:25.307 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_clock_drift_allowed", 2024-04-07T17:51:25.307 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_cluster_log_file_level", 2024-04-07T17:51:25.308 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_cluster_log_to_file", 2024-04-07T17:51:25.308 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_data_avail_warn", 2024-04-07T17:51:25.308 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_election_default_strategy", 2024-04-07T17:51:25.308 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_max_pg_per_osd", 2024-04-07T17:51:25.308 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_mds_skip_sanity", 2024-04-07T17:51:25.308 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_mgr_mkfs_grace", 2024-04-07T17:51:25.308 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_osd_prime_pg_temp", 2024-04-07T17:51:25.308 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_osd_reporter_subtree_level", 2024-04-07T17:51:25.308 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_pg_warn_max_object_skew", 2024-04-07T17:51:25.308 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_reweight_min_bytes_per_osd", 2024-04-07T17:51:25.308 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_reweight_min_pgs_per_osd", 2024-04-07T17:51:25.308 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_warn_on_crush_straw_calc_version_zero", 2024-04-07T17:51:25.308 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_warn_on_insecure_global_id_reclaim", 2024-04-07T17:51:25.308 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_warn_on_insecure_global_id_reclaim_allowed", 2024-04-07T17:51:25.308 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_warn_on_legacy_crush_tunables", 2024-04-07T17:51:25.309 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_warn_on_osd_down_out_interval_zero", 2024-04-07T17:51:25.309 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_warn_on_pool_pg_num_not_power_of_two", 2024-04-07T17:51:25.309 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_warn_on_too_few_osds", 2024-04-07T17:51:25.309 INFO:tasks.workunit.client.0.smithi100.stdout: "ms_die_on_bug", 2024-04-07T17:51:25.309 INFO:tasks.workunit.client.0.smithi100.stdout: "ms_die_on_old_message", 2024-04-07T17:51:25.309 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_class_default_list", 2024-04-07T17:51:25.309 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_class_load_list", 2024-04-07T17:51:25.309 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_debug_misdirected_ops", 2024-04-07T17:51:25.309 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_debug_op_order", 2024-04-07T17:51:25.309 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_debug_pg_log_writeout", 2024-04-07T17:51:25.309 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_debug_shutdown", 2024-04-07T17:51:25.309 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_debug_verify_cached_snaps", 2024-04-07T17:51:25.309 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_debug_verify_missing_on_start", 2024-04-07T17:51:25.309 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_debug_verify_stray_on_activate", 2024-04-07T17:51:25.310 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_deep_scrub_update_digest_min_age", 2024-04-07T17:51:25.310 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mclock_max_capacity_iops_ssd", 2024-04-07T17:51:25.310 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mclock_profile", 2024-04-07T17:51:25.310 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_memory_target", 2024-04-07T17:51:25.310 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_memory_target_autotune", 2024-04-07T17:51:25.310 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_op_queue", 2024-04-07T17:51:25.310 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_op_queue_cut_off", 2024-04-07T17:51:25.310 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_pool_default_erasure_code_profile", 2024-04-07T17:51:25.310 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_pool_default_pg_autoscale_mode", 2024-04-07T17:51:25.310 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_recover_clone_overlap", 2024-04-07T17:51:25.310 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_recovery_max_chunk", 2024-04-07T17:51:25.310 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_scrub_load_threshold", 2024-04-07T17:51:25.310 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_scrub_max_interval", 2024-04-07T17:51:25.310 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_shutdown_pgref_assert", 2024-04-07T17:51:25.311 INFO:tasks.workunit.client.0.smithi100.stdout: "public_network", 2024-04-07T17:51:25.311 INFO:tasks.workunit.client.0.smithi100.stdout: "rbd_config_pool_override_update_timestamp", 2024-04-07T17:51:25.311 INFO:tasks.workunit.client.0.smithi100.stdout: "rgw_cache_enabled", 2024-04-07T17:51:25.311 INFO:tasks.workunit.client.0.smithi100.stdout: "rgw_enable_ops_log", 2024-04-07T17:51:25.311 INFO:tasks.workunit.client.0.smithi100.stdout: "rgw_enable_usage_log" 2024-04-07T17:51:25.311 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:25.311 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.311 INFO:tasks.workunit.client.0.smithi100.stdout: "contact": null, 2024-04-07T17:51:25.311 INFO:tasks.workunit.client.0.smithi100.stdout: "crashes": [], 2024-04-07T17:51:25.311 INFO:tasks.workunit.client.0.smithi100.stdout: "created": "2024-04-07T16:02:17.280630Z", 2024-04-07T17:51:25.311 INFO:tasks.workunit.client.0.smithi100.stdout: "crush": { 2024-04-07T17:51:25.311 INFO:tasks.workunit.client.0.smithi100.stdout: "bucket_algs": { 2024-04-07T17:51:25.311 INFO:tasks.workunit.client.0.smithi100.stdout: "straw2": 3 2024-04-07T17:51:25.311 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.311 INFO:tasks.workunit.client.0.smithi100.stdout: "bucket_sizes": { 2024-04-07T17:51:25.312 INFO:tasks.workunit.client.0.smithi100.stdout: "2": 1, 2024-04-07T17:51:25.312 INFO:tasks.workunit.client.0.smithi100.stdout: "4": 2 2024-04-07T17:51:25.312 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.312 INFO:tasks.workunit.client.0.smithi100.stdout: "bucket_types": { 2024-04-07T17:51:25.312 INFO:tasks.workunit.client.0.smithi100.stdout: "1": 2, 2024-04-07T17:51:25.312 INFO:tasks.workunit.client.0.smithi100.stdout: "11": 1 2024-04-07T17:51:25.312 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.312 INFO:tasks.workunit.client.0.smithi100.stdout: "compat_weight_set": false, 2024-04-07T17:51:25.312 INFO:tasks.workunit.client.0.smithi100.stdout: "device_classes": [ 2024-04-07T17:51:25.312 INFO:tasks.workunit.client.0.smithi100.stdout: 8 2024-04-07T17:51:25.312 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:25.312 INFO:tasks.workunit.client.0.smithi100.stdout: "num_buckets": 6, 2024-04-07T17:51:25.312 INFO:tasks.workunit.client.0.smithi100.stdout: "num_devices": 8, 2024-04-07T17:51:25.312 INFO:tasks.workunit.client.0.smithi100.stdout: "num_rules": 2, 2024-04-07T17:51:25.312 INFO:tasks.workunit.client.0.smithi100.stdout: "num_types": 12, 2024-04-07T17:51:25.312 INFO:tasks.workunit.client.0.smithi100.stdout: "num_weight_sets": 0, 2024-04-07T17:51:25.313 INFO:tasks.workunit.client.0.smithi100.stdout: "tunables": { 2024-04-07T17:51:25.313 INFO:tasks.workunit.client.0.smithi100.stdout: "allowed_bucket_algs": 54, 2024-04-07T17:51:25.313 INFO:tasks.workunit.client.0.smithi100.stdout: "choose_local_fallback_tries": 0, 2024-04-07T17:51:25.313 INFO:tasks.workunit.client.0.smithi100.stdout: "choose_local_tries": 0, 2024-04-07T17:51:25.313 INFO:tasks.workunit.client.0.smithi100.stdout: "choose_total_tries": 50, 2024-04-07T17:51:25.313 INFO:tasks.workunit.client.0.smithi100.stdout: "chooseleaf_descend_once": 1, 2024-04-07T17:51:25.313 INFO:tasks.workunit.client.0.smithi100.stdout: "chooseleaf_stable": 1, 2024-04-07T17:51:25.313 INFO:tasks.workunit.client.0.smithi100.stdout: "chooseleaf_vary_r": 1, 2024-04-07T17:51:25.313 INFO:tasks.workunit.client.0.smithi100.stdout: "has_msr_rules": 0, 2024-04-07T17:51:25.313 INFO:tasks.workunit.client.0.smithi100.stdout: "has_v2_rules": 1, 2024-04-07T17:51:25.313 INFO:tasks.workunit.client.0.smithi100.stdout: "has_v3_rules": 0, 2024-04-07T17:51:25.313 INFO:tasks.workunit.client.0.smithi100.stdout: "has_v4_buckets": 1, 2024-04-07T17:51:25.313 INFO:tasks.workunit.client.0.smithi100.stdout: "has_v5_rules": 0, 2024-04-07T17:51:25.313 INFO:tasks.workunit.client.0.smithi100.stdout: "legacy_tunables": 0, 2024-04-07T17:51:25.313 INFO:tasks.workunit.client.0.smithi100.stdout: "minimum_required_version": "jewel", 2024-04-07T17:51:25.314 INFO:tasks.workunit.client.0.smithi100.stdout: "msr_collision_tries": 100, 2024-04-07T17:51:25.314 INFO:tasks.workunit.client.0.smithi100.stdout: "msr_descents": 100, 2024-04-07T17:51:25.314 INFO:tasks.workunit.client.0.smithi100.stdout: "optimal_tunables": 1, 2024-04-07T17:51:25.314 INFO:tasks.workunit.client.0.smithi100.stdout: "profile": "jewel", 2024-04-07T17:51:25.314 INFO:tasks.workunit.client.0.smithi100.stdout: "require_feature_tunables": 1, 2024-04-07T17:51:25.314 INFO:tasks.workunit.client.0.smithi100.stdout: "require_feature_tunables2": 1, 2024-04-07T17:51:25.314 INFO:tasks.workunit.client.0.smithi100.stdout: "require_feature_tunables3": 1, 2024-04-07T17:51:25.314 INFO:tasks.workunit.client.0.smithi100.stdout: "require_feature_tunables5": 1, 2024-04-07T17:51:25.314 INFO:tasks.workunit.client.0.smithi100.stdout: "straw_calc_version": 1 2024-04-07T17:51:25.314 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.314 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.314 INFO:tasks.workunit.client.0.smithi100.stdout: "description": "upgrade test cluster", 2024-04-07T17:51:25.314 INFO:tasks.workunit.client.0.smithi100.stdout: "fs": { 2024-04-07T17:51:25.314 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1, 2024-04-07T17:51:25.314 INFO:tasks.workunit.client.0.smithi100.stdout: "feature_flags": { 2024-04-07T17:51:25.315 INFO:tasks.workunit.client.0.smithi100.stdout: "enable_multiple": true, 2024-04-07T17:51:25.315 INFO:tasks.workunit.client.0.smithi100.stdout: "ever_enabled_multiple": true 2024-04-07T17:51:25.315 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.315 INFO:tasks.workunit.client.0.smithi100.stdout: "filesystems": [ 2024-04-07T17:51:25.315 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:25.315 INFO:tasks.workunit.client.0.smithi100.stdout: "approx_ctime": "2024-04", 2024-04-07T17:51:25.315 INFO:tasks.workunit.client.0.smithi100.stdout: "balancer_enabled": false, 2024-04-07T17:51:25.315 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.315 INFO:tasks.workunit.client.0.smithi100.stdout: "cached_caps": 0, 2024-04-07T17:51:25.315 INFO:tasks.workunit.client.0.smithi100.stdout: "cached_dns": 10, 2024-04-07T17:51:25.315 INFO:tasks.workunit.client.0.smithi100.stdout: "cached_inos": 13, 2024-04-07T17:51:25.315 INFO:tasks.workunit.client.0.smithi100.stdout: "cached_subtrees": 2, 2024-04-07T17:51:25.315 INFO:tasks.workunit.client.0.smithi100.stdout: "ever_allowed_features": 0, 2024-04-07T17:51:25.315 INFO:tasks.workunit.client.0.smithi100.stdout: "explicitly_allowed_features": 0, 2024-04-07T17:51:25.316 INFO:tasks.workunit.client.0.smithi100.stdout: "files": 0, 2024-04-07T17:51:25.316 INFO:tasks.workunit.client.0.smithi100.stdout: "max_mds": 1, 2024-04-07T17:51:25.316 INFO:tasks.workunit.client.0.smithi100.stdout: "num_data_pools": 1, 2024-04-07T17:51:25.316 INFO:tasks.workunit.client.0.smithi100.stdout: "num_in": 1, 2024-04-07T17:51:25.316 INFO:tasks.workunit.client.0.smithi100.stdout: "num_mds": 1, 2024-04-07T17:51:25.316 INFO:tasks.workunit.client.0.smithi100.stdout: "num_sessions": 0, 2024-04-07T17:51:25.316 INFO:tasks.workunit.client.0.smithi100.stdout: "num_standby_replay": 0, 2024-04-07T17:51:25.316 INFO:tasks.workunit.client.0.smithi100.stdout: "num_up": 1, 2024-04-07T17:51:25.316 INFO:tasks.workunit.client.0.smithi100.stdout: "snaps": 0, 2024-04-07T17:51:25.316 INFO:tasks.workunit.client.0.smithi100.stdout: "standby_count_wanted": 0 2024-04-07T17:51:25.316 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.316 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:25.316 INFO:tasks.workunit.client.0.smithi100.stdout: "num_standby_mds": 1, 2024-04-07T17:51:25.316 INFO:tasks.workunit.client.0.smithi100.stdout: "total_num_mds": 2 2024-04-07T17:51:25.316 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.317 INFO:tasks.workunit.client.0.smithi100.stdout: "heap_stats": { 2024-04-07T17:51:25.317 INFO:tasks.workunit.client.0.smithi100.stdout: "mds": { 2024-04-07T17:51:25.317 INFO:tasks.workunit.client.0.smithi100.stdout: "mds.a52fc0d8acb515431879057649ff294cfe14c620": { 2024-04-07T17:51:25.317 INFO:tasks.workunit.client.0.smithi100.stdout: "actual_memory_used": 20258816, 2024-04-07T17:51:25.317 INFO:tasks.workunit.client.0.smithi100.stdout: "central_cache_freelist": 389520, 2024-04-07T17:51:25.317 INFO:tasks.workunit.client.0.smithi100.stdout: "malloc_metadata": 2752512, 2024-04-07T17:51:25.317 INFO:tasks.workunit.client.0.smithi100.stdout: "page_heap_freelist": 868352, 2024-04-07T17:51:25.317 INFO:tasks.workunit.client.0.smithi100.stdout: "released_to_os": 0, 2024-04-07T17:51:25.317 INFO:tasks.workunit.client.0.smithi100.stdout: "spans_in_use": 340, 2024-04-07T17:51:25.317 INFO:tasks.workunit.client.0.smithi100.stdout: "tcmalloc_page_size": 8192, 2024-04-07T17:51:25.317 INFO:tasks.workunit.client.0.smithi100.stdout: "thread_cache_freelists": 1129576, 2024-04-07T17:51:25.317 INFO:tasks.workunit.client.0.smithi100.stdout: "thread_heaps_in_use": 13, 2024-04-07T17:51:25.317 INFO:tasks.workunit.client.0.smithi100.stdout: "transfer_cache_freelist": 636416, 2024-04-07T17:51:25.317 INFO:tasks.workunit.client.0.smithi100.stdout: "use_by_application": 14482440, 2024-04-07T17:51:25.317 INFO:tasks.workunit.client.0.smithi100.stdout: "virtual_address_space_used": 20258816 2024-04-07T17:51:25.317 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.318 INFO:tasks.workunit.client.0.smithi100.stdout: "mds.f47a4ff15a45950a63160265564a123fc5da5e5e": { 2024-04-07T17:51:25.318 INFO:tasks.workunit.client.0.smithi100.stdout: "actual_memory_used": 20414464, 2024-04-07T17:51:25.318 INFO:tasks.workunit.client.0.smithi100.stdout: "central_cache_freelist": 439832, 2024-04-07T17:51:25.318 INFO:tasks.workunit.client.0.smithi100.stdout: "malloc_metadata": 2752512, 2024-04-07T17:51:25.318 INFO:tasks.workunit.client.0.smithi100.stdout: "page_heap_freelist": 0, 2024-04-07T17:51:25.318 INFO:tasks.workunit.client.0.smithi100.stdout: "released_to_os": 892928, 2024-04-07T17:51:25.318 INFO:tasks.workunit.client.0.smithi100.stdout: "spans_in_use": 427, 2024-04-07T17:51:25.318 INFO:tasks.workunit.client.0.smithi100.stdout: "tcmalloc_page_size": 8192, 2024-04-07T17:51:25.318 INFO:tasks.workunit.client.0.smithi100.stdout: "thread_cache_freelists": 1301152, 2024-04-07T17:51:25.318 INFO:tasks.workunit.client.0.smithi100.stdout: "thread_heaps_in_use": 22, 2024-04-07T17:51:25.318 INFO:tasks.workunit.client.0.smithi100.stdout: "transfer_cache_freelist": 1038336, 2024-04-07T17:51:25.318 INFO:tasks.workunit.client.0.smithi100.stdout: "use_by_application": 14882632, 2024-04-07T17:51:25.318 INFO:tasks.workunit.client.0.smithi100.stdout: "virtual_address_space_used": 21307392 2024-04-07T17:51:25.318 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.318 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.319 INFO:tasks.workunit.client.0.smithi100.stdout: "mon": { 2024-04-07T17:51:25.319 INFO:tasks.workunit.client.0.smithi100.stdout: "mon.8dd0baa036f0a7e6d1f3bdce86136fe527836c3d": { 2024-04-07T17:51:25.319 INFO:tasks.workunit.client.0.smithi100.stdout: "actual_memory_used": 171474944, 2024-04-07T17:51:25.319 INFO:tasks.workunit.client.0.smithi100.stdout: "central_cache_freelist": 1880528, 2024-04-07T17:51:25.319 INFO:tasks.workunit.client.0.smithi100.stdout: "malloc_metadata": 5111808, 2024-04-07T17:51:25.319 INFO:tasks.workunit.client.0.smithi100.stdout: "page_heap_freelist": 507904, 2024-04-07T17:51:25.319 INFO:tasks.workunit.client.0.smithi100.stdout: "released_to_os": 319488, 2024-04-07T17:51:25.319 INFO:tasks.workunit.client.0.smithi100.stdout: "spans_in_use": 6327, 2024-04-07T17:51:25.319 INFO:tasks.workunit.client.0.smithi100.stdout: "tcmalloc_page_size": 8192, 2024-04-07T17:51:25.319 INFO:tasks.workunit.client.0.smithi100.stdout: "thread_cache_freelists": 11611784, 2024-04-07T17:51:25.319 INFO:tasks.workunit.client.0.smithi100.stdout: "thread_heaps_in_use": 21, 2024-04-07T17:51:25.319 INFO:tasks.workunit.client.0.smithi100.stdout: "transfer_cache_freelist": 2971904, 2024-04-07T17:51:25.319 INFO:tasks.workunit.client.0.smithi100.stdout: "use_by_application": 149391016, 2024-04-07T17:51:25.319 INFO:tasks.workunit.client.0.smithi100.stdout: "virtual_address_space_used": 171794432 2024-04-07T17:51:25.319 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.320 INFO:tasks.workunit.client.0.smithi100.stdout: "mon.a52fc0d8acb515431879057649ff294cfe14c620": { 2024-04-07T17:51:25.320 INFO:tasks.workunit.client.0.smithi100.stdout: "actual_memory_used": 165158912, 2024-04-07T17:51:25.320 INFO:tasks.workunit.client.0.smithi100.stdout: "central_cache_freelist": 2408920, 2024-04-07T17:51:25.320 INFO:tasks.workunit.client.0.smithi100.stdout: "malloc_metadata": 3014656, 2024-04-07T17:51:25.320 INFO:tasks.workunit.client.0.smithi100.stdout: "page_heap_freelist": 0, 2024-04-07T17:51:25.320 INFO:tasks.workunit.client.0.smithi100.stdout: "released_to_os": 2441216, 2024-04-07T17:51:25.320 INFO:tasks.workunit.client.0.smithi100.stdout: "spans_in_use": 6223, 2024-04-07T17:51:25.320 INFO:tasks.workunit.client.0.smithi100.stdout: "tcmalloc_page_size": 8192, 2024-04-07T17:51:25.320 INFO:tasks.workunit.client.0.smithi100.stdout: "thread_cache_freelists": 9216752, 2024-04-07T17:51:25.320 INFO:tasks.workunit.client.0.smithi100.stdout: "thread_heaps_in_use": 21, 2024-04-07T17:51:25.320 INFO:tasks.workunit.client.0.smithi100.stdout: "transfer_cache_freelist": 2701056, 2024-04-07T17:51:25.320 INFO:tasks.workunit.client.0.smithi100.stdout: "use_by_application": 147817528, 2024-04-07T17:51:25.320 INFO:tasks.workunit.client.0.smithi100.stdout: "virtual_address_space_used": 167600128 2024-04-07T17:51:25.320 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.320 INFO:tasks.workunit.client.0.smithi100.stdout: "mon.f47a4ff15a45950a63160265564a123fc5da5e5e": { 2024-04-07T17:51:25.320 INFO:tasks.workunit.client.0.smithi100.stdout: "actual_memory_used": 262119424, 2024-04-07T17:51:25.321 INFO:tasks.workunit.client.0.smithi100.stdout: "central_cache_freelist": 6486992, 2024-04-07T17:51:25.321 INFO:tasks.workunit.client.0.smithi100.stdout: "malloc_metadata": 3145728, 2024-04-07T17:51:25.321 INFO:tasks.workunit.client.0.smithi100.stdout: "page_heap_freelist": 622592, 2024-04-07T17:51:25.321 INFO:tasks.workunit.client.0.smithi100.stdout: "released_to_os": 188416, 2024-04-07T17:51:25.321 INFO:tasks.workunit.client.0.smithi100.stdout: "spans_in_use": 9408, 2024-04-07T17:51:25.321 INFO:tasks.workunit.client.0.smithi100.stdout: "tcmalloc_page_size": 8192, 2024-04-07T17:51:25.321 INFO:tasks.workunit.client.0.smithi100.stdout: "thread_cache_freelists": 15080160, 2024-04-07T17:51:25.321 INFO:tasks.workunit.client.0.smithi100.stdout: "thread_heaps_in_use": 23, 2024-04-07T17:51:25.321 INFO:tasks.workunit.client.0.smithi100.stdout: "transfer_cache_freelist": 5216512, 2024-04-07T17:51:25.321 INFO:tasks.workunit.client.0.smithi100.stdout: "use_by_application": 231567440, 2024-04-07T17:51:25.321 INFO:tasks.workunit.client.0.smithi100.stdout: "virtual_address_space_used": 262307840 2024-04-07T17:51:25.321 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.321 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.321 INFO:tasks.workunit.client.0.smithi100.stdout: "osd": { 2024-04-07T17:51:25.321 INFO:tasks.workunit.client.0.smithi100.stdout: "osd.0": { 2024-04-07T17:51:25.322 INFO:tasks.workunit.client.0.smithi100.stdout: "actual_memory_used": 473178112, 2024-04-07T17:51:25.322 INFO:tasks.workunit.client.0.smithi100.stdout: "central_cache_freelist": 27167920, 2024-04-07T17:51:25.322 INFO:tasks.workunit.client.0.smithi100.stdout: "malloc_metadata": 3407872, 2024-04-07T17:51:25.322 INFO:tasks.workunit.client.0.smithi100.stdout: "page_heap_freelist": 0, 2024-04-07T17:51:25.322 INFO:tasks.workunit.client.0.smithi100.stdout: "released_to_os": 38920192, 2024-04-07T17:51:25.322 INFO:tasks.workunit.client.0.smithi100.stdout: "spans_in_use": 12519, 2024-04-07T17:51:25.322 INFO:tasks.workunit.client.0.smithi100.stdout: "tcmalloc_page_size": 8192, 2024-04-07T17:51:25.322 INFO:tasks.workunit.client.0.smithi100.stdout: "thread_cache_freelists": 94876192, 2024-04-07T17:51:25.322 INFO:tasks.workunit.client.0.smithi100.stdout: "thread_heaps_in_use": 60, 2024-04-07T17:51:25.322 INFO:tasks.workunit.client.0.smithi100.stdout: "transfer_cache_freelist": 8498176, 2024-04-07T17:51:25.322 INFO:tasks.workunit.client.0.smithi100.stdout: "use_by_application": 339227952, 2024-04-07T17:51:25.322 INFO:tasks.workunit.client.0.smithi100.stdout: "virtual_address_space_used": 512098304 2024-04-07T17:51:25.322 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.322 INFO:tasks.workunit.client.0.smithi100.stdout: "osd.1": { 2024-04-07T17:51:25.322 INFO:tasks.workunit.client.0.smithi100.stdout: "actual_memory_used": 664215552, 2024-04-07T17:51:25.323 INFO:tasks.workunit.client.0.smithi100.stdout: "central_cache_freelist": 27261088, 2024-04-07T17:51:25.323 INFO:tasks.workunit.client.0.smithi100.stdout: "malloc_metadata": 3407872, 2024-04-07T17:51:25.323 INFO:tasks.workunit.client.0.smithi100.stdout: "page_heap_freelist": 0, 2024-04-07T17:51:25.323 INFO:tasks.workunit.client.0.smithi100.stdout: "released_to_os": 35110912, 2024-04-07T17:51:25.323 INFO:tasks.workunit.client.0.smithi100.stdout: "spans_in_use": 13060, 2024-04-07T17:51:25.323 INFO:tasks.workunit.client.0.smithi100.stdout: "tcmalloc_page_size": 8192, 2024-04-07T17:51:25.323 INFO:tasks.workunit.client.0.smithi100.stdout: "thread_cache_freelists": 85490224, 2024-04-07T17:51:25.323 INFO:tasks.workunit.client.0.smithi100.stdout: "thread_heaps_in_use": 53, 2024-04-07T17:51:25.323 INFO:tasks.workunit.client.0.smithi100.stdout: "transfer_cache_freelist": 8460544, 2024-04-07T17:51:25.323 INFO:tasks.workunit.client.0.smithi100.stdout: "use_by_application": 539595824, 2024-04-07T17:51:25.323 INFO:tasks.workunit.client.0.smithi100.stdout: "virtual_address_space_used": 699326464 2024-04-07T17:51:25.323 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.323 INFO:tasks.workunit.client.0.smithi100.stdout: "osd.2": { 2024-04-07T17:51:25.323 INFO:tasks.workunit.client.0.smithi100.stdout: "actual_memory_used": 517300224, 2024-04-07T17:51:25.323 INFO:tasks.workunit.client.0.smithi100.stdout: "central_cache_freelist": 26301816, 2024-04-07T17:51:25.324 INFO:tasks.workunit.client.0.smithi100.stdout: "malloc_metadata": 3276800, 2024-04-07T17:51:25.324 INFO:tasks.workunit.client.0.smithi100.stdout: "page_heap_freelist": 0, 2024-04-07T17:51:25.324 INFO:tasks.workunit.client.0.smithi100.stdout: "released_to_os": 49258496, 2024-04-07T17:51:25.324 INFO:tasks.workunit.client.0.smithi100.stdout: "spans_in_use": 10733, 2024-04-07T17:51:25.324 INFO:tasks.workunit.client.0.smithi100.stdout: "tcmalloc_page_size": 8192, 2024-04-07T17:51:25.324 INFO:tasks.workunit.client.0.smithi100.stdout: "thread_cache_freelists": 85507984, 2024-04-07T17:51:25.324 INFO:tasks.workunit.client.0.smithi100.stdout: "thread_heaps_in_use": 60, 2024-04-07T17:51:25.324 INFO:tasks.workunit.client.0.smithi100.stdout: "transfer_cache_freelist": 9234432, 2024-04-07T17:51:25.324 INFO:tasks.workunit.client.0.smithi100.stdout: "use_by_application": 392979192, 2024-04-07T17:51:25.324 INFO:tasks.workunit.client.0.smithi100.stdout: "virtual_address_space_used": 566558720 2024-04-07T17:51:25.324 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.324 INFO:tasks.workunit.client.0.smithi100.stdout: "osd.3": { 2024-04-07T17:51:25.324 INFO:tasks.workunit.client.0.smithi100.stdout: "actual_memory_used": 473882624, 2024-04-07T17:51:25.324 INFO:tasks.workunit.client.0.smithi100.stdout: "central_cache_freelist": 20204712, 2024-04-07T17:51:25.324 INFO:tasks.workunit.client.0.smithi100.stdout: "malloc_metadata": 3276800, 2024-04-07T17:51:25.324 INFO:tasks.workunit.client.0.smithi100.stdout: "page_heap_freelist": 0, 2024-04-07T17:51:25.325 INFO:tasks.workunit.client.0.smithi100.stdout: "released_to_os": 33169408, 2024-04-07T17:51:25.325 INFO:tasks.workunit.client.0.smithi100.stdout: "spans_in_use": 10246, 2024-04-07T17:51:25.325 INFO:tasks.workunit.client.0.smithi100.stdout: "tcmalloc_page_size": 8192, 2024-04-07T17:51:25.325 INFO:tasks.workunit.client.0.smithi100.stdout: "thread_cache_freelists": 80135240, 2024-04-07T17:51:25.325 INFO:tasks.workunit.client.0.smithi100.stdout: "thread_heaps_in_use": 60, 2024-04-07T17:51:25.325 INFO:tasks.workunit.client.0.smithi100.stdout: "transfer_cache_freelist": 9872896, 2024-04-07T17:51:25.325 INFO:tasks.workunit.client.0.smithi100.stdout: "use_by_application": 360392976, 2024-04-07T17:51:25.325 INFO:tasks.workunit.client.0.smithi100.stdout: "virtual_address_space_used": 507052032 2024-04-07T17:51:25.325 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.325 INFO:tasks.workunit.client.0.smithi100.stdout: "osd.4": { 2024-04-07T17:51:25.325 INFO:tasks.workunit.client.0.smithi100.stdout: "actual_memory_used": 596656128, 2024-04-07T17:51:25.325 INFO:tasks.workunit.client.0.smithi100.stdout: "central_cache_freelist": 24815184, 2024-04-07T17:51:25.325 INFO:tasks.workunit.client.0.smithi100.stdout: "malloc_metadata": 3407872, 2024-04-07T17:51:25.325 INFO:tasks.workunit.client.0.smithi100.stdout: "page_heap_freelist": 0, 2024-04-07T17:51:25.325 INFO:tasks.workunit.client.0.smithi100.stdout: "released_to_os": 30015488, 2024-04-07T17:51:25.325 INFO:tasks.workunit.client.0.smithi100.stdout: "spans_in_use": 11402, 2024-04-07T17:51:25.326 INFO:tasks.workunit.client.0.smithi100.stdout: "tcmalloc_page_size": 8192, 2024-04-07T17:51:25.326 INFO:tasks.workunit.client.0.smithi100.stdout: "thread_cache_freelists": 99015504, 2024-04-07T17:51:25.326 INFO:tasks.workunit.client.0.smithi100.stdout: "thread_heaps_in_use": 53, 2024-04-07T17:51:25.326 INFO:tasks.workunit.client.0.smithi100.stdout: "transfer_cache_freelist": 12010496, 2024-04-07T17:51:25.326 INFO:tasks.workunit.client.0.smithi100.stdout: "use_by_application": 457407072, 2024-04-07T17:51:25.326 INFO:tasks.workunit.client.0.smithi100.stdout: "virtual_address_space_used": 626671616 2024-04-07T17:51:25.326 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.326 INFO:tasks.workunit.client.0.smithi100.stdout: "osd.5": { 2024-04-07T17:51:25.326 INFO:tasks.workunit.client.0.smithi100.stdout: "actual_memory_used": 562798592, 2024-04-07T17:51:25.326 INFO:tasks.workunit.client.0.smithi100.stdout: "central_cache_freelist": 27165776, 2024-04-07T17:51:25.326 INFO:tasks.workunit.client.0.smithi100.stdout: "malloc_metadata": 5636096, 2024-04-07T17:51:25.326 INFO:tasks.workunit.client.0.smithi100.stdout: "page_heap_freelist": 0, 2024-04-07T17:51:25.326 INFO:tasks.workunit.client.0.smithi100.stdout: "released_to_os": 13197312, 2024-04-07T17:51:25.326 INFO:tasks.workunit.client.0.smithi100.stdout: "spans_in_use": 13822, 2024-04-07T17:51:25.327 INFO:tasks.workunit.client.0.smithi100.stdout: "tcmalloc_page_size": 8192, 2024-04-07T17:51:25.327 INFO:tasks.workunit.client.0.smithi100.stdout: "thread_cache_freelists": 86836240, 2024-04-07T17:51:25.327 INFO:tasks.workunit.client.0.smithi100.stdout: "thread_heaps_in_use": 60, 2024-04-07T17:51:25.327 INFO:tasks.workunit.client.0.smithi100.stdout: "transfer_cache_freelist": 10625536, 2024-04-07T17:51:25.327 INFO:tasks.workunit.client.0.smithi100.stdout: "use_by_application": 432534944, 2024-04-07T17:51:25.327 INFO:tasks.workunit.client.0.smithi100.stdout: "virtual_address_space_used": 575995904 2024-04-07T17:51:25.327 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.327 INFO:tasks.workunit.client.0.smithi100.stdout: "osd.6": { 2024-04-07T17:51:25.327 INFO:tasks.workunit.client.0.smithi100.stdout: "actual_memory_used": 362029056, 2024-04-07T17:51:25.327 INFO:tasks.workunit.client.0.smithi100.stdout: "central_cache_freelist": 16256432, 2024-04-07T17:51:25.327 INFO:tasks.workunit.client.0.smithi100.stdout: "malloc_metadata": 3276800, 2024-04-07T17:51:25.327 INFO:tasks.workunit.client.0.smithi100.stdout: "page_heap_freelist": 0, 2024-04-07T17:51:25.327 INFO:tasks.workunit.client.0.smithi100.stdout: "released_to_os": 64888832, 2024-04-07T17:51:25.327 INFO:tasks.workunit.client.0.smithi100.stdout: "spans_in_use": 9372, 2024-04-07T17:51:25.327 INFO:tasks.workunit.client.0.smithi100.stdout: "tcmalloc_page_size": 8192, 2024-04-07T17:51:25.327 INFO:tasks.workunit.client.0.smithi100.stdout: "thread_cache_freelists": 49958336, 2024-04-07T17:51:25.328 INFO:tasks.workunit.client.0.smithi100.stdout: "thread_heaps_in_use": 60, 2024-04-07T17:51:25.328 INFO:tasks.workunit.client.0.smithi100.stdout: "transfer_cache_freelist": 10123264, 2024-04-07T17:51:25.328 INFO:tasks.workunit.client.0.smithi100.stdout: "use_by_application": 282414224, 2024-04-07T17:51:25.328 INFO:tasks.workunit.client.0.smithi100.stdout: "virtual_address_space_used": 426917888 2024-04-07T17:51:25.328 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.328 INFO:tasks.workunit.client.0.smithi100.stdout: "osd.7": { 2024-04-07T17:51:25.328 INFO:tasks.workunit.client.0.smithi100.stdout: "actual_memory_used": 434184192, 2024-04-07T17:51:25.328 INFO:tasks.workunit.client.0.smithi100.stdout: "central_cache_freelist": 20882896, 2024-04-07T17:51:25.328 INFO:tasks.workunit.client.0.smithi100.stdout: "malloc_metadata": 3276800, 2024-04-07T17:51:25.328 INFO:tasks.workunit.client.0.smithi100.stdout: "page_heap_freelist": 0, 2024-04-07T17:51:25.328 INFO:tasks.workunit.client.0.smithi100.stdout: "released_to_os": 32423936, 2024-04-07T17:51:25.328 INFO:tasks.workunit.client.0.smithi100.stdout: "spans_in_use": 10455, 2024-04-07T17:51:25.328 INFO:tasks.workunit.client.0.smithi100.stdout: "tcmalloc_page_size": 8192, 2024-04-07T17:51:25.328 INFO:tasks.workunit.client.0.smithi100.stdout: "thread_cache_freelists": 72543584, 2024-04-07T17:51:25.328 INFO:tasks.workunit.client.0.smithi100.stdout: "thread_heaps_in_use": 59, 2024-04-07T17:51:25.329 INFO:tasks.workunit.client.0.smithi100.stdout: "transfer_cache_freelist": 10244352, 2024-04-07T17:51:25.329 INFO:tasks.workunit.client.0.smithi100.stdout: "use_by_application": 327236560, 2024-04-07T17:51:25.329 INFO:tasks.workunit.client.0.smithi100.stdout: "virtual_address_space_used": 466608128 2024-04-07T17:51:25.329 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.329 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.329 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.329 INFO:tasks.workunit.client.0.smithi100.stdout: "hosts": { 2024-04-07T17:51:25.329 INFO:tasks.workunit.client.0.smithi100.stdout: "num": 2, 2024-04-07T17:51:25.329 INFO:tasks.workunit.client.0.smithi100.stdout: "num_with_mds": 2, 2024-04-07T17:51:25.329 INFO:tasks.workunit.client.0.smithi100.stdout: "num_with_mgr": 2, 2024-04-07T17:51:25.329 INFO:tasks.workunit.client.0.smithi100.stdout: "num_with_mon": 2, 2024-04-07T17:51:25.329 INFO:tasks.workunit.client.0.smithi100.stdout: "num_with_osd": 2 2024-04-07T17:51:25.329 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.329 INFO:tasks.workunit.client.0.smithi100.stdout: "io_rate": { 2024-04-07T17:51:25.329 INFO:tasks.workunit.client.0.smithi100.stdout: "pg_stats_delta": { 2024-04-07T17:51:25.329 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": 0, 2024-04-07T17:51:25.330 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:25.330 INFO:tasks.workunit.client.0.smithi100.stdout: "num_store_stats": 0, 2024-04-07T17:51:25.330 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:25.330 INFO:tasks.workunit.client.0.smithi100.stdout: "stamp_delta": "12.005171", 2024-04-07T17:51:25.330 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:25.330 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:25.330 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:25.330 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:25.330 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:25.330 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:25.330 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:25.330 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:25.330 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:25.330 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:25.330 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:25.331 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:25.331 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:25.331 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:25.331 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:25.331 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:25.331 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:25.331 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:25.331 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:25.331 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:25.331 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:25.331 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:25.331 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:25.331 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:25.331 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:25.331 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:25.332 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:25.332 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:25.332 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:25.332 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:25.332 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:25.332 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:25.332 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:25.332 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:25.332 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 2, 2024-04-07T17:51:25.332 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 2, 2024-04-07T17:51:25.332 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:25.332 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:25.332 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:25.332 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:25.332 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:25.332 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.333 INFO:tasks.workunit.client.0.smithi100.stdout: "store_stats": { 2024-04-07T17:51:25.333 INFO:tasks.workunit.client.0.smithi100.stdout: "allocated": 0, 2024-04-07T17:51:25.333 INFO:tasks.workunit.client.0.smithi100.stdout: "available": 0, 2024-04-07T17:51:25.333 INFO:tasks.workunit.client.0.smithi100.stdout: "data_compressed": 0, 2024-04-07T17:51:25.333 INFO:tasks.workunit.client.0.smithi100.stdout: "data_compressed_allocated": 0, 2024-04-07T17:51:25.333 INFO:tasks.workunit.client.0.smithi100.stdout: "data_compressed_original": 0, 2024-04-07T17:51:25.333 INFO:tasks.workunit.client.0.smithi100.stdout: "data_stored": 0, 2024-04-07T17:51:25.333 INFO:tasks.workunit.client.0.smithi100.stdout: "internal_metadata": 0, 2024-04-07T17:51:25.333 INFO:tasks.workunit.client.0.smithi100.stdout: "internally_reserved": 0, 2024-04-07T17:51:25.333 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_allocated": 0, 2024-04-07T17:51:25.333 INFO:tasks.workunit.client.0.smithi100.stdout: "total": 0 2024-04-07T17:51:25.333 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.333 INFO:tasks.workunit.client.0.smithi100.stdout: "up": 0 2024-04-07T17:51:25.333 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.333 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.334 INFO:tasks.workunit.client.0.smithi100.stdout: "leaderboard": false, 2024-04-07T17:51:25.334 INFO:tasks.workunit.client.0.smithi100.stdout: "leaderboard_description": null, 2024-04-07T17:51:25.334 INFO:tasks.workunit.client.0.smithi100.stdout: "license": "sharing-1-0", 2024-04-07T17:51:25.334 INFO:tasks.workunit.client.0.smithi100.stdout: "mempool": { 2024-04-07T17:51:25.334 INFO:tasks.workunit.client.0.smithi100.stdout: "mds": { 2024-04-07T17:51:25.334 INFO:tasks.workunit.client.0.smithi100.stdout: "mds.a52fc0d8acb515431879057649ff294cfe14c620": { 2024-04-07T17:51:25.334 INFO:tasks.workunit.client.0.smithi100.stdout: "bloom_filter": { 2024-04-07T17:51:25.334 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.334 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.334 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.334 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs": { 2024-04-07T17:51:25.334 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.334 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.334 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.334 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs_file_reader": { 2024-04-07T17:51:25.334 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.335 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.335 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.335 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs_file_writer": { 2024-04-07T17:51:25.335 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.335 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.335 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.335 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_alloc": { 2024-04-07T17:51:25.335 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.335 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.335 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.335 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_blob": { 2024-04-07T17:51:25.335 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.335 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.335 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.335 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_buffer": { 2024-04-07T17:51:25.336 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.336 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.336 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.336 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_data": { 2024-04-07T17:51:25.336 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.336 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.336 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.336 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_meta": { 2024-04-07T17:51:25.336 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.336 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.336 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.336 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_onode": { 2024-04-07T17:51:25.336 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.336 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.336 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.337 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_other": { 2024-04-07T17:51:25.337 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.337 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.337 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.337 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_extent": { 2024-04-07T17:51:25.337 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.337 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.337 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.337 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_fsck": { 2024-04-07T17:51:25.337 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.337 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.337 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.337 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_inline_bl": { 2024-04-07T17:51:25.337 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.337 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.337 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.338 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_shared_blob": { 2024-04-07T17:51:25.338 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.338 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.338 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.338 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_txc": { 2024-04-07T17:51:25.338 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.338 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.338 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.338 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_writing": { 2024-04-07T17:51:25.338 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.338 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.338 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.338 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_writing_deferred": { 2024-04-07T17:51:25.338 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.339 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.339 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.339 INFO:tasks.workunit.client.0.smithi100.stdout: "buffer_anon": { 2024-04-07T17:51:25.339 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 149048, 2024-04-07T17:51:25.339 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 34 2024-04-07T17:51:25.339 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.339 INFO:tasks.workunit.client.0.smithi100.stdout: "buffer_meta": { 2024-04-07T17:51:25.339 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.339 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.339 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.339 INFO:tasks.workunit.client.0.smithi100.stdout: "mds_co": { 2024-04-07T17:51:25.339 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 928, 2024-04-07T17:51:25.339 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 2 2024-04-07T17:51:25.339 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.339 INFO:tasks.workunit.client.0.smithi100.stdout: "osd": { 2024-04-07T17:51:25.340 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.340 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.340 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.340 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mapbl": { 2024-04-07T17:51:25.340 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.340 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.340 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.340 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_pglog": { 2024-04-07T17:51:25.340 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.340 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.340 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.340 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap": { 2024-04-07T17:51:25.340 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.340 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.340 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.341 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap_mapping": { 2024-04-07T17:51:25.341 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.341 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.341 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.341 INFO:tasks.workunit.client.0.smithi100.stdout: "pgmap": { 2024-04-07T17:51:25.341 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.341 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.341 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.341 INFO:tasks.workunit.client.0.smithi100.stdout: "unittest_1": { 2024-04-07T17:51:25.341 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.341 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.341 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.341 INFO:tasks.workunit.client.0.smithi100.stdout: "unittest_2": { 2024-04-07T17:51:25.341 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.341 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.341 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.342 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.342 INFO:tasks.workunit.client.0.smithi100.stdout: "mds.f47a4ff15a45950a63160265564a123fc5da5e5e": { 2024-04-07T17:51:25.342 INFO:tasks.workunit.client.0.smithi100.stdout: "bloom_filter": { 2024-04-07T17:51:25.342 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.342 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.342 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.342 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs": { 2024-04-07T17:51:25.342 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.342 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.342 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.342 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs_file_reader": { 2024-04-07T17:51:25.342 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.342 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.342 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.342 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs_file_writer": { 2024-04-07T17:51:25.343 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.343 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.343 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.343 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_alloc": { 2024-04-07T17:51:25.343 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.343 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.343 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.343 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_blob": { 2024-04-07T17:51:25.343 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.343 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.343 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.343 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_buffer": { 2024-04-07T17:51:25.343 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.343 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.343 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.344 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_data": { 2024-04-07T17:51:25.344 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.344 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.344 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.344 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_meta": { 2024-04-07T17:51:25.344 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.344 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.344 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.344 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_onode": { 2024-04-07T17:51:25.344 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.344 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.344 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.344 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_other": { 2024-04-07T17:51:25.344 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.344 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.345 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.345 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_extent": { 2024-04-07T17:51:25.345 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.345 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.345 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.345 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_fsck": { 2024-04-07T17:51:25.345 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.345 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.345 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.345 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_inline_bl": { 2024-04-07T17:51:25.345 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.345 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.345 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.345 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_shared_blob": { 2024-04-07T17:51:25.345 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.345 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.346 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.346 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_txc": { 2024-04-07T17:51:25.346 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.346 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.346 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.346 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_writing": { 2024-04-07T17:51:25.346 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.346 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.346 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.346 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_writing_deferred": { 2024-04-07T17:51:25.346 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.346 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.346 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.346 INFO:tasks.workunit.client.0.smithi100.stdout: "buffer_anon": { 2024-04-07T17:51:25.347 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 221552, 2024-04-07T17:51:25.347 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 64 2024-04-07T17:51:25.347 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.347 INFO:tasks.workunit.client.0.smithi100.stdout: "buffer_meta": { 2024-04-07T17:51:25.347 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.347 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.347 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.347 INFO:tasks.workunit.client.0.smithi100.stdout: "mds_co": { 2024-04-07T17:51:25.347 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 52112, 2024-04-07T17:51:25.347 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 181 2024-04-07T17:51:25.347 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.347 INFO:tasks.workunit.client.0.smithi100.stdout: "osd": { 2024-04-07T17:51:25.347 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.347 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.347 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.348 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mapbl": { 2024-04-07T17:51:25.348 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.348 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.348 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.348 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_pglog": { 2024-04-07T17:51:25.348 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.348 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.348 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.348 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap": { 2024-04-07T17:51:25.348 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 12216, 2024-04-07T17:51:25.348 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 252 2024-04-07T17:51:25.348 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.348 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap_mapping": { 2024-04-07T17:51:25.348 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.348 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.349 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.349 INFO:tasks.workunit.client.0.smithi100.stdout: "pgmap": { 2024-04-07T17:51:25.349 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.349 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.349 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.349 INFO:tasks.workunit.client.0.smithi100.stdout: "unittest_1": { 2024-04-07T17:51:25.349 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.349 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.349 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.349 INFO:tasks.workunit.client.0.smithi100.stdout: "unittest_2": { 2024-04-07T17:51:25.349 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.349 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.349 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.349 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.350 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.350 INFO:tasks.workunit.client.0.smithi100.stdout: "mon": { 2024-04-07T17:51:25.350 INFO:tasks.workunit.client.0.smithi100.stdout: "mon.8dd0baa036f0a7e6d1f3bdce86136fe527836c3d": { 2024-04-07T17:51:25.350 INFO:tasks.workunit.client.0.smithi100.stdout: "bloom_filter": { 2024-04-07T17:51:25.350 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.350 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.350 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.350 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs": { 2024-04-07T17:51:25.350 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.350 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.350 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.350 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs_file_reader": { 2024-04-07T17:51:25.350 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.350 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.350 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.351 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs_file_writer": { 2024-04-07T17:51:25.351 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.351 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.351 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.351 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_alloc": { 2024-04-07T17:51:25.351 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.351 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.351 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.351 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_blob": { 2024-04-07T17:51:25.351 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.351 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.351 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.351 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_buffer": { 2024-04-07T17:51:25.351 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.351 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.351 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.351 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_data": { 2024-04-07T17:51:25.352 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.352 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.352 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.352 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_meta": { 2024-04-07T17:51:25.352 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.352 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.352 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.352 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_onode": { 2024-04-07T17:51:25.352 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.352 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.352 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.352 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_other": { 2024-04-07T17:51:25.352 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.352 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.352 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.352 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_extent": { 2024-04-07T17:51:25.353 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.353 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.353 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.353 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_fsck": { 2024-04-07T17:51:25.353 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.353 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.353 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.353 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_inline_bl": { 2024-04-07T17:51:25.353 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.353 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.353 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.353 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_shared_blob": { 2024-04-07T17:51:25.353 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.353 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.353 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.353 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_txc": { 2024-04-07T17:51:25.354 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.354 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.354 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.354 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_writing": { 2024-04-07T17:51:25.354 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.354 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.354 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.354 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_writing_deferred": { 2024-04-07T17:51:25.354 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.354 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.354 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.354 INFO:tasks.workunit.client.0.smithi100.stdout: "buffer_anon": { 2024-04-07T17:51:25.354 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 40717970, 2024-04-07T17:51:25.354 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 5034 2024-04-07T17:51:25.354 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.355 INFO:tasks.workunit.client.0.smithi100.stdout: "buffer_meta": { 2024-04-07T17:51:25.355 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.355 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.355 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.355 INFO:tasks.workunit.client.0.smithi100.stdout: "mds_co": { 2024-04-07T17:51:25.355 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.355 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.355 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.355 INFO:tasks.workunit.client.0.smithi100.stdout: "osd": { 2024-04-07T17:51:25.355 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.355 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.355 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.355 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mapbl": { 2024-04-07T17:51:25.355 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.355 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.355 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.356 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_pglog": { 2024-04-07T17:51:25.356 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.356 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.356 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.356 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap": { 2024-04-07T17:51:25.356 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 10792, 2024-04-07T17:51:25.356 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 251 2024-04-07T17:51:25.356 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.356 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap_mapping": { 2024-04-07T17:51:25.356 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 70440, 2024-04-07T17:51:25.356 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 5158 2024-04-07T17:51:25.356 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.356 INFO:tasks.workunit.client.0.smithi100.stdout: "pgmap": { 2024-04-07T17:51:25.356 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 5896, 2024-04-07T17:51:25.357 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 174 2024-04-07T17:51:25.357 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.357 INFO:tasks.workunit.client.0.smithi100.stdout: "unittest_1": { 2024-04-07T17:51:25.357 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.357 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.357 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.357 INFO:tasks.workunit.client.0.smithi100.stdout: "unittest_2": { 2024-04-07T17:51:25.357 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.357 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.357 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.357 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.357 INFO:tasks.workunit.client.0.smithi100.stdout: "mon.a52fc0d8acb515431879057649ff294cfe14c620": { 2024-04-07T17:51:25.357 INFO:tasks.workunit.client.0.smithi100.stdout: "bloom_filter": { 2024-04-07T17:51:25.357 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.357 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.357 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.358 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs": { 2024-04-07T17:51:25.358 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.358 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.358 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.358 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs_file_reader": { 2024-04-07T17:51:25.358 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.358 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.358 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.358 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs_file_writer": { 2024-04-07T17:51:25.358 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.358 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.358 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.358 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_alloc": { 2024-04-07T17:51:25.358 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.358 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.359 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.359 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_blob": { 2024-04-07T17:51:25.359 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.359 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.359 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.359 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_buffer": { 2024-04-07T17:51:25.359 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.359 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.359 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.359 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_data": { 2024-04-07T17:51:25.359 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.359 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.359 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.359 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_meta": { 2024-04-07T17:51:25.359 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.359 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.360 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.360 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_onode": { 2024-04-07T17:51:25.360 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.360 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.360 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.360 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_other": { 2024-04-07T17:51:25.360 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.360 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.360 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.360 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_extent": { 2024-04-07T17:51:25.360 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.360 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.360 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.360 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_fsck": { 2024-04-07T17:51:25.360 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.360 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.361 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.361 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_inline_bl": { 2024-04-07T17:51:25.361 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.361 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.361 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.361 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_shared_blob": { 2024-04-07T17:51:25.361 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.361 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.361 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.361 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_txc": { 2024-04-07T17:51:25.361 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.361 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.361 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.361 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_writing": { 2024-04-07T17:51:25.361 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.362 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.362 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.362 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_writing_deferred": { 2024-04-07T17:51:25.362 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.362 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.362 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.362 INFO:tasks.workunit.client.0.smithi100.stdout: "buffer_anon": { 2024-04-07T17:51:25.362 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 40412315, 2024-04-07T17:51:25.362 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 4986 2024-04-07T17:51:25.362 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.362 INFO:tasks.workunit.client.0.smithi100.stdout: "buffer_meta": { 2024-04-07T17:51:25.362 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 1056, 2024-04-07T17:51:25.362 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 12 2024-04-07T17:51:25.362 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.362 INFO:tasks.workunit.client.0.smithi100.stdout: "mds_co": { 2024-04-07T17:51:25.363 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.363 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.363 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.363 INFO:tasks.workunit.client.0.smithi100.stdout: "osd": { 2024-04-07T17:51:25.363 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.363 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.363 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.363 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mapbl": { 2024-04-07T17:51:25.363 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.363 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.363 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.363 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_pglog": { 2024-04-07T17:51:25.363 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.363 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.363 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.363 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap": { 2024-04-07T17:51:25.364 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 10792, 2024-04-07T17:51:25.364 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 251 2024-04-07T17:51:25.364 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.364 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap_mapping": { 2024-04-07T17:51:25.364 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 70440, 2024-04-07T17:51:25.364 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 5158 2024-04-07T17:51:25.364 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.364 INFO:tasks.workunit.client.0.smithi100.stdout: "pgmap": { 2024-04-07T17:51:25.364 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 5896, 2024-04-07T17:51:25.364 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 174 2024-04-07T17:51:25.364 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.364 INFO:tasks.workunit.client.0.smithi100.stdout: "unittest_1": { 2024-04-07T17:51:25.364 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.364 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.364 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.364 INFO:tasks.workunit.client.0.smithi100.stdout: "unittest_2": { 2024-04-07T17:51:25.365 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.365 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.365 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.365 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.365 INFO:tasks.workunit.client.0.smithi100.stdout: "mon.f47a4ff15a45950a63160265564a123fc5da5e5e": { 2024-04-07T17:51:25.365 INFO:tasks.workunit.client.0.smithi100.stdout: "bloom_filter": { 2024-04-07T17:51:25.365 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.365 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.365 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.365 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs": { 2024-04-07T17:51:25.365 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.365 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.365 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.365 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs_file_reader": { 2024-04-07T17:51:25.365 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.365 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.366 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.366 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs_file_writer": { 2024-04-07T17:51:25.366 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.366 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.366 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.366 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_alloc": { 2024-04-07T17:51:25.366 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.366 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.366 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.366 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_blob": { 2024-04-07T17:51:25.366 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.366 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.366 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.366 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_buffer": { 2024-04-07T17:51:25.367 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.367 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.367 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.367 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_data": { 2024-04-07T17:51:25.367 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.367 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.367 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.367 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_meta": { 2024-04-07T17:51:25.367 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.367 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.367 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.367 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_onode": { 2024-04-07T17:51:25.367 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.367 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.367 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.367 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_other": { 2024-04-07T17:51:25.368 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.368 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.368 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.368 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_extent": { 2024-04-07T17:51:25.368 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.368 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.368 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.368 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_fsck": { 2024-04-07T17:51:25.368 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.368 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.368 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.368 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_inline_bl": { 2024-04-07T17:51:25.368 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.368 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.368 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.368 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_shared_blob": { 2024-04-07T17:51:25.369 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.369 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.369 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.369 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_txc": { 2024-04-07T17:51:25.369 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.369 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.369 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.369 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_writing": { 2024-04-07T17:51:25.369 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.369 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.369 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.369 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_writing_deferred": { 2024-04-07T17:51:25.369 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.369 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.369 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.369 INFO:tasks.workunit.client.0.smithi100.stdout: "buffer_anon": { 2024-04-07T17:51:25.370 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 41172598, 2024-04-07T17:51:25.370 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 5102 2024-04-07T17:51:25.370 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.370 INFO:tasks.workunit.client.0.smithi100.stdout: "buffer_meta": { 2024-04-07T17:51:25.370 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 88, 2024-04-07T17:51:25.370 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 1 2024-04-07T17:51:25.370 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.370 INFO:tasks.workunit.client.0.smithi100.stdout: "mds_co": { 2024-04-07T17:51:25.370 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.370 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.370 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.370 INFO:tasks.workunit.client.0.smithi100.stdout: "osd": { 2024-04-07T17:51:25.370 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.370 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.370 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.371 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mapbl": { 2024-04-07T17:51:25.371 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.371 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.371 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.371 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_pglog": { 2024-04-07T17:51:25.371 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.371 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.371 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.371 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap": { 2024-04-07T17:51:25.371 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 10792, 2024-04-07T17:51:25.371 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 251 2024-04-07T17:51:25.371 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.371 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap_mapping": { 2024-04-07T17:51:25.371 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 70440, 2024-04-07T17:51:25.371 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 5158 2024-04-07T17:51:25.372 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.372 INFO:tasks.workunit.client.0.smithi100.stdout: "pgmap": { 2024-04-07T17:51:25.372 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 11792, 2024-04-07T17:51:25.372 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 348 2024-04-07T17:51:25.372 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.372 INFO:tasks.workunit.client.0.smithi100.stdout: "unittest_1": { 2024-04-07T17:51:25.372 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.372 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.372 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.372 INFO:tasks.workunit.client.0.smithi100.stdout: "unittest_2": { 2024-04-07T17:51:25.372 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.372 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.372 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.372 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.372 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.372 INFO:tasks.workunit.client.0.smithi100.stdout: "osd": { 2024-04-07T17:51:25.372 INFO:tasks.workunit.client.0.smithi100.stdout: "osd.0": { 2024-04-07T17:51:25.373 INFO:tasks.workunit.client.0.smithi100.stdout: "bloom_filter": { 2024-04-07T17:51:25.373 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.373 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.373 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.373 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs": { 2024-04-07T17:51:25.373 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 22768, 2024-04-07T17:51:25.373 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 424 2024-04-07T17:51:25.373 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.373 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs_file_reader": { 2024-04-07T17:51:25.373 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 31360, 2024-04-07T17:51:25.373 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 28 2024-04-07T17:51:25.373 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.373 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs_file_writer": { 2024-04-07T17:51:25.373 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 2496, 2024-04-07T17:51:25.373 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 13 2024-04-07T17:51:25.373 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.374 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_alloc": { 2024-04-07T17:51:25.374 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 59840, 2024-04-07T17:51:25.374 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 748 2024-04-07T17:51:25.374 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.374 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_blob": { 2024-04-07T17:51:25.374 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 382512, 2024-04-07T17:51:25.374 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 1839 2024-04-07T17:51:25.374 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.374 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_buffer": { 2024-04-07T17:51:25.374 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 256, 2024-04-07T17:51:25.374 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 32 2024-04-07T17:51:25.374 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.374 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_data": { 2024-04-07T17:51:25.374 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 774695, 2024-04-07T17:51:25.375 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 215 2024-04-07T17:51:25.375 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.375 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_meta": { 2024-04-07T17:51:25.375 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 2426291, 2024-04-07T17:51:25.375 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 537859 2024-04-07T17:51:25.375 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.375 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_onode": { 2024-04-07T17:51:25.375 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 6943736, 2024-04-07T17:51:25.375 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 10243 2024-04-07T17:51:25.375 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.375 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_other": { 2024-04-07T17:51:25.375 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 72960, 2024-04-07T17:51:25.375 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 7616 2024-04-07T17:51:25.375 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.375 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_extent": { 2024-04-07T17:51:25.375 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 88272, 2024-04-07T17:51:25.376 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 1839 2024-04-07T17:51:25.376 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.376 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_fsck": { 2024-04-07T17:51:25.376 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.376 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.376 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.376 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_inline_bl": { 2024-04-07T17:51:25.376 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 175624, 2024-04-07T17:51:25.376 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 1258 2024-04-07T17:51:25.376 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.376 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_shared_blob": { 2024-04-07T17:51:25.376 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.376 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.376 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.377 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_txc": { 2024-04-07T17:51:25.377 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.377 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.377 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.377 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_writing": { 2024-04-07T17:51:25.377 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 21009, 2024-04-07T17:51:25.377 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 33 2024-04-07T17:51:25.377 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.377 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_writing_deferred": { 2024-04-07T17:51:25.377 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.377 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.377 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.377 INFO:tasks.workunit.client.0.smithi100.stdout: "buffer_anon": { 2024-04-07T17:51:25.377 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 15044831, 2024-04-07T17:51:25.377 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 290 2024-04-07T17:51:25.378 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.378 INFO:tasks.workunit.client.0.smithi100.stdout: "buffer_meta": { 2024-04-07T17:51:25.378 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 19448, 2024-04-07T17:51:25.378 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 221 2024-04-07T17:51:25.378 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.378 INFO:tasks.workunit.client.0.smithi100.stdout: "mds_co": { 2024-04-07T17:51:25.378 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.378 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.378 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.378 INFO:tasks.workunit.client.0.smithi100.stdout: "osd": { 2024-04-07T17:51:25.378 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 513216, 2024-04-07T17:51:25.378 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 44 2024-04-07T17:51:25.378 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.378 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mapbl": { 2024-04-07T17:51:25.378 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.379 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.379 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.379 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_pglog": { 2024-04-07T17:51:25.379 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 7161440, 2024-04-07T17:51:25.379 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 18572 2024-04-07T17:51:25.379 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.379 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap": { 2024-04-07T17:51:25.379 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 635376, 2024-04-07T17:51:25.379 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 11076 2024-04-07T17:51:25.379 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.379 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap_mapping": { 2024-04-07T17:51:25.379 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.379 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.379 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.379 INFO:tasks.workunit.client.0.smithi100.stdout: "pgmap": { 2024-04-07T17:51:25.379 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.380 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.380 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.380 INFO:tasks.workunit.client.0.smithi100.stdout: "unittest_1": { 2024-04-07T17:51:25.380 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.380 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.380 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.380 INFO:tasks.workunit.client.0.smithi100.stdout: "unittest_2": { 2024-04-07T17:51:25.380 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.380 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.380 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.380 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.380 INFO:tasks.workunit.client.0.smithi100.stdout: "osd.1": { 2024-04-07T17:51:25.380 INFO:tasks.workunit.client.0.smithi100.stdout: "bloom_filter": { 2024-04-07T17:51:25.380 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.380 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.381 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.381 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs": { 2024-04-07T17:51:25.381 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 20720, 2024-04-07T17:51:25.381 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 385 2024-04-07T17:51:25.381 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.381 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs_file_reader": { 2024-04-07T17:51:25.381 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 35072, 2024-04-07T17:51:25.381 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 26 2024-04-07T17:51:25.381 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.381 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs_file_writer": { 2024-04-07T17:51:25.381 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 4800, 2024-04-07T17:51:25.381 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 25 2024-04-07T17:51:25.381 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.381 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_alloc": { 2024-04-07T17:51:25.381 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 58160, 2024-04-07T17:51:25.382 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 727 2024-04-07T17:51:25.382 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.382 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_blob": { 2024-04-07T17:51:25.382 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 380224, 2024-04-07T17:51:25.382 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 1828 2024-04-07T17:51:25.382 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.382 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_buffer": { 2024-04-07T17:51:25.382 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 256, 2024-04-07T17:51:25.382 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 32 2024-04-07T17:51:25.382 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.382 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_data": { 2024-04-07T17:51:25.382 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 1045046, 2024-04-07T17:51:25.382 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 285 2024-04-07T17:51:25.382 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.382 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_meta": { 2024-04-07T17:51:25.383 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 2355008, 2024-04-07T17:51:25.383 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 521245 2024-04-07T17:51:25.383 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.383 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_onode": { 2024-04-07T17:51:25.383 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 6739736, 2024-04-07T17:51:25.383 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 9943 2024-04-07T17:51:25.383 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.383 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_other": { 2024-04-07T17:51:25.383 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 70888, 2024-04-07T17:51:25.383 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 7440 2024-04-07T17:51:25.383 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.383 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_extent": { 2024-04-07T17:51:25.383 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 87744, 2024-04-07T17:51:25.383 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 1828 2024-04-07T17:51:25.383 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.383 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_fsck": { 2024-04-07T17:51:25.384 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.384 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.384 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.384 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_inline_bl": { 2024-04-07T17:51:25.384 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 175712, 2024-04-07T17:51:25.384 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 1259 2024-04-07T17:51:25.384 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.384 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_shared_blob": { 2024-04-07T17:51:25.384 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.384 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.384 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.384 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_txc": { 2024-04-07T17:51:25.384 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.384 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.384 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.384 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_writing": { 2024-04-07T17:51:25.385 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 34652, 2024-04-07T17:51:25.385 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 29 2024-04-07T17:51:25.385 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.385 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_writing_deferred": { 2024-04-07T17:51:25.385 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.385 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.385 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.385 INFO:tasks.workunit.client.0.smithi100.stdout: "buffer_anon": { 2024-04-07T17:51:25.385 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 26499475, 2024-04-07T17:51:25.385 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 286 2024-04-07T17:51:25.385 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.385 INFO:tasks.workunit.client.0.smithi100.stdout: "buffer_meta": { 2024-04-07T17:51:25.385 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 26928, 2024-04-07T17:51:25.385 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 306 2024-04-07T17:51:25.385 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.386 INFO:tasks.workunit.client.0.smithi100.stdout: "mds_co": { 2024-04-07T17:51:25.386 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.386 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.386 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.386 INFO:tasks.workunit.client.0.smithi100.stdout: "osd": { 2024-04-07T17:51:25.386 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 443232, 2024-04-07T17:51:25.386 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 38 2024-04-07T17:51:25.386 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.386 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mapbl": { 2024-04-07T17:51:25.386 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.386 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.386 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.386 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_pglog": { 2024-04-07T17:51:25.387 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 7182344, 2024-04-07T17:51:25.387 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 20929 2024-04-07T17:51:25.387 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.387 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap": { 2024-04-07T17:51:25.387 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 635376, 2024-04-07T17:51:25.387 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 11076 2024-04-07T17:51:25.387 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.387 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap_mapping": { 2024-04-07T17:51:25.387 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.387 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.387 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.387 INFO:tasks.workunit.client.0.smithi100.stdout: "pgmap": { 2024-04-07T17:51:25.387 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.387 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.387 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.388 INFO:tasks.workunit.client.0.smithi100.stdout: "unittest_1": { 2024-04-07T17:51:25.388 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.388 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.388 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.388 INFO:tasks.workunit.client.0.smithi100.stdout: "unittest_2": { 2024-04-07T17:51:25.388 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.388 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.388 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.388 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.388 INFO:tasks.workunit.client.0.smithi100.stdout: "osd.2": { 2024-04-07T17:51:25.388 INFO:tasks.workunit.client.0.smithi100.stdout: "bloom_filter": { 2024-04-07T17:51:25.388 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.388 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.388 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.388 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs": { 2024-04-07T17:51:25.388 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 19496, 2024-04-07T17:51:25.389 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 362 2024-04-07T17:51:25.389 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.389 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs_file_reader": { 2024-04-07T17:51:25.389 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 31360, 2024-04-07T17:51:25.389 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 28 2024-04-07T17:51:25.389 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.389 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs_file_writer": { 2024-04-07T17:51:25.389 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 3648, 2024-04-07T17:51:25.389 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 19 2024-04-07T17:51:25.389 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.389 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_alloc": { 2024-04-07T17:51:25.389 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 58080, 2024-04-07T17:51:25.389 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 726 2024-04-07T17:51:25.389 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.389 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_blob": { 2024-04-07T17:51:25.389 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 379600, 2024-04-07T17:51:25.390 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 1825 2024-04-07T17:51:25.390 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.390 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_buffer": { 2024-04-07T17:51:25.390 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 256, 2024-04-07T17:51:25.390 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 32 2024-04-07T17:51:25.390 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.390 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_data": { 2024-04-07T17:51:25.390 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 434802, 2024-04-07T17:51:25.390 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 100 2024-04-07T17:51:25.390 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.390 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_meta": { 2024-04-07T17:51:25.390 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 1711873, 2024-04-07T17:51:25.390 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 379494 2024-04-07T17:51:25.390 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.390 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_onode": { 2024-04-07T17:51:25.391 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 5016616, 2024-04-07T17:51:25.391 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 7409 2024-04-07T17:51:25.391 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.391 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_other": { 2024-04-07T17:51:25.391 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 69948, 2024-04-07T17:51:25.391 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 7427 2024-04-07T17:51:25.391 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.391 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_extent": { 2024-04-07T17:51:25.391 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 87600, 2024-04-07T17:51:25.391 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 1825 2024-04-07T17:51:25.391 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.391 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_fsck": { 2024-04-07T17:51:25.391 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.391 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.391 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.391 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_inline_bl": { 2024-04-07T17:51:25.392 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 175624, 2024-04-07T17:51:25.392 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 1258 2024-04-07T17:51:25.392 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.392 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_shared_blob": { 2024-04-07T17:51:25.392 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.392 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.392 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.392 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_txc": { 2024-04-07T17:51:25.392 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 736, 2024-04-07T17:51:25.392 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 1 2024-04-07T17:51:25.392 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.392 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_writing": { 2024-04-07T17:51:25.392 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 69418, 2024-04-07T17:51:25.392 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 53 2024-04-07T17:51:25.392 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.393 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_writing_deferred": { 2024-04-07T17:51:25.393 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 12112, 2024-04-07T17:51:25.393 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 3 2024-04-07T17:51:25.393 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.393 INFO:tasks.workunit.client.0.smithi100.stdout: "buffer_anon": { 2024-04-07T17:51:25.393 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 20645063, 2024-04-07T17:51:25.393 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 260 2024-04-07T17:51:25.393 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.393 INFO:tasks.workunit.client.0.smithi100.stdout: "buffer_meta": { 2024-04-07T17:51:25.393 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 12760, 2024-04-07T17:51:25.393 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 145 2024-04-07T17:51:25.393 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.393 INFO:tasks.workunit.client.0.smithi100.stdout: "mds_co": { 2024-04-07T17:51:25.393 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.393 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.393 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.394 INFO:tasks.workunit.client.0.smithi100.stdout: "osd": { 2024-04-07T17:51:25.394 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 349920, 2024-04-07T17:51:25.394 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 30 2024-04-07T17:51:25.394 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.394 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mapbl": { 2024-04-07T17:51:25.394 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.394 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.394 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.394 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_pglog": { 2024-04-07T17:51:25.394 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 3170440, 2024-04-07T17:51:25.394 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 6097 2024-04-07T17:51:25.394 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.394 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap": { 2024-04-07T17:51:25.394 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 635376, 2024-04-07T17:51:25.394 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 11076 2024-04-07T17:51:25.395 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.395 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap_mapping": { 2024-04-07T17:51:25.395 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.395 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.395 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.395 INFO:tasks.workunit.client.0.smithi100.stdout: "pgmap": { 2024-04-07T17:51:25.395 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.395 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.395 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.395 INFO:tasks.workunit.client.0.smithi100.stdout: "unittest_1": { 2024-04-07T17:51:25.395 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.395 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.395 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.395 INFO:tasks.workunit.client.0.smithi100.stdout: "unittest_2": { 2024-04-07T17:51:25.395 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.395 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.396 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.396 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.396 INFO:tasks.workunit.client.0.smithi100.stdout: "osd.3": { 2024-04-07T17:51:25.396 INFO:tasks.workunit.client.0.smithi100.stdout: "bloom_filter": { 2024-04-07T17:51:25.396 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.396 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.396 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.396 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs": { 2024-04-07T17:51:25.396 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 17048, 2024-04-07T17:51:25.396 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 316 2024-04-07T17:51:25.396 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.396 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs_file_reader": { 2024-04-07T17:51:25.396 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 30976, 2024-04-07T17:51:25.396 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 25 2024-04-07T17:51:25.396 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.396 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs_file_writer": { 2024-04-07T17:51:25.397 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 3072, 2024-04-07T17:51:25.397 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 16 2024-04-07T17:51:25.397 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.397 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_alloc": { 2024-04-07T17:51:25.397 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 56960, 2024-04-07T17:51:25.397 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 712 2024-04-07T17:51:25.397 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.397 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_blob": { 2024-04-07T17:51:25.397 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 380640, 2024-04-07T17:51:25.397 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 1830 2024-04-07T17:51:25.397 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.397 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_buffer": { 2024-04-07T17:51:25.397 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 256, 2024-04-07T17:51:25.397 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 32 2024-04-07T17:51:25.397 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.398 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_data": { 2024-04-07T17:51:25.398 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 381771, 2024-04-07T17:51:25.398 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 104 2024-04-07T17:51:25.398 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.398 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_meta": { 2024-04-07T17:51:25.398 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 1733357, 2024-04-07T17:51:25.398 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 381544 2024-04-07T17:51:25.398 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.398 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_onode": { 2024-04-07T17:51:25.398 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 5029536, 2024-04-07T17:51:25.398 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 7428 2024-04-07T17:51:25.398 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.398 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_other": { 2024-04-07T17:51:25.398 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 70748, 2024-04-07T17:51:25.398 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 7446 2024-04-07T17:51:25.399 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.399 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_extent": { 2024-04-07T17:51:25.399 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 87840, 2024-04-07T17:51:25.399 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 1830 2024-04-07T17:51:25.399 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.399 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_fsck": { 2024-04-07T17:51:25.399 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.399 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.399 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.399 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_inline_bl": { 2024-04-07T17:51:25.399 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 175624, 2024-04-07T17:51:25.399 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 1258 2024-04-07T17:51:25.399 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.399 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_shared_blob": { 2024-04-07T17:51:25.399 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.400 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.400 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.400 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_txc": { 2024-04-07T17:51:25.400 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 736, 2024-04-07T17:51:25.400 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 1 2024-04-07T17:51:25.400 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.400 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_writing": { 2024-04-07T17:51:25.400 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 83741, 2024-04-07T17:51:25.400 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 55 2024-04-07T17:51:25.400 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.400 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_writing_deferred": { 2024-04-07T17:51:25.400 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 12112, 2024-04-07T17:51:25.400 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 3 2024-04-07T17:51:25.400 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.400 INFO:tasks.workunit.client.0.smithi100.stdout: "buffer_anon": { 2024-04-07T17:51:25.401 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 18443066, 2024-04-07T17:51:25.401 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 280 2024-04-07T17:51:25.401 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.401 INFO:tasks.workunit.client.0.smithi100.stdout: "buffer_meta": { 2024-04-07T17:51:25.401 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 11880, 2024-04-07T17:51:25.401 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 135 2024-04-07T17:51:25.401 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.401 INFO:tasks.workunit.client.0.smithi100.stdout: "mds_co": { 2024-04-07T17:51:25.401 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.401 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.401 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.401 INFO:tasks.workunit.client.0.smithi100.stdout: "osd": { 2024-04-07T17:51:25.401 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 419904, 2024-04-07T17:51:25.401 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 36 2024-04-07T17:51:25.401 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.401 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mapbl": { 2024-04-07T17:51:25.402 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.402 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.402 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.402 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_pglog": { 2024-04-07T17:51:25.402 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 3485560, 2024-04-07T17:51:25.402 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 9103 2024-04-07T17:51:25.402 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.402 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap": { 2024-04-07T17:51:25.402 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 635376, 2024-04-07T17:51:25.402 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 11076 2024-04-07T17:51:25.402 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.402 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap_mapping": { 2024-04-07T17:51:25.402 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.402 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.402 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.403 INFO:tasks.workunit.client.0.smithi100.stdout: "pgmap": { 2024-04-07T17:51:25.403 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.403 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.403 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.403 INFO:tasks.workunit.client.0.smithi100.stdout: "unittest_1": { 2024-04-07T17:51:25.403 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.403 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.403 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.403 INFO:tasks.workunit.client.0.smithi100.stdout: "unittest_2": { 2024-04-07T17:51:25.403 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.403 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.403 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.403 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.403 INFO:tasks.workunit.client.0.smithi100.stdout: "osd.4": { 2024-04-07T17:51:25.403 INFO:tasks.workunit.client.0.smithi100.stdout: "bloom_filter": { 2024-04-07T17:51:25.403 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.404 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.404 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.404 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs": { 2024-04-07T17:51:25.404 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 20816, 2024-04-07T17:51:25.404 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 393 2024-04-07T17:51:25.404 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.404 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs_file_reader": { 2024-04-07T17:51:25.404 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 31488, 2024-04-07T17:51:25.404 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 29 2024-04-07T17:51:25.404 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.404 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs_file_writer": { 2024-04-07T17:51:25.404 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 4032, 2024-04-07T17:51:25.404 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 21 2024-04-07T17:51:25.404 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.404 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_alloc": { 2024-04-07T17:51:25.404 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 58240, 2024-04-07T17:51:25.404 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 728 2024-04-07T17:51:25.405 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.405 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_blob": { 2024-04-07T17:51:25.405 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 393952, 2024-04-07T17:51:25.405 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 1894 2024-04-07T17:51:25.405 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.405 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_buffer": { 2024-04-07T17:51:25.405 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 256, 2024-04-07T17:51:25.405 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 32 2024-04-07T17:51:25.405 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.405 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_data": { 2024-04-07T17:51:25.405 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 1050407, 2024-04-07T17:51:25.405 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 273 2024-04-07T17:51:25.405 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.405 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_meta": { 2024-04-07T17:51:25.405 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 2027647, 2024-04-07T17:51:25.405 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 446306 2024-04-07T17:51:25.406 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.406 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_onode": { 2024-04-07T17:51:25.406 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 5812216, 2024-04-07T17:51:25.406 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 8579 2024-04-07T17:51:25.406 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.406 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_other": { 2024-04-07T17:51:25.406 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 80384, 2024-04-07T17:51:25.406 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 8599 2024-04-07T17:51:25.406 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.406 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_extent": { 2024-04-07T17:51:25.406 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 90912, 2024-04-07T17:51:25.406 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 1894 2024-04-07T17:51:25.406 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.406 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_fsck": { 2024-04-07T17:51:25.406 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.407 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.407 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.407 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_inline_bl": { 2024-04-07T17:51:25.407 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 175456, 2024-04-07T17:51:25.407 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 1258 2024-04-07T17:51:25.407 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.407 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_shared_blob": { 2024-04-07T17:51:25.407 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.407 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.407 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.407 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_txc": { 2024-04-07T17:51:25.407 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 736, 2024-04-07T17:51:25.407 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 1 2024-04-07T17:51:25.407 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.407 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_writing": { 2024-04-07T17:51:25.407 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 66705, 2024-04-07T17:51:25.408 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 51 2024-04-07T17:51:25.408 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.408 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_writing_deferred": { 2024-04-07T17:51:25.408 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 12112, 2024-04-07T17:51:25.408 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 3 2024-04-07T17:51:25.408 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.408 INFO:tasks.workunit.client.0.smithi100.stdout: "buffer_anon": { 2024-04-07T17:51:25.408 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 23146638, 2024-04-07T17:51:25.408 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 273 2024-04-07T17:51:25.408 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.408 INFO:tasks.workunit.client.0.smithi100.stdout: "buffer_meta": { 2024-04-07T17:51:25.408 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 26928, 2024-04-07T17:51:25.408 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 306 2024-04-07T17:51:25.408 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.408 INFO:tasks.workunit.client.0.smithi100.stdout: "mds_co": { 2024-04-07T17:51:25.408 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.409 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.409 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.409 INFO:tasks.workunit.client.0.smithi100.stdout: "osd": { 2024-04-07T17:51:25.409 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 466560, 2024-04-07T17:51:25.409 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 40 2024-04-07T17:51:25.409 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.409 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mapbl": { 2024-04-07T17:51:25.409 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.409 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.409 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.409 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_pglog": { 2024-04-07T17:51:25.409 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 5065320, 2024-04-07T17:51:25.409 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 12141 2024-04-07T17:51:25.409 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.409 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap": { 2024-04-07T17:51:25.409 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 635376, 2024-04-07T17:51:25.410 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 11076 2024-04-07T17:51:25.410 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.410 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap_mapping": { 2024-04-07T17:51:25.410 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.410 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.410 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.410 INFO:tasks.workunit.client.0.smithi100.stdout: "pgmap": { 2024-04-07T17:51:25.410 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.410 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.410 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.410 INFO:tasks.workunit.client.0.smithi100.stdout: "unittest_1": { 2024-04-07T17:51:25.410 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.410 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.410 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.411 INFO:tasks.workunit.client.0.smithi100.stdout: "unittest_2": { 2024-04-07T17:51:25.411 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.411 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.411 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.411 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.411 INFO:tasks.workunit.client.0.smithi100.stdout: "osd.5": { 2024-04-07T17:51:25.411 INFO:tasks.workunit.client.0.smithi100.stdout: "bloom_filter": { 2024-04-07T17:51:25.411 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.411 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.411 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.411 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs": { 2024-04-07T17:51:25.411 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 19216, 2024-04-07T17:51:25.411 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 365 2024-04-07T17:51:25.411 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.411 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs_file_reader": { 2024-04-07T17:51:25.412 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 31616, 2024-04-07T17:51:25.412 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 30 2024-04-07T17:51:25.412 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.412 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs_file_writer": { 2024-04-07T17:51:25.412 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 3072, 2024-04-07T17:51:25.412 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 16 2024-04-07T17:51:25.412 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.412 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_alloc": { 2024-04-07T17:51:25.412 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 58400, 2024-04-07T17:51:25.412 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 730 2024-04-07T17:51:25.412 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.412 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_blob": { 2024-04-07T17:51:25.412 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 393952, 2024-04-07T17:51:25.412 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 1894 2024-04-07T17:51:25.412 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.412 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_buffer": { 2024-04-07T17:51:25.413 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 256, 2024-04-07T17:51:25.413 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 32 2024-04-07T17:51:25.413 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.413 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_data": { 2024-04-07T17:51:25.413 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 274397, 2024-04-07T17:51:25.413 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 87 2024-04-07T17:51:25.413 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.413 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_meta": { 2024-04-07T17:51:25.413 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 2323821, 2024-04-07T17:51:25.413 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 518276 2024-04-07T17:51:25.413 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.413 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_onode": { 2024-04-07T17:51:25.413 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 6677176, 2024-04-07T17:51:25.413 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 9851 2024-04-07T17:51:25.413 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.413 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_other": { 2024-04-07T17:51:25.414 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 80636, 2024-04-07T17:51:25.414 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 8604 2024-04-07T17:51:25.414 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.414 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_extent": { 2024-04-07T17:51:25.414 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 90912, 2024-04-07T17:51:25.414 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 1894 2024-04-07T17:51:25.414 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.414 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_fsck": { 2024-04-07T17:51:25.414 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.414 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.414 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.414 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_inline_bl": { 2024-04-07T17:51:25.414 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 175712, 2024-04-07T17:51:25.414 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 1259 2024-04-07T17:51:25.414 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.415 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_shared_blob": { 2024-04-07T17:51:25.415 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.415 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.415 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.415 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_txc": { 2024-04-07T17:51:25.415 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 736, 2024-04-07T17:51:25.415 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 1 2024-04-07T17:51:25.415 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.415 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_writing": { 2024-04-07T17:51:25.415 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 87571, 2024-04-07T17:51:25.415 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 56 2024-04-07T17:51:25.415 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.415 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_writing_deferred": { 2024-04-07T17:51:25.415 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 12112, 2024-04-07T17:51:25.415 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 3 2024-04-07T17:51:25.415 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.416 INFO:tasks.workunit.client.0.smithi100.stdout: "buffer_anon": { 2024-04-07T17:51:25.416 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 18319894, 2024-04-07T17:51:25.416 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 12576 2024-04-07T17:51:25.416 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.416 INFO:tasks.workunit.client.0.smithi100.stdout: "buffer_meta": { 2024-04-07T17:51:25.416 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 10208, 2024-04-07T17:51:25.416 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 116 2024-04-07T17:51:25.416 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.416 INFO:tasks.workunit.client.0.smithi100.stdout: "mds_co": { 2024-04-07T17:51:25.416 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.416 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.416 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.416 INFO:tasks.workunit.client.0.smithi100.stdout: "osd": { 2024-04-07T17:51:25.416 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 489888, 2024-04-07T17:51:25.417 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 42 2024-04-07T17:51:25.417 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.417 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mapbl": { 2024-04-07T17:51:25.417 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.417 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.417 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.417 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_pglog": { 2024-04-07T17:51:25.417 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 6315920, 2024-04-07T17:51:25.417 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 12146 2024-04-07T17:51:25.417 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.417 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap": { 2024-04-07T17:51:25.417 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 635376, 2024-04-07T17:51:25.417 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 11076 2024-04-07T17:51:25.417 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.417 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap_mapping": { 2024-04-07T17:51:25.417 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.417 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.418 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.418 INFO:tasks.workunit.client.0.smithi100.stdout: "pgmap": { 2024-04-07T17:51:25.418 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.418 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.418 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.418 INFO:tasks.workunit.client.0.smithi100.stdout: "unittest_1": { 2024-04-07T17:51:25.418 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.418 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.418 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.418 INFO:tasks.workunit.client.0.smithi100.stdout: "unittest_2": { 2024-04-07T17:51:25.418 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.418 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.418 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.418 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.418 INFO:tasks.workunit.client.0.smithi100.stdout: "osd.6": { 2024-04-07T17:51:25.419 INFO:tasks.workunit.client.0.smithi100.stdout: "bloom_filter": { 2024-04-07T17:51:25.419 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.419 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.419 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.419 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs": { 2024-04-07T17:51:25.419 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 13656, 2024-04-07T17:51:25.419 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 230 2024-04-07T17:51:25.419 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.419 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs_file_reader": { 2024-04-07T17:51:25.419 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 30336, 2024-04-07T17:51:25.419 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 20 2024-04-07T17:51:25.419 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.419 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs_file_writer": { 2024-04-07T17:51:25.419 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 2496, 2024-04-07T17:51:25.419 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 13 2024-04-07T17:51:25.419 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.420 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_alloc": { 2024-04-07T17:51:25.420 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 56960, 2024-04-07T17:51:25.420 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 712 2024-04-07T17:51:25.420 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.420 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_blob": { 2024-04-07T17:51:25.420 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 395616, 2024-04-07T17:51:25.420 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 1902 2024-04-07T17:51:25.420 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.420 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_buffer": { 2024-04-07T17:51:25.420 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 256, 2024-04-07T17:51:25.420 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 32 2024-04-07T17:51:25.420 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.420 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_data": { 2024-04-07T17:51:25.420 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 463979, 2024-04-07T17:51:25.420 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 128 2024-04-07T17:51:25.421 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.421 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_meta": { 2024-04-07T17:51:25.421 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 1718221, 2024-04-07T17:51:25.421 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 377358 2024-04-07T17:51:25.421 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.421 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_onode": { 2024-04-07T17:51:25.421 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 4954056, 2024-04-07T17:51:25.421 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 7317 2024-04-07T17:51:25.421 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.421 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_other": { 2024-04-07T17:51:25.421 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 81560, 2024-04-07T17:51:25.421 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 8760 2024-04-07T17:51:25.421 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.421 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_extent": { 2024-04-07T17:51:25.421 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 91296, 2024-04-07T17:51:25.421 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 1902 2024-04-07T17:51:25.422 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.422 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_fsck": { 2024-04-07T17:51:25.422 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.422 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.422 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.422 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_inline_bl": { 2024-04-07T17:51:25.422 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 175368, 2024-04-07T17:51:25.422 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 1257 2024-04-07T17:51:25.422 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.422 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_shared_blob": { 2024-04-07T17:51:25.422 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.422 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.422 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.422 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_txc": { 2024-04-07T17:51:25.423 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.423 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.423 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.423 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_writing": { 2024-04-07T17:51:25.423 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 69389, 2024-04-07T17:51:25.423 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 49 2024-04-07T17:51:25.423 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.423 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_writing_deferred": { 2024-04-07T17:51:25.423 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.423 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.423 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.423 INFO:tasks.workunit.client.0.smithi100.stdout: "buffer_anon": { 2024-04-07T17:51:25.423 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 13267228, 2024-04-07T17:51:25.423 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 243 2024-04-07T17:51:25.423 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.423 INFO:tasks.workunit.client.0.smithi100.stdout: "buffer_meta": { 2024-04-07T17:51:25.424 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 12320, 2024-04-07T17:51:25.424 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 140 2024-04-07T17:51:25.424 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.424 INFO:tasks.workunit.client.0.smithi100.stdout: "mds_co": { 2024-04-07T17:51:25.424 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.424 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.424 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.424 INFO:tasks.workunit.client.0.smithi100.stdout: "osd": { 2024-04-07T17:51:25.424 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 454896, 2024-04-07T17:51:25.424 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 39 2024-04-07T17:51:25.424 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.424 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mapbl": { 2024-04-07T17:51:25.424 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.424 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.424 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.424 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_pglog": { 2024-04-07T17:51:25.425 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 3662880, 2024-04-07T17:51:25.425 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 9444 2024-04-07T17:51:25.425 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.425 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap": { 2024-04-07T17:51:25.425 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 635376, 2024-04-07T17:51:25.425 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 11076 2024-04-07T17:51:25.425 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.425 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap_mapping": { 2024-04-07T17:51:25.425 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.425 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.425 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.425 INFO:tasks.workunit.client.0.smithi100.stdout: "pgmap": { 2024-04-07T17:51:25.425 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.425 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.425 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.425 INFO:tasks.workunit.client.0.smithi100.stdout: "unittest_1": { 2024-04-07T17:51:25.426 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.426 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.426 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.426 INFO:tasks.workunit.client.0.smithi100.stdout: "unittest_2": { 2024-04-07T17:51:25.426 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.426 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.426 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.426 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.426 INFO:tasks.workunit.client.0.smithi100.stdout: "osd.7": { 2024-04-07T17:51:25.426 INFO:tasks.workunit.client.0.smithi100.stdout: "bloom_filter": { 2024-04-07T17:51:25.426 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.426 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.426 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.426 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs": { 2024-04-07T17:51:25.427 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 15992, 2024-04-07T17:51:25.427 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 307 2024-04-07T17:51:25.427 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.427 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs_file_reader": { 2024-04-07T17:51:25.427 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 39040, 2024-04-07T17:51:25.427 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 25 2024-04-07T17:51:25.427 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.427 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs_file_writer": { 2024-04-07T17:51:25.427 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 2688, 2024-04-07T17:51:25.427 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 14 2024-04-07T17:51:25.427 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.427 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_alloc": { 2024-04-07T17:51:25.427 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 57280, 2024-04-07T17:51:25.427 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 716 2024-04-07T17:51:25.427 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.427 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_blob": { 2024-04-07T17:51:25.428 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 382096, 2024-04-07T17:51:25.428 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 1837 2024-04-07T17:51:25.428 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.428 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_buffer": { 2024-04-07T17:51:25.428 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 256, 2024-04-07T17:51:25.428 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 32 2024-04-07T17:51:25.428 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.428 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_data": { 2024-04-07T17:51:25.428 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 427947, 2024-04-07T17:51:25.428 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 127 2024-04-07T17:51:25.428 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.428 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_meta": { 2024-04-07T17:51:25.428 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 1987758, 2024-04-07T17:51:25.428 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 443621 2024-04-07T17:51:25.428 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.428 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_onode": { 2024-04-07T17:51:25.429 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 5763936, 2024-04-07T17:51:25.429 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 8508 2024-04-07T17:51:25.429 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.429 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_other": { 2024-04-07T17:51:25.429 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 73080, 2024-04-07T17:51:25.429 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 7610 2024-04-07T17:51:25.429 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.429 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_extent": { 2024-04-07T17:51:25.429 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 88176, 2024-04-07T17:51:25.429 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 1837 2024-04-07T17:51:25.429 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.429 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_fsck": { 2024-04-07T17:51:25.429 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.429 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.429 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.430 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_inline_bl": { 2024-04-07T17:51:25.430 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 175624, 2024-04-07T17:51:25.430 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 1258 2024-04-07T17:51:25.430 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.430 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_shared_blob": { 2024-04-07T17:51:25.430 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.430 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.430 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.430 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_txc": { 2024-04-07T17:51:25.430 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.430 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.430 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.430 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_writing": { 2024-04-07T17:51:25.430 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 21282, 2024-04-07T17:51:25.431 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 25 2024-04-07T17:51:25.431 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.431 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_writing_deferred": { 2024-04-07T17:51:25.431 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.431 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.431 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.431 INFO:tasks.workunit.client.0.smithi100.stdout: "buffer_anon": { 2024-04-07T17:51:25.431 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 14934597, 2024-04-07T17:51:25.431 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 264 2024-04-07T17:51:25.431 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.431 INFO:tasks.workunit.client.0.smithi100.stdout: "buffer_meta": { 2024-04-07T17:51:25.431 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 10912, 2024-04-07T17:51:25.431 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 124 2024-04-07T17:51:25.431 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.431 INFO:tasks.workunit.client.0.smithi100.stdout: "mds_co": { 2024-04-07T17:51:25.431 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.432 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.432 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.432 INFO:tasks.workunit.client.0.smithi100.stdout: "osd": { 2024-04-07T17:51:25.432 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 536544, 2024-04-07T17:51:25.432 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 46 2024-04-07T17:51:25.432 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.432 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mapbl": { 2024-04-07T17:51:25.432 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.432 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.432 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.432 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_pglog": { 2024-04-07T17:51:25.432 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 5000736, 2024-04-07T17:51:25.432 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 9636 2024-04-07T17:51:25.432 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.432 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap": { 2024-04-07T17:51:25.433 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 635376, 2024-04-07T17:51:25.433 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 11076 2024-04-07T17:51:25.433 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.433 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap_mapping": { 2024-04-07T17:51:25.433 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.433 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.433 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.433 INFO:tasks.workunit.client.0.smithi100.stdout: "pgmap": { 2024-04-07T17:51:25.433 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.433 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.433 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.433 INFO:tasks.workunit.client.0.smithi100.stdout: "unittest_1": { 2024-04-07T17:51:25.433 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.433 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.433 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.433 INFO:tasks.workunit.client.0.smithi100.stdout: "unittest_2": { 2024-04-07T17:51:25.434 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:25.434 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:25.434 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.434 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.434 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.434 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.434 INFO:tasks.workunit.client.0.smithi100.stdout: "metadata": { 2024-04-07T17:51:25.434 INFO:tasks.workunit.client.0.smithi100.stdout: "mds": { 2024-04-07T17:51:25.434 INFO:tasks.workunit.client.0.smithi100.stdout: "arch": { 2024-04-07T17:51:25.434 INFO:tasks.workunit.client.0.smithi100.stdout: "x86_64": 2 2024-04-07T17:51:25.434 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.434 INFO:tasks.workunit.client.0.smithi100.stdout: "ceph_version": { 2024-04-07T17:51:25.434 INFO:tasks.workunit.client.0.smithi100.stdout: "ceph version 19.0.0-2785-gadce92d0 (adce92d056533f22dc1ee37e3141d098b046ec1b) squid (dev)": 2 2024-04-07T17:51:25.434 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.435 INFO:tasks.workunit.client.0.smithi100.stdout: "cpu": { 2024-04-07T17:51:25.435 INFO:tasks.workunit.client.0.smithi100.stdout: "Intel(R) Xeon(R) CPU E5-1620 v4 @ 3.50GHz": 2 2024-04-07T17:51:25.435 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.435 INFO:tasks.workunit.client.0.smithi100.stdout: "distro": { 2024-04-07T17:51:25.435 INFO:tasks.workunit.client.0.smithi100.stdout: "centos": 2 2024-04-07T17:51:25.435 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.435 INFO:tasks.workunit.client.0.smithi100.stdout: "distro_description": { 2024-04-07T17:51:25.435 INFO:tasks.workunit.client.0.smithi100.stdout: "CentOS Stream 8": 2 2024-04-07T17:51:25.435 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.435 INFO:tasks.workunit.client.0.smithi100.stdout: "kernel_description": { 2024-04-07T17:51:25.435 INFO:tasks.workunit.client.0.smithi100.stdout: "#111-Ubuntu SMP Tue Mar 5 20:16:58 UTC 2024": 2 2024-04-07T17:51:25.435 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.435 INFO:tasks.workunit.client.0.smithi100.stdout: "kernel_version": { 2024-04-07T17:51:25.435 INFO:tasks.workunit.client.0.smithi100.stdout: "5.15.0-101-generic": 2 2024-04-07T17:51:25.435 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.436 INFO:tasks.workunit.client.0.smithi100.stdout: "os": { 2024-04-07T17:51:25.436 INFO:tasks.workunit.client.0.smithi100.stdout: "Linux": 2 2024-04-07T17:51:25.436 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.436 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.436 INFO:tasks.workunit.client.0.smithi100.stdout: "mon": { 2024-04-07T17:51:25.436 INFO:tasks.workunit.client.0.smithi100.stdout: "arch": { 2024-04-07T17:51:25.436 INFO:tasks.workunit.client.0.smithi100.stdout: "x86_64": 3 2024-04-07T17:51:25.436 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.436 INFO:tasks.workunit.client.0.smithi100.stdout: "ceph_version": { 2024-04-07T17:51:25.436 INFO:tasks.workunit.client.0.smithi100.stdout: "ceph version 19.0.0-2785-gadce92d0 (adce92d056533f22dc1ee37e3141d098b046ec1b) squid (dev)": 3 2024-04-07T17:51:25.436 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.436 INFO:tasks.workunit.client.0.smithi100.stdout: "cpu": { 2024-04-07T17:51:25.436 INFO:tasks.workunit.client.0.smithi100.stdout: "Intel(R) Xeon(R) CPU E5-1620 v4 @ 3.50GHz": 3 2024-04-07T17:51:25.436 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.436 INFO:tasks.workunit.client.0.smithi100.stdout: "distro": { 2024-04-07T17:51:25.437 INFO:tasks.workunit.client.0.smithi100.stdout: "centos": 3 2024-04-07T17:51:25.437 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.437 INFO:tasks.workunit.client.0.smithi100.stdout: "distro_description": { 2024-04-07T17:51:25.437 INFO:tasks.workunit.client.0.smithi100.stdout: "CentOS Stream 8": 3 2024-04-07T17:51:25.437 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.437 INFO:tasks.workunit.client.0.smithi100.stdout: "kernel_description": { 2024-04-07T17:51:25.437 INFO:tasks.workunit.client.0.smithi100.stdout: "#111-Ubuntu SMP Tue Mar 5 20:16:58 UTC 2024": 3 2024-04-07T17:51:25.437 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.437 INFO:tasks.workunit.client.0.smithi100.stdout: "kernel_version": { 2024-04-07T17:51:25.437 INFO:tasks.workunit.client.0.smithi100.stdout: "5.15.0-101-generic": 3 2024-04-07T17:51:25.437 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.437 INFO:tasks.workunit.client.0.smithi100.stdout: "os": { 2024-04-07T17:51:25.437 INFO:tasks.workunit.client.0.smithi100.stdout: "Linux": 3 2024-04-07T17:51:25.437 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.437 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.438 INFO:tasks.workunit.client.0.smithi100.stdout: "osd": { 2024-04-07T17:51:25.438 INFO:tasks.workunit.client.0.smithi100.stdout: "arch": { 2024-04-07T17:51:25.438 INFO:tasks.workunit.client.0.smithi100.stdout: "x86_64": 8 2024-04-07T17:51:25.438 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.438 INFO:tasks.workunit.client.0.smithi100.stdout: "ceph_version": { 2024-04-07T17:51:25.438 INFO:tasks.workunit.client.0.smithi100.stdout: "ceph version 19.0.0-2785-gadce92d0 (adce92d056533f22dc1ee37e3141d098b046ec1b) squid (dev)": 8 2024-04-07T17:51:25.438 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.438 INFO:tasks.workunit.client.0.smithi100.stdout: "cpu": { 2024-04-07T17:51:25.438 INFO:tasks.workunit.client.0.smithi100.stdout: "Intel(R) Xeon(R) CPU E5-1620 v4 @ 3.50GHz": 8 2024-04-07T17:51:25.438 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.438 INFO:tasks.workunit.client.0.smithi100.stdout: "distro": { 2024-04-07T17:51:25.438 INFO:tasks.workunit.client.0.smithi100.stdout: "centos": 8 2024-04-07T17:51:25.438 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.438 INFO:tasks.workunit.client.0.smithi100.stdout: "distro_description": { 2024-04-07T17:51:25.438 INFO:tasks.workunit.client.0.smithi100.stdout: "CentOS Stream 8": 8 2024-04-07T17:51:25.439 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.439 INFO:tasks.workunit.client.0.smithi100.stdout: "kernel_description": { 2024-04-07T17:51:25.439 INFO:tasks.workunit.client.0.smithi100.stdout: "#111-Ubuntu SMP Tue Mar 5 20:16:58 UTC 2024": 8 2024-04-07T17:51:25.439 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.439 INFO:tasks.workunit.client.0.smithi100.stdout: "kernel_version": { 2024-04-07T17:51:25.439 INFO:tasks.workunit.client.0.smithi100.stdout: "5.15.0-101-generic": 8 2024-04-07T17:51:25.439 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.439 INFO:tasks.workunit.client.0.smithi100.stdout: "os": { 2024-04-07T17:51:25.439 INFO:tasks.workunit.client.0.smithi100.stdout: "Linux": 8 2024-04-07T17:51:25.439 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.439 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_objectstore": { 2024-04-07T17:51:25.439 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore": 8 2024-04-07T17:51:25.439 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.439 INFO:tasks.workunit.client.0.smithi100.stdout: "rotational": { 2024-04-07T17:51:25.439 INFO:tasks.workunit.client.0.smithi100.stdout: "0": 8 2024-04-07T17:51:25.439 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.439 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.440 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.440 INFO:tasks.workunit.client.0.smithi100.stdout: "mon": { 2024-04-07T17:51:25.440 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 3, 2024-04-07T17:51:25.440 INFO:tasks.workunit.client.0.smithi100.stdout: "features": { 2024-04-07T17:51:25.440 INFO:tasks.workunit.client.0.smithi100.stdout: "optional": [], 2024-04-07T17:51:25.440 INFO:tasks.workunit.client.0.smithi100.stdout: "persistent": [ 2024-04-07T17:51:25.440 INFO:tasks.workunit.client.0.smithi100.stdout: "kraken", 2024-04-07T17:51:25.440 INFO:tasks.workunit.client.0.smithi100.stdout: "luminous", 2024-04-07T17:51:25.440 INFO:tasks.workunit.client.0.smithi100.stdout: "mimic", 2024-04-07T17:51:25.440 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap-prune", 2024-04-07T17:51:25.440 INFO:tasks.workunit.client.0.smithi100.stdout: "nautilus", 2024-04-07T17:51:25.440 INFO:tasks.workunit.client.0.smithi100.stdout: "octopus", 2024-04-07T17:51:25.440 INFO:tasks.workunit.client.0.smithi100.stdout: "pacific", 2024-04-07T17:51:25.440 INFO:tasks.workunit.client.0.smithi100.stdout: "elector-pinging", 2024-04-07T17:51:25.440 INFO:tasks.workunit.client.0.smithi100.stdout: "quincy", 2024-04-07T17:51:25.440 INFO:tasks.workunit.client.0.smithi100.stdout: "reef", 2024-04-07T17:51:25.441 INFO:tasks.workunit.client.0.smithi100.stdout: "squid" 2024-04-07T17:51:25.441 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:25.441 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.441 INFO:tasks.workunit.client.0.smithi100.stdout: "ipv4_addr_mons": 6, 2024-04-07T17:51:25.441 INFO:tasks.workunit.client.0.smithi100.stdout: "ipv6_addr_mons": 0, 2024-04-07T17:51:25.441 INFO:tasks.workunit.client.0.smithi100.stdout: "min_mon_release": 19, 2024-04-07T17:51:25.441 INFO:tasks.workunit.client.0.smithi100.stdout: "v1_addr_mons": 3, 2024-04-07T17:51:25.441 INFO:tasks.workunit.client.0.smithi100.stdout: "v2_addr_mons": 3 2024-04-07T17:51:25.441 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.441 INFO:tasks.workunit.client.0.smithi100.stdout: "organization": "ceph-qa", 2024-04-07T17:51:25.441 INFO:tasks.workunit.client.0.smithi100.stdout: "osd": { 2024-04-07T17:51:25.441 INFO:tasks.workunit.client.0.smithi100.stdout: "cluster_network": false, 2024-04-07T17:51:25.441 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 8, 2024-04-07T17:51:25.441 INFO:tasks.workunit.client.0.smithi100.stdout: "require_min_compat_client": "luminous", 2024-04-07T17:51:25.441 INFO:tasks.workunit.client.0.smithi100.stdout: "require_osd_release": "squid" 2024-04-07T17:51:25.441 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.442 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_perf_histograms": [ 2024-04-07T17:51:25.442 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:25.442 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_latency_out_bytes_histogram": { 2024-04-07T17:51:25.442 INFO:tasks.workunit.client.0.smithi100.stdout: "axes": [ 2024-04-07T17:51:25.442 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:25.442 INFO:tasks.workunit.client.0.smithi100.stdout: "buckets": 32, 2024-04-07T17:51:25.442 INFO:tasks.workunit.client.0.smithi100.stdout: "min": 0, 2024-04-07T17:51:25.442 INFO:tasks.workunit.client.0.smithi100.stdout: "name": "Latency (usec)", 2024-04-07T17:51:25.442 INFO:tasks.workunit.client.0.smithi100.stdout: "quant_size": 100000, 2024-04-07T17:51:25.442 INFO:tasks.workunit.client.0.smithi100.stdout: "ranges": [ 2024-04-07T17:51:25.442 INFO:tasks.workunit.client.0.smithi100.stdout: "[None, -1]", 2024-04-07T17:51:25.442 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 99999]", 2024-04-07T17:51:25.442 INFO:tasks.workunit.client.0.smithi100.stdout: "[100000, 199999]", 2024-04-07T17:51:25.442 INFO:tasks.workunit.client.0.smithi100.stdout: "[200000, 399999]", 2024-04-07T17:51:25.442 INFO:tasks.workunit.client.0.smithi100.stdout: "[400000, 799999]", 2024-04-07T17:51:25.443 INFO:tasks.workunit.client.0.smithi100.stdout: "[800000, 1599999]", 2024-04-07T17:51:25.443 INFO:tasks.workunit.client.0.smithi100.stdout: "[1600000, 3199999]", 2024-04-07T17:51:25.443 INFO:tasks.workunit.client.0.smithi100.stdout: "[3200000, 6399999]", 2024-04-07T17:51:25.443 INFO:tasks.workunit.client.0.smithi100.stdout: "[6400000, 12799999]", 2024-04-07T17:51:25.443 INFO:tasks.workunit.client.0.smithi100.stdout: "[12800000, 25599999]", 2024-04-07T17:51:25.443 INFO:tasks.workunit.client.0.smithi100.stdout: "[25600000, 51199999]", 2024-04-07T17:51:25.443 INFO:tasks.workunit.client.0.smithi100.stdout: "[51200000, 102399999]", 2024-04-07T17:51:25.443 INFO:tasks.workunit.client.0.smithi100.stdout: "[102400000, 204799999]", 2024-04-07T17:51:25.443 INFO:tasks.workunit.client.0.smithi100.stdout: "[204800000, 409599999]", 2024-04-07T17:51:25.443 INFO:tasks.workunit.client.0.smithi100.stdout: "[409600000, 819199999]", 2024-04-07T17:51:25.443 INFO:tasks.workunit.client.0.smithi100.stdout: "[819200000, 1638399999]", 2024-04-07T17:51:25.443 INFO:tasks.workunit.client.0.smithi100.stdout: "[1638400000, 3276799999]", 2024-04-07T17:51:25.443 INFO:tasks.workunit.client.0.smithi100.stdout: "[3276800000, 6553599999]", 2024-04-07T17:51:25.443 INFO:tasks.workunit.client.0.smithi100.stdout: "[6553600000, 13107199999]", 2024-04-07T17:51:25.443 INFO:tasks.workunit.client.0.smithi100.stdout: "[13107200000, 26214399999]", 2024-04-07T17:51:25.443 INFO:tasks.workunit.client.0.smithi100.stdout: "[26214400000, 52428799999]", 2024-04-07T17:51:25.443 INFO:tasks.workunit.client.0.smithi100.stdout: "[52428800000, 104857599999]", 2024-04-07T17:51:25.444 INFO:tasks.workunit.client.0.smithi100.stdout: "[104857600000, 209715199999]", 2024-04-07T17:51:25.444 INFO:tasks.workunit.client.0.smithi100.stdout: "[209715200000, 419430399999]", 2024-04-07T17:51:25.444 INFO:tasks.workunit.client.0.smithi100.stdout: "[419430400000, 838860799999]", 2024-04-07T17:51:25.444 INFO:tasks.workunit.client.0.smithi100.stdout: "[838860800000, 1677721599999]", 2024-04-07T17:51:25.444 INFO:tasks.workunit.client.0.smithi100.stdout: "[1677721600000, 3355443199999]", 2024-04-07T17:51:25.444 INFO:tasks.workunit.client.0.smithi100.stdout: "[3355443200000, 6710886399999]", 2024-04-07T17:51:25.444 INFO:tasks.workunit.client.0.smithi100.stdout: "[6710886400000, 13421772799999]", 2024-04-07T17:51:25.444 INFO:tasks.workunit.client.0.smithi100.stdout: "[13421772800000, 26843545599999]", 2024-04-07T17:51:25.444 INFO:tasks.workunit.client.0.smithi100.stdout: "[26843545600000, 53687091199999]", 2024-04-07T17:51:25.444 INFO:tasks.workunit.client.0.smithi100.stdout: "[53687091200000, None]" 2024-04-07T17:51:25.444 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:25.444 INFO:tasks.workunit.client.0.smithi100.stdout: "scale_type": "log2" 2024-04-07T17:51:25.444 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.444 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:25.444 INFO:tasks.workunit.client.0.smithi100.stdout: "buckets": 32, 2024-04-07T17:51:25.445 INFO:tasks.workunit.client.0.smithi100.stdout: "min": 0, 2024-04-07T17:51:25.445 INFO:tasks.workunit.client.0.smithi100.stdout: "name": "Request size (bytes)", 2024-04-07T17:51:25.445 INFO:tasks.workunit.client.0.smithi100.stdout: "quant_size": 512, 2024-04-07T17:51:25.445 INFO:tasks.workunit.client.0.smithi100.stdout: "ranges": [ 2024-04-07T17:51:25.445 INFO:tasks.workunit.client.0.smithi100.stdout: "[None, -1]", 2024-04-07T17:51:25.445 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 511]", 2024-04-07T17:51:25.445 INFO:tasks.workunit.client.0.smithi100.stdout: "[512, 1023]", 2024-04-07T17:51:25.445 INFO:tasks.workunit.client.0.smithi100.stdout: "[1024, 2047]", 2024-04-07T17:51:25.445 INFO:tasks.workunit.client.0.smithi100.stdout: "[2048, 4095]", 2024-04-07T17:51:25.445 INFO:tasks.workunit.client.0.smithi100.stdout: "[4096, 8191]", 2024-04-07T17:51:25.445 INFO:tasks.workunit.client.0.smithi100.stdout: "[8192, 16383]", 2024-04-07T17:51:25.445 INFO:tasks.workunit.client.0.smithi100.stdout: "[16384, 32767]", 2024-04-07T17:51:25.445 INFO:tasks.workunit.client.0.smithi100.stdout: "[32768, 65535]", 2024-04-07T17:51:25.445 INFO:tasks.workunit.client.0.smithi100.stdout: "[65536, 131071]", 2024-04-07T17:51:25.445 INFO:tasks.workunit.client.0.smithi100.stdout: "[131072, 262143]", 2024-04-07T17:51:25.445 INFO:tasks.workunit.client.0.smithi100.stdout: "[262144, 524287]", 2024-04-07T17:51:25.446 INFO:tasks.workunit.client.0.smithi100.stdout: "[524288, 1048575]", 2024-04-07T17:51:25.446 INFO:tasks.workunit.client.0.smithi100.stdout: "[1048576, 2097151]", 2024-04-07T17:51:25.446 INFO:tasks.workunit.client.0.smithi100.stdout: "[2097152, 4194303]", 2024-04-07T17:51:25.446 INFO:tasks.workunit.client.0.smithi100.stdout: "[4194304, 8388607]", 2024-04-07T17:51:25.446 INFO:tasks.workunit.client.0.smithi100.stdout: "[8388608, 16777215]", 2024-04-07T17:51:25.446 INFO:tasks.workunit.client.0.smithi100.stdout: "[16777216, 33554431]", 2024-04-07T17:51:25.446 INFO:tasks.workunit.client.0.smithi100.stdout: "[33554432, 67108863]", 2024-04-07T17:51:25.446 INFO:tasks.workunit.client.0.smithi100.stdout: "[67108864, 134217727]", 2024-04-07T17:51:25.446 INFO:tasks.workunit.client.0.smithi100.stdout: "[134217728, 268435455]", 2024-04-07T17:51:25.446 INFO:tasks.workunit.client.0.smithi100.stdout: "[268435456, 536870911]", 2024-04-07T17:51:25.446 INFO:tasks.workunit.client.0.smithi100.stdout: "[536870912, 1073741823]", 2024-04-07T17:51:25.446 INFO:tasks.workunit.client.0.smithi100.stdout: "[1073741824, 2147483647]", 2024-04-07T17:51:25.446 INFO:tasks.workunit.client.0.smithi100.stdout: "[2147483648, 4294967295]", 2024-04-07T17:51:25.446 INFO:tasks.workunit.client.0.smithi100.stdout: "[4294967296, 8589934591]", 2024-04-07T17:51:25.447 INFO:tasks.workunit.client.0.smithi100.stdout: "[8589934592, 17179869183]", 2024-04-07T17:51:25.447 INFO:tasks.workunit.client.0.smithi100.stdout: "[17179869184, 34359738367]", 2024-04-07T17:51:25.447 INFO:tasks.workunit.client.0.smithi100.stdout: "[34359738368, 68719476735]", 2024-04-07T17:51:25.447 INFO:tasks.workunit.client.0.smithi100.stdout: "[68719476736, 137438953471]", 2024-04-07T17:51:25.447 INFO:tasks.workunit.client.0.smithi100.stdout: "[137438953472, 274877906943]", 2024-04-07T17:51:25.447 INFO:tasks.workunit.client.0.smithi100.stdout: "[274877906944, None]" 2024-04-07T17:51:25.447 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:25.447 INFO:tasks.workunit.client.0.smithi100.stdout: "scale_type": "log2" 2024-04-07T17:51:25.447 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.447 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:25.447 INFO:tasks.workunit.client.0.smithi100.stdout: "osds": [ 2024-04-07T17:51:25.447 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:25.447 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 0, 2024-04-07T17:51:25.447 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:25.447 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.447 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.448 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 997, 13, 0, 0, 648, 16, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.448 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 7212, 3165, 0, 1, 4231, 80, 0, 4, 1, 0, 3, 2, 4, 0, 12, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.448 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 11864, 1050, 12, 2, 2904, 503, 11, 11, 26, 39, 32, 36, 59, 21, 77, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.448 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1610, 30, 63, 0, 19, 15, 2, 5, 5, 13, 105, 163, 32, 5, 39, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.448 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 417, 0, 5, 0, 3, 9, 0, 1, 0, 1, 3, 37, 361, 22, 36, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.448 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 141, 1, 0, 0, 5, 11, 0, 0, 0, 1, 2, 13, 25, 31, 205, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.448 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 120, 1, 0, 0, 0, 0, 1, 2, 2, 0, 0, 7, 21, 15, 56, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.448 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 192, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 4, 14, 26, 33, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.448 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 32, 0, 0, 0, 0, 0, 0, 0, 0, 0, 2, 0, 4, 19, 20, 5, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.448 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 6, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 4, 3, 9, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.448 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.448 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.448 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.448 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.449 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.449 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.449 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.449 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.449 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.449 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.449 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.449 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.449 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.449 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.449 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.449 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.449 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.449 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.449 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.449 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:25.450 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:25.450 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.450 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:25.450 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 1, 2024-04-07T17:51:25.450 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:25.450 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.450 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.450 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1446, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.450 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 5788, 4, 145, 0, 8, 0, 0, 0, 1, 1, 2, 1, 2, 0, 9, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.450 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 9601, 33, 88, 8, 28, 51, 80, 25, 39, 53, 38, 35, 64, 23, 44, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.450 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1252, 11, 1178, 4, 3, 3, 38, 118, 6, 17, 129, 228, 29, 7, 16, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.450 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 284, 1, 144, 24, 0, 0, 0, 14, 2, 1, 4, 45, 487, 18, 5, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.450 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 188, 0, 9, 6, 0, 0, 0, 0, 0, 1, 8, 2, 4, 12, 161, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.450 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 174, 0, 76, 0, 0, 0, 0, 0, 0, 0, 9, 0, 4, 5, 36, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.451 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 95, 0, 62, 0, 0, 0, 0, 0, 0, 0, 3, 0, 3, 11, 17, 6, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.451 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 7, 0, 153, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 5, 2, 10, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.451 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 3, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.451 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.451 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.451 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.451 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.451 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.451 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.451 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.451 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.451 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.451 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.451 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.451 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.451 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.452 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.452 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.452 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.452 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.452 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.452 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.452 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:25.452 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:25.452 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.452 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:25.452 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 2, 2024-04-07T17:51:25.452 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:25.452 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.452 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.452 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 2187, 6, 0, 0, 0, 0, 0, 0, 0, 0, 2, 0, 0, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.452 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 6047, 2629, 2809, 0, 8, 0, 0, 1, 2, 0, 23, 0, 2, 7, 5, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.453 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 9066, 1269, 1692, 0, 3, 1, 0, 2, 6, 6, 32, 2, 20, 30, 68, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.453 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1309, 32, 27, 1, 2, 0, 0, 1, 9, 9, 29, 7, 17, 16, 33, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.453 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 335, 7, 6, 1, 0, 0, 0, 0, 0, 0, 3, 7, 47, 45, 17, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.453 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 152, 2, 0, 0, 0, 0, 0, 3, 0, 1, 3, 25, 11, 60, 181, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.453 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 149, 1, 0, 0, 0, 0, 0, 1, 3, 0, 3, 18, 37, 45, 52, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.453 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 208, 0, 0, 1, 0, 1, 2, 0, 0, 0, 4, 20, 43, 66, 44, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.453 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 52, 0, 0, 0, 0, 0, 0, 3, 0, 0, 0, 2, 3, 18, 51, 7, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.453 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 10, 0, 0, 0, 1, 0, 0, 1, 0, 0, 0, 0, 0, 8, 3, 4, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.453 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.453 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.453 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.453 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.453 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.453 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.453 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.453 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.454 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.454 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.454 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.454 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.454 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.454 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.454 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.454 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.454 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.454 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.454 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.454 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:25.454 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:25.454 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.455 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:25.455 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 3, 2024-04-07T17:51:25.455 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:25.455 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.455 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.455 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 3687, 36, 0, 1, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.455 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 12530, 2576, 0, 0, 13, 0, 0, 0, 0, 0, 1, 1, 1, 0, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.455 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 13127, 1790, 0, 0, 8, 6, 2, 8, 12, 16, 12, 7, 12, 5, 27, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.455 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1412, 42, 24, 0, 26, 0, 0, 2, 3, 5, 41, 74, 13, 6, 10, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.455 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 416, 3, 13, 0, 157, 0, 0, 0, 0, 0, 2, 12, 225, 42, 4, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.455 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 125, 1, 0, 0, 306, 1, 0, 0, 0, 0, 0, 0, 6, 21, 84, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.455 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 27, 0, 2, 2, 334, 0, 0, 0, 0, 0, 0, 2, 6, 7, 22, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.455 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 10, 0, 0, 0, 1487, 0, 0, 0, 0, 0, 0, 0, 4, 9, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.455 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 8, 0, 0, 0, 374, 0, 0, 0, 0, 0, 0, 0, 0, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.455 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.455 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.456 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.456 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.456 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.456 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.456 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.456 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.456 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.456 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.456 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.456 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.456 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.456 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.456 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.456 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.457 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.457 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.457 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.457 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.457 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:25.457 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:25.457 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.457 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:25.457 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 4, 2024-04-07T17:51:25.457 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:25.457 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.457 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.457 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 609, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.457 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 3885, 8, 15, 0, 0, 0, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.457 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 10736, 14, 53, 5, 22, 32, 30, 14, 13, 15, 11, 13, 30, 4, 36, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.458 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1540, 27, 25, 0, 9, 3, 41, 70, 1, 5, 58, 81, 10, 3, 28, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.458 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 375, 10, 14, 0, 0, 0, 0, 0, 0, 0, 1, 16, 198, 13, 3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.458 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 187, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 3, 9, 9, 56, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.458 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 61, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 5, 13, 3, 46, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.458 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 34, 0, 0, 0, 0, 0, 0, 0, 0, 2, 2, 3, 6, 3, 18, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.458 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 8, 0, 0, 0, 0, 0, 0, 1, 0, 0, 1, 0, 0, 0, 2, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.458 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.458 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.458 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.458 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.458 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.458 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.458 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.458 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.458 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.459 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.459 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.459 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.459 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.459 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.459 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.459 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.459 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.459 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.459 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.465 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.465 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:25.466 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:25.466 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.466 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:25.466 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 5, 2024-04-07T17:51:25.466 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:25.466 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.466 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.466 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 946, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.466 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 5643, 1, 0, 1, 0, 0, 1, 0, 0, 1, 3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.466 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 14942, 20, 3, 0, 21, 3, 0, 4, 1, 9, 44, 5, 10, 29, 54, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.466 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 2337, 103, 114, 1, 1, 1, 0, 2, 3, 4, 28, 12, 3, 5, 47, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.466 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 783, 40, 72, 4, 6, 0, 0, 0, 0, 0, 2, 0, 27, 26, 14, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.466 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 237, 9, 6, 4, 0, 0, 0, 2, 1, 0, 0, 3, 4, 39, 200, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.466 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 158, 10, 24, 5, 1, 0, 0, 1, 0, 0, 0, 2, 1, 6, 66, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.467 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 156, 1, 10, 45, 24, 0, 0, 0, 0, 0, 0, 0, 4, 6, 14, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.467 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 10, 0, 0, 16, 104, 86, 0, 0, 0, 0, 0, 0, 3, 6, 14, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.467 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 3, 0, 0, 0, 6, 43, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.467 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.467 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.467 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.467 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.467 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.467 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.467 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.467 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.467 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.467 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.467 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.468 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.468 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.468 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.468 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.468 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.468 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.468 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.468 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.468 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:25.468 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:25.468 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.468 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:25.468 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 6, 2024-04-07T17:51:25.468 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:25.468 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.469 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.469 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 830, 5, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.469 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 6114, 1983, 0, 0, 2, 1, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.469 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 10059, 2215, 1, 1, 22, 3, 1, 0, 18, 18, 10, 10, 26, 17, 43, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.469 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1593, 28, 21, 0, 2, 1, 0, 0, 5, 13, 58, 83, 12, 5, 27, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.469 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 625, 3, 7, 0, 0, 0, 0, 0, 0, 1, 7, 26, 211, 19, 7, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.469 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 531, 11, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 6, 22, 90, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.469 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 196, 8, 0, 0, 0, 1, 0, 0, 0, 1, 0, 0, 2, 1, 53, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.469 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 155, 8, 0, 0, 0, 0, 0, 1, 0, 3, 0, 0, 0, 1, 6, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.469 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 8, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.469 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.469 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.469 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.469 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.469 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.469 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.470 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.470 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.470 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.470 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.470 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.470 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.470 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.470 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.470 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.470 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.470 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.470 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.470 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.471 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.471 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:25.471 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:25.471 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.471 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:25.471 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 7, 2024-04-07T17:51:25.471 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:25.471 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.471 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.471 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 385, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.471 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 6673, 2090, 1, 0, 0, 0, 0, 0, 0, 0, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.471 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 13766, 2103, 0, 0, 7, 2, 0, 1, 0, 1, 1, 13, 10, 17, 55, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.471 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1294, 17, 4, 1, 0, 0, 0, 1, 0, 2, 17, 6, 3, 7, 31, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.471 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 308, 17, 0, 0, 0, 0, 0, 0, 0, 0, 0, 4, 20, 40, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.471 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 152, 63, 8, 0, 0, 0, 0, 2, 0, 0, 3, 1, 6, 32, 113, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.472 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 98, 138, 5, 0, 0, 0, 0, 1, 0, 0, 0, 2, 2, 7, 52, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.472 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 223, 58, 1, 0, 0, 0, 0, 0, 0, 0, 3, 0, 0, 8, 11, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.472 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 49, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 6, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.472 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.472 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.472 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.472 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.472 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.472 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.472 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.472 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.472 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.472 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.472 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.472 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.473 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.473 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.473 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.473 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.473 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.473 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.473 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.473 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.473 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:25.473 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:25.473 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.473 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:25.473 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.473 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_latency_in_bytes_histogram": { 2024-04-07T17:51:25.473 INFO:tasks.workunit.client.0.smithi100.stdout: "axes": [ 2024-04-07T17:51:25.473 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:25.474 INFO:tasks.workunit.client.0.smithi100.stdout: "buckets": 32, 2024-04-07T17:51:25.474 INFO:tasks.workunit.client.0.smithi100.stdout: "min": 0, 2024-04-07T17:51:25.474 INFO:tasks.workunit.client.0.smithi100.stdout: "name": "Latency (usec)", 2024-04-07T17:51:25.474 INFO:tasks.workunit.client.0.smithi100.stdout: "quant_size": 100000, 2024-04-07T17:51:25.474 INFO:tasks.workunit.client.0.smithi100.stdout: "ranges": [ 2024-04-07T17:51:25.474 INFO:tasks.workunit.client.0.smithi100.stdout: "[None, -1]", 2024-04-07T17:51:25.474 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 99999]", 2024-04-07T17:51:25.474 INFO:tasks.workunit.client.0.smithi100.stdout: "[100000, 199999]", 2024-04-07T17:51:25.474 INFO:tasks.workunit.client.0.smithi100.stdout: "[200000, 399999]", 2024-04-07T17:51:25.474 INFO:tasks.workunit.client.0.smithi100.stdout: "[400000, 799999]", 2024-04-07T17:51:25.474 INFO:tasks.workunit.client.0.smithi100.stdout: "[800000, 1599999]", 2024-04-07T17:51:25.474 INFO:tasks.workunit.client.0.smithi100.stdout: "[1600000, 3199999]", 2024-04-07T17:51:25.474 INFO:tasks.workunit.client.0.smithi100.stdout: "[3200000, 6399999]", 2024-04-07T17:51:25.474 INFO:tasks.workunit.client.0.smithi100.stdout: "[6400000, 12799999]", 2024-04-07T17:51:25.475 INFO:tasks.workunit.client.0.smithi100.stdout: "[12800000, 25599999]", 2024-04-07T17:51:25.475 INFO:tasks.workunit.client.0.smithi100.stdout: "[25600000, 51199999]", 2024-04-07T17:51:25.475 INFO:tasks.workunit.client.0.smithi100.stdout: "[51200000, 102399999]", 2024-04-07T17:51:25.475 INFO:tasks.workunit.client.0.smithi100.stdout: "[102400000, 204799999]", 2024-04-07T17:51:25.475 INFO:tasks.workunit.client.0.smithi100.stdout: "[204800000, 409599999]", 2024-04-07T17:51:25.475 INFO:tasks.workunit.client.0.smithi100.stdout: "[409600000, 819199999]", 2024-04-07T17:51:25.475 INFO:tasks.workunit.client.0.smithi100.stdout: "[819200000, 1638399999]", 2024-04-07T17:51:25.475 INFO:tasks.workunit.client.0.smithi100.stdout: "[1638400000, 3276799999]", 2024-04-07T17:51:25.475 INFO:tasks.workunit.client.0.smithi100.stdout: "[3276800000, 6553599999]", 2024-04-07T17:51:25.475 INFO:tasks.workunit.client.0.smithi100.stdout: "[6553600000, 13107199999]", 2024-04-07T17:51:25.475 INFO:tasks.workunit.client.0.smithi100.stdout: "[13107200000, 26214399999]", 2024-04-07T17:51:25.475 INFO:tasks.workunit.client.0.smithi100.stdout: "[26214400000, 52428799999]", 2024-04-07T17:51:25.475 INFO:tasks.workunit.client.0.smithi100.stdout: "[52428800000, 104857599999]", 2024-04-07T17:51:25.475 INFO:tasks.workunit.client.0.smithi100.stdout: "[104857600000, 209715199999]", 2024-04-07T17:51:25.475 INFO:tasks.workunit.client.0.smithi100.stdout: "[209715200000, 419430399999]", 2024-04-07T17:51:25.475 INFO:tasks.workunit.client.0.smithi100.stdout: "[419430400000, 838860799999]", 2024-04-07T17:51:25.476 INFO:tasks.workunit.client.0.smithi100.stdout: "[838860800000, 1677721599999]", 2024-04-07T17:51:25.476 INFO:tasks.workunit.client.0.smithi100.stdout: "[1677721600000, 3355443199999]", 2024-04-07T17:51:25.476 INFO:tasks.workunit.client.0.smithi100.stdout: "[3355443200000, 6710886399999]", 2024-04-07T17:51:25.476 INFO:tasks.workunit.client.0.smithi100.stdout: "[6710886400000, 13421772799999]", 2024-04-07T17:51:25.476 INFO:tasks.workunit.client.0.smithi100.stdout: "[13421772800000, 26843545599999]", 2024-04-07T17:51:25.476 INFO:tasks.workunit.client.0.smithi100.stdout: "[26843545600000, 53687091199999]", 2024-04-07T17:51:25.476 INFO:tasks.workunit.client.0.smithi100.stdout: "[53687091200000, None]" 2024-04-07T17:51:25.476 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:25.476 INFO:tasks.workunit.client.0.smithi100.stdout: "scale_type": "log2" 2024-04-07T17:51:25.476 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.476 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:25.476 INFO:tasks.workunit.client.0.smithi100.stdout: "buckets": 32, 2024-04-07T17:51:25.476 INFO:tasks.workunit.client.0.smithi100.stdout: "min": 0, 2024-04-07T17:51:25.476 INFO:tasks.workunit.client.0.smithi100.stdout: "name": "Request size (bytes)", 2024-04-07T17:51:25.477 INFO:tasks.workunit.client.0.smithi100.stdout: "quant_size": 512, 2024-04-07T17:51:25.477 INFO:tasks.workunit.client.0.smithi100.stdout: "ranges": [ 2024-04-07T17:51:25.477 INFO:tasks.workunit.client.0.smithi100.stdout: "[None, -1]", 2024-04-07T17:51:25.477 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 511]", 2024-04-07T17:51:25.477 INFO:tasks.workunit.client.0.smithi100.stdout: "[512, 1023]", 2024-04-07T17:51:25.477 INFO:tasks.workunit.client.0.smithi100.stdout: "[1024, 2047]", 2024-04-07T17:51:25.477 INFO:tasks.workunit.client.0.smithi100.stdout: "[2048, 4095]", 2024-04-07T17:51:25.477 INFO:tasks.workunit.client.0.smithi100.stdout: "[4096, 8191]", 2024-04-07T17:51:25.477 INFO:tasks.workunit.client.0.smithi100.stdout: "[8192, 16383]", 2024-04-07T17:51:25.477 INFO:tasks.workunit.client.0.smithi100.stdout: "[16384, 32767]", 2024-04-07T17:51:25.477 INFO:tasks.workunit.client.0.smithi100.stdout: "[32768, 65535]", 2024-04-07T17:51:25.477 INFO:tasks.workunit.client.0.smithi100.stdout: "[65536, 131071]", 2024-04-07T17:51:25.477 INFO:tasks.workunit.client.0.smithi100.stdout: "[131072, 262143]", 2024-04-07T17:51:25.477 INFO:tasks.workunit.client.0.smithi100.stdout: "[262144, 524287]", 2024-04-07T17:51:25.478 INFO:tasks.workunit.client.0.smithi100.stdout: "[524288, 1048575]", 2024-04-07T17:51:25.478 INFO:tasks.workunit.client.0.smithi100.stdout: "[1048576, 2097151]", 2024-04-07T17:51:25.478 INFO:tasks.workunit.client.0.smithi100.stdout: "[2097152, 4194303]", 2024-04-07T17:51:25.478 INFO:tasks.workunit.client.0.smithi100.stdout: "[4194304, 8388607]", 2024-04-07T17:51:25.478 INFO:tasks.workunit.client.0.smithi100.stdout: "[8388608, 16777215]", 2024-04-07T17:51:25.478 INFO:tasks.workunit.client.0.smithi100.stdout: "[16777216, 33554431]", 2024-04-07T17:51:25.478 INFO:tasks.workunit.client.0.smithi100.stdout: "[33554432, 67108863]", 2024-04-07T17:51:25.478 INFO:tasks.workunit.client.0.smithi100.stdout: "[67108864, 134217727]", 2024-04-07T17:51:25.478 INFO:tasks.workunit.client.0.smithi100.stdout: "[134217728, 268435455]", 2024-04-07T17:51:25.478 INFO:tasks.workunit.client.0.smithi100.stdout: "[268435456, 536870911]", 2024-04-07T17:51:25.478 INFO:tasks.workunit.client.0.smithi100.stdout: "[536870912, 1073741823]", 2024-04-07T17:51:25.478 INFO:tasks.workunit.client.0.smithi100.stdout: "[1073741824, 2147483647]", 2024-04-07T17:51:25.478 INFO:tasks.workunit.client.0.smithi100.stdout: "[2147483648, 4294967295]", 2024-04-07T17:51:25.478 INFO:tasks.workunit.client.0.smithi100.stdout: "[4294967296, 8589934591]", 2024-04-07T17:51:25.478 INFO:tasks.workunit.client.0.smithi100.stdout: "[8589934592, 17179869183]", 2024-04-07T17:51:25.479 INFO:tasks.workunit.client.0.smithi100.stdout: "[17179869184, 34359738367]", 2024-04-07T17:51:25.479 INFO:tasks.workunit.client.0.smithi100.stdout: "[34359738368, 68719476735]", 2024-04-07T17:51:25.479 INFO:tasks.workunit.client.0.smithi100.stdout: "[68719476736, 137438953471]", 2024-04-07T17:51:25.479 INFO:tasks.workunit.client.0.smithi100.stdout: "[137438953472, 274877906943]", 2024-04-07T17:51:25.479 INFO:tasks.workunit.client.0.smithi100.stdout: "[274877906944, None]" 2024-04-07T17:51:25.479 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:25.479 INFO:tasks.workunit.client.0.smithi100.stdout: "scale_type": "log2" 2024-04-07T17:51:25.479 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.479 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:25.479 INFO:tasks.workunit.client.0.smithi100.stdout: "osds": [ 2024-04-07T17:51:25.479 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:25.479 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 0, 2024-04-07T17:51:25.479 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:25.479 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.479 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.480 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.480 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.480 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.480 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 416, 12, 5, 6, 1, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.480 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 4275, 54, 74, 123, 377, 516, 419, 4, 11, 5, 0, 0, 3, 3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.480 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 775, 16, 26, 72, 112, 96, 307, 0, 3, 12, 2, 3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.480 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 369, 1, 0, 4, 2, 1, 2, 0, 1, 1, 0, 3, 13, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.480 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 201, 1, 0, 0, 0, 0, 1, 0, 0, 1, 0, 1, 10, 14, 29, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.480 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 88, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 1, 1, 7, 41, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.480 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 6, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 25, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.480 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.480 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.480 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.480 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.480 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.480 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.481 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.481 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.481 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.481 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.481 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.481 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.481 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.481 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.481 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.481 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.481 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.481 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.481 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.481 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:25.481 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:25.481 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.482 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:25.482 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 1, 2024-04-07T17:51:25.482 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:25.482 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.482 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.482 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.482 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.482 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.482 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 463, 210, 9, 10, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.482 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 2623, 76, 25, 59, 122, 1, 2, 2, 0, 0, 0, 1, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.482 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 819, 8, 815, 0, 1, 0, 1, 0, 2, 1, 0, 1, 0, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.482 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 994, 507, 544, 0, 0, 0, 0, 0, 0, 0, 1, 1, 5, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.482 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 844, 647, 73, 0, 0, 0, 0, 1, 0, 0, 1, 0, 5, 7, 15, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.482 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1618, 1450, 103, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 4, 34, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.483 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 291, 394, 108, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 12, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.483 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.483 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.483 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.483 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.483 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.483 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.483 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.483 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.483 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.483 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.484 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.484 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.484 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.484 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.484 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.484 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.484 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.484 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.484 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.484 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:25.484 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:25.484 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.484 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:25.484 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 2, 2024-04-07T17:51:25.484 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:25.485 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.485 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.485 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.485 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.485 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.485 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 248, 5, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.485 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 7869, 12, 0, 0, 0, 2, 5, 3, 9, 2, 0, 2, 3, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.485 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 17697, 0, 1, 0, 1, 0, 0, 1, 2, 11, 0, 2, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.485 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 887, 0, 1, 0, 0, 0, 0, 0, 1, 0, 0, 2, 9, 3, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.485 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 267, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 7, 21, 28, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.485 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 103, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 6, 6, 37, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.485 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 2, 23, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.485 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.485 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.485 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.486 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.486 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.486 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.486 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.486 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.486 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.486 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.486 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.486 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.486 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.486 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.486 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.486 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.486 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.487 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.487 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.487 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:25.487 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:25.487 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.487 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:25.487 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 3, 2024-04-07T17:51:25.487 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:25.487 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.487 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.487 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.487 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.487 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.487 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 280, 1, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.487 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 3748, 33, 1, 0, 5, 1, 5, 4, 1, 0, 0, 0, 3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.487 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 519, 29, 31, 40, 1, 0, 1, 0, 0, 0, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.488 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 290, 227, 3, 37, 0, 0, 0, 0, 0, 0, 0, 2, 3, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.488 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 2130, 1325, 1, 172, 0, 0, 0, 0, 0, 0, 0, 0, 6, 6, 16, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.488 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 2779, 1327, 0, 285, 0, 0, 0, 0, 0, 0, 0, 1, 3, 4, 24, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.488 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 233, 115, 0, 51, 0, 0, 0, 0, 0, 0, 0, 0, 0, 2, 15, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.488 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.488 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.488 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.488 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.488 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.488 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.488 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.488 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.488 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.488 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.489 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.489 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.489 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.489 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.489 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.489 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.489 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.489 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.489 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.489 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:25.489 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:25.489 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.489 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:25.489 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 4, 2024-04-07T17:51:25.489 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:25.489 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.489 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.490 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.490 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.490 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.490 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 113, 11, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.490 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 3576, 127, 33, 67, 137, 233, 86, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.490 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 303, 84, 0, 0, 1, 40, 395, 0, 1, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.490 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 124, 2, 0, 0, 1, 2, 1, 0, 0, 0, 1, 2, 7, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.490 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 73, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 4, 4, 6, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.490 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 25, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 29, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.490 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 16, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.490 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.490 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.490 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.490 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.491 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.491 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.491 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.491 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.491 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.491 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.491 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.491 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.491 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.491 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.491 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.491 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.491 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.491 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.491 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.491 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:25.491 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:25.492 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.492 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:25.492 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 5, 2024-04-07T17:51:25.492 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:25.492 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.492 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.492 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.492 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.492 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.492 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 109, 21, 86, 200, 454, 576, 651, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.492 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 5257, 50, 38, 54, 17, 36, 74, 1, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.492 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 689, 12, 17, 5, 0, 1, 0, 1, 0, 1, 2, 3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.492 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 457, 32, 66, 127, 29, 1, 0, 2, 0, 0, 2, 4, 8, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.492 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 302, 8, 16, 1, 0, 0, 0, 1, 0, 2, 2, 1, 9, 18, 14, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.492 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 96, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 4, 8, 35, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.492 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 18, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.493 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.493 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.493 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.493 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.493 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.493 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.493 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.493 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.493 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.493 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.493 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.493 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.493 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.493 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.493 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.494 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.494 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.494 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.494 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.494 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:25.494 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:25.494 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.494 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:25.494 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 6, 2024-04-07T17:51:25.494 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:25.494 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.494 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.494 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.494 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.495 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.495 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 51, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.495 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 2959, 10, 0, 0, 1, 1, 0, 1, 0, 0, 0, 1, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.495 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 347, 9, 0, 0, 1, 1, 0, 0, 0, 1, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.495 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 267, 64, 0, 0, 0, 0, 1, 0, 1, 2, 0, 3, 7, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.495 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 150, 40, 0, 0, 0, 0, 0, 0, 0, 0, 1, 2, 3, 4, 6, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.495 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 58, 5, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 6, 27, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.495 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 6, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 2, 20, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.495 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.495 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.495 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.495 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.495 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.495 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.495 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.495 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.496 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.496 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.496 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.496 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.496 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.496 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.496 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.496 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.496 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.496 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.496 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.496 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:25.496 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:25.496 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.496 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:25.497 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 7, 2024-04-07T17:51:25.497 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:25.497 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.497 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.497 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.497 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.497 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.497 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 267, 14, 33, 41, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.497 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 4198, 22, 41, 115, 0, 1, 1, 1, 2, 0, 0, 0, 2, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.497 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 6038, 25, 32, 5, 0, 1, 0, 1, 2, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.497 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 292, 5, 2, 0, 0, 0, 0, 0, 0, 1, 0, 2, 4, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.497 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 85, 1, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 2, 13, 6, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.497 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 43, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 2, 5, 28, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.497 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 16, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.498 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.498 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.498 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.498 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.498 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.498 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.498 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.498 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.498 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.498 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.498 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.498 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.498 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.498 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.499 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.499 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.499 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.499 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.499 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.499 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:25.499 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:25.499 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.499 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:25.499 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.499 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_latency_out_bytes_histogram": { 2024-04-07T17:51:25.499 INFO:tasks.workunit.client.0.smithi100.stdout: "axes": [ 2024-04-07T17:51:25.499 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:25.499 INFO:tasks.workunit.client.0.smithi100.stdout: "buckets": 32, 2024-04-07T17:51:25.499 INFO:tasks.workunit.client.0.smithi100.stdout: "min": 0, 2024-04-07T17:51:25.499 INFO:tasks.workunit.client.0.smithi100.stdout: "name": "Latency (usec)", 2024-04-07T17:51:25.500 INFO:tasks.workunit.client.0.smithi100.stdout: "quant_size": 100000, 2024-04-07T17:51:25.500 INFO:tasks.workunit.client.0.smithi100.stdout: "ranges": [ 2024-04-07T17:51:25.500 INFO:tasks.workunit.client.0.smithi100.stdout: "[None, -1]", 2024-04-07T17:51:25.500 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 99999]", 2024-04-07T17:51:25.500 INFO:tasks.workunit.client.0.smithi100.stdout: "[100000, 199999]", 2024-04-07T17:51:25.500 INFO:tasks.workunit.client.0.smithi100.stdout: "[200000, 399999]", 2024-04-07T17:51:25.500 INFO:tasks.workunit.client.0.smithi100.stdout: "[400000, 799999]", 2024-04-07T17:51:25.500 INFO:tasks.workunit.client.0.smithi100.stdout: "[800000, 1599999]", 2024-04-07T17:51:25.500 INFO:tasks.workunit.client.0.smithi100.stdout: "[1600000, 3199999]", 2024-04-07T17:51:25.500 INFO:tasks.workunit.client.0.smithi100.stdout: "[3200000, 6399999]", 2024-04-07T17:51:25.500 INFO:tasks.workunit.client.0.smithi100.stdout: "[6400000, 12799999]", 2024-04-07T17:51:25.500 INFO:tasks.workunit.client.0.smithi100.stdout: "[12800000, 25599999]", 2024-04-07T17:51:25.500 INFO:tasks.workunit.client.0.smithi100.stdout: "[25600000, 51199999]", 2024-04-07T17:51:25.500 INFO:tasks.workunit.client.0.smithi100.stdout: "[51200000, 102399999]", 2024-04-07T17:51:25.500 INFO:tasks.workunit.client.0.smithi100.stdout: "[102400000, 204799999]", 2024-04-07T17:51:25.500 INFO:tasks.workunit.client.0.smithi100.stdout: "[204800000, 409599999]", 2024-04-07T17:51:25.501 INFO:tasks.workunit.client.0.smithi100.stdout: "[409600000, 819199999]", 2024-04-07T17:51:25.501 INFO:tasks.workunit.client.0.smithi100.stdout: "[819200000, 1638399999]", 2024-04-07T17:51:25.501 INFO:tasks.workunit.client.0.smithi100.stdout: "[1638400000, 3276799999]", 2024-04-07T17:51:25.501 INFO:tasks.workunit.client.0.smithi100.stdout: "[3276800000, 6553599999]", 2024-04-07T17:51:25.501 INFO:tasks.workunit.client.0.smithi100.stdout: "[6553600000, 13107199999]", 2024-04-07T17:51:25.501 INFO:tasks.workunit.client.0.smithi100.stdout: "[13107200000, 26214399999]", 2024-04-07T17:51:25.501 INFO:tasks.workunit.client.0.smithi100.stdout: "[26214400000, 52428799999]", 2024-04-07T17:51:25.501 INFO:tasks.workunit.client.0.smithi100.stdout: "[52428800000, 104857599999]", 2024-04-07T17:51:25.501 INFO:tasks.workunit.client.0.smithi100.stdout: "[104857600000, 209715199999]", 2024-04-07T17:51:25.501 INFO:tasks.workunit.client.0.smithi100.stdout: "[209715200000, 419430399999]", 2024-04-07T17:51:25.501 INFO:tasks.workunit.client.0.smithi100.stdout: "[419430400000, 838860799999]", 2024-04-07T17:51:25.501 INFO:tasks.workunit.client.0.smithi100.stdout: "[838860800000, 1677721599999]", 2024-04-07T17:51:25.501 INFO:tasks.workunit.client.0.smithi100.stdout: "[1677721600000, 3355443199999]", 2024-04-07T17:51:25.501 INFO:tasks.workunit.client.0.smithi100.stdout: "[3355443200000, 6710886399999]", 2024-04-07T17:51:25.501 INFO:tasks.workunit.client.0.smithi100.stdout: "[6710886400000, 13421772799999]", 2024-04-07T17:51:25.501 INFO:tasks.workunit.client.0.smithi100.stdout: "[13421772800000, 26843545599999]", 2024-04-07T17:51:25.502 INFO:tasks.workunit.client.0.smithi100.stdout: "[26843545600000, 53687091199999]", 2024-04-07T17:51:25.502 INFO:tasks.workunit.client.0.smithi100.stdout: "[53687091200000, None]" 2024-04-07T17:51:25.502 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:25.502 INFO:tasks.workunit.client.0.smithi100.stdout: "scale_type": "log2" 2024-04-07T17:51:25.502 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.502 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:25.502 INFO:tasks.workunit.client.0.smithi100.stdout: "buckets": 32, 2024-04-07T17:51:25.502 INFO:tasks.workunit.client.0.smithi100.stdout: "min": 0, 2024-04-07T17:51:25.502 INFO:tasks.workunit.client.0.smithi100.stdout: "name": "Request size (bytes)", 2024-04-07T17:51:25.502 INFO:tasks.workunit.client.0.smithi100.stdout: "quant_size": 512, 2024-04-07T17:51:25.502 INFO:tasks.workunit.client.0.smithi100.stdout: "ranges": [ 2024-04-07T17:51:25.502 INFO:tasks.workunit.client.0.smithi100.stdout: "[None, -1]", 2024-04-07T17:51:25.502 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 511]", 2024-04-07T17:51:25.502 INFO:tasks.workunit.client.0.smithi100.stdout: "[512, 1023]", 2024-04-07T17:51:25.502 INFO:tasks.workunit.client.0.smithi100.stdout: "[1024, 2047]", 2024-04-07T17:51:25.503 INFO:tasks.workunit.client.0.smithi100.stdout: "[2048, 4095]", 2024-04-07T17:51:25.503 INFO:tasks.workunit.client.0.smithi100.stdout: "[4096, 8191]", 2024-04-07T17:51:25.503 INFO:tasks.workunit.client.0.smithi100.stdout: "[8192, 16383]", 2024-04-07T17:51:25.503 INFO:tasks.workunit.client.0.smithi100.stdout: "[16384, 32767]", 2024-04-07T17:51:25.503 INFO:tasks.workunit.client.0.smithi100.stdout: "[32768, 65535]", 2024-04-07T17:51:25.503 INFO:tasks.workunit.client.0.smithi100.stdout: "[65536, 131071]", 2024-04-07T17:51:25.503 INFO:tasks.workunit.client.0.smithi100.stdout: "[131072, 262143]", 2024-04-07T17:51:25.503 INFO:tasks.workunit.client.0.smithi100.stdout: "[262144, 524287]", 2024-04-07T17:51:25.503 INFO:tasks.workunit.client.0.smithi100.stdout: "[524288, 1048575]", 2024-04-07T17:51:25.503 INFO:tasks.workunit.client.0.smithi100.stdout: "[1048576, 2097151]", 2024-04-07T17:51:25.503 INFO:tasks.workunit.client.0.smithi100.stdout: "[2097152, 4194303]", 2024-04-07T17:51:25.503 INFO:tasks.workunit.client.0.smithi100.stdout: "[4194304, 8388607]", 2024-04-07T17:51:25.503 INFO:tasks.workunit.client.0.smithi100.stdout: "[8388608, 16777215]", 2024-04-07T17:51:25.503 INFO:tasks.workunit.client.0.smithi100.stdout: "[16777216, 33554431]", 2024-04-07T17:51:25.503 INFO:tasks.workunit.client.0.smithi100.stdout: "[33554432, 67108863]", 2024-04-07T17:51:25.503 INFO:tasks.workunit.client.0.smithi100.stdout: "[67108864, 134217727]", 2024-04-07T17:51:25.504 INFO:tasks.workunit.client.0.smithi100.stdout: "[134217728, 268435455]", 2024-04-07T17:51:25.504 INFO:tasks.workunit.client.0.smithi100.stdout: "[268435456, 536870911]", 2024-04-07T17:51:25.504 INFO:tasks.workunit.client.0.smithi100.stdout: "[536870912, 1073741823]", 2024-04-07T17:51:25.504 INFO:tasks.workunit.client.0.smithi100.stdout: "[1073741824, 2147483647]", 2024-04-07T17:51:25.504 INFO:tasks.workunit.client.0.smithi100.stdout: "[2147483648, 4294967295]", 2024-04-07T17:51:25.504 INFO:tasks.workunit.client.0.smithi100.stdout: "[4294967296, 8589934591]", 2024-04-07T17:51:25.504 INFO:tasks.workunit.client.0.smithi100.stdout: "[8589934592, 17179869183]", 2024-04-07T17:51:25.504 INFO:tasks.workunit.client.0.smithi100.stdout: "[17179869184, 34359738367]", 2024-04-07T17:51:25.504 INFO:tasks.workunit.client.0.smithi100.stdout: "[34359738368, 68719476735]", 2024-04-07T17:51:25.504 INFO:tasks.workunit.client.0.smithi100.stdout: "[68719476736, 137438953471]", 2024-04-07T17:51:25.504 INFO:tasks.workunit.client.0.smithi100.stdout: "[137438953472, 274877906943]", 2024-04-07T17:51:25.504 INFO:tasks.workunit.client.0.smithi100.stdout: "[274877906944, None]" 2024-04-07T17:51:25.504 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:25.504 INFO:tasks.workunit.client.0.smithi100.stdout: "scale_type": "log2" 2024-04-07T17:51:25.504 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.505 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:25.505 INFO:tasks.workunit.client.0.smithi100.stdout: "osds": [ 2024-04-07T17:51:25.505 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:25.505 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 0, 2024-04-07T17:51:25.505 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:25.505 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.505 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.505 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.505 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.505 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.505 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 441, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.505 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 5864, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.505 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1424, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.505 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 398, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.506 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 258, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.506 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 139, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.506 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 32, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.506 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.506 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.506 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.506 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.506 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.506 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.506 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.506 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.506 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.506 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.507 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.507 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.507 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.507 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.507 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.507 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.507 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.507 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.507 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.507 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:25.507 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:25.507 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.507 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:25.507 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 1, 2024-04-07T17:51:25.507 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:25.507 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.508 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.508 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.508 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.508 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.508 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 692, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.508 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 2912, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.508 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1650, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.508 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 2053, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.508 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1593, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.508 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 3210, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.508 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 807, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.508 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.508 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.508 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.508 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.508 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.509 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.509 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.509 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.509 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.509 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.509 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.509 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.509 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.509 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.509 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.509 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.509 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.509 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.509 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.509 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:25.509 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:25.510 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.510 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:25.510 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 2, 2024-04-07T17:51:25.510 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:25.510 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.510 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.510 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.510 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.510 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.510 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 253, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.510 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 7908, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.510 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 17717, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.510 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 904, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.510 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 325, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.511 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 152, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.511 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 28, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.511 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.511 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.511 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.511 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.511 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.511 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.511 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.511 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.511 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.511 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.511 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.511 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.511 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.512 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.512 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.512 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.512 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.512 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.512 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.512 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:25.512 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:25.512 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.512 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:25.512 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 3, 2024-04-07T17:51:25.512 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:25.512 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.512 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.512 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.512 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.513 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.513 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 282, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.513 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 3801, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.513 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 624, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.513 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 564, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.513 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 3656, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.513 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 4423, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.513 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 416, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.513 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.513 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.513 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.513 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.513 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.513 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.514 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.514 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.514 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.514 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.514 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.514 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.514 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.514 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.514 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.514 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.514 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.514 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.514 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.514 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:25.515 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:25.515 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.515 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:25.515 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 4, 2024-04-07T17:51:25.515 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:25.515 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.515 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.515 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.515 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.515 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.515 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 126, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.515 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 4261, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.515 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 825, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.515 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 140, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.515 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 88, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.515 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 55, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.516 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 16, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.516 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.516 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.516 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.516 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.516 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.516 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.516 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.516 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.516 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.516 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.516 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.516 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.516 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.517 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.517 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.517 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.517 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.517 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.517 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.517 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:25.517 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:25.517 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.517 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:25.517 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 5, 2024-04-07T17:51:25.517 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:25.517 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.517 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.517 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.517 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.518 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.518 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 2097, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.518 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 5529, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.518 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 731, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.518 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 730, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.518 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 374, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.518 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 143, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.518 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 20, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.518 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.518 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.518 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.518 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.518 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.519 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.519 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.519 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.519 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.519 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.519 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.519 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.519 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.519 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.519 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.519 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.519 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.519 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.519 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.519 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:25.519 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:25.520 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.520 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:25.520 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 6, 2024-04-07T17:51:25.520 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:25.520 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.520 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.520 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.520 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.520 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.520 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 52, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.520 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 2975, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.520 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 360, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.520 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 346, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.520 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 206, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.520 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 97, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.521 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 28, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.521 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.521 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.521 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.521 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.521 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.521 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.521 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.521 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.521 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.521 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.521 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.521 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.521 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.521 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.522 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.522 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.522 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.522 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.522 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.522 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:25.522 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:25.522 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.522 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:25.522 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 7, 2024-04-07T17:51:25.522 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:25.522 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.522 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.522 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.523 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.523 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.523 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 355, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.523 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 4385, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.523 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 6107, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.523 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 307, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.523 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 109, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.523 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 79, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.523 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 19, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.523 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.523 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.523 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.523 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.523 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.523 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.524 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.524 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.524 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.524 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.524 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.524 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.524 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.524 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.524 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.524 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.524 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.524 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.524 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.524 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:25.524 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:25.525 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.525 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:25.525 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.525 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_latency_in_bytes_histogram": { 2024-04-07T17:51:25.525 INFO:tasks.workunit.client.0.smithi100.stdout: "axes": [ 2024-04-07T17:51:25.525 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:25.525 INFO:tasks.workunit.client.0.smithi100.stdout: "buckets": 32, 2024-04-07T17:51:25.525 INFO:tasks.workunit.client.0.smithi100.stdout: "min": 0, 2024-04-07T17:51:25.525 INFO:tasks.workunit.client.0.smithi100.stdout: "name": "Latency (usec)", 2024-04-07T17:51:25.525 INFO:tasks.workunit.client.0.smithi100.stdout: "quant_size": 100000, 2024-04-07T17:51:25.525 INFO:tasks.workunit.client.0.smithi100.stdout: "ranges": [ 2024-04-07T17:51:25.525 INFO:tasks.workunit.client.0.smithi100.stdout: "[None, -1]", 2024-04-07T17:51:25.525 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 99999]", 2024-04-07T17:51:25.525 INFO:tasks.workunit.client.0.smithi100.stdout: "[100000, 199999]", 2024-04-07T17:51:25.525 INFO:tasks.workunit.client.0.smithi100.stdout: "[200000, 399999]", 2024-04-07T17:51:25.526 INFO:tasks.workunit.client.0.smithi100.stdout: "[400000, 799999]", 2024-04-07T17:51:25.526 INFO:tasks.workunit.client.0.smithi100.stdout: "[800000, 1599999]", 2024-04-07T17:51:25.526 INFO:tasks.workunit.client.0.smithi100.stdout: "[1600000, 3199999]", 2024-04-07T17:51:25.526 INFO:tasks.workunit.client.0.smithi100.stdout: "[3200000, 6399999]", 2024-04-07T17:51:25.526 INFO:tasks.workunit.client.0.smithi100.stdout: "[6400000, 12799999]", 2024-04-07T17:51:25.526 INFO:tasks.workunit.client.0.smithi100.stdout: "[12800000, 25599999]", 2024-04-07T17:51:25.526 INFO:tasks.workunit.client.0.smithi100.stdout: "[25600000, 51199999]", 2024-04-07T17:51:25.526 INFO:tasks.workunit.client.0.smithi100.stdout: "[51200000, 102399999]", 2024-04-07T17:51:25.526 INFO:tasks.workunit.client.0.smithi100.stdout: "[102400000, 204799999]", 2024-04-07T17:51:25.526 INFO:tasks.workunit.client.0.smithi100.stdout: "[204800000, 409599999]", 2024-04-07T17:51:25.526 INFO:tasks.workunit.client.0.smithi100.stdout: "[409600000, 819199999]", 2024-04-07T17:51:25.526 INFO:tasks.workunit.client.0.smithi100.stdout: "[819200000, 1638399999]", 2024-04-07T17:51:25.526 INFO:tasks.workunit.client.0.smithi100.stdout: "[1638400000, 3276799999]", 2024-04-07T17:51:25.526 INFO:tasks.workunit.client.0.smithi100.stdout: "[3276800000, 6553599999]", 2024-04-07T17:51:25.526 INFO:tasks.workunit.client.0.smithi100.stdout: "[6553600000, 13107199999]", 2024-04-07T17:51:25.527 INFO:tasks.workunit.client.0.smithi100.stdout: "[13107200000, 26214399999]", 2024-04-07T17:51:25.527 INFO:tasks.workunit.client.0.smithi100.stdout: "[26214400000, 52428799999]", 2024-04-07T17:51:25.527 INFO:tasks.workunit.client.0.smithi100.stdout: "[52428800000, 104857599999]", 2024-04-07T17:51:25.527 INFO:tasks.workunit.client.0.smithi100.stdout: "[104857600000, 209715199999]", 2024-04-07T17:51:25.527 INFO:tasks.workunit.client.0.smithi100.stdout: "[209715200000, 419430399999]", 2024-04-07T17:51:25.527 INFO:tasks.workunit.client.0.smithi100.stdout: "[419430400000, 838860799999]", 2024-04-07T17:51:25.527 INFO:tasks.workunit.client.0.smithi100.stdout: "[838860800000, 1677721599999]", 2024-04-07T17:51:25.527 INFO:tasks.workunit.client.0.smithi100.stdout: "[1677721600000, 3355443199999]", 2024-04-07T17:51:25.527 INFO:tasks.workunit.client.0.smithi100.stdout: "[3355443200000, 6710886399999]", 2024-04-07T17:51:25.527 INFO:tasks.workunit.client.0.smithi100.stdout: "[6710886400000, 13421772799999]", 2024-04-07T17:51:25.527 INFO:tasks.workunit.client.0.smithi100.stdout: "[13421772800000, 26843545599999]", 2024-04-07T17:51:25.527 INFO:tasks.workunit.client.0.smithi100.stdout: "[26843545600000, 53687091199999]", 2024-04-07T17:51:25.527 INFO:tasks.workunit.client.0.smithi100.stdout: "[53687091200000, None]" 2024-04-07T17:51:25.527 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:25.527 INFO:tasks.workunit.client.0.smithi100.stdout: "scale_type": "log2" 2024-04-07T17:51:25.527 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.528 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:25.528 INFO:tasks.workunit.client.0.smithi100.stdout: "buckets": 32, 2024-04-07T17:51:25.528 INFO:tasks.workunit.client.0.smithi100.stdout: "min": 0, 2024-04-07T17:51:25.528 INFO:tasks.workunit.client.0.smithi100.stdout: "name": "Request size (bytes)", 2024-04-07T17:51:25.528 INFO:tasks.workunit.client.0.smithi100.stdout: "quant_size": 512, 2024-04-07T17:51:25.528 INFO:tasks.workunit.client.0.smithi100.stdout: "ranges": [ 2024-04-07T17:51:25.528 INFO:tasks.workunit.client.0.smithi100.stdout: "[None, -1]", 2024-04-07T17:51:25.528 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 511]", 2024-04-07T17:51:25.528 INFO:tasks.workunit.client.0.smithi100.stdout: "[512, 1023]", 2024-04-07T17:51:25.528 INFO:tasks.workunit.client.0.smithi100.stdout: "[1024, 2047]", 2024-04-07T17:51:25.528 INFO:tasks.workunit.client.0.smithi100.stdout: "[2048, 4095]", 2024-04-07T17:51:25.528 INFO:tasks.workunit.client.0.smithi100.stdout: "[4096, 8191]", 2024-04-07T17:51:25.528 INFO:tasks.workunit.client.0.smithi100.stdout: "[8192, 16383]", 2024-04-07T17:51:25.528 INFO:tasks.workunit.client.0.smithi100.stdout: "[16384, 32767]", 2024-04-07T17:51:25.528 INFO:tasks.workunit.client.0.smithi100.stdout: "[32768, 65535]", 2024-04-07T17:51:25.528 INFO:tasks.workunit.client.0.smithi100.stdout: "[65536, 131071]", 2024-04-07T17:51:25.529 INFO:tasks.workunit.client.0.smithi100.stdout: "[131072, 262143]", 2024-04-07T17:51:25.529 INFO:tasks.workunit.client.0.smithi100.stdout: "[262144, 524287]", 2024-04-07T17:51:25.529 INFO:tasks.workunit.client.0.smithi100.stdout: "[524288, 1048575]", 2024-04-07T17:51:25.529 INFO:tasks.workunit.client.0.smithi100.stdout: "[1048576, 2097151]", 2024-04-07T17:51:25.529 INFO:tasks.workunit.client.0.smithi100.stdout: "[2097152, 4194303]", 2024-04-07T17:51:25.529 INFO:tasks.workunit.client.0.smithi100.stdout: "[4194304, 8388607]", 2024-04-07T17:51:25.529 INFO:tasks.workunit.client.0.smithi100.stdout: "[8388608, 16777215]", 2024-04-07T17:51:25.529 INFO:tasks.workunit.client.0.smithi100.stdout: "[16777216, 33554431]", 2024-04-07T17:51:25.529 INFO:tasks.workunit.client.0.smithi100.stdout: "[33554432, 67108863]", 2024-04-07T17:51:25.529 INFO:tasks.workunit.client.0.smithi100.stdout: "[67108864, 134217727]", 2024-04-07T17:51:25.529 INFO:tasks.workunit.client.0.smithi100.stdout: "[134217728, 268435455]", 2024-04-07T17:51:25.529 INFO:tasks.workunit.client.0.smithi100.stdout: "[268435456, 536870911]", 2024-04-07T17:51:25.529 INFO:tasks.workunit.client.0.smithi100.stdout: "[536870912, 1073741823]", 2024-04-07T17:51:25.529 INFO:tasks.workunit.client.0.smithi100.stdout: "[1073741824, 2147483647]", 2024-04-07T17:51:25.529 INFO:tasks.workunit.client.0.smithi100.stdout: "[2147483648, 4294967295]", 2024-04-07T17:51:25.530 INFO:tasks.workunit.client.0.smithi100.stdout: "[4294967296, 8589934591]", 2024-04-07T17:51:25.530 INFO:tasks.workunit.client.0.smithi100.stdout: "[8589934592, 17179869183]", 2024-04-07T17:51:25.530 INFO:tasks.workunit.client.0.smithi100.stdout: "[17179869184, 34359738367]", 2024-04-07T17:51:25.530 INFO:tasks.workunit.client.0.smithi100.stdout: "[34359738368, 68719476735]", 2024-04-07T17:51:25.530 INFO:tasks.workunit.client.0.smithi100.stdout: "[68719476736, 137438953471]", 2024-04-07T17:51:25.530 INFO:tasks.workunit.client.0.smithi100.stdout: "[137438953472, 274877906943]", 2024-04-07T17:51:25.530 INFO:tasks.workunit.client.0.smithi100.stdout: "[274877906944, None]" 2024-04-07T17:51:25.530 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:25.530 INFO:tasks.workunit.client.0.smithi100.stdout: "scale_type": "log2" 2024-04-07T17:51:25.530 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.530 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:25.530 INFO:tasks.workunit.client.0.smithi100.stdout: "osds": [ 2024-04-07T17:51:25.530 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:25.530 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 0, 2024-04-07T17:51:25.530 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:25.530 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.531 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.531 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.531 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.531 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.531 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 45, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.531 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 604, 18, 1, 2, 6, 0, 3, 3, 6, 3, 4, 1, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.531 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 2693, 8, 3, 0, 4, 3, 9, 15, 24, 57, 124, 51, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.531 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 270, 5, 0, 0, 1, 1, 0, 4, 3, 10, 38, 191, 289, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.531 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 410, 5, 0, 0, 0, 0, 1, 1, 5, 7, 14, 34, 121, 20, 38, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.531 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 566, 0, 0, 0, 0, 0, 0, 0, 0, 2, 3, 6, 14, 38, 152, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.531 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 778, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 4, 29, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.531 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 481, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.531 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 388, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.531 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.532 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 4, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.532 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.532 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.532 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.532 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.532 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.532 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.532 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.532 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.532 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.532 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.532 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.532 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.532 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.532 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.533 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.533 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:25.533 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:25.533 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.533 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:25.533 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 1, 2024-04-07T17:51:25.533 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:25.533 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.533 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.533 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.533 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.533 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.533 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 21, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.533 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 641, 34, 4, 2, 6, 0, 0, 1, 0, 0, 2, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.533 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 3452, 9, 3, 2, 2, 7, 13, 17, 33, 73, 134, 69, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.534 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 133, 5, 0, 0, 2, 1, 2, 1, 3, 6, 19, 211, 422, 3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.534 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 66, 11, 0, 0, 0, 0, 0, 0, 3, 1, 6, 9, 158, 22, 32, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.534 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 191, 32, 0, 0, 0, 0, 0, 0, 0, 0, 6, 9, 24, 39, 105, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.534 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1198, 50, 0, 0, 0, 0, 0, 0, 0, 1, 0, 1, 0, 3, 18, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.534 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1626, 138, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.534 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 662, 17, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.534 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.534 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.534 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.534 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.534 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.534 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.534 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.534 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.535 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.535 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.535 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.535 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.535 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.535 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.535 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.535 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.535 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.535 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:25.535 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:25.535 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.535 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:25.535 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 2, 2024-04-07T17:51:25.535 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:25.536 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.536 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.536 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.536 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.536 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.536 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 22, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.536 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 335, 6, 1, 0, 4, 0, 1, 5, 6, 2, 5, 5, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.536 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 207, 9, 1, 0, 3, 0, 4, 3, 10, 17, 51, 63, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.536 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 199, 14, 0, 0, 0, 1, 4, 6, 13, 13, 79, 92, 13, 6, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.536 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 155, 8, 0, 0, 0, 0, 2, 2, 4, 11, 32, 52, 31, 36, 46, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.536 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 44, 0, 0, 0, 0, 0, 0, 0, 0, 3, 12, 14, 17, 59, 136, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.536 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 6, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 2, 4, 4, 27, 3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.536 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.536 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.536 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.537 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.537 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.537 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.537 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.537 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.537 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.537 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.537 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.537 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.537 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.537 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.537 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.537 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.537 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.537 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.538 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.538 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:25.538 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:25.538 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.538 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:25.538 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 3, 2024-04-07T17:51:25.538 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:25.538 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.538 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.538 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.538 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.538 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.538 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 26, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.538 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 494, 15, 0, 0, 11, 2, 2, 3, 5, 0, 3, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.538 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1353, 2, 2, 0, 3, 3, 5, 7, 20, 29, 59, 33, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.539 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 366, 7, 0, 0, 1, 0, 0, 3, 4, 6, 16, 89, 157, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.539 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 404, 5, 0, 0, 0, 0, 0, 1, 3, 1, 4, 17, 61, 29, 16, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.539 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 330, 0, 0, 0, 0, 0, 1, 0, 2, 1, 0, 4, 21, 46, 72, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.539 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 124, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 5, 3, 13, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.539 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 46, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.539 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.539 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.539 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.539 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.539 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.539 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.539 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.539 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.539 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.539 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.539 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.540 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.540 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.540 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.540 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.540 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.540 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.540 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.540 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:25.540 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:25.540 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.540 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:25.540 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 4, 2024-04-07T17:51:25.540 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:25.541 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.541 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.541 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.541 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.541 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.541 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 16, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.541 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 441, 22, 0, 0, 7, 0, 1, 0, 0, 0, 1, 2, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.541 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1429, 3, 1, 1, 4, 5, 1, 8, 16, 28, 51, 23, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.541 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 545, 0, 0, 0, 0, 2, 3, 3, 3, 3, 26, 99, 129, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.541 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 331, 0, 0, 0, 0, 0, 0, 0, 1, 3, 19, 19, 107, 9, 19, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.541 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 162, 0, 0, 0, 0, 0, 0, 0, 0, 1, 2, 3, 7, 23, 85, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.541 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 151, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 2, 19, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.541 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 71, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.541 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.541 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.541 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.542 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.542 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.542 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.542 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.542 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.542 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.542 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.542 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.542 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.542 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.542 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.542 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.542 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.542 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.543 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.543 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:25.543 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:25.543 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.543 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:25.543 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 5, 2024-04-07T17:51:25.543 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:25.543 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.543 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.543 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.543 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.543 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.543 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 12, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.543 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 550, 9, 0, 0, 5, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.543 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 111, 8, 0, 0, 8, 0, 0, 2, 3, 4, 9, 9, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.543 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 90, 5, 0, 0, 0, 0, 3, 4, 5, 5, 13, 12, 14, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.544 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 41, 4, 0, 0, 0, 0, 0, 0, 0, 3, 4, 15, 16, 15, 20, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.544 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 13, 0, 0, 0, 0, 0, 0, 1, 0, 0, 6, 4, 6, 42, 128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.544 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 3, 26, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.544 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.544 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.544 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.544 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.544 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.544 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.544 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.544 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.544 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.544 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.544 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.544 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.545 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.545 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.545 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.545 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.545 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.545 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.545 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.545 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:25.545 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:25.545 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.545 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:25.545 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 6, 2024-04-07T17:51:25.545 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:25.545 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.545 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.546 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.546 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.546 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.546 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 11, 0, 0, 1, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.546 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 423, 33, 0, 0, 16, 0, 2, 2, 1, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.546 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1503, 6, 1, 2, 5, 0, 4, 10, 23, 27, 44, 18, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.546 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 492, 4, 0, 0, 1, 1, 1, 1, 1, 2, 19, 86, 117, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.546 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 382, 9, 0, 0, 0, 0, 0, 0, 1, 1, 4, 7, 121, 9, 15, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.546 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 160, 28, 0, 0, 0, 0, 0, 0, 1, 1, 1, 3, 11, 32, 99, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.546 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 129, 75, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 2, 20, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.546 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 75, 153, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.546 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.546 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.546 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.546 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.547 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.547 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.547 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.547 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.547 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.547 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.547 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.547 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.547 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.547 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.547 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.547 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.547 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.547 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.547 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:25.547 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:25.548 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.548 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:25.548 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 7, 2024-04-07T17:51:25.548 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:25.548 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.548 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.548 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.548 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.548 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.548 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 12, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.548 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 239, 5, 0, 0, 6, 0, 2, 3, 5, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.548 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 61, 4, 0, 0, 2, 1, 1, 2, 3, 12, 8, 13, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.548 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 75, 12, 0, 0, 0, 0, 2, 2, 2, 9, 9, 19, 6, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.548 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 49, 15, 0, 0, 0, 0, 1, 2, 2, 2, 8, 21, 29, 29, 10, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.548 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 13, 33, 0, 0, 0, 0, 1, 0, 0, 2, 2, 4, 15, 67, 122, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.549 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 64, 0, 0, 0, 0, 0, 0, 0, 1, 0, 1, 0, 4, 26, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.549 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 149, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.549 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.549 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.549 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.549 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.549 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.549 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.549 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.549 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.549 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.549 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.549 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.549 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.550 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.550 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.550 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.550 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.550 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.550 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.550 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:25.550 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:25.550 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.550 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:25.550 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.550 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.550 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:25.550 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_resrv_repnum_vs_duration": { 2024-04-07T17:51:25.550 INFO:tasks.workunit.client.0.smithi100.stdout: "axes": [ 2024-04-07T17:51:25.551 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:25.551 INFO:tasks.workunit.client.0.smithi100.stdout: "buckets": 8, 2024-04-07T17:51:25.551 INFO:tasks.workunit.client.0.smithi100.stdout: "min": 0, 2024-04-07T17:51:25.551 INFO:tasks.workunit.client.0.smithi100.stdout: "name": "number of replicas", 2024-04-07T17:51:25.551 INFO:tasks.workunit.client.0.smithi100.stdout: "quant_size": 1, 2024-04-07T17:51:25.551 INFO:tasks.workunit.client.0.smithi100.stdout: "ranges": [ 2024-04-07T17:51:25.551 INFO:tasks.workunit.client.0.smithi100.stdout: "[None, -1]", 2024-04-07T17:51:25.551 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0]", 2024-04-07T17:51:25.551 INFO:tasks.workunit.client.0.smithi100.stdout: "[1, 1]", 2024-04-07T17:51:25.551 INFO:tasks.workunit.client.0.smithi100.stdout: "[2, 2]", 2024-04-07T17:51:25.551 INFO:tasks.workunit.client.0.smithi100.stdout: "[3, 3]", 2024-04-07T17:51:25.551 INFO:tasks.workunit.client.0.smithi100.stdout: "[4, 4]", 2024-04-07T17:51:25.551 INFO:tasks.workunit.client.0.smithi100.stdout: "[5, 5]", 2024-04-07T17:51:25.551 INFO:tasks.workunit.client.0.smithi100.stdout: "[6, None]" 2024-04-07T17:51:25.551 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:25.551 INFO:tasks.workunit.client.0.smithi100.stdout: "scale_type": "linear" 2024-04-07T17:51:25.552 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.552 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:25.552 INFO:tasks.workunit.client.0.smithi100.stdout: "buckets": 10, 2024-04-07T17:51:25.552 INFO:tasks.workunit.client.0.smithi100.stdout: "min": 0, 2024-04-07T17:51:25.552 INFO:tasks.workunit.client.0.smithi100.stdout: "name": "duration", 2024-04-07T17:51:25.552 INFO:tasks.workunit.client.0.smithi100.stdout: "quant_size": 250000, 2024-04-07T17:51:25.552 INFO:tasks.workunit.client.0.smithi100.stdout: "ranges": [ 2024-04-07T17:51:25.552 INFO:tasks.workunit.client.0.smithi100.stdout: "[None, -1]", 2024-04-07T17:51:25.552 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 249999]", 2024-04-07T17:51:25.552 INFO:tasks.workunit.client.0.smithi100.stdout: "[250000, 499999]", 2024-04-07T17:51:25.552 INFO:tasks.workunit.client.0.smithi100.stdout: "[500000, 999999]", 2024-04-07T17:51:25.552 INFO:tasks.workunit.client.0.smithi100.stdout: "[1000000, 1999999]", 2024-04-07T17:51:25.552 INFO:tasks.workunit.client.0.smithi100.stdout: "[2000000, 3999999]", 2024-04-07T17:51:25.552 INFO:tasks.workunit.client.0.smithi100.stdout: "[4000000, 7999999]", 2024-04-07T17:51:25.552 INFO:tasks.workunit.client.0.smithi100.stdout: "[8000000, 15999999]", 2024-04-07T17:51:25.552 INFO:tasks.workunit.client.0.smithi100.stdout: "[16000000, 31999999]", 2024-04-07T17:51:25.553 INFO:tasks.workunit.client.0.smithi100.stdout: "[32000000, None]" 2024-04-07T17:51:25.553 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:25.553 INFO:tasks.workunit.client.0.smithi100.stdout: "scale_type": "log2" 2024-04-07T17:51:25.553 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.553 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:25.553 INFO:tasks.workunit.client.0.smithi100.stdout: "osds": [ 2024-04-07T17:51:25.553 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:25.553 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 0, 2024-04-07T17:51:25.553 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:25.553 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.553 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.553 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.553 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.553 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.553 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.553 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.554 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:25.554 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:25.554 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.554 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:25.554 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 1, 2024-04-07T17:51:25.554 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:25.554 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.554 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.554 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.554 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.554 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.554 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.554 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.554 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:25.555 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:25.555 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.555 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:25.555 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 2, 2024-04-07T17:51:25.555 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:25.555 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.555 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.555 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.555 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.555 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.555 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.555 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.555 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:25.555 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:25.555 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.555 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:25.556 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 3, 2024-04-07T17:51:25.556 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:25.556 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.556 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.556 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.556 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.556 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.556 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.556 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.556 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:25.556 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:25.556 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.556 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:25.556 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 4, 2024-04-07T17:51:25.556 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:25.556 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.557 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.557 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.557 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.557 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.557 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.557 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.557 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:25.557 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:25.557 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.557 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:25.557 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 5, 2024-04-07T17:51:25.557 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:25.557 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.557 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.557 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.557 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.558 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.558 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.558 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.558 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:25.558 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:25.558 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.558 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:25.558 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 6, 2024-04-07T17:51:25.558 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:25.558 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.558 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.558 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.558 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.559 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.559 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.559 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.559 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:25.559 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:25.559 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.559 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:25.559 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 7, 2024-04-07T17:51:25.559 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:25.559 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.559 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.559 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.559 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.559 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.559 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.559 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:25.560 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:25.560 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:25.560 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.560 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:25.560 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.560 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.560 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:25.560 INFO:tasks.workunit.client.0.smithi100.stdout: "perf_counters": { 2024-04-07T17:51:25.560 INFO:tasks.workunit.client.0.smithi100.stdout: "mds.f47a4ff15a45950a63160265564a123fc5da5e5e": { 2024-04-07T17:51:25.560 INFO:tasks.workunit.client.0.smithi100.stdout: "mds": { 2024-04-07T17:51:25.560 INFO:tasks.workunit.client.0.smithi100.stdout: "caps": { 2024-04-07T17:51:25.560 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.560 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.560 INFO:tasks.workunit.client.0.smithi100.stdout: "ceph_cap_op_flush_ack": { 2024-04-07T17:51:25.561 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.561 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.561 INFO:tasks.workunit.client.0.smithi100.stdout: "ceph_cap_op_flushsnap_ack": { 2024-04-07T17:51:25.561 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.561 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.561 INFO:tasks.workunit.client.0.smithi100.stdout: "ceph_cap_op_grant": { 2024-04-07T17:51:25.561 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.561 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.561 INFO:tasks.workunit.client.0.smithi100.stdout: "ceph_cap_op_revoke": { 2024-04-07T17:51:25.561 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.561 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.561 INFO:tasks.workunit.client.0.smithi100.stdout: "ceph_cap_op_trunc": { 2024-04-07T17:51:25.561 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.561 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.561 INFO:tasks.workunit.client.0.smithi100.stdout: "dir_commit": { 2024-04-07T17:51:25.562 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.562 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.562 INFO:tasks.workunit.client.0.smithi100.stdout: "dir_fetch_complete": { 2024-04-07T17:51:25.562 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 12 2024-04-07T17:51:25.562 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.562 INFO:tasks.workunit.client.0.smithi100.stdout: "dir_fetch_keys": { 2024-04-07T17:51:25.562 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 10 2024-04-07T17:51:25.562 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.562 INFO:tasks.workunit.client.0.smithi100.stdout: "dir_merge": { 2024-04-07T17:51:25.562 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.562 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.562 INFO:tasks.workunit.client.0.smithi100.stdout: "dir_split": { 2024-04-07T17:51:25.562 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.562 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.562 INFO:tasks.workunit.client.0.smithi100.stdout: "exported_inodes": { 2024-04-07T17:51:25.563 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.563 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.563 INFO:tasks.workunit.client.0.smithi100.stdout: "forward": { 2024-04-07T17:51:25.563 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.563 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.563 INFO:tasks.workunit.client.0.smithi100.stdout: "handle_client_cap_release": { 2024-04-07T17:51:25.563 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.563 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.563 INFO:tasks.workunit.client.0.smithi100.stdout: "handle_client_caps": { 2024-04-07T17:51:25.563 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.563 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.563 INFO:tasks.workunit.client.0.smithi100.stdout: "handle_client_caps_dirty": { 2024-04-07T17:51:25.563 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.563 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.563 INFO:tasks.workunit.client.0.smithi100.stdout: "handle_inode_file_caps": { 2024-04-07T17:51:25.564 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.564 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.564 INFO:tasks.workunit.client.0.smithi100.stdout: "imported_inodes": { 2024-04-07T17:51:25.564 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.564 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.564 INFO:tasks.workunit.client.0.smithi100.stdout: "inodes": { 2024-04-07T17:51:25.564 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 10 2024-04-07T17:51:25.564 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.564 INFO:tasks.workunit.client.0.smithi100.stdout: "inodes_expired": { 2024-04-07T17:51:25.564 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.564 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.564 INFO:tasks.workunit.client.0.smithi100.stdout: "inodes_pinned": { 2024-04-07T17:51:25.564 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 10 2024-04-07T17:51:25.564 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.564 INFO:tasks.workunit.client.0.smithi100.stdout: "inodes_with_caps": { 2024-04-07T17:51:25.564 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.565 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.565 INFO:tasks.workunit.client.0.smithi100.stdout: "load_cent": { 2024-04-07T17:51:25.565 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.565 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.565 INFO:tasks.workunit.client.0.smithi100.stdout: "openino_dir_fetch": { 2024-04-07T17:51:25.565 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.565 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.565 INFO:tasks.workunit.client.0.smithi100.stdout: "process_request_cap_release": { 2024-04-07T17:51:25.565 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.565 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.565 INFO:tasks.workunit.client.0.smithi100.stdout: "reply_latency": { 2024-04-07T17:51:25.565 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:25.565 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.565 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.566 INFO:tasks.workunit.client.0.smithi100.stdout: "request": { 2024-04-07T17:51:25.566 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.566 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.566 INFO:tasks.workunit.client.0.smithi100.stdout: "root_rbytes": { 2024-04-07T17:51:25.566 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.566 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.566 INFO:tasks.workunit.client.0.smithi100.stdout: "root_rfiles": { 2024-04-07T17:51:25.566 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.566 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.566 INFO:tasks.workunit.client.0.smithi100.stdout: "root_rsnaps": { 2024-04-07T17:51:25.566 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.566 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.566 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_reply": { 2024-04-07T17:51:25.566 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.566 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.567 INFO:tasks.workunit.client.0.smithi100.stdout: "subtrees": { 2024-04-07T17:51:25.567 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2 2024-04-07T17:51:25.567 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.567 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.567 INFO:tasks.workunit.client.0.smithi100.stdout: "mds_cache": { 2024-04-07T17:51:25.567 INFO:tasks.workunit.client.0.smithi100.stdout: "ireq_enqueue_scrub": { 2024-04-07T17:51:25.567 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.567 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.567 INFO:tasks.workunit.client.0.smithi100.stdout: "ireq_exportdir": { 2024-04-07T17:51:25.567 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.567 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.567 INFO:tasks.workunit.client.0.smithi100.stdout: "ireq_flush": { 2024-04-07T17:51:25.567 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.567 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.567 INFO:tasks.workunit.client.0.smithi100.stdout: "ireq_fragmentdir": { 2024-04-07T17:51:25.567 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.568 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.568 INFO:tasks.workunit.client.0.smithi100.stdout: "ireq_fragstats": { 2024-04-07T17:51:25.568 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.568 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.568 INFO:tasks.workunit.client.0.smithi100.stdout: "ireq_inodestats": { 2024-04-07T17:51:25.568 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.568 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.568 INFO:tasks.workunit.client.0.smithi100.stdout: "ireq_quiesce_inode": { 2024-04-07T17:51:25.568 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.568 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.568 INFO:tasks.workunit.client.0.smithi100.stdout: "ireq_quiesce_path": { 2024-04-07T17:51:25.568 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.568 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.568 INFO:tasks.workunit.client.0.smithi100.stdout: "num_recovering_enqueued": { 2024-04-07T17:51:25.568 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.568 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.569 INFO:tasks.workunit.client.0.smithi100.stdout: "num_recovering_prioritized": { 2024-04-07T17:51:25.569 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.569 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.569 INFO:tasks.workunit.client.0.smithi100.stdout: "num_recovering_processing": { 2024-04-07T17:51:25.569 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.569 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.569 INFO:tasks.workunit.client.0.smithi100.stdout: "num_strays": { 2024-04-07T17:51:25.569 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.569 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.569 INFO:tasks.workunit.client.0.smithi100.stdout: "num_strays_delayed": { 2024-04-07T17:51:25.569 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.569 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.569 INFO:tasks.workunit.client.0.smithi100.stdout: "num_strays_enqueuing": { 2024-04-07T17:51:25.569 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.570 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.570 INFO:tasks.workunit.client.0.smithi100.stdout: "recovery_completed": { 2024-04-07T17:51:25.570 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.570 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.570 INFO:tasks.workunit.client.0.smithi100.stdout: "recovery_started": { 2024-04-07T17:51:25.570 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.570 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.570 INFO:tasks.workunit.client.0.smithi100.stdout: "strays_created": { 2024-04-07T17:51:25.570 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.570 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.570 INFO:tasks.workunit.client.0.smithi100.stdout: "strays_enqueued": { 2024-04-07T17:51:25.570 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.570 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.570 INFO:tasks.workunit.client.0.smithi100.stdout: "strays_migrated": { 2024-04-07T17:51:25.570 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.571 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.571 INFO:tasks.workunit.client.0.smithi100.stdout: "strays_reintegrated": { 2024-04-07T17:51:25.571 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.571 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.571 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.571 INFO:tasks.workunit.client.0.smithi100.stdout: "mds_log": { 2024-04-07T17:51:25.571 INFO:tasks.workunit.client.0.smithi100.stdout: "ev": { 2024-04-07T17:51:25.571 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 17 2024-04-07T17:51:25.571 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.571 INFO:tasks.workunit.client.0.smithi100.stdout: "evadd": { 2024-04-07T17:51:25.571 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.571 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.571 INFO:tasks.workunit.client.0.smithi100.stdout: "evex": { 2024-04-07T17:51:25.571 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.571 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.571 INFO:tasks.workunit.client.0.smithi100.stdout: "evexd": { 2024-04-07T17:51:25.572 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.572 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.572 INFO:tasks.workunit.client.0.smithi100.stdout: "evexg": { 2024-04-07T17:51:25.572 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.572 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.572 INFO:tasks.workunit.client.0.smithi100.stdout: "evlrg": { 2024-04-07T17:51:25.572 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.572 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.572 INFO:tasks.workunit.client.0.smithi100.stdout: "evtrm": { 2024-04-07T17:51:25.572 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.572 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.572 INFO:tasks.workunit.client.0.smithi100.stdout: "jlat": { 2024-04-07T17:51:25.572 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:25.572 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.572 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.573 INFO:tasks.workunit.client.0.smithi100.stdout: "replayed": { 2024-04-07T17:51:25.573 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 17 2024-04-07T17:51:25.573 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.573 INFO:tasks.workunit.client.0.smithi100.stdout: "seg": { 2024-04-07T17:51:25.573 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1 2024-04-07T17:51:25.573 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.573 INFO:tasks.workunit.client.0.smithi100.stdout: "segadd": { 2024-04-07T17:51:25.573 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.573 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.573 INFO:tasks.workunit.client.0.smithi100.stdout: "segex": { 2024-04-07T17:51:25.573 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.573 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.573 INFO:tasks.workunit.client.0.smithi100.stdout: "segexd": { 2024-04-07T17:51:25.573 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.573 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.573 INFO:tasks.workunit.client.0.smithi100.stdout: "segexg": { 2024-04-07T17:51:25.574 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.574 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.574 INFO:tasks.workunit.client.0.smithi100.stdout: "segmjr": { 2024-04-07T17:51:25.574 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1 2024-04-07T17:51:25.574 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.574 INFO:tasks.workunit.client.0.smithi100.stdout: "segtrm": { 2024-04-07T17:51:25.574 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.574 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.574 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.574 INFO:tasks.workunit.client.0.smithi100.stdout: "mds_mem": { 2024-04-07T17:51:25.574 INFO:tasks.workunit.client.0.smithi100.stdout: "cap": { 2024-04-07T17:51:25.574 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.574 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.575 INFO:tasks.workunit.client.0.smithi100.stdout: "cap+": { 2024-04-07T17:51:25.575 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.575 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.575 INFO:tasks.workunit.client.0.smithi100.stdout: "cap-": { 2024-04-07T17:51:25.575 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.575 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.575 INFO:tasks.workunit.client.0.smithi100.stdout: "dir": { 2024-04-07T17:51:25.575 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 12 2024-04-07T17:51:25.575 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.575 INFO:tasks.workunit.client.0.smithi100.stdout: "dir+": { 2024-04-07T17:51:25.575 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 12 2024-04-07T17:51:25.575 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.575 INFO:tasks.workunit.client.0.smithi100.stdout: "dir-": { 2024-04-07T17:51:25.575 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.575 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.575 INFO:tasks.workunit.client.0.smithi100.stdout: "dn": { 2024-04-07T17:51:25.576 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 10 2024-04-07T17:51:25.576 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.576 INFO:tasks.workunit.client.0.smithi100.stdout: "dn+": { 2024-04-07T17:51:25.576 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 10 2024-04-07T17:51:25.576 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.576 INFO:tasks.workunit.client.0.smithi100.stdout: "dn-": { 2024-04-07T17:51:25.576 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.576 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.576 INFO:tasks.workunit.client.0.smithi100.stdout: "heap": { 2024-04-07T17:51:25.576 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 248184 2024-04-07T17:51:25.576 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.576 INFO:tasks.workunit.client.0.smithi100.stdout: "ino": { 2024-04-07T17:51:25.576 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 13 2024-04-07T17:51:25.576 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.576 INFO:tasks.workunit.client.0.smithi100.stdout: "ino+": { 2024-04-07T17:51:25.576 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 13 2024-04-07T17:51:25.577 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.577 INFO:tasks.workunit.client.0.smithi100.stdout: "ino-": { 2024-04-07T17:51:25.577 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.577 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.577 INFO:tasks.workunit.client.0.smithi100.stdout: "rss": { 2024-04-07T17:51:25.577 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 46296 2024-04-07T17:51:25.577 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.577 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.577 INFO:tasks.workunit.client.0.smithi100.stdout: "mds_server": { 2024-04-07T17:51:25.577 INFO:tasks.workunit.client.0.smithi100.stdout: "cap_acquisition_throttle": { 2024-04-07T17:51:25.577 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.577 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.577 INFO:tasks.workunit.client.0.smithi100.stdout: "cap_revoke_eviction": { 2024-04-07T17:51:25.577 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.577 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.578 INFO:tasks.workunit.client.0.smithi100.stdout: "handle_client_request": { 2024-04-07T17:51:25.578 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.578 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.578 INFO:tasks.workunit.client.0.smithi100.stdout: "handle_client_session": { 2024-04-07T17:51:25.578 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.578 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.578 INFO:tasks.workunit.client.0.smithi100.stdout: "handle_peer_request": { 2024-04-07T17:51:25.578 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.578 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.578 INFO:tasks.workunit.client.0.smithi100.stdout: "req_create_latency": { 2024-04-07T17:51:25.578 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:25.578 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.578 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.578 INFO:tasks.workunit.client.0.smithi100.stdout: "req_getattr_latency": { 2024-04-07T17:51:25.578 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:25.579 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.579 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.579 INFO:tasks.workunit.client.0.smithi100.stdout: "req_getfilelock_latency": { 2024-04-07T17:51:25.579 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:25.579 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.579 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.579 INFO:tasks.workunit.client.0.smithi100.stdout: "req_getvxattr_latency": { 2024-04-07T17:51:25.579 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:25.579 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.579 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.579 INFO:tasks.workunit.client.0.smithi100.stdout: "req_link_latency": { 2024-04-07T17:51:25.579 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:25.579 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.579 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.579 INFO:tasks.workunit.client.0.smithi100.stdout: "req_lookup_latency": { 2024-04-07T17:51:25.580 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:25.580 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.580 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.580 INFO:tasks.workunit.client.0.smithi100.stdout: "req_lookuphash_latency": { 2024-04-07T17:51:25.580 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:25.580 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.580 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.580 INFO:tasks.workunit.client.0.smithi100.stdout: "req_lookupino_latency": { 2024-04-07T17:51:25.580 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:25.580 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.580 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.580 INFO:tasks.workunit.client.0.smithi100.stdout: "req_lookupname_latency": { 2024-04-07T17:51:25.580 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:25.580 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.580 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.580 INFO:tasks.workunit.client.0.smithi100.stdout: "req_lookupparent_latency": { 2024-04-07T17:51:25.581 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:25.581 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.581 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.581 INFO:tasks.workunit.client.0.smithi100.stdout: "req_lookupsnap_latency": { 2024-04-07T17:51:25.581 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:25.581 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.581 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.581 INFO:tasks.workunit.client.0.smithi100.stdout: "req_lssnap_latency": { 2024-04-07T17:51:25.581 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:25.581 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.581 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.581 INFO:tasks.workunit.client.0.smithi100.stdout: "req_mkdir_latency": { 2024-04-07T17:51:25.581 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:25.581 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.581 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.582 INFO:tasks.workunit.client.0.smithi100.stdout: "req_mknod_latency": { 2024-04-07T17:51:25.582 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:25.582 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.582 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.582 INFO:tasks.workunit.client.0.smithi100.stdout: "req_mksnap_latency": { 2024-04-07T17:51:25.582 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:25.582 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.582 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.582 INFO:tasks.workunit.client.0.smithi100.stdout: "req_open_latency": { 2024-04-07T17:51:25.582 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:25.582 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.582 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.582 INFO:tasks.workunit.client.0.smithi100.stdout: "req_readdir_latency": { 2024-04-07T17:51:25.582 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:25.582 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.583 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.583 INFO:tasks.workunit.client.0.smithi100.stdout: "req_rename_latency": { 2024-04-07T17:51:25.583 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:25.583 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.583 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.583 INFO:tasks.workunit.client.0.smithi100.stdout: "req_renamesnap_latency": { 2024-04-07T17:51:25.583 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:25.583 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.583 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.583 INFO:tasks.workunit.client.0.smithi100.stdout: "req_rmdir_latency": { 2024-04-07T17:51:25.583 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:25.583 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.583 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.583 INFO:tasks.workunit.client.0.smithi100.stdout: "req_rmsnap_latency": { 2024-04-07T17:51:25.583 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:25.584 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.584 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.584 INFO:tasks.workunit.client.0.smithi100.stdout: "req_rmxattr_latency": { 2024-04-07T17:51:25.584 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:25.584 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.584 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.584 INFO:tasks.workunit.client.0.smithi100.stdout: "req_setattr_latency": { 2024-04-07T17:51:25.584 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:25.584 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.584 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.584 INFO:tasks.workunit.client.0.smithi100.stdout: "req_setdirlayout_latency": { 2024-04-07T17:51:25.584 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:25.584 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.584 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.584 INFO:tasks.workunit.client.0.smithi100.stdout: "req_setfilelock_latency": { 2024-04-07T17:51:25.585 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:25.585 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.585 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.585 INFO:tasks.workunit.client.0.smithi100.stdout: "req_setlayout_latency": { 2024-04-07T17:51:25.585 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:25.585 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.585 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.585 INFO:tasks.workunit.client.0.smithi100.stdout: "req_setxattr_latency": { 2024-04-07T17:51:25.585 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:25.585 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.585 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.585 INFO:tasks.workunit.client.0.smithi100.stdout: "req_snapdiff_latency": { 2024-04-07T17:51:25.585 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:25.585 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.585 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.585 INFO:tasks.workunit.client.0.smithi100.stdout: "req_symlink_latency": { 2024-04-07T17:51:25.586 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:25.586 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.586 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.586 INFO:tasks.workunit.client.0.smithi100.stdout: "req_unlink_latency": { 2024-04-07T17:51:25.586 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:25.586 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.586 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.586 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.586 INFO:tasks.workunit.client.0.smithi100.stdout: "mds_sessions": { 2024-04-07T17:51:25.586 INFO:tasks.workunit.client.0.smithi100.stdout: "average_load": { 2024-04-07T17:51:25.586 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.586 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.586 INFO:tasks.workunit.client.0.smithi100.stdout: "avg_session_uptime": { 2024-04-07T17:51:25.586 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.587 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.587 INFO:tasks.workunit.client.0.smithi100.stdout: "mdthresh_evicted": { 2024-04-07T17:51:25.587 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.587 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.587 INFO:tasks.workunit.client.0.smithi100.stdout: "session_add": { 2024-04-07T17:51:25.587 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 8 2024-04-07T17:51:25.587 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.587 INFO:tasks.workunit.client.0.smithi100.stdout: "session_count": { 2024-04-07T17:51:25.587 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.587 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.587 INFO:tasks.workunit.client.0.smithi100.stdout: "session_remove": { 2024-04-07T17:51:25.587 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 8 2024-04-07T17:51:25.587 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.587 INFO:tasks.workunit.client.0.smithi100.stdout: "sessions_open": { 2024-04-07T17:51:25.587 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2 2024-04-07T17:51:25.587 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.588 INFO:tasks.workunit.client.0.smithi100.stdout: "sessions_stale": { 2024-04-07T17:51:25.588 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.588 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.588 INFO:tasks.workunit.client.0.smithi100.stdout: "total_load": { 2024-04-07T17:51:25.588 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.588 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.588 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.588 INFO:tasks.workunit.client.0.smithi100.stdout: "objecter": { 2024-04-07T17:51:25.588 INFO:tasks.workunit.client.0.smithi100.stdout: "op_active": { 2024-04-07T17:51:25.588 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.588 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.588 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r": { 2024-04-07T17:51:25.588 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 37 2024-04-07T17:51:25.588 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.588 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rmw": { 2024-04-07T17:51:25.588 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.589 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.589 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w": { 2024-04-07T17:51:25.589 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 7 2024-04-07T17:51:25.589 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.589 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.589 INFO:tasks.workunit.client.0.smithi100.stdout: "purge_queue": { 2024-04-07T17:51:25.589 INFO:tasks.workunit.client.0.smithi100.stdout: "pq_executed": { 2024-04-07T17:51:25.589 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.589 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.589 INFO:tasks.workunit.client.0.smithi100.stdout: "pq_executed_ops": { 2024-04-07T17:51:25.589 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.589 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.589 INFO:tasks.workunit.client.0.smithi100.stdout: "pq_executing": { 2024-04-07T17:51:25.589 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.589 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.590 INFO:tasks.workunit.client.0.smithi100.stdout: "pq_executing_high_water": { 2024-04-07T17:51:25.590 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.590 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.590 INFO:tasks.workunit.client.0.smithi100.stdout: "pq_executing_ops": { 2024-04-07T17:51:25.590 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.590 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.590 INFO:tasks.workunit.client.0.smithi100.stdout: "pq_executing_ops_high_water": { 2024-04-07T17:51:25.590 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.590 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.590 INFO:tasks.workunit.client.0.smithi100.stdout: "pq_item_in_journal": { 2024-04-07T17:51:25.590 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.590 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.590 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.590 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.590 INFO:tasks.workunit.client.0.smithi100.stdout: "mon.8dd0baa036f0a7e6d1f3bdce86136fe527836c3d": { 2024-04-07T17:51:25.590 INFO:tasks.workunit.client.0.smithi100.stdout: "mon": { 2024-04-07T17:51:25.591 INFO:tasks.workunit.client.0.smithi100.stdout: "election_call": { 2024-04-07T17:51:25.591 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2 2024-04-07T17:51:25.591 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.591 INFO:tasks.workunit.client.0.smithi100.stdout: "election_lose": { 2024-04-07T17:51:25.591 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3 2024-04-07T17:51:25.591 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.591 INFO:tasks.workunit.client.0.smithi100.stdout: "election_win": { 2024-04-07T17:51:25.591 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.591 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.591 INFO:tasks.workunit.client.0.smithi100.stdout: "num_elections": { 2024-04-07T17:51:25.591 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 12 2024-04-07T17:51:25.591 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.591 INFO:tasks.workunit.client.0.smithi100.stdout: "num_sessions": { 2024-04-07T17:51:25.591 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 7 2024-04-07T17:51:25.592 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.592 INFO:tasks.workunit.client.0.smithi100.stdout: "session_add": { 2024-04-07T17:51:25.592 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4 2024-04-07T17:51:25.592 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.592 INFO:tasks.workunit.client.0.smithi100.stdout: "session_rm": { 2024-04-07T17:51:25.592 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 253 2024-04-07T17:51:25.592 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.592 INFO:tasks.workunit.client.0.smithi100.stdout: "session_trim": { 2024-04-07T17:51:25.592 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.592 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.592 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.592 INFO:tasks.workunit.client.0.smithi100.stdout: "paxos": { 2024-04-07T17:51:25.592 INFO:tasks.workunit.client.0.smithi100.stdout: "accept_timeout": { 2024-04-07T17:51:25.592 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.592 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.593 INFO:tasks.workunit.client.0.smithi100.stdout: "begin": { 2024-04-07T17:51:25.593 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 8797 2024-04-07T17:51:25.593 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.593 INFO:tasks.workunit.client.0.smithi100.stdout: "begin_bytes": { 2024-04-07T17:51:25.593 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 8797, 2024-04-07T17:51:25.593 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 59268356 2024-04-07T17:51:25.593 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.593 INFO:tasks.workunit.client.0.smithi100.stdout: "begin_keys": { 2024-04-07T17:51:25.593 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:25.593 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.593 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.593 INFO:tasks.workunit.client.0.smithi100.stdout: "begin_latency": { 2024-04-07T17:51:25.593 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 8797, 2024-04-07T17:51:25.593 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3095913350 2024-04-07T17:51:25.593 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.593 INFO:tasks.workunit.client.0.smithi100.stdout: "collect": { 2024-04-07T17:51:25.594 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3 2024-04-07T17:51:25.594 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.594 INFO:tasks.workunit.client.0.smithi100.stdout: "collect_bytes": { 2024-04-07T17:51:25.594 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 3, 2024-04-07T17:51:25.594 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 219 2024-04-07T17:51:25.594 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.594 INFO:tasks.workunit.client.0.smithi100.stdout: "collect_keys": { 2024-04-07T17:51:25.594 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 3, 2024-04-07T17:51:25.594 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3 2024-04-07T17:51:25.594 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.594 INFO:tasks.workunit.client.0.smithi100.stdout: "collect_latency": { 2024-04-07T17:51:25.594 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 3, 2024-04-07T17:51:25.594 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.594 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.594 INFO:tasks.workunit.client.0.smithi100.stdout: "collect_timeout": { 2024-04-07T17:51:25.595 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.595 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.595 INFO:tasks.workunit.client.0.smithi100.stdout: "collect_uncommitted": { 2024-04-07T17:51:25.595 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1 2024-04-07T17:51:25.595 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.595 INFO:tasks.workunit.client.0.smithi100.stdout: "commit": { 2024-04-07T17:51:25.595 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 8798 2024-04-07T17:51:25.595 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.595 INFO:tasks.workunit.client.0.smithi100.stdout: "commit_bytes": { 2024-04-07T17:51:25.595 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:25.595 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.595 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.595 INFO:tasks.workunit.client.0.smithi100.stdout: "commit_keys": { 2024-04-07T17:51:25.595 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:25.595 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.596 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.596 INFO:tasks.workunit.client.0.smithi100.stdout: "commit_latency": { 2024-04-07T17:51:25.596 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:25.596 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.596 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.596 INFO:tasks.workunit.client.0.smithi100.stdout: "lease_ack_timeout": { 2024-04-07T17:51:25.596 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.596 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.596 INFO:tasks.workunit.client.0.smithi100.stdout: "lease_timeout": { 2024-04-07T17:51:25.596 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.596 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.596 INFO:tasks.workunit.client.0.smithi100.stdout: "new_pn": { 2024-04-07T17:51:25.596 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.596 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.596 INFO:tasks.workunit.client.0.smithi100.stdout: "new_pn_latency": { 2024-04-07T17:51:25.596 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:25.597 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.597 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.597 INFO:tasks.workunit.client.0.smithi100.stdout: "refresh": { 2024-04-07T17:51:25.597 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 8798 2024-04-07T17:51:25.597 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.597 INFO:tasks.workunit.client.0.smithi100.stdout: "refresh_latency": { 2024-04-07T17:51:25.597 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 8798, 2024-04-07T17:51:25.597 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 52838506301 2024-04-07T17:51:25.597 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.597 INFO:tasks.workunit.client.0.smithi100.stdout: "restart": { 2024-04-07T17:51:25.597 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 14 2024-04-07T17:51:25.597 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.597 INFO:tasks.workunit.client.0.smithi100.stdout: "share_state": { 2024-04-07T17:51:25.597 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.597 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.598 INFO:tasks.workunit.client.0.smithi100.stdout: "share_state_bytes": { 2024-04-07T17:51:25.598 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:25.598 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.598 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.598 INFO:tasks.workunit.client.0.smithi100.stdout: "share_state_keys": { 2024-04-07T17:51:25.598 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:25.598 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.598 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.598 INFO:tasks.workunit.client.0.smithi100.stdout: "start_leader": { 2024-04-07T17:51:25.598 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.598 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.598 INFO:tasks.workunit.client.0.smithi100.stdout: "start_peon": { 2024-04-07T17:51:25.598 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3 2024-04-07T17:51:25.598 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.599 INFO:tasks.workunit.client.0.smithi100.stdout: "store_state": { 2024-04-07T17:51:25.599 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 8797 2024-04-07T17:51:25.599 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.599 INFO:tasks.workunit.client.0.smithi100.stdout: "store_state_bytes": { 2024-04-07T17:51:25.599 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 8797, 2024-04-07T17:51:25.599 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 116970414 2024-04-07T17:51:25.599 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.599 INFO:tasks.workunit.client.0.smithi100.stdout: "store_state_keys": { 2024-04-07T17:51:25.599 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 8797, 2024-04-07T17:51:25.599 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 76589 2024-04-07T17:51:25.599 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.599 INFO:tasks.workunit.client.0.smithi100.stdout: "store_state_latency": { 2024-04-07T17:51:25.599 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 8797, 2024-04-07T17:51:25.599 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5995831139 2024-04-07T17:51:25.599 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.600 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.600 INFO:tasks.workunit.client.0.smithi100.stdout: "prioritycache": { 2024-04-07T17:51:25.600 INFO:tasks.workunit.client.0.smithi100.stdout: "cache_bytes": { 2024-04-07T17:51:25.600 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1020054731 2024-04-07T17:51:25.600 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.600 INFO:tasks.workunit.client.0.smithi100.stdout: "heap_bytes": { 2024-04-07T17:51:25.600 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 166682624 2024-04-07T17:51:25.600 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.600 INFO:tasks.workunit.client.0.smithi100.stdout: "mapped_bytes": { 2024-04-07T17:51:25.600 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 165036032 2024-04-07T17:51:25.600 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.600 INFO:tasks.workunit.client.0.smithi100.stdout: "target_bytes": { 2024-04-07T17:51:25.600 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2147483648 2024-04-07T17:51:25.600 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.600 INFO:tasks.workunit.client.0.smithi100.stdout: "unmapped_bytes": { 2024-04-07T17:51:25.600 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1646592 2024-04-07T17:51:25.601 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.601 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.601 INFO:tasks.workunit.client.0.smithi100.stdout: "prioritycache:full": { 2024-04-07T17:51:25.601 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:25.601 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 360710144 2024-04-07T17:51:25.601 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.601 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:25.601 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.601 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.601 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:25.601 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.601 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.601 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:25.601 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 267782252 2024-04-07T17:51:25.601 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.601 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:25.601 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 23512176 2024-04-07T17:51:25.602 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.602 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:25.602 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.602 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.602 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:25.602 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.602 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.602 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:25.602 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.602 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.602 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:25.602 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.602 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.602 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:25.602 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.603 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.603 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:25.603 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.603 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.603 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:25.603 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.603 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.603 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:25.603 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.603 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.603 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:25.603 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 69415716 2024-04-07T17:51:25.603 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.603 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.603 INFO:tasks.workunit.client.0.smithi100.stdout: "prioritycache:inc": { 2024-04-07T17:51:25.603 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:25.603 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 339738624 2024-04-07T17:51:25.604 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.604 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:25.604 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.604 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.604 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:25.604 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.604 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.604 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:25.604 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 267782252 2024-04-07T17:51:25.604 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.604 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:25.604 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1438180 2024-04-07T17:51:25.604 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.604 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:25.604 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.604 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.604 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:25.604 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.605 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.605 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:25.605 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.605 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.605 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:25.605 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.605 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.605 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:25.605 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.605 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.605 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:25.605 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.605 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.605 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:25.605 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.605 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.606 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:25.606 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.606 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.606 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:25.606 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 70518192 2024-04-07T17:51:25.606 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.606 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.606 INFO:tasks.workunit.client.0.smithi100.stdout: "prioritycache:kv": { 2024-04-07T17:51:25.606 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:25.606 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 314572800 2024-04-07T17:51:25.606 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.606 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:25.606 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 67108864 2024-04-07T17:51:25.606 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.606 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:25.607 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.607 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.607 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:25.607 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 178521501 2024-04-07T17:51:25.607 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.607 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:25.607 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.607 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.607 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:25.607 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.607 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.607 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:25.607 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.607 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.607 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:25.607 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.608 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.608 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:25.608 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.608 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.608 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:25.608 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.608 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.608 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:25.608 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.608 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.608 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:25.608 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.608 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.608 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:25.608 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.608 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.608 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:25.609 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 68942435 2024-04-07T17:51:25.609 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.609 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.609 INFO:tasks.workunit.client.0.smithi100.stdout: "rocksdb": { 2024-04-07T17:51:25.609 INFO:tasks.workunit.client.0.smithi100.stdout: "compact": { 2024-04-07T17:51:25.609 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.609 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.609 INFO:tasks.workunit.client.0.smithi100.stdout: "compact_queue_len": { 2024-04-07T17:51:25.609 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.609 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.609 INFO:tasks.workunit.client.0.smithi100.stdout: "compact_queue_merge": { 2024-04-07T17:51:25.609 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.609 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.609 INFO:tasks.workunit.client.0.smithi100.stdout: "compact_range": { 2024-04-07T17:51:25.609 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 105 2024-04-07T17:51:25.609 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.609 INFO:tasks.workunit.client.0.smithi100.stdout: "get_latency": { 2024-04-07T17:51:25.610 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 380867, 2024-04-07T17:51:25.610 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4241508762 2024-04-07T17:51:25.610 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.610 INFO:tasks.workunit.client.0.smithi100.stdout: "rocksdb_write_delay_time": { 2024-04-07T17:51:25.610 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:25.610 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.610 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.610 INFO:tasks.workunit.client.0.smithi100.stdout: "rocksdb_write_memtable_time": { 2024-04-07T17:51:25.610 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:25.610 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.610 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.610 INFO:tasks.workunit.client.0.smithi100.stdout: "rocksdb_write_pre_and_post_time": { 2024-04-07T17:51:25.610 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:25.610 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.610 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.611 INFO:tasks.workunit.client.0.smithi100.stdout: "rocksdb_write_wal_time": { 2024-04-07T17:51:25.611 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:25.611 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.611 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.611 INFO:tasks.workunit.client.0.smithi100.stdout: "submit_latency": { 2024-04-07T17:51:25.611 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:25.611 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.611 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.611 INFO:tasks.workunit.client.0.smithi100.stdout: "submit_sync_latency": { 2024-04-07T17:51:25.611 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 21106, 2024-04-07T17:51:25.611 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 10897184086 2024-04-07T17:51:25.611 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.611 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.611 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.611 INFO:tasks.workunit.client.0.smithi100.stdout: "mon.a52fc0d8acb515431879057649ff294cfe14c620": { 2024-04-07T17:51:25.611 INFO:tasks.workunit.client.0.smithi100.stdout: "mon": { 2024-04-07T17:51:25.611 INFO:tasks.workunit.client.0.smithi100.stdout: "election_call": { 2024-04-07T17:51:25.612 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2 2024-04-07T17:51:25.612 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.612 INFO:tasks.workunit.client.0.smithi100.stdout: "election_lose": { 2024-04-07T17:51:25.612 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2 2024-04-07T17:51:25.612 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.612 INFO:tasks.workunit.client.0.smithi100.stdout: "election_win": { 2024-04-07T17:51:25.612 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.612 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.612 INFO:tasks.workunit.client.0.smithi100.stdout: "num_elections": { 2024-04-07T17:51:25.612 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 7 2024-04-07T17:51:25.612 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.612 INFO:tasks.workunit.client.0.smithi100.stdout: "num_sessions": { 2024-04-07T17:51:25.612 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 8 2024-04-07T17:51:25.612 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.612 INFO:tasks.workunit.client.0.smithi100.stdout: "session_add": { 2024-04-07T17:51:25.612 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3 2024-04-07T17:51:25.613 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.613 INFO:tasks.workunit.client.0.smithi100.stdout: "session_rm": { 2024-04-07T17:51:25.613 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 245 2024-04-07T17:51:25.613 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.613 INFO:tasks.workunit.client.0.smithi100.stdout: "session_trim": { 2024-04-07T17:51:25.613 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.613 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.613 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.613 INFO:tasks.workunit.client.0.smithi100.stdout: "paxos": { 2024-04-07T17:51:25.613 INFO:tasks.workunit.client.0.smithi100.stdout: "accept_timeout": { 2024-04-07T17:51:25.613 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.613 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.613 INFO:tasks.workunit.client.0.smithi100.stdout: "begin": { 2024-04-07T17:51:25.613 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 8738 2024-04-07T17:51:25.613 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.613 INFO:tasks.workunit.client.0.smithi100.stdout: "begin_bytes": { 2024-04-07T17:51:25.614 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 8738, 2024-04-07T17:51:25.614 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 58850172 2024-04-07T17:51:25.614 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.614 INFO:tasks.workunit.client.0.smithi100.stdout: "begin_keys": { 2024-04-07T17:51:25.614 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:25.614 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.614 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.614 INFO:tasks.workunit.client.0.smithi100.stdout: "begin_latency": { 2024-04-07T17:51:25.614 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 8738, 2024-04-07T17:51:25.614 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2543954542 2024-04-07T17:51:25.614 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.614 INFO:tasks.workunit.client.0.smithi100.stdout: "collect": { 2024-04-07T17:51:25.614 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2 2024-04-07T17:51:25.614 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.614 INFO:tasks.workunit.client.0.smithi100.stdout: "collect_bytes": { 2024-04-07T17:51:25.615 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 2, 2024-04-07T17:51:25.615 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 146 2024-04-07T17:51:25.615 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.615 INFO:tasks.workunit.client.0.smithi100.stdout: "collect_keys": { 2024-04-07T17:51:25.615 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 2, 2024-04-07T17:51:25.615 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2 2024-04-07T17:51:25.615 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.615 INFO:tasks.workunit.client.0.smithi100.stdout: "collect_latency": { 2024-04-07T17:51:25.615 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 2, 2024-04-07T17:51:25.615 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3999909 2024-04-07T17:51:25.615 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.615 INFO:tasks.workunit.client.0.smithi100.stdout: "collect_timeout": { 2024-04-07T17:51:25.615 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.615 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.615 INFO:tasks.workunit.client.0.smithi100.stdout: "collect_uncommitted": { 2024-04-07T17:51:25.616 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.616 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.616 INFO:tasks.workunit.client.0.smithi100.stdout: "commit": { 2024-04-07T17:51:25.616 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 8738 2024-04-07T17:51:25.616 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.616 INFO:tasks.workunit.client.0.smithi100.stdout: "commit_bytes": { 2024-04-07T17:51:25.616 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:25.616 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.616 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.616 INFO:tasks.workunit.client.0.smithi100.stdout: "commit_keys": { 2024-04-07T17:51:25.616 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:25.616 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.616 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.616 INFO:tasks.workunit.client.0.smithi100.stdout: "commit_latency": { 2024-04-07T17:51:25.616 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:25.617 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.617 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.617 INFO:tasks.workunit.client.0.smithi100.stdout: "lease_ack_timeout": { 2024-04-07T17:51:25.617 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.617 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.617 INFO:tasks.workunit.client.0.smithi100.stdout: "lease_timeout": { 2024-04-07T17:51:25.617 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.617 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.617 INFO:tasks.workunit.client.0.smithi100.stdout: "new_pn": { 2024-04-07T17:51:25.617 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.617 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.617 INFO:tasks.workunit.client.0.smithi100.stdout: "new_pn_latency": { 2024-04-07T17:51:25.617 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:25.617 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.617 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.617 INFO:tasks.workunit.client.0.smithi100.stdout: "refresh": { 2024-04-07T17:51:25.617 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 8738 2024-04-07T17:51:25.618 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.618 INFO:tasks.workunit.client.0.smithi100.stdout: "refresh_latency": { 2024-04-07T17:51:25.618 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 8738, 2024-04-07T17:51:25.618 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 48943127324 2024-04-07T17:51:25.618 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.618 INFO:tasks.workunit.client.0.smithi100.stdout: "restart": { 2024-04-07T17:51:25.618 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 9 2024-04-07T17:51:25.618 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.618 INFO:tasks.workunit.client.0.smithi100.stdout: "share_state": { 2024-04-07T17:51:25.618 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.618 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.618 INFO:tasks.workunit.client.0.smithi100.stdout: "share_state_bytes": { 2024-04-07T17:51:25.618 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:25.618 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.618 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.619 INFO:tasks.workunit.client.0.smithi100.stdout: "share_state_keys": { 2024-04-07T17:51:25.619 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:25.619 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.619 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.619 INFO:tasks.workunit.client.0.smithi100.stdout: "start_leader": { 2024-04-07T17:51:25.619 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.619 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.619 INFO:tasks.workunit.client.0.smithi100.stdout: "start_peon": { 2024-04-07T17:51:25.619 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2 2024-04-07T17:51:25.619 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.619 INFO:tasks.workunit.client.0.smithi100.stdout: "store_state": { 2024-04-07T17:51:25.619 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 8738 2024-04-07T17:51:25.619 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.619 INFO:tasks.workunit.client.0.smithi100.stdout: "store_state_bytes": { 2024-04-07T17:51:25.619 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 8738, 2024-04-07T17:51:25.620 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 116035468 2024-04-07T17:51:25.620 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.620 INFO:tasks.workunit.client.0.smithi100.stdout: "store_state_keys": { 2024-04-07T17:51:25.620 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 8738, 2024-04-07T17:51:25.620 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 75603 2024-04-07T17:51:25.620 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.620 INFO:tasks.workunit.client.0.smithi100.stdout: "store_state_latency": { 2024-04-07T17:51:25.620 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 8738, 2024-04-07T17:51:25.620 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3183943365 2024-04-07T17:51:25.620 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.620 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.620 INFO:tasks.workunit.client.0.smithi100.stdout: "prioritycache": { 2024-04-07T17:51:25.620 INFO:tasks.workunit.client.0.smithi100.stdout: "cache_bytes": { 2024-04-07T17:51:25.620 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1020054731 2024-04-07T17:51:25.621 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.621 INFO:tasks.workunit.client.0.smithi100.stdout: "heap_bytes": { 2024-04-07T17:51:25.621 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 164585472 2024-04-07T17:51:25.621 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.621 INFO:tasks.workunit.client.0.smithi100.stdout: "mapped_bytes": { 2024-04-07T17:51:25.621 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 162013184 2024-04-07T17:51:25.621 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.621 INFO:tasks.workunit.client.0.smithi100.stdout: "target_bytes": { 2024-04-07T17:51:25.621 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2147483648 2024-04-07T17:51:25.621 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.621 INFO:tasks.workunit.client.0.smithi100.stdout: "unmapped_bytes": { 2024-04-07T17:51:25.621 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2572288 2024-04-07T17:51:25.621 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.621 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.621 INFO:tasks.workunit.client.0.smithi100.stdout: "prioritycache:full": { 2024-04-07T17:51:25.621 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:25.621 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 360710144 2024-04-07T17:51:25.622 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.622 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:25.622 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.622 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.622 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:25.622 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.622 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.622 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:25.622 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 267855893 2024-04-07T17:51:25.622 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.622 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:25.622 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 23328861 2024-04-07T17:51:25.622 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.622 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:25.623 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.623 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.623 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:25.623 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.623 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.623 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:25.623 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.623 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.623 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:25.623 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.623 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.623 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:25.623 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.623 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.623 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:25.623 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.623 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.624 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:25.624 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.624 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.624 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:25.624 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.624 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.624 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:25.624 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 69525390 2024-04-07T17:51:25.624 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.624 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.624 INFO:tasks.workunit.client.0.smithi100.stdout: "prioritycache:inc": { 2024-04-07T17:51:25.624 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:25.624 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 339738624 2024-04-07T17:51:25.624 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.624 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:25.624 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.624 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.625 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:25.625 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.625 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.625 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:25.625 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 267855893 2024-04-07T17:51:25.625 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.625 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:25.625 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1425119 2024-04-07T17:51:25.625 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.625 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:25.625 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.625 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.625 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:25.625 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.625 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.625 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:25.626 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.626 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.626 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:25.626 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.626 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.626 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:25.626 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.626 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.626 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:25.626 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.626 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.626 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:25.626 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.627 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.627 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:25.627 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.627 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.627 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:25.627 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 70457612 2024-04-07T17:51:25.627 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.627 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.627 INFO:tasks.workunit.client.0.smithi100.stdout: "prioritycache:kv": { 2024-04-07T17:51:25.627 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:25.627 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 314572800 2024-04-07T17:51:25.627 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.627 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:25.627 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 67108864 2024-04-07T17:51:25.627 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.628 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:25.628 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.628 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.628 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:25.628 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 178570595 2024-04-07T17:51:25.628 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.628 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:25.628 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.628 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.628 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:25.628 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.628 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.628 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:25.628 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.628 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.629 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:25.629 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.629 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.629 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:25.629 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.629 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.629 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:25.629 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.629 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.629 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:25.629 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.629 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.629 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:25.629 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.629 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.630 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:25.630 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.630 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.630 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:25.630 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 68893341 2024-04-07T17:51:25.630 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.630 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.630 INFO:tasks.workunit.client.0.smithi100.stdout: "rocksdb": { 2024-04-07T17:51:25.630 INFO:tasks.workunit.client.0.smithi100.stdout: "compact": { 2024-04-07T17:51:25.630 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.630 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.630 INFO:tasks.workunit.client.0.smithi100.stdout: "compact_queue_len": { 2024-04-07T17:51:25.630 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.630 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.631 INFO:tasks.workunit.client.0.smithi100.stdout: "compact_queue_merge": { 2024-04-07T17:51:25.631 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.631 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.631 INFO:tasks.workunit.client.0.smithi100.stdout: "compact_range": { 2024-04-07T17:51:25.631 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 101 2024-04-07T17:51:25.631 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.631 INFO:tasks.workunit.client.0.smithi100.stdout: "get_latency": { 2024-04-07T17:51:25.631 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 378157, 2024-04-07T17:51:25.631 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3964885955 2024-04-07T17:51:25.631 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.631 INFO:tasks.workunit.client.0.smithi100.stdout: "rocksdb_write_delay_time": { 2024-04-07T17:51:25.631 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:25.631 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.631 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.631 INFO:tasks.workunit.client.0.smithi100.stdout: "rocksdb_write_memtable_time": { 2024-04-07T17:51:25.631 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:25.632 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.632 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.632 INFO:tasks.workunit.client.0.smithi100.stdout: "rocksdb_write_pre_and_post_time": { 2024-04-07T17:51:25.632 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:25.632 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.632 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.632 INFO:tasks.workunit.client.0.smithi100.stdout: "rocksdb_write_wal_time": { 2024-04-07T17:51:25.632 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:25.632 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.632 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.632 INFO:tasks.workunit.client.0.smithi100.stdout: "submit_latency": { 2024-04-07T17:51:25.632 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:25.632 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.632 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.632 INFO:tasks.workunit.client.0.smithi100.stdout: "submit_sync_latency": { 2024-04-07T17:51:25.633 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 20955, 2024-04-07T17:51:25.633 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 8051722523 2024-04-07T17:51:25.633 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.633 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.633 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.633 INFO:tasks.workunit.client.0.smithi100.stdout: "mon.f47a4ff15a45950a63160265564a123fc5da5e5e": { 2024-04-07T17:51:25.633 INFO:tasks.workunit.client.0.smithi100.stdout: "mon": { 2024-04-07T17:51:25.633 INFO:tasks.workunit.client.0.smithi100.stdout: "election_call": { 2024-04-07T17:51:25.633 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5 2024-04-07T17:51:25.633 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.633 INFO:tasks.workunit.client.0.smithi100.stdout: "election_lose": { 2024-04-07T17:51:25.633 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.633 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.633 INFO:tasks.workunit.client.0.smithi100.stdout: "election_win": { 2024-04-07T17:51:25.633 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5 2024-04-07T17:51:25.634 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.634 INFO:tasks.workunit.client.0.smithi100.stdout: "num_elections": { 2024-04-07T17:51:25.634 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 17 2024-04-07T17:51:25.634 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.634 INFO:tasks.workunit.client.0.smithi100.stdout: "num_sessions": { 2024-04-07T17:51:25.634 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 11 2024-04-07T17:51:25.634 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.634 INFO:tasks.workunit.client.0.smithi100.stdout: "session_add": { 2024-04-07T17:51:25.634 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5 2024-04-07T17:51:25.634 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.634 INFO:tasks.workunit.client.0.smithi100.stdout: "session_rm": { 2024-04-07T17:51:25.634 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 278 2024-04-07T17:51:25.634 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.634 INFO:tasks.workunit.client.0.smithi100.stdout: "session_trim": { 2024-04-07T17:51:25.634 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.635 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.635 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.635 INFO:tasks.workunit.client.0.smithi100.stdout: "paxos": { 2024-04-07T17:51:25.635 INFO:tasks.workunit.client.0.smithi100.stdout: "accept_timeout": { 2024-04-07T17:51:25.635 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.635 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.635 INFO:tasks.workunit.client.0.smithi100.stdout: "begin": { 2024-04-07T17:51:25.635 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 8891 2024-04-07T17:51:25.635 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.635 INFO:tasks.workunit.client.0.smithi100.stdout: "begin_bytes": { 2024-04-07T17:51:25.635 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 8891, 2024-04-07T17:51:25.635 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 60665253 2024-04-07T17:51:25.635 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.635 INFO:tasks.workunit.client.0.smithi100.stdout: "begin_keys": { 2024-04-07T17:51:25.635 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 8891, 2024-04-07T17:51:25.636 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 26673 2024-04-07T17:51:25.636 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.636 INFO:tasks.workunit.client.0.smithi100.stdout: "begin_latency": { 2024-04-07T17:51:25.636 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 8891, 2024-04-07T17:51:25.636 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2895918242 2024-04-07T17:51:25.636 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.636 INFO:tasks.workunit.client.0.smithi100.stdout: "collect": { 2024-04-07T17:51:25.636 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.636 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.636 INFO:tasks.workunit.client.0.smithi100.stdout: "collect_bytes": { 2024-04-07T17:51:25.636 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:25.636 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.636 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.636 INFO:tasks.workunit.client.0.smithi100.stdout: "collect_keys": { 2024-04-07T17:51:25.636 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:25.637 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.637 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.637 INFO:tasks.workunit.client.0.smithi100.stdout: "collect_latency": { 2024-04-07T17:51:25.637 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:25.637 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.637 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.637 INFO:tasks.workunit.client.0.smithi100.stdout: "collect_timeout": { 2024-04-07T17:51:25.637 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.637 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.637 INFO:tasks.workunit.client.0.smithi100.stdout: "collect_uncommitted": { 2024-04-07T17:51:25.637 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2 2024-04-07T17:51:25.637 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.637 INFO:tasks.workunit.client.0.smithi100.stdout: "commit": { 2024-04-07T17:51:25.637 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 8889 2024-04-07T17:51:25.637 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.637 INFO:tasks.workunit.client.0.smithi100.stdout: "commit_bytes": { 2024-04-07T17:51:25.637 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 8889, 2024-04-07T17:51:25.638 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 59997923 2024-04-07T17:51:25.638 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.638 INFO:tasks.workunit.client.0.smithi100.stdout: "commit_keys": { 2024-04-07T17:51:25.638 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 8889, 2024-04-07T17:51:25.638 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 68421 2024-04-07T17:51:25.638 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.638 INFO:tasks.workunit.client.0.smithi100.stdout: "commit_latency": { 2024-04-07T17:51:25.638 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 8889, 2024-04-07T17:51:25.638 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3946227492 2024-04-07T17:51:25.638 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.638 INFO:tasks.workunit.client.0.smithi100.stdout: "lease_ack_timeout": { 2024-04-07T17:51:25.638 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.638 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.638 INFO:tasks.workunit.client.0.smithi100.stdout: "lease_timeout": { 2024-04-07T17:51:25.639 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.639 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.639 INFO:tasks.workunit.client.0.smithi100.stdout: "new_pn": { 2024-04-07T17:51:25.639 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5 2024-04-07T17:51:25.639 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.639 INFO:tasks.workunit.client.0.smithi100.stdout: "new_pn_latency": { 2024-04-07T17:51:25.639 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 5, 2024-04-07T17:51:25.639 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.639 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.639 INFO:tasks.workunit.client.0.smithi100.stdout: "refresh": { 2024-04-07T17:51:25.639 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 8892 2024-04-07T17:51:25.639 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.639 INFO:tasks.workunit.client.0.smithi100.stdout: "refresh_latency": { 2024-04-07T17:51:25.639 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 8892, 2024-04-07T17:51:25.639 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 56214415654 2024-04-07T17:51:25.640 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.640 INFO:tasks.workunit.client.0.smithi100.stdout: "restart": { 2024-04-07T17:51:25.640 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 19 2024-04-07T17:51:25.640 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.640 INFO:tasks.workunit.client.0.smithi100.stdout: "share_state": { 2024-04-07T17:51:25.640 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2 2024-04-07T17:51:25.640 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.640 INFO:tasks.workunit.client.0.smithi100.stdout: "share_state_bytes": { 2024-04-07T17:51:25.640 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 2, 2024-04-07T17:51:25.640 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 129358 2024-04-07T17:51:25.640 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.640 INFO:tasks.workunit.client.0.smithi100.stdout: "share_state_keys": { 2024-04-07T17:51:25.640 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 2, 2024-04-07T17:51:25.640 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 10 2024-04-07T17:51:25.640 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.640 INFO:tasks.workunit.client.0.smithi100.stdout: "start_leader": { 2024-04-07T17:51:25.641 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5 2024-04-07T17:51:25.641 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.641 INFO:tasks.workunit.client.0.smithi100.stdout: "start_peon": { 2024-04-07T17:51:25.641 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.641 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.641 INFO:tasks.workunit.client.0.smithi100.stdout: "store_state": { 2024-04-07T17:51:25.641 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.641 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.641 INFO:tasks.workunit.client.0.smithi100.stdout: "store_state_bytes": { 2024-04-07T17:51:25.641 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:25.641 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.641 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.641 INFO:tasks.workunit.client.0.smithi100.stdout: "store_state_keys": { 2024-04-07T17:51:25.641 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:25.641 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.641 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.641 INFO:tasks.workunit.client.0.smithi100.stdout: "store_state_latency": { 2024-04-07T17:51:25.642 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:25.642 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.642 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.642 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.642 INFO:tasks.workunit.client.0.smithi100.stdout: "prioritycache": { 2024-04-07T17:51:25.642 INFO:tasks.workunit.client.0.smithi100.stdout: "cache_bytes": { 2024-04-07T17:51:25.642 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1020054731 2024-04-07T17:51:25.642 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.642 INFO:tasks.workunit.client.0.smithi100.stdout: "heap_bytes": { 2024-04-07T17:51:25.642 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 258113536 2024-04-07T17:51:25.642 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.642 INFO:tasks.workunit.client.0.smithi100.stdout: "mapped_bytes": { 2024-04-07T17:51:25.642 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 257466368 2024-04-07T17:51:25.642 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.643 INFO:tasks.workunit.client.0.smithi100.stdout: "target_bytes": { 2024-04-07T17:51:25.643 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2147483648 2024-04-07T17:51:25.643 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.643 INFO:tasks.workunit.client.0.smithi100.stdout: "unmapped_bytes": { 2024-04-07T17:51:25.643 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 647168 2024-04-07T17:51:25.643 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.643 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.643 INFO:tasks.workunit.client.0.smithi100.stdout: "prioritycache:full": { 2024-04-07T17:51:25.643 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:25.643 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 360710144 2024-04-07T17:51:25.643 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.643 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:25.643 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.643 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.643 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:25.643 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.644 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.644 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:25.644 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 267670953 2024-04-07T17:51:25.644 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.644 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:25.644 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 23788534 2024-04-07T17:51:25.644 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.644 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:25.644 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.644 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.644 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:25.644 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.644 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.644 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:25.644 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.644 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.644 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:25.645 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.645 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.645 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:25.645 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.645 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.645 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:25.645 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.645 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.645 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:25.645 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.645 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.645 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:25.645 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.645 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.645 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:25.645 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 69250657 2024-04-07T17:51:25.645 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.646 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.646 INFO:tasks.workunit.client.0.smithi100.stdout: "prioritycache:inc": { 2024-04-07T17:51:25.646 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:25.646 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 339738624 2024-04-07T17:51:25.646 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.646 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:25.646 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.646 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.646 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:25.646 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.646 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.646 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:25.646 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 267670953 2024-04-07T17:51:25.646 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.646 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:25.647 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1458619 2024-04-07T17:51:25.647 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.647 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:25.647 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.647 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.647 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:25.647 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.647 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.647 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:25.647 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.647 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.647 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:25.647 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.647 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.647 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:25.647 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.647 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.648 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:25.648 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.648 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.648 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:25.648 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.648 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.648 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:25.648 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.648 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.648 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:25.648 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 70609052 2024-04-07T17:51:25.648 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.648 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.648 INFO:tasks.workunit.client.0.smithi100.stdout: "prioritycache:kv": { 2024-04-07T17:51:25.648 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:25.648 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 314572800 2024-04-07T17:51:25.648 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.649 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:25.649 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 67108864 2024-04-07T17:51:25.649 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.649 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:25.649 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.649 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.649 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:25.649 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 178447302 2024-04-07T17:51:25.649 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.649 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:25.649 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.649 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.649 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:25.649 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.649 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.649 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:25.649 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.650 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.650 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:25.650 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.650 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.650 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:25.650 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.650 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.650 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:25.650 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.650 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.650 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:25.650 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.650 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.650 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:25.650 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.651 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.651 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:25.651 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.651 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.651 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:25.651 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 69016634 2024-04-07T17:51:25.651 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.651 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.651 INFO:tasks.workunit.client.0.smithi100.stdout: "rocksdb": { 2024-04-07T17:51:25.651 INFO:tasks.workunit.client.0.smithi100.stdout: "compact": { 2024-04-07T17:51:25.651 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.651 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.651 INFO:tasks.workunit.client.0.smithi100.stdout: "compact_queue_len": { 2024-04-07T17:51:25.651 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.651 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.651 INFO:tasks.workunit.client.0.smithi100.stdout: "compact_queue_merge": { 2024-04-07T17:51:25.651 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.652 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.652 INFO:tasks.workunit.client.0.smithi100.stdout: "compact_range": { 2024-04-07T17:51:25.652 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 105 2024-04-07T17:51:25.652 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.652 INFO:tasks.workunit.client.0.smithi100.stdout: "get_latency": { 2024-04-07T17:51:25.652 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 398669, 2024-04-07T17:51:25.652 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4722518373 2024-04-07T17:51:25.652 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.652 INFO:tasks.workunit.client.0.smithi100.stdout: "rocksdb_write_delay_time": { 2024-04-07T17:51:25.652 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:25.652 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.652 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.652 INFO:tasks.workunit.client.0.smithi100.stdout: "rocksdb_write_memtable_time": { 2024-04-07T17:51:25.652 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:25.652 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.652 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.652 INFO:tasks.workunit.client.0.smithi100.stdout: "rocksdb_write_pre_and_post_time": { 2024-04-07T17:51:25.652 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:25.653 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.653 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.653 INFO:tasks.workunit.client.0.smithi100.stdout: "rocksdb_write_wal_time": { 2024-04-07T17:51:25.653 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:25.653 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.653 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.653 INFO:tasks.workunit.client.0.smithi100.stdout: "submit_latency": { 2024-04-07T17:51:25.653 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:25.653 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.653 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.653 INFO:tasks.workunit.client.0.smithi100.stdout: "submit_sync_latency": { 2024-04-07T17:51:25.653 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 21335, 2024-04-07T17:51:25.653 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 8939358365 2024-04-07T17:51:25.653 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.653 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.653 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.653 INFO:tasks.workunit.client.0.smithi100.stdout: "osd.0": { 2024-04-07T17:51:25.654 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs": { 2024-04-07T17:51:25.654 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_db_max_lat": { 2024-04-07T17:51:25.654 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 21463 2024-04-07T17:51:25.654 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.654 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_slow_fallback": { 2024-04-07T17:51:25.654 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.654 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.654 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_slow_max_lat": { 2024-04-07T17:51:25.654 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.654 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.654 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_slow_size_fallback": { 2024-04-07T17:51:25.654 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.654 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.654 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit_db": { 2024-04-07T17:51:25.654 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 65536 2024-04-07T17:51:25.655 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.655 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit_slow": { 2024-04-07T17:51:25.655 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.655 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.655 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit_wal": { 2024-04-07T17:51:25.655 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.655 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.655 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_wal_max_lat": { 2024-04-07T17:51:25.655 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.655 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.655 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_written_slow": { 2024-04-07T17:51:25.655 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.655 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.655 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_written_sst": { 2024-04-07T17:51:25.655 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 14692352 2024-04-07T17:51:25.656 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.656 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_written_wal": { 2024-04-07T17:51:25.656 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 742674432 2024-04-07T17:51:25.656 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.656 INFO:tasks.workunit.client.0.smithi100.stdout: "compact_lat": { 2024-04-07T17:51:25.656 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 19, 2024-04-07T17:51:25.656 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 7885793 2024-04-07T17:51:25.656 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.656 INFO:tasks.workunit.client.0.smithi100.stdout: "compact_lock_lat": { 2024-04-07T17:51:25.656 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 19, 2024-04-07T17:51:25.656 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2646042 2024-04-07T17:51:25.656 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.656 INFO:tasks.workunit.client.0.smithi100.stdout: "db_alloc_lat": { 2024-04-07T17:51:25.656 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 211, 2024-04-07T17:51:25.656 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1477946 2024-04-07T17:51:25.656 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.657 INFO:tasks.workunit.client.0.smithi100.stdout: "db_total_bytes": { 2024-04-07T17:51:25.657 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 95995027456 2024-04-07T17:51:25.657 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.657 INFO:tasks.workunit.client.0.smithi100.stdout: "db_used_bytes": { 2024-04-07T17:51:25.657 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 735182848 2024-04-07T17:51:25.657 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.657 INFO:tasks.workunit.client.0.smithi100.stdout: "flush_lat": { 2024-04-07T17:51:25.657 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 79477, 2024-04-07T17:51:25.657 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 14955834078 2024-04-07T17:51:25.657 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.657 INFO:tasks.workunit.client.0.smithi100.stdout: "fsync_lat": { 2024-04-07T17:51:25.657 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 79485, 2024-04-07T17:51:25.657 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 25864611168 2024-04-07T17:51:25.657 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.657 INFO:tasks.workunit.client.0.smithi100.stdout: "log_bytes": { 2024-04-07T17:51:25.657 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 9666560 2024-04-07T17:51:25.658 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.658 INFO:tasks.workunit.client.0.smithi100.stdout: "logged_bytes": { 2024-04-07T17:51:25.658 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 325562368 2024-04-07T17:51:25.658 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.658 INFO:tasks.workunit.client.0.smithi100.stdout: "max_bytes_db": { 2024-04-07T17:51:25.658 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 747962368 2024-04-07T17:51:25.658 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.658 INFO:tasks.workunit.client.0.smithi100.stdout: "max_bytes_slow": { 2024-04-07T17:51:25.658 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.658 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.658 INFO:tasks.workunit.client.0.smithi100.stdout: "max_bytes_wal": { 2024-04-07T17:51:25.658 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.658 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.658 INFO:tasks.workunit.client.0.smithi100.stdout: "num_files": { 2024-04-07T17:51:25.658 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 55 2024-04-07T17:51:25.659 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.659 INFO:tasks.workunit.client.0.smithi100.stdout: "read_bytes": { 2024-04-07T17:51:25.659 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 13112107291 2024-04-07T17:51:25.659 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.659 INFO:tasks.workunit.client.0.smithi100.stdout: "read_count": { 2024-04-07T17:51:25.659 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 86948 2024-04-07T17:51:25.659 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.659 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes": { 2024-04-07T17:51:25.659 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 11361755136 2024-04-07T17:51:25.659 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.659 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes_db": { 2024-04-07T17:51:25.659 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 11361759232 2024-04-07T17:51:25.659 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.659 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes_slow": { 2024-04-07T17:51:25.659 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.660 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.660 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes_wal": { 2024-04-07T17:51:25.660 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.660 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.660 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_count": { 2024-04-07T17:51:25.660 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 88640 2024-04-07T17:51:25.660 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.660 INFO:tasks.workunit.client.0.smithi100.stdout: "read_lat": { 2024-04-07T17:51:25.660 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 86948, 2024-04-07T17:51:25.660 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5555540195 2024-04-07T17:51:25.660 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.660 INFO:tasks.workunit.client.0.smithi100.stdout: "read_prefetch_bytes": { 2024-04-07T17:51:25.660 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 12971063490 2024-04-07T17:51:25.660 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.660 INFO:tasks.workunit.client.0.smithi100.stdout: "read_prefetch_count": { 2024-04-07T17:51:25.660 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 82627 2024-04-07T17:51:25.661 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.661 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_buffer_bytes": { 2024-04-07T17:51:25.661 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5154118 2024-04-07T17:51:25.661 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.661 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_buffer_count": { 2024-04-07T17:51:25.661 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 723 2024-04-07T17:51:25.661 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.661 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_bytes": { 2024-04-07T17:51:25.661 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 14619250 2024-04-07T17:51:25.661 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.661 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_count": { 2024-04-07T17:51:25.661 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1029 2024-04-07T17:51:25.661 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.661 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes": { 2024-04-07T17:51:25.661 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 9465132 2024-04-07T17:51:25.662 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.662 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes_db": { 2024-04-07T17:51:25.662 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 9465132 2024-04-07T17:51:25.662 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.662 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes_slow": { 2024-04-07T17:51:25.662 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.662 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.662 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes_wal": { 2024-04-07T17:51:25.662 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.662 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.662 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_count": { 2024-04-07T17:51:25.662 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 321 2024-04-07T17:51:25.662 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.662 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_lat": { 2024-04-07T17:51:25.663 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1029, 2024-04-07T17:51:25.663 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 7138854 2024-04-07T17:51:25.663 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.663 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_alloc_lat": { 2024-04-07T17:51:25.663 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:25.663 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.663 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.663 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_total_bytes": { 2024-04-07T17:51:25.663 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.663 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.663 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_used_bytes": { 2024-04-07T17:51:25.663 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.663 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.663 INFO:tasks.workunit.client.0.smithi100.stdout: "truncate_lat": { 2024-04-07T17:51:25.664 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1, 2024-04-07T17:51:25.664 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 593 2024-04-07T17:51:25.664 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.664 INFO:tasks.workunit.client.0.smithi100.stdout: "unlink_lat": { 2024-04-07T17:51:25.664 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 11, 2024-04-07T17:51:25.664 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 14997 2024-04-07T17:51:25.664 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.664 INFO:tasks.workunit.client.0.smithi100.stdout: "wal_alloc_lat": { 2024-04-07T17:51:25.664 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:25.664 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.664 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.664 INFO:tasks.workunit.client.0.smithi100.stdout: "wal_total_bytes": { 2024-04-07T17:51:25.664 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.664 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.664 INFO:tasks.workunit.client.0.smithi100.stdout: "wal_used_bytes": { 2024-04-07T17:51:25.664 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.665 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.665 INFO:tasks.workunit.client.0.smithi100.stdout: "write_bytes": { 2024-04-07T17:51:25.665 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1085816832 2024-04-07T17:51:25.665 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.665 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.665 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore": { 2024-04-07T17:51:25.665 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit": { 2024-04-07T17:51:25.665 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4096 2024-04-07T17:51:25.665 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.665 INFO:tasks.workunit.client.0.smithi100.stdout: "allocated": { 2024-04-07T17:51:25.665 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 10412032 2024-04-07T17:51:25.665 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.665 INFO:tasks.workunit.client.0.smithi100.stdout: "allocator_lat": { 2024-04-07T17:51:25.665 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 23588, 2024-04-07T17:51:25.665 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 122707682 2024-04-07T17:51:25.666 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.666 INFO:tasks.workunit.client.0.smithi100.stdout: "clist_lat": { 2024-04-07T17:51:25.666 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 3594, 2024-04-07T17:51:25.666 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 12445185285 2024-04-07T17:51:25.666 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.666 INFO:tasks.workunit.client.0.smithi100.stdout: "compress_lat": { 2024-04-07T17:51:25.666 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:25.666 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.666 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.666 INFO:tasks.workunit.client.0.smithi100.stdout: "compressed": { 2024-04-07T17:51:25.666 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.666 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.666 INFO:tasks.workunit.client.0.smithi100.stdout: "compressed_allocated": { 2024-04-07T17:51:25.666 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.666 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.667 INFO:tasks.workunit.client.0.smithi100.stdout: "compressed_original": { 2024-04-07T17:51:25.667 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.667 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.667 INFO:tasks.workunit.client.0.smithi100.stdout: "csum_lat": { 2024-04-07T17:51:25.667 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 29739, 2024-04-07T17:51:25.667 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 345936986 2024-04-07T17:51:25.667 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.667 INFO:tasks.workunit.client.0.smithi100.stdout: "decompress_lat": { 2024-04-07T17:51:25.667 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:25.667 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.667 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.667 INFO:tasks.workunit.client.0.smithi100.stdout: "fragmentation_micros": { 2024-04-07T17:51:25.667 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.667 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.667 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_commit_lat": { 2024-04-07T17:51:25.668 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 79403, 2024-04-07T17:51:25.668 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 33855770677 2024-04-07T17:51:25.668 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.668 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_final_lat": { 2024-04-07T17:51:25.668 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 78455, 2024-04-07T17:51:25.668 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3723540743 2024-04-07T17:51:25.668 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.668 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_flush_lat": { 2024-04-07T17:51:25.668 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 79403, 2024-04-07T17:51:25.668 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 104760188 2024-04-07T17:51:25.668 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.668 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_sync_lat": { 2024-04-07T17:51:25.668 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 79403, 2024-04-07T17:51:25.668 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 33960530865 2024-04-07T17:51:25.668 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.668 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_get_keys_lat": { 2024-04-07T17:51:25.669 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:25.669 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.669 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.669 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_get_values_lat": { 2024-04-07T17:51:25.669 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 35913, 2024-04-07T17:51:25.669 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1906667723 2024-04-07T17:51:25.669 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.669 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_lower_bound_lat": { 2024-04-07T17:51:25.669 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 941, 2024-04-07T17:51:25.669 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5755773 2024-04-07T17:51:25.669 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.669 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_next_lat": { 2024-04-07T17:51:25.669 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 24954, 2024-04-07T17:51:25.669 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 21515897 2024-04-07T17:51:25.669 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.669 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_seek_to_first_lat": { 2024-04-07T17:51:25.670 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 52, 2024-04-07T17:51:25.670 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 102213 2024-04-07T17:51:25.670 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.670 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_upper_bound_lat": { 2024-04-07T17:51:25.670 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 16667, 2024-04-07T17:51:25.670 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5746456893 2024-04-07T17:51:25.670 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.670 INFO:tasks.workunit.client.0.smithi100.stdout: "onode_hits": { 2024-04-07T17:51:25.670 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 260492 2024-04-07T17:51:25.670 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.670 INFO:tasks.workunit.client.0.smithi100.stdout: "onode_misses": { 2024-04-07T17:51:25.670 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 24792 2024-04-07T17:51:25.670 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.670 INFO:tasks.workunit.client.0.smithi100.stdout: "read_lat": { 2024-04-07T17:51:25.671 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 26984, 2024-04-07T17:51:25.671 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3801162649 2024-04-07T17:51:25.671 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.671 INFO:tasks.workunit.client.0.smithi100.stdout: "read_onode_meta_lat": { 2024-04-07T17:51:25.671 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 62413, 2024-04-07T17:51:25.671 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 46979091 2024-04-07T17:51:25.671 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.671 INFO:tasks.workunit.client.0.smithi100.stdout: "read_wait_aio_lat": { 2024-04-07T17:51:25.671 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 35429, 2024-04-07T17:51:25.671 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2438344202 2024-04-07T17:51:25.671 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.671 INFO:tasks.workunit.client.0.smithi100.stdout: "reads_with_retries": { 2024-04-07T17:51:25.671 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.671 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.671 INFO:tasks.workunit.client.0.smithi100.stdout: "remove_lat": { 2024-04-07T17:51:25.672 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 15389, 2024-04-07T17:51:25.672 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 819542013 2024-04-07T17:51:25.672 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.672 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_aio_wait_count": { 2024-04-07T17:51:25.672 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.672 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.672 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_committed_kv_count": { 2024-04-07T17:51:25.672 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.672 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.672 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_read_onode_meta_count": { 2024-04-07T17:51:25.672 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.672 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.672 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_read_wait_aio_count": { 2024-04-07T17:51:25.672 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.672 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.672 INFO:tasks.workunit.client.0.smithi100.stdout: "state_aio_wait_lat": { 2024-04-07T17:51:25.673 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 93393, 2024-04-07T17:51:25.673 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 12329349719 2024-04-07T17:51:25.673 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.673 INFO:tasks.workunit.client.0.smithi100.stdout: "state_deferred_aio_wait_lat": { 2024-04-07T17:51:25.673 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 9754, 2024-04-07T17:51:25.673 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1209751248 2024-04-07T17:51:25.673 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.673 INFO:tasks.workunit.client.0.smithi100.stdout: "state_deferred_cleanup_lat": { 2024-04-07T17:51:25.673 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 9753, 2024-04-07T17:51:25.673 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4213165502302 2024-04-07T17:51:25.673 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.673 INFO:tasks.workunit.client.0.smithi100.stdout: "state_deferred_queued_lat": { 2024-04-07T17:51:25.673 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 9754, 2024-04-07T17:51:25.673 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 6800803572839 2024-04-07T17:51:25.673 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.674 INFO:tasks.workunit.client.0.smithi100.stdout: "state_done_lat": { 2024-04-07T17:51:25.674 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 93392, 2024-04-07T17:51:25.674 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2914483870678 2024-04-07T17:51:25.674 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.674 INFO:tasks.workunit.client.0.smithi100.stdout: "state_finishing_lat": { 2024-04-07T17:51:25.674 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 93392, 2024-04-07T17:51:25.674 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 20544100 2024-04-07T17:51:25.674 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.674 INFO:tasks.workunit.client.0.smithi100.stdout: "state_io_done_lat": { 2024-04-07T17:51:25.674 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 93393, 2024-04-07T17:51:25.674 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 589384870 2024-04-07T17:51:25.674 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.674 INFO:tasks.workunit.client.0.smithi100.stdout: "state_kv_commiting_lat": { 2024-04-07T17:51:25.674 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 93393, 2024-04-07T17:51:25.674 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 60184889904 2024-04-07T17:51:25.675 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.675 INFO:tasks.workunit.client.0.smithi100.stdout: "state_kv_done_lat": { 2024-04-07T17:51:25.675 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 93393, 2024-04-07T17:51:25.675 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 69530872 2024-04-07T17:51:25.675 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.675 INFO:tasks.workunit.client.0.smithi100.stdout: "state_kv_queued_lat": { 2024-04-07T17:51:25.675 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 93393, 2024-04-07T17:51:25.675 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 31858207932 2024-04-07T17:51:25.675 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.675 INFO:tasks.workunit.client.0.smithi100.stdout: "state_prepare_lat": { 2024-04-07T17:51:25.675 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 93393, 2024-04-07T17:51:25.675 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 26654236951 2024-04-07T17:51:25.675 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.675 INFO:tasks.workunit.client.0.smithi100.stdout: "stored": { 2024-04-07T17:51:25.675 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 6290778 2024-04-07T17:51:25.676 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.676 INFO:tasks.workunit.client.0.smithi100.stdout: "truncate_lat": { 2024-04-07T17:51:25.676 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 2482, 2024-04-07T17:51:25.676 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 25806717 2024-04-07T17:51:25.676 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.676 INFO:tasks.workunit.client.0.smithi100.stdout: "txc_commit_lat": { 2024-04-07T17:51:25.676 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 93393, 2024-04-07T17:51:25.676 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 131646585667 2024-04-07T17:51:25.676 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.676 INFO:tasks.workunit.client.0.smithi100.stdout: "txc_submit_lat": { 2024-04-07T17:51:25.676 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 93393, 2024-04-07T17:51:25.676 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 29197915299 2024-04-07T17:51:25.676 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.676 INFO:tasks.workunit.client.0.smithi100.stdout: "txc_throttle_lat": { 2024-04-07T17:51:25.676 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 93393, 2024-04-07T17:51:25.676 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1435528401 2024-04-07T17:51:25.677 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.677 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.677 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache": { 2024-04-07T17:51:25.677 INFO:tasks.workunit.client.0.smithi100.stdout: "cache_bytes": { 2024-04-07T17:51:25.677 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1444614092 2024-04-07T17:51:25.677 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.677 INFO:tasks.workunit.client.0.smithi100.stdout: "heap_bytes": { 2024-04-07T17:51:25.677 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 508690432 2024-04-07T17:51:25.677 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.677 INFO:tasks.workunit.client.0.smithi100.stdout: "mapped_bytes": { 2024-04-07T17:51:25.677 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 469770240 2024-04-07T17:51:25.677 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.677 INFO:tasks.workunit.client.0.smithi100.stdout: "target_bytes": { 2024-04-07T17:51:25.677 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2646965248 2024-04-07T17:51:25.677 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.678 INFO:tasks.workunit.client.0.smithi100.stdout: "unmapped_bytes": { 2024-04-07T17:51:25.678 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 38920192 2024-04-07T17:51:25.678 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.678 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.678 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:data": { 2024-04-07T17:51:25.678 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:25.678 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 134217728 2024-04-07T17:51:25.678 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.678 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:25.678 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.678 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.678 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:25.678 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.678 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.678 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:25.679 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 58795597 2024-04-07T17:51:25.679 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.679 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:25.679 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.679 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.679 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:25.679 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.679 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.679 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:25.679 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.679 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.679 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:25.679 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.679 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.679 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:25.680 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4096 2024-04-07T17:51:25.680 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.680 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:25.680 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 438272 2024-04-07T17:51:25.680 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.680 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:25.680 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.680 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.680 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:25.680 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.680 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.680 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:25.680 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.680 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.680 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:25.680 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 74979763 2024-04-07T17:51:25.681 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.681 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.681 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:kv": { 2024-04-07T17:51:25.681 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:25.681 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 578813952 2024-04-07T17:51:25.681 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.681 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:25.681 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 67108864 2024-04-07T17:51:25.681 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.681 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:25.681 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.681 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.681 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:25.681 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 440966984 2024-04-07T17:51:25.681 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.681 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:25.682 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.682 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.682 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:25.682 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.682 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.682 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:25.682 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1776 2024-04-07T17:51:25.682 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.682 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:25.682 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.682 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.682 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:25.682 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 121696 2024-04-07T17:51:25.682 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.682 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:25.683 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.683 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.683 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:25.683 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.683 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.683 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:25.683 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.683 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.683 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:25.683 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.683 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.683 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:25.683 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 70614632 2024-04-07T17:51:25.683 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.683 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.684 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:kv_onode": { 2024-04-07T17:51:25.684 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:25.684 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 192937984 2024-04-07T17:51:25.684 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.684 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:25.684 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 67108864 2024-04-07T17:51:25.684 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.684 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:25.684 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 14485360 2024-04-07T17:51:25.684 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.684 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:25.684 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 39197065 2024-04-07T17:51:25.684 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.684 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:25.684 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.684 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.685 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:25.685 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.685 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.685 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:25.685 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.685 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.685 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:25.685 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.685 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.685 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:25.685 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.685 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.685 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:25.685 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.686 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.686 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:25.686 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.686 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.686 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:25.686 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.686 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.686 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:25.686 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.686 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.686 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:25.686 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 72146695 2024-04-07T17:51:25.686 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.686 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.687 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:meta": { 2024-04-07T17:51:25.687 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:25.687 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 511705088 2024-04-07T17:51:25.687 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.687 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:25.687 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.687 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.687 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:25.687 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.687 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.687 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:25.687 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 440966984 2024-04-07T17:51:25.687 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.687 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:25.687 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.688 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.688 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:25.688 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.688 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.688 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:25.688 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.688 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.688 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:25.688 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.688 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.688 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:25.688 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 157581 2024-04-07T17:51:25.688 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.688 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:25.688 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1088294 2024-04-07T17:51:25.688 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.689 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:25.689 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.689 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.689 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:25.689 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.689 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.689 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:25.689 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.689 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.689 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:25.689 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 69492229 2024-04-07T17:51:25.689 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.689 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.689 INFO:tasks.workunit.client.0.smithi100.stdout: "objecter": { 2024-04-07T17:51:25.689 INFO:tasks.workunit.client.0.smithi100.stdout: "op_active": { 2024-04-07T17:51:25.690 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.690 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.690 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r": { 2024-04-07T17:51:25.690 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 56 2024-04-07T17:51:25.690 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.690 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rmw": { 2024-04-07T17:51:25.690 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.690 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.690 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w": { 2024-04-07T17:51:25.690 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.690 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.690 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.690 INFO:tasks.workunit.client.0.smithi100.stdout: "osd": { 2024-04-07T17:51:25.690 INFO:tasks.workunit.client.0.smithi100.stdout: "numpg": { 2024-04-07T17:51:25.690 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 55 2024-04-07T17:51:25.691 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.691 INFO:tasks.workunit.client.0.smithi100.stdout: "numpg_removing": { 2024-04-07T17:51:25.691 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.691 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.691 INFO:tasks.workunit.client.0.smithi100.stdout: "op": { 2024-04-07T17:51:25.691 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 53262 2024-04-07T17:51:25.691 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.691 INFO:tasks.workunit.client.0.smithi100.stdout: "op_before_queue_op_lat": { 2024-04-07T17:51:25.691 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 138143, 2024-04-07T17:51:25.691 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 14239009375 2024-04-07T17:51:25.691 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.691 INFO:tasks.workunit.client.0.smithi100.stdout: "op_delayed_degraded": { 2024-04-07T17:51:25.691 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4 2024-04-07T17:51:25.691 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.691 INFO:tasks.workunit.client.0.smithi100.stdout: "op_delayed_unreadable": { 2024-04-07T17:51:25.692 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.692 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.692 INFO:tasks.workunit.client.0.smithi100.stdout: "op_in_bytes": { 2024-04-07T17:51:25.692 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2665625896 2024-04-07T17:51:25.692 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.692 INFO:tasks.workunit.client.0.smithi100.stdout: "op_latency": { 2024-04-07T17:51:25.692 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 53262, 2024-04-07T17:51:25.692 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 365463334048 2024-04-07T17:51:25.692 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.692 INFO:tasks.workunit.client.0.smithi100.stdout: "op_out_bytes": { 2024-04-07T17:51:25.692 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3752394306 2024-04-07T17:51:25.692 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.692 INFO:tasks.workunit.client.0.smithi100.stdout: "op_prepare_latency": { 2024-04-07T17:51:25.692 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 54137, 2024-04-07T17:51:25.693 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 47146686225 2024-04-07T17:51:25.693 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.693 INFO:tasks.workunit.client.0.smithi100.stdout: "op_process_latency": { 2024-04-07T17:51:25.693 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 53262, 2024-04-07T17:51:25.693 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 125634824678 2024-04-07T17:51:25.693 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.693 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r": { 2024-04-07T17:51:25.693 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 37074 2024-04-07T17:51:25.693 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.693 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_latency": { 2024-04-07T17:51:25.693 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 37074, 2024-04-07T17:51:25.693 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 32583038857 2024-04-07T17:51:25.693 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.693 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_latency_out_bytes_histogram": { 2024-04-07T17:51:25.693 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.694 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.694 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_out_bytes": { 2024-04-07T17:51:25.694 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3752394306 2024-04-07T17:51:25.694 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.694 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_prepare_latency": { 2024-04-07T17:51:25.694 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 37434, 2024-04-07T17:51:25.694 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 17131282728 2024-04-07T17:51:25.694 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.694 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_process_latency": { 2024-04-07T17:51:25.694 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 37074, 2024-04-07T17:51:25.694 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 17452953714 2024-04-07T17:51:25.694 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.694 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw": { 2024-04-07T17:51:25.694 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 8557 2024-04-07T17:51:25.694 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.695 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_in_bytes": { 2024-04-07T17:51:25.695 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 574680166 2024-04-07T17:51:25.695 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.695 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_latency": { 2024-04-07T17:51:25.695 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 8557, 2024-04-07T17:51:25.695 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 36381436318 2024-04-07T17:51:25.695 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.695 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_latency_in_bytes_histogram": { 2024-04-07T17:51:25.695 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.695 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.695 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_latency_out_bytes_histogram": { 2024-04-07T17:51:25.695 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.695 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.695 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_out_bytes": { 2024-04-07T17:51:25.695 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.696 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.696 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_prepare_latency": { 2024-04-07T17:51:25.696 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 8868, 2024-04-07T17:51:25.696 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 12409822934 2024-04-07T17:51:25.696 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.696 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_process_latency": { 2024-04-07T17:51:25.696 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 8557, 2024-04-07T17:51:25.696 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 31269983898 2024-04-07T17:51:25.696 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.696 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w": { 2024-04-07T17:51:25.696 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 7631 2024-04-07T17:51:25.696 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.696 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_in_bytes": { 2024-04-07T17:51:25.696 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2090945730 2024-04-07T17:51:25.696 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.697 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_latency": { 2024-04-07T17:51:25.697 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 7631, 2024-04-07T17:51:25.697 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 296498858873 2024-04-07T17:51:25.697 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.697 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_latency_in_bytes_histogram": { 2024-04-07T17:51:25.697 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.697 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.697 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_prepare_latency": { 2024-04-07T17:51:25.697 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 7835, 2024-04-07T17:51:25.697 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 17605580563 2024-04-07T17:51:25.697 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.697 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_process_latency": { 2024-04-07T17:51:25.697 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 7631, 2024-04-07T17:51:25.697 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 76911887066 2024-04-07T17:51:25.697 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.697 INFO:tasks.workunit.client.0.smithi100.stdout: "op_wip": { 2024-04-07T17:51:25.698 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.698 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.698 INFO:tasks.workunit.client.0.smithi100.stdout: "recovery_bytes": { 2024-04-07T17:51:25.698 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.698 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.698 INFO:tasks.workunit.client.0.smithi100.stdout: "recovery_ops": { 2024-04-07T17:51:25.698 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 26 2024-04-07T17:51:25.698 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.698 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_bytes": { 2024-04-07T17:51:25.698 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 95995035648 2024-04-07T17:51:25.698 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.698 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_bytes_used": { 2024-04-07T17:51:25.698 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 745607168 2024-04-07T17:51:25.698 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.699 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.699 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.699 INFO:tasks.workunit.client.0.smithi100.stdout: "osd.1": { 2024-04-07T17:51:25.699 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs": { 2024-04-07T17:51:25.699 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_db_max_lat": { 2024-04-07T17:51:25.699 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 30825 2024-04-07T17:51:25.699 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.699 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_slow_fallback": { 2024-04-07T17:51:25.699 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.699 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.699 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_slow_max_lat": { 2024-04-07T17:51:25.699 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.699 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.699 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_slow_size_fallback": { 2024-04-07T17:51:25.699 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.699 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.700 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit_db": { 2024-04-07T17:51:25.700 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 65536 2024-04-07T17:51:25.700 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.700 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit_slow": { 2024-04-07T17:51:25.700 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.700 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.700 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit_wal": { 2024-04-07T17:51:25.700 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.700 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.700 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_wal_max_lat": { 2024-04-07T17:51:25.700 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.700 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.700 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_written_slow": { 2024-04-07T17:51:25.700 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.700 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.701 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_written_sst": { 2024-04-07T17:51:25.701 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5402624 2024-04-07T17:51:25.701 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.701 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_written_wal": { 2024-04-07T17:51:25.701 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 723148800 2024-04-07T17:51:25.701 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.701 INFO:tasks.workunit.client.0.smithi100.stdout: "compact_lat": { 2024-04-07T17:51:25.701 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 18, 2024-04-07T17:51:25.701 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5817020 2024-04-07T17:51:25.701 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.701 INFO:tasks.workunit.client.0.smithi100.stdout: "compact_lock_lat": { 2024-04-07T17:51:25.701 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 18, 2024-04-07T17:51:25.701 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2179935 2024-04-07T17:51:25.701 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.701 INFO:tasks.workunit.client.0.smithi100.stdout: "db_alloc_lat": { 2024-04-07T17:51:25.702 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 187, 2024-04-07T17:51:25.702 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1295182 2024-04-07T17:51:25.702 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.702 INFO:tasks.workunit.client.0.smithi100.stdout: "db_total_bytes": { 2024-04-07T17:51:25.702 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 95995027456 2024-04-07T17:51:25.702 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.702 INFO:tasks.workunit.client.0.smithi100.stdout: "db_used_bytes": { 2024-04-07T17:51:25.702 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 721354752 2024-04-07T17:51:25.702 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.702 INFO:tasks.workunit.client.0.smithi100.stdout: "flush_lat": { 2024-04-07T17:51:25.702 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 74047, 2024-04-07T17:51:25.702 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 13624180905 2024-04-07T17:51:25.702 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.702 INFO:tasks.workunit.client.0.smithi100.stdout: "fsync_lat": { 2024-04-07T17:51:25.702 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 74056, 2024-04-07T17:51:25.702 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 23366921692 2024-04-07T17:51:25.703 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.703 INFO:tasks.workunit.client.0.smithi100.stdout: "log_bytes": { 2024-04-07T17:51:25.703 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4120576 2024-04-07T17:51:25.703 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.703 INFO:tasks.workunit.client.0.smithi100.stdout: "logged_bytes": { 2024-04-07T17:51:25.703 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 303386624 2024-04-07T17:51:25.703 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.703 INFO:tasks.workunit.client.0.smithi100.stdout: "max_bytes_db": { 2024-04-07T17:51:25.703 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 738394112 2024-04-07T17:51:25.703 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.703 INFO:tasks.workunit.client.0.smithi100.stdout: "max_bytes_slow": { 2024-04-07T17:51:25.703 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.703 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.703 INFO:tasks.workunit.client.0.smithi100.stdout: "max_bytes_wal": { 2024-04-07T17:51:25.703 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.703 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.704 INFO:tasks.workunit.client.0.smithi100.stdout: "num_files": { 2024-04-07T17:51:25.704 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 50 2024-04-07T17:51:25.704 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.704 INFO:tasks.workunit.client.0.smithi100.stdout: "read_bytes": { 2024-04-07T17:51:25.704 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3171092523 2024-04-07T17:51:25.704 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.704 INFO:tasks.workunit.client.0.smithi100.stdout: "read_count": { 2024-04-07T17:51:25.704 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 42792 2024-04-07T17:51:25.704 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.704 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes": { 2024-04-07T17:51:25.704 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2761277440 2024-04-07T17:51:25.704 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.704 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes_db": { 2024-04-07T17:51:25.704 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2761281536 2024-04-07T17:51:25.704 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.704 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes_slow": { 2024-04-07T17:51:25.705 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.705 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.705 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes_wal": { 2024-04-07T17:51:25.705 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.705 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.705 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_count": { 2024-04-07T17:51:25.705 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 35333 2024-04-07T17:51:25.705 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.705 INFO:tasks.workunit.client.0.smithi100.stdout: "read_lat": { 2024-04-07T17:51:25.705 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 42792, 2024-04-07T17:51:25.705 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2006369197 2024-04-07T17:51:25.705 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.705 INFO:tasks.workunit.client.0.smithi100.stdout: "read_prefetch_bytes": { 2024-04-07T17:51:25.705 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3054473279 2024-04-07T17:51:25.705 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.705 INFO:tasks.workunit.client.0.smithi100.stdout: "read_prefetch_count": { 2024-04-07T17:51:25.705 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 39218 2024-04-07T17:51:25.706 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.706 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_buffer_bytes": { 2024-04-07T17:51:25.706 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1733730 2024-04-07T17:51:25.706 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.706 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_buffer_count": { 2024-04-07T17:51:25.706 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 618 2024-04-07T17:51:25.706 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.706 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_bytes": { 2024-04-07T17:51:25.706 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5349743 2024-04-07T17:51:25.706 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.706 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_count": { 2024-04-07T17:51:25.706 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 831 2024-04-07T17:51:25.706 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.706 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes": { 2024-04-07T17:51:25.707 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3616013 2024-04-07T17:51:25.707 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.707 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes_db": { 2024-04-07T17:51:25.707 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3616013 2024-04-07T17:51:25.707 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.707 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes_slow": { 2024-04-07T17:51:25.707 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.707 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.707 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes_wal": { 2024-04-07T17:51:25.707 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.707 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.707 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_count": { 2024-04-07T17:51:25.707 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 217 2024-04-07T17:51:25.707 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.707 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_lat": { 2024-04-07T17:51:25.707 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 831, 2024-04-07T17:51:25.708 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2170610 2024-04-07T17:51:25.708 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.708 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_alloc_lat": { 2024-04-07T17:51:25.708 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:25.708 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.708 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.708 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_total_bytes": { 2024-04-07T17:51:25.708 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.708 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.708 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_used_bytes": { 2024-04-07T17:51:25.708 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.708 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.708 INFO:tasks.workunit.client.0.smithi100.stdout: "truncate_lat": { 2024-04-07T17:51:25.708 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1, 2024-04-07T17:51:25.708 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1014 2024-04-07T17:51:25.709 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.709 INFO:tasks.workunit.client.0.smithi100.stdout: "unlink_lat": { 2024-04-07T17:51:25.709 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 10, 2024-04-07T17:51:25.709 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 14895 2024-04-07T17:51:25.709 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.709 INFO:tasks.workunit.client.0.smithi100.stdout: "wal_alloc_lat": { 2024-04-07T17:51:25.709 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:25.709 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.709 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.709 INFO:tasks.workunit.client.0.smithi100.stdout: "wal_total_bytes": { 2024-04-07T17:51:25.709 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.709 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.709 INFO:tasks.workunit.client.0.smithi100.stdout: "wal_used_bytes": { 2024-04-07T17:51:25.709 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.709 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.709 INFO:tasks.workunit.client.0.smithi100.stdout: "write_bytes": { 2024-04-07T17:51:25.710 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1034661888 2024-04-07T17:51:25.710 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.710 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.710 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore": { 2024-04-07T17:51:25.710 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit": { 2024-04-07T17:51:25.710 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4096 2024-04-07T17:51:25.710 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.710 INFO:tasks.workunit.client.0.smithi100.stdout: "allocated": { 2024-04-07T17:51:25.710 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 9814016 2024-04-07T17:51:25.710 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.710 INFO:tasks.workunit.client.0.smithi100.stdout: "allocator_lat": { 2024-04-07T17:51:25.710 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 18804, 2024-04-07T17:51:25.710 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 100612374 2024-04-07T17:51:25.710 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.710 INFO:tasks.workunit.client.0.smithi100.stdout: "clist_lat": { 2024-04-07T17:51:25.710 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 3554, 2024-04-07T17:51:25.710 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 9702602885 2024-04-07T17:51:25.711 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.711 INFO:tasks.workunit.client.0.smithi100.stdout: "compress_lat": { 2024-04-07T17:51:25.711 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:25.711 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.711 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.711 INFO:tasks.workunit.client.0.smithi100.stdout: "compressed": { 2024-04-07T17:51:25.711 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.711 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.711 INFO:tasks.workunit.client.0.smithi100.stdout: "compressed_allocated": { 2024-04-07T17:51:25.711 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.711 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.711 INFO:tasks.workunit.client.0.smithi100.stdout: "compressed_original": { 2024-04-07T17:51:25.711 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.711 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.711 INFO:tasks.workunit.client.0.smithi100.stdout: "csum_lat": { 2024-04-07T17:51:25.711 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 20948, 2024-04-07T17:51:25.712 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 250530103 2024-04-07T17:51:25.712 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.712 INFO:tasks.workunit.client.0.smithi100.stdout: "decompress_lat": { 2024-04-07T17:51:25.712 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:25.712 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.712 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.712 INFO:tasks.workunit.client.0.smithi100.stdout: "fragmentation_micros": { 2024-04-07T17:51:25.712 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.712 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.712 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_commit_lat": { 2024-04-07T17:51:25.712 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 73998, 2024-04-07T17:51:25.712 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 30592293335 2024-04-07T17:51:25.712 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.712 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_final_lat": { 2024-04-07T17:51:25.712 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 73127, 2024-04-07T17:51:25.712 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3885020813 2024-04-07T17:51:25.713 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.713 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_flush_lat": { 2024-04-07T17:51:25.713 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 73998, 2024-04-07T17:51:25.713 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 99101609 2024-04-07T17:51:25.713 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.713 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_sync_lat": { 2024-04-07T17:51:25.713 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 73998, 2024-04-07T17:51:25.713 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 30691394944 2024-04-07T17:51:25.713 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.713 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_get_keys_lat": { 2024-04-07T17:51:25.713 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:25.713 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.713 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.713 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_get_values_lat": { 2024-04-07T17:51:25.713 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 29479, 2024-04-07T17:51:25.713 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1535907762 2024-04-07T17:51:25.713 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.714 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_lower_bound_lat": { 2024-04-07T17:51:25.714 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 847, 2024-04-07T17:51:25.714 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5353667 2024-04-07T17:51:25.714 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.714 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_next_lat": { 2024-04-07T17:51:25.714 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 17576, 2024-04-07T17:51:25.714 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 23428630 2024-04-07T17:51:25.714 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.714 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_seek_to_first_lat": { 2024-04-07T17:51:25.714 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 45, 2024-04-07T17:51:25.714 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 74604 2024-04-07T17:51:25.714 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.714 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_upper_bound_lat": { 2024-04-07T17:51:25.714 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 11749, 2024-04-07T17:51:25.714 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3050678508 2024-04-07T17:51:25.714 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.715 INFO:tasks.workunit.client.0.smithi100.stdout: "onode_hits": { 2024-04-07T17:51:25.715 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 227653 2024-04-07T17:51:25.715 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.715 INFO:tasks.workunit.client.0.smithi100.stdout: "onode_misses": { 2024-04-07T17:51:25.715 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 26792 2024-04-07T17:51:25.715 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.715 INFO:tasks.workunit.client.0.smithi100.stdout: "read_lat": { 2024-04-07T17:51:25.715 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 23747, 2024-04-07T17:51:25.715 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3478797990 2024-04-07T17:51:25.715 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.715 INFO:tasks.workunit.client.0.smithi100.stdout: "read_onode_meta_lat": { 2024-04-07T17:51:25.715 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 76353, 2024-04-07T17:51:25.715 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 39167256 2024-04-07T17:51:25.715 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.715 INFO:tasks.workunit.client.0.smithi100.stdout: "read_wait_aio_lat": { 2024-04-07T17:51:25.715 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 52606, 2024-04-07T17:51:25.716 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1660372202 2024-04-07T17:51:25.716 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.716 INFO:tasks.workunit.client.0.smithi100.stdout: "reads_with_retries": { 2024-04-07T17:51:25.716 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.716 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.716 INFO:tasks.workunit.client.0.smithi100.stdout: "remove_lat": { 2024-04-07T17:51:25.716 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 15062, 2024-04-07T17:51:25.716 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 718857059 2024-04-07T17:51:25.716 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.716 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_aio_wait_count": { 2024-04-07T17:51:25.716 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.716 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.716 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_committed_kv_count": { 2024-04-07T17:51:25.716 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.716 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.716 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_read_onode_meta_count": { 2024-04-07T17:51:25.717 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.717 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.717 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_read_wait_aio_count": { 2024-04-07T17:51:25.717 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.717 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.717 INFO:tasks.workunit.client.0.smithi100.stdout: "state_aio_wait_lat": { 2024-04-07T17:51:25.717 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 85696, 2024-04-07T17:51:25.717 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 8515002567 2024-04-07T17:51:25.717 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.717 INFO:tasks.workunit.client.0.smithi100.stdout: "state_deferred_aio_wait_lat": { 2024-04-07T17:51:25.717 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 22706, 2024-04-07T17:51:25.717 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2143104911 2024-04-07T17:51:25.717 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.717 INFO:tasks.workunit.client.0.smithi100.stdout: "state_deferred_cleanup_lat": { 2024-04-07T17:51:25.717 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 22706, 2024-04-07T17:51:25.717 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4386813655909 2024-04-07T17:51:25.717 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.718 INFO:tasks.workunit.client.0.smithi100.stdout: "state_deferred_queued_lat": { 2024-04-07T17:51:25.718 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 22706, 2024-04-07T17:51:25.718 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 7212579664352 2024-04-07T17:51:25.718 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.718 INFO:tasks.workunit.client.0.smithi100.stdout: "state_done_lat": { 2024-04-07T17:51:25.718 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 85696, 2024-04-07T17:51:25.718 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2950395751529 2024-04-07T17:51:25.718 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.718 INFO:tasks.workunit.client.0.smithi100.stdout: "state_finishing_lat": { 2024-04-07T17:51:25.718 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 85696, 2024-04-07T17:51:25.718 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 17842298 2024-04-07T17:51:25.718 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.718 INFO:tasks.workunit.client.0.smithi100.stdout: "state_io_done_lat": { 2024-04-07T17:51:25.718 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 85696, 2024-04-07T17:51:25.718 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 318013774 2024-04-07T17:51:25.718 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.718 INFO:tasks.workunit.client.0.smithi100.stdout: "state_kv_commiting_lat": { 2024-04-07T17:51:25.719 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 85696, 2024-04-07T17:51:25.719 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 50196560708 2024-04-07T17:51:25.719 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.719 INFO:tasks.workunit.client.0.smithi100.stdout: "state_kv_done_lat": { 2024-04-07T17:51:25.719 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 85696, 2024-04-07T17:51:25.719 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 64451217 2024-04-07T17:51:25.719 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.719 INFO:tasks.workunit.client.0.smithi100.stdout: "state_kv_queued_lat": { 2024-04-07T17:51:25.719 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 85696, 2024-04-07T17:51:25.719 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 24917952642 2024-04-07T17:51:25.719 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.719 INFO:tasks.workunit.client.0.smithi100.stdout: "state_prepare_lat": { 2024-04-07T17:51:25.719 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 85696, 2024-04-07T17:51:25.719 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 23724441505 2024-04-07T17:51:25.719 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.720 INFO:tasks.workunit.client.0.smithi100.stdout: "stored": { 2024-04-07T17:51:25.720 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5699861 2024-04-07T17:51:25.720 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.720 INFO:tasks.workunit.client.0.smithi100.stdout: "truncate_lat": { 2024-04-07T17:51:25.720 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 2507, 2024-04-07T17:51:25.720 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 25130678 2024-04-07T17:51:25.720 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.720 INFO:tasks.workunit.client.0.smithi100.stdout: "txc_commit_lat": { 2024-04-07T17:51:25.720 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 85696, 2024-04-07T17:51:25.720 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 107702564402 2024-04-07T17:51:25.720 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.720 INFO:tasks.workunit.client.0.smithi100.stdout: "txc_submit_lat": { 2024-04-07T17:51:25.720 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 85696, 2024-04-07T17:51:25.720 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 25902403453 2024-04-07T17:51:25.720 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.720 INFO:tasks.workunit.client.0.smithi100.stdout: "txc_throttle_lat": { 2024-04-07T17:51:25.720 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 85696, 2024-04-07T17:51:25.721 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 991891115 2024-04-07T17:51:25.721 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.721 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.721 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache": { 2024-04-07T17:51:25.721 INFO:tasks.workunit.client.0.smithi100.stdout: "cache_bytes": { 2024-04-07T17:51:25.721 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1444614092 2024-04-07T17:51:25.721 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.721 INFO:tasks.workunit.client.0.smithi100.stdout: "heap_bytes": { 2024-04-07T17:51:25.721 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 695918592 2024-04-07T17:51:25.721 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.721 INFO:tasks.workunit.client.0.smithi100.stdout: "mapped_bytes": { 2024-04-07T17:51:25.721 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 660807680 2024-04-07T17:51:25.721 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.721 INFO:tasks.workunit.client.0.smithi100.stdout: "target_bytes": { 2024-04-07T17:51:25.721 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2646965248 2024-04-07T17:51:25.721 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.721 INFO:tasks.workunit.client.0.smithi100.stdout: "unmapped_bytes": { 2024-04-07T17:51:25.722 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 35110912 2024-04-07T17:51:25.722 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.722 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.722 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:data": { 2024-04-07T17:51:25.722 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:25.722 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 134217728 2024-04-07T17:51:25.722 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.722 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:25.722 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.722 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.722 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:25.722 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.722 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.722 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:25.722 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 59401471 2024-04-07T17:51:25.723 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.723 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:25.723 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.723 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.723 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:25.723 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.723 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.723 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:25.723 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.723 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.723 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:25.723 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.723 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.723 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:25.723 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 114688 2024-04-07T17:51:25.723 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.724 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:25.724 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 733184 2024-04-07T17:51:25.724 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.724 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:25.724 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.724 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.724 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:25.724 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.724 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.724 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:25.724 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.724 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.724 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:25.724 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 73968385 2024-04-07T17:51:25.724 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.724 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.724 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:kv": { 2024-04-07T17:51:25.725 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:25.725 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 587202560 2024-04-07T17:51:25.725 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.725 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:25.725 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 67108864 2024-04-07T17:51:25.725 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.725 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:25.725 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.725 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.725 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:25.725 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 445511035 2024-04-07T17:51:25.725 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.725 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:25.725 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.725 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.726 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:25.726 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 256 2024-04-07T17:51:25.726 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.726 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:25.726 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.726 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.726 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:25.726 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.726 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.726 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:25.726 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 112208 2024-04-07T17:51:25.726 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.726 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:25.726 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.727 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.727 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:25.727 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.727 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.727 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:25.727 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.727 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.727 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:25.727 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.727 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.727 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:25.727 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 74470197 2024-04-07T17:51:25.727 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.727 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.727 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:kv_onode": { 2024-04-07T17:51:25.727 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:25.728 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 184549376 2024-04-07T17:51:25.728 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.728 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:25.728 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 67108864 2024-04-07T17:51:25.728 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.728 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:25.728 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4603888 2024-04-07T17:51:25.728 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.728 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:25.728 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 39600980 2024-04-07T17:51:25.728 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.728 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:25.728 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.728 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.728 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:25.729 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.729 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.729 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:25.729 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.729 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.729 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:25.729 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.729 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.729 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:25.729 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.729 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.729 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:25.729 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.729 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.729 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:25.729 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.730 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.730 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:25.730 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.730 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.730 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:25.730 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.730 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.730 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:25.730 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 73235644 2024-04-07T17:51:25.730 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.730 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.730 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:meta": { 2024-04-07T17:51:25.730 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:25.730 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 520093696 2024-04-07T17:51:25.730 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.731 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:25.731 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.731 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.731 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:25.731 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.731 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.731 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:25.731 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 445511035 2024-04-07T17:51:25.731 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.731 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:25.731 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.731 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.731 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:25.731 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.731 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.731 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:25.732 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.732 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.732 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:25.732 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.732 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.732 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:25.732 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 176559 2024-04-07T17:51:25.732 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.732 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:25.732 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1067248 2024-04-07T17:51:25.732 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.732 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:25.732 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.732 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.732 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:25.732 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.733 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.733 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:25.733 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.733 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.733 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:25.733 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 73338854 2024-04-07T17:51:25.733 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.733 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.733 INFO:tasks.workunit.client.0.smithi100.stdout: "objecter": { 2024-04-07T17:51:25.733 INFO:tasks.workunit.client.0.smithi100.stdout: "op_active": { 2024-04-07T17:51:25.733 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.733 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.733 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r": { 2024-04-07T17:51:25.733 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 21 2024-04-07T17:51:25.733 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.733 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rmw": { 2024-04-07T17:51:25.734 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.734 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.734 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w": { 2024-04-07T17:51:25.734 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1 2024-04-07T17:51:25.734 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.734 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.734 INFO:tasks.workunit.client.0.smithi100.stdout: "osd": { 2024-04-07T17:51:25.734 INFO:tasks.workunit.client.0.smithi100.stdout: "numpg": { 2024-04-07T17:51:25.734 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 52 2024-04-07T17:51:25.734 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.734 INFO:tasks.workunit.client.0.smithi100.stdout: "numpg_removing": { 2024-04-07T17:51:25.734 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.734 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.734 INFO:tasks.workunit.client.0.smithi100.stdout: "op": { 2024-04-07T17:51:25.735 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 45437 2024-04-07T17:51:25.735 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.735 INFO:tasks.workunit.client.0.smithi100.stdout: "op_before_queue_op_lat": { 2024-04-07T17:51:25.735 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 141249, 2024-04-07T17:51:25.735 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 14856125226 2024-04-07T17:51:25.735 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.735 INFO:tasks.workunit.client.0.smithi100.stdout: "op_delayed_degraded": { 2024-04-07T17:51:25.735 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.735 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.735 INFO:tasks.workunit.client.0.smithi100.stdout: "op_delayed_unreadable": { 2024-04-07T17:51:25.735 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1 2024-04-07T17:51:25.735 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.735 INFO:tasks.workunit.client.0.smithi100.stdout: "op_in_bytes": { 2024-04-07T17:51:25.735 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2465060363 2024-04-07T17:51:25.735 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.735 INFO:tasks.workunit.client.0.smithi100.stdout: "op_latency": { 2024-04-07T17:51:25.736 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 45437, 2024-04-07T17:51:25.736 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 845704656846 2024-04-07T17:51:25.736 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.736 INFO:tasks.workunit.client.0.smithi100.stdout: "op_out_bytes": { 2024-04-07T17:51:25.736 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2916484455 2024-04-07T17:51:25.736 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.736 INFO:tasks.workunit.client.0.smithi100.stdout: "op_prepare_latency": { 2024-04-07T17:51:25.736 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 47101, 2024-04-07T17:51:25.736 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 66781908488 2024-04-07T17:51:25.736 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.736 INFO:tasks.workunit.client.0.smithi100.stdout: "op_process_latency": { 2024-04-07T17:51:25.736 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 45437, 2024-04-07T17:51:25.736 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 194958987564 2024-04-07T17:51:25.736 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.737 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r": { 2024-04-07T17:51:25.737 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 22754 2024-04-07T17:51:25.737 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.737 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_latency": { 2024-04-07T17:51:25.737 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 22754, 2024-04-07T17:51:25.737 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 30613921114 2024-04-07T17:51:25.737 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.737 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_latency_out_bytes_histogram": { 2024-04-07T17:51:25.737 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.737 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.737 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_out_bytes": { 2024-04-07T17:51:25.737 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2916484455 2024-04-07T17:51:25.737 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.737 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_prepare_latency": { 2024-04-07T17:51:25.737 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 23171, 2024-04-07T17:51:25.737 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 12769117700 2024-04-07T17:51:25.738 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.738 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_process_latency": { 2024-04-07T17:51:25.738 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 22754, 2024-04-07T17:51:25.738 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 12464549208 2024-04-07T17:51:25.738 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.738 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw": { 2024-04-07T17:51:25.738 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 12919 2024-04-07T17:51:25.738 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.738 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_in_bytes": { 2024-04-07T17:51:25.738 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 335492738 2024-04-07T17:51:25.738 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.738 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_latency": { 2024-04-07T17:51:25.738 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 12919, 2024-04-07T17:51:25.738 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 239996971130 2024-04-07T17:51:25.738 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.739 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_latency_in_bytes_histogram": { 2024-04-07T17:51:25.739 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.739 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.739 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_latency_out_bytes_histogram": { 2024-04-07T17:51:25.739 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.739 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.739 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_out_bytes": { 2024-04-07T17:51:25.739 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.739 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.739 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_prepare_latency": { 2024-04-07T17:51:25.739 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 13070, 2024-04-07T17:51:25.739 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 30679482792 2024-04-07T17:51:25.739 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.739 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_process_latency": { 2024-04-07T17:51:25.739 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 12919, 2024-04-07T17:51:25.739 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 75646162399 2024-04-07T17:51:25.740 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.740 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w": { 2024-04-07T17:51:25.740 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 9764 2024-04-07T17:51:25.740 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.740 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_in_bytes": { 2024-04-07T17:51:25.740 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2129567625 2024-04-07T17:51:25.740 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.740 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_latency": { 2024-04-07T17:51:25.740 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 9764, 2024-04-07T17:51:25.740 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 575093764602 2024-04-07T17:51:25.740 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.740 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_latency_in_bytes_histogram": { 2024-04-07T17:51:25.740 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.740 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.740 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_prepare_latency": { 2024-04-07T17:51:25.740 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 10860, 2024-04-07T17:51:25.740 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 23333307996 2024-04-07T17:51:25.741 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.741 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_process_latency": { 2024-04-07T17:51:25.741 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 9764, 2024-04-07T17:51:25.741 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 106848275957 2024-04-07T17:51:25.741 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.741 INFO:tasks.workunit.client.0.smithi100.stdout: "op_wip": { 2024-04-07T17:51:25.741 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.741 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.741 INFO:tasks.workunit.client.0.smithi100.stdout: "recovery_bytes": { 2024-04-07T17:51:25.741 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 40249613 2024-04-07T17:51:25.741 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.741 INFO:tasks.workunit.client.0.smithi100.stdout: "recovery_ops": { 2024-04-07T17:51:25.741 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 28 2024-04-07T17:51:25.741 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.741 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_bytes": { 2024-04-07T17:51:25.741 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 95995035648 2024-04-07T17:51:25.741 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.742 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_bytes_used": { 2024-04-07T17:51:25.742 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 731176960 2024-04-07T17:51:25.742 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.742 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.742 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.742 INFO:tasks.workunit.client.0.smithi100.stdout: "osd.2": { 2024-04-07T17:51:25.742 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs": { 2024-04-07T17:51:25.742 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_db_max_lat": { 2024-04-07T17:51:25.742 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 33529 2024-04-07T17:51:25.742 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.742 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_slow_fallback": { 2024-04-07T17:51:25.742 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.742 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.742 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_slow_max_lat": { 2024-04-07T17:51:25.742 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.742 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.743 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_slow_size_fallback": { 2024-04-07T17:51:25.743 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.743 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.743 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit_db": { 2024-04-07T17:51:25.743 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 65536 2024-04-07T17:51:25.743 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.743 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit_slow": { 2024-04-07T17:51:25.743 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.743 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.743 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit_wal": { 2024-04-07T17:51:25.743 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.743 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.743 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_wal_max_lat": { 2024-04-07T17:51:25.743 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.743 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.743 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_written_slow": { 2024-04-07T17:51:25.743 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.744 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.744 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_written_sst": { 2024-04-07T17:51:25.744 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 11583488 2024-04-07T17:51:25.744 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.744 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_written_wal": { 2024-04-07T17:51:25.744 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 567971840 2024-04-07T17:51:25.744 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.744 INFO:tasks.workunit.client.0.smithi100.stdout: "compact_lat": { 2024-04-07T17:51:25.744 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 17, 2024-04-07T17:51:25.744 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 10549451 2024-04-07T17:51:25.744 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.744 INFO:tasks.workunit.client.0.smithi100.stdout: "compact_lock_lat": { 2024-04-07T17:51:25.744 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 17, 2024-04-07T17:51:25.744 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3589378 2024-04-07T17:51:25.744 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.744 INFO:tasks.workunit.client.0.smithi100.stdout: "db_alloc_lat": { 2024-04-07T17:51:25.745 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 176, 2024-04-07T17:51:25.745 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1295988 2024-04-07T17:51:25.745 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.745 INFO:tasks.workunit.client.0.smithi100.stdout: "db_total_bytes": { 2024-04-07T17:51:25.745 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 95995027456 2024-04-07T17:51:25.745 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.745 INFO:tasks.workunit.client.0.smithi100.stdout: "db_used_bytes": { 2024-04-07T17:51:25.745 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 446365696 2024-04-07T17:51:25.745 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.745 INFO:tasks.workunit.client.0.smithi100.stdout: "flush_lat": { 2024-04-07T17:51:25.745 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 69116, 2024-04-07T17:51:25.745 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 12568150083 2024-04-07T17:51:25.745 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.745 INFO:tasks.workunit.client.0.smithi100.stdout: "fsync_lat": { 2024-04-07T17:51:25.745 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 69111, 2024-04-07T17:51:25.745 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 21920914517 2024-04-07T17:51:25.746 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.746 INFO:tasks.workunit.client.0.smithi100.stdout: "log_bytes": { 2024-04-07T17:51:25.746 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 491520 2024-04-07T17:51:25.746 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.746 INFO:tasks.workunit.client.0.smithi100.stdout: "logged_bytes": { 2024-04-07T17:51:25.746 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 283127808 2024-04-07T17:51:25.746 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.746 INFO:tasks.workunit.client.0.smithi100.stdout: "max_bytes_db": { 2024-04-07T17:51:25.746 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 449183744 2024-04-07T17:51:25.746 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.746 INFO:tasks.workunit.client.0.smithi100.stdout: "max_bytes_slow": { 2024-04-07T17:51:25.747 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.747 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.747 INFO:tasks.workunit.client.0.smithi100.stdout: "max_bytes_wal": { 2024-04-07T17:51:25.747 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.747 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.747 INFO:tasks.workunit.client.0.smithi100.stdout: "num_files": { 2024-04-07T17:51:25.747 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 47 2024-04-07T17:51:25.747 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.747 INFO:tasks.workunit.client.0.smithi100.stdout: "read_bytes": { 2024-04-07T17:51:25.747 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 8760902516 2024-04-07T17:51:25.747 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.747 INFO:tasks.workunit.client.0.smithi100.stdout: "read_count": { 2024-04-07T17:51:25.747 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 74144 2024-04-07T17:51:25.747 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.747 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes": { 2024-04-07T17:51:25.747 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 7434956800 2024-04-07T17:51:25.747 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.748 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes_db": { 2024-04-07T17:51:25.748 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 7434960896 2024-04-07T17:51:25.748 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.748 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes_slow": { 2024-04-07T17:51:25.748 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.748 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.748 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes_wal": { 2024-04-07T17:51:25.748 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.748 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.748 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_count": { 2024-04-07T17:51:25.748 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 64202 2024-04-07T17:51:25.748 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.748 INFO:tasks.workunit.client.0.smithi100.stdout: "read_lat": { 2024-04-07T17:51:25.748 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 74144, 2024-04-07T17:51:25.748 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4199208949 2024-04-07T17:51:25.748 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.748 INFO:tasks.workunit.client.0.smithi100.stdout: "read_prefetch_bytes": { 2024-04-07T17:51:25.749 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 8601015701 2024-04-07T17:51:25.749 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.749 INFO:tasks.workunit.client.0.smithi100.stdout: "read_prefetch_count": { 2024-04-07T17:51:25.749 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 69247 2024-04-07T17:51:25.749 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.749 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_buffer_bytes": { 2024-04-07T17:51:25.749 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3233145 2024-04-07T17:51:25.749 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.749 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_buffer_count": { 2024-04-07T17:51:25.749 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 709 2024-04-07T17:51:25.749 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.749 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_bytes": { 2024-04-07T17:51:25.749 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 11496373 2024-04-07T17:51:25.749 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.749 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_count": { 2024-04-07T17:51:25.749 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1006 2024-04-07T17:51:25.749 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.750 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes": { 2024-04-07T17:51:25.750 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 8263228 2024-04-07T17:51:25.750 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.750 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes_db": { 2024-04-07T17:51:25.750 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 8263228 2024-04-07T17:51:25.750 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.750 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes_slow": { 2024-04-07T17:51:25.750 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.750 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.750 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes_wal": { 2024-04-07T17:51:25.750 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.750 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.750 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_count": { 2024-04-07T17:51:25.750 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 312 2024-04-07T17:51:25.750 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.751 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_lat": { 2024-04-07T17:51:25.751 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1006, 2024-04-07T17:51:25.751 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4007550 2024-04-07T17:51:25.751 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.751 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_alloc_lat": { 2024-04-07T17:51:25.751 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:25.751 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.751 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.751 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_total_bytes": { 2024-04-07T17:51:25.751 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.751 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.751 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_used_bytes": { 2024-04-07T17:51:25.751 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.751 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.751 INFO:tasks.workunit.client.0.smithi100.stdout: "truncate_lat": { 2024-04-07T17:51:25.751 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1, 2024-04-07T17:51:25.752 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 636 2024-04-07T17:51:25.752 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.752 INFO:tasks.workunit.client.0.smithi100.stdout: "unlink_lat": { 2024-04-07T17:51:25.752 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 13, 2024-04-07T17:51:25.752 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 14014 2024-04-07T17:51:25.752 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.752 INFO:tasks.workunit.client.0.smithi100.stdout: "wal_alloc_lat": { 2024-04-07T17:51:25.752 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:25.752 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.752 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.752 INFO:tasks.workunit.client.0.smithi100.stdout: "wal_total_bytes": { 2024-04-07T17:51:25.752 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.752 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.752 INFO:tasks.workunit.client.0.smithi100.stdout: "wal_used_bytes": { 2024-04-07T17:51:25.752 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.752 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.752 INFO:tasks.workunit.client.0.smithi100.stdout: "write_bytes": { 2024-04-07T17:51:25.752 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 865255424 2024-04-07T17:51:25.753 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.753 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.753 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore": { 2024-04-07T17:51:25.753 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit": { 2024-04-07T17:51:25.753 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4096 2024-04-07T17:51:25.753 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.753 INFO:tasks.workunit.client.0.smithi100.stdout: "allocated": { 2024-04-07T17:51:25.753 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 9801728 2024-04-07T17:51:25.753 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.753 INFO:tasks.workunit.client.0.smithi100.stdout: "allocator_lat": { 2024-04-07T17:51:25.753 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 13808, 2024-04-07T17:51:25.753 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 73423246 2024-04-07T17:51:25.753 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.753 INFO:tasks.workunit.client.0.smithi100.stdout: "clist_lat": { 2024-04-07T17:51:25.753 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 3623, 2024-04-07T17:51:25.753 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 12377775846 2024-04-07T17:51:25.753 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.754 INFO:tasks.workunit.client.0.smithi100.stdout: "compress_lat": { 2024-04-07T17:51:25.754 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:25.754 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.754 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.754 INFO:tasks.workunit.client.0.smithi100.stdout: "compressed": { 2024-04-07T17:51:25.754 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.754 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.754 INFO:tasks.workunit.client.0.smithi100.stdout: "compressed_allocated": { 2024-04-07T17:51:25.754 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.754 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.754 INFO:tasks.workunit.client.0.smithi100.stdout: "compressed_original": { 2024-04-07T17:51:25.754 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.754 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.754 INFO:tasks.workunit.client.0.smithi100.stdout: "csum_lat": { 2024-04-07T17:51:25.754 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 26449, 2024-04-07T17:51:25.754 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 333837013 2024-04-07T17:51:25.755 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.755 INFO:tasks.workunit.client.0.smithi100.stdout: "decompress_lat": { 2024-04-07T17:51:25.755 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:25.755 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.755 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.755 INFO:tasks.workunit.client.0.smithi100.stdout: "fragmentation_micros": { 2024-04-07T17:51:25.755 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.755 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.755 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_commit_lat": { 2024-04-07T17:51:25.755 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 69053, 2024-04-07T17:51:25.755 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 28692593210 2024-04-07T17:51:25.755 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.755 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_final_lat": { 2024-04-07T17:51:25.755 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 68316, 2024-04-07T17:51:25.755 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3308256168 2024-04-07T17:51:25.755 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.755 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_flush_lat": { 2024-04-07T17:51:25.756 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 69053, 2024-04-07T17:51:25.756 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 77146286 2024-04-07T17:51:25.756 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.756 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_sync_lat": { 2024-04-07T17:51:25.756 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 69053, 2024-04-07T17:51:25.756 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 28769739496 2024-04-07T17:51:25.756 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.756 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_get_keys_lat": { 2024-04-07T17:51:25.756 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:25.756 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.756 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.756 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_get_values_lat": { 2024-04-07T17:51:25.756 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 54368, 2024-04-07T17:51:25.756 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1618898953 2024-04-07T17:51:25.756 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.756 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_lower_bound_lat": { 2024-04-07T17:51:25.757 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 911, 2024-04-07T17:51:25.757 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5592855 2024-04-07T17:51:25.757 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.757 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_next_lat": { 2024-04-07T17:51:25.757 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 22756, 2024-04-07T17:51:25.757 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 25939924 2024-04-07T17:51:25.757 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.757 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_seek_to_first_lat": { 2024-04-07T17:51:25.757 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 36, 2024-04-07T17:51:25.757 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 71387 2024-04-07T17:51:25.757 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.757 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_upper_bound_lat": { 2024-04-07T17:51:25.757 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 19162, 2024-04-07T17:51:25.757 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2934148688 2024-04-07T17:51:25.757 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.757 INFO:tasks.workunit.client.0.smithi100.stdout: "onode_hits": { 2024-04-07T17:51:25.757 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 251164 2024-04-07T17:51:25.757 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.758 INFO:tasks.workunit.client.0.smithi100.stdout: "onode_misses": { 2024-04-07T17:51:25.758 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 19790 2024-04-07T17:51:25.758 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.758 INFO:tasks.workunit.client.0.smithi100.stdout: "read_lat": { 2024-04-07T17:51:25.758 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 17396, 2024-04-07T17:51:25.758 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2437128824 2024-04-07T17:51:25.758 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.758 INFO:tasks.workunit.client.0.smithi100.stdout: "read_onode_meta_lat": { 2024-04-07T17:51:25.758 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 41161, 2024-04-07T17:51:25.758 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 40062786 2024-04-07T17:51:25.758 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.758 INFO:tasks.workunit.client.0.smithi100.stdout: "read_wait_aio_lat": { 2024-04-07T17:51:25.758 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 23765, 2024-04-07T17:51:25.758 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1924044806 2024-04-07T17:51:25.758 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.758 INFO:tasks.workunit.client.0.smithi100.stdout: "reads_with_retries": { 2024-04-07T17:51:25.759 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.759 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.759 INFO:tasks.workunit.client.0.smithi100.stdout: "remove_lat": { 2024-04-07T17:51:25.759 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 12440, 2024-04-07T17:51:25.759 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 700556520 2024-04-07T17:51:25.759 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.759 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_aio_wait_count": { 2024-04-07T17:51:25.759 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.759 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.759 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_committed_kv_count": { 2024-04-07T17:51:25.759 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.759 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.759 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_read_onode_meta_count": { 2024-04-07T17:51:25.759 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.759 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.759 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_read_wait_aio_count": { 2024-04-07T17:51:25.759 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.760 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.760 INFO:tasks.workunit.client.0.smithi100.stdout: "state_aio_wait_lat": { 2024-04-07T17:51:25.760 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 77648, 2024-04-07T17:51:25.760 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 7605898743 2024-04-07T17:51:25.760 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.760 INFO:tasks.workunit.client.0.smithi100.stdout: "state_deferred_aio_wait_lat": { 2024-04-07T17:51:25.760 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 5880, 2024-04-07T17:51:25.760 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 870604827 2024-04-07T17:51:25.760 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.760 INFO:tasks.workunit.client.0.smithi100.stdout: "state_deferred_cleanup_lat": { 2024-04-07T17:51:25.760 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 5879, 2024-04-07T17:51:25.760 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4540688258752 2024-04-07T17:51:25.760 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.760 INFO:tasks.workunit.client.0.smithi100.stdout: "state_deferred_queued_lat": { 2024-04-07T17:51:25.760 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 5880, 2024-04-07T17:51:25.761 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 6337522548440 2024-04-07T17:51:25.761 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.761 INFO:tasks.workunit.client.0.smithi100.stdout: "state_done_lat": { 2024-04-07T17:51:25.761 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 77647, 2024-04-07T17:51:25.761 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2768612828005 2024-04-07T17:51:25.761 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.761 INFO:tasks.workunit.client.0.smithi100.stdout: "state_finishing_lat": { 2024-04-07T17:51:25.761 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 77647, 2024-04-07T17:51:25.761 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 18125716 2024-04-07T17:51:25.761 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.761 INFO:tasks.workunit.client.0.smithi100.stdout: "state_io_done_lat": { 2024-04-07T17:51:25.761 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 77648, 2024-04-07T17:51:25.761 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 245246928 2024-04-07T17:51:25.761 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.761 INFO:tasks.workunit.client.0.smithi100.stdout: "state_kv_commiting_lat": { 2024-04-07T17:51:25.762 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 77648, 2024-04-07T17:51:25.762 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 45419960556 2024-04-07T17:51:25.762 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.762 INFO:tasks.workunit.client.0.smithi100.stdout: "state_kv_done_lat": { 2024-04-07T17:51:25.762 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 77648, 2024-04-07T17:51:25.762 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 60433024 2024-04-07T17:51:25.762 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.762 INFO:tasks.workunit.client.0.smithi100.stdout: "state_kv_queued_lat": { 2024-04-07T17:51:25.762 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 77648, 2024-04-07T17:51:25.762 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 21195606198 2024-04-07T17:51:25.762 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.762 INFO:tasks.workunit.client.0.smithi100.stdout: "state_prepare_lat": { 2024-04-07T17:51:25.762 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 77648, 2024-04-07T17:51:25.762 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 23621700730 2024-04-07T17:51:25.762 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.763 INFO:tasks.workunit.client.0.smithi100.stdout: "stored": { 2024-04-07T17:51:25.763 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5699682 2024-04-07T17:51:25.763 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.763 INFO:tasks.workunit.client.0.smithi100.stdout: "truncate_lat": { 2024-04-07T17:51:25.763 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 2411, 2024-04-07T17:51:25.763 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 23697978 2024-04-07T17:51:25.763 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.763 INFO:tasks.workunit.client.0.smithi100.stdout: "txc_commit_lat": { 2024-04-07T17:51:25.763 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 77648, 2024-04-07T17:51:25.763 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 98115535290 2024-04-07T17:51:25.763 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.763 INFO:tasks.workunit.client.0.smithi100.stdout: "txc_submit_lat": { 2024-04-07T17:51:25.763 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 77648, 2024-04-07T17:51:25.763 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 25610928464 2024-04-07T17:51:25.763 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.763 INFO:tasks.workunit.client.0.smithi100.stdout: "txc_throttle_lat": { 2024-04-07T17:51:25.763 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 77648, 2024-04-07T17:51:25.763 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 903590085 2024-04-07T17:51:25.764 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.764 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.764 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache": { 2024-04-07T17:51:25.764 INFO:tasks.workunit.client.0.smithi100.stdout: "cache_bytes": { 2024-04-07T17:51:25.764 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1444614092 2024-04-07T17:51:25.764 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.764 INFO:tasks.workunit.client.0.smithi100.stdout: "heap_bytes": { 2024-04-07T17:51:25.764 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 563281920 2024-04-07T17:51:25.764 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.764 INFO:tasks.workunit.client.0.smithi100.stdout: "mapped_bytes": { 2024-04-07T17:51:25.764 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 513974272 2024-04-07T17:51:25.764 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.764 INFO:tasks.workunit.client.0.smithi100.stdout: "target_bytes": { 2024-04-07T17:51:25.764 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2646965248 2024-04-07T17:51:25.764 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.764 INFO:tasks.workunit.client.0.smithi100.stdout: "unmapped_bytes": { 2024-04-07T17:51:25.764 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 49307648 2024-04-07T17:51:25.765 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.765 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.765 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:data": { 2024-04-07T17:51:25.765 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:25.765 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 134217728 2024-04-07T17:51:25.765 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.765 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:25.765 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.765 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.765 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:25.765 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.765 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.765 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:25.765 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 59173493 2024-04-07T17:51:25.765 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.765 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:25.766 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.766 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.766 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:25.766 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.766 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.766 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:25.766 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.766 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.766 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:25.766 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.766 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.766 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:25.766 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 8192 2024-04-07T17:51:25.766 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.766 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:25.767 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 77824 2024-04-07T17:51:25.767 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.767 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:25.767 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.767 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.767 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:25.767 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.767 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.767 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:25.767 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.767 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.767 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:25.767 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 74958219 2024-04-07T17:51:25.767 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.767 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.767 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:kv": { 2024-04-07T17:51:25.767 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:25.768 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 578813952 2024-04-07T17:51:25.768 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.768 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:25.768 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 67108864 2024-04-07T17:51:25.768 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.768 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:25.768 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.768 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.768 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:25.768 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 443801202 2024-04-07T17:51:25.768 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.768 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:25.768 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.768 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.768 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:25.768 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.768 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.769 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:25.769 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2384 2024-04-07T17:51:25.769 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.769 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:25.769 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.769 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.769 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:25.769 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 150592 2024-04-07T17:51:25.769 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.769 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:25.769 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 384 2024-04-07T17:51:25.769 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.769 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:25.769 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.769 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.769 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:25.769 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.769 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.770 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:25.770 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.770 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.770 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:25.770 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 67750526 2024-04-07T17:51:25.770 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.770 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.770 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:kv_onode": { 2024-04-07T17:51:25.770 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:25.770 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 184549376 2024-04-07T17:51:25.770 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.770 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:25.770 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 67108864 2024-04-07T17:51:25.770 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.770 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:25.771 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 10790880 2024-04-07T17:51:25.771 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.771 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:25.771 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 39448995 2024-04-07T17:51:25.771 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.771 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:25.771 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.771 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.771 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:25.771 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.771 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.771 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:25.771 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.771 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.771 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:25.771 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.772 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.773 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:25.773 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.773 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.773 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:25.773 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.773 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.773 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:25.773 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.773 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.774 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:25.774 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.774 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.774 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:25.774 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.774 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.774 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:25.774 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 67200637 2024-04-07T17:51:25.774 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.774 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.774 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:meta": { 2024-04-07T17:51:25.774 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:25.774 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 520093696 2024-04-07T17:51:25.774 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.774 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:25.775 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.775 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.775 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:25.775 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.775 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.775 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:25.775 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 443801202 2024-04-07T17:51:25.775 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.775 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:25.775 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.775 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.775 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:25.775 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.775 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.775 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:25.775 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.776 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.776 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:25.776 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.776 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.776 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:25.776 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 138605 2024-04-07T17:51:25.776 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.776 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:25.776 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1113867 2024-04-07T17:51:25.776 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.776 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:25.776 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.776 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.776 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:25.777 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.777 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.777 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:25.777 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.777 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.777 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:25.777 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 75040022 2024-04-07T17:51:25.777 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.777 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.777 INFO:tasks.workunit.client.0.smithi100.stdout: "objecter": { 2024-04-07T17:51:25.777 INFO:tasks.workunit.client.0.smithi100.stdout: "op_active": { 2024-04-07T17:51:25.777 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.777 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.777 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r": { 2024-04-07T17:51:25.777 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 91 2024-04-07T17:51:25.778 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.778 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rmw": { 2024-04-07T17:51:25.778 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.778 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.778 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w": { 2024-04-07T17:51:25.778 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.778 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.778 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.778 INFO:tasks.workunit.client.0.smithi100.stdout: "osd": { 2024-04-07T17:51:25.778 INFO:tasks.workunit.client.0.smithi100.stdout: "numpg": { 2024-04-07T17:51:25.778 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 40 2024-04-07T17:51:25.778 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.778 INFO:tasks.workunit.client.0.smithi100.stdout: "numpg_removing": { 2024-04-07T17:51:25.778 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.779 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.779 INFO:tasks.workunit.client.0.smithi100.stdout: "op": { 2024-04-07T17:51:25.779 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 58391 2024-04-07T17:51:25.779 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.779 INFO:tasks.workunit.client.0.smithi100.stdout: "op_before_queue_op_lat": { 2024-04-07T17:51:25.779 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 150392, 2024-04-07T17:51:25.779 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 13814312300 2024-04-07T17:51:25.779 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.779 INFO:tasks.workunit.client.0.smithi100.stdout: "op_delayed_degraded": { 2024-04-07T17:51:25.779 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5 2024-04-07T17:51:25.779 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.779 INFO:tasks.workunit.client.0.smithi100.stdout: "op_delayed_unreadable": { 2024-04-07T17:51:25.779 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.779 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.779 INFO:tasks.workunit.client.0.smithi100.stdout: "op_in_bytes": { 2024-04-07T17:51:25.780 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2120776034 2024-04-07T17:51:25.780 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.780 INFO:tasks.workunit.client.0.smithi100.stdout: "op_latency": { 2024-04-07T17:51:25.780 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 58391, 2024-04-07T17:51:25.780 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 187069135567 2024-04-07T17:51:25.780 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.780 INFO:tasks.workunit.client.0.smithi100.stdout: "op_out_bytes": { 2024-04-07T17:51:25.780 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3533359549 2024-04-07T17:51:25.780 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.780 INFO:tasks.workunit.client.0.smithi100.stdout: "op_prepare_latency": { 2024-04-07T17:51:25.780 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 59237, 2024-04-07T17:51:25.780 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 89091892647 2024-04-07T17:51:25.780 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.780 INFO:tasks.workunit.client.0.smithi100.stdout: "op_process_latency": { 2024-04-07T17:51:25.780 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 58391, 2024-04-07T17:51:25.781 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 156357104214 2024-04-07T17:51:25.781 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.781 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r": { 2024-04-07T17:51:25.781 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 29188 2024-04-07T17:51:25.781 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.781 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_latency": { 2024-04-07T17:51:25.781 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 29188, 2024-04-07T17:51:25.781 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 31668185766 2024-04-07T17:51:25.781 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.781 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_latency_out_bytes_histogram": { 2024-04-07T17:51:25.781 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.781 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.781 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_out_bytes": { 2024-04-07T17:51:25.781 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3533359549 2024-04-07T17:51:25.781 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.782 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_prepare_latency": { 2024-04-07T17:51:25.782 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 29608, 2024-04-07T17:51:25.782 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 13047146829 2024-04-07T17:51:25.782 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.782 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_process_latency": { 2024-04-07T17:51:25.782 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 29188, 2024-04-07T17:51:25.782 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 17316757778 2024-04-07T17:51:25.782 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.782 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw": { 2024-04-07T17:51:25.782 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 27288 2024-04-07T17:51:25.782 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.782 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_in_bytes": { 2024-04-07T17:51:25.782 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 535101355 2024-04-07T17:51:25.782 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.782 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_latency": { 2024-04-07T17:51:25.783 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 27288, 2024-04-07T17:51:25.783 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 125896770692 2024-04-07T17:51:25.783 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.783 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_latency_in_bytes_histogram": { 2024-04-07T17:51:25.783 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.783 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.783 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_latency_out_bytes_histogram": { 2024-04-07T17:51:25.783 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.783 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.783 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_out_bytes": { 2024-04-07T17:51:25.783 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.783 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.783 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_prepare_latency": { 2024-04-07T17:51:25.783 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 27539, 2024-04-07T17:51:25.783 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 69930450013 2024-04-07T17:51:25.783 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.783 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_process_latency": { 2024-04-07T17:51:25.784 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 27288, 2024-04-07T17:51:25.784 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 117911423955 2024-04-07T17:51:25.784 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.784 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w": { 2024-04-07T17:51:25.784 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1915 2024-04-07T17:51:25.784 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.784 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_in_bytes": { 2024-04-07T17:51:25.784 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1585674679 2024-04-07T17:51:25.784 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.784 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_latency": { 2024-04-07T17:51:25.784 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1915, 2024-04-07T17:51:25.784 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 29504179109 2024-04-07T17:51:25.784 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.784 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_latency_in_bytes_histogram": { 2024-04-07T17:51:25.784 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.784 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.784 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_prepare_latency": { 2024-04-07T17:51:25.785 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 2090, 2024-04-07T17:51:25.785 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 6114295805 2024-04-07T17:51:25.785 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.785 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_process_latency": { 2024-04-07T17:51:25.785 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1915, 2024-04-07T17:51:25.785 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 21128922481 2024-04-07T17:51:25.785 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.785 INFO:tasks.workunit.client.0.smithi100.stdout: "op_wip": { 2024-04-07T17:51:25.785 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.785 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.785 INFO:tasks.workunit.client.0.smithi100.stdout: "recovery_bytes": { 2024-04-07T17:51:25.785 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4194321 2024-04-07T17:51:25.785 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.785 INFO:tasks.workunit.client.0.smithi100.stdout: "recovery_ops": { 2024-04-07T17:51:25.785 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 38 2024-04-07T17:51:25.785 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.786 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_bytes": { 2024-04-07T17:51:25.786 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 95995035648 2024-04-07T17:51:25.786 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.786 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_bytes_used": { 2024-04-07T17:51:25.786 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 456179712 2024-04-07T17:51:25.786 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.786 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.786 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.786 INFO:tasks.workunit.client.0.smithi100.stdout: "osd.3": { 2024-04-07T17:51:25.786 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs": { 2024-04-07T17:51:25.786 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_db_max_lat": { 2024-04-07T17:51:25.786 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 29319 2024-04-07T17:51:25.786 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.786 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_slow_fallback": { 2024-04-07T17:51:25.787 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.787 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.787 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_slow_max_lat": { 2024-04-07T17:51:25.787 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.787 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.787 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_slow_size_fallback": { 2024-04-07T17:51:25.787 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.787 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.787 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit_db": { 2024-04-07T17:51:25.787 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 65536 2024-04-07T17:51:25.787 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.787 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit_slow": { 2024-04-07T17:51:25.787 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.787 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.787 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit_wal": { 2024-04-07T17:51:25.787 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.788 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.788 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_wal_max_lat": { 2024-04-07T17:51:25.788 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.788 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.788 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_written_slow": { 2024-04-07T17:51:25.788 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.788 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.788 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_written_sst": { 2024-04-07T17:51:25.788 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 6107136 2024-04-07T17:51:25.788 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.788 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_written_wal": { 2024-04-07T17:51:25.788 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 546639872 2024-04-07T17:51:25.788 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.788 INFO:tasks.workunit.client.0.smithi100.stdout: "compact_lat": { 2024-04-07T17:51:25.788 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 16, 2024-04-07T17:51:25.788 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5676850 2024-04-07T17:51:25.788 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.788 INFO:tasks.workunit.client.0.smithi100.stdout: "compact_lock_lat": { 2024-04-07T17:51:25.789 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 16, 2024-04-07T17:51:25.789 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1688739 2024-04-07T17:51:25.789 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.789 INFO:tasks.workunit.client.0.smithi100.stdout: "db_alloc_lat": { 2024-04-07T17:51:25.789 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 158, 2024-04-07T17:51:25.789 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1098247 2024-04-07T17:51:25.789 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.789 INFO:tasks.workunit.client.0.smithi100.stdout: "db_total_bytes": { 2024-04-07T17:51:25.789 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 95995027456 2024-04-07T17:51:25.789 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.789 INFO:tasks.workunit.client.0.smithi100.stdout: "db_used_bytes": { 2024-04-07T17:51:25.789 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 449314816 2024-04-07T17:51:25.789 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.789 INFO:tasks.workunit.client.0.smithi100.stdout: "flush_lat": { 2024-04-07T17:51:25.789 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 67796, 2024-04-07T17:51:25.789 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 12755738535 2024-04-07T17:51:25.789 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.790 INFO:tasks.workunit.client.0.smithi100.stdout: "fsync_lat": { 2024-04-07T17:51:25.790 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 67808, 2024-04-07T17:51:25.790 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 22453593571 2024-04-07T17:51:25.790 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.790 INFO:tasks.workunit.client.0.smithi100.stdout: "log_bytes": { 2024-04-07T17:51:25.790 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 11792384 2024-04-07T17:51:25.790 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.790 INFO:tasks.workunit.client.0.smithi100.stdout: "logged_bytes": { 2024-04-07T17:51:25.790 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 277786624 2024-04-07T17:51:25.790 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.790 INFO:tasks.workunit.client.0.smithi100.stdout: "max_bytes_db": { 2024-04-07T17:51:25.790 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 462094336 2024-04-07T17:51:25.790 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.790 INFO:tasks.workunit.client.0.smithi100.stdout: "max_bytes_slow": { 2024-04-07T17:51:25.790 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.790 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.791 INFO:tasks.workunit.client.0.smithi100.stdout: "max_bytes_wal": { 2024-04-07T17:51:25.791 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.791 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.791 INFO:tasks.workunit.client.0.smithi100.stdout: "num_files": { 2024-04-07T17:51:25.791 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 41 2024-04-07T17:51:25.791 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.791 INFO:tasks.workunit.client.0.smithi100.stdout: "read_bytes": { 2024-04-07T17:51:25.791 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5081425364 2024-04-07T17:51:25.791 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.791 INFO:tasks.workunit.client.0.smithi100.stdout: "read_count": { 2024-04-07T17:51:25.791 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 55136 2024-04-07T17:51:25.791 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.791 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes": { 2024-04-07T17:51:25.791 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4273045504 2024-04-07T17:51:25.791 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.791 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes_db": { 2024-04-07T17:51:25.792 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4273049600 2024-04-07T17:51:25.792 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.792 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes_slow": { 2024-04-07T17:51:25.792 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.792 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.792 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes_wal": { 2024-04-07T17:51:25.792 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.792 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.792 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_count": { 2024-04-07T17:51:25.792 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 47715 2024-04-07T17:51:25.792 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.792 INFO:tasks.workunit.client.0.smithi100.stdout: "read_lat": { 2024-04-07T17:51:25.792 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 55136, 2024-04-07T17:51:25.792 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2526773312 2024-04-07T17:51:25.792 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.792 INFO:tasks.workunit.client.0.smithi100.stdout: "read_prefetch_bytes": { 2024-04-07T17:51:25.792 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4957544695 2024-04-07T17:51:25.793 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.793 INFO:tasks.workunit.client.0.smithi100.stdout: "read_prefetch_count": { 2024-04-07T17:51:25.793 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 51339 2024-04-07T17:51:25.793 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.793 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_buffer_bytes": { 2024-04-07T17:51:25.793 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1547577 2024-04-07T17:51:25.793 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.793 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_buffer_count": { 2024-04-07T17:51:25.793 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 561 2024-04-07T17:51:25.793 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.793 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_bytes": { 2024-04-07T17:51:25.793 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 6055753 2024-04-07T17:51:25.793 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.793 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_count": { 2024-04-07T17:51:25.793 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 720 2024-04-07T17:51:25.793 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.793 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes": { 2024-04-07T17:51:25.794 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4508176 2024-04-07T17:51:25.794 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.794 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes_db": { 2024-04-07T17:51:25.794 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4508176 2024-04-07T17:51:25.794 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.794 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes_slow": { 2024-04-07T17:51:25.794 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.794 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.794 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes_wal": { 2024-04-07T17:51:25.794 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.794 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.794 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_count": { 2024-04-07T17:51:25.794 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 161 2024-04-07T17:51:25.794 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.794 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_lat": { 2024-04-07T17:51:25.795 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 720, 2024-04-07T17:51:25.795 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2153712 2024-04-07T17:51:25.795 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.795 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_alloc_lat": { 2024-04-07T17:51:25.795 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:25.795 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.795 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.795 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_total_bytes": { 2024-04-07T17:51:25.795 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.795 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.795 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_used_bytes": { 2024-04-07T17:51:25.795 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.795 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.795 INFO:tasks.workunit.client.0.smithi100.stdout: "truncate_lat": { 2024-04-07T17:51:25.795 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1, 2024-04-07T17:51:25.795 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 872 2024-04-07T17:51:25.796 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.796 INFO:tasks.workunit.client.0.smithi100.stdout: "unlink_lat": { 2024-04-07T17:51:25.796 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 10, 2024-04-07T17:51:25.796 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 15209 2024-04-07T17:51:25.796 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.796 INFO:tasks.workunit.client.0.smithi100.stdout: "wal_alloc_lat": { 2024-04-07T17:51:25.796 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:25.796 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.796 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.796 INFO:tasks.workunit.client.0.smithi100.stdout: "wal_total_bytes": { 2024-04-07T17:51:25.796 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.796 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.796 INFO:tasks.workunit.client.0.smithi100.stdout: "wal_used_bytes": { 2024-04-07T17:51:25.796 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.796 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.797 INFO:tasks.workunit.client.0.smithi100.stdout: "write_bytes": { 2024-04-07T17:51:25.797 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 832966656 2024-04-07T17:51:25.797 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.797 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.797 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore": { 2024-04-07T17:51:25.797 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit": { 2024-04-07T17:51:25.797 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4096 2024-04-07T17:51:25.797 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.797 INFO:tasks.workunit.client.0.smithi100.stdout: "allocated": { 2024-04-07T17:51:25.797 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 9826304 2024-04-07T17:51:25.797 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.797 INFO:tasks.workunit.client.0.smithi100.stdout: "allocator_lat": { 2024-04-07T17:51:25.797 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 19863, 2024-04-07T17:51:25.797 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 103164491 2024-04-07T17:51:25.797 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.797 INFO:tasks.workunit.client.0.smithi100.stdout: "clist_lat": { 2024-04-07T17:51:25.797 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 3584, 2024-04-07T17:51:25.797 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 10742765419 2024-04-07T17:51:25.798 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.798 INFO:tasks.workunit.client.0.smithi100.stdout: "compress_lat": { 2024-04-07T17:51:25.798 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:25.798 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.798 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.798 INFO:tasks.workunit.client.0.smithi100.stdout: "compressed": { 2024-04-07T17:51:25.798 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.798 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.798 INFO:tasks.workunit.client.0.smithi100.stdout: "compressed_allocated": { 2024-04-07T17:51:25.798 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.798 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.798 INFO:tasks.workunit.client.0.smithi100.stdout: "compressed_original": { 2024-04-07T17:51:25.798 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.798 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.798 INFO:tasks.workunit.client.0.smithi100.stdout: "csum_lat": { 2024-04-07T17:51:25.799 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 13277, 2024-04-07T17:51:25.799 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 158291808 2024-04-07T17:51:25.799 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.799 INFO:tasks.workunit.client.0.smithi100.stdout: "decompress_lat": { 2024-04-07T17:51:25.799 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:25.799 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.799 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.799 INFO:tasks.workunit.client.0.smithi100.stdout: "fragmentation_micros": { 2024-04-07T17:51:25.799 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.799 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.799 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_commit_lat": { 2024-04-07T17:51:25.799 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 67759, 2024-04-07T17:51:25.799 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 28639225606 2024-04-07T17:51:25.799 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.799 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_final_lat": { 2024-04-07T17:51:25.799 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 66833, 2024-04-07T17:51:25.799 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3952581972 2024-04-07T17:51:25.800 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.800 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_flush_lat": { 2024-04-07T17:51:25.800 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 67759, 2024-04-07T17:51:25.800 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 98642276 2024-04-07T17:51:25.800 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.800 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_sync_lat": { 2024-04-07T17:51:25.800 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 67759, 2024-04-07T17:51:25.800 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 28737867882 2024-04-07T17:51:25.800 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.800 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_get_keys_lat": { 2024-04-07T17:51:25.800 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:25.800 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.800 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.800 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_get_values_lat": { 2024-04-07T17:51:25.800 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 26953, 2024-04-07T17:51:25.800 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1128418885 2024-04-07T17:51:25.800 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.801 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_lower_bound_lat": { 2024-04-07T17:51:25.801 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1271, 2024-04-07T17:51:25.801 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 7079072 2024-04-07T17:51:25.801 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.801 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_next_lat": { 2024-04-07T17:51:25.801 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 8457, 2024-04-07T17:51:25.801 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 12008607 2024-04-07T17:51:25.801 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.801 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_seek_to_first_lat": { 2024-04-07T17:51:25.801 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 40, 2024-04-07T17:51:25.801 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 80474 2024-04-07T17:51:25.801 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.801 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_upper_bound_lat": { 2024-04-07T17:51:25.801 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 11692, 2024-04-07T17:51:25.801 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4518542429 2024-04-07T17:51:25.801 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.802 INFO:tasks.workunit.client.0.smithi100.stdout: "onode_hits": { 2024-04-07T17:51:25.802 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 208992 2024-04-07T17:51:25.802 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.802 INFO:tasks.workunit.client.0.smithi100.stdout: "onode_misses": { 2024-04-07T17:51:25.802 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 18294 2024-04-07T17:51:25.802 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.802 INFO:tasks.workunit.client.0.smithi100.stdout: "read_lat": { 2024-04-07T17:51:25.802 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 30350, 2024-04-07T17:51:25.802 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2036962015 2024-04-07T17:51:25.802 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.802 INFO:tasks.workunit.client.0.smithi100.stdout: "read_onode_meta_lat": { 2024-04-07T17:51:25.802 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 80371, 2024-04-07T17:51:25.802 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 45758452 2024-04-07T17:51:25.802 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.803 INFO:tasks.workunit.client.0.smithi100.stdout: "read_wait_aio_lat": { 2024-04-07T17:51:25.803 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 50021, 2024-04-07T17:51:25.803 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1142513586 2024-04-07T17:51:25.803 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.803 INFO:tasks.workunit.client.0.smithi100.stdout: "reads_with_retries": { 2024-04-07T17:51:25.803 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.803 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.803 INFO:tasks.workunit.client.0.smithi100.stdout: "remove_lat": { 2024-04-07T17:51:25.803 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 12382, 2024-04-07T17:51:25.803 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 706656351 2024-04-07T17:51:25.803 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.803 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_aio_wait_count": { 2024-04-07T17:51:25.803 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.803 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.803 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_committed_kv_count": { 2024-04-07T17:51:25.803 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.803 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.804 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_read_onode_meta_count": { 2024-04-07T17:51:25.804 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.804 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.804 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_read_wait_aio_count": { 2024-04-07T17:51:25.804 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.804 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.804 INFO:tasks.workunit.client.0.smithi100.stdout: "state_aio_wait_lat": { 2024-04-07T17:51:25.804 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 77291, 2024-04-07T17:51:25.804 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 12556920270 2024-04-07T17:51:25.804 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.804 INFO:tasks.workunit.client.0.smithi100.stdout: "state_deferred_aio_wait_lat": { 2024-04-07T17:51:25.804 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 17849, 2024-04-07T17:51:25.804 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1593523661 2024-04-07T17:51:25.804 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.804 INFO:tasks.workunit.client.0.smithi100.stdout: "state_deferred_cleanup_lat": { 2024-04-07T17:51:25.804 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 17848, 2024-04-07T17:51:25.804 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3943299065015 2024-04-07T17:51:25.804 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.805 INFO:tasks.workunit.client.0.smithi100.stdout: "state_deferred_queued_lat": { 2024-04-07T17:51:25.805 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 17849, 2024-04-07T17:51:25.805 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 6510467867246 2024-04-07T17:51:25.805 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.805 INFO:tasks.workunit.client.0.smithi100.stdout: "state_done_lat": { 2024-04-07T17:51:25.805 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 77290, 2024-04-07T17:51:25.805 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2750364291611 2024-04-07T17:51:25.805 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.805 INFO:tasks.workunit.client.0.smithi100.stdout: "state_finishing_lat": { 2024-04-07T17:51:25.805 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 77290, 2024-04-07T17:51:25.805 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 16788531 2024-04-07T17:51:25.805 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.805 INFO:tasks.workunit.client.0.smithi100.stdout: "state_io_done_lat": { 2024-04-07T17:51:25.805 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 77291, 2024-04-07T17:51:25.805 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 461265229 2024-04-07T17:51:25.805 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.805 INFO:tasks.workunit.client.0.smithi100.stdout: "state_kv_commiting_lat": { 2024-04-07T17:51:25.806 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 77291, 2024-04-07T17:51:25.806 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 50377334346 2024-04-07T17:51:25.806 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.806 INFO:tasks.workunit.client.0.smithi100.stdout: "state_kv_done_lat": { 2024-04-07T17:51:25.806 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 77291, 2024-04-07T17:51:25.806 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 62405628 2024-04-07T17:51:25.806 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.806 INFO:tasks.workunit.client.0.smithi100.stdout: "state_kv_queued_lat": { 2024-04-07T17:51:25.806 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 77291, 2024-04-07T17:51:25.806 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 24119810266 2024-04-07T17:51:25.806 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.806 INFO:tasks.workunit.client.0.smithi100.stdout: "state_prepare_lat": { 2024-04-07T17:51:25.806 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 77291, 2024-04-07T17:51:25.807 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 26664283551 2024-04-07T17:51:25.807 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.807 INFO:tasks.workunit.client.0.smithi100.stdout: "stored": { 2024-04-07T17:51:25.807 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5705610 2024-04-07T17:51:25.807 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.807 INFO:tasks.workunit.client.0.smithi100.stdout: "truncate_lat": { 2024-04-07T17:51:25.807 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 2398, 2024-04-07T17:51:25.807 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 27539717 2024-04-07T17:51:25.807 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.807 INFO:tasks.workunit.client.0.smithi100.stdout: "txc_commit_lat": { 2024-04-07T17:51:25.807 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 77291, 2024-04-07T17:51:25.807 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 114204848603 2024-04-07T17:51:25.807 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.807 INFO:tasks.workunit.client.0.smithi100.stdout: "txc_submit_lat": { 2024-04-07T17:51:25.807 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 77291, 2024-04-07T17:51:25.808 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 28995862182 2024-04-07T17:51:25.808 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.808 INFO:tasks.workunit.client.0.smithi100.stdout: "txc_throttle_lat": { 2024-04-07T17:51:25.808 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 77291, 2024-04-07T17:51:25.808 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1079109304 2024-04-07T17:51:25.808 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.808 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.808 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache": { 2024-04-07T17:51:25.808 INFO:tasks.workunit.client.0.smithi100.stdout: "cache_bytes": { 2024-04-07T17:51:25.808 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1444614092 2024-04-07T17:51:25.808 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.808 INFO:tasks.workunit.client.0.smithi100.stdout: "heap_bytes": { 2024-04-07T17:51:25.808 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 503775232 2024-04-07T17:51:25.808 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.808 INFO:tasks.workunit.client.0.smithi100.stdout: "mapped_bytes": { 2024-04-07T17:51:25.808 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 470474752 2024-04-07T17:51:25.808 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.809 INFO:tasks.workunit.client.0.smithi100.stdout: "target_bytes": { 2024-04-07T17:51:25.809 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2646965248 2024-04-07T17:51:25.809 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.809 INFO:tasks.workunit.client.0.smithi100.stdout: "unmapped_bytes": { 2024-04-07T17:51:25.809 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 33300480 2024-04-07T17:51:25.809 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.809 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.809 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:data": { 2024-04-07T17:51:25.809 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:25.809 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 134217728 2024-04-07T17:51:25.809 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.809 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:25.809 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.809 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.809 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:25.809 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.809 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.810 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:25.810 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 59559460 2024-04-07T17:51:25.810 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.810 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:25.810 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.810 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.810 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:25.810 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.810 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.810 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:25.810 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.810 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.810 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:25.810 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.810 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.811 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:25.811 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 8192 2024-04-07T17:51:25.811 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.811 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:25.811 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 102400 2024-04-07T17:51:25.811 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.811 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:25.811 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.811 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.811 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:25.811 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.811 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.811 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:25.811 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.811 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.811 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:25.811 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 74547676 2024-04-07T17:51:25.812 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.812 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.812 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:kv": { 2024-04-07T17:51:25.812 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:25.812 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 587202560 2024-04-07T17:51:25.812 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.812 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:25.812 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 67108864 2024-04-07T17:51:25.812 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.812 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:25.812 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.812 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.812 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:25.812 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 446695953 2024-04-07T17:51:25.812 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.812 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:25.812 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.813 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.813 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:25.813 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1776 2024-04-07T17:51:25.813 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.813 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:25.813 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.813 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.813 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:25.813 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.813 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.813 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:25.813 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 110304 2024-04-07T17:51:25.813 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.813 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:25.813 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.813 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.813 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:25.814 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.814 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.814 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:25.814 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.814 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.814 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:25.814 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.814 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.814 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:25.814 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 73285663 2024-04-07T17:51:25.814 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.814 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.814 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:kv_onode": { 2024-04-07T17:51:25.814 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:25.814 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 184549376 2024-04-07T17:51:25.815 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.815 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:25.815 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 67108864 2024-04-07T17:51:25.815 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.815 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:25.815 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5620784 2024-04-07T17:51:25.815 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.815 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:25.815 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 39706307 2024-04-07T17:51:25.815 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.815 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:25.815 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.815 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.815 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:25.815 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.815 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.815 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:25.816 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.816 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.816 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:25.816 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.816 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.816 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:25.816 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.816 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.816 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:25.816 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.816 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.816 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:25.816 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.816 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.816 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:25.816 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.816 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.817 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:25.817 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.817 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.817 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:25.817 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 72113421 2024-04-07T17:51:25.817 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.817 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.817 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:meta": { 2024-04-07T17:51:25.817 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:25.817 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 520093696 2024-04-07T17:51:25.817 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.817 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:25.817 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.817 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.817 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:25.817 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.817 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.818 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:25.818 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 446695953 2024-04-07T17:51:25.818 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.818 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:25.818 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.818 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.818 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:25.818 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.818 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.818 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:25.818 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.818 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.818 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:25.818 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.818 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.818 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:25.819 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 151009 2024-04-07T17:51:25.819 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.819 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:25.819 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1112438 2024-04-07T17:51:25.819 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.819 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:25.819 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.819 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.819 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:25.819 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.819 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.819 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:25.819 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.819 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.819 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:25.819 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 72134296 2024-04-07T17:51:25.819 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.820 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.820 INFO:tasks.workunit.client.0.smithi100.stdout: "objecter": { 2024-04-07T17:51:25.820 INFO:tasks.workunit.client.0.smithi100.stdout: "op_active": { 2024-04-07T17:51:25.820 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.820 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.820 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r": { 2024-04-07T17:51:25.820 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 18 2024-04-07T17:51:25.820 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.820 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rmw": { 2024-04-07T17:51:25.820 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.820 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.820 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w": { 2024-04-07T17:51:25.820 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.820 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.820 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.820 INFO:tasks.workunit.client.0.smithi100.stdout: "osd": { 2024-04-07T17:51:25.820 INFO:tasks.workunit.client.0.smithi100.stdout: "numpg": { 2024-04-07T17:51:25.820 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 47 2024-04-07T17:51:25.821 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.821 INFO:tasks.workunit.client.0.smithi100.stdout: "numpg_removing": { 2024-04-07T17:51:25.821 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.821 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.821 INFO:tasks.workunit.client.0.smithi100.stdout: "op": { 2024-04-07T17:51:25.821 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 56964 2024-04-07T17:51:25.821 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.821 INFO:tasks.workunit.client.0.smithi100.stdout: "op_before_queue_op_lat": { 2024-04-07T17:51:25.821 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 133884, 2024-04-07T17:51:25.821 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 12292525237 2024-04-07T17:51:25.821 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.821 INFO:tasks.workunit.client.0.smithi100.stdout: "op_delayed_degraded": { 2024-04-07T17:51:25.821 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.821 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.821 INFO:tasks.workunit.client.0.smithi100.stdout: "op_delayed_unreadable": { 2024-04-07T17:51:25.821 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.822 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.822 INFO:tasks.workunit.client.0.smithi100.stdout: "op_in_bytes": { 2024-04-07T17:51:25.822 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1485365521 2024-04-07T17:51:25.822 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.822 INFO:tasks.workunit.client.0.smithi100.stdout: "op_latency": { 2024-04-07T17:51:25.822 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 56964, 2024-04-07T17:51:25.822 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 394646376159 2024-04-07T17:51:25.822 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.822 INFO:tasks.workunit.client.0.smithi100.stdout: "op_out_bytes": { 2024-04-07T17:51:25.822 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1430791176 2024-04-07T17:51:25.822 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.822 INFO:tasks.workunit.client.0.smithi100.stdout: "op_prepare_latency": { 2024-04-07T17:51:25.822 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 59216, 2024-04-07T17:51:25.822 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 48205385292 2024-04-07T17:51:25.823 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.823 INFO:tasks.workunit.client.0.smithi100.stdout: "op_process_latency": { 2024-04-07T17:51:25.823 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 56964, 2024-04-07T17:51:25.823 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 169980516116 2024-04-07T17:51:25.823 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.823 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r": { 2024-04-07T17:51:25.823 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 39255 2024-04-07T17:51:25.823 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.823 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_latency": { 2024-04-07T17:51:25.823 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 39255, 2024-04-07T17:51:25.823 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 64075850096 2024-04-07T17:51:25.823 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.823 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_latency_out_bytes_histogram": { 2024-04-07T17:51:25.823 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.823 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.823 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_out_bytes": { 2024-04-07T17:51:25.823 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1430791176 2024-04-07T17:51:25.824 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.824 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_prepare_latency": { 2024-04-07T17:51:25.824 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 39480, 2024-04-07T17:51:25.824 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 16088819423 2024-04-07T17:51:25.824 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.824 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_process_latency": { 2024-04-07T17:51:25.824 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 39255, 2024-04-07T17:51:25.824 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 13918274599 2024-04-07T17:51:25.824 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.824 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw": { 2024-04-07T17:51:25.824 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 13767 2024-04-07T17:51:25.824 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.824 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_in_bytes": { 2024-04-07T17:51:25.824 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 309949184 2024-04-07T17:51:25.824 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.824 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_latency": { 2024-04-07T17:51:25.824 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 13767, 2024-04-07T17:51:25.825 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 275992837370 2024-04-07T17:51:25.825 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.825 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_latency_in_bytes_histogram": { 2024-04-07T17:51:25.825 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.825 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.825 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_latency_out_bytes_histogram": { 2024-04-07T17:51:25.825 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.825 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.825 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_out_bytes": { 2024-04-07T17:51:25.825 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.825 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.825 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_prepare_latency": { 2024-04-07T17:51:25.825 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 15694, 2024-04-07T17:51:25.825 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 23263942693 2024-04-07T17:51:25.825 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.825 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_process_latency": { 2024-04-07T17:51:25.826 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 13767, 2024-04-07T17:51:25.826 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 118411226197 2024-04-07T17:51:25.826 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.826 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w": { 2024-04-07T17:51:25.826 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3942 2024-04-07T17:51:25.826 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.826 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_in_bytes": { 2024-04-07T17:51:25.826 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1175416337 2024-04-07T17:51:25.826 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.826 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_latency": { 2024-04-07T17:51:25.826 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 3942, 2024-04-07T17:51:25.826 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 54577688693 2024-04-07T17:51:25.826 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.826 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_latency_in_bytes_histogram": { 2024-04-07T17:51:25.827 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.827 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.827 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_prepare_latency": { 2024-04-07T17:51:25.827 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 4042, 2024-04-07T17:51:25.827 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 8852623176 2024-04-07T17:51:25.827 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.827 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_process_latency": { 2024-04-07T17:51:25.827 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 3942, 2024-04-07T17:51:25.827 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 37651015320 2024-04-07T17:51:25.827 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.827 INFO:tasks.workunit.client.0.smithi100.stdout: "op_wip": { 2024-04-07T17:51:25.827 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.827 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.827 INFO:tasks.workunit.client.0.smithi100.stdout: "recovery_bytes": { 2024-04-07T17:51:25.827 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 35 2024-04-07T17:51:25.827 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.827 INFO:tasks.workunit.client.0.smithi100.stdout: "recovery_ops": { 2024-04-07T17:51:25.828 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3 2024-04-07T17:51:25.828 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.828 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_bytes": { 2024-04-07T17:51:25.828 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 95995035648 2024-04-07T17:51:25.828 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.828 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_bytes_used": { 2024-04-07T17:51:25.828 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 459153408 2024-04-07T17:51:25.828 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.828 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.828 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.828 INFO:tasks.workunit.client.0.smithi100.stdout: "osd.4": { 2024-04-07T17:51:25.828 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs": { 2024-04-07T17:51:25.828 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_db_max_lat": { 2024-04-07T17:51:25.828 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 53233 2024-04-07T17:51:25.829 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.829 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_slow_fallback": { 2024-04-07T17:51:25.829 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.829 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.829 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_slow_max_lat": { 2024-04-07T17:51:25.829 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.829 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.829 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_slow_size_fallback": { 2024-04-07T17:51:25.829 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.829 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.829 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit_db": { 2024-04-07T17:51:25.829 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 65536 2024-04-07T17:51:25.829 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.829 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit_slow": { 2024-04-07T17:51:25.829 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.829 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.830 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit_wal": { 2024-04-07T17:51:25.830 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.830 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.830 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_wal_max_lat": { 2024-04-07T17:51:25.830 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.830 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.830 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_written_slow": { 2024-04-07T17:51:25.830 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.830 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.830 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_written_sst": { 2024-04-07T17:51:25.830 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 14217216 2024-04-07T17:51:25.830 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.830 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_written_wal": { 2024-04-07T17:51:25.830 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 588619776 2024-04-07T17:51:25.830 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.831 INFO:tasks.workunit.client.0.smithi100.stdout: "compact_lat": { 2024-04-07T17:51:25.831 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 14, 2024-04-07T17:51:25.831 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5250958 2024-04-07T17:51:25.831 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.831 INFO:tasks.workunit.client.0.smithi100.stdout: "compact_lock_lat": { 2024-04-07T17:51:25.831 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 14, 2024-04-07T17:51:25.831 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1869555 2024-04-07T17:51:25.831 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.831 INFO:tasks.workunit.client.0.smithi100.stdout: "db_alloc_lat": { 2024-04-07T17:51:25.831 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 167, 2024-04-07T17:51:25.831 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1195870 2024-04-07T17:51:25.831 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.831 INFO:tasks.workunit.client.0.smithi100.stdout: "db_total_bytes": { 2024-04-07T17:51:25.831 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 95995027456 2024-04-07T17:51:25.831 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.831 INFO:tasks.workunit.client.0.smithi100.stdout: "db_used_bytes": { 2024-04-07T17:51:25.832 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 572653568 2024-04-07T17:51:25.832 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.832 INFO:tasks.workunit.client.0.smithi100.stdout: "flush_lat": { 2024-04-07T17:51:25.832 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 60244, 2024-04-07T17:51:25.832 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 12173027260 2024-04-07T17:51:25.832 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.832 INFO:tasks.workunit.client.0.smithi100.stdout: "fsync_lat": { 2024-04-07T17:51:25.832 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 60248, 2024-04-07T17:51:25.832 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 21497842884 2024-04-07T17:51:25.832 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.832 INFO:tasks.workunit.client.0.smithi100.stdout: "log_bytes": { 2024-04-07T17:51:25.832 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 14090240 2024-04-07T17:51:25.832 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.832 INFO:tasks.workunit.client.0.smithi100.stdout: "logged_bytes": { 2024-04-07T17:51:25.832 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 246812672 2024-04-07T17:51:25.832 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.833 INFO:tasks.workunit.client.0.smithi100.stdout: "max_bytes_db": { 2024-04-07T17:51:25.833 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 568393728 2024-04-07T17:51:25.833 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.833 INFO:tasks.workunit.client.0.smithi100.stdout: "max_bytes_slow": { 2024-04-07T17:51:25.833 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.833 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.833 INFO:tasks.workunit.client.0.smithi100.stdout: "max_bytes_wal": { 2024-04-07T17:51:25.833 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.833 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.833 INFO:tasks.workunit.client.0.smithi100.stdout: "num_files": { 2024-04-07T17:51:25.833 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 50 2024-04-07T17:51:25.833 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.833 INFO:tasks.workunit.client.0.smithi100.stdout: "read_bytes": { 2024-04-07T17:51:25.833 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 10796585872 2024-04-07T17:51:25.833 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.834 INFO:tasks.workunit.client.0.smithi100.stdout: "read_count": { 2024-04-07T17:51:25.834 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 87890 2024-04-07T17:51:25.834 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.834 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes": { 2024-04-07T17:51:25.834 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 9426481152 2024-04-07T17:51:25.834 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.834 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes_db": { 2024-04-07T17:51:25.834 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 9426485248 2024-04-07T17:51:25.834 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.834 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes_slow": { 2024-04-07T17:51:25.834 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.834 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.834 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes_wal": { 2024-04-07T17:51:25.834 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.835 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.835 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_count": { 2024-04-07T17:51:25.835 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 80816 2024-04-07T17:51:25.835 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.835 INFO:tasks.workunit.client.0.smithi100.stdout: "read_lat": { 2024-04-07T17:51:25.835 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 87890, 2024-04-07T17:51:25.835 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5954312472 2024-04-07T17:51:25.835 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.835 INFO:tasks.workunit.client.0.smithi100.stdout: "read_prefetch_bytes": { 2024-04-07T17:51:25.835 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 10604152323 2024-04-07T17:51:25.835 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.835 INFO:tasks.workunit.client.0.smithi100.stdout: "read_prefetch_count": { 2024-04-07T17:51:25.835 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 82000 2024-04-07T17:51:25.835 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.835 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_buffer_bytes": { 2024-04-07T17:51:25.835 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4778070 2024-04-07T17:51:25.836 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.836 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_buffer_count": { 2024-04-07T17:51:25.836 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 805 2024-04-07T17:51:25.836 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.836 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_bytes": { 2024-04-07T17:51:25.836 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 14103486 2024-04-07T17:51:25.836 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.836 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_count": { 2024-04-07T17:51:25.836 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1149 2024-04-07T17:51:25.836 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.836 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes": { 2024-04-07T17:51:25.836 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 9325416 2024-04-07T17:51:25.836 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.836 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes_db": { 2024-04-07T17:51:25.836 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 9325416 2024-04-07T17:51:25.837 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.837 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes_slow": { 2024-04-07T17:51:25.837 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.837 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.837 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes_wal": { 2024-04-07T17:51:25.837 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.837 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.837 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_count": { 2024-04-07T17:51:25.837 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 361 2024-04-07T17:51:25.837 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.837 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_lat": { 2024-04-07T17:51:25.837 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1149, 2024-04-07T17:51:25.837 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4714569 2024-04-07T17:51:25.837 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.837 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_alloc_lat": { 2024-04-07T17:51:25.838 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:25.838 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.838 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.838 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_total_bytes": { 2024-04-07T17:51:25.838 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.838 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.838 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_used_bytes": { 2024-04-07T17:51:25.838 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.838 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.838 INFO:tasks.workunit.client.0.smithi100.stdout: "truncate_lat": { 2024-04-07T17:51:25.838 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1, 2024-04-07T17:51:25.838 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 731 2024-04-07T17:51:25.838 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.838 INFO:tasks.workunit.client.0.smithi100.stdout: "unlink_lat": { 2024-04-07T17:51:25.838 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 14, 2024-04-07T17:51:25.839 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 19309 2024-04-07T17:51:25.839 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.839 INFO:tasks.workunit.client.0.smithi100.stdout: "wal_alloc_lat": { 2024-04-07T17:51:25.839 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:25.839 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.839 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.839 INFO:tasks.workunit.client.0.smithi100.stdout: "wal_total_bytes": { 2024-04-07T17:51:25.839 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.839 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.839 INFO:tasks.workunit.client.0.smithi100.stdout: "wal_used_bytes": { 2024-04-07T17:51:25.839 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.839 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.839 INFO:tasks.workunit.client.0.smithi100.stdout: "write_bytes": { 2024-04-07T17:51:25.839 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 851795968 2024-04-07T17:51:25.839 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.839 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.840 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore": { 2024-04-07T17:51:25.840 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit": { 2024-04-07T17:51:25.840 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4096 2024-04-07T17:51:25.840 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.840 INFO:tasks.workunit.client.0.smithi100.stdout: "allocated": { 2024-04-07T17:51:25.840 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 14016512 2024-04-07T17:51:25.840 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.840 INFO:tasks.workunit.client.0.smithi100.stdout: "allocator_lat": { 2024-04-07T17:51:25.840 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 20261, 2024-04-07T17:51:25.840 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 99191054 2024-04-07T17:51:25.840 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.840 INFO:tasks.workunit.client.0.smithi100.stdout: "clist_lat": { 2024-04-07T17:51:25.840 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 3532, 2024-04-07T17:51:25.840 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 14554851866 2024-04-07T17:51:25.840 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.840 INFO:tasks.workunit.client.0.smithi100.stdout: "compress_lat": { 2024-04-07T17:51:25.841 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:25.841 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.841 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.841 INFO:tasks.workunit.client.0.smithi100.stdout: "compressed": { 2024-04-07T17:51:25.841 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.841 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.841 INFO:tasks.workunit.client.0.smithi100.stdout: "compressed_allocated": { 2024-04-07T17:51:25.841 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.841 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.841 INFO:tasks.workunit.client.0.smithi100.stdout: "compressed_original": { 2024-04-07T17:51:25.841 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.841 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.841 INFO:tasks.workunit.client.0.smithi100.stdout: "csum_lat": { 2024-04-07T17:51:25.841 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 12243, 2024-04-07T17:51:25.841 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 157875076 2024-04-07T17:51:25.842 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.842 INFO:tasks.workunit.client.0.smithi100.stdout: "decompress_lat": { 2024-04-07T17:51:25.842 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:25.842 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.842 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.842 INFO:tasks.workunit.client.0.smithi100.stdout: "fragmentation_micros": { 2024-04-07T17:51:25.842 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.842 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.842 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_commit_lat": { 2024-04-07T17:51:25.842 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 60186, 2024-04-07T17:51:25.842 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 27783488280 2024-04-07T17:51:25.842 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.842 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_final_lat": { 2024-04-07T17:51:25.842 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 59468, 2024-04-07T17:51:25.842 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3239794550 2024-04-07T17:51:25.843 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.843 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_flush_lat": { 2024-04-07T17:51:25.843 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 60186, 2024-04-07T17:51:25.843 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 106527146 2024-04-07T17:51:25.843 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.843 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_sync_lat": { 2024-04-07T17:51:25.843 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 60186, 2024-04-07T17:51:25.843 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 27890015426 2024-04-07T17:51:25.843 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.843 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_get_keys_lat": { 2024-04-07T17:51:25.843 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:25.843 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.843 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.843 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_get_values_lat": { 2024-04-07T17:51:25.843 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 22645, 2024-04-07T17:51:25.844 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1026880183 2024-04-07T17:51:25.844 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.844 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_lower_bound_lat": { 2024-04-07T17:51:25.844 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 697, 2024-04-07T17:51:25.844 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4156713 2024-04-07T17:51:25.844 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.844 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_next_lat": { 2024-04-07T17:51:25.844 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 13851, 2024-04-07T17:51:25.844 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 16294854 2024-04-07T17:51:25.844 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.844 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_seek_to_first_lat": { 2024-04-07T17:51:25.844 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 47, 2024-04-07T17:51:25.844 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 58460 2024-04-07T17:51:25.844 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.844 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_upper_bound_lat": { 2024-04-07T17:51:25.844 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 14286, 2024-04-07T17:51:25.845 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 6817372199 2024-04-07T17:51:25.845 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.845 INFO:tasks.workunit.client.0.smithi100.stdout: "onode_hits": { 2024-04-07T17:51:25.845 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 175006 2024-04-07T17:51:25.845 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.845 INFO:tasks.workunit.client.0.smithi100.stdout: "onode_misses": { 2024-04-07T17:51:25.845 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 19274 2024-04-07T17:51:25.845 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.845 INFO:tasks.workunit.client.0.smithi100.stdout: "read_lat": { 2024-04-07T17:51:25.845 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 8345, 2024-04-07T17:51:25.845 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1757977707 2024-04-07T17:51:25.845 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.845 INFO:tasks.workunit.client.0.smithi100.stdout: "read_onode_meta_lat": { 2024-04-07T17:51:25.845 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 35241, 2024-04-07T17:51:25.845 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 18704077 2024-04-07T17:51:25.845 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.846 INFO:tasks.workunit.client.0.smithi100.stdout: "read_wait_aio_lat": { 2024-04-07T17:51:25.846 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 26896, 2024-04-07T17:51:25.846 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1180234012 2024-04-07T17:51:25.846 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.846 INFO:tasks.workunit.client.0.smithi100.stdout: "reads_with_retries": { 2024-04-07T17:51:25.846 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.846 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.846 INFO:tasks.workunit.client.0.smithi100.stdout: "remove_lat": { 2024-04-07T17:51:25.846 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 13430, 2024-04-07T17:51:25.846 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 730792101 2024-04-07T17:51:25.846 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.846 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_aio_wait_count": { 2024-04-07T17:51:25.846 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.846 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.847 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_committed_kv_count": { 2024-04-07T17:51:25.847 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.847 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.847 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_read_onode_meta_count": { 2024-04-07T17:51:25.847 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.847 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.847 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_read_wait_aio_count": { 2024-04-07T17:51:25.847 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.847 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.847 INFO:tasks.workunit.client.0.smithi100.stdout: "state_aio_wait_lat": { 2024-04-07T17:51:25.847 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 69678, 2024-04-07T17:51:25.847 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 8516743959 2024-04-07T17:51:25.847 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.847 INFO:tasks.workunit.client.0.smithi100.stdout: "state_deferred_aio_wait_lat": { 2024-04-07T17:51:25.847 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 18006, 2024-04-07T17:51:25.847 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1525120051 2024-04-07T17:51:25.848 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.848 INFO:tasks.workunit.client.0.smithi100.stdout: "state_deferred_cleanup_lat": { 2024-04-07T17:51:25.848 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 18005, 2024-04-07T17:51:25.848 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4347507639061 2024-04-07T17:51:25.848 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.848 INFO:tasks.workunit.client.0.smithi100.stdout: "state_deferred_queued_lat": { 2024-04-07T17:51:25.848 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 18006, 2024-04-07T17:51:25.848 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 6594884467617 2024-04-07T17:51:25.848 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.848 INFO:tasks.workunit.client.0.smithi100.stdout: "state_done_lat": { 2024-04-07T17:51:25.848 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 69677, 2024-04-07T17:51:25.848 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3006179254631 2024-04-07T17:51:25.848 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.848 INFO:tasks.workunit.client.0.smithi100.stdout: "state_finishing_lat": { 2024-04-07T17:51:25.848 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 69677, 2024-04-07T17:51:25.848 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 16535015 2024-04-07T17:51:25.849 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.849 INFO:tasks.workunit.client.0.smithi100.stdout: "state_io_done_lat": { 2024-04-07T17:51:25.849 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 69678, 2024-04-07T17:51:25.849 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 283438924 2024-04-07T17:51:25.849 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.849 INFO:tasks.workunit.client.0.smithi100.stdout: "state_kv_commiting_lat": { 2024-04-07T17:51:25.849 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 69678, 2024-04-07T17:51:25.849 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 46634763201 2024-04-07T17:51:25.849 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.849 INFO:tasks.workunit.client.0.smithi100.stdout: "state_kv_done_lat": { 2024-04-07T17:51:25.849 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 69678, 2024-04-07T17:51:25.849 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 54811413 2024-04-07T17:51:25.849 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.849 INFO:tasks.workunit.client.0.smithi100.stdout: "state_kv_queued_lat": { 2024-04-07T17:51:25.849 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 69678, 2024-04-07T17:51:25.849 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 21669658103 2024-04-07T17:51:25.850 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.850 INFO:tasks.workunit.client.0.smithi100.stdout: "state_prepare_lat": { 2024-04-07T17:51:25.850 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 69678, 2024-04-07T17:51:25.850 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 25344251701 2024-04-07T17:51:25.850 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.850 INFO:tasks.workunit.client.0.smithi100.stdout: "stored": { 2024-04-07T17:51:25.850 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 9894713 2024-04-07T17:51:25.850 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.850 INFO:tasks.workunit.client.0.smithi100.stdout: "truncate_lat": { 2024-04-07T17:51:25.850 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 2371, 2024-04-07T17:51:25.850 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 24012279 2024-04-07T17:51:25.850 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.850 INFO:tasks.workunit.client.0.smithi100.stdout: "txc_commit_lat": { 2024-04-07T17:51:25.850 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 69678, 2024-04-07T17:51:25.850 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 102473756098 2024-04-07T17:51:25.851 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.851 INFO:tasks.workunit.client.0.smithi100.stdout: "txc_submit_lat": { 2024-04-07T17:51:25.851 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 69678, 2024-04-07T17:51:25.851 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 27569044264 2024-04-07T17:51:25.851 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.851 INFO:tasks.workunit.client.0.smithi100.stdout: "txc_throttle_lat": { 2024-04-07T17:51:25.851 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 69678, 2024-04-07T17:51:25.851 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 877546314 2024-04-07T17:51:25.851 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.851 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.851 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache": { 2024-04-07T17:51:25.851 INFO:tasks.workunit.client.0.smithi100.stdout: "cache_bytes": { 2024-04-07T17:51:25.851 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1444650649 2024-04-07T17:51:25.851 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.851 INFO:tasks.workunit.client.0.smithi100.stdout: "heap_bytes": { 2024-04-07T17:51:25.852 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 623263744 2024-04-07T17:51:25.852 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.852 INFO:tasks.workunit.client.0.smithi100.stdout: "mapped_bytes": { 2024-04-07T17:51:25.852 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 593174528 2024-04-07T17:51:25.852 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.852 INFO:tasks.workunit.client.0.smithi100.stdout: "target_bytes": { 2024-04-07T17:51:25.852 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2647008256 2024-04-07T17:51:25.852 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.852 INFO:tasks.workunit.client.0.smithi100.stdout: "unmapped_bytes": { 2024-04-07T17:51:25.852 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 30089216 2024-04-07T17:51:25.852 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.852 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.852 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:data": { 2024-04-07T17:51:25.852 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:25.852 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 134217728 2024-04-07T17:51:25.853 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.853 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:25.853 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.853 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.853 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:25.853 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.853 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.853 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:25.853 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 58883623 2024-04-07T17:51:25.853 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.853 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:25.853 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.853 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.853 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:25.853 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.854 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.854 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:25.854 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.854 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.854 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:25.854 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.854 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.854 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:25.854 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 28672 2024-04-07T17:51:25.854 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.854 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:25.854 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 757760 2024-04-07T17:51:25.854 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.854 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:25.854 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.855 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.855 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:25.855 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.855 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.855 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:25.855 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.855 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.855 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:25.855 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 74547673 2024-04-07T17:51:25.855 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.855 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.855 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:kv": { 2024-04-07T17:51:25.855 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:25.855 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 578813952 2024-04-07T17:51:25.855 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.855 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:25.856 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 67108864 2024-04-07T17:51:25.856 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.856 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:25.856 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.856 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.856 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:25.856 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 441627178 2024-04-07T17:51:25.856 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.856 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:25.856 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.856 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.856 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:25.856 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.856 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.856 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:25.857 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2416 2024-04-07T17:51:25.857 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.857 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:25.857 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.857 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.857 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:25.857 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 153616 2024-04-07T17:51:25.857 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.857 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:25.857 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.857 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.857 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:25.857 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.857 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.857 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:25.857 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.857 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.858 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:25.858 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.858 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.858 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:25.858 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 69921878 2024-04-07T17:51:25.858 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.858 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.858 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:kv_onode": { 2024-04-07T17:51:25.858 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:25.858 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 192937984 2024-04-07T17:51:25.858 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.858 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:25.858 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 67108864 2024-04-07T17:51:25.858 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.859 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:25.859 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 13844688 2024-04-07T17:51:25.859 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.859 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:25.859 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 39255749 2024-04-07T17:51:25.859 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.859 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:25.859 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.859 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.859 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:25.859 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.859 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.859 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:25.859 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.859 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.859 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:25.860 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.860 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.860 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:25.860 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.860 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.860 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:25.860 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.860 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.860 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:25.860 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.860 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.860 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:25.860 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.860 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.860 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:25.860 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.861 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.861 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:25.861 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 72728683 2024-04-07T17:51:25.861 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.861 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.861 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:meta": { 2024-04-07T17:51:25.861 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:25.861 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 511705088 2024-04-07T17:51:25.861 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.861 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:25.861 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.861 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.861 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:25.861 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.861 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.862 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:25.862 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 441627178 2024-04-07T17:51:25.862 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.862 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:25.862 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.862 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.862 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:25.862 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.862 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.862 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:25.862 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.862 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.862 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:25.862 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.862 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.863 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:25.863 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 158033 2024-04-07T17:51:25.863 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.863 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:25.863 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1110235 2024-04-07T17:51:25.863 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.863 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:25.863 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.863 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.863 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:25.863 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.863 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.863 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:25.863 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.863 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.863 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:25.864 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 68809642 2024-04-07T17:51:25.864 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.864 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.864 INFO:tasks.workunit.client.0.smithi100.stdout: "objecter": { 2024-04-07T17:51:25.864 INFO:tasks.workunit.client.0.smithi100.stdout: "op_active": { 2024-04-07T17:51:25.864 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.864 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.864 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r": { 2024-04-07T17:51:25.864 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 22 2024-04-07T17:51:25.864 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.864 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rmw": { 2024-04-07T17:51:25.864 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.864 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.864 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w": { 2024-04-07T17:51:25.864 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.864 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.864 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.865 INFO:tasks.workunit.client.0.smithi100.stdout: "osd": { 2024-04-07T17:51:25.865 INFO:tasks.workunit.client.0.smithi100.stdout: "numpg": { 2024-04-07T17:51:25.865 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 51 2024-04-07T17:51:25.865 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.865 INFO:tasks.workunit.client.0.smithi100.stdout: "numpg_removing": { 2024-04-07T17:51:25.865 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.865 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.865 INFO:tasks.workunit.client.0.smithi100.stdout: "op": { 2024-04-07T17:51:25.865 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 27984 2024-04-07T17:51:25.865 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.865 INFO:tasks.workunit.client.0.smithi100.stdout: "op_before_queue_op_lat": { 2024-04-07T17:51:25.865 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 87803, 2024-04-07T17:51:25.865 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 14036212222 2024-04-07T17:51:25.865 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.865 INFO:tasks.workunit.client.0.smithi100.stdout: "op_delayed_degraded": { 2024-04-07T17:51:25.866 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1 2024-04-07T17:51:25.866 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.866 INFO:tasks.workunit.client.0.smithi100.stdout: "op_delayed_unreadable": { 2024-04-07T17:51:25.866 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.866 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.866 INFO:tasks.workunit.client.0.smithi100.stdout: "op_in_bytes": { 2024-04-07T17:51:25.866 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1438594914 2024-04-07T17:51:25.866 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.866 INFO:tasks.workunit.client.0.smithi100.stdout: "op_latency": { 2024-04-07T17:51:25.866 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 27984, 2024-04-07T17:51:25.866 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 91475787538 2024-04-07T17:51:25.866 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.866 INFO:tasks.workunit.client.0.smithi100.stdout: "op_out_bytes": { 2024-04-07T17:51:25.867 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1486727151 2024-04-07T17:51:25.867 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.867 INFO:tasks.workunit.client.0.smithi100.stdout: "op_prepare_latency": { 2024-04-07T17:51:25.867 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 28568, 2024-04-07T17:51:25.867 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 27069205821 2024-04-07T17:51:25.867 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.867 INFO:tasks.workunit.client.0.smithi100.stdout: "op_process_latency": { 2024-04-07T17:51:25.867 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 27984, 2024-04-07T17:51:25.867 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 54395342610 2024-04-07T17:51:25.867 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.867 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r": { 2024-04-07T17:51:25.867 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 18559 2024-04-07T17:51:25.867 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.867 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_latency": { 2024-04-07T17:51:25.867 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 18559, 2024-04-07T17:51:25.867 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 15330456442 2024-04-07T17:51:25.867 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.868 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_latency_out_bytes_histogram": { 2024-04-07T17:51:25.868 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.868 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.868 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_out_bytes": { 2024-04-07T17:51:25.868 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1486727151 2024-04-07T17:51:25.868 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.868 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_prepare_latency": { 2024-04-07T17:51:25.868 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 18752, 2024-04-07T17:51:25.868 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 10042264838 2024-04-07T17:51:25.868 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.868 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_process_latency": { 2024-04-07T17:51:25.868 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 18559, 2024-04-07T17:51:25.868 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 9329326658 2024-04-07T17:51:25.868 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.868 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw": { 2024-04-07T17:51:25.869 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5511 2024-04-07T17:51:25.869 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.869 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_in_bytes": { 2024-04-07T17:51:25.869 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 266802303 2024-04-07T17:51:25.869 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.869 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_latency": { 2024-04-07T17:51:25.869 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 5511, 2024-04-07T17:51:25.869 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 19608757265 2024-04-07T17:51:25.869 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.869 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_latency_in_bytes_histogram": { 2024-04-07T17:51:25.869 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.869 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.869 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_latency_out_bytes_histogram": { 2024-04-07T17:51:25.869 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.869 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.870 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_out_bytes": { 2024-04-07T17:51:25.870 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.870 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.870 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_prepare_latency": { 2024-04-07T17:51:25.870 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 5777, 2024-04-07T17:51:25.870 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 7872110857 2024-04-07T17:51:25.870 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.870 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_process_latency": { 2024-04-07T17:51:25.870 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 5511, 2024-04-07T17:51:25.870 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 17096965462 2024-04-07T17:51:25.870 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.870 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w": { 2024-04-07T17:51:25.870 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3914 2024-04-07T17:51:25.870 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.871 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_in_bytes": { 2024-04-07T17:51:25.871 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1171792611 2024-04-07T17:51:25.871 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.871 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_latency": { 2024-04-07T17:51:25.871 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 3914, 2024-04-07T17:51:25.871 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 56536573831 2024-04-07T17:51:25.871 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.871 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_latency_in_bytes_histogram": { 2024-04-07T17:51:25.871 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.871 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.871 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_prepare_latency": { 2024-04-07T17:51:25.871 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 4039, 2024-04-07T17:51:25.871 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 9154830126 2024-04-07T17:51:25.871 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.871 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_process_latency": { 2024-04-07T17:51:25.872 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 3914, 2024-04-07T17:51:25.872 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 27969050490 2024-04-07T17:51:25.872 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.872 INFO:tasks.workunit.client.0.smithi100.stdout: "op_wip": { 2024-04-07T17:51:25.872 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.872 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.872 INFO:tasks.workunit.client.0.smithi100.stdout: "recovery_bytes": { 2024-04-07T17:51:25.872 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 8388656 2024-04-07T17:51:25.872 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.872 INFO:tasks.workunit.client.0.smithi100.stdout: "recovery_ops": { 2024-04-07T17:51:25.872 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 11 2024-04-07T17:51:25.872 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.872 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_bytes": { 2024-04-07T17:51:25.872 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 95995035648 2024-04-07T17:51:25.872 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.872 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_bytes_used": { 2024-04-07T17:51:25.873 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 586682368 2024-04-07T17:51:25.873 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.873 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.873 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.873 INFO:tasks.workunit.client.0.smithi100.stdout: "osd.5": { 2024-04-07T17:51:25.873 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs": { 2024-04-07T17:51:25.873 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_db_max_lat": { 2024-04-07T17:51:25.873 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 26003 2024-04-07T17:51:25.873 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.873 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_slow_fallback": { 2024-04-07T17:51:25.873 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.873 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.873 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_slow_max_lat": { 2024-04-07T17:51:25.873 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.873 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.873 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_slow_size_fallback": { 2024-04-07T17:51:25.874 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.874 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.874 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit_db": { 2024-04-07T17:51:25.874 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 65536 2024-04-07T17:51:25.874 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.874 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit_slow": { 2024-04-07T17:51:25.874 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.874 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.874 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit_wal": { 2024-04-07T17:51:25.874 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.874 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.874 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_wal_max_lat": { 2024-04-07T17:51:25.874 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.874 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.875 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_written_slow": { 2024-04-07T17:51:25.875 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.875 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.875 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_written_sst": { 2024-04-07T17:51:25.875 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 15654912 2024-04-07T17:51:25.875 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.875 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_written_wal": { 2024-04-07T17:51:25.875 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 603410432 2024-04-07T17:51:25.875 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.875 INFO:tasks.workunit.client.0.smithi100.stdout: "compact_lat": { 2024-04-07T17:51:25.875 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 19, 2024-04-07T17:51:25.875 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 8231783 2024-04-07T17:51:25.875 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.875 INFO:tasks.workunit.client.0.smithi100.stdout: "compact_lock_lat": { 2024-04-07T17:51:25.875 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 19, 2024-04-07T17:51:25.876 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2455108 2024-04-07T17:51:25.876 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.876 INFO:tasks.workunit.client.0.smithi100.stdout: "db_alloc_lat": { 2024-04-07T17:51:25.876 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 196, 2024-04-07T17:51:25.876 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1310582 2024-04-07T17:51:25.876 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.876 INFO:tasks.workunit.client.0.smithi100.stdout: "db_total_bytes": { 2024-04-07T17:51:25.876 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 95995027456 2024-04-07T17:51:25.876 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.876 INFO:tasks.workunit.client.0.smithi100.stdout: "db_used_bytes": { 2024-04-07T17:51:25.876 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 444465152 2024-04-07T17:51:25.876 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.876 INFO:tasks.workunit.client.0.smithi100.stdout: "flush_lat": { 2024-04-07T17:51:25.876 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 80665, 2024-04-07T17:51:25.876 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 15235152455 2024-04-07T17:51:25.876 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.877 INFO:tasks.workunit.client.0.smithi100.stdout: "fsync_lat": { 2024-04-07T17:51:25.877 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 80656, 2024-04-07T17:51:25.877 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 26858743324 2024-04-07T17:51:25.877 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.877 INFO:tasks.workunit.client.0.smithi100.stdout: "log_bytes": { 2024-04-07T17:51:25.877 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 14532608 2024-04-07T17:51:25.877 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.877 INFO:tasks.workunit.client.0.smithi100.stdout: "logged_bytes": { 2024-04-07T17:51:25.877 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 330424320 2024-04-07T17:51:25.877 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.877 INFO:tasks.workunit.client.0.smithi100.stdout: "max_bytes_db": { 2024-04-07T17:51:25.877 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 452984832 2024-04-07T17:51:25.877 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.877 INFO:tasks.workunit.client.0.smithi100.stdout: "max_bytes_slow": { 2024-04-07T17:51:25.877 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.877 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.878 INFO:tasks.workunit.client.0.smithi100.stdout: "max_bytes_wal": { 2024-04-07T17:51:25.878 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.878 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.878 INFO:tasks.workunit.client.0.smithi100.stdout: "num_files": { 2024-04-07T17:51:25.878 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 46 2024-04-07T17:51:25.878 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.878 INFO:tasks.workunit.client.0.smithi100.stdout: "read_bytes": { 2024-04-07T17:51:25.878 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 12789730188 2024-04-07T17:51:25.878 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.878 INFO:tasks.workunit.client.0.smithi100.stdout: "read_count": { 2024-04-07T17:51:25.878 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 100535 2024-04-07T17:51:25.878 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.878 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes": { 2024-04-07T17:51:25.878 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 10848047104 2024-04-07T17:51:25.878 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.878 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes_db": { 2024-04-07T17:51:25.879 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 10848051200 2024-04-07T17:51:25.879 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.879 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes_slow": { 2024-04-07T17:51:25.879 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.879 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.879 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes_wal": { 2024-04-07T17:51:25.879 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.879 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.879 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_count": { 2024-04-07T17:51:25.879 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 94858 2024-04-07T17:51:25.879 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.879 INFO:tasks.workunit.client.0.smithi100.stdout: "read_lat": { 2024-04-07T17:51:25.879 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 100535, 2024-04-07T17:51:25.879 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5637691717 2024-04-07T17:51:25.879 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.879 INFO:tasks.workunit.client.0.smithi100.stdout: "read_prefetch_bytes": { 2024-04-07T17:51:25.879 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 12578820194 2024-04-07T17:51:25.880 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.880 INFO:tasks.workunit.client.0.smithi100.stdout: "read_prefetch_count": { 2024-04-07T17:51:25.880 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 94081 2024-04-07T17:51:25.880 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.880 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_buffer_bytes": { 2024-04-07T17:51:25.880 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4494109 2024-04-07T17:51:25.880 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.880 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_buffer_count": { 2024-04-07T17:51:25.880 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 924 2024-04-07T17:51:25.880 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.880 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_bytes": { 2024-04-07T17:51:25.880 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 15530604 2024-04-07T17:51:25.880 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.880 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_count": { 2024-04-07T17:51:25.880 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1234 2024-04-07T17:51:25.880 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.881 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes": { 2024-04-07T17:51:25.881 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 11036495 2024-04-07T17:51:25.881 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.881 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes_db": { 2024-04-07T17:51:25.881 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 11036495 2024-04-07T17:51:25.881 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.881 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes_slow": { 2024-04-07T17:51:25.881 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.881 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.881 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes_wal": { 2024-04-07T17:51:25.881 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.881 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.881 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_count": { 2024-04-07T17:51:25.881 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 324 2024-04-07T17:51:25.881 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.881 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_lat": { 2024-04-07T17:51:25.881 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1234, 2024-04-07T17:51:25.881 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4942084 2024-04-07T17:51:25.882 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.882 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_alloc_lat": { 2024-04-07T17:51:25.882 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:25.882 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.882 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.882 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_total_bytes": { 2024-04-07T17:51:25.882 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.882 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.882 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_used_bytes": { 2024-04-07T17:51:25.882 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.882 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.882 INFO:tasks.workunit.client.0.smithi100.stdout: "truncate_lat": { 2024-04-07T17:51:25.882 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1, 2024-04-07T17:51:25.882 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 682 2024-04-07T17:51:25.882 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.883 INFO:tasks.workunit.client.0.smithi100.stdout: "unlink_lat": { 2024-04-07T17:51:25.883 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 15, 2024-04-07T17:51:25.883 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 19508 2024-04-07T17:51:25.883 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.883 INFO:tasks.workunit.client.0.smithi100.stdout: "wal_alloc_lat": { 2024-04-07T17:51:25.883 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:25.883 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.883 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.883 INFO:tasks.workunit.client.0.smithi100.stdout: "wal_total_bytes": { 2024-04-07T17:51:25.883 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.883 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.883 INFO:tasks.workunit.client.0.smithi100.stdout: "wal_used_bytes": { 2024-04-07T17:51:25.883 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.883 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.883 INFO:tasks.workunit.client.0.smithi100.stdout: "write_bytes": { 2024-04-07T17:51:25.883 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 952373248 2024-04-07T17:51:25.884 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.884 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.884 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore": { 2024-04-07T17:51:25.884 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit": { 2024-04-07T17:51:25.884 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4096 2024-04-07T17:51:25.884 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.884 INFO:tasks.workunit.client.0.smithi100.stdout: "allocated": { 2024-04-07T17:51:25.884 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 14020608 2024-04-07T17:51:25.884 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.884 INFO:tasks.workunit.client.0.smithi100.stdout: "allocator_lat": { 2024-04-07T17:51:25.884 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 22611, 2024-04-07T17:51:25.884 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 116632904 2024-04-07T17:51:25.884 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.884 INFO:tasks.workunit.client.0.smithi100.stdout: "clist_lat": { 2024-04-07T17:51:25.885 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 3546, 2024-04-07T17:51:25.885 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 17766845093 2024-04-07T17:51:25.885 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.885 INFO:tasks.workunit.client.0.smithi100.stdout: "compress_lat": { 2024-04-07T17:51:25.885 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:25.885 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.885 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.885 INFO:tasks.workunit.client.0.smithi100.stdout: "compressed": { 2024-04-07T17:51:25.885 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.885 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.885 INFO:tasks.workunit.client.0.smithi100.stdout: "compressed_allocated": { 2024-04-07T17:51:25.885 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.885 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.885 INFO:tasks.workunit.client.0.smithi100.stdout: "compressed_original": { 2024-04-07T17:51:25.885 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.886 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.886 INFO:tasks.workunit.client.0.smithi100.stdout: "csum_lat": { 2024-04-07T17:51:25.886 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 24579, 2024-04-07T17:51:25.886 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 325075173 2024-04-07T17:51:25.886 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.886 INFO:tasks.workunit.client.0.smithi100.stdout: "decompress_lat": { 2024-04-07T17:51:25.886 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:25.886 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.886 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.886 INFO:tasks.workunit.client.0.smithi100.stdout: "fragmentation_micros": { 2024-04-07T17:51:25.886 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.886 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.886 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_commit_lat": { 2024-04-07T17:51:25.886 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 80597, 2024-04-07T17:51:25.887 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 34623882580 2024-04-07T17:51:25.887 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.887 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_final_lat": { 2024-04-07T17:51:25.887 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 79698, 2024-04-07T17:51:25.887 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3958472053 2024-04-07T17:51:25.887 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.887 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_flush_lat": { 2024-04-07T17:51:25.887 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 80597, 2024-04-07T17:51:25.887 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 99120687 2024-04-07T17:51:25.887 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.887 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_sync_lat": { 2024-04-07T17:51:25.887 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 80597, 2024-04-07T17:51:25.887 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 34723003267 2024-04-07T17:51:25.887 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.887 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_get_keys_lat": { 2024-04-07T17:51:25.887 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:25.887 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.887 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.888 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_get_values_lat": { 2024-04-07T17:51:25.888 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 239680, 2024-04-07T17:51:25.888 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2166046359 2024-04-07T17:51:25.888 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.888 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_lower_bound_lat": { 2024-04-07T17:51:25.888 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 808, 2024-04-07T17:51:25.888 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5289872 2024-04-07T17:51:25.888 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.888 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_next_lat": { 2024-04-07T17:51:25.888 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 128881, 2024-04-07T17:51:25.888 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 97129343 2024-04-07T17:51:25.888 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.888 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_seek_to_first_lat": { 2024-04-07T17:51:25.888 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 50, 2024-04-07T17:51:25.888 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 74621 2024-04-07T17:51:25.888 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.888 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_upper_bound_lat": { 2024-04-07T17:51:25.889 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 19469, 2024-04-07T17:51:25.889 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 8325495929 2024-04-07T17:51:25.889 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.889 INFO:tasks.workunit.client.0.smithi100.stdout: "onode_hits": { 2024-04-07T17:51:25.889 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 436725 2024-04-07T17:51:25.889 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.889 INFO:tasks.workunit.client.0.smithi100.stdout: "onode_misses": { 2024-04-07T17:51:25.889 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 20982 2024-04-07T17:51:25.889 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.889 INFO:tasks.workunit.client.0.smithi100.stdout: "read_lat": { 2024-04-07T17:51:25.889 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 10659, 2024-04-07T17:51:25.889 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2220392001 2024-04-07T17:51:25.889 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.889 INFO:tasks.workunit.client.0.smithi100.stdout: "read_onode_meta_lat": { 2024-04-07T17:51:25.889 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 30132, 2024-04-07T17:51:25.889 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 15167976 2024-04-07T17:51:25.889 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.890 INFO:tasks.workunit.client.0.smithi100.stdout: "read_wait_aio_lat": { 2024-04-07T17:51:25.890 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 19473, 2024-04-07T17:51:25.890 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1807973266 2024-04-07T17:51:25.890 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.890 INFO:tasks.workunit.client.0.smithi100.stdout: "reads_with_retries": { 2024-04-07T17:51:25.890 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.890 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.890 INFO:tasks.workunit.client.0.smithi100.stdout: "remove_lat": { 2024-04-07T17:51:25.890 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 15467, 2024-04-07T17:51:25.890 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 823899553 2024-04-07T17:51:25.890 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.890 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_aio_wait_count": { 2024-04-07T17:51:25.890 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.890 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.891 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_committed_kv_count": { 2024-04-07T17:51:25.891 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.891 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.891 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_read_onode_meta_count": { 2024-04-07T17:51:25.891 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.891 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.891 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_read_wait_aio_count": { 2024-04-07T17:51:25.891 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.891 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.891 INFO:tasks.workunit.client.0.smithi100.stdout: "state_aio_wait_lat": { 2024-04-07T17:51:25.891 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 93751, 2024-04-07T17:51:25.891 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 13111865981 2024-04-07T17:51:25.891 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.891 INFO:tasks.workunit.client.0.smithi100.stdout: "state_deferred_aio_wait_lat": { 2024-04-07T17:51:25.891 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 8297, 2024-04-07T17:51:25.891 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1069282503 2024-04-07T17:51:25.892 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.892 INFO:tasks.workunit.client.0.smithi100.stdout: "state_deferred_cleanup_lat": { 2024-04-07T17:51:25.892 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 8296, 2024-04-07T17:51:25.892 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4488212413467 2024-04-07T17:51:25.892 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.892 INFO:tasks.workunit.client.0.smithi100.stdout: "state_deferred_queued_lat": { 2024-04-07T17:51:25.892 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 8297, 2024-04-07T17:51:25.892 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 6666966893111 2024-04-07T17:51:25.892 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.892 INFO:tasks.workunit.client.0.smithi100.stdout: "state_done_lat": { 2024-04-07T17:51:25.893 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 93750, 2024-04-07T17:51:25.893 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3192176271911 2024-04-07T17:51:25.893 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.893 INFO:tasks.workunit.client.0.smithi100.stdout: "state_finishing_lat": { 2024-04-07T17:51:25.893 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 93750, 2024-04-07T17:51:25.893 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 23533677 2024-04-07T17:51:25.893 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.893 INFO:tasks.workunit.client.0.smithi100.stdout: "state_io_done_lat": { 2024-04-07T17:51:25.893 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 93751, 2024-04-07T17:51:25.893 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 354231200 2024-04-07T17:51:25.894 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.894 INFO:tasks.workunit.client.0.smithi100.stdout: "state_kv_commiting_lat": { 2024-04-07T17:51:25.894 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 93751, 2024-04-07T17:51:25.894 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 60032684809 2024-04-07T17:51:25.894 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.894 INFO:tasks.workunit.client.0.smithi100.stdout: "state_kv_done_lat": { 2024-04-07T17:51:25.894 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 93751, 2024-04-07T17:51:25.894 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 73338408 2024-04-07T17:51:25.894 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.894 INFO:tasks.workunit.client.0.smithi100.stdout: "state_kv_queued_lat": { 2024-04-07T17:51:25.894 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 93751, 2024-04-07T17:51:25.894 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 29192592705 2024-04-07T17:51:25.894 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.894 INFO:tasks.workunit.client.0.smithi100.stdout: "state_prepare_lat": { 2024-04-07T17:51:25.894 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 93751, 2024-04-07T17:51:25.895 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 35861066911 2024-04-07T17:51:25.895 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.895 INFO:tasks.workunit.client.0.smithi100.stdout: "stored": { 2024-04-07T17:51:25.895 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 9899919 2024-04-07T17:51:25.895 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.895 INFO:tasks.workunit.client.0.smithi100.stdout: "truncate_lat": { 2024-04-07T17:51:25.895 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 2320, 2024-04-07T17:51:25.895 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 23026482 2024-04-07T17:51:25.895 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.895 INFO:tasks.workunit.client.0.smithi100.stdout: "txc_commit_lat": { 2024-04-07T17:51:25.895 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 93751, 2024-04-07T17:51:25.895 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 138586597593 2024-04-07T17:51:25.895 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.895 INFO:tasks.workunit.client.0.smithi100.stdout: "txc_submit_lat": { 2024-04-07T17:51:25.895 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 93751, 2024-04-07T17:51:25.896 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 38590847980 2024-04-07T17:51:25.896 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.896 INFO:tasks.workunit.client.0.smithi100.stdout: "txc_throttle_lat": { 2024-04-07T17:51:25.896 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 93751, 2024-04-07T17:51:25.896 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1304542608 2024-04-07T17:51:25.896 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.896 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.896 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache": { 2024-04-07T17:51:25.896 INFO:tasks.workunit.client.0.smithi100.stdout: "cache_bytes": { 2024-04-07T17:51:25.896 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1444650649 2024-04-07T17:51:25.896 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.896 INFO:tasks.workunit.client.0.smithi100.stdout: "heap_bytes": { 2024-04-07T17:51:25.896 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 570359808 2024-04-07T17:51:25.896 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.896 INFO:tasks.workunit.client.0.smithi100.stdout: "mapped_bytes": { 2024-04-07T17:51:25.897 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 557162496 2024-04-07T17:51:25.897 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.897 INFO:tasks.workunit.client.0.smithi100.stdout: "target_bytes": { 2024-04-07T17:51:25.897 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2647008256 2024-04-07T17:51:25.897 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.897 INFO:tasks.workunit.client.0.smithi100.stdout: "unmapped_bytes": { 2024-04-07T17:51:25.897 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 13197312 2024-04-07T17:51:25.897 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.897 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.897 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:data": { 2024-04-07T17:51:25.897 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:25.897 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 134217728 2024-04-07T17:51:25.897 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.897 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:25.897 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.898 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.898 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:25.898 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.898 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.898 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:25.898 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 58739848 2024-04-07T17:51:25.898 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.898 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:25.898 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.898 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.898 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:25.898 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.898 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.898 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:25.899 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.899 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.899 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:25.899 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.899 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.899 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:25.899 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4096 2024-04-07T17:51:25.899 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.899 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:25.899 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 24576 2024-04-07T17:51:25.899 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.899 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:25.899 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.899 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.899 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:25.899 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.900 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.900 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:25.900 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.900 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.900 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:25.900 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 75449208 2024-04-07T17:51:25.900 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.900 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.900 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:kv": { 2024-04-07T17:51:25.900 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:25.900 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 578813952 2024-04-07T17:51:25.900 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.900 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:25.900 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 67108864 2024-04-07T17:51:25.900 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.901 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:25.901 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.901 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.901 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:25.901 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 440548866 2024-04-07T17:51:25.901 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.901 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:25.901 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.901 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.901 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:25.901 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.901 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.901 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:25.901 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2128 2024-04-07T17:51:25.901 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.901 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:25.902 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.902 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.902 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:25.902 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 163344 2024-04-07T17:51:25.902 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.902 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:25.902 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.902 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.902 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:25.902 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.902 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.902 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:25.902 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.902 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.903 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:25.903 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.903 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.903 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:25.903 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 70990750 2024-04-07T17:51:25.903 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.903 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.903 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:kv_onode": { 2024-04-07T17:51:25.903 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:25.903 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 192937984 2024-04-07T17:51:25.903 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.903 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:25.903 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 67108864 2024-04-07T17:51:25.903 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.903 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:25.903 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 15205920 2024-04-07T17:51:25.903 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.904 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:25.904 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 39159899 2024-04-07T17:51:25.904 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.904 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:25.904 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.904 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.904 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:25.904 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.904 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.904 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:25.904 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.904 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.904 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:25.904 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.904 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.904 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:25.904 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.905 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.905 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:25.905 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.905 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.905 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:25.905 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.905 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.905 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:25.905 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.905 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.905 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:25.905 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.905 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.905 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:25.905 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 71463301 2024-04-07T17:51:25.905 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.906 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.906 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:meta": { 2024-04-07T17:51:25.906 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:25.906 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 511705088 2024-04-07T17:51:25.906 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.906 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:25.906 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.906 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.906 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:25.906 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.906 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.906 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:25.906 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 440548866 2024-04-07T17:51:25.906 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.907 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:25.907 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.907 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.907 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:25.907 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.907 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.907 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:25.907 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.907 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.907 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:25.907 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.907 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.907 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:25.907 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 148347 2024-04-07T17:51:25.907 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.907 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:25.908 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1094803 2024-04-07T17:51:25.908 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.908 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:25.908 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.908 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.908 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:25.908 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.908 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.908 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:25.908 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.908 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.908 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:25.908 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 69913072 2024-04-07T17:51:25.908 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.908 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.908 INFO:tasks.workunit.client.0.smithi100.stdout: "objecter": { 2024-04-07T17:51:25.908 INFO:tasks.workunit.client.0.smithi100.stdout: "op_active": { 2024-04-07T17:51:25.908 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.909 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.909 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r": { 2024-04-07T17:51:25.909 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 24 2024-04-07T17:51:25.909 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.909 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rmw": { 2024-04-07T17:51:25.909 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.909 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.909 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w": { 2024-04-07T17:51:25.909 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.909 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.909 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.909 INFO:tasks.workunit.client.0.smithi100.stdout: "osd": { 2024-04-07T17:51:25.909 INFO:tasks.workunit.client.0.smithi100.stdout: "numpg": { 2024-04-07T17:51:25.909 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 54 2024-04-07T17:51:25.909 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.909 INFO:tasks.workunit.client.0.smithi100.stdout: "numpg_removing": { 2024-04-07T17:51:25.909 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.910 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.910 INFO:tasks.workunit.client.0.smithi100.stdout: "op": { 2024-04-07T17:51:25.910 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 37568 2024-04-07T17:51:25.910 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.910 INFO:tasks.workunit.client.0.smithi100.stdout: "op_before_queue_op_lat": { 2024-04-07T17:51:25.910 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 118839, 2024-04-07T17:51:25.910 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 18250742844 2024-04-07T17:51:25.910 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.910 INFO:tasks.workunit.client.0.smithi100.stdout: "op_delayed_degraded": { 2024-04-07T17:51:25.910 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.910 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.910 INFO:tasks.workunit.client.0.smithi100.stdout: "op_delayed_unreadable": { 2024-04-07T17:51:25.910 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.910 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.910 INFO:tasks.workunit.client.0.smithi100.stdout: "op_in_bytes": { 2024-04-07T17:51:25.911 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1478194555 2024-04-07T17:51:25.911 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.911 INFO:tasks.workunit.client.0.smithi100.stdout: "op_latency": { 2024-04-07T17:51:25.911 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 37568, 2024-04-07T17:51:25.911 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 92458218161 2024-04-07T17:51:25.911 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.911 INFO:tasks.workunit.client.0.smithi100.stdout: "op_out_bytes": { 2024-04-07T17:51:25.911 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2285559046 2024-04-07T17:51:25.911 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.911 INFO:tasks.workunit.client.0.smithi100.stdout: "op_prepare_latency": { 2024-04-07T17:51:25.911 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 38251, 2024-04-07T17:51:25.911 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 45291105721 2024-04-07T17:51:25.911 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.911 INFO:tasks.workunit.client.0.smithi100.stdout: "op_process_latency": { 2024-04-07T17:51:25.911 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 37568, 2024-04-07T17:51:25.912 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 67926473334 2024-04-07T17:51:25.912 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.912 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r": { 2024-04-07T17:51:25.912 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 26708 2024-04-07T17:51:25.912 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.912 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_latency": { 2024-04-07T17:51:25.912 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 26708, 2024-04-07T17:51:25.912 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 37896778616 2024-04-07T17:51:25.912 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.912 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_latency_out_bytes_histogram": { 2024-04-07T17:51:25.912 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.912 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.912 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_out_bytes": { 2024-04-07T17:51:25.912 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2285559046 2024-04-07T17:51:25.912 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.913 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_prepare_latency": { 2024-04-07T17:51:25.913 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 26937, 2024-04-07T17:51:25.913 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 27634387557 2024-04-07T17:51:25.913 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.913 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_process_latency": { 2024-04-07T17:51:25.913 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 26708, 2024-04-07T17:51:25.913 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 26262052230 2024-04-07T17:51:25.913 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.913 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw": { 2024-04-07T17:51:25.913 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 9627 2024-04-07T17:51:25.913 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.913 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_in_bytes": { 2024-04-07T17:51:25.913 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 446562633 2024-04-07T17:51:25.913 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.913 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_latency": { 2024-04-07T17:51:25.913 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 9627, 2024-04-07T17:51:25.914 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 39696616362 2024-04-07T17:51:25.914 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.914 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_latency_in_bytes_histogram": { 2024-04-07T17:51:25.914 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.914 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.914 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_latency_out_bytes_histogram": { 2024-04-07T17:51:25.914 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.914 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.914 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_out_bytes": { 2024-04-07T17:51:25.914 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.914 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.914 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_prepare_latency": { 2024-04-07T17:51:25.914 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 9890, 2024-04-07T17:51:25.914 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 13660567957 2024-04-07T17:51:25.915 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.915 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_process_latency": { 2024-04-07T17:51:25.915 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 9627, 2024-04-07T17:51:25.915 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 30252117533 2024-04-07T17:51:25.915 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.915 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w": { 2024-04-07T17:51:25.915 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1233 2024-04-07T17:51:25.915 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.915 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_in_bytes": { 2024-04-07T17:51:25.915 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1031631922 2024-04-07T17:51:25.915 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.915 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_latency": { 2024-04-07T17:51:25.915 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1233, 2024-04-07T17:51:25.915 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 14864823183 2024-04-07T17:51:25.915 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.916 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_latency_in_bytes_histogram": { 2024-04-07T17:51:25.916 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.916 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.916 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_prepare_latency": { 2024-04-07T17:51:25.916 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1424, 2024-04-07T17:51:25.916 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3996150207 2024-04-07T17:51:25.916 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.916 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_process_latency": { 2024-04-07T17:51:25.916 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1233, 2024-04-07T17:51:25.916 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 11412303571 2024-04-07T17:51:25.916 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.916 INFO:tasks.workunit.client.0.smithi100.stdout: "op_wip": { 2024-04-07T17:51:25.916 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.916 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.916 INFO:tasks.workunit.client.0.smithi100.stdout: "recovery_bytes": { 2024-04-07T17:51:25.917 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 11394690 2024-04-07T17:51:25.917 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.917 INFO:tasks.workunit.client.0.smithi100.stdout: "recovery_ops": { 2024-04-07T17:51:25.917 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 18 2024-04-07T17:51:25.917 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.917 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_bytes": { 2024-04-07T17:51:25.917 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 95995035648 2024-04-07T17:51:25.917 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.917 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_bytes_used": { 2024-04-07T17:51:25.917 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 458498048 2024-04-07T17:51:25.917 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.917 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.917 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.917 INFO:tasks.workunit.client.0.smithi100.stdout: "osd.6": { 2024-04-07T17:51:25.917 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs": { 2024-04-07T17:51:25.918 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_db_max_lat": { 2024-04-07T17:51:25.918 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 42976 2024-04-07T17:51:25.918 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.918 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_slow_fallback": { 2024-04-07T17:51:25.918 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.918 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.918 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_slow_max_lat": { 2024-04-07T17:51:25.918 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.918 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.918 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_slow_size_fallback": { 2024-04-07T17:51:25.918 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.918 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.918 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit_db": { 2024-04-07T17:51:25.918 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 65536 2024-04-07T17:51:25.919 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.919 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit_slow": { 2024-04-07T17:51:25.919 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.919 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.919 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit_wal": { 2024-04-07T17:51:25.919 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.919 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.919 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_wal_max_lat": { 2024-04-07T17:51:25.919 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.919 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.919 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_written_slow": { 2024-04-07T17:51:25.919 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.919 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.919 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_written_sst": { 2024-04-07T17:51:25.919 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1253376 2024-04-07T17:51:25.920 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.920 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_written_wal": { 2024-04-07T17:51:25.920 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 458960896 2024-04-07T17:51:25.920 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.920 INFO:tasks.workunit.client.0.smithi100.stdout: "compact_lat": { 2024-04-07T17:51:25.920 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 15, 2024-04-07T17:51:25.920 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5459458 2024-04-07T17:51:25.920 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.920 INFO:tasks.workunit.client.0.smithi100.stdout: "compact_lock_lat": { 2024-04-07T17:51:25.920 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 15, 2024-04-07T17:51:25.920 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1804720 2024-04-07T17:51:25.920 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.920 INFO:tasks.workunit.client.0.smithi100.stdout: "db_alloc_lat": { 2024-04-07T17:51:25.920 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 135, 2024-04-07T17:51:25.920 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 985275 2024-04-07T17:51:25.920 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.921 INFO:tasks.workunit.client.0.smithi100.stdout: "db_total_bytes": { 2024-04-07T17:51:25.921 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 95995027456 2024-04-07T17:51:25.921 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.921 INFO:tasks.workunit.client.0.smithi100.stdout: "db_used_bytes": { 2024-04-07T17:51:25.921 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 343408640 2024-04-07T17:51:25.921 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.921 INFO:tasks.workunit.client.0.smithi100.stdout: "flush_lat": { 2024-04-07T17:51:25.921 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 61485, 2024-04-07T17:51:25.921 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 11982013100 2024-04-07T17:51:25.921 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.921 INFO:tasks.workunit.client.0.smithi100.stdout: "fsync_lat": { 2024-04-07T17:51:25.921 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 61497, 2024-04-07T17:51:25.921 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 21138088477 2024-04-07T17:51:25.921 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.921 INFO:tasks.workunit.client.0.smithi100.stdout: "log_bytes": { 2024-04-07T17:51:25.922 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2564096 2024-04-07T17:51:25.922 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.922 INFO:tasks.workunit.client.0.smithi100.stdout: "logged_bytes": { 2024-04-07T17:51:25.922 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 251932672 2024-04-07T17:51:25.922 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.922 INFO:tasks.workunit.client.0.smithi100.stdout: "max_bytes_db": { 2024-04-07T17:51:25.922 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 364707840 2024-04-07T17:51:25.922 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.922 INFO:tasks.workunit.client.0.smithi100.stdout: "max_bytes_slow": { 2024-04-07T17:51:25.922 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.922 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.922 INFO:tasks.workunit.client.0.smithi100.stdout: "max_bytes_wal": { 2024-04-07T17:51:25.922 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.922 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.922 INFO:tasks.workunit.client.0.smithi100.stdout: "num_files": { 2024-04-07T17:51:25.923 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 33 2024-04-07T17:51:25.923 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.923 INFO:tasks.workunit.client.0.smithi100.stdout: "read_bytes": { 2024-04-07T17:51:25.923 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 611394701 2024-04-07T17:51:25.923 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.923 INFO:tasks.workunit.client.0.smithi100.stdout: "read_count": { 2024-04-07T17:51:25.923 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 18243 2024-04-07T17:51:25.923 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.923 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes": { 2024-04-07T17:51:25.923 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 531197952 2024-04-07T17:51:25.923 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.923 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes_db": { 2024-04-07T17:51:25.923 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 531202048 2024-04-07T17:51:25.923 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.923 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes_slow": { 2024-04-07T17:51:25.924 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.924 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.924 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes_wal": { 2024-04-07T17:51:25.924 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.924 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.924 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_count": { 2024-04-07T17:51:25.924 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 16144 2024-04-07T17:51:25.924 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.924 INFO:tasks.workunit.client.0.smithi100.stdout: "read_lat": { 2024-04-07T17:51:25.924 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 18243, 2024-04-07T17:51:25.924 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 442981158 2024-04-07T17:51:25.924 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.924 INFO:tasks.workunit.client.0.smithi100.stdout: "read_prefetch_bytes": { 2024-04-07T17:51:25.924 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 565009679 2024-04-07T17:51:25.924 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.925 INFO:tasks.workunit.client.0.smithi100.stdout: "read_prefetch_count": { 2024-04-07T17:51:25.925 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 16814 2024-04-07T17:51:25.925 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.925 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_buffer_bytes": { 2024-04-07T17:51:25.925 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 563551 2024-04-07T17:51:25.925 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.925 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_buffer_count": { 2024-04-07T17:51:25.925 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 339 2024-04-07T17:51:25.925 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.925 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_bytes": { 2024-04-07T17:51:25.925 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1239831 2024-04-07T17:51:25.925 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.925 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_count": { 2024-04-07T17:51:25.925 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 451 2024-04-07T17:51:25.925 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.926 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes": { 2024-04-07T17:51:25.926 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 676280 2024-04-07T17:51:25.926 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.926 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes_db": { 2024-04-07T17:51:25.926 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 676280 2024-04-07T17:51:25.926 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.926 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes_slow": { 2024-04-07T17:51:25.926 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.926 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.926 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes_wal": { 2024-04-07T17:51:25.926 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.926 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.926 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_count": { 2024-04-07T17:51:25.926 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 112 2024-04-07T17:51:25.926 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.927 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_lat": { 2024-04-07T17:51:25.927 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 451, 2024-04-07T17:51:25.927 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 827027 2024-04-07T17:51:25.927 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.927 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_alloc_lat": { 2024-04-07T17:51:25.927 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:25.927 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.927 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.927 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_total_bytes": { 2024-04-07T17:51:25.927 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.927 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.927 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_used_bytes": { 2024-04-07T17:51:25.927 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.927 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.927 INFO:tasks.workunit.client.0.smithi100.stdout: "truncate_lat": { 2024-04-07T17:51:25.928 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1, 2024-04-07T17:51:25.928 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 646 2024-04-07T17:51:25.928 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.928 INFO:tasks.workunit.client.0.smithi100.stdout: "unlink_lat": { 2024-04-07T17:51:25.928 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 5, 2024-04-07T17:51:25.928 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 8184 2024-04-07T17:51:25.928 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.928 INFO:tasks.workunit.client.0.smithi100.stdout: "wal_alloc_lat": { 2024-04-07T17:51:25.928 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:25.928 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.928 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.928 INFO:tasks.workunit.client.0.smithi100.stdout: "wal_total_bytes": { 2024-04-07T17:51:25.928 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.928 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.928 INFO:tasks.workunit.client.0.smithi100.stdout: "wal_used_bytes": { 2024-04-07T17:51:25.929 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.929 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.929 INFO:tasks.workunit.client.0.smithi100.stdout: "write_bytes": { 2024-04-07T17:51:25.929 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 714424320 2024-04-07T17:51:25.929 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.929 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.929 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore": { 2024-04-07T17:51:25.929 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit": { 2024-04-07T17:51:25.929 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4096 2024-04-07T17:51:25.929 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.929 INFO:tasks.workunit.client.0.smithi100.stdout: "allocated": { 2024-04-07T17:51:25.929 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 14606336 2024-04-07T17:51:25.929 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.929 INFO:tasks.workunit.client.0.smithi100.stdout: "allocator_lat": { 2024-04-07T17:51:25.929 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 16890, 2024-04-07T17:51:25.929 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 74243402 2024-04-07T17:51:25.930 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.930 INFO:tasks.workunit.client.0.smithi100.stdout: "clist_lat": { 2024-04-07T17:51:25.930 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 3441, 2024-04-07T17:51:25.930 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 6473693390 2024-04-07T17:51:25.930 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.930 INFO:tasks.workunit.client.0.smithi100.stdout: "compress_lat": { 2024-04-07T17:51:25.930 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:25.930 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.930 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.930 INFO:tasks.workunit.client.0.smithi100.stdout: "compressed": { 2024-04-07T17:51:25.930 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.930 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.930 INFO:tasks.workunit.client.0.smithi100.stdout: "compressed_allocated": { 2024-04-07T17:51:25.930 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.931 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.931 INFO:tasks.workunit.client.0.smithi100.stdout: "compressed_original": { 2024-04-07T17:51:25.931 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.931 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.931 INFO:tasks.workunit.client.0.smithi100.stdout: "csum_lat": { 2024-04-07T17:51:25.931 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 14729, 2024-04-07T17:51:25.931 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 190648102 2024-04-07T17:51:25.931 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.931 INFO:tasks.workunit.client.0.smithi100.stdout: "decompress_lat": { 2024-04-07T17:51:25.931 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:25.931 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.931 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.931 INFO:tasks.workunit.client.0.smithi100.stdout: "fragmentation_micros": { 2024-04-07T17:51:25.931 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.931 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.931 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_commit_lat": { 2024-04-07T17:51:25.932 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 61461, 2024-04-07T17:51:25.932 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 27465827150 2024-04-07T17:51:25.932 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.932 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_final_lat": { 2024-04-07T17:51:25.932 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 60716, 2024-04-07T17:51:25.932 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3226568509 2024-04-07T17:51:25.932 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.932 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_flush_lat": { 2024-04-07T17:51:25.932 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 61461, 2024-04-07T17:51:25.932 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 74420054 2024-04-07T17:51:25.932 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.932 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_sync_lat": { 2024-04-07T17:51:25.932 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 61461, 2024-04-07T17:51:25.932 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 27540247204 2024-04-07T17:51:25.932 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.933 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_get_keys_lat": { 2024-04-07T17:51:25.933 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:25.933 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.933 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.933 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_get_values_lat": { 2024-04-07T17:51:25.933 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 22581, 2024-04-07T17:51:25.933 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 685980505 2024-04-07T17:51:25.933 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.933 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_lower_bound_lat": { 2024-04-07T17:51:25.933 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 700, 2024-04-07T17:51:25.933 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5026445 2024-04-07T17:51:25.933 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.933 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_next_lat": { 2024-04-07T17:51:25.933 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 10108, 2024-04-07T17:51:25.933 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 12802315 2024-04-07T17:51:25.934 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.934 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_seek_to_first_lat": { 2024-04-07T17:51:25.934 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 43, 2024-04-07T17:51:25.934 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 77153 2024-04-07T17:51:25.934 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.934 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_upper_bound_lat": { 2024-04-07T17:51:25.934 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 10657, 2024-04-07T17:51:25.934 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1970969784 2024-04-07T17:51:25.934 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.934 INFO:tasks.workunit.client.0.smithi100.stdout: "onode_hits": { 2024-04-07T17:51:25.934 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 175572 2024-04-07T17:51:25.934 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.934 INFO:tasks.workunit.client.0.smithi100.stdout: "onode_misses": { 2024-04-07T17:51:25.934 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 18139 2024-04-07T17:51:25.934 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.935 INFO:tasks.workunit.client.0.smithi100.stdout: "read_lat": { 2024-04-07T17:51:25.935 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 11039, 2024-04-07T17:51:25.935 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1962439157 2024-04-07T17:51:25.935 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.935 INFO:tasks.workunit.client.0.smithi100.stdout: "read_onode_meta_lat": { 2024-04-07T17:51:25.935 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 38635, 2024-04-07T17:51:25.935 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 20820077 2024-04-07T17:51:25.935 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.935 INFO:tasks.workunit.client.0.smithi100.stdout: "read_wait_aio_lat": { 2024-04-07T17:51:25.935 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 27596, 2024-04-07T17:51:25.935 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1296711458 2024-04-07T17:51:25.935 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.935 INFO:tasks.workunit.client.0.smithi100.stdout: "reads_with_retries": { 2024-04-07T17:51:25.935 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.935 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.936 INFO:tasks.workunit.client.0.smithi100.stdout: "remove_lat": { 2024-04-07T17:51:25.936 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 11796, 2024-04-07T17:51:25.936 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 656690853 2024-04-07T17:51:25.936 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.936 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_aio_wait_count": { 2024-04-07T17:51:25.936 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.936 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.936 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_committed_kv_count": { 2024-04-07T17:51:25.936 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.936 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.936 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_read_onode_meta_count": { 2024-04-07T17:51:25.936 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.936 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.936 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_read_wait_aio_count": { 2024-04-07T17:51:25.936 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.937 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.937 INFO:tasks.workunit.client.0.smithi100.stdout: "state_aio_wait_lat": { 2024-04-07T17:51:25.937 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 74635, 2024-04-07T17:51:25.937 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 10364611458 2024-04-07T17:51:25.937 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.937 INFO:tasks.workunit.client.0.smithi100.stdout: "state_deferred_aio_wait_lat": { 2024-04-07T17:51:25.937 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 13939, 2024-04-07T17:51:25.937 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1372751186 2024-04-07T17:51:25.937 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.937 INFO:tasks.workunit.client.0.smithi100.stdout: "state_deferred_cleanup_lat": { 2024-04-07T17:51:25.937 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 13938, 2024-04-07T17:51:25.937 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3655450811587 2024-04-07T17:51:25.937 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.937 INFO:tasks.workunit.client.0.smithi100.stdout: "state_deferred_queued_lat": { 2024-04-07T17:51:25.938 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 13939, 2024-04-07T17:51:25.938 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5923221764357 2024-04-07T17:51:25.938 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.938 INFO:tasks.workunit.client.0.smithi100.stdout: "state_done_lat": { 2024-04-07T17:51:25.938 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 74634, 2024-04-07T17:51:25.938 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2803609494529 2024-04-07T17:51:25.938 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.938 INFO:tasks.workunit.client.0.smithi100.stdout: "state_finishing_lat": { 2024-04-07T17:51:25.938 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 74634, 2024-04-07T17:51:25.938 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 20557962 2024-04-07T17:51:25.938 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.938 INFO:tasks.workunit.client.0.smithi100.stdout: "state_io_done_lat": { 2024-04-07T17:51:25.938 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 74635, 2024-04-07T17:51:25.938 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 593824141 2024-04-07T17:51:25.938 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.939 INFO:tasks.workunit.client.0.smithi100.stdout: "state_kv_commiting_lat": { 2024-04-07T17:51:25.939 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 74635, 2024-04-07T17:51:25.939 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 51948333590 2024-04-07T17:51:25.939 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.939 INFO:tasks.workunit.client.0.smithi100.stdout: "state_kv_done_lat": { 2024-04-07T17:51:25.939 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 74635, 2024-04-07T17:51:25.939 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 60647317 2024-04-07T17:51:25.939 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.939 INFO:tasks.workunit.client.0.smithi100.stdout: "state_kv_queued_lat": { 2024-04-07T17:51:25.939 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 74635, 2024-04-07T17:51:25.939 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 28224012224 2024-04-07T17:51:25.939 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.939 INFO:tasks.workunit.client.0.smithi100.stdout: "state_prepare_lat": { 2024-04-07T17:51:25.939 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 74635, 2024-04-07T17:51:25.939 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 21897965540 2024-04-07T17:51:25.939 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.940 INFO:tasks.workunit.client.0.smithi100.stdout: "stored": { 2024-04-07T17:51:25.940 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 10490178 2024-04-07T17:51:25.940 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.940 INFO:tasks.workunit.client.0.smithi100.stdout: "truncate_lat": { 2024-04-07T17:51:25.940 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 2295, 2024-04-07T17:51:25.940 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 25153482 2024-04-07T17:51:25.940 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.940 INFO:tasks.workunit.client.0.smithi100.stdout: "txc_commit_lat": { 2024-04-07T17:51:25.940 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 74635, 2024-04-07T17:51:25.940 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 113055819629 2024-04-07T17:51:25.940 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.940 INFO:tasks.workunit.client.0.smithi100.stdout: "txc_submit_lat": { 2024-04-07T17:51:25.940 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 74635, 2024-04-07T17:51:25.940 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 23968070054 2024-04-07T17:51:25.940 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.940 INFO:tasks.workunit.client.0.smithi100.stdout: "txc_throttle_lat": { 2024-04-07T17:51:25.941 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 74635, 2024-04-07T17:51:25.941 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1130773563 2024-04-07T17:51:25.941 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.941 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.941 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache": { 2024-04-07T17:51:25.941 INFO:tasks.workunit.client.0.smithi100.stdout: "cache_bytes": { 2024-04-07T17:51:25.941 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1444650649 2024-04-07T17:51:25.941 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.941 INFO:tasks.workunit.client.0.smithi100.stdout: "heap_bytes": { 2024-04-07T17:51:25.941 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 423641088 2024-04-07T17:51:25.941 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.941 INFO:tasks.workunit.client.0.smithi100.stdout: "mapped_bytes": { 2024-04-07T17:51:25.941 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 358727680 2024-04-07T17:51:25.941 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.941 INFO:tasks.workunit.client.0.smithi100.stdout: "target_bytes": { 2024-04-07T17:51:25.941 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2647008256 2024-04-07T17:51:25.941 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.942 INFO:tasks.workunit.client.0.smithi100.stdout: "unmapped_bytes": { 2024-04-07T17:51:25.942 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 64913408 2024-04-07T17:51:25.942 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.942 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.942 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:data": { 2024-04-07T17:51:25.942 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:25.942 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 134217728 2024-04-07T17:51:25.942 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.942 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:25.942 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.942 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.942 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:25.942 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.942 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.942 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:25.943 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 59908063 2024-04-07T17:51:25.943 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.943 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:25.943 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.943 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.943 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:25.943 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.943 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.943 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:25.943 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.943 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.943 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:25.943 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.943 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.943 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:25.943 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.943 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.944 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:25.944 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 184320 2024-04-07T17:51:25.944 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.944 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:25.944 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.944 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.944 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:25.944 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.944 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.944 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:25.944 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.944 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.944 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:25.944 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 74125345 2024-04-07T17:51:25.944 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.944 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.944 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:kv": { 2024-04-07T17:51:25.945 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:25.945 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 587202560 2024-04-07T17:51:25.945 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.945 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:25.945 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 67108864 2024-04-07T17:51:25.945 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.945 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:25.945 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.945 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.945 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:25.945 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 449310472 2024-04-07T17:51:25.945 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.945 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:25.945 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.945 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.946 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:25.946 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.946 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.946 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:25.946 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1040 2024-04-07T17:51:25.946 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.946 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:25.946 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.946 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.946 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:25.946 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 66384 2024-04-07T17:51:25.946 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.946 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:25.946 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.946 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.947 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:25.947 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.947 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.947 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:25.947 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.947 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.947 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:25.947 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.947 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.947 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:25.947 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 70715800 2024-04-07T17:51:25.947 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.947 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.947 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:kv_onode": { 2024-04-07T17:51:25.947 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:25.947 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 176160768 2024-04-07T17:51:25.947 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.948 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:25.948 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 67108864 2024-04-07T17:51:25.948 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.948 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:25.948 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 691824 2024-04-07T17:51:25.948 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.948 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:25.948 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 39938708 2024-04-07T17:51:25.948 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.948 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:25.948 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.948 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.948 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:25.948 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.948 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.948 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:25.948 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.949 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.949 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:25.949 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.949 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.949 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:25.949 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.949 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.949 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:25.949 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.949 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.949 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:25.949 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.949 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.949 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:25.949 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.949 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.950 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:25.950 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.950 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.950 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:25.950 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 68421372 2024-04-07T17:51:25.950 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.950 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.950 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:meta": { 2024-04-07T17:51:25.950 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:25.950 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 520093696 2024-04-07T17:51:25.950 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.950 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:25.950 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.950 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.950 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:25.950 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.951 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.951 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:25.951 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 449310472 2024-04-07T17:51:25.951 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.951 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:25.951 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.951 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.951 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:25.951 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.951 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.951 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:25.951 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.951 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.951 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:25.951 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.952 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.952 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:25.952 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 157124 2024-04-07T17:51:25.952 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.952 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:25.952 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1119133 2024-04-07T17:51:25.952 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.952 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:25.952 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.952 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.952 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:25.952 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.952 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.952 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:25.952 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.953 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.953 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:25.953 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 69506967 2024-04-07T17:51:25.953 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.953 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.953 INFO:tasks.workunit.client.0.smithi100.stdout: "objecter": { 2024-04-07T17:51:25.953 INFO:tasks.workunit.client.0.smithi100.stdout: "op_active": { 2024-04-07T17:51:25.953 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.953 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.953 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r": { 2024-04-07T17:51:25.953 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5 2024-04-07T17:51:25.953 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.953 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rmw": { 2024-04-07T17:51:25.953 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.953 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.953 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w": { 2024-04-07T17:51:25.954 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.954 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.954 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.954 INFO:tasks.workunit.client.0.smithi100.stdout: "osd": { 2024-04-07T17:51:25.954 INFO:tasks.workunit.client.0.smithi100.stdout: "numpg": { 2024-04-07T17:51:25.954 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 54 2024-04-07T17:51:25.954 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.954 INFO:tasks.workunit.client.0.smithi100.stdout: "numpg_removing": { 2024-04-07T17:51:25.954 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.954 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.954 INFO:tasks.workunit.client.0.smithi100.stdout: "op": { 2024-04-07T17:51:25.954 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 33509 2024-04-07T17:51:25.954 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.954 INFO:tasks.workunit.client.0.smithi100.stdout: "op_before_queue_op_lat": { 2024-04-07T17:51:25.954 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 91089, 2024-04-07T17:51:25.954 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 12791557788 2024-04-07T17:51:25.955 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.955 INFO:tasks.workunit.client.0.smithi100.stdout: "op_delayed_degraded": { 2024-04-07T17:51:25.955 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.955 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.955 INFO:tasks.workunit.client.0.smithi100.stdout: "op_delayed_unreadable": { 2024-04-07T17:51:25.955 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.955 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.955 INFO:tasks.workunit.client.0.smithi100.stdout: "op_in_bytes": { 2024-04-07T17:51:25.955 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1535388602 2024-04-07T17:51:25.955 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.955 INFO:tasks.workunit.client.0.smithi100.stdout: "op_latency": { 2024-04-07T17:51:25.955 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 33509, 2024-04-07T17:51:25.955 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 131541624216 2024-04-07T17:51:25.955 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.955 INFO:tasks.workunit.client.0.smithi100.stdout: "op_out_bytes": { 2024-04-07T17:51:25.955 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1703112431 2024-04-07T17:51:25.955 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.956 INFO:tasks.workunit.client.0.smithi100.stdout: "op_prepare_latency": { 2024-04-07T17:51:25.956 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 34219, 2024-04-07T17:51:25.956 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 27830378329 2024-04-07T17:51:25.956 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.956 INFO:tasks.workunit.client.0.smithi100.stdout: "op_process_latency": { 2024-04-07T17:51:25.956 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 33509, 2024-04-07T17:51:25.956 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 53861705532 2024-04-07T17:51:25.956 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.956 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r": { 2024-04-07T17:51:25.956 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 25245 2024-04-07T17:51:25.956 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.956 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_latency": { 2024-04-07T17:51:25.956 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 25245, 2024-04-07T17:51:25.956 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 23376821581 2024-04-07T17:51:25.956 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.956 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_latency_out_bytes_histogram": { 2024-04-07T17:51:25.957 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.957 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.957 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_out_bytes": { 2024-04-07T17:51:25.957 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1703112431 2024-04-07T17:51:25.957 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.957 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_prepare_latency": { 2024-04-07T17:51:25.957 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 25399, 2024-04-07T17:51:25.957 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 11923589716 2024-04-07T17:51:25.957 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.957 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_process_latency": { 2024-04-07T17:51:25.957 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 25245, 2024-04-07T17:51:25.957 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 10146174967 2024-04-07T17:51:25.957 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.957 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw": { 2024-04-07T17:51:25.957 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4064 2024-04-07T17:51:25.957 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.957 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_in_bytes": { 2024-04-07T17:51:25.958 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 290840421 2024-04-07T17:51:25.958 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.958 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_latency": { 2024-04-07T17:51:25.958 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 4064, 2024-04-07T17:51:25.958 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 20965538275 2024-04-07T17:51:25.958 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.958 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_latency_in_bytes_histogram": { 2024-04-07T17:51:25.958 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.958 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.958 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_latency_out_bytes_histogram": { 2024-04-07T17:51:25.958 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.958 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.958 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_out_bytes": { 2024-04-07T17:51:25.958 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.958 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.959 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_prepare_latency": { 2024-04-07T17:51:25.959 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 4504, 2024-04-07T17:51:25.959 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 6305172918 2024-04-07T17:51:25.959 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.959 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_process_latency": { 2024-04-07T17:51:25.959 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 4064, 2024-04-07T17:51:25.959 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 14864893688 2024-04-07T17:51:25.959 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.959 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w": { 2024-04-07T17:51:25.959 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4200 2024-04-07T17:51:25.959 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.959 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_in_bytes": { 2024-04-07T17:51:25.959 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1244548181 2024-04-07T17:51:25.959 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.959 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_latency": { 2024-04-07T17:51:25.959 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 4200, 2024-04-07T17:51:25.959 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 87199264360 2024-04-07T17:51:25.960 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.960 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_latency_in_bytes_histogram": { 2024-04-07T17:51:25.960 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.960 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.960 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_prepare_latency": { 2024-04-07T17:51:25.960 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 4316, 2024-04-07T17:51:25.960 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 9601615695 2024-04-07T17:51:25.960 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.960 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_process_latency": { 2024-04-07T17:51:25.960 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 4200, 2024-04-07T17:51:25.960 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 28850636877 2024-04-07T17:51:25.960 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.960 INFO:tasks.workunit.client.0.smithi100.stdout: "op_wip": { 2024-04-07T17:51:25.960 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.960 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.961 INFO:tasks.workunit.client.0.smithi100.stdout: "recovery_bytes": { 2024-04-07T17:51:25.961 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 73766334 2024-04-07T17:51:25.961 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.961 INFO:tasks.workunit.client.0.smithi100.stdout: "recovery_ops": { 2024-04-07T17:51:25.961 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 34 2024-04-07T17:51:25.961 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.961 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_bytes": { 2024-04-07T17:51:25.961 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 95995035648 2024-04-07T17:51:25.961 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.961 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_bytes_used": { 2024-04-07T17:51:25.961 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 358027264 2024-04-07T17:51:25.961 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.961 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.961 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.961 INFO:tasks.workunit.client.0.smithi100.stdout: "osd.7": { 2024-04-07T17:51:25.962 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs": { 2024-04-07T17:51:25.962 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_db_max_lat": { 2024-04-07T17:51:25.962 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1425185 2024-04-07T17:51:25.962 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.962 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_slow_fallback": { 2024-04-07T17:51:25.962 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.962 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.962 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_slow_max_lat": { 2024-04-07T17:51:25.962 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.962 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.962 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_slow_size_fallback": { 2024-04-07T17:51:25.962 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.962 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.962 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit_db": { 2024-04-07T17:51:25.963 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 65536 2024-04-07T17:51:25.963 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.963 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit_slow": { 2024-04-07T17:51:25.963 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.963 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.963 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit_wal": { 2024-04-07T17:51:25.963 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.963 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.963 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_wal_max_lat": { 2024-04-07T17:51:25.963 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.963 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.963 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_written_slow": { 2024-04-07T17:51:25.963 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.963 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.963 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_written_sst": { 2024-04-07T17:51:25.963 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5050368 2024-04-07T17:51:25.964 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.964 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_written_wal": { 2024-04-07T17:51:25.964 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 495513600 2024-04-07T17:51:25.964 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.964 INFO:tasks.workunit.client.0.smithi100.stdout: "compact_lat": { 2024-04-07T17:51:25.964 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 15, 2024-04-07T17:51:25.964 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 6850825 2024-04-07T17:51:25.964 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.964 INFO:tasks.workunit.client.0.smithi100.stdout: "compact_lock_lat": { 2024-04-07T17:51:25.964 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 15, 2024-04-07T17:51:25.964 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1769094 2024-04-07T17:51:25.964 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.964 INFO:tasks.workunit.client.0.smithi100.stdout: "db_alloc_lat": { 2024-04-07T17:51:25.964 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 148, 2024-04-07T17:51:25.964 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2568628 2024-04-07T17:51:25.964 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.964 INFO:tasks.workunit.client.0.smithi100.stdout: "db_total_bytes": { 2024-04-07T17:51:25.965 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 95995027456 2024-04-07T17:51:25.965 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.965 INFO:tasks.workunit.client.0.smithi100.stdout: "db_used_bytes": { 2024-04-07T17:51:25.965 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 411107328 2024-04-07T17:51:25.965 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.965 INFO:tasks.workunit.client.0.smithi100.stdout: "flush_lat": { 2024-04-07T17:51:25.965 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 63017, 2024-04-07T17:51:25.965 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 12122188924 2024-04-07T17:51:25.965 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.965 INFO:tasks.workunit.client.0.smithi100.stdout: "fsync_lat": { 2024-04-07T17:51:25.965 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 63028, 2024-04-07T17:51:25.965 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 21247379624 2024-04-07T17:51:25.965 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.965 INFO:tasks.workunit.client.0.smithi100.stdout: "log_bytes": { 2024-04-07T17:51:25.965 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 8843264 2024-04-07T17:51:25.966 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.966 INFO:tasks.workunit.client.0.smithi100.stdout: "logged_bytes": { 2024-04-07T17:51:25.966 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 258203648 2024-04-07T17:51:25.966 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.966 INFO:tasks.workunit.client.0.smithi100.stdout: "max_bytes_db": { 2024-04-07T17:51:25.966 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 423886848 2024-04-07T17:51:25.966 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.966 INFO:tasks.workunit.client.0.smithi100.stdout: "max_bytes_slow": { 2024-04-07T17:51:25.966 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.966 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.966 INFO:tasks.workunit.client.0.smithi100.stdout: "max_bytes_wal": { 2024-04-07T17:51:25.966 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.966 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.966 INFO:tasks.workunit.client.0.smithi100.stdout: "num_files": { 2024-04-07T17:51:25.966 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 38 2024-04-07T17:51:25.967 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.967 INFO:tasks.workunit.client.0.smithi100.stdout: "read_bytes": { 2024-04-07T17:51:25.967 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3167491823 2024-04-07T17:51:25.967 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.967 INFO:tasks.workunit.client.0.smithi100.stdout: "read_count": { 2024-04-07T17:51:25.967 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 40027 2024-04-07T17:51:25.967 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.967 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes": { 2024-04-07T17:51:25.967 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2790236160 2024-04-07T17:51:25.967 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.967 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes_db": { 2024-04-07T17:51:25.967 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2790240256 2024-04-07T17:51:25.967 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.967 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes_slow": { 2024-04-07T17:51:25.967 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.967 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.968 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes_wal": { 2024-04-07T17:51:25.968 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.968 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.968 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_count": { 2024-04-07T17:51:25.968 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 25974 2024-04-07T17:51:25.968 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.968 INFO:tasks.workunit.client.0.smithi100.stdout: "read_lat": { 2024-04-07T17:51:25.968 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 40027, 2024-04-07T17:51:25.968 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1975651191 2024-04-07T17:51:25.968 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.968 INFO:tasks.workunit.client.0.smithi100.stdout: "read_prefetch_bytes": { 2024-04-07T17:51:25.968 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3056697503 2024-04-07T17:51:25.968 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.968 INFO:tasks.workunit.client.0.smithi100.stdout: "read_prefetch_count": { 2024-04-07T17:51:25.968 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 36631 2024-04-07T17:51:25.969 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.969 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_buffer_bytes": { 2024-04-07T17:51:25.969 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1782285 2024-04-07T17:51:25.969 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.969 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_buffer_count": { 2024-04-07T17:51:25.969 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 538 2024-04-07T17:51:25.969 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.969 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_bytes": { 2024-04-07T17:51:25.969 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5008896 2024-04-07T17:51:25.969 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.969 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_count": { 2024-04-07T17:51:25.969 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 703 2024-04-07T17:51:25.969 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.969 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes": { 2024-04-07T17:51:25.969 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3226611 2024-04-07T17:51:25.969 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.970 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes_db": { 2024-04-07T17:51:25.970 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3226611 2024-04-07T17:51:25.970 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.970 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes_slow": { 2024-04-07T17:51:25.970 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.970 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.970 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes_wal": { 2024-04-07T17:51:25.970 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.970 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.970 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_count": { 2024-04-07T17:51:25.970 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 167 2024-04-07T17:51:25.970 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.970 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_lat": { 2024-04-07T17:51:25.970 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 703, 2024-04-07T17:51:25.971 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1945572 2024-04-07T17:51:25.971 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.971 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_alloc_lat": { 2024-04-07T17:51:25.971 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:25.971 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.971 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.971 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_total_bytes": { 2024-04-07T17:51:25.971 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.971 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.971 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_used_bytes": { 2024-04-07T17:51:25.971 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.971 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.971 INFO:tasks.workunit.client.0.smithi100.stdout: "truncate_lat": { 2024-04-07T17:51:25.971 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1, 2024-04-07T17:51:25.971 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 858 2024-04-07T17:51:25.971 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.972 INFO:tasks.workunit.client.0.smithi100.stdout: "unlink_lat": { 2024-04-07T17:51:25.972 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 9, 2024-04-07T17:51:25.972 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 12676 2024-04-07T17:51:25.972 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.972 INFO:tasks.workunit.client.0.smithi100.stdout: "wal_alloc_lat": { 2024-04-07T17:51:25.972 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:25.972 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.972 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.972 INFO:tasks.workunit.client.0.smithi100.stdout: "wal_total_bytes": { 2024-04-07T17:51:25.972 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.972 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.972 INFO:tasks.workunit.client.0.smithi100.stdout: "wal_used_bytes": { 2024-04-07T17:51:25.972 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.972 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.972 INFO:tasks.workunit.client.0.smithi100.stdout: "write_bytes": { 2024-04-07T17:51:25.972 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 761053184 2024-04-07T17:51:25.973 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.973 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.973 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore": { 2024-04-07T17:51:25.973 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit": { 2024-04-07T17:51:25.973 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4096 2024-04-07T17:51:25.973 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.973 INFO:tasks.workunit.client.0.smithi100.stdout: "allocated": { 2024-04-07T17:51:25.973 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 10403840 2024-04-07T17:51:25.973 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.973 INFO:tasks.workunit.client.0.smithi100.stdout: "allocator_lat": { 2024-04-07T17:51:25.973 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 17672, 2024-04-07T17:51:25.973 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 84277754 2024-04-07T17:51:25.973 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.973 INFO:tasks.workunit.client.0.smithi100.stdout: "clist_lat": { 2024-04-07T17:51:25.974 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 3466, 2024-04-07T17:51:25.974 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 6069941459 2024-04-07T17:51:25.974 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.974 INFO:tasks.workunit.client.0.smithi100.stdout: "compress_lat": { 2024-04-07T17:51:25.974 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:25.974 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.974 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.974 INFO:tasks.workunit.client.0.smithi100.stdout: "compressed": { 2024-04-07T17:51:25.974 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.974 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.974 INFO:tasks.workunit.client.0.smithi100.stdout: "compressed_allocated": { 2024-04-07T17:51:25.974 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.974 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.974 INFO:tasks.workunit.client.0.smithi100.stdout: "compressed_original": { 2024-04-07T17:51:25.974 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.975 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.975 INFO:tasks.workunit.client.0.smithi100.stdout: "csum_lat": { 2024-04-07T17:51:25.975 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 16446, 2024-04-07T17:51:25.975 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 220970389 2024-04-07T17:51:25.975 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.975 INFO:tasks.workunit.client.0.smithi100.stdout: "decompress_lat": { 2024-04-07T17:51:25.975 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:25.975 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.975 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.975 INFO:tasks.workunit.client.0.smithi100.stdout: "fragmentation_micros": { 2024-04-07T17:51:25.975 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.975 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.975 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_commit_lat": { 2024-04-07T17:51:25.975 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 62983, 2024-04-07T17:51:25.975 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 27504738088 2024-04-07T17:51:25.976 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.976 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_final_lat": { 2024-04-07T17:51:25.976 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 62244, 2024-04-07T17:51:25.976 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3517645071 2024-04-07T17:51:25.976 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.976 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_flush_lat": { 2024-04-07T17:51:25.976 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 62983, 2024-04-07T17:51:25.976 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 96259530 2024-04-07T17:51:25.976 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.976 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_sync_lat": { 2024-04-07T17:51:25.976 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 62983, 2024-04-07T17:51:25.976 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 27600997618 2024-04-07T17:51:25.976 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.976 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_get_keys_lat": { 2024-04-07T17:51:25.976 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:25.977 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.977 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.977 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_get_values_lat": { 2024-04-07T17:51:25.977 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 33493, 2024-04-07T17:51:25.977 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 950805853 2024-04-07T17:51:25.977 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.977 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_lower_bound_lat": { 2024-04-07T17:51:25.977 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 632, 2024-04-07T17:51:25.977 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4331297 2024-04-07T17:51:25.977 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.977 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_next_lat": { 2024-04-07T17:51:25.977 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 7000, 2024-04-07T17:51:25.977 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 13697854 2024-04-07T17:51:25.977 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.977 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_seek_to_first_lat": { 2024-04-07T17:51:25.977 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 50, 2024-04-07T17:51:25.978 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 102727 2024-04-07T17:51:25.978 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.978 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_upper_bound_lat": { 2024-04-07T17:51:25.978 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 11201, 2024-04-07T17:51:25.978 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1557562272 2024-04-07T17:51:25.978 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.978 INFO:tasks.workunit.client.0.smithi100.stdout: "onode_hits": { 2024-04-07T17:51:25.978 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 194378 2024-04-07T17:51:25.978 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.978 INFO:tasks.workunit.client.0.smithi100.stdout: "onode_misses": { 2024-04-07T17:51:25.978 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 17489 2024-04-07T17:51:25.978 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.978 INFO:tasks.workunit.client.0.smithi100.stdout: "read_lat": { 2024-04-07T17:51:25.978 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 9939, 2024-04-07T17:51:25.978 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1593794096 2024-04-07T17:51:25.979 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.979 INFO:tasks.workunit.client.0.smithi100.stdout: "read_onode_meta_lat": { 2024-04-07T17:51:25.979 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 39993, 2024-04-07T17:51:25.979 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 19793062 2024-04-07T17:51:25.979 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.979 INFO:tasks.workunit.client.0.smithi100.stdout: "read_wait_aio_lat": { 2024-04-07T17:51:25.979 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 30054, 2024-04-07T17:51:25.979 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1325463873 2024-04-07T17:51:25.979 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.979 INFO:tasks.workunit.client.0.smithi100.stdout: "reads_with_retries": { 2024-04-07T17:51:25.979 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.979 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.979 INFO:tasks.workunit.client.0.smithi100.stdout: "remove_lat": { 2024-04-07T17:51:25.979 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 13264, 2024-04-07T17:51:25.979 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 667186707 2024-04-07T17:51:25.980 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.980 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_aio_wait_count": { 2024-04-07T17:51:25.980 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.980 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.980 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_committed_kv_count": { 2024-04-07T17:51:25.980 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.980 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.980 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_read_onode_meta_count": { 2024-04-07T17:51:25.980 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.980 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.980 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_read_wait_aio_count": { 2024-04-07T17:51:25.980 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.980 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.980 INFO:tasks.workunit.client.0.smithi100.stdout: "state_aio_wait_lat": { 2024-04-07T17:51:25.980 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 74429, 2024-04-07T17:51:25.980 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 8392301592 2024-04-07T17:51:25.981 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.981 INFO:tasks.workunit.client.0.smithi100.stdout: "state_deferred_aio_wait_lat": { 2024-04-07T17:51:25.981 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 16391, 2024-04-07T17:51:25.981 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1329233809 2024-04-07T17:51:25.981 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.981 INFO:tasks.workunit.client.0.smithi100.stdout: "state_deferred_cleanup_lat": { 2024-04-07T17:51:25.981 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 16390, 2024-04-07T17:51:25.981 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4040344878498 2024-04-07T17:51:25.981 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.981 INFO:tasks.workunit.client.0.smithi100.stdout: "state_deferred_queued_lat": { 2024-04-07T17:51:25.981 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 16391, 2024-04-07T17:51:25.981 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 6763629312230 2024-04-07T17:51:25.981 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.981 INFO:tasks.workunit.client.0.smithi100.stdout: "state_done_lat": { 2024-04-07T17:51:25.981 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 74428, 2024-04-07T17:51:25.981 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2916516169130 2024-04-07T17:51:25.981 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.982 INFO:tasks.workunit.client.0.smithi100.stdout: "state_finishing_lat": { 2024-04-07T17:51:25.982 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 74428, 2024-04-07T17:51:25.982 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 17625368 2024-04-07T17:51:25.982 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.982 INFO:tasks.workunit.client.0.smithi100.stdout: "state_io_done_lat": { 2024-04-07T17:51:25.982 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 74429, 2024-04-07T17:51:25.982 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 362782490 2024-04-07T17:51:25.982 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.982 INFO:tasks.workunit.client.0.smithi100.stdout: "state_kv_commiting_lat": { 2024-04-07T17:51:25.982 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 74429, 2024-04-07T17:51:25.982 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 49654142030 2024-04-07T17:51:25.982 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.982 INFO:tasks.workunit.client.0.smithi100.stdout: "state_kv_done_lat": { 2024-04-07T17:51:25.982 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 74429, 2024-04-07T17:51:25.982 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 57857521 2024-04-07T17:51:25.983 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.983 INFO:tasks.workunit.client.0.smithi100.stdout: "state_kv_queued_lat": { 2024-04-07T17:51:25.983 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 74429, 2024-04-07T17:51:25.983 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 25576588902 2024-04-07T17:51:25.983 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.983 INFO:tasks.workunit.client.0.smithi100.stdout: "state_prepare_lat": { 2024-04-07T17:51:25.983 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 74429, 2024-04-07T17:51:25.983 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 18974977018 2024-04-07T17:51:25.983 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.983 INFO:tasks.workunit.client.0.smithi100.stdout: "stored": { 2024-04-07T17:51:25.983 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 6290654 2024-04-07T17:51:25.983 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.983 INFO:tasks.workunit.client.0.smithi100.stdout: "truncate_lat": { 2024-04-07T17:51:25.983 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 2242, 2024-04-07T17:51:25.983 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 22293375 2024-04-07T17:51:25.984 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.984 INFO:tasks.workunit.client.0.smithi100.stdout: "txc_commit_lat": { 2024-04-07T17:51:25.984 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 74429, 2024-04-07T17:51:25.984 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 102987546391 2024-04-07T17:51:25.984 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.984 INFO:tasks.workunit.client.0.smithi100.stdout: "txc_submit_lat": { 2024-04-07T17:51:25.984 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 74429, 2024-04-07T17:51:25.984 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 21153024583 2024-04-07T17:51:25.984 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.984 INFO:tasks.workunit.client.0.smithi100.stdout: "txc_throttle_lat": { 2024-04-07T17:51:25.984 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 74429, 2024-04-07T17:51:25.984 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 946122269 2024-04-07T17:51:25.984 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.984 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.984 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache": { 2024-04-07T17:51:25.984 INFO:tasks.workunit.client.0.smithi100.stdout: "cache_bytes": { 2024-04-07T17:51:25.984 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1444650649 2024-04-07T17:51:25.985 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.985 INFO:tasks.workunit.client.0.smithi100.stdout: "heap_bytes": { 2024-04-07T17:51:25.985 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 463331328 2024-04-07T17:51:25.985 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.985 INFO:tasks.workunit.client.0.smithi100.stdout: "mapped_bytes": { 2024-04-07T17:51:25.985 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 430809088 2024-04-07T17:51:25.985 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.985 INFO:tasks.workunit.client.0.smithi100.stdout: "target_bytes": { 2024-04-07T17:51:25.985 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2647008256 2024-04-07T17:51:25.985 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.985 INFO:tasks.workunit.client.0.smithi100.stdout: "unmapped_bytes": { 2024-04-07T17:51:25.985 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 32522240 2024-04-07T17:51:25.985 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.985 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.985 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:data": { 2024-04-07T17:51:25.986 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:25.986 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 134217728 2024-04-07T17:51:25.986 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.986 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:25.986 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.986 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.986 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:25.986 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.986 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.986 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:25.986 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 59553061 2024-04-07T17:51:25.986 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.986 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:25.986 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.987 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.987 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:25.987 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.987 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.987 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:25.987 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.987 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.987 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:25.987 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.987 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.987 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:25.987 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 32768 2024-04-07T17:51:25.987 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.987 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:25.987 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 61440 2024-04-07T17:51:25.988 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.988 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:25.988 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.988 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.988 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:25.988 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.988 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.988 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:25.988 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.988 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.988 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:25.988 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 74570459 2024-04-07T17:51:25.988 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.988 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.988 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:kv": { 2024-04-07T17:51:25.988 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:25.988 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 587202560 2024-04-07T17:51:25.989 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.989 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:25.989 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 67108864 2024-04-07T17:51:25.989 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.989 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:25.989 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.989 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.989 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:25.989 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 446647962 2024-04-07T17:51:25.989 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.989 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:25.989 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.989 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.989 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:25.989 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.989 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.989 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:25.990 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1504 2024-04-07T17:51:25.990 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.990 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:25.990 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.990 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.990 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:25.990 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 95456 2024-04-07T17:51:25.990 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.990 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:25.990 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.990 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.990 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:25.990 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.990 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.990 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:25.991 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.991 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.991 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:25.991 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.991 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.991 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:25.991 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 73348774 2024-04-07T17:51:25.991 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.991 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.991 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:kv_onode": { 2024-04-07T17:51:25.991 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:25.991 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 184549376 2024-04-07T17:51:25.991 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.991 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:25.991 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 67108864 2024-04-07T17:51:25.991 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.992 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:25.992 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4539488 2024-04-07T17:51:25.992 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.992 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:25.992 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 39702041 2024-04-07T17:51:25.992 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.992 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:25.992 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.992 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.992 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:25.992 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.992 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.992 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:25.992 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.992 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.992 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:25.993 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.993 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.993 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:25.993 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.993 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.993 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:25.993 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.993 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.993 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:25.993 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.993 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.993 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:25.993 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.993 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.993 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:25.994 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.994 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.994 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:25.994 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 73198983 2024-04-07T17:51:25.994 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.994 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.994 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:meta": { 2024-04-07T17:51:25.994 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:25.994 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 520093696 2024-04-07T17:51:25.994 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.994 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:25.994 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.994 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.994 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:25.994 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.995 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.995 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:25.995 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 446647962 2024-04-07T17:51:25.995 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.995 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:25.995 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.995 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.995 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:25.995 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.995 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.995 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:25.995 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.995 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.995 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:25.995 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.995 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.996 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:25.996 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 155289 2024-04-07T17:51:25.996 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.996 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:25.996 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1107933 2024-04-07T17:51:25.996 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.996 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:25.996 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.996 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.996 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:25.996 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.996 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.996 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:25.996 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.996 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.996 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:25.997 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 72182512 2024-04-07T17:51:25.997 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.997 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.997 INFO:tasks.workunit.client.0.smithi100.stdout: "objecter": { 2024-04-07T17:51:25.997 INFO:tasks.workunit.client.0.smithi100.stdout: "op_active": { 2024-04-07T17:51:25.997 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.997 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.997 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r": { 2024-04-07T17:51:25.997 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 23 2024-04-07T17:51:25.997 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.997 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rmw": { 2024-04-07T17:51:25.997 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.997 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.997 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w": { 2024-04-07T17:51:25.997 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.997 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:25.997 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.998 INFO:tasks.workunit.client.0.smithi100.stdout: "osd": { 2024-04-07T17:51:25.998 INFO:tasks.workunit.client.0.smithi100.stdout: "numpg": { 2024-04-07T17:51:25.998 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 58 2024-04-07T17:51:25.998 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.998 INFO:tasks.workunit.client.0.smithi100.stdout: "numpg_removing": { 2024-04-07T17:51:25.998 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.998 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.998 INFO:tasks.workunit.client.0.smithi100.stdout: "op": { 2024-04-07T17:51:25.998 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 40489 2024-04-07T17:51:25.998 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.998 INFO:tasks.workunit.client.0.smithi100.stdout: "op_before_queue_op_lat": { 2024-04-07T17:51:25.998 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 108065, 2024-04-07T17:51:25.998 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 15066892416 2024-04-07T17:51:25.998 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.998 INFO:tasks.workunit.client.0.smithi100.stdout: "op_delayed_degraded": { 2024-04-07T17:51:25.999 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.999 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.999 INFO:tasks.workunit.client.0.smithi100.stdout: "op_delayed_unreadable": { 2024-04-07T17:51:25.999 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:25.999 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.999 INFO:tasks.workunit.client.0.smithi100.stdout: "op_in_bytes": { 2024-04-07T17:51:25.999 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1416321031 2024-04-07T17:51:25.999 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.999 INFO:tasks.workunit.client.0.smithi100.stdout: "op_latency": { 2024-04-07T17:51:25.999 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 40489, 2024-04-07T17:51:25.999 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 116806238602 2024-04-07T17:51:25.999 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.999 INFO:tasks.workunit.client.0.smithi100.stdout: "op_out_bytes": { 2024-04-07T17:51:25.999 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1572109991 2024-04-07T17:51:25.999 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:25.999 INFO:tasks.workunit.client.0.smithi100.stdout: "op_prepare_latency": { 2024-04-07T17:51:25.999 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 40866, 2024-04-07T17:51:25.999 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 41965414839 2024-04-07T17:51:26.000 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.000 INFO:tasks.workunit.client.0.smithi100.stdout: "op_process_latency": { 2024-04-07T17:51:26.000 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 40489, 2024-04-07T17:51:26.000 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 66194385741 2024-04-07T17:51:26.000 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.000 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r": { 2024-04-07T17:51:26.000 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 27940 2024-04-07T17:51:26.000 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.000 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_latency": { 2024-04-07T17:51:26.000 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 27940, 2024-04-07T17:51:26.000 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 25174152653 2024-04-07T17:51:26.000 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.000 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_latency_out_bytes_histogram": { 2024-04-07T17:51:26.000 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:26.000 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.000 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_out_bytes": { 2024-04-07T17:51:26.001 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1572109991 2024-04-07T17:51:26.001 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.001 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_prepare_latency": { 2024-04-07T17:51:26.001 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 28058, 2024-04-07T17:51:26.001 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 13058211260 2024-04-07T17:51:26.001 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.001 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_process_latency": { 2024-04-07T17:51:26.001 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 27940, 2024-04-07T17:51:26.001 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 11404588197 2024-04-07T17:51:26.001 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.001 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw": { 2024-04-07T17:51:26.001 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 11358 2024-04-07T17:51:26.001 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.001 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_in_bytes": { 2024-04-07T17:51:26.001 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 298732687 2024-04-07T17:51:26.001 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.002 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_latency": { 2024-04-07T17:51:26.002 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 11358, 2024-04-07T17:51:26.002 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 46852373505 2024-04-07T17:51:26.002 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.002 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_latency_in_bytes_histogram": { 2024-04-07T17:51:26.002 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:26.002 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.002 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_latency_out_bytes_histogram": { 2024-04-07T17:51:26.002 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:26.002 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.002 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_out_bytes": { 2024-04-07T17:51:26.002 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:26.002 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.002 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_prepare_latency": { 2024-04-07T17:51:26.002 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 11546, 2024-04-07T17:51:26.002 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 24931969765 2024-04-07T17:51:26.003 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.003 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_process_latency": { 2024-04-07T17:51:26.003 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 11358, 2024-04-07T17:51:26.003 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 41982228701 2024-04-07T17:51:26.003 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.003 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w": { 2024-04-07T17:51:26.003 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1191 2024-04-07T17:51:26.003 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.003 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_in_bytes": { 2024-04-07T17:51:26.003 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1117588344 2024-04-07T17:51:26.003 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.003 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_latency": { 2024-04-07T17:51:26.003 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1191, 2024-04-07T17:51:26.003 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 44779712444 2024-04-07T17:51:26.003 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.004 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_latency_in_bytes_histogram": { 2024-04-07T17:51:26.004 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:26.004 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.004 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_prepare_latency": { 2024-04-07T17:51:26.004 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1262, 2024-04-07T17:51:26.004 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3975233814 2024-04-07T17:51:26.004 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.004 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_process_latency": { 2024-04-07T17:51:26.004 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1191, 2024-04-07T17:51:26.004 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 12807568843 2024-04-07T17:51:26.004 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.004 INFO:tasks.workunit.client.0.smithi100.stdout: "op_wip": { 2024-04-07T17:51:26.004 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:26.004 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.004 INFO:tasks.workunit.client.0.smithi100.stdout: "recovery_bytes": { 2024-04-07T17:51:26.004 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:26.005 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.005 INFO:tasks.workunit.client.0.smithi100.stdout: "recovery_ops": { 2024-04-07T17:51:26.005 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2 2024-04-07T17:51:26.005 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.005 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_bytes": { 2024-04-07T17:51:26.005 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 95995035648 2024-04-07T17:51:26.005 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.005 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_bytes_used": { 2024-04-07T17:51:26.005 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 421523456 2024-04-07T17:51:26.005 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:26.005 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:26.005 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:26.005 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.005 INFO:tasks.workunit.client.0.smithi100.stdout: "pools": [ 2024-04-07T17:51:26.005 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:26.005 INFO:tasks.workunit.client.0.smithi100.stdout: "application": [ 2024-04-07T17:51:26.005 INFO:tasks.workunit.client.0.smithi100.stdout: "mgr" 2024-04-07T17:51:26.006 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.006 INFO:tasks.workunit.client.0.smithi100.stdout: "cache_mode": "none", 2024-04-07T17:51:26.006 INFO:tasks.workunit.client.0.smithi100.stdout: "erasure_code_profile": {}, 2024-04-07T17:51:26.006 INFO:tasks.workunit.client.0.smithi100.stdout: "flags_names": [ 2024-04-07T17:51:26.006 INFO:tasks.workunit.client.0.smithi100.stdout: "hashpspool" 2024-04-07T17:51:26.006 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.006 INFO:tasks.workunit.client.0.smithi100.stdout: "min_size": 2, 2024-04-07T17:51:26.006 INFO:tasks.workunit.client.0.smithi100.stdout: "options": {}, 2024-04-07T17:51:26.006 INFO:tasks.workunit.client.0.smithi100.stdout: "pg_autoscale_mode": "off", 2024-04-07T17:51:26.006 INFO:tasks.workunit.client.0.smithi100.stdout: "pg_num": 1, 2024-04-07T17:51:26.006 INFO:tasks.workunit.client.0.smithi100.stdout: "pgp_num": 1, 2024-04-07T17:51:26.006 INFO:tasks.workunit.client.0.smithi100.stdout: "pool": 1, 2024-04-07T17:51:26.006 INFO:tasks.workunit.client.0.smithi100.stdout: "size": 3, 2024-04-07T17:51:26.006 INFO:tasks.workunit.client.0.smithi100.stdout: "stats": { 2024-04-07T17:51:26.006 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_raw": 723597404426, 2024-04-07T17:51:26.007 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_used": 1781760, 2024-04-07T17:51:26.007 INFO:tasks.workunit.client.0.smithi100.stdout: "compress_bytes_used": 0, 2024-04-07T17:51:26.007 INFO:tasks.workunit.client.0.smithi100.stdout: "compress_under_bytes": 0, 2024-04-07T17:51:26.007 INFO:tasks.workunit.client.0.smithi100.stdout: "data_bytes_used": 1781760, 2024-04-07T17:51:26.007 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty": 0, 2024-04-07T17:51:26.007 INFO:tasks.workunit.client.0.smithi100.stdout: "max_avail": 241199136768, 2024-04-07T17:51:26.007 INFO:tasks.workunit.client.0.smithi100.stdout: "objects": 2, 2024-04-07T17:51:26.007 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_bytes_used": 0, 2024-04-07T17:51:26.007 INFO:tasks.workunit.client.0.smithi100.stdout: "percent_used": 2.46235777012771e-06, 2024-04-07T17:51:26.007 INFO:tasks.workunit.client.0.smithi100.stdout: "quota_bytes": 0, 2024-04-07T17:51:26.007 INFO:tasks.workunit.client.0.smithi100.stdout: "quota_objects": 0, 2024-04-07T17:51:26.007 INFO:tasks.workunit.client.0.smithi100.stdout: "rd": 1214, 2024-04-07T17:51:26.007 INFO:tasks.workunit.client.0.smithi100.stdout: "rd_bytes": 2072576, 2024-04-07T17:51:26.007 INFO:tasks.workunit.client.0.smithi100.stdout: "stored": 590368, 2024-04-07T17:51:26.007 INFO:tasks.workunit.client.0.smithi100.stdout: "stored_data": 590368, 2024-04-07T17:51:26.007 INFO:tasks.workunit.client.0.smithi100.stdout: "stored_omap": 0, 2024-04-07T17:51:26.008 INFO:tasks.workunit.client.0.smithi100.stdout: "stored_raw": 1771104, 2024-04-07T17:51:26.008 INFO:tasks.workunit.client.0.smithi100.stdout: "wr": 399, 2024-04-07T17:51:26.008 INFO:tasks.workunit.client.0.smithi100.stdout: "wr_bytes": 2473984 2024-04-07T17:51:26.008 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.008 INFO:tasks.workunit.client.0.smithi100.stdout: "target_max_bytes": 0, 2024-04-07T17:51:26.008 INFO:tasks.workunit.client.0.smithi100.stdout: "target_max_objects": 0, 2024-04-07T17:51:26.008 INFO:tasks.workunit.client.0.smithi100.stdout: "type": "replicated" 2024-04-07T17:51:26.008 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.008 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:26.008 INFO:tasks.workunit.client.0.smithi100.stdout: "application": [ 2024-04-07T17:51:26.008 INFO:tasks.workunit.client.0.smithi100.stdout: "cephfs" 2024-04-07T17:51:26.008 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.008 INFO:tasks.workunit.client.0.smithi100.stdout: "cache_mode": "none", 2024-04-07T17:51:26.008 INFO:tasks.workunit.client.0.smithi100.stdout: "erasure_code_profile": {}, 2024-04-07T17:51:26.008 INFO:tasks.workunit.client.0.smithi100.stdout: "flags_names": [ 2024-04-07T17:51:26.008 INFO:tasks.workunit.client.0.smithi100.stdout: "hashpspool" 2024-04-07T17:51:26.008 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.009 INFO:tasks.workunit.client.0.smithi100.stdout: "min_size": 2, 2024-04-07T17:51:26.009 INFO:tasks.workunit.client.0.smithi100.stdout: "options": {}, 2024-04-07T17:51:26.009 INFO:tasks.workunit.client.0.smithi100.stdout: "pg_autoscale_mode": "off", 2024-04-07T17:51:26.009 INFO:tasks.workunit.client.0.smithi100.stdout: "pg_num": 32, 2024-04-07T17:51:26.009 INFO:tasks.workunit.client.0.smithi100.stdout: "pgp_num": 32, 2024-04-07T17:51:26.009 INFO:tasks.workunit.client.0.smithi100.stdout: "pool": 2, 2024-04-07T17:51:26.009 INFO:tasks.workunit.client.0.smithi100.stdout: "size": 3, 2024-04-07T17:51:26.009 INFO:tasks.workunit.client.0.smithi100.stdout: "stats": { 2024-04-07T17:51:26.009 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_raw": 723597404426, 2024-04-07T17:51:26.009 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_used": 159173, 2024-04-07T17:51:26.009 INFO:tasks.workunit.client.0.smithi100.stdout: "compress_bytes_used": 0, 2024-04-07T17:51:26.009 INFO:tasks.workunit.client.0.smithi100.stdout: "compress_under_bytes": 0, 2024-04-07T17:51:26.009 INFO:tasks.workunit.client.0.smithi100.stdout: "data_bytes_used": 110592, 2024-04-07T17:51:26.009 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty": 0, 2024-04-07T17:51:26.009 INFO:tasks.workunit.client.0.smithi100.stdout: "max_avail": 241199136768, 2024-04-07T17:51:26.009 INFO:tasks.workunit.client.0.smithi100.stdout: "objects": 22, 2024-04-07T17:51:26.009 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_bytes_used": 48581, 2024-04-07T17:51:26.010 INFO:tasks.workunit.client.0.smithi100.stdout: "percent_used": 2.1997449550781312e-07, 2024-04-07T17:51:26.010 INFO:tasks.workunit.client.0.smithi100.stdout: "quota_bytes": 0, 2024-04-07T17:51:26.010 INFO:tasks.workunit.client.0.smithi100.stdout: "quota_objects": 0, 2024-04-07T17:51:26.010 INFO:tasks.workunit.client.0.smithi100.stdout: "rd": 79, 2024-04-07T17:51:26.010 INFO:tasks.workunit.client.0.smithi100.stdout: "rd_bytes": 89088, 2024-04-07T17:51:26.010 INFO:tasks.workunit.client.0.smithi100.stdout: "stored": 22815, 2024-04-07T17:51:26.010 INFO:tasks.workunit.client.0.smithi100.stdout: "stored_data": 6622, 2024-04-07T17:51:26.010 INFO:tasks.workunit.client.0.smithi100.stdout: "stored_omap": 16193, 2024-04-07T17:51:26.010 INFO:tasks.workunit.client.0.smithi100.stdout: "stored_raw": 68445, 2024-04-07T17:51:26.010 INFO:tasks.workunit.client.0.smithi100.stdout: "wr": 68, 2024-04-07T17:51:26.010 INFO:tasks.workunit.client.0.smithi100.stdout: "wr_bytes": 36864 2024-04-07T17:51:26.010 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.010 INFO:tasks.workunit.client.0.smithi100.stdout: "target_max_bytes": 0, 2024-04-07T17:51:26.010 INFO:tasks.workunit.client.0.smithi100.stdout: "target_max_objects": 0, 2024-04-07T17:51:26.010 INFO:tasks.workunit.client.0.smithi100.stdout: "type": "replicated" 2024-04-07T17:51:26.010 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.011 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:26.011 INFO:tasks.workunit.client.0.smithi100.stdout: "application": [ 2024-04-07T17:51:26.011 INFO:tasks.workunit.client.0.smithi100.stdout: "cephfs" 2024-04-07T17:51:26.011 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.011 INFO:tasks.workunit.client.0.smithi100.stdout: "cache_mode": "none", 2024-04-07T17:51:26.011 INFO:tasks.workunit.client.0.smithi100.stdout: "erasure_code_profile": {}, 2024-04-07T17:51:26.011 INFO:tasks.workunit.client.0.smithi100.stdout: "flags_names": [ 2024-04-07T17:51:26.011 INFO:tasks.workunit.client.0.smithi100.stdout: "hashpspool" 2024-04-07T17:51:26.011 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.011 INFO:tasks.workunit.client.0.smithi100.stdout: "min_size": 2, 2024-04-07T17:51:26.011 INFO:tasks.workunit.client.0.smithi100.stdout: "options": {}, 2024-04-07T17:51:26.011 INFO:tasks.workunit.client.0.smithi100.stdout: "pg_autoscale_mode": "off", 2024-04-07T17:51:26.011 INFO:tasks.workunit.client.0.smithi100.stdout: "pg_num": 64, 2024-04-07T17:51:26.011 INFO:tasks.workunit.client.0.smithi100.stdout: "pgp_num": 64, 2024-04-07T17:51:26.011 INFO:tasks.workunit.client.0.smithi100.stdout: "pool": 3, 2024-04-07T17:51:26.011 INFO:tasks.workunit.client.0.smithi100.stdout: "size": 3, 2024-04-07T17:51:26.012 INFO:tasks.workunit.client.0.smithi100.stdout: "stats": { 2024-04-07T17:51:26.012 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_raw": 723597404426, 2024-04-07T17:51:26.012 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_used": 0, 2024-04-07T17:51:26.012 INFO:tasks.workunit.client.0.smithi100.stdout: "compress_bytes_used": 0, 2024-04-07T17:51:26.012 INFO:tasks.workunit.client.0.smithi100.stdout: "compress_under_bytes": 0, 2024-04-07T17:51:26.012 INFO:tasks.workunit.client.0.smithi100.stdout: "data_bytes_used": 0, 2024-04-07T17:51:26.012 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty": 0, 2024-04-07T17:51:26.012 INFO:tasks.workunit.client.0.smithi100.stdout: "max_avail": 241199136768, 2024-04-07T17:51:26.012 INFO:tasks.workunit.client.0.smithi100.stdout: "objects": 0, 2024-04-07T17:51:26.012 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_bytes_used": 0, 2024-04-07T17:51:26.012 INFO:tasks.workunit.client.0.smithi100.stdout: "percent_used": 0.0, 2024-04-07T17:51:26.012 INFO:tasks.workunit.client.0.smithi100.stdout: "quota_bytes": 0, 2024-04-07T17:51:26.012 INFO:tasks.workunit.client.0.smithi100.stdout: "quota_objects": 0, 2024-04-07T17:51:26.012 INFO:tasks.workunit.client.0.smithi100.stdout: "rd": 0, 2024-04-07T17:51:26.012 INFO:tasks.workunit.client.0.smithi100.stdout: "rd_bytes": 0, 2024-04-07T17:51:26.012 INFO:tasks.workunit.client.0.smithi100.stdout: "stored": 0, 2024-04-07T17:51:26.012 INFO:tasks.workunit.client.0.smithi100.stdout: "stored_data": 0, 2024-04-07T17:51:26.013 INFO:tasks.workunit.client.0.smithi100.stdout: "stored_omap": 0, 2024-04-07T17:51:26.013 INFO:tasks.workunit.client.0.smithi100.stdout: "stored_raw": 0, 2024-04-07T17:51:26.013 INFO:tasks.workunit.client.0.smithi100.stdout: "wr": 0, 2024-04-07T17:51:26.013 INFO:tasks.workunit.client.0.smithi100.stdout: "wr_bytes": 0 2024-04-07T17:51:26.013 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.013 INFO:tasks.workunit.client.0.smithi100.stdout: "target_max_bytes": 0, 2024-04-07T17:51:26.013 INFO:tasks.workunit.client.0.smithi100.stdout: "target_max_objects": 0, 2024-04-07T17:51:26.013 INFO:tasks.workunit.client.0.smithi100.stdout: "type": "replicated" 2024-04-07T17:51:26.013 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.013 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:26.013 INFO:tasks.workunit.client.0.smithi100.stdout: "application": [ 2024-04-07T17:51:26.013 INFO:tasks.workunit.client.0.smithi100.stdout: "rbd" 2024-04-07T17:51:26.013 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.013 INFO:tasks.workunit.client.0.smithi100.stdout: "cache_mode": "none", 2024-04-07T17:51:26.013 INFO:tasks.workunit.client.0.smithi100.stdout: "erasure_code_profile": {}, 2024-04-07T17:51:26.013 INFO:tasks.workunit.client.0.smithi100.stdout: "flags_names": [ 2024-04-07T17:51:26.013 INFO:tasks.workunit.client.0.smithi100.stdout: "hashpspool", 2024-04-07T17:51:26.014 INFO:tasks.workunit.client.0.smithi100.stdout: "selfmanaged_snaps" 2024-04-07T17:51:26.014 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.014 INFO:tasks.workunit.client.0.smithi100.stdout: "min_size": 2, 2024-04-07T17:51:26.014 INFO:tasks.workunit.client.0.smithi100.stdout: "options": {}, 2024-04-07T17:51:26.014 INFO:tasks.workunit.client.0.smithi100.stdout: "pg_autoscale_mode": "off", 2024-04-07T17:51:26.014 INFO:tasks.workunit.client.0.smithi100.stdout: "pg_num": 8, 2024-04-07T17:51:26.014 INFO:tasks.workunit.client.0.smithi100.stdout: "pgp_num": 8, 2024-04-07T17:51:26.014 INFO:tasks.workunit.client.0.smithi100.stdout: "pool": 4, 2024-04-07T17:51:26.014 INFO:tasks.workunit.client.0.smithi100.stdout: "size": 3, 2024-04-07T17:51:26.014 INFO:tasks.workunit.client.0.smithi100.stdout: "stats": { 2024-04-07T17:51:26.014 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_raw": 723597404426, 2024-04-07T17:51:26.014 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_used": 12595200, 2024-04-07T17:51:26.014 INFO:tasks.workunit.client.0.smithi100.stdout: "compress_bytes_used": 0, 2024-04-07T17:51:26.014 INFO:tasks.workunit.client.0.smithi100.stdout: "compress_under_bytes": 0, 2024-04-07T17:51:26.015 INFO:tasks.workunit.client.0.smithi100.stdout: "data_bytes_used": 12595200, 2024-04-07T17:51:26.015 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty": 0, 2024-04-07T17:51:26.015 INFO:tasks.workunit.client.0.smithi100.stdout: "max_avail": 241199136768, 2024-04-07T17:51:26.015 INFO:tasks.workunit.client.0.smithi100.stdout: "objects": 4, 2024-04-07T17:51:26.015 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_bytes_used": 0, 2024-04-07T17:51:26.015 INFO:tasks.workunit.client.0.smithi100.stdout: "percent_used": 1.74060623976402e-05, 2024-04-07T17:51:26.015 INFO:tasks.workunit.client.0.smithi100.stdout: "quota_bytes": 0, 2024-04-07T17:51:26.015 INFO:tasks.workunit.client.0.smithi100.stdout: "quota_objects": 0, 2024-04-07T17:51:26.015 INFO:tasks.workunit.client.0.smithi100.stdout: "rd": 9087, 2024-04-07T17:51:26.015 INFO:tasks.workunit.client.0.smithi100.stdout: "rd_bytes": 3222810624, 2024-04-07T17:51:26.015 INFO:tasks.workunit.client.0.smithi100.stdout: "stored": 4194323, 2024-04-07T17:51:26.015 INFO:tasks.workunit.client.0.smithi100.stdout: "stored_data": 4194323, 2024-04-07T17:51:26.015 INFO:tasks.workunit.client.0.smithi100.stdout: "stored_omap": 0, 2024-04-07T17:51:26.015 INFO:tasks.workunit.client.0.smithi100.stdout: "stored_raw": 12582969, 2024-04-07T17:51:26.015 INFO:tasks.workunit.client.0.smithi100.stdout: "wr": 24273, 2024-04-07T17:51:26.015 INFO:tasks.workunit.client.0.smithi100.stdout: "wr_bytes": 3103659008 2024-04-07T17:51:26.016 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.016 INFO:tasks.workunit.client.0.smithi100.stdout: "target_max_bytes": 0, 2024-04-07T17:51:26.016 INFO:tasks.workunit.client.0.smithi100.stdout: "target_max_objects": 0, 2024-04-07T17:51:26.016 INFO:tasks.workunit.client.0.smithi100.stdout: "type": "replicated" 2024-04-07T17:51:26.016 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:26.016 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.016 INFO:tasks.workunit.client.0.smithi100.stdout: "rbd": { 2024-04-07T17:51:26.016 INFO:tasks.workunit.client.0.smithi100.stdout: "mirroring_by_pool": [ 2024-04-07T17:51:26.016 INFO:tasks.workunit.client.0.smithi100.stdout: false 2024-04-07T17:51:26.016 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.016 INFO:tasks.workunit.client.0.smithi100.stdout: "num_images_by_pool": [ 2024-04-07T17:51:26.016 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T17:51:26.016 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.016 INFO:tasks.workunit.client.0.smithi100.stdout: "num_pools": 1 2024-04-07T17:51:26.016 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.016 INFO:tasks.workunit.client.0.smithi100.stdout: "report_id": "ee983b25-10fa-4633-b6cf-1c3036b91b52", 2024-04-07T17:51:26.017 INFO:tasks.workunit.client.0.smithi100.stdout: "report_timestamp": "2024-04-07T17:51:24.819883", 2024-04-07T17:51:26.017 INFO:tasks.workunit.client.0.smithi100.stdout: "report_version": 1, 2024-04-07T17:51:26.017 INFO:tasks.workunit.client.0.smithi100.stdout: "rocksdb_stats": { 2024-04-07T17:51:26.017 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "7.9.2" 2024-04-07T17:51:26.017 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.017 INFO:tasks.workunit.client.0.smithi100.stdout: "rook": { 2024-04-07T17:51:26.017 INFO:tasks.workunit.client.0.smithi100.stdout: "cluster": { 2024-04-07T17:51:26.017 INFO:tasks.workunit.client.0.smithi100.stdout: "external-mode": null, 2024-04-07T17:51:26.017 INFO:tasks.workunit.client.0.smithi100.stdout: "mon": { 2024-04-07T17:51:26.017 INFO:tasks.workunit.client.0.smithi100.stdout: "allow-multiple-per-node": null, 2024-04-07T17:51:26.017 INFO:tasks.workunit.client.0.smithi100.stdout: "count": null, 2024-04-07T17:51:26.017 INFO:tasks.workunit.client.0.smithi100.stdout: "max-id": null, 2024-04-07T17:51:26.017 INFO:tasks.workunit.client.0.smithi100.stdout: "pvc": { 2024-04-07T17:51:26.017 INFO:tasks.workunit.client.0.smithi100.stdout: "enabled": null 2024-04-07T17:51:26.017 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.017 INFO:tasks.workunit.client.0.smithi100.stdout: "stretch": { 2024-04-07T17:51:26.017 INFO:tasks.workunit.client.0.smithi100.stdout: "enabled": null 2024-04-07T17:51:26.018 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:26.018 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.018 INFO:tasks.workunit.client.0.smithi100.stdout: "network": { 2024-04-07T17:51:26.018 INFO:tasks.workunit.client.0.smithi100.stdout: "provider": null 2024-04-07T17:51:26.018 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.018 INFO:tasks.workunit.client.0.smithi100.stdout: "storage": { 2024-04-07T17:51:26.018 INFO:tasks.workunit.client.0.smithi100.stdout: "device-set": { 2024-04-07T17:51:26.018 INFO:tasks.workunit.client.0.smithi100.stdout: "count": { 2024-04-07T17:51:26.018 INFO:tasks.workunit.client.0.smithi100.stdout: "non-portable": null, 2024-04-07T17:51:26.018 INFO:tasks.workunit.client.0.smithi100.stdout: "portable": null, 2024-04-07T17:51:26.018 INFO:tasks.workunit.client.0.smithi100.stdout: "total": null 2024-04-07T17:51:26.018 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:26.018 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:26.018 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:26.018 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.019 INFO:tasks.workunit.client.0.smithi100.stdout: "csi": { 2024-04-07T17:51:26.019 INFO:tasks.workunit.client.0.smithi100.stdout: "version": null 2024-04-07T17:51:26.019 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.019 INFO:tasks.workunit.client.0.smithi100.stdout: "kubernetes": { 2024-04-07T17:51:26.019 INFO:tasks.workunit.client.0.smithi100.stdout: "version": null 2024-04-07T17:51:26.019 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.019 INFO:tasks.workunit.client.0.smithi100.stdout: "node": { 2024-04-07T17:51:26.019 INFO:tasks.workunit.client.0.smithi100.stdout: "count": { 2024-04-07T17:51:26.019 INFO:tasks.workunit.client.0.smithi100.stdout: "kubernetes-total": null, 2024-04-07T17:51:26.019 INFO:tasks.workunit.client.0.smithi100.stdout: "with-ceph-daemons": null, 2024-04-07T17:51:26.019 INFO:tasks.workunit.client.0.smithi100.stdout: "with-csi-cephfs-plugin": null, 2024-04-07T17:51:26.019 INFO:tasks.workunit.client.0.smithi100.stdout: "with-csi-nfs-plugin": null, 2024-04-07T17:51:26.019 INFO:tasks.workunit.client.0.smithi100.stdout: "with-csi-rbd-plugin": null 2024-04-07T17:51:26.019 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:26.019 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.019 INFO:tasks.workunit.client.0.smithi100.stdout: "usage": { 2024-04-07T17:51:26.019 INFO:tasks.workunit.client.0.smithi100.stdout: "storage-class": { 2024-04-07T17:51:26.020 INFO:tasks.workunit.client.0.smithi100.stdout: "count": { 2024-04-07T17:51:26.020 INFO:tasks.workunit.client.0.smithi100.stdout: "bucket": null, 2024-04-07T17:51:26.020 INFO:tasks.workunit.client.0.smithi100.stdout: "cephfs": null, 2024-04-07T17:51:26.020 INFO:tasks.workunit.client.0.smithi100.stdout: "nfs": null, 2024-04-07T17:51:26.020 INFO:tasks.workunit.client.0.smithi100.stdout: "rbd": null, 2024-04-07T17:51:26.020 INFO:tasks.workunit.client.0.smithi100.stdout: "total": null 2024-04-07T17:51:26.020 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:26.020 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:26.020 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.020 INFO:tasks.workunit.client.0.smithi100.stdout: "version": null 2024-04-07T17:51:26.020 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.020 INFO:tasks.workunit.client.0.smithi100.stdout: "services": { 2024-04-07T17:51:26.020 INFO:tasks.workunit.client.0.smithi100.stdout: "mon": 1 2024-04-07T17:51:26.020 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.020 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_per_pg": [ 2024-04-07T17:51:26.020 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:26.021 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:26.021 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:26.021 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:26.021 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T17:51:26.021 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.021 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 6, 2024-04-07T17:51:26.021 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:26.021 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:26.021 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:26.021 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:26.021 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:26.021 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:26.021 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.371282+0000", 2024-04-07T17:51:26.021 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:08.962622+0000", 2024-04-07T17:51:26.021 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:08.962622+0000", 2024-04-07T17:51:26.022 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:08.963061+0000", 2024-04-07T17:51:26.022 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.371282+0000", 2024-04-07T17:51:26.022 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.022 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:26.022 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.022 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 554, 2024-04-07T17:51:26.022 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.371282+0000", 2024-04-07T17:51:26.022 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.371282+0000", 2024-04-07T17:51:26.022 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.371282+0000", 2024-04-07T17:51:26.022 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:26.022 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:26.022 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.022 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.371282+0000", 2024-04-07T17:51:26.022 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.371282+0000", 2024-04-07T17:51:26.022 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:26.022 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:26.023 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:26.023 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:26.023 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 553, 2024-04-07T17:51:26.023 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:26.023 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:26.023 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:26.023 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:26.023 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:26.023 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:26.023 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:26.023 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:26.023 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.2c", 2024-04-07T17:51:26.023 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:26.023 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:26.023 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:26.024 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4846, 2024-04-07T17:51:26.024 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:26.024 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T02:53:55.780425+0000", 2024-04-07T17:51:26.024 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:26.024 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:26.024 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:26.024 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:26.024 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:26.024 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:26.024 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:26.024 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:26.024 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:26.024 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:26.024 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:26.024 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:26.024 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:26.025 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:26.025 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:26.025 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:26.025 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:26.025 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:26.025 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:26.025 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:26.025 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:26.025 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:26.025 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:26.025 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:26.025 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:26.025 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:26.025 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:26.025 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:26.026 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:26.026 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:26.026 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:26.026 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:26.026 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:26.026 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:26.026 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:26.026 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:26.026 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:26.026 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:26.026 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:26.026 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:26.026 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:26.026 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:26.026 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:26.027 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.027 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:26.027 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:26.027 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:26.027 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:26.027 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:26.027 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T17:51:26.027 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.027 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 6, 2024-04-07T17:51:26.027 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:26.027 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.027 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:26.027 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:26.027 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:26.027 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:26.027 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T17:51:26.028 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.028 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 0, 2024-04-07T17:51:26.028 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:26.028 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:26.028 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:26.028 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:26.028 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:26.028 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:26.028 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:35.989606+0000", 2024-04-07T17:51:26.028 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:39.003945+0000", 2024-04-07T17:51:26.028 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:39.003945+0000", 2024-04-07T17:51:26.028 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:39.004542+0000", 2024-04-07T17:51:26.028 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:35.989606+0000", 2024-04-07T17:51:26.028 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.028 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:26.028 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.029 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:26.029 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:35.989606+0000", 2024-04-07T17:51:26.029 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:35.989606+0000", 2024-04-07T17:51:26.029 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:35.989606+0000", 2024-04-07T17:51:26.029 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:26.029 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:26.029 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.029 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:35.989606+0000", 2024-04-07T17:51:26.029 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:35.989606+0000", 2024-04-07T17:51:26.029 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:26.029 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:26.029 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:26.029 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:26.029 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:26.029 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:26.029 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:26.030 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:26.030 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:26.030 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:26.030 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:26.030 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:26.030 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:26.030 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.2b", 2024-04-07T17:51:26.030 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:26.030 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:26.030 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:26.030 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4997, 2024-04-07T17:51:26.030 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:26.031 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T18:56:04.984002+0000", 2024-04-07T17:51:26.031 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:26.031 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:26.031 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:26.031 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:26.031 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:26.031 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:26.031 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:26.031 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:26.031 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:26.031 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:26.031 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:26.031 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:26.031 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:26.031 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:26.032 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:26.032 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:26.032 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:26.032 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:26.032 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:26.032 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:26.032 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:26.032 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:26.032 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:26.032 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:26.032 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:26.032 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:26.032 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:26.032 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:26.032 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:26.032 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:26.033 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:26.033 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:26.033 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:26.033 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:26.033 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:26.033 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:26.033 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:26.033 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:26.033 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:26.033 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:26.033 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:26.033 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:26.033 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:26.033 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.033 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:26.033 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:26.034 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:26.034 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:26.034 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:26.034 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T17:51:26.034 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.034 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 0, 2024-04-07T17:51:26.034 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:26.034 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.034 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:26.034 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:26.034 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:26.034 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:26.034 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T17:51:26.034 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.035 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 5, 2024-04-07T17:51:26.035 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:26.035 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:26.035 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:26.035 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:26.035 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:26.035 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:26.035 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.369807+0000", 2024-04-07T17:51:26.035 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:38.977861+0000", 2024-04-07T17:51:26.035 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:38.977861+0000", 2024-04-07T17:51:26.035 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:38.978928+0000", 2024-04-07T17:51:26.035 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.369807+0000", 2024-04-07T17:51:26.035 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.035 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:26.036 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.036 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:26.036 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.369807+0000", 2024-04-07T17:51:26.036 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.369807+0000", 2024-04-07T17:51:26.036 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.369807+0000", 2024-04-07T17:51:26.036 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:26.036 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:26.036 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.036 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.369807+0000", 2024-04-07T17:51:26.036 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.369807+0000", 2024-04-07T17:51:26.036 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:26.036 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:26.036 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:26.036 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:26.037 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:26.037 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:26.037 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:26.037 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:26.037 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:26.037 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:26.037 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:26.037 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:26.037 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:26.037 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.2a", 2024-04-07T17:51:26.037 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:26.037 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:26.037 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:26.037 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4871, 2024-04-07T17:51:26.037 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:26.037 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T01:59:28.849179+0000", 2024-04-07T17:51:26.038 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:26.038 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:26.038 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:26.038 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:26.038 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:26.038 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:26.038 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:26.038 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:26.038 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:26.038 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:26.038 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:26.038 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:26.038 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:26.038 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:26.039 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:26.039 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:26.039 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:26.039 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:26.039 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:26.039 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:26.039 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:26.039 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:26.039 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:26.039 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:26.039 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:26.039 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:26.039 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:26.039 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:26.039 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:26.039 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:26.040 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:26.040 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:26.040 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:26.040 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:26.040 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:26.040 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:26.040 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:26.040 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:26.040 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:26.040 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:26.040 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:26.040 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:26.040 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:26.040 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.040 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:26.040 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:26.041 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:26.041 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:26.041 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:26.041 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T17:51:26.041 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.041 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 5, 2024-04-07T17:51:26.041 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:26.041 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.041 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:26.041 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:26.041 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:26.041 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:26.041 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T17:51:26.041 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.041 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 3, 2024-04-07T17:51:26.041 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:26.042 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:26.042 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:26.042 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:26.042 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:26.042 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:26.042 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:35.976785+0000", 2024-04-07T17:51:26.042 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:24:13.933027+0000", 2024-04-07T17:51:26.042 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:24:13.933027+0000", 2024-04-07T17:51:26.042 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:24:13.933359+0000", 2024-04-07T17:51:26.042 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:35.976785+0000", 2024-04-07T17:51:26.042 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.042 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:26.042 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.042 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 500, 2024-04-07T17:51:26.043 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:35.976785+0000", 2024-04-07T17:51:26.043 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:35.976785+0000", 2024-04-07T17:51:26.043 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:35.976785+0000", 2024-04-07T17:51:26.043 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:26.043 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:26.043 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.043 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:35.976785+0000", 2024-04-07T17:51:26.043 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:35.976785+0000", 2024-04-07T17:51:26.043 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:26.043 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:26.043 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:26.043 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:26.043 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 499, 2024-04-07T17:51:26.043 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:26.043 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:26.043 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:26.044 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:26.044 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:26.044 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:26.044 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:26.044 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:26.044 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.29", 2024-04-07T17:51:26.044 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:26.044 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:26.044 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:26.044 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4966, 2024-04-07T17:51:26.044 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:26.044 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T00:08:14.265846+0000", 2024-04-07T17:51:26.044 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:26.044 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:26.044 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:26.044 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:26.045 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:26.045 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:26.045 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:26.045 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:26.045 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:26.045 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:26.045 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:26.045 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:26.045 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:26.045 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:26.045 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:26.045 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:26.045 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:26.045 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:26.045 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:26.046 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:26.046 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:26.046 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:26.046 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:26.046 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:26.046 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:26.046 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:26.046 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:26.046 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:26.046 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:26.046 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:26.046 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:26.046 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:26.047 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:26.047 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:26.047 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:26.047 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:26.047 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:26.047 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:26.047 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:26.047 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:26.047 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:26.047 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:26.047 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:26.047 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.047 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:26.047 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:26.047 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:26.047 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:26.048 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:26.048 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T17:51:26.048 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.048 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 3, 2024-04-07T17:51:26.048 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:26.048 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.048 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:26.048 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:26.048 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:26.048 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:26.048 INFO:tasks.workunit.client.0.smithi100.stdout: 2 2024-04-07T17:51:26.048 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.048 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 5, 2024-04-07T17:51:26.048 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:26.048 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:26.048 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:26.048 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:26.049 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:26.049 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:26.049 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.363975+0000", 2024-04-07T17:51:26.049 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:24:41.553450+0000", 2024-04-07T17:51:26.049 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:24:41.553450+0000", 2024-04-07T17:51:26.049 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:24:41.557281+0000", 2024-04-07T17:51:26.049 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.363975+0000", 2024-04-07T17:51:26.049 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.049 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:26.049 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.049 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 527, 2024-04-07T17:51:26.049 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.363975+0000", 2024-04-07T17:51:26.049 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.363975+0000", 2024-04-07T17:51:26.049 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.363975+0000", 2024-04-07T17:51:26.049 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:26.050 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:26.050 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.050 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.363975+0000", 2024-04-07T17:51:26.050 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.363975+0000", 2024-04-07T17:51:26.050 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:26.050 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:26.050 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:26.050 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:26.050 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 526, 2024-04-07T17:51:26.050 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:26.050 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:26.050 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:26.050 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:26.050 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:26.050 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:26.050 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:26.051 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:26.051 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.28", 2024-04-07T17:51:26.051 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:26.051 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:26.051 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:26.051 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4884, 2024-04-07T17:51:26.051 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:26.051 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T19:49:37.795766+0000", 2024-04-07T17:51:26.051 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:26.051 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:26.051 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:26.051 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:26.051 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:26.051 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:26.051 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:26.051 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:26.052 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:26.052 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:26.052 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:26.052 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:26.052 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:26.052 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:26.052 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:26.052 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:26.052 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:26.052 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:26.052 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:26.052 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:26.052 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:26.052 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:26.052 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:26.053 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:26.053 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:26.053 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:26.053 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:26.053 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:26.053 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:26.053 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:26.053 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:26.053 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:26.053 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:26.053 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:26.053 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:26.053 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:26.053 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:26.053 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:26.053 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:26.054 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:26.054 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:26.054 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:26.054 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:26.054 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.054 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:26.054 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:26.054 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:26.054 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:26.054 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:26.054 INFO:tasks.workunit.client.0.smithi100.stdout: 2 2024-04-07T17:51:26.054 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.054 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 5, 2024-04-07T17:51:26.054 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:26.054 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.054 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:26.054 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:26.055 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:26.055 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:26.055 INFO:tasks.workunit.client.0.smithi100.stdout: 2 2024-04-07T17:51:26.055 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.055 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 3, 2024-04-07T17:51:26.055 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:26.055 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:26.055 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:26.055 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:26.055 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:26.055 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:26.055 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:35.982188+0000", 2024-04-07T17:51:26.055 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:24:13.933691+0000", 2024-04-07T17:51:26.055 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:24:13.933691+0000", 2024-04-07T17:51:26.056 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:24:13.934646+0000", 2024-04-07T17:51:26.056 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:35.982188+0000", 2024-04-07T17:51:26.056 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.056 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:26.056 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.056 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 500, 2024-04-07T17:51:26.056 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:35.982188+0000", 2024-04-07T17:51:26.056 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:35.982188+0000", 2024-04-07T17:51:26.056 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:35.982188+0000", 2024-04-07T17:51:26.056 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:26.056 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:26.056 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.056 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:35.982188+0000", 2024-04-07T17:51:26.056 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:35.982188+0000", 2024-04-07T17:51:26.056 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:26.057 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:26.057 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:26.057 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:26.057 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 499, 2024-04-07T17:51:26.057 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:26.057 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:26.057 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:26.057 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:26.057 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:26.057 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:26.057 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:26.057 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:26.057 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.27", 2024-04-07T17:51:26.057 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:26.057 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:26.057 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:26.057 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4954, 2024-04-07T17:51:26.058 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:26.058 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T19:33:33.960559+0000", 2024-04-07T17:51:26.058 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:26.058 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:26.058 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:26.058 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:26.058 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:26.058 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:26.058 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:26.058 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:26.058 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:26.058 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:26.058 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:26.058 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:26.058 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:26.059 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:26.059 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:26.059 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:26.059 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:26.059 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:26.059 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:26.059 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:26.059 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:26.059 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:26.059 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:26.059 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:26.059 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:26.059 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:26.059 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:26.059 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:26.059 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:26.060 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:26.060 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:26.060 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:26.060 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:26.060 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:26.060 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:26.060 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:26.060 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:26.060 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:26.060 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:26.060 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:26.060 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:26.060 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:26.060 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:26.060 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.060 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:26.061 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:26.061 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:26.061 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:26.061 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:26.061 INFO:tasks.workunit.client.0.smithi100.stdout: 2 2024-04-07T17:51:26.061 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.061 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 3, 2024-04-07T17:51:26.061 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:26.061 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.061 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:26.061 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:26.061 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:26.061 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:26.061 INFO:tasks.workunit.client.0.smithi100.stdout: 4 2024-04-07T17:51:26.062 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.062 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 6, 2024-04-07T17:51:26.062 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:26.062 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:26.062 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:26.062 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:26.062 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:26.062 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:26.062 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.368356+0000", 2024-04-07T17:51:26.062 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:38.960595+0000", 2024-04-07T17:51:26.062 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:38.960595+0000", 2024-04-07T17:51:26.062 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:38.962091+0000", 2024-04-07T17:51:26.062 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.368356+0000", 2024-04-07T17:51:26.062 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.062 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:26.062 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.063 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:26.063 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.368356+0000", 2024-04-07T17:51:26.063 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.368356+0000", 2024-04-07T17:51:26.063 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.368356+0000", 2024-04-07T17:51:26.063 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:26.063 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:26.063 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.063 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.368356+0000", 2024-04-07T17:51:26.063 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.368356+0000", 2024-04-07T17:51:26.063 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:26.063 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:26.063 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:26.063 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:26.063 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:26.063 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:26.063 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:26.064 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:26.064 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:26.064 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:26.064 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:26.064 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:26.064 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:26.064 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.26", 2024-04-07T17:51:26.064 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:26.064 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:26.064 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:26.064 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4841, 2024-04-07T17:51:26.064 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:26.064 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T19:11:01.462021+0000", 2024-04-07T17:51:26.064 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:26.064 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:26.064 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:26.064 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:26.065 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:26.065 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:26.065 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:26.065 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:26.065 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:26.065 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:26.065 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:26.065 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:26.065 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:26.065 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:26.065 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:26.065 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:26.065 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:26.065 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:26.066 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:26.066 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:26.066 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:26.066 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:26.066 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:26.066 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:26.066 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:26.066 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:26.066 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:26.066 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:26.066 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:26.066 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:26.066 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:26.066 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:26.066 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:26.067 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:26.067 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:26.067 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:26.067 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:26.067 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:26.067 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:26.067 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:26.067 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:26.067 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:26.067 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:26.067 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.067 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:26.067 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:26.067 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:26.067 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:26.067 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:26.068 INFO:tasks.workunit.client.0.smithi100.stdout: 4 2024-04-07T17:51:26.068 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.068 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 6, 2024-04-07T17:51:26.068 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:26.068 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.068 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:26.068 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:26.068 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:26.068 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:26.068 INFO:tasks.workunit.client.0.smithi100.stdout: 2 2024-04-07T17:51:26.068 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.068 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 1, 2024-04-07T17:51:26.068 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:26.068 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:26.068 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:26.068 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:26.068 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:26.068 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:26.069 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.334473+0000", 2024-04-07T17:51:26.069 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:23:43.924498+0000", 2024-04-07T17:51:26.069 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:23:43.924498+0000", 2024-04-07T17:51:26.069 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:23:43.926295+0000", 2024-04-07T17:51:26.069 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.334473+0000", 2024-04-07T17:51:26.069 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.069 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:26.069 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.069 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 471, 2024-04-07T17:51:26.069 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.334473+0000", 2024-04-07T17:51:26.069 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.334473+0000", 2024-04-07T17:51:26.069 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.334473+0000", 2024-04-07T17:51:26.069 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:26.069 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:26.069 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.069 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.334473+0000", 2024-04-07T17:51:26.069 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.334473+0000", 2024-04-07T17:51:26.070 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:26.070 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:26.070 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:26.070 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:26.070 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 470, 2024-04-07T17:51:26.070 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:26.070 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:26.070 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:26.070 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:26.070 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:26.070 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:26.070 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:26.070 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:26.070 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.25", 2024-04-07T17:51:26.070 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:26.071 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:26.071 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:26.071 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4981, 2024-04-07T17:51:26.071 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:26.071 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T17:58:42.938076+0000", 2024-04-07T17:51:26.071 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:26.071 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:26.071 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:26.071 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:26.071 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:26.071 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:26.071 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:26.071 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:26.071 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:26.071 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:26.071 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:26.071 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:26.072 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:26.072 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:26.072 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:26.072 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:26.072 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:26.072 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:26.072 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:26.072 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:26.072 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:26.072 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:26.072 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:26.072 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:26.072 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:26.072 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:26.072 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:26.072 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:26.072 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:26.073 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:26.073 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:26.073 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:26.073 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:26.073 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:26.073 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:26.073 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:26.073 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:26.073 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:26.073 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:26.073 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:26.073 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:26.073 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:26.073 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:26.073 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.073 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:26.073 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:26.074 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:26.074 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:26.074 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:26.074 INFO:tasks.workunit.client.0.smithi100.stdout: 2 2024-04-07T17:51:26.074 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.074 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 1, 2024-04-07T17:51:26.074 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:26.074 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.074 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:26.074 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:26.074 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:26.074 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:26.074 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T17:51:26.074 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.074 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 3, 2024-04-07T17:51:26.074 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:26.075 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:26.075 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:26.075 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:26.075 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:26.075 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:26.075 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:35.977918+0000", 2024-04-07T17:51:26.075 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:24:13.930420+0000", 2024-04-07T17:51:26.075 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:24:13.930420+0000", 2024-04-07T17:51:26.075 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:24:13.931275+0000", 2024-04-07T17:51:26.075 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:35.977918+0000", 2024-04-07T17:51:26.075 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.075 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:26.075 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.075 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 500, 2024-04-07T17:51:26.075 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:35.977918+0000", 2024-04-07T17:51:26.076 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:35.977918+0000", 2024-04-07T17:51:26.076 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:35.977918+0000", 2024-04-07T17:51:26.076 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:26.076 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:26.076 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.076 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:35.977918+0000", 2024-04-07T17:51:26.076 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:35.977918+0000", 2024-04-07T17:51:26.076 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:26.076 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:26.076 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:26.076 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:26.076 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 499, 2024-04-07T17:51:26.076 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:26.076 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:26.076 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:26.077 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:26.077 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:26.077 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:26.077 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:26.077 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:26.077 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.24", 2024-04-07T17:51:26.077 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:26.077 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:26.077 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:26.077 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4940, 2024-04-07T17:51:26.077 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:26.077 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T00:26:09.028937+0000", 2024-04-07T17:51:26.077 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:26.077 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:26.077 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:26.077 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:26.077 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:26.078 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:26.078 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:26.078 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:26.078 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:26.078 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:26.078 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:26.078 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:26.078 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:26.078 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:26.078 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:26.078 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:26.078 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:26.078 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:26.078 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:26.078 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:26.079 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:26.079 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:26.079 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:26.079 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:26.079 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:26.079 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:26.079 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:26.079 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:26.079 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:26.079 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:26.079 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:26.079 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:26.079 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:26.079 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:26.079 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:26.079 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:26.079 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:26.080 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:26.080 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:26.080 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:26.080 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:26.080 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:26.080 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:26.080 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.080 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:26.080 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:26.080 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:26.080 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:26.080 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:26.080 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T17:51:26.080 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.080 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 3, 2024-04-07T17:51:26.080 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:26.080 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.081 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:26.081 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:26.081 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:26.081 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:26.081 INFO:tasks.workunit.client.0.smithi100.stdout: 4 2024-04-07T17:51:26.081 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.081 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 0, 2024-04-07T17:51:26.081 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:26.081 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:26.081 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:26.081 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:26.081 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:26.081 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:26.081 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:35.989350+0000", 2024-04-07T17:51:26.081 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:24:13.962893+0000", 2024-04-07T17:51:26.081 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:24:13.962893+0000", 2024-04-07T17:51:26.081 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:24:13.964515+0000", 2024-04-07T17:51:26.082 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:35.989350+0000", 2024-04-07T17:51:26.082 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.082 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:26.082 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.082 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 500, 2024-04-07T17:51:26.082 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:35.989350+0000", 2024-04-07T17:51:26.082 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:35.989350+0000", 2024-04-07T17:51:26.082 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:35.989350+0000", 2024-04-07T17:51:26.082 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:26.082 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:26.082 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.082 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:35.989350+0000", 2024-04-07T17:51:26.082 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:35.989350+0000", 2024-04-07T17:51:26.082 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:26.082 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:26.083 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:26.083 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:26.083 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 499, 2024-04-07T17:51:26.083 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:26.083 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:26.083 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:26.083 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:26.083 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:26.083 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:26.083 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:26.083 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:26.083 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.23", 2024-04-07T17:51:26.083 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:26.083 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:26.083 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:26.083 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 5000, 2024-04-07T17:51:26.084 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:26.084 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T17:30:33.589121+0000", 2024-04-07T17:51:26.084 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:26.084 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:26.084 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:26.084 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:26.084 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:26.084 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:26.084 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:26.084 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:26.084 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:26.084 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:26.084 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:26.084 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:26.084 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:26.084 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:26.085 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:26.085 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:26.085 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:26.085 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:26.085 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:26.085 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:26.085 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:26.085 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:26.085 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:26.085 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:26.085 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:26.085 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:26.085 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:26.085 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:26.085 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:26.086 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:26.086 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:26.086 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:26.086 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:26.086 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:26.086 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:26.086 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:26.086 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:26.086 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:26.086 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:26.086 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:26.086 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:26.086 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:26.086 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:26.086 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.087 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:26.087 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:26.087 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:26.087 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:26.087 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:26.087 INFO:tasks.workunit.client.0.smithi100.stdout: 4 2024-04-07T17:51:26.087 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.087 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 0, 2024-04-07T17:51:26.087 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:26.087 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.087 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:26.087 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:26.087 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:26.087 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:26.087 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T17:51:26.087 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.088 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 5, 2024-04-07T17:51:26.088 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:26.088 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:26.088 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:26.088 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:26.088 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:26.088 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:26.088 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.369012+0000", 2024-04-07T17:51:26.088 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:24:41.561766+0000", 2024-04-07T17:51:26.088 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:24:41.561766+0000", 2024-04-07T17:51:26.088 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:24:41.564435+0000", 2024-04-07T17:51:26.088 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.369012+0000", 2024-04-07T17:51:26.088 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.088 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:26.088 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.088 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 527, 2024-04-07T17:51:26.089 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.369012+0000", 2024-04-07T17:51:26.089 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.369012+0000", 2024-04-07T17:51:26.089 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.369012+0000", 2024-04-07T17:51:26.089 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:26.089 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:26.089 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.089 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.369012+0000", 2024-04-07T17:51:26.089 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.369012+0000", 2024-04-07T17:51:26.089 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:26.089 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:26.089 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:26.089 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:26.089 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 526, 2024-04-07T17:51:26.089 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:26.089 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:26.089 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:26.090 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:26.090 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:26.090 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:26.090 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:26.090 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:26.090 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.22", 2024-04-07T17:51:26.090 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:26.090 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:26.090 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:26.090 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4886, 2024-04-07T17:51:26.090 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:26.090 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T17:08:34.046549+0000", 2024-04-07T17:51:26.090 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:26.090 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:26.090 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:26.091 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:26.091 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:26.091 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:26.091 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:26.091 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:26.091 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:26.091 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:26.091 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:26.091 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:26.091 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:26.091 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:26.091 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:26.091 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:26.091 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:26.091 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:26.091 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:26.092 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:26.092 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:26.092 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:26.092 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:26.092 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:26.092 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:26.092 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:26.092 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:26.092 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:26.092 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:26.092 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:26.092 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:26.092 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:26.092 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:26.092 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:26.092 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:26.092 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:26.093 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:26.093 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:26.093 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:26.093 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:26.093 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:26.093 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:26.093 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:26.093 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.093 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:26.093 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:26.093 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:26.093 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:26.093 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:26.093 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T17:51:26.093 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.093 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 5, 2024-04-07T17:51:26.093 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:26.093 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.094 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:26.094 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:26.094 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T17:51:26.094 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:26.094 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T17:51:26.094 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.094 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 2, 2024-04-07T17:51:26.094 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:26.094 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:26.094 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:26.094 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:26.094 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:26.094 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:26.094 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.354974+0000", 2024-04-07T17:51:26.094 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:08.976322+0000", 2024-04-07T17:51:26.095 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:08.976322+0000", 2024-04-07T17:51:26.095 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:08.976897+0000", 2024-04-07T17:51:26.095 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.354974+0000", 2024-04-07T17:51:26.095 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.095 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:26.095 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.095 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 554, 2024-04-07T17:51:26.095 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.354974+0000", 2024-04-07T17:51:26.095 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.354974+0000", 2024-04-07T17:51:26.095 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.354974+0000", 2024-04-07T17:51:26.095 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:26.095 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:26.095 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.095 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.354974+0000", 2024-04-07T17:51:26.095 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.354974+0000", 2024-04-07T17:51:26.095 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:26.096 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:26.096 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:26.096 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:26.096 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 553, 2024-04-07T17:51:26.096 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:26.096 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:26.096 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:26.096 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:26.096 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:26.096 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:26.096 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:26.096 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:26.096 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.21", 2024-04-07T17:51:26.096 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:26.096 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:26.096 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:26.097 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4970, 2024-04-07T17:51:26.097 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:26.097 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T18:02:01.757959+0000", 2024-04-07T17:51:26.097 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:26.097 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:26.097 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:26.097 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:26.097 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:26.097 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:26.097 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:26.097 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:26.097 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:26.097 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:26.097 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:26.097 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:26.097 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:26.097 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:26.098 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:26.098 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:26.098 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:26.098 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:26.098 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:26.098 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:26.098 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:26.098 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:26.098 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:26.098 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:26.098 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:26.098 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:26.098 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:26.098 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:26.098 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:26.098 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:26.099 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:26.099 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:26.099 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:26.099 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:26.099 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:26.099 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:26.099 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:26.099 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:26.099 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:26.099 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:26.099 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:26.099 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:26.099 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:26.099 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.099 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:26.099 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:26.099 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:26.100 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T17:51:26.100 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:26.100 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T17:51:26.100 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.100 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 2, 2024-04-07T17:51:26.100 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:26.100 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.100 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:26.100 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:26.100 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:26.100 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:26.100 INFO:tasks.workunit.client.0.smithi100.stdout: 5 2024-04-07T17:51:26.100 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.100 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 6, 2024-04-07T17:51:26.100 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:26.100 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:26.101 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:26.101 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:26.101 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:26.101 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:26.101 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.370975+0000", 2024-04-07T17:51:26.101 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:08.975767+0000", 2024-04-07T17:51:26.101 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:08.975767+0000", 2024-04-07T17:51:26.101 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:08.976755+0000", 2024-04-07T17:51:26.101 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.370975+0000", 2024-04-07T17:51:26.101 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.101 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:26.101 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.101 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 554, 2024-04-07T17:51:26.101 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.370975+0000", 2024-04-07T17:51:26.101 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.370975+0000", 2024-04-07T17:51:26.102 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.370975+0000", 2024-04-07T17:51:26.102 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:26.102 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:26.102 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.102 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.370975+0000", 2024-04-07T17:51:26.102 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.370975+0000", 2024-04-07T17:51:26.102 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:26.102 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:26.102 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:26.102 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:26.102 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 553, 2024-04-07T17:51:26.102 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:26.102 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:26.102 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:26.102 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:26.103 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:26.103 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:26.103 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:26.103 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:26.103 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.20", 2024-04-07T17:51:26.103 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:26.103 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:26.103 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:26.103 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4834, 2024-04-07T17:51:26.103 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:26.103 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T16:33:00.288075+0000", 2024-04-07T17:51:26.103 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:26.103 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:26.103 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:26.103 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:26.103 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:26.104 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:26.104 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:26.104 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:26.104 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:26.104 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:26.104 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:26.104 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:26.104 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:26.104 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:26.104 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:26.104 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:26.104 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:26.104 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:26.104 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:26.104 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:26.104 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:26.105 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:26.105 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:26.105 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:26.105 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:26.105 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:26.105 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:26.105 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:26.105 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:26.105 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:26.105 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:26.105 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:26.105 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:26.105 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:26.105 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:26.105 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:26.106 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:26.106 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:26.106 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:26.106 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:26.106 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:26.106 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:26.106 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:26.106 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.106 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:26.106 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:26.106 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:26.106 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:26.106 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:26.106 INFO:tasks.workunit.client.0.smithi100.stdout: 5 2024-04-07T17:51:26.107 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.107 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 6, 2024-04-07T17:51:26.107 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:26.107 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.107 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:26.107 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:26.107 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:26.107 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:26.107 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T17:51:26.107 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.107 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 7, 2024-04-07T17:51:26.107 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:26.107 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:26.107 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:26.107 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:26.107 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:26.108 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:26.108 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.367504+0000", 2024-04-07T17:51:26.108 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:38.976388+0000", 2024-04-07T17:51:26.108 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:38.976388+0000", 2024-04-07T17:51:26.108 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:38.994496+0000", 2024-04-07T17:51:26.108 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.367504+0000", 2024-04-07T17:51:26.108 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.108 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:26.108 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.108 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:26.108 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.367504+0000", 2024-04-07T17:51:26.108 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.367504+0000", 2024-04-07T17:51:26.108 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.367504+0000", 2024-04-07T17:51:26.108 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:26.108 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:26.109 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.109 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.367504+0000", 2024-04-07T17:51:26.109 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.367504+0000", 2024-04-07T17:51:26.109 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:26.109 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:26.109 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:26.109 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:26.109 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:26.109 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:26.109 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:26.109 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:26.109 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:26.109 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:26.109 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:26.109 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:26.109 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:26.109 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.f", 2024-04-07T17:51:26.110 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:26.110 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:26.110 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:26.110 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4790, 2024-04-07T17:51:26.110 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:26.110 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T17:59:57.662112+0000", 2024-04-07T17:51:26.110 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:26.110 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:26.110 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:26.110 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:26.110 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:26.110 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:26.110 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:26.110 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:26.110 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:26.110 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:26.111 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:26.111 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:26.111 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:26.111 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:26.111 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:26.111 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:26.111 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:26.111 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:26.111 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:26.111 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:26.111 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:26.111 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:26.111 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:26.111 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:26.111 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:26.111 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:26.111 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:26.112 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:26.112 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:26.112 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:26.112 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:26.112 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:26.112 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:26.112 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:26.112 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:26.112 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:26.112 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:26.112 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:26.112 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:26.112 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:26.112 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:26.112 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:26.112 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:26.112 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.113 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:26.113 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:26.113 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:26.113 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:26.113 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:26.113 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T17:51:26.113 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.113 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 7, 2024-04-07T17:51:26.113 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:26.113 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.113 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:26.113 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:26.113 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T17:51:26.113 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:26.113 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T17:51:26.113 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.113 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 2, 2024-04-07T17:51:26.113 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:26.114 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:26.114 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T17:51:26.114 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:26.114 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:26.114 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:26.114 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.359582+0000", 2024-04-07T17:51:26.114 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:38.967914+0000", 2024-04-07T17:51:26.114 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:38.967914+0000", 2024-04-07T17:51:26.114 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:38.968390+0000", 2024-04-07T17:51:26.114 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.359582+0000", 2024-04-07T17:51:26.114 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:26.114 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:26.114 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:26.114 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:26.114 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.359582+0000", 2024-04-07T17:51:26.115 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.359582+0000", 2024-04-07T17:51:26.115 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.359582+0000", 2024-04-07T17:51:26.115 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:26.115 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:26.115 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:26.115 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.359582+0000", 2024-04-07T17:51:26.115 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.359582+0000", 2024-04-07T17:51:26.115 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:26.115 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 2, 2024-04-07T17:51:26.115 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:26.115 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:26.115 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:26.115 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:26.115 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:26.115 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:26.115 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:26.116 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 2, 2024-04-07T17:51:26.116 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:26.116 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:26.116 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:26.116 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.e", 2024-04-07T17:51:26.116 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:26.116 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:26.116 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:26.116 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4971, 2024-04-07T17:51:26.116 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:26.116 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T21:28:08.591091+0000", 2024-04-07T17:51:26.116 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:26.116 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:26.116 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:26.116 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:26.116 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:26.116 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:26.116 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:26.117 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:26.117 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:26.117 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:26.117 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:26.117 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:26.117 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:26.117 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:26.117 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:26.117 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:26.117 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:26.117 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:26.117 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:26.117 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 3, 2024-04-07T17:51:26.117 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 1, 2024-04-07T17:51:26.117 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:26.117 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 1, 2024-04-07T17:51:26.118 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:26.118 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:26.118 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:26.118 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:26.118 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:26.118 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 1, 2024-04-07T17:51:26.118 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:26.118 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:26.118 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:26.118 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:26.118 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:26.119 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:26.119 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:26.119 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 6, 2024-04-07T17:51:26.119 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 6, 2024-04-07T17:51:26.119 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:26.119 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:26.119 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:26.119 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 3, 2024-04-07T17:51:26.119 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:26.119 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.119 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:26.119 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:26.119 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:26.119 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T17:51:26.119 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:26.119 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T17:51:26.119 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.120 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 2, 2024-04-07T17:51:26.120 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "61'2" 2024-04-07T17:51:26.120 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.120 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:26.120 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:26.120 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:26.120 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:26.120 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T17:51:26.120 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.120 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 5, 2024-04-07T17:51:26.120 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:26.120 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:26.120 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:26.120 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:26.120 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:26.120 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:26.121 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.366524+0000", 2024-04-07T17:51:26.121 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:08.971755+0000", 2024-04-07T17:51:26.121 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:08.971755+0000", 2024-04-07T17:51:26.121 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:08.973166+0000", 2024-04-07T17:51:26.121 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.366524+0000", 2024-04-07T17:51:26.121 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.121 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:26.121 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.121 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 554, 2024-04-07T17:51:26.121 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.366524+0000", 2024-04-07T17:51:26.121 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.366524+0000", 2024-04-07T17:51:26.121 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.366524+0000", 2024-04-07T17:51:26.121 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:26.121 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:26.121 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.121 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.366524+0000", 2024-04-07T17:51:26.122 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.366524+0000", 2024-04-07T17:51:26.122 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:26.122 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:26.122 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:26.122 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:26.122 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 553, 2024-04-07T17:51:26.122 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:26.122 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:26.122 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:26.122 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:26.122 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:26.122 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:26.122 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:26.122 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:26.123 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.c", 2024-04-07T17:51:26.123 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:26.123 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:26.123 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:26.123 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4877, 2024-04-07T17:51:26.123 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:26.123 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T02:59:49.583756+0000", 2024-04-07T17:51:26.123 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:26.123 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:26.123 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:26.123 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:26.123 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:26.123 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:26.123 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:26.123 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:26.123 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:26.123 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:26.123 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:26.124 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:26.124 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:26.124 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:26.124 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:26.124 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:26.124 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:26.124 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:26.124 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:26.124 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:26.124 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:26.124 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:26.124 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:26.124 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:26.124 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:26.124 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:26.124 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:26.125 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:26.125 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:26.125 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:26.125 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:26.125 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:26.125 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:26.125 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:26.125 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:26.125 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:26.125 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:26.125 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:26.125 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:26.125 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:26.125 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:26.125 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:26.126 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:26.126 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.126 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:26.126 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:26.126 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:26.126 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:26.126 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:26.126 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T17:51:26.126 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.126 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 5, 2024-04-07T17:51:26.126 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:26.126 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.126 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:26.127 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:26.127 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:26.127 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:26.127 INFO:tasks.workunit.client.0.smithi100.stdout: 3 2024-04-07T17:51:26.127 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.127 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 1, 2024-04-07T17:51:26.127 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:26.127 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:26.127 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T17:51:26.127 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:26.127 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:26.127 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:26.127 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.333944+0000", 2024-04-07T17:51:26.127 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:24:13.932662+0000", 2024-04-07T17:51:26.127 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:24:13.932662+0000", 2024-04-07T17:51:26.128 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:24:13.933216+0000", 2024-04-07T17:51:26.128 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.333944+0000", 2024-04-07T17:51:26.128 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:26.128 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:26.128 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:26.128 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 500, 2024-04-07T17:51:26.128 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.333944+0000", 2024-04-07T17:51:26.128 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.333944+0000", 2024-04-07T17:51:26.128 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.333944+0000", 2024-04-07T17:51:26.128 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:26.128 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:26.128 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:26.128 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.333944+0000", 2024-04-07T17:51:26.128 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.333944+0000", 2024-04-07T17:51:26.128 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:26.128 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 1, 2024-04-07T17:51:26.129 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:26.129 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:26.129 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 499, 2024-04-07T17:51:26.129 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:26.129 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:26.129 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:26.129 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:26.129 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 1, 2024-04-07T17:51:26.129 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:26.129 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:26.129 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:26.129 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.d", 2024-04-07T17:51:26.129 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:26.129 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:26.129 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:26.130 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4983, 2024-04-07T17:51:26.130 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:26.130 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T17:17:11.134012+0000", 2024-04-07T17:51:26.130 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:26.130 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:26.130 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:26.130 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 70, 2024-04-07T17:51:26.130 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:26.130 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:26.130 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:26.130 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:26.130 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:26.130 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:26.130 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:26.131 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:26.131 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:26.131 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:26.131 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:26.131 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:26.131 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:26.131 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:26.131 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:26.131 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 3, 2024-04-07T17:51:26.131 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 1, 2024-04-07T17:51:26.131 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:26.131 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 1, 2024-04-07T17:51:26.131 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:26.131 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:26.131 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:26.131 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:26.132 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:26.132 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:26.132 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:26.132 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:26.132 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:26.132 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:26.132 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:26.132 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:26.132 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:26.132 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 1, 2024-04-07T17:51:26.132 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 1, 2024-04-07T17:51:26.132 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:26.132 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:26.132 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:26.132 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 1, 2024-04-07T17:51:26.132 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 1 2024-04-07T17:51:26.133 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.133 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:26.133 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:26.133 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:26.133 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:26.133 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:26.133 INFO:tasks.workunit.client.0.smithi100.stdout: 3 2024-04-07T17:51:26.133 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.133 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 1, 2024-04-07T17:51:26.133 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "61'1" 2024-04-07T17:51:26.133 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.133 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:26.133 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:26.133 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:26.133 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:26.133 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T17:51:26.134 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.134 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 7, 2024-04-07T17:51:26.134 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:26.134 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:26.134 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:26.134 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:26.134 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:26.134 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:26.134 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.373148+0000", 2024-04-07T17:51:26.134 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:38.989051+0000", 2024-04-07T17:51:26.134 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:38.989051+0000", 2024-04-07T17:51:26.134 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:39.000662+0000", 2024-04-07T17:51:26.134 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.373148+0000", 2024-04-07T17:51:26.134 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.134 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:26.135 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.135 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:26.135 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.373148+0000", 2024-04-07T17:51:26.135 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.373148+0000", 2024-04-07T17:51:26.135 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.373148+0000", 2024-04-07T17:51:26.135 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:26.135 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:26.135 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.135 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.373148+0000", 2024-04-07T17:51:26.135 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.373148+0000", 2024-04-07T17:51:26.135 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:26.135 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:26.135 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:26.135 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:26.135 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:26.136 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:26.136 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:26.136 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:26.136 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:26.136 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:26.136 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:26.136 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:26.136 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:26.136 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.d", 2024-04-07T17:51:26.136 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:26.136 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:26.136 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:26.136 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4803, 2024-04-07T17:51:26.136 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:26.136 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T23:45:21.166051+0000", 2024-04-07T17:51:26.136 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:26.137 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:26.137 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:26.137 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:26.137 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:26.137 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:26.137 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:26.137 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:26.137 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:26.137 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:26.137 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:26.137 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:26.137 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:26.137 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:26.137 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:26.137 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:26.137 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:26.137 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:26.138 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:26.138 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:26.138 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:26.138 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:26.138 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:26.138 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:26.138 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:26.138 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:26.138 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:26.138 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:26.138 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:26.138 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:26.138 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:26.138 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:26.139 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:26.139 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:26.139 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:26.139 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:26.139 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:26.139 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:26.139 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:26.139 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:26.139 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:26.139 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:26.139 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:26.139 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.139 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:26.139 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:26.139 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:26.139 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:26.139 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:26.140 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T17:51:26.140 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.140 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 7, 2024-04-07T17:51:26.140 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:26.140 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.140 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:26.140 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:26.140 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T17:51:26.140 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:26.140 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T17:51:26.140 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.140 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 2, 2024-04-07T17:51:26.140 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:26.140 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:26.140 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T17:51:26.140 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:26.140 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:26.141 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:26.141 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.355658+0000", 2024-04-07T17:51:26.141 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:24:41.549188+0000", 2024-04-07T17:51:26.141 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:24:41.549188+0000", 2024-04-07T17:51:26.141 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:24:41.550837+0000", 2024-04-07T17:51:26.141 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.355658+0000", 2024-04-07T17:51:26.141 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:26.141 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:26.141 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:26.141 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 527, 2024-04-07T17:51:26.141 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.355658+0000", 2024-04-07T17:51:26.141 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.355658+0000", 2024-04-07T17:51:26.141 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.355658+0000", 2024-04-07T17:51:26.141 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:26.141 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:26.141 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:26.141 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.355658+0000", 2024-04-07T17:51:26.142 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.355658+0000", 2024-04-07T17:51:26.142 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:26.142 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:26.142 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:26.142 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:26.142 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 526, 2024-04-07T17:51:26.142 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:26.142 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:26.142 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:26.142 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:26.142 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:26.142 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:26.142 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:26.142 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:26.143 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.c", 2024-04-07T17:51:26.143 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:26.143 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:26.143 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:26.143 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4973, 2024-04-07T17:51:26.143 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:26.143 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T18:56:44.543940+0000", 2024-04-07T17:51:26.143 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:26.143 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:26.143 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:26.143 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:26.143 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:26.143 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:26.143 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:26.143 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:26.144 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:26.144 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:26.144 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:26.144 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:26.144 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:26.144 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:26.144 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:26.144 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:26.144 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:26.144 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:26.144 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:26.144 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:26.144 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:26.144 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:26.144 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:26.144 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:26.144 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:26.144 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:26.145 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:26.145 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:26.145 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:26.145 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:26.145 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:26.145 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:26.145 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:26.145 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:26.145 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:26.145 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:26.145 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:26.145 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:26.145 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:26.145 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:26.145 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:26.145 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:26.146 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:26.146 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.146 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:26.146 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:26.146 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:26.146 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T17:51:26.146 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:26.146 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T17:51:26.146 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.146 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 2, 2024-04-07T17:51:26.146 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:26.146 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.146 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:26.146 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:26.146 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:26.147 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:26.147 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T17:51:26.147 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.147 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 6, 2024-04-07T17:51:26.147 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:26.147 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:26.147 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:26.147 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:26.147 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:26.147 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:26.147 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.386750+0000", 2024-04-07T17:51:26.147 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:08.966124+0000", 2024-04-07T17:51:26.147 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:08.966124+0000", 2024-04-07T17:51:26.147 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:08.966628+0000", 2024-04-07T17:51:26.147 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.386750+0000", 2024-04-07T17:51:26.147 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.148 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:26.148 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.148 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 554, 2024-04-07T17:51:26.148 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.386750+0000", 2024-04-07T17:51:26.148 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.386750+0000", 2024-04-07T17:51:26.148 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.386750+0000", 2024-04-07T17:51:26.148 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:26.148 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:26.148 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.148 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.386750+0000", 2024-04-07T17:51:26.148 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.386750+0000", 2024-04-07T17:51:26.148 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:26.148 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:26.148 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:26.148 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:26.149 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 553, 2024-04-07T17:51:26.149 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:26.149 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:26.149 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:26.149 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:26.149 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:26.149 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:26.149 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:26.149 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:26.149 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.a", 2024-04-07T17:51:26.149 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:26.149 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:26.149 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:26.149 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4847, 2024-04-07T17:51:26.149 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:26.149 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T19:57:28.644470+0000", 2024-04-07T17:51:26.149 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:26.150 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:26.150 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:26.150 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:26.150 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:26.150 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:26.150 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:26.150 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:26.150 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:26.150 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:26.150 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:26.150 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:26.150 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:26.150 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:26.150 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:26.150 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:26.151 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:26.151 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:26.151 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:26.151 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:26.151 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:26.151 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:26.151 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:26.151 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:26.151 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:26.151 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:26.151 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:26.151 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:26.151 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:26.151 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:26.151 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:26.151 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:26.151 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:26.152 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:26.152 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:26.152 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:26.152 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:26.152 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:26.152 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:26.152 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:26.152 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:26.152 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:26.152 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:26.152 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.152 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:26.152 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:26.152 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:26.152 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:26.153 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:26.153 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T17:51:26.153 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.153 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 6, 2024-04-07T17:51:26.153 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:26.153 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.153 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:26.153 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:26.153 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:26.153 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:26.153 INFO:tasks.workunit.client.0.smithi100.stdout: 5 2024-04-07T17:51:26.153 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.153 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 7, 2024-04-07T17:51:26.153 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:26.153 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:26.153 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T17:51:26.154 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:26.154 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:26.154 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:26.154 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.369146+0000", 2024-04-07T17:51:26.154 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:38.990309+0000", 2024-04-07T17:51:26.154 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:38.990309+0000", 2024-04-07T17:51:26.154 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:38.992642+0000", 2024-04-07T17:51:26.154 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.369146+0000", 2024-04-07T17:51:26.154 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:26.154 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:26.154 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:26.154 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:26.154 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.369146+0000", 2024-04-07T17:51:26.154 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.369146+0000", 2024-04-07T17:51:26.154 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.369146+0000", 2024-04-07T17:51:26.155 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:26.155 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:26.155 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:26.155 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.369146+0000", 2024-04-07T17:51:26.155 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.369146+0000", 2024-04-07T17:51:26.155 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:26.155 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 3, 2024-04-07T17:51:26.155 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:26.155 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:26.155 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:26.155 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:26.155 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:26.155 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:26.155 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:26.155 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 3, 2024-04-07T17:51:26.155 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:26.155 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:26.156 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:26.156 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.b", 2024-04-07T17:51:26.156 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:26.156 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:26.156 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:26.156 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4823, 2024-04-07T17:51:26.156 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:26.156 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T17:09:27.535561+0000", 2024-04-07T17:51:26.156 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:26.156 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:26.156 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:26.156 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:26.156 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:26.156 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:26.156 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:26.156 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:26.157 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:26.157 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:26.157 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:26.157 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:26.157 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:26.157 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:26.157 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:26.157 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:26.157 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:26.157 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:26.157 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:26.157 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 3, 2024-04-07T17:51:26.157 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 1, 2024-04-07T17:51:26.157 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:26.157 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 1, 2024-04-07T17:51:26.157 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:26.157 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:26.158 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:26.158 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:26.158 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:26.158 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 1, 2024-04-07T17:51:26.158 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:26.158 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:26.158 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:26.158 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:26.158 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:26.158 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:26.158 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:26.158 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 2, 2024-04-07T17:51:26.158 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 2, 2024-04-07T17:51:26.158 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:26.159 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:26.159 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:26.159 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 1, 2024-04-07T17:51:26.159 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:26.159 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.159 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:26.159 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:26.159 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:26.159 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:26.159 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:26.159 INFO:tasks.workunit.client.0.smithi100.stdout: 5 2024-04-07T17:51:26.159 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.159 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 7, 2024-04-07T17:51:26.159 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "623'3" 2024-04-07T17:51:26.159 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.159 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:26.160 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:26.160 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:26.160 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:26.160 INFO:tasks.workunit.client.0.smithi100.stdout: 4 2024-04-07T17:51:26.160 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.160 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 3, 2024-04-07T17:51:26.160 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:26.160 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:26.160 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:26.160 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:26.160 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:26.160 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:26.160 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:35.976901+0000", 2024-04-07T17:51:26.160 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:24:13.933754+0000", 2024-04-07T17:51:26.160 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:24:13.933754+0000", 2024-04-07T17:51:26.160 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:24:13.934742+0000", 2024-04-07T17:51:26.161 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:35.976901+0000", 2024-04-07T17:51:26.161 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.161 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:26.161 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.161 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 500, 2024-04-07T17:51:26.161 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:35.976901+0000", 2024-04-07T17:51:26.161 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:35.976901+0000", 2024-04-07T17:51:26.161 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:35.976901+0000", 2024-04-07T17:51:26.161 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:26.161 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:26.161 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.161 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:35.976901+0000", 2024-04-07T17:51:26.161 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:35.976901+0000", 2024-04-07T17:51:26.161 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:26.161 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:26.161 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:26.162 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:26.162 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 499, 2024-04-07T17:51:26.162 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:26.162 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:26.162 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:26.162 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:26.162 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:26.162 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:26.162 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:26.162 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:26.162 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.b", 2024-04-07T17:51:26.162 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:26.162 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:26.162 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:26.163 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4938, 2024-04-07T17:51:26.163 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:26.163 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T02:43:13.536050+0000", 2024-04-07T17:51:26.163 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:26.163 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:26.163 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:26.163 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:26.163 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:26.163 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:26.163 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:26.163 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:26.163 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:26.163 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:26.163 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:26.163 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:26.164 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:26.164 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:26.164 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:26.164 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:26.164 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:26.164 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:26.164 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:26.164 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:26.164 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:26.164 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:26.164 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:26.164 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:26.164 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:26.164 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:26.164 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:26.164 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:26.164 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:26.164 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:26.165 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:26.165 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:26.165 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:26.165 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:26.165 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:26.165 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:26.165 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:26.165 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:26.165 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:26.165 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:26.165 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:26.165 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:26.165 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:26.165 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.165 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:26.165 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:26.166 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:26.166 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:26.166 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:26.166 INFO:tasks.workunit.client.0.smithi100.stdout: 4 2024-04-07T17:51:26.166 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.166 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 3, 2024-04-07T17:51:26.166 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:26.166 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.166 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:26.166 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:26.166 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:26.166 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:26.166 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T17:51:26.166 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.166 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 1, 2024-04-07T17:51:26.167 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:26.167 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:26.167 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T17:51:26.167 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:26.167 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:26.167 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:26.167 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.335142+0000", 2024-04-07T17:51:26.167 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:38.975734+0000", 2024-04-07T17:51:26.167 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:38.975734+0000", 2024-04-07T17:51:26.167 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:38.976248+0000", 2024-04-07T17:51:26.167 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.335142+0000", 2024-04-07T17:51:26.167 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:26.167 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:26.167 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:26.167 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:26.168 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.335142+0000", 2024-04-07T17:51:26.168 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.335142+0000", 2024-04-07T17:51:26.168 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.335142+0000", 2024-04-07T17:51:26.168 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:26.168 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:26.168 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:26.168 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.335142+0000", 2024-04-07T17:51:26.168 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.335142+0000", 2024-04-07T17:51:26.168 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:26.168 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:26.168 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:26.168 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:26.168 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:26.168 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:26.168 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:26.168 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:26.169 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:26.169 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:26.169 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:26.169 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:26.169 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:26.169 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.a", 2024-04-07T17:51:26.169 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:26.169 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:26.169 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:26.169 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4981, 2024-04-07T17:51:26.169 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:26.169 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T19:45:35.140357+0000", 2024-04-07T17:51:26.169 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:26.169 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:26.169 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:26.170 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:26.170 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:26.170 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:26.170 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:26.170 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:26.170 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:26.170 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:26.170 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:26.170 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:26.170 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:26.170 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:26.170 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:26.170 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:26.170 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:26.171 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:26.171 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:26.171 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:26.171 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:26.171 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:26.171 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:26.171 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:26.171 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:26.171 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:26.171 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:26.171 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:26.171 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:26.171 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:26.171 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:26.171 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:26.171 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:26.172 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:26.172 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:26.172 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:26.172 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:26.172 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:26.172 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:26.172 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:26.172 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:26.172 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:26.172 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:26.172 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.172 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:26.172 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:26.172 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:26.172 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:26.172 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:26.172 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T17:51:26.173 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.173 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 1, 2024-04-07T17:51:26.173 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:26.173 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.173 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:26.173 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:26.173 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:26.173 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:26.173 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T17:51:26.173 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.173 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 3, 2024-04-07T17:51:26.173 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:26.173 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:26.173 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:26.173 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:26.173 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:26.174 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:26.174 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:35.975502+0000", 2024-04-07T17:51:26.174 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:38.997582+0000", 2024-04-07T17:51:26.174 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:38.997582+0000", 2024-04-07T17:51:26.174 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:38.998502+0000", 2024-04-07T17:51:26.174 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:35.975502+0000", 2024-04-07T17:51:26.174 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.174 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:26.174 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.174 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:26.174 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:35.975502+0000", 2024-04-07T17:51:26.174 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:35.975502+0000", 2024-04-07T17:51:26.174 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:35.975502+0000", 2024-04-07T17:51:26.174 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:26.174 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:26.175 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.175 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:35.975502+0000", 2024-04-07T17:51:26.175 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:35.975502+0000", 2024-04-07T17:51:26.175 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:26.175 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:26.175 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:26.175 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:26.175 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:26.175 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:26.175 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:26.175 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:26.175 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:26.175 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:26.175 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:26.175 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:26.176 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:26.176 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.8", 2024-04-07T17:51:26.176 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:26.176 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:26.176 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:26.176 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4968, 2024-04-07T17:51:26.176 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:26.176 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T23:09:31.581874+0000", 2024-04-07T17:51:26.176 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:26.176 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:26.176 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:26.176 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:26.176 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:26.176 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:26.176 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:26.176 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:26.177 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:26.177 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:26.177 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:26.177 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:26.177 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:26.177 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:26.177 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:26.177 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:26.177 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:26.177 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:26.177 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:26.177 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:26.177 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:26.177 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:26.177 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:26.177 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:26.177 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:26.178 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:26.178 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:26.178 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:26.178 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:26.178 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:26.178 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:26.178 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:26.178 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:26.178 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:26.178 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:26.178 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:26.178 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:26.178 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:26.178 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:26.178 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:26.178 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:26.179 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:26.179 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:26.179 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.179 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:26.179 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:26.179 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:26.179 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:26.179 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:26.179 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T17:51:26.179 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.179 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 3, 2024-04-07T17:51:26.179 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:26.179 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.179 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:26.179 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:26.180 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:26.180 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:26.180 INFO:tasks.workunit.client.0.smithi100.stdout: 3 2024-04-07T17:51:26.180 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.180 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 1, 2024-04-07T17:51:26.180 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:26.180 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:26.180 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T17:51:26.180 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:26.180 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:26.180 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:26.180 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.331976+0000", 2024-04-07T17:51:26.180 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:38.992340+0000", 2024-04-07T17:51:26.180 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:38.992340+0000", 2024-04-07T17:51:26.180 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:38.992779+0000", 2024-04-07T17:51:26.181 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.331976+0000", 2024-04-07T17:51:26.181 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:26.181 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:26.181 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:26.181 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:26.181 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.331976+0000", 2024-04-07T17:51:26.181 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.331976+0000", 2024-04-07T17:51:26.181 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.331976+0000", 2024-04-07T17:51:26.181 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:26.181 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:26.181 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:26.181 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.331976+0000", 2024-04-07T17:51:26.181 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.331976+0000", 2024-04-07T17:51:26.181 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:26.181 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:26.182 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:26.182 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:26.182 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:26.182 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:26.182 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:26.182 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:26.182 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:26.182 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:26.182 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:26.182 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:26.182 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:26.182 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.9", 2024-04-07T17:51:26.182 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:26.182 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:26.182 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:26.182 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4981, 2024-04-07T17:51:26.183 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:26.183 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T21:25:43.908090+0000", 2024-04-07T17:51:26.183 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:26.183 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:26.183 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:26.183 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:26.183 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:26.183 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:26.183 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:26.183 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:26.183 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:26.183 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:26.183 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:26.183 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:26.183 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:26.183 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:26.183 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:26.184 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:26.184 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:26.184 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:26.184 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:26.184 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:26.184 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:26.184 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:26.184 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:26.184 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:26.184 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:26.184 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:26.184 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:26.184 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:26.184 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:26.184 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:26.185 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:26.185 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:26.185 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:26.185 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:26.185 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:26.185 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:26.185 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:26.185 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:26.185 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:26.185 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:26.185 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:26.185 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:26.185 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:26.185 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.185 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:26.185 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:26.186 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:26.186 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:26.186 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:26.186 INFO:tasks.workunit.client.0.smithi100.stdout: 3 2024-04-07T17:51:26.186 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.186 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 1, 2024-04-07T17:51:26.186 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:26.186 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.186 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:26.186 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:26.186 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:26.186 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T17:51:26.186 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T17:51:26.186 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.187 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 4, 2024-04-07T17:51:26.187 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:26.187 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:26.187 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:26.187 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:26.187 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:26.187 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:26.187 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:35.977778+0000", 2024-04-07T17:51:26.187 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:38.985206+0000", 2024-04-07T17:51:26.187 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:38.985206+0000", 2024-04-07T17:51:26.187 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:38.985745+0000", 2024-04-07T17:51:26.187 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:35.977778+0000", 2024-04-07T17:51:26.187 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.187 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:26.187 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.187 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:26.187 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:35.977778+0000", 2024-04-07T17:51:26.188 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:35.977778+0000", 2024-04-07T17:51:26.188 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:35.977778+0000", 2024-04-07T17:51:26.188 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:26.188 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:26.188 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.188 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:35.977778+0000", 2024-04-07T17:51:26.188 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:35.977778+0000", 2024-04-07T17:51:26.188 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:26.188 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:26.188 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:26.188 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:26.188 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:26.188 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:26.188 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:26.188 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:26.188 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:26.189 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:26.189 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:26.189 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:26.189 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:26.189 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.9", 2024-04-07T17:51:26.189 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:26.189 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:26.189 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:26.189 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4931, 2024-04-07T17:51:26.189 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:26.189 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T00:51:54.981164+0000", 2024-04-07T17:51:26.189 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:26.189 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:26.189 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:26.189 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:26.189 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:26.189 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:26.190 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:26.190 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:26.190 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:26.190 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:26.190 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:26.190 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:26.190 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:26.190 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:26.190 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:26.190 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:26.190 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:26.190 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:26.190 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:26.190 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:26.190 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:26.190 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:26.191 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:26.191 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:26.191 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:26.191 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:26.191 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:26.191 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:26.191 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:26.191 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:26.191 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:26.191 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:26.191 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:26.191 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:26.191 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:26.191 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:26.191 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:26.192 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:26.192 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:26.192 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:26.192 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:26.192 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:26.192 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:26.192 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.192 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:26.192 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:26.192 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:26.192 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:26.192 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T17:51:26.192 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T17:51:26.192 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.192 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 4, 2024-04-07T17:51:26.192 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:26.192 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.193 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:26.193 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:26.193 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:26.193 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:26.193 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T17:51:26.193 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.193 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 7, 2024-04-07T17:51:26.193 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:26.193 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:26.193 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T17:51:26.193 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:26.193 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:26.193 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:26.193 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.364254+0000", 2024-04-07T17:51:26.193 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:38.996008+0000", 2024-04-07T17:51:26.193 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:38.996008+0000", 2024-04-07T17:51:26.193 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:39.001597+0000", 2024-04-07T17:51:26.194 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.364254+0000", 2024-04-07T17:51:26.194 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:26.194 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:26.194 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:26.194 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:26.194 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.364254+0000", 2024-04-07T17:51:26.194 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.364254+0000", 2024-04-07T17:51:26.194 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.364254+0000", 2024-04-07T17:51:26.194 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:26.194 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:26.194 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:26.194 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.364254+0000", 2024-04-07T17:51:26.194 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.364254+0000", 2024-04-07T17:51:26.195 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:26.195 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:26.195 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:26.195 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:26.195 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:26.195 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:26.195 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:26.195 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:26.195 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:26.195 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:26.195 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:26.195 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:26.195 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:26.195 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.8", 2024-04-07T17:51:26.195 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:26.196 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:26.196 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:26.196 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4792, 2024-04-07T17:51:26.196 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:26.196 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T19:53:15.918325+0000", 2024-04-07T17:51:26.196 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:26.196 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:26.196 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:26.196 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:26.196 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:26.196 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:26.196 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:26.196 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:26.196 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:26.196 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:26.196 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:26.197 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:26.197 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:26.197 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:26.197 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:26.197 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:26.197 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:26.197 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:26.197 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:26.197 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:26.197 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:26.197 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:26.197 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:26.197 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:26.197 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:26.197 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:26.197 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:26.197 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:26.198 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:26.198 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:26.198 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:26.198 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:26.198 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:26.198 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:26.198 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:26.198 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:26.198 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:26.198 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:26.198 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:26.198 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:26.198 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:26.198 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:26.198 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:26.199 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.199 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:26.199 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:26.199 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:26.199 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:26.199 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:26.199 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T17:51:26.199 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.199 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 7, 2024-04-07T17:51:26.199 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:26.199 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.199 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:26.199 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:26.199 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:26.199 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:26.199 INFO:tasks.workunit.client.0.smithi100.stdout: 4 2024-04-07T17:51:26.199 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.200 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 0, 2024-04-07T17:51:26.200 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:26.200 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:26.200 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:26.200 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:26.200 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:26.200 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:26.200 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:35.981688+0000", 2024-04-07T17:51:26.200 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:24:13.962953+0000", 2024-04-07T17:51:26.200 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:24:13.962953+0000", 2024-04-07T17:51:26.200 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:24:13.963599+0000", 2024-04-07T17:51:26.200 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:35.981688+0000", 2024-04-07T17:51:26.200 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.200 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:26.200 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.200 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 500, 2024-04-07T17:51:26.200 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:35.981688+0000", 2024-04-07T17:51:26.201 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:35.981688+0000", 2024-04-07T17:51:26.201 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:35.981688+0000", 2024-04-07T17:51:26.201 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:26.201 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:26.201 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.201 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:35.981688+0000", 2024-04-07T17:51:26.201 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:35.981688+0000", 2024-04-07T17:51:26.201 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:26.201 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:26.201 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:26.201 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:26.201 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 499, 2024-04-07T17:51:26.201 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:26.201 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:26.201 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:26.201 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:26.202 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:26.202 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:26.202 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:26.202 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:26.202 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.6", 2024-04-07T17:51:26.202 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:26.202 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:26.202 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:26.202 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4999, 2024-04-07T17:51:26.202 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:26.202 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T22:29:41.083754+0000", 2024-04-07T17:51:26.202 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:26.202 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:26.202 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:26.203 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:26.203 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:26.203 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:26.203 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:26.203 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:26.203 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:26.203 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:26.203 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:26.203 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:26.203 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:26.203 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:26.203 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:26.203 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:26.203 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:26.203 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:26.203 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:26.203 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:26.204 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:26.204 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:26.204 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:26.204 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:26.204 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:26.204 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:26.204 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:26.204 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:26.204 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:26.204 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:26.204 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:26.204 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:26.204 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:26.204 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:26.204 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:26.204 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:26.204 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:26.205 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:26.205 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:26.205 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:26.205 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:26.205 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:26.205 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:26.205 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.205 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:26.205 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:26.205 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:26.205 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:26.205 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:26.205 INFO:tasks.workunit.client.0.smithi100.stdout: 4 2024-04-07T17:51:26.205 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.205 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 0, 2024-04-07T17:51:26.205 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:26.206 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.206 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:26.206 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:26.206 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:26.206 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:26.206 INFO:tasks.workunit.client.0.smithi100.stdout: 2 2024-04-07T17:51:26.206 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.206 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 6, 2024-04-07T17:51:26.206 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:26.206 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:26.206 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T17:51:26.206 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:26.206 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:26.206 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:26.207 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.369806+0000", 2024-04-07T17:51:26.207 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:08.974754+0000", 2024-04-07T17:51:26.207 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:08.974754+0000", 2024-04-07T17:51:26.207 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:08.975487+0000", 2024-04-07T17:51:26.207 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.369806+0000", 2024-04-07T17:51:26.207 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:26.207 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:26.207 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:26.207 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 554, 2024-04-07T17:51:26.207 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.369806+0000", 2024-04-07T17:51:26.207 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.369806+0000", 2024-04-07T17:51:26.207 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.369806+0000", 2024-04-07T17:51:26.207 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:26.207 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:26.207 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:26.207 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.369806+0000", 2024-04-07T17:51:26.207 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.369806+0000", 2024-04-07T17:51:26.208 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:26.208 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 2, 2024-04-07T17:51:26.208 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:26.208 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:26.208 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 553, 2024-04-07T17:51:26.208 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:26.208 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:26.208 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:26.208 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:26.208 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 2, 2024-04-07T17:51:26.208 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:26.208 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:26.208 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:26.208 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.7", 2024-04-07T17:51:26.208 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:26.208 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:26.209 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:26.209 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4865, 2024-04-07T17:51:26.209 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:26.209 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T01:37:46.529471+0000", 2024-04-07T17:51:26.209 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:26.209 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:26.209 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:26.209 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:26.209 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:26.209 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:26.209 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:26.209 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:26.209 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:26.209 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:26.209 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:26.209 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:26.209 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:26.210 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:26.210 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:26.210 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:26.210 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:26.210 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:26.210 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:26.210 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 3, 2024-04-07T17:51:26.210 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 1, 2024-04-07T17:51:26.210 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:26.210 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 1, 2024-04-07T17:51:26.210 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:26.210 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:26.210 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:26.210 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:26.210 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:26.211 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 1, 2024-04-07T17:51:26.211 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:26.211 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:26.211 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:26.211 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:26.211 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:26.211 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:26.211 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:26.211 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 6, 2024-04-07T17:51:26.211 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 6, 2024-04-07T17:51:26.211 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:26.211 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:26.211 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:26.211 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 3, 2024-04-07T17:51:26.211 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:26.211 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.211 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:26.212 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:26.212 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:26.212 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:26.212 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:26.212 INFO:tasks.workunit.client.0.smithi100.stdout: 2 2024-04-07T17:51:26.212 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.212 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 6, 2024-04-07T17:51:26.212 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "61'2" 2024-04-07T17:51:26.212 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.212 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:26.212 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:26.212 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:26.212 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:26.212 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T17:51:26.212 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.212 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 4, 2024-04-07T17:51:26.212 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:26.213 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:26.213 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 66, 2024-04-07T17:51:26.213 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:26.213 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:26.213 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:26.213 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:35.977631+0000", 2024-04-07T17:51:26.213 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:08.952433+0000", 2024-04-07T17:51:26.213 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:08.952433+0000", 2024-04-07T17:51:26.213 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:59:56.534054+0000", 2024-04-07T17:51:26.213 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:35.977631+0000", 2024-04-07T17:51:26.213 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:13:28.963534+0000", 2024-04-07T17:51:26.213 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:26.213 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:13:28.963534+0000", 2024-04-07T17:51:26.213 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 554, 2024-04-07T17:51:26.213 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:35.977631+0000", 2024-04-07T17:51:26.213 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:35.977631+0000", 2024-04-07T17:51:26.213 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:35.977631+0000", 2024-04-07T17:51:26.214 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:26.214 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:26.214 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:13:28.963534+0000", 2024-04-07T17:51:26.214 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:35.977631+0000", 2024-04-07T17:51:26.214 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:35.977631+0000", 2024-04-07T17:51:26.214 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:26.214 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 3036, 2024-04-07T17:51:26.214 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:26.214 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:26.214 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 553, 2024-04-07T17:51:26.214 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:26.214 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:26.214 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:26.214 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:26.214 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 3036, 2024-04-07T17:51:26.215 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:26.215 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:26.215 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:26.215 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "4.1", 2024-04-07T17:51:26.215 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:26.215 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:26.215 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:26.215 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 8679, 2024-04-07T17:51:26.215 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:26.215 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T00:59:44.742269+0000", 2024-04-07T17:51:26.215 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.007050569, 2024-04-07T17:51:26.215 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:26.215 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:26.215 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 4194304, 2024-04-07T17:51:26.215 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:26.215 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:26.216 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:26.216 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:26.216 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:26.216 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:26.216 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:26.216 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:26.216 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:26.216 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:26.216 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:26.216 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:26.216 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:26.216 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:26.216 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:26.216 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 3, 2024-04-07T17:51:26.216 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 1, 2024-04-07T17:51:26.216 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:26.217 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 1, 2024-04-07T17:51:26.217 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:26.217 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:26.217 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:26.217 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:26.217 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:26.217 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:26.217 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:26.217 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:26.217 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:26.217 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:26.217 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:26.217 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:26.217 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:26.217 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 1087, 2024-04-07T17:51:26.217 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 402424, 2024-04-07T17:51:26.217 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:26.218 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:26.218 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:26.218 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 3017, 2024-04-07T17:51:26.218 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 407157 2024-04-07T17:51:26.218 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.218 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:26.218 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:26.218 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:26.218 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:26.218 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:26.218 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T17:51:26.218 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.218 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 4, 2024-04-07T17:51:26.218 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "1083'3036" 2024-04-07T17:51:26.218 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.218 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:26.219 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:26.219 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:26.219 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:26.219 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T17:51:26.219 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.219 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 3, 2024-04-07T17:51:26.219 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:26.219 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:26.219 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:26.219 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:26.219 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:26.219 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:26.219 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:35.981722+0000", 2024-04-07T17:51:26.219 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:38.974388+0000", 2024-04-07T17:51:26.219 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:38.974388+0000", 2024-04-07T17:51:26.219 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:38.974763+0000", 2024-04-07T17:51:26.220 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:35.981722+0000", 2024-04-07T17:51:26.220 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.220 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:26.220 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.220 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:26.220 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:35.981722+0000", 2024-04-07T17:51:26.220 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:35.981722+0000", 2024-04-07T17:51:26.220 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:35.981722+0000", 2024-04-07T17:51:26.220 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:26.220 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:26.220 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.220 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:35.981722+0000", 2024-04-07T17:51:26.220 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:35.981722+0000", 2024-04-07T17:51:26.220 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:26.220 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:26.220 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:26.220 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:26.221 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:26.221 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:26.221 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:26.221 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:26.221 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:26.221 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:26.221 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:26.221 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:26.221 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:26.221 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.7", 2024-04-07T17:51:26.221 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:26.221 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:26.221 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:26.221 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4940, 2024-04-07T17:51:26.221 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:26.221 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T03:43:59.412072+0000", 2024-04-07T17:51:26.221 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:26.222 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:26.222 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:26.222 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:26.222 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:26.222 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:26.222 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:26.222 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:26.222 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:26.222 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:26.222 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:26.222 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:26.222 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:26.222 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:26.222 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:26.223 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:26.223 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:26.223 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:26.223 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:26.223 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:26.223 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:26.223 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:26.223 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:26.223 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:26.223 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:26.223 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:26.223 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:26.223 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:26.223 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:26.223 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:26.223 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:26.224 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:26.224 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:26.224 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:26.224 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:26.224 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:26.224 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:26.224 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:26.224 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:26.224 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:26.224 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:26.224 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:26.224 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:26.224 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.224 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:26.224 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:26.224 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:26.225 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:26.225 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:26.225 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T17:51:26.225 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.225 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 3, 2024-04-07T17:51:26.225 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:26.225 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.225 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:26.225 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:26.225 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:26.225 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:26.225 INFO:tasks.workunit.client.0.smithi100.stdout: 4 2024-04-07T17:51:26.225 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.225 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 1, 2024-04-07T17:51:26.226 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:26.226 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:26.226 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T17:51:26.226 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:26.226 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:26.226 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:26.226 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.335258+0000", 2024-04-07T17:51:26.226 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:08.962773+0000", 2024-04-07T17:51:26.226 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:08.962773+0000", 2024-04-07T17:51:26.226 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:08.963320+0000", 2024-04-07T17:51:26.226 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.335258+0000", 2024-04-07T17:51:26.226 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:26.226 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:26.226 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:26.227 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 554, 2024-04-07T17:51:26.227 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.335258+0000", 2024-04-07T17:51:26.227 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.335258+0000", 2024-04-07T17:51:26.227 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.335258+0000", 2024-04-07T17:51:26.227 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:26.227 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:26.227 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:26.227 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.335258+0000", 2024-04-07T17:51:26.227 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.335258+0000", 2024-04-07T17:51:26.227 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:26.227 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 2, 2024-04-07T17:51:26.227 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:26.227 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:26.227 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 553, 2024-04-07T17:51:26.228 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:26.228 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:26.228 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:26.228 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:26.228 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 2, 2024-04-07T17:51:26.228 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:26.228 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:26.228 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:26.228 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.6", 2024-04-07T17:51:26.228 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:26.228 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:26.228 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:26.228 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4986, 2024-04-07T17:51:26.228 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:26.228 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T18:43:13.545482+0000", 2024-04-07T17:51:26.228 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:26.229 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:26.229 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:26.229 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:26.229 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:26.229 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:26.229 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:26.229 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:26.229 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:26.229 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:26.229 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:26.229 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:26.229 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:26.229 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:26.229 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:26.229 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:26.229 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:26.229 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:26.230 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:26.230 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 3, 2024-04-07T17:51:26.230 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 1, 2024-04-07T17:51:26.230 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:26.230 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 1, 2024-04-07T17:51:26.230 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:26.230 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:26.230 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:26.230 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:26.230 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:26.230 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 1, 2024-04-07T17:51:26.230 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:26.230 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:26.230 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:26.231 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:26.231 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:26.231 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:26.231 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:26.231 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 2, 2024-04-07T17:51:26.231 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 2, 2024-04-07T17:51:26.231 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:26.231 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:26.231 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:26.231 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 2, 2024-04-07T17:51:26.231 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:26.231 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.231 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:26.231 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:26.231 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:26.232 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:26.232 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:26.232 INFO:tasks.workunit.client.0.smithi100.stdout: 4 2024-04-07T17:51:26.232 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.232 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 1, 2024-04-07T17:51:26.232 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "627'2" 2024-04-07T17:51:26.232 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.232 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:26.232 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:26.232 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:26.232 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:26.232 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T17:51:26.232 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.232 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 3, 2024-04-07T17:51:26.232 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:26.232 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:26.233 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 66, 2024-04-07T17:51:26.233 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:26.233 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:26.233 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:26.233 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:35.975295+0000", 2024-04-07T17:51:26.233 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:38.997956+0000", 2024-04-07T17:51:26.233 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:38.997956+0000", 2024-04-07T17:51:26.233 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:59:56.537779+0000", 2024-04-07T17:51:26.233 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:35.975295+0000", 2024-04-07T17:51:26.233 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:13:28.963534+0000", 2024-04-07T17:51:26.233 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:26.233 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:13:28.963534+0000", 2024-04-07T17:51:26.233 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:26.233 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:35.975295+0000", 2024-04-07T17:51:26.233 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:35.975295+0000", 2024-04-07T17:51:26.234 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:35.975295+0000", 2024-04-07T17:51:26.234 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:26.234 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:26.234 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:13:28.963534+0000", 2024-04-07T17:51:26.234 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:35.975295+0000", 2024-04-07T17:51:26.234 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:35.975295+0000", 2024-04-07T17:51:26.234 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:26.234 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 3089, 2024-04-07T17:51:26.234 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:26.234 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:26.234 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:26.234 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:26.234 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:26.234 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:26.234 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:26.235 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 3089, 2024-04-07T17:51:26.235 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:26.235 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:26.235 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:26.235 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "4.0", 2024-04-07T17:51:26.235 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:26.235 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:26.235 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:26.235 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 8748, 2024-04-07T17:51:26.235 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:26.235 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T16:44:21.465234+0000", 2024-04-07T17:51:26.235 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.013008564, 2024-04-07T17:51:26.235 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:26.235 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:26.235 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:26.235 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:26.236 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:26.236 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:26.236 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:26.236 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:26.236 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:26.236 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:26.236 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:26.236 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:26.236 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:26.236 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:26.236 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:26.236 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:26.236 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:26.236 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:26.236 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:26.237 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:26.237 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:26.237 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:26.237 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:26.237 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:26.237 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:26.237 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:26.237 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:26.237 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:26.237 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:26.237 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:26.237 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:26.237 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:26.237 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:26.237 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:26.237 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:26.237 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 1133, 2024-04-07T17:51:26.238 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 401354, 2024-04-07T17:51:26.238 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:26.238 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:26.238 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:26.238 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 3064, 2024-04-07T17:51:26.238 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 366365 2024-04-07T17:51:26.238 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.238 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:26.238 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:26.238 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:26.238 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:26.238 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:26.238 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T17:51:26.238 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.238 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 3, 2024-04-07T17:51:26.238 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "1083'3089" 2024-04-07T17:51:26.239 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.239 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:26.239 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:26.239 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:26.239 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T17:51:26.239 INFO:tasks.workunit.client.0.smithi100.stdout: 5 2024-04-07T17:51:26.239 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.239 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 1, 2024-04-07T17:51:26.239 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:26.239 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:26.239 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:26.239 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:26.239 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:26.239 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:26.239 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.333938+0000", 2024-04-07T17:51:26.240 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:24:41.552588+0000", 2024-04-07T17:51:26.240 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:24:41.552588+0000", 2024-04-07T17:51:26.240 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:24:41.553129+0000", 2024-04-07T17:51:26.240 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.333938+0000", 2024-04-07T17:51:26.240 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.240 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:26.240 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.240 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 527, 2024-04-07T17:51:26.240 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.333938+0000", 2024-04-07T17:51:26.240 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.333938+0000", 2024-04-07T17:51:26.240 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.333938+0000", 2024-04-07T17:51:26.240 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:26.240 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:26.240 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.240 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.333938+0000", 2024-04-07T17:51:26.240 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.333938+0000", 2024-04-07T17:51:26.240 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:26.241 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:26.241 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:26.241 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:26.241 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 526, 2024-04-07T17:51:26.241 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:26.241 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:26.241 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:26.241 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:26.241 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:26.241 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:26.241 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:26.241 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:26.241 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.4", 2024-04-07T17:51:26.241 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:26.241 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:26.241 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:26.241 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4983, 2024-04-07T17:51:26.242 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:26.242 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T18:17:15.023877+0000", 2024-04-07T17:51:26.242 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:26.242 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:26.242 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:26.242 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:26.242 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:26.242 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:26.242 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:26.242 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:26.242 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:26.242 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:26.242 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:26.242 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:26.242 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:26.242 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:26.243 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:26.243 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:26.243 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:26.243 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:26.243 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:26.243 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:26.243 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:26.243 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:26.243 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:26.243 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:26.243 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:26.243 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:26.243 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:26.243 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:26.243 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:26.243 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:26.244 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:26.244 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:26.244 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:26.244 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:26.244 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:26.244 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:26.244 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:26.244 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:26.244 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:26.244 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:26.244 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:26.244 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:26.244 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:26.244 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.244 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:26.245 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:26.245 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:26.245 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:26.245 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T17:51:26.245 INFO:tasks.workunit.client.0.smithi100.stdout: 5 2024-04-07T17:51:26.245 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.245 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 1, 2024-04-07T17:51:26.245 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:26.245 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.245 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:26.245 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:26.245 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:26.245 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:26.245 INFO:tasks.workunit.client.0.smithi100.stdout: 4 2024-04-07T17:51:26.245 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.245 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 7, 2024-04-07T17:51:26.246 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:26.246 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:26.246 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T17:51:26.246 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:26.246 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:26.246 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:26.246 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.372971+0000", 2024-04-07T17:51:26.246 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:39.001835+0000", 2024-04-07T17:51:26.246 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:39.001835+0000", 2024-04-07T17:51:26.246 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:39.003607+0000", 2024-04-07T17:51:26.246 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.372971+0000", 2024-04-07T17:51:26.246 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:26.246 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:26.246 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:26.246 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:26.247 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.372971+0000", 2024-04-07T17:51:26.247 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.372971+0000", 2024-04-07T17:51:26.247 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.372971+0000", 2024-04-07T17:51:26.247 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:26.247 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:26.247 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:26.247 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.372971+0000", 2024-04-07T17:51:26.247 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.372971+0000", 2024-04-07T17:51:26.247 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:26.247 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:26.247 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:26.247 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:26.247 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:26.247 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:26.247 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:26.247 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:26.247 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:26.247 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:26.248 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:26.248 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:26.248 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:26.248 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.5", 2024-04-07T17:51:26.248 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:26.248 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:26.248 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:26.248 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4795, 2024-04-07T17:51:26.248 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:26.248 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T21:54:36.255484+0000", 2024-04-07T17:51:26.248 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:26.248 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:26.248 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:26.248 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:26.248 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:26.248 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:26.248 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:26.249 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:26.249 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:26.249 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:26.249 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:26.249 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:26.249 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:26.249 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:26.249 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:26.249 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:26.249 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:26.249 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:26.249 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:26.249 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:26.249 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:26.249 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:26.249 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:26.249 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:26.249 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:26.250 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:26.250 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:26.250 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:26.250 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:26.250 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:26.250 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:26.250 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:26.250 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:26.250 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:26.250 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:26.250 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:26.250 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:26.250 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:26.250 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:26.251 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:26.251 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:26.251 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:26.251 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:26.251 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.251 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:26.251 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:26.251 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:26.251 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:26.251 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:26.251 INFO:tasks.workunit.client.0.smithi100.stdout: 4 2024-04-07T17:51:26.251 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.251 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 7, 2024-04-07T17:51:26.251 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:26.251 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.251 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:26.252 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:26.252 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:26.252 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:26.252 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T17:51:26.252 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.252 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 0, 2024-04-07T17:51:26.252 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:26.252 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:26.252 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 66, 2024-04-07T17:51:26.252 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:26.252 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:26.252 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:26.252 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:51:22.430544+0000", 2024-04-07T17:51:26.252 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:39.003851+0000", 2024-04-07T17:51:26.252 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:39.003851+0000", 2024-04-07T17:51:26.253 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:59:56.571405+0000", 2024-04-07T17:51:26.253 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:51:22.430544+0000", 2024-04-07T17:51:26.253 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:13:28.963534+0000", 2024-04-07T17:51:26.253 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:26.253 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:13:28.963534+0000", 2024-04-07T17:51:26.253 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:26.253 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:51:22.430544+0000", 2024-04-07T17:51:26.253 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:51:22.430544+0000", 2024-04-07T17:51:26.253 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:51:22.430544+0000", 2024-04-07T17:51:26.253 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:26.253 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:26.253 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:13:28.963534+0000", 2024-04-07T17:51:26.253 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:51:22.430544+0000", 2024-04-07T17:51:26.253 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:51:22.430544+0000", 2024-04-07T17:51:26.253 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:26.254 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 3135, 2024-04-07T17:51:26.254 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:26.254 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:26.254 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:26.254 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:26.254 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:26.254 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:26.254 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:26.254 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 3135, 2024-04-07T17:51:26.254 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:26.254 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:26.254 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:26.254 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "4.3", 2024-04-07T17:51:26.254 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:26.254 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:26.255 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:26.255 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 8778, 2024-04-07T17:51:26.255 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:26.255 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T23:06:05.133511+0000", 2024-04-07T17:51:26.255 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.019028691, 2024-04-07T17:51:26.255 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:26.255 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:26.255 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:26.255 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:26.255 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:26.255 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:26.255 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:26.255 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:26.255 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:26.255 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:26.256 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:26.256 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:26.256 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:26.256 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:26.256 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:26.256 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:26.256 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:26.256 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:26.256 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 3, 2024-04-07T17:51:26.256 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 1, 2024-04-07T17:51:26.256 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:26.256 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 1, 2024-04-07T17:51:26.256 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:26.256 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:26.256 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:26.257 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:26.257 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:26.257 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 1, 2024-04-07T17:51:26.257 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:26.257 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:26.257 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:26.257 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:26.257 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:26.257 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:26.257 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:26.257 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 873, 2024-04-07T17:51:26.257 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 391393, 2024-04-07T17:51:26.257 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:26.257 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:26.257 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:26.257 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 3127, 2024-04-07T17:51:26.257 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 378194 2024-04-07T17:51:26.257 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.258 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:26.258 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:26.258 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:26.258 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:26.258 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:26.258 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T17:51:26.258 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.258 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 0, 2024-04-07T17:51:26.258 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "1084'3135" 2024-04-07T17:51:26.258 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.258 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:26.258 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:26.258 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:26.258 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:26.258 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T17:51:26.259 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.259 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 7, 2024-04-07T17:51:26.259 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:26.259 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:26.259 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T17:51:26.259 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:26.259 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:26.259 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:26.259 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.364517+0000", 2024-04-07T17:51:26.259 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:39.000134+0000", 2024-04-07T17:51:26.259 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:39.000134+0000", 2024-04-07T17:51:26.259 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:39.003944+0000", 2024-04-07T17:51:26.259 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.364517+0000", 2024-04-07T17:51:26.259 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:26.259 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:26.259 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:26.259 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:26.260 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.364517+0000", 2024-04-07T17:51:26.260 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.364517+0000", 2024-04-07T17:51:26.260 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.364517+0000", 2024-04-07T17:51:26.260 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:26.260 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:26.260 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:26.260 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.364517+0000", 2024-04-07T17:51:26.260 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.364517+0000", 2024-04-07T17:51:26.260 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:26.260 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:26.260 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:26.260 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:26.260 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:26.260 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:26.260 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:26.260 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:26.261 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:26.261 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:26.261 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:26.261 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:26.261 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:26.261 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.0", 2024-04-07T17:51:26.261 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:26.261 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:26.261 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:26.261 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4796, 2024-04-07T17:51:26.261 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:26.261 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T22:28:12.376172+0000", 2024-04-07T17:51:26.261 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:26.261 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:26.261 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:26.261 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:26.262 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:26.262 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:26.262 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:26.262 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:26.262 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:26.262 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:26.262 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:26.262 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:26.262 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:26.262 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:26.262 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:26.262 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:26.262 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:26.262 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:26.262 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:26.262 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:26.263 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:26.263 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:26.263 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:26.263 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:26.263 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:26.263 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:26.263 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:26.263 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:26.263 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:26.263 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:26.263 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:26.263 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:26.263 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:26.263 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:26.263 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:26.264 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:26.264 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:26.264 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:26.264 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:26.264 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:26.264 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:26.264 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:26.264 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:26.264 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.264 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:26.264 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:26.264 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:26.264 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:26.264 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:26.264 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T17:51:26.264 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.265 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 7, 2024-04-07T17:51:26.265 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:26.265 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.265 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:26.265 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:26.265 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T17:51:26.265 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:26.265 INFO:tasks.workunit.client.0.smithi100.stdout: 3 2024-04-07T17:51:26.265 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.265 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 2, 2024-04-07T17:51:26.265 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:26.265 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:26.265 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:26.265 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:26.265 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:26.265 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:26.265 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.354824+0000", 2024-04-07T17:51:26.266 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:24:13.930517+0000", 2024-04-07T17:51:26.266 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:24:13.930517+0000", 2024-04-07T17:51:26.266 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:24:13.931259+0000", 2024-04-07T17:51:26.266 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.354824+0000", 2024-04-07T17:51:26.266 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.266 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:26.266 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.266 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 500, 2024-04-07T17:51:26.266 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.354824+0000", 2024-04-07T17:51:26.266 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.354824+0000", 2024-04-07T17:51:26.266 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.354824+0000", 2024-04-07T17:51:26.266 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:26.266 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:26.266 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.266 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.354824+0000", 2024-04-07T17:51:26.267 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.354824+0000", 2024-04-07T17:51:26.267 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:26.267 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:26.267 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:26.267 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:26.267 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 499, 2024-04-07T17:51:26.267 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:26.267 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:26.267 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:26.267 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:26.267 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:26.267 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:26.267 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:26.267 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:26.267 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.1", 2024-04-07T17:51:26.267 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:26.267 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:26.268 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:26.268 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4929, 2024-04-07T17:51:26.268 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:26.268 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T01:43:11.885075+0000", 2024-04-07T17:51:26.268 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:26.268 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:26.268 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:26.268 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:26.268 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:26.268 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:26.268 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:26.268 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:26.268 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:26.268 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:26.268 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:26.268 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:26.269 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:26.269 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:26.269 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:26.269 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:26.269 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:26.269 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:26.269 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:26.269 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:26.269 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:26.269 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:26.269 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:26.269 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:26.269 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:26.269 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:26.269 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:26.269 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:26.269 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:26.269 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:26.270 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:26.270 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:26.270 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:26.270 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:26.270 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:26.270 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:26.270 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:26.270 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:26.270 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:26.270 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:26.270 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:26.270 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:26.270 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:26.270 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.270 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:26.270 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:26.271 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:26.271 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T17:51:26.271 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:26.271 INFO:tasks.workunit.client.0.smithi100.stdout: 3 2024-04-07T17:51:26.271 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.271 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 2, 2024-04-07T17:51:26.271 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:26.271 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.271 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:26.271 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:26.271 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:26.271 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:26.271 INFO:tasks.workunit.client.0.smithi100.stdout: 2 2024-04-07T17:51:26.271 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.271 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 0, 2024-04-07T17:51:26.271 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:26.271 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:26.272 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 66, 2024-04-07T17:51:26.272 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:26.272 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:26.272 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:26.272 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:35.982117+0000", 2024-04-07T17:51:26.272 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:23:15.882881+0000", 2024-04-07T17:51:26.272 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:23:15.882881+0000", 2024-04-07T17:51:26.272 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:59:56.569536+0000", 2024-04-07T17:51:26.272 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:35.982117+0000", 2024-04-07T17:51:26.272 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:13:28.963534+0000", 2024-04-07T17:51:26.272 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:26.272 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:13:28.963534+0000", 2024-04-07T17:51:26.272 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 445, 2024-04-07T17:51:26.272 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:35.982117+0000", 2024-04-07T17:51:26.272 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:35.982117+0000", 2024-04-07T17:51:26.272 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:35.982117+0000", 2024-04-07T17:51:26.272 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:26.272 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:26.273 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:13:28.963534+0000", 2024-04-07T17:51:26.273 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:35.982117+0000", 2024-04-07T17:51:26.273 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:35.982117+0000", 2024-04-07T17:51:26.273 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:26.273 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 3107, 2024-04-07T17:51:26.273 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:26.273 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:26.273 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 444, 2024-04-07T17:51:26.273 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:26.273 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:26.273 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:26.273 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:26.273 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 3107, 2024-04-07T17:51:26.273 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:26.273 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:26.273 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:26.273 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "4.6", 2024-04-07T17:51:26.274 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:26.274 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:26.274 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:26.274 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 8717, 2024-04-07T17:51:26.274 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:26.274 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T18:35:28.250440+0000", 2024-04-07T17:51:26.274 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.017233416, 2024-04-07T17:51:26.274 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:26.274 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:26.274 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:26.274 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:26.274 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:26.274 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:26.274 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:26.275 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:26.275 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:26.275 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:26.275 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:26.275 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:26.275 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:26.275 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:26.275 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:26.275 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:26.275 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:26.275 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:26.275 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:26.275 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:26.275 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:26.275 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:26.275 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:26.276 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:26.276 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:26.276 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:26.276 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:26.276 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:26.276 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:26.276 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:26.276 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:26.276 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:26.276 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:26.276 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:26.276 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:26.276 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 858, 2024-04-07T17:51:26.276 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 396543, 2024-04-07T17:51:26.276 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:26.276 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:26.276 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:26.277 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 3070, 2024-04-07T17:51:26.277 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 377559 2024-04-07T17:51:26.277 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.277 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:26.277 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:26.277 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:26.277 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:26.277 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:26.277 INFO:tasks.workunit.client.0.smithi100.stdout: 2 2024-04-07T17:51:26.277 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.277 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 0, 2024-04-07T17:51:26.277 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "1084'3107" 2024-04-07T17:51:26.277 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.277 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:26.277 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:26.277 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:26.277 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:26.277 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T17:51:26.278 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.278 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 7, 2024-04-07T17:51:26.278 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:26.278 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:26.278 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 19, 2024-04-07T17:51:26.278 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:26.278 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:26.278 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:26.278 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.366640+0000", 2024-04-07T17:51:26.278 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:39.000587+0000", 2024-04-07T17:51:26.278 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:39.000587+0000", 2024-04-07T17:51:26.278 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:39.004716+0000", 2024-04-07T17:51:26.278 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.366640+0000", 2024-04-07T17:51:26.278 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:07:26.866003+0000", 2024-04-07T17:51:26.278 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:26.278 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:07:26.866003+0000", 2024-04-07T17:51:26.279 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:26.279 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.366640+0000", 2024-04-07T17:51:26.279 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.366640+0000", 2024-04-07T17:51:26.279 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.366640+0000", 2024-04-07T17:51:26.279 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:26.279 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:26.279 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:07:26.866003+0000", 2024-04-07T17:51:26.279 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.366640+0000", 2024-04-07T17:51:26.279 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.366640+0000", 2024-04-07T17:51:26.279 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:26.279 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 241, 2024-04-07T17:51:26.279 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:26.279 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:26.279 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:26.279 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:26.279 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:26.279 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:26.280 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:26.280 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 241, 2024-04-07T17:51:26.280 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:26.280 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:26.280 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:26.280 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "1.0", 2024-04-07T17:51:26.280 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:26.280 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:26.280 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:26.280 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 5460, 2024-04-07T17:51:26.280 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:26.280 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T21:59:54.730545+0000", 2024-04-07T17:51:26.280 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:26.280 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:26.280 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:26.280 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 590368, 2024-04-07T17:51:26.280 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:26.280 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 1771104, 2024-04-07T17:51:26.281 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:26.281 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:26.281 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:26.281 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:26.281 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:26.281 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:26.281 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:26.281 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:26.281 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:26.281 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:26.281 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:26.281 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:26.281 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:26.281 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 6, 2024-04-07T17:51:26.281 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 2, 2024-04-07T17:51:26.281 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:26.281 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 2, 2024-04-07T17:51:26.282 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:26.282 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:26.282 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:26.282 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:26.282 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:26.282 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:26.282 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:26.282 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 6, 2024-04-07T17:51:26.282 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:26.282 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:26.282 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:26.282 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:26.282 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:26.282 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 1214, 2024-04-07T17:51:26.283 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 2024, 2024-04-07T17:51:26.283 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:26.283 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:26.283 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:26.283 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 399, 2024-04-07T17:51:26.283 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 2416 2024-04-07T17:51:26.283 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.283 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:26.283 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:26.283 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:26.283 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:26.283 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:26.283 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T17:51:26.283 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.283 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 7, 2024-04-07T17:51:26.283 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "2111'241" 2024-04-07T17:51:26.283 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.284 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:26.284 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:26.284 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:26.284 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T17:51:26.284 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T17:51:26.284 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.284 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 5, 2024-04-07T17:51:26.284 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:26.284 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:26.284 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T17:51:26.284 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:26.284 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:26.284 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:26.284 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.370078+0000", 2024-04-07T17:51:26.284 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:38.985921+0000", 2024-04-07T17:51:26.284 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:38.985921+0000", 2024-04-07T17:51:26.285 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:38.986647+0000", 2024-04-07T17:51:26.285 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.370078+0000", 2024-04-07T17:51:26.285 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:26.285 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:26.285 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:26.285 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:26.285 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.370078+0000", 2024-04-07T17:51:26.285 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.370078+0000", 2024-04-07T17:51:26.285 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.370078+0000", 2024-04-07T17:51:26.285 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:26.285 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:26.285 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:26.285 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.370078+0000", 2024-04-07T17:51:26.285 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.370078+0000", 2024-04-07T17:51:26.285 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:26.285 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:26.286 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:26.286 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:26.286 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:26.286 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:26.286 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:26.286 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:26.286 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:26.286 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:26.286 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:26.286 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:26.286 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:26.286 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.3", 2024-04-07T17:51:26.286 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:26.287 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:26.287 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:26.287 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4855, 2024-04-07T17:51:26.287 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:26.287 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T02:44:42.906581+0000", 2024-04-07T17:51:26.287 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:26.287 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:26.287 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:26.287 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:26.287 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:26.287 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:26.287 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:26.287 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:26.287 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:26.287 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:26.287 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:26.288 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:26.288 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:26.288 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:26.288 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:26.288 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:26.288 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:26.288 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:26.288 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:26.288 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:26.288 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:26.288 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:26.288 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:26.288 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:26.288 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:26.288 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:26.288 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:26.288 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:26.289 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:26.289 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:26.289 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:26.289 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:26.289 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:26.289 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:26.289 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:26.289 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:26.289 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:26.289 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:26.289 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:26.289 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:26.289 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:26.289 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:26.289 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:26.289 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.290 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:26.290 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:26.290 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:26.290 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:26.290 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T17:51:26.290 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T17:51:26.290 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.290 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 5, 2024-04-07T17:51:26.290 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:26.290 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.290 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:26.290 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:26.290 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:26.290 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:26.290 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T17:51:26.291 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.291 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 3, 2024-04-07T17:51:26.291 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:26.291 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:26.291 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:26.291 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:26.291 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:26.291 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:26.291 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:35.982147+0000", 2024-04-07T17:51:26.291 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:08.977825+0000", 2024-04-07T17:51:26.291 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:08.977825+0000", 2024-04-07T17:51:26.291 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:08.978231+0000", 2024-04-07T17:51:26.291 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:35.982147+0000", 2024-04-07T17:51:26.291 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.291 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:26.291 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.291 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 554, 2024-04-07T17:51:26.292 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:35.982147+0000", 2024-04-07T17:51:26.292 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:35.982147+0000", 2024-04-07T17:51:26.292 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:35.982147+0000", 2024-04-07T17:51:26.292 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:26.292 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:26.292 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.292 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:35.982147+0000", 2024-04-07T17:51:26.292 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:35.982147+0000", 2024-04-07T17:51:26.292 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:26.292 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:26.292 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:26.292 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:26.292 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 553, 2024-04-07T17:51:26.292 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:26.292 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:26.292 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:26.292 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:26.293 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:26.293 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:26.293 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:26.293 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:26.293 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.2", 2024-04-07T17:51:26.293 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:26.293 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:26.293 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:26.293 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4937, 2024-04-07T17:51:26.293 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:26.293 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T01:34:08.840626+0000", 2024-04-07T17:51:26.293 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:26.293 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:26.293 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:26.293 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:26.293 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:26.294 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:26.294 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:26.294 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:26.294 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:26.294 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:26.294 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:26.294 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:26.294 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:26.294 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:26.294 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:26.294 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:26.294 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:26.294 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:26.294 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:26.294 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:26.295 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:26.295 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:26.295 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:26.295 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:26.295 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:26.295 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:26.295 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:26.295 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:26.295 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:26.295 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:26.295 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:26.295 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:26.295 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:26.295 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:26.295 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:26.296 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:26.296 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:26.296 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:26.296 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:26.296 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:26.296 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:26.296 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:26.296 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:26.296 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.296 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:26.296 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:26.296 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:26.296 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:26.296 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:26.296 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T17:51:26.297 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.297 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 3, 2024-04-07T17:51:26.297 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:26.297 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.297 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:26.297 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:26.297 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:26.297 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:26.297 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T17:51:26.297 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.297 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 6, 2024-04-07T17:51:26.297 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:26.297 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:26.297 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 66, 2024-04-07T17:51:26.297 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:26.297 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:26.297 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:26.298 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.360030+0000", 2024-04-07T17:51:26.298 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:39.000185+0000", 2024-04-07T17:51:26.298 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:39.000185+0000", 2024-04-07T17:51:26.298 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:59:56.544598+0000", 2024-04-07T17:51:26.298 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.360030+0000", 2024-04-07T17:51:26.298 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:13:28.963534+0000", 2024-04-07T17:51:26.298 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:26.298 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:13:28.963534+0000", 2024-04-07T17:51:26.298 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:26.298 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.360030+0000", 2024-04-07T17:51:26.298 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.360030+0000", 2024-04-07T17:51:26.298 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.360030+0000", 2024-04-07T17:51:26.298 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:26.298 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:26.299 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:13:28.963534+0000", 2024-04-07T17:51:26.299 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.360030+0000", 2024-04-07T17:51:26.299 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.360030+0000", 2024-04-07T17:51:26.299 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:26.299 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 3124, 2024-04-07T17:51:26.299 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:26.299 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:26.299 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:26.299 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:26.299 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:26.299 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:26.299 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:26.299 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 3124, 2024-04-07T17:51:26.299 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:26.299 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:26.299 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:26.299 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "4.5", 2024-04-07T17:51:26.300 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:26.300 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:26.300 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:26.300 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 8600, 2024-04-07T17:51:26.300 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:26.300 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T01:53:54.315291+0000", 2024-04-07T17:51:26.300 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.008144472, 2024-04-07T17:51:26.300 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:26.300 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:26.300 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:26.300 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:26.300 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:26.300 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:26.300 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:26.300 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:26.300 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:26.301 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:26.301 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:26.301 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:26.301 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:26.301 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:26.301 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:26.301 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:26.301 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:26.301 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:26.301 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:26.301 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:26.301 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:26.301 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:26.301 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:26.301 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:26.301 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:26.301 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:26.302 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:26.302 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:26.302 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:26.302 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:26.302 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:26.302 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:26.302 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:26.302 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:26.302 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:26.302 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 979, 2024-04-07T17:51:26.302 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 424424, 2024-04-07T17:51:26.302 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:26.302 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:26.302 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:26.302 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 3125, 2024-04-07T17:51:26.303 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 420851 2024-04-07T17:51:26.303 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.303 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:26.303 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:26.303 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:26.303 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:26.303 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:26.303 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T17:51:26.303 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.303 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 6, 2024-04-07T17:51:26.303 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "1083'3124" 2024-04-07T17:51:26.303 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.303 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:26.303 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:26.303 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T17:51:26.303 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:26.304 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T17:51:26.304 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.304 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 2, 2024-04-07T17:51:26.304 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:26.304 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:26.304 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T17:51:26.304 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:26.304 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:26.304 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:26.304 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.355116+0000", 2024-04-07T17:51:26.304 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:23:43.929900+0000", 2024-04-07T17:51:26.304 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:23:43.929900+0000", 2024-04-07T17:51:26.304 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:23:43.930960+0000", 2024-04-07T17:51:26.304 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.355116+0000", 2024-04-07T17:51:26.304 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:26.304 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:26.304 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:26.305 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 471, 2024-04-07T17:51:26.305 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.355116+0000", 2024-04-07T17:51:26.305 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.355116+0000", 2024-04-07T17:51:26.305 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.355116+0000", 2024-04-07T17:51:26.305 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:26.305 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:26.305 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:26.305 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.355116+0000", 2024-04-07T17:51:26.305 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.355116+0000", 2024-04-07T17:51:26.305 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:26.305 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:26.305 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:26.305 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:26.305 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 470, 2024-04-07T17:51:26.305 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:26.305 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:26.306 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:26.306 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:26.306 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:26.306 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:26.306 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:26.306 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:26.306 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.1", 2024-04-07T17:51:26.306 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:26.306 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:26.306 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:26.306 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4977, 2024-04-07T17:51:26.306 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:26.306 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T20:05:04.268255+0000", 2024-04-07T17:51:26.306 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:26.306 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:26.307 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:26.307 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:26.307 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:26.307 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:26.307 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:26.307 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:26.307 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:26.307 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:26.307 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:26.307 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:26.307 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:26.307 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:26.307 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:26.307 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:26.307 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:26.307 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:26.307 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:26.308 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:26.308 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:26.308 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:26.308 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:26.308 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:26.308 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:26.308 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:26.308 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:26.308 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:26.308 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:26.308 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:26.308 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:26.308 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:26.308 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:26.308 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:26.308 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:26.308 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:26.309 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:26.309 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:26.309 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:26.309 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:26.309 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:26.309 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:26.309 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:26.309 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.309 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:26.309 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:26.309 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:26.309 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T17:51:26.309 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:26.309 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T17:51:26.309 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.309 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 2, 2024-04-07T17:51:26.310 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:26.310 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.310 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:26.310 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:26.310 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:26.310 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T17:51:26.310 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T17:51:26.310 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.310 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 1, 2024-04-07T17:51:26.310 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:26.310 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:26.310 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:26.310 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:26.310 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:26.310 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:26.311 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.331050+0000", 2024-04-07T17:51:26.311 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:08.961818+0000", 2024-04-07T17:51:26.311 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:08.961818+0000", 2024-04-07T17:51:26.311 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:08.962269+0000", 2024-04-07T17:51:26.311 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.331050+0000", 2024-04-07T17:51:26.311 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.311 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:26.311 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.311 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 554, 2024-04-07T17:51:26.311 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.331050+0000", 2024-04-07T17:51:26.311 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.331050+0000", 2024-04-07T17:51:26.311 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.331050+0000", 2024-04-07T17:51:26.311 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:26.311 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:26.311 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.311 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.331050+0000", 2024-04-07T17:51:26.311 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.331050+0000", 2024-04-07T17:51:26.311 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:26.312 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:26.312 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:26.312 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:26.312 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 553, 2024-04-07T17:51:26.312 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:26.312 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:26.312 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:26.312 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:26.312 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:26.312 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:26.312 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:26.312 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:26.312 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.0", 2024-04-07T17:51:26.312 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:26.312 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:26.312 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:26.312 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4985, 2024-04-07T17:51:26.313 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:26.313 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T01:33:47.318091+0000", 2024-04-07T17:51:26.313 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:26.313 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:26.313 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:26.313 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:26.313 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:26.313 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:26.313 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:26.313 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:26.313 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:26.313 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:26.313 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:26.313 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:26.313 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:26.313 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:26.313 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:26.314 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:26.314 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:26.314 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:26.314 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:26.314 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:26.314 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:26.314 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:26.314 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:26.314 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:26.314 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:26.314 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:26.314 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:26.314 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:26.314 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:26.314 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:26.315 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:26.315 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:26.315 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:26.315 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:26.315 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:26.315 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:26.315 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:26.315 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:26.315 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:26.315 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:26.315 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:26.315 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:26.315 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:26.315 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.315 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:26.315 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:26.316 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:26.316 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:26.316 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T17:51:26.316 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T17:51:26.316 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.316 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 1, 2024-04-07T17:51:26.316 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:26.316 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.316 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:26.316 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:26.316 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:26.316 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:26.316 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T17:51:26.316 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.316 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 1, 2024-04-07T17:51:26.316 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:26.317 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:26.317 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 66, 2024-04-07T17:51:26.317 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:26.317 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:26.317 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:26.317 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.332996+0000", 2024-04-07T17:51:26.317 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:24:41.570251+0000", 2024-04-07T17:51:26.317 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:24:41.570251+0000", 2024-04-07T17:51:26.317 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:59:56.540592+0000", 2024-04-07T17:51:26.317 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.332996+0000", 2024-04-07T17:51:26.317 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:13:28.963534+0000", 2024-04-07T17:51:26.317 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:26.317 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:13:28.963534+0000", 2024-04-07T17:51:26.317 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 527, 2024-04-07T17:51:26.317 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.332996+0000", 2024-04-07T17:51:26.317 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.332996+0000", 2024-04-07T17:51:26.317 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.332996+0000", 2024-04-07T17:51:26.318 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:26.318 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:26.318 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:13:28.963534+0000", 2024-04-07T17:51:26.318 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.332996+0000", 2024-04-07T17:51:26.318 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.332996+0000", 2024-04-07T17:51:26.318 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:26.318 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 2973, 2024-04-07T17:51:26.318 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:26.318 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:26.318 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 526, 2024-04-07T17:51:26.318 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:26.318 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:26.318 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:26.318 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:26.318 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 2973, 2024-04-07T17:51:26.319 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:26.319 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:26.319 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:26.319 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "4.7", 2024-04-07T17:51:26.319 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:26.319 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:26.319 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:26.319 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 8716, 2024-04-07T17:51:26.319 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:26.319 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T19:36:01.668815+0000", 2024-04-07T17:51:26.319 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.014750449, 2024-04-07T17:51:26.319 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:26.319 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:26.319 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:26.319 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:26.319 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:26.320 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:26.320 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:26.320 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:26.320 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:26.320 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:26.320 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:26.320 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:26.320 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:26.320 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:26.320 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:26.320 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:26.320 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:26.320 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:26.320 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:26.320 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:26.321 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:26.321 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:26.321 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:26.321 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:26.321 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:26.321 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:26.321 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:26.321 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:26.321 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:26.321 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:26.321 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:26.321 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:26.321 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:26.321 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:26.321 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:26.322 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 1318, 2024-04-07T17:51:26.322 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 368358, 2024-04-07T17:51:26.322 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:26.322 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:26.322 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:26.322 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 2957, 2024-04-07T17:51:26.322 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 380954 2024-04-07T17:51:26.322 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.322 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:26.322 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:26.322 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:26.322 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:26.322 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:26.322 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T17:51:26.323 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.323 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 1, 2024-04-07T17:51:26.323 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "1084'2973" 2024-04-07T17:51:26.323 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.323 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:26.323 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:26.323 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:26.323 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:26.323 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T17:51:26.323 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.323 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 4, 2024-04-07T17:51:26.323 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:26.323 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:26.323 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:26.323 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:26.324 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:26.324 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:26.324 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:35.979039+0000", 2024-04-07T17:51:26.324 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:08.952350+0000", 2024-04-07T17:51:26.324 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:08.952350+0000", 2024-04-07T17:51:26.324 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:08.952832+0000", 2024-04-07T17:51:26.324 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:35.979039+0000", 2024-04-07T17:51:26.324 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.324 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:26.324 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.324 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 554, 2024-04-07T17:51:26.324 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:35.979039+0000", 2024-04-07T17:51:26.324 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:35.979039+0000", 2024-04-07T17:51:26.324 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:35.979039+0000", 2024-04-07T17:51:26.324 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:26.324 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:26.325 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.325 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:35.979039+0000", 2024-04-07T17:51:26.325 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:35.979039+0000", 2024-04-07T17:51:26.325 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:26.325 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:26.325 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:26.325 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:26.325 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 553, 2024-04-07T17:51:26.325 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:26.325 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:26.325 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:26.325 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:26.325 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:26.325 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:26.325 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:26.326 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:26.326 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.3", 2024-04-07T17:51:26.326 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:26.326 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:26.326 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:26.326 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4919, 2024-04-07T17:51:26.326 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:26.326 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T22:57:40.507778+0000", 2024-04-07T17:51:26.326 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:26.326 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:26.326 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:26.326 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:26.326 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:26.326 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:26.326 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:26.327 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:26.327 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:26.327 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:26.327 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:26.327 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:26.327 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:26.327 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:26.327 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:26.327 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:26.327 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:26.327 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:26.327 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:26.327 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:26.327 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:26.327 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:26.327 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:26.328 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:26.328 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:26.328 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:26.328 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:26.328 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:26.328 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:26.328 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:26.328 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:26.328 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:26.328 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:26.328 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:26.328 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:26.328 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:26.328 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:26.328 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:26.328 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:26.328 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:26.329 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:26.329 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:26.329 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:26.329 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.329 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:26.329 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:26.329 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:26.329 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:26.329 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:26.329 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T17:51:26.329 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.329 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 4, 2024-04-07T17:51:26.329 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:26.329 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.329 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:26.329 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:26.329 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:26.330 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:26.330 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T17:51:26.330 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.330 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 5, 2024-04-07T17:51:26.330 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:26.330 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:26.330 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T17:51:26.330 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:26.330 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:26.330 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:26.330 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.369727+0000", 2024-04-07T17:51:26.330 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:08.972663+0000", 2024-04-07T17:51:26.330 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:08.972663+0000", 2024-04-07T17:51:26.330 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:08.973250+0000", 2024-04-07T17:51:26.330 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.369727+0000", 2024-04-07T17:51:26.331 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:26.331 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:26.331 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:26.331 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 554, 2024-04-07T17:51:26.331 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.369727+0000", 2024-04-07T17:51:26.331 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.369727+0000", 2024-04-07T17:51:26.331 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.369727+0000", 2024-04-07T17:51:26.331 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:26.331 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:26.331 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:26.331 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.369727+0000", 2024-04-07T17:51:26.331 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.369727+0000", 2024-04-07T17:51:26.331 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:26.331 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:26.331 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:26.332 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:26.332 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 553, 2024-04-07T17:51:26.332 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:26.332 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:26.332 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:26.332 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:26.332 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:26.332 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:26.332 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:26.332 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:26.332 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.2", 2024-04-07T17:51:26.332 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:26.332 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:26.332 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:26.332 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4868, 2024-04-07T17:51:26.332 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:26.333 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T22:42:44.603307+0000", 2024-04-07T17:51:26.333 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:26.333 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:26.333 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:26.333 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:26.333 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:26.333 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:26.333 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:26.333 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:26.333 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:26.333 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:26.333 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:26.333 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:26.333 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:26.333 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:26.333 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:26.334 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:26.334 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:26.334 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:26.334 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:26.334 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:26.334 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:26.334 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:26.334 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:26.334 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:26.334 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:26.334 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:26.334 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:26.334 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:26.334 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:26.334 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:26.335 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:26.335 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:26.335 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:26.335 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:26.335 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:26.335 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:26.335 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:26.335 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:26.335 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:26.335 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:26.335 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:26.335 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:26.335 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:26.335 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.335 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:26.335 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:26.336 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:26.336 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:26.336 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:26.336 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T17:51:26.336 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.336 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 5, 2024-04-07T17:51:26.336 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:26.336 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.336 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:26.336 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:26.336 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:26.336 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T17:51:26.336 INFO:tasks.workunit.client.0.smithi100.stdout: 3 2024-04-07T17:51:26.336 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.336 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 1, 2024-04-07T17:51:26.337 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:26.337 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:26.337 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 66, 2024-04-07T17:51:26.337 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:26.337 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:26.337 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:26.337 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.334401+0000", 2024-04-07T17:51:26.337 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:23:43.924443+0000", 2024-04-07T17:51:26.337 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:23:43.924443+0000", 2024-04-07T17:51:26.337 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:59:56.538443+0000", 2024-04-07T17:51:26.337 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.334401+0000", 2024-04-07T17:51:26.337 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:13:28.963534+0000", 2024-04-07T17:51:26.337 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:26.337 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:13:28.963534+0000", 2024-04-07T17:51:26.337 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 471, 2024-04-07T17:51:26.337 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.334401+0000", 2024-04-07T17:51:26.337 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.334401+0000", 2024-04-07T17:51:26.338 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.334401+0000", 2024-04-07T17:51:26.338 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:26.338 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:26.338 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:13:28.963534+0000", 2024-04-07T17:51:26.338 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.334401+0000", 2024-04-07T17:51:26.338 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.334401+0000", 2024-04-07T17:51:26.338 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:26.338 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 2976, 2024-04-07T17:51:26.338 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:26.338 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:26.338 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 470, 2024-04-07T17:51:26.338 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:26.338 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:26.338 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:26.338 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:26.339 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 2976, 2024-04-07T17:51:26.339 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:26.339 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:26.339 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:26.339 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "4.4", 2024-04-07T17:51:26.339 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:26.339 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:26.339 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:26.339 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 8713, 2024-04-07T17:51:26.339 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:26.339 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T16:43:02.513339+0000", 2024-04-07T17:51:26.339 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.010915499, 2024-04-07T17:51:26.339 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:26.339 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:26.339 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:26.340 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:26.340 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:26.340 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:26.340 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:26.340 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:26.340 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:26.340 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:26.340 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:26.340 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:26.340 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:26.340 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:26.340 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:26.340 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:26.340 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:26.340 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:26.340 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 3, 2024-04-07T17:51:26.341 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 1, 2024-04-07T17:51:26.341 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:26.341 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 1, 2024-04-07T17:51:26.341 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:26.341 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:26.341 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:26.341 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:26.341 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:26.341 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 1, 2024-04-07T17:51:26.341 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:26.341 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:26.341 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:26.341 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:26.341 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:26.341 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:26.341 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:26.342 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 1187, 2024-04-07T17:51:26.342 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 375700, 2024-04-07T17:51:26.342 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:26.342 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:26.342 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:26.342 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 2976, 2024-04-07T17:51:26.342 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 350581 2024-04-07T17:51:26.342 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.342 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:26.342 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:26.342 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:26.342 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:26.342 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T17:51:26.342 INFO:tasks.workunit.client.0.smithi100.stdout: 3 2024-04-07T17:51:26.342 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.343 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 1, 2024-04-07T17:51:26.343 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "1084'2976" 2024-04-07T17:51:26.343 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.343 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:26.343 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:26.343 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:26.343 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:26.343 INFO:tasks.workunit.client.0.smithi100.stdout: 2 2024-04-07T17:51:26.343 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.343 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 5, 2024-04-07T17:51:26.343 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:26.343 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:26.343 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:26.343 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:26.343 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:26.344 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:26.344 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.364565+0000", 2024-04-07T17:51:26.344 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:24:41.554485+0000", 2024-04-07T17:51:26.344 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:24:41.554485+0000", 2024-04-07T17:51:26.344 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:24:41.557859+0000", 2024-04-07T17:51:26.344 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.364565+0000", 2024-04-07T17:51:26.344 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.344 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:26.344 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.344 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 527, 2024-04-07T17:51:26.344 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.364565+0000", 2024-04-07T17:51:26.344 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.364565+0000", 2024-04-07T17:51:26.344 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.364565+0000", 2024-04-07T17:51:26.344 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:26.344 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:26.345 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.345 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.364565+0000", 2024-04-07T17:51:26.345 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.364565+0000", 2024-04-07T17:51:26.345 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:26.345 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:26.345 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:26.345 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:26.345 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 526, 2024-04-07T17:51:26.345 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:26.345 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:26.345 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:26.345 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:26.345 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:26.345 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:26.345 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:26.346 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:26.346 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.5", 2024-04-07T17:51:26.346 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:26.346 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:26.346 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:26.346 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4886, 2024-04-07T17:51:26.346 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:26.346 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T18:53:57.243053+0000", 2024-04-07T17:51:26.346 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:26.346 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:26.346 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:26.346 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:26.346 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:26.346 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:26.347 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:26.347 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:26.347 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:26.347 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:26.347 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:26.347 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:26.347 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:26.347 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:26.347 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:26.347 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:26.347 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:26.347 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:26.347 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:26.347 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:26.347 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:26.347 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:26.348 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:26.348 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:26.348 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:26.348 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:26.348 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:26.348 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:26.348 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:26.348 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:26.348 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:26.348 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:26.348 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:26.348 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:26.348 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:26.348 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:26.348 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:26.348 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:26.349 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:26.349 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:26.349 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:26.349 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:26.349 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:26.349 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.349 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:26.349 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:26.349 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:26.349 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:26.349 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:26.349 INFO:tasks.workunit.client.0.smithi100.stdout: 2 2024-04-07T17:51:26.349 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.349 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 5, 2024-04-07T17:51:26.349 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:26.350 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.350 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:26.350 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:26.350 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:26.350 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:26.350 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T17:51:26.350 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.350 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 1, 2024-04-07T17:51:26.350 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:26.350 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:26.350 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T17:51:26.350 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:26.350 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:26.350 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:26.351 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.333650+0000", 2024-04-07T17:51:26.351 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:38.999666+0000", 2024-04-07T17:51:26.351 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:38.999666+0000", 2024-04-07T17:51:26.351 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:39.000176+0000", 2024-04-07T17:51:26.351 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.333650+0000", 2024-04-07T17:51:26.351 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:26.351 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:26.351 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:26.351 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:26.351 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.333650+0000", 2024-04-07T17:51:26.351 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.333650+0000", 2024-04-07T17:51:26.351 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.333650+0000", 2024-04-07T17:51:26.351 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:26.351 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:26.351 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:26.352 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.333650+0000", 2024-04-07T17:51:26.352 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.333650+0000", 2024-04-07T17:51:26.352 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:26.352 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:26.352 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:26.352 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:26.352 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:26.352 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:26.352 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:26.352 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:26.352 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:26.352 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:26.352 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:26.352 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:26.352 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:26.352 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.4", 2024-04-07T17:51:26.353 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:26.353 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:26.353 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:26.353 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4990, 2024-04-07T17:51:26.353 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:26.353 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T19:36:37.701087+0000", 2024-04-07T17:51:26.353 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:26.353 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:26.353 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:26.353 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:26.353 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:26.353 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:26.353 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:26.353 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:26.354 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:26.354 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:26.354 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:26.354 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:26.354 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:26.354 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:26.354 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:26.354 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:26.354 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:26.354 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:26.354 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:26.354 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:26.354 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:26.354 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:26.355 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:26.355 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:26.355 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:26.355 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:26.355 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:26.355 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:26.355 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:26.355 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:26.355 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:26.355 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:26.355 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:26.355 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:26.355 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:26.355 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:26.355 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:26.356 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:26.356 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:26.356 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:26.356 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:26.356 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:26.356 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:26.356 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.356 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:26.356 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:26.356 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:26.356 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:26.356 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:26.356 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T17:51:26.356 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.356 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 1, 2024-04-07T17:51:26.357 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:26.357 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.357 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:26.357 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:26.357 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:26.357 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:26.357 INFO:tasks.workunit.client.0.smithi100.stdout: 4 2024-04-07T17:51:26.357 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.357 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 1, 2024-04-07T17:51:26.357 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:26.357 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:26.357 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 66, 2024-04-07T17:51:26.357 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:26.357 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:26.357 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:26.357 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.335612+0000", 2024-04-07T17:51:26.358 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:24:41.537435+0000", 2024-04-07T17:51:26.358 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:24:41.537435+0000", 2024-04-07T17:51:26.358 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:59:56.544979+0000", 2024-04-07T17:51:26.358 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.335612+0000", 2024-04-07T17:51:26.358 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:13:28.963534+0000", 2024-04-07T17:51:26.358 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:26.358 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:13:28.963534+0000", 2024-04-07T17:51:26.358 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 527, 2024-04-07T17:51:26.358 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.335612+0000", 2024-04-07T17:51:26.358 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.335612+0000", 2024-04-07T17:51:26.358 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.335612+0000", 2024-04-07T17:51:26.358 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:26.358 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:26.358 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:13:28.963534+0000", 2024-04-07T17:51:26.358 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.335612+0000", 2024-04-07T17:51:26.359 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.335612+0000", 2024-04-07T17:51:26.359 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:26.359 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 2963, 2024-04-07T17:51:26.359 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:26.359 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:26.359 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 526, 2024-04-07T17:51:26.359 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:26.359 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:26.359 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:26.359 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:26.359 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 2963, 2024-04-07T17:51:26.359 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:26.359 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:26.359 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:26.359 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "4.2", 2024-04-07T17:51:26.360 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:26.360 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:26.360 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:26.360 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 8984, 2024-04-07T17:51:26.360 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:26.360 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T23:06:36.963212+0000", 2024-04-07T17:51:26.360 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.015938325, 2024-04-07T17:51:26.360 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:26.360 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:26.360 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 19, 2024-04-07T17:51:26.360 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:26.360 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:26.360 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:26.360 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:26.360 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:26.361 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:26.361 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:26.361 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:26.361 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:26.361 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:26.361 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:26.361 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:26.361 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:26.361 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:26.361 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:26.361 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 3, 2024-04-07T17:51:26.361 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 1, 2024-04-07T17:51:26.361 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:26.361 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 1, 2024-04-07T17:51:26.361 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:26.361 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:26.361 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:26.362 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:26.362 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:26.362 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:26.362 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:26.362 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:26.362 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:26.362 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:26.362 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:26.362 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:26.362 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:26.362 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 1654, 2024-04-07T17:51:26.362 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 387082, 2024-04-07T17:51:26.362 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:26.362 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:26.362 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:26.363 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 2937, 2024-04-07T17:51:26.363 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 349256 2024-04-07T17:51:26.363 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.363 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:26.363 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:26.363 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:26.363 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:26.363 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:26.363 INFO:tasks.workunit.client.0.smithi100.stdout: 4 2024-04-07T17:51:26.363 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.363 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 1, 2024-04-07T17:51:26.363 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "1083'2963" 2024-04-07T17:51:26.363 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.363 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:26.363 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:26.363 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:26.364 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:26.364 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T17:51:26.364 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.364 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 7, 2024-04-07T17:51:26.364 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:26.364 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:26.364 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:26.364 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:26.364 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:26.364 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:26.364 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.372374+0000", 2024-04-07T17:51:26.364 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:38.968892+0000", 2024-04-07T17:51:26.364 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:38.968892+0000", 2024-04-07T17:51:26.364 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:38.970807+0000", 2024-04-07T17:51:26.364 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.372374+0000", 2024-04-07T17:51:26.364 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.364 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:26.365 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.365 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:26.365 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.372374+0000", 2024-04-07T17:51:26.365 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.372374+0000", 2024-04-07T17:51:26.365 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.372374+0000", 2024-04-07T17:51:26.365 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:26.365 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:26.365 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.365 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.372374+0000", 2024-04-07T17:51:26.365 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.372374+0000", 2024-04-07T17:51:26.365 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:26.365 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:26.365 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:26.365 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:26.365 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:26.365 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:26.365 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:26.366 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:26.366 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:26.366 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:26.366 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:26.366 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:26.366 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:26.366 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.e", 2024-04-07T17:51:26.366 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:26.366 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:26.366 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:26.366 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4791, 2024-04-07T17:51:26.366 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:26.366 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T21:17:23.974869+0000", 2024-04-07T17:51:26.366 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:26.367 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:26.367 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:26.367 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:26.367 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:26.367 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:26.367 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:26.367 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:26.367 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:26.367 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:26.367 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:26.367 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:26.367 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:26.367 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:26.367 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:26.367 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:26.367 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:26.368 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:26.368 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:26.368 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:26.368 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:26.368 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:26.368 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:26.368 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:26.368 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:26.368 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:26.368 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:26.368 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:26.368 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:26.368 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:26.368 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:26.368 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:26.369 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:26.369 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:26.369 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:26.369 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:26.369 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:26.369 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:26.369 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:26.369 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:26.369 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:26.369 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:26.369 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:26.369 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.369 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:26.369 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:26.369 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:26.369 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:26.369 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:26.370 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T17:51:26.370 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.370 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 7, 2024-04-07T17:51:26.370 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:26.370 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.370 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:26.370 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:26.370 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:26.370 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:26.370 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T17:51:26.370 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.370 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 4, 2024-04-07T17:51:26.370 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:26.370 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:26.370 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T17:51:26.371 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:26.371 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:26.371 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:26.371 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:35.981613+0000", 2024-04-07T17:51:26.371 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:38.967913+0000", 2024-04-07T17:51:26.371 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:38.967913+0000", 2024-04-07T17:51:26.371 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:38.968805+0000", 2024-04-07T17:51:26.371 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:35.981613+0000", 2024-04-07T17:51:26.371 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:26.371 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:26.371 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:26.371 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:26.371 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:35.981613+0000", 2024-04-07T17:51:26.371 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:35.981613+0000", 2024-04-07T17:51:26.371 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:35.981613+0000", 2024-04-07T17:51:26.371 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:26.372 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:26.372 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:26.372 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:35.981613+0000", 2024-04-07T17:51:26.372 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:35.981613+0000", 2024-04-07T17:51:26.372 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:26.372 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 1, 2024-04-07T17:51:26.372 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:26.372 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:26.372 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:26.372 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:26.372 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:26.372 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:26.372 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:26.372 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 1, 2024-04-07T17:51:26.372 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:26.373 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:26.373 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:26.373 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.f", 2024-04-07T17:51:26.373 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:26.373 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:26.373 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:26.373 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4919, 2024-04-07T17:51:26.373 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:26.373 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T19:43:23.875589+0000", 2024-04-07T17:51:26.373 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:26.373 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:26.373 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:26.373 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 22, 2024-04-07T17:51:26.373 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:26.373 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:26.373 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:26.374 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:26.374 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:26.374 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:26.374 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:26.374 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:26.374 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:26.374 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:26.374 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:26.374 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:26.374 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:26.374 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:26.374 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:26.374 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 3, 2024-04-07T17:51:26.374 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 1, 2024-04-07T17:51:26.374 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:26.375 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 1, 2024-04-07T17:51:26.375 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:26.375 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:26.375 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:26.375 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:26.375 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:26.375 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:26.375 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:26.375 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:26.375 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:26.375 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:26.375 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:26.375 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:26.375 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:26.375 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 1, 2024-04-07T17:51:26.376 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 1, 2024-04-07T17:51:26.376 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:26.376 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:26.376 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:26.376 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 1, 2024-04-07T17:51:26.376 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 1 2024-04-07T17:51:26.376 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.376 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:26.376 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:26.376 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:26.376 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:26.376 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:26.376 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T17:51:26.376 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.376 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 4, 2024-04-07T17:51:26.376 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "61'1" 2024-04-07T17:51:26.377 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.377 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:26.377 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:26.377 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:26.377 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:26.377 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T17:51:26.377 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.377 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 7, 2024-04-07T17:51:26.377 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:26.377 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:26.377 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:26.377 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:26.377 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:26.377 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:26.377 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.358184+0000", 2024-04-07T17:51:26.378 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:38.987996+0000", 2024-04-07T17:51:26.378 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:38.987996+0000", 2024-04-07T17:51:26.378 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:38.989241+0000", 2024-04-07T17:51:26.378 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.358184+0000", 2024-04-07T17:51:26.378 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.378 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:26.378 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.378 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:26.378 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.358184+0000", 2024-04-07T17:51:26.378 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.358184+0000", 2024-04-07T17:51:26.378 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.358184+0000", 2024-04-07T17:51:26.378 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:26.378 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:26.378 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.379 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.358184+0000", 2024-04-07T17:51:26.379 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.358184+0000", 2024-04-07T17:51:26.379 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:26.379 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:26.379 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:26.379 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:26.379 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:26.379 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:26.379 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:26.379 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:26.379 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:26.379 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:26.379 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:26.379 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:26.379 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:26.379 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.11", 2024-04-07T17:51:26.380 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:26.380 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:26.380 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:26.380 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4801, 2024-04-07T17:51:26.380 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:26.380 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T16:46:00.763122+0000", 2024-04-07T17:51:26.380 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:26.380 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:26.380 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:26.380 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:26.380 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:26.380 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:26.380 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:26.380 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:26.380 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:26.380 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:26.381 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:26.381 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:26.381 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:26.381 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:26.381 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:26.381 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:26.381 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:26.381 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:26.381 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:26.381 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:26.381 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:26.381 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:26.381 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:26.381 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:26.381 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:26.381 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:26.382 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:26.382 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:26.382 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:26.382 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:26.382 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:26.382 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:26.382 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:26.382 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:26.382 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:26.382 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:26.382 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:26.382 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:26.382 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:26.382 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:26.383 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:26.383 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:26.383 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:26.383 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.383 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:26.383 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:26.383 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:26.383 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:26.383 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:26.383 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T17:51:26.383 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.383 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 7, 2024-04-07T17:51:26.383 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:26.383 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.383 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:26.383 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:26.383 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T17:51:26.384 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:26.384 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T17:51:26.384 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.384 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 2, 2024-04-07T17:51:26.384 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:26.384 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:26.384 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T17:51:26.384 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:26.384 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:26.384 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:26.384 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.354070+0000", 2024-04-07T17:51:26.384 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:23:15.875351+0000", 2024-04-07T17:51:26.384 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:23:15.875351+0000", 2024-04-07T17:51:26.384 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:23:15.880710+0000", 2024-04-07T17:51:26.384 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.354070+0000", 2024-04-07T17:51:26.384 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:26.385 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:26.385 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:26.385 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 445, 2024-04-07T17:51:26.385 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.354070+0000", 2024-04-07T17:51:26.385 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.354070+0000", 2024-04-07T17:51:26.385 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.354070+0000", 2024-04-07T17:51:26.385 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:26.385 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:26.385 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:26.385 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.354070+0000", 2024-04-07T17:51:26.385 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.354070+0000", 2024-04-07T17:51:26.385 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:26.385 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 8, 2024-04-07T17:51:26.385 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:26.385 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:26.386 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 444, 2024-04-07T17:51:26.386 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:26.386 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:26.386 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:26.386 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:26.386 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 8, 2024-04-07T17:51:26.386 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:26.386 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:26.386 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:26.386 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.10", 2024-04-07T17:51:26.386 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:26.386 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:26.386 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:26.386 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4979, 2024-04-07T17:51:26.386 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:26.387 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T17:16:20.268788+0000", 2024-04-07T17:51:26.387 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:26.387 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:26.387 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:26.387 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:26.387 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:26.387 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:26.387 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:26.387 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:26.387 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:26.387 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:26.387 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:26.387 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:26.387 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:26.387 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:26.388 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:26.388 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:26.388 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:26.388 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:26.388 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:26.388 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 12, 2024-04-07T17:51:26.388 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 4, 2024-04-07T17:51:26.388 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:26.388 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 4, 2024-04-07T17:51:26.388 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:26.388 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:26.388 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:26.388 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:26.388 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:26.388 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 4, 2024-04-07T17:51:26.388 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:26.388 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:26.389 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:26.389 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:26.389 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:26.389 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:26.389 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:26.389 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 24, 2024-04-07T17:51:26.389 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 24, 2024-04-07T17:51:26.389 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:26.389 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:26.389 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:26.389 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 12, 2024-04-07T17:51:26.389 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:26.389 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.389 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:26.389 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:26.389 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:26.389 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T17:51:26.390 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:26.390 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T17:51:26.390 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.390 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 2, 2024-04-07T17:51:26.390 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "61'8" 2024-04-07T17:51:26.390 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.390 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:26.390 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:26.390 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:26.390 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:26.390 INFO:tasks.workunit.client.0.smithi100.stdout: 5 2024-04-07T17:51:26.390 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.390 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 6, 2024-04-07T17:51:26.390 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:26.390 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:26.391 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:26.391 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:26.391 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:26.391 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:26.391 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.370556+0000", 2024-04-07T17:51:26.391 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:08.963259+0000", 2024-04-07T17:51:26.391 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:08.963259+0000", 2024-04-07T17:51:26.391 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:08.964055+0000", 2024-04-07T17:51:26.391 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.370556+0000", 2024-04-07T17:51:26.391 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.391 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:26.391 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.391 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 554, 2024-04-07T17:51:26.391 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.370556+0000", 2024-04-07T17:51:26.391 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.370556+0000", 2024-04-07T17:51:26.391 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.370556+0000", 2024-04-07T17:51:26.391 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:26.391 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:26.392 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.392 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.370556+0000", 2024-04-07T17:51:26.392 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.370556+0000", 2024-04-07T17:51:26.392 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:26.392 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:26.392 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:26.392 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:26.392 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 553, 2024-04-07T17:51:26.392 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:26.392 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:26.392 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:26.392 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:26.392 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:26.392 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:26.392 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:26.392 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:26.392 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.10", 2024-04-07T17:51:26.393 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:26.393 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:26.393 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:26.393 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4829, 2024-04-07T17:51:26.393 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:26.393 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T23:13:34.886609+0000", 2024-04-07T17:51:26.393 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:26.393 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:26.393 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:26.393 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:26.393 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:26.393 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:26.393 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:26.393 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:26.393 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:26.393 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:26.393 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:26.394 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:26.394 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:26.394 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:26.394 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:26.394 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:26.394 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:26.394 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:26.394 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:26.394 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:26.394 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:26.394 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:26.394 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:26.394 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:26.394 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:26.394 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:26.395 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:26.395 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:26.395 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:26.395 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:26.395 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:26.395 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:26.395 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:26.395 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:26.395 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:26.395 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:26.395 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:26.395 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:26.395 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:26.395 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:26.395 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:26.396 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:26.396 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:26.396 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.396 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:26.396 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:26.396 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:26.396 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:26.396 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:26.396 INFO:tasks.workunit.client.0.smithi100.stdout: 5 2024-04-07T17:51:26.396 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.396 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 6, 2024-04-07T17:51:26.396 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:26.396 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.396 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:26.396 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:26.397 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:26.397 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:26.397 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T17:51:26.397 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.397 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 6, 2024-04-07T17:51:26.397 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:26.397 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:26.397 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T17:51:26.397 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:26.397 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:26.397 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:26.397 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.371043+0000", 2024-04-07T17:51:26.397 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:08.965799+0000", 2024-04-07T17:51:26.397 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:08.965799+0000", 2024-04-07T17:51:26.397 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:08.966301+0000", 2024-04-07T17:51:26.397 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.371043+0000", 2024-04-07T17:51:26.397 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:26.398 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:26.398 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:26.398 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 554, 2024-04-07T17:51:26.398 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.371043+0000", 2024-04-07T17:51:26.398 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.371043+0000", 2024-04-07T17:51:26.398 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.371043+0000", 2024-04-07T17:51:26.398 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:26.398 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:26.398 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:26.398 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.371043+0000", 2024-04-07T17:51:26.398 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.371043+0000", 2024-04-07T17:51:26.398 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:26.398 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:26.398 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:26.398 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:26.399 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 553, 2024-04-07T17:51:26.399 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:26.399 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:26.399 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:26.399 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:26.399 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:26.399 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:26.399 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:26.399 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:26.399 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.11", 2024-04-07T17:51:26.399 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:26.399 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:26.399 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:26.399 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4845, 2024-04-07T17:51:26.399 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:26.399 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T21:12:09.705565+0000", 2024-04-07T17:51:26.399 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:26.400 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:26.400 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:26.400 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:26.400 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:26.400 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:26.400 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:26.400 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:26.400 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:26.400 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:26.400 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:26.400 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:26.400 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:26.400 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:26.400 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:26.400 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:26.400 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:26.400 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:26.401 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:26.401 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:26.401 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:26.401 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:26.401 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:26.401 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:26.401 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:26.401 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:26.401 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:26.401 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:26.401 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:26.401 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:26.401 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:26.401 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:26.401 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:26.401 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:26.401 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:26.402 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:26.402 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:26.402 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:26.402 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:26.402 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:26.402 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:26.402 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:26.402 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:26.402 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.402 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:26.402 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:26.402 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:26.402 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:26.402 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:26.402 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T17:51:26.402 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.403 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 6, 2024-04-07T17:51:26.403 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:26.403 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.403 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:26.403 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:26.403 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:26.403 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:26.403 INFO:tasks.workunit.client.0.smithi100.stdout: 2 2024-04-07T17:51:26.403 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.403 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 7, 2024-04-07T17:51:26.403 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:26.403 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:26.403 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:26.403 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:26.403 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:26.403 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:26.403 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.368748+0000", 2024-04-07T17:51:26.404 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:38.987994+0000", 2024-04-07T17:51:26.404 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:38.987994+0000", 2024-04-07T17:51:26.404 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:38.991515+0000", 2024-04-07T17:51:26.404 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.368748+0000", 2024-04-07T17:51:26.404 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.404 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:26.404 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.404 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:26.404 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.368748+0000", 2024-04-07T17:51:26.404 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.368748+0000", 2024-04-07T17:51:26.404 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.368748+0000", 2024-04-07T17:51:26.404 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:26.404 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:26.404 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.404 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.368748+0000", 2024-04-07T17:51:26.404 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.368748+0000", 2024-04-07T17:51:26.404 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:26.405 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:26.405 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:26.405 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:26.405 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:26.405 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:26.405 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:26.405 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:26.405 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:26.405 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:26.405 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:26.405 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:26.405 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:26.405 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.13", 2024-04-07T17:51:26.405 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:26.405 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:26.405 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:26.405 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4789, 2024-04-07T17:51:26.406 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:26.406 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T20:51:08.365772+0000", 2024-04-07T17:51:26.406 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:26.406 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:26.406 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:26.406 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:26.406 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:26.406 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:26.406 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:26.406 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:26.406 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:26.406 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:26.406 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:26.406 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:26.406 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:26.407 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:26.407 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:26.407 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:26.407 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:26.407 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:26.407 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:26.407 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:26.407 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:26.407 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:26.407 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:26.407 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:26.407 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:26.407 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:26.407 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:26.407 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:26.407 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:26.408 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:26.408 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:26.408 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:26.408 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:26.408 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:26.408 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:26.408 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:26.408 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:26.408 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:26.408 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:26.408 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:26.408 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:26.408 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:26.408 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:26.408 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.409 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:26.409 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:26.409 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:26.409 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:26.409 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:26.409 INFO:tasks.workunit.client.0.smithi100.stdout: 2 2024-04-07T17:51:26.409 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.409 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 7, 2024-04-07T17:51:26.409 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:26.409 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.409 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:26.409 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:26.409 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:26.409 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:26.409 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T17:51:26.409 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.410 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 5, 2024-04-07T17:51:26.410 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:26.410 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:26.410 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T17:51:26.410 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:26.410 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:26.410 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:26.410 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.369416+0000", 2024-04-07T17:51:26.410 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:38.978180+0000", 2024-04-07T17:51:26.410 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:38.978180+0000", 2024-04-07T17:51:26.410 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:38.979026+0000", 2024-04-07T17:51:26.410 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.369416+0000", 2024-04-07T17:51:26.410 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:26.410 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:26.411 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:26.411 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:26.411 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.369416+0000", 2024-04-07T17:51:26.411 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.369416+0000", 2024-04-07T17:51:26.411 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.369416+0000", 2024-04-07T17:51:26.411 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:26.411 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:26.411 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:26.411 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.369416+0000", 2024-04-07T17:51:26.411 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.369416+0000", 2024-04-07T17:51:26.411 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:26.411 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:26.411 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:26.411 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:26.411 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:26.411 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:26.411 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:26.412 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:26.412 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:26.412 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:26.412 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:26.412 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:26.412 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:26.412 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.12", 2024-04-07T17:51:26.412 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:26.412 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:26.412 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:26.412 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4877, 2024-04-07T17:51:26.412 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:26.412 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T01:41:45.196090+0000", 2024-04-07T17:51:26.412 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:26.412 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:26.413 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:26.413 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:26.413 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:26.413 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:26.413 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:26.413 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:26.413 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:26.413 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:26.413 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:26.413 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:26.413 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:26.413 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:26.413 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:26.413 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:26.413 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:26.413 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:26.414 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:26.414 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:26.414 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:26.414 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:26.414 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:26.414 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:26.414 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:26.414 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:26.414 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:26.414 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:26.414 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:26.414 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:26.414 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:26.414 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:26.415 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:26.415 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:26.415 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:26.415 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:26.415 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:26.415 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:26.415 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:26.415 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:26.415 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:26.415 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:26.415 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:26.415 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.415 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:26.415 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:26.415 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:26.415 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:26.416 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:26.416 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T17:51:26.416 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.416 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 5, 2024-04-07T17:51:26.416 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:26.416 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.416 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:26.416 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:26.416 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T17:51:26.416 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:26.416 INFO:tasks.workunit.client.0.smithi100.stdout: 3 2024-04-07T17:51:26.416 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.416 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 2, 2024-04-07T17:51:26.416 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:26.416 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:26.417 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:26.417 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:26.417 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:26.417 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:26.417 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.354815+0000", 2024-04-07T17:51:26.417 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:38.969402+0000", 2024-04-07T17:51:26.417 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:38.969402+0000", 2024-04-07T17:51:26.417 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:38.970054+0000", 2024-04-07T17:51:26.417 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.354815+0000", 2024-04-07T17:51:26.417 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.417 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:26.417 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.417 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:26.417 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.354815+0000", 2024-04-07T17:51:26.417 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.354815+0000", 2024-04-07T17:51:26.418 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.354815+0000", 2024-04-07T17:51:26.418 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:26.418 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:26.418 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.418 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.354815+0000", 2024-04-07T17:51:26.418 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.354815+0000", 2024-04-07T17:51:26.418 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:26.418 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:26.418 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:26.418 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:26.418 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:26.418 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:26.418 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:26.418 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:26.418 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:26.418 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:26.419 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:26.419 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:26.419 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:26.419 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.12", 2024-04-07T17:51:26.419 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:26.419 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:26.419 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:26.419 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4924, 2024-04-07T17:51:26.419 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:26.419 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T23:11:21.401536+0000", 2024-04-07T17:51:26.419 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:26.419 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:26.419 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:26.419 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:26.419 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:26.419 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:26.420 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:26.420 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:26.420 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:26.420 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:26.420 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:26.420 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:26.420 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:26.420 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:26.420 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:26.420 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:26.420 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:26.420 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:26.420 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:26.420 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:26.420 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:26.421 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:26.421 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:26.421 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:26.421 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:26.421 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:26.421 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:26.421 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:26.421 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:26.421 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:26.421 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:26.421 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:26.421 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:26.421 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:26.421 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:26.421 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:26.421 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:26.421 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:26.422 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:26.422 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:26.422 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:26.422 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:26.422 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:26.422 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.422 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:26.422 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:26.422 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:26.422 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T17:51:26.422 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:26.422 INFO:tasks.workunit.client.0.smithi100.stdout: 3 2024-04-07T17:51:26.422 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.422 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 2, 2024-04-07T17:51:26.422 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:26.423 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.423 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:26.423 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:26.423 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:26.423 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:26.423 INFO:tasks.workunit.client.0.smithi100.stdout: 3 2024-04-07T17:51:26.423 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.423 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 0, 2024-04-07T17:51:26.423 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:26.423 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:26.423 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T17:51:26.423 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:26.423 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:26.423 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:26.423 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:35.988325+0000", 2024-04-07T17:51:26.423 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:24:13.963512+0000", 2024-04-07T17:51:26.423 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:24:13.963512+0000", 2024-04-07T17:51:26.424 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:24:13.964689+0000", 2024-04-07T17:51:26.424 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:35.988325+0000", 2024-04-07T17:51:26.424 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:26.424 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:26.424 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:26.424 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 500, 2024-04-07T17:51:26.424 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:35.988325+0000", 2024-04-07T17:51:26.424 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:35.988325+0000", 2024-04-07T17:51:26.424 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:35.988325+0000", 2024-04-07T17:51:26.424 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:26.424 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:26.424 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:26.424 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:35.988325+0000", 2024-04-07T17:51:26.424 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:35.988325+0000", 2024-04-07T17:51:26.424 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:26.424 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 2, 2024-04-07T17:51:26.425 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:26.425 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:26.425 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 499, 2024-04-07T17:51:26.425 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:26.425 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:26.425 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:26.425 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:26.425 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 2, 2024-04-07T17:51:26.425 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:26.425 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:26.425 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:26.425 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.13", 2024-04-07T17:51:26.425 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:26.425 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:26.425 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:26.425 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 5000, 2024-04-07T17:51:26.426 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:26.426 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T16:15:37.955732+0000", 2024-04-07T17:51:26.426 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:26.426 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:26.426 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:26.426 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 34, 2024-04-07T17:51:26.426 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:26.426 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:26.426 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:26.426 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:26.426 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:26.426 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:26.426 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:26.426 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:26.426 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:26.427 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:26.427 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:26.427 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:26.427 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:26.427 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:26.427 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:26.427 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 6, 2024-04-07T17:51:26.427 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 2, 2024-04-07T17:51:26.427 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:26.427 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 2, 2024-04-07T17:51:26.427 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:26.427 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:26.427 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:26.427 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:26.427 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:26.427 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 1, 2024-04-07T17:51:26.428 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:26.428 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:26.428 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:26.428 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:26.428 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:26.428 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:26.428 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:26.428 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 4, 2024-04-07T17:51:26.428 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 8, 2024-04-07T17:51:26.428 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:26.428 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:26.428 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:26.428 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 3, 2024-04-07T17:51:26.428 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 7 2024-04-07T17:51:26.428 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.428 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:26.428 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:26.429 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:26.429 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:26.429 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:26.429 INFO:tasks.workunit.client.0.smithi100.stdout: 3 2024-04-07T17:51:26.429 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.429 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 0, 2024-04-07T17:51:26.429 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "61'2" 2024-04-07T17:51:26.429 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.429 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:26.429 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:26.429 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:26.429 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:26.429 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T17:51:26.429 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.429 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 7, 2024-04-07T17:51:26.429 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:26.430 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:26.430 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:26.430 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:26.430 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:26.430 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:26.430 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.372482+0000", 2024-04-07T17:51:26.430 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:38.987197+0000", 2024-04-07T17:51:26.430 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:38.987197+0000", 2024-04-07T17:51:26.430 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:38.999803+0000", 2024-04-07T17:51:26.430 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.372482+0000", 2024-04-07T17:51:26.430 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.430 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:26.430 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.430 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:26.431 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.372482+0000", 2024-04-07T17:51:26.431 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.372482+0000", 2024-04-07T17:51:26.431 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.372482+0000", 2024-04-07T17:51:26.431 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:26.431 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:26.431 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.431 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.372482+0000", 2024-04-07T17:51:26.431 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.372482+0000", 2024-04-07T17:51:26.431 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:26.431 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:26.431 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:26.431 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:26.431 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:26.431 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:26.432 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:26.432 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:26.432 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:26.432 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:26.432 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:26.432 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:26.432 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:26.432 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.15", 2024-04-07T17:51:26.432 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:26.432 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:26.432 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:26.432 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4809, 2024-04-07T17:51:26.432 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:26.432 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T03:29:56.940037+0000", 2024-04-07T17:51:26.432 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:26.432 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:26.432 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:26.433 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:26.433 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:26.433 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:26.433 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:26.433 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:26.433 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:26.433 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:26.433 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:26.433 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:26.433 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:26.433 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:26.433 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:26.433 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:26.433 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:26.433 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:26.433 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:26.434 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:26.434 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:26.434 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:26.434 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:26.434 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:26.434 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:26.434 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:26.434 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:26.434 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:26.434 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:26.434 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:26.434 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:26.434 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:26.434 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:26.435 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:26.435 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:26.435 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:26.435 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:26.435 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:26.435 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:26.435 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:26.435 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:26.435 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:26.435 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:26.435 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.435 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:26.435 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:26.435 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:26.435 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:26.435 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:26.436 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T17:51:26.436 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.436 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 7, 2024-04-07T17:51:26.436 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:26.436 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.436 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:26.436 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:26.436 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:26.436 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:26.436 INFO:tasks.workunit.client.0.smithi100.stdout: 5 2024-04-07T17:51:26.436 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.436 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 6, 2024-04-07T17:51:26.436 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:26.436 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:26.437 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T17:51:26.437 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:26.437 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:26.437 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:26.437 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.370811+0000", 2024-04-07T17:51:26.437 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:08.975701+0000", 2024-04-07T17:51:26.437 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:08.975701+0000", 2024-04-07T17:51:26.437 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:08.977082+0000", 2024-04-07T17:51:26.437 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.370811+0000", 2024-04-07T17:51:26.437 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:26.437 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:26.437 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:26.437 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 554, 2024-04-07T17:51:26.437 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.370811+0000", 2024-04-07T17:51:26.437 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.370811+0000", 2024-04-07T17:51:26.437 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.370811+0000", 2024-04-07T17:51:26.437 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:26.438 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:26.438 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:26.438 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.370811+0000", 2024-04-07T17:51:26.438 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.370811+0000", 2024-04-07T17:51:26.438 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:26.438 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 29, 2024-04-07T17:51:26.438 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:26.438 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:26.438 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 553, 2024-04-07T17:51:26.438 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:26.438 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:26.438 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:26.438 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:26.438 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 29, 2024-04-07T17:51:26.439 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:26.439 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:26.439 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:26.439 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.14", 2024-04-07T17:51:26.439 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:26.439 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:26.439 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:26.439 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4865, 2024-04-07T17:51:26.439 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:26.439 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T20:48:03.443222+0000", 2024-04-07T17:51:26.439 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:26.439 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:26.439 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:26.439 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 5824, 2024-04-07T17:51:26.439 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:26.439 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:26.439 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:26.440 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:26.440 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:26.440 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:26.440 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:26.440 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:26.440 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:26.440 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:26.440 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:26.440 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:26.440 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:26.440 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:26.440 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:26.440 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 12, 2024-04-07T17:51:26.440 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 4, 2024-04-07T17:51:26.440 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:26.440 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 4, 2024-04-07T17:51:26.441 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:26.441 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:26.441 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:26.441 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:26.441 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:26.441 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 1, 2024-04-07T17:51:26.441 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:26.441 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:26.441 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:26.441 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:26.441 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:26.441 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:26.441 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:26.441 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 10, 2024-04-07T17:51:26.441 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 14, 2024-04-07T17:51:26.441 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:26.441 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:26.442 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:26.442 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 28, 2024-04-07T17:51:26.442 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 25 2024-04-07T17:51:26.442 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.442 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:26.442 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:26.442 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:26.442 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:26.442 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:26.442 INFO:tasks.workunit.client.0.smithi100.stdout: 5 2024-04-07T17:51:26.442 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.442 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 6, 2024-04-07T17:51:26.442 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "623'29" 2024-04-07T17:51:26.442 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.442 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:26.443 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:26.443 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:26.443 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:26.443 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T17:51:26.443 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.443 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 4, 2024-04-07T17:51:26.443 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:26.443 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:26.443 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:26.443 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:26.443 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:26.443 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:26.443 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:35.981022+0000", 2024-04-07T17:51:26.443 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:38.985283+0000", 2024-04-07T17:51:26.443 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:38.985283+0000", 2024-04-07T17:51:26.443 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:38.986153+0000", 2024-04-07T17:51:26.443 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:35.981022+0000", 2024-04-07T17:51:26.444 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.444 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:26.444 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.444 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:26.444 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:35.981022+0000", 2024-04-07T17:51:26.444 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:35.981022+0000", 2024-04-07T17:51:26.444 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:35.981022+0000", 2024-04-07T17:51:26.444 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:26.444 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:26.444 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.444 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:35.981022+0000", 2024-04-07T17:51:26.444 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:35.981022+0000", 2024-04-07T17:51:26.444 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:26.444 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:26.444 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:26.444 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:26.445 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:26.445 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:26.445 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:26.445 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:26.445 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:26.445 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:26.445 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:26.445 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:26.445 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:26.445 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.14", 2024-04-07T17:51:26.445 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:26.445 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:26.445 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:26.445 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4907, 2024-04-07T17:51:26.445 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:26.446 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T22:16:21.117705+0000", 2024-04-07T17:51:26.446 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:26.446 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:26.446 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:26.446 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:26.446 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:26.446 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:26.446 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:26.446 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:26.446 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:26.446 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:26.446 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:26.446 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:26.446 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:26.447 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:26.447 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:26.447 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:26.447 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:26.447 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:26.447 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:26.447 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:26.447 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:26.447 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:26.447 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:26.447 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:26.447 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:26.447 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:26.447 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:26.447 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:26.448 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:26.448 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:26.448 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:26.448 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:26.448 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:26.448 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:26.448 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:26.448 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:26.448 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:26.448 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:26.448 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:26.448 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:26.448 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:26.448 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:26.448 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:26.448 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.448 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:26.449 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:26.449 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:26.449 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:26.449 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:26.449 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T17:51:26.449 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.449 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 4, 2024-04-07T17:51:26.449 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:26.449 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.449 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:26.449 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:26.449 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:26.449 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:26.449 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T17:51:26.449 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.449 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 1, 2024-04-07T17:51:26.450 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:26.450 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:26.450 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T17:51:26.450 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:26.450 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:26.450 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:26.450 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.332631+0000", 2024-04-07T17:51:26.450 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:24:41.527553+0000", 2024-04-07T17:51:26.450 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:24:41.527553+0000", 2024-04-07T17:51:26.450 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:24:41.530429+0000", 2024-04-07T17:51:26.450 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.332631+0000", 2024-04-07T17:51:26.450 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:26.450 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:26.450 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:26.451 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 527, 2024-04-07T17:51:26.451 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.332631+0000", 2024-04-07T17:51:26.451 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.332631+0000", 2024-04-07T17:51:26.451 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.332631+0000", 2024-04-07T17:51:26.451 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:26.451 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:26.451 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:26.451 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.332631+0000", 2024-04-07T17:51:26.451 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.332631+0000", 2024-04-07T17:51:26.451 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:26.451 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 3, 2024-04-07T17:51:26.451 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:26.451 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:26.451 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 526, 2024-04-07T17:51:26.451 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:26.451 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:26.452 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:26.452 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:26.452 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 3, 2024-04-07T17:51:26.452 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:26.452 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:26.452 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:26.452 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.15", 2024-04-07T17:51:26.452 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:26.452 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:26.452 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:26.452 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4990, 2024-04-07T17:51:26.452 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:26.452 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T16:34:47.703047+0000", 2024-04-07T17:51:26.452 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:26.452 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:26.452 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:26.452 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 90, 2024-04-07T17:51:26.453 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:26.453 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:26.453 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:26.453 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:26.453 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:26.453 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:26.453 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:26.453 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:26.453 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:26.453 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:26.453 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:26.453 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:26.453 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:26.453 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:26.453 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:26.453 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 6, 2024-04-07T17:51:26.453 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 2, 2024-04-07T17:51:26.454 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:26.454 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 2, 2024-04-07T17:51:26.454 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:26.454 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:26.454 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:26.454 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:26.454 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:26.454 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 1, 2024-04-07T17:51:26.454 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:26.454 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:26.454 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:26.454 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:26.454 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:26.454 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:26.454 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:26.455 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 7, 2024-04-07T17:51:26.455 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 7, 2024-04-07T17:51:26.455 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:26.455 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:26.455 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:26.455 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 4, 2024-04-07T17:51:26.455 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 1 2024-04-07T17:51:26.455 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.455 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:26.455 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:26.455 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:26.455 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:26.455 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:26.455 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T17:51:26.455 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.456 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 1, 2024-04-07T17:51:26.456 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "61'3" 2024-04-07T17:51:26.456 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.456 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:26.456 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:26.456 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T17:51:26.456 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:26.456 INFO:tasks.workunit.client.0.smithi100.stdout: 3 2024-04-07T17:51:26.456 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.456 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 2, 2024-04-07T17:51:26.456 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:26.456 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:26.456 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:26.456 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:26.456 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:26.456 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:26.457 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.363660+0000", 2024-04-07T17:51:26.457 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:24:41.558300+0000", 2024-04-07T17:51:26.457 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:24:41.558300+0000", 2024-04-07T17:51:26.457 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:24:41.558648+0000", 2024-04-07T17:51:26.457 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.363660+0000", 2024-04-07T17:51:26.457 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.457 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:26.457 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.457 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 527, 2024-04-07T17:51:26.457 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.363660+0000", 2024-04-07T17:51:26.457 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.363660+0000", 2024-04-07T17:51:26.457 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.363660+0000", 2024-04-07T17:51:26.457 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:26.457 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:26.457 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.457 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.363660+0000", 2024-04-07T17:51:26.457 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.363660+0000", 2024-04-07T17:51:26.458 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:26.458 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:26.458 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:26.458 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:26.458 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 526, 2024-04-07T17:51:26.458 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:26.458 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:26.458 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:26.458 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:26.458 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:26.458 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:26.458 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:26.458 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:26.458 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.17", 2024-04-07T17:51:26.458 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:26.459 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:26.459 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:26.459 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4923, 2024-04-07T17:51:26.459 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:26.459 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T01:44:12.981988+0000", 2024-04-07T17:51:26.459 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:26.459 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:26.459 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:26.459 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:26.459 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:26.459 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:26.459 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:26.459 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:26.459 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:26.459 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:26.459 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:26.459 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:26.460 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:26.460 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:26.460 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:26.460 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:26.460 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:26.460 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:26.460 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:26.460 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:26.460 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:26.460 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:26.460 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:26.460 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:26.460 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:26.460 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:26.460 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:26.460 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:26.460 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:26.461 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:26.461 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:26.461 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:26.461 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:26.461 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:26.461 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:26.461 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:26.461 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:26.461 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:26.461 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:26.461 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:26.461 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:26.461 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:26.461 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:26.461 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.462 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:26.462 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:26.462 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:26.462 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T17:51:26.462 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:26.462 INFO:tasks.workunit.client.0.smithi100.stdout: 3 2024-04-07T17:51:26.462 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.462 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 2, 2024-04-07T17:51:26.462 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:26.462 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.462 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:26.462 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:26.462 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:26.462 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:26.463 INFO:tasks.workunit.client.0.smithi100.stdout: 2 2024-04-07T17:51:26.463 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.463 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 5, 2024-04-07T17:51:26.463 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:26.463 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:26.463 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T17:51:26.463 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:26.463 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:26.463 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:26.463 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.368702+0000", 2024-04-07T17:51:26.463 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:08.971659+0000", 2024-04-07T17:51:26.463 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:08.971659+0000", 2024-04-07T17:51:26.463 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:08.976930+0000", 2024-04-07T17:51:26.463 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.368702+0000", 2024-04-07T17:51:26.463 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:26.463 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:26.464 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:26.464 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 554, 2024-04-07T17:51:26.464 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.368702+0000", 2024-04-07T17:51:26.464 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.368702+0000", 2024-04-07T17:51:26.464 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.368702+0000", 2024-04-07T17:51:26.464 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:26.464 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:26.464 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:26.464 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.368702+0000", 2024-04-07T17:51:26.464 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.368702+0000", 2024-04-07T17:51:26.464 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:26.464 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:26.464 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:26.464 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:26.464 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 553, 2024-04-07T17:51:26.465 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:26.465 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:26.465 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:26.465 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:26.465 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:26.465 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:26.465 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:26.465 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:26.465 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.16", 2024-04-07T17:51:26.465 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:26.465 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:26.465 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:26.465 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4848, 2024-04-07T17:51:26.465 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:26.465 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T02:13:26.493523+0000", 2024-04-07T17:51:26.466 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:26.466 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:26.466 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:26.466 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:26.466 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:26.466 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:26.466 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:26.466 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:26.466 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:26.466 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:26.466 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:26.466 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:26.466 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:26.466 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:26.466 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:26.467 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:26.467 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:26.467 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:26.467 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:26.467 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:26.467 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:26.467 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:26.467 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:26.467 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:26.467 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:26.467 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:26.467 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:26.467 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:26.467 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:26.467 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:26.467 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:26.468 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:26.468 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:26.468 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:26.468 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:26.468 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:26.468 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:26.468 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:26.468 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:26.468 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:26.468 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:26.468 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:26.468 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:26.468 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.468 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:26.469 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:26.469 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:26.469 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:26.469 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:26.469 INFO:tasks.workunit.client.0.smithi100.stdout: 2 2024-04-07T17:51:26.469 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.469 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 5, 2024-04-07T17:51:26.469 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:26.469 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.469 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:26.469 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:26.469 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:26.469 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:26.469 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T17:51:26.469 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.469 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 5, 2024-04-07T17:51:26.470 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:26.470 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:26.470 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:26.470 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:26.470 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:26.470 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:26.470 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.369127+0000", 2024-04-07T17:51:26.470 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:38.977859+0000", 2024-04-07T17:51:26.470 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:38.977859+0000", 2024-04-07T17:51:26.470 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:38.978447+0000", 2024-04-07T17:51:26.470 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.369127+0000", 2024-04-07T17:51:26.470 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.470 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:26.470 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.471 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:26.471 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.369127+0000", 2024-04-07T17:51:26.471 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.369127+0000", 2024-04-07T17:51:26.471 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.369127+0000", 2024-04-07T17:51:26.471 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:26.471 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:26.471 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.471 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.369127+0000", 2024-04-07T17:51:26.471 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.369127+0000", 2024-04-07T17:51:26.471 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:26.471 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:26.471 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:26.471 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:26.471 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:26.471 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:26.471 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:26.472 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:26.472 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:26.472 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:26.472 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:26.472 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:26.472 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:26.472 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.16", 2024-04-07T17:51:26.472 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:26.472 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:26.472 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:26.472 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4860, 2024-04-07T17:51:26.472 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:26.472 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T16:22:51.136501+0000", 2024-04-07T17:51:26.472 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:26.472 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:26.472 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:26.473 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:26.473 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:26.473 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:26.473 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:26.473 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:26.473 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:26.473 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:26.473 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:26.473 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:26.473 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:26.473 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:26.473 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:26.473 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:26.473 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:26.473 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:26.473 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:26.473 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:26.474 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:26.474 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:26.474 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:26.474 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:26.474 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:26.474 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:26.474 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:26.474 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:26.474 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:26.474 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:26.474 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:26.474 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:26.474 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:26.474 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:26.474 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:26.475 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:26.475 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:26.475 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:26.475 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:26.475 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:26.475 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:26.475 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:26.475 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:26.475 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.475 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:26.475 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:26.475 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:26.475 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:26.475 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:26.475 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T17:51:26.475 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.476 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 5, 2024-04-07T17:51:26.476 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:26.476 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.476 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:26.476 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:26.476 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:26.476 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:26.476 INFO:tasks.workunit.client.0.smithi100.stdout: 2 2024-04-07T17:51:26.476 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.476 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 6, 2024-04-07T17:51:26.476 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:26.476 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:26.476 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T17:51:26.476 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:26.476 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:26.476 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:26.477 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.370115+0000", 2024-04-07T17:51:26.477 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:08.977704+0000", 2024-04-07T17:51:26.477 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:08.977704+0000", 2024-04-07T17:51:26.477 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:08.978095+0000", 2024-04-07T17:51:26.477 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.370115+0000", 2024-04-07T17:51:26.477 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:26.477 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:26.477 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:26.477 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 554, 2024-04-07T17:51:26.477 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.370115+0000", 2024-04-07T17:51:26.477 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.370115+0000", 2024-04-07T17:51:26.477 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.370115+0000", 2024-04-07T17:51:26.477 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:26.477 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:26.477 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:26.477 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.370115+0000", 2024-04-07T17:51:26.477 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.370115+0000", 2024-04-07T17:51:26.478 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:26.478 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:26.478 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:26.478 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:26.478 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 553, 2024-04-07T17:51:26.478 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:26.478 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:26.478 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:26.478 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:26.478 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:26.478 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:26.478 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:26.478 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:26.478 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.17", 2024-04-07T17:51:26.478 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:26.478 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:26.479 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:26.479 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4829, 2024-04-07T17:51:26.479 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:26.479 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T18:47:28.916356+0000", 2024-04-07T17:51:26.479 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:26.479 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:26.479 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:26.479 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:26.479 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:26.479 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:26.479 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:26.479 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:26.479 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:26.479 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:26.479 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:26.479 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:26.479 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:26.480 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:26.480 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:26.480 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:26.480 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:26.480 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:26.480 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:26.480 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:26.480 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:26.480 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:26.480 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:26.480 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:26.480 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:26.480 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:26.480 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:26.480 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:26.481 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:26.481 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:26.481 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:26.481 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:26.481 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:26.481 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:26.481 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:26.481 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:26.481 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:26.481 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:26.481 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:26.481 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:26.481 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:26.481 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:26.481 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:26.482 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.482 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:26.482 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:26.482 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:26.482 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:26.482 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:26.482 INFO:tasks.workunit.client.0.smithi100.stdout: 2 2024-04-07T17:51:26.482 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.482 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 6, 2024-04-07T17:51:26.482 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:26.482 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.482 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:26.482 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:26.482 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:26.482 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:26.483 INFO:tasks.workunit.client.0.smithi100.stdout: 4 2024-04-07T17:51:26.483 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.483 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 1, 2024-04-07T17:51:26.483 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:26.483 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:26.483 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:26.483 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:26.483 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:26.483 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:26.483 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.331404+0000", 2024-04-07T17:51:26.483 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:24:13.932694+0000", 2024-04-07T17:51:26.483 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:24:13.932694+0000", 2024-04-07T17:51:26.483 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:24:13.933594+0000", 2024-04-07T17:51:26.483 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.331404+0000", 2024-04-07T17:51:26.483 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.484 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:26.484 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.484 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 500, 2024-04-07T17:51:26.484 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.331404+0000", 2024-04-07T17:51:26.484 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.331404+0000", 2024-04-07T17:51:26.484 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.331404+0000", 2024-04-07T17:51:26.484 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:26.484 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:26.484 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.484 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.331404+0000", 2024-04-07T17:51:26.484 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.331404+0000", 2024-04-07T17:51:26.484 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:26.484 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:26.484 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:26.484 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:26.485 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 499, 2024-04-07T17:51:26.485 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:26.485 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:26.485 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:26.485 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:26.485 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:26.485 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:26.485 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:26.485 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:26.485 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.19", 2024-04-07T17:51:26.485 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:26.485 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:26.485 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:26.485 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4977, 2024-04-07T17:51:26.485 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:26.486 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T17:08:30.909773+0000", 2024-04-07T17:51:26.486 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:26.486 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:26.486 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:26.486 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:26.486 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:26.486 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:26.486 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:26.486 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:26.486 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:26.486 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:26.486 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:26.486 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:26.486 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:26.486 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:26.487 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:26.487 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:26.487 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:26.487 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:26.487 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:26.487 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:26.487 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:26.487 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:26.487 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:26.487 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:26.487 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:26.487 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:26.487 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:26.487 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:26.487 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:26.488 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:26.488 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:26.488 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:26.488 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:26.488 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:26.488 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:26.488 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:26.488 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:26.488 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:26.488 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:26.488 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:26.488 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:26.488 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:26.488 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:26.488 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.489 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:26.489 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:26.489 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:26.489 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:26.489 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:26.489 INFO:tasks.workunit.client.0.smithi100.stdout: 4 2024-04-07T17:51:26.489 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.489 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 1, 2024-04-07T17:51:26.489 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:26.489 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.489 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:26.489 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:26.489 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:26.489 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:26.489 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T17:51:26.489 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.489 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 5, 2024-04-07T17:51:26.490 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:26.490 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:26.490 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T17:51:26.490 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:26.490 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:26.490 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:26.490 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.361600+0000", 2024-04-07T17:51:26.490 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:38.990134+0000", 2024-04-07T17:51:26.490 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:38.990134+0000", 2024-04-07T17:51:26.490 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:38.990885+0000", 2024-04-07T17:51:26.490 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.361600+0000", 2024-04-07T17:51:26.490 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:26.490 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:26.490 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:26.491 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:26.491 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.361600+0000", 2024-04-07T17:51:26.491 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.361600+0000", 2024-04-07T17:51:26.491 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.361600+0000", 2024-04-07T17:51:26.491 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:26.491 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:26.491 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:26.491 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.361600+0000", 2024-04-07T17:51:26.491 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.361600+0000", 2024-04-07T17:51:26.491 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:26.491 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:26.491 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:26.491 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:26.491 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:26.491 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:26.491 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:26.491 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:26.492 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:26.492 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:26.492 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:26.492 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:26.492 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:26.492 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.18", 2024-04-07T17:51:26.492 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:26.492 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:26.492 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:26.492 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4861, 2024-04-07T17:51:26.492 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:26.492 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T19:33:03.967565+0000", 2024-04-07T17:51:26.492 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:26.492 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:26.492 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:26.492 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:26.493 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:26.493 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:26.493 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:26.493 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:26.493 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:26.493 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:26.493 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:26.493 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:26.493 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:26.493 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:26.493 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:26.493 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:26.493 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:26.493 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:26.493 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:26.493 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:26.494 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:26.494 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:26.494 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:26.494 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:26.494 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:26.494 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:26.494 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:26.494 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:26.494 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:26.494 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:26.494 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:26.494 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:26.494 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:26.494 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:26.494 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:26.495 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:26.495 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:26.495 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:26.495 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:26.495 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:26.495 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:26.495 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:26.495 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:26.495 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.495 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:26.495 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:26.495 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:26.495 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:26.495 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:26.495 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T17:51:26.495 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.496 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 5, 2024-04-07T17:51:26.496 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:26.496 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.496 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:26.496 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:26.496 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:26.496 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:26.496 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T17:51:26.496 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.496 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 3, 2024-04-07T17:51:26.496 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:26.496 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:26.496 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:26.496 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:26.497 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:26.497 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:26.497 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:35.975334+0000", 2024-04-07T17:51:26.497 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:23:43.903341+0000", 2024-04-07T17:51:26.497 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:23:43.903341+0000", 2024-04-07T17:51:26.497 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:23:43.904215+0000", 2024-04-07T17:51:26.497 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:35.975334+0000", 2024-04-07T17:51:26.497 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.497 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:26.497 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.497 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 471, 2024-04-07T17:51:26.497 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:35.975334+0000", 2024-04-07T17:51:26.497 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:35.975334+0000", 2024-04-07T17:51:26.497 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:35.975334+0000", 2024-04-07T17:51:26.497 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:26.497 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:26.498 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.498 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:35.975334+0000", 2024-04-07T17:51:26.498 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:35.975334+0000", 2024-04-07T17:51:26.498 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:26.498 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:26.498 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:26.498 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:26.498 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 470, 2024-04-07T17:51:26.498 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:26.498 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:26.498 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:26.498 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:26.498 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:26.498 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:26.498 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:26.499 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:26.499 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.18", 2024-04-07T17:51:26.499 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:26.499 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:26.499 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:26.499 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4969, 2024-04-07T17:51:26.499 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:26.499 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T23:54:23.660433+0000", 2024-04-07T17:51:26.499 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:26.499 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:26.499 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:26.499 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:26.499 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:26.499 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:26.499 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:26.499 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:26.499 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:26.500 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:26.500 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:26.500 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:26.500 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:26.500 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:26.500 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:26.500 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:26.500 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:26.500 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:26.500 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:26.500 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:26.500 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:26.500 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:26.500 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:26.500 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:26.500 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:26.500 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:26.501 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:26.501 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:26.501 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:26.501 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:26.501 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:26.501 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:26.501 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:26.501 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:26.501 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:26.501 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:26.501 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:26.501 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:26.501 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:26.501 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:26.501 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:26.501 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:26.501 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:26.502 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.502 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:26.502 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:26.502 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:26.502 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:26.502 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:26.502 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T17:51:26.502 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.502 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 3, 2024-04-07T17:51:26.502 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:26.502 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.502 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:26.502 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:26.502 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:26.502 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:26.503 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T17:51:26.503 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.503 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 3, 2024-04-07T17:51:26.503 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:26.503 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:26.503 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T17:51:26.503 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:26.503 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:26.503 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:26.503 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:35.977675+0000", 2024-04-07T17:51:26.503 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:08.976003+0000", 2024-04-07T17:51:26.503 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:08.976003+0000", 2024-04-07T17:51:26.503 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:08.976353+0000", 2024-04-07T17:51:26.503 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:35.977675+0000", 2024-04-07T17:51:26.503 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:26.503 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:26.503 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:26.504 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 554, 2024-04-07T17:51:26.504 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:35.977675+0000", 2024-04-07T17:51:26.504 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:35.977675+0000", 2024-04-07T17:51:26.504 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:35.977675+0000", 2024-04-07T17:51:26.504 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:26.504 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:26.504 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:26.504 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:35.977675+0000", 2024-04-07T17:51:26.504 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:35.977675+0000", 2024-04-07T17:51:26.504 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:26.504 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:26.504 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:26.504 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:26.504 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 553, 2024-04-07T17:51:26.504 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:26.504 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:26.504 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:26.505 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:26.505 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:26.505 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:26.505 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:26.505 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:26.505 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.19", 2024-04-07T17:51:26.505 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:26.505 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:26.505 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:26.505 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4937, 2024-04-07T17:51:26.505 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:26.505 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T01:23:38.086617+0000", 2024-04-07T17:51:26.505 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:26.505 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:26.505 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:26.505 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:26.505 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:26.506 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:26.506 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:26.506 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:26.506 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:26.506 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:26.506 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:26.506 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:26.506 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:26.506 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:26.506 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:26.506 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:26.506 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:26.506 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:26.506 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:26.506 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:26.507 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:26.507 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:26.507 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:26.507 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:26.507 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:26.507 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:26.507 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:26.507 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:26.507 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:26.507 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:26.507 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:26.507 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:26.507 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:26.507 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:26.507 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:26.507 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:26.507 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:26.508 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:26.508 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:26.508 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:26.508 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:26.508 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:26.508 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:26.508 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.508 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:26.508 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:26.508 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:26.508 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:26.508 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:26.508 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T17:51:26.508 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.508 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 3, 2024-04-07T17:51:26.508 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:26.508 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.508 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:26.509 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:26.509 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:26.509 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:26.509 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T17:51:26.509 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.509 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 0, 2024-04-07T17:51:26.509 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:26.509 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:26.509 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:26.509 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:26.509 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:26.509 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:26.509 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:35.988644+0000", 2024-04-07T17:51:26.509 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:39.004366+0000", 2024-04-07T17:51:26.509 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:39.004366+0000", 2024-04-07T17:51:26.509 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:39.004810+0000", 2024-04-07T17:51:26.509 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:35.988644+0000", 2024-04-07T17:51:26.510 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.510 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:26.510 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.510 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:26.510 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:35.988644+0000", 2024-04-07T17:51:26.510 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:35.988644+0000", 2024-04-07T17:51:26.510 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:35.988644+0000", 2024-04-07T17:51:26.510 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:26.510 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:26.510 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.510 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:35.988644+0000", 2024-04-07T17:51:26.510 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:35.988644+0000", 2024-04-07T17:51:26.510 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:26.510 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:26.510 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:26.511 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:26.511 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:26.511 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:26.511 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:26.511 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:26.511 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:26.511 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:26.511 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:26.511 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:26.511 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:26.511 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.1b", 2024-04-07T17:51:26.511 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:26.511 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:26.511 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:26.511 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4992, 2024-04-07T17:51:26.511 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:26.511 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T19:31:04.334052+0000", 2024-04-07T17:51:26.512 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:26.512 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:26.512 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:26.512 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:26.512 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:26.512 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:26.512 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:26.512 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:26.512 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:26.512 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:26.512 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:26.512 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:26.512 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:26.512 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:26.512 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:26.513 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:26.513 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:26.513 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:26.513 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:26.513 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:26.513 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:26.513 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:26.513 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:26.513 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:26.513 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:26.513 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:26.513 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:26.513 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:26.513 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:26.513 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:26.513 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:26.513 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:26.514 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:26.514 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:26.514 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:26.514 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:26.514 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:26.514 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:26.514 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:26.514 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:26.514 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:26.514 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:26.514 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:26.514 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.514 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:26.514 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:26.514 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:26.515 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:26.515 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:26.515 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T17:51:26.515 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.515 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 0, 2024-04-07T17:51:26.515 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:26.515 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.515 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:26.515 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:26.515 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:26.515 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:26.515 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T17:51:26.515 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.515 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 6, 2024-04-07T17:51:26.515 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:26.515 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:26.516 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T17:51:26.516 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:26.516 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:26.516 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:26.516 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.372918+0000", 2024-04-07T17:51:26.516 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:38.960251+0000", 2024-04-07T17:51:26.516 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:38.960251+0000", 2024-04-07T17:51:26.516 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:38.960896+0000", 2024-04-07T17:51:26.516 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.372918+0000", 2024-04-07T17:51:26.516 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:26.516 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:26.516 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:26.516 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:26.516 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.372918+0000", 2024-04-07T17:51:26.517 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.372918+0000", 2024-04-07T17:51:26.517 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.372918+0000", 2024-04-07T17:51:26.517 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:26.517 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:26.517 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:26.517 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.372918+0000", 2024-04-07T17:51:26.517 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.372918+0000", 2024-04-07T17:51:26.517 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:26.517 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 2, 2024-04-07T17:51:26.517 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:26.517 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:26.517 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:26.517 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:26.517 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:26.517 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:26.517 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:26.517 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 2, 2024-04-07T17:51:26.518 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:26.518 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:26.518 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:26.518 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.1a", 2024-04-07T17:51:26.518 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:26.518 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:26.518 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:26.518 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4839, 2024-04-07T17:51:26.518 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:26.518 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T18:24:54.929614+0000", 2024-04-07T17:51:26.518 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:26.518 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:26.518 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:26.518 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:26.518 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:26.518 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:26.519 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:26.519 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:26.519 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:26.519 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:26.519 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:26.519 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:26.519 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:26.519 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:26.519 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:26.519 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:26.519 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:26.519 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:26.519 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:26.519 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:26.519 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:26.519 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:26.519 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:26.520 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:26.520 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:26.520 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:26.520 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:26.520 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:26.520 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:26.520 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:26.520 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:26.520 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:26.520 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:26.520 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:26.520 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:26.520 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:26.520 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:26.520 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:26.520 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:26.521 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:26.521 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:26.521 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:26.521 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:26.521 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.521 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:26.521 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:26.521 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:26.521 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:26.521 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:26.521 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T17:51:26.521 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.521 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 6, 2024-04-07T17:51:26.521 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "623'2" 2024-04-07T17:51:26.521 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.521 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:26.522 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:26.522 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:26.522 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:26.522 INFO:tasks.workunit.client.0.smithi100.stdout: 2 2024-04-07T17:51:26.522 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.522 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 4, 2024-04-07T17:51:26.522 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:26.522 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:26.522 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:26.522 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:26.522 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:26.522 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:26.522 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:35.981965+0000", 2024-04-07T17:51:26.522 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:24:13.936610+0000", 2024-04-07T17:51:26.522 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:24:13.936610+0000", 2024-04-07T17:51:26.523 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:24:13.937948+0000", 2024-04-07T17:51:26.523 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:35.981965+0000", 2024-04-07T17:51:26.523 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.523 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:26.523 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.523 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 500, 2024-04-07T17:51:26.523 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:35.981965+0000", 2024-04-07T17:51:26.523 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:35.981965+0000", 2024-04-07T17:51:26.523 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:35.981965+0000", 2024-04-07T17:51:26.523 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:26.523 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:26.523 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.523 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:35.981965+0000", 2024-04-07T17:51:26.523 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:35.981965+0000", 2024-04-07T17:51:26.523 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:26.523 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:26.524 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:26.524 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:26.524 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 499, 2024-04-07T17:51:26.524 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:26.524 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:26.524 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:26.524 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:26.524 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:26.524 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:26.524 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:26.524 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:26.524 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.1a", 2024-04-07T17:51:26.524 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:26.524 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:26.524 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:26.524 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4930, 2024-04-07T17:51:26.524 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:26.525 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T02:25:15.032887+0000", 2024-04-07T17:51:26.525 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:26.525 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:26.525 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:26.525 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:26.525 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:26.525 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:26.525 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:26.525 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:26.525 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:26.525 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:26.525 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:26.525 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:26.525 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:26.525 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:26.525 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:26.526 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:26.526 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:26.526 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:26.526 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:26.526 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:26.526 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:26.526 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:26.526 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:26.526 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:26.526 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:26.526 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:26.526 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:26.526 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:26.526 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:26.526 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:26.527 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:26.527 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:26.527 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:26.527 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:26.527 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:26.527 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:26.527 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:26.527 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:26.527 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:26.527 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:26.527 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:26.527 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:26.527 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:26.527 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.527 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:26.527 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:26.528 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:26.528 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:26.528 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:26.528 INFO:tasks.workunit.client.0.smithi100.stdout: 2 2024-04-07T17:51:26.528 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.528 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 4, 2024-04-07T17:51:26.528 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:26.528 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.528 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:26.528 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:26.528 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:26.528 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:26.528 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T17:51:26.528 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.528 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 3, 2024-04-07T17:51:26.529 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:26.529 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:26.529 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T17:51:26.529 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:26.529 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:26.529 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:26.529 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:35.977526+0000", 2024-04-07T17:51:26.529 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:38.997910+0000", 2024-04-07T17:51:26.529 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:38.997910+0000", 2024-04-07T17:51:26.529 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:38.998434+0000", 2024-04-07T17:51:26.529 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:35.977526+0000", 2024-04-07T17:51:26.529 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:26.529 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:26.529 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:26.529 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:26.529 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:35.977526+0000", 2024-04-07T17:51:26.529 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:35.977526+0000", 2024-04-07T17:51:26.530 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:35.977526+0000", 2024-04-07T17:51:26.530 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:26.530 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:26.530 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:26.530 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:35.977526+0000", 2024-04-07T17:51:26.530 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:35.977526+0000", 2024-04-07T17:51:26.530 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:26.530 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 2, 2024-04-07T17:51:26.530 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:26.530 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:26.530 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:26.530 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:26.530 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:26.530 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:26.531 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:26.531 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 2, 2024-04-07T17:51:26.531 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:26.531 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:26.531 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:26.531 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.1b", 2024-04-07T17:51:26.531 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:26.531 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:26.531 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:26.531 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4932, 2024-04-07T17:51:26.531 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:26.531 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T02:18:46.594583+0000", 2024-04-07T17:51:26.531 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:26.531 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:26.531 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:26.531 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:26.531 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:26.532 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:26.532 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:26.532 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:26.532 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:26.532 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:26.532 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:26.532 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:26.532 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:26.532 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:26.532 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:26.532 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:26.532 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:26.532 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:26.532 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:26.532 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:26.532 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:26.533 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:26.533 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:26.533 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:26.533 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:26.533 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:26.533 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:26.533 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:26.533 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:26.533 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:26.533 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:26.533 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:26.533 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:26.533 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:26.533 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:26.533 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:26.533 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:26.533 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:26.534 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:26.534 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:26.534 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:26.534 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:26.534 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:26.534 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.534 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:26.534 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:26.534 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:26.534 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:26.534 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:26.534 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T17:51:26.534 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.535 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 3, 2024-04-07T17:51:26.535 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "623'2" 2024-04-07T17:51:26.535 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.535 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:26.535 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:26.535 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:26.535 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:26.535 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T17:51:26.535 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.535 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 5, 2024-04-07T17:51:26.535 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:26.535 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:26.535 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:26.535 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:26.535 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:26.535 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:26.536 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.365302+0000", 2024-04-07T17:51:26.536 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:08.974827+0000", 2024-04-07T17:51:26.536 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:08.974827+0000", 2024-04-07T17:51:26.536 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:08.975369+0000", 2024-04-07T17:51:26.536 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.365302+0000", 2024-04-07T17:51:26.536 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.536 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:26.536 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.536 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 554, 2024-04-07T17:51:26.536 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.365302+0000", 2024-04-07T17:51:26.536 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.365302+0000", 2024-04-07T17:51:26.536 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.365302+0000", 2024-04-07T17:51:26.536 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:26.536 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:26.536 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.536 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.365302+0000", 2024-04-07T17:51:26.537 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.365302+0000", 2024-04-07T17:51:26.537 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:26.537 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:26.537 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:26.537 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:26.537 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 553, 2024-04-07T17:51:26.537 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:26.537 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:26.537 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:26.537 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:26.537 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:26.537 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:26.537 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:26.537 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:26.537 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.1d", 2024-04-07T17:51:26.537 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:26.537 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:26.538 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:26.538 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4868, 2024-04-07T17:51:26.538 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:26.538 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T19:16:10.778146+0000", 2024-04-07T17:51:26.538 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:26.538 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:26.538 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:26.538 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:26.538 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:26.538 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:26.538 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:26.538 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:26.538 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:26.538 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:26.538 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:26.539 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:26.539 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:26.539 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:26.539 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:26.539 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:26.539 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:26.539 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:26.539 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:26.539 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:26.539 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:26.539 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:26.539 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:26.539 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:26.539 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:26.539 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:26.539 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:26.540 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:26.540 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:26.540 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:26.540 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:26.540 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:26.540 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:26.540 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:26.540 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:26.540 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:26.540 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:26.540 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:26.540 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:26.540 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:26.540 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:26.540 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:26.540 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:26.541 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.541 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:26.541 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:26.541 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:26.541 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:26.541 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:26.541 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T17:51:26.541 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.541 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 5, 2024-04-07T17:51:26.541 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:26.541 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.541 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:26.541 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:26.541 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:26.541 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:26.541 INFO:tasks.workunit.client.0.smithi100.stdout: 2 2024-04-07T17:51:26.542 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.542 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 7, 2024-04-07T17:51:26.542 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:26.542 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:26.542 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T17:51:26.542 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:26.542 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:26.542 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:26.542 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.364942+0000", 2024-04-07T17:51:26.542 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:38.975920+0000", 2024-04-07T17:51:26.542 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:38.975920+0000", 2024-04-07T17:51:26.542 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:38.991127+0000", 2024-04-07T17:51:26.542 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.364942+0000", 2024-04-07T17:51:26.542 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:26.542 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:26.543 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:26.543 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:26.543 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.364942+0000", 2024-04-07T17:51:26.543 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.364942+0000", 2024-04-07T17:51:26.543 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.364942+0000", 2024-04-07T17:51:26.543 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:26.543 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:26.543 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:26.543 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.364942+0000", 2024-04-07T17:51:26.543 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.364942+0000", 2024-04-07T17:51:26.543 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:26.543 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 2, 2024-04-07T17:51:26.543 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:26.543 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:26.543 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:26.543 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:26.544 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:26.544 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:26.544 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:26.544 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 2, 2024-04-07T17:51:26.544 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:26.544 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:26.544 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:26.544 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.1c", 2024-04-07T17:51:26.544 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:26.544 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:26.544 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:26.544 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4819, 2024-04-07T17:51:26.544 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:26.544 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T22:48:18.986526+0000", 2024-04-07T17:51:26.544 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:26.544 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:26.545 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:26.545 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:26.545 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:26.545 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:26.545 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:26.545 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:26.545 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:26.545 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:26.545 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:26.545 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:26.545 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:26.545 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:26.545 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:26.545 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:26.545 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:26.545 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:26.546 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:26.546 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 3, 2024-04-07T17:51:26.546 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 1, 2024-04-07T17:51:26.546 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:26.546 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 1, 2024-04-07T17:51:26.546 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:26.546 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:26.546 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:26.546 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:26.546 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:26.546 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 1, 2024-04-07T17:51:26.546 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:26.546 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:26.546 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:26.546 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:26.547 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:26.547 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:26.547 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:26.547 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 6, 2024-04-07T17:51:26.547 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 6, 2024-04-07T17:51:26.547 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:26.547 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:26.547 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:26.547 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 3, 2024-04-07T17:51:26.547 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:26.547 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.547 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:26.547 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:26.547 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:26.547 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:26.547 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:26.548 INFO:tasks.workunit.client.0.smithi100.stdout: 2 2024-04-07T17:51:26.548 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.548 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 7, 2024-04-07T17:51:26.548 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "61'2" 2024-04-07T17:51:26.548 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.548 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:26.548 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:26.548 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:26.548 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:26.548 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T17:51:26.548 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.548 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 5, 2024-04-07T17:51:26.548 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:26.548 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:26.548 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:26.548 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:26.548 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:26.548 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:26.549 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.367080+0000", 2024-04-07T17:51:26.549 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:24:41.542687+0000", 2024-04-07T17:51:26.549 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:24:41.542687+0000", 2024-04-07T17:51:26.549 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:24:41.543356+0000", 2024-04-07T17:51:26.549 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.367080+0000", 2024-04-07T17:51:26.549 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.549 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:26.549 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.549 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 527, 2024-04-07T17:51:26.549 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.367080+0000", 2024-04-07T17:51:26.549 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.367080+0000", 2024-04-07T17:51:26.549 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.367080+0000", 2024-04-07T17:51:26.549 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:26.549 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:26.549 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.550 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.367080+0000", 2024-04-07T17:51:26.550 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.367080+0000", 2024-04-07T17:51:26.550 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:26.550 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:26.550 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:26.550 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:26.550 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 526, 2024-04-07T17:51:26.550 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:26.550 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:26.550 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:26.550 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:26.550 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:26.550 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:26.550 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:26.550 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:26.551 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.1c", 2024-04-07T17:51:26.551 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:26.551 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:26.551 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:26.551 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4865, 2024-04-07T17:51:26.551 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:26.551 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T21:43:40.340691+0000", 2024-04-07T17:51:26.551 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:26.551 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:26.551 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:26.551 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:26.551 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:26.551 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:26.551 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:26.551 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:26.551 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:26.551 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:26.552 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:26.552 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:26.552 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:26.552 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:26.552 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:26.552 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:26.552 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:26.552 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:26.552 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:26.552 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:26.552 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:26.552 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:26.552 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:26.552 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:26.552 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:26.552 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:26.553 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:26.553 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:26.553 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:26.553 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:26.553 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:26.553 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:26.553 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:26.553 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:26.553 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:26.553 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:26.553 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:26.553 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:26.553 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:26.553 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:26.553 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:26.553 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:26.554 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:26.554 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.554 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:26.554 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:26.554 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:26.554 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:26.554 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:26.554 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T17:51:26.554 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.554 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 5, 2024-04-07T17:51:26.554 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:26.554 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.554 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:26.555 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:26.555 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:26.555 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:26.555 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T17:51:26.555 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.555 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 7, 2024-04-07T17:51:26.555 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:26.555 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:26.555 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T17:51:26.555 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:26.555 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:26.555 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:26.555 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.373509+0000", 2024-04-07T17:51:26.555 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:39.001898+0000", 2024-04-07T17:51:26.555 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:39.001898+0000", 2024-04-07T17:51:26.556 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:39.004508+0000", 2024-04-07T17:51:26.556 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.373509+0000", 2024-04-07T17:51:26.556 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:26.556 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:26.556 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:26.556 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:26.556 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.373509+0000", 2024-04-07T17:51:26.556 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.373509+0000", 2024-04-07T17:51:26.556 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.373509+0000", 2024-04-07T17:51:26.556 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:26.556 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:26.556 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:26.556 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.373509+0000", 2024-04-07T17:51:26.556 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.373509+0000", 2024-04-07T17:51:26.556 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:26.556 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 2, 2024-04-07T17:51:26.557 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:26.557 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:26.557 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:26.557 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:26.557 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:26.557 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:26.557 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:26.557 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 2, 2024-04-07T17:51:26.557 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:26.557 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:26.557 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:26.557 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.1d", 2024-04-07T17:51:26.557 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:26.557 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:26.557 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:26.557 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4804, 2024-04-07T17:51:26.557 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:26.558 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T01:47:40.701645+0000", 2024-04-07T17:51:26.558 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:26.558 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:26.558 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:26.558 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:26.558 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:26.558 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:26.558 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:26.558 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:26.558 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:26.558 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:26.558 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:26.558 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:26.558 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:26.558 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:26.559 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:26.559 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:26.559 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:26.559 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:26.559 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:26.559 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 3, 2024-04-07T17:51:26.559 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 1, 2024-04-07T17:51:26.559 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:26.559 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 1, 2024-04-07T17:51:26.559 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:26.559 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:26.559 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:26.559 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:26.559 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:26.559 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 1, 2024-04-07T17:51:26.559 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:26.559 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:26.559 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:26.560 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:26.560 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:26.560 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:26.560 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:26.560 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 6, 2024-04-07T17:51:26.560 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 6, 2024-04-07T17:51:26.560 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:26.560 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:26.560 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:26.560 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 3, 2024-04-07T17:51:26.560 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:26.560 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.560 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:26.560 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:26.560 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:26.560 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:26.560 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:26.560 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T17:51:26.561 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.561 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 7, 2024-04-07T17:51:26.561 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "61'2" 2024-04-07T17:51:26.561 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.561 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:26.561 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:26.561 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:26.561 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:26.561 INFO:tasks.workunit.client.0.smithi100.stdout: 2 2024-04-07T17:51:26.561 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.561 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 0, 2024-04-07T17:51:26.561 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:26.561 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:26.561 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:26.561 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:26.561 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:26.562 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:26.562 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:35.983257+0000", 2024-04-07T17:51:26.562 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:24:41.564700+0000", 2024-04-07T17:51:26.562 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:24:41.564700+0000", 2024-04-07T17:51:26.562 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:24:41.565093+0000", 2024-04-07T17:51:26.562 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:35.983257+0000", 2024-04-07T17:51:26.562 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.562 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:26.562 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.562 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 527, 2024-04-07T17:51:26.562 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:35.983257+0000", 2024-04-07T17:51:26.562 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:35.983257+0000", 2024-04-07T17:51:26.562 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:35.983257+0000", 2024-04-07T17:51:26.562 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:26.563 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:26.563 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.563 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:35.983257+0000", 2024-04-07T17:51:26.563 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:35.983257+0000", 2024-04-07T17:51:26.563 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:26.563 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:26.563 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:26.563 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:26.563 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 526, 2024-04-07T17:51:26.563 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:26.563 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:26.563 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:26.563 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:26.563 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:26.563 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:26.563 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:26.563 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:26.564 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.1f", 2024-04-07T17:51:26.564 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:26.564 INFO:tasks.workunit.client.0.smithi100.stderr:+ ceph telemetry on --license sharing-1-0 2024-04-07T17:51:26.565 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:25 smithi144 bash[66654]: audit 2024-04-07T17:51:23.722297+0000 mgr.y (mgr.25175) 2885 : audit [DBG] from='client.45983 -' entity='client.admin' cmd=[{"prefix": "telemetry preview-device", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T17:51:26.566 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:25 smithi144 bash[66654]: audit 2024-04-07T17:51:23.875259+0000 mgr.y (mgr.25175) 2886 : audit [DBG] from='client.45983 -' entity='client.admin' cmd=[{"prefix": "telemetry preview-device", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T17:51:26.566 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:25 smithi144 bash[66654]: audit 2024-04-07T17:51:24.233450+0000 mgr.y (mgr.25175) 2887 : audit [DBG] from='client.36087 -' entity='client.admin' cmd=[{"prefix": "telemetry preview-all", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T17:51:26.566 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:25 smithi144 bash[66654]: audit 2024-04-07T17:51:24.236072+0000 mon.a (mon.0) 4811 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:51:26.566 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:25 smithi144 bash[66654]: audit 2024-04-07T17:51:24.244631+0000 mon.a (mon.0) 4812 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config-key dump", "format": "json"}]: dispatch 2024-04-07T17:51:26.566 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:25 smithi144 bash[66654]: audit 2024-04-07T17:51:24.463056+0000 mon.a (mon.0) 4813 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:26.566 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:25 smithi144 bash[66654]: audit 2024-04-07T17:51:24.463558+0000 mon.a (mon.0) 4814 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:26.566 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:25 smithi144 bash[66654]: audit 2024-04-07T17:51:24.463881+0000 mon.a (mon.0) 4815 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:26.566 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:25 smithi144 bash[66654]: audit 2024-04-07T17:51:24.467349+0000 mon.a (mon.0) 4816 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:26.566 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:25 smithi144 bash[66654]: audit 2024-04-07T17:51:24.467818+0000 mon.a (mon.0) 4817 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:26.566 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:25 smithi144 bash[66654]: audit 2024-04-07T17:51:24.468123+0000 mon.a (mon.0) 4818 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:26.566 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:25 smithi144 bash[66654]: audit 2024-04-07T17:51:24.471323+0000 mon.a (mon.0) 4819 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:26.566 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:25 smithi144 bash[66654]: audit 2024-04-07T17:51:24.471801+0000 mon.a (mon.0) 4820 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:26.567 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:25 smithi144 bash[66654]: audit 2024-04-07T17:51:24.472256+0000 mon.a (mon.0) 4821 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:26.567 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:25 smithi144 bash[66654]: audit 2024-04-07T17:51:24.491001+0000 mon.a (mon.0) 4822 : audit [INF] from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T17:51:26.567 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:25 smithi144 bash[66654]: audit 2024-04-07T17:51:24.491101+0000 mon.a (mon.0) 4823 : audit [INF] from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T17:51:26.567 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:25 smithi144 bash[66654]: audit 2024-04-07T17:51:24.491464+0000 mon.a (mon.0) 4824 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:26.567 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:25 smithi144 bash[66654]: audit 2024-04-07T17:51:24.491973+0000 mon.a (mon.0) 4825 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:26.567 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:25 smithi144 bash[66654]: audit 2024-04-07T17:51:24.492374+0000 mon.a (mon.0) 4826 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:26.567 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:25 smithi144 bash[66654]: audit 2024-04-07T17:51:24.495655+0000 mon.a (mon.0) 4827 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:26.567 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:25 smithi144 bash[66654]: audit 2024-04-07T17:51:24.496511+0000 mon.a (mon.0) 4828 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:26.567 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:25 smithi144 bash[66654]: audit 2024-04-07T17:51:24.496829+0000 mon.a (mon.0) 4829 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:26.567 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:25 smithi144 bash[66654]: audit 2024-04-07T17:51:24.497477+0000 mon.b (mon.1) 27 : audit [INF] from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T17:51:26.567 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:25 smithi144 bash[66654]: audit 2024-04-07T17:51:24.497582+0000 mon.b (mon.1) 28 : audit [INF] from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T17:51:26.567 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:25 smithi144 bash[66654]: audit 2024-04-07T17:51:24.499288+0000 mon.c (mon.2) 125 : audit [INF] from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T17:51:26.567 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:25 smithi144 bash[66654]: audit 2024-04-07T17:51:24.499394+0000 mon.c (mon.2) 126 : audit [INF] from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T17:51:26.567 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:25 smithi144 bash[66654]: audit 2024-04-07T17:51:24.499820+0000 mon.a (mon.0) 4830 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:26.567 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:25 smithi144 bash[66654]: audit 2024-04-07T17:51:24.500812+0000 mon.a (mon.0) 4831 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:26.568 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:25 smithi144 bash[66654]: audit 2024-04-07T17:51:24.501145+0000 mon.a (mon.0) 4832 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:26.568 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:25 smithi144 bash[66654]: audit 2024-04-07T17:51:24.822625+0000 mon.a (mon.0) 4833 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:51:26.568 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:25 smithi144 bash[66654]: audit 2024-04-07T17:51:24.830664+0000 mon.a (mon.0) 4834 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config-key dump", "format": "json"}]: dispatch 2024-04-07T17:51:26.568 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:25 smithi144 bash[66654]: audit 2024-04-07T17:51:25.048444+0000 mon.a (mon.0) 4835 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:26.568 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:25 smithi144 bash[66654]: audit 2024-04-07T17:51:25.048954+0000 mon.a (mon.0) 4836 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:26.568 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:25 smithi144 bash[66654]: audit 2024-04-07T17:51:25.049416+0000 mon.a (mon.0) 4837 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:26.568 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:25 smithi144 bash[66654]: audit 2024-04-07T17:51:25.053033+0000 mon.a (mon.0) 4838 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:26.568 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:25 smithi144 bash[66654]: audit 2024-04-07T17:51:25.053470+0000 mon.a (mon.0) 4839 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:26.568 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:25 smithi144 bash[66654]: audit 2024-04-07T17:51:25.053843+0000 mon.a (mon.0) 4840 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:26.568 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:25 smithi144 bash[66654]: audit 2024-04-07T17:51:25.056987+0000 mon.a (mon.0) 4841 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:26.568 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:25 smithi144 bash[66654]: audit 2024-04-07T17:51:25.057521+0000 mon.a (mon.0) 4842 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:26.568 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:25 smithi144 bash[66654]: audit 2024-04-07T17:51:25.057918+0000 mon.a (mon.0) 4843 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:26.568 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:25 smithi144 bash[66654]: audit 2024-04-07T17:51:25.076689+0000 mon.a (mon.0) 4844 : audit [INF] from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T17:51:26.568 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:25 smithi144 bash[66654]: audit 2024-04-07T17:51:25.076842+0000 mon.a (mon.0) 4845 : audit [INF] from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T17:51:26.568 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:25 smithi144 bash[66654]: audit 2024-04-07T17:51:25.077280+0000 mon.a (mon.0) 4846 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:26.569 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:25 smithi144 bash[66654]: audit 2024-04-07T17:51:25.077798+0000 mon.a (mon.0) 4847 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:26.569 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:25 smithi144 bash[66654]: audit 2024-04-07T17:51:25.078158+0000 mon.a (mon.0) 4848 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:26.569 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:25 smithi144 bash[66654]: audit 2024-04-07T17:51:25.081565+0000 mon.a (mon.0) 4849 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:26.569 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:25 smithi144 bash[66654]: audit 2024-04-07T17:51:25.082472+0000 mon.a (mon.0) 4850 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:26.569 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:25 smithi144 bash[66654]: audit 2024-04-07T17:51:25.082795+0000 mon.a (mon.0) 4851 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:26.569 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:25 smithi144 bash[66654]: audit 2024-04-07T17:51:25.083445+0000 mon.b (mon.1) 29 : audit [INF] from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T17:51:26.569 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:25 smithi144 bash[66654]: audit 2024-04-07T17:51:25.083551+0000 mon.b (mon.1) 30 : audit [INF] from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T17:51:26.569 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:25 smithi144 bash[66654]: audit 2024-04-07T17:51:25.085501+0000 mon.c (mon.2) 127 : audit [INF] from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T17:51:26.569 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:25 smithi144 bash[66654]: audit 2024-04-07T17:51:25.085606+0000 mon.c (mon.2) 128 : audit [INF] from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T17:51:26.569 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:25 smithi144 bash[66654]: audit 2024-04-07T17:51:25.086022+0000 mon.a (mon.0) 4852 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:26.569 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:25 smithi144 bash[66654]: audit 2024-04-07T17:51:25.086899+0000 mon.a (mon.0) 4853 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:26.569 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:25 smithi144 bash[66654]: audit 2024-04-07T17:51:25.087254+0000 mon.a (mon.0) 4854 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:26.569 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:26.569 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:26.569 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 5000, 2024-04-07T17:51:26.570 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:26.570 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T01:20:29.103128+0000", 2024-04-07T17:51:26.570 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:26.570 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:26.570 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:26.570 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:26.570 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:26.570 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:26.570 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:26.570 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:26.570 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:26.570 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:26.570 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:26.571 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:26.571 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:26.571 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:26.571 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:26.571 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:26.571 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:26.571 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:26.571 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:26.571 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:26.571 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:26.571 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:26.571 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:26.571 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:26.571 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:26.571 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:26.572 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:26.572 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:26.572 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:26.572 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:26.572 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:26.572 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:26.572 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:26.572 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:26.572 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:26.572 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:26.572 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:26.572 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:26.572 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:26.572 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:26.572 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:26.572 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:26.573 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:26.573 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.573 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:26.573 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:26.573 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:26.573 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:26.573 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:26.573 INFO:tasks.workunit.client.0.smithi100.stdout: 2 2024-04-07T17:51:26.573 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.573 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 0, 2024-04-07T17:51:26.573 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:26.573 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.573 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:26.573 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:26.573 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:26.573 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:26.574 INFO:tasks.workunit.client.0.smithi100.stdout: 5 2024-04-07T17:51:26.574 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.574 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 3, 2024-04-07T17:51:26.574 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:26.574 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:26.574 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T17:51:26.574 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:26.574 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:26.574 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:26.574 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:35.982794+0000", 2024-04-07T17:51:26.574 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:24:41.546310+0000", 2024-04-07T17:51:26.574 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:24:41.546310+0000", 2024-04-07T17:51:26.574 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:24:41.546952+0000", 2024-04-07T17:51:26.574 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:35.982794+0000", 2024-04-07T17:51:26.575 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:26.575 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:26.575 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:26.575 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 527, 2024-04-07T17:51:26.575 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:35.982794+0000", 2024-04-07T17:51:26.575 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:35.982794+0000", 2024-04-07T17:51:26.575 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:35.982794+0000", 2024-04-07T17:51:26.575 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:26.575 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:26.575 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:26.575 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:35.982794+0000", 2024-04-07T17:51:26.575 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:35.982794+0000", 2024-04-07T17:51:26.575 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:26.575 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 2, 2024-04-07T17:51:26.575 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:26.575 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:26.575 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 526, 2024-04-07T17:51:26.576 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:26.576 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:26.576 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:26.576 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:26.576 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 2, 2024-04-07T17:51:26.576 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:26.576 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:26.576 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:26.576 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.1e", 2024-04-07T17:51:26.576 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:26.576 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:26.576 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:26.576 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4947, 2024-04-07T17:51:26.576 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:26.576 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T21:45:55.182027+0000", 2024-04-07T17:51:26.576 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:26.577 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:26.577 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:26.577 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:26.577 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:26.577 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:26.577 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:26.577 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:26.577 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:26.577 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:26.577 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:26.577 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:26.577 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:26.577 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:26.577 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:26.577 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:26.577 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:26.577 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:26.577 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:26.578 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:26.578 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:26.578 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:26.578 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:26.578 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:26.578 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:26.578 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:26.578 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:26.578 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:26.578 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:26.578 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:26.578 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:26.578 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:26.578 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:26.578 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:26.578 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:26.578 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:26.579 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:26.579 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:26.579 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:26.579 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:26.579 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:26.579 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:26.579 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:26.579 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.579 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:26.579 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:26.579 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:26.579 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:26.579 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:26.579 INFO:tasks.workunit.client.0.smithi100.stdout: 5 2024-04-07T17:51:26.579 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.579 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 3, 2024-04-07T17:51:26.579 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "623'2" 2024-04-07T17:51:26.580 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.580 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:26.580 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:26.580 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:26.580 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:26.580 INFO:tasks.workunit.client.0.smithi100.stdout: 2 2024-04-07T17:51:26.580 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.580 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 3, 2024-04-07T17:51:26.580 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:26.580 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:26.580 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:26.580 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:26.580 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:26.580 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:26.580 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:35.982633+0000", 2024-04-07T17:51:26.580 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:08.977988+0000", 2024-04-07T17:51:26.580 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:08.977988+0000", 2024-04-07T17:51:26.581 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:08.978435+0000", 2024-04-07T17:51:26.581 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:35.982633+0000", 2024-04-07T17:51:26.581 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.581 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:26.581 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.581 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 554, 2024-04-07T17:51:26.581 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:35.982633+0000", 2024-04-07T17:51:26.581 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:35.982633+0000", 2024-04-07T17:51:26.581 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:35.982633+0000", 2024-04-07T17:51:26.581 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:26.581 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:26.581 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.581 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:35.982633+0000", 2024-04-07T17:51:26.581 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:35.982633+0000", 2024-04-07T17:51:26.581 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:26.581 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:26.581 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:26.582 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:26.582 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 553, 2024-04-07T17:51:26.582 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:26.582 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:26.582 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:26.582 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:26.582 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:26.582 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:26.582 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:26.582 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:26.582 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.1e", 2024-04-07T17:51:26.582 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:26.582 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:26.582 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:26.582 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4957, 2024-04-07T17:51:26.582 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:26.583 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T21:46:59.340658+0000", 2024-04-07T17:51:26.583 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:26.583 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:26.583 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:26.583 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:26.583 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:26.583 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:26.583 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:26.583 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:26.583 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:26.583 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:26.583 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:26.583 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:26.583 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:26.583 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:26.583 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:26.584 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:26.584 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:26.584 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:26.584 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:26.584 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:26.584 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:26.584 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:26.584 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:26.584 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:26.584 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:26.584 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:26.584 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:26.584 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:26.584 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:26.584 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:26.585 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:26.585 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:26.585 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:26.585 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:26.585 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:26.585 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:26.585 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:26.585 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:26.585 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:26.585 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:26.585 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:26.585 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:26.585 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:26.585 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.585 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:26.586 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:26.586 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:26.586 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:26.586 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:26.586 INFO:tasks.workunit.client.0.smithi100.stdout: 2 2024-04-07T17:51:26.586 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.586 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 3, 2024-04-07T17:51:26.586 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:26.586 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.586 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:26.586 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:26.586 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:26.586 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:26.586 INFO:tasks.workunit.client.0.smithi100.stdout: 4 2024-04-07T17:51:26.586 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.587 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 0, 2024-04-07T17:51:26.587 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:26.587 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:26.587 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T17:51:26.587 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:26.587 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:26.587 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:26.587 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:35.984435+0000", 2024-04-07T17:51:26.587 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:38.997317+0000", 2024-04-07T17:51:26.587 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:38.997317+0000", 2024-04-07T17:51:26.587 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:38.997712+0000", 2024-04-07T17:51:26.587 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:35.984435+0000", 2024-04-07T17:51:26.587 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:26.587 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:26.587 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:26.587 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:26.588 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:35.984435+0000", 2024-04-07T17:51:26.588 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:35.984435+0000", 2024-04-07T17:51:26.588 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:35.984435+0000", 2024-04-07T17:51:26.588 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:26.588 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:26.588 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:26.588 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:35.984435+0000", 2024-04-07T17:51:26.588 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:35.984435+0000", 2024-04-07T17:51:26.588 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:26.588 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 5, 2024-04-07T17:51:26.588 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:26.588 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:26.588 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:26.588 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:26.588 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:26.588 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:26.589 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:26.589 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 5, 2024-04-07T17:51:26.589 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:26.589 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:26.589 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:26.589 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.1f", 2024-04-07T17:51:26.589 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:26.589 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:26.589 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:26.589 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 5008, 2024-04-07T17:51:26.589 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:26.589 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T18:23:18.184011+0000", 2024-04-07T17:51:26.589 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:26.589 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:26.589 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:26.589 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 582, 2024-04-07T17:51:26.589 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:26.590 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:26.590 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:26.590 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:26.590 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:26.590 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:26.590 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:26.590 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:26.590 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:26.590 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:26.590 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:26.590 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:26.590 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:26.590 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:26.590 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:26.590 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 6, 2024-04-07T17:51:26.590 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 2, 2024-04-07T17:51:26.591 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:26.591 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 2, 2024-04-07T17:51:26.591 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:26.591 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:26.591 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:26.591 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:26.591 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:26.591 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 1, 2024-04-07T17:51:26.591 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:26.591 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:26.591 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:26.591 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:26.591 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:26.591 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:26.591 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:26.591 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 4, 2024-04-07T17:51:26.591 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 4, 2024-04-07T17:51:26.592 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:26.592 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:26.592 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:26.592 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 4, 2024-04-07T17:51:26.592 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 1 2024-04-07T17:51:26.592 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.592 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:26.592 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:26.592 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:26.592 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:26.592 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:26.592 INFO:tasks.workunit.client.0.smithi100.stdout: 4 2024-04-07T17:51:26.592 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.592 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 0, 2024-04-07T17:51:26.592 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "623'5" 2024-04-07T17:51:26.592 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.592 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:26.593 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:26.593 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:26.593 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T17:51:26.593 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T17:51:26.593 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.593 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 7, 2024-04-07T17:51:26.593 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:26.593 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:26.593 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:26.593 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:26.593 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:26.593 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:26.593 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.373284+0000", 2024-04-07T17:51:26.593 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:38.996053+0000", 2024-04-07T17:51:26.593 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:38.996053+0000", 2024-04-07T17:51:26.593 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:39.002704+0000", 2024-04-07T17:51:26.593 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.373284+0000", 2024-04-07T17:51:26.594 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.594 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:26.594 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.594 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:26.594 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.373284+0000", 2024-04-07T17:51:26.594 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.373284+0000", 2024-04-07T17:51:26.594 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.373284+0000", 2024-04-07T17:51:26.594 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:26.594 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:26.594 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.594 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.373284+0000", 2024-04-07T17:51:26.594 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.373284+0000", 2024-04-07T17:51:26.594 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:26.594 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:26.594 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:26.595 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:26.595 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:26.595 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:26.595 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:26.595 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:26.595 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:26.595 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:26.595 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:26.595 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:26.595 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:26.595 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.2d", 2024-04-07T17:51:26.595 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:26.595 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:26.595 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:26.595 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4802, 2024-04-07T17:51:26.595 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:26.595 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T17:45:19.934847+0000", 2024-04-07T17:51:26.596 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:26.596 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:26.596 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:26.596 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:26.596 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:26.596 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:26.596 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:26.596 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:26.596 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:26.596 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:26.596 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:26.596 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:26.596 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:26.596 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:26.596 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:26.597 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:26.597 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:26.597 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:26.597 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:26.597 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:26.597 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:26.597 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:26.597 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:26.597 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:26.597 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:26.597 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:26.597 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:26.597 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:26.597 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:26.597 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:26.597 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:26.597 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:26.598 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:26.598 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:26.598 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:26.598 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:26.598 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:26.598 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:26.598 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:26.598 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:26.598 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:26.598 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:26.598 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:26.598 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.598 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:26.598 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:26.598 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:26.598 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:26.599 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T17:51:26.599 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T17:51:26.599 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.599 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 7, 2024-04-07T17:51:26.599 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:26.599 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.599 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:26.599 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:26.599 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T17:51:26.599 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:26.599 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T17:51:26.599 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.599 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 2, 2024-04-07T17:51:26.599 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:26.599 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:26.599 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:26.599 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:26.600 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:26.600 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:26.600 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.359468+0000", 2024-04-07T17:51:26.600 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:38.967369+0000", 2024-04-07T17:51:26.600 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:38.967369+0000", 2024-04-07T17:51:26.600 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:38.967737+0000", 2024-04-07T17:51:26.600 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.359468+0000", 2024-04-07T17:51:26.600 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.600 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:26.600 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.600 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:26.600 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.359468+0000", 2024-04-07T17:51:26.600 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.359468+0000", 2024-04-07T17:51:26.600 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.359468+0000", 2024-04-07T17:51:26.600 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:26.600 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:26.600 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.600 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.359468+0000", 2024-04-07T17:51:26.601 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.359468+0000", 2024-04-07T17:51:26.601 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:26.601 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:26.601 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:26.601 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:26.601 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:26.601 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:26.601 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:26.601 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:26.601 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:26.601 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:26.601 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:26.601 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:26.601 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:26.601 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.2e", 2024-04-07T17:51:26.601 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:26.601 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:26.602 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:26.602 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4964, 2024-04-07T17:51:26.602 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:26.602 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T22:15:36.602436+0000", 2024-04-07T17:51:26.602 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:26.602 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:26.602 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:26.602 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:26.602 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:26.602 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:26.602 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:26.602 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:26.602 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:26.602 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:26.602 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:26.602 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:26.603 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:26.603 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:26.603 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:26.603 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:26.603 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:26.603 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:26.603 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:26.603 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:26.603 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:26.603 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:26.603 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:26.603 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:26.603 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:26.603 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:26.603 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:26.603 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:26.603 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:26.603 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:26.604 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:26.604 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:26.604 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:26.604 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:26.604 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:26.604 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:26.604 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:26.604 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:26.604 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:26.604 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:26.604 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:26.604 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:26.604 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:26.604 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.604 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:26.604 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:26.604 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:26.605 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T17:51:26.605 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:26.605 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T17:51:26.605 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.605 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 2, 2024-04-07T17:51:26.605 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:26.605 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.605 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:26.605 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:26.605 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:26.605 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:26.605 INFO:tasks.workunit.client.0.smithi100.stdout: 4 2024-04-07T17:51:26.605 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.605 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 5, 2024-04-07T17:51:26.605 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:26.605 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:26.606 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:26.606 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:26.606 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:26.606 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:26.606 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.361813+0000", 2024-04-07T17:51:26.606 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:38.971115+0000", 2024-04-07T17:51:26.606 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:38.971115+0000", 2024-04-07T17:51:26.606 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:38.971858+0000", 2024-04-07T17:51:26.606 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.361813+0000", 2024-04-07T17:51:26.606 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.606 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:26.606 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.606 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:26.606 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.361813+0000", 2024-04-07T17:51:26.606 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.361813+0000", 2024-04-07T17:51:26.607 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.361813+0000", 2024-04-07T17:51:26.607 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:26.607 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:26.607 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.607 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.361813+0000", 2024-04-07T17:51:26.607 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.361813+0000", 2024-04-07T17:51:26.607 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:26.607 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:26.607 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:26.607 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:26.607 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:26.607 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:26.607 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:26.607 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:26.607 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:26.607 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:26.607 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:26.607 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:26.608 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:26.608 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.2f", 2024-04-07T17:51:26.608 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:26.608 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:26.608 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:26.608 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4863, 2024-04-07T17:51:26.608 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:26.608 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T22:01:10.276598+0000", 2024-04-07T17:51:26.608 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:26.608 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:26.608 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:26.608 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:26.608 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:26.608 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:26.608 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:26.608 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:26.608 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:26.609 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:26.609 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:26.609 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:26.609 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:26.609 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:26.609 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:26.609 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:26.609 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:26.609 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:26.609 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:26.609 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:26.609 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:26.609 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:26.609 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:26.609 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:26.609 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:26.609 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:26.610 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:26.610 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:26.610 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:26.610 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:26.610 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:26.610 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:26.610 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:26.610 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:26.610 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:26.610 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:26.610 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:26.610 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:26.610 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:26.610 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:26.610 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:26.610 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:26.610 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:26.611 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.611 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:26.611 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:26.611 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:26.611 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:26.611 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:26.611 INFO:tasks.workunit.client.0.smithi100.stdout: 4 2024-04-07T17:51:26.611 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.611 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 5, 2024-04-07T17:51:26.611 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:26.611 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.611 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:26.611 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:26.611 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:26.611 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:26.611 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T17:51:26.611 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.611 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 5, 2024-04-07T17:51:26.612 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:26.612 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:26.612 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:26.612 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:26.612 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:26.612 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:26.612 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.364596+0000", 2024-04-07T17:51:26.612 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:39.005541+0000", 2024-04-07T17:51:26.612 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:39.005541+0000", 2024-04-07T17:51:26.612 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:39.006138+0000", 2024-04-07T17:51:26.612 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.364596+0000", 2024-04-07T17:51:26.612 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.612 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:26.612 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.612 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:26.612 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.364596+0000", 2024-04-07T17:51:26.612 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.364596+0000", 2024-04-07T17:51:26.613 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.364596+0000", 2024-04-07T17:51:26.613 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:26.613 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:26.613 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.613 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.364596+0000", 2024-04-07T17:51:26.613 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.364596+0000", 2024-04-07T17:51:26.613 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:26.613 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:26.613 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:26.613 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:26.613 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:26.613 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:26.613 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:26.613 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:26.613 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:26.613 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:26.613 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:26.614 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:26.614 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:26.614 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.30", 2024-04-07T17:51:26.614 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:26.614 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:26.614 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:26.614 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4876, 2024-04-07T17:51:26.614 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:26.614 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T20:40:58.093262+0000", 2024-04-07T17:51:26.614 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:26.614 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:26.614 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:26.614 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:26.614 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:26.614 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:26.614 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:26.614 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:26.614 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:26.615 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:26.615 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:26.615 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:26.615 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:26.615 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:26.615 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:26.615 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:26.615 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:26.615 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:26.615 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:26.615 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:26.615 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:26.615 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:26.615 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:26.615 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:26.616 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:26.616 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:26.616 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:26.616 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:26.616 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:26.616 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:26.616 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:26.616 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:26.616 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:26.616 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:26.616 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:26.616 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:26.616 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:26.616 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:26.617 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:26.617 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:26.617 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:26.617 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:26.617 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:26.617 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.617 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:26.617 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:26.617 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:26.617 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:26.617 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:26.617 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T17:51:26.617 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.617 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 5, 2024-04-07T17:51:26.617 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:26.618 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.618 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:26.618 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:26.618 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:26.618 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:26.618 INFO:tasks.workunit.client.0.smithi100.stdout: 5 2024-04-07T17:51:26.618 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.618 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 1, 2024-04-07T17:51:26.618 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:26.618 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:26.618 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:26.618 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:26.618 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:26.618 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:26.618 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.331123+0000", 2024-04-07T17:51:26.619 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:24:41.533626+0000", 2024-04-07T17:51:26.619 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:24:41.533626+0000", 2024-04-07T17:51:26.619 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:24:41.534090+0000", 2024-04-07T17:51:26.619 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.331123+0000", 2024-04-07T17:51:26.619 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.619 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:26.619 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.619 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 527, 2024-04-07T17:51:26.619 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.331123+0000", 2024-04-07T17:51:26.619 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.331123+0000", 2024-04-07T17:51:26.619 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.331123+0000", 2024-04-07T17:51:26.619 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:26.619 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:26.619 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.619 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.331123+0000", 2024-04-07T17:51:26.620 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.331123+0000", 2024-04-07T17:51:26.620 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:26.620 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:26.620 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:26.620 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:26.620 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 526, 2024-04-07T17:51:26.620 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:26.620 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:26.620 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:26.620 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:26.620 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:26.620 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:26.620 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:26.620 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:26.620 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.31", 2024-04-07T17:51:26.620 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:26.621 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:26.621 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:26.621 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4978, 2024-04-07T17:51:26.621 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:26.621 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T16:42:51.811301+0000", 2024-04-07T17:51:26.621 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:26.621 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:26.621 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:26.621 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:26.621 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:26.621 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:26.621 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:26.621 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:26.621 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:26.622 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:26.622 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:26.622 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:26.622 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:26.622 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:26.622 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:26.622 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:26.622 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:26.622 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:26.622 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:26.622 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:26.622 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:26.622 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:26.622 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:26.622 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:26.623 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:26.623 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:26.623 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:26.623 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:26.623 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:26.623 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:26.623 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:26.623 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:26.623 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:26.623 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:26.623 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:26.623 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:26.623 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:26.623 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:26.623 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:26.623 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:26.624 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:26.624 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:26.624 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:26.624 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.624 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:26.624 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:26.624 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:26.624 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:26.624 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:26.624 INFO:tasks.workunit.client.0.smithi100.stdout: 5 2024-04-07T17:51:26.624 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.624 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 1, 2024-04-07T17:51:26.624 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:26.624 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.624 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:26.624 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:26.624 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:26.625 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:26.625 INFO:tasks.workunit.client.0.smithi100.stdout: 5 2024-04-07T17:51:26.625 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.625 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 0, 2024-04-07T17:51:26.625 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:26.625 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:26.625 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:26.625 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:26.625 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:26.625 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:26.625 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:35.984431+0000", 2024-04-07T17:51:26.625 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:24:41.538233+0000", 2024-04-07T17:51:26.625 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:24:41.538233+0000", 2024-04-07T17:51:26.625 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:24:41.538718+0000", 2024-04-07T17:51:26.625 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:35.984431+0000", 2024-04-07T17:51:26.625 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.625 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:26.626 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.626 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 527, 2024-04-07T17:51:26.626 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:35.984431+0000", 2024-04-07T17:51:26.626 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:35.984431+0000", 2024-04-07T17:51:26.626 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:35.984431+0000", 2024-04-07T17:51:26.626 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:26.626 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:26.626 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.626 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:35.984431+0000", 2024-04-07T17:51:26.626 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:35.984431+0000", 2024-04-07T17:51:26.626 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:26.626 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:26.626 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:26.626 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:26.627 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 526, 2024-04-07T17:51:26.627 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:26.627 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:26.627 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:26.627 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:26.627 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:26.627 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:26.627 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:26.627 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:26.627 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.32", 2024-04-07T17:51:26.627 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:26.627 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:26.627 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:26.627 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4995, 2024-04-07T17:51:26.627 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:26.627 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T22:18:55.330487+0000", 2024-04-07T17:51:26.627 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:26.628 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:26.628 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:26.628 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:26.628 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:26.628 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:26.628 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:26.628 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:26.628 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:26.628 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:26.628 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:26.628 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:26.628 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:26.628 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:26.628 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:26.628 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:26.628 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:26.629 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:26.629 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:26.629 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:26.629 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:26.629 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:26.629 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:26.629 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:26.629 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:26.629 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:26.629 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:26.629 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:26.629 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:26.629 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:26.629 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:26.629 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:26.629 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:26.630 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:26.630 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:26.630 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:26.630 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:26.630 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:26.630 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:26.630 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:26.630 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:26.630 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:26.630 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:26.630 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.630 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:26.630 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:26.630 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:26.630 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:26.631 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:26.631 INFO:tasks.workunit.client.0.smithi100.stdout: 5 2024-04-07T17:51:26.631 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.631 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 0, 2024-04-07T17:51:26.631 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:26.631 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.631 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:26.631 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:26.631 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:26.631 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:26.631 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T17:51:26.631 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.631 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 3, 2024-04-07T17:51:26.631 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:26.631 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:26.631 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:26.632 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:26.632 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:26.632 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:26.632 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:35.977234+0000", 2024-04-07T17:51:26.632 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:08.976145+0000", 2024-04-07T17:51:26.632 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:08.976145+0000", 2024-04-07T17:51:26.632 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:08.976499+0000", 2024-04-07T17:51:26.632 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:35.977234+0000", 2024-04-07T17:51:26.632 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.632 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:26.632 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.632 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 554, 2024-04-07T17:51:26.632 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:35.977234+0000", 2024-04-07T17:51:26.632 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:35.977234+0000", 2024-04-07T17:51:26.632 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:35.977234+0000", 2024-04-07T17:51:26.632 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:26.632 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:26.633 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.633 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:35.977234+0000", 2024-04-07T17:51:26.633 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:35.977234+0000", 2024-04-07T17:51:26.633 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:26.633 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:26.633 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:26.633 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:26.633 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 553, 2024-04-07T17:51:26.633 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:26.633 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:26.633 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:26.633 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:26.633 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:26.633 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:26.633 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:26.633 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:26.634 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.33", 2024-04-07T17:51:26.634 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:26.634 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:26.634 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:26.634 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4963, 2024-04-07T17:51:26.634 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:26.634 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T04:02:25.054838+0000", 2024-04-07T17:51:26.634 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:26.634 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:26.634 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:26.634 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:26.634 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:26.634 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:26.634 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:26.634 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:26.634 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:26.635 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:26.635 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:26.635 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:26.635 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:26.635 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:26.635 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:26.635 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:26.635 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:26.635 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:26.635 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:26.635 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:26.635 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:26.635 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:26.635 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:26.635 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:26.636 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:26.636 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:26.636 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:26.636 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:26.636 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:26.636 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:26.636 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:26.636 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:26.636 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:26.636 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:26.636 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:26.636 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:26.636 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:26.636 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:26.636 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:26.636 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:26.637 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:26.637 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:26.637 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:26.637 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.637 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:26.637 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:26.637 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:26.637 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:26.637 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:26.637 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T17:51:26.637 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.637 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 3, 2024-04-07T17:51:26.637 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:26.637 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.637 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:26.637 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:26.638 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:26.638 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:26.638 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T17:51:26.638 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.638 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 4, 2024-04-07T17:51:26.638 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:26.638 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:26.638 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:26.638 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:26.638 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:26.638 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:26.638 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:35.980246+0000", 2024-04-07T17:51:26.638 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:08.955654+0000", 2024-04-07T17:51:26.638 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:08.955654+0000", 2024-04-07T17:51:26.638 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:08.956220+0000", 2024-04-07T17:51:26.638 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:35.980246+0000", 2024-04-07T17:51:26.638 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.639 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:26.639 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.639 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 554, 2024-04-07T17:51:26.639 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:35.980246+0000", 2024-04-07T17:51:26.639 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:35.980246+0000", 2024-04-07T17:51:26.639 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:35.980246+0000", 2024-04-07T17:51:26.639 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:26.639 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:26.639 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.639 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:35.980246+0000", 2024-04-07T17:51:26.639 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:35.980246+0000", 2024-04-07T17:51:26.639 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:26.639 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:26.639 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:26.639 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:26.639 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 553, 2024-04-07T17:51:26.639 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:26.639 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:26.640 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:26.640 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:26.640 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:26.640 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:26.640 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:26.640 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:26.640 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.34", 2024-04-07T17:51:26.640 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:26.640 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:26.640 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:26.640 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4913, 2024-04-07T17:51:26.640 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:26.640 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T00:20:14.081274+0000", 2024-04-07T17:51:26.640 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:26.640 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:26.640 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:26.641 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:26.641 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:26.641 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:26.641 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:26.641 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:26.641 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:26.641 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:26.641 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:26.641 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:26.641 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:26.641 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:26.641 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:26.641 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:26.641 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:26.641 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:26.641 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:26.642 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:26.642 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:26.642 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:26.642 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:26.642 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:26.642 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:26.642 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:26.642 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:26.642 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:26.642 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:26.642 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:26.642 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:26.642 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:26.642 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:26.642 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:26.642 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:26.643 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:26.643 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:26.643 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:26.643 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:26.643 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:26.643 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:26.643 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:26.643 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:26.643 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.643 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:26.643 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:26.643 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:26.643 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:26.643 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:26.643 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T17:51:26.643 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.644 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 4, 2024-04-07T17:51:26.644 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:26.644 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.644 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:26.644 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:26.644 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:26.644 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:26.644 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T17:51:26.644 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.644 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 6, 2024-04-07T17:51:26.644 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:26.644 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:26.644 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:26.644 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:26.644 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:26.644 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:26.645 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.359086+0000", 2024-04-07T17:51:26.645 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:08.978750+0000", 2024-04-07T17:51:26.645 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:08.978750+0000", 2024-04-07T17:51:26.645 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:08.979235+0000", 2024-04-07T17:51:26.645 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.359086+0000", 2024-04-07T17:51:26.645 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.645 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:26.645 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.645 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 554, 2024-04-07T17:51:26.645 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.359086+0000", 2024-04-07T17:51:26.645 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.359086+0000", 2024-04-07T17:51:26.645 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.359086+0000", 2024-04-07T17:51:26.645 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:26.645 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:26.645 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.646 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.359086+0000", 2024-04-07T17:51:26.646 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.359086+0000", 2024-04-07T17:51:26.646 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:26.646 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:26.646 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:26.646 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:26.646 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 553, 2024-04-07T17:51:26.646 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:26.646 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:26.646 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:26.646 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:26.646 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:26.646 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:26.646 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:26.646 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:26.647 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.35", 2024-04-07T17:51:26.647 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:26.647 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:26.647 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:26.647 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4828, 2024-04-07T17:51:26.647 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:26.647 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T03:27:45.790517+0000", 2024-04-07T17:51:26.647 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:26.647 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:26.647 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:26.647 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:26.647 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:26.647 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:26.647 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:26.647 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:26.648 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:26.648 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:26.648 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:26.648 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:26.648 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:26.648 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:26.648 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:26.648 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:26.648 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:26.648 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:26.648 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:26.648 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:26.648 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:26.648 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:26.648 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:26.648 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:26.648 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:26.649 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:26.649 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:26.649 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:26.649 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:26.649 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:26.649 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:26.649 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:26.649 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:26.649 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:26.649 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:26.649 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:26.649 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:26.649 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:26.649 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:26.649 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:26.649 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:26.649 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:26.650 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:26.650 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.650 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:26.650 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:26.650 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:26.650 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:26.650 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:26.650 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T17:51:26.650 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.650 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 6, 2024-04-07T17:51:26.650 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:26.650 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.650 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:26.650 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:26.650 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:26.651 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:26.651 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T17:51:26.651 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.651 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 6, 2024-04-07T17:51:26.651 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:26.651 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:26.651 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:26.651 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:26.651 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:26.651 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:26.651 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.368833+0000", 2024-04-07T17:51:26.651 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:08.977646+0000", 2024-04-07T17:51:26.651 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:08.977646+0000", 2024-04-07T17:51:26.651 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:08.978144+0000", 2024-04-07T17:51:26.651 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.368833+0000", 2024-04-07T17:51:26.651 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.651 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:26.652 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.652 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 554, 2024-04-07T17:51:26.652 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.368833+0000", 2024-04-07T17:51:26.652 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.368833+0000", 2024-04-07T17:51:26.652 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.368833+0000", 2024-04-07T17:51:26.652 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:26.652 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:26.652 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.652 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.368833+0000", 2024-04-07T17:51:26.652 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.368833+0000", 2024-04-07T17:51:26.652 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:26.652 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:26.652 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:26.652 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:26.652 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 553, 2024-04-07T17:51:26.653 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:26.653 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:26.653 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:26.653 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:26.653 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:26.653 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:26.653 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:26.653 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:26.653 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.36", 2024-04-07T17:51:26.653 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:26.653 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:26.653 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:26.653 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4828, 2024-04-07T17:51:26.653 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:26.653 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T01:10:43.702742+0000", 2024-04-07T17:51:26.654 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:26.654 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:26.654 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:26.654 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:26.654 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:26.654 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:26.654 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:26.654 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:26.654 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:26.654 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:26.654 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:26.654 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:26.654 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:26.654 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:26.654 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:26.654 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:26.655 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:26.655 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:26.655 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:26.655 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:26.655 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:26.655 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:26.655 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:26.655 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:26.655 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:26.655 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:26.655 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:26.655 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:26.655 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:26.655 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:26.655 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:26.656 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:26.656 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:26.656 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:26.656 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:26.656 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:26.656 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:26.656 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:26.656 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:26.656 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:26.656 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:26.656 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:26.656 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:26.656 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.656 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:26.657 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:26.657 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:26.657 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:26.657 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:26.657 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T17:51:26.657 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.657 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 6, 2024-04-07T17:51:26.657 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:26.657 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.657 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:26.657 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:26.657 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:26.657 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:26.657 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T17:51:26.657 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.657 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 7, 2024-04-07T17:51:26.658 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:26.658 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:26.658 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:26.658 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:26.658 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:26.658 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:26.658 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.366733+0000", 2024-04-07T17:51:26.658 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:38.968877+0000", 2024-04-07T17:51:26.658 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:38.968877+0000", 2024-04-07T17:51:26.658 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:38.976137+0000", 2024-04-07T17:51:26.658 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.366733+0000", 2024-04-07T17:51:26.658 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.658 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:26.658 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.658 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:26.658 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.366733+0000", 2024-04-07T17:51:26.659 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.366733+0000", 2024-04-07T17:51:26.659 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.366733+0000", 2024-04-07T17:51:26.659 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:26.659 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:26.659 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.659 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.366733+0000", 2024-04-07T17:51:26.659 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.366733+0000", 2024-04-07T17:51:26.659 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:26.659 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:26.659 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:26.659 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:26.659 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:26.659 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:26.659 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:26.659 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:26.659 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:26.659 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:26.659 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:26.660 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:26.660 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:26.660 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.37", 2024-04-07T17:51:26.660 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:26.660 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:26.660 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:26.660 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4815, 2024-04-07T17:51:26.660 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:26.660 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T17:38:48.549583+0000", 2024-04-07T17:51:26.660 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:26.660 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:26.660 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:26.660 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:26.660 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:26.660 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:26.660 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:26.660 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:26.661 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:26.661 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:26.661 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:26.661 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:26.661 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:26.661 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:26.661 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:26.661 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:26.661 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:26.661 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:26.661 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:26.661 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:26.661 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:26.661 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:26.661 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:26.661 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:26.661 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:26.661 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:26.662 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:26.662 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:26.662 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:26.662 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:26.662 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:26.662 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:26.662 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:26.662 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:26.662 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:26.662 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:26.662 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:26.662 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:26.662 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:26.662 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:26.662 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:26.663 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:26.663 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:26.663 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.663 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:26.663 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:26.663 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:26.663 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:26.663 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:26.663 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T17:51:26.663 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.663 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 7, 2024-04-07T17:51:26.663 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:26.663 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.663 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:26.663 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:26.663 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:26.663 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:26.664 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T17:51:26.664 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.664 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 3, 2024-04-07T17:51:26.664 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:26.664 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:26.664 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:26.664 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:26.664 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:26.664 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:26.664 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:35.975131+0000", 2024-04-07T17:51:26.664 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:08.977725+0000", 2024-04-07T17:51:26.664 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:08.977725+0000", 2024-04-07T17:51:26.664 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:08.978090+0000", 2024-04-07T17:51:26.664 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:35.975131+0000", 2024-04-07T17:51:26.664 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.664 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:26.664 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.665 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 554, 2024-04-07T17:51:26.665 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:35.975131+0000", 2024-04-07T17:51:26.665 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:35.975131+0000", 2024-04-07T17:51:26.665 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:35.975131+0000", 2024-04-07T17:51:26.665 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:26.665 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:26.665 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.665 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:35.975131+0000", 2024-04-07T17:51:26.665 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:35.975131+0000", 2024-04-07T17:51:26.665 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:26.665 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:26.665 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:26.665 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:26.665 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 553, 2024-04-07T17:51:26.665 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:26.665 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:26.666 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:26.666 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:26.666 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:26.666 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:26.666 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:26.666 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:26.666 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.38", 2024-04-07T17:51:26.666 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:26.666 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:26.666 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:26.666 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4929, 2024-04-07T17:51:26.666 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:26.666 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T18:02:53.604912+0000", 2024-04-07T17:51:26.666 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:26.667 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:26.667 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:26.667 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:26.667 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:26.667 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:26.667 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:26.667 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:26.667 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:26.667 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:26.667 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:26.667 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:26.667 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:26.667 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:26.667 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:26.667 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:26.667 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:26.668 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:26.668 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:26.668 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:26.668 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:26.668 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:26.668 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:26.668 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:26.668 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:26.668 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:26.668 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:26.668 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:26.668 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:26.668 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:26.668 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:26.668 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:26.668 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:26.668 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:26.669 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:26.669 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:26.669 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:26.669 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:26.669 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:26.669 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:26.669 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:26.669 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:26.669 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:26.669 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.669 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:26.669 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:26.669 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:26.669 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:26.669 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:26.669 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T17:51:26.669 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.670 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 3, 2024-04-07T17:51:26.670 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:26.670 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.670 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:26.670 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:26.670 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:26.670 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:26.670 INFO:tasks.workunit.client.0.smithi100.stdout: 4 2024-04-07T17:51:26.670 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.670 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 7, 2024-04-07T17:51:26.670 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:26.670 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:26.670 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:26.670 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:26.670 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:26.671 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:26.671 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.358816+0000", 2024-04-07T17:51:26.671 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:38.996215+0000", 2024-04-07T17:51:26.671 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:38.996215+0000", 2024-04-07T17:51:26.671 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:39.000120+0000", 2024-04-07T17:51:26.671 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.358816+0000", 2024-04-07T17:51:26.671 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.671 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:26.671 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.671 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:26.671 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.358816+0000", 2024-04-07T17:51:26.671 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.358816+0000", 2024-04-07T17:51:26.671 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.358816+0000", 2024-04-07T17:51:26.671 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:26.671 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:26.671 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.672 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.358816+0000", 2024-04-07T17:51:26.672 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.358816+0000", 2024-04-07T17:51:26.672 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:26.672 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:26.672 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:26.672 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:26.672 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:26.672 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:26.672 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:26.672 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:26.672 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:26.672 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:26.672 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:26.672 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:26.672 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:26.672 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.39", 2024-04-07T17:51:26.672 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:26.673 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:26.673 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:26.673 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4787, 2024-04-07T17:51:26.673 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:26.673 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T22:36:35.618662+0000", 2024-04-07T17:51:26.673 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:26.673 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:26.673 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:26.673 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:26.673 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:26.673 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:26.673 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:26.673 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:26.673 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:26.673 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:26.673 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:26.674 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:26.674 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:26.674 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:26.674 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:26.674 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:26.674 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:26.674 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:26.674 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:26.674 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:26.674 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:26.674 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:26.674 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:26.674 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:26.674 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:26.675 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:26.675 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:26.675 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:26.675 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:26.675 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:26.675 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:26.675 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:26.675 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:26.675 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:26.675 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:26.675 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:26.675 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:26.675 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:26.675 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:26.675 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:26.675 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:26.676 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:26.676 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:26.676 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.676 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:26.676 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:26.676 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:26.676 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:26.676 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:26.676 INFO:tasks.workunit.client.0.smithi100.stdout: 4 2024-04-07T17:51:26.676 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.676 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 7, 2024-04-07T17:51:26.676 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:26.676 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.676 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:26.676 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:26.677 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:26.677 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:26.677 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T17:51:26.677 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.677 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 5, 2024-04-07T17:51:26.677 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:26.677 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:26.677 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:26.677 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:26.677 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:26.677 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:26.677 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.368733+0000", 2024-04-07T17:51:26.677 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:24:41.548397+0000", 2024-04-07T17:51:26.677 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:24:41.548397+0000", 2024-04-07T17:51:26.677 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:24:41.551443+0000", 2024-04-07T17:51:26.678 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.368733+0000", 2024-04-07T17:51:26.678 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.678 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:26.678 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.678 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 527, 2024-04-07T17:51:26.678 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.368733+0000", 2024-04-07T17:51:26.678 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.368733+0000", 2024-04-07T17:51:26.678 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.368733+0000", 2024-04-07T17:51:26.678 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:26.678 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:26.678 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.678 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.368733+0000", 2024-04-07T17:51:26.678 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.368733+0000", 2024-04-07T17:51:26.678 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:26.678 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:26.679 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:26.679 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:26.679 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 526, 2024-04-07T17:51:26.679 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:26.679 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:26.679 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:26.679 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:26.679 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:26.679 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:26.679 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:26.679 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:26.679 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.3a", 2024-04-07T17:51:26.679 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:26.679 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:26.679 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:26.679 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4865, 2024-04-07T17:51:26.679 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:26.679 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T20:38:47.135272+0000", 2024-04-07T17:51:26.680 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:26.680 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:26.680 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:26.680 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:26.680 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:26.680 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:26.680 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:26.680 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:26.680 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:26.680 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:26.680 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:26.680 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:26.680 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:26.680 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:26.680 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:26.680 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:26.681 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:26.681 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:26.681 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:26.681 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:26.681 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:26.681 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:26.681 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:26.681 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:26.681 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:26.681 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:26.681 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:26.681 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:26.681 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:26.681 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:26.681 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:26.681 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:26.682 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:26.682 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:26.682 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:26.682 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:26.682 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:26.682 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:26.682 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:26.682 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:26.682 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:26.682 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:26.682 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:26.682 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.682 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:26.682 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:26.683 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:26.683 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:26.683 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:26.683 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T17:51:26.683 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.683 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 5, 2024-04-07T17:51:26.683 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:26.683 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.683 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:26.683 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:26.683 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:26.683 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T17:51:26.683 INFO:tasks.workunit.client.0.smithi100.stdout: 4 2024-04-07T17:51:26.683 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.683 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 7, 2024-04-07T17:51:26.683 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:26.684 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:26.684 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:26.684 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:26.684 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:26.684 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:26.684 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.369564+0000", 2024-04-07T17:51:26.684 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:38.985715+0000", 2024-04-07T17:51:26.684 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:38.985715+0000", 2024-04-07T17:51:26.684 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:38.990756+0000", 2024-04-07T17:51:26.684 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.369564+0000", 2024-04-07T17:51:26.684 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.684 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:26.684 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.684 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:26.684 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.369564+0000", 2024-04-07T17:51:26.684 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.369564+0000", 2024-04-07T17:51:26.685 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.369564+0000", 2024-04-07T17:51:26.685 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:26.685 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:26.685 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.685 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.369564+0000", 2024-04-07T17:51:26.685 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.369564+0000", 2024-04-07T17:51:26.685 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:26.685 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:26.685 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:26.685 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:26.685 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:26.685 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:26.685 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:26.685 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:26.685 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:26.685 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:26.686 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:26.686 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:26.686 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:26.686 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.3b", 2024-04-07T17:51:26.686 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:26.686 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:26.686 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:26.686 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4787, 2024-04-07T17:51:26.686 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:26.686 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T17:50:57.375885+0000", 2024-04-07T17:51:26.686 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:26.686 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:26.686 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:26.686 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:26.686 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:26.687 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:26.687 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:26.687 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:26.687 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:26.687 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:26.687 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:26.687 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:26.687 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:26.687 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:26.687 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:26.687 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:26.687 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:26.687 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:26.687 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:26.687 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:26.687 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:26.688 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:26.688 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:26.688 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:26.688 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:26.688 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:26.688 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:26.688 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:26.688 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:26.688 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:26.688 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:26.688 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:26.688 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:26.688 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:26.688 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:26.689 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:26.689 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:26.689 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:26.689 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:26.689 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:26.689 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:26.689 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:26.689 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:26.689 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.689 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:26.689 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:26.689 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:26.689 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:26.689 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T17:51:26.689 INFO:tasks.workunit.client.0.smithi100.stdout: 4 2024-04-07T17:51:26.689 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.690 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 7, 2024-04-07T17:51:26.690 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:26.690 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.690 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:26.690 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:26.690 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:26.690 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:26.690 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T17:51:26.690 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.690 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 5, 2024-04-07T17:51:26.690 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:26.690 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:26.690 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:26.690 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:26.691 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:26.691 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:26.691 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.365779+0000", 2024-04-07T17:51:26.691 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:38.986923+0000", 2024-04-07T17:51:26.691 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:38.986923+0000", 2024-04-07T17:51:26.691 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:38.987454+0000", 2024-04-07T17:51:26.691 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.365779+0000", 2024-04-07T17:51:26.691 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.691 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:26.691 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.691 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:26.691 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.365779+0000", 2024-04-07T17:51:26.691 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.365779+0000", 2024-04-07T17:51:26.691 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.365779+0000", 2024-04-07T17:51:26.691 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:26.691 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:26.692 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.692 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.365779+0000", 2024-04-07T17:51:26.692 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.365779+0000", 2024-04-07T17:51:26.692 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:26.692 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:26.692 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:26.692 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:26.692 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:26.692 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:26.692 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:26.692 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:26.692 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:26.692 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:26.692 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:26.692 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:26.693 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:26.693 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.3c", 2024-04-07T17:51:26.693 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:26.693 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:26.693 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:26.693 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4883, 2024-04-07T17:51:26.693 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:26.693 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T02:20:58.654636+0000", 2024-04-07T17:51:26.693 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:26.693 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:26.693 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:26.693 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:26.693 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:26.693 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:26.693 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:26.693 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:26.693 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:26.694 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:26.694 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:26.694 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:26.694 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:26.694 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:26.694 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:26.694 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:26.694 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:26.694 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:26.694 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:26.694 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:26.694 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:26.694 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:26.694 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:26.694 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:26.695 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:26.695 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:26.695 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:26.695 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:26.695 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:26.695 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:26.695 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:26.695 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:26.695 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:26.695 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:26.695 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:26.695 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:26.695 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:26.695 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:26.695 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:26.695 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:26.696 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:26.696 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:26.696 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:26.696 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.696 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:26.696 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:26.696 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:26.696 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:26.696 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:26.696 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T17:51:26.696 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.696 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 5, 2024-04-07T17:51:26.696 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:26.696 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.696 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:26.696 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:26.697 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:26.697 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T17:51:26.697 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T17:51:26.697 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.697 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 0, 2024-04-07T17:51:26.697 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:26.697 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:26.697 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:26.697 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:26.697 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:26.697 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:26.697 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:35.982778+0000", 2024-04-07T17:51:26.697 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:23:15.876439+0000", 2024-04-07T17:51:26.697 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:23:15.876439+0000", 2024-04-07T17:51:26.697 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:23:15.876856+0000", 2024-04-07T17:51:26.697 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:35.982778+0000", 2024-04-07T17:51:26.698 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.698 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:26.698 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.698 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 445, 2024-04-07T17:51:26.698 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:35.982778+0000", 2024-04-07T17:51:26.698 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:35.982778+0000", 2024-04-07T17:51:26.698 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:35.982778+0000", 2024-04-07T17:51:26.698 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:26.698 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:26.698 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.698 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:35.982778+0000", 2024-04-07T17:51:26.698 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:35.982778+0000", 2024-04-07T17:51:26.698 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:26.698 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:26.698 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:26.699 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:26.699 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 444, 2024-04-07T17:51:26.699 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:26.699 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:26.699 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:26.699 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:26.699 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:26.699 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:26.699 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:26.699 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:26.699 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.3d", 2024-04-07T17:51:26.699 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:26.699 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:26.699 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:26.699 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 5000, 2024-04-07T17:51:26.699 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:26.699 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T03:48:36.975141+0000", 2024-04-07T17:51:26.700 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:26.700 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:26.700 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:26.700 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:26.700 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:26.700 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:26.700 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:26.700 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:26.700 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:26.700 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:26.700 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:26.700 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:26.700 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:26.700 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:26.700 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:26.700 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:26.700 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:26.701 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:26.701 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:26.701 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:26.701 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:26.701 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:26.701 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:26.701 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:26.701 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:26.701 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:26.701 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:26.701 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:26.701 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:26.701 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:26.701 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:26.701 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:26.701 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:26.701 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:26.702 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:26.702 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:26.702 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:26.702 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:26.702 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:26.702 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:26.702 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:26.702 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:26.702 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:26.702 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.702 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:26.702 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:26.702 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:26.702 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:26.702 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T17:51:26.703 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T17:51:26.703 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.703 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 0, 2024-04-07T17:51:26.703 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:26.703 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.703 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:26.703 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:26.703 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:26.703 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:26.703 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T17:51:26.703 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.703 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 1, 2024-04-07T17:51:26.703 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:26.703 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:26.703 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:26.704 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:26.704 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:26.704 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:26.704 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.334821+0000", 2024-04-07T17:51:26.704 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:08.962145+0000", 2024-04-07T17:51:26.704 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:08.962145+0000", 2024-04-07T17:51:26.704 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:08.962703+0000", 2024-04-07T17:51:26.704 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.334821+0000", 2024-04-07T17:51:26.704 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.704 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:26.704 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.704 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 554, 2024-04-07T17:51:26.704 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.334821+0000", 2024-04-07T17:51:26.704 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.334821+0000", 2024-04-07T17:51:26.704 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.334821+0000", 2024-04-07T17:51:26.704 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:26.705 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:26.705 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.705 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.334821+0000", 2024-04-07T17:51:26.705 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.334821+0000", 2024-04-07T17:51:26.705 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:26.705 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:26.705 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:26.705 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:26.705 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 553, 2024-04-07T17:51:26.705 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:26.705 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:26.705 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:26.705 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:26.705 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:26.705 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:26.705 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:26.706 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:26.706 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.3e", 2024-04-07T17:51:26.706 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:26.706 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:26.706 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:26.706 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4989, 2024-04-07T17:51:26.706 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:26.706 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T23:09:45.491615+0000", 2024-04-07T17:51:26.706 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:26.706 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:26.706 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:26.706 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:26.706 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:26.706 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:26.707 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:26.707 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:26.707 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:26.707 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:26.707 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:26.707 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:26.707 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:26.707 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:26.707 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:26.707 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:26.707 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:26.707 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:26.707 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:26.707 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:26.707 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:26.707 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:26.707 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:26.708 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:26.708 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:26.708 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:26.708 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:26.708 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:26.708 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:26.708 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:26.708 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:26.708 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:26.708 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:26.708 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:26.708 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:26.708 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:26.708 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:26.708 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:26.709 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:26.709 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:26.709 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:26.709 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:26.709 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:26.709 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.709 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:26.709 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:26.709 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:26.709 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:26.709 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:26.709 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T17:51:26.709 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.709 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 1, 2024-04-07T17:51:26.709 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:26.709 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.710 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:26.710 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:26.710 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:26.710 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:26.710 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T17:51:26.710 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.710 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 7, 2024-04-07T17:51:26.710 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:26.710 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:26.710 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:26.710 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:26.710 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:26.710 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:26.710 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.367170+0000", 2024-04-07T17:51:26.710 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:38.966876+0000", 2024-04-07T17:51:26.711 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:38.966876+0000", 2024-04-07T17:51:26.711 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:38.969419+0000", 2024-04-07T17:51:26.711 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.367170+0000", 2024-04-07T17:51:26.711 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.711 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:26.711 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.711 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:26.711 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.367170+0000", 2024-04-07T17:51:26.711 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.367170+0000", 2024-04-07T17:51:26.711 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.367170+0000", 2024-04-07T17:51:26.711 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:26.711 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:26.711 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:26.711 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.367170+0000", 2024-04-07T17:51:26.711 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.367170+0000", 2024-04-07T17:51:26.711 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:26.712 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:26.712 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:26.712 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:26.712 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:26.712 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:26.712 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:26.712 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:26.712 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:26.712 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:26.712 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:26.712 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:26.712 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:26.712 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.3f", 2024-04-07T17:51:26.712 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:26.712 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:26.712 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:26.712 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4807, 2024-04-07T17:51:26.713 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:26.713 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T01:03:39.405808+0000", 2024-04-07T17:51:26.713 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:26.713 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:26.713 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:26.713 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:26.713 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:26.713 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:26.713 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:26.713 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:26.713 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:26.713 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:26.713 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:26.713 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:26.713 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:26.713 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:26.714 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:26.714 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:26.714 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:26.714 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:26.714 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:26.714 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:26.714 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:26.714 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:26.714 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:26.714 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:26.714 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:26.714 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:26.714 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:26.714 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:26.715 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:26.715 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:26.715 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:26.715 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:26.715 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:26.715 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:26.715 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:26.715 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:26.715 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:26.715 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:26.715 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:26.715 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:26.715 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:26.715 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:26.715 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:26.715 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.716 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:26.716 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:26.716 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:26.716 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:26.716 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:26.716 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T17:51:26.716 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.716 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 7, 2024-04-07T17:51:26.716 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:26.716 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:26.716 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.716 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_per_pool": [ 2024-04-07T17:51:26.716 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:26.716 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": 24, 2024-04-07T17:51:26.716 INFO:tasks.workunit.client.0.smithi100.stdout: "application": [ 2024-04-07T17:51:26.717 INFO:tasks.workunit.client.0.smithi100.stdout: "rbd" 2024-04-07T17:51:26.717 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.717 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 24403, 2024-04-07T17:51:26.717 INFO:tasks.workunit.client.0.smithi100.stdout: "num_pg": 8, 2024-04-07T17:51:26.717 INFO:tasks.workunit.client.0.smithi100.stdout: "num_store_stats": 8, 2024-04-07T17:51:26.717 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 24403, 2024-04-07T17:51:26.717 INFO:tasks.workunit.client.0.smithi100.stdout: "poolid": 4, 2024-04-07T17:51:26.717 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:26.717 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 4194323, 2024-04-07T17:51:26.717 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:26.717 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:26.717 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:26.717 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:26.717 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:26.717 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:26.717 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:26.718 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:26.718 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:26.718 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:26.718 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:26.718 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:26.718 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:26.718 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:26.718 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:26.718 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 12, 2024-04-07T17:51:26.718 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 4, 2024-04-07T17:51:26.718 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:26.718 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 4, 2024-04-07T17:51:26.718 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:26.718 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:26.718 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:26.719 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:26.719 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:26.719 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 2, 2024-04-07T17:51:26.719 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:26.719 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:26.719 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:26.719 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:26.719 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:26.719 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:26.719 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:26.719 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 9089, 2024-04-07T17:51:26.719 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 3147278, 2024-04-07T17:51:26.719 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:26.719 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:26.719 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:26.720 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 24273, 2024-04-07T17:51:26.720 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 3030917 2024-04-07T17:51:26.720 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.720 INFO:tasks.workunit.client.0.smithi100.stdout: "store_stats": { 2024-04-07T17:51:26.720 INFO:tasks.workunit.client.0.smithi100.stdout: "allocated": 12595200, 2024-04-07T17:51:26.720 INFO:tasks.workunit.client.0.smithi100.stdout: "available": 0, 2024-04-07T17:51:26.720 INFO:tasks.workunit.client.0.smithi100.stdout: "data_compressed": 0, 2024-04-07T17:51:26.720 INFO:tasks.workunit.client.0.smithi100.stdout: "data_compressed_allocated": 0, 2024-04-07T17:51:26.720 INFO:tasks.workunit.client.0.smithi100.stdout: "data_compressed_original": 0, 2024-04-07T17:51:26.720 INFO:tasks.workunit.client.0.smithi100.stdout: "data_stored": 12582969, 2024-04-07T17:51:26.720 INFO:tasks.workunit.client.0.smithi100.stdout: "internal_metadata": 0, 2024-04-07T17:51:26.720 INFO:tasks.workunit.client.0.smithi100.stdout: "internally_reserved": 0, 2024-04-07T17:51:26.720 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_allocated": 0, 2024-04-07T17:51:26.720 INFO:tasks.workunit.client.0.smithi100.stdout: "total": 0 2024-04-07T17:51:26.720 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.720 INFO:tasks.workunit.client.0.smithi100.stdout: "up": 24 2024-04-07T17:51:26.721 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.721 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:26.721 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": 192, 2024-04-07T17:51:26.721 INFO:tasks.workunit.client.0.smithi100.stdout: "application": [ 2024-04-07T17:51:26.721 INFO:tasks.workunit.client.0.smithi100.stdout: "cephfs" 2024-04-07T17:51:26.721 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.721 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:26.721 INFO:tasks.workunit.client.0.smithi100.stdout: "num_pg": 64, 2024-04-07T17:51:26.721 INFO:tasks.workunit.client.0.smithi100.stdout: "num_store_stats": 8, 2024-04-07T17:51:26.721 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:26.721 INFO:tasks.workunit.client.0.smithi100.stdout: "poolid": 3, 2024-04-07T17:51:26.721 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:26.721 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:26.721 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:26.721 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:26.721 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:26.722 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:26.722 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:26.722 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:26.722 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:26.722 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:26.722 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:26.722 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:26.722 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:26.722 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:26.722 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:26.722 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:26.722 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:26.722 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:26.722 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:26.723 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:26.723 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:26.723 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:26.723 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:26.723 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:26.723 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:26.723 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:26.723 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:26.723 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:26.723 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:26.723 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:26.723 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:26.723 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:26.723 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:26.723 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:26.723 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:26.724 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:26.724 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:26.724 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:26.724 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:26.724 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:26.724 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:26.724 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.724 INFO:tasks.workunit.client.0.smithi100.stdout: "store_stats": { 2024-04-07T17:51:26.724 INFO:tasks.workunit.client.0.smithi100.stdout: "allocated": 0, 2024-04-07T17:51:26.724 INFO:tasks.workunit.client.0.smithi100.stdout: "available": 0, 2024-04-07T17:51:26.724 INFO:tasks.workunit.client.0.smithi100.stdout: "data_compressed": 0, 2024-04-07T17:51:26.724 INFO:tasks.workunit.client.0.smithi100.stdout: "data_compressed_allocated": 0, 2024-04-07T17:51:26.724 INFO:tasks.workunit.client.0.smithi100.stdout: "data_compressed_original": 0, 2024-04-07T17:51:26.724 INFO:tasks.workunit.client.0.smithi100.stdout: "data_stored": 0, 2024-04-07T17:51:26.724 INFO:tasks.workunit.client.0.smithi100.stdout: "internal_metadata": 0, 2024-04-07T17:51:26.724 INFO:tasks.workunit.client.0.smithi100.stdout: "internally_reserved": 0, 2024-04-07T17:51:26.725 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_allocated": 0, 2024-04-07T17:51:26.725 INFO:tasks.workunit.client.0.smithi100.stdout: "total": 0 2024-04-07T17:51:26.725 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.725 INFO:tasks.workunit.client.0.smithi100.stdout: "up": 192 2024-04-07T17:51:26.725 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.725 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:26.725 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": 3, 2024-04-07T17:51:26.725 INFO:tasks.workunit.client.0.smithi100.stdout: "application": [ 2024-04-07T17:51:26.725 INFO:tasks.workunit.client.0.smithi100.stdout: "mgr" 2024-04-07T17:51:26.725 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.725 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 241, 2024-04-07T17:51:26.725 INFO:tasks.workunit.client.0.smithi100.stdout: "num_pg": 1, 2024-04-07T17:51:26.725 INFO:tasks.workunit.client.0.smithi100.stdout: "num_store_stats": 3, 2024-04-07T17:51:26.725 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 241, 2024-04-07T17:51:26.725 INFO:tasks.workunit.client.0.smithi100.stdout: "poolid": 1, 2024-04-07T17:51:26.725 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:26.726 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 590368, 2024-04-07T17:51:26.726 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:26.726 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 1771104, 2024-04-07T17:51:26.726 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:26.726 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:26.726 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:26.726 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:26.726 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:26.726 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:26.726 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:26.726 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:26.726 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:26.726 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:26.726 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:26.727 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:26.727 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:26.727 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 6, 2024-04-07T17:51:26.727 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 2, 2024-04-07T17:51:26.727 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:26.727 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 2, 2024-04-07T17:51:26.727 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:26.727 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:26.727 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:26.727 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:26.727 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:26.727 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:26.727 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:26.727 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 6, 2024-04-07T17:51:26.727 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:26.727 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:26.728 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:26.728 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:26.728 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:26.728 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 1214, 2024-04-07T17:51:26.728 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 2024, 2024-04-07T17:51:26.728 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:26.728 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:26.728 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:26.728 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 399, 2024-04-07T17:51:26.728 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 2416 2024-04-07T17:51:26.728 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.728 INFO:tasks.workunit.client.0.smithi100.stdout: "store_stats": { 2024-04-07T17:51:26.728 INFO:tasks.workunit.client.0.smithi100.stdout: "allocated": 1781760, 2024-04-07T17:51:26.728 INFO:tasks.workunit.client.0.smithi100.stdout: "available": 0, 2024-04-07T17:51:26.728 INFO:tasks.workunit.client.0.smithi100.stdout: "data_compressed": 0, 2024-04-07T17:51:26.728 INFO:tasks.workunit.client.0.smithi100.stdout: "data_compressed_allocated": 0, 2024-04-07T17:51:26.729 INFO:tasks.workunit.client.0.smithi100.stdout: "data_compressed_original": 0, 2024-04-07T17:51:26.729 INFO:tasks.workunit.client.0.smithi100.stdout: "data_stored": 1771104, 2024-04-07T17:51:26.729 INFO:tasks.workunit.client.0.smithi100.stdout: "internal_metadata": 0, 2024-04-07T17:51:26.729 INFO:tasks.workunit.client.0.smithi100.stdout: "internally_reserved": 0, 2024-04-07T17:51:26.729 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_allocated": 0, 2024-04-07T17:51:26.729 INFO:tasks.workunit.client.0.smithi100.stdout: "total": 0 2024-04-07T17:51:26.729 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.729 INFO:tasks.workunit.client.0.smithi100.stdout: "up": 3 2024-04-07T17:51:26.729 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.729 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:26.729 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": 96, 2024-04-07T17:51:26.729 INFO:tasks.workunit.client.0.smithi100.stdout: "application": [ 2024-04-07T17:51:26.729 INFO:tasks.workunit.client.0.smithi100.stdout: "cephfs" 2024-04-07T17:51:26.729 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.729 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 68, 2024-04-07T17:51:26.730 INFO:tasks.workunit.client.0.smithi100.stdout: "num_pg": 32, 2024-04-07T17:51:26.730 INFO:tasks.workunit.client.0.smithi100.stdout: "num_store_stats": 8, 2024-04-07T17:51:26.730 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 68, 2024-04-07T17:51:26.730 INFO:tasks.workunit.client.0.smithi100.stdout: "poolid": 2, 2024-04-07T17:51:26.730 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:26.730 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 6622, 2024-04-07T17:51:26.730 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:26.730 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:26.730 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:26.730 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:26.730 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:26.730 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:26.730 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:26.730 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:26.731 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:26.731 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:26.731 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:26.731 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:26.731 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:26.731 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:26.731 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:26.731 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 66, 2024-04-07T17:51:26.731 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 22, 2024-04-07T17:51:26.731 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:26.731 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 22, 2024-04-07T17:51:26.731 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:26.731 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:26.731 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:26.731 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:26.731 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:26.731 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 14, 2024-04-07T17:51:26.732 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:26.732 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:26.732 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:26.732 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:26.732 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:26.732 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:26.732 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:26.732 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 79, 2024-04-07T17:51:26.732 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 87, 2024-04-07T17:51:26.732 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:26.732 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:26.732 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:26.732 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 68, 2024-04-07T17:51:26.732 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 36 2024-04-07T17:51:26.732 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.732 INFO:tasks.workunit.client.0.smithi100.stdout: "store_stats": { 2024-04-07T17:51:26.733 INFO:tasks.workunit.client.0.smithi100.stdout: "allocated": 110592, 2024-04-07T17:51:26.733 INFO:tasks.workunit.client.0.smithi100.stdout: "available": 0, 2024-04-07T17:51:26.733 INFO:tasks.workunit.client.0.smithi100.stdout: "data_compressed": 0, 2024-04-07T17:51:26.733 INFO:tasks.workunit.client.0.smithi100.stdout: "data_compressed_allocated": 0, 2024-04-07T17:51:26.733 INFO:tasks.workunit.client.0.smithi100.stdout: "data_compressed_original": 0, 2024-04-07T17:51:26.733 INFO:tasks.workunit.client.0.smithi100.stdout: "data_stored": 19866, 2024-04-07T17:51:26.733 INFO:tasks.workunit.client.0.smithi100.stdout: "internal_metadata": 0, 2024-04-07T17:51:26.733 INFO:tasks.workunit.client.0.smithi100.stdout: "internally_reserved": 0, 2024-04-07T17:51:26.733 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_allocated": 48581, 2024-04-07T17:51:26.733 INFO:tasks.workunit.client.0.smithi100.stdout: "total": 0 2024-04-07T17:51:26.733 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.733 INFO:tasks.workunit.client.0.smithi100.stdout: "up": 96 2024-04-07T17:51:26.733 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:26.733 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:26.733 INFO:tasks.workunit.client.0.smithi100.stdout: "usage": { 2024-04-07T17:51:26.733 INFO:tasks.workunit.client.0.smithi100.stdout: "pg_num": 105, 2024-04-07T17:51:26.734 INFO:tasks.workunit.client.0.smithi100.stdout: "pools": 4, 2024-04-07T17:51:26.734 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_by_class": { 2024-04-07T17:51:26.734 INFO:tasks.workunit.client.0.smithi100.stdout: "ssd": { 2024-04-07T17:51:26.734 INFO:tasks.workunit.client.0.smithi100.stdout: "total_avail_bytes": 763743436800, 2024-04-07T17:51:26.734 INFO:tasks.workunit.client.0.smithi100.stdout: "total_bytes": 767960285184, 2024-04-07T17:51:26.734 INFO:tasks.workunit.client.0.smithi100.stdout: "total_used_bytes": 4216848384, 2024-04-07T17:51:26.734 INFO:tasks.workunit.client.0.smithi100.stdout: "total_used_raw_bytes": 4216848384, 2024-04-07T17:51:26.734 INFO:tasks.workunit.client.0.smithi100.stdout: "total_used_raw_ratio": 0.005490971729159355 2024-04-07T17:51:26.734 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:26.734 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:26.734 INFO:tasks.workunit.client.0.smithi100.stdout: "total_avail_bytes": 763743436800, 2024-04-07T17:51:26.734 INFO:tasks.workunit.client.0.smithi100.stdout: "total_bytes": 767960285184, 2024-04-07T17:51:26.734 INFO:tasks.workunit.client.0.smithi100.stdout: "total_used_bytes": 4216848384 2024-04-07T17:51:26.734 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:26.735 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:26.735 INFO:tasks.workunit.client.0.smithi100.stdout:} 2024-04-07T17:51:26.735 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:25 smithi100 bash[76202]: audit 2024-04-07T17:51:23.722297+0000 mgr.y (mgr.25175) 2885 : audit [DBG] from='client.45983 -' entity='client.admin' cmd=[{"prefix": "telemetry preview-device", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T17:51:26.735 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:25 smithi100 bash[76202]: audit 2024-04-07T17:51:23.875259+0000 mgr.y (mgr.25175) 2886 : audit [DBG] from='client.45983 -' entity='client.admin' cmd=[{"prefix": "telemetry preview-device", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T17:51:26.735 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:25 smithi100 bash[76202]: audit 2024-04-07T17:51:24.233450+0000 mgr.y (mgr.25175) 2887 : audit [DBG] from='client.36087 -' entity='client.admin' cmd=[{"prefix": "telemetry preview-all", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T17:51:26.735 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:25 smithi100 bash[76202]: audit 2024-04-07T17:51:24.236072+0000 mon.a (mon.0) 4811 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:51:26.735 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:25 smithi100 bash[76202]: audit 2024-04-07T17:51:24.244631+0000 mon.a (mon.0) 4812 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config-key dump", "format": "json"}]: dispatch 2024-04-07T17:51:26.735 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:25 smithi100 bash[76202]: audit 2024-04-07T17:51:24.463056+0000 mon.a (mon.0) 4813 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:26.735 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:25 smithi100 bash[76202]: audit 2024-04-07T17:51:24.463558+0000 mon.a (mon.0) 4814 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:26.736 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:25 smithi100 bash[76202]: audit 2024-04-07T17:51:24.463881+0000 mon.a (mon.0) 4815 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:26.736 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:25 smithi100 bash[76202]: audit 2024-04-07T17:51:24.467349+0000 mon.a (mon.0) 4816 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:26.736 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:25 smithi100 bash[76202]: audit 2024-04-07T17:51:24.467818+0000 mon.a (mon.0) 4817 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:26.736 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:25 smithi100 bash[76202]: audit 2024-04-07T17:51:24.468123+0000 mon.a (mon.0) 4818 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:26.736 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:25 smithi100 bash[76202]: audit 2024-04-07T17:51:24.471323+0000 mon.a (mon.0) 4819 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:26.736 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:25 smithi100 bash[76202]: audit 2024-04-07T17:51:24.471801+0000 mon.a (mon.0) 4820 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:26.736 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:25 smithi100 bash[76202]: audit 2024-04-07T17:51:24.472256+0000 mon.a (mon.0) 4821 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:26.736 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:25 smithi100 bash[76202]: audit 2024-04-07T17:51:24.491001+0000 mon.a (mon.0) 4822 : audit [INF] from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T17:51:26.736 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:25 smithi100 bash[76202]: audit 2024-04-07T17:51:24.491101+0000 mon.a (mon.0) 4823 : audit [INF] from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T17:51:26.736 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:25 smithi100 bash[76202]: audit 2024-04-07T17:51:24.491464+0000 mon.a (mon.0) 4824 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:26.736 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:25 smithi100 bash[76202]: audit 2024-04-07T17:51:24.491973+0000 mon.a (mon.0) 4825 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:26.736 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:25 smithi100 bash[74942]: audit 2024-04-07T17:51:23.722297+0000 mgr.y (mgr.25175) 2885 : audit [DBG] from='client.45983 -' entity='client.admin' cmd=[{"prefix": "telemetry preview-device", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T17:51:26.737 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:25 smithi100 bash[74942]: audit 2024-04-07T17:51:23.875259+0000 mgr.y (mgr.25175) 2886 : audit [DBG] from='client.45983 -' entity='client.admin' cmd=[{"prefix": "telemetry preview-device", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T17:51:26.737 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:25 smithi100 bash[74942]: audit 2024-04-07T17:51:24.233450+0000 mgr.y (mgr.25175) 2887 : audit [DBG] from='client.36087 -' entity='client.admin' cmd=[{"prefix": "telemetry preview-all", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T17:51:26.737 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:25 smithi100 bash[74942]: audit 2024-04-07T17:51:24.236072+0000 mon.a (mon.0) 4811 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:51:26.737 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:25 smithi100 bash[74942]: audit 2024-04-07T17:51:24.244631+0000 mon.a (mon.0) 4812 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config-key dump", "format": "json"}]: dispatch 2024-04-07T17:51:26.737 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:25 smithi100 bash[74942]: audit 2024-04-07T17:51:24.463056+0000 mon.a (mon.0) 4813 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:26.737 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:25 smithi100 bash[74942]: audit 2024-04-07T17:51:24.463558+0000 mon.a (mon.0) 4814 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:26.737 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:25 smithi100 bash[74942]: audit 2024-04-07T17:51:24.463881+0000 mon.a (mon.0) 4815 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:26.737 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:25 smithi100 bash[74942]: audit 2024-04-07T17:51:24.467349+0000 mon.a (mon.0) 4816 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:26.737 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:25 smithi100 bash[74942]: audit 2024-04-07T17:51:24.467818+0000 mon.a (mon.0) 4817 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:26.737 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:25 smithi100 bash[74942]: audit 2024-04-07T17:51:24.468123+0000 mon.a (mon.0) 4818 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:26.737 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:25 smithi100 bash[74942]: audit 2024-04-07T17:51:24.471323+0000 mon.a (mon.0) 4819 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:26.737 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:25 smithi100 bash[74942]: audit 2024-04-07T17:51:24.471801+0000 mon.a (mon.0) 4820 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:26.737 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:25 smithi100 bash[74942]: audit 2024-04-07T17:51:24.472256+0000 mon.a (mon.0) 4821 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:26.737 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:25 smithi100 bash[74942]: audit 2024-04-07T17:51:24.491001+0000 mon.a (mon.0) 4822 : audit [INF] from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T17:51:26.737 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:25 smithi100 bash[74942]: audit 2024-04-07T17:51:24.491101+0000 mon.a (mon.0) 4823 : audit [INF] from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T17:51:26.737 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:25 smithi100 bash[74942]: audit 2024-04-07T17:51:24.491464+0000 mon.a (mon.0) 4824 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:26.738 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:25 smithi100 bash[74942]: audit 2024-04-07T17:51:24.491973+0000 mon.a (mon.0) 4825 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:26.738 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:25 smithi100 bash[76202]: audit 2024-04-07T17:51:24.492374+0000 mon.a (mon.0) 4826 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:26.738 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:25 smithi100 bash[76202]: audit 2024-04-07T17:51:24.495655+0000 mon.a (mon.0) 4827 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:26.738 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:25 smithi100 bash[76202]: audit 2024-04-07T17:51:24.496511+0000 mon.a (mon.0) 4828 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:26.738 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:25 smithi100 bash[76202]: audit 2024-04-07T17:51:24.496829+0000 mon.a (mon.0) 4829 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:26.738 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:25 smithi100 bash[76202]: audit 2024-04-07T17:51:24.497477+0000 mon.b (mon.1) 27 : audit [INF] from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T17:51:26.738 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:25 smithi100 bash[76202]: audit 2024-04-07T17:51:24.497582+0000 mon.b (mon.1) 28 : audit [INF] from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T17:51:26.738 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:25 smithi100 bash[76202]: audit 2024-04-07T17:51:24.499288+0000 mon.c (mon.2) 125 : audit [INF] from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T17:51:26.738 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:25 smithi100 bash[76202]: audit 2024-04-07T17:51:24.499394+0000 mon.c (mon.2) 126 : audit [INF] from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T17:51:26.738 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:25 smithi100 bash[76202]: audit 2024-04-07T17:51:24.499820+0000 mon.a (mon.0) 4830 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:26.738 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:25 smithi100 bash[76202]: audit 2024-04-07T17:51:24.500812+0000 mon.a (mon.0) 4831 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:26.739 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:25 smithi100 bash[76202]: audit 2024-04-07T17:51:24.501145+0000 mon.a (mon.0) 4832 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:26.739 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:25 smithi100 bash[76202]: audit 2024-04-07T17:51:24.822625+0000 mon.a (mon.0) 4833 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:51:26.739 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:25 smithi100 bash[76202]: audit 2024-04-07T17:51:24.830664+0000 mon.a (mon.0) 4834 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config-key dump", "format": "json"}]: dispatch 2024-04-07T17:51:26.739 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:25 smithi100 bash[76202]: audit 2024-04-07T17:51:25.048444+0000 mon.a (mon.0) 4835 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:26.739 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:25 smithi100 bash[76202]: audit 2024-04-07T17:51:25.048954+0000 mon.a (mon.0) 4836 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:26.739 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:25 smithi100 bash[76202]: audit 2024-04-07T17:51:25.049416+0000 mon.a (mon.0) 4837 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:26.739 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:25 smithi100 bash[76202]: audit 2024-04-07T17:51:25.053033+0000 mon.a (mon.0) 4838 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:26.739 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:25 smithi100 bash[76202]: audit 2024-04-07T17:51:25.053470+0000 mon.a (mon.0) 4839 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:26.739 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:25 smithi100 bash[76202]: audit 2024-04-07T17:51:25.053843+0000 mon.a (mon.0) 4840 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:26.739 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:25 smithi100 bash[74942]: audit 2024-04-07T17:51:24.492374+0000 mon.a (mon.0) 4826 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:26.739 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:25 smithi100 bash[74942]: audit 2024-04-07T17:51:24.495655+0000 mon.a (mon.0) 4827 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:26.739 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:25 smithi100 bash[74942]: audit 2024-04-07T17:51:24.496511+0000 mon.a (mon.0) 4828 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:26.739 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:25 smithi100 bash[74942]: audit 2024-04-07T17:51:24.496829+0000 mon.a (mon.0) 4829 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:26.740 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:25 smithi100 bash[74942]: audit 2024-04-07T17:51:24.497477+0000 mon.b (mon.1) 27 : audit [INF] from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T17:51:26.740 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:25 smithi100 bash[74942]: audit 2024-04-07T17:51:24.497582+0000 mon.b (mon.1) 28 : audit [INF] from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T17:51:26.740 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:25 smithi100 bash[74942]: audit 2024-04-07T17:51:24.499288+0000 mon.c (mon.2) 125 : audit [INF] from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T17:51:26.740 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:25 smithi100 bash[74942]: audit 2024-04-07T17:51:24.499394+0000 mon.c (mon.2) 126 : audit [INF] from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T17:51:26.740 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:25 smithi100 bash[74942]: audit 2024-04-07T17:51:24.499820+0000 mon.a (mon.0) 4830 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:26.740 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:25 smithi100 bash[74942]: audit 2024-04-07T17:51:24.500812+0000 mon.a (mon.0) 4831 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:26.740 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:25 smithi100 bash[74942]: audit 2024-04-07T17:51:24.501145+0000 mon.a (mon.0) 4832 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:26.740 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:25 smithi100 bash[74942]: audit 2024-04-07T17:51:24.822625+0000 mon.a (mon.0) 4833 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:51:26.740 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:25 smithi100 bash[74942]: audit 2024-04-07T17:51:24.830664+0000 mon.a (mon.0) 4834 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config-key dump", "format": "json"}]: dispatch 2024-04-07T17:51:26.740 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:25 smithi100 bash[74942]: audit 2024-04-07T17:51:25.048444+0000 mon.a (mon.0) 4835 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:26.740 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:25 smithi100 bash[74942]: audit 2024-04-07T17:51:25.048954+0000 mon.a (mon.0) 4836 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:26.740 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:25 smithi100 bash[74942]: audit 2024-04-07T17:51:25.049416+0000 mon.a (mon.0) 4837 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:26.740 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:25 smithi100 bash[74942]: audit 2024-04-07T17:51:25.053033+0000 mon.a (mon.0) 4838 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:26.740 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:25 smithi100 bash[74942]: audit 2024-04-07T17:51:25.053470+0000 mon.a (mon.0) 4839 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:26.740 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:25 smithi100 bash[74942]: audit 2024-04-07T17:51:25.053843+0000 mon.a (mon.0) 4840 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:26.741 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:25 smithi100 bash[76202]: audit 2024-04-07T17:51:25.056987+0000 mon.a (mon.0) 4841 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:26.741 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:25 smithi100 bash[76202]: audit 2024-04-07T17:51:25.057521+0000 mon.a (mon.0) 4842 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:26.741 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:25 smithi100 bash[76202]: audit 2024-04-07T17:51:25.057918+0000 mon.a (mon.0) 4843 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:26.741 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:25 smithi100 bash[76202]: audit 2024-04-07T17:51:25.076689+0000 mon.a (mon.0) 4844 : audit [INF] from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T17:51:26.741 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:25 smithi100 bash[76202]: audit 2024-04-07T17:51:25.076842+0000 mon.a (mon.0) 4845 : audit [INF] from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T17:51:26.741 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:25 smithi100 bash[76202]: audit 2024-04-07T17:51:25.077280+0000 mon.a (mon.0) 4846 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:26.741 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:25 smithi100 bash[76202]: audit 2024-04-07T17:51:25.077798+0000 mon.a (mon.0) 4847 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:26.741 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:25 smithi100 bash[76202]: audit 2024-04-07T17:51:25.078158+0000 mon.a (mon.0) 4848 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:26.741 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:25 smithi100 bash[76202]: audit 2024-04-07T17:51:25.081565+0000 mon.a (mon.0) 4849 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:26.741 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:25 smithi100 bash[76202]: audit 2024-04-07T17:51:25.082472+0000 mon.a (mon.0) 4850 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:26.741 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:25 smithi100 bash[76202]: audit 2024-04-07T17:51:25.082795+0000 mon.a (mon.0) 4851 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:26.741 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:25 smithi100 bash[76202]: audit 2024-04-07T17:51:25.083445+0000 mon.b (mon.1) 29 : audit [INF] from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T17:51:26.741 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:25 smithi100 bash[76202]: audit 2024-04-07T17:51:25.083551+0000 mon.b (mon.1) 30 : audit [INF] from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T17:51:26.741 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:25 smithi100 bash[76202]: audit 2024-04-07T17:51:25.085501+0000 mon.c (mon.2) 127 : audit [INF] from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T17:51:26.742 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:25 smithi100 bash[76202]: audit 2024-04-07T17:51:25.085606+0000 mon.c (mon.2) 128 : audit [INF] from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T17:51:26.742 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:25 smithi100 bash[76202]: audit 2024-04-07T17:51:25.086022+0000 mon.a (mon.0) 4852 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:26.742 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:25 smithi100 bash[76202]: audit 2024-04-07T17:51:25.086899+0000 mon.a (mon.0) 4853 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:26.742 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:25 smithi100 bash[76202]: audit 2024-04-07T17:51:25.087254+0000 mon.a (mon.0) 4854 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:26.742 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:25 smithi100 bash[74942]: audit 2024-04-07T17:51:25.056987+0000 mon.a (mon.0) 4841 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:26.742 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:25 smithi100 bash[74942]: audit 2024-04-07T17:51:25.057521+0000 mon.a (mon.0) 4842 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:26.742 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:25 smithi100 bash[74942]: audit 2024-04-07T17:51:25.057918+0000 mon.a (mon.0) 4843 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:26.742 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:25 smithi100 bash[74942]: audit 2024-04-07T17:51:25.076689+0000 mon.a (mon.0) 4844 : audit [INF] from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T17:51:26.742 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:25 smithi100 bash[74942]: audit 2024-04-07T17:51:25.076842+0000 mon.a (mon.0) 4845 : audit [INF] from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T17:51:26.742 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:25 smithi100 bash[74942]: audit 2024-04-07T17:51:25.077280+0000 mon.a (mon.0) 4846 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:26.742 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:25 smithi100 bash[74942]: audit 2024-04-07T17:51:25.077798+0000 mon.a (mon.0) 4847 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:26.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:25 smithi100 bash[74942]: audit 2024-04-07T17:51:25.078158+0000 mon.a (mon.0) 4848 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:26.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:25 smithi100 bash[74942]: audit 2024-04-07T17:51:25.081565+0000 mon.a (mon.0) 4849 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:26.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:25 smithi100 bash[74942]: audit 2024-04-07T17:51:25.082472+0000 mon.a (mon.0) 4850 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:26.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:25 smithi100 bash[74942]: audit 2024-04-07T17:51:25.082795+0000 mon.a (mon.0) 4851 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:26.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:25 smithi100 bash[74942]: audit 2024-04-07T17:51:25.083445+0000 mon.b (mon.1) 29 : audit [INF] from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T17:51:26.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:25 smithi100 bash[74942]: audit 2024-04-07T17:51:25.083551+0000 mon.b (mon.1) 30 : audit [INF] from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T17:51:26.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:25 smithi100 bash[74942]: audit 2024-04-07T17:51:25.085501+0000 mon.c (mon.2) 127 : audit [INF] from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T17:51:26.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:25 smithi100 bash[74942]: audit 2024-04-07T17:51:25.085606+0000 mon.c (mon.2) 128 : audit [INF] from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T17:51:26.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:25 smithi100 bash[74942]: audit 2024-04-07T17:51:25.086022+0000 mon.a (mon.0) 4852 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:26.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:25 smithi100 bash[74942]: audit 2024-04-07T17:51:25.086899+0000 mon.a (mon.0) 4853 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:26.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:25 smithi100 bash[74942]: audit 2024-04-07T17:51:25.087254+0000 mon.a (mon.0) 4854 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:26.743 INFO:tasks.workunit.client.0.smithi100.stdout:Telemetry is on. 2024-04-07T17:51:26.744 INFO:tasks.workunit.client.0.smithi100.stderr:+ ceph telemetry show 2024-04-07T17:51:26.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:26 smithi100 bash[74942]: audit 2024-04-07T17:51:24.819635+0000 mgr.y (mgr.25175) 2888 : audit [DBG] from='client.36087 -' entity='client.admin' cmd=[{"prefix": "telemetry preview-all", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T17:51:26.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:26 smithi100 bash[74942]: cluster 2024-04-07T17:51:24.829300+0000 mgr.y (mgr.25175) 2889 : cluster [DBG] pgmap v5169: 105 pgs: 105 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 170 B/s rd, 0 op/s 2024-04-07T17:51:26.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:26 smithi100 bash[74942]: audit 2024-04-07T17:51:25.650309+0000 mon.a (mon.0) 4855 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:51:26.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:26 smithi100 bash[74942]: audit 2024-04-07T17:51:25.655826+0000 mon.a (mon.0) 4856 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:51:26.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:26 smithi100 bash[74942]: audit 2024-04-07T17:51:25.697267+0000 mon.a (mon.0) 4857 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:51:26.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:26 smithi100 bash[74942]: audit 2024-04-07T17:51:25.706731+0000 mon.a (mon.0) 4858 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config-key dump", "format": "json"}]: dispatch 2024-04-07T17:51:26.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:26 smithi100 bash[74942]: audit 2024-04-07T17:51:25.837880+0000 mon.a (mon.0) 4859 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:51:26.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:26 smithi100 bash[74942]: audit 2024-04-07T17:51:25.843966+0000 mon.a (mon.0) 4860 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:51:26.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:26 smithi100 bash[74942]: audit 2024-04-07T17:51:25.882594+0000 mon.a (mon.0) 4861 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:26.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:26 smithi100 bash[74942]: audit 2024-04-07T17:51:25.882890+0000 mon.a (mon.0) 4862 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:26.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:26 smithi100 bash[74942]: audit 2024-04-07T17:51:25.883128+0000 mon.a (mon.0) 4863 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:26.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:26 smithi100 bash[74942]: audit 2024-04-07T17:51:25.885667+0000 mon.a (mon.0) 4864 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:26.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:26 smithi100 bash[74942]: audit 2024-04-07T17:51:25.885936+0000 mon.a (mon.0) 4865 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:26.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:26 smithi100 bash[74942]: audit 2024-04-07T17:51:25.886167+0000 mon.a (mon.0) 4866 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:26.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:26 smithi100 bash[74942]: audit 2024-04-07T17:51:25.887883+0000 mon.a (mon.0) 4867 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:26.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:26 smithi100 bash[74942]: audit 2024-04-07T17:51:25.888133+0000 mon.a (mon.0) 4868 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:26.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:26 smithi100 bash[74942]: audit 2024-04-07T17:51:25.888423+0000 mon.a (mon.0) 4869 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:26.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:26 smithi100 bash[74942]: audit 2024-04-07T17:51:25.899107+0000 mon.a (mon.0) 4870 : audit [INF] from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T17:51:26.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:26 smithi100 bash[74942]: audit 2024-04-07T17:51:25.899153+0000 mon.a (mon.0) 4871 : audit [INF] from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T17:51:26.745 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:26 smithi100 bash[76202]: audit 2024-04-07T17:51:24.819635+0000 mgr.y (mgr.25175) 2888 : audit [DBG] from='client.36087 -' entity='client.admin' cmd=[{"prefix": "telemetry preview-all", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T17:51:26.746 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:26 smithi100 bash[76202]: cluster 2024-04-07T17:51:24.829300+0000 mgr.y (mgr.25175) 2889 : cluster [DBG] pgmap v5169: 105 pgs: 105 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 170 B/s rd, 0 op/s 2024-04-07T17:51:26.746 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:26 smithi100 bash[76202]: audit 2024-04-07T17:51:25.650309+0000 mon.a (mon.0) 4855 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:51:26.746 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:26 smithi100 bash[76202]: audit 2024-04-07T17:51:25.655826+0000 mon.a (mon.0) 4856 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:51:26.746 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:26 smithi100 bash[76202]: audit 2024-04-07T17:51:25.697267+0000 mon.a (mon.0) 4857 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:51:26.746 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:26 smithi100 bash[76202]: audit 2024-04-07T17:51:25.706731+0000 mon.a (mon.0) 4858 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config-key dump", "format": "json"}]: dispatch 2024-04-07T17:51:26.746 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:26 smithi100 bash[76202]: audit 2024-04-07T17:51:25.837880+0000 mon.a (mon.0) 4859 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:51:26.746 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:26 smithi100 bash[76202]: audit 2024-04-07T17:51:25.843966+0000 mon.a (mon.0) 4860 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:51:26.746 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:26 smithi100 bash[76202]: audit 2024-04-07T17:51:25.882594+0000 mon.a (mon.0) 4861 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:26.746 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:26 smithi100 bash[76202]: audit 2024-04-07T17:51:25.882890+0000 mon.a (mon.0) 4862 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:26.746 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:26 smithi100 bash[76202]: audit 2024-04-07T17:51:25.883128+0000 mon.a (mon.0) 4863 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:26.746 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:26 smithi100 bash[76202]: audit 2024-04-07T17:51:25.885667+0000 mon.a (mon.0) 4864 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:26.746 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:26 smithi100 bash[76202]: audit 2024-04-07T17:51:25.885936+0000 mon.a (mon.0) 4865 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:26.746 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:26 smithi100 bash[76202]: audit 2024-04-07T17:51:25.886167+0000 mon.a (mon.0) 4866 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:26.746 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:26 smithi100 bash[76202]: audit 2024-04-07T17:51:25.887883+0000 mon.a (mon.0) 4867 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:26.747 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:26 smithi100 bash[76202]: audit 2024-04-07T17:51:25.888133+0000 mon.a (mon.0) 4868 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:26.747 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:26 smithi100 bash[76202]: audit 2024-04-07T17:51:25.888423+0000 mon.a (mon.0) 4869 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:26.747 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:26 smithi100 bash[76202]: audit 2024-04-07T17:51:25.899107+0000 mon.a (mon.0) 4870 : audit [INF] from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T17:51:26.747 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:26 smithi100 bash[76202]: audit 2024-04-07T17:51:25.899153+0000 mon.a (mon.0) 4871 : audit [INF] from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T17:51:26.747 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:26 smithi100 bash[74942]: audit 2024-04-07T17:51:25.899498+0000 mon.a (mon.0) 4872 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:26.747 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:26 smithi100 bash[74942]: audit 2024-04-07T17:51:25.899741+0000 mon.a (mon.0) 4873 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:26.748 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:26 smithi100 bash[74942]: audit 2024-04-07T17:51:25.899957+0000 mon.a (mon.0) 4874 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:26.748 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:26 smithi100 bash[74942]: audit 2024-04-07T17:51:25.902213+0000 mon.a (mon.0) 4875 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:26.748 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:26 smithi100 bash[74942]: audit 2024-04-07T17:51:25.902435+0000 mon.a (mon.0) 4876 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:26.748 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:26 smithi100 bash[74942]: audit 2024-04-07T17:51:25.902625+0000 mon.a (mon.0) 4877 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:26.748 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:26 smithi100 bash[74942]: audit 2024-04-07T17:51:25.904047+0000 mon.c (mon.2) 129 : audit [INF] from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T17:51:26.748 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:26 smithi100 bash[74942]: audit 2024-04-07T17:51:25.904104+0000 mon.c (mon.2) 130 : audit [INF] from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T17:51:26.748 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:26 smithi100 bash[74942]: audit 2024-04-07T17:51:25.904287+0000 mon.a (mon.0) 4878 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:26.748 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:26 smithi100 bash[74942]: audit 2024-04-07T17:51:25.904375+0000 mon.b (mon.1) 31 : audit [INF] from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T17:51:26.748 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:26 smithi100 bash[74942]: audit 2024-04-07T17:51:25.904477+0000 mon.b (mon.1) 32 : audit [INF] from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T17:51:26.748 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:26 smithi100 bash[74942]: audit 2024-04-07T17:51:25.904675+0000 mon.a (mon.0) 4879 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:26.748 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:26 smithi100 bash[74942]: audit 2024-04-07T17:51:25.904872+0000 mon.a (mon.0) 4880 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:26.748 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:26 smithi100 bash[74942]: audit 2024-04-07T17:51:25.993600+0000 mon.a (mon.0) 4881 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:51:26.748 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:26 smithi100 bash[74942]: audit 2024-04-07T17:51:25.997527+0000 mon.a (mon.0) 4882 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config-key dump", "format": "json"}]: dispatch 2024-04-07T17:51:26.748 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:26 smithi100 bash[74942]: audit 2024-04-07T17:51:26.110357+0000 mon.a (mon.0) 4883 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:26.748 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:26 smithi100 bash[74942]: audit 2024-04-07T17:51:26.110967+0000 mon.a (mon.0) 4884 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:26.749 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:26 smithi100 bash[74942]: audit 2024-04-07T17:51:26.111449+0000 mon.a (mon.0) 4885 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:26.749 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:26 smithi100 bash[74942]: audit 2024-04-07T17:51:26.114338+0000 mon.a (mon.0) 4886 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:26.749 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:26 smithi100 bash[74942]: audit 2024-04-07T17:51:26.114827+0000 mon.a (mon.0) 4887 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:26.749 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:26 smithi100 bash[74942]: audit 2024-04-07T17:51:26.115192+0000 mon.a (mon.0) 4888 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:26.749 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:26 smithi144 bash[66654]: audit 2024-04-07T17:51:24.819635+0000 mgr.y (mgr.25175) 2888 : audit [DBG] from='client.36087 -' entity='client.admin' cmd=[{"prefix": "telemetry preview-all", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T17:51:26.749 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:26 smithi144 bash[66654]: cluster 2024-04-07T17:51:24.829300+0000 mgr.y (mgr.25175) 2889 : cluster [DBG] pgmap v5169: 105 pgs: 105 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 170 B/s rd, 0 op/s 2024-04-07T17:51:26.749 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:26 smithi144 bash[66654]: audit 2024-04-07T17:51:25.650309+0000 mon.a (mon.0) 4855 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:51:26.749 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:26 smithi144 bash[66654]: audit 2024-04-07T17:51:25.655826+0000 mon.a (mon.0) 4856 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:51:26.749 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:26 smithi144 bash[66654]: audit 2024-04-07T17:51:25.697267+0000 mon.a (mon.0) 4857 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:51:26.749 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:26 smithi144 bash[66654]: audit 2024-04-07T17:51:25.706731+0000 mon.a (mon.0) 4858 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config-key dump", "format": "json"}]: dispatch 2024-04-07T17:51:26.749 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:26 smithi144 bash[66654]: audit 2024-04-07T17:51:25.837880+0000 mon.a (mon.0) 4859 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:51:26.749 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:26 smithi144 bash[66654]: audit 2024-04-07T17:51:25.843966+0000 mon.a (mon.0) 4860 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:51:26.749 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:26 smithi144 bash[66654]: audit 2024-04-07T17:51:25.882594+0000 mon.a (mon.0) 4861 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:26.749 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:26 smithi144 bash[66654]: audit 2024-04-07T17:51:25.882890+0000 mon.a (mon.0) 4862 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:26.750 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:26 smithi144 bash[66654]: audit 2024-04-07T17:51:25.883128+0000 mon.a (mon.0) 4863 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:26.750 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:26 smithi144 bash[66654]: audit 2024-04-07T17:51:25.885667+0000 mon.a (mon.0) 4864 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:26.750 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:26 smithi144 bash[66654]: audit 2024-04-07T17:51:25.885936+0000 mon.a (mon.0) 4865 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:26.750 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:26 smithi144 bash[66654]: audit 2024-04-07T17:51:25.886167+0000 mon.a (mon.0) 4866 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:26.750 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:26 smithi144 bash[66654]: audit 2024-04-07T17:51:25.887883+0000 mon.a (mon.0) 4867 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:26.750 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:26 smithi144 bash[66654]: audit 2024-04-07T17:51:25.888133+0000 mon.a (mon.0) 4868 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:26.750 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:26 smithi144 bash[66654]: audit 2024-04-07T17:51:25.888423+0000 mon.a (mon.0) 4869 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:26.750 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:26 smithi144 bash[66654]: audit 2024-04-07T17:51:25.899107+0000 mon.a (mon.0) 4870 : audit [INF] from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T17:51:26.750 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:26 smithi144 bash[66654]: audit 2024-04-07T17:51:25.899153+0000 mon.a (mon.0) 4871 : audit [INF] from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T17:51:26.750 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:26 smithi144 bash[66654]: audit 2024-04-07T17:51:25.899498+0000 mon.a (mon.0) 4872 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:26.750 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:26 smithi144 bash[66654]: audit 2024-04-07T17:51:25.899741+0000 mon.a (mon.0) 4873 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:26.750 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:26 smithi144 bash[66654]: audit 2024-04-07T17:51:25.899957+0000 mon.a (mon.0) 4874 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:26.750 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:26 smithi144 bash[66654]: audit 2024-04-07T17:51:25.902213+0000 mon.a (mon.0) 4875 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:26.750 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:26 smithi144 bash[66654]: audit 2024-04-07T17:51:25.902435+0000 mon.a (mon.0) 4876 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:26.751 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:26 smithi144 bash[66654]: audit 2024-04-07T17:51:25.902625+0000 mon.a (mon.0) 4877 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:26.751 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:26 smithi144 bash[66654]: audit 2024-04-07T17:51:25.904047+0000 mon.c (mon.2) 129 : audit [INF] from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T17:51:26.751 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:26 smithi144 bash[66654]: audit 2024-04-07T17:51:25.904104+0000 mon.c (mon.2) 130 : audit [INF] from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T17:51:26.751 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:26 smithi144 bash[66654]: audit 2024-04-07T17:51:25.904287+0000 mon.a (mon.0) 4878 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:26.751 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:26 smithi144 bash[66654]: audit 2024-04-07T17:51:25.904375+0000 mon.b (mon.1) 31 : audit [INF] from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T17:51:26.751 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:26 smithi144 bash[66654]: audit 2024-04-07T17:51:25.904477+0000 mon.b (mon.1) 32 : audit [INF] from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T17:51:26.751 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:26 smithi144 bash[66654]: audit 2024-04-07T17:51:25.904675+0000 mon.a (mon.0) 4879 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:26.751 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:26 smithi144 bash[66654]: audit 2024-04-07T17:51:25.904872+0000 mon.a (mon.0) 4880 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:26.751 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:26 smithi144 bash[66654]: audit 2024-04-07T17:51:25.993600+0000 mon.a (mon.0) 4881 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:51:26.751 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:26 smithi144 bash[66654]: audit 2024-04-07T17:51:25.997527+0000 mon.a (mon.0) 4882 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config-key dump", "format": "json"}]: dispatch 2024-04-07T17:51:26.751 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:26 smithi144 bash[66654]: audit 2024-04-07T17:51:26.110357+0000 mon.a (mon.0) 4883 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:26.751 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:26 smithi144 bash[66654]: audit 2024-04-07T17:51:26.110967+0000 mon.a (mon.0) 4884 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:26.751 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:26 smithi144 bash[66654]: audit 2024-04-07T17:51:26.111449+0000 mon.a (mon.0) 4885 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:26.751 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:26 smithi144 bash[66654]: audit 2024-04-07T17:51:26.114338+0000 mon.a (mon.0) 4886 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:26.751 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:26 smithi144 bash[66654]: audit 2024-04-07T17:51:26.114827+0000 mon.a (mon.0) 4887 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:26.751 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:26 smithi144 bash[66654]: audit 2024-04-07T17:51:26.115192+0000 mon.a (mon.0) 4888 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:26.752 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:26 smithi100 bash[76202]: audit 2024-04-07T17:51:25.899498+0000 mon.a (mon.0) 4872 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:26.752 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:26 smithi100 bash[76202]: audit 2024-04-07T17:51:25.899741+0000 mon.a (mon.0) 4873 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:26.752 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:26 smithi100 bash[76202]: audit 2024-04-07T17:51:25.899957+0000 mon.a (mon.0) 4874 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:26.752 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:26 smithi100 bash[76202]: audit 2024-04-07T17:51:25.902213+0000 mon.a (mon.0) 4875 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:26.752 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:26 smithi100 bash[76202]: audit 2024-04-07T17:51:25.902435+0000 mon.a (mon.0) 4876 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:26.752 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:26 smithi100 bash[76202]: audit 2024-04-07T17:51:25.902625+0000 mon.a (mon.0) 4877 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:26.752 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:26 smithi100 bash[76202]: audit 2024-04-07T17:51:25.904047+0000 mon.c (mon.2) 129 : audit [INF] from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T17:51:26.752 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:26 smithi100 bash[76202]: audit 2024-04-07T17:51:25.904104+0000 mon.c (mon.2) 130 : audit [INF] from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T17:51:26.752 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:26 smithi100 bash[76202]: audit 2024-04-07T17:51:25.904287+0000 mon.a (mon.0) 4878 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:26.752 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:26 smithi100 bash[76202]: audit 2024-04-07T17:51:25.904375+0000 mon.b (mon.1) 31 : audit [INF] from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T17:51:26.752 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:26 smithi100 bash[76202]: audit 2024-04-07T17:51:25.904477+0000 mon.b (mon.1) 32 : audit [INF] from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T17:51:26.752 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:26 smithi100 bash[76202]: audit 2024-04-07T17:51:25.904675+0000 mon.a (mon.0) 4879 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:26.752 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:26 smithi100 bash[76202]: audit 2024-04-07T17:51:25.904872+0000 mon.a (mon.0) 4880 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:26.752 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:26 smithi100 bash[76202]: audit 2024-04-07T17:51:25.993600+0000 mon.a (mon.0) 4881 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:51:26.752 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:26 smithi100 bash[76202]: audit 2024-04-07T17:51:25.997527+0000 mon.a (mon.0) 4882 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config-key dump", "format": "json"}]: dispatch 2024-04-07T17:51:26.752 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:26 smithi100 bash[76202]: audit 2024-04-07T17:51:26.110357+0000 mon.a (mon.0) 4883 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:26.753 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:26 smithi100 bash[76202]: audit 2024-04-07T17:51:26.110967+0000 mon.a (mon.0) 4884 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:26.753 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:26 smithi100 bash[76202]: audit 2024-04-07T17:51:26.111449+0000 mon.a (mon.0) 4885 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:26.753 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:26 smithi100 bash[76202]: audit 2024-04-07T17:51:26.114338+0000 mon.a (mon.0) 4886 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:26.753 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:26 smithi100 bash[76202]: audit 2024-04-07T17:51:26.114827+0000 mon.a (mon.0) 4887 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:26.753 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:26 smithi100 bash[76202]: audit 2024-04-07T17:51:26.115192+0000 mon.a (mon.0) 4888 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:27.115 INFO:tasks.workunit.client.0.smithi100.stdout:{ 2024-04-07T17:51:27.115 INFO:tasks.workunit.client.0.smithi100.stdout: "balancer": { 2024-04-07T17:51:27.115 INFO:tasks.workunit.client.0.smithi100.stdout: "active": true, 2024-04-07T17:51:27.115 INFO:tasks.workunit.client.0.smithi100.stdout: "mode": "" 2024-04-07T17:51:27.115 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.116 INFO:tasks.workunit.client.0.smithi100.stdout: "channels": [ 2024-04-07T17:51:27.116 INFO:tasks.workunit.client.0.smithi100.stdout: "basic", 2024-04-07T17:51:27.116 INFO:tasks.workunit.client.0.smithi100.stdout: "crash", 2024-04-07T17:51:27.116 INFO:tasks.workunit.client.0.smithi100.stdout: "device", 2024-04-07T17:51:27.116 INFO:tasks.workunit.client.0.smithi100.stdout: "ident", 2024-04-07T17:51:27.116 INFO:tasks.workunit.client.0.smithi100.stdout: "perf" 2024-04-07T17:51:27.116 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:27.116 INFO:tasks.workunit.client.0.smithi100.stdout: "channels_available": [ 2024-04-07T17:51:27.116 INFO:tasks.workunit.client.0.smithi100.stdout: "basic", 2024-04-07T17:51:27.116 INFO:tasks.workunit.client.0.smithi100.stdout: "ident", 2024-04-07T17:51:27.116 INFO:tasks.workunit.client.0.smithi100.stdout: "crash", 2024-04-07T17:51:27.116 INFO:tasks.workunit.client.0.smithi100.stdout: "device", 2024-04-07T17:51:27.116 INFO:tasks.workunit.client.0.smithi100.stdout: "perf" 2024-04-07T17:51:27.116 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:27.116 INFO:tasks.workunit.client.0.smithi100.stdout: "collections_available": [ 2024-04-07T17:51:27.117 INFO:tasks.workunit.client.0.smithi100.stdout: "basic_base", 2024-04-07T17:51:27.117 INFO:tasks.workunit.client.0.smithi100.stdout: "device_base", 2024-04-07T17:51:27.117 INFO:tasks.workunit.client.0.smithi100.stdout: "crash_base", 2024-04-07T17:51:27.117 INFO:tasks.workunit.client.0.smithi100.stdout: "ident_base", 2024-04-07T17:51:27.117 INFO:tasks.workunit.client.0.smithi100.stdout: "perf_perf", 2024-04-07T17:51:27.117 INFO:tasks.workunit.client.0.smithi100.stdout: "basic_mds_metadata", 2024-04-07T17:51:27.117 INFO:tasks.workunit.client.0.smithi100.stdout: "basic_pool_usage", 2024-04-07T17:51:27.117 INFO:tasks.workunit.client.0.smithi100.stdout: "basic_usage_by_class", 2024-04-07T17:51:27.117 INFO:tasks.workunit.client.0.smithi100.stdout: "basic_rook_v01", 2024-04-07T17:51:27.117 INFO:tasks.workunit.client.0.smithi100.stdout: "perf_memory_metrics", 2024-04-07T17:51:27.117 INFO:tasks.workunit.client.0.smithi100.stdout: "basic_pool_options_bluestore", 2024-04-07T17:51:27.117 INFO:tasks.workunit.client.0.smithi100.stdout: "basic_pool_flags" 2024-04-07T17:51:27.117 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:27.117 INFO:tasks.workunit.client.0.smithi100.stdout: "collections_opted_in": [ 2024-04-07T17:51:27.117 INFO:tasks.workunit.client.0.smithi100.stdout: "basic_base", 2024-04-07T17:51:27.117 INFO:tasks.workunit.client.0.smithi100.stdout: "device_base", 2024-04-07T17:51:27.118 INFO:tasks.workunit.client.0.smithi100.stdout: "crash_base", 2024-04-07T17:51:27.118 INFO:tasks.workunit.client.0.smithi100.stdout: "ident_base", 2024-04-07T17:51:27.118 INFO:tasks.workunit.client.0.smithi100.stdout: "perf_perf", 2024-04-07T17:51:27.118 INFO:tasks.workunit.client.0.smithi100.stdout: "basic_mds_metadata", 2024-04-07T17:51:27.118 INFO:tasks.workunit.client.0.smithi100.stdout: "basic_pool_usage", 2024-04-07T17:51:27.118 INFO:tasks.workunit.client.0.smithi100.stdout: "basic_usage_by_class", 2024-04-07T17:51:27.118 INFO:tasks.workunit.client.0.smithi100.stdout: "basic_rook_v01", 2024-04-07T17:51:27.118 INFO:tasks.workunit.client.0.smithi100.stdout: "perf_memory_metrics", 2024-04-07T17:51:27.118 INFO:tasks.workunit.client.0.smithi100.stdout: "basic_pool_options_bluestore", 2024-04-07T17:51:27.118 INFO:tasks.workunit.client.0.smithi100.stdout: "basic_pool_flags" 2024-04-07T17:51:27.118 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:27.118 INFO:tasks.workunit.client.0.smithi100.stdout: "config": { 2024-04-07T17:51:27.118 INFO:tasks.workunit.client.0.smithi100.stdout: "active_changed": [ 2024-04-07T17:51:27.118 INFO:tasks.workunit.client.0.smithi100.stdout: "auth_allow_insecure_global_id_reclaim", 2024-04-07T17:51:27.118 INFO:tasks.workunit.client.0.smithi100.stdout: "auth_debug", 2024-04-07T17:51:27.119 INFO:tasks.workunit.client.0.smithi100.stdout: "auth_mon_ticket_ttl", 2024-04-07T17:51:27.119 INFO:tasks.workunit.client.0.smithi100.stdout: "auth_service_ticket_ttl", 2024-04-07T17:51:27.119 INFO:tasks.workunit.client.0.smithi100.stdout: "bdev_debug_aio", 2024-04-07T17:51:27.119 INFO:tasks.workunit.client.0.smithi100.stdout: "client_check_pool_perm", 2024-04-07T17:51:27.119 INFO:tasks.workunit.client.0.smithi100.stdout: "client_mount_gid", 2024-04-07T17:51:27.119 INFO:tasks.workunit.client.0.smithi100.stdout: "client_mount_uid", 2024-04-07T17:51:27.119 INFO:tasks.workunit.client.0.smithi100.stdout: "client_quota", 2024-04-07T17:51:27.119 INFO:tasks.workunit.client.0.smithi100.stdout: "container_image", 2024-04-07T17:51:27.119 INFO:tasks.workunit.client.0.smithi100.stdout: "daemonize", 2024-04-07T17:51:27.119 INFO:tasks.workunit.client.0.smithi100.stdout: "debug_asserts_on_shutdown", 2024-04-07T17:51:27.119 INFO:tasks.workunit.client.0.smithi100.stdout: "debug_mgr", 2024-04-07T17:51:27.119 INFO:tasks.workunit.client.0.smithi100.stdout: "debug_mon", 2024-04-07T17:51:27.119 INFO:tasks.workunit.client.0.smithi100.stdout: "debug_ms", 2024-04-07T17:51:27.119 INFO:tasks.workunit.client.0.smithi100.stdout: "debug_osd", 2024-04-07T17:51:27.119 INFO:tasks.workunit.client.0.smithi100.stdout: "debug_paxos", 2024-04-07T17:51:27.120 INFO:tasks.workunit.client.0.smithi100.stdout: "keyring", 2024-04-07T17:51:27.120 INFO:tasks.workunit.client.0.smithi100.stdout: "log_stderr_prefix", 2024-04-07T17:51:27.120 INFO:tasks.workunit.client.0.smithi100.stdout: "log_to_file", 2024-04-07T17:51:27.120 INFO:tasks.workunit.client.0.smithi100.stdout: "log_to_journald", 2024-04-07T17:51:27.120 INFO:tasks.workunit.client.0.smithi100.stdout: "log_to_stderr", 2024-04-07T17:51:27.120 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_allow_pool_delete", 2024-04-07T17:51:27.120 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_clock_drift_allowed", 2024-04-07T17:51:27.120 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_cluster_log_file_level", 2024-04-07T17:51:27.120 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_cluster_log_to_file", 2024-04-07T17:51:27.120 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_cluster_log_to_stderr", 2024-04-07T17:51:27.120 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_data_avail_warn", 2024-04-07T17:51:27.120 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_election_default_strategy", 2024-04-07T17:51:27.120 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_host", 2024-04-07T17:51:27.120 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_max_pg_per_osd", 2024-04-07T17:51:27.120 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_mds_skip_sanity", 2024-04-07T17:51:27.121 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_mgr_mkfs_grace", 2024-04-07T17:51:27.121 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_osd_prime_pg_temp", 2024-04-07T17:51:27.121 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_osd_reporter_subtree_level", 2024-04-07T17:51:27.121 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_pg_warn_max_object_skew", 2024-04-07T17:51:27.121 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_reweight_min_bytes_per_osd", 2024-04-07T17:51:27.121 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_reweight_min_pgs_per_osd", 2024-04-07T17:51:27.121 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_warn_on_crush_straw_calc_version_zero", 2024-04-07T17:51:27.121 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_warn_on_insecure_global_id_reclaim", 2024-04-07T17:51:27.121 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_warn_on_insecure_global_id_reclaim_allowed", 2024-04-07T17:51:27.121 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_warn_on_legacy_crush_tunables", 2024-04-07T17:51:27.121 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_warn_on_osd_down_out_interval_zero", 2024-04-07T17:51:27.121 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_warn_on_pool_pg_num_not_power_of_two", 2024-04-07T17:51:27.121 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_warn_on_too_few_osds", 2024-04-07T17:51:27.121 INFO:tasks.workunit.client.0.smithi100.stdout: "ms_die_on_bug", 2024-04-07T17:51:27.121 INFO:tasks.workunit.client.0.smithi100.stdout: "ms_die_on_old_message", 2024-04-07T17:51:27.122 INFO:tasks.workunit.client.0.smithi100.stdout: "no_config_file", 2024-04-07T17:51:27.122 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_class_default_list", 2024-04-07T17:51:27.122 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_class_load_list", 2024-04-07T17:51:27.122 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_debug_misdirected_ops", 2024-04-07T17:51:27.122 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_debug_op_order", 2024-04-07T17:51:27.122 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_debug_pg_log_writeout", 2024-04-07T17:51:27.122 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_debug_shutdown", 2024-04-07T17:51:27.122 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_debug_verify_cached_snaps", 2024-04-07T17:51:27.122 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_debug_verify_missing_on_start", 2024-04-07T17:51:27.122 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_debug_verify_stray_on_activate", 2024-04-07T17:51:27.122 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_deep_scrub_update_digest_min_age", 2024-04-07T17:51:27.122 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_delete_sleep", 2024-04-07T17:51:27.122 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_delete_sleep_hdd", 2024-04-07T17:51:27.122 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_delete_sleep_hybrid", 2024-04-07T17:51:27.122 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_delete_sleep_ssd", 2024-04-07T17:51:27.123 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_max_backfills", 2024-04-07T17:51:27.123 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mclock_max_capacity_iops_ssd", 2024-04-07T17:51:27.123 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mclock_profile", 2024-04-07T17:51:27.123 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mclock_scheduler_background_best_effort_lim", 2024-04-07T17:51:27.123 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mclock_scheduler_background_best_effort_res", 2024-04-07T17:51:27.123 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mclock_scheduler_background_best_effort_wgt", 2024-04-07T17:51:27.123 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mclock_scheduler_background_recovery_lim", 2024-04-07T17:51:27.123 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mclock_scheduler_background_recovery_res", 2024-04-07T17:51:27.123 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mclock_scheduler_background_recovery_wgt", 2024-04-07T17:51:27.123 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mclock_scheduler_client_lim", 2024-04-07T17:51:27.123 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mclock_scheduler_client_res", 2024-04-07T17:51:27.123 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mclock_scheduler_client_wgt", 2024-04-07T17:51:27.123 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_memory_target", 2024-04-07T17:51:27.123 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_memory_target_autotune", 2024-04-07T17:51:27.123 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_op_queue", 2024-04-07T17:51:27.124 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_op_queue_cut_off", 2024-04-07T17:51:27.124 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_pool_default_erasure_code_profile", 2024-04-07T17:51:27.124 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_pool_default_pg_autoscale_mode", 2024-04-07T17:51:27.124 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_recover_clone_overlap", 2024-04-07T17:51:27.124 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_recovery_max_active", 2024-04-07T17:51:27.124 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_recovery_max_active_hdd", 2024-04-07T17:51:27.124 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_recovery_max_active_ssd", 2024-04-07T17:51:27.124 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_recovery_max_chunk", 2024-04-07T17:51:27.124 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_recovery_sleep", 2024-04-07T17:51:27.124 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_recovery_sleep_hdd", 2024-04-07T17:51:27.124 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_recovery_sleep_hybrid", 2024-04-07T17:51:27.124 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_recovery_sleep_ssd", 2024-04-07T17:51:27.124 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_scrub_load_threshold", 2024-04-07T17:51:27.124 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_scrub_max_interval", 2024-04-07T17:51:27.124 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_scrub_sleep", 2024-04-07T17:51:27.125 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_shutdown_pgref_assert", 2024-04-07T17:51:27.125 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_snap_trim_sleep", 2024-04-07T17:51:27.125 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_snap_trim_sleep_hdd", 2024-04-07T17:51:27.125 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_snap_trim_sleep_hybrid", 2024-04-07T17:51:27.125 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_snap_trim_sleep_ssd", 2024-04-07T17:51:27.125 INFO:tasks.workunit.client.0.smithi100.stdout: "public_network", 2024-04-07T17:51:27.125 INFO:tasks.workunit.client.0.smithi100.stdout: "rbd_config_pool_override_update_timestamp", 2024-04-07T17:51:27.125 INFO:tasks.workunit.client.0.smithi100.stdout: "rbd_default_features", 2024-04-07T17:51:27.125 INFO:tasks.workunit.client.0.smithi100.stdout: "rbd_qos_exclude_ops", 2024-04-07T17:51:27.125 INFO:tasks.workunit.client.0.smithi100.stdout: "setgroup", 2024-04-07T17:51:27.125 INFO:tasks.workunit.client.0.smithi100.stdout: "setuser" 2024-04-07T17:51:27.125 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:27.125 INFO:tasks.workunit.client.0.smithi100.stdout: "cluster_changed": [ 2024-04-07T17:51:27.125 INFO:tasks.workunit.client.0.smithi100.stdout: "auth_allow_insecure_global_id_reclaim", 2024-04-07T17:51:27.125 INFO:tasks.workunit.client.0.smithi100.stdout: "auth_debug", 2024-04-07T17:51:27.125 INFO:tasks.workunit.client.0.smithi100.stdout: "auth_mon_ticket_ttl", 2024-04-07T17:51:27.126 INFO:tasks.workunit.client.0.smithi100.stdout: "auth_service_ticket_ttl", 2024-04-07T17:51:27.126 INFO:tasks.workunit.client.0.smithi100.stdout: "bdev_debug_aio", 2024-04-07T17:51:27.126 INFO:tasks.workunit.client.0.smithi100.stdout: "container_image", 2024-04-07T17:51:27.126 INFO:tasks.workunit.client.0.smithi100.stdout: "debug_asserts_on_shutdown", 2024-04-07T17:51:27.126 INFO:tasks.workunit.client.0.smithi100.stdout: "debug_mgr", 2024-04-07T17:51:27.126 INFO:tasks.workunit.client.0.smithi100.stdout: "debug_mon", 2024-04-07T17:51:27.126 INFO:tasks.workunit.client.0.smithi100.stdout: "debug_ms", 2024-04-07T17:51:27.126 INFO:tasks.workunit.client.0.smithi100.stdout: "debug_osd", 2024-04-07T17:51:27.126 INFO:tasks.workunit.client.0.smithi100.stdout: "debug_paxos", 2024-04-07T17:51:27.126 INFO:tasks.workunit.client.0.smithi100.stdout: "log_to_file", 2024-04-07T17:51:27.126 INFO:tasks.workunit.client.0.smithi100.stdout: "log_to_journald", 2024-04-07T17:51:27.126 INFO:tasks.workunit.client.0.smithi100.stdout: "log_to_stderr", 2024-04-07T17:51:27.126 INFO:tasks.workunit.client.0.smithi100.stdout: "mds_join_fs", 2024-04-07T17:51:27.126 INFO:tasks.workunit.client.0.smithi100.stdout: "mgr/cephadm/allow_ptrace", 2024-04-07T17:51:27.127 INFO:tasks.workunit.client.0.smithi100.stdout: "mgr/cephadm/container_init", 2024-04-07T17:51:27.127 INFO:tasks.workunit.client.0.smithi100.stdout: "mgr/cephadm/migration_current", 2024-04-07T17:51:27.127 INFO:tasks.workunit.client.0.smithi100.stdout: "mgr/cephadm/use_repo_digest", 2024-04-07T17:51:27.127 INFO:tasks.workunit.client.0.smithi100.stdout: "mgr/dashboard/ALERTMANAGER_API_HOST", 2024-04-07T17:51:27.127 INFO:tasks.workunit.client.0.smithi100.stdout: "mgr/dashboard/GRAFANA_API_SSL_VERIFY", 2024-04-07T17:51:27.127 INFO:tasks.workunit.client.0.smithi100.stdout: "mgr/dashboard/GRAFANA_API_URL", 2024-04-07T17:51:27.127 INFO:tasks.workunit.client.0.smithi100.stdout: "mgr/dashboard/PROMETHEUS_API_HOST", 2024-04-07T17:51:27.127 INFO:tasks.workunit.client.0.smithi100.stdout: "mgr/dashboard/ssl_server_port", 2024-04-07T17:51:27.127 INFO:tasks.workunit.client.0.smithi100.stdout: "mgr/orchestrator/orchestrator", 2024-04-07T17:51:27.127 INFO:tasks.workunit.client.0.smithi100.stdout: "mgr/telemetry/channel_ident", 2024-04-07T17:51:27.127 INFO:tasks.workunit.client.0.smithi100.stdout: "mgr/telemetry/channel_perf", 2024-04-07T17:51:27.127 INFO:tasks.workunit.client.0.smithi100.stdout: "mgr/telemetry/description", 2024-04-07T17:51:27.127 INFO:tasks.workunit.client.0.smithi100.stdout: "mgr/telemetry/enabled", 2024-04-07T17:51:27.127 INFO:tasks.workunit.client.0.smithi100.stdout: "mgr/telemetry/organization", 2024-04-07T17:51:27.127 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_allow_pool_delete", 2024-04-07T17:51:27.127 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_clock_drift_allowed", 2024-04-07T17:51:27.128 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_cluster_log_file_level", 2024-04-07T17:51:27.128 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_cluster_log_to_file", 2024-04-07T17:51:27.128 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_data_avail_warn", 2024-04-07T17:51:27.128 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_election_default_strategy", 2024-04-07T17:51:27.128 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_max_pg_per_osd", 2024-04-07T17:51:27.128 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_mds_skip_sanity", 2024-04-07T17:51:27.128 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_mgr_mkfs_grace", 2024-04-07T17:51:27.128 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_osd_prime_pg_temp", 2024-04-07T17:51:27.128 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_osd_reporter_subtree_level", 2024-04-07T17:51:27.128 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_pg_warn_max_object_skew", 2024-04-07T17:51:27.128 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_reweight_min_bytes_per_osd", 2024-04-07T17:51:27.128 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_reweight_min_pgs_per_osd", 2024-04-07T17:51:27.128 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_warn_on_crush_straw_calc_version_zero", 2024-04-07T17:51:27.128 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_warn_on_insecure_global_id_reclaim", 2024-04-07T17:51:27.128 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_warn_on_insecure_global_id_reclaim_allowed", 2024-04-07T17:51:27.129 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_warn_on_legacy_crush_tunables", 2024-04-07T17:51:27.129 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_warn_on_osd_down_out_interval_zero", 2024-04-07T17:51:27.129 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_warn_on_pool_pg_num_not_power_of_two", 2024-04-07T17:51:27.129 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_warn_on_too_few_osds", 2024-04-07T17:51:27.129 INFO:tasks.workunit.client.0.smithi100.stdout: "ms_die_on_bug", 2024-04-07T17:51:27.129 INFO:tasks.workunit.client.0.smithi100.stdout: "ms_die_on_old_message", 2024-04-07T17:51:27.129 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_class_default_list", 2024-04-07T17:51:27.129 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_class_load_list", 2024-04-07T17:51:27.129 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_debug_misdirected_ops", 2024-04-07T17:51:27.129 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_debug_op_order", 2024-04-07T17:51:27.129 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_debug_pg_log_writeout", 2024-04-07T17:51:27.129 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_debug_shutdown", 2024-04-07T17:51:27.129 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_debug_verify_cached_snaps", 2024-04-07T17:51:27.129 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_debug_verify_missing_on_start", 2024-04-07T17:51:27.129 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_debug_verify_stray_on_activate", 2024-04-07T17:51:27.130 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_deep_scrub_update_digest_min_age", 2024-04-07T17:51:27.130 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mclock_max_capacity_iops_ssd", 2024-04-07T17:51:27.130 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mclock_profile", 2024-04-07T17:51:27.130 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_memory_target", 2024-04-07T17:51:27.130 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_memory_target_autotune", 2024-04-07T17:51:27.130 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_op_queue", 2024-04-07T17:51:27.130 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_op_queue_cut_off", 2024-04-07T17:51:27.130 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_pool_default_erasure_code_profile", 2024-04-07T17:51:27.130 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_pool_default_pg_autoscale_mode", 2024-04-07T17:51:27.130 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_recover_clone_overlap", 2024-04-07T17:51:27.130 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_recovery_max_chunk", 2024-04-07T17:51:27.130 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_scrub_load_threshold", 2024-04-07T17:51:27.131 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_scrub_max_interval", 2024-04-07T17:51:27.131 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_shutdown_pgref_assert", 2024-04-07T17:51:27.131 INFO:tasks.workunit.client.0.smithi100.stdout: "public_network", 2024-04-07T17:51:27.131 INFO:tasks.workunit.client.0.smithi100.stdout: "rbd_config_pool_override_update_timestamp", 2024-04-07T17:51:27.131 INFO:tasks.workunit.client.0.smithi100.stdout: "rgw_cache_enabled", 2024-04-07T17:51:27.131 INFO:tasks.workunit.client.0.smithi100.stdout: "rgw_enable_ops_log", 2024-04-07T17:51:27.131 INFO:tasks.workunit.client.0.smithi100.stdout: "rgw_enable_usage_log" 2024-04-07T17:51:27.131 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:27.131 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.131 INFO:tasks.workunit.client.0.smithi100.stdout: "contact": null, 2024-04-07T17:51:27.131 INFO:tasks.workunit.client.0.smithi100.stdout: "crashes": [], 2024-04-07T17:51:27.131 INFO:tasks.workunit.client.0.smithi100.stdout: "created": "2024-04-07T16:02:17.280630Z", 2024-04-07T17:51:27.131 INFO:tasks.workunit.client.0.smithi100.stdout: "crush": { 2024-04-07T17:51:27.132 INFO:tasks.workunit.client.0.smithi100.stdout: "bucket_algs": { 2024-04-07T17:51:27.132 INFO:tasks.workunit.client.0.smithi100.stdout: "straw2": 3 2024-04-07T17:51:27.132 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.132 INFO:tasks.workunit.client.0.smithi100.stdout: "bucket_sizes": { 2024-04-07T17:51:27.132 INFO:tasks.workunit.client.0.smithi100.stdout: "2": 1, 2024-04-07T17:51:27.132 INFO:tasks.workunit.client.0.smithi100.stdout: "4": 2 2024-04-07T17:51:27.132 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.132 INFO:tasks.workunit.client.0.smithi100.stdout: "bucket_types": { 2024-04-07T17:51:27.132 INFO:tasks.workunit.client.0.smithi100.stdout: "1": 2, 2024-04-07T17:51:27.132 INFO:tasks.workunit.client.0.smithi100.stdout: "11": 1 2024-04-07T17:51:27.132 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.132 INFO:tasks.workunit.client.0.smithi100.stdout: "compat_weight_set": false, 2024-04-07T17:51:27.132 INFO:tasks.workunit.client.0.smithi100.stdout: "device_classes": [ 2024-04-07T17:51:27.132 INFO:tasks.workunit.client.0.smithi100.stdout: 8 2024-04-07T17:51:27.132 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:27.133 INFO:tasks.workunit.client.0.smithi100.stdout: "num_buckets": 6, 2024-04-07T17:51:27.133 INFO:tasks.workunit.client.0.smithi100.stdout: "num_devices": 8, 2024-04-07T17:51:27.133 INFO:tasks.workunit.client.0.smithi100.stdout: "num_rules": 2, 2024-04-07T17:51:27.133 INFO:tasks.workunit.client.0.smithi100.stdout: "num_types": 12, 2024-04-07T17:51:27.133 INFO:tasks.workunit.client.0.smithi100.stdout: "num_weight_sets": 0, 2024-04-07T17:51:27.133 INFO:tasks.workunit.client.0.smithi100.stdout: "tunables": { 2024-04-07T17:51:27.133 INFO:tasks.workunit.client.0.smithi100.stdout: "allowed_bucket_algs": 54, 2024-04-07T17:51:27.133 INFO:tasks.workunit.client.0.smithi100.stdout: "choose_local_fallback_tries": 0, 2024-04-07T17:51:27.133 INFO:tasks.workunit.client.0.smithi100.stdout: "choose_local_tries": 0, 2024-04-07T17:51:27.133 INFO:tasks.workunit.client.0.smithi100.stdout: "choose_total_tries": 50, 2024-04-07T17:51:27.133 INFO:tasks.workunit.client.0.smithi100.stdout: "chooseleaf_descend_once": 1, 2024-04-07T17:51:27.133 INFO:tasks.workunit.client.0.smithi100.stdout: "chooseleaf_stable": 1, 2024-04-07T17:51:27.133 INFO:tasks.workunit.client.0.smithi100.stdout: "chooseleaf_vary_r": 1, 2024-04-07T17:51:27.133 INFO:tasks.workunit.client.0.smithi100.stdout: "has_msr_rules": 0, 2024-04-07T17:51:27.133 INFO:tasks.workunit.client.0.smithi100.stdout: "has_v2_rules": 1, 2024-04-07T17:51:27.134 INFO:tasks.workunit.client.0.smithi100.stdout: "has_v3_rules": 0, 2024-04-07T17:51:27.134 INFO:tasks.workunit.client.0.smithi100.stdout: "has_v4_buckets": 1, 2024-04-07T17:51:27.134 INFO:tasks.workunit.client.0.smithi100.stdout: "has_v5_rules": 0, 2024-04-07T17:51:27.134 INFO:tasks.workunit.client.0.smithi100.stdout: "legacy_tunables": 0, 2024-04-07T17:51:27.134 INFO:tasks.workunit.client.0.smithi100.stdout: "minimum_required_version": "jewel", 2024-04-07T17:51:27.134 INFO:tasks.workunit.client.0.smithi100.stdout: "msr_collision_tries": 100, 2024-04-07T17:51:27.134 INFO:tasks.workunit.client.0.smithi100.stdout: "msr_descents": 100, 2024-04-07T17:51:27.134 INFO:tasks.workunit.client.0.smithi100.stdout: "optimal_tunables": 1, 2024-04-07T17:51:27.134 INFO:tasks.workunit.client.0.smithi100.stdout: "profile": "jewel", 2024-04-07T17:51:27.134 INFO:tasks.workunit.client.0.smithi100.stdout: "require_feature_tunables": 1, 2024-04-07T17:51:27.134 INFO:tasks.workunit.client.0.smithi100.stdout: "require_feature_tunables2": 1, 2024-04-07T17:51:27.134 INFO:tasks.workunit.client.0.smithi100.stdout: "require_feature_tunables3": 1, 2024-04-07T17:51:27.134 INFO:tasks.workunit.client.0.smithi100.stdout: "require_feature_tunables5": 1, 2024-04-07T17:51:27.134 INFO:tasks.workunit.client.0.smithi100.stdout: "straw_calc_version": 1 2024-04-07T17:51:27.135 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.135 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.135 INFO:tasks.workunit.client.0.smithi100.stdout: "description": "upgrade test cluster", 2024-04-07T17:51:27.135 INFO:tasks.workunit.client.0.smithi100.stdout: "fs": { 2024-04-07T17:51:27.135 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1, 2024-04-07T17:51:27.135 INFO:tasks.workunit.client.0.smithi100.stdout: "feature_flags": { 2024-04-07T17:51:27.135 INFO:tasks.workunit.client.0.smithi100.stdout: "enable_multiple": true, 2024-04-07T17:51:27.135 INFO:tasks.workunit.client.0.smithi100.stdout: "ever_enabled_multiple": true 2024-04-07T17:51:27.135 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.135 INFO:tasks.workunit.client.0.smithi100.stdout: "filesystems": [ 2024-04-07T17:51:27.135 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:27.135 INFO:tasks.workunit.client.0.smithi100.stdout: "approx_ctime": "2024-04", 2024-04-07T17:51:27.135 INFO:tasks.workunit.client.0.smithi100.stdout: "balancer_enabled": false, 2024-04-07T17:51:27.135 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.135 INFO:tasks.workunit.client.0.smithi100.stdout: "cached_caps": 0, 2024-04-07T17:51:27.135 INFO:tasks.workunit.client.0.smithi100.stdout: "cached_dns": 10, 2024-04-07T17:51:27.136 INFO:tasks.workunit.client.0.smithi100.stdout: "cached_inos": 13, 2024-04-07T17:51:27.136 INFO:tasks.workunit.client.0.smithi100.stdout: "cached_subtrees": 2, 2024-04-07T17:51:27.136 INFO:tasks.workunit.client.0.smithi100.stdout: "ever_allowed_features": 0, 2024-04-07T17:51:27.136 INFO:tasks.workunit.client.0.smithi100.stdout: "explicitly_allowed_features": 0, 2024-04-07T17:51:27.136 INFO:tasks.workunit.client.0.smithi100.stdout: "files": 0, 2024-04-07T17:51:27.136 INFO:tasks.workunit.client.0.smithi100.stdout: "max_mds": 1, 2024-04-07T17:51:27.136 INFO:tasks.workunit.client.0.smithi100.stdout: "num_data_pools": 1, 2024-04-07T17:51:27.136 INFO:tasks.workunit.client.0.smithi100.stdout: "num_in": 1, 2024-04-07T17:51:27.136 INFO:tasks.workunit.client.0.smithi100.stdout: "num_mds": 1, 2024-04-07T17:51:27.136 INFO:tasks.workunit.client.0.smithi100.stdout: "num_sessions": 0, 2024-04-07T17:51:27.136 INFO:tasks.workunit.client.0.smithi100.stdout: "num_standby_replay": 0, 2024-04-07T17:51:27.136 INFO:tasks.workunit.client.0.smithi100.stdout: "num_up": 1, 2024-04-07T17:51:27.136 INFO:tasks.workunit.client.0.smithi100.stdout: "snaps": 0, 2024-04-07T17:51:27.136 INFO:tasks.workunit.client.0.smithi100.stdout: "standby_count_wanted": 0 2024-04-07T17:51:27.137 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.137 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:27.137 INFO:tasks.workunit.client.0.smithi100.stdout: "num_standby_mds": 1, 2024-04-07T17:51:27.137 INFO:tasks.workunit.client.0.smithi100.stdout: "total_num_mds": 2 2024-04-07T17:51:27.137 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.137 INFO:tasks.workunit.client.0.smithi100.stdout: "heap_stats": { 2024-04-07T17:51:27.137 INFO:tasks.workunit.client.0.smithi100.stdout: "mds": { 2024-04-07T17:51:27.137 INFO:tasks.workunit.client.0.smithi100.stdout: "mds.a52fc0d8acb515431879057649ff294cfe14c620": { 2024-04-07T17:51:27.137 INFO:tasks.workunit.client.0.smithi100.stdout: "actual_memory_used": 20258816, 2024-04-07T17:51:27.137 INFO:tasks.workunit.client.0.smithi100.stdout: "central_cache_freelist": 394352, 2024-04-07T17:51:27.137 INFO:tasks.workunit.client.0.smithi100.stdout: "malloc_metadata": 2752512, 2024-04-07T17:51:27.137 INFO:tasks.workunit.client.0.smithi100.stdout: "page_heap_freelist": 827392, 2024-04-07T17:51:27.137 INFO:tasks.workunit.client.0.smithi100.stdout: "released_to_os": 0, 2024-04-07T17:51:27.137 INFO:tasks.workunit.client.0.smithi100.stdout: "spans_in_use": 343, 2024-04-07T17:51:27.137 INFO:tasks.workunit.client.0.smithi100.stdout: "tcmalloc_page_size": 8192, 2024-04-07T17:51:27.137 INFO:tasks.workunit.client.0.smithi100.stdout: "thread_cache_freelists": 1100168, 2024-04-07T17:51:27.138 INFO:tasks.workunit.client.0.smithi100.stdout: "thread_heaps_in_use": 13, 2024-04-07T17:51:27.138 INFO:tasks.workunit.client.0.smithi100.stdout: "transfer_cache_freelist": 701952, 2024-04-07T17:51:27.138 INFO:tasks.workunit.client.0.smithi100.stdout: "use_by_application": 14482440, 2024-04-07T17:51:27.138 INFO:tasks.workunit.client.0.smithi100.stdout: "virtual_address_space_used": 20258816 2024-04-07T17:51:27.138 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.138 INFO:tasks.workunit.client.0.smithi100.stdout: "mds.f47a4ff15a45950a63160265564a123fc5da5e5e": { 2024-04-07T17:51:27.138 INFO:tasks.workunit.client.0.smithi100.stdout: "actual_memory_used": 20447232, 2024-04-07T17:51:27.138 INFO:tasks.workunit.client.0.smithi100.stdout: "central_cache_freelist": 436568, 2024-04-07T17:51:27.138 INFO:tasks.workunit.client.0.smithi100.stdout: "malloc_metadata": 2752512, 2024-04-07T17:51:27.138 INFO:tasks.workunit.client.0.smithi100.stdout: "page_heap_freelist": 0, 2024-04-07T17:51:27.138 INFO:tasks.workunit.client.0.smithi100.stdout: "released_to_os": 860160, 2024-04-07T17:51:27.138 INFO:tasks.workunit.client.0.smithi100.stdout: "spans_in_use": 429, 2024-04-07T17:51:27.138 INFO:tasks.workunit.client.0.smithi100.stdout: "tcmalloc_page_size": 8192, 2024-04-07T17:51:27.138 INFO:tasks.workunit.client.0.smithi100.stdout: "thread_cache_freelists": 1271136, 2024-04-07T17:51:27.139 INFO:tasks.workunit.client.0.smithi100.stdout: "thread_heaps_in_use": 22, 2024-04-07T17:51:27.139 INFO:tasks.workunit.client.0.smithi100.stdout: "transfer_cache_freelist": 1104384, 2024-04-07T17:51:27.139 INFO:tasks.workunit.client.0.smithi100.stdout: "use_by_application": 14882632, 2024-04-07T17:51:27.139 INFO:tasks.workunit.client.0.smithi100.stdout: "virtual_address_space_used": 21307392 2024-04-07T17:51:27.139 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.139 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.139 INFO:tasks.workunit.client.0.smithi100.stdout: "mon": { 2024-04-07T17:51:27.139 INFO:tasks.workunit.client.0.smithi100.stdout: "mon.8dd0baa036f0a7e6d1f3bdce86136fe527836c3d": { 2024-04-07T17:51:27.139 INFO:tasks.workunit.client.0.smithi100.stdout: "actual_memory_used": 172793856, 2024-04-07T17:51:27.139 INFO:tasks.workunit.client.0.smithi100.stdout: "central_cache_freelist": 2006744, 2024-04-07T17:51:27.139 INFO:tasks.workunit.client.0.smithi100.stdout: "malloc_metadata": 5111808, 2024-04-07T17:51:27.139 INFO:tasks.workunit.client.0.smithi100.stdout: "page_heap_freelist": 983040, 2024-04-07T17:51:27.139 INFO:tasks.workunit.client.0.smithi100.stdout: "released_to_os": 49152, 2024-04-07T17:51:27.139 INFO:tasks.workunit.client.0.smithi100.stdout: "spans_in_use": 6331, 2024-04-07T17:51:27.139 INFO:tasks.workunit.client.0.smithi100.stdout: "tcmalloc_page_size": 8192, 2024-04-07T17:51:27.140 INFO:tasks.workunit.client.0.smithi100.stdout: "thread_cache_freelists": 10920960, 2024-04-07T17:51:27.140 INFO:tasks.workunit.client.0.smithi100.stdout: "thread_heaps_in_use": 21, 2024-04-07T17:51:27.140 INFO:tasks.workunit.client.0.smithi100.stdout: "transfer_cache_freelist": 3323136, 2024-04-07T17:51:27.140 INFO:tasks.workunit.client.0.smithi100.stdout: "use_by_application": 150448168, 2024-04-07T17:51:27.140 INFO:tasks.workunit.client.0.smithi100.stdout: "virtual_address_space_used": 172843008 2024-04-07T17:51:27.140 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.140 INFO:tasks.workunit.client.0.smithi100.stdout: "mon.a52fc0d8acb515431879057649ff294cfe14c620": { 2024-04-07T17:51:27.140 INFO:tasks.workunit.client.0.smithi100.stdout: "actual_memory_used": 166387712, 2024-04-07T17:51:27.140 INFO:tasks.workunit.client.0.smithi100.stdout: "central_cache_freelist": 2385656, 2024-04-07T17:51:27.140 INFO:tasks.workunit.client.0.smithi100.stdout: "malloc_metadata": 3014656, 2024-04-07T17:51:27.140 INFO:tasks.workunit.client.0.smithi100.stdout: "page_heap_freelist": 0, 2024-04-07T17:51:27.140 INFO:tasks.workunit.client.0.smithi100.stdout: "released_to_os": 1212416, 2024-04-07T17:51:27.140 INFO:tasks.workunit.client.0.smithi100.stdout: "spans_in_use": 6234, 2024-04-07T17:51:27.140 INFO:tasks.workunit.client.0.smithi100.stdout: "tcmalloc_page_size": 8192, 2024-04-07T17:51:27.140 INFO:tasks.workunit.client.0.smithi100.stdout: "thread_cache_freelists": 9417856, 2024-04-07T17:51:27.140 INFO:tasks.workunit.client.0.smithi100.stdout: "thread_heaps_in_use": 21, 2024-04-07T17:51:27.141 INFO:tasks.workunit.client.0.smithi100.stdout: "transfer_cache_freelist": 2757888, 2024-04-07T17:51:27.141 INFO:tasks.workunit.client.0.smithi100.stdout: "use_by_application": 148811656, 2024-04-07T17:51:27.141 INFO:tasks.workunit.client.0.smithi100.stdout: "virtual_address_space_used": 167600128 2024-04-07T17:51:27.141 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.141 INFO:tasks.workunit.client.0.smithi100.stdout: "mon.f47a4ff15a45950a63160265564a123fc5da5e5e": { 2024-04-07T17:51:27.141 INFO:tasks.workunit.client.0.smithi100.stdout: "actual_memory_used": 263282688, 2024-04-07T17:51:27.141 INFO:tasks.workunit.client.0.smithi100.stdout: "central_cache_freelist": 6464640, 2024-04-07T17:51:27.141 INFO:tasks.workunit.client.0.smithi100.stdout: "malloc_metadata": 3145728, 2024-04-07T17:51:27.141 INFO:tasks.workunit.client.0.smithi100.stdout: "page_heap_freelist": 1400832, 2024-04-07T17:51:27.141 INFO:tasks.workunit.client.0.smithi100.stdout: "released_to_os": 73728, 2024-04-07T17:51:27.141 INFO:tasks.workunit.client.0.smithi100.stdout: "spans_in_use": 9419, 2024-04-07T17:51:27.141 INFO:tasks.workunit.client.0.smithi100.stdout: "tcmalloc_page_size": 8192, 2024-04-07T17:51:27.141 INFO:tasks.workunit.client.0.smithi100.stdout: "thread_cache_freelists": 14944496, 2024-04-07T17:51:27.141 INFO:tasks.workunit.client.0.smithi100.stdout: "thread_heaps_in_use": 23, 2024-04-07T17:51:27.141 INFO:tasks.workunit.client.0.smithi100.stdout: "transfer_cache_freelist": 5602048, 2024-04-07T17:51:27.141 INFO:tasks.workunit.client.0.smithi100.stdout: "use_by_application": 231724944, 2024-04-07T17:51:27.142 INFO:tasks.workunit.client.0.smithi100.stdout: "virtual_address_space_used": 263356416 2024-04-07T17:51:27.142 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.142 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.142 INFO:tasks.workunit.client.0.smithi100.stdout: "osd": { 2024-04-07T17:51:27.142 INFO:tasks.workunit.client.0.smithi100.stdout: "osd.0": { 2024-04-07T17:51:27.142 INFO:tasks.workunit.client.0.smithi100.stdout: "actual_memory_used": 473178112, 2024-04-07T17:51:27.142 INFO:tasks.workunit.client.0.smithi100.stdout: "central_cache_freelist": 27162800, 2024-04-07T17:51:27.142 INFO:tasks.workunit.client.0.smithi100.stdout: "malloc_metadata": 3407872, 2024-04-07T17:51:27.142 INFO:tasks.workunit.client.0.smithi100.stdout: "page_heap_freelist": 0, 2024-04-07T17:51:27.142 INFO:tasks.workunit.client.0.smithi100.stdout: "released_to_os": 38920192, 2024-04-07T17:51:27.142 INFO:tasks.workunit.client.0.smithi100.stdout: "spans_in_use": 12519, 2024-04-07T17:51:27.142 INFO:tasks.workunit.client.0.smithi100.stdout: "tcmalloc_page_size": 8192, 2024-04-07T17:51:27.142 INFO:tasks.workunit.client.0.smithi100.stdout: "thread_cache_freelists": 94784960, 2024-04-07T17:51:27.142 INFO:tasks.workunit.client.0.smithi100.stdout: "thread_heaps_in_use": 60, 2024-04-07T17:51:27.143 INFO:tasks.workunit.client.0.smithi100.stdout: "transfer_cache_freelist": 8584704, 2024-04-07T17:51:27.143 INFO:tasks.workunit.client.0.smithi100.stdout: "use_by_application": 339237776, 2024-04-07T17:51:27.143 INFO:tasks.workunit.client.0.smithi100.stdout: "virtual_address_space_used": 512098304 2024-04-07T17:51:27.143 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.143 INFO:tasks.workunit.client.0.smithi100.stdout: "osd.1": { 2024-04-07T17:51:27.143 INFO:tasks.workunit.client.0.smithi100.stdout: "actual_memory_used": 664215552, 2024-04-07T17:51:27.143 INFO:tasks.workunit.client.0.smithi100.stdout: "central_cache_freelist": 27226272, 2024-04-07T17:51:27.143 INFO:tasks.workunit.client.0.smithi100.stdout: "malloc_metadata": 3407872, 2024-04-07T17:51:27.143 INFO:tasks.workunit.client.0.smithi100.stdout: "page_heap_freelist": 0, 2024-04-07T17:51:27.143 INFO:tasks.workunit.client.0.smithi100.stdout: "released_to_os": 35110912, 2024-04-07T17:51:27.143 INFO:tasks.workunit.client.0.smithi100.stdout: "spans_in_use": 13060, 2024-04-07T17:51:27.143 INFO:tasks.workunit.client.0.smithi100.stdout: "tcmalloc_page_size": 8192, 2024-04-07T17:51:27.143 INFO:tasks.workunit.client.0.smithi100.stdout: "thread_cache_freelists": 85586928, 2024-04-07T17:51:27.143 INFO:tasks.workunit.client.0.smithi100.stdout: "thread_heaps_in_use": 53, 2024-04-07T17:51:27.143 INFO:tasks.workunit.client.0.smithi100.stdout: "transfer_cache_freelist": 8398592, 2024-04-07T17:51:27.143 INFO:tasks.workunit.client.0.smithi100.stdout: "use_by_application": 539595888, 2024-04-07T17:51:27.144 INFO:tasks.workunit.client.0.smithi100.stdout: "virtual_address_space_used": 699326464 2024-04-07T17:51:27.144 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.144 INFO:tasks.workunit.client.0.smithi100.stdout: "osd.2": { 2024-04-07T17:51:27.144 INFO:tasks.workunit.client.0.smithi100.stdout: "actual_memory_used": 517300224, 2024-04-07T17:51:27.144 INFO:tasks.workunit.client.0.smithi100.stdout: "central_cache_freelist": 26270840, 2024-04-07T17:51:27.144 INFO:tasks.workunit.client.0.smithi100.stdout: "malloc_metadata": 3276800, 2024-04-07T17:51:27.144 INFO:tasks.workunit.client.0.smithi100.stdout: "page_heap_freelist": 0, 2024-04-07T17:51:27.144 INFO:tasks.workunit.client.0.smithi100.stdout: "released_to_os": 49258496, 2024-04-07T17:51:27.144 INFO:tasks.workunit.client.0.smithi100.stdout: "spans_in_use": 10733, 2024-04-07T17:51:27.144 INFO:tasks.workunit.client.0.smithi100.stdout: "tcmalloc_page_size": 8192, 2024-04-07T17:51:27.144 INFO:tasks.workunit.client.0.smithi100.stdout: "thread_cache_freelists": 85543536, 2024-04-07T17:51:27.144 INFO:tasks.workunit.client.0.smithi100.stdout: "thread_heaps_in_use": 60, 2024-04-07T17:51:27.144 INFO:tasks.workunit.client.0.smithi100.stdout: "transfer_cache_freelist": 9229824, 2024-04-07T17:51:27.144 INFO:tasks.workunit.client.0.smithi100.stdout: "use_by_application": 392979224, 2024-04-07T17:51:27.144 INFO:tasks.workunit.client.0.smithi100.stdout: "virtual_address_space_used": 566558720 2024-04-07T17:51:27.145 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.145 INFO:tasks.workunit.client.0.smithi100.stdout: "osd.3": { 2024-04-07T17:51:27.145 INFO:tasks.workunit.client.0.smithi100.stdout: "actual_memory_used": 473931776, 2024-04-07T17:51:27.145 INFO:tasks.workunit.client.0.smithi100.stdout: "central_cache_freelist": 20241576, 2024-04-07T17:51:27.145 INFO:tasks.workunit.client.0.smithi100.stdout: "malloc_metadata": 3276800, 2024-04-07T17:51:27.145 INFO:tasks.workunit.client.0.smithi100.stdout: "page_heap_freelist": 49152, 2024-04-07T17:51:27.145 INFO:tasks.workunit.client.0.smithi100.stdout: "released_to_os": 33120256, 2024-04-07T17:51:27.145 INFO:tasks.workunit.client.0.smithi100.stdout: "spans_in_use": 10246, 2024-04-07T17:51:27.145 INFO:tasks.workunit.client.0.smithi100.stdout: "tcmalloc_page_size": 8192, 2024-04-07T17:51:27.145 INFO:tasks.workunit.client.0.smithi100.stdout: "thread_cache_freelists": 80029224, 2024-04-07T17:51:27.145 INFO:tasks.workunit.client.0.smithi100.stdout: "thread_heaps_in_use": 60, 2024-04-07T17:51:27.145 INFO:tasks.workunit.client.0.smithi100.stdout: "transfer_cache_freelist": 9942016, 2024-04-07T17:51:27.145 INFO:tasks.workunit.client.0.smithi100.stdout: "use_by_application": 360393008, 2024-04-07T17:51:27.145 INFO:tasks.workunit.client.0.smithi100.stdout: "virtual_address_space_used": 507052032 2024-04-07T17:51:27.145 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.146 INFO:tasks.workunit.client.0.smithi100.stdout: "osd.4": { 2024-04-07T17:51:27.146 INFO:tasks.workunit.client.0.smithi100.stdout: "actual_memory_used": 596754432, 2024-04-07T17:51:27.146 INFO:tasks.workunit.client.0.smithi100.stdout: "central_cache_freelist": 24852048, 2024-04-07T17:51:27.146 INFO:tasks.workunit.client.0.smithi100.stdout: "malloc_metadata": 3407872, 2024-04-07T17:51:27.146 INFO:tasks.workunit.client.0.smithi100.stdout: "page_heap_freelist": 0, 2024-04-07T17:51:27.146 INFO:tasks.workunit.client.0.smithi100.stdout: "released_to_os": 29917184, 2024-04-07T17:51:27.146 INFO:tasks.workunit.client.0.smithi100.stdout: "spans_in_use": 11405, 2024-04-07T17:51:27.146 INFO:tasks.workunit.client.0.smithi100.stdout: "tcmalloc_page_size": 8192, 2024-04-07T17:51:27.146 INFO:tasks.workunit.client.0.smithi100.stdout: "thread_cache_freelists": 99044656, 2024-04-07T17:51:27.146 INFO:tasks.workunit.client.0.smithi100.stdout: "thread_heaps_in_use": 53, 2024-04-07T17:51:27.146 INFO:tasks.workunit.client.0.smithi100.stdout: "transfer_cache_freelist": 12042752, 2024-04-07T17:51:27.146 INFO:tasks.workunit.client.0.smithi100.stdout: "use_by_application": 457407104, 2024-04-07T17:51:27.146 INFO:tasks.workunit.client.0.smithi100.stdout: "virtual_address_space_used": 626671616 2024-04-07T17:51:27.146 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.147 INFO:tasks.workunit.client.0.smithi100.stdout: "osd.5": { 2024-04-07T17:51:27.147 INFO:tasks.workunit.client.0.smithi100.stdout: "actual_memory_used": 562798592, 2024-04-07T17:51:27.147 INFO:tasks.workunit.client.0.smithi100.stdout: "central_cache_freelist": 27133056, 2024-04-07T17:51:27.147 INFO:tasks.workunit.client.0.smithi100.stdout: "malloc_metadata": 5636096, 2024-04-07T17:51:27.147 INFO:tasks.workunit.client.0.smithi100.stdout: "page_heap_freelist": 0, 2024-04-07T17:51:27.147 INFO:tasks.workunit.client.0.smithi100.stdout: "released_to_os": 13197312, 2024-04-07T17:51:27.147 INFO:tasks.workunit.client.0.smithi100.stdout: "spans_in_use": 13822, 2024-04-07T17:51:27.147 INFO:tasks.workunit.client.0.smithi100.stdout: "tcmalloc_page_size": 8192, 2024-04-07T17:51:27.147 INFO:tasks.workunit.client.0.smithi100.stdout: "thread_cache_freelists": 86793184, 2024-04-07T17:51:27.147 INFO:tasks.workunit.client.0.smithi100.stdout: "thread_heaps_in_use": 60, 2024-04-07T17:51:27.147 INFO:tasks.workunit.client.0.smithi100.stdout: "transfer_cache_freelist": 10701312, 2024-04-07T17:51:27.147 INFO:tasks.workunit.client.0.smithi100.stdout: "use_by_application": 432534944, 2024-04-07T17:51:27.147 INFO:tasks.workunit.client.0.smithi100.stdout: "virtual_address_space_used": 575995904 2024-04-07T17:51:27.147 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.147 INFO:tasks.workunit.client.0.smithi100.stdout: "osd.6": { 2024-04-07T17:51:27.148 INFO:tasks.workunit.client.0.smithi100.stdout: "actual_memory_used": 362029056, 2024-04-07T17:51:27.148 INFO:tasks.workunit.client.0.smithi100.stdout: "central_cache_freelist": 16221696, 2024-04-07T17:51:27.148 INFO:tasks.workunit.client.0.smithi100.stdout: "malloc_metadata": 3276800, 2024-04-07T17:51:27.148 INFO:tasks.workunit.client.0.smithi100.stdout: "page_heap_freelist": 0, 2024-04-07T17:51:27.148 INFO:tasks.workunit.client.0.smithi100.stdout: "released_to_os": 64888832, 2024-04-07T17:51:27.148 INFO:tasks.workunit.client.0.smithi100.stdout: "spans_in_use": 9372, 2024-04-07T17:51:27.148 INFO:tasks.workunit.client.0.smithi100.stdout: "tcmalloc_page_size": 8192, 2024-04-07T17:51:27.148 INFO:tasks.workunit.client.0.smithi100.stdout: "thread_cache_freelists": 50403120, 2024-04-07T17:51:27.148 INFO:tasks.workunit.client.0.smithi100.stdout: "thread_heaps_in_use": 60, 2024-04-07T17:51:27.148 INFO:tasks.workunit.client.0.smithi100.stdout: "transfer_cache_freelist": 9703424, 2024-04-07T17:51:27.148 INFO:tasks.workunit.client.0.smithi100.stdout: "use_by_application": 282424016, 2024-04-07T17:51:27.148 INFO:tasks.workunit.client.0.smithi100.stdout: "virtual_address_space_used": 426917888 2024-04-07T17:51:27.148 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.148 INFO:tasks.workunit.client.0.smithi100.stdout: "osd.7": { 2024-04-07T17:51:27.148 INFO:tasks.workunit.client.0.smithi100.stdout: "actual_memory_used": 434208768, 2024-04-07T17:51:27.149 INFO:tasks.workunit.client.0.smithi100.stdout: "central_cache_freelist": 20861904, 2024-04-07T17:51:27.149 INFO:tasks.workunit.client.0.smithi100.stdout: "malloc_metadata": 3276800, 2024-04-07T17:51:27.149 INFO:tasks.workunit.client.0.smithi100.stdout: "page_heap_freelist": 0, 2024-04-07T17:51:27.149 INFO:tasks.workunit.client.0.smithi100.stdout: "released_to_os": 32399360, 2024-04-07T17:51:27.149 INFO:tasks.workunit.client.0.smithi100.stdout: "spans_in_use": 10456, 2024-04-07T17:51:27.149 INFO:tasks.workunit.client.0.smithi100.stdout: "tcmalloc_page_size": 8192, 2024-04-07T17:51:27.149 INFO:tasks.workunit.client.0.smithi100.stdout: "thread_cache_freelists": 72555712, 2024-04-07T17:51:27.149 INFO:tasks.workunit.client.0.smithi100.stdout: "thread_heaps_in_use": 59, 2024-04-07T17:51:27.149 INFO:tasks.workunit.client.0.smithi100.stdout: "transfer_cache_freelist": 10265856, 2024-04-07T17:51:27.149 INFO:tasks.workunit.client.0.smithi100.stdout: "use_by_application": 327248496, 2024-04-07T17:51:27.149 INFO:tasks.workunit.client.0.smithi100.stdout: "virtual_address_space_used": 466608128 2024-04-07T17:51:27.149 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.149 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.149 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.149 INFO:tasks.workunit.client.0.smithi100.stdout: "hosts": { 2024-04-07T17:51:27.149 INFO:tasks.workunit.client.0.smithi100.stdout: "num": 2, 2024-04-07T17:51:27.150 INFO:tasks.workunit.client.0.smithi100.stdout: "num_with_mds": 2, 2024-04-07T17:51:27.150 INFO:tasks.workunit.client.0.smithi100.stdout: "num_with_mgr": 2, 2024-04-07T17:51:27.150 INFO:tasks.workunit.client.0.smithi100.stdout: "num_with_mon": 2, 2024-04-07T17:51:27.150 INFO:tasks.workunit.client.0.smithi100.stdout: "num_with_osd": 2 2024-04-07T17:51:27.150 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.150 INFO:tasks.workunit.client.0.smithi100.stdout: "io_rate": { 2024-04-07T17:51:27.150 INFO:tasks.workunit.client.0.smithi100.stdout: "pg_stats_delta": { 2024-04-07T17:51:27.150 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": 0, 2024-04-07T17:51:27.150 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:27.150 INFO:tasks.workunit.client.0.smithi100.stdout: "num_store_stats": 0, 2024-04-07T17:51:27.150 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:27.150 INFO:tasks.workunit.client.0.smithi100.stdout: "stamp_delta": "10.867805", 2024-04-07T17:51:27.150 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:27.150 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:27.151 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:27.151 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:27.151 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:27.151 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:27.151 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:27.151 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:27.151 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:27.151 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:27.151 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:27.151 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:27.151 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:27.151 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:27.151 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:27.151 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:27.151 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:27.151 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:27.152 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:27.152 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:27.152 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:27.152 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:27.152 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:27.152 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:27.152 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:27.152 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:27.152 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:27.152 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:27.152 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:27.152 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:27.152 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:27.152 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:27.152 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:27.153 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:27.153 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 11, 2024-04-07T17:51:27.153 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 7, 2024-04-07T17:51:27.153 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:27.153 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:27.153 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:27.153 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:27.153 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:27.153 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.153 INFO:tasks.workunit.client.0.smithi100.stdout: "store_stats": { 2024-04-07T17:51:27.153 INFO:tasks.workunit.client.0.smithi100.stdout: "allocated": 0, 2024-04-07T17:51:27.153 INFO:tasks.workunit.client.0.smithi100.stdout: "available": 0, 2024-04-07T17:51:27.153 INFO:tasks.workunit.client.0.smithi100.stdout: "data_compressed": 0, 2024-04-07T17:51:27.153 INFO:tasks.workunit.client.0.smithi100.stdout: "data_compressed_allocated": 0, 2024-04-07T17:51:27.153 INFO:tasks.workunit.client.0.smithi100.stdout: "data_compressed_original": 0, 2024-04-07T17:51:27.154 INFO:tasks.workunit.client.0.smithi100.stdout: "data_stored": 0, 2024-04-07T17:51:27.154 INFO:tasks.workunit.client.0.smithi100.stdout: "internal_metadata": 0, 2024-04-07T17:51:27.154 INFO:tasks.workunit.client.0.smithi100.stdout: "internally_reserved": 0, 2024-04-07T17:51:27.154 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_allocated": 0, 2024-04-07T17:51:27.154 INFO:tasks.workunit.client.0.smithi100.stdout: "total": 0 2024-04-07T17:51:27.154 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.154 INFO:tasks.workunit.client.0.smithi100.stdout: "up": 0 2024-04-07T17:51:27.154 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.154 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.154 INFO:tasks.workunit.client.0.smithi100.stdout: "leaderboard": false, 2024-04-07T17:51:27.154 INFO:tasks.workunit.client.0.smithi100.stdout: "leaderboard_description": null, 2024-04-07T17:51:27.154 INFO:tasks.workunit.client.0.smithi100.stdout: "license": "sharing-1-0", 2024-04-07T17:51:27.154 INFO:tasks.workunit.client.0.smithi100.stdout: "mempool": { 2024-04-07T17:51:27.154 INFO:tasks.workunit.client.0.smithi100.stdout: "mds": { 2024-04-07T17:51:27.155 INFO:tasks.workunit.client.0.smithi100.stdout: "mds.a52fc0d8acb515431879057649ff294cfe14c620": { 2024-04-07T17:51:27.155 INFO:tasks.workunit.client.0.smithi100.stdout: "bloom_filter": { 2024-04-07T17:51:27.155 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.155 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.155 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.155 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs": { 2024-04-07T17:51:27.155 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.155 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.155 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.155 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs_file_reader": { 2024-04-07T17:51:27.155 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.155 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.155 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.155 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs_file_writer": { 2024-04-07T17:51:27.155 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.156 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.156 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.156 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_alloc": { 2024-04-07T17:51:27.156 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.156 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.156 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.156 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_blob": { 2024-04-07T17:51:27.156 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.156 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.156 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.156 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_buffer": { 2024-04-07T17:51:27.156 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.156 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.156 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.156 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_data": { 2024-04-07T17:51:27.157 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.157 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.157 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.157 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_meta": { 2024-04-07T17:51:27.157 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.157 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.157 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.157 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_onode": { 2024-04-07T17:51:27.157 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.157 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.157 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.157 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_other": { 2024-04-07T17:51:27.157 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.157 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.157 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.157 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_extent": { 2024-04-07T17:51:27.158 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.158 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.158 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.158 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_fsck": { 2024-04-07T17:51:27.158 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.158 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.158 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.158 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_inline_bl": { 2024-04-07T17:51:27.158 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.158 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.158 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.158 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_shared_blob": { 2024-04-07T17:51:27.158 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.158 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.158 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.159 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_txc": { 2024-04-07T17:51:27.159 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.159 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.159 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.159 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_writing": { 2024-04-07T17:51:27.159 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.159 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.159 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.159 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_writing_deferred": { 2024-04-07T17:51:27.159 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.159 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.159 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.159 INFO:tasks.workunit.client.0.smithi100.stdout: "buffer_anon": { 2024-04-07T17:51:27.159 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 149048, 2024-04-07T17:51:27.159 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 34 2024-04-07T17:51:27.160 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.160 INFO:tasks.workunit.client.0.smithi100.stdout: "buffer_meta": { 2024-04-07T17:51:27.160 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.160 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.160 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.160 INFO:tasks.workunit.client.0.smithi100.stdout: "mds_co": { 2024-04-07T17:51:27.160 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 928, 2024-04-07T17:51:27.160 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 2 2024-04-07T17:51:27.160 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.160 INFO:tasks.workunit.client.0.smithi100.stdout: "osd": { 2024-04-07T17:51:27.160 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.160 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.160 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.160 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mapbl": { 2024-04-07T17:51:27.160 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.160 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.161 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.161 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_pglog": { 2024-04-07T17:51:27.161 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.161 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.161 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.161 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap": { 2024-04-07T17:51:27.161 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.161 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.161 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.161 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap_mapping": { 2024-04-07T17:51:27.161 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.161 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.161 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.161 INFO:tasks.workunit.client.0.smithi100.stdout: "pgmap": { 2024-04-07T17:51:27.161 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.161 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.162 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.162 INFO:tasks.workunit.client.0.smithi100.stdout: "unittest_1": { 2024-04-07T17:51:27.162 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.162 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.162 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.162 INFO:tasks.workunit.client.0.smithi100.stdout: "unittest_2": { 2024-04-07T17:51:27.162 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.162 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.162 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.162 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.162 INFO:tasks.workunit.client.0.smithi100.stdout: "mds.f47a4ff15a45950a63160265564a123fc5da5e5e": { 2024-04-07T17:51:27.162 INFO:tasks.workunit.client.0.smithi100.stdout: "bloom_filter": { 2024-04-07T17:51:27.162 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.162 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.162 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.163 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs": { 2024-04-07T17:51:27.163 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.163 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.163 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.163 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs_file_reader": { 2024-04-07T17:51:27.163 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.163 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.163 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.163 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs_file_writer": { 2024-04-07T17:51:27.163 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.163 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.163 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.163 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_alloc": { 2024-04-07T17:51:27.163 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.163 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.164 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.164 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_blob": { 2024-04-07T17:51:27.164 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.164 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.164 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.164 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_buffer": { 2024-04-07T17:51:27.164 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.164 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.164 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.164 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_data": { 2024-04-07T17:51:27.164 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.164 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.164 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.164 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_meta": { 2024-04-07T17:51:27.164 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.164 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.165 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.165 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_onode": { 2024-04-07T17:51:27.165 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.165 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.165 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.165 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_other": { 2024-04-07T17:51:27.165 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.165 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.165 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.165 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_extent": { 2024-04-07T17:51:27.165 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.165 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.165 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.165 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_fsck": { 2024-04-07T17:51:27.165 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.166 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.166 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.166 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_inline_bl": { 2024-04-07T17:51:27.166 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.166 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.166 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.166 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_shared_blob": { 2024-04-07T17:51:27.166 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.166 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.166 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.166 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_txc": { 2024-04-07T17:51:27.166 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.166 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.166 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.166 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_writing": { 2024-04-07T17:51:27.167 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.167 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.167 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.167 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_writing_deferred": { 2024-04-07T17:51:27.167 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.167 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.167 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.167 INFO:tasks.workunit.client.0.smithi100.stdout: "buffer_anon": { 2024-04-07T17:51:27.167 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 221552, 2024-04-07T17:51:27.167 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 64 2024-04-07T17:51:27.167 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.167 INFO:tasks.workunit.client.0.smithi100.stdout: "buffer_meta": { 2024-04-07T17:51:27.167 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.167 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.167 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.168 INFO:tasks.workunit.client.0.smithi100.stdout: "mds_co": { 2024-04-07T17:51:27.168 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 52112, 2024-04-07T17:51:27.168 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 181 2024-04-07T17:51:27.168 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.168 INFO:tasks.workunit.client.0.smithi100.stdout: "osd": { 2024-04-07T17:51:27.168 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.168 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.168 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.168 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mapbl": { 2024-04-07T17:51:27.168 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.168 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.168 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.168 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_pglog": { 2024-04-07T17:51:27.168 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.168 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.168 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.169 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap": { 2024-04-07T17:51:27.169 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 12216, 2024-04-07T17:51:27.169 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 252 2024-04-07T17:51:27.169 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.169 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap_mapping": { 2024-04-07T17:51:27.169 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.169 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.169 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.169 INFO:tasks.workunit.client.0.smithi100.stdout: "pgmap": { 2024-04-07T17:51:27.169 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.169 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.169 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.169 INFO:tasks.workunit.client.0.smithi100.stdout: "unittest_1": { 2024-04-07T17:51:27.169 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.169 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.170 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.170 INFO:tasks.workunit.client.0.smithi100.stdout: "unittest_2": { 2024-04-07T17:51:27.170 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.170 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.170 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.170 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.170 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.170 INFO:tasks.workunit.client.0.smithi100.stdout: "mon": { 2024-04-07T17:51:27.170 INFO:tasks.workunit.client.0.smithi100.stdout: "mon.8dd0baa036f0a7e6d1f3bdce86136fe527836c3d": { 2024-04-07T17:51:27.170 INFO:tasks.workunit.client.0.smithi100.stdout: "bloom_filter": { 2024-04-07T17:51:27.170 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.170 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.170 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.171 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs": { 2024-04-07T17:51:27.171 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.171 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.171 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.171 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs_file_reader": { 2024-04-07T17:51:27.171 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.171 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.171 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.171 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs_file_writer": { 2024-04-07T17:51:27.171 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.171 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.171 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.171 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_alloc": { 2024-04-07T17:51:27.171 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.171 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.172 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.172 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_blob": { 2024-04-07T17:51:27.172 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.172 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.172 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.172 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_buffer": { 2024-04-07T17:51:27.172 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.172 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.172 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.172 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_data": { 2024-04-07T17:51:27.172 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.172 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.172 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.172 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_meta": { 2024-04-07T17:51:27.172 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.172 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.173 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.173 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_onode": { 2024-04-07T17:51:27.173 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.173 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.173 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.173 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_other": { 2024-04-07T17:51:27.173 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.173 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.173 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.173 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_extent": { 2024-04-07T17:51:27.173 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.173 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.173 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.173 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_fsck": { 2024-04-07T17:51:27.173 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.173 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.173 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.174 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_inline_bl": { 2024-04-07T17:51:27.174 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.174 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.174 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.174 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_shared_blob": { 2024-04-07T17:51:27.174 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.174 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.174 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.174 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_txc": { 2024-04-07T17:51:27.174 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.174 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.174 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.174 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_writing": { 2024-04-07T17:51:27.174 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.174 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.175 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.175 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_writing_deferred": { 2024-04-07T17:51:27.175 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.175 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.175 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.175 INFO:tasks.workunit.client.0.smithi100.stdout: "buffer_anon": { 2024-04-07T17:51:27.175 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 40741730, 2024-04-07T17:51:27.175 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 5037 2024-04-07T17:51:27.175 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.175 INFO:tasks.workunit.client.0.smithi100.stdout: "buffer_meta": { 2024-04-07T17:51:27.175 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.175 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.175 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.175 INFO:tasks.workunit.client.0.smithi100.stdout: "mds_co": { 2024-04-07T17:51:27.175 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.176 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.176 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.176 INFO:tasks.workunit.client.0.smithi100.stdout: "osd": { 2024-04-07T17:51:27.176 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.176 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.176 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.176 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mapbl": { 2024-04-07T17:51:27.176 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.176 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.176 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.176 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_pglog": { 2024-04-07T17:51:27.176 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.176 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.176 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.176 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap": { 2024-04-07T17:51:27.177 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 10792, 2024-04-07T17:51:27.177 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 251 2024-04-07T17:51:27.177 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.177 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap_mapping": { 2024-04-07T17:51:27.177 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 70440, 2024-04-07T17:51:27.177 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 5158 2024-04-07T17:51:27.177 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.177 INFO:tasks.workunit.client.0.smithi100.stdout: "pgmap": { 2024-04-07T17:51:27.177 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 5896, 2024-04-07T17:51:27.177 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 174 2024-04-07T17:51:27.177 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.177 INFO:tasks.workunit.client.0.smithi100.stdout: "unittest_1": { 2024-04-07T17:51:27.177 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.177 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.177 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.178 INFO:tasks.workunit.client.0.smithi100.stdout: "unittest_2": { 2024-04-07T17:51:27.178 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.178 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.178 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.178 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.178 INFO:tasks.workunit.client.0.smithi100.stdout: "mon.a52fc0d8acb515431879057649ff294cfe14c620": { 2024-04-07T17:51:27.178 INFO:tasks.workunit.client.0.smithi100.stdout: "bloom_filter": { 2024-04-07T17:51:27.178 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.178 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.178 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.178 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs": { 2024-04-07T17:51:27.178 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.178 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.178 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.178 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs_file_reader": { 2024-04-07T17:51:27.179 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.179 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.179 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.179 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs_file_writer": { 2024-04-07T17:51:27.179 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.179 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.179 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.179 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_alloc": { 2024-04-07T17:51:27.179 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.179 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.179 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.179 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_blob": { 2024-04-07T17:51:27.179 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.179 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.179 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.179 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_buffer": { 2024-04-07T17:51:27.180 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.180 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.180 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.180 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_data": { 2024-04-07T17:51:27.180 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.180 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.180 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.180 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_meta": { 2024-04-07T17:51:27.180 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.180 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.180 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.180 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_onode": { 2024-04-07T17:51:27.180 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.180 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.180 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.180 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_other": { 2024-04-07T17:51:27.181 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.181 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.181 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.181 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_extent": { 2024-04-07T17:51:27.181 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.181 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.181 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.181 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_fsck": { 2024-04-07T17:51:27.181 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.181 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.181 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.181 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_inline_bl": { 2024-04-07T17:51:27.181 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.181 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.181 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.181 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_shared_blob": { 2024-04-07T17:51:27.182 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.182 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.182 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.182 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_txc": { 2024-04-07T17:51:27.182 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.182 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.182 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.182 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_writing": { 2024-04-07T17:51:27.182 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.182 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.182 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.182 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_writing_deferred": { 2024-04-07T17:51:27.182 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.182 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.183 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.183 INFO:tasks.workunit.client.0.smithi100.stdout: "buffer_anon": { 2024-04-07T17:51:27.183 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 40378562, 2024-04-07T17:51:27.183 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 4986 2024-04-07T17:51:27.183 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.183 INFO:tasks.workunit.client.0.smithi100.stdout: "buffer_meta": { 2024-04-07T17:51:27.183 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 440, 2024-04-07T17:51:27.183 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 5 2024-04-07T17:51:27.183 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.183 INFO:tasks.workunit.client.0.smithi100.stdout: "mds_co": { 2024-04-07T17:51:27.183 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.183 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.183 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.183 INFO:tasks.workunit.client.0.smithi100.stdout: "osd": { 2024-04-07T17:51:27.183 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.183 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.184 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.184 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mapbl": { 2024-04-07T17:51:27.184 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.184 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.184 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.184 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_pglog": { 2024-04-07T17:51:27.184 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.184 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.184 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.184 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap": { 2024-04-07T17:51:27.184 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 10792, 2024-04-07T17:51:27.184 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 251 2024-04-07T17:51:27.184 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.184 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap_mapping": { 2024-04-07T17:51:27.184 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 70440, 2024-04-07T17:51:27.185 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 5158 2024-04-07T17:51:27.185 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.185 INFO:tasks.workunit.client.0.smithi100.stdout: "pgmap": { 2024-04-07T17:51:27.185 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 5896, 2024-04-07T17:51:27.185 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 174 2024-04-07T17:51:27.185 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.185 INFO:tasks.workunit.client.0.smithi100.stdout: "unittest_1": { 2024-04-07T17:51:27.185 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.185 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.185 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.185 INFO:tasks.workunit.client.0.smithi100.stdout: "unittest_2": { 2024-04-07T17:51:27.185 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.185 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.185 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.185 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.185 INFO:tasks.workunit.client.0.smithi100.stdout: "mon.f47a4ff15a45950a63160265564a123fc5da5e5e": { 2024-04-07T17:51:27.186 INFO:tasks.workunit.client.0.smithi100.stdout: "bloom_filter": { 2024-04-07T17:51:27.186 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.186 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.186 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.186 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs": { 2024-04-07T17:51:27.186 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.186 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.186 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.186 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs_file_reader": { 2024-04-07T17:51:27.186 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.186 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.186 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.186 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs_file_writer": { 2024-04-07T17:51:27.186 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.187 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.187 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.187 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_alloc": { 2024-04-07T17:51:27.187 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.187 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.187 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.187 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_blob": { 2024-04-07T17:51:27.187 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.187 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.187 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.187 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_buffer": { 2024-04-07T17:51:27.187 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.187 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.187 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.187 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_data": { 2024-04-07T17:51:27.188 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.188 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.188 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.188 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_meta": { 2024-04-07T17:51:27.188 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.188 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.188 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.188 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_onode": { 2024-04-07T17:51:27.188 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.188 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.188 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.188 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_other": { 2024-04-07T17:51:27.188 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.188 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.188 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.189 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_extent": { 2024-04-07T17:51:27.189 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.189 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.189 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.189 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_fsck": { 2024-04-07T17:51:27.189 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.189 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.189 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.189 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_inline_bl": { 2024-04-07T17:51:27.189 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.189 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.189 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.189 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_shared_blob": { 2024-04-07T17:51:27.189 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.189 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.189 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.190 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_txc": { 2024-04-07T17:51:27.190 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.190 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.190 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.190 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_writing": { 2024-04-07T17:51:27.190 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.190 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.190 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.190 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_writing_deferred": { 2024-04-07T17:51:27.190 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.190 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.190 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.190 INFO:tasks.workunit.client.0.smithi100.stdout: "buffer_anon": { 2024-04-07T17:51:27.190 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 41174442, 2024-04-07T17:51:27.190 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 5101 2024-04-07T17:51:27.190 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.191 INFO:tasks.workunit.client.0.smithi100.stdout: "buffer_meta": { 2024-04-07T17:51:27.191 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.191 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.191 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.191 INFO:tasks.workunit.client.0.smithi100.stdout: "mds_co": { 2024-04-07T17:51:27.191 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.191 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.191 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.191 INFO:tasks.workunit.client.0.smithi100.stdout: "osd": { 2024-04-07T17:51:27.191 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.191 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.191 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.191 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mapbl": { 2024-04-07T17:51:27.191 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.191 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.192 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.192 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_pglog": { 2024-04-07T17:51:27.192 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.192 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.192 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.192 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap": { 2024-04-07T17:51:27.192 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 10792, 2024-04-07T17:51:27.192 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 251 2024-04-07T17:51:27.192 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.192 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap_mapping": { 2024-04-07T17:51:27.192 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 70440, 2024-04-07T17:51:27.192 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 5158 2024-04-07T17:51:27.192 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.192 INFO:tasks.workunit.client.0.smithi100.stdout: "pgmap": { 2024-04-07T17:51:27.192 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 11792, 2024-04-07T17:51:27.192 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 348 2024-04-07T17:51:27.193 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.193 INFO:tasks.workunit.client.0.smithi100.stdout: "unittest_1": { 2024-04-07T17:51:27.193 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.193 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.193 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.193 INFO:tasks.workunit.client.0.smithi100.stdout: "unittest_2": { 2024-04-07T17:51:27.193 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.193 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.193 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.193 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.193 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.193 INFO:tasks.workunit.client.0.smithi100.stdout: "osd": { 2024-04-07T17:51:27.193 INFO:tasks.workunit.client.0.smithi100.stdout: "osd.0": { 2024-04-07T17:51:27.193 INFO:tasks.workunit.client.0.smithi100.stdout: "bloom_filter": { 2024-04-07T17:51:27.193 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.193 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.194 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.194 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs": { 2024-04-07T17:51:27.194 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 22768, 2024-04-07T17:51:27.194 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 424 2024-04-07T17:51:27.194 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.194 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs_file_reader": { 2024-04-07T17:51:27.194 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 31360, 2024-04-07T17:51:27.194 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 28 2024-04-07T17:51:27.194 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.194 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs_file_writer": { 2024-04-07T17:51:27.194 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 2496, 2024-04-07T17:51:27.194 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 13 2024-04-07T17:51:27.194 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.194 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_alloc": { 2024-04-07T17:51:27.194 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 59840, 2024-04-07T17:51:27.195 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 748 2024-04-07T17:51:27.195 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.195 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_blob": { 2024-04-07T17:51:27.195 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 382512, 2024-04-07T17:51:27.195 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 1839 2024-04-07T17:51:27.195 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.195 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_buffer": { 2024-04-07T17:51:27.195 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 256, 2024-04-07T17:51:27.195 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 32 2024-04-07T17:51:27.195 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.195 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_data": { 2024-04-07T17:51:27.195 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 774695, 2024-04-07T17:51:27.195 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 215 2024-04-07T17:51:27.195 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.195 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_meta": { 2024-04-07T17:51:27.196 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 2426291, 2024-04-07T17:51:27.196 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 537859 2024-04-07T17:51:27.196 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.196 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_onode": { 2024-04-07T17:51:27.196 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 6943736, 2024-04-07T17:51:27.196 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 10243 2024-04-07T17:51:27.196 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.196 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_other": { 2024-04-07T17:51:27.196 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 72960, 2024-04-07T17:51:27.196 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 7616 2024-04-07T17:51:27.196 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.196 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_extent": { 2024-04-07T17:51:27.196 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 88272, 2024-04-07T17:51:27.196 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 1839 2024-04-07T17:51:27.196 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.196 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_fsck": { 2024-04-07T17:51:27.196 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.197 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.197 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.197 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_inline_bl": { 2024-04-07T17:51:27.197 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 175624, 2024-04-07T17:51:27.197 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 1258 2024-04-07T17:51:27.197 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.197 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_shared_blob": { 2024-04-07T17:51:27.197 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.197 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.197 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.197 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_txc": { 2024-04-07T17:51:27.197 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.197 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.197 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.197 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_writing": { 2024-04-07T17:51:27.198 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 21009, 2024-04-07T17:51:27.198 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 33 2024-04-07T17:51:27.198 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.198 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_writing_deferred": { 2024-04-07T17:51:27.198 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.198 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.198 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.198 INFO:tasks.workunit.client.0.smithi100.stdout: "buffer_anon": { 2024-04-07T17:51:27.198 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 15044844, 2024-04-07T17:51:27.198 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 291 2024-04-07T17:51:27.198 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.198 INFO:tasks.workunit.client.0.smithi100.stdout: "buffer_meta": { 2024-04-07T17:51:27.198 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 19448, 2024-04-07T17:51:27.199 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 221 2024-04-07T17:51:27.199 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.199 INFO:tasks.workunit.client.0.smithi100.stdout: "mds_co": { 2024-04-07T17:51:27.199 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.199 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.199 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.199 INFO:tasks.workunit.client.0.smithi100.stdout: "osd": { 2024-04-07T17:51:27.199 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 513216, 2024-04-07T17:51:27.199 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 44 2024-04-07T17:51:27.199 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.199 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mapbl": { 2024-04-07T17:51:27.199 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.201 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.201 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.201 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_pglog": { 2024-04-07T17:51:27.201 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 7167680, 2024-04-07T17:51:27.201 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 18584 2024-04-07T17:51:27.201 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.201 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap": { 2024-04-07T17:51:27.201 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 635376, 2024-04-07T17:51:27.201 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 11076 2024-04-07T17:51:27.201 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.202 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap_mapping": { 2024-04-07T17:51:27.202 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.202 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.202 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.202 INFO:tasks.workunit.client.0.smithi100.stdout: "pgmap": { 2024-04-07T17:51:27.202 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.202 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.202 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.202 INFO:tasks.workunit.client.0.smithi100.stdout: "unittest_1": { 2024-04-07T17:51:27.202 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.202 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.202 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.202 INFO:tasks.workunit.client.0.smithi100.stdout: "unittest_2": { 2024-04-07T17:51:27.202 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.202 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.203 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.203 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.203 INFO:tasks.workunit.client.0.smithi100.stdout: "osd.1": { 2024-04-07T17:51:27.203 INFO:tasks.workunit.client.0.smithi100.stdout: "bloom_filter": { 2024-04-07T17:51:27.203 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.203 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.203 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.203 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs": { 2024-04-07T17:51:27.203 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 20720, 2024-04-07T17:51:27.203 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 385 2024-04-07T17:51:27.203 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.203 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs_file_reader": { 2024-04-07T17:51:27.203 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 35072, 2024-04-07T17:51:27.203 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 26 2024-04-07T17:51:27.203 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.203 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs_file_writer": { 2024-04-07T17:51:27.204 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 4800, 2024-04-07T17:51:27.204 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 25 2024-04-07T17:51:27.204 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.204 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_alloc": { 2024-04-07T17:51:27.204 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 58160, 2024-04-07T17:51:27.204 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 727 2024-04-07T17:51:27.204 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.204 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_blob": { 2024-04-07T17:51:27.204 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 380224, 2024-04-07T17:51:27.204 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 1828 2024-04-07T17:51:27.204 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.204 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_buffer": { 2024-04-07T17:51:27.204 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 256, 2024-04-07T17:51:27.204 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 32 2024-04-07T17:51:27.204 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.204 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_data": { 2024-04-07T17:51:27.205 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 1045046, 2024-04-07T17:51:27.205 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 285 2024-04-07T17:51:27.205 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.205 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_meta": { 2024-04-07T17:51:27.205 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 2355008, 2024-04-07T17:51:27.205 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 521245 2024-04-07T17:51:27.205 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.205 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_onode": { 2024-04-07T17:51:27.205 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 6739736, 2024-04-07T17:51:27.205 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 9943 2024-04-07T17:51:27.205 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.205 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_other": { 2024-04-07T17:51:27.205 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 70888, 2024-04-07T17:51:27.205 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 7440 2024-04-07T17:51:27.205 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.205 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_extent": { 2024-04-07T17:51:27.206 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 87744, 2024-04-07T17:51:27.206 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 1828 2024-04-07T17:51:27.206 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.206 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_fsck": { 2024-04-07T17:51:27.206 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.206 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.206 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.206 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_inline_bl": { 2024-04-07T17:51:27.206 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 175712, 2024-04-07T17:51:27.206 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 1259 2024-04-07T17:51:27.206 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.206 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_shared_blob": { 2024-04-07T17:51:27.206 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.206 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.206 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.207 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_txc": { 2024-04-07T17:51:27.207 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.207 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.207 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.207 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_writing": { 2024-04-07T17:51:27.207 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 34652, 2024-04-07T17:51:27.207 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 29 2024-04-07T17:51:27.207 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.207 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_writing_deferred": { 2024-04-07T17:51:27.207 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.207 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.207 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.207 INFO:tasks.workunit.client.0.smithi100.stdout: "buffer_anon": { 2024-04-07T17:51:27.208 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 26499475, 2024-04-07T17:51:27.208 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 286 2024-04-07T17:51:27.208 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.208 INFO:tasks.workunit.client.0.smithi100.stdout: "buffer_meta": { 2024-04-07T17:51:27.208 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 26928, 2024-04-07T17:51:27.208 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 306 2024-04-07T17:51:27.208 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.208 INFO:tasks.workunit.client.0.smithi100.stdout: "mds_co": { 2024-04-07T17:51:27.208 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.208 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.208 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.208 INFO:tasks.workunit.client.0.smithi100.stdout: "osd": { 2024-04-07T17:51:27.208 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 443232, 2024-04-07T17:51:27.208 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 38 2024-04-07T17:51:27.208 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.209 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mapbl": { 2024-04-07T17:51:27.209 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.209 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.209 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.209 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_pglog": { 2024-04-07T17:51:27.209 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 7182344, 2024-04-07T17:51:27.209 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 20929 2024-04-07T17:51:27.209 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.209 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap": { 2024-04-07T17:51:27.209 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 635376, 2024-04-07T17:51:27.209 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 11076 2024-04-07T17:51:27.209 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.209 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap_mapping": { 2024-04-07T17:51:27.209 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.209 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.210 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.210 INFO:tasks.workunit.client.0.smithi100.stdout: "pgmap": { 2024-04-07T17:51:27.210 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.210 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.210 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.210 INFO:tasks.workunit.client.0.smithi100.stdout: "unittest_1": { 2024-04-07T17:51:27.210 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.210 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.210 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.210 INFO:tasks.workunit.client.0.smithi100.stdout: "unittest_2": { 2024-04-07T17:51:27.210 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.210 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.210 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.210 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.210 INFO:tasks.workunit.client.0.smithi100.stdout: "osd.2": { 2024-04-07T17:51:27.211 INFO:tasks.workunit.client.0.smithi100.stdout: "bloom_filter": { 2024-04-07T17:51:27.211 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.211 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.211 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.211 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs": { 2024-04-07T17:51:27.211 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 19496, 2024-04-07T17:51:27.211 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 362 2024-04-07T17:51:27.211 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.211 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs_file_reader": { 2024-04-07T17:51:27.211 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 31360, 2024-04-07T17:51:27.211 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 28 2024-04-07T17:51:27.211 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.211 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs_file_writer": { 2024-04-07T17:51:27.211 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 3648, 2024-04-07T17:51:27.211 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 19 2024-04-07T17:51:27.211 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.212 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_alloc": { 2024-04-07T17:51:27.212 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 58080, 2024-04-07T17:51:27.212 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 726 2024-04-07T17:51:27.212 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.212 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_blob": { 2024-04-07T17:51:27.212 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 379600, 2024-04-07T17:51:27.212 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 1825 2024-04-07T17:51:27.212 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.212 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_buffer": { 2024-04-07T17:51:27.212 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 256, 2024-04-07T17:51:27.212 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 32 2024-04-07T17:51:27.212 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.212 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_data": { 2024-04-07T17:51:27.212 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 434802, 2024-04-07T17:51:27.212 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 100 2024-04-07T17:51:27.212 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.213 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_meta": { 2024-04-07T17:51:27.213 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 1711873, 2024-04-07T17:51:27.213 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 379494 2024-04-07T17:51:27.213 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.213 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_onode": { 2024-04-07T17:51:27.213 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 5016616, 2024-04-07T17:51:27.213 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 7409 2024-04-07T17:51:27.213 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.213 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_other": { 2024-04-07T17:51:27.213 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 69948, 2024-04-07T17:51:27.213 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 7427 2024-04-07T17:51:27.213 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.213 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_extent": { 2024-04-07T17:51:27.213 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 87600, 2024-04-07T17:51:27.213 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 1825 2024-04-07T17:51:27.214 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.214 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_fsck": { 2024-04-07T17:51:27.214 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.214 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.214 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.214 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_inline_bl": { 2024-04-07T17:51:27.214 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 175624, 2024-04-07T17:51:27.214 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 1258 2024-04-07T17:51:27.214 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.214 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_shared_blob": { 2024-04-07T17:51:27.214 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.214 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.214 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.215 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_txc": { 2024-04-07T17:51:27.215 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 736, 2024-04-07T17:51:27.215 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 1 2024-04-07T17:51:27.215 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.215 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_writing": { 2024-04-07T17:51:27.215 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 69418, 2024-04-07T17:51:27.215 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 53 2024-04-07T17:51:27.215 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.215 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_writing_deferred": { 2024-04-07T17:51:27.215 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 12112, 2024-04-07T17:51:27.215 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 3 2024-04-07T17:51:27.215 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.215 INFO:tasks.workunit.client.0.smithi100.stdout: "buffer_anon": { 2024-04-07T17:51:27.215 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 20645063, 2024-04-07T17:51:27.215 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 260 2024-04-07T17:51:27.215 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.216 INFO:tasks.workunit.client.0.smithi100.stdout: "buffer_meta": { 2024-04-07T17:51:27.216 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 12760, 2024-04-07T17:51:27.216 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 145 2024-04-07T17:51:27.216 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.216 INFO:tasks.workunit.client.0.smithi100.stdout: "mds_co": { 2024-04-07T17:51:27.216 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.216 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.216 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.216 INFO:tasks.workunit.client.0.smithi100.stdout: "osd": { 2024-04-07T17:51:27.216 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 349920, 2024-04-07T17:51:27.216 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 30 2024-04-07T17:51:27.216 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.216 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mapbl": { 2024-04-07T17:51:27.216 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.216 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.217 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.217 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_pglog": { 2024-04-07T17:51:27.217 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 3170440, 2024-04-07T17:51:27.217 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 6097 2024-04-07T17:51:27.217 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.217 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap": { 2024-04-07T17:51:27.217 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 635376, 2024-04-07T17:51:27.217 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 11076 2024-04-07T17:51:27.217 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.217 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap_mapping": { 2024-04-07T17:51:27.217 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.217 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.217 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.217 INFO:tasks.workunit.client.0.smithi100.stdout: "pgmap": { 2024-04-07T17:51:27.217 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.217 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.218 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.218 INFO:tasks.workunit.client.0.smithi100.stdout: "unittest_1": { 2024-04-07T17:51:27.218 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.218 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.218 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.218 INFO:tasks.workunit.client.0.smithi100.stdout: "unittest_2": { 2024-04-07T17:51:27.218 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.218 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.218 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.218 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.218 INFO:tasks.workunit.client.0.smithi100.stdout: "osd.3": { 2024-04-07T17:51:27.218 INFO:tasks.workunit.client.0.smithi100.stdout: "bloom_filter": { 2024-04-07T17:51:27.218 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.218 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.218 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.219 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs": { 2024-04-07T17:51:27.219 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 17048, 2024-04-07T17:51:27.219 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 316 2024-04-07T17:51:27.219 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.219 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs_file_reader": { 2024-04-07T17:51:27.219 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 30976, 2024-04-07T17:51:27.219 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 25 2024-04-07T17:51:27.219 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.219 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs_file_writer": { 2024-04-07T17:51:27.219 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 3072, 2024-04-07T17:51:27.219 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 16 2024-04-07T17:51:27.219 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.219 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_alloc": { 2024-04-07T17:51:27.219 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 56960, 2024-04-07T17:51:27.219 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 712 2024-04-07T17:51:27.220 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.220 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_blob": { 2024-04-07T17:51:27.220 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 380640, 2024-04-07T17:51:27.220 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 1830 2024-04-07T17:51:27.220 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.220 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_buffer": { 2024-04-07T17:51:27.220 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 256, 2024-04-07T17:51:27.220 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 32 2024-04-07T17:51:27.220 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.220 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_data": { 2024-04-07T17:51:27.220 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 381771, 2024-04-07T17:51:27.220 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 104 2024-04-07T17:51:27.220 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.220 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_meta": { 2024-04-07T17:51:27.220 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 1733357, 2024-04-07T17:51:27.220 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 381544 2024-04-07T17:51:27.221 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.221 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_onode": { 2024-04-07T17:51:27.221 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 5029536, 2024-04-07T17:51:27.221 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 7428 2024-04-07T17:51:27.221 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.221 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_other": { 2024-04-07T17:51:27.221 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 70748, 2024-04-07T17:51:27.221 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 7446 2024-04-07T17:51:27.221 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.221 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_extent": { 2024-04-07T17:51:27.221 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 87840, 2024-04-07T17:51:27.221 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 1830 2024-04-07T17:51:27.221 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.221 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_fsck": { 2024-04-07T17:51:27.221 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.221 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.222 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.222 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_inline_bl": { 2024-04-07T17:51:27.222 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 175624, 2024-04-07T17:51:27.222 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 1258 2024-04-07T17:51:27.222 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.222 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_shared_blob": { 2024-04-07T17:51:27.222 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.222 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.222 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.222 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_txc": { 2024-04-07T17:51:27.222 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 736, 2024-04-07T17:51:27.222 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 1 2024-04-07T17:51:27.222 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.222 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_writing": { 2024-04-07T17:51:27.222 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 83741, 2024-04-07T17:51:27.222 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 55 2024-04-07T17:51:27.223 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.223 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_writing_deferred": { 2024-04-07T17:51:27.223 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 12112, 2024-04-07T17:51:27.223 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 3 2024-04-07T17:51:27.223 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.223 INFO:tasks.workunit.client.0.smithi100.stdout: "buffer_anon": { 2024-04-07T17:51:27.223 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 18443066, 2024-04-07T17:51:27.223 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 280 2024-04-07T17:51:27.223 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.223 INFO:tasks.workunit.client.0.smithi100.stdout: "buffer_meta": { 2024-04-07T17:51:27.223 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 11880, 2024-04-07T17:51:27.223 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 135 2024-04-07T17:51:27.223 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.223 INFO:tasks.workunit.client.0.smithi100.stdout: "mds_co": { 2024-04-07T17:51:27.223 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.223 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.224 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.224 INFO:tasks.workunit.client.0.smithi100.stdout: "osd": { 2024-04-07T17:51:27.224 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 419904, 2024-04-07T17:51:27.224 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 36 2024-04-07T17:51:27.224 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.224 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mapbl": { 2024-04-07T17:51:27.224 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.224 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.224 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.224 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_pglog": { 2024-04-07T17:51:27.224 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 3485560, 2024-04-07T17:51:27.224 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 9103 2024-04-07T17:51:27.224 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.224 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap": { 2024-04-07T17:51:27.224 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 635376, 2024-04-07T17:51:27.224 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 11076 2024-04-07T17:51:27.225 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.225 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap_mapping": { 2024-04-07T17:51:27.225 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.225 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.225 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.225 INFO:tasks.workunit.client.0.smithi100.stdout: "pgmap": { 2024-04-07T17:51:27.225 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.225 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.225 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.225 INFO:tasks.workunit.client.0.smithi100.stdout: "unittest_1": { 2024-04-07T17:51:27.225 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.225 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.225 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.225 INFO:tasks.workunit.client.0.smithi100.stdout: "unittest_2": { 2024-04-07T17:51:27.225 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.225 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.226 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.226 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.226 INFO:tasks.workunit.client.0.smithi100.stdout: "osd.4": { 2024-04-07T17:51:27.226 INFO:tasks.workunit.client.0.smithi100.stdout: "bloom_filter": { 2024-04-07T17:51:27.226 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.226 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.226 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.226 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs": { 2024-04-07T17:51:27.226 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 20816, 2024-04-07T17:51:27.226 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 393 2024-04-07T17:51:27.226 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.226 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs_file_reader": { 2024-04-07T17:51:27.226 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 31488, 2024-04-07T17:51:27.226 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 29 2024-04-07T17:51:27.226 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.227 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs_file_writer": { 2024-04-07T17:51:27.227 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 4032, 2024-04-07T17:51:27.227 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 21 2024-04-07T17:51:27.227 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.227 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_alloc": { 2024-04-07T17:51:27.227 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 58240, 2024-04-07T17:51:27.227 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 728 2024-04-07T17:51:27.227 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.227 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_blob": { 2024-04-07T17:51:27.227 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 393952, 2024-04-07T17:51:27.227 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 1894 2024-04-07T17:51:27.227 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.227 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_buffer": { 2024-04-07T17:51:27.227 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 256, 2024-04-07T17:51:27.227 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 32 2024-04-07T17:51:27.227 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.228 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_data": { 2024-04-07T17:51:27.228 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 1050407, 2024-04-07T17:51:27.228 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 273 2024-04-07T17:51:27.228 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.228 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_meta": { 2024-04-07T17:51:27.228 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 2027647, 2024-04-07T17:51:27.228 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 446306 2024-04-07T17:51:27.228 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.228 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_onode": { 2024-04-07T17:51:27.228 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 5812216, 2024-04-07T17:51:27.228 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 8579 2024-04-07T17:51:27.228 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.228 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_other": { 2024-04-07T17:51:27.228 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 80384, 2024-04-07T17:51:27.228 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 8599 2024-04-07T17:51:27.228 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.229 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_extent": { 2024-04-07T17:51:27.229 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 90912, 2024-04-07T17:51:27.229 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 1894 2024-04-07T17:51:27.229 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.229 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_fsck": { 2024-04-07T17:51:27.229 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.229 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.229 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.229 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_inline_bl": { 2024-04-07T17:51:27.229 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 175456, 2024-04-07T17:51:27.229 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 1258 2024-04-07T17:51:27.229 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.229 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_shared_blob": { 2024-04-07T17:51:27.229 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.229 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.229 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.230 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_txc": { 2024-04-07T17:51:27.230 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 736, 2024-04-07T17:51:27.230 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 1 2024-04-07T17:51:27.230 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.230 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_writing": { 2024-04-07T17:51:27.230 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 66705, 2024-04-07T17:51:27.230 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 51 2024-04-07T17:51:27.230 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.230 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_writing_deferred": { 2024-04-07T17:51:27.230 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 12112, 2024-04-07T17:51:27.230 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 3 2024-04-07T17:51:27.230 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.230 INFO:tasks.workunit.client.0.smithi100.stdout: "buffer_anon": { 2024-04-07T17:51:27.230 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 23146638, 2024-04-07T17:51:27.230 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 273 2024-04-07T17:51:27.231 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.231 INFO:tasks.workunit.client.0.smithi100.stdout: "buffer_meta": { 2024-04-07T17:51:27.231 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 26928, 2024-04-07T17:51:27.231 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 306 2024-04-07T17:51:27.231 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.231 INFO:tasks.workunit.client.0.smithi100.stdout: "mds_co": { 2024-04-07T17:51:27.231 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.231 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.231 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.231 INFO:tasks.workunit.client.0.smithi100.stdout: "osd": { 2024-04-07T17:51:27.231 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 466560, 2024-04-07T17:51:27.231 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 40 2024-04-07T17:51:27.231 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.231 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mapbl": { 2024-04-07T17:51:27.231 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.231 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.232 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.232 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_pglog": { 2024-04-07T17:51:27.232 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 5065320, 2024-04-07T17:51:27.232 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 12141 2024-04-07T17:51:27.232 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.232 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap": { 2024-04-07T17:51:27.232 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 635376, 2024-04-07T17:51:27.232 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 11076 2024-04-07T17:51:27.232 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.232 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap_mapping": { 2024-04-07T17:51:27.232 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.232 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.232 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.232 INFO:tasks.workunit.client.0.smithi100.stdout: "pgmap": { 2024-04-07T17:51:27.232 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.232 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.233 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.233 INFO:tasks.workunit.client.0.smithi100.stdout: "unittest_1": { 2024-04-07T17:51:27.233 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.233 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.233 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.233 INFO:tasks.workunit.client.0.smithi100.stdout: "unittest_2": { 2024-04-07T17:51:27.233 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.233 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.233 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.233 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.233 INFO:tasks.workunit.client.0.smithi100.stdout: "osd.5": { 2024-04-07T17:51:27.233 INFO:tasks.workunit.client.0.smithi100.stdout: "bloom_filter": { 2024-04-07T17:51:27.233 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.233 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.233 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.234 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs": { 2024-04-07T17:51:27.234 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 19216, 2024-04-07T17:51:27.234 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 365 2024-04-07T17:51:27.234 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.234 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs_file_reader": { 2024-04-07T17:51:27.234 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 31616, 2024-04-07T17:51:27.234 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 30 2024-04-07T17:51:27.234 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.234 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs_file_writer": { 2024-04-07T17:51:27.234 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 3072, 2024-04-07T17:51:27.234 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 16 2024-04-07T17:51:27.234 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.234 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_alloc": { 2024-04-07T17:51:27.234 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 58400, 2024-04-07T17:51:27.234 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 730 2024-04-07T17:51:27.234 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.235 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_blob": { 2024-04-07T17:51:27.235 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 393952, 2024-04-07T17:51:27.235 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 1894 2024-04-07T17:51:27.235 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.235 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_buffer": { 2024-04-07T17:51:27.235 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 256, 2024-04-07T17:51:27.235 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 32 2024-04-07T17:51:27.235 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.235 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_data": { 2024-04-07T17:51:27.235 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 274397, 2024-04-07T17:51:27.235 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 87 2024-04-07T17:51:27.235 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.235 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_meta": { 2024-04-07T17:51:27.235 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 2323821, 2024-04-07T17:51:27.235 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 518276 2024-04-07T17:51:27.236 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.236 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_onode": { 2024-04-07T17:51:27.236 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 6677176, 2024-04-07T17:51:27.236 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 9851 2024-04-07T17:51:27.236 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.236 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_other": { 2024-04-07T17:51:27.236 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 80636, 2024-04-07T17:51:27.236 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 8604 2024-04-07T17:51:27.236 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.236 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_extent": { 2024-04-07T17:51:27.236 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 90912, 2024-04-07T17:51:27.236 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 1894 2024-04-07T17:51:27.236 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.236 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_fsck": { 2024-04-07T17:51:27.237 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.237 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.237 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.237 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_inline_bl": { 2024-04-07T17:51:27.237 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 175712, 2024-04-07T17:51:27.237 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 1259 2024-04-07T17:51:27.237 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.237 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_shared_blob": { 2024-04-07T17:51:27.237 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.237 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.237 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.237 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_txc": { 2024-04-07T17:51:27.237 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 736, 2024-04-07T17:51:27.237 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 1 2024-04-07T17:51:27.237 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.237 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_writing": { 2024-04-07T17:51:27.238 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 87571, 2024-04-07T17:51:27.238 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 56 2024-04-07T17:51:27.238 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.238 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_writing_deferred": { 2024-04-07T17:51:27.238 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 12112, 2024-04-07T17:51:27.238 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 3 2024-04-07T17:51:27.238 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.238 INFO:tasks.workunit.client.0.smithi100.stdout: "buffer_anon": { 2024-04-07T17:51:27.238 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 18319894, 2024-04-07T17:51:27.238 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 12576 2024-04-07T17:51:27.238 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.238 INFO:tasks.workunit.client.0.smithi100.stdout: "buffer_meta": { 2024-04-07T17:51:27.238 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 10208, 2024-04-07T17:51:27.238 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 116 2024-04-07T17:51:27.238 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.239 INFO:tasks.workunit.client.0.smithi100.stdout: "mds_co": { 2024-04-07T17:51:27.239 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.239 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.239 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.239 INFO:tasks.workunit.client.0.smithi100.stdout: "osd": { 2024-04-07T17:51:27.239 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 489888, 2024-04-07T17:51:27.239 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 42 2024-04-07T17:51:27.239 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.239 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mapbl": { 2024-04-07T17:51:27.239 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.239 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.239 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.239 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_pglog": { 2024-04-07T17:51:27.239 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 6315920, 2024-04-07T17:51:27.239 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 12146 2024-04-07T17:51:27.240 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.240 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap": { 2024-04-07T17:51:27.240 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 635376, 2024-04-07T17:51:27.240 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 11076 2024-04-07T17:51:27.240 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.240 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap_mapping": { 2024-04-07T17:51:27.240 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.240 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.240 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.240 INFO:tasks.workunit.client.0.smithi100.stdout: "pgmap": { 2024-04-07T17:51:27.240 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.240 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.240 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.240 INFO:tasks.workunit.client.0.smithi100.stdout: "unittest_1": { 2024-04-07T17:51:27.240 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.240 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.241 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.241 INFO:tasks.workunit.client.0.smithi100.stdout: "unittest_2": { 2024-04-07T17:51:27.241 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.241 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.241 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.241 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.241 INFO:tasks.workunit.client.0.smithi100.stdout: "osd.6": { 2024-04-07T17:51:27.241 INFO:tasks.workunit.client.0.smithi100.stdout: "bloom_filter": { 2024-04-07T17:51:27.241 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.241 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.241 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.241 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs": { 2024-04-07T17:51:27.241 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 13656, 2024-04-07T17:51:27.241 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 230 2024-04-07T17:51:27.241 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.241 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs_file_reader": { 2024-04-07T17:51:27.242 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 30336, 2024-04-07T17:51:27.242 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 20 2024-04-07T17:51:27.242 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.242 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs_file_writer": { 2024-04-07T17:51:27.242 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 2496, 2024-04-07T17:51:27.242 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 13 2024-04-07T17:51:27.242 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.242 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_alloc": { 2024-04-07T17:51:27.242 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 56960, 2024-04-07T17:51:27.242 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 712 2024-04-07T17:51:27.242 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.242 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_blob": { 2024-04-07T17:51:27.242 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 395616, 2024-04-07T17:51:27.242 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 1902 2024-04-07T17:51:27.242 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.242 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_buffer": { 2024-04-07T17:51:27.243 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 256, 2024-04-07T17:51:27.243 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 32 2024-04-07T17:51:27.243 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.243 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_data": { 2024-04-07T17:51:27.243 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 463979, 2024-04-07T17:51:27.243 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 128 2024-04-07T17:51:27.243 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.243 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_meta": { 2024-04-07T17:51:27.243 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 1718221, 2024-04-07T17:51:27.243 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 377358 2024-04-07T17:51:27.243 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.243 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_onode": { 2024-04-07T17:51:27.243 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 4954056, 2024-04-07T17:51:27.243 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 7317 2024-04-07T17:51:27.243 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.243 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_other": { 2024-04-07T17:51:27.243 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 81560, 2024-04-07T17:51:27.244 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 8760 2024-04-07T17:51:27.244 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.244 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_extent": { 2024-04-07T17:51:27.244 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 91296, 2024-04-07T17:51:27.244 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 1902 2024-04-07T17:51:27.244 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.244 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_fsck": { 2024-04-07T17:51:27.244 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.244 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.244 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.244 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_inline_bl": { 2024-04-07T17:51:27.244 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 175368, 2024-04-07T17:51:27.244 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 1257 2024-04-07T17:51:27.244 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.244 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_shared_blob": { 2024-04-07T17:51:27.244 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.245 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.245 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.245 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_txc": { 2024-04-07T17:51:27.245 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.245 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.245 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.245 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_writing": { 2024-04-07T17:51:27.245 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 69389, 2024-04-07T17:51:27.245 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 49 2024-04-07T17:51:27.245 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.245 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_writing_deferred": { 2024-04-07T17:51:27.245 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.245 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.245 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.245 INFO:tasks.workunit.client.0.smithi100.stdout: "buffer_anon": { 2024-04-07T17:51:27.246 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 13267228, 2024-04-07T17:51:27.246 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 243 2024-04-07T17:51:27.246 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.246 INFO:tasks.workunit.client.0.smithi100.stdout: "buffer_meta": { 2024-04-07T17:51:27.246 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 12320, 2024-04-07T17:51:27.246 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 140 2024-04-07T17:51:27.246 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.246 INFO:tasks.workunit.client.0.smithi100.stdout: "mds_co": { 2024-04-07T17:51:27.246 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.246 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.246 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.246 INFO:tasks.workunit.client.0.smithi100.stdout: "osd": { 2024-04-07T17:51:27.246 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 454896, 2024-04-07T17:51:27.246 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 39 2024-04-07T17:51:27.246 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.247 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mapbl": { 2024-04-07T17:51:27.247 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.247 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.247 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.247 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_pglog": { 2024-04-07T17:51:27.247 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 3669120, 2024-04-07T17:51:27.247 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 9456 2024-04-07T17:51:27.247 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.247 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap": { 2024-04-07T17:51:27.247 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 635376, 2024-04-07T17:51:27.247 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 11076 2024-04-07T17:51:27.247 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.247 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap_mapping": { 2024-04-07T17:51:27.247 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.247 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.247 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.248 INFO:tasks.workunit.client.0.smithi100.stdout: "pgmap": { 2024-04-07T17:51:27.248 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.248 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.248 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.248 INFO:tasks.workunit.client.0.smithi100.stdout: "unittest_1": { 2024-04-07T17:51:27.248 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.248 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.248 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.248 INFO:tasks.workunit.client.0.smithi100.stdout: "unittest_2": { 2024-04-07T17:51:27.248 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.248 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.248 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.248 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.248 INFO:tasks.workunit.client.0.smithi100.stdout: "osd.7": { 2024-04-07T17:51:27.248 INFO:tasks.workunit.client.0.smithi100.stdout: "bloom_filter": { 2024-04-07T17:51:27.248 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.249 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.249 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.249 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs": { 2024-04-07T17:51:27.249 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 15992, 2024-04-07T17:51:27.249 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 307 2024-04-07T17:51:27.249 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.249 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs_file_reader": { 2024-04-07T17:51:27.249 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 39040, 2024-04-07T17:51:27.249 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 25 2024-04-07T17:51:27.249 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.249 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs_file_writer": { 2024-04-07T17:51:27.249 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 2688, 2024-04-07T17:51:27.249 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 14 2024-04-07T17:51:27.249 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.249 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_alloc": { 2024-04-07T17:51:27.249 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 57280, 2024-04-07T17:51:27.250 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 716 2024-04-07T17:51:27.250 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.250 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_blob": { 2024-04-07T17:51:27.250 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 382096, 2024-04-07T17:51:27.250 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 1837 2024-04-07T17:51:27.250 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.250 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_buffer": { 2024-04-07T17:51:27.250 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 256, 2024-04-07T17:51:27.250 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 32 2024-04-07T17:51:27.250 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.250 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_data": { 2024-04-07T17:51:27.250 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 427947, 2024-04-07T17:51:27.250 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 127 2024-04-07T17:51:27.250 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.251 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_meta": { 2024-04-07T17:51:27.251 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 1987758, 2024-04-07T17:51:27.251 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 443621 2024-04-07T17:51:27.251 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.251 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_onode": { 2024-04-07T17:51:27.251 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 5763936, 2024-04-07T17:51:27.251 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 8508 2024-04-07T17:51:27.251 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.251 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_other": { 2024-04-07T17:51:27.251 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 73080, 2024-04-07T17:51:27.251 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 7610 2024-04-07T17:51:27.251 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.251 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_extent": { 2024-04-07T17:51:27.251 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 88176, 2024-04-07T17:51:27.252 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 1837 2024-04-07T17:51:27.252 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.252 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_fsck": { 2024-04-07T17:51:27.252 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.252 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.252 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.252 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_inline_bl": { 2024-04-07T17:51:27.252 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 175624, 2024-04-07T17:51:27.252 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 1258 2024-04-07T17:51:27.252 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.252 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_shared_blob": { 2024-04-07T17:51:27.252 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.252 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.252 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.252 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_txc": { 2024-04-07T17:51:27.253 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.253 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.253 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.253 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_writing": { 2024-04-07T17:51:27.253 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 21282, 2024-04-07T17:51:27.253 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 25 2024-04-07T17:51:27.253 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.253 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_writing_deferred": { 2024-04-07T17:51:27.253 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.253 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.253 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.253 INFO:tasks.workunit.client.0.smithi100.stdout: "buffer_anon": { 2024-04-07T17:51:27.253 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 14934597, 2024-04-07T17:51:27.253 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 264 2024-04-07T17:51:27.253 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.253 INFO:tasks.workunit.client.0.smithi100.stdout: "buffer_meta": { 2024-04-07T17:51:27.254 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 10912, 2024-04-07T17:51:27.254 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 124 2024-04-07T17:51:27.254 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.254 INFO:tasks.workunit.client.0.smithi100.stdout: "mds_co": { 2024-04-07T17:51:27.254 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.254 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.254 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.254 INFO:tasks.workunit.client.0.smithi100.stdout: "osd": { 2024-04-07T17:51:27.254 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 536544, 2024-04-07T17:51:27.254 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 46 2024-04-07T17:51:27.254 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.254 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mapbl": { 2024-04-07T17:51:27.254 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.254 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.254 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.254 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_pglog": { 2024-04-07T17:51:27.255 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 5008224, 2024-04-07T17:51:27.255 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 9660 2024-04-07T17:51:27.255 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.255 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap": { 2024-04-07T17:51:27.255 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 635376, 2024-04-07T17:51:27.255 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 11076 2024-04-07T17:51:27.255 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.255 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap_mapping": { 2024-04-07T17:51:27.255 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.255 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.255 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.255 INFO:tasks.workunit.client.0.smithi100.stdout: "pgmap": { 2024-04-07T17:51:27.255 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.255 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.255 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.256 INFO:tasks.workunit.client.0.smithi100.stdout: "unittest_1": { 2024-04-07T17:51:27.256 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.256 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.256 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.256 INFO:tasks.workunit.client.0.smithi100.stdout: "unittest_2": { 2024-04-07T17:51:27.256 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:27.256 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:27.256 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.256 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.256 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.256 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.256 INFO:tasks.workunit.client.0.smithi100.stdout: "metadata": { 2024-04-07T17:51:27.256 INFO:tasks.workunit.client.0.smithi100.stdout: "mds": { 2024-04-07T17:51:27.256 INFO:tasks.workunit.client.0.smithi100.stdout: "arch": { 2024-04-07T17:51:27.256 INFO:tasks.workunit.client.0.smithi100.stdout: "x86_64": 2 2024-04-07T17:51:27.257 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.257 INFO:tasks.workunit.client.0.smithi100.stdout: "ceph_version": { 2024-04-07T17:51:27.257 INFO:tasks.workunit.client.0.smithi100.stdout: "ceph version 19.0.0-2785-gadce92d0 (adce92d056533f22dc1ee37e3141d098b046ec1b) squid (dev)": 2 2024-04-07T17:51:27.257 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.257 INFO:tasks.workunit.client.0.smithi100.stdout: "cpu": { 2024-04-07T17:51:27.257 INFO:tasks.workunit.client.0.smithi100.stdout: "Intel(R) Xeon(R) CPU E5-1620 v4 @ 3.50GHz": 2 2024-04-07T17:51:27.257 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.257 INFO:tasks.workunit.client.0.smithi100.stdout: "distro": { 2024-04-07T17:51:27.257 INFO:tasks.workunit.client.0.smithi100.stdout: "centos": 2 2024-04-07T17:51:27.257 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.257 INFO:tasks.workunit.client.0.smithi100.stdout: "distro_description": { 2024-04-07T17:51:27.257 INFO:tasks.workunit.client.0.smithi100.stdout: "CentOS Stream 8": 2 2024-04-07T17:51:27.257 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.257 INFO:tasks.workunit.client.0.smithi100.stdout: "kernel_description": { 2024-04-07T17:51:27.258 INFO:tasks.workunit.client.0.smithi100.stdout: "#111-Ubuntu SMP Tue Mar 5 20:16:58 UTC 2024": 2 2024-04-07T17:51:27.258 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.258 INFO:tasks.workunit.client.0.smithi100.stdout: "kernel_version": { 2024-04-07T17:51:27.258 INFO:tasks.workunit.client.0.smithi100.stdout: "5.15.0-101-generic": 2 2024-04-07T17:51:27.258 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.258 INFO:tasks.workunit.client.0.smithi100.stdout: "os": { 2024-04-07T17:51:27.258 INFO:tasks.workunit.client.0.smithi100.stdout: "Linux": 2 2024-04-07T17:51:27.258 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.258 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.258 INFO:tasks.workunit.client.0.smithi100.stdout: "mon": { 2024-04-07T17:51:27.258 INFO:tasks.workunit.client.0.smithi100.stdout: "arch": { 2024-04-07T17:51:27.258 INFO:tasks.workunit.client.0.smithi100.stdout: "x86_64": 3 2024-04-07T17:51:27.258 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.258 INFO:tasks.workunit.client.0.smithi100.stdout: "ceph_version": { 2024-04-07T17:51:27.258 INFO:tasks.workunit.client.0.smithi100.stdout: "ceph version 19.0.0-2785-gadce92d0 (adce92d056533f22dc1ee37e3141d098b046ec1b) squid (dev)": 3 2024-04-07T17:51:27.258 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.259 INFO:tasks.workunit.client.0.smithi100.stdout: "cpu": { 2024-04-07T17:51:27.259 INFO:tasks.workunit.client.0.smithi100.stdout: "Intel(R) Xeon(R) CPU E5-1620 v4 @ 3.50GHz": 3 2024-04-07T17:51:27.259 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.259 INFO:tasks.workunit.client.0.smithi100.stdout: "distro": { 2024-04-07T17:51:27.259 INFO:tasks.workunit.client.0.smithi100.stdout: "centos": 3 2024-04-07T17:51:27.259 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.259 INFO:tasks.workunit.client.0.smithi100.stdout: "distro_description": { 2024-04-07T17:51:27.259 INFO:tasks.workunit.client.0.smithi100.stdout: "CentOS Stream 8": 3 2024-04-07T17:51:27.259 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.259 INFO:tasks.workunit.client.0.smithi100.stdout: "kernel_description": { 2024-04-07T17:51:27.259 INFO:tasks.workunit.client.0.smithi100.stdout: "#111-Ubuntu SMP Tue Mar 5 20:16:58 UTC 2024": 3 2024-04-07T17:51:27.259 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.259 INFO:tasks.workunit.client.0.smithi100.stdout: "kernel_version": { 2024-04-07T17:51:27.259 INFO:tasks.workunit.client.0.smithi100.stdout: "5.15.0-101-generic": 3 2024-04-07T17:51:27.259 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.260 INFO:tasks.workunit.client.0.smithi100.stdout: "os": { 2024-04-07T17:51:27.260 INFO:tasks.workunit.client.0.smithi100.stdout: "Linux": 3 2024-04-07T17:51:27.260 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.260 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.260 INFO:tasks.workunit.client.0.smithi100.stdout: "osd": { 2024-04-07T17:51:27.260 INFO:tasks.workunit.client.0.smithi100.stdout: "arch": { 2024-04-07T17:51:27.260 INFO:tasks.workunit.client.0.smithi100.stdout: "x86_64": 8 2024-04-07T17:51:27.260 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.260 INFO:tasks.workunit.client.0.smithi100.stdout: "ceph_version": { 2024-04-07T17:51:27.260 INFO:tasks.workunit.client.0.smithi100.stdout: "ceph version 19.0.0-2785-gadce92d0 (adce92d056533f22dc1ee37e3141d098b046ec1b) squid (dev)": 8 2024-04-07T17:51:27.260 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.260 INFO:tasks.workunit.client.0.smithi100.stdout: "cpu": { 2024-04-07T17:51:27.260 INFO:tasks.workunit.client.0.smithi100.stdout: "Intel(R) Xeon(R) CPU E5-1620 v4 @ 3.50GHz": 8 2024-04-07T17:51:27.260 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.260 INFO:tasks.workunit.client.0.smithi100.stdout: "distro": { 2024-04-07T17:51:27.260 INFO:tasks.workunit.client.0.smithi100.stdout: "centos": 8 2024-04-07T17:51:27.260 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.261 INFO:tasks.workunit.client.0.smithi100.stdout: "distro_description": { 2024-04-07T17:51:27.261 INFO:tasks.workunit.client.0.smithi100.stdout: "CentOS Stream 8": 8 2024-04-07T17:51:27.261 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.261 INFO:tasks.workunit.client.0.smithi100.stdout: "kernel_description": { 2024-04-07T17:51:27.261 INFO:tasks.workunit.client.0.smithi100.stdout: "#111-Ubuntu SMP Tue Mar 5 20:16:58 UTC 2024": 8 2024-04-07T17:51:27.261 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.261 INFO:tasks.workunit.client.0.smithi100.stdout: "kernel_version": { 2024-04-07T17:51:27.261 INFO:tasks.workunit.client.0.smithi100.stdout: "5.15.0-101-generic": 8 2024-04-07T17:51:27.261 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.261 INFO:tasks.workunit.client.0.smithi100.stdout: "os": { 2024-04-07T17:51:27.261 INFO:tasks.workunit.client.0.smithi100.stdout: "Linux": 8 2024-04-07T17:51:27.261 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.261 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_objectstore": { 2024-04-07T17:51:27.261 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore": 8 2024-04-07T17:51:27.262 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.262 INFO:tasks.workunit.client.0.smithi100.stdout: "rotational": { 2024-04-07T17:51:27.262 INFO:tasks.workunit.client.0.smithi100.stdout: "0": 8 2024-04-07T17:51:27.262 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.262 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.262 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.262 INFO:tasks.workunit.client.0.smithi100.stdout: "mon": { 2024-04-07T17:51:27.262 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 3, 2024-04-07T17:51:27.262 INFO:tasks.workunit.client.0.smithi100.stdout: "features": { 2024-04-07T17:51:27.262 INFO:tasks.workunit.client.0.smithi100.stdout: "optional": [], 2024-04-07T17:51:27.262 INFO:tasks.workunit.client.0.smithi100.stdout: "persistent": [ 2024-04-07T17:51:27.262 INFO:tasks.workunit.client.0.smithi100.stdout: "kraken", 2024-04-07T17:51:27.262 INFO:tasks.workunit.client.0.smithi100.stdout: "luminous", 2024-04-07T17:51:27.262 INFO:tasks.workunit.client.0.smithi100.stdout: "mimic", 2024-04-07T17:51:27.263 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap-prune", 2024-04-07T17:51:27.263 INFO:tasks.workunit.client.0.smithi100.stdout: "nautilus", 2024-04-07T17:51:27.263 INFO:tasks.workunit.client.0.smithi100.stdout: "octopus", 2024-04-07T17:51:27.263 INFO:tasks.workunit.client.0.smithi100.stdout: "pacific", 2024-04-07T17:51:27.263 INFO:tasks.workunit.client.0.smithi100.stdout: "elector-pinging", 2024-04-07T17:51:27.263 INFO:tasks.workunit.client.0.smithi100.stdout: "quincy", 2024-04-07T17:51:27.263 INFO:tasks.workunit.client.0.smithi100.stdout: "reef", 2024-04-07T17:51:27.263 INFO:tasks.workunit.client.0.smithi100.stdout: "squid" 2024-04-07T17:51:27.263 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:27.263 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.263 INFO:tasks.workunit.client.0.smithi100.stdout: "ipv4_addr_mons": 6, 2024-04-07T17:51:27.263 INFO:tasks.workunit.client.0.smithi100.stdout: "ipv6_addr_mons": 0, 2024-04-07T17:51:27.263 INFO:tasks.workunit.client.0.smithi100.stdout: "min_mon_release": 19, 2024-04-07T17:51:27.263 INFO:tasks.workunit.client.0.smithi100.stdout: "v1_addr_mons": 3, 2024-04-07T17:51:27.263 INFO:tasks.workunit.client.0.smithi100.stdout: "v2_addr_mons": 3 2024-04-07T17:51:27.264 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.264 INFO:tasks.workunit.client.0.smithi100.stdout: "organization": "ceph-qa", 2024-04-07T17:51:27.264 INFO:tasks.workunit.client.0.smithi100.stdout: "osd": { 2024-04-07T17:51:27.264 INFO:tasks.workunit.client.0.smithi100.stdout: "cluster_network": false, 2024-04-07T17:51:27.264 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 8, 2024-04-07T17:51:27.264 INFO:tasks.workunit.client.0.smithi100.stdout: "require_min_compat_client": "luminous", 2024-04-07T17:51:27.264 INFO:tasks.workunit.client.0.smithi100.stdout: "require_osd_release": "squid" 2024-04-07T17:51:27.264 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.264 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_perf_histograms": [ 2024-04-07T17:51:27.264 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:27.264 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_latency_out_bytes_histogram": { 2024-04-07T17:51:27.264 INFO:tasks.workunit.client.0.smithi100.stdout: "axes": [ 2024-04-07T17:51:27.264 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:27.264 INFO:tasks.workunit.client.0.smithi100.stdout: "buckets": 32, 2024-04-07T17:51:27.264 INFO:tasks.workunit.client.0.smithi100.stdout: "min": 0, 2024-04-07T17:51:27.264 INFO:tasks.workunit.client.0.smithi100.stdout: "name": "Latency (usec)", 2024-04-07T17:51:27.264 INFO:tasks.workunit.client.0.smithi100.stdout: "quant_size": 100000, 2024-04-07T17:51:27.265 INFO:tasks.workunit.client.0.smithi100.stdout: "ranges": [ 2024-04-07T17:51:27.265 INFO:tasks.workunit.client.0.smithi100.stdout: "[None, -1]", 2024-04-07T17:51:27.265 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 99999]", 2024-04-07T17:51:27.265 INFO:tasks.workunit.client.0.smithi100.stdout: "[100000, 199999]", 2024-04-07T17:51:27.265 INFO:tasks.workunit.client.0.smithi100.stdout: "[200000, 399999]", 2024-04-07T17:51:27.265 INFO:tasks.workunit.client.0.smithi100.stdout: "[400000, 799999]", 2024-04-07T17:51:27.265 INFO:tasks.workunit.client.0.smithi100.stdout: "[800000, 1599999]", 2024-04-07T17:51:27.265 INFO:tasks.workunit.client.0.smithi100.stdout: "[1600000, 3199999]", 2024-04-07T17:51:27.265 INFO:tasks.workunit.client.0.smithi100.stdout: "[3200000, 6399999]", 2024-04-07T17:51:27.265 INFO:tasks.workunit.client.0.smithi100.stdout: "[6400000, 12799999]", 2024-04-07T17:51:27.265 INFO:tasks.workunit.client.0.smithi100.stdout: "[12800000, 25599999]", 2024-04-07T17:51:27.265 INFO:tasks.workunit.client.0.smithi100.stdout: "[25600000, 51199999]", 2024-04-07T17:51:27.265 INFO:tasks.workunit.client.0.smithi100.stdout: "[51200000, 102399999]", 2024-04-07T17:51:27.265 INFO:tasks.workunit.client.0.smithi100.stdout: "[102400000, 204799999]", 2024-04-07T17:51:27.265 INFO:tasks.workunit.client.0.smithi100.stdout: "[204800000, 409599999]", 2024-04-07T17:51:27.266 INFO:tasks.workunit.client.0.smithi100.stdout: "[409600000, 819199999]", 2024-04-07T17:51:27.266 INFO:tasks.workunit.client.0.smithi100.stdout: "[819200000, 1638399999]", 2024-04-07T17:51:27.266 INFO:tasks.workunit.client.0.smithi100.stdout: "[1638400000, 3276799999]", 2024-04-07T17:51:27.266 INFO:tasks.workunit.client.0.smithi100.stdout: "[3276800000, 6553599999]", 2024-04-07T17:51:27.266 INFO:tasks.workunit.client.0.smithi100.stdout: "[6553600000, 13107199999]", 2024-04-07T17:51:27.266 INFO:tasks.workunit.client.0.smithi100.stdout: "[13107200000, 26214399999]", 2024-04-07T17:51:27.266 INFO:tasks.workunit.client.0.smithi100.stdout: "[26214400000, 52428799999]", 2024-04-07T17:51:27.266 INFO:tasks.workunit.client.0.smithi100.stdout: "[52428800000, 104857599999]", 2024-04-07T17:51:27.266 INFO:tasks.workunit.client.0.smithi100.stdout: "[104857600000, 209715199999]", 2024-04-07T17:51:27.266 INFO:tasks.workunit.client.0.smithi100.stdout: "[209715200000, 419430399999]", 2024-04-07T17:51:27.266 INFO:tasks.workunit.client.0.smithi100.stdout: "[419430400000, 838860799999]", 2024-04-07T17:51:27.266 INFO:tasks.workunit.client.0.smithi100.stdout: "[838860800000, 1677721599999]", 2024-04-07T17:51:27.266 INFO:tasks.workunit.client.0.smithi100.stdout: "[1677721600000, 3355443199999]", 2024-04-07T17:51:27.266 INFO:tasks.workunit.client.0.smithi100.stdout: "[3355443200000, 6710886399999]", 2024-04-07T17:51:27.266 INFO:tasks.workunit.client.0.smithi100.stdout: "[6710886400000, 13421772799999]", 2024-04-07T17:51:27.266 INFO:tasks.workunit.client.0.smithi100.stdout: "[13421772800000, 26843545599999]", 2024-04-07T17:51:27.267 INFO:tasks.workunit.client.0.smithi100.stdout: "[26843545600000, 53687091199999]", 2024-04-07T17:51:27.267 INFO:tasks.workunit.client.0.smithi100.stdout: "[53687091200000, None]" 2024-04-07T17:51:27.267 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:27.267 INFO:tasks.workunit.client.0.smithi100.stdout: "scale_type": "log2" 2024-04-07T17:51:27.267 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.267 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:27.267 INFO:tasks.workunit.client.0.smithi100.stdout: "buckets": 32, 2024-04-07T17:51:27.267 INFO:tasks.workunit.client.0.smithi100.stdout: "min": 0, 2024-04-07T17:51:27.267 INFO:tasks.workunit.client.0.smithi100.stdout: "name": "Request size (bytes)", 2024-04-07T17:51:27.267 INFO:tasks.workunit.client.0.smithi100.stdout: "quant_size": 512, 2024-04-07T17:51:27.267 INFO:tasks.workunit.client.0.smithi100.stdout: "ranges": [ 2024-04-07T17:51:27.267 INFO:tasks.workunit.client.0.smithi100.stdout: "[None, -1]", 2024-04-07T17:51:27.267 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 511]", 2024-04-07T17:51:27.267 INFO:tasks.workunit.client.0.smithi100.stdout: "[512, 1023]", 2024-04-07T17:51:27.268 INFO:tasks.workunit.client.0.smithi100.stdout: "[1024, 2047]", 2024-04-07T17:51:27.268 INFO:tasks.workunit.client.0.smithi100.stdout: "[2048, 4095]", 2024-04-07T17:51:27.268 INFO:tasks.workunit.client.0.smithi100.stdout: "[4096, 8191]", 2024-04-07T17:51:27.268 INFO:tasks.workunit.client.0.smithi100.stdout: "[8192, 16383]", 2024-04-07T17:51:27.268 INFO:tasks.workunit.client.0.smithi100.stdout: "[16384, 32767]", 2024-04-07T17:51:27.268 INFO:tasks.workunit.client.0.smithi100.stdout: "[32768, 65535]", 2024-04-07T17:51:27.268 INFO:tasks.workunit.client.0.smithi100.stdout: "[65536, 131071]", 2024-04-07T17:51:27.268 INFO:tasks.workunit.client.0.smithi100.stdout: "[131072, 262143]", 2024-04-07T17:51:27.268 INFO:tasks.workunit.client.0.smithi100.stdout: "[262144, 524287]", 2024-04-07T17:51:27.268 INFO:tasks.workunit.client.0.smithi100.stdout: "[524288, 1048575]", 2024-04-07T17:51:27.268 INFO:tasks.workunit.client.0.smithi100.stdout: "[1048576, 2097151]", 2024-04-07T17:51:27.268 INFO:tasks.workunit.client.0.smithi100.stdout: "[2097152, 4194303]", 2024-04-07T17:51:27.268 INFO:tasks.workunit.client.0.smithi100.stdout: "[4194304, 8388607]", 2024-04-07T17:51:27.268 INFO:tasks.workunit.client.0.smithi100.stdout: "[8388608, 16777215]", 2024-04-07T17:51:27.268 INFO:tasks.workunit.client.0.smithi100.stdout: "[16777216, 33554431]", 2024-04-07T17:51:27.269 INFO:tasks.workunit.client.0.smithi100.stdout: "[33554432, 67108863]", 2024-04-07T17:51:27.269 INFO:tasks.workunit.client.0.smithi100.stdout: "[67108864, 134217727]", 2024-04-07T17:51:27.269 INFO:tasks.workunit.client.0.smithi100.stdout: "[134217728, 268435455]", 2024-04-07T17:51:27.269 INFO:tasks.workunit.client.0.smithi100.stdout: "[268435456, 536870911]", 2024-04-07T17:51:27.269 INFO:tasks.workunit.client.0.smithi100.stdout: "[536870912, 1073741823]", 2024-04-07T17:51:27.269 INFO:tasks.workunit.client.0.smithi100.stdout: "[1073741824, 2147483647]", 2024-04-07T17:51:27.269 INFO:tasks.workunit.client.0.smithi100.stdout: "[2147483648, 4294967295]", 2024-04-07T17:51:27.269 INFO:tasks.workunit.client.0.smithi100.stdout: "[4294967296, 8589934591]", 2024-04-07T17:51:27.269 INFO:tasks.workunit.client.0.smithi100.stdout: "[8589934592, 17179869183]", 2024-04-07T17:51:27.269 INFO:tasks.workunit.client.0.smithi100.stdout: "[17179869184, 34359738367]", 2024-04-07T17:51:27.269 INFO:tasks.workunit.client.0.smithi100.stdout: "[34359738368, 68719476735]", 2024-04-07T17:51:27.269 INFO:tasks.workunit.client.0.smithi100.stdout: "[68719476736, 137438953471]", 2024-04-07T17:51:27.269 INFO:tasks.workunit.client.0.smithi100.stdout: "[137438953472, 274877906943]", 2024-04-07T17:51:27.269 INFO:tasks.workunit.client.0.smithi100.stdout: "[274877906944, None]" 2024-04-07T17:51:27.269 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:27.270 INFO:tasks.workunit.client.0.smithi100.stdout: "scale_type": "log2" 2024-04-07T17:51:27.270 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.270 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:27.270 INFO:tasks.workunit.client.0.smithi100.stdout: "osds": [ 2024-04-07T17:51:27.270 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:27.270 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 0, 2024-04-07T17:51:27.270 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:27.270 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.270 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.270 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 997, 13, 0, 0, 648, 16, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.270 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 7214, 3165, 0, 1, 4231, 80, 0, 4, 1, 0, 3, 2, 4, 0, 12, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.270 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 11866, 1050, 12, 2, 2904, 503, 11, 11, 26, 39, 32, 36, 59, 21, 77, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.270 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1610, 30, 63, 0, 19, 15, 2, 5, 5, 13, 105, 163, 32, 5, 39, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.270 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 417, 0, 5, 0, 3, 9, 0, 1, 0, 1, 3, 37, 361, 22, 36, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.271 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 141, 1, 0, 0, 5, 11, 0, 0, 0, 1, 2, 13, 25, 31, 205, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.271 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 120, 1, 0, 0, 0, 0, 1, 2, 2, 0, 0, 7, 21, 15, 56, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.271 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 192, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 4, 14, 26, 33, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.271 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 32, 0, 0, 0, 0, 0, 0, 0, 0, 0, 2, 0, 4, 19, 20, 5, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.271 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 6, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 4, 3, 9, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.271 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.271 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.271 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.271 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.271 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.271 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.271 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.271 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.271 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.271 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.272 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.272 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.272 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.272 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.272 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.272 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.272 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.272 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.272 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.272 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:27.272 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:27.272 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.272 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:27.272 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 1, 2024-04-07T17:51:27.272 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:27.273 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.273 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.273 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1446, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.273 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 5794, 4, 145, 0, 8, 0, 0, 0, 1, 1, 2, 1, 2, 0, 9, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.273 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 9603, 33, 88, 8, 28, 51, 80, 25, 39, 53, 38, 35, 64, 23, 44, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.273 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1252, 11, 1178, 4, 3, 3, 38, 118, 6, 17, 129, 228, 29, 7, 16, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.273 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 284, 1, 144, 24, 0, 0, 0, 14, 2, 1, 4, 45, 487, 18, 5, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.273 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 188, 0, 9, 6, 0, 0, 0, 0, 0, 1, 8, 2, 4, 12, 161, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.273 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 174, 0, 76, 0, 0, 0, 0, 0, 0, 0, 9, 0, 4, 5, 36, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.273 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 95, 0, 62, 0, 0, 0, 0, 0, 0, 0, 3, 0, 3, 11, 17, 6, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.273 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 7, 0, 153, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 5, 2, 10, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.273 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 3, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.273 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.273 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.273 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.274 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.274 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.274 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.274 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.274 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.274 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.274 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.274 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.274 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.274 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.274 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.274 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.274 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.274 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.275 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.275 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.275 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:27.275 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:27.275 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.275 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:27.275 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 2, 2024-04-07T17:51:27.275 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:27.275 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.275 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.275 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 2187, 6, 0, 0, 0, 0, 0, 0, 0, 0, 2, 0, 0, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.275 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 6047, 2629, 2809, 0, 8, 0, 0, 1, 2, 0, 23, 0, 2, 7, 5, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.275 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 9066, 1269, 1692, 0, 3, 1, 0, 2, 6, 6, 32, 2, 20, 30, 68, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.275 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1309, 32, 27, 1, 2, 0, 0, 1, 9, 9, 29, 7, 17, 16, 33, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.275 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 335, 7, 6, 1, 0, 0, 0, 0, 0, 0, 3, 7, 47, 45, 17, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.275 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 152, 2, 0, 0, 0, 0, 0, 3, 0, 1, 3, 25, 11, 60, 181, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.275 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 149, 1, 0, 0, 0, 0, 0, 1, 3, 0, 3, 18, 37, 45, 52, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.276 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 208, 0, 0, 1, 0, 1, 2, 0, 0, 0, 4, 20, 43, 66, 44, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.276 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 52, 0, 0, 0, 0, 0, 0, 3, 0, 0, 0, 2, 3, 18, 51, 7, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.276 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 10, 0, 0, 0, 1, 0, 0, 1, 0, 0, 0, 0, 0, 8, 3, 4, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.276 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.276 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.276 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.276 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.276 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.276 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.276 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.276 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.276 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.276 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.276 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.277 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.277 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.277 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.277 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.277 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.277 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.277 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.277 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.277 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:27.277 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:27.277 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.277 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:27.277 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 3, 2024-04-07T17:51:27.277 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:27.277 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.277 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.278 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 3687, 36, 0, 1, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.278 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 12530, 2576, 0, 0, 13, 0, 0, 0, 0, 0, 1, 1, 1, 0, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.278 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 13127, 1790, 0, 0, 8, 6, 2, 8, 12, 16, 12, 7, 12, 5, 27, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.278 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1412, 42, 24, 0, 26, 0, 0, 2, 3, 5, 41, 74, 13, 6, 10, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.278 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 416, 3, 13, 0, 157, 0, 0, 0, 0, 0, 2, 12, 225, 42, 4, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.278 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 125, 1, 0, 0, 306, 1, 0, 0, 0, 0, 0, 0, 6, 21, 84, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.278 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 27, 0, 2, 2, 334, 0, 0, 0, 0, 0, 0, 2, 6, 7, 22, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.278 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 10, 0, 0, 0, 1487, 0, 0, 0, 0, 0, 0, 0, 4, 9, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.278 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 8, 0, 0, 0, 374, 0, 0, 0, 0, 0, 0, 0, 0, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.278 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.278 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.278 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.278 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.278 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.278 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.279 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.279 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.279 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.279 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.279 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.279 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.279 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.279 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.279 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.279 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.279 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.279 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.279 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.279 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.279 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:27.280 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:27.280 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.280 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:27.280 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 4, 2024-04-07T17:51:27.280 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:27.280 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.280 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.280 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 609, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.280 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 3885, 8, 15, 0, 0, 0, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.280 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 10736, 14, 53, 5, 22, 32, 30, 14, 13, 15, 11, 13, 30, 4, 36, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.280 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1540, 27, 25, 0, 9, 3, 41, 70, 1, 5, 58, 81, 10, 3, 28, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.280 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 375, 10, 14, 0, 0, 0, 0, 0, 0, 0, 1, 16, 198, 13, 3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.280 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 187, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 3, 9, 9, 56, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.280 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 61, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 5, 13, 3, 46, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.280 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 34, 0, 0, 0, 0, 0, 0, 0, 0, 2, 2, 3, 6, 3, 18, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.281 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 8, 0, 0, 0, 0, 0, 0, 1, 0, 0, 1, 0, 0, 0, 2, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.281 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.281 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.281 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.281 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.281 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.281 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.281 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.281 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.281 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.281 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.281 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.281 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.281 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.281 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.282 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.282 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.282 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.282 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.282 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.282 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.282 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:27.282 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:27.282 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.282 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:27.282 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 5, 2024-04-07T17:51:27.282 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:27.282 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.282 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.282 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 946, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.283 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 5643, 1, 0, 1, 0, 0, 1, 0, 0, 1, 3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.283 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 14942, 20, 3, 0, 21, 3, 0, 4, 1, 9, 44, 5, 10, 29, 54, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.283 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 2337, 103, 114, 1, 1, 1, 0, 2, 3, 4, 28, 12, 3, 5, 47, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.283 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 783, 40, 72, 4, 6, 0, 0, 0, 0, 0, 2, 0, 27, 26, 14, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.283 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 237, 9, 6, 4, 0, 0, 0, 2, 1, 0, 0, 3, 4, 39, 200, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.283 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 158, 10, 24, 5, 1, 0, 0, 1, 0, 0, 0, 2, 1, 6, 66, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.283 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 156, 1, 10, 45, 24, 0, 0, 0, 0, 0, 0, 0, 4, 6, 14, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.283 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 10, 0, 0, 16, 104, 86, 0, 0, 0, 0, 0, 0, 3, 6, 14, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.283 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 3, 0, 0, 0, 6, 43, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.283 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.283 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.283 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.283 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.283 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.283 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.283 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.284 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.284 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.284 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.284 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.284 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.284 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.284 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.284 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.284 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.284 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.284 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.284 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.284 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:27.284 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:27.284 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.285 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:27.285 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 6, 2024-04-07T17:51:27.285 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:27.285 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.285 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.285 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 830, 5, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.285 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 6114, 1983, 0, 0, 2, 1, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.285 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 10059, 2215, 1, 1, 22, 3, 1, 0, 18, 18, 10, 10, 26, 17, 43, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.285 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1593, 28, 21, 0, 2, 1, 0, 0, 5, 13, 58, 83, 12, 5, 27, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.285 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 625, 3, 7, 0, 0, 0, 0, 0, 0, 1, 7, 26, 211, 19, 7, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.285 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 531, 11, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 6, 22, 90, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.285 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 196, 8, 0, 0, 0, 1, 0, 0, 0, 1, 0, 0, 2, 1, 53, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.285 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 155, 8, 0, 0, 0, 0, 0, 1, 0, 3, 0, 0, 0, 1, 6, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.285 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 8, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.285 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.286 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.286 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.286 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.286 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.286 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.286 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.286 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.286 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.286 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.286 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.286 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.286 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.286 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.286 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.287 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.287 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.287 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.287 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.287 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.287 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:27.287 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:27.287 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.287 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:27.287 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 7, 2024-04-07T17:51:27.287 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:27.287 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.287 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.287 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 385, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.287 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 6682, 2090, 1, 0, 0, 0, 0, 0, 0, 0, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.287 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 13787, 2103, 0, 0, 7, 2, 0, 1, 0, 1, 1, 13, 10, 17, 55, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.288 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1294, 17, 4, 1, 0, 0, 0, 1, 0, 2, 17, 6, 3, 7, 31, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.288 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 308, 17, 0, 0, 0, 0, 0, 0, 0, 0, 0, 4, 20, 40, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.288 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 152, 63, 8, 0, 0, 0, 0, 2, 0, 0, 3, 1, 6, 32, 113, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.288 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 98, 138, 5, 0, 0, 0, 0, 1, 0, 0, 0, 2, 2, 7, 52, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.288 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 223, 58, 1, 0, 0, 0, 0, 0, 0, 0, 3, 0, 0, 8, 11, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.288 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 49, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 6, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.288 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.288 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.288 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.288 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.288 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.288 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.288 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.288 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.289 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.289 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.289 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.289 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.289 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.289 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.289 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.289 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.289 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.289 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.289 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.289 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.289 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:27.289 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:27.289 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.289 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:27.290 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.290 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_latency_in_bytes_histogram": { 2024-04-07T17:51:27.290 INFO:tasks.workunit.client.0.smithi100.stdout: "axes": [ 2024-04-07T17:51:27.290 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:27.290 INFO:tasks.workunit.client.0.smithi100.stdout: "buckets": 32, 2024-04-07T17:51:27.290 INFO:tasks.workunit.client.0.smithi100.stdout: "min": 0, 2024-04-07T17:51:27.290 INFO:tasks.workunit.client.0.smithi100.stdout: "name": "Latency (usec)", 2024-04-07T17:51:27.290 INFO:tasks.workunit.client.0.smithi100.stdout: "quant_size": 100000, 2024-04-07T17:51:27.290 INFO:tasks.workunit.client.0.smithi100.stdout: "ranges": [ 2024-04-07T17:51:27.290 INFO:tasks.workunit.client.0.smithi100.stdout: "[None, -1]", 2024-04-07T17:51:27.290 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 99999]", 2024-04-07T17:51:27.290 INFO:tasks.workunit.client.0.smithi100.stdout: "[100000, 199999]", 2024-04-07T17:51:27.290 INFO:tasks.workunit.client.0.smithi100.stdout: "[200000, 399999]", 2024-04-07T17:51:27.290 INFO:tasks.workunit.client.0.smithi100.stdout: "[400000, 799999]", 2024-04-07T17:51:27.291 INFO:tasks.workunit.client.0.smithi100.stdout: "[800000, 1599999]", 2024-04-07T17:51:27.291 INFO:tasks.workunit.client.0.smithi100.stdout: "[1600000, 3199999]", 2024-04-07T17:51:27.291 INFO:tasks.workunit.client.0.smithi100.stdout: "[3200000, 6399999]", 2024-04-07T17:51:27.291 INFO:tasks.workunit.client.0.smithi100.stdout: "[6400000, 12799999]", 2024-04-07T17:51:27.291 INFO:tasks.workunit.client.0.smithi100.stdout: "[12800000, 25599999]", 2024-04-07T17:51:27.291 INFO:tasks.workunit.client.0.smithi100.stdout: "[25600000, 51199999]", 2024-04-07T17:51:27.291 INFO:tasks.workunit.client.0.smithi100.stdout: "[51200000, 102399999]", 2024-04-07T17:51:27.291 INFO:tasks.workunit.client.0.smithi100.stdout: "[102400000, 204799999]", 2024-04-07T17:51:27.291 INFO:tasks.workunit.client.0.smithi100.stdout: "[204800000, 409599999]", 2024-04-07T17:51:27.291 INFO:tasks.workunit.client.0.smithi100.stdout: "[409600000, 819199999]", 2024-04-07T17:51:27.291 INFO:tasks.workunit.client.0.smithi100.stdout: "[819200000, 1638399999]", 2024-04-07T17:51:27.291 INFO:tasks.workunit.client.0.smithi100.stdout: "[1638400000, 3276799999]", 2024-04-07T17:51:27.291 INFO:tasks.workunit.client.0.smithi100.stdout: "[3276800000, 6553599999]", 2024-04-07T17:51:27.291 INFO:tasks.workunit.client.0.smithi100.stdout: "[6553600000, 13107199999]", 2024-04-07T17:51:27.291 INFO:tasks.workunit.client.0.smithi100.stdout: "[13107200000, 26214399999]", 2024-04-07T17:51:27.292 INFO:tasks.workunit.client.0.smithi100.stdout: "[26214400000, 52428799999]", 2024-04-07T17:51:27.292 INFO:tasks.workunit.client.0.smithi100.stdout: "[52428800000, 104857599999]", 2024-04-07T17:51:27.292 INFO:tasks.workunit.client.0.smithi100.stdout: "[104857600000, 209715199999]", 2024-04-07T17:51:27.292 INFO:tasks.workunit.client.0.smithi100.stdout: "[209715200000, 419430399999]", 2024-04-07T17:51:27.292 INFO:tasks.workunit.client.0.smithi100.stdout: "[419430400000, 838860799999]", 2024-04-07T17:51:27.292 INFO:tasks.workunit.client.0.smithi100.stdout: "[838860800000, 1677721599999]", 2024-04-07T17:51:27.292 INFO:tasks.workunit.client.0.smithi100.stdout: "[1677721600000, 3355443199999]", 2024-04-07T17:51:27.292 INFO:tasks.workunit.client.0.smithi100.stdout: "[3355443200000, 6710886399999]", 2024-04-07T17:51:27.292 INFO:tasks.workunit.client.0.smithi100.stdout: "[6710886400000, 13421772799999]", 2024-04-07T17:51:27.292 INFO:tasks.workunit.client.0.smithi100.stdout: "[13421772800000, 26843545599999]", 2024-04-07T17:51:27.292 INFO:tasks.workunit.client.0.smithi100.stdout: "[26843545600000, 53687091199999]", 2024-04-07T17:51:27.292 INFO:tasks.workunit.client.0.smithi100.stdout: "[53687091200000, None]" 2024-04-07T17:51:27.292 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:27.292 INFO:tasks.workunit.client.0.smithi100.stdout: "scale_type": "log2" 2024-04-07T17:51:27.292 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.292 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:27.292 INFO:tasks.workunit.client.0.smithi100.stdout: "buckets": 32, 2024-04-07T17:51:27.293 INFO:tasks.workunit.client.0.smithi100.stdout: "min": 0, 2024-04-07T17:51:27.293 INFO:tasks.workunit.client.0.smithi100.stdout: "name": "Request size (bytes)", 2024-04-07T17:51:27.293 INFO:tasks.workunit.client.0.smithi100.stdout: "quant_size": 512, 2024-04-07T17:51:27.293 INFO:tasks.workunit.client.0.smithi100.stdout: "ranges": [ 2024-04-07T17:51:27.293 INFO:tasks.workunit.client.0.smithi100.stdout: "[None, -1]", 2024-04-07T17:51:27.293 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 511]", 2024-04-07T17:51:27.293 INFO:tasks.workunit.client.0.smithi100.stdout: "[512, 1023]", 2024-04-07T17:51:27.293 INFO:tasks.workunit.client.0.smithi100.stdout: "[1024, 2047]", 2024-04-07T17:51:27.293 INFO:tasks.workunit.client.0.smithi100.stdout: "[2048, 4095]", 2024-04-07T17:51:27.293 INFO:tasks.workunit.client.0.smithi100.stdout: "[4096, 8191]", 2024-04-07T17:51:27.293 INFO:tasks.workunit.client.0.smithi100.stdout: "[8192, 16383]", 2024-04-07T17:51:27.293 INFO:tasks.workunit.client.0.smithi100.stdout: "[16384, 32767]", 2024-04-07T17:51:27.293 INFO:tasks.workunit.client.0.smithi100.stdout: "[32768, 65535]", 2024-04-07T17:51:27.293 INFO:tasks.workunit.client.0.smithi100.stdout: "[65536, 131071]", 2024-04-07T17:51:27.294 INFO:tasks.workunit.client.0.smithi100.stdout: "[131072, 262143]", 2024-04-07T17:51:27.294 INFO:tasks.workunit.client.0.smithi100.stdout: "[262144, 524287]", 2024-04-07T17:51:27.294 INFO:tasks.workunit.client.0.smithi100.stdout: "[524288, 1048575]", 2024-04-07T17:51:27.294 INFO:tasks.workunit.client.0.smithi100.stdout: "[1048576, 2097151]", 2024-04-07T17:51:27.294 INFO:tasks.workunit.client.0.smithi100.stdout: "[2097152, 4194303]", 2024-04-07T17:51:27.294 INFO:tasks.workunit.client.0.smithi100.stdout: "[4194304, 8388607]", 2024-04-07T17:51:27.294 INFO:tasks.workunit.client.0.smithi100.stdout: "[8388608, 16777215]", 2024-04-07T17:51:27.294 INFO:tasks.workunit.client.0.smithi100.stdout: "[16777216, 33554431]", 2024-04-07T17:51:27.294 INFO:tasks.workunit.client.0.smithi100.stdout: "[33554432, 67108863]", 2024-04-07T17:51:27.294 INFO:tasks.workunit.client.0.smithi100.stdout: "[67108864, 134217727]", 2024-04-07T17:51:27.294 INFO:tasks.workunit.client.0.smithi100.stdout: "[134217728, 268435455]", 2024-04-07T17:51:27.294 INFO:tasks.workunit.client.0.smithi100.stdout: "[268435456, 536870911]", 2024-04-07T17:51:27.294 INFO:tasks.workunit.client.0.smithi100.stdout: "[536870912, 1073741823]", 2024-04-07T17:51:27.294 INFO:tasks.workunit.client.0.smithi100.stdout: "[1073741824, 2147483647]", 2024-04-07T17:51:27.294 INFO:tasks.workunit.client.0.smithi100.stdout: "[2147483648, 4294967295]", 2024-04-07T17:51:27.294 INFO:tasks.workunit.client.0.smithi100.stdout: "[4294967296, 8589934591]", 2024-04-07T17:51:27.295 INFO:tasks.workunit.client.0.smithi100.stdout: "[8589934592, 17179869183]", 2024-04-07T17:51:27.295 INFO:tasks.workunit.client.0.smithi100.stdout: "[17179869184, 34359738367]", 2024-04-07T17:51:27.295 INFO:tasks.workunit.client.0.smithi100.stdout: "[34359738368, 68719476735]", 2024-04-07T17:51:27.295 INFO:tasks.workunit.client.0.smithi100.stdout: "[68719476736, 137438953471]", 2024-04-07T17:51:27.295 INFO:tasks.workunit.client.0.smithi100.stdout: "[137438953472, 274877906943]", 2024-04-07T17:51:27.295 INFO:tasks.workunit.client.0.smithi100.stdout: "[274877906944, None]" 2024-04-07T17:51:27.295 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:27.295 INFO:tasks.workunit.client.0.smithi100.stdout: "scale_type": "log2" 2024-04-07T17:51:27.295 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.295 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:27.295 INFO:tasks.workunit.client.0.smithi100.stdout: "osds": [ 2024-04-07T17:51:27.295 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:27.295 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 0, 2024-04-07T17:51:27.295 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:27.296 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.296 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.296 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.296 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.296 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.296 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 416, 12, 5, 6, 1, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.296 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 4275, 54, 74, 123, 377, 516, 419, 4, 11, 5, 0, 0, 3, 3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.296 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 775, 16, 26, 72, 112, 96, 307, 0, 3, 12, 2, 3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.296 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 369, 1, 0, 4, 2, 1, 2, 0, 1, 1, 0, 3, 13, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.296 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 201, 1, 0, 0, 0, 0, 1, 0, 0, 1, 0, 1, 10, 14, 29, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.296 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 88, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 1, 1, 7, 41, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.296 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 6, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 25, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.296 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.296 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.296 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.297 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.297 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.297 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.297 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.297 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.297 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.297 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.297 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.297 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.297 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.297 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.297 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.297 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.297 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.297 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.297 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.298 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:27.298 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:27.298 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.298 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:27.298 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 1, 2024-04-07T17:51:27.298 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:27.298 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.298 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.298 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.298 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.298 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.298 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 463, 210, 9, 10, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.298 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 2623, 76, 25, 59, 122, 1, 2, 2, 0, 0, 0, 1, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.298 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 819, 8, 815, 0, 1, 0, 1, 0, 2, 1, 0, 1, 0, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.298 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 994, 507, 544, 0, 0, 0, 0, 0, 0, 0, 1, 1, 5, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.299 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 844, 647, 73, 0, 0, 0, 0, 1, 0, 0, 1, 0, 5, 7, 15, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.299 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1618, 1450, 103, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 4, 34, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.299 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 291, 394, 108, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 12, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.299 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.299 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.299 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.299 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.299 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.299 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.299 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.299 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.299 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.299 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.299 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.299 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.300 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.300 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.300 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.300 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.300 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.300 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.300 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.300 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:27.300 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:27.300 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.300 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:27.300 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 2, 2024-04-07T17:51:27.300 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:27.300 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.300 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.300 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.301 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.301 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.301 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 248, 5, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.301 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 7869, 12, 0, 0, 0, 2, 5, 3, 9, 2, 0, 2, 3, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.301 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 17697, 0, 1, 0, 1, 0, 0, 1, 2, 11, 0, 2, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.301 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 887, 0, 1, 0, 0, 0, 0, 0, 1, 0, 0, 2, 9, 3, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.301 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 267, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 7, 21, 28, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.301 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 103, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 6, 6, 37, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.301 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 2, 23, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.301 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.301 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.301 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.301 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.301 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.301 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.302 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.302 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.302 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.302 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.302 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.302 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.302 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.302 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.302 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.302 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.302 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.302 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.302 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.302 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:27.303 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:27.303 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.303 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:27.303 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 3, 2024-04-07T17:51:27.303 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:27.303 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.303 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.303 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.303 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.303 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.303 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 280, 1, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.303 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 3748, 33, 1, 0, 5, 1, 5, 4, 1, 0, 0, 0, 3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.303 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 519, 29, 31, 40, 1, 0, 1, 0, 0, 0, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.303 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 290, 227, 3, 37, 0, 0, 0, 0, 0, 0, 0, 2, 3, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.303 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 2130, 1325, 1, 172, 0, 0, 0, 0, 0, 0, 0, 0, 6, 6, 16, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.304 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 2779, 1327, 0, 285, 0, 0, 0, 0, 0, 0, 0, 1, 3, 4, 24, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.304 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 233, 115, 0, 51, 0, 0, 0, 0, 0, 0, 0, 0, 0, 2, 15, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.304 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.304 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.304 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.304 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.304 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.304 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.304 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.304 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.304 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.304 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.304 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.304 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.304 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.304 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.305 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.305 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.305 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.305 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.305 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.305 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:27.305 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:27.305 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.305 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:27.305 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 4, 2024-04-07T17:51:27.305 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:27.305 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.305 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.305 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.305 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.306 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.306 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 113, 11, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.306 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 3576, 127, 33, 67, 137, 233, 86, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.306 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 303, 84, 0, 0, 1, 40, 395, 0, 1, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.306 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 124, 2, 0, 0, 1, 2, 1, 0, 0, 0, 1, 2, 7, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.306 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 73, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 4, 4, 6, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.306 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 25, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 29, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.306 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 16, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.306 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.306 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.306 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.306 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.306 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.306 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.307 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.307 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.307 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.307 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.307 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.307 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.307 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.307 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.307 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.307 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.307 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.307 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.307 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.307 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:27.307 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:27.308 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.308 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:27.308 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 5, 2024-04-07T17:51:27.308 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:27.308 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.308 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.308 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.308 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.308 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.308 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 109, 21, 86, 200, 454, 576, 651, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.308 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 5257, 50, 38, 54, 17, 36, 74, 1, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.308 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 689, 12, 17, 5, 0, 1, 0, 1, 0, 1, 2, 3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.308 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 457, 32, 66, 127, 29, 1, 0, 2, 0, 0, 2, 4, 8, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.308 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 302, 8, 16, 1, 0, 0, 0, 1, 0, 2, 2, 1, 9, 18, 14, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.308 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 96, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 4, 8, 35, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.309 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 18, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.309 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.309 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.309 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.309 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.309 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.309 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.309 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.309 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.309 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.309 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.309 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.309 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.309 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.309 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.309 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.310 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.310 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.310 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.310 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.310 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:27.310 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:27.310 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.310 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:27.310 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 6, 2024-04-07T17:51:27.310 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:27.310 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.310 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.310 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.310 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.311 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.311 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 51, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.311 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 2959, 10, 0, 0, 1, 1, 0, 1, 0, 0, 0, 1, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.311 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 347, 9, 0, 0, 1, 1, 0, 0, 0, 1, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.311 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 267, 64, 0, 0, 0, 0, 1, 0, 1, 2, 0, 3, 7, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.311 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 150, 40, 0, 0, 0, 0, 0, 0, 0, 0, 1, 2, 3, 4, 6, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.311 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 58, 5, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 6, 27, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.311 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 6, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 2, 20, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.311 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.311 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.311 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.311 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.311 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.311 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.311 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.312 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.312 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.312 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.312 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.312 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.312 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.312 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.312 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.312 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.312 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.312 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.312 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.312 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:27.312 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:27.312 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.313 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:27.313 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 7, 2024-04-07T17:51:27.313 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:27.313 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.313 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.313 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.313 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.313 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.313 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 267, 14, 33, 41, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.313 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 4210, 22, 41, 115, 0, 1, 1, 1, 2, 0, 0, 0, 2, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.313 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 6038, 25, 32, 5, 0, 1, 0, 1, 2, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.313 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 292, 5, 2, 0, 0, 0, 0, 0, 0, 1, 0, 2, 4, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.313 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 85, 1, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 2, 13, 6, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.313 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 43, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 2, 5, 28, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.313 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 16, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.314 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.314 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.314 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.314 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.314 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.314 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.314 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.314 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.314 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.314 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.314 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.314 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.314 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.314 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.315 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.315 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.315 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.315 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.315 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.315 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:27.315 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:27.315 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.315 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:27.315 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.315 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_latency_out_bytes_histogram": { 2024-04-07T17:51:27.315 INFO:tasks.workunit.client.0.smithi100.stdout: "axes": [ 2024-04-07T17:51:27.315 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:27.315 INFO:tasks.workunit.client.0.smithi100.stdout: "buckets": 32, 2024-04-07T17:51:27.315 INFO:tasks.workunit.client.0.smithi100.stdout: "min": 0, 2024-04-07T17:51:27.316 INFO:tasks.workunit.client.0.smithi100.stdout: "name": "Latency (usec)", 2024-04-07T17:51:27.316 INFO:tasks.workunit.client.0.smithi100.stdout: "quant_size": 100000, 2024-04-07T17:51:27.316 INFO:tasks.workunit.client.0.smithi100.stdout: "ranges": [ 2024-04-07T17:51:27.316 INFO:tasks.workunit.client.0.smithi100.stdout: "[None, -1]", 2024-04-07T17:51:27.316 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 99999]", 2024-04-07T17:51:27.316 INFO:tasks.workunit.client.0.smithi100.stdout: "[100000, 199999]", 2024-04-07T17:51:27.316 INFO:tasks.workunit.client.0.smithi100.stdout: "[200000, 399999]", 2024-04-07T17:51:27.316 INFO:tasks.workunit.client.0.smithi100.stdout: "[400000, 799999]", 2024-04-07T17:51:27.316 INFO:tasks.workunit.client.0.smithi100.stdout: "[800000, 1599999]", 2024-04-07T17:51:27.316 INFO:tasks.workunit.client.0.smithi100.stdout: "[1600000, 3199999]", 2024-04-07T17:51:27.316 INFO:tasks.workunit.client.0.smithi100.stdout: "[3200000, 6399999]", 2024-04-07T17:51:27.316 INFO:tasks.workunit.client.0.smithi100.stdout: "[6400000, 12799999]", 2024-04-07T17:51:27.316 INFO:tasks.workunit.client.0.smithi100.stdout: "[12800000, 25599999]", 2024-04-07T17:51:27.316 INFO:tasks.workunit.client.0.smithi100.stdout: "[25600000, 51199999]", 2024-04-07T17:51:27.316 INFO:tasks.workunit.client.0.smithi100.stdout: "[51200000, 102399999]", 2024-04-07T17:51:27.317 INFO:tasks.workunit.client.0.smithi100.stdout: "[102400000, 204799999]", 2024-04-07T17:51:27.317 INFO:tasks.workunit.client.0.smithi100.stdout: "[204800000, 409599999]", 2024-04-07T17:51:27.317 INFO:tasks.workunit.client.0.smithi100.stdout: "[409600000, 819199999]", 2024-04-07T17:51:27.317 INFO:tasks.workunit.client.0.smithi100.stdout: "[819200000, 1638399999]", 2024-04-07T17:51:27.317 INFO:tasks.workunit.client.0.smithi100.stdout: "[1638400000, 3276799999]", 2024-04-07T17:51:27.317 INFO:tasks.workunit.client.0.smithi100.stdout: "[3276800000, 6553599999]", 2024-04-07T17:51:27.317 INFO:tasks.workunit.client.0.smithi100.stdout: "[6553600000, 13107199999]", 2024-04-07T17:51:27.317 INFO:tasks.workunit.client.0.smithi100.stdout: "[13107200000, 26214399999]", 2024-04-07T17:51:27.317 INFO:tasks.workunit.client.0.smithi100.stdout: "[26214400000, 52428799999]", 2024-04-07T17:51:27.317 INFO:tasks.workunit.client.0.smithi100.stdout: "[52428800000, 104857599999]", 2024-04-07T17:51:27.317 INFO:tasks.workunit.client.0.smithi100.stdout: "[104857600000, 209715199999]", 2024-04-07T17:51:27.317 INFO:tasks.workunit.client.0.smithi100.stdout: "[209715200000, 419430399999]", 2024-04-07T17:51:27.317 INFO:tasks.workunit.client.0.smithi100.stdout: "[419430400000, 838860799999]", 2024-04-07T17:51:27.317 INFO:tasks.workunit.client.0.smithi100.stdout: "[838860800000, 1677721599999]", 2024-04-07T17:51:27.317 INFO:tasks.workunit.client.0.smithi100.stdout: "[1677721600000, 3355443199999]", 2024-04-07T17:51:27.318 INFO:tasks.workunit.client.0.smithi100.stdout: "[3355443200000, 6710886399999]", 2024-04-07T17:51:27.318 INFO:tasks.workunit.client.0.smithi100.stdout: "[6710886400000, 13421772799999]", 2024-04-07T17:51:27.318 INFO:tasks.workunit.client.0.smithi100.stdout: "[13421772800000, 26843545599999]", 2024-04-07T17:51:27.318 INFO:tasks.workunit.client.0.smithi100.stdout: "[26843545600000, 53687091199999]", 2024-04-07T17:51:27.318 INFO:tasks.workunit.client.0.smithi100.stdout: "[53687091200000, None]" 2024-04-07T17:51:27.318 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:27.318 INFO:tasks.workunit.client.0.smithi100.stdout: "scale_type": "log2" 2024-04-07T17:51:27.318 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.318 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:27.318 INFO:tasks.workunit.client.0.smithi100.stdout: "buckets": 32, 2024-04-07T17:51:27.318 INFO:tasks.workunit.client.0.smithi100.stdout: "min": 0, 2024-04-07T17:51:27.318 INFO:tasks.workunit.client.0.smithi100.stdout: "name": "Request size (bytes)", 2024-04-07T17:51:27.318 INFO:tasks.workunit.client.0.smithi100.stdout: "quant_size": 512, 2024-04-07T17:51:27.318 INFO:tasks.workunit.client.0.smithi100.stdout: "ranges": [ 2024-04-07T17:51:27.318 INFO:tasks.workunit.client.0.smithi100.stdout: "[None, -1]", 2024-04-07T17:51:27.318 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 511]", 2024-04-07T17:51:27.319 INFO:tasks.workunit.client.0.smithi100.stdout: "[512, 1023]", 2024-04-07T17:51:27.319 INFO:tasks.workunit.client.0.smithi100.stdout: "[1024, 2047]", 2024-04-07T17:51:27.319 INFO:tasks.workunit.client.0.smithi100.stdout: "[2048, 4095]", 2024-04-07T17:51:27.319 INFO:tasks.workunit.client.0.smithi100.stdout: "[4096, 8191]", 2024-04-07T17:51:27.319 INFO:tasks.workunit.client.0.smithi100.stdout: "[8192, 16383]", 2024-04-07T17:51:27.319 INFO:tasks.workunit.client.0.smithi100.stdout: "[16384, 32767]", 2024-04-07T17:51:27.319 INFO:tasks.workunit.client.0.smithi100.stdout: "[32768, 65535]", 2024-04-07T17:51:27.319 INFO:tasks.workunit.client.0.smithi100.stdout: "[65536, 131071]", 2024-04-07T17:51:27.319 INFO:tasks.workunit.client.0.smithi100.stdout: "[131072, 262143]", 2024-04-07T17:51:27.319 INFO:tasks.workunit.client.0.smithi100.stdout: "[262144, 524287]", 2024-04-07T17:51:27.319 INFO:tasks.workunit.client.0.smithi100.stdout: "[524288, 1048575]", 2024-04-07T17:51:27.319 INFO:tasks.workunit.client.0.smithi100.stdout: "[1048576, 2097151]", 2024-04-07T17:51:27.319 INFO:tasks.workunit.client.0.smithi100.stdout: "[2097152, 4194303]", 2024-04-07T17:51:27.319 INFO:tasks.workunit.client.0.smithi100.stdout: "[4194304, 8388607]", 2024-04-07T17:51:27.320 INFO:tasks.workunit.client.0.smithi100.stdout: "[8388608, 16777215]", 2024-04-07T17:51:27.320 INFO:tasks.workunit.client.0.smithi100.stdout: "[16777216, 33554431]", 2024-04-07T17:51:27.320 INFO:tasks.workunit.client.0.smithi100.stdout: "[33554432, 67108863]", 2024-04-07T17:51:27.320 INFO:tasks.workunit.client.0.smithi100.stdout: "[67108864, 134217727]", 2024-04-07T17:51:27.320 INFO:tasks.workunit.client.0.smithi100.stdout: "[134217728, 268435455]", 2024-04-07T17:51:27.320 INFO:tasks.workunit.client.0.smithi100.stdout: "[268435456, 536870911]", 2024-04-07T17:51:27.320 INFO:tasks.workunit.client.0.smithi100.stdout: "[536870912, 1073741823]", 2024-04-07T17:51:27.320 INFO:tasks.workunit.client.0.smithi100.stdout: "[1073741824, 2147483647]", 2024-04-07T17:51:27.320 INFO:tasks.workunit.client.0.smithi100.stdout: "[2147483648, 4294967295]", 2024-04-07T17:51:27.320 INFO:tasks.workunit.client.0.smithi100.stdout: "[4294967296, 8589934591]", 2024-04-07T17:51:27.320 INFO:tasks.workunit.client.0.smithi100.stdout: "[8589934592, 17179869183]", 2024-04-07T17:51:27.320 INFO:tasks.workunit.client.0.smithi100.stdout: "[17179869184, 34359738367]", 2024-04-07T17:51:27.320 INFO:tasks.workunit.client.0.smithi100.stdout: "[34359738368, 68719476735]", 2024-04-07T17:51:27.320 INFO:tasks.workunit.client.0.smithi100.stdout: "[68719476736, 137438953471]", 2024-04-07T17:51:27.320 INFO:tasks.workunit.client.0.smithi100.stdout: "[137438953472, 274877906943]", 2024-04-07T17:51:27.321 INFO:tasks.workunit.client.0.smithi100.stdout: "[274877906944, None]" 2024-04-07T17:51:27.321 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:27.321 INFO:tasks.workunit.client.0.smithi100.stdout: "scale_type": "log2" 2024-04-07T17:51:27.321 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.321 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:27.321 INFO:tasks.workunit.client.0.smithi100.stdout: "osds": [ 2024-04-07T17:51:27.321 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:27.321 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 0, 2024-04-07T17:51:27.321 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:27.321 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.321 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.321 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.321 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.321 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.321 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 441, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.322 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 5864, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.322 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1424, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.322 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 398, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.322 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 258, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.322 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 139, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.322 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 32, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.322 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.322 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.322 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.322 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.322 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.322 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.322 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.322 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.322 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.323 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.323 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.323 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.323 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.323 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.323 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.323 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.323 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.323 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.323 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.323 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:27.323 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:27.323 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.323 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:27.323 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 1, 2024-04-07T17:51:27.324 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:27.324 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.324 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.324 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.324 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.324 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.324 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 692, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.324 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 2912, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.324 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1650, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.324 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 2053, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.324 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1593, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.324 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 3210, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.324 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 807, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.324 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.324 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.324 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.325 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.325 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.325 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.325 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.325 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.325 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.325 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.325 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.325 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.325 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.325 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.325 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.325 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.325 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.325 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.325 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.326 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:27.326 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:27.326 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.326 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:27.326 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 2, 2024-04-07T17:51:27.326 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:27.326 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.326 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.326 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.326 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.326 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.326 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 253, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.326 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 7908, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.326 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 17717, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.327 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 904, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.327 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 325, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.327 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 152, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.327 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 28, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.327 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.327 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.327 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.327 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.327 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.327 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.327 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.327 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.327 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.327 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.327 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.328 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.328 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.328 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.328 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.328 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.328 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.328 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.328 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.328 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:27.328 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:27.328 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.328 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:27.328 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 3, 2024-04-07T17:51:27.328 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:27.328 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.329 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.329 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.329 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.329 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.329 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 282, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.329 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 3801, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.329 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 624, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.329 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 564, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.329 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 3656, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.329 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 4423, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.329 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 416, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.329 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.329 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.329 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.329 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.329 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.330 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.330 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.330 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.330 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.330 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.330 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.330 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.330 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.330 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.330 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.330 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.330 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.330 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.330 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.331 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:27.331 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:27.331 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.331 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:27.331 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 4, 2024-04-07T17:51:27.331 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:27.331 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.331 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.331 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.331 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.331 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.331 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 126, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.331 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 4261, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.331 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 825, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.331 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 140, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.331 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 88, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.332 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 55, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.332 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 16, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.332 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.332 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.332 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.332 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.332 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.332 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.332 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.332 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.332 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.332 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.332 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.332 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.332 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.333 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.333 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.333 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.333 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.333 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.333 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.333 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:27.333 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:27.333 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.333 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:27.333 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 5, 2024-04-07T17:51:27.333 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:27.333 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.333 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.333 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.334 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.334 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.334 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 2097, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.334 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 5529, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.334 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 731, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.334 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 730, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.334 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 374, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.334 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 143, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.334 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 20, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.334 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.334 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.334 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.334 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.334 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.335 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.335 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.335 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.335 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.335 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.335 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.335 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.335 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.335 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.335 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.335 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.335 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.335 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.335 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.335 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:27.335 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:27.335 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.336 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:27.336 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 6, 2024-04-07T17:51:27.336 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:27.336 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.336 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.336 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.336 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.336 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.336 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 52, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.336 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 2975, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.336 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 360, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.336 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 346, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.336 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 206, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.336 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 97, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.336 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 28, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.337 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.337 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.337 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.337 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.337 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.337 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.337 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.337 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.337 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.337 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.337 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.337 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.337 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.337 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.337 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.337 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.338 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.338 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.338 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.338 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:27.338 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:27.338 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.338 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:27.338 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 7, 2024-04-07T17:51:27.338 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:27.338 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.338 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.338 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.338 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.338 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.338 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 355, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.339 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 4397, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.339 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 6107, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.339 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 307, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.339 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 109, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.339 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 79, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.339 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 19, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.339 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.339 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.339 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.339 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.339 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.339 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.339 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.339 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.339 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.340 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.340 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.340 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.340 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.340 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.340 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.340 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.340 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.340 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.340 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.340 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:27.340 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:27.340 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.340 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:27.340 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.341 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_latency_in_bytes_histogram": { 2024-04-07T17:51:27.341 INFO:tasks.workunit.client.0.smithi100.stdout: "axes": [ 2024-04-07T17:51:27.341 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:27.341 INFO:tasks.workunit.client.0.smithi100.stdout: "buckets": 32, 2024-04-07T17:51:27.341 INFO:tasks.workunit.client.0.smithi100.stdout: "min": 0, 2024-04-07T17:51:27.341 INFO:tasks.workunit.client.0.smithi100.stdout: "name": "Latency (usec)", 2024-04-07T17:51:27.341 INFO:tasks.workunit.client.0.smithi100.stdout: "quant_size": 100000, 2024-04-07T17:51:27.341 INFO:tasks.workunit.client.0.smithi100.stdout: "ranges": [ 2024-04-07T17:51:27.341 INFO:tasks.workunit.client.0.smithi100.stdout: "[None, -1]", 2024-04-07T17:51:27.341 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 99999]", 2024-04-07T17:51:27.341 INFO:tasks.workunit.client.0.smithi100.stdout: "[100000, 199999]", 2024-04-07T17:51:27.341 INFO:tasks.workunit.client.0.smithi100.stdout: "[200000, 399999]", 2024-04-07T17:51:27.341 INFO:tasks.workunit.client.0.smithi100.stdout: "[400000, 799999]", 2024-04-07T17:51:27.341 INFO:tasks.workunit.client.0.smithi100.stdout: "[800000, 1599999]", 2024-04-07T17:51:27.341 INFO:tasks.workunit.client.0.smithi100.stdout: "[1600000, 3199999]", 2024-04-07T17:51:27.341 INFO:tasks.workunit.client.0.smithi100.stdout: "[3200000, 6399999]", 2024-04-07T17:51:27.341 INFO:tasks.workunit.client.0.smithi100.stdout: "[6400000, 12799999]", 2024-04-07T17:51:27.342 INFO:tasks.workunit.client.0.smithi100.stdout: "[12800000, 25599999]", 2024-04-07T17:51:27.342 INFO:tasks.workunit.client.0.smithi100.stdout: "[25600000, 51199999]", 2024-04-07T17:51:27.342 INFO:tasks.workunit.client.0.smithi100.stdout: "[51200000, 102399999]", 2024-04-07T17:51:27.342 INFO:tasks.workunit.client.0.smithi100.stdout: "[102400000, 204799999]", 2024-04-07T17:51:27.342 INFO:tasks.workunit.client.0.smithi100.stdout: "[204800000, 409599999]", 2024-04-07T17:51:27.342 INFO:tasks.workunit.client.0.smithi100.stdout: "[409600000, 819199999]", 2024-04-07T17:51:27.342 INFO:tasks.workunit.client.0.smithi100.stdout: "[819200000, 1638399999]", 2024-04-07T17:51:27.342 INFO:tasks.workunit.client.0.smithi100.stdout: "[1638400000, 3276799999]", 2024-04-07T17:51:27.342 INFO:tasks.workunit.client.0.smithi100.stdout: "[3276800000, 6553599999]", 2024-04-07T17:51:27.342 INFO:tasks.workunit.client.0.smithi100.stdout: "[6553600000, 13107199999]", 2024-04-07T17:51:27.342 INFO:tasks.workunit.client.0.smithi100.stdout: "[13107200000, 26214399999]", 2024-04-07T17:51:27.342 INFO:tasks.workunit.client.0.smithi100.stdout: "[26214400000, 52428799999]", 2024-04-07T17:51:27.342 INFO:tasks.workunit.client.0.smithi100.stdout: "[52428800000, 104857599999]", 2024-04-07T17:51:27.342 INFO:tasks.workunit.client.0.smithi100.stdout: "[104857600000, 209715199999]", 2024-04-07T17:51:27.343 INFO:tasks.workunit.client.0.smithi100.stdout: "[209715200000, 419430399999]", 2024-04-07T17:51:27.343 INFO:tasks.workunit.client.0.smithi100.stdout: "[419430400000, 838860799999]", 2024-04-07T17:51:27.343 INFO:tasks.workunit.client.0.smithi100.stdout: "[838860800000, 1677721599999]", 2024-04-07T17:51:27.343 INFO:tasks.workunit.client.0.smithi100.stdout: "[1677721600000, 3355443199999]", 2024-04-07T17:51:27.343 INFO:tasks.workunit.client.0.smithi100.stdout: "[3355443200000, 6710886399999]", 2024-04-07T17:51:27.343 INFO:tasks.workunit.client.0.smithi100.stdout: "[6710886400000, 13421772799999]", 2024-04-07T17:51:27.343 INFO:tasks.workunit.client.0.smithi100.stdout: "[13421772800000, 26843545599999]", 2024-04-07T17:51:27.343 INFO:tasks.workunit.client.0.smithi100.stdout: "[26843545600000, 53687091199999]", 2024-04-07T17:51:27.343 INFO:tasks.workunit.client.0.smithi100.stdout: "[53687091200000, None]" 2024-04-07T17:51:27.343 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:27.343 INFO:tasks.workunit.client.0.smithi100.stdout: "scale_type": "log2" 2024-04-07T17:51:27.343 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.343 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:27.343 INFO:tasks.workunit.client.0.smithi100.stdout: "buckets": 32, 2024-04-07T17:51:27.343 INFO:tasks.workunit.client.0.smithi100.stdout: "min": 0, 2024-04-07T17:51:27.343 INFO:tasks.workunit.client.0.smithi100.stdout: "name": "Request size (bytes)", 2024-04-07T17:51:27.344 INFO:tasks.workunit.client.0.smithi100.stdout: "quant_size": 512, 2024-04-07T17:51:27.344 INFO:tasks.workunit.client.0.smithi100.stdout: "ranges": [ 2024-04-07T17:51:27.344 INFO:tasks.workunit.client.0.smithi100.stdout: "[None, -1]", 2024-04-07T17:51:27.344 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 511]", 2024-04-07T17:51:27.344 INFO:tasks.workunit.client.0.smithi100.stdout: "[512, 1023]", 2024-04-07T17:51:27.344 INFO:tasks.workunit.client.0.smithi100.stdout: "[1024, 2047]", 2024-04-07T17:51:27.344 INFO:tasks.workunit.client.0.smithi100.stdout: "[2048, 4095]", 2024-04-07T17:51:27.344 INFO:tasks.workunit.client.0.smithi100.stdout: "[4096, 8191]", 2024-04-07T17:51:27.344 INFO:tasks.workunit.client.0.smithi100.stdout: "[8192, 16383]", 2024-04-07T17:51:27.344 INFO:tasks.workunit.client.0.smithi100.stdout: "[16384, 32767]", 2024-04-07T17:51:27.344 INFO:tasks.workunit.client.0.smithi100.stdout: "[32768, 65535]", 2024-04-07T17:51:27.344 INFO:tasks.workunit.client.0.smithi100.stdout: "[65536, 131071]", 2024-04-07T17:51:27.344 INFO:tasks.workunit.client.0.smithi100.stdout: "[131072, 262143]", 2024-04-07T17:51:27.344 INFO:tasks.workunit.client.0.smithi100.stdout: "[262144, 524287]", 2024-04-07T17:51:27.344 INFO:tasks.workunit.client.0.smithi100.stdout: "[524288, 1048575]", 2024-04-07T17:51:27.345 INFO:tasks.workunit.client.0.smithi100.stdout: "[1048576, 2097151]", 2024-04-07T17:51:27.345 INFO:tasks.workunit.client.0.smithi100.stdout: "[2097152, 4194303]", 2024-04-07T17:51:27.345 INFO:tasks.workunit.client.0.smithi100.stdout: "[4194304, 8388607]", 2024-04-07T17:51:27.345 INFO:tasks.workunit.client.0.smithi100.stdout: "[8388608, 16777215]", 2024-04-07T17:51:27.345 INFO:tasks.workunit.client.0.smithi100.stdout: "[16777216, 33554431]", 2024-04-07T17:51:27.345 INFO:tasks.workunit.client.0.smithi100.stdout: "[33554432, 67108863]", 2024-04-07T17:51:27.345 INFO:tasks.workunit.client.0.smithi100.stdout: "[67108864, 134217727]", 2024-04-07T17:51:27.345 INFO:tasks.workunit.client.0.smithi100.stdout: "[134217728, 268435455]", 2024-04-07T17:51:27.345 INFO:tasks.workunit.client.0.smithi100.stdout: "[268435456, 536870911]", 2024-04-07T17:51:27.345 INFO:tasks.workunit.client.0.smithi100.stdout: "[536870912, 1073741823]", 2024-04-07T17:51:27.345 INFO:tasks.workunit.client.0.smithi100.stdout: "[1073741824, 2147483647]", 2024-04-07T17:51:27.345 INFO:tasks.workunit.client.0.smithi100.stdout: "[2147483648, 4294967295]", 2024-04-07T17:51:27.345 INFO:tasks.workunit.client.0.smithi100.stdout: "[4294967296, 8589934591]", 2024-04-07T17:51:27.345 INFO:tasks.workunit.client.0.smithi100.stdout: "[8589934592, 17179869183]", 2024-04-07T17:51:27.345 INFO:tasks.workunit.client.0.smithi100.stdout: "[17179869184, 34359738367]", 2024-04-07T17:51:27.346 INFO:tasks.workunit.client.0.smithi100.stdout: "[34359738368, 68719476735]", 2024-04-07T17:51:27.346 INFO:tasks.workunit.client.0.smithi100.stdout: "[68719476736, 137438953471]", 2024-04-07T17:51:27.346 INFO:tasks.workunit.client.0.smithi100.stdout: "[137438953472, 274877906943]", 2024-04-07T17:51:27.346 INFO:tasks.workunit.client.0.smithi100.stdout: "[274877906944, None]" 2024-04-07T17:51:27.346 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:27.346 INFO:tasks.workunit.client.0.smithi100.stdout: "scale_type": "log2" 2024-04-07T17:51:27.346 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.346 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:27.346 INFO:tasks.workunit.client.0.smithi100.stdout: "osds": [ 2024-04-07T17:51:27.346 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:27.346 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 0, 2024-04-07T17:51:27.346 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:27.346 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.346 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.347 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.347 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.347 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.347 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 45, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.347 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 604, 18, 1, 2, 6, 0, 3, 3, 6, 3, 4, 1, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.347 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 2693, 8, 3, 0, 4, 3, 9, 15, 24, 57, 124, 51, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.347 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 270, 5, 0, 0, 1, 1, 0, 4, 3, 10, 38, 191, 289, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.347 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 410, 5, 0, 0, 0, 0, 1, 1, 5, 7, 14, 34, 121, 20, 38, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.347 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 566, 0, 0, 0, 0, 0, 0, 0, 0, 2, 3, 6, 14, 38, 152, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.347 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 778, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 4, 29, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.347 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 481, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.347 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 388, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.347 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.347 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 4, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.347 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.347 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.348 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.348 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.348 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.348 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.348 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.348 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.348 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.348 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.348 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.348 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.348 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.348 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.348 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.348 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:27.348 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:27.348 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.348 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:27.349 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 1, 2024-04-07T17:51:27.349 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:27.349 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.349 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.349 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.349 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.349 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.349 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 21, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.349 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 641, 34, 4, 2, 6, 0, 0, 1, 0, 0, 2, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.349 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 3452, 9, 3, 2, 2, 7, 13, 17, 33, 73, 134, 69, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.349 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 133, 5, 0, 0, 2, 1, 2, 1, 3, 6, 19, 211, 422, 3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.349 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 66, 11, 0, 0, 0, 0, 0, 0, 3, 1, 6, 9, 158, 22, 32, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.349 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 191, 32, 0, 0, 0, 0, 0, 0, 0, 0, 6, 9, 24, 39, 105, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.349 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1198, 50, 0, 0, 0, 0, 0, 0, 0, 1, 0, 1, 0, 3, 18, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.349 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1626, 138, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.349 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 662, 17, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.349 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.350 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.350 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.350 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.350 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.350 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.350 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.350 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.350 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.350 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.350 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.350 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.350 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.350 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.350 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.351 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.351 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.351 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:27.351 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:27.351 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.351 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:27.351 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 2, 2024-04-07T17:51:27.351 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:27.351 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.351 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.351 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.351 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.351 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.351 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 22, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.351 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 335, 6, 1, 0, 4, 0, 1, 5, 6, 2, 5, 5, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.351 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 207, 9, 1, 0, 3, 0, 4, 3, 10, 17, 51, 63, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.352 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 199, 14, 0, 0, 0, 1, 4, 6, 13, 13, 79, 92, 13, 6, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.352 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 155, 8, 0, 0, 0, 0, 2, 2, 4, 11, 32, 52, 31, 36, 46, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.352 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 44, 0, 0, 0, 0, 0, 0, 0, 0, 3, 12, 14, 17, 59, 136, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.352 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 6, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 2, 4, 4, 27, 3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.352 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.352 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.352 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.352 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.352 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.352 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.352 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.352 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.352 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.352 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.352 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.353 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.353 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.353 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.353 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.353 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.353 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.353 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.353 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.353 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:27.353 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:27.353 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.353 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:27.353 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 3, 2024-04-07T17:51:27.353 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:27.353 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.353 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.353 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.354 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.354 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.354 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 26, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.354 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 494, 15, 0, 0, 11, 2, 2, 3, 5, 0, 3, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.354 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1353, 2, 2, 0, 3, 3, 5, 7, 20, 29, 59, 33, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.354 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 366, 7, 0, 0, 1, 0, 0, 3, 4, 6, 16, 89, 157, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.354 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 404, 5, 0, 0, 0, 0, 0, 1, 3, 1, 4, 17, 61, 29, 16, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.354 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 330, 0, 0, 0, 0, 0, 1, 0, 2, 1, 0, 4, 21, 46, 72, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.354 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 124, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 5, 3, 13, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.354 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 46, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.354 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.354 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.354 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.354 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.355 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.355 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.355 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.355 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.355 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.355 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.355 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.355 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.355 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.355 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.355 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.355 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.355 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.355 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.355 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:27.356 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:27.356 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.356 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:27.356 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 4, 2024-04-07T17:51:27.356 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:27.356 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.356 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.356 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.356 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.356 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.356 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 16, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.356 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 441, 22, 0, 0, 7, 0, 1, 0, 0, 0, 1, 2, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.356 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1429, 3, 1, 1, 4, 5, 1, 8, 16, 28, 51, 23, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.356 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 545, 0, 0, 0, 0, 2, 3, 3, 3, 3, 26, 99, 129, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.357 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 331, 0, 0, 0, 0, 0, 0, 0, 1, 3, 19, 19, 107, 9, 19, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.357 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 162, 0, 0, 0, 0, 0, 0, 0, 0, 1, 2, 3, 7, 23, 85, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.357 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 151, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 2, 19, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.357 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 71, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.357 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.357 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.357 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.357 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.357 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.357 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.357 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.357 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.357 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.357 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.357 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.358 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.358 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.358 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.358 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.358 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.358 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.358 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.358 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:27.358 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:27.358 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.358 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:27.358 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 5, 2024-04-07T17:51:27.358 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:27.358 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.359 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.359 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.359 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.359 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.359 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 12, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.359 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 550, 9, 0, 0, 5, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.359 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 111, 8, 0, 0, 8, 0, 0, 2, 3, 4, 9, 9, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.359 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 90, 5, 0, 0, 0, 0, 3, 4, 5, 5, 13, 12, 14, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.359 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 41, 4, 0, 0, 0, 0, 0, 0, 0, 3, 4, 15, 16, 15, 20, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.359 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 13, 0, 0, 0, 0, 0, 0, 1, 0, 0, 6, 4, 6, 42, 128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.359 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 3, 26, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.359 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.359 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.359 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.359 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.360 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.360 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.360 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.360 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.360 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.360 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.360 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.360 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.360 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.360 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.360 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.360 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.360 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.360 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.360 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.361 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:27.361 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:27.361 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.361 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:27.361 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 6, 2024-04-07T17:51:27.361 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:27.361 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.361 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.361 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.361 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.361 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.361 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 11, 0, 0, 1, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.361 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 423, 33, 0, 0, 16, 0, 2, 2, 1, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.361 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1503, 6, 1, 2, 5, 0, 4, 10, 23, 27, 44, 18, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.362 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 492, 4, 0, 0, 1, 1, 1, 1, 1, 2, 19, 86, 117, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.362 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 382, 9, 0, 0, 0, 0, 0, 0, 1, 1, 4, 7, 121, 9, 15, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.362 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 160, 28, 0, 0, 0, 0, 0, 0, 1, 1, 1, 3, 11, 32, 99, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.362 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 129, 75, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 2, 20, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.362 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 75, 153, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.362 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.362 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.362 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.362 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.362 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.362 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.362 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.362 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.362 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.363 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.363 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.363 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.363 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.363 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.363 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.363 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.363 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.363 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.363 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:27.363 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:27.363 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.363 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:27.363 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 7, 2024-04-07T17:51:27.363 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:27.363 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.364 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.364 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.364 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.364 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.364 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 12, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.364 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 239, 5, 0, 0, 6, 0, 2, 3, 5, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.364 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 61, 4, 0, 0, 2, 1, 1, 2, 3, 12, 8, 13, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.364 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 75, 12, 0, 0, 0, 0, 2, 2, 2, 9, 9, 19, 6, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.364 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 49, 15, 0, 0, 0, 0, 1, 2, 2, 2, 8, 21, 29, 29, 10, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.364 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 13, 33, 0, 0, 0, 0, 1, 0, 0, 2, 2, 4, 15, 67, 122, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.364 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 64, 0, 0, 0, 0, 0, 0, 0, 1, 0, 1, 0, 4, 26, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.365 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 149, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.365 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.365 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.365 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.365 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.365 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.365 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.365 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.365 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.365 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.365 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.365 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.365 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.365 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.365 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.366 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.366 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.366 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.366 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.366 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:27.366 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:27.366 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.366 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:27.366 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.366 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.366 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:27.366 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_resrv_repnum_vs_duration": { 2024-04-07T17:51:27.366 INFO:tasks.workunit.client.0.smithi100.stdout: "axes": [ 2024-04-07T17:51:27.366 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:27.367 INFO:tasks.workunit.client.0.smithi100.stdout: "buckets": 8, 2024-04-07T17:51:27.367 INFO:tasks.workunit.client.0.smithi100.stdout: "min": 0, 2024-04-07T17:51:27.367 INFO:tasks.workunit.client.0.smithi100.stdout: "name": "number of replicas", 2024-04-07T17:51:27.367 INFO:tasks.workunit.client.0.smithi100.stdout: "quant_size": 1, 2024-04-07T17:51:27.367 INFO:tasks.workunit.client.0.smithi100.stdout: "ranges": [ 2024-04-07T17:51:27.367 INFO:tasks.workunit.client.0.smithi100.stdout: "[None, -1]", 2024-04-07T17:51:27.367 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0]", 2024-04-07T17:51:27.367 INFO:tasks.workunit.client.0.smithi100.stdout: "[1, 1]", 2024-04-07T17:51:27.367 INFO:tasks.workunit.client.0.smithi100.stdout: "[2, 2]", 2024-04-07T17:51:27.367 INFO:tasks.workunit.client.0.smithi100.stdout: "[3, 3]", 2024-04-07T17:51:27.367 INFO:tasks.workunit.client.0.smithi100.stdout: "[4, 4]", 2024-04-07T17:51:27.367 INFO:tasks.workunit.client.0.smithi100.stdout: "[5, 5]", 2024-04-07T17:51:27.367 INFO:tasks.workunit.client.0.smithi100.stdout: "[6, None]" 2024-04-07T17:51:27.367 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:27.367 INFO:tasks.workunit.client.0.smithi100.stdout: "scale_type": "linear" 2024-04-07T17:51:27.368 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.368 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:27.368 INFO:tasks.workunit.client.0.smithi100.stdout: "buckets": 10, 2024-04-07T17:51:27.368 INFO:tasks.workunit.client.0.smithi100.stdout: "min": 0, 2024-04-07T17:51:27.368 INFO:tasks.workunit.client.0.smithi100.stdout: "name": "duration", 2024-04-07T17:51:27.368 INFO:tasks.workunit.client.0.smithi100.stdout: "quant_size": 250000, 2024-04-07T17:51:27.368 INFO:tasks.workunit.client.0.smithi100.stdout: "ranges": [ 2024-04-07T17:51:27.368 INFO:tasks.workunit.client.0.smithi100.stdout: "[None, -1]", 2024-04-07T17:51:27.368 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 249999]", 2024-04-07T17:51:27.368 INFO:tasks.workunit.client.0.smithi100.stdout: "[250000, 499999]", 2024-04-07T17:51:27.368 INFO:tasks.workunit.client.0.smithi100.stdout: "[500000, 999999]", 2024-04-07T17:51:27.368 INFO:tasks.workunit.client.0.smithi100.stdout: "[1000000, 1999999]", 2024-04-07T17:51:27.368 INFO:tasks.workunit.client.0.smithi100.stdout: "[2000000, 3999999]", 2024-04-07T17:51:27.368 INFO:tasks.workunit.client.0.smithi100.stdout: "[4000000, 7999999]", 2024-04-07T17:51:27.368 INFO:tasks.workunit.client.0.smithi100.stdout: "[8000000, 15999999]", 2024-04-07T17:51:27.368 INFO:tasks.workunit.client.0.smithi100.stdout: "[16000000, 31999999]", 2024-04-07T17:51:27.369 INFO:tasks.workunit.client.0.smithi100.stdout: "[32000000, None]" 2024-04-07T17:51:27.369 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:27.369 INFO:tasks.workunit.client.0.smithi100.stdout: "scale_type": "log2" 2024-04-07T17:51:27.369 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.369 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:27.369 INFO:tasks.workunit.client.0.smithi100.stdout: "osds": [ 2024-04-07T17:51:27.369 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:27.369 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 0, 2024-04-07T17:51:27.369 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:27.369 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.369 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.369 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.369 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.369 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.369 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.369 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.370 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:27.370 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:27.370 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.370 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:27.370 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 1, 2024-04-07T17:51:27.370 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:27.370 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.370 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.370 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.370 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.370 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.370 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.370 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.371 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:27.371 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:27.371 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.371 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:27.371 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 2, 2024-04-07T17:51:27.371 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:27.371 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.371 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.371 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.371 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.371 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.371 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.371 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.371 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:27.371 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:27.371 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.372 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:27.372 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 3, 2024-04-07T17:51:27.372 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:27.372 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.372 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.372 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.372 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.372 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.372 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.372 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.372 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:27.372 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:27.372 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.372 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:27.372 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 4, 2024-04-07T17:51:27.373 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:27.373 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.373 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.373 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.373 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.373 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.373 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.373 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.379 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:27.379 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:27.379 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.379 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:27.379 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 5, 2024-04-07T17:51:27.379 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:27.379 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.379 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.379 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.379 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.379 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.379 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.379 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.379 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:27.380 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:27.380 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.380 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:27.380 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 6, 2024-04-07T17:51:27.380 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:27.380 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.380 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.380 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.380 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.380 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.380 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.380 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.380 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:27.380 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:27.380 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.381 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:27.381 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 7, 2024-04-07T17:51:27.381 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:27.381 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.381 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.381 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.381 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.381 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.381 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.381 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:27.381 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:27.381 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:27.381 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.381 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:27.381 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.382 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.382 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:27.382 INFO:tasks.workunit.client.0.smithi100.stdout: "perf_counters": { 2024-04-07T17:51:27.382 INFO:tasks.workunit.client.0.smithi100.stdout: "mds.f47a4ff15a45950a63160265564a123fc5da5e5e": { 2024-04-07T17:51:27.382 INFO:tasks.workunit.client.0.smithi100.stdout: "mds": { 2024-04-07T17:51:27.382 INFO:tasks.workunit.client.0.smithi100.stdout: "caps": { 2024-04-07T17:51:27.382 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.382 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.382 INFO:tasks.workunit.client.0.smithi100.stdout: "ceph_cap_op_flush_ack": { 2024-04-07T17:51:27.382 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.382 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.382 INFO:tasks.workunit.client.0.smithi100.stdout: "ceph_cap_op_flushsnap_ack": { 2024-04-07T17:51:27.382 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.382 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.383 INFO:tasks.workunit.client.0.smithi100.stdout: "ceph_cap_op_grant": { 2024-04-07T17:51:27.383 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.383 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.383 INFO:tasks.workunit.client.0.smithi100.stdout: "ceph_cap_op_revoke": { 2024-04-07T17:51:27.383 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.383 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.383 INFO:tasks.workunit.client.0.smithi100.stdout: "ceph_cap_op_trunc": { 2024-04-07T17:51:27.383 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.383 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.383 INFO:tasks.workunit.client.0.smithi100.stdout: "dir_commit": { 2024-04-07T17:51:27.383 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.383 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.383 INFO:tasks.workunit.client.0.smithi100.stdout: "dir_fetch_complete": { 2024-04-07T17:51:27.383 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 12 2024-04-07T17:51:27.383 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.384 INFO:tasks.workunit.client.0.smithi100.stdout: "dir_fetch_keys": { 2024-04-07T17:51:27.384 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 10 2024-04-07T17:51:27.384 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.384 INFO:tasks.workunit.client.0.smithi100.stdout: "dir_merge": { 2024-04-07T17:51:27.384 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.384 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.384 INFO:tasks.workunit.client.0.smithi100.stdout: "dir_split": { 2024-04-07T17:51:27.384 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.384 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.384 INFO:tasks.workunit.client.0.smithi100.stdout: "exported_inodes": { 2024-04-07T17:51:27.384 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.384 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.384 INFO:tasks.workunit.client.0.smithi100.stdout: "forward": { 2024-04-07T17:51:27.384 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.384 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.384 INFO:tasks.workunit.client.0.smithi100.stdout: "handle_client_cap_release": { 2024-04-07T17:51:27.385 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.385 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.385 INFO:tasks.workunit.client.0.smithi100.stdout: "handle_client_caps": { 2024-04-07T17:51:27.385 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.385 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.385 INFO:tasks.workunit.client.0.smithi100.stdout: "handle_client_caps_dirty": { 2024-04-07T17:51:27.385 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.385 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.385 INFO:tasks.workunit.client.0.smithi100.stdout: "handle_inode_file_caps": { 2024-04-07T17:51:27.385 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.385 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.385 INFO:tasks.workunit.client.0.smithi100.stdout: "imported_inodes": { 2024-04-07T17:51:27.385 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.385 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.385 INFO:tasks.workunit.client.0.smithi100.stdout: "inodes": { 2024-04-07T17:51:27.386 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 10 2024-04-07T17:51:27.386 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.386 INFO:tasks.workunit.client.0.smithi100.stdout: "inodes_expired": { 2024-04-07T17:51:27.386 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.386 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.386 INFO:tasks.workunit.client.0.smithi100.stdout: "inodes_pinned": { 2024-04-07T17:51:27.386 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 10 2024-04-07T17:51:27.386 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.386 INFO:tasks.workunit.client.0.smithi100.stdout: "inodes_with_caps": { 2024-04-07T17:51:27.386 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.386 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.386 INFO:tasks.workunit.client.0.smithi100.stdout: "load_cent": { 2024-04-07T17:51:27.386 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.386 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.387 INFO:tasks.workunit.client.0.smithi100.stdout: "openino_dir_fetch": { 2024-04-07T17:51:27.387 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.387 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.387 INFO:tasks.workunit.client.0.smithi100.stdout: "process_request_cap_release": { 2024-04-07T17:51:27.387 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.387 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.387 INFO:tasks.workunit.client.0.smithi100.stdout: "reply_latency": { 2024-04-07T17:51:27.387 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:27.387 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.387 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.387 INFO:tasks.workunit.client.0.smithi100.stdout: "request": { 2024-04-07T17:51:27.387 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.387 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.387 INFO:tasks.workunit.client.0.smithi100.stdout: "root_rbytes": { 2024-04-07T17:51:27.387 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.388 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.388 INFO:tasks.workunit.client.0.smithi100.stdout: "root_rfiles": { 2024-04-07T17:51:27.388 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.388 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.388 INFO:tasks.workunit.client.0.smithi100.stdout: "root_rsnaps": { 2024-04-07T17:51:27.388 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.388 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.388 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_reply": { 2024-04-07T17:51:27.388 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.388 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.388 INFO:tasks.workunit.client.0.smithi100.stdout: "subtrees": { 2024-04-07T17:51:27.388 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2 2024-04-07T17:51:27.388 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.388 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.388 INFO:tasks.workunit.client.0.smithi100.stdout: "mds_cache": { 2024-04-07T17:51:27.388 INFO:tasks.workunit.client.0.smithi100.stdout: "ireq_enqueue_scrub": { 2024-04-07T17:51:27.389 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.389 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.389 INFO:tasks.workunit.client.0.smithi100.stdout: "ireq_exportdir": { 2024-04-07T17:51:27.389 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.389 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.389 INFO:tasks.workunit.client.0.smithi100.stdout: "ireq_flush": { 2024-04-07T17:51:27.389 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.389 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.389 INFO:tasks.workunit.client.0.smithi100.stdout: "ireq_fragmentdir": { 2024-04-07T17:51:27.389 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.389 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.389 INFO:tasks.workunit.client.0.smithi100.stdout: "ireq_fragstats": { 2024-04-07T17:51:27.389 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.389 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.389 INFO:tasks.workunit.client.0.smithi100.stdout: "ireq_inodestats": { 2024-04-07T17:51:27.389 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.390 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.390 INFO:tasks.workunit.client.0.smithi100.stdout: "ireq_quiesce_inode": { 2024-04-07T17:51:27.390 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.390 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.390 INFO:tasks.workunit.client.0.smithi100.stdout: "ireq_quiesce_path": { 2024-04-07T17:51:27.390 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.390 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.390 INFO:tasks.workunit.client.0.smithi100.stdout: "num_recovering_enqueued": { 2024-04-07T17:51:27.390 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.390 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.390 INFO:tasks.workunit.client.0.smithi100.stdout: "num_recovering_prioritized": { 2024-04-07T17:51:27.390 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.390 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.390 INFO:tasks.workunit.client.0.smithi100.stdout: "num_recovering_processing": { 2024-04-07T17:51:27.390 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.390 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.391 INFO:tasks.workunit.client.0.smithi100.stdout: "num_strays": { 2024-04-07T17:51:27.391 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.391 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.391 INFO:tasks.workunit.client.0.smithi100.stdout: "num_strays_delayed": { 2024-04-07T17:51:27.391 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.391 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.391 INFO:tasks.workunit.client.0.smithi100.stdout: "num_strays_enqueuing": { 2024-04-07T17:51:27.391 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.391 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.391 INFO:tasks.workunit.client.0.smithi100.stdout: "recovery_completed": { 2024-04-07T17:51:27.391 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.391 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.391 INFO:tasks.workunit.client.0.smithi100.stdout: "recovery_started": { 2024-04-07T17:51:27.391 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.391 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.392 INFO:tasks.workunit.client.0.smithi100.stdout: "strays_created": { 2024-04-07T17:51:27.392 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.392 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.392 INFO:tasks.workunit.client.0.smithi100.stdout: "strays_enqueued": { 2024-04-07T17:51:27.392 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.392 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.392 INFO:tasks.workunit.client.0.smithi100.stdout: "strays_migrated": { 2024-04-07T17:51:27.392 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.392 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.392 INFO:tasks.workunit.client.0.smithi100.stdout: "strays_reintegrated": { 2024-04-07T17:51:27.392 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.392 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.392 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.392 INFO:tasks.workunit.client.0.smithi100.stdout: "mds_log": { 2024-04-07T17:51:27.392 INFO:tasks.workunit.client.0.smithi100.stdout: "ev": { 2024-04-07T17:51:27.392 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 17 2024-04-07T17:51:27.393 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.393 INFO:tasks.workunit.client.0.smithi100.stdout: "evadd": { 2024-04-07T17:51:27.393 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.393 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.393 INFO:tasks.workunit.client.0.smithi100.stdout: "evex": { 2024-04-07T17:51:27.393 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.393 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.393 INFO:tasks.workunit.client.0.smithi100.stdout: "evexd": { 2024-04-07T17:51:27.393 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.393 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.393 INFO:tasks.workunit.client.0.smithi100.stdout: "evexg": { 2024-04-07T17:51:27.393 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.393 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.393 INFO:tasks.workunit.client.0.smithi100.stdout: "evlrg": { 2024-04-07T17:51:27.393 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.394 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.394 INFO:tasks.workunit.client.0.smithi100.stdout: "evtrm": { 2024-04-07T17:51:27.394 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.394 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.394 INFO:tasks.workunit.client.0.smithi100.stdout: "jlat": { 2024-04-07T17:51:27.394 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:27.394 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.394 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.394 INFO:tasks.workunit.client.0.smithi100.stdout: "replayed": { 2024-04-07T17:51:27.394 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 17 2024-04-07T17:51:27.394 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.394 INFO:tasks.workunit.client.0.smithi100.stdout: "seg": { 2024-04-07T17:51:27.394 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1 2024-04-07T17:51:27.394 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.394 INFO:tasks.workunit.client.0.smithi100.stdout: "segadd": { 2024-04-07T17:51:27.395 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.395 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.395 INFO:tasks.workunit.client.0.smithi100.stdout: "segex": { 2024-04-07T17:51:27.395 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.395 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.395 INFO:tasks.workunit.client.0.smithi100.stdout: "segexd": { 2024-04-07T17:51:27.395 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.395 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.395 INFO:tasks.workunit.client.0.smithi100.stdout: "segexg": { 2024-04-07T17:51:27.395 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.395 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.395 INFO:tasks.workunit.client.0.smithi100.stdout: "segmjr": { 2024-04-07T17:51:27.395 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1 2024-04-07T17:51:27.395 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.395 INFO:tasks.workunit.client.0.smithi100.stdout: "segtrm": { 2024-04-07T17:51:27.396 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.396 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.396 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.396 INFO:tasks.workunit.client.0.smithi100.stdout: "mds_mem": { 2024-04-07T17:51:27.396 INFO:tasks.workunit.client.0.smithi100.stdout: "cap": { 2024-04-07T17:51:27.396 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.396 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.396 INFO:tasks.workunit.client.0.smithi100.stdout: "cap+": { 2024-04-07T17:51:27.396 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.396 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.396 INFO:tasks.workunit.client.0.smithi100.stdout: "cap-": { 2024-04-07T17:51:27.396 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.396 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.397 INFO:tasks.workunit.client.0.smithi100.stdout: "dir": { 2024-04-07T17:51:27.397 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 12 2024-04-07T17:51:27.397 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.397 INFO:tasks.workunit.client.0.smithi100.stdout: "dir+": { 2024-04-07T17:51:27.397 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 12 2024-04-07T17:51:27.397 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.397 INFO:tasks.workunit.client.0.smithi100.stdout: "dir-": { 2024-04-07T17:51:27.397 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.397 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.397 INFO:tasks.workunit.client.0.smithi100.stdout: "dn": { 2024-04-07T17:51:27.397 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 10 2024-04-07T17:51:27.397 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.397 INFO:tasks.workunit.client.0.smithi100.stdout: "dn+": { 2024-04-07T17:51:27.397 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 10 2024-04-07T17:51:27.397 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.397 INFO:tasks.workunit.client.0.smithi100.stdout: "dn-": { 2024-04-07T17:51:27.397 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.398 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.398 INFO:tasks.workunit.client.0.smithi100.stdout: "heap": { 2024-04-07T17:51:27.398 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 248184 2024-04-07T17:51:27.398 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.398 INFO:tasks.workunit.client.0.smithi100.stdout: "ino": { 2024-04-07T17:51:27.398 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 13 2024-04-07T17:51:27.398 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.398 INFO:tasks.workunit.client.0.smithi100.stdout: "ino+": { 2024-04-07T17:51:27.398 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 13 2024-04-07T17:51:27.398 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.398 INFO:tasks.workunit.client.0.smithi100.stdout: "ino-": { 2024-04-07T17:51:27.398 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.398 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.398 INFO:tasks.workunit.client.0.smithi100.stdout: "rss": { 2024-04-07T17:51:27.398 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 46296 2024-04-07T17:51:27.399 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.399 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.399 INFO:tasks.workunit.client.0.smithi100.stdout: "mds_server": { 2024-04-07T17:51:27.399 INFO:tasks.workunit.client.0.smithi100.stdout: "cap_acquisition_throttle": { 2024-04-07T17:51:27.399 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.399 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.399 INFO:tasks.workunit.client.0.smithi100.stdout: "cap_revoke_eviction": { 2024-04-07T17:51:27.399 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.399 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.399 INFO:tasks.workunit.client.0.smithi100.stdout: "handle_client_request": { 2024-04-07T17:51:27.399 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.399 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.399 INFO:tasks.workunit.client.0.smithi100.stdout: "handle_client_session": { 2024-04-07T17:51:27.399 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.399 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.399 INFO:tasks.workunit.client.0.smithi100.stdout: "handle_peer_request": { 2024-04-07T17:51:27.399 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.400 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.400 INFO:tasks.workunit.client.0.smithi100.stdout: "req_create_latency": { 2024-04-07T17:51:27.400 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:27.400 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.400 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.400 INFO:tasks.workunit.client.0.smithi100.stdout: "req_getattr_latency": { 2024-04-07T17:51:27.400 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:27.400 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.400 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.400 INFO:tasks.workunit.client.0.smithi100.stdout: "req_getfilelock_latency": { 2024-04-07T17:51:27.400 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:27.400 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.400 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.400 INFO:tasks.workunit.client.0.smithi100.stdout: "req_getvxattr_latency": { 2024-04-07T17:51:27.400 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:27.400 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.401 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.401 INFO:tasks.workunit.client.0.smithi100.stdout: "req_link_latency": { 2024-04-07T17:51:27.401 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:27.401 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.401 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.401 INFO:tasks.workunit.client.0.smithi100.stdout: "req_lookup_latency": { 2024-04-07T17:51:27.401 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:27.401 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.401 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.401 INFO:tasks.workunit.client.0.smithi100.stdout: "req_lookuphash_latency": { 2024-04-07T17:51:27.401 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:27.401 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.401 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.401 INFO:tasks.workunit.client.0.smithi100.stdout: "req_lookupino_latency": { 2024-04-07T17:51:27.401 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:27.401 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.402 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.402 INFO:tasks.workunit.client.0.smithi100.stdout: "req_lookupname_latency": { 2024-04-07T17:51:27.402 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:27.402 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.402 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.402 INFO:tasks.workunit.client.0.smithi100.stdout: "req_lookupparent_latency": { 2024-04-07T17:51:27.402 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:27.402 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.402 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.402 INFO:tasks.workunit.client.0.smithi100.stdout: "req_lookupsnap_latency": { 2024-04-07T17:51:27.402 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:27.402 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.402 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.402 INFO:tasks.workunit.client.0.smithi100.stdout: "req_lssnap_latency": { 2024-04-07T17:51:27.403 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:27.403 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.403 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.403 INFO:tasks.workunit.client.0.smithi100.stdout: "req_mkdir_latency": { 2024-04-07T17:51:27.403 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:27.403 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.403 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.403 INFO:tasks.workunit.client.0.smithi100.stdout: "req_mknod_latency": { 2024-04-07T17:51:27.403 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:27.403 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.403 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.403 INFO:tasks.workunit.client.0.smithi100.stdout: "req_mksnap_latency": { 2024-04-07T17:51:27.403 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:27.403 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.403 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.403 INFO:tasks.workunit.client.0.smithi100.stdout: "req_open_latency": { 2024-04-07T17:51:27.404 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:27.404 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.404 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.404 INFO:tasks.workunit.client.0.smithi100.stdout: "req_readdir_latency": { 2024-04-07T17:51:27.404 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:27.404 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.404 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.404 INFO:tasks.workunit.client.0.smithi100.stdout: "req_rename_latency": { 2024-04-07T17:51:27.404 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:27.404 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.404 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.404 INFO:tasks.workunit.client.0.smithi100.stdout: "req_renamesnap_latency": { 2024-04-07T17:51:27.404 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:27.404 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.404 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.404 INFO:tasks.workunit.client.0.smithi100.stdout: "req_rmdir_latency": { 2024-04-07T17:51:27.405 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:27.405 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.405 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.405 INFO:tasks.workunit.client.0.smithi100.stdout: "req_rmsnap_latency": { 2024-04-07T17:51:27.405 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:27.405 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.405 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.405 INFO:tasks.workunit.client.0.smithi100.stdout: "req_rmxattr_latency": { 2024-04-07T17:51:27.405 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:27.405 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.405 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.405 INFO:tasks.workunit.client.0.smithi100.stdout: "req_setattr_latency": { 2024-04-07T17:51:27.405 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:27.405 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.405 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.406 INFO:tasks.workunit.client.0.smithi100.stdout: "req_setdirlayout_latency": { 2024-04-07T17:51:27.406 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:27.406 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.406 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.406 INFO:tasks.workunit.client.0.smithi100.stdout: "req_setfilelock_latency": { 2024-04-07T17:51:27.406 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:27.406 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.406 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.406 INFO:tasks.workunit.client.0.smithi100.stdout: "req_setlayout_latency": { 2024-04-07T17:51:27.406 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:27.406 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.406 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.406 INFO:tasks.workunit.client.0.smithi100.stdout: "req_setxattr_latency": { 2024-04-07T17:51:27.406 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:27.407 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.407 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.407 INFO:tasks.workunit.client.0.smithi100.stdout: "req_snapdiff_latency": { 2024-04-07T17:51:27.407 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:27.407 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.407 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.407 INFO:tasks.workunit.client.0.smithi100.stdout: "req_symlink_latency": { 2024-04-07T17:51:27.407 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:27.407 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.407 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.407 INFO:tasks.workunit.client.0.smithi100.stdout: "req_unlink_latency": { 2024-04-07T17:51:27.407 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:27.407 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.407 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.407 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.408 INFO:tasks.workunit.client.0.smithi100.stdout: "mds_sessions": { 2024-04-07T17:51:27.408 INFO:tasks.workunit.client.0.smithi100.stdout: "average_load": { 2024-04-07T17:51:27.408 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.408 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.408 INFO:tasks.workunit.client.0.smithi100.stdout: "avg_session_uptime": { 2024-04-07T17:51:27.408 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.408 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.408 INFO:tasks.workunit.client.0.smithi100.stdout: "mdthresh_evicted": { 2024-04-07T17:51:27.408 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.408 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.408 INFO:tasks.workunit.client.0.smithi100.stdout: "session_add": { 2024-04-07T17:51:27.408 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 8 2024-04-07T17:51:27.408 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.408 INFO:tasks.workunit.client.0.smithi100.stdout: "session_count": { 2024-04-07T17:51:27.408 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.409 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.409 INFO:tasks.workunit.client.0.smithi100.stdout: "session_remove": { 2024-04-07T17:51:27.409 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 8 2024-04-07T17:51:27.409 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.409 INFO:tasks.workunit.client.0.smithi100.stdout: "sessions_open": { 2024-04-07T17:51:27.409 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2 2024-04-07T17:51:27.409 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.409 INFO:tasks.workunit.client.0.smithi100.stdout: "sessions_stale": { 2024-04-07T17:51:27.409 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.409 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.409 INFO:tasks.workunit.client.0.smithi100.stdout: "total_load": { 2024-04-07T17:51:27.409 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.409 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.409 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.409 INFO:tasks.workunit.client.0.smithi100.stdout: "objecter": { 2024-04-07T17:51:27.410 INFO:tasks.workunit.client.0.smithi100.stdout: "op_active": { 2024-04-07T17:51:27.410 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.410 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.410 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r": { 2024-04-07T17:51:27.410 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 37 2024-04-07T17:51:27.410 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.410 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rmw": { 2024-04-07T17:51:27.410 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.410 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.410 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w": { 2024-04-07T17:51:27.410 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 7 2024-04-07T17:51:27.410 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.410 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.410 INFO:tasks.workunit.client.0.smithi100.stdout: "purge_queue": { 2024-04-07T17:51:27.410 INFO:tasks.workunit.client.0.smithi100.stdout: "pq_executed": { 2024-04-07T17:51:27.411 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.411 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.411 INFO:tasks.workunit.client.0.smithi100.stdout: "pq_executed_ops": { 2024-04-07T17:51:27.411 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.411 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.411 INFO:tasks.workunit.client.0.smithi100.stdout: "pq_executing": { 2024-04-07T17:51:27.411 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.411 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.411 INFO:tasks.workunit.client.0.smithi100.stdout: "pq_executing_high_water": { 2024-04-07T17:51:27.411 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.411 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.411 INFO:tasks.workunit.client.0.smithi100.stdout: "pq_executing_ops": { 2024-04-07T17:51:27.411 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.411 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.411 INFO:tasks.workunit.client.0.smithi100.stdout: "pq_executing_ops_high_water": { 2024-04-07T17:51:27.411 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.412 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.412 INFO:tasks.workunit.client.0.smithi100.stdout: "pq_item_in_journal": { 2024-04-07T17:51:27.412 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.412 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.412 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.412 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.412 INFO:tasks.workunit.client.0.smithi100.stdout: "mon.8dd0baa036f0a7e6d1f3bdce86136fe527836c3d": { 2024-04-07T17:51:27.412 INFO:tasks.workunit.client.0.smithi100.stdout: "mon": { 2024-04-07T17:51:27.412 INFO:tasks.workunit.client.0.smithi100.stdout: "election_call": { 2024-04-07T17:51:27.412 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2 2024-04-07T17:51:27.412 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.412 INFO:tasks.workunit.client.0.smithi100.stdout: "election_lose": { 2024-04-07T17:51:27.412 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3 2024-04-07T17:51:27.412 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.412 INFO:tasks.workunit.client.0.smithi100.stdout: "election_win": { 2024-04-07T17:51:27.412 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.413 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.413 INFO:tasks.workunit.client.0.smithi100.stdout: "num_elections": { 2024-04-07T17:51:27.413 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 12 2024-04-07T17:51:27.413 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.413 INFO:tasks.workunit.client.0.smithi100.stdout: "num_sessions": { 2024-04-07T17:51:27.413 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 7 2024-04-07T17:51:27.413 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.413 INFO:tasks.workunit.client.0.smithi100.stdout: "session_add": { 2024-04-07T17:51:27.413 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4 2024-04-07T17:51:27.413 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.413 INFO:tasks.workunit.client.0.smithi100.stdout: "session_rm": { 2024-04-07T17:51:27.413 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 253 2024-04-07T17:51:27.413 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.413 INFO:tasks.workunit.client.0.smithi100.stdout: "session_trim": { 2024-04-07T17:51:27.413 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.414 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.414 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.414 INFO:tasks.workunit.client.0.smithi100.stdout: "paxos": { 2024-04-07T17:51:27.414 INFO:tasks.workunit.client.0.smithi100.stdout: "accept_timeout": { 2024-04-07T17:51:27.414 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.414 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.414 INFO:tasks.workunit.client.0.smithi100.stdout: "begin": { 2024-04-07T17:51:27.414 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 8797 2024-04-07T17:51:27.414 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.414 INFO:tasks.workunit.client.0.smithi100.stdout: "begin_bytes": { 2024-04-07T17:51:27.414 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 8797, 2024-04-07T17:51:27.414 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 59268356 2024-04-07T17:51:27.414 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.414 INFO:tasks.workunit.client.0.smithi100.stdout: "begin_keys": { 2024-04-07T17:51:27.415 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:27.415 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.415 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.415 INFO:tasks.workunit.client.0.smithi100.stdout: "begin_latency": { 2024-04-07T17:51:27.415 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 8797, 2024-04-07T17:51:27.415 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3095913350 2024-04-07T17:51:27.415 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.415 INFO:tasks.workunit.client.0.smithi100.stdout: "collect": { 2024-04-07T17:51:27.415 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3 2024-04-07T17:51:27.415 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.415 INFO:tasks.workunit.client.0.smithi100.stdout: "collect_bytes": { 2024-04-07T17:51:27.415 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 3, 2024-04-07T17:51:27.415 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 219 2024-04-07T17:51:27.415 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.415 INFO:tasks.workunit.client.0.smithi100.stdout: "collect_keys": { 2024-04-07T17:51:27.415 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 3, 2024-04-07T17:51:27.416 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3 2024-04-07T17:51:27.416 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.416 INFO:tasks.workunit.client.0.smithi100.stdout: "collect_latency": { 2024-04-07T17:51:27.416 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 3, 2024-04-07T17:51:27.416 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.416 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.416 INFO:tasks.workunit.client.0.smithi100.stdout: "collect_timeout": { 2024-04-07T17:51:27.416 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.416 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.416 INFO:tasks.workunit.client.0.smithi100.stdout: "collect_uncommitted": { 2024-04-07T17:51:27.416 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1 2024-04-07T17:51:27.416 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.416 INFO:tasks.workunit.client.0.smithi100.stdout: "commit": { 2024-04-07T17:51:27.416 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 8798 2024-04-07T17:51:27.416 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.417 INFO:tasks.workunit.client.0.smithi100.stdout: "commit_bytes": { 2024-04-07T17:51:27.417 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:27.417 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.417 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.417 INFO:tasks.workunit.client.0.smithi100.stdout: "commit_keys": { 2024-04-07T17:51:27.417 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:27.417 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.417 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.417 INFO:tasks.workunit.client.0.smithi100.stdout: "commit_latency": { 2024-04-07T17:51:27.417 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:27.417 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.417 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.417 INFO:tasks.workunit.client.0.smithi100.stdout: "lease_ack_timeout": { 2024-04-07T17:51:27.417 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.417 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.417 INFO:tasks.workunit.client.0.smithi100.stdout: "lease_timeout": { 2024-04-07T17:51:27.418 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.418 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.418 INFO:tasks.workunit.client.0.smithi100.stdout: "new_pn": { 2024-04-07T17:51:27.418 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.418 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.418 INFO:tasks.workunit.client.0.smithi100.stdout: "new_pn_latency": { 2024-04-07T17:51:27.418 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:27.418 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.418 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.418 INFO:tasks.workunit.client.0.smithi100.stdout: "refresh": { 2024-04-07T17:51:27.418 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 8798 2024-04-07T17:51:27.418 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.418 INFO:tasks.workunit.client.0.smithi100.stdout: "refresh_latency": { 2024-04-07T17:51:27.419 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 8798, 2024-04-07T17:51:27.419 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 52838506301 2024-04-07T17:51:27.419 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.419 INFO:tasks.workunit.client.0.smithi100.stdout: "restart": { 2024-04-07T17:51:27.419 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 14 2024-04-07T17:51:27.419 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.419 INFO:tasks.workunit.client.0.smithi100.stdout: "share_state": { 2024-04-07T17:51:27.419 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.419 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.419 INFO:tasks.workunit.client.0.smithi100.stdout: "share_state_bytes": { 2024-04-07T17:51:27.419 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:27.419 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.419 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.419 INFO:tasks.workunit.client.0.smithi100.stdout: "share_state_keys": { 2024-04-07T17:51:27.419 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:27.419 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.420 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.420 INFO:tasks.workunit.client.0.smithi100.stdout: "start_leader": { 2024-04-07T17:51:27.420 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.420 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.420 INFO:tasks.workunit.client.0.smithi100.stdout: "start_peon": { 2024-04-07T17:51:27.420 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3 2024-04-07T17:51:27.420 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.420 INFO:tasks.workunit.client.0.smithi100.stdout: "store_state": { 2024-04-07T17:51:27.420 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 8797 2024-04-07T17:51:27.420 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.420 INFO:tasks.workunit.client.0.smithi100.stdout: "store_state_bytes": { 2024-04-07T17:51:27.420 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 8797, 2024-04-07T17:51:27.420 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 116970414 2024-04-07T17:51:27.420 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.420 INFO:tasks.workunit.client.0.smithi100.stdout: "store_state_keys": { 2024-04-07T17:51:27.421 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 8797, 2024-04-07T17:51:27.421 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 76589 2024-04-07T17:51:27.421 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.421 INFO:tasks.workunit.client.0.smithi100.stdout: "store_state_latency": { 2024-04-07T17:51:27.421 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 8797, 2024-04-07T17:51:27.421 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5995831139 2024-04-07T17:51:27.421 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.421 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.421 INFO:tasks.workunit.client.0.smithi100.stdout: "prioritycache": { 2024-04-07T17:51:27.421 INFO:tasks.workunit.client.0.smithi100.stdout: "cache_bytes": { 2024-04-07T17:51:27.421 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1020054731 2024-04-07T17:51:27.421 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.421 INFO:tasks.workunit.client.0.smithi100.stdout: "heap_bytes": { 2024-04-07T17:51:27.421 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 166682624 2024-04-07T17:51:27.421 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.421 INFO:tasks.workunit.client.0.smithi100.stdout: "mapped_bytes": { 2024-04-07T17:51:27.421 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 165036032 2024-04-07T17:51:27.422 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.422 INFO:tasks.workunit.client.0.smithi100.stdout: "target_bytes": { 2024-04-07T17:51:27.422 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2147483648 2024-04-07T17:51:27.422 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.422 INFO:tasks.workunit.client.0.smithi100.stdout: "unmapped_bytes": { 2024-04-07T17:51:27.422 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1646592 2024-04-07T17:51:27.422 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.422 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.422 INFO:tasks.workunit.client.0.smithi100.stdout: "prioritycache:full": { 2024-04-07T17:51:27.422 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:27.422 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 360710144 2024-04-07T17:51:27.422 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.422 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:27.422 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.423 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.423 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:27.423 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.423 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.423 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:27.423 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 267782252 2024-04-07T17:51:27.423 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.423 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:27.423 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 23512176 2024-04-07T17:51:27.423 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.423 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:27.423 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.423 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.423 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:27.423 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.424 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.424 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:27.424 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.424 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.424 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:27.424 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.424 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.424 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:27.424 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.424 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.424 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:27.424 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.424 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.424 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:27.424 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.425 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.425 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:27.425 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.425 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.425 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:27.425 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 69415716 2024-04-07T17:51:27.425 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.425 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.425 INFO:tasks.workunit.client.0.smithi100.stdout: "prioritycache:inc": { 2024-04-07T17:51:27.425 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:27.425 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 339738624 2024-04-07T17:51:27.425 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.425 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:27.425 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.425 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.425 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:27.425 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.426 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.426 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:27.426 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 267782252 2024-04-07T17:51:27.426 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.426 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:27.426 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1438180 2024-04-07T17:51:27.426 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.426 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:27.426 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.426 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.426 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:27.426 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.426 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.426 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:27.427 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.427 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.427 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:27.427 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.427 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.427 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:27.427 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.427 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.427 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:27.427 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.427 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.427 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:27.427 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.427 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.427 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:27.427 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.428 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.428 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:27.428 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 70518192 2024-04-07T17:51:27.428 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.428 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.428 INFO:tasks.workunit.client.0.smithi100.stdout: "prioritycache:kv": { 2024-04-07T17:51:27.428 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:27.428 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 314572800 2024-04-07T17:51:27.428 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.428 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:27.428 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 67108864 2024-04-07T17:51:27.428 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.428 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:27.428 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.428 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.428 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:27.428 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 178521501 2024-04-07T17:51:27.429 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.429 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:27.429 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.429 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.429 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:27.429 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.429 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.429 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:27.429 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.429 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.429 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:27.429 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.429 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.429 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:27.429 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.430 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.430 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:27.430 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.430 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.430 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:27.430 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.430 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.430 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:27.430 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.430 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.430 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:27.430 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.430 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.430 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:27.430 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 68942435 2024-04-07T17:51:27.431 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.431 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.431 INFO:tasks.workunit.client.0.smithi100.stdout: "rocksdb": { 2024-04-07T17:51:27.431 INFO:tasks.workunit.client.0.smithi100.stdout: "compact": { 2024-04-07T17:51:27.431 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.431 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.431 INFO:tasks.workunit.client.0.smithi100.stdout: "compact_queue_len": { 2024-04-07T17:51:27.431 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.431 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.431 INFO:tasks.workunit.client.0.smithi100.stdout: "compact_queue_merge": { 2024-04-07T17:51:27.431 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.431 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.431 INFO:tasks.workunit.client.0.smithi100.stdout: "compact_range": { 2024-04-07T17:51:27.431 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 105 2024-04-07T17:51:27.431 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.431 INFO:tasks.workunit.client.0.smithi100.stdout: "get_latency": { 2024-04-07T17:51:27.431 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 380867, 2024-04-07T17:51:27.432 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4241508762 2024-04-07T17:51:27.432 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.432 INFO:tasks.workunit.client.0.smithi100.stdout: "rocksdb_write_delay_time": { 2024-04-07T17:51:27.432 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:27.432 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.432 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.432 INFO:tasks.workunit.client.0.smithi100.stdout: "rocksdb_write_memtable_time": { 2024-04-07T17:51:27.432 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:27.432 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.432 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.432 INFO:tasks.workunit.client.0.smithi100.stdout: "rocksdb_write_pre_and_post_time": { 2024-04-07T17:51:27.432 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:27.432 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.432 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.432 INFO:tasks.workunit.client.0.smithi100.stdout: "rocksdb_write_wal_time": { 2024-04-07T17:51:27.432 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:27.432 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.433 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.433 INFO:tasks.workunit.client.0.smithi100.stdout: "submit_latency": { 2024-04-07T17:51:27.433 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:27.433 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.433 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.433 INFO:tasks.workunit.client.0.smithi100.stdout: "submit_sync_latency": { 2024-04-07T17:51:27.433 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 21106, 2024-04-07T17:51:27.433 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 10897184086 2024-04-07T17:51:27.433 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.433 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.433 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.433 INFO:tasks.workunit.client.0.smithi100.stdout: "mon.a52fc0d8acb515431879057649ff294cfe14c620": { 2024-04-07T17:51:27.433 INFO:tasks.workunit.client.0.smithi100.stdout: "mon": { 2024-04-07T17:51:27.433 INFO:tasks.workunit.client.0.smithi100.stdout: "election_call": { 2024-04-07T17:51:27.434 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2 2024-04-07T17:51:27.434 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.434 INFO:tasks.workunit.client.0.smithi100.stdout: "election_lose": { 2024-04-07T17:51:27.434 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2 2024-04-07T17:51:27.434 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.434 INFO:tasks.workunit.client.0.smithi100.stdout: "election_win": { 2024-04-07T17:51:27.434 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.434 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.434 INFO:tasks.workunit.client.0.smithi100.stdout: "num_elections": { 2024-04-07T17:51:27.434 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 7 2024-04-07T17:51:27.434 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.434 INFO:tasks.workunit.client.0.smithi100.stdout: "num_sessions": { 2024-04-07T17:51:27.434 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 8 2024-04-07T17:51:27.434 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.434 INFO:tasks.workunit.client.0.smithi100.stdout: "session_add": { 2024-04-07T17:51:27.435 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3 2024-04-07T17:51:27.435 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.435 INFO:tasks.workunit.client.0.smithi100.stdout: "session_rm": { 2024-04-07T17:51:27.435 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 253 2024-04-07T17:51:27.435 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.435 INFO:tasks.workunit.client.0.smithi100.stdout: "session_trim": { 2024-04-07T17:51:27.435 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.435 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.435 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.435 INFO:tasks.workunit.client.0.smithi100.stdout: "paxos": { 2024-04-07T17:51:27.435 INFO:tasks.workunit.client.0.smithi100.stdout: "accept_timeout": { 2024-04-07T17:51:27.435 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.435 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.435 INFO:tasks.workunit.client.0.smithi100.stdout: "begin": { 2024-04-07T17:51:27.435 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 8746 2024-04-07T17:51:27.436 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.436 INFO:tasks.workunit.client.0.smithi100.stdout: "begin_bytes": { 2024-04-07T17:51:27.436 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 8746, 2024-04-07T17:51:27.436 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 58929605 2024-04-07T17:51:27.436 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.436 INFO:tasks.workunit.client.0.smithi100.stdout: "begin_keys": { 2024-04-07T17:51:27.436 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:27.436 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.436 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.436 INFO:tasks.workunit.client.0.smithi100.stdout: "begin_latency": { 2024-04-07T17:51:27.436 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 8746, 2024-04-07T17:51:27.436 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2547954472 2024-04-07T17:51:27.436 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.436 INFO:tasks.workunit.client.0.smithi100.stdout: "collect": { 2024-04-07T17:51:27.436 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2 2024-04-07T17:51:27.437 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.437 INFO:tasks.workunit.client.0.smithi100.stdout: "collect_bytes": { 2024-04-07T17:51:27.437 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 2, 2024-04-07T17:51:27.437 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 146 2024-04-07T17:51:27.437 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.437 INFO:tasks.workunit.client.0.smithi100.stdout: "collect_keys": { 2024-04-07T17:51:27.437 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 2, 2024-04-07T17:51:27.437 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2 2024-04-07T17:51:27.437 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.437 INFO:tasks.workunit.client.0.smithi100.stdout: "collect_latency": { 2024-04-07T17:51:27.437 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 2, 2024-04-07T17:51:27.437 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3999909 2024-04-07T17:51:27.437 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.437 INFO:tasks.workunit.client.0.smithi100.stdout: "collect_timeout": { 2024-04-07T17:51:27.437 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.437 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.438 INFO:tasks.workunit.client.0.smithi100.stdout: "collect_uncommitted": { 2024-04-07T17:51:27.438 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.438 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.438 INFO:tasks.workunit.client.0.smithi100.stdout: "commit": { 2024-04-07T17:51:27.438 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 8746 2024-04-07T17:51:27.438 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.438 INFO:tasks.workunit.client.0.smithi100.stdout: "commit_bytes": { 2024-04-07T17:51:27.438 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:27.438 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.438 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.438 INFO:tasks.workunit.client.0.smithi100.stdout: "commit_keys": { 2024-04-07T17:51:27.438 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:27.438 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.438 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.438 INFO:tasks.workunit.client.0.smithi100.stdout: "commit_latency": { 2024-04-07T17:51:27.439 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:27.439 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.439 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.439 INFO:tasks.workunit.client.0.smithi100.stdout: "lease_ack_timeout": { 2024-04-07T17:51:27.439 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.439 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.439 INFO:tasks.workunit.client.0.smithi100.stdout: "lease_timeout": { 2024-04-07T17:51:27.439 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.439 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.439 INFO:tasks.workunit.client.0.smithi100.stdout: "new_pn": { 2024-04-07T17:51:27.439 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.439 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.439 INFO:tasks.workunit.client.0.smithi100.stdout: "new_pn_latency": { 2024-04-07T17:51:27.439 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:27.439 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.439 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.440 INFO:tasks.workunit.client.0.smithi100.stdout: "refresh": { 2024-04-07T17:51:27.440 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 8746 2024-04-07T17:51:27.440 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.440 INFO:tasks.workunit.client.0.smithi100.stdout: "refresh_latency": { 2024-04-07T17:51:27.440 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 8746, 2024-04-07T17:51:27.440 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 48991126475 2024-04-07T17:51:27.440 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.440 INFO:tasks.workunit.client.0.smithi100.stdout: "restart": { 2024-04-07T17:51:27.440 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 9 2024-04-07T17:51:27.440 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.440 INFO:tasks.workunit.client.0.smithi100.stdout: "share_state": { 2024-04-07T17:51:27.440 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.440 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.440 INFO:tasks.workunit.client.0.smithi100.stdout: "share_state_bytes": { 2024-04-07T17:51:27.440 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:27.440 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.441 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.441 INFO:tasks.workunit.client.0.smithi100.stdout: "share_state_keys": { 2024-04-07T17:51:27.441 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:27.441 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.441 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.441 INFO:tasks.workunit.client.0.smithi100.stdout: "start_leader": { 2024-04-07T17:51:27.441 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.441 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.441 INFO:tasks.workunit.client.0.smithi100.stdout: "start_peon": { 2024-04-07T17:51:27.441 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2 2024-04-07T17:51:27.441 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.441 INFO:tasks.workunit.client.0.smithi100.stdout: "store_state": { 2024-04-07T17:51:27.441 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 8746 2024-04-07T17:51:27.441 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.441 INFO:tasks.workunit.client.0.smithi100.stdout: "store_state_bytes": { 2024-04-07T17:51:27.441 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 8746, 2024-04-07T17:51:27.442 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 116192806 2024-04-07T17:51:27.442 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.442 INFO:tasks.workunit.client.0.smithi100.stdout: "store_state_keys": { 2024-04-07T17:51:27.442 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 8746, 2024-04-07T17:51:27.442 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 75755 2024-04-07T17:51:27.442 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.442 INFO:tasks.workunit.client.0.smithi100.stdout: "store_state_latency": { 2024-04-07T17:51:27.442 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 8746, 2024-04-07T17:51:27.442 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3183943365 2024-04-07T17:51:27.442 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.442 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.442 INFO:tasks.workunit.client.0.smithi100.stdout: "prioritycache": { 2024-04-07T17:51:27.442 INFO:tasks.workunit.client.0.smithi100.stdout: "cache_bytes": { 2024-04-07T17:51:27.442 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1020054731 2024-04-07T17:51:27.442 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.443 INFO:tasks.workunit.client.0.smithi100.stdout: "heap_bytes": { 2024-04-07T17:51:27.443 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 164585472 2024-04-07T17:51:27.443 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.443 INFO:tasks.workunit.client.0.smithi100.stdout: "mapped_bytes": { 2024-04-07T17:51:27.443 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 162111488 2024-04-07T17:51:27.443 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.443 INFO:tasks.workunit.client.0.smithi100.stdout: "target_bytes": { 2024-04-07T17:51:27.443 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2147483648 2024-04-07T17:51:27.443 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.443 INFO:tasks.workunit.client.0.smithi100.stdout: "unmapped_bytes": { 2024-04-07T17:51:27.443 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2473984 2024-04-07T17:51:27.443 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.443 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.443 INFO:tasks.workunit.client.0.smithi100.stdout: "prioritycache:full": { 2024-04-07T17:51:27.443 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:27.444 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 360710144 2024-04-07T17:51:27.444 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.444 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:27.444 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.444 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.444 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:27.444 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.444 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.444 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:27.444 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 267855893 2024-04-07T17:51:27.444 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.444 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:27.444 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 23328861 2024-04-07T17:51:27.444 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.444 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:27.444 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.445 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.445 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:27.445 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.445 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.445 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:27.445 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.445 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.445 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:27.445 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.445 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.445 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:27.445 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.445 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.445 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:27.445 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.446 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.446 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:27.446 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.446 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.446 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:27.446 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.446 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.446 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:27.446 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 69525390 2024-04-07T17:51:27.446 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.446 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.446 INFO:tasks.workunit.client.0.smithi100.stdout: "prioritycache:inc": { 2024-04-07T17:51:27.446 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:27.446 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 339738624 2024-04-07T17:51:27.446 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.447 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:27.447 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.447 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.447 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:27.447 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.447 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.447 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:27.447 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 267855893 2024-04-07T17:51:27.447 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.447 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:27.447 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1425119 2024-04-07T17:51:27.447 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.447 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:27.447 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.447 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.447 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:27.448 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.448 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.448 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:27.448 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.448 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.448 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:27.448 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.448 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.448 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:27.448 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.448 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.448 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:27.448 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.448 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.448 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:27.448 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.449 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.449 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:27.449 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.449 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.449 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:27.449 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 70457612 2024-04-07T17:51:27.449 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.449 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.449 INFO:tasks.workunit.client.0.smithi100.stdout: "prioritycache:kv": { 2024-04-07T17:51:27.449 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:27.449 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 314572800 2024-04-07T17:51:27.449 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.449 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:27.449 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 67108864 2024-04-07T17:51:27.449 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.449 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:27.449 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.450 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.450 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:27.450 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 178570595 2024-04-07T17:51:27.450 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.450 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:27.450 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.450 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.450 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:27.450 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.450 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.450 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:27.450 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.450 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.450 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:27.450 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.450 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.451 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:27.451 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.451 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.451 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:27.451 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.451 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.451 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:27.451 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.451 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.451 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:27.451 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.451 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.451 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:27.451 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.451 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.452 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:27.452 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 68893341 2024-04-07T17:51:27.452 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.452 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.452 INFO:tasks.workunit.client.0.smithi100.stdout: "rocksdb": { 2024-04-07T17:51:27.452 INFO:tasks.workunit.client.0.smithi100.stdout: "compact": { 2024-04-07T17:51:27.452 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.452 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.452 INFO:tasks.workunit.client.0.smithi100.stdout: "compact_queue_len": { 2024-04-07T17:51:27.452 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.452 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.452 INFO:tasks.workunit.client.0.smithi100.stdout: "compact_queue_merge": { 2024-04-07T17:51:27.452 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.452 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.452 INFO:tasks.workunit.client.0.smithi100.stdout: "compact_range": { 2024-04-07T17:51:27.452 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 101 2024-04-07T17:51:27.453 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.453 INFO:tasks.workunit.client.0.smithi100.stdout: "get_latency": { 2024-04-07T17:51:27.453 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 378495, 2024-04-07T17:51:27.453 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3968329012 2024-04-07T17:51:27.453 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.453 INFO:tasks.workunit.client.0.smithi100.stdout: "rocksdb_write_delay_time": { 2024-04-07T17:51:27.453 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:27.453 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.453 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.453 INFO:tasks.workunit.client.0.smithi100.stdout: "rocksdb_write_memtable_time": { 2024-04-07T17:51:27.453 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:27.453 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.453 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.453 INFO:tasks.workunit.client.0.smithi100.stdout: "rocksdb_write_pre_and_post_time": { 2024-04-07T17:51:27.453 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:27.454 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.454 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.454 INFO:tasks.workunit.client.0.smithi100.stdout: "rocksdb_write_wal_time": { 2024-04-07T17:51:27.454 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:27.454 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.454 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.454 INFO:tasks.workunit.client.0.smithi100.stdout: "submit_latency": { 2024-04-07T17:51:27.454 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:27.454 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.454 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.454 INFO:tasks.workunit.client.0.smithi100.stdout: "submit_sync_latency": { 2024-04-07T17:51:27.454 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 20972, 2024-04-07T17:51:27.454 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 8057256302 2024-04-07T17:51:27.454 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.455 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.455 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.455 INFO:tasks.workunit.client.0.smithi100.stdout: "mon.f47a4ff15a45950a63160265564a123fc5da5e5e": { 2024-04-07T17:51:27.455 INFO:tasks.workunit.client.0.smithi100.stdout: "mon": { 2024-04-07T17:51:27.455 INFO:tasks.workunit.client.0.smithi100.stdout: "election_call": { 2024-04-07T17:51:27.455 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5 2024-04-07T17:51:27.455 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.455 INFO:tasks.workunit.client.0.smithi100.stdout: "election_lose": { 2024-04-07T17:51:27.455 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.455 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.455 INFO:tasks.workunit.client.0.smithi100.stdout: "election_win": { 2024-04-07T17:51:27.455 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5 2024-04-07T17:51:27.455 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.455 INFO:tasks.workunit.client.0.smithi100.stdout: "num_elections": { 2024-04-07T17:51:27.455 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 17 2024-04-07T17:51:27.456 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.456 INFO:tasks.workunit.client.0.smithi100.stdout: "num_sessions": { 2024-04-07T17:51:27.456 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 11 2024-04-07T17:51:27.456 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.456 INFO:tasks.workunit.client.0.smithi100.stdout: "session_add": { 2024-04-07T17:51:27.456 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5 2024-04-07T17:51:27.456 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.456 INFO:tasks.workunit.client.0.smithi100.stdout: "session_rm": { 2024-04-07T17:51:27.456 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 278 2024-04-07T17:51:27.456 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.456 INFO:tasks.workunit.client.0.smithi100.stdout: "session_trim": { 2024-04-07T17:51:27.456 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.456 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.456 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.456 INFO:tasks.workunit.client.0.smithi100.stdout: "paxos": { 2024-04-07T17:51:27.456 INFO:tasks.workunit.client.0.smithi100.stdout: "accept_timeout": { 2024-04-07T17:51:27.457 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.457 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.457 INFO:tasks.workunit.client.0.smithi100.stdout: "begin": { 2024-04-07T17:51:27.457 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 8891 2024-04-07T17:51:27.457 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.457 INFO:tasks.workunit.client.0.smithi100.stdout: "begin_bytes": { 2024-04-07T17:51:27.457 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 8891, 2024-04-07T17:51:27.457 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 60665253 2024-04-07T17:51:27.457 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.457 INFO:tasks.workunit.client.0.smithi100.stdout: "begin_keys": { 2024-04-07T17:51:27.457 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 8891, 2024-04-07T17:51:27.457 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 26673 2024-04-07T17:51:27.457 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.457 INFO:tasks.workunit.client.0.smithi100.stdout: "begin_latency": { 2024-04-07T17:51:27.457 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 8891, 2024-04-07T17:51:27.457 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2895918242 2024-04-07T17:51:27.458 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.458 INFO:tasks.workunit.client.0.smithi100.stdout: "collect": { 2024-04-07T17:51:27.458 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.458 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.458 INFO:tasks.workunit.client.0.smithi100.stdout: "collect_bytes": { 2024-04-07T17:51:27.458 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:27.458 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.458 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.458 INFO:tasks.workunit.client.0.smithi100.stdout: "collect_keys": { 2024-04-07T17:51:27.458 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:27.458 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.458 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.458 INFO:tasks.workunit.client.0.smithi100.stdout: "collect_latency": { 2024-04-07T17:51:27.458 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:27.459 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.459 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.459 INFO:tasks.workunit.client.0.smithi100.stdout: "collect_timeout": { 2024-04-07T17:51:27.459 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.459 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.459 INFO:tasks.workunit.client.0.smithi100.stdout: "collect_uncommitted": { 2024-04-07T17:51:27.459 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2 2024-04-07T17:51:27.459 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.459 INFO:tasks.workunit.client.0.smithi100.stdout: "commit": { 2024-04-07T17:51:27.459 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 8889 2024-04-07T17:51:27.459 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.459 INFO:tasks.workunit.client.0.smithi100.stdout: "commit_bytes": { 2024-04-07T17:51:27.459 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 8889, 2024-04-07T17:51:27.459 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 59997923 2024-04-07T17:51:27.459 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.460 INFO:tasks.workunit.client.0.smithi100.stdout: "commit_keys": { 2024-04-07T17:51:27.460 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 8889, 2024-04-07T17:51:27.460 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 68421 2024-04-07T17:51:27.460 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.460 INFO:tasks.workunit.client.0.smithi100.stdout: "commit_latency": { 2024-04-07T17:51:27.460 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 8889, 2024-04-07T17:51:27.460 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3946227492 2024-04-07T17:51:27.460 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.460 INFO:tasks.workunit.client.0.smithi100.stdout: "lease_ack_timeout": { 2024-04-07T17:51:27.460 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.460 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.460 INFO:tasks.workunit.client.0.smithi100.stdout: "lease_timeout": { 2024-04-07T17:51:27.460 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.460 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.460 INFO:tasks.workunit.client.0.smithi100.stdout: "new_pn": { 2024-04-07T17:51:27.460 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5 2024-04-07T17:51:27.461 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.461 INFO:tasks.workunit.client.0.smithi100.stdout: "new_pn_latency": { 2024-04-07T17:51:27.461 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 5, 2024-04-07T17:51:27.461 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.461 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.461 INFO:tasks.workunit.client.0.smithi100.stdout: "refresh": { 2024-04-07T17:51:27.461 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 8892 2024-04-07T17:51:27.461 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.461 INFO:tasks.workunit.client.0.smithi100.stdout: "refresh_latency": { 2024-04-07T17:51:27.461 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 8892, 2024-04-07T17:51:27.461 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 56214415654 2024-04-07T17:51:27.461 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.461 INFO:tasks.workunit.client.0.smithi100.stdout: "restart": { 2024-04-07T17:51:27.461 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 19 2024-04-07T17:51:27.461 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.461 INFO:tasks.workunit.client.0.smithi100.stdout: "share_state": { 2024-04-07T17:51:27.462 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2 2024-04-07T17:51:27.462 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.462 INFO:tasks.workunit.client.0.smithi100.stdout: "share_state_bytes": { 2024-04-07T17:51:27.462 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 2, 2024-04-07T17:51:27.462 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 129358 2024-04-07T17:51:27.462 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.462 INFO:tasks.workunit.client.0.smithi100.stdout: "share_state_keys": { 2024-04-07T17:51:27.462 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 2, 2024-04-07T17:51:27.462 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 10 2024-04-07T17:51:27.462 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.462 INFO:tasks.workunit.client.0.smithi100.stdout: "start_leader": { 2024-04-07T17:51:27.462 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5 2024-04-07T17:51:27.462 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.462 INFO:tasks.workunit.client.0.smithi100.stdout: "start_peon": { 2024-04-07T17:51:27.462 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.463 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.463 INFO:tasks.workunit.client.0.smithi100.stdout: "store_state": { 2024-04-07T17:51:27.463 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.463 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.463 INFO:tasks.workunit.client.0.smithi100.stdout: "store_state_bytes": { 2024-04-07T17:51:27.463 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:27.463 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.463 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.463 INFO:tasks.workunit.client.0.smithi100.stdout: "store_state_keys": { 2024-04-07T17:51:27.463 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:27.463 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.463 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.463 INFO:tasks.workunit.client.0.smithi100.stdout: "store_state_latency": { 2024-04-07T17:51:27.463 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:27.463 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.463 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.464 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.464 INFO:tasks.workunit.client.0.smithi100.stdout: "prioritycache": { 2024-04-07T17:51:27.464 INFO:tasks.workunit.client.0.smithi100.stdout: "cache_bytes": { 2024-04-07T17:51:27.464 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1020054731 2024-04-07T17:51:27.464 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.464 INFO:tasks.workunit.client.0.smithi100.stdout: "heap_bytes": { 2024-04-07T17:51:27.464 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 258113536 2024-04-07T17:51:27.464 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.464 INFO:tasks.workunit.client.0.smithi100.stdout: "mapped_bytes": { 2024-04-07T17:51:27.464 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 257466368 2024-04-07T17:51:27.464 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.464 INFO:tasks.workunit.client.0.smithi100.stdout: "target_bytes": { 2024-04-07T17:51:27.464 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2147483648 2024-04-07T17:51:27.464 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.464 INFO:tasks.workunit.client.0.smithi100.stdout: "unmapped_bytes": { 2024-04-07T17:51:27.465 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 647168 2024-04-07T17:51:27.465 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.465 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.465 INFO:tasks.workunit.client.0.smithi100.stdout: "prioritycache:full": { 2024-04-07T17:51:27.465 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:27.465 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 360710144 2024-04-07T17:51:27.465 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.465 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:27.465 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.465 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.465 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:27.465 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.465 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.465 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:27.465 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 267670953 2024-04-07T17:51:27.466 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.466 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:27.466 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 23788534 2024-04-07T17:51:27.466 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.466 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:27.466 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.466 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.466 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:27.466 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.466 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.466 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:27.466 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.466 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.466 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:27.466 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.467 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.467 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:27.467 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.467 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.467 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:27.467 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.467 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.467 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:27.467 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.467 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.467 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:27.467 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.467 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.467 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:27.467 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 69250657 2024-04-07T17:51:27.467 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.468 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.468 INFO:tasks.workunit.client.0.smithi100.stdout: "prioritycache:inc": { 2024-04-07T17:51:27.468 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:27.468 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 339738624 2024-04-07T17:51:27.468 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.468 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:27.468 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.468 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.468 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:27.468 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.468 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.468 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:27.468 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 267670953 2024-04-07T17:51:27.468 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.468 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:27.469 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1458619 2024-04-07T17:51:27.469 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.469 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:27.469 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.469 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.469 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:27.469 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.469 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.469 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:27.469 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.469 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.469 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:27.469 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.469 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.469 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:27.470 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.470 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.470 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:27.470 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.470 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.470 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:27.470 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.470 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.470 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:27.470 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.470 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.470 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:27.470 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 70609052 2024-04-07T17:51:27.471 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.471 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.471 INFO:tasks.workunit.client.0.smithi100.stdout: "prioritycache:kv": { 2024-04-07T17:51:27.471 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:27.471 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 314572800 2024-04-07T17:51:27.471 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.471 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:27.471 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 67108864 2024-04-07T17:51:27.471 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.471 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:27.471 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.471 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.471 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:27.471 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 178447302 2024-04-07T17:51:27.471 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.471 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:27.472 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.472 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.472 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:27.472 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.472 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.472 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:27.472 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.472 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.472 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:27.472 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.472 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.472 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:27.472 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.472 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.472 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:27.473 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.473 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.473 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:27.473 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.473 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.473 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:27.473 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.473 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.473 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:27.473 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.473 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.473 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:27.473 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 69016634 2024-04-07T17:51:27.473 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.473 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.474 INFO:tasks.workunit.client.0.smithi100.stdout: "rocksdb": { 2024-04-07T17:51:27.474 INFO:tasks.workunit.client.0.smithi100.stdout: "compact": { 2024-04-07T17:51:27.474 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.474 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.474 INFO:tasks.workunit.client.0.smithi100.stdout: "compact_queue_len": { 2024-04-07T17:51:27.474 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.474 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.474 INFO:tasks.workunit.client.0.smithi100.stdout: "compact_queue_merge": { 2024-04-07T17:51:27.474 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.474 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.474 INFO:tasks.workunit.client.0.smithi100.stdout: "compact_range": { 2024-04-07T17:51:27.474 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 105 2024-04-07T17:51:27.474 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.474 INFO:tasks.workunit.client.0.smithi100.stdout: "get_latency": { 2024-04-07T17:51:27.474 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 398669, 2024-04-07T17:51:27.475 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4722518373 2024-04-07T17:51:27.475 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.475 INFO:tasks.workunit.client.0.smithi100.stdout: "rocksdb_write_delay_time": { 2024-04-07T17:51:27.475 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:27.475 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.475 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.475 INFO:tasks.workunit.client.0.smithi100.stdout: "rocksdb_write_memtable_time": { 2024-04-07T17:51:27.475 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:27.475 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.475 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.475 INFO:tasks.workunit.client.0.smithi100.stdout: "rocksdb_write_pre_and_post_time": { 2024-04-07T17:51:27.475 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:27.475 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.475 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.475 INFO:tasks.workunit.client.0.smithi100.stdout: "rocksdb_write_wal_time": { 2024-04-07T17:51:27.476 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:27.476 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.476 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.476 INFO:tasks.workunit.client.0.smithi100.stdout: "submit_latency": { 2024-04-07T17:51:27.476 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:27.476 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.476 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.476 INFO:tasks.workunit.client.0.smithi100.stdout: "submit_sync_latency": { 2024-04-07T17:51:27.476 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 21335, 2024-04-07T17:51:27.476 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 8939358365 2024-04-07T17:51:27.476 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.476 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.476 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.476 INFO:tasks.workunit.client.0.smithi100.stdout: "osd.0": { 2024-04-07T17:51:27.476 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs": { 2024-04-07T17:51:27.477 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_db_max_lat": { 2024-04-07T17:51:27.477 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 21463 2024-04-07T17:51:27.477 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.477 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_slow_fallback": { 2024-04-07T17:51:27.477 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.477 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.477 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_slow_max_lat": { 2024-04-07T17:51:27.477 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.477 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.477 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_slow_size_fallback": { 2024-04-07T17:51:27.477 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.477 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.477 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit_db": { 2024-04-07T17:51:27.477 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 65536 2024-04-07T17:51:27.477 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.478 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit_slow": { 2024-04-07T17:51:27.478 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.478 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.478 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit_wal": { 2024-04-07T17:51:27.478 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.478 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.478 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_wal_max_lat": { 2024-04-07T17:51:27.478 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.478 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.478 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_written_slow": { 2024-04-07T17:51:27.478 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.478 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.478 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_written_sst": { 2024-04-07T17:51:27.478 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 14692352 2024-04-07T17:51:27.478 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.479 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_written_wal": { 2024-04-07T17:51:27.479 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 742674432 2024-04-07T17:51:27.479 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.479 INFO:tasks.workunit.client.0.smithi100.stdout: "compact_lat": { 2024-04-07T17:51:27.479 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 19, 2024-04-07T17:51:27.479 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 7885793 2024-04-07T17:51:27.479 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.479 INFO:tasks.workunit.client.0.smithi100.stdout: "compact_lock_lat": { 2024-04-07T17:51:27.479 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 19, 2024-04-07T17:51:27.479 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2646042 2024-04-07T17:51:27.479 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.479 INFO:tasks.workunit.client.0.smithi100.stdout: "db_alloc_lat": { 2024-04-07T17:51:27.479 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 211, 2024-04-07T17:51:27.479 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1477946 2024-04-07T17:51:27.479 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.479 INFO:tasks.workunit.client.0.smithi100.stdout: "db_total_bytes": { 2024-04-07T17:51:27.480 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 95995027456 2024-04-07T17:51:27.480 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.480 INFO:tasks.workunit.client.0.smithi100.stdout: "db_used_bytes": { 2024-04-07T17:51:27.480 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 735182848 2024-04-07T17:51:27.480 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.480 INFO:tasks.workunit.client.0.smithi100.stdout: "flush_lat": { 2024-04-07T17:51:27.480 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 79477, 2024-04-07T17:51:27.480 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 14955834078 2024-04-07T17:51:27.480 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.480 INFO:tasks.workunit.client.0.smithi100.stdout: "fsync_lat": { 2024-04-07T17:51:27.480 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 79485, 2024-04-07T17:51:27.480 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 25864611168 2024-04-07T17:51:27.480 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.480 INFO:tasks.workunit.client.0.smithi100.stdout: "log_bytes": { 2024-04-07T17:51:27.480 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 9666560 2024-04-07T17:51:27.481 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.481 INFO:tasks.workunit.client.0.smithi100.stdout: "logged_bytes": { 2024-04-07T17:51:27.481 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 325562368 2024-04-07T17:51:27.481 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.481 INFO:tasks.workunit.client.0.smithi100.stdout: "max_bytes_db": { 2024-04-07T17:51:27.481 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 747962368 2024-04-07T17:51:27.481 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.481 INFO:tasks.workunit.client.0.smithi100.stdout: "max_bytes_slow": { 2024-04-07T17:51:27.481 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.481 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.481 INFO:tasks.workunit.client.0.smithi100.stdout: "max_bytes_wal": { 2024-04-07T17:51:27.481 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.481 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.481 INFO:tasks.workunit.client.0.smithi100.stdout: "num_files": { 2024-04-07T17:51:27.481 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 55 2024-04-07T17:51:27.482 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.482 INFO:tasks.workunit.client.0.smithi100.stdout: "read_bytes": { 2024-04-07T17:51:27.482 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 13112107291 2024-04-07T17:51:27.482 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.482 INFO:tasks.workunit.client.0.smithi100.stdout: "read_count": { 2024-04-07T17:51:27.482 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 86948 2024-04-07T17:51:27.482 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.482 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes": { 2024-04-07T17:51:27.482 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 11361755136 2024-04-07T17:51:27.482 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.482 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes_db": { 2024-04-07T17:51:27.482 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 11361759232 2024-04-07T17:51:27.482 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.482 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes_slow": { 2024-04-07T17:51:27.482 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.483 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.483 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes_wal": { 2024-04-07T17:51:27.483 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.483 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.483 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_count": { 2024-04-07T17:51:27.483 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 88640 2024-04-07T17:51:27.483 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.483 INFO:tasks.workunit.client.0.smithi100.stdout: "read_lat": { 2024-04-07T17:51:27.483 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 86948, 2024-04-07T17:51:27.483 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5555540195 2024-04-07T17:51:27.483 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.483 INFO:tasks.workunit.client.0.smithi100.stdout: "read_prefetch_bytes": { 2024-04-07T17:51:27.483 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 12971063490 2024-04-07T17:51:27.483 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.483 INFO:tasks.workunit.client.0.smithi100.stdout: "read_prefetch_count": { 2024-04-07T17:51:27.483 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 82627 2024-04-07T17:51:27.483 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.484 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_buffer_bytes": { 2024-04-07T17:51:27.484 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5154118 2024-04-07T17:51:27.484 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.484 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_buffer_count": { 2024-04-07T17:51:27.484 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 723 2024-04-07T17:51:27.484 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.484 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_bytes": { 2024-04-07T17:51:27.484 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 14619250 2024-04-07T17:51:27.484 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.484 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_count": { 2024-04-07T17:51:27.484 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1029 2024-04-07T17:51:27.484 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.484 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes": { 2024-04-07T17:51:27.484 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 9465132 2024-04-07T17:51:27.484 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.484 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes_db": { 2024-04-07T17:51:27.485 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 9465132 2024-04-07T17:51:27.485 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.485 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes_slow": { 2024-04-07T17:51:27.485 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.485 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.485 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes_wal": { 2024-04-07T17:51:27.485 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.485 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.485 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_count": { 2024-04-07T17:51:27.485 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 321 2024-04-07T17:51:27.485 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.485 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_lat": { 2024-04-07T17:51:27.485 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1029, 2024-04-07T17:51:27.485 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 7138854 2024-04-07T17:51:27.485 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.486 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_alloc_lat": { 2024-04-07T17:51:27.486 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:27.486 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.486 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.486 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_total_bytes": { 2024-04-07T17:51:27.486 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.486 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.486 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_used_bytes": { 2024-04-07T17:51:27.486 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.486 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.486 INFO:tasks.workunit.client.0.smithi100.stdout: "truncate_lat": { 2024-04-07T17:51:27.486 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1, 2024-04-07T17:51:27.486 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 593 2024-04-07T17:51:27.486 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.487 INFO:tasks.workunit.client.0.smithi100.stdout: "unlink_lat": { 2024-04-07T17:51:27.487 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 11, 2024-04-07T17:51:27.487 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 14997 2024-04-07T17:51:27.487 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.487 INFO:tasks.workunit.client.0.smithi100.stdout: "wal_alloc_lat": { 2024-04-07T17:51:27.487 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:27.487 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.487 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.487 INFO:tasks.workunit.client.0.smithi100.stdout: "wal_total_bytes": { 2024-04-07T17:51:27.487 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.487 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.487 INFO:tasks.workunit.client.0.smithi100.stdout: "wal_used_bytes": { 2024-04-07T17:51:27.487 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.487 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.487 INFO:tasks.workunit.client.0.smithi100.stdout: "write_bytes": { 2024-04-07T17:51:27.488 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1085816832 2024-04-07T17:51:27.488 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.488 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.488 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore": { 2024-04-07T17:51:27.488 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit": { 2024-04-07T17:51:27.488 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4096 2024-04-07T17:51:27.488 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.488 INFO:tasks.workunit.client.0.smithi100.stdout: "allocated": { 2024-04-07T17:51:27.488 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 10412032 2024-04-07T17:51:27.488 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.488 INFO:tasks.workunit.client.0.smithi100.stdout: "allocator_lat": { 2024-04-07T17:51:27.488 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 23588, 2024-04-07T17:51:27.488 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 122707682 2024-04-07T17:51:27.488 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.488 INFO:tasks.workunit.client.0.smithi100.stdout: "clist_lat": { 2024-04-07T17:51:27.489 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 3594, 2024-04-07T17:51:27.489 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 12445185285 2024-04-07T17:51:27.489 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.489 INFO:tasks.workunit.client.0.smithi100.stdout: "compress_lat": { 2024-04-07T17:51:27.489 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:27.489 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.489 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.489 INFO:tasks.workunit.client.0.smithi100.stdout: "compressed": { 2024-04-07T17:51:27.489 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.489 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.489 INFO:tasks.workunit.client.0.smithi100.stdout: "compressed_allocated": { 2024-04-07T17:51:27.489 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.489 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.489 INFO:tasks.workunit.client.0.smithi100.stdout: "compressed_original": { 2024-04-07T17:51:27.489 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.489 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.490 INFO:tasks.workunit.client.0.smithi100.stdout: "csum_lat": { 2024-04-07T17:51:27.490 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 29739, 2024-04-07T17:51:27.490 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 345936986 2024-04-07T17:51:27.490 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.490 INFO:tasks.workunit.client.0.smithi100.stdout: "decompress_lat": { 2024-04-07T17:51:27.490 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:27.490 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.490 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.490 INFO:tasks.workunit.client.0.smithi100.stdout: "fragmentation_micros": { 2024-04-07T17:51:27.490 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.490 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.490 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_commit_lat": { 2024-04-07T17:51:27.490 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 79403, 2024-04-07T17:51:27.490 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 33855770677 2024-04-07T17:51:27.491 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.491 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_final_lat": { 2024-04-07T17:51:27.491 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 78455, 2024-04-07T17:51:27.491 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3723540743 2024-04-07T17:51:27.491 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.491 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_flush_lat": { 2024-04-07T17:51:27.491 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 79403, 2024-04-07T17:51:27.491 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 104760188 2024-04-07T17:51:27.491 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.491 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_sync_lat": { 2024-04-07T17:51:27.491 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 79403, 2024-04-07T17:51:27.491 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 33960530865 2024-04-07T17:51:27.491 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.491 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_get_keys_lat": { 2024-04-07T17:51:27.491 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:27.491 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.491 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.492 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_get_values_lat": { 2024-04-07T17:51:27.492 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 35913, 2024-04-07T17:51:27.492 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1906667723 2024-04-07T17:51:27.492 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.492 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_lower_bound_lat": { 2024-04-07T17:51:27.492 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 941, 2024-04-07T17:51:27.492 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5755773 2024-04-07T17:51:27.492 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.492 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_next_lat": { 2024-04-07T17:51:27.492 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 24954, 2024-04-07T17:51:27.492 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 21515897 2024-04-07T17:51:27.492 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.492 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_seek_to_first_lat": { 2024-04-07T17:51:27.492 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 52, 2024-04-07T17:51:27.492 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 102213 2024-04-07T17:51:27.492 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.493 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_upper_bound_lat": { 2024-04-07T17:51:27.493 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 16667, 2024-04-07T17:51:27.493 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5746456893 2024-04-07T17:51:27.493 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.493 INFO:tasks.workunit.client.0.smithi100.stdout: "onode_hits": { 2024-04-07T17:51:27.493 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 260492 2024-04-07T17:51:27.493 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.493 INFO:tasks.workunit.client.0.smithi100.stdout: "onode_misses": { 2024-04-07T17:51:27.493 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 24792 2024-04-07T17:51:27.493 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.493 INFO:tasks.workunit.client.0.smithi100.stdout: "read_lat": { 2024-04-07T17:51:27.493 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 26984, 2024-04-07T17:51:27.493 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3801162649 2024-04-07T17:51:27.493 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.493 INFO:tasks.workunit.client.0.smithi100.stdout: "read_onode_meta_lat": { 2024-04-07T17:51:27.494 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 62413, 2024-04-07T17:51:27.494 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 46979091 2024-04-07T17:51:27.494 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.494 INFO:tasks.workunit.client.0.smithi100.stdout: "read_wait_aio_lat": { 2024-04-07T17:51:27.494 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 35429, 2024-04-07T17:51:27.494 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2438344202 2024-04-07T17:51:27.494 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.494 INFO:tasks.workunit.client.0.smithi100.stdout: "reads_with_retries": { 2024-04-07T17:51:27.494 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.494 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.494 INFO:tasks.workunit.client.0.smithi100.stdout: "remove_lat": { 2024-04-07T17:51:27.494 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 15389, 2024-04-07T17:51:27.494 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 819542013 2024-04-07T17:51:27.494 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.495 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_aio_wait_count": { 2024-04-07T17:51:27.495 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.495 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.495 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_committed_kv_count": { 2024-04-07T17:51:27.495 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.495 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.495 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_read_onode_meta_count": { 2024-04-07T17:51:27.495 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.495 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.495 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_read_wait_aio_count": { 2024-04-07T17:51:27.495 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.495 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.495 INFO:tasks.workunit.client.0.smithi100.stdout: "state_aio_wait_lat": { 2024-04-07T17:51:27.495 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 93393, 2024-04-07T17:51:27.495 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 12329349719 2024-04-07T17:51:27.496 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.496 INFO:tasks.workunit.client.0.smithi100.stdout: "state_deferred_aio_wait_lat": { 2024-04-07T17:51:27.496 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 9754, 2024-04-07T17:51:27.496 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1209751248 2024-04-07T17:51:27.496 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.496 INFO:tasks.workunit.client.0.smithi100.stdout: "state_deferred_cleanup_lat": { 2024-04-07T17:51:27.496 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 9753, 2024-04-07T17:51:27.496 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4213165502302 2024-04-07T17:51:27.496 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.496 INFO:tasks.workunit.client.0.smithi100.stdout: "state_deferred_queued_lat": { 2024-04-07T17:51:27.496 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 9754, 2024-04-07T17:51:27.496 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 6800803572839 2024-04-07T17:51:27.496 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.496 INFO:tasks.workunit.client.0.smithi100.stdout: "state_done_lat": { 2024-04-07T17:51:27.496 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 93392, 2024-04-07T17:51:27.497 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2914483870678 2024-04-07T17:51:27.497 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.497 INFO:tasks.workunit.client.0.smithi100.stdout: "state_finishing_lat": { 2024-04-07T17:51:27.497 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 93392, 2024-04-07T17:51:27.497 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 20544100 2024-04-07T17:51:27.497 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.497 INFO:tasks.workunit.client.0.smithi100.stdout: "state_io_done_lat": { 2024-04-07T17:51:27.497 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 93393, 2024-04-07T17:51:27.497 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 589384870 2024-04-07T17:51:27.497 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.497 INFO:tasks.workunit.client.0.smithi100.stdout: "state_kv_commiting_lat": { 2024-04-07T17:51:27.497 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 93393, 2024-04-07T17:51:27.497 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 60184889904 2024-04-07T17:51:27.497 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.497 INFO:tasks.workunit.client.0.smithi100.stdout: "state_kv_done_lat": { 2024-04-07T17:51:27.497 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 93393, 2024-04-07T17:51:27.498 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 69530872 2024-04-07T17:51:27.498 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.498 INFO:tasks.workunit.client.0.smithi100.stdout: "state_kv_queued_lat": { 2024-04-07T17:51:27.498 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 93393, 2024-04-07T17:51:27.498 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 31858207932 2024-04-07T17:51:27.498 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.498 INFO:tasks.workunit.client.0.smithi100.stdout: "state_prepare_lat": { 2024-04-07T17:51:27.498 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 93393, 2024-04-07T17:51:27.498 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 26654236951 2024-04-07T17:51:27.498 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.498 INFO:tasks.workunit.client.0.smithi100.stdout: "stored": { 2024-04-07T17:51:27.498 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 6290778 2024-04-07T17:51:27.498 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.498 INFO:tasks.workunit.client.0.smithi100.stdout: "truncate_lat": { 2024-04-07T17:51:27.498 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 2482, 2024-04-07T17:51:27.499 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 25806717 2024-04-07T17:51:27.499 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.499 INFO:tasks.workunit.client.0.smithi100.stdout: "txc_commit_lat": { 2024-04-07T17:51:27.499 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 93393, 2024-04-07T17:51:27.499 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 131646585667 2024-04-07T17:51:27.499 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.499 INFO:tasks.workunit.client.0.smithi100.stdout: "txc_submit_lat": { 2024-04-07T17:51:27.499 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 93393, 2024-04-07T17:51:27.499 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 29197915299 2024-04-07T17:51:27.499 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.499 INFO:tasks.workunit.client.0.smithi100.stdout: "txc_throttle_lat": { 2024-04-07T17:51:27.499 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 93393, 2024-04-07T17:51:27.499 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1435528401 2024-04-07T17:51:27.499 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.499 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.499 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache": { 2024-04-07T17:51:27.500 INFO:tasks.workunit.client.0.smithi100.stdout: "cache_bytes": { 2024-04-07T17:51:27.500 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1444614092 2024-04-07T17:51:27.500 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.500 INFO:tasks.workunit.client.0.smithi100.stdout: "heap_bytes": { 2024-04-07T17:51:27.500 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 508690432 2024-04-07T17:51:27.500 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.500 INFO:tasks.workunit.client.0.smithi100.stdout: "mapped_bytes": { 2024-04-07T17:51:27.500 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 469770240 2024-04-07T17:51:27.500 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.500 INFO:tasks.workunit.client.0.smithi100.stdout: "target_bytes": { 2024-04-07T17:51:27.500 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2646965248 2024-04-07T17:51:27.500 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.500 INFO:tasks.workunit.client.0.smithi100.stdout: "unmapped_bytes": { 2024-04-07T17:51:27.500 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 38920192 2024-04-07T17:51:27.500 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.500 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.500 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:data": { 2024-04-07T17:51:27.501 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:27.501 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 134217728 2024-04-07T17:51:27.501 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.501 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:27.501 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.501 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.501 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:27.501 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.501 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.501 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:27.501 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 58795597 2024-04-07T17:51:27.501 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.501 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:27.501 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.501 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.501 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:27.502 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.502 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.502 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:27.502 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.502 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.502 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:27.502 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.502 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.502 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:27.502 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4096 2024-04-07T17:51:27.502 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.502 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:27.502 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 438272 2024-04-07T17:51:27.502 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.503 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:27.503 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.503 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.503 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:27.503 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.503 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.503 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:27.503 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.503 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.503 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:27.503 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 74979763 2024-04-07T17:51:27.503 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.503 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.503 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:kv": { 2024-04-07T17:51:27.503 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:27.503 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 578813952 2024-04-07T17:51:27.504 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.504 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:27.504 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 67108864 2024-04-07T17:51:27.504 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.504 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:27.504 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.504 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.504 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:27.504 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 440966984 2024-04-07T17:51:27.504 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.504 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:27.504 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.504 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.504 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:27.504 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.505 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.505 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:27.505 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1776 2024-04-07T17:51:27.505 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.505 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:27.505 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.505 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.505 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:27.505 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 121696 2024-04-07T17:51:27.505 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.505 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:27.505 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.505 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.506 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:27.506 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.506 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.506 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:27.506 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.506 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.506 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:27.506 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.506 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.506 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:27.506 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 70614632 2024-04-07T17:51:27.506 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.506 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.506 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:kv_onode": { 2024-04-07T17:51:27.507 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:27.507 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 192937984 2024-04-07T17:51:27.507 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.507 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:27.507 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 67108864 2024-04-07T17:51:27.507 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.507 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:27.507 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 14485360 2024-04-07T17:51:27.507 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.507 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:27.507 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 39197065 2024-04-07T17:51:27.507 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.507 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:27.507 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.507 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.507 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:27.508 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.508 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.508 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:27.508 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.508 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.508 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:27.508 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.508 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.508 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:27.508 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.508 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.508 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:27.508 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.508 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.508 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:27.509 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.509 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.509 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:27.509 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.509 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.509 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:27.509 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.509 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.509 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:27.509 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 72146695 2024-04-07T17:51:27.509 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.509 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.509 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:meta": { 2024-04-07T17:51:27.509 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:27.509 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 511705088 2024-04-07T17:51:27.509 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.510 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:27.510 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.510 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.510 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:27.510 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.510 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.510 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:27.510 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 440966984 2024-04-07T17:51:27.510 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.510 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:27.510 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.510 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.510 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:27.510 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.511 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.511 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:27.511 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.511 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.511 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:27.511 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.511 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.511 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:27.511 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 157581 2024-04-07T17:51:27.511 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.511 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:27.511 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1088294 2024-04-07T17:51:27.511 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.511 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:27.511 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.511 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.512 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:27.512 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.512 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.512 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:27.512 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.512 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.512 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:27.512 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 69492229 2024-04-07T17:51:27.512 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.512 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.512 INFO:tasks.workunit.client.0.smithi100.stdout: "objecter": { 2024-04-07T17:51:27.512 INFO:tasks.workunit.client.0.smithi100.stdout: "op_active": { 2024-04-07T17:51:27.512 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.512 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.512 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r": { 2024-04-07T17:51:27.512 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 56 2024-04-07T17:51:27.512 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.513 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rmw": { 2024-04-07T17:51:27.513 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.513 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.513 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w": { 2024-04-07T17:51:27.513 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.513 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.513 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.513 INFO:tasks.workunit.client.0.smithi100.stdout: "osd": { 2024-04-07T17:51:27.513 INFO:tasks.workunit.client.0.smithi100.stdout: "numpg": { 2024-04-07T17:51:27.513 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 55 2024-04-07T17:51:27.513 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.513 INFO:tasks.workunit.client.0.smithi100.stdout: "numpg_removing": { 2024-04-07T17:51:27.513 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.513 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.513 INFO:tasks.workunit.client.0.smithi100.stdout: "op": { 2024-04-07T17:51:27.513 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 53262 2024-04-07T17:51:27.513 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.514 INFO:tasks.workunit.client.0.smithi100.stdout: "op_before_queue_op_lat": { 2024-04-07T17:51:27.514 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 138143, 2024-04-07T17:51:27.514 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 14239009375 2024-04-07T17:51:27.514 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.514 INFO:tasks.workunit.client.0.smithi100.stdout: "op_delayed_degraded": { 2024-04-07T17:51:27.514 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4 2024-04-07T17:51:27.514 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.514 INFO:tasks.workunit.client.0.smithi100.stdout: "op_delayed_unreadable": { 2024-04-07T17:51:27.514 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.514 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.514 INFO:tasks.workunit.client.0.smithi100.stdout: "op_in_bytes": { 2024-04-07T17:51:27.514 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2665625896 2024-04-07T17:51:27.514 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.514 INFO:tasks.workunit.client.0.smithi100.stdout: "op_latency": { 2024-04-07T17:51:27.514 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 53262, 2024-04-07T17:51:27.515 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 365463334048 2024-04-07T17:51:27.515 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.515 INFO:tasks.workunit.client.0.smithi100.stdout: "op_out_bytes": { 2024-04-07T17:51:27.515 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3752394306 2024-04-07T17:51:27.515 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.515 INFO:tasks.workunit.client.0.smithi100.stdout: "op_prepare_latency": { 2024-04-07T17:51:27.515 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 54137, 2024-04-07T17:51:27.515 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 47146686225 2024-04-07T17:51:27.515 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.515 INFO:tasks.workunit.client.0.smithi100.stdout: "op_process_latency": { 2024-04-07T17:51:27.515 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 53262, 2024-04-07T17:51:27.515 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 125634824678 2024-04-07T17:51:27.515 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.515 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r": { 2024-04-07T17:51:27.515 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 37074 2024-04-07T17:51:27.515 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.516 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_latency": { 2024-04-07T17:51:27.516 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 37074, 2024-04-07T17:51:27.516 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 32583038857 2024-04-07T17:51:27.516 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.516 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_latency_out_bytes_histogram": { 2024-04-07T17:51:27.516 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.516 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.516 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_out_bytes": { 2024-04-07T17:51:27.516 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3752394306 2024-04-07T17:51:27.516 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.516 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_prepare_latency": { 2024-04-07T17:51:27.516 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 37434, 2024-04-07T17:51:27.516 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 17131282728 2024-04-07T17:51:27.516 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.516 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_process_latency": { 2024-04-07T17:51:27.517 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 37074, 2024-04-07T17:51:27.517 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 17452953714 2024-04-07T17:51:27.517 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.517 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw": { 2024-04-07T17:51:27.517 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 8557 2024-04-07T17:51:27.517 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.517 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_in_bytes": { 2024-04-07T17:51:27.517 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 574680166 2024-04-07T17:51:27.517 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.517 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_latency": { 2024-04-07T17:51:27.517 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 8557, 2024-04-07T17:51:27.517 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 36381436318 2024-04-07T17:51:27.517 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.517 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_latency_in_bytes_histogram": { 2024-04-07T17:51:27.517 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.517 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.517 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_latency_out_bytes_histogram": { 2024-04-07T17:51:27.518 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.518 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.518 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_out_bytes": { 2024-04-07T17:51:27.518 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.518 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.518 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_prepare_latency": { 2024-04-07T17:51:27.518 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 8868, 2024-04-07T17:51:27.518 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 12409822934 2024-04-07T17:51:27.518 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.518 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_process_latency": { 2024-04-07T17:51:27.518 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 8557, 2024-04-07T17:51:27.518 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 31269983898 2024-04-07T17:51:27.518 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.518 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w": { 2024-04-07T17:51:27.518 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 7631 2024-04-07T17:51:27.519 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.519 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_in_bytes": { 2024-04-07T17:51:27.519 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2090945730 2024-04-07T17:51:27.519 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.519 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_latency": { 2024-04-07T17:51:27.519 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 7631, 2024-04-07T17:51:27.519 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 296498858873 2024-04-07T17:51:27.519 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.519 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_latency_in_bytes_histogram": { 2024-04-07T17:51:27.519 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.519 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.519 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_prepare_latency": { 2024-04-07T17:51:27.519 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 7835, 2024-04-07T17:51:27.519 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 17605580563 2024-04-07T17:51:27.519 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.520 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_process_latency": { 2024-04-07T17:51:27.520 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 7631, 2024-04-07T17:51:27.520 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 76911887066 2024-04-07T17:51:27.520 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.520 INFO:tasks.workunit.client.0.smithi100.stdout: "op_wip": { 2024-04-07T17:51:27.520 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.520 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.520 INFO:tasks.workunit.client.0.smithi100.stdout: "recovery_bytes": { 2024-04-07T17:51:27.520 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.520 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.520 INFO:tasks.workunit.client.0.smithi100.stdout: "recovery_ops": { 2024-04-07T17:51:27.520 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 26 2024-04-07T17:51:27.520 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.520 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_bytes": { 2024-04-07T17:51:27.520 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 95995035648 2024-04-07T17:51:27.520 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.521 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_bytes_used": { 2024-04-07T17:51:27.521 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 745607168 2024-04-07T17:51:27.521 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.521 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.521 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.521 INFO:tasks.workunit.client.0.smithi100.stdout: "osd.1": { 2024-04-07T17:51:27.521 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs": { 2024-04-07T17:51:27.521 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_db_max_lat": { 2024-04-07T17:51:27.521 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 30825 2024-04-07T17:51:27.521 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.521 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_slow_fallback": { 2024-04-07T17:51:27.521 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.521 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.521 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_slow_max_lat": { 2024-04-07T17:51:27.521 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.522 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.522 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_slow_size_fallback": { 2024-04-07T17:51:27.522 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.522 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.522 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit_db": { 2024-04-07T17:51:27.522 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 65536 2024-04-07T17:51:27.522 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.522 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit_slow": { 2024-04-07T17:51:27.522 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.522 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.522 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit_wal": { 2024-04-07T17:51:27.522 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.522 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.522 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_wal_max_lat": { 2024-04-07T17:51:27.522 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.523 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.523 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_written_slow": { 2024-04-07T17:51:27.523 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.523 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.523 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_written_sst": { 2024-04-07T17:51:27.523 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5402624 2024-04-07T17:51:27.523 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.523 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_written_wal": { 2024-04-07T17:51:27.523 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 723148800 2024-04-07T17:51:27.523 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.523 INFO:tasks.workunit.client.0.smithi100.stdout: "compact_lat": { 2024-04-07T17:51:27.523 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 18, 2024-04-07T17:51:27.523 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5817020 2024-04-07T17:51:27.523 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.524 INFO:tasks.workunit.client.0.smithi100.stdout: "compact_lock_lat": { 2024-04-07T17:51:27.524 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 18, 2024-04-07T17:51:27.524 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2179935 2024-04-07T17:51:27.524 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.524 INFO:tasks.workunit.client.0.smithi100.stdout: "db_alloc_lat": { 2024-04-07T17:51:27.524 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 187, 2024-04-07T17:51:27.524 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1295182 2024-04-07T17:51:27.524 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.524 INFO:tasks.workunit.client.0.smithi100.stdout: "db_total_bytes": { 2024-04-07T17:51:27.524 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 95995027456 2024-04-07T17:51:27.524 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.524 INFO:tasks.workunit.client.0.smithi100.stdout: "db_used_bytes": { 2024-04-07T17:51:27.524 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 721354752 2024-04-07T17:51:27.524 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.524 INFO:tasks.workunit.client.0.smithi100.stdout: "flush_lat": { 2024-04-07T17:51:27.525 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 74047, 2024-04-07T17:51:27.525 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 13624180905 2024-04-07T17:51:27.525 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.525 INFO:tasks.workunit.client.0.smithi100.stdout: "fsync_lat": { 2024-04-07T17:51:27.525 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 74056, 2024-04-07T17:51:27.525 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 23366921692 2024-04-07T17:51:27.525 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.525 INFO:tasks.workunit.client.0.smithi100.stdout: "log_bytes": { 2024-04-07T17:51:27.525 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4120576 2024-04-07T17:51:27.525 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.525 INFO:tasks.workunit.client.0.smithi100.stdout: "logged_bytes": { 2024-04-07T17:51:27.525 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 303386624 2024-04-07T17:51:27.525 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.525 INFO:tasks.workunit.client.0.smithi100.stdout: "max_bytes_db": { 2024-04-07T17:51:27.525 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 738394112 2024-04-07T17:51:27.526 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.526 INFO:tasks.workunit.client.0.smithi100.stdout: "max_bytes_slow": { 2024-04-07T17:51:27.526 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.526 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.526 INFO:tasks.workunit.client.0.smithi100.stdout: "max_bytes_wal": { 2024-04-07T17:51:27.526 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.526 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.526 INFO:tasks.workunit.client.0.smithi100.stdout: "num_files": { 2024-04-07T17:51:27.526 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 50 2024-04-07T17:51:27.526 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.526 INFO:tasks.workunit.client.0.smithi100.stdout: "read_bytes": { 2024-04-07T17:51:27.526 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3171092523 2024-04-07T17:51:27.526 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.526 INFO:tasks.workunit.client.0.smithi100.stdout: "read_count": { 2024-04-07T17:51:27.526 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 42792 2024-04-07T17:51:27.527 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.527 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes": { 2024-04-07T17:51:27.527 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2761277440 2024-04-07T17:51:27.527 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.527 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes_db": { 2024-04-07T17:51:27.527 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2761281536 2024-04-07T17:51:27.527 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.527 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes_slow": { 2024-04-07T17:51:27.527 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.527 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.527 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes_wal": { 2024-04-07T17:51:27.527 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.527 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.527 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_count": { 2024-04-07T17:51:27.527 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 35333 2024-04-07T17:51:27.528 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.528 INFO:tasks.workunit.client.0.smithi100.stdout: "read_lat": { 2024-04-07T17:51:27.528 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 42792, 2024-04-07T17:51:27.528 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2006369197 2024-04-07T17:51:27.528 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.528 INFO:tasks.workunit.client.0.smithi100.stdout: "read_prefetch_bytes": { 2024-04-07T17:51:27.528 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3054473279 2024-04-07T17:51:27.528 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.528 INFO:tasks.workunit.client.0.smithi100.stdout: "read_prefetch_count": { 2024-04-07T17:51:27.528 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 39218 2024-04-07T17:51:27.528 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.528 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_buffer_bytes": { 2024-04-07T17:51:27.528 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1733730 2024-04-07T17:51:27.528 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.528 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_buffer_count": { 2024-04-07T17:51:27.529 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 618 2024-04-07T17:51:27.529 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.529 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_bytes": { 2024-04-07T17:51:27.529 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5349743 2024-04-07T17:51:27.529 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.529 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_count": { 2024-04-07T17:51:27.529 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 831 2024-04-07T17:51:27.529 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.529 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes": { 2024-04-07T17:51:27.529 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3616013 2024-04-07T17:51:27.529 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.529 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes_db": { 2024-04-07T17:51:27.529 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3616013 2024-04-07T17:51:27.529 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.529 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes_slow": { 2024-04-07T17:51:27.529 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.530 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.530 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes_wal": { 2024-04-07T17:51:27.530 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.530 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.530 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_count": { 2024-04-07T17:51:27.530 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 217 2024-04-07T17:51:27.530 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.530 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_lat": { 2024-04-07T17:51:27.530 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 831, 2024-04-07T17:51:27.530 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2170610 2024-04-07T17:51:27.530 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.530 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_alloc_lat": { 2024-04-07T17:51:27.530 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:27.530 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.531 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.531 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_total_bytes": { 2024-04-07T17:51:27.531 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.531 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.531 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_used_bytes": { 2024-04-07T17:51:27.531 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.531 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.531 INFO:tasks.workunit.client.0.smithi100.stdout: "truncate_lat": { 2024-04-07T17:51:27.531 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1, 2024-04-07T17:51:27.531 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1014 2024-04-07T17:51:27.531 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.531 INFO:tasks.workunit.client.0.smithi100.stdout: "unlink_lat": { 2024-04-07T17:51:27.531 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 10, 2024-04-07T17:51:27.531 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 14895 2024-04-07T17:51:27.531 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.532 INFO:tasks.workunit.client.0.smithi100.stdout: "wal_alloc_lat": { 2024-04-07T17:51:27.532 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:27.532 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.532 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.532 INFO:tasks.workunit.client.0.smithi100.stdout: "wal_total_bytes": { 2024-04-07T17:51:27.532 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.532 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.532 INFO:tasks.workunit.client.0.smithi100.stdout: "wal_used_bytes": { 2024-04-07T17:51:27.532 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.532 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.532 INFO:tasks.workunit.client.0.smithi100.stdout: "write_bytes": { 2024-04-07T17:51:27.532 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1034661888 2024-04-07T17:51:27.532 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.532 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.533 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore": { 2024-04-07T17:51:27.533 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit": { 2024-04-07T17:51:27.533 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4096 2024-04-07T17:51:27.533 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.533 INFO:tasks.workunit.client.0.smithi100.stdout: "allocated": { 2024-04-07T17:51:27.533 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 9814016 2024-04-07T17:51:27.533 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.533 INFO:tasks.workunit.client.0.smithi100.stdout: "allocator_lat": { 2024-04-07T17:51:27.533 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 18804, 2024-04-07T17:51:27.533 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 100612374 2024-04-07T17:51:27.533 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.533 INFO:tasks.workunit.client.0.smithi100.stdout: "clist_lat": { 2024-04-07T17:51:27.533 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 3554, 2024-04-07T17:51:27.533 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 9702602885 2024-04-07T17:51:27.533 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.534 INFO:tasks.workunit.client.0.smithi100.stdout: "compress_lat": { 2024-04-07T17:51:27.534 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:27.534 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.534 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.534 INFO:tasks.workunit.client.0.smithi100.stdout: "compressed": { 2024-04-07T17:51:27.534 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.534 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.534 INFO:tasks.workunit.client.0.smithi100.stdout: "compressed_allocated": { 2024-04-07T17:51:27.534 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.534 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.534 INFO:tasks.workunit.client.0.smithi100.stdout: "compressed_original": { 2024-04-07T17:51:27.534 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.534 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.534 INFO:tasks.workunit.client.0.smithi100.stdout: "csum_lat": { 2024-04-07T17:51:27.534 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 20962, 2024-04-07T17:51:27.535 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 250539900 2024-04-07T17:51:27.535 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.535 INFO:tasks.workunit.client.0.smithi100.stdout: "decompress_lat": { 2024-04-07T17:51:27.535 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:27.535 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.535 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.535 INFO:tasks.workunit.client.0.smithi100.stdout: "fragmentation_micros": { 2024-04-07T17:51:27.535 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.535 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.535 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_commit_lat": { 2024-04-07T17:51:27.535 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 73998, 2024-04-07T17:51:27.535 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 30592293335 2024-04-07T17:51:27.535 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.535 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_final_lat": { 2024-04-07T17:51:27.535 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 73127, 2024-04-07T17:51:27.535 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3885020813 2024-04-07T17:51:27.536 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.536 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_flush_lat": { 2024-04-07T17:51:27.536 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 73998, 2024-04-07T17:51:27.536 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 99101609 2024-04-07T17:51:27.536 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.536 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_sync_lat": { 2024-04-07T17:51:27.536 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 73998, 2024-04-07T17:51:27.536 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 30691394944 2024-04-07T17:51:27.536 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.536 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_get_keys_lat": { 2024-04-07T17:51:27.536 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:27.536 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.536 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.536 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_get_values_lat": { 2024-04-07T17:51:27.536 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 29479, 2024-04-07T17:51:27.537 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1535907762 2024-04-07T17:51:27.537 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.537 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_lower_bound_lat": { 2024-04-07T17:51:27.537 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 847, 2024-04-07T17:51:27.537 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5353667 2024-04-07T17:51:27.537 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.537 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_next_lat": { 2024-04-07T17:51:27.537 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 17576, 2024-04-07T17:51:27.537 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 23428630 2024-04-07T17:51:27.537 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.537 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_seek_to_first_lat": { 2024-04-07T17:51:27.537 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 45, 2024-04-07T17:51:27.537 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 74604 2024-04-07T17:51:27.537 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.537 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_upper_bound_lat": { 2024-04-07T17:51:27.538 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 11754, 2024-04-07T17:51:27.538 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3050723203 2024-04-07T17:51:27.538 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.538 INFO:tasks.workunit.client.0.smithi100.stdout: "onode_hits": { 2024-04-07T17:51:27.538 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 227672 2024-04-07T17:51:27.538 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.538 INFO:tasks.workunit.client.0.smithi100.stdout: "onode_misses": { 2024-04-07T17:51:27.538 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 26792 2024-04-07T17:51:27.538 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.538 INFO:tasks.workunit.client.0.smithi100.stdout: "read_lat": { 2024-04-07T17:51:27.538 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 23761, 2024-04-07T17:51:27.538 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3480334341 2024-04-07T17:51:27.538 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.538 INFO:tasks.workunit.client.0.smithi100.stdout: "read_onode_meta_lat": { 2024-04-07T17:51:27.538 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 76381, 2024-04-07T17:51:27.539 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 39178980 2024-04-07T17:51:27.539 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.539 INFO:tasks.workunit.client.0.smithi100.stdout: "read_wait_aio_lat": { 2024-04-07T17:51:27.539 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 52620, 2024-04-07T17:51:27.539 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1661838016 2024-04-07T17:51:27.539 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.539 INFO:tasks.workunit.client.0.smithi100.stdout: "reads_with_retries": { 2024-04-07T17:51:27.539 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.539 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.539 INFO:tasks.workunit.client.0.smithi100.stdout: "remove_lat": { 2024-04-07T17:51:27.539 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 15062, 2024-04-07T17:51:27.539 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 718857059 2024-04-07T17:51:27.539 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.539 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_aio_wait_count": { 2024-04-07T17:51:27.539 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.540 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.540 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_committed_kv_count": { 2024-04-07T17:51:27.540 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.540 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.540 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_read_onode_meta_count": { 2024-04-07T17:51:27.540 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.540 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.540 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_read_wait_aio_count": { 2024-04-07T17:51:27.540 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.540 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.540 INFO:tasks.workunit.client.0.smithi100.stdout: "state_aio_wait_lat": { 2024-04-07T17:51:27.540 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 85696, 2024-04-07T17:51:27.540 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 8515002567 2024-04-07T17:51:27.540 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.540 INFO:tasks.workunit.client.0.smithi100.stdout: "state_deferred_aio_wait_lat": { 2024-04-07T17:51:27.541 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 22706, 2024-04-07T17:51:27.541 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2143104911 2024-04-07T17:51:27.541 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.541 INFO:tasks.workunit.client.0.smithi100.stdout: "state_deferred_cleanup_lat": { 2024-04-07T17:51:27.541 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 22706, 2024-04-07T17:51:27.541 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4386813655909 2024-04-07T17:51:27.541 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.541 INFO:tasks.workunit.client.0.smithi100.stdout: "state_deferred_queued_lat": { 2024-04-07T17:51:27.541 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 22706, 2024-04-07T17:51:27.541 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 7212579664352 2024-04-07T17:51:27.541 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.541 INFO:tasks.workunit.client.0.smithi100.stdout: "state_done_lat": { 2024-04-07T17:51:27.541 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 85696, 2024-04-07T17:51:27.541 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2950395751529 2024-04-07T17:51:27.541 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.541 INFO:tasks.workunit.client.0.smithi100.stdout: "state_finishing_lat": { 2024-04-07T17:51:27.542 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 85696, 2024-04-07T17:51:27.542 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 17842298 2024-04-07T17:51:27.542 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.542 INFO:tasks.workunit.client.0.smithi100.stdout: "state_io_done_lat": { 2024-04-07T17:51:27.542 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 85696, 2024-04-07T17:51:27.542 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 318013774 2024-04-07T17:51:27.542 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.542 INFO:tasks.workunit.client.0.smithi100.stdout: "state_kv_commiting_lat": { 2024-04-07T17:51:27.542 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 85696, 2024-04-07T17:51:27.542 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 50196560708 2024-04-07T17:51:27.542 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.542 INFO:tasks.workunit.client.0.smithi100.stdout: "state_kv_done_lat": { 2024-04-07T17:51:27.542 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 85696, 2024-04-07T17:51:27.542 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 64451217 2024-04-07T17:51:27.543 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.543 INFO:tasks.workunit.client.0.smithi100.stdout: "state_kv_queued_lat": { 2024-04-07T17:51:27.543 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 85696, 2024-04-07T17:51:27.543 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 24917952642 2024-04-07T17:51:27.543 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.543 INFO:tasks.workunit.client.0.smithi100.stdout: "state_prepare_lat": { 2024-04-07T17:51:27.543 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 85696, 2024-04-07T17:51:27.543 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 23724441505 2024-04-07T17:51:27.543 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.543 INFO:tasks.workunit.client.0.smithi100.stdout: "stored": { 2024-04-07T17:51:27.543 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5699861 2024-04-07T17:51:27.543 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.543 INFO:tasks.workunit.client.0.smithi100.stdout: "truncate_lat": { 2024-04-07T17:51:27.543 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 2507, 2024-04-07T17:51:27.544 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 25130678 2024-04-07T17:51:27.544 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.544 INFO:tasks.workunit.client.0.smithi100.stdout: "txc_commit_lat": { 2024-04-07T17:51:27.544 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 85696, 2024-04-07T17:51:27.544 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 107702564402 2024-04-07T17:51:27.544 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.544 INFO:tasks.workunit.client.0.smithi100.stdout: "txc_submit_lat": { 2024-04-07T17:51:27.544 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 85696, 2024-04-07T17:51:27.544 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 25902403453 2024-04-07T17:51:27.544 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.544 INFO:tasks.workunit.client.0.smithi100.stdout: "txc_throttle_lat": { 2024-04-07T17:51:27.544 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 85696, 2024-04-07T17:51:27.544 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 991891115 2024-04-07T17:51:27.544 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.544 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.544 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache": { 2024-04-07T17:51:27.545 INFO:tasks.workunit.client.0.smithi100.stdout: "cache_bytes": { 2024-04-07T17:51:27.545 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1444614092 2024-04-07T17:51:27.545 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.545 INFO:tasks.workunit.client.0.smithi100.stdout: "heap_bytes": { 2024-04-07T17:51:27.545 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 695918592 2024-04-07T17:51:27.545 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.545 INFO:tasks.workunit.client.0.smithi100.stdout: "mapped_bytes": { 2024-04-07T17:51:27.545 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 660807680 2024-04-07T17:51:27.545 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.545 INFO:tasks.workunit.client.0.smithi100.stdout: "target_bytes": { 2024-04-07T17:51:27.545 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2646965248 2024-04-07T17:51:27.545 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.545 INFO:tasks.workunit.client.0.smithi100.stdout: "unmapped_bytes": { 2024-04-07T17:51:27.545 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 35110912 2024-04-07T17:51:27.545 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.546 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.546 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:data": { 2024-04-07T17:51:27.546 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:27.546 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 134217728 2024-04-07T17:51:27.546 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.546 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:27.546 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.546 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.546 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:27.546 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.546 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.546 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:27.546 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 59401471 2024-04-07T17:51:27.546 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.547 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:27.547 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.547 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.547 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:27.547 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.547 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.547 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:27.547 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.547 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.547 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:27.547 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.547 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.547 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:27.547 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 32768 2024-04-07T17:51:27.548 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.548 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:27.548 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 815104 2024-04-07T17:51:27.548 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.548 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:27.548 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.548 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.548 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:27.548 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.548 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.548 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:27.548 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.548 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.548 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:27.548 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 73968385 2024-04-07T17:51:27.548 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.549 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.549 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:kv": { 2024-04-07T17:51:27.549 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:27.549 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 587202560 2024-04-07T17:51:27.549 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.549 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:27.549 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 67108864 2024-04-07T17:51:27.549 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.549 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:27.549 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.549 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.549 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:27.549 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 445511035 2024-04-07T17:51:27.549 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.549 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:27.550 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.550 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.550 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:27.550 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 256 2024-04-07T17:51:27.550 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.550 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:27.550 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.550 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.550 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:27.550 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.550 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.550 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:27.550 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 112208 2024-04-07T17:51:27.550 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.551 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:27.551 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.551 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.551 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:27.551 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.551 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.551 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:27.551 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.551 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.551 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:27.551 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.551 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.551 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:27.551 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 74470197 2024-04-07T17:51:27.551 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.551 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.552 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:kv_onode": { 2024-04-07T17:51:27.552 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:27.552 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 184549376 2024-04-07T17:51:27.552 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.552 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:27.552 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 67108864 2024-04-07T17:51:27.552 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.552 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:27.552 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4603888 2024-04-07T17:51:27.552 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.552 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:27.552 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 39600980 2024-04-07T17:51:27.552 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.552 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:27.553 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.553 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.553 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:27.553 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.553 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.553 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:27.553 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.553 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.553 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:27.553 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.553 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.553 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:27.553 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.553 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.553 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:27.553 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.554 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.554 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:27.554 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.554 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.554 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:27.554 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.554 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.554 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:27.554 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.554 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.554 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:27.554 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 73235644 2024-04-07T17:51:27.554 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.554 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.554 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:meta": { 2024-04-07T17:51:27.555 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:27.555 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 520093696 2024-04-07T17:51:27.555 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.555 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:27.555 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.555 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.555 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:27.555 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.555 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.555 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:27.555 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 445511035 2024-04-07T17:51:27.555 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.555 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:27.555 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.555 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.556 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:27.556 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.556 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.556 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:27.556 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.556 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.556 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:27.556 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.556 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.556 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:27.556 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 143023 2024-04-07T17:51:27.556 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.556 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:27.556 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1096839 2024-04-07T17:51:27.556 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.556 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:27.557 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.557 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.557 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:27.557 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.557 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.557 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:27.557 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.557 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.557 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:27.557 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 73342799 2024-04-07T17:51:27.557 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.557 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.557 INFO:tasks.workunit.client.0.smithi100.stdout: "objecter": { 2024-04-07T17:51:27.557 INFO:tasks.workunit.client.0.smithi100.stdout: "op_active": { 2024-04-07T17:51:27.557 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.557 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.558 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r": { 2024-04-07T17:51:27.558 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 21 2024-04-07T17:51:27.558 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.558 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rmw": { 2024-04-07T17:51:27.558 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.558 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.558 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w": { 2024-04-07T17:51:27.558 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1 2024-04-07T17:51:27.558 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.558 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.558 INFO:tasks.workunit.client.0.smithi100.stdout: "osd": { 2024-04-07T17:51:27.558 INFO:tasks.workunit.client.0.smithi100.stdout: "numpg": { 2024-04-07T17:51:27.558 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 52 2024-04-07T17:51:27.558 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.558 INFO:tasks.workunit.client.0.smithi100.stdout: "numpg_removing": { 2024-04-07T17:51:27.559 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.559 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.559 INFO:tasks.workunit.client.0.smithi100.stdout: "op": { 2024-04-07T17:51:27.559 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 45446 2024-04-07T17:51:27.559 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.559 INFO:tasks.workunit.client.0.smithi100.stdout: "op_before_queue_op_lat": { 2024-04-07T17:51:27.559 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 141258, 2024-04-07T17:51:27.559 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 14856495861 2024-04-07T17:51:27.559 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.559 INFO:tasks.workunit.client.0.smithi100.stdout: "op_delayed_degraded": { 2024-04-07T17:51:27.559 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.559 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.559 INFO:tasks.workunit.client.0.smithi100.stdout: "op_delayed_unreadable": { 2024-04-07T17:51:27.559 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1 2024-04-07T17:51:27.559 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.560 INFO:tasks.workunit.client.0.smithi100.stdout: "op_in_bytes": { 2024-04-07T17:51:27.560 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2465060363 2024-04-07T17:51:27.560 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.560 INFO:tasks.workunit.client.0.smithi100.stdout: "op_latency": { 2024-04-07T17:51:27.560 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 45446, 2024-04-07T17:51:27.560 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 845709748759 2024-04-07T17:51:27.560 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.560 INFO:tasks.workunit.client.0.smithi100.stdout: "op_out_bytes": { 2024-04-07T17:51:27.560 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2916484475 2024-04-07T17:51:27.560 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.560 INFO:tasks.workunit.client.0.smithi100.stdout: "op_prepare_latency": { 2024-04-07T17:51:27.560 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 47110, 2024-04-07T17:51:27.560 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 66786302195 2024-04-07T17:51:27.560 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.560 INFO:tasks.workunit.client.0.smithi100.stdout: "op_process_latency": { 2024-04-07T17:51:27.561 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 45446, 2024-04-07T17:51:27.561 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 194962917930 2024-04-07T17:51:27.561 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.561 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r": { 2024-04-07T17:51:27.561 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 22763 2024-04-07T17:51:27.561 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.561 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_latency": { 2024-04-07T17:51:27.561 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 22763, 2024-04-07T17:51:27.561 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 30619013027 2024-04-07T17:51:27.561 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.561 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_latency_out_bytes_histogram": { 2024-04-07T17:51:27.561 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.561 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.561 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_out_bytes": { 2024-04-07T17:51:27.561 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2916484475 2024-04-07T17:51:27.562 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.562 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_prepare_latency": { 2024-04-07T17:51:27.562 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 23180, 2024-04-07T17:51:27.562 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 12773511407 2024-04-07T17:51:27.562 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.562 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_process_latency": { 2024-04-07T17:51:27.562 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 22763, 2024-04-07T17:51:27.562 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 12468479574 2024-04-07T17:51:27.562 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.562 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw": { 2024-04-07T17:51:27.562 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 12919 2024-04-07T17:51:27.562 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.562 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_in_bytes": { 2024-04-07T17:51:27.562 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 335492738 2024-04-07T17:51:27.562 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.563 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_latency": { 2024-04-07T17:51:27.563 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 12919, 2024-04-07T17:51:27.563 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 239996971130 2024-04-07T17:51:27.563 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.563 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_latency_in_bytes_histogram": { 2024-04-07T17:51:27.563 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.563 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.563 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_latency_out_bytes_histogram": { 2024-04-07T17:51:27.563 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.563 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.563 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_out_bytes": { 2024-04-07T17:51:27.563 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.563 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.563 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_prepare_latency": { 2024-04-07T17:51:27.563 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 13070, 2024-04-07T17:51:27.564 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 30679482792 2024-04-07T17:51:27.564 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.564 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_process_latency": { 2024-04-07T17:51:27.564 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 12919, 2024-04-07T17:51:27.564 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 75646162399 2024-04-07T17:51:27.564 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.564 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w": { 2024-04-07T17:51:27.564 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 9764 2024-04-07T17:51:27.564 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.564 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_in_bytes": { 2024-04-07T17:51:27.564 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2129567625 2024-04-07T17:51:27.564 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.564 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_latency": { 2024-04-07T17:51:27.564 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 9764, 2024-04-07T17:51:27.564 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 575093764602 2024-04-07T17:51:27.564 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.565 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_latency_in_bytes_histogram": { 2024-04-07T17:51:27.565 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.565 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.565 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_prepare_latency": { 2024-04-07T17:51:27.565 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 10860, 2024-04-07T17:51:27.565 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 23333307996 2024-04-07T17:51:27.565 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.565 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_process_latency": { 2024-04-07T17:51:27.565 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 9764, 2024-04-07T17:51:27.565 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 106848275957 2024-04-07T17:51:27.565 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.565 INFO:tasks.workunit.client.0.smithi100.stdout: "op_wip": { 2024-04-07T17:51:27.565 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.565 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.565 INFO:tasks.workunit.client.0.smithi100.stdout: "recovery_bytes": { 2024-04-07T17:51:27.566 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 40249613 2024-04-07T17:51:27.566 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.566 INFO:tasks.workunit.client.0.smithi100.stdout: "recovery_ops": { 2024-04-07T17:51:27.566 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 28 2024-04-07T17:51:27.566 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.566 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_bytes": { 2024-04-07T17:51:27.566 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 95995035648 2024-04-07T17:51:27.566 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.566 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_bytes_used": { 2024-04-07T17:51:27.566 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 731176960 2024-04-07T17:51:27.566 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.566 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.566 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.566 INFO:tasks.workunit.client.0.smithi100.stdout: "osd.2": { 2024-04-07T17:51:27.566 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs": { 2024-04-07T17:51:27.567 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_db_max_lat": { 2024-04-07T17:51:27.567 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 33529 2024-04-07T17:51:27.567 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.567 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_slow_fallback": { 2024-04-07T17:51:27.567 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.567 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.567 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_slow_max_lat": { 2024-04-07T17:51:27.567 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.567 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.567 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_slow_size_fallback": { 2024-04-07T17:51:27.567 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.567 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.567 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit_db": { 2024-04-07T17:51:27.567 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 65536 2024-04-07T17:51:27.567 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.567 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit_slow": { 2024-04-07T17:51:27.568 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.568 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.568 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit_wal": { 2024-04-07T17:51:27.568 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.568 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.568 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_wal_max_lat": { 2024-04-07T17:51:27.568 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.568 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.568 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_written_slow": { 2024-04-07T17:51:27.568 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.568 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.568 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_written_sst": { 2024-04-07T17:51:27.568 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 11583488 2024-04-07T17:51:27.568 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.568 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_written_wal": { 2024-04-07T17:51:27.569 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 567971840 2024-04-07T17:51:27.569 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.569 INFO:tasks.workunit.client.0.smithi100.stdout: "compact_lat": { 2024-04-07T17:51:27.569 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 17, 2024-04-07T17:51:27.569 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 10549451 2024-04-07T17:51:27.569 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.569 INFO:tasks.workunit.client.0.smithi100.stdout: "compact_lock_lat": { 2024-04-07T17:51:27.569 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 17, 2024-04-07T17:51:27.569 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3589378 2024-04-07T17:51:27.569 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.569 INFO:tasks.workunit.client.0.smithi100.stdout: "db_alloc_lat": { 2024-04-07T17:51:27.569 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 176, 2024-04-07T17:51:27.569 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1295988 2024-04-07T17:51:27.569 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.569 INFO:tasks.workunit.client.0.smithi100.stdout: "db_total_bytes": { 2024-04-07T17:51:27.569 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 95995027456 2024-04-07T17:51:27.570 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.570 INFO:tasks.workunit.client.0.smithi100.stdout: "db_used_bytes": { 2024-04-07T17:51:27.570 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 446365696 2024-04-07T17:51:27.570 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.570 INFO:tasks.workunit.client.0.smithi100.stdout: "flush_lat": { 2024-04-07T17:51:27.570 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 69116, 2024-04-07T17:51:27.570 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 12568150083 2024-04-07T17:51:27.570 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.570 INFO:tasks.workunit.client.0.smithi100.stdout: "fsync_lat": { 2024-04-07T17:51:27.570 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 69111, 2024-04-07T17:51:27.570 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 21920914517 2024-04-07T17:51:27.570 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.570 INFO:tasks.workunit.client.0.smithi100.stdout: "log_bytes": { 2024-04-07T17:51:27.570 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 491520 2024-04-07T17:51:27.571 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.571 INFO:tasks.workunit.client.0.smithi100.stdout: "logged_bytes": { 2024-04-07T17:51:27.571 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 283127808 2024-04-07T17:51:27.571 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.571 INFO:tasks.workunit.client.0.smithi100.stdout: "max_bytes_db": { 2024-04-07T17:51:27.571 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 449183744 2024-04-07T17:51:27.571 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.571 INFO:tasks.workunit.client.0.smithi100.stdout: "max_bytes_slow": { 2024-04-07T17:51:27.571 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.571 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.571 INFO:tasks.workunit.client.0.smithi100.stdout: "max_bytes_wal": { 2024-04-07T17:51:27.571 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.571 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.571 INFO:tasks.workunit.client.0.smithi100.stdout: "num_files": { 2024-04-07T17:51:27.571 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 47 2024-04-07T17:51:27.571 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.572 INFO:tasks.workunit.client.0.smithi100.stdout: "read_bytes": { 2024-04-07T17:51:27.572 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 8760902516 2024-04-07T17:51:27.572 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.572 INFO:tasks.workunit.client.0.smithi100.stdout: "read_count": { 2024-04-07T17:51:27.572 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 74144 2024-04-07T17:51:27.572 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.572 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes": { 2024-04-07T17:51:27.572 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 7434956800 2024-04-07T17:51:27.572 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.572 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes_db": { 2024-04-07T17:51:27.572 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 7434960896 2024-04-07T17:51:27.572 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.572 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes_slow": { 2024-04-07T17:51:27.572 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.572 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.573 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes_wal": { 2024-04-07T17:51:27.573 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.573 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.573 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_count": { 2024-04-07T17:51:27.573 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 64202 2024-04-07T17:51:27.573 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.573 INFO:tasks.workunit.client.0.smithi100.stdout: "read_lat": { 2024-04-07T17:51:27.573 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 74144, 2024-04-07T17:51:27.573 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4199208949 2024-04-07T17:51:27.573 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.573 INFO:tasks.workunit.client.0.smithi100.stdout: "read_prefetch_bytes": { 2024-04-07T17:51:27.573 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 8601015701 2024-04-07T17:51:27.573 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.573 INFO:tasks.workunit.client.0.smithi100.stdout: "read_prefetch_count": { 2024-04-07T17:51:27.573 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 69247 2024-04-07T17:51:27.573 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.574 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_buffer_bytes": { 2024-04-07T17:51:27.574 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3233145 2024-04-07T17:51:27.574 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.574 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_buffer_count": { 2024-04-07T17:51:27.574 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 709 2024-04-07T17:51:27.574 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.574 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_bytes": { 2024-04-07T17:51:27.574 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 11496373 2024-04-07T17:51:27.574 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.574 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_count": { 2024-04-07T17:51:27.574 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1006 2024-04-07T17:51:27.574 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.574 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes": { 2024-04-07T17:51:27.574 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 8263228 2024-04-07T17:51:27.574 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.575 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes_db": { 2024-04-07T17:51:27.575 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 8263228 2024-04-07T17:51:27.575 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.575 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes_slow": { 2024-04-07T17:51:27.575 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.575 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.575 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes_wal": { 2024-04-07T17:51:27.575 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.575 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.575 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_count": { 2024-04-07T17:51:27.575 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 312 2024-04-07T17:51:27.575 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.575 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_lat": { 2024-04-07T17:51:27.575 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1006, 2024-04-07T17:51:27.575 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4007550 2024-04-07T17:51:27.576 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.576 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_alloc_lat": { 2024-04-07T17:51:27.576 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:27.576 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.576 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.576 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_total_bytes": { 2024-04-07T17:51:27.576 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.576 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.576 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_used_bytes": { 2024-04-07T17:51:27.576 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.576 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.576 INFO:tasks.workunit.client.0.smithi100.stdout: "truncate_lat": { 2024-04-07T17:51:27.576 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1, 2024-04-07T17:51:27.576 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 636 2024-04-07T17:51:27.577 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.577 INFO:tasks.workunit.client.0.smithi100.stdout: "unlink_lat": { 2024-04-07T17:51:27.577 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 13, 2024-04-07T17:51:27.577 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 14014 2024-04-07T17:51:27.577 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.577 INFO:tasks.workunit.client.0.smithi100.stdout: "wal_alloc_lat": { 2024-04-07T17:51:27.577 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:27.577 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.577 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.577 INFO:tasks.workunit.client.0.smithi100.stdout: "wal_total_bytes": { 2024-04-07T17:51:27.577 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.577 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.577 INFO:tasks.workunit.client.0.smithi100.stdout: "wal_used_bytes": { 2024-04-07T17:51:27.577 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.577 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.578 INFO:tasks.workunit.client.0.smithi100.stdout: "write_bytes": { 2024-04-07T17:51:27.578 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 865255424 2024-04-07T17:51:27.578 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.578 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.578 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore": { 2024-04-07T17:51:27.578 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit": { 2024-04-07T17:51:27.578 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4096 2024-04-07T17:51:27.578 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.578 INFO:tasks.workunit.client.0.smithi100.stdout: "allocated": { 2024-04-07T17:51:27.578 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 9801728 2024-04-07T17:51:27.578 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.578 INFO:tasks.workunit.client.0.smithi100.stdout: "allocator_lat": { 2024-04-07T17:51:27.578 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 13808, 2024-04-07T17:51:27.579 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 73423246 2024-04-07T17:51:27.579 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.579 INFO:tasks.workunit.client.0.smithi100.stdout: "clist_lat": { 2024-04-07T17:51:27.579 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 3623, 2024-04-07T17:51:27.579 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 12377775846 2024-04-07T17:51:27.579 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.579 INFO:tasks.workunit.client.0.smithi100.stdout: "compress_lat": { 2024-04-07T17:51:27.579 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:27.579 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.579 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.579 INFO:tasks.workunit.client.0.smithi100.stdout: "compressed": { 2024-04-07T17:51:27.579 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.579 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.579 INFO:tasks.workunit.client.0.smithi100.stdout: "compressed_allocated": { 2024-04-07T17:51:27.579 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.580 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.580 INFO:tasks.workunit.client.0.smithi100.stdout: "compressed_original": { 2024-04-07T17:51:27.580 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.580 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.580 INFO:tasks.workunit.client.0.smithi100.stdout: "csum_lat": { 2024-04-07T17:51:27.580 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 26449, 2024-04-07T17:51:27.580 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 333837013 2024-04-07T17:51:27.580 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.580 INFO:tasks.workunit.client.0.smithi100.stdout: "decompress_lat": { 2024-04-07T17:51:27.580 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:27.580 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.580 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.580 INFO:tasks.workunit.client.0.smithi100.stdout: "fragmentation_micros": { 2024-04-07T17:51:27.580 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.580 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.580 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_commit_lat": { 2024-04-07T17:51:27.580 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 69053, 2024-04-07T17:51:27.581 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 28692593210 2024-04-07T17:51:27.581 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.581 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_final_lat": { 2024-04-07T17:51:27.581 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 68316, 2024-04-07T17:51:27.581 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3308256168 2024-04-07T17:51:27.581 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.581 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_flush_lat": { 2024-04-07T17:51:27.581 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 69053, 2024-04-07T17:51:27.581 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 77146286 2024-04-07T17:51:27.581 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.581 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_sync_lat": { 2024-04-07T17:51:27.581 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 69053, 2024-04-07T17:51:27.581 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 28769739496 2024-04-07T17:51:27.581 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.581 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_get_keys_lat": { 2024-04-07T17:51:27.582 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:27.582 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.582 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.582 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_get_values_lat": { 2024-04-07T17:51:27.582 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 54368, 2024-04-07T17:51:27.582 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1618898953 2024-04-07T17:51:27.582 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.582 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_lower_bound_lat": { 2024-04-07T17:51:27.582 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 911, 2024-04-07T17:51:27.582 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5592855 2024-04-07T17:51:27.582 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.582 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_next_lat": { 2024-04-07T17:51:27.582 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 22756, 2024-04-07T17:51:27.582 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 25939924 2024-04-07T17:51:27.582 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.583 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_seek_to_first_lat": { 2024-04-07T17:51:27.583 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 36, 2024-04-07T17:51:27.583 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 71387 2024-04-07T17:51:27.583 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.583 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_upper_bound_lat": { 2024-04-07T17:51:27.583 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 19162, 2024-04-07T17:51:27.583 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2934148688 2024-04-07T17:51:27.583 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.583 INFO:tasks.workunit.client.0.smithi100.stdout: "onode_hits": { 2024-04-07T17:51:27.583 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 251164 2024-04-07T17:51:27.583 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.583 INFO:tasks.workunit.client.0.smithi100.stdout: "onode_misses": { 2024-04-07T17:51:27.583 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 19790 2024-04-07T17:51:27.583 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.583 INFO:tasks.workunit.client.0.smithi100.stdout: "read_lat": { 2024-04-07T17:51:27.583 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 17396, 2024-04-07T17:51:27.584 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2437128824 2024-04-07T17:51:27.584 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.584 INFO:tasks.workunit.client.0.smithi100.stdout: "read_onode_meta_lat": { 2024-04-07T17:51:27.584 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 41161, 2024-04-07T17:51:27.584 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 40062786 2024-04-07T17:51:27.584 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.584 INFO:tasks.workunit.client.0.smithi100.stdout: "read_wait_aio_lat": { 2024-04-07T17:51:27.584 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 23765, 2024-04-07T17:51:27.584 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1924044806 2024-04-07T17:51:27.584 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.584 INFO:tasks.workunit.client.0.smithi100.stdout: "reads_with_retries": { 2024-04-07T17:51:27.584 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.584 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.584 INFO:tasks.workunit.client.0.smithi100.stdout: "remove_lat": { 2024-04-07T17:51:27.584 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 12440, 2024-04-07T17:51:27.584 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 700556520 2024-04-07T17:51:27.585 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.585 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_aio_wait_count": { 2024-04-07T17:51:27.585 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.585 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.585 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_committed_kv_count": { 2024-04-07T17:51:27.585 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.585 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.585 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_read_onode_meta_count": { 2024-04-07T17:51:27.585 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.585 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.585 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_read_wait_aio_count": { 2024-04-07T17:51:27.585 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.585 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.585 INFO:tasks.workunit.client.0.smithi100.stdout: "state_aio_wait_lat": { 2024-04-07T17:51:27.585 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 77648, 2024-04-07T17:51:27.585 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 7605898743 2024-04-07T17:51:27.586 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.586 INFO:tasks.workunit.client.0.smithi100.stdout: "state_deferred_aio_wait_lat": { 2024-04-07T17:51:27.586 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 5880, 2024-04-07T17:51:27.586 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 870604827 2024-04-07T17:51:27.586 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.586 INFO:tasks.workunit.client.0.smithi100.stdout: "state_deferred_cleanup_lat": { 2024-04-07T17:51:27.586 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 5879, 2024-04-07T17:51:27.586 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4540688258752 2024-04-07T17:51:27.586 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.586 INFO:tasks.workunit.client.0.smithi100.stdout: "state_deferred_queued_lat": { 2024-04-07T17:51:27.586 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 5880, 2024-04-07T17:51:27.586 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 6337522548440 2024-04-07T17:51:27.586 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.586 INFO:tasks.workunit.client.0.smithi100.stdout: "state_done_lat": { 2024-04-07T17:51:27.586 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 77647, 2024-04-07T17:51:27.587 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2768612828005 2024-04-07T17:51:27.587 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.587 INFO:tasks.workunit.client.0.smithi100.stdout: "state_finishing_lat": { 2024-04-07T17:51:27.587 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 77647, 2024-04-07T17:51:27.587 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 18125716 2024-04-07T17:51:27.587 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.587 INFO:tasks.workunit.client.0.smithi100.stdout: "state_io_done_lat": { 2024-04-07T17:51:27.587 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 77648, 2024-04-07T17:51:27.587 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 245246928 2024-04-07T17:51:27.587 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.587 INFO:tasks.workunit.client.0.smithi100.stdout: "state_kv_commiting_lat": { 2024-04-07T17:51:27.587 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 77648, 2024-04-07T17:51:27.587 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 45419960556 2024-04-07T17:51:27.587 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.587 INFO:tasks.workunit.client.0.smithi100.stdout: "state_kv_done_lat": { 2024-04-07T17:51:27.587 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 77648, 2024-04-07T17:51:27.588 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 60433024 2024-04-07T17:51:27.588 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.588 INFO:tasks.workunit.client.0.smithi100.stdout: "state_kv_queued_lat": { 2024-04-07T17:51:27.588 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 77648, 2024-04-07T17:51:27.588 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 21195606198 2024-04-07T17:51:27.588 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.588 INFO:tasks.workunit.client.0.smithi100.stdout: "state_prepare_lat": { 2024-04-07T17:51:27.588 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 77648, 2024-04-07T17:51:27.588 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 23621700730 2024-04-07T17:51:27.588 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.588 INFO:tasks.workunit.client.0.smithi100.stdout: "stored": { 2024-04-07T17:51:27.588 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5699682 2024-04-07T17:51:27.588 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.588 INFO:tasks.workunit.client.0.smithi100.stdout: "truncate_lat": { 2024-04-07T17:51:27.588 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 2411, 2024-04-07T17:51:27.588 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 23697978 2024-04-07T17:51:27.589 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.589 INFO:tasks.workunit.client.0.smithi100.stdout: "txc_commit_lat": { 2024-04-07T17:51:27.589 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 77648, 2024-04-07T17:51:27.589 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 98115535290 2024-04-07T17:51:27.589 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.589 INFO:tasks.workunit.client.0.smithi100.stdout: "txc_submit_lat": { 2024-04-07T17:51:27.589 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 77648, 2024-04-07T17:51:27.589 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 25610928464 2024-04-07T17:51:27.589 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.589 INFO:tasks.workunit.client.0.smithi100.stdout: "txc_throttle_lat": { 2024-04-07T17:51:27.589 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 77648, 2024-04-07T17:51:27.589 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 903590085 2024-04-07T17:51:27.589 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.589 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.589 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache": { 2024-04-07T17:51:27.590 INFO:tasks.workunit.client.0.smithi100.stdout: "cache_bytes": { 2024-04-07T17:51:27.590 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1444614092 2024-04-07T17:51:27.590 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.590 INFO:tasks.workunit.client.0.smithi100.stdout: "heap_bytes": { 2024-04-07T17:51:27.590 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 563281920 2024-04-07T17:51:27.590 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.590 INFO:tasks.workunit.client.0.smithi100.stdout: "mapped_bytes": { 2024-04-07T17:51:27.590 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 513974272 2024-04-07T17:51:27.590 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.590 INFO:tasks.workunit.client.0.smithi100.stdout: "target_bytes": { 2024-04-07T17:51:27.590 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2646965248 2024-04-07T17:51:27.590 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.590 INFO:tasks.workunit.client.0.smithi100.stdout: "unmapped_bytes": { 2024-04-07T17:51:27.590 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 49307648 2024-04-07T17:51:27.590 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.591 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.591 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:data": { 2024-04-07T17:51:27.591 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:27.591 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 134217728 2024-04-07T17:51:27.591 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.591 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:27.591 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.591 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.591 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:27.591 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.591 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.591 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:27.591 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 59173493 2024-04-07T17:51:27.591 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.591 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:27.591 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.592 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.592 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:27.592 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.592 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.592 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:27.592 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.592 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.592 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:27.592 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.592 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.592 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:27.592 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 8192 2024-04-07T17:51:27.592 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.592 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:27.592 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 77824 2024-04-07T17:51:27.592 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.593 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:27.593 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.593 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.593 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:27.593 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.593 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.593 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:27.593 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.593 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.593 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:27.593 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 74958219 2024-04-07T17:51:27.593 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.593 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.593 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:kv": { 2024-04-07T17:51:27.593 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:27.594 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 578813952 2024-04-07T17:51:27.594 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.594 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:27.594 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 67108864 2024-04-07T17:51:27.594 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.594 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:27.594 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.594 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.594 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:27.594 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 443801202 2024-04-07T17:51:27.594 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.594 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:27.594 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.594 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.595 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:27.595 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.595 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.595 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:27.595 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2384 2024-04-07T17:51:27.595 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.595 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:27.595 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.595 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.595 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:27.595 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 150592 2024-04-07T17:51:27.595 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.595 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:27.595 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 384 2024-04-07T17:51:27.595 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.596 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:27.596 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.596 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.596 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:27.596 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.596 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.596 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:27.596 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.596 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.596 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:27.596 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 67750526 2024-04-07T17:51:27.596 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.596 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.596 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:kv_onode": { 2024-04-07T17:51:27.596 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:27.597 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 184549376 2024-04-07T17:51:27.597 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.597 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:27.597 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 67108864 2024-04-07T17:51:27.597 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.597 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:27.597 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 10790880 2024-04-07T17:51:27.597 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.597 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:27.597 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 39448995 2024-04-07T17:51:27.597 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.597 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:27.597 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.597 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.597 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:27.597 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.598 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.598 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:27.598 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.598 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.598 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:27.598 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.598 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.598 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:27.598 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.598 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.598 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:27.598 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.598 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.598 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:27.599 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.599 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.599 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:27.599 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.599 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.599 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:27.599 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.599 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.599 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:27.599 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 67200637 2024-04-07T17:51:27.599 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.599 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.599 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:meta": { 2024-04-07T17:51:27.599 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:27.599 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 520093696 2024-04-07T17:51:27.599 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.600 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:27.600 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.600 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.600 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:27.600 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.600 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.600 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:27.600 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 443801202 2024-04-07T17:51:27.600 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.600 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:27.600 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.600 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.600 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:27.600 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.600 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.601 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:27.601 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.601 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.601 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:27.601 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.601 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.601 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:27.601 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 138605 2024-04-07T17:51:27.601 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.601 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:27.601 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1113867 2024-04-07T17:51:27.601 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.601 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:27.601 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.601 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.602 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:27.602 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.602 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.602 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:27.602 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.602 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.602 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:27.602 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 75040022 2024-04-07T17:51:27.602 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.602 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.602 INFO:tasks.workunit.client.0.smithi100.stdout: "objecter": { 2024-04-07T17:51:27.602 INFO:tasks.workunit.client.0.smithi100.stdout: "op_active": { 2024-04-07T17:51:27.602 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.602 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.602 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r": { 2024-04-07T17:51:27.603 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 91 2024-04-07T17:51:27.603 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.603 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rmw": { 2024-04-07T17:51:27.603 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.603 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.603 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w": { 2024-04-07T17:51:27.603 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.603 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.603 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.603 INFO:tasks.workunit.client.0.smithi100.stdout: "osd": { 2024-04-07T17:51:27.603 INFO:tasks.workunit.client.0.smithi100.stdout: "numpg": { 2024-04-07T17:51:27.603 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 40 2024-04-07T17:51:27.603 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.603 INFO:tasks.workunit.client.0.smithi100.stdout: "numpg_removing": { 2024-04-07T17:51:27.603 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.604 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.604 INFO:tasks.workunit.client.0.smithi100.stdout: "op": { 2024-04-07T17:51:27.604 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 58391 2024-04-07T17:51:27.604 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.604 INFO:tasks.workunit.client.0.smithi100.stdout: "op_before_queue_op_lat": { 2024-04-07T17:51:27.604 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 150392, 2024-04-07T17:51:27.604 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 13814312300 2024-04-07T17:51:27.604 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.604 INFO:tasks.workunit.client.0.smithi100.stdout: "op_delayed_degraded": { 2024-04-07T17:51:27.604 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5 2024-04-07T17:51:27.604 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.604 INFO:tasks.workunit.client.0.smithi100.stdout: "op_delayed_unreadable": { 2024-04-07T17:51:27.604 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.604 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.604 INFO:tasks.workunit.client.0.smithi100.stdout: "op_in_bytes": { 2024-04-07T17:51:27.605 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2120776034 2024-04-07T17:51:27.605 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.605 INFO:tasks.workunit.client.0.smithi100.stdout: "op_latency": { 2024-04-07T17:51:27.605 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 58391, 2024-04-07T17:51:27.605 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 187069135567 2024-04-07T17:51:27.605 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.605 INFO:tasks.workunit.client.0.smithi100.stdout: "op_out_bytes": { 2024-04-07T17:51:27.605 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3533359549 2024-04-07T17:51:27.605 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.605 INFO:tasks.workunit.client.0.smithi100.stdout: "op_prepare_latency": { 2024-04-07T17:51:27.605 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 59237, 2024-04-07T17:51:27.605 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 89091892647 2024-04-07T17:51:27.605 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.605 INFO:tasks.workunit.client.0.smithi100.stdout: "op_process_latency": { 2024-04-07T17:51:27.605 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 58391, 2024-04-07T17:51:27.606 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 156357104214 2024-04-07T17:51:27.606 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.606 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r": { 2024-04-07T17:51:27.606 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 29188 2024-04-07T17:51:27.606 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.606 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_latency": { 2024-04-07T17:51:27.606 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 29188, 2024-04-07T17:51:27.606 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 31668185766 2024-04-07T17:51:27.606 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.606 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_latency_out_bytes_histogram": { 2024-04-07T17:51:27.606 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.606 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.606 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_out_bytes": { 2024-04-07T17:51:27.607 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3533359549 2024-04-07T17:51:27.607 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.607 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_prepare_latency": { 2024-04-07T17:51:27.607 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 29608, 2024-04-07T17:51:27.607 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 13047146829 2024-04-07T17:51:27.607 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.607 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_process_latency": { 2024-04-07T17:51:27.607 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 29188, 2024-04-07T17:51:27.607 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 17316757778 2024-04-07T17:51:27.607 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.607 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw": { 2024-04-07T17:51:27.607 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 27288 2024-04-07T17:51:27.607 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.607 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_in_bytes": { 2024-04-07T17:51:27.607 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 535101355 2024-04-07T17:51:27.607 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.608 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_latency": { 2024-04-07T17:51:27.608 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 27288, 2024-04-07T17:51:27.608 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 125896770692 2024-04-07T17:51:27.608 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.608 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_latency_in_bytes_histogram": { 2024-04-07T17:51:27.608 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.608 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.608 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_latency_out_bytes_histogram": { 2024-04-07T17:51:27.608 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.608 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.608 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_out_bytes": { 2024-04-07T17:51:27.608 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.608 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.608 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_prepare_latency": { 2024-04-07T17:51:27.609 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 27539, 2024-04-07T17:51:27.609 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 69930450013 2024-04-07T17:51:27.609 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.609 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_process_latency": { 2024-04-07T17:51:27.609 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 27288, 2024-04-07T17:51:27.609 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 117911423955 2024-04-07T17:51:27.609 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.609 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w": { 2024-04-07T17:51:27.609 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1915 2024-04-07T17:51:27.609 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.609 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_in_bytes": { 2024-04-07T17:51:27.609 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1585674679 2024-04-07T17:51:27.609 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.609 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_latency": { 2024-04-07T17:51:27.609 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1915, 2024-04-07T17:51:27.609 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 29504179109 2024-04-07T17:51:27.610 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.610 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_latency_in_bytes_histogram": { 2024-04-07T17:51:27.610 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.610 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.610 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_prepare_latency": { 2024-04-07T17:51:27.610 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 2090, 2024-04-07T17:51:27.610 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 6114295805 2024-04-07T17:51:27.610 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.610 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_process_latency": { 2024-04-07T17:51:27.610 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1915, 2024-04-07T17:51:27.610 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 21128922481 2024-04-07T17:51:27.610 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.610 INFO:tasks.workunit.client.0.smithi100.stdout: "op_wip": { 2024-04-07T17:51:27.610 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.611 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.611 INFO:tasks.workunit.client.0.smithi100.stdout: "recovery_bytes": { 2024-04-07T17:51:27.611 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4194321 2024-04-07T17:51:27.611 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.611 INFO:tasks.workunit.client.0.smithi100.stdout: "recovery_ops": { 2024-04-07T17:51:27.611 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 38 2024-04-07T17:51:27.611 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.611 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_bytes": { 2024-04-07T17:51:27.611 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 95995035648 2024-04-07T17:51:27.611 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.611 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_bytes_used": { 2024-04-07T17:51:27.611 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 456179712 2024-04-07T17:51:27.611 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.611 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.611 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.612 INFO:tasks.workunit.client.0.smithi100.stdout: "osd.3": { 2024-04-07T17:51:27.612 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs": { 2024-04-07T17:51:27.612 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_db_max_lat": { 2024-04-07T17:51:27.612 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 29319 2024-04-07T17:51:27.612 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.612 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_slow_fallback": { 2024-04-07T17:51:27.612 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.612 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.612 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_slow_max_lat": { 2024-04-07T17:51:27.612 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.612 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.612 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_slow_size_fallback": { 2024-04-07T17:51:27.612 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.612 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.612 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit_db": { 2024-04-07T17:51:27.613 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 65536 2024-04-07T17:51:27.613 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.613 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit_slow": { 2024-04-07T17:51:27.613 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.613 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.613 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit_wal": { 2024-04-07T17:51:27.613 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.613 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.613 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_wal_max_lat": { 2024-04-07T17:51:27.613 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.613 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.613 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_written_slow": { 2024-04-07T17:51:27.613 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.613 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.613 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_written_sst": { 2024-04-07T17:51:27.613 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 6107136 2024-04-07T17:51:27.614 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.614 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_written_wal": { 2024-04-07T17:51:27.614 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 546639872 2024-04-07T17:51:27.614 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.614 INFO:tasks.workunit.client.0.smithi100.stdout: "compact_lat": { 2024-04-07T17:51:27.614 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 16, 2024-04-07T17:51:27.614 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5676850 2024-04-07T17:51:27.614 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.614 INFO:tasks.workunit.client.0.smithi100.stdout: "compact_lock_lat": { 2024-04-07T17:51:27.614 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 16, 2024-04-07T17:51:27.614 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1688739 2024-04-07T17:51:27.614 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.614 INFO:tasks.workunit.client.0.smithi100.stdout: "db_alloc_lat": { 2024-04-07T17:51:27.614 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 158, 2024-04-07T17:51:27.614 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1098247 2024-04-07T17:51:27.615 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.615 INFO:tasks.workunit.client.0.smithi100.stdout: "db_total_bytes": { 2024-04-07T17:51:27.615 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 95995027456 2024-04-07T17:51:27.615 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.615 INFO:tasks.workunit.client.0.smithi100.stdout: "db_used_bytes": { 2024-04-07T17:51:27.615 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 449314816 2024-04-07T17:51:27.615 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.615 INFO:tasks.workunit.client.0.smithi100.stdout: "flush_lat": { 2024-04-07T17:51:27.615 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 67796, 2024-04-07T17:51:27.615 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 12755738535 2024-04-07T17:51:27.615 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.615 INFO:tasks.workunit.client.0.smithi100.stdout: "fsync_lat": { 2024-04-07T17:51:27.615 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 67808, 2024-04-07T17:51:27.615 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 22453593571 2024-04-07T17:51:27.616 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.616 INFO:tasks.workunit.client.0.smithi100.stdout: "log_bytes": { 2024-04-07T17:51:27.616 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 11792384 2024-04-07T17:51:27.616 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.616 INFO:tasks.workunit.client.0.smithi100.stdout: "logged_bytes": { 2024-04-07T17:51:27.616 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 277786624 2024-04-07T17:51:27.616 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.616 INFO:tasks.workunit.client.0.smithi100.stdout: "max_bytes_db": { 2024-04-07T17:51:27.616 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 462094336 2024-04-07T17:51:27.616 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.616 INFO:tasks.workunit.client.0.smithi100.stdout: "max_bytes_slow": { 2024-04-07T17:51:27.616 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.616 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.616 INFO:tasks.workunit.client.0.smithi100.stdout: "max_bytes_wal": { 2024-04-07T17:51:27.616 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.617 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.617 INFO:tasks.workunit.client.0.smithi100.stdout: "num_files": { 2024-04-07T17:51:27.617 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 41 2024-04-07T17:51:27.617 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.617 INFO:tasks.workunit.client.0.smithi100.stdout: "read_bytes": { 2024-04-07T17:51:27.617 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5081425364 2024-04-07T17:51:27.617 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.617 INFO:tasks.workunit.client.0.smithi100.stdout: "read_count": { 2024-04-07T17:51:27.617 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 55136 2024-04-07T17:51:27.617 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.617 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes": { 2024-04-07T17:51:27.617 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4273045504 2024-04-07T17:51:27.617 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.617 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes_db": { 2024-04-07T17:51:27.617 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4273049600 2024-04-07T17:51:27.617 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.618 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes_slow": { 2024-04-07T17:51:27.618 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.618 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.618 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes_wal": { 2024-04-07T17:51:27.618 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.618 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.618 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_count": { 2024-04-07T17:51:27.618 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 47715 2024-04-07T17:51:27.618 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.618 INFO:tasks.workunit.client.0.smithi100.stdout: "read_lat": { 2024-04-07T17:51:27.618 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 55136, 2024-04-07T17:51:27.618 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2526773312 2024-04-07T17:51:27.618 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.618 INFO:tasks.workunit.client.0.smithi100.stdout: "read_prefetch_bytes": { 2024-04-07T17:51:27.618 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4957544695 2024-04-07T17:51:27.619 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.619 INFO:tasks.workunit.client.0.smithi100.stdout: "read_prefetch_count": { 2024-04-07T17:51:27.619 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 51339 2024-04-07T17:51:27.619 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.619 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_buffer_bytes": { 2024-04-07T17:51:27.619 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1547577 2024-04-07T17:51:27.619 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.619 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_buffer_count": { 2024-04-07T17:51:27.619 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 561 2024-04-07T17:51:27.619 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.619 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_bytes": { 2024-04-07T17:51:27.619 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 6055753 2024-04-07T17:51:27.619 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.619 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_count": { 2024-04-07T17:51:27.619 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 720 2024-04-07T17:51:27.619 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.620 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes": { 2024-04-07T17:51:27.620 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4508176 2024-04-07T17:51:27.620 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.620 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes_db": { 2024-04-07T17:51:27.620 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4508176 2024-04-07T17:51:27.620 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.620 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes_slow": { 2024-04-07T17:51:27.620 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.620 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.620 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes_wal": { 2024-04-07T17:51:27.620 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.620 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.620 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_count": { 2024-04-07T17:51:27.620 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 161 2024-04-07T17:51:27.620 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.620 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_lat": { 2024-04-07T17:51:27.621 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 720, 2024-04-07T17:51:27.621 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2153712 2024-04-07T17:51:27.621 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.621 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_alloc_lat": { 2024-04-07T17:51:27.621 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:27.621 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.621 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.621 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_total_bytes": { 2024-04-07T17:51:27.621 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.621 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.621 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_used_bytes": { 2024-04-07T17:51:27.621 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.621 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.621 INFO:tasks.workunit.client.0.smithi100.stdout: "truncate_lat": { 2024-04-07T17:51:27.621 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1, 2024-04-07T17:51:27.621 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 872 2024-04-07T17:51:27.622 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.622 INFO:tasks.workunit.client.0.smithi100.stdout: "unlink_lat": { 2024-04-07T17:51:27.622 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 10, 2024-04-07T17:51:27.622 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 15209 2024-04-07T17:51:27.622 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.622 INFO:tasks.workunit.client.0.smithi100.stdout: "wal_alloc_lat": { 2024-04-07T17:51:27.622 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:27.622 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.622 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.622 INFO:tasks.workunit.client.0.smithi100.stdout: "wal_total_bytes": { 2024-04-07T17:51:27.622 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.622 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.622 INFO:tasks.workunit.client.0.smithi100.stdout: "wal_used_bytes": { 2024-04-07T17:51:27.622 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.622 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.623 INFO:tasks.workunit.client.0.smithi100.stdout: "write_bytes": { 2024-04-07T17:51:27.623 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 832966656 2024-04-07T17:51:27.623 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.623 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.623 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore": { 2024-04-07T17:51:27.623 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit": { 2024-04-07T17:51:27.623 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4096 2024-04-07T17:51:27.623 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.623 INFO:tasks.workunit.client.0.smithi100.stdout: "allocated": { 2024-04-07T17:51:27.623 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 9826304 2024-04-07T17:51:27.623 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.623 INFO:tasks.workunit.client.0.smithi100.stdout: "allocator_lat": { 2024-04-07T17:51:27.623 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 19863, 2024-04-07T17:51:27.623 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 103164491 2024-04-07T17:51:27.623 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.623 INFO:tasks.workunit.client.0.smithi100.stdout: "clist_lat": { 2024-04-07T17:51:27.623 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 3584, 2024-04-07T17:51:27.624 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 10742765419 2024-04-07T17:51:27.624 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.624 INFO:tasks.workunit.client.0.smithi100.stdout: "compress_lat": { 2024-04-07T17:51:27.624 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:27.624 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.624 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.624 INFO:tasks.workunit.client.0.smithi100.stdout: "compressed": { 2024-04-07T17:51:27.624 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.624 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.624 INFO:tasks.workunit.client.0.smithi100.stdout: "compressed_allocated": { 2024-04-07T17:51:27.624 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.624 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.624 INFO:tasks.workunit.client.0.smithi100.stdout: "compressed_original": { 2024-04-07T17:51:27.624 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.624 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.625 INFO:tasks.workunit.client.0.smithi100.stdout: "csum_lat": { 2024-04-07T17:51:27.625 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 13277, 2024-04-07T17:51:27.625 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 158291808 2024-04-07T17:51:27.625 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.625 INFO:tasks.workunit.client.0.smithi100.stdout: "decompress_lat": { 2024-04-07T17:51:27.625 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:27.625 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.625 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.625 INFO:tasks.workunit.client.0.smithi100.stdout: "fragmentation_micros": { 2024-04-07T17:51:27.625 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.625 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.625 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_commit_lat": { 2024-04-07T17:51:27.625 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 67759, 2024-04-07T17:51:27.625 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 28639225606 2024-04-07T17:51:27.625 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.625 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_final_lat": { 2024-04-07T17:51:27.626 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 66833, 2024-04-07T17:51:27.626 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3952581972 2024-04-07T17:51:27.626 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.626 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_flush_lat": { 2024-04-07T17:51:27.626 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 67759, 2024-04-07T17:51:27.626 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 98642276 2024-04-07T17:51:27.626 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.626 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_sync_lat": { 2024-04-07T17:51:27.626 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 67759, 2024-04-07T17:51:27.626 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 28737867882 2024-04-07T17:51:27.626 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.626 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_get_keys_lat": { 2024-04-07T17:51:27.626 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:27.626 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.627 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.627 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_get_values_lat": { 2024-04-07T17:51:27.627 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 26953, 2024-04-07T17:51:27.627 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1128418885 2024-04-07T17:51:27.627 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.627 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_lower_bound_lat": { 2024-04-07T17:51:27.627 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1271, 2024-04-07T17:51:27.627 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 7079072 2024-04-07T17:51:27.627 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.627 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_next_lat": { 2024-04-07T17:51:27.627 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 8457, 2024-04-07T17:51:27.627 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 12008607 2024-04-07T17:51:27.627 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.627 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_seek_to_first_lat": { 2024-04-07T17:51:27.627 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 40, 2024-04-07T17:51:27.628 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 80474 2024-04-07T17:51:27.628 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.628 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_upper_bound_lat": { 2024-04-07T17:51:27.628 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 11692, 2024-04-07T17:51:27.628 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4518542429 2024-04-07T17:51:27.628 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.628 INFO:tasks.workunit.client.0.smithi100.stdout: "onode_hits": { 2024-04-07T17:51:27.628 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 208992 2024-04-07T17:51:27.628 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.628 INFO:tasks.workunit.client.0.smithi100.stdout: "onode_misses": { 2024-04-07T17:51:27.628 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 18294 2024-04-07T17:51:27.628 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.628 INFO:tasks.workunit.client.0.smithi100.stdout: "read_lat": { 2024-04-07T17:51:27.628 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 30350, 2024-04-07T17:51:27.628 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2036962015 2024-04-07T17:51:27.628 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.629 INFO:tasks.workunit.client.0.smithi100.stdout: "read_onode_meta_lat": { 2024-04-07T17:51:27.629 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 80371, 2024-04-07T17:51:27.629 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 45758452 2024-04-07T17:51:27.629 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.629 INFO:tasks.workunit.client.0.smithi100.stdout: "read_wait_aio_lat": { 2024-04-07T17:51:27.629 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 50021, 2024-04-07T17:51:27.629 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1142513586 2024-04-07T17:51:27.629 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.629 INFO:tasks.workunit.client.0.smithi100.stdout: "reads_with_retries": { 2024-04-07T17:51:27.629 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.629 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.629 INFO:tasks.workunit.client.0.smithi100.stdout: "remove_lat": { 2024-04-07T17:51:27.629 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 12382, 2024-04-07T17:51:27.629 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 706656351 2024-04-07T17:51:27.629 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.630 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_aio_wait_count": { 2024-04-07T17:51:27.630 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.630 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.630 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_committed_kv_count": { 2024-04-07T17:51:27.630 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.630 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.630 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_read_onode_meta_count": { 2024-04-07T17:51:27.630 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.630 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.630 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_read_wait_aio_count": { 2024-04-07T17:51:27.630 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.630 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.630 INFO:tasks.workunit.client.0.smithi100.stdout: "state_aio_wait_lat": { 2024-04-07T17:51:27.630 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 77291, 2024-04-07T17:51:27.630 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 12556920270 2024-04-07T17:51:27.630 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.631 INFO:tasks.workunit.client.0.smithi100.stdout: "state_deferred_aio_wait_lat": { 2024-04-07T17:51:27.631 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 17849, 2024-04-07T17:51:27.631 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1593523661 2024-04-07T17:51:27.631 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.631 INFO:tasks.workunit.client.0.smithi100.stdout: "state_deferred_cleanup_lat": { 2024-04-07T17:51:27.631 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 17848, 2024-04-07T17:51:27.631 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3943299065015 2024-04-07T17:51:27.631 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.631 INFO:tasks.workunit.client.0.smithi100.stdout: "state_deferred_queued_lat": { 2024-04-07T17:51:27.631 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 17849, 2024-04-07T17:51:27.631 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 6510467867246 2024-04-07T17:51:27.631 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.631 INFO:tasks.workunit.client.0.smithi100.stdout: "state_done_lat": { 2024-04-07T17:51:27.631 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 77290, 2024-04-07T17:51:27.631 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2750364291611 2024-04-07T17:51:27.631 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.631 INFO:tasks.workunit.client.0.smithi100.stdout: "state_finishing_lat": { 2024-04-07T17:51:27.632 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 77290, 2024-04-07T17:51:27.632 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 16788531 2024-04-07T17:51:27.632 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.632 INFO:tasks.workunit.client.0.smithi100.stdout: "state_io_done_lat": { 2024-04-07T17:51:27.632 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 77291, 2024-04-07T17:51:27.632 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 461265229 2024-04-07T17:51:27.632 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.632 INFO:tasks.workunit.client.0.smithi100.stdout: "state_kv_commiting_lat": { 2024-04-07T17:51:27.632 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 77291, 2024-04-07T17:51:27.632 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 50377334346 2024-04-07T17:51:27.632 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.632 INFO:tasks.workunit.client.0.smithi100.stdout: "state_kv_done_lat": { 2024-04-07T17:51:27.632 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 77291, 2024-04-07T17:51:27.632 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 62405628 2024-04-07T17:51:27.632 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.633 INFO:tasks.workunit.client.0.smithi100.stdout: "state_kv_queued_lat": { 2024-04-07T17:51:27.633 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 77291, 2024-04-07T17:51:27.633 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 24119810266 2024-04-07T17:51:27.633 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.633 INFO:tasks.workunit.client.0.smithi100.stdout: "state_prepare_lat": { 2024-04-07T17:51:27.633 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 77291, 2024-04-07T17:51:27.633 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 26664283551 2024-04-07T17:51:27.633 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.633 INFO:tasks.workunit.client.0.smithi100.stdout: "stored": { 2024-04-07T17:51:27.633 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5705610 2024-04-07T17:51:27.633 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.633 INFO:tasks.workunit.client.0.smithi100.stdout: "truncate_lat": { 2024-04-07T17:51:27.633 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 2398, 2024-04-07T17:51:27.633 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 27539717 2024-04-07T17:51:27.633 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.633 INFO:tasks.workunit.client.0.smithi100.stdout: "txc_commit_lat": { 2024-04-07T17:51:27.634 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 77291, 2024-04-07T17:51:27.634 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 114204848603 2024-04-07T17:51:27.634 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.634 INFO:tasks.workunit.client.0.smithi100.stdout: "txc_submit_lat": { 2024-04-07T17:51:27.634 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 77291, 2024-04-07T17:51:27.634 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 28995862182 2024-04-07T17:51:27.634 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.634 INFO:tasks.workunit.client.0.smithi100.stdout: "txc_throttle_lat": { 2024-04-07T17:51:27.634 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 77291, 2024-04-07T17:51:27.634 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1079109304 2024-04-07T17:51:27.634 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.634 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.634 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache": { 2024-04-07T17:51:27.634 INFO:tasks.workunit.client.0.smithi100.stdout: "cache_bytes": { 2024-04-07T17:51:27.634 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1444614092 2024-04-07T17:51:27.635 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.635 INFO:tasks.workunit.client.0.smithi100.stdout: "heap_bytes": { 2024-04-07T17:51:27.635 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 503775232 2024-04-07T17:51:27.635 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.635 INFO:tasks.workunit.client.0.smithi100.stdout: "mapped_bytes": { 2024-04-07T17:51:27.635 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 470474752 2024-04-07T17:51:27.635 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.635 INFO:tasks.workunit.client.0.smithi100.stdout: "target_bytes": { 2024-04-07T17:51:27.635 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2646965248 2024-04-07T17:51:27.635 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.635 INFO:tasks.workunit.client.0.smithi100.stdout: "unmapped_bytes": { 2024-04-07T17:51:27.635 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 33300480 2024-04-07T17:51:27.635 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.635 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.635 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:data": { 2024-04-07T17:51:27.636 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:27.636 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 134217728 2024-04-07T17:51:27.636 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.636 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:27.636 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.636 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.636 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:27.636 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.636 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.636 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:27.636 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 59559460 2024-04-07T17:51:27.636 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.636 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:27.636 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.636 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.636 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:27.637 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.637 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.637 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:27.637 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.637 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.637 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:27.637 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.637 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.637 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:27.637 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 8192 2024-04-07T17:51:27.637 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.637 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:27.637 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 102400 2024-04-07T17:51:27.637 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.637 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:27.638 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.638 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.638 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:27.638 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.638 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.638 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:27.638 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.638 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.638 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:27.638 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 74547676 2024-04-07T17:51:27.638 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.638 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.638 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:kv": { 2024-04-07T17:51:27.638 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:27.639 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 587202560 2024-04-07T17:51:27.639 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.639 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:27.639 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 67108864 2024-04-07T17:51:27.639 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.639 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:27.639 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.639 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.639 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:27.639 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 446695953 2024-04-07T17:51:27.639 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.639 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:27.639 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.639 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.639 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:27.639 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1776 2024-04-07T17:51:27.640 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.640 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:27.640 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.640 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.640 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:27.640 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.640 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.640 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:27.640 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 110304 2024-04-07T17:51:27.640 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.640 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:27.640 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.640 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.640 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:27.640 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.640 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.641 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:27.641 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.641 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.641 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:27.641 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.641 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.641 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:27.641 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 73285663 2024-04-07T17:51:27.641 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.641 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.641 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:kv_onode": { 2024-04-07T17:51:27.641 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:27.641 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 184549376 2024-04-07T17:51:27.641 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.641 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:27.641 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 67108864 2024-04-07T17:51:27.641 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.642 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:27.642 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5620784 2024-04-07T17:51:27.642 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.642 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:27.642 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 39706307 2024-04-07T17:51:27.642 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.642 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:27.642 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.642 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.642 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:27.642 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.642 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.642 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:27.642 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.642 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.642 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:27.643 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.643 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.643 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:27.643 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.643 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.643 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:27.643 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.643 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.643 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:27.643 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.643 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.643 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:27.643 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.643 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.643 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:27.644 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.644 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.644 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:27.644 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 72113421 2024-04-07T17:51:27.644 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.644 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.644 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:meta": { 2024-04-07T17:51:27.644 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:27.644 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 520093696 2024-04-07T17:51:27.644 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.644 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:27.644 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.644 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.644 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:27.644 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.644 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.645 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:27.645 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 446695953 2024-04-07T17:51:27.645 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.645 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:27.645 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.645 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.645 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:27.645 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.645 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.645 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:27.645 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.645 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.645 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:27.645 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.645 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.645 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:27.645 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 151009 2024-04-07T17:51:27.646 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.646 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:27.646 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1112438 2024-04-07T17:51:27.646 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.646 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:27.646 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.646 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.646 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:27.646 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.646 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.646 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:27.646 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.646 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.646 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:27.646 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 72134296 2024-04-07T17:51:27.646 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.647 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.647 INFO:tasks.workunit.client.0.smithi100.stdout: "objecter": { 2024-04-07T17:51:27.647 INFO:tasks.workunit.client.0.smithi100.stdout: "op_active": { 2024-04-07T17:51:27.647 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.647 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.647 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r": { 2024-04-07T17:51:27.647 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 18 2024-04-07T17:51:27.647 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.647 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rmw": { 2024-04-07T17:51:27.647 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.647 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.647 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w": { 2024-04-07T17:51:27.647 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.647 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.648 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.648 INFO:tasks.workunit.client.0.smithi100.stdout: "osd": { 2024-04-07T17:51:27.648 INFO:tasks.workunit.client.0.smithi100.stdout: "numpg": { 2024-04-07T17:51:27.648 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 47 2024-04-07T17:51:27.648 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.648 INFO:tasks.workunit.client.0.smithi100.stdout: "numpg_removing": { 2024-04-07T17:51:27.648 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.648 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.648 INFO:tasks.workunit.client.0.smithi100.stdout: "op": { 2024-04-07T17:51:27.648 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 56964 2024-04-07T17:51:27.648 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.648 INFO:tasks.workunit.client.0.smithi100.stdout: "op_before_queue_op_lat": { 2024-04-07T17:51:27.648 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 133884, 2024-04-07T17:51:27.648 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 12292525237 2024-04-07T17:51:27.648 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.649 INFO:tasks.workunit.client.0.smithi100.stdout: "op_delayed_degraded": { 2024-04-07T17:51:27.649 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.649 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.649 INFO:tasks.workunit.client.0.smithi100.stdout: "op_delayed_unreadable": { 2024-04-07T17:51:27.649 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.649 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.649 INFO:tasks.workunit.client.0.smithi100.stdout: "op_in_bytes": { 2024-04-07T17:51:27.649 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1485365521 2024-04-07T17:51:27.649 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.649 INFO:tasks.workunit.client.0.smithi100.stdout: "op_latency": { 2024-04-07T17:51:27.649 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 56964, 2024-04-07T17:51:27.649 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 394646376159 2024-04-07T17:51:27.649 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.649 INFO:tasks.workunit.client.0.smithi100.stdout: "op_out_bytes": { 2024-04-07T17:51:27.649 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1430791176 2024-04-07T17:51:27.650 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.650 INFO:tasks.workunit.client.0.smithi100.stdout: "op_prepare_latency": { 2024-04-07T17:51:27.650 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 59216, 2024-04-07T17:51:27.650 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 48205385292 2024-04-07T17:51:27.650 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.650 INFO:tasks.workunit.client.0.smithi100.stdout: "op_process_latency": { 2024-04-07T17:51:27.650 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 56964, 2024-04-07T17:51:27.650 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 169980516116 2024-04-07T17:51:27.650 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.650 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r": { 2024-04-07T17:51:27.650 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 39255 2024-04-07T17:51:27.650 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.650 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_latency": { 2024-04-07T17:51:27.650 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 39255, 2024-04-07T17:51:27.650 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 64075850096 2024-04-07T17:51:27.650 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.651 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_latency_out_bytes_histogram": { 2024-04-07T17:51:27.651 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.651 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.651 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_out_bytes": { 2024-04-07T17:51:27.651 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1430791176 2024-04-07T17:51:27.651 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.651 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_prepare_latency": { 2024-04-07T17:51:27.651 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 39480, 2024-04-07T17:51:27.651 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 16088819423 2024-04-07T17:51:27.651 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.651 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_process_latency": { 2024-04-07T17:51:27.651 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 39255, 2024-04-07T17:51:27.651 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 13918274599 2024-04-07T17:51:27.651 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.651 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw": { 2024-04-07T17:51:27.651 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 13767 2024-04-07T17:51:27.652 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.652 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_in_bytes": { 2024-04-07T17:51:27.652 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 309949184 2024-04-07T17:51:27.652 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.652 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_latency": { 2024-04-07T17:51:27.652 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 13767, 2024-04-07T17:51:27.652 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 275992837370 2024-04-07T17:51:27.652 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.652 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_latency_in_bytes_histogram": { 2024-04-07T17:51:27.652 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.652 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.652 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_latency_out_bytes_histogram": { 2024-04-07T17:51:27.652 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.652 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.652 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_out_bytes": { 2024-04-07T17:51:27.652 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.652 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.653 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_prepare_latency": { 2024-04-07T17:51:27.653 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 15694, 2024-04-07T17:51:27.653 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 23263942693 2024-04-07T17:51:27.653 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.653 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_process_latency": { 2024-04-07T17:51:27.653 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 13767, 2024-04-07T17:51:27.653 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 118411226197 2024-04-07T17:51:27.653 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.653 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w": { 2024-04-07T17:51:27.653 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3942 2024-04-07T17:51:27.653 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.653 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_in_bytes": { 2024-04-07T17:51:27.653 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1175416337 2024-04-07T17:51:27.653 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.653 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_latency": { 2024-04-07T17:51:27.654 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 3942, 2024-04-07T17:51:27.654 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 54577688693 2024-04-07T17:51:27.654 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.654 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_latency_in_bytes_histogram": { 2024-04-07T17:51:27.654 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.654 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.654 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_prepare_latency": { 2024-04-07T17:51:27.654 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 4042, 2024-04-07T17:51:27.654 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 8852623176 2024-04-07T17:51:27.654 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.654 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_process_latency": { 2024-04-07T17:51:27.654 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 3942, 2024-04-07T17:51:27.654 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 37651015320 2024-04-07T17:51:27.654 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.654 INFO:tasks.workunit.client.0.smithi100.stdout: "op_wip": { 2024-04-07T17:51:27.655 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.655 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.655 INFO:tasks.workunit.client.0.smithi100.stdout: "recovery_bytes": { 2024-04-07T17:51:27.655 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 35 2024-04-07T17:51:27.655 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.655 INFO:tasks.workunit.client.0.smithi100.stdout: "recovery_ops": { 2024-04-07T17:51:27.655 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3 2024-04-07T17:51:27.655 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.655 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_bytes": { 2024-04-07T17:51:27.655 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 95995035648 2024-04-07T17:51:27.655 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.655 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_bytes_used": { 2024-04-07T17:51:27.655 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 459153408 2024-04-07T17:51:27.655 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.655 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.656 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.656 INFO:tasks.workunit.client.0.smithi100.stdout: "osd.4": { 2024-04-07T17:51:27.656 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs": { 2024-04-07T17:51:27.656 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_db_max_lat": { 2024-04-07T17:51:27.656 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 53233 2024-04-07T17:51:27.656 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.656 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_slow_fallback": { 2024-04-07T17:51:27.656 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.656 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.656 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_slow_max_lat": { 2024-04-07T17:51:27.656 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.656 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.656 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_slow_size_fallback": { 2024-04-07T17:51:27.656 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.656 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.656 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit_db": { 2024-04-07T17:51:27.657 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 65536 2024-04-07T17:51:27.657 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.657 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit_slow": { 2024-04-07T17:51:27.657 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.657 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.657 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit_wal": { 2024-04-07T17:51:27.657 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.657 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.657 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_wal_max_lat": { 2024-04-07T17:51:27.657 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.657 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.657 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_written_slow": { 2024-04-07T17:51:27.657 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.657 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.657 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_written_sst": { 2024-04-07T17:51:27.657 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 14217216 2024-04-07T17:51:27.658 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.658 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_written_wal": { 2024-04-07T17:51:27.658 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 588619776 2024-04-07T17:51:27.658 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.658 INFO:tasks.workunit.client.0.smithi100.stdout: "compact_lat": { 2024-04-07T17:51:27.658 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 14, 2024-04-07T17:51:27.658 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5250958 2024-04-07T17:51:27.658 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.658 INFO:tasks.workunit.client.0.smithi100.stdout: "compact_lock_lat": { 2024-04-07T17:51:27.658 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 14, 2024-04-07T17:51:27.658 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1869555 2024-04-07T17:51:27.658 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.658 INFO:tasks.workunit.client.0.smithi100.stdout: "db_alloc_lat": { 2024-04-07T17:51:27.658 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 167, 2024-04-07T17:51:27.658 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1195870 2024-04-07T17:51:27.658 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.658 INFO:tasks.workunit.client.0.smithi100.stdout: "db_total_bytes": { 2024-04-07T17:51:27.659 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 95995027456 2024-04-07T17:51:27.659 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.659 INFO:tasks.workunit.client.0.smithi100.stdout: "db_used_bytes": { 2024-04-07T17:51:27.659 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 572653568 2024-04-07T17:51:27.659 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.659 INFO:tasks.workunit.client.0.smithi100.stdout: "flush_lat": { 2024-04-07T17:51:27.659 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 60244, 2024-04-07T17:51:27.659 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 12173027260 2024-04-07T17:51:27.659 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.659 INFO:tasks.workunit.client.0.smithi100.stdout: "fsync_lat": { 2024-04-07T17:51:27.659 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 60248, 2024-04-07T17:51:27.659 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 21497842884 2024-04-07T17:51:27.659 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.659 INFO:tasks.workunit.client.0.smithi100.stdout: "log_bytes": { 2024-04-07T17:51:27.659 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 14090240 2024-04-07T17:51:27.660 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.660 INFO:tasks.workunit.client.0.smithi100.stdout: "logged_bytes": { 2024-04-07T17:51:27.660 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 246812672 2024-04-07T17:51:27.660 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.660 INFO:tasks.workunit.client.0.smithi100.stdout: "max_bytes_db": { 2024-04-07T17:51:27.660 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 568393728 2024-04-07T17:51:27.660 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.660 INFO:tasks.workunit.client.0.smithi100.stdout: "max_bytes_slow": { 2024-04-07T17:51:27.660 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.660 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.660 INFO:tasks.workunit.client.0.smithi100.stdout: "max_bytes_wal": { 2024-04-07T17:51:27.660 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.660 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.660 INFO:tasks.workunit.client.0.smithi100.stdout: "num_files": { 2024-04-07T17:51:27.660 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 50 2024-04-07T17:51:27.661 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.661 INFO:tasks.workunit.client.0.smithi100.stdout: "read_bytes": { 2024-04-07T17:51:27.661 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 10796585872 2024-04-07T17:51:27.661 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.661 INFO:tasks.workunit.client.0.smithi100.stdout: "read_count": { 2024-04-07T17:51:27.661 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 87890 2024-04-07T17:51:27.661 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.661 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes": { 2024-04-07T17:51:27.661 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 9426481152 2024-04-07T17:51:27.661 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.661 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes_db": { 2024-04-07T17:51:27.661 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 9426485248 2024-04-07T17:51:27.661 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.661 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes_slow": { 2024-04-07T17:51:27.662 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.662 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.662 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes_wal": { 2024-04-07T17:51:27.662 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.662 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.662 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_count": { 2024-04-07T17:51:27.662 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 80816 2024-04-07T17:51:27.662 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.662 INFO:tasks.workunit.client.0.smithi100.stdout: "read_lat": { 2024-04-07T17:51:27.662 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 87890, 2024-04-07T17:51:27.662 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5954312472 2024-04-07T17:51:27.662 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.662 INFO:tasks.workunit.client.0.smithi100.stdout: "read_prefetch_bytes": { 2024-04-07T17:51:27.662 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 10604152323 2024-04-07T17:51:27.662 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.662 INFO:tasks.workunit.client.0.smithi100.stdout: "read_prefetch_count": { 2024-04-07T17:51:27.663 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 82000 2024-04-07T17:51:27.663 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.663 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_buffer_bytes": { 2024-04-07T17:51:27.663 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4778070 2024-04-07T17:51:27.663 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.663 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_buffer_count": { 2024-04-07T17:51:27.663 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 805 2024-04-07T17:51:27.663 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.663 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_bytes": { 2024-04-07T17:51:27.663 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 14103486 2024-04-07T17:51:27.663 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.663 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_count": { 2024-04-07T17:51:27.663 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1149 2024-04-07T17:51:27.663 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.663 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes": { 2024-04-07T17:51:27.664 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 9325416 2024-04-07T17:51:27.664 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.664 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes_db": { 2024-04-07T17:51:27.664 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 9325416 2024-04-07T17:51:27.664 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.664 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes_slow": { 2024-04-07T17:51:27.664 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.664 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.664 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes_wal": { 2024-04-07T17:51:27.664 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.664 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.664 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_count": { 2024-04-07T17:51:27.664 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 361 2024-04-07T17:51:27.664 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.664 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_lat": { 2024-04-07T17:51:27.664 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1149, 2024-04-07T17:51:27.665 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4714569 2024-04-07T17:51:27.665 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.665 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_alloc_lat": { 2024-04-07T17:51:27.665 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:27.665 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.665 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.665 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_total_bytes": { 2024-04-07T17:51:27.665 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.665 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.665 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_used_bytes": { 2024-04-07T17:51:27.665 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.665 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.665 INFO:tasks.workunit.client.0.smithi100.stdout: "truncate_lat": { 2024-04-07T17:51:27.665 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1, 2024-04-07T17:51:27.665 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 731 2024-04-07T17:51:27.666 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.666 INFO:tasks.workunit.client.0.smithi100.stdout: "unlink_lat": { 2024-04-07T17:51:27.666 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 14, 2024-04-07T17:51:27.666 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 19309 2024-04-07T17:51:27.666 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.666 INFO:tasks.workunit.client.0.smithi100.stdout: "wal_alloc_lat": { 2024-04-07T17:51:27.666 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:27.666 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.666 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.666 INFO:tasks.workunit.client.0.smithi100.stdout: "wal_total_bytes": { 2024-04-07T17:51:27.666 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.666 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.666 INFO:tasks.workunit.client.0.smithi100.stdout: "wal_used_bytes": { 2024-04-07T17:51:27.666 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.667 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.667 INFO:tasks.workunit.client.0.smithi100.stdout: "write_bytes": { 2024-04-07T17:51:27.667 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 851795968 2024-04-07T17:51:27.667 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.667 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.667 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore": { 2024-04-07T17:51:27.667 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit": { 2024-04-07T17:51:27.667 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4096 2024-04-07T17:51:27.667 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.667 INFO:tasks.workunit.client.0.smithi100.stdout: "allocated": { 2024-04-07T17:51:27.667 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 14016512 2024-04-07T17:51:27.667 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.667 INFO:tasks.workunit.client.0.smithi100.stdout: "allocator_lat": { 2024-04-07T17:51:27.667 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 20261, 2024-04-07T17:51:27.667 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 99191054 2024-04-07T17:51:27.667 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.668 INFO:tasks.workunit.client.0.smithi100.stdout: "clist_lat": { 2024-04-07T17:51:27.668 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 3532, 2024-04-07T17:51:27.668 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 14554851866 2024-04-07T17:51:27.668 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.668 INFO:tasks.workunit.client.0.smithi100.stdout: "compress_lat": { 2024-04-07T17:51:27.668 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:27.668 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.668 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.668 INFO:tasks.workunit.client.0.smithi100.stdout: "compressed": { 2024-04-07T17:51:27.668 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.668 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.668 INFO:tasks.workunit.client.0.smithi100.stdout: "compressed_allocated": { 2024-04-07T17:51:27.668 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.668 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.668 INFO:tasks.workunit.client.0.smithi100.stdout: "compressed_original": { 2024-04-07T17:51:27.668 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.669 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.669 INFO:tasks.workunit.client.0.smithi100.stdout: "csum_lat": { 2024-04-07T17:51:27.669 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 12243, 2024-04-07T17:51:27.669 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 157875076 2024-04-07T17:51:27.669 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.669 INFO:tasks.workunit.client.0.smithi100.stdout: "decompress_lat": { 2024-04-07T17:51:27.669 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:27.669 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.669 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.669 INFO:tasks.workunit.client.0.smithi100.stdout: "fragmentation_micros": { 2024-04-07T17:51:27.669 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.669 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.669 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_commit_lat": { 2024-04-07T17:51:27.669 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 60186, 2024-04-07T17:51:27.669 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 27783488280 2024-04-07T17:51:27.669 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.670 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_final_lat": { 2024-04-07T17:51:27.670 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 59468, 2024-04-07T17:51:27.670 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3239794550 2024-04-07T17:51:27.670 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.670 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_flush_lat": { 2024-04-07T17:51:27.670 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 60186, 2024-04-07T17:51:27.670 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 106527146 2024-04-07T17:51:27.670 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.670 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_sync_lat": { 2024-04-07T17:51:27.670 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 60186, 2024-04-07T17:51:27.670 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 27890015426 2024-04-07T17:51:27.670 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.670 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_get_keys_lat": { 2024-04-07T17:51:27.670 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:27.670 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.671 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.671 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_get_values_lat": { 2024-04-07T17:51:27.671 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 22645, 2024-04-07T17:51:27.671 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1026880183 2024-04-07T17:51:27.671 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.671 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_lower_bound_lat": { 2024-04-07T17:51:27.671 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 697, 2024-04-07T17:51:27.671 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4156713 2024-04-07T17:51:27.671 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.671 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_next_lat": { 2024-04-07T17:51:27.671 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 13851, 2024-04-07T17:51:27.671 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 16294854 2024-04-07T17:51:27.671 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.671 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_seek_to_first_lat": { 2024-04-07T17:51:27.671 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 47, 2024-04-07T17:51:27.671 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 58460 2024-04-07T17:51:27.672 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.672 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_upper_bound_lat": { 2024-04-07T17:51:27.672 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 14286, 2024-04-07T17:51:27.672 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 6817372199 2024-04-07T17:51:27.672 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.672 INFO:tasks.workunit.client.0.smithi100.stdout: "onode_hits": { 2024-04-07T17:51:27.672 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 175006 2024-04-07T17:51:27.672 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.672 INFO:tasks.workunit.client.0.smithi100.stdout: "onode_misses": { 2024-04-07T17:51:27.672 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 19274 2024-04-07T17:51:27.672 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.672 INFO:tasks.workunit.client.0.smithi100.stdout: "read_lat": { 2024-04-07T17:51:27.672 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 8345, 2024-04-07T17:51:27.672 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1757977707 2024-04-07T17:51:27.672 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.672 INFO:tasks.workunit.client.0.smithi100.stdout: "read_onode_meta_lat": { 2024-04-07T17:51:27.673 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 35241, 2024-04-07T17:51:27.673 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 18704077 2024-04-07T17:51:27.673 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.673 INFO:tasks.workunit.client.0.smithi100.stdout: "read_wait_aio_lat": { 2024-04-07T17:51:27.673 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 26896, 2024-04-07T17:51:27.673 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1180234012 2024-04-07T17:51:27.673 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.673 INFO:tasks.workunit.client.0.smithi100.stdout: "reads_with_retries": { 2024-04-07T17:51:27.673 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.673 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.673 INFO:tasks.workunit.client.0.smithi100.stdout: "remove_lat": { 2024-04-07T17:51:27.673 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 13430, 2024-04-07T17:51:27.673 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 730792101 2024-04-07T17:51:27.673 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.674 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_aio_wait_count": { 2024-04-07T17:51:27.674 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.674 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.674 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_committed_kv_count": { 2024-04-07T17:51:27.674 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.674 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.674 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_read_onode_meta_count": { 2024-04-07T17:51:27.674 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.674 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.674 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_read_wait_aio_count": { 2024-04-07T17:51:27.674 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.674 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.674 INFO:tasks.workunit.client.0.smithi100.stdout: "state_aio_wait_lat": { 2024-04-07T17:51:27.674 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 69678, 2024-04-07T17:51:27.674 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 8516743959 2024-04-07T17:51:27.674 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.675 INFO:tasks.workunit.client.0.smithi100.stdout: "state_deferred_aio_wait_lat": { 2024-04-07T17:51:27.675 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 18006, 2024-04-07T17:51:27.675 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1525120051 2024-04-07T17:51:27.675 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.675 INFO:tasks.workunit.client.0.smithi100.stdout: "state_deferred_cleanup_lat": { 2024-04-07T17:51:27.675 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 18005, 2024-04-07T17:51:27.675 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4347507639061 2024-04-07T17:51:27.675 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.675 INFO:tasks.workunit.client.0.smithi100.stdout: "state_deferred_queued_lat": { 2024-04-07T17:51:27.675 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 18006, 2024-04-07T17:51:27.675 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 6594884467617 2024-04-07T17:51:27.675 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.675 INFO:tasks.workunit.client.0.smithi100.stdout: "state_done_lat": { 2024-04-07T17:51:27.675 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 69677, 2024-04-07T17:51:27.676 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3006179254631 2024-04-07T17:51:27.676 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.676 INFO:tasks.workunit.client.0.smithi100.stdout: "state_finishing_lat": { 2024-04-07T17:51:27.676 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 69677, 2024-04-07T17:51:27.676 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 16535015 2024-04-07T17:51:27.676 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.676 INFO:tasks.workunit.client.0.smithi100.stdout: "state_io_done_lat": { 2024-04-07T17:51:27.676 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 69678, 2024-04-07T17:51:27.676 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 283438924 2024-04-07T17:51:27.676 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.676 INFO:tasks.workunit.client.0.smithi100.stdout: "state_kv_commiting_lat": { 2024-04-07T17:51:27.676 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 69678, 2024-04-07T17:51:27.676 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 46634763201 2024-04-07T17:51:27.676 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.676 INFO:tasks.workunit.client.0.smithi100.stdout: "state_kv_done_lat": { 2024-04-07T17:51:27.677 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 69678, 2024-04-07T17:51:27.677 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 54811413 2024-04-07T17:51:27.677 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.677 INFO:tasks.workunit.client.0.smithi100.stdout: "state_kv_queued_lat": { 2024-04-07T17:51:27.677 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 69678, 2024-04-07T17:51:27.677 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 21669658103 2024-04-07T17:51:27.677 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.677 INFO:tasks.workunit.client.0.smithi100.stdout: "state_prepare_lat": { 2024-04-07T17:51:27.677 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 69678, 2024-04-07T17:51:27.677 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 25344251701 2024-04-07T17:51:27.677 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.677 INFO:tasks.workunit.client.0.smithi100.stdout: "stored": { 2024-04-07T17:51:27.677 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 9894713 2024-04-07T17:51:27.677 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.677 INFO:tasks.workunit.client.0.smithi100.stdout: "truncate_lat": { 2024-04-07T17:51:27.677 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 2371, 2024-04-07T17:51:27.678 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 24012279 2024-04-07T17:51:27.678 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.678 INFO:tasks.workunit.client.0.smithi100.stdout: "txc_commit_lat": { 2024-04-07T17:51:27.678 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 69678, 2024-04-07T17:51:27.678 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 102473756098 2024-04-07T17:51:27.678 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.678 INFO:tasks.workunit.client.0.smithi100.stdout: "txc_submit_lat": { 2024-04-07T17:51:27.678 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 69678, 2024-04-07T17:51:27.678 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 27569044264 2024-04-07T17:51:27.678 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.678 INFO:tasks.workunit.client.0.smithi100.stdout: "txc_throttle_lat": { 2024-04-07T17:51:27.678 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 69678, 2024-04-07T17:51:27.678 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 877546314 2024-04-07T17:51:27.678 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.678 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.679 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache": { 2024-04-07T17:51:27.679 INFO:tasks.workunit.client.0.smithi100.stdout: "cache_bytes": { 2024-04-07T17:51:27.679 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1444650649 2024-04-07T17:51:27.679 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.679 INFO:tasks.workunit.client.0.smithi100.stdout: "heap_bytes": { 2024-04-07T17:51:27.679 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 623263744 2024-04-07T17:51:27.679 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.679 INFO:tasks.workunit.client.0.smithi100.stdout: "mapped_bytes": { 2024-04-07T17:51:27.679 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 593174528 2024-04-07T17:51:27.679 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.679 INFO:tasks.workunit.client.0.smithi100.stdout: "target_bytes": { 2024-04-07T17:51:27.679 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2647008256 2024-04-07T17:51:27.679 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.679 INFO:tasks.workunit.client.0.smithi100.stdout: "unmapped_bytes": { 2024-04-07T17:51:27.679 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 30089216 2024-04-07T17:51:27.679 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.680 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.680 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:data": { 2024-04-07T17:51:27.680 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:27.680 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 134217728 2024-04-07T17:51:27.680 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.680 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:27.680 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.680 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.680 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:27.680 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.680 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.680 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:27.680 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 58883623 2024-04-07T17:51:27.680 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.680 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:27.681 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.681 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.681 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:27.681 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.681 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.681 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:27.681 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.681 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.681 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:27.681 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.681 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.681 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:27.681 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 28672 2024-04-07T17:51:27.681 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.681 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:27.682 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 757760 2024-04-07T17:51:27.682 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.682 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:27.682 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.682 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.682 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:27.682 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.682 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.682 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:27.682 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.682 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.682 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:27.682 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 74547673 2024-04-07T17:51:27.682 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.682 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.682 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:kv": { 2024-04-07T17:51:27.683 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:27.683 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 578813952 2024-04-07T17:51:27.683 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.683 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:27.683 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 67108864 2024-04-07T17:51:27.683 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.683 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:27.683 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.683 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.683 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:27.683 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 441627178 2024-04-07T17:51:27.683 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.683 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:27.683 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.683 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.683 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:27.684 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.684 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.684 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:27.684 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2416 2024-04-07T17:51:27.684 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.684 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:27.684 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.684 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.684 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:27.684 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 153616 2024-04-07T17:51:27.684 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.684 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:27.684 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.684 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.685 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:27.685 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.685 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.685 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:27.685 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.685 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.685 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:27.685 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.685 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.685 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:27.685 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 69921878 2024-04-07T17:51:27.685 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.685 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.685 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:kv_onode": { 2024-04-07T17:51:27.685 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:27.685 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 192937984 2024-04-07T17:51:27.686 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.686 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:27.686 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 67108864 2024-04-07T17:51:27.686 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.686 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:27.686 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 13844688 2024-04-07T17:51:27.686 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.686 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:27.686 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 39255749 2024-04-07T17:51:27.686 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.686 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:27.686 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.686 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.686 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:27.686 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.686 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.687 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:27.687 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.687 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.687 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:27.687 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.687 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.687 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:27.687 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.687 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.687 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:27.687 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.687 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.687 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:27.687 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.687 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.687 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:27.688 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.688 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.688 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:27.688 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.688 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.688 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:27.688 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 72728683 2024-04-07T17:51:27.688 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.688 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.688 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:meta": { 2024-04-07T17:51:27.688 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:27.688 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 511705088 2024-04-07T17:51:27.688 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.688 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:27.688 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.688 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.688 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:27.689 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.689 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.689 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:27.689 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 441627178 2024-04-07T17:51:27.689 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.689 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:27.689 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.689 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.689 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:27.689 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.689 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.689 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:27.689 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.689 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.689 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:27.690 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.690 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.690 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:27.690 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 158033 2024-04-07T17:51:27.690 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.690 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:27.690 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1110235 2024-04-07T17:51:27.690 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.690 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:27.690 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.690 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.690 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:27.690 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.690 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.690 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:27.691 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.691 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.691 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:27.691 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 68809642 2024-04-07T17:51:27.691 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.691 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.691 INFO:tasks.workunit.client.0.smithi100.stdout: "objecter": { 2024-04-07T17:51:27.691 INFO:tasks.workunit.client.0.smithi100.stdout: "op_active": { 2024-04-07T17:51:27.691 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.691 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.691 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r": { 2024-04-07T17:51:27.691 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 22 2024-04-07T17:51:27.691 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.691 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rmw": { 2024-04-07T17:51:27.691 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.691 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.692 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w": { 2024-04-07T17:51:27.692 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.692 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.692 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.692 INFO:tasks.workunit.client.0.smithi100.stdout: "osd": { 2024-04-07T17:51:27.692 INFO:tasks.workunit.client.0.smithi100.stdout: "numpg": { 2024-04-07T17:51:27.692 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 51 2024-04-07T17:51:27.692 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.692 INFO:tasks.workunit.client.0.smithi100.stdout: "numpg_removing": { 2024-04-07T17:51:27.692 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.692 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.692 INFO:tasks.workunit.client.0.smithi100.stdout: "op": { 2024-04-07T17:51:27.692 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 27984 2024-04-07T17:51:27.692 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.692 INFO:tasks.workunit.client.0.smithi100.stdout: "op_before_queue_op_lat": { 2024-04-07T17:51:27.692 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 87803, 2024-04-07T17:51:27.692 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 14036212222 2024-04-07T17:51:27.692 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.693 INFO:tasks.workunit.client.0.smithi100.stdout: "op_delayed_degraded": { 2024-04-07T17:51:27.693 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1 2024-04-07T17:51:27.693 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.693 INFO:tasks.workunit.client.0.smithi100.stdout: "op_delayed_unreadable": { 2024-04-07T17:51:27.693 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.693 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.693 INFO:tasks.workunit.client.0.smithi100.stdout: "op_in_bytes": { 2024-04-07T17:51:27.693 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1438594914 2024-04-07T17:51:27.693 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.693 INFO:tasks.workunit.client.0.smithi100.stdout: "op_latency": { 2024-04-07T17:51:27.693 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 27984, 2024-04-07T17:51:27.693 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 91475787538 2024-04-07T17:51:27.693 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.693 INFO:tasks.workunit.client.0.smithi100.stdout: "op_out_bytes": { 2024-04-07T17:51:27.693 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1486727151 2024-04-07T17:51:27.693 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.694 INFO:tasks.workunit.client.0.smithi100.stdout: "op_prepare_latency": { 2024-04-07T17:51:27.694 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 28568, 2024-04-07T17:51:27.694 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 27069205821 2024-04-07T17:51:27.694 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.694 INFO:tasks.workunit.client.0.smithi100.stdout: "op_process_latency": { 2024-04-07T17:51:27.694 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 27984, 2024-04-07T17:51:27.694 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 54395342610 2024-04-07T17:51:27.694 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.694 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r": { 2024-04-07T17:51:27.694 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 18559 2024-04-07T17:51:27.694 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.694 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_latency": { 2024-04-07T17:51:27.694 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 18559, 2024-04-07T17:51:27.694 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 15330456442 2024-04-07T17:51:27.694 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.694 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_latency_out_bytes_histogram": { 2024-04-07T17:51:27.694 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.695 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.695 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_out_bytes": { 2024-04-07T17:51:27.695 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1486727151 2024-04-07T17:51:27.695 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.695 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_prepare_latency": { 2024-04-07T17:51:27.695 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 18752, 2024-04-07T17:51:27.695 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 10042264838 2024-04-07T17:51:27.695 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.695 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_process_latency": { 2024-04-07T17:51:27.695 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 18559, 2024-04-07T17:51:27.695 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 9329326658 2024-04-07T17:51:27.695 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.695 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw": { 2024-04-07T17:51:27.695 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5511 2024-04-07T17:51:27.696 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.696 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_in_bytes": { 2024-04-07T17:51:27.696 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 266802303 2024-04-07T17:51:27.696 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.696 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_latency": { 2024-04-07T17:51:27.696 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 5511, 2024-04-07T17:51:27.696 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 19608757265 2024-04-07T17:51:27.696 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.696 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_latency_in_bytes_histogram": { 2024-04-07T17:51:27.696 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.696 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.696 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_latency_out_bytes_histogram": { 2024-04-07T17:51:27.696 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.696 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.696 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_out_bytes": { 2024-04-07T17:51:27.697 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.697 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.697 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_prepare_latency": { 2024-04-07T17:51:27.697 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 5777, 2024-04-07T17:51:27.697 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 7872110857 2024-04-07T17:51:27.697 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.697 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_process_latency": { 2024-04-07T17:51:27.697 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 5511, 2024-04-07T17:51:27.697 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 17096965462 2024-04-07T17:51:27.697 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.697 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w": { 2024-04-07T17:51:27.697 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3914 2024-04-07T17:51:27.697 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.697 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_in_bytes": { 2024-04-07T17:51:27.697 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1171792611 2024-04-07T17:51:27.697 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.697 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_latency": { 2024-04-07T17:51:27.698 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 3914, 2024-04-07T17:51:27.698 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 56536573831 2024-04-07T17:51:27.698 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.698 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_latency_in_bytes_histogram": { 2024-04-07T17:51:27.698 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.698 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.698 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_prepare_latency": { 2024-04-07T17:51:27.698 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 4039, 2024-04-07T17:51:27.698 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 9154830126 2024-04-07T17:51:27.698 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.698 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_process_latency": { 2024-04-07T17:51:27.698 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 3914, 2024-04-07T17:51:27.698 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 27969050490 2024-04-07T17:51:27.698 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.698 INFO:tasks.workunit.client.0.smithi100.stdout: "op_wip": { 2024-04-07T17:51:27.699 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.699 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.699 INFO:tasks.workunit.client.0.smithi100.stdout: "recovery_bytes": { 2024-04-07T17:51:27.699 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 8388656 2024-04-07T17:51:27.699 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.699 INFO:tasks.workunit.client.0.smithi100.stdout: "recovery_ops": { 2024-04-07T17:51:27.699 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 11 2024-04-07T17:51:27.699 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.699 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_bytes": { 2024-04-07T17:51:27.699 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 95995035648 2024-04-07T17:51:27.699 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.699 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_bytes_used": { 2024-04-07T17:51:27.699 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 586682368 2024-04-07T17:51:27.699 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.699 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.700 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.700 INFO:tasks.workunit.client.0.smithi100.stdout: "osd.5": { 2024-04-07T17:51:27.700 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs": { 2024-04-07T17:51:27.700 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_db_max_lat": { 2024-04-07T17:51:27.700 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 26003 2024-04-07T17:51:27.700 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.700 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_slow_fallback": { 2024-04-07T17:51:27.700 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.700 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.700 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_slow_max_lat": { 2024-04-07T17:51:27.700 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.700 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.700 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_slow_size_fallback": { 2024-04-07T17:51:27.700 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.700 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.700 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit_db": { 2024-04-07T17:51:27.701 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 65536 2024-04-07T17:51:27.701 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.701 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit_slow": { 2024-04-07T17:51:27.701 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.701 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.701 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit_wal": { 2024-04-07T17:51:27.701 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.701 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.701 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_wal_max_lat": { 2024-04-07T17:51:27.701 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.701 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.701 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_written_slow": { 2024-04-07T17:51:27.701 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.701 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.701 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_written_sst": { 2024-04-07T17:51:27.702 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 15654912 2024-04-07T17:51:27.702 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.702 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_written_wal": { 2024-04-07T17:51:27.702 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 603410432 2024-04-07T17:51:27.702 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.702 INFO:tasks.workunit.client.0.smithi100.stdout: "compact_lat": { 2024-04-07T17:51:27.702 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 19, 2024-04-07T17:51:27.702 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 8231783 2024-04-07T17:51:27.702 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.702 INFO:tasks.workunit.client.0.smithi100.stdout: "compact_lock_lat": { 2024-04-07T17:51:27.702 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 19, 2024-04-07T17:51:27.702 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2455108 2024-04-07T17:51:27.702 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.702 INFO:tasks.workunit.client.0.smithi100.stdout: "db_alloc_lat": { 2024-04-07T17:51:27.702 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 196, 2024-04-07T17:51:27.703 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1310582 2024-04-07T17:51:27.703 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.703 INFO:tasks.workunit.client.0.smithi100.stdout: "db_total_bytes": { 2024-04-07T17:51:27.703 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 95995027456 2024-04-07T17:51:27.703 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.703 INFO:tasks.workunit.client.0.smithi100.stdout: "db_used_bytes": { 2024-04-07T17:51:27.703 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 444465152 2024-04-07T17:51:27.703 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.703 INFO:tasks.workunit.client.0.smithi100.stdout: "flush_lat": { 2024-04-07T17:51:27.703 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 80665, 2024-04-07T17:51:27.703 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 15235152455 2024-04-07T17:51:27.703 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.703 INFO:tasks.workunit.client.0.smithi100.stdout: "fsync_lat": { 2024-04-07T17:51:27.703 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 80656, 2024-04-07T17:51:27.703 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 26858743324 2024-04-07T17:51:27.703 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.704 INFO:tasks.workunit.client.0.smithi100.stdout: "log_bytes": { 2024-04-07T17:51:27.704 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 14532608 2024-04-07T17:51:27.704 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.704 INFO:tasks.workunit.client.0.smithi100.stdout: "logged_bytes": { 2024-04-07T17:51:27.704 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 330424320 2024-04-07T17:51:27.704 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.704 INFO:tasks.workunit.client.0.smithi100.stdout: "max_bytes_db": { 2024-04-07T17:51:27.704 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 452984832 2024-04-07T17:51:27.704 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.704 INFO:tasks.workunit.client.0.smithi100.stdout: "max_bytes_slow": { 2024-04-07T17:51:27.704 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.704 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.704 INFO:tasks.workunit.client.0.smithi100.stdout: "max_bytes_wal": { 2024-04-07T17:51:27.704 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.704 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.704 INFO:tasks.workunit.client.0.smithi100.stdout: "num_files": { 2024-04-07T17:51:27.705 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 46 2024-04-07T17:51:27.705 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.705 INFO:tasks.workunit.client.0.smithi100.stdout: "read_bytes": { 2024-04-07T17:51:27.705 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 12789730188 2024-04-07T17:51:27.705 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.705 INFO:tasks.workunit.client.0.smithi100.stdout: "read_count": { 2024-04-07T17:51:27.705 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 100535 2024-04-07T17:51:27.705 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.705 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes": { 2024-04-07T17:51:27.705 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 10848047104 2024-04-07T17:51:27.705 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.705 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes_db": { 2024-04-07T17:51:27.705 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 10848051200 2024-04-07T17:51:27.705 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.705 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes_slow": { 2024-04-07T17:51:27.705 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.706 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.706 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes_wal": { 2024-04-07T17:51:27.706 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.706 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.706 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_count": { 2024-04-07T17:51:27.706 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 94858 2024-04-07T17:51:27.706 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.706 INFO:tasks.workunit.client.0.smithi100.stdout: "read_lat": { 2024-04-07T17:51:27.706 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 100535, 2024-04-07T17:51:27.706 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5637691717 2024-04-07T17:51:27.706 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.706 INFO:tasks.workunit.client.0.smithi100.stdout: "read_prefetch_bytes": { 2024-04-07T17:51:27.706 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 12578820194 2024-04-07T17:51:27.706 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.707 INFO:tasks.workunit.client.0.smithi100.stdout: "read_prefetch_count": { 2024-04-07T17:51:27.707 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 94081 2024-04-07T17:51:27.707 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.707 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_buffer_bytes": { 2024-04-07T17:51:27.707 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4494109 2024-04-07T17:51:27.707 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.707 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_buffer_count": { 2024-04-07T17:51:27.707 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 924 2024-04-07T17:51:27.707 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.707 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_bytes": { 2024-04-07T17:51:27.707 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 15530604 2024-04-07T17:51:27.707 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.707 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_count": { 2024-04-07T17:51:27.707 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1234 2024-04-07T17:51:27.707 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.708 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes": { 2024-04-07T17:51:27.708 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 11036495 2024-04-07T17:51:27.708 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.708 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes_db": { 2024-04-07T17:51:27.708 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 11036495 2024-04-07T17:51:27.708 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.708 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes_slow": { 2024-04-07T17:51:27.708 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.708 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.708 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes_wal": { 2024-04-07T17:51:27.708 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.708 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.708 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_count": { 2024-04-07T17:51:27.708 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 324 2024-04-07T17:51:27.708 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.708 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_lat": { 2024-04-07T17:51:27.709 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1234, 2024-04-07T17:51:27.709 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4942084 2024-04-07T17:51:27.709 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.709 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_alloc_lat": { 2024-04-07T17:51:27.709 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:27.709 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.709 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.709 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_total_bytes": { 2024-04-07T17:51:27.709 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.709 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.709 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_used_bytes": { 2024-04-07T17:51:27.709 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.709 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.709 INFO:tasks.workunit.client.0.smithi100.stdout: "truncate_lat": { 2024-04-07T17:51:27.709 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1, 2024-04-07T17:51:27.710 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 682 2024-04-07T17:51:27.710 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.710 INFO:tasks.workunit.client.0.smithi100.stdout: "unlink_lat": { 2024-04-07T17:51:27.710 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 15, 2024-04-07T17:51:27.710 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 19508 2024-04-07T17:51:27.710 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.710 INFO:tasks.workunit.client.0.smithi100.stdout: "wal_alloc_lat": { 2024-04-07T17:51:27.710 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:27.710 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.710 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.710 INFO:tasks.workunit.client.0.smithi100.stdout: "wal_total_bytes": { 2024-04-07T17:51:27.710 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.710 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.710 INFO:tasks.workunit.client.0.smithi100.stdout: "wal_used_bytes": { 2024-04-07T17:51:27.710 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.710 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.711 INFO:tasks.workunit.client.0.smithi100.stdout: "write_bytes": { 2024-04-07T17:51:27.711 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 952373248 2024-04-07T17:51:27.711 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.711 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.711 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore": { 2024-04-07T17:51:27.711 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit": { 2024-04-07T17:51:27.711 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4096 2024-04-07T17:51:27.711 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.711 INFO:tasks.workunit.client.0.smithi100.stdout: "allocated": { 2024-04-07T17:51:27.711 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 14020608 2024-04-07T17:51:27.711 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.711 INFO:tasks.workunit.client.0.smithi100.stdout: "allocator_lat": { 2024-04-07T17:51:27.711 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 22611, 2024-04-07T17:51:27.711 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 116632904 2024-04-07T17:51:27.711 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.712 INFO:tasks.workunit.client.0.smithi100.stdout: "clist_lat": { 2024-04-07T17:51:27.712 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 3546, 2024-04-07T17:51:27.712 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 17766845093 2024-04-07T17:51:27.712 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.712 INFO:tasks.workunit.client.0.smithi100.stdout: "compress_lat": { 2024-04-07T17:51:27.712 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:27.712 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.712 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.712 INFO:tasks.workunit.client.0.smithi100.stdout: "compressed": { 2024-04-07T17:51:27.712 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.712 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.712 INFO:tasks.workunit.client.0.smithi100.stdout: "compressed_allocated": { 2024-04-07T17:51:27.712 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.712 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.712 INFO:tasks.workunit.client.0.smithi100.stdout: "compressed_original": { 2024-04-07T17:51:27.713 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.713 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.713 INFO:tasks.workunit.client.0.smithi100.stdout: "csum_lat": { 2024-04-07T17:51:27.713 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 24579, 2024-04-07T17:51:27.713 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 325075173 2024-04-07T17:51:27.713 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.713 INFO:tasks.workunit.client.0.smithi100.stdout: "decompress_lat": { 2024-04-07T17:51:27.713 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:27.713 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.713 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.713 INFO:tasks.workunit.client.0.smithi100.stdout: "fragmentation_micros": { 2024-04-07T17:51:27.713 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.713 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.713 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_commit_lat": { 2024-04-07T17:51:27.713 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 80597, 2024-04-07T17:51:27.714 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 34623882580 2024-04-07T17:51:27.714 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.714 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_final_lat": { 2024-04-07T17:51:27.714 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 79698, 2024-04-07T17:51:27.714 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3958472053 2024-04-07T17:51:27.714 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.714 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_flush_lat": { 2024-04-07T17:51:27.714 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 80597, 2024-04-07T17:51:27.714 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 99120687 2024-04-07T17:51:27.714 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.714 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_sync_lat": { 2024-04-07T17:51:27.714 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 80597, 2024-04-07T17:51:27.714 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 34723003267 2024-04-07T17:51:27.714 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.714 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_get_keys_lat": { 2024-04-07T17:51:27.714 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:27.715 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.715 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.715 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_get_values_lat": { 2024-04-07T17:51:27.715 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 239680, 2024-04-07T17:51:27.715 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2166046359 2024-04-07T17:51:27.715 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.715 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_lower_bound_lat": { 2024-04-07T17:51:27.715 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 808, 2024-04-07T17:51:27.715 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5289872 2024-04-07T17:51:27.715 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.715 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_next_lat": { 2024-04-07T17:51:27.715 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 128881, 2024-04-07T17:51:27.715 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 97129343 2024-04-07T17:51:27.715 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.715 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_seek_to_first_lat": { 2024-04-07T17:51:27.715 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 50, 2024-04-07T17:51:27.716 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 74621 2024-04-07T17:51:27.716 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.716 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_upper_bound_lat": { 2024-04-07T17:51:27.716 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 19469, 2024-04-07T17:51:27.716 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 8325495929 2024-04-07T17:51:27.716 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.716 INFO:tasks.workunit.client.0.smithi100.stdout: "onode_hits": { 2024-04-07T17:51:27.716 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 436725 2024-04-07T17:51:27.716 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.716 INFO:tasks.workunit.client.0.smithi100.stdout: "onode_misses": { 2024-04-07T17:51:27.716 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 20982 2024-04-07T17:51:27.716 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.716 INFO:tasks.workunit.client.0.smithi100.stdout: "read_lat": { 2024-04-07T17:51:27.717 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 10659, 2024-04-07T17:51:27.717 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2220392001 2024-04-07T17:51:27.717 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.717 INFO:tasks.workunit.client.0.smithi100.stdout: "read_onode_meta_lat": { 2024-04-07T17:51:27.717 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 30132, 2024-04-07T17:51:27.717 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 15167976 2024-04-07T17:51:27.717 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.717 INFO:tasks.workunit.client.0.smithi100.stdout: "read_wait_aio_lat": { 2024-04-07T17:51:27.717 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 19473, 2024-04-07T17:51:27.717 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1807973266 2024-04-07T17:51:27.717 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.717 INFO:tasks.workunit.client.0.smithi100.stdout: "reads_with_retries": { 2024-04-07T17:51:27.717 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.717 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.717 INFO:tasks.workunit.client.0.smithi100.stdout: "remove_lat": { 2024-04-07T17:51:27.718 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 15467, 2024-04-07T17:51:27.718 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 823899553 2024-04-07T17:51:27.718 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.718 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_aio_wait_count": { 2024-04-07T17:51:27.718 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.718 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.718 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_committed_kv_count": { 2024-04-07T17:51:27.718 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.718 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.718 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_read_onode_meta_count": { 2024-04-07T17:51:27.718 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.718 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.718 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_read_wait_aio_count": { 2024-04-07T17:51:27.718 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.718 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.719 INFO:tasks.workunit.client.0.smithi100.stdout: "state_aio_wait_lat": { 2024-04-07T17:51:27.719 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 93751, 2024-04-07T17:51:27.719 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 13111865981 2024-04-07T17:51:27.719 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.719 INFO:tasks.workunit.client.0.smithi100.stdout: "state_deferred_aio_wait_lat": { 2024-04-07T17:51:27.719 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 8297, 2024-04-07T17:51:27.719 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1069282503 2024-04-07T17:51:27.719 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.719 INFO:tasks.workunit.client.0.smithi100.stdout: "state_deferred_cleanup_lat": { 2024-04-07T17:51:27.719 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 8296, 2024-04-07T17:51:27.719 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4488212413467 2024-04-07T17:51:27.719 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.719 INFO:tasks.workunit.client.0.smithi100.stdout: "state_deferred_queued_lat": { 2024-04-07T17:51:27.719 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 8297, 2024-04-07T17:51:27.719 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 6666966893111 2024-04-07T17:51:27.719 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.720 INFO:tasks.workunit.client.0.smithi100.stdout: "state_done_lat": { 2024-04-07T17:51:27.720 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 93750, 2024-04-07T17:51:27.720 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3192176271911 2024-04-07T17:51:27.720 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.720 INFO:tasks.workunit.client.0.smithi100.stdout: "state_finishing_lat": { 2024-04-07T17:51:27.720 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 93750, 2024-04-07T17:51:27.720 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 23533677 2024-04-07T17:51:27.720 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.720 INFO:tasks.workunit.client.0.smithi100.stdout: "state_io_done_lat": { 2024-04-07T17:51:27.720 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 93751, 2024-04-07T17:51:27.720 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 354231200 2024-04-07T17:51:27.720 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.720 INFO:tasks.workunit.client.0.smithi100.stdout: "state_kv_commiting_lat": { 2024-04-07T17:51:27.720 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 93751, 2024-04-07T17:51:27.720 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 60032684809 2024-04-07T17:51:27.721 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.721 INFO:tasks.workunit.client.0.smithi100.stdout: "state_kv_done_lat": { 2024-04-07T17:51:27.721 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 93751, 2024-04-07T17:51:27.721 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 73338408 2024-04-07T17:51:27.721 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.721 INFO:tasks.workunit.client.0.smithi100.stdout: "state_kv_queued_lat": { 2024-04-07T17:51:27.721 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 93751, 2024-04-07T17:51:27.721 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 29192592705 2024-04-07T17:51:27.721 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.721 INFO:tasks.workunit.client.0.smithi100.stdout: "state_prepare_lat": { 2024-04-07T17:51:27.721 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 93751, 2024-04-07T17:51:27.721 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 35861066911 2024-04-07T17:51:27.721 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.721 INFO:tasks.workunit.client.0.smithi100.stdout: "stored": { 2024-04-07T17:51:27.721 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 9899919 2024-04-07T17:51:27.721 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.722 INFO:tasks.workunit.client.0.smithi100.stdout: "truncate_lat": { 2024-04-07T17:51:27.722 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 2320, 2024-04-07T17:51:27.722 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 23026482 2024-04-07T17:51:27.722 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.722 INFO:tasks.workunit.client.0.smithi100.stdout: "txc_commit_lat": { 2024-04-07T17:51:27.722 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 93751, 2024-04-07T17:51:27.722 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 138586597593 2024-04-07T17:51:27.722 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.722 INFO:tasks.workunit.client.0.smithi100.stdout: "txc_submit_lat": { 2024-04-07T17:51:27.722 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 93751, 2024-04-07T17:51:27.722 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 38590847980 2024-04-07T17:51:27.722 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.722 INFO:tasks.workunit.client.0.smithi100.stdout: "txc_throttle_lat": { 2024-04-07T17:51:27.722 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 93751, 2024-04-07T17:51:27.723 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1304542608 2024-04-07T17:51:27.723 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.723 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.723 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache": { 2024-04-07T17:51:27.723 INFO:tasks.workunit.client.0.smithi100.stdout: "cache_bytes": { 2024-04-07T17:51:27.723 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1444650649 2024-04-07T17:51:27.723 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.723 INFO:tasks.workunit.client.0.smithi100.stdout: "heap_bytes": { 2024-04-07T17:51:27.723 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 570359808 2024-04-07T17:51:27.723 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.723 INFO:tasks.workunit.client.0.smithi100.stdout: "mapped_bytes": { 2024-04-07T17:51:27.723 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 557162496 2024-04-07T17:51:27.723 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.723 INFO:tasks.workunit.client.0.smithi100.stdout: "target_bytes": { 2024-04-07T17:51:27.723 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2647008256 2024-04-07T17:51:27.723 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.724 INFO:tasks.workunit.client.0.smithi100.stdout: "unmapped_bytes": { 2024-04-07T17:51:27.724 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 13197312 2024-04-07T17:51:27.724 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.724 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.724 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:data": { 2024-04-07T17:51:27.724 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:27.724 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 134217728 2024-04-07T17:51:27.724 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.724 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:27.724 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.724 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.724 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:27.724 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.724 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.724 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:27.724 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 58739848 2024-04-07T17:51:27.725 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.725 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:27.725 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.725 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.725 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:27.725 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.725 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.725 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:27.725 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.725 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.725 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:27.725 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.725 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.725 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:27.725 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4096 2024-04-07T17:51:27.726 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.726 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:27.726 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 24576 2024-04-07T17:51:27.726 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.726 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:27.726 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.726 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.726 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:27.726 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.726 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.726 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:27.726 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.726 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.726 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:27.726 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 75449208 2024-04-07T17:51:27.727 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.727 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.727 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:kv": { 2024-04-07T17:51:27.727 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:27.727 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 578813952 2024-04-07T17:51:27.727 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.727 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:27.727 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 67108864 2024-04-07T17:51:27.727 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.727 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:27.727 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.727 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.727 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:27.727 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 440548866 2024-04-07T17:51:27.727 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.727 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:27.728 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.728 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.728 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:27.728 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.728 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.728 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:27.728 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2128 2024-04-07T17:51:27.728 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.728 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:27.728 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.728 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.728 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:27.728 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 163344 2024-04-07T17:51:27.728 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.728 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:27.728 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.729 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.729 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:27.729 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.729 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.729 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:27.729 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.729 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.729 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:27.729 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.729 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.729 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:27.729 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 70990750 2024-04-07T17:51:27.729 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.729 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.729 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:kv_onode": { 2024-04-07T17:51:27.729 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:27.729 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 192937984 2024-04-07T17:51:27.730 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.730 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:27.730 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 67108864 2024-04-07T17:51:27.730 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.730 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:27.730 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 15205920 2024-04-07T17:51:27.730 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.730 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:27.730 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 39159899 2024-04-07T17:51:27.730 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.730 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:27.730 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.730 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.730 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:27.730 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.731 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.731 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:27.731 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.731 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.731 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:27.731 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.731 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.731 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:27.731 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.731 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.731 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:27.731 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.731 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.731 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:27.731 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.732 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.732 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:27.732 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.732 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.732 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:27.732 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.732 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.732 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:27.732 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 71463301 2024-04-07T17:51:27.732 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.732 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.732 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:meta": { 2024-04-07T17:51:27.732 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:27.732 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 511705088 2024-04-07T17:51:27.732 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.732 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:27.732 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.733 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.733 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:27.733 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.733 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.733 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:27.733 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 440548866 2024-04-07T17:51:27.733 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.733 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:27.733 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.733 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.733 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:27.733 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.733 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.733 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:27.733 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.733 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.734 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:27.734 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.734 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.734 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:27.734 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 148347 2024-04-07T17:51:27.734 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.734 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:27.734 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1094803 2024-04-07T17:51:27.734 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.734 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:27.734 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.734 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.734 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:27.734 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.735 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.735 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:27.735 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.735 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.735 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:27.735 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 69913072 2024-04-07T17:51:27.735 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.735 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.735 INFO:tasks.workunit.client.0.smithi100.stdout: "objecter": { 2024-04-07T17:51:27.735 INFO:tasks.workunit.client.0.smithi100.stdout: "op_active": { 2024-04-07T17:51:27.735 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.735 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.735 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r": { 2024-04-07T17:51:27.735 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 24 2024-04-07T17:51:27.735 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.735 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rmw": { 2024-04-07T17:51:27.736 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.736 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.736 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w": { 2024-04-07T17:51:27.736 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.736 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.736 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.736 INFO:tasks.workunit.client.0.smithi100.stdout: "osd": { 2024-04-07T17:51:27.736 INFO:tasks.workunit.client.0.smithi100.stdout: "numpg": { 2024-04-07T17:51:27.736 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 54 2024-04-07T17:51:27.736 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.736 INFO:tasks.workunit.client.0.smithi100.stdout: "numpg_removing": { 2024-04-07T17:51:27.736 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.736 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.736 INFO:tasks.workunit.client.0.smithi100.stdout: "op": { 2024-04-07T17:51:27.736 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 37568 2024-04-07T17:51:27.737 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.737 INFO:tasks.workunit.client.0.smithi100.stdout: "op_before_queue_op_lat": { 2024-04-07T17:51:27.737 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 118839, 2024-04-07T17:51:27.737 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 18250742844 2024-04-07T17:51:27.737 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.737 INFO:tasks.workunit.client.0.smithi100.stdout: "op_delayed_degraded": { 2024-04-07T17:51:27.737 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.737 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.737 INFO:tasks.workunit.client.0.smithi100.stdout: "op_delayed_unreadable": { 2024-04-07T17:51:27.737 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.737 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.737 INFO:tasks.workunit.client.0.smithi100.stdout: "op_in_bytes": { 2024-04-07T17:51:27.737 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1478194555 2024-04-07T17:51:27.737 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.737 INFO:tasks.workunit.client.0.smithi100.stdout: "op_latency": { 2024-04-07T17:51:27.737 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 37568, 2024-04-07T17:51:27.738 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 92458218161 2024-04-07T17:51:27.738 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.738 INFO:tasks.workunit.client.0.smithi100.stdout: "op_out_bytes": { 2024-04-07T17:51:27.738 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2285559046 2024-04-07T17:51:27.738 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.738 INFO:tasks.workunit.client.0.smithi100.stdout: "op_prepare_latency": { 2024-04-07T17:51:27.738 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 38251, 2024-04-07T17:51:27.738 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 45291105721 2024-04-07T17:51:27.738 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.738 INFO:tasks.workunit.client.0.smithi100.stdout: "op_process_latency": { 2024-04-07T17:51:27.738 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 37568, 2024-04-07T17:51:27.738 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 67926473334 2024-04-07T17:51:27.738 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.738 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r": { 2024-04-07T17:51:27.738 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 26708 2024-04-07T17:51:27.739 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.739 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_latency": { 2024-04-07T17:51:27.739 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 26708, 2024-04-07T17:51:27.739 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 37896778616 2024-04-07T17:51:27.739 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.739 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_latency_out_bytes_histogram": { 2024-04-07T17:51:27.739 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.739 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.739 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_out_bytes": { 2024-04-07T17:51:27.739 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2285559046 2024-04-07T17:51:27.739 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.739 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_prepare_latency": { 2024-04-07T17:51:27.739 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 26937, 2024-04-07T17:51:27.739 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 27634387557 2024-04-07T17:51:27.739 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.739 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_process_latency": { 2024-04-07T17:51:27.740 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 26708, 2024-04-07T17:51:27.740 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 26262052230 2024-04-07T17:51:27.740 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.740 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw": { 2024-04-07T17:51:27.740 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 9627 2024-04-07T17:51:27.740 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.740 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_in_bytes": { 2024-04-07T17:51:27.740 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 446562633 2024-04-07T17:51:27.740 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.740 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_latency": { 2024-04-07T17:51:27.740 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 9627, 2024-04-07T17:51:27.740 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 39696616362 2024-04-07T17:51:27.740 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.740 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_latency_in_bytes_histogram": { 2024-04-07T17:51:27.740 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.740 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.741 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_latency_out_bytes_histogram": { 2024-04-07T17:51:27.741 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.741 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.741 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_out_bytes": { 2024-04-07T17:51:27.741 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.741 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.741 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_prepare_latency": { 2024-04-07T17:51:27.741 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 9890, 2024-04-07T17:51:27.741 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 13660567957 2024-04-07T17:51:27.741 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.741 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_process_latency": { 2024-04-07T17:51:27.741 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 9627, 2024-04-07T17:51:27.741 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 30252117533 2024-04-07T17:51:27.741 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.741 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w": { 2024-04-07T17:51:27.741 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1233 2024-04-07T17:51:27.741 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.742 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_in_bytes": { 2024-04-07T17:51:27.742 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1031631922 2024-04-07T17:51:27.742 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.742 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_latency": { 2024-04-07T17:51:27.742 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1233, 2024-04-07T17:51:27.742 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 14864823183 2024-04-07T17:51:27.742 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.742 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_latency_in_bytes_histogram": { 2024-04-07T17:51:27.742 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.742 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.742 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_prepare_latency": { 2024-04-07T17:51:27.742 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1424, 2024-04-07T17:51:27.742 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3996150207 2024-04-07T17:51:27.742 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.742 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_process_latency": { 2024-04-07T17:51:27.743 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1233, 2024-04-07T17:51:27.743 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 11412303571 2024-04-07T17:51:27.743 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.743 INFO:tasks.workunit.client.0.smithi100.stdout: "op_wip": { 2024-04-07T17:51:27.743 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.743 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.743 INFO:tasks.workunit.client.0.smithi100.stdout: "recovery_bytes": { 2024-04-07T17:51:27.743 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 11394690 2024-04-07T17:51:27.743 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.743 INFO:tasks.workunit.client.0.smithi100.stdout: "recovery_ops": { 2024-04-07T17:51:27.743 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 18 2024-04-07T17:51:27.743 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.743 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_bytes": { 2024-04-07T17:51:27.743 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 95995035648 2024-04-07T17:51:27.743 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.743 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_bytes_used": { 2024-04-07T17:51:27.743 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 458498048 2024-04-07T17:51:27.744 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.744 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.744 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.744 INFO:tasks.workunit.client.0.smithi100.stdout: "osd.6": { 2024-04-07T17:51:27.744 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs": { 2024-04-07T17:51:27.744 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_db_max_lat": { 2024-04-07T17:51:27.744 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 42976 2024-04-07T17:51:27.744 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.744 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_slow_fallback": { 2024-04-07T17:51:27.744 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.744 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.744 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_slow_max_lat": { 2024-04-07T17:51:27.744 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.744 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.744 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_slow_size_fallback": { 2024-04-07T17:51:27.744 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.745 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.745 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit_db": { 2024-04-07T17:51:27.745 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 65536 2024-04-07T17:51:27.745 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.745 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit_slow": { 2024-04-07T17:51:27.745 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.745 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.745 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit_wal": { 2024-04-07T17:51:27.745 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.745 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.745 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_wal_max_lat": { 2024-04-07T17:51:27.745 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.745 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.745 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_written_slow": { 2024-04-07T17:51:27.745 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.745 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.746 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_written_sst": { 2024-04-07T17:51:27.746 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1253376 2024-04-07T17:51:27.746 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.746 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_written_wal": { 2024-04-07T17:51:27.746 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 458960896 2024-04-07T17:51:27.746 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.746 INFO:tasks.workunit.client.0.smithi100.stdout: "compact_lat": { 2024-04-07T17:51:27.746 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 15, 2024-04-07T17:51:27.746 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5459458 2024-04-07T17:51:27.746 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.746 INFO:tasks.workunit.client.0.smithi100.stdout: "compact_lock_lat": { 2024-04-07T17:51:27.746 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 15, 2024-04-07T17:51:27.746 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1804720 2024-04-07T17:51:27.746 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.746 INFO:tasks.workunit.client.0.smithi100.stdout: "db_alloc_lat": { 2024-04-07T17:51:27.747 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 135, 2024-04-07T17:51:27.747 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 985275 2024-04-07T17:51:27.747 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.747 INFO:tasks.workunit.client.0.smithi100.stdout: "db_total_bytes": { 2024-04-07T17:51:27.747 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 95995027456 2024-04-07T17:51:27.747 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.747 INFO:tasks.workunit.client.0.smithi100.stdout: "db_used_bytes": { 2024-04-07T17:51:27.747 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 343408640 2024-04-07T17:51:27.747 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.747 INFO:tasks.workunit.client.0.smithi100.stdout: "flush_lat": { 2024-04-07T17:51:27.747 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 61485, 2024-04-07T17:51:27.747 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 11982013100 2024-04-07T17:51:27.747 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.747 INFO:tasks.workunit.client.0.smithi100.stdout: "fsync_lat": { 2024-04-07T17:51:27.747 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 61497, 2024-04-07T17:51:27.747 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 21138088477 2024-04-07T17:51:27.748 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.748 INFO:tasks.workunit.client.0.smithi100.stdout: "log_bytes": { 2024-04-07T17:51:27.748 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2564096 2024-04-07T17:51:27.748 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.748 INFO:tasks.workunit.client.0.smithi100.stdout: "logged_bytes": { 2024-04-07T17:51:27.748 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 251932672 2024-04-07T17:51:27.748 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.748 INFO:tasks.workunit.client.0.smithi100.stdout: "max_bytes_db": { 2024-04-07T17:51:27.748 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 364707840 2024-04-07T17:51:27.748 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.748 INFO:tasks.workunit.client.0.smithi100.stdout: "max_bytes_slow": { 2024-04-07T17:51:27.748 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.748 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.748 INFO:tasks.workunit.client.0.smithi100.stdout: "max_bytes_wal": { 2024-04-07T17:51:27.748 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.748 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.749 INFO:tasks.workunit.client.0.smithi100.stdout: "num_files": { 2024-04-07T17:51:27.749 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 33 2024-04-07T17:51:27.749 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.749 INFO:tasks.workunit.client.0.smithi100.stdout: "read_bytes": { 2024-04-07T17:51:27.749 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 611394701 2024-04-07T17:51:27.749 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.749 INFO:tasks.workunit.client.0.smithi100.stdout: "read_count": { 2024-04-07T17:51:27.749 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 18243 2024-04-07T17:51:27.749 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.749 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes": { 2024-04-07T17:51:27.749 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 531197952 2024-04-07T17:51:27.749 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.749 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes_db": { 2024-04-07T17:51:27.749 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 531202048 2024-04-07T17:51:27.749 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.750 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes_slow": { 2024-04-07T17:51:27.750 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.750 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.750 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes_wal": { 2024-04-07T17:51:27.750 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.750 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.750 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_count": { 2024-04-07T17:51:27.750 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 16144 2024-04-07T17:51:27.750 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.750 INFO:tasks.workunit.client.0.smithi100.stdout: "read_lat": { 2024-04-07T17:51:27.750 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 18243, 2024-04-07T17:51:27.750 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 442981158 2024-04-07T17:51:27.750 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.750 INFO:tasks.workunit.client.0.smithi100.stdout: "read_prefetch_bytes": { 2024-04-07T17:51:27.750 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 565009679 2024-04-07T17:51:27.751 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.751 INFO:tasks.workunit.client.0.smithi100.stdout: "read_prefetch_count": { 2024-04-07T17:51:27.751 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 16814 2024-04-07T17:51:27.751 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.751 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_buffer_bytes": { 2024-04-07T17:51:27.751 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 563551 2024-04-07T17:51:27.751 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.751 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_buffer_count": { 2024-04-07T17:51:27.751 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 339 2024-04-07T17:51:27.751 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.751 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_bytes": { 2024-04-07T17:51:27.751 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1239831 2024-04-07T17:51:27.751 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.751 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_count": { 2024-04-07T17:51:27.752 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 451 2024-04-07T17:51:27.752 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.752 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes": { 2024-04-07T17:51:27.752 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 676280 2024-04-07T17:51:27.752 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.752 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes_db": { 2024-04-07T17:51:27.752 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 676280 2024-04-07T17:51:27.752 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.752 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes_slow": { 2024-04-07T17:51:27.752 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.752 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.752 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes_wal": { 2024-04-07T17:51:27.752 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.752 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.752 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_count": { 2024-04-07T17:51:27.753 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 112 2024-04-07T17:51:27.753 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.753 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_lat": { 2024-04-07T17:51:27.753 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 451, 2024-04-07T17:51:27.753 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 827027 2024-04-07T17:51:27.753 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.753 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_alloc_lat": { 2024-04-07T17:51:27.753 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:27.753 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.753 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.753 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_total_bytes": { 2024-04-07T17:51:27.753 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.753 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.753 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_used_bytes": { 2024-04-07T17:51:27.753 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.754 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.754 INFO:tasks.workunit.client.0.smithi100.stdout: "truncate_lat": { 2024-04-07T17:51:27.754 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1, 2024-04-07T17:51:27.754 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 646 2024-04-07T17:51:27.754 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.754 INFO:tasks.workunit.client.0.smithi100.stdout: "unlink_lat": { 2024-04-07T17:51:27.754 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 5, 2024-04-07T17:51:27.754 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 8184 2024-04-07T17:51:27.754 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.754 INFO:tasks.workunit.client.0.smithi100.stdout: "wal_alloc_lat": { 2024-04-07T17:51:27.754 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:27.754 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.754 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.754 INFO:tasks.workunit.client.0.smithi100.stdout: "wal_total_bytes": { 2024-04-07T17:51:27.755 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.755 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.755 INFO:tasks.workunit.client.0.smithi100.stdout: "wal_used_bytes": { 2024-04-07T17:51:27.755 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.755 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.755 INFO:tasks.workunit.client.0.smithi100.stdout: "write_bytes": { 2024-04-07T17:51:27.755 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 714424320 2024-04-07T17:51:27.755 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.755 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.755 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore": { 2024-04-07T17:51:27.755 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit": { 2024-04-07T17:51:27.755 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4096 2024-04-07T17:51:27.755 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.755 INFO:tasks.workunit.client.0.smithi100.stdout: "allocated": { 2024-04-07T17:51:27.755 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 14606336 2024-04-07T17:51:27.756 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.756 INFO:tasks.workunit.client.0.smithi100.stdout: "allocator_lat": { 2024-04-07T17:51:27.756 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 16890, 2024-04-07T17:51:27.756 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 74243402 2024-04-07T17:51:27.756 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.756 INFO:tasks.workunit.client.0.smithi100.stdout: "clist_lat": { 2024-04-07T17:51:27.756 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 3441, 2024-04-07T17:51:27.756 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 6473693390 2024-04-07T17:51:27.756 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.756 INFO:tasks.workunit.client.0.smithi100.stdout: "compress_lat": { 2024-04-07T17:51:27.756 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:27.756 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.756 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.756 INFO:tasks.workunit.client.0.smithi100.stdout: "compressed": { 2024-04-07T17:51:27.756 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.757 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.757 INFO:tasks.workunit.client.0.smithi100.stdout: "compressed_allocated": { 2024-04-07T17:51:27.757 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.757 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.757 INFO:tasks.workunit.client.0.smithi100.stdout: "compressed_original": { 2024-04-07T17:51:27.757 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.757 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.757 INFO:tasks.workunit.client.0.smithi100.stdout: "csum_lat": { 2024-04-07T17:51:27.757 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 14729, 2024-04-07T17:51:27.757 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 190648102 2024-04-07T17:51:27.757 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.757 INFO:tasks.workunit.client.0.smithi100.stdout: "decompress_lat": { 2024-04-07T17:51:27.757 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:27.757 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.757 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.758 INFO:tasks.workunit.client.0.smithi100.stdout: "fragmentation_micros": { 2024-04-07T17:51:27.758 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.758 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.758 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_commit_lat": { 2024-04-07T17:51:27.758 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 61461, 2024-04-07T17:51:27.758 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 27465827150 2024-04-07T17:51:27.758 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.758 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_final_lat": { 2024-04-07T17:51:27.758 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 60716, 2024-04-07T17:51:27.758 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3226568509 2024-04-07T17:51:27.758 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.758 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_flush_lat": { 2024-04-07T17:51:27.758 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 61461, 2024-04-07T17:51:27.758 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 74420054 2024-04-07T17:51:27.758 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.759 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_sync_lat": { 2024-04-07T17:51:27.759 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 61461, 2024-04-07T17:51:27.759 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 27540247204 2024-04-07T17:51:27.759 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.759 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_get_keys_lat": { 2024-04-07T17:51:27.759 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:27.759 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.759 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.759 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_get_values_lat": { 2024-04-07T17:51:27.759 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 22581, 2024-04-07T17:51:27.759 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 685980505 2024-04-07T17:51:27.759 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.759 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_lower_bound_lat": { 2024-04-07T17:51:27.759 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 700, 2024-04-07T17:51:27.759 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5026445 2024-04-07T17:51:27.759 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.760 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_next_lat": { 2024-04-07T17:51:27.760 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 10108, 2024-04-07T17:51:27.760 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 12802315 2024-04-07T17:51:27.760 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.760 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_seek_to_first_lat": { 2024-04-07T17:51:27.760 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 43, 2024-04-07T17:51:27.760 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 77153 2024-04-07T17:51:27.760 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.760 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_upper_bound_lat": { 2024-04-07T17:51:27.760 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 10657, 2024-04-07T17:51:27.760 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1970969784 2024-04-07T17:51:27.760 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.760 INFO:tasks.workunit.client.0.smithi100.stdout: "onode_hits": { 2024-04-07T17:51:27.760 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 175572 2024-04-07T17:51:27.760 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.761 INFO:tasks.workunit.client.0.smithi100.stdout: "onode_misses": { 2024-04-07T17:51:27.761 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 18139 2024-04-07T17:51:27.761 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.761 INFO:tasks.workunit.client.0.smithi100.stdout: "read_lat": { 2024-04-07T17:51:27.761 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 11039, 2024-04-07T17:51:27.761 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1962439157 2024-04-07T17:51:27.761 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.761 INFO:tasks.workunit.client.0.smithi100.stdout: "read_onode_meta_lat": { 2024-04-07T17:51:27.761 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 38635, 2024-04-07T17:51:27.761 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 20820077 2024-04-07T17:51:27.761 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.761 INFO:tasks.workunit.client.0.smithi100.stdout: "read_wait_aio_lat": { 2024-04-07T17:51:27.761 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 27596, 2024-04-07T17:51:27.761 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1296711458 2024-04-07T17:51:27.761 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.762 INFO:tasks.workunit.client.0.smithi100.stdout: "reads_with_retries": { 2024-04-07T17:51:27.762 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.762 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.762 INFO:tasks.workunit.client.0.smithi100.stdout: "remove_lat": { 2024-04-07T17:51:27.762 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 11796, 2024-04-07T17:51:27.762 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 656690853 2024-04-07T17:51:27.762 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.762 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_aio_wait_count": { 2024-04-07T17:51:27.762 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.762 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.762 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_committed_kv_count": { 2024-04-07T17:51:27.762 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.762 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.762 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_read_onode_meta_count": { 2024-04-07T17:51:27.762 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.763 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.763 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_read_wait_aio_count": { 2024-04-07T17:51:27.763 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.763 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.763 INFO:tasks.workunit.client.0.smithi100.stdout: "state_aio_wait_lat": { 2024-04-07T17:51:27.763 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 74635, 2024-04-07T17:51:27.763 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 10364611458 2024-04-07T17:51:27.763 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.763 INFO:tasks.workunit.client.0.smithi100.stdout: "state_deferred_aio_wait_lat": { 2024-04-07T17:51:27.763 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 13939, 2024-04-07T17:51:27.763 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1372751186 2024-04-07T17:51:27.763 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.763 INFO:tasks.workunit.client.0.smithi100.stdout: "state_deferred_cleanup_lat": { 2024-04-07T17:51:27.763 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 13938, 2024-04-07T17:51:27.763 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3655450811587 2024-04-07T17:51:27.763 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.764 INFO:tasks.workunit.client.0.smithi100.stdout: "state_deferred_queued_lat": { 2024-04-07T17:51:27.764 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 13939, 2024-04-07T17:51:27.764 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5923221764357 2024-04-07T17:51:27.764 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.764 INFO:tasks.workunit.client.0.smithi100.stdout: "state_done_lat": { 2024-04-07T17:51:27.764 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 74634, 2024-04-07T17:51:27.764 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2803609494529 2024-04-07T17:51:27.764 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.764 INFO:tasks.workunit.client.0.smithi100.stdout: "state_finishing_lat": { 2024-04-07T17:51:27.764 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 74634, 2024-04-07T17:51:27.764 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 20557962 2024-04-07T17:51:27.764 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.764 INFO:tasks.workunit.client.0.smithi100.stdout: "state_io_done_lat": { 2024-04-07T17:51:27.764 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 74635, 2024-04-07T17:51:27.764 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 593824141 2024-04-07T17:51:27.764 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.765 INFO:tasks.workunit.client.0.smithi100.stdout: "state_kv_commiting_lat": { 2024-04-07T17:51:27.765 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 74635, 2024-04-07T17:51:27.765 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 51948333590 2024-04-07T17:51:27.765 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.765 INFO:tasks.workunit.client.0.smithi100.stdout: "state_kv_done_lat": { 2024-04-07T17:51:27.765 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 74635, 2024-04-07T17:51:27.765 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 60647317 2024-04-07T17:51:27.765 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.765 INFO:tasks.workunit.client.0.smithi100.stdout: "state_kv_queued_lat": { 2024-04-07T17:51:27.765 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 74635, 2024-04-07T17:51:27.765 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 28224012224 2024-04-07T17:51:27.765 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.765 INFO:tasks.workunit.client.0.smithi100.stdout: "state_prepare_lat": { 2024-04-07T17:51:27.765 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 74635, 2024-04-07T17:51:27.765 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 21897965540 2024-04-07T17:51:27.766 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.766 INFO:tasks.workunit.client.0.smithi100.stdout: "stored": { 2024-04-07T17:51:27.766 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 10490178 2024-04-07T17:51:27.766 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.766 INFO:tasks.workunit.client.0.smithi100.stdout: "truncate_lat": { 2024-04-07T17:51:27.766 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 2295, 2024-04-07T17:51:27.766 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 25153482 2024-04-07T17:51:27.766 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.766 INFO:tasks.workunit.client.0.smithi100.stdout: "txc_commit_lat": { 2024-04-07T17:51:27.766 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 74635, 2024-04-07T17:51:27.766 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 113055819629 2024-04-07T17:51:27.766 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.766 INFO:tasks.workunit.client.0.smithi100.stdout: "txc_submit_lat": { 2024-04-07T17:51:27.766 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 74635, 2024-04-07T17:51:27.766 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 23968070054 2024-04-07T17:51:27.767 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.767 INFO:tasks.workunit.client.0.smithi100.stdout: "txc_throttle_lat": { 2024-04-07T17:51:27.767 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 74635, 2024-04-07T17:51:27.767 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1130773563 2024-04-07T17:51:27.767 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.767 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.767 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache": { 2024-04-07T17:51:27.767 INFO:tasks.workunit.client.0.smithi100.stdout: "cache_bytes": { 2024-04-07T17:51:27.767 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1444650649 2024-04-07T17:51:27.767 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.767 INFO:tasks.workunit.client.0.smithi100.stdout: "heap_bytes": { 2024-04-07T17:51:27.767 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 423641088 2024-04-07T17:51:27.767 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.767 INFO:tasks.workunit.client.0.smithi100.stdout: "mapped_bytes": { 2024-04-07T17:51:27.767 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 358727680 2024-04-07T17:51:27.768 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.768 INFO:tasks.workunit.client.0.smithi100.stdout: "target_bytes": { 2024-04-07T17:51:27.768 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2647008256 2024-04-07T17:51:27.768 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.768 INFO:tasks.workunit.client.0.smithi100.stdout: "unmapped_bytes": { 2024-04-07T17:51:27.768 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 64913408 2024-04-07T17:51:27.768 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.768 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.768 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:data": { 2024-04-07T17:51:27.768 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:27.768 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 134217728 2024-04-07T17:51:27.768 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.768 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:27.769 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.769 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.769 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:27.769 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.769 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.769 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:27.769 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 59908063 2024-04-07T17:51:27.769 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.769 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:27.769 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.769 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.769 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:27.769 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.769 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.770 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:27.770 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.770 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.770 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:27.770 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.770 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.770 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:27.770 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.770 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.770 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:27.770 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 184320 2024-04-07T17:51:27.770 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.770 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:27.770 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.771 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.771 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:27.771 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.771 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.771 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:27.771 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.771 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.771 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:27.771 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 74125345 2024-04-07T17:51:27.771 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.771 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.771 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:kv": { 2024-04-07T17:51:27.771 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:27.771 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 587202560 2024-04-07T17:51:27.771 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.771 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:27.772 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 67108864 2024-04-07T17:51:27.772 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.772 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:27.772 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.772 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.772 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:27.772 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 449310472 2024-04-07T17:51:27.772 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.772 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:27.772 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.772 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.772 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:27.772 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.772 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.772 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:27.772 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1040 2024-04-07T17:51:27.773 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.773 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:27.773 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.773 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.773 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:27.773 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 66384 2024-04-07T17:51:27.773 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.773 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:27.773 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.773 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.773 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:27.773 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.773 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.773 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:27.773 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.774 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.774 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:27.774 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.774 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.774 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:27.774 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 70715800 2024-04-07T17:51:27.774 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.774 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.774 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:kv_onode": { 2024-04-07T17:51:27.774 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:27.774 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 176160768 2024-04-07T17:51:27.774 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.774 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:27.774 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 67108864 2024-04-07T17:51:27.774 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.775 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:27.775 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 691824 2024-04-07T17:51:27.775 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.775 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:27.775 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 39938708 2024-04-07T17:51:27.775 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.775 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:27.775 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.775 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.775 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:27.775 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.775 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.775 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:27.775 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.775 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.776 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:27.776 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.776 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.776 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:27.776 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.776 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.776 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:27.776 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.776 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.776 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:27.776 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.776 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.776 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:27.776 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.776 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.777 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:27.777 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.777 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.777 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:27.777 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 68421372 2024-04-07T17:51:27.777 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.777 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.777 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:meta": { 2024-04-07T17:51:27.777 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:27.777 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 520093696 2024-04-07T17:51:27.777 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.777 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:27.777 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.777 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.777 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:27.777 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.778 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.778 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:27.778 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 449310472 2024-04-07T17:51:27.778 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.778 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:27.778 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.778 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.778 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:27.778 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.778 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.778 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:27.778 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.778 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.778 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:27.778 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.779 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.779 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:27.779 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 157124 2024-04-07T17:51:27.779 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.779 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:27.779 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1119133 2024-04-07T17:51:27.779 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.779 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:27.779 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.779 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.779 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:27.779 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.779 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.779 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:27.779 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.780 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.780 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:27.780 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 69506967 2024-04-07T17:51:27.780 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.780 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.780 INFO:tasks.workunit.client.0.smithi100.stdout: "objecter": { 2024-04-07T17:51:27.780 INFO:tasks.workunit.client.0.smithi100.stdout: "op_active": { 2024-04-07T17:51:27.780 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.780 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.780 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r": { 2024-04-07T17:51:27.780 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5 2024-04-07T17:51:27.780 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.780 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rmw": { 2024-04-07T17:51:27.780 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.780 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.780 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w": { 2024-04-07T17:51:27.780 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.781 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.781 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.781 INFO:tasks.workunit.client.0.smithi100.stdout: "osd": { 2024-04-07T17:51:27.781 INFO:tasks.workunit.client.0.smithi100.stdout: "numpg": { 2024-04-07T17:51:27.781 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 54 2024-04-07T17:51:27.781 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.781 INFO:tasks.workunit.client.0.smithi100.stdout: "numpg_removing": { 2024-04-07T17:51:27.781 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.781 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.781 INFO:tasks.workunit.client.0.smithi100.stdout: "op": { 2024-04-07T17:51:27.781 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 33509 2024-04-07T17:51:27.781 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.781 INFO:tasks.workunit.client.0.smithi100.stdout: "op_before_queue_op_lat": { 2024-04-07T17:51:27.781 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 91089, 2024-04-07T17:51:27.781 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 12791557788 2024-04-07T17:51:27.781 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.781 INFO:tasks.workunit.client.0.smithi100.stdout: "op_delayed_degraded": { 2024-04-07T17:51:27.782 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.782 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.782 INFO:tasks.workunit.client.0.smithi100.stdout: "op_delayed_unreadable": { 2024-04-07T17:51:27.782 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.782 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.782 INFO:tasks.workunit.client.0.smithi100.stdout: "op_in_bytes": { 2024-04-07T17:51:27.782 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1535388602 2024-04-07T17:51:27.782 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.782 INFO:tasks.workunit.client.0.smithi100.stdout: "op_latency": { 2024-04-07T17:51:27.782 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 33509, 2024-04-07T17:51:27.782 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 131541624216 2024-04-07T17:51:27.782 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.782 INFO:tasks.workunit.client.0.smithi100.stdout: "op_out_bytes": { 2024-04-07T17:51:27.782 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1703112431 2024-04-07T17:51:27.783 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.783 INFO:tasks.workunit.client.0.smithi100.stdout: "op_prepare_latency": { 2024-04-07T17:51:27.783 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 34219, 2024-04-07T17:51:27.783 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 27830378329 2024-04-07T17:51:27.783 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.783 INFO:tasks.workunit.client.0.smithi100.stdout: "op_process_latency": { 2024-04-07T17:51:27.783 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 33509, 2024-04-07T17:51:27.783 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 53861705532 2024-04-07T17:51:27.783 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.783 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r": { 2024-04-07T17:51:27.783 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 25245 2024-04-07T17:51:27.783 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.783 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_latency": { 2024-04-07T17:51:27.783 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 25245, 2024-04-07T17:51:27.783 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 23376821581 2024-04-07T17:51:27.783 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.784 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_latency_out_bytes_histogram": { 2024-04-07T17:51:27.784 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.784 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.784 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_out_bytes": { 2024-04-07T17:51:27.784 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1703112431 2024-04-07T17:51:27.784 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.784 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_prepare_latency": { 2024-04-07T17:51:27.784 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 25399, 2024-04-07T17:51:27.784 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 11923589716 2024-04-07T17:51:27.784 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.784 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_process_latency": { 2024-04-07T17:51:27.784 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 25245, 2024-04-07T17:51:27.784 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 10146174967 2024-04-07T17:51:27.784 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.784 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw": { 2024-04-07T17:51:27.784 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4064 2024-04-07T17:51:27.785 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.785 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_in_bytes": { 2024-04-07T17:51:27.785 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 290840421 2024-04-07T17:51:27.785 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.785 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_latency": { 2024-04-07T17:51:27.785 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 4064, 2024-04-07T17:51:27.785 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 20965538275 2024-04-07T17:51:27.785 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.785 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_latency_in_bytes_histogram": { 2024-04-07T17:51:27.785 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.785 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.785 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_latency_out_bytes_histogram": { 2024-04-07T17:51:27.785 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.785 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.786 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_out_bytes": { 2024-04-07T17:51:27.786 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.786 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.786 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_prepare_latency": { 2024-04-07T17:51:27.786 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 4504, 2024-04-07T17:51:27.786 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 6305172918 2024-04-07T17:51:27.786 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.786 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_process_latency": { 2024-04-07T17:51:27.786 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 4064, 2024-04-07T17:51:27.786 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 14864893688 2024-04-07T17:51:27.786 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.786 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w": { 2024-04-07T17:51:27.786 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4200 2024-04-07T17:51:27.786 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.787 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_in_bytes": { 2024-04-07T17:51:27.787 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1244548181 2024-04-07T17:51:27.787 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.787 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_latency": { 2024-04-07T17:51:27.787 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 4200, 2024-04-07T17:51:27.787 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 87199264360 2024-04-07T17:51:27.787 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.787 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_latency_in_bytes_histogram": { 2024-04-07T17:51:27.787 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.787 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.787 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_prepare_latency": { 2024-04-07T17:51:27.787 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 4316, 2024-04-07T17:51:27.787 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 9601615695 2024-04-07T17:51:27.787 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.787 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_process_latency": { 2024-04-07T17:51:27.787 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 4200, 2024-04-07T17:51:27.788 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 28850636877 2024-04-07T17:51:27.788 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.788 INFO:tasks.workunit.client.0.smithi100.stdout: "op_wip": { 2024-04-07T17:51:27.788 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.788 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.788 INFO:tasks.workunit.client.0.smithi100.stdout: "recovery_bytes": { 2024-04-07T17:51:27.788 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 73766334 2024-04-07T17:51:27.788 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.788 INFO:tasks.workunit.client.0.smithi100.stdout: "recovery_ops": { 2024-04-07T17:51:27.788 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 34 2024-04-07T17:51:27.788 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.788 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_bytes": { 2024-04-07T17:51:27.788 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 95995035648 2024-04-07T17:51:27.788 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.788 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_bytes_used": { 2024-04-07T17:51:27.788 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 358027264 2024-04-07T17:51:27.789 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.789 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.789 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.789 INFO:tasks.workunit.client.0.smithi100.stdout: "osd.7": { 2024-04-07T17:51:27.789 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs": { 2024-04-07T17:51:27.789 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_db_max_lat": { 2024-04-07T17:51:27.789 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1425185 2024-04-07T17:51:27.789 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.789 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_slow_fallback": { 2024-04-07T17:51:27.789 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.789 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.789 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_slow_max_lat": { 2024-04-07T17:51:27.789 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.789 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.789 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_slow_size_fallback": { 2024-04-07T17:51:27.789 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.790 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.790 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit_db": { 2024-04-07T17:51:27.790 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 65536 2024-04-07T17:51:27.790 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.790 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit_slow": { 2024-04-07T17:51:27.790 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.790 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.790 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit_wal": { 2024-04-07T17:51:27.790 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.790 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.790 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_wal_max_lat": { 2024-04-07T17:51:27.790 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.790 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.790 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_written_slow": { 2024-04-07T17:51:27.790 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.790 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.791 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_written_sst": { 2024-04-07T17:51:27.791 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5050368 2024-04-07T17:51:27.791 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.791 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_written_wal": { 2024-04-07T17:51:27.791 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 495513600 2024-04-07T17:51:27.791 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.791 INFO:tasks.workunit.client.0.smithi100.stdout: "compact_lat": { 2024-04-07T17:51:27.791 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 15, 2024-04-07T17:51:27.791 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 6850825 2024-04-07T17:51:27.791 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.791 INFO:tasks.workunit.client.0.smithi100.stdout: "compact_lock_lat": { 2024-04-07T17:51:27.791 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 15, 2024-04-07T17:51:27.791 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1769094 2024-04-07T17:51:27.791 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.791 INFO:tasks.workunit.client.0.smithi100.stdout: "db_alloc_lat": { 2024-04-07T17:51:27.791 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 148, 2024-04-07T17:51:27.792 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2568628 2024-04-07T17:51:27.792 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.792 INFO:tasks.workunit.client.0.smithi100.stdout: "db_total_bytes": { 2024-04-07T17:51:27.792 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 95995027456 2024-04-07T17:51:27.792 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.792 INFO:tasks.workunit.client.0.smithi100.stdout: "db_used_bytes": { 2024-04-07T17:51:27.792 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 411107328 2024-04-07T17:51:27.792 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.792 INFO:tasks.workunit.client.0.smithi100.stdout: "flush_lat": { 2024-04-07T17:51:27.792 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 63017, 2024-04-07T17:51:27.792 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 12122188924 2024-04-07T17:51:27.792 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.792 INFO:tasks.workunit.client.0.smithi100.stdout: "fsync_lat": { 2024-04-07T17:51:27.792 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 63028, 2024-04-07T17:51:27.792 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 21247379624 2024-04-07T17:51:27.792 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.792 INFO:tasks.workunit.client.0.smithi100.stdout: "log_bytes": { 2024-04-07T17:51:27.793 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 8843264 2024-04-07T17:51:27.793 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.793 INFO:tasks.workunit.client.0.smithi100.stdout: "logged_bytes": { 2024-04-07T17:51:27.793 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 258203648 2024-04-07T17:51:27.793 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.793 INFO:tasks.workunit.client.0.smithi100.stdout: "max_bytes_db": { 2024-04-07T17:51:27.793 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 423886848 2024-04-07T17:51:27.793 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.793 INFO:tasks.workunit.client.0.smithi100.stdout: "max_bytes_slow": { 2024-04-07T17:51:27.793 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.793 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.793 INFO:tasks.workunit.client.0.smithi100.stdout: "max_bytes_wal": { 2024-04-07T17:51:27.793 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.793 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.793 INFO:tasks.workunit.client.0.smithi100.stdout: "num_files": { 2024-04-07T17:51:27.794 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 38 2024-04-07T17:51:27.794 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.794 INFO:tasks.workunit.client.0.smithi100.stdout: "read_bytes": { 2024-04-07T17:51:27.794 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3167491823 2024-04-07T17:51:27.794 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.794 INFO:tasks.workunit.client.0.smithi100.stdout: "read_count": { 2024-04-07T17:51:27.794 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 40027 2024-04-07T17:51:27.794 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.794 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes": { 2024-04-07T17:51:27.794 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2790236160 2024-04-07T17:51:27.794 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.794 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes_db": { 2024-04-07T17:51:27.794 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2790240256 2024-04-07T17:51:27.794 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.794 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes_slow": { 2024-04-07T17:51:27.795 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.795 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.795 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes_wal": { 2024-04-07T17:51:27.795 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.795 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.795 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_count": { 2024-04-07T17:51:27.795 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 25974 2024-04-07T17:51:27.795 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.795 INFO:tasks.workunit.client.0.smithi100.stdout: "read_lat": { 2024-04-07T17:51:27.795 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 40027, 2024-04-07T17:51:27.795 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1975651191 2024-04-07T17:51:27.795 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.795 INFO:tasks.workunit.client.0.smithi100.stdout: "read_prefetch_bytes": { 2024-04-07T17:51:27.795 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3056697503 2024-04-07T17:51:27.795 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.796 INFO:tasks.workunit.client.0.smithi100.stdout: "read_prefetch_count": { 2024-04-07T17:51:27.796 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 36631 2024-04-07T17:51:27.796 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.796 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_buffer_bytes": { 2024-04-07T17:51:27.796 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1782285 2024-04-07T17:51:27.796 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.796 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_buffer_count": { 2024-04-07T17:51:27.796 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 538 2024-04-07T17:51:27.796 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.796 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_bytes": { 2024-04-07T17:51:27.796 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5008896 2024-04-07T17:51:27.796 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.796 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_count": { 2024-04-07T17:51:27.796 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 703 2024-04-07T17:51:27.796 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.797 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes": { 2024-04-07T17:51:27.797 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3226611 2024-04-07T17:51:27.797 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.797 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes_db": { 2024-04-07T17:51:27.797 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3226611 2024-04-07T17:51:27.797 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.797 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes_slow": { 2024-04-07T17:51:27.797 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.797 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.797 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes_wal": { 2024-04-07T17:51:27.797 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.797 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.797 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_count": { 2024-04-07T17:51:27.797 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 167 2024-04-07T17:51:27.797 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.798 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_lat": { 2024-04-07T17:51:27.798 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 703, 2024-04-07T17:51:27.798 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1945572 2024-04-07T17:51:27.798 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.798 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_alloc_lat": { 2024-04-07T17:51:27.798 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:27.798 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.798 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.798 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_total_bytes": { 2024-04-07T17:51:27.798 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.798 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.798 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_used_bytes": { 2024-04-07T17:51:27.798 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.798 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.798 INFO:tasks.workunit.client.0.smithi100.stdout: "truncate_lat": { 2024-04-07T17:51:27.799 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1, 2024-04-07T17:51:27.799 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 858 2024-04-07T17:51:27.799 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.799 INFO:tasks.workunit.client.0.smithi100.stdout: "unlink_lat": { 2024-04-07T17:51:27.799 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 9, 2024-04-07T17:51:27.799 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 12676 2024-04-07T17:51:27.799 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.799 INFO:tasks.workunit.client.0.smithi100.stdout: "wal_alloc_lat": { 2024-04-07T17:51:27.799 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:27.799 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.799 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.799 INFO:tasks.workunit.client.0.smithi100.stdout: "wal_total_bytes": { 2024-04-07T17:51:27.799 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.799 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.799 INFO:tasks.workunit.client.0.smithi100.stdout: "wal_used_bytes": { 2024-04-07T17:51:27.800 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.800 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.800 INFO:tasks.workunit.client.0.smithi100.stdout: "write_bytes": { 2024-04-07T17:51:27.800 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 761053184 2024-04-07T17:51:27.800 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.800 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.800 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore": { 2024-04-07T17:51:27.800 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit": { 2024-04-07T17:51:27.800 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4096 2024-04-07T17:51:27.800 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.800 INFO:tasks.workunit.client.0.smithi100.stdout: "allocated": { 2024-04-07T17:51:27.800 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 10403840 2024-04-07T17:51:27.800 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.800 INFO:tasks.workunit.client.0.smithi100.stdout: "allocator_lat": { 2024-04-07T17:51:27.800 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 17672, 2024-04-07T17:51:27.801 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 84277754 2024-04-07T17:51:27.801 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.801 INFO:tasks.workunit.client.0.smithi100.stdout: "clist_lat": { 2024-04-07T17:51:27.801 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 3466, 2024-04-07T17:51:27.801 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 6069941459 2024-04-07T17:51:27.801 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.801 INFO:tasks.workunit.client.0.smithi100.stdout: "compress_lat": { 2024-04-07T17:51:27.801 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:27.801 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.801 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.801 INFO:tasks.workunit.client.0.smithi100.stdout: "compressed": { 2024-04-07T17:51:27.801 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.801 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.801 INFO:tasks.workunit.client.0.smithi100.stdout: "compressed_allocated": { 2024-04-07T17:51:27.801 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.802 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.802 INFO:tasks.workunit.client.0.smithi100.stdout: "compressed_original": { 2024-04-07T17:51:27.802 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.802 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.802 INFO:tasks.workunit.client.0.smithi100.stdout: "csum_lat": { 2024-04-07T17:51:27.802 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 16446, 2024-04-07T17:51:27.802 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 220970389 2024-04-07T17:51:27.802 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.802 INFO:tasks.workunit.client.0.smithi100.stdout: "decompress_lat": { 2024-04-07T17:51:27.802 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:27.802 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.802 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.802 INFO:tasks.workunit.client.0.smithi100.stdout: "fragmentation_micros": { 2024-04-07T17:51:27.802 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.802 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.803 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_commit_lat": { 2024-04-07T17:51:27.803 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 62983, 2024-04-07T17:51:27.803 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 27504738088 2024-04-07T17:51:27.803 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.803 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_final_lat": { 2024-04-07T17:51:27.803 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 62244, 2024-04-07T17:51:27.803 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3517645071 2024-04-07T17:51:27.803 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.803 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_flush_lat": { 2024-04-07T17:51:27.803 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 62983, 2024-04-07T17:51:27.803 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 96259530 2024-04-07T17:51:27.803 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.803 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_sync_lat": { 2024-04-07T17:51:27.803 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 62983, 2024-04-07T17:51:27.803 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 27600997618 2024-04-07T17:51:27.804 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.804 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_get_keys_lat": { 2024-04-07T17:51:27.804 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:27.804 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.804 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.804 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_get_values_lat": { 2024-04-07T17:51:27.804 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 33493, 2024-04-07T17:51:27.804 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 950805853 2024-04-07T17:51:27.804 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.804 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_lower_bound_lat": { 2024-04-07T17:51:27.804 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 632, 2024-04-07T17:51:27.804 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4331297 2024-04-07T17:51:27.804 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.804 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_next_lat": { 2024-04-07T17:51:27.805 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 7000, 2024-04-07T17:51:27.805 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 13697854 2024-04-07T17:51:27.805 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.805 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_seek_to_first_lat": { 2024-04-07T17:51:27.805 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 50, 2024-04-07T17:51:27.805 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 102727 2024-04-07T17:51:27.805 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.805 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_upper_bound_lat": { 2024-04-07T17:51:27.805 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 11201, 2024-04-07T17:51:27.805 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1557562272 2024-04-07T17:51:27.805 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.805 INFO:tasks.workunit.client.0.smithi100.stdout: "onode_hits": { 2024-04-07T17:51:27.805 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 194378 2024-04-07T17:51:27.805 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.805 INFO:tasks.workunit.client.0.smithi100.stdout: "onode_misses": { 2024-04-07T17:51:27.805 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 17489 2024-04-07T17:51:27.806 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.806 INFO:tasks.workunit.client.0.smithi100.stdout: "read_lat": { 2024-04-07T17:51:27.806 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 9939, 2024-04-07T17:51:27.806 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1593794096 2024-04-07T17:51:27.806 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.806 INFO:tasks.workunit.client.0.smithi100.stdout: "read_onode_meta_lat": { 2024-04-07T17:51:27.806 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 39993, 2024-04-07T17:51:27.806 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 19793062 2024-04-07T17:51:27.806 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.806 INFO:tasks.workunit.client.0.smithi100.stdout: "read_wait_aio_lat": { 2024-04-07T17:51:27.806 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 30054, 2024-04-07T17:51:27.806 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1325463873 2024-04-07T17:51:27.806 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.806 INFO:tasks.workunit.client.0.smithi100.stdout: "reads_with_retries": { 2024-04-07T17:51:27.807 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.807 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.807 INFO:tasks.workunit.client.0.smithi100.stdout: "remove_lat": { 2024-04-07T17:51:27.807 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 13264, 2024-04-07T17:51:27.807 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 667186707 2024-04-07T17:51:27.807 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.807 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_aio_wait_count": { 2024-04-07T17:51:27.807 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.807 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.807 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_committed_kv_count": { 2024-04-07T17:51:27.807 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.807 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.807 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_read_onode_meta_count": { 2024-04-07T17:51:27.807 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.808 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.808 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_read_wait_aio_count": { 2024-04-07T17:51:27.808 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.808 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.808 INFO:tasks.workunit.client.0.smithi100.stdout: "state_aio_wait_lat": { 2024-04-07T17:51:27.808 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 74429, 2024-04-07T17:51:27.808 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 8392301592 2024-04-07T17:51:27.808 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.808 INFO:tasks.workunit.client.0.smithi100.stdout: "state_deferred_aio_wait_lat": { 2024-04-07T17:51:27.808 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 16391, 2024-04-07T17:51:27.808 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1329233809 2024-04-07T17:51:27.808 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.808 INFO:tasks.workunit.client.0.smithi100.stdout: "state_deferred_cleanup_lat": { 2024-04-07T17:51:27.808 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 16390, 2024-04-07T17:51:27.808 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4040344878498 2024-04-07T17:51:27.808 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.809 INFO:tasks.workunit.client.0.smithi100.stdout: "state_deferred_queued_lat": { 2024-04-07T17:51:27.809 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 16391, 2024-04-07T17:51:27.809 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 6763629312230 2024-04-07T17:51:27.809 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.809 INFO:tasks.workunit.client.0.smithi100.stdout: "state_done_lat": { 2024-04-07T17:51:27.809 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 74428, 2024-04-07T17:51:27.809 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2916516169130 2024-04-07T17:51:27.809 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.809 INFO:tasks.workunit.client.0.smithi100.stdout: "state_finishing_lat": { 2024-04-07T17:51:27.809 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 74428, 2024-04-07T17:51:27.809 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 17625368 2024-04-07T17:51:27.809 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.809 INFO:tasks.workunit.client.0.smithi100.stdout: "state_io_done_lat": { 2024-04-07T17:51:27.809 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 74429, 2024-04-07T17:51:27.809 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 362782490 2024-04-07T17:51:27.809 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.810 INFO:tasks.workunit.client.0.smithi100.stdout: "state_kv_commiting_lat": { 2024-04-07T17:51:27.810 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 74429, 2024-04-07T17:51:27.810 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 49654142030 2024-04-07T17:51:27.810 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.810 INFO:tasks.workunit.client.0.smithi100.stdout: "state_kv_done_lat": { 2024-04-07T17:51:27.810 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 74429, 2024-04-07T17:51:27.810 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 57857521 2024-04-07T17:51:27.810 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.810 INFO:tasks.workunit.client.0.smithi100.stdout: "state_kv_queued_lat": { 2024-04-07T17:51:27.810 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 74429, 2024-04-07T17:51:27.810 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 25576588902 2024-04-07T17:51:27.810 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.810 INFO:tasks.workunit.client.0.smithi100.stdout: "state_prepare_lat": { 2024-04-07T17:51:27.810 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 74429, 2024-04-07T17:51:27.811 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 18974977018 2024-04-07T17:51:27.811 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.811 INFO:tasks.workunit.client.0.smithi100.stdout: "stored": { 2024-04-07T17:51:27.811 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 6290654 2024-04-07T17:51:27.811 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.811 INFO:tasks.workunit.client.0.smithi100.stdout: "truncate_lat": { 2024-04-07T17:51:27.811 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 2242, 2024-04-07T17:51:27.811 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 22293375 2024-04-07T17:51:27.811 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.811 INFO:tasks.workunit.client.0.smithi100.stdout: "txc_commit_lat": { 2024-04-07T17:51:27.811 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 74429, 2024-04-07T17:51:27.811 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 102987546391 2024-04-07T17:51:27.811 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.811 INFO:tasks.workunit.client.0.smithi100.stdout: "txc_submit_lat": { 2024-04-07T17:51:27.811 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 74429, 2024-04-07T17:51:27.811 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 21153024583 2024-04-07T17:51:27.812 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.812 INFO:tasks.workunit.client.0.smithi100.stdout: "txc_throttle_lat": { 2024-04-07T17:51:27.812 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 74429, 2024-04-07T17:51:27.812 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 946122269 2024-04-07T17:51:27.812 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.812 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.812 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache": { 2024-04-07T17:51:27.812 INFO:tasks.workunit.client.0.smithi100.stdout: "cache_bytes": { 2024-04-07T17:51:27.812 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1444650649 2024-04-07T17:51:27.812 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.812 INFO:tasks.workunit.client.0.smithi100.stdout: "heap_bytes": { 2024-04-07T17:51:27.812 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 463331328 2024-04-07T17:51:27.812 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.812 INFO:tasks.workunit.client.0.smithi100.stdout: "mapped_bytes": { 2024-04-07T17:51:27.812 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 430809088 2024-04-07T17:51:27.812 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.813 INFO:tasks.workunit.client.0.smithi100.stdout: "target_bytes": { 2024-04-07T17:51:27.813 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2647008256 2024-04-07T17:51:27.813 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.813 INFO:tasks.workunit.client.0.smithi100.stdout: "unmapped_bytes": { 2024-04-07T17:51:27.813 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 32522240 2024-04-07T17:51:27.813 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.813 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.813 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:data": { 2024-04-07T17:51:27.813 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:27.813 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 134217728 2024-04-07T17:51:27.813 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.813 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:27.813 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.813 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.814 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:27.814 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.814 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.814 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:27.814 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 59553061 2024-04-07T17:51:27.814 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.814 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:27.814 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.814 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.814 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:27.814 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.814 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.814 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:27.814 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.814 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.814 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:27.815 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.815 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.815 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:27.815 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 32768 2024-04-07T17:51:27.815 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.815 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:27.815 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 61440 2024-04-07T17:51:27.815 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.815 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:27.815 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.815 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.815 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:27.815 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.815 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.815 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:27.816 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.816 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.816 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:27.816 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 74570459 2024-04-07T17:51:27.816 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.816 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.816 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:kv": { 2024-04-07T17:51:27.816 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:27.816 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 587202560 2024-04-07T17:51:27.816 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.816 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:27.816 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 67108864 2024-04-07T17:51:27.816 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.816 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:27.816 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.817 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.817 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:27.817 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 446647962 2024-04-07T17:51:27.817 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.817 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:27.817 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.817 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.817 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:27.817 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.817 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.817 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:27.817 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1504 2024-04-07T17:51:27.817 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.817 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:27.817 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.817 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.818 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:27.818 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 95456 2024-04-07T17:51:27.818 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.818 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:27.818 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.818 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.818 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:27.818 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.818 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.818 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:27.818 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.818 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.818 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:27.818 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.818 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.819 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:27.819 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 73348774 2024-04-07T17:51:27.819 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.819 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.819 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:kv_onode": { 2024-04-07T17:51:27.819 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:27.819 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 184549376 2024-04-07T17:51:27.819 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.819 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:27.819 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 67108864 2024-04-07T17:51:27.819 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.819 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:27.819 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4539488 2024-04-07T17:51:27.819 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.819 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:27.820 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 39702041 2024-04-07T17:51:27.820 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.820 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:27.820 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.820 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.820 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:27.820 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.820 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.820 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:27.820 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.820 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.820 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:27.820 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.820 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.820 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:27.820 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.821 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.821 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:27.821 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.821 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.821 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:27.821 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.821 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.821 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:27.821 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.821 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.821 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:27.821 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.821 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.821 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:27.821 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 73198983 2024-04-07T17:51:27.822 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.822 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.822 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:meta": { 2024-04-07T17:51:27.822 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:27.822 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 520093696 2024-04-07T17:51:27.822 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.822 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:27.822 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.822 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.822 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:27.822 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.823 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.823 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:27.823 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 446647962 2024-04-07T17:51:27.823 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.823 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:27.823 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.823 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.823 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:27.823 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.823 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.823 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:27.823 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.823 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.823 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:27.823 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.824 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.824 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:27.824 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 155289 2024-04-07T17:51:27.824 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.824 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:27.824 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1107933 2024-04-07T17:51:27.824 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.824 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:27.824 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.824 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.824 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:27.824 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.824 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.824 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:27.824 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.825 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.825 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:27.825 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 72182512 2024-04-07T17:51:27.825 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.825 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.825 INFO:tasks.workunit.client.0.smithi100.stdout: "objecter": { 2024-04-07T17:51:27.825 INFO:tasks.workunit.client.0.smithi100.stdout: "op_active": { 2024-04-07T17:51:27.825 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.825 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.825 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r": { 2024-04-07T17:51:27.825 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 23 2024-04-07T17:51:27.825 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.825 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rmw": { 2024-04-07T17:51:27.825 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.825 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.825 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w": { 2024-04-07T17:51:27.826 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.826 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.826 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.826 INFO:tasks.workunit.client.0.smithi100.stdout: "osd": { 2024-04-07T17:51:27.826 INFO:tasks.workunit.client.0.smithi100.stdout: "numpg": { 2024-04-07T17:51:27.826 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 58 2024-04-07T17:51:27.826 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.826 INFO:tasks.workunit.client.0.smithi100.stdout: "numpg_removing": { 2024-04-07T17:51:27.826 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.826 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.826 INFO:tasks.workunit.client.0.smithi100.stdout: "op": { 2024-04-07T17:51:27.826 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 40489 2024-04-07T17:51:27.826 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.826 INFO:tasks.workunit.client.0.smithi100.stdout: "op_before_queue_op_lat": { 2024-04-07T17:51:27.827 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 108065, 2024-04-07T17:51:27.827 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 15066892416 2024-04-07T17:51:27.827 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.827 INFO:tasks.workunit.client.0.smithi100.stdout: "op_delayed_degraded": { 2024-04-07T17:51:27.827 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.827 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.827 INFO:tasks.workunit.client.0.smithi100.stdout: "op_delayed_unreadable": { 2024-04-07T17:51:27.827 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.827 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.827 INFO:tasks.workunit.client.0.smithi100.stdout: "op_in_bytes": { 2024-04-07T17:51:27.827 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1416321031 2024-04-07T17:51:27.827 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.827 INFO:tasks.workunit.client.0.smithi100.stdout: "op_latency": { 2024-04-07T17:51:27.827 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 40489, 2024-04-07T17:51:27.827 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 116806238602 2024-04-07T17:51:27.828 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.828 INFO:tasks.workunit.client.0.smithi100.stdout: "op_out_bytes": { 2024-04-07T17:51:27.828 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1572109991 2024-04-07T17:51:27.828 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.828 INFO:tasks.workunit.client.0.smithi100.stdout: "op_prepare_latency": { 2024-04-07T17:51:27.828 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 40866, 2024-04-07T17:51:27.828 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 41965414839 2024-04-07T17:51:27.828 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.828 INFO:tasks.workunit.client.0.smithi100.stdout: "op_process_latency": { 2024-04-07T17:51:27.828 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 40489, 2024-04-07T17:51:27.828 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 66194385741 2024-04-07T17:51:27.828 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.828 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r": { 2024-04-07T17:51:27.828 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 27940 2024-04-07T17:51:27.828 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.829 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_latency": { 2024-04-07T17:51:27.829 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 27940, 2024-04-07T17:51:27.829 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 25174152653 2024-04-07T17:51:27.829 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.829 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_latency_out_bytes_histogram": { 2024-04-07T17:51:27.829 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.829 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.829 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_out_bytes": { 2024-04-07T17:51:27.829 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1572109991 2024-04-07T17:51:27.829 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.829 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_prepare_latency": { 2024-04-07T17:51:27.829 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 28058, 2024-04-07T17:51:27.829 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 13058211260 2024-04-07T17:51:27.829 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.829 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_process_latency": { 2024-04-07T17:51:27.830 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 27940, 2024-04-07T17:51:27.830 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 11404588197 2024-04-07T17:51:27.830 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.830 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw": { 2024-04-07T17:51:27.830 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 11358 2024-04-07T17:51:27.830 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.830 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_in_bytes": { 2024-04-07T17:51:27.830 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 298732687 2024-04-07T17:51:27.830 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.830 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_latency": { 2024-04-07T17:51:27.830 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 11358, 2024-04-07T17:51:27.830 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 46852373505 2024-04-07T17:51:27.830 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.830 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_latency_in_bytes_histogram": { 2024-04-07T17:51:27.831 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.831 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.831 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_latency_out_bytes_histogram": { 2024-04-07T17:51:27.831 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.831 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.831 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_out_bytes": { 2024-04-07T17:51:27.831 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.831 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.831 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_prepare_latency": { 2024-04-07T17:51:27.831 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 11546, 2024-04-07T17:51:27.831 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 24931969765 2024-04-07T17:51:27.831 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.831 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_process_latency": { 2024-04-07T17:51:27.831 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 11358, 2024-04-07T17:51:27.831 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 41982228701 2024-04-07T17:51:27.832 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.832 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w": { 2024-04-07T17:51:27.832 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1191 2024-04-07T17:51:27.832 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.832 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_in_bytes": { 2024-04-07T17:51:27.832 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1117588344 2024-04-07T17:51:27.832 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.832 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_latency": { 2024-04-07T17:51:27.832 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1191, 2024-04-07T17:51:27.832 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 44779712444 2024-04-07T17:51:27.832 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.832 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_latency_in_bytes_histogram": { 2024-04-07T17:51:27.832 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.832 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.832 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_prepare_latency": { 2024-04-07T17:51:27.833 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1262, 2024-04-07T17:51:27.833 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3975233814 2024-04-07T17:51:27.833 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.833 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_process_latency": { 2024-04-07T17:51:27.833 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1191, 2024-04-07T17:51:27.833 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 12807568843 2024-04-07T17:51:27.833 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.833 INFO:tasks.workunit.client.0.smithi100.stdout: "op_wip": { 2024-04-07T17:51:27.833 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.833 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.833 INFO:tasks.workunit.client.0.smithi100.stdout: "recovery_bytes": { 2024-04-07T17:51:27.833 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:27.833 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.833 INFO:tasks.workunit.client.0.smithi100.stdout: "recovery_ops": { 2024-04-07T17:51:27.833 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2 2024-04-07T17:51:27.834 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.834 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_bytes": { 2024-04-07T17:51:27.834 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 95995035648 2024-04-07T17:51:27.834 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.834 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_bytes_used": { 2024-04-07T17:51:27.834 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 421523456 2024-04-07T17:51:27.834 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.834 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.834 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.834 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.834 INFO:tasks.workunit.client.0.smithi100.stdout: "pools": [ 2024-04-07T17:51:27.834 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:27.834 INFO:tasks.workunit.client.0.smithi100.stdout: "application": [ 2024-04-07T17:51:27.834 INFO:tasks.workunit.client.0.smithi100.stdout: "mgr" 2024-04-07T17:51:27.834 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:27.835 INFO:tasks.workunit.client.0.smithi100.stdout: "cache_mode": "none", 2024-04-07T17:51:27.835 INFO:tasks.workunit.client.0.smithi100.stdout: "erasure_code_profile": {}, 2024-04-07T17:51:27.835 INFO:tasks.workunit.client.0.smithi100.stdout: "flags_names": [ 2024-04-07T17:51:27.835 INFO:tasks.workunit.client.0.smithi100.stdout: "hashpspool" 2024-04-07T17:51:27.835 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:27.835 INFO:tasks.workunit.client.0.smithi100.stdout: "min_size": 2, 2024-04-07T17:51:27.835 INFO:tasks.workunit.client.0.smithi100.stdout: "options": {}, 2024-04-07T17:51:27.835 INFO:tasks.workunit.client.0.smithi100.stdout: "pg_autoscale_mode": "off", 2024-04-07T17:51:27.835 INFO:tasks.workunit.client.0.smithi100.stdout: "pg_num": 1, 2024-04-07T17:51:27.835 INFO:tasks.workunit.client.0.smithi100.stdout: "pgp_num": 1, 2024-04-07T17:51:27.835 INFO:tasks.workunit.client.0.smithi100.stdout: "pool": 1, 2024-04-07T17:51:27.835 INFO:tasks.workunit.client.0.smithi100.stdout: "size": 3, 2024-04-07T17:51:27.835 INFO:tasks.workunit.client.0.smithi100.stdout: "stats": { 2024-04-07T17:51:27.835 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_raw": 723597404426, 2024-04-07T17:51:27.836 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_used": 1781760, 2024-04-07T17:51:27.836 INFO:tasks.workunit.client.0.smithi100.stdout: "compress_bytes_used": 0, 2024-04-07T17:51:27.836 INFO:tasks.workunit.client.0.smithi100.stdout: "compress_under_bytes": 0, 2024-04-07T17:51:27.836 INFO:tasks.workunit.client.0.smithi100.stdout: "data_bytes_used": 1781760, 2024-04-07T17:51:27.836 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty": 0, 2024-04-07T17:51:27.836 INFO:tasks.workunit.client.0.smithi100.stdout: "max_avail": 241199136768, 2024-04-07T17:51:27.836 INFO:tasks.workunit.client.0.smithi100.stdout: "objects": 2, 2024-04-07T17:51:27.836 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_bytes_used": 0, 2024-04-07T17:51:27.836 INFO:tasks.workunit.client.0.smithi100.stdout: "percent_used": 2.46235777012771e-06, 2024-04-07T17:51:27.836 INFO:tasks.workunit.client.0.smithi100.stdout: "quota_bytes": 0, 2024-04-07T17:51:27.836 INFO:tasks.workunit.client.0.smithi100.stdout: "quota_objects": 0, 2024-04-07T17:51:27.836 INFO:tasks.workunit.client.0.smithi100.stdout: "rd": 1214, 2024-04-07T17:51:27.836 INFO:tasks.workunit.client.0.smithi100.stdout: "rd_bytes": 2072576, 2024-04-07T17:51:27.836 INFO:tasks.workunit.client.0.smithi100.stdout: "stored": 590368, 2024-04-07T17:51:27.836 INFO:tasks.workunit.client.0.smithi100.stdout: "stored_data": 590368, 2024-04-07T17:51:27.837 INFO:tasks.workunit.client.0.smithi100.stdout: "stored_omap": 0, 2024-04-07T17:51:27.837 INFO:tasks.workunit.client.0.smithi100.stdout: "stored_raw": 1771104, 2024-04-07T17:51:27.837 INFO:tasks.workunit.client.0.smithi100.stdout: "wr": 399, 2024-04-07T17:51:27.837 INFO:tasks.workunit.client.0.smithi100.stdout: "wr_bytes": 2473984 2024-04-07T17:51:27.837 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.837 INFO:tasks.workunit.client.0.smithi100.stdout: "target_max_bytes": 0, 2024-04-07T17:51:27.837 INFO:tasks.workunit.client.0.smithi100.stdout: "target_max_objects": 0, 2024-04-07T17:51:27.837 INFO:tasks.workunit.client.0.smithi100.stdout: "type": "replicated" 2024-04-07T17:51:27.837 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.837 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:27.837 INFO:tasks.workunit.client.0.smithi100.stdout: "application": [ 2024-04-07T17:51:27.837 INFO:tasks.workunit.client.0.smithi100.stdout: "cephfs" 2024-04-07T17:51:27.837 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:27.837 INFO:tasks.workunit.client.0.smithi100.stdout: "cache_mode": "none", 2024-04-07T17:51:27.837 INFO:tasks.workunit.client.0.smithi100.stdout: "erasure_code_profile": {}, 2024-04-07T17:51:27.837 INFO:tasks.workunit.client.0.smithi100.stdout: "flags_names": [ 2024-04-07T17:51:27.838 INFO:tasks.workunit.client.0.smithi100.stdout: "hashpspool" 2024-04-07T17:51:27.838 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:27.838 INFO:tasks.workunit.client.0.smithi100.stdout: "min_size": 2, 2024-04-07T17:51:27.838 INFO:tasks.workunit.client.0.smithi100.stdout: "options": {}, 2024-04-07T17:51:27.838 INFO:tasks.workunit.client.0.smithi100.stdout: "pg_autoscale_mode": "off", 2024-04-07T17:51:27.838 INFO:tasks.workunit.client.0.smithi100.stdout: "pg_num": 32, 2024-04-07T17:51:27.838 INFO:tasks.workunit.client.0.smithi100.stdout: "pgp_num": 32, 2024-04-07T17:51:27.838 INFO:tasks.workunit.client.0.smithi100.stdout: "pool": 2, 2024-04-07T17:51:27.838 INFO:tasks.workunit.client.0.smithi100.stdout: "size": 3, 2024-04-07T17:51:27.838 INFO:tasks.workunit.client.0.smithi100.stdout: "stats": { 2024-04-07T17:51:27.838 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_raw": 723597404426, 2024-04-07T17:51:27.838 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_used": 159173, 2024-04-07T17:51:27.838 INFO:tasks.workunit.client.0.smithi100.stdout: "compress_bytes_used": 0, 2024-04-07T17:51:27.838 INFO:tasks.workunit.client.0.smithi100.stdout: "compress_under_bytes": 0, 2024-04-07T17:51:27.839 INFO:tasks.workunit.client.0.smithi100.stdout: "data_bytes_used": 110592, 2024-04-07T17:51:27.839 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty": 0, 2024-04-07T17:51:27.839 INFO:tasks.workunit.client.0.smithi100.stdout: "max_avail": 241199136768, 2024-04-07T17:51:27.839 INFO:tasks.workunit.client.0.smithi100.stdout: "objects": 22, 2024-04-07T17:51:27.839 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_bytes_used": 48581, 2024-04-07T17:51:27.839 INFO:tasks.workunit.client.0.smithi100.stdout: "percent_used": 2.1997449550781312e-07, 2024-04-07T17:51:27.839 INFO:tasks.workunit.client.0.smithi100.stdout: "quota_bytes": 0, 2024-04-07T17:51:27.839 INFO:tasks.workunit.client.0.smithi100.stdout: "quota_objects": 0, 2024-04-07T17:51:27.839 INFO:tasks.workunit.client.0.smithi100.stdout: "rd": 79, 2024-04-07T17:51:27.839 INFO:tasks.workunit.client.0.smithi100.stdout: "rd_bytes": 89088, 2024-04-07T17:51:27.839 INFO:tasks.workunit.client.0.smithi100.stdout: "stored": 22815, 2024-04-07T17:51:27.839 INFO:tasks.workunit.client.0.smithi100.stdout: "stored_data": 6622, 2024-04-07T17:51:27.839 INFO:tasks.workunit.client.0.smithi100.stdout: "stored_omap": 16193, 2024-04-07T17:51:27.839 INFO:tasks.workunit.client.0.smithi100.stdout: "stored_raw": 68445, 2024-04-07T17:51:27.839 INFO:tasks.workunit.client.0.smithi100.stdout: "wr": 68, 2024-04-07T17:51:27.839 INFO:tasks.workunit.client.0.smithi100.stdout: "wr_bytes": 36864 2024-04-07T17:51:27.840 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.840 INFO:tasks.workunit.client.0.smithi100.stdout: "target_max_bytes": 0, 2024-04-07T17:51:27.840 INFO:tasks.workunit.client.0.smithi100.stdout: "target_max_objects": 0, 2024-04-07T17:51:27.840 INFO:tasks.workunit.client.0.smithi100.stdout: "type": "replicated" 2024-04-07T17:51:27.840 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.840 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:27.840 INFO:tasks.workunit.client.0.smithi100.stdout: "application": [ 2024-04-07T17:51:27.840 INFO:tasks.workunit.client.0.smithi100.stdout: "cephfs" 2024-04-07T17:51:27.840 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:27.840 INFO:tasks.workunit.client.0.smithi100.stdout: "cache_mode": "none", 2024-04-07T17:51:27.840 INFO:tasks.workunit.client.0.smithi100.stdout: "erasure_code_profile": {}, 2024-04-07T17:51:27.840 INFO:tasks.workunit.client.0.smithi100.stdout: "flags_names": [ 2024-04-07T17:51:27.840 INFO:tasks.workunit.client.0.smithi100.stdout: "hashpspool" 2024-04-07T17:51:27.840 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:27.840 INFO:tasks.workunit.client.0.smithi100.stdout: "min_size": 2, 2024-04-07T17:51:27.841 INFO:tasks.workunit.client.0.smithi100.stdout: "options": {}, 2024-04-07T17:51:27.841 INFO:tasks.workunit.client.0.smithi100.stdout: "pg_autoscale_mode": "off", 2024-04-07T17:51:27.841 INFO:tasks.workunit.client.0.smithi100.stdout: "pg_num": 64, 2024-04-07T17:51:27.841 INFO:tasks.workunit.client.0.smithi100.stdout: "pgp_num": 64, 2024-04-07T17:51:27.841 INFO:tasks.workunit.client.0.smithi100.stdout: "pool": 3, 2024-04-07T17:51:27.841 INFO:tasks.workunit.client.0.smithi100.stdout: "size": 3, 2024-04-07T17:51:27.841 INFO:tasks.workunit.client.0.smithi100.stdout: "stats": { 2024-04-07T17:51:27.841 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_raw": 723597404426, 2024-04-07T17:51:27.841 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_used": 0, 2024-04-07T17:51:27.841 INFO:tasks.workunit.client.0.smithi100.stdout: "compress_bytes_used": 0, 2024-04-07T17:51:27.841 INFO:tasks.workunit.client.0.smithi100.stdout: "compress_under_bytes": 0, 2024-04-07T17:51:27.841 INFO:tasks.workunit.client.0.smithi100.stdout: "data_bytes_used": 0, 2024-04-07T17:51:27.841 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty": 0, 2024-04-07T17:51:27.841 INFO:tasks.workunit.client.0.smithi100.stdout: "max_avail": 241199136768, 2024-04-07T17:51:27.841 INFO:tasks.workunit.client.0.smithi100.stdout: "objects": 0, 2024-04-07T17:51:27.842 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_bytes_used": 0, 2024-04-07T17:51:27.842 INFO:tasks.workunit.client.0.smithi100.stdout: "percent_used": 0.0, 2024-04-07T17:51:27.842 INFO:tasks.workunit.client.0.smithi100.stdout: "quota_bytes": 0, 2024-04-07T17:51:27.842 INFO:tasks.workunit.client.0.smithi100.stdout: "quota_objects": 0, 2024-04-07T17:51:27.842 INFO:tasks.workunit.client.0.smithi100.stdout: "rd": 0, 2024-04-07T17:51:27.842 INFO:tasks.workunit.client.0.smithi100.stdout: "rd_bytes": 0, 2024-04-07T17:51:27.842 INFO:tasks.workunit.client.0.smithi100.stdout: "stored": 0, 2024-04-07T17:51:27.842 INFO:tasks.workunit.client.0.smithi100.stdout: "stored_data": 0, 2024-04-07T17:51:27.842 INFO:tasks.workunit.client.0.smithi100.stdout: "stored_omap": 0, 2024-04-07T17:51:27.842 INFO:tasks.workunit.client.0.smithi100.stdout: "stored_raw": 0, 2024-04-07T17:51:27.842 INFO:tasks.workunit.client.0.smithi100.stdout: "wr": 0, 2024-04-07T17:51:27.842 INFO:tasks.workunit.client.0.smithi100.stdout: "wr_bytes": 0 2024-04-07T17:51:27.842 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.842 INFO:tasks.workunit.client.0.smithi100.stdout: "target_max_bytes": 0, 2024-04-07T17:51:27.842 INFO:tasks.workunit.client.0.smithi100.stdout: "target_max_objects": 0, 2024-04-07T17:51:27.842 INFO:tasks.workunit.client.0.smithi100.stdout: "type": "replicated" 2024-04-07T17:51:27.843 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.843 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:27.843 INFO:tasks.workunit.client.0.smithi100.stdout: "application": [ 2024-04-07T17:51:27.843 INFO:tasks.workunit.client.0.smithi100.stdout: "rbd" 2024-04-07T17:51:27.843 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:27.843 INFO:tasks.workunit.client.0.smithi100.stdout: "cache_mode": "none", 2024-04-07T17:51:27.843 INFO:tasks.workunit.client.0.smithi100.stdout: "erasure_code_profile": {}, 2024-04-07T17:51:27.843 INFO:tasks.workunit.client.0.smithi100.stdout: "flags_names": [ 2024-04-07T17:51:27.843 INFO:tasks.workunit.client.0.smithi100.stdout: "hashpspool", 2024-04-07T17:51:27.843 INFO:tasks.workunit.client.0.smithi100.stdout: "selfmanaged_snaps" 2024-04-07T17:51:27.843 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:27.843 INFO:tasks.workunit.client.0.smithi100.stdout: "min_size": 2, 2024-04-07T17:51:27.843 INFO:tasks.workunit.client.0.smithi100.stdout: "options": {}, 2024-04-07T17:51:27.843 INFO:tasks.workunit.client.0.smithi100.stdout: "pg_autoscale_mode": "off", 2024-04-07T17:51:27.843 INFO:tasks.workunit.client.0.smithi100.stdout: "pg_num": 8, 2024-04-07T17:51:27.843 INFO:tasks.workunit.client.0.smithi100.stdout: "pgp_num": 8, 2024-04-07T17:51:27.844 INFO:tasks.workunit.client.0.smithi100.stdout: "pool": 4, 2024-04-07T17:51:27.844 INFO:tasks.workunit.client.0.smithi100.stdout: "size": 3, 2024-04-07T17:51:27.844 INFO:tasks.workunit.client.0.smithi100.stdout: "stats": { 2024-04-07T17:51:27.844 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_raw": 723597404426, 2024-04-07T17:51:27.844 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_used": 12595200, 2024-04-07T17:51:27.844 INFO:tasks.workunit.client.0.smithi100.stdout: "compress_bytes_used": 0, 2024-04-07T17:51:27.844 INFO:tasks.workunit.client.0.smithi100.stdout: "compress_under_bytes": 0, 2024-04-07T17:51:27.844 INFO:tasks.workunit.client.0.smithi100.stdout: "data_bytes_used": 12595200, 2024-04-07T17:51:27.844 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty": 0, 2024-04-07T17:51:27.844 INFO:tasks.workunit.client.0.smithi100.stdout: "max_avail": 241199136768, 2024-04-07T17:51:27.844 INFO:tasks.workunit.client.0.smithi100.stdout: "objects": 4, 2024-04-07T17:51:27.844 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_bytes_used": 0, 2024-04-07T17:51:27.844 INFO:tasks.workunit.client.0.smithi100.stdout: "percent_used": 1.74060623976402e-05, 2024-04-07T17:51:27.844 INFO:tasks.workunit.client.0.smithi100.stdout: "quota_bytes": 0, 2024-04-07T17:51:27.844 INFO:tasks.workunit.client.0.smithi100.stdout: "quota_objects": 0, 2024-04-07T17:51:27.845 INFO:tasks.workunit.client.0.smithi100.stdout: "rd": 9098, 2024-04-07T17:51:27.845 INFO:tasks.workunit.client.0.smithi100.stdout: "rd_bytes": 3222817792, 2024-04-07T17:51:27.845 INFO:tasks.workunit.client.0.smithi100.stdout: "stored": 4194323, 2024-04-07T17:51:27.845 INFO:tasks.workunit.client.0.smithi100.stdout: "stored_data": 4194323, 2024-04-07T17:51:27.845 INFO:tasks.workunit.client.0.smithi100.stdout: "stored_omap": 0, 2024-04-07T17:51:27.845 INFO:tasks.workunit.client.0.smithi100.stdout: "stored_raw": 12582969, 2024-04-07T17:51:27.845 INFO:tasks.workunit.client.0.smithi100.stdout: "wr": 24273, 2024-04-07T17:51:27.845 INFO:tasks.workunit.client.0.smithi100.stdout: "wr_bytes": 3103659008 2024-04-07T17:51:27.845 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.845 INFO:tasks.workunit.client.0.smithi100.stdout: "target_max_bytes": 0, 2024-04-07T17:51:27.845 INFO:tasks.workunit.client.0.smithi100.stdout: "target_max_objects": 0, 2024-04-07T17:51:27.845 INFO:tasks.workunit.client.0.smithi100.stdout: "type": "replicated" 2024-04-07T17:51:27.845 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.845 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:27.845 INFO:tasks.workunit.client.0.smithi100.stdout: "rbd": { 2024-04-07T17:51:27.845 INFO:tasks.workunit.client.0.smithi100.stdout: "mirroring_by_pool": [ 2024-04-07T17:51:27.846 INFO:tasks.workunit.client.0.smithi100.stdout: false 2024-04-07T17:51:27.846 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:27.846 INFO:tasks.workunit.client.0.smithi100.stdout: "num_images_by_pool": [ 2024-04-07T17:51:27.846 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T17:51:27.846 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:27.846 INFO:tasks.workunit.client.0.smithi100.stdout: "num_pools": 1 2024-04-07T17:51:27.846 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.846 INFO:tasks.workunit.client.0.smithi100.stdout: "report_id": "ee983b25-10fa-4633-b6cf-1c3036b91b52", 2024-04-07T17:51:27.846 INFO:tasks.workunit.client.0.smithi100.stdout: "report_timestamp": "2024-04-07T17:51:26.716127", 2024-04-07T17:51:27.846 INFO:tasks.workunit.client.0.smithi100.stdout: "report_version": 1, 2024-04-07T17:51:27.846 INFO:tasks.workunit.client.0.smithi100.stdout: "rocksdb_stats": { 2024-04-07T17:51:27.846 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "7.9.2" 2024-04-07T17:51:27.846 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.846 INFO:tasks.workunit.client.0.smithi100.stdout: "rook": { 2024-04-07T17:51:27.846 INFO:tasks.workunit.client.0.smithi100.stdout: "cluster": { 2024-04-07T17:51:27.847 INFO:tasks.workunit.client.0.smithi100.stdout: "external-mode": null, 2024-04-07T17:51:27.847 INFO:tasks.workunit.client.0.smithi100.stdout: "mon": { 2024-04-07T17:51:27.847 INFO:tasks.workunit.client.0.smithi100.stdout: "allow-multiple-per-node": null, 2024-04-07T17:51:27.847 INFO:tasks.workunit.client.0.smithi100.stdout: "count": null, 2024-04-07T17:51:27.847 INFO:tasks.workunit.client.0.smithi100.stdout: "max-id": null, 2024-04-07T17:51:27.847 INFO:tasks.workunit.client.0.smithi100.stdout: "pvc": { 2024-04-07T17:51:27.847 INFO:tasks.workunit.client.0.smithi100.stdout: "enabled": null 2024-04-07T17:51:27.847 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.847 INFO:tasks.workunit.client.0.smithi100.stdout: "stretch": { 2024-04-07T17:51:27.847 INFO:tasks.workunit.client.0.smithi100.stdout: "enabled": null 2024-04-07T17:51:27.847 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.847 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.847 INFO:tasks.workunit.client.0.smithi100.stdout: "network": { 2024-04-07T17:51:27.847 INFO:tasks.workunit.client.0.smithi100.stdout: "provider": null 2024-04-07T17:51:27.847 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.847 INFO:tasks.workunit.client.0.smithi100.stdout: "storage": { 2024-04-07T17:51:27.848 INFO:tasks.workunit.client.0.smithi100.stdout: "device-set": { 2024-04-07T17:51:27.848 INFO:tasks.workunit.client.0.smithi100.stdout: "count": { 2024-04-07T17:51:27.848 INFO:tasks.workunit.client.0.smithi100.stdout: "non-portable": null, 2024-04-07T17:51:27.848 INFO:tasks.workunit.client.0.smithi100.stdout: "portable": null, 2024-04-07T17:51:27.848 INFO:tasks.workunit.client.0.smithi100.stdout: "total": null 2024-04-07T17:51:27.848 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.848 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.848 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.848 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.848 INFO:tasks.workunit.client.0.smithi100.stdout: "csi": { 2024-04-07T17:51:27.848 INFO:tasks.workunit.client.0.smithi100.stdout: "version": null 2024-04-07T17:51:27.848 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.848 INFO:tasks.workunit.client.0.smithi100.stdout: "kubernetes": { 2024-04-07T17:51:27.848 INFO:tasks.workunit.client.0.smithi100.stdout: "version": null 2024-04-07T17:51:27.848 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.849 INFO:tasks.workunit.client.0.smithi100.stdout: "node": { 2024-04-07T17:51:27.849 INFO:tasks.workunit.client.0.smithi100.stdout: "count": { 2024-04-07T17:51:27.849 INFO:tasks.workunit.client.0.smithi100.stdout: "kubernetes-total": null, 2024-04-07T17:51:27.849 INFO:tasks.workunit.client.0.smithi100.stdout: "with-ceph-daemons": null, 2024-04-07T17:51:27.849 INFO:tasks.workunit.client.0.smithi100.stdout: "with-csi-cephfs-plugin": null, 2024-04-07T17:51:27.849 INFO:tasks.workunit.client.0.smithi100.stdout: "with-csi-nfs-plugin": null, 2024-04-07T17:51:27.849 INFO:tasks.workunit.client.0.smithi100.stdout: "with-csi-rbd-plugin": null 2024-04-07T17:51:27.849 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.849 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.849 INFO:tasks.workunit.client.0.smithi100.stdout: "usage": { 2024-04-07T17:51:27.849 INFO:tasks.workunit.client.0.smithi100.stdout: "storage-class": { 2024-04-07T17:51:27.849 INFO:tasks.workunit.client.0.smithi100.stdout: "count": { 2024-04-07T17:51:27.849 INFO:tasks.workunit.client.0.smithi100.stdout: "bucket": null, 2024-04-07T17:51:27.849 INFO:tasks.workunit.client.0.smithi100.stdout: "cephfs": null, 2024-04-07T17:51:27.849 INFO:tasks.workunit.client.0.smithi100.stdout: "nfs": null, 2024-04-07T17:51:27.850 INFO:tasks.workunit.client.0.smithi100.stdout: "rbd": null, 2024-04-07T17:51:27.850 INFO:tasks.workunit.client.0.smithi100.stdout: "total": null 2024-04-07T17:51:27.850 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.850 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:27.850 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.850 INFO:tasks.workunit.client.0.smithi100.stdout: "version": null 2024-04-07T17:51:27.850 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.850 INFO:tasks.workunit.client.0.smithi100.stdout: "services": { 2024-04-07T17:51:27.850 INFO:tasks.workunit.client.0.smithi100.stdout: "mon": 1 2024-04-07T17:51:27.850 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.850 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_per_pg": [ 2024-04-07T17:51:27.850 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:27.850 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:27.850 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:27.850 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:27.851 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T17:51:27.851 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:27.851 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 6, 2024-04-07T17:51:27.851 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:27.851 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:27.851 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:27.851 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:27.851 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:27.851 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:27.851 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.371282+0000", 2024-04-07T17:51:27.851 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:08.962622+0000", 2024-04-07T17:51:27.851 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:08.962622+0000", 2024-04-07T17:51:27.851 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:08.963061+0000", 2024-04-07T17:51:27.851 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.371282+0000", 2024-04-07T17:51:27.851 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:27.851 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:27.852 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:27.852 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 554, 2024-04-07T17:51:27.852 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.371282+0000", 2024-04-07T17:51:27.852 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.371282+0000", 2024-04-07T17:51:27.852 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.371282+0000", 2024-04-07T17:51:27.852 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:27.852 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:27.852 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:27.852 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.371282+0000", 2024-04-07T17:51:27.852 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.371282+0000", 2024-04-07T17:51:27.852 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:27.852 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:27.852 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:27.852 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:27.852 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 553, 2024-04-07T17:51:27.853 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:27.853 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:27.853 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:27.853 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:27.853 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:27.853 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:27.853 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:27.853 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:27.853 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.2c", 2024-04-07T17:51:27.853 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:27.853 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:27.853 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:27.853 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4846, 2024-04-07T17:51:27.853 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:27.853 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T02:53:55.780425+0000", 2024-04-07T17:51:27.853 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:27.854 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:27.854 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:27.854 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:27.854 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:27.854 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:27.854 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:27.854 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:27.854 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:27.854 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:27.854 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:27.854 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:27.854 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:27.854 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:27.854 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:27.854 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:27.855 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:27.855 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:27.855 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:27.855 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:27.855 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:27.855 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:27.855 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:27.855 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:27.855 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:27.855 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:27.855 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:27.855 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:27.855 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:27.855 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:27.855 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:27.856 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:27.856 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:27.856 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:27.856 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:27.856 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:27.856 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:27.856 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:27.856 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:27.856 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:27.856 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:27.856 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:27.856 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:27.856 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.856 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:27.856 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:27.857 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:27.857 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:27.857 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:27.857 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T17:51:27.857 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:27.857 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 6, 2024-04-07T17:51:27.857 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:27.857 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.857 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:27.857 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:27.857 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:27.857 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:27.857 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T17:51:27.857 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:27.857 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 0, 2024-04-07T17:51:27.858 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:27.858 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:27.858 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:27.858 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:27.858 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:27.858 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:27.858 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:35.989606+0000", 2024-04-07T17:51:27.858 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:39.003945+0000", 2024-04-07T17:51:27.858 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:39.003945+0000", 2024-04-07T17:51:27.858 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:39.004542+0000", 2024-04-07T17:51:27.858 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:35.989606+0000", 2024-04-07T17:51:27.858 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:27.858 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:27.858 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:27.858 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:27.859 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:35.989606+0000", 2024-04-07T17:51:27.859 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:35.989606+0000", 2024-04-07T17:51:27.859 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:35.989606+0000", 2024-04-07T17:51:27.859 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:27.859 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:27.859 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:27.859 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:35.989606+0000", 2024-04-07T17:51:27.859 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:35.989606+0000", 2024-04-07T17:51:27.859 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:27.859 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:27.859 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:27.859 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:27.859 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:27.859 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:27.859 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:27.859 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:27.860 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:27.860 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:27.860 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:27.860 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:27.860 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:27.860 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.2b", 2024-04-07T17:51:27.860 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:27.860 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:27.860 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:27.860 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4997, 2024-04-07T17:51:27.860 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:27.860 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T18:56:04.984002+0000", 2024-04-07T17:51:27.860 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:27.860 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:27.860 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:27.861 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:27.861 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:27.861 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:27.861 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:27.861 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:27.861 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:27.861 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:27.861 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:27.861 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:27.861 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:27.861 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:27.861 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:27.861 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:27.861 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:27.861 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:27.862 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:27.862 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:27.862 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:27.862 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:27.862 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:27.862 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:27.862 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:27.862 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:27.862 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:27.862 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:27.862 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:27.862 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:27.862 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:27.862 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:27.862 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:27.863 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:27.863 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:27.863 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:27.863 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:27.863 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:27.863 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:27.863 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:27.863 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:27.863 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:27.863 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:27.863 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.863 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:27.863 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:27.863 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:27.863 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:27.864 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:27.864 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T17:51:27.864 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:27.864 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 0, 2024-04-07T17:51:27.864 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:27.864 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.864 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:27.864 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:27.864 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:27.864 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:27.864 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T17:51:27.864 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:27.864 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 5, 2024-04-07T17:51:27.864 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:27.864 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:27.865 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:27.865 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:27.865 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:27.865 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:27.865 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.369807+0000", 2024-04-07T17:51:27.865 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:38.977861+0000", 2024-04-07T17:51:27.865 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:38.977861+0000", 2024-04-07T17:51:27.865 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:38.978928+0000", 2024-04-07T17:51:27.865 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.369807+0000", 2024-04-07T17:51:27.865 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:27.865 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:27.865 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:27.865 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:27.865 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.369807+0000", 2024-04-07T17:51:27.865 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.369807+0000", 2024-04-07T17:51:27.865 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.369807+0000", 2024-04-07T17:51:27.866 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:27.866 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:27.866 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:27.866 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.369807+0000", 2024-04-07T17:51:27.866 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.369807+0000", 2024-04-07T17:51:27.866 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:27.866 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:27.866 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:27.866 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:27.866 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:27.866 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:27.866 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:27.866 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:27.867 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:27.867 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:27.867 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:27.867 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:27.867 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:27.867 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.2a", 2024-04-07T17:51:27.867 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:27.867 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:27.867 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:27.867 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4871, 2024-04-07T17:51:27.867 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:27.867 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T01:59:28.849179+0000", 2024-04-07T17:51:27.867 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:27.867 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:27.867 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:27.867 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:27.868 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:27.868 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:27.868 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:27.868 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:27.868 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:27.868 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:27.868 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:27.868 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:27.868 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:27.868 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:27.868 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:27.868 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:27.868 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:27.868 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:27.868 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:27.868 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:27.869 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:27.869 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:27.869 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:27.869 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:27.869 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:27.869 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:27.869 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:27.869 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:27.869 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:27.869 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:27.869 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:27.869 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:27.869 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:27.869 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:27.869 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:27.870 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:27.870 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:27.870 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:27.870 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:27.870 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:27.870 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:27.870 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:27.870 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:27.870 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.870 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:27.870 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:27.870 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:27.870 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:27.870 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:27.871 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T17:51:27.871 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:27.871 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 5, 2024-04-07T17:51:27.871 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:27.871 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.871 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:27.871 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:27.871 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:27.871 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:27.871 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T17:51:27.871 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:27.871 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 3, 2024-04-07T17:51:27.871 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:27.871 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:27.871 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:27.872 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:27.872 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:27.872 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:27.872 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:35.976785+0000", 2024-04-07T17:51:27.872 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:24:13.933027+0000", 2024-04-07T17:51:27.872 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:24:13.933027+0000", 2024-04-07T17:51:27.872 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:24:13.933359+0000", 2024-04-07T17:51:27.872 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:35.976785+0000", 2024-04-07T17:51:27.872 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:27.872 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:27.872 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:27.872 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 500, 2024-04-07T17:51:27.872 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:35.976785+0000", 2024-04-07T17:51:27.872 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:35.976785+0000", 2024-04-07T17:51:27.872 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:35.976785+0000", 2024-04-07T17:51:27.872 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:27.873 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:27.873 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:27.873 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:35.976785+0000", 2024-04-07T17:51:27.873 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:35.976785+0000", 2024-04-07T17:51:27.873 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:27.873 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:27.873 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:27.873 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:27.873 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 499, 2024-04-07T17:51:27.873 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:27.873 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:27.873 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:27.873 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:27.873 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:27.873 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:27.874 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:27.874 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:27.874 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.29", 2024-04-07T17:51:27.874 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:27.874 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:27.874 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:27.874 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4966, 2024-04-07T17:51:27.874 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:27.874 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T00:08:14.265846+0000", 2024-04-07T17:51:27.874 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:27.874 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:27.874 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:27.874 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:27.874 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:27.874 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:27.875 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:27.875 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:27.875 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:27.875 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:27.875 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:27.875 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:27.875 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:27.875 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:27.875 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:27.875 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:27.875 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:27.875 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:27.875 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:27.875 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:27.875 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:27.876 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:27.876 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:27.876 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:27.876 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:27.876 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:27.876 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:27.876 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:27.876 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:27.876 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:27.876 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:27.876 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:27.876 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:27.876 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:27.876 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:27.876 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:27.877 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:27.877 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:27.877 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:27.877 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:27.877 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:27.877 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:27.877 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:27.877 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.877 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:27.877 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:27.877 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:27.877 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:27.877 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:27.877 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T17:51:27.877 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:27.878 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 3, 2024-04-07T17:51:27.878 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:27.878 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.878 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:27.878 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:27.878 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:27.878 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:27.878 INFO:tasks.workunit.client.0.smithi100.stdout: 2 2024-04-07T17:51:27.878 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:27.878 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 5, 2024-04-07T17:51:27.878 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:27.878 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:27.878 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:27.878 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:27.878 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:27.879 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:27.879 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.363975+0000", 2024-04-07T17:51:27.879 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:24:41.553450+0000", 2024-04-07T17:51:27.879 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:24:41.553450+0000", 2024-04-07T17:51:27.879 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:24:41.557281+0000", 2024-04-07T17:51:27.879 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.363975+0000", 2024-04-07T17:51:27.879 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:27.879 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:27.879 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:27.879 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 527, 2024-04-07T17:51:27.879 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.363975+0000", 2024-04-07T17:51:27.879 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.363975+0000", 2024-04-07T17:51:27.879 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.363975+0000", 2024-04-07T17:51:27.879 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:27.879 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:27.880 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:27.880 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.363975+0000", 2024-04-07T17:51:27.880 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.363975+0000", 2024-04-07T17:51:27.880 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:27.880 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:27.880 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:27.880 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:27.880 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 526, 2024-04-07T17:51:27.880 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:27.880 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:27.880 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:27.880 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:27.880 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:27.880 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:27.880 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:27.880 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:27.881 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.28", 2024-04-07T17:51:27.881 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:27.881 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:27.881 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:27.881 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4884, 2024-04-07T17:51:27.881 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:27.881 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T19:49:37.795766+0000", 2024-04-07T17:51:27.881 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:27.881 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:27.881 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:27.881 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:27.881 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:27.881 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:27.881 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:27.881 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:27.882 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:27.882 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:27.882 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:27.882 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:27.882 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:27.882 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:27.882 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:27.882 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:27.882 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:27.882 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:27.882 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:27.882 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:27.882 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:27.882 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:27.883 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:27.883 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:27.883 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:27.883 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:27.883 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:27.883 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:27.883 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:27.883 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:27.883 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:27.883 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:27.883 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:27.883 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:27.883 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:27.883 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:27.883 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:27.883 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:27.884 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:27.884 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:27.884 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:27.884 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:27.884 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:27.884 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.884 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:27.884 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:27.884 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:27.884 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:27.884 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:27.884 INFO:tasks.workunit.client.0.smithi100.stdout: 2 2024-04-07T17:51:27.884 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:27.884 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 5, 2024-04-07T17:51:27.885 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:27.885 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.885 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:27.885 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:27.885 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:27.885 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:27.885 INFO:tasks.workunit.client.0.smithi100.stdout: 2 2024-04-07T17:51:27.885 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:27.885 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 3, 2024-04-07T17:51:27.885 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:27.885 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:27.885 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:27.885 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:27.885 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:27.885 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:27.886 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:35.982188+0000", 2024-04-07T17:51:27.886 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:24:13.933691+0000", 2024-04-07T17:51:27.886 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:24:13.933691+0000", 2024-04-07T17:51:27.886 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:24:13.934646+0000", 2024-04-07T17:51:27.886 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:35.982188+0000", 2024-04-07T17:51:27.886 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:27.886 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:27.886 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:27.886 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 500, 2024-04-07T17:51:27.886 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:35.982188+0000", 2024-04-07T17:51:27.886 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:35.982188+0000", 2024-04-07T17:51:27.886 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:35.982188+0000", 2024-04-07T17:51:27.886 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:27.886 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:27.886 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:27.887 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:35.982188+0000", 2024-04-07T17:51:27.887 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:35.982188+0000", 2024-04-07T17:51:27.887 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:27.887 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:27.887 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:27.887 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:27.887 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 499, 2024-04-07T17:51:27.887 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:27.887 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:27.887 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:27.887 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:27.887 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:27.887 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:27.887 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:27.887 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:27.888 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.27", 2024-04-07T17:51:27.888 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:27.888 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:27.888 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:27.888 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4954, 2024-04-07T17:51:27.888 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:27.888 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T19:33:33.960559+0000", 2024-04-07T17:51:27.888 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:27.888 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:27.888 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:27.888 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:27.888 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:27.888 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:27.888 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:27.888 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:27.888 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:27.889 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:27.889 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:27.889 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:27.889 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:27.889 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:27.889 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:27.889 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:27.889 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:27.889 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:27.889 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:27.889 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:27.889 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:27.889 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:27.889 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:27.889 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:27.890 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:27.890 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:27.890 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:27.890 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:27.890 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:27.890 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:27.890 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:27.890 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:27.890 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:27.890 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:27.890 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:27.890 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:27.890 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:27.890 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:27.890 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:27.891 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:27.891 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:27.891 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:27.891 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:27.891 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.891 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:27.891 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:27.891 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:27.891 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:27.891 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:27.891 INFO:tasks.workunit.client.0.smithi100.stdout: 2 2024-04-07T17:51:27.891 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:27.891 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 3, 2024-04-07T17:51:27.891 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:27.891 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.892 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:27.892 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:27.892 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:27.892 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:27.892 INFO:tasks.workunit.client.0.smithi100.stdout: 4 2024-04-07T17:51:27.892 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:27.892 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 6, 2024-04-07T17:51:27.892 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:27.892 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:27.892 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:27.892 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:27.892 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:27.892 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:27.892 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.368356+0000", 2024-04-07T17:51:27.892 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:38.960595+0000", 2024-04-07T17:51:27.892 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:38.960595+0000", 2024-04-07T17:51:27.893 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:38.962091+0000", 2024-04-07T17:51:27.893 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.368356+0000", 2024-04-07T17:51:27.893 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:27.893 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:27.893 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:27.893 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:27.893 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.368356+0000", 2024-04-07T17:51:27.893 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.368356+0000", 2024-04-07T17:51:27.893 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.368356+0000", 2024-04-07T17:51:27.893 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:27.893 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:27.893 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:27.893 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.368356+0000", 2024-04-07T17:51:27.893 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.368356+0000", 2024-04-07T17:51:27.893 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:27.893 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:27.894 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:27.894 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:27.894 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:27.894 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:27.894 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:27.894 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:27.894 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:27.894 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:27.894 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:27.894 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:27.894 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:27.894 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.26", 2024-04-07T17:51:27.894 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:27.894 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:27.894 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:27.895 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4841, 2024-04-07T17:51:27.895 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:27.895 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T19:11:01.462021+0000", 2024-04-07T17:51:27.895 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:27.895 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:27.895 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:27.895 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:27.895 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:27.895 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:27.895 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:27.895 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:27.895 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:27.895 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:27.895 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:27.895 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:27.895 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:27.896 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:27.896 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:27.896 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:27.896 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:27.896 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:27.896 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:27.896 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:27.896 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:27.896 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:27.896 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:27.896 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:27.896 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:27.896 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:27.896 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:27.896 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:27.896 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:27.897 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:27.897 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:27.897 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:27.897 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:27.897 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:27.897 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:27.897 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:27.897 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:27.897 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:27.897 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:27.897 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:27.897 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:27.897 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:27.897 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:27.897 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.897 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:27.898 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:27.898 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:27.898 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:27.898 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:27.898 INFO:tasks.workunit.client.0.smithi100.stdout: 4 2024-04-07T17:51:27.898 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:27.898 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 6, 2024-04-07T17:51:27.898 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:27.898 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.898 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:27.898 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:27.898 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:27.898 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:27.898 INFO:tasks.workunit.client.0.smithi100.stdout: 2 2024-04-07T17:51:27.899 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:27.899 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 1, 2024-04-07T17:51:27.899 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:27.899 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:27.899 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:27.899 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:27.899 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:27.899 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:27.899 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.334473+0000", 2024-04-07T17:51:27.899 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:23:43.924498+0000", 2024-04-07T17:51:27.899 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:23:43.924498+0000", 2024-04-07T17:51:27.899 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:23:43.926295+0000", 2024-04-07T17:51:27.899 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.334473+0000", 2024-04-07T17:51:27.899 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:27.899 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:27.900 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:27.900 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 471, 2024-04-07T17:51:27.900 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.334473+0000", 2024-04-07T17:51:27.900 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.334473+0000", 2024-04-07T17:51:27.900 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.334473+0000", 2024-04-07T17:51:27.900 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:27.900 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:27.900 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:27.900 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.334473+0000", 2024-04-07T17:51:27.900 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.334473+0000", 2024-04-07T17:51:27.900 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:27.900 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:27.900 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:27.900 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:27.900 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 470, 2024-04-07T17:51:27.901 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:27.901 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:27.901 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:27.901 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:27.901 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:27.901 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:27.901 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:27.901 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:27.901 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.25", 2024-04-07T17:51:27.901 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:27.901 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:27.901 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:27.901 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4981, 2024-04-07T17:51:27.901 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:27.901 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T17:58:42.938076+0000", 2024-04-07T17:51:27.901 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:27.902 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:27.902 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:27.902 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:27.902 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:27.902 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:27.902 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:27.902 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:27.902 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:27.902 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:27.902 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:27.902 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:27.902 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:27.902 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:27.902 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:27.903 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:27.903 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:27.903 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:27.903 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:27.903 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:27.903 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:27.903 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:27.903 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:27.903 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:27.903 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:27.903 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:27.903 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:27.903 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:27.903 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:27.904 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:27.904 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:27.904 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:27.904 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:27.904 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:27.904 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:27.904 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:27.904 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:27.904 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:27.904 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:27.904 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:27.904 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:27.904 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:27.904 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:27.904 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.904 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:27.905 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:27.905 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:27.905 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:27.905 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:27.905 INFO:tasks.workunit.client.0.smithi100.stdout: 2 2024-04-07T17:51:27.905 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:27.905 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 1, 2024-04-07T17:51:27.905 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:27.905 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.905 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:27.905 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:27.905 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:27.905 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:27.905 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T17:51:27.905 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:27.906 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 3, 2024-04-07T17:51:27.906 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:27.906 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:27.906 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:27.906 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:27.906 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:27.906 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:27.906 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:35.977918+0000", 2024-04-07T17:51:27.906 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:24:13.930420+0000", 2024-04-07T17:51:27.906 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:24:13.930420+0000", 2024-04-07T17:51:27.906 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:24:13.931275+0000", 2024-04-07T17:51:27.906 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:35.977918+0000", 2024-04-07T17:51:27.906 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:27.906 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:27.906 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:27.907 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 500, 2024-04-07T17:51:27.907 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:35.977918+0000", 2024-04-07T17:51:27.907 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:35.977918+0000", 2024-04-07T17:51:27.907 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:35.977918+0000", 2024-04-07T17:51:27.907 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:27.907 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:27.907 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:27.907 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:35.977918+0000", 2024-04-07T17:51:27.907 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:35.977918+0000", 2024-04-07T17:51:27.907 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:27.907 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:27.907 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:27.907 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:27.907 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 499, 2024-04-07T17:51:27.907 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:27.907 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:27.908 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:27.908 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:27.908 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:27.908 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:27.908 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:27.908 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:27.908 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.24", 2024-04-07T17:51:27.908 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:27.908 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:27.908 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:27.908 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4940, 2024-04-07T17:51:27.908 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:27.908 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T00:26:09.028937+0000", 2024-04-07T17:51:27.908 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:27.908 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:27.908 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:27.909 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:27.909 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:27.909 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:27.909 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:27.909 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:27.909 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:27.909 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:27.909 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:27.909 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:27.909 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:27.909 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:27.909 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:27.909 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:27.909 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:27.909 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:27.909 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:27.910 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:27.910 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:27.910 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:27.910 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:27.910 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:27.910 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:27.910 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:27.910 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:27.910 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:27.910 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:27.910 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:27.910 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:27.910 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:27.910 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:27.910 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:27.911 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:27.911 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:27.911 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:27.911 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:27.911 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:27.911 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:27.911 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:27.911 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:27.911 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:27.911 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.911 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:27.911 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:27.911 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:27.911 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:27.911 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:27.911 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T17:51:27.912 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:27.912 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 3, 2024-04-07T17:51:27.912 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:27.912 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.912 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:27.912 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:27.912 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:27.912 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:27.912 INFO:tasks.workunit.client.0.smithi100.stdout: 4 2024-04-07T17:51:27.912 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:27.912 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 0, 2024-04-07T17:51:27.912 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:27.912 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:27.912 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:27.913 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:27.913 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:27.913 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:27.913 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:35.989350+0000", 2024-04-07T17:51:27.913 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:24:13.962893+0000", 2024-04-07T17:51:27.913 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:24:13.962893+0000", 2024-04-07T17:51:27.913 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:24:13.964515+0000", 2024-04-07T17:51:27.913 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:35.989350+0000", 2024-04-07T17:51:27.913 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:27.913 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:27.913 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:27.913 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 500, 2024-04-07T17:51:27.913 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:35.989350+0000", 2024-04-07T17:51:27.913 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:35.989350+0000", 2024-04-07T17:51:27.913 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:35.989350+0000", 2024-04-07T17:51:27.913 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:27.914 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:27.914 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:27.914 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:35.989350+0000", 2024-04-07T17:51:27.914 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:35.989350+0000", 2024-04-07T17:51:27.914 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:27.914 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:27.914 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:27.914 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:27.914 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 499, 2024-04-07T17:51:27.914 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:27.914 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:27.914 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:27.914 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:27.914 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:27.915 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:27.915 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:27.915 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:27.915 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.23", 2024-04-07T17:51:27.915 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:27.915 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:27.915 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:27.915 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 5000, 2024-04-07T17:51:27.915 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:27.915 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T17:30:33.589121+0000", 2024-04-07T17:51:27.915 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:27.915 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:27.915 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:27.915 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:27.915 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:27.915 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:27.916 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:27.916 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:27.916 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:27.916 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:27.916 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:27.916 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:27.916 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:27.916 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:27.916 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:27.916 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:27.916 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:27.916 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:27.916 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:27.916 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:27.916 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:27.917 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:27.917 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:27.917 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:27.917 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:27.917 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:27.917 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:27.917 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:27.917 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:27.917 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:27.917 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:27.917 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:27.917 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:27.917 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:27.917 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:27.917 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:27.917 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:27.918 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:27.918 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:27.918 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:27.918 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:27.918 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:27.918 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:27.918 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.918 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:27.918 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:27.918 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:27.918 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:27.918 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:27.918 INFO:tasks.workunit.client.0.smithi100.stdout: 4 2024-04-07T17:51:27.918 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:27.919 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 0, 2024-04-07T17:51:27.919 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:27.919 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.919 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:27.919 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:27.919 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:27.919 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:27.919 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T17:51:27.919 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:27.919 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 5, 2024-04-07T17:51:27.919 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:27.919 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:27.919 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:27.919 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:27.919 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:27.919 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:27.920 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.369012+0000", 2024-04-07T17:51:27.920 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:24:41.561766+0000", 2024-04-07T17:51:27.920 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:24:41.561766+0000", 2024-04-07T17:51:27.920 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:24:41.564435+0000", 2024-04-07T17:51:27.920 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.369012+0000", 2024-04-07T17:51:27.920 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:27.920 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:27.920 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:27.920 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 527, 2024-04-07T17:51:27.920 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.369012+0000", 2024-04-07T17:51:27.920 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.369012+0000", 2024-04-07T17:51:27.920 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.369012+0000", 2024-04-07T17:51:27.920 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:27.920 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:27.920 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:27.920 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.369012+0000", 2024-04-07T17:51:27.921 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.369012+0000", 2024-04-07T17:51:27.921 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:27.921 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:27.921 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:27.921 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:27.921 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 526, 2024-04-07T17:51:27.921 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:27.921 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:27.921 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:27.921 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:27.921 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:27.921 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:27.921 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:27.921 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:27.921 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.22", 2024-04-07T17:51:27.921 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:27.922 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:27.922 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:27.922 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4886, 2024-04-07T17:51:27.922 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:27.922 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T17:08:34.046549+0000", 2024-04-07T17:51:27.922 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:27.922 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:27.922 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:27.922 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:27.922 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:27.922 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:27.922 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:27.922 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:27.922 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:27.922 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:27.923 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:27.923 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:27.923 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:27.923 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:27.923 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:27.923 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:27.923 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:27.923 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:27.923 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:27.923 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:27.923 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:27.923 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:27.923 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:27.923 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:27.923 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:27.923 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:27.924 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:27.924 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:27.924 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:27.924 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:27.924 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:27.924 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:27.924 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:27.924 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:27.924 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:27.924 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:27.924 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:27.924 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:27.924 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:27.924 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:27.924 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:27.925 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:27.925 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:27.925 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.925 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:27.925 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:27.925 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:27.925 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:27.925 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:27.925 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T17:51:27.925 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:27.925 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 5, 2024-04-07T17:51:27.925 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:27.925 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.925 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:27.926 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:27.926 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T17:51:27.926 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:27.926 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T17:51:27.926 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:27.926 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 2, 2024-04-07T17:51:27.926 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:27.926 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:27.926 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:27.926 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:27.926 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:27.926 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:27.926 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.354974+0000", 2024-04-07T17:51:27.926 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:08.976322+0000", 2024-04-07T17:51:27.927 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:08.976322+0000", 2024-04-07T17:51:27.927 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:08.976897+0000", 2024-04-07T17:51:27.927 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.354974+0000", 2024-04-07T17:51:27.927 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:27.927 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:27.927 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:27.927 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 554, 2024-04-07T17:51:27.927 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.354974+0000", 2024-04-07T17:51:27.927 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.354974+0000", 2024-04-07T17:51:27.927 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.354974+0000", 2024-04-07T17:51:27.927 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:27.927 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:27.927 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:27.927 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.354974+0000", 2024-04-07T17:51:27.928 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.354974+0000", 2024-04-07T17:51:27.928 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:27.928 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:27.928 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:27.928 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:27.928 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 553, 2024-04-07T17:51:27.928 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:27.928 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:27.928 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:27.928 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:27.928 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:27.928 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:27.928 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:27.928 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:27.928 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.21", 2024-04-07T17:51:27.928 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:27.929 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:27.929 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:27.929 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4970, 2024-04-07T17:51:27.929 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:27.929 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T18:02:01.757959+0000", 2024-04-07T17:51:27.929 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:27.929 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:27.929 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:27.929 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:27.929 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:27.929 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:27.929 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:27.929 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:27.929 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:27.929 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:27.930 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:27.930 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:27.930 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:27.930 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:27.930 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:27.930 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:27.930 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:27.930 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:27.930 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:27.930 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:27.930 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:27.930 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:27.930 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:27.930 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:27.931 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:27.931 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:27.931 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:27.931 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:27.931 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:27.931 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:27.931 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:27.931 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:27.931 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:27.931 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:27.931 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:27.931 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:27.931 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:27.931 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:27.931 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:27.932 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:27.932 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:27.932 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:27.932 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:27.932 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.932 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:27.932 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:27.932 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:27.932 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T17:51:27.932 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:27.932 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T17:51:27.932 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:27.932 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 2, 2024-04-07T17:51:27.932 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:27.932 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.933 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:27.933 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:27.933 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:27.933 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:27.933 INFO:tasks.workunit.client.0.smithi100.stdout: 5 2024-04-07T17:51:27.933 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:27.933 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 6, 2024-04-07T17:51:27.933 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:27.933 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:27.933 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:27.933 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:27.933 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:27.933 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:27.933 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.370975+0000", 2024-04-07T17:51:27.933 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:08.975767+0000", 2024-04-07T17:51:27.933 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:08.975767+0000", 2024-04-07T17:51:27.933 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:08.976755+0000", 2024-04-07T17:51:27.934 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.370975+0000", 2024-04-07T17:51:27.934 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:27.934 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:27.934 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:27.934 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 554, 2024-04-07T17:51:27.934 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.370975+0000", 2024-04-07T17:51:27.934 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.370975+0000", 2024-04-07T17:51:27.934 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.370975+0000", 2024-04-07T17:51:27.934 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:27.934 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:27.934 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:27.934 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.370975+0000", 2024-04-07T17:51:27.934 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.370975+0000", 2024-04-07T17:51:27.934 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:27.934 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:27.935 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:27.935 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:27.935 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 553, 2024-04-07T17:51:27.935 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:27.935 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:27.935 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:27.935 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:27.935 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:27.935 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:27.935 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:27.935 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:27.935 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.20", 2024-04-07T17:51:27.935 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:27.935 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:27.935 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:27.936 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4834, 2024-04-07T17:51:27.936 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:27.936 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T16:33:00.288075+0000", 2024-04-07T17:51:27.936 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:27.936 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:27.936 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:27.936 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:27.936 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:27.936 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:27.936 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:27.936 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:27.936 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:27.936 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:27.936 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:27.936 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:27.937 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:27.937 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:27.937 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:27.937 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:27.937 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:27.937 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:27.937 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:27.937 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:27.937 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:27.937 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:27.937 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:27.937 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:27.937 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:27.937 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:27.937 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:27.937 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:27.937 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:27.938 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:27.938 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:27.938 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:27.938 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:27.938 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:27.938 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:27.938 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:27.938 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:27.938 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:27.938 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:27.938 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:27.938 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:27.938 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:27.938 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:27.938 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.939 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:27.939 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:27.939 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:27.939 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:27.939 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:27.939 INFO:tasks.workunit.client.0.smithi100.stdout: 5 2024-04-07T17:51:27.939 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:27.939 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 6, 2024-04-07T17:51:27.939 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:27.939 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.939 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:27.939 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:27.939 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:27.939 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:27.939 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T17:51:27.939 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:27.940 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 7, 2024-04-07T17:51:27.940 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:27.940 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:27.940 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:27.940 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:27.940 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:27.940 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:27.940 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.367504+0000", 2024-04-07T17:51:27.940 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:38.976388+0000", 2024-04-07T17:51:27.940 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:38.976388+0000", 2024-04-07T17:51:27.940 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:38.994496+0000", 2024-04-07T17:51:27.940 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.367504+0000", 2024-04-07T17:51:27.940 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:27.940 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:27.940 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:27.941 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:27.941 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.367504+0000", 2024-04-07T17:51:27.941 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.367504+0000", 2024-04-07T17:51:27.941 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.367504+0000", 2024-04-07T17:51:27.941 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:27.941 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:27.941 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:27.941 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.367504+0000", 2024-04-07T17:51:27.941 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.367504+0000", 2024-04-07T17:51:27.941 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:27.941 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:27.941 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:27.941 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:27.941 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:27.941 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:27.942 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:27.942 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:27.942 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:27.942 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:27.942 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:27.942 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:27.942 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:27.942 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.f", 2024-04-07T17:51:27.942 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:27.942 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:27.942 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:27.942 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4790, 2024-04-07T17:51:27.942 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:27.942 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T17:59:57.662112+0000", 2024-04-07T17:51:27.942 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:27.943 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:27.943 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:27.943 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:27.943 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:27.943 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:27.943 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:27.943 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:27.943 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:27.943 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:27.943 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:27.943 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:27.943 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:27.943 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:27.943 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:27.943 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:27.943 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:27.944 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:27.944 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:27.944 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:27.944 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:27.944 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:27.944 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:27.944 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:27.944 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:27.944 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:27.944 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:27.944 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:27.944 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:27.944 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:27.944 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:27.944 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:27.944 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:27.945 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:27.945 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:27.945 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:27.945 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:27.945 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:27.945 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:27.945 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:27.945 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:27.945 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:27.945 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:27.945 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.945 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:27.945 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:27.945 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:27.946 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:27.946 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:27.946 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T17:51:27.946 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:27.946 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 7, 2024-04-07T17:51:27.946 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:27.946 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.946 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:27.946 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:27.946 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T17:51:27.946 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:27.946 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T17:51:27.946 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:27.946 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 2, 2024-04-07T17:51:27.947 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:27.947 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:27.947 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T17:51:27.947 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:27.947 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:27.947 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:27.947 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.359582+0000", 2024-04-07T17:51:27.947 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:38.967914+0000", 2024-04-07T17:51:27.947 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:38.967914+0000", 2024-04-07T17:51:27.947 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:38.968390+0000", 2024-04-07T17:51:27.947 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.359582+0000", 2024-04-07T17:51:27.947 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:27.947 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:27.947 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:27.947 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:27.947 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.359582+0000", 2024-04-07T17:51:27.948 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.359582+0000", 2024-04-07T17:51:27.948 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.359582+0000", 2024-04-07T17:51:27.948 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:27.948 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:27.948 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:27.948 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.359582+0000", 2024-04-07T17:51:27.948 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.359582+0000", 2024-04-07T17:51:27.948 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:27.948 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 2, 2024-04-07T17:51:27.948 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:27.948 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:27.948 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:27.948 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:27.948 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:27.948 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:27.949 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:27.949 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 2, 2024-04-07T17:51:27.949 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:27.949 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:27.949 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:27.949 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.e", 2024-04-07T17:51:27.949 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:27.949 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:27.949 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:27.949 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4971, 2024-04-07T17:51:27.949 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:27.949 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T21:28:08.591091+0000", 2024-04-07T17:51:27.949 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:27.949 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:27.949 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:27.949 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:27.950 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:27.950 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:27.950 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:27.950 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:27.950 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:27.950 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:27.950 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:27.950 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:27.950 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:27.950 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:27.950 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:27.950 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:27.950 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:27.950 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:27.951 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:27.951 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 3, 2024-04-07T17:51:27.951 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 1, 2024-04-07T17:51:27.951 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:27.951 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 1, 2024-04-07T17:51:27.951 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:27.951 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:27.951 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:27.951 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:27.951 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:27.951 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 1, 2024-04-07T17:51:27.951 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:27.951 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:27.951 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:27.951 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:27.951 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:27.951 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:27.952 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:27.952 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 6, 2024-04-07T17:51:27.952 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 6, 2024-04-07T17:51:27.952 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:27.952 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:27.952 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:27.952 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 3, 2024-04-07T17:51:27.952 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:27.952 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.952 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:27.952 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:27.952 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:27.952 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T17:51:27.952 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:27.952 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T17:51:27.952 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:27.953 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 2, 2024-04-07T17:51:27.953 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "61'2" 2024-04-07T17:51:27.953 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.953 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:27.953 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:27.953 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:27.953 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:27.953 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T17:51:27.953 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:27.953 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 5, 2024-04-07T17:51:27.953 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:27.953 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:27.953 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:27.953 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:27.954 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:27.954 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:27.954 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.366524+0000", 2024-04-07T17:51:27.954 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:08.971755+0000", 2024-04-07T17:51:27.954 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:08.971755+0000", 2024-04-07T17:51:27.954 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:08.973166+0000", 2024-04-07T17:51:27.954 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.366524+0000", 2024-04-07T17:51:27.954 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:27.954 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:27.954 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:27.954 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 554, 2024-04-07T17:51:27.954 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.366524+0000", 2024-04-07T17:51:27.954 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.366524+0000", 2024-04-07T17:51:27.954 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.366524+0000", 2024-04-07T17:51:27.954 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:27.955 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:27.955 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:27.955 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.366524+0000", 2024-04-07T17:51:27.955 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.366524+0000", 2024-04-07T17:51:27.955 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:27.955 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:27.955 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:27.955 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:27.955 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 553, 2024-04-07T17:51:27.955 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:27.955 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:27.955 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:27.955 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:27.955 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:27.955 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:27.956 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:27.956 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:27.956 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.c", 2024-04-07T17:51:27.956 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:27.956 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:27.956 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:27.956 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4877, 2024-04-07T17:51:27.956 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:27.956 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T02:59:49.583756+0000", 2024-04-07T17:51:27.956 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:27.956 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:27.956 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:27.956 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:27.956 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:27.956 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:27.956 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:27.957 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:27.957 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:27.957 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:27.957 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:27.957 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:27.957 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:27.957 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:27.957 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:27.957 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:27.957 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:27.957 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:27.957 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:27.957 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:27.957 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:27.957 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:27.958 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:27.958 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:27.958 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:27.958 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:27.958 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:27.958 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:27.958 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:27.958 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:27.958 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:27.958 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:27.958 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:27.958 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:27.958 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:27.958 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:27.959 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:27.959 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:27.959 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:27.959 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:27.959 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:27.959 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:27.959 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:27.959 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.959 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:27.959 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:27.959 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:27.959 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:27.959 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:27.959 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T17:51:27.959 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:27.959 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 5, 2024-04-07T17:51:27.960 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:27.960 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.960 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:27.960 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:27.960 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:27.960 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:27.960 INFO:tasks.workunit.client.0.smithi100.stdout: 3 2024-04-07T17:51:27.960 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:27.960 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 1, 2024-04-07T17:51:27.960 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:27.960 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:27.960 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T17:51:27.960 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:27.960 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:27.960 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:27.960 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.333944+0000", 2024-04-07T17:51:27.961 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:24:13.932662+0000", 2024-04-07T17:51:27.961 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:24:13.932662+0000", 2024-04-07T17:51:27.961 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:24:13.933216+0000", 2024-04-07T17:51:27.961 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.333944+0000", 2024-04-07T17:51:27.961 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:27.961 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:27.961 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:27.961 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 500, 2024-04-07T17:51:27.961 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.333944+0000", 2024-04-07T17:51:27.961 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.333944+0000", 2024-04-07T17:51:27.961 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.333944+0000", 2024-04-07T17:51:27.961 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:27.961 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:27.961 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:27.961 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.333944+0000", 2024-04-07T17:51:27.962 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.333944+0000", 2024-04-07T17:51:27.962 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:27.962 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 1, 2024-04-07T17:51:27.962 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:27.962 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:27.962 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 499, 2024-04-07T17:51:27.962 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:27.962 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:27.962 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:27.962 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:27.962 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 1, 2024-04-07T17:51:27.962 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:27.962 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:27.962 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:27.962 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.d", 2024-04-07T17:51:27.963 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:27.963 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:27.963 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:27.963 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4983, 2024-04-07T17:51:27.963 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:27.963 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T17:17:11.134012+0000", 2024-04-07T17:51:27.963 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:27.963 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:27.963 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:27.963 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 70, 2024-04-07T17:51:27.963 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:27.963 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:27.963 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:27.963 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:27.963 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:27.963 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:27.964 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:27.964 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:27.964 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:27.964 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:27.964 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:27.964 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:27.964 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:27.964 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:27.964 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:27.964 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 3, 2024-04-07T17:51:27.964 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 1, 2024-04-07T17:51:27.964 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:27.964 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 1, 2024-04-07T17:51:27.964 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:27.964 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:27.964 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:27.965 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:27.965 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:27.965 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:27.965 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:27.965 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:27.965 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:27.965 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:27.965 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:27.965 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:27.965 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:27.965 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 1, 2024-04-07T17:51:27.965 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 1, 2024-04-07T17:51:27.965 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:27.965 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:27.965 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:27.965 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 1, 2024-04-07T17:51:27.966 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 1 2024-04-07T17:51:27.966 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.966 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:27.966 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:27.966 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:27.966 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:27.966 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:27.966 INFO:tasks.workunit.client.0.smithi100.stdout: 3 2024-04-07T17:51:27.966 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:27.966 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 1, 2024-04-07T17:51:27.966 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "61'1" 2024-04-07T17:51:27.966 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.966 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:27.966 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:27.966 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:27.967 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:27.967 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T17:51:27.967 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:27.967 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 7, 2024-04-07T17:51:27.967 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:27.967 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:27.967 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:27.967 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:27.967 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:27.967 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:27.967 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.373148+0000", 2024-04-07T17:51:27.967 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:38.989051+0000", 2024-04-07T17:51:27.967 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:38.989051+0000", 2024-04-07T17:51:27.967 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:39.000662+0000", 2024-04-07T17:51:27.968 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.373148+0000", 2024-04-07T17:51:27.968 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:27.968 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:27.968 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:27.968 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:27.968 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.373148+0000", 2024-04-07T17:51:27.968 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.373148+0000", 2024-04-07T17:51:27.968 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.373148+0000", 2024-04-07T17:51:27.968 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:27.968 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:27.968 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:27.968 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.373148+0000", 2024-04-07T17:51:27.968 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.373148+0000", 2024-04-07T17:51:27.968 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:27.968 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:27.969 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:27.969 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:27.969 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:27.969 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:27.969 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:27.969 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:27.969 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:27.969 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:27.969 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:27.969 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:27.969 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:27.969 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.d", 2024-04-07T17:51:27.969 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:27.969 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:27.970 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:27.970 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4803, 2024-04-07T17:51:27.970 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:27.970 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T23:45:21.166051+0000", 2024-04-07T17:51:27.970 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:27.970 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:27.970 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:27.970 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:27.970 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:27.970 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:27.970 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:27.970 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:27.970 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:27.970 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:27.971 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:27.971 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:27.971 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:27.971 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:27.971 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:27.971 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:27.971 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:27.971 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:27.971 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:27.971 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:27.971 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:27.971 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:27.971 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:27.971 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:27.971 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:27.972 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:27.972 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:27.972 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:27.972 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:27.972 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:27.972 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:27.972 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:27.972 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:27.972 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:27.972 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:27.972 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:27.972 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:27.972 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:27.972 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:27.972 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:27.972 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:27.972 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:27.973 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:27.973 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.973 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:27.973 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:27.973 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:27.973 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:27.973 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:27.973 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T17:51:27.973 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:27.973 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 7, 2024-04-07T17:51:27.973 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:27.973 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.973 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:27.973 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:27.973 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T17:51:27.973 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:27.974 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T17:51:27.974 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:27.974 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 2, 2024-04-07T17:51:27.974 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:27.974 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:27.974 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T17:51:27.974 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:27.974 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:27.974 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:27.974 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.355658+0000", 2024-04-07T17:51:27.974 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:24:41.549188+0000", 2024-04-07T17:51:27.974 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:24:41.549188+0000", 2024-04-07T17:51:27.974 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:24:41.550837+0000", 2024-04-07T17:51:27.974 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.355658+0000", 2024-04-07T17:51:27.975 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:27.975 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:27.975 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:27.975 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 527, 2024-04-07T17:51:27.975 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.355658+0000", 2024-04-07T17:51:27.975 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.355658+0000", 2024-04-07T17:51:27.975 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.355658+0000", 2024-04-07T17:51:27.975 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:27.975 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:27.975 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:27.975 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.355658+0000", 2024-04-07T17:51:27.975 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.355658+0000", 2024-04-07T17:51:27.975 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:27.975 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:27.975 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:27.975 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:27.976 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 526, 2024-04-07T17:51:27.976 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:27.976 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:27.976 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:27.976 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:27.976 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:27.976 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:27.976 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:27.976 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:27.976 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.c", 2024-04-07T17:51:27.976 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:27.976 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:27.976 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:27.976 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4973, 2024-04-07T17:51:27.976 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:27.977 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T18:56:44.543940+0000", 2024-04-07T17:51:27.977 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:27.977 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:27.977 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:27.977 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:27.977 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:27.977 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:27.977 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:27.977 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:27.977 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:27.977 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:27.977 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:27.977 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:27.977 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:27.977 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:27.978 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:27.978 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:27.978 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:27.978 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:27.978 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:27.978 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:27.978 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:27.978 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:27.978 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:27.978 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:27.978 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:27.978 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:27.978 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:27.978 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:27.978 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:27.979 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:27.979 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:27.979 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:27.979 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:27.979 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:27.979 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:27.979 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:27.979 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:27.979 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:27.979 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:27.979 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:27.979 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:27.979 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:27.979 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:27.979 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.980 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:27.980 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:27.980 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:27.980 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T17:51:27.980 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:27.980 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T17:51:27.980 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:27.980 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 2, 2024-04-07T17:51:27.980 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:27.980 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.980 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:27.980 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:27.980 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:27.980 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:27.980 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T17:51:27.980 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:27.981 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 6, 2024-04-07T17:51:27.981 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:27.981 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:27.981 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:27.981 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:27.981 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:27.981 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:27.981 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.386750+0000", 2024-04-07T17:51:27.981 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:08.966124+0000", 2024-04-07T17:51:27.981 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:08.966124+0000", 2024-04-07T17:51:27.981 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:08.966628+0000", 2024-04-07T17:51:27.981 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.386750+0000", 2024-04-07T17:51:27.981 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:27.981 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:27.981 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:27.982 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 554, 2024-04-07T17:51:27.982 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.386750+0000", 2024-04-07T17:51:27.982 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.386750+0000", 2024-04-07T17:51:27.982 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.386750+0000", 2024-04-07T17:51:27.982 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:27.982 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:27.982 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:27.982 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.386750+0000", 2024-04-07T17:51:27.982 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.386750+0000", 2024-04-07T17:51:27.982 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:27.982 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:27.982 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:27.982 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:27.982 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 553, 2024-04-07T17:51:27.983 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:27.983 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:27.983 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:27.983 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:27.983 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:27.983 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:27.983 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:27.983 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:27.983 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.a", 2024-04-07T17:51:27.983 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:27.983 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:27.983 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:27.983 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4847, 2024-04-07T17:51:27.983 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:27.983 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T19:57:28.644470+0000", 2024-04-07T17:51:27.983 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:27.983 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:27.984 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:27.984 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:27.984 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:27.984 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:27.984 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:27.984 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:27.984 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:27.984 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:27.984 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:27.984 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:27.984 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:27.984 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:27.984 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:27.984 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:27.984 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:27.985 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:27.985 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:27.985 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:27.985 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:27.985 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:27.985 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:27.985 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:27.985 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:27.985 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:27.985 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:27.985 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:27.985 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:27.985 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:27.985 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:27.985 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:27.985 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:27.986 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:27.986 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:27.986 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:27.986 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:27.986 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:27.986 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:27.986 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:27.986 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:27.986 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:27.986 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:27.986 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.986 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:27.986 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:27.986 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:27.987 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:27.987 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:27.987 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T17:51:27.987 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:27.987 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 6, 2024-04-07T17:51:27.987 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:27.987 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.987 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:27.987 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:27.987 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:27.987 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:27.987 INFO:tasks.workunit.client.0.smithi100.stdout: 5 2024-04-07T17:51:27.987 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:27.987 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 7, 2024-04-07T17:51:27.987 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:27.987 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:27.987 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T17:51:27.988 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:27.988 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:27.988 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:27.988 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.369146+0000", 2024-04-07T17:51:27.988 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:38.990309+0000", 2024-04-07T17:51:27.988 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:38.990309+0000", 2024-04-07T17:51:27.988 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:38.992642+0000", 2024-04-07T17:51:27.988 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.369146+0000", 2024-04-07T17:51:27.988 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:27.988 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:27.988 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:27.988 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:27.988 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.369146+0000", 2024-04-07T17:51:27.988 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.369146+0000", 2024-04-07T17:51:27.988 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.369146+0000", 2024-04-07T17:51:27.988 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:27.989 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:27.989 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:27.989 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.369146+0000", 2024-04-07T17:51:27.989 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.369146+0000", 2024-04-07T17:51:27.989 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:27.989 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 3, 2024-04-07T17:51:27.989 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:27.989 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:27.989 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:27.989 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:27.989 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:27.989 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:27.989 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:27.989 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 3, 2024-04-07T17:51:27.989 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:27.990 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:27.990 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:27.990 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.b", 2024-04-07T17:51:27.990 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:27.990 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:27.990 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:27.990 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4823, 2024-04-07T17:51:27.990 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:27.990 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T17:09:27.535561+0000", 2024-04-07T17:51:27.990 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:27.990 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:27.990 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:27.990 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:27.990 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:27.991 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:27.991 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:27.991 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:27.991 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:27.991 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:27.991 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:27.991 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:27.991 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:27.991 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:27.991 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:27.991 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:27.991 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:27.991 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:27.991 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:27.991 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 3, 2024-04-07T17:51:27.991 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 1, 2024-04-07T17:51:27.992 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:27.992 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 1, 2024-04-07T17:51:27.992 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:27.992 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:27.992 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:27.992 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:27.992 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:27.992 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 1, 2024-04-07T17:51:27.992 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:27.992 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:27.992 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:27.992 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:27.992 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:27.992 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:27.992 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:27.993 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 2, 2024-04-07T17:51:27.993 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 2, 2024-04-07T17:51:27.993 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:27.993 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:27.993 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:27.993 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 1, 2024-04-07T17:51:27.993 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:27.993 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.993 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:27.993 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:27.993 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:27.993 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:27.993 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:27.993 INFO:tasks.workunit.client.0.smithi100.stdout: 5 2024-04-07T17:51:27.993 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:27.993 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 7, 2024-04-07T17:51:27.993 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "623'3" 2024-04-07T17:51:27.994 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:27.994 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:27.994 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:27.994 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:27.994 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:27.994 INFO:tasks.workunit.client.0.smithi100.stdout: 4 2024-04-07T17:51:27.994 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:27.994 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 3, 2024-04-07T17:51:27.994 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:27.994 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:27.994 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:27.994 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:27.994 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:27.994 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:27.994 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:35.976901+0000", 2024-04-07T17:51:27.995 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:24:13.933754+0000", 2024-04-07T17:51:27.995 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:24:13.933754+0000", 2024-04-07T17:51:27.995 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:24:13.934742+0000", 2024-04-07T17:51:27.995 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:35.976901+0000", 2024-04-07T17:51:27.995 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:27.995 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:27.995 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:27.995 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 500, 2024-04-07T17:51:27.995 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:35.976901+0000", 2024-04-07T17:51:27.995 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:35.976901+0000", 2024-04-07T17:51:27.995 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:35.976901+0000", 2024-04-07T17:51:27.995 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:27.995 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:27.995 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:27.996 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:35.976901+0000", 2024-04-07T17:51:27.996 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:35.976901+0000", 2024-04-07T17:51:27.996 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:27.996 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:27.996 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:27.996 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:27.996 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 499, 2024-04-07T17:51:27.996 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:27.996 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:27.996 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:27.996 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:27.996 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:27.996 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:27.996 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:27.996 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:27.997 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.b", 2024-04-07T17:51:27.997 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:27.997 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:27.997 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:27.997 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4938, 2024-04-07T17:51:27.997 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:27.997 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T02:43:13.536050+0000", 2024-04-07T17:51:27.997 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:27.997 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:27.997 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:27.997 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:27.997 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:27.997 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:27.997 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:27.997 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:27.998 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:27.998 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:27.998 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:27.998 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:27.998 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:27.998 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:27.998 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:27.998 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:27.998 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:27.998 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:27.998 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:27.998 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:27.998 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:27.998 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:27.999 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:27.999 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:27.999 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:27.999 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:27.999 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:27.999 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:27.999 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:27.999 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:27.999 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:27.999 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:27.999 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:27.999 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:27.999 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:27.999 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:27.999 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:28.000 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:28.000 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:28.000 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:28.000 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:28.000 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:28.000 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:28.000 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.000 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:28.000 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:28.000 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:28.000 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:28.000 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:28.000 INFO:tasks.workunit.client.0.smithi100.stdout: 4 2024-04-07T17:51:28.000 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.000 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 3, 2024-04-07T17:51:28.001 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:28.001 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.001 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:28.001 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:28.001 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:28.001 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:28.001 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T17:51:28.001 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.001 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 1, 2024-04-07T17:51:28.001 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:28.001 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:28.001 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T17:51:28.001 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:28.001 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:28.001 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:28.002 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.335142+0000", 2024-04-07T17:51:28.002 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:38.975734+0000", 2024-04-07T17:51:28.002 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:38.975734+0000", 2024-04-07T17:51:28.002 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:38.976248+0000", 2024-04-07T17:51:28.002 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.335142+0000", 2024-04-07T17:51:28.002 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:28.002 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:28.002 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:28.002 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:28.002 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.335142+0000", 2024-04-07T17:51:28.002 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.335142+0000", 2024-04-07T17:51:28.002 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.335142+0000", 2024-04-07T17:51:28.002 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:28.002 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:28.003 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:28.003 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.335142+0000", 2024-04-07T17:51:28.003 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.335142+0000", 2024-04-07T17:51:28.003 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:28.003 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:28.003 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:28.003 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:28.003 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:28.003 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:28.003 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:28.003 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:28.003 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:28.003 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:28.003 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:28.003 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:28.003 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:28.004 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.a", 2024-04-07T17:51:28.004 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:28.004 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:28.004 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:28.004 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4981, 2024-04-07T17:51:28.004 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:28.004 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T19:45:35.140357+0000", 2024-04-07T17:51:28.004 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:28.004 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:28.004 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:28.004 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:28.004 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:28.004 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:28.004 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:28.004 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:28.005 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:28.005 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:28.005 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:28.005 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:28.005 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:28.005 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:28.005 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:28.005 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:28.005 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:28.005 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:28.005 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:28.005 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:28.005 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:28.005 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:28.005 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:28.005 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:28.006 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:28.006 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:28.006 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:28.006 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:28.006 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:28.006 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:28.006 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:28.006 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:28.006 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:28.006 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:28.006 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:28.006 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:28.006 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:28.006 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:28.006 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:28.007 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:28.007 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:28.007 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:28.007 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:28.007 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.007 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:28.007 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:28.007 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:28.007 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:28.007 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:28.007 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T17:51:28.007 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.007 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 1, 2024-04-07T17:51:28.007 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:28.007 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.007 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:28.008 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:28.008 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:28.008 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:28.008 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T17:51:28.008 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.008 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 3, 2024-04-07T17:51:28.008 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:28.008 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:28.008 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:28.008 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:28.008 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:28.008 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:28.008 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:35.975502+0000", 2024-04-07T17:51:28.008 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:38.997582+0000", 2024-04-07T17:51:28.008 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:38.997582+0000", 2024-04-07T17:51:28.008 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:38.998502+0000", 2024-04-07T17:51:28.009 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:35.975502+0000", 2024-04-07T17:51:28.009 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:28.009 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:28.009 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:28.009 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:28.009 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:35.975502+0000", 2024-04-07T17:51:28.009 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:35.975502+0000", 2024-04-07T17:51:28.009 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:35.975502+0000", 2024-04-07T17:51:28.009 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:28.009 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:28.009 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:28.009 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:35.975502+0000", 2024-04-07T17:51:28.009 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:35.975502+0000", 2024-04-07T17:51:28.009 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:28.009 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:28.009 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:28.010 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:28.010 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:28.010 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:28.010 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:28.010 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:28.010 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:28.010 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:28.010 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:28.010 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:28.010 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:28.010 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.8", 2024-04-07T17:51:28.010 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:28.010 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:28.010 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:28.011 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4968, 2024-04-07T17:51:28.011 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:28.011 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T23:09:31.581874+0000", 2024-04-07T17:51:28.011 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:28.011 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:28.011 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:28.011 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:28.011 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:28.011 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:28.011 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:28.011 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:28.011 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:28.011 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:28.011 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:28.011 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:28.011 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:28.012 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:28.012 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:28.012 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:28.012 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:28.012 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:28.012 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:28.012 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:28.012 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:28.012 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:28.012 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:28.012 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:28.012 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:28.012 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:28.012 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:28.012 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:28.012 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:28.013 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:28.013 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:28.013 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:28.013 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:28.013 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:28.013 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:28.013 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:28.013 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:28.013 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:28.013 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:28.013 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:28.013 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:28.013 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:28.013 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:28.013 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.013 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:28.014 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:28.014 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:28.014 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:28.014 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:28.014 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T17:51:28.014 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.014 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 3, 2024-04-07T17:51:28.014 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:28.014 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.014 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:28.014 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:28.014 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:28.014 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:28.015 INFO:tasks.workunit.client.0.smithi100.stdout: 3 2024-04-07T17:51:28.015 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.015 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 1, 2024-04-07T17:51:28.015 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:28.015 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:28.015 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T17:51:28.015 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:28.015 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:28.015 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:28.015 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.331976+0000", 2024-04-07T17:51:28.015 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:38.992340+0000", 2024-04-07T17:51:28.015 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:38.992340+0000", 2024-04-07T17:51:28.015 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:38.992779+0000", 2024-04-07T17:51:28.015 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.331976+0000", 2024-04-07T17:51:28.015 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:28.015 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:28.016 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:28.016 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:28.016 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.331976+0000", 2024-04-07T17:51:28.016 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.331976+0000", 2024-04-07T17:51:28.016 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.331976+0000", 2024-04-07T17:51:28.016 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:28.016 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:28.016 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:28.016 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.331976+0000", 2024-04-07T17:51:28.016 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.331976+0000", 2024-04-07T17:51:28.016 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:28.016 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:28.016 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:28.016 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:28.016 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:28.017 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:28.017 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:28.017 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:28.017 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:28.017 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:28.017 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:28.017 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:28.017 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:28.017 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.9", 2024-04-07T17:51:28.017 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:28.017 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:28.017 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:28.017 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4981, 2024-04-07T17:51:28.017 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:28.017 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T21:25:43.908090+0000", 2024-04-07T17:51:28.017 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:28.018 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:28.018 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:28.018 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:28.018 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:28.018 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:28.018 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:28.018 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:28.018 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:28.018 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:28.018 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:28.018 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:28.018 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:28.018 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:28.018 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:28.018 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:28.019 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:28.019 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:28.019 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:28.019 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:28.019 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:28.019 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:28.019 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:28.019 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:28.019 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:28.019 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:28.019 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:28.019 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:28.019 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:28.019 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:28.019 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:28.019 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:28.020 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:28.020 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:28.020 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:28.020 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:28.020 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:28.020 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:28.020 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:28.020 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:28.020 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:28.020 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:28.020 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:28.020 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.020 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:28.020 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:28.020 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:28.020 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:28.020 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:28.021 INFO:tasks.workunit.client.0.smithi100.stdout: 3 2024-04-07T17:51:28.021 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.021 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 1, 2024-04-07T17:51:28.021 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:28.021 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.021 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:28.021 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:28.021 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:28.021 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T17:51:28.021 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T17:51:28.021 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.021 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 4, 2024-04-07T17:51:28.021 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:28.021 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:28.021 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:28.021 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:28.022 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:28.022 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:28.022 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:35.977778+0000", 2024-04-07T17:51:28.022 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:38.985206+0000", 2024-04-07T17:51:28.022 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:38.985206+0000", 2024-04-07T17:51:28.022 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:38.985745+0000", 2024-04-07T17:51:28.022 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:35.977778+0000", 2024-04-07T17:51:28.022 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:28.022 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:28.022 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:28.022 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:28.022 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:35.977778+0000", 2024-04-07T17:51:28.022 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:35.977778+0000", 2024-04-07T17:51:28.022 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:35.977778+0000", 2024-04-07T17:51:28.023 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:28.023 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:28.023 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:28.023 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:35.977778+0000", 2024-04-07T17:51:28.023 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:35.977778+0000", 2024-04-07T17:51:28.023 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:28.023 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:28.023 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:28.023 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:28.023 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:28.023 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:28.023 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:28.023 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:28.023 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:28.023 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:28.023 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:28.023 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:28.024 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:28.024 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.9", 2024-04-07T17:51:28.024 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:28.024 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:28.024 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:28.024 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4931, 2024-04-07T17:51:28.024 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:28.024 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T00:51:54.981164+0000", 2024-04-07T17:51:28.024 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:28.024 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:28.024 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:28.024 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:28.024 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:28.024 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:28.024 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:28.025 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:28.025 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:28.025 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:28.025 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:28.025 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:28.025 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:28.025 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:28.025 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:28.025 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:28.025 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:28.025 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:28.025 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:28.025 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:28.025 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:28.025 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:28.025 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:28.026 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:28.026 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:28.026 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:28.026 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:28.026 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:28.026 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:28.026 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:28.026 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:28.026 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:28.026 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:28.026 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:28.026 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:28.026 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:28.027 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:28.027 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:28.027 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:28.027 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:28.027 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:28.027 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:28.027 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:28.027 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.027 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:28.027 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:28.027 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:28.027 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:28.027 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T17:51:28.027 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T17:51:28.027 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.027 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 4, 2024-04-07T17:51:28.028 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:28.028 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.028 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:28.028 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:28.028 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:28.028 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:28.028 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T17:51:28.028 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.028 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 7, 2024-04-07T17:51:28.028 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:28.028 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:28.028 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T17:51:28.028 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:28.028 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:28.028 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:28.028 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.364254+0000", 2024-04-07T17:51:28.029 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:38.996008+0000", 2024-04-07T17:51:28.029 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:38.996008+0000", 2024-04-07T17:51:28.029 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:39.001597+0000", 2024-04-07T17:51:28.029 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.364254+0000", 2024-04-07T17:51:28.029 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:28.029 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:28.029 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:28.029 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:28.029 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.364254+0000", 2024-04-07T17:51:28.029 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.364254+0000", 2024-04-07T17:51:28.029 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.364254+0000", 2024-04-07T17:51:28.029 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:28.029 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:28.029 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:28.029 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.364254+0000", 2024-04-07T17:51:28.029 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.364254+0000", 2024-04-07T17:51:28.030 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:28.030 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:28.030 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:28.030 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:28.030 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:28.030 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:28.030 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:28.030 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:28.030 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:28.030 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:28.030 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:28.030 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:28.030 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:28.030 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.8", 2024-04-07T17:51:28.030 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:28.031 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:28.031 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:28.031 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4792, 2024-04-07T17:51:28.031 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:28.031 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T19:53:15.918325+0000", 2024-04-07T17:51:28.031 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:28.031 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:28.031 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:28.031 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:28.031 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:28.031 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:28.031 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:28.031 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:28.031 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:28.031 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:28.031 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:28.032 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:28.032 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:28.032 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:28.032 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:28.032 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:28.032 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:28.032 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:28.032 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:28.032 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:28.032 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:28.032 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:28.032 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:28.032 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:28.032 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:28.032 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:28.032 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:28.033 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:28.033 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:28.033 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:28.033 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:28.033 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:28.033 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:28.033 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:28.033 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:28.033 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:28.033 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:28.033 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:28.033 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:28.033 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:28.033 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:28.033 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:28.033 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:28.034 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.034 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:28.034 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:28.034 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:28.034 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:28.034 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:28.034 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T17:51:28.034 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.034 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 7, 2024-04-07T17:51:28.034 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:28.034 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.034 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:28.034 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:28.034 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:28.034 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:28.035 INFO:tasks.workunit.client.0.smithi100.stdout: 4 2024-04-07T17:51:28.035 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.035 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 0, 2024-04-07T17:51:28.035 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:28.035 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:28.035 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:28.035 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:28.035 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:28.035 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:28.035 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:35.981688+0000", 2024-04-07T17:51:28.035 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:24:13.962953+0000", 2024-04-07T17:51:28.035 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:24:13.962953+0000", 2024-04-07T17:51:28.035 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:24:13.963599+0000", 2024-04-07T17:51:28.035 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:35.981688+0000", 2024-04-07T17:51:28.035 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:28.036 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:28.036 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:28.036 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 500, 2024-04-07T17:51:28.036 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:35.981688+0000", 2024-04-07T17:51:28.036 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:35.981688+0000", 2024-04-07T17:51:28.036 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:35.981688+0000", 2024-04-07T17:51:28.036 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:28.036 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:28.036 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:28.036 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:35.981688+0000", 2024-04-07T17:51:28.036 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:35.981688+0000", 2024-04-07T17:51:28.036 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:28.036 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:28.036 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:28.037 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:28.037 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 499, 2024-04-07T17:51:28.037 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:28.037 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:28.037 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:28.037 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:28.037 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:28.037 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:28.037 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:28.037 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:28.037 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.6", 2024-04-07T17:51:28.037 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:28.037 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:28.037 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:28.037 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4999, 2024-04-07T17:51:28.037 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:28.038 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T22:29:41.083754+0000", 2024-04-07T17:51:28.038 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:28.038 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:28.038 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:28.038 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:28.038 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:28.038 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:28.038 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:28.038 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:28.038 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:28.038 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:28.038 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:28.038 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:28.038 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:28.038 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:28.038 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:28.039 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:28.039 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:28.039 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:28.039 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:28.039 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:28.039 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:28.039 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:28.039 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:28.039 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:28.039 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:28.039 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:28.039 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:28.039 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:28.039 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:28.039 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:28.039 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:28.040 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:28.040 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:28.040 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:28.040 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:28.040 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:28.040 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:28.040 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:28.040 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:28.040 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:28.040 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:28.040 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:28.040 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:28.040 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.040 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:28.041 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:28.041 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:28.041 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:28.041 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:28.041 INFO:tasks.workunit.client.0.smithi100.stdout: 4 2024-04-07T17:51:28.041 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.041 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 0, 2024-04-07T17:51:28.041 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:28.041 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.041 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:28.041 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:28.041 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:28.041 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:28.041 INFO:tasks.workunit.client.0.smithi100.stdout: 2 2024-04-07T17:51:28.041 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.042 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 6, 2024-04-07T17:51:28.042 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:28.042 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:28.042 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T17:51:28.042 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:28.042 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:28.042 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:28.042 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.369806+0000", 2024-04-07T17:51:28.042 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:08.974754+0000", 2024-04-07T17:51:28.042 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:08.974754+0000", 2024-04-07T17:51:28.042 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:08.975487+0000", 2024-04-07T17:51:28.042 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.369806+0000", 2024-04-07T17:51:28.042 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:28.042 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:28.043 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:28.043 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 554, 2024-04-07T17:51:28.043 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.369806+0000", 2024-04-07T17:51:28.043 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.369806+0000", 2024-04-07T17:51:28.043 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.369806+0000", 2024-04-07T17:51:28.043 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:28.043 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:28.043 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:28.043 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.369806+0000", 2024-04-07T17:51:28.043 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.369806+0000", 2024-04-07T17:51:28.043 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:28.043 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 2, 2024-04-07T17:51:28.043 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:28.043 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:28.044 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 553, 2024-04-07T17:51:28.044 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:28.044 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:28.044 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:28.044 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:28.044 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 2, 2024-04-07T17:51:28.044 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:28.044 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:28.044 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:28.044 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.7", 2024-04-07T17:51:28.044 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:28.044 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:28.044 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:28.044 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4865, 2024-04-07T17:51:28.044 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:28.044 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T01:37:46.529471+0000", 2024-04-07T17:51:28.044 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:28.045 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:28.045 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:28.045 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:28.045 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:28.045 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:28.045 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:28.045 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:28.045 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:28.045 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:28.045 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:28.045 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:28.045 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:28.045 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:28.045 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:28.045 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:28.045 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:28.046 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:28.046 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:28.046 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 3, 2024-04-07T17:51:28.046 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 1, 2024-04-07T17:51:28.046 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:28.046 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 1, 2024-04-07T17:51:28.046 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:28.046 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:28.046 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:28.046 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:28.046 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:28.046 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 1, 2024-04-07T17:51:28.046 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:28.046 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:28.046 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:28.047 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:28.047 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:28.047 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:28.047 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:28.047 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 6, 2024-04-07T17:51:28.047 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 6, 2024-04-07T17:51:28.047 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:28.047 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:28.047 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:28.047 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 3, 2024-04-07T17:51:28.047 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:28.047 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.047 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:28.047 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:28.047 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:28.047 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:28.047 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:28.048 INFO:tasks.workunit.client.0.smithi100.stdout: 2 2024-04-07T17:51:28.048 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.048 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 6, 2024-04-07T17:51:28.048 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "61'2" 2024-04-07T17:51:28.048 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.048 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:28.048 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:28.048 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:28.048 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:28.048 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T17:51:28.048 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.048 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 4, 2024-04-07T17:51:28.048 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:28.048 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:28.048 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 66, 2024-04-07T17:51:28.048 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:28.049 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:28.049 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:28.049 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:35.977631+0000", 2024-04-07T17:51:28.049 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:08.952433+0000", 2024-04-07T17:51:28.049 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:08.952433+0000", 2024-04-07T17:51:28.049 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:59:56.534054+0000", 2024-04-07T17:51:28.049 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:35.977631+0000", 2024-04-07T17:51:28.049 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:13:28.963534+0000", 2024-04-07T17:51:28.049 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:28.049 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:13:28.963534+0000", 2024-04-07T17:51:28.049 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 554, 2024-04-07T17:51:28.049 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:35.977631+0000", 2024-04-07T17:51:28.049 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:35.977631+0000", 2024-04-07T17:51:28.049 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:35.977631+0000", 2024-04-07T17:51:28.049 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:28.050 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:28.050 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:13:28.963534+0000", 2024-04-07T17:51:28.050 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:35.977631+0000", 2024-04-07T17:51:28.050 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:35.977631+0000", 2024-04-07T17:51:28.050 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:28.050 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 3036, 2024-04-07T17:51:28.050 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:28.050 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:28.050 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 553, 2024-04-07T17:51:28.050 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:28.050 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:28.050 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:28.050 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:28.050 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 3036, 2024-04-07T17:51:28.050 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:28.051 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:28.051 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:28.051 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "4.1", 2024-04-07T17:51:28.051 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:28.051 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:28.051 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:28.051 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 8679, 2024-04-07T17:51:28.051 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:28.051 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T00:59:44.742269+0000", 2024-04-07T17:51:28.051 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.007050569, 2024-04-07T17:51:28.051 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:28.051 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:28.051 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 4194304, 2024-04-07T17:51:28.051 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:28.051 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:28.051 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:28.052 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:28.052 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:28.052 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:28.052 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:28.052 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:28.052 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:28.052 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:28.052 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:28.052 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:28.052 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:28.052 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:28.052 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:28.052 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 3, 2024-04-07T17:51:28.052 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 1, 2024-04-07T17:51:28.052 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:28.052 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 1, 2024-04-07T17:51:28.053 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:28.053 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:28.053 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:28.053 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:28.053 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:28.053 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:28.053 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:28.053 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:28.053 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:28.053 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:28.053 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:28.053 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:28.053 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:28.053 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 1087, 2024-04-07T17:51:28.053 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 402424, 2024-04-07T17:51:28.054 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:28.054 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:28.054 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:28.054 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 3017, 2024-04-07T17:51:28.054 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 407157 2024-04-07T17:51:28.054 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.054 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:28.054 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:28.054 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:28.054 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:28.054 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:28.054 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T17:51:28.054 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.054 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 4, 2024-04-07T17:51:28.054 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "1083'3036" 2024-04-07T17:51:28.055 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.055 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:28.055 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:28.055 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:28.055 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:28.055 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T17:51:28.055 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.055 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 3, 2024-04-07T17:51:28.055 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:28.055 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:28.055 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:28.055 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:28.055 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:28.055 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:28.055 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:35.981722+0000", 2024-04-07T17:51:28.056 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:38.974388+0000", 2024-04-07T17:51:28.056 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:38.974388+0000", 2024-04-07T17:51:28.056 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:38.974763+0000", 2024-04-07T17:51:28.056 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:35.981722+0000", 2024-04-07T17:51:28.056 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:28.056 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:28.056 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:28.056 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:28.056 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:35.981722+0000", 2024-04-07T17:51:28.056 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:35.981722+0000", 2024-04-07T17:51:28.056 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:35.981722+0000", 2024-04-07T17:51:28.056 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:28.056 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:28.056 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:28.056 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:35.981722+0000", 2024-04-07T17:51:28.056 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:35.981722+0000", 2024-04-07T17:51:28.057 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:28.057 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:28.057 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:28.057 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:28.057 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:28.057 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:28.057 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:28.057 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:28.057 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:28.057 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:28.057 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:28.057 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:28.057 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:28.057 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.7", 2024-04-07T17:51:28.057 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:28.057 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:28.057 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:28.058 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4940, 2024-04-07T17:51:28.058 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:28.058 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T03:43:59.412072+0000", 2024-04-07T17:51:28.058 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:28.058 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:28.058 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:28.058 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:28.058 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:28.058 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:28.058 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:28.058 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:28.058 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:28.058 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:28.058 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:28.058 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:28.059 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:28.059 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:28.059 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:28.059 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:28.059 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:28.059 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:28.059 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:28.059 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:28.059 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:28.059 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:28.059 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:28.059 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:28.059 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:28.059 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:28.059 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:28.059 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:28.060 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:28.060 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:28.060 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:28.060 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:28.060 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:28.060 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:28.060 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:28.060 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:28.060 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:28.060 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:28.060 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:28.060 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:28.060 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:28.060 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:28.060 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:28.060 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.060 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:28.061 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:28.061 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:28.061 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:28.061 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:28.061 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T17:51:28.061 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.061 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 3, 2024-04-07T17:51:28.061 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:28.061 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.061 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:28.061 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:28.061 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:28.061 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:28.061 INFO:tasks.workunit.client.0.smithi100.stdout: 4 2024-04-07T17:51:28.061 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.061 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 1, 2024-04-07T17:51:28.061 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:28.062 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:28.062 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T17:51:28.062 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:28.062 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:28.062 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:28.062 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.335258+0000", 2024-04-07T17:51:28.062 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:08.962773+0000", 2024-04-07T17:51:28.062 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:08.962773+0000", 2024-04-07T17:51:28.062 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:08.963320+0000", 2024-04-07T17:51:28.062 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.335258+0000", 2024-04-07T17:51:28.062 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:28.062 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:28.062 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:28.062 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 554, 2024-04-07T17:51:28.063 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.335258+0000", 2024-04-07T17:51:28.063 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.335258+0000", 2024-04-07T17:51:28.063 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.335258+0000", 2024-04-07T17:51:28.063 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:28.063 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:28.063 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:28.063 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.335258+0000", 2024-04-07T17:51:28.063 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.335258+0000", 2024-04-07T17:51:28.063 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:28.063 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 2, 2024-04-07T17:51:28.063 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:28.063 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:28.063 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 553, 2024-04-07T17:51:28.063 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:28.063 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:28.064 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:28.064 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:28.064 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 2, 2024-04-07T17:51:28.064 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:28.064 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:28.064 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:28.064 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.6", 2024-04-07T17:51:28.064 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:28.064 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:28.064 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:28.064 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4986, 2024-04-07T17:51:28.064 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:28.064 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T18:43:13.545482+0000", 2024-04-07T17:51:28.064 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:28.064 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:28.064 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:28.065 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:28.065 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:28.065 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:28.065 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:28.065 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:28.065 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:28.065 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:28.065 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:28.065 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:28.065 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:28.065 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:28.065 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:28.065 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:28.065 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:28.065 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:28.066 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:28.066 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 3, 2024-04-07T17:51:28.066 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 1, 2024-04-07T17:51:28.066 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:28.066 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 1, 2024-04-07T17:51:28.066 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:28.066 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:28.066 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:28.066 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:28.066 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:28.066 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 1, 2024-04-07T17:51:28.066 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:28.066 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:28.066 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:28.066 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:28.066 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:28.067 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:28.067 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:28.067 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 2, 2024-04-07T17:51:28.067 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 2, 2024-04-07T17:51:28.067 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:28.067 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:28.067 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:28.067 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 2, 2024-04-07T17:51:28.067 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:28.067 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.067 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:28.067 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:28.067 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:28.067 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:28.067 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:28.068 INFO:tasks.workunit.client.0.smithi100.stdout: 4 2024-04-07T17:51:28.068 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.068 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 1, 2024-04-07T17:51:28.068 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "627'2" 2024-04-07T17:51:28.068 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.068 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:28.068 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:28.068 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:28.068 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:28.068 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T17:51:28.068 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.068 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 3, 2024-04-07T17:51:28.068 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:28.068 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:28.068 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 66, 2024-04-07T17:51:28.068 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:28.068 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:28.069 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:28.069 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:35.975295+0000", 2024-04-07T17:51:28.069 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:38.997956+0000", 2024-04-07T17:51:28.069 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:38.997956+0000", 2024-04-07T17:51:28.069 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:59:56.537779+0000", 2024-04-07T17:51:28.069 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:35.975295+0000", 2024-04-07T17:51:28.069 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:13:28.963534+0000", 2024-04-07T17:51:28.069 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:28.069 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:13:28.963534+0000", 2024-04-07T17:51:28.069 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:28.069 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:35.975295+0000", 2024-04-07T17:51:28.069 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:35.975295+0000", 2024-04-07T17:51:28.069 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:35.975295+0000", 2024-04-07T17:51:28.069 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:28.069 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:28.069 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:13:28.963534+0000", 2024-04-07T17:51:28.070 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:35.975295+0000", 2024-04-07T17:51:28.070 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:35.975295+0000", 2024-04-07T17:51:28.070 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:28.070 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 3089, 2024-04-07T17:51:28.070 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:28.070 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:28.070 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:28.070 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:28.070 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:28.070 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:28.070 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:28.070 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 3089, 2024-04-07T17:51:28.070 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:28.070 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:28.071 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:28.071 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "4.0", 2024-04-07T17:51:28.071 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:28.071 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:28.071 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:28.071 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 8748, 2024-04-07T17:51:28.071 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:28.071 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T16:44:21.465234+0000", 2024-04-07T17:51:28.071 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.013008564, 2024-04-07T17:51:28.071 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:28.071 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:28.071 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:28.071 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:28.071 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:28.071 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:28.071 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:28.072 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:28.072 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:28.072 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:28.072 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:28.072 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:28.072 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:28.072 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:28.072 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:28.072 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:28.072 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:28.072 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:28.072 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:28.072 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:28.072 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:28.072 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:28.073 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:28.073 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:28.073 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:28.073 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:28.073 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:28.073 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:28.073 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:28.073 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:28.073 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:28.073 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:28.073 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:28.073 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:28.073 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:28.073 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 1133, 2024-04-07T17:51:28.073 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 401354, 2024-04-07T17:51:28.073 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:28.074 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:28.074 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:28.074 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 3064, 2024-04-07T17:51:28.074 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 366365 2024-04-07T17:51:28.074 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.074 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:28.074 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:28.074 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:28.074 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:28.074 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:28.074 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T17:51:28.074 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.074 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 3, 2024-04-07T17:51:28.074 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "1083'3089" 2024-04-07T17:51:28.074 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.075 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:28.075 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:28.075 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:28.075 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T17:51:28.075 INFO:tasks.workunit.client.0.smithi100.stdout: 5 2024-04-07T17:51:28.075 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.075 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 1, 2024-04-07T17:51:28.075 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:28.075 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:28.075 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:28.075 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:28.075 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:28.075 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:28.075 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.333938+0000", 2024-04-07T17:51:28.075 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:24:41.552588+0000", 2024-04-07T17:51:28.075 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:24:41.552588+0000", 2024-04-07T17:51:28.076 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:24:41.553129+0000", 2024-04-07T17:51:28.076 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.333938+0000", 2024-04-07T17:51:28.076 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:28.076 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:28.076 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:28.076 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 527, 2024-04-07T17:51:28.076 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.333938+0000", 2024-04-07T17:51:28.076 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.333938+0000", 2024-04-07T17:51:28.076 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.333938+0000", 2024-04-07T17:51:28.076 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:28.076 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:28.076 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:28.076 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.333938+0000", 2024-04-07T17:51:28.076 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.333938+0000", 2024-04-07T17:51:28.077 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:28.077 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:28.077 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:28.077 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:28.077 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 526, 2024-04-07T17:51:28.077 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:28.077 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:28.077 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:28.077 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:28.077 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:28.077 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:28.077 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:28.077 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:28.077 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.4", 2024-04-07T17:51:28.077 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:28.078 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:28.078 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:28.078 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4983, 2024-04-07T17:51:28.078 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:28.078 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T18:17:15.023877+0000", 2024-04-07T17:51:28.078 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:28.078 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:28.078 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:28.078 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:28.078 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:28.078 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:28.078 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:28.078 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:28.078 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:28.079 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:28.079 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:28.079 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:28.079 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:28.079 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:28.079 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:28.079 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:28.079 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:28.079 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:28.079 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:28.079 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:28.079 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:28.079 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:28.079 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:28.079 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:28.079 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:28.080 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:28.080 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:28.080 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:28.080 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:28.080 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:28.080 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:28.080 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:28.080 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:28.080 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:28.080 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:28.080 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:28.080 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:28.080 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:28.080 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:28.080 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:28.081 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:28.081 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:28.081 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:28.081 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.081 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:28.081 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:28.081 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:28.081 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:28.081 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T17:51:28.081 INFO:tasks.workunit.client.0.smithi100.stdout: 5 2024-04-07T17:51:28.081 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.081 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 1, 2024-04-07T17:51:28.081 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:28.081 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.081 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:28.082 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:28.082 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:28.082 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:28.082 INFO:tasks.workunit.client.0.smithi100.stdout: 4 2024-04-07T17:51:28.082 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.082 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 7, 2024-04-07T17:51:28.082 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:28.082 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:28.082 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T17:51:28.082 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:28.082 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:28.082 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:28.082 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.372971+0000", 2024-04-07T17:51:28.082 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:39.001835+0000", 2024-04-07T17:51:28.082 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:39.001835+0000", 2024-04-07T17:51:28.083 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:39.003607+0000", 2024-04-07T17:51:28.083 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.372971+0000", 2024-04-07T17:51:28.083 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:28.083 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:28.083 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:28.083 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:28.083 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.372971+0000", 2024-04-07T17:51:28.083 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.372971+0000", 2024-04-07T17:51:28.083 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.372971+0000", 2024-04-07T17:51:28.083 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:28.083 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:28.083 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:28.083 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.372971+0000", 2024-04-07T17:51:28.083 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.372971+0000", 2024-04-07T17:51:28.083 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:28.084 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:28.084 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:28.084 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:28.084 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:28.084 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:28.084 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:28.084 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:28.084 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:28.084 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:28.084 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:28.084 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:28.084 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:28.084 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.5", 2024-04-07T17:51:28.084 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:28.084 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:28.085 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:28.085 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4795, 2024-04-07T17:51:28.085 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:28.085 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T21:54:36.255484+0000", 2024-04-07T17:51:28.085 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:28.085 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:28.085 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:28.085 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:28.085 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:28.085 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:28.085 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:28.085 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:28.085 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:28.085 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:28.085 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:28.086 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:28.086 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:28.086 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:28.086 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:28.086 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:28.086 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:28.086 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:28.086 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:28.086 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:28.086 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:28.086 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:28.086 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:28.086 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:28.086 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:28.087 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:28.087 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:28.087 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:28.087 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:28.087 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:28.087 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:28.087 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:28.087 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:28.087 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:28.087 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:28.087 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:28.087 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:28.087 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:28.087 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:28.087 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:28.087 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:28.088 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:28.088 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:28.088 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.088 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:28.088 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:28.088 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:28.088 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:28.088 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:28.088 INFO:tasks.workunit.client.0.smithi100.stdout: 4 2024-04-07T17:51:28.088 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.088 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 7, 2024-04-07T17:51:28.088 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:28.088 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.088 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:28.088 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:28.089 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:28.089 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:28.089 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T17:51:28.089 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.089 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 0, 2024-04-07T17:51:28.089 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:28.089 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:28.089 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 66, 2024-04-07T17:51:28.089 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:28.089 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:28.089 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:28.089 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:51:22.430544+0000", 2024-04-07T17:51:28.089 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:39.003851+0000", 2024-04-07T17:51:28.089 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:39.003851+0000", 2024-04-07T17:51:28.089 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:59:56.571405+0000", 2024-04-07T17:51:28.089 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:51:22.430544+0000", 2024-04-07T17:51:28.090 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:13:28.963534+0000", 2024-04-07T17:51:28.090 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:28.090 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:13:28.963534+0000", 2024-04-07T17:51:28.090 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:28.090 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:51:22.430544+0000", 2024-04-07T17:51:28.090 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:51:22.430544+0000", 2024-04-07T17:51:28.090 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:51:22.430544+0000", 2024-04-07T17:51:28.090 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:28.090 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:28.090 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:13:28.963534+0000", 2024-04-07T17:51:28.090 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:51:22.430544+0000", 2024-04-07T17:51:28.090 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:51:22.430544+0000", 2024-04-07T17:51:28.090 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:28.090 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 3135, 2024-04-07T17:51:28.091 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:28.091 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:28.091 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:28.091 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:28.091 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:28.091 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:28.091 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:28.091 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 3135, 2024-04-07T17:51:28.091 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:28.091 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:28.091 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:28.091 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "4.3", 2024-04-07T17:51:28.091 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:28.091 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:28.091 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:28.092 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 8778, 2024-04-07T17:51:28.092 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:28.092 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T23:06:05.133511+0000", 2024-04-07T17:51:28.092 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.019028691, 2024-04-07T17:51:28.092 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:28.092 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:28.092 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:28.092 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:28.092 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:28.092 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:28.092 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:28.092 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:28.092 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:28.092 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:28.093 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:28.093 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:28.093 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:28.093 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:28.093 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:28.093 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:28.093 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:28.093 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:28.093 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 3, 2024-04-07T17:51:28.093 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 1, 2024-04-07T17:51:28.093 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:28.093 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 1, 2024-04-07T17:51:28.093 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:28.093 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:28.093 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:28.093 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:28.094 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:28.094 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 1, 2024-04-07T17:51:28.094 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:28.094 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:28.094 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:28.094 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:28.094 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:28.094 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:28.094 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:28.094 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 873, 2024-04-07T17:51:28.094 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 391393, 2024-04-07T17:51:28.094 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:28.094 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:28.094 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:28.095 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 3127, 2024-04-07T17:51:28.095 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 378194 2024-04-07T17:51:28.095 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.095 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:28.095 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:28.095 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:28.095 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:28.095 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:28.095 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T17:51:28.095 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.095 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 0, 2024-04-07T17:51:28.095 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "1084'3135" 2024-04-07T17:51:28.095 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.095 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:28.096 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:28.096 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:28.096 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:28.096 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T17:51:28.096 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.096 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 7, 2024-04-07T17:51:28.096 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:28.096 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:28.096 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T17:51:28.096 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:28.096 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:28.096 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:28.096 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.364517+0000", 2024-04-07T17:51:28.096 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:39.000134+0000", 2024-04-07T17:51:28.096 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:39.000134+0000", 2024-04-07T17:51:28.096 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:39.003944+0000", 2024-04-07T17:51:28.097 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.364517+0000", 2024-04-07T17:51:28.097 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:28.097 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:28.097 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:28.097 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:28.097 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.364517+0000", 2024-04-07T17:51:28.097 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.364517+0000", 2024-04-07T17:51:28.097 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.364517+0000", 2024-04-07T17:51:28.097 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:28.097 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:28.097 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:28.097 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.364517+0000", 2024-04-07T17:51:28.097 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.364517+0000", 2024-04-07T17:51:28.097 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:28.097 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:28.098 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:28.098 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:28.098 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:28.098 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:28.098 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:28.098 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:28.098 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:28.098 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:28.098 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:28.098 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:28.098 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:28.098 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.0", 2024-04-07T17:51:28.098 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:28.098 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:28.099 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:28.099 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4796, 2024-04-07T17:51:28.099 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:28.099 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T22:28:12.376172+0000", 2024-04-07T17:51:28.099 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:28.099 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:28.099 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:28.099 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:28.099 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:28.099 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:28.099 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:28.099 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:28.099 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:28.099 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:28.099 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:28.100 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:28.100 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:28.100 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:28.100 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:28.100 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:28.100 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:28.100 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:28.100 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:28.100 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:28.100 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:28.100 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:28.100 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:28.100 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:28.100 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:28.100 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:28.100 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:28.101 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:28.101 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:28.101 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:28.101 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:28.101 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:28.101 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:28.101 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:28.101 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:28.101 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:28.101 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:28.101 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:28.101 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:28.101 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:28.101 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:28.101 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:28.101 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:28.102 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.102 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:28.102 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:28.102 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:28.102 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:28.102 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:28.102 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T17:51:28.102 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.102 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 7, 2024-04-07T17:51:28.102 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:28.102 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.102 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:28.102 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:28.102 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T17:51:28.103 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:28.103 INFO:tasks.workunit.client.0.smithi100.stdout: 3 2024-04-07T17:51:28.103 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.103 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 2, 2024-04-07T17:51:28.103 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:28.103 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:28.103 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:28.103 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:28.103 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:28.103 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:28.103 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.354824+0000", 2024-04-07T17:51:28.103 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:24:13.930517+0000", 2024-04-07T17:51:28.103 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:24:13.930517+0000", 2024-04-07T17:51:28.103 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:24:13.931259+0000", 2024-04-07T17:51:28.103 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.354824+0000", 2024-04-07T17:51:28.103 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:28.104 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:28.104 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:28.104 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 500, 2024-04-07T17:51:28.104 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.354824+0000", 2024-04-07T17:51:28.104 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.354824+0000", 2024-04-07T17:51:28.104 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.354824+0000", 2024-04-07T17:51:28.104 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:28.104 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:28.104 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:28.104 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.354824+0000", 2024-04-07T17:51:28.104 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.354824+0000", 2024-04-07T17:51:28.104 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:28.104 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:28.104 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:28.105 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:28.105 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 499, 2024-04-07T17:51:28.105 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:28.105 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:28.105 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:28.105 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:28.105 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:28.105 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:28.105 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:28.105 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:28.105 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.1", 2024-04-07T17:51:28.105 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:28.105 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:28.105 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:28.105 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4929, 2024-04-07T17:51:28.106 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:28.106 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T01:43:11.885075+0000", 2024-04-07T17:51:28.106 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:28.106 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:28.106 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:28.106 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:28.106 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:28.106 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:28.106 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:28.106 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:28.106 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:28.106 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:28.106 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:28.107 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:28.107 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:28.107 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:28.107 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:28.107 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:28.107 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:28.107 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:28.107 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:28.107 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:28.107 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:28.107 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:28.107 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:28.107 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:28.107 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:28.107 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:28.108 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:28.108 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:28.108 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:28.108 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:28.108 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:28.108 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:28.108 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:28.108 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:28.108 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:28.108 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:28.108 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:28.108 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:28.108 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:28.108 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:28.108 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:28.109 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:28.109 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:28.109 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.109 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:28.109 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:28.109 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:28.109 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T17:51:28.109 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:28.109 INFO:tasks.workunit.client.0.smithi100.stdout: 3 2024-04-07T17:51:28.109 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.109 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 2, 2024-04-07T17:51:28.109 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:28.109 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.109 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:28.109 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:28.109 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:28.110 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:28.110 INFO:tasks.workunit.client.0.smithi100.stdout: 2 2024-04-07T17:51:28.110 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.110 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 0, 2024-04-07T17:51:28.110 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:28.110 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:28.110 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 66, 2024-04-07T17:51:28.110 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:28.110 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:28.110 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:28.110 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:35.982117+0000", 2024-04-07T17:51:28.110 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:23:15.882881+0000", 2024-04-07T17:51:28.110 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:23:15.882881+0000", 2024-04-07T17:51:28.110 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:59:56.569536+0000", 2024-04-07T17:51:28.110 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:35.982117+0000", 2024-04-07T17:51:28.111 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:13:28.963534+0000", 2024-04-07T17:51:28.111 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:28.111 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:13:28.963534+0000", 2024-04-07T17:51:28.111 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 445, 2024-04-07T17:51:28.111 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:35.982117+0000", 2024-04-07T17:51:28.111 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:35.982117+0000", 2024-04-07T17:51:28.111 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:35.982117+0000", 2024-04-07T17:51:28.111 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:28.111 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:28.111 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:13:28.963534+0000", 2024-04-07T17:51:28.111 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:35.982117+0000", 2024-04-07T17:51:28.111 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:35.982117+0000", 2024-04-07T17:51:28.111 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:28.111 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 3107, 2024-04-07T17:51:28.111 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:28.112 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:28.112 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 444, 2024-04-07T17:51:28.112 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:28.112 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:28.112 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:28.112 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:28.112 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 3107, 2024-04-07T17:51:28.112 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:28.112 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:28.112 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:28.112 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "4.6", 2024-04-07T17:51:28.112 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:28.112 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:28.112 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:28.112 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 8717, 2024-04-07T17:51:28.112 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:28.112 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T18:35:28.250440+0000", 2024-04-07T17:51:28.113 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.017233416, 2024-04-07T17:51:28.113 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:28.113 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:28.113 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:28.113 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:28.113 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:28.113 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:28.113 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:28.113 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:28.113 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:28.113 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:28.113 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:28.113 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:28.113 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:28.113 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:28.113 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:28.113 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:28.114 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:28.114 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:28.114 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:28.114 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:28.114 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:28.114 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:28.114 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:28.114 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:28.114 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:28.114 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:28.114 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:28.114 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:28.114 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:28.114 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:28.115 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:28.115 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:28.115 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:28.115 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:28.115 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:28.115 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 858, 2024-04-07T17:51:28.115 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 396543, 2024-04-07T17:51:28.115 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:28.115 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:28.115 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:28.115 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 3070, 2024-04-07T17:51:28.115 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 377559 2024-04-07T17:51:28.115 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.115 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:28.115 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:28.116 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:28.116 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:28.116 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:28.116 INFO:tasks.workunit.client.0.smithi100.stdout: 2 2024-04-07T17:51:28.116 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.116 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 0, 2024-04-07T17:51:28.116 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "1084'3107" 2024-04-07T17:51:28.116 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.116 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:28.116 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:28.116 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:28.116 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:28.116 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T17:51:28.116 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.116 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 7, 2024-04-07T17:51:28.116 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:28.117 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:28.117 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 19, 2024-04-07T17:51:28.117 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:28.117 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:28.117 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:28.117 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.366640+0000", 2024-04-07T17:51:28.117 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:39.000587+0000", 2024-04-07T17:51:28.117 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:39.000587+0000", 2024-04-07T17:51:28.117 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:39.004716+0000", 2024-04-07T17:51:28.117 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.366640+0000", 2024-04-07T17:51:28.117 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:07:26.866003+0000", 2024-04-07T17:51:28.117 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:28.117 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:07:26.866003+0000", 2024-04-07T17:51:28.117 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:28.117 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.366640+0000", 2024-04-07T17:51:28.117 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.366640+0000", 2024-04-07T17:51:28.118 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.366640+0000", 2024-04-07T17:51:28.118 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:28.118 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:28.118 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:07:26.866003+0000", 2024-04-07T17:51:28.118 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.366640+0000", 2024-04-07T17:51:28.118 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.366640+0000", 2024-04-07T17:51:28.118 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:28.118 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 241, 2024-04-07T17:51:28.118 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:28.118 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:28.118 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:28.118 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:28.118 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:28.119 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:28.119 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:28.119 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 241, 2024-04-07T17:51:28.119 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:28.119 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:28.119 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:28.119 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "1.0", 2024-04-07T17:51:28.119 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:28.119 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:28.119 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:28.119 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 5460, 2024-04-07T17:51:28.119 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:28.119 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T21:59:54.730545+0000", 2024-04-07T17:51:28.119 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:28.119 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:28.120 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:28.120 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 590368, 2024-04-07T17:51:28.120 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:28.120 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 1771104, 2024-04-07T17:51:28.120 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:28.120 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:28.120 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:28.120 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:28.120 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:28.120 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:28.120 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:28.120 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:28.120 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:28.120 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:28.120 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:28.121 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:28.121 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:28.121 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 6, 2024-04-07T17:51:28.121 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 2, 2024-04-07T17:51:28.121 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:28.121 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 2, 2024-04-07T17:51:28.121 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:28.121 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:28.121 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:28.121 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:28.121 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:28.121 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:28.121 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:28.121 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 6, 2024-04-07T17:51:28.122 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:28.122 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:28.122 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:28.122 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:28.122 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:28.122 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 1214, 2024-04-07T17:51:28.122 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 2024, 2024-04-07T17:51:28.122 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:28.122 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:28.122 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:28.122 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 399, 2024-04-07T17:51:28.122 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 2416 2024-04-07T17:51:28.122 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.122 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:28.122 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:28.123 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:28.123 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:28.123 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:28.123 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T17:51:28.123 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.123 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 7, 2024-04-07T17:51:28.123 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "2111'241" 2024-04-07T17:51:28.123 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.123 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:28.123 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:28.123 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:28.123 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T17:51:28.123 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T17:51:28.123 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.123 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 5, 2024-04-07T17:51:28.124 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:28.124 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:28.124 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T17:51:28.124 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:28.124 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:28.124 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:28.124 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.370078+0000", 2024-04-07T17:51:28.124 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:38.985921+0000", 2024-04-07T17:51:28.124 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:38.985921+0000", 2024-04-07T17:51:28.124 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:38.986647+0000", 2024-04-07T17:51:28.124 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.370078+0000", 2024-04-07T17:51:28.124 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:28.124 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:28.124 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:28.124 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:28.124 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.370078+0000", 2024-04-07T17:51:28.124 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.370078+0000", 2024-04-07T17:51:28.125 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.370078+0000", 2024-04-07T17:51:28.125 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:28.125 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:28.125 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:28.125 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.370078+0000", 2024-04-07T17:51:28.125 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.370078+0000", 2024-04-07T17:51:28.125 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:28.125 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:28.125 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:28.125 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:28.125 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:28.125 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:28.125 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:28.125 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:28.125 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:28.126 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:28.126 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:28.126 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:28.126 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:28.126 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.3", 2024-04-07T17:51:28.126 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:28.126 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:28.126 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:28.126 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4855, 2024-04-07T17:51:28.126 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:28.126 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T02:44:42.906581+0000", 2024-04-07T17:51:28.126 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:28.126 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:28.126 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:28.127 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:28.127 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:28.127 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:28.127 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:28.127 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:28.127 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:28.127 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:28.127 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:28.127 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:28.127 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:28.127 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:28.127 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:28.127 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:28.127 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:28.127 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:28.127 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:28.127 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:28.128 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:28.128 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:28.128 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:28.128 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:28.128 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:28.128 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:28.128 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:28.128 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:28.128 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:28.128 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:28.128 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:28.128 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:28.128 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:28.128 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:28.128 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:28.128 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:28.129 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:28.129 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:28.129 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:28.129 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:28.129 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:28.129 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:28.129 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:28.129 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.129 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:28.129 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:28.129 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:28.129 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:28.129 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T17:51:28.129 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T17:51:28.129 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.129 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 5, 2024-04-07T17:51:28.129 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:28.130 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.130 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:28.130 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:28.130 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:28.130 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:28.130 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T17:51:28.130 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.130 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 3, 2024-04-07T17:51:28.130 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:28.130 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:28.130 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:28.130 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:28.130 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:28.130 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:28.131 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:35.982147+0000", 2024-04-07T17:51:28.131 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:08.977825+0000", 2024-04-07T17:51:28.131 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:08.977825+0000", 2024-04-07T17:51:28.131 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:08.978231+0000", 2024-04-07T17:51:28.131 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:35.982147+0000", 2024-04-07T17:51:28.131 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:28.131 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:28.131 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:28.131 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 554, 2024-04-07T17:51:28.131 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:35.982147+0000", 2024-04-07T17:51:28.131 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:35.982147+0000", 2024-04-07T17:51:28.131 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:35.982147+0000", 2024-04-07T17:51:28.131 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:28.131 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:28.131 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:28.131 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:35.982147+0000", 2024-04-07T17:51:28.132 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:35.982147+0000", 2024-04-07T17:51:28.132 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:28.132 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:28.132 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:28.132 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:28.132 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 553, 2024-04-07T17:51:28.132 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:28.132 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:28.132 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:28.132 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:28.132 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:28.132 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:28.132 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:28.132 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:28.132 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.2", 2024-04-07T17:51:28.132 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:28.133 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:28.133 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:28.133 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4937, 2024-04-07T17:51:28.133 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:28.133 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T01:34:08.840626+0000", 2024-04-07T17:51:28.133 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:28.133 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:28.133 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:28.133 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:28.133 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:28.133 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:28.133 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:28.133 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:28.133 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:28.133 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:28.133 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:28.133 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:28.134 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:28.134 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:28.134 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:28.134 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:28.134 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:28.134 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:28.134 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:28.134 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:28.134 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:28.134 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:28.134 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:28.134 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:28.134 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:28.134 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:28.135 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:28.135 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:28.135 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:28.135 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:28.135 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:28.135 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:28.135 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:28.135 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:28.135 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:28.135 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:28.135 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:28.135 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:28.135 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:28.135 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:28.135 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:28.136 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:28.136 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:28.136 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.136 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:28.136 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:28.136 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:28.136 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:28.136 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:28.136 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T17:51:28.136 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.136 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 3, 2024-04-07T17:51:28.136 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:28.136 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.136 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:28.136 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:28.137 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:28.137 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:28.137 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T17:51:28.137 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.137 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 6, 2024-04-07T17:51:28.137 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:28.137 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:28.137 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 66, 2024-04-07T17:51:28.137 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:28.137 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:28.137 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:28.137 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.360030+0000", 2024-04-07T17:51:28.137 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:39.000185+0000", 2024-04-07T17:51:28.137 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:39.000185+0000", 2024-04-07T17:51:28.137 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:59:56.544598+0000", 2024-04-07T17:51:28.137 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.360030+0000", 2024-04-07T17:51:28.138 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:13:28.963534+0000", 2024-04-07T17:51:28.138 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:28.138 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:13:28.963534+0000", 2024-04-07T17:51:28.138 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:28.138 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.360030+0000", 2024-04-07T17:51:28.138 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.360030+0000", 2024-04-07T17:51:28.138 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.360030+0000", 2024-04-07T17:51:28.138 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:28.138 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:28.138 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:13:28.963534+0000", 2024-04-07T17:51:28.138 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.360030+0000", 2024-04-07T17:51:28.138 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.360030+0000", 2024-04-07T17:51:28.138 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:28.138 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 3124, 2024-04-07T17:51:28.138 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:28.139 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:28.139 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:28.139 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:28.139 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:28.139 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:28.139 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:28.139 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 3124, 2024-04-07T17:51:28.139 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:28.139 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:28.139 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:28.139 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "4.5", 2024-04-07T17:51:28.139 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:28.139 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:28.139 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:28.139 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 8600, 2024-04-07T17:51:28.139 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:28.140 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T01:53:54.315291+0000", 2024-04-07T17:51:28.140 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.008144472, 2024-04-07T17:51:28.140 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:28.140 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:28.140 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:28.140 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:28.140 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:28.140 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:28.140 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:28.140 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:28.140 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:28.140 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:28.140 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:28.140 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:28.140 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:28.140 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:28.141 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:28.141 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:28.141 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:28.141 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:28.141 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:28.141 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:28.141 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:28.141 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:28.141 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:28.141 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:28.141 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:28.141 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:28.141 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:28.141 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:28.141 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:28.142 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:28.142 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:28.142 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:28.142 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:28.142 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:28.142 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:28.142 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 979, 2024-04-07T17:51:28.142 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 424424, 2024-04-07T17:51:28.142 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:28.142 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:28.142 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:28.142 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 3125, 2024-04-07T17:51:28.142 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 420851 2024-04-07T17:51:28.142 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.142 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:28.143 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:28.143 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:28.143 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:28.143 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:28.143 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T17:51:28.143 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.143 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 6, 2024-04-07T17:51:28.143 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "1083'3124" 2024-04-07T17:51:28.143 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.143 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:28.143 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:28.143 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T17:51:28.143 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:28.143 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T17:51:28.143 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.144 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 2, 2024-04-07T17:51:28.144 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:28.144 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:28.144 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T17:51:28.144 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:28.144 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:28.144 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:28.144 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.355116+0000", 2024-04-07T17:51:28.144 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:23:43.929900+0000", 2024-04-07T17:51:28.144 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:23:43.929900+0000", 2024-04-07T17:51:28.144 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:23:43.930960+0000", 2024-04-07T17:51:28.144 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.355116+0000", 2024-04-07T17:51:28.144 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:28.144 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:28.144 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:28.144 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 471, 2024-04-07T17:51:28.145 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.355116+0000", 2024-04-07T17:51:28.145 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.355116+0000", 2024-04-07T17:51:28.145 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.355116+0000", 2024-04-07T17:51:28.145 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:28.145 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:28.145 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:28.145 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.355116+0000", 2024-04-07T17:51:28.145 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.355116+0000", 2024-04-07T17:51:28.145 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:28.145 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:28.145 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:28.145 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:28.145 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 470, 2024-04-07T17:51:28.145 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:28.146 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:28.146 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:28.146 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:28.146 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:28.146 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:28.146 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:28.146 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:28.146 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.1", 2024-04-07T17:51:28.146 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:28.146 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:28.146 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:28.146 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4977, 2024-04-07T17:51:28.146 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:28.146 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T20:05:04.268255+0000", 2024-04-07T17:51:28.147 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:28.147 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:28.147 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:28.147 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:28.147 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:28.147 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:28.147 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:28.147 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:28.147 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:28.147 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:28.147 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:28.147 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:28.147 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:28.147 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:28.147 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:28.148 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:28.148 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:28.148 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:28.148 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:28.148 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:28.148 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:28.148 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:28.148 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:28.148 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:28.148 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:28.148 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:28.148 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:28.148 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:28.148 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:28.148 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:28.149 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:28.149 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:28.149 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:28.149 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:28.149 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:28.149 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:28.149 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:28.149 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:28.149 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:28.149 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:28.149 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:28.149 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:28.149 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:28.149 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.149 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:28.150 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:28.150 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:28.150 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T17:51:28.150 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:28.150 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T17:51:28.150 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.150 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 2, 2024-04-07T17:51:28.150 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:28.150 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.150 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:28.150 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:28.150 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:28.150 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T17:51:28.150 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T17:51:28.150 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.151 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 1, 2024-04-07T17:51:28.151 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:28.151 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:28.151 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:28.151 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:28.151 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:28.151 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:28.151 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.331050+0000", 2024-04-07T17:51:28.151 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:08.961818+0000", 2024-04-07T17:51:28.151 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:08.961818+0000", 2024-04-07T17:51:28.151 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:08.962269+0000", 2024-04-07T17:51:28.151 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.331050+0000", 2024-04-07T17:51:28.151 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:28.151 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:28.151 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:28.152 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 554, 2024-04-07T17:51:28.152 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.331050+0000", 2024-04-07T17:51:28.152 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.331050+0000", 2024-04-07T17:51:28.152 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.331050+0000", 2024-04-07T17:51:28.152 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:28.152 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:28.152 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:28.152 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.331050+0000", 2024-04-07T17:51:28.152 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.331050+0000", 2024-04-07T17:51:28.152 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:28.152 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:28.152 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:28.152 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:28.152 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 553, 2024-04-07T17:51:28.152 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:28.153 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:28.153 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:28.153 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:28.153 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:28.153 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:28.153 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:28.153 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:28.153 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.0", 2024-04-07T17:51:28.153 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:28.153 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:28.153 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:28.153 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4985, 2024-04-07T17:51:28.153 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:28.153 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T01:33:47.318091+0000", 2024-04-07T17:51:28.153 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:28.154 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:28.154 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:28.154 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:28.154 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:28.154 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:28.154 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:28.154 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:28.154 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:28.154 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:28.154 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:28.154 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:28.154 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:28.154 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:28.154 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:28.154 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:28.155 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:28.155 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:28.155 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:28.155 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:28.155 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:28.155 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:28.155 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:28.155 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:28.155 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:28.155 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:28.155 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:28.155 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:28.155 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:28.155 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:28.155 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:28.156 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:28.156 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:28.156 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:28.156 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:28.156 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:28.156 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:28.156 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:28.156 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:28.156 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:28.156 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:28.156 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:28.156 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:28.156 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.156 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:28.156 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:28.157 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:28.157 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:28.157 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T17:51:28.157 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T17:51:28.157 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.157 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 1, 2024-04-07T17:51:28.157 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:28.157 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.157 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:28.157 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:28.157 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:28.157 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:28.157 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T17:51:28.157 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.157 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 1, 2024-04-07T17:51:28.158 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:28.158 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:28.158 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 66, 2024-04-07T17:51:28.158 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:28.158 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:28.158 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:28.158 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.332996+0000", 2024-04-07T17:51:28.158 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:24:41.570251+0000", 2024-04-07T17:51:28.158 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:24:41.570251+0000", 2024-04-07T17:51:28.158 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:59:56.540592+0000", 2024-04-07T17:51:28.158 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.332996+0000", 2024-04-07T17:51:28.158 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:13:28.963534+0000", 2024-04-07T17:51:28.158 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:28.158 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:13:28.963534+0000", 2024-04-07T17:51:28.158 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 527, 2024-04-07T17:51:28.159 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.332996+0000", 2024-04-07T17:51:28.159 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.332996+0000", 2024-04-07T17:51:28.159 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.332996+0000", 2024-04-07T17:51:28.159 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:28.159 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:28.159 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:13:28.963534+0000", 2024-04-07T17:51:28.159 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.332996+0000", 2024-04-07T17:51:28.159 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.332996+0000", 2024-04-07T17:51:28.159 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:28.159 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 2973, 2024-04-07T17:51:28.159 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:28.159 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:28.159 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 526, 2024-04-07T17:51:28.159 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:28.159 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:28.160 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:28.160 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:28.160 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 2973, 2024-04-07T17:51:28.160 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:28.160 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:28.160 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:28.160 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "4.7", 2024-04-07T17:51:28.160 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:28.160 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:28.160 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:28.160 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 8716, 2024-04-07T17:51:28.160 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:28.160 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T19:36:01.668815+0000", 2024-04-07T17:51:28.160 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.014750449, 2024-04-07T17:51:28.161 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:28.161 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:28.161 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:28.161 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:28.161 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:28.161 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:28.161 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:28.161 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:28.161 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:28.161 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:28.161 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:28.161 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:28.161 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:28.161 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:28.161 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:28.161 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:28.162 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:28.162 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:28.162 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:28.162 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:28.162 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:28.162 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:28.162 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:28.162 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:28.162 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:28.162 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:28.162 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:28.162 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:28.162 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:28.162 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:28.163 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:28.163 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:28.163 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:28.163 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:28.163 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:28.163 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 1318, 2024-04-07T17:51:28.163 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 368358, 2024-04-07T17:51:28.163 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:28.163 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:28.163 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:28.163 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 2957, 2024-04-07T17:51:28.163 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 380954 2024-04-07T17:51:28.163 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.163 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:28.163 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:28.164 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:28.164 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:28.164 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:28.164 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T17:51:28.164 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.164 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 1, 2024-04-07T17:51:28.164 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "1084'2973" 2024-04-07T17:51:28.164 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.164 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:28.164 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:28.164 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:28.164 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:28.164 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T17:51:28.164 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.164 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 4, 2024-04-07T17:51:28.165 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:28.165 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:28.165 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:28.165 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:28.165 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:28.165 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:28.165 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:35.979039+0000", 2024-04-07T17:51:28.165 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:08.952350+0000", 2024-04-07T17:51:28.165 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:08.952350+0000", 2024-04-07T17:51:28.165 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:08.952832+0000", 2024-04-07T17:51:28.165 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:35.979039+0000", 2024-04-07T17:51:28.165 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:28.165 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:28.165 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:28.165 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 554, 2024-04-07T17:51:28.166 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:35.979039+0000", 2024-04-07T17:51:28.166 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:35.979039+0000", 2024-04-07T17:51:28.166 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:35.979039+0000", 2024-04-07T17:51:28.166 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:28.166 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:28.166 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:28.166 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:35.979039+0000", 2024-04-07T17:51:28.166 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:35.979039+0000", 2024-04-07T17:51:28.166 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:28.166 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:28.166 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:28.166 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:28.166 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 553, 2024-04-07T17:51:28.166 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:28.167 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:28.167 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:28.167 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:28.167 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:28.167 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:28.167 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:28.167 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:28.167 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.3", 2024-04-07T17:51:28.167 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:28.167 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:28.167 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:28.167 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4919, 2024-04-07T17:51:28.167 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:28.167 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T22:57:40.507778+0000", 2024-04-07T17:51:28.167 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:28.167 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:28.168 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:28.168 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:28.168 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:28.168 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:28.168 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:28.168 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:28.168 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:28.168 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:28.168 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:28.168 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:28.168 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:28.168 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:28.168 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:28.168 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:28.168 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:28.169 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:28.169 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:28.169 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:28.169 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:28.169 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:28.169 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:28.169 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:28.169 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:28.169 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:28.169 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:28.169 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:28.169 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:28.169 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:28.169 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:28.169 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:28.169 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:28.170 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:28.170 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:28.170 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:28.170 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:28.170 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:28.170 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:28.170 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:28.170 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:28.170 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:28.170 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:28.170 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.170 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:28.170 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:28.170 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:28.170 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:28.170 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:28.171 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T17:51:28.171 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.171 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 4, 2024-04-07T17:51:28.171 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:28.171 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.171 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:28.171 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:28.171 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:28.171 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:28.171 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T17:51:28.171 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.171 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 5, 2024-04-07T17:51:28.171 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:28.171 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:28.171 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T17:51:28.172 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:28.172 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:28.172 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:28.172 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.369727+0000", 2024-04-07T17:51:28.172 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:08.972663+0000", 2024-04-07T17:51:28.172 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:08.972663+0000", 2024-04-07T17:51:28.172 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:08.973250+0000", 2024-04-07T17:51:28.172 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.369727+0000", 2024-04-07T17:51:28.172 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:28.172 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:28.172 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:28.172 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 554, 2024-04-07T17:51:28.172 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.369727+0000", 2024-04-07T17:51:28.172 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.369727+0000", 2024-04-07T17:51:28.172 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.369727+0000", 2024-04-07T17:51:28.172 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:28.173 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:28.173 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:28.173 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.369727+0000", 2024-04-07T17:51:28.173 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.369727+0000", 2024-04-07T17:51:28.173 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:28.173 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:28.173 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:28.173 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:28.173 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 553, 2024-04-07T17:51:28.173 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:28.173 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:28.173 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:28.173 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:28.173 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:28.173 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:28.174 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:28.174 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:28.174 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.2", 2024-04-07T17:51:28.174 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:28.174 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:28.174 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:28.174 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4868, 2024-04-07T17:51:28.174 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:28.174 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T22:42:44.603307+0000", 2024-04-07T17:51:28.174 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:28.174 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:28.174 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:28.174 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:28.174 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:28.175 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:28.175 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:28.175 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:28.175 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:28.175 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:28.175 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:28.175 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:28.175 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:28.175 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:28.175 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:28.175 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:28.175 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:28.175 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:28.175 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:28.175 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:28.176 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:28.176 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:28.176 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:28.176 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:28.176 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:28.176 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:28.176 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:28.176 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:28.176 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:28.176 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:28.176 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:28.176 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:28.176 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:28.176 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:28.176 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:28.176 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:28.177 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:28.177 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:28.177 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:28.177 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:28.177 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:28.177 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:28.177 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:28.177 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.177 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:28.177 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:28.177 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:28.177 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:28.177 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:28.177 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T17:51:28.177 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.178 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 5, 2024-04-07T17:51:28.178 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:28.178 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.178 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:28.178 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:28.178 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:28.178 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T17:51:28.178 INFO:tasks.workunit.client.0.smithi100.stdout: 3 2024-04-07T17:51:28.178 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.178 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 1, 2024-04-07T17:51:28.178 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:28.178 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:28.178 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 66, 2024-04-07T17:51:28.178 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:28.178 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:28.179 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:28.179 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:51:24.825515+0000", 2024-04-07T17:51:28.179 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:23:43.924443+0000", 2024-04-07T17:51:28.179 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:23:43.924443+0000", 2024-04-07T17:51:28.179 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:59:56.538443+0000", 2024-04-07T17:51:28.179 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:51:24.825515+0000", 2024-04-07T17:51:28.179 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:13:28.963534+0000", 2024-04-07T17:51:28.179 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:28.179 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:13:28.963534+0000", 2024-04-07T17:51:28.179 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 471, 2024-04-07T17:51:28.179 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:51:24.825515+0000", 2024-04-07T17:51:28.179 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:51:24.825515+0000", 2024-04-07T17:51:28.179 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:51:24.825515+0000", 2024-04-07T17:51:28.179 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:28.179 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:28.179 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:13:28.963534+0000", 2024-04-07T17:51:28.180 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:51:24.825515+0000", 2024-04-07T17:51:28.180 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:51:24.825515+0000", 2024-04-07T17:51:28.180 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:28.180 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 2976, 2024-04-07T17:51:28.180 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:28.180 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:28.180 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 470, 2024-04-07T17:51:28.180 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:28.180 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:28.180 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:28.180 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:28.180 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 2976, 2024-04-07T17:51:28.180 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:28.180 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:28.181 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:28.181 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "4.4", 2024-04-07T17:51:28.181 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:28.181 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:28.181 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:28.181 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 8722, 2024-04-07T17:51:28.181 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:28.181 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T16:43:02.513339+0000", 2024-04-07T17:51:28.181 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.010915499, 2024-04-07T17:51:28.181 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:28.181 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:28.181 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:28.181 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:28.181 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:28.181 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:28.181 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:28.182 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:28.182 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:28.182 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:28.182 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:28.182 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:28.182 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:28.182 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:28.182 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:28.182 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:28.182 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:28.182 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:28.182 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 3, 2024-04-07T17:51:28.182 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 1, 2024-04-07T17:51:28.182 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:28.182 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 1, 2024-04-07T17:51:28.183 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:28.183 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:28.183 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:28.183 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:28.183 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:28.183 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 1, 2024-04-07T17:51:28.183 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:28.183 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:28.183 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:28.183 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:28.183 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:28.183 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:28.183 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:28.183 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 1196, 2024-04-07T17:51:28.183 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 375705, 2024-04-07T17:51:28.183 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:28.184 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:28.184 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:28.184 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 2976, 2024-04-07T17:51:28.184 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 350581 2024-04-07T17:51:28.184 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.184 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:28.184 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:28.184 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:28.184 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:28.184 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T17:51:28.184 INFO:tasks.workunit.client.0.smithi100.stdout: 3 2024-04-07T17:51:28.184 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.184 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 1, 2024-04-07T17:51:28.184 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "1084'2976" 2024-04-07T17:51:28.184 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.184 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:28.185 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:28.185 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:28.185 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:28.185 INFO:tasks.workunit.client.0.smithi100.stdout: 2 2024-04-07T17:51:28.185 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.185 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 5, 2024-04-07T17:51:28.185 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:28.185 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:28.185 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:28.185 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:28.185 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:28.185 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:28.185 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.364565+0000", 2024-04-07T17:51:28.185 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:24:41.554485+0000", 2024-04-07T17:51:28.185 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:24:41.554485+0000", 2024-04-07T17:51:28.186 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:24:41.557859+0000", 2024-04-07T17:51:28.186 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.364565+0000", 2024-04-07T17:51:28.186 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:28.186 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:28.186 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:28.186 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 527, 2024-04-07T17:51:28.186 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.364565+0000", 2024-04-07T17:51:28.186 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.364565+0000", 2024-04-07T17:51:28.186 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.364565+0000", 2024-04-07T17:51:28.186 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:28.186 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:28.186 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:28.186 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.364565+0000", 2024-04-07T17:51:28.187 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.364565+0000", 2024-04-07T17:51:28.187 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:28.187 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:28.187 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:28.187 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:28.187 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 526, 2024-04-07T17:51:28.187 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:28.187 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:28.187 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:28.187 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:28.187 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:28.187 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:28.187 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:28.187 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:28.188 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.5", 2024-04-07T17:51:28.188 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:28.188 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:28.188 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:28.188 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4886, 2024-04-07T17:51:28.188 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:28.188 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T18:53:57.243053+0000", 2024-04-07T17:51:28.188 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:28.188 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:28.188 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:28.188 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:28.188 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:28.188 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:28.188 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:28.188 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:28.188 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:28.189 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:28.189 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:28.189 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:28.189 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:28.189 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:28.189 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:28.189 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:28.189 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:28.189 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:28.189 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:28.189 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:28.189 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:28.189 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:28.189 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:28.189 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:28.189 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:28.189 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:28.190 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:28.190 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:28.190 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:28.190 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:28.190 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:28.190 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:28.190 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:28.190 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:28.190 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:28.190 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:28.190 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:28.190 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:28.190 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:28.190 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:28.191 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:28.191 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:28.191 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:28.191 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.191 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:28.191 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:28.191 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:28.191 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:28.191 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:28.191 INFO:tasks.workunit.client.0.smithi100.stdout: 2 2024-04-07T17:51:28.191 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.191 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 5, 2024-04-07T17:51:28.191 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:28.191 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.191 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:28.191 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:28.192 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:28.192 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:28.192 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T17:51:28.192 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.192 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 1, 2024-04-07T17:51:28.192 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:28.192 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:28.192 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T17:51:28.192 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:28.192 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:28.192 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:28.192 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.333650+0000", 2024-04-07T17:51:28.192 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:38.999666+0000", 2024-04-07T17:51:28.192 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:38.999666+0000", 2024-04-07T17:51:28.192 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:39.000176+0000", 2024-04-07T17:51:28.192 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.333650+0000", 2024-04-07T17:51:28.192 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:28.193 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:28.193 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:28.193 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:28.193 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.333650+0000", 2024-04-07T17:51:28.193 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.333650+0000", 2024-04-07T17:51:28.193 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.333650+0000", 2024-04-07T17:51:28.193 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:28.193 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:28.193 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:28.193 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.333650+0000", 2024-04-07T17:51:28.193 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.333650+0000", 2024-04-07T17:51:28.193 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:28.193 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:28.193 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:28.193 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:28.194 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:28.194 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:28.194 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:28.194 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:28.194 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:28.194 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:28.194 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:28.194 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:28.194 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:28.194 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.4", 2024-04-07T17:51:28.194 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:28.194 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:28.194 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:28.194 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4990, 2024-04-07T17:51:28.194 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:28.195 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T19:36:37.701087+0000", 2024-04-07T17:51:28.195 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:28.195 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:28.195 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:28.195 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:28.195 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:28.195 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:28.195 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:28.195 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:28.195 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:28.195 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:28.195 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:28.195 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:28.195 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:28.195 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:28.195 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:28.196 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:28.196 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:28.196 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:28.196 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:28.196 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:28.196 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:28.196 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:28.196 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:28.196 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:28.196 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:28.196 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:28.196 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:28.196 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:28.196 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:28.196 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:28.196 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:28.197 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:28.197 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:28.197 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:28.197 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:28.197 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:28.197 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:28.197 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:28.197 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:28.197 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:28.197 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:28.197 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:28.197 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:28.197 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.197 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:28.197 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:28.197 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:28.198 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:28.198 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:28.198 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T17:51:28.198 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.198 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 1, 2024-04-07T17:51:28.198 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:28.198 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.198 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:28.198 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:28.198 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:28.198 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:28.198 INFO:tasks.workunit.client.0.smithi100.stdout: 4 2024-04-07T17:51:28.198 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.198 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 1, 2024-04-07T17:51:28.198 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:28.199 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:28.199 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 66, 2024-04-07T17:51:28.199 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:28.199 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:28.199 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:28.199 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.335612+0000", 2024-04-07T17:51:28.199 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:24:41.537435+0000", 2024-04-07T17:51:28.199 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:24:41.537435+0000", 2024-04-07T17:51:28.199 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:59:56.544979+0000", 2024-04-07T17:51:28.199 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.335612+0000", 2024-04-07T17:51:28.199 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:13:28.963534+0000", 2024-04-07T17:51:28.199 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:28.199 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:13:28.963534+0000", 2024-04-07T17:51:28.199 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 527, 2024-04-07T17:51:28.199 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.335612+0000", 2024-04-07T17:51:28.200 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.335612+0000", 2024-04-07T17:51:28.200 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.335612+0000", 2024-04-07T17:51:28.200 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:28.200 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:28.200 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:13:28.963534+0000", 2024-04-07T17:51:28.200 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.335612+0000", 2024-04-07T17:51:28.200 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.335612+0000", 2024-04-07T17:51:28.200 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:28.200 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 2963, 2024-04-07T17:51:28.200 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:28.200 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:28.200 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 526, 2024-04-07T17:51:28.200 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:28.200 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:28.200 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:28.201 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:28.201 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 2963, 2024-04-07T17:51:28.201 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:28.201 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:28.201 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:28.201 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "4.2", 2024-04-07T17:51:28.201 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:28.201 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:28.201 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:28.201 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 8984, 2024-04-07T17:51:28.201 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:28.201 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T23:06:36.963212+0000", 2024-04-07T17:51:28.201 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.015938325, 2024-04-07T17:51:28.201 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:28.201 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:28.202 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 19, 2024-04-07T17:51:28.202 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:28.202 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:28.202 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:28.202 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:28.202 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:28.202 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:28.202 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:28.202 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:28.202 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:28.202 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:28.202 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:28.202 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:28.202 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:28.202 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:28.203 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:28.203 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 3, 2024-04-07T17:51:28.203 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 1, 2024-04-07T17:51:28.203 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:28.203 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 1, 2024-04-07T17:51:28.203 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:28.203 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:28.203 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:28.203 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:28.203 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:28.203 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:28.203 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:28.203 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:28.203 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:28.203 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:28.204 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:28.204 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:28.204 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:28.204 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 1654, 2024-04-07T17:51:28.204 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 387082, 2024-04-07T17:51:28.204 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:28.204 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:28.204 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:28.204 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 2937, 2024-04-07T17:51:28.204 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 349256 2024-04-07T17:51:28.204 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.204 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:28.204 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:28.204 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:28.204 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:28.204 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:28.205 INFO:tasks.workunit.client.0.smithi100.stdout: 4 2024-04-07T17:51:28.205 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.205 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 1, 2024-04-07T17:51:28.205 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "1083'2963" 2024-04-07T17:51:28.205 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.205 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:28.205 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:28.205 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:28.205 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:28.205 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T17:51:28.205 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.205 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 7, 2024-04-07T17:51:28.205 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:28.205 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:28.205 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:28.206 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:28.206 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:28.206 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:28.206 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.372374+0000", 2024-04-07T17:51:28.206 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:38.968892+0000", 2024-04-07T17:51:28.206 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:38.968892+0000", 2024-04-07T17:51:28.206 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:38.970807+0000", 2024-04-07T17:51:28.206 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.372374+0000", 2024-04-07T17:51:28.206 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:28.206 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:28.206 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:28.206 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:28.206 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.372374+0000", 2024-04-07T17:51:28.206 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.372374+0000", 2024-04-07T17:51:28.207 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.372374+0000", 2024-04-07T17:51:28.207 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:28.207 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:28.207 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:28.207 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.372374+0000", 2024-04-07T17:51:28.207 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.372374+0000", 2024-04-07T17:51:28.207 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:28.207 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:28.207 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:28.207 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:28.207 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:28.207 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:28.207 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:28.207 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:28.207 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:28.207 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:28.208 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:28.208 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:28.208 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:28.208 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.e", 2024-04-07T17:51:28.208 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:28.208 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:28.208 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:28.208 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4791, 2024-04-07T17:51:28.208 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:28.208 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T21:17:23.974869+0000", 2024-04-07T17:51:28.208 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:28.208 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:28.208 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:28.208 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:28.208 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:28.208 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:28.209 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:28.209 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:28.209 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:28.209 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:28.209 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:28.209 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:28.209 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:28.209 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:28.209 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:28.209 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:28.209 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:28.209 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:28.209 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:28.209 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:28.209 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:28.209 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:28.210 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:28.210 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:28.210 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:28.210 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:28.210 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:28.210 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:28.210 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:28.210 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:28.210 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:28.210 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:28.210 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:28.210 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:28.210 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:28.210 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:28.210 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:28.211 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:28.211 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:28.211 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:28.211 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:28.211 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:28.211 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:28.211 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.211 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:28.211 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:28.211 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:28.211 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:28.211 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:28.211 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T17:51:28.211 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.211 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 7, 2024-04-07T17:51:28.212 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:28.212 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.212 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:28.212 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:28.212 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:28.212 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:28.212 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T17:51:28.212 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.212 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 4, 2024-04-07T17:51:28.212 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:28.212 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:28.212 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T17:51:28.212 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:28.212 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:28.212 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:28.212 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:35.981613+0000", 2024-04-07T17:51:28.213 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:38.967913+0000", 2024-04-07T17:51:28.213 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:38.967913+0000", 2024-04-07T17:51:28.213 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:38.968805+0000", 2024-04-07T17:51:28.213 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:35.981613+0000", 2024-04-07T17:51:28.213 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:28.213 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:28.213 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:28.213 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:28.213 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:35.981613+0000", 2024-04-07T17:51:28.213 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:35.981613+0000", 2024-04-07T17:51:28.213 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:35.981613+0000", 2024-04-07T17:51:28.213 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:28.213 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:28.213 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:28.213 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:35.981613+0000", 2024-04-07T17:51:28.213 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:35.981613+0000", 2024-04-07T17:51:28.214 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:28.214 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 1, 2024-04-07T17:51:28.214 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:28.214 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:28.214 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:28.214 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:28.214 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:28.214 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:28.214 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:28.214 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 1, 2024-04-07T17:51:28.214 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:28.214 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:28.214 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:28.214 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.f", 2024-04-07T17:51:28.215 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:28.215 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:28.215 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:28.215 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4919, 2024-04-07T17:51:28.215 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:28.215 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T19:43:23.875589+0000", 2024-04-07T17:51:28.215 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:28.215 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:28.215 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:28.215 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 22, 2024-04-07T17:51:28.215 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:28.215 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:28.215 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:28.215 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:28.215 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:28.216 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:28.216 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:28.216 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:28.216 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:28.216 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:28.216 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:28.216 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:28.216 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:28.216 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:28.216 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:28.216 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 3, 2024-04-07T17:51:28.216 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 1, 2024-04-07T17:51:28.216 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:28.216 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 1, 2024-04-07T17:51:28.216 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:28.217 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:28.217 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:28.217 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:28.217 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:28.217 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:28.217 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:28.217 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:28.217 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:28.217 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:28.217 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:28.217 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:28.217 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:28.217 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 1, 2024-04-07T17:51:28.217 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 1, 2024-04-07T17:51:28.217 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:28.218 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:28.218 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:28.218 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 1, 2024-04-07T17:51:28.218 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 1 2024-04-07T17:51:28.218 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.218 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:28.218 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:28.218 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:28.218 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:28.218 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:28.218 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T17:51:28.218 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.218 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 4, 2024-04-07T17:51:28.218 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "61'1" 2024-04-07T17:51:28.219 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.219 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:28.219 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:28.219 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:28.219 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:28.219 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T17:51:28.219 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.219 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 7, 2024-04-07T17:51:28.219 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:28.219 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:28.219 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:28.219 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:28.219 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:28.219 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:28.219 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.358184+0000", 2024-04-07T17:51:28.219 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:38.987996+0000", 2024-04-07T17:51:28.220 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:38.987996+0000", 2024-04-07T17:51:28.220 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:38.989241+0000", 2024-04-07T17:51:28.220 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.358184+0000", 2024-04-07T17:51:28.220 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:28.220 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:28.220 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:28.220 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:28.220 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.358184+0000", 2024-04-07T17:51:28.220 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.358184+0000", 2024-04-07T17:51:28.220 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.358184+0000", 2024-04-07T17:51:28.220 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:28.220 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:28.220 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:28.220 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.358184+0000", 2024-04-07T17:51:28.220 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.358184+0000", 2024-04-07T17:51:28.221 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:28.221 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:28.221 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:28.221 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:28.221 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:28.221 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:28.221 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:28.221 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:28.221 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:28.221 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:28.221 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:28.221 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:28.221 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:28.221 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.11", 2024-04-07T17:51:28.221 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:28.221 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:28.222 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:28.222 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4801, 2024-04-07T17:51:28.222 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:28.222 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T16:46:00.763122+0000", 2024-04-07T17:51:28.222 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:28.222 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:28.222 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:28.222 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:28.222 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:28.222 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:28.222 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:28.222 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:28.222 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:28.222 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:28.223 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:28.223 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:28.223 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:28.223 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:28.223 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:28.223 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:28.223 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:28.223 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:28.223 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:28.223 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:28.223 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:28.223 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:28.223 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:28.223 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:28.223 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:28.223 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:28.223 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:28.224 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:28.224 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:28.224 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:28.224 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:28.224 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:28.224 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:28.224 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:28.224 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:28.224 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:28.224 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:28.224 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:28.224 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:28.224 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:28.224 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:28.224 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:28.224 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:28.224 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.225 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:28.225 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:28.225 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:28.225 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:28.225 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:28.225 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T17:51:28.225 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.225 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 7, 2024-04-07T17:51:28.225 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:28.225 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.225 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:28.225 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:28.225 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T17:51:28.225 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:28.225 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T17:51:28.225 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.226 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 2, 2024-04-07T17:51:28.226 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:28.226 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:28.226 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T17:51:28.226 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:28.226 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:28.226 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:28.226 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.354070+0000", 2024-04-07T17:51:28.226 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:23:15.875351+0000", 2024-04-07T17:51:28.226 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:23:15.875351+0000", 2024-04-07T17:51:28.226 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:23:15.880710+0000", 2024-04-07T17:51:28.226 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.354070+0000", 2024-04-07T17:51:28.226 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:28.226 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:28.226 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:28.227 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 445, 2024-04-07T17:51:28.227 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.354070+0000", 2024-04-07T17:51:28.227 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.354070+0000", 2024-04-07T17:51:28.227 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.354070+0000", 2024-04-07T17:51:28.227 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:28.227 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:28.227 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:28.227 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.354070+0000", 2024-04-07T17:51:28.227 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.354070+0000", 2024-04-07T17:51:28.227 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:28.227 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 8, 2024-04-07T17:51:28.227 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:28.227 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:28.227 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 444, 2024-04-07T17:51:28.227 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:28.227 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:28.228 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:28.228 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:28.228 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 8, 2024-04-07T17:51:28.228 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:28.228 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:28.228 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:28.228 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.10", 2024-04-07T17:51:28.228 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:28.228 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:28.228 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:28.228 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4979, 2024-04-07T17:51:28.228 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:28.228 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T17:16:20.268788+0000", 2024-04-07T17:51:28.228 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:28.228 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:28.229 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:28.229 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:28.229 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:28.229 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:28.229 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:28.229 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:28.229 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:28.229 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:28.229 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:28.229 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:28.229 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:28.229 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:28.229 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:28.229 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:28.229 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:28.230 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:28.230 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:28.230 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 12, 2024-04-07T17:51:28.230 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 4, 2024-04-07T17:51:28.230 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:28.230 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 4, 2024-04-07T17:51:28.230 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:28.230 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:28.230 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:28.230 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:28.230 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:28.230 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 4, 2024-04-07T17:51:28.230 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:28.230 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:28.230 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:28.231 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:28.231 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:28.231 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:28.231 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:28.231 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 24, 2024-04-07T17:51:28.231 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 24, 2024-04-07T17:51:28.231 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:28.231 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:28.231 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:28.231 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 12, 2024-04-07T17:51:28.231 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:28.231 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.231 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:28.231 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:28.231 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:28.231 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T17:51:28.232 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:28.232 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T17:51:28.232 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.232 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 2, 2024-04-07T17:51:28.232 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "61'8" 2024-04-07T17:51:28.232 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.232 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:28.232 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:28.232 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:28.232 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:28.232 INFO:tasks.workunit.client.0.smithi100.stdout: 5 2024-04-07T17:51:28.232 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.232 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 6, 2024-04-07T17:51:28.232 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:28.232 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:28.232 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:28.233 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:28.233 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:28.233 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:28.233 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.370556+0000", 2024-04-07T17:51:28.233 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:08.963259+0000", 2024-04-07T17:51:28.233 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:08.963259+0000", 2024-04-07T17:51:28.233 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:08.964055+0000", 2024-04-07T17:51:28.233 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.370556+0000", 2024-04-07T17:51:28.233 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:28.233 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:28.233 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:28.233 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 554, 2024-04-07T17:51:28.233 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.370556+0000", 2024-04-07T17:51:28.233 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.370556+0000", 2024-04-07T17:51:28.233 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.370556+0000", 2024-04-07T17:51:28.234 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:28.234 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:28.234 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:28.234 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.370556+0000", 2024-04-07T17:51:28.234 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.370556+0000", 2024-04-07T17:51:28.234 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:28.234 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:28.234 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:28.234 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:28.234 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 553, 2024-04-07T17:51:28.234 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:28.234 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:28.234 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:28.234 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:28.234 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:28.235 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:28.235 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:28.235 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:28.235 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.10", 2024-04-07T17:51:28.235 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:28.235 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:28.235 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:28.235 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4829, 2024-04-07T17:51:28.235 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:28.235 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T23:13:34.886609+0000", 2024-04-07T17:51:28.235 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:28.235 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:28.235 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:28.235 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:28.235 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:28.236 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:28.236 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:28.236 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:28.236 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:28.236 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:28.236 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:28.236 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:28.236 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:28.236 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:28.236 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:28.236 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:28.236 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:28.236 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:28.236 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:28.236 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:28.237 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:28.237 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:28.237 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:28.237 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:28.237 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:28.237 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:28.237 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:28.237 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:28.237 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:28.237 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:28.237 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:28.237 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:28.237 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:28.237 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:28.237 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:28.237 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:28.237 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:28.238 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:28.238 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:28.238 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:28.238 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:28.238 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:28.238 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:28.238 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.238 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:28.238 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:28.238 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:28.238 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:28.238 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:28.238 INFO:tasks.workunit.client.0.smithi100.stdout: 5 2024-04-07T17:51:28.238 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.238 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 6, 2024-04-07T17:51:28.239 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:28.239 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.239 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:28.239 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:28.239 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:28.239 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:28.239 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T17:51:28.239 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.239 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 6, 2024-04-07T17:51:28.239 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:28.239 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:28.239 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T17:51:28.239 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:28.239 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:28.239 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:28.239 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.371043+0000", 2024-04-07T17:51:28.240 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:08.965799+0000", 2024-04-07T17:51:28.240 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:08.965799+0000", 2024-04-07T17:51:28.240 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:08.966301+0000", 2024-04-07T17:51:28.240 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.371043+0000", 2024-04-07T17:51:28.240 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:28.240 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:28.240 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:28.240 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 554, 2024-04-07T17:51:28.240 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.371043+0000", 2024-04-07T17:51:28.240 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.371043+0000", 2024-04-07T17:51:28.240 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.371043+0000", 2024-04-07T17:51:28.240 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:28.240 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:28.240 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:28.240 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.371043+0000", 2024-04-07T17:51:28.240 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.371043+0000", 2024-04-07T17:51:28.241 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:28.241 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:28.241 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:28.241 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:28.241 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 553, 2024-04-07T17:51:28.241 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:28.241 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:28.241 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:28.241 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:28.241 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:28.241 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:28.241 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:28.241 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:28.241 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.11", 2024-04-07T17:51:28.241 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:28.241 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:28.242 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:28.242 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4845, 2024-04-07T17:51:28.242 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:28.242 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T21:12:09.705565+0000", 2024-04-07T17:51:28.242 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:28.242 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:28.242 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:28.242 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:28.242 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:28.242 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:28.242 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:28.242 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:28.242 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:28.242 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:28.242 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:28.243 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:28.243 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:28.243 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:28.243 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:28.243 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:28.243 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:28.243 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:28.243 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:28.243 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:28.243 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:28.243 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:28.243 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:28.243 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:28.243 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:28.243 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:28.244 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:28.244 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:28.244 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:28.244 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:28.244 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:28.244 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:28.244 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:28.244 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:28.244 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:28.244 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:28.244 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:28.244 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:28.244 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:28.244 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:28.244 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:28.244 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:28.245 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:28.245 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.245 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:28.245 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:28.245 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:28.245 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:28.245 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:28.245 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T17:51:28.245 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.245 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 6, 2024-04-07T17:51:28.245 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:28.245 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.245 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:28.245 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:28.245 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:28.246 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:28.246 INFO:tasks.workunit.client.0.smithi100.stdout: 2 2024-04-07T17:51:28.246 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.246 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 7, 2024-04-07T17:51:28.246 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:28.246 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:28.246 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:28.246 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:28.246 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:28.246 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:28.246 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.368748+0000", 2024-04-07T17:51:28.246 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:38.987994+0000", 2024-04-07T17:51:28.246 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:38.987994+0000", 2024-04-07T17:51:28.246 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:38.991515+0000", 2024-04-07T17:51:28.246 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.368748+0000", 2024-04-07T17:51:28.247 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:28.247 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:28.247 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:28.247 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:28.247 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.368748+0000", 2024-04-07T17:51:28.247 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.368748+0000", 2024-04-07T17:51:28.247 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.368748+0000", 2024-04-07T17:51:28.247 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:28.247 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:28.247 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:28.247 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.368748+0000", 2024-04-07T17:51:28.247 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.368748+0000", 2024-04-07T17:51:28.247 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:28.247 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:28.247 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:28.248 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:28.248 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:28.248 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:28.248 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:28.248 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:28.248 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:28.248 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:28.248 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:28.248 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:28.248 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:28.248 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.13", 2024-04-07T17:51:28.248 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:28.248 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:28.248 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:28.248 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4789, 2024-04-07T17:51:28.248 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:28.249 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T20:51:08.365772+0000", 2024-04-07T17:51:28.249 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:28.249 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:28.249 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:28.249 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:28.249 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:28.249 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:28.249 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:28.249 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:28.249 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:28.249 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:28.249 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:28.249 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:28.249 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:28.249 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:28.249 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:28.250 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:28.250 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:28.250 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:28.250 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:28.250 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:28.250 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:28.250 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:28.250 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:28.250 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:28.250 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:28.250 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:28.250 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:28.250 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:28.250 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:28.250 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:28.251 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:28.251 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:28.251 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:28.251 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:28.251 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:28.251 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:28.251 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:28.251 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:28.251 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:28.251 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:28.251 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:28.251 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:28.251 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:28.251 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.251 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:28.252 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:28.252 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:28.252 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:28.252 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:28.252 INFO:tasks.workunit.client.0.smithi100.stdout: 2 2024-04-07T17:51:28.252 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.252 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 7, 2024-04-07T17:51:28.252 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:28.252 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.252 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:28.252 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:28.252 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:28.252 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:28.252 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T17:51:28.252 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.252 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 5, 2024-04-07T17:51:28.253 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:28.253 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:28.253 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T17:51:28.253 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:28.253 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:28.253 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:28.253 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.369416+0000", 2024-04-07T17:51:28.253 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:38.978180+0000", 2024-04-07T17:51:28.253 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:38.978180+0000", 2024-04-07T17:51:28.253 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:38.979026+0000", 2024-04-07T17:51:28.253 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.369416+0000", 2024-04-07T17:51:28.253 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:28.253 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:28.253 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:28.253 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:28.254 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.369416+0000", 2024-04-07T17:51:28.254 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.369416+0000", 2024-04-07T17:51:28.254 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.369416+0000", 2024-04-07T17:51:28.254 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:28.254 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:28.254 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:28.254 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.369416+0000", 2024-04-07T17:51:28.254 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.369416+0000", 2024-04-07T17:51:28.254 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:28.254 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:28.254 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:28.254 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:28.254 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:28.254 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:28.254 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:28.255 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:28.255 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:28.255 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:28.255 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:28.255 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:28.255 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:28.255 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.12", 2024-04-07T17:51:28.255 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:28.255 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:28.255 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:28.255 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4877, 2024-04-07T17:51:28.255 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:28.255 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T01:41:45.196090+0000", 2024-04-07T17:51:28.255 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:28.256 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:28.256 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:28.256 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:28.256 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:28.256 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:28.256 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:28.256 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:28.256 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:28.256 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:28.256 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:28.256 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:28.256 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:28.256 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:28.256 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:28.256 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:28.257 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:28.257 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:28.257 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:28.257 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:28.257 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:28.257 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:28.257 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:28.257 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:28.257 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:28.257 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:28.257 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:28.257 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:28.257 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:28.257 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:28.257 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:28.258 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:28.258 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:28.258 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:28.258 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:28.258 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:28.258 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:28.258 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:28.258 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:28.258 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:28.258 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:28.258 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:28.258 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:28.258 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.258 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:28.258 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:28.259 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:28.259 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:28.259 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:28.259 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T17:51:28.259 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.259 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 5, 2024-04-07T17:51:28.259 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:28.259 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.259 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:28.259 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:28.259 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T17:51:28.259 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:28.259 INFO:tasks.workunit.client.0.smithi100.stdout: 3 2024-04-07T17:51:28.259 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.259 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 2, 2024-04-07T17:51:28.260 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:28.260 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:28.260 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:28.260 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:28.260 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:28.260 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:28.260 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.354815+0000", 2024-04-07T17:51:28.260 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:38.969402+0000", 2024-04-07T17:51:28.260 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:38.969402+0000", 2024-04-07T17:51:28.260 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:38.970054+0000", 2024-04-07T17:51:28.260 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.354815+0000", 2024-04-07T17:51:28.260 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:28.260 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:28.260 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:28.260 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:28.260 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.354815+0000", 2024-04-07T17:51:28.261 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.354815+0000", 2024-04-07T17:51:28.261 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.354815+0000", 2024-04-07T17:51:28.261 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:28.261 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:28.261 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:28.261 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.354815+0000", 2024-04-07T17:51:28.261 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.354815+0000", 2024-04-07T17:51:28.261 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:28.261 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:28.261 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:28.261 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:28.261 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:28.261 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:28.261 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:28.261 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:28.262 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:28.262 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:28.262 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:28.262 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:28.262 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:28.262 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.12", 2024-04-07T17:51:28.262 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:28.262 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:28.262 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:28.262 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4924, 2024-04-07T17:51:28.262 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:28.262 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T23:11:21.401536+0000", 2024-04-07T17:51:28.262 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:28.262 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:28.262 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:28.263 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:28.263 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:28.263 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:28.263 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:28.263 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:28.263 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:28.263 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:28.263 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:28.263 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:28.263 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:28.263 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:28.263 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:28.263 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:28.263 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:28.263 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:28.264 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:28.264 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:28.264 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:28.264 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:28.264 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:28.264 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:28.264 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:28.264 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:28.264 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:28.264 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:28.264 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:28.264 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:28.264 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:28.264 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:28.264 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:28.265 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:28.265 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:28.265 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:28.265 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:28.265 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:28.265 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:28.265 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:28.265 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:28.265 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:28.265 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:28.265 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.265 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:28.265 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:28.265 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:28.265 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T17:51:28.265 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:28.266 INFO:tasks.workunit.client.0.smithi100.stdout: 3 2024-04-07T17:51:28.266 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.266 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 2, 2024-04-07T17:51:28.266 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:28.266 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.266 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:28.266 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:28.266 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:28.266 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:28.266 INFO:tasks.workunit.client.0.smithi100.stdout: 3 2024-04-07T17:51:28.266 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.266 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 0, 2024-04-07T17:51:28.266 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:28.266 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:28.266 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T17:51:28.267 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:28.267 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:28.267 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:28.267 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:35.988325+0000", 2024-04-07T17:51:28.267 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:24:13.963512+0000", 2024-04-07T17:51:28.267 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:24:13.963512+0000", 2024-04-07T17:51:28.267 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:24:13.964689+0000", 2024-04-07T17:51:28.267 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:35.988325+0000", 2024-04-07T17:51:28.267 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:28.267 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:28.267 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:28.267 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 500, 2024-04-07T17:51:28.267 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:35.988325+0000", 2024-04-07T17:51:28.267 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:35.988325+0000", 2024-04-07T17:51:28.267 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:35.988325+0000", 2024-04-07T17:51:28.268 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:28.268 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:28.268 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:28.268 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:35.988325+0000", 2024-04-07T17:51:28.268 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:35.988325+0000", 2024-04-07T17:51:28.268 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:28.268 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 2, 2024-04-07T17:51:28.268 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:28.268 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:28.268 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 499, 2024-04-07T17:51:28.268 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:28.268 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:28.268 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:28.268 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:28.268 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 2, 2024-04-07T17:51:28.268 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:28.269 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:28.269 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:28.269 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.13", 2024-04-07T17:51:28.269 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:28.269 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:28.269 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:28.269 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 5000, 2024-04-07T17:51:28.269 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:28.269 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T16:15:37.955732+0000", 2024-04-07T17:51:28.269 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:28.269 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:28.269 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:28.269 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 34, 2024-04-07T17:51:28.269 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:28.270 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:28.270 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:28.270 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:28.270 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:28.270 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:28.270 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:28.270 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:28.270 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:28.270 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:28.270 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:28.270 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:28.270 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:28.270 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:28.270 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:28.271 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 6, 2024-04-07T17:51:28.271 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 2, 2024-04-07T17:51:28.271 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:28.271 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 2, 2024-04-07T17:51:28.271 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:28.271 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:28.271 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:28.271 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:28.271 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:28.271 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 1, 2024-04-07T17:51:28.271 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:28.271 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:28.271 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:28.271 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:28.271 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:28.271 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:28.272 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:28.272 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 4, 2024-04-07T17:51:28.272 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 8, 2024-04-07T17:51:28.272 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:28.272 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:28.272 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:28.272 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 3, 2024-04-07T17:51:28.272 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 7 2024-04-07T17:51:28.272 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.272 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:28.272 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:28.272 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:28.272 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:28.272 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:28.272 INFO:tasks.workunit.client.0.smithi100.stdout: 3 2024-04-07T17:51:28.272 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.273 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 0, 2024-04-07T17:51:28.273 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "61'2" 2024-04-07T17:51:28.273 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.273 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:28.273 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:28.273 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:28.273 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:28.273 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T17:51:28.273 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.273 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 7, 2024-04-07T17:51:28.273 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:28.273 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:28.273 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:28.273 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:28.273 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:28.274 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:28.274 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.372482+0000", 2024-04-07T17:51:28.274 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:38.987197+0000", 2024-04-07T17:51:28.274 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:38.987197+0000", 2024-04-07T17:51:28.274 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:38.999803+0000", 2024-04-07T17:51:28.274 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.372482+0000", 2024-04-07T17:51:28.274 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:28.274 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:28.274 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:28.274 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:28.274 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.372482+0000", 2024-04-07T17:51:28.274 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.372482+0000", 2024-04-07T17:51:28.274 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.372482+0000", 2024-04-07T17:51:28.274 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:28.275 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:28.275 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:28.275 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.372482+0000", 2024-04-07T17:51:28.275 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.372482+0000", 2024-04-07T17:51:28.275 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:28.275 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:28.275 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:28.275 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:28.275 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:28.275 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:28.275 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:28.275 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:28.275 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:28.275 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:28.275 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:28.275 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:28.276 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:28.276 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.15", 2024-04-07T17:51:28.276 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:28.276 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:28.276 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:28.276 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4809, 2024-04-07T17:51:28.276 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:28.276 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T03:29:56.940037+0000", 2024-04-07T17:51:28.276 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:28.276 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:28.276 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:28.276 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:28.276 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:28.276 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:28.276 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:28.277 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:28.277 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:28.277 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:28.277 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:28.277 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:28.277 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:28.277 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:28.277 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:28.277 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:28.277 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:28.277 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:28.277 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:28.277 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:28.277 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:28.277 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:28.277 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:28.278 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:28.278 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:28.278 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:28.278 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:28.278 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:28.278 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:28.278 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:28.278 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:28.278 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:28.278 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:28.278 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:28.278 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:28.278 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:28.278 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:28.278 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:28.279 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:28.279 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:28.279 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:28.279 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:28.279 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:28.279 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.279 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:28.279 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:28.279 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:28.279 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:28.279 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:28.279 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T17:51:28.279 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.279 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 7, 2024-04-07T17:51:28.279 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:28.279 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.280 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:28.280 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:28.280 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:28.280 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:28.280 INFO:tasks.workunit.client.0.smithi100.stdout: 5 2024-04-07T17:51:28.280 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.280 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 6, 2024-04-07T17:51:28.280 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:28.280 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:28.280 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T17:51:28.280 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:28.280 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:28.280 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:28.280 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.370811+0000", 2024-04-07T17:51:28.280 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:08.975701+0000", 2024-04-07T17:51:28.280 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:08.975701+0000", 2024-04-07T17:51:28.281 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:08.977082+0000", 2024-04-07T17:51:28.281 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.370811+0000", 2024-04-07T17:51:28.281 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:28.281 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:28.281 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:28.281 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 554, 2024-04-07T17:51:28.281 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.370811+0000", 2024-04-07T17:51:28.281 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.370811+0000", 2024-04-07T17:51:28.281 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.370811+0000", 2024-04-07T17:51:28.281 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:28.281 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:28.281 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:28.281 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.370811+0000", 2024-04-07T17:51:28.281 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.370811+0000", 2024-04-07T17:51:28.281 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:28.282 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 29, 2024-04-07T17:51:28.282 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:28.282 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:28.282 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 553, 2024-04-07T17:51:28.282 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:28.282 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:28.282 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:28.282 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:28.282 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 29, 2024-04-07T17:51:28.282 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:28.282 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:28.282 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:28.282 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.14", 2024-04-07T17:51:28.282 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:28.283 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:28.283 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:28.283 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4865, 2024-04-07T17:51:28.283 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:28.283 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T20:48:03.443222+0000", 2024-04-07T17:51:28.283 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:28.283 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:28.283 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:28.283 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 5824, 2024-04-07T17:51:28.283 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:28.283 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:28.283 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:28.283 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:28.283 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:28.283 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:28.283 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:28.284 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:28.284 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:28.284 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:28.284 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:28.284 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:28.284 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:28.284 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:28.284 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:28.284 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 12, 2024-04-07T17:51:28.284 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 4, 2024-04-07T17:51:28.284 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:28.284 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 4, 2024-04-07T17:51:28.284 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:28.284 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:28.284 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:28.285 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:28.285 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:28.285 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 1, 2024-04-07T17:51:28.285 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:28.285 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:28.285 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:28.285 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:28.285 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:28.285 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:28.285 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:28.285 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 10, 2024-04-07T17:51:28.285 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 14, 2024-04-07T17:51:28.285 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:28.285 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:28.285 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:28.286 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 28, 2024-04-07T17:51:28.286 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 25 2024-04-07T17:51:28.286 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.286 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:28.286 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:28.286 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:28.286 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:28.286 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:28.286 INFO:tasks.workunit.client.0.smithi100.stdout: 5 2024-04-07T17:51:28.286 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.286 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 6, 2024-04-07T17:51:28.286 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "623'29" 2024-04-07T17:51:28.286 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.286 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:28.286 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:28.287 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:28.287 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:28.287 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T17:51:28.287 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.287 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 4, 2024-04-07T17:51:28.287 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:28.287 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:28.287 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:28.287 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:28.287 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:28.287 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:28.287 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:35.981022+0000", 2024-04-07T17:51:28.287 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:38.985283+0000", 2024-04-07T17:51:28.287 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:38.985283+0000", 2024-04-07T17:51:28.287 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:38.986153+0000", 2024-04-07T17:51:28.287 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:35.981022+0000", 2024-04-07T17:51:28.288 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:28.288 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:28.288 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:28.288 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:28.288 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:35.981022+0000", 2024-04-07T17:51:28.288 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:35.981022+0000", 2024-04-07T17:51:28.288 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:35.981022+0000", 2024-04-07T17:51:28.288 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:28.288 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:28.288 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:28.288 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:35.981022+0000", 2024-04-07T17:51:28.288 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:35.981022+0000", 2024-04-07T17:51:28.288 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:28.288 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:28.288 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:28.288 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:28.289 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:28.289 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:28.289 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:28.289 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:28.289 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:28.289 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:28.289 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:28.289 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:28.289 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:28.289 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.14", 2024-04-07T17:51:28.289 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:28.289 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:28.289 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:28.289 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4907, 2024-04-07T17:51:28.289 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:28.290 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T22:16:21.117705+0000", 2024-04-07T17:51:28.290 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:28.290 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:28.290 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:28.290 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:28.290 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:28.290 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:28.290 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:28.290 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:28.290 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:28.290 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:28.290 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:28.290 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:28.290 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:28.290 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:28.291 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:28.291 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:28.291 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:28.291 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:28.291 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:28.291 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:28.291 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:28.291 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:28.291 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:28.291 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:28.291 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:28.291 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:28.291 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:28.291 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:28.291 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:28.291 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:28.292 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:28.292 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:28.292 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:28.292 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:28.292 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:28.292 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:28.292 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:28.292 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:28.292 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:28.292 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:28.292 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:28.292 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:28.292 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:28.292 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.292 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:28.292 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:28.293 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:28.293 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:28.293 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:28.293 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T17:51:28.293 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.293 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 4, 2024-04-07T17:51:28.293 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:28.293 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.293 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:28.293 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:28.293 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:28.293 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:28.293 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T17:51:28.293 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.293 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 1, 2024-04-07T17:51:28.293 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:28.294 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:28.294 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T17:51:28.294 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:28.294 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:28.294 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:28.294 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.332631+0000", 2024-04-07T17:51:28.294 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:24:41.527553+0000", 2024-04-07T17:51:28.294 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:24:41.527553+0000", 2024-04-07T17:51:28.294 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:24:41.530429+0000", 2024-04-07T17:51:28.294 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.332631+0000", 2024-04-07T17:51:28.294 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:28.294 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:28.294 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:28.294 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 527, 2024-04-07T17:51:28.294 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.332631+0000", 2024-04-07T17:51:28.295 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.332631+0000", 2024-04-07T17:51:28.295 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.332631+0000", 2024-04-07T17:51:28.295 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:28.295 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:28.295 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:28.295 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.332631+0000", 2024-04-07T17:51:28.295 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.332631+0000", 2024-04-07T17:51:28.295 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:28.295 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 3, 2024-04-07T17:51:28.295 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:28.295 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:28.295 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 526, 2024-04-07T17:51:28.295 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:28.295 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:28.295 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:28.296 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:28.296 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 3, 2024-04-07T17:51:28.296 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:28.296 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:28.296 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:28.296 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.15", 2024-04-07T17:51:28.296 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:28.296 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:28.296 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:28.296 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4990, 2024-04-07T17:51:28.296 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:28.296 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T16:34:47.703047+0000", 2024-04-07T17:51:28.296 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:28.296 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:28.297 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:28.297 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 90, 2024-04-07T17:51:28.297 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:28.297 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:28.297 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:28.297 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:28.297 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:28.297 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:28.297 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:28.297 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:28.297 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:28.297 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:28.297 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:28.297 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:28.297 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:28.297 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:28.298 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:28.298 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 6, 2024-04-07T17:51:28.298 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 2, 2024-04-07T17:51:28.298 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:28.298 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 2, 2024-04-07T17:51:28.298 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:28.298 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:28.298 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:28.298 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:28.298 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:28.298 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 1, 2024-04-07T17:51:28.298 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:28.298 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:28.298 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:28.298 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:28.299 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:28.299 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:28.299 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:28.299 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 7, 2024-04-07T17:51:28.299 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 7, 2024-04-07T17:51:28.299 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:28.299 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:28.299 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:28.299 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 4, 2024-04-07T17:51:28.299 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 1 2024-04-07T17:51:28.299 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.299 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:28.299 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:28.299 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:28.299 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:28.300 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:28.300 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T17:51:28.300 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.300 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 1, 2024-04-07T17:51:28.300 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "61'3" 2024-04-07T17:51:28.300 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.300 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:28.300 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:28.300 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T17:51:28.300 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:28.300 INFO:tasks.workunit.client.0.smithi100.stdout: 3 2024-04-07T17:51:28.300 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.300 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 2, 2024-04-07T17:51:28.300 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:28.300 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:28.301 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:28.301 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:28.301 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:28.301 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:28.301 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.363660+0000", 2024-04-07T17:51:28.301 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:24:41.558300+0000", 2024-04-07T17:51:28.301 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:24:41.558300+0000", 2024-04-07T17:51:28.301 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:24:41.558648+0000", 2024-04-07T17:51:28.301 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.363660+0000", 2024-04-07T17:51:28.301 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:28.301 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:28.301 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:28.301 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 527, 2024-04-07T17:51:28.301 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.363660+0000", 2024-04-07T17:51:28.301 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.363660+0000", 2024-04-07T17:51:28.302 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.363660+0000", 2024-04-07T17:51:28.302 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:28.302 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:28.302 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:28.302 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.363660+0000", 2024-04-07T17:51:28.302 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.363660+0000", 2024-04-07T17:51:28.302 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:28.302 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:28.302 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:28.302 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:28.302 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 526, 2024-04-07T17:51:28.302 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:28.302 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:28.302 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:28.303 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:28.303 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:28.303 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:28.303 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:28.303 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:28.303 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.17", 2024-04-07T17:51:28.303 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:28.303 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:28.303 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:28.303 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4923, 2024-04-07T17:51:28.303 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:28.303 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T01:44:12.981988+0000", 2024-04-07T17:51:28.303 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:28.303 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:28.303 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:28.303 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:28.304 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:28.304 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:28.304 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:28.304 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:28.304 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:28.304 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:28.304 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:28.304 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:28.304 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:28.304 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:28.304 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:28.304 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:28.304 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:28.304 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:28.304 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:28.304 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:28.305 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:28.305 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:28.305 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:28.305 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:28.305 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:28.305 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:28.305 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:28.305 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:28.305 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:28.305 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:28.305 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:28.305 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:28.305 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:28.305 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:28.305 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:28.306 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:28.306 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:28.306 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:28.306 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:28.306 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:28.306 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:28.306 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:28.306 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:28.306 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.306 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:28.306 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:28.306 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:28.306 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T17:51:28.306 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:28.307 INFO:tasks.workunit.client.0.smithi100.stdout: 3 2024-04-07T17:51:28.307 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.307 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 2, 2024-04-07T17:51:28.307 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:28.307 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.307 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:28.307 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:28.307 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:28.307 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:28.307 INFO:tasks.workunit.client.0.smithi100.stdout: 2 2024-04-07T17:51:28.307 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.307 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 5, 2024-04-07T17:51:28.307 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:28.307 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:28.307 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T17:51:28.307 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:28.307 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:28.308 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:28.308 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.368702+0000", 2024-04-07T17:51:28.308 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:08.971659+0000", 2024-04-07T17:51:28.308 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:08.971659+0000", 2024-04-07T17:51:28.308 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:08.976930+0000", 2024-04-07T17:51:28.308 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.368702+0000", 2024-04-07T17:51:28.308 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:28.308 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:28.308 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:28.308 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 554, 2024-04-07T17:51:28.308 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.368702+0000", 2024-04-07T17:51:28.308 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.368702+0000", 2024-04-07T17:51:28.308 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.368702+0000", 2024-04-07T17:51:28.308 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:28.308 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:28.308 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:28.309 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.368702+0000", 2024-04-07T17:51:28.309 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.368702+0000", 2024-04-07T17:51:28.309 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:28.309 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:28.309 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:28.309 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:28.309 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 553, 2024-04-07T17:51:28.309 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:28.309 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:28.309 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:28.309 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:28.309 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:28.309 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:28.309 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:28.309 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:28.309 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.16", 2024-04-07T17:51:28.310 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:28.310 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:28.310 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:28.310 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4848, 2024-04-07T17:51:28.310 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:28.310 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T02:13:26.493523+0000", 2024-04-07T17:51:28.310 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:28.310 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:28.310 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:28.310 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:28.310 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:28.310 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:28.310 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:28.310 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:28.311 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:28.311 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:28.311 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:28.311 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:28.311 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:28.311 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:28.311 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:28.311 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:28.311 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:28.311 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:28.311 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:28.311 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:28.311 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:28.311 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:28.311 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:28.312 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:28.312 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:28.312 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:28.312 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:28.312 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:28.312 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:28.312 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:28.312 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:28.312 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:28.312 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:28.312 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:28.312 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:28.312 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:28.312 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:28.312 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:28.313 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:28.313 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:28.313 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:28.313 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:28.313 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:28.313 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.313 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:28.313 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:28.313 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:28.313 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:28.313 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:28.313 INFO:tasks.workunit.client.0.smithi100.stdout: 2 2024-04-07T17:51:28.313 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.313 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 5, 2024-04-07T17:51:28.313 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:28.313 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.314 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:28.314 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:28.314 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:28.314 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:28.314 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T17:51:28.314 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.314 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 5, 2024-04-07T17:51:28.314 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:28.314 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:28.314 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:28.314 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:28.314 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:28.314 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:28.315 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.369127+0000", 2024-04-07T17:51:28.315 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:38.977859+0000", 2024-04-07T17:51:28.315 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:38.977859+0000", 2024-04-07T17:51:28.315 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:38.978447+0000", 2024-04-07T17:51:28.315 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.369127+0000", 2024-04-07T17:51:28.315 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:28.315 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:28.315 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:28.315 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:28.315 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.369127+0000", 2024-04-07T17:51:28.315 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.369127+0000", 2024-04-07T17:51:28.315 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.369127+0000", 2024-04-07T17:51:28.315 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:28.315 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:28.315 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:28.316 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.369127+0000", 2024-04-07T17:51:28.316 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.369127+0000", 2024-04-07T17:51:28.316 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:28.316 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:28.316 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:28.316 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:28.316 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:28.316 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:28.316 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:28.316 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:28.316 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:28.316 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:28.316 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:28.316 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:28.316 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:28.317 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.16", 2024-04-07T17:51:28.317 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:28.317 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:28.317 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:28.317 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4860, 2024-04-07T17:51:28.317 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:28.317 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T16:22:51.136501+0000", 2024-04-07T17:51:28.317 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:28.317 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:28.317 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:28.317 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:28.317 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:28.317 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:28.317 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:28.317 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:28.317 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:28.318 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:28.318 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:28.318 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:28.318 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:28.318 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:28.318 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:28.318 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:28.318 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:28.318 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:28.318 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:28.318 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:28.318 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:28.318 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:28.318 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:28.319 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:28.319 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:28.319 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:28.319 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:28.319 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:28.319 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:28.319 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:28.319 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:28.319 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:28.319 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:28.319 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:28.319 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:28.319 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:28.319 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:28.319 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:28.320 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:28.320 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:28.320 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:28.320 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:28.320 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:28.320 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.320 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:28.320 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:28.320 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:28.320 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:28.320 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:28.320 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T17:51:28.320 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.320 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 5, 2024-04-07T17:51:28.320 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:28.320 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.321 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:28.321 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:28.321 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:28.321 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:28.321 INFO:tasks.workunit.client.0.smithi100.stdout: 2 2024-04-07T17:51:28.321 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.321 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 6, 2024-04-07T17:51:28.321 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:28.321 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:28.321 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T17:51:28.321 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:28.321 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:28.321 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:28.321 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.370115+0000", 2024-04-07T17:51:28.321 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:08.977704+0000", 2024-04-07T17:51:28.322 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:08.977704+0000", 2024-04-07T17:51:28.322 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:08.978095+0000", 2024-04-07T17:51:28.322 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.370115+0000", 2024-04-07T17:51:28.322 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:28.322 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:28.322 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:28.322 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 554, 2024-04-07T17:51:28.322 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.370115+0000", 2024-04-07T17:51:28.322 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.370115+0000", 2024-04-07T17:51:28.322 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.370115+0000", 2024-04-07T17:51:28.322 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:28.322 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:28.322 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:28.322 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.370115+0000", 2024-04-07T17:51:28.322 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.370115+0000", 2024-04-07T17:51:28.323 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:28.323 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:28.323 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:28.323 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:28.323 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 553, 2024-04-07T17:51:28.323 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:28.323 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:28.323 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:28.323 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:28.323 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:28.323 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:28.323 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:28.323 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:28.323 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.17", 2024-04-07T17:51:28.323 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:28.323 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:28.323 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:28.323 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4829, 2024-04-07T17:51:28.324 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:28.324 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T18:47:28.916356+0000", 2024-04-07T17:51:28.324 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:28.324 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:28.324 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:28.324 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:28.324 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:28.324 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:28.324 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:28.324 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:28.324 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:28.324 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:28.324 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:28.324 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:28.324 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:28.325 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:28.325 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:28.325 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:28.325 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:28.325 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:28.325 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:28.325 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:28.325 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:28.325 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:28.325 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:28.325 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:28.325 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:28.325 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:28.325 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:28.325 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:28.325 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:28.326 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:28.326 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:28.326 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:28.326 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:28.326 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:28.326 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:28.326 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:28.326 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:28.326 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:28.326 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:28.326 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:28.326 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:28.326 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:28.326 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:28.327 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.327 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:28.327 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:28.327 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:28.327 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:28.327 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:28.327 INFO:tasks.workunit.client.0.smithi100.stdout: 2 2024-04-07T17:51:28.327 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.327 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 6, 2024-04-07T17:51:28.327 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:28.327 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.327 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:28.327 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:28.327 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:28.327 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:28.328 INFO:tasks.workunit.client.0.smithi100.stdout: 4 2024-04-07T17:51:28.328 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.328 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 1, 2024-04-07T17:51:28.328 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:28.328 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:28.328 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:28.328 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:28.328 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:28.328 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:28.328 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.331404+0000", 2024-04-07T17:51:28.328 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:24:13.932694+0000", 2024-04-07T17:51:28.328 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:24:13.932694+0000", 2024-04-07T17:51:28.328 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:24:13.933594+0000", 2024-04-07T17:51:28.328 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.331404+0000", 2024-04-07T17:51:28.328 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:28.329 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:28.329 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:28.329 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 500, 2024-04-07T17:51:28.329 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.331404+0000", 2024-04-07T17:51:28.329 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.331404+0000", 2024-04-07T17:51:28.329 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.331404+0000", 2024-04-07T17:51:28.329 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:28.329 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:28.329 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:28.329 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.331404+0000", 2024-04-07T17:51:28.329 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.331404+0000", 2024-04-07T17:51:28.329 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:28.329 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:28.329 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:28.329 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:28.330 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 499, 2024-04-07T17:51:28.330 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:28.330 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:28.330 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:28.330 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:28.330 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:28.330 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:28.330 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:28.330 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:28.330 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.19", 2024-04-07T17:51:28.330 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:28.330 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:28.330 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:28.330 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4977, 2024-04-07T17:51:28.331 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:28.331 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T17:08:30.909773+0000", 2024-04-07T17:51:28.331 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:28.331 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:28.331 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:28.331 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:28.331 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:28.331 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:28.331 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:28.331 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:28.331 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:28.331 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:28.331 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:28.331 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:28.331 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:28.332 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:28.332 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:28.332 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:28.332 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:28.332 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:28.332 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:28.332 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:28.332 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:28.332 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:28.332 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:28.332 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:28.332 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:28.332 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:28.332 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:28.332 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:28.332 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:28.333 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:28.333 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:28.333 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:28.333 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:28.333 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:28.333 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:28.333 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:28.333 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:28.333 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:28.333 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:28.333 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:28.333 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:28.333 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:28.333 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:28.333 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.333 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:28.334 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:28.334 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:28.334 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:28.334 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:28.334 INFO:tasks.workunit.client.0.smithi100.stdout: 4 2024-04-07T17:51:28.334 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.334 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 1, 2024-04-07T17:51:28.334 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:28.334 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.334 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:28.334 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:28.334 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:28.334 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:28.334 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T17:51:28.335 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.335 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 5, 2024-04-07T17:51:28.335 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:28.335 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:28.335 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T17:51:28.335 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:28.335 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:28.335 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:28.335 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.361600+0000", 2024-04-07T17:51:28.335 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:38.990134+0000", 2024-04-07T17:51:28.335 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:38.990134+0000", 2024-04-07T17:51:28.335 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:38.990885+0000", 2024-04-07T17:51:28.335 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.361600+0000", 2024-04-07T17:51:28.335 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:28.335 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:28.335 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:28.336 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:28.336 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.361600+0000", 2024-04-07T17:51:28.336 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.361600+0000", 2024-04-07T17:51:28.336 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.361600+0000", 2024-04-07T17:51:28.336 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:28.336 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:28.336 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:28.336 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.361600+0000", 2024-04-07T17:51:28.336 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.361600+0000", 2024-04-07T17:51:28.336 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:28.336 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:28.336 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:28.336 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:28.336 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:28.336 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:28.337 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:28.337 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:28.337 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:28.337 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:28.337 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:28.337 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:28.337 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:28.337 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.18", 2024-04-07T17:51:28.337 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:28.337 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:28.337 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:28.337 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4861, 2024-04-07T17:51:28.337 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:28.337 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T19:33:03.967565+0000", 2024-04-07T17:51:28.337 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:28.337 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:28.338 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:28.338 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:28.338 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:28.338 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:28.338 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:28.338 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:28.338 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:28.338 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:28.338 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:28.338 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:28.338 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:28.338 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:28.338 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:28.338 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:28.338 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:28.339 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:28.339 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:28.339 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:28.339 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:28.339 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:28.339 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:28.339 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:28.339 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:28.339 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:28.339 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:28.339 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:28.339 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:28.339 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:28.339 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:28.339 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:28.340 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:28.340 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:28.340 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:28.340 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:28.340 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:28.340 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:28.340 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:28.340 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:28.340 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:28.340 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:28.340 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:28.340 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.340 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:28.340 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:28.340 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:28.341 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:28.341 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:28.341 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T17:51:28.341 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.341 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 5, 2024-04-07T17:51:28.341 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:28.341 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.341 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:28.341 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:28.341 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:28.341 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:28.341 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T17:51:28.341 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.341 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 3, 2024-04-07T17:51:28.341 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:28.341 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:28.342 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:28.342 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:28.342 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:28.342 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:28.342 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:35.975334+0000", 2024-04-07T17:51:28.342 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:23:43.903341+0000", 2024-04-07T17:51:28.342 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:23:43.903341+0000", 2024-04-07T17:51:28.342 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:23:43.904215+0000", 2024-04-07T17:51:28.342 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:35.975334+0000", 2024-04-07T17:51:28.342 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:28.342 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:28.342 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:28.342 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 471, 2024-04-07T17:51:28.342 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:35.975334+0000", 2024-04-07T17:51:28.343 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:35.975334+0000", 2024-04-07T17:51:28.343 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:35.975334+0000", 2024-04-07T17:51:28.343 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:28.343 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:28.343 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:28.343 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:35.975334+0000", 2024-04-07T17:51:28.343 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:35.975334+0000", 2024-04-07T17:51:28.343 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:28.343 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:28.343 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:28.343 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:28.343 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 470, 2024-04-07T17:51:28.343 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:28.343 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:28.343 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:28.343 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:28.344 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:28.344 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:28.344 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:28.344 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:28.344 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.18", 2024-04-07T17:51:28.344 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:28.344 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:28.344 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:28.344 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4969, 2024-04-07T17:51:28.344 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:28.344 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T23:54:23.660433+0000", 2024-04-07T17:51:28.344 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:28.344 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:28.344 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:28.344 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:28.344 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:28.345 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:28.345 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:28.345 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:28.345 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:28.345 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:28.345 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:28.345 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:28.345 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:28.345 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:28.345 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:28.345 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:28.345 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:28.345 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:28.345 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:28.345 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:28.346 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:28.346 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:28.346 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:28.346 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:28.346 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:28.346 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:28.346 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:28.346 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:28.346 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:28.346 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:28.346 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:28.346 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:28.346 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:28.346 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:28.347 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:28.347 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:28.347 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:28.347 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:28.347 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:28.347 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:28.347 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:28.347 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:28.347 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:28.347 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.347 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:28.347 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:28.347 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:28.347 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:28.347 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:28.347 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T17:51:28.348 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.348 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 3, 2024-04-07T17:51:28.348 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:28.348 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.348 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:28.348 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:28.348 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:28.348 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:28.348 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T17:51:28.348 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.348 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 3, 2024-04-07T17:51:28.348 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:28.348 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:28.348 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T17:51:28.348 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:28.348 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:28.349 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:28.349 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:35.977675+0000", 2024-04-07T17:51:28.349 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:08.976003+0000", 2024-04-07T17:51:28.349 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:08.976003+0000", 2024-04-07T17:51:28.349 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:08.976353+0000", 2024-04-07T17:51:28.349 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:35.977675+0000", 2024-04-07T17:51:28.349 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:28.349 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:28.349 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:28.349 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 554, 2024-04-07T17:51:28.349 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:35.977675+0000", 2024-04-07T17:51:28.349 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:35.977675+0000", 2024-04-07T17:51:28.349 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:35.977675+0000", 2024-04-07T17:51:28.349 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:28.349 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:28.349 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:28.350 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:35.977675+0000", 2024-04-07T17:51:28.350 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:35.977675+0000", 2024-04-07T17:51:28.350 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:28.350 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:28.350 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:28.350 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:28.350 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 553, 2024-04-07T17:51:28.350 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:28.350 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:28.350 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:28.350 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:28.350 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:28.350 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:28.350 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:28.351 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:28.351 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.19", 2024-04-07T17:51:28.351 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:28.351 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:28.351 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:28.351 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4937, 2024-04-07T17:51:28.351 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:28.351 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T01:23:38.086617+0000", 2024-04-07T17:51:28.351 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:28.351 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:28.351 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:28.351 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:28.351 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:28.351 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:28.351 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:28.351 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:28.352 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:28.352 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:28.352 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:28.352 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:28.352 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:28.352 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:28.352 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:28.352 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:28.352 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:28.352 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:28.352 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:28.352 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:28.352 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:28.352 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:28.352 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:28.353 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:28.353 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:28.353 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:28.353 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:28.353 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:28.353 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:28.353 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:28.353 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:28.353 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:28.353 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:28.353 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:28.353 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:28.353 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:28.353 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:28.353 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:28.354 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:28.354 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:28.354 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:28.354 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:28.354 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:28.354 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.354 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:28.354 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:28.354 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:28.354 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:28.354 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:28.354 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T17:51:28.354 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.354 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 3, 2024-04-07T17:51:28.354 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:28.355 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.355 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:28.355 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:28.355 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:28.355 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:28.355 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T17:51:28.355 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.355 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 0, 2024-04-07T17:51:28.355 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:28.355 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:28.355 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:28.355 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:28.355 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:28.355 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:28.355 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:35.988644+0000", 2024-04-07T17:51:28.356 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:39.004366+0000", 2024-04-07T17:51:28.356 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:39.004366+0000", 2024-04-07T17:51:28.356 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:39.004810+0000", 2024-04-07T17:51:28.356 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:35.988644+0000", 2024-04-07T17:51:28.356 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:28.356 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:28.356 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:28.356 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:28.356 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:35.988644+0000", 2024-04-07T17:51:28.356 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:35.988644+0000", 2024-04-07T17:51:28.356 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:35.988644+0000", 2024-04-07T17:51:28.356 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:28.356 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:28.356 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:28.356 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:35.988644+0000", 2024-04-07T17:51:28.357 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:35.988644+0000", 2024-04-07T17:51:28.357 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:28.357 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:28.357 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:28.357 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:28.357 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:28.357 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:28.357 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:28.357 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:28.357 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:28.357 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:28.357 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:28.357 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:28.357 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:28.357 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.1b", 2024-04-07T17:51:28.357 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:28.358 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:28.358 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:28.358 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4992, 2024-04-07T17:51:28.358 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:28.358 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T19:31:04.334052+0000", 2024-04-07T17:51:28.358 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:28.358 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:28.358 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:28.358 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:28.358 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:28.358 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:28.358 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:28.358 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:28.358 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:28.359 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:28.359 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:28.359 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:28.359 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:28.359 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:28.359 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:28.359 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:28.359 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:28.359 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:28.359 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:28.359 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:28.359 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:28.359 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:28.359 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:28.359 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:28.359 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:28.360 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:28.360 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:28.360 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:28.360 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:28.360 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:28.360 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:28.360 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:28.360 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:28.360 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:28.360 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:28.360 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:28.360 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:28.360 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:28.360 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:28.360 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:28.361 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:28.361 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:28.361 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:28.361 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.361 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:28.361 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:28.361 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:28.361 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:28.361 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:28.361 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T17:51:28.361 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.361 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 0, 2024-04-07T17:51:28.361 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:28.361 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.361 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:28.361 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:28.362 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:28.362 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:28.362 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T17:51:28.362 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.362 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 6, 2024-04-07T17:51:28.362 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:28.362 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:28.362 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T17:51:28.362 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:28.362 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:28.362 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:28.362 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.372918+0000", 2024-04-07T17:51:28.362 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:38.960251+0000", 2024-04-07T17:51:28.362 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:38.960251+0000", 2024-04-07T17:51:28.362 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:38.960896+0000", 2024-04-07T17:51:28.363 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.372918+0000", 2024-04-07T17:51:28.363 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:28.363 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:28.363 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:28.363 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:28.363 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.372918+0000", 2024-04-07T17:51:28.363 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.372918+0000", 2024-04-07T17:51:28.363 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.372918+0000", 2024-04-07T17:51:28.363 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:28.363 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:28.363 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:28.363 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.372918+0000", 2024-04-07T17:51:28.363 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.372918+0000", 2024-04-07T17:51:28.363 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:28.363 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 2, 2024-04-07T17:51:28.363 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:28.364 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:28.364 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:28.364 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:28.364 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:28.364 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:28.364 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:28.364 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 2, 2024-04-07T17:51:28.364 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:28.364 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:28.364 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:28.364 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.1a", 2024-04-07T17:51:28.364 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:28.364 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:28.364 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:28.364 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4839, 2024-04-07T17:51:28.364 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:28.365 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T18:24:54.929614+0000", 2024-04-07T17:51:28.365 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:28.365 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:28.365 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:28.365 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:28.365 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:28.365 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:28.365 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:28.365 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:28.365 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:28.365 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:28.365 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:28.365 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:28.365 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:28.366 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:28.366 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:28.366 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:28.366 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:28.366 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:28.366 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:28.366 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:28.366 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:28.366 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:28.366 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:28.366 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:28.366 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:28.366 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:28.366 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:28.367 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:28.367 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:28.367 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:28.367 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:28.367 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:28.367 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:28.367 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:28.367 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:28.367 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:28.367 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:28.367 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:28.367 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:28.367 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:28.367 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:28.367 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:28.367 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:28.368 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.368 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:28.368 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:28.368 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:28.368 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:28.368 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:28.368 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T17:51:28.368 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.368 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 6, 2024-04-07T17:51:28.368 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "623'2" 2024-04-07T17:51:28.368 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.368 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:28.368 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:28.368 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:28.368 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:28.368 INFO:tasks.workunit.client.0.smithi100.stdout: 2 2024-04-07T17:51:28.368 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.369 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 4, 2024-04-07T17:51:28.369 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:28.369 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:28.369 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:28.369 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:28.369 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:28.369 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:28.369 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:35.981965+0000", 2024-04-07T17:51:28.369 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:24:13.936610+0000", 2024-04-07T17:51:28.369 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:24:13.936610+0000", 2024-04-07T17:51:28.369 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:24:13.937948+0000", 2024-04-07T17:51:28.369 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:35.981965+0000", 2024-04-07T17:51:28.369 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:28.369 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:28.369 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:28.369 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 500, 2024-04-07T17:51:28.369 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:35.981965+0000", 2024-04-07T17:51:28.370 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:35.981965+0000", 2024-04-07T17:51:28.370 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:35.981965+0000", 2024-04-07T17:51:28.370 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:28.370 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:28.370 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:28.370 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:35.981965+0000", 2024-04-07T17:51:28.370 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:35.981965+0000", 2024-04-07T17:51:28.370 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:28.370 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:28.370 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:28.370 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:28.370 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 499, 2024-04-07T17:51:28.370 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:28.370 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:28.370 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:28.370 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:28.371 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:28.371 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:28.371 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:28.371 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:28.371 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.1a", 2024-04-07T17:51:28.371 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:28.371 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:28.371 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:28.371 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4930, 2024-04-07T17:51:28.371 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:28.371 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T02:25:15.032887+0000", 2024-04-07T17:51:28.371 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:28.371 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:28.371 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:28.371 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:28.371 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:28.371 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:28.372 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:28.372 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:28.372 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:28.372 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:28.372 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:28.372 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:28.372 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:28.372 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:28.372 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:28.372 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:28.372 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:28.372 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:28.372 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:28.372 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:28.372 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:28.372 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:28.372 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:28.373 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:28.373 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:28.373 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:28.373 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:28.373 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:28.373 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:28.373 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:28.373 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:28.373 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:28.373 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:28.373 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:28.373 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:28.373 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:28.373 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:28.373 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:28.373 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:28.373 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:28.373 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:28.374 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:28.374 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:28.374 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.374 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:28.374 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:28.374 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:28.374 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:28.374 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:28.374 INFO:tasks.workunit.client.0.smithi100.stdout: 2 2024-04-07T17:51:28.374 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.374 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 4, 2024-04-07T17:51:28.374 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:28.374 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.375 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:28.375 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:28.375 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:28.375 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:28.375 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T17:51:28.375 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.375 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 3, 2024-04-07T17:51:28.375 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:28.375 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:28.375 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T17:51:28.375 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:28.375 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:28.375 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:28.375 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:35.977526+0000", 2024-04-07T17:51:28.375 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:38.997910+0000", 2024-04-07T17:51:28.375 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:38.997910+0000", 2024-04-07T17:51:28.375 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:38.998434+0000", 2024-04-07T17:51:28.376 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:35.977526+0000", 2024-04-07T17:51:28.376 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:28.376 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:28.376 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:28.376 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:28.376 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:35.977526+0000", 2024-04-07T17:51:28.376 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:35.977526+0000", 2024-04-07T17:51:28.376 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:35.977526+0000", 2024-04-07T17:51:28.376 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:28.376 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:28.376 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:28.376 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:35.977526+0000", 2024-04-07T17:51:28.376 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:35.977526+0000", 2024-04-07T17:51:28.376 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:28.376 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 2, 2024-04-07T17:51:28.376 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:28.376 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:28.377 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:28.377 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:28.377 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:28.377 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:28.377 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:28.377 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 2, 2024-04-07T17:51:28.377 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:28.377 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:28.377 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:28.377 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.1b", 2024-04-07T17:51:28.377 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:28.377 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:28.377 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:28.377 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4932, 2024-04-07T17:51:28.377 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:28.377 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T02:18:46.594583+0000", 2024-04-07T17:51:28.377 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:28.378 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:28.378 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:28.378 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:28.378 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:28.378 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:28.378 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:28.378 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:28.378 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:28.378 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:28.378 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:28.378 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:28.378 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:28.378 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:28.378 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:28.378 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:28.379 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:28.379 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:28.379 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:28.379 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:28.379 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:28.379 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:28.379 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:28.379 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:28.379 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:28.379 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:28.379 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:28.379 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:28.379 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:28.379 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:28.379 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:28.379 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:28.379 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:28.380 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:28.380 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:28.380 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:28.380 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:28.380 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:28.380 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:28.380 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:28.380 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:28.380 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:28.380 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:28.380 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.380 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:28.380 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:28.380 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:28.380 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:28.380 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:28.381 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T17:51:28.381 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.381 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 3, 2024-04-07T17:51:28.381 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "623'2" 2024-04-07T17:51:28.381 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.381 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:28.381 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:28.381 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:28.381 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:28.381 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T17:51:28.381 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.381 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 5, 2024-04-07T17:51:28.381 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:28.381 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:28.381 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:28.381 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:28.381 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:28.382 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:28.382 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.365302+0000", 2024-04-07T17:51:28.382 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:08.974827+0000", 2024-04-07T17:51:28.382 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:08.974827+0000", 2024-04-07T17:51:28.382 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:08.975369+0000", 2024-04-07T17:51:28.382 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.365302+0000", 2024-04-07T17:51:28.382 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:28.382 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:28.382 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:28.382 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 554, 2024-04-07T17:51:28.382 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.365302+0000", 2024-04-07T17:51:28.382 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.365302+0000", 2024-04-07T17:51:28.382 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.365302+0000", 2024-04-07T17:51:28.382 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:28.383 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:28.383 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:28.383 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.365302+0000", 2024-04-07T17:51:28.383 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.365302+0000", 2024-04-07T17:51:28.383 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:28.383 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:28.383 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:28.383 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:28.383 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 553, 2024-04-07T17:51:28.383 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:28.383 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:28.383 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:28.383 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:28.383 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:28.383 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:28.383 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:28.383 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:28.383 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.1d", 2024-04-07T17:51:28.384 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:28.384 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:28.384 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:28.384 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4868, 2024-04-07T17:51:28.384 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:28.384 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T19:16:10.778146+0000", 2024-04-07T17:51:28.384 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:28.384 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:28.384 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:28.384 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:28.384 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:28.384 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:28.384 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:28.384 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:28.384 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:28.384 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:28.385 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:28.385 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:28.385 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:28.385 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:28.385 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:28.385 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:28.385 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:28.385 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:28.385 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:28.385 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:28.385 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:28.385 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:28.385 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:28.385 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:28.385 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:28.385 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:28.386 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:28.386 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:28.386 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:28.386 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:28.386 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:28.386 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:28.386 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:28.386 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:28.386 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:28.386 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:28.386 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:28.386 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:28.386 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:28.386 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:28.386 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:28.387 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:28.387 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:28.387 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.387 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:28.387 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:28.387 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:28.387 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:28.387 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:28.387 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T17:51:28.387 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.387 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 5, 2024-04-07T17:51:28.387 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:28.387 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.387 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:28.387 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:28.388 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:28.388 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:28.388 INFO:tasks.workunit.client.0.smithi100.stdout: 2 2024-04-07T17:51:28.388 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.388 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 7, 2024-04-07T17:51:28.388 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:28.388 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:28.388 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T17:51:28.388 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:28.388 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:28.388 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:28.388 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.364942+0000", 2024-04-07T17:51:28.388 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:38.975920+0000", 2024-04-07T17:51:28.388 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:38.975920+0000", 2024-04-07T17:51:28.388 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:38.991127+0000", 2024-04-07T17:51:28.388 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.364942+0000", 2024-04-07T17:51:28.389 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:28.389 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:28.389 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:28.389 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:28.389 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.364942+0000", 2024-04-07T17:51:28.389 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.364942+0000", 2024-04-07T17:51:28.389 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.364942+0000", 2024-04-07T17:51:28.389 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:28.389 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:28.389 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:28.389 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.364942+0000", 2024-04-07T17:51:28.389 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.364942+0000", 2024-04-07T17:51:28.389 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:28.389 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 2, 2024-04-07T17:51:28.389 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:28.389 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:28.389 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:28.389 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:28.390 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:28.390 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:28.390 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:28.390 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 2, 2024-04-07T17:51:28.390 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:28.390 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:28.390 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:28.390 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.1c", 2024-04-07T17:51:28.390 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:28.390 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:28.390 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:28.390 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4819, 2024-04-07T17:51:28.390 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:28.390 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T22:48:18.986526+0000", 2024-04-07T17:51:28.391 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:28.391 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:28.391 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:28.391 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:28.391 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:28.391 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:28.391 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:28.391 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:28.391 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:28.391 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:28.391 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:28.391 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:28.391 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:28.391 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:28.391 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:28.392 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:28.392 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:28.392 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:28.392 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:28.392 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 3, 2024-04-07T17:51:28.392 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 1, 2024-04-07T17:51:28.392 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:28.392 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 1, 2024-04-07T17:51:28.392 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:28.392 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:28.392 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:28.392 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:28.392 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:28.392 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 1, 2024-04-07T17:51:28.392 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:28.393 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:28.393 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:28.393 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:28.393 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:28.393 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:28.393 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:28.393 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 6, 2024-04-07T17:51:28.393 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 6, 2024-04-07T17:51:28.393 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:28.393 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:28.393 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:28.393 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 3, 2024-04-07T17:51:28.393 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:28.393 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.393 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:28.394 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:28.394 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:28.394 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:28.394 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:28.394 INFO:tasks.workunit.client.0.smithi100.stdout: 2 2024-04-07T17:51:28.394 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.394 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 7, 2024-04-07T17:51:28.394 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "61'2" 2024-04-07T17:51:28.394 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.394 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:28.394 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:28.394 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:28.394 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:28.394 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T17:51:28.394 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.395 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 5, 2024-04-07T17:51:28.395 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:28.395 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:28.395 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:28.395 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:28.395 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:28.395 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:28.395 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.367080+0000", 2024-04-07T17:51:28.395 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:24:41.542687+0000", 2024-04-07T17:51:28.395 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:24:41.542687+0000", 2024-04-07T17:51:28.395 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:24:41.543356+0000", 2024-04-07T17:51:28.395 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.367080+0000", 2024-04-07T17:51:28.395 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:28.395 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:28.395 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:28.396 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 527, 2024-04-07T17:51:28.396 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.367080+0000", 2024-04-07T17:51:28.396 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.367080+0000", 2024-04-07T17:51:28.396 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.367080+0000", 2024-04-07T17:51:28.396 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:28.396 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:28.396 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:28.396 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.367080+0000", 2024-04-07T17:51:28.396 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.367080+0000", 2024-04-07T17:51:28.396 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:28.396 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:28.396 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:28.396 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:28.396 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 526, 2024-04-07T17:51:28.397 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:28.397 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:28.397 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:28.397 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:28.397 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:28.397 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:28.397 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:28.397 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:28.397 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.1c", 2024-04-07T17:51:28.397 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:28.397 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:28.397 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:28.397 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4865, 2024-04-07T17:51:28.397 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:28.397 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T21:43:40.340691+0000", 2024-04-07T17:51:28.397 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:28.398 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:28.398 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:28.398 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:28.398 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:28.398 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:28.398 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:28.398 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:28.398 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:28.398 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:28.398 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:28.398 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:28.398 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:28.398 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:28.398 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:28.399 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:28.399 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:28.399 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:28.399 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:28.399 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:28.399 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:28.399 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:28.399 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:28.399 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:28.399 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:28.399 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:28.399 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:28.399 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:28.399 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:28.399 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:28.399 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:28.400 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:28.400 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:28.400 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:28.400 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:28.400 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:28.400 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:28.400 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:28.400 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:28.400 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:28.400 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:28.400 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:28.400 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:28.400 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.400 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:28.400 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:28.401 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:28.401 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:28.401 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:28.401 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T17:51:28.401 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.401 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 5, 2024-04-07T17:51:28.401 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:28.401 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.401 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:28.401 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:28.401 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:28.401 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:28.401 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T17:51:28.401 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.401 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 7, 2024-04-07T17:51:28.402 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:28.402 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:28.402 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T17:51:28.402 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:28.402 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:28.402 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:28.402 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.373509+0000", 2024-04-07T17:51:28.402 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:39.001898+0000", 2024-04-07T17:51:28.402 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:39.001898+0000", 2024-04-07T17:51:28.402 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:39.004508+0000", 2024-04-07T17:51:28.402 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.373509+0000", 2024-04-07T17:51:28.402 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:28.402 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:28.402 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:28.403 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:28.403 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.373509+0000", 2024-04-07T17:51:28.403 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.373509+0000", 2024-04-07T17:51:28.403 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.373509+0000", 2024-04-07T17:51:28.403 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:28.403 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:28.403 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:28.403 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.373509+0000", 2024-04-07T17:51:28.403 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.373509+0000", 2024-04-07T17:51:28.403 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:28.403 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 2, 2024-04-07T17:51:28.403 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:28.403 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:28.403 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:28.403 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:28.404 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:28.404 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:28.404 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:28.404 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 2, 2024-04-07T17:51:28.404 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:28.404 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:28.404 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:28.404 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.1d", 2024-04-07T17:51:28.404 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:28.404 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:28.404 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:28.404 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4804, 2024-04-07T17:51:28.404 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:28.404 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T01:47:40.701645+0000", 2024-04-07T17:51:28.404 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:28.404 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:28.405 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:28.405 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:28.405 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:28.405 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:28.405 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:28.405 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:28.405 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:28.405 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:28.405 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:28.405 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:28.405 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:28.405 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:28.405 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:28.405 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:28.405 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:28.406 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:28.406 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:28.406 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 3, 2024-04-07T17:51:28.406 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 1, 2024-04-07T17:51:28.406 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:28.406 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 1, 2024-04-07T17:51:28.406 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:28.406 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:28.406 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:28.406 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:28.406 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:28.406 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 1, 2024-04-07T17:51:28.406 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:28.406 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:28.407 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:28.407 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:28.407 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:28.407 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:28.407 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:28.407 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 6, 2024-04-07T17:51:28.407 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 6, 2024-04-07T17:51:28.407 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:28.407 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:28.407 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:28.407 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 3, 2024-04-07T17:51:28.407 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:28.407 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.407 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:28.407 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:28.408 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:28.408 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:28.408 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:28.408 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T17:51:28.408 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.408 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 7, 2024-04-07T17:51:28.408 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "61'2" 2024-04-07T17:51:28.408 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.408 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:28.408 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:28.408 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:28.408 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:28.408 INFO:tasks.workunit.client.0.smithi100.stdout: 2 2024-04-07T17:51:28.408 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.408 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 0, 2024-04-07T17:51:28.409 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:28.409 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:28.409 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:28.409 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:28.409 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:28.409 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:28.409 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:35.983257+0000", 2024-04-07T17:51:28.409 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:24:41.564700+0000", 2024-04-07T17:51:28.409 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:24:41.564700+0000", 2024-04-07T17:51:28.409 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:24:41.565093+0000", 2024-04-07T17:51:28.409 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:35.983257+0000", 2024-04-07T17:51:28.409 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:28.409 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:28.409 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:28.409 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 527, 2024-04-07T17:51:28.409 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:35.983257+0000", 2024-04-07T17:51:28.410 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:35.983257+0000", 2024-04-07T17:51:28.410 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:35.983257+0000", 2024-04-07T17:51:28.410 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:28.410 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:28.410 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:28.410 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:35.983257+0000", 2024-04-07T17:51:28.410 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:35.983257+0000", 2024-04-07T17:51:28.410 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:28.410 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:28.410 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:28.410 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:28.410 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 526, 2024-04-07T17:51:28.410 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:28.410 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:28.410 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:28.411 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:28.411 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:28.411 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:28.411 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:28.411 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:28.411 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.1f", 2024-04-07T17:51:28.411 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:28.411 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:28.411 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:28.411 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 5000, 2024-04-07T17:51:28.411 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:28.411 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T01:20:29.103128+0000", 2024-04-07T17:51:28.411 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:28.411 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:28.411 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:28.411 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:28.412 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:28.412 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:28.412 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:28.412 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:28.412 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:28.412 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:28.412 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:28.412 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:28.412 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:28.412 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:28.412 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:28.412 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:28.412 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:28.412 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:28.412 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:28.412 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:28.412 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:28.413 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:28.413 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:28.413 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:28.413 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:28.413 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:28.413 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:28.413 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:28.413 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:28.413 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:28.413 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:28.413 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:28.413 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:28.413 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:28.413 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:28.413 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:28.413 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:28.414 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:28.414 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:28.414 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:28.414 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:28.414 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:28.414 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:28.414 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.414 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:28.414 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:28.414 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:28.414 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:28.414 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:28.414 INFO:tasks.workunit.client.0.smithi100.stdout: 2 2024-04-07T17:51:28.414 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.414 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 0, 2024-04-07T17:51:28.414 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:28.415 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.415 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:28.415 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:28.415 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:28.415 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:28.415 INFO:tasks.workunit.client.0.smithi100.stdout: 5 2024-04-07T17:51:28.415 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.415 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 3, 2024-04-07T17:51:28.415 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:28.415 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:28.415 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T17:51:28.415 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:28.415 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:28.415 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:28.415 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:35.982794+0000", 2024-04-07T17:51:28.415 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:24:41.546310+0000", 2024-04-07T17:51:28.416 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:24:41.546310+0000", 2024-04-07T17:51:28.416 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:24:41.546952+0000", 2024-04-07T17:51:28.416 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:35.982794+0000", 2024-04-07T17:51:28.416 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:28.416 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:28.416 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:28.416 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 527, 2024-04-07T17:51:28.416 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:35.982794+0000", 2024-04-07T17:51:28.416 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:35.982794+0000", 2024-04-07T17:51:28.416 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:35.982794+0000", 2024-04-07T17:51:28.416 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:28.416 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:28.416 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:28.416 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:35.982794+0000", 2024-04-07T17:51:28.416 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:35.982794+0000", 2024-04-07T17:51:28.417 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:28.417 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 2, 2024-04-07T17:51:28.417 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:28.417 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:28.417 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 526, 2024-04-07T17:51:28.417 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:28.417 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:28.417 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:28.417 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:28.417 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 2, 2024-04-07T17:51:28.417 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:28.417 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:28.417 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:28.417 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.1e", 2024-04-07T17:51:28.417 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:28.418 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:28.418 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:28.418 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4947, 2024-04-07T17:51:28.418 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:28.418 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T21:45:55.182027+0000", 2024-04-07T17:51:28.418 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:28.418 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:28.418 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:28.418 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:28.418 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:28.418 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:28.418 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:28.418 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:28.418 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:28.418 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:28.419 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:28.419 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:28.419 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:28.419 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:28.419 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:28.419 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:28.419 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:28.419 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:28.419 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:28.419 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:28.419 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:28.419 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:28.419 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:28.419 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:28.419 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:28.420 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:28.420 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:28.420 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:28.420 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:28.420 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:28.420 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:28.420 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:28.420 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:28.420 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:28.420 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:28.420 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:28.420 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:28.420 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:28.420 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:28.420 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:28.421 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:28.421 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:28.421 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:28.421 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.421 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:28.421 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:28.421 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:28.421 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:28.421 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:28.421 INFO:tasks.workunit.client.0.smithi100.stdout: 5 2024-04-07T17:51:28.421 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.421 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 3, 2024-04-07T17:51:28.421 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "623'2" 2024-04-07T17:51:28.421 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.421 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:28.422 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:28.422 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:28.422 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:28.422 INFO:tasks.workunit.client.0.smithi100.stdout: 2 2024-04-07T17:51:28.422 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.422 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 3, 2024-04-07T17:51:28.422 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:28.422 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:28.422 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:28.422 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:28.422 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:28.422 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:28.422 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:35.982633+0000", 2024-04-07T17:51:28.422 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:08.977988+0000", 2024-04-07T17:51:28.423 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:08.977988+0000", 2024-04-07T17:51:28.423 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:08.978435+0000", 2024-04-07T17:51:28.423 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:35.982633+0000", 2024-04-07T17:51:28.423 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:28.423 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:28.423 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:28.423 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 554, 2024-04-07T17:51:28.423 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:35.982633+0000", 2024-04-07T17:51:28.423 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:35.982633+0000", 2024-04-07T17:51:28.423 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:35.982633+0000", 2024-04-07T17:51:28.423 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:28.423 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:28.423 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:28.423 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:35.982633+0000", 2024-04-07T17:51:28.424 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:35.982633+0000", 2024-04-07T17:51:28.424 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:28.424 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:28.424 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:28.424 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:28.424 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 553, 2024-04-07T17:51:28.424 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:28.424 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:28.424 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:28.424 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:28.424 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:28.424 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:28.424 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:28.424 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:28.424 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.1e", 2024-04-07T17:51:28.424 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:28.425 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:28.425 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:28.425 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4957, 2024-04-07T17:51:28.425 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:28.425 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T21:46:59.340658+0000", 2024-04-07T17:51:28.425 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:28.425 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:28.425 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:28.425 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:28.425 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:28.425 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:28.425 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:28.425 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:28.425 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:28.425 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:28.426 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:28.426 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:28.426 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:28.426 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:28.426 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:28.426 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:28.426 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:28.426 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:28.426 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:28.426 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:28.426 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:28.426 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:28.426 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:28.429 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:28.429 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:28.429 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:28.429 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:28.429 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:28.429 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:28.429 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:28.429 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:28.429 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:28.429 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:28.430 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:28.430 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:28.430 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:28.430 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:28.430 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:28.430 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:28.430 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:28.430 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:28.430 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:28.430 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:28.430 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.430 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:28.430 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:28.430 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:28.431 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:28.431 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:28.431 INFO:tasks.workunit.client.0.smithi100.stdout: 2 2024-04-07T17:51:28.431 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.431 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 3, 2024-04-07T17:51:28.431 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:28.431 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.431 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:28.431 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:28.431 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:28.431 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:28.431 INFO:tasks.workunit.client.0.smithi100.stdout: 4 2024-04-07T17:51:28.431 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.431 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 0, 2024-04-07T17:51:28.431 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:28.431 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:28.431 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T17:51:28.432 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:28.432 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:28.432 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:28.432 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:35.984435+0000", 2024-04-07T17:51:28.432 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:38.997317+0000", 2024-04-07T17:51:28.432 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:38.997317+0000", 2024-04-07T17:51:28.432 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:38.997712+0000", 2024-04-07T17:51:28.432 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:35.984435+0000", 2024-04-07T17:51:28.432 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:28.432 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:28.432 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:28.432 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:28.432 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:35.984435+0000", 2024-04-07T17:51:28.432 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:35.984435+0000", 2024-04-07T17:51:28.432 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:35.984435+0000", 2024-04-07T17:51:28.432 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:28.432 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:28.433 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:28.433 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:35.984435+0000", 2024-04-07T17:51:28.433 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:35.984435+0000", 2024-04-07T17:51:28.433 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:28.433 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 5, 2024-04-07T17:51:28.433 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:28.433 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:28.433 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:28.433 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:28.433 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:28.433 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:28.433 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:28.433 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 5, 2024-04-07T17:51:28.433 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:28.433 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:28.433 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:28.434 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.1f", 2024-04-07T17:51:28.434 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:28.434 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:28.434 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:28.434 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 5008, 2024-04-07T17:51:28.434 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:28.434 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T18:23:18.184011+0000", 2024-04-07T17:51:28.434 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:28.434 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:28.434 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:28.434 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 582, 2024-04-07T17:51:28.434 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:28.434 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:28.434 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:28.434 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:28.435 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:28.435 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:28.435 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:28.435 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:28.435 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:28.435 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:28.435 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:28.435 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:28.435 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:28.435 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:28.435 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:28.435 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 6, 2024-04-07T17:51:28.435 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 2, 2024-04-07T17:51:28.436 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:28.436 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 2, 2024-04-07T17:51:28.436 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:28.436 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:28.436 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:28.436 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:28.436 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:28.436 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 1, 2024-04-07T17:51:28.436 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:28.436 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:28.436 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:28.436 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:28.436 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:28.436 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:28.436 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:28.437 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 4, 2024-04-07T17:51:28.437 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 4, 2024-04-07T17:51:28.437 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:28.437 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:28.437 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:28.437 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 4, 2024-04-07T17:51:28.437 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 1 2024-04-07T17:51:28.437 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.437 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:28.437 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:28.437 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:28.437 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:28.437 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:28.437 INFO:tasks.workunit.client.0.smithi100.stdout: 4 2024-04-07T17:51:28.437 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.437 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 0, 2024-04-07T17:51:28.437 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "623'5" 2024-04-07T17:51:28.438 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.438 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:28.438 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:28.438 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:28.438 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T17:51:28.438 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T17:51:28.438 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.438 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 7, 2024-04-07T17:51:28.438 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:28.438 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:28.438 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:28.438 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:28.438 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:28.438 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:28.438 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.373284+0000", 2024-04-07T17:51:28.439 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:38.996053+0000", 2024-04-07T17:51:28.439 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:38.996053+0000", 2024-04-07T17:51:28.439 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:39.002704+0000", 2024-04-07T17:51:28.439 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.373284+0000", 2024-04-07T17:51:28.439 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:28.439 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:28.439 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:28.439 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:28.439 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.373284+0000", 2024-04-07T17:51:28.439 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.373284+0000", 2024-04-07T17:51:28.439 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.373284+0000", 2024-04-07T17:51:28.439 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:28.439 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:28.439 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:28.439 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.373284+0000", 2024-04-07T17:51:28.439 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.373284+0000", 2024-04-07T17:51:28.440 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:28.440 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:28.440 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:28.440 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:28.440 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:28.440 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:28.440 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:28.440 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:28.440 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:28.440 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:28.440 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:28.440 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:28.440 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:28.440 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.2d", 2024-04-07T17:51:28.440 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:28.440 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:28.440 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:28.441 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4802, 2024-04-07T17:51:28.441 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:28.441 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T17:45:19.934847+0000", 2024-04-07T17:51:28.441 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:28.441 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:28.441 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:28.441 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:28.441 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:28.441 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:28.441 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:28.441 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:28.441 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:28.441 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:28.441 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:28.441 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:28.441 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:28.441 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:28.442 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:28.442 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:28.442 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:28.442 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:28.442 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:28.442 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:28.442 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:28.442 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:28.442 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:28.442 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:28.442 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:28.442 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:28.442 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:28.442 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:28.443 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:28.443 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:28.443 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:28.443 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:28.443 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:28.443 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:28.443 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:28.443 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:28.443 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:28.443 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:28.443 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:28.443 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:28.443 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:28.443 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:28.443 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:28.443 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.444 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:28.444 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:28.444 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:28.444 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:28.444 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T17:51:28.444 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T17:51:28.444 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.444 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 7, 2024-04-07T17:51:28.444 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:28.444 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.444 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:28.444 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:28.444 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T17:51:28.444 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:28.445 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T17:51:28.445 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.445 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 2, 2024-04-07T17:51:28.445 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:28.445 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:28.445 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:28.445 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:28.445 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:28.445 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:28.445 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.359468+0000", 2024-04-07T17:51:28.445 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:38.967369+0000", 2024-04-07T17:51:28.445 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:38.967369+0000", 2024-04-07T17:51:28.445 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:38.967737+0000", 2024-04-07T17:51:28.445 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.359468+0000", 2024-04-07T17:51:28.445 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:28.446 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:28.446 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:28.446 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:28.446 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.359468+0000", 2024-04-07T17:51:28.446 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.359468+0000", 2024-04-07T17:51:28.446 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.359468+0000", 2024-04-07T17:51:28.446 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:28.446 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:28.446 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:28.446 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.359468+0000", 2024-04-07T17:51:28.446 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.359468+0000", 2024-04-07T17:51:28.446 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:28.446 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:28.446 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:28.447 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:28.447 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:28.447 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:28.447 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:28.447 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:28.447 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:28.447 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:28.447 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:28.447 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:28.447 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:28.447 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.2e", 2024-04-07T17:51:28.447 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:28.447 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:28.447 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:28.447 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4964, 2024-04-07T17:51:28.447 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:28.448 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T22:15:36.602436+0000", 2024-04-07T17:51:28.448 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:28.448 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:28.448 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:28.448 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:28.448 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:28.448 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:28.448 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:28.448 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:28.448 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:28.448 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:28.448 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:28.448 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:28.448 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:28.448 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:28.448 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:28.448 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:28.449 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:28.449 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:28.449 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:28.449 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:28.449 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:28.449 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:28.449 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:28.449 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:28.449 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:28.449 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:28.449 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:28.449 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:28.449 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:28.449 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:28.449 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:28.450 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:28.450 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:28.450 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:28.450 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:28.450 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:28.450 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:28.450 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:28.450 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:28.450 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:28.450 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:28.450 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:28.450 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:28.450 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.450 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:28.450 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:28.451 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:28.451 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T17:51:28.451 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:28.451 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T17:51:28.451 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.451 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 2, 2024-04-07T17:51:28.451 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:28.451 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.451 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:28.451 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:28.451 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:28.451 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:28.451 INFO:tasks.workunit.client.0.smithi100.stdout: 4 2024-04-07T17:51:28.451 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.451 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 5, 2024-04-07T17:51:28.451 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:28.452 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:28.452 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:28.452 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:28.452 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:28.452 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:28.452 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.361813+0000", 2024-04-07T17:51:28.452 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:38.971115+0000", 2024-04-07T17:51:28.452 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:38.971115+0000", 2024-04-07T17:51:28.452 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:38.971858+0000", 2024-04-07T17:51:28.452 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.361813+0000", 2024-04-07T17:51:28.452 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:28.452 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:28.452 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:28.452 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:28.452 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.361813+0000", 2024-04-07T17:51:28.452 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.361813+0000", 2024-04-07T17:51:28.453 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.361813+0000", 2024-04-07T17:51:28.453 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:28.453 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:28.453 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:28.453 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.361813+0000", 2024-04-07T17:51:28.453 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.361813+0000", 2024-04-07T17:51:28.453 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:28.453 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:28.453 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:28.453 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:28.453 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:28.453 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:28.453 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:28.453 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:28.453 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:28.453 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:28.453 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:28.454 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:28.454 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:28.454 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.2f", 2024-04-07T17:51:28.454 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:28.454 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:28.454 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:28.454 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4863, 2024-04-07T17:51:28.454 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:28.454 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T22:01:10.276598+0000", 2024-04-07T17:51:28.454 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:28.454 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:28.454 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:28.454 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:28.454 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:28.454 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:28.455 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:28.455 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:28.455 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:28.455 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:28.455 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:28.455 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:28.455 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:28.455 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:28.455 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:28.455 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:28.455 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:28.455 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:28.455 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:28.455 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:28.455 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:28.456 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:28.456 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:28.456 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:28.456 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:28.456 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:28.456 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:28.456 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:28.456 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:28.456 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:28.456 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:28.456 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:28.456 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:28.456 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:28.456 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:28.456 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:28.457 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:28.457 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:28.457 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:28.457 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:28.457 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:28.457 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:28.457 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:28.457 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.457 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:28.457 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:28.457 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:28.457 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:28.457 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:28.457 INFO:tasks.workunit.client.0.smithi100.stdout: 4 2024-04-07T17:51:28.457 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.457 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 5, 2024-04-07T17:51:28.457 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:28.458 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.458 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:28.458 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:28.458 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:28.458 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:28.458 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T17:51:28.458 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.458 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 5, 2024-04-07T17:51:28.458 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:28.458 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:28.458 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:28.458 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:28.458 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:28.458 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:28.458 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.364596+0000", 2024-04-07T17:51:28.459 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:39.005541+0000", 2024-04-07T17:51:28.459 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:39.005541+0000", 2024-04-07T17:51:28.459 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:39.006138+0000", 2024-04-07T17:51:28.459 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.364596+0000", 2024-04-07T17:51:28.459 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:28.459 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:28.459 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:28.459 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:28.459 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.364596+0000", 2024-04-07T17:51:28.459 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.364596+0000", 2024-04-07T17:51:28.459 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.364596+0000", 2024-04-07T17:51:28.459 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:28.459 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:28.459 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:28.459 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.364596+0000", 2024-04-07T17:51:28.459 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.364596+0000", 2024-04-07T17:51:28.460 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:28.460 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:28.460 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:28.460 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:28.460 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:28.460 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:28.460 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:28.460 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:28.460 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:28.460 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:28.460 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:28.460 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:28.460 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:28.460 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.30", 2024-04-07T17:51:28.460 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:28.461 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:28.461 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:28.461 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4876, 2024-04-07T17:51:28.461 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:28.461 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T20:40:58.093262+0000", 2024-04-07T17:51:28.461 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:28.461 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:28.461 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:28.461 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:28.461 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:28.461 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:28.461 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:28.461 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:28.461 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:28.461 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:28.461 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:28.461 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:28.461 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:28.462 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:28.462 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:28.462 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:28.462 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:28.462 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:28.462 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:28.462 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:28.462 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:28.462 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:28.462 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:28.462 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:28.462 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:28.462 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:28.462 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:28.462 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:28.462 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:28.463 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:28.463 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:28.463 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:28.463 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:28.463 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:28.463 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:28.463 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:28.463 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:28.463 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:28.463 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:28.463 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:28.463 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:28.463 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:28.463 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:28.463 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.463 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:28.463 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:28.464 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:28.464 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:28.464 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:28.464 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T17:51:28.464 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.464 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 5, 2024-04-07T17:51:28.464 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:28.464 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.464 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:28.464 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:28.464 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:28.464 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:28.464 INFO:tasks.workunit.client.0.smithi100.stdout: 5 2024-04-07T17:51:28.464 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.464 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 1, 2024-04-07T17:51:28.464 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:28.464 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:28.465 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:28.465 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:28.465 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:28.465 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:28.465 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.331261+0000", 2024-04-07T17:51:28.465 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:24:41.533626+0000", 2024-04-07T17:51:28.465 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:24:41.533626+0000", 2024-04-07T17:51:28.465 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:24:41.534090+0000", 2024-04-07T17:51:28.465 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.331261+0000", 2024-04-07T17:51:28.465 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:28.465 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:28.465 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:28.465 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 527, 2024-04-07T17:51:28.465 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.331261+0000", 2024-04-07T17:51:28.465 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.331261+0000", 2024-04-07T17:51:28.465 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.331261+0000", 2024-04-07T17:51:28.465 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:28.466 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:28.466 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:28.466 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.331261+0000", 2024-04-07T17:51:28.466 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.331261+0000", 2024-04-07T17:51:28.466 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:28.466 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:28.466 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:28.466 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:28.466 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 526, 2024-04-07T17:51:28.466 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:28.466 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:28.466 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:28.466 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:28.466 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:28.467 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:28.467 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:28.467 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:28.467 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.31", 2024-04-07T17:51:28.467 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:28.467 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:28.467 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:28.467 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4979, 2024-04-07T17:51:28.467 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:28.467 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T16:42:51.811301+0000", 2024-04-07T17:51:28.467 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:28.467 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:28.467 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:28.467 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:28.467 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:28.467 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:28.468 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:28.468 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:28.468 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:28.468 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:28.468 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:28.468 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:28.468 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:28.468 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:28.468 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:28.468 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:28.468 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:28.468 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:28.468 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:28.468 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:28.468 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:28.468 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:28.468 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:28.469 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:28.469 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:28.469 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:28.469 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:28.469 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:28.469 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:28.469 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:28.469 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:28.469 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:28.469 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:28.469 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:28.469 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:28.469 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:28.469 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:28.469 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:28.469 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:28.470 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:28.470 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:28.470 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:28.470 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:28.470 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.470 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:28.470 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:28.470 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:28.470 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:28.470 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:28.470 INFO:tasks.workunit.client.0.smithi100.stdout: 5 2024-04-07T17:51:28.470 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.470 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 1, 2024-04-07T17:51:28.470 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:28.470 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.471 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:28.471 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:28.471 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:28.471 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:28.471 INFO:tasks.workunit.client.0.smithi100.stdout: 5 2024-04-07T17:51:28.471 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.471 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 0, 2024-04-07T17:51:28.471 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:28.471 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:28.471 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:28.471 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:28.471 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:28.471 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:28.471 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:35.984431+0000", 2024-04-07T17:51:28.471 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:24:41.538233+0000", 2024-04-07T17:51:28.471 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:24:41.538233+0000", 2024-04-07T17:51:28.471 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:24:41.538718+0000", 2024-04-07T17:51:28.472 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:35.984431+0000", 2024-04-07T17:51:28.472 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:28.472 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:28.472 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:28.472 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 527, 2024-04-07T17:51:28.472 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:35.984431+0000", 2024-04-07T17:51:28.472 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:35.984431+0000", 2024-04-07T17:51:28.472 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:35.984431+0000", 2024-04-07T17:51:28.472 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:28.472 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:28.472 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:28.472 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:35.984431+0000", 2024-04-07T17:51:28.472 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:35.984431+0000", 2024-04-07T17:51:28.472 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:28.472 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:28.472 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:28.473 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:28.473 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 526, 2024-04-07T17:51:28.473 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:28.473 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:28.473 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:28.473 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:28.473 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:28.473 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:28.473 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:28.473 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:28.473 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.32", 2024-04-07T17:51:28.473 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:28.473 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:28.473 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:28.473 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4995, 2024-04-07T17:51:28.473 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:28.473 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T22:18:55.330487+0000", 2024-04-07T17:51:28.474 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:28.474 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:28.474 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:28.474 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:28.474 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:28.474 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:28.474 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:28.474 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:28.474 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:28.474 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:28.474 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:28.474 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:28.474 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:28.474 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:28.475 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:28.475 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:28.475 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:28.475 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:28.475 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:28.475 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:28.475 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:28.475 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:28.475 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:28.475 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:28.475 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:28.475 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:28.475 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:28.475 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:28.475 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:28.475 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:28.476 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:28.476 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:28.476 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:28.476 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:28.476 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:28.476 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:28.476 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:28.476 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:28.476 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:28.476 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:28.476 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:28.476 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:28.476 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:28.476 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.476 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:28.476 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:28.477 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:28.477 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:28.477 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:28.477 INFO:tasks.workunit.client.0.smithi100.stdout: 5 2024-04-07T17:51:28.477 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.477 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 0, 2024-04-07T17:51:28.477 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:28.477 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.477 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:28.477 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:28.477 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:28.477 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:28.477 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T17:51:28.477 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.477 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 3, 2024-04-07T17:51:28.477 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:28.477 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:28.478 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:28.478 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:28.478 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:28.478 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:28.478 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:35.977234+0000", 2024-04-07T17:51:28.478 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:08.976145+0000", 2024-04-07T17:51:28.478 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:08.976145+0000", 2024-04-07T17:51:28.478 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:08.976499+0000", 2024-04-07T17:51:28.478 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:35.977234+0000", 2024-04-07T17:51:28.478 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:28.478 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:28.478 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:28.478 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 554, 2024-04-07T17:51:28.478 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:35.977234+0000", 2024-04-07T17:51:28.478 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:35.977234+0000", 2024-04-07T17:51:28.479 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:35.977234+0000", 2024-04-07T17:51:28.479 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:28.479 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:28.479 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:28.479 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:35.977234+0000", 2024-04-07T17:51:28.479 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:35.977234+0000", 2024-04-07T17:51:28.479 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:28.479 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:28.479 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:28.479 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:28.479 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 553, 2024-04-07T17:51:28.479 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:28.479 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:28.479 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:28.479 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:28.479 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:28.480 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:28.480 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:28.480 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:28.480 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.33", 2024-04-07T17:51:28.480 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:28.480 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:28.480 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:28.480 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4963, 2024-04-07T17:51:28.480 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:28.480 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T04:02:25.054838+0000", 2024-04-07T17:51:28.480 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:28.480 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:28.480 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:28.480 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:28.480 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:28.481 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:28.481 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:28.481 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:28.481 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:28.481 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:28.481 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:28.481 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:28.481 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:28.481 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:28.481 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:28.481 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:28.481 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:28.481 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:28.481 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:28.481 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:28.482 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:28.482 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:28.482 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:28.482 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:28.482 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:28.482 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:28.482 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:28.482 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:28.482 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:28.482 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:28.482 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:28.482 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:28.482 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:28.482 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:28.482 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:28.482 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:28.483 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:28.483 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:28.483 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:28.483 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:28.483 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:28.483 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:28.483 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:28.483 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.483 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:28.483 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:28.483 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:28.483 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:28.483 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:28.483 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T17:51:28.483 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.483 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 3, 2024-04-07T17:51:28.484 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:28.484 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.484 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:28.484 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:28.484 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:28.484 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:28.484 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T17:51:28.484 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.484 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 4, 2024-04-07T17:51:28.484 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:28.484 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:28.484 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:28.484 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:28.484 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:28.484 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:28.484 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:35.980246+0000", 2024-04-07T17:51:28.484 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:08.955654+0000", 2024-04-07T17:51:28.485 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:08.955654+0000", 2024-04-07T17:51:28.485 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:08.956220+0000", 2024-04-07T17:51:28.485 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:35.980246+0000", 2024-04-07T17:51:28.485 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:28.485 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:28.485 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:28.485 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 554, 2024-04-07T17:51:28.485 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:35.980246+0000", 2024-04-07T17:51:28.485 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:35.980246+0000", 2024-04-07T17:51:28.485 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:35.980246+0000", 2024-04-07T17:51:28.485 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:28.485 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:28.485 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:28.485 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:35.980246+0000", 2024-04-07T17:51:28.485 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:35.980246+0000", 2024-04-07T17:51:28.485 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:28.486 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:28.486 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:28.486 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:28.486 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 553, 2024-04-07T17:51:28.486 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:28.486 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:28.486 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:28.486 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:28.486 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:28.486 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:28.486 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:28.486 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:28.486 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.34", 2024-04-07T17:51:28.486 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:28.487 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:28.487 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:28.487 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4913, 2024-04-07T17:51:28.487 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:28.487 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T00:20:14.081274+0000", 2024-04-07T17:51:28.487 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:28.487 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:28.487 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:28.487 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:28.487 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:28.487 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:28.487 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:28.487 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:28.487 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:28.487 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:28.487 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:28.488 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:28.488 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:28.488 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:28.488 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:28.488 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:28.488 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:28.488 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:28.488 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:28.488 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:28.488 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:28.488 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:28.488 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:28.488 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:28.488 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:28.488 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:28.488 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:28.488 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:28.489 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:28.489 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:28.489 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:28.489 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:28.489 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:28.489 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:28.489 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:28.489 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:28.489 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:28.489 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:28.489 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:28.489 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:28.489 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:28.489 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:28.489 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:28.489 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.489 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:28.489 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:28.490 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:28.490 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:28.490 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:28.490 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T17:51:28.490 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.490 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 4, 2024-04-07T17:51:28.490 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:28.490 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.490 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:28.490 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:28.490 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:28.490 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:28.490 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T17:51:28.490 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.490 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 6, 2024-04-07T17:51:28.490 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:28.491 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:28.491 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:28.491 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:28.491 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:28.491 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:28.491 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.359086+0000", 2024-04-07T17:51:28.491 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:08.978750+0000", 2024-04-07T17:51:28.491 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:08.978750+0000", 2024-04-07T17:51:28.491 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:08.979235+0000", 2024-04-07T17:51:28.491 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.359086+0000", 2024-04-07T17:51:28.491 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:28.491 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:28.491 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:28.491 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 554, 2024-04-07T17:51:28.491 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.359086+0000", 2024-04-07T17:51:28.491 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.359086+0000", 2024-04-07T17:51:28.491 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.359086+0000", 2024-04-07T17:51:28.492 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:28.492 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:28.492 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:28.492 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.359086+0000", 2024-04-07T17:51:28.492 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.359086+0000", 2024-04-07T17:51:28.492 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:28.492 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:28.492 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:28.492 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:28.492 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 553, 2024-04-07T17:51:28.492 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:28.492 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:28.492 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:28.492 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:28.492 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:28.492 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:28.492 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:28.493 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:28.493 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.35", 2024-04-07T17:51:28.493 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:28.493 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:28.493 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:28.493 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4828, 2024-04-07T17:51:28.493 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:28.493 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T03:27:45.790517+0000", 2024-04-07T17:51:28.493 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:28.493 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:28.493 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:28.493 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:28.493 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:28.493 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:28.493 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:28.493 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:28.494 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:28.494 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:28.494 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:28.494 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:28.494 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:28.494 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:28.494 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:28.494 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:28.494 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:28.494 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:28.494 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:28.494 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:28.494 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:28.494 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:28.494 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:28.494 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:28.495 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:28.495 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:28.495 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:28.495 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:28.495 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:28.495 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:28.495 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:28.495 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:28.495 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:28.495 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:28.495 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:28.495 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:28.495 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:28.495 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:28.495 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:28.495 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:28.495 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:28.496 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:28.496 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:28.496 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.496 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:28.496 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:28.496 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:28.496 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:28.496 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:28.496 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T17:51:28.496 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.496 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 6, 2024-04-07T17:51:28.496 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:28.496 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.496 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:28.496 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:28.496 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:28.497 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:28.497 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T17:51:28.497 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.497 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 6, 2024-04-07T17:51:28.497 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:28.497 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:28.497 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:28.497 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:28.497 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:28.497 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:28.497 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.368833+0000", 2024-04-07T17:51:28.497 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:08.977646+0000", 2024-04-07T17:51:28.497 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:08.977646+0000", 2024-04-07T17:51:28.497 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:08.978144+0000", 2024-04-07T17:51:28.497 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.368833+0000", 2024-04-07T17:51:28.497 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:28.498 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:28.498 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:28.498 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 554, 2024-04-07T17:51:28.498 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.368833+0000", 2024-04-07T17:51:28.498 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.368833+0000", 2024-04-07T17:51:28.498 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.368833+0000", 2024-04-07T17:51:28.498 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:28.498 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:28.498 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:28.498 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.368833+0000", 2024-04-07T17:51:28.498 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.368833+0000", 2024-04-07T17:51:28.498 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:28.498 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:28.498 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:28.498 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:28.499 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 553, 2024-04-07T17:51:28.499 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:28.499 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:28.499 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:28.499 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:28.499 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:28.499 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:28.499 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:28.499 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:28.499 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.36", 2024-04-07T17:51:28.499 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:28.499 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:28.499 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:28.499 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4828, 2024-04-07T17:51:28.499 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:28.499 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T01:10:43.702742+0000", 2024-04-07T17:51:28.499 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:28.500 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:28.500 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:28.500 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:28.500 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:28.500 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:28.500 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:28.500 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:28.500 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:28.500 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:28.500 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:28.500 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:28.500 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:28.500 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:28.500 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:28.500 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:28.500 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:28.501 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:28.501 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:28.501 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:28.501 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:28.501 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:28.501 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:28.501 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:28.501 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:28.501 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:28.501 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:28.501 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:28.501 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:28.501 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:28.501 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:28.501 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:28.501 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:28.502 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:28.502 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:28.502 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:28.502 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:28.502 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:28.502 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:28.502 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:28.502 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:28.502 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:28.502 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:28.502 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.502 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:28.502 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:28.502 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:28.502 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:28.502 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:28.503 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T17:51:28.503 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.503 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 6, 2024-04-07T17:51:28.503 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:28.503 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.503 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:28.503 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:28.503 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:28.503 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:28.503 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T17:51:28.503 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.503 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 7, 2024-04-07T17:51:28.503 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:28.503 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:28.503 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:28.503 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:28.503 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:28.504 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:28.504 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.366733+0000", 2024-04-07T17:51:28.504 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:38.968877+0000", 2024-04-07T17:51:28.504 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:38.968877+0000", 2024-04-07T17:51:28.504 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:38.976137+0000", 2024-04-07T17:51:28.504 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.366733+0000", 2024-04-07T17:51:28.504 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:28.504 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:28.504 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:28.504 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:28.504 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.366733+0000", 2024-04-07T17:51:28.504 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.366733+0000", 2024-04-07T17:51:28.504 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.366733+0000", 2024-04-07T17:51:28.504 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:28.504 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:28.504 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:28.504 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.366733+0000", 2024-04-07T17:51:28.505 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.366733+0000", 2024-04-07T17:51:28.505 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:28.505 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:28.505 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:28.505 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:28.505 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:28.505 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:28.505 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:28.505 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:28.505 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:28.505 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:28.505 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:28.505 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:28.505 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:28.505 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.37", 2024-04-07T17:51:28.505 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:28.505 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:28.506 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:28.506 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4815, 2024-04-07T17:51:28.506 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:28.506 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T17:38:48.549583+0000", 2024-04-07T17:51:28.506 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:28.506 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:28.506 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:28.506 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:28.506 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:28.506 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:28.506 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:28.506 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:28.506 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:28.506 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:28.506 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:28.507 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:28.507 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:28.507 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:28.507 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:28.507 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:28.507 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:28.507 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:28.507 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:28.507 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:28.507 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:28.507 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:28.507 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:28.507 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:28.507 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:28.507 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:28.507 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:28.508 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:28.508 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:28.508 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:28.508 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:28.508 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:28.508 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:28.508 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:28.508 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:28.508 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:28.508 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:28.508 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:28.508 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:28.508 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:28.508 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:28.508 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:28.508 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:28.509 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.509 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:28.509 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:28.509 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:28.509 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:28.509 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:28.509 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T17:51:28.509 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.509 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 7, 2024-04-07T17:51:28.509 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:28.509 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.509 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:28.509 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:28.509 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:28.509 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:28.509 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T17:51:28.510 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.510 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 3, 2024-04-07T17:51:28.510 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:28.510 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:28.510 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:28.510 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:28.510 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:28.510 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:28.510 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:35.975131+0000", 2024-04-07T17:51:28.510 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:08.977725+0000", 2024-04-07T17:51:28.510 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:08.977725+0000", 2024-04-07T17:51:28.510 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:08.978090+0000", 2024-04-07T17:51:28.510 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:35.975131+0000", 2024-04-07T17:51:28.510 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:28.510 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:28.511 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:28.511 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 554, 2024-04-07T17:51:28.511 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:35.975131+0000", 2024-04-07T17:51:28.511 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:35.975131+0000", 2024-04-07T17:51:28.511 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:35.975131+0000", 2024-04-07T17:51:28.511 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:28.511 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:28.511 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:28.511 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:35.975131+0000", 2024-04-07T17:51:28.511 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:35.975131+0000", 2024-04-07T17:51:28.511 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:28.511 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:28.511 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:28.511 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:28.512 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 553, 2024-04-07T17:51:28.512 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:28.512 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:28.512 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:28.512 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:28.512 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:28.512 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:28.512 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:28.512 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:28.512 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.38", 2024-04-07T17:51:28.512 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:28.512 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:28.512 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:28.512 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4929, 2024-04-07T17:51:28.512 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:28.512 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T18:02:53.604912+0000", 2024-04-07T17:51:28.513 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:28.513 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:28.513 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:28.513 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:28.513 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:28.513 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:28.513 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:28.513 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:28.513 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:28.513 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:28.513 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:28.513 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:28.513 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:28.513 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:28.513 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:28.513 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:28.513 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:28.514 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:28.514 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:28.514 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:28.514 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:28.514 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:28.514 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:28.514 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:28.514 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:28.514 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:28.514 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:28.514 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:28.514 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:28.514 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:28.514 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:28.515 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:28.515 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:28.515 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:28.515 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:28.515 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:28.515 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:28.515 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:28.515 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:28.515 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:28.515 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:28.515 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:28.515 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:28.515 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.515 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:28.515 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:28.515 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:28.516 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:28.516 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:28.516 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T17:51:28.516 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.516 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 3, 2024-04-07T17:51:28.516 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:28.516 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.516 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:28.516 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:28.516 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:28.516 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:28.516 INFO:tasks.workunit.client.0.smithi100.stdout: 4 2024-04-07T17:51:28.516 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.516 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 7, 2024-04-07T17:51:28.517 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:28.517 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:28.517 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:28.517 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:28.517 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:28.517 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:28.517 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.358816+0000", 2024-04-07T17:51:28.517 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:38.996215+0000", 2024-04-07T17:51:28.517 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:38.996215+0000", 2024-04-07T17:51:28.517 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:39.000120+0000", 2024-04-07T17:51:28.517 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.358816+0000", 2024-04-07T17:51:28.517 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:28.517 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:28.517 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:28.517 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:28.518 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.358816+0000", 2024-04-07T17:51:28.518 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.358816+0000", 2024-04-07T17:51:28.518 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.358816+0000", 2024-04-07T17:51:28.518 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:28.518 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:28.518 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:28.518 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.358816+0000", 2024-04-07T17:51:28.518 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.358816+0000", 2024-04-07T17:51:28.518 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:28.518 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:28.518 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:28.518 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:28.518 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:28.518 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:28.518 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:28.519 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:28.519 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:28.519 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:28.519 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:28.519 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:28.519 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:28.519 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.39", 2024-04-07T17:51:28.519 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:28.519 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:28.519 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:28.519 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4787, 2024-04-07T17:51:28.519 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:28.519 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T22:36:35.618662+0000", 2024-04-07T17:51:28.519 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:28.519 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:28.519 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:28.520 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:28.520 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:28.520 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:28.520 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:28.520 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:28.520 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:28.520 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:28.520 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:28.520 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:28.520 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:28.520 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:28.520 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:28.520 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:28.520 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:28.520 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:28.520 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:28.521 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:28.521 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:28.521 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:28.521 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:28.521 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:28.521 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:28.521 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:28.521 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:28.521 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:28.521 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:28.521 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:28.521 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:28.521 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:28.521 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:28.521 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:28.521 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:28.521 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:28.522 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:28.522 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:28.522 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:28.522 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:28.522 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:28.522 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:28.522 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:28.522 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.522 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:28.522 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:28.522 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:28.522 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:28.522 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:28.522 INFO:tasks.workunit.client.0.smithi100.stdout: 4 2024-04-07T17:51:28.522 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.522 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 7, 2024-04-07T17:51:28.523 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:28.523 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.523 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:28.523 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:28.523 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:28.523 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:28.523 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T17:51:28.523 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.523 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 5, 2024-04-07T17:51:28.523 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:28.523 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:28.523 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:28.523 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:28.523 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:28.523 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:28.523 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.368733+0000", 2024-04-07T17:51:28.523 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:24:41.548397+0000", 2024-04-07T17:51:28.524 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:24:41.548397+0000", 2024-04-07T17:51:28.524 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:24:41.551443+0000", 2024-04-07T17:51:28.524 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.368733+0000", 2024-04-07T17:51:28.524 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:28.524 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:28.524 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:28.524 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 527, 2024-04-07T17:51:28.524 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.368733+0000", 2024-04-07T17:51:28.524 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.368733+0000", 2024-04-07T17:51:28.524 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.368733+0000", 2024-04-07T17:51:28.524 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:28.524 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:28.524 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:28.524 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.368733+0000", 2024-04-07T17:51:28.524 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.368733+0000", 2024-04-07T17:51:28.524 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:28.524 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:28.525 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:28.525 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:28.525 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 526, 2024-04-07T17:51:28.525 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:28.525 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:28.525 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:28.525 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:28.525 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:28.525 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:28.525 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:28.525 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:28.525 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.3a", 2024-04-07T17:51:28.525 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:28.525 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:28.525 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:28.525 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4865, 2024-04-07T17:51:28.526 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:28.526 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T20:38:47.135272+0000", 2024-04-07T17:51:28.526 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:28.526 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:28.526 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:28.526 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:28.526 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:28.526 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:28.526 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:28.526 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:28.526 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:28.526 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:28.526 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:28.526 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:28.527 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:28.527 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:28.527 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:28.527 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:28.527 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:28.527 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:28.527 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:28.527 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:28.527 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:28.527 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:28.527 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:28.527 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:28.527 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:28.527 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:28.527 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:28.527 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:28.528 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:28.528 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:28.528 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:28.528 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:28.528 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:28.528 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:28.528 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:28.528 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:28.528 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:28.528 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:28.528 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:28.528 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:28.528 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:28.528 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:28.529 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:28.529 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.529 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:28.529 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:28.529 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:28.529 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:28.529 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:28.529 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T17:51:28.529 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.529 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 5, 2024-04-07T17:51:28.529 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:28.529 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.529 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:28.529 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:28.529 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:28.529 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T17:51:28.530 INFO:tasks.workunit.client.0.smithi100.stdout: 4 2024-04-07T17:51:28.530 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.530 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 7, 2024-04-07T17:51:28.530 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:28.530 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:28.530 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:28.530 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:28.530 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:28.530 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:28.530 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.369564+0000", 2024-04-07T17:51:28.530 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:38.985715+0000", 2024-04-07T17:51:28.530 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:38.985715+0000", 2024-04-07T17:51:28.530 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:38.990756+0000", 2024-04-07T17:51:28.530 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.369564+0000", 2024-04-07T17:51:28.531 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:28.531 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:28.531 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:28.531 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:28.531 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.369564+0000", 2024-04-07T17:51:28.531 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.369564+0000", 2024-04-07T17:51:28.531 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.369564+0000", 2024-04-07T17:51:28.531 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:28.531 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:28.531 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:28.531 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.369564+0000", 2024-04-07T17:51:28.531 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.369564+0000", 2024-04-07T17:51:28.531 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:28.531 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:28.531 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:28.531 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:28.532 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:28.532 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:28.532 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:28.532 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:28.532 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:28.532 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:28.532 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:28.532 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:28.532 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:28.532 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.3b", 2024-04-07T17:51:28.532 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:28.532 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:28.532 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:28.532 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4787, 2024-04-07T17:51:28.532 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:28.532 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T17:50:57.375885+0000", 2024-04-07T17:51:28.532 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:28.533 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:28.533 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:28.533 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:28.533 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:28.533 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:28.533 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:28.533 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:28.533 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:28.533 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:28.533 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:28.533 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:28.533 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:28.533 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:28.533 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:28.533 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:28.533 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:28.533 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:28.534 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:28.534 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:28.534 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:28.534 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:28.534 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:28.534 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:28.534 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:28.534 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:28.534 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:28.534 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:28.534 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:28.534 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:28.534 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:28.534 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:28.534 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:28.535 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:28.535 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:28.535 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:28.535 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:28.535 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:28.535 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:28.535 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:28.535 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:28.535 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:28.535 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:28.535 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.535 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:28.535 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:28.535 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:28.535 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:28.535 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T17:51:28.536 INFO:tasks.workunit.client.0.smithi100.stdout: 4 2024-04-07T17:51:28.536 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.536 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 7, 2024-04-07T17:51:28.536 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:28.536 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.536 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:28.536 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:28.536 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:28.536 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:28.536 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T17:51:28.536 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.536 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 5, 2024-04-07T17:51:28.536 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:28.536 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:28.536 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:28.536 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:28.537 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:28.537 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:28.537 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.365779+0000", 2024-04-07T17:51:28.537 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:38.986923+0000", 2024-04-07T17:51:28.537 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:38.986923+0000", 2024-04-07T17:51:28.537 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:38.987454+0000", 2024-04-07T17:51:28.537 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.365779+0000", 2024-04-07T17:51:28.537 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:28.537 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:28.537 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:28.537 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:28.537 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.365779+0000", 2024-04-07T17:51:28.537 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.365779+0000", 2024-04-07T17:51:28.537 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.365779+0000", 2024-04-07T17:51:28.537 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:28.538 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:28.538 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:28.538 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.365779+0000", 2024-04-07T17:51:28.538 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.365779+0000", 2024-04-07T17:51:28.538 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:28.538 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:28.538 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:28.538 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:28.538 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:28.538 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:28.538 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:28.538 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:28.538 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:28.538 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:28.538 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:28.539 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:28.539 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:28.539 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.3c", 2024-04-07T17:51:28.539 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:28.539 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:28.539 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:28.539 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4883, 2024-04-07T17:51:28.539 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:28.539 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T02:20:58.654636+0000", 2024-04-07T17:51:28.539 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:28.539 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:28.539 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:28.539 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:28.539 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:28.539 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:28.540 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:28.540 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:28.540 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:28.540 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:28.540 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:28.540 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:28.540 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:28.540 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:28.540 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:28.540 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:28.540 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:28.540 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:28.540 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:28.540 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:28.540 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:28.541 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:28.541 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:28.541 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:28.541 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:28.541 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:28.541 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:28.541 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:28.541 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:28.541 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:28.541 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:28.541 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:28.541 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:28.542 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:28.542 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:28.542 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:28.542 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:28.542 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:28.542 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:28.542 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:28.542 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:28.542 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:28.542 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:28.542 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.542 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:28.542 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:28.542 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:28.543 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:28.543 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:28.543 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T17:51:28.543 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.543 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 5, 2024-04-07T17:51:28.543 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:28.543 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.543 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:28.543 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:28.543 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:28.543 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T17:51:28.543 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T17:51:28.543 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.543 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 0, 2024-04-07T17:51:28.544 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:28.544 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:28.544 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:28.544 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:28.544 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:28.544 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:28.544 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:35.982778+0000", 2024-04-07T17:51:28.544 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:23:15.876439+0000", 2024-04-07T17:51:28.544 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:23:15.876439+0000", 2024-04-07T17:51:28.544 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:23:15.876856+0000", 2024-04-07T17:51:28.544 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:35.982778+0000", 2024-04-07T17:51:28.544 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:28.544 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:28.544 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:28.544 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 445, 2024-04-07T17:51:28.544 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:35.982778+0000", 2024-04-07T17:51:28.545 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:35.982778+0000", 2024-04-07T17:51:28.545 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:35.982778+0000", 2024-04-07T17:51:28.545 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:28.545 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:28.545 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:28.545 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:35.982778+0000", 2024-04-07T17:51:28.545 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:35.982778+0000", 2024-04-07T17:51:28.545 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:28.545 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:28.545 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:28.545 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:28.545 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 444, 2024-04-07T17:51:28.545 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:28.545 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:28.546 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:28.546 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:28.546 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:28.546 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:28.546 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:28.546 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:28.546 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.3d", 2024-04-07T17:51:28.546 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:28.546 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:28.546 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:28.546 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 5000, 2024-04-07T17:51:28.546 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:28.546 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T03:48:36.975141+0000", 2024-04-07T17:51:28.546 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:28.547 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:28.547 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:28.547 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:28.547 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:28.547 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:28.547 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:28.547 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:28.547 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:28.547 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:28.547 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:28.547 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:28.547 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:28.547 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:28.547 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:28.547 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:28.548 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:28.548 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:28.548 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:28.548 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:28.548 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:28.548 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:28.548 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:28.548 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:28.548 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:28.548 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:28.548 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:28.548 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:28.548 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:28.548 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:28.548 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:28.549 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:28.549 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:28.549 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:28.549 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:28.549 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:28.549 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:28.549 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:28.549 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:28.549 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:28.549 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:28.549 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:28.549 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:28.549 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.549 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:28.549 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:28.549 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:28.550 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:28.550 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T17:51:28.550 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T17:51:28.550 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.550 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 0, 2024-04-07T17:51:28.550 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:28.550 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.550 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:28.550 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:28.550 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:28.550 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:28.550 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T17:51:28.550 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.550 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 1, 2024-04-07T17:51:28.551 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:28.551 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:28.551 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:28.551 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:28.551 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:28.551 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:28.551 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.334821+0000", 2024-04-07T17:51:28.551 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:08.962145+0000", 2024-04-07T17:51:28.551 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:08.962145+0000", 2024-04-07T17:51:28.551 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:08.962703+0000", 2024-04-07T17:51:28.551 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.334821+0000", 2024-04-07T17:51:28.551 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:28.551 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:28.551 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:28.551 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 554, 2024-04-07T17:51:28.551 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.334821+0000", 2024-04-07T17:51:28.552 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.334821+0000", 2024-04-07T17:51:28.552 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.334821+0000", 2024-04-07T17:51:28.552 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:28.552 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:28.552 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:28.552 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.334821+0000", 2024-04-07T17:51:28.552 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.334821+0000", 2024-04-07T17:51:28.552 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:28.552 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:28.552 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:28.552 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:28.552 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 553, 2024-04-07T17:51:28.552 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:28.552 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:28.552 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:28.552 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:28.553 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:28.553 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:28.553 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:28.553 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:28.553 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.3e", 2024-04-07T17:51:28.553 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:28.553 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:28.553 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:28.553 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4989, 2024-04-07T17:51:28.553 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:28.553 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T23:09:45.491615+0000", 2024-04-07T17:51:28.553 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:28.553 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:28.553 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:28.553 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:28.554 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:28.554 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:28.554 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:28.554 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:28.554 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:28.554 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:28.554 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:28.554 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:28.554 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:28.554 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:28.554 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:28.554 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:28.554 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:28.554 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:28.554 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:28.555 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:28.555 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:28.555 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:28.555 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:28.555 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:28.555 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:28.555 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:28.555 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:28.555 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:28.555 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:28.555 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:28.555 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:28.555 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:28.555 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:28.555 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:28.556 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:28.556 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:28.556 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:28.556 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:28.556 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:28.556 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:28.556 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:28.556 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:28.556 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:28.556 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.556 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:28.556 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:28.556 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:28.556 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:28.557 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:28.557 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T17:51:28.557 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.557 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 1, 2024-04-07T17:51:28.557 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:28.557 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.557 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:28.557 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:28.557 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:28.557 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:28.557 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T17:51:28.557 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.557 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 7, 2024-04-07T17:51:28.557 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:28.557 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:28.557 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:28.557 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:28.558 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:28.558 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:28.558 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.367170+0000", 2024-04-07T17:51:28.558 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:38.966876+0000", 2024-04-07T17:51:28.558 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:38.966876+0000", 2024-04-07T17:51:28.558 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:38.969419+0000", 2024-04-07T17:51:28.558 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.367170+0000", 2024-04-07T17:51:28.558 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:28.558 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:28.558 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:28.558 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:28.558 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.367170+0000", 2024-04-07T17:51:28.558 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.367170+0000", 2024-04-07T17:51:28.558 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.367170+0000", 2024-04-07T17:51:28.558 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:28.559 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:28.559 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:28.559 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.367170+0000", 2024-04-07T17:51:28.559 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.367170+0000", 2024-04-07T17:51:28.559 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:28.559 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:28.559 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:28.559 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:28.559 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:28.559 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:28.559 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:28.559 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:28.559 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:28.559 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:28.559 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:28.560 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:28.560 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:28.560 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.3f", 2024-04-07T17:51:28.560 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:28.560 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:28.560 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:28.560 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4807, 2024-04-07T17:51:28.560 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:28.560 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T01:03:39.405808+0000", 2024-04-07T17:51:28.560 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:28.560 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:28.560 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:28.560 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:28.560 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:28.560 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:28.561 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:28.561 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:28.561 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:28.561 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:28.561 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:28.561 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:28.561 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:28.561 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:28.561 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:28.561 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:28.561 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:28.561 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:28.561 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:28.561 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:28.561 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:28.562 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:28.562 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:28.562 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:28.562 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:28.562 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:28.562 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:28.562 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:28.562 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:28.562 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:28.562 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:28.562 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:28.562 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:28.562 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:28.562 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:28.562 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:28.563 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:28.563 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:28.563 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:28.563 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:28.563 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:28.563 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:28.563 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:28.563 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.563 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:28.563 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:28.563 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:28.563 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:28.563 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:28.563 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T17:51:28.563 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.563 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 7, 2024-04-07T17:51:28.564 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:28.564 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:28.564 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.564 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_per_pool": [ 2024-04-07T17:51:28.564 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:28.564 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": 24, 2024-04-07T17:51:28.564 INFO:tasks.workunit.client.0.smithi100.stdout: "application": [ 2024-04-07T17:51:28.564 INFO:tasks.workunit.client.0.smithi100.stdout: "rbd" 2024-04-07T17:51:28.564 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.564 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 24403, 2024-04-07T17:51:28.564 INFO:tasks.workunit.client.0.smithi100.stdout: "num_pg": 8, 2024-04-07T17:51:28.564 INFO:tasks.workunit.client.0.smithi100.stdout: "num_store_stats": 8, 2024-04-07T17:51:28.564 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 24403, 2024-04-07T17:51:28.564 INFO:tasks.workunit.client.0.smithi100.stdout: "poolid": 4, 2024-04-07T17:51:28.564 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:28.564 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 4194323, 2024-04-07T17:51:28.565 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:28.565 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:28.565 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:28.565 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:28.565 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:28.565 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:28.565 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:28.565 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:28.565 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:28.565 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:28.565 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:28.565 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:28.565 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:28.565 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:28.565 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:28.565 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 12, 2024-04-07T17:51:28.566 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 4, 2024-04-07T17:51:28.566 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:28.566 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 4, 2024-04-07T17:51:28.566 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:28.566 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:28.566 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:28.566 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:28.566 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:28.566 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 2, 2024-04-07T17:51:28.566 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:28.566 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:28.566 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:28.566 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:28.566 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:28.566 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:28.567 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:28.567 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 9098, 2024-04-07T17:51:28.567 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 3147283, 2024-04-07T17:51:28.567 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:28.567 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:28.567 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:28.567 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 24273, 2024-04-07T17:51:28.567 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 3030917 2024-04-07T17:51:28.567 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.567 INFO:tasks.workunit.client.0.smithi100.stdout: "store_stats": { 2024-04-07T17:51:28.567 INFO:tasks.workunit.client.0.smithi100.stdout: "allocated": 12595200, 2024-04-07T17:51:28.567 INFO:tasks.workunit.client.0.smithi100.stdout: "available": 0, 2024-04-07T17:51:28.567 INFO:tasks.workunit.client.0.smithi100.stdout: "data_compressed": 0, 2024-04-07T17:51:28.567 INFO:tasks.workunit.client.0.smithi100.stdout: "data_compressed_allocated": 0, 2024-04-07T17:51:28.567 INFO:tasks.workunit.client.0.smithi100.stdout: "data_compressed_original": 0, 2024-04-07T17:51:28.568 INFO:tasks.workunit.client.0.smithi100.stdout: "data_stored": 12582969, 2024-04-07T17:51:28.568 INFO:tasks.workunit.client.0.smithi100.stdout: "internal_metadata": 0, 2024-04-07T17:51:28.568 INFO:tasks.workunit.client.0.smithi100.stdout: "internally_reserved": 0, 2024-04-07T17:51:28.568 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_allocated": 0, 2024-04-07T17:51:28.568 INFO:tasks.workunit.client.0.smithi100.stdout: "total": 0 2024-04-07T17:51:28.568 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.568 INFO:tasks.workunit.client.0.smithi100.stdout: "up": 24 2024-04-07T17:51:28.568 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.568 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:28.568 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": 192, 2024-04-07T17:51:28.568 INFO:tasks.workunit.client.0.smithi100.stdout: "application": [ 2024-04-07T17:51:28.568 INFO:tasks.workunit.client.0.smithi100.stdout: "cephfs" 2024-04-07T17:51:28.568 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.568 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:28.568 INFO:tasks.workunit.client.0.smithi100.stdout: "num_pg": 64, 2024-04-07T17:51:28.568 INFO:tasks.workunit.client.0.smithi100.stdout: "num_store_stats": 8, 2024-04-07T17:51:28.569 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:28.569 INFO:tasks.workunit.client.0.smithi100.stdout: "poolid": 3, 2024-04-07T17:51:28.569 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:28.569 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:28.569 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:28.569 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:28.569 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:28.569 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:28.569 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:28.569 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:28.569 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:28.569 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:28.569 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:28.569 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:28.569 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:28.569 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:28.570 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:28.570 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:28.570 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:28.570 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:28.570 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:28.570 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:28.570 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:28.570 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:28.570 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:28.570 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:28.570 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:28.570 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:28.570 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:28.570 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:28.571 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:28.571 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:28.571 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:28.571 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:28.571 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:28.571 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:28.571 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:28.571 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:28.571 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:28.571 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:28.571 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:28.571 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:28.571 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:28.571 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.571 INFO:tasks.workunit.client.0.smithi100.stdout: "store_stats": { 2024-04-07T17:51:28.572 INFO:tasks.workunit.client.0.smithi100.stdout: "allocated": 0, 2024-04-07T17:51:28.572 INFO:tasks.workunit.client.0.smithi100.stdout: "available": 0, 2024-04-07T17:51:28.572 INFO:tasks.workunit.client.0.smithi100.stdout: "data_compressed": 0, 2024-04-07T17:51:28.572 INFO:tasks.workunit.client.0.smithi100.stdout: "data_compressed_allocated": 0, 2024-04-07T17:51:28.572 INFO:tasks.workunit.client.0.smithi100.stdout: "data_compressed_original": 0, 2024-04-07T17:51:28.572 INFO:tasks.workunit.client.0.smithi100.stdout: "data_stored": 0, 2024-04-07T17:51:28.572 INFO:tasks.workunit.client.0.smithi100.stdout: "internal_metadata": 0, 2024-04-07T17:51:28.572 INFO:tasks.workunit.client.0.smithi100.stdout: "internally_reserved": 0, 2024-04-07T17:51:28.572 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_allocated": 0, 2024-04-07T17:51:28.572 INFO:tasks.workunit.client.0.smithi100.stdout: "total": 0 2024-04-07T17:51:28.572 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.572 INFO:tasks.workunit.client.0.smithi100.stdout: "up": 192 2024-04-07T17:51:28.572 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.572 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:28.572 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": 3, 2024-04-07T17:51:28.572 INFO:tasks.workunit.client.0.smithi100.stdout: "application": [ 2024-04-07T17:51:28.573 INFO:tasks.workunit.client.0.smithi100.stdout: "mgr" 2024-04-07T17:51:28.573 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.573 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 241, 2024-04-07T17:51:28.573 INFO:tasks.workunit.client.0.smithi100.stdout: "num_pg": 1, 2024-04-07T17:51:28.573 INFO:tasks.workunit.client.0.smithi100.stdout: "num_store_stats": 3, 2024-04-07T17:51:28.573 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 241, 2024-04-07T17:51:28.573 INFO:tasks.workunit.client.0.smithi100.stdout: "poolid": 1, 2024-04-07T17:51:28.573 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:28.573 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 590368, 2024-04-07T17:51:28.573 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:28.573 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 1771104, 2024-04-07T17:51:28.573 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:28.573 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:28.573 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:28.573 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:28.574 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:28.574 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:28.574 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:28.574 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:28.574 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:28.574 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:28.574 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:28.574 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:28.574 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:28.574 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 6, 2024-04-07T17:51:28.574 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 2, 2024-04-07T17:51:28.574 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:28.574 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 2, 2024-04-07T17:51:28.574 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:28.575 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:28.575 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:28.575 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:28.575 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:28.575 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:28.575 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:28.575 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 6, 2024-04-07T17:51:28.575 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:28.575 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:28.575 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:28.575 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:28.575 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:28.575 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 1214, 2024-04-07T17:51:28.575 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 2024, 2024-04-07T17:51:28.575 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:28.575 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:28.576 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:28.576 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 399, 2024-04-07T17:51:28.576 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 2416 2024-04-07T17:51:28.576 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.576 INFO:tasks.workunit.client.0.smithi100.stdout: "store_stats": { 2024-04-07T17:51:28.576 INFO:tasks.workunit.client.0.smithi100.stdout: "allocated": 1781760, 2024-04-07T17:51:28.576 INFO:tasks.workunit.client.0.smithi100.stdout: "available": 0, 2024-04-07T17:51:28.576 INFO:tasks.workunit.client.0.smithi100.stdout: "data_compressed": 0, 2024-04-07T17:51:28.576 INFO:tasks.workunit.client.0.smithi100.stdout: "data_compressed_allocated": 0, 2024-04-07T17:51:28.576 INFO:tasks.workunit.client.0.smithi100.stdout: "data_compressed_original": 0, 2024-04-07T17:51:28.576 INFO:tasks.workunit.client.0.smithi100.stdout: "data_stored": 1771104, 2024-04-07T17:51:28.576 INFO:tasks.workunit.client.0.smithi100.stdout: "internal_metadata": 0, 2024-04-07T17:51:28.576 INFO:tasks.workunit.client.0.smithi100.stdout: "internally_reserved": 0, 2024-04-07T17:51:28.576 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_allocated": 0, 2024-04-07T17:51:28.576 INFO:tasks.workunit.client.0.smithi100.stdout: "total": 0 2024-04-07T17:51:28.577 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.577 INFO:tasks.workunit.client.0.smithi100.stdout: "up": 3 2024-04-07T17:51:28.577 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.577 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:28.577 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": 96, 2024-04-07T17:51:28.577 INFO:tasks.workunit.client.0.smithi100.stdout: "application": [ 2024-04-07T17:51:28.577 INFO:tasks.workunit.client.0.smithi100.stdout: "cephfs" 2024-04-07T17:51:28.577 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.577 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 68, 2024-04-07T17:51:28.577 INFO:tasks.workunit.client.0.smithi100.stdout: "num_pg": 32, 2024-04-07T17:51:28.577 INFO:tasks.workunit.client.0.smithi100.stdout: "num_store_stats": 8, 2024-04-07T17:51:28.577 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 68, 2024-04-07T17:51:28.577 INFO:tasks.workunit.client.0.smithi100.stdout: "poolid": 2, 2024-04-07T17:51:28.577 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:28.577 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 6622, 2024-04-07T17:51:28.578 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:28.578 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:28.578 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:28.578 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:28.578 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:28.578 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:28.578 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:28.578 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:28.578 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:28.578 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:28.578 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:28.578 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:28.578 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:28.578 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:28.579 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:28.579 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 66, 2024-04-07T17:51:28.579 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 22, 2024-04-07T17:51:28.579 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:28.579 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 22, 2024-04-07T17:51:28.579 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:28.579 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:28.579 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:28.579 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:28.579 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:28.579 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 14, 2024-04-07T17:51:28.579 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:28.579 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:28.579 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:28.579 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:28.580 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:28.580 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:28.580 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:28.580 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 79, 2024-04-07T17:51:28.580 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 87, 2024-04-07T17:51:28.580 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:28.580 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:28.580 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:28.580 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 68, 2024-04-07T17:51:28.580 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 36 2024-04-07T17:51:28.580 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.580 INFO:tasks.workunit.client.0.smithi100.stdout: "store_stats": { 2024-04-07T17:51:28.580 INFO:tasks.workunit.client.0.smithi100.stdout: "allocated": 110592, 2024-04-07T17:51:28.580 INFO:tasks.workunit.client.0.smithi100.stdout: "available": 0, 2024-04-07T17:51:28.580 INFO:tasks.workunit.client.0.smithi100.stdout: "data_compressed": 0, 2024-04-07T17:51:28.580 INFO:tasks.workunit.client.0.smithi100.stdout: "data_compressed_allocated": 0, 2024-04-07T17:51:28.581 INFO:tasks.workunit.client.0.smithi100.stdout: "data_compressed_original": 0, 2024-04-07T17:51:28.581 INFO:tasks.workunit.client.0.smithi100.stdout: "data_stored": 19866, 2024-04-07T17:51:28.581 INFO:tasks.workunit.client.0.smithi100.stdout: "internal_metadata": 0, 2024-04-07T17:51:28.581 INFO:tasks.workunit.client.0.smithi100.stdout: "internally_reserved": 0, 2024-04-07T17:51:28.581 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_allocated": 48581, 2024-04-07T17:51:28.581 INFO:tasks.workunit.client.0.smithi100.stdout: "total": 0 2024-04-07T17:51:28.581 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.581 INFO:tasks.workunit.client.0.smithi100.stdout: "up": 96 2024-04-07T17:51:28.581 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:28.581 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.581 INFO:tasks.workunit.client.0.smithi100.stdout: "usage": { 2024-04-07T17:51:28.581 INFO:tasks.workunit.client.0.smithi100.stdout: "pg_num": 105, 2024-04-07T17:51:28.581 INFO:tasks.workunit.client.0.smithi100.stdout: "pools": 4, 2024-04-07T17:51:28.581 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_by_class": { 2024-04-07T17:51:28.581 INFO:tasks.workunit.client.0.smithi100.stdout: "ssd": { 2024-04-07T17:51:28.582 INFO:tasks.workunit.client.0.smithi100.stdout: "total_avail_bytes": 763743436800, 2024-04-07T17:51:28.582 INFO:tasks.workunit.client.0.smithi100.stdout: "total_bytes": 767960285184, 2024-04-07T17:51:28.582 INFO:tasks.workunit.client.0.smithi100.stdout: "total_used_bytes": 4216848384, 2024-04-07T17:51:28.582 INFO:tasks.workunit.client.0.smithi100.stdout: "total_used_raw_bytes": 4216848384, 2024-04-07T17:51:28.582 INFO:tasks.workunit.client.0.smithi100.stdout: "total_used_raw_ratio": 0.005490971729159355 2024-04-07T17:51:28.582 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:28.582 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.582 INFO:tasks.workunit.client.0.smithi100.stdout: "total_avail_bytes": 763743436800, 2024-04-07T17:51:28.582 INFO:tasks.workunit.client.0.smithi100.stdout: "total_bytes": 767960285184, 2024-04-07T17:51:28.582 INFO:tasks.workunit.client.0.smithi100.stdout: "total_used_bytes": 4216848384 2024-04-07T17:51:28.582 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:28.582 INFO:tasks.workunit.client.0.smithi100.stdout:} 2024-04-07T17:51:28.582 INFO:tasks.workunit.client.0.smithi100.stdout:Device report is generated separately. To see it run 'ceph telemetry show-device'. 2024-04-07T17:51:28.583 INFO:tasks.workunit.client.0.smithi100.stderr:+ ceph telemetry show-device 2024-04-07T17:51:28.583 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:27 smithi100 bash[74942]: audit 2024-04-07T17:51:25.644364+0000 mgr.y (mgr.25175) 2890 : audit [DBG] from='client.46007 -' entity='client.admin' cmd=[{"prefix": "telemetry on", "license": "sharing-1-0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T17:51:28.583 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:27 smithi100 bash[74942]: cluster 2024-04-07T17:51:25.692912+0000 mgr.y (mgr.25175) 2891 : cluster [DBG] pgmap v5170: 105 pgs: 105 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 659 B/s rd, 1 op/s 2024-04-07T17:51:28.583 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:27 smithi100 bash[74942]: audit 2024-04-07T17:51:25.830845+0000 mgr.y (mgr.25175) 2892 : audit [DBG] from='client.46007 -' entity='client.admin' cmd=[{"prefix": "telemetry on", "license": "sharing-1-0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T17:51:28.583 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:27 smithi100 bash[74942]: audit 2024-04-07T17:51:26.118409+0000 mon.a (mon.0) 4889 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:28.583 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:27 smithi100 bash[74942]: audit 2024-04-07T17:51:26.118619+0000 mon.a (mon.0) 4890 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:28.583 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:27 smithi100 bash[74942]: audit 2024-04-07T17:51:26.127881+0000 mon.a (mon.0) 4891 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:28.583 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:27 smithi100 bash[74942]: audit 2024-04-07T17:51:26.141580+0000 mon.a (mon.0) 4892 : audit [INF] from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T17:51:28.583 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:27 smithi100 bash[74942]: audit 2024-04-07T17:51:26.141642+0000 mon.a (mon.0) 4893 : audit [INF] from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T17:51:28.584 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:27 smithi100 bash[74942]: audit 2024-04-07T17:51:26.141901+0000 mon.a (mon.0) 4894 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:28.584 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:27 smithi100 bash[74942]: audit 2024-04-07T17:51:26.142098+0000 mon.a (mon.0) 4895 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:28.584 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:27 smithi100 bash[74942]: audit 2024-04-07T17:51:26.142315+0000 mon.a (mon.0) 4896 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:28.584 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:27 smithi100 bash[74942]: audit 2024-04-07T17:51:26.145032+0000 mon.a (mon.0) 4897 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:28.584 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:27 smithi100 bash[74942]: audit 2024-04-07T17:51:26.145256+0000 mon.a (mon.0) 4898 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:28.584 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:27 smithi100 bash[74942]: audit 2024-04-07T17:51:26.145648+0000 mon.a (mon.0) 4899 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:28.584 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:27 smithi100 bash[74942]: audit 2024-04-07T17:51:26.147129+0000 mon.b (mon.1) 33 : audit [INF] from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T17:51:28.584 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:27 smithi100 bash[74942]: audit 2024-04-07T17:51:26.147250+0000 mon.b (mon.1) 34 : audit [INF] from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T17:51:28.584 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:27 smithi100 bash[74942]: audit 2024-04-07T17:51:26.147586+0000 mon.c (mon.2) 131 : audit [INF] from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T17:51:28.584 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:27 smithi100 bash[74942]: audit 2024-04-07T17:51:26.147683+0000 mon.c (mon.2) 132 : audit [INF] from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T17:51:28.584 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:27 smithi100 bash[74942]: audit 2024-04-07T17:51:26.147691+0000 mon.a (mon.0) 4900 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:28.584 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:27 smithi100 bash[76202]: audit 2024-04-07T17:51:25.644364+0000 mgr.y (mgr.25175) 2890 : audit [DBG] from='client.46007 -' entity='client.admin' cmd=[{"prefix": "telemetry on", "license": "sharing-1-0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T17:51:28.584 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:27 smithi100 bash[76202]: cluster 2024-04-07T17:51:25.692912+0000 mgr.y (mgr.25175) 2891 : cluster [DBG] pgmap v5170: 105 pgs: 105 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 659 B/s rd, 1 op/s 2024-04-07T17:51:28.585 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:27 smithi100 bash[76202]: audit 2024-04-07T17:51:25.830845+0000 mgr.y (mgr.25175) 2892 : audit [DBG] from='client.46007 -' entity='client.admin' cmd=[{"prefix": "telemetry on", "license": "sharing-1-0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T17:51:28.585 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:27 smithi100 bash[76202]: audit 2024-04-07T17:51:26.118409+0000 mon.a (mon.0) 4889 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:28.585 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:27 smithi100 bash[76202]: audit 2024-04-07T17:51:26.118619+0000 mon.a (mon.0) 4890 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:28.585 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:27 smithi100 bash[76202]: audit 2024-04-07T17:51:26.127881+0000 mon.a (mon.0) 4891 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:28.585 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:27 smithi100 bash[76202]: audit 2024-04-07T17:51:26.141580+0000 mon.a (mon.0) 4892 : audit [INF] from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T17:51:28.585 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:27 smithi100 bash[76202]: audit 2024-04-07T17:51:26.141642+0000 mon.a (mon.0) 4893 : audit [INF] from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T17:51:28.585 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:27 smithi100 bash[76202]: audit 2024-04-07T17:51:26.141901+0000 mon.a (mon.0) 4894 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:28.585 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:27 smithi100 bash[76202]: audit 2024-04-07T17:51:26.142098+0000 mon.a (mon.0) 4895 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:28.585 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:27 smithi100 bash[76202]: audit 2024-04-07T17:51:26.142315+0000 mon.a (mon.0) 4896 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:28.585 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:27 smithi100 bash[76202]: audit 2024-04-07T17:51:26.145032+0000 mon.a (mon.0) 4897 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:28.585 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:27 smithi100 bash[76202]: audit 2024-04-07T17:51:26.145256+0000 mon.a (mon.0) 4898 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:28.585 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:27 smithi100 bash[76202]: audit 2024-04-07T17:51:26.145648+0000 mon.a (mon.0) 4899 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:28.585 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:27 smithi100 bash[76202]: audit 2024-04-07T17:51:26.147129+0000 mon.b (mon.1) 33 : audit [INF] from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T17:51:28.585 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:27 smithi100 bash[76202]: audit 2024-04-07T17:51:26.147250+0000 mon.b (mon.1) 34 : audit [INF] from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T17:51:28.585 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:27 smithi100 bash[76202]: audit 2024-04-07T17:51:26.147586+0000 mon.c (mon.2) 131 : audit [INF] from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T17:51:28.585 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:27 smithi100 bash[76202]: audit 2024-04-07T17:51:26.147683+0000 mon.c (mon.2) 132 : audit [INF] from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T17:51:28.586 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:27 smithi100 bash[76202]: audit 2024-04-07T17:51:26.147691+0000 mon.a (mon.0) 4900 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:28.586 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:27 smithi100 bash[74942]: audit 2024-04-07T17:51:26.148003+0000 mon.a (mon.0) 4901 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:28.586 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:27 smithi100 bash[74942]: audit 2024-04-07T17:51:26.148397+0000 mon.a (mon.0) 4902 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:28.586 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:27 smithi100 bash[74942]: audit 2024-04-07T17:51:26.222921+0000 mgr.y (mgr.25175) 2893 : audit [DBG] from='client.36117 -' entity='client.admin' cmd=[{"prefix": "telemetry show", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T17:51:28.586 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:27 smithi100 bash[74942]: audit 2024-04-07T17:51:26.225253+0000 mon.a (mon.0) 4903 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:51:28.586 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:27 smithi100 bash[74942]: audit 2024-04-07T17:51:26.230592+0000 mon.a (mon.0) 4904 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config-key dump", "format": "json"}]: dispatch 2024-04-07T17:51:28.587 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:27 smithi100 bash[74942]: audit 2024-04-07T17:51:26.394528+0000 mon.a (mon.0) 4905 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:28.587 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:27 smithi100 bash[74942]: audit 2024-04-07T17:51:26.395032+0000 mon.a (mon.0) 4906 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:28.587 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:27 smithi100 bash[74942]: audit 2024-04-07T17:51:26.395508+0000 mon.a (mon.0) 4907 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:28.587 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:27 smithi100 bash[74942]: audit 2024-04-07T17:51:26.398843+0000 mon.a (mon.0) 4908 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:28.587 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:27 smithi100 bash[74942]: audit 2024-04-07T17:51:26.399323+0000 mon.a (mon.0) 4909 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:28.587 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:27 smithi100 bash[74942]: audit 2024-04-07T17:51:26.399730+0000 mon.a (mon.0) 4910 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:28.587 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:27 smithi100 bash[74942]: audit 2024-04-07T17:51:26.403624+0000 mon.a (mon.0) 4911 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:28.587 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:27 smithi100 bash[74942]: audit 2024-04-07T17:51:26.404147+0000 mon.a (mon.0) 4912 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:28.587 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:27 smithi100 bash[74942]: audit 2024-04-07T17:51:26.404643+0000 mon.a (mon.0) 4913 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:28.587 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:27 smithi100 bash[74942]: audit 2024-04-07T17:51:26.424314+0000 mon.a (mon.0) 4914 : audit [INF] from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T17:51:28.587 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:27 smithi100 bash[74942]: audit 2024-04-07T17:51:26.424445+0000 mon.a (mon.0) 4915 : audit [INF] from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T17:51:28.587 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:27 smithi100 bash[74942]: audit 2024-04-07T17:51:26.424865+0000 mon.a (mon.0) 4916 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:28.587 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:27 smithi100 bash[74942]: audit 2024-04-07T17:51:26.425309+0000 mon.a (mon.0) 4917 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:28.588 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:27 smithi100 bash[76202]: audit 2024-04-07T17:51:26.148003+0000 mon.a (mon.0) 4901 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:28.588 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:27 smithi100 bash[76202]: audit 2024-04-07T17:51:26.148397+0000 mon.a (mon.0) 4902 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:28.588 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:27 smithi100 bash[76202]: audit 2024-04-07T17:51:26.222921+0000 mgr.y (mgr.25175) 2893 : audit [DBG] from='client.36117 -' entity='client.admin' cmd=[{"prefix": "telemetry show", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T17:51:28.588 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:27 smithi100 bash[76202]: audit 2024-04-07T17:51:26.225253+0000 mon.a (mon.0) 4903 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:51:28.588 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:27 smithi100 bash[76202]: audit 2024-04-07T17:51:26.230592+0000 mon.a (mon.0) 4904 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config-key dump", "format": "json"}]: dispatch 2024-04-07T17:51:28.588 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:27 smithi100 bash[76202]: audit 2024-04-07T17:51:26.394528+0000 mon.a (mon.0) 4905 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:28.588 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:27 smithi100 bash[76202]: audit 2024-04-07T17:51:26.395032+0000 mon.a (mon.0) 4906 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:28.588 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:27 smithi100 bash[76202]: audit 2024-04-07T17:51:26.395508+0000 mon.a (mon.0) 4907 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:28.588 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:27 smithi100 bash[76202]: audit 2024-04-07T17:51:26.398843+0000 mon.a (mon.0) 4908 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:28.588 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:27 smithi100 bash[76202]: audit 2024-04-07T17:51:26.399323+0000 mon.a (mon.0) 4909 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:28.588 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:27 smithi100 bash[76202]: audit 2024-04-07T17:51:26.399730+0000 mon.a (mon.0) 4910 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:28.588 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:27 smithi100 bash[76202]: audit 2024-04-07T17:51:26.403624+0000 mon.a (mon.0) 4911 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:28.588 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:27 smithi100 bash[76202]: audit 2024-04-07T17:51:26.404147+0000 mon.a (mon.0) 4912 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:28.588 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:27 smithi100 bash[76202]: audit 2024-04-07T17:51:26.404643+0000 mon.a (mon.0) 4913 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:28.589 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:27 smithi100 bash[76202]: audit 2024-04-07T17:51:26.424314+0000 mon.a (mon.0) 4914 : audit [INF] from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T17:51:28.589 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:27 smithi100 bash[76202]: audit 2024-04-07T17:51:26.424445+0000 mon.a (mon.0) 4915 : audit [INF] from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T17:51:28.589 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:27 smithi100 bash[76202]: audit 2024-04-07T17:51:26.424865+0000 mon.a (mon.0) 4916 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:28.589 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:27 smithi100 bash[76202]: audit 2024-04-07T17:51:26.425309+0000 mon.a (mon.0) 4917 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:28.589 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:27 smithi144 bash[66654]: audit 2024-04-07T17:51:25.644364+0000 mgr.y (mgr.25175) 2890 : audit [DBG] from='client.46007 -' entity='client.admin' cmd=[{"prefix": "telemetry on", "license": "sharing-1-0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T17:51:28.589 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:27 smithi144 bash[66654]: cluster 2024-04-07T17:51:25.692912+0000 mgr.y (mgr.25175) 2891 : cluster [DBG] pgmap v5170: 105 pgs: 105 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 659 B/s rd, 1 op/s 2024-04-07T17:51:28.589 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:27 smithi144 bash[66654]: audit 2024-04-07T17:51:25.830845+0000 mgr.y (mgr.25175) 2892 : audit [DBG] from='client.46007 -' entity='client.admin' cmd=[{"prefix": "telemetry on", "license": "sharing-1-0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T17:51:28.589 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:27 smithi144 bash[66654]: audit 2024-04-07T17:51:26.118409+0000 mon.a (mon.0) 4889 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:28.590 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:27 smithi144 bash[66654]: audit 2024-04-07T17:51:26.118619+0000 mon.a (mon.0) 4890 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:28.590 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:27 smithi144 bash[66654]: audit 2024-04-07T17:51:26.127881+0000 mon.a (mon.0) 4891 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:28.590 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:27 smithi144 bash[66654]: audit 2024-04-07T17:51:26.141580+0000 mon.a (mon.0) 4892 : audit [INF] from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T17:51:28.590 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:27 smithi144 bash[66654]: audit 2024-04-07T17:51:26.141642+0000 mon.a (mon.0) 4893 : audit [INF] from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T17:51:28.590 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:27 smithi144 bash[66654]: audit 2024-04-07T17:51:26.141901+0000 mon.a (mon.0) 4894 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:28.590 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:27 smithi144 bash[66654]: audit 2024-04-07T17:51:26.142098+0000 mon.a (mon.0) 4895 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:28.590 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:27 smithi144 bash[66654]: audit 2024-04-07T17:51:26.142315+0000 mon.a (mon.0) 4896 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:28.590 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:27 smithi144 bash[66654]: audit 2024-04-07T17:51:26.145032+0000 mon.a (mon.0) 4897 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:28.590 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:27 smithi144 bash[66654]: audit 2024-04-07T17:51:26.145256+0000 mon.a (mon.0) 4898 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:28.590 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:27 smithi144 bash[66654]: audit 2024-04-07T17:51:26.145648+0000 mon.a (mon.0) 4899 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:28.590 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:27 smithi144 bash[66654]: audit 2024-04-07T17:51:26.147129+0000 mon.b (mon.1) 33 : audit [INF] from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T17:51:28.590 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:27 smithi144 bash[66654]: audit 2024-04-07T17:51:26.147250+0000 mon.b (mon.1) 34 : audit [INF] from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T17:51:28.590 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:27 smithi144 bash[66654]: audit 2024-04-07T17:51:26.147586+0000 mon.c (mon.2) 131 : audit [INF] from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T17:51:28.591 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:27 smithi144 bash[66654]: audit 2024-04-07T17:51:26.147683+0000 mon.c (mon.2) 132 : audit [INF] from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T17:51:28.591 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:27 smithi144 bash[66654]: audit 2024-04-07T17:51:26.147691+0000 mon.a (mon.0) 4900 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:28.591 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:27 smithi144 bash[66654]: audit 2024-04-07T17:51:26.148003+0000 mon.a (mon.0) 4901 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:28.591 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:27 smithi144 bash[66654]: audit 2024-04-07T17:51:26.148397+0000 mon.a (mon.0) 4902 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:28.591 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:27 smithi144 bash[66654]: audit 2024-04-07T17:51:26.222921+0000 mgr.y (mgr.25175) 2893 : audit [DBG] from='client.36117 -' entity='client.admin' cmd=[{"prefix": "telemetry show", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T17:51:28.591 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:27 smithi144 bash[66654]: audit 2024-04-07T17:51:26.225253+0000 mon.a (mon.0) 4903 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:51:28.591 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:27 smithi144 bash[66654]: audit 2024-04-07T17:51:26.230592+0000 mon.a (mon.0) 4904 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config-key dump", "format": "json"}]: dispatch 2024-04-07T17:51:28.591 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:27 smithi144 bash[66654]: audit 2024-04-07T17:51:26.394528+0000 mon.a (mon.0) 4905 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:28.591 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:27 smithi144 bash[66654]: audit 2024-04-07T17:51:26.395032+0000 mon.a (mon.0) 4906 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:28.591 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:27 smithi144 bash[66654]: audit 2024-04-07T17:51:26.395508+0000 mon.a (mon.0) 4907 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:28.591 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:27 smithi144 bash[66654]: audit 2024-04-07T17:51:26.398843+0000 mon.a (mon.0) 4908 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:28.591 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:27 smithi144 bash[66654]: audit 2024-04-07T17:51:26.399323+0000 mon.a (mon.0) 4909 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:28.591 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:27 smithi144 bash[66654]: audit 2024-04-07T17:51:26.399730+0000 mon.a (mon.0) 4910 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:28.591 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:27 smithi144 bash[66654]: audit 2024-04-07T17:51:26.403624+0000 mon.a (mon.0) 4911 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:28.591 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:27 smithi144 bash[66654]: audit 2024-04-07T17:51:26.404147+0000 mon.a (mon.0) 4912 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:28.592 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:27 smithi144 bash[66654]: audit 2024-04-07T17:51:26.404643+0000 mon.a (mon.0) 4913 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:28.592 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:27 smithi144 bash[66654]: audit 2024-04-07T17:51:26.424314+0000 mon.a (mon.0) 4914 : audit [INF] from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T17:51:28.592 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:27 smithi144 bash[66654]: audit 2024-04-07T17:51:26.424445+0000 mon.a (mon.0) 4915 : audit [INF] from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T17:51:28.592 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:27 smithi144 bash[66654]: audit 2024-04-07T17:51:26.424865+0000 mon.a (mon.0) 4916 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:28.592 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:27 smithi144 bash[66654]: audit 2024-04-07T17:51:26.425309+0000 mon.a (mon.0) 4917 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:28.592 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:27 smithi144 bash[66654]: audit 2024-04-07T17:51:26.425696+0000 mon.a (mon.0) 4918 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:28.592 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:27 smithi144 bash[66654]: audit 2024-04-07T17:51:26.429315+0000 mon.a (mon.0) 4919 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:28.592 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:27 smithi144 bash[66654]: audit 2024-04-07T17:51:26.430053+0000 mon.a (mon.0) 4920 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:28.592 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:27 smithi144 bash[66654]: audit 2024-04-07T17:51:26.430499+0000 mon.a (mon.0) 4921 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:28.592 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:27 smithi144 bash[66654]: audit 2024-04-07T17:51:26.431047+0000 mon.b (mon.1) 35 : audit [INF] from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T17:51:28.592 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:27 smithi144 bash[66654]: audit 2024-04-07T17:51:26.431150+0000 mon.b (mon.1) 36 : audit [INF] from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T17:51:28.592 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:27 smithi144 bash[66654]: audit 2024-04-07T17:51:26.433131+0000 mon.c (mon.2) 133 : audit [INF] from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T17:51:28.592 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:27 smithi144 bash[66654]: audit 2024-04-07T17:51:26.433308+0000 mon.c (mon.2) 134 : audit [INF] from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T17:51:28.592 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:27 smithi144 bash[66654]: audit 2024-04-07T17:51:26.434131+0000 mon.a (mon.0) 4922 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:28.593 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:27 smithi144 bash[66654]: audit 2024-04-07T17:51:26.434848+0000 mon.a (mon.0) 4923 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:28.593 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:27 smithi144 bash[66654]: audit 2024-04-07T17:51:26.435170+0000 mon.a (mon.0) 4924 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:28.593 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:27 smithi144 bash[66654]: cluster 2024-04-07T17:51:26.653768+0000 mon.a (mon.0) 4925 : cluster [INF] Health check cleared: TELEMETRY_CHANGED (was: Telemetry requires re-opt-in) 2024-04-07T17:51:28.593 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:27 smithi144 bash[66654]: cluster 2024-04-07T17:51:26.653829+0000 mon.a (mon.0) 4926 : cluster [INF] Cluster is now healthy 2024-04-07T17:51:28.593 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:27 smithi144 bash[66654]: audit 2024-04-07T17:51:26.718817+0000 mon.a (mon.0) 4927 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:51:28.593 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:27 smithi144 bash[66654]: audit 2024-04-07T17:51:26.725109+0000 mon.a (mon.0) 4928 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config-key dump", "format": "json"}]: dispatch 2024-04-07T17:51:28.593 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:27 smithi144 bash[66654]: audit 2024-04-07T17:51:26.936463+0000 mon.a (mon.0) 4929 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:28.593 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:27 smithi144 bash[66654]: audit 2024-04-07T17:51:26.937033+0000 mon.a (mon.0) 4930 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:28.593 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:27 smithi144 bash[66654]: audit 2024-04-07T17:51:26.937559+0000 mon.a (mon.0) 4931 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:28.593 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:27 smithi144 bash[66654]: audit 2024-04-07T17:51:26.941318+0000 mon.a (mon.0) 4932 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:28.593 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:27 smithi144 bash[66654]: audit 2024-04-07T17:51:26.941779+0000 mon.a (mon.0) 4933 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:28.593 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:27 smithi144 bash[66654]: audit 2024-04-07T17:51:26.942090+0000 mon.a (mon.0) 4934 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:28.593 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:27 smithi144 bash[66654]: audit 2024-04-07T17:51:26.945450+0000 mon.a (mon.0) 4935 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:28.593 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:27 smithi144 bash[66654]: audit 2024-04-07T17:51:26.945933+0000 mon.a (mon.0) 4936 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:28.593 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:27 smithi144 bash[66654]: audit 2024-04-07T17:51:26.946248+0000 mon.a (mon.0) 4937 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:28.593 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:27 smithi144 bash[66654]: audit 2024-04-07T17:51:26.965535+0000 mon.a (mon.0) 4938 : audit [INF] from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T17:51:28.594 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:27 smithi144 bash[66654]: audit 2024-04-07T17:51:26.965663+0000 mon.a (mon.0) 4939 : audit [INF] from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T17:51:28.594 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:27 smithi144 bash[66654]: audit 2024-04-07T17:51:26.966112+0000 mon.a (mon.0) 4940 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:28.594 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:27 smithi144 bash[66654]: audit 2024-04-07T17:51:26.966638+0000 mon.a (mon.0) 4941 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:28.594 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:27 smithi144 bash[66654]: audit 2024-04-07T17:51:26.967019+0000 mon.a (mon.0) 4942 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:28.594 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:27 smithi144 bash[66654]: audit 2024-04-07T17:51:26.970423+0000 mon.a (mon.0) 4943 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:28.594 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:27 smithi144 bash[66654]: audit 2024-04-07T17:51:26.971680+0000 mon.a (mon.0) 4944 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:28.594 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:27 smithi144 bash[66654]: audit 2024-04-07T17:51:26.972171+0000 mon.b (mon.1) 37 : audit [INF] from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T17:51:28.594 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:27 smithi144 bash[66654]: audit 2024-04-07T17:51:26.972185+0000 mon.a (mon.0) 4945 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:28.594 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:27 smithi144 bash[66654]: audit 2024-04-07T17:51:26.972292+0000 mon.b (mon.1) 38 : audit [INF] from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T17:51:28.594 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:27 smithi144 bash[66654]: audit 2024-04-07T17:51:26.975061+0000 mon.c (mon.2) 135 : audit [INF] from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T17:51:28.594 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:27 smithi144 bash[66654]: audit 2024-04-07T17:51:26.975211+0000 mon.c (mon.2) 136 : audit [INF] from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T17:51:28.594 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:27 smithi144 bash[66654]: audit 2024-04-07T17:51:26.975584+0000 mon.a (mon.0) 4946 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:28.594 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:27 smithi144 bash[66654]: audit 2024-04-07T17:51:26.976608+0000 mon.a (mon.0) 4947 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:28.595 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:27 smithi144 bash[66654]: audit 2024-04-07T17:51:26.977127+0000 mon.a (mon.0) 4948 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:28.595 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:27 smithi100 bash[74942]: audit 2024-04-07T17:51:26.425696+0000 mon.a (mon.0) 4918 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:28.595 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:27 smithi100 bash[74942]: audit 2024-04-07T17:51:26.429315+0000 mon.a (mon.0) 4919 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:28.595 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:27 smithi100 bash[74942]: audit 2024-04-07T17:51:26.430053+0000 mon.a (mon.0) 4920 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:28.595 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:27 smithi100 bash[74942]: audit 2024-04-07T17:51:26.430499+0000 mon.a (mon.0) 4921 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:28.595 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:27 smithi100 bash[74942]: audit 2024-04-07T17:51:26.431047+0000 mon.b (mon.1) 35 : audit [INF] from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T17:51:28.595 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:27 smithi100 bash[74942]: audit 2024-04-07T17:51:26.431150+0000 mon.b (mon.1) 36 : audit [INF] from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T17:51:28.595 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:27 smithi100 bash[74942]: audit 2024-04-07T17:51:26.433131+0000 mon.c (mon.2) 133 : audit [INF] from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T17:51:28.595 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:27 smithi100 bash[74942]: audit 2024-04-07T17:51:26.433308+0000 mon.c (mon.2) 134 : audit [INF] from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T17:51:28.595 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:27 smithi100 bash[74942]: audit 2024-04-07T17:51:26.434131+0000 mon.a (mon.0) 4922 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:28.595 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:27 smithi100 bash[74942]: audit 2024-04-07T17:51:26.434848+0000 mon.a (mon.0) 4923 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:28.595 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:27 smithi100 bash[74942]: audit 2024-04-07T17:51:26.435170+0000 mon.a (mon.0) 4924 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:28.595 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:27 smithi100 bash[74942]: cluster 2024-04-07T17:51:26.653768+0000 mon.a (mon.0) 4925 : cluster [INF] Health check cleared: TELEMETRY_CHANGED (was: Telemetry requires re-opt-in) 2024-04-07T17:51:28.595 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:27 smithi100 bash[74942]: cluster 2024-04-07T17:51:26.653829+0000 mon.a (mon.0) 4926 : cluster [INF] Cluster is now healthy 2024-04-07T17:51:28.596 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:27 smithi100 bash[74942]: audit 2024-04-07T17:51:26.718817+0000 mon.a (mon.0) 4927 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:51:28.596 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:27 smithi100 bash[74942]: audit 2024-04-07T17:51:26.725109+0000 mon.a (mon.0) 4928 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config-key dump", "format": "json"}]: dispatch 2024-04-07T17:51:28.596 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:27 smithi100 bash[74942]: audit 2024-04-07T17:51:26.936463+0000 mon.a (mon.0) 4929 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:28.596 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:27 smithi100 bash[74942]: audit 2024-04-07T17:51:26.937033+0000 mon.a (mon.0) 4930 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:28.596 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:27 smithi100 bash[74942]: audit 2024-04-07T17:51:26.937559+0000 mon.a (mon.0) 4931 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:28.596 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:27 smithi100 bash[74942]: audit 2024-04-07T17:51:26.941318+0000 mon.a (mon.0) 4932 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:28.596 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:27 smithi100 bash[74942]: audit 2024-04-07T17:51:26.941779+0000 mon.a (mon.0) 4933 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:28.596 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:27 smithi100 bash[76202]: audit 2024-04-07T17:51:26.425696+0000 mon.a (mon.0) 4918 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:28.596 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:27 smithi100 bash[76202]: audit 2024-04-07T17:51:26.429315+0000 mon.a (mon.0) 4919 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:28.596 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:27 smithi100 bash[76202]: audit 2024-04-07T17:51:26.430053+0000 mon.a (mon.0) 4920 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:28.597 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:27 smithi100 bash[76202]: audit 2024-04-07T17:51:26.430499+0000 mon.a (mon.0) 4921 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:28.597 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:27 smithi100 bash[76202]: audit 2024-04-07T17:51:26.431047+0000 mon.b (mon.1) 35 : audit [INF] from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T17:51:28.597 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:27 smithi100 bash[76202]: audit 2024-04-07T17:51:26.431150+0000 mon.b (mon.1) 36 : audit [INF] from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T17:51:28.597 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:27 smithi100 bash[76202]: audit 2024-04-07T17:51:26.433131+0000 mon.c (mon.2) 133 : audit [INF] from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T17:51:28.597 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:27 smithi100 bash[76202]: audit 2024-04-07T17:51:26.433308+0000 mon.c (mon.2) 134 : audit [INF] from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T17:51:28.597 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:27 smithi100 bash[76202]: audit 2024-04-07T17:51:26.434131+0000 mon.a (mon.0) 4922 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:28.597 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:27 smithi100 bash[76202]: audit 2024-04-07T17:51:26.434848+0000 mon.a (mon.0) 4923 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:28.597 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:27 smithi100 bash[76202]: audit 2024-04-07T17:51:26.435170+0000 mon.a (mon.0) 4924 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:28.597 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:27 smithi100 bash[76202]: cluster 2024-04-07T17:51:26.653768+0000 mon.a (mon.0) 4925 : cluster [INF] Health check cleared: TELEMETRY_CHANGED (was: Telemetry requires re-opt-in) 2024-04-07T17:51:28.597 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:27 smithi100 bash[76202]: cluster 2024-04-07T17:51:26.653829+0000 mon.a (mon.0) 4926 : cluster [INF] Cluster is now healthy 2024-04-07T17:51:28.597 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:27 smithi100 bash[76202]: audit 2024-04-07T17:51:26.718817+0000 mon.a (mon.0) 4927 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:51:28.597 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:27 smithi100 bash[76202]: audit 2024-04-07T17:51:26.725109+0000 mon.a (mon.0) 4928 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config-key dump", "format": "json"}]: dispatch 2024-04-07T17:51:28.597 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:27 smithi100 bash[76202]: audit 2024-04-07T17:51:26.936463+0000 mon.a (mon.0) 4929 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:28.597 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:27 smithi100 bash[76202]: audit 2024-04-07T17:51:26.937033+0000 mon.a (mon.0) 4930 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:28.597 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:27 smithi100 bash[76202]: audit 2024-04-07T17:51:26.937559+0000 mon.a (mon.0) 4931 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:28.598 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:27 smithi100 bash[76202]: audit 2024-04-07T17:51:26.941318+0000 mon.a (mon.0) 4932 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:28.598 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:27 smithi100 bash[76202]: audit 2024-04-07T17:51:26.941779+0000 mon.a (mon.0) 4933 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:28.598 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:27 smithi100 bash[74942]: audit 2024-04-07T17:51:26.942090+0000 mon.a (mon.0) 4934 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:28.598 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:27 smithi100 bash[74942]: audit 2024-04-07T17:51:26.945450+0000 mon.a (mon.0) 4935 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:28.598 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:27 smithi100 bash[74942]: audit 2024-04-07T17:51:26.945933+0000 mon.a (mon.0) 4936 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:28.598 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:27 smithi100 bash[74942]: audit 2024-04-07T17:51:26.946248+0000 mon.a (mon.0) 4937 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:28.598 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:27 smithi100 bash[74942]: audit 2024-04-07T17:51:26.965535+0000 mon.a (mon.0) 4938 : audit [INF] from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T17:51:28.598 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:27 smithi100 bash[74942]: audit 2024-04-07T17:51:26.965663+0000 mon.a (mon.0) 4939 : audit [INF] from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T17:51:28.598 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:27 smithi100 bash[74942]: audit 2024-04-07T17:51:26.966112+0000 mon.a (mon.0) 4940 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:28.598 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:27 smithi100 bash[74942]: audit 2024-04-07T17:51:26.966638+0000 mon.a (mon.0) 4941 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:28.599 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:27 smithi100 bash[74942]: audit 2024-04-07T17:51:26.967019+0000 mon.a (mon.0) 4942 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:28.599 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:27 smithi100 bash[74942]: audit 2024-04-07T17:51:26.970423+0000 mon.a (mon.0) 4943 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:28.599 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:27 smithi100 bash[74942]: audit 2024-04-07T17:51:26.971680+0000 mon.a (mon.0) 4944 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:28.599 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:27 smithi100 bash[74942]: audit 2024-04-07T17:51:26.972171+0000 mon.b (mon.1) 37 : audit [INF] from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T17:51:28.599 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:27 smithi100 bash[74942]: audit 2024-04-07T17:51:26.972185+0000 mon.a (mon.0) 4945 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:28.599 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:27 smithi100 bash[74942]: audit 2024-04-07T17:51:26.972292+0000 mon.b (mon.1) 38 : audit [INF] from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T17:51:28.599 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:27 smithi100 bash[74942]: audit 2024-04-07T17:51:26.975061+0000 mon.c (mon.2) 135 : audit [INF] from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T17:51:28.599 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:27 smithi100 bash[74942]: audit 2024-04-07T17:51:26.975211+0000 mon.c (mon.2) 136 : audit [INF] from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T17:51:28.599 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:27 smithi100 bash[74942]: audit 2024-04-07T17:51:26.975584+0000 mon.a (mon.0) 4946 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:28.599 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:27 smithi100 bash[74942]: audit 2024-04-07T17:51:26.976608+0000 mon.a (mon.0) 4947 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:28.599 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:27 smithi100 bash[74942]: audit 2024-04-07T17:51:26.977127+0000 mon.a (mon.0) 4948 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:28.600 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:27 smithi100 bash[76202]: audit 2024-04-07T17:51:26.942090+0000 mon.a (mon.0) 4934 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:28.600 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:27 smithi100 bash[76202]: audit 2024-04-07T17:51:26.945450+0000 mon.a (mon.0) 4935 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:28.600 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:27 smithi100 bash[76202]: audit 2024-04-07T17:51:26.945933+0000 mon.a (mon.0) 4936 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:28.600 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:27 smithi100 bash[76202]: audit 2024-04-07T17:51:26.946248+0000 mon.a (mon.0) 4937 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:28.600 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:27 smithi100 bash[76202]: audit 2024-04-07T17:51:26.965535+0000 mon.a (mon.0) 4938 : audit [INF] from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T17:51:28.600 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:27 smithi100 bash[76202]: audit 2024-04-07T17:51:26.965663+0000 mon.a (mon.0) 4939 : audit [INF] from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T17:51:28.600 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:27 smithi100 bash[76202]: audit 2024-04-07T17:51:26.966112+0000 mon.a (mon.0) 4940 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:28.600 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:27 smithi100 bash[76202]: audit 2024-04-07T17:51:26.966638+0000 mon.a (mon.0) 4941 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:28.600 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:27 smithi100 bash[76202]: audit 2024-04-07T17:51:26.967019+0000 mon.a (mon.0) 4942 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:28.600 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:27 smithi100 bash[76202]: audit 2024-04-07T17:51:26.970423+0000 mon.a (mon.0) 4943 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:28.600 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:27 smithi100 bash[76202]: audit 2024-04-07T17:51:26.971680+0000 mon.a (mon.0) 4944 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:28.600 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:27 smithi100 bash[76202]: audit 2024-04-07T17:51:26.972171+0000 mon.b (mon.1) 37 : audit [INF] from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T17:51:28.600 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:27 smithi100 bash[76202]: audit 2024-04-07T17:51:26.972185+0000 mon.a (mon.0) 4945 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:28.600 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:27 smithi100 bash[76202]: audit 2024-04-07T17:51:26.972292+0000 mon.b (mon.1) 38 : audit [INF] from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T17:51:28.601 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:27 smithi100 bash[76202]: audit 2024-04-07T17:51:26.975061+0000 mon.c (mon.2) 135 : audit [INF] from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T17:51:28.601 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:27 smithi100 bash[76202]: audit 2024-04-07T17:51:26.975211+0000 mon.c (mon.2) 136 : audit [INF] from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T17:51:28.601 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:27 smithi100 bash[76202]: audit 2024-04-07T17:51:26.975584+0000 mon.a (mon.0) 4946 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:28.601 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:27 smithi100 bash[76202]: audit 2024-04-07T17:51:26.976608+0000 mon.a (mon.0) 4947 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:28.601 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:27 smithi100 bash[76202]: audit 2024-04-07T17:51:26.977127+0000 mon.a (mon.0) 4948 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:28.601 INFO:tasks.workunit.client.0.smithi100.stdout:{ 2024-04-07T17:51:28.601 INFO:tasks.workunit.client.0.smithi100.stdout: "251e7b66-f4fa-11ee-8447-0cc47a8fd733": { 2024-04-07T17:51:28.601 INFO:tasks.workunit.client.0.smithi100.stdout: "INTEL_SSDPEDMD400G4_251ffd38-f4fa-11ee-8447-0cc47a8fd733": { 2024-04-07T17:51:28.601 INFO:tasks.workunit.client.0.smithi100.stdout: "20240407-160729": { 2024-04-07T17:51:28.601 INFO:tasks.workunit.client.0.smithi100.stdout: "device": { 2024-04-07T17:51:28.601 INFO:tasks.workunit.client.0.smithi100.stdout: "info_name": "/dev/nvme0n1", 2024-04-07T17:51:28.602 INFO:tasks.workunit.client.0.smithi100.stdout: "name": "/dev/nvme0n1", 2024-04-07T17:51:28.602 INFO:tasks.workunit.client.0.smithi100.stdout: "protocol": "NVMe", 2024-04-07T17:51:28.602 INFO:tasks.workunit.client.0.smithi100.stdout: "type": "nvme" 2024-04-07T17:51:28.602 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.602 INFO:tasks.workunit.client.0.smithi100.stdout: "firmware_version": "8DV101H0", 2024-04-07T17:51:28.602 INFO:tasks.workunit.client.0.smithi100.stdout: "host_id": "251e7b66-f4fa-11ee-8447-0cc47a8fd733", 2024-04-07T17:51:28.602 INFO:tasks.workunit.client.0.smithi100.stdout: "json_format_version": [ 2024-04-07T17:51:28.602 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:28.602 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T17:51:28.602 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.602 INFO:tasks.workunit.client.0.smithi100.stdout: "local_time": { 2024-04-07T17:51:28.602 INFO:tasks.workunit.client.0.smithi100.stdout: "asctime": "Sun Apr 7 16:07:28 2024 UTC", 2024-04-07T17:51:28.602 INFO:tasks.workunit.client.0.smithi100.stdout: "time_t": 1712506048 2024-04-07T17:51:28.602 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.603 INFO:tasks.workunit.client.0.smithi100.stdout: "logical_block_size": 512, 2024-04-07T17:51:28.603 INFO:tasks.workunit.client.0.smithi100.stdout: "model_name": "INTEL SSDPEDMD400G4", 2024-04-07T17:51:28.603 INFO:tasks.workunit.client.0.smithi100.stdout: "nvme_controller_id": 0, 2024-04-07T17:51:28.603 INFO:tasks.workunit.client.0.smithi100.stdout: "nvme_ieee_oui_identifier": 6083300, 2024-04-07T17:51:28.603 INFO:tasks.workunit.client.0.smithi100.stdout: "nvme_namespaces": [ 2024-04-07T17:51:28.603 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:28.603 INFO:tasks.workunit.client.0.smithi100.stdout: "capacity": { 2024-04-07T17:51:28.603 INFO:tasks.workunit.client.0.smithi100.stdout: "blocks": 781422768, 2024-04-07T17:51:28.603 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 400088457216 2024-04-07T17:51:28.603 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.603 INFO:tasks.workunit.client.0.smithi100.stdout: "formatted_lba_size": 512, 2024-04-07T17:51:28.603 INFO:tasks.workunit.client.0.smithi100.stdout: "id": 1, 2024-04-07T17:51:28.603 INFO:tasks.workunit.client.0.smithi100.stdout: "size": { 2024-04-07T17:51:28.603 INFO:tasks.workunit.client.0.smithi100.stdout: "blocks": 781422768, 2024-04-07T17:51:28.604 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 400088457216 2024-04-07T17:51:28.604 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.604 INFO:tasks.workunit.client.0.smithi100.stdout: "utilization": { 2024-04-07T17:51:28.604 INFO:tasks.workunit.client.0.smithi100.stdout: "blocks": 781422768, 2024-04-07T17:51:28.604 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 400088457216 2024-04-07T17:51:28.604 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:28.604 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:28.604 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.604 INFO:tasks.workunit.client.0.smithi100.stdout: "nvme_number_of_namespaces": 1, 2024-04-07T17:51:28.604 INFO:tasks.workunit.client.0.smithi100.stdout: "nvme_pci_vendor": { 2024-04-07T17:51:28.604 INFO:tasks.workunit.client.0.smithi100.stdout: "id": 32902, 2024-04-07T17:51:28.604 INFO:tasks.workunit.client.0.smithi100.stdout: "subsystem_id": 32902 2024-04-07T17:51:28.604 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.604 INFO:tasks.workunit.client.0.smithi100.stdout: "nvme_smart_health_information_add_log": { 2024-04-07T17:51:28.604 INFO:tasks.workunit.client.0.smithi100.stdout: "Device stats": { 2024-04-07T17:51:28.604 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_fw_downgrades": { 2024-04-07T17:51:28.604 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 0, 2024-04-07T17:51:28.605 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 0 2024-04-07T17:51:28.605 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.605 INFO:tasks.workunit.client.0.smithi100.stdout: "crc_error_count": { 2024-04-07T17:51:28.605 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 100, 2024-04-07T17:51:28.605 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 0 2024-04-07T17:51:28.605 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.605 INFO:tasks.workunit.client.0.smithi100.stdout: "end_to_end_error_detection_count": { 2024-04-07T17:51:28.605 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 100, 2024-04-07T17:51:28.605 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 0 2024-04-07T17:51:28.605 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.605 INFO:tasks.workunit.client.0.smithi100.stdout: "erase_fail_count": { 2024-04-07T17:51:28.605 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 100, 2024-04-07T17:51:28.605 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 0 2024-04-07T17:51:28.605 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.605 INFO:tasks.workunit.client.0.smithi100.stdout: "host_bytes_written": { 2024-04-07T17:51:28.605 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 100, 2024-04-07T17:51:28.606 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 29568720 2024-04-07T17:51:28.606 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.606 INFO:tasks.workunit.client.0.smithi100.stdout: "host_ctx_wear_used": { 2024-04-07T17:51:28.606 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 100, 2024-04-07T17:51:28.606 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 1 2024-04-07T17:51:28.606 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.606 INFO:tasks.workunit.client.0.smithi100.stdout: "media_bytes_read": { 2024-04-07T17:51:28.606 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 0, 2024-04-07T17:51:28.606 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 0 2024-04-07T17:51:28.606 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.606 INFO:tasks.workunit.client.0.smithi100.stdout: "nand_bytes_written": { 2024-04-07T17:51:28.606 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 100, 2024-04-07T17:51:28.606 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 47802746 2024-04-07T17:51:28.607 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.607 INFO:tasks.workunit.client.0.smithi100.stdout: "perf_stat_indicator": { 2024-04-07T17:51:28.607 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 0, 2024-04-07T17:51:28.607 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 0 2024-04-07T17:51:28.607 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.607 INFO:tasks.workunit.client.0.smithi100.stdout: "pll_lock_loss_count": { 2024-04-07T17:51:28.607 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 100, 2024-04-07T17:51:28.607 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 0 2024-04-07T17:51:28.607 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.607 INFO:tasks.workunit.client.0.smithi100.stdout: "program_fail_count": { 2024-04-07T17:51:28.607 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 100, 2024-04-07T17:51:28.607 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 0 2024-04-07T17:51:28.607 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.607 INFO:tasks.workunit.client.0.smithi100.stdout: "re_alloc_sectr_cnt": { 2024-04-07T17:51:28.607 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 0, 2024-04-07T17:51:28.608 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 0 2024-04-07T17:51:28.608 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.608 INFO:tasks.workunit.client.0.smithi100.stdout: "retry_buffer_overflow_count": { 2024-04-07T17:51:28.608 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 100, 2024-04-07T17:51:28.608 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 0 2024-04-07T17:51:28.608 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.608 INFO:tasks.workunit.client.0.smithi100.stdout: "soft_ecc_err_rate": { 2024-04-07T17:51:28.608 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 0, 2024-04-07T17:51:28.608 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 0 2024-04-07T17:51:28.608 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.608 INFO:tasks.workunit.client.0.smithi100.stdout: "thermal_throttle_status": { 2024-04-07T17:51:28.608 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 100, 2024-04-07T17:51:28.608 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": { 2024-04-07T17:51:28.608 INFO:tasks.workunit.client.0.smithi100.stdout: "cnt": 0, 2024-04-07T17:51:28.608 INFO:tasks.workunit.client.0.smithi100.stdout: "pct": 0 2024-04-07T17:51:28.608 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:28.609 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.609 INFO:tasks.workunit.client.0.smithi100.stdout: "timed_workload_host_reads": { 2024-04-07T17:51:28.609 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 100, 2024-04-07T17:51:28.609 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 65535 2024-04-07T17:51:28.609 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.609 INFO:tasks.workunit.client.0.smithi100.stdout: "timed_workload_media_wear": { 2024-04-07T17:51:28.609 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 100, 2024-04-07T17:51:28.609 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 64 2024-04-07T17:51:28.609 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.609 INFO:tasks.workunit.client.0.smithi100.stdout: "timed_workload_timer": { 2024-04-07T17:51:28.609 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 100, 2024-04-07T17:51:28.609 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 65535 2024-04-07T17:51:28.609 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.609 INFO:tasks.workunit.client.0.smithi100.stdout: "unexp_power_loss": { 2024-04-07T17:51:28.609 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 0, 2024-04-07T17:51:28.609 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 0 2024-04-07T17:51:28.609 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.610 INFO:tasks.workunit.client.0.smithi100.stdout: "wear_leveling": { 2024-04-07T17:51:28.610 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 90, 2024-04-07T17:51:28.610 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": { 2024-04-07T17:51:28.610 INFO:tasks.workunit.client.0.smithi100.stdout: "avg": 3209, 2024-04-07T17:51:28.610 INFO:tasks.workunit.client.0.smithi100.stdout: "max": 3225, 2024-04-07T17:51:28.610 INFO:tasks.workunit.client.0.smithi100.stdout: "min": 3194 2024-04-07T17:51:28.610 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:28.610 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:28.610 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.610 INFO:tasks.workunit.client.0.smithi100.stdout: "Intel Smart log": "nvme0n1" 2024-04-07T17:51:28.610 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.610 INFO:tasks.workunit.client.0.smithi100.stdout: "nvme_smart_health_information_log": { 2024-04-07T17:51:28.610 INFO:tasks.workunit.client.0.smithi100.stdout: "available_spare": 100, 2024-04-07T17:51:28.610 INFO:tasks.workunit.client.0.smithi100.stdout: "available_spare_threshold": 10, 2024-04-07T17:51:28.610 INFO:tasks.workunit.client.0.smithi100.stdout: "controller_busy_time": 4, 2024-04-07T17:51:28.610 INFO:tasks.workunit.client.0.smithi100.stdout: "critical_warning": 0, 2024-04-07T17:51:28.611 INFO:tasks.workunit.client.0.smithi100.stdout: "data_units_read": 385784931, 2024-04-07T17:51:28.611 INFO:tasks.workunit.client.0.smithi100.stdout: "data_units_written": 1937815661, 2024-04-07T17:51:28.611 INFO:tasks.workunit.client.0.smithi100.stdout: "host_reads": 5771442418, 2024-04-07T17:51:28.611 INFO:tasks.workunit.client.0.smithi100.stdout: "host_writes": 31722089956, 2024-04-07T17:51:28.611 INFO:tasks.workunit.client.0.smithi100.stdout: "media_errors": 0, 2024-04-07T17:51:28.611 INFO:tasks.workunit.client.0.smithi100.stdout: "num_err_log_entries": 0, 2024-04-07T17:51:28.611 INFO:tasks.workunit.client.0.smithi100.stdout: "percentage_used": 10, 2024-04-07T17:51:28.611 INFO:tasks.workunit.client.0.smithi100.stdout: "power_cycles": 45774, 2024-04-07T17:51:28.611 INFO:tasks.workunit.client.0.smithi100.stdout: "power_on_hours": 57978, 2024-04-07T17:51:28.611 INFO:tasks.workunit.client.0.smithi100.stdout: "temperature": 24, 2024-04-07T17:51:28.611 INFO:tasks.workunit.client.0.smithi100.stdout: "unsafe_shutdowns": 210 2024-04-07T17:51:28.611 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.611 INFO:tasks.workunit.client.0.smithi100.stdout: "nvme_vendor": "intel", 2024-04-07T17:51:28.611 INFO:tasks.workunit.client.0.smithi100.stdout: "power_cycle_count": 45774, 2024-04-07T17:51:28.611 INFO:tasks.workunit.client.0.smithi100.stdout: "power_on_time": { 2024-04-07T17:51:28.611 INFO:tasks.workunit.client.0.smithi100.stdout: "hours": 57978 2024-04-07T17:51:28.611 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.612 INFO:tasks.workunit.client.0.smithi100.stdout: "serial_number": "deleted", 2024-04-07T17:51:28.612 INFO:tasks.workunit.client.0.smithi100.stdout: "smart_status": { 2024-04-07T17:51:28.612 INFO:tasks.workunit.client.0.smithi100.stdout: "nvme": { 2024-04-07T17:51:28.612 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:28.612 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.612 INFO:tasks.workunit.client.0.smithi100.stdout: "passed": true 2024-04-07T17:51:28.612 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.612 INFO:tasks.workunit.client.0.smithi100.stdout: "smartctl": { 2024-04-07T17:51:28.612 INFO:tasks.workunit.client.0.smithi100.stdout: "argv": [ 2024-04-07T17:51:28.612 INFO:tasks.workunit.client.0.smithi100.stdout: "smartctl", 2024-04-07T17:51:28.612 INFO:tasks.workunit.client.0.smithi100.stdout: "-x", 2024-04-07T17:51:28.612 INFO:tasks.workunit.client.0.smithi100.stdout: "--json=o", 2024-04-07T17:51:28.612 INFO:tasks.workunit.client.0.smithi100.stdout: "/dev/nvme0n1" 2024-04-07T17:51:28.612 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.612 INFO:tasks.workunit.client.0.smithi100.stdout: "build_info": "(local build)", 2024-04-07T17:51:28.612 INFO:tasks.workunit.client.0.smithi100.stdout: "exit_status": 0, 2024-04-07T17:51:28.613 INFO:tasks.workunit.client.0.smithi100.stdout: "output": [ 2024-04-07T17:51:28.613 INFO:tasks.workunit.client.0.smithi100.stdout: "smartctl 7.1 2019-12-30 r5022 [x86_64-linux-5.15.0-101-generic] (local build)", 2024-04-07T17:51:28.613 INFO:tasks.workunit.client.0.smithi100.stdout: "Copyright (C) 2002-19, Bruce Allen, Christian Franke, www.smartmontools.org", 2024-04-07T17:51:28.613 INFO:tasks.workunit.client.0.smithi100.stdout: "", 2024-04-07T17:51:28.613 INFO:tasks.workunit.client.0.smithi100.stdout: "=== START OF INFORMATION SECTION ===", 2024-04-07T17:51:28.613 INFO:tasks.workunit.client.0.smithi100.stdout: "Model Number: INTEL SSDPEDMD400G4", 2024-04-07T17:51:28.613 INFO:tasks.workunit.client.0.smithi100.stdout: "Serial Number: deleted", 2024-04-07T17:51:28.613 INFO:tasks.workunit.client.0.smithi100.stdout: "Firmware Version: 8DV101H0", 2024-04-07T17:51:28.613 INFO:tasks.workunit.client.0.smithi100.stdout: "PCI Vendor/Subsystem ID: 0x8086", 2024-04-07T17:51:28.613 INFO:tasks.workunit.client.0.smithi100.stdout: "IEEE OUI Identifier: 0x5cd2e4", 2024-04-07T17:51:28.613 INFO:tasks.workunit.client.0.smithi100.stdout: "Controller ID: 0", 2024-04-07T17:51:28.613 INFO:tasks.workunit.client.0.smithi100.stdout: "Number of Namespaces: 1", 2024-04-07T17:51:28.613 INFO:tasks.workunit.client.0.smithi100.stdout: "Namespace 1 Size/Capacity: 400,088,457,216 [400 GB]", 2024-04-07T17:51:28.613 INFO:tasks.workunit.client.0.smithi100.stdout: "Namespace 1 Formatted LBA Size: 512", 2024-04-07T17:51:28.613 INFO:tasks.workunit.client.0.smithi100.stdout: "Local Time is: Sun Apr 7 16:07:28 2024 UTC", 2024-04-07T17:51:28.613 INFO:tasks.workunit.client.0.smithi100.stdout: "Firmware Updates (0x02): 1 Slot", 2024-04-07T17:51:28.613 INFO:tasks.workunit.client.0.smithi100.stdout: "Optional Admin Commands (0x0006): Format Frmw_DL", 2024-04-07T17:51:28.614 INFO:tasks.workunit.client.0.smithi100.stdout: "Optional NVM Commands (0x0006): Wr_Unc DS_Mngmt", 2024-04-07T17:51:28.614 INFO:tasks.workunit.client.0.smithi100.stdout: "Maximum Data Transfer Size: 32 Pages", 2024-04-07T17:51:28.614 INFO:tasks.workunit.client.0.smithi100.stdout: "", 2024-04-07T17:51:28.614 INFO:tasks.workunit.client.0.smithi100.stdout: "Supported Power States", 2024-04-07T17:51:28.614 INFO:tasks.workunit.client.0.smithi100.stdout: "St Op Max Active Idle RL RT WL WT Ent_Lat Ex_Lat", 2024-04-07T17:51:28.614 INFO:tasks.workunit.client.0.smithi100.stdout: " 0 + 25.00W - - 0 0 0 0 0 0", 2024-04-07T17:51:28.614 INFO:tasks.workunit.client.0.smithi100.stdout: "", 2024-04-07T17:51:28.614 INFO:tasks.workunit.client.0.smithi100.stdout: "Supported LBA Sizes (NSID 0x1)", 2024-04-07T17:51:28.614 INFO:tasks.workunit.client.0.smithi100.stdout: "Id Fmt Data Metadt Rel_Perf", 2024-04-07T17:51:28.614 INFO:tasks.workunit.client.0.smithi100.stdout: " 0 + 512 0 2", 2024-04-07T17:51:28.614 INFO:tasks.workunit.client.0.smithi100.stdout: " 1 - 512 8 2", 2024-04-07T17:51:28.614 INFO:tasks.workunit.client.0.smithi100.stdout: " 2 - 512 16 2", 2024-04-07T17:51:28.614 INFO:tasks.workunit.client.0.smithi100.stdout: " 3 - 4096 0 0", 2024-04-07T17:51:28.614 INFO:tasks.workunit.client.0.smithi100.stdout: " 4 - 4096 8 0", 2024-04-07T17:51:28.615 INFO:tasks.workunit.client.0.smithi100.stdout: " 5 - 4096 64 0", 2024-04-07T17:51:28.615 INFO:tasks.workunit.client.0.smithi100.stdout: " 6 - 4096 128 0", 2024-04-07T17:51:28.615 INFO:tasks.workunit.client.0.smithi100.stdout: "", 2024-04-07T17:51:28.615 INFO:tasks.workunit.client.0.smithi100.stdout: "=== START OF SMART DATA SECTION ===", 2024-04-07T17:51:28.615 INFO:tasks.workunit.client.0.smithi100.stdout: "SMART overall-health self-assessment test result: PASSED", 2024-04-07T17:51:28.615 INFO:tasks.workunit.client.0.smithi100.stdout: "", 2024-04-07T17:51:28.615 INFO:tasks.workunit.client.0.smithi100.stdout: "SMART/Health Information (NVMe Log 0x02)", 2024-04-07T17:51:28.615 INFO:tasks.workunit.client.0.smithi100.stdout: "Critical Warning: 0x00", 2024-04-07T17:51:28.615 INFO:tasks.workunit.client.0.smithi100.stdout: "Temperature: 24 Celsius", 2024-04-07T17:51:28.615 INFO:tasks.workunit.client.0.smithi100.stdout: "Available Spare: 100%", 2024-04-07T17:51:28.615 INFO:tasks.workunit.client.0.smithi100.stdout: "Available Spare Threshold: 10%", 2024-04-07T17:51:28.615 INFO:tasks.workunit.client.0.smithi100.stdout: "Percentage Used: 10%", 2024-04-07T17:51:28.615 INFO:tasks.workunit.client.0.smithi100.stdout: "Data Units Read: 385,784,931 [197 TB]", 2024-04-07T17:51:28.615 INFO:tasks.workunit.client.0.smithi100.stdout: "Data Units Written: 1,937,815,661 [992 TB]", 2024-04-07T17:51:28.615 INFO:tasks.workunit.client.0.smithi100.stdout: "Host Read Commands: 5,771,442,418", 2024-04-07T17:51:28.615 INFO:tasks.workunit.client.0.smithi100.stdout: "Host Write Commands: 31,722,089,956", 2024-04-07T17:51:28.615 INFO:tasks.workunit.client.0.smithi100.stdout: "Controller Busy Time: 4", 2024-04-07T17:51:28.616 INFO:tasks.workunit.client.0.smithi100.stdout: "Power Cycles: 45,774", 2024-04-07T17:51:28.616 INFO:tasks.workunit.client.0.smithi100.stdout: "Power On Hours: 57,978", 2024-04-07T17:51:28.616 INFO:tasks.workunit.client.0.smithi100.stdout: "Unsafe Shutdowns: 210", 2024-04-07T17:51:28.616 INFO:tasks.workunit.client.0.smithi100.stdout: "Media and Data Integrity Errors: 0", 2024-04-07T17:51:28.616 INFO:tasks.workunit.client.0.smithi100.stdout: "Error Information Log Entries: 0", 2024-04-07T17:51:28.616 INFO:tasks.workunit.client.0.smithi100.stdout: "", 2024-04-07T17:51:28.616 INFO:tasks.workunit.client.0.smithi100.stdout: "Error Information (NVMe Log 0x01, max 64 entries)", 2024-04-07T17:51:28.616 INFO:tasks.workunit.client.0.smithi100.stdout: "No Errors Logged", 2024-04-07T17:51:28.616 INFO:tasks.workunit.client.0.smithi100.stdout: "" 2024-04-07T17:51:28.616 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.616 INFO:tasks.workunit.client.0.smithi100.stdout: "platform_info": "x86_64-linux-5.15.0-101-generic", 2024-04-07T17:51:28.616 INFO:tasks.workunit.client.0.smithi100.stdout: "svn_revision": "5022", 2024-04-07T17:51:28.616 INFO:tasks.workunit.client.0.smithi100.stdout: "version": [ 2024-04-07T17:51:28.616 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:28.616 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T17:51:28.616 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:28.617 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.617 INFO:tasks.workunit.client.0.smithi100.stdout: "temperature": { 2024-04-07T17:51:28.617 INFO:tasks.workunit.client.0.smithi100.stdout: "current": 24 2024-04-07T17:51:28.617 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.617 INFO:tasks.workunit.client.0.smithi100.stdout: "user_capacity": { 2024-04-07T17:51:28.617 INFO:tasks.workunit.client.0.smithi100.stdout: "blocks": 781422768, 2024-04-07T17:51:28.617 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 400088457216 2024-04-07T17:51:28.617 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:28.617 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:28.617 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:28.617 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.617 INFO:tasks.workunit.client.0.smithi100.stdout: "25233610-f4fa-11ee-8447-0cc47a8fd733": { 2024-04-07T17:51:28.617 INFO:tasks.workunit.client.0.smithi100.stdout: "INTEL_SSDPEDMD400G4_2524b6c0-f4fa-11ee-8447-0cc47a8fd733": { 2024-04-07T17:51:28.617 INFO:tasks.workunit.client.0.smithi100.stdout: "20240407-160732": { 2024-04-07T17:51:28.617 INFO:tasks.workunit.client.0.smithi100.stdout: "device": { 2024-04-07T17:51:28.617 INFO:tasks.workunit.client.0.smithi100.stdout: "info_name": "/dev/nvme0n1", 2024-04-07T17:51:28.617 INFO:tasks.workunit.client.0.smithi100.stdout: "name": "/dev/nvme0n1", 2024-04-07T17:51:28.618 INFO:tasks.workunit.client.0.smithi100.stdout: "protocol": "NVMe", 2024-04-07T17:51:28.618 INFO:tasks.workunit.client.0.smithi100.stdout: "type": "nvme" 2024-04-07T17:51:28.618 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.618 INFO:tasks.workunit.client.0.smithi100.stdout: "firmware_version": "8DV101H0", 2024-04-07T17:51:28.618 INFO:tasks.workunit.client.0.smithi100.stdout: "host_id": "25233610-f4fa-11ee-8447-0cc47a8fd733", 2024-04-07T17:51:28.618 INFO:tasks.workunit.client.0.smithi100.stdout: "json_format_version": [ 2024-04-07T17:51:28.618 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:28.618 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T17:51:28.618 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.618 INFO:tasks.workunit.client.0.smithi100.stdout: "local_time": { 2024-04-07T17:51:28.618 INFO:tasks.workunit.client.0.smithi100.stdout: "asctime": "Sun Apr 7 16:07:32 2024 UTC", 2024-04-07T17:51:28.618 INFO:tasks.workunit.client.0.smithi100.stdout: "time_t": 1712506052 2024-04-07T17:51:28.618 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.618 INFO:tasks.workunit.client.0.smithi100.stdout: "logical_block_size": 512, 2024-04-07T17:51:28.619 INFO:tasks.workunit.client.0.smithi100.stdout: "model_name": "INTEL SSDPEDMD400G4", 2024-04-07T17:51:28.619 INFO:tasks.workunit.client.0.smithi100.stdout: "nvme_controller_id": 0, 2024-04-07T17:51:28.619 INFO:tasks.workunit.client.0.smithi100.stdout: "nvme_ieee_oui_identifier": 6083300, 2024-04-07T17:51:28.619 INFO:tasks.workunit.client.0.smithi100.stdout: "nvme_namespaces": [ 2024-04-07T17:51:28.619 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:28.619 INFO:tasks.workunit.client.0.smithi100.stdout: "capacity": { 2024-04-07T17:51:28.619 INFO:tasks.workunit.client.0.smithi100.stdout: "blocks": 781422768, 2024-04-07T17:51:28.619 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 400088457216 2024-04-07T17:51:28.619 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.619 INFO:tasks.workunit.client.0.smithi100.stdout: "formatted_lba_size": 512, 2024-04-07T17:51:28.619 INFO:tasks.workunit.client.0.smithi100.stdout: "id": 1, 2024-04-07T17:51:28.619 INFO:tasks.workunit.client.0.smithi100.stdout: "size": { 2024-04-07T17:51:28.619 INFO:tasks.workunit.client.0.smithi100.stdout: "blocks": 781422768, 2024-04-07T17:51:28.619 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 400088457216 2024-04-07T17:51:28.619 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.619 INFO:tasks.workunit.client.0.smithi100.stdout: "utilization": { 2024-04-07T17:51:28.620 INFO:tasks.workunit.client.0.smithi100.stdout: "blocks": 781422768, 2024-04-07T17:51:28.620 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 400088457216 2024-04-07T17:51:28.620 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:28.620 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:28.620 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.620 INFO:tasks.workunit.client.0.smithi100.stdout: "nvme_number_of_namespaces": 1, 2024-04-07T17:51:28.620 INFO:tasks.workunit.client.0.smithi100.stdout: "nvme_pci_vendor": { 2024-04-07T17:51:28.620 INFO:tasks.workunit.client.0.smithi100.stdout: "id": 32902, 2024-04-07T17:51:28.620 INFO:tasks.workunit.client.0.smithi100.stdout: "subsystem_id": 32902 2024-04-07T17:51:28.620 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.620 INFO:tasks.workunit.client.0.smithi100.stdout: "nvme_smart_health_information_add_log": { 2024-04-07T17:51:28.620 INFO:tasks.workunit.client.0.smithi100.stdout: "Device stats": { 2024-04-07T17:51:28.620 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_fw_downgrades": { 2024-04-07T17:51:28.620 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 0, 2024-04-07T17:51:28.620 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 0 2024-04-07T17:51:28.621 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.621 INFO:tasks.workunit.client.0.smithi100.stdout: "crc_error_count": { 2024-04-07T17:51:28.621 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 100, 2024-04-07T17:51:28.621 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 0 2024-04-07T17:51:28.621 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.621 INFO:tasks.workunit.client.0.smithi100.stdout: "end_to_end_error_detection_count": { 2024-04-07T17:51:28.621 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 100, 2024-04-07T17:51:28.621 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 0 2024-04-07T17:51:28.621 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.621 INFO:tasks.workunit.client.0.smithi100.stdout: "erase_fail_count": { 2024-04-07T17:51:28.621 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 100, 2024-04-07T17:51:28.621 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 0 2024-04-07T17:51:28.621 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.621 INFO:tasks.workunit.client.0.smithi100.stdout: "host_bytes_written": { 2024-04-07T17:51:28.621 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 100, 2024-04-07T17:51:28.622 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 25060599 2024-04-07T17:51:28.622 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.622 INFO:tasks.workunit.client.0.smithi100.stdout: "host_ctx_wear_used": { 2024-04-07T17:51:28.622 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 100, 2024-04-07T17:51:28.622 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 1 2024-04-07T17:51:28.622 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.622 INFO:tasks.workunit.client.0.smithi100.stdout: "media_bytes_read": { 2024-04-07T17:51:28.622 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 0, 2024-04-07T17:51:28.622 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 0 2024-04-07T17:51:28.622 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.622 INFO:tasks.workunit.client.0.smithi100.stdout: "nand_bytes_written": { 2024-04-07T17:51:28.622 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 100, 2024-04-07T17:51:28.622 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 42751216 2024-04-07T17:51:28.622 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.622 INFO:tasks.workunit.client.0.smithi100.stdout: "perf_stat_indicator": { 2024-04-07T17:51:28.623 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 0, 2024-04-07T17:51:28.623 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 0 2024-04-07T17:51:28.623 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.623 INFO:tasks.workunit.client.0.smithi100.stdout: "pll_lock_loss_count": { 2024-04-07T17:51:28.623 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 100, 2024-04-07T17:51:28.623 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 0 2024-04-07T17:51:28.623 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.623 INFO:tasks.workunit.client.0.smithi100.stdout: "program_fail_count": { 2024-04-07T17:51:28.623 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 100, 2024-04-07T17:51:28.623 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 0 2024-04-07T17:51:28.623 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.623 INFO:tasks.workunit.client.0.smithi100.stdout: "re_alloc_sectr_cnt": { 2024-04-07T17:51:28.623 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 0, 2024-04-07T17:51:28.623 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 0 2024-04-07T17:51:28.623 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.623 INFO:tasks.workunit.client.0.smithi100.stdout: "retry_buffer_overflow_count": { 2024-04-07T17:51:28.624 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 100, 2024-04-07T17:51:28.624 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 0 2024-04-07T17:51:28.624 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.624 INFO:tasks.workunit.client.0.smithi100.stdout: "soft_ecc_err_rate": { 2024-04-07T17:51:28.624 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 0, 2024-04-07T17:51:28.624 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 0 2024-04-07T17:51:28.624 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.624 INFO:tasks.workunit.client.0.smithi100.stdout: "thermal_throttle_status": { 2024-04-07T17:51:28.624 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 100, 2024-04-07T17:51:28.624 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": { 2024-04-07T17:51:28.624 INFO:tasks.workunit.client.0.smithi100.stdout: "cnt": 0, 2024-04-07T17:51:28.624 INFO:tasks.workunit.client.0.smithi100.stdout: "pct": 0 2024-04-07T17:51:28.624 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:28.624 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.624 INFO:tasks.workunit.client.0.smithi100.stdout: "timed_workload_host_reads": { 2024-04-07T17:51:28.625 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 100, 2024-04-07T17:51:28.625 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 65535 2024-04-07T17:51:28.625 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.625 INFO:tasks.workunit.client.0.smithi100.stdout: "timed_workload_media_wear": { 2024-04-07T17:51:28.625 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 100, 2024-04-07T17:51:28.625 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 64 2024-04-07T17:51:28.625 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.625 INFO:tasks.workunit.client.0.smithi100.stdout: "timed_workload_timer": { 2024-04-07T17:51:28.625 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 100, 2024-04-07T17:51:28.625 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 65535 2024-04-07T17:51:28.625 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.625 INFO:tasks.workunit.client.0.smithi100.stdout: "unexp_power_loss": { 2024-04-07T17:51:28.625 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 0, 2024-04-07T17:51:28.625 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": 0 2024-04-07T17:51:28.625 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.625 INFO:tasks.workunit.client.0.smithi100.stdout: "wear_leveling": { 2024-04-07T17:51:28.625 INFO:tasks.workunit.client.0.smithi100.stdout: "normalized": 91, 2024-04-07T17:51:28.625 INFO:tasks.workunit.client.0.smithi100.stdout: "raw": { 2024-04-07T17:51:28.626 INFO:tasks.workunit.client.0.smithi100.stdout: "avg": 2895, 2024-04-07T17:51:28.626 INFO:tasks.workunit.client.0.smithi100.stdout: "max": 2907, 2024-04-07T17:51:28.626 INFO:tasks.workunit.client.0.smithi100.stdout: "min": 2884 2024-04-07T17:51:28.626 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:28.626 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:28.626 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.626 INFO:tasks.workunit.client.0.smithi100.stdout: "Intel Smart log": "nvme0n1" 2024-04-07T17:51:28.626 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.626 INFO:tasks.workunit.client.0.smithi100.stdout: "nvme_smart_health_information_log": { 2024-04-07T17:51:28.626 INFO:tasks.workunit.client.0.smithi100.stdout: "available_spare": 100, 2024-04-07T17:51:28.626 INFO:tasks.workunit.client.0.smithi100.stdout: "available_spare_threshold": 10, 2024-04-07T17:51:28.626 INFO:tasks.workunit.client.0.smithi100.stdout: "controller_busy_time": 0, 2024-04-07T17:51:28.626 INFO:tasks.workunit.client.0.smithi100.stdout: "critical_warning": 0, 2024-04-07T17:51:28.626 INFO:tasks.workunit.client.0.smithi100.stdout: "data_units_read": 359011585, 2024-04-07T17:51:28.627 INFO:tasks.workunit.client.0.smithi100.stdout: "data_units_written": 1642371481, 2024-04-07T17:51:28.627 INFO:tasks.workunit.client.0.smithi100.stdout: "host_reads": 5750712954, 2024-04-07T17:51:28.627 INFO:tasks.workunit.client.0.smithi100.stdout: "host_writes": 28276536650, 2024-04-07T17:51:28.627 INFO:tasks.workunit.client.0.smithi100.stdout: "media_errors": 0, 2024-04-07T17:51:28.627 INFO:tasks.workunit.client.0.smithi100.stdout: "num_err_log_entries": 0, 2024-04-07T17:51:28.627 INFO:tasks.workunit.client.0.smithi100.stdout: "percentage_used": 9, 2024-04-07T17:51:28.627 INFO:tasks.workunit.client.0.smithi100.stdout: "power_cycles": 47059, 2024-04-07T17:51:28.627 INFO:tasks.workunit.client.0.smithi100.stdout: "power_on_hours": 54821, 2024-04-07T17:51:28.627 INFO:tasks.workunit.client.0.smithi100.stdout: "temperature": 25, 2024-04-07T17:51:28.627 INFO:tasks.workunit.client.0.smithi100.stdout: "unsafe_shutdowns": 12 2024-04-07T17:51:28.627 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.627 INFO:tasks.workunit.client.0.smithi100.stdout: "nvme_vendor": "intel", 2024-04-07T17:51:28.627 INFO:tasks.workunit.client.0.smithi100.stdout: "power_cycle_count": 47059, 2024-04-07T17:51:28.627 INFO:tasks.workunit.client.0.smithi100.stdout: "power_on_time": { 2024-04-07T17:51:28.627 INFO:tasks.workunit.client.0.smithi100.stdout: "hours": 54821 2024-04-07T17:51:28.627 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.628 INFO:tasks.workunit.client.0.smithi100.stdout: "serial_number": "deleted", 2024-04-07T17:51:28.628 INFO:tasks.workunit.client.0.smithi100.stdout: "smart_status": { 2024-04-07T17:51:28.628 INFO:tasks.workunit.client.0.smithi100.stdout: "nvme": { 2024-04-07T17:51:28.628 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:28.628 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.628 INFO:tasks.workunit.client.0.smithi100.stdout: "passed": true 2024-04-07T17:51:28.628 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.628 INFO:tasks.workunit.client.0.smithi100.stdout: "smartctl": { 2024-04-07T17:51:28.628 INFO:tasks.workunit.client.0.smithi100.stdout: "argv": [ 2024-04-07T17:51:28.628 INFO:tasks.workunit.client.0.smithi100.stdout: "smartctl", 2024-04-07T17:51:28.628 INFO:tasks.workunit.client.0.smithi100.stdout: "-x", 2024-04-07T17:51:28.628 INFO:tasks.workunit.client.0.smithi100.stdout: "--json=o", 2024-04-07T17:51:28.628 INFO:tasks.workunit.client.0.smithi100.stdout: "/dev/nvme0n1" 2024-04-07T17:51:28.628 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.628 INFO:tasks.workunit.client.0.smithi100.stdout: "build_info": "(local build)", 2024-04-07T17:51:28.629 INFO:tasks.workunit.client.0.smithi100.stdout: "exit_status": 0, 2024-04-07T17:51:28.629 INFO:tasks.workunit.client.0.smithi100.stdout: "output": [ 2024-04-07T17:51:28.629 INFO:tasks.workunit.client.0.smithi100.stdout: "smartctl 7.1 2019-12-30 r5022 [x86_64-linux-5.15.0-101-generic] (local build)", 2024-04-07T17:51:28.629 INFO:tasks.workunit.client.0.smithi100.stdout: "Copyright (C) 2002-19, Bruce Allen, Christian Franke, www.smartmontools.org", 2024-04-07T17:51:28.629 INFO:tasks.workunit.client.0.smithi100.stdout: "", 2024-04-07T17:51:28.629 INFO:tasks.workunit.client.0.smithi100.stdout: "=== START OF INFORMATION SECTION ===", 2024-04-07T17:51:28.629 INFO:tasks.workunit.client.0.smithi100.stdout: "Model Number: INTEL SSDPEDMD400G4", 2024-04-07T17:51:28.629 INFO:tasks.workunit.client.0.smithi100.stdout: "Serial Number: deleted", 2024-04-07T17:51:28.629 INFO:tasks.workunit.client.0.smithi100.stdout: "Firmware Version: 8DV101H0", 2024-04-07T17:51:28.629 INFO:tasks.workunit.client.0.smithi100.stdout: "PCI Vendor/Subsystem ID: 0x8086", 2024-04-07T17:51:28.629 INFO:tasks.workunit.client.0.smithi100.stdout: "IEEE OUI Identifier: 0x5cd2e4", 2024-04-07T17:51:28.629 INFO:tasks.workunit.client.0.smithi100.stdout: "Controller ID: 0", 2024-04-07T17:51:28.629 INFO:tasks.workunit.client.0.smithi100.stdout: "Number of Namespaces: 1", 2024-04-07T17:51:28.629 INFO:tasks.workunit.client.0.smithi100.stdout: "Namespace 1 Size/Capacity: 400,088,457,216 [400 GB]", 2024-04-07T17:51:28.629 INFO:tasks.workunit.client.0.smithi100.stdout: "Namespace 1 Formatted LBA Size: 512", 2024-04-07T17:51:28.629 INFO:tasks.workunit.client.0.smithi100.stdout: "Local Time is: Sun Apr 7 16:07:32 2024 UTC", 2024-04-07T17:51:28.629 INFO:tasks.workunit.client.0.smithi100.stdout: "Firmware Updates (0x02): 1 Slot", 2024-04-07T17:51:28.630 INFO:tasks.workunit.client.0.smithi100.stdout: "Optional Admin Commands (0x0006): Format Frmw_DL", 2024-04-07T17:51:28.630 INFO:tasks.workunit.client.0.smithi100.stdout: "Optional NVM Commands (0x0006): Wr_Unc DS_Mngmt", 2024-04-07T17:51:28.630 INFO:tasks.workunit.client.0.smithi100.stdout: "Maximum Data Transfer Size: 32 Pages", 2024-04-07T17:51:28.630 INFO:tasks.workunit.client.0.smithi100.stdout: "", 2024-04-07T17:51:28.630 INFO:tasks.workunit.client.0.smithi100.stdout: "Supported Power States", 2024-04-07T17:51:28.630 INFO:tasks.workunit.client.0.smithi100.stdout: "St Op Max Active Idle RL RT WL WT Ent_Lat Ex_Lat", 2024-04-07T17:51:28.630 INFO:tasks.workunit.client.0.smithi100.stdout: " 0 + 25.00W - - 0 0 0 0 0 0", 2024-04-07T17:51:28.630 INFO:tasks.workunit.client.0.smithi100.stdout: "", 2024-04-07T17:51:28.630 INFO:tasks.workunit.client.0.smithi100.stdout: "Supported LBA Sizes (NSID 0x1)", 2024-04-07T17:51:28.630 INFO:tasks.workunit.client.0.smithi100.stdout: "Id Fmt Data Metadt Rel_Perf", 2024-04-07T17:51:28.630 INFO:tasks.workunit.client.0.smithi100.stdout: " 0 + 512 0 2", 2024-04-07T17:51:28.630 INFO:tasks.workunit.client.0.smithi100.stdout: " 1 - 512 8 2", 2024-04-07T17:51:28.630 INFO:tasks.workunit.client.0.smithi100.stdout: " 2 - 512 16 2", 2024-04-07T17:51:28.630 INFO:tasks.workunit.client.0.smithi100.stdout: " 3 - 4096 0 0", 2024-04-07T17:51:28.631 INFO:tasks.workunit.client.0.smithi100.stdout: " 4 - 4096 8 0", 2024-04-07T17:51:28.631 INFO:tasks.workunit.client.0.smithi100.stdout: " 5 - 4096 64 0", 2024-04-07T17:51:28.631 INFO:tasks.workunit.client.0.smithi100.stdout: " 6 - 4096 128 0", 2024-04-07T17:51:28.631 INFO:tasks.workunit.client.0.smithi100.stdout: "", 2024-04-07T17:51:28.631 INFO:tasks.workunit.client.0.smithi100.stdout: "=== START OF SMART DATA SECTION ===", 2024-04-07T17:51:28.631 INFO:tasks.workunit.client.0.smithi100.stdout: "SMART overall-health self-assessment test result: PASSED", 2024-04-07T17:51:28.631 INFO:tasks.workunit.client.0.smithi100.stdout: "", 2024-04-07T17:51:28.631 INFO:tasks.workunit.client.0.smithi100.stdout: "SMART/Health Information (NVMe Log 0x02)", 2024-04-07T17:51:28.631 INFO:tasks.workunit.client.0.smithi100.stdout: "Critical Warning: 0x00", 2024-04-07T17:51:28.631 INFO:tasks.workunit.client.0.smithi100.stdout: "Temperature: 25 Celsius", 2024-04-07T17:51:28.631 INFO:tasks.workunit.client.0.smithi100.stdout: "Available Spare: 100%", 2024-04-07T17:51:28.631 INFO:tasks.workunit.client.0.smithi100.stdout: "Available Spare Threshold: 10%", 2024-04-07T17:51:28.631 INFO:tasks.workunit.client.0.smithi100.stdout: "Percentage Used: 9%", 2024-04-07T17:51:28.631 INFO:tasks.workunit.client.0.smithi100.stdout: "Data Units Read: 359,011,585 [183 TB]", 2024-04-07T17:51:28.631 INFO:tasks.workunit.client.0.smithi100.stdout: "Data Units Written: 1,642,371,481 [840 TB]", 2024-04-07T17:51:28.631 INFO:tasks.workunit.client.0.smithi100.stdout: "Host Read Commands: 5,750,712,954", 2024-04-07T17:51:28.632 INFO:tasks.workunit.client.0.smithi100.stdout: "Host Write Commands: 28,276,536,650", 2024-04-07T17:51:28.632 INFO:tasks.workunit.client.0.smithi100.stdout: "Controller Busy Time: 0", 2024-04-07T17:51:28.632 INFO:tasks.workunit.client.0.smithi100.stdout: "Power Cycles: 47,059", 2024-04-07T17:51:28.632 INFO:tasks.workunit.client.0.smithi100.stdout: "Power On Hours: 54,821", 2024-04-07T17:51:28.632 INFO:tasks.workunit.client.0.smithi100.stdout: "Unsafe Shutdowns: 12", 2024-04-07T17:51:28.632 INFO:tasks.workunit.client.0.smithi100.stdout: "Media and Data Integrity Errors: 0", 2024-04-07T17:51:28.632 INFO:tasks.workunit.client.0.smithi100.stdout: "Error Information Log Entries: 0", 2024-04-07T17:51:28.632 INFO:tasks.workunit.client.0.smithi100.stdout: "", 2024-04-07T17:51:28.632 INFO:tasks.workunit.client.0.smithi100.stdout: "Error Information (NVMe Log 0x01, max 64 entries)", 2024-04-07T17:51:28.632 INFO:tasks.workunit.client.0.smithi100.stdout: "No Errors Logged", 2024-04-07T17:51:28.632 INFO:tasks.workunit.client.0.smithi100.stdout: "" 2024-04-07T17:51:28.632 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.632 INFO:tasks.workunit.client.0.smithi100.stdout: "platform_info": "x86_64-linux-5.15.0-101-generic", 2024-04-07T17:51:28.632 INFO:tasks.workunit.client.0.smithi100.stdout: "svn_revision": "5022", 2024-04-07T17:51:28.632 INFO:tasks.workunit.client.0.smithi100.stdout: "version": [ 2024-04-07T17:51:28.633 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:28.633 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T17:51:28.633 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:28.633 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.633 INFO:tasks.workunit.client.0.smithi100.stdout: "temperature": { 2024-04-07T17:51:28.633 INFO:tasks.workunit.client.0.smithi100.stdout: "current": 25 2024-04-07T17:51:28.633 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.633 INFO:tasks.workunit.client.0.smithi100.stdout: "user_capacity": { 2024-04-07T17:51:28.633 INFO:tasks.workunit.client.0.smithi100.stdout: "blocks": 781422768, 2024-04-07T17:51:28.633 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 400088457216 2024-04-07T17:51:28.633 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:28.633 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:28.633 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:28.633 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:28.633 INFO:tasks.workunit.client.0.smithi100.stdout:} 2024-04-07T17:51:28.634 INFO:tasks.workunit.client.0.smithi100.stderr:+ ceph telemetry show 2024-04-07T17:51:28.634 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:28 smithi100 bash[74942]: audit 2024-04-07T17:51:26.715936+0000 mgr.y (mgr.25175) 2894 : audit [DBG] from='client.36117 -' entity='client.admin' cmd=[{"prefix": "telemetry show", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T17:51:28.634 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:28 smithi100 bash[74942]: audit 2024-04-07T17:51:28.075292+0000 mon.a (mon.0) 4949 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:51:28.634 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:28 smithi100 bash[74942]: audit 2024-04-07T17:51:28.082736+0000 mon.a (mon.0) 4950 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config-key dump", "format": "json"}]: dispatch 2024-04-07T17:51:28.634 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:28 smithi100 bash[76202]: audit 2024-04-07T17:51:26.715936+0000 mgr.y (mgr.25175) 2894 : audit [DBG] from='client.36117 -' entity='client.admin' cmd=[{"prefix": "telemetry show", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T17:51:28.634 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:28 smithi100 bash[76202]: audit 2024-04-07T17:51:28.075292+0000 mon.a (mon.0) 4949 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:51:28.635 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:28 smithi100 bash[76202]: audit 2024-04-07T17:51:28.082736+0000 mon.a (mon.0) 4950 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config-key dump", "format": "json"}]: dispatch 2024-04-07T17:51:28.635 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:28 smithi144 bash[66654]: audit 2024-04-07T17:51:26.715936+0000 mgr.y (mgr.25175) 2894 : audit [DBG] from='client.36117 -' entity='client.admin' cmd=[{"prefix": "telemetry show", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T17:51:28.635 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:28 smithi144 bash[66654]: audit 2024-04-07T17:51:28.075292+0000 mon.a (mon.0) 4949 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:51:28.637 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:28 smithi144 bash[66654]: audit 2024-04-07T17:51:28.082736+0000 mon.a (mon.0) 4950 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config-key dump", "format": "json"}]: dispatch 2024-04-07T17:51:28.703 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:51:28 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:51:28] "GET /metrics HTTP/1.1" 200 34269 "" "Prometheus/2.43.0" 2024-04-07T17:51:28.993 INFO:tasks.workunit.client.0.smithi100.stdout:{ 2024-04-07T17:51:28.993 INFO:tasks.workunit.client.0.smithi100.stdout: "balancer": { 2024-04-07T17:51:28.993 INFO:tasks.workunit.client.0.smithi100.stdout: "active": true, 2024-04-07T17:51:28.994 INFO:tasks.workunit.client.0.smithi100.stdout: "mode": "" 2024-04-07T17:51:28.994 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:28.994 INFO:tasks.workunit.client.0.smithi100.stdout: "channels": [ 2024-04-07T17:51:28.994 INFO:tasks.workunit.client.0.smithi100.stdout: "basic", 2024-04-07T17:51:28.994 INFO:tasks.workunit.client.0.smithi100.stdout: "crash", 2024-04-07T17:51:28.994 INFO:tasks.workunit.client.0.smithi100.stdout: "device", 2024-04-07T17:51:28.994 INFO:tasks.workunit.client.0.smithi100.stdout: "ident", 2024-04-07T17:51:28.994 INFO:tasks.workunit.client.0.smithi100.stdout: "perf" 2024-04-07T17:51:28.994 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.994 INFO:tasks.workunit.client.0.smithi100.stdout: "channels_available": [ 2024-04-07T17:51:28.994 INFO:tasks.workunit.client.0.smithi100.stdout: "basic", 2024-04-07T17:51:28.994 INFO:tasks.workunit.client.0.smithi100.stdout: "ident", 2024-04-07T17:51:28.994 INFO:tasks.workunit.client.0.smithi100.stdout: "crash", 2024-04-07T17:51:28.994 INFO:tasks.workunit.client.0.smithi100.stdout: "device", 2024-04-07T17:51:28.995 INFO:tasks.workunit.client.0.smithi100.stdout: "perf" 2024-04-07T17:51:28.995 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.995 INFO:tasks.workunit.client.0.smithi100.stdout: "collections_available": [ 2024-04-07T17:51:28.995 INFO:tasks.workunit.client.0.smithi100.stdout: "basic_base", 2024-04-07T17:51:28.995 INFO:tasks.workunit.client.0.smithi100.stdout: "device_base", 2024-04-07T17:51:28.995 INFO:tasks.workunit.client.0.smithi100.stdout: "crash_base", 2024-04-07T17:51:28.995 INFO:tasks.workunit.client.0.smithi100.stdout: "ident_base", 2024-04-07T17:51:28.995 INFO:tasks.workunit.client.0.smithi100.stdout: "perf_perf", 2024-04-07T17:51:28.995 INFO:tasks.workunit.client.0.smithi100.stdout: "basic_mds_metadata", 2024-04-07T17:51:28.995 INFO:tasks.workunit.client.0.smithi100.stdout: "basic_pool_usage", 2024-04-07T17:51:28.995 INFO:tasks.workunit.client.0.smithi100.stdout: "basic_usage_by_class", 2024-04-07T17:51:28.995 INFO:tasks.workunit.client.0.smithi100.stdout: "basic_rook_v01", 2024-04-07T17:51:28.995 INFO:tasks.workunit.client.0.smithi100.stdout: "perf_memory_metrics", 2024-04-07T17:51:28.995 INFO:tasks.workunit.client.0.smithi100.stdout: "basic_pool_options_bluestore", 2024-04-07T17:51:28.995 INFO:tasks.workunit.client.0.smithi100.stdout: "basic_pool_flags" 2024-04-07T17:51:28.995 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.996 INFO:tasks.workunit.client.0.smithi100.stdout: "collections_opted_in": [ 2024-04-07T17:51:28.996 INFO:tasks.workunit.client.0.smithi100.stdout: "basic_base", 2024-04-07T17:51:28.996 INFO:tasks.workunit.client.0.smithi100.stdout: "device_base", 2024-04-07T17:51:28.996 INFO:tasks.workunit.client.0.smithi100.stdout: "crash_base", 2024-04-07T17:51:28.996 INFO:tasks.workunit.client.0.smithi100.stdout: "ident_base", 2024-04-07T17:51:28.996 INFO:tasks.workunit.client.0.smithi100.stdout: "perf_perf", 2024-04-07T17:51:28.996 INFO:tasks.workunit.client.0.smithi100.stdout: "basic_mds_metadata", 2024-04-07T17:51:28.996 INFO:tasks.workunit.client.0.smithi100.stdout: "basic_pool_usage", 2024-04-07T17:51:28.996 INFO:tasks.workunit.client.0.smithi100.stdout: "basic_usage_by_class", 2024-04-07T17:51:28.996 INFO:tasks.workunit.client.0.smithi100.stdout: "basic_rook_v01", 2024-04-07T17:51:28.996 INFO:tasks.workunit.client.0.smithi100.stdout: "perf_memory_metrics", 2024-04-07T17:51:28.996 INFO:tasks.workunit.client.0.smithi100.stdout: "basic_pool_options_bluestore", 2024-04-07T17:51:28.996 INFO:tasks.workunit.client.0.smithi100.stdout: "basic_pool_flags" 2024-04-07T17:51:28.996 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:28.997 INFO:tasks.workunit.client.0.smithi100.stdout: "config": { 2024-04-07T17:51:28.997 INFO:tasks.workunit.client.0.smithi100.stdout: "active_changed": [ 2024-04-07T17:51:28.997 INFO:tasks.workunit.client.0.smithi100.stdout: "auth_allow_insecure_global_id_reclaim", 2024-04-07T17:51:28.997 INFO:tasks.workunit.client.0.smithi100.stdout: "auth_debug", 2024-04-07T17:51:28.997 INFO:tasks.workunit.client.0.smithi100.stdout: "auth_mon_ticket_ttl", 2024-04-07T17:51:28.997 INFO:tasks.workunit.client.0.smithi100.stdout: "auth_service_ticket_ttl", 2024-04-07T17:51:28.997 INFO:tasks.workunit.client.0.smithi100.stdout: "bdev_debug_aio", 2024-04-07T17:51:28.997 INFO:tasks.workunit.client.0.smithi100.stdout: "client_check_pool_perm", 2024-04-07T17:51:28.997 INFO:tasks.workunit.client.0.smithi100.stdout: "client_mount_gid", 2024-04-07T17:51:28.997 INFO:tasks.workunit.client.0.smithi100.stdout: "client_mount_uid", 2024-04-07T17:51:28.997 INFO:tasks.workunit.client.0.smithi100.stdout: "client_quota", 2024-04-07T17:51:28.997 INFO:tasks.workunit.client.0.smithi100.stdout: "container_image", 2024-04-07T17:51:28.997 INFO:tasks.workunit.client.0.smithi100.stdout: "daemonize", 2024-04-07T17:51:28.997 INFO:tasks.workunit.client.0.smithi100.stdout: "debug_asserts_on_shutdown", 2024-04-07T17:51:28.997 INFO:tasks.workunit.client.0.smithi100.stdout: "debug_mgr", 2024-04-07T17:51:28.998 INFO:tasks.workunit.client.0.smithi100.stdout: "debug_mon", 2024-04-07T17:51:28.998 INFO:tasks.workunit.client.0.smithi100.stdout: "debug_ms", 2024-04-07T17:51:28.998 INFO:tasks.workunit.client.0.smithi100.stdout: "debug_osd", 2024-04-07T17:51:28.998 INFO:tasks.workunit.client.0.smithi100.stdout: "debug_paxos", 2024-04-07T17:51:28.998 INFO:tasks.workunit.client.0.smithi100.stdout: "keyring", 2024-04-07T17:51:28.998 INFO:tasks.workunit.client.0.smithi100.stdout: "log_stderr_prefix", 2024-04-07T17:51:28.998 INFO:tasks.workunit.client.0.smithi100.stdout: "log_to_file", 2024-04-07T17:51:28.998 INFO:tasks.workunit.client.0.smithi100.stdout: "log_to_journald", 2024-04-07T17:51:28.998 INFO:tasks.workunit.client.0.smithi100.stdout: "log_to_stderr", 2024-04-07T17:51:28.998 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_allow_pool_delete", 2024-04-07T17:51:28.998 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_clock_drift_allowed", 2024-04-07T17:51:28.998 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_cluster_log_file_level", 2024-04-07T17:51:28.998 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_cluster_log_to_file", 2024-04-07T17:51:28.998 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_cluster_log_to_stderr", 2024-04-07T17:51:28.998 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_data_avail_warn", 2024-04-07T17:51:28.998 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_election_default_strategy", 2024-04-07T17:51:28.999 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_host", 2024-04-07T17:51:28.999 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_max_pg_per_osd", 2024-04-07T17:51:28.999 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_mds_skip_sanity", 2024-04-07T17:51:28.999 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_mgr_mkfs_grace", 2024-04-07T17:51:28.999 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_osd_prime_pg_temp", 2024-04-07T17:51:28.999 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_osd_reporter_subtree_level", 2024-04-07T17:51:28.999 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_pg_warn_max_object_skew", 2024-04-07T17:51:28.999 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_reweight_min_bytes_per_osd", 2024-04-07T17:51:28.999 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_reweight_min_pgs_per_osd", 2024-04-07T17:51:28.999 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_warn_on_crush_straw_calc_version_zero", 2024-04-07T17:51:28.999 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_warn_on_insecure_global_id_reclaim", 2024-04-07T17:51:28.999 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_warn_on_insecure_global_id_reclaim_allowed", 2024-04-07T17:51:28.999 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_warn_on_legacy_crush_tunables", 2024-04-07T17:51:28.999 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_warn_on_osd_down_out_interval_zero", 2024-04-07T17:51:28.999 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_warn_on_pool_pg_num_not_power_of_two", 2024-04-07T17:51:28.999 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_warn_on_too_few_osds", 2024-04-07T17:51:29.000 INFO:tasks.workunit.client.0.smithi100.stdout: "ms_die_on_bug", 2024-04-07T17:51:29.000 INFO:tasks.workunit.client.0.smithi100.stdout: "ms_die_on_old_message", 2024-04-07T17:51:29.000 INFO:tasks.workunit.client.0.smithi100.stdout: "no_config_file", 2024-04-07T17:51:29.000 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_class_default_list", 2024-04-07T17:51:29.000 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_class_load_list", 2024-04-07T17:51:29.000 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_debug_misdirected_ops", 2024-04-07T17:51:29.000 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_debug_op_order", 2024-04-07T17:51:29.000 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_debug_pg_log_writeout", 2024-04-07T17:51:29.000 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_debug_shutdown", 2024-04-07T17:51:29.000 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_debug_verify_cached_snaps", 2024-04-07T17:51:29.000 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_debug_verify_missing_on_start", 2024-04-07T17:51:29.000 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_debug_verify_stray_on_activate", 2024-04-07T17:51:29.000 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_deep_scrub_update_digest_min_age", 2024-04-07T17:51:29.000 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_delete_sleep", 2024-04-07T17:51:29.000 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_delete_sleep_hdd", 2024-04-07T17:51:29.001 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_delete_sleep_hybrid", 2024-04-07T17:51:29.001 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_delete_sleep_ssd", 2024-04-07T17:51:29.001 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_max_backfills", 2024-04-07T17:51:29.001 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mclock_max_capacity_iops_ssd", 2024-04-07T17:51:29.001 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mclock_profile", 2024-04-07T17:51:29.001 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mclock_scheduler_background_best_effort_lim", 2024-04-07T17:51:29.001 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mclock_scheduler_background_best_effort_res", 2024-04-07T17:51:29.001 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mclock_scheduler_background_best_effort_wgt", 2024-04-07T17:51:29.001 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mclock_scheduler_background_recovery_lim", 2024-04-07T17:51:29.001 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mclock_scheduler_background_recovery_res", 2024-04-07T17:51:29.001 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mclock_scheduler_background_recovery_wgt", 2024-04-07T17:51:29.001 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mclock_scheduler_client_lim", 2024-04-07T17:51:29.001 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mclock_scheduler_client_res", 2024-04-07T17:51:29.001 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mclock_scheduler_client_wgt", 2024-04-07T17:51:29.001 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_memory_target", 2024-04-07T17:51:29.001 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_memory_target_autotune", 2024-04-07T17:51:29.001 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_op_queue", 2024-04-07T17:51:29.002 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_op_queue_cut_off", 2024-04-07T17:51:29.002 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_pool_default_erasure_code_profile", 2024-04-07T17:51:29.002 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_pool_default_pg_autoscale_mode", 2024-04-07T17:51:29.002 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_recover_clone_overlap", 2024-04-07T17:51:29.002 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_recovery_max_active", 2024-04-07T17:51:29.002 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_recovery_max_active_hdd", 2024-04-07T17:51:29.002 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_recovery_max_active_ssd", 2024-04-07T17:51:29.002 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_recovery_max_chunk", 2024-04-07T17:51:29.002 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_recovery_sleep", 2024-04-07T17:51:29.002 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_recovery_sleep_hdd", 2024-04-07T17:51:29.002 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_recovery_sleep_hybrid", 2024-04-07T17:51:29.002 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_recovery_sleep_ssd", 2024-04-07T17:51:29.002 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_scrub_load_threshold", 2024-04-07T17:51:29.002 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_scrub_max_interval", 2024-04-07T17:51:29.002 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_scrub_sleep", 2024-04-07T17:51:29.003 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_shutdown_pgref_assert", 2024-04-07T17:51:29.003 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_snap_trim_sleep", 2024-04-07T17:51:29.003 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_snap_trim_sleep_hdd", 2024-04-07T17:51:29.003 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_snap_trim_sleep_hybrid", 2024-04-07T17:51:29.003 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_snap_trim_sleep_ssd", 2024-04-07T17:51:29.003 INFO:tasks.workunit.client.0.smithi100.stdout: "public_network", 2024-04-07T17:51:29.003 INFO:tasks.workunit.client.0.smithi100.stdout: "rbd_config_pool_override_update_timestamp", 2024-04-07T17:51:29.003 INFO:tasks.workunit.client.0.smithi100.stdout: "rbd_default_features", 2024-04-07T17:51:29.003 INFO:tasks.workunit.client.0.smithi100.stdout: "rbd_qos_exclude_ops", 2024-04-07T17:51:29.003 INFO:tasks.workunit.client.0.smithi100.stdout: "setgroup", 2024-04-07T17:51:29.003 INFO:tasks.workunit.client.0.smithi100.stdout: "setuser" 2024-04-07T17:51:29.003 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:29.003 INFO:tasks.workunit.client.0.smithi100.stdout: "cluster_changed": [ 2024-04-07T17:51:29.003 INFO:tasks.workunit.client.0.smithi100.stdout: "auth_allow_insecure_global_id_reclaim", 2024-04-07T17:51:29.003 INFO:tasks.workunit.client.0.smithi100.stdout: "auth_debug", 2024-04-07T17:51:29.003 INFO:tasks.workunit.client.0.smithi100.stdout: "auth_mon_ticket_ttl", 2024-04-07T17:51:29.004 INFO:tasks.workunit.client.0.smithi100.stdout: "auth_service_ticket_ttl", 2024-04-07T17:51:29.004 INFO:tasks.workunit.client.0.smithi100.stdout: "bdev_debug_aio", 2024-04-07T17:51:29.004 INFO:tasks.workunit.client.0.smithi100.stdout: "container_image", 2024-04-07T17:51:29.004 INFO:tasks.workunit.client.0.smithi100.stdout: "debug_asserts_on_shutdown", 2024-04-07T17:51:29.004 INFO:tasks.workunit.client.0.smithi100.stdout: "debug_mgr", 2024-04-07T17:51:29.004 INFO:tasks.workunit.client.0.smithi100.stdout: "debug_mon", 2024-04-07T17:51:29.004 INFO:tasks.workunit.client.0.smithi100.stdout: "debug_ms", 2024-04-07T17:51:29.004 INFO:tasks.workunit.client.0.smithi100.stdout: "debug_osd", 2024-04-07T17:51:29.004 INFO:tasks.workunit.client.0.smithi100.stdout: "debug_paxos", 2024-04-07T17:51:29.004 INFO:tasks.workunit.client.0.smithi100.stdout: "log_to_file", 2024-04-07T17:51:29.004 INFO:tasks.workunit.client.0.smithi100.stdout: "log_to_journald", 2024-04-07T17:51:29.004 INFO:tasks.workunit.client.0.smithi100.stdout: "log_to_stderr", 2024-04-07T17:51:29.004 INFO:tasks.workunit.client.0.smithi100.stdout: "mds_join_fs", 2024-04-07T17:51:29.004 INFO:tasks.workunit.client.0.smithi100.stdout: "mgr/cephadm/allow_ptrace", 2024-04-07T17:51:29.004 INFO:tasks.workunit.client.0.smithi100.stdout: "mgr/cephadm/container_init", 2024-04-07T17:51:29.004 INFO:tasks.workunit.client.0.smithi100.stdout: "mgr/cephadm/migration_current", 2024-04-07T17:51:29.004 INFO:tasks.workunit.client.0.smithi100.stdout: "mgr/cephadm/use_repo_digest", 2024-04-07T17:51:29.005 INFO:tasks.workunit.client.0.smithi100.stdout: "mgr/dashboard/ALERTMANAGER_API_HOST", 2024-04-07T17:51:29.005 INFO:tasks.workunit.client.0.smithi100.stdout: "mgr/dashboard/GRAFANA_API_SSL_VERIFY", 2024-04-07T17:51:29.005 INFO:tasks.workunit.client.0.smithi100.stdout: "mgr/dashboard/GRAFANA_API_URL", 2024-04-07T17:51:29.005 INFO:tasks.workunit.client.0.smithi100.stdout: "mgr/dashboard/PROMETHEUS_API_HOST", 2024-04-07T17:51:29.005 INFO:tasks.workunit.client.0.smithi100.stdout: "mgr/dashboard/ssl_server_port", 2024-04-07T17:51:29.005 INFO:tasks.workunit.client.0.smithi100.stdout: "mgr/orchestrator/orchestrator", 2024-04-07T17:51:29.005 INFO:tasks.workunit.client.0.smithi100.stdout: "mgr/telemetry/channel_ident", 2024-04-07T17:51:29.005 INFO:tasks.workunit.client.0.smithi100.stdout: "mgr/telemetry/channel_perf", 2024-04-07T17:51:29.005 INFO:tasks.workunit.client.0.smithi100.stdout: "mgr/telemetry/description", 2024-04-07T17:51:29.005 INFO:tasks.workunit.client.0.smithi100.stdout: "mgr/telemetry/enabled", 2024-04-07T17:51:29.005 INFO:tasks.workunit.client.0.smithi100.stdout: "mgr/telemetry/organization", 2024-04-07T17:51:29.005 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_allow_pool_delete", 2024-04-07T17:51:29.005 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_clock_drift_allowed", 2024-04-07T17:51:29.005 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_cluster_log_file_level", 2024-04-07T17:51:29.005 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_cluster_log_to_file", 2024-04-07T17:51:29.005 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_data_avail_warn", 2024-04-07T17:51:29.006 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_election_default_strategy", 2024-04-07T17:51:29.006 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_max_pg_per_osd", 2024-04-07T17:51:29.006 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_mds_skip_sanity", 2024-04-07T17:51:29.006 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_mgr_mkfs_grace", 2024-04-07T17:51:29.006 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_osd_prime_pg_temp", 2024-04-07T17:51:29.006 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_osd_reporter_subtree_level", 2024-04-07T17:51:29.006 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_pg_warn_max_object_skew", 2024-04-07T17:51:29.006 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_reweight_min_bytes_per_osd", 2024-04-07T17:51:29.006 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_reweight_min_pgs_per_osd", 2024-04-07T17:51:29.006 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_warn_on_crush_straw_calc_version_zero", 2024-04-07T17:51:29.006 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_warn_on_insecure_global_id_reclaim", 2024-04-07T17:51:29.006 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_warn_on_insecure_global_id_reclaim_allowed", 2024-04-07T17:51:29.006 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_warn_on_legacy_crush_tunables", 2024-04-07T17:51:29.006 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_warn_on_osd_down_out_interval_zero", 2024-04-07T17:51:29.007 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_warn_on_pool_pg_num_not_power_of_two", 2024-04-07T17:51:29.007 INFO:tasks.workunit.client.0.smithi100.stdout: "mon_warn_on_too_few_osds", 2024-04-07T17:51:29.007 INFO:tasks.workunit.client.0.smithi100.stdout: "ms_die_on_bug", 2024-04-07T17:51:29.007 INFO:tasks.workunit.client.0.smithi100.stdout: "ms_die_on_old_message", 2024-04-07T17:51:29.007 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_class_default_list", 2024-04-07T17:51:29.007 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_class_load_list", 2024-04-07T17:51:29.007 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_debug_misdirected_ops", 2024-04-07T17:51:29.007 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_debug_op_order", 2024-04-07T17:51:29.007 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_debug_pg_log_writeout", 2024-04-07T17:51:29.007 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_debug_shutdown", 2024-04-07T17:51:29.007 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_debug_verify_cached_snaps", 2024-04-07T17:51:29.007 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_debug_verify_missing_on_start", 2024-04-07T17:51:29.007 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_debug_verify_stray_on_activate", 2024-04-07T17:51:29.007 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_deep_scrub_update_digest_min_age", 2024-04-07T17:51:29.007 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mclock_max_capacity_iops_ssd", 2024-04-07T17:51:29.007 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mclock_profile", 2024-04-07T17:51:29.008 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_memory_target", 2024-04-07T17:51:29.008 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_memory_target_autotune", 2024-04-07T17:51:29.008 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_op_queue", 2024-04-07T17:51:29.008 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_op_queue_cut_off", 2024-04-07T17:51:29.008 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_pool_default_erasure_code_profile", 2024-04-07T17:51:29.008 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_pool_default_pg_autoscale_mode", 2024-04-07T17:51:29.008 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_recover_clone_overlap", 2024-04-07T17:51:29.008 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_recovery_max_chunk", 2024-04-07T17:51:29.008 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_scrub_load_threshold", 2024-04-07T17:51:29.008 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_scrub_max_interval", 2024-04-07T17:51:29.008 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_shutdown_pgref_assert", 2024-04-07T17:51:29.008 INFO:tasks.workunit.client.0.smithi100.stdout: "public_network", 2024-04-07T17:51:29.008 INFO:tasks.workunit.client.0.smithi100.stdout: "rbd_config_pool_override_update_timestamp", 2024-04-07T17:51:29.008 INFO:tasks.workunit.client.0.smithi100.stdout: "rgw_cache_enabled", 2024-04-07T17:51:29.008 INFO:tasks.workunit.client.0.smithi100.stdout: "rgw_enable_ops_log", 2024-04-07T17:51:29.009 INFO:tasks.workunit.client.0.smithi100.stdout: "rgw_enable_usage_log" 2024-04-07T17:51:29.009 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:29.009 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.009 INFO:tasks.workunit.client.0.smithi100.stdout: "contact": null, 2024-04-07T17:51:29.009 INFO:tasks.workunit.client.0.smithi100.stdout: "crashes": [], 2024-04-07T17:51:29.009 INFO:tasks.workunit.client.0.smithi100.stdout: "created": "2024-04-07T16:02:17.280630Z", 2024-04-07T17:51:29.009 INFO:tasks.workunit.client.0.smithi100.stdout: "crush": { 2024-04-07T17:51:29.009 INFO:tasks.workunit.client.0.smithi100.stdout: "bucket_algs": { 2024-04-07T17:51:29.009 INFO:tasks.workunit.client.0.smithi100.stdout: "straw2": 3 2024-04-07T17:51:29.009 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.009 INFO:tasks.workunit.client.0.smithi100.stdout: "bucket_sizes": { 2024-04-07T17:51:29.009 INFO:tasks.workunit.client.0.smithi100.stdout: "2": 1, 2024-04-07T17:51:29.009 INFO:tasks.workunit.client.0.smithi100.stdout: "4": 2 2024-04-07T17:51:29.009 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.009 INFO:tasks.workunit.client.0.smithi100.stdout: "bucket_types": { 2024-04-07T17:51:29.009 INFO:tasks.workunit.client.0.smithi100.stdout: "1": 2, 2024-04-07T17:51:29.010 INFO:tasks.workunit.client.0.smithi100.stdout: "11": 1 2024-04-07T17:51:29.010 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.010 INFO:tasks.workunit.client.0.smithi100.stdout: "compat_weight_set": false, 2024-04-07T17:51:29.010 INFO:tasks.workunit.client.0.smithi100.stdout: "device_classes": [ 2024-04-07T17:51:29.010 INFO:tasks.workunit.client.0.smithi100.stdout: 8 2024-04-07T17:51:29.010 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:29.010 INFO:tasks.workunit.client.0.smithi100.stdout: "num_buckets": 6, 2024-04-07T17:51:29.010 INFO:tasks.workunit.client.0.smithi100.stdout: "num_devices": 8, 2024-04-07T17:51:29.010 INFO:tasks.workunit.client.0.smithi100.stdout: "num_rules": 2, 2024-04-07T17:51:29.010 INFO:tasks.workunit.client.0.smithi100.stdout: "num_types": 12, 2024-04-07T17:51:29.010 INFO:tasks.workunit.client.0.smithi100.stdout: "num_weight_sets": 0, 2024-04-07T17:51:29.010 INFO:tasks.workunit.client.0.smithi100.stdout: "tunables": { 2024-04-07T17:51:29.010 INFO:tasks.workunit.client.0.smithi100.stdout: "allowed_bucket_algs": 54, 2024-04-07T17:51:29.010 INFO:tasks.workunit.client.0.smithi100.stdout: "choose_local_fallback_tries": 0, 2024-04-07T17:51:29.011 INFO:tasks.workunit.client.0.smithi100.stdout: "choose_local_tries": 0, 2024-04-07T17:51:29.011 INFO:tasks.workunit.client.0.smithi100.stdout: "choose_total_tries": 50, 2024-04-07T17:51:29.011 INFO:tasks.workunit.client.0.smithi100.stdout: "chooseleaf_descend_once": 1, 2024-04-07T17:51:29.011 INFO:tasks.workunit.client.0.smithi100.stdout: "chooseleaf_stable": 1, 2024-04-07T17:51:29.011 INFO:tasks.workunit.client.0.smithi100.stdout: "chooseleaf_vary_r": 1, 2024-04-07T17:51:29.011 INFO:tasks.workunit.client.0.smithi100.stdout: "has_msr_rules": 0, 2024-04-07T17:51:29.011 INFO:tasks.workunit.client.0.smithi100.stdout: "has_v2_rules": 1, 2024-04-07T17:51:29.011 INFO:tasks.workunit.client.0.smithi100.stdout: "has_v3_rules": 0, 2024-04-07T17:51:29.011 INFO:tasks.workunit.client.0.smithi100.stdout: "has_v4_buckets": 1, 2024-04-07T17:51:29.011 INFO:tasks.workunit.client.0.smithi100.stdout: "has_v5_rules": 0, 2024-04-07T17:51:29.011 INFO:tasks.workunit.client.0.smithi100.stdout: "legacy_tunables": 0, 2024-04-07T17:51:29.011 INFO:tasks.workunit.client.0.smithi100.stdout: "minimum_required_version": "jewel", 2024-04-07T17:51:29.011 INFO:tasks.workunit.client.0.smithi100.stdout: "msr_collision_tries": 100, 2024-04-07T17:51:29.011 INFO:tasks.workunit.client.0.smithi100.stdout: "msr_descents": 100, 2024-04-07T17:51:29.012 INFO:tasks.workunit.client.0.smithi100.stdout: "optimal_tunables": 1, 2024-04-07T17:51:29.012 INFO:tasks.workunit.client.0.smithi100.stdout: "profile": "jewel", 2024-04-07T17:51:29.012 INFO:tasks.workunit.client.0.smithi100.stdout: "require_feature_tunables": 1, 2024-04-07T17:51:29.012 INFO:tasks.workunit.client.0.smithi100.stdout: "require_feature_tunables2": 1, 2024-04-07T17:51:29.012 INFO:tasks.workunit.client.0.smithi100.stdout: "require_feature_tunables3": 1, 2024-04-07T17:51:29.012 INFO:tasks.workunit.client.0.smithi100.stdout: "require_feature_tunables5": 1, 2024-04-07T17:51:29.012 INFO:tasks.workunit.client.0.smithi100.stdout: "straw_calc_version": 1 2024-04-07T17:51:29.012 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.012 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.012 INFO:tasks.workunit.client.0.smithi100.stdout: "description": "upgrade test cluster", 2024-04-07T17:51:29.012 INFO:tasks.workunit.client.0.smithi100.stdout: "fs": { 2024-04-07T17:51:29.012 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1, 2024-04-07T17:51:29.012 INFO:tasks.workunit.client.0.smithi100.stdout: "feature_flags": { 2024-04-07T17:51:29.012 INFO:tasks.workunit.client.0.smithi100.stdout: "enable_multiple": true, 2024-04-07T17:51:29.012 INFO:tasks.workunit.client.0.smithi100.stdout: "ever_enabled_multiple": true 2024-04-07T17:51:29.013 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.013 INFO:tasks.workunit.client.0.smithi100.stdout: "filesystems": [ 2024-04-07T17:51:29.013 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:29.013 INFO:tasks.workunit.client.0.smithi100.stdout: "approx_ctime": "2024-04", 2024-04-07T17:51:29.013 INFO:tasks.workunit.client.0.smithi100.stdout: "balancer_enabled": false, 2024-04-07T17:51:29.013 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.013 INFO:tasks.workunit.client.0.smithi100.stdout: "cached_caps": 0, 2024-04-07T17:51:29.013 INFO:tasks.workunit.client.0.smithi100.stdout: "cached_dns": 10, 2024-04-07T17:51:29.013 INFO:tasks.workunit.client.0.smithi100.stdout: "cached_inos": 13, 2024-04-07T17:51:29.013 INFO:tasks.workunit.client.0.smithi100.stdout: "cached_subtrees": 2, 2024-04-07T17:51:29.013 INFO:tasks.workunit.client.0.smithi100.stdout: "ever_allowed_features": 0, 2024-04-07T17:51:29.013 INFO:tasks.workunit.client.0.smithi100.stdout: "explicitly_allowed_features": 0, 2024-04-07T17:51:29.013 INFO:tasks.workunit.client.0.smithi100.stdout: "files": 0, 2024-04-07T17:51:29.013 INFO:tasks.workunit.client.0.smithi100.stdout: "max_mds": 1, 2024-04-07T17:51:29.013 INFO:tasks.workunit.client.0.smithi100.stdout: "num_data_pools": 1, 2024-04-07T17:51:29.014 INFO:tasks.workunit.client.0.smithi100.stdout: "num_in": 1, 2024-04-07T17:51:29.014 INFO:tasks.workunit.client.0.smithi100.stdout: "num_mds": 1, 2024-04-07T17:51:29.014 INFO:tasks.workunit.client.0.smithi100.stdout: "num_sessions": 0, 2024-04-07T17:51:29.014 INFO:tasks.workunit.client.0.smithi100.stdout: "num_standby_replay": 0, 2024-04-07T17:51:29.014 INFO:tasks.workunit.client.0.smithi100.stdout: "num_up": 1, 2024-04-07T17:51:29.014 INFO:tasks.workunit.client.0.smithi100.stdout: "snaps": 0, 2024-04-07T17:51:29.014 INFO:tasks.workunit.client.0.smithi100.stdout: "standby_count_wanted": 0 2024-04-07T17:51:29.014 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.014 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:29.014 INFO:tasks.workunit.client.0.smithi100.stdout: "num_standby_mds": 1, 2024-04-07T17:51:29.014 INFO:tasks.workunit.client.0.smithi100.stdout: "total_num_mds": 2 2024-04-07T17:51:29.014 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.014 INFO:tasks.workunit.client.0.smithi100.stdout: "heap_stats": { 2024-04-07T17:51:29.014 INFO:tasks.workunit.client.0.smithi100.stdout: "mds": { 2024-04-07T17:51:29.015 INFO:tasks.workunit.client.0.smithi100.stdout: "mds.a52fc0d8acb515431879057649ff294cfe14c620": { 2024-04-07T17:51:29.015 INFO:tasks.workunit.client.0.smithi100.stdout: "actual_memory_used": 20258816, 2024-04-07T17:51:29.015 INFO:tasks.workunit.client.0.smithi100.stdout: "central_cache_freelist": 383520, 2024-04-07T17:51:29.015 INFO:tasks.workunit.client.0.smithi100.stdout: "malloc_metadata": 2752512, 2024-04-07T17:51:29.015 INFO:tasks.workunit.client.0.smithi100.stdout: "page_heap_freelist": 819200, 2024-04-07T17:51:29.015 INFO:tasks.workunit.client.0.smithi100.stdout: "released_to_os": 0, 2024-04-07T17:51:29.015 INFO:tasks.workunit.client.0.smithi100.stdout: "spans_in_use": 344, 2024-04-07T17:51:29.015 INFO:tasks.workunit.client.0.smithi100.stdout: "tcmalloc_page_size": 8192, 2024-04-07T17:51:29.015 INFO:tasks.workunit.client.0.smithi100.stdout: "thread_cache_freelists": 1117656, 2024-04-07T17:51:29.015 INFO:tasks.workunit.client.0.smithi100.stdout: "thread_heaps_in_use": 13, 2024-04-07T17:51:29.015 INFO:tasks.workunit.client.0.smithi100.stdout: "transfer_cache_freelist": 703488, 2024-04-07T17:51:29.015 INFO:tasks.workunit.client.0.smithi100.stdout: "use_by_application": 14482440, 2024-04-07T17:51:29.015 INFO:tasks.workunit.client.0.smithi100.stdout: "virtual_address_space_used": 20258816 2024-04-07T17:51:29.015 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.015 INFO:tasks.workunit.client.0.smithi100.stdout: "mds.f47a4ff15a45950a63160265564a123fc5da5e5e": { 2024-04-07T17:51:29.016 INFO:tasks.workunit.client.0.smithi100.stdout: "actual_memory_used": 20463616, 2024-04-07T17:51:29.016 INFO:tasks.workunit.client.0.smithi100.stdout: "central_cache_freelist": 437768, 2024-04-07T17:51:29.016 INFO:tasks.workunit.client.0.smithi100.stdout: "malloc_metadata": 2752512, 2024-04-07T17:51:29.016 INFO:tasks.workunit.client.0.smithi100.stdout: "page_heap_freelist": 0, 2024-04-07T17:51:29.016 INFO:tasks.workunit.client.0.smithi100.stdout: "released_to_os": 843776, 2024-04-07T17:51:29.016 INFO:tasks.workunit.client.0.smithi100.stdout: "spans_in_use": 430, 2024-04-07T17:51:29.016 INFO:tasks.workunit.client.0.smithi100.stdout: "tcmalloc_page_size": 8192, 2024-04-07T17:51:29.016 INFO:tasks.workunit.client.0.smithi100.stdout: "thread_cache_freelists": 1300096, 2024-04-07T17:51:29.016 INFO:tasks.workunit.client.0.smithi100.stdout: "thread_heaps_in_use": 22, 2024-04-07T17:51:29.016 INFO:tasks.workunit.client.0.smithi100.stdout: "transfer_cache_freelist": 1090560, 2024-04-07T17:51:29.016 INFO:tasks.workunit.client.0.smithi100.stdout: "use_by_application": 14882680, 2024-04-07T17:51:29.016 INFO:tasks.workunit.client.0.smithi100.stdout: "virtual_address_space_used": 21307392 2024-04-07T17:51:29.016 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.016 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.016 INFO:tasks.workunit.client.0.smithi100.stdout: "mon": { 2024-04-07T17:51:29.017 INFO:tasks.workunit.client.0.smithi100.stdout: "mon.8dd0baa036f0a7e6d1f3bdce86136fe527836c3d": { 2024-04-07T17:51:29.017 INFO:tasks.workunit.client.0.smithi100.stdout: "actual_memory_used": 172621824, 2024-04-07T17:51:29.017 INFO:tasks.workunit.client.0.smithi100.stdout: "central_cache_freelist": 2091000, 2024-04-07T17:51:29.017 INFO:tasks.workunit.client.0.smithi100.stdout: "malloc_metadata": 5111808, 2024-04-07T17:51:29.017 INFO:tasks.workunit.client.0.smithi100.stdout: "page_heap_freelist": 548864, 2024-04-07T17:51:29.017 INFO:tasks.workunit.client.0.smithi100.stdout: "released_to_os": 221184, 2024-04-07T17:51:29.017 INFO:tasks.workunit.client.0.smithi100.stdout: "spans_in_use": 6337, 2024-04-07T17:51:29.017 INFO:tasks.workunit.client.0.smithi100.stdout: "tcmalloc_page_size": 8192, 2024-04-07T17:51:29.017 INFO:tasks.workunit.client.0.smithi100.stdout: "thread_cache_freelists": 11263072, 2024-04-07T17:51:29.017 INFO:tasks.workunit.client.0.smithi100.stdout: "thread_heaps_in_use": 21, 2024-04-07T17:51:29.017 INFO:tasks.workunit.client.0.smithi100.stdout: "transfer_cache_freelist": 3081472, 2024-04-07T17:51:29.017 INFO:tasks.workunit.client.0.smithi100.stdout: "use_by_application": 150525608, 2024-04-07T17:51:29.017 INFO:tasks.workunit.client.0.smithi100.stdout: "virtual_address_space_used": 172843008 2024-04-07T17:51:29.017 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.017 INFO:tasks.workunit.client.0.smithi100.stdout: "mon.a52fc0d8acb515431879057649ff294cfe14c620": { 2024-04-07T17:51:29.017 INFO:tasks.workunit.client.0.smithi100.stdout: "actual_memory_used": 166469632, 2024-04-07T17:51:29.018 INFO:tasks.workunit.client.0.smithi100.stdout: "central_cache_freelist": 2320824, 2024-04-07T17:51:29.018 INFO:tasks.workunit.client.0.smithi100.stdout: "malloc_metadata": 3014656, 2024-04-07T17:51:29.018 INFO:tasks.workunit.client.0.smithi100.stdout: "page_heap_freelist": 40960, 2024-04-07T17:51:29.018 INFO:tasks.workunit.client.0.smithi100.stdout: "released_to_os": 1130496, 2024-04-07T17:51:29.018 INFO:tasks.workunit.client.0.smithi100.stdout: "spans_in_use": 6238, 2024-04-07T17:51:29.018 INFO:tasks.workunit.client.0.smithi100.stdout: "tcmalloc_page_size": 8192, 2024-04-07T17:51:29.018 INFO:tasks.workunit.client.0.smithi100.stdout: "thread_cache_freelists": 9541536, 2024-04-07T17:51:29.018 INFO:tasks.workunit.client.0.smithi100.stdout: "thread_heaps_in_use": 21, 2024-04-07T17:51:29.018 INFO:tasks.workunit.client.0.smithi100.stdout: "transfer_cache_freelist": 2689280, 2024-04-07T17:51:29.018 INFO:tasks.workunit.client.0.smithi100.stdout: "use_by_application": 148862376, 2024-04-07T17:51:29.018 INFO:tasks.workunit.client.0.smithi100.stdout: "virtual_address_space_used": 167600128 2024-04-07T17:51:29.018 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.018 INFO:tasks.workunit.client.0.smithi100.stdout: "mon.f47a4ff15a45950a63160265564a123fc5da5e5e": { 2024-04-07T17:51:29.018 INFO:tasks.workunit.client.0.smithi100.stdout: "actual_memory_used": 261971968, 2024-04-07T17:51:29.018 INFO:tasks.workunit.client.0.smithi100.stdout: "central_cache_freelist": 6466432, 2024-04-07T17:51:29.019 INFO:tasks.workunit.client.0.smithi100.stdout: "malloc_metadata": 3145728, 2024-04-07T17:51:29.019 INFO:tasks.workunit.client.0.smithi100.stdout: "page_heap_freelist": 0, 2024-04-07T17:51:29.019 INFO:tasks.workunit.client.0.smithi100.stdout: "released_to_os": 1384448, 2024-04-07T17:51:29.019 INFO:tasks.workunit.client.0.smithi100.stdout: "spans_in_use": 9422, 2024-04-07T17:51:29.019 INFO:tasks.workunit.client.0.smithi100.stdout: "tcmalloc_page_size": 8192, 2024-04-07T17:51:29.019 INFO:tasks.workunit.client.0.smithi100.stdout: "thread_cache_freelists": 15436352, 2024-04-07T17:51:29.019 INFO:tasks.workunit.client.0.smithi100.stdout: "thread_heaps_in_use": 23, 2024-04-07T17:51:29.019 INFO:tasks.workunit.client.0.smithi100.stdout: "transfer_cache_freelist": 5025024, 2024-04-07T17:51:29.019 INFO:tasks.workunit.client.0.smithi100.stdout: "use_by_application": 231898432, 2024-04-07T17:51:29.019 INFO:tasks.workunit.client.0.smithi100.stdout: "virtual_address_space_used": 263356416 2024-04-07T17:51:29.019 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.019 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.019 INFO:tasks.workunit.client.0.smithi100.stdout: "osd": { 2024-04-07T17:51:29.019 INFO:tasks.workunit.client.0.smithi100.stdout: "osd.0": { 2024-04-07T17:51:29.019 INFO:tasks.workunit.client.0.smithi100.stdout: "actual_memory_used": 473178112, 2024-04-07T17:51:29.019 INFO:tasks.workunit.client.0.smithi100.stdout: "central_cache_freelist": 27131872, 2024-04-07T17:51:29.020 INFO:tasks.workunit.client.0.smithi100.stdout: "malloc_metadata": 3407872, 2024-04-07T17:51:29.020 INFO:tasks.workunit.client.0.smithi100.stdout: "page_heap_freelist": 0, 2024-04-07T17:51:29.020 INFO:tasks.workunit.client.0.smithi100.stdout: "released_to_os": 38920192, 2024-04-07T17:51:29.020 INFO:tasks.workunit.client.0.smithi100.stdout: "spans_in_use": 12519, 2024-04-07T17:51:29.020 INFO:tasks.workunit.client.0.smithi100.stdout: "tcmalloc_page_size": 8192, 2024-04-07T17:51:29.020 INFO:tasks.workunit.client.0.smithi100.stdout: "thread_cache_freelists": 94886640, 2024-04-07T17:51:29.020 INFO:tasks.workunit.client.0.smithi100.stdout: "thread_heaps_in_use": 60, 2024-04-07T17:51:29.020 INFO:tasks.workunit.client.0.smithi100.stdout: "transfer_cache_freelist": 8507392, 2024-04-07T17:51:29.020 INFO:tasks.workunit.client.0.smithi100.stdout: "use_by_application": 339244336, 2024-04-07T17:51:29.020 INFO:tasks.workunit.client.0.smithi100.stdout: "virtual_address_space_used": 512098304 2024-04-07T17:51:29.020 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.020 INFO:tasks.workunit.client.0.smithi100.stdout: "osd.1": { 2024-04-07T17:51:29.020 INFO:tasks.workunit.client.0.smithi100.stdout: "actual_memory_used": 664215552, 2024-04-07T17:51:29.020 INFO:tasks.workunit.client.0.smithi100.stdout: "central_cache_freelist": 27067408, 2024-04-07T17:51:29.020 INFO:tasks.workunit.client.0.smithi100.stdout: "malloc_metadata": 3407872, 2024-04-07T17:51:29.020 INFO:tasks.workunit.client.0.smithi100.stdout: "page_heap_freelist": 114688, 2024-04-07T17:51:29.021 INFO:tasks.workunit.client.0.smithi100.stdout: "released_to_os": 35110912, 2024-04-07T17:51:29.021 INFO:tasks.workunit.client.0.smithi100.stdout: "spans_in_use": 13060, 2024-04-07T17:51:29.021 INFO:tasks.workunit.client.0.smithi100.stdout: "tcmalloc_page_size": 8192, 2024-04-07T17:51:29.021 INFO:tasks.workunit.client.0.smithi100.stdout: "thread_cache_freelists": 85772928, 2024-04-07T17:51:29.021 INFO:tasks.workunit.client.0.smithi100.stdout: "thread_heaps_in_use": 53, 2024-04-07T17:51:29.021 INFO:tasks.workunit.client.0.smithi100.stdout: "transfer_cache_freelist": 8256768, 2024-04-07T17:51:29.021 INFO:tasks.workunit.client.0.smithi100.stdout: "use_by_application": 539595888, 2024-04-07T17:51:29.021 INFO:tasks.workunit.client.0.smithi100.stdout: "virtual_address_space_used": 699326464 2024-04-07T17:51:29.021 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.021 INFO:tasks.workunit.client.0.smithi100.stdout: "osd.2": { 2024-04-07T17:51:29.021 INFO:tasks.workunit.client.0.smithi100.stdout: "actual_memory_used": 517300224, 2024-04-07T17:51:29.021 INFO:tasks.workunit.client.0.smithi100.stdout: "central_cache_freelist": 26235880, 2024-04-07T17:51:29.021 INFO:tasks.workunit.client.0.smithi100.stdout: "malloc_metadata": 3276800, 2024-04-07T17:51:29.021 INFO:tasks.workunit.client.0.smithi100.stdout: "page_heap_freelist": 0, 2024-04-07T17:51:29.021 INFO:tasks.workunit.client.0.smithi100.stdout: "released_to_os": 49258496, 2024-04-07T17:51:29.022 INFO:tasks.workunit.client.0.smithi100.stdout: "spans_in_use": 10733, 2024-04-07T17:51:29.022 INFO:tasks.workunit.client.0.smithi100.stdout: "tcmalloc_page_size": 8192, 2024-04-07T17:51:29.022 INFO:tasks.workunit.client.0.smithi100.stdout: "thread_cache_freelists": 85661440, 2024-04-07T17:51:29.022 INFO:tasks.workunit.client.0.smithi100.stdout: "thread_heaps_in_use": 60, 2024-04-07T17:51:29.022 INFO:tasks.workunit.client.0.smithi100.stdout: "transfer_cache_freelist": 9146880, 2024-04-07T17:51:29.022 INFO:tasks.workunit.client.0.smithi100.stdout: "use_by_application": 392979224, 2024-04-07T17:51:29.022 INFO:tasks.workunit.client.0.smithi100.stdout: "virtual_address_space_used": 566558720 2024-04-07T17:51:29.022 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.022 INFO:tasks.workunit.client.0.smithi100.stdout: "osd.3": { 2024-04-07T17:51:29.022 INFO:tasks.workunit.client.0.smithi100.stdout: "actual_memory_used": 473931776, 2024-04-07T17:51:29.022 INFO:tasks.workunit.client.0.smithi100.stdout: "central_cache_freelist": 20203448, 2024-04-07T17:51:29.022 INFO:tasks.workunit.client.0.smithi100.stdout: "malloc_metadata": 3276800, 2024-04-07T17:51:29.022 INFO:tasks.workunit.client.0.smithi100.stdout: "page_heap_freelist": 49152, 2024-04-07T17:51:29.022 INFO:tasks.workunit.client.0.smithi100.stdout: "released_to_os": 33120256, 2024-04-07T17:51:29.022 INFO:tasks.workunit.client.0.smithi100.stdout: "spans_in_use": 10246, 2024-04-07T17:51:29.022 INFO:tasks.workunit.client.0.smithi100.stdout: "tcmalloc_page_size": 8192, 2024-04-07T17:51:29.023 INFO:tasks.workunit.client.0.smithi100.stdout: "thread_cache_freelists": 80192792, 2024-04-07T17:51:29.023 INFO:tasks.workunit.client.0.smithi100.stdout: "thread_heaps_in_use": 60, 2024-04-07T17:51:29.023 INFO:tasks.workunit.client.0.smithi100.stdout: "transfer_cache_freelist": 9816576, 2024-04-07T17:51:29.023 INFO:tasks.workunit.client.0.smithi100.stdout: "use_by_application": 360393008, 2024-04-07T17:51:29.023 INFO:tasks.workunit.client.0.smithi100.stdout: "virtual_address_space_used": 507052032 2024-04-07T17:51:29.023 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.023 INFO:tasks.workunit.client.0.smithi100.stdout: "osd.4": { 2024-04-07T17:51:29.023 INFO:tasks.workunit.client.0.smithi100.stdout: "actual_memory_used": 596779008, 2024-04-07T17:51:29.023 INFO:tasks.workunit.client.0.smithi100.stdout: "central_cache_freelist": 24820160, 2024-04-07T17:51:29.023 INFO:tasks.workunit.client.0.smithi100.stdout: "malloc_metadata": 3407872, 2024-04-07T17:51:29.023 INFO:tasks.workunit.client.0.smithi100.stdout: "page_heap_freelist": 0, 2024-04-07T17:51:29.023 INFO:tasks.workunit.client.0.smithi100.stdout: "released_to_os": 29892608, 2024-04-07T17:51:29.023 INFO:tasks.workunit.client.0.smithi100.stdout: "spans_in_use": 11406, 2024-04-07T17:51:29.023 INFO:tasks.workunit.client.0.smithi100.stdout: "tcmalloc_page_size": 8192, 2024-04-07T17:51:29.023 INFO:tasks.workunit.client.0.smithi100.stdout: "thread_cache_freelists": 99064256, 2024-04-07T17:51:29.024 INFO:tasks.workunit.client.0.smithi100.stdout: "thread_heaps_in_use": 53, 2024-04-07T17:51:29.024 INFO:tasks.workunit.client.0.smithi100.stdout: "transfer_cache_freelist": 12079616, 2024-04-07T17:51:29.024 INFO:tasks.workunit.client.0.smithi100.stdout: "use_by_application": 457407104, 2024-04-07T17:51:29.024 INFO:tasks.workunit.client.0.smithi100.stdout: "virtual_address_space_used": 626671616 2024-04-07T17:51:29.024 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.024 INFO:tasks.workunit.client.0.smithi100.stdout: "osd.5": { 2024-04-07T17:51:29.024 INFO:tasks.workunit.client.0.smithi100.stdout: "actual_memory_used": 562798592, 2024-04-07T17:51:29.024 INFO:tasks.workunit.client.0.smithi100.stdout: "central_cache_freelist": 27101168, 2024-04-07T17:51:29.024 INFO:tasks.workunit.client.0.smithi100.stdout: "malloc_metadata": 5636096, 2024-04-07T17:51:29.024 INFO:tasks.workunit.client.0.smithi100.stdout: "page_heap_freelist": 0, 2024-04-07T17:51:29.024 INFO:tasks.workunit.client.0.smithi100.stdout: "released_to_os": 13197312, 2024-04-07T17:51:29.024 INFO:tasks.workunit.client.0.smithi100.stdout: "spans_in_use": 13822, 2024-04-07T17:51:29.024 INFO:tasks.workunit.client.0.smithi100.stdout: "tcmalloc_page_size": 8192, 2024-04-07T17:51:29.024 INFO:tasks.workunit.client.0.smithi100.stdout: "thread_cache_freelists": 86834768, 2024-04-07T17:51:29.024 INFO:tasks.workunit.client.0.smithi100.stdout: "thread_heaps_in_use": 60, 2024-04-07T17:51:29.025 INFO:tasks.workunit.client.0.smithi100.stdout: "transfer_cache_freelist": 10691584, 2024-04-07T17:51:29.025 INFO:tasks.workunit.client.0.smithi100.stdout: "use_by_application": 432534976, 2024-04-07T17:51:29.025 INFO:tasks.workunit.client.0.smithi100.stdout: "virtual_address_space_used": 575995904 2024-04-07T17:51:29.025 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.025 INFO:tasks.workunit.client.0.smithi100.stdout: "osd.6": { 2024-04-07T17:51:29.025 INFO:tasks.workunit.client.0.smithi100.stdout: "actual_memory_used": 362029056, 2024-04-07T17:51:29.025 INFO:tasks.workunit.client.0.smithi100.stdout: "central_cache_freelist": 16187696, 2024-04-07T17:51:29.025 INFO:tasks.workunit.client.0.smithi100.stdout: "malloc_metadata": 3276800, 2024-04-07T17:51:29.025 INFO:tasks.workunit.client.0.smithi100.stdout: "page_heap_freelist": 0, 2024-04-07T17:51:29.025 INFO:tasks.workunit.client.0.smithi100.stdout: "released_to_os": 64888832, 2024-04-07T17:51:29.025 INFO:tasks.workunit.client.0.smithi100.stdout: "spans_in_use": 9372, 2024-04-07T17:51:29.025 INFO:tasks.workunit.client.0.smithi100.stdout: "tcmalloc_page_size": 8192, 2024-04-07T17:51:29.025 INFO:tasks.workunit.client.0.smithi100.stdout: "thread_cache_freelists": 50423424, 2024-04-07T17:51:29.025 INFO:tasks.workunit.client.0.smithi100.stdout: "thread_heaps_in_use": 60, 2024-04-07T17:51:29.025 INFO:tasks.workunit.client.0.smithi100.stdout: "transfer_cache_freelist": 9710592, 2024-04-07T17:51:29.026 INFO:tasks.workunit.client.0.smithi100.stdout: "use_by_application": 282430544, 2024-04-07T17:51:29.026 INFO:tasks.workunit.client.0.smithi100.stdout: "virtual_address_space_used": 426917888 2024-04-07T17:51:29.026 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.026 INFO:tasks.workunit.client.0.smithi100.stdout: "osd.7": { 2024-04-07T17:51:29.026 INFO:tasks.workunit.client.0.smithi100.stdout: "actual_memory_used": 434208768, 2024-04-07T17:51:29.026 INFO:tasks.workunit.client.0.smithi100.stdout: "central_cache_freelist": 20836576, 2024-04-07T17:51:29.026 INFO:tasks.workunit.client.0.smithi100.stdout: "malloc_metadata": 3276800, 2024-04-07T17:51:29.026 INFO:tasks.workunit.client.0.smithi100.stdout: "page_heap_freelist": 0, 2024-04-07T17:51:29.026 INFO:tasks.workunit.client.0.smithi100.stdout: "released_to_os": 32399360, 2024-04-07T17:51:29.026 INFO:tasks.workunit.client.0.smithi100.stdout: "spans_in_use": 10456, 2024-04-07T17:51:29.026 INFO:tasks.workunit.client.0.smithi100.stdout: "tcmalloc_page_size": 8192, 2024-04-07T17:51:29.026 INFO:tasks.workunit.client.0.smithi100.stdout: "thread_cache_freelists": 72495984, 2024-04-07T17:51:29.026 INFO:tasks.workunit.client.0.smithi100.stdout: "thread_heaps_in_use": 59, 2024-04-07T17:51:29.026 INFO:tasks.workunit.client.0.smithi100.stdout: "transfer_cache_freelist": 10342656, 2024-04-07T17:51:29.027 INFO:tasks.workunit.client.0.smithi100.stdout: "use_by_application": 327256752, 2024-04-07T17:51:29.027 INFO:tasks.workunit.client.0.smithi100.stdout: "virtual_address_space_used": 466608128 2024-04-07T17:51:29.027 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.027 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.027 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.027 INFO:tasks.workunit.client.0.smithi100.stdout: "hosts": { 2024-04-07T17:51:29.027 INFO:tasks.workunit.client.0.smithi100.stdout: "num": 2, 2024-04-07T17:51:29.027 INFO:tasks.workunit.client.0.smithi100.stdout: "num_with_mds": 2, 2024-04-07T17:51:29.027 INFO:tasks.workunit.client.0.smithi100.stdout: "num_with_mgr": 2, 2024-04-07T17:51:29.027 INFO:tasks.workunit.client.0.smithi100.stdout: "num_with_mon": 2, 2024-04-07T17:51:29.027 INFO:tasks.workunit.client.0.smithi100.stdout: "num_with_osd": 2 2024-04-07T17:51:29.027 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.027 INFO:tasks.workunit.client.0.smithi100.stdout: "io_rate": { 2024-04-07T17:51:29.027 INFO:tasks.workunit.client.0.smithi100.stdout: "pg_stats_delta": { 2024-04-07T17:51:29.027 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": 0, 2024-04-07T17:51:29.027 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:29.028 INFO:tasks.workunit.client.0.smithi100.stdout: "num_store_stats": 0, 2024-04-07T17:51:29.028 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:29.028 INFO:tasks.workunit.client.0.smithi100.stdout: "stamp_delta": "10.867706", 2024-04-07T17:51:29.028 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:29.028 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:29.028 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:29.028 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:29.028 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:29.028 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:29.028 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:29.028 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:29.028 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:29.028 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:29.028 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:29.028 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:29.028 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:29.029 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:29.029 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:29.029 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:29.029 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:29.029 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:29.029 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:29.029 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:29.029 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:29.029 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:29.029 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:29.029 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:29.029 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:29.029 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:29.029 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:29.029 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:29.030 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:29.030 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:29.030 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:29.030 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:29.030 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:29.030 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:29.030 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 155, 2024-04-07T17:51:29.030 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 127, 2024-04-07T17:51:29.030 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:29.030 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:29.030 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:29.030 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 32, 2024-04-07T17:51:29.030 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:29.030 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.030 INFO:tasks.workunit.client.0.smithi100.stdout: "store_stats": { 2024-04-07T17:51:29.031 INFO:tasks.workunit.client.0.smithi100.stdout: "allocated": 0, 2024-04-07T17:51:29.031 INFO:tasks.workunit.client.0.smithi100.stdout: "available": 0, 2024-04-07T17:51:29.031 INFO:tasks.workunit.client.0.smithi100.stdout: "data_compressed": 0, 2024-04-07T17:51:29.031 INFO:tasks.workunit.client.0.smithi100.stdout: "data_compressed_allocated": 0, 2024-04-07T17:51:29.031 INFO:tasks.workunit.client.0.smithi100.stdout: "data_compressed_original": 0, 2024-04-07T17:51:29.031 INFO:tasks.workunit.client.0.smithi100.stdout: "data_stored": 0, 2024-04-07T17:51:29.031 INFO:tasks.workunit.client.0.smithi100.stdout: "internal_metadata": 0, 2024-04-07T17:51:29.031 INFO:tasks.workunit.client.0.smithi100.stdout: "internally_reserved": 0, 2024-04-07T17:51:29.031 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_allocated": 0, 2024-04-07T17:51:29.031 INFO:tasks.workunit.client.0.smithi100.stdout: "total": 0 2024-04-07T17:51:29.031 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.031 INFO:tasks.workunit.client.0.smithi100.stdout: "up": 0 2024-04-07T17:51:29.031 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.031 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.031 INFO:tasks.workunit.client.0.smithi100.stdout: "leaderboard": false, 2024-04-07T17:51:29.031 INFO:tasks.workunit.client.0.smithi100.stdout: "leaderboard_description": null, 2024-04-07T17:51:29.032 INFO:tasks.workunit.client.0.smithi100.stdout: "license": "sharing-1-0", 2024-04-07T17:51:29.032 INFO:tasks.workunit.client.0.smithi100.stdout: "mempool": { 2024-04-07T17:51:29.032 INFO:tasks.workunit.client.0.smithi100.stdout: "mds": { 2024-04-07T17:51:29.032 INFO:tasks.workunit.client.0.smithi100.stdout: "mds.a52fc0d8acb515431879057649ff294cfe14c620": { 2024-04-07T17:51:29.032 INFO:tasks.workunit.client.0.smithi100.stdout: "bloom_filter": { 2024-04-07T17:51:29.032 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.032 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.032 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.032 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs": { 2024-04-07T17:51:29.032 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.032 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.032 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.032 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs_file_reader": { 2024-04-07T17:51:29.032 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.032 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.033 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.033 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs_file_writer": { 2024-04-07T17:51:29.033 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.033 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.033 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.033 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_alloc": { 2024-04-07T17:51:29.033 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.033 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.033 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.033 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_blob": { 2024-04-07T17:51:29.033 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.033 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.033 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.033 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_buffer": { 2024-04-07T17:51:29.033 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.033 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.034 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.034 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_data": { 2024-04-07T17:51:29.034 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.034 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.034 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.034 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_meta": { 2024-04-07T17:51:29.034 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.034 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.034 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.034 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_onode": { 2024-04-07T17:51:29.034 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.034 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.034 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.034 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_other": { 2024-04-07T17:51:29.034 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.035 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.035 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.035 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_extent": { 2024-04-07T17:51:29.035 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.035 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.035 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.035 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_fsck": { 2024-04-07T17:51:29.035 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.035 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.035 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.035 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_inline_bl": { 2024-04-07T17:51:29.035 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.035 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.035 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.035 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_shared_blob": { 2024-04-07T17:51:29.036 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.036 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.036 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.036 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_txc": { 2024-04-07T17:51:29.036 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.036 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.036 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.036 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_writing": { 2024-04-07T17:51:29.036 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.036 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.036 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.036 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_writing_deferred": { 2024-04-07T17:51:29.036 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.036 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.036 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.037 INFO:tasks.workunit.client.0.smithi100.stdout: "buffer_anon": { 2024-04-07T17:51:29.037 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 149048, 2024-04-07T17:51:29.037 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 34 2024-04-07T17:51:29.037 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.037 INFO:tasks.workunit.client.0.smithi100.stdout: "buffer_meta": { 2024-04-07T17:51:29.037 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.037 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.037 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.037 INFO:tasks.workunit.client.0.smithi100.stdout: "mds_co": { 2024-04-07T17:51:29.037 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 928, 2024-04-07T17:51:29.037 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 2 2024-04-07T17:51:29.037 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.037 INFO:tasks.workunit.client.0.smithi100.stdout: "osd": { 2024-04-07T17:51:29.037 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.037 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.037 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.038 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mapbl": { 2024-04-07T17:51:29.038 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.038 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.038 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.038 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_pglog": { 2024-04-07T17:51:29.038 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.038 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.038 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.038 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap": { 2024-04-07T17:51:29.038 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.038 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.038 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.038 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap_mapping": { 2024-04-07T17:51:29.038 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.038 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.039 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.039 INFO:tasks.workunit.client.0.smithi100.stdout: "pgmap": { 2024-04-07T17:51:29.039 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.039 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.039 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.039 INFO:tasks.workunit.client.0.smithi100.stdout: "unittest_1": { 2024-04-07T17:51:29.039 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.039 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.039 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.039 INFO:tasks.workunit.client.0.smithi100.stdout: "unittest_2": { 2024-04-07T17:51:29.039 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.039 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.039 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.039 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.039 INFO:tasks.workunit.client.0.smithi100.stdout: "mds.f47a4ff15a45950a63160265564a123fc5da5e5e": { 2024-04-07T17:51:29.041 INFO:tasks.workunit.client.0.smithi100.stdout: "bloom_filter": { 2024-04-07T17:51:29.041 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.041 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.041 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.041 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs": { 2024-04-07T17:51:29.041 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.041 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.041 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.041 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs_file_reader": { 2024-04-07T17:51:29.041 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.041 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.042 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.042 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs_file_writer": { 2024-04-07T17:51:29.042 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.042 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.042 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.042 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_alloc": { 2024-04-07T17:51:29.042 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.042 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.042 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.042 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_blob": { 2024-04-07T17:51:29.042 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.042 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.042 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.042 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_buffer": { 2024-04-07T17:51:29.042 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.043 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.043 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.043 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_data": { 2024-04-07T17:51:29.043 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.043 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.043 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.043 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_meta": { 2024-04-07T17:51:29.043 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.043 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.043 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.043 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_onode": { 2024-04-07T17:51:29.043 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.043 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.043 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.043 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_other": { 2024-04-07T17:51:29.044 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.044 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.044 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.044 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_extent": { 2024-04-07T17:51:29.044 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.044 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.044 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.044 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_fsck": { 2024-04-07T17:51:29.044 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.044 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.044 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.044 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_inline_bl": { 2024-04-07T17:51:29.044 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.044 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.044 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.044 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_shared_blob": { 2024-04-07T17:51:29.045 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.045 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.045 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.045 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_txc": { 2024-04-07T17:51:29.045 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.045 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.045 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.045 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_writing": { 2024-04-07T17:51:29.045 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.045 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.045 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.045 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_writing_deferred": { 2024-04-07T17:51:29.045 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.045 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.046 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.046 INFO:tasks.workunit.client.0.smithi100.stdout: "buffer_anon": { 2024-04-07T17:51:29.046 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 221552, 2024-04-07T17:51:29.046 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 64 2024-04-07T17:51:29.046 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.046 INFO:tasks.workunit.client.0.smithi100.stdout: "buffer_meta": { 2024-04-07T17:51:29.046 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.046 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.046 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.046 INFO:tasks.workunit.client.0.smithi100.stdout: "mds_co": { 2024-04-07T17:51:29.046 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 52112, 2024-04-07T17:51:29.046 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 181 2024-04-07T17:51:29.046 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.046 INFO:tasks.workunit.client.0.smithi100.stdout: "osd": { 2024-04-07T17:51:29.047 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.047 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.047 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.047 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mapbl": { 2024-04-07T17:51:29.047 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.047 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.047 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.047 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_pglog": { 2024-04-07T17:51:29.047 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.047 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.047 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.047 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap": { 2024-04-07T17:51:29.047 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 12216, 2024-04-07T17:51:29.047 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 252 2024-04-07T17:51:29.048 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.048 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap_mapping": { 2024-04-07T17:51:29.048 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.048 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.048 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.048 INFO:tasks.workunit.client.0.smithi100.stdout: "pgmap": { 2024-04-07T17:51:29.048 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.048 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.048 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.048 INFO:tasks.workunit.client.0.smithi100.stdout: "unittest_1": { 2024-04-07T17:51:29.048 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.048 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.048 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.048 INFO:tasks.workunit.client.0.smithi100.stdout: "unittest_2": { 2024-04-07T17:51:29.048 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.048 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.048 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.049 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.049 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.049 INFO:tasks.workunit.client.0.smithi100.stdout: "mon": { 2024-04-07T17:51:29.049 INFO:tasks.workunit.client.0.smithi100.stdout: "mon.8dd0baa036f0a7e6d1f3bdce86136fe527836c3d": { 2024-04-07T17:51:29.049 INFO:tasks.workunit.client.0.smithi100.stdout: "bloom_filter": { 2024-04-07T17:51:29.049 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.049 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.049 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.049 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs": { 2024-04-07T17:51:29.049 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.049 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.049 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.049 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs_file_reader": { 2024-04-07T17:51:29.049 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.049 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.049 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.050 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs_file_writer": { 2024-04-07T17:51:29.050 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.050 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.050 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.050 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_alloc": { 2024-04-07T17:51:29.050 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.050 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.050 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.050 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_blob": { 2024-04-07T17:51:29.050 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.050 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.050 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.050 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_buffer": { 2024-04-07T17:51:29.050 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.050 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.050 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.051 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_data": { 2024-04-07T17:51:29.051 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.051 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.051 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.051 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_meta": { 2024-04-07T17:51:29.051 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.051 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.051 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.051 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_onode": { 2024-04-07T17:51:29.051 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.051 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.051 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.051 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_other": { 2024-04-07T17:51:29.051 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.051 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.051 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.051 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_extent": { 2024-04-07T17:51:29.052 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.052 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.052 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.052 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_fsck": { 2024-04-07T17:51:29.052 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.052 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.052 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.052 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_inline_bl": { 2024-04-07T17:51:29.052 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.052 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.052 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.052 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_shared_blob": { 2024-04-07T17:51:29.052 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.052 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.052 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.052 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_txc": { 2024-04-07T17:51:29.052 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.053 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.053 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.053 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_writing": { 2024-04-07T17:51:29.053 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.053 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.053 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.053 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_writing_deferred": { 2024-04-07T17:51:29.053 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.053 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.053 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.053 INFO:tasks.workunit.client.0.smithi100.stdout: "buffer_anon": { 2024-04-07T17:51:29.053 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 40717986, 2024-04-07T17:51:29.053 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 5035 2024-04-07T17:51:29.053 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.053 INFO:tasks.workunit.client.0.smithi100.stdout: "buffer_meta": { 2024-04-07T17:51:29.053 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.053 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.054 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.054 INFO:tasks.workunit.client.0.smithi100.stdout: "mds_co": { 2024-04-07T17:51:29.054 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.054 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.054 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.054 INFO:tasks.workunit.client.0.smithi100.stdout: "osd": { 2024-04-07T17:51:29.054 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.054 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.054 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.054 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mapbl": { 2024-04-07T17:51:29.054 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.054 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.054 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.054 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_pglog": { 2024-04-07T17:51:29.055 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.055 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.055 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.055 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap": { 2024-04-07T17:51:29.055 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 10792, 2024-04-07T17:51:29.055 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 251 2024-04-07T17:51:29.055 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.055 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap_mapping": { 2024-04-07T17:51:29.055 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 70440, 2024-04-07T17:51:29.055 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 5158 2024-04-07T17:51:29.055 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.055 INFO:tasks.workunit.client.0.smithi100.stdout: "pgmap": { 2024-04-07T17:51:29.055 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 5896, 2024-04-07T17:51:29.055 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 174 2024-04-07T17:51:29.055 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.055 INFO:tasks.workunit.client.0.smithi100.stdout: "unittest_1": { 2024-04-07T17:51:29.055 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.056 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.056 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.056 INFO:tasks.workunit.client.0.smithi100.stdout: "unittest_2": { 2024-04-07T17:51:29.056 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.056 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.056 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.056 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.056 INFO:tasks.workunit.client.0.smithi100.stdout: "mon.a52fc0d8acb515431879057649ff294cfe14c620": { 2024-04-07T17:51:29.056 INFO:tasks.workunit.client.0.smithi100.stdout: "bloom_filter": { 2024-04-07T17:51:29.056 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.056 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.056 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.056 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs": { 2024-04-07T17:51:29.056 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.056 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.056 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.056 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs_file_reader": { 2024-04-07T17:51:29.057 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.057 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.057 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.057 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs_file_writer": { 2024-04-07T17:51:29.057 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.057 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.057 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.057 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_alloc": { 2024-04-07T17:51:29.063 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.064 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.064 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.064 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_blob": { 2024-04-07T17:51:29.064 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.064 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.064 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.064 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_buffer": { 2024-04-07T17:51:29.064 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.064 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.064 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.064 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_data": { 2024-04-07T17:51:29.064 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.064 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.064 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.064 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_meta": { 2024-04-07T17:51:29.064 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.064 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.065 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.065 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_onode": { 2024-04-07T17:51:29.065 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.065 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.065 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.065 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_other": { 2024-04-07T17:51:29.065 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.065 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.065 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.065 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_extent": { 2024-04-07T17:51:29.065 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.065 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.065 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.065 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_fsck": { 2024-04-07T17:51:29.065 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.065 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.066 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.066 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_inline_bl": { 2024-04-07T17:51:29.066 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.066 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.066 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.066 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_shared_blob": { 2024-04-07T17:51:29.066 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.066 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.066 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.066 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_txc": { 2024-04-07T17:51:29.066 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.066 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.066 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.066 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_writing": { 2024-04-07T17:51:29.066 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.067 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.067 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.067 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_writing_deferred": { 2024-04-07T17:51:29.067 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.067 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.067 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.067 INFO:tasks.workunit.client.0.smithi100.stdout: "buffer_anon": { 2024-04-07T17:51:29.067 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 40333659, 2024-04-07T17:51:29.067 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 4986 2024-04-07T17:51:29.067 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.067 INFO:tasks.workunit.client.0.smithi100.stdout: "buffer_meta": { 2024-04-07T17:51:29.067 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 88, 2024-04-07T17:51:29.067 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 1 2024-04-07T17:51:29.067 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.067 INFO:tasks.workunit.client.0.smithi100.stdout: "mds_co": { 2024-04-07T17:51:29.067 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.068 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.068 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.068 INFO:tasks.workunit.client.0.smithi100.stdout: "osd": { 2024-04-07T17:51:29.068 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.068 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.068 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.068 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mapbl": { 2024-04-07T17:51:29.068 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.068 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.068 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.068 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_pglog": { 2024-04-07T17:51:29.068 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.068 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.068 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.068 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap": { 2024-04-07T17:51:29.068 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 10792, 2024-04-07T17:51:29.069 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 251 2024-04-07T17:51:29.069 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.069 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap_mapping": { 2024-04-07T17:51:29.069 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 70440, 2024-04-07T17:51:29.069 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 5158 2024-04-07T17:51:29.069 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.069 INFO:tasks.workunit.client.0.smithi100.stdout: "pgmap": { 2024-04-07T17:51:29.069 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 5896, 2024-04-07T17:51:29.069 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 174 2024-04-07T17:51:29.069 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.069 INFO:tasks.workunit.client.0.smithi100.stdout: "unittest_1": { 2024-04-07T17:51:29.069 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.069 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.069 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.069 INFO:tasks.workunit.client.0.smithi100.stdout: "unittest_2": { 2024-04-07T17:51:29.069 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.070 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.070 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.070 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.070 INFO:tasks.workunit.client.0.smithi100.stdout: "mon.f47a4ff15a45950a63160265564a123fc5da5e5e": { 2024-04-07T17:51:29.070 INFO:tasks.workunit.client.0.smithi100.stdout: "bloom_filter": { 2024-04-07T17:51:29.070 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.070 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.070 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.070 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs": { 2024-04-07T17:51:29.070 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.070 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.070 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.070 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs_file_reader": { 2024-04-07T17:51:29.070 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.071 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.071 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.071 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs_file_writer": { 2024-04-07T17:51:29.071 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.071 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.071 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.071 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_alloc": { 2024-04-07T17:51:29.071 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.071 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.071 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.071 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_blob": { 2024-04-07T17:51:29.071 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.071 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.071 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.071 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_buffer": { 2024-04-07T17:51:29.072 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.072 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.072 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.072 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_data": { 2024-04-07T17:51:29.072 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.072 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.072 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.072 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_meta": { 2024-04-07T17:51:29.072 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.072 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.072 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.072 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_onode": { 2024-04-07T17:51:29.072 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.072 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.072 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.073 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_other": { 2024-04-07T17:51:29.073 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.073 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.073 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.073 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_extent": { 2024-04-07T17:51:29.073 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.073 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.073 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.073 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_fsck": { 2024-04-07T17:51:29.073 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.073 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.073 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.073 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_inline_bl": { 2024-04-07T17:51:29.073 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.073 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.074 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.074 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_shared_blob": { 2024-04-07T17:51:29.074 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.074 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.074 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.074 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_txc": { 2024-04-07T17:51:29.074 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.074 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.074 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.074 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_writing": { 2024-04-07T17:51:29.074 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.074 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.074 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.074 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_writing_deferred": { 2024-04-07T17:51:29.074 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.075 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.075 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.075 INFO:tasks.workunit.client.0.smithi100.stdout: "buffer_anon": { 2024-04-07T17:51:29.075 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 41223042, 2024-04-07T17:51:29.075 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 5096 2024-04-07T17:51:29.075 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.075 INFO:tasks.workunit.client.0.smithi100.stdout: "buffer_meta": { 2024-04-07T17:51:29.075 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 88, 2024-04-07T17:51:29.075 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 1 2024-04-07T17:51:29.075 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.075 INFO:tasks.workunit.client.0.smithi100.stdout: "mds_co": { 2024-04-07T17:51:29.075 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.075 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.075 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.075 INFO:tasks.workunit.client.0.smithi100.stdout: "osd": { 2024-04-07T17:51:29.075 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.076 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.076 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.076 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mapbl": { 2024-04-07T17:51:29.076 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.076 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.076 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.076 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_pglog": { 2024-04-07T17:51:29.076 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.076 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.076 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.076 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap": { 2024-04-07T17:51:29.076 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 10792, 2024-04-07T17:51:29.076 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 251 2024-04-07T17:51:29.076 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.076 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap_mapping": { 2024-04-07T17:51:29.077 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 70440, 2024-04-07T17:51:29.077 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 5158 2024-04-07T17:51:29.077 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.077 INFO:tasks.workunit.client.0.smithi100.stdout: "pgmap": { 2024-04-07T17:51:29.077 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 11792, 2024-04-07T17:51:29.077 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 348 2024-04-07T17:51:29.077 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.077 INFO:tasks.workunit.client.0.smithi100.stdout: "unittest_1": { 2024-04-07T17:51:29.077 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.077 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.077 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.077 INFO:tasks.workunit.client.0.smithi100.stdout: "unittest_2": { 2024-04-07T17:51:29.077 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.077 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.077 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.077 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.078 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.078 INFO:tasks.workunit.client.0.smithi100.stdout: "osd": { 2024-04-07T17:51:29.078 INFO:tasks.workunit.client.0.smithi100.stdout: "osd.0": { 2024-04-07T17:51:29.078 INFO:tasks.workunit.client.0.smithi100.stdout: "bloom_filter": { 2024-04-07T17:51:29.078 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.078 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.078 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.078 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs": { 2024-04-07T17:51:29.078 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 22768, 2024-04-07T17:51:29.078 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 424 2024-04-07T17:51:29.078 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.078 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs_file_reader": { 2024-04-07T17:51:29.078 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 31360, 2024-04-07T17:51:29.078 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 28 2024-04-07T17:51:29.078 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.079 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs_file_writer": { 2024-04-07T17:51:29.079 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 2496, 2024-04-07T17:51:29.079 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 13 2024-04-07T17:51:29.079 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.079 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_alloc": { 2024-04-07T17:51:29.079 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 59840, 2024-04-07T17:51:29.079 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 748 2024-04-07T17:51:29.079 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.079 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_blob": { 2024-04-07T17:51:29.079 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 382512, 2024-04-07T17:51:29.079 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 1839 2024-04-07T17:51:29.079 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.079 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_buffer": { 2024-04-07T17:51:29.079 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 256, 2024-04-07T17:51:29.079 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 32 2024-04-07T17:51:29.080 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.080 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_data": { 2024-04-07T17:51:29.080 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 774695, 2024-04-07T17:51:29.080 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 215 2024-04-07T17:51:29.080 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.080 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_meta": { 2024-04-07T17:51:29.080 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 2426291, 2024-04-07T17:51:29.080 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 537859 2024-04-07T17:51:29.080 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.080 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_onode": { 2024-04-07T17:51:29.080 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 6943736, 2024-04-07T17:51:29.080 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 10243 2024-04-07T17:51:29.080 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.080 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_other": { 2024-04-07T17:51:29.080 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 72960, 2024-04-07T17:51:29.080 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 7616 2024-04-07T17:51:29.081 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.081 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_extent": { 2024-04-07T17:51:29.081 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 88272, 2024-04-07T17:51:29.081 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 1839 2024-04-07T17:51:29.081 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.081 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_fsck": { 2024-04-07T17:51:29.081 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.081 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.081 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.081 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_inline_bl": { 2024-04-07T17:51:29.081 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 175624, 2024-04-07T17:51:29.081 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 1258 2024-04-07T17:51:29.081 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.081 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_shared_blob": { 2024-04-07T17:51:29.082 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.082 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.082 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.082 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_txc": { 2024-04-07T17:51:29.082 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.082 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.082 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.082 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_writing": { 2024-04-07T17:51:29.082 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 21009, 2024-04-07T17:51:29.082 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 33 2024-04-07T17:51:29.082 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.082 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_writing_deferred": { 2024-04-07T17:51:29.082 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.082 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.082 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.083 INFO:tasks.workunit.client.0.smithi100.stdout: "buffer_anon": { 2024-04-07T17:51:29.083 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 15044844, 2024-04-07T17:51:29.083 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 291 2024-04-07T17:51:29.083 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.083 INFO:tasks.workunit.client.0.smithi100.stdout: "buffer_meta": { 2024-04-07T17:51:29.083 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 19448, 2024-04-07T17:51:29.083 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 221 2024-04-07T17:51:29.083 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.083 INFO:tasks.workunit.client.0.smithi100.stdout: "mds_co": { 2024-04-07T17:51:29.083 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.083 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.083 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.083 INFO:tasks.workunit.client.0.smithi100.stdout: "osd": { 2024-04-07T17:51:29.083 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 513216, 2024-04-07T17:51:29.083 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 44 2024-04-07T17:51:29.083 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.083 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mapbl": { 2024-04-07T17:51:29.084 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.084 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.084 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.084 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_pglog": { 2024-04-07T17:51:29.084 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 7171840, 2024-04-07T17:51:29.084 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 18592 2024-04-07T17:51:29.084 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.084 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap": { 2024-04-07T17:51:29.084 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 635376, 2024-04-07T17:51:29.084 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 11076 2024-04-07T17:51:29.084 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.084 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap_mapping": { 2024-04-07T17:51:29.084 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.084 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.084 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.084 INFO:tasks.workunit.client.0.smithi100.stdout: "pgmap": { 2024-04-07T17:51:29.084 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.085 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.085 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.085 INFO:tasks.workunit.client.0.smithi100.stdout: "unittest_1": { 2024-04-07T17:51:29.085 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.085 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.085 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.085 INFO:tasks.workunit.client.0.smithi100.stdout: "unittest_2": { 2024-04-07T17:51:29.085 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.085 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.085 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.085 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.085 INFO:tasks.workunit.client.0.smithi100.stdout: "osd.1": { 2024-04-07T17:51:29.085 INFO:tasks.workunit.client.0.smithi100.stdout: "bloom_filter": { 2024-04-07T17:51:29.085 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.085 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.086 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.086 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs": { 2024-04-07T17:51:29.086 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 20720, 2024-04-07T17:51:29.086 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 385 2024-04-07T17:51:29.086 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.086 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs_file_reader": { 2024-04-07T17:51:29.086 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 35072, 2024-04-07T17:51:29.086 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 26 2024-04-07T17:51:29.086 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.086 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs_file_writer": { 2024-04-07T17:51:29.086 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 4800, 2024-04-07T17:51:29.086 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 25 2024-04-07T17:51:29.086 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.086 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_alloc": { 2024-04-07T17:51:29.087 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 58160, 2024-04-07T17:51:29.087 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 727 2024-04-07T17:51:29.087 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.087 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_blob": { 2024-04-07T17:51:29.087 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 380224, 2024-04-07T17:51:29.087 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 1828 2024-04-07T17:51:29.087 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.087 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_buffer": { 2024-04-07T17:51:29.087 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 256, 2024-04-07T17:51:29.087 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 32 2024-04-07T17:51:29.087 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.087 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_data": { 2024-04-07T17:51:29.087 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 1045046, 2024-04-07T17:51:29.087 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 285 2024-04-07T17:51:29.087 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.087 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_meta": { 2024-04-07T17:51:29.087 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 2355008, 2024-04-07T17:51:29.088 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 521245 2024-04-07T17:51:29.088 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.088 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_onode": { 2024-04-07T17:51:29.088 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 6739736, 2024-04-07T17:51:29.088 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 9943 2024-04-07T17:51:29.088 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.088 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_other": { 2024-04-07T17:51:29.088 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 70888, 2024-04-07T17:51:29.088 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 7440 2024-04-07T17:51:29.088 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.088 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_extent": { 2024-04-07T17:51:29.088 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 87744, 2024-04-07T17:51:29.088 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 1828 2024-04-07T17:51:29.088 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.088 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_fsck": { 2024-04-07T17:51:29.088 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.089 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.089 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.089 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_inline_bl": { 2024-04-07T17:51:29.089 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 175712, 2024-04-07T17:51:29.089 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 1259 2024-04-07T17:51:29.089 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.089 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_shared_blob": { 2024-04-07T17:51:29.089 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.089 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.089 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.089 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_txc": { 2024-04-07T17:51:29.089 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.089 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.089 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.089 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_writing": { 2024-04-07T17:51:29.089 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 34652, 2024-04-07T17:51:29.090 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 29 2024-04-07T17:51:29.090 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.090 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_writing_deferred": { 2024-04-07T17:51:29.090 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.090 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.090 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.090 INFO:tasks.workunit.client.0.smithi100.stdout: "buffer_anon": { 2024-04-07T17:51:29.090 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 26499475, 2024-04-07T17:51:29.090 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 286 2024-04-07T17:51:29.090 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.090 INFO:tasks.workunit.client.0.smithi100.stdout: "buffer_meta": { 2024-04-07T17:51:29.090 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 26928, 2024-04-07T17:51:29.090 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 306 2024-04-07T17:51:29.090 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.090 INFO:tasks.workunit.client.0.smithi100.stdout: "mds_co": { 2024-04-07T17:51:29.090 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.091 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.091 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.091 INFO:tasks.workunit.client.0.smithi100.stdout: "osd": { 2024-04-07T17:51:29.091 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 443232, 2024-04-07T17:51:29.091 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 38 2024-04-07T17:51:29.091 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.091 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mapbl": { 2024-04-07T17:51:29.091 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.091 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.091 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.091 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_pglog": { 2024-04-07T17:51:29.091 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 7182344, 2024-04-07T17:51:29.091 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 20929 2024-04-07T17:51:29.091 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.091 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap": { 2024-04-07T17:51:29.091 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 635376, 2024-04-07T17:51:29.092 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 11076 2024-04-07T17:51:29.092 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.092 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap_mapping": { 2024-04-07T17:51:29.092 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.092 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.092 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.092 INFO:tasks.workunit.client.0.smithi100.stdout: "pgmap": { 2024-04-07T17:51:29.092 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.092 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.092 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.092 INFO:tasks.workunit.client.0.smithi100.stdout: "unittest_1": { 2024-04-07T17:51:29.092 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.092 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.092 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.092 INFO:tasks.workunit.client.0.smithi100.stdout: "unittest_2": { 2024-04-07T17:51:29.092 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.093 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.093 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.093 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.093 INFO:tasks.workunit.client.0.smithi100.stdout: "osd.2": { 2024-04-07T17:51:29.093 INFO:tasks.workunit.client.0.smithi100.stdout: "bloom_filter": { 2024-04-07T17:51:29.093 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.093 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.093 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.093 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs": { 2024-04-07T17:51:29.093 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 19496, 2024-04-07T17:51:29.093 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 362 2024-04-07T17:51:29.093 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.093 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs_file_reader": { 2024-04-07T17:51:29.093 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 31360, 2024-04-07T17:51:29.093 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 28 2024-04-07T17:51:29.093 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.093 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs_file_writer": { 2024-04-07T17:51:29.094 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 3648, 2024-04-07T17:51:29.094 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 19 2024-04-07T17:51:29.094 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.094 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_alloc": { 2024-04-07T17:51:29.094 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 58080, 2024-04-07T17:51:29.094 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 726 2024-04-07T17:51:29.094 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.094 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_blob": { 2024-04-07T17:51:29.094 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 379600, 2024-04-07T17:51:29.094 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 1825 2024-04-07T17:51:29.094 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.094 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_buffer": { 2024-04-07T17:51:29.094 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 256, 2024-04-07T17:51:29.094 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 32 2024-04-07T17:51:29.094 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.095 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_data": { 2024-04-07T17:51:29.095 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 434802, 2024-04-07T17:51:29.095 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 100 2024-04-07T17:51:29.095 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.095 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_meta": { 2024-04-07T17:51:29.095 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 1711873, 2024-04-07T17:51:29.095 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 379494 2024-04-07T17:51:29.095 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.095 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_onode": { 2024-04-07T17:51:29.095 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 5016616, 2024-04-07T17:51:29.095 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 7409 2024-04-07T17:51:29.095 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.095 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_other": { 2024-04-07T17:51:29.095 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 69948, 2024-04-07T17:51:29.095 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 7427 2024-04-07T17:51:29.095 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.096 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_extent": { 2024-04-07T17:51:29.096 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 87600, 2024-04-07T17:51:29.096 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 1825 2024-04-07T17:51:29.096 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.096 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_fsck": { 2024-04-07T17:51:29.096 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.096 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.096 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.096 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_inline_bl": { 2024-04-07T17:51:29.096 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 175624, 2024-04-07T17:51:29.096 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 1258 2024-04-07T17:51:29.096 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.096 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_shared_blob": { 2024-04-07T17:51:29.096 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.096 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.096 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.097 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_txc": { 2024-04-07T17:51:29.097 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 736, 2024-04-07T17:51:29.097 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 1 2024-04-07T17:51:29.097 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.097 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_writing": { 2024-04-07T17:51:29.097 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 69418, 2024-04-07T17:51:29.097 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 53 2024-04-07T17:51:29.097 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.097 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_writing_deferred": { 2024-04-07T17:51:29.097 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 12112, 2024-04-07T17:51:29.097 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 3 2024-04-07T17:51:29.097 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.097 INFO:tasks.workunit.client.0.smithi100.stdout: "buffer_anon": { 2024-04-07T17:51:29.097 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 20645063, 2024-04-07T17:51:29.097 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 260 2024-04-07T17:51:29.098 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.098 INFO:tasks.workunit.client.0.smithi100.stdout: "buffer_meta": { 2024-04-07T17:51:29.098 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 12760, 2024-04-07T17:51:29.098 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 145 2024-04-07T17:51:29.098 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.098 INFO:tasks.workunit.client.0.smithi100.stdout: "mds_co": { 2024-04-07T17:51:29.098 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.098 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.098 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.098 INFO:tasks.workunit.client.0.smithi100.stdout: "osd": { 2024-04-07T17:51:29.098 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 349920, 2024-04-07T17:51:29.098 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 30 2024-04-07T17:51:29.098 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.098 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mapbl": { 2024-04-07T17:51:29.099 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.099 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.099 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.099 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_pglog": { 2024-04-07T17:51:29.099 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 3170440, 2024-04-07T17:51:29.099 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 6097 2024-04-07T17:51:29.099 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.099 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap": { 2024-04-07T17:51:29.099 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 635376, 2024-04-07T17:51:29.099 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 11076 2024-04-07T17:51:29.099 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.099 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap_mapping": { 2024-04-07T17:51:29.099 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.099 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.099 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.099 INFO:tasks.workunit.client.0.smithi100.stdout: "pgmap": { 2024-04-07T17:51:29.099 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.100 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.100 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.100 INFO:tasks.workunit.client.0.smithi100.stdout: "unittest_1": { 2024-04-07T17:51:29.100 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.100 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.100 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.100 INFO:tasks.workunit.client.0.smithi100.stdout: "unittest_2": { 2024-04-07T17:51:29.100 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.100 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.100 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.100 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.100 INFO:tasks.workunit.client.0.smithi100.stdout: "osd.3": { 2024-04-07T17:51:29.100 INFO:tasks.workunit.client.0.smithi100.stdout: "bloom_filter": { 2024-04-07T17:51:29.100 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.100 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.100 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.100 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs": { 2024-04-07T17:51:29.101 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 17048, 2024-04-07T17:51:29.101 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 316 2024-04-07T17:51:29.101 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.101 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs_file_reader": { 2024-04-07T17:51:29.101 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 30976, 2024-04-07T17:51:29.101 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 25 2024-04-07T17:51:29.101 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.101 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs_file_writer": { 2024-04-07T17:51:29.101 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 3072, 2024-04-07T17:51:29.101 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 16 2024-04-07T17:51:29.101 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.101 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_alloc": { 2024-04-07T17:51:29.101 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 56960, 2024-04-07T17:51:29.101 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 712 2024-04-07T17:51:29.101 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.101 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_blob": { 2024-04-07T17:51:29.101 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 380640, 2024-04-07T17:51:29.102 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 1830 2024-04-07T17:51:29.102 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.102 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_buffer": { 2024-04-07T17:51:29.102 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 256, 2024-04-07T17:51:29.102 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 32 2024-04-07T17:51:29.102 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.102 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_data": { 2024-04-07T17:51:29.102 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 381771, 2024-04-07T17:51:29.102 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 104 2024-04-07T17:51:29.102 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.102 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_meta": { 2024-04-07T17:51:29.102 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 1733357, 2024-04-07T17:51:29.102 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 381544 2024-04-07T17:51:29.102 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.102 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_onode": { 2024-04-07T17:51:29.103 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 5029536, 2024-04-07T17:51:29.103 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 7428 2024-04-07T17:51:29.103 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.103 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_other": { 2024-04-07T17:51:29.103 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 70748, 2024-04-07T17:51:29.103 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 7446 2024-04-07T17:51:29.103 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.103 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_extent": { 2024-04-07T17:51:29.103 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 87840, 2024-04-07T17:51:29.103 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 1830 2024-04-07T17:51:29.103 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.103 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_fsck": { 2024-04-07T17:51:29.103 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.103 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.103 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.103 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_inline_bl": { 2024-04-07T17:51:29.104 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 175624, 2024-04-07T17:51:29.104 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 1258 2024-04-07T17:51:29.104 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.104 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_shared_blob": { 2024-04-07T17:51:29.104 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.104 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.104 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.104 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_txc": { 2024-04-07T17:51:29.104 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 736, 2024-04-07T17:51:29.104 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 1 2024-04-07T17:51:29.104 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.104 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_writing": { 2024-04-07T17:51:29.104 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 83741, 2024-04-07T17:51:29.104 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 55 2024-04-07T17:51:29.104 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.104 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_writing_deferred": { 2024-04-07T17:51:29.105 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 12112, 2024-04-07T17:51:29.105 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 3 2024-04-07T17:51:29.105 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.105 INFO:tasks.workunit.client.0.smithi100.stdout: "buffer_anon": { 2024-04-07T17:51:29.105 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 18443066, 2024-04-07T17:51:29.105 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 280 2024-04-07T17:51:29.105 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.105 INFO:tasks.workunit.client.0.smithi100.stdout: "buffer_meta": { 2024-04-07T17:51:29.105 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 11880, 2024-04-07T17:51:29.105 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 135 2024-04-07T17:51:29.105 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.105 INFO:tasks.workunit.client.0.smithi100.stdout: "mds_co": { 2024-04-07T17:51:29.105 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.105 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.105 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.105 INFO:tasks.workunit.client.0.smithi100.stdout: "osd": { 2024-04-07T17:51:29.106 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 419904, 2024-04-07T17:51:29.106 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 36 2024-04-07T17:51:29.106 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.106 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mapbl": { 2024-04-07T17:51:29.106 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.106 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.106 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.106 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_pglog": { 2024-04-07T17:51:29.106 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 3485560, 2024-04-07T17:51:29.106 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 9103 2024-04-07T17:51:29.106 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.106 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap": { 2024-04-07T17:51:29.106 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 635376, 2024-04-07T17:51:29.106 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 11076 2024-04-07T17:51:29.106 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.107 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap_mapping": { 2024-04-07T17:51:29.107 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.107 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.107 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.107 INFO:tasks.workunit.client.0.smithi100.stdout: "pgmap": { 2024-04-07T17:51:29.107 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.107 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.107 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.107 INFO:tasks.workunit.client.0.smithi100.stdout: "unittest_1": { 2024-04-07T17:51:29.107 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.107 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.107 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.107 INFO:tasks.workunit.client.0.smithi100.stdout: "unittest_2": { 2024-04-07T17:51:29.107 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.107 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.107 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.107 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.108 INFO:tasks.workunit.client.0.smithi100.stdout: "osd.4": { 2024-04-07T17:51:29.108 INFO:tasks.workunit.client.0.smithi100.stdout: "bloom_filter": { 2024-04-07T17:51:29.108 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.108 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.108 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.108 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs": { 2024-04-07T17:51:29.108 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 20816, 2024-04-07T17:51:29.108 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 393 2024-04-07T17:51:29.108 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.108 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs_file_reader": { 2024-04-07T17:51:29.108 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 31488, 2024-04-07T17:51:29.108 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 29 2024-04-07T17:51:29.108 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.108 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs_file_writer": { 2024-04-07T17:51:29.108 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 4032, 2024-04-07T17:51:29.108 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 21 2024-04-07T17:51:29.108 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.109 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_alloc": { 2024-04-07T17:51:29.109 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 58240, 2024-04-07T17:51:29.109 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 728 2024-04-07T17:51:29.109 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.109 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_blob": { 2024-04-07T17:51:29.109 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 393952, 2024-04-07T17:51:29.109 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 1894 2024-04-07T17:51:29.109 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.109 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_buffer": { 2024-04-07T17:51:29.109 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 256, 2024-04-07T17:51:29.109 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 32 2024-04-07T17:51:29.109 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.109 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_data": { 2024-04-07T17:51:29.109 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 1050407, 2024-04-07T17:51:29.109 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 273 2024-04-07T17:51:29.109 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.109 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_meta": { 2024-04-07T17:51:29.110 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 2027647, 2024-04-07T17:51:29.110 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 446306 2024-04-07T17:51:29.110 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.110 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_onode": { 2024-04-07T17:51:29.110 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 5812216, 2024-04-07T17:51:29.110 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 8579 2024-04-07T17:51:29.110 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.110 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_other": { 2024-04-07T17:51:29.110 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 80384, 2024-04-07T17:51:29.110 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 8599 2024-04-07T17:51:29.110 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.110 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_extent": { 2024-04-07T17:51:29.110 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 90912, 2024-04-07T17:51:29.110 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 1894 2024-04-07T17:51:29.110 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.110 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_fsck": { 2024-04-07T17:51:29.111 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.111 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.111 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.111 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_inline_bl": { 2024-04-07T17:51:29.111 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 175456, 2024-04-07T17:51:29.111 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 1258 2024-04-07T17:51:29.111 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.111 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_shared_blob": { 2024-04-07T17:51:29.111 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.111 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.111 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.111 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_txc": { 2024-04-07T17:51:29.111 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 736, 2024-04-07T17:51:29.111 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 1 2024-04-07T17:51:29.111 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.111 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_writing": { 2024-04-07T17:51:29.111 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 66705, 2024-04-07T17:51:29.112 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 51 2024-04-07T17:51:29.112 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.112 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_writing_deferred": { 2024-04-07T17:51:29.112 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 12112, 2024-04-07T17:51:29.112 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 3 2024-04-07T17:51:29.112 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.112 INFO:tasks.workunit.client.0.smithi100.stdout: "buffer_anon": { 2024-04-07T17:51:29.112 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 23146638, 2024-04-07T17:51:29.112 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 273 2024-04-07T17:51:29.112 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.112 INFO:tasks.workunit.client.0.smithi100.stdout: "buffer_meta": { 2024-04-07T17:51:29.112 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 26928, 2024-04-07T17:51:29.112 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 306 2024-04-07T17:51:29.112 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.112 INFO:tasks.workunit.client.0.smithi100.stdout: "mds_co": { 2024-04-07T17:51:29.112 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.112 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.113 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.113 INFO:tasks.workunit.client.0.smithi100.stdout: "osd": { 2024-04-07T17:51:29.113 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 466560, 2024-04-07T17:51:29.113 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 40 2024-04-07T17:51:29.113 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.113 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mapbl": { 2024-04-07T17:51:29.113 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.113 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.113 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.113 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_pglog": { 2024-04-07T17:51:29.113 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 5065320, 2024-04-07T17:51:29.113 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 12141 2024-04-07T17:51:29.113 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.113 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap": { 2024-04-07T17:51:29.113 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 635376, 2024-04-07T17:51:29.113 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 11076 2024-04-07T17:51:29.113 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.114 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap_mapping": { 2024-04-07T17:51:29.114 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.114 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.114 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.114 INFO:tasks.workunit.client.0.smithi100.stdout: "pgmap": { 2024-04-07T17:51:29.114 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.114 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.114 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.114 INFO:tasks.workunit.client.0.smithi100.stdout: "unittest_1": { 2024-04-07T17:51:29.114 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.114 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.114 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.114 INFO:tasks.workunit.client.0.smithi100.stdout: "unittest_2": { 2024-04-07T17:51:29.114 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.114 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.115 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.115 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.115 INFO:tasks.workunit.client.0.smithi100.stdout: "osd.5": { 2024-04-07T17:51:29.115 INFO:tasks.workunit.client.0.smithi100.stdout: "bloom_filter": { 2024-04-07T17:51:29.115 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.115 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.115 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.115 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs": { 2024-04-07T17:51:29.115 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 19216, 2024-04-07T17:51:29.115 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 365 2024-04-07T17:51:29.115 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.115 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs_file_reader": { 2024-04-07T17:51:29.115 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 31616, 2024-04-07T17:51:29.115 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 30 2024-04-07T17:51:29.116 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.116 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs_file_writer": { 2024-04-07T17:51:29.116 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 3072, 2024-04-07T17:51:29.116 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 16 2024-04-07T17:51:29.116 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.116 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_alloc": { 2024-04-07T17:51:29.116 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 58400, 2024-04-07T17:51:29.116 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 730 2024-04-07T17:51:29.116 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.116 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_blob": { 2024-04-07T17:51:29.116 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 393952, 2024-04-07T17:51:29.116 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 1894 2024-04-07T17:51:29.116 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.116 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_buffer": { 2024-04-07T17:51:29.116 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 256, 2024-04-07T17:51:29.117 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 32 2024-04-07T17:51:29.117 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.117 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_data": { 2024-04-07T17:51:29.117 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 274397, 2024-04-07T17:51:29.117 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 87 2024-04-07T17:51:29.117 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.117 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_meta": { 2024-04-07T17:51:29.117 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 2323821, 2024-04-07T17:51:29.117 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 518276 2024-04-07T17:51:29.117 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.117 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_onode": { 2024-04-07T17:51:29.117 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 6677176, 2024-04-07T17:51:29.117 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 9851 2024-04-07T17:51:29.117 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.117 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_other": { 2024-04-07T17:51:29.117 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 80636, 2024-04-07T17:51:29.117 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 8604 2024-04-07T17:51:29.118 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.118 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_extent": { 2024-04-07T17:51:29.118 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 90912, 2024-04-07T17:51:29.118 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 1894 2024-04-07T17:51:29.118 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.118 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_fsck": { 2024-04-07T17:51:29.118 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.118 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.118 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.118 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_inline_bl": { 2024-04-07T17:51:29.118 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 175712, 2024-04-07T17:51:29.118 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 1259 2024-04-07T17:51:29.118 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.118 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_shared_blob": { 2024-04-07T17:51:29.118 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.119 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.119 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.119 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_txc": { 2024-04-07T17:51:29.119 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 736, 2024-04-07T17:51:29.119 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 1 2024-04-07T17:51:29.119 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.119 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_writing": { 2024-04-07T17:51:29.119 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 87571, 2024-04-07T17:51:29.119 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 56 2024-04-07T17:51:29.119 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.119 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_writing_deferred": { 2024-04-07T17:51:29.119 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 12112, 2024-04-07T17:51:29.119 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 3 2024-04-07T17:51:29.119 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.119 INFO:tasks.workunit.client.0.smithi100.stdout: "buffer_anon": { 2024-04-07T17:51:29.119 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 18319894, 2024-04-07T17:51:29.119 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 12576 2024-04-07T17:51:29.120 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.120 INFO:tasks.workunit.client.0.smithi100.stdout: "buffer_meta": { 2024-04-07T17:51:29.120 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 10208, 2024-04-07T17:51:29.120 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 116 2024-04-07T17:51:29.120 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.120 INFO:tasks.workunit.client.0.smithi100.stdout: "mds_co": { 2024-04-07T17:51:29.120 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.120 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.120 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.120 INFO:tasks.workunit.client.0.smithi100.stdout: "osd": { 2024-04-07T17:51:29.120 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 489888, 2024-04-07T17:51:29.120 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 42 2024-04-07T17:51:29.120 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.120 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mapbl": { 2024-04-07T17:51:29.120 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.120 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.120 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.120 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_pglog": { 2024-04-07T17:51:29.121 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 6315920, 2024-04-07T17:51:29.121 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 12146 2024-04-07T17:51:29.121 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.121 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap": { 2024-04-07T17:51:29.121 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 635376, 2024-04-07T17:51:29.121 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 11076 2024-04-07T17:51:29.121 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.121 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap_mapping": { 2024-04-07T17:51:29.121 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.121 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.121 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.121 INFO:tasks.workunit.client.0.smithi100.stdout: "pgmap": { 2024-04-07T17:51:29.121 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.121 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.121 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.121 INFO:tasks.workunit.client.0.smithi100.stdout: "unittest_1": { 2024-04-07T17:51:29.121 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.122 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.122 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.122 INFO:tasks.workunit.client.0.smithi100.stdout: "unittest_2": { 2024-04-07T17:51:29.122 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.122 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.122 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.122 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.122 INFO:tasks.workunit.client.0.smithi100.stdout: "osd.6": { 2024-04-07T17:51:29.122 INFO:tasks.workunit.client.0.smithi100.stdout: "bloom_filter": { 2024-04-07T17:51:29.122 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.122 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.122 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.122 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs": { 2024-04-07T17:51:29.122 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 13656, 2024-04-07T17:51:29.122 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 230 2024-04-07T17:51:29.123 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.123 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs_file_reader": { 2024-04-07T17:51:29.123 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 30336, 2024-04-07T17:51:29.123 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 20 2024-04-07T17:51:29.123 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.123 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs_file_writer": { 2024-04-07T17:51:29.123 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 2496, 2024-04-07T17:51:29.123 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 13 2024-04-07T17:51:29.123 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.123 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_alloc": { 2024-04-07T17:51:29.123 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 56960, 2024-04-07T17:51:29.123 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 712 2024-04-07T17:51:29.123 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.123 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_blob": { 2024-04-07T17:51:29.123 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 395616, 2024-04-07T17:51:29.123 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 1902 2024-04-07T17:51:29.124 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.124 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_buffer": { 2024-04-07T17:51:29.124 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 256, 2024-04-07T17:51:29.124 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 32 2024-04-07T17:51:29.124 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.124 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_data": { 2024-04-07T17:51:29.124 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 463979, 2024-04-07T17:51:29.124 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 128 2024-04-07T17:51:29.124 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.124 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_meta": { 2024-04-07T17:51:29.124 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 1718221, 2024-04-07T17:51:29.124 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 377358 2024-04-07T17:51:29.124 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.124 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_onode": { 2024-04-07T17:51:29.124 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 4954056, 2024-04-07T17:51:29.124 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 7317 2024-04-07T17:51:29.124 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.124 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_other": { 2024-04-07T17:51:29.125 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 81560, 2024-04-07T17:51:29.125 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 8760 2024-04-07T17:51:29.125 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.125 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_extent": { 2024-04-07T17:51:29.125 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 91296, 2024-04-07T17:51:29.125 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 1902 2024-04-07T17:51:29.125 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.125 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_fsck": { 2024-04-07T17:51:29.125 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.125 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.125 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.125 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_inline_bl": { 2024-04-07T17:51:29.125 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 175368, 2024-04-07T17:51:29.125 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 1257 2024-04-07T17:51:29.125 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.125 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_shared_blob": { 2024-04-07T17:51:29.126 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.126 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.126 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.126 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_txc": { 2024-04-07T17:51:29.126 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.126 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.126 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.126 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_writing": { 2024-04-07T17:51:29.126 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 69389, 2024-04-07T17:51:29.126 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 49 2024-04-07T17:51:29.126 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.126 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_writing_deferred": { 2024-04-07T17:51:29.126 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.126 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.126 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.127 INFO:tasks.workunit.client.0.smithi100.stdout: "buffer_anon": { 2024-04-07T17:51:29.127 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 13267228, 2024-04-07T17:51:29.127 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 243 2024-04-07T17:51:29.127 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.127 INFO:tasks.workunit.client.0.smithi100.stdout: "buffer_meta": { 2024-04-07T17:51:29.127 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 12320, 2024-04-07T17:51:29.127 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 140 2024-04-07T17:51:29.127 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.127 INFO:tasks.workunit.client.0.smithi100.stdout: "mds_co": { 2024-04-07T17:51:29.127 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.127 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.127 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.127 INFO:tasks.workunit.client.0.smithi100.stdout: "osd": { 2024-04-07T17:51:29.127 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 454896, 2024-04-07T17:51:29.127 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 39 2024-04-07T17:51:29.128 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.128 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mapbl": { 2024-04-07T17:51:29.128 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.128 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.128 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.128 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_pglog": { 2024-04-07T17:51:29.128 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 3673280, 2024-04-07T17:51:29.128 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 9464 2024-04-07T17:51:29.128 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.128 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap": { 2024-04-07T17:51:29.128 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 635376, 2024-04-07T17:51:29.128 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 11076 2024-04-07T17:51:29.128 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.128 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap_mapping": { 2024-04-07T17:51:29.128 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.128 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.128 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.129 INFO:tasks.workunit.client.0.smithi100.stdout: "pgmap": { 2024-04-07T17:51:29.129 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.129 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.129 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.129 INFO:tasks.workunit.client.0.smithi100.stdout: "unittest_1": { 2024-04-07T17:51:29.129 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.129 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.129 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.129 INFO:tasks.workunit.client.0.smithi100.stdout: "unittest_2": { 2024-04-07T17:51:29.129 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.129 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.129 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.129 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.129 INFO:tasks.workunit.client.0.smithi100.stdout: "osd.7": { 2024-04-07T17:51:29.129 INFO:tasks.workunit.client.0.smithi100.stdout: "bloom_filter": { 2024-04-07T17:51:29.129 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.129 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.130 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.130 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs": { 2024-04-07T17:51:29.130 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 15992, 2024-04-07T17:51:29.130 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 307 2024-04-07T17:51:29.130 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.130 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs_file_reader": { 2024-04-07T17:51:29.130 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 39040, 2024-04-07T17:51:29.130 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 25 2024-04-07T17:51:29.130 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.130 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs_file_writer": { 2024-04-07T17:51:29.130 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 2688, 2024-04-07T17:51:29.130 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 14 2024-04-07T17:51:29.130 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.130 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_alloc": { 2024-04-07T17:51:29.131 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 57280, 2024-04-07T17:51:29.131 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 716 2024-04-07T17:51:29.131 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.131 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_blob": { 2024-04-07T17:51:29.131 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 382096, 2024-04-07T17:51:29.131 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 1837 2024-04-07T17:51:29.131 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.131 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_buffer": { 2024-04-07T17:51:29.131 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 256, 2024-04-07T17:51:29.131 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 32 2024-04-07T17:51:29.131 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.131 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_data": { 2024-04-07T17:51:29.131 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 427947, 2024-04-07T17:51:29.131 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 127 2024-04-07T17:51:29.131 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.131 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_meta": { 2024-04-07T17:51:29.131 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 1987758, 2024-04-07T17:51:29.132 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 443621 2024-04-07T17:51:29.132 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.132 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_onode": { 2024-04-07T17:51:29.132 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 5763936, 2024-04-07T17:51:29.132 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 8508 2024-04-07T17:51:29.132 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.132 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_cache_other": { 2024-04-07T17:51:29.132 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 73080, 2024-04-07T17:51:29.132 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 7610 2024-04-07T17:51:29.132 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.132 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_extent": { 2024-04-07T17:51:29.132 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 88176, 2024-04-07T17:51:29.132 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 1837 2024-04-07T17:51:29.132 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.133 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_fsck": { 2024-04-07T17:51:29.133 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.133 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.133 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.133 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_inline_bl": { 2024-04-07T17:51:29.133 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 175624, 2024-04-07T17:51:29.133 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 1258 2024-04-07T17:51:29.133 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.133 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_shared_blob": { 2024-04-07T17:51:29.133 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.133 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.133 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.133 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_txc": { 2024-04-07T17:51:29.133 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.133 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.133 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.134 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_writing": { 2024-04-07T17:51:29.134 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 21282, 2024-04-07T17:51:29.134 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 25 2024-04-07T17:51:29.134 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.134 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore_writing_deferred": { 2024-04-07T17:51:29.134 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.134 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.134 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.134 INFO:tasks.workunit.client.0.smithi100.stdout: "buffer_anon": { 2024-04-07T17:51:29.134 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 14934597, 2024-04-07T17:51:29.134 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 264 2024-04-07T17:51:29.134 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.134 INFO:tasks.workunit.client.0.smithi100.stdout: "buffer_meta": { 2024-04-07T17:51:29.134 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 10912, 2024-04-07T17:51:29.134 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 124 2024-04-07T17:51:29.135 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.135 INFO:tasks.workunit.client.0.smithi100.stdout: "mds_co": { 2024-04-07T17:51:29.135 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.135 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.135 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.135 INFO:tasks.workunit.client.0.smithi100.stdout: "osd": { 2024-04-07T17:51:29.135 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 536544, 2024-04-07T17:51:29.135 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 46 2024-04-07T17:51:29.135 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.135 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_mapbl": { 2024-04-07T17:51:29.135 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.135 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.135 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.135 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_pglog": { 2024-04-07T17:51:29.135 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 5013216, 2024-04-07T17:51:29.135 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 9676 2024-04-07T17:51:29.135 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.136 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap": { 2024-04-07T17:51:29.136 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 635376, 2024-04-07T17:51:29.136 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 11076 2024-04-07T17:51:29.136 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.136 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap_mapping": { 2024-04-07T17:51:29.136 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.136 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.136 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.136 INFO:tasks.workunit.client.0.smithi100.stdout: "pgmap": { 2024-04-07T17:51:29.136 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.136 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.136 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.136 INFO:tasks.workunit.client.0.smithi100.stdout: "unittest_1": { 2024-04-07T17:51:29.136 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.136 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.137 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.137 INFO:tasks.workunit.client.0.smithi100.stdout: "unittest_2": { 2024-04-07T17:51:29.137 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes": 0, 2024-04-07T17:51:29.137 INFO:tasks.workunit.client.0.smithi100.stdout: "items": 0 2024-04-07T17:51:29.137 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.137 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.137 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.137 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.137 INFO:tasks.workunit.client.0.smithi100.stdout: "metadata": { 2024-04-07T17:51:29.137 INFO:tasks.workunit.client.0.smithi100.stdout: "mds": { 2024-04-07T17:51:29.137 INFO:tasks.workunit.client.0.smithi100.stdout: "arch": { 2024-04-07T17:51:29.137 INFO:tasks.workunit.client.0.smithi100.stdout: "x86_64": 2 2024-04-07T17:51:29.137 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.137 INFO:tasks.workunit.client.0.smithi100.stdout: "ceph_version": { 2024-04-07T17:51:29.137 INFO:tasks.workunit.client.0.smithi100.stdout: "ceph version 19.0.0-2785-gadce92d0 (adce92d056533f22dc1ee37e3141d098b046ec1b) squid (dev)": 2 2024-04-07T17:51:29.138 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.138 INFO:tasks.workunit.client.0.smithi100.stdout: "cpu": { 2024-04-07T17:51:29.138 INFO:tasks.workunit.client.0.smithi100.stdout: "Intel(R) Xeon(R) CPU E5-1620 v4 @ 3.50GHz": 2 2024-04-07T17:51:29.138 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.138 INFO:tasks.workunit.client.0.smithi100.stdout: "distro": { 2024-04-07T17:51:29.138 INFO:tasks.workunit.client.0.smithi100.stdout: "centos": 2 2024-04-07T17:51:29.138 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.138 INFO:tasks.workunit.client.0.smithi100.stdout: "distro_description": { 2024-04-07T17:51:29.138 INFO:tasks.workunit.client.0.smithi100.stdout: "CentOS Stream 8": 2 2024-04-07T17:51:29.138 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.138 INFO:tasks.workunit.client.0.smithi100.stdout: "kernel_description": { 2024-04-07T17:51:29.138 INFO:tasks.workunit.client.0.smithi100.stdout: "#111-Ubuntu SMP Tue Mar 5 20:16:58 UTC 2024": 2 2024-04-07T17:51:29.138 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.138 INFO:tasks.workunit.client.0.smithi100.stdout: "kernel_version": { 2024-04-07T17:51:29.138 INFO:tasks.workunit.client.0.smithi100.stdout: "5.15.0-101-generic": 2 2024-04-07T17:51:29.139 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.139 INFO:tasks.workunit.client.0.smithi100.stdout: "os": { 2024-04-07T17:51:29.139 INFO:tasks.workunit.client.0.smithi100.stdout: "Linux": 2 2024-04-07T17:51:29.139 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.139 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.139 INFO:tasks.workunit.client.0.smithi100.stdout: "mon": { 2024-04-07T17:51:29.139 INFO:tasks.workunit.client.0.smithi100.stdout: "arch": { 2024-04-07T17:51:29.139 INFO:tasks.workunit.client.0.smithi100.stdout: "x86_64": 3 2024-04-07T17:51:29.139 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.139 INFO:tasks.workunit.client.0.smithi100.stdout: "ceph_version": { 2024-04-07T17:51:29.139 INFO:tasks.workunit.client.0.smithi100.stdout: "ceph version 19.0.0-2785-gadce92d0 (adce92d056533f22dc1ee37e3141d098b046ec1b) squid (dev)": 3 2024-04-07T17:51:29.139 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.139 INFO:tasks.workunit.client.0.smithi100.stdout: "cpu": { 2024-04-07T17:51:29.139 INFO:tasks.workunit.client.0.smithi100.stdout: "Intel(R) Xeon(R) CPU E5-1620 v4 @ 3.50GHz": 3 2024-04-07T17:51:29.139 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.139 INFO:tasks.workunit.client.0.smithi100.stdout: "distro": { 2024-04-07T17:51:29.140 INFO:tasks.workunit.client.0.smithi100.stdout: "centos": 3 2024-04-07T17:51:29.140 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.140 INFO:tasks.workunit.client.0.smithi100.stdout: "distro_description": { 2024-04-07T17:51:29.140 INFO:tasks.workunit.client.0.smithi100.stdout: "CentOS Stream 8": 3 2024-04-07T17:51:29.140 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.140 INFO:tasks.workunit.client.0.smithi100.stdout: "kernel_description": { 2024-04-07T17:51:29.140 INFO:tasks.workunit.client.0.smithi100.stdout: "#111-Ubuntu SMP Tue Mar 5 20:16:58 UTC 2024": 3 2024-04-07T17:51:29.140 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.140 INFO:tasks.workunit.client.0.smithi100.stdout: "kernel_version": { 2024-04-07T17:51:29.140 INFO:tasks.workunit.client.0.smithi100.stdout: "5.15.0-101-generic": 3 2024-04-07T17:51:29.140 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.140 INFO:tasks.workunit.client.0.smithi100.stdout: "os": { 2024-04-07T17:51:29.140 INFO:tasks.workunit.client.0.smithi100.stdout: "Linux": 3 2024-04-07T17:51:29.140 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.140 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.140 INFO:tasks.workunit.client.0.smithi100.stdout: "osd": { 2024-04-07T17:51:29.141 INFO:tasks.workunit.client.0.smithi100.stdout: "arch": { 2024-04-07T17:51:29.141 INFO:tasks.workunit.client.0.smithi100.stdout: "x86_64": 8 2024-04-07T17:51:29.141 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.141 INFO:tasks.workunit.client.0.smithi100.stdout: "ceph_version": { 2024-04-07T17:51:29.141 INFO:tasks.workunit.client.0.smithi100.stdout: "ceph version 19.0.0-2785-gadce92d0 (adce92d056533f22dc1ee37e3141d098b046ec1b) squid (dev)": 8 2024-04-07T17:51:29.141 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.141 INFO:tasks.workunit.client.0.smithi100.stdout: "cpu": { 2024-04-07T17:51:29.141 INFO:tasks.workunit.client.0.smithi100.stdout: "Intel(R) Xeon(R) CPU E5-1620 v4 @ 3.50GHz": 8 2024-04-07T17:51:29.141 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.141 INFO:tasks.workunit.client.0.smithi100.stdout: "distro": { 2024-04-07T17:51:29.141 INFO:tasks.workunit.client.0.smithi100.stdout: "centos": 8 2024-04-07T17:51:29.141 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.141 INFO:tasks.workunit.client.0.smithi100.stdout: "distro_description": { 2024-04-07T17:51:29.141 INFO:tasks.workunit.client.0.smithi100.stdout: "CentOS Stream 8": 8 2024-04-07T17:51:29.141 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.142 INFO:tasks.workunit.client.0.smithi100.stdout: "kernel_description": { 2024-04-07T17:51:29.142 INFO:tasks.workunit.client.0.smithi100.stdout: "#111-Ubuntu SMP Tue Mar 5 20:16:58 UTC 2024": 8 2024-04-07T17:51:29.142 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.142 INFO:tasks.workunit.client.0.smithi100.stdout: "kernel_version": { 2024-04-07T17:51:29.142 INFO:tasks.workunit.client.0.smithi100.stdout: "5.15.0-101-generic": 8 2024-04-07T17:51:29.142 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.142 INFO:tasks.workunit.client.0.smithi100.stdout: "os": { 2024-04-07T17:51:29.142 INFO:tasks.workunit.client.0.smithi100.stdout: "Linux": 8 2024-04-07T17:51:29.142 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.142 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_objectstore": { 2024-04-07T17:51:29.142 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore": 8 2024-04-07T17:51:29.142 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.142 INFO:tasks.workunit.client.0.smithi100.stdout: "rotational": { 2024-04-07T17:51:29.142 INFO:tasks.workunit.client.0.smithi100.stdout: "0": 8 2024-04-07T17:51:29.142 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.143 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.143 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.143 INFO:tasks.workunit.client.0.smithi100.stdout: "mon": { 2024-04-07T17:51:29.143 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 3, 2024-04-07T17:51:29.143 INFO:tasks.workunit.client.0.smithi100.stdout: "features": { 2024-04-07T17:51:29.143 INFO:tasks.workunit.client.0.smithi100.stdout: "optional": [], 2024-04-07T17:51:29.143 INFO:tasks.workunit.client.0.smithi100.stdout: "persistent": [ 2024-04-07T17:51:29.143 INFO:tasks.workunit.client.0.smithi100.stdout: "kraken", 2024-04-07T17:51:29.143 INFO:tasks.workunit.client.0.smithi100.stdout: "luminous", 2024-04-07T17:51:29.143 INFO:tasks.workunit.client.0.smithi100.stdout: "mimic", 2024-04-07T17:51:29.143 INFO:tasks.workunit.client.0.smithi100.stdout: "osdmap-prune", 2024-04-07T17:51:29.143 INFO:tasks.workunit.client.0.smithi100.stdout: "nautilus", 2024-04-07T17:51:29.143 INFO:tasks.workunit.client.0.smithi100.stdout: "octopus", 2024-04-07T17:51:29.143 INFO:tasks.workunit.client.0.smithi100.stdout: "pacific", 2024-04-07T17:51:29.143 INFO:tasks.workunit.client.0.smithi100.stdout: "elector-pinging", 2024-04-07T17:51:29.143 INFO:tasks.workunit.client.0.smithi100.stdout: "quincy", 2024-04-07T17:51:29.144 INFO:tasks.workunit.client.0.smithi100.stdout: "reef", 2024-04-07T17:51:29.144 INFO:tasks.workunit.client.0.smithi100.stdout: "squid" 2024-04-07T17:51:29.144 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:29.144 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.144 INFO:tasks.workunit.client.0.smithi100.stdout: "ipv4_addr_mons": 6, 2024-04-07T17:51:29.144 INFO:tasks.workunit.client.0.smithi100.stdout: "ipv6_addr_mons": 0, 2024-04-07T17:51:29.144 INFO:tasks.workunit.client.0.smithi100.stdout: "min_mon_release": 19, 2024-04-07T17:51:29.144 INFO:tasks.workunit.client.0.smithi100.stdout: "v1_addr_mons": 3, 2024-04-07T17:51:29.144 INFO:tasks.workunit.client.0.smithi100.stdout: "v2_addr_mons": 3 2024-04-07T17:51:29.144 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.144 INFO:tasks.workunit.client.0.smithi100.stdout: "organization": "ceph-qa", 2024-04-07T17:51:29.144 INFO:tasks.workunit.client.0.smithi100.stdout: "osd": { 2024-04-07T17:51:29.144 INFO:tasks.workunit.client.0.smithi100.stdout: "cluster_network": false, 2024-04-07T17:51:29.144 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 8, 2024-04-07T17:51:29.144 INFO:tasks.workunit.client.0.smithi100.stdout: "require_min_compat_client": "luminous", 2024-04-07T17:51:29.145 INFO:tasks.workunit.client.0.smithi100.stdout: "require_osd_release": "squid" 2024-04-07T17:51:29.145 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.145 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_perf_histograms": [ 2024-04-07T17:51:29.145 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:29.145 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_latency_out_bytes_histogram": { 2024-04-07T17:51:29.145 INFO:tasks.workunit.client.0.smithi100.stdout: "axes": [ 2024-04-07T17:51:29.145 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:29.145 INFO:tasks.workunit.client.0.smithi100.stdout: "buckets": 32, 2024-04-07T17:51:29.145 INFO:tasks.workunit.client.0.smithi100.stdout: "min": 0, 2024-04-07T17:51:29.145 INFO:tasks.workunit.client.0.smithi100.stdout: "name": "Latency (usec)", 2024-04-07T17:51:29.145 INFO:tasks.workunit.client.0.smithi100.stdout: "quant_size": 100000, 2024-04-07T17:51:29.145 INFO:tasks.workunit.client.0.smithi100.stdout: "ranges": [ 2024-04-07T17:51:29.145 INFO:tasks.workunit.client.0.smithi100.stdout: "[None, -1]", 2024-04-07T17:51:29.145 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 99999]", 2024-04-07T17:51:29.145 INFO:tasks.workunit.client.0.smithi100.stdout: "[100000, 199999]", 2024-04-07T17:51:29.146 INFO:tasks.workunit.client.0.smithi100.stdout: "[200000, 399999]", 2024-04-07T17:51:29.146 INFO:tasks.workunit.client.0.smithi100.stdout: "[400000, 799999]", 2024-04-07T17:51:29.146 INFO:tasks.workunit.client.0.smithi100.stdout: "[800000, 1599999]", 2024-04-07T17:51:29.146 INFO:tasks.workunit.client.0.smithi100.stdout: "[1600000, 3199999]", 2024-04-07T17:51:29.146 INFO:tasks.workunit.client.0.smithi100.stdout: "[3200000, 6399999]", 2024-04-07T17:51:29.146 INFO:tasks.workunit.client.0.smithi100.stdout: "[6400000, 12799999]", 2024-04-07T17:51:29.146 INFO:tasks.workunit.client.0.smithi100.stdout: "[12800000, 25599999]", 2024-04-07T17:51:29.146 INFO:tasks.workunit.client.0.smithi100.stdout: "[25600000, 51199999]", 2024-04-07T17:51:29.146 INFO:tasks.workunit.client.0.smithi100.stdout: "[51200000, 102399999]", 2024-04-07T17:51:29.146 INFO:tasks.workunit.client.0.smithi100.stdout: "[102400000, 204799999]", 2024-04-07T17:51:29.146 INFO:tasks.workunit.client.0.smithi100.stdout: "[204800000, 409599999]", 2024-04-07T17:51:29.146 INFO:tasks.workunit.client.0.smithi100.stdout: "[409600000, 819199999]", 2024-04-07T17:51:29.146 INFO:tasks.workunit.client.0.smithi100.stdout: "[819200000, 1638399999]", 2024-04-07T17:51:29.146 INFO:tasks.workunit.client.0.smithi100.stdout: "[1638400000, 3276799999]", 2024-04-07T17:51:29.146 INFO:tasks.workunit.client.0.smithi100.stdout: "[3276800000, 6553599999]", 2024-04-07T17:51:29.146 INFO:tasks.workunit.client.0.smithi100.stdout: "[6553600000, 13107199999]", 2024-04-07T17:51:29.147 INFO:tasks.workunit.client.0.smithi100.stdout: "[13107200000, 26214399999]", 2024-04-07T17:51:29.147 INFO:tasks.workunit.client.0.smithi100.stdout: "[26214400000, 52428799999]", 2024-04-07T17:51:29.147 INFO:tasks.workunit.client.0.smithi100.stdout: "[52428800000, 104857599999]", 2024-04-07T17:51:29.147 INFO:tasks.workunit.client.0.smithi100.stdout: "[104857600000, 209715199999]", 2024-04-07T17:51:29.147 INFO:tasks.workunit.client.0.smithi100.stdout: "[209715200000, 419430399999]", 2024-04-07T17:51:29.147 INFO:tasks.workunit.client.0.smithi100.stdout: "[419430400000, 838860799999]", 2024-04-07T17:51:29.147 INFO:tasks.workunit.client.0.smithi100.stdout: "[838860800000, 1677721599999]", 2024-04-07T17:51:29.147 INFO:tasks.workunit.client.0.smithi100.stdout: "[1677721600000, 3355443199999]", 2024-04-07T17:51:29.147 INFO:tasks.workunit.client.0.smithi100.stdout: "[3355443200000, 6710886399999]", 2024-04-07T17:51:29.147 INFO:tasks.workunit.client.0.smithi100.stdout: "[6710886400000, 13421772799999]", 2024-04-07T17:51:29.147 INFO:tasks.workunit.client.0.smithi100.stdout: "[13421772800000, 26843545599999]", 2024-04-07T17:51:29.147 INFO:tasks.workunit.client.0.smithi100.stdout: "[26843545600000, 53687091199999]", 2024-04-07T17:51:29.147 INFO:tasks.workunit.client.0.smithi100.stdout: "[53687091200000, None]" 2024-04-07T17:51:29.147 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:29.148 INFO:tasks.workunit.client.0.smithi100.stdout: "scale_type": "log2" 2024-04-07T17:51:29.148 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.148 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:29.148 INFO:tasks.workunit.client.0.smithi100.stdout: "buckets": 32, 2024-04-07T17:51:29.148 INFO:tasks.workunit.client.0.smithi100.stdout: "min": 0, 2024-04-07T17:51:29.148 INFO:tasks.workunit.client.0.smithi100.stdout: "name": "Request size (bytes)", 2024-04-07T17:51:29.148 INFO:tasks.workunit.client.0.smithi100.stdout: "quant_size": 512, 2024-04-07T17:51:29.148 INFO:tasks.workunit.client.0.smithi100.stdout: "ranges": [ 2024-04-07T17:51:29.148 INFO:tasks.workunit.client.0.smithi100.stdout: "[None, -1]", 2024-04-07T17:51:29.148 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 511]", 2024-04-07T17:51:29.148 INFO:tasks.workunit.client.0.smithi100.stdout: "[512, 1023]", 2024-04-07T17:51:29.148 INFO:tasks.workunit.client.0.smithi100.stdout: "[1024, 2047]", 2024-04-07T17:51:29.148 INFO:tasks.workunit.client.0.smithi100.stdout: "[2048, 4095]", 2024-04-07T17:51:29.148 INFO:tasks.workunit.client.0.smithi100.stdout: "[4096, 8191]", 2024-04-07T17:51:29.149 INFO:tasks.workunit.client.0.smithi100.stdout: "[8192, 16383]", 2024-04-07T17:51:29.149 INFO:tasks.workunit.client.0.smithi100.stdout: "[16384, 32767]", 2024-04-07T17:51:29.149 INFO:tasks.workunit.client.0.smithi100.stdout: "[32768, 65535]", 2024-04-07T17:51:29.149 INFO:tasks.workunit.client.0.smithi100.stdout: "[65536, 131071]", 2024-04-07T17:51:29.149 INFO:tasks.workunit.client.0.smithi100.stdout: "[131072, 262143]", 2024-04-07T17:51:29.149 INFO:tasks.workunit.client.0.smithi100.stdout: "[262144, 524287]", 2024-04-07T17:51:29.149 INFO:tasks.workunit.client.0.smithi100.stdout: "[524288, 1048575]", 2024-04-07T17:51:29.149 INFO:tasks.workunit.client.0.smithi100.stdout: "[1048576, 2097151]", 2024-04-07T17:51:29.149 INFO:tasks.workunit.client.0.smithi100.stdout: "[2097152, 4194303]", 2024-04-07T17:51:29.149 INFO:tasks.workunit.client.0.smithi100.stdout: "[4194304, 8388607]", 2024-04-07T17:51:29.149 INFO:tasks.workunit.client.0.smithi100.stdout: "[8388608, 16777215]", 2024-04-07T17:51:29.149 INFO:tasks.workunit.client.0.smithi100.stdout: "[16777216, 33554431]", 2024-04-07T17:51:29.149 INFO:tasks.workunit.client.0.smithi100.stdout: "[33554432, 67108863]", 2024-04-07T17:51:29.149 INFO:tasks.workunit.client.0.smithi100.stdout: "[67108864, 134217727]", 2024-04-07T17:51:29.149 INFO:tasks.workunit.client.0.smithi100.stdout: "[134217728, 268435455]", 2024-04-07T17:51:29.150 INFO:tasks.workunit.client.0.smithi100.stdout: "[268435456, 536870911]", 2024-04-07T17:51:29.150 INFO:tasks.workunit.client.0.smithi100.stdout: "[536870912, 1073741823]", 2024-04-07T17:51:29.150 INFO:tasks.workunit.client.0.smithi100.stdout: "[1073741824, 2147483647]", 2024-04-07T17:51:29.150 INFO:tasks.workunit.client.0.smithi100.stdout: "[2147483648, 4294967295]", 2024-04-07T17:51:29.150 INFO:tasks.workunit.client.0.smithi100.stdout: "[4294967296, 8589934591]", 2024-04-07T17:51:29.150 INFO:tasks.workunit.client.0.smithi100.stdout: "[8589934592, 17179869183]", 2024-04-07T17:51:29.150 INFO:tasks.workunit.client.0.smithi100.stdout: "[17179869184, 34359738367]", 2024-04-07T17:51:29.150 INFO:tasks.workunit.client.0.smithi100.stdout: "[34359738368, 68719476735]", 2024-04-07T17:51:29.150 INFO:tasks.workunit.client.0.smithi100.stdout: "[68719476736, 137438953471]", 2024-04-07T17:51:29.150 INFO:tasks.workunit.client.0.smithi100.stdout: "[137438953472, 274877906943]", 2024-04-07T17:51:29.150 INFO:tasks.workunit.client.0.smithi100.stdout: "[274877906944, None]" 2024-04-07T17:51:29.150 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:29.150 INFO:tasks.workunit.client.0.smithi100.stdout: "scale_type": "log2" 2024-04-07T17:51:29.150 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.151 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:29.151 INFO:tasks.workunit.client.0.smithi100.stdout: "osds": [ 2024-04-07T17:51:29.151 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:29.151 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 0, 2024-04-07T17:51:29.151 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:29.151 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.151 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.151 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 997, 13, 0, 0, 648, 16, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.151 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 7216, 3165, 0, 1, 4231, 80, 0, 4, 1, 0, 3, 2, 4, 0, 12, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.151 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 11866, 1050, 12, 2, 2904, 503, 11, 11, 26, 39, 32, 36, 59, 21, 77, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.151 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1610, 30, 63, 0, 19, 15, 2, 5, 5, 13, 105, 163, 32, 5, 39, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.151 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 417, 0, 5, 0, 3, 9, 0, 1, 0, 1, 3, 37, 361, 22, 36, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.151 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 141, 1, 0, 0, 5, 11, 0, 0, 0, 1, 2, 13, 25, 31, 205, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.151 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 120, 1, 0, 0, 0, 0, 1, 2, 2, 0, 0, 7, 21, 15, 56, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.151 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 192, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 4, 14, 26, 33, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.152 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 32, 0, 0, 0, 0, 0, 0, 0, 0, 0, 2, 0, 4, 19, 20, 5, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.152 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 6, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 4, 3, 9, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.152 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.152 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.152 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.152 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.152 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.152 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.152 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.152 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.152 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.152 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.152 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.152 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.152 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.152 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.153 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.153 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.153 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.153 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.153 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.153 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:29.153 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:29.153 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.153 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:29.153 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 1, 2024-04-07T17:51:29.153 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:29.153 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.153 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.153 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1446, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.153 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 5796, 4, 145, 0, 8, 0, 0, 0, 1, 1, 2, 1, 2, 0, 9, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.153 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 9605, 33, 88, 8, 28, 51, 80, 25, 39, 53, 38, 35, 64, 23, 44, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.154 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1252, 11, 1178, 4, 3, 3, 38, 118, 6, 17, 129, 228, 29, 7, 16, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.154 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 284, 1, 144, 24, 0, 0, 0, 14, 2, 1, 4, 45, 487, 18, 5, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.154 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 188, 0, 9, 6, 0, 0, 0, 0, 0, 1, 8, 2, 4, 12, 161, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.154 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 174, 0, 76, 0, 0, 0, 0, 0, 0, 0, 9, 0, 4, 5, 36, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.154 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 95, 0, 62, 0, 0, 0, 0, 0, 0, 0, 3, 0, 3, 11, 17, 6, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.154 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 7, 0, 153, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 5, 2, 10, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.154 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 3, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.154 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.154 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.154 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.154 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.154 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.154 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.155 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.155 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.155 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.155 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.155 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.155 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.155 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.155 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.155 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.155 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.155 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.155 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.155 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.155 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:29.155 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:29.156 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.156 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:29.156 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 2, 2024-04-07T17:51:29.156 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:29.156 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.156 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.156 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 2187, 6, 0, 0, 0, 0, 0, 0, 0, 0, 2, 0, 0, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.156 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 6047, 2629, 2809, 0, 8, 0, 0, 1, 2, 0, 23, 0, 2, 7, 5, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.156 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 9066, 1269, 1692, 0, 3, 1, 0, 2, 6, 6, 32, 2, 20, 30, 68, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.156 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1309, 32, 27, 1, 2, 0, 0, 1, 9, 9, 29, 7, 17, 16, 33, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.156 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 335, 7, 6, 1, 0, 0, 0, 0, 0, 0, 3, 7, 47, 45, 17, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.156 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 152, 2, 0, 0, 0, 0, 0, 3, 0, 1, 3, 25, 11, 60, 181, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.156 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 149, 1, 0, 0, 0, 0, 0, 1, 3, 0, 3, 18, 37, 45, 52, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.156 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 208, 0, 0, 1, 0, 1, 2, 0, 0, 0, 4, 20, 43, 66, 44, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.156 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 52, 0, 0, 0, 0, 0, 0, 3, 0, 0, 0, 2, 3, 18, 51, 7, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.157 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 10, 0, 0, 0, 1, 0, 0, 1, 0, 0, 0, 0, 0, 8, 3, 4, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.157 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.157 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.157 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.157 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.157 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.157 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.157 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.157 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.157 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.157 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.157 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.157 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.157 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.157 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.157 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.158 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.158 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.158 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.158 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.158 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:29.158 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:29.158 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.158 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:29.158 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 3, 2024-04-07T17:51:29.158 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:29.158 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.158 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.158 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 3687, 36, 0, 1, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.158 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 12530, 2576, 0, 0, 13, 0, 0, 0, 0, 0, 1, 1, 1, 0, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.158 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 13127, 1790, 0, 0, 8, 6, 2, 8, 12, 16, 12, 7, 12, 5, 27, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.159 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1412, 42, 24, 0, 26, 0, 0, 2, 3, 5, 41, 74, 13, 6, 10, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.159 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 416, 3, 13, 0, 157, 0, 0, 0, 0, 0, 2, 12, 225, 42, 4, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.159 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 125, 1, 0, 0, 306, 1, 0, 0, 0, 0, 0, 0, 6, 21, 84, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.159 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 27, 0, 2, 2, 334, 0, 0, 0, 0, 0, 0, 2, 6, 7, 22, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.159 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 10, 0, 0, 0, 1487, 0, 0, 0, 0, 0, 0, 0, 4, 9, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.159 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 8, 0, 0, 0, 374, 0, 0, 0, 0, 0, 0, 0, 0, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.159 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.159 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.159 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.159 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.159 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.159 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.159 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.159 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.159 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.160 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.160 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.160 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.160 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.160 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.160 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.160 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.160 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.160 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.160 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.160 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.160 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:29.160 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:29.160 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.160 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:29.161 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 4, 2024-04-07T17:51:29.161 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:29.161 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.161 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.161 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 609, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.161 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 3885, 8, 15, 0, 0, 0, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.161 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 10736, 14, 53, 5, 22, 32, 30, 14, 13, 15, 11, 13, 30, 4, 36, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.161 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1540, 27, 25, 0, 9, 3, 41, 70, 1, 5, 58, 81, 10, 3, 28, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.161 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 375, 10, 14, 0, 0, 0, 0, 0, 0, 0, 1, 16, 198, 13, 3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.161 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 187, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 3, 9, 9, 56, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.161 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 61, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 5, 13, 3, 46, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.161 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 34, 0, 0, 0, 0, 0, 0, 0, 0, 2, 2, 3, 6, 3, 18, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.161 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 8, 0, 0, 0, 0, 0, 0, 1, 0, 0, 1, 0, 0, 0, 2, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.161 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.161 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.162 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.162 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.162 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.162 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.162 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.162 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.162 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.162 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.162 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.162 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.162 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.162 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.162 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.162 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.162 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.163 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.163 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.163 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.163 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:29.163 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:29.163 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.163 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:29.163 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 5, 2024-04-07T17:51:29.163 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:29.163 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.163 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.163 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 946, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.163 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 5643, 1, 0, 1, 0, 0, 1, 0, 0, 1, 3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.163 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 14942, 20, 3, 0, 21, 3, 0, 4, 1, 9, 44, 5, 10, 29, 54, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.163 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 2337, 103, 114, 1, 1, 1, 0, 2, 3, 4, 28, 12, 3, 5, 47, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.163 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 783, 40, 72, 4, 6, 0, 0, 0, 0, 0, 2, 0, 27, 26, 14, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.164 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 237, 9, 6, 4, 0, 0, 0, 2, 1, 0, 0, 3, 4, 39, 200, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.164 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 158, 10, 24, 5, 1, 0, 0, 1, 0, 0, 0, 2, 1, 6, 66, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.164 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 156, 1, 10, 45, 24, 0, 0, 0, 0, 0, 0, 0, 4, 6, 14, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.164 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 10, 0, 0, 16, 104, 86, 0, 0, 0, 0, 0, 0, 3, 6, 14, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.164 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 3, 0, 0, 0, 6, 43, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.164 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.164 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.164 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.164 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.164 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.164 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.164 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.164 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.164 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.164 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.165 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.165 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.165 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.165 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.165 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.165 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.165 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.165 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.165 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.165 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:29.165 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:29.165 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.165 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:29.165 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 6, 2024-04-07T17:51:29.165 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:29.165 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.166 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.166 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 830, 5, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.166 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 6114, 1983, 0, 0, 2, 1, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.166 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 10059, 2215, 1, 1, 22, 3, 1, 0, 18, 18, 10, 10, 26, 17, 43, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.166 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1593, 28, 21, 0, 2, 1, 0, 0, 5, 13, 58, 83, 12, 5, 27, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.166 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 625, 3, 7, 0, 0, 0, 0, 0, 0, 1, 7, 26, 211, 19, 7, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.166 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 531, 11, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 6, 22, 90, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.166 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 196, 8, 0, 0, 0, 1, 0, 0, 0, 1, 0, 0, 2, 1, 53, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.166 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 155, 8, 0, 0, 0, 0, 0, 1, 0, 3, 0, 0, 0, 1, 6, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.166 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 8, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.166 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.166 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.166 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.166 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.167 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.167 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.167 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.167 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.167 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.167 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.167 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.167 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.167 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.167 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.167 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.167 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.167 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.167 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.167 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.167 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.168 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:29.168 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:29.168 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.168 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:29.168 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 7, 2024-04-07T17:51:29.168 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:29.168 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.168 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.168 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 385, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.168 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 6693, 2090, 1, 0, 0, 0, 0, 0, 0, 0, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.168 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 13796, 2103, 0, 0, 7, 2, 0, 1, 0, 1, 1, 13, 10, 17, 55, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.168 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1294, 17, 4, 1, 0, 0, 0, 1, 0, 2, 17, 6, 3, 7, 31, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.168 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 308, 17, 0, 0, 0, 0, 0, 0, 0, 0, 0, 4, 20, 40, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.168 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 152, 63, 8, 0, 0, 0, 0, 2, 0, 0, 3, 1, 6, 32, 113, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.169 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 98, 138, 5, 0, 0, 0, 0, 1, 0, 0, 0, 2, 2, 7, 52, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.169 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 223, 58, 1, 0, 0, 0, 0, 0, 0, 0, 3, 0, 0, 8, 11, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.169 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 49, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 6, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.169 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.169 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.169 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.169 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.169 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.169 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.169 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.169 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.169 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.169 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.169 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.169 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.170 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.170 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.170 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.170 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.170 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.170 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.170 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.170 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.170 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:29.170 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:29.170 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.170 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:29.170 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.170 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_latency_in_bytes_histogram": { 2024-04-07T17:51:29.170 INFO:tasks.workunit.client.0.smithi100.stdout: "axes": [ 2024-04-07T17:51:29.171 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:29.171 INFO:tasks.workunit.client.0.smithi100.stdout: "buckets": 32, 2024-04-07T17:51:29.171 INFO:tasks.workunit.client.0.smithi100.stdout: "min": 0, 2024-04-07T17:51:29.171 INFO:tasks.workunit.client.0.smithi100.stdout: "name": "Latency (usec)", 2024-04-07T17:51:29.171 INFO:tasks.workunit.client.0.smithi100.stdout: "quant_size": 100000, 2024-04-07T17:51:29.171 INFO:tasks.workunit.client.0.smithi100.stdout: "ranges": [ 2024-04-07T17:51:29.171 INFO:tasks.workunit.client.0.smithi100.stdout: "[None, -1]", 2024-04-07T17:51:29.171 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 99999]", 2024-04-07T17:51:29.171 INFO:tasks.workunit.client.0.smithi100.stdout: "[100000, 199999]", 2024-04-07T17:51:29.171 INFO:tasks.workunit.client.0.smithi100.stdout: "[200000, 399999]", 2024-04-07T17:51:29.171 INFO:tasks.workunit.client.0.smithi100.stdout: "[400000, 799999]", 2024-04-07T17:51:29.171 INFO:tasks.workunit.client.0.smithi100.stdout: "[800000, 1599999]", 2024-04-07T17:51:29.171 INFO:tasks.workunit.client.0.smithi100.stdout: "[1600000, 3199999]", 2024-04-07T17:51:29.171 INFO:tasks.workunit.client.0.smithi100.stdout: "[3200000, 6399999]", 2024-04-07T17:51:29.171 INFO:tasks.workunit.client.0.smithi100.stdout: "[6400000, 12799999]", 2024-04-07T17:51:29.171 INFO:tasks.workunit.client.0.smithi100.stdout: "[12800000, 25599999]", 2024-04-07T17:51:29.172 INFO:tasks.workunit.client.0.smithi100.stdout: "[25600000, 51199999]", 2024-04-07T17:51:29.172 INFO:tasks.workunit.client.0.smithi100.stdout: "[51200000, 102399999]", 2024-04-07T17:51:29.172 INFO:tasks.workunit.client.0.smithi100.stdout: "[102400000, 204799999]", 2024-04-07T17:51:29.172 INFO:tasks.workunit.client.0.smithi100.stdout: "[204800000, 409599999]", 2024-04-07T17:51:29.172 INFO:tasks.workunit.client.0.smithi100.stdout: "[409600000, 819199999]", 2024-04-07T17:51:29.172 INFO:tasks.workunit.client.0.smithi100.stdout: "[819200000, 1638399999]", 2024-04-07T17:51:29.172 INFO:tasks.workunit.client.0.smithi100.stdout: "[1638400000, 3276799999]", 2024-04-07T17:51:29.172 INFO:tasks.workunit.client.0.smithi100.stdout: "[3276800000, 6553599999]", 2024-04-07T17:51:29.172 INFO:tasks.workunit.client.0.smithi100.stdout: "[6553600000, 13107199999]", 2024-04-07T17:51:29.172 INFO:tasks.workunit.client.0.smithi100.stdout: "[13107200000, 26214399999]", 2024-04-07T17:51:29.172 INFO:tasks.workunit.client.0.smithi100.stdout: "[26214400000, 52428799999]", 2024-04-07T17:51:29.172 INFO:tasks.workunit.client.0.smithi100.stdout: "[52428800000, 104857599999]", 2024-04-07T17:51:29.172 INFO:tasks.workunit.client.0.smithi100.stdout: "[104857600000, 209715199999]", 2024-04-07T17:51:29.172 INFO:tasks.workunit.client.0.smithi100.stdout: "[209715200000, 419430399999]", 2024-04-07T17:51:29.172 INFO:tasks.workunit.client.0.smithi100.stdout: "[419430400000, 838860799999]", 2024-04-07T17:51:29.172 INFO:tasks.workunit.client.0.smithi100.stdout: "[838860800000, 1677721599999]", 2024-04-07T17:51:29.173 INFO:tasks.workunit.client.0.smithi100.stdout: "[1677721600000, 3355443199999]", 2024-04-07T17:51:29.173 INFO:tasks.workunit.client.0.smithi100.stdout: "[3355443200000, 6710886399999]", 2024-04-07T17:51:29.173 INFO:tasks.workunit.client.0.smithi100.stdout: "[6710886400000, 13421772799999]", 2024-04-07T17:51:29.173 INFO:tasks.workunit.client.0.smithi100.stdout: "[13421772800000, 26843545599999]", 2024-04-07T17:51:29.173 INFO:tasks.workunit.client.0.smithi100.stdout: "[26843545600000, 53687091199999]", 2024-04-07T17:51:29.173 INFO:tasks.workunit.client.0.smithi100.stdout: "[53687091200000, None]" 2024-04-07T17:51:29.173 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:29.173 INFO:tasks.workunit.client.0.smithi100.stdout: "scale_type": "log2" 2024-04-07T17:51:29.173 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.173 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:29.173 INFO:tasks.workunit.client.0.smithi100.stdout: "buckets": 32, 2024-04-07T17:51:29.173 INFO:tasks.workunit.client.0.smithi100.stdout: "min": 0, 2024-04-07T17:51:29.173 INFO:tasks.workunit.client.0.smithi100.stdout: "name": "Request size (bytes)", 2024-04-07T17:51:29.173 INFO:tasks.workunit.client.0.smithi100.stdout: "quant_size": 512, 2024-04-07T17:51:29.173 INFO:tasks.workunit.client.0.smithi100.stdout: "ranges": [ 2024-04-07T17:51:29.173 INFO:tasks.workunit.client.0.smithi100.stdout: "[None, -1]", 2024-04-07T17:51:29.173 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 511]", 2024-04-07T17:51:29.174 INFO:tasks.workunit.client.0.smithi100.stdout: "[512, 1023]", 2024-04-07T17:51:29.174 INFO:tasks.workunit.client.0.smithi100.stdout: "[1024, 2047]", 2024-04-07T17:51:29.174 INFO:tasks.workunit.client.0.smithi100.stdout: "[2048, 4095]", 2024-04-07T17:51:29.174 INFO:tasks.workunit.client.0.smithi100.stdout: "[4096, 8191]", 2024-04-07T17:51:29.174 INFO:tasks.workunit.client.0.smithi100.stdout: "[8192, 16383]", 2024-04-07T17:51:29.174 INFO:tasks.workunit.client.0.smithi100.stdout: "[16384, 32767]", 2024-04-07T17:51:29.174 INFO:tasks.workunit.client.0.smithi100.stdout: "[32768, 65535]", 2024-04-07T17:51:29.174 INFO:tasks.workunit.client.0.smithi100.stdout: "[65536, 131071]", 2024-04-07T17:51:29.174 INFO:tasks.workunit.client.0.smithi100.stdout: "[131072, 262143]", 2024-04-07T17:51:29.174 INFO:tasks.workunit.client.0.smithi100.stdout: "[262144, 524287]", 2024-04-07T17:51:29.174 INFO:tasks.workunit.client.0.smithi100.stdout: "[524288, 1048575]", 2024-04-07T17:51:29.174 INFO:tasks.workunit.client.0.smithi100.stdout: "[1048576, 2097151]", 2024-04-07T17:51:29.174 INFO:tasks.workunit.client.0.smithi100.stdout: "[2097152, 4194303]", 2024-04-07T17:51:29.174 INFO:tasks.workunit.client.0.smithi100.stdout: "[4194304, 8388607]", 2024-04-07T17:51:29.175 INFO:tasks.workunit.client.0.smithi100.stdout: "[8388608, 16777215]", 2024-04-07T17:51:29.175 INFO:tasks.workunit.client.0.smithi100.stdout: "[16777216, 33554431]", 2024-04-07T17:51:29.175 INFO:tasks.workunit.client.0.smithi100.stdout: "[33554432, 67108863]", 2024-04-07T17:51:29.175 INFO:tasks.workunit.client.0.smithi100.stdout: "[67108864, 134217727]", 2024-04-07T17:51:29.175 INFO:tasks.workunit.client.0.smithi100.stdout: "[134217728, 268435455]", 2024-04-07T17:51:29.175 INFO:tasks.workunit.client.0.smithi100.stdout: "[268435456, 536870911]", 2024-04-07T17:51:29.175 INFO:tasks.workunit.client.0.smithi100.stdout: "[536870912, 1073741823]", 2024-04-07T17:51:29.175 INFO:tasks.workunit.client.0.smithi100.stdout: "[1073741824, 2147483647]", 2024-04-07T17:51:29.175 INFO:tasks.workunit.client.0.smithi100.stdout: "[2147483648, 4294967295]", 2024-04-07T17:51:29.175 INFO:tasks.workunit.client.0.smithi100.stdout: "[4294967296, 8589934591]", 2024-04-07T17:51:29.175 INFO:tasks.workunit.client.0.smithi100.stdout: "[8589934592, 17179869183]", 2024-04-07T17:51:29.175 INFO:tasks.workunit.client.0.smithi100.stdout: "[17179869184, 34359738367]", 2024-04-07T17:51:29.175 INFO:tasks.workunit.client.0.smithi100.stdout: "[34359738368, 68719476735]", 2024-04-07T17:51:29.175 INFO:tasks.workunit.client.0.smithi100.stdout: "[68719476736, 137438953471]", 2024-04-07T17:51:29.175 INFO:tasks.workunit.client.0.smithi100.stdout: "[137438953472, 274877906943]", 2024-04-07T17:51:29.176 INFO:tasks.workunit.client.0.smithi100.stdout: "[274877906944, None]" 2024-04-07T17:51:29.176 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:29.176 INFO:tasks.workunit.client.0.smithi100.stdout: "scale_type": "log2" 2024-04-07T17:51:29.176 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.176 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:29.176 INFO:tasks.workunit.client.0.smithi100.stdout: "osds": [ 2024-04-07T17:51:29.176 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:29.176 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 0, 2024-04-07T17:51:29.176 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:29.176 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.176 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.176 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.176 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.176 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.177 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 416, 12, 5, 6, 1, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.177 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 4275, 54, 74, 123, 377, 516, 419, 4, 11, 5, 0, 0, 3, 3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.177 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 775, 16, 26, 72, 112, 96, 307, 0, 3, 12, 2, 3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.177 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 369, 1, 0, 4, 2, 1, 2, 0, 1, 1, 0, 3, 13, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.177 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 201, 1, 0, 0, 0, 0, 1, 0, 0, 1, 0, 1, 10, 14, 29, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.177 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 88, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 1, 1, 7, 41, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.177 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 6, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 25, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.177 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.177 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.177 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.177 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.177 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.177 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.177 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.178 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.178 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.178 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.178 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.178 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.178 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.178 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.178 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.178 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.178 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.178 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.178 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.178 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:29.178 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:29.178 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.179 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:29.179 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 1, 2024-04-07T17:51:29.179 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:29.179 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.179 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.179 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.179 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.179 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.179 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 463, 210, 9, 10, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.179 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 2623, 76, 25, 59, 122, 1, 2, 2, 0, 0, 0, 1, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.179 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 819, 8, 815, 0, 1, 0, 1, 0, 2, 1, 0, 1, 0, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.179 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 994, 507, 544, 0, 0, 0, 0, 0, 0, 0, 1, 1, 5, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.179 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 844, 647, 73, 0, 0, 0, 0, 1, 0, 0, 1, 0, 5, 7, 15, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.179 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1618, 1450, 103, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 4, 34, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.179 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 291, 394, 108, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 12, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.179 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.180 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.180 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.180 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.180 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.180 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.180 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.180 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.180 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.180 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.180 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.180 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.180 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.180 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.180 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.180 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.180 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.180 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.181 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.181 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:29.181 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:29.181 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.181 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:29.181 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 2, 2024-04-07T17:51:29.181 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:29.181 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.181 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.181 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.181 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.181 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.181 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 248, 5, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.181 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 7869, 12, 0, 0, 0, 2, 5, 3, 9, 2, 0, 2, 3, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.181 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 17697, 0, 1, 0, 1, 0, 0, 1, 2, 11, 0, 2, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.182 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 887, 0, 1, 0, 0, 0, 0, 0, 1, 0, 0, 2, 9, 3, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.182 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 267, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 7, 21, 28, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.182 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 103, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 6, 6, 37, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.182 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 2, 23, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.182 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.182 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.182 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.182 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.182 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.182 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.182 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.182 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.182 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.182 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.182 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.183 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.183 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.183 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.183 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.183 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.183 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.183 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.183 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.183 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:29.183 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:29.183 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.183 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:29.183 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 3, 2024-04-07T17:51:29.183 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:29.183 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.183 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.184 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.184 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.184 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.184 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 280, 1, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.184 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 3748, 33, 1, 0, 5, 1, 5, 4, 1, 0, 0, 0, 3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.184 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 519, 29, 31, 40, 1, 0, 1, 0, 0, 0, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.184 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 290, 227, 3, 37, 0, 0, 0, 0, 0, 0, 0, 2, 3, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.184 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 2130, 1325, 1, 172, 0, 0, 0, 0, 0, 0, 0, 0, 6, 6, 16, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.184 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 2779, 1327, 0, 285, 0, 0, 0, 0, 0, 0, 0, 1, 3, 4, 24, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.184 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 233, 115, 0, 51, 0, 0, 0, 0, 0, 0, 0, 0, 0, 2, 15, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.184 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.184 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.184 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.184 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.184 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.184 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.185 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.185 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.185 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.185 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.185 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.185 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.185 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.185 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.185 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.185 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.185 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.185 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.185 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.185 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:29.185 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:29.186 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.186 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:29.186 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 4, 2024-04-07T17:51:29.186 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:29.186 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.186 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.186 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.186 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.186 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.186 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 113, 11, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.186 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 3576, 127, 33, 67, 137, 233, 86, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.186 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 303, 84, 0, 0, 1, 40, 395, 0, 1, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.186 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 124, 2, 0, 0, 1, 2, 1, 0, 0, 0, 1, 2, 7, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.186 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 73, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 4, 4, 6, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.187 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 25, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 29, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.187 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 16, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.187 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.187 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.187 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.187 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.187 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.187 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.187 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.187 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.187 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.187 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.187 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.187 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.187 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.188 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.188 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.188 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.188 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.188 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.188 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.188 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:29.188 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:29.188 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.188 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:29.188 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 5, 2024-04-07T17:51:29.188 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:29.188 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.188 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.188 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.188 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.189 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.189 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 109, 21, 86, 200, 454, 576, 651, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.189 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 5257, 50, 38, 54, 17, 36, 74, 1, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.189 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 689, 12, 17, 5, 0, 1, 0, 1, 0, 1, 2, 3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.189 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 457, 32, 66, 127, 29, 1, 0, 2, 0, 0, 2, 4, 8, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.189 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 302, 8, 16, 1, 0, 0, 0, 1, 0, 2, 2, 1, 9, 18, 14, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.189 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 96, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 4, 8, 35, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.189 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 18, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.189 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.189 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.189 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.189 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.189 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.189 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.189 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.189 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.190 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.190 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.190 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.190 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.190 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.190 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.190 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.190 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.190 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.190 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.190 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.190 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:29.190 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:29.191 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.191 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:29.191 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 6, 2024-04-07T17:51:29.191 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:29.191 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.191 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.191 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.191 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.191 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.191 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 51, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.191 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 2959, 10, 0, 0, 1, 1, 0, 1, 0, 0, 0, 1, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.191 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 347, 9, 0, 0, 1, 1, 0, 0, 0, 1, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.191 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 267, 64, 0, 0, 0, 0, 1, 0, 1, 2, 0, 3, 7, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.191 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 150, 40, 0, 0, 0, 0, 0, 0, 0, 0, 1, 2, 3, 4, 6, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.191 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 58, 5, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 6, 27, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.191 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 6, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 2, 20, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.192 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.192 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.192 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.192 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.192 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.192 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.192 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.192 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.192 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.192 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.192 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.192 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.192 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.192 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.192 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.192 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.192 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.193 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.193 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.193 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:29.193 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:29.193 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.193 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:29.193 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 7, 2024-04-07T17:51:29.193 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:29.193 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.193 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.193 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.193 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.193 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.193 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 267, 14, 33, 41, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.193 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 4218, 22, 41, 115, 0, 1, 1, 1, 2, 0, 0, 0, 2, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.193 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 6038, 25, 32, 5, 0, 1, 0, 1, 2, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.193 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 292, 5, 2, 0, 0, 0, 0, 0, 0, 1, 0, 2, 4, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.194 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 85, 1, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 2, 13, 6, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.194 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 43, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 2, 5, 28, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.194 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 16, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.194 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.194 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.194 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.194 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.194 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.194 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.194 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.194 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.194 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.194 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.194 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.194 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.195 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.195 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.195 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.195 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.195 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.195 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.195 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.195 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:29.195 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:29.195 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.195 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:29.195 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.195 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_latency_out_bytes_histogram": { 2024-04-07T17:51:29.196 INFO:tasks.workunit.client.0.smithi100.stdout: "axes": [ 2024-04-07T17:51:29.196 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:29.196 INFO:tasks.workunit.client.0.smithi100.stdout: "buckets": 32, 2024-04-07T17:51:29.196 INFO:tasks.workunit.client.0.smithi100.stdout: "min": 0, 2024-04-07T17:51:29.196 INFO:tasks.workunit.client.0.smithi100.stdout: "name": "Latency (usec)", 2024-04-07T17:51:29.196 INFO:tasks.workunit.client.0.smithi100.stdout: "quant_size": 100000, 2024-04-07T17:51:29.196 INFO:tasks.workunit.client.0.smithi100.stdout: "ranges": [ 2024-04-07T17:51:29.196 INFO:tasks.workunit.client.0.smithi100.stdout: "[None, -1]", 2024-04-07T17:51:29.196 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 99999]", 2024-04-07T17:51:29.196 INFO:tasks.workunit.client.0.smithi100.stdout: "[100000, 199999]", 2024-04-07T17:51:29.196 INFO:tasks.workunit.client.0.smithi100.stdout: "[200000, 399999]", 2024-04-07T17:51:29.196 INFO:tasks.workunit.client.0.smithi100.stdout: "[400000, 799999]", 2024-04-07T17:51:29.196 INFO:tasks.workunit.client.0.smithi100.stdout: "[800000, 1599999]", 2024-04-07T17:51:29.196 INFO:tasks.workunit.client.0.smithi100.stdout: "[1600000, 3199999]", 2024-04-07T17:51:29.196 INFO:tasks.workunit.client.0.smithi100.stdout: "[3200000, 6399999]", 2024-04-07T17:51:29.197 INFO:tasks.workunit.client.0.smithi100.stdout: "[6400000, 12799999]", 2024-04-07T17:51:29.197 INFO:tasks.workunit.client.0.smithi100.stdout: "[12800000, 25599999]", 2024-04-07T17:51:29.197 INFO:tasks.workunit.client.0.smithi100.stdout: "[25600000, 51199999]", 2024-04-07T17:51:29.197 INFO:tasks.workunit.client.0.smithi100.stdout: "[51200000, 102399999]", 2024-04-07T17:51:29.197 INFO:tasks.workunit.client.0.smithi100.stdout: "[102400000, 204799999]", 2024-04-07T17:51:29.197 INFO:tasks.workunit.client.0.smithi100.stdout: "[204800000, 409599999]", 2024-04-07T17:51:29.197 INFO:tasks.workunit.client.0.smithi100.stdout: "[409600000, 819199999]", 2024-04-07T17:51:29.197 INFO:tasks.workunit.client.0.smithi100.stdout: "[819200000, 1638399999]", 2024-04-07T17:51:29.197 INFO:tasks.workunit.client.0.smithi100.stdout: "[1638400000, 3276799999]", 2024-04-07T17:51:29.197 INFO:tasks.workunit.client.0.smithi100.stdout: "[3276800000, 6553599999]", 2024-04-07T17:51:29.197 INFO:tasks.workunit.client.0.smithi100.stdout: "[6553600000, 13107199999]", 2024-04-07T17:51:29.197 INFO:tasks.workunit.client.0.smithi100.stdout: "[13107200000, 26214399999]", 2024-04-07T17:51:29.197 INFO:tasks.workunit.client.0.smithi100.stdout: "[26214400000, 52428799999]", 2024-04-07T17:51:29.197 INFO:tasks.workunit.client.0.smithi100.stdout: "[52428800000, 104857599999]", 2024-04-07T17:51:29.197 INFO:tasks.workunit.client.0.smithi100.stdout: "[104857600000, 209715199999]", 2024-04-07T17:51:29.197 INFO:tasks.workunit.client.0.smithi100.stdout: "[209715200000, 419430399999]", 2024-04-07T17:51:29.198 INFO:tasks.workunit.client.0.smithi100.stdout: "[419430400000, 838860799999]", 2024-04-07T17:51:29.198 INFO:tasks.workunit.client.0.smithi100.stdout: "[838860800000, 1677721599999]", 2024-04-07T17:51:29.198 INFO:tasks.workunit.client.0.smithi100.stdout: "[1677721600000, 3355443199999]", 2024-04-07T17:51:29.198 INFO:tasks.workunit.client.0.smithi100.stdout: "[3355443200000, 6710886399999]", 2024-04-07T17:51:29.198 INFO:tasks.workunit.client.0.smithi100.stdout: "[6710886400000, 13421772799999]", 2024-04-07T17:51:29.198 INFO:tasks.workunit.client.0.smithi100.stdout: "[13421772800000, 26843545599999]", 2024-04-07T17:51:29.198 INFO:tasks.workunit.client.0.smithi100.stdout: "[26843545600000, 53687091199999]", 2024-04-07T17:51:29.198 INFO:tasks.workunit.client.0.smithi100.stdout: "[53687091200000, None]" 2024-04-07T17:51:29.198 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:29.198 INFO:tasks.workunit.client.0.smithi100.stdout: "scale_type": "log2" 2024-04-07T17:51:29.198 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.198 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:29.198 INFO:tasks.workunit.client.0.smithi100.stdout: "buckets": 32, 2024-04-07T17:51:29.198 INFO:tasks.workunit.client.0.smithi100.stdout: "min": 0, 2024-04-07T17:51:29.199 INFO:tasks.workunit.client.0.smithi100.stdout: "name": "Request size (bytes)", 2024-04-07T17:51:29.199 INFO:tasks.workunit.client.0.smithi100.stdout: "quant_size": 512, 2024-04-07T17:51:29.199 INFO:tasks.workunit.client.0.smithi100.stdout: "ranges": [ 2024-04-07T17:51:29.199 INFO:tasks.workunit.client.0.smithi100.stdout: "[None, -1]", 2024-04-07T17:51:29.199 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 511]", 2024-04-07T17:51:29.199 INFO:tasks.workunit.client.0.smithi100.stdout: "[512, 1023]", 2024-04-07T17:51:29.199 INFO:tasks.workunit.client.0.smithi100.stdout: "[1024, 2047]", 2024-04-07T17:51:29.199 INFO:tasks.workunit.client.0.smithi100.stdout: "[2048, 4095]", 2024-04-07T17:51:29.199 INFO:tasks.workunit.client.0.smithi100.stdout: "[4096, 8191]", 2024-04-07T17:51:29.199 INFO:tasks.workunit.client.0.smithi100.stdout: "[8192, 16383]", 2024-04-07T17:51:29.199 INFO:tasks.workunit.client.0.smithi100.stdout: "[16384, 32767]", 2024-04-07T17:51:29.199 INFO:tasks.workunit.client.0.smithi100.stdout: "[32768, 65535]", 2024-04-07T17:51:29.199 INFO:tasks.workunit.client.0.smithi100.stdout: "[65536, 131071]", 2024-04-07T17:51:29.199 INFO:tasks.workunit.client.0.smithi100.stdout: "[131072, 262143]", 2024-04-07T17:51:29.199 INFO:tasks.workunit.client.0.smithi100.stdout: "[262144, 524287]", 2024-04-07T17:51:29.200 INFO:tasks.workunit.client.0.smithi100.stdout: "[524288, 1048575]", 2024-04-07T17:51:29.200 INFO:tasks.workunit.client.0.smithi100.stdout: "[1048576, 2097151]", 2024-04-07T17:51:29.200 INFO:tasks.workunit.client.0.smithi100.stdout: "[2097152, 4194303]", 2024-04-07T17:51:29.200 INFO:tasks.workunit.client.0.smithi100.stdout: "[4194304, 8388607]", 2024-04-07T17:51:29.200 INFO:tasks.workunit.client.0.smithi100.stdout: "[8388608, 16777215]", 2024-04-07T17:51:29.200 INFO:tasks.workunit.client.0.smithi100.stdout: "[16777216, 33554431]", 2024-04-07T17:51:29.200 INFO:tasks.workunit.client.0.smithi100.stdout: "[33554432, 67108863]", 2024-04-07T17:51:29.200 INFO:tasks.workunit.client.0.smithi100.stdout: "[67108864, 134217727]", 2024-04-07T17:51:29.200 INFO:tasks.workunit.client.0.smithi100.stdout: "[134217728, 268435455]", 2024-04-07T17:51:29.200 INFO:tasks.workunit.client.0.smithi100.stdout: "[268435456, 536870911]", 2024-04-07T17:51:29.200 INFO:tasks.workunit.client.0.smithi100.stdout: "[536870912, 1073741823]", 2024-04-07T17:51:29.200 INFO:tasks.workunit.client.0.smithi100.stdout: "[1073741824, 2147483647]", 2024-04-07T17:51:29.200 INFO:tasks.workunit.client.0.smithi100.stdout: "[2147483648, 4294967295]", 2024-04-07T17:51:29.200 INFO:tasks.workunit.client.0.smithi100.stdout: "[4294967296, 8589934591]", 2024-04-07T17:51:29.200 INFO:tasks.workunit.client.0.smithi100.stdout: "[8589934592, 17179869183]", 2024-04-07T17:51:29.201 INFO:tasks.workunit.client.0.smithi100.stdout: "[17179869184, 34359738367]", 2024-04-07T17:51:29.201 INFO:tasks.workunit.client.0.smithi100.stdout: "[34359738368, 68719476735]", 2024-04-07T17:51:29.201 INFO:tasks.workunit.client.0.smithi100.stdout: "[68719476736, 137438953471]", 2024-04-07T17:51:29.201 INFO:tasks.workunit.client.0.smithi100.stdout: "[137438953472, 274877906943]", 2024-04-07T17:51:29.201 INFO:tasks.workunit.client.0.smithi100.stdout: "[274877906944, None]" 2024-04-07T17:51:29.201 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:29.201 INFO:tasks.workunit.client.0.smithi100.stdout: "scale_type": "log2" 2024-04-07T17:51:29.201 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.201 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:29.201 INFO:tasks.workunit.client.0.smithi100.stdout: "osds": [ 2024-04-07T17:51:29.201 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:29.201 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 0, 2024-04-07T17:51:29.201 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:29.201 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.201 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.201 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.201 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.202 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.202 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 441, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.202 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 5864, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.202 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1424, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.202 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 398, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.202 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 258, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.202 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 139, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.202 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 32, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.202 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.202 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.202 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.202 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.202 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.202 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.202 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.203 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.203 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.203 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.203 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.203 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.203 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.203 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.203 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.203 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.203 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.203 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.203 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.203 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:29.203 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:29.203 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.204 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:29.204 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 1, 2024-04-07T17:51:29.204 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:29.204 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.204 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.204 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.204 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.204 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.204 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 692, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.204 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 2912, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.204 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1650, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.204 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 2053, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.204 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1593, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.204 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 3210, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.204 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 807, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.205 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.205 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.205 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.205 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.205 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.205 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.205 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.205 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.205 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.205 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.205 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.205 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.205 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.205 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.205 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.206 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.206 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.206 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.206 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.206 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:29.206 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:29.206 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.206 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:29.206 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 2, 2024-04-07T17:51:29.206 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:29.206 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.206 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.206 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.206 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.207 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.207 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 253, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.207 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 7908, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.207 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 17717, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.207 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 904, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.207 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 325, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.207 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 152, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.207 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 28, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.207 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.207 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.207 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.207 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.207 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.207 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.207 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.207 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.208 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.208 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.208 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.208 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.208 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.208 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.208 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.208 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.208 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.208 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.208 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.208 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:29.208 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:29.208 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.209 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:29.209 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 3, 2024-04-07T17:51:29.209 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:29.209 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.209 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.209 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.209 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.209 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.209 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 282, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.209 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 3801, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.209 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 624, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.209 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 564, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.209 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 3656, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.209 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 4423, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.209 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 416, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.209 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.210 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.210 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.210 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.210 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.210 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.210 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.210 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.210 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.210 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.210 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.210 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.210 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.210 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.210 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.210 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.211 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.211 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.211 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.211 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:29.211 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:29.211 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.211 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:29.211 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 4, 2024-04-07T17:51:29.211 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:29.211 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.211 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.211 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.211 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.211 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.211 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 126, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.211 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 4261, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.212 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 825, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.212 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 140, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.212 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 88, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.212 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 55, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.212 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 16, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.212 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.212 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.212 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.212 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.212 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.212 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.212 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.212 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.213 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.213 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.213 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.213 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.213 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.213 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.213 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.213 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.213 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.213 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.213 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.213 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:29.213 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:29.214 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.214 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:29.214 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 5, 2024-04-07T17:51:29.214 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:29.214 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.214 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.214 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.214 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.214 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.214 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 2097, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.214 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 5529, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.214 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 731, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.214 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 730, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.215 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 374, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.215 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 143, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.215 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 20, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.215 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.215 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.215 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.215 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.215 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.215 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.215 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.215 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.215 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.215 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.215 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.215 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.216 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.216 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.216 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.216 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.216 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.216 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.216 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.216 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:29.216 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:29.216 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.216 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:29.216 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 6, 2024-04-07T17:51:29.216 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:29.216 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.216 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.217 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.217 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.217 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.217 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 52, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.217 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 2975, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.217 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 360, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.217 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 346, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.217 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 206, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.217 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 97, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.217 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 28, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.217 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.217 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.217 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.217 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.217 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.218 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.218 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.218 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.218 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.218 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.218 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.218 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.218 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.218 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.218 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.218 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.218 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.218 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.219 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.219 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:29.219 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:29.219 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.219 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:29.219 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 7, 2024-04-07T17:51:29.219 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:29.219 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.219 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.219 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.219 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.219 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.219 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 355, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.219 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 4405, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.219 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 6107, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.220 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 307, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.220 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 109, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.220 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 79, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.220 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 19, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.220 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.220 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.220 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.220 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.220 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.220 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.220 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.220 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.220 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.220 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.220 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.220 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.221 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.221 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.221 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.221 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.221 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.221 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.221 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.221 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:29.221 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:29.221 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.221 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:29.221 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.221 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_latency_in_bytes_histogram": { 2024-04-07T17:51:29.221 INFO:tasks.workunit.client.0.smithi100.stdout: "axes": [ 2024-04-07T17:51:29.221 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:29.221 INFO:tasks.workunit.client.0.smithi100.stdout: "buckets": 32, 2024-04-07T17:51:29.222 INFO:tasks.workunit.client.0.smithi100.stdout: "min": 0, 2024-04-07T17:51:29.222 INFO:tasks.workunit.client.0.smithi100.stdout: "name": "Latency (usec)", 2024-04-07T17:51:29.222 INFO:tasks.workunit.client.0.smithi100.stdout: "quant_size": 100000, 2024-04-07T17:51:29.222 INFO:tasks.workunit.client.0.smithi100.stdout: "ranges": [ 2024-04-07T17:51:29.222 INFO:tasks.workunit.client.0.smithi100.stdout: "[None, -1]", 2024-04-07T17:51:29.222 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 99999]", 2024-04-07T17:51:29.222 INFO:tasks.workunit.client.0.smithi100.stdout: "[100000, 199999]", 2024-04-07T17:51:29.222 INFO:tasks.workunit.client.0.smithi100.stdout: "[200000, 399999]", 2024-04-07T17:51:29.222 INFO:tasks.workunit.client.0.smithi100.stdout: "[400000, 799999]", 2024-04-07T17:51:29.222 INFO:tasks.workunit.client.0.smithi100.stdout: "[800000, 1599999]", 2024-04-07T17:51:29.222 INFO:tasks.workunit.client.0.smithi100.stdout: "[1600000, 3199999]", 2024-04-07T17:51:29.222 INFO:tasks.workunit.client.0.smithi100.stdout: "[3200000, 6399999]", 2024-04-07T17:51:29.222 INFO:tasks.workunit.client.0.smithi100.stdout: "[6400000, 12799999]", 2024-04-07T17:51:29.222 INFO:tasks.workunit.client.0.smithi100.stdout: "[12800000, 25599999]", 2024-04-07T17:51:29.223 INFO:tasks.workunit.client.0.smithi100.stdout: "[25600000, 51199999]", 2024-04-07T17:51:29.223 INFO:tasks.workunit.client.0.smithi100.stdout: "[51200000, 102399999]", 2024-04-07T17:51:29.223 INFO:tasks.workunit.client.0.smithi100.stdout: "[102400000, 204799999]", 2024-04-07T17:51:29.223 INFO:tasks.workunit.client.0.smithi100.stdout: "[204800000, 409599999]", 2024-04-07T17:51:29.223 INFO:tasks.workunit.client.0.smithi100.stdout: "[409600000, 819199999]", 2024-04-07T17:51:29.223 INFO:tasks.workunit.client.0.smithi100.stdout: "[819200000, 1638399999]", 2024-04-07T17:51:29.223 INFO:tasks.workunit.client.0.smithi100.stdout: "[1638400000, 3276799999]", 2024-04-07T17:51:29.223 INFO:tasks.workunit.client.0.smithi100.stdout: "[3276800000, 6553599999]", 2024-04-07T17:51:29.223 INFO:tasks.workunit.client.0.smithi100.stdout: "[6553600000, 13107199999]", 2024-04-07T17:51:29.223 INFO:tasks.workunit.client.0.smithi100.stdout: "[13107200000, 26214399999]", 2024-04-07T17:51:29.223 INFO:tasks.workunit.client.0.smithi100.stdout: "[26214400000, 52428799999]", 2024-04-07T17:51:29.223 INFO:tasks.workunit.client.0.smithi100.stdout: "[52428800000, 104857599999]", 2024-04-07T17:51:29.223 INFO:tasks.workunit.client.0.smithi100.stdout: "[104857600000, 209715199999]", 2024-04-07T17:51:29.223 INFO:tasks.workunit.client.0.smithi100.stdout: "[209715200000, 419430399999]", 2024-04-07T17:51:29.223 INFO:tasks.workunit.client.0.smithi100.stdout: "[419430400000, 838860799999]", 2024-04-07T17:51:29.223 INFO:tasks.workunit.client.0.smithi100.stdout: "[838860800000, 1677721599999]", 2024-04-07T17:51:29.224 INFO:tasks.workunit.client.0.smithi100.stdout: "[1677721600000, 3355443199999]", 2024-04-07T17:51:29.224 INFO:tasks.workunit.client.0.smithi100.stdout: "[3355443200000, 6710886399999]", 2024-04-07T17:51:29.224 INFO:tasks.workunit.client.0.smithi100.stdout: "[6710886400000, 13421772799999]", 2024-04-07T17:51:29.224 INFO:tasks.workunit.client.0.smithi100.stdout: "[13421772800000, 26843545599999]", 2024-04-07T17:51:29.224 INFO:tasks.workunit.client.0.smithi100.stdout: "[26843545600000, 53687091199999]", 2024-04-07T17:51:29.224 INFO:tasks.workunit.client.0.smithi100.stdout: "[53687091200000, None]" 2024-04-07T17:51:29.224 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:29.224 INFO:tasks.workunit.client.0.smithi100.stdout: "scale_type": "log2" 2024-04-07T17:51:29.224 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.224 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:29.224 INFO:tasks.workunit.client.0.smithi100.stdout: "buckets": 32, 2024-04-07T17:51:29.224 INFO:tasks.workunit.client.0.smithi100.stdout: "min": 0, 2024-04-07T17:51:29.224 INFO:tasks.workunit.client.0.smithi100.stdout: "name": "Request size (bytes)", 2024-04-07T17:51:29.224 INFO:tasks.workunit.client.0.smithi100.stdout: "quant_size": 512, 2024-04-07T17:51:29.225 INFO:tasks.workunit.client.0.smithi100.stdout: "ranges": [ 2024-04-07T17:51:29.225 INFO:tasks.workunit.client.0.smithi100.stdout: "[None, -1]", 2024-04-07T17:51:29.225 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 511]", 2024-04-07T17:51:29.225 INFO:tasks.workunit.client.0.smithi100.stdout: "[512, 1023]", 2024-04-07T17:51:29.225 INFO:tasks.workunit.client.0.smithi100.stdout: "[1024, 2047]", 2024-04-07T17:51:29.225 INFO:tasks.workunit.client.0.smithi100.stdout: "[2048, 4095]", 2024-04-07T17:51:29.225 INFO:tasks.workunit.client.0.smithi100.stdout: "[4096, 8191]", 2024-04-07T17:51:29.225 INFO:tasks.workunit.client.0.smithi100.stdout: "[8192, 16383]", 2024-04-07T17:51:29.225 INFO:tasks.workunit.client.0.smithi100.stdout: "[16384, 32767]", 2024-04-07T17:51:29.225 INFO:tasks.workunit.client.0.smithi100.stdout: "[32768, 65535]", 2024-04-07T17:51:29.225 INFO:tasks.workunit.client.0.smithi100.stdout: "[65536, 131071]", 2024-04-07T17:51:29.225 INFO:tasks.workunit.client.0.smithi100.stdout: "[131072, 262143]", 2024-04-07T17:51:29.225 INFO:tasks.workunit.client.0.smithi100.stdout: "[262144, 524287]", 2024-04-07T17:51:29.225 INFO:tasks.workunit.client.0.smithi100.stdout: "[524288, 1048575]", 2024-04-07T17:51:29.225 INFO:tasks.workunit.client.0.smithi100.stdout: "[1048576, 2097151]", 2024-04-07T17:51:29.226 INFO:tasks.workunit.client.0.smithi100.stdout: "[2097152, 4194303]", 2024-04-07T17:51:29.226 INFO:tasks.workunit.client.0.smithi100.stdout: "[4194304, 8388607]", 2024-04-07T17:51:29.226 INFO:tasks.workunit.client.0.smithi100.stdout: "[8388608, 16777215]", 2024-04-07T17:51:29.226 INFO:tasks.workunit.client.0.smithi100.stdout: "[16777216, 33554431]", 2024-04-07T17:51:29.226 INFO:tasks.workunit.client.0.smithi100.stdout: "[33554432, 67108863]", 2024-04-07T17:51:29.226 INFO:tasks.workunit.client.0.smithi100.stdout: "[67108864, 134217727]", 2024-04-07T17:51:29.226 INFO:tasks.workunit.client.0.smithi100.stdout: "[134217728, 268435455]", 2024-04-07T17:51:29.226 INFO:tasks.workunit.client.0.smithi100.stdout: "[268435456, 536870911]", 2024-04-07T17:51:29.226 INFO:tasks.workunit.client.0.smithi100.stdout: "[536870912, 1073741823]", 2024-04-07T17:51:29.226 INFO:tasks.workunit.client.0.smithi100.stdout: "[1073741824, 2147483647]", 2024-04-07T17:51:29.226 INFO:tasks.workunit.client.0.smithi100.stdout: "[2147483648, 4294967295]", 2024-04-07T17:51:29.226 INFO:tasks.workunit.client.0.smithi100.stdout: "[4294967296, 8589934591]", 2024-04-07T17:51:29.226 INFO:tasks.workunit.client.0.smithi100.stdout: "[8589934592, 17179869183]", 2024-04-07T17:51:29.226 INFO:tasks.workunit.client.0.smithi100.stdout: "[17179869184, 34359738367]", 2024-04-07T17:51:29.226 INFO:tasks.workunit.client.0.smithi100.stdout: "[34359738368, 68719476735]", 2024-04-07T17:51:29.226 INFO:tasks.workunit.client.0.smithi100.stdout: "[68719476736, 137438953471]", 2024-04-07T17:51:29.227 INFO:tasks.workunit.client.0.smithi100.stdout: "[137438953472, 274877906943]", 2024-04-07T17:51:29.227 INFO:tasks.workunit.client.0.smithi100.stdout: "[274877906944, None]" 2024-04-07T17:51:29.227 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:29.227 INFO:tasks.workunit.client.0.smithi100.stdout: "scale_type": "log2" 2024-04-07T17:51:29.227 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.227 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:29.227 INFO:tasks.workunit.client.0.smithi100.stdout: "osds": [ 2024-04-07T17:51:29.227 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:29.227 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 0, 2024-04-07T17:51:29.227 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:29.227 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.227 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.227 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.227 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.227 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.228 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 45, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.228 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 604, 18, 1, 2, 6, 0, 3, 3, 6, 3, 4, 1, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.228 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 2693, 8, 3, 0, 4, 3, 9, 15, 24, 57, 124, 51, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.228 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 270, 5, 0, 0, 1, 1, 0, 4, 3, 10, 38, 191, 289, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.228 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 410, 5, 0, 0, 0, 0, 1, 1, 5, 7, 14, 34, 121, 20, 38, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.228 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 566, 0, 0, 0, 0, 0, 0, 0, 0, 2, 3, 6, 14, 38, 152, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.228 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 778, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 4, 29, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.228 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 481, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.228 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 388, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.228 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.228 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 4, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.228 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.228 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.228 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.228 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.228 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.229 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.229 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.229 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.229 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.229 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.229 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.229 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.229 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.229 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.229 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.229 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:29.229 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:29.229 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.229 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:29.229 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 1, 2024-04-07T17:51:29.229 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:29.230 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.230 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.230 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.230 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.230 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.230 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 21, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.230 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 641, 34, 4, 2, 6, 0, 0, 1, 0, 0, 2, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.230 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 3452, 9, 3, 2, 2, 7, 13, 17, 33, 73, 134, 69, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.230 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 133, 5, 0, 0, 2, 1, 2, 1, 3, 6, 19, 211, 422, 3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.230 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 66, 11, 0, 0, 0, 0, 0, 0, 3, 1, 6, 9, 158, 22, 32, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.230 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 191, 32, 0, 0, 0, 0, 0, 0, 0, 0, 6, 9, 24, 39, 105, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.230 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1198, 50, 0, 0, 0, 0, 0, 0, 0, 1, 0, 1, 0, 3, 18, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.230 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1626, 138, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.230 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 662, 17, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.231 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.231 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.231 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.231 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.231 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.231 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.231 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.231 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.231 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.231 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.231 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.231 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.231 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.231 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.231 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.232 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.232 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.232 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:29.232 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:29.232 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.232 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:29.232 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 2, 2024-04-07T17:51:29.232 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:29.232 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.232 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.232 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.232 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.232 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.232 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 22, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.232 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 335, 6, 1, 0, 4, 0, 1, 5, 6, 2, 5, 5, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.232 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 207, 9, 1, 0, 3, 0, 4, 3, 10, 17, 51, 63, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.233 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 199, 14, 0, 0, 0, 1, 4, 6, 13, 13, 79, 92, 13, 6, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.233 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 155, 8, 0, 0, 0, 0, 2, 2, 4, 11, 32, 52, 31, 36, 46, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.233 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 44, 0, 0, 0, 0, 0, 0, 0, 0, 3, 12, 14, 17, 59, 136, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.233 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 6, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 2, 4, 4, 27, 3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.233 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.233 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.233 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.233 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.233 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.233 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.233 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.233 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.233 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.233 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.233 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.234 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.234 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.234 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.234 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.234 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.234 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.234 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.234 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.234 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:29.234 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:29.234 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.234 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:29.234 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 3, 2024-04-07T17:51:29.234 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:29.234 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.235 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.235 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.235 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.235 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.235 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 26, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.235 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 494, 15, 0, 0, 11, 2, 2, 3, 5, 0, 3, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.235 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1353, 2, 2, 0, 3, 3, 5, 7, 20, 29, 59, 33, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.235 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 366, 7, 0, 0, 1, 0, 0, 3, 4, 6, 16, 89, 157, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.235 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 404, 5, 0, 0, 0, 0, 0, 1, 3, 1, 4, 17, 61, 29, 16, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.235 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 330, 0, 0, 0, 0, 0, 1, 0, 2, 1, 0, 4, 21, 46, 72, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.235 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 124, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 5, 3, 13, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.235 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 46, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.235 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.235 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.236 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.236 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.236 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.236 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.236 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.236 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.236 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.236 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.236 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.236 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.236 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.236 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.236 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.236 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.236 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.236 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.237 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:29.237 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:29.237 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.237 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:29.237 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 4, 2024-04-07T17:51:29.237 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:29.237 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.237 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.237 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.237 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.237 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.237 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 16, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.237 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 441, 22, 0, 0, 7, 0, 1, 0, 0, 0, 1, 2, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.237 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1429, 3, 1, 1, 4, 5, 1, 8, 16, 28, 51, 23, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.237 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 545, 0, 0, 0, 0, 2, 3, 3, 3, 3, 26, 99, 129, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.237 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 331, 0, 0, 0, 0, 0, 0, 0, 1, 3, 19, 19, 107, 9, 19, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.238 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 162, 0, 0, 0, 0, 0, 0, 0, 0, 1, 2, 3, 7, 23, 85, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.238 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 151, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 2, 19, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.238 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 71, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.238 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.238 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.238 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.238 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.238 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.238 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.238 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.238 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.238 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.238 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.238 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.239 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.239 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.239 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.239 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.239 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.239 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.239 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.239 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:29.239 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:29.239 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.239 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:29.239 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 5, 2024-04-07T17:51:29.239 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:29.239 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.239 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.239 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.240 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.240 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.240 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 12, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.240 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 550, 9, 0, 0, 5, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.240 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 111, 8, 0, 0, 8, 0, 0, 2, 3, 4, 9, 9, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.240 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 90, 5, 0, 0, 0, 0, 3, 4, 5, 5, 13, 12, 14, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.240 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 41, 4, 0, 0, 0, 0, 0, 0, 0, 3, 4, 15, 16, 15, 20, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.240 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 13, 0, 0, 0, 0, 0, 0, 1, 0, 0, 6, 4, 6, 42, 128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.240 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 3, 26, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.240 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.240 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.240 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.240 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.240 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.241 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.241 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.241 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.241 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.241 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.241 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.241 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.241 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.241 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.241 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.241 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.241 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.241 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.241 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.241 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:29.242 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:29.242 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.242 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:29.242 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 6, 2024-04-07T17:51:29.242 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:29.242 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.242 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.242 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.242 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.242 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.242 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 11, 0, 0, 1, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.242 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 423, 33, 0, 0, 16, 0, 2, 2, 1, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.242 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 1503, 6, 1, 2, 5, 0, 4, 10, 23, 27, 44, 18, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.242 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 492, 4, 0, 0, 1, 1, 1, 1, 1, 2, 19, 86, 117, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.243 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 382, 9, 0, 0, 0, 0, 0, 0, 1, 1, 4, 7, 121, 9, 15, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.243 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 160, 28, 0, 0, 0, 0, 0, 0, 1, 1, 1, 3, 11, 32, 99, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.243 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 129, 75, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 2, 20, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.243 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 75, 153, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.243 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.243 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.243 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.243 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.243 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.243 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.243 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.243 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.243 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.243 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.243 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.244 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.244 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.244 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.245 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.245 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.245 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.245 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.245 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:29.245 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:29.246 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.246 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:29.246 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 7, 2024-04-07T17:51:29.246 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:29.246 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.246 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.246 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.246 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.246 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.246 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 12, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.246 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 239, 5, 0, 0, 6, 0, 2, 3, 5, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.246 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 61, 4, 0, 0, 2, 1, 1, 2, 3, 12, 8, 13, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.246 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 75, 12, 0, 0, 0, 0, 2, 2, 2, 9, 9, 19, 6, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.247 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 49, 15, 0, 0, 0, 0, 1, 2, 2, 2, 8, 21, 29, 29, 10, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.247 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 13, 33, 0, 0, 0, 0, 1, 0, 0, 2, 2, 4, 15, 67, 122, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.247 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 64, 0, 0, 0, 0, 0, 0, 0, 1, 0, 1, 0, 4, 26, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.247 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 149, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.247 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.247 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.247 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.247 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.247 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.247 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.247 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.247 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.247 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.247 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.248 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.248 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.248 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.248 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.248 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.248 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.248 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.248 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.248 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:29.248 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:29.248 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.248 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:29.248 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.248 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.248 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:29.249 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_resrv_repnum_vs_duration": { 2024-04-07T17:51:29.249 INFO:tasks.workunit.client.0.smithi100.stdout: "axes": [ 2024-04-07T17:51:29.249 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:29.249 INFO:tasks.workunit.client.0.smithi100.stdout: "buckets": 8, 2024-04-07T17:51:29.249 INFO:tasks.workunit.client.0.smithi100.stdout: "min": 0, 2024-04-07T17:51:29.249 INFO:tasks.workunit.client.0.smithi100.stdout: "name": "number of replicas", 2024-04-07T17:51:29.249 INFO:tasks.workunit.client.0.smithi100.stdout: "quant_size": 1, 2024-04-07T17:51:29.249 INFO:tasks.workunit.client.0.smithi100.stdout: "ranges": [ 2024-04-07T17:51:29.249 INFO:tasks.workunit.client.0.smithi100.stdout: "[None, -1]", 2024-04-07T17:51:29.249 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0]", 2024-04-07T17:51:29.249 INFO:tasks.workunit.client.0.smithi100.stdout: "[1, 1]", 2024-04-07T17:51:29.249 INFO:tasks.workunit.client.0.smithi100.stdout: "[2, 2]", 2024-04-07T17:51:29.249 INFO:tasks.workunit.client.0.smithi100.stdout: "[3, 3]", 2024-04-07T17:51:29.249 INFO:tasks.workunit.client.0.smithi100.stdout: "[4, 4]", 2024-04-07T17:51:29.249 INFO:tasks.workunit.client.0.smithi100.stdout: "[5, 5]", 2024-04-07T17:51:29.250 INFO:tasks.workunit.client.0.smithi100.stdout: "[6, None]" 2024-04-07T17:51:29.250 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:29.250 INFO:tasks.workunit.client.0.smithi100.stdout: "scale_type": "linear" 2024-04-07T17:51:29.250 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.250 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:29.250 INFO:tasks.workunit.client.0.smithi100.stdout: "buckets": 10, 2024-04-07T17:51:29.250 INFO:tasks.workunit.client.0.smithi100.stdout: "min": 0, 2024-04-07T17:51:29.250 INFO:tasks.workunit.client.0.smithi100.stdout: "name": "duration", 2024-04-07T17:51:29.250 INFO:tasks.workunit.client.0.smithi100.stdout: "quant_size": 250000, 2024-04-07T17:51:29.250 INFO:tasks.workunit.client.0.smithi100.stdout: "ranges": [ 2024-04-07T17:51:29.250 INFO:tasks.workunit.client.0.smithi100.stdout: "[None, -1]", 2024-04-07T17:51:29.250 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 249999]", 2024-04-07T17:51:29.250 INFO:tasks.workunit.client.0.smithi100.stdout: "[250000, 499999]", 2024-04-07T17:51:29.250 INFO:tasks.workunit.client.0.smithi100.stdout: "[500000, 999999]", 2024-04-07T17:51:29.250 INFO:tasks.workunit.client.0.smithi100.stdout: "[1000000, 1999999]", 2024-04-07T17:51:29.251 INFO:tasks.workunit.client.0.smithi100.stdout: "[2000000, 3999999]", 2024-04-07T17:51:29.251 INFO:tasks.workunit.client.0.smithi100.stdout: "[4000000, 7999999]", 2024-04-07T17:51:29.251 INFO:tasks.workunit.client.0.smithi100.stdout: "[8000000, 15999999]", 2024-04-07T17:51:29.251 INFO:tasks.workunit.client.0.smithi100.stdout: "[16000000, 31999999]", 2024-04-07T17:51:29.251 INFO:tasks.workunit.client.0.smithi100.stdout: "[32000000, None]" 2024-04-07T17:51:29.251 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:29.251 INFO:tasks.workunit.client.0.smithi100.stdout: "scale_type": "log2" 2024-04-07T17:51:29.251 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.251 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:29.251 INFO:tasks.workunit.client.0.smithi100.stdout: "osds": [ 2024-04-07T17:51:29.251 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:29.251 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 0, 2024-04-07T17:51:29.251 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:29.251 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.251 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.251 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.252 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.252 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.252 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.252 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.252 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:29.252 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:29.252 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.252 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:29.252 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 1, 2024-04-07T17:51:29.252 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:29.252 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.252 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.252 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.252 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.252 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.253 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.253 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.253 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:29.253 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:29.253 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.253 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:29.253 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 2, 2024-04-07T17:51:29.253 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:29.253 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.253 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.253 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.253 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.253 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.253 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.253 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.253 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:29.253 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:29.254 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.254 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:29.254 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 3, 2024-04-07T17:51:29.254 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:29.254 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.254 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.254 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.254 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.254 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.254 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.254 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.254 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:29.254 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:29.254 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.255 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:29.255 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 4, 2024-04-07T17:51:29.255 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:29.255 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.255 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.255 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.255 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.255 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.255 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.255 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.255 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:29.255 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:29.255 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.255 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:29.255 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 5, 2024-04-07T17:51:29.256 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:29.256 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.256 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.256 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.256 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.256 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.256 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.256 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.256 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:29.256 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:29.256 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.256 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:29.256 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 6, 2024-04-07T17:51:29.256 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:29.257 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.257 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.257 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.257 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.257 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.257 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.257 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.257 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:29.257 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:29.257 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.257 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:29.257 INFO:tasks.workunit.client.0.smithi100.stdout: "osd_id": 7, 2024-04-07T17:51:29.257 INFO:tasks.workunit.client.0.smithi100.stdout: "values": [ 2024-04-07T17:51:29.257 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.257 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.257 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.258 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.258 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.258 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.258 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-04-07T17:51:29.258 INFO:tasks.workunit.client.0.smithi100.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-04-07T17:51:29.258 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:29.258 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.258 INFO:tasks.workunit.client.0.smithi100.stdout: ] 2024-04-07T17:51:29.258 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.258 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.258 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:29.258 INFO:tasks.workunit.client.0.smithi100.stdout: "perf_counters": { 2024-04-07T17:51:29.258 INFO:tasks.workunit.client.0.smithi100.stdout: "mds.f47a4ff15a45950a63160265564a123fc5da5e5e": { 2024-04-07T17:51:29.258 INFO:tasks.workunit.client.0.smithi100.stdout: "mds": { 2024-04-07T17:51:29.259 INFO:tasks.workunit.client.0.smithi100.stdout: "caps": { 2024-04-07T17:51:29.259 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.259 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.259 INFO:tasks.workunit.client.0.smithi100.stdout: "ceph_cap_op_flush_ack": { 2024-04-07T17:51:29.259 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.259 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.259 INFO:tasks.workunit.client.0.smithi100.stdout: "ceph_cap_op_flushsnap_ack": { 2024-04-07T17:51:29.259 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.259 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.259 INFO:tasks.workunit.client.0.smithi100.stdout: "ceph_cap_op_grant": { 2024-04-07T17:51:29.259 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.259 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.259 INFO:tasks.workunit.client.0.smithi100.stdout: "ceph_cap_op_revoke": { 2024-04-07T17:51:29.259 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.259 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.259 INFO:tasks.workunit.client.0.smithi100.stdout: "ceph_cap_op_trunc": { 2024-04-07T17:51:29.260 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.260 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.260 INFO:tasks.workunit.client.0.smithi100.stdout: "dir_commit": { 2024-04-07T17:51:29.260 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.260 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.260 INFO:tasks.workunit.client.0.smithi100.stdout: "dir_fetch_complete": { 2024-04-07T17:51:29.260 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 12 2024-04-07T17:51:29.260 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.260 INFO:tasks.workunit.client.0.smithi100.stdout: "dir_fetch_keys": { 2024-04-07T17:51:29.260 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 10 2024-04-07T17:51:29.260 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.260 INFO:tasks.workunit.client.0.smithi100.stdout: "dir_merge": { 2024-04-07T17:51:29.260 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.260 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.260 INFO:tasks.workunit.client.0.smithi100.stdout: "dir_split": { 2024-04-07T17:51:29.261 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.261 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.261 INFO:tasks.workunit.client.0.smithi100.stdout: "exported_inodes": { 2024-04-07T17:51:29.261 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.261 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.261 INFO:tasks.workunit.client.0.smithi100.stdout: "forward": { 2024-04-07T17:51:29.261 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.261 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.261 INFO:tasks.workunit.client.0.smithi100.stdout: "handle_client_cap_release": { 2024-04-07T17:51:29.261 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.261 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.261 INFO:tasks.workunit.client.0.smithi100.stdout: "handle_client_caps": { 2024-04-07T17:51:29.261 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.261 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.261 INFO:tasks.workunit.client.0.smithi100.stdout: "handle_client_caps_dirty": { 2024-04-07T17:51:29.262 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.262 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.262 INFO:tasks.workunit.client.0.smithi100.stdout: "handle_inode_file_caps": { 2024-04-07T17:51:29.262 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.262 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.262 INFO:tasks.workunit.client.0.smithi100.stdout: "imported_inodes": { 2024-04-07T17:51:29.262 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.262 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.262 INFO:tasks.workunit.client.0.smithi100.stdout: "inodes": { 2024-04-07T17:51:29.262 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 10 2024-04-07T17:51:29.262 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.262 INFO:tasks.workunit.client.0.smithi100.stdout: "inodes_expired": { 2024-04-07T17:51:29.262 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.262 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.263 INFO:tasks.workunit.client.0.smithi100.stdout: "inodes_pinned": { 2024-04-07T17:51:29.263 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 10 2024-04-07T17:51:29.263 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.263 INFO:tasks.workunit.client.0.smithi100.stdout: "inodes_with_caps": { 2024-04-07T17:51:29.263 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.263 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.263 INFO:tasks.workunit.client.0.smithi100.stdout: "load_cent": { 2024-04-07T17:51:29.263 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.263 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.263 INFO:tasks.workunit.client.0.smithi100.stdout: "openino_dir_fetch": { 2024-04-07T17:51:29.263 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.263 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.263 INFO:tasks.workunit.client.0.smithi100.stdout: "process_request_cap_release": { 2024-04-07T17:51:29.263 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.263 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.263 INFO:tasks.workunit.client.0.smithi100.stdout: "reply_latency": { 2024-04-07T17:51:29.264 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:29.264 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.264 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.264 INFO:tasks.workunit.client.0.smithi100.stdout: "request": { 2024-04-07T17:51:29.264 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.264 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.264 INFO:tasks.workunit.client.0.smithi100.stdout: "root_rbytes": { 2024-04-07T17:51:29.264 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.264 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.264 INFO:tasks.workunit.client.0.smithi100.stdout: "root_rfiles": { 2024-04-07T17:51:29.264 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.264 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.264 INFO:tasks.workunit.client.0.smithi100.stdout: "root_rsnaps": { 2024-04-07T17:51:29.264 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.264 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.265 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_reply": { 2024-04-07T17:51:29.265 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.265 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.265 INFO:tasks.workunit.client.0.smithi100.stdout: "subtrees": { 2024-04-07T17:51:29.265 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2 2024-04-07T17:51:29.265 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.265 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.265 INFO:tasks.workunit.client.0.smithi100.stdout: "mds_cache": { 2024-04-07T17:51:29.265 INFO:tasks.workunit.client.0.smithi100.stdout: "ireq_enqueue_scrub": { 2024-04-07T17:51:29.265 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.265 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.265 INFO:tasks.workunit.client.0.smithi100.stdout: "ireq_exportdir": { 2024-04-07T17:51:29.265 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.265 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.265 INFO:tasks.workunit.client.0.smithi100.stdout: "ireq_flush": { 2024-04-07T17:51:29.265 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.266 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.266 INFO:tasks.workunit.client.0.smithi100.stdout: "ireq_fragmentdir": { 2024-04-07T17:51:29.266 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.266 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.266 INFO:tasks.workunit.client.0.smithi100.stdout: "ireq_fragstats": { 2024-04-07T17:51:29.266 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.266 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.266 INFO:tasks.workunit.client.0.smithi100.stdout: "ireq_inodestats": { 2024-04-07T17:51:29.266 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.266 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.266 INFO:tasks.workunit.client.0.smithi100.stdout: "ireq_quiesce_inode": { 2024-04-07T17:51:29.266 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.266 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.266 INFO:tasks.workunit.client.0.smithi100.stdout: "ireq_quiesce_path": { 2024-04-07T17:51:29.266 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.267 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.267 INFO:tasks.workunit.client.0.smithi100.stdout: "num_recovering_enqueued": { 2024-04-07T17:51:29.267 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.267 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.267 INFO:tasks.workunit.client.0.smithi100.stdout: "num_recovering_prioritized": { 2024-04-07T17:51:29.267 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.267 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.267 INFO:tasks.workunit.client.0.smithi100.stdout: "num_recovering_processing": { 2024-04-07T17:51:29.267 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.267 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.267 INFO:tasks.workunit.client.0.smithi100.stdout: "num_strays": { 2024-04-07T17:51:29.267 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.267 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.267 INFO:tasks.workunit.client.0.smithi100.stdout: "num_strays_delayed": { 2024-04-07T17:51:29.267 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.268 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.268 INFO:tasks.workunit.client.0.smithi100.stdout: "num_strays_enqueuing": { 2024-04-07T17:51:29.268 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.268 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.268 INFO:tasks.workunit.client.0.smithi100.stdout: "recovery_completed": { 2024-04-07T17:51:29.268 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.268 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.268 INFO:tasks.workunit.client.0.smithi100.stdout: "recovery_started": { 2024-04-07T17:51:29.268 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.268 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.268 INFO:tasks.workunit.client.0.smithi100.stdout: "strays_created": { 2024-04-07T17:51:29.268 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.268 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.268 INFO:tasks.workunit.client.0.smithi100.stdout: "strays_enqueued": { 2024-04-07T17:51:29.268 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.268 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.269 INFO:tasks.workunit.client.0.smithi100.stdout: "strays_migrated": { 2024-04-07T17:51:29.269 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.269 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.269 INFO:tasks.workunit.client.0.smithi100.stdout: "strays_reintegrated": { 2024-04-07T17:51:29.269 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.269 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.269 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.269 INFO:tasks.workunit.client.0.smithi100.stdout: "mds_log": { 2024-04-07T17:51:29.269 INFO:tasks.workunit.client.0.smithi100.stdout: "ev": { 2024-04-07T17:51:29.269 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 17 2024-04-07T17:51:29.269 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.269 INFO:tasks.workunit.client.0.smithi100.stdout: "evadd": { 2024-04-07T17:51:29.269 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.269 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.269 INFO:tasks.workunit.client.0.smithi100.stdout: "evex": { 2024-04-07T17:51:29.269 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.270 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.270 INFO:tasks.workunit.client.0.smithi100.stdout: "evexd": { 2024-04-07T17:51:29.270 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.270 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.270 INFO:tasks.workunit.client.0.smithi100.stdout: "evexg": { 2024-04-07T17:51:29.270 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.270 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.270 INFO:tasks.workunit.client.0.smithi100.stdout: "evlrg": { 2024-04-07T17:51:29.270 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.270 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.270 INFO:tasks.workunit.client.0.smithi100.stdout: "evtrm": { 2024-04-07T17:51:29.270 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.270 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.270 INFO:tasks.workunit.client.0.smithi100.stdout: "jlat": { 2024-04-07T17:51:29.271 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:29.271 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.271 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.271 INFO:tasks.workunit.client.0.smithi100.stdout: "replayed": { 2024-04-07T17:51:29.271 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 17 2024-04-07T17:51:29.271 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.271 INFO:tasks.workunit.client.0.smithi100.stdout: "seg": { 2024-04-07T17:51:29.271 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1 2024-04-07T17:51:29.271 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.271 INFO:tasks.workunit.client.0.smithi100.stdout: "segadd": { 2024-04-07T17:51:29.271 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.271 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.271 INFO:tasks.workunit.client.0.smithi100.stdout: "segex": { 2024-04-07T17:51:29.271 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.271 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.271 INFO:tasks.workunit.client.0.smithi100.stdout: "segexd": { 2024-04-07T17:51:29.271 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.272 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.272 INFO:tasks.workunit.client.0.smithi100.stdout: "segexg": { 2024-04-07T17:51:29.272 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.272 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.272 INFO:tasks.workunit.client.0.smithi100.stdout: "segmjr": { 2024-04-07T17:51:29.272 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1 2024-04-07T17:51:29.272 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.272 INFO:tasks.workunit.client.0.smithi100.stdout: "segtrm": { 2024-04-07T17:51:29.272 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.272 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.272 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.272 INFO:tasks.workunit.client.0.smithi100.stdout: "mds_mem": { 2024-04-07T17:51:29.272 INFO:tasks.workunit.client.0.smithi100.stdout: "cap": { 2024-04-07T17:51:29.272 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.273 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.273 INFO:tasks.workunit.client.0.smithi100.stdout: "cap+": { 2024-04-07T17:51:29.273 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.273 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.273 INFO:tasks.workunit.client.0.smithi100.stdout: "cap-": { 2024-04-07T17:51:29.273 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.273 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.273 INFO:tasks.workunit.client.0.smithi100.stdout: "dir": { 2024-04-07T17:51:29.273 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 12 2024-04-07T17:51:29.273 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.273 INFO:tasks.workunit.client.0.smithi100.stdout: "dir+": { 2024-04-07T17:51:29.273 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 12 2024-04-07T17:51:29.273 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.273 INFO:tasks.workunit.client.0.smithi100.stdout: "dir-": { 2024-04-07T17:51:29.273 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.273 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.274 INFO:tasks.workunit.client.0.smithi100.stdout: "dn": { 2024-04-07T17:51:29.274 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 10 2024-04-07T17:51:29.274 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.274 INFO:tasks.workunit.client.0.smithi100.stdout: "dn+": { 2024-04-07T17:51:29.274 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 10 2024-04-07T17:51:29.274 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.274 INFO:tasks.workunit.client.0.smithi100.stdout: "dn-": { 2024-04-07T17:51:29.274 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.274 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.274 INFO:tasks.workunit.client.0.smithi100.stdout: "heap": { 2024-04-07T17:51:29.274 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 248184 2024-04-07T17:51:29.274 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.274 INFO:tasks.workunit.client.0.smithi100.stdout: "ino": { 2024-04-07T17:51:29.274 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 13 2024-04-07T17:51:29.275 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.275 INFO:tasks.workunit.client.0.smithi100.stdout: "ino+": { 2024-04-07T17:51:29.275 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 13 2024-04-07T17:51:29.275 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.275 INFO:tasks.workunit.client.0.smithi100.stdout: "ino-": { 2024-04-07T17:51:29.275 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.275 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.275 INFO:tasks.workunit.client.0.smithi100.stdout: "rss": { 2024-04-07T17:51:29.275 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 46296 2024-04-07T17:51:29.275 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.275 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.275 INFO:tasks.workunit.client.0.smithi100.stdout: "mds_server": { 2024-04-07T17:51:29.275 INFO:tasks.workunit.client.0.smithi100.stdout: "cap_acquisition_throttle": { 2024-04-07T17:51:29.275 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.276 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.276 INFO:tasks.workunit.client.0.smithi100.stdout: "cap_revoke_eviction": { 2024-04-07T17:51:29.276 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.276 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.276 INFO:tasks.workunit.client.0.smithi100.stdout: "handle_client_request": { 2024-04-07T17:51:29.276 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.276 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.276 INFO:tasks.workunit.client.0.smithi100.stdout: "handle_client_session": { 2024-04-07T17:51:29.276 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.276 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.276 INFO:tasks.workunit.client.0.smithi100.stdout: "handle_peer_request": { 2024-04-07T17:51:29.276 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.276 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.276 INFO:tasks.workunit.client.0.smithi100.stdout: "req_create_latency": { 2024-04-07T17:51:29.276 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:29.276 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.277 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.277 INFO:tasks.workunit.client.0.smithi100.stdout: "req_getattr_latency": { 2024-04-07T17:51:29.277 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:29.277 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.277 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.277 INFO:tasks.workunit.client.0.smithi100.stdout: "req_getfilelock_latency": { 2024-04-07T17:51:29.277 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:29.277 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.277 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.277 INFO:tasks.workunit.client.0.smithi100.stdout: "req_getvxattr_latency": { 2024-04-07T17:51:29.277 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:29.277 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.277 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.277 INFO:tasks.workunit.client.0.smithi100.stdout: "req_link_latency": { 2024-04-07T17:51:29.277 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:29.278 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.278 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.278 INFO:tasks.workunit.client.0.smithi100.stdout: "req_lookup_latency": { 2024-04-07T17:51:29.278 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:29.278 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.278 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.278 INFO:tasks.workunit.client.0.smithi100.stdout: "req_lookuphash_latency": { 2024-04-07T17:51:29.278 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:29.278 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.278 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.278 INFO:tasks.workunit.client.0.smithi100.stdout: "req_lookupino_latency": { 2024-04-07T17:51:29.278 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:29.278 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.278 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.278 INFO:tasks.workunit.client.0.smithi100.stdout: "req_lookupname_latency": { 2024-04-07T17:51:29.279 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:29.279 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.279 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.279 INFO:tasks.workunit.client.0.smithi100.stdout: "req_lookupparent_latency": { 2024-04-07T17:51:29.279 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:29.279 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.279 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.279 INFO:tasks.workunit.client.0.smithi100.stdout: "req_lookupsnap_latency": { 2024-04-07T17:51:29.279 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:29.279 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.279 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.279 INFO:tasks.workunit.client.0.smithi100.stdout: "req_lssnap_latency": { 2024-04-07T17:51:29.279 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:29.279 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.279 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.280 INFO:tasks.workunit.client.0.smithi100.stdout: "req_mkdir_latency": { 2024-04-07T17:51:29.280 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:29.280 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.280 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.280 INFO:tasks.workunit.client.0.smithi100.stdout: "req_mknod_latency": { 2024-04-07T17:51:29.280 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:29.280 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.280 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.280 INFO:tasks.workunit.client.0.smithi100.stdout: "req_mksnap_latency": { 2024-04-07T17:51:29.280 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:29.280 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.280 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.280 INFO:tasks.workunit.client.0.smithi100.stdout: "req_open_latency": { 2024-04-07T17:51:29.280 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:29.280 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.281 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.281 INFO:tasks.workunit.client.0.smithi100.stdout: "req_readdir_latency": { 2024-04-07T17:51:29.281 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:29.281 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.281 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.281 INFO:tasks.workunit.client.0.smithi100.stdout: "req_rename_latency": { 2024-04-07T17:51:29.281 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:29.281 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.281 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.281 INFO:tasks.workunit.client.0.smithi100.stdout: "req_renamesnap_latency": { 2024-04-07T17:51:29.281 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:29.281 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.281 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.281 INFO:tasks.workunit.client.0.smithi100.stdout: "req_rmdir_latency": { 2024-04-07T17:51:29.281 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:29.282 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.282 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.282 INFO:tasks.workunit.client.0.smithi100.stdout: "req_rmsnap_latency": { 2024-04-07T17:51:29.282 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:29.282 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.282 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.282 INFO:tasks.workunit.client.0.smithi100.stdout: "req_rmxattr_latency": { 2024-04-07T17:51:29.282 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:29.282 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.282 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.282 INFO:tasks.workunit.client.0.smithi100.stdout: "req_setattr_latency": { 2024-04-07T17:51:29.282 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:29.282 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.282 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.283 INFO:tasks.workunit.client.0.smithi100.stdout: "req_setdirlayout_latency": { 2024-04-07T17:51:29.283 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:29.283 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.283 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.283 INFO:tasks.workunit.client.0.smithi100.stdout: "req_setfilelock_latency": { 2024-04-07T17:51:29.283 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:29.283 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.283 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.283 INFO:tasks.workunit.client.0.smithi100.stdout: "req_setlayout_latency": { 2024-04-07T17:51:29.283 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:29.283 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.283 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.283 INFO:tasks.workunit.client.0.smithi100.stdout: "req_setxattr_latency": { 2024-04-07T17:51:29.283 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:29.283 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.283 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.284 INFO:tasks.workunit.client.0.smithi100.stdout: "req_snapdiff_latency": { 2024-04-07T17:51:29.284 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:29.284 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.284 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.284 INFO:tasks.workunit.client.0.smithi100.stdout: "req_symlink_latency": { 2024-04-07T17:51:29.284 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:29.284 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.284 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.284 INFO:tasks.workunit.client.0.smithi100.stdout: "req_unlink_latency": { 2024-04-07T17:51:29.284 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:29.284 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.284 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.284 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.284 INFO:tasks.workunit.client.0.smithi100.stdout: "mds_sessions": { 2024-04-07T17:51:29.284 INFO:tasks.workunit.client.0.smithi100.stdout: "average_load": { 2024-04-07T17:51:29.285 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.285 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.285 INFO:tasks.workunit.client.0.smithi100.stdout: "avg_session_uptime": { 2024-04-07T17:51:29.285 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.285 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.285 INFO:tasks.workunit.client.0.smithi100.stdout: "mdthresh_evicted": { 2024-04-07T17:51:29.285 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.285 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.285 INFO:tasks.workunit.client.0.smithi100.stdout: "session_add": { 2024-04-07T17:51:29.285 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 8 2024-04-07T17:51:29.285 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.285 INFO:tasks.workunit.client.0.smithi100.stdout: "session_count": { 2024-04-07T17:51:29.285 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.285 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.285 INFO:tasks.workunit.client.0.smithi100.stdout: "session_remove": { 2024-04-07T17:51:29.286 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 8 2024-04-07T17:51:29.286 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.286 INFO:tasks.workunit.client.0.smithi100.stdout: "sessions_open": { 2024-04-07T17:51:29.286 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2 2024-04-07T17:51:29.286 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.286 INFO:tasks.workunit.client.0.smithi100.stdout: "sessions_stale": { 2024-04-07T17:51:29.286 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.286 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.286 INFO:tasks.workunit.client.0.smithi100.stdout: "total_load": { 2024-04-07T17:51:29.286 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.286 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.286 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.286 INFO:tasks.workunit.client.0.smithi100.stdout: "objecter": { 2024-04-07T17:51:29.286 INFO:tasks.workunit.client.0.smithi100.stdout: "op_active": { 2024-04-07T17:51:29.286 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.287 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.287 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r": { 2024-04-07T17:51:29.287 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 37 2024-04-07T17:51:29.287 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.287 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rmw": { 2024-04-07T17:51:29.287 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.287 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.287 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w": { 2024-04-07T17:51:29.287 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 7 2024-04-07T17:51:29.287 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.287 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.287 INFO:tasks.workunit.client.0.smithi100.stdout: "purge_queue": { 2024-04-07T17:51:29.287 INFO:tasks.workunit.client.0.smithi100.stdout: "pq_executed": { 2024-04-07T17:51:29.287 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.287 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.287 INFO:tasks.workunit.client.0.smithi100.stdout: "pq_executed_ops": { 2024-04-07T17:51:29.288 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.288 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.288 INFO:tasks.workunit.client.0.smithi100.stdout: "pq_executing": { 2024-04-07T17:51:29.288 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.288 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.288 INFO:tasks.workunit.client.0.smithi100.stdout: "pq_executing_high_water": { 2024-04-07T17:51:29.288 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.288 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.288 INFO:tasks.workunit.client.0.smithi100.stdout: "pq_executing_ops": { 2024-04-07T17:51:29.288 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.288 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.288 INFO:tasks.workunit.client.0.smithi100.stdout: "pq_executing_ops_high_water": { 2024-04-07T17:51:29.288 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.288 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.288 INFO:tasks.workunit.client.0.smithi100.stdout: "pq_item_in_journal": { 2024-04-07T17:51:29.289 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.289 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.289 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.289 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.289 INFO:tasks.workunit.client.0.smithi100.stdout: "mon.8dd0baa036f0a7e6d1f3bdce86136fe527836c3d": { 2024-04-07T17:51:29.289 INFO:tasks.workunit.client.0.smithi100.stdout: "mon": { 2024-04-07T17:51:29.289 INFO:tasks.workunit.client.0.smithi100.stdout: "election_call": { 2024-04-07T17:51:29.289 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2 2024-04-07T17:51:29.289 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.289 INFO:tasks.workunit.client.0.smithi100.stdout: "election_lose": { 2024-04-07T17:51:29.289 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3 2024-04-07T17:51:29.289 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.289 INFO:tasks.workunit.client.0.smithi100.stdout: "election_win": { 2024-04-07T17:51:29.289 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.289 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.290 INFO:tasks.workunit.client.0.smithi100.stdout: "num_elections": { 2024-04-07T17:51:29.290 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 12 2024-04-07T17:51:29.290 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.290 INFO:tasks.workunit.client.0.smithi100.stdout: "num_sessions": { 2024-04-07T17:51:29.290 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 7 2024-04-07T17:51:29.290 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.290 INFO:tasks.workunit.client.0.smithi100.stdout: "session_add": { 2024-04-07T17:51:29.290 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4 2024-04-07T17:51:29.290 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.290 INFO:tasks.workunit.client.0.smithi100.stdout: "session_rm": { 2024-04-07T17:51:29.290 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 271 2024-04-07T17:51:29.290 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.290 INFO:tasks.workunit.client.0.smithi100.stdout: "session_trim": { 2024-04-07T17:51:29.290 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.290 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.291 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.291 INFO:tasks.workunit.client.0.smithi100.stdout: "paxos": { 2024-04-07T17:51:29.291 INFO:tasks.workunit.client.0.smithi100.stdout: "accept_timeout": { 2024-04-07T17:51:29.291 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.291 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.291 INFO:tasks.workunit.client.0.smithi100.stdout: "begin": { 2024-04-07T17:51:29.291 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 8809 2024-04-07T17:51:29.291 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.291 INFO:tasks.workunit.client.0.smithi100.stdout: "begin_bytes": { 2024-04-07T17:51:29.291 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 8809, 2024-04-07T17:51:29.291 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 59368816 2024-04-07T17:51:29.291 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.291 INFO:tasks.workunit.client.0.smithi100.stdout: "begin_keys": { 2024-04-07T17:51:29.291 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:29.291 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.292 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.292 INFO:tasks.workunit.client.0.smithi100.stdout: "begin_latency": { 2024-04-07T17:51:29.292 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 8809, 2024-04-07T17:51:29.292 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3099913251 2024-04-07T17:51:29.292 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.292 INFO:tasks.workunit.client.0.smithi100.stdout: "collect": { 2024-04-07T17:51:29.292 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3 2024-04-07T17:51:29.292 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.292 INFO:tasks.workunit.client.0.smithi100.stdout: "collect_bytes": { 2024-04-07T17:51:29.292 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 3, 2024-04-07T17:51:29.292 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 219 2024-04-07T17:51:29.292 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.292 INFO:tasks.workunit.client.0.smithi100.stdout: "collect_keys": { 2024-04-07T17:51:29.292 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 3, 2024-04-07T17:51:29.292 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3 2024-04-07T17:51:29.293 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.293 INFO:tasks.workunit.client.0.smithi100.stdout: "collect_latency": { 2024-04-07T17:51:29.293 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 3, 2024-04-07T17:51:29.293 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.293 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.293 INFO:tasks.workunit.client.0.smithi100.stdout: "collect_timeout": { 2024-04-07T17:51:29.293 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.293 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.293 INFO:tasks.workunit.client.0.smithi100.stdout: "collect_uncommitted": { 2024-04-07T17:51:29.293 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1 2024-04-07T17:51:29.293 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.293 INFO:tasks.workunit.client.0.smithi100.stdout: "commit": { 2024-04-07T17:51:29.293 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 8810 2024-04-07T17:51:29.293 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.293 INFO:tasks.workunit.client.0.smithi100.stdout: "commit_bytes": { 2024-04-07T17:51:29.293 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:29.294 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.294 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.294 INFO:tasks.workunit.client.0.smithi100.stdout: "commit_keys": { 2024-04-07T17:51:29.294 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:29.294 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.294 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.294 INFO:tasks.workunit.client.0.smithi100.stdout: "commit_latency": { 2024-04-07T17:51:29.294 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:29.294 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.294 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.294 INFO:tasks.workunit.client.0.smithi100.stdout: "lease_ack_timeout": { 2024-04-07T17:51:29.294 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.294 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.294 INFO:tasks.workunit.client.0.smithi100.stdout: "lease_timeout": { 2024-04-07T17:51:29.294 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.295 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.295 INFO:tasks.workunit.client.0.smithi100.stdout: "new_pn": { 2024-04-07T17:51:29.295 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.295 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.295 INFO:tasks.workunit.client.0.smithi100.stdout: "new_pn_latency": { 2024-04-07T17:51:29.295 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:29.295 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.295 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.295 INFO:tasks.workunit.client.0.smithi100.stdout: "refresh": { 2024-04-07T17:51:29.295 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 8810 2024-04-07T17:51:29.295 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.295 INFO:tasks.workunit.client.0.smithi100.stdout: "refresh_latency": { 2024-04-07T17:51:29.295 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 8810, 2024-04-07T17:51:29.295 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 52890505016 2024-04-07T17:51:29.295 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.296 INFO:tasks.workunit.client.0.smithi100.stdout: "restart": { 2024-04-07T17:51:29.296 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 14 2024-04-07T17:51:29.296 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.296 INFO:tasks.workunit.client.0.smithi100.stdout: "share_state": { 2024-04-07T17:51:29.296 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.296 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.296 INFO:tasks.workunit.client.0.smithi100.stdout: "share_state_bytes": { 2024-04-07T17:51:29.296 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:29.296 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.296 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.296 INFO:tasks.workunit.client.0.smithi100.stdout: "share_state_keys": { 2024-04-07T17:51:29.296 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:29.296 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.296 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.297 INFO:tasks.workunit.client.0.smithi100.stdout: "start_leader": { 2024-04-07T17:51:29.297 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.297 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.297 INFO:tasks.workunit.client.0.smithi100.stdout: "start_peon": { 2024-04-07T17:51:29.297 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3 2024-04-07T17:51:29.297 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.297 INFO:tasks.workunit.client.0.smithi100.stdout: "store_state": { 2024-04-07T17:51:29.297 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 8809 2024-04-07T17:51:29.297 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.297 INFO:tasks.workunit.client.0.smithi100.stdout: "store_state_bytes": { 2024-04-07T17:51:29.297 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 8809, 2024-04-07T17:51:29.297 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 117169042 2024-04-07T17:51:29.297 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.297 INFO:tasks.workunit.client.0.smithi100.stdout: "store_state_keys": { 2024-04-07T17:51:29.297 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 8809, 2024-04-07T17:51:29.298 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 76799 2024-04-07T17:51:29.298 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.298 INFO:tasks.workunit.client.0.smithi100.stdout: "store_state_latency": { 2024-04-07T17:51:29.298 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 8809, 2024-04-07T17:51:29.298 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 6003830940 2024-04-07T17:51:29.298 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.298 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.298 INFO:tasks.workunit.client.0.smithi100.stdout: "prioritycache": { 2024-04-07T17:51:29.298 INFO:tasks.workunit.client.0.smithi100.stdout: "cache_bytes": { 2024-04-07T17:51:29.298 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1020054731 2024-04-07T17:51:29.298 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.298 INFO:tasks.workunit.client.0.smithi100.stdout: "heap_bytes": { 2024-04-07T17:51:29.298 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 166682624 2024-04-07T17:51:29.298 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.298 INFO:tasks.workunit.client.0.smithi100.stdout: "mapped_bytes": { 2024-04-07T17:51:29.299 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 165732352 2024-04-07T17:51:29.299 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.299 INFO:tasks.workunit.client.0.smithi100.stdout: "target_bytes": { 2024-04-07T17:51:29.299 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2147483648 2024-04-07T17:51:29.299 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.299 INFO:tasks.workunit.client.0.smithi100.stdout: "unmapped_bytes": { 2024-04-07T17:51:29.299 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 950272 2024-04-07T17:51:29.299 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.299 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.299 INFO:tasks.workunit.client.0.smithi100.stdout: "prioritycache:full": { 2024-04-07T17:51:29.299 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:29.299 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 360710144 2024-04-07T17:51:29.299 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.299 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:29.299 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.299 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.300 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:29.300 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.300 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.300 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:29.300 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 267782252 2024-04-07T17:51:29.300 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.300 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:29.300 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 23512176 2024-04-07T17:51:29.300 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.300 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:29.300 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.300 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.300 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:29.300 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.301 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.301 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:29.301 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.301 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.301 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:29.301 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.301 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.301 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:29.301 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.301 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.301 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:29.301 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.301 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.301 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:29.301 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.302 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.302 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:29.302 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.302 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.302 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:29.302 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 69415716 2024-04-07T17:51:29.302 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.302 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.302 INFO:tasks.workunit.client.0.smithi100.stdout: "prioritycache:inc": { 2024-04-07T17:51:29.302 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:29.302 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 339738624 2024-04-07T17:51:29.302 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.302 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:29.302 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.302 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.303 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:29.303 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.303 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.303 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:29.303 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 267782252 2024-04-07T17:51:29.303 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.303 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:29.303 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1438180 2024-04-07T17:51:29.303 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.303 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:29.303 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.303 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.303 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:29.303 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.303 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.304 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:29.304 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.304 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.304 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:29.304 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.304 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.304 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:29.304 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.304 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.304 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:29.304 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.304 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.304 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:29.304 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.304 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.304 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:29.305 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.305 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.305 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:29.305 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 70518192 2024-04-07T17:51:29.305 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.305 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.305 INFO:tasks.workunit.client.0.smithi100.stdout: "prioritycache:kv": { 2024-04-07T17:51:29.305 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:29.305 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 314572800 2024-04-07T17:51:29.305 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.305 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:29.305 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 67108864 2024-04-07T17:51:29.305 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.305 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:29.305 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.305 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.306 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:29.306 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 178521501 2024-04-07T17:51:29.306 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.306 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:29.306 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.306 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.306 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:29.306 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.306 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.306 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:29.306 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.306 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.306 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:29.306 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.307 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.307 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:29.307 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.307 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.307 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:29.307 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.307 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.307 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:29.307 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.307 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.307 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:29.307 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.307 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.307 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:29.307 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.308 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.308 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:29.308 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 68942435 2024-04-07T17:51:29.308 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.308 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.308 INFO:tasks.workunit.client.0.smithi100.stdout: "rocksdb": { 2024-04-07T17:51:29.308 INFO:tasks.workunit.client.0.smithi100.stdout: "compact": { 2024-04-07T17:51:29.308 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.308 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.308 INFO:tasks.workunit.client.0.smithi100.stdout: "compact_queue_len": { 2024-04-07T17:51:29.308 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.308 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.308 INFO:tasks.workunit.client.0.smithi100.stdout: "compact_queue_merge": { 2024-04-07T17:51:29.308 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.308 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.308 INFO:tasks.workunit.client.0.smithi100.stdout: "compact_range": { 2024-04-07T17:51:29.309 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 105 2024-04-07T17:51:29.309 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.309 INFO:tasks.workunit.client.0.smithi100.stdout: "get_latency": { 2024-04-07T17:51:29.309 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 381369, 2024-04-07T17:51:29.309 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4245626135 2024-04-07T17:51:29.309 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.309 INFO:tasks.workunit.client.0.smithi100.stdout: "rocksdb_write_delay_time": { 2024-04-07T17:51:29.309 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:29.309 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.309 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.309 INFO:tasks.workunit.client.0.smithi100.stdout: "rocksdb_write_memtable_time": { 2024-04-07T17:51:29.309 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:29.309 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.309 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.309 INFO:tasks.workunit.client.0.smithi100.stdout: "rocksdb_write_pre_and_post_time": { 2024-04-07T17:51:29.309 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:29.310 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.310 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.310 INFO:tasks.workunit.client.0.smithi100.stdout: "rocksdb_write_wal_time": { 2024-04-07T17:51:29.310 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:29.310 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.310 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.310 INFO:tasks.workunit.client.0.smithi100.stdout: "submit_latency": { 2024-04-07T17:51:29.310 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:29.310 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.310 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.310 INFO:tasks.workunit.client.0.smithi100.stdout: "submit_sync_latency": { 2024-04-07T17:51:29.310 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 21131, 2024-04-07T17:51:29.310 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 10905136882 2024-04-07T17:51:29.311 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.311 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.311 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.311 INFO:tasks.workunit.client.0.smithi100.stdout: "mon.a52fc0d8acb515431879057649ff294cfe14c620": { 2024-04-07T17:51:29.311 INFO:tasks.workunit.client.0.smithi100.stdout: "mon": { 2024-04-07T17:51:29.311 INFO:tasks.workunit.client.0.smithi100.stdout: "election_call": { 2024-04-07T17:51:29.311 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2 2024-04-07T17:51:29.311 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.311 INFO:tasks.workunit.client.0.smithi100.stdout: "election_lose": { 2024-04-07T17:51:29.311 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2 2024-04-07T17:51:29.311 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.311 INFO:tasks.workunit.client.0.smithi100.stdout: "election_win": { 2024-04-07T17:51:29.311 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.311 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.311 INFO:tasks.workunit.client.0.smithi100.stdout: "num_elections": { 2024-04-07T17:51:29.311 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 7 2024-04-07T17:51:29.312 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.312 INFO:tasks.workunit.client.0.smithi100.stdout: "num_sessions": { 2024-04-07T17:51:29.312 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 8 2024-04-07T17:51:29.312 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.312 INFO:tasks.workunit.client.0.smithi100.stdout: "session_add": { 2024-04-07T17:51:29.312 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3 2024-04-07T17:51:29.312 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.312 INFO:tasks.workunit.client.0.smithi100.stdout: "session_rm": { 2024-04-07T17:51:29.312 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 253 2024-04-07T17:51:29.312 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.312 INFO:tasks.workunit.client.0.smithi100.stdout: "session_trim": { 2024-04-07T17:51:29.312 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.312 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.312 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.312 INFO:tasks.workunit.client.0.smithi100.stdout: "paxos": { 2024-04-07T17:51:29.313 INFO:tasks.workunit.client.0.smithi100.stdout: "accept_timeout": { 2024-04-07T17:51:29.313 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.313 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.313 INFO:tasks.workunit.client.0.smithi100.stdout: "begin": { 2024-04-07T17:51:29.313 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 8746 2024-04-07T17:51:29.313 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.313 INFO:tasks.workunit.client.0.smithi100.stdout: "begin_bytes": { 2024-04-07T17:51:29.313 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 8746, 2024-04-07T17:51:29.313 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 58929605 2024-04-07T17:51:29.313 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.313 INFO:tasks.workunit.client.0.smithi100.stdout: "begin_keys": { 2024-04-07T17:51:29.313 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:29.313 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.313 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.313 INFO:tasks.workunit.client.0.smithi100.stdout: "begin_latency": { 2024-04-07T17:51:29.313 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 8746, 2024-04-07T17:51:29.314 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2547954472 2024-04-07T17:51:29.314 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.314 INFO:tasks.workunit.client.0.smithi100.stdout: "collect": { 2024-04-07T17:51:29.314 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2 2024-04-07T17:51:29.314 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.314 INFO:tasks.workunit.client.0.smithi100.stdout: "collect_bytes": { 2024-04-07T17:51:29.314 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 2, 2024-04-07T17:51:29.314 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 146 2024-04-07T17:51:29.314 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.314 INFO:tasks.workunit.client.0.smithi100.stdout: "collect_keys": { 2024-04-07T17:51:29.314 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 2, 2024-04-07T17:51:29.314 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2 2024-04-07T17:51:29.314 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.314 INFO:tasks.workunit.client.0.smithi100.stdout: "collect_latency": { 2024-04-07T17:51:29.315 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 2, 2024-04-07T17:51:29.315 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3999909 2024-04-07T17:51:29.315 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.315 INFO:tasks.workunit.client.0.smithi100.stdout: "collect_timeout": { 2024-04-07T17:51:29.315 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.315 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.315 INFO:tasks.workunit.client.0.smithi100.stdout: "collect_uncommitted": { 2024-04-07T17:51:29.315 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.315 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.315 INFO:tasks.workunit.client.0.smithi100.stdout: "commit": { 2024-04-07T17:51:29.315 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 8746 2024-04-07T17:51:29.315 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.315 INFO:tasks.workunit.client.0.smithi100.stdout: "commit_bytes": { 2024-04-07T17:51:29.315 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:29.315 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.316 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.316 INFO:tasks.workunit.client.0.smithi100.stdout: "commit_keys": { 2024-04-07T17:51:29.316 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:29.316 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.316 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.316 INFO:tasks.workunit.client.0.smithi100.stdout: "commit_latency": { 2024-04-07T17:51:29.316 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:29.316 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.316 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.316 INFO:tasks.workunit.client.0.smithi100.stdout: "lease_ack_timeout": { 2024-04-07T17:51:29.316 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.316 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.316 INFO:tasks.workunit.client.0.smithi100.stdout: "lease_timeout": { 2024-04-07T17:51:29.316 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.316 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.317 INFO:tasks.workunit.client.0.smithi100.stdout: "new_pn": { 2024-04-07T17:51:29.317 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.317 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.317 INFO:tasks.workunit.client.0.smithi100.stdout: "new_pn_latency": { 2024-04-07T17:51:29.317 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:29.317 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.317 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.317 INFO:tasks.workunit.client.0.smithi100.stdout: "refresh": { 2024-04-07T17:51:29.317 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 8746 2024-04-07T17:51:29.317 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.317 INFO:tasks.workunit.client.0.smithi100.stdout: "refresh_latency": { 2024-04-07T17:51:29.317 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 8746, 2024-04-07T17:51:29.317 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 48991126475 2024-04-07T17:51:29.317 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.317 INFO:tasks.workunit.client.0.smithi100.stdout: "restart": { 2024-04-07T17:51:29.317 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 9 2024-04-07T17:51:29.318 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.318 INFO:tasks.workunit.client.0.smithi100.stdout: "share_state": { 2024-04-07T17:51:29.318 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.318 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.318 INFO:tasks.workunit.client.0.smithi100.stdout: "share_state_bytes": { 2024-04-07T17:51:29.318 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:29.318 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.318 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.318 INFO:tasks.workunit.client.0.smithi100.stdout: "share_state_keys": { 2024-04-07T17:51:29.318 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:29.318 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.318 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.318 INFO:tasks.workunit.client.0.smithi100.stdout: "start_leader": { 2024-04-07T17:51:29.318 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.319 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.319 INFO:tasks.workunit.client.0.smithi100.stdout: "start_peon": { 2024-04-07T17:51:29.319 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2 2024-04-07T17:51:29.319 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.319 INFO:tasks.workunit.client.0.smithi100.stdout: "store_state": { 2024-04-07T17:51:29.319 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 8746 2024-04-07T17:51:29.319 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.319 INFO:tasks.workunit.client.0.smithi100.stdout: "store_state_bytes": { 2024-04-07T17:51:29.319 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 8746, 2024-04-07T17:51:29.319 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 116192806 2024-04-07T17:51:29.319 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.319 INFO:tasks.workunit.client.0.smithi100.stdout: "store_state_keys": { 2024-04-07T17:51:29.319 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 8746, 2024-04-07T17:51:29.319 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 75755 2024-04-07T17:51:29.319 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.320 INFO:tasks.workunit.client.0.smithi100.stdout: "store_state_latency": { 2024-04-07T17:51:29.320 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 8746, 2024-04-07T17:51:29.320 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3183943365 2024-04-07T17:51:29.320 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.320 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.320 INFO:tasks.workunit.client.0.smithi100.stdout: "prioritycache": { 2024-04-07T17:51:29.320 INFO:tasks.workunit.client.0.smithi100.stdout: "cache_bytes": { 2024-04-07T17:51:29.320 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1020054731 2024-04-07T17:51:29.320 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.320 INFO:tasks.workunit.client.0.smithi100.stdout: "heap_bytes": { 2024-04-07T17:51:29.320 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 164585472 2024-04-07T17:51:29.320 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.320 INFO:tasks.workunit.client.0.smithi100.stdout: "mapped_bytes": { 2024-04-07T17:51:29.320 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 162111488 2024-04-07T17:51:29.320 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.320 INFO:tasks.workunit.client.0.smithi100.stdout: "target_bytes": { 2024-04-07T17:51:29.321 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2147483648 2024-04-07T17:51:29.321 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.321 INFO:tasks.workunit.client.0.smithi100.stdout: "unmapped_bytes": { 2024-04-07T17:51:29.321 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2473984 2024-04-07T17:51:29.321 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.321 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.321 INFO:tasks.workunit.client.0.smithi100.stdout: "prioritycache:full": { 2024-04-07T17:51:29.321 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:29.321 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 360710144 2024-04-07T17:51:29.321 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.321 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:29.321 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.321 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.321 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:29.321 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.321 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.322 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:29.322 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 267855893 2024-04-07T17:51:29.322 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.322 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:29.322 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 23328861 2024-04-07T17:51:29.322 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.322 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:29.322 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.322 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.322 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:29.322 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.322 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.322 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:29.322 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.322 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.323 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:29.323 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.323 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.323 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:29.323 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.323 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.323 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:29.323 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.323 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.323 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:29.323 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.323 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.323 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:29.323 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.323 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.324 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:29.324 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 69525390 2024-04-07T17:51:29.324 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.324 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.324 INFO:tasks.workunit.client.0.smithi100.stdout: "prioritycache:inc": { 2024-04-07T17:51:29.324 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:29.324 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 339738624 2024-04-07T17:51:29.324 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.324 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:29.324 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.324 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.324 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:29.324 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.324 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.324 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:29.325 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 267855893 2024-04-07T17:51:29.325 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.325 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:29.325 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1425119 2024-04-07T17:51:29.325 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.325 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:29.325 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.325 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.325 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:29.325 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.325 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.325 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:29.325 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.325 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.325 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:29.326 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.326 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.326 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:29.326 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.326 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.326 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:29.326 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.326 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.326 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:29.326 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.326 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.326 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:29.326 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.326 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.326 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:29.327 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 70457612 2024-04-07T17:51:29.327 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.327 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.327 INFO:tasks.workunit.client.0.smithi100.stdout: "prioritycache:kv": { 2024-04-07T17:51:29.327 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:29.327 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 314572800 2024-04-07T17:51:29.327 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.327 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:29.327 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 67108864 2024-04-07T17:51:29.327 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.327 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:29.327 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.327 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.327 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:29.327 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 178570595 2024-04-07T17:51:29.328 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.328 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:29.328 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.328 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.328 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:29.328 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.328 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.328 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:29.328 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.328 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.328 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:29.328 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.328 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.328 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:29.328 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.328 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.329 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:29.329 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.329 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.329 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:29.329 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.329 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.329 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:29.329 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.329 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.329 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:29.329 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.329 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.330 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:29.330 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 68893341 2024-04-07T17:51:29.330 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.330 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.330 INFO:tasks.workunit.client.0.smithi100.stdout: "rocksdb": { 2024-04-07T17:51:29.330 INFO:tasks.workunit.client.0.smithi100.stdout: "compact": { 2024-04-07T17:51:29.330 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.330 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.330 INFO:tasks.workunit.client.0.smithi100.stdout: "compact_queue_len": { 2024-04-07T17:51:29.330 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.330 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.330 INFO:tasks.workunit.client.0.smithi100.stdout: "compact_queue_merge": { 2024-04-07T17:51:29.330 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.331 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.331 INFO:tasks.workunit.client.0.smithi100.stdout: "compact_range": { 2024-04-07T17:51:29.331 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 101 2024-04-07T17:51:29.331 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.331 INFO:tasks.workunit.client.0.smithi100.stdout: "get_latency": { 2024-04-07T17:51:29.331 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 378495, 2024-04-07T17:51:29.331 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3968329012 2024-04-07T17:51:29.331 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.331 INFO:tasks.workunit.client.0.smithi100.stdout: "rocksdb_write_delay_time": { 2024-04-07T17:51:29.331 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:29.331 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.331 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.331 INFO:tasks.workunit.client.0.smithi100.stdout: "rocksdb_write_memtable_time": { 2024-04-07T17:51:29.331 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:29.331 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.332 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.332 INFO:tasks.workunit.client.0.smithi100.stdout: "rocksdb_write_pre_and_post_time": { 2024-04-07T17:51:29.332 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:29.332 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.332 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.332 INFO:tasks.workunit.client.0.smithi100.stdout: "rocksdb_write_wal_time": { 2024-04-07T17:51:29.332 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:29.332 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.332 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.332 INFO:tasks.workunit.client.0.smithi100.stdout: "submit_latency": { 2024-04-07T17:51:29.332 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:29.332 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.332 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.332 INFO:tasks.workunit.client.0.smithi100.stdout: "submit_sync_latency": { 2024-04-07T17:51:29.332 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 20972, 2024-04-07T17:51:29.333 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 8057256302 2024-04-07T17:51:29.333 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.333 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.333 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.333 INFO:tasks.workunit.client.0.smithi100.stdout: "mon.f47a4ff15a45950a63160265564a123fc5da5e5e": { 2024-04-07T17:51:29.333 INFO:tasks.workunit.client.0.smithi100.stdout: "mon": { 2024-04-07T17:51:29.333 INFO:tasks.workunit.client.0.smithi100.stdout: "election_call": { 2024-04-07T17:51:29.333 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5 2024-04-07T17:51:29.333 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.333 INFO:tasks.workunit.client.0.smithi100.stdout: "election_lose": { 2024-04-07T17:51:29.333 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.333 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.333 INFO:tasks.workunit.client.0.smithi100.stdout: "election_win": { 2024-04-07T17:51:29.333 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5 2024-04-07T17:51:29.333 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.334 INFO:tasks.workunit.client.0.smithi100.stdout: "num_elections": { 2024-04-07T17:51:29.334 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 17 2024-04-07T17:51:29.334 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.334 INFO:tasks.workunit.client.0.smithi100.stdout: "num_sessions": { 2024-04-07T17:51:29.334 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 11 2024-04-07T17:51:29.334 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.334 INFO:tasks.workunit.client.0.smithi100.stdout: "session_add": { 2024-04-07T17:51:29.334 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5 2024-04-07T17:51:29.334 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.334 INFO:tasks.workunit.client.0.smithi100.stdout: "session_rm": { 2024-04-07T17:51:29.334 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 301 2024-04-07T17:51:29.334 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.334 INFO:tasks.workunit.client.0.smithi100.stdout: "session_trim": { 2024-04-07T17:51:29.334 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.335 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.335 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.335 INFO:tasks.workunit.client.0.smithi100.stdout: "paxos": { 2024-04-07T17:51:29.335 INFO:tasks.workunit.client.0.smithi100.stdout: "accept_timeout": { 2024-04-07T17:51:29.335 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.335 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.335 INFO:tasks.workunit.client.0.smithi100.stdout: "begin": { 2024-04-07T17:51:29.335 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 8903 2024-04-07T17:51:29.335 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.335 INFO:tasks.workunit.client.0.smithi100.stdout: "begin_bytes": { 2024-04-07T17:51:29.335 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 8903, 2024-04-07T17:51:29.335 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 60802484 2024-04-07T17:51:29.335 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.335 INFO:tasks.workunit.client.0.smithi100.stdout: "begin_keys": { 2024-04-07T17:51:29.335 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 8903, 2024-04-07T17:51:29.335 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 26709 2024-04-07T17:51:29.336 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.336 INFO:tasks.workunit.client.0.smithi100.stdout: "begin_latency": { 2024-04-07T17:51:29.336 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 8903, 2024-04-07T17:51:29.336 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2899918143 2024-04-07T17:51:29.336 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.336 INFO:tasks.workunit.client.0.smithi100.stdout: "collect": { 2024-04-07T17:51:29.336 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.336 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.336 INFO:tasks.workunit.client.0.smithi100.stdout: "collect_bytes": { 2024-04-07T17:51:29.336 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:29.336 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.336 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.336 INFO:tasks.workunit.client.0.smithi100.stdout: "collect_keys": { 2024-04-07T17:51:29.336 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:29.336 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.336 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.337 INFO:tasks.workunit.client.0.smithi100.stdout: "collect_latency": { 2024-04-07T17:51:29.337 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:29.337 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.337 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.337 INFO:tasks.workunit.client.0.smithi100.stdout: "collect_timeout": { 2024-04-07T17:51:29.337 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.337 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.337 INFO:tasks.workunit.client.0.smithi100.stdout: "collect_uncommitted": { 2024-04-07T17:51:29.337 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2 2024-04-07T17:51:29.337 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.337 INFO:tasks.workunit.client.0.smithi100.stdout: "commit": { 2024-04-07T17:51:29.337 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 8901 2024-04-07T17:51:29.337 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.337 INFO:tasks.workunit.client.0.smithi100.stdout: "commit_bytes": { 2024-04-07T17:51:29.337 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 8901, 2024-04-07T17:51:29.338 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 60134266 2024-04-07T17:51:29.338 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.338 INFO:tasks.workunit.client.0.smithi100.stdout: "commit_keys": { 2024-04-07T17:51:29.338 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 8901, 2024-04-07T17:51:29.338 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 68692 2024-04-07T17:51:29.338 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.338 INFO:tasks.workunit.client.0.smithi100.stdout: "commit_latency": { 2024-04-07T17:51:29.338 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 8901, 2024-04-07T17:51:29.338 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3951689309 2024-04-07T17:51:29.338 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.338 INFO:tasks.workunit.client.0.smithi100.stdout: "lease_ack_timeout": { 2024-04-07T17:51:29.338 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.338 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.338 INFO:tasks.workunit.client.0.smithi100.stdout: "lease_timeout": { 2024-04-07T17:51:29.338 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.339 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.339 INFO:tasks.workunit.client.0.smithi100.stdout: "new_pn": { 2024-04-07T17:51:29.339 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5 2024-04-07T17:51:29.339 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.339 INFO:tasks.workunit.client.0.smithi100.stdout: "new_pn_latency": { 2024-04-07T17:51:29.339 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 5, 2024-04-07T17:51:29.339 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.339 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.339 INFO:tasks.workunit.client.0.smithi100.stdout: "refresh": { 2024-04-07T17:51:29.339 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 8904 2024-04-07T17:51:29.339 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.339 INFO:tasks.workunit.client.0.smithi100.stdout: "refresh_latency": { 2024-04-07T17:51:29.339 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 8904, 2024-04-07T17:51:29.339 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 56282413972 2024-04-07T17:51:29.339 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.340 INFO:tasks.workunit.client.0.smithi100.stdout: "restart": { 2024-04-07T17:51:29.340 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 19 2024-04-07T17:51:29.340 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.340 INFO:tasks.workunit.client.0.smithi100.stdout: "share_state": { 2024-04-07T17:51:29.340 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2 2024-04-07T17:51:29.340 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.340 INFO:tasks.workunit.client.0.smithi100.stdout: "share_state_bytes": { 2024-04-07T17:51:29.340 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 2, 2024-04-07T17:51:29.340 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 129358 2024-04-07T17:51:29.340 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.340 INFO:tasks.workunit.client.0.smithi100.stdout: "share_state_keys": { 2024-04-07T17:51:29.340 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 2, 2024-04-07T17:51:29.340 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 10 2024-04-07T17:51:29.340 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.340 INFO:tasks.workunit.client.0.smithi100.stdout: "start_leader": { 2024-04-07T17:51:29.341 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5 2024-04-07T17:51:29.341 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.341 INFO:tasks.workunit.client.0.smithi100.stdout: "start_peon": { 2024-04-07T17:51:29.341 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.341 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.341 INFO:tasks.workunit.client.0.smithi100.stdout: "store_state": { 2024-04-07T17:51:29.341 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.341 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.341 INFO:tasks.workunit.client.0.smithi100.stdout: "store_state_bytes": { 2024-04-07T17:51:29.341 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:29.341 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.341 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.341 INFO:tasks.workunit.client.0.smithi100.stdout: "store_state_keys": { 2024-04-07T17:51:29.341 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:29.341 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.341 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.342 INFO:tasks.workunit.client.0.smithi100.stdout: "store_state_latency": { 2024-04-07T17:51:29.342 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:29.342 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.342 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.342 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.342 INFO:tasks.workunit.client.0.smithi100.stdout: "prioritycache": { 2024-04-07T17:51:29.342 INFO:tasks.workunit.client.0.smithi100.stdout: "cache_bytes": { 2024-04-07T17:51:29.342 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1020054731 2024-04-07T17:51:29.342 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.342 INFO:tasks.workunit.client.0.smithi100.stdout: "heap_bytes": { 2024-04-07T17:51:29.342 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 259162112 2024-04-07T17:51:29.342 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.342 INFO:tasks.workunit.client.0.smithi100.stdout: "mapped_bytes": { 2024-04-07T17:51:29.342 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 258187264 2024-04-07T17:51:29.343 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.343 INFO:tasks.workunit.client.0.smithi100.stdout: "target_bytes": { 2024-04-07T17:51:29.343 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2147483648 2024-04-07T17:51:29.343 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.343 INFO:tasks.workunit.client.0.smithi100.stdout: "unmapped_bytes": { 2024-04-07T17:51:29.343 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 974848 2024-04-07T17:51:29.343 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.343 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.343 INFO:tasks.workunit.client.0.smithi100.stdout: "prioritycache:full": { 2024-04-07T17:51:29.343 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:29.343 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 360710144 2024-04-07T17:51:29.343 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.343 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:29.343 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.343 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.343 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:29.344 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.344 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.344 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:29.344 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 267670953 2024-04-07T17:51:29.344 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.344 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:29.344 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 23788534 2024-04-07T17:51:29.344 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.344 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:29.344 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.344 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.344 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:29.344 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.344 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.344 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:29.344 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.345 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.345 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:29.345 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.345 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.345 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:29.345 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.345 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.345 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:29.345 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.345 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.345 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:29.345 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.345 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.345 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:29.345 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.345 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.346 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:29.346 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 69250657 2024-04-07T17:51:29.346 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.346 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.346 INFO:tasks.workunit.client.0.smithi100.stdout: "prioritycache:inc": { 2024-04-07T17:51:29.346 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:29.346 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 339738624 2024-04-07T17:51:29.346 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.346 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:29.346 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.346 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.346 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:29.346 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.346 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.346 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:29.347 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 267670953 2024-04-07T17:51:29.347 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.347 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:29.347 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1458619 2024-04-07T17:51:29.347 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.347 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:29.347 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.347 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.347 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:29.347 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.347 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.347 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:29.347 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.347 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.347 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:29.348 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.348 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.348 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:29.348 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.348 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.348 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:29.348 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.348 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.348 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:29.348 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.348 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.348 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:29.348 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.349 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.349 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:29.349 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 70609052 2024-04-07T17:51:29.349 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.349 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.349 INFO:tasks.workunit.client.0.smithi100.stdout: "prioritycache:kv": { 2024-04-07T17:51:29.349 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:29.349 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 314572800 2024-04-07T17:51:29.349 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.349 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:29.349 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 67108864 2024-04-07T17:51:29.349 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.349 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:29.349 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.349 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.350 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:29.350 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 178447302 2024-04-07T17:51:29.350 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.350 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:29.350 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.350 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.350 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:29.350 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.350 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.350 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:29.350 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.350 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.350 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:29.350 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.350 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.351 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:29.351 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.351 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.351 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:29.351 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.351 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.351 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:29.351 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.351 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.351 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:29.351 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.351 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.351 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:29.351 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.351 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.352 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:29.352 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 69016634 2024-04-07T17:51:29.352 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.352 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.352 INFO:tasks.workunit.client.0.smithi100.stdout: "rocksdb": { 2024-04-07T17:51:29.352 INFO:tasks.workunit.client.0.smithi100.stdout: "compact": { 2024-04-07T17:51:29.352 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.352 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.352 INFO:tasks.workunit.client.0.smithi100.stdout: "compact_queue_len": { 2024-04-07T17:51:29.352 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.352 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.352 INFO:tasks.workunit.client.0.smithi100.stdout: "compact_queue_merge": { 2024-04-07T17:51:29.352 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.352 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.352 INFO:tasks.workunit.client.0.smithi100.stdout: "compact_range": { 2024-04-07T17:51:29.353 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 105 2024-04-07T17:51:29.353 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.353 INFO:tasks.workunit.client.0.smithi100.stdout: "get_latency": { 2024-04-07T17:51:29.353 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 399200, 2024-04-07T17:51:29.353 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4727464457 2024-04-07T17:51:29.353 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.353 INFO:tasks.workunit.client.0.smithi100.stdout: "rocksdb_write_delay_time": { 2024-04-07T17:51:29.353 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:29.353 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.353 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.353 INFO:tasks.workunit.client.0.smithi100.stdout: "rocksdb_write_memtable_time": { 2024-04-07T17:51:29.353 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:29.353 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.353 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.353 INFO:tasks.workunit.client.0.smithi100.stdout: "rocksdb_write_pre_and_post_time": { 2024-04-07T17:51:29.354 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:29.354 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.354 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.354 INFO:tasks.workunit.client.0.smithi100.stdout: "rocksdb_write_wal_time": { 2024-04-07T17:51:29.354 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:29.354 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.354 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.354 INFO:tasks.workunit.client.0.smithi100.stdout: "submit_latency": { 2024-04-07T17:51:29.354 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:29.354 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.354 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.354 INFO:tasks.workunit.client.0.smithi100.stdout: "submit_sync_latency": { 2024-04-07T17:51:29.354 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 21360, 2024-04-07T17:51:29.354 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 8948085534 2024-04-07T17:51:29.354 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.355 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.355 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.355 INFO:tasks.workunit.client.0.smithi100.stdout: "osd.0": { 2024-04-07T17:51:29.355 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs": { 2024-04-07T17:51:29.355 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_db_max_lat": { 2024-04-07T17:51:29.355 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 21463 2024-04-07T17:51:29.355 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.355 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_slow_fallback": { 2024-04-07T17:51:29.355 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.355 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.355 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_slow_max_lat": { 2024-04-07T17:51:29.355 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.355 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.355 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_slow_size_fallback": { 2024-04-07T17:51:29.355 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.356 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.356 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit_db": { 2024-04-07T17:51:29.356 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 65536 2024-04-07T17:51:29.356 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.356 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit_slow": { 2024-04-07T17:51:29.356 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.356 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.356 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit_wal": { 2024-04-07T17:51:29.356 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.356 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.356 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_wal_max_lat": { 2024-04-07T17:51:29.356 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.356 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.356 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_written_slow": { 2024-04-07T17:51:29.356 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.357 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.357 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_written_sst": { 2024-04-07T17:51:29.357 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 14692352 2024-04-07T17:51:29.357 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.357 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_written_wal": { 2024-04-07T17:51:29.357 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 742813696 2024-04-07T17:51:29.357 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.357 INFO:tasks.workunit.client.0.smithi100.stdout: "compact_lat": { 2024-04-07T17:51:29.357 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 19, 2024-04-07T17:51:29.357 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 7885793 2024-04-07T17:51:29.357 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.357 INFO:tasks.workunit.client.0.smithi100.stdout: "compact_lock_lat": { 2024-04-07T17:51:29.357 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 19, 2024-04-07T17:51:29.357 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2646042 2024-04-07T17:51:29.357 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.358 INFO:tasks.workunit.client.0.smithi100.stdout: "db_alloc_lat": { 2024-04-07T17:51:29.358 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 211, 2024-04-07T17:51:29.358 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1477946 2024-04-07T17:51:29.358 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.358 INFO:tasks.workunit.client.0.smithi100.stdout: "db_total_bytes": { 2024-04-07T17:51:29.358 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 95995027456 2024-04-07T17:51:29.358 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.358 INFO:tasks.workunit.client.0.smithi100.stdout: "db_used_bytes": { 2024-04-07T17:51:29.358 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 735182848 2024-04-07T17:51:29.358 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.358 INFO:tasks.workunit.client.0.smithi100.stdout: "flush_lat": { 2024-04-07T17:51:29.358 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 79501, 2024-04-07T17:51:29.358 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 14958487526 2024-04-07T17:51:29.358 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.359 INFO:tasks.workunit.client.0.smithi100.stdout: "fsync_lat": { 2024-04-07T17:51:29.359 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 79509, 2024-04-07T17:51:29.359 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 25869094804 2024-04-07T17:51:29.359 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.359 INFO:tasks.workunit.client.0.smithi100.stdout: "log_bytes": { 2024-04-07T17:51:29.359 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 9764864 2024-04-07T17:51:29.359 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.359 INFO:tasks.workunit.client.0.smithi100.stdout: "logged_bytes": { 2024-04-07T17:51:29.359 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 325660672 2024-04-07T17:51:29.359 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.359 INFO:tasks.workunit.client.0.smithi100.stdout: "max_bytes_db": { 2024-04-07T17:51:29.359 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 747962368 2024-04-07T17:51:29.359 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.359 INFO:tasks.workunit.client.0.smithi100.stdout: "max_bytes_slow": { 2024-04-07T17:51:29.359 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.359 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.360 INFO:tasks.workunit.client.0.smithi100.stdout: "max_bytes_wal": { 2024-04-07T17:51:29.360 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.360 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.360 INFO:tasks.workunit.client.0.smithi100.stdout: "num_files": { 2024-04-07T17:51:29.360 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 55 2024-04-07T17:51:29.360 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.360 INFO:tasks.workunit.client.0.smithi100.stdout: "read_bytes": { 2024-04-07T17:51:29.360 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 13112107291 2024-04-07T17:51:29.360 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.360 INFO:tasks.workunit.client.0.smithi100.stdout: "read_count": { 2024-04-07T17:51:29.360 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 86948 2024-04-07T17:51:29.360 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.360 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes": { 2024-04-07T17:51:29.360 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 11361755136 2024-04-07T17:51:29.360 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.361 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes_db": { 2024-04-07T17:51:29.361 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 11361759232 2024-04-07T17:51:29.361 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.361 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes_slow": { 2024-04-07T17:51:29.361 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.361 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.361 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes_wal": { 2024-04-07T17:51:29.361 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.361 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.361 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_count": { 2024-04-07T17:51:29.361 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 88640 2024-04-07T17:51:29.361 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.361 INFO:tasks.workunit.client.0.smithi100.stdout: "read_lat": { 2024-04-07T17:51:29.361 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 86948, 2024-04-07T17:51:29.361 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5555540195 2024-04-07T17:51:29.361 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.362 INFO:tasks.workunit.client.0.smithi100.stdout: "read_prefetch_bytes": { 2024-04-07T17:51:29.362 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 12971063490 2024-04-07T17:51:29.362 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.362 INFO:tasks.workunit.client.0.smithi100.stdout: "read_prefetch_count": { 2024-04-07T17:51:29.362 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 82627 2024-04-07T17:51:29.362 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.362 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_buffer_bytes": { 2024-04-07T17:51:29.362 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5154118 2024-04-07T17:51:29.362 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.362 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_buffer_count": { 2024-04-07T17:51:29.362 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 723 2024-04-07T17:51:29.362 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.362 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_bytes": { 2024-04-07T17:51:29.362 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 14619250 2024-04-07T17:51:29.362 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.363 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_count": { 2024-04-07T17:51:29.363 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1029 2024-04-07T17:51:29.363 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.363 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes": { 2024-04-07T17:51:29.363 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 9465132 2024-04-07T17:51:29.363 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.363 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes_db": { 2024-04-07T17:51:29.363 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 9465132 2024-04-07T17:51:29.363 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.363 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes_slow": { 2024-04-07T17:51:29.363 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.363 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.363 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes_wal": { 2024-04-07T17:51:29.363 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.363 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.363 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_count": { 2024-04-07T17:51:29.364 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 321 2024-04-07T17:51:29.364 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.364 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_lat": { 2024-04-07T17:51:29.364 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1029, 2024-04-07T17:51:29.364 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 7138854 2024-04-07T17:51:29.364 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.364 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_alloc_lat": { 2024-04-07T17:51:29.364 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:29.364 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.364 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.364 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_total_bytes": { 2024-04-07T17:51:29.364 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.364 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.364 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_used_bytes": { 2024-04-07T17:51:29.364 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.364 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.365 INFO:tasks.workunit.client.0.smithi100.stdout: "truncate_lat": { 2024-04-07T17:51:29.365 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1, 2024-04-07T17:51:29.365 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 593 2024-04-07T17:51:29.365 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.365 INFO:tasks.workunit.client.0.smithi100.stdout: "unlink_lat": { 2024-04-07T17:51:29.365 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 11, 2024-04-07T17:51:29.365 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 14997 2024-04-07T17:51:29.365 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.365 INFO:tasks.workunit.client.0.smithi100.stdout: "wal_alloc_lat": { 2024-04-07T17:51:29.365 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:29.365 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.365 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.365 INFO:tasks.workunit.client.0.smithi100.stdout: "wal_total_bytes": { 2024-04-07T17:51:29.365 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.365 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.366 INFO:tasks.workunit.client.0.smithi100.stdout: "wal_used_bytes": { 2024-04-07T17:51:29.366 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.366 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.366 INFO:tasks.workunit.client.0.smithi100.stdout: "write_bytes": { 2024-04-07T17:51:29.366 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1086054400 2024-04-07T17:51:29.366 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.366 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.366 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore": { 2024-04-07T17:51:29.366 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit": { 2024-04-07T17:51:29.366 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4096 2024-04-07T17:51:29.366 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.366 INFO:tasks.workunit.client.0.smithi100.stdout: "allocated": { 2024-04-07T17:51:29.367 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 10412032 2024-04-07T17:51:29.367 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.367 INFO:tasks.workunit.client.0.smithi100.stdout: "allocator_lat": { 2024-04-07T17:51:29.367 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 23588, 2024-04-07T17:51:29.367 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 122707682 2024-04-07T17:51:29.367 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.367 INFO:tasks.workunit.client.0.smithi100.stdout: "clist_lat": { 2024-04-07T17:51:29.367 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 3594, 2024-04-07T17:51:29.367 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 12445185285 2024-04-07T17:51:29.367 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.367 INFO:tasks.workunit.client.0.smithi100.stdout: "compress_lat": { 2024-04-07T17:51:29.367 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:29.367 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.367 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.367 INFO:tasks.workunit.client.0.smithi100.stdout: "compressed": { 2024-04-07T17:51:29.368 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.368 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.368 INFO:tasks.workunit.client.0.smithi100.stdout: "compressed_allocated": { 2024-04-07T17:51:29.368 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.368 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.368 INFO:tasks.workunit.client.0.smithi100.stdout: "compressed_original": { 2024-04-07T17:51:29.368 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.368 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.368 INFO:tasks.workunit.client.0.smithi100.stdout: "csum_lat": { 2024-04-07T17:51:29.368 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 29739, 2024-04-07T17:51:29.368 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 345936986 2024-04-07T17:51:29.368 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.368 INFO:tasks.workunit.client.0.smithi100.stdout: "decompress_lat": { 2024-04-07T17:51:29.368 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:29.368 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.369 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.369 INFO:tasks.workunit.client.0.smithi100.stdout: "fragmentation_micros": { 2024-04-07T17:51:29.369 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.369 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.369 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_commit_lat": { 2024-04-07T17:51:29.369 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 79427, 2024-04-07T17:51:29.369 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 33861604319 2024-04-07T17:51:29.369 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.369 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_final_lat": { 2024-04-07T17:51:29.369 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 78479, 2024-04-07T17:51:29.369 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3723931118 2024-04-07T17:51:29.369 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.369 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_flush_lat": { 2024-04-07T17:51:29.369 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 79427, 2024-04-07T17:51:29.369 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 104765973 2024-04-07T17:51:29.369 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.370 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_sync_lat": { 2024-04-07T17:51:29.370 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 79427, 2024-04-07T17:51:29.370 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 33966370292 2024-04-07T17:51:29.370 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.370 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_get_keys_lat": { 2024-04-07T17:51:29.370 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:29.370 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.370 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.370 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_get_values_lat": { 2024-04-07T17:51:29.370 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 35913, 2024-04-07T17:51:29.370 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1906667723 2024-04-07T17:51:29.370 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.370 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_lower_bound_lat": { 2024-04-07T17:51:29.370 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 941, 2024-04-07T17:51:29.371 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5755773 2024-04-07T17:51:29.371 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.371 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_next_lat": { 2024-04-07T17:51:29.371 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 24954, 2024-04-07T17:51:29.371 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 21515897 2024-04-07T17:51:29.371 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.371 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_seek_to_first_lat": { 2024-04-07T17:51:29.371 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 52, 2024-04-07T17:51:29.371 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 102213 2024-04-07T17:51:29.371 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.371 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_upper_bound_lat": { 2024-04-07T17:51:29.371 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 16673, 2024-04-07T17:51:29.371 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5746524694 2024-04-07T17:51:29.371 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.371 INFO:tasks.workunit.client.0.smithi100.stdout: "onode_hits": { 2024-04-07T17:51:29.372 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 260546 2024-04-07T17:51:29.372 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.372 INFO:tasks.workunit.client.0.smithi100.stdout: "onode_misses": { 2024-04-07T17:51:29.372 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 24792 2024-04-07T17:51:29.372 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.372 INFO:tasks.workunit.client.0.smithi100.stdout: "read_lat": { 2024-04-07T17:51:29.372 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 26984, 2024-04-07T17:51:29.372 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3801162649 2024-04-07T17:51:29.372 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.372 INFO:tasks.workunit.client.0.smithi100.stdout: "read_onode_meta_lat": { 2024-04-07T17:51:29.372 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 62413, 2024-04-07T17:51:29.372 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 46979091 2024-04-07T17:51:29.372 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.372 INFO:tasks.workunit.client.0.smithi100.stdout: "read_wait_aio_lat": { 2024-04-07T17:51:29.372 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 35429, 2024-04-07T17:51:29.372 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2438344202 2024-04-07T17:51:29.373 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.373 INFO:tasks.workunit.client.0.smithi100.stdout: "reads_with_retries": { 2024-04-07T17:51:29.373 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.373 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.373 INFO:tasks.workunit.client.0.smithi100.stdout: "remove_lat": { 2024-04-07T17:51:29.373 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 15389, 2024-04-07T17:51:29.373 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 819542013 2024-04-07T17:51:29.373 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.373 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_aio_wait_count": { 2024-04-07T17:51:29.373 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.373 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.373 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_committed_kv_count": { 2024-04-07T17:51:29.373 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.373 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.373 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_read_onode_meta_count": { 2024-04-07T17:51:29.374 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.374 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.374 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_read_wait_aio_count": { 2024-04-07T17:51:29.374 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.374 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.374 INFO:tasks.workunit.client.0.smithi100.stdout: "state_aio_wait_lat": { 2024-04-07T17:51:29.374 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 93417, 2024-04-07T17:51:29.374 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 12329354117 2024-04-07T17:51:29.374 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.374 INFO:tasks.workunit.client.0.smithi100.stdout: "state_deferred_aio_wait_lat": { 2024-04-07T17:51:29.374 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 9754, 2024-04-07T17:51:29.374 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1209751248 2024-04-07T17:51:29.374 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.375 INFO:tasks.workunit.client.0.smithi100.stdout: "state_deferred_cleanup_lat": { 2024-04-07T17:51:29.375 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 9754, 2024-04-07T17:51:29.375 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4259450438283 2024-04-07T17:51:29.375 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.375 INFO:tasks.workunit.client.0.smithi100.stdout: "state_deferred_queued_lat": { 2024-04-07T17:51:29.375 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 9754, 2024-04-07T17:51:29.375 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 6800803572839 2024-04-07T17:51:29.375 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.375 INFO:tasks.workunit.client.0.smithi100.stdout: "state_done_lat": { 2024-04-07T17:51:29.375 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 93417, 2024-04-07T17:51:29.375 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2914483927714 2024-04-07T17:51:29.375 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.375 INFO:tasks.workunit.client.0.smithi100.stdout: "state_finishing_lat": { 2024-04-07T17:51:29.375 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 93417, 2024-04-07T17:51:29.375 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 20549739 2024-04-07T17:51:29.376 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.376 INFO:tasks.workunit.client.0.smithi100.stdout: "state_io_done_lat": { 2024-04-07T17:51:29.376 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 93417, 2024-04-07T17:51:29.376 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 589398054 2024-04-07T17:51:29.376 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.376 INFO:tasks.workunit.client.0.smithi100.stdout: "state_kv_commiting_lat": { 2024-04-07T17:51:29.376 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 93417, 2024-04-07T17:51:29.376 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 60191086996 2024-04-07T17:51:29.376 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.376 INFO:tasks.workunit.client.0.smithi100.stdout: "state_kv_done_lat": { 2024-04-07T17:51:29.376 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 93417, 2024-04-07T17:51:29.376 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 69549694 2024-04-07T17:51:29.376 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.376 INFO:tasks.workunit.client.0.smithi100.stdout: "state_kv_queued_lat": { 2024-04-07T17:51:29.376 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 93417, 2024-04-07T17:51:29.377 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 31858885295 2024-04-07T17:51:29.377 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.377 INFO:tasks.workunit.client.0.smithi100.stdout: "state_prepare_lat": { 2024-04-07T17:51:29.377 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 93417, 2024-04-07T17:51:29.377 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 26655280613 2024-04-07T17:51:29.377 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.377 INFO:tasks.workunit.client.0.smithi100.stdout: "stored": { 2024-04-07T17:51:29.377 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 6290778 2024-04-07T17:51:29.377 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.377 INFO:tasks.workunit.client.0.smithi100.stdout: "truncate_lat": { 2024-04-07T17:51:29.377 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 2482, 2024-04-07T17:51:29.377 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 25806717 2024-04-07T17:51:29.377 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.377 INFO:tasks.workunit.client.0.smithi100.stdout: "txc_commit_lat": { 2024-04-07T17:51:29.377 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 93417, 2024-04-07T17:51:29.377 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 131654529374 2024-04-07T17:51:29.377 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.378 INFO:tasks.workunit.client.0.smithi100.stdout: "txc_submit_lat": { 2024-04-07T17:51:29.378 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 93417, 2024-04-07T17:51:29.378 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 29199087138 2024-04-07T17:51:29.378 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.378 INFO:tasks.workunit.client.0.smithi100.stdout: "txc_throttle_lat": { 2024-04-07T17:51:29.378 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 93417, 2024-04-07T17:51:29.378 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1435768578 2024-04-07T17:51:29.378 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.378 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.378 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache": { 2024-04-07T17:51:29.378 INFO:tasks.workunit.client.0.smithi100.stdout: "cache_bytes": { 2024-04-07T17:51:29.378 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1444614092 2024-04-07T17:51:29.378 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.378 INFO:tasks.workunit.client.0.smithi100.stdout: "heap_bytes": { 2024-04-07T17:51:29.378 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 508690432 2024-04-07T17:51:29.379 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.379 INFO:tasks.workunit.client.0.smithi100.stdout: "mapped_bytes": { 2024-04-07T17:51:29.379 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 469770240 2024-04-07T17:51:29.379 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.379 INFO:tasks.workunit.client.0.smithi100.stdout: "target_bytes": { 2024-04-07T17:51:29.379 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2646965248 2024-04-07T17:51:29.379 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.379 INFO:tasks.workunit.client.0.smithi100.stdout: "unmapped_bytes": { 2024-04-07T17:51:29.379 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 38920192 2024-04-07T17:51:29.379 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.379 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.379 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:data": { 2024-04-07T17:51:29.379 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:29.379 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 134217728 2024-04-07T17:51:29.379 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.379 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:29.380 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.380 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.380 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:29.380 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.380 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.380 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:29.380 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 58791520 2024-04-07T17:51:29.380 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.380 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:29.380 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.380 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.380 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:29.380 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.380 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.380 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:29.380 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 65536 2024-04-07T17:51:29.380 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.381 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:29.381 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.381 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.381 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:29.381 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4096 2024-04-07T17:51:29.381 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.381 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:29.381 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 438272 2024-04-07T17:51:29.381 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.381 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:29.381 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.381 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.381 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:29.381 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.381 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.381 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:29.382 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.382 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.382 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:29.382 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 74918304 2024-04-07T17:51:29.382 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.382 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.382 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:kv": { 2024-04-07T17:51:29.382 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:29.382 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 578813952 2024-04-07T17:51:29.382 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.382 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:29.382 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 67108864 2024-04-07T17:51:29.382 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.382 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:29.382 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.383 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.383 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:29.383 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 440936400 2024-04-07T17:51:29.383 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.383 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:29.383 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.383 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.383 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:29.383 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1488 2024-04-07T17:51:29.383 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.383 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:29.383 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 288 2024-04-07T17:51:29.383 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.384 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:29.384 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.384 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.384 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:29.384 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 121696 2024-04-07T17:51:29.384 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.384 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:29.384 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.384 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.384 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:29.384 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.384 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.384 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:29.384 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.384 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.384 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:29.385 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.385 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.385 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:29.385 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 70645216 2024-04-07T17:51:29.385 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.385 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.385 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:kv_onode": { 2024-04-07T17:51:29.385 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:29.385 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 192937984 2024-04-07T17:51:29.385 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.385 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:29.385 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 67108864 2024-04-07T17:51:29.385 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.385 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:29.385 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 14485360 2024-04-07T17:51:29.386 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.386 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:29.386 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 39194346 2024-04-07T17:51:29.386 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.386 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:29.386 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.386 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.386 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:29.386 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.386 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.386 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:29.386 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.386 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.387 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:29.387 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.387 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.387 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:29.387 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.387 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.387 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:29.387 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.387 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.387 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:29.387 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.387 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.387 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:29.387 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.387 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.388 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:29.388 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.388 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.388 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:29.388 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 72149414 2024-04-07T17:51:29.388 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.388 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.388 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:meta": { 2024-04-07T17:51:29.388 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:29.388 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 511705088 2024-04-07T17:51:29.388 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.388 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:29.388 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.388 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.388 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:29.388 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.389 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.389 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:29.389 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 440936400 2024-04-07T17:51:29.389 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.389 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:29.389 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.389 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.389 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:29.389 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 985 2024-04-07T17:51:29.389 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.389 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:29.389 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 15761 2024-04-07T17:51:29.389 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.389 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:29.389 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.389 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.390 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:29.390 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 128065 2024-04-07T17:51:29.390 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.390 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:29.390 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1095452 2024-04-07T17:51:29.390 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.390 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:29.390 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.390 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.390 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:29.390 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.390 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.390 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:29.390 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.391 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.391 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:29.391 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 69528425 2024-04-07T17:51:29.391 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.391 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.391 INFO:tasks.workunit.client.0.smithi100.stdout: "objecter": { 2024-04-07T17:51:29.391 INFO:tasks.workunit.client.0.smithi100.stdout: "op_active": { 2024-04-07T17:51:29.391 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.391 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.391 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r": { 2024-04-07T17:51:29.391 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 56 2024-04-07T17:51:29.391 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.391 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rmw": { 2024-04-07T17:51:29.391 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.391 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.391 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w": { 2024-04-07T17:51:29.392 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.392 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.392 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.392 INFO:tasks.workunit.client.0.smithi100.stdout: "osd": { 2024-04-07T17:51:29.392 INFO:tasks.workunit.client.0.smithi100.stdout: "numpg": { 2024-04-07T17:51:29.392 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 55 2024-04-07T17:51:29.392 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.392 INFO:tasks.workunit.client.0.smithi100.stdout: "numpg_removing": { 2024-04-07T17:51:29.392 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.392 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.392 INFO:tasks.workunit.client.0.smithi100.stdout: "op": { 2024-04-07T17:51:29.392 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 53268 2024-04-07T17:51:29.392 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.392 INFO:tasks.workunit.client.0.smithi100.stdout: "op_before_queue_op_lat": { 2024-04-07T17:51:29.392 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 138179, 2024-04-07T17:51:29.393 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 14240690971 2024-04-07T17:51:29.393 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.393 INFO:tasks.workunit.client.0.smithi100.stdout: "op_delayed_degraded": { 2024-04-07T17:51:29.393 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4 2024-04-07T17:51:29.393 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.393 INFO:tasks.workunit.client.0.smithi100.stdout: "op_delayed_unreadable": { 2024-04-07T17:51:29.393 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.393 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.393 INFO:tasks.workunit.client.0.smithi100.stdout: "op_in_bytes": { 2024-04-07T17:51:29.393 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2665625896 2024-04-07T17:51:29.393 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.393 INFO:tasks.workunit.client.0.smithi100.stdout: "op_latency": { 2024-04-07T17:51:29.393 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 53268, 2024-04-07T17:51:29.393 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 365465901087 2024-04-07T17:51:29.393 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.393 INFO:tasks.workunit.client.0.smithi100.stdout: "op_out_bytes": { 2024-04-07T17:51:29.394 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3752394330 2024-04-07T17:51:29.394 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.394 INFO:tasks.workunit.client.0.smithi100.stdout: "op_prepare_latency": { 2024-04-07T17:51:29.394 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 54143, 2024-04-07T17:51:29.394 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 47148708510 2024-04-07T17:51:29.394 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.394 INFO:tasks.workunit.client.0.smithi100.stdout: "op_process_latency": { 2024-04-07T17:51:29.394 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 53268, 2024-04-07T17:51:29.394 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 125636506387 2024-04-07T17:51:29.394 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.394 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r": { 2024-04-07T17:51:29.394 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 37080 2024-04-07T17:51:29.394 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.394 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_latency": { 2024-04-07T17:51:29.395 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 37080, 2024-04-07T17:51:29.395 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 32585605896 2024-04-07T17:51:29.395 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.395 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_latency_out_bytes_histogram": { 2024-04-07T17:51:29.395 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.395 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.395 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_out_bytes": { 2024-04-07T17:51:29.395 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3752394330 2024-04-07T17:51:29.395 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.395 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_prepare_latency": { 2024-04-07T17:51:29.395 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 37440, 2024-04-07T17:51:29.395 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 17133305013 2024-04-07T17:51:29.395 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.395 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_process_latency": { 2024-04-07T17:51:29.395 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 37080, 2024-04-07T17:51:29.395 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 17454635423 2024-04-07T17:51:29.396 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.396 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw": { 2024-04-07T17:51:29.396 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 8557 2024-04-07T17:51:29.396 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.396 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_in_bytes": { 2024-04-07T17:51:29.396 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 574680166 2024-04-07T17:51:29.396 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.396 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_latency": { 2024-04-07T17:51:29.396 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 8557, 2024-04-07T17:51:29.396 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 36381436318 2024-04-07T17:51:29.396 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.396 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_latency_in_bytes_histogram": { 2024-04-07T17:51:29.396 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.396 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.396 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_latency_out_bytes_histogram": { 2024-04-07T17:51:29.397 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.397 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.397 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_out_bytes": { 2024-04-07T17:51:29.397 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.397 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.397 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_prepare_latency": { 2024-04-07T17:51:29.397 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 8868, 2024-04-07T17:51:29.397 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 12409822934 2024-04-07T17:51:29.397 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.397 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_process_latency": { 2024-04-07T17:51:29.397 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 8557, 2024-04-07T17:51:29.397 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 31269983898 2024-04-07T17:51:29.397 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.397 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w": { 2024-04-07T17:51:29.397 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 7631 2024-04-07T17:51:29.397 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.398 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_in_bytes": { 2024-04-07T17:51:29.398 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2090945730 2024-04-07T17:51:29.398 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.398 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_latency": { 2024-04-07T17:51:29.398 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 7631, 2024-04-07T17:51:29.398 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 296498858873 2024-04-07T17:51:29.398 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.398 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_latency_in_bytes_histogram": { 2024-04-07T17:51:29.398 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.398 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.398 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_prepare_latency": { 2024-04-07T17:51:29.398 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 7835, 2024-04-07T17:51:29.398 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 17605580563 2024-04-07T17:51:29.398 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.398 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_process_latency": { 2024-04-07T17:51:29.398 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 7631, 2024-04-07T17:51:29.399 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 76911887066 2024-04-07T17:51:29.399 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.399 INFO:tasks.workunit.client.0.smithi100.stdout: "op_wip": { 2024-04-07T17:51:29.399 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.399 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.399 INFO:tasks.workunit.client.0.smithi100.stdout: "recovery_bytes": { 2024-04-07T17:51:29.399 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.399 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.399 INFO:tasks.workunit.client.0.smithi100.stdout: "recovery_ops": { 2024-04-07T17:51:29.399 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 26 2024-04-07T17:51:29.399 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.399 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_bytes": { 2024-04-07T17:51:29.399 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 95995035648 2024-04-07T17:51:29.399 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.399 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_bytes_used": { 2024-04-07T17:51:29.399 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 745603072 2024-04-07T17:51:29.400 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.400 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.400 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.400 INFO:tasks.workunit.client.0.smithi100.stdout: "osd.1": { 2024-04-07T17:51:29.400 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs": { 2024-04-07T17:51:29.400 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_db_max_lat": { 2024-04-07T17:51:29.400 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 30825 2024-04-07T17:51:29.400 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.400 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_slow_fallback": { 2024-04-07T17:51:29.400 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.400 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.400 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_slow_max_lat": { 2024-04-07T17:51:29.400 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.400 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.400 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_slow_size_fallback": { 2024-04-07T17:51:29.400 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.401 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.401 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit_db": { 2024-04-07T17:51:29.401 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 65536 2024-04-07T17:51:29.401 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.401 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit_slow": { 2024-04-07T17:51:29.401 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.401 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.401 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit_wal": { 2024-04-07T17:51:29.401 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.401 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.401 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_wal_max_lat": { 2024-04-07T17:51:29.401 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.401 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.401 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_written_slow": { 2024-04-07T17:51:29.402 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.402 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.402 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_written_sst": { 2024-04-07T17:51:29.402 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5402624 2024-04-07T17:51:29.402 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.402 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_written_wal": { 2024-04-07T17:51:29.402 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 723148800 2024-04-07T17:51:29.402 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.402 INFO:tasks.workunit.client.0.smithi100.stdout: "compact_lat": { 2024-04-07T17:51:29.402 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 18, 2024-04-07T17:51:29.402 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5817020 2024-04-07T17:51:29.402 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.402 INFO:tasks.workunit.client.0.smithi100.stdout: "compact_lock_lat": { 2024-04-07T17:51:29.402 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 18, 2024-04-07T17:51:29.402 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2179935 2024-04-07T17:51:29.403 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.403 INFO:tasks.workunit.client.0.smithi100.stdout: "db_alloc_lat": { 2024-04-07T17:51:29.403 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 187, 2024-04-07T17:51:29.403 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1295182 2024-04-07T17:51:29.403 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.403 INFO:tasks.workunit.client.0.smithi100.stdout: "db_total_bytes": { 2024-04-07T17:51:29.403 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 95995027456 2024-04-07T17:51:29.403 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.403 INFO:tasks.workunit.client.0.smithi100.stdout: "db_used_bytes": { 2024-04-07T17:51:29.403 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 721354752 2024-04-07T17:51:29.403 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.403 INFO:tasks.workunit.client.0.smithi100.stdout: "flush_lat": { 2024-04-07T17:51:29.403 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 74047, 2024-04-07T17:51:29.403 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 13624180905 2024-04-07T17:51:29.403 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.404 INFO:tasks.workunit.client.0.smithi100.stdout: "fsync_lat": { 2024-04-07T17:51:29.404 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 74056, 2024-04-07T17:51:29.404 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 23366921692 2024-04-07T17:51:29.404 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.404 INFO:tasks.workunit.client.0.smithi100.stdout: "log_bytes": { 2024-04-07T17:51:29.404 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4120576 2024-04-07T17:51:29.404 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.404 INFO:tasks.workunit.client.0.smithi100.stdout: "logged_bytes": { 2024-04-07T17:51:29.404 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 303386624 2024-04-07T17:51:29.404 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.404 INFO:tasks.workunit.client.0.smithi100.stdout: "max_bytes_db": { 2024-04-07T17:51:29.404 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 738394112 2024-04-07T17:51:29.404 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.404 INFO:tasks.workunit.client.0.smithi100.stdout: "max_bytes_slow": { 2024-04-07T17:51:29.404 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.405 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.405 INFO:tasks.workunit.client.0.smithi100.stdout: "max_bytes_wal": { 2024-04-07T17:51:29.405 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.405 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.405 INFO:tasks.workunit.client.0.smithi100.stdout: "num_files": { 2024-04-07T17:51:29.405 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 50 2024-04-07T17:51:29.405 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.405 INFO:tasks.workunit.client.0.smithi100.stdout: "read_bytes": { 2024-04-07T17:51:29.405 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3171092523 2024-04-07T17:51:29.405 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.405 INFO:tasks.workunit.client.0.smithi100.stdout: "read_count": { 2024-04-07T17:51:29.405 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 42792 2024-04-07T17:51:29.405 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.405 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes": { 2024-04-07T17:51:29.405 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2761277440 2024-04-07T17:51:29.406 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.406 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes_db": { 2024-04-07T17:51:29.406 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2761281536 2024-04-07T17:51:29.406 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.406 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes_slow": { 2024-04-07T17:51:29.406 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.406 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.406 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes_wal": { 2024-04-07T17:51:29.406 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.406 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.406 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_count": { 2024-04-07T17:51:29.406 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 35333 2024-04-07T17:51:29.406 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.406 INFO:tasks.workunit.client.0.smithi100.stdout: "read_lat": { 2024-04-07T17:51:29.406 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 42792, 2024-04-07T17:51:29.407 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2006369197 2024-04-07T17:51:29.407 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.407 INFO:tasks.workunit.client.0.smithi100.stdout: "read_prefetch_bytes": { 2024-04-07T17:51:29.407 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3054473279 2024-04-07T17:51:29.407 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.407 INFO:tasks.workunit.client.0.smithi100.stdout: "read_prefetch_count": { 2024-04-07T17:51:29.407 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 39218 2024-04-07T17:51:29.407 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.407 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_buffer_bytes": { 2024-04-07T17:51:29.407 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1733730 2024-04-07T17:51:29.407 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.407 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_buffer_count": { 2024-04-07T17:51:29.407 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 618 2024-04-07T17:51:29.407 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.407 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_bytes": { 2024-04-07T17:51:29.408 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5349743 2024-04-07T17:51:29.408 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.408 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_count": { 2024-04-07T17:51:29.408 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 831 2024-04-07T17:51:29.408 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.408 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes": { 2024-04-07T17:51:29.408 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3616013 2024-04-07T17:51:29.408 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.408 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes_db": { 2024-04-07T17:51:29.408 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3616013 2024-04-07T17:51:29.408 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.408 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes_slow": { 2024-04-07T17:51:29.408 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.408 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.408 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes_wal": { 2024-04-07T17:51:29.408 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.409 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.409 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_count": { 2024-04-07T17:51:29.409 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 217 2024-04-07T17:51:29.409 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.409 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_lat": { 2024-04-07T17:51:29.409 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 831, 2024-04-07T17:51:29.409 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2170610 2024-04-07T17:51:29.409 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.409 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_alloc_lat": { 2024-04-07T17:51:29.409 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:29.409 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.409 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.409 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_total_bytes": { 2024-04-07T17:51:29.409 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.409 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.410 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_used_bytes": { 2024-04-07T17:51:29.410 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.410 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.410 INFO:tasks.workunit.client.0.smithi100.stdout: "truncate_lat": { 2024-04-07T17:51:29.410 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1, 2024-04-07T17:51:29.410 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1014 2024-04-07T17:51:29.410 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.410 INFO:tasks.workunit.client.0.smithi100.stdout: "unlink_lat": { 2024-04-07T17:51:29.410 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 10, 2024-04-07T17:51:29.410 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 14895 2024-04-07T17:51:29.410 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.410 INFO:tasks.workunit.client.0.smithi100.stdout: "wal_alloc_lat": { 2024-04-07T17:51:29.410 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:29.410 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.411 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.411 INFO:tasks.workunit.client.0.smithi100.stdout: "wal_total_bytes": { 2024-04-07T17:51:29.411 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.411 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.411 INFO:tasks.workunit.client.0.smithi100.stdout: "wal_used_bytes": { 2024-04-07T17:51:29.411 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.411 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.411 INFO:tasks.workunit.client.0.smithi100.stdout: "write_bytes": { 2024-04-07T17:51:29.411 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1034661888 2024-04-07T17:51:29.411 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.411 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.411 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore": { 2024-04-07T17:51:29.411 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit": { 2024-04-07T17:51:29.411 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4096 2024-04-07T17:51:29.411 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.411 INFO:tasks.workunit.client.0.smithi100.stdout: "allocated": { 2024-04-07T17:51:29.412 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 9814016 2024-04-07T17:51:29.412 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.412 INFO:tasks.workunit.client.0.smithi100.stdout: "allocator_lat": { 2024-04-07T17:51:29.412 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 18804, 2024-04-07T17:51:29.412 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 100612374 2024-04-07T17:51:29.412 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.412 INFO:tasks.workunit.client.0.smithi100.stdout: "clist_lat": { 2024-04-07T17:51:29.412 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 3554, 2024-04-07T17:51:29.412 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 9702602885 2024-04-07T17:51:29.412 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.412 INFO:tasks.workunit.client.0.smithi100.stdout: "compress_lat": { 2024-04-07T17:51:29.412 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:29.412 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.412 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.412 INFO:tasks.workunit.client.0.smithi100.stdout: "compressed": { 2024-04-07T17:51:29.412 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.413 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.413 INFO:tasks.workunit.client.0.smithi100.stdout: "compressed_allocated": { 2024-04-07T17:51:29.413 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.413 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.413 INFO:tasks.workunit.client.0.smithi100.stdout: "compressed_original": { 2024-04-07T17:51:29.413 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.413 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.413 INFO:tasks.workunit.client.0.smithi100.stdout: "csum_lat": { 2024-04-07T17:51:29.413 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 20962, 2024-04-07T17:51:29.413 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 250539900 2024-04-07T17:51:29.413 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.413 INFO:tasks.workunit.client.0.smithi100.stdout: "decompress_lat": { 2024-04-07T17:51:29.413 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:29.413 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.413 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.414 INFO:tasks.workunit.client.0.smithi100.stdout: "fragmentation_micros": { 2024-04-07T17:51:29.414 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.414 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.414 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_commit_lat": { 2024-04-07T17:51:29.414 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 73998, 2024-04-07T17:51:29.414 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 30592293335 2024-04-07T17:51:29.414 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.414 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_final_lat": { 2024-04-07T17:51:29.414 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 73127, 2024-04-07T17:51:29.414 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3885020813 2024-04-07T17:51:29.414 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.414 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_flush_lat": { 2024-04-07T17:51:29.414 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 73998, 2024-04-07T17:51:29.414 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 99101609 2024-04-07T17:51:29.415 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.415 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_sync_lat": { 2024-04-07T17:51:29.415 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 73998, 2024-04-07T17:51:29.415 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 30691394944 2024-04-07T17:51:29.415 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.415 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_get_keys_lat": { 2024-04-07T17:51:29.415 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:29.415 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.415 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.415 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_get_values_lat": { 2024-04-07T17:51:29.415 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 29479, 2024-04-07T17:51:29.415 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1535907762 2024-04-07T17:51:29.415 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.415 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_lower_bound_lat": { 2024-04-07T17:51:29.415 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 847, 2024-04-07T17:51:29.415 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5353667 2024-04-07T17:51:29.416 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.416 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_next_lat": { 2024-04-07T17:51:29.416 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 17576, 2024-04-07T17:51:29.416 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 23428630 2024-04-07T17:51:29.416 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.416 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_seek_to_first_lat": { 2024-04-07T17:51:29.416 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 45, 2024-04-07T17:51:29.416 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 74604 2024-04-07T17:51:29.416 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.416 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_upper_bound_lat": { 2024-04-07T17:51:29.416 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 11754, 2024-04-07T17:51:29.416 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3050723203 2024-04-07T17:51:29.416 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.416 INFO:tasks.workunit.client.0.smithi100.stdout: "onode_hits": { 2024-04-07T17:51:29.416 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 227672 2024-04-07T17:51:29.417 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.417 INFO:tasks.workunit.client.0.smithi100.stdout: "onode_misses": { 2024-04-07T17:51:29.417 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 26792 2024-04-07T17:51:29.417 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.417 INFO:tasks.workunit.client.0.smithi100.stdout: "read_lat": { 2024-04-07T17:51:29.417 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 23761, 2024-04-07T17:51:29.417 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3480334341 2024-04-07T17:51:29.417 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.417 INFO:tasks.workunit.client.0.smithi100.stdout: "read_onode_meta_lat": { 2024-04-07T17:51:29.417 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 76381, 2024-04-07T17:51:29.417 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 39178980 2024-04-07T17:51:29.417 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.417 INFO:tasks.workunit.client.0.smithi100.stdout: "read_wait_aio_lat": { 2024-04-07T17:51:29.417 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 52620, 2024-04-07T17:51:29.417 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1661838016 2024-04-07T17:51:29.417 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.418 INFO:tasks.workunit.client.0.smithi100.stdout: "reads_with_retries": { 2024-04-07T17:51:29.418 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.418 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.418 INFO:tasks.workunit.client.0.smithi100.stdout: "remove_lat": { 2024-04-07T17:51:29.418 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 15062, 2024-04-07T17:51:29.418 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 718857059 2024-04-07T17:51:29.418 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.418 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_aio_wait_count": { 2024-04-07T17:51:29.418 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.418 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.418 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_committed_kv_count": { 2024-04-07T17:51:29.418 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.418 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.418 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_read_onode_meta_count": { 2024-04-07T17:51:29.419 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.419 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.419 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_read_wait_aio_count": { 2024-04-07T17:51:29.419 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.419 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.419 INFO:tasks.workunit.client.0.smithi100.stdout: "state_aio_wait_lat": { 2024-04-07T17:51:29.419 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 85696, 2024-04-07T17:51:29.419 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 8515002567 2024-04-07T17:51:29.419 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.419 INFO:tasks.workunit.client.0.smithi100.stdout: "state_deferred_aio_wait_lat": { 2024-04-07T17:51:29.419 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 22706, 2024-04-07T17:51:29.419 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2143104911 2024-04-07T17:51:29.419 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.419 INFO:tasks.workunit.client.0.smithi100.stdout: "state_deferred_cleanup_lat": { 2024-04-07T17:51:29.420 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 22706, 2024-04-07T17:51:29.420 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4386813655909 2024-04-07T17:51:29.420 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.420 INFO:tasks.workunit.client.0.smithi100.stdout: "state_deferred_queued_lat": { 2024-04-07T17:51:29.420 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 22706, 2024-04-07T17:51:29.420 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 7212579664352 2024-04-07T17:51:29.420 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.420 INFO:tasks.workunit.client.0.smithi100.stdout: "state_done_lat": { 2024-04-07T17:51:29.420 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 85696, 2024-04-07T17:51:29.420 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2950395751529 2024-04-07T17:51:29.420 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.420 INFO:tasks.workunit.client.0.smithi100.stdout: "state_finishing_lat": { 2024-04-07T17:51:29.420 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 85696, 2024-04-07T17:51:29.420 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 17842298 2024-04-07T17:51:29.420 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.421 INFO:tasks.workunit.client.0.smithi100.stdout: "state_io_done_lat": { 2024-04-07T17:51:29.421 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 85696, 2024-04-07T17:51:29.421 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 318013774 2024-04-07T17:51:29.421 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.421 INFO:tasks.workunit.client.0.smithi100.stdout: "state_kv_commiting_lat": { 2024-04-07T17:51:29.421 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 85696, 2024-04-07T17:51:29.421 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 50196560708 2024-04-07T17:51:29.421 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.421 INFO:tasks.workunit.client.0.smithi100.stdout: "state_kv_done_lat": { 2024-04-07T17:51:29.421 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 85696, 2024-04-07T17:51:29.421 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 64451217 2024-04-07T17:51:29.421 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.421 INFO:tasks.workunit.client.0.smithi100.stdout: "state_kv_queued_lat": { 2024-04-07T17:51:29.421 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 85696, 2024-04-07T17:51:29.421 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 24917952642 2024-04-07T17:51:29.421 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.422 INFO:tasks.workunit.client.0.smithi100.stdout: "state_prepare_lat": { 2024-04-07T17:51:29.422 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 85696, 2024-04-07T17:51:29.422 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 23724441505 2024-04-07T17:51:29.422 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.422 INFO:tasks.workunit.client.0.smithi100.stdout: "stored": { 2024-04-07T17:51:29.422 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5699861 2024-04-07T17:51:29.422 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.422 INFO:tasks.workunit.client.0.smithi100.stdout: "truncate_lat": { 2024-04-07T17:51:29.422 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 2507, 2024-04-07T17:51:29.422 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 25130678 2024-04-07T17:51:29.422 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.422 INFO:tasks.workunit.client.0.smithi100.stdout: "txc_commit_lat": { 2024-04-07T17:51:29.422 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 85696, 2024-04-07T17:51:29.422 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 107702564402 2024-04-07T17:51:29.423 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.423 INFO:tasks.workunit.client.0.smithi100.stdout: "txc_submit_lat": { 2024-04-07T17:51:29.423 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 85696, 2024-04-07T17:51:29.423 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 25902403453 2024-04-07T17:51:29.423 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.423 INFO:tasks.workunit.client.0.smithi100.stdout: "txc_throttle_lat": { 2024-04-07T17:51:29.423 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 85696, 2024-04-07T17:51:29.423 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 991891115 2024-04-07T17:51:29.423 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.423 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.423 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache": { 2024-04-07T17:51:29.423 INFO:tasks.workunit.client.0.smithi100.stdout: "cache_bytes": { 2024-04-07T17:51:29.423 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1444614092 2024-04-07T17:51:29.423 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.423 INFO:tasks.workunit.client.0.smithi100.stdout: "heap_bytes": { 2024-04-07T17:51:29.423 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 695918592 2024-04-07T17:51:29.424 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.424 INFO:tasks.workunit.client.0.smithi100.stdout: "mapped_bytes": { 2024-04-07T17:51:29.424 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 660807680 2024-04-07T17:51:29.424 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.424 INFO:tasks.workunit.client.0.smithi100.stdout: "target_bytes": { 2024-04-07T17:51:29.424 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2646965248 2024-04-07T17:51:29.424 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.424 INFO:tasks.workunit.client.0.smithi100.stdout: "unmapped_bytes": { 2024-04-07T17:51:29.424 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 35110912 2024-04-07T17:51:29.424 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.424 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.424 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:data": { 2024-04-07T17:51:29.424 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:29.424 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 134217728 2024-04-07T17:51:29.425 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.425 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:29.425 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.425 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.425 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:29.425 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.425 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.425 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:29.425 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 59401471 2024-04-07T17:51:29.425 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.425 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:29.425 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.425 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.425 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:29.425 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.426 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.426 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:29.426 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.426 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.426 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:29.426 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.426 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.426 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:29.426 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 32768 2024-04-07T17:51:29.426 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.426 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:29.426 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 815104 2024-04-07T17:51:29.426 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.426 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:29.426 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.427 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.427 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:29.427 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.427 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.427 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:29.427 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.427 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.427 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:29.427 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 73968385 2024-04-07T17:51:29.427 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.427 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.427 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:kv": { 2024-04-07T17:51:29.427 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:29.427 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 587202560 2024-04-07T17:51:29.427 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.428 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:29.428 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 67108864 2024-04-07T17:51:29.428 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.428 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:29.428 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.428 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.428 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:29.428 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 445511035 2024-04-07T17:51:29.428 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.428 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:29.428 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.428 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.428 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:29.428 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 256 2024-04-07T17:51:29.428 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.429 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:29.429 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.429 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.429 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:29.429 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.429 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.429 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:29.429 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 112208 2024-04-07T17:51:29.429 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.429 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:29.429 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.429 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.429 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:29.429 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.429 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.430 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:29.430 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.430 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.430 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:29.430 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.430 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.430 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:29.430 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 74470197 2024-04-07T17:51:29.430 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.430 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.430 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:kv_onode": { 2024-04-07T17:51:29.430 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:29.430 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 184549376 2024-04-07T17:51:29.430 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.431 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:29.431 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 67108864 2024-04-07T17:51:29.431 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.431 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:29.431 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4603888 2024-04-07T17:51:29.431 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.431 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:29.431 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 39600980 2024-04-07T17:51:29.431 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.431 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:29.431 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.431 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.431 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:29.431 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.431 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.431 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:29.432 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.432 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.432 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:29.432 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.432 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.432 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:29.432 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.432 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.432 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:29.432 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.432 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.432 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:29.432 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.432 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.432 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:29.432 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.433 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.433 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:29.433 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.433 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.433 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:29.433 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 73235644 2024-04-07T17:51:29.433 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.433 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.433 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:meta": { 2024-04-07T17:51:29.433 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:29.433 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 520093696 2024-04-07T17:51:29.433 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.433 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:29.433 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.434 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.434 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:29.434 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.434 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.434 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:29.434 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 445511035 2024-04-07T17:51:29.434 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.434 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:29.434 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.434 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.434 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:29.434 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.434 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.434 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:29.434 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.435 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.435 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:29.435 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.435 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.435 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:29.435 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 143023 2024-04-07T17:51:29.435 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.435 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:29.435 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1096839 2024-04-07T17:51:29.435 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.435 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:29.435 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.435 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.435 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:29.435 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.436 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.436 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:29.436 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.436 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.436 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:29.436 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 73342799 2024-04-07T17:51:29.436 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.436 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.436 INFO:tasks.workunit.client.0.smithi100.stdout: "objecter": { 2024-04-07T17:51:29.436 INFO:tasks.workunit.client.0.smithi100.stdout: "op_active": { 2024-04-07T17:51:29.436 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.436 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.436 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r": { 2024-04-07T17:51:29.436 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 21 2024-04-07T17:51:29.437 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.437 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rmw": { 2024-04-07T17:51:29.437 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.437 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.437 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w": { 2024-04-07T17:51:29.437 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1 2024-04-07T17:51:29.437 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.437 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.437 INFO:tasks.workunit.client.0.smithi100.stdout: "osd": { 2024-04-07T17:51:29.437 INFO:tasks.workunit.client.0.smithi100.stdout: "numpg": { 2024-04-07T17:51:29.437 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 52 2024-04-07T17:51:29.437 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.437 INFO:tasks.workunit.client.0.smithi100.stdout: "numpg_removing": { 2024-04-07T17:51:29.437 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.437 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.438 INFO:tasks.workunit.client.0.smithi100.stdout: "op": { 2024-04-07T17:51:29.438 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 45446 2024-04-07T17:51:29.438 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.438 INFO:tasks.workunit.client.0.smithi100.stdout: "op_before_queue_op_lat": { 2024-04-07T17:51:29.438 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 141258, 2024-04-07T17:51:29.438 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 14856495861 2024-04-07T17:51:29.438 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.438 INFO:tasks.workunit.client.0.smithi100.stdout: "op_delayed_degraded": { 2024-04-07T17:51:29.438 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.438 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.438 INFO:tasks.workunit.client.0.smithi100.stdout: "op_delayed_unreadable": { 2024-04-07T17:51:29.438 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1 2024-04-07T17:51:29.438 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.439 INFO:tasks.workunit.client.0.smithi100.stdout: "op_in_bytes": { 2024-04-07T17:51:29.439 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2465060363 2024-04-07T17:51:29.439 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.439 INFO:tasks.workunit.client.0.smithi100.stdout: "op_latency": { 2024-04-07T17:51:29.439 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 45446, 2024-04-07T17:51:29.439 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 845709748759 2024-04-07T17:51:29.439 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.439 INFO:tasks.workunit.client.0.smithi100.stdout: "op_out_bytes": { 2024-04-07T17:51:29.439 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2916484475 2024-04-07T17:51:29.439 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.439 INFO:tasks.workunit.client.0.smithi100.stdout: "op_prepare_latency": { 2024-04-07T17:51:29.439 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 47110, 2024-04-07T17:51:29.439 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 66786302195 2024-04-07T17:51:29.439 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.439 INFO:tasks.workunit.client.0.smithi100.stdout: "op_process_latency": { 2024-04-07T17:51:29.439 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 45446, 2024-04-07T17:51:29.440 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 194962917930 2024-04-07T17:51:29.440 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.440 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r": { 2024-04-07T17:51:29.440 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 22763 2024-04-07T17:51:29.440 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.440 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_latency": { 2024-04-07T17:51:29.440 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 22763, 2024-04-07T17:51:29.440 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 30619013027 2024-04-07T17:51:29.440 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.440 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_latency_out_bytes_histogram": { 2024-04-07T17:51:29.440 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.440 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.440 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_out_bytes": { 2024-04-07T17:51:29.440 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2916484475 2024-04-07T17:51:29.440 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.440 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_prepare_latency": { 2024-04-07T17:51:29.441 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 23180, 2024-04-07T17:51:29.441 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 12773511407 2024-04-07T17:51:29.441 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.441 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_process_latency": { 2024-04-07T17:51:29.441 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 22763, 2024-04-07T17:51:29.441 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 12468479574 2024-04-07T17:51:29.441 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.441 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw": { 2024-04-07T17:51:29.441 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 12919 2024-04-07T17:51:29.441 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.441 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_in_bytes": { 2024-04-07T17:51:29.441 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 335492738 2024-04-07T17:51:29.441 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.441 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_latency": { 2024-04-07T17:51:29.441 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 12919, 2024-04-07T17:51:29.442 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 239996971130 2024-04-07T17:51:29.442 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.442 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_latency_in_bytes_histogram": { 2024-04-07T17:51:29.442 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.442 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.442 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_latency_out_bytes_histogram": { 2024-04-07T17:51:29.442 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.442 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.442 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_out_bytes": { 2024-04-07T17:51:29.442 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.442 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.442 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_prepare_latency": { 2024-04-07T17:51:29.442 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 13070, 2024-04-07T17:51:29.442 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 30679482792 2024-04-07T17:51:29.442 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.443 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_process_latency": { 2024-04-07T17:51:29.443 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 12919, 2024-04-07T17:51:29.443 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 75646162399 2024-04-07T17:51:29.443 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.443 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w": { 2024-04-07T17:51:29.443 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 9764 2024-04-07T17:51:29.443 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.443 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_in_bytes": { 2024-04-07T17:51:29.443 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2129567625 2024-04-07T17:51:29.443 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.443 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_latency": { 2024-04-07T17:51:29.443 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 9764, 2024-04-07T17:51:29.443 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 575093764602 2024-04-07T17:51:29.443 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.443 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_latency_in_bytes_histogram": { 2024-04-07T17:51:29.444 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.444 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.444 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_prepare_latency": { 2024-04-07T17:51:29.444 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 10860, 2024-04-07T17:51:29.444 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 23333307996 2024-04-07T17:51:29.444 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.444 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_process_latency": { 2024-04-07T17:51:29.444 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 9764, 2024-04-07T17:51:29.444 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 106848275957 2024-04-07T17:51:29.444 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.444 INFO:tasks.workunit.client.0.smithi100.stdout: "op_wip": { 2024-04-07T17:51:29.444 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.444 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.444 INFO:tasks.workunit.client.0.smithi100.stdout: "recovery_bytes": { 2024-04-07T17:51:29.445 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 40249613 2024-04-07T17:51:29.445 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.445 INFO:tasks.workunit.client.0.smithi100.stdout: "recovery_ops": { 2024-04-07T17:51:29.445 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 28 2024-04-07T17:51:29.445 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.445 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_bytes": { 2024-04-07T17:51:29.445 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 95995035648 2024-04-07T17:51:29.445 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.445 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_bytes_used": { 2024-04-07T17:51:29.445 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 731176960 2024-04-07T17:51:29.445 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.445 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.445 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.445 INFO:tasks.workunit.client.0.smithi100.stdout: "osd.2": { 2024-04-07T17:51:29.445 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs": { 2024-04-07T17:51:29.445 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_db_max_lat": { 2024-04-07T17:51:29.446 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 33529 2024-04-07T17:51:29.446 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.446 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_slow_fallback": { 2024-04-07T17:51:29.446 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.446 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.446 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_slow_max_lat": { 2024-04-07T17:51:29.446 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.446 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.446 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_slow_size_fallback": { 2024-04-07T17:51:29.446 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.446 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.446 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit_db": { 2024-04-07T17:51:29.446 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 65536 2024-04-07T17:51:29.447 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.447 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit_slow": { 2024-04-07T17:51:29.447 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.447 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.447 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit_wal": { 2024-04-07T17:51:29.447 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.447 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.447 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_wal_max_lat": { 2024-04-07T17:51:29.447 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.447 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.447 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_written_slow": { 2024-04-07T17:51:29.447 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.447 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.447 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_written_sst": { 2024-04-07T17:51:29.447 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 11583488 2024-04-07T17:51:29.448 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.448 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_written_wal": { 2024-04-07T17:51:29.448 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 567971840 2024-04-07T17:51:29.448 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.448 INFO:tasks.workunit.client.0.smithi100.stdout: "compact_lat": { 2024-04-07T17:51:29.448 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 17, 2024-04-07T17:51:29.448 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 10549451 2024-04-07T17:51:29.448 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.448 INFO:tasks.workunit.client.0.smithi100.stdout: "compact_lock_lat": { 2024-04-07T17:51:29.448 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 17, 2024-04-07T17:51:29.448 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3589378 2024-04-07T17:51:29.448 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.448 INFO:tasks.workunit.client.0.smithi100.stdout: "db_alloc_lat": { 2024-04-07T17:51:29.448 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 176, 2024-04-07T17:51:29.448 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1295988 2024-04-07T17:51:29.448 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.449 INFO:tasks.workunit.client.0.smithi100.stdout: "db_total_bytes": { 2024-04-07T17:51:29.449 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 95995027456 2024-04-07T17:51:29.449 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.449 INFO:tasks.workunit.client.0.smithi100.stdout: "db_used_bytes": { 2024-04-07T17:51:29.449 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 446365696 2024-04-07T17:51:29.449 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.449 INFO:tasks.workunit.client.0.smithi100.stdout: "flush_lat": { 2024-04-07T17:51:29.449 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 69116, 2024-04-07T17:51:29.449 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 12568150083 2024-04-07T17:51:29.449 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.449 INFO:tasks.workunit.client.0.smithi100.stdout: "fsync_lat": { 2024-04-07T17:51:29.449 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 69111, 2024-04-07T17:51:29.449 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 21920914517 2024-04-07T17:51:29.449 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.449 INFO:tasks.workunit.client.0.smithi100.stdout: "log_bytes": { 2024-04-07T17:51:29.450 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 491520 2024-04-07T17:51:29.450 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.450 INFO:tasks.workunit.client.0.smithi100.stdout: "logged_bytes": { 2024-04-07T17:51:29.450 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 283127808 2024-04-07T17:51:29.450 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.450 INFO:tasks.workunit.client.0.smithi100.stdout: "max_bytes_db": { 2024-04-07T17:51:29.450 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 449183744 2024-04-07T17:51:29.450 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.450 INFO:tasks.workunit.client.0.smithi100.stdout: "max_bytes_slow": { 2024-04-07T17:51:29.450 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.450 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.450 INFO:tasks.workunit.client.0.smithi100.stdout: "max_bytes_wal": { 2024-04-07T17:51:29.450 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.450 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.450 INFO:tasks.workunit.client.0.smithi100.stdout: "num_files": { 2024-04-07T17:51:29.451 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 47 2024-04-07T17:51:29.451 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.451 INFO:tasks.workunit.client.0.smithi100.stdout: "read_bytes": { 2024-04-07T17:51:29.451 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 8760902516 2024-04-07T17:51:29.451 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.451 INFO:tasks.workunit.client.0.smithi100.stdout: "read_count": { 2024-04-07T17:51:29.451 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 74144 2024-04-07T17:51:29.451 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.451 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes": { 2024-04-07T17:51:29.451 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 7434956800 2024-04-07T17:51:29.451 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.451 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes_db": { 2024-04-07T17:51:29.451 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 7434960896 2024-04-07T17:51:29.451 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.451 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes_slow": { 2024-04-07T17:51:29.451 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.452 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.452 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes_wal": { 2024-04-07T17:51:29.452 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.452 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.452 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_count": { 2024-04-07T17:51:29.452 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 64202 2024-04-07T17:51:29.452 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.452 INFO:tasks.workunit.client.0.smithi100.stdout: "read_lat": { 2024-04-07T17:51:29.452 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 74144, 2024-04-07T17:51:29.452 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4199208949 2024-04-07T17:51:29.452 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.452 INFO:tasks.workunit.client.0.smithi100.stdout: "read_prefetch_bytes": { 2024-04-07T17:51:29.452 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 8601015701 2024-04-07T17:51:29.452 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.452 INFO:tasks.workunit.client.0.smithi100.stdout: "read_prefetch_count": { 2024-04-07T17:51:29.453 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 69247 2024-04-07T17:51:29.453 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.453 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_buffer_bytes": { 2024-04-07T17:51:29.453 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3233145 2024-04-07T17:51:29.453 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.453 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_buffer_count": { 2024-04-07T17:51:29.453 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 709 2024-04-07T17:51:29.453 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.453 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_bytes": { 2024-04-07T17:51:29.453 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 11496373 2024-04-07T17:51:29.453 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.453 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_count": { 2024-04-07T17:51:29.453 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1006 2024-04-07T17:51:29.453 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.453 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes": { 2024-04-07T17:51:29.454 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 8263228 2024-04-07T17:51:29.454 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.454 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes_db": { 2024-04-07T17:51:29.454 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 8263228 2024-04-07T17:51:29.454 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.454 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes_slow": { 2024-04-07T17:51:29.454 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.454 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.454 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes_wal": { 2024-04-07T17:51:29.454 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.454 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.454 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_count": { 2024-04-07T17:51:29.454 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 312 2024-04-07T17:51:29.454 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.454 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_lat": { 2024-04-07T17:51:29.455 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1006, 2024-04-07T17:51:29.455 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4007550 2024-04-07T17:51:29.455 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.455 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_alloc_lat": { 2024-04-07T17:51:29.455 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:29.455 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.455 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.455 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_total_bytes": { 2024-04-07T17:51:29.455 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.455 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.455 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_used_bytes": { 2024-04-07T17:51:29.455 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.455 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.456 INFO:tasks.workunit.client.0.smithi100.stdout: "truncate_lat": { 2024-04-07T17:51:29.456 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1, 2024-04-07T17:51:29.456 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 636 2024-04-07T17:51:29.456 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.456 INFO:tasks.workunit.client.0.smithi100.stdout: "unlink_lat": { 2024-04-07T17:51:29.456 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 13, 2024-04-07T17:51:29.456 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 14014 2024-04-07T17:51:29.456 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.456 INFO:tasks.workunit.client.0.smithi100.stdout: "wal_alloc_lat": { 2024-04-07T17:51:29.456 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:29.456 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.456 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.456 INFO:tasks.workunit.client.0.smithi100.stdout: "wal_total_bytes": { 2024-04-07T17:51:29.456 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.456 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.457 INFO:tasks.workunit.client.0.smithi100.stdout: "wal_used_bytes": { 2024-04-07T17:51:29.457 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.457 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.457 INFO:tasks.workunit.client.0.smithi100.stdout: "write_bytes": { 2024-04-07T17:51:29.457 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 865255424 2024-04-07T17:51:29.457 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.457 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.457 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore": { 2024-04-07T17:51:29.457 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit": { 2024-04-07T17:51:29.457 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4096 2024-04-07T17:51:29.457 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.457 INFO:tasks.workunit.client.0.smithi100.stdout: "allocated": { 2024-04-07T17:51:29.457 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 9801728 2024-04-07T17:51:29.457 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.457 INFO:tasks.workunit.client.0.smithi100.stdout: "allocator_lat": { 2024-04-07T17:51:29.458 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 13808, 2024-04-07T17:51:29.458 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 73423246 2024-04-07T17:51:29.458 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.458 INFO:tasks.workunit.client.0.smithi100.stdout: "clist_lat": { 2024-04-07T17:51:29.458 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 3623, 2024-04-07T17:51:29.458 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 12377775846 2024-04-07T17:51:29.458 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.458 INFO:tasks.workunit.client.0.smithi100.stdout: "compress_lat": { 2024-04-07T17:51:29.458 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:29.458 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.458 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.458 INFO:tasks.workunit.client.0.smithi100.stdout: "compressed": { 2024-04-07T17:51:29.458 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.458 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.458 INFO:tasks.workunit.client.0.smithi100.stdout: "compressed_allocated": { 2024-04-07T17:51:29.459 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.459 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.459 INFO:tasks.workunit.client.0.smithi100.stdout: "compressed_original": { 2024-04-07T17:51:29.459 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.459 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.459 INFO:tasks.workunit.client.0.smithi100.stdout: "csum_lat": { 2024-04-07T17:51:29.459 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 26449, 2024-04-07T17:51:29.459 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 333837013 2024-04-07T17:51:29.459 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.459 INFO:tasks.workunit.client.0.smithi100.stdout: "decompress_lat": { 2024-04-07T17:51:29.459 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:29.459 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.459 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.459 INFO:tasks.workunit.client.0.smithi100.stdout: "fragmentation_micros": { 2024-04-07T17:51:29.459 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.459 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.459 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_commit_lat": { 2024-04-07T17:51:29.460 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 69053, 2024-04-07T17:51:29.460 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 28692593210 2024-04-07T17:51:29.460 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.460 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_final_lat": { 2024-04-07T17:51:29.460 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 68316, 2024-04-07T17:51:29.460 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3308256168 2024-04-07T17:51:29.460 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.460 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_flush_lat": { 2024-04-07T17:51:29.460 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 69053, 2024-04-07T17:51:29.460 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 77146286 2024-04-07T17:51:29.460 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.460 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_sync_lat": { 2024-04-07T17:51:29.460 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 69053, 2024-04-07T17:51:29.460 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 28769739496 2024-04-07T17:51:29.460 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.460 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_get_keys_lat": { 2024-04-07T17:51:29.461 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:29.461 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.461 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.461 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_get_values_lat": { 2024-04-07T17:51:29.461 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 54368, 2024-04-07T17:51:29.461 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1618898953 2024-04-07T17:51:29.461 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.461 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_lower_bound_lat": { 2024-04-07T17:51:29.461 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 911, 2024-04-07T17:51:29.461 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5592855 2024-04-07T17:51:29.461 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.461 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_next_lat": { 2024-04-07T17:51:29.461 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 22756, 2024-04-07T17:51:29.461 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 25939924 2024-04-07T17:51:29.461 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.461 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_seek_to_first_lat": { 2024-04-07T17:51:29.462 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 36, 2024-04-07T17:51:29.462 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 71387 2024-04-07T17:51:29.462 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.462 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_upper_bound_lat": { 2024-04-07T17:51:29.462 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 19162, 2024-04-07T17:51:29.462 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2934148688 2024-04-07T17:51:29.462 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.462 INFO:tasks.workunit.client.0.smithi100.stdout: "onode_hits": { 2024-04-07T17:51:29.462 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 251164 2024-04-07T17:51:29.462 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.462 INFO:tasks.workunit.client.0.smithi100.stdout: "onode_misses": { 2024-04-07T17:51:29.462 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 19790 2024-04-07T17:51:29.462 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.462 INFO:tasks.workunit.client.0.smithi100.stdout: "read_lat": { 2024-04-07T17:51:29.462 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 17396, 2024-04-07T17:51:29.463 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2437128824 2024-04-07T17:51:29.463 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.463 INFO:tasks.workunit.client.0.smithi100.stdout: "read_onode_meta_lat": { 2024-04-07T17:51:29.463 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 41161, 2024-04-07T17:51:29.463 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 40062786 2024-04-07T17:51:29.463 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.463 INFO:tasks.workunit.client.0.smithi100.stdout: "read_wait_aio_lat": { 2024-04-07T17:51:29.463 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 23765, 2024-04-07T17:51:29.463 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1924044806 2024-04-07T17:51:29.463 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.463 INFO:tasks.workunit.client.0.smithi100.stdout: "reads_with_retries": { 2024-04-07T17:51:29.463 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.463 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.463 INFO:tasks.workunit.client.0.smithi100.stdout: "remove_lat": { 2024-04-07T17:51:29.463 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 12440, 2024-04-07T17:51:29.463 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 700556520 2024-04-07T17:51:29.464 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.464 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_aio_wait_count": { 2024-04-07T17:51:29.464 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.464 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.464 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_committed_kv_count": { 2024-04-07T17:51:29.464 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.464 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.464 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_read_onode_meta_count": { 2024-04-07T17:51:29.464 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.464 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.464 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_read_wait_aio_count": { 2024-04-07T17:51:29.464 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.464 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.464 INFO:tasks.workunit.client.0.smithi100.stdout: "state_aio_wait_lat": { 2024-04-07T17:51:29.464 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 77648, 2024-04-07T17:51:29.464 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 7605898743 2024-04-07T17:51:29.465 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.465 INFO:tasks.workunit.client.0.smithi100.stdout: "state_deferred_aio_wait_lat": { 2024-04-07T17:51:29.465 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 5880, 2024-04-07T17:51:29.465 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 870604827 2024-04-07T17:51:29.465 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.465 INFO:tasks.workunit.client.0.smithi100.stdout: "state_deferred_cleanup_lat": { 2024-04-07T17:51:29.465 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 5879, 2024-04-07T17:51:29.465 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4540688258752 2024-04-07T17:51:29.465 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.465 INFO:tasks.workunit.client.0.smithi100.stdout: "state_deferred_queued_lat": { 2024-04-07T17:51:29.465 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 5880, 2024-04-07T17:51:29.465 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 6337522548440 2024-04-07T17:51:29.465 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.465 INFO:tasks.workunit.client.0.smithi100.stdout: "state_done_lat": { 2024-04-07T17:51:29.465 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 77647, 2024-04-07T17:51:29.466 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2768612828005 2024-04-07T17:51:29.466 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.466 INFO:tasks.workunit.client.0.smithi100.stdout: "state_finishing_lat": { 2024-04-07T17:51:29.466 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 77647, 2024-04-07T17:51:29.466 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 18125716 2024-04-07T17:51:29.466 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.466 INFO:tasks.workunit.client.0.smithi100.stdout: "state_io_done_lat": { 2024-04-07T17:51:29.466 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 77648, 2024-04-07T17:51:29.466 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 245246928 2024-04-07T17:51:29.466 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.466 INFO:tasks.workunit.client.0.smithi100.stdout: "state_kv_commiting_lat": { 2024-04-07T17:51:29.466 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 77648, 2024-04-07T17:51:29.466 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 45419960556 2024-04-07T17:51:29.466 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.467 INFO:tasks.workunit.client.0.smithi100.stdout: "state_kv_done_lat": { 2024-04-07T17:51:29.467 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 77648, 2024-04-07T17:51:29.467 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 60433024 2024-04-07T17:51:29.467 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.467 INFO:tasks.workunit.client.0.smithi100.stdout: "state_kv_queued_lat": { 2024-04-07T17:51:29.467 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 77648, 2024-04-07T17:51:29.467 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 21195606198 2024-04-07T17:51:29.467 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.467 INFO:tasks.workunit.client.0.smithi100.stdout: "state_prepare_lat": { 2024-04-07T17:51:29.467 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 77648, 2024-04-07T17:51:29.467 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 23621700730 2024-04-07T17:51:29.467 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.467 INFO:tasks.workunit.client.0.smithi100.stdout: "stored": { 2024-04-07T17:51:29.467 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5699682 2024-04-07T17:51:29.467 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.467 INFO:tasks.workunit.client.0.smithi100.stdout: "truncate_lat": { 2024-04-07T17:51:29.468 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 2411, 2024-04-07T17:51:29.468 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 23697978 2024-04-07T17:51:29.468 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.468 INFO:tasks.workunit.client.0.smithi100.stdout: "txc_commit_lat": { 2024-04-07T17:51:29.468 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 77648, 2024-04-07T17:51:29.468 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 98115535290 2024-04-07T17:51:29.468 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.468 INFO:tasks.workunit.client.0.smithi100.stdout: "txc_submit_lat": { 2024-04-07T17:51:29.468 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 77648, 2024-04-07T17:51:29.468 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 25610928464 2024-04-07T17:51:29.468 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.468 INFO:tasks.workunit.client.0.smithi100.stdout: "txc_throttle_lat": { 2024-04-07T17:51:29.468 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 77648, 2024-04-07T17:51:29.468 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 903590085 2024-04-07T17:51:29.468 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.469 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.469 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache": { 2024-04-07T17:51:29.469 INFO:tasks.workunit.client.0.smithi100.stdout: "cache_bytes": { 2024-04-07T17:51:29.469 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1444614092 2024-04-07T17:51:29.469 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.469 INFO:tasks.workunit.client.0.smithi100.stdout: "heap_bytes": { 2024-04-07T17:51:29.469 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 563281920 2024-04-07T17:51:29.469 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.469 INFO:tasks.workunit.client.0.smithi100.stdout: "mapped_bytes": { 2024-04-07T17:51:29.469 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 513974272 2024-04-07T17:51:29.469 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.469 INFO:tasks.workunit.client.0.smithi100.stdout: "target_bytes": { 2024-04-07T17:51:29.469 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2646965248 2024-04-07T17:51:29.469 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.469 INFO:tasks.workunit.client.0.smithi100.stdout: "unmapped_bytes": { 2024-04-07T17:51:29.469 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 49307648 2024-04-07T17:51:29.469 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.470 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.470 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:data": { 2024-04-07T17:51:29.470 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:29.470 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 134217728 2024-04-07T17:51:29.470 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.470 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:29.470 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.470 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.470 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:29.470 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.470 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.470 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:29.470 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 59173493 2024-04-07T17:51:29.470 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.471 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:29.471 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.471 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.471 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:29.471 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.471 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.471 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:29.471 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.471 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.471 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:29.471 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.471 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.471 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:29.471 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 8192 2024-04-07T17:51:29.471 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.472 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:29.472 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 77824 2024-04-07T17:51:29.472 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.472 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:29.472 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.472 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.472 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:29.472 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.472 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.472 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:29.472 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.472 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.472 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:29.472 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 74958219 2024-04-07T17:51:29.472 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.473 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.473 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:kv": { 2024-04-07T17:51:29.473 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:29.473 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 578813952 2024-04-07T17:51:29.473 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.473 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:29.473 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 67108864 2024-04-07T17:51:29.473 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.473 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:29.473 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.473 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.473 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:29.473 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 443801202 2024-04-07T17:51:29.473 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.474 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:29.474 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.474 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.474 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:29.474 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.474 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.474 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:29.474 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2384 2024-04-07T17:51:29.474 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.474 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:29.474 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.474 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.474 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:29.474 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 150592 2024-04-07T17:51:29.475 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.475 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:29.475 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 384 2024-04-07T17:51:29.475 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.475 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:29.475 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.475 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.475 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:29.475 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.475 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.475 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:29.475 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.475 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.475 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:29.475 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 67750526 2024-04-07T17:51:29.476 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.476 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.476 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:kv_onode": { 2024-04-07T17:51:29.476 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:29.476 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 184549376 2024-04-07T17:51:29.476 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.476 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:29.476 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 67108864 2024-04-07T17:51:29.476 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.476 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:29.476 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 10790880 2024-04-07T17:51:29.476 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.476 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:29.476 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 39448995 2024-04-07T17:51:29.476 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.477 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:29.477 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.477 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.477 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:29.477 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.477 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.477 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:29.477 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.477 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.477 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:29.477 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.477 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.477 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:29.477 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.477 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.478 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:29.478 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.478 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.478 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:29.478 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.478 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.478 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:29.478 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.478 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.478 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:29.478 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.478 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.478 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:29.478 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 67200637 2024-04-07T17:51:29.478 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.478 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.479 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:meta": { 2024-04-07T17:51:29.479 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:29.479 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 520093696 2024-04-07T17:51:29.479 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.479 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:29.479 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.479 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.479 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:29.479 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.479 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.479 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:29.479 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 443801202 2024-04-07T17:51:29.479 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.479 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:29.479 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.479 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.480 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:29.480 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.480 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.480 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:29.480 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.480 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.480 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:29.480 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.480 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.480 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:29.480 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 138605 2024-04-07T17:51:29.480 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.480 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:29.480 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1113867 2024-04-07T17:51:29.480 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.480 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:29.480 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.481 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.481 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:29.481 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.481 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.481 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:29.481 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.481 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.481 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:29.481 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 75040022 2024-04-07T17:51:29.481 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.481 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.481 INFO:tasks.workunit.client.0.smithi100.stdout: "objecter": { 2024-04-07T17:51:29.481 INFO:tasks.workunit.client.0.smithi100.stdout: "op_active": { 2024-04-07T17:51:29.481 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.481 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.481 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r": { 2024-04-07T17:51:29.481 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 91 2024-04-07T17:51:29.482 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.482 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rmw": { 2024-04-07T17:51:29.482 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.482 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.482 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w": { 2024-04-07T17:51:29.482 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.482 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.482 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.482 INFO:tasks.workunit.client.0.smithi100.stdout: "osd": { 2024-04-07T17:51:29.482 INFO:tasks.workunit.client.0.smithi100.stdout: "numpg": { 2024-04-07T17:51:29.482 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 40 2024-04-07T17:51:29.482 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.482 INFO:tasks.workunit.client.0.smithi100.stdout: "numpg_removing": { 2024-04-07T17:51:29.482 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.482 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.483 INFO:tasks.workunit.client.0.smithi100.stdout: "op": { 2024-04-07T17:51:29.483 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 58391 2024-04-07T17:51:29.483 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.483 INFO:tasks.workunit.client.0.smithi100.stdout: "op_before_queue_op_lat": { 2024-04-07T17:51:29.483 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 150392, 2024-04-07T17:51:29.483 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 13814312300 2024-04-07T17:51:29.483 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.483 INFO:tasks.workunit.client.0.smithi100.stdout: "op_delayed_degraded": { 2024-04-07T17:51:29.483 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5 2024-04-07T17:51:29.483 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.483 INFO:tasks.workunit.client.0.smithi100.stdout: "op_delayed_unreadable": { 2024-04-07T17:51:29.483 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.483 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.483 INFO:tasks.workunit.client.0.smithi100.stdout: "op_in_bytes": { 2024-04-07T17:51:29.483 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2120776034 2024-04-07T17:51:29.483 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.484 INFO:tasks.workunit.client.0.smithi100.stdout: "op_latency": { 2024-04-07T17:51:29.484 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 58391, 2024-04-07T17:51:29.484 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 187069135567 2024-04-07T17:51:29.484 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.484 INFO:tasks.workunit.client.0.smithi100.stdout: "op_out_bytes": { 2024-04-07T17:51:29.484 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3533359549 2024-04-07T17:51:29.484 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.484 INFO:tasks.workunit.client.0.smithi100.stdout: "op_prepare_latency": { 2024-04-07T17:51:29.484 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 59237, 2024-04-07T17:51:29.484 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 89091892647 2024-04-07T17:51:29.484 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.484 INFO:tasks.workunit.client.0.smithi100.stdout: "op_process_latency": { 2024-04-07T17:51:29.484 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 58391, 2024-04-07T17:51:29.484 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 156357104214 2024-04-07T17:51:29.484 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.485 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r": { 2024-04-07T17:51:29.485 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 29188 2024-04-07T17:51:29.485 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.485 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_latency": { 2024-04-07T17:51:29.485 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 29188, 2024-04-07T17:51:29.485 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 31668185766 2024-04-07T17:51:29.485 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.485 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_latency_out_bytes_histogram": { 2024-04-07T17:51:29.485 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.485 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.485 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_out_bytes": { 2024-04-07T17:51:29.485 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3533359549 2024-04-07T17:51:29.485 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.485 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_prepare_latency": { 2024-04-07T17:51:29.485 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 29608, 2024-04-07T17:51:29.486 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 13047146829 2024-04-07T17:51:29.486 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.486 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_process_latency": { 2024-04-07T17:51:29.486 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 29188, 2024-04-07T17:51:29.486 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 17316757778 2024-04-07T17:51:29.486 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.486 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw": { 2024-04-07T17:51:29.486 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 27288 2024-04-07T17:51:29.486 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.486 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_in_bytes": { 2024-04-07T17:51:29.486 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 535101355 2024-04-07T17:51:29.486 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.486 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_latency": { 2024-04-07T17:51:29.486 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 27288, 2024-04-07T17:51:29.486 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 125896770692 2024-04-07T17:51:29.487 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.487 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_latency_in_bytes_histogram": { 2024-04-07T17:51:29.487 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.487 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.487 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_latency_out_bytes_histogram": { 2024-04-07T17:51:29.487 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.487 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.487 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_out_bytes": { 2024-04-07T17:51:29.487 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.487 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.487 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_prepare_latency": { 2024-04-07T17:51:29.487 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 27539, 2024-04-07T17:51:29.487 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 69930450013 2024-04-07T17:51:29.487 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.487 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_process_latency": { 2024-04-07T17:51:29.488 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 27288, 2024-04-07T17:51:29.488 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 117911423955 2024-04-07T17:51:29.488 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.488 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w": { 2024-04-07T17:51:29.488 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1915 2024-04-07T17:51:29.488 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.488 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_in_bytes": { 2024-04-07T17:51:29.488 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1585674679 2024-04-07T17:51:29.488 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.488 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_latency": { 2024-04-07T17:51:29.488 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1915, 2024-04-07T17:51:29.488 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 29504179109 2024-04-07T17:51:29.488 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.488 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_latency_in_bytes_histogram": { 2024-04-07T17:51:29.488 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.489 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.489 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_prepare_latency": { 2024-04-07T17:51:29.489 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 2090, 2024-04-07T17:51:29.489 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 6114295805 2024-04-07T17:51:29.489 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.489 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_process_latency": { 2024-04-07T17:51:29.489 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1915, 2024-04-07T17:51:29.489 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 21128922481 2024-04-07T17:51:29.489 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.489 INFO:tasks.workunit.client.0.smithi100.stdout: "op_wip": { 2024-04-07T17:51:29.489 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.489 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.489 INFO:tasks.workunit.client.0.smithi100.stdout: "recovery_bytes": { 2024-04-07T17:51:29.489 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4194321 2024-04-07T17:51:29.489 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.490 INFO:tasks.workunit.client.0.smithi100.stdout: "recovery_ops": { 2024-04-07T17:51:29.490 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 38 2024-04-07T17:51:29.490 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.490 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_bytes": { 2024-04-07T17:51:29.490 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 95995035648 2024-04-07T17:51:29.490 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.490 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_bytes_used": { 2024-04-07T17:51:29.490 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 456179712 2024-04-07T17:51:29.490 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.490 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.490 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.490 INFO:tasks.workunit.client.0.smithi100.stdout: "osd.3": { 2024-04-07T17:51:29.490 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs": { 2024-04-07T17:51:29.490 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_db_max_lat": { 2024-04-07T17:51:29.491 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 29319 2024-04-07T17:51:29.491 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.491 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_slow_fallback": { 2024-04-07T17:51:29.491 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.491 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.491 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_slow_max_lat": { 2024-04-07T17:51:29.491 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.491 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.491 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_slow_size_fallback": { 2024-04-07T17:51:29.491 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.491 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.491 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit_db": { 2024-04-07T17:51:29.491 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 65536 2024-04-07T17:51:29.492 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.492 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit_slow": { 2024-04-07T17:51:29.492 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.492 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.492 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit_wal": { 2024-04-07T17:51:29.492 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.492 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.492 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_wal_max_lat": { 2024-04-07T17:51:29.492 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.492 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.492 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_written_slow": { 2024-04-07T17:51:29.492 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.492 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.492 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_written_sst": { 2024-04-07T17:51:29.492 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 6107136 2024-04-07T17:51:29.492 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.493 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_written_wal": { 2024-04-07T17:51:29.493 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 546639872 2024-04-07T17:51:29.493 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.493 INFO:tasks.workunit.client.0.smithi100.stdout: "compact_lat": { 2024-04-07T17:51:29.493 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 16, 2024-04-07T17:51:29.493 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5676850 2024-04-07T17:51:29.493 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.493 INFO:tasks.workunit.client.0.smithi100.stdout: "compact_lock_lat": { 2024-04-07T17:51:29.493 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 16, 2024-04-07T17:51:29.493 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1688739 2024-04-07T17:51:29.493 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.493 INFO:tasks.workunit.client.0.smithi100.stdout: "db_alloc_lat": { 2024-04-07T17:51:29.493 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 158, 2024-04-07T17:51:29.493 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1098247 2024-04-07T17:51:29.493 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.494 INFO:tasks.workunit.client.0.smithi100.stdout: "db_total_bytes": { 2024-04-07T17:51:29.494 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 95995027456 2024-04-07T17:51:29.494 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.494 INFO:tasks.workunit.client.0.smithi100.stdout: "db_used_bytes": { 2024-04-07T17:51:29.494 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 449314816 2024-04-07T17:51:29.494 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.494 INFO:tasks.workunit.client.0.smithi100.stdout: "flush_lat": { 2024-04-07T17:51:29.494 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 67796, 2024-04-07T17:51:29.494 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 12755738535 2024-04-07T17:51:29.494 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.494 INFO:tasks.workunit.client.0.smithi100.stdout: "fsync_lat": { 2024-04-07T17:51:29.494 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 67808, 2024-04-07T17:51:29.494 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 22453593571 2024-04-07T17:51:29.494 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.494 INFO:tasks.workunit.client.0.smithi100.stdout: "log_bytes": { 2024-04-07T17:51:29.495 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 11792384 2024-04-07T17:51:29.495 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.495 INFO:tasks.workunit.client.0.smithi100.stdout: "logged_bytes": { 2024-04-07T17:51:29.495 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 277786624 2024-04-07T17:51:29.495 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.495 INFO:tasks.workunit.client.0.smithi100.stdout: "max_bytes_db": { 2024-04-07T17:51:29.495 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 462094336 2024-04-07T17:51:29.495 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.495 INFO:tasks.workunit.client.0.smithi100.stdout: "max_bytes_slow": { 2024-04-07T17:51:29.495 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.495 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.495 INFO:tasks.workunit.client.0.smithi100.stdout: "max_bytes_wal": { 2024-04-07T17:51:29.495 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.495 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.495 INFO:tasks.workunit.client.0.smithi100.stdout: "num_files": { 2024-04-07T17:51:29.495 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 41 2024-04-07T17:51:29.496 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.496 INFO:tasks.workunit.client.0.smithi100.stdout: "read_bytes": { 2024-04-07T17:51:29.496 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5081425364 2024-04-07T17:51:29.496 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.496 INFO:tasks.workunit.client.0.smithi100.stdout: "read_count": { 2024-04-07T17:51:29.496 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 55136 2024-04-07T17:51:29.496 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.496 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes": { 2024-04-07T17:51:29.496 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4273045504 2024-04-07T17:51:29.496 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.496 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes_db": { 2024-04-07T17:51:29.496 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4273049600 2024-04-07T17:51:29.496 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.496 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes_slow": { 2024-04-07T17:51:29.496 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.497 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.497 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes_wal": { 2024-04-07T17:51:29.497 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.497 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.497 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_count": { 2024-04-07T17:51:29.497 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 47715 2024-04-07T17:51:29.497 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.497 INFO:tasks.workunit.client.0.smithi100.stdout: "read_lat": { 2024-04-07T17:51:29.497 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 55136, 2024-04-07T17:51:29.497 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2526773312 2024-04-07T17:51:29.497 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.497 INFO:tasks.workunit.client.0.smithi100.stdout: "read_prefetch_bytes": { 2024-04-07T17:51:29.497 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4957544695 2024-04-07T17:51:29.497 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.497 INFO:tasks.workunit.client.0.smithi100.stdout: "read_prefetch_count": { 2024-04-07T17:51:29.497 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 51339 2024-04-07T17:51:29.498 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.498 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_buffer_bytes": { 2024-04-07T17:51:29.498 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1547577 2024-04-07T17:51:29.498 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.498 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_buffer_count": { 2024-04-07T17:51:29.498 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 561 2024-04-07T17:51:29.498 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.498 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_bytes": { 2024-04-07T17:51:29.498 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 6055753 2024-04-07T17:51:29.498 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.498 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_count": { 2024-04-07T17:51:29.498 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 720 2024-04-07T17:51:29.498 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.498 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes": { 2024-04-07T17:51:29.499 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4508176 2024-04-07T17:51:29.499 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.499 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes_db": { 2024-04-07T17:51:29.499 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4508176 2024-04-07T17:51:29.499 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.499 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes_slow": { 2024-04-07T17:51:29.499 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.499 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.499 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes_wal": { 2024-04-07T17:51:29.499 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.499 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.499 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_count": { 2024-04-07T17:51:29.499 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 161 2024-04-07T17:51:29.499 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.499 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_lat": { 2024-04-07T17:51:29.500 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 720, 2024-04-07T17:51:29.500 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2153712 2024-04-07T17:51:29.500 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.500 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_alloc_lat": { 2024-04-07T17:51:29.500 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:29.500 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.500 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.500 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_total_bytes": { 2024-04-07T17:51:29.500 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.500 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.500 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_used_bytes": { 2024-04-07T17:51:29.500 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.500 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.500 INFO:tasks.workunit.client.0.smithi100.stdout: "truncate_lat": { 2024-04-07T17:51:29.500 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1, 2024-04-07T17:51:29.500 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 872 2024-04-07T17:51:29.500 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.501 INFO:tasks.workunit.client.0.smithi100.stdout: "unlink_lat": { 2024-04-07T17:51:29.501 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 10, 2024-04-07T17:51:29.501 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 15209 2024-04-07T17:51:29.501 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.501 INFO:tasks.workunit.client.0.smithi100.stdout: "wal_alloc_lat": { 2024-04-07T17:51:29.501 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:29.501 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.501 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.501 INFO:tasks.workunit.client.0.smithi100.stdout: "wal_total_bytes": { 2024-04-07T17:51:29.501 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.501 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.501 INFO:tasks.workunit.client.0.smithi100.stdout: "wal_used_bytes": { 2024-04-07T17:51:29.501 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.501 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.501 INFO:tasks.workunit.client.0.smithi100.stdout: "write_bytes": { 2024-04-07T17:51:29.501 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 832966656 2024-04-07T17:51:29.502 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.502 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.502 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore": { 2024-04-07T17:51:29.502 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit": { 2024-04-07T17:51:29.502 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4096 2024-04-07T17:51:29.502 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.502 INFO:tasks.workunit.client.0.smithi100.stdout: "allocated": { 2024-04-07T17:51:29.502 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 9826304 2024-04-07T17:51:29.502 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.502 INFO:tasks.workunit.client.0.smithi100.stdout: "allocator_lat": { 2024-04-07T17:51:29.502 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 19863, 2024-04-07T17:51:29.502 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 103164491 2024-04-07T17:51:29.502 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.502 INFO:tasks.workunit.client.0.smithi100.stdout: "clist_lat": { 2024-04-07T17:51:29.502 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 3584, 2024-04-07T17:51:29.502 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 10742765419 2024-04-07T17:51:29.503 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.503 INFO:tasks.workunit.client.0.smithi100.stdout: "compress_lat": { 2024-04-07T17:51:29.503 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:29.503 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.503 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.503 INFO:tasks.workunit.client.0.smithi100.stdout: "compressed": { 2024-04-07T17:51:29.503 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.503 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.503 INFO:tasks.workunit.client.0.smithi100.stdout: "compressed_allocated": { 2024-04-07T17:51:29.503 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.503 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.503 INFO:tasks.workunit.client.0.smithi100.stdout: "compressed_original": { 2024-04-07T17:51:29.503 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.503 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.503 INFO:tasks.workunit.client.0.smithi100.stdout: "csum_lat": { 2024-04-07T17:51:29.504 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 13277, 2024-04-07T17:51:29.504 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 158291808 2024-04-07T17:51:29.504 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.504 INFO:tasks.workunit.client.0.smithi100.stdout: "decompress_lat": { 2024-04-07T17:51:29.504 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:29.504 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.504 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.504 INFO:tasks.workunit.client.0.smithi100.stdout: "fragmentation_micros": { 2024-04-07T17:51:29.504 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.504 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.504 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_commit_lat": { 2024-04-07T17:51:29.504 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 67759, 2024-04-07T17:51:29.504 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 28639225606 2024-04-07T17:51:29.504 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.504 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_final_lat": { 2024-04-07T17:51:29.504 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 66833, 2024-04-07T17:51:29.504 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3952581972 2024-04-07T17:51:29.505 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.505 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_flush_lat": { 2024-04-07T17:51:29.505 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 67759, 2024-04-07T17:51:29.505 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 98642276 2024-04-07T17:51:29.505 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.505 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_sync_lat": { 2024-04-07T17:51:29.505 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 67759, 2024-04-07T17:51:29.505 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 28737867882 2024-04-07T17:51:29.505 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.505 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_get_keys_lat": { 2024-04-07T17:51:29.505 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:29.505 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.505 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.505 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_get_values_lat": { 2024-04-07T17:51:29.505 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 26953, 2024-04-07T17:51:29.505 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1128418885 2024-04-07T17:51:29.506 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.506 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_lower_bound_lat": { 2024-04-07T17:51:29.506 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1271, 2024-04-07T17:51:29.506 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 7079072 2024-04-07T17:51:29.506 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.506 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_next_lat": { 2024-04-07T17:51:29.506 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 8457, 2024-04-07T17:51:29.506 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 12008607 2024-04-07T17:51:29.506 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.506 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_seek_to_first_lat": { 2024-04-07T17:51:29.506 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 40, 2024-04-07T17:51:29.506 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 80474 2024-04-07T17:51:29.506 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.506 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_upper_bound_lat": { 2024-04-07T17:51:29.507 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 11692, 2024-04-07T17:51:29.507 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4518542429 2024-04-07T17:51:29.507 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.507 INFO:tasks.workunit.client.0.smithi100.stdout: "onode_hits": { 2024-04-07T17:51:29.507 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 208992 2024-04-07T17:51:29.507 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.507 INFO:tasks.workunit.client.0.smithi100.stdout: "onode_misses": { 2024-04-07T17:51:29.507 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 18294 2024-04-07T17:51:29.507 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.507 INFO:tasks.workunit.client.0.smithi100.stdout: "read_lat": { 2024-04-07T17:51:29.507 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 30350, 2024-04-07T17:51:29.507 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2036962015 2024-04-07T17:51:29.507 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.507 INFO:tasks.workunit.client.0.smithi100.stdout: "read_onode_meta_lat": { 2024-04-07T17:51:29.507 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 80371, 2024-04-07T17:51:29.507 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 45758452 2024-04-07T17:51:29.508 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.508 INFO:tasks.workunit.client.0.smithi100.stdout: "read_wait_aio_lat": { 2024-04-07T17:51:29.508 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 50021, 2024-04-07T17:51:29.508 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1142513586 2024-04-07T17:51:29.508 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.508 INFO:tasks.workunit.client.0.smithi100.stdout: "reads_with_retries": { 2024-04-07T17:51:29.508 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.508 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.508 INFO:tasks.workunit.client.0.smithi100.stdout: "remove_lat": { 2024-04-07T17:51:29.508 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 12382, 2024-04-07T17:51:29.508 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 706656351 2024-04-07T17:51:29.508 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.508 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_aio_wait_count": { 2024-04-07T17:51:29.508 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.508 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.509 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_committed_kv_count": { 2024-04-07T17:51:29.509 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.509 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.509 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_read_onode_meta_count": { 2024-04-07T17:51:29.509 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.509 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.509 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_read_wait_aio_count": { 2024-04-07T17:51:29.509 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.509 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.509 INFO:tasks.workunit.client.0.smithi100.stdout: "state_aio_wait_lat": { 2024-04-07T17:51:29.509 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 77291, 2024-04-07T17:51:29.509 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 12556920270 2024-04-07T17:51:29.509 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.509 INFO:tasks.workunit.client.0.smithi100.stdout: "state_deferred_aio_wait_lat": { 2024-04-07T17:51:29.509 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 17849, 2024-04-07T17:51:29.509 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1593523661 2024-04-07T17:51:29.510 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.510 INFO:tasks.workunit.client.0.smithi100.stdout: "state_deferred_cleanup_lat": { 2024-04-07T17:51:29.510 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 17848, 2024-04-07T17:51:29.510 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3943299065015 2024-04-07T17:51:29.510 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.510 INFO:tasks.workunit.client.0.smithi100.stdout: "state_deferred_queued_lat": { 2024-04-07T17:51:29.510 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 17849, 2024-04-07T17:51:29.510 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 6510467867246 2024-04-07T17:51:29.510 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.510 INFO:tasks.workunit.client.0.smithi100.stdout: "state_done_lat": { 2024-04-07T17:51:29.510 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 77290, 2024-04-07T17:51:29.510 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2750364291611 2024-04-07T17:51:29.510 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.510 INFO:tasks.workunit.client.0.smithi100.stdout: "state_finishing_lat": { 2024-04-07T17:51:29.511 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 77290, 2024-04-07T17:51:29.511 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 16788531 2024-04-07T17:51:29.511 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.511 INFO:tasks.workunit.client.0.smithi100.stdout: "state_io_done_lat": { 2024-04-07T17:51:29.511 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 77291, 2024-04-07T17:51:29.511 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 461265229 2024-04-07T17:51:29.511 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.511 INFO:tasks.workunit.client.0.smithi100.stdout: "state_kv_commiting_lat": { 2024-04-07T17:51:29.511 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 77291, 2024-04-07T17:51:29.511 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 50377334346 2024-04-07T17:51:29.511 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.511 INFO:tasks.workunit.client.0.smithi100.stdout: "state_kv_done_lat": { 2024-04-07T17:51:29.511 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 77291, 2024-04-07T17:51:29.511 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 62405628 2024-04-07T17:51:29.511 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.512 INFO:tasks.workunit.client.0.smithi100.stdout: "state_kv_queued_lat": { 2024-04-07T17:51:29.512 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 77291, 2024-04-07T17:51:29.512 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 24119810266 2024-04-07T17:51:29.512 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.512 INFO:tasks.workunit.client.0.smithi100.stdout: "state_prepare_lat": { 2024-04-07T17:51:29.512 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 77291, 2024-04-07T17:51:29.512 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 26664283551 2024-04-07T17:51:29.512 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.512 INFO:tasks.workunit.client.0.smithi100.stdout: "stored": { 2024-04-07T17:51:29.512 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5705610 2024-04-07T17:51:29.512 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.512 INFO:tasks.workunit.client.0.smithi100.stdout: "truncate_lat": { 2024-04-07T17:51:29.512 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 2398, 2024-04-07T17:51:29.512 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 27539717 2024-04-07T17:51:29.512 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.512 INFO:tasks.workunit.client.0.smithi100.stdout: "txc_commit_lat": { 2024-04-07T17:51:29.513 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 77291, 2024-04-07T17:51:29.513 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 114204848603 2024-04-07T17:51:29.513 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.513 INFO:tasks.workunit.client.0.smithi100.stdout: "txc_submit_lat": { 2024-04-07T17:51:29.513 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 77291, 2024-04-07T17:51:29.513 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 28995862182 2024-04-07T17:51:29.513 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.513 INFO:tasks.workunit.client.0.smithi100.stdout: "txc_throttle_lat": { 2024-04-07T17:51:29.513 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 77291, 2024-04-07T17:51:29.513 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1079109304 2024-04-07T17:51:29.513 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.513 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.513 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache": { 2024-04-07T17:51:29.513 INFO:tasks.workunit.client.0.smithi100.stdout: "cache_bytes": { 2024-04-07T17:51:29.513 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1444614092 2024-04-07T17:51:29.513 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.514 INFO:tasks.workunit.client.0.smithi100.stdout: "heap_bytes": { 2024-04-07T17:51:29.514 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 503775232 2024-04-07T17:51:29.514 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.514 INFO:tasks.workunit.client.0.smithi100.stdout: "mapped_bytes": { 2024-04-07T17:51:29.514 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 470630400 2024-04-07T17:51:29.514 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.514 INFO:tasks.workunit.client.0.smithi100.stdout: "target_bytes": { 2024-04-07T17:51:29.514 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2646965248 2024-04-07T17:51:29.514 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.514 INFO:tasks.workunit.client.0.smithi100.stdout: "unmapped_bytes": { 2024-04-07T17:51:29.514 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 33144832 2024-04-07T17:51:29.514 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.514 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.514 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:data": { 2024-04-07T17:51:29.514 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:29.515 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 134217728 2024-04-07T17:51:29.515 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.515 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:29.515 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.515 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.515 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:29.515 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.515 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.515 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:29.515 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 59559460 2024-04-07T17:51:29.515 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.515 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:29.515 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.515 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.515 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:29.516 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.516 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.516 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:29.516 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.516 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.516 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:29.516 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.516 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.516 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:29.516 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 8192 2024-04-07T17:51:29.516 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.516 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:29.516 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 102400 2024-04-07T17:51:29.516 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.517 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:29.517 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.517 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.517 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:29.517 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.517 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.517 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:29.517 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.517 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.517 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:29.517 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 74547676 2024-04-07T17:51:29.517 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.517 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.517 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:kv": { 2024-04-07T17:51:29.517 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:29.518 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 587202560 2024-04-07T17:51:29.518 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.518 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:29.518 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 67108864 2024-04-07T17:51:29.518 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.518 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:29.518 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.518 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.518 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:29.518 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 446695953 2024-04-07T17:51:29.518 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.518 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:29.518 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.518 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.519 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:29.519 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1776 2024-04-07T17:51:29.519 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.519 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:29.519 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.519 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.519 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:29.519 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.519 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.519 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:29.519 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 110304 2024-04-07T17:51:29.519 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.519 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:29.519 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.519 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.519 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:29.520 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.520 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.520 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:29.520 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.520 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.520 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:29.520 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.520 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.520 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:29.520 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 73285663 2024-04-07T17:51:29.520 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.520 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.520 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:kv_onode": { 2024-04-07T17:51:29.520 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:29.520 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 184549376 2024-04-07T17:51:29.520 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.521 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:29.521 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 67108864 2024-04-07T17:51:29.521 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.521 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:29.521 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5620784 2024-04-07T17:51:29.521 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.521 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:29.521 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 39706307 2024-04-07T17:51:29.521 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.521 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:29.521 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.521 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.521 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:29.521 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.521 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.521 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:29.521 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.522 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.522 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:29.522 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.522 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.522 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:29.522 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.522 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.522 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:29.522 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.522 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.522 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:29.522 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.522 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.522 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:29.522 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.523 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.523 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:29.523 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.523 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.523 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:29.523 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 72113421 2024-04-07T17:51:29.523 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.523 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.523 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:meta": { 2024-04-07T17:51:29.523 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:29.523 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 520093696 2024-04-07T17:51:29.523 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.523 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:29.523 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.523 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.523 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:29.523 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.524 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.524 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:29.524 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 446695953 2024-04-07T17:51:29.524 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.524 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:29.524 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.524 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.524 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:29.524 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.524 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.524 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:29.524 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.524 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.524 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:29.524 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.525 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.525 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:29.525 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 144969 2024-04-07T17:51:29.525 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.525 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:29.525 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1112438 2024-04-07T17:51:29.525 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.525 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:29.525 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.525 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.525 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:29.525 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.525 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.525 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:29.526 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.526 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.526 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:29.526 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 72140336 2024-04-07T17:51:29.526 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.526 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.526 INFO:tasks.workunit.client.0.smithi100.stdout: "objecter": { 2024-04-07T17:51:29.526 INFO:tasks.workunit.client.0.smithi100.stdout: "op_active": { 2024-04-07T17:51:29.526 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.526 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.526 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r": { 2024-04-07T17:51:29.526 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 18 2024-04-07T17:51:29.526 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.527 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rmw": { 2024-04-07T17:51:29.527 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.527 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.527 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w": { 2024-04-07T17:51:29.527 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.527 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.527 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.527 INFO:tasks.workunit.client.0.smithi100.stdout: "osd": { 2024-04-07T17:51:29.527 INFO:tasks.workunit.client.0.smithi100.stdout: "numpg": { 2024-04-07T17:51:29.527 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 47 2024-04-07T17:51:29.527 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.527 INFO:tasks.workunit.client.0.smithi100.stdout: "numpg_removing": { 2024-04-07T17:51:29.527 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.527 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.527 INFO:tasks.workunit.client.0.smithi100.stdout: "op": { 2024-04-07T17:51:29.528 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 56964 2024-04-07T17:51:29.528 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.528 INFO:tasks.workunit.client.0.smithi100.stdout: "op_before_queue_op_lat": { 2024-04-07T17:51:29.528 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 133884, 2024-04-07T17:51:29.528 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 12292525237 2024-04-07T17:51:29.528 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.528 INFO:tasks.workunit.client.0.smithi100.stdout: "op_delayed_degraded": { 2024-04-07T17:51:29.528 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.528 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.528 INFO:tasks.workunit.client.0.smithi100.stdout: "op_delayed_unreadable": { 2024-04-07T17:51:29.528 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.528 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.528 INFO:tasks.workunit.client.0.smithi100.stdout: "op_in_bytes": { 2024-04-07T17:51:29.528 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1485365521 2024-04-07T17:51:29.528 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.529 INFO:tasks.workunit.client.0.smithi100.stdout: "op_latency": { 2024-04-07T17:51:29.529 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 56964, 2024-04-07T17:51:29.529 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 394646376159 2024-04-07T17:51:29.529 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.529 INFO:tasks.workunit.client.0.smithi100.stdout: "op_out_bytes": { 2024-04-07T17:51:29.529 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1430791176 2024-04-07T17:51:29.529 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.529 INFO:tasks.workunit.client.0.smithi100.stdout: "op_prepare_latency": { 2024-04-07T17:51:29.529 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 59216, 2024-04-07T17:51:29.529 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 48205385292 2024-04-07T17:51:29.529 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.529 INFO:tasks.workunit.client.0.smithi100.stdout: "op_process_latency": { 2024-04-07T17:51:29.529 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 56964, 2024-04-07T17:51:29.529 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 169980516116 2024-04-07T17:51:29.529 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.529 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r": { 2024-04-07T17:51:29.530 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 39255 2024-04-07T17:51:29.530 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.530 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_latency": { 2024-04-07T17:51:29.530 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 39255, 2024-04-07T17:51:29.530 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 64075850096 2024-04-07T17:51:29.530 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.530 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_latency_out_bytes_histogram": { 2024-04-07T17:51:29.530 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.530 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.530 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_out_bytes": { 2024-04-07T17:51:29.530 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1430791176 2024-04-07T17:51:29.530 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.530 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_prepare_latency": { 2024-04-07T17:51:29.531 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 39480, 2024-04-07T17:51:29.531 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 16088819423 2024-04-07T17:51:29.531 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.531 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_process_latency": { 2024-04-07T17:51:29.531 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 39255, 2024-04-07T17:51:29.531 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 13918274599 2024-04-07T17:51:29.531 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.531 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw": { 2024-04-07T17:51:29.531 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 13767 2024-04-07T17:51:29.531 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.531 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_in_bytes": { 2024-04-07T17:51:29.531 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 309949184 2024-04-07T17:51:29.531 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.531 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_latency": { 2024-04-07T17:51:29.531 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 13767, 2024-04-07T17:51:29.531 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 275992837370 2024-04-07T17:51:29.531 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.532 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_latency_in_bytes_histogram": { 2024-04-07T17:51:29.532 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.532 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.532 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_latency_out_bytes_histogram": { 2024-04-07T17:51:29.532 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.532 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.532 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_out_bytes": { 2024-04-07T17:51:29.532 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.532 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.532 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_prepare_latency": { 2024-04-07T17:51:29.532 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 15694, 2024-04-07T17:51:29.532 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 23263942693 2024-04-07T17:51:29.532 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.532 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_process_latency": { 2024-04-07T17:51:29.532 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 13767, 2024-04-07T17:51:29.532 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 118411226197 2024-04-07T17:51:29.533 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.533 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w": { 2024-04-07T17:51:29.533 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3942 2024-04-07T17:51:29.533 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.533 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_in_bytes": { 2024-04-07T17:51:29.533 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1175416337 2024-04-07T17:51:29.533 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.533 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_latency": { 2024-04-07T17:51:29.533 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 3942, 2024-04-07T17:51:29.533 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 54577688693 2024-04-07T17:51:29.533 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.533 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_latency_in_bytes_histogram": { 2024-04-07T17:51:29.533 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.533 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.533 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_prepare_latency": { 2024-04-07T17:51:29.533 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 4042, 2024-04-07T17:51:29.533 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 8852623176 2024-04-07T17:51:29.534 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.534 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_process_latency": { 2024-04-07T17:51:29.534 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 3942, 2024-04-07T17:51:29.534 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 37651015320 2024-04-07T17:51:29.534 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.534 INFO:tasks.workunit.client.0.smithi100.stdout: "op_wip": { 2024-04-07T17:51:29.534 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.534 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.534 INFO:tasks.workunit.client.0.smithi100.stdout: "recovery_bytes": { 2024-04-07T17:51:29.534 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 35 2024-04-07T17:51:29.534 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.534 INFO:tasks.workunit.client.0.smithi100.stdout: "recovery_ops": { 2024-04-07T17:51:29.534 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3 2024-04-07T17:51:29.534 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.534 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_bytes": { 2024-04-07T17:51:29.534 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 95995035648 2024-04-07T17:51:29.535 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.535 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_bytes_used": { 2024-04-07T17:51:29.535 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 459153408 2024-04-07T17:51:29.535 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.535 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.535 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.535 INFO:tasks.workunit.client.0.smithi100.stdout: "osd.4": { 2024-04-07T17:51:29.535 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs": { 2024-04-07T17:51:29.535 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_db_max_lat": { 2024-04-07T17:51:29.535 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 53233 2024-04-07T17:51:29.535 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.535 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_slow_fallback": { 2024-04-07T17:51:29.535 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.535 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.535 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_slow_max_lat": { 2024-04-07T17:51:29.535 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.536 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.536 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_slow_size_fallback": { 2024-04-07T17:51:29.536 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.536 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.536 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit_db": { 2024-04-07T17:51:29.536 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 65536 2024-04-07T17:51:29.536 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.536 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit_slow": { 2024-04-07T17:51:29.536 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.536 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.536 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit_wal": { 2024-04-07T17:51:29.536 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.536 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.536 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_wal_max_lat": { 2024-04-07T17:51:29.536 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.537 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.537 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_written_slow": { 2024-04-07T17:51:29.537 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.537 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.537 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_written_sst": { 2024-04-07T17:51:29.537 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 14217216 2024-04-07T17:51:29.537 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.537 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_written_wal": { 2024-04-07T17:51:29.537 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 588619776 2024-04-07T17:51:29.537 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.537 INFO:tasks.workunit.client.0.smithi100.stdout: "compact_lat": { 2024-04-07T17:51:29.537 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 14, 2024-04-07T17:51:29.537 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5250958 2024-04-07T17:51:29.537 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.537 INFO:tasks.workunit.client.0.smithi100.stdout: "compact_lock_lat": { 2024-04-07T17:51:29.537 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 14, 2024-04-07T17:51:29.537 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1869555 2024-04-07T17:51:29.537 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.538 INFO:tasks.workunit.client.0.smithi100.stdout: "db_alloc_lat": { 2024-04-07T17:51:29.538 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 167, 2024-04-07T17:51:29.538 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1195870 2024-04-07T17:51:29.538 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.538 INFO:tasks.workunit.client.0.smithi100.stdout: "db_total_bytes": { 2024-04-07T17:51:29.538 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 95995027456 2024-04-07T17:51:29.538 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.538 INFO:tasks.workunit.client.0.smithi100.stdout: "db_used_bytes": { 2024-04-07T17:51:29.538 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 572653568 2024-04-07T17:51:29.538 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.538 INFO:tasks.workunit.client.0.smithi100.stdout: "flush_lat": { 2024-04-07T17:51:29.538 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 60244, 2024-04-07T17:51:29.538 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 12173027260 2024-04-07T17:51:29.538 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.538 INFO:tasks.workunit.client.0.smithi100.stdout: "fsync_lat": { 2024-04-07T17:51:29.539 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 60248, 2024-04-07T17:51:29.539 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 21497842884 2024-04-07T17:51:29.539 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.539 INFO:tasks.workunit.client.0.smithi100.stdout: "log_bytes": { 2024-04-07T17:51:29.539 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 14090240 2024-04-07T17:51:29.539 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.539 INFO:tasks.workunit.client.0.smithi100.stdout: "logged_bytes": { 2024-04-07T17:51:29.539 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 246812672 2024-04-07T17:51:29.539 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.539 INFO:tasks.workunit.client.0.smithi100.stdout: "max_bytes_db": { 2024-04-07T17:51:29.539 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 568393728 2024-04-07T17:51:29.539 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.539 INFO:tasks.workunit.client.0.smithi100.stdout: "max_bytes_slow": { 2024-04-07T17:51:29.539 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.539 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.539 INFO:tasks.workunit.client.0.smithi100.stdout: "max_bytes_wal": { 2024-04-07T17:51:29.539 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.540 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.540 INFO:tasks.workunit.client.0.smithi100.stdout: "num_files": { 2024-04-07T17:51:29.540 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 50 2024-04-07T17:51:29.540 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.540 INFO:tasks.workunit.client.0.smithi100.stdout: "read_bytes": { 2024-04-07T17:51:29.540 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 10796585872 2024-04-07T17:51:29.540 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.540 INFO:tasks.workunit.client.0.smithi100.stdout: "read_count": { 2024-04-07T17:51:29.540 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 87890 2024-04-07T17:51:29.540 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.540 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes": { 2024-04-07T17:51:29.540 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 9426481152 2024-04-07T17:51:29.540 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.540 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes_db": { 2024-04-07T17:51:29.540 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 9426485248 2024-04-07T17:51:29.540 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.541 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes_slow": { 2024-04-07T17:51:29.541 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.541 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.541 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes_wal": { 2024-04-07T17:51:29.541 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.541 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.541 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_count": { 2024-04-07T17:51:29.541 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 80816 2024-04-07T17:51:29.541 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.541 INFO:tasks.workunit.client.0.smithi100.stdout: "read_lat": { 2024-04-07T17:51:29.541 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 87890, 2024-04-07T17:51:29.541 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5954312472 2024-04-07T17:51:29.541 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.541 INFO:tasks.workunit.client.0.smithi100.stdout: "read_prefetch_bytes": { 2024-04-07T17:51:29.541 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 10604152323 2024-04-07T17:51:29.541 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.541 INFO:tasks.workunit.client.0.smithi100.stdout: "read_prefetch_count": { 2024-04-07T17:51:29.542 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 82000 2024-04-07T17:51:29.542 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.542 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_buffer_bytes": { 2024-04-07T17:51:29.542 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4778070 2024-04-07T17:51:29.542 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.542 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_buffer_count": { 2024-04-07T17:51:29.542 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 805 2024-04-07T17:51:29.542 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.542 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_bytes": { 2024-04-07T17:51:29.542 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 14103486 2024-04-07T17:51:29.542 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.542 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_count": { 2024-04-07T17:51:29.542 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1149 2024-04-07T17:51:29.542 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.542 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes": { 2024-04-07T17:51:29.542 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 9325416 2024-04-07T17:51:29.543 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.543 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes_db": { 2024-04-07T17:51:29.543 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 9325416 2024-04-07T17:51:29.543 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.543 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes_slow": { 2024-04-07T17:51:29.543 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.543 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.543 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes_wal": { 2024-04-07T17:51:29.543 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.543 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.543 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_count": { 2024-04-07T17:51:29.543 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 361 2024-04-07T17:51:29.543 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.543 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_lat": { 2024-04-07T17:51:29.543 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1149, 2024-04-07T17:51:29.544 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4714569 2024-04-07T17:51:29.544 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.544 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_alloc_lat": { 2024-04-07T17:51:29.544 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:29.544 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.544 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.544 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_total_bytes": { 2024-04-07T17:51:29.544 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.544 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.544 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_used_bytes": { 2024-04-07T17:51:29.544 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.544 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.544 INFO:tasks.workunit.client.0.smithi100.stdout: "truncate_lat": { 2024-04-07T17:51:29.544 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1, 2024-04-07T17:51:29.544 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 731 2024-04-07T17:51:29.544 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.545 INFO:tasks.workunit.client.0.smithi100.stdout: "unlink_lat": { 2024-04-07T17:51:29.545 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 14, 2024-04-07T17:51:29.545 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 19309 2024-04-07T17:51:29.545 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.545 INFO:tasks.workunit.client.0.smithi100.stdout: "wal_alloc_lat": { 2024-04-07T17:51:29.545 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:29.545 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.545 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.545 INFO:tasks.workunit.client.0.smithi100.stdout: "wal_total_bytes": { 2024-04-07T17:51:29.545 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.545 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.545 INFO:tasks.workunit.client.0.smithi100.stdout: "wal_used_bytes": { 2024-04-07T17:51:29.545 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.545 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.545 INFO:tasks.workunit.client.0.smithi100.stdout: "write_bytes": { 2024-04-07T17:51:29.546 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 851795968 2024-04-07T17:51:29.546 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.546 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.546 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore": { 2024-04-07T17:51:29.546 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit": { 2024-04-07T17:51:29.546 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4096 2024-04-07T17:51:29.546 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.546 INFO:tasks.workunit.client.0.smithi100.stdout: "allocated": { 2024-04-07T17:51:29.546 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 14016512 2024-04-07T17:51:29.546 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.546 INFO:tasks.workunit.client.0.smithi100.stdout: "allocator_lat": { 2024-04-07T17:51:29.546 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 20261, 2024-04-07T17:51:29.546 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 99191054 2024-04-07T17:51:29.546 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.546 INFO:tasks.workunit.client.0.smithi100.stdout: "clist_lat": { 2024-04-07T17:51:29.547 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 3532, 2024-04-07T17:51:29.547 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 14554851866 2024-04-07T17:51:29.547 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.547 INFO:tasks.workunit.client.0.smithi100.stdout: "compress_lat": { 2024-04-07T17:51:29.547 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:29.547 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.547 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.547 INFO:tasks.workunit.client.0.smithi100.stdout: "compressed": { 2024-04-07T17:51:29.547 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.547 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.547 INFO:tasks.workunit.client.0.smithi100.stdout: "compressed_allocated": { 2024-04-07T17:51:29.547 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.547 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.547 INFO:tasks.workunit.client.0.smithi100.stdout: "compressed_original": { 2024-04-07T17:51:29.547 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.548 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.548 INFO:tasks.workunit.client.0.smithi100.stdout: "csum_lat": { 2024-04-07T17:51:29.548 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 12243, 2024-04-07T17:51:29.548 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 157875076 2024-04-07T17:51:29.548 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.548 INFO:tasks.workunit.client.0.smithi100.stdout: "decompress_lat": { 2024-04-07T17:51:29.548 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:29.548 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.548 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.548 INFO:tasks.workunit.client.0.smithi100.stdout: "fragmentation_micros": { 2024-04-07T17:51:29.548 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.548 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.548 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_commit_lat": { 2024-04-07T17:51:29.548 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 60186, 2024-04-07T17:51:29.549 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 27783488280 2024-04-07T17:51:29.549 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.549 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_final_lat": { 2024-04-07T17:51:29.549 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 59468, 2024-04-07T17:51:29.549 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3239794550 2024-04-07T17:51:29.549 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.549 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_flush_lat": { 2024-04-07T17:51:29.549 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 60186, 2024-04-07T17:51:29.549 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 106527146 2024-04-07T17:51:29.549 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.549 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_sync_lat": { 2024-04-07T17:51:29.549 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 60186, 2024-04-07T17:51:29.549 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 27890015426 2024-04-07T17:51:29.549 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.549 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_get_keys_lat": { 2024-04-07T17:51:29.549 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:29.550 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.550 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.550 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_get_values_lat": { 2024-04-07T17:51:29.550 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 22645, 2024-04-07T17:51:29.550 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1026880183 2024-04-07T17:51:29.550 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.550 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_lower_bound_lat": { 2024-04-07T17:51:29.550 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 697, 2024-04-07T17:51:29.550 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4156713 2024-04-07T17:51:29.550 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.550 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_next_lat": { 2024-04-07T17:51:29.550 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 13851, 2024-04-07T17:51:29.550 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 16294854 2024-04-07T17:51:29.550 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.551 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_seek_to_first_lat": { 2024-04-07T17:51:29.551 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 47, 2024-04-07T17:51:29.551 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 58460 2024-04-07T17:51:29.551 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.551 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_upper_bound_lat": { 2024-04-07T17:51:29.551 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 14286, 2024-04-07T17:51:29.551 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 6817372199 2024-04-07T17:51:29.551 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.551 INFO:tasks.workunit.client.0.smithi100.stdout: "onode_hits": { 2024-04-07T17:51:29.551 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 175006 2024-04-07T17:51:29.551 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.551 INFO:tasks.workunit.client.0.smithi100.stdout: "onode_misses": { 2024-04-07T17:51:29.551 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 19274 2024-04-07T17:51:29.551 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.551 INFO:tasks.workunit.client.0.smithi100.stdout: "read_lat": { 2024-04-07T17:51:29.552 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 8345, 2024-04-07T17:51:29.552 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1757977707 2024-04-07T17:51:29.552 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.552 INFO:tasks.workunit.client.0.smithi100.stdout: "read_onode_meta_lat": { 2024-04-07T17:51:29.552 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 35241, 2024-04-07T17:51:29.552 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 18704077 2024-04-07T17:51:29.552 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.552 INFO:tasks.workunit.client.0.smithi100.stdout: "read_wait_aio_lat": { 2024-04-07T17:51:29.552 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 26896, 2024-04-07T17:51:29.552 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1180234012 2024-04-07T17:51:29.552 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.552 INFO:tasks.workunit.client.0.smithi100.stdout: "reads_with_retries": { 2024-04-07T17:51:29.552 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.552 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.552 INFO:tasks.workunit.client.0.smithi100.stdout: "remove_lat": { 2024-04-07T17:51:29.552 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 13430, 2024-04-07T17:51:29.553 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 730792101 2024-04-07T17:51:29.553 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.553 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_aio_wait_count": { 2024-04-07T17:51:29.553 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.553 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.553 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_committed_kv_count": { 2024-04-07T17:51:29.553 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.553 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.553 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_read_onode_meta_count": { 2024-04-07T17:51:29.553 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.553 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.553 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_read_wait_aio_count": { 2024-04-07T17:51:29.553 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.553 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.553 INFO:tasks.workunit.client.0.smithi100.stdout: "state_aio_wait_lat": { 2024-04-07T17:51:29.554 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 69678, 2024-04-07T17:51:29.554 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 8516743959 2024-04-07T17:51:29.554 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.554 INFO:tasks.workunit.client.0.smithi100.stdout: "state_deferred_aio_wait_lat": { 2024-04-07T17:51:29.554 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 18006, 2024-04-07T17:51:29.554 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1525120051 2024-04-07T17:51:29.554 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.554 INFO:tasks.workunit.client.0.smithi100.stdout: "state_deferred_cleanup_lat": { 2024-04-07T17:51:29.554 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 18005, 2024-04-07T17:51:29.554 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4347507639061 2024-04-07T17:51:29.554 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.554 INFO:tasks.workunit.client.0.smithi100.stdout: "state_deferred_queued_lat": { 2024-04-07T17:51:29.554 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 18006, 2024-04-07T17:51:29.554 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 6594884467617 2024-04-07T17:51:29.554 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.555 INFO:tasks.workunit.client.0.smithi100.stdout: "state_done_lat": { 2024-04-07T17:51:29.555 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 69677, 2024-04-07T17:51:29.555 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3006179254631 2024-04-07T17:51:29.555 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.555 INFO:tasks.workunit.client.0.smithi100.stdout: "state_finishing_lat": { 2024-04-07T17:51:29.555 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 69677, 2024-04-07T17:51:29.555 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 16535015 2024-04-07T17:51:29.555 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.555 INFO:tasks.workunit.client.0.smithi100.stdout: "state_io_done_lat": { 2024-04-07T17:51:29.555 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 69678, 2024-04-07T17:51:29.555 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 283438924 2024-04-07T17:51:29.555 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.555 INFO:tasks.workunit.client.0.smithi100.stdout: "state_kv_commiting_lat": { 2024-04-07T17:51:29.555 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 69678, 2024-04-07T17:51:29.555 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 46634763201 2024-04-07T17:51:29.555 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.555 INFO:tasks.workunit.client.0.smithi100.stdout: "state_kv_done_lat": { 2024-04-07T17:51:29.556 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 69678, 2024-04-07T17:51:29.556 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 54811413 2024-04-07T17:51:29.556 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.556 INFO:tasks.workunit.client.0.smithi100.stdout: "state_kv_queued_lat": { 2024-04-07T17:51:29.556 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 69678, 2024-04-07T17:51:29.556 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 21669658103 2024-04-07T17:51:29.556 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.556 INFO:tasks.workunit.client.0.smithi100.stdout: "state_prepare_lat": { 2024-04-07T17:51:29.556 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 69678, 2024-04-07T17:51:29.556 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 25344251701 2024-04-07T17:51:29.556 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.556 INFO:tasks.workunit.client.0.smithi100.stdout: "stored": { 2024-04-07T17:51:29.556 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 9894713 2024-04-07T17:51:29.556 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.556 INFO:tasks.workunit.client.0.smithi100.stdout: "truncate_lat": { 2024-04-07T17:51:29.556 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 2371, 2024-04-07T17:51:29.557 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 24012279 2024-04-07T17:51:29.557 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.557 INFO:tasks.workunit.client.0.smithi100.stdout: "txc_commit_lat": { 2024-04-07T17:51:29.557 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 69678, 2024-04-07T17:51:29.557 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 102473756098 2024-04-07T17:51:29.557 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.557 INFO:tasks.workunit.client.0.smithi100.stdout: "txc_submit_lat": { 2024-04-07T17:51:29.557 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 69678, 2024-04-07T17:51:29.557 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 27569044264 2024-04-07T17:51:29.557 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.557 INFO:tasks.workunit.client.0.smithi100.stdout: "txc_throttle_lat": { 2024-04-07T17:51:29.557 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 69678, 2024-04-07T17:51:29.557 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 877546314 2024-04-07T17:51:29.557 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.557 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.557 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache": { 2024-04-07T17:51:29.557 INFO:tasks.workunit.client.0.smithi100.stdout: "cache_bytes": { 2024-04-07T17:51:29.558 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1444650649 2024-04-07T17:51:29.558 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.558 INFO:tasks.workunit.client.0.smithi100.stdout: "heap_bytes": { 2024-04-07T17:51:29.558 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 623263744 2024-04-07T17:51:29.558 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.558 INFO:tasks.workunit.client.0.smithi100.stdout: "mapped_bytes": { 2024-04-07T17:51:29.558 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 593297408 2024-04-07T17:51:29.558 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.558 INFO:tasks.workunit.client.0.smithi100.stdout: "target_bytes": { 2024-04-07T17:51:29.558 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2647008256 2024-04-07T17:51:29.558 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.558 INFO:tasks.workunit.client.0.smithi100.stdout: "unmapped_bytes": { 2024-04-07T17:51:29.558 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 29966336 2024-04-07T17:51:29.558 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.558 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.559 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:data": { 2024-04-07T17:51:29.559 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:29.559 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 134217728 2024-04-07T17:51:29.559 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.559 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:29.559 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.559 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.559 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:29.559 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.559 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.559 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:29.559 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 58883623 2024-04-07T17:51:29.559 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.559 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:29.560 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.560 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.560 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:29.560 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.560 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.560 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:29.560 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.560 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.560 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:29.560 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.560 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.560 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:29.560 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 24576 2024-04-07T17:51:29.560 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.560 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:29.561 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 761856 2024-04-07T17:51:29.561 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.561 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:29.561 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.561 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.561 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:29.561 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.561 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.561 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:29.561 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.561 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.561 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:29.561 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 74547673 2024-04-07T17:51:29.561 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.561 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.561 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:kv": { 2024-04-07T17:51:29.562 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:29.562 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 578813952 2024-04-07T17:51:29.562 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.562 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:29.562 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 67108864 2024-04-07T17:51:29.562 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.562 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:29.562 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.562 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.562 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:29.562 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 441627178 2024-04-07T17:51:29.562 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.562 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:29.562 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.562 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.563 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:29.563 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.563 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.563 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:29.563 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2416 2024-04-07T17:51:29.563 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.563 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:29.563 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.563 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.563 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:29.563 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 153616 2024-04-07T17:51:29.563 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.563 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:29.563 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.563 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.563 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:29.563 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.564 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.564 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:29.564 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.564 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.564 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:29.564 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.564 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.564 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:29.564 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 69921878 2024-04-07T17:51:29.564 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.564 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.564 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:kv_onode": { 2024-04-07T17:51:29.564 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:29.564 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 192937984 2024-04-07T17:51:29.564 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.564 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:29.565 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 67108864 2024-04-07T17:51:29.565 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.565 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:29.565 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 13844688 2024-04-07T17:51:29.565 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.565 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:29.565 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 39255749 2024-04-07T17:51:29.565 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.565 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:29.565 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.565 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.565 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:29.565 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.565 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.565 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:29.566 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.566 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.566 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:29.566 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.566 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.566 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:29.566 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.566 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.566 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:29.566 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.566 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.566 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:29.566 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.566 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.567 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:29.567 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.567 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.567 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:29.567 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.567 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.567 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:29.567 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 72728683 2024-04-07T17:51:29.567 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.567 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.567 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:meta": { 2024-04-07T17:51:29.567 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:29.567 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 511705088 2024-04-07T17:51:29.567 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.567 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:29.567 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.568 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.568 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:29.568 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.568 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.568 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:29.568 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 441627178 2024-04-07T17:51:29.568 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.568 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:29.568 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.568 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.568 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:29.568 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.568 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.568 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:29.568 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.568 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.569 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:29.569 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.569 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.569 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:29.569 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 144030 2024-04-07T17:51:29.569 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.569 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:29.569 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1105234 2024-04-07T17:51:29.569 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.569 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:29.569 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.569 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.569 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:29.569 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.569 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.570 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:29.570 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.570 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.570 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:29.570 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 68828646 2024-04-07T17:51:29.570 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.570 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.570 INFO:tasks.workunit.client.0.smithi100.stdout: "objecter": { 2024-04-07T17:51:29.570 INFO:tasks.workunit.client.0.smithi100.stdout: "op_active": { 2024-04-07T17:51:29.570 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.570 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.570 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r": { 2024-04-07T17:51:29.570 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 22 2024-04-07T17:51:29.570 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.570 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rmw": { 2024-04-07T17:51:29.571 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.571 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.571 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w": { 2024-04-07T17:51:29.571 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.571 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.571 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.571 INFO:tasks.workunit.client.0.smithi100.stdout: "osd": { 2024-04-07T17:51:29.571 INFO:tasks.workunit.client.0.smithi100.stdout: "numpg": { 2024-04-07T17:51:29.571 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 51 2024-04-07T17:51:29.571 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.571 INFO:tasks.workunit.client.0.smithi100.stdout: "numpg_removing": { 2024-04-07T17:51:29.571 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.571 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.571 INFO:tasks.workunit.client.0.smithi100.stdout: "op": { 2024-04-07T17:51:29.571 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 27984 2024-04-07T17:51:29.572 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.572 INFO:tasks.workunit.client.0.smithi100.stdout: "op_before_queue_op_lat": { 2024-04-07T17:51:29.572 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 87803, 2024-04-07T17:51:29.572 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 14036212222 2024-04-07T17:51:29.572 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.572 INFO:tasks.workunit.client.0.smithi100.stdout: "op_delayed_degraded": { 2024-04-07T17:51:29.572 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1 2024-04-07T17:51:29.572 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.572 INFO:tasks.workunit.client.0.smithi100.stdout: "op_delayed_unreadable": { 2024-04-07T17:51:29.572 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.572 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.572 INFO:tasks.workunit.client.0.smithi100.stdout: "op_in_bytes": { 2024-04-07T17:51:29.572 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1438594914 2024-04-07T17:51:29.572 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.572 INFO:tasks.workunit.client.0.smithi100.stdout: "op_latency": { 2024-04-07T17:51:29.573 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 27984, 2024-04-07T17:51:29.573 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 91475787538 2024-04-07T17:51:29.573 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.573 INFO:tasks.workunit.client.0.smithi100.stdout: "op_out_bytes": { 2024-04-07T17:51:29.573 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1486727151 2024-04-07T17:51:29.573 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.573 INFO:tasks.workunit.client.0.smithi100.stdout: "op_prepare_latency": { 2024-04-07T17:51:29.573 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 28568, 2024-04-07T17:51:29.573 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 27069205821 2024-04-07T17:51:29.573 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.573 INFO:tasks.workunit.client.0.smithi100.stdout: "op_process_latency": { 2024-04-07T17:51:29.573 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 27984, 2024-04-07T17:51:29.573 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 54395342610 2024-04-07T17:51:29.573 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.573 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r": { 2024-04-07T17:51:29.573 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 18559 2024-04-07T17:51:29.574 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.574 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_latency": { 2024-04-07T17:51:29.574 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 18559, 2024-04-07T17:51:29.574 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 15330456442 2024-04-07T17:51:29.574 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.574 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_latency_out_bytes_histogram": { 2024-04-07T17:51:29.574 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.574 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.574 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_out_bytes": { 2024-04-07T17:51:29.574 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1486727151 2024-04-07T17:51:29.574 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.574 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_prepare_latency": { 2024-04-07T17:51:29.574 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 18752, 2024-04-07T17:51:29.574 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 10042264838 2024-04-07T17:51:29.574 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.575 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_process_latency": { 2024-04-07T17:51:29.575 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 18559, 2024-04-07T17:51:29.575 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 9329326658 2024-04-07T17:51:29.575 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.575 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw": { 2024-04-07T17:51:29.575 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5511 2024-04-07T17:51:29.575 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.575 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_in_bytes": { 2024-04-07T17:51:29.575 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 266802303 2024-04-07T17:51:29.575 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.575 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_latency": { 2024-04-07T17:51:29.575 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 5511, 2024-04-07T17:51:29.575 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 19608757265 2024-04-07T17:51:29.575 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.575 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_latency_in_bytes_histogram": { 2024-04-07T17:51:29.576 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.576 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.576 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_latency_out_bytes_histogram": { 2024-04-07T17:51:29.576 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.576 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.576 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_out_bytes": { 2024-04-07T17:51:29.576 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.576 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.576 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_prepare_latency": { 2024-04-07T17:51:29.576 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 5777, 2024-04-07T17:51:29.576 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 7872110857 2024-04-07T17:51:29.576 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.576 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_process_latency": { 2024-04-07T17:51:29.576 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 5511, 2024-04-07T17:51:29.576 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 17096965462 2024-04-07T17:51:29.577 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.577 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w": { 2024-04-07T17:51:29.577 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3914 2024-04-07T17:51:29.577 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.577 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_in_bytes": { 2024-04-07T17:51:29.577 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1171792611 2024-04-07T17:51:29.577 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.577 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_latency": { 2024-04-07T17:51:29.577 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 3914, 2024-04-07T17:51:29.577 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 56536573831 2024-04-07T17:51:29.577 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.577 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_latency_in_bytes_histogram": { 2024-04-07T17:51:29.577 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.577 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.577 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_prepare_latency": { 2024-04-07T17:51:29.577 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 4039, 2024-04-07T17:51:29.578 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 9154830126 2024-04-07T17:51:29.578 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.578 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_process_latency": { 2024-04-07T17:51:29.578 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 3914, 2024-04-07T17:51:29.578 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 27969050490 2024-04-07T17:51:29.578 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.578 INFO:tasks.workunit.client.0.smithi100.stdout: "op_wip": { 2024-04-07T17:51:29.578 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.578 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.578 INFO:tasks.workunit.client.0.smithi100.stdout: "recovery_bytes": { 2024-04-07T17:51:29.578 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 8388656 2024-04-07T17:51:29.578 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.578 INFO:tasks.workunit.client.0.smithi100.stdout: "recovery_ops": { 2024-04-07T17:51:29.578 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 11 2024-04-07T17:51:29.579 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.579 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_bytes": { 2024-04-07T17:51:29.579 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 95995035648 2024-04-07T17:51:29.579 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.579 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_bytes_used": { 2024-04-07T17:51:29.579 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 586682368 2024-04-07T17:51:29.579 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.579 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.579 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.579 INFO:tasks.workunit.client.0.smithi100.stdout: "osd.5": { 2024-04-07T17:51:29.579 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs": { 2024-04-07T17:51:29.579 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_db_max_lat": { 2024-04-07T17:51:29.579 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 26003 2024-04-07T17:51:29.579 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.579 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_slow_fallback": { 2024-04-07T17:51:29.579 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.580 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.580 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_slow_max_lat": { 2024-04-07T17:51:29.580 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.580 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.580 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_slow_size_fallback": { 2024-04-07T17:51:29.580 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.580 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.580 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit_db": { 2024-04-07T17:51:29.580 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 65536 2024-04-07T17:51:29.580 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.580 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit_slow": { 2024-04-07T17:51:29.580 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.580 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.580 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit_wal": { 2024-04-07T17:51:29.580 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.581 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.581 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_wal_max_lat": { 2024-04-07T17:51:29.581 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.581 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.581 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_written_slow": { 2024-04-07T17:51:29.581 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.581 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.581 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_written_sst": { 2024-04-07T17:51:29.581 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 15654912 2024-04-07T17:51:29.581 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.581 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_written_wal": { 2024-04-07T17:51:29.581 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 603410432 2024-04-07T17:51:29.581 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.581 INFO:tasks.workunit.client.0.smithi100.stdout: "compact_lat": { 2024-04-07T17:51:29.582 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 19, 2024-04-07T17:51:29.582 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 8231783 2024-04-07T17:51:29.582 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.582 INFO:tasks.workunit.client.0.smithi100.stdout: "compact_lock_lat": { 2024-04-07T17:51:29.582 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 19, 2024-04-07T17:51:29.582 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2455108 2024-04-07T17:51:29.582 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.582 INFO:tasks.workunit.client.0.smithi100.stdout: "db_alloc_lat": { 2024-04-07T17:51:29.582 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 196, 2024-04-07T17:51:29.582 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1310582 2024-04-07T17:51:29.582 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.582 INFO:tasks.workunit.client.0.smithi100.stdout: "db_total_bytes": { 2024-04-07T17:51:29.582 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 95995027456 2024-04-07T17:51:29.582 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.582 INFO:tasks.workunit.client.0.smithi100.stdout: "db_used_bytes": { 2024-04-07T17:51:29.583 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 444465152 2024-04-07T17:51:29.583 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.583 INFO:tasks.workunit.client.0.smithi100.stdout: "flush_lat": { 2024-04-07T17:51:29.583 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 80665, 2024-04-07T17:51:29.583 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 15235152455 2024-04-07T17:51:29.583 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.583 INFO:tasks.workunit.client.0.smithi100.stdout: "fsync_lat": { 2024-04-07T17:51:29.583 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 80656, 2024-04-07T17:51:29.583 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 26858743324 2024-04-07T17:51:29.583 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.583 INFO:tasks.workunit.client.0.smithi100.stdout: "log_bytes": { 2024-04-07T17:51:29.583 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 14532608 2024-04-07T17:51:29.583 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.583 INFO:tasks.workunit.client.0.smithi100.stdout: "logged_bytes": { 2024-04-07T17:51:29.583 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 330424320 2024-04-07T17:51:29.583 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.584 INFO:tasks.workunit.client.0.smithi100.stdout: "max_bytes_db": { 2024-04-07T17:51:29.584 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 452984832 2024-04-07T17:51:29.584 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.584 INFO:tasks.workunit.client.0.smithi100.stdout: "max_bytes_slow": { 2024-04-07T17:51:29.584 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.584 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.584 INFO:tasks.workunit.client.0.smithi100.stdout: "max_bytes_wal": { 2024-04-07T17:51:29.584 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.584 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.584 INFO:tasks.workunit.client.0.smithi100.stdout: "num_files": { 2024-04-07T17:51:29.584 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 46 2024-04-07T17:51:29.584 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.584 INFO:tasks.workunit.client.0.smithi100.stdout: "read_bytes": { 2024-04-07T17:51:29.584 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 12789730188 2024-04-07T17:51:29.584 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.584 INFO:tasks.workunit.client.0.smithi100.stdout: "read_count": { 2024-04-07T17:51:29.584 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 100535 2024-04-07T17:51:29.585 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.585 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes": { 2024-04-07T17:51:29.585 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 10848047104 2024-04-07T17:51:29.585 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.585 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes_db": { 2024-04-07T17:51:29.585 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 10848051200 2024-04-07T17:51:29.585 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.585 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes_slow": { 2024-04-07T17:51:29.585 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.585 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.585 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes_wal": { 2024-04-07T17:51:29.585 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.585 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.585 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_count": { 2024-04-07T17:51:29.585 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 94858 2024-04-07T17:51:29.586 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.586 INFO:tasks.workunit.client.0.smithi100.stdout: "read_lat": { 2024-04-07T17:51:29.586 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 100535, 2024-04-07T17:51:29.586 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5637691717 2024-04-07T17:51:29.586 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.586 INFO:tasks.workunit.client.0.smithi100.stdout: "read_prefetch_bytes": { 2024-04-07T17:51:29.586 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 12578820194 2024-04-07T17:51:29.586 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.586 INFO:tasks.workunit.client.0.smithi100.stdout: "read_prefetch_count": { 2024-04-07T17:51:29.586 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 94081 2024-04-07T17:51:29.586 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.586 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_buffer_bytes": { 2024-04-07T17:51:29.586 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4494109 2024-04-07T17:51:29.586 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.587 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_buffer_count": { 2024-04-07T17:51:29.587 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 924 2024-04-07T17:51:29.587 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.587 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_bytes": { 2024-04-07T17:51:29.587 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 15530604 2024-04-07T17:51:29.587 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.587 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_count": { 2024-04-07T17:51:29.587 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1234 2024-04-07T17:51:29.587 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.587 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes": { 2024-04-07T17:51:29.587 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 11036495 2024-04-07T17:51:29.587 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.587 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes_db": { 2024-04-07T17:51:29.587 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 11036495 2024-04-07T17:51:29.588 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.588 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes_slow": { 2024-04-07T17:51:29.588 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.588 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.588 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes_wal": { 2024-04-07T17:51:29.588 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.588 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.588 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_count": { 2024-04-07T17:51:29.588 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 324 2024-04-07T17:51:29.588 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.588 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_lat": { 2024-04-07T17:51:29.588 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1234, 2024-04-07T17:51:29.588 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4942084 2024-04-07T17:51:29.588 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.588 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_alloc_lat": { 2024-04-07T17:51:29.588 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:29.589 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.589 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.589 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_total_bytes": { 2024-04-07T17:51:29.589 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.589 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.589 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_used_bytes": { 2024-04-07T17:51:29.589 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.589 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.589 INFO:tasks.workunit.client.0.smithi100.stdout: "truncate_lat": { 2024-04-07T17:51:29.589 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1, 2024-04-07T17:51:29.589 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 682 2024-04-07T17:51:29.589 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.589 INFO:tasks.workunit.client.0.smithi100.stdout: "unlink_lat": { 2024-04-07T17:51:29.589 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 15, 2024-04-07T17:51:29.589 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 19508 2024-04-07T17:51:29.589 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.590 INFO:tasks.workunit.client.0.smithi100.stdout: "wal_alloc_lat": { 2024-04-07T17:51:29.590 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:29.590 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.590 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.590 INFO:tasks.workunit.client.0.smithi100.stdout: "wal_total_bytes": { 2024-04-07T17:51:29.590 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.590 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.590 INFO:tasks.workunit.client.0.smithi100.stdout: "wal_used_bytes": { 2024-04-07T17:51:29.590 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.590 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.590 INFO:tasks.workunit.client.0.smithi100.stdout: "write_bytes": { 2024-04-07T17:51:29.590 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 952373248 2024-04-07T17:51:29.590 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.590 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.590 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore": { 2024-04-07T17:51:29.591 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit": { 2024-04-07T17:51:29.591 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4096 2024-04-07T17:51:29.591 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.591 INFO:tasks.workunit.client.0.smithi100.stdout: "allocated": { 2024-04-07T17:51:29.591 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 14020608 2024-04-07T17:51:29.591 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.591 INFO:tasks.workunit.client.0.smithi100.stdout: "allocator_lat": { 2024-04-07T17:51:29.591 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 22611, 2024-04-07T17:51:29.591 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 116632904 2024-04-07T17:51:29.591 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.591 INFO:tasks.workunit.client.0.smithi100.stdout: "clist_lat": { 2024-04-07T17:51:29.591 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 3546, 2024-04-07T17:51:29.591 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 17766845093 2024-04-07T17:51:29.591 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.591 INFO:tasks.workunit.client.0.smithi100.stdout: "compress_lat": { 2024-04-07T17:51:29.592 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:29.592 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.592 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.592 INFO:tasks.workunit.client.0.smithi100.stdout: "compressed": { 2024-04-07T17:51:29.592 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.592 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.592 INFO:tasks.workunit.client.0.smithi100.stdout: "compressed_allocated": { 2024-04-07T17:51:29.592 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.592 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.592 INFO:tasks.workunit.client.0.smithi100.stdout: "compressed_original": { 2024-04-07T17:51:29.592 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.592 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.592 INFO:tasks.workunit.client.0.smithi100.stdout: "csum_lat": { 2024-04-07T17:51:29.592 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 24579, 2024-04-07T17:51:29.592 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 325075173 2024-04-07T17:51:29.592 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.593 INFO:tasks.workunit.client.0.smithi100.stdout: "decompress_lat": { 2024-04-07T17:51:29.593 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:29.593 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.593 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.593 INFO:tasks.workunit.client.0.smithi100.stdout: "fragmentation_micros": { 2024-04-07T17:51:29.593 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.593 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.593 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_commit_lat": { 2024-04-07T17:51:29.593 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 80597, 2024-04-07T17:51:29.593 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 34623882580 2024-04-07T17:51:29.593 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.593 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_final_lat": { 2024-04-07T17:51:29.593 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 79698, 2024-04-07T17:51:29.593 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3958472053 2024-04-07T17:51:29.593 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.594 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_flush_lat": { 2024-04-07T17:51:29.594 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 80597, 2024-04-07T17:51:29.594 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 99120687 2024-04-07T17:51:29.594 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.594 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_sync_lat": { 2024-04-07T17:51:29.594 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 80597, 2024-04-07T17:51:29.594 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 34723003267 2024-04-07T17:51:29.594 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.594 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_get_keys_lat": { 2024-04-07T17:51:29.594 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:29.594 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.594 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.594 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_get_values_lat": { 2024-04-07T17:51:29.594 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 239680, 2024-04-07T17:51:29.594 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2166046359 2024-04-07T17:51:29.594 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.595 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_lower_bound_lat": { 2024-04-07T17:51:29.595 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 808, 2024-04-07T17:51:29.595 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5289872 2024-04-07T17:51:29.595 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.595 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_next_lat": { 2024-04-07T17:51:29.595 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 128881, 2024-04-07T17:51:29.595 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 97129343 2024-04-07T17:51:29.595 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.595 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_seek_to_first_lat": { 2024-04-07T17:51:29.595 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 50, 2024-04-07T17:51:29.595 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 74621 2024-04-07T17:51:29.595 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.595 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_upper_bound_lat": { 2024-04-07T17:51:29.595 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 19469, 2024-04-07T17:51:29.595 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 8325495929 2024-04-07T17:51:29.596 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.596 INFO:tasks.workunit.client.0.smithi100.stdout: "onode_hits": { 2024-04-07T17:51:29.596 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 436725 2024-04-07T17:51:29.596 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.596 INFO:tasks.workunit.client.0.smithi100.stdout: "onode_misses": { 2024-04-07T17:51:29.596 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 20982 2024-04-07T17:51:29.596 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.596 INFO:tasks.workunit.client.0.smithi100.stdout: "read_lat": { 2024-04-07T17:51:29.596 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 10659, 2024-04-07T17:51:29.596 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2220392001 2024-04-07T17:51:29.596 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.596 INFO:tasks.workunit.client.0.smithi100.stdout: "read_onode_meta_lat": { 2024-04-07T17:51:29.596 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 30132, 2024-04-07T17:51:29.596 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 15167976 2024-04-07T17:51:29.596 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.597 INFO:tasks.workunit.client.0.smithi100.stdout: "read_wait_aio_lat": { 2024-04-07T17:51:29.597 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 19473, 2024-04-07T17:51:29.597 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1807973266 2024-04-07T17:51:29.597 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.597 INFO:tasks.workunit.client.0.smithi100.stdout: "reads_with_retries": { 2024-04-07T17:51:29.597 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.597 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.597 INFO:tasks.workunit.client.0.smithi100.stdout: "remove_lat": { 2024-04-07T17:51:29.597 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 15467, 2024-04-07T17:51:29.597 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 823899553 2024-04-07T17:51:29.597 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.597 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_aio_wait_count": { 2024-04-07T17:51:29.597 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.597 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.597 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_committed_kv_count": { 2024-04-07T17:51:29.597 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.597 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.598 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_read_onode_meta_count": { 2024-04-07T17:51:29.598 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.598 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.598 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_read_wait_aio_count": { 2024-04-07T17:51:29.598 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.598 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.598 INFO:tasks.workunit.client.0.smithi100.stdout: "state_aio_wait_lat": { 2024-04-07T17:51:29.598 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 93751, 2024-04-07T17:51:29.598 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 13111865981 2024-04-07T17:51:29.598 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.598 INFO:tasks.workunit.client.0.smithi100.stdout: "state_deferred_aio_wait_lat": { 2024-04-07T17:51:29.598 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 8297, 2024-04-07T17:51:29.598 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1069282503 2024-04-07T17:51:29.598 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.598 INFO:tasks.workunit.client.0.smithi100.stdout: "state_deferred_cleanup_lat": { 2024-04-07T17:51:29.599 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 8296, 2024-04-07T17:51:29.599 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4488212413467 2024-04-07T17:51:29.599 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.599 INFO:tasks.workunit.client.0.smithi100.stdout: "state_deferred_queued_lat": { 2024-04-07T17:51:29.599 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 8297, 2024-04-07T17:51:29.599 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 6666966893111 2024-04-07T17:51:29.599 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.599 INFO:tasks.workunit.client.0.smithi100.stdout: "state_done_lat": { 2024-04-07T17:51:29.599 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 93750, 2024-04-07T17:51:29.599 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3192176271911 2024-04-07T17:51:29.599 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.599 INFO:tasks.workunit.client.0.smithi100.stdout: "state_finishing_lat": { 2024-04-07T17:51:29.599 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 93750, 2024-04-07T17:51:29.599 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 23533677 2024-04-07T17:51:29.599 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.599 INFO:tasks.workunit.client.0.smithi100.stdout: "state_io_done_lat": { 2024-04-07T17:51:29.599 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 93751, 2024-04-07T17:51:29.600 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 354231200 2024-04-07T17:51:29.600 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.600 INFO:tasks.workunit.client.0.smithi100.stdout: "state_kv_commiting_lat": { 2024-04-07T17:51:29.600 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 93751, 2024-04-07T17:51:29.600 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 60032684809 2024-04-07T17:51:29.600 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.600 INFO:tasks.workunit.client.0.smithi100.stdout: "state_kv_done_lat": { 2024-04-07T17:51:29.600 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 93751, 2024-04-07T17:51:29.600 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 73338408 2024-04-07T17:51:29.600 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.600 INFO:tasks.workunit.client.0.smithi100.stdout: "state_kv_queued_lat": { 2024-04-07T17:51:29.600 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 93751, 2024-04-07T17:51:29.600 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 29192592705 2024-04-07T17:51:29.600 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.600 INFO:tasks.workunit.client.0.smithi100.stdout: "state_prepare_lat": { 2024-04-07T17:51:29.600 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 93751, 2024-04-07T17:51:29.601 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 35861066911 2024-04-07T17:51:29.601 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.601 INFO:tasks.workunit.client.0.smithi100.stdout: "stored": { 2024-04-07T17:51:29.601 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 9899919 2024-04-07T17:51:29.601 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.601 INFO:tasks.workunit.client.0.smithi100.stdout: "truncate_lat": { 2024-04-07T17:51:29.601 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 2320, 2024-04-07T17:51:29.601 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 23026482 2024-04-07T17:51:29.601 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.601 INFO:tasks.workunit.client.0.smithi100.stdout: "txc_commit_lat": { 2024-04-07T17:51:29.601 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 93751, 2024-04-07T17:51:29.601 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 138586597593 2024-04-07T17:51:29.601 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.601 INFO:tasks.workunit.client.0.smithi100.stdout: "txc_submit_lat": { 2024-04-07T17:51:29.601 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 93751, 2024-04-07T17:51:29.601 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 38590847980 2024-04-07T17:51:29.601 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.602 INFO:tasks.workunit.client.0.smithi100.stdout: "txc_throttle_lat": { 2024-04-07T17:51:29.602 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 93751, 2024-04-07T17:51:29.602 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1304542608 2024-04-07T17:51:29.602 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.602 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.602 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache": { 2024-04-07T17:51:29.602 INFO:tasks.workunit.client.0.smithi100.stdout: "cache_bytes": { 2024-04-07T17:51:29.602 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1444650649 2024-04-07T17:51:29.602 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.602 INFO:tasks.workunit.client.0.smithi100.stdout: "heap_bytes": { 2024-04-07T17:51:29.602 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 570359808 2024-04-07T17:51:29.602 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.602 INFO:tasks.workunit.client.0.smithi100.stdout: "mapped_bytes": { 2024-04-07T17:51:29.602 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 557162496 2024-04-07T17:51:29.602 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.603 INFO:tasks.workunit.client.0.smithi100.stdout: "target_bytes": { 2024-04-07T17:51:29.603 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2647008256 2024-04-07T17:51:29.603 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.603 INFO:tasks.workunit.client.0.smithi100.stdout: "unmapped_bytes": { 2024-04-07T17:51:29.603 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 13197312 2024-04-07T17:51:29.603 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.603 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.603 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:data": { 2024-04-07T17:51:29.603 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:29.603 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 134217728 2024-04-07T17:51:29.603 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.603 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:29.603 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.603 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.603 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:29.604 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.604 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.604 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:29.604 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 58739848 2024-04-07T17:51:29.604 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.604 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:29.604 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.604 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.604 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:29.604 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.604 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.604 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:29.604 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.604 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.604 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:29.605 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.605 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.605 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:29.605 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4096 2024-04-07T17:51:29.605 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.605 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:29.605 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 24576 2024-04-07T17:51:29.605 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.605 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:29.605 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.605 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.605 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:29.605 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.605 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.605 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:29.605 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.606 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.606 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:29.606 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 75449208 2024-04-07T17:51:29.606 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.606 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.606 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:kv": { 2024-04-07T17:51:29.606 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:29.606 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 578813952 2024-04-07T17:51:29.606 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.606 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:29.606 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 67108864 2024-04-07T17:51:29.606 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.606 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:29.606 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.606 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.607 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:29.607 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 440548866 2024-04-07T17:51:29.607 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.607 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:29.607 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.607 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.607 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:29.607 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.607 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.607 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:29.607 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2128 2024-04-07T17:51:29.607 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.607 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:29.607 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.607 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.607 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:29.607 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 163344 2024-04-07T17:51:29.607 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.608 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:29.608 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.608 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.608 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:29.608 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.608 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.608 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:29.608 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.608 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.608 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:29.608 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.608 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.608 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:29.608 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 70990750 2024-04-07T17:51:29.608 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.608 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.608 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:kv_onode": { 2024-04-07T17:51:29.608 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:29.609 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 192937984 2024-04-07T17:51:29.609 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.609 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:29.609 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 67108864 2024-04-07T17:51:29.609 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.609 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:29.609 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 15205920 2024-04-07T17:51:29.609 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.609 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:29.609 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 39159899 2024-04-07T17:51:29.609 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.609 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:29.609 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.609 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.609 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:29.609 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.609 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.610 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:29.610 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.610 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.610 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:29.610 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.610 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.610 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:29.610 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.610 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.610 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:29.610 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.610 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.610 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:29.610 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.610 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.610 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:29.611 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.611 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.611 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:29.611 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.611 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.611 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:29.611 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 71463301 2024-04-07T17:51:29.611 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.611 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.611 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:meta": { 2024-04-07T17:51:29.611 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:29.611 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 511705088 2024-04-07T17:51:29.611 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.611 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:29.611 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.611 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.611 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:29.612 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.612 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.612 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:29.612 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 440548866 2024-04-07T17:51:29.612 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.612 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:29.612 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.612 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.612 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:29.612 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.612 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.612 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:29.612 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.612 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.612 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:29.612 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.612 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.613 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:29.613 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 148347 2024-04-07T17:51:29.613 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.613 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:29.613 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1094803 2024-04-07T17:51:29.613 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.613 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:29.613 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.613 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.613 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:29.613 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.613 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.613 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:29.613 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.613 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.613 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:29.613 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 69913072 2024-04-07T17:51:29.614 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.614 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.614 INFO:tasks.workunit.client.0.smithi100.stdout: "objecter": { 2024-04-07T17:51:29.614 INFO:tasks.workunit.client.0.smithi100.stdout: "op_active": { 2024-04-07T17:51:29.614 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.614 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.614 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r": { 2024-04-07T17:51:29.614 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 24 2024-04-07T17:51:29.614 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.614 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rmw": { 2024-04-07T17:51:29.614 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.614 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.614 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w": { 2024-04-07T17:51:29.614 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.614 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.615 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.615 INFO:tasks.workunit.client.0.smithi100.stdout: "osd": { 2024-04-07T17:51:29.615 INFO:tasks.workunit.client.0.smithi100.stdout: "numpg": { 2024-04-07T17:51:29.615 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 54 2024-04-07T17:51:29.615 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.615 INFO:tasks.workunit.client.0.smithi100.stdout: "numpg_removing": { 2024-04-07T17:51:29.615 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.615 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.615 INFO:tasks.workunit.client.0.smithi100.stdout: "op": { 2024-04-07T17:51:29.615 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 37568 2024-04-07T17:51:29.615 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.615 INFO:tasks.workunit.client.0.smithi100.stdout: "op_before_queue_op_lat": { 2024-04-07T17:51:29.615 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 118839, 2024-04-07T17:51:29.615 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 18250742844 2024-04-07T17:51:29.615 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.616 INFO:tasks.workunit.client.0.smithi100.stdout: "op_delayed_degraded": { 2024-04-07T17:51:29.616 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.616 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.616 INFO:tasks.workunit.client.0.smithi100.stdout: "op_delayed_unreadable": { 2024-04-07T17:51:29.616 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.616 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.616 INFO:tasks.workunit.client.0.smithi100.stdout: "op_in_bytes": { 2024-04-07T17:51:29.616 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1478194555 2024-04-07T17:51:29.616 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.616 INFO:tasks.workunit.client.0.smithi100.stdout: "op_latency": { 2024-04-07T17:51:29.616 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 37568, 2024-04-07T17:51:29.616 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 92458218161 2024-04-07T17:51:29.616 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.616 INFO:tasks.workunit.client.0.smithi100.stdout: "op_out_bytes": { 2024-04-07T17:51:29.617 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2285559046 2024-04-07T17:51:29.617 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.617 INFO:tasks.workunit.client.0.smithi100.stdout: "op_prepare_latency": { 2024-04-07T17:51:29.617 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 38251, 2024-04-07T17:51:29.617 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 45291105721 2024-04-07T17:51:29.617 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.617 INFO:tasks.workunit.client.0.smithi100.stdout: "op_process_latency": { 2024-04-07T17:51:29.617 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 37568, 2024-04-07T17:51:29.617 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 67926473334 2024-04-07T17:51:29.617 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.617 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r": { 2024-04-07T17:51:29.617 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 26708 2024-04-07T17:51:29.617 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.617 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_latency": { 2024-04-07T17:51:29.617 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 26708, 2024-04-07T17:51:29.617 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 37896778616 2024-04-07T17:51:29.618 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.618 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_latency_out_bytes_histogram": { 2024-04-07T17:51:29.618 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.618 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.618 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_out_bytes": { 2024-04-07T17:51:29.618 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2285559046 2024-04-07T17:51:29.618 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.618 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_prepare_latency": { 2024-04-07T17:51:29.618 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 26937, 2024-04-07T17:51:29.618 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 27634387557 2024-04-07T17:51:29.618 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.618 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_process_latency": { 2024-04-07T17:51:29.618 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 26708, 2024-04-07T17:51:29.618 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 26262052230 2024-04-07T17:51:29.618 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.619 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw": { 2024-04-07T17:51:29.619 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 9627 2024-04-07T17:51:29.619 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.619 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_in_bytes": { 2024-04-07T17:51:29.619 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 446562633 2024-04-07T17:51:29.619 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.619 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_latency": { 2024-04-07T17:51:29.619 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 9627, 2024-04-07T17:51:29.619 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 39696616362 2024-04-07T17:51:29.619 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.619 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_latency_in_bytes_histogram": { 2024-04-07T17:51:29.619 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.619 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.619 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_latency_out_bytes_histogram": { 2024-04-07T17:51:29.619 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.619 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.620 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_out_bytes": { 2024-04-07T17:51:29.620 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.620 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.620 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_prepare_latency": { 2024-04-07T17:51:29.620 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 9890, 2024-04-07T17:51:29.620 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 13660567957 2024-04-07T17:51:29.620 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.620 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_process_latency": { 2024-04-07T17:51:29.620 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 9627, 2024-04-07T17:51:29.620 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 30252117533 2024-04-07T17:51:29.620 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.620 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w": { 2024-04-07T17:51:29.620 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1233 2024-04-07T17:51:29.620 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.620 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_in_bytes": { 2024-04-07T17:51:29.621 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1031631922 2024-04-07T17:51:29.621 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.621 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_latency": { 2024-04-07T17:51:29.621 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1233, 2024-04-07T17:51:29.621 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 14864823183 2024-04-07T17:51:29.621 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.621 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_latency_in_bytes_histogram": { 2024-04-07T17:51:29.621 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.621 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.621 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_prepare_latency": { 2024-04-07T17:51:29.621 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1424, 2024-04-07T17:51:29.621 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3996150207 2024-04-07T17:51:29.621 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.621 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_process_latency": { 2024-04-07T17:51:29.621 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1233, 2024-04-07T17:51:29.621 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 11412303571 2024-04-07T17:51:29.621 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.622 INFO:tasks.workunit.client.0.smithi100.stdout: "op_wip": { 2024-04-07T17:51:29.622 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.622 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.622 INFO:tasks.workunit.client.0.smithi100.stdout: "recovery_bytes": { 2024-04-07T17:51:29.622 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 11394690 2024-04-07T17:51:29.622 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.622 INFO:tasks.workunit.client.0.smithi100.stdout: "recovery_ops": { 2024-04-07T17:51:29.622 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 18 2024-04-07T17:51:29.622 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.622 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_bytes": { 2024-04-07T17:51:29.622 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 95995035648 2024-04-07T17:51:29.622 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.622 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_bytes_used": { 2024-04-07T17:51:29.622 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 458498048 2024-04-07T17:51:29.622 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.623 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.623 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.623 INFO:tasks.workunit.client.0.smithi100.stdout: "osd.6": { 2024-04-07T17:51:29.623 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs": { 2024-04-07T17:51:29.623 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_db_max_lat": { 2024-04-07T17:51:29.623 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 42976 2024-04-07T17:51:29.623 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.623 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_slow_fallback": { 2024-04-07T17:51:29.623 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.623 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.623 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_slow_max_lat": { 2024-04-07T17:51:29.623 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.623 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.623 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_slow_size_fallback": { 2024-04-07T17:51:29.623 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.623 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.623 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit_db": { 2024-04-07T17:51:29.624 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 65536 2024-04-07T17:51:29.624 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.624 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit_slow": { 2024-04-07T17:51:29.624 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.624 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.624 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit_wal": { 2024-04-07T17:51:29.624 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.624 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.624 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_wal_max_lat": { 2024-04-07T17:51:29.624 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.624 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.624 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_written_slow": { 2024-04-07T17:51:29.624 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.624 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.624 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_written_sst": { 2024-04-07T17:51:29.624 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1253376 2024-04-07T17:51:29.625 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.625 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_written_wal": { 2024-04-07T17:51:29.625 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 459055104 2024-04-07T17:51:29.625 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.625 INFO:tasks.workunit.client.0.smithi100.stdout: "compact_lat": { 2024-04-07T17:51:29.625 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 15, 2024-04-07T17:51:29.625 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5459458 2024-04-07T17:51:29.625 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.625 INFO:tasks.workunit.client.0.smithi100.stdout: "compact_lock_lat": { 2024-04-07T17:51:29.625 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 15, 2024-04-07T17:51:29.625 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1804720 2024-04-07T17:51:29.625 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.625 INFO:tasks.workunit.client.0.smithi100.stdout: "db_alloc_lat": { 2024-04-07T17:51:29.625 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 135, 2024-04-07T17:51:29.625 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 985275 2024-04-07T17:51:29.625 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.626 INFO:tasks.workunit.client.0.smithi100.stdout: "db_total_bytes": { 2024-04-07T17:51:29.626 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 95995027456 2024-04-07T17:51:29.626 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.626 INFO:tasks.workunit.client.0.smithi100.stdout: "db_used_bytes": { 2024-04-07T17:51:29.626 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 343408640 2024-04-07T17:51:29.626 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.626 INFO:tasks.workunit.client.0.smithi100.stdout: "flush_lat": { 2024-04-07T17:51:29.626 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 61501, 2024-04-07T17:51:29.626 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 11983540988 2024-04-07T17:51:29.626 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.626 INFO:tasks.workunit.client.0.smithi100.stdout: "fsync_lat": { 2024-04-07T17:51:29.626 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 61513, 2024-04-07T17:51:29.626 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 21141003730 2024-04-07T17:51:29.626 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.627 INFO:tasks.workunit.client.0.smithi100.stdout: "log_bytes": { 2024-04-07T17:51:29.627 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2629632 2024-04-07T17:51:29.627 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.627 INFO:tasks.workunit.client.0.smithi100.stdout: "logged_bytes": { 2024-04-07T17:51:29.627 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 251998208 2024-04-07T17:51:29.627 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.627 INFO:tasks.workunit.client.0.smithi100.stdout: "max_bytes_db": { 2024-04-07T17:51:29.627 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 364707840 2024-04-07T17:51:29.627 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.627 INFO:tasks.workunit.client.0.smithi100.stdout: "max_bytes_slow": { 2024-04-07T17:51:29.627 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.627 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.627 INFO:tasks.workunit.client.0.smithi100.stdout: "max_bytes_wal": { 2024-04-07T17:51:29.627 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.627 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.628 INFO:tasks.workunit.client.0.smithi100.stdout: "num_files": { 2024-04-07T17:51:29.628 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 33 2024-04-07T17:51:29.628 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.628 INFO:tasks.workunit.client.0.smithi100.stdout: "read_bytes": { 2024-04-07T17:51:29.628 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 611394701 2024-04-07T17:51:29.628 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.628 INFO:tasks.workunit.client.0.smithi100.stdout: "read_count": { 2024-04-07T17:51:29.628 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 18243 2024-04-07T17:51:29.628 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.628 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes": { 2024-04-07T17:51:29.628 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 531197952 2024-04-07T17:51:29.628 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.628 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes_db": { 2024-04-07T17:51:29.628 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 531202048 2024-04-07T17:51:29.628 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.628 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes_slow": { 2024-04-07T17:51:29.629 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.629 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.629 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes_wal": { 2024-04-07T17:51:29.629 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.629 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.629 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_count": { 2024-04-07T17:51:29.629 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 16144 2024-04-07T17:51:29.629 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.629 INFO:tasks.workunit.client.0.smithi100.stdout: "read_lat": { 2024-04-07T17:51:29.629 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 18243, 2024-04-07T17:51:29.629 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 442981158 2024-04-07T17:51:29.629 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.629 INFO:tasks.workunit.client.0.smithi100.stdout: "read_prefetch_bytes": { 2024-04-07T17:51:29.629 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 565009679 2024-04-07T17:51:29.629 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.629 INFO:tasks.workunit.client.0.smithi100.stdout: "read_prefetch_count": { 2024-04-07T17:51:29.630 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 16814 2024-04-07T17:51:29.630 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.630 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_buffer_bytes": { 2024-04-07T17:51:29.630 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 563551 2024-04-07T17:51:29.630 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.630 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_buffer_count": { 2024-04-07T17:51:29.630 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 339 2024-04-07T17:51:29.630 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.630 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_bytes": { 2024-04-07T17:51:29.630 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1239831 2024-04-07T17:51:29.630 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.630 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_count": { 2024-04-07T17:51:29.630 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 451 2024-04-07T17:51:29.631 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.631 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes": { 2024-04-07T17:51:29.631 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 676280 2024-04-07T17:51:29.631 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.631 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes_db": { 2024-04-07T17:51:29.631 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 676280 2024-04-07T17:51:29.631 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.631 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes_slow": { 2024-04-07T17:51:29.631 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.631 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.631 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes_wal": { 2024-04-07T17:51:29.631 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.631 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.631 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_count": { 2024-04-07T17:51:29.631 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 112 2024-04-07T17:51:29.631 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.632 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_lat": { 2024-04-07T17:51:29.632 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 451, 2024-04-07T17:51:29.632 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 827027 2024-04-07T17:51:29.632 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.632 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_alloc_lat": { 2024-04-07T17:51:29.632 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:29.632 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.632 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.632 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_total_bytes": { 2024-04-07T17:51:29.632 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.632 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.632 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_used_bytes": { 2024-04-07T17:51:29.632 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.632 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.632 INFO:tasks.workunit.client.0.smithi100.stdout: "truncate_lat": { 2024-04-07T17:51:29.633 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1, 2024-04-07T17:51:29.633 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 646 2024-04-07T17:51:29.633 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.633 INFO:tasks.workunit.client.0.smithi100.stdout: "unlink_lat": { 2024-04-07T17:51:29.633 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 5, 2024-04-07T17:51:29.633 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 8184 2024-04-07T17:51:29.633 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.633 INFO:tasks.workunit.client.0.smithi100.stdout: "wal_alloc_lat": { 2024-04-07T17:51:29.633 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:29.633 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.633 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.633 INFO:tasks.workunit.client.0.smithi100.stdout: "wal_total_bytes": { 2024-04-07T17:51:29.633 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.633 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.633 INFO:tasks.workunit.client.0.smithi100.stdout: "wal_used_bytes": { 2024-04-07T17:51:29.634 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.634 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.634 INFO:tasks.workunit.client.0.smithi100.stdout: "write_bytes": { 2024-04-07T17:51:29.634 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 714584064 2024-04-07T17:51:29.634 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.634 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.634 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore": { 2024-04-07T17:51:29.634 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit": { 2024-04-07T17:51:29.634 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4096 2024-04-07T17:51:29.634 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.634 INFO:tasks.workunit.client.0.smithi100.stdout: "allocated": { 2024-04-07T17:51:29.634 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 14606336 2024-04-07T17:51:29.634 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.634 INFO:tasks.workunit.client.0.smithi100.stdout: "allocator_lat": { 2024-04-07T17:51:29.635 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 16890, 2024-04-07T17:51:29.635 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 74243402 2024-04-07T17:51:29.635 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.635 INFO:tasks.workunit.client.0.smithi100.stdout: "clist_lat": { 2024-04-07T17:51:29.635 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 3441, 2024-04-07T17:51:29.635 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 6473693390 2024-04-07T17:51:29.635 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.635 INFO:tasks.workunit.client.0.smithi100.stdout: "compress_lat": { 2024-04-07T17:51:29.635 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:29.635 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.635 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.635 INFO:tasks.workunit.client.0.smithi100.stdout: "compressed": { 2024-04-07T17:51:29.635 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.635 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.635 INFO:tasks.workunit.client.0.smithi100.stdout: "compressed_allocated": { 2024-04-07T17:51:29.635 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.636 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.636 INFO:tasks.workunit.client.0.smithi100.stdout: "compressed_original": { 2024-04-07T17:51:29.636 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.636 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.636 INFO:tasks.workunit.client.0.smithi100.stdout: "csum_lat": { 2024-04-07T17:51:29.636 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 14729, 2024-04-07T17:51:29.636 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 190648102 2024-04-07T17:51:29.636 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.636 INFO:tasks.workunit.client.0.smithi100.stdout: "decompress_lat": { 2024-04-07T17:51:29.636 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:29.636 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.636 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.636 INFO:tasks.workunit.client.0.smithi100.stdout: "fragmentation_micros": { 2024-04-07T17:51:29.636 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.636 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.637 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_commit_lat": { 2024-04-07T17:51:29.637 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 61477, 2024-04-07T17:51:29.637 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 27469991543 2024-04-07T17:51:29.637 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.637 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_final_lat": { 2024-04-07T17:51:29.637 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 60732, 2024-04-07T17:51:29.637 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3226860182 2024-04-07T17:51:29.637 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.637 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_flush_lat": { 2024-04-07T17:51:29.637 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 61477, 2024-04-07T17:51:29.637 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 74425283 2024-04-07T17:51:29.637 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.637 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_sync_lat": { 2024-04-07T17:51:29.637 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 61477, 2024-04-07T17:51:29.637 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 27544416826 2024-04-07T17:51:29.638 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.638 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_get_keys_lat": { 2024-04-07T17:51:29.638 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:29.638 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.638 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.638 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_get_values_lat": { 2024-04-07T17:51:29.638 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 22581, 2024-04-07T17:51:29.638 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 685980505 2024-04-07T17:51:29.638 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.638 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_lower_bound_lat": { 2024-04-07T17:51:29.638 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 700, 2024-04-07T17:51:29.638 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5026445 2024-04-07T17:51:29.638 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.638 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_next_lat": { 2024-04-07T17:51:29.639 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 10108, 2024-04-07T17:51:29.639 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 12802315 2024-04-07T17:51:29.639 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.639 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_seek_to_first_lat": { 2024-04-07T17:51:29.639 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 43, 2024-04-07T17:51:29.639 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 77153 2024-04-07T17:51:29.639 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.639 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_upper_bound_lat": { 2024-04-07T17:51:29.639 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 10657, 2024-04-07T17:51:29.639 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1970969784 2024-04-07T17:51:29.639 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.639 INFO:tasks.workunit.client.0.smithi100.stdout: "onode_hits": { 2024-04-07T17:51:29.639 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 175604 2024-04-07T17:51:29.639 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.639 INFO:tasks.workunit.client.0.smithi100.stdout: "onode_misses": { 2024-04-07T17:51:29.639 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 18139 2024-04-07T17:51:29.640 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.640 INFO:tasks.workunit.client.0.smithi100.stdout: "read_lat": { 2024-04-07T17:51:29.640 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 11039, 2024-04-07T17:51:29.640 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1962439157 2024-04-07T17:51:29.640 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.640 INFO:tasks.workunit.client.0.smithi100.stdout: "read_onode_meta_lat": { 2024-04-07T17:51:29.640 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 38635, 2024-04-07T17:51:29.640 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 20820077 2024-04-07T17:51:29.640 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.640 INFO:tasks.workunit.client.0.smithi100.stdout: "read_wait_aio_lat": { 2024-04-07T17:51:29.640 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 27596, 2024-04-07T17:51:29.640 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1296711458 2024-04-07T17:51:29.640 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.640 INFO:tasks.workunit.client.0.smithi100.stdout: "reads_with_retries": { 2024-04-07T17:51:29.641 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.641 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.641 INFO:tasks.workunit.client.0.smithi100.stdout: "remove_lat": { 2024-04-07T17:51:29.641 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 11796, 2024-04-07T17:51:29.641 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 656690853 2024-04-07T17:51:29.641 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.641 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_aio_wait_count": { 2024-04-07T17:51:29.641 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.641 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.641 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_committed_kv_count": { 2024-04-07T17:51:29.641 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.641 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.641 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_read_onode_meta_count": { 2024-04-07T17:51:29.641 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.641 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.642 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_read_wait_aio_count": { 2024-04-07T17:51:29.642 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.642 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.642 INFO:tasks.workunit.client.0.smithi100.stdout: "state_aio_wait_lat": { 2024-04-07T17:51:29.642 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 74651, 2024-04-07T17:51:29.642 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 10364615640 2024-04-07T17:51:29.642 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.642 INFO:tasks.workunit.client.0.smithi100.stdout: "state_deferred_aio_wait_lat": { 2024-04-07T17:51:29.642 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 13939, 2024-04-07T17:51:29.642 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1372751186 2024-04-07T17:51:29.642 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.642 INFO:tasks.workunit.client.0.smithi100.stdout: "state_deferred_cleanup_lat": { 2024-04-07T17:51:29.642 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 13939, 2024-04-07T17:51:29.642 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3701426932890 2024-04-07T17:51:29.642 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.643 INFO:tasks.workunit.client.0.smithi100.stdout: "state_deferred_queued_lat": { 2024-04-07T17:51:29.643 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 13939, 2024-04-07T17:51:29.643 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5923221764357 2024-04-07T17:51:29.643 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.643 INFO:tasks.workunit.client.0.smithi100.stdout: "state_done_lat": { 2024-04-07T17:51:29.643 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 74651, 2024-04-07T17:51:29.643 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2803609540196 2024-04-07T17:51:29.643 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.643 INFO:tasks.workunit.client.0.smithi100.stdout: "state_finishing_lat": { 2024-04-07T17:51:29.643 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 74651, 2024-04-07T17:51:29.643 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 20563045 2024-04-07T17:51:29.643 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.643 INFO:tasks.workunit.client.0.smithi100.stdout: "state_io_done_lat": { 2024-04-07T17:51:29.643 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 74651, 2024-04-07T17:51:29.643 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 593835565 2024-04-07T17:51:29.644 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.644 INFO:tasks.workunit.client.0.smithi100.stdout: "state_kv_commiting_lat": { 2024-04-07T17:51:29.644 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 74651, 2024-04-07T17:51:29.644 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 51952800738 2024-04-07T17:51:29.644 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.644 INFO:tasks.workunit.client.0.smithi100.stdout: "state_kv_done_lat": { 2024-04-07T17:51:29.644 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 74651, 2024-04-07T17:51:29.644 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 60663028 2024-04-07T17:51:29.644 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.644 INFO:tasks.workunit.client.0.smithi100.stdout: "state_kv_queued_lat": { 2024-04-07T17:51:29.644 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 74651, 2024-04-07T17:51:29.644 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 28224386813 2024-04-07T17:51:29.644 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.644 INFO:tasks.workunit.client.0.smithi100.stdout: "state_prepare_lat": { 2024-04-07T17:51:29.644 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 74651, 2024-04-07T17:51:29.644 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 21898861720 2024-04-07T17:51:29.645 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.645 INFO:tasks.workunit.client.0.smithi100.stdout: "stored": { 2024-04-07T17:51:29.645 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 10490178 2024-04-07T17:51:29.645 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.645 INFO:tasks.workunit.client.0.smithi100.stdout: "truncate_lat": { 2024-04-07T17:51:29.645 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 2295, 2024-04-07T17:51:29.645 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 25153482 2024-04-07T17:51:29.645 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.645 INFO:tasks.workunit.client.0.smithi100.stdout: "txc_commit_lat": { 2024-04-07T17:51:29.645 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 74651, 2024-04-07T17:51:29.645 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 113061581342 2024-04-07T17:51:29.645 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.645 INFO:tasks.workunit.client.0.smithi100.stdout: "txc_submit_lat": { 2024-04-07T17:51:29.645 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 74651, 2024-04-07T17:51:29.646 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 23969092831 2024-04-07T17:51:29.646 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.646 INFO:tasks.workunit.client.0.smithi100.stdout: "txc_throttle_lat": { 2024-04-07T17:51:29.646 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 74651, 2024-04-07T17:51:29.646 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1130975469 2024-04-07T17:51:29.646 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.646 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.646 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache": { 2024-04-07T17:51:29.646 INFO:tasks.workunit.client.0.smithi100.stdout: "cache_bytes": { 2024-04-07T17:51:29.646 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1444650649 2024-04-07T17:51:29.646 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.646 INFO:tasks.workunit.client.0.smithi100.stdout: "heap_bytes": { 2024-04-07T17:51:29.646 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 423641088 2024-04-07T17:51:29.646 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.647 INFO:tasks.workunit.client.0.smithi100.stdout: "mapped_bytes": { 2024-04-07T17:51:29.647 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 358752256 2024-04-07T17:51:29.647 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.647 INFO:tasks.workunit.client.0.smithi100.stdout: "target_bytes": { 2024-04-07T17:51:29.647 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2647008256 2024-04-07T17:51:29.647 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.647 INFO:tasks.workunit.client.0.smithi100.stdout: "unmapped_bytes": { 2024-04-07T17:51:29.647 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 64888832 2024-04-07T17:51:29.647 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.647 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.647 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:data": { 2024-04-07T17:51:29.647 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:29.647 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 134217728 2024-04-07T17:51:29.647 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.647 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:29.648 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.648 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.648 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:29.648 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.648 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.648 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:29.648 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 59908062 2024-04-07T17:51:29.648 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.648 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:29.648 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.648 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.648 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:29.648 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.648 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.648 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:29.649 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.649 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.649 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:29.649 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.649 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.649 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:29.649 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.649 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.649 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:29.649 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 184320 2024-04-07T17:51:29.649 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.649 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:29.649 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.649 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.649 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:29.649 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.650 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.650 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:29.650 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.650 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.650 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:29.650 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 74125346 2024-04-07T17:51:29.650 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.650 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.650 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:kv": { 2024-04-07T17:51:29.650 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:29.650 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 587202560 2024-04-07T17:51:29.650 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.650 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:29.650 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 67108864 2024-04-07T17:51:29.650 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.651 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:29.651 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.651 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.651 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:29.651 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 449310472 2024-04-07T17:51:29.651 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.651 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:29.651 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.651 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.651 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:29.651 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.651 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.651 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:29.651 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1040 2024-04-07T17:51:29.651 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.652 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:29.652 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.652 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.652 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:29.652 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 66384 2024-04-07T17:51:29.652 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.652 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:29.652 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.652 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.652 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:29.652 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.652 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.652 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:29.652 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.652 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.652 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:29.653 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.653 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.653 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:29.653 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 70715800 2024-04-07T17:51:29.653 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.653 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.653 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:kv_onode": { 2024-04-07T17:51:29.653 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:29.653 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 176160768 2024-04-07T17:51:29.653 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.653 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:29.653 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 67108864 2024-04-07T17:51:29.653 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.653 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:29.653 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 691824 2024-04-07T17:51:29.653 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.654 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:29.654 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 39938708 2024-04-07T17:51:29.654 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.654 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:29.654 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.654 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.654 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:29.654 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.654 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.654 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:29.654 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.654 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.654 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:29.654 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.655 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.655 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:29.655 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.655 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.655 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:29.655 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.655 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.655 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:29.655 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.655 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.655 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:29.655 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.655 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.655 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:29.655 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.655 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.656 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:29.656 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 68421372 2024-04-07T17:51:29.656 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.656 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.656 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:meta": { 2024-04-07T17:51:29.656 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:29.656 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 520093696 2024-04-07T17:51:29.656 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.656 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:29.656 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.656 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.656 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:29.656 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.656 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.656 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:29.657 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 449310472 2024-04-07T17:51:29.657 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.657 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:29.657 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.657 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.657 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:29.657 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.657 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.657 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:29.657 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.657 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.657 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:29.657 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.657 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.657 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:29.658 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 144960 2024-04-07T17:51:29.658 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.658 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:29.658 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1119133 2024-04-07T17:51:29.658 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.658 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:29.658 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.658 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.658 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:29.658 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.658 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.658 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:29.658 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.658 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.658 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:29.659 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 69519131 2024-04-07T17:51:29.659 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.659 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.659 INFO:tasks.workunit.client.0.smithi100.stdout: "objecter": { 2024-04-07T17:51:29.659 INFO:tasks.workunit.client.0.smithi100.stdout: "op_active": { 2024-04-07T17:51:29.659 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.659 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.659 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r": { 2024-04-07T17:51:29.659 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5 2024-04-07T17:51:29.659 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.659 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rmw": { 2024-04-07T17:51:29.659 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.659 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.659 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w": { 2024-04-07T17:51:29.659 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.659 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.660 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.660 INFO:tasks.workunit.client.0.smithi100.stdout: "osd": { 2024-04-07T17:51:29.660 INFO:tasks.workunit.client.0.smithi100.stdout: "numpg": { 2024-04-07T17:51:29.660 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 54 2024-04-07T17:51:29.660 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.660 INFO:tasks.workunit.client.0.smithi100.stdout: "numpg_removing": { 2024-04-07T17:51:29.660 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.660 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.660 INFO:tasks.workunit.client.0.smithi100.stdout: "op": { 2024-04-07T17:51:29.660 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 33509 2024-04-07T17:51:29.660 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.660 INFO:tasks.workunit.client.0.smithi100.stdout: "op_before_queue_op_lat": { 2024-04-07T17:51:29.660 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 91105, 2024-04-07T17:51:29.660 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 12792382018 2024-04-07T17:51:29.660 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.660 INFO:tasks.workunit.client.0.smithi100.stdout: "op_delayed_degraded": { 2024-04-07T17:51:29.660 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.661 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.661 INFO:tasks.workunit.client.0.smithi100.stdout: "op_delayed_unreadable": { 2024-04-07T17:51:29.661 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.661 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.661 INFO:tasks.workunit.client.0.smithi100.stdout: "op_in_bytes": { 2024-04-07T17:51:29.661 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1535388602 2024-04-07T17:51:29.661 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.661 INFO:tasks.workunit.client.0.smithi100.stdout: "op_latency": { 2024-04-07T17:51:29.661 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 33509, 2024-04-07T17:51:29.661 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 131541624216 2024-04-07T17:51:29.661 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.661 INFO:tasks.workunit.client.0.smithi100.stdout: "op_out_bytes": { 2024-04-07T17:51:29.661 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1703112431 2024-04-07T17:51:29.661 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.661 INFO:tasks.workunit.client.0.smithi100.stdout: "op_prepare_latency": { 2024-04-07T17:51:29.661 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 34219, 2024-04-07T17:51:29.662 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 27830378329 2024-04-07T17:51:29.662 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.662 INFO:tasks.workunit.client.0.smithi100.stdout: "op_process_latency": { 2024-04-07T17:51:29.662 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 33509, 2024-04-07T17:51:29.662 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 53861705532 2024-04-07T17:51:29.662 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.662 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r": { 2024-04-07T17:51:29.662 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 25245 2024-04-07T17:51:29.662 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.662 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_latency": { 2024-04-07T17:51:29.662 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 25245, 2024-04-07T17:51:29.662 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 23376821581 2024-04-07T17:51:29.662 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.662 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_latency_out_bytes_histogram": { 2024-04-07T17:51:29.662 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.662 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.663 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_out_bytes": { 2024-04-07T17:51:29.663 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1703112431 2024-04-07T17:51:29.663 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.663 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_prepare_latency": { 2024-04-07T17:51:29.663 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 25399, 2024-04-07T17:51:29.663 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 11923589716 2024-04-07T17:51:29.663 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.663 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_process_latency": { 2024-04-07T17:51:29.663 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 25245, 2024-04-07T17:51:29.663 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 10146174967 2024-04-07T17:51:29.663 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.663 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw": { 2024-04-07T17:51:29.663 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4064 2024-04-07T17:51:29.663 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.663 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_in_bytes": { 2024-04-07T17:51:29.663 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 290840421 2024-04-07T17:51:29.663 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.664 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_latency": { 2024-04-07T17:51:29.664 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 4064, 2024-04-07T17:51:29.664 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 20965538275 2024-04-07T17:51:29.664 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.664 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_latency_in_bytes_histogram": { 2024-04-07T17:51:29.664 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.664 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.664 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_latency_out_bytes_histogram": { 2024-04-07T17:51:29.664 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.664 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.664 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_out_bytes": { 2024-04-07T17:51:29.664 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.664 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.664 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_prepare_latency": { 2024-04-07T17:51:29.664 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 4504, 2024-04-07T17:51:29.664 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 6305172918 2024-04-07T17:51:29.664 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.664 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_process_latency": { 2024-04-07T17:51:29.665 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 4064, 2024-04-07T17:51:29.665 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 14864893688 2024-04-07T17:51:29.665 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.665 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w": { 2024-04-07T17:51:29.665 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4200 2024-04-07T17:51:29.665 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.665 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_in_bytes": { 2024-04-07T17:51:29.665 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1244548181 2024-04-07T17:51:29.665 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.665 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_latency": { 2024-04-07T17:51:29.665 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 4200, 2024-04-07T17:51:29.665 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 87199264360 2024-04-07T17:51:29.665 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.665 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_latency_in_bytes_histogram": { 2024-04-07T17:51:29.665 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.666 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.666 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_prepare_latency": { 2024-04-07T17:51:29.666 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 4316, 2024-04-07T17:51:29.666 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 9601615695 2024-04-07T17:51:29.666 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.666 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_process_latency": { 2024-04-07T17:51:29.666 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 4200, 2024-04-07T17:51:29.666 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 28850636877 2024-04-07T17:51:29.666 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.666 INFO:tasks.workunit.client.0.smithi100.stdout: "op_wip": { 2024-04-07T17:51:29.666 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.666 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.666 INFO:tasks.workunit.client.0.smithi100.stdout: "recovery_bytes": { 2024-04-07T17:51:29.666 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 73766334 2024-04-07T17:51:29.666 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.666 INFO:tasks.workunit.client.0.smithi100.stdout: "recovery_ops": { 2024-04-07T17:51:29.667 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 34 2024-04-07T17:51:29.667 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.667 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_bytes": { 2024-04-07T17:51:29.667 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 95995035648 2024-04-07T17:51:29.667 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.667 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_bytes_used": { 2024-04-07T17:51:29.667 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 358023168 2024-04-07T17:51:29.667 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.667 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.667 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.667 INFO:tasks.workunit.client.0.smithi100.stdout: "osd.7": { 2024-04-07T17:51:29.667 INFO:tasks.workunit.client.0.smithi100.stdout: "bluefs": { 2024-04-07T17:51:29.667 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_db_max_lat": { 2024-04-07T17:51:29.667 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1425185 2024-04-07T17:51:29.667 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.667 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_slow_fallback": { 2024-04-07T17:51:29.667 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.668 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.668 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_slow_max_lat": { 2024-04-07T17:51:29.668 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.668 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.668 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_slow_size_fallback": { 2024-04-07T17:51:29.668 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.668 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.668 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit_db": { 2024-04-07T17:51:29.668 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 65536 2024-04-07T17:51:29.668 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.668 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit_slow": { 2024-04-07T17:51:29.668 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.668 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.668 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit_wal": { 2024-04-07T17:51:29.668 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.668 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.669 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_wal_max_lat": { 2024-04-07T17:51:29.669 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.669 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.669 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_written_slow": { 2024-04-07T17:51:29.669 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.669 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.669 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_written_sst": { 2024-04-07T17:51:29.669 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5050368 2024-04-07T17:51:29.669 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.669 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_written_wal": { 2024-04-07T17:51:29.669 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 495611904 2024-04-07T17:51:29.669 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.669 INFO:tasks.workunit.client.0.smithi100.stdout: "compact_lat": { 2024-04-07T17:51:29.669 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 15, 2024-04-07T17:51:29.669 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 6850825 2024-04-07T17:51:29.669 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.670 INFO:tasks.workunit.client.0.smithi100.stdout: "compact_lock_lat": { 2024-04-07T17:51:29.670 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 15, 2024-04-07T17:51:29.670 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1769094 2024-04-07T17:51:29.670 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.670 INFO:tasks.workunit.client.0.smithi100.stdout: "db_alloc_lat": { 2024-04-07T17:51:29.670 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 148, 2024-04-07T17:51:29.670 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2568628 2024-04-07T17:51:29.670 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.670 INFO:tasks.workunit.client.0.smithi100.stdout: "db_total_bytes": { 2024-04-07T17:51:29.670 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 95995027456 2024-04-07T17:51:29.670 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.670 INFO:tasks.workunit.client.0.smithi100.stdout: "db_used_bytes": { 2024-04-07T17:51:29.670 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 411107328 2024-04-07T17:51:29.670 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.671 INFO:tasks.workunit.client.0.smithi100.stdout: "flush_lat": { 2024-04-07T17:51:29.671 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 63033, 2024-04-07T17:51:29.671 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 12124224545 2024-04-07T17:51:29.671 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.671 INFO:tasks.workunit.client.0.smithi100.stdout: "fsync_lat": { 2024-04-07T17:51:29.671 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 63044, 2024-04-07T17:51:29.671 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 21250946957 2024-04-07T17:51:29.671 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.671 INFO:tasks.workunit.client.0.smithi100.stdout: "log_bytes": { 2024-04-07T17:51:29.671 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 8908800 2024-04-07T17:51:29.671 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.671 INFO:tasks.workunit.client.0.smithi100.stdout: "logged_bytes": { 2024-04-07T17:51:29.671 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 258269184 2024-04-07T17:51:29.671 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.671 INFO:tasks.workunit.client.0.smithi100.stdout: "max_bytes_db": { 2024-04-07T17:51:29.672 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 423886848 2024-04-07T17:51:29.672 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.672 INFO:tasks.workunit.client.0.smithi100.stdout: "max_bytes_slow": { 2024-04-07T17:51:29.672 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.672 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.672 INFO:tasks.workunit.client.0.smithi100.stdout: "max_bytes_wal": { 2024-04-07T17:51:29.672 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.672 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.672 INFO:tasks.workunit.client.0.smithi100.stdout: "num_files": { 2024-04-07T17:51:29.672 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 38 2024-04-07T17:51:29.672 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.672 INFO:tasks.workunit.client.0.smithi100.stdout: "read_bytes": { 2024-04-07T17:51:29.672 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3167491823 2024-04-07T17:51:29.672 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.672 INFO:tasks.workunit.client.0.smithi100.stdout: "read_count": { 2024-04-07T17:51:29.673 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 40027 2024-04-07T17:51:29.673 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.673 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes": { 2024-04-07T17:51:29.673 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2790236160 2024-04-07T17:51:29.673 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.673 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes_db": { 2024-04-07T17:51:29.673 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2790240256 2024-04-07T17:51:29.673 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.673 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes_slow": { 2024-04-07T17:51:29.673 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.673 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.673 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_bytes_wal": { 2024-04-07T17:51:29.673 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.673 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.674 INFO:tasks.workunit.client.0.smithi100.stdout: "read_disk_count": { 2024-04-07T17:51:29.674 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 25974 2024-04-07T17:51:29.674 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.674 INFO:tasks.workunit.client.0.smithi100.stdout: "read_lat": { 2024-04-07T17:51:29.674 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 40027, 2024-04-07T17:51:29.674 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1975651191 2024-04-07T17:51:29.674 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.674 INFO:tasks.workunit.client.0.smithi100.stdout: "read_prefetch_bytes": { 2024-04-07T17:51:29.674 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3056697503 2024-04-07T17:51:29.674 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.674 INFO:tasks.workunit.client.0.smithi100.stdout: "read_prefetch_count": { 2024-04-07T17:51:29.674 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 36631 2024-04-07T17:51:29.674 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.674 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_buffer_bytes": { 2024-04-07T17:51:29.674 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1782285 2024-04-07T17:51:29.675 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.675 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_buffer_count": { 2024-04-07T17:51:29.675 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 538 2024-04-07T17:51:29.675 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.675 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_bytes": { 2024-04-07T17:51:29.675 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 5008896 2024-04-07T17:51:29.675 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.675 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_count": { 2024-04-07T17:51:29.675 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 703 2024-04-07T17:51:29.675 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.675 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes": { 2024-04-07T17:51:29.675 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3226611 2024-04-07T17:51:29.675 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.675 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes_db": { 2024-04-07T17:51:29.675 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3226611 2024-04-07T17:51:29.676 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.676 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes_slow": { 2024-04-07T17:51:29.676 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.676 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.676 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_bytes_wal": { 2024-04-07T17:51:29.676 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.676 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.676 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_disk_count": { 2024-04-07T17:51:29.676 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 167 2024-04-07T17:51:29.676 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.676 INFO:tasks.workunit.client.0.smithi100.stdout: "read_random_lat": { 2024-04-07T17:51:29.676 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 703, 2024-04-07T17:51:29.676 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1945572 2024-04-07T17:51:29.676 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.677 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_alloc_lat": { 2024-04-07T17:51:29.677 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:29.677 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.677 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.677 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_total_bytes": { 2024-04-07T17:51:29.677 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.677 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.677 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_used_bytes": { 2024-04-07T17:51:29.677 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.677 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.677 INFO:tasks.workunit.client.0.smithi100.stdout: "truncate_lat": { 2024-04-07T17:51:29.677 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1, 2024-04-07T17:51:29.677 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 858 2024-04-07T17:51:29.677 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.677 INFO:tasks.workunit.client.0.smithi100.stdout: "unlink_lat": { 2024-04-07T17:51:29.678 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 9, 2024-04-07T17:51:29.678 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 12676 2024-04-07T17:51:29.678 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.678 INFO:tasks.workunit.client.0.smithi100.stdout: "wal_alloc_lat": { 2024-04-07T17:51:29.678 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:29.678 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.678 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.678 INFO:tasks.workunit.client.0.smithi100.stdout: "wal_total_bytes": { 2024-04-07T17:51:29.678 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.678 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.678 INFO:tasks.workunit.client.0.smithi100.stdout: "wal_used_bytes": { 2024-04-07T17:51:29.678 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.678 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.678 INFO:tasks.workunit.client.0.smithi100.stdout: "write_bytes": { 2024-04-07T17:51:29.679 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 761217024 2024-04-07T17:51:29.679 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.679 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.679 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore": { 2024-04-07T17:51:29.679 INFO:tasks.workunit.client.0.smithi100.stdout: "alloc_unit": { 2024-04-07T17:51:29.679 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4096 2024-04-07T17:51:29.679 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.679 INFO:tasks.workunit.client.0.smithi100.stdout: "allocated": { 2024-04-07T17:51:29.679 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 10403840 2024-04-07T17:51:29.679 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.679 INFO:tasks.workunit.client.0.smithi100.stdout: "allocator_lat": { 2024-04-07T17:51:29.679 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 17672, 2024-04-07T17:51:29.679 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 84277754 2024-04-07T17:51:29.679 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.679 INFO:tasks.workunit.client.0.smithi100.stdout: "clist_lat": { 2024-04-07T17:51:29.679 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 3466, 2024-04-07T17:51:29.679 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 6069941459 2024-04-07T17:51:29.680 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.680 INFO:tasks.workunit.client.0.smithi100.stdout: "compress_lat": { 2024-04-07T17:51:29.680 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:29.680 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.680 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.680 INFO:tasks.workunit.client.0.smithi100.stdout: "compressed": { 2024-04-07T17:51:29.680 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.680 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.680 INFO:tasks.workunit.client.0.smithi100.stdout: "compressed_allocated": { 2024-04-07T17:51:29.680 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.680 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.680 INFO:tasks.workunit.client.0.smithi100.stdout: "compressed_original": { 2024-04-07T17:51:29.680 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.680 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.680 INFO:tasks.workunit.client.0.smithi100.stdout: "csum_lat": { 2024-04-07T17:51:29.680 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 16463, 2024-04-07T17:51:29.680 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 221000679 2024-04-07T17:51:29.681 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.681 INFO:tasks.workunit.client.0.smithi100.stdout: "decompress_lat": { 2024-04-07T17:51:29.681 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:29.681 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.681 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.681 INFO:tasks.workunit.client.0.smithi100.stdout: "fragmentation_micros": { 2024-04-07T17:51:29.681 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.681 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.681 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_commit_lat": { 2024-04-07T17:51:29.681 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 62999, 2024-04-07T17:51:29.681 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 27509607699 2024-04-07T17:51:29.681 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.681 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_final_lat": { 2024-04-07T17:51:29.681 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 62260, 2024-04-07T17:51:29.681 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3517974079 2024-04-07T17:51:29.681 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.681 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_flush_lat": { 2024-04-07T17:51:29.681 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 62999, 2024-04-07T17:51:29.682 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 96265273 2024-04-07T17:51:29.682 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.682 INFO:tasks.workunit.client.0.smithi100.stdout: "kv_sync_lat": { 2024-04-07T17:51:29.682 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 62999, 2024-04-07T17:51:29.682 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 27605872972 2024-04-07T17:51:29.682 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.682 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_get_keys_lat": { 2024-04-07T17:51:29.682 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 0, 2024-04-07T17:51:29.682 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.682 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.682 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_get_values_lat": { 2024-04-07T17:51:29.682 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 33493, 2024-04-07T17:51:29.682 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 950805853 2024-04-07T17:51:29.682 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.683 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_lower_bound_lat": { 2024-04-07T17:51:29.683 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 632, 2024-04-07T17:51:29.683 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4331297 2024-04-07T17:51:29.683 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.683 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_next_lat": { 2024-04-07T17:51:29.683 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 7000, 2024-04-07T17:51:29.683 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 13697854 2024-04-07T17:51:29.683 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.683 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_seek_to_first_lat": { 2024-04-07T17:51:29.683 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 50, 2024-04-07T17:51:29.683 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 102727 2024-04-07T17:51:29.683 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.683 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_upper_bound_lat": { 2024-04-07T17:51:29.683 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 11201, 2024-04-07T17:51:29.683 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1557562272 2024-04-07T17:51:29.683 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.683 INFO:tasks.workunit.client.0.smithi100.stdout: "onode_hits": { 2024-04-07T17:51:29.684 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 194571 2024-04-07T17:51:29.684 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.684 INFO:tasks.workunit.client.0.smithi100.stdout: "onode_misses": { 2024-04-07T17:51:29.684 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 17497 2024-04-07T17:51:29.684 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.684 INFO:tasks.workunit.client.0.smithi100.stdout: "read_lat": { 2024-04-07T17:51:29.684 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 9972, 2024-04-07T17:51:29.684 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1596166661 2024-04-07T17:51:29.684 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.684 INFO:tasks.workunit.client.0.smithi100.stdout: "read_onode_meta_lat": { 2024-04-07T17:51:29.684 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 40059, 2024-04-07T17:51:29.684 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 19844921 2024-04-07T17:51:29.684 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.684 INFO:tasks.workunit.client.0.smithi100.stdout: "read_wait_aio_lat": { 2024-04-07T17:51:29.684 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 30087, 2024-04-07T17:51:29.684 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1327511355 2024-04-07T17:51:29.684 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.685 INFO:tasks.workunit.client.0.smithi100.stdout: "reads_with_retries": { 2024-04-07T17:51:29.685 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.685 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.685 INFO:tasks.workunit.client.0.smithi100.stdout: "remove_lat": { 2024-04-07T17:51:29.685 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 13264, 2024-04-07T17:51:29.685 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 667186707 2024-04-07T17:51:29.685 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.685 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_aio_wait_count": { 2024-04-07T17:51:29.685 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.685 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.685 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_committed_kv_count": { 2024-04-07T17:51:29.685 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.685 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.685 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_read_onode_meta_count": { 2024-04-07T17:51:29.685 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.685 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.686 INFO:tasks.workunit.client.0.smithi100.stdout: "slow_read_wait_aio_count": { 2024-04-07T17:51:29.686 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.686 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.686 INFO:tasks.workunit.client.0.smithi100.stdout: "state_aio_wait_lat": { 2024-04-07T17:51:29.686 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 74445, 2024-04-07T17:51:29.686 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 8392306112 2024-04-07T17:51:29.686 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.686 INFO:tasks.workunit.client.0.smithi100.stdout: "state_deferred_aio_wait_lat": { 2024-04-07T17:51:29.686 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 16391, 2024-04-07T17:51:29.686 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1329233809 2024-04-07T17:51:29.686 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.686 INFO:tasks.workunit.client.0.smithi100.stdout: "state_deferred_cleanup_lat": { 2024-04-07T17:51:29.686 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 16391, 2024-04-07T17:51:29.686 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4085640485184 2024-04-07T17:51:29.687 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.687 INFO:tasks.workunit.client.0.smithi100.stdout: "state_deferred_queued_lat": { 2024-04-07T17:51:29.687 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 16391, 2024-04-07T17:51:29.687 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 6763629312230 2024-04-07T17:51:29.687 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.687 INFO:tasks.workunit.client.0.smithi100.stdout: "state_done_lat": { 2024-04-07T17:51:29.687 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 74445, 2024-04-07T17:51:29.687 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2916516222047 2024-04-07T17:51:29.687 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.687 INFO:tasks.workunit.client.0.smithi100.stdout: "state_finishing_lat": { 2024-04-07T17:51:29.687 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 74445, 2024-04-07T17:51:29.687 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 17630145 2024-04-07T17:51:29.687 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.687 INFO:tasks.workunit.client.0.smithi100.stdout: "state_io_done_lat": { 2024-04-07T17:51:29.687 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 74445, 2024-04-07T17:51:29.687 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 362795257 2024-04-07T17:51:29.687 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.688 INFO:tasks.workunit.client.0.smithi100.stdout: "state_kv_commiting_lat": { 2024-04-07T17:51:29.688 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 74445, 2024-04-07T17:51:29.688 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 49659409201 2024-04-07T17:51:29.688 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.688 INFO:tasks.workunit.client.0.smithi100.stdout: "state_kv_done_lat": { 2024-04-07T17:51:29.688 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 74445, 2024-04-07T17:51:29.688 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 57872489 2024-04-07T17:51:29.688 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.688 INFO:tasks.workunit.client.0.smithi100.stdout: "state_kv_queued_lat": { 2024-04-07T17:51:29.688 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 74445, 2024-04-07T17:51:29.688 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 25577010850 2024-04-07T17:51:29.688 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.688 INFO:tasks.workunit.client.0.smithi100.stdout: "state_prepare_lat": { 2024-04-07T17:51:29.688 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 74445, 2024-04-07T17:51:29.688 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 18976008686 2024-04-07T17:51:29.689 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.689 INFO:tasks.workunit.client.0.smithi100.stdout: "stored": { 2024-04-07T17:51:29.689 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 6290654 2024-04-07T17:51:29.689 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.689 INFO:tasks.workunit.client.0.smithi100.stdout: "truncate_lat": { 2024-04-07T17:51:29.689 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 2242, 2024-04-07T17:51:29.689 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 22293375 2024-04-07T17:51:29.689 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.689 INFO:tasks.workunit.client.0.smithi100.stdout: "txc_commit_lat": { 2024-04-07T17:51:29.689 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 74445, 2024-04-07T17:51:29.689 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 102994291156 2024-04-07T17:51:29.689 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.689 INFO:tasks.workunit.client.0.smithi100.stdout: "txc_submit_lat": { 2024-04-07T17:51:29.689 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 74445, 2024-04-07T17:51:29.689 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 21154199772 2024-04-07T17:51:29.690 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.690 INFO:tasks.workunit.client.0.smithi100.stdout: "txc_throttle_lat": { 2024-04-07T17:51:29.690 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 74445, 2024-04-07T17:51:29.690 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 946343542 2024-04-07T17:51:29.690 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.690 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.690 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache": { 2024-04-07T17:51:29.690 INFO:tasks.workunit.client.0.smithi100.stdout: "cache_bytes": { 2024-04-07T17:51:29.690 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1444650649 2024-04-07T17:51:29.690 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.690 INFO:tasks.workunit.client.0.smithi100.stdout: "heap_bytes": { 2024-04-07T17:51:29.690 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 463331328 2024-04-07T17:51:29.690 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.690 INFO:tasks.workunit.client.0.smithi100.stdout: "mapped_bytes": { 2024-04-07T17:51:29.691 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 430931968 2024-04-07T17:51:29.691 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.691 INFO:tasks.workunit.client.0.smithi100.stdout: "target_bytes": { 2024-04-07T17:51:29.691 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2647008256 2024-04-07T17:51:29.691 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.691 INFO:tasks.workunit.client.0.smithi100.stdout: "unmapped_bytes": { 2024-04-07T17:51:29.691 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 32399360 2024-04-07T17:51:29.691 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.691 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.691 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:data": { 2024-04-07T17:51:29.691 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:29.691 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 134217728 2024-04-07T17:51:29.691 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.691 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:29.691 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.691 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.691 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:29.692 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.692 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.692 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:29.692 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 59553061 2024-04-07T17:51:29.692 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.692 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:29.692 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.692 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.692 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:29.692 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.692 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.692 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:29.692 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.692 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.692 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:29.692 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.693 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.693 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:29.693 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 32768 2024-04-07T17:51:29.693 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.693 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:29.693 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 61440 2024-04-07T17:51:29.693 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.693 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:29.693 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.693 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.693 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:29.693 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.693 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.693 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:29.693 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.693 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.694 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:29.694 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 74570459 2024-04-07T17:51:29.694 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.694 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.694 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:kv": { 2024-04-07T17:51:29.694 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:29.694 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 587202560 2024-04-07T17:51:29.694 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.694 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:29.694 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 67108864 2024-04-07T17:51:29.694 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.694 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:29.694 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.694 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.694 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:29.695 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 446647962 2024-04-07T17:51:29.695 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.695 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:29.695 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.695 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.695 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:29.695 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.695 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.695 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:29.695 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1504 2024-04-07T17:51:29.695 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.695 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:29.695 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.695 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.695 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:29.696 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 95456 2024-04-07T17:51:29.696 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.696 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:29.696 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.696 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.696 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:29.696 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.696 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.696 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:29.696 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.696 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.696 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:29.696 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.696 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.696 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:29.697 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 73348774 2024-04-07T17:51:29.697 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.697 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.697 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:kv_onode": { 2024-04-07T17:51:29.697 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:29.697 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 184549376 2024-04-07T17:51:29.697 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.697 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:29.697 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 67108864 2024-04-07T17:51:29.697 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.697 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:29.697 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 4539488 2024-04-07T17:51:29.697 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.697 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:29.697 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 39702041 2024-04-07T17:51:29.697 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.698 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:29.698 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.698 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.698 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:29.698 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.698 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.698 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:29.698 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.698 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.698 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:29.698 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.698 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.698 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:29.698 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.698 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.699 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:29.699 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.699 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.699 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:29.699 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.699 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.699 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:29.699 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.699 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.699 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:29.699 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.699 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.699 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:29.699 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 73198983 2024-04-07T17:51:29.699 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.699 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.700 INFO:tasks.workunit.client.0.smithi100.stdout: "bluestore-pricache:meta": { 2024-04-07T17:51:29.700 INFO:tasks.workunit.client.0.smithi100.stdout: "committed_bytes": { 2024-04-07T17:51:29.700 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 520093696 2024-04-07T17:51:29.700 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.700 INFO:tasks.workunit.client.0.smithi100.stdout: "pri0_bytes": { 2024-04-07T17:51:29.700 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.700 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.700 INFO:tasks.workunit.client.0.smithi100.stdout: "pri10_bytes": { 2024-04-07T17:51:29.700 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.700 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.700 INFO:tasks.workunit.client.0.smithi100.stdout: "pri11_bytes": { 2024-04-07T17:51:29.700 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 446647962 2024-04-07T17:51:29.700 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.700 INFO:tasks.workunit.client.0.smithi100.stdout: "pri1_bytes": { 2024-04-07T17:51:29.700 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.700 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.701 INFO:tasks.workunit.client.0.smithi100.stdout: "pri2_bytes": { 2024-04-07T17:51:29.701 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.701 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.701 INFO:tasks.workunit.client.0.smithi100.stdout: "pri3_bytes": { 2024-04-07T17:51:29.701 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.701 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.701 INFO:tasks.workunit.client.0.smithi100.stdout: "pri4_bytes": { 2024-04-07T17:51:29.701 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.701 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.701 INFO:tasks.workunit.client.0.smithi100.stdout: "pri5_bytes": { 2024-04-07T17:51:29.701 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 149317 2024-04-07T17:51:29.701 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.701 INFO:tasks.workunit.client.0.smithi100.stdout: "pri6_bytes": { 2024-04-07T17:51:29.701 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1104947 2024-04-07T17:51:29.701 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.702 INFO:tasks.workunit.client.0.smithi100.stdout: "pri7_bytes": { 2024-04-07T17:51:29.702 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.702 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.702 INFO:tasks.workunit.client.0.smithi100.stdout: "pri8_bytes": { 2024-04-07T17:51:29.702 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.702 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.702 INFO:tasks.workunit.client.0.smithi100.stdout: "pri9_bytes": { 2024-04-07T17:51:29.702 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.702 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.702 INFO:tasks.workunit.client.0.smithi100.stdout: "reserved_bytes": { 2024-04-07T17:51:29.702 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 72191470 2024-04-07T17:51:29.702 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.702 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.702 INFO:tasks.workunit.client.0.smithi100.stdout: "objecter": { 2024-04-07T17:51:29.703 INFO:tasks.workunit.client.0.smithi100.stdout: "op_active": { 2024-04-07T17:51:29.703 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.703 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.703 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r": { 2024-04-07T17:51:29.703 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 23 2024-04-07T17:51:29.703 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.703 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rmw": { 2024-04-07T17:51:29.703 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.703 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.703 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w": { 2024-04-07T17:51:29.703 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.703 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.703 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.703 INFO:tasks.workunit.client.0.smithi100.stdout: "osd": { 2024-04-07T17:51:29.703 INFO:tasks.workunit.client.0.smithi100.stdout: "numpg": { 2024-04-07T17:51:29.703 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 58 2024-04-07T17:51:29.704 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.704 INFO:tasks.workunit.client.0.smithi100.stdout: "numpg_removing": { 2024-04-07T17:51:29.704 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.704 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.704 INFO:tasks.workunit.client.0.smithi100.stdout: "op": { 2024-04-07T17:51:29.704 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 40545 2024-04-07T17:51:29.704 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.704 INFO:tasks.workunit.client.0.smithi100.stdout: "op_before_queue_op_lat": { 2024-04-07T17:51:29.704 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 108162, 2024-04-07T17:51:29.704 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 15071280376 2024-04-07T17:51:29.704 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.704 INFO:tasks.workunit.client.0.smithi100.stdout: "op_delayed_degraded": { 2024-04-07T17:51:29.704 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.704 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.704 INFO:tasks.workunit.client.0.smithi100.stdout: "op_delayed_unreadable": { 2024-04-07T17:51:29.705 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.705 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.705 INFO:tasks.workunit.client.0.smithi100.stdout: "op_in_bytes": { 2024-04-07T17:51:29.705 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1416321031 2024-04-07T17:51:29.705 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.705 INFO:tasks.workunit.client.0.smithi100.stdout: "op_latency": { 2024-04-07T17:51:29.705 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 40545, 2024-04-07T17:51:29.705 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 116867590636 2024-04-07T17:51:29.705 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.705 INFO:tasks.workunit.client.0.smithi100.stdout: "op_out_bytes": { 2024-04-07T17:51:29.705 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1572111487 2024-04-07T17:51:29.705 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.705 INFO:tasks.workunit.client.0.smithi100.stdout: "op_prepare_latency": { 2024-04-07T17:51:29.705 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 40922, 2024-04-07T17:51:29.706 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 42003164309 2024-04-07T17:51:29.706 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.706 INFO:tasks.workunit.client.0.smithi100.stdout: "op_process_latency": { 2024-04-07T17:51:29.706 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 40545, 2024-04-07T17:51:29.706 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 66247026476 2024-04-07T17:51:29.706 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.706 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r": { 2024-04-07T17:51:29.706 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 27980 2024-04-07T17:51:29.706 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.706 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_latency": { 2024-04-07T17:51:29.706 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 27980, 2024-04-07T17:51:29.706 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 25192283619 2024-04-07T17:51:29.706 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.706 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_latency_out_bytes_histogram": { 2024-04-07T17:51:29.707 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.707 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.707 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_out_bytes": { 2024-04-07T17:51:29.707 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1572111487 2024-04-07T17:51:29.707 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.707 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_prepare_latency": { 2024-04-07T17:51:29.707 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 28098, 2024-04-07T17:51:29.707 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 13073535315 2024-04-07T17:51:29.707 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.707 INFO:tasks.workunit.client.0.smithi100.stdout: "op_r_process_latency": { 2024-04-07T17:51:29.707 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 27980, 2024-04-07T17:51:29.707 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 11416651740 2024-04-07T17:51:29.707 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.707 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw": { 2024-04-07T17:51:29.707 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 11374 2024-04-07T17:51:29.708 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.708 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_in_bytes": { 2024-04-07T17:51:29.708 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 298732687 2024-04-07T17:51:29.708 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.708 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_latency": { 2024-04-07T17:51:29.708 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 11374, 2024-04-07T17:51:29.708 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 46895594573 2024-04-07T17:51:29.708 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.708 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_latency_in_bytes_histogram": { 2024-04-07T17:51:29.708 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.708 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.708 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_latency_out_bytes_histogram": { 2024-04-07T17:51:29.708 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.708 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.708 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_out_bytes": { 2024-04-07T17:51:29.709 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.709 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.709 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_prepare_latency": { 2024-04-07T17:51:29.709 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 11562, 2024-04-07T17:51:29.709 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 24954395180 2024-04-07T17:51:29.709 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.709 INFO:tasks.workunit.client.0.smithi100.stdout: "op_rw_process_latency": { 2024-04-07T17:51:29.709 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 11374, 2024-04-07T17:51:29.709 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 42022805893 2024-04-07T17:51:29.709 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.709 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w": { 2024-04-07T17:51:29.709 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1191 2024-04-07T17:51:29.709 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.709 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_in_bytes": { 2024-04-07T17:51:29.709 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 1117588344 2024-04-07T17:51:29.709 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.710 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_latency": { 2024-04-07T17:51:29.710 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1191, 2024-04-07T17:51:29.710 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 44779712444 2024-04-07T17:51:29.710 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.710 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_latency_in_bytes_histogram": { 2024-04-07T17:51:29.710 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.710 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.710 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_prepare_latency": { 2024-04-07T17:51:29.710 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1262, 2024-04-07T17:51:29.710 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 3975233814 2024-04-07T17:51:29.710 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.710 INFO:tasks.workunit.client.0.smithi100.stdout: "op_w_process_latency": { 2024-04-07T17:51:29.710 INFO:tasks.workunit.client.0.smithi100.stdout: "count": 1191, 2024-04-07T17:51:29.710 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 12807568843 2024-04-07T17:51:29.711 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.711 INFO:tasks.workunit.client.0.smithi100.stdout: "op_wip": { 2024-04-07T17:51:29.711 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.711 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.711 INFO:tasks.workunit.client.0.smithi100.stdout: "recovery_bytes": { 2024-04-07T17:51:29.711 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 0 2024-04-07T17:51:29.711 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.711 INFO:tasks.workunit.client.0.smithi100.stdout: "recovery_ops": { 2024-04-07T17:51:29.711 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 2 2024-04-07T17:51:29.711 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.711 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_bytes": { 2024-04-07T17:51:29.711 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 95995035648 2024-04-07T17:51:29.711 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.711 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_bytes_used": { 2024-04-07T17:51:29.711 INFO:tasks.workunit.client.0.smithi100.stdout: "value": 421519360 2024-04-07T17:51:29.711 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.712 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.712 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.712 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.712 INFO:tasks.workunit.client.0.smithi100.stdout: "pools": [ 2024-04-07T17:51:29.712 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:29.712 INFO:tasks.workunit.client.0.smithi100.stdout: "application": [ 2024-04-07T17:51:29.712 INFO:tasks.workunit.client.0.smithi100.stdout: "mgr" 2024-04-07T17:51:29.712 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:29.712 INFO:tasks.workunit.client.0.smithi100.stdout: "cache_mode": "none", 2024-04-07T17:51:29.712 INFO:tasks.workunit.client.0.smithi100.stdout: "erasure_code_profile": {}, 2024-04-07T17:51:29.712 INFO:tasks.workunit.client.0.smithi100.stdout: "flags_names": [ 2024-04-07T17:51:29.712 INFO:tasks.workunit.client.0.smithi100.stdout: "hashpspool" 2024-04-07T17:51:29.712 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:29.712 INFO:tasks.workunit.client.0.smithi100.stdout: "min_size": 2, 2024-04-07T17:51:29.713 INFO:tasks.workunit.client.0.smithi100.stdout: "options": {}, 2024-04-07T17:51:29.713 INFO:tasks.workunit.client.0.smithi100.stdout: "pg_autoscale_mode": "off", 2024-04-07T17:51:29.713 INFO:tasks.workunit.client.0.smithi100.stdout: "pg_num": 1, 2024-04-07T17:51:29.713 INFO:tasks.workunit.client.0.smithi100.stdout: "pgp_num": 1, 2024-04-07T17:51:29.713 INFO:tasks.workunit.client.0.smithi100.stdout: "pool": 1, 2024-04-07T17:51:29.713 INFO:tasks.workunit.client.0.smithi100.stdout: "size": 3, 2024-04-07T17:51:29.713 INFO:tasks.workunit.client.0.smithi100.stdout: "stats": { 2024-04-07T17:51:29.713 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_raw": 723597404426, 2024-04-07T17:51:29.713 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_used": 1781760, 2024-04-07T17:51:29.713 INFO:tasks.workunit.client.0.smithi100.stdout: "compress_bytes_used": 0, 2024-04-07T17:51:29.713 INFO:tasks.workunit.client.0.smithi100.stdout: "compress_under_bytes": 0, 2024-04-07T17:51:29.713 INFO:tasks.workunit.client.0.smithi100.stdout: "data_bytes_used": 1781760, 2024-04-07T17:51:29.713 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty": 0, 2024-04-07T17:51:29.713 INFO:tasks.workunit.client.0.smithi100.stdout: "max_avail": 241199136768, 2024-04-07T17:51:29.713 INFO:tasks.workunit.client.0.smithi100.stdout: "objects": 2, 2024-04-07T17:51:29.713 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_bytes_used": 0, 2024-04-07T17:51:29.714 INFO:tasks.workunit.client.0.smithi100.stdout: "percent_used": 2.46235777012771e-06, 2024-04-07T17:51:29.714 INFO:tasks.workunit.client.0.smithi100.stdout: "quota_bytes": 0, 2024-04-07T17:51:29.714 INFO:tasks.workunit.client.0.smithi100.stdout: "quota_objects": 0, 2024-04-07T17:51:29.714 INFO:tasks.workunit.client.0.smithi100.stdout: "rd": 1358, 2024-04-07T17:51:29.714 INFO:tasks.workunit.client.0.smithi100.stdout: "rd_bytes": 2195456, 2024-04-07T17:51:29.714 INFO:tasks.workunit.client.0.smithi100.stdout: "stored": 590368, 2024-04-07T17:51:29.714 INFO:tasks.workunit.client.0.smithi100.stdout: "stored_data": 590368, 2024-04-07T17:51:29.714 INFO:tasks.workunit.client.0.smithi100.stdout: "stored_omap": 0, 2024-04-07T17:51:29.714 INFO:tasks.workunit.client.0.smithi100.stdout: "stored_raw": 1771104, 2024-04-07T17:51:29.714 INFO:tasks.workunit.client.0.smithi100.stdout: "wr": 431, 2024-04-07T17:51:29.714 INFO:tasks.workunit.client.0.smithi100.stdout: "wr_bytes": 2473984 2024-04-07T17:51:29.714 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.714 INFO:tasks.workunit.client.0.smithi100.stdout: "target_max_bytes": 0, 2024-04-07T17:51:29.714 INFO:tasks.workunit.client.0.smithi100.stdout: "target_max_objects": 0, 2024-04-07T17:51:29.715 INFO:tasks.workunit.client.0.smithi100.stdout: "type": "replicated" 2024-04-07T17:51:29.715 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.715 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:29.715 INFO:tasks.workunit.client.0.smithi100.stdout: "application": [ 2024-04-07T17:51:29.715 INFO:tasks.workunit.client.0.smithi100.stdout: "cephfs" 2024-04-07T17:51:29.715 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:29.715 INFO:tasks.workunit.client.0.smithi100.stdout: "cache_mode": "none", 2024-04-07T17:51:29.715 INFO:tasks.workunit.client.0.smithi100.stdout: "erasure_code_profile": {}, 2024-04-07T17:51:29.715 INFO:tasks.workunit.client.0.smithi100.stdout: "flags_names": [ 2024-04-07T17:51:29.715 INFO:tasks.workunit.client.0.smithi100.stdout: "hashpspool" 2024-04-07T17:51:29.715 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:29.715 INFO:tasks.workunit.client.0.smithi100.stdout: "min_size": 2, 2024-04-07T17:51:29.715 INFO:tasks.workunit.client.0.smithi100.stdout: "options": {}, 2024-04-07T17:51:29.715 INFO:tasks.workunit.client.0.smithi100.stdout: "pg_autoscale_mode": "off", 2024-04-07T17:51:29.715 INFO:tasks.workunit.client.0.smithi100.stdout: "pg_num": 32, 2024-04-07T17:51:29.716 INFO:tasks.workunit.client.0.smithi100.stdout: "pgp_num": 32, 2024-04-07T17:51:29.716 INFO:tasks.workunit.client.0.smithi100.stdout: "pool": 2, 2024-04-07T17:51:29.716 INFO:tasks.workunit.client.0.smithi100.stdout: "size": 3, 2024-04-07T17:51:29.716 INFO:tasks.workunit.client.0.smithi100.stdout: "stats": { 2024-04-07T17:51:29.716 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_raw": 723597404426, 2024-04-07T17:51:29.716 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_used": 159173, 2024-04-07T17:51:29.716 INFO:tasks.workunit.client.0.smithi100.stdout: "compress_bytes_used": 0, 2024-04-07T17:51:29.716 INFO:tasks.workunit.client.0.smithi100.stdout: "compress_under_bytes": 0, 2024-04-07T17:51:29.716 INFO:tasks.workunit.client.0.smithi100.stdout: "data_bytes_used": 110592, 2024-04-07T17:51:29.716 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty": 0, 2024-04-07T17:51:29.716 INFO:tasks.workunit.client.0.smithi100.stdout: "max_avail": 241199136768, 2024-04-07T17:51:29.716 INFO:tasks.workunit.client.0.smithi100.stdout: "objects": 22, 2024-04-07T17:51:29.716 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_bytes_used": 48581, 2024-04-07T17:51:29.716 INFO:tasks.workunit.client.0.smithi100.stdout: "percent_used": 2.1997449550781312e-07, 2024-04-07T17:51:29.716 INFO:tasks.workunit.client.0.smithi100.stdout: "quota_bytes": 0, 2024-04-07T17:51:29.717 INFO:tasks.workunit.client.0.smithi100.stdout: "quota_objects": 0, 2024-04-07T17:51:29.717 INFO:tasks.workunit.client.0.smithi100.stdout: "rd": 79, 2024-04-07T17:51:29.717 INFO:tasks.workunit.client.0.smithi100.stdout: "rd_bytes": 89088, 2024-04-07T17:51:29.717 INFO:tasks.workunit.client.0.smithi100.stdout: "stored": 22815, 2024-04-07T17:51:29.717 INFO:tasks.workunit.client.0.smithi100.stdout: "stored_data": 6622, 2024-04-07T17:51:29.717 INFO:tasks.workunit.client.0.smithi100.stdout: "stored_omap": 16193, 2024-04-07T17:51:29.717 INFO:tasks.workunit.client.0.smithi100.stdout: "stored_raw": 68445, 2024-04-07T17:51:29.717 INFO:tasks.workunit.client.0.smithi100.stdout: "wr": 68, 2024-04-07T17:51:29.717 INFO:tasks.workunit.client.0.smithi100.stdout: "wr_bytes": 36864 2024-04-07T17:51:29.717 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.717 INFO:tasks.workunit.client.0.smithi100.stdout: "target_max_bytes": 0, 2024-04-07T17:51:29.717 INFO:tasks.workunit.client.0.smithi100.stdout: "target_max_objects": 0, 2024-04-07T17:51:29.717 INFO:tasks.workunit.client.0.smithi100.stdout: "type": "replicated" 2024-04-07T17:51:29.717 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.718 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:29.718 INFO:tasks.workunit.client.0.smithi100.stdout: "application": [ 2024-04-07T17:51:29.718 INFO:tasks.workunit.client.0.smithi100.stdout: "cephfs" 2024-04-07T17:51:29.718 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:29.718 INFO:tasks.workunit.client.0.smithi100.stdout: "cache_mode": "none", 2024-04-07T17:51:29.718 INFO:tasks.workunit.client.0.smithi100.stdout: "erasure_code_profile": {}, 2024-04-07T17:51:29.718 INFO:tasks.workunit.client.0.smithi100.stdout: "flags_names": [ 2024-04-07T17:51:29.718 INFO:tasks.workunit.client.0.smithi100.stdout: "hashpspool" 2024-04-07T17:51:29.718 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:29.718 INFO:tasks.workunit.client.0.smithi100.stdout: "min_size": 2, 2024-04-07T17:51:29.718 INFO:tasks.workunit.client.0.smithi100.stdout: "options": {}, 2024-04-07T17:51:29.718 INFO:tasks.workunit.client.0.smithi100.stdout: "pg_autoscale_mode": "off", 2024-04-07T17:51:29.718 INFO:tasks.workunit.client.0.smithi100.stdout: "pg_num": 64, 2024-04-07T17:51:29.719 INFO:tasks.workunit.client.0.smithi100.stdout: "pgp_num": 64, 2024-04-07T17:51:29.719 INFO:tasks.workunit.client.0.smithi100.stdout: "pool": 3, 2024-04-07T17:51:29.719 INFO:tasks.workunit.client.0.smithi100.stdout: "size": 3, 2024-04-07T17:51:29.719 INFO:tasks.workunit.client.0.smithi100.stdout: "stats": { 2024-04-07T17:51:29.719 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_raw": 723597404426, 2024-04-07T17:51:29.719 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_used": 0, 2024-04-07T17:51:29.719 INFO:tasks.workunit.client.0.smithi100.stdout: "compress_bytes_used": 0, 2024-04-07T17:51:29.719 INFO:tasks.workunit.client.0.smithi100.stdout: "compress_under_bytes": 0, 2024-04-07T17:51:29.719 INFO:tasks.workunit.client.0.smithi100.stdout: "data_bytes_used": 0, 2024-04-07T17:51:29.719 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty": 0, 2024-04-07T17:51:29.719 INFO:tasks.workunit.client.0.smithi100.stdout: "max_avail": 241199136768, 2024-04-07T17:51:29.719 INFO:tasks.workunit.client.0.smithi100.stdout: "objects": 0, 2024-04-07T17:51:29.719 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_bytes_used": 0, 2024-04-07T17:51:29.719 INFO:tasks.workunit.client.0.smithi100.stdout: "percent_used": 0.0, 2024-04-07T17:51:29.719 INFO:tasks.workunit.client.0.smithi100.stdout: "quota_bytes": 0, 2024-04-07T17:51:29.719 INFO:tasks.workunit.client.0.smithi100.stdout: "quota_objects": 0, 2024-04-07T17:51:29.719 INFO:tasks.workunit.client.0.smithi100.stdout: "rd": 0, 2024-04-07T17:51:29.720 INFO:tasks.workunit.client.0.smithi100.stdout: "rd_bytes": 0, 2024-04-07T17:51:29.720 INFO:tasks.workunit.client.0.smithi100.stdout: "stored": 0, 2024-04-07T17:51:29.720 INFO:tasks.workunit.client.0.smithi100.stdout: "stored_data": 0, 2024-04-07T17:51:29.720 INFO:tasks.workunit.client.0.smithi100.stdout: "stored_omap": 0, 2024-04-07T17:51:29.720 INFO:tasks.workunit.client.0.smithi100.stdout: "stored_raw": 0, 2024-04-07T17:51:29.720 INFO:tasks.workunit.client.0.smithi100.stdout: "wr": 0, 2024-04-07T17:51:29.720 INFO:tasks.workunit.client.0.smithi100.stdout: "wr_bytes": 0 2024-04-07T17:51:29.720 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.720 INFO:tasks.workunit.client.0.smithi100.stdout: "target_max_bytes": 0, 2024-04-07T17:51:29.720 INFO:tasks.workunit.client.0.smithi100.stdout: "target_max_objects": 0, 2024-04-07T17:51:29.720 INFO:tasks.workunit.client.0.smithi100.stdout: "type": "replicated" 2024-04-07T17:51:29.720 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.720 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:29.720 INFO:tasks.workunit.client.0.smithi100.stdout: "application": [ 2024-04-07T17:51:29.720 INFO:tasks.workunit.client.0.smithi100.stdout: "rbd" 2024-04-07T17:51:29.720 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:29.721 INFO:tasks.workunit.client.0.smithi100.stdout: "cache_mode": "none", 2024-04-07T17:51:29.721 INFO:tasks.workunit.client.0.smithi100.stdout: "erasure_code_profile": {}, 2024-04-07T17:51:29.721 INFO:tasks.workunit.client.0.smithi100.stdout: "flags_names": [ 2024-04-07T17:51:29.721 INFO:tasks.workunit.client.0.smithi100.stdout: "hashpspool", 2024-04-07T17:51:29.721 INFO:tasks.workunit.client.0.smithi100.stdout: "selfmanaged_snaps" 2024-04-07T17:51:29.721 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:29.721 INFO:tasks.workunit.client.0.smithi100.stdout: "min_size": 2, 2024-04-07T17:51:29.721 INFO:tasks.workunit.client.0.smithi100.stdout: "options": {}, 2024-04-07T17:51:29.721 INFO:tasks.workunit.client.0.smithi100.stdout: "pg_autoscale_mode": "off", 2024-04-07T17:51:29.721 INFO:tasks.workunit.client.0.smithi100.stdout: "pg_num": 8, 2024-04-07T17:51:29.721 INFO:tasks.workunit.client.0.smithi100.stdout: "pgp_num": 8, 2024-04-07T17:51:29.721 INFO:tasks.workunit.client.0.smithi100.stdout: "pool": 4, 2024-04-07T17:51:29.721 INFO:tasks.workunit.client.0.smithi100.stdout: "size": 3, 2024-04-07T17:51:29.721 INFO:tasks.workunit.client.0.smithi100.stdout: "stats": { 2024-04-07T17:51:29.721 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_raw": 723597404426, 2024-04-07T17:51:29.721 INFO:tasks.workunit.client.0.smithi100.stdout: "bytes_used": 12595200, 2024-04-07T17:51:29.721 INFO:tasks.workunit.client.0.smithi100.stdout: "compress_bytes_used": 0, 2024-04-07T17:51:29.722 INFO:tasks.workunit.client.0.smithi100.stdout: "compress_under_bytes": 0, 2024-04-07T17:51:29.722 INFO:tasks.workunit.client.0.smithi100.stdout: "data_bytes_used": 12595200, 2024-04-07T17:51:29.722 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty": 0, 2024-04-07T17:51:29.722 INFO:tasks.workunit.client.0.smithi100.stdout: "max_avail": 241199136768, 2024-04-07T17:51:29.722 INFO:tasks.workunit.client.0.smithi100.stdout: "objects": 4, 2024-04-07T17:51:29.722 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_bytes_used": 0, 2024-04-07T17:51:29.722 INFO:tasks.workunit.client.0.smithi100.stdout: "percent_used": 1.74060623976402e-05, 2024-04-07T17:51:29.722 INFO:tasks.workunit.client.0.smithi100.stdout: "quota_bytes": 0, 2024-04-07T17:51:29.722 INFO:tasks.workunit.client.0.smithi100.stdout: "quota_objects": 0, 2024-04-07T17:51:29.722 INFO:tasks.workunit.client.0.smithi100.stdout: "rd": 9098, 2024-04-07T17:51:29.722 INFO:tasks.workunit.client.0.smithi100.stdout: "rd_bytes": 3222817792, 2024-04-07T17:51:29.722 INFO:tasks.workunit.client.0.smithi100.stdout: "stored": 4194323, 2024-04-07T17:51:29.722 INFO:tasks.workunit.client.0.smithi100.stdout: "stored_data": 4194323, 2024-04-07T17:51:29.722 INFO:tasks.workunit.client.0.smithi100.stdout: "stored_omap": 0, 2024-04-07T17:51:29.722 INFO:tasks.workunit.client.0.smithi100.stdout: "stored_raw": 12582969, 2024-04-07T17:51:29.723 INFO:tasks.workunit.client.0.smithi100.stdout: "wr": 24273, 2024-04-07T17:51:29.723 INFO:tasks.workunit.client.0.smithi100.stdout: "wr_bytes": 3103659008 2024-04-07T17:51:29.723 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.723 INFO:tasks.workunit.client.0.smithi100.stdout: "target_max_bytes": 0, 2024-04-07T17:51:29.723 INFO:tasks.workunit.client.0.smithi100.stdout: "target_max_objects": 0, 2024-04-07T17:51:29.723 INFO:tasks.workunit.client.0.smithi100.stdout: "type": "replicated" 2024-04-07T17:51:29.723 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.723 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:29.723 INFO:tasks.workunit.client.0.smithi100.stdout: "rbd": { 2024-04-07T17:51:29.723 INFO:tasks.workunit.client.0.smithi100.stdout: "mirroring_by_pool": [ 2024-04-07T17:51:29.723 INFO:tasks.workunit.client.0.smithi100.stdout: false 2024-04-07T17:51:29.723 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:29.723 INFO:tasks.workunit.client.0.smithi100.stdout: "num_images_by_pool": [ 2024-04-07T17:51:29.723 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T17:51:29.723 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:29.723 INFO:tasks.workunit.client.0.smithi100.stdout: "num_pools": 1 2024-04-07T17:51:29.723 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.724 INFO:tasks.workunit.client.0.smithi100.stdout: "report_id": "ee983b25-10fa-4633-b6cf-1c3036b91b52", 2024-04-07T17:51:29.724 INFO:tasks.workunit.client.0.smithi100.stdout: "report_timestamp": "2024-04-07T17:51:28.631595", 2024-04-07T17:51:29.724 INFO:tasks.workunit.client.0.smithi100.stdout: "report_version": 1, 2024-04-07T17:51:29.724 INFO:tasks.workunit.client.0.smithi100.stdout: "rocksdb_stats": { 2024-04-07T17:51:29.724 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "7.9.2" 2024-04-07T17:51:29.724 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.724 INFO:tasks.workunit.client.0.smithi100.stdout: "rook": { 2024-04-07T17:51:29.724 INFO:tasks.workunit.client.0.smithi100.stdout: "cluster": { 2024-04-07T17:51:29.724 INFO:tasks.workunit.client.0.smithi100.stdout: "external-mode": null, 2024-04-07T17:51:29.724 INFO:tasks.workunit.client.0.smithi100.stdout: "mon": { 2024-04-07T17:51:29.724 INFO:tasks.workunit.client.0.smithi100.stdout: "allow-multiple-per-node": null, 2024-04-07T17:51:29.724 INFO:tasks.workunit.client.0.smithi100.stdout: "count": null, 2024-04-07T17:51:29.724 INFO:tasks.workunit.client.0.smithi100.stdout: "max-id": null, 2024-04-07T17:51:29.724 INFO:tasks.workunit.client.0.smithi100.stdout: "pvc": { 2024-04-07T17:51:29.724 INFO:tasks.workunit.client.0.smithi100.stdout: "enabled": null 2024-04-07T17:51:29.725 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.725 INFO:tasks.workunit.client.0.smithi100.stdout: "stretch": { 2024-04-07T17:51:29.725 INFO:tasks.workunit.client.0.smithi100.stdout: "enabled": null 2024-04-07T17:51:29.725 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.725 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.725 INFO:tasks.workunit.client.0.smithi100.stdout: "network": { 2024-04-07T17:51:29.725 INFO:tasks.workunit.client.0.smithi100.stdout: "provider": null 2024-04-07T17:51:29.725 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.725 INFO:tasks.workunit.client.0.smithi100.stdout: "storage": { 2024-04-07T17:51:29.725 INFO:tasks.workunit.client.0.smithi100.stdout: "device-set": { 2024-04-07T17:51:29.725 INFO:tasks.workunit.client.0.smithi100.stdout: "count": { 2024-04-07T17:51:29.725 INFO:tasks.workunit.client.0.smithi100.stdout: "non-portable": null, 2024-04-07T17:51:29.725 INFO:tasks.workunit.client.0.smithi100.stdout: "portable": null, 2024-04-07T17:51:29.725 INFO:tasks.workunit.client.0.smithi100.stdout: "total": null 2024-04-07T17:51:29.725 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.725 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.725 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.726 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.726 INFO:tasks.workunit.client.0.smithi100.stdout: "csi": { 2024-04-07T17:51:29.726 INFO:tasks.workunit.client.0.smithi100.stdout: "version": null 2024-04-07T17:51:29.726 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.726 INFO:tasks.workunit.client.0.smithi100.stdout: "kubernetes": { 2024-04-07T17:51:29.726 INFO:tasks.workunit.client.0.smithi100.stdout: "version": null 2024-04-07T17:51:29.726 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.726 INFO:tasks.workunit.client.0.smithi100.stdout: "node": { 2024-04-07T17:51:29.726 INFO:tasks.workunit.client.0.smithi100.stdout: "count": { 2024-04-07T17:51:29.726 INFO:tasks.workunit.client.0.smithi100.stdout: "kubernetes-total": null, 2024-04-07T17:51:29.726 INFO:tasks.workunit.client.0.smithi100.stdout: "with-ceph-daemons": null, 2024-04-07T17:51:29.726 INFO:tasks.workunit.client.0.smithi100.stdout: "with-csi-cephfs-plugin": null, 2024-04-07T17:51:29.726 INFO:tasks.workunit.client.0.smithi100.stdout: "with-csi-nfs-plugin": null, 2024-04-07T17:51:29.726 INFO:tasks.workunit.client.0.smithi100.stdout: "with-csi-rbd-plugin": null 2024-04-07T17:51:29.727 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.727 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.727 INFO:tasks.workunit.client.0.smithi100.stdout: "usage": { 2024-04-07T17:51:29.727 INFO:tasks.workunit.client.0.smithi100.stdout: "storage-class": { 2024-04-07T17:51:29.727 INFO:tasks.workunit.client.0.smithi100.stdout: "count": { 2024-04-07T17:51:29.727 INFO:tasks.workunit.client.0.smithi100.stdout: "bucket": null, 2024-04-07T17:51:29.727 INFO:tasks.workunit.client.0.smithi100.stdout: "cephfs": null, 2024-04-07T17:51:29.727 INFO:tasks.workunit.client.0.smithi100.stdout: "nfs": null, 2024-04-07T17:51:29.727 INFO:tasks.workunit.client.0.smithi100.stdout: "rbd": null, 2024-04-07T17:51:29.727 INFO:tasks.workunit.client.0.smithi100.stdout: "total": null 2024-04-07T17:51:29.727 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.727 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:29.727 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.727 INFO:tasks.workunit.client.0.smithi100.stdout: "version": null 2024-04-07T17:51:29.727 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.727 INFO:tasks.workunit.client.0.smithi100.stdout: "services": { 2024-04-07T17:51:29.727 INFO:tasks.workunit.client.0.smithi100.stdout: "mon": 1 2024-04-07T17:51:29.727 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.728 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_per_pg": [ 2024-04-07T17:51:29.728 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:29.728 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:29.728 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:29.728 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:29.728 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T17:51:29.728 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:29.728 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 6, 2024-04-07T17:51:29.728 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:29.728 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:29.728 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:29.728 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:29.728 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:29.728 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:29.728 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.371282+0000", 2024-04-07T17:51:29.728 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:08.962622+0000", 2024-04-07T17:51:29.728 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:08.962622+0000", 2024-04-07T17:51:29.729 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:08.963061+0000", 2024-04-07T17:51:29.729 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.371282+0000", 2024-04-07T17:51:29.729 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:29.729 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:29.729 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:29.729 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 554, 2024-04-07T17:51:29.729 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.371282+0000", 2024-04-07T17:51:29.729 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.371282+0000", 2024-04-07T17:51:29.729 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.371282+0000", 2024-04-07T17:51:29.729 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:29.729 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:29.729 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:29.729 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.371282+0000", 2024-04-07T17:51:29.729 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.371282+0000", 2024-04-07T17:51:29.729 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:29.729 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:29.730 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:29.730 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:29.730 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 553, 2024-04-07T17:51:29.730 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:29.730 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:29.730 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:29.730 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:29.730 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:29.730 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:29.730 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:29.730 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:29.730 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.2c", 2024-04-07T17:51:29.730 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:29.730 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:29.731 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:29.731 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4846, 2024-04-07T17:51:29.731 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:29.731 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T02:53:55.780425+0000", 2024-04-07T17:51:29.731 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:29.731 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:29.731 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:29.731 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:29.731 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:29.731 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:29.731 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:29.731 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:29.731 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:29.731 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:29.731 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:29.731 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:29.731 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:29.732 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:29.732 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:29.732 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:29.732 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:29.732 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:29.732 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:29.732 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:29.732 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:29.732 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:29.732 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:29.732 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:29.732 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:29.732 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:29.732 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:29.732 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:29.732 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:29.733 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:29.733 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:29.733 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:29.733 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:29.733 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:29.733 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:29.733 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:29.733 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:29.733 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:29.733 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:29.733 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:29.733 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:29.733 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:29.733 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:29.733 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.733 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:29.734 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:29.734 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:29.734 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:29.734 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:29.734 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T17:51:29.734 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:29.734 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 6, 2024-04-07T17:51:29.734 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:29.734 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.734 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:29.734 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:29.734 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:29.734 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:29.734 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T17:51:29.735 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:29.735 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 0, 2024-04-07T17:51:29.735 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:29.735 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:29.735 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:29.735 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:29.735 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:29.735 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:29.735 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:35.989606+0000", 2024-04-07T17:51:29.735 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:39.003945+0000", 2024-04-07T17:51:29.735 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:39.003945+0000", 2024-04-07T17:51:29.735 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:39.004542+0000", 2024-04-07T17:51:29.735 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:35.989606+0000", 2024-04-07T17:51:29.735 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:29.735 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:29.736 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:29.736 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:29.736 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:35.989606+0000", 2024-04-07T17:51:29.736 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:35.989606+0000", 2024-04-07T17:51:29.736 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:35.989606+0000", 2024-04-07T17:51:29.736 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:29.736 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:29.736 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:29.736 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:35.989606+0000", 2024-04-07T17:51:29.736 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:35.989606+0000", 2024-04-07T17:51:29.736 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:29.736 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:29.736 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:29.736 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:29.736 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:29.737 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:29.737 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:29.737 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:29.737 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:29.737 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:29.737 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:29.737 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:29.737 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:29.737 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.2b", 2024-04-07T17:51:29.737 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:29.737 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:29.737 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:29.737 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4997, 2024-04-07T17:51:29.737 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:29.737 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T18:56:04.984002+0000", 2024-04-07T17:51:29.738 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:29.738 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:29.738 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:29.738 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:29.738 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:29.738 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:29.738 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:29.738 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:29.738 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:29.738 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:29.738 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:29.738 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:29.738 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:29.738 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:29.738 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:29.739 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:29.739 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:29.739 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:29.739 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:29.739 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:29.739 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:29.739 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:29.739 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:29.739 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:29.739 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:29.739 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:29.739 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:29.739 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:29.739 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:29.739 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:29.739 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:29.740 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:29.740 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:29.740 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:29.740 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:29.740 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:29.740 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:29.740 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:29.740 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:29.740 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:29.740 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:29.740 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:29.740 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:29.740 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.740 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:29.740 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:29.741 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:29.741 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:29.741 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:29.741 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T17:51:29.741 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:29.741 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 0, 2024-04-07T17:51:29.741 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:29.741 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.741 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:29.741 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:29.741 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:29.741 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:29.741 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T17:51:29.741 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:29.741 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 5, 2024-04-07T17:51:29.742 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:29.742 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:29.742 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:29.742 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:29.742 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:29.742 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:29.742 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.369807+0000", 2024-04-07T17:51:29.742 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:38.977861+0000", 2024-04-07T17:51:29.742 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:38.977861+0000", 2024-04-07T17:51:29.742 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:38.978928+0000", 2024-04-07T17:51:29.742 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.369807+0000", 2024-04-07T17:51:29.742 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:29.742 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:29.742 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:29.743 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:29.743 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.369807+0000", 2024-04-07T17:51:29.743 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.369807+0000", 2024-04-07T17:51:29.743 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.369807+0000", 2024-04-07T17:51:29.743 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:29.743 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:29.743 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:29.743 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.369807+0000", 2024-04-07T17:51:29.743 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.369807+0000", 2024-04-07T17:51:29.743 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:29.743 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:29.743 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:29.743 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:29.743 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:29.743 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:29.744 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:29.744 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:29.744 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:29.744 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:29.744 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:29.744 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:29.744 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:29.744 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.2a", 2024-04-07T17:51:29.744 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:29.744 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:29.744 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:29.744 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4871, 2024-04-07T17:51:29.744 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:29.744 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T01:59:28.849179+0000", 2024-04-07T17:51:29.744 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:29.744 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:29.745 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:29.745 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:29.745 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:29.745 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:29.745 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:29.745 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:29.745 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:29.745 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:29.745 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:29.745 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:29.745 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:29.745 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:29.745 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:29.745 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:29.745 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:29.746 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:29.746 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:29.746 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:29.746 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:29.746 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:29.746 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:29.746 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:29.746 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:29.746 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:29.746 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:29.746 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:29.746 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:29.746 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:29.746 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:29.747 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:29.747 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:29.747 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:29.747 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:29.747 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:29.747 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:29.747 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:29.747 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:29.747 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:29.747 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:29.747 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:29.747 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:29.747 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.747 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:29.747 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:29.748 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:29.748 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:29.748 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:29.748 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T17:51:29.748 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:29.748 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 5, 2024-04-07T17:51:29.748 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:29.748 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.748 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:29.748 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:29.748 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:29.748 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:29.748 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T17:51:29.748 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:29.748 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 3, 2024-04-07T17:51:29.748 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:29.749 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:29.749 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:29.749 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:29.749 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:29.749 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:29.749 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:35.976924+0000", 2024-04-07T17:51:29.749 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:24:13.933027+0000", 2024-04-07T17:51:29.749 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:24:13.933027+0000", 2024-04-07T17:51:29.749 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:24:13.933359+0000", 2024-04-07T17:51:29.749 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:35.976924+0000", 2024-04-07T17:51:29.749 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:29.749 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:29.749 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:29.749 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 500, 2024-04-07T17:51:29.749 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:35.976924+0000", 2024-04-07T17:51:29.750 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:35.976924+0000", 2024-04-07T17:51:29.750 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:35.976924+0000", 2024-04-07T17:51:29.750 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:29.750 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:29.750 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:29.750 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:35.976924+0000", 2024-04-07T17:51:29.750 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:35.976924+0000", 2024-04-07T17:51:29.750 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:29.750 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:29.750 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:29.750 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:29.750 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 499, 2024-04-07T17:51:29.750 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:29.750 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:29.751 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:29.751 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:29.751 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:29.751 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:29.751 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:29.751 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:29.751 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.29", 2024-04-07T17:51:29.751 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:29.751 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:29.751 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:29.751 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4967, 2024-04-07T17:51:29.751 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:29.751 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T00:08:14.265846+0000", 2024-04-07T17:51:29.751 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:29.751 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:29.751 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:29.752 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:29.752 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:29.752 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:29.752 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:29.752 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:29.752 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:29.752 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:29.752 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:29.752 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:29.752 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:29.752 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:29.752 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:29.752 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:29.752 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:29.752 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:29.753 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:29.753 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:29.753 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:29.753 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:29.753 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:29.753 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:29.753 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:29.753 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:29.753 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:29.753 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:29.753 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:29.753 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:29.753 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:29.753 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:29.753 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:29.753 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:29.754 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:29.754 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:29.754 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:29.754 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:29.754 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:29.754 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:29.754 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:29.754 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:29.754 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:29.754 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.754 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:29.754 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:29.754 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:29.754 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:29.754 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:29.755 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T17:51:29.755 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:29.755 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 3, 2024-04-07T17:51:29.755 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:29.755 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.755 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:29.755 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:29.755 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:29.755 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:29.755 INFO:tasks.workunit.client.0.smithi100.stdout: 2 2024-04-07T17:51:29.755 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:29.755 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 5, 2024-04-07T17:51:29.755 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:29.755 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:29.755 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:29.756 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:29.756 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:29.756 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:29.756 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.363975+0000", 2024-04-07T17:51:29.756 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:24:41.553450+0000", 2024-04-07T17:51:29.756 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:24:41.553450+0000", 2024-04-07T17:51:29.756 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:24:41.557281+0000", 2024-04-07T17:51:29.756 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.363975+0000", 2024-04-07T17:51:29.756 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:29.756 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:29.756 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:29.756 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 527, 2024-04-07T17:51:29.756 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.363975+0000", 2024-04-07T17:51:29.756 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.363975+0000", 2024-04-07T17:51:29.757 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.363975+0000", 2024-04-07T17:51:29.757 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:29.757 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:29.757 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:29.757 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.363975+0000", 2024-04-07T17:51:29.757 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.363975+0000", 2024-04-07T17:51:29.757 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:29.757 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:29.757 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:29.757 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:29.757 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 526, 2024-04-07T17:51:29.757 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:29.757 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:29.757 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:29.757 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:29.758 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:29.758 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:29.758 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:29.758 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:29.758 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.28", 2024-04-07T17:51:29.758 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:29.758 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:29.758 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:29.758 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4884, 2024-04-07T17:51:29.758 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:29.758 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T19:49:37.795766+0000", 2024-04-07T17:51:29.758 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:29.758 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:29.758 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:29.759 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:29.759 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:29.759 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:29.759 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:29.759 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:29.759 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:29.759 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:29.759 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:29.759 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:29.759 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:29.759 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:29.759 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:29.759 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:29.759 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:29.759 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:29.760 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:29.760 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:29.760 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:29.760 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:29.760 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:29.760 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:29.760 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:29.760 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:29.760 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:29.760 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:29.760 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:29.760 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:29.760 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:29.760 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:29.760 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:29.761 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:29.761 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:29.761 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:29.761 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:29.761 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:29.761 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:29.761 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:29.761 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:29.761 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:29.761 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:29.761 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.761 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:29.761 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:29.761 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:29.761 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:29.761 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:29.762 INFO:tasks.workunit.client.0.smithi100.stdout: 2 2024-04-07T17:51:29.762 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:29.762 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 5, 2024-04-07T17:51:29.762 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:29.762 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.762 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:29.762 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:29.762 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:29.762 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:29.762 INFO:tasks.workunit.client.0.smithi100.stdout: 2 2024-04-07T17:51:29.762 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:29.762 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 3, 2024-04-07T17:51:29.762 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:29.762 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:29.762 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:29.763 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:29.763 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:29.763 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:29.763 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:35.982188+0000", 2024-04-07T17:51:29.763 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:24:13.933691+0000", 2024-04-07T17:51:29.763 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:24:13.933691+0000", 2024-04-07T17:51:29.763 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:24:13.934646+0000", 2024-04-07T17:51:29.763 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:35.982188+0000", 2024-04-07T17:51:29.763 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:29.763 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:29.763 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:29.763 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 500, 2024-04-07T17:51:29.763 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:35.982188+0000", 2024-04-07T17:51:29.763 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:35.982188+0000", 2024-04-07T17:51:29.763 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:35.982188+0000", 2024-04-07T17:51:29.763 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:29.764 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:29.764 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:29.764 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:35.982188+0000", 2024-04-07T17:51:29.764 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:35.982188+0000", 2024-04-07T17:51:29.764 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:29.764 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:29.764 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:29.764 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:29.764 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 499, 2024-04-07T17:51:29.764 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:29.764 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:29.764 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:29.764 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:29.764 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:29.764 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:29.764 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:29.764 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:29.764 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.27", 2024-04-07T17:51:29.765 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:29.765 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:29.765 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:29.765 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4954, 2024-04-07T17:51:29.765 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:29.765 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T19:33:33.960559+0000", 2024-04-07T17:51:29.765 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:29.765 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:29.765 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:29.765 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:29.765 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:29.765 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:29.765 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:29.765 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:29.765 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:29.765 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:29.766 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:29.766 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:29.766 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:29.766 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:29.766 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:29.766 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:29.766 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:29.766 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:29.766 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:29.766 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:29.766 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:29.766 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:29.766 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:29.766 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:29.767 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:29.767 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:29.767 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:29.767 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:29.767 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:29.767 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:29.767 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:29.767 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:29.767 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:29.767 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:29.767 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:29.767 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:29.767 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:29.767 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:29.767 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:29.767 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:29.768 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:29.768 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:29.768 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:29.768 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.768 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:29.768 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:29.768 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:29.768 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:29.768 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:29.768 INFO:tasks.workunit.client.0.smithi100.stdout: 2 2024-04-07T17:51:29.768 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:29.768 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 3, 2024-04-07T17:51:29.768 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:29.768 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.768 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:29.769 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:29.769 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:29.769 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:29.769 INFO:tasks.workunit.client.0.smithi100.stdout: 4 2024-04-07T17:51:29.769 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:29.769 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 6, 2024-04-07T17:51:29.769 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:29.769 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:29.769 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:29.769 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:29.769 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:29.769 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:29.769 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.368356+0000", 2024-04-07T17:51:29.769 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:38.960595+0000", 2024-04-07T17:51:29.769 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:38.960595+0000", 2024-04-07T17:51:29.769 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:38.962091+0000", 2024-04-07T17:51:29.770 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.368356+0000", 2024-04-07T17:51:29.770 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:29.770 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:29.770 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:29.770 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:29.770 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.368356+0000", 2024-04-07T17:51:29.770 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.368356+0000", 2024-04-07T17:51:29.770 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.368356+0000", 2024-04-07T17:51:29.770 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:29.770 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:29.770 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:29.770 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.368356+0000", 2024-04-07T17:51:29.770 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.368356+0000", 2024-04-07T17:51:29.770 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:29.770 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:29.771 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:29.771 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:29.771 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:29.771 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:29.771 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:29.771 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:29.771 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:29.771 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:29.771 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:29.771 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:29.771 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:29.771 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.26", 2024-04-07T17:51:29.771 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:29.771 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:29.771 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:29.772 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4841, 2024-04-07T17:51:29.772 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:29.772 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T19:11:01.462021+0000", 2024-04-07T17:51:29.772 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:29.772 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:29.772 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:29.772 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:29.772 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:29.772 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:29.772 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:29.772 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:29.772 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:29.772 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:29.772 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:29.772 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:29.773 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:29.773 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:29.773 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:29.773 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:29.773 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:29.773 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:29.773 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:29.773 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:29.773 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:29.773 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:29.773 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:29.773 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:29.773 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:29.773 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:29.773 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:29.774 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:29.774 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:29.774 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:29.774 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:29.774 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:29.774 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:29.774 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:29.774 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:29.774 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:29.774 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:29.774 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:29.774 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:29.774 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:29.774 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:29.775 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:29.775 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:29.775 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.775 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:29.775 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:29.775 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:29.775 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:29.775 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:29.775 INFO:tasks.workunit.client.0.smithi100.stdout: 4 2024-04-07T17:51:29.775 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:29.775 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 6, 2024-04-07T17:51:29.775 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:29.775 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.775 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:29.776 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:29.776 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:29.776 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:29.776 INFO:tasks.workunit.client.0.smithi100.stdout: 2 2024-04-07T17:51:29.776 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:29.776 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 1, 2024-04-07T17:51:29.776 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:29.776 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:29.776 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:29.776 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:29.776 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:29.776 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:29.776 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.334473+0000", 2024-04-07T17:51:29.776 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:23:43.924498+0000", 2024-04-07T17:51:29.776 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:23:43.924498+0000", 2024-04-07T17:51:29.777 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:23:43.926295+0000", 2024-04-07T17:51:29.777 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.334473+0000", 2024-04-07T17:51:29.777 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:29.777 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:29.777 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:29.777 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 471, 2024-04-07T17:51:29.777 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.334473+0000", 2024-04-07T17:51:29.777 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.334473+0000", 2024-04-07T17:51:29.777 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.334473+0000", 2024-04-07T17:51:29.777 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:29.777 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:29.777 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:29.777 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.334473+0000", 2024-04-07T17:51:29.777 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.334473+0000", 2024-04-07T17:51:29.777 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:29.778 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:29.778 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:29.778 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:29.778 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 470, 2024-04-07T17:51:29.778 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:29.778 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:29.778 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:29.778 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:29.778 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:29.778 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:29.778 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:29.778 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:29.778 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.25", 2024-04-07T17:51:29.778 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:29.778 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:29.779 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:29.779 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4981, 2024-04-07T17:51:29.779 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:29.779 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T17:58:42.938076+0000", 2024-04-07T17:51:29.779 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:29.779 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:29.779 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:29.779 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:29.779 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:29.779 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:29.779 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:29.779 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:29.779 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:29.779 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:29.779 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:29.780 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:29.780 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:29.780 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:29.780 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:29.780 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:29.780 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:29.780 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:29.780 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:29.780 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:29.780 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:29.780 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:29.780 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:29.780 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:29.780 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:29.780 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:29.781 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:29.781 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:29.781 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:29.781 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:29.781 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:29.781 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:29.781 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:29.781 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:29.781 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:29.781 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:29.781 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:29.781 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:29.781 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:29.781 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:29.781 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:29.781 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:29.782 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:29.782 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.782 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:29.782 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:29.782 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:29.782 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:29.782 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:29.782 INFO:tasks.workunit.client.0.smithi100.stdout: 2 2024-04-07T17:51:29.782 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:29.782 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 1, 2024-04-07T17:51:29.782 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:29.782 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.782 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:29.782 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:29.782 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:29.783 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:29.783 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T17:51:29.783 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:29.783 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 3, 2024-04-07T17:51:29.783 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:29.783 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:29.783 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:29.783 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:29.783 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:29.783 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:29.783 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:35.977918+0000", 2024-04-07T17:51:29.783 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:24:13.930420+0000", 2024-04-07T17:51:29.783 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:24:13.930420+0000", 2024-04-07T17:51:29.783 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:24:13.931275+0000", 2024-04-07T17:51:29.783 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:35.977918+0000", 2024-04-07T17:51:29.784 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:29.784 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:29.784 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:29.784 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 500, 2024-04-07T17:51:29.784 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:35.977918+0000", 2024-04-07T17:51:29.784 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:35.977918+0000", 2024-04-07T17:51:29.784 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:35.977918+0000", 2024-04-07T17:51:29.784 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:29.784 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:29.784 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:29.784 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:35.977918+0000", 2024-04-07T17:51:29.784 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:35.977918+0000", 2024-04-07T17:51:29.784 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:29.784 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:29.784 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:29.784 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:29.785 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 499, 2024-04-07T17:51:29.785 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:29.785 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:29.785 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:29.785 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:29.785 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:29.785 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:29.785 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:29.785 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:29.785 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.24", 2024-04-07T17:51:29.785 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:29.785 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:29.785 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:29.785 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4940, 2024-04-07T17:51:29.785 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:29.786 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T00:26:09.028937+0000", 2024-04-07T17:51:29.786 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:29.786 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:29.786 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:29.786 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:29.786 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:29.786 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:29.786 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:29.786 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:29.786 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:29.786 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:29.786 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:29.786 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:29.786 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:29.786 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:29.786 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:29.787 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:29.787 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:29.787 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:29.787 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:29.787 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:29.787 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:29.787 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:29.787 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:29.787 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:29.787 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:29.787 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:29.787 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:29.787 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:29.787 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:29.787 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:29.788 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:29.788 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:29.788 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:29.788 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:29.788 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:29.788 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:29.788 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:29.788 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:29.788 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:29.788 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:29.788 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:29.788 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:29.788 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:29.788 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.788 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:29.789 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:29.789 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:29.789 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:29.789 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:29.789 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T17:51:29.789 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:29.789 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 3, 2024-04-07T17:51:29.789 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:29.789 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.789 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:29.789 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:29.789 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:29.789 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:29.789 INFO:tasks.workunit.client.0.smithi100.stdout: 4 2024-04-07T17:51:29.790 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:29.790 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 0, 2024-04-07T17:51:29.790 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:29.790 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:29.790 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:29.790 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:29.790 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:29.790 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:29.790 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:35.989350+0000", 2024-04-07T17:51:29.790 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:24:13.962893+0000", 2024-04-07T17:51:29.790 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:24:13.962893+0000", 2024-04-07T17:51:29.790 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:24:13.964515+0000", 2024-04-07T17:51:29.790 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:35.989350+0000", 2024-04-07T17:51:29.790 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:29.790 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:29.791 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:29.791 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 500, 2024-04-07T17:51:29.791 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:35.989350+0000", 2024-04-07T17:51:29.791 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:35.989350+0000", 2024-04-07T17:51:29.791 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:35.989350+0000", 2024-04-07T17:51:29.791 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:29.791 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:29.791 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:29.791 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:35.989350+0000", 2024-04-07T17:51:29.791 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:35.989350+0000", 2024-04-07T17:51:29.791 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:29.791 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:29.791 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:29.791 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:29.791 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 499, 2024-04-07T17:51:29.792 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:29.792 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:29.792 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:29.792 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:29.792 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:29.792 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:29.792 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:29.792 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:29.792 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.23", 2024-04-07T17:51:29.792 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:29.792 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:29.792 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:29.792 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 5000, 2024-04-07T17:51:29.792 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:29.792 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T17:30:33.589121+0000", 2024-04-07T17:51:29.793 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:29.793 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:29.793 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:29.793 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:29.793 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:29.793 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:29.793 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:29.793 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:29.793 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:29.793 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:29.793 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:29.793 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:29.793 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:29.793 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:29.793 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:29.793 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:29.794 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:29.794 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:29.794 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:29.794 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:29.794 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:29.794 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:29.794 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:29.794 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:29.794 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:29.794 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:29.794 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:29.794 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:29.794 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:29.794 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:29.795 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:29.795 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:29.795 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:29.795 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:29.795 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:29.795 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:29.795 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:29.795 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:29.795 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:29.795 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:29.795 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:29.795 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:29.795 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:29.795 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.796 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:29.796 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:29.796 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:29.796 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:29.796 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:29.796 INFO:tasks.workunit.client.0.smithi100.stdout: 4 2024-04-07T17:51:29.796 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:29.796 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 0, 2024-04-07T17:51:29.796 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:29.796 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.796 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:29.796 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:29.796 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:29.796 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:29.796 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T17:51:29.796 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:29.797 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 5, 2024-04-07T17:51:29.797 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:29.797 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:29.797 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:29.797 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:29.797 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:29.797 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:29.797 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.369012+0000", 2024-04-07T17:51:29.797 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:24:41.561766+0000", 2024-04-07T17:51:29.797 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:24:41.561766+0000", 2024-04-07T17:51:29.797 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:24:41.564435+0000", 2024-04-07T17:51:29.797 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.369012+0000", 2024-04-07T17:51:29.797 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:29.797 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:29.797 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:29.798 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 527, 2024-04-07T17:51:29.798 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.369012+0000", 2024-04-07T17:51:29.798 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.369012+0000", 2024-04-07T17:51:29.798 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.369012+0000", 2024-04-07T17:51:29.798 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:29.798 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:29.798 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:29.798 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.369012+0000", 2024-04-07T17:51:29.798 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.369012+0000", 2024-04-07T17:51:29.798 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:29.798 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:29.798 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:29.798 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:29.798 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 526, 2024-04-07T17:51:29.799 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:29.799 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:29.799 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:29.799 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:29.799 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:29.799 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:29.799 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:29.799 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:29.799 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.22", 2024-04-07T17:51:29.799 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:29.799 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:29.799 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:29.799 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4886, 2024-04-07T17:51:29.799 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:29.799 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T17:08:34.046549+0000", 2024-04-07T17:51:29.799 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:29.800 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:29.800 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:29.800 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:29.800 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:29.800 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:29.800 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:29.800 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:29.800 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:29.800 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:29.800 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:29.800 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:29.800 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:29.800 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:29.800 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:29.800 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:29.801 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:29.801 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:29.801 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:29.801 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:29.801 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:29.801 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:29.801 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:29.801 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:29.801 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:29.801 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:29.801 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:29.801 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:29.801 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:29.801 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:29.801 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:29.801 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:29.802 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:29.802 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:29.802 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:29.802 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:29.802 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:29.802 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:29.802 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:29.802 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:29.802 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:29.802 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:29.802 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:29.802 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.802 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:29.803 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:29.803 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:29.803 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:29.803 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:29.803 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T17:51:29.803 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:29.803 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 5, 2024-04-07T17:51:29.803 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:29.803 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.803 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:29.803 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:29.803 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T17:51:29.803 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:29.803 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T17:51:29.804 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:29.804 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 2, 2024-04-07T17:51:29.804 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:29.804 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:29.804 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:29.804 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:29.804 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:29.804 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:29.804 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.354974+0000", 2024-04-07T17:51:29.804 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:08.976322+0000", 2024-04-07T17:51:29.804 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:08.976322+0000", 2024-04-07T17:51:29.804 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:08.976897+0000", 2024-04-07T17:51:29.804 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.354974+0000", 2024-04-07T17:51:29.804 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:29.804 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:29.805 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:29.805 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 554, 2024-04-07T17:51:29.805 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.354974+0000", 2024-04-07T17:51:29.805 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.354974+0000", 2024-04-07T17:51:29.805 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.354974+0000", 2024-04-07T17:51:29.805 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:29.805 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:29.805 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:29.805 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.354974+0000", 2024-04-07T17:51:29.805 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.354974+0000", 2024-04-07T17:51:29.805 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:29.805 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:29.805 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:29.805 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:29.806 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 553, 2024-04-07T17:51:29.806 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:29.806 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:29.806 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:29.806 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:29.806 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:29.806 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:29.806 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:29.806 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:29.806 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.21", 2024-04-07T17:51:29.806 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:29.806 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:29.806 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:29.806 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4970, 2024-04-07T17:51:29.806 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:29.807 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T18:02:01.757959+0000", 2024-04-07T17:51:29.807 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:29.807 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:29.807 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:29.807 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:29.807 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:29.807 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:29.807 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:29.807 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:29.807 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:29.807 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:29.807 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:29.807 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:29.807 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:29.807 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:29.808 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:29.808 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:29.808 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:29.808 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:29.808 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:29.808 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:29.808 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:29.808 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:29.808 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:29.808 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:29.808 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:29.808 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:29.808 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:29.808 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:29.808 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:29.809 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:29.809 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:29.809 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:29.809 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:29.809 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:29.809 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:29.809 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:29.809 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:29.809 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:29.809 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:29.809 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:29.809 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:29.809 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:29.809 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:29.809 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.809 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:29.810 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:29.810 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:29.810 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T17:51:29.810 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:29.810 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T17:51:29.810 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:29.810 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 2, 2024-04-07T17:51:29.810 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:29.810 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.810 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:29.810 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:29.810 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:29.810 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:29.810 INFO:tasks.workunit.client.0.smithi100.stdout: 5 2024-04-07T17:51:29.811 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:29.811 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 6, 2024-04-07T17:51:29.811 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:29.811 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:29.811 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:29.811 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:29.811 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:29.811 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:29.811 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.370975+0000", 2024-04-07T17:51:29.811 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:08.975767+0000", 2024-04-07T17:51:29.811 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:08.975767+0000", 2024-04-07T17:51:29.811 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:08.976755+0000", 2024-04-07T17:51:29.811 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.370975+0000", 2024-04-07T17:51:29.811 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:29.811 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:29.811 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:29.812 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 554, 2024-04-07T17:51:29.812 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.370975+0000", 2024-04-07T17:51:29.812 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.370975+0000", 2024-04-07T17:51:29.812 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.370975+0000", 2024-04-07T17:51:29.812 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:29.812 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:29.812 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:29.812 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.370975+0000", 2024-04-07T17:51:29.812 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.370975+0000", 2024-04-07T17:51:29.812 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:29.812 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:29.812 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:29.812 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:29.812 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 553, 2024-04-07T17:51:29.812 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:29.812 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:29.812 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:29.813 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:29.813 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:29.813 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:29.813 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:29.813 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:29.813 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.20", 2024-04-07T17:51:29.813 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:29.813 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:29.813 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:29.813 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4834, 2024-04-07T17:51:29.813 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:29.813 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T16:33:00.288075+0000", 2024-04-07T17:51:29.813 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:29.813 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:29.813 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:29.814 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:29.814 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:29.814 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:29.814 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:29.814 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:29.814 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:29.814 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:29.814 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:29.814 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:29.814 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:29.814 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:29.814 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:29.814 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:29.814 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:29.814 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:29.815 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:29.815 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:29.815 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:29.815 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:29.815 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:29.815 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:29.815 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:29.815 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:29.815 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:29.815 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:29.815 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:29.815 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:29.815 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:29.815 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:29.815 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:29.816 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:29.816 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:29.816 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:29.816 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:29.816 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:29.816 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:29.816 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:29.816 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:29.816 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:29.816 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:29.816 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.816 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:29.816 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:29.816 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:29.816 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:29.817 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:29.817 INFO:tasks.workunit.client.0.smithi100.stdout: 5 2024-04-07T17:51:29.817 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:29.817 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 6, 2024-04-07T17:51:29.817 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:29.817 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.817 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:29.817 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:29.817 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:29.817 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:29.817 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T17:51:29.817 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:29.817 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 7, 2024-04-07T17:51:29.817 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:29.818 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:29.818 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:29.818 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:29.818 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:29.818 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:29.818 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.367504+0000", 2024-04-07T17:51:29.818 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:38.976388+0000", 2024-04-07T17:51:29.818 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:38.976388+0000", 2024-04-07T17:51:29.818 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:38.994496+0000", 2024-04-07T17:51:29.818 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.367504+0000", 2024-04-07T17:51:29.818 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:29.818 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:29.818 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:29.818 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:29.819 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.367504+0000", 2024-04-07T17:51:29.819 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.367504+0000", 2024-04-07T17:51:29.819 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.367504+0000", 2024-04-07T17:51:29.819 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:29.819 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:29.819 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:29.819 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.367504+0000", 2024-04-07T17:51:29.819 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.367504+0000", 2024-04-07T17:51:29.819 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:29.819 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:29.819 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:29.819 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:29.819 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:29.819 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:29.819 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:29.820 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:29.820 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:29.820 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:29.820 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:29.820 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:29.820 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:29.820 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.f", 2024-04-07T17:51:29.820 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:29.820 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:29.820 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:29.820 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4790, 2024-04-07T17:51:29.820 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:29.820 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T17:59:57.662112+0000", 2024-04-07T17:51:29.820 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:29.820 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:29.820 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:29.821 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:29.821 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:29.821 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:29.821 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:29.821 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:29.821 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:29.821 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:29.821 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:29.821 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:29.821 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:29.821 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:29.821 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:29.821 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:29.821 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:29.821 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:29.821 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:29.822 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:29.822 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:29.822 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:29.822 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:29.822 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:29.822 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:29.822 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:29.822 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:29.822 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:29.822 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:29.822 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:29.822 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:29.822 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:29.822 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:29.823 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:29.823 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:29.823 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:29.823 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:29.823 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:29.823 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:29.823 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:29.823 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:29.823 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:29.823 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:29.823 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.823 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:29.823 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:29.823 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:29.823 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:29.823 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:29.824 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T17:51:29.824 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:29.824 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 7, 2024-04-07T17:51:29.824 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:29.824 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.824 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:29.824 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:29.824 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T17:51:29.824 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:29.824 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T17:51:29.824 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:29.824 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 2, 2024-04-07T17:51:29.824 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:29.824 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:29.824 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T17:51:29.824 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:29.825 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:29.825 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:29.825 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.359582+0000", 2024-04-07T17:51:29.825 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:38.967914+0000", 2024-04-07T17:51:29.825 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:38.967914+0000", 2024-04-07T17:51:29.825 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:38.968390+0000", 2024-04-07T17:51:29.825 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.359582+0000", 2024-04-07T17:51:29.825 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:29.825 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:29.825 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:29.825 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:29.825 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.359582+0000", 2024-04-07T17:51:29.825 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.359582+0000", 2024-04-07T17:51:29.825 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.359582+0000", 2024-04-07T17:51:29.826 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:29.826 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:29.826 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:29.826 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.359582+0000", 2024-04-07T17:51:29.826 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.359582+0000", 2024-04-07T17:51:29.826 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:29.826 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 2, 2024-04-07T17:51:29.826 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:29.826 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:29.826 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:29.826 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:29.826 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:29.826 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:29.826 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:29.826 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 2, 2024-04-07T17:51:29.827 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:29.827 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:29.827 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:29.827 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.e", 2024-04-07T17:51:29.827 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:29.827 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:29.827 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:29.827 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4971, 2024-04-07T17:51:29.827 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:29.827 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T21:28:08.591091+0000", 2024-04-07T17:51:29.827 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:29.827 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:29.827 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:29.827 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:29.827 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:29.827 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:29.828 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:29.828 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:29.828 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:29.828 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:29.828 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:29.828 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:29.828 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:29.828 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:29.828 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:29.828 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:29.828 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:29.828 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:29.828 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:29.828 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 3, 2024-04-07T17:51:29.828 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 1, 2024-04-07T17:51:29.828 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:29.829 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 1, 2024-04-07T17:51:29.829 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:29.829 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:29.829 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:29.829 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:29.829 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:29.829 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 1, 2024-04-07T17:51:29.829 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:29.829 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:29.829 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:29.829 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:29.829 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:29.829 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:29.829 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:29.829 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 6, 2024-04-07T17:51:29.830 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 6, 2024-04-07T17:51:29.830 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:29.830 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:29.830 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:29.830 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 3, 2024-04-07T17:51:29.830 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:29.830 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.830 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:29.830 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:29.830 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:29.830 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T17:51:29.830 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:29.830 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T17:51:29.830 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:29.830 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 2, 2024-04-07T17:51:29.831 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "61'2" 2024-04-07T17:51:29.831 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.831 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:29.831 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:29.831 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:29.831 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:29.831 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T17:51:29.831 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:29.831 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 5, 2024-04-07T17:51:29.831 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:29.831 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:29.831 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:29.831 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:29.831 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:29.831 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:29.832 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.366524+0000", 2024-04-07T17:51:29.832 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:08.971755+0000", 2024-04-07T17:51:29.832 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:08.971755+0000", 2024-04-07T17:51:29.832 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:08.973166+0000", 2024-04-07T17:51:29.832 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.366524+0000", 2024-04-07T17:51:29.832 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:29.832 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:29.832 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:29.832 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 554, 2024-04-07T17:51:29.832 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.366524+0000", 2024-04-07T17:51:29.832 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.366524+0000", 2024-04-07T17:51:29.832 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.366524+0000", 2024-04-07T17:51:29.832 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:29.832 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:29.832 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:29.833 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.366524+0000", 2024-04-07T17:51:29.833 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.366524+0000", 2024-04-07T17:51:29.833 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:29.833 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:29.833 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:29.833 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:29.833 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 553, 2024-04-07T17:51:29.833 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:29.833 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:29.833 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:29.833 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:29.833 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:29.833 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:29.833 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:29.833 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:29.833 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.c", 2024-04-07T17:51:29.834 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:29.834 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:29.834 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:29.834 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4877, 2024-04-07T17:51:29.834 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:29.834 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T02:59:49.583756+0000", 2024-04-07T17:51:29.834 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:29.834 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:29.834 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:29.834 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:29.834 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:29.834 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:29.834 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:29.834 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:29.834 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:29.835 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:29.835 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:29.835 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:29.835 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:29.835 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:29.835 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:29.835 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:29.835 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:29.835 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:29.835 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:29.835 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:29.835 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:29.835 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:29.835 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:29.835 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:29.836 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:29.836 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:29.836 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:29.836 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:29.836 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:29.836 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:29.836 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:29.836 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:29.836 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:29.836 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:29.836 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:29.836 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:29.836 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:29.836 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:29.836 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:29.836 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:29.837 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:29.837 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:29.837 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:29.837 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.837 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:29.837 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:29.837 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:29.837 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:29.837 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:29.837 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T17:51:29.837 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:29.837 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 5, 2024-04-07T17:51:29.837 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:29.837 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.837 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:29.838 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:29.838 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:29.838 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:29.838 INFO:tasks.workunit.client.0.smithi100.stdout: 3 2024-04-07T17:51:29.838 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:29.838 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 1, 2024-04-07T17:51:29.838 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:29.838 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:29.838 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T17:51:29.838 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:29.838 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:29.838 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:29.838 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.333944+0000", 2024-04-07T17:51:29.838 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:24:13.932662+0000", 2024-04-07T17:51:29.839 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:24:13.932662+0000", 2024-04-07T17:51:29.839 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:24:13.933216+0000", 2024-04-07T17:51:29.839 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.333944+0000", 2024-04-07T17:51:29.839 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:29.839 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:29.839 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:29.839 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 500, 2024-04-07T17:51:29.839 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.333944+0000", 2024-04-07T17:51:29.839 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.333944+0000", 2024-04-07T17:51:29.839 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.333944+0000", 2024-04-07T17:51:29.839 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:29.839 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:29.839 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:29.839 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.333944+0000", 2024-04-07T17:51:29.839 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.333944+0000", 2024-04-07T17:51:29.839 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:29.840 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 1, 2024-04-07T17:51:29.840 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:29.840 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:29.840 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 499, 2024-04-07T17:51:29.840 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:29.840 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:29.840 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:29.840 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:29.840 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 1, 2024-04-07T17:51:29.840 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:29.840 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:29.840 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:29.840 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.d", 2024-04-07T17:51:29.840 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:29.840 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:29.841 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:29.841 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4983, 2024-04-07T17:51:29.841 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:29.841 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T17:17:11.134012+0000", 2024-04-07T17:51:29.841 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:29.841 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:29.841 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:29.841 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 70, 2024-04-07T17:51:29.841 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:29.841 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:29.841 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:29.841 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:29.841 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:29.841 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:29.841 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:29.841 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:29.842 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:29.842 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:29.842 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:29.842 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:29.842 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:29.842 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:29.842 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:29.842 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 3, 2024-04-07T17:51:29.842 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 1, 2024-04-07T17:51:29.842 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:29.842 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 1, 2024-04-07T17:51:29.842 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:29.842 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:29.842 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:29.842 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:29.843 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:29.843 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:29.843 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:29.843 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:29.843 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:29.843 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:29.843 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:29.843 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:29.843 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:29.843 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 1, 2024-04-07T17:51:29.843 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 1, 2024-04-07T17:51:29.843 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:29.843 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:29.843 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:29.843 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 1, 2024-04-07T17:51:29.843 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 1 2024-04-07T17:51:29.844 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.844 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:29.844 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:29.844 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:29.844 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:29.844 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:29.844 INFO:tasks.workunit.client.0.smithi100.stdout: 3 2024-04-07T17:51:29.844 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:29.844 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 1, 2024-04-07T17:51:29.844 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "61'1" 2024-04-07T17:51:29.844 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.844 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:29.844 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:29.844 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:29.844 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:29.845 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T17:51:29.845 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:29.845 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 7, 2024-04-07T17:51:29.845 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:29.845 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:29.845 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:29.845 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:29.845 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:29.845 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:29.845 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.373148+0000", 2024-04-07T17:51:29.845 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:38.989051+0000", 2024-04-07T17:51:29.845 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:38.989051+0000", 2024-04-07T17:51:29.845 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:39.000662+0000", 2024-04-07T17:51:29.845 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.373148+0000", 2024-04-07T17:51:29.845 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:29.846 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:29.846 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:29.846 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:29.846 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.373148+0000", 2024-04-07T17:51:29.846 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.373148+0000", 2024-04-07T17:51:29.846 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.373148+0000", 2024-04-07T17:51:29.846 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:29.846 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:29.846 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:29.846 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.373148+0000", 2024-04-07T17:51:29.846 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.373148+0000", 2024-04-07T17:51:29.846 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:29.846 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:29.846 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:29.847 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:29.847 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:29.847 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:29.847 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:29.847 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:29.847 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:29.847 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:29.847 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:29.847 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:29.847 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:29.847 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.d", 2024-04-07T17:51:29.847 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:29.847 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:29.847 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:29.847 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4803, 2024-04-07T17:51:29.848 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:29.848 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T23:45:21.166051+0000", 2024-04-07T17:51:29.848 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:29.848 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:29.848 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:29.848 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:29.848 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:29.848 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:29.848 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:29.848 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:29.848 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:29.848 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:29.848 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:29.848 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:29.848 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:29.848 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:29.849 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:29.849 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:29.849 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:29.849 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:29.849 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:29.849 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:29.849 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:29.849 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:29.849 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:29.849 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:29.849 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:29.849 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:29.849 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:29.849 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:29.849 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:29.850 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:29.850 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:29.850 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:29.850 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:29.850 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:29.850 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:29.850 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:29.850 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:29.850 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:29.850 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:29.850 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:29.850 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:29.850 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:29.850 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:29.850 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.851 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:29.851 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:29.851 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:29.851 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:29.851 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:29.851 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T17:51:29.851 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:29.851 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 7, 2024-04-07T17:51:29.851 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:29.851 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.851 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:29.851 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:29.851 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T17:51:29.851 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:29.851 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T17:51:29.851 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:29.852 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 2, 2024-04-07T17:51:29.852 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:29.852 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:29.852 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T17:51:29.852 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:29.852 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:29.852 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:29.852 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.355658+0000", 2024-04-07T17:51:29.852 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:24:41.549188+0000", 2024-04-07T17:51:29.852 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:24:41.549188+0000", 2024-04-07T17:51:29.852 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:24:41.550837+0000", 2024-04-07T17:51:29.852 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.355658+0000", 2024-04-07T17:51:29.852 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:29.852 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:29.852 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:29.852 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 527, 2024-04-07T17:51:29.853 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.355658+0000", 2024-04-07T17:51:29.853 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.355658+0000", 2024-04-07T17:51:29.853 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.355658+0000", 2024-04-07T17:51:29.853 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:29.853 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:29.853 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:29.853 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.355658+0000", 2024-04-07T17:51:29.853 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.355658+0000", 2024-04-07T17:51:29.853 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:29.853 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:29.853 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:29.853 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:29.853 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 526, 2024-04-07T17:51:29.853 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:29.853 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:29.853 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:29.854 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:29.854 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:29.854 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:29.854 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:29.854 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:29.854 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.c", 2024-04-07T17:51:29.854 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:29.854 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:29.854 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:29.854 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4973, 2024-04-07T17:51:29.854 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:29.854 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T18:56:44.543940+0000", 2024-04-07T17:51:29.854 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:29.854 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:29.854 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:29.854 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:29.855 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:29.855 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:29.855 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:29.855 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:29.855 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:29.855 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:29.855 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:29.855 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:29.855 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:29.855 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:29.855 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:29.855 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:29.855 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:29.855 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:29.855 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:29.856 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:29.856 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:29.856 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:29.856 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:29.856 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:29.856 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:29.856 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:29.856 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:29.856 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:29.856 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:29.856 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:29.856 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:29.856 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:29.856 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:29.857 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:29.857 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:29.857 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:29.857 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:29.857 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:29.857 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:29.857 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:29.857 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:29.857 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:29.857 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:29.857 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.857 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:29.857 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:29.857 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:29.858 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T17:51:29.858 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:29.858 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T17:51:29.858 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:29.858 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 2, 2024-04-07T17:51:29.858 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:29.858 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.858 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:29.858 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:29.858 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:29.858 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:29.858 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T17:51:29.858 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:29.858 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 6, 2024-04-07T17:51:29.858 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:29.859 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:29.859 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:29.859 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:29.859 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:29.859 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:29.859 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.386750+0000", 2024-04-07T17:51:29.859 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:08.966124+0000", 2024-04-07T17:51:29.859 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:08.966124+0000", 2024-04-07T17:51:29.859 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:08.966628+0000", 2024-04-07T17:51:29.859 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.386750+0000", 2024-04-07T17:51:29.859 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:29.859 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:29.859 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:29.859 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 554, 2024-04-07T17:51:29.859 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.386750+0000", 2024-04-07T17:51:29.860 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.386750+0000", 2024-04-07T17:51:29.860 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.386750+0000", 2024-04-07T17:51:29.860 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:29.860 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:29.860 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:29.860 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.386750+0000", 2024-04-07T17:51:29.860 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.386750+0000", 2024-04-07T17:51:29.860 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:29.860 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:29.860 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:29.860 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:29.860 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 553, 2024-04-07T17:51:29.860 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:29.860 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:29.860 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:29.861 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:29.861 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:29.861 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:29.861 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:29.861 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:29.861 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.a", 2024-04-07T17:51:29.861 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:29.861 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:29.861 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:29.861 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4847, 2024-04-07T17:51:29.861 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:29.861 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T19:57:28.644470+0000", 2024-04-07T17:51:29.861 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:29.861 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:29.861 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:29.861 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:29.862 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:29.862 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:29.862 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:29.862 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:29.862 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:29.862 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:29.862 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:29.862 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:29.862 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:29.862 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:29.862 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:29.862 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:29.862 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:29.862 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:29.863 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:29.863 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:29.863 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:29.863 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:29.863 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:29.863 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:29.863 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:29.863 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:29.863 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:29.863 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:29.863 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:29.863 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:29.863 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:29.863 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:29.863 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:29.863 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:29.864 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:29.864 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:29.864 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:29.864 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:29.864 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:29.864 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:29.864 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:29.864 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:29.864 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:29.864 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.864 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:29.864 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:29.864 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:29.864 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:29.864 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:29.864 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T17:51:29.865 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:29.865 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 6, 2024-04-07T17:51:29.865 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:29.865 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.865 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:29.865 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:29.865 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:29.865 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:29.865 INFO:tasks.workunit.client.0.smithi100.stdout: 5 2024-04-07T17:51:29.865 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:29.865 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 7, 2024-04-07T17:51:29.865 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:29.865 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:29.866 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T17:51:29.866 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:29.866 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:29.866 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:29.866 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.369146+0000", 2024-04-07T17:51:29.866 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:38.990309+0000", 2024-04-07T17:51:29.866 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:38.990309+0000", 2024-04-07T17:51:29.866 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:38.992642+0000", 2024-04-07T17:51:29.866 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.369146+0000", 2024-04-07T17:51:29.866 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:29.866 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:29.866 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:29.866 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:29.866 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.369146+0000", 2024-04-07T17:51:29.866 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.369146+0000", 2024-04-07T17:51:29.867 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.369146+0000", 2024-04-07T17:51:29.867 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:29.867 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:29.867 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:29.867 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.369146+0000", 2024-04-07T17:51:29.867 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.369146+0000", 2024-04-07T17:51:29.867 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:29.867 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 3, 2024-04-07T17:51:29.867 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:29.867 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:29.867 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:29.867 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:29.867 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:29.867 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:29.867 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:29.867 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 3, 2024-04-07T17:51:29.868 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:29.868 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:29.868 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:29.868 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.b", 2024-04-07T17:51:29.868 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:29.868 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:29.868 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:29.868 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4823, 2024-04-07T17:51:29.868 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:29.868 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T17:09:27.535561+0000", 2024-04-07T17:51:29.868 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:29.868 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:29.868 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:29.868 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:29.868 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:29.868 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:29.869 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:29.869 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:29.869 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:29.869 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:29.869 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:29.869 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:29.869 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:29.869 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:29.869 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:29.869 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:29.869 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:29.869 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:29.869 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:29.869 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 3, 2024-04-07T17:51:29.869 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 1, 2024-04-07T17:51:29.870 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:29.870 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 1, 2024-04-07T17:51:29.870 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:29.870 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:29.870 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:29.870 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:29.870 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:29.870 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 1, 2024-04-07T17:51:29.870 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:29.870 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:29.870 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:29.870 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:29.870 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:29.870 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:29.870 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:29.871 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 2, 2024-04-07T17:51:29.871 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 2, 2024-04-07T17:51:29.871 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:29.871 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:29.871 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:29.871 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 1, 2024-04-07T17:51:29.871 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:29.871 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.871 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:29.871 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:29.871 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:29.871 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:29.871 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:29.871 INFO:tasks.workunit.client.0.smithi100.stdout: 5 2024-04-07T17:51:29.871 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:29.872 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 7, 2024-04-07T17:51:29.872 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "623'3" 2024-04-07T17:51:29.872 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.872 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:29.872 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:29.872 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:29.872 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:29.872 INFO:tasks.workunit.client.0.smithi100.stdout: 4 2024-04-07T17:51:29.872 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:29.872 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 3, 2024-04-07T17:51:29.872 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:29.872 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:29.872 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:29.872 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:29.872 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:29.872 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:29.873 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:35.976988+0000", 2024-04-07T17:51:29.873 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:24:13.933754+0000", 2024-04-07T17:51:29.873 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:24:13.933754+0000", 2024-04-07T17:51:29.873 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:24:13.934742+0000", 2024-04-07T17:51:29.873 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:35.976988+0000", 2024-04-07T17:51:29.873 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:29.873 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:29.873 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:29.873 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 500, 2024-04-07T17:51:29.873 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:35.976988+0000", 2024-04-07T17:51:29.873 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:35.976988+0000", 2024-04-07T17:51:29.873 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:35.976988+0000", 2024-04-07T17:51:29.873 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:29.873 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:29.874 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:29.874 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:35.976988+0000", 2024-04-07T17:51:29.874 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:35.976988+0000", 2024-04-07T17:51:29.874 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:29.874 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:29.874 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:29.874 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:29.874 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 499, 2024-04-07T17:51:29.874 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:29.874 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:29.874 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:29.874 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:29.874 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:29.874 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:29.874 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:29.875 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:29.875 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.b", 2024-04-07T17:51:29.875 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:29.875 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:29.875 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:29.875 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4939, 2024-04-07T17:51:29.875 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:29.875 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T02:43:13.536050+0000", 2024-04-07T17:51:29.875 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:29.875 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:29.875 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:29.875 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:29.875 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:29.875 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:29.876 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:29.876 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:29.876 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:29.876 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:29.876 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:29.876 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:29.876 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:29.876 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:29.876 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:29.876 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:29.876 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:29.876 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:29.876 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:29.876 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:29.876 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:29.877 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:29.877 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:29.877 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:29.877 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:29.877 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:29.877 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:29.877 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:29.877 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:29.877 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:29.877 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:29.877 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:29.877 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:29.877 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:29.877 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:29.877 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:29.877 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:29.878 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:29.878 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:29.878 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:29.878 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:29.878 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:29.878 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:29.878 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.878 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:29.878 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:29.878 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:29.878 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:29.878 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:29.878 INFO:tasks.workunit.client.0.smithi100.stdout: 4 2024-04-07T17:51:29.878 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:29.878 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 3, 2024-04-07T17:51:29.879 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:29.879 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.879 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:29.879 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:29.879 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:29.879 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:29.879 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T17:51:29.879 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:29.879 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 1, 2024-04-07T17:51:29.879 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:29.879 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:29.879 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T17:51:29.879 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:29.879 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:29.879 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:29.879 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.335142+0000", 2024-04-07T17:51:29.879 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:38.975734+0000", 2024-04-07T17:51:29.880 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:38.975734+0000", 2024-04-07T17:51:29.880 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:38.976248+0000", 2024-04-07T17:51:29.880 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.335142+0000", 2024-04-07T17:51:29.880 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:29.880 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:29.880 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:29.880 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:29.880 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.335142+0000", 2024-04-07T17:51:29.880 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.335142+0000", 2024-04-07T17:51:29.880 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.335142+0000", 2024-04-07T17:51:29.880 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:29.880 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:29.880 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:29.880 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.335142+0000", 2024-04-07T17:51:29.880 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.335142+0000", 2024-04-07T17:51:29.880 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:29.881 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:29.881 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:29.881 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:29.881 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:29.881 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:29.881 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:29.881 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:29.881 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:29.881 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:29.881 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:29.881 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:29.881 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:29.881 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.a", 2024-04-07T17:51:29.881 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:29.881 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:29.881 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:29.882 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4981, 2024-04-07T17:51:29.882 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:29.882 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T19:45:35.140357+0000", 2024-04-07T17:51:29.882 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:29.882 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:29.882 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:29.882 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:29.882 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:29.882 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:29.882 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:29.882 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:29.882 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:29.882 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:29.882 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:29.883 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:29.883 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:29.883 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:29.883 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:29.883 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:29.883 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:29.883 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:29.883 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:29.883 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:29.883 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:29.883 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:29.883 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:29.883 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:29.883 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:29.883 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:29.883 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:29.883 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:29.884 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:29.884 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:29.884 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:29.884 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:29.884 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:29.884 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:29.884 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:29.884 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:29.884 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:29.884 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:29.884 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:29.884 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:29.884 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:29.884 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:29.885 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:29.885 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.885 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:29.885 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:29.885 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:29.885 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:29.885 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:29.885 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T17:51:29.885 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:29.885 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 1, 2024-04-07T17:51:29.885 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:29.885 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.885 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:29.885 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:29.885 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:29.885 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:29.886 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T17:51:29.886 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:29.886 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 3, 2024-04-07T17:51:29.886 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:29.886 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:29.886 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:29.886 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:29.886 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:29.886 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:29.886 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:35.975502+0000", 2024-04-07T17:51:29.886 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:38.997582+0000", 2024-04-07T17:51:29.886 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:38.997582+0000", 2024-04-07T17:51:29.886 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:38.998502+0000", 2024-04-07T17:51:29.886 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:35.975502+0000", 2024-04-07T17:51:29.887 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:29.887 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:29.887 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:29.887 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:29.887 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:35.975502+0000", 2024-04-07T17:51:29.887 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:35.975502+0000", 2024-04-07T17:51:29.887 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:35.975502+0000", 2024-04-07T17:51:29.887 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:29.887 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:29.887 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:29.887 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:35.975502+0000", 2024-04-07T17:51:29.887 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:35.975502+0000", 2024-04-07T17:51:29.887 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:29.887 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:29.888 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:29.888 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:29.888 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:29.888 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:29.888 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:29.888 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:29.888 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:29.888 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:29.888 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:29.888 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:29.888 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:29.888 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.8", 2024-04-07T17:51:29.888 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:29.888 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:29.888 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:29.888 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4968, 2024-04-07T17:51:29.889 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:29.889 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T23:09:31.581874+0000", 2024-04-07T17:51:29.889 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:29.889 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:29.889 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:29.889 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:29.889 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:29.889 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:29.889 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:29.889 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:29.889 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:29.889 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:29.889 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:29.889 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:29.889 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:29.890 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:29.890 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:29.890 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:29.890 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:29.890 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:29.890 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:29.890 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:29.890 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:29.890 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:29.890 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:29.890 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:29.890 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:29.890 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:29.890 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:29.890 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:29.891 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:29.891 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:29.891 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:29.891 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:29.891 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:29.891 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:29.891 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:29.891 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:29.891 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:29.891 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:29.891 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:29.891 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:29.891 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:29.891 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:29.892 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:29.892 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.892 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:29.892 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:29.892 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:29.892 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:29.892 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:29.892 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T17:51:29.892 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:29.892 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 3, 2024-04-07T17:51:29.892 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:29.892 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.892 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:29.892 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:29.892 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:29.892 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:29.893 INFO:tasks.workunit.client.0.smithi100.stdout: 3 2024-04-07T17:51:29.893 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:29.893 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 1, 2024-04-07T17:51:29.893 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:29.893 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:29.893 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T17:51:29.893 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:29.893 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:29.893 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:29.893 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.331976+0000", 2024-04-07T17:51:29.893 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:38.992340+0000", 2024-04-07T17:51:29.893 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:38.992340+0000", 2024-04-07T17:51:29.893 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:38.992779+0000", 2024-04-07T17:51:29.893 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.331976+0000", 2024-04-07T17:51:29.893 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:29.893 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:29.894 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:29.894 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:29.894 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.331976+0000", 2024-04-07T17:51:29.894 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.331976+0000", 2024-04-07T17:51:29.894 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.331976+0000", 2024-04-07T17:51:29.894 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:29.894 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:29.894 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:29.894 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.331976+0000", 2024-04-07T17:51:29.894 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.331976+0000", 2024-04-07T17:51:29.894 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:29.894 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:29.894 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:29.894 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:29.894 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:29.895 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:29.895 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:29.895 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:29.895 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:29.895 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:29.895 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:29.895 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:29.895 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:29.895 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.9", 2024-04-07T17:51:29.895 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:29.895 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:29.895 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:29.895 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4981, 2024-04-07T17:51:29.895 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:29.895 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T21:25:43.908090+0000", 2024-04-07T17:51:29.896 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:29.896 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:29.896 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:29.896 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:29.896 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:29.896 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:29.896 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:29.896 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:29.896 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:29.896 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:29.896 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:29.896 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:29.896 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:29.896 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:29.896 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:29.896 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:29.897 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:29.897 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:29.897 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:29.897 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:29.897 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:29.897 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:29.897 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:29.897 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:29.897 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:29.897 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:29.897 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:29.897 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:29.897 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:29.897 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:29.897 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:29.898 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:29.898 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:29.898 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:29.898 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:29.898 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:29.898 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:29.898 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:29.898 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:29.898 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:29.898 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:29.898 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:29.898 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:29.898 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.898 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:29.898 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:29.899 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:29.899 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:29.899 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:29.899 INFO:tasks.workunit.client.0.smithi100.stdout: 3 2024-04-07T17:51:29.899 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:29.899 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 1, 2024-04-07T17:51:29.899 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:29.899 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.899 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:29.899 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:29.899 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:29.899 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T17:51:29.899 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T17:51:29.899 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:29.899 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 4, 2024-04-07T17:51:29.900 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:29.900 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:29.900 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:29.900 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:29.900 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:29.900 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:29.900 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:35.977778+0000", 2024-04-07T17:51:29.900 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:38.985206+0000", 2024-04-07T17:51:29.900 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:38.985206+0000", 2024-04-07T17:51:29.900 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:38.985745+0000", 2024-04-07T17:51:29.900 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:35.977778+0000", 2024-04-07T17:51:29.900 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:29.900 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:29.900 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:29.900 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:29.901 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:35.977778+0000", 2024-04-07T17:51:29.901 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:35.977778+0000", 2024-04-07T17:51:29.901 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:35.977778+0000", 2024-04-07T17:51:29.901 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:29.901 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:29.901 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:29.901 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:35.977778+0000", 2024-04-07T17:51:29.901 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:35.977778+0000", 2024-04-07T17:51:29.901 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:29.901 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:29.901 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:29.901 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:29.901 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:29.901 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:29.901 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:29.902 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:29.902 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:29.902 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:29.902 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:29.902 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:29.902 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:29.902 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.9", 2024-04-07T17:51:29.902 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:29.902 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:29.902 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:29.902 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4931, 2024-04-07T17:51:29.902 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:29.902 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T00:51:54.981164+0000", 2024-04-07T17:51:29.902 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:29.902 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:29.903 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:29.903 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:29.903 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:29.903 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:29.903 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:29.903 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:29.903 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:29.903 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:29.903 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:29.903 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:29.903 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:29.903 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:29.903 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:29.903 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:29.903 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:29.903 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:29.904 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:29.904 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:29.904 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:29.904 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:29.904 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:29.904 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:29.904 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:29.904 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:29.904 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:29.904 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:29.904 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:29.904 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:29.904 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:29.904 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:29.904 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:29.904 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:29.905 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:29.905 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:29.905 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:29.905 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:29.905 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:29.905 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:29.905 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:29.905 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:29.905 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:29.905 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.905 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:29.905 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:29.905 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:29.905 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:29.905 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T17:51:29.906 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T17:51:29.906 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:29.906 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 4, 2024-04-07T17:51:29.906 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:29.906 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.906 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:29.906 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:29.906 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:29.906 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:29.906 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T17:51:29.906 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:29.906 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 7, 2024-04-07T17:51:29.906 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:29.906 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:29.906 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T17:51:29.907 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:29.907 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:29.907 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:29.907 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.364254+0000", 2024-04-07T17:51:29.907 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:38.996008+0000", 2024-04-07T17:51:29.907 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:38.996008+0000", 2024-04-07T17:51:29.907 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:39.001597+0000", 2024-04-07T17:51:29.907 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.364254+0000", 2024-04-07T17:51:29.907 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:29.907 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:29.907 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:29.907 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:29.907 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.364254+0000", 2024-04-07T17:51:29.907 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.364254+0000", 2024-04-07T17:51:29.907 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.364254+0000", 2024-04-07T17:51:29.907 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:29.908 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:29.908 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:29.908 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.364254+0000", 2024-04-07T17:51:29.908 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.364254+0000", 2024-04-07T17:51:29.908 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:29.908 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:29.908 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:29.908 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:29.908 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:29.908 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:29.908 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:29.908 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:29.908 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:29.908 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:29.908 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:29.909 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:29.909 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:29.909 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.8", 2024-04-07T17:51:29.909 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:29.909 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:29.909 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:29.909 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4792, 2024-04-07T17:51:29.909 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:29.909 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T19:53:15.918325+0000", 2024-04-07T17:51:29.909 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:29.909 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:29.909 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:29.909 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:29.909 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:29.909 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:29.909 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:29.910 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:29.910 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:29.910 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:29.910 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:29.910 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:29.910 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:29.910 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:29.910 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:29.910 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:29.910 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:29.910 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:29.910 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:29.910 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:29.910 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:29.910 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:29.911 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:29.911 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:29.911 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:29.911 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:29.911 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:29.911 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:29.911 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:29.911 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:29.911 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:29.911 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:29.911 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:29.911 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:29.911 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:29.911 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:29.911 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:29.912 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:29.912 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:29.912 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:29.912 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:29.912 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:29.912 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:29.912 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.912 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:29.912 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:29.912 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:29.912 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:29.912 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:29.912 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T17:51:29.912 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:29.913 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 7, 2024-04-07T17:51:29.913 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:29.913 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.913 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:29.913 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:29.913 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:29.913 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:29.913 INFO:tasks.workunit.client.0.smithi100.stdout: 4 2024-04-07T17:51:29.913 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:29.913 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 0, 2024-04-07T17:51:29.913 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:29.913 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:29.913 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:29.913 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:29.913 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:29.913 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:29.914 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:35.981688+0000", 2024-04-07T17:51:29.914 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:24:13.962953+0000", 2024-04-07T17:51:29.914 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:24:13.962953+0000", 2024-04-07T17:51:29.914 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:24:13.963599+0000", 2024-04-07T17:51:29.914 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:35.981688+0000", 2024-04-07T17:51:29.914 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:29.914 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:29.914 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:29.914 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 500, 2024-04-07T17:51:29.914 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:35.981688+0000", 2024-04-07T17:51:29.914 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:35.981688+0000", 2024-04-07T17:51:29.914 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:35.981688+0000", 2024-04-07T17:51:29.914 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:29.914 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:29.915 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:29.915 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:35.981688+0000", 2024-04-07T17:51:29.915 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:35.981688+0000", 2024-04-07T17:51:29.915 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:29.915 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:29.915 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:29.915 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:29.915 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 499, 2024-04-07T17:51:29.915 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:29.915 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:29.915 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:29.915 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:29.915 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:29.915 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:29.915 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:29.916 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:29.916 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.6", 2024-04-07T17:51:29.916 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:29.916 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:29.916 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:29.916 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4999, 2024-04-07T17:51:29.916 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:29.916 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T22:29:41.083754+0000", 2024-04-07T17:51:29.916 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:29.916 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:29.916 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:29.916 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:29.916 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:29.916 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:29.917 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:29.917 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:29.917 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:29.917 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:29.917 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:29.917 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:29.917 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:29.917 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:29.917 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:29.917 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:29.917 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:29.917 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:29.917 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:29.917 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:29.917 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:29.918 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:29.918 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:29.918 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:29.918 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:29.918 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:29.918 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:29.918 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:29.918 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:29.918 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:29.918 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:29.918 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:29.918 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:29.918 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:29.918 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:29.919 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:29.919 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:29.919 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:29.919 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:29.919 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:29.919 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:29.919 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:29.919 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:29.919 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.919 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:29.919 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:29.919 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:29.919 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:29.919 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:29.919 INFO:tasks.workunit.client.0.smithi100.stdout: 4 2024-04-07T17:51:29.919 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:29.920 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 0, 2024-04-07T17:51:29.920 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:29.920 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.920 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:29.920 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:29.920 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:29.920 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:29.920 INFO:tasks.workunit.client.0.smithi100.stdout: 2 2024-04-07T17:51:29.920 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:29.920 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 6, 2024-04-07T17:51:29.920 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:29.920 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:29.920 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T17:51:29.920 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:29.920 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:29.921 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:29.921 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.369806+0000", 2024-04-07T17:51:29.921 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:08.974754+0000", 2024-04-07T17:51:29.921 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:08.974754+0000", 2024-04-07T17:51:29.921 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:08.975487+0000", 2024-04-07T17:51:29.921 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.369806+0000", 2024-04-07T17:51:29.921 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:29.921 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:29.921 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:29.921 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 554, 2024-04-07T17:51:29.921 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.369806+0000", 2024-04-07T17:51:29.921 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.369806+0000", 2024-04-07T17:51:29.921 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.369806+0000", 2024-04-07T17:51:29.921 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:29.921 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:29.922 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:29.922 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.369806+0000", 2024-04-07T17:51:29.922 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.369806+0000", 2024-04-07T17:51:29.922 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:29.922 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 2, 2024-04-07T17:51:29.922 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:29.922 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:29.922 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 553, 2024-04-07T17:51:29.922 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:29.922 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:29.922 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:29.922 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:29.922 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 2, 2024-04-07T17:51:29.922 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:29.923 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:29.923 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:29.923 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.7", 2024-04-07T17:51:29.923 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:29.923 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:29.923 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:29.923 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4865, 2024-04-07T17:51:29.923 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:29.923 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T01:37:46.529471+0000", 2024-04-07T17:51:29.923 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:29.923 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:29.923 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:29.923 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:29.923 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:29.923 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:29.923 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:29.924 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:29.924 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:29.924 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:29.924 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:29.924 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:29.924 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:29.924 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:29.924 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:29.924 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:29.924 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:29.924 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:29.924 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:29.924 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 3, 2024-04-07T17:51:29.924 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 1, 2024-04-07T17:51:29.924 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:29.925 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 1, 2024-04-07T17:51:29.925 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:29.925 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:29.925 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:29.925 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:29.925 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:29.925 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 1, 2024-04-07T17:51:29.925 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:29.925 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:29.925 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:29.925 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:29.925 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:29.925 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:29.925 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:29.925 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 6, 2024-04-07T17:51:29.925 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 6, 2024-04-07T17:51:29.926 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:29.926 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:29.926 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:29.926 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 3, 2024-04-07T17:51:29.926 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:29.926 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.926 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:29.926 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:29.926 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:29.926 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:29.926 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:29.926 INFO:tasks.workunit.client.0.smithi100.stdout: 2 2024-04-07T17:51:29.926 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:29.926 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 6, 2024-04-07T17:51:29.927 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "61'2" 2024-04-07T17:51:29.927 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.927 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:29.927 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:29.927 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:29.927 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:29.927 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T17:51:29.927 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:29.927 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 4, 2024-04-07T17:51:29.927 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:29.927 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:29.927 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 66, 2024-04-07T17:51:29.927 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:29.927 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:29.927 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:29.927 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:35.977631+0000", 2024-04-07T17:51:29.928 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:08.952433+0000", 2024-04-07T17:51:29.928 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:08.952433+0000", 2024-04-07T17:51:29.928 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:59:56.534054+0000", 2024-04-07T17:51:29.928 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:35.977631+0000", 2024-04-07T17:51:29.928 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:13:28.963534+0000", 2024-04-07T17:51:29.928 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:29.928 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:13:28.963534+0000", 2024-04-07T17:51:29.928 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 554, 2024-04-07T17:51:29.928 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:35.977631+0000", 2024-04-07T17:51:29.928 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:35.977631+0000", 2024-04-07T17:51:29.928 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:35.977631+0000", 2024-04-07T17:51:29.928 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:29.928 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:29.928 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:13:28.963534+0000", 2024-04-07T17:51:29.929 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:35.977631+0000", 2024-04-07T17:51:29.929 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:35.977631+0000", 2024-04-07T17:51:29.929 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:29.929 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 3036, 2024-04-07T17:51:29.929 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:29.929 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:29.929 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 553, 2024-04-07T17:51:29.929 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:29.929 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:29.929 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:29.929 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:29.929 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 3036, 2024-04-07T17:51:29.929 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:29.929 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:29.929 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:29.930 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "4.1", 2024-04-07T17:51:29.930 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:29.930 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:29.930 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:29.930 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 8679, 2024-04-07T17:51:29.930 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:29.930 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T00:59:44.742269+0000", 2024-04-07T17:51:29.930 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.007050569, 2024-04-07T17:51:29.930 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:29.930 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:29.930 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 4194304, 2024-04-07T17:51:29.930 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:29.930 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:29.930 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:29.930 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:29.931 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:29.931 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:29.931 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:29.931 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:29.931 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:29.931 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:29.931 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:29.931 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:29.931 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:29.931 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:29.931 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:29.931 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 3, 2024-04-07T17:51:29.931 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 1, 2024-04-07T17:51:29.931 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:29.931 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 1, 2024-04-07T17:51:29.931 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:29.932 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:29.932 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:29.932 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:29.932 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:29.932 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:29.932 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:29.932 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:29.932 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:29.932 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:29.932 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:29.932 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:29.932 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:29.932 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 1087, 2024-04-07T17:51:29.932 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 402424, 2024-04-07T17:51:29.932 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:29.932 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:29.933 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:29.933 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 3017, 2024-04-07T17:51:29.933 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 407157 2024-04-07T17:51:29.933 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.933 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:29.933 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:29.933 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:29.933 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:29.933 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:29.933 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T17:51:29.933 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:29.933 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 4, 2024-04-07T17:51:29.933 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "1083'3036" 2024-04-07T17:51:29.933 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.933 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:29.934 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:29.934 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:29.934 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:29.934 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T17:51:29.934 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:29.934 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 3, 2024-04-07T17:51:29.934 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:29.934 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:29.934 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:29.934 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:29.934 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:29.934 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:29.934 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:35.981722+0000", 2024-04-07T17:51:29.934 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:38.974388+0000", 2024-04-07T17:51:29.934 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:38.974388+0000", 2024-04-07T17:51:29.935 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:38.974763+0000", 2024-04-07T17:51:29.935 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:35.981722+0000", 2024-04-07T17:51:29.935 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:29.935 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:29.935 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:29.935 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:29.935 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:35.981722+0000", 2024-04-07T17:51:29.935 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:35.981722+0000", 2024-04-07T17:51:29.935 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:35.981722+0000", 2024-04-07T17:51:29.935 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:29.935 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:29.935 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:29.935 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:35.981722+0000", 2024-04-07T17:51:29.935 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:35.981722+0000", 2024-04-07T17:51:29.935 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:29.935 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:29.936 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:29.936 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:29.936 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:29.936 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:29.936 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:29.936 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:29.936 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:29.936 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:29.936 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:29.936 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:29.936 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:29.936 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.7", 2024-04-07T17:51:29.936 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:29.936 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:29.936 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:29.936 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4940, 2024-04-07T17:51:29.937 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:29.937 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T03:43:59.412072+0000", 2024-04-07T17:51:29.937 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:29.937 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:29.937 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:29.937 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:29.937 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:29.937 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:29.937 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:29.937 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:29.937 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:29.937 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:29.937 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:29.937 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:29.937 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:29.938 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:29.938 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:29.938 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:29.938 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:29.938 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:29.938 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:29.938 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:29.938 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:29.938 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:29.938 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:29.938 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:29.938 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:29.938 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:29.938 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:29.938 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:29.938 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:29.939 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:29.939 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:29.939 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:29.939 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:29.939 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:29.939 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:29.939 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:29.939 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:29.939 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:29.939 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:29.939 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:29.939 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:29.939 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:29.939 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:29.939 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.940 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:29.940 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:29.940 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:29.940 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:29.940 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:29.940 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T17:51:29.940 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:29.940 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 3, 2024-04-07T17:51:29.940 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:29.940 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.940 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:29.940 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:29.940 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:29.940 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:29.940 INFO:tasks.workunit.client.0.smithi100.stdout: 4 2024-04-07T17:51:29.940 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:29.941 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 1, 2024-04-07T17:51:29.941 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:29.941 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:29.941 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T17:51:29.941 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:29.941 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:29.941 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:29.941 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.335258+0000", 2024-04-07T17:51:29.941 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:08.962773+0000", 2024-04-07T17:51:29.941 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:08.962773+0000", 2024-04-07T17:51:29.941 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:08.963320+0000", 2024-04-07T17:51:29.941 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.335258+0000", 2024-04-07T17:51:29.941 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:29.941 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:29.942 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:29.942 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 554, 2024-04-07T17:51:29.942 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.335258+0000", 2024-04-07T17:51:29.942 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.335258+0000", 2024-04-07T17:51:29.942 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.335258+0000", 2024-04-07T17:51:29.942 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:29.942 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:29.942 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:29.942 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.335258+0000", 2024-04-07T17:51:29.942 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.335258+0000", 2024-04-07T17:51:29.942 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:29.942 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 2, 2024-04-07T17:51:29.942 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:29.942 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:29.942 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 553, 2024-04-07T17:51:29.943 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:29.943 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:29.943 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:29.943 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:29.943 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 2, 2024-04-07T17:51:29.943 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:29.943 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:29.943 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:29.943 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.6", 2024-04-07T17:51:29.943 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:29.943 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:29.943 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:29.943 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4986, 2024-04-07T17:51:29.943 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:29.943 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T18:43:13.545482+0000", 2024-04-07T17:51:29.944 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:29.944 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:29.944 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:29.944 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:29.944 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:29.944 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:29.944 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:29.944 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:29.944 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:29.944 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:29.944 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:29.944 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:29.944 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:29.944 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:29.944 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:29.944 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:29.945 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:29.945 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:29.945 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:29.945 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 3, 2024-04-07T17:51:29.945 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 1, 2024-04-07T17:51:29.945 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:29.945 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 1, 2024-04-07T17:51:29.945 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:29.945 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:29.945 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:29.945 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:29.945 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:29.945 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 1, 2024-04-07T17:51:29.945 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:29.945 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:29.945 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:29.946 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:29.946 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:29.946 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:29.946 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:29.946 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 2, 2024-04-07T17:51:29.946 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 2, 2024-04-07T17:51:29.946 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:29.946 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:29.946 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:29.946 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 2, 2024-04-07T17:51:29.946 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:29.946 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.946 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:29.946 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:29.946 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:29.947 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:29.947 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:29.947 INFO:tasks.workunit.client.0.smithi100.stdout: 4 2024-04-07T17:51:29.947 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:29.947 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 1, 2024-04-07T17:51:29.947 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "627'2" 2024-04-07T17:51:29.947 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.947 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:29.947 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:29.947 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:29.947 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:29.947 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T17:51:29.947 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:29.947 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 3, 2024-04-07T17:51:29.947 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:29.948 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:29.948 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 66, 2024-04-07T17:51:29.948 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:29.948 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:29.948 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:29.948 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:35.975295+0000", 2024-04-07T17:51:29.948 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:38.997956+0000", 2024-04-07T17:51:29.948 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:38.997956+0000", 2024-04-07T17:51:29.948 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:59:56.537779+0000", 2024-04-07T17:51:29.948 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:35.975295+0000", 2024-04-07T17:51:29.948 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:13:28.963534+0000", 2024-04-07T17:51:29.948 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:29.948 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:13:28.963534+0000", 2024-04-07T17:51:29.948 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:29.948 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:35.975295+0000", 2024-04-07T17:51:29.949 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:35.975295+0000", 2024-04-07T17:51:29.949 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:35.975295+0000", 2024-04-07T17:51:29.949 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:29.949 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:29.949 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:13:28.963534+0000", 2024-04-07T17:51:29.949 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:35.975295+0000", 2024-04-07T17:51:29.949 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:35.975295+0000", 2024-04-07T17:51:29.949 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:29.949 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 3089, 2024-04-07T17:51:29.949 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:29.949 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:29.949 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:29.949 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:29.949 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:29.949 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:29.949 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:29.950 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 3089, 2024-04-07T17:51:29.950 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:29.950 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:29.950 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:29.950 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "4.0", 2024-04-07T17:51:29.950 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:29.950 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:29.950 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:29.950 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 8748, 2024-04-07T17:51:29.950 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:29.950 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T16:44:21.465234+0000", 2024-04-07T17:51:29.950 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.013008564, 2024-04-07T17:51:29.950 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:29.950 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:29.950 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:29.951 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:29.951 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:29.951 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:29.951 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:29.951 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:29.951 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:29.951 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:29.951 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:29.951 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:29.951 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:29.951 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:29.951 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:29.951 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:29.951 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:29.951 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:29.952 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:29.952 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:29.952 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:29.952 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:29.952 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:29.952 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:29.952 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:29.952 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:29.952 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:29.952 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:29.952 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:29.952 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:29.952 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:29.952 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:29.952 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:29.952 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:29.953 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:29.953 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 1133, 2024-04-07T17:51:29.953 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 401354, 2024-04-07T17:51:29.953 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:29.953 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:29.953 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:29.953 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 3064, 2024-04-07T17:51:29.953 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 366365 2024-04-07T17:51:29.953 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.953 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:29.953 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:29.953 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:29.953 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:29.953 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:29.953 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T17:51:29.954 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:29.954 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 3, 2024-04-07T17:51:29.954 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "1083'3089" 2024-04-07T17:51:29.954 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.954 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:29.954 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:29.954 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:29.954 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T17:51:29.954 INFO:tasks.workunit.client.0.smithi100.stdout: 5 2024-04-07T17:51:29.954 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:29.954 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 1, 2024-04-07T17:51:29.954 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:29.954 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:29.954 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:29.954 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:29.955 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:29.955 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:29.955 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.333938+0000", 2024-04-07T17:51:29.955 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:24:41.552588+0000", 2024-04-07T17:51:29.955 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:24:41.552588+0000", 2024-04-07T17:51:29.955 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:24:41.553129+0000", 2024-04-07T17:51:29.955 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.333938+0000", 2024-04-07T17:51:29.955 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:29.955 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:29.955 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:29.955 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 527, 2024-04-07T17:51:29.955 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.333938+0000", 2024-04-07T17:51:29.955 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.333938+0000", 2024-04-07T17:51:29.955 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.333938+0000", 2024-04-07T17:51:29.956 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:29.956 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:29.956 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:29.956 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.333938+0000", 2024-04-07T17:51:29.956 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.333938+0000", 2024-04-07T17:51:29.956 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:29.956 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:29.956 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:29.956 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:29.956 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 526, 2024-04-07T17:51:29.956 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:29.956 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:29.956 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:29.956 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:29.956 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:29.957 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:29.957 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:29.957 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:29.957 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.4", 2024-04-07T17:51:29.957 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:29.957 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:29.957 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:29.957 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4983, 2024-04-07T17:51:29.957 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:29.957 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T18:17:15.023877+0000", 2024-04-07T17:51:29.957 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:29.957 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:29.957 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:29.957 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:29.957 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:29.958 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:29.958 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:29.958 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:29.958 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:29.958 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:29.958 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:29.958 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:29.958 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:29.958 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:29.958 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:29.958 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:29.958 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:29.958 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:29.958 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:29.958 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:29.959 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:29.959 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:29.959 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:29.959 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:29.959 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:29.959 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:29.959 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:29.959 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:29.959 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:29.959 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:29.959 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:29.959 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:29.959 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:29.959 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:29.959 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:29.960 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:29.960 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:29.960 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:29.960 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:29.960 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:29.960 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:29.960 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:29.960 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:29.960 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.960 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:29.960 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:29.960 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:29.960 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:29.960 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T17:51:29.960 INFO:tasks.workunit.client.0.smithi100.stdout: 5 2024-04-07T17:51:29.960 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:29.961 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 1, 2024-04-07T17:51:29.961 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:29.961 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.961 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:29.961 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:29.961 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:29.961 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:29.961 INFO:tasks.workunit.client.0.smithi100.stdout: 4 2024-04-07T17:51:29.961 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:29.961 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 7, 2024-04-07T17:51:29.961 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:29.961 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:29.961 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T17:51:29.961 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:29.961 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:29.961 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:29.962 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.373099+0000", 2024-04-07T17:51:29.962 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:39.001835+0000", 2024-04-07T17:51:29.962 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:39.001835+0000", 2024-04-07T17:51:29.962 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:39.003607+0000", 2024-04-07T17:51:29.962 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.373099+0000", 2024-04-07T17:51:29.962 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:29.962 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:29.962 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:29.962 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:29.962 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.373099+0000", 2024-04-07T17:51:29.962 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.373099+0000", 2024-04-07T17:51:29.962 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.373099+0000", 2024-04-07T17:51:29.962 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:29.962 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:29.962 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:29.963 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.373099+0000", 2024-04-07T17:51:29.963 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.373099+0000", 2024-04-07T17:51:29.963 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:29.963 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:29.963 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:29.963 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:29.963 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:29.963 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:29.963 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:29.963 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:29.963 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:29.963 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:29.963 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:29.963 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:29.963 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:29.963 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.5", 2024-04-07T17:51:29.964 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:29.964 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:29.964 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:29.964 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4796, 2024-04-07T17:51:29.964 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:29.964 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T21:54:36.255484+0000", 2024-04-07T17:51:29.964 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:29.964 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:29.964 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:29.964 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:29.964 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:29.964 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:29.964 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:29.964 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:29.964 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:29.964 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:29.965 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:29.965 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:29.965 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:29.965 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:29.965 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:29.965 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:29.965 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:29.965 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:29.965 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:29.965 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:29.965 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:29.965 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:29.965 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:29.965 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:29.965 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:29.966 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:29.966 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:29.966 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:29.966 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:29.966 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:29.966 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:29.966 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:29.966 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:29.966 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:29.966 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:29.966 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:29.966 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:29.966 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:29.966 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:29.966 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:29.967 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:29.967 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:29.967 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:29.967 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.967 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:29.967 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:29.967 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:29.967 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:29.967 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:29.967 INFO:tasks.workunit.client.0.smithi100.stdout: 4 2024-04-07T17:51:29.967 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:29.967 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 7, 2024-04-07T17:51:29.967 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:29.967 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.967 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:29.967 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:29.968 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:29.968 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:29.968 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T17:51:29.968 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:29.968 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 0, 2024-04-07T17:51:29.968 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:29.968 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:29.968 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 66, 2024-04-07T17:51:29.968 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:29.968 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:29.968 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:29.968 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:51:22.430544+0000", 2024-04-07T17:51:29.968 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:39.003851+0000", 2024-04-07T17:51:29.968 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:39.003851+0000", 2024-04-07T17:51:29.968 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:59:56.571405+0000", 2024-04-07T17:51:29.969 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:51:22.430544+0000", 2024-04-07T17:51:29.969 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:13:28.963534+0000", 2024-04-07T17:51:29.969 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:29.969 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:13:28.963534+0000", 2024-04-07T17:51:29.969 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:29.969 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:51:22.430544+0000", 2024-04-07T17:51:29.969 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:51:22.430544+0000", 2024-04-07T17:51:29.969 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:51:22.430544+0000", 2024-04-07T17:51:29.969 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:29.969 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:29.969 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:13:28.963534+0000", 2024-04-07T17:51:29.969 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:51:22.430544+0000", 2024-04-07T17:51:29.969 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:51:22.430544+0000", 2024-04-07T17:51:29.969 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:29.969 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 3135, 2024-04-07T17:51:29.970 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:29.970 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:29.970 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:29.970 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:29.970 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:29.970 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:29.970 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:29.970 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 3135, 2024-04-07T17:51:29.970 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:29.970 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:29.970 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:29.970 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "4.3", 2024-04-07T17:51:29.970 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:29.970 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:29.970 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:29.971 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 8778, 2024-04-07T17:51:29.971 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:29.971 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T23:06:05.133511+0000", 2024-04-07T17:51:29.971 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.019028691, 2024-04-07T17:51:29.971 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:29.971 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:29.971 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:29.971 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:29.971 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:29.971 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:29.971 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:29.971 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:29.971 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:29.971 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:29.971 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:29.972 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:29.972 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:29.972 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:29.972 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:29.972 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:29.972 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:29.972 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:29.972 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 3, 2024-04-07T17:51:29.972 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 1, 2024-04-07T17:51:29.972 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:29.972 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 1, 2024-04-07T17:51:29.972 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:29.972 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:29.972 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:29.972 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:29.972 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:29.972 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 1, 2024-04-07T17:51:29.973 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:29.973 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:29.973 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:29.973 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:29.973 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:29.973 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:29.973 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:29.973 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 873, 2024-04-07T17:51:29.973 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 391393, 2024-04-07T17:51:29.973 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:29.973 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:29.973 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:29.973 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 3127, 2024-04-07T17:51:29.973 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 378194 2024-04-07T17:51:29.973 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.974 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:29.974 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:29.974 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:29.974 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:29.974 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:29.974 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T17:51:29.974 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:29.974 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 0, 2024-04-07T17:51:29.974 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "1084'3135" 2024-04-07T17:51:29.974 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.974 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:29.974 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:29.974 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:29.974 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:29.974 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T17:51:29.975 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:29.975 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 7, 2024-04-07T17:51:29.975 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:29.975 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:29.975 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T17:51:29.975 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:29.975 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:29.975 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:29.975 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.364517+0000", 2024-04-07T17:51:29.975 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:39.000134+0000", 2024-04-07T17:51:29.975 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:39.000134+0000", 2024-04-07T17:51:29.975 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:39.003944+0000", 2024-04-07T17:51:29.975 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.364517+0000", 2024-04-07T17:51:29.975 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:29.976 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:29.976 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:29.976 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:29.976 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.364517+0000", 2024-04-07T17:51:29.976 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.364517+0000", 2024-04-07T17:51:29.976 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.364517+0000", 2024-04-07T17:51:29.976 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:29.976 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:29.976 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:29.976 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.364517+0000", 2024-04-07T17:51:29.976 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.364517+0000", 2024-04-07T17:51:29.976 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:29.976 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:29.976 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:29.976 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:29.977 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:29.977 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:29.977 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:29.977 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:29.977 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:29.977 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:29.977 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:29.977 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:29.977 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:29.977 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.0", 2024-04-07T17:51:29.977 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:29.977 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:29.977 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:29.977 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4796, 2024-04-07T17:51:29.977 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:29.977 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T22:28:12.376172+0000", 2024-04-07T17:51:29.978 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:29.978 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:29.978 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:29.978 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:29.978 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:29.978 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:29.978 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:29.978 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:29.978 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:29.978 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:29.978 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:29.978 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:29.978 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:29.978 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:29.979 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:29.979 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:29.979 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:29.979 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:29.979 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:29.979 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:29.979 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:29.979 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:29.979 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:29.979 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:29.979 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:29.979 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:29.979 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:29.979 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:29.979 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:29.979 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:29.980 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:29.980 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:29.980 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:29.980 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:29.980 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:29.980 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:29.980 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:29.980 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:29.980 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:29.980 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:29.980 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:29.980 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:29.980 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:29.980 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.980 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:29.980 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:29.981 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:29.981 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:29.981 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:29.981 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T17:51:29.981 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:29.981 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 7, 2024-04-07T17:51:29.981 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:29.981 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.981 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:29.981 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:29.981 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T17:51:29.981 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:29.981 INFO:tasks.workunit.client.0.smithi100.stdout: 3 2024-04-07T17:51:29.981 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:29.981 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 2, 2024-04-07T17:51:29.981 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:29.981 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:29.982 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:29.982 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:29.982 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:29.982 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:29.982 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.354824+0000", 2024-04-07T17:51:29.982 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:24:13.930517+0000", 2024-04-07T17:51:29.982 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:24:13.930517+0000", 2024-04-07T17:51:29.982 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:24:13.931259+0000", 2024-04-07T17:51:29.982 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.354824+0000", 2024-04-07T17:51:29.982 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:29.982 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:29.982 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:29.982 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 500, 2024-04-07T17:51:29.982 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.354824+0000", 2024-04-07T17:51:29.983 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.354824+0000", 2024-04-07T17:51:29.983 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.354824+0000", 2024-04-07T17:51:29.983 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:29.983 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:29.983 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:29.983 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.354824+0000", 2024-04-07T17:51:29.983 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.354824+0000", 2024-04-07T17:51:29.983 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:29.983 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:29.983 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:29.983 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:29.983 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 499, 2024-04-07T17:51:29.983 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:29.984 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:29.984 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:29.984 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:29.984 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:29.984 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:29.984 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:29.984 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:29.984 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.1", 2024-04-07T17:51:29.984 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:29.984 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:29.984 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:29.984 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4929, 2024-04-07T17:51:29.984 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:29.984 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T01:43:11.885075+0000", 2024-04-07T17:51:29.984 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:29.985 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:29.985 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:29.985 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:29.985 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:29.985 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:29.985 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:29.985 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:29.985 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:29.985 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:29.985 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:29.985 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:29.985 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:29.985 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:29.985 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:29.985 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:29.986 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:29.986 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:29.986 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:29.986 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:29.986 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:29.986 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:29.986 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:29.986 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:29.986 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:29.986 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:29.986 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:29.986 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:29.986 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:29.986 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:29.986 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:29.987 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:29.987 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:29.987 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:29.987 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:29.987 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:29.987 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:29.987 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:29.987 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:29.987 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:29.987 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:29.987 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:29.987 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:29.987 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.987 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:29.987 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:29.988 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:29.988 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T17:51:29.988 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:29.988 INFO:tasks.workunit.client.0.smithi100.stdout: 3 2024-04-07T17:51:29.988 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:29.988 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 2, 2024-04-07T17:51:29.988 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:29.988 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.988 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:29.988 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:29.988 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:29.988 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:29.988 INFO:tasks.workunit.client.0.smithi100.stdout: 2 2024-04-07T17:51:29.988 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:29.988 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 0, 2024-04-07T17:51:29.988 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:29.989 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:29.989 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 66, 2024-04-07T17:51:29.989 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:29.989 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:29.989 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:29.989 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:35.982117+0000", 2024-04-07T17:51:29.989 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:23:15.882881+0000", 2024-04-07T17:51:29.989 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:23:15.882881+0000", 2024-04-07T17:51:29.989 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:59:56.569536+0000", 2024-04-07T17:51:29.989 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:35.982117+0000", 2024-04-07T17:51:29.989 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:13:28.963534+0000", 2024-04-07T17:51:29.989 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:29.989 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:13:28.963534+0000", 2024-04-07T17:51:29.989 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 445, 2024-04-07T17:51:29.989 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:35.982117+0000", 2024-04-07T17:51:29.990 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:35.982117+0000", 2024-04-07T17:51:29.990 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:35.982117+0000", 2024-04-07T17:51:29.990 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:29.990 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:29.990 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:13:28.963534+0000", 2024-04-07T17:51:29.990 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:35.982117+0000", 2024-04-07T17:51:29.990 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:35.982117+0000", 2024-04-07T17:51:29.990 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:29.990 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 3107, 2024-04-07T17:51:29.990 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:29.990 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:29.990 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 444, 2024-04-07T17:51:29.990 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:29.990 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:29.990 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:29.991 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:29.991 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 3107, 2024-04-07T17:51:29.991 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:29.991 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:29.991 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:29.991 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "4.6", 2024-04-07T17:51:29.991 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:29.991 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:29.991 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:29.991 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 8717, 2024-04-07T17:51:29.991 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:29.991 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T18:35:28.250440+0000", 2024-04-07T17:51:29.991 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.017233416, 2024-04-07T17:51:29.991 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:29.991 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:29.991 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:29.992 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:29.992 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:29.992 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:29.992 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:29.992 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:29.992 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:29.992 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:29.992 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:29.992 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:29.992 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:29.992 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:29.992 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:29.992 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:29.992 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:29.992 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:29.992 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:29.993 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:29.993 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:29.993 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:29.993 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:29.993 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:29.993 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:29.993 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:29.993 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:29.993 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:29.993 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:29.993 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:29.993 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:29.993 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:29.993 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:29.993 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:29.993 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:29.994 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 858, 2024-04-07T17:51:29.994 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 396543, 2024-04-07T17:51:29.994 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:29.994 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:29.994 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:29.994 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 3070, 2024-04-07T17:51:29.994 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 377559 2024-04-07T17:51:29.994 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.994 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:29.994 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:29.994 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:29.994 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:29.994 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:29.994 INFO:tasks.workunit.client.0.smithi100.stdout: 2 2024-04-07T17:51:29.995 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:29.995 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 0, 2024-04-07T17:51:29.995 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "1084'3107" 2024-04-07T17:51:29.995 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:29.995 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:29.995 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:29.995 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:29.995 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:29.995 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T17:51:29.995 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:29.995 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 7, 2024-04-07T17:51:29.995 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:29.995 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:29.995 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 19, 2024-04-07T17:51:29.995 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:29.996 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:29.996 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:29.996 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:51:26.217238+0000", 2024-04-07T17:51:29.996 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:39.000587+0000", 2024-04-07T17:51:29.996 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:39.000587+0000", 2024-04-07T17:51:29.996 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:39.004716+0000", 2024-04-07T17:51:29.996 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:51:26.217238+0000", 2024-04-07T17:51:29.996 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:07:26.866003+0000", 2024-04-07T17:51:29.996 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:29.996 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:07:26.866003+0000", 2024-04-07T17:51:29.996 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:29.996 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:51:26.217238+0000", 2024-04-07T17:51:29.996 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:51:26.217238+0000", 2024-04-07T17:51:29.996 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:51:26.217238+0000", 2024-04-07T17:51:29.996 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:29.997 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:29.997 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:07:26.866003+0000", 2024-04-07T17:51:29.997 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:51:26.217238+0000", 2024-04-07T17:51:29.997 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:51:26.217238+0000", 2024-04-07T17:51:29.997 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:29.997 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 257, 2024-04-07T17:51:29.997 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:29.997 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:29.997 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:29.997 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:29.997 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:29.997 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:29.997 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:29.997 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 257, 2024-04-07T17:51:29.997 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:29.997 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:29.998 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:29.998 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "1.0", 2024-04-07T17:51:29.998 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:29.998 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:29.998 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:29.998 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 5516, 2024-04-07T17:51:29.998 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:29.998 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T21:59:54.730545+0000", 2024-04-07T17:51:29.998 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:29.998 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:29.998 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:29.998 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 590368, 2024-04-07T17:51:29.998 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:29.998 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 1771104, 2024-04-07T17:51:29.998 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:29.999 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:29.999 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:29.999 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:29.999 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:29.999 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:29.999 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:29.999 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:29.999 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:29.999 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:29.999 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:29.999 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:29.999 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:29.999 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 6, 2024-04-07T17:51:29.999 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 2, 2024-04-07T17:51:29.999 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:29.999 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 2, 2024-04-07T17:51:30.000 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:30.000 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:30.000 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:30.000 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:30.000 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:30.000 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:30.000 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:30.000 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 6, 2024-04-07T17:51:30.000 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:30.000 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:30.000 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:30.000 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:30.000 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:30.000 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 1358, 2024-04-07T17:51:30.000 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 2144, 2024-04-07T17:51:30.001 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:30.001 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:30.001 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:30.001 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 431, 2024-04-07T17:51:30.001 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 2416 2024-04-07T17:51:30.001 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:30.001 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:30.001 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:30.001 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:30.001 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:30.001 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:30.001 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T17:51:30.001 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:30.001 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 7, 2024-04-07T17:51:30.001 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "2713'257" 2024-04-07T17:51:30.001 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:30.002 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:30.002 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:30.002 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:30.002 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T17:51:30.002 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T17:51:30.002 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:30.002 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 5, 2024-04-07T17:51:30.002 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:30.002 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:30.002 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T17:51:30.002 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:30.002 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:30.002 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:30.002 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.370078+0000", 2024-04-07T17:51:30.002 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:38.985921+0000", 2024-04-07T17:51:30.003 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:38.985921+0000", 2024-04-07T17:51:30.003 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:38.986647+0000", 2024-04-07T17:51:30.003 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.370078+0000", 2024-04-07T17:51:30.003 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:30.003 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:30.003 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:30.003 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:30.003 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.370078+0000", 2024-04-07T17:51:30.003 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.370078+0000", 2024-04-07T17:51:30.003 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.370078+0000", 2024-04-07T17:51:30.003 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:30.003 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:30.003 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:30.003 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.370078+0000", 2024-04-07T17:51:30.003 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.370078+0000", 2024-04-07T17:51:30.004 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:30.004 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:30.004 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:30.004 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:30.004 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:30.004 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:30.004 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:30.004 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:30.004 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:30.004 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:30.004 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:30.004 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:30.004 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:30.004 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.3", 2024-04-07T17:51:30.004 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:30.004 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:30.005 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:30.005 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4855, 2024-04-07T17:51:30.005 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:30.005 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T02:44:42.906581+0000", 2024-04-07T17:51:30.005 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:30.005 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:30.005 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:30.005 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:30.005 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:30.005 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:30.005 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:30.005 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:30.005 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:30.005 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:30.005 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:30.006 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:30.006 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:30.006 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:30.006 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:30.006 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:30.006 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:30.006 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:30.006 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:30.006 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:30.006 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:30.006 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:30.006 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:30.006 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:30.006 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:30.006 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:30.006 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:30.007 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:30.007 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:30.007 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:30.007 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:30.007 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:30.007 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:30.007 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:30.007 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:30.007 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:30.007 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:30.007 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:30.007 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:30.007 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:30.007 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:30.007 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:30.008 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:30.008 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:30.008 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:30.008 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:30.008 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:30.008 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:30.008 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T17:51:30.008 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T17:51:30.008 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:30.008 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 5, 2024-04-07T17:51:30.008 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:30.008 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:30.008 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:30.008 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:30.008 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:30.008 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:30.009 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T17:51:30.009 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:30.009 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 3, 2024-04-07T17:51:30.009 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:30.009 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:30.009 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:30.009 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:30.009 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:30.009 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:30.009 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:35.982147+0000", 2024-04-07T17:51:30.009 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:08.977825+0000", 2024-04-07T17:51:30.009 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:08.977825+0000", 2024-04-07T17:51:30.009 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:08.978231+0000", 2024-04-07T17:51:30.009 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:35.982147+0000", 2024-04-07T17:51:30.009 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:30.009 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:30.010 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:30.010 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 554, 2024-04-07T17:51:30.010 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:35.982147+0000", 2024-04-07T17:51:30.010 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:35.982147+0000", 2024-04-07T17:51:30.010 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:35.982147+0000", 2024-04-07T17:51:30.010 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:30.010 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:30.010 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:30.010 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:35.982147+0000", 2024-04-07T17:51:30.010 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:35.982147+0000", 2024-04-07T17:51:30.010 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:30.010 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:30.010 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:30.010 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:30.011 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 553, 2024-04-07T17:51:30.011 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:30.011 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:30.011 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:30.011 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:30.011 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:30.011 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:30.011 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:30.011 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:30.011 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.2", 2024-04-07T17:51:30.011 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:30.011 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:30.011 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:30.011 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4937, 2024-04-07T17:51:30.011 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:30.011 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T01:34:08.840626+0000", 2024-04-07T17:51:30.012 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:30.012 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:30.012 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:30.012 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:30.012 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:30.012 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:30.012 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:30.012 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:30.012 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:30.012 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:30.012 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:30.012 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:30.012 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:30.012 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:30.012 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:30.013 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:30.013 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:30.013 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:30.013 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:30.013 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:30.013 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:30.013 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:30.013 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:30.013 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:30.013 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:30.013 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:30.013 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:30.013 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:30.013 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:30.013 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:30.013 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:30.014 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:30.014 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:30.014 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:30.014 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:30.014 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:30.014 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:30.014 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:30.014 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:30.014 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:30.014 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:30.014 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:30.014 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:30.014 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:30.014 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:30.015 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:30.015 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:30.015 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:30.015 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:30.015 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T17:51:30.015 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:30.015 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 3, 2024-04-07T17:51:30.015 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:30.015 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:30.015 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:30.015 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:30.015 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:30.015 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:30.015 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T17:51:30.015 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:30.016 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 6, 2024-04-07T17:51:30.016 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:30.016 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:30.016 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 66, 2024-04-07T17:51:30.016 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:30.016 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:30.016 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:30.016 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.360030+0000", 2024-04-07T17:51:30.016 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:39.000185+0000", 2024-04-07T17:51:30.016 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:39.000185+0000", 2024-04-07T17:51:30.016 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:59:56.544598+0000", 2024-04-07T17:51:30.016 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.360030+0000", 2024-04-07T17:51:30.016 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:13:28.963534+0000", 2024-04-07T17:51:30.016 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:30.016 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:13:28.963534+0000", 2024-04-07T17:51:30.017 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:30.017 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.360030+0000", 2024-04-07T17:51:30.017 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.360030+0000", 2024-04-07T17:51:30.017 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.360030+0000", 2024-04-07T17:51:30.017 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:30.017 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:30.017 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:13:28.963534+0000", 2024-04-07T17:51:30.017 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.360030+0000", 2024-04-07T17:51:30.017 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.360030+0000", 2024-04-07T17:51:30.017 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:30.017 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 3124, 2024-04-07T17:51:30.017 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:30.017 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:30.017 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:30.017 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:30.018 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:30.018 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:30.018 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:30.018 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 3124, 2024-04-07T17:51:30.018 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:30.018 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:30.018 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:30.018 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "4.5", 2024-04-07T17:51:30.018 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:30.018 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:30.018 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:30.018 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 8600, 2024-04-07T17:51:30.018 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:30.018 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T01:53:54.315291+0000", 2024-04-07T17:51:30.018 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.008144472, 2024-04-07T17:51:30.019 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:30.019 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:30.019 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:30.019 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:30.019 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:30.019 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:30.019 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:30.019 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:30.019 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:30.019 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:30.019 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:30.019 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:30.019 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:30.019 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:30.019 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:30.020 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:30.020 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:30.020 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:30.020 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:30.020 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:30.020 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:30.020 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:30.020 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:30.020 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:30.020 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:30.020 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:30.020 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:30.020 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:30.020 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:30.020 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:30.020 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:30.021 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:30.021 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:30.021 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:30.021 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:30.021 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 979, 2024-04-07T17:51:30.021 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 424424, 2024-04-07T17:51:30.021 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:30.021 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:30.021 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:30.021 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 3125, 2024-04-07T17:51:30.021 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 420851 2024-04-07T17:51:30.021 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:30.021 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:30.021 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:30.021 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:30.022 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:30.022 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:30.022 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T17:51:30.022 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:30.022 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 6, 2024-04-07T17:51:30.022 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "1083'3124" 2024-04-07T17:51:30.022 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:30.022 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:30.022 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:30.022 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T17:51:30.022 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:30.022 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T17:51:30.022 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:30.022 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 2, 2024-04-07T17:51:30.023 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:30.023 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:30.023 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T17:51:30.023 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:30.023 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:30.023 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:30.023 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.355116+0000", 2024-04-07T17:51:30.023 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:23:43.929900+0000", 2024-04-07T17:51:30.023 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:23:43.929900+0000", 2024-04-07T17:51:30.023 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:23:43.930960+0000", 2024-04-07T17:51:30.023 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.355116+0000", 2024-04-07T17:51:30.023 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:30.023 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:30.023 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:30.023 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 471, 2024-04-07T17:51:30.023 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.355116+0000", 2024-04-07T17:51:30.024 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.355116+0000", 2024-04-07T17:51:30.024 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.355116+0000", 2024-04-07T17:51:30.024 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:30.024 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:30.024 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:30.024 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.355116+0000", 2024-04-07T17:51:30.024 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.355116+0000", 2024-04-07T17:51:30.024 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:30.024 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:30.024 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:30.024 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:30.024 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 470, 2024-04-07T17:51:30.024 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:30.024 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:30.024 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:30.025 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:30.025 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:30.025 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:30.025 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:30.025 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:30.025 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.1", 2024-04-07T17:51:30.025 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:30.025 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:30.025 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:30.025 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4977, 2024-04-07T17:51:30.025 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:30.025 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T20:05:04.268255+0000", 2024-04-07T17:51:30.025 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:30.025 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:30.025 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:30.026 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:30.026 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:30.026 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:30.026 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:30.026 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:30.026 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:30.026 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:30.026 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:30.026 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:30.026 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:30.026 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:30.026 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:30.026 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:30.026 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:30.026 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:30.027 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:30.027 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:30.027 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:30.027 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:30.027 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:30.027 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:30.027 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:30.027 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:30.027 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:30.027 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:30.027 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:30.027 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:30.027 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:30.027 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:30.027 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:30.028 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:30.028 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:30.028 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:30.028 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:30.028 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:30.028 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:30.028 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:30.028 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:30.028 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:30.028 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:30.028 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:30.028 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:30.028 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:30.028 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:30.028 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T17:51:30.029 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:30.029 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T17:51:30.029 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:30.029 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 2, 2024-04-07T17:51:30.029 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:30.029 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:30.029 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:30.029 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:30.029 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:30.029 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T17:51:30.029 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T17:51:30.029 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:30.029 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 1, 2024-04-07T17:51:30.029 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:30.029 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:30.029 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:30.030 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:30.030 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:30.030 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:30.030 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.331050+0000", 2024-04-07T17:51:30.030 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:08.961818+0000", 2024-04-07T17:51:30.030 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:08.961818+0000", 2024-04-07T17:51:30.030 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:08.962269+0000", 2024-04-07T17:51:30.030 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.331050+0000", 2024-04-07T17:51:30.030 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:30.030 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:30.030 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:30.030 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 554, 2024-04-07T17:51:30.030 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.331050+0000", 2024-04-07T17:51:30.030 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.331050+0000", 2024-04-07T17:51:30.031 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.331050+0000", 2024-04-07T17:51:30.031 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:30.031 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:30.031 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:30.031 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.331050+0000", 2024-04-07T17:51:30.031 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.331050+0000", 2024-04-07T17:51:30.031 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:30.031 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:30.031 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:30.031 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:30.031 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 553, 2024-04-07T17:51:30.031 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:30.031 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:30.031 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:30.031 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:30.032 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:30.032 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:30.032 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:30.032 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:30.032 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.0", 2024-04-07T17:51:30.032 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:30.032 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:30.032 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:30.032 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4985, 2024-04-07T17:51:30.032 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:30.032 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T01:33:47.318091+0000", 2024-04-07T17:51:30.032 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:30.032 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:30.032 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:30.032 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:30.033 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:30.033 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:30.033 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:30.033 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:30.033 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:30.033 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:30.033 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:30.033 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:30.033 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:30.033 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:30.033 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:30.033 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:30.033 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:30.033 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:30.033 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:30.033 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:30.034 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:30.034 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:30.034 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:30.034 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:30.034 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:30.034 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:30.034 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:30.034 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:30.034 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:30.034 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:30.034 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:30.034 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:30.034 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:30.034 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:30.035 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:30.035 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:30.035 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:30.035 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:30.035 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:30.035 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:30.035 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:30.035 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:30.035 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:30.035 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:30.035 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:30.035 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:30.035 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:30.035 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:30.035 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T17:51:30.035 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T17:51:30.036 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:30.036 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 1, 2024-04-07T17:51:30.036 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:30.036 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:30.036 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:30.036 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:30.036 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:30.036 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:30.036 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T17:51:30.036 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:30.036 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 1, 2024-04-07T17:51:30.036 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:30.036 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:30.036 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 66, 2024-04-07T17:51:30.036 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:30.037 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:30.037 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:30.037 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.332996+0000", 2024-04-07T17:51:30.037 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:24:41.570251+0000", 2024-04-07T17:51:30.037 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:24:41.570251+0000", 2024-04-07T17:51:30.037 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:59:56.540592+0000", 2024-04-07T17:51:30.037 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.332996+0000", 2024-04-07T17:51:30.037 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:13:28.963534+0000", 2024-04-07T17:51:30.037 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:30.037 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:13:28.963534+0000", 2024-04-07T17:51:30.037 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 527, 2024-04-07T17:51:30.037 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.332996+0000", 2024-04-07T17:51:30.037 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.332996+0000", 2024-04-07T17:51:30.037 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.332996+0000", 2024-04-07T17:51:30.037 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:30.038 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:30.038 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:13:28.963534+0000", 2024-04-07T17:51:30.038 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.332996+0000", 2024-04-07T17:51:30.038 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.332996+0000", 2024-04-07T17:51:30.038 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:30.038 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 2973, 2024-04-07T17:51:30.038 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:30.038 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:30.038 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 526, 2024-04-07T17:51:30.038 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:30.038 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:30.038 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:30.038 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:30.038 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 2973, 2024-04-07T17:51:30.039 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:30.039 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:30.039 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:30.039 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "4.7", 2024-04-07T17:51:30.039 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:30.039 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:30.039 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:30.039 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 8716, 2024-04-07T17:51:30.039 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:30.039 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T19:36:01.668815+0000", 2024-04-07T17:51:30.039 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.014750449, 2024-04-07T17:51:30.039 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:30.039 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:30.039 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:30.039 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:30.040 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:30.040 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:30.040 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:30.040 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:30.040 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:30.040 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:30.040 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:30.040 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:30.040 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:30.040 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:30.040 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:30.040 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:30.040 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:30.040 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:30.040 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:30.041 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:30.041 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:30.041 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:30.041 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:30.041 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:30.041 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:30.041 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:30.041 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:30.041 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:30.041 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:30.041 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:30.041 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:30.041 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:30.041 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:30.041 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:30.042 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:30.042 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 1318, 2024-04-07T17:51:30.042 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 368358, 2024-04-07T17:51:30.042 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:30.042 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:30.042 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:30.042 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 2957, 2024-04-07T17:51:30.042 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 380954 2024-04-07T17:51:30.042 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:30.042 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:30.042 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:30.042 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:30.042 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:30.042 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:30.042 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T17:51:30.043 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:30.043 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 1, 2024-04-07T17:51:30.043 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "1084'2973" 2024-04-07T17:51:30.043 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:30.043 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:30.043 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:30.043 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:30.043 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:30.043 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T17:51:30.043 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:30.043 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 4, 2024-04-07T17:51:30.043 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:30.043 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:30.043 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:30.043 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:30.043 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:30.044 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:30.044 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:35.979039+0000", 2024-04-07T17:51:30.044 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:08.952350+0000", 2024-04-07T17:51:30.044 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:08.952350+0000", 2024-04-07T17:51:30.044 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:08.952832+0000", 2024-04-07T17:51:30.044 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:35.979039+0000", 2024-04-07T17:51:30.044 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:30.044 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:30.044 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:30.044 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 554, 2024-04-07T17:51:30.044 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:35.979039+0000", 2024-04-07T17:51:30.044 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:35.979039+0000", 2024-04-07T17:51:30.044 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:35.979039+0000", 2024-04-07T17:51:30.044 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:30.044 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:30.045 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:30.045 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:35.979039+0000", 2024-04-07T17:51:30.045 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:35.979039+0000", 2024-04-07T17:51:30.045 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:30.045 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:30.045 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:30.045 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:30.045 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 553, 2024-04-07T17:51:30.045 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:30.045 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:30.045 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:30.045 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:30.045 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:30.045 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:30.045 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:30.046 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:30.046 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.3", 2024-04-07T17:51:30.046 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:30.046 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:30.046 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:30.046 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4919, 2024-04-07T17:51:30.046 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:30.046 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T22:57:40.507778+0000", 2024-04-07T17:51:30.046 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:30.046 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:30.046 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:30.046 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:30.046 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:30.046 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:30.046 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:30.047 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:30.047 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:30.047 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:30.047 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:30.047 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:30.047 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:30.047 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:30.047 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:30.047 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:30.047 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:30.047 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:30.047 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:30.047 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:30.047 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:30.047 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:30.047 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:30.048 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:30.048 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:30.048 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:30.048 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:30.048 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:30.048 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:30.048 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:30.048 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:30.048 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:30.048 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:30.048 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:30.048 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:30.048 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:30.048 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:30.048 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:30.049 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:30.049 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:30.049 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:30.049 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:30.049 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:30.049 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:30.049 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:30.049 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:30.049 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:30.049 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:30.049 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:30.049 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T17:51:30.049 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:30.049 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 4, 2024-04-07T17:51:30.049 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:30.049 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:30.050 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:30.050 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:30.050 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:30.050 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:30.050 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T17:51:30.050 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:30.050 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 5, 2024-04-07T17:51:30.050 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:30.050 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:30.050 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T17:51:30.050 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:30.050 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:30.050 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:30.050 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.369727+0000", 2024-04-07T17:51:30.050 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:08.972663+0000", 2024-04-07T17:51:30.050 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:08.972663+0000", 2024-04-07T17:51:30.051 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:08.973250+0000", 2024-04-07T17:51:30.051 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.369727+0000", 2024-04-07T17:51:30.051 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:30.051 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:30.051 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:30.051 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 554, 2024-04-07T17:51:30.051 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.369727+0000", 2024-04-07T17:51:30.051 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.369727+0000", 2024-04-07T17:51:30.051 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.369727+0000", 2024-04-07T17:51:30.051 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:30.051 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:30.051 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:30.051 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.369727+0000", 2024-04-07T17:51:30.051 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.369727+0000", 2024-04-07T17:51:30.052 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:30.052 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:30.052 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:30.052 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:30.052 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 553, 2024-04-07T17:51:30.052 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:30.052 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:30.052 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:30.052 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:30.052 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:30.052 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:30.052 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:30.052 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:30.052 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.2", 2024-04-07T17:51:30.052 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:30.052 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:30.053 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:30.053 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4868, 2024-04-07T17:51:30.053 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:30.053 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T22:42:44.603307+0000", 2024-04-07T17:51:30.053 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:30.053 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:30.053 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:30.053 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:30.053 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:30.053 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:30.053 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:30.053 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:30.053 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:30.053 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:30.053 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:30.054 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:30.054 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:30.054 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:30.054 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:30.054 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:30.054 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:30.054 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:30.054 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:30.054 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:30.054 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:30.054 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:30.054 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:30.054 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:30.054 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:30.054 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:30.055 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:30.055 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:30.055 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:30.055 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:30.055 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:30.055 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:30.055 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:30.055 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:30.055 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:30.055 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:30.055 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:30.055 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:30.055 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:30.055 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:30.055 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:30.056 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:30.056 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:30.056 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:30.056 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:30.056 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:30.056 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:30.056 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:30.056 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:30.056 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T17:51:30.056 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:30.056 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 5, 2024-04-07T17:51:30.056 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:30.056 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:30.056 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:30.056 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:30.057 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:30.057 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T17:51:30.057 INFO:tasks.workunit.client.0.smithi100.stdout: 3 2024-04-07T17:51:30.057 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:30.057 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 1, 2024-04-07T17:51:30.057 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:30.057 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:30.057 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 66, 2024-04-07T17:51:30.057 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:30.057 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:30.057 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:30.057 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:51:24.825515+0000", 2024-04-07T17:51:30.057 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:23:43.924443+0000", 2024-04-07T17:51:30.057 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:23:43.924443+0000", 2024-04-07T17:51:30.057 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:59:56.538443+0000", 2024-04-07T17:51:30.058 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:51:24.825515+0000", 2024-04-07T17:51:30.058 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:13:28.963534+0000", 2024-04-07T17:51:30.058 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:30.058 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:13:28.963534+0000", 2024-04-07T17:51:30.058 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 471, 2024-04-07T17:51:30.058 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:51:24.825515+0000", 2024-04-07T17:51:30.058 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:51:24.825515+0000", 2024-04-07T17:51:30.058 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:51:24.825515+0000", 2024-04-07T17:51:30.058 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:30.058 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:30.058 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:13:28.963534+0000", 2024-04-07T17:51:30.058 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:51:24.825515+0000", 2024-04-07T17:51:30.058 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:51:24.825515+0000", 2024-04-07T17:51:30.058 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:30.058 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 2976, 2024-04-07T17:51:30.059 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:30.059 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:30.059 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 470, 2024-04-07T17:51:30.059 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:30.059 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:30.059 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:30.059 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:30.059 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 2976, 2024-04-07T17:51:30.059 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:30.059 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:30.059 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:30.059 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "4.4", 2024-04-07T17:51:30.059 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:30.059 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:30.059 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:30.060 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 8722, 2024-04-07T17:51:30.060 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:30.060 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T16:43:02.513339+0000", 2024-04-07T17:51:30.060 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.010915499, 2024-04-07T17:51:30.060 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:30.060 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:30.060 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:30.060 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:30.060 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:30.060 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:30.060 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:30.060 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:30.060 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:30.060 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:30.060 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:30.060 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:30.061 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:30.061 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:30.061 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:30.061 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:30.061 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:30.061 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:30.061 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 3, 2024-04-07T17:51:30.061 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 1, 2024-04-07T17:51:30.061 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:30.061 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 1, 2024-04-07T17:51:30.061 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:30.061 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:30.061 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:30.061 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:30.061 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:30.062 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 1, 2024-04-07T17:51:30.062 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:30.062 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:30.062 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:30.062 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:30.062 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:30.062 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:30.062 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:30.062 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 1196, 2024-04-07T17:51:30.062 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 375705, 2024-04-07T17:51:30.062 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:30.062 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:30.062 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:30.062 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 2976, 2024-04-07T17:51:30.062 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 350581 2024-04-07T17:51:30.063 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:30.063 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:30.063 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:30.063 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:30.063 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:30.063 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T17:51:30.063 INFO:tasks.workunit.client.0.smithi100.stdout: 3 2024-04-07T17:51:30.063 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:30.063 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 1, 2024-04-07T17:51:30.063 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "1084'2976" 2024-04-07T17:51:30.063 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:30.063 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:30.063 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:30.063 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:30.063 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:30.063 INFO:tasks.workunit.client.0.smithi100.stdout: 2 2024-04-07T17:51:30.064 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:30.064 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 5, 2024-04-07T17:51:30.064 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:30.064 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:30.064 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:30.064 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:30.064 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:30.064 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:30.064 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.364565+0000", 2024-04-07T17:51:30.064 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:24:41.554485+0000", 2024-04-07T17:51:30.064 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:24:41.554485+0000", 2024-04-07T17:51:30.064 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:24:41.557859+0000", 2024-04-07T17:51:30.064 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.364565+0000", 2024-04-07T17:51:30.064 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:30.064 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:30.064 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:30.065 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 527, 2024-04-07T17:51:30.065 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.364565+0000", 2024-04-07T17:51:30.065 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.364565+0000", 2024-04-07T17:51:30.065 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.364565+0000", 2024-04-07T17:51:30.065 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:30.065 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:30.065 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:30.065 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.364565+0000", 2024-04-07T17:51:30.065 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.364565+0000", 2024-04-07T17:51:30.065 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:30.065 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:30.065 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:30.065 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:30.065 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 526, 2024-04-07T17:51:30.066 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:30.066 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:30.066 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:30.066 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:30.066 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:30.066 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:30.066 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:30.066 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:30.066 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.5", 2024-04-07T17:51:30.066 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:30.066 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:30.066 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:30.066 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4886, 2024-04-07T17:51:30.066 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:30.067 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T18:53:57.243053+0000", 2024-04-07T17:51:30.067 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:30.067 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:30.067 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:30.067 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:30.067 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:30.067 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:30.067 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:30.067 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:30.067 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:30.067 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:30.067 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:30.067 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:30.067 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:30.067 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:30.068 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:30.068 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:30.068 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:30.068 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:30.068 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:30.068 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:30.068 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:30.068 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:30.068 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:30.068 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:30.068 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:30.068 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:30.068 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:30.068 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:30.068 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:30.069 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:30.069 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:30.069 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:30.069 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:30.069 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:30.069 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:30.069 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:30.069 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:30.069 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:30.069 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:30.069 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:30.069 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:30.069 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:30.069 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:30.069 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:30.070 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:30.070 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:30.070 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:30.070 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:30.070 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:30.070 INFO:tasks.workunit.client.0.smithi100.stdout: 2 2024-04-07T17:51:30.070 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:30.070 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 5, 2024-04-07T17:51:30.070 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:30.070 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:30.070 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:30.070 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:30.070 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:30.070 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:30.071 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T17:51:30.071 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:30.071 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 1, 2024-04-07T17:51:30.071 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:30.071 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:30.071 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T17:51:30.071 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:30.071 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:30.071 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:30.071 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.333650+0000", 2024-04-07T17:51:30.071 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:38.999666+0000", 2024-04-07T17:51:30.071 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:38.999666+0000", 2024-04-07T17:51:30.071 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:39.000176+0000", 2024-04-07T17:51:30.071 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.333650+0000", 2024-04-07T17:51:30.071 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:30.072 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:30.072 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:30.072 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:30.072 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.333650+0000", 2024-04-07T17:51:30.072 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.333650+0000", 2024-04-07T17:51:30.072 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.333650+0000", 2024-04-07T17:51:30.072 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:30.072 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:30.072 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:30.072 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.333650+0000", 2024-04-07T17:51:30.072 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.333650+0000", 2024-04-07T17:51:30.072 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:30.072 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:30.072 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:30.072 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:30.073 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:30.073 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:30.073 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:30.073 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:30.073 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:30.073 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:30.073 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:30.073 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:30.073 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:30.073 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.4", 2024-04-07T17:51:30.073 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:30.073 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:30.073 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:30.073 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4990, 2024-04-07T17:51:30.073 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:30.074 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T19:36:37.701087+0000", 2024-04-07T17:51:30.074 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:30.074 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:30.074 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:30.074 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:30.074 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:30.074 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:30.074 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:30.074 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:30.074 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:30.074 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:30.074 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:30.074 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:30.074 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:30.074 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:30.075 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:30.075 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:30.075 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:30.075 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:30.075 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:30.075 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:30.075 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:30.075 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:30.075 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:30.075 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:30.075 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:30.075 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:30.075 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:30.075 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:30.075 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:30.075 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:30.076 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:30.076 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:30.076 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:30.076 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:30.076 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:30.076 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:30.076 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:30.076 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:30.076 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:30.076 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:30.076 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:30.076 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:30.076 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:30.076 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:30.076 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:30.077 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:30.077 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:30.077 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:30.077 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:30.077 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T17:51:30.077 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:30.077 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 1, 2024-04-07T17:51:30.077 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:30.077 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:30.077 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:30.077 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:30.077 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:30.077 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:30.077 INFO:tasks.workunit.client.0.smithi100.stdout: 4 2024-04-07T17:51:30.077 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:30.078 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 1, 2024-04-07T17:51:30.078 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:30.078 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:30.078 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 66, 2024-04-07T17:51:30.078 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:30.078 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:30.078 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:30.078 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.335612+0000", 2024-04-07T17:51:30.078 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:24:41.537435+0000", 2024-04-07T17:51:30.078 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:24:41.537435+0000", 2024-04-07T17:51:30.078 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:59:56.544979+0000", 2024-04-07T17:51:30.078 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.335612+0000", 2024-04-07T17:51:30.078 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:13:28.963534+0000", 2024-04-07T17:51:30.078 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:30.078 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:13:28.963534+0000", 2024-04-07T17:51:30.079 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 527, 2024-04-07T17:51:30.079 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.335612+0000", 2024-04-07T17:51:30.079 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.335612+0000", 2024-04-07T17:51:30.079 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.335612+0000", 2024-04-07T17:51:30.079 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:30.079 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:30.079 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:13:28.963534+0000", 2024-04-07T17:51:30.079 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.335612+0000", 2024-04-07T17:51:30.079 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.335612+0000", 2024-04-07T17:51:30.079 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:30.079 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 2963, 2024-04-07T17:51:30.079 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:30.079 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:30.079 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 526, 2024-04-07T17:51:30.079 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:30.080 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:30.080 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:30.080 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:30.080 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 2963, 2024-04-07T17:51:30.080 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:30.080 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:30.080 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:30.080 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "4.2", 2024-04-07T17:51:30.080 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:30.080 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:30.080 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:30.080 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 8984, 2024-04-07T17:51:30.080 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:30.080 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T23:06:36.963212+0000", 2024-04-07T17:51:30.080 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.015938325, 2024-04-07T17:51:30.081 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:30.081 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:30.081 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 19, 2024-04-07T17:51:30.081 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:30.081 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:30.081 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:30.081 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:30.081 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:30.081 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:30.081 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:30.081 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:30.081 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:30.081 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:30.081 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:30.081 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:30.081 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:30.082 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:30.082 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:30.082 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 3, 2024-04-07T17:51:30.082 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 1, 2024-04-07T17:51:30.082 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:30.082 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 1, 2024-04-07T17:51:30.082 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:30.082 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:30.082 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:30.082 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:30.082 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:30.082 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:30.082 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:30.082 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:30.083 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:30.083 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:30.083 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:30.083 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:30.083 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:30.083 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 1654, 2024-04-07T17:51:30.083 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 387082, 2024-04-07T17:51:30.083 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:30.083 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:30.083 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:30.083 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 2937, 2024-04-07T17:51:30.083 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 349256 2024-04-07T17:51:30.083 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:30.083 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:30.083 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:30.083 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:30.084 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:30.084 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:30.084 INFO:tasks.workunit.client.0.smithi100.stdout: 4 2024-04-07T17:51:30.084 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:30.084 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 1, 2024-04-07T17:51:30.084 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "1083'2963" 2024-04-07T17:51:30.084 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:30.084 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:30.084 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:30.084 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:30.084 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:30.084 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T17:51:30.084 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:30.084 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 7, 2024-04-07T17:51:30.084 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:30.085 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:30.085 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:30.085 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:30.085 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:30.085 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:30.085 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.372374+0000", 2024-04-07T17:51:30.085 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:38.968892+0000", 2024-04-07T17:51:30.085 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:38.968892+0000", 2024-04-07T17:51:30.085 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:38.970807+0000", 2024-04-07T17:51:30.085 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.372374+0000", 2024-04-07T17:51:30.085 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:30.085 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:30.085 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:30.085 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:30.085 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.372374+0000", 2024-04-07T17:51:30.086 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.372374+0000", 2024-04-07T17:51:30.086 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.372374+0000", 2024-04-07T17:51:30.086 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:30.086 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:30.086 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:30.086 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.372374+0000", 2024-04-07T17:51:30.086 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.372374+0000", 2024-04-07T17:51:30.086 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:30.086 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:30.086 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:30.086 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:30.086 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:30.086 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:30.086 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:30.087 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:30.087 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:30.087 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:30.087 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:30.087 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:30.087 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:30.087 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.e", 2024-04-07T17:51:30.087 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:30.087 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:30.087 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:30.087 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4791, 2024-04-07T17:51:30.087 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:30.087 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T21:17:23.974869+0000", 2024-04-07T17:51:30.087 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:30.088 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:30.088 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:30.088 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:30.088 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:30.088 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:30.088 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:30.088 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:30.088 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:30.088 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:30.088 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:30.088 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:30.088 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:30.088 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:30.088 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:30.088 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:30.088 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:30.089 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:30.089 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:30.089 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:30.089 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:30.089 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:30.089 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:30.089 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:30.089 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:30.089 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:30.089 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:30.089 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:30.089 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:30.089 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:30.089 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:30.089 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:30.089 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:30.090 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:30.090 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:30.090 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:30.090 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:30.090 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:30.090 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:30.090 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:30.090 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:30.090 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:30.090 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:30.090 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:30.090 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:30.090 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:30.090 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:30.091 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:30.091 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:30.091 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T17:51:30.091 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:30.091 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 7, 2024-04-07T17:51:30.091 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:30.091 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:30.091 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:30.091 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:30.091 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:30.091 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:30.091 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T17:51:30.091 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:30.091 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 4, 2024-04-07T17:51:30.091 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:30.092 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:30.092 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T17:51:30.092 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:30.092 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:30.092 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:30.092 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:35.981613+0000", 2024-04-07T17:51:30.092 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:38.967913+0000", 2024-04-07T17:51:30.092 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:38.967913+0000", 2024-04-07T17:51:30.092 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:38.968805+0000", 2024-04-07T17:51:30.092 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:35.981613+0000", 2024-04-07T17:51:30.092 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:30.092 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:30.092 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:30.092 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:30.092 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:35.981613+0000", 2024-04-07T17:51:30.093 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:35.981613+0000", 2024-04-07T17:51:30.093 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:35.981613+0000", 2024-04-07T17:51:30.093 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:30.093 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:30.093 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:30.093 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:35.981613+0000", 2024-04-07T17:51:30.093 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:35.981613+0000", 2024-04-07T17:51:30.093 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:30.093 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 1, 2024-04-07T17:51:30.093 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:30.093 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:30.093 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:30.093 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:30.093 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:30.094 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:30.094 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:30.094 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 1, 2024-04-07T17:51:30.094 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:30.094 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:30.094 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:30.094 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.f", 2024-04-07T17:51:30.094 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:30.094 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:30.094 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:30.094 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4919, 2024-04-07T17:51:30.094 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:30.094 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T19:43:23.875589+0000", 2024-04-07T17:51:30.094 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:30.094 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:30.095 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:30.095 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 22, 2024-04-07T17:51:30.095 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:30.095 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:30.095 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:30.095 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:30.095 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:30.095 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:30.095 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:30.095 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:30.095 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:30.095 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:30.095 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:30.095 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:30.096 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:30.096 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:30.096 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:30.096 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 3, 2024-04-07T17:51:30.096 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 1, 2024-04-07T17:51:30.096 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:30.096 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 1, 2024-04-07T17:51:30.096 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:30.096 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:30.096 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:30.096 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:30.096 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:30.096 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:30.096 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:30.096 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:30.097 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:30.097 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:30.097 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:30.097 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:30.097 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:30.097 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 1, 2024-04-07T17:51:30.097 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 1, 2024-04-07T17:51:30.097 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:30.097 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:30.097 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:30.097 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 1, 2024-04-07T17:51:30.097 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 1 2024-04-07T17:51:30.097 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:30.097 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:30.097 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:30.098 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:30.098 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:30.098 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:30.098 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T17:51:30.098 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:30.098 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 4, 2024-04-07T17:51:30.098 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "61'1" 2024-04-07T17:51:30.098 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:30.098 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:30.098 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:30.098 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:30.098 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:30.098 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T17:51:30.098 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:30.099 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 7, 2024-04-07T17:51:30.099 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:30.099 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:30.099 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:30.099 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:30.099 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:30.099 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:30.099 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.358184+0000", 2024-04-07T17:51:30.099 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:38.987996+0000", 2024-04-07T17:51:30.099 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:38.987996+0000", 2024-04-07T17:51:30.099 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:38.989241+0000", 2024-04-07T17:51:30.099 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.358184+0000", 2024-04-07T17:51:30.099 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:30.099 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:30.100 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:30.100 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:30.100 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.358184+0000", 2024-04-07T17:51:30.100 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.358184+0000", 2024-04-07T17:51:30.100 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.358184+0000", 2024-04-07T17:51:30.100 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:30.100 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:30.100 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:30.100 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.358184+0000", 2024-04-07T17:51:30.100 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.358184+0000", 2024-04-07T17:51:30.100 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:30.100 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:30.100 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:30.100 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:30.100 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:30.101 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:30.101 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:30.101 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:30.101 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:30.101 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:30.101 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:30.101 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:30.101 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:30.101 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.11", 2024-04-07T17:51:30.101 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:30.101 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:30.101 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:30.101 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4801, 2024-04-07T17:51:30.101 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:30.101 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T16:46:00.763122+0000", 2024-04-07T17:51:30.101 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:30.102 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:30.102 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:30.102 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:30.102 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:30.102 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:30.102 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:30.102 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:30.102 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:30.102 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:30.102 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:30.102 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:30.102 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:30.102 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:30.102 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:30.102 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:30.103 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:30.103 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:30.103 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:30.103 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:30.103 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:30.103 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:30.103 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:30.103 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:30.103 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:30.103 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:30.103 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:30.103 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:30.103 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:30.103 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:30.103 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:30.103 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:30.104 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:30.104 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:30.104 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:30.104 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:30.104 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:30.104 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:30.104 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:30.104 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:30.104 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:30.104 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:30.104 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:30.104 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:30.104 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:30.105 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:30.105 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:30.105 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:30.105 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:30.105 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T17:51:30.105 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:30.105 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 7, 2024-04-07T17:51:30.105 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:30.105 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:30.105 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:30.105 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:30.105 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T17:51:30.105 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:30.105 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T17:51:30.105 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:30.106 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 2, 2024-04-07T17:51:30.106 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:30.106 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:30.106 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T17:51:30.106 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:30.106 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:30.106 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:30.106 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.354070+0000", 2024-04-07T17:51:30.106 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:23:15.875351+0000", 2024-04-07T17:51:30.106 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:23:15.875351+0000", 2024-04-07T17:51:30.106 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:23:15.880710+0000", 2024-04-07T17:51:30.106 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.354070+0000", 2024-04-07T17:51:30.106 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:30.106 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:30.107 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:30.107 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 445, 2024-04-07T17:51:30.107 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.354070+0000", 2024-04-07T17:51:30.107 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.354070+0000", 2024-04-07T17:51:30.107 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.354070+0000", 2024-04-07T17:51:30.107 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:30.107 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:30.107 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:30.107 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.354070+0000", 2024-04-07T17:51:30.107 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.354070+0000", 2024-04-07T17:51:30.107 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:30.107 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 8, 2024-04-07T17:51:30.107 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:30.107 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:30.108 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 444, 2024-04-07T17:51:30.108 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:30.108 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:30.108 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:30.108 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:30.108 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 8, 2024-04-07T17:51:30.108 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:30.108 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:30.108 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:30.108 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.10", 2024-04-07T17:51:30.108 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:30.108 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:30.108 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:30.108 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4979, 2024-04-07T17:51:30.108 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:30.108 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T17:16:20.268788+0000", 2024-04-07T17:51:30.109 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:30.109 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:30.109 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:30.109 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:30.109 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:30.109 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:30.109 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:30.109 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:30.109 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:30.109 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:30.109 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:30.109 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:30.109 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:30.109 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:30.109 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:30.110 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:30.110 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:30.110 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:30.110 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:30.110 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 12, 2024-04-07T17:51:30.110 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 4, 2024-04-07T17:51:30.110 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:30.110 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 4, 2024-04-07T17:51:30.110 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:30.110 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:30.110 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:30.110 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:30.110 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:30.110 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 4, 2024-04-07T17:51:30.110 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:30.111 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:30.111 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:30.111 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:30.111 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:30.111 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:30.111 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:30.111 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 24, 2024-04-07T17:51:30.111 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 24, 2024-04-07T17:51:30.111 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:30.111 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:30.111 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:30.111 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 12, 2024-04-07T17:51:30.111 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:30.111 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:30.111 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:30.111 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:30.112 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:30.112 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T17:51:30.112 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:30.112 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T17:51:30.112 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:30.112 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 2, 2024-04-07T17:51:30.112 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "61'8" 2024-04-07T17:51:30.112 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:30.112 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:30.112 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:30.112 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:30.112 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:30.112 INFO:tasks.workunit.client.0.smithi100.stdout: 5 2024-04-07T17:51:30.112 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:30.112 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 6, 2024-04-07T17:51:30.113 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:30.113 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:30.113 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:30.113 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:30.113 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:30.113 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:30.113 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.370701+0000", 2024-04-07T17:51:30.113 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:08.963259+0000", 2024-04-07T17:51:30.113 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:08.963259+0000", 2024-04-07T17:51:30.113 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:08.964055+0000", 2024-04-07T17:51:30.113 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.370701+0000", 2024-04-07T17:51:30.113 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:30.113 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:30.113 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:30.113 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 554, 2024-04-07T17:51:30.113 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.370701+0000", 2024-04-07T17:51:30.114 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.370701+0000", 2024-04-07T17:51:30.114 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.370701+0000", 2024-04-07T17:51:30.114 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:30.114 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:30.114 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:30.114 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.370701+0000", 2024-04-07T17:51:30.114 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.370701+0000", 2024-04-07T17:51:30.114 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:30.114 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:30.114 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:30.114 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:30.114 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 553, 2024-04-07T17:51:30.114 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:30.114 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:30.114 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:30.115 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:30.115 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:30.115 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:30.115 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:30.115 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:30.115 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.10", 2024-04-07T17:51:30.115 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:30.115 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:30.115 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:30.115 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4830, 2024-04-07T17:51:30.115 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:30.115 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T23:13:34.886609+0000", 2024-04-07T17:51:30.115 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:30.115 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:30.115 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:30.116 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:30.116 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:30.116 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:30.116 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:30.116 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:30.116 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:30.116 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:30.116 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:30.116 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:30.116 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:30.116 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:30.116 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:30.116 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:30.116 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:30.116 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:30.116 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:30.117 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:30.117 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:30.117 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:30.117 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:30.117 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:30.117 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:30.117 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:30.117 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:30.117 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:30.117 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:30.117 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:30.117 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:30.117 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:30.117 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:30.117 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:30.118 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:30.118 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:30.118 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:30.118 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:30.118 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:30.118 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:30.118 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:30.118 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:30.118 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:30.118 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:30.118 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:30.118 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:30.118 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:30.118 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:30.118 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:30.119 INFO:tasks.workunit.client.0.smithi100.stdout: 5 2024-04-07T17:51:30.119 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:30.119 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 6, 2024-04-07T17:51:30.119 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:30.119 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:30.119 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:30.119 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:30.119 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:30.119 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:30.119 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T17:51:30.119 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:30.119 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 6, 2024-04-07T17:51:30.119 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:30.119 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:30.119 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T17:51:30.120 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:30.120 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:30.120 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:30.120 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.371180+0000", 2024-04-07T17:51:30.120 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:08.965799+0000", 2024-04-07T17:51:30.120 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:08.965799+0000", 2024-04-07T17:51:30.120 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:08.966301+0000", 2024-04-07T17:51:30.120 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.371180+0000", 2024-04-07T17:51:30.120 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:30.120 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:30.120 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:30.120 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 554, 2024-04-07T17:51:30.120 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.371180+0000", 2024-04-07T17:51:30.120 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.371180+0000", 2024-04-07T17:51:30.120 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.371180+0000", 2024-04-07T17:51:30.120 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:30.121 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:30.121 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:30.121 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.371180+0000", 2024-04-07T17:51:30.121 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.371180+0000", 2024-04-07T17:51:30.121 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:30.121 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:30.121 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:30.121 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:30.121 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 553, 2024-04-07T17:51:30.121 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:30.121 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:30.121 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:30.121 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:30.121 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:30.122 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:30.122 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:30.122 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:30.122 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.11", 2024-04-07T17:51:30.122 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:30.122 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:30.122 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:30.122 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4846, 2024-04-07T17:51:30.122 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:30.122 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T21:12:09.705565+0000", 2024-04-07T17:51:30.122 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:30.122 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:30.122 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:30.122 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:30.122 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:30.123 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:30.123 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:30.123 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:30.123 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:30.123 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:30.123 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:30.123 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:30.123 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:30.123 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:30.123 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:30.123 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:30.123 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:30.123 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:30.123 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:30.123 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:30.124 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:30.124 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:30.124 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:30.124 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:30.124 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:30.124 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:30.124 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:30.124 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:30.124 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:30.124 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:30.124 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:30.124 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:30.124 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:30.124 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:30.124 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:30.124 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:30.125 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:30.125 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:30.125 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:30.125 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:30.125 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:30.125 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:30.125 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:30.125 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:30.125 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:30.125 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:30.125 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:30.125 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:30.125 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:30.125 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T17:51:30.125 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:30.125 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 6, 2024-04-07T17:51:30.126 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:30.126 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:30.126 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:30.126 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:30.126 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:30.126 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:30.126 INFO:tasks.workunit.client.0.smithi100.stdout: 2 2024-04-07T17:51:30.126 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:30.126 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 7, 2024-04-07T17:51:30.126 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:30.126 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:30.126 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:30.126 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:30.126 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:30.127 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:30.127 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.368748+0000", 2024-04-07T17:51:30.127 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:38.987994+0000", 2024-04-07T17:51:30.127 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:38.987994+0000", 2024-04-07T17:51:30.127 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:38.991515+0000", 2024-04-07T17:51:30.127 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.368748+0000", 2024-04-07T17:51:30.127 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:30.127 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:30.127 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:30.127 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:30.127 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.368748+0000", 2024-04-07T17:51:30.127 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.368748+0000", 2024-04-07T17:51:30.127 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.368748+0000", 2024-04-07T17:51:30.127 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:30.127 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:30.128 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:30.128 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.368748+0000", 2024-04-07T17:51:30.128 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.368748+0000", 2024-04-07T17:51:30.128 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:30.128 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:30.128 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:30.128 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:30.128 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:30.128 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:30.128 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:30.128 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:30.128 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:30.128 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:30.128 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:30.128 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:30.128 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:30.129 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.13", 2024-04-07T17:51:30.129 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:30.129 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:30.129 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:30.129 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4789, 2024-04-07T17:51:30.129 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:30.129 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T20:51:08.365772+0000", 2024-04-07T17:51:30.129 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:30.129 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:30.129 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:30.129 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:30.129 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:30.129 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:30.129 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:30.129 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:30.130 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:30.130 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:30.130 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:30.130 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:30.130 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:30.130 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:30.130 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:30.130 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:30.130 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:30.130 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:30.130 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:30.130 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:30.130 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:30.130 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:30.130 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:30.131 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:30.131 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:30.131 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:30.131 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:30.131 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:30.131 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:30.131 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:30.131 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:30.131 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:30.131 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:30.131 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:30.131 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:30.131 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:30.131 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:30.131 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:30.131 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:30.132 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:30.132 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:30.132 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:30.132 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:30.132 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:30.132 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:30.132 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:30.132 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:30.132 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:30.132 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:30.132 INFO:tasks.workunit.client.0.smithi100.stdout: 2 2024-04-07T17:51:30.132 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:30.132 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 7, 2024-04-07T17:51:30.132 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:30.132 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:30.132 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:30.133 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:30.133 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:30.133 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:30.133 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T17:51:30.133 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:30.133 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 5, 2024-04-07T17:51:30.133 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:30.133 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:30.133 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T17:51:30.133 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:30.133 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:30.133 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:30.133 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.369416+0000", 2024-04-07T17:51:30.133 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:38.978180+0000", 2024-04-07T17:51:30.133 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:38.978180+0000", 2024-04-07T17:51:30.133 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:38.979026+0000", 2024-04-07T17:51:30.134 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.369416+0000", 2024-04-07T17:51:30.134 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:30.134 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:30.134 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:30.134 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:30.134 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.369416+0000", 2024-04-07T17:51:30.134 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.369416+0000", 2024-04-07T17:51:30.134 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.369416+0000", 2024-04-07T17:51:30.134 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:30.134 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:30.134 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:30.134 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.369416+0000", 2024-04-07T17:51:30.134 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.369416+0000", 2024-04-07T17:51:30.134 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:30.134 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:30.135 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:30.135 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:30.135 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:30.135 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:30.135 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:30.135 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:30.135 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:30.135 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:30.135 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:30.135 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:30.135 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:30.135 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.12", 2024-04-07T17:51:30.135 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:30.135 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:30.135 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:30.136 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4877, 2024-04-07T17:51:30.136 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:30.136 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T01:41:45.196090+0000", 2024-04-07T17:51:30.136 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:30.136 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:30.136 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:30.136 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:30.136 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:30.136 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:30.136 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:30.136 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:30.136 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:30.136 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:30.136 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:30.136 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:30.137 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:30.137 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:30.137 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:30.137 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:30.137 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:30.137 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:30.137 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:30.137 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:30.137 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:30.137 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:30.137 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:30.137 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:30.137 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:30.137 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:30.137 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:30.137 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:30.138 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:30.138 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:30.138 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:30.138 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:30.138 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:30.138 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:30.138 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:30.138 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:30.138 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:30.138 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:30.138 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:30.138 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:30.138 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:30.138 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:30.138 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:30.139 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:30.139 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:30.139 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:30.139 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:30.139 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:30.139 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:30.139 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T17:51:30.139 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:30.139 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 5, 2024-04-07T17:51:30.139 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:30.139 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:30.139 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:30.139 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:30.139 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T17:51:30.139 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:30.140 INFO:tasks.workunit.client.0.smithi100.stdout: 3 2024-04-07T17:51:30.140 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:30.140 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 2, 2024-04-07T17:51:30.140 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:30.140 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:30.140 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:30.140 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:30.140 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:30.140 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:30.140 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.354815+0000", 2024-04-07T17:51:30.140 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:38.969402+0000", 2024-04-07T17:51:30.140 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:38.969402+0000", 2024-04-07T17:51:30.140 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:38.970054+0000", 2024-04-07T17:51:30.140 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.354815+0000", 2024-04-07T17:51:30.140 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:30.140 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:30.141 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:30.141 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:30.141 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.354815+0000", 2024-04-07T17:51:30.141 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.354815+0000", 2024-04-07T17:51:30.141 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.354815+0000", 2024-04-07T17:51:30.141 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:30.141 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:30.141 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:30.141 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.354815+0000", 2024-04-07T17:51:30.141 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.354815+0000", 2024-04-07T17:51:30.141 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:30.141 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:30.141 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:30.141 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:30.141 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:30.142 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:30.142 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:30.142 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:30.142 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:30.142 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:30.142 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:30.142 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:30.142 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:30.142 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.12", 2024-04-07T17:51:30.142 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:30.142 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:30.142 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:30.142 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4924, 2024-04-07T17:51:30.142 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:30.142 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T23:11:21.401536+0000", 2024-04-07T17:51:30.142 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:30.143 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:30.143 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:30.143 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:30.143 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:30.143 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:30.143 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:30.143 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:30.143 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:30.143 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:30.143 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:30.143 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:30.143 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:30.143 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:30.143 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:30.143 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:30.144 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:30.144 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:30.144 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:30.144 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:30.144 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:30.144 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:30.144 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:30.144 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:30.144 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:30.144 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:30.144 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:30.144 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:30.144 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:30.144 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:30.144 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:30.144 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:30.144 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:30.145 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:30.145 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:30.145 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:30.145 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:30.145 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:30.145 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:30.145 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:30.145 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:30.145 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:30.145 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:30.145 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:30.145 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:30.145 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:30.145 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:30.145 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T17:51:30.146 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:30.146 INFO:tasks.workunit.client.0.smithi100.stdout: 3 2024-04-07T17:51:30.146 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:30.146 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 2, 2024-04-07T17:51:30.146 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:30.146 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:30.146 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:30.146 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:30.146 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:30.146 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:30.146 INFO:tasks.workunit.client.0.smithi100.stdout: 3 2024-04-07T17:51:30.146 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:30.146 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 0, 2024-04-07T17:51:30.146 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:30.147 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:30.147 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T17:51:30.147 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:30.147 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:30.147 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:30.147 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:35.988325+0000", 2024-04-07T17:51:30.147 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:24:13.963512+0000", 2024-04-07T17:51:30.147 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:24:13.963512+0000", 2024-04-07T17:51:30.147 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:24:13.964689+0000", 2024-04-07T17:51:30.147 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:35.988325+0000", 2024-04-07T17:51:30.147 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:30.147 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:30.147 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:30.147 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 500, 2024-04-07T17:51:30.147 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:35.988325+0000", 2024-04-07T17:51:30.147 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:35.988325+0000", 2024-04-07T17:51:30.148 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:35.988325+0000", 2024-04-07T17:51:30.148 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:30.148 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:30.148 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:30.148 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:35.988325+0000", 2024-04-07T17:51:30.148 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:35.988325+0000", 2024-04-07T17:51:30.148 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:30.148 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 2, 2024-04-07T17:51:30.148 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:30.148 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:30.148 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 499, 2024-04-07T17:51:30.148 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:30.148 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:30.148 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:30.148 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:30.148 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 2, 2024-04-07T17:51:30.149 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:30.149 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:30.149 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:30.149 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.13", 2024-04-07T17:51:30.149 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:30.149 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:30.149 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:30.149 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 5000, 2024-04-07T17:51:30.149 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:30.149 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T16:15:37.955732+0000", 2024-04-07T17:51:30.149 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:30.149 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:30.149 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:30.149 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 34, 2024-04-07T17:51:30.149 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:30.150 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:30.150 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:30.150 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:30.150 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:30.150 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:30.150 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:30.150 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:30.150 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:30.150 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:30.150 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:30.150 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:30.150 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:30.150 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:30.150 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:30.150 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 6, 2024-04-07T17:51:30.151 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 2, 2024-04-07T17:51:30.151 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:30.151 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 2, 2024-04-07T17:51:30.151 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:30.151 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:30.151 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:30.151 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:30.151 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:30.151 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 1, 2024-04-07T17:51:30.151 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:30.151 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:30.151 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:30.151 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:30.151 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:30.151 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:30.151 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:30.152 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 4, 2024-04-07T17:51:30.152 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 8, 2024-04-07T17:51:30.152 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:30.152 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:30.152 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:30.152 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 3, 2024-04-07T17:51:30.152 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 7 2024-04-07T17:51:30.152 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:30.152 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:30.152 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:30.152 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:30.152 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:30.152 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:30.152 INFO:tasks.workunit.client.0.smithi100.stdout: 3 2024-04-07T17:51:30.152 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:30.153 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 0, 2024-04-07T17:51:30.153 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "61'2" 2024-04-07T17:51:30.153 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:30.153 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:30.153 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:30.153 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:30.153 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:30.153 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T17:51:30.153 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:30.153 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 7, 2024-04-07T17:51:30.153 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:30.153 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:30.153 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:30.153 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:30.153 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:30.153 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:30.153 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.372482+0000", 2024-04-07T17:51:30.154 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:38.987197+0000", 2024-04-07T17:51:30.154 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:38.987197+0000", 2024-04-07T17:51:30.154 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:38.999803+0000", 2024-04-07T17:51:30.154 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.372482+0000", 2024-04-07T17:51:30.154 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:30.154 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:30.154 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:30.154 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:30.154 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.372482+0000", 2024-04-07T17:51:30.154 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.372482+0000", 2024-04-07T17:51:30.154 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.372482+0000", 2024-04-07T17:51:30.154 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:30.154 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:30.154 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:30.154 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.372482+0000", 2024-04-07T17:51:30.155 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.372482+0000", 2024-04-07T17:51:30.155 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:30.155 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:30.155 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:30.155 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:30.155 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:30.155 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:30.155 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:30.155 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:30.155 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:30.155 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:30.155 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:30.155 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:30.155 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:30.155 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.15", 2024-04-07T17:51:30.155 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:30.156 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:30.156 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:30.156 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4809, 2024-04-07T17:51:30.156 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:30.156 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T03:29:56.940037+0000", 2024-04-07T17:51:30.156 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:30.156 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:30.156 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:30.156 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:30.156 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:30.156 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:30.156 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:30.156 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:30.156 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:30.156 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:30.157 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:30.157 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:30.157 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:30.157 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:30.157 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:30.157 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:30.157 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:30.157 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:30.157 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:30.157 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:30.157 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:30.157 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:30.157 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:30.157 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:30.157 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:30.157 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:30.157 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:30.158 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:30.158 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:30.158 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:30.158 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:30.158 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:30.158 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:30.158 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:30.158 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:30.158 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:30.158 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:30.158 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:30.158 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:30.158 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:30.158 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:30.158 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:30.158 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:30.159 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:30.159 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:30.159 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:30.159 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:30.159 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:30.159 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:30.159 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T17:51:30.159 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:30.159 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 7, 2024-04-07T17:51:30.159 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:30.159 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:30.159 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:30.159 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:30.159 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:30.159 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:30.159 INFO:tasks.workunit.client.0.smithi100.stdout: 5 2024-04-07T17:51:30.159 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:30.160 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 6, 2024-04-07T17:51:30.160 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:30.160 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:30.160 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T17:51:30.160 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:30.160 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:30.160 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:30.160 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.370811+0000", 2024-04-07T17:51:30.160 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:08.975701+0000", 2024-04-07T17:51:30.160 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:08.975701+0000", 2024-04-07T17:51:30.160 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:08.977082+0000", 2024-04-07T17:51:30.160 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.370811+0000", 2024-04-07T17:51:30.160 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:30.160 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:30.160 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:30.160 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 554, 2024-04-07T17:51:30.161 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.370811+0000", 2024-04-07T17:51:30.161 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.370811+0000", 2024-04-07T17:51:30.161 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.370811+0000", 2024-04-07T17:51:30.161 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:30.161 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:30.161 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:30.161 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.370811+0000", 2024-04-07T17:51:30.161 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.370811+0000", 2024-04-07T17:51:30.161 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:30.161 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 29, 2024-04-07T17:51:30.161 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:30.161 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:30.161 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 553, 2024-04-07T17:51:30.161 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:30.161 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:30.161 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:30.162 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:30.162 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 29, 2024-04-07T17:51:30.162 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:30.162 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:30.162 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:30.162 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.14", 2024-04-07T17:51:30.162 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:30.162 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:30.162 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:30.162 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4865, 2024-04-07T17:51:30.162 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:30.162 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T20:48:03.443222+0000", 2024-04-07T17:51:30.162 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:30.162 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:30.163 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:30.163 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 5824, 2024-04-07T17:51:30.163 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:30.163 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:30.163 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:30.163 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:30.163 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:30.163 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:30.163 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:30.163 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:30.163 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:30.163 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:30.163 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:30.163 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:30.163 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:30.163 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:30.163 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:30.164 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 12, 2024-04-07T17:51:30.164 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 4, 2024-04-07T17:51:30.164 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:30.164 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 4, 2024-04-07T17:51:30.164 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:30.164 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:30.164 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:30.164 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:30.164 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:30.164 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 1, 2024-04-07T17:51:30.164 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:30.164 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:30.164 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:30.164 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:30.164 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:30.164 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:30.165 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:30.165 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 10, 2024-04-07T17:51:30.165 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 14, 2024-04-07T17:51:30.165 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:30.165 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:30.165 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:30.165 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 28, 2024-04-07T17:51:30.165 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 25 2024-04-07T17:51:30.165 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:30.165 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:30.165 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:30.165 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:30.165 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:30.165 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:30.165 INFO:tasks.workunit.client.0.smithi100.stdout: 5 2024-04-07T17:51:30.166 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:30.166 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 6, 2024-04-07T17:51:30.166 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "623'29" 2024-04-07T17:51:30.166 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:30.166 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:30.166 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:30.166 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:30.166 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:30.166 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T17:51:30.166 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:30.166 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 4, 2024-04-07T17:51:30.166 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:30.166 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:30.166 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:30.167 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:30.167 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:30.167 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:30.167 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:35.981022+0000", 2024-04-07T17:51:30.167 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:38.985283+0000", 2024-04-07T17:51:30.167 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:38.985283+0000", 2024-04-07T17:51:30.167 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:38.986153+0000", 2024-04-07T17:51:30.167 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:35.981022+0000", 2024-04-07T17:51:30.167 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:30.167 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:30.167 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:30.167 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:30.167 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:35.981022+0000", 2024-04-07T17:51:30.167 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:35.981022+0000", 2024-04-07T17:51:30.167 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:35.981022+0000", 2024-04-07T17:51:30.167 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:30.168 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:30.168 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:30.168 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:35.981022+0000", 2024-04-07T17:51:30.168 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:35.981022+0000", 2024-04-07T17:51:30.168 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:30.168 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:30.168 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:30.168 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:30.168 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:30.168 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:30.168 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:30.168 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:30.168 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:30.168 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:30.168 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:30.168 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:30.169 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:30.169 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.14", 2024-04-07T17:51:30.169 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:30.169 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:30.169 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:30.169 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4907, 2024-04-07T17:51:30.169 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:30.169 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T22:16:21.117705+0000", 2024-04-07T17:51:30.169 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:30.169 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:30.169 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:30.169 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:30.169 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:30.169 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:30.169 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:30.169 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:30.170 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:30.170 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:30.170 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:30.170 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:30.170 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:30.170 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:30.170 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:30.170 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:30.170 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:30.170 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:30.170 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:30.170 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:30.170 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:30.170 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:30.171 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:30.171 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:30.171 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:30.171 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:30.171 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:30.171 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:30.171 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:30.171 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:30.171 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:30.171 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:30.171 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:30.171 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:30.171 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:30.171 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:30.171 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:30.171 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:30.172 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:30.172 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:30.172 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:30.172 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:30.172 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:30.172 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:30.172 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:30.172 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:30.172 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:30.172 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:30.172 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:30.172 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T17:51:30.172 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:30.172 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 4, 2024-04-07T17:51:30.172 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:30.172 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:30.173 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:30.173 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:30.173 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:30.173 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:30.173 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T17:51:30.173 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:30.173 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 1, 2024-04-07T17:51:30.173 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:30.173 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:30.173 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T17:51:30.173 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:30.173 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:30.173 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:30.173 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.332631+0000", 2024-04-07T17:51:30.173 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:24:41.527553+0000", 2024-04-07T17:51:30.173 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:24:41.527553+0000", 2024-04-07T17:51:30.173 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:24:41.530429+0000", 2024-04-07T17:51:30.174 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.332631+0000", 2024-04-07T17:51:30.174 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:30.174 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:30.174 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:30.174 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 527, 2024-04-07T17:51:30.174 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.332631+0000", 2024-04-07T17:51:30.174 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.332631+0000", 2024-04-07T17:51:30.174 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.332631+0000", 2024-04-07T17:51:30.174 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:30.174 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:30.174 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:30.174 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.332631+0000", 2024-04-07T17:51:30.174 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.332631+0000", 2024-04-07T17:51:30.174 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:30.175 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 3, 2024-04-07T17:51:30.175 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:30.175 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:30.175 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 526, 2024-04-07T17:51:30.175 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:30.175 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:30.175 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:30.175 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:30.175 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 3, 2024-04-07T17:51:30.175 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:30.175 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:30.175 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:30.175 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.15", 2024-04-07T17:51:30.175 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:30.175 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:30.176 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:30.176 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4990, 2024-04-07T17:51:30.176 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:30.176 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T16:34:47.703047+0000", 2024-04-07T17:51:30.176 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:30.176 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:30.176 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:30.176 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 90, 2024-04-07T17:51:30.176 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:30.176 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:30.176 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:30.176 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:30.176 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:30.176 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:30.176 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:30.177 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:30.177 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:30.177 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:30.177 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:30.177 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:30.177 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:30.177 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:30.177 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:30.177 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 6, 2024-04-07T17:51:30.177 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 2, 2024-04-07T17:51:30.177 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:30.177 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 2, 2024-04-07T17:51:30.177 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:30.177 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:30.178 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:30.178 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:30.178 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:30.178 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 1, 2024-04-07T17:51:30.178 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:30.178 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:30.178 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:30.178 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:30.178 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:30.178 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:30.178 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:30.178 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 7, 2024-04-07T17:51:30.178 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 7, 2024-04-07T17:51:30.178 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:30.179 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:30.179 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:30.179 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 4, 2024-04-07T17:51:30.179 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 1 2024-04-07T17:51:30.179 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:30.179 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:30.179 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:30.179 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:30.179 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:30.179 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:30.179 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T17:51:30.179 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:30.179 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 1, 2024-04-07T17:51:30.179 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "61'3" 2024-04-07T17:51:30.179 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:30.179 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:30.179 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:30.180 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T17:51:30.180 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:30.180 INFO:tasks.workunit.client.0.smithi100.stdout: 3 2024-04-07T17:51:30.180 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:30.180 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 2, 2024-04-07T17:51:30.180 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:30.180 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:30.180 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:30.180 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:30.180 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:30.180 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:30.180 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.363660+0000", 2024-04-07T17:51:30.180 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:24:41.558300+0000", 2024-04-07T17:51:30.180 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:24:41.558300+0000", 2024-04-07T17:51:30.180 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:24:41.558648+0000", 2024-04-07T17:51:30.181 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.363660+0000", 2024-04-07T17:51:30.181 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:30.181 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:30.181 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:30.181 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 527, 2024-04-07T17:51:30.181 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.363660+0000", 2024-04-07T17:51:30.181 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.363660+0000", 2024-04-07T17:51:30.181 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.363660+0000", 2024-04-07T17:51:30.181 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:30.181 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:30.181 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:30.181 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.363660+0000", 2024-04-07T17:51:30.181 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.363660+0000", 2024-04-07T17:51:30.181 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:30.181 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:30.181 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:30.182 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:30.182 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 526, 2024-04-07T17:51:30.182 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:30.182 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:30.182 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:30.182 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:30.182 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:30.182 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:30.182 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:30.182 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:30.182 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.17", 2024-04-07T17:51:30.182 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:30.182 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:30.182 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:30.183 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4923, 2024-04-07T17:51:30.183 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:30.183 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T01:44:12.981988+0000", 2024-04-07T17:51:30.183 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:30.183 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:30.183 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:30.183 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:30.183 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:30.183 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:30.183 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:30.183 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:30.183 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:30.183 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:30.183 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:30.183 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:30.183 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:30.183 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:30.184 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:30.184 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:30.184 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:30.184 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:30.184 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:30.184 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:30.184 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:30.184 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:30.184 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:30.184 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:30.184 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:30.184 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:30.184 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:30.184 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:30.184 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:30.184 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:30.185 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:30.185 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:30.185 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:30.185 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:30.185 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:30.185 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:30.185 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:30.185 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:30.185 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:30.185 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:30.185 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:30.185 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:30.185 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:30.185 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:30.185 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:30.186 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:30.186 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:30.186 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T17:51:30.186 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:30.186 INFO:tasks.workunit.client.0.smithi100.stdout: 3 2024-04-07T17:51:30.186 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:30.186 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 2, 2024-04-07T17:51:30.186 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:30.186 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:30.186 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:30.186 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:30.186 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:30.186 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:30.186 INFO:tasks.workunit.client.0.smithi100.stdout: 2 2024-04-07T17:51:30.187 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:30.187 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 5, 2024-04-07T17:51:30.187 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:30.187 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:30.187 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T17:51:30.187 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:30.187 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:30.187 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:30.187 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.368702+0000", 2024-04-07T17:51:30.187 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:08.971659+0000", 2024-04-07T17:51:30.187 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:08.971659+0000", 2024-04-07T17:51:30.187 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:08.976930+0000", 2024-04-07T17:51:30.187 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.368702+0000", 2024-04-07T17:51:30.187 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:30.187 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:30.187 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:30.188 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 554, 2024-04-07T17:51:30.188 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.368702+0000", 2024-04-07T17:51:30.188 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.368702+0000", 2024-04-07T17:51:30.188 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.368702+0000", 2024-04-07T17:51:30.188 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:30.188 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:30.188 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:30.188 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.368702+0000", 2024-04-07T17:51:30.188 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.368702+0000", 2024-04-07T17:51:30.188 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:30.188 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:30.188 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:30.188 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:30.188 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 553, 2024-04-07T17:51:30.188 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:30.188 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:30.188 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:30.189 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:30.189 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:30.189 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:30.189 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:30.189 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:30.189 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.16", 2024-04-07T17:51:30.189 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:30.189 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:30.189 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:30.189 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4848, 2024-04-07T17:51:30.189 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:30.189 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T02:13:26.493523+0000", 2024-04-07T17:51:30.189 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:30.189 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:30.189 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:30.190 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:30.190 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:30.190 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:30.190 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:30.190 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:30.190 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:30.190 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:30.190 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:30.190 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:30.190 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:30.190 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:30.190 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:30.190 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:30.190 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:30.191 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:30.191 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:30.191 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:30.191 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:30.191 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:30.191 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:30.191 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:30.191 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:30.191 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:30.191 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:30.191 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:30.191 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:30.191 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:30.191 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:30.191 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:30.192 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:30.192 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:30.192 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:30.192 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:30.192 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:30.192 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:30.192 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:30.192 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:30.192 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:30.192 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:30.192 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:30.192 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:30.192 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:30.192 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:30.192 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:30.192 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:30.193 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:30.193 INFO:tasks.workunit.client.0.smithi100.stdout: 2 2024-04-07T17:51:30.193 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:30.193 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 5, 2024-04-07T17:51:30.193 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:30.193 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:30.193 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:30.193 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:30.193 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:30.193 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:30.193 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T17:51:30.193 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:30.193 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 5, 2024-04-07T17:51:30.193 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:30.193 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:30.193 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:30.194 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:30.194 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:30.194 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:30.194 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.369127+0000", 2024-04-07T17:51:30.194 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:38.977859+0000", 2024-04-07T17:51:30.194 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:38.977859+0000", 2024-04-07T17:51:30.194 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:38.978447+0000", 2024-04-07T17:51:30.194 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.369127+0000", 2024-04-07T17:51:30.194 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:30.194 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:30.194 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:30.194 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:30.194 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.369127+0000", 2024-04-07T17:51:30.194 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.369127+0000", 2024-04-07T17:51:30.194 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.369127+0000", 2024-04-07T17:51:30.195 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:30.195 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:30.195 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:30.195 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.369127+0000", 2024-04-07T17:51:30.195 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.369127+0000", 2024-04-07T17:51:30.195 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:30.195 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:30.195 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:30.195 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:30.195 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:30.195 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:30.195 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:30.195 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:30.195 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:30.195 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:30.196 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:30.196 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:30.196 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:30.196 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.16", 2024-04-07T17:51:30.196 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:30.196 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:30.196 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:30.196 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4860, 2024-04-07T17:51:30.196 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:30.196 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T16:22:51.136501+0000", 2024-04-07T17:51:30.196 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:30.196 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:30.196 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:30.196 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:30.196 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:30.197 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:30.197 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:30.197 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:30.197 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:30.197 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:30.197 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:30.197 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:30.197 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:30.197 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:30.197 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:30.197 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:30.197 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:30.197 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:30.197 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:30.197 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:30.197 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:30.198 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:30.198 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:30.198 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:30.198 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:30.198 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:30.198 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:30.198 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:30.198 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:30.198 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:30.198 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:30.198 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:30.198 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:30.198 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:30.198 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:30.198 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:30.199 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:30.199 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:30.199 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:30.199 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:30.199 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:30.199 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:30.199 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:30.199 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:30.199 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:30.199 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:30.199 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:30.199 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:30.199 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:30.199 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T17:51:30.199 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:30.199 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 5, 2024-04-07T17:51:30.200 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:30.200 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:30.200 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:30.200 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:30.200 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:30.200 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:30.200 INFO:tasks.workunit.client.0.smithi100.stdout: 2 2024-04-07T17:51:30.200 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:30.200 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 6, 2024-04-07T17:51:30.200 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:30.200 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:30.200 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T17:51:30.200 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:30.200 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:30.200 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:30.200 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.370115+0000", 2024-04-07T17:51:30.201 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:08.977704+0000", 2024-04-07T17:51:30.201 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:08.977704+0000", 2024-04-07T17:51:30.201 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:08.978095+0000", 2024-04-07T17:51:30.201 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.370115+0000", 2024-04-07T17:51:30.201 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:30.201 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:30.201 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:30.201 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 554, 2024-04-07T17:51:30.201 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.370115+0000", 2024-04-07T17:51:30.201 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.370115+0000", 2024-04-07T17:51:30.201 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.370115+0000", 2024-04-07T17:51:30.201 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:30.201 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:30.201 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:30.201 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.370115+0000", 2024-04-07T17:51:30.201 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.370115+0000", 2024-04-07T17:51:30.201 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:30.202 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:30.202 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:30.202 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:30.202 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 553, 2024-04-07T17:51:30.202 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:30.202 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:30.202 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:30.202 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:30.202 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:30.202 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:30.202 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:30.202 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:30.202 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.17", 2024-04-07T17:51:30.202 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:30.202 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:30.203 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:30.203 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4829, 2024-04-07T17:51:30.203 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:30.203 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T18:47:28.916356+0000", 2024-04-07T17:51:30.203 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:30.203 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:30.203 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:30.203 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:30.203 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:30.203 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:30.203 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:30.203 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:30.203 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:30.203 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:30.203 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:30.204 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:30.204 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:30.204 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:30.204 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:30.204 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:30.204 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:30.204 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:30.204 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:30.204 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:30.204 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:30.204 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:30.204 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:30.204 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:30.204 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:30.204 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:30.204 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:30.205 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:30.205 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:30.205 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:30.205 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:30.205 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:30.205 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:30.205 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:30.205 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:30.205 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:30.205 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:30.205 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:30.205 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:30.205 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:30.205 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:30.205 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:30.206 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:30.206 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:30.206 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:30.206 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:30.206 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:30.206 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:30.206 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:30.206 INFO:tasks.workunit.client.0.smithi100.stdout: 2 2024-04-07T17:51:30.206 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:30.206 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 6, 2024-04-07T17:51:30.206 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:30.206 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:30.206 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:30.206 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:30.207 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:30.207 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:30.207 INFO:tasks.workunit.client.0.smithi100.stdout: 4 2024-04-07T17:51:30.207 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:30.207 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 1, 2024-04-07T17:51:30.207 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:30.207 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:30.207 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:30.207 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:30.207 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:30.207 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:30.207 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.331404+0000", 2024-04-07T17:51:30.207 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:24:13.932694+0000", 2024-04-07T17:51:30.207 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:24:13.932694+0000", 2024-04-07T17:51:30.207 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:24:13.933594+0000", 2024-04-07T17:51:30.208 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.331404+0000", 2024-04-07T17:51:30.208 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:30.208 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:30.208 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:30.208 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 500, 2024-04-07T17:51:30.208 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.331404+0000", 2024-04-07T17:51:30.208 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.331404+0000", 2024-04-07T17:51:30.208 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.331404+0000", 2024-04-07T17:51:30.208 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:30.208 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:30.208 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:30.208 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.331404+0000", 2024-04-07T17:51:30.208 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.331404+0000", 2024-04-07T17:51:30.208 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:30.208 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:30.208 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:30.208 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:30.209 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 499, 2024-04-07T17:51:30.209 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:30.209 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:30.209 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:30.209 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:30.209 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:30.209 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:30.209 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:30.209 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:30.209 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.19", 2024-04-07T17:51:30.209 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:30.209 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:30.209 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:30.209 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4977, 2024-04-07T17:51:30.209 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:30.210 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T17:08:30.909773+0000", 2024-04-07T17:51:30.210 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:30.210 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:30.210 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:30.210 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:30.210 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:30.210 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:30.210 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:30.210 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:30.210 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:30.210 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:30.210 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:30.210 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:30.210 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:30.210 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:30.210 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:30.211 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:30.211 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:30.211 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:30.211 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:30.211 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:30.211 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:30.211 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:30.211 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:30.211 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:30.211 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:30.211 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:30.211 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:30.211 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:30.211 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:30.211 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:30.212 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:30.212 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:30.212 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:30.212 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:30.212 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:30.212 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:30.212 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:30.212 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:30.212 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:30.212 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:30.212 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:30.212 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:30.212 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:30.212 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:30.212 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:30.212 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:30.213 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:30.213 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:30.213 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:30.213 INFO:tasks.workunit.client.0.smithi100.stdout: 4 2024-04-07T17:51:30.213 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:30.213 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 1, 2024-04-07T17:51:30.213 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:30.213 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:30.213 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:30.213 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:30.213 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:30.213 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:30.213 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T17:51:30.213 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:30.214 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 5, 2024-04-07T17:51:30.214 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:30.214 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:30.214 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T17:51:30.214 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:30.214 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:30.214 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:30.214 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.361600+0000", 2024-04-07T17:51:30.214 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:38.990134+0000", 2024-04-07T17:51:30.214 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:38.990134+0000", 2024-04-07T17:51:30.214 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:38.990885+0000", 2024-04-07T17:51:30.214 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.361600+0000", 2024-04-07T17:51:30.214 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:30.214 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:30.214 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:30.214 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:30.215 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.361600+0000", 2024-04-07T17:51:30.215 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.361600+0000", 2024-04-07T17:51:30.215 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.361600+0000", 2024-04-07T17:51:30.215 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:30.215 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:30.215 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:30.215 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.361600+0000", 2024-04-07T17:51:30.215 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.361600+0000", 2024-04-07T17:51:30.215 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:30.215 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:30.215 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:30.215 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:30.215 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:30.215 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:30.215 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:30.215 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:30.216 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:30.216 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:30.216 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:30.216 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:30.216 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:30.216 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.18", 2024-04-07T17:51:30.216 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:30.216 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:30.216 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:30.216 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4861, 2024-04-07T17:51:30.216 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:30.216 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T19:33:03.967565+0000", 2024-04-07T17:51:30.216 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:30.216 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:30.216 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:30.217 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:30.217 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:30.217 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:30.217 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:30.217 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:30.217 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:30.217 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:30.217 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:30.217 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:30.217 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:30.217 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:30.217 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:30.217 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:30.217 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:30.217 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:30.217 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:30.218 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:30.218 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:30.218 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:30.218 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:30.218 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:30.218 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:30.218 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:30.218 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:30.218 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:30.218 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:30.218 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:30.218 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:30.218 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:30.218 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:30.218 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:30.218 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:30.219 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:30.219 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:30.219 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:30.219 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:30.219 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:30.219 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:30.219 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:30.219 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:30.219 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:30.219 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:30.219 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:30.219 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:30.219 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:30.219 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:30.219 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T17:51:30.219 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:30.220 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 5, 2024-04-07T17:51:30.220 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:30.220 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:30.220 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:30.220 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:30.220 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:30.220 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:30.220 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T17:51:30.220 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:30.220 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 3, 2024-04-07T17:51:30.220 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:30.220 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:30.220 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:30.220 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:30.220 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:30.220 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:30.221 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:35.975334+0000", 2024-04-07T17:51:30.221 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:23:43.903341+0000", 2024-04-07T17:51:30.221 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:23:43.903341+0000", 2024-04-07T17:51:30.221 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:23:43.904215+0000", 2024-04-07T17:51:30.221 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:35.975334+0000", 2024-04-07T17:51:30.221 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:30.221 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:30.221 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:30.221 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 471, 2024-04-07T17:51:30.221 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:35.975334+0000", 2024-04-07T17:51:30.221 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:35.975334+0000", 2024-04-07T17:51:30.221 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:35.975334+0000", 2024-04-07T17:51:30.221 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:30.221 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:30.221 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:30.222 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:35.975334+0000", 2024-04-07T17:51:30.222 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:35.975334+0000", 2024-04-07T17:51:30.222 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:30.222 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:30.222 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:30.222 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:30.222 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 470, 2024-04-07T17:51:30.222 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:30.222 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:30.222 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:30.222 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:30.222 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:30.222 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:30.222 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:30.222 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:30.223 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.18", 2024-04-07T17:51:30.223 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:30.223 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:30.223 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:30.223 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4969, 2024-04-07T17:51:30.223 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:30.223 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T23:54:23.660433+0000", 2024-04-07T17:51:30.223 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:30.223 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:30.223 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:30.223 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:30.223 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:30.223 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:30.223 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:30.223 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:30.224 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:30.224 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:30.224 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:30.224 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:30.224 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:30.224 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:30.224 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:30.224 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:30.224 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:30.224 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:30.224 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:30.224 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:30.224 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:30.224 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:30.224 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:30.224 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:30.225 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:30.225 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:30.225 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:30.225 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:30.225 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:30.225 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:30.225 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:30.225 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:30.225 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:30.225 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:30.225 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:30.225 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:30.225 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:30.225 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:30.225 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:30.225 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:30.226 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:30.226 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:30.226 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:30.226 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:30.226 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:30.226 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:30.226 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:30.226 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:30.226 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:30.226 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T17:51:30.226 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:30.226 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 3, 2024-04-07T17:51:30.226 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:30.226 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:30.227 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:30.227 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:30.227 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:30.227 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:30.227 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T17:51:30.227 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:30.227 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 3, 2024-04-07T17:51:30.227 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:30.227 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:30.227 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T17:51:30.227 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:30.227 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:30.227 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:30.227 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:35.977675+0000", 2024-04-07T17:51:30.227 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:08.976003+0000", 2024-04-07T17:51:30.227 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:08.976003+0000", 2024-04-07T17:51:30.228 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:08.976353+0000", 2024-04-07T17:51:30.228 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:35.977675+0000", 2024-04-07T17:51:30.228 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:30.228 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:30.228 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:30.228 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 554, 2024-04-07T17:51:30.228 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:35.977675+0000", 2024-04-07T17:51:30.228 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:35.977675+0000", 2024-04-07T17:51:30.228 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:35.977675+0000", 2024-04-07T17:51:30.228 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:30.228 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:30.228 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:30.228 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:35.977675+0000", 2024-04-07T17:51:30.228 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:35.977675+0000", 2024-04-07T17:51:30.228 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:30.228 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:30.229 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:30.229 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:30.229 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 553, 2024-04-07T17:51:30.229 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:30.229 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:30.229 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:30.229 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:30.229 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:30.229 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:30.229 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:30.229 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:30.229 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.19", 2024-04-07T17:51:30.229 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:30.229 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:30.229 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:30.230 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4937, 2024-04-07T17:51:30.230 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:30.230 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T01:23:38.086617+0000", 2024-04-07T17:51:30.230 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:30.230 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:30.230 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:30.230 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:30.230 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:30.230 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:30.230 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:30.230 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:30.230 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:30.230 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:30.230 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:30.231 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:30.231 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:30.231 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:30.231 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:30.231 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:30.231 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:30.231 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:30.231 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:30.231 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:30.231 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:30.231 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:30.231 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:30.231 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:30.231 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:30.231 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:30.232 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:30.232 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:30.232 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:30.232 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:30.232 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:30.232 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:30.232 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:30.232 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:30.232 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:30.232 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:30.232 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:30.232 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:30.232 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:30.232 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:30.232 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:30.232 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:30.233 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:30.233 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:30.233 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:30.233 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:30.233 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:30.233 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:30.233 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:30.233 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T17:51:30.233 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:30.233 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 3, 2024-04-07T17:51:30.233 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:30.233 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:30.233 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:30.233 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:30.233 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:30.234 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:30.234 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T17:51:30.234 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:30.234 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 0, 2024-04-07T17:51:30.234 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:30.234 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:30.234 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:30.234 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:30.234 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:30.234 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:30.234 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:35.988644+0000", 2024-04-07T17:51:30.234 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:39.004366+0000", 2024-04-07T17:51:30.234 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:39.004366+0000", 2024-04-07T17:51:30.234 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:39.004810+0000", 2024-04-07T17:51:30.234 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:35.988644+0000", 2024-04-07T17:51:30.235 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:30.235 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:30.235 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:30.235 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:30.235 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:35.988644+0000", 2024-04-07T17:51:30.235 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:35.988644+0000", 2024-04-07T17:51:30.235 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:35.988644+0000", 2024-04-07T17:51:30.235 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:30.235 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:30.235 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:30.235 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:35.988644+0000", 2024-04-07T17:51:30.235 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:35.988644+0000", 2024-04-07T17:51:30.235 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:30.235 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:30.235 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:30.236 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:30.236 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:30.236 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:30.236 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:30.236 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:30.236 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:30.236 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:30.236 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:30.236 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:30.236 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:30.236 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.1b", 2024-04-07T17:51:30.236 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:30.236 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:30.236 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:30.237 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4992, 2024-04-07T17:51:30.237 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:30.237 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T19:31:04.334052+0000", 2024-04-07T17:51:30.237 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:30.237 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:30.237 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:30.237 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:30.237 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:30.237 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:30.237 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:30.237 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:30.237 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:30.237 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:30.237 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:30.237 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:30.237 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:30.238 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:30.238 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:30.238 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:30.238 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:30.238 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:30.238 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:30.238 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:30.238 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:30.238 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:30.238 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:30.238 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:30.238 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:30.238 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:30.238 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:30.238 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:30.238 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:30.239 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:30.239 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:30.239 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:30.239 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:30.239 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:30.239 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:30.239 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:30.239 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:30.239 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:30.239 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:30.239 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:30.239 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:30.239 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:30.239 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:30.239 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:30.240 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:30.240 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:30.240 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:30.240 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:30.240 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:30.240 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T17:51:30.240 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:30.240 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 0, 2024-04-07T17:51:30.240 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:30.240 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:30.240 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:30.240 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:30.240 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:30.240 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:30.240 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T17:51:30.240 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:30.240 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 6, 2024-04-07T17:51:30.241 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:30.241 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:30.241 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T17:51:30.241 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:30.241 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:30.241 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:30.241 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.372918+0000", 2024-04-07T17:51:30.241 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:38.960251+0000", 2024-04-07T17:51:30.241 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:38.960251+0000", 2024-04-07T17:51:30.241 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:38.960896+0000", 2024-04-07T17:51:30.241 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.372918+0000", 2024-04-07T17:51:30.241 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:30.241 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:30.241 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:30.241 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:30.242 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.372918+0000", 2024-04-07T17:51:30.242 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.372918+0000", 2024-04-07T17:51:30.242 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.372918+0000", 2024-04-07T17:51:30.242 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:30.242 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:30.242 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:30.242 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.372918+0000", 2024-04-07T17:51:30.242 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.372918+0000", 2024-04-07T17:51:30.242 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:30.242 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 2, 2024-04-07T17:51:30.242 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:30.242 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:30.242 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:30.242 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:30.242 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:30.243 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:30.243 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:30.243 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 2, 2024-04-07T17:51:30.243 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:30.243 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:30.243 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:30.243 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.1a", 2024-04-07T17:51:30.243 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:30.243 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:30.243 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:30.243 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4839, 2024-04-07T17:51:30.243 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:30.243 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T18:24:54.929614+0000", 2024-04-07T17:51:30.243 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:30.243 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:30.243 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:30.244 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:30.244 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:30.244 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:30.244 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:30.244 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:30.244 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:30.244 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:30.244 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:30.244 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:30.244 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:30.244 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:30.244 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:30.244 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:30.244 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:30.245 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:30.245 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:30.245 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:30.245 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:30.245 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:30.245 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:30.245 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:30.245 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:30.245 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:30.245 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:30.245 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:30.245 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:30.245 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:30.245 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:30.245 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:30.245 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:30.246 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:30.246 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:30.246 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:30.246 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:30.246 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:30.246 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:30.246 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:30.246 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:30.246 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:30.246 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:30.246 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:30.246 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:30.246 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:30.246 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:30.247 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:30.247 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:30.247 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T17:51:30.247 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:30.247 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 6, 2024-04-07T17:51:30.247 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "623'2" 2024-04-07T17:51:30.247 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:30.247 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:30.247 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:30.247 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:30.247 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:30.247 INFO:tasks.workunit.client.0.smithi100.stdout: 2 2024-04-07T17:51:30.247 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:30.247 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 4, 2024-04-07T17:51:30.247 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:30.248 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:30.248 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:30.248 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:30.248 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:30.248 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:30.248 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:35.981965+0000", 2024-04-07T17:51:30.248 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:24:13.936610+0000", 2024-04-07T17:51:30.248 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:24:13.936610+0000", 2024-04-07T17:51:30.248 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:24:13.937948+0000", 2024-04-07T17:51:30.248 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:35.981965+0000", 2024-04-07T17:51:30.248 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:30.248 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:30.248 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:30.248 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 500, 2024-04-07T17:51:30.248 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:35.981965+0000", 2024-04-07T17:51:30.249 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:35.981965+0000", 2024-04-07T17:51:30.249 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:35.981965+0000", 2024-04-07T17:51:30.249 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:30.249 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:30.249 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:30.249 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:35.981965+0000", 2024-04-07T17:51:30.249 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:35.981965+0000", 2024-04-07T17:51:30.249 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:30.249 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:30.249 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:30.249 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:30.249 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 499, 2024-04-07T17:51:30.249 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:30.249 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:30.249 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:30.249 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:30.249 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:30.250 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:30.250 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:30.250 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:30.250 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.1a", 2024-04-07T17:51:30.250 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:30.250 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:30.250 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:30.250 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4930, 2024-04-07T17:51:30.250 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:30.250 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T02:25:15.032887+0000", 2024-04-07T17:51:30.250 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:30.250 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:30.250 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:30.250 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:30.250 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:30.251 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:30.251 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:30.251 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:30.251 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:30.251 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:30.251 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:30.251 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:30.251 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:30.251 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:30.251 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:30.251 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:30.251 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:30.251 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:30.251 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:30.251 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:30.251 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:30.252 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:30.252 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:30.252 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:30.252 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:30.252 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:30.252 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:30.252 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:30.252 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:30.252 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:30.252 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:30.252 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:30.252 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:30.252 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:30.252 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:30.252 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:30.252 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:30.253 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:30.253 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:30.253 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:30.253 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:30.253 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:30.253 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:30.253 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:30.253 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:30.253 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:30.253 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:30.253 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:30.253 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:30.253 INFO:tasks.workunit.client.0.smithi100.stdout: 2 2024-04-07T17:51:30.253 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:30.253 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 4, 2024-04-07T17:51:30.253 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:30.254 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:30.254 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:30.254 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:30.254 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:30.254 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:30.254 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T17:51:30.254 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:30.254 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 3, 2024-04-07T17:51:30.254 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:30.254 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:30.254 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T17:51:30.254 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:30.254 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:30.254 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:30.254 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:35.977526+0000", 2024-04-07T17:51:30.254 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:38.997910+0000", 2024-04-07T17:51:30.255 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:38.997910+0000", 2024-04-07T17:51:30.255 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:38.998434+0000", 2024-04-07T17:51:30.255 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:35.977526+0000", 2024-04-07T17:51:30.255 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:30.255 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:30.255 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:30.255 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:30.255 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:35.977526+0000", 2024-04-07T17:51:30.255 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:35.977526+0000", 2024-04-07T17:51:30.255 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:35.977526+0000", 2024-04-07T17:51:30.255 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:30.255 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:30.255 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:30.255 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:35.977526+0000", 2024-04-07T17:51:30.255 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:35.977526+0000", 2024-04-07T17:51:30.256 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:30.256 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 2, 2024-04-07T17:51:30.256 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:30.256 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:30.256 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:30.256 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:30.256 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:30.256 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:30.256 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:30.256 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 2, 2024-04-07T17:51:30.256 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:30.256 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:30.256 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:30.256 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.1b", 2024-04-07T17:51:30.256 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:30.256 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:30.257 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:30.257 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4932, 2024-04-07T17:51:30.257 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:30.257 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T02:18:46.594583+0000", 2024-04-07T17:51:30.257 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:30.257 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:30.257 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:30.257 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:30.257 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:30.257 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:30.257 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:30.257 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:30.257 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:30.257 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:30.257 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:30.258 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:30.258 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:30.258 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:30.258 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:30.258 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:30.258 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:30.258 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:30.258 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:30.258 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:30.258 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:30.258 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:30.258 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:30.258 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:30.258 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:30.258 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:30.259 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:30.259 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:30.259 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:30.259 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:30.259 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:30.259 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:30.259 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:30.259 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:30.259 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:30.259 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:30.259 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:30.259 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:30.259 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:30.259 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:30.259 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:30.260 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:30.260 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:30.260 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:30.260 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:30.260 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:30.260 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:30.260 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:30.260 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:30.260 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T17:51:30.260 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:30.260 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 3, 2024-04-07T17:51:30.260 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "623'2" 2024-04-07T17:51:30.260 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:30.260 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:30.260 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:30.260 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:30.261 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:30.261 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T17:51:30.261 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:30.261 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 5, 2024-04-07T17:51:30.261 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:30.261 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:30.261 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:30.261 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:30.261 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:30.261 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:30.261 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.365302+0000", 2024-04-07T17:51:30.261 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:08.974827+0000", 2024-04-07T17:51:30.261 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:08.974827+0000", 2024-04-07T17:51:30.261 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:08.975369+0000", 2024-04-07T17:51:30.261 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.365302+0000", 2024-04-07T17:51:30.262 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:30.262 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:30.262 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:30.262 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 554, 2024-04-07T17:51:30.262 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.365302+0000", 2024-04-07T17:51:30.262 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.365302+0000", 2024-04-07T17:51:30.262 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.365302+0000", 2024-04-07T17:51:30.262 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:30.262 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:30.262 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:30.262 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.365302+0000", 2024-04-07T17:51:30.262 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.365302+0000", 2024-04-07T17:51:30.262 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:30.262 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:30.262 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:30.263 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:30.263 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 553, 2024-04-07T17:51:30.263 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:30.263 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:30.263 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:30.263 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:30.263 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:30.263 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:30.263 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:30.263 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:30.263 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.1d", 2024-04-07T17:51:30.263 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:30.263 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:30.263 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:30.263 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4868, 2024-04-07T17:51:30.265 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:30.265 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T19:16:10.778146+0000", 2024-04-07T17:51:30.265 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:30.265 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:30.265 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:30.265 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:30.265 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:30.265 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:30.265 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:30.265 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:30.265 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:30.265 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:30.266 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:30.266 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:30.266 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:30.266 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:30.266 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:30.266 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:30.266 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:30.266 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:30.266 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:30.266 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:30.266 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:30.266 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:30.266 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:30.266 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:30.266 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:30.267 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:30.267 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:30.267 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:30.267 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:30.267 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:30.267 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:30.267 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:30.267 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:30.267 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:30.267 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:30.267 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:30.267 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:30.267 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:30.267 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:30.267 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:30.267 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:30.268 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:30.268 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:30.268 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:30.268 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:30.268 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:30.268 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:30.268 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:30.268 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:30.268 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T17:51:30.268 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:30.268 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 5, 2024-04-07T17:51:30.268 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:30.268 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:30.268 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:30.268 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:30.268 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:30.269 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:30.269 INFO:tasks.workunit.client.0.smithi100.stdout: 2 2024-04-07T17:51:30.269 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:30.269 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 7, 2024-04-07T17:51:30.269 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:30.269 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:30.269 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T17:51:30.269 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:30.269 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:30.269 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:30.269 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.364942+0000", 2024-04-07T17:51:30.269 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:38.975920+0000", 2024-04-07T17:51:30.269 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:38.975920+0000", 2024-04-07T17:51:30.269 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:38.991127+0000", 2024-04-07T17:51:30.269 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.364942+0000", 2024-04-07T17:51:30.270 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:30.270 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:30.270 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:30.270 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:30.270 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.364942+0000", 2024-04-07T17:51:30.270 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.364942+0000", 2024-04-07T17:51:30.270 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.364942+0000", 2024-04-07T17:51:30.270 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:30.270 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:30.270 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:30.270 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.364942+0000", 2024-04-07T17:51:30.270 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.364942+0000", 2024-04-07T17:51:30.270 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:30.270 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 2, 2024-04-07T17:51:30.270 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:30.271 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:30.271 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:30.271 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:30.271 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:30.271 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:30.271 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:30.271 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 2, 2024-04-07T17:51:30.271 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:30.271 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:30.271 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:30.271 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.1c", 2024-04-07T17:51:30.271 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:30.271 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:30.271 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:30.272 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4819, 2024-04-07T17:51:30.272 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:30.272 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T22:48:18.986526+0000", 2024-04-07T17:51:30.272 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:30.272 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:30.272 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:30.272 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:30.272 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:30.272 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:30.272 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:30.272 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:30.272 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:30.272 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:30.272 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:30.273 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:30.273 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:30.273 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:30.273 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:30.273 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:30.273 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:30.273 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:30.273 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:30.273 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 3, 2024-04-07T17:51:30.273 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 1, 2024-04-07T17:51:30.273 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:30.273 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 1, 2024-04-07T17:51:30.273 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:30.273 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:30.273 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:30.273 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:30.274 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:30.274 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 1, 2024-04-07T17:51:30.274 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:30.274 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:30.274 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:30.274 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:30.274 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:30.274 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:30.274 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:30.274 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 6, 2024-04-07T17:51:30.274 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 6, 2024-04-07T17:51:30.274 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:30.274 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:30.274 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:30.275 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 3, 2024-04-07T17:51:30.275 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:30.275 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:30.275 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:30.275 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:30.275 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:30.275 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:30.275 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:30.275 INFO:tasks.workunit.client.0.smithi100.stdout: 2 2024-04-07T17:51:30.275 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:30.275 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 7, 2024-04-07T17:51:30.275 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "61'2" 2024-04-07T17:51:30.275 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:30.275 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:30.275 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:30.276 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:30.276 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:30.276 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T17:51:30.276 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:30.276 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 5, 2024-04-07T17:51:30.276 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:30.276 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:30.276 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:30.276 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:30.276 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:30.276 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:30.276 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.367080+0000", 2024-04-07T17:51:30.276 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:24:41.542687+0000", 2024-04-07T17:51:30.276 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:24:41.542687+0000", 2024-04-07T17:51:30.276 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:24:41.543356+0000", 2024-04-07T17:51:30.277 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.367080+0000", 2024-04-07T17:51:30.277 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:30.277 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:30.277 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:30.277 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 527, 2024-04-07T17:51:30.277 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.367080+0000", 2024-04-07T17:51:30.277 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.367080+0000", 2024-04-07T17:51:30.277 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.367080+0000", 2024-04-07T17:51:30.277 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:30.277 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:30.277 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:30.277 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.367080+0000", 2024-04-07T17:51:30.277 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.367080+0000", 2024-04-07T17:51:30.277 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:30.277 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:30.278 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:30.278 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:30.278 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 526, 2024-04-07T17:51:30.278 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:30.278 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:30.278 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:30.278 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:30.278 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:30.278 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:30.278 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:30.278 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:30.278 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.1c", 2024-04-07T17:51:30.278 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:30.278 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:30.278 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:30.279 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4865, 2024-04-07T17:51:30.279 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:30.279 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T21:43:40.340691+0000", 2024-04-07T17:51:30.279 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:30.279 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:30.279 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:30.279 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:30.279 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:30.279 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:30.279 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:30.279 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:30.279 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:30.279 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:30.279 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:30.279 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:30.280 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:30.280 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:30.280 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:30.280 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:30.280 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:30.280 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:30.280 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:30.280 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:30.280 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:30.280 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:30.280 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:30.280 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:30.280 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:30.280 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:30.280 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:30.280 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:30.281 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:30.281 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:30.281 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:30.281 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:30.281 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:30.281 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:30.281 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:30.281 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:30.281 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:30.281 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:30.281 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:30.281 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:30.281 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:30.281 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:30.281 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:30.281 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:30.282 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:30.282 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:30.282 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:30.282 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:30.282 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:30.282 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T17:51:30.282 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:30.282 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 5, 2024-04-07T17:51:30.282 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:30.282 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:30.282 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:30.282 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:30.282 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:30.282 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:30.282 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T17:51:30.283 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:30.283 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 7, 2024-04-07T17:51:30.283 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:30.283 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:30.283 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T17:51:30.283 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:30.283 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:30.283 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:30.283 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.373509+0000", 2024-04-07T17:51:30.283 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:39.001898+0000", 2024-04-07T17:51:30.283 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:39.001898+0000", 2024-04-07T17:51:30.283 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:39.004508+0000", 2024-04-07T17:51:30.283 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.373509+0000", 2024-04-07T17:51:30.283 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:30.283 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:30.283 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:30.284 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:30.284 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.373509+0000", 2024-04-07T17:51:30.284 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.373509+0000", 2024-04-07T17:51:30.284 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.373509+0000", 2024-04-07T17:51:30.284 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:30.284 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:30.284 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:30.284 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.373509+0000", 2024-04-07T17:51:30.284 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.373509+0000", 2024-04-07T17:51:30.284 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:30.284 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 2, 2024-04-07T17:51:30.284 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:30.284 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:30.284 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:30.284 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:30.284 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:30.284 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:30.285 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:30.285 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 2, 2024-04-07T17:51:30.285 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:30.285 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:30.285 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:30.285 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.1d", 2024-04-07T17:51:30.285 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:30.285 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:30.285 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:30.285 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4804, 2024-04-07T17:51:30.285 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:30.285 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T01:47:40.701645+0000", 2024-04-07T17:51:30.285 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:30.285 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:30.285 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:30.286 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:30.286 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:30.286 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:30.286 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:30.286 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:30.286 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:30.286 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:30.286 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:30.286 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:30.286 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:30.286 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:30.286 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:30.286 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:30.287 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:30.287 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:30.287 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:30.287 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 3, 2024-04-07T17:51:30.287 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 1, 2024-04-07T17:51:30.287 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:30.287 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 1, 2024-04-07T17:51:30.287 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:30.287 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:30.287 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:30.287 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:30.287 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:30.287 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 1, 2024-04-07T17:51:30.287 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:30.287 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:30.288 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:30.288 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:30.288 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:30.288 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:30.288 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:30.288 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 6, 2024-04-07T17:51:30.288 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 6, 2024-04-07T17:51:30.288 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:30.288 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:30.288 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:30.288 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 3, 2024-04-07T17:51:30.288 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:30.288 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:30.288 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:30.288 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:30.288 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:30.288 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:30.289 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:30.289 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T17:51:30.289 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:30.289 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 7, 2024-04-07T17:51:30.289 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "61'2" 2024-04-07T17:51:30.289 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:30.289 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:30.289 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:30.289 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:30.289 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:30.289 INFO:tasks.workunit.client.0.smithi100.stdout: 2 2024-04-07T17:51:30.289 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:30.289 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 0, 2024-04-07T17:51:30.289 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:30.289 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:30.290 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:30.290 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:30.290 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:30.290 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:30.290 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:35.983257+0000", 2024-04-07T17:51:30.290 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:24:41.564700+0000", 2024-04-07T17:51:30.290 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:24:41.564700+0000", 2024-04-07T17:51:30.290 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:24:41.565093+0000", 2024-04-07T17:51:30.290 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:35.983257+0000", 2024-04-07T17:51:30.290 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:30.290 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:30.290 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:30.290 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 527, 2024-04-07T17:51:30.290 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:35.983257+0000", 2024-04-07T17:51:30.290 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:35.983257+0000", 2024-04-07T17:51:30.291 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:35.983257+0000", 2024-04-07T17:51:30.291 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:30.291 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:30.291 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:30.291 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:35.983257+0000", 2024-04-07T17:51:30.291 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:35.983257+0000", 2024-04-07T17:51:30.291 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:30.291 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:30.291 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:30.291 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:30.291 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 526, 2024-04-07T17:51:30.291 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:30.291 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:30.291 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:30.291 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:30.292 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:30.292 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:30.292 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:30.292 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:30.292 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.1f", 2024-04-07T17:51:30.292 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:30.292 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:30.292 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:30.292 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 5000, 2024-04-07T17:51:30.292 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:30.292 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T01:20:29.103128+0000", 2024-04-07T17:51:30.292 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:30.292 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:30.292 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:30.292 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:30.292 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:30.293 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:30.293 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:30.293 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:30.293 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:30.293 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:30.293 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:30.293 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:30.293 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:30.293 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:30.293 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:30.293 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:30.293 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:30.293 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:30.293 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:30.293 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:30.294 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:30.294 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:30.294 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:30.294 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:30.294 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:30.294 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:30.294 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:30.294 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:30.294 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:30.294 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:30.294 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:30.294 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:30.294 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:30.294 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:30.294 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:30.295 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:30.295 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:30.295 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:30.295 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:30.295 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:30.295 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:30.295 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:30.295 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:30.295 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:30.295 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:30.295 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:30.295 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:30.295 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:30.295 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:30.295 INFO:tasks.workunit.client.0.smithi100.stdout: 2 2024-04-07T17:51:30.295 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:30.296 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 0, 2024-04-07T17:51:30.296 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:30.296 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:30.296 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:30.296 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:30.296 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:30.296 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:30.296 INFO:tasks.workunit.client.0.smithi100.stdout: 5 2024-04-07T17:51:30.296 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:30.296 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 3, 2024-04-07T17:51:30.296 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:30.296 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:30.296 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T17:51:30.296 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:30.296 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:30.297 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:30.297 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:35.982794+0000", 2024-04-07T17:51:30.297 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:24:41.546310+0000", 2024-04-07T17:51:30.297 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:24:41.546310+0000", 2024-04-07T17:51:30.297 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:24:41.546952+0000", 2024-04-07T17:51:30.297 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:35.982794+0000", 2024-04-07T17:51:30.297 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:30.297 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:30.297 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:30.297 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 527, 2024-04-07T17:51:30.297 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:35.982794+0000", 2024-04-07T17:51:30.297 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:35.982794+0000", 2024-04-07T17:51:30.297 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:35.982794+0000", 2024-04-07T17:51:30.297 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:30.297 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:30.297 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:30.298 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:35.982794+0000", 2024-04-07T17:51:30.298 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:35.982794+0000", 2024-04-07T17:51:30.298 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:30.298 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 2, 2024-04-07T17:51:30.298 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:30.298 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:30.298 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 526, 2024-04-07T17:51:30.298 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:30.298 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:30.298 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:30.298 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:30.298 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 2, 2024-04-07T17:51:30.298 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:30.298 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:30.298 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:30.299 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.1e", 2024-04-07T17:51:30.299 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:30.299 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:30.299 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:30.299 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4947, 2024-04-07T17:51:30.299 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:30.299 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T21:45:55.182027+0000", 2024-04-07T17:51:30.299 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:30.299 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:30.299 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:30.299 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:30.299 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:30.299 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:30.299 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:30.299 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:30.299 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:30.300 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:30.300 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:30.300 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:30.300 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:30.300 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:30.300 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:30.300 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:30.300 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:30.300 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:30.300 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:30.300 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:30.300 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:30.300 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:30.300 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:30.300 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:30.301 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:30.301 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:30.301 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:30.301 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:30.301 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:30.301 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:30.301 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:30.301 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:30.301 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:30.301 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:30.301 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:30.301 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:30.301 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:30.301 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:30.301 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:30.302 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:30.302 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:30.302 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:30.302 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:30.302 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:30.302 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:30.302 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:30.302 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:30.302 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:30.302 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:30.302 INFO:tasks.workunit.client.0.smithi100.stdout: 5 2024-04-07T17:51:30.302 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:30.302 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 3, 2024-04-07T17:51:30.302 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "623'2" 2024-04-07T17:51:30.303 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:30.303 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:30.303 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:30.303 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:30.303 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:30.303 INFO:tasks.workunit.client.0.smithi100.stdout: 2 2024-04-07T17:51:30.303 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:30.303 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 3, 2024-04-07T17:51:30.303 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:30.303 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:30.303 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:30.303 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:30.303 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:30.303 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:30.303 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:35.982633+0000", 2024-04-07T17:51:30.303 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:08.977988+0000", 2024-04-07T17:51:30.304 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:08.977988+0000", 2024-04-07T17:51:30.304 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:08.978435+0000", 2024-04-07T17:51:30.304 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:35.982633+0000", 2024-04-07T17:51:30.304 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:30.304 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:30.304 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:30.304 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 554, 2024-04-07T17:51:30.304 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:35.982633+0000", 2024-04-07T17:51:30.304 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:35.982633+0000", 2024-04-07T17:51:30.304 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:35.982633+0000", 2024-04-07T17:51:30.304 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:30.304 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:30.304 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:30.304 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:35.982633+0000", 2024-04-07T17:51:30.304 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:35.982633+0000", 2024-04-07T17:51:30.305 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:30.305 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:30.305 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:30.305 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:30.305 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 553, 2024-04-07T17:51:30.305 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:30.305 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:30.305 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:30.305 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:30.305 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:30.305 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:30.305 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:30.305 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:30.305 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.1e", 2024-04-07T17:51:30.305 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:30.305 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:30.306 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:30.306 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4957, 2024-04-07T17:51:30.306 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:30.306 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T21:46:59.340658+0000", 2024-04-07T17:51:30.306 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:30.306 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:30.306 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:30.306 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:30.306 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:30.306 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:30.306 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:30.306 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:30.306 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:30.306 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:30.306 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:30.307 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:30.307 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:30.307 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:30.307 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:30.307 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:30.307 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:30.307 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:30.307 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:30.307 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:30.307 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:30.307 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:30.307 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:30.307 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:30.307 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:30.307 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:30.308 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:30.308 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:30.308 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:30.308 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:30.308 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:30.308 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:30.308 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:30.308 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:30.308 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:30.308 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:30.308 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:30.308 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:30.308 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:30.308 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:30.308 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:30.309 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:30.309 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:30.309 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:30.309 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:30.309 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:30.309 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:30.309 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:30.309 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:30.309 INFO:tasks.workunit.client.0.smithi100.stdout: 2 2024-04-07T17:51:30.309 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:30.309 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 3, 2024-04-07T17:51:30.309 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:30.309 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:30.309 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:30.309 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:30.309 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:30.310 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:30.310 INFO:tasks.workunit.client.0.smithi100.stdout: 4 2024-04-07T17:51:30.310 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:30.310 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 0, 2024-04-07T17:51:30.310 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:30.310 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:30.310 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 59, 2024-04-07T17:51:30.310 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:30.310 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:30.310 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:30.310 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:35.984435+0000", 2024-04-07T17:51:30.310 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:38.997317+0000", 2024-04-07T17:51:30.310 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:38.997317+0000", 2024-04-07T17:51:30.310 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:38.997712+0000", 2024-04-07T17:51:30.310 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:35.984435+0000", 2024-04-07T17:51:30.311 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:30.311 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:30.311 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:30.311 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:30.311 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:35.984435+0000", 2024-04-07T17:51:30.311 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:35.984435+0000", 2024-04-07T17:51:30.311 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:35.984435+0000", 2024-04-07T17:51:30.311 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:30.311 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:30.311 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:11.994648+0000", 2024-04-07T17:51:30.311 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:35.984435+0000", 2024-04-07T17:51:30.311 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:35.984435+0000", 2024-04-07T17:51:30.311 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:30.311 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 5, 2024-04-07T17:51:30.311 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:30.312 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:30.312 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:30.312 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:30.312 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:30.312 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:30.312 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:30.312 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 5, 2024-04-07T17:51:30.312 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:30.312 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:30.312 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:30.312 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "2.1f", 2024-04-07T17:51:30.312 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:30.312 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:30.312 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:30.312 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 5008, 2024-04-07T17:51:30.312 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:30.313 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T18:23:18.184011+0000", 2024-04-07T17:51:30.313 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:30.313 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:30.313 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:30.313 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 582, 2024-04-07T17:51:30.313 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:30.313 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:30.313 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:30.313 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:30.313 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:30.313 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:30.313 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:30.313 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:30.313 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:30.313 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:30.314 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:30.314 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:30.314 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:30.314 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:30.314 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:30.314 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 6, 2024-04-07T17:51:30.314 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 2, 2024-04-07T17:51:30.314 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:30.314 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 2, 2024-04-07T17:51:30.314 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:30.314 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:30.314 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:30.314 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:30.314 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:30.315 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 1, 2024-04-07T17:51:30.315 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:30.315 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:30.315 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:30.315 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:30.315 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:30.315 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:30.315 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:30.315 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 4, 2024-04-07T17:51:30.315 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 4, 2024-04-07T17:51:30.315 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:30.315 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:30.315 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:30.315 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 4, 2024-04-07T17:51:30.315 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 1 2024-04-07T17:51:30.315 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:30.316 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:30.316 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:30.316 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:30.316 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:30.316 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:30.316 INFO:tasks.workunit.client.0.smithi100.stdout: 4 2024-04-07T17:51:30.316 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:30.316 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 0, 2024-04-07T17:51:30.316 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "623'5" 2024-04-07T17:51:30.316 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:30.316 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:30.316 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:30.316 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:30.316 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T17:51:30.316 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T17:51:30.317 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:30.317 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 7, 2024-04-07T17:51:30.317 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:30.317 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:30.317 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:30.317 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:30.317 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:30.317 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:30.317 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.373284+0000", 2024-04-07T17:51:30.317 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:38.996053+0000", 2024-04-07T17:51:30.317 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:38.996053+0000", 2024-04-07T17:51:30.317 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:39.002704+0000", 2024-04-07T17:51:30.317 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.373284+0000", 2024-04-07T17:51:30.317 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:30.317 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:30.317 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:30.318 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:30.318 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.373284+0000", 2024-04-07T17:51:30.318 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.373284+0000", 2024-04-07T17:51:30.318 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.373284+0000", 2024-04-07T17:51:30.318 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:30.318 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:30.318 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:30.318 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.373284+0000", 2024-04-07T17:51:30.318 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.373284+0000", 2024-04-07T17:51:30.318 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:30.318 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:30.318 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:30.318 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:30.318 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:30.319 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:30.319 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:30.319 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:30.319 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:30.319 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:30.319 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:30.319 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:30.319 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:30.319 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.2d", 2024-04-07T17:51:30.319 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:30.319 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:30.319 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:30.319 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4802, 2024-04-07T17:51:30.319 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:30.319 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T17:45:19.934847+0000", 2024-04-07T17:51:30.320 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:30.320 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:30.320 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:30.320 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:30.320 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:30.320 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:30.320 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:30.320 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:30.320 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:30.320 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:30.320 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:30.320 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:30.320 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:30.320 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:30.320 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:30.320 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:30.321 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:30.321 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:30.321 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:30.321 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:30.321 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:30.321 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:30.321 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:30.321 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:30.321 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:30.321 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:30.321 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:30.321 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:30.321 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:30.321 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:30.321 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:30.322 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:30.322 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:30.322 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:30.322 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:30.322 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:30.322 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:30.322 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:30.322 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:30.322 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:30.322 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:30.322 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:30.322 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:30.322 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:30.322 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:30.323 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:30.323 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:30.323 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:30.323 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T17:51:30.323 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T17:51:30.323 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:30.323 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 7, 2024-04-07T17:51:30.323 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:30.323 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:30.323 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:30.323 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:30.323 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T17:51:30.323 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:30.323 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T17:51:30.323 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:30.324 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 2, 2024-04-07T17:51:30.324 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:30.324 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:30.324 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:30.324 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:30.324 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:30.324 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:30.324 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.359468+0000", 2024-04-07T17:51:30.324 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:38.967369+0000", 2024-04-07T17:51:30.324 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:38.967369+0000", 2024-04-07T17:51:30.324 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:38.967737+0000", 2024-04-07T17:51:30.324 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.359468+0000", 2024-04-07T17:51:30.324 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:30.324 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:30.324 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:30.324 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:30.325 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.359468+0000", 2024-04-07T17:51:30.325 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.359468+0000", 2024-04-07T17:51:30.325 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.359468+0000", 2024-04-07T17:51:30.325 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:30.325 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:30.325 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:30.325 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.359468+0000", 2024-04-07T17:51:30.325 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.359468+0000", 2024-04-07T17:51:30.325 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:30.325 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:30.325 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:30.325 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:30.325 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:30.325 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:30.325 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:30.326 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:30.326 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:30.326 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:30.326 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:30.326 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:30.326 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:30.326 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.2e", 2024-04-07T17:51:30.326 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:30.326 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:30.326 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:30.326 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4964, 2024-04-07T17:51:30.326 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:30.326 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T22:15:36.602436+0000", 2024-04-07T17:51:30.326 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:30.326 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:30.327 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:30.327 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:30.327 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:30.327 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:30.327 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:30.327 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:30.327 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:30.327 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:30.327 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:30.327 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:30.327 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:30.327 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:30.327 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:30.327 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:30.327 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:30.328 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:30.328 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:30.328 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:30.328 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:30.328 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:30.328 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:30.328 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:30.328 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:30.328 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:30.328 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:30.328 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:30.328 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:30.328 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:30.328 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:30.328 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:30.329 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:30.329 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:30.329 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:30.329 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:30.329 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:30.329 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:30.329 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:30.329 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:30.329 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:30.329 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:30.329 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:30.329 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:30.329 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:30.329 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:30.329 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:30.330 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T17:51:30.330 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:30.330 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T17:51:30.330 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:30.330 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 2, 2024-04-07T17:51:30.330 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:30.330 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:30.330 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:30.330 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:30.330 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:30.330 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:30.330 INFO:tasks.workunit.client.0.smithi100.stdout: 4 2024-04-07T17:51:30.330 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:30.330 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 5, 2024-04-07T17:51:30.331 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:30.331 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:30.331 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:30.331 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:30.331 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:30.331 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:30.331 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.361813+0000", 2024-04-07T17:51:30.331 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:38.971115+0000", 2024-04-07T17:51:30.331 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:38.971115+0000", 2024-04-07T17:51:30.331 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:38.971858+0000", 2024-04-07T17:51:30.331 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.361813+0000", 2024-04-07T17:51:30.331 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:30.331 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:30.331 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:30.331 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:30.332 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.361813+0000", 2024-04-07T17:51:30.332 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.361813+0000", 2024-04-07T17:51:30.332 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.361813+0000", 2024-04-07T17:51:30.332 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:30.332 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:30.332 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:30.332 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.361813+0000", 2024-04-07T17:51:30.332 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.361813+0000", 2024-04-07T17:51:30.332 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:30.332 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:30.332 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:30.332 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:30.332 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:30.332 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:30.332 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:30.333 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:30.333 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:30.333 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:30.333 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:30.333 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:30.333 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:30.333 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.2f", 2024-04-07T17:51:30.333 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:30.333 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:30.333 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:30.333 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4863, 2024-04-07T17:51:30.333 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:30.333 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T22:01:10.276598+0000", 2024-04-07T17:51:30.333 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:30.333 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:30.334 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:30.334 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:30.334 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:30.334 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:30.334 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:30.334 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:30.334 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:30.334 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:30.334 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:30.334 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:30.334 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:30.334 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:30.334 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:30.334 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:30.335 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:30.335 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:30.335 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:30.335 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:30.335 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:30.335 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:30.335 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:30.335 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:30.335 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:30.335 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:30.335 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:30.335 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:30.335 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:30.335 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:30.335 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:30.336 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:30.336 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:30.336 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:30.336 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:30.336 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:30.336 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:30.336 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:30.336 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:30.336 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:30.336 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:30.336 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:30.336 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:30.336 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:30.336 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:30.336 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:30.337 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:30.337 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:30.337 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:30.337 INFO:tasks.workunit.client.0.smithi100.stdout: 4 2024-04-07T17:51:30.337 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:30.337 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 5, 2024-04-07T17:51:30.337 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:30.337 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:30.337 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:30.337 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:30.337 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:30.337 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:30.337 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T17:51:30.337 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:30.337 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 5, 2024-04-07T17:51:30.338 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:30.338 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:30.338 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:30.338 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:30.338 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:30.338 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:30.338 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.364596+0000", 2024-04-07T17:51:30.338 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:39.005541+0000", 2024-04-07T17:51:30.338 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:39.005541+0000", 2024-04-07T17:51:30.338 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:39.006138+0000", 2024-04-07T17:51:30.338 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.364596+0000", 2024-04-07T17:51:30.338 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:30.338 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:30.338 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:30.338 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:30.338 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.364596+0000", 2024-04-07T17:51:30.339 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.364596+0000", 2024-04-07T17:51:30.339 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.364596+0000", 2024-04-07T17:51:30.339 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:30.339 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:30.339 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:30.339 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.364596+0000", 2024-04-07T17:51:30.339 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.364596+0000", 2024-04-07T17:51:30.339 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:30.339 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:30.339 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:30.339 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:30.339 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:30.339 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:30.339 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:30.339 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:30.340 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:30.340 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:30.340 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:30.340 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:30.340 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:30.340 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.30", 2024-04-07T17:51:30.340 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:30.340 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:30.340 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:30.340 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4876, 2024-04-07T17:51:30.340 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:30.340 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T20:40:58.093262+0000", 2024-04-07T17:51:30.340 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:30.340 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:30.340 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:30.340 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:30.341 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:30.341 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:30.341 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:30.341 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:30.341 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:30.341 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:30.341 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:30.341 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:30.341 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:30.341 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:30.341 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:30.341 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:30.341 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:30.342 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:30.342 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:30.342 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:30.342 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:30.342 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:30.342 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:30.342 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:30.342 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:30.342 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:30.342 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:30.342 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:30.342 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:30.342 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:30.342 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:30.343 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:30.343 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:30.343 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:30.343 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:30.343 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:30.343 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:30.343 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:30.343 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:30.343 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:30.343 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:30.343 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:30.343 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:30.343 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:30.343 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:30.343 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:30.343 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:30.344 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:30.344 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:30.344 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T17:51:30.344 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:30.344 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 5, 2024-04-07T17:51:30.344 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:30.344 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:30.344 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:30.344 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:30.344 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:30.344 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:30.344 INFO:tasks.workunit.client.0.smithi100.stdout: 5 2024-04-07T17:51:30.344 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:30.344 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 1, 2024-04-07T17:51:30.345 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:30.345 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:30.345 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:30.345 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:30.345 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:30.345 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:30.345 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.331261+0000", 2024-04-07T17:51:30.345 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:24:41.533626+0000", 2024-04-07T17:51:30.345 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:24:41.533626+0000", 2024-04-07T17:51:30.345 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:24:41.534090+0000", 2024-04-07T17:51:30.345 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.331261+0000", 2024-04-07T17:51:30.345 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:30.345 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:30.345 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:30.345 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 527, 2024-04-07T17:51:30.345 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.331261+0000", 2024-04-07T17:51:30.346 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.331261+0000", 2024-04-07T17:51:30.346 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.331261+0000", 2024-04-07T17:51:30.346 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:30.346 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:30.346 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:30.346 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.331261+0000", 2024-04-07T17:51:30.346 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.331261+0000", 2024-04-07T17:51:30.346 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:30.346 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:30.346 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:30.346 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:30.346 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 526, 2024-04-07T17:51:30.346 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:30.346 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:30.346 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:30.347 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:30.347 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:30.347 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:30.347 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:30.347 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:30.347 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.31", 2024-04-07T17:51:30.347 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:30.347 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:30.347 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:30.347 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4979, 2024-04-07T17:51:30.347 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:30.347 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T16:42:51.811301+0000", 2024-04-07T17:51:30.347 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:30.347 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:30.347 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:30.348 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:30.348 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:30.348 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:30.348 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:30.348 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:30.348 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:30.348 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:30.348 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:30.348 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:30.348 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:30.348 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:30.348 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:30.348 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:30.348 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:30.348 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:30.349 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:30.349 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:30.349 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:30.349 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:30.349 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:30.349 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:30.349 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:30.349 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:30.349 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:30.349 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:30.349 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:30.349 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:30.349 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:30.349 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:30.349 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:30.350 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:30.350 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:30.350 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:30.350 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:30.350 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:30.350 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:30.350 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:30.350 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:30.350 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:30.350 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:30.350 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:30.350 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:30.350 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:30.350 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:30.350 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:30.351 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:30.351 INFO:tasks.workunit.client.0.smithi100.stdout: 5 2024-04-07T17:51:30.351 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:30.351 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 1, 2024-04-07T17:51:30.351 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:30.351 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:30.351 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:30.351 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:30.351 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:30.351 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:30.351 INFO:tasks.workunit.client.0.smithi100.stdout: 5 2024-04-07T17:51:30.351 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:30.351 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 0, 2024-04-07T17:51:30.351 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:30.351 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:30.352 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:30.352 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:30.352 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:30.352 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:30.352 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:35.984431+0000", 2024-04-07T17:51:30.352 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:24:41.538233+0000", 2024-04-07T17:51:30.352 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:24:41.538233+0000", 2024-04-07T17:51:30.352 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:24:41.538718+0000", 2024-04-07T17:51:30.352 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:35.984431+0000", 2024-04-07T17:51:30.352 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:30.352 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:30.352 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:30.352 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 527, 2024-04-07T17:51:30.352 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:35.984431+0000", 2024-04-07T17:51:30.352 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:35.984431+0000", 2024-04-07T17:51:30.352 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:35.984431+0000", 2024-04-07T17:51:30.353 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:30.353 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:30.353 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:30.353 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:35.984431+0000", 2024-04-07T17:51:30.353 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:35.984431+0000", 2024-04-07T17:51:30.353 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:30.353 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:30.353 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:30.353 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:30.353 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 526, 2024-04-07T17:51:30.353 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:30.353 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:30.353 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:30.353 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:30.353 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:30.353 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:30.354 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:30.354 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:30.354 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.32", 2024-04-07T17:51:30.354 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:30.354 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:30.354 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:30.354 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4995, 2024-04-07T17:51:30.354 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:30.354 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T22:18:55.330487+0000", 2024-04-07T17:51:30.354 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:30.354 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:30.354 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:30.354 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:30.354 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:30.354 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:30.355 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:30.355 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:30.355 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:30.355 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:30.355 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:30.355 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:30.355 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:30.355 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:30.355 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:30.355 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:30.355 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:30.355 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:30.355 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:30.355 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:30.355 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:30.356 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:30.356 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:30.356 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:30.356 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:30.356 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:30.356 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:30.356 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:30.356 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:30.356 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:30.356 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:30.356 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:30.356 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:30.356 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:30.356 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:30.356 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:30.357 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:30.357 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:30.357 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:30.357 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:30.357 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:30.357 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:30.357 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:30.357 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:30.357 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:30.357 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:30.357 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:30.357 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:30.357 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:30.357 INFO:tasks.workunit.client.0.smithi100.stdout: 5 2024-04-07T17:51:30.357 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:30.357 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 0, 2024-04-07T17:51:30.358 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:30.358 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:30.358 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:30.358 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:30.358 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:30.358 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:30.358 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T17:51:30.358 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:30.358 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 3, 2024-04-07T17:51:30.358 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:30.358 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:30.358 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:30.358 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:30.358 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:30.359 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:30.359 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:35.977234+0000", 2024-04-07T17:51:30.359 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:08.976145+0000", 2024-04-07T17:51:30.359 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:08.976145+0000", 2024-04-07T17:51:30.359 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:08.976499+0000", 2024-04-07T17:51:30.359 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:35.977234+0000", 2024-04-07T17:51:30.359 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:30.359 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:30.359 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:30.359 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 554, 2024-04-07T17:51:30.359 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:35.977234+0000", 2024-04-07T17:51:30.359 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:35.977234+0000", 2024-04-07T17:51:30.359 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:35.977234+0000", 2024-04-07T17:51:30.359 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:30.359 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:30.359 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:30.360 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:35.977234+0000", 2024-04-07T17:51:30.360 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:35.977234+0000", 2024-04-07T17:51:30.360 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:30.360 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:30.360 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:30.360 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:30.360 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 553, 2024-04-07T17:51:30.360 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:30.360 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:30.360 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:30.360 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:30.360 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:30.360 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:30.360 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:30.360 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:30.360 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.33", 2024-04-07T17:51:30.360 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:30.361 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:30.361 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:30.361 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4963, 2024-04-07T17:51:30.361 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:30.361 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T04:02:25.054838+0000", 2024-04-07T17:51:30.361 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:30.361 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:30.361 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:30.361 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:30.361 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:30.361 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:30.361 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:30.361 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:30.361 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:30.361 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:30.361 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:30.362 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:30.362 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:30.362 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:30.362 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:30.362 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:30.362 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:30.362 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:30.362 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:30.362 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:30.362 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:30.362 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:30.362 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:30.362 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:30.362 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:30.363 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:30.363 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:30.363 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:30.363 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:30.363 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:30.363 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:30.363 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:30.363 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:30.363 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:30.363 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:30.363 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:30.363 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:30.363 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:30.363 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:30.363 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:30.364 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:30.364 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:30.364 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:30.364 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:30.364 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:30.364 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:30.364 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:30.364 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:30.364 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:30.364 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T17:51:30.364 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:30.364 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 3, 2024-04-07T17:51:30.364 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:30.364 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:30.364 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:30.364 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:30.365 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:30.365 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:30.365 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T17:51:30.365 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:30.365 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 4, 2024-04-07T17:51:30.365 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:30.365 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:30.365 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:30.365 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:30.365 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:30.365 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:30.365 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:35.980246+0000", 2024-04-07T17:51:30.365 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:08.955654+0000", 2024-04-07T17:51:30.365 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:08.955654+0000", 2024-04-07T17:51:30.365 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:08.956220+0000", 2024-04-07T17:51:30.365 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:35.980246+0000", 2024-04-07T17:51:30.365 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:30.365 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:30.366 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:30.366 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 554, 2024-04-07T17:51:30.366 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:35.980246+0000", 2024-04-07T17:51:30.366 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:35.980246+0000", 2024-04-07T17:51:30.366 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:35.980246+0000", 2024-04-07T17:51:30.366 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:30.366 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:30.366 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:30.366 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:35.980246+0000", 2024-04-07T17:51:30.366 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:35.980246+0000", 2024-04-07T17:51:30.366 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:30.366 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:30.366 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:30.366 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:30.367 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 553, 2024-04-07T17:51:30.367 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:30.367 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:30.367 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:30.367 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:30.367 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:30.367 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:30.367 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:30.367 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:30.367 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.34", 2024-04-07T17:51:30.367 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:30.367 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:30.367 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:30.367 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4913, 2024-04-07T17:51:30.367 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:30.367 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T00:20:14.081274+0000", 2024-04-07T17:51:30.367 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:30.367 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:30.368 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:30.368 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:30.368 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:30.368 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:30.368 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:30.368 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:30.368 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:30.368 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:30.368 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:30.368 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:30.368 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:30.368 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:30.368 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:30.368 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:30.368 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:30.368 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:30.369 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:30.369 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:30.369 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:30.369 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:30.369 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:30.369 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:30.369 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:30.369 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:30.369 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:30.369 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:30.369 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:30.369 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:30.369 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:30.369 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:30.369 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:30.370 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:30.370 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:30.370 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:30.370 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:30.370 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:30.370 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:30.370 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:30.370 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:30.370 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:30.370 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:30.370 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:30.370 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:30.370 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:30.371 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:30.371 INFO:tasks.workunit.client.0.smithi100.stdout: 4, 2024-04-07T17:51:30.371 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:30.371 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T17:51:30.371 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:30.371 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 4, 2024-04-07T17:51:30.371 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:30.371 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:30.371 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:30.371 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:30.371 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:30.371 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:30.371 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T17:51:30.371 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:30.371 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 6, 2024-04-07T17:51:30.372 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:30.372 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:30.372 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:30.372 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:30.372 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:30.372 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:30.372 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.359086+0000", 2024-04-07T17:51:30.372 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:08.978750+0000", 2024-04-07T17:51:30.372 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:08.978750+0000", 2024-04-07T17:51:30.372 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:08.979235+0000", 2024-04-07T17:51:30.372 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.359086+0000", 2024-04-07T17:51:30.372 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:30.372 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:30.372 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:30.372 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 554, 2024-04-07T17:51:30.372 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.359086+0000", 2024-04-07T17:51:30.373 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.359086+0000", 2024-04-07T17:51:30.373 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.359086+0000", 2024-04-07T17:51:30.373 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:30.373 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:30.373 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:30.373 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.359086+0000", 2024-04-07T17:51:30.373 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.359086+0000", 2024-04-07T17:51:30.373 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:30.373 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:30.373 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:30.373 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:30.373 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 553, 2024-04-07T17:51:30.373 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:30.373 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:30.373 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:30.374 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:30.374 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:30.374 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:30.374 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:30.374 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:30.374 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.35", 2024-04-07T17:51:30.374 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:30.374 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:30.374 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:30.374 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4828, 2024-04-07T17:51:30.374 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:30.374 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T03:27:45.790517+0000", 2024-04-07T17:51:30.374 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:30.374 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:30.375 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:30.375 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:30.375 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:30.375 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:30.375 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:30.375 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:30.375 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:30.375 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:30.375 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:30.375 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:30.375 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:30.375 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:30.375 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:30.375 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:30.375 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:30.376 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:30.376 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:30.376 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:30.376 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:30.376 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:30.376 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:30.376 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:30.376 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:30.376 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:30.376 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:30.376 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:30.376 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:30.376 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:30.376 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:30.376 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:30.377 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:30.377 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:30.377 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:30.377 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:30.377 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:30.377 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:30.377 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:30.377 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:30.377 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:30.377 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:30.377 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:30.377 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:30.377 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:30.377 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:30.377 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:30.377 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:30.378 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:30.378 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T17:51:30.378 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:30.378 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 6, 2024-04-07T17:51:30.378 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:30.378 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:30.378 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:30.378 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:30.378 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:30.378 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:30.378 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T17:51:30.378 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:30.378 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 6, 2024-04-07T17:51:30.378 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:30.379 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:30.379 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:30.379 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:30.379 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:30.379 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:30.379 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.368833+0000", 2024-04-07T17:51:30.379 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:08.977646+0000", 2024-04-07T17:51:30.379 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:08.977646+0000", 2024-04-07T17:51:30.379 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:08.978144+0000", 2024-04-07T17:51:30.379 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.368833+0000", 2024-04-07T17:51:30.379 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:30.379 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:30.379 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:30.379 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 554, 2024-04-07T17:51:30.379 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.368833+0000", 2024-04-07T17:51:30.379 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.368833+0000", 2024-04-07T17:51:30.380 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.368833+0000", 2024-04-07T17:51:30.380 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:30.380 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:30.380 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:30.380 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.368833+0000", 2024-04-07T17:51:30.380 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.368833+0000", 2024-04-07T17:51:30.380 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:30.380 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:30.380 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:30.380 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:30.380 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 553, 2024-04-07T17:51:30.380 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:30.380 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:30.380 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:30.380 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:30.380 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:30.381 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:30.381 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:30.381 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:30.381 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.36", 2024-04-07T17:51:30.381 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:30.381 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:30.381 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:30.381 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4828, 2024-04-07T17:51:30.381 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:30.381 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T01:10:43.702742+0000", 2024-04-07T17:51:30.381 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:30.381 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:30.381 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:30.381 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:30.381 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:30.382 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:30.382 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:30.382 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:30.382 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:30.382 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:30.382 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:30.382 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:30.382 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:30.382 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:30.382 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:30.382 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:30.382 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:30.382 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:30.382 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:30.382 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:30.383 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:30.383 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:30.383 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:30.383 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:30.383 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:30.383 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:30.383 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:30.383 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:30.383 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:30.383 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:30.383 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:30.383 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:30.383 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:30.383 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:30.383 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:30.384 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:30.384 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:30.384 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:30.384 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:30.384 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:30.384 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:30.384 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:30.384 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:30.384 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:30.384 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:30.384 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:30.384 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:30.384 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:30.384 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:30.384 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T17:51:30.385 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:30.385 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 6, 2024-04-07T17:51:30.385 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:30.385 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:30.385 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:30.385 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:30.385 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:30.385 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:30.385 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T17:51:30.385 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:30.385 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 7, 2024-04-07T17:51:30.385 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:30.385 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:30.385 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:30.385 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:30.385 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:30.386 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:30.386 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.366733+0000", 2024-04-07T17:51:30.386 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:38.968877+0000", 2024-04-07T17:51:30.386 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:38.968877+0000", 2024-04-07T17:51:30.386 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:38.976137+0000", 2024-04-07T17:51:30.386 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.366733+0000", 2024-04-07T17:51:30.386 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:30.386 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:30.386 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:30.386 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:30.386 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.366733+0000", 2024-04-07T17:51:30.386 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.366733+0000", 2024-04-07T17:51:30.386 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.366733+0000", 2024-04-07T17:51:30.386 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:30.387 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:30.387 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:30.387 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.366733+0000", 2024-04-07T17:51:30.387 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.366733+0000", 2024-04-07T17:51:30.387 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:30.387 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:30.387 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:30.387 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:30.387 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:30.387 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:30.387 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:30.387 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:30.387 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:30.387 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:30.387 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:30.388 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:30.388 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:30.388 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.37", 2024-04-07T17:51:30.388 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:30.388 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:30.388 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:30.388 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4815, 2024-04-07T17:51:30.388 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:30.388 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T17:38:48.549583+0000", 2024-04-07T17:51:30.388 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:30.388 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:30.388 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:30.388 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:30.388 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:30.388 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:30.389 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:30.389 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:30.389 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:30.389 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:30.389 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:30.389 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:30.389 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:30.389 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:30.389 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:30.389 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:30.389 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:30.389 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:30.389 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:30.389 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:30.390 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:30.390 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:30.390 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:30.390 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:30.390 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:30.390 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:30.390 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:30.390 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:30.390 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:30.390 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:30.390 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:30.390 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:30.390 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:30.390 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:30.390 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:30.391 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:30.391 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:30.391 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:30.391 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:30.391 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:30.391 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:30.391 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:30.391 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:30.391 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:30.391 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:30.391 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:30.391 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:30.391 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:30.391 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:30.391 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T17:51:30.392 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:30.392 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 7, 2024-04-07T17:51:30.392 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:30.392 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:30.392 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:30.392 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:30.392 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:30.392 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:30.392 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T17:51:30.392 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:30.392 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 3, 2024-04-07T17:51:30.392 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:30.392 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:30.392 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:30.392 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:30.393 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:30.393 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:30.393 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:35.975131+0000", 2024-04-07T17:51:30.393 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:08.977725+0000", 2024-04-07T17:51:30.393 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:08.977725+0000", 2024-04-07T17:51:30.393 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:08.978090+0000", 2024-04-07T17:51:30.393 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:35.975131+0000", 2024-04-07T17:51:30.393 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:30.393 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:30.393 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:30.393 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 554, 2024-04-07T17:51:30.393 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:35.975131+0000", 2024-04-07T17:51:30.393 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:35.975131+0000", 2024-04-07T17:51:30.393 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:35.975131+0000", 2024-04-07T17:51:30.393 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:30.393 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:30.394 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:30.394 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:35.975131+0000", 2024-04-07T17:51:30.394 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:35.975131+0000", 2024-04-07T17:51:30.394 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:30.394 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:30.394 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:30.394 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:30.394 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 553, 2024-04-07T17:51:30.394 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:30.394 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:30.394 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:30.394 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:30.394 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:30.394 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:30.394 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:30.395 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:30.395 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.38", 2024-04-07T17:51:30.395 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:30.395 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:30.395 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:30.395 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4929, 2024-04-07T17:51:30.395 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:30.395 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T18:02:53.604912+0000", 2024-04-07T17:51:30.395 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:30.395 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:30.395 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:30.395 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:30.395 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:30.395 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:30.396 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:30.396 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:30.396 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:30.396 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:30.396 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:30.396 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:30.396 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:30.396 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:30.396 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:30.396 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:30.396 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:30.396 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:30.396 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:30.396 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:30.396 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:30.396 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:30.397 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:30.397 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:30.397 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:30.397 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:30.397 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:30.397 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:30.397 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:30.397 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:30.397 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:30.397 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:30.397 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:30.397 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:30.397 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:30.397 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:30.398 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:30.398 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:30.398 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:30.398 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:30.398 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:30.398 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:30.398 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:30.398 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:30.398 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:30.398 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:30.398 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:30.398 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:30.398 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:30.398 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T17:51:30.399 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:30.399 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 3, 2024-04-07T17:51:30.399 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:30.399 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:30.399 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:30.399 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:30.399 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:30.399 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:30.399 INFO:tasks.workunit.client.0.smithi100.stdout: 4 2024-04-07T17:51:30.399 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:30.399 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 7, 2024-04-07T17:51:30.399 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:30.399 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:30.399 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:30.399 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:30.400 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:30.400 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:30.400 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.358816+0000", 2024-04-07T17:51:30.400 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:38.996215+0000", 2024-04-07T17:51:30.400 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:38.996215+0000", 2024-04-07T17:51:30.400 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:39.000120+0000", 2024-04-07T17:51:30.400 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.358816+0000", 2024-04-07T17:51:30.400 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:30.400 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:30.400 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:30.400 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:30.400 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.358816+0000", 2024-04-07T17:51:30.400 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.358816+0000", 2024-04-07T17:51:30.400 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.358816+0000", 2024-04-07T17:51:30.400 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:30.401 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:30.401 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:30.401 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.358816+0000", 2024-04-07T17:51:30.401 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.358816+0000", 2024-04-07T17:51:30.401 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:30.401 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:30.401 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:30.401 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:30.401 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:30.401 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:30.401 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:30.401 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:30.401 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:30.401 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:30.401 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:30.401 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:30.402 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:30.402 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.39", 2024-04-07T17:51:30.402 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:30.402 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:30.402 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:30.402 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4787, 2024-04-07T17:51:30.402 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:30.402 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T22:36:35.618662+0000", 2024-04-07T17:51:30.402 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:30.402 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:30.402 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:30.402 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:30.402 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:30.402 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:30.403 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:30.403 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:30.403 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:30.403 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:30.403 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:30.403 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:30.403 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:30.403 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:30.403 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:30.403 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:30.403 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:30.403 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:30.403 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:30.403 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:30.403 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:30.403 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:30.404 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:30.404 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:30.404 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:30.404 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:30.404 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:30.404 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:30.404 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:30.404 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:30.404 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:30.404 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:30.404 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:30.404 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:30.404 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:30.404 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:30.404 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:30.404 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:30.405 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:30.405 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:30.405 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:30.405 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:30.405 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:30.405 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:30.405 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:30.405 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:30.405 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:30.405 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:30.405 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:30.405 INFO:tasks.workunit.client.0.smithi100.stdout: 4 2024-04-07T17:51:30.405 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:30.405 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 7, 2024-04-07T17:51:30.405 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:30.406 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:30.406 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:30.406 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:30.406 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:30.406 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:30.406 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T17:51:30.406 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:30.406 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 5, 2024-04-07T17:51:30.406 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:30.406 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:30.406 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:30.406 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:30.406 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:30.406 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:30.406 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.368733+0000", 2024-04-07T17:51:30.407 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:24:41.548397+0000", 2024-04-07T17:51:30.407 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:24:41.548397+0000", 2024-04-07T17:51:30.407 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:24:41.551443+0000", 2024-04-07T17:51:30.407 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.368733+0000", 2024-04-07T17:51:30.407 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:30.407 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:30.407 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:30.407 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 527, 2024-04-07T17:51:30.407 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.368733+0000", 2024-04-07T17:51:30.407 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.368733+0000", 2024-04-07T17:51:30.407 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.368733+0000", 2024-04-07T17:51:30.407 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:30.407 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:30.407 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:30.407 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.368733+0000", 2024-04-07T17:51:30.407 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.368733+0000", 2024-04-07T17:51:30.407 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:30.408 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:30.408 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:30.408 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:30.408 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 526, 2024-04-07T17:51:30.408 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:30.408 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:30.408 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:30.408 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:30.408 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:30.408 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:30.408 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:30.408 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:30.408 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.3a", 2024-04-07T17:51:30.408 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:30.408 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:30.409 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:30.409 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4865, 2024-04-07T17:51:30.409 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:30.409 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T20:38:47.135272+0000", 2024-04-07T17:51:30.409 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:30.409 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:30.409 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:30.409 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:30.409 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:30.409 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:30.409 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:30.409 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:30.409 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:30.409 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:30.409 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:30.409 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:30.410 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:30.410 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:30.410 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:30.410 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:30.410 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:30.410 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:30.410 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:30.410 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:30.410 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:30.410 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:30.410 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:30.410 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:30.410 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:30.410 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:30.411 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:30.411 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:30.411 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:30.411 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:30.411 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:30.411 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:30.411 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:30.411 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:30.411 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:30.411 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:30.411 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:30.411 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:30.411 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:30.411 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:30.411 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:30.412 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:30.412 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:30.412 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:30.412 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:30.412 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:30.412 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:30.412 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:30.412 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:30.412 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T17:51:30.412 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:30.412 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 5, 2024-04-07T17:51:30.412 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:30.412 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:30.412 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:30.413 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:30.413 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:30.413 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T17:51:30.413 INFO:tasks.workunit.client.0.smithi100.stdout: 4 2024-04-07T17:51:30.413 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:30.413 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 7, 2024-04-07T17:51:30.413 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:30.413 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:30.413 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:30.413 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:30.413 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:30.413 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:30.413 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.369564+0000", 2024-04-07T17:51:30.413 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:38.985715+0000", 2024-04-07T17:51:30.413 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:38.985715+0000", 2024-04-07T17:51:30.413 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:38.990756+0000", 2024-04-07T17:51:30.414 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.369564+0000", 2024-04-07T17:51:30.414 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:30.414 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:30.414 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:30.414 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:30.414 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.369564+0000", 2024-04-07T17:51:30.414 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.369564+0000", 2024-04-07T17:51:30.414 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.369564+0000", 2024-04-07T17:51:30.414 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:30.414 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:30.414 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:30.414 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.369564+0000", 2024-04-07T17:51:30.414 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.369564+0000", 2024-04-07T17:51:30.414 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:30.414 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:30.414 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:30.415 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:30.415 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:30.415 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:30.415 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:30.415 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:30.415 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:30.415 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:30.415 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:30.415 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:30.415 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:30.415 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.3b", 2024-04-07T17:51:30.415 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:30.415 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:30.415 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:30.416 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4787, 2024-04-07T17:51:30.416 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:30.416 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T17:50:57.375885+0000", 2024-04-07T17:51:30.416 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:30.416 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:30.416 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:30.416 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:30.416 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:30.416 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:30.416 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:30.416 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:30.416 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:30.416 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:30.416 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:30.416 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:30.416 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:30.417 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:30.417 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:30.417 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:30.417 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:30.417 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:30.417 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:30.417 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:30.417 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:30.417 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:30.417 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:30.417 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:30.417 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:30.417 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:30.417 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:30.417 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:30.417 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:30.417 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:30.418 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:30.418 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:30.418 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:30.418 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:30.418 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:30.418 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:30.418 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:30.418 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:30.418 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:30.418 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:30.418 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:30.418 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:30.418 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:30.418 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:30.419 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:30.419 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:30.419 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:30.419 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:30.419 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T17:51:30.419 INFO:tasks.workunit.client.0.smithi100.stdout: 4 2024-04-07T17:51:30.419 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:30.419 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 7, 2024-04-07T17:51:30.419 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:30.419 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:30.419 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:30.419 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:30.419 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:30.419 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:30.419 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T17:51:30.419 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:30.420 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 5, 2024-04-07T17:51:30.420 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:30.420 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:30.420 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:30.420 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:30.420 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:30.420 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:30.420 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.365779+0000", 2024-04-07T17:51:30.420 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:38.986923+0000", 2024-04-07T17:51:30.420 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:38.986923+0000", 2024-04-07T17:51:30.420 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:38.987454+0000", 2024-04-07T17:51:30.420 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.365779+0000", 2024-04-07T17:51:30.420 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:30.420 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:30.420 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:30.420 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:30.421 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.365779+0000", 2024-04-07T17:51:30.421 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.365779+0000", 2024-04-07T17:51:30.421 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.365779+0000", 2024-04-07T17:51:30.421 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:30.421 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:30.421 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:30.421 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.365779+0000", 2024-04-07T17:51:30.421 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.365779+0000", 2024-04-07T17:51:30.421 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:30.421 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:30.421 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:30.421 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:30.421 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:30.421 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:30.421 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:30.422 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:30.422 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:30.422 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:30.422 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:30.422 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:30.422 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:30.422 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.3c", 2024-04-07T17:51:30.422 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:30.422 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:30.422 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:30.422 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4883, 2024-04-07T17:51:30.422 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:30.422 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T02:20:58.654636+0000", 2024-04-07T17:51:30.422 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:30.422 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:30.423 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:30.423 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:30.423 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:30.423 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:30.423 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:30.423 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:30.423 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:30.423 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:30.423 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:30.423 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:30.423 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:30.423 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:30.423 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:30.423 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:30.423 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:30.424 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:30.424 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:30.424 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:30.424 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:30.424 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:30.424 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:30.424 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:30.424 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:30.424 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:30.424 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:30.424 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:30.424 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:30.424 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:30.424 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:30.424 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:30.424 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:30.425 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:30.425 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:30.425 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:30.425 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:30.425 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:30.425 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:30.425 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:30.425 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:30.425 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:30.425 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:30.425 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:30.425 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:30.425 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:30.425 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:30.426 INFO:tasks.workunit.client.0.smithi100.stdout: 5, 2024-04-07T17:51:30.426 INFO:tasks.workunit.client.0.smithi100.stdout: 3, 2024-04-07T17:51:30.426 INFO:tasks.workunit.client.0.smithi100.stdout: 7 2024-04-07T17:51:30.426 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:30.426 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 5, 2024-04-07T17:51:30.426 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:30.426 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:30.426 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:30.426 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:30.426 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:30.426 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T17:51:30.426 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T17:51:30.426 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:30.426 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 0, 2024-04-07T17:51:30.426 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:30.427 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:30.427 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:30.427 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:30.427 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:30.427 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:30.427 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:35.982778+0000", 2024-04-07T17:51:30.427 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:23:15.876439+0000", 2024-04-07T17:51:30.427 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:23:15.876439+0000", 2024-04-07T17:51:30.427 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:23:15.876856+0000", 2024-04-07T17:51:30.427 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:35.982778+0000", 2024-04-07T17:51:30.427 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:30.427 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:30.427 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:30.427 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 445, 2024-04-07T17:51:30.427 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:35.982778+0000", 2024-04-07T17:51:30.428 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:35.982778+0000", 2024-04-07T17:51:30.428 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:35.982778+0000", 2024-04-07T17:51:30.428 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:30.428 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:30.428 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:30.428 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:35.982778+0000", 2024-04-07T17:51:30.428 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:35.982778+0000", 2024-04-07T17:51:30.428 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:30.428 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:30.428 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:30.428 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:30.428 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 444, 2024-04-07T17:51:30.428 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:30.428 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:30.428 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:30.429 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:30.429 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:30.429 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:30.429 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:30.429 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:30.429 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.3d", 2024-04-07T17:51:30.429 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:30.429 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:30.429 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:30.429 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 5000, 2024-04-07T17:51:30.429 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:30.429 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T03:48:36.975141+0000", 2024-04-07T17:51:30.429 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:30.429 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:30.429 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:30.430 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:30.430 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:30.430 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:30.430 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:30.430 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:30.430 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:30.430 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:30.430 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:30.430 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:30.430 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:30.430 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:30.430 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:30.430 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:30.430 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:30.430 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:30.431 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:30.431 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:30.431 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:30.431 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:30.431 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:30.431 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:30.431 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:30.431 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:30.431 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:30.431 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:30.431 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:30.431 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:30.431 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:30.431 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:30.431 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:30.431 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:30.432 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:30.432 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:30.432 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:30.432 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:30.432 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:30.432 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:30.432 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:30.432 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:30.432 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:30.432 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:30.432 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:30.432 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:30.432 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:30.432 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:30.432 INFO:tasks.workunit.client.0.smithi100.stdout: 2, 2024-04-07T17:51:30.433 INFO:tasks.workunit.client.0.smithi100.stdout: 1 2024-04-07T17:51:30.433 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:30.433 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 0, 2024-04-07T17:51:30.433 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:30.433 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:30.433 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:30.433 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:30.433 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:30.433 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:30.433 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T17:51:30.433 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:30.433 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 1, 2024-04-07T17:51:30.433 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:30.433 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:30.433 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:30.433 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:30.434 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:30.434 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:30.434 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.334821+0000", 2024-04-07T17:51:30.434 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:08.962145+0000", 2024-04-07T17:51:30.434 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:08.962145+0000", 2024-04-07T17:51:30.434 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:08.962703+0000", 2024-04-07T17:51:30.434 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.334821+0000", 2024-04-07T17:51:30.434 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:30.434 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:30.434 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:30.434 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 554, 2024-04-07T17:51:30.434 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.334821+0000", 2024-04-07T17:51:30.434 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.334821+0000", 2024-04-07T17:51:30.434 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.334821+0000", 2024-04-07T17:51:30.434 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:30.435 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:30.435 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:30.435 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.334821+0000", 2024-04-07T17:51:30.435 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.334821+0000", 2024-04-07T17:51:30.435 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:30.435 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:30.435 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:30.435 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:30.435 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 553, 2024-04-07T17:51:30.435 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:30.435 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:30.435 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:30.435 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:30.435 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:30.435 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:30.436 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:30.436 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:30.436 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.3e", 2024-04-07T17:51:30.436 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:30.436 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:30.436 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:30.436 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4989, 2024-04-07T17:51:30.436 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:30.436 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-08T23:09:45.491615+0000", 2024-04-07T17:51:30.436 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:30.436 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:30.436 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:30.436 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:30.436 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:30.436 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:30.436 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:30.437 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:30.437 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:30.437 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:30.437 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:30.437 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:30.437 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:30.437 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:30.437 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:30.437 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:30.437 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:30.437 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:30.437 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:30.437 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:30.437 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:30.437 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:30.438 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:30.438 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:30.438 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:30.438 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:30.438 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:30.438 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:30.438 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:30.438 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:30.438 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:30.438 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:30.438 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:30.438 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:30.438 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:30.438 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:30.438 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:30.439 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:30.439 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:30.439 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:30.439 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:30.439 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:30.439 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:30.439 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:30.439 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:30.439 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:30.439 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:30.439 INFO:tasks.workunit.client.0.smithi100.stdout: 1, 2024-04-07T17:51:30.439 INFO:tasks.workunit.client.0.smithi100.stdout: 0, 2024-04-07T17:51:30.439 INFO:tasks.workunit.client.0.smithi100.stdout: 6 2024-04-07T17:51:30.439 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:30.439 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 1, 2024-04-07T17:51:30.440 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:30.440 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:30.440 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:30.440 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": [ 2024-04-07T17:51:30.440 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:30.440 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:30.440 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T17:51:30.440 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:30.440 INFO:tasks.workunit.client.0.smithi100.stdout: "acting_primary": 7, 2024-04-07T17:51:30.440 INFO:tasks.workunit.client.0.smithi100.stdout: "avail_no_missing": [], 2024-04-07T17:51:30.440 INFO:tasks.workunit.client.0.smithi100.stdout: "blocked_by": [], 2024-04-07T17:51:30.440 INFO:tasks.workunit.client.0.smithi100.stdout: "created": 60, 2024-04-07T17:51:30.440 INFO:tasks.workunit.client.0.smithi100.stdout: "dirty_stats_invalid": false, 2024-04-07T17:51:30.440 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_bytes_stats_invalid": false, 2024-04-07T17:51:30.440 INFO:tasks.workunit.client.0.smithi100.stdout: "hitset_stats_invalid": false, 2024-04-07T17:51:30.440 INFO:tasks.workunit.client.0.smithi100.stdout: "last_active": "2024-04-07T17:50:36.367170+0000", 2024-04-07T17:51:30.441 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_active": "2024-04-07T16:25:38.966876+0000", 2024-04-07T17:51:30.441 INFO:tasks.workunit.client.0.smithi100.stdout: "last_became_peered": "2024-04-07T16:25:38.966876+0000", 2024-04-07T17:51:30.441 INFO:tasks.workunit.client.0.smithi100.stdout: "last_change": "2024-04-07T16:25:38.969419+0000", 2024-04-07T17:51:30.441 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean": "2024-04-07T17:50:36.367170+0000", 2024-04-07T17:51:30.441 INFO:tasks.workunit.client.0.smithi100.stdout: "last_clean_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:30.441 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub": "0'0", 2024-04-07T17:51:30.441 INFO:tasks.workunit.client.0.smithi100.stdout: "last_deep_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:30.441 INFO:tasks.workunit.client.0.smithi100.stdout: "last_epoch_clean": 583, 2024-04-07T17:51:30.441 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fresh": "2024-04-07T17:50:36.367170+0000", 2024-04-07T17:51:30.441 INFO:tasks.workunit.client.0.smithi100.stdout: "last_fullsized": "2024-04-07T17:50:36.367170+0000", 2024-04-07T17:51:30.441 INFO:tasks.workunit.client.0.smithi100.stdout: "last_peered": "2024-04-07T17:50:36.367170+0000", 2024-04-07T17:51:30.441 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub": "0'0", 2024-04-07T17:51:30.441 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_duration": 0, 2024-04-07T17:51:30.441 INFO:tasks.workunit.client.0.smithi100.stdout: "last_scrub_stamp": "2024-04-07T16:12:12.996208+0000", 2024-04-07T17:51:30.441 INFO:tasks.workunit.client.0.smithi100.stdout: "last_undegraded": "2024-04-07T17:50:36.367170+0000", 2024-04-07T17:51:30.442 INFO:tasks.workunit.client.0.smithi100.stdout: "last_unstale": "2024-04-07T17:50:36.367170+0000", 2024-04-07T17:51:30.442 INFO:tasks.workunit.client.0.smithi100.stdout: "log_dups_size": 0, 2024-04-07T17:51:30.442 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:30.442 INFO:tasks.workunit.client.0.smithi100.stdout: "log_start": "0'0", 2024-04-07T17:51:30.442 INFO:tasks.workunit.client.0.smithi100.stdout: "manifest_stats_invalid": false, 2024-04-07T17:51:30.442 INFO:tasks.workunit.client.0.smithi100.stdout: "mapping_epoch": 582, 2024-04-07T17:51:30.442 INFO:tasks.workunit.client.0.smithi100.stdout: "object_location_counts": [], 2024-04-07T17:51:30.442 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_scrubbed": 0, 2024-04-07T17:51:30.442 INFO:tasks.workunit.client.0.smithi100.stdout: "objects_trimmed": 0, 2024-04-07T17:51:30.442 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_stats_invalid": false, 2024-04-07T17:51:30.442 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:30.442 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_start": "0'0", 2024-04-07T17:51:30.442 INFO:tasks.workunit.client.0.smithi100.stdout: "parent": "0.0", 2024-04-07T17:51:30.442 INFO:tasks.workunit.client.0.smithi100.stdout: "parent_split_bits": 0, 2024-04-07T17:51:30.442 INFO:tasks.workunit.client.0.smithi100.stdout: "pgid": "3.3f", 2024-04-07T17:51:30.443 INFO:tasks.workunit.client.0.smithi100.stdout: "pin_stats_invalid": false, 2024-04-07T17:51:30.443 INFO:tasks.workunit.client.0.smithi100.stdout: "purged_snaps": [], 2024-04-07T17:51:30.443 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_epoch": 2713, 2024-04-07T17:51:30.443 INFO:tasks.workunit.client.0.smithi100.stdout: "reported_seq": 4807, 2024-04-07T17:51:30.443 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_duration": 0.0, 2024-04-07T17:51:30.443 INFO:tasks.workunit.client.0.smithi100.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-04-09T01:03:39.405808+0000", 2024-04-07T17:51:30.443 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrim_duration": 0.0, 2024-04-07T17:51:30.443 INFO:tasks.workunit.client.0.smithi100.stdout: "snaptrimq_len": 0, 2024-04-07T17:51:30.443 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:30.443 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:30.443 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:30.443 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:30.443 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:30.443 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:30.443 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:30.444 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:30.444 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:30.444 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:30.444 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:30.444 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:30.444 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:30.444 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:30.444 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:30.444 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:30.444 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:30.444 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:30.444 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:30.444 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:30.444 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:30.444 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:30.444 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:30.444 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:30.445 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:30.445 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:30.445 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:30.445 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:30.445 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:30.445 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:30.445 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:30.445 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:30.445 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:30.445 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:30.445 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:30.445 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:30.445 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:30.445 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:30.445 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:30.445 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:30.446 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:30.446 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:30.446 INFO:tasks.workunit.client.0.smithi100.stdout: "state": "active+clean", 2024-04-07T17:51:30.446 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_invalid": false, 2024-04-07T17:51:30.446 INFO:tasks.workunit.client.0.smithi100.stdout: "up": [ 2024-04-07T17:51:30.446 INFO:tasks.workunit.client.0.smithi100.stdout: 7, 2024-04-07T17:51:30.446 INFO:tasks.workunit.client.0.smithi100.stdout: 6, 2024-04-07T17:51:30.446 INFO:tasks.workunit.client.0.smithi100.stdout: 0 2024-04-07T17:51:30.446 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:30.446 INFO:tasks.workunit.client.0.smithi100.stdout: "up_primary": 7, 2024-04-07T17:51:30.446 INFO:tasks.workunit.client.0.smithi100.stdout: "version": "0'0" 2024-04-07T17:51:30.446 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:30.446 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:30.446 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_per_pool": [ 2024-04-07T17:51:30.447 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:30.447 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": 24, 2024-04-07T17:51:30.447 INFO:tasks.workunit.client.0.smithi100.stdout: "application": [ 2024-04-07T17:51:30.447 INFO:tasks.workunit.client.0.smithi100.stdout: "rbd" 2024-04-07T17:51:30.447 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:30.447 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 24403, 2024-04-07T17:51:30.447 INFO:tasks.workunit.client.0.smithi100.stdout: "num_pg": 8, 2024-04-07T17:51:30.447 INFO:tasks.workunit.client.0.smithi100.stdout: "num_store_stats": 8, 2024-04-07T17:51:30.447 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 24403, 2024-04-07T17:51:30.447 INFO:tasks.workunit.client.0.smithi100.stdout: "poolid": 4, 2024-04-07T17:51:30.447 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:30.447 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 4194323, 2024-04-07T17:51:30.447 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:30.447 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:30.447 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:30.448 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:30.448 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:30.448 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:30.448 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:30.448 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:30.448 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:30.448 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:30.448 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:30.448 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:30.448 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:30.448 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:30.448 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:30.448 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 12, 2024-04-07T17:51:30.448 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 4, 2024-04-07T17:51:30.448 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:30.449 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 4, 2024-04-07T17:51:30.449 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:30.449 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:30.449 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:30.449 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:30.449 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:30.449 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 2, 2024-04-07T17:51:30.449 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:30.449 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:30.449 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:30.449 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:30.449 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:30.449 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:30.449 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:30.449 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 9098, 2024-04-07T17:51:30.449 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 3147283, 2024-04-07T17:51:30.449 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:30.450 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:30.450 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:30.450 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 24273, 2024-04-07T17:51:30.450 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 3030917 2024-04-07T17:51:30.450 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:30.450 INFO:tasks.workunit.client.0.smithi100.stdout: "store_stats": { 2024-04-07T17:51:30.450 INFO:tasks.workunit.client.0.smithi100.stdout: "allocated": 12595200, 2024-04-07T17:51:30.450 INFO:tasks.workunit.client.0.smithi100.stdout: "available": 0, 2024-04-07T17:51:30.450 INFO:tasks.workunit.client.0.smithi100.stdout: "data_compressed": 0, 2024-04-07T17:51:30.450 INFO:tasks.workunit.client.0.smithi100.stdout: "data_compressed_allocated": 0, 2024-04-07T17:51:30.450 INFO:tasks.workunit.client.0.smithi100.stdout: "data_compressed_original": 0, 2024-04-07T17:51:30.450 INFO:tasks.workunit.client.0.smithi100.stdout: "data_stored": 12582969, 2024-04-07T17:51:30.450 INFO:tasks.workunit.client.0.smithi100.stdout: "internal_metadata": 0, 2024-04-07T17:51:30.450 INFO:tasks.workunit.client.0.smithi100.stdout: "internally_reserved": 0, 2024-04-07T17:51:30.451 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_allocated": 0, 2024-04-07T17:51:30.451 INFO:tasks.workunit.client.0.smithi100.stdout: "total": 0 2024-04-07T17:51:30.451 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:30.451 INFO:tasks.workunit.client.0.smithi100.stdout: "up": 24 2024-04-07T17:51:30.451 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:30.451 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:30.451 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": 192, 2024-04-07T17:51:30.451 INFO:tasks.workunit.client.0.smithi100.stdout: "application": [ 2024-04-07T17:51:30.451 INFO:tasks.workunit.client.0.smithi100.stdout: "cephfs" 2024-04-07T17:51:30.451 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:30.451 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 0, 2024-04-07T17:51:30.451 INFO:tasks.workunit.client.0.smithi100.stdout: "num_pg": 64, 2024-04-07T17:51:30.451 INFO:tasks.workunit.client.0.smithi100.stdout: "num_store_stats": 8, 2024-04-07T17:51:30.451 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 0, 2024-04-07T17:51:30.451 INFO:tasks.workunit.client.0.smithi100.stdout: "poolid": 3, 2024-04-07T17:51:30.451 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:30.452 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 0, 2024-04-07T17:51:30.452 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:30.452 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:30.452 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:30.452 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:30.452 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:30.452 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:30.452 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:30.452 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:30.452 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:30.452 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:30.452 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:30.452 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:30.452 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:30.452 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:30.452 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:30.452 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 0, 2024-04-07T17:51:30.453 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 0, 2024-04-07T17:51:30.453 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:30.453 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 0, 2024-04-07T17:51:30.453 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:30.453 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:30.453 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:30.453 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:30.453 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:30.453 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:30.453 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:30.453 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:30.453 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:30.453 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:30.453 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:30.453 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:30.454 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:30.454 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 0, 2024-04-07T17:51:30.454 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 0, 2024-04-07T17:51:30.454 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:30.454 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:30.454 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:30.454 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 0, 2024-04-07T17:51:30.454 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 0 2024-04-07T17:51:30.454 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:30.454 INFO:tasks.workunit.client.0.smithi100.stdout: "store_stats": { 2024-04-07T17:51:30.454 INFO:tasks.workunit.client.0.smithi100.stdout: "allocated": 0, 2024-04-07T17:51:30.454 INFO:tasks.workunit.client.0.smithi100.stdout: "available": 0, 2024-04-07T17:51:30.454 INFO:tasks.workunit.client.0.smithi100.stdout: "data_compressed": 0, 2024-04-07T17:51:30.454 INFO:tasks.workunit.client.0.smithi100.stdout: "data_compressed_allocated": 0, 2024-04-07T17:51:30.455 INFO:tasks.workunit.client.0.smithi100.stdout: "data_compressed_original": 0, 2024-04-07T17:51:30.455 INFO:tasks.workunit.client.0.smithi100.stdout: "data_stored": 0, 2024-04-07T17:51:30.455 INFO:tasks.workunit.client.0.smithi100.stdout: "internal_metadata": 0, 2024-04-07T17:51:30.455 INFO:tasks.workunit.client.0.smithi100.stdout: "internally_reserved": 0, 2024-04-07T17:51:30.455 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_allocated": 0, 2024-04-07T17:51:30.455 INFO:tasks.workunit.client.0.smithi100.stdout: "total": 0 2024-04-07T17:51:30.455 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:30.455 INFO:tasks.workunit.client.0.smithi100.stdout: "up": 192 2024-04-07T17:51:30.455 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:30.455 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:30.455 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": 3, 2024-04-07T17:51:30.455 INFO:tasks.workunit.client.0.smithi100.stdout: "application": [ 2024-04-07T17:51:30.455 INFO:tasks.workunit.client.0.smithi100.stdout: "mgr" 2024-04-07T17:51:30.455 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:30.455 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 257, 2024-04-07T17:51:30.456 INFO:tasks.workunit.client.0.smithi100.stdout: "num_pg": 1, 2024-04-07T17:51:30.456 INFO:tasks.workunit.client.0.smithi100.stdout: "num_store_stats": 3, 2024-04-07T17:51:30.456 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 257, 2024-04-07T17:51:30.456 INFO:tasks.workunit.client.0.smithi100.stdout: "poolid": 1, 2024-04-07T17:51:30.456 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:30.456 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 590368, 2024-04-07T17:51:30.456 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:30.456 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 1771104, 2024-04-07T17:51:30.456 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:30.456 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:30.456 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:30.456 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:30.456 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:30.456 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:30.456 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:30.457 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:30.457 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:30.457 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:30.457 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:30.457 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:30.457 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:30.457 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 6, 2024-04-07T17:51:30.457 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 2, 2024-04-07T17:51:30.457 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:30.457 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 2, 2024-04-07T17:51:30.457 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:30.457 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:30.457 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:30.457 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:30.457 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:30.458 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 0, 2024-04-07T17:51:30.458 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:30.458 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 6, 2024-04-07T17:51:30.458 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:30.458 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:30.458 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:30.458 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:30.458 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:30.458 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 1358, 2024-04-07T17:51:30.458 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 2144, 2024-04-07T17:51:30.458 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:30.458 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:30.458 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:30.458 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 431, 2024-04-07T17:51:30.458 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 2416 2024-04-07T17:51:30.459 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:30.459 INFO:tasks.workunit.client.0.smithi100.stdout: "store_stats": { 2024-04-07T17:51:30.459 INFO:tasks.workunit.client.0.smithi100.stdout: "allocated": 1781760, 2024-04-07T17:51:30.459 INFO:tasks.workunit.client.0.smithi100.stdout: "available": 0, 2024-04-07T17:51:30.459 INFO:tasks.workunit.client.0.smithi100.stdout: "data_compressed": 0, 2024-04-07T17:51:30.459 INFO:tasks.workunit.client.0.smithi100.stdout: "data_compressed_allocated": 0, 2024-04-07T17:51:30.459 INFO:tasks.workunit.client.0.smithi100.stdout: "data_compressed_original": 0, 2024-04-07T17:51:30.459 INFO:tasks.workunit.client.0.smithi100.stdout: "data_stored": 1771104, 2024-04-07T17:51:30.459 INFO:tasks.workunit.client.0.smithi100.stdout: "internal_metadata": 0, 2024-04-07T17:51:30.459 INFO:tasks.workunit.client.0.smithi100.stdout: "internally_reserved": 0, 2024-04-07T17:51:30.459 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_allocated": 0, 2024-04-07T17:51:30.459 INFO:tasks.workunit.client.0.smithi100.stdout: "total": 0 2024-04-07T17:51:30.459 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:30.459 INFO:tasks.workunit.client.0.smithi100.stdout: "up": 3 2024-04-07T17:51:30.459 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:30.459 INFO:tasks.workunit.client.0.smithi100.stdout: { 2024-04-07T17:51:30.459 INFO:tasks.workunit.client.0.smithi100.stdout: "acting": 96, 2024-04-07T17:51:30.460 INFO:tasks.workunit.client.0.smithi100.stdout: "application": [ 2024-04-07T17:51:30.460 INFO:tasks.workunit.client.0.smithi100.stdout: "cephfs" 2024-04-07T17:51:30.460 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:30.460 INFO:tasks.workunit.client.0.smithi100.stdout: "log_size": 68, 2024-04-07T17:51:30.460 INFO:tasks.workunit.client.0.smithi100.stdout: "num_pg": 32, 2024-04-07T17:51:30.460 INFO:tasks.workunit.client.0.smithi100.stdout: "num_store_stats": 8, 2024-04-07T17:51:30.460 INFO:tasks.workunit.client.0.smithi100.stdout: "ondisk_log_size": 68, 2024-04-07T17:51:30.460 INFO:tasks.workunit.client.0.smithi100.stdout: "poolid": 2, 2024-04-07T17:51:30.460 INFO:tasks.workunit.client.0.smithi100.stdout: "stat_sum": { 2024-04-07T17:51:30.460 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes": 6622, 2024-04-07T17:51:30.460 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_hit_set_archive": 0, 2024-04-07T17:51:30.460 INFO:tasks.workunit.client.0.smithi100.stdout: "num_bytes_recovered": 0, 2024-04-07T17:51:30.460 INFO:tasks.workunit.client.0.smithi100.stdout: "num_deep_scrub_errors": 0, 2024-04-07T17:51:30.460 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict": 0, 2024-04-07T17:51:30.460 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_kb": 0, 2024-04-07T17:51:30.460 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_full": 0, 2024-04-07T17:51:30.461 INFO:tasks.workunit.client.0.smithi100.stdout: "num_evict_mode_some": 0, 2024-04-07T17:51:30.461 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush": 0, 2024-04-07T17:51:30.461 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_kb": 0, 2024-04-07T17:51:30.461 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_high": 0, 2024-04-07T17:51:30.461 INFO:tasks.workunit.client.0.smithi100.stdout: "num_flush_mode_low": 0, 2024-04-07T17:51:30.461 INFO:tasks.workunit.client.0.smithi100.stdout: "num_keys_recovered": 0, 2024-04-07T17:51:30.461 INFO:tasks.workunit.client.0.smithi100.stdout: "num_large_omap_objects": 0, 2024-04-07T17:51:30.461 INFO:tasks.workunit.client.0.smithi100.stdout: "num_legacy_snapsets": 0, 2024-04-07T17:51:30.461 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_clones": 0, 2024-04-07T17:51:30.461 INFO:tasks.workunit.client.0.smithi100.stdout: "num_object_copies": 66, 2024-04-07T17:51:30.461 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects": 22, 2024-04-07T17:51:30.461 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_degraded": 0, 2024-04-07T17:51:30.461 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_dirty": 22, 2024-04-07T17:51:30.461 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_hit_set_archive": 0, 2024-04-07T17:51:30.461 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_manifest": 0, 2024-04-07T17:51:30.461 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_misplaced": 0, 2024-04-07T17:51:30.462 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing": 0, 2024-04-07T17:51:30.462 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_missing_on_primary": 0, 2024-04-07T17:51:30.462 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_omap": 14, 2024-04-07T17:51:30.462 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_pinned": 0, 2024-04-07T17:51:30.462 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_recovered": 0, 2024-04-07T17:51:30.462 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_repaired": 0, 2024-04-07T17:51:30.462 INFO:tasks.workunit.client.0.smithi100.stdout: "num_objects_unfound": 0, 2024-04-07T17:51:30.462 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_bytes": 0, 2024-04-07T17:51:30.462 INFO:tasks.workunit.client.0.smithi100.stdout: "num_omap_keys": 0, 2024-04-07T17:51:30.462 INFO:tasks.workunit.client.0.smithi100.stdout: "num_promote": 0, 2024-04-07T17:51:30.462 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read": 79, 2024-04-07T17:51:30.462 INFO:tasks.workunit.client.0.smithi100.stdout: "num_read_kb": 87, 2024-04-07T17:51:30.462 INFO:tasks.workunit.client.0.smithi100.stdout: "num_scrub_errors": 0, 2024-04-07T17:51:30.462 INFO:tasks.workunit.client.0.smithi100.stdout: "num_shallow_scrub_errors": 0, 2024-04-07T17:51:30.462 INFO:tasks.workunit.client.0.smithi100.stdout: "num_whiteouts": 0, 2024-04-07T17:51:30.463 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write": 68, 2024-04-07T17:51:30.463 INFO:tasks.workunit.client.0.smithi100.stdout: "num_write_kb": 36 2024-04-07T17:51:30.463 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:30.463 INFO:tasks.workunit.client.0.smithi100.stdout: "store_stats": { 2024-04-07T17:51:30.463 INFO:tasks.workunit.client.0.smithi100.stdout: "allocated": 110592, 2024-04-07T17:51:30.463 INFO:tasks.workunit.client.0.smithi100.stdout: "available": 0, 2024-04-07T17:51:30.463 INFO:tasks.workunit.client.0.smithi100.stdout: "data_compressed": 0, 2024-04-07T17:51:30.463 INFO:tasks.workunit.client.0.smithi100.stdout: "data_compressed_allocated": 0, 2024-04-07T17:51:30.463 INFO:tasks.workunit.client.0.smithi100.stdout: "data_compressed_original": 0, 2024-04-07T17:51:30.463 INFO:tasks.workunit.client.0.smithi100.stdout: "data_stored": 19866, 2024-04-07T17:51:30.463 INFO:tasks.workunit.client.0.smithi100.stdout: "internal_metadata": 0, 2024-04-07T17:51:30.463 INFO:tasks.workunit.client.0.smithi100.stdout: "internally_reserved": 0, 2024-04-07T17:51:30.463 INFO:tasks.workunit.client.0.smithi100.stdout: "omap_allocated": 48581, 2024-04-07T17:51:30.463 INFO:tasks.workunit.client.0.smithi100.stdout: "total": 0 2024-04-07T17:51:30.463 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:30.464 INFO:tasks.workunit.client.0.smithi100.stdout: "up": 96 2024-04-07T17:51:30.464 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:30.464 INFO:tasks.workunit.client.0.smithi100.stdout: ], 2024-04-07T17:51:30.464 INFO:tasks.workunit.client.0.smithi100.stdout: "usage": { 2024-04-07T17:51:30.464 INFO:tasks.workunit.client.0.smithi100.stdout: "pg_num": 105, 2024-04-07T17:51:30.464 INFO:tasks.workunit.client.0.smithi100.stdout: "pools": 4, 2024-04-07T17:51:30.464 INFO:tasks.workunit.client.0.smithi100.stdout: "stats_by_class": { 2024-04-07T17:51:30.464 INFO:tasks.workunit.client.0.smithi100.stdout: "ssd": { 2024-04-07T17:51:30.464 INFO:tasks.workunit.client.0.smithi100.stdout: "total_avail_bytes": 763743444992, 2024-04-07T17:51:30.464 INFO:tasks.workunit.client.0.smithi100.stdout: "total_bytes": 767960285184, 2024-04-07T17:51:30.464 INFO:tasks.workunit.client.0.smithi100.stdout: "total_used_bytes": 4216840192, 2024-04-07T17:51:30.464 INFO:tasks.workunit.client.0.smithi100.stdout: "total_used_raw_bytes": 4216840192, 2024-04-07T17:51:30.464 INFO:tasks.workunit.client.0.smithi100.stdout: "total_used_raw_ratio": 0.005490961484611034 2024-04-07T17:51:30.464 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:30.464 INFO:tasks.workunit.client.0.smithi100.stdout: }, 2024-04-07T17:51:30.464 INFO:tasks.workunit.client.0.smithi100.stdout: "total_avail_bytes": 763743444992, 2024-04-07T17:51:30.464 INFO:tasks.workunit.client.0.smithi100.stdout: "total_bytes": 767960285184, 2024-04-07T17:51:30.465 INFO:tasks.workunit.client.0.smithi100.stdout: "total_used_bytes": 4216840192 2024-04-07T17:51:30.465 INFO:tasks.workunit.client.0.smithi100.stdout: } 2024-04-07T17:51:30.465 INFO:tasks.workunit.client.0.smithi100.stdout:} 2024-04-07T17:51:30.465 INFO:tasks.workunit.client.0.smithi100.stdout:Device report is generated separately. To see it run 'ceph telemetry show-device'. 2024-04-07T17:51:30.465 INFO:tasks.workunit.client.0.smithi100.stderr:+ ceph telemetry off 2024-04-07T17:51:30.466 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:51:29 smithi100 bash[68137]: [07/Apr/2024:17:51:29] ENGINE Bus STOPPING 2024-04-07T17:51:30.466 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:51:29 smithi100 bash[68137]: [07/Apr/2024:17:51:29] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-04-07T17:51:30.466 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:29 smithi100 bash[74942]: audit 2024-04-07T17:51:27.510696+0000 mgr.y (mgr.25175) 2895 : audit [DBG] from='client.36141 -' entity='client.admin' cmd=[{"prefix": "telemetry show-device", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T17:51:30.466 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:29 smithi100 bash[74942]: audit 2024-04-07T17:51:27.692596+0000 mgr.y (mgr.25175) 2896 : audit [DBG] from='client.36141 -' entity='client.admin' cmd=[{"prefix": "telemetry show-device", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T17:51:30.466 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:29 smithi100 bash[74942]: cluster 2024-04-07T17:51:27.693736+0000 mgr.y (mgr.25175) 2897 : cluster [DBG] pgmap v5171: 105 pgs: 105 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 12 KiB/s rd, 0 B/s wr, 17 op/s 2024-04-07T17:51:30.466 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:29 smithi100 bash[74942]: audit 2024-04-07T17:51:28.070812+0000 mgr.y (mgr.25175) 2898 : audit [DBG] from='client.46049 -' entity='client.admin' cmd=[{"prefix": "telemetry show", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T17:51:30.467 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:29 smithi100 bash[74942]: audit 2024-04-07T17:51:28.302058+0000 mon.a (mon.0) 4951 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:30.467 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:29 smithi100 bash[74942]: audit 2024-04-07T17:51:28.302722+0000 mon.a (mon.0) 4952 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:30.467 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:29 smithi100 bash[74942]: audit 2024-04-07T17:51:28.303180+0000 mon.a (mon.0) 4953 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:30.467 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:29 smithi100 bash[74942]: audit 2024-04-07T17:51:28.306809+0000 mon.a (mon.0) 4954 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:30.467 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:29 smithi100 bash[74942]: audit 2024-04-07T17:51:28.307269+0000 mon.a (mon.0) 4955 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:30.467 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:29 smithi100 bash[74942]: audit 2024-04-07T17:51:28.307702+0000 mon.a (mon.0) 4956 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:30.467 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:29 smithi100 bash[74942]: audit 2024-04-07T17:51:28.310836+0000 mon.a (mon.0) 4957 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:30.467 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:29 smithi100 bash[74942]: audit 2024-04-07T17:51:28.311353+0000 mon.a (mon.0) 4958 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:30.467 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:29 smithi100 bash[74942]: audit 2024-04-07T17:51:28.311727+0000 mon.a (mon.0) 4959 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:30.467 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:29 smithi100 bash[74942]: audit 2024-04-07T17:51:28.330558+0000 mon.a (mon.0) 4960 : audit [INF] from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T17:51:30.467 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:29 smithi100 bash[74942]: audit 2024-04-07T17:51:28.330708+0000 mon.a (mon.0) 4961 : audit [INF] from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T17:51:30.467 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:29 smithi100 bash[74942]: audit 2024-04-07T17:51:28.331319+0000 mon.a (mon.0) 4962 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:30.467 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:29 smithi100 bash[74942]: audit 2024-04-07T17:51:28.331816+0000 mon.a (mon.0) 4963 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:30.467 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:29 smithi100 bash[74942]: audit 2024-04-07T17:51:28.332197+0000 mon.a (mon.0) 4964 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:30.468 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:29 smithi100 bash[74942]: audit 2024-04-07T17:51:28.335655+0000 mon.a (mon.0) 4965 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:30.468 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:29 smithi100 bash[74942]: audit 2024-04-07T17:51:28.336434+0000 mon.a (mon.0) 4966 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:30.468 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:29 smithi100 bash[74942]: audit 2024-04-07T17:51:28.336796+0000 mon.a (mon.0) 4967 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:30.468 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:29 smithi100 bash[74942]: audit 2024-04-07T17:51:28.337490+0000 mon.b (mon.1) 39 : audit [INF] from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T17:51:30.468 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:29 smithi100 bash[74942]: audit 2024-04-07T17:51:28.337645+0000 mon.b (mon.1) 40 : audit [INF] from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T17:51:30.468 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:29 smithi100 bash[74942]: audit 2024-04-07T17:51:28.339274+0000 mon.c (mon.2) 137 : audit [INF] from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T17:51:30.468 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:29 smithi100 bash[74942]: audit 2024-04-07T17:51:28.339413+0000 mon.c (mon.2) 138 : audit [INF] from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T17:51:30.468 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:29 smithi100 bash[74942]: audit 2024-04-07T17:51:28.339803+0000 mon.a (mon.0) 4968 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:30.468 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:29 smithi100 bash[74942]: audit 2024-04-07T17:51:28.340751+0000 mon.a (mon.0) 4969 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:30.468 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:29 smithi100 bash[74942]: audit 2024-04-07T17:51:28.341129+0000 mon.a (mon.0) 4970 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:30.468 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:29 smithi100 bash[74942]: audit 2024-04-07T17:51:28.634584+0000 mon.a (mon.0) 4971 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:51:30.468 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:29 smithi100 bash[74942]: audit 2024-04-07T17:51:28.641307+0000 mon.a (mon.0) 4972 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config-key dump", "format": "json"}]: dispatch 2024-04-07T17:51:30.468 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:29 smithi100 bash[74942]: audit 2024-04-07T17:51:28.820395+0000 mon.a (mon.0) 4973 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:30.468 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:29 smithi100 bash[74942]: audit 2024-04-07T17:51:28.821070+0000 mon.a (mon.0) 4974 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:30.468 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:29 smithi100 bash[74942]: audit 2024-04-07T17:51:28.821614+0000 mon.a (mon.0) 4975 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:30.469 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:29 smithi100 bash[74942]: audit 2024-04-07T17:51:28.825374+0000 mon.a (mon.0) 4976 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:30.469 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:29 smithi100 bash[74942]: audit 2024-04-07T17:51:28.826003+0000 mon.a (mon.0) 4977 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:30.469 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:29 smithi100 bash[74942]: audit 2024-04-07T17:51:28.826492+0000 mon.a (mon.0) 4978 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:30.469 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:29 smithi100 bash[74942]: audit 2024-04-07T17:51:28.829729+0000 mon.a (mon.0) 4979 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:30.469 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:29 smithi100 bash[74942]: audit 2024-04-07T17:51:28.830312+0000 mon.a (mon.0) 4980 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:30.469 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:29 smithi100 bash[74942]: audit 2024-04-07T17:51:28.830636+0000 mon.a (mon.0) 4981 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:30.469 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:29 smithi100 bash[74942]: audit 2024-04-07T17:51:28.850005+0000 mon.a (mon.0) 4982 : audit [INF] from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T17:51:30.469 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:29 smithi100 bash[74942]: audit 2024-04-07T17:51:28.850108+0000 mon.a (mon.0) 4983 : audit [INF] from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T17:51:30.469 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:29 smithi100 bash[74942]: audit 2024-04-07T17:51:28.850624+0000 mon.a (mon.0) 4984 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:30.469 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:29 smithi100 bash[74942]: audit 2024-04-07T17:51:28.851069+0000 mon.a (mon.0) 4985 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:30.469 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:29 smithi100 bash[74942]: audit 2024-04-07T17:51:28.851448+0000 mon.a (mon.0) 4986 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:30.469 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:29 smithi100 bash[74942]: audit 2024-04-07T17:51:28.854932+0000 mon.a (mon.0) 4987 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:30.469 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:29 smithi100 bash[74942]: audit 2024-04-07T17:51:28.855716+0000 mon.a (mon.0) 4988 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:30.469 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:29 smithi100 bash[74942]: audit 2024-04-07T17:51:28.856053+0000 mon.a (mon.0) 4989 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:30.469 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:29 smithi100 bash[74942]: audit 2024-04-07T17:51:28.856733+0000 mon.b (mon.1) 41 : audit [INF] from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T17:51:30.469 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:29 smithi100 bash[74942]: audit 2024-04-07T17:51:28.856883+0000 mon.b (mon.1) 42 : audit [INF] from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T17:51:30.469 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:29 smithi100 bash[74942]: audit 2024-04-07T17:51:28.858743+0000 mon.c (mon.2) 139 : audit [INF] from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T17:51:30.470 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:29 smithi100 bash[74942]: audit 2024-04-07T17:51:28.858889+0000 mon.c (mon.2) 140 : audit [INF] from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T17:51:30.470 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:29 smithi100 bash[74942]: audit 2024-04-07T17:51:28.859149+0000 mon.a (mon.0) 4990 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:30.470 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:29 smithi100 bash[74942]: audit 2024-04-07T17:51:28.860142+0000 mon.a (mon.0) 4991 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:30.470 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:29 smithi100 bash[74942]: audit 2024-04-07T17:51:28.860533+0000 mon.a (mon.0) 4992 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:30.470 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:29 smithi100 bash[76202]: audit 2024-04-07T17:51:27.510696+0000 mgr.y (mgr.25175) 2895 : audit [DBG] from='client.36141 -' entity='client.admin' cmd=[{"prefix": "telemetry show-device", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T17:51:30.470 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:29 smithi100 bash[76202]: audit 2024-04-07T17:51:27.692596+0000 mgr.y (mgr.25175) 2896 : audit [DBG] from='client.36141 -' entity='client.admin' cmd=[{"prefix": "telemetry show-device", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T17:51:30.470 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:29 smithi100 bash[76202]: cluster 2024-04-07T17:51:27.693736+0000 mgr.y (mgr.25175) 2897 : cluster [DBG] pgmap v5171: 105 pgs: 105 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 12 KiB/s rd, 0 B/s wr, 17 op/s 2024-04-07T17:51:30.471 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:29 smithi100 bash[76202]: audit 2024-04-07T17:51:28.070812+0000 mgr.y (mgr.25175) 2898 : audit [DBG] from='client.46049 -' entity='client.admin' cmd=[{"prefix": "telemetry show", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T17:51:30.471 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:29 smithi100 bash[76202]: audit 2024-04-07T17:51:28.302058+0000 mon.a (mon.0) 4951 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:30.471 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:29 smithi100 bash[76202]: audit 2024-04-07T17:51:28.302722+0000 mon.a (mon.0) 4952 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:30.471 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:29 smithi100 bash[76202]: audit 2024-04-07T17:51:28.303180+0000 mon.a (mon.0) 4953 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:30.471 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:29 smithi100 bash[76202]: audit 2024-04-07T17:51:28.306809+0000 mon.a (mon.0) 4954 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:30.471 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:29 smithi100 bash[76202]: audit 2024-04-07T17:51:28.307269+0000 mon.a (mon.0) 4955 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:30.471 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:29 smithi100 bash[76202]: audit 2024-04-07T17:51:28.307702+0000 mon.a (mon.0) 4956 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:30.471 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:29 smithi100 bash[76202]: audit 2024-04-07T17:51:28.310836+0000 mon.a (mon.0) 4957 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:30.471 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:29 smithi100 bash[76202]: audit 2024-04-07T17:51:28.311353+0000 mon.a (mon.0) 4958 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:30.471 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:29 smithi100 bash[76202]: audit 2024-04-07T17:51:28.311727+0000 mon.a (mon.0) 4959 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:30.471 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:29 smithi100 bash[76202]: audit 2024-04-07T17:51:28.330558+0000 mon.a (mon.0) 4960 : audit [INF] from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T17:51:30.471 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:29 smithi100 bash[76202]: audit 2024-04-07T17:51:28.330708+0000 mon.a (mon.0) 4961 : audit [INF] from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T17:51:30.471 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:29 smithi100 bash[76202]: audit 2024-04-07T17:51:28.331319+0000 mon.a (mon.0) 4962 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:30.471 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:29 smithi100 bash[76202]: audit 2024-04-07T17:51:28.331816+0000 mon.a (mon.0) 4963 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:30.473 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:29 smithi100 bash[76202]: audit 2024-04-07T17:51:28.332197+0000 mon.a (mon.0) 4964 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:30.473 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:29 smithi100 bash[76202]: audit 2024-04-07T17:51:28.335655+0000 mon.a (mon.0) 4965 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:30.473 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:29 smithi100 bash[76202]: audit 2024-04-07T17:51:28.336434+0000 mon.a (mon.0) 4966 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:30.473 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:29 smithi100 bash[76202]: audit 2024-04-07T17:51:28.336796+0000 mon.a (mon.0) 4967 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:30.473 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:29 smithi100 bash[76202]: audit 2024-04-07T17:51:28.337490+0000 mon.b (mon.1) 39 : audit [INF] from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T17:51:30.473 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:29 smithi100 bash[76202]: audit 2024-04-07T17:51:28.337645+0000 mon.b (mon.1) 40 : audit [INF] from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T17:51:30.473 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:29 smithi100 bash[76202]: audit 2024-04-07T17:51:28.339274+0000 mon.c (mon.2) 137 : audit [INF] from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T17:51:30.473 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:29 smithi100 bash[76202]: audit 2024-04-07T17:51:28.339413+0000 mon.c (mon.2) 138 : audit [INF] from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T17:51:30.473 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:29 smithi100 bash[76202]: audit 2024-04-07T17:51:28.339803+0000 mon.a (mon.0) 4968 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:30.473 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:29 smithi100 bash[76202]: audit 2024-04-07T17:51:28.340751+0000 mon.a (mon.0) 4969 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:30.474 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:29 smithi100 bash[76202]: audit 2024-04-07T17:51:28.341129+0000 mon.a (mon.0) 4970 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:30.474 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:29 smithi100 bash[76202]: audit 2024-04-07T17:51:28.634584+0000 mon.a (mon.0) 4971 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:51:30.474 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:29 smithi100 bash[76202]: audit 2024-04-07T17:51:28.641307+0000 mon.a (mon.0) 4972 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config-key dump", "format": "json"}]: dispatch 2024-04-07T17:51:30.474 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:29 smithi100 bash[76202]: audit 2024-04-07T17:51:28.820395+0000 mon.a (mon.0) 4973 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:30.474 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:29 smithi100 bash[76202]: audit 2024-04-07T17:51:28.821070+0000 mon.a (mon.0) 4974 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:30.474 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:29 smithi100 bash[76202]: audit 2024-04-07T17:51:28.821614+0000 mon.a (mon.0) 4975 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:30.474 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:29 smithi100 bash[76202]: audit 2024-04-07T17:51:28.825374+0000 mon.a (mon.0) 4976 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:30.474 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:29 smithi100 bash[76202]: audit 2024-04-07T17:51:28.826003+0000 mon.a (mon.0) 4977 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:30.474 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:29 smithi100 bash[76202]: audit 2024-04-07T17:51:28.826492+0000 mon.a (mon.0) 4978 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:30.474 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:29 smithi100 bash[76202]: audit 2024-04-07T17:51:28.829729+0000 mon.a (mon.0) 4979 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:30.474 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:29 smithi100 bash[76202]: audit 2024-04-07T17:51:28.830312+0000 mon.a (mon.0) 4980 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:30.474 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:29 smithi100 bash[76202]: audit 2024-04-07T17:51:28.830636+0000 mon.a (mon.0) 4981 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:30.474 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:29 smithi100 bash[76202]: audit 2024-04-07T17:51:28.850005+0000 mon.a (mon.0) 4982 : audit [INF] from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T17:51:30.475 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:29 smithi100 bash[76202]: audit 2024-04-07T17:51:28.850108+0000 mon.a (mon.0) 4983 : audit [INF] from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T17:51:30.475 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:29 smithi100 bash[76202]: audit 2024-04-07T17:51:28.850624+0000 mon.a (mon.0) 4984 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:30.475 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:29 smithi100 bash[76202]: audit 2024-04-07T17:51:28.851069+0000 mon.a (mon.0) 4985 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:30.475 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:29 smithi100 bash[76202]: audit 2024-04-07T17:51:28.851448+0000 mon.a (mon.0) 4986 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:30.475 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:29 smithi100 bash[76202]: audit 2024-04-07T17:51:28.854932+0000 mon.a (mon.0) 4987 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:30.475 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:29 smithi100 bash[76202]: audit 2024-04-07T17:51:28.855716+0000 mon.a (mon.0) 4988 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:30.475 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:29 smithi100 bash[76202]: audit 2024-04-07T17:51:28.856053+0000 mon.a (mon.0) 4989 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:30.475 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:29 smithi100 bash[76202]: audit 2024-04-07T17:51:28.856733+0000 mon.b (mon.1) 41 : audit [INF] from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T17:51:30.475 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:29 smithi100 bash[76202]: audit 2024-04-07T17:51:28.856883+0000 mon.b (mon.1) 42 : audit [INF] from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T17:51:30.475 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:29 smithi100 bash[76202]: audit 2024-04-07T17:51:28.858743+0000 mon.c (mon.2) 139 : audit [INF] from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T17:51:30.475 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:29 smithi100 bash[76202]: audit 2024-04-07T17:51:28.858889+0000 mon.c (mon.2) 140 : audit [INF] from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T17:51:30.475 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:29 smithi100 bash[76202]: audit 2024-04-07T17:51:28.859149+0000 mon.a (mon.0) 4990 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:30.475 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:29 smithi100 bash[76202]: audit 2024-04-07T17:51:28.860142+0000 mon.a (mon.0) 4991 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:30.475 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:29 smithi100 bash[76202]: audit 2024-04-07T17:51:28.860533+0000 mon.a (mon.0) 4992 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:30.476 INFO:tasks.workunit.client.0.smithi100.stdout:Telemetry is currently not enabled, nothing to turn off. Please consider opting-in with `ceph telemetry on`. 2024-04-07T17:51:30.476 INFO:tasks.workunit.client.0.smithi100.stdout:Preview sample reports with `ceph telemetry preview`. 2024-04-07T17:51:30.477 INFO:tasks.workunit.client.0.smithi100.stdout:OK 2024-04-07T17:51:30.477 INFO:tasks.workunit.client.0.smithi100.stderr:+ echo OK 2024-04-07T17:51:30.478 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-07T17:51:30.478 DEBUG:teuthology.orchestra.run.smithi100:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-07T17:51:30.478 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:51:29 smithi100 bash[68137]: [07/Apr/2024:17:51:29] ENGINE Bus STOPPED 2024-04-07T17:51:30.478 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:51:29 smithi100 bash[68137]: [07/Apr/2024:17:51:29] ENGINE Bus STARTING 2024-04-07T17:51:30.478 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:51:29 smithi100 bash[68137]: [07/Apr/2024:17:51:29] ENGINE Serving on http://:::9283 2024-04-07T17:51:30.479 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:51:29 smithi100 bash[68137]: [07/Apr/2024:17:51:29] ENGINE Bus STARTED 2024-04-07T17:51:30.479 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:30 smithi100 bash[74942]: audit 2024-04-07T17:51:28.631399+0000 mgr.y (mgr.25175) 2899 : audit [DBG] from='client.46049 -' entity='client.admin' cmd=[{"prefix": "telemetry show", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T17:51:30.479 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:30 smithi100 bash[74942]: audit 2024-04-07T17:51:29.368444+0000 mon.a (mon.0) 4993 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:51:30.479 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:30 smithi100 bash[74942]: audit 2024-04-07T17:51:29.377079+0000 mon.a (mon.0) 4994 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:51:30.479 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:30 smithi100 bash[74942]: audit 2024-04-07T17:51:29.387402+0000 mon.a (mon.0) 4995 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:51:30.479 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:30 smithi100 bash[74942]: audit 2024-04-07T17:51:29.407010+0000 mon.a (mon.0) 4996 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:51:30.479 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:30 smithi100 bash[74942]: audit 2024-04-07T17:51:29.408863+0000 mon.a (mon.0) 4997 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:51:30.479 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:30 smithi100 bash[74942]: audit 2024-04-07T17:51:29.409587+0000 mon.a (mon.0) 4998 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:51:30.479 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:30 smithi100 bash[74942]: audit 2024-04-07T17:51:29.415364+0000 mon.a (mon.0) 4999 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:51:30.480 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:30 smithi100 bash[76202]: audit 2024-04-07T17:51:28.631399+0000 mgr.y (mgr.25175) 2899 : audit [DBG] from='client.46049 -' entity='client.admin' cmd=[{"prefix": "telemetry show", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T17:51:30.480 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:30 smithi100 bash[76202]: audit 2024-04-07T17:51:29.368444+0000 mon.a (mon.0) 4993 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:51:30.480 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:30 smithi100 bash[76202]: audit 2024-04-07T17:51:29.377079+0000 mon.a (mon.0) 4994 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:51:30.480 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:30 smithi100 bash[76202]: audit 2024-04-07T17:51:29.387402+0000 mon.a (mon.0) 4995 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:51:30.480 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:30 smithi100 bash[76202]: audit 2024-04-07T17:51:29.407010+0000 mon.a (mon.0) 4996 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:51:30.480 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:30 smithi100 bash[76202]: audit 2024-04-07T17:51:29.408863+0000 mon.a (mon.0) 4997 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:51:30.480 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:30 smithi100 bash[76202]: audit 2024-04-07T17:51:29.409587+0000 mon.a (mon.0) 4998 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:51:30.480 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:30 smithi100 bash[76202]: audit 2024-04-07T17:51:29.415364+0000 mon.a (mon.0) 4999 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:51:30.481 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:29 smithi144 bash[66654]: audit 2024-04-07T17:51:27.510696+0000 mgr.y (mgr.25175) 2895 : audit [DBG] from='client.36141 -' entity='client.admin' cmd=[{"prefix": "telemetry show-device", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T17:51:30.481 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:29 smithi144 bash[66654]: audit 2024-04-07T17:51:27.692596+0000 mgr.y (mgr.25175) 2896 : audit [DBG] from='client.36141 -' entity='client.admin' cmd=[{"prefix": "telemetry show-device", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T17:51:30.481 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:29 smithi144 bash[66654]: cluster 2024-04-07T17:51:27.693736+0000 mgr.y (mgr.25175) 2897 : cluster [DBG] pgmap v5171: 105 pgs: 105 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 12 KiB/s rd, 0 B/s wr, 17 op/s 2024-04-07T17:51:30.481 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:29 smithi144 bash[66654]: audit 2024-04-07T17:51:28.070812+0000 mgr.y (mgr.25175) 2898 : audit [DBG] from='client.46049 -' entity='client.admin' cmd=[{"prefix": "telemetry show", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T17:51:30.481 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:29 smithi144 bash[66654]: audit 2024-04-07T17:51:28.302058+0000 mon.a (mon.0) 4951 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:30.481 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:29 smithi144 bash[66654]: audit 2024-04-07T17:51:28.302722+0000 mon.a (mon.0) 4952 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:30.481 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:29 smithi144 bash[66654]: audit 2024-04-07T17:51:28.303180+0000 mon.a (mon.0) 4953 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:30.481 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:29 smithi144 bash[66654]: audit 2024-04-07T17:51:28.306809+0000 mon.a (mon.0) 4954 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:30.482 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:29 smithi144 bash[66654]: audit 2024-04-07T17:51:28.307269+0000 mon.a (mon.0) 4955 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:30.482 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:29 smithi144 bash[66654]: audit 2024-04-07T17:51:28.307702+0000 mon.a (mon.0) 4956 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:30.482 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:29 smithi144 bash[66654]: audit 2024-04-07T17:51:28.310836+0000 mon.a (mon.0) 4957 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:30.482 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:29 smithi144 bash[66654]: audit 2024-04-07T17:51:28.311353+0000 mon.a (mon.0) 4958 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:30.482 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:29 smithi144 bash[66654]: audit 2024-04-07T17:51:28.311727+0000 mon.a (mon.0) 4959 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:30.482 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:29 smithi144 bash[66654]: audit 2024-04-07T17:51:28.330558+0000 mon.a (mon.0) 4960 : audit [INF] from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T17:51:30.482 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:29 smithi144 bash[66654]: audit 2024-04-07T17:51:28.330708+0000 mon.a (mon.0) 4961 : audit [INF] from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T17:51:30.482 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:29 smithi144 bash[66654]: audit 2024-04-07T17:51:28.331319+0000 mon.a (mon.0) 4962 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:30.482 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:29 smithi144 bash[66654]: audit 2024-04-07T17:51:28.331816+0000 mon.a (mon.0) 4963 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:30.482 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:29 smithi144 bash[66654]: audit 2024-04-07T17:51:28.332197+0000 mon.a (mon.0) 4964 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:30.482 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:29 smithi144 bash[66654]: audit 2024-04-07T17:51:28.335655+0000 mon.a (mon.0) 4965 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:30.482 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:29 smithi144 bash[66654]: audit 2024-04-07T17:51:28.336434+0000 mon.a (mon.0) 4966 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:30.482 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:29 smithi144 bash[66654]: audit 2024-04-07T17:51:28.336796+0000 mon.a (mon.0) 4967 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:30.482 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:29 smithi144 bash[66654]: audit 2024-04-07T17:51:28.337490+0000 mon.b (mon.1) 39 : audit [INF] from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T17:51:30.482 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:29 smithi144 bash[66654]: audit 2024-04-07T17:51:28.337645+0000 mon.b (mon.1) 40 : audit [INF] from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T17:51:30.483 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:29 smithi144 bash[66654]: audit 2024-04-07T17:51:28.339274+0000 mon.c (mon.2) 137 : audit [INF] from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T17:51:30.483 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:29 smithi144 bash[66654]: audit 2024-04-07T17:51:28.339413+0000 mon.c (mon.2) 138 : audit [INF] from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T17:51:30.483 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:29 smithi144 bash[66654]: audit 2024-04-07T17:51:28.339803+0000 mon.a (mon.0) 4968 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:30.483 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:29 smithi144 bash[66654]: audit 2024-04-07T17:51:28.340751+0000 mon.a (mon.0) 4969 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:30.483 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:29 smithi144 bash[66654]: audit 2024-04-07T17:51:28.341129+0000 mon.a (mon.0) 4970 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:30.483 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:29 smithi144 bash[66654]: audit 2024-04-07T17:51:28.634584+0000 mon.a (mon.0) 4971 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:51:30.483 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:29 smithi144 bash[66654]: audit 2024-04-07T17:51:28.641307+0000 mon.a (mon.0) 4972 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config-key dump", "format": "json"}]: dispatch 2024-04-07T17:51:30.483 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:29 smithi144 bash[66654]: audit 2024-04-07T17:51:28.820395+0000 mon.a (mon.0) 4973 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:30.483 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:29 smithi144 bash[66654]: audit 2024-04-07T17:51:28.821070+0000 mon.a (mon.0) 4974 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:30.483 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:29 smithi144 bash[66654]: audit 2024-04-07T17:51:28.821614+0000 mon.a (mon.0) 4975 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:30.483 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:29 smithi144 bash[66654]: audit 2024-04-07T17:51:28.825374+0000 mon.a (mon.0) 4976 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:30.483 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:29 smithi144 bash[66654]: audit 2024-04-07T17:51:28.826003+0000 mon.a (mon.0) 4977 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:30.483 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:29 smithi144 bash[66654]: audit 2024-04-07T17:51:28.826492+0000 mon.a (mon.0) 4978 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:30.483 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:29 smithi144 bash[66654]: audit 2024-04-07T17:51:28.829729+0000 mon.a (mon.0) 4979 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:30.484 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:29 smithi144 bash[66654]: audit 2024-04-07T17:51:28.830312+0000 mon.a (mon.0) 4980 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:30.484 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:29 smithi144 bash[66654]: audit 2024-04-07T17:51:28.830636+0000 mon.a (mon.0) 4981 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:30.484 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:29 smithi144 bash[66654]: audit 2024-04-07T17:51:28.850005+0000 mon.a (mon.0) 4982 : audit [INF] from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T17:51:30.484 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:29 smithi144 bash[66654]: audit 2024-04-07T17:51:28.850108+0000 mon.a (mon.0) 4983 : audit [INF] from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T17:51:30.484 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:29 smithi144 bash[66654]: audit 2024-04-07T17:51:28.850624+0000 mon.a (mon.0) 4984 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:30.484 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:29 smithi144 bash[66654]: audit 2024-04-07T17:51:28.851069+0000 mon.a (mon.0) 4985 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:30.484 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:29 smithi144 bash[66654]: audit 2024-04-07T17:51:28.851448+0000 mon.a (mon.0) 4986 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:30.484 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:29 smithi144 bash[66654]: audit 2024-04-07T17:51:28.854932+0000 mon.a (mon.0) 4987 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:30.484 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:29 smithi144 bash[66654]: audit 2024-04-07T17:51:28.855716+0000 mon.a (mon.0) 4988 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:30.484 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:29 smithi144 bash[66654]: audit 2024-04-07T17:51:28.856053+0000 mon.a (mon.0) 4989 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:30.484 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:29 smithi144 bash[66654]: audit 2024-04-07T17:51:28.856733+0000 mon.b (mon.1) 41 : audit [INF] from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T17:51:30.484 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:29 smithi144 bash[66654]: audit 2024-04-07T17:51:28.856883+0000 mon.b (mon.1) 42 : audit [INF] from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T17:51:30.484 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:29 smithi144 bash[66654]: audit 2024-04-07T17:51:28.858743+0000 mon.c (mon.2) 139 : audit [INF] from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-04-07T17:51:30.484 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:29 smithi144 bash[66654]: audit 2024-04-07T17:51:28.858889+0000 mon.c (mon.2) 140 : audit [INF] from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-04-07T17:51:30.484 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:29 smithi144 bash[66654]: audit 2024-04-07T17:51:28.859149+0000 mon.a (mon.0) 4990 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-07T17:51:30.485 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:29 smithi144 bash[66654]: audit 2024-04-07T17:51:28.860142+0000 mon.a (mon.0) 4991 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-07T17:51:30.485 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:29 smithi144 bash[66654]: audit 2024-04-07T17:51:28.860533+0000 mon.a (mon.0) 4992 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-07T17:51:30.494 INFO:tasks.workunit:Stopping ['test_telemetry_reef_x.sh'] on client.0... 2024-04-07T17:51:30.494 DEBUG:teuthology.orchestra.run.smithi100:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-04-07T17:51:30.614 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:30 smithi144 bash[66654]: audit 2024-04-07T17:51:28.631399+0000 mgr.y (mgr.25175) 2899 : audit [DBG] from='client.46049 -' entity='client.admin' cmd=[{"prefix": "telemetry show", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T17:51:30.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:30 smithi144 bash[66654]: audit 2024-04-07T17:51:29.368444+0000 mon.a (mon.0) 4993 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:51:30.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:30 smithi144 bash[66654]: audit 2024-04-07T17:51:29.377079+0000 mon.a (mon.0) 4994 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:51:30.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:30 smithi144 bash[66654]: audit 2024-04-07T17:51:29.387402+0000 mon.a (mon.0) 4995 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:51:30.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:30 smithi144 bash[66654]: audit 2024-04-07T17:51:29.407010+0000 mon.a (mon.0) 4996 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-07T17:51:30.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:30 smithi144 bash[66654]: audit 2024-04-07T17:51:29.408863+0000 mon.a (mon.0) 4997 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-07T17:51:30.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:30 smithi144 bash[66654]: audit 2024-04-07T17:51:29.409587+0000 mon.a (mon.0) 4998 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-07T17:51:30.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:30 smithi144 bash[66654]: audit 2024-04-07T17:51:29.415364+0000 mon.a (mon.0) 4999 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:51:30.815 DEBUG:teuthology.parallel:result is None 2024-04-07T17:51:30.815 DEBUG:teuthology.orchestra.run.smithi100:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-04-07T17:51:30.828 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-04-07T17:51:30.828 DEBUG:teuthology.orchestra.run.smithi100:> rmdir -- /home/ubuntu/cephtest/mnt.0 2024-04-07T17:51:30.876 INFO:tasks.workunit:Deleted artificial mount point /home/ubuntu/cephtest/mnt.0/client.0 2024-04-07T17:51:30.876 INFO:teuthology.run_tasks:Running task print... 2024-04-07T17:51:30.886 INFO:teuthology.task.print:**** done end telemetry x... 2024-04-07T17:51:30.887 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-04-07T17:51:30.898 INFO:tasks.cephadm:Teardown begin 2024-04-07T17:51:30.898 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-07T17:51:30.898 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-07T17:51:31.272 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-07T17:51:31.273 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":14,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":54211,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/1569910340","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":1569910340},{"type":"v1","addr":"172.21.15.144:6833","nonce":1569910340}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":14}],"filesystems":[{"mdsmap":{"epoch":13,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-07T16:12:13.365189+0000","modified":"2024-04-07T16:26:22.589632+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":623,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":44315},"failed":[],"damaged":[],"stopped":[],"info":{"gid_44315":{"gid":44315,"name":"a","rank":0,"incarnation":10,"state":"up:active","state_seq":8,"addr":"172.21.15.100:6835/2107704360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2107704360},{"type":"v1","addr":"172.21.15.100:6835","nonce":2107704360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":44315,"qdb_cluster":[44315]},"id":1}]} 2024-04-07T17:51:31.273 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 14 2024-04-07T17:51:31.291 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-07T17:51:31.453 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:31 smithi100 bash[76202]: audit 2024-04-07T17:51:29.363437+0000 mgr.y (mgr.25175) 2900 : audit [DBG] from='client.46064 -' entity='client.admin' cmd=[{"prefix": "telemetry off", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T17:51:31.453 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:31 smithi100 bash[76202]: audit 2024-04-07T17:51:29.559767+0000 mgr.y (mgr.25175) 2901 : audit [DBG] from='client.46064 -' entity='client.admin' cmd=[{"prefix": "telemetry off", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T17:51:31.453 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:31 smithi100 bash[76202]: cluster 2024-04-07T17:51:29.694641+0000 mgr.y (mgr.25175) 2902 : cluster [DBG] pgmap v5172: 105 pgs: 105 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 12 KiB/s rd, 0 B/s wr, 17 op/s 2024-04-07T17:51:31.453 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:31 smithi100 bash[74942]: audit 2024-04-07T17:51:29.363437+0000 mgr.y (mgr.25175) 2900 : audit [DBG] from='client.46064 -' entity='client.admin' cmd=[{"prefix": "telemetry off", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T17:51:31.453 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:31 smithi100 bash[74942]: audit 2024-04-07T17:51:29.559767+0000 mgr.y (mgr.25175) 2901 : audit [DBG] from='client.46064 -' entity='client.admin' cmd=[{"prefix": "telemetry off", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T17:51:31.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:31 smithi100 bash[74942]: cluster 2024-04-07T17:51:29.694641+0000 mgr.y (mgr.25175) 2902 : cluster [DBG] pgmap v5172: 105 pgs: 105 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 12 KiB/s rd, 0 B/s wr, 17 op/s 2024-04-07T17:51:31.614 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:31 smithi144 bash[66654]: audit 2024-04-07T17:51:29.363437+0000 mgr.y (mgr.25175) 2900 : audit [DBG] from='client.46064 -' entity='client.admin' cmd=[{"prefix": "telemetry off", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T17:51:31.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:31 smithi144 bash[66654]: audit 2024-04-07T17:51:29.559767+0000 mgr.y (mgr.25175) 2901 : audit [DBG] from='client.46064 -' entity='client.admin' cmd=[{"prefix": "telemetry off", "target": ["mon-mgr", ""]}]: dispatch 2024-04-07T17:51:31.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:31 smithi144 bash[66654]: cluster 2024-04-07T17:51:29.694641+0000 mgr.y (mgr.25175) 2902 : cluster [DBG] pgmap v5172: 105 pgs: 105 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 12 KiB/s rd, 0 B/s wr, 17 op/s 2024-04-07T17:51:31.616 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-07T17:51:31.617 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":14,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":54211,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/1569910340","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":1569910340},{"type":"v1","addr":"172.21.15.144:6833","nonce":1569910340}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":14}],"filesystems":[{"mdsmap":{"epoch":13,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-07T16:12:13.365189+0000","modified":"2024-04-07T16:26:22.589632+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":623,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":44315},"failed":[],"damaged":[],"stopped":[],"info":{"gid_44315":{"gid":44315,"name":"a","rank":0,"incarnation":10,"state":"up:active","state_seq":8,"addr":"172.21.15.100:6835/2107704360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2107704360},{"type":"v1","addr":"172.21.15.100:6835","nonce":2107704360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":44315,"qdb_cluster":[44315]},"id":1}]} 2024-04-07T17:51:31.617 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 14 2024-04-07T17:51:31.635 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-07T17:51:31.949 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-07T17:51:31.949 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":2713,"fsid":"0702e3a8-f4f8-11ee-b649-cb9ed24678a4","created":"2024-04-07T16:02:21.629692+0000","modified":"2024-04-07T17:50:35.959386+0000","last_up_change":"2024-04-07T16:25:37.915048+0000","last_in_change":"2024-04-07T16:11:04.793434+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":37,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":123,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-07T16:07:26.784071+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":7.8899998664855957,"score_stable":7.8899998664855957,"optimal_score":0.37999999523162842,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-07T16:12:11.856505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-07T16:12:12.737709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"rbd","create_time":"2024-04-07T16:13:28.913394+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"1083","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":8,"snap_epoch":1083,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":3,"score_stable":3,"optimal_score":1,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8facd48e-0a0c-4fda-aee2-f001b60a3811","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":8,"last_clean_end":376,"up_from":389,"up_thru":2627,"down_at":377,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":165581137},{"type":"v1","addr":"172.21.15.100:6803","nonce":165581137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":165581137},{"type":"v1","addr":"172.21.15.100:6805","nonce":165581137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":165581137},{"type":"v1","addr":"172.21.15.100:6809","nonce":165581137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":165581137},{"type":"v1","addr":"172.21.15.100:6807","nonce":165581137}]},"public_addr":"172.21.15.100:6803/165581137","cluster_addr":"172.21.15.100:6805/165581137","heartbeat_back_addr":"172.21.15.100:6809/165581137","heartbeat_front_addr":"172.21.15.100:6807/165581137","state":["exists","up"]},{"osd":1,"uuid":"cb5f6c23-6764-48b6-8b21-97a36b004786","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":13,"last_clean_end":403,"up_from":416,"up_thru":2627,"down_at":404,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":1469336877},{"type":"v1","addr":"172.21.15.100:6811","nonce":1469336877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1469336877},{"type":"v1","addr":"172.21.15.100:6813","nonce":1469336877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1469336877},{"type":"v1","addr":"172.21.15.100:6817","nonce":1469336877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1469336877},{"type":"v1","addr":"172.21.15.100:6815","nonce":1469336877}]},"public_addr":"172.21.15.100:6811/1469336877","cluster_addr":"172.21.15.100:6813/1469336877","heartbeat_back_addr":"172.21.15.100:6817/1469336877","heartbeat_front_addr":"172.21.15.100:6815/1469336877","state":["exists","up"]},{"osd":2,"uuid":"1650aedb-9ce9-4c14-a059-e1f089e5e63f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":18,"last_clean_end":431,"up_from":444,"up_thru":2627,"down_at":432,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":2767460830},{"type":"v1","addr":"172.21.15.100:6819","nonce":2767460830}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2767460830},{"type":"v1","addr":"172.21.15.100:6821","nonce":2767460830}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":2767460830},{"type":"v1","addr":"172.21.15.100:6825","nonce":2767460830}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":2767460830},{"type":"v1","addr":"172.21.15.100:6823","nonce":2767460830}]},"public_addr":"172.21.15.100:6819/2767460830","cluster_addr":"172.21.15.100:6821/2767460830","heartbeat_back_addr":"172.21.15.100:6825/2767460830","heartbeat_front_addr":"172.21.15.100:6823/2767460830","state":["exists","up"]},{"osd":3,"uuid":"17591c6b-862b-43b2-858b-02f4c3199247","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":25,"last_clean_end":459,"up_from":470,"up_thru":2627,"down_at":460,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":269478984},{"type":"v1","addr":"172.21.15.100:6827","nonce":269478984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":269478984},{"type":"v1","addr":"172.21.15.100:6829","nonce":269478984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":269478984},{"type":"v1","addr":"172.21.15.100:6833","nonce":269478984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":269478984},{"type":"v1","addr":"172.21.15.100:6831","nonce":269478984}]},"public_addr":"172.21.15.100:6827/269478984","cluster_addr":"172.21.15.100:6829/269478984","heartbeat_back_addr":"172.21.15.100:6833/269478984","heartbeat_front_addr":"172.21.15.100:6831/269478984","state":["exists","up"]},{"osd":4,"uuid":"6ecc1205-7c78-461e-8037-640136cb3658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":30,"last_clean_end":487,"up_from":499,"up_thru":2627,"down_at":488,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":1068767678},{"type":"v1","addr":"172.21.15.144:6801","nonce":1068767678}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":1068767678},{"type":"v1","addr":"172.21.15.144:6803","nonce":1068767678}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1068767678},{"type":"v1","addr":"172.21.15.144:6807","nonce":1068767678}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":1068767678},{"type":"v1","addr":"172.21.15.144:6805","nonce":1068767678}]},"public_addr":"172.21.15.144:6801/1068767678","cluster_addr":"172.21.15.144:6803/1068767678","heartbeat_back_addr":"172.21.15.144:6807/1068767678","heartbeat_front_addr":"172.21.15.144:6805/1068767678","state":["exists","up"]},{"osd":5,"uuid":"54789cc8-1c3c-46a4-a92e-8881564338db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":38,"last_clean_end":513,"up_from":526,"up_thru":2627,"down_at":514,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1594566125},{"type":"v1","addr":"172.21.15.144:6809","nonce":1594566125}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1594566125},{"type":"v1","addr":"172.21.15.144:6811","nonce":1594566125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1594566125},{"type":"v1","addr":"172.21.15.144:6815","nonce":1594566125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1594566125},{"type":"v1","addr":"172.21.15.144:6813","nonce":1594566125}]},"public_addr":"172.21.15.144:6809/1594566125","cluster_addr":"172.21.15.144:6811/1594566125","heartbeat_back_addr":"172.21.15.144:6815/1594566125","heartbeat_front_addr":"172.21.15.144:6813/1594566125","state":["exists","up"]},{"osd":6,"uuid":"ac619497-7605-4ca6-84bf-3abb3f6f32d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":46,"last_clean_end":542,"up_from":553,"up_thru":2627,"down_at":543,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2416344523},{"type":"v1","addr":"172.21.15.144:6817","nonce":2416344523}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2416344523},{"type":"v1","addr":"172.21.15.144:6819","nonce":2416344523}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2416344523},{"type":"v1","addr":"172.21.15.144:6823","nonce":2416344523}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2416344523},{"type":"v1","addr":"172.21.15.144:6821","nonce":2416344523}]},"public_addr":"172.21.15.144:6817/2416344523","cluster_addr":"172.21.15.144:6819/2416344523","heartbeat_back_addr":"172.21.15.144:6823/2416344523","heartbeat_front_addr":"172.21.15.144:6821/2416344523","state":["exists","up"]},{"osd":7,"uuid":"b85b3422-9da2-4605-88e7-fe9c3b2020cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":54,"last_clean_end":569,"up_from":582,"up_thru":2627,"down_at":570,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":3641044519},{"type":"v1","addr":"172.21.15.144:6825","nonce":3641044519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":3641044519},{"type":"v1","addr":"172.21.15.144:6827","nonce":3641044519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":3641044519},{"type":"v1","addr":"172.21.15.144:6831","nonce":3641044519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":3641044519},{"type":"v1","addr":"172.21.15.144:6829","nonce":3641044519}]},"public_addr":"172.21.15.144:6825/3641044519","cluster_addr":"172.21.15.144:6827/3641044519","heartbeat_back_addr":"172.21.15.144:6831/3641044519","heartbeat_front_addr":"172.21.15.144:6829/3641044519","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-07T16:05:44.696671+0000","dead_epoch":376},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-07T16:06:33.787321+0000","dead_epoch":403},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-07T16:07:22.983158+0000","dead_epoch":431},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-07T16:08:12.935606+0000","dead_epoch":458},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-07T16:09:00.218619+0000","dead_epoch":487},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-07T16:09:50.222185+0000","dead_epoch":513},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-07T16:10:37.856158+0000","dead_epoch":542},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-07T16:11:28.267877+0000","dead_epoch":569}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.7","mappings":[{"from":7,"to":4}]},{"pgid":"3.1","mappings":[{"from":0,"to":2}]},{"pgid":"3.12","mappings":[{"from":0,"to":2}]},{"pgid":"3.15","mappings":[{"from":4,"to":1}]},{"pgid":"3.17","mappings":[{"from":0,"to":2}]},{"pgid":"3.1d","mappings":[{"from":4,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.100:0/594324814":"2024-04-07T18:08:56.860132+0000","172.21.15.144:6833/226244023":"2024-04-08T16:26:19.573335+0000","172.21.15.100:0/2558802424":"2024-04-08T16:18:20.617755+0000","172.21.15.100:6800/3429225708":"2024-04-08T16:18:20.617755+0000","172.21.15.100:0/258495078":"2024-04-08T16:18:20.617755+0000","172.21.15.100:0/2754245222":"2024-04-08T16:20:16.558634+0000","172.21.15.144:0/2599140395":"2024-04-08T16:19:13.981948+0000","172.21.15.144:0/653654312":"2024-04-08T16:19:13.981948+0000","172.21.15.144:6835/3360802682":"2024-04-08T16:19:13.981948+0000","172.21.15.100:6801/190788518":"2024-04-08T16:12:52.577534+0000","172.21.15.144:0/1073632838":"2024-04-08T16:19:13.981948+0000","172.21.15.100:6800/2710911757":"2024-04-08T16:03:24.146096+0000","172.21.15.100:0/1631130639":"2024-04-08T16:18:20.617755+0000","172.21.15.100:0/3273267396":"2024-04-08T16:12:52.577534+0000","172.21.15.100:0/3095497165":"2024-04-07T18:08:57.773920+0000","172.21.15.100:0/4121992264":"2024-04-08T16:02:53.269514+0000","172.21.15.100:0/218414173":"2024-04-08T16:18:20.617755+0000","172.21.15.100:6801/2710911757":"2024-04-08T16:03:24.146096+0000","172.21.15.100:6800/3440287045":"2024-04-08T16:02:53.269514+0000","172.21.15.100:0/3316052690":"2024-04-08T16:20:16.558634+0000","172.21.15.100:0/4162615562":"2024-04-08T16:02:53.269514+0000","172.21.15.100:6801/3440287045":"2024-04-08T16:02:53.269514+0000","172.21.15.100:0/1466792171":"2024-04-08T16:20:16.558634+0000","172.21.15.100:0/48519977":"2024-04-08T16:20:16.558634+0000","172.21.15.144:6832/226244023":"2024-04-08T16:26:19.573335+0000","172.21.15.100:0/2474331937":"2024-04-08T16:03:24.146096+0000","172.21.15.144:0/493790447":"2024-04-08T16:19:13.981948+0000","172.21.15.100:6801/3429225708":"2024-04-08T16:18:20.617755+0000","172.21.15.100:0/2159245581":"2024-04-08T16:20:16.558634+0000","172.21.15.100:6801/3273770395":"2024-04-08T16:20:16.558634+0000","172.21.15.100:0/639120693":"2024-04-08T16:03:24.146096+0000","172.21.15.144:0/3723623887":"2024-04-08T16:19:13.981948+0000","172.21.15.100:0/1472671368":"2024-04-08T16:03:24.146096+0000","172.21.15.144:6834/3360802682":"2024-04-08T16:19:13.981948+0000","172.21.15.100:0/1091704426":"2024-04-07T18:18:18.417894+0000","172.21.15.100:0/1309290877":"2024-04-08T16:12:52.577534+0000","172.21.15.100:0/1822703232":"2024-04-08T16:20:16.558634+0000","172.21.15.100:6800/3273770395":"2024-04-08T16:20:16.558634+0000","172.21.15.100:0/200721231":"2024-04-07T18:50:01.905701+0000","172.21.15.100:6800/190788518":"2024-04-08T16:12:52.577534+0000","172.21.15.100:0/367509160":"2024-04-08T16:20:16.558634+0000","172.21.15.100:0/2535336108":"2024-04-08T16:12:52.577534+0000","172.21.15.144:0/1989982380":"2024-04-08T16:19:13.981948+0000","172.21.15.100:0/109071285":"2024-04-08T16:02:53.269514+0000","172.21.15.100:0/353657034":"2024-04-08T16:18:20.617755+0000","172.21.15.144:0/115503816":"2024-04-08T16:19:13.981948+0000","172.21.15.100:0/74983397":"2024-04-08T16:12:52.577534+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"},"teuthologyprofile":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van","w":"8"}},"removed_snaps_queue":[{"pool":60,"snaps":[{"begin":2,"length":1}]},{"pool":63,"snaps":[{"begin":1,"length":2}]},{"pool":68,"snaps":[{"begin":1,"length":2}]},{"pool":77,"snaps":[{"begin":2,"length":1}]},{"pool":78,"snaps":[{"begin":2,"length":1}]},{"pool":79,"snaps":[{"begin":2,"length":1}]},{"pool":80,"snaps":[{"begin":2,"length":1}]},{"pool":81,"snaps":[{"begin":2,"length":1}]},{"pool":82,"snaps":[{"begin":2,"length":1}]},{"pool":83,"snaps":[{"begin":2,"length":1}]},{"pool":86,"snaps":[{"begin":3,"length":6}]},{"pool":88,"snaps":[{"begin":34,"length":4},{"begin":43,"length":2},{"begin":46,"length":2}]},{"pool":89,"snaps":[{"begin":17,"length":1},{"begin":22,"length":1}]},{"pool":90,"snaps":[{"begin":2,"length":1}]},{"pool":122,"snaps":[{"begin":293,"length":2}]},{"pool":123,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-07T17:51:31.967 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-07T17:51:32.308 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:32 smithi100 bash[74942]: audit 2024-04-07T17:51:31.271870+0000 mon.a (mon.0) 5000 : audit [DBG] from='client.? 172.21.15.100:0/3445509547' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-07T17:51:32.309 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:32 smithi100 bash[74942]: audit 2024-04-07T17:51:31.616025+0000 mon.c (mon.2) 141 : audit [DBG] from='client.? 172.21.15.100:0/367532962' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-07T17:51:32.309 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:32 smithi100 bash[74942]: audit 2024-04-07T17:51:31.897562+0000 mon.a (mon.0) 5001 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:51:32.309 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:32 smithi100 bash[74942]: audit 2024-04-07T17:51:31.899449+0000 mon.a (mon.0) 5002 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:51:32.309 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:32 smithi100 bash[74942]: audit 2024-04-07T17:51:31.942150+0000 mon.c (mon.2) 142 : audit [DBG] from='client.? 172.21.15.100:0/3513324741' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-07T17:51:32.309 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:32 smithi100 bash[74942]: debug 2024-04-07T17:51:32.306+0000 7f53b1d07700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-07T17:51:32.309 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:32 smithi100 bash[76202]: audit 2024-04-07T17:51:31.271870+0000 mon.a (mon.0) 5000 : audit [DBG] from='client.? 172.21.15.100:0/3445509547' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-07T17:51:32.309 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:32 smithi100 bash[76202]: audit 2024-04-07T17:51:31.616025+0000 mon.c (mon.2) 141 : audit [DBG] from='client.? 172.21.15.100:0/367532962' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-07T17:51:32.309 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:32 smithi100 bash[76202]: audit 2024-04-07T17:51:31.897562+0000 mon.a (mon.0) 5001 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:51:32.310 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:32 smithi100 bash[76202]: audit 2024-04-07T17:51:31.899449+0000 mon.a (mon.0) 5002 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:51:32.310 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:32 smithi100 bash[76202]: audit 2024-04-07T17:51:31.942150+0000 mon.c (mon.2) 142 : audit [DBG] from='client.? 172.21.15.100:0/3513324741' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-07T17:51:32.321 INFO:teuthology.orchestra.run.smithi100.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-07T17:51:32.341 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-07T17:51:32.453 INFO:journalctl@ceph.mds.a.smithi100.stdout:Apr 07 17:51:32 smithi100 bash[98288]: ignoring --setuser ceph since I am not root 2024-04-07T17:51:32.453 INFO:journalctl@ceph.mds.a.smithi100.stdout:Apr 07 17:51:32 smithi100 bash[98288]: ignoring --setgroup ceph since I am not root 2024-04-07T17:51:32.453 INFO:journalctl@ceph.mds.a.smithi100.stdout:Apr 07 17:51:32 smithi100 bash[98288]: starting mds.a at 2024-04-07T17:51:32.614 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:51:32 smithi144 bash[95233]: ts=2024-04-07T17:51:32.317Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:51:32.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:32 smithi144 bash[66654]: audit 2024-04-07T17:51:31.271870+0000 mon.a (mon.0) 5000 : audit [DBG] from='client.? 172.21.15.100:0/3445509547' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-07T17:51:32.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:32 smithi144 bash[66654]: audit 2024-04-07T17:51:31.616025+0000 mon.c (mon.2) 141 : audit [DBG] from='client.? 172.21.15.100:0/367532962' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-07T17:51:32.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:32 smithi144 bash[66654]: audit 2024-04-07T17:51:31.897562+0000 mon.a (mon.0) 5001 : audit [INF] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' 2024-04-07T17:51:32.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:32 smithi144 bash[66654]: audit 2024-04-07T17:51:31.899449+0000 mon.a (mon.0) 5002 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-07T17:51:32.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:32 smithi144 bash[66654]: audit 2024-04-07T17:51:31.942150+0000 mon.c (mon.2) 142 : audit [DBG] from='client.? 172.21.15.100:0/3513324741' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-07T17:51:33.342 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-07T17:51:33.453 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:33 smithi100 bash[76202]: cluster 2024-04-07T17:51:31.695177+0000 mgr.y (mgr.25175) 2903 : cluster [DBG] pgmap v5173: 105 pgs: 105 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 13 KiB/s rd, 0 B/s wr, 18 op/s 2024-04-07T17:51:33.453 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:33 smithi100 bash[76202]: audit 2024-04-07T17:51:32.306811+0000 mon.c (mon.2) 143 : audit [INF] from='client.? 172.21.15.100:0/2589318261' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-07T17:51:33.453 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:33 smithi100 bash[76202]: audit 2024-04-07T17:51:32.307384+0000 mon.a (mon.0) 5003 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-07T17:51:33.453 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:33 smithi100 bash[76202]: cluster 2024-04-07T17:51:32.308022+0000 mon.a (mon.0) 5004 : cluster [WRN] Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-07T17:51:33.453 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:33 smithi100 bash[76202]: cluster 2024-04-07T17:51:32.308050+0000 mon.a (mon.0) 5005 : cluster [ERR] Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-07T17:51:33.453 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:33 smithi100 bash[76202]: audit 2024-04-07T17:51:32.309861+0000 mon.a (mon.0) 5006 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-07T17:51:33.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:33 smithi100 bash[76202]: cluster 2024-04-07T17:51:32.314697+0000 mon.a (mon.0) 5007 : cluster [DBG] osdmap e2714: 8 total, 8 up, 8 in 2024-04-07T17:51:33.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:33 smithi100 bash[76202]: cluster 2024-04-07T17:51:32.315296+0000 mon.a (mon.0) 5008 : cluster [DBG] fsmap cephfs:0/1 1 up:standby, 1 failed 2024-04-07T17:51:33.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:33 smithi100 bash[76202]: audit 2024-04-07T17:51:32.664335+0000 mon.c (mon.2) 144 : audit [INF] from='client.? 172.21.15.100:0/1294733737' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:51:33.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:33 smithi100 bash[76202]: audit 2024-04-07T17:51:32.664780+0000 mon.a (mon.0) 5009 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:51:33.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:33 smithi100 bash[74942]: cluster 2024-04-07T17:51:31.695177+0000 mgr.y (mgr.25175) 2903 : cluster [DBG] pgmap v5173: 105 pgs: 105 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 13 KiB/s rd, 0 B/s wr, 18 op/s 2024-04-07T17:51:33.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:33 smithi100 bash[74942]: audit 2024-04-07T17:51:32.306811+0000 mon.c (mon.2) 143 : audit [INF] from='client.? 172.21.15.100:0/2589318261' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-07T17:51:33.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:33 smithi100 bash[74942]: audit 2024-04-07T17:51:32.307384+0000 mon.a (mon.0) 5003 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-07T17:51:33.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:33 smithi100 bash[74942]: cluster 2024-04-07T17:51:32.308022+0000 mon.a (mon.0) 5004 : cluster [WRN] Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-07T17:51:33.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:33 smithi100 bash[74942]: cluster 2024-04-07T17:51:32.308050+0000 mon.a (mon.0) 5005 : cluster [ERR] Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-07T17:51:33.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:33 smithi100 bash[74942]: audit 2024-04-07T17:51:32.309861+0000 mon.a (mon.0) 5006 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-07T17:51:33.455 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:33 smithi100 bash[74942]: cluster 2024-04-07T17:51:32.314697+0000 mon.a (mon.0) 5007 : cluster [DBG] osdmap e2714: 8 total, 8 up, 8 in 2024-04-07T17:51:33.455 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:33 smithi100 bash[74942]: cluster 2024-04-07T17:51:32.315296+0000 mon.a (mon.0) 5008 : cluster [DBG] fsmap cephfs:0/1 1 up:standby, 1 failed 2024-04-07T17:51:33.455 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:33 smithi100 bash[74942]: audit 2024-04-07T17:51:32.664335+0000 mon.c (mon.2) 144 : audit [INF] from='client.? 172.21.15.100:0/1294733737' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:51:33.455 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:33 smithi100 bash[74942]: audit 2024-04-07T17:51:32.664780+0000 mon.a (mon.0) 5009 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:51:33.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:33 smithi144 bash[66654]: cluster 2024-04-07T17:51:31.695177+0000 mgr.y (mgr.25175) 2903 : cluster [DBG] pgmap v5173: 105 pgs: 105 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 13 KiB/s rd, 0 B/s wr, 18 op/s 2024-04-07T17:51:33.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:33 smithi144 bash[66654]: audit 2024-04-07T17:51:32.306811+0000 mon.c (mon.2) 143 : audit [INF] from='client.? 172.21.15.100:0/2589318261' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-07T17:51:33.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:33 smithi144 bash[66654]: audit 2024-04-07T17:51:32.307384+0000 mon.a (mon.0) 5003 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-07T17:51:33.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:33 smithi144 bash[66654]: cluster 2024-04-07T17:51:32.308022+0000 mon.a (mon.0) 5004 : cluster [WRN] Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-07T17:51:33.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:33 smithi144 bash[66654]: cluster 2024-04-07T17:51:32.308050+0000 mon.a (mon.0) 5005 : cluster [ERR] Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-07T17:51:33.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:33 smithi144 bash[66654]: audit 2024-04-07T17:51:32.309861+0000 mon.a (mon.0) 5006 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-07T17:51:33.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:33 smithi144 bash[66654]: cluster 2024-04-07T17:51:32.314697+0000 mon.a (mon.0) 5007 : cluster [DBG] osdmap e2714: 8 total, 8 up, 8 in 2024-04-07T17:51:33.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:33 smithi144 bash[66654]: cluster 2024-04-07T17:51:32.315296+0000 mon.a (mon.0) 5008 : cluster [DBG] fsmap cephfs:0/1 1 up:standby, 1 failed 2024-04-07T17:51:33.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:33 smithi144 bash[66654]: audit 2024-04-07T17:51:32.664335+0000 mon.c (mon.2) 144 : audit [INF] from='client.? 172.21.15.100:0/1294733737' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:51:33.616 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:33 smithi144 bash[66654]: audit 2024-04-07T17:51:32.664780+0000 mon.a (mon.0) 5009 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-07T17:51:34.334 INFO:teuthology.orchestra.run.smithi100.stderr:pool 'cephfs_metadata' removed 2024-04-07T17:51:34.365 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-07T17:51:34.453 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:34 smithi100 bash[74942]: cluster 2024-04-07T17:51:33.310035+0000 mon.a (mon.0) 5010 : cluster [INF] Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-07T17:51:34.453 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:34 smithi100 bash[74942]: cluster 2024-04-07T17:51:33.310093+0000 mon.a (mon.0) 5011 : cluster [INF] Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-07T17:51:34.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:34 smithi100 bash[74942]: cluster 2024-04-07T17:51:33.310122+0000 mon.a (mon.0) 5012 : cluster [INF] Cluster is now healthy 2024-04-07T17:51:34.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:34 smithi100 bash[74942]: audit 2024-04-07T17:51:33.312162+0000 mon.a (mon.0) 5013 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-07T17:51:34.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:34 smithi100 bash[74942]: cluster 2024-04-07T17:51:33.318199+0000 mon.a (mon.0) 5014 : cluster [DBG] mds.? [v2:172.21.15.100:6834/3055802164,v1:172.21.15.100:6835/3055802164] up:boot 2024-04-07T17:51:34.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:34 smithi100 bash[74942]: cluster 2024-04-07T17:51:33.318364+0000 mon.a (mon.0) 5015 : cluster [DBG] fsmap 2 up:standby 2024-04-07T17:51:34.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:34 smithi100 bash[74942]: audit 2024-04-07T17:51:33.318834+0000 mon.a (mon.0) 5016 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-07T17:51:34.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:34 smithi100 bash[74942]: audit 2024-04-07T17:51:33.651556+0000 mon.c (mon.2) 145 : audit [INF] from='client.? 172.21.15.100:0/766011454' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-07T17:51:34.454 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:34 smithi100 bash[74942]: audit 2024-04-07T17:51:33.651988+0000 mon.a (mon.0) 5017 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-07T17:51:34.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:34 smithi100 bash[76202]: cluster 2024-04-07T17:51:33.310035+0000 mon.a (mon.0) 5010 : cluster [INF] Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-07T17:51:34.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:34 smithi100 bash[76202]: cluster 2024-04-07T17:51:33.310093+0000 mon.a (mon.0) 5011 : cluster [INF] Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-07T17:51:34.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:34 smithi100 bash[76202]: cluster 2024-04-07T17:51:33.310122+0000 mon.a (mon.0) 5012 : cluster [INF] Cluster is now healthy 2024-04-07T17:51:34.454 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:34 smithi100 bash[76202]: audit 2024-04-07T17:51:33.312162+0000 mon.a (mon.0) 5013 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-07T17:51:34.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:34 smithi100 bash[76202]: cluster 2024-04-07T17:51:33.318199+0000 mon.a (mon.0) 5014 : cluster [DBG] mds.? [v2:172.21.15.100:6834/3055802164,v1:172.21.15.100:6835/3055802164] up:boot 2024-04-07T17:51:34.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:34 smithi100 bash[76202]: cluster 2024-04-07T17:51:33.318364+0000 mon.a (mon.0) 5015 : cluster [DBG] fsmap 2 up:standby 2024-04-07T17:51:34.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:34 smithi100 bash[76202]: audit 2024-04-07T17:51:33.318834+0000 mon.a (mon.0) 5016 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-07T17:51:34.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:34 smithi100 bash[76202]: audit 2024-04-07T17:51:33.651556+0000 mon.c (mon.2) 145 : audit [INF] from='client.? 172.21.15.100:0/766011454' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-07T17:51:34.455 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:34 smithi100 bash[76202]: audit 2024-04-07T17:51:33.651988+0000 mon.a (mon.0) 5017 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-07T17:51:34.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:34 smithi144 bash[66654]: cluster 2024-04-07T17:51:33.310035+0000 mon.a (mon.0) 5010 : cluster [INF] Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-07T17:51:34.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:34 smithi144 bash[66654]: cluster 2024-04-07T17:51:33.310093+0000 mon.a (mon.0) 5011 : cluster [INF] Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-07T17:51:34.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:34 smithi144 bash[66654]: cluster 2024-04-07T17:51:33.310122+0000 mon.a (mon.0) 5012 : cluster [INF] Cluster is now healthy 2024-04-07T17:51:34.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:34 smithi144 bash[66654]: audit 2024-04-07T17:51:33.312162+0000 mon.a (mon.0) 5013 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-07T17:51:34.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:34 smithi144 bash[66654]: cluster 2024-04-07T17:51:33.318199+0000 mon.a (mon.0) 5014 : cluster [DBG] mds.? [v2:172.21.15.100:6834/3055802164,v1:172.21.15.100:6835/3055802164] up:boot 2024-04-07T17:51:34.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:34 smithi144 bash[66654]: cluster 2024-04-07T17:51:33.318364+0000 mon.a (mon.0) 5015 : cluster [DBG] fsmap 2 up:standby 2024-04-07T17:51:34.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:34 smithi144 bash[66654]: audit 2024-04-07T17:51:33.318834+0000 mon.a (mon.0) 5016 : audit [DBG] from='mgr.25175 172.21.15.100:0/2255651159' entity='mgr.y' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-07T17:51:34.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:34 smithi144 bash[66654]: audit 2024-04-07T17:51:33.651556+0000 mon.c (mon.2) 145 : audit [INF] from='client.? 172.21.15.100:0/766011454' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-07T17:51:34.615 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:34 smithi144 bash[66654]: audit 2024-04-07T17:51:33.651988+0000 mon.a (mon.0) 5017 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-07T17:51:35.337 INFO:teuthology.orchestra.run.smithi100.stderr:pool 'cephfs_data' removed 2024-04-07T17:51:35.371 DEBUG:teuthology.orchestra.run.smithi100:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-07T17:51:35.382 DEBUG:teuthology.orchestra.run.smithi144:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-07T17:51:35.395 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-04-07T17:51:35.396 DEBUG:teuthology.orchestra.run.smithi100:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-07T17:51:35.427 DEBUG:teuthology.orchestra.run.smithi144:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-07T17:51:35.442 INFO:tasks.cephadm:Stopping all daemons... 2024-04-07T17:51:35.442 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-04-07T17:51:35.442 DEBUG:teuthology.orchestra.run.smithi100:> sudo systemctl stop ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@mon.a 2024-04-07T17:51:35.595 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:35 smithi100 bash[74942]: cluster 2024-04-07T17:51:33.695743+0000 mgr.y (mgr.25175) 2904 : cluster [DBG] pgmap v5175: 105 pgs: 105 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 22 KiB/s rd, 0 B/s wr, 33 op/s 2024-04-07T17:51:35.595 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:35 smithi100 bash[74942]: audit 2024-04-07T17:51:34.323718+0000 mon.a (mon.0) 5018 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-07T17:51:35.595 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:35 smithi100 bash[74942]: cluster 2024-04-07T17:51:34.332114+0000 mon.a (mon.0) 5019 : cluster [DBG] osdmap e2715: 8 total, 8 up, 8 in 2024-04-07T17:51:35.595 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:35 smithi100 bash[74942]: audit 2024-04-07T17:51:34.676581+0000 mon.c (mon.2) 146 : audit [INF] from='client.? 172.21.15.100:0/1249574906' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-07T17:51:35.595 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:35 smithi100 bash[74942]: audit 2024-04-07T17:51:34.677114+0000 mon.a (mon.0) 5020 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-07T17:51:35.595 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:35 smithi100 systemd[1]: Stopping Ceph mon.a for 0702e3a8-f4f8-11ee-b649-cb9ed24678a4... 2024-04-07T17:51:35.595 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:35 smithi100 bash[76202]: cluster 2024-04-07T17:51:33.695743+0000 mgr.y (mgr.25175) 2904 : cluster [DBG] pgmap v5175: 105 pgs: 105 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 22 KiB/s rd, 0 B/s wr, 33 op/s 2024-04-07T17:51:35.596 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:35 smithi100 bash[76202]: audit 2024-04-07T17:51:34.323718+0000 mon.a (mon.0) 5018 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-07T17:51:35.596 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:35 smithi100 bash[76202]: cluster 2024-04-07T17:51:34.332114+0000 mon.a (mon.0) 5019 : cluster [DBG] osdmap e2715: 8 total, 8 up, 8 in 2024-04-07T17:51:35.596 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:35 smithi100 bash[76202]: audit 2024-04-07T17:51:34.676581+0000 mon.c (mon.2) 146 : audit [INF] from='client.? 172.21.15.100:0/1249574906' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-07T17:51:35.596 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:35 smithi100 bash[76202]: audit 2024-04-07T17:51:34.677114+0000 mon.a (mon.0) 5020 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-07T17:51:35.620 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:35 smithi100 bash[74942]: debug 2024-04-07T17:51:35.590+0000 7f53b7512700 -1 received signal: Terminated from /sbin/docker-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-stderr=true (PID: 1) UID: 0 2024-04-07T17:51:35.620 INFO:journalctl@ceph.mon.a.smithi100.stdout:Apr 07 17:51:35 smithi100 bash[74942]: debug 2024-04-07T17:51:35.590+0000 7f53b7512700 -1 mon.a@0(leader) e4 *** Got Signal Terminated *** 2024-04-07T17:51:35.620 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:51:35 smithi100 bash[68137]: [07/Apr/2024:17:51:35] ENGINE Bus STOPPING 2024-04-07T17:51:35.620 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:51:35 smithi100 bash[68137]: [07/Apr/2024:17:51:35] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-04-07T17:51:35.620 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:51:35 smithi100 bash[68137]: [07/Apr/2024:17:51:35] ENGINE Bus STOPPED 2024-04-07T17:51:35.620 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:51:35 smithi100 bash[68137]: [07/Apr/2024:17:51:35] ENGINE Bus STARTING 2024-04-07T17:51:35.644 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:35 smithi144 bash[66654]: cluster 2024-04-07T17:51:33.695743+0000 mgr.y (mgr.25175) 2904 : cluster [DBG] pgmap v5175: 105 pgs: 105 active+clean; 4.6 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 22 KiB/s rd, 0 B/s wr, 33 op/s 2024-04-07T17:51:35.645 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:35 smithi144 bash[66654]: audit 2024-04-07T17:51:34.323718+0000 mon.a (mon.0) 5018 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-07T17:51:35.645 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:35 smithi144 bash[66654]: cluster 2024-04-07T17:51:34.332114+0000 mon.a (mon.0) 5019 : cluster [DBG] osdmap e2715: 8 total, 8 up, 8 in 2024-04-07T17:51:35.645 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:35 smithi144 bash[66654]: audit 2024-04-07T17:51:34.676581+0000 mon.c (mon.2) 146 : audit [INF] from='client.? 172.21.15.100:0/1249574906' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-07T17:51:35.645 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:35 smithi144 bash[66654]: audit 2024-04-07T17:51:34.677114+0000 mon.a (mon.0) 5020 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-07T17:51:35.851 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:51:35 smithi100 bash[68137]: [07/Apr/2024:17:51:35] ENGINE Serving on http://:::9283 2024-04-07T17:51:35.851 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:51:35 smithi100 bash[68137]: [07/Apr/2024:17:51:35] ENGINE Bus STARTED 2024-04-07T17:51:37.070 DEBUG:teuthology.orchestra.run.smithi100:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@mon.a.service' 2024-04-07T17:51:37.102 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-07T17:51:37.103 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-04-07T17:51:37.103 INFO:tasks.cephadm.mon.b:Stopping mon.c... 2024-04-07T17:51:37.103 DEBUG:teuthology.orchestra.run.smithi100:> sudo systemctl stop ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@mon.c 2024-04-07T17:51:37.415 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:37 smithi100 systemd[1]: Stopping Ceph mon.c for 0702e3a8-f4f8-11ee-b649-cb9ed24678a4... 2024-04-07T17:51:37.415 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:37 smithi100 bash[76202]: debug 2024-04-07T17:51:37.278+0000 7fd7bbff9700 -1 received signal: Terminated from /sbin/docker-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-stderr=true (PID: 1) UID: 0 2024-04-07T17:51:37.415 INFO:journalctl@ceph.mon.c.smithi100.stdout:Apr 07 17:51:37 smithi100 bash[76202]: debug 2024-04-07T17:51:37.278+0000 7fd7bbff9700 -1 mon.c@2(peon) e4 *** Got Signal Terminated *** 2024-04-07T17:51:38.341 DEBUG:teuthology.orchestra.run.smithi100:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@mon.c.service' 2024-04-07T17:51:38.363 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-07T17:51:38.364 INFO:tasks.cephadm.mon.b:Stopped mon.c 2024-04-07T17:51:38.364 INFO:tasks.cephadm.mon.b:Stopping mon.b... 2024-04-07T17:51:38.364 DEBUG:teuthology.orchestra.run.smithi144:> sudo systemctl stop ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@mon.b 2024-04-07T17:51:38.703 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:51:38 smithi100 bash[68137]: ::ffff:172.21.15.144 - - [07/Apr/2024:17:51:38] "GET /metrics HTTP/1.1" 200 34265 "" "Prometheus/2.43.0" 2024-04-07T17:51:38.810 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:38 smithi144 systemd[1]: Stopping Ceph mon.b for 0702e3a8-f4f8-11ee-b649-cb9ed24678a4... 2024-04-07T17:51:38.810 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:38 smithi144 bash[66654]: debug 2024-04-07T17:51:38.574+0000 7fdc7e3d6700 -1 received signal: Terminated from /sbin/docker-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-stderr=true (PID: 1) UID: 0 2024-04-07T17:51:38.810 INFO:journalctl@ceph.mon.b.smithi144.stdout:Apr 07 17:51:38 smithi144 bash[66654]: debug 2024-04-07T17:51:38.574+0000 7fdc7e3d6700 -1 mon.b@1(peon) e4 *** Got Signal Terminated *** 2024-04-07T17:51:39.787 DEBUG:teuthology.orchestra.run.smithi144:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@mon.b.service' 2024-04-07T17:51:39.825 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-07T17:51:39.826 INFO:tasks.cephadm.mon.b:Stopped mon.b 2024-04-07T17:51:39.826 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-04-07T17:51:39.826 DEBUG:teuthology.orchestra.run.smithi100:> sudo systemctl stop ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@mgr.y 2024-04-07T17:51:40.203 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:51:39 smithi100 systemd[1]: Stopping Ceph mgr.y for 0702e3a8-f4f8-11ee-b649-cb9ed24678a4... 2024-04-07T17:51:40.203 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:51:39 smithi100 bash[68137]: debug 2024-04-07T17:51:39.970+0000 7f853ee22700 -1 received signal: Terminated from /sbin/docker-init -- /usr/bin/ceph-mgr -n mgr.y -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug (PID: 1) UID: 0 2024-04-07T17:51:40.203 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Apr 07 17:51:39 smithi100 bash[68137]: debug 2024-04-07T17:51:39.970+0000 7f853ee22700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-04-07T17:51:40.662 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Apr 07 17:51:40 smithi100 bash[105121]: ts=2024-04-07T17:51:40.245Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi100.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.100:8443: connect: connection refused" 2024-04-07T17:51:40.880 DEBUG:teuthology.orchestra.run.smithi100:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@mgr.y.service' 2024-04-07T17:51:40.903 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-07T17:51:40.903 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-04-07T17:51:40.903 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-04-07T17:51:40.903 DEBUG:teuthology.orchestra.run.smithi144:> sudo systemctl stop ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@mgr.x 2024-04-07T17:51:41.329 INFO:journalctl@ceph.mgr.x.smithi144.stdout:Apr 07 17:51:40 smithi144 systemd[1]: Stopping Ceph mgr.x for 0702e3a8-f4f8-11ee-b649-cb9ed24678a4... 2024-04-07T17:51:41.920 DEBUG:teuthology.orchestra.run.smithi144:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@mgr.x.service' 2024-04-07T17:51:41.943 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-07T17:51:41.943 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-04-07T17:51:41.943 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-04-07T17:51:41.943 DEBUG:teuthology.orchestra.run.smithi100:> sudo systemctl stop ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@osd.0 2024-04-07T17:51:42.203 INFO:journalctl@ceph.osd.0.smithi100.stdout:Apr 07 17:51:41 smithi100 systemd[1]: Stopping Ceph osd.0 for 0702e3a8-f4f8-11ee-b649-cb9ed24678a4... 2024-04-07T17:51:42.203 INFO:journalctl@ceph.osd.0.smithi100.stdout:Apr 07 17:51:42 smithi100 bash[80932]: debug 2024-04-07T17:51:42.086+0000 7f992e2db700 -1 received signal: Terminated from /sbin/docker-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug (PID: 1) UID: 0 2024-04-07T17:51:42.203 INFO:journalctl@ceph.osd.0.smithi100.stdout:Apr 07 17:51:42 smithi100 bash[80932]: debug 2024-04-07T17:51:42.086+0000 7f992e2db700 -1 osd.0 2716 *** Got signal Terminated *** 2024-04-07T17:51:42.203 INFO:journalctl@ceph.osd.0.smithi100.stdout:Apr 07 17:51:42 smithi100 bash[80932]: debug 2024-04-07T17:51:42.086+0000 7f992e2db700 -1 osd.0 2716 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-07T17:51:42.614 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:51:42 smithi144 bash[95233]: ts=2024-04-07T17:51:42.316Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi100:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"0702e3a8-f4f8-11ee-b649-cb9ed24678a4\", instance=\"smithi100:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-04-07T17:51:48.733 INFO:journalctl@ceph.osd.0.smithi100.stdout:Apr 07 17:51:48 smithi100 bash[174693]: ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4-osd-0 2024-04-07T17:51:48.733 INFO:journalctl@ceph.osd.0.smithi100.stdout:Apr 07 17:51:48 smithi100 bash[174759]: Error response from daemon: No such container: ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4-osd-0 2024-04-07T17:51:50.632 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Apr 07 17:51:50 smithi100 bash[105121]: ts=2024-04-07T17:51:50.245Z caller=dispatch.go:352 level=error component=dispatcher msg="Notify for alerts failed" num_alerts=1 err="ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://smithi100.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.100:8443: connect: connection refused" 2024-04-07T17:51:50.632 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Apr 07 17:51:50 smithi100 bash[105121]: ts=2024-04-07T17:51:50.246Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi100.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.100:8443: connect: connection refused" 2024-04-07T17:51:50.636 DEBUG:teuthology.orchestra.run.smithi100:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@osd.0.service' 2024-04-07T17:51:50.663 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-07T17:51:50.663 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-04-07T17:51:50.663 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-04-07T17:51:50.663 DEBUG:teuthology.orchestra.run.smithi100:> sudo systemctl stop ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@osd.1 2024-04-07T17:51:50.893 INFO:journalctl@ceph.osd.1.smithi100.stdout:Apr 07 17:51:50 smithi100 systemd[1]: Stopping Ceph osd.1 for 0702e3a8-f4f8-11ee-b649-cb9ed24678a4... 2024-04-07T17:51:51.203 INFO:journalctl@ceph.osd.1.smithi100.stdout:Apr 07 17:51:50 smithi100 bash[85238]: debug 2024-04-07T17:51:50.889+0000 7fbe92bc4700 -1 received signal: Terminated from /sbin/docker-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug (PID: 1) UID: 0 2024-04-07T17:51:51.203 INFO:journalctl@ceph.osd.1.smithi100.stdout:Apr 07 17:51:50 smithi100 bash[85238]: debug 2024-04-07T17:51:50.889+0000 7fbe92bc4700 -1 osd.1 2716 *** Got signal Terminated *** 2024-04-07T17:51:51.203 INFO:journalctl@ceph.osd.1.smithi100.stdout:Apr 07 17:51:50 smithi100 bash[85238]: debug 2024-04-07T17:51:50.889+0000 7fbe92bc4700 -1 osd.1 2716 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-07T17:51:57.457 INFO:journalctl@ceph.osd.1.smithi100.stdout:Apr 07 17:51:57 smithi100 bash[174893]: ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4-osd-1 2024-04-07T17:51:57.458 INFO:journalctl@ceph.osd.1.smithi100.stdout:Apr 07 17:51:57 smithi100 bash[174960]: Error response from daemon: No such container: ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4-osd-1 2024-04-07T17:51:59.086 DEBUG:teuthology.orchestra.run.smithi100:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@osd.1.service' 2024-04-07T17:51:59.114 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-07T17:51:59.114 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-04-07T17:51:59.114 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-04-07T17:51:59.114 DEBUG:teuthology.orchestra.run.smithi100:> sudo systemctl stop ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@osd.2 2024-04-07T17:51:59.453 INFO:journalctl@ceph.osd.2.smithi100.stdout:Apr 07 17:51:59 smithi100 systemd[1]: Stopping Ceph osd.2 for 0702e3a8-f4f8-11ee-b649-cb9ed24678a4... 2024-04-07T17:51:59.453 INFO:journalctl@ceph.osd.2.smithi100.stdout:Apr 07 17:51:59 smithi100 bash[89668]: debug 2024-04-07T17:51:59.293+0000 7f35311c3700 -1 received signal: Terminated from /sbin/docker-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug (PID: 1) UID: 0 2024-04-07T17:51:59.453 INFO:journalctl@ceph.osd.2.smithi100.stdout:Apr 07 17:51:59 smithi100 bash[89668]: debug 2024-04-07T17:51:59.293+0000 7f35311c3700 -1 osd.2 2716 *** Got signal Terminated *** 2024-04-07T17:51:59.453 INFO:journalctl@ceph.osd.2.smithi100.stdout:Apr 07 17:51:59 smithi100 bash[89668]: debug 2024-04-07T17:51:59.293+0000 7f35311c3700 -1 osd.2 2716 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-07T17:52:00.703 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Apr 07 17:52:00 smithi100 bash[105121]: ts=2024-04-07T17:52:00.246Z caller=dispatch.go:352 level=error component=dispatcher msg="Notify for alerts failed" num_alerts=1 err="ceph-dashboard/webhook[0]: notify retry canceled after 8 attempts: Post \"https://smithi100.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.100:8443: connect: connection refused" 2024-04-07T17:52:00.703 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Apr 07 17:52:00 smithi100 bash[105121]: ts=2024-04-07T17:52:00.247Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi100.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.100:8443: connect: connection refused" 2024-04-07T17:52:05.739 INFO:journalctl@ceph.osd.2.smithi100.stdout:Apr 07 17:52:05 smithi100 bash[175096]: ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4-osd-2 2024-04-07T17:52:05.739 INFO:journalctl@ceph.osd.2.smithi100.stdout:Apr 07 17:52:05 smithi100 bash[175160]: Error response from daemon: No such container: ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4-osd-2 2024-04-07T17:52:07.644 DEBUG:teuthology.orchestra.run.smithi100:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@osd.2.service' 2024-04-07T17:52:07.673 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-07T17:52:07.673 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-04-07T17:52:07.673 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-04-07T17:52:07.673 DEBUG:teuthology.orchestra.run.smithi100:> sudo systemctl stop ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@osd.3 2024-04-07T17:52:07.953 INFO:journalctl@ceph.osd.3.smithi100.stdout:Apr 07 17:52:07 smithi100 systemd[1]: Stopping Ceph osd.3 for 0702e3a8-f4f8-11ee-b649-cb9ed24678a4... 2024-04-07T17:52:07.953 INFO:journalctl@ceph.osd.3.smithi100.stdout:Apr 07 17:52:07 smithi100 bash[93998]: debug 2024-04-07T17:52:07.853+0000 7f6eadcc7700 -1 received signal: Terminated from /sbin/docker-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug (PID: 1) UID: 0 2024-04-07T17:52:07.953 INFO:journalctl@ceph.osd.3.smithi100.stdout:Apr 07 17:52:07 smithi100 bash[93998]: debug 2024-04-07T17:52:07.853+0000 7f6eadcc7700 -1 osd.3 2716 *** Got signal Terminated *** 2024-04-07T17:52:07.953 INFO:journalctl@ceph.osd.3.smithi100.stdout:Apr 07 17:52:07 smithi100 bash[93998]: debug 2024-04-07T17:52:07.853+0000 7f6eadcc7700 -1 osd.3 2716 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-07T17:52:09.614 INFO:journalctl@ceph.osd.5.smithi144.stdout:Apr 07 17:52:09 smithi144 bash[75374]: debug 2024-04-07T17:52:09.237+0000 7f4af7be3700 -1 osd.5 2716 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-04-07T17:51:46.166431+0000 front 2024-04-07T17:51:46.166680+0000 (oldest deadline 2024-04-07T17:52:08.466253+0000) 2024-04-07T17:52:10.114 INFO:journalctl@ceph.osd.4.smithi144.stdout:Apr 07 17:52:09 smithi144 bash[71221]: debug 2024-04-07T17:52:09.621+0000 7f17aff83700 -1 osd.4 2716 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-04-07T17:51:43.473864+0000 front 2024-04-07T17:51:43.473761+0000 (oldest deadline 2024-04-07T17:52:08.773844+0000) 2024-04-07T17:52:10.580 INFO:journalctl@ceph.osd.5.smithi144.stdout:Apr 07 17:52:10 smithi144 bash[75374]: debug 2024-04-07T17:52:10.221+0000 7f4af7be3700 -1 osd.5 2716 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-04-07T17:51:46.166431+0000 front 2024-04-07T17:51:46.166680+0000 (oldest deadline 2024-04-07T17:52:08.466253+0000) 2024-04-07T17:52:10.703 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Apr 07 17:52:10 smithi100 bash[105121]: ts=2024-04-07T17:52:10.246Z caller=dispatch.go:352 level=error component=dispatcher msg="Notify for alerts failed" num_alerts=1 err="ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://smithi100.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.100:8443: connect: connection refused" 2024-04-07T17:52:10.703 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Apr 07 17:52:10 smithi100 bash[105121]: ts=2024-04-07T17:52:10.247Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi100.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.100:8443: connect: connection refused" 2024-04-07T17:52:10.864 INFO:journalctl@ceph.osd.4.smithi144.stdout:Apr 07 17:52:10 smithi144 bash[71221]: debug 2024-04-07T17:52:10.581+0000 7f17aff83700 -1 osd.4 2716 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-04-07T17:51:43.473864+0000 front 2024-04-07T17:51:43.473761+0000 (oldest deadline 2024-04-07T17:52:08.773844+0000) 2024-04-07T17:52:11.542 INFO:journalctl@ceph.osd.5.smithi144.stdout:Apr 07 17:52:11 smithi144 bash[75374]: debug 2024-04-07T17:52:11.213+0000 7f4af7be3700 -1 osd.5 2716 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-04-07T17:51:46.166431+0000 front 2024-04-07T17:51:46.166680+0000 (oldest deadline 2024-04-07T17:52:08.466253+0000) 2024-04-07T17:52:11.864 INFO:journalctl@ceph.osd.4.smithi144.stdout:Apr 07 17:52:11 smithi144 bash[71221]: debug 2024-04-07T17:52:11.537+0000 7f17aff83700 -1 osd.4 2716 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-04-07T17:51:43.473864+0000 front 2024-04-07T17:51:43.473761+0000 (oldest deadline 2024-04-07T17:52:08.773844+0000) 2024-04-07T17:52:12.203 INFO:journalctl@ceph.osd.3.smithi100.stdout:Apr 07 17:52:11 smithi100 bash[93998]: debug 2024-04-07T17:52:11.793+0000 7f6ea62b3700 -1 osd.3 2716 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-04-07T17:51:45.332969+0000 front 2024-04-07T17:51:45.333322+0000 (oldest deadline 2024-04-07T17:52:11.233143+0000) 2024-04-07T17:52:12.614 INFO:journalctl@ceph.osd.4.smithi144.stdout:Apr 07 17:52:12 smithi144 bash[71221]: debug 2024-04-07T17:52:12.489+0000 7f17aff83700 -1 osd.4 2716 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-04-07T17:51:43.473864+0000 front 2024-04-07T17:51:43.473761+0000 (oldest deadline 2024-04-07T17:52:08.773844+0000) 2024-04-07T17:52:12.615 INFO:journalctl@ceph.osd.5.smithi144.stdout:Apr 07 17:52:12 smithi144 bash[75374]: debug 2024-04-07T17:52:12.261+0000 7f4af7be3700 -1 osd.5 2716 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-04-07T17:51:46.166431+0000 front 2024-04-07T17:51:46.166680+0000 (oldest deadline 2024-04-07T17:52:08.466253+0000) 2024-04-07T17:52:13.203 INFO:journalctl@ceph.osd.3.smithi100.stdout:Apr 07 17:52:12 smithi100 bash[93998]: debug 2024-04-07T17:52:12.805+0000 7f6ea62b3700 -1 osd.3 2716 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-04-07T17:51:45.332969+0000 front 2024-04-07T17:51:45.333322+0000 (oldest deadline 2024-04-07T17:52:11.233143+0000) 2024-04-07T17:52:13.248 INFO:journalctl@ceph.osd.7.smithi144.stdout:Apr 07 17:52:12 smithi144 bash[84215]: debug 2024-04-07T17:52:12.929+0000 7f786bcc4700 -1 osd.7 2716 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-04-07T17:51:46.828238+0000 front 2024-04-07T17:51:46.828037+0000 (oldest deadline 2024-04-07T17:52:12.727582+0000) 2024-04-07T17:52:13.573 INFO:journalctl@ceph.osd.4.smithi144.stdout:Apr 07 17:52:13 smithi144 bash[71221]: debug 2024-04-07T17:52:13.465+0000 7f17aff83700 -1 osd.4 2716 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-04-07T17:51:43.473864+0000 front 2024-04-07T17:51:43.473761+0000 (oldest deadline 2024-04-07T17:52:08.773844+0000) 2024-04-07T17:52:13.573 INFO:journalctl@ceph.osd.5.smithi144.stdout:Apr 07 17:52:13 smithi144 bash[75374]: debug 2024-04-07T17:52:13.245+0000 7f4af7be3700 -1 osd.5 2716 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-04-07T17:51:46.166431+0000 front 2024-04-07T17:51:46.166680+0000 (oldest deadline 2024-04-07T17:52:08.466253+0000) 2024-04-07T17:52:13.864 INFO:journalctl@ceph.osd.6.smithi144.stdout:Apr 07 17:52:13 smithi144 bash[79803]: debug 2024-04-07T17:52:13.569+0000 7f894d91f700 -1 osd.6 2716 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-04-07T17:51:46.955007+0000 front 2024-04-07T17:51:46.955258+0000 (oldest deadline 2024-04-07T17:52:12.854871+0000) 2024-04-07T17:52:14.276 INFO:journalctl@ceph.osd.7.smithi144.stdout:Apr 07 17:52:13 smithi144 bash[84215]: debug 2024-04-07T17:52:13.917+0000 7f786bcc4700 -1 osd.7 2716 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-04-07T17:51:46.828238+0000 front 2024-04-07T17:51:46.828037+0000 (oldest deadline 2024-04-07T17:52:12.727582+0000) 2024-04-07T17:52:14.455 INFO:journalctl@ceph.osd.3.smithi100.stdout:Apr 07 17:52:13 smithi100 bash[175290]: ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4-osd-3 2024-04-07T17:52:14.455 INFO:journalctl@ceph.osd.3.smithi100.stdout:Apr 07 17:52:14 smithi100 bash[175352]: Error response from daemon: No such container: ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4-osd-3 2024-04-07T17:52:14.615 INFO:journalctl@ceph.osd.4.smithi144.stdout:Apr 07 17:52:14 smithi144 bash[71221]: debug 2024-04-07T17:52:14.445+0000 7f17aff83700 -1 osd.4 2716 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-04-07T17:51:43.473864+0000 front 2024-04-07T17:51:43.473761+0000 (oldest deadline 2024-04-07T17:52:08.773844+0000) 2024-04-07T17:52:14.615 INFO:journalctl@ceph.osd.5.smithi144.stdout:Apr 07 17:52:14 smithi144 bash[75374]: debug 2024-04-07T17:52:14.277+0000 7f4af7be3700 -1 osd.5 2716 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-04-07T17:51:46.166431+0000 front 2024-04-07T17:51:46.166680+0000 (oldest deadline 2024-04-07T17:52:08.466253+0000) 2024-04-07T17:52:14.950 INFO:journalctl@ceph.osd.6.smithi144.stdout:Apr 07 17:52:14 smithi144 bash[79803]: debug 2024-04-07T17:52:14.617+0000 7f894d91f700 -1 osd.6 2716 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-04-07T17:51:46.955007+0000 front 2024-04-07T17:51:46.955258+0000 (oldest deadline 2024-04-07T17:52:12.854871+0000) 2024-04-07T17:52:15.314 INFO:journalctl@ceph.osd.7.smithi144.stdout:Apr 07 17:52:14 smithi144 bash[84215]: debug 2024-04-07T17:52:14.949+0000 7f786bcc4700 -1 osd.7 2716 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-04-07T17:51:46.828238+0000 front 2024-04-07T17:51:46.828037+0000 (oldest deadline 2024-04-07T17:52:12.727582+0000) 2024-04-07T17:52:15.614 INFO:journalctl@ceph.osd.5.smithi144.stdout:Apr 07 17:52:15 smithi144 bash[75374]: debug 2024-04-07T17:52:15.313+0000 7f4af7be3700 -1 osd.5 2716 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-04-07T17:51:46.166431+0000 front 2024-04-07T17:51:46.166680+0000 (oldest deadline 2024-04-07T17:52:08.466253+0000) 2024-04-07T17:52:15.614 INFO:journalctl@ceph.osd.4.smithi144.stdout:Apr 07 17:52:15 smithi144 bash[71221]: debug 2024-04-07T17:52:15.417+0000 7f17aff83700 -1 osd.4 2716 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-04-07T17:51:43.473864+0000 front 2024-04-07T17:51:43.473761+0000 (oldest deadline 2024-04-07T17:52:08.773844+0000) 2024-04-07T17:52:15.864 INFO:journalctl@ceph.osd.6.smithi144.stdout:Apr 07 17:52:15 smithi144 bash[79803]: debug 2024-04-07T17:52:15.609+0000 7f894d91f700 -1 osd.6 2716 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-04-07T17:51:46.955007+0000 front 2024-04-07T17:51:46.955258+0000 (oldest deadline 2024-04-07T17:52:12.854871+0000) 2024-04-07T17:52:15.934 DEBUG:teuthology.orchestra.run.smithi100:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@osd.3.service' 2024-04-07T17:52:15.960 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-07T17:52:15.960 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-04-07T17:52:15.960 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-04-07T17:52:15.960 DEBUG:teuthology.orchestra.run.smithi144:> sudo systemctl stop ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@osd.4 2024-04-07T17:52:16.277 INFO:journalctl@ceph.osd.7.smithi144.stdout:Apr 07 17:52:15 smithi144 bash[84215]: debug 2024-04-07T17:52:15.949+0000 7f786bcc4700 -1 osd.7 2716 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-04-07T17:51:46.828238+0000 front 2024-04-07T17:51:46.828037+0000 (oldest deadline 2024-04-07T17:52:12.727582+0000) 2024-04-07T17:52:16.277 INFO:journalctl@ceph.osd.4.smithi144.stdout:Apr 07 17:52:15 smithi144 systemd[1]: Stopping Ceph osd.4 for 0702e3a8-f4f8-11ee-b649-cb9ed24678a4... 2024-04-07T17:52:16.277 INFO:journalctl@ceph.osd.4.smithi144.stdout:Apr 07 17:52:16 smithi144 bash[71221]: debug 2024-04-07T17:52:16.109+0000 7f17b398f700 -1 received signal: Terminated from /sbin/docker-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug (PID: 1) UID: 0 2024-04-07T17:52:16.277 INFO:journalctl@ceph.osd.4.smithi144.stdout:Apr 07 17:52:16 smithi144 bash[71221]: debug 2024-04-07T17:52:16.109+0000 7f17b398f700 -1 osd.4 2716 *** Got signal Terminated *** 2024-04-07T17:52:16.277 INFO:journalctl@ceph.osd.4.smithi144.stdout:Apr 07 17:52:16 smithi144 bash[71221]: debug 2024-04-07T17:52:16.109+0000 7f17b398f700 -1 osd.4 2716 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-07T17:52:16.593 INFO:journalctl@ceph.osd.4.smithi144.stdout:Apr 07 17:52:16 smithi144 bash[71221]: debug 2024-04-07T17:52:16.433+0000 7f17aff83700 -1 osd.4 2716 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-04-07T17:51:43.473864+0000 front 2024-04-07T17:51:43.473761+0000 (oldest deadline 2024-04-07T17:52:08.773844+0000) 2024-04-07T17:52:16.593 INFO:journalctl@ceph.osd.5.smithi144.stdout:Apr 07 17:52:16 smithi144 bash[75374]: debug 2024-04-07T17:52:16.273+0000 7f4af7be3700 -1 osd.5 2716 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-04-07T17:51:46.166431+0000 front 2024-04-07T17:51:46.166680+0000 (oldest deadline 2024-04-07T17:52:08.466253+0000) 2024-04-07T17:52:16.864 INFO:journalctl@ceph.osd.6.smithi144.stdout:Apr 07 17:52:16 smithi144 bash[79803]: debug 2024-04-07T17:52:16.593+0000 7f894d91f700 -1 osd.6 2716 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-04-07T17:51:46.955007+0000 front 2024-04-07T17:51:46.955258+0000 (oldest deadline 2024-04-07T17:52:12.854871+0000) 2024-04-07T17:52:17.114 INFO:journalctl@ceph.osd.7.smithi144.stdout:Apr 07 17:52:16 smithi144 bash[84215]: debug 2024-04-07T17:52:16.937+0000 7f786bcc4700 -1 osd.7 2716 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-04-07T17:51:46.828238+0000 front 2024-04-07T17:51:46.828037+0000 (oldest deadline 2024-04-07T17:52:12.727582+0000) 2024-04-07T17:52:17.115 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:52:16 smithi144 bash[95233]: ts=2024-04-07T17:52:16.865Z caller=refresh.go:99 level=error component="discovery manager scrape" discovery=http config=node msg="Unable to refresh target groups" err="Get \"http://172.21.15.100:8765/sd/prometheus/sd-config?service=node-exporter\": dial tcp 172.21.15.100:8765: connect: connection refused" 2024-04-07T17:52:17.115 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:52:16 smithi144 bash[95233]: ts=2024-04-07T17:52:16.872Z caller=refresh.go:99 level=error component="discovery manager scrape" discovery=http config=nvmeof msg="Unable to refresh target groups" err="Get \"http://172.21.15.100:8765/sd/prometheus/sd-config?service=nvmeof\": dial tcp 172.21.15.100:8765: connect: connection refused" 2024-04-07T17:52:17.115 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:52:16 smithi144 bash[95233]: ts=2024-04-07T17:52:16.878Z caller=refresh.go:99 level=error component="discovery manager scrape" discovery=http config=ceph msg="Unable to refresh target groups" err="Get \"http://172.21.15.100:8765/sd/prometheus/sd-config?service=mgr-prometheus\": dial tcp 172.21.15.100:8765: connect: connection refused" 2024-04-07T17:52:17.115 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:52:16 smithi144 bash[95233]: ts=2024-04-07T17:52:16.878Z caller=refresh.go:99 level=error component="discovery manager notify" discovery=http config=config-0 msg="Unable to refresh target groups" err="Get \"http://172.21.15.100:8765/sd/prometheus/sd-config?service=alertmanager\": dial tcp 172.21.15.100:8765: connect: connection refused" 2024-04-07T17:52:17.115 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:52:16 smithi144 bash[95233]: ts=2024-04-07T17:52:16.878Z caller=refresh.go:99 level=error component="discovery manager scrape" discovery=http config=ceph-exporter msg="Unable to refresh target groups" err="Get \"http://172.21.15.100:8765/sd/prometheus/sd-config?service=ceph-exporter\": dial tcp 172.21.15.100:8765: connect: connection refused" 2024-04-07T17:52:17.547 INFO:journalctl@ceph.osd.4.smithi144.stdout:Apr 07 17:52:17 smithi144 bash[71221]: debug 2024-04-07T17:52:17.409+0000 7f17aff83700 -1 osd.4 2716 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-04-07T17:51:43.473864+0000 front 2024-04-07T17:51:43.473761+0000 (oldest deadline 2024-04-07T17:52:08.773844+0000) 2024-04-07T17:52:17.547 INFO:journalctl@ceph.osd.5.smithi144.stdout:Apr 07 17:52:17 smithi144 bash[75374]: debug 2024-04-07T17:52:17.289+0000 7f4af7be3700 -1 osd.5 2716 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-04-07T17:51:46.166431+0000 front 2024-04-07T17:51:46.166680+0000 (oldest deadline 2024-04-07T17:52:08.466253+0000) 2024-04-07T17:52:17.864 INFO:journalctl@ceph.osd.6.smithi144.stdout:Apr 07 17:52:17 smithi144 bash[79803]: debug 2024-04-07T17:52:17.545+0000 7f894d91f700 -1 osd.6 2716 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-04-07T17:51:46.955007+0000 front 2024-04-07T17:51:46.955258+0000 (oldest deadline 2024-04-07T17:52:12.854871+0000) 2024-04-07T17:52:18.319 INFO:journalctl@ceph.osd.7.smithi144.stdout:Apr 07 17:52:17 smithi144 bash[84215]: debug 2024-04-07T17:52:17.917+0000 7f786bcc4700 -1 osd.7 2716 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-04-07T17:51:46.828238+0000 front 2024-04-07T17:51:46.828037+0000 (oldest deadline 2024-04-07T17:52:12.727582+0000) 2024-04-07T17:52:18.614 INFO:journalctl@ceph.osd.4.smithi144.stdout:Apr 07 17:52:18 smithi144 bash[71221]: debug 2024-04-07T17:52:18.417+0000 7f17aff83700 -1 osd.4 2716 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-04-07T17:51:43.473864+0000 front 2024-04-07T17:51:43.473761+0000 (oldest deadline 2024-04-07T17:52:08.773844+0000) 2024-04-07T17:52:18.615 INFO:journalctl@ceph.osd.5.smithi144.stdout:Apr 07 17:52:18 smithi144 bash[75374]: debug 2024-04-07T17:52:18.317+0000 7f4af7be3700 -1 osd.5 2716 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-04-07T17:51:46.166431+0000 front 2024-04-07T17:51:46.166680+0000 (oldest deadline 2024-04-07T17:52:08.466253+0000) 2024-04-07T17:52:18.615 INFO:journalctl@ceph.osd.6.smithi144.stdout:Apr 07 17:52:18 smithi144 bash[79803]: debug 2024-04-07T17:52:18.541+0000 7f894d91f700 -1 osd.6 2716 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-04-07T17:51:46.955007+0000 front 2024-04-07T17:51:46.955258+0000 (oldest deadline 2024-04-07T17:52:12.854871+0000) 2024-04-07T17:52:18.615 INFO:journalctl@ceph.osd.6.smithi144.stdout:Apr 07 17:52:18 smithi144 bash[79803]: debug 2024-04-07T17:52:18.541+0000 7f894d91f700 -1 osd.6 2716 heartbeat_check: no reply from 172.21.15.100:6814 osd.1 since back 2024-04-07T17:51:52.855832+0000 front 2024-04-07T17:51:52.855974+0000 (oldest deadline 2024-04-07T17:52:17.555534+0000) 2024-04-07T17:52:19.279 INFO:journalctl@ceph.osd.7.smithi144.stdout:Apr 07 17:52:18 smithi144 bash[84215]: debug 2024-04-07T17:52:18.925+0000 7f786bcc4700 -1 osd.7 2716 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-04-07T17:51:46.828238+0000 front 2024-04-07T17:51:46.828037+0000 (oldest deadline 2024-04-07T17:52:12.727582+0000) 2024-04-07T17:52:19.280 INFO:journalctl@ceph.osd.7.smithi144.stdout:Apr 07 17:52:18 smithi144 bash[84215]: debug 2024-04-07T17:52:18.925+0000 7f786bcc4700 -1 osd.7 2716 heartbeat_check: no reply from 172.21.15.100:6814 osd.1 since back 2024-04-07T17:51:55.029147+0000 front 2024-04-07T17:51:55.029247+0000 (oldest deadline 2024-04-07T17:52:17.928736+0000) 2024-04-07T17:52:19.552 INFO:journalctl@ceph.osd.5.smithi144.stdout:Apr 07 17:52:19 smithi144 bash[75374]: debug 2024-04-07T17:52:19.277+0000 7f4af7be3700 -1 osd.5 2716 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-04-07T17:51:46.166431+0000 front 2024-04-07T17:51:46.166680+0000 (oldest deadline 2024-04-07T17:52:08.466253+0000) 2024-04-07T17:52:19.552 INFO:journalctl@ceph.osd.4.smithi144.stdout:Apr 07 17:52:19 smithi144 bash[71221]: debug 2024-04-07T17:52:19.453+0000 7f17aff83700 -1 osd.4 2716 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-04-07T17:51:43.473864+0000 front 2024-04-07T17:51:43.473761+0000 (oldest deadline 2024-04-07T17:52:08.773844+0000) 2024-04-07T17:52:19.864 INFO:journalctl@ceph.osd.6.smithi144.stdout:Apr 07 17:52:19 smithi144 bash[79803]: debug 2024-04-07T17:52:19.553+0000 7f894d91f700 -1 osd.6 2716 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-04-07T17:51:46.955007+0000 front 2024-04-07T17:51:46.955258+0000 (oldest deadline 2024-04-07T17:52:12.854871+0000) 2024-04-07T17:52:19.864 INFO:journalctl@ceph.osd.6.smithi144.stdout:Apr 07 17:52:19 smithi144 bash[79803]: debug 2024-04-07T17:52:19.553+0000 7f894d91f700 -1 osd.6 2716 heartbeat_check: no reply from 172.21.15.100:6814 osd.1 since back 2024-04-07T17:51:52.855832+0000 front 2024-04-07T17:51:52.855974+0000 (oldest deadline 2024-04-07T17:52:17.555534+0000) 2024-04-07T17:52:20.279 INFO:journalctl@ceph.osd.7.smithi144.stdout:Apr 07 17:52:19 smithi144 bash[84215]: debug 2024-04-07T17:52:19.929+0000 7f786bcc4700 -1 osd.7 2716 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-04-07T17:51:46.828238+0000 front 2024-04-07T17:51:46.828037+0000 (oldest deadline 2024-04-07T17:52:12.727582+0000) 2024-04-07T17:52:20.279 INFO:journalctl@ceph.osd.7.smithi144.stdout:Apr 07 17:52:19 smithi144 bash[84215]: debug 2024-04-07T17:52:19.929+0000 7f786bcc4700 -1 osd.7 2716 heartbeat_check: no reply from 172.21.15.100:6814 osd.1 since back 2024-04-07T17:51:55.029147+0000 front 2024-04-07T17:51:55.029247+0000 (oldest deadline 2024-04-07T17:52:17.928736+0000) 2024-04-07T17:52:20.582 INFO:journalctl@ceph.osd.4.smithi144.stdout:Apr 07 17:52:20 smithi144 bash[71221]: debug 2024-04-07T17:52:20.421+0000 7f17aff83700 -1 osd.4 2716 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-04-07T17:51:43.473864+0000 front 2024-04-07T17:51:43.473761+0000 (oldest deadline 2024-04-07T17:52:08.773844+0000) 2024-04-07T17:52:20.583 INFO:journalctl@ceph.osd.4.smithi144.stdout:Apr 07 17:52:20 smithi144 bash[71221]: debug 2024-04-07T17:52:20.421+0000 7f17aff83700 -1 osd.4 2716 heartbeat_check: no reply from 172.21.15.100:6814 osd.1 since back 2024-04-07T17:51:50.975969+0000 front 2024-04-07T17:51:50.975922+0000 (oldest deadline 2024-04-07T17:52:20.376600+0000) 2024-04-07T17:52:20.583 INFO:journalctl@ceph.osd.5.smithi144.stdout:Apr 07 17:52:20 smithi144 bash[75374]: debug 2024-04-07T17:52:20.277+0000 7f4af7be3700 -1 osd.5 2716 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-04-07T17:51:46.166431+0000 front 2024-04-07T17:51:46.166680+0000 (oldest deadline 2024-04-07T17:52:08.466253+0000) 2024-04-07T17:52:20.583 INFO:journalctl@ceph.osd.5.smithi144.stdout:Apr 07 17:52:20 smithi144 bash[75374]: debug 2024-04-07T17:52:20.277+0000 7f4af7be3700 -1 osd.5 2716 heartbeat_check: no reply from 172.21.15.100:6814 osd.1 since back 2024-04-07T17:51:51.269051+0000 front 2024-04-07T17:51:51.268528+0000 (oldest deadline 2024-04-07T17:52:20.069044+0000) 2024-04-07T17:52:20.703 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Apr 07 17:52:20 smithi100 bash[105121]: ts=2024-04-07T17:52:20.247Z caller=dispatch.go:352 level=error component=dispatcher msg="Notify for alerts failed" num_alerts=1 err="ceph-dashboard/webhook[0]: notify retry canceled after 8 attempts: Post \"https://smithi100.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.100:8443: connect: connection refused" 2024-04-07T17:52:20.703 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Apr 07 17:52:20 smithi100 bash[105121]: ts=2024-04-07T17:52:20.248Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi100.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.100:8443: connect: connection refused" 2024-04-07T17:52:20.864 INFO:journalctl@ceph.osd.6.smithi144.stdout:Apr 07 17:52:20 smithi144 bash[79803]: debug 2024-04-07T17:52:20.581+0000 7f894d91f700 -1 osd.6 2716 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-04-07T17:51:46.955007+0000 front 2024-04-07T17:51:46.955258+0000 (oldest deadline 2024-04-07T17:52:12.854871+0000) 2024-04-07T17:52:20.864 INFO:journalctl@ceph.osd.6.smithi144.stdout:Apr 07 17:52:20 smithi144 bash[79803]: debug 2024-04-07T17:52:20.581+0000 7f894d91f700 -1 osd.6 2716 heartbeat_check: no reply from 172.21.15.100:6814 osd.1 since back 2024-04-07T17:51:52.855832+0000 front 2024-04-07T17:51:52.855974+0000 (oldest deadline 2024-04-07T17:52:17.555534+0000) 2024-04-07T17:52:21.321 INFO:journalctl@ceph.osd.7.smithi144.stdout:Apr 07 17:52:20 smithi144 bash[84215]: debug 2024-04-07T17:52:20.881+0000 7f786bcc4700 -1 osd.7 2716 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-04-07T17:51:46.828238+0000 front 2024-04-07T17:51:46.828037+0000 (oldest deadline 2024-04-07T17:52:12.727582+0000) 2024-04-07T17:52:21.321 INFO:journalctl@ceph.osd.7.smithi144.stdout:Apr 07 17:52:20 smithi144 bash[84215]: debug 2024-04-07T17:52:20.881+0000 7f786bcc4700 -1 osd.7 2716 heartbeat_check: no reply from 172.21.15.100:6814 osd.1 since back 2024-04-07T17:51:55.029147+0000 front 2024-04-07T17:51:55.029247+0000 (oldest deadline 2024-04-07T17:52:17.928736+0000) 2024-04-07T17:52:21.614 INFO:journalctl@ceph.osd.5.smithi144.stdout:Apr 07 17:52:21 smithi144 bash[75374]: debug 2024-04-07T17:52:21.321+0000 7f4af7be3700 -1 osd.5 2716 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-04-07T17:51:46.166431+0000 front 2024-04-07T17:51:46.166680+0000 (oldest deadline 2024-04-07T17:52:08.466253+0000) 2024-04-07T17:52:21.614 INFO:journalctl@ceph.osd.5.smithi144.stdout:Apr 07 17:52:21 smithi144 bash[75374]: debug 2024-04-07T17:52:21.321+0000 7f4af7be3700 -1 osd.5 2716 heartbeat_check: no reply from 172.21.15.100:6814 osd.1 since back 2024-04-07T17:51:51.269051+0000 front 2024-04-07T17:51:51.268528+0000 (oldest deadline 2024-04-07T17:52:20.069044+0000) 2024-04-07T17:52:21.615 INFO:journalctl@ceph.osd.6.smithi144.stdout:Apr 07 17:52:21 smithi144 bash[79803]: debug 2024-04-07T17:52:21.553+0000 7f894d91f700 -1 osd.6 2716 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-04-07T17:51:46.955007+0000 front 2024-04-07T17:51:46.955258+0000 (oldest deadline 2024-04-07T17:52:12.854871+0000) 2024-04-07T17:52:21.615 INFO:journalctl@ceph.osd.6.smithi144.stdout:Apr 07 17:52:21 smithi144 bash[79803]: debug 2024-04-07T17:52:21.553+0000 7f894d91f700 -1 osd.6 2716 heartbeat_check: no reply from 172.21.15.100:6814 osd.1 since back 2024-04-07T17:51:52.855832+0000 front 2024-04-07T17:51:52.855974+0000 (oldest deadline 2024-04-07T17:52:17.555534+0000) 2024-04-07T17:52:22.114 INFO:journalctl@ceph.osd.7.smithi144.stdout:Apr 07 17:52:21 smithi144 bash[84215]: debug 2024-04-07T17:52:21.869+0000 7f786bcc4700 -1 osd.7 2716 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-04-07T17:51:46.828238+0000 front 2024-04-07T17:51:46.828037+0000 (oldest deadline 2024-04-07T17:52:12.727582+0000) 2024-04-07T17:52:22.114 INFO:journalctl@ceph.osd.7.smithi144.stdout:Apr 07 17:52:21 smithi144 bash[84215]: debug 2024-04-07T17:52:21.869+0000 7f786bcc4700 -1 osd.7 2716 heartbeat_check: no reply from 172.21.15.100:6814 osd.1 since back 2024-04-07T17:51:55.029147+0000 front 2024-04-07T17:51:55.029247+0000 (oldest deadline 2024-04-07T17:52:17.928736+0000) 2024-04-07T17:52:22.553 INFO:journalctl@ceph.osd.4.smithi144.stdout:Apr 07 17:52:22 smithi144 bash[113802]: ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4-osd-4 2024-04-07T17:52:22.553 INFO:journalctl@ceph.osd.4.smithi144.stdout:Apr 07 17:52:22 smithi144 bash[113868]: Error response from daemon: No such container: ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4-osd-4 2024-04-07T17:52:22.553 INFO:journalctl@ceph.osd.5.smithi144.stdout:Apr 07 17:52:22 smithi144 bash[75374]: debug 2024-04-07T17:52:22.305+0000 7f4af7be3700 -1 osd.5 2716 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-04-07T17:51:46.166431+0000 front 2024-04-07T17:51:46.166680+0000 (oldest deadline 2024-04-07T17:52:08.466253+0000) 2024-04-07T17:52:22.553 INFO:journalctl@ceph.osd.5.smithi144.stdout:Apr 07 17:52:22 smithi144 bash[75374]: debug 2024-04-07T17:52:22.305+0000 7f4af7be3700 -1 osd.5 2716 heartbeat_check: no reply from 172.21.15.100:6814 osd.1 since back 2024-04-07T17:51:51.269051+0000 front 2024-04-07T17:51:51.268528+0000 (oldest deadline 2024-04-07T17:52:20.069044+0000) 2024-04-07T17:52:22.831 INFO:journalctl@ceph.osd.6.smithi144.stdout:Apr 07 17:52:22 smithi144 bash[79803]: debug 2024-04-07T17:52:22.553+0000 7f894d91f700 -1 osd.6 2716 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-04-07T17:51:46.955007+0000 front 2024-04-07T17:51:46.955258+0000 (oldest deadline 2024-04-07T17:52:12.854871+0000) 2024-04-07T17:52:22.831 INFO:journalctl@ceph.osd.6.smithi144.stdout:Apr 07 17:52:22 smithi144 bash[79803]: debug 2024-04-07T17:52:22.553+0000 7f894d91f700 -1 osd.6 2716 heartbeat_check: no reply from 172.21.15.100:6814 osd.1 since back 2024-04-07T17:51:52.855832+0000 front 2024-04-07T17:51:52.855974+0000 (oldest deadline 2024-04-07T17:52:17.555534+0000) 2024-04-07T17:52:23.114 INFO:journalctl@ceph.osd.7.smithi144.stdout:Apr 07 17:52:22 smithi144 bash[84215]: debug 2024-04-07T17:52:22.829+0000 7f786bcc4700 -1 osd.7 2716 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-04-07T17:51:46.828238+0000 front 2024-04-07T17:51:46.828037+0000 (oldest deadline 2024-04-07T17:52:12.727582+0000) 2024-04-07T17:52:23.114 INFO:journalctl@ceph.osd.7.smithi144.stdout:Apr 07 17:52:22 smithi144 bash[84215]: debug 2024-04-07T17:52:22.829+0000 7f786bcc4700 -1 osd.7 2716 heartbeat_check: no reply from 172.21.15.100:6814 osd.1 since back 2024-04-07T17:51:55.029147+0000 front 2024-04-07T17:51:55.029247+0000 (oldest deadline 2024-04-07T17:52:17.928736+0000) 2024-04-07T17:52:23.520 INFO:journalctl@ceph.osd.5.smithi144.stdout:Apr 07 17:52:23 smithi144 bash[75374]: debug 2024-04-07T17:52:23.265+0000 7f4af7be3700 -1 osd.5 2716 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-04-07T17:51:46.166431+0000 front 2024-04-07T17:51:46.166680+0000 (oldest deadline 2024-04-07T17:52:08.466253+0000) 2024-04-07T17:52:23.520 INFO:journalctl@ceph.osd.5.smithi144.stdout:Apr 07 17:52:23 smithi144 bash[75374]: debug 2024-04-07T17:52:23.265+0000 7f4af7be3700 -1 osd.5 2716 heartbeat_check: no reply from 172.21.15.100:6814 osd.1 since back 2024-04-07T17:51:51.269051+0000 front 2024-04-07T17:51:51.268528+0000 (oldest deadline 2024-04-07T17:52:20.069044+0000) 2024-04-07T17:52:23.776 INFO:journalctl@ceph.osd.6.smithi144.stdout:Apr 07 17:52:23 smithi144 bash[79803]: debug 2024-04-07T17:52:23.517+0000 7f894d91f700 -1 osd.6 2716 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-04-07T17:51:46.955007+0000 front 2024-04-07T17:51:46.955258+0000 (oldest deadline 2024-04-07T17:52:12.854871+0000) 2024-04-07T17:52:23.776 INFO:journalctl@ceph.osd.6.smithi144.stdout:Apr 07 17:52:23 smithi144 bash[79803]: debug 2024-04-07T17:52:23.517+0000 7f894d91f700 -1 osd.6 2716 heartbeat_check: no reply from 172.21.15.100:6814 osd.1 since back 2024-04-07T17:51:52.855832+0000 front 2024-04-07T17:51:52.855974+0000 (oldest deadline 2024-04-07T17:52:17.555534+0000) 2024-04-07T17:52:24.114 INFO:journalctl@ceph.osd.7.smithi144.stdout:Apr 07 17:52:23 smithi144 bash[84215]: debug 2024-04-07T17:52:23.841+0000 7f786bcc4700 -1 osd.7 2716 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-04-07T17:51:46.828238+0000 front 2024-04-07T17:51:46.828037+0000 (oldest deadline 2024-04-07T17:52:12.727582+0000) 2024-04-07T17:52:24.115 INFO:journalctl@ceph.osd.7.smithi144.stdout:Apr 07 17:52:23 smithi144 bash[84215]: debug 2024-04-07T17:52:23.841+0000 7f786bcc4700 -1 osd.7 2716 heartbeat_check: no reply from 172.21.15.100:6814 osd.1 since back 2024-04-07T17:51:55.029147+0000 front 2024-04-07T17:51:55.029247+0000 (oldest deadline 2024-04-07T17:52:17.928736+0000) 2024-04-07T17:52:24.475 DEBUG:teuthology.orchestra.run.smithi144:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@osd.4.service' 2024-04-07T17:52:24.501 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-07T17:52:24.502 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-04-07T17:52:24.502 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-04-07T17:52:24.502 DEBUG:teuthology.orchestra.run.smithi144:> sudo systemctl stop ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@osd.5 2024-04-07T17:52:24.548 INFO:journalctl@ceph.osd.5.smithi144.stdout:Apr 07 17:52:24 smithi144 bash[75374]: debug 2024-04-07T17:52:24.281+0000 7f4af7be3700 -1 osd.5 2716 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-04-07T17:51:46.166431+0000 front 2024-04-07T17:51:46.166680+0000 (oldest deadline 2024-04-07T17:52:08.466253+0000) 2024-04-07T17:52:24.548 INFO:journalctl@ceph.osd.5.smithi144.stdout:Apr 07 17:52:24 smithi144 bash[75374]: debug 2024-04-07T17:52:24.281+0000 7f4af7be3700 -1 osd.5 2716 heartbeat_check: no reply from 172.21.15.100:6814 osd.1 since back 2024-04-07T17:51:51.269051+0000 front 2024-04-07T17:51:51.268528+0000 (oldest deadline 2024-04-07T17:52:20.069044+0000) 2024-04-07T17:52:24.797 INFO:journalctl@ceph.osd.6.smithi144.stdout:Apr 07 17:52:24 smithi144 bash[79803]: debug 2024-04-07T17:52:24.545+0000 7f894d91f700 -1 osd.6 2716 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-04-07T17:51:46.955007+0000 front 2024-04-07T17:51:46.955258+0000 (oldest deadline 2024-04-07T17:52:12.854871+0000) 2024-04-07T17:52:24.797 INFO:journalctl@ceph.osd.6.smithi144.stdout:Apr 07 17:52:24 smithi144 bash[79803]: debug 2024-04-07T17:52:24.545+0000 7f894d91f700 -1 osd.6 2716 heartbeat_check: no reply from 172.21.15.100:6814 osd.1 since back 2024-04-07T17:51:52.855832+0000 front 2024-04-07T17:51:52.855974+0000 (oldest deadline 2024-04-07T17:52:17.555534+0000) 2024-04-07T17:52:24.797 INFO:journalctl@ceph.osd.5.smithi144.stdout:Apr 07 17:52:24 smithi144 systemd[1]: Stopping Ceph osd.5 for 0702e3a8-f4f8-11ee-b649-cb9ed24678a4... 2024-04-07T17:52:24.797 INFO:journalctl@ceph.osd.5.smithi144.stdout:Apr 07 17:52:24 smithi144 bash[75374]: debug 2024-04-07T17:52:24.653+0000 7f4aff5f7700 -1 received signal: Terminated from /sbin/docker-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug (PID: 1) UID: 0 2024-04-07T17:52:24.797 INFO:journalctl@ceph.osd.5.smithi144.stdout:Apr 07 17:52:24 smithi144 bash[75374]: debug 2024-04-07T17:52:24.653+0000 7f4aff5f7700 -1 osd.5 2716 *** Got signal Terminated *** 2024-04-07T17:52:24.797 INFO:journalctl@ceph.osd.5.smithi144.stdout:Apr 07 17:52:24 smithi144 bash[75374]: debug 2024-04-07T17:52:24.653+0000 7f4aff5f7700 -1 osd.5 2716 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-07T17:52:24.864 INFO:journalctl@ceph.osd.7.smithi144.stdout:Apr 07 17:52:24 smithi144 bash[84215]: debug 2024-04-07T17:52:24.793+0000 7f786bcc4700 -1 osd.7 2716 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-04-07T17:51:46.828238+0000 front 2024-04-07T17:51:46.828037+0000 (oldest deadline 2024-04-07T17:52:12.727582+0000) 2024-04-07T17:52:24.864 INFO:journalctl@ceph.osd.7.smithi144.stdout:Apr 07 17:52:24 smithi144 bash[84215]: debug 2024-04-07T17:52:24.793+0000 7f786bcc4700 -1 osd.7 2716 heartbeat_check: no reply from 172.21.15.100:6814 osd.1 since back 2024-04-07T17:51:55.029147+0000 front 2024-04-07T17:51:55.029247+0000 (oldest deadline 2024-04-07T17:52:17.928736+0000) 2024-04-07T17:52:25.614 INFO:journalctl@ceph.osd.5.smithi144.stdout:Apr 07 17:52:25 smithi144 bash[75374]: debug 2024-04-07T17:52:25.297+0000 7f4af7be3700 -1 osd.5 2716 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-04-07T17:51:46.166431+0000 front 2024-04-07T17:51:46.166680+0000 (oldest deadline 2024-04-07T17:52:08.466253+0000) 2024-04-07T17:52:25.615 INFO:journalctl@ceph.osd.5.smithi144.stdout:Apr 07 17:52:25 smithi144 bash[75374]: debug 2024-04-07T17:52:25.297+0000 7f4af7be3700 -1 osd.5 2716 heartbeat_check: no reply from 172.21.15.100:6814 osd.1 since back 2024-04-07T17:51:51.269051+0000 front 2024-04-07T17:51:51.268528+0000 (oldest deadline 2024-04-07T17:52:20.069044+0000) 2024-04-07T17:52:25.615 INFO:journalctl@ceph.osd.6.smithi144.stdout:Apr 07 17:52:25 smithi144 bash[79803]: debug 2024-04-07T17:52:25.513+0000 7f894d91f700 -1 osd.6 2716 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-04-07T17:51:46.955007+0000 front 2024-04-07T17:51:46.955258+0000 (oldest deadline 2024-04-07T17:52:12.854871+0000) 2024-04-07T17:52:25.615 INFO:journalctl@ceph.osd.6.smithi144.stdout:Apr 07 17:52:25 smithi144 bash[79803]: debug 2024-04-07T17:52:25.513+0000 7f894d91f700 -1 osd.6 2716 heartbeat_check: no reply from 172.21.15.100:6814 osd.1 since back 2024-04-07T17:51:52.855832+0000 front 2024-04-07T17:51:52.855974+0000 (oldest deadline 2024-04-07T17:52:17.555534+0000) 2024-04-07T17:52:26.114 INFO:journalctl@ceph.osd.7.smithi144.stdout:Apr 07 17:52:25 smithi144 bash[84215]: debug 2024-04-07T17:52:25.773+0000 7f786bcc4700 -1 osd.7 2716 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-04-07T17:51:46.828238+0000 front 2024-04-07T17:51:46.828037+0000 (oldest deadline 2024-04-07T17:52:12.727582+0000) 2024-04-07T17:52:26.115 INFO:journalctl@ceph.osd.7.smithi144.stdout:Apr 07 17:52:25 smithi144 bash[84215]: debug 2024-04-07T17:52:25.773+0000 7f786bcc4700 -1 osd.7 2716 heartbeat_check: no reply from 172.21.15.100:6814 osd.1 since back 2024-04-07T17:51:55.029147+0000 front 2024-04-07T17:51:55.029247+0000 (oldest deadline 2024-04-07T17:52:17.928736+0000) 2024-04-07T17:52:26.615 INFO:journalctl@ceph.osd.5.smithi144.stdout:Apr 07 17:52:26 smithi144 bash[75374]: debug 2024-04-07T17:52:26.257+0000 7f4af7be3700 -1 osd.5 2716 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-04-07T17:51:46.166431+0000 front 2024-04-07T17:51:46.166680+0000 (oldest deadline 2024-04-07T17:52:08.466253+0000) 2024-04-07T17:52:26.615 INFO:journalctl@ceph.osd.5.smithi144.stdout:Apr 07 17:52:26 smithi144 bash[75374]: debug 2024-04-07T17:52:26.257+0000 7f4af7be3700 -1 osd.5 2716 heartbeat_check: no reply from 172.21.15.100:6814 osd.1 since back 2024-04-07T17:51:51.269051+0000 front 2024-04-07T17:51:51.268528+0000 (oldest deadline 2024-04-07T17:52:20.069044+0000) 2024-04-07T17:52:26.615 INFO:journalctl@ceph.osd.6.smithi144.stdout:Apr 07 17:52:26 smithi144 bash[79803]: debug 2024-04-07T17:52:26.489+0000 7f894d91f700 -1 osd.6 2716 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-04-07T17:51:46.955007+0000 front 2024-04-07T17:51:46.955258+0000 (oldest deadline 2024-04-07T17:52:12.854871+0000) 2024-04-07T17:52:26.615 INFO:journalctl@ceph.osd.6.smithi144.stdout:Apr 07 17:52:26 smithi144 bash[79803]: debug 2024-04-07T17:52:26.489+0000 7f894d91f700 -1 osd.6 2716 heartbeat_check: no reply from 172.21.15.100:6814 osd.1 since back 2024-04-07T17:51:52.855832+0000 front 2024-04-07T17:51:52.855974+0000 (oldest deadline 2024-04-07T17:52:17.555534+0000) 2024-04-07T17:52:26.615 INFO:journalctl@ceph.osd.6.smithi144.stdout:Apr 07 17:52:26 smithi144 bash[79803]: debug 2024-04-07T17:52:26.489+0000 7f894d91f700 -1 osd.6 2716 heartbeat_check: no reply from 172.21.15.100:6822 osd.2 since back 2024-04-07T17:52:02.757914+0000 front 2024-04-07T17:52:02.757770+0000 (oldest deadline 2024-04-07T17:52:26.257434+0000) 2024-04-07T17:52:27.114 INFO:journalctl@ceph.osd.7.smithi144.stdout:Apr 07 17:52:26 smithi144 bash[84215]: debug 2024-04-07T17:52:26.733+0000 7f786bcc4700 -1 osd.7 2716 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-04-07T17:51:46.828238+0000 front 2024-04-07T17:51:46.828037+0000 (oldest deadline 2024-04-07T17:52:12.727582+0000) 2024-04-07T17:52:27.115 INFO:journalctl@ceph.osd.7.smithi144.stdout:Apr 07 17:52:26 smithi144 bash[84215]: debug 2024-04-07T17:52:26.733+0000 7f786bcc4700 -1 osd.7 2716 heartbeat_check: no reply from 172.21.15.100:6814 osd.1 since back 2024-04-07T17:51:55.029147+0000 front 2024-04-07T17:51:55.029247+0000 (oldest deadline 2024-04-07T17:52:17.928736+0000) 2024-04-07T17:52:27.614 INFO:journalctl@ceph.osd.5.smithi144.stdout:Apr 07 17:52:27 smithi144 bash[75374]: debug 2024-04-07T17:52:27.277+0000 7f4af7be3700 -1 osd.5 2716 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-04-07T17:51:46.166431+0000 front 2024-04-07T17:51:46.166680+0000 (oldest deadline 2024-04-07T17:52:08.466253+0000) 2024-04-07T17:52:27.615 INFO:journalctl@ceph.osd.5.smithi144.stdout:Apr 07 17:52:27 smithi144 bash[75374]: debug 2024-04-07T17:52:27.277+0000 7f4af7be3700 -1 osd.5 2716 heartbeat_check: no reply from 172.21.15.100:6814 osd.1 since back 2024-04-07T17:51:51.269051+0000 front 2024-04-07T17:51:51.268528+0000 (oldest deadline 2024-04-07T17:52:20.069044+0000) 2024-04-07T17:52:27.615 INFO:journalctl@ceph.osd.6.smithi144.stdout:Apr 07 17:52:27 smithi144 bash[79803]: debug 2024-04-07T17:52:27.497+0000 7f894d91f700 -1 osd.6 2716 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-04-07T17:51:46.955007+0000 front 2024-04-07T17:51:46.955258+0000 (oldest deadline 2024-04-07T17:52:12.854871+0000) 2024-04-07T17:52:27.615 INFO:journalctl@ceph.osd.6.smithi144.stdout:Apr 07 17:52:27 smithi144 bash[79803]: debug 2024-04-07T17:52:27.497+0000 7f894d91f700 -1 osd.6 2716 heartbeat_check: no reply from 172.21.15.100:6814 osd.1 since back 2024-04-07T17:51:52.855832+0000 front 2024-04-07T17:51:52.855974+0000 (oldest deadline 2024-04-07T17:52:17.555534+0000) 2024-04-07T17:52:27.615 INFO:journalctl@ceph.osd.6.smithi144.stdout:Apr 07 17:52:27 smithi144 bash[79803]: debug 2024-04-07T17:52:27.497+0000 7f894d91f700 -1 osd.6 2716 heartbeat_check: no reply from 172.21.15.100:6822 osd.2 since back 2024-04-07T17:52:02.757914+0000 front 2024-04-07T17:52:02.757770+0000 (oldest deadline 2024-04-07T17:52:26.257434+0000) 2024-04-07T17:52:28.114 INFO:journalctl@ceph.osd.7.smithi144.stdout:Apr 07 17:52:27 smithi144 bash[84215]: debug 2024-04-07T17:52:27.785+0000 7f786bcc4700 -1 osd.7 2716 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-04-07T17:51:46.828238+0000 front 2024-04-07T17:51:46.828037+0000 (oldest deadline 2024-04-07T17:52:12.727582+0000) 2024-04-07T17:52:28.114 INFO:journalctl@ceph.osd.7.smithi144.stdout:Apr 07 17:52:27 smithi144 bash[84215]: debug 2024-04-07T17:52:27.785+0000 7f786bcc4700 -1 osd.7 2716 heartbeat_check: no reply from 172.21.15.100:6814 osd.1 since back 2024-04-07T17:51:55.029147+0000 front 2024-04-07T17:51:55.029247+0000 (oldest deadline 2024-04-07T17:52:17.928736+0000) 2024-04-07T17:52:28.614 INFO:journalctl@ceph.osd.5.smithi144.stdout:Apr 07 17:52:28 smithi144 bash[75374]: debug 2024-04-07T17:52:28.305+0000 7f4af7be3700 -1 osd.5 2716 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-04-07T17:51:46.166431+0000 front 2024-04-07T17:51:46.166680+0000 (oldest deadline 2024-04-07T17:52:08.466253+0000) 2024-04-07T17:52:28.615 INFO:journalctl@ceph.osd.5.smithi144.stdout:Apr 07 17:52:28 smithi144 bash[75374]: debug 2024-04-07T17:52:28.305+0000 7f4af7be3700 -1 osd.5 2716 heartbeat_check: no reply from 172.21.15.100:6814 osd.1 since back 2024-04-07T17:51:51.269051+0000 front 2024-04-07T17:51:51.268528+0000 (oldest deadline 2024-04-07T17:52:20.069044+0000) 2024-04-07T17:52:28.615 INFO:journalctl@ceph.osd.6.smithi144.stdout:Apr 07 17:52:28 smithi144 bash[79803]: debug 2024-04-07T17:52:28.549+0000 7f894d91f700 -1 osd.6 2716 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-04-07T17:51:46.955007+0000 front 2024-04-07T17:51:46.955258+0000 (oldest deadline 2024-04-07T17:52:12.854871+0000) 2024-04-07T17:52:28.615 INFO:journalctl@ceph.osd.6.smithi144.stdout:Apr 07 17:52:28 smithi144 bash[79803]: debug 2024-04-07T17:52:28.549+0000 7f894d91f700 -1 osd.6 2716 heartbeat_check: no reply from 172.21.15.100:6814 osd.1 since back 2024-04-07T17:51:52.855832+0000 front 2024-04-07T17:51:52.855974+0000 (oldest deadline 2024-04-07T17:52:17.555534+0000) 2024-04-07T17:52:28.615 INFO:journalctl@ceph.osd.6.smithi144.stdout:Apr 07 17:52:28 smithi144 bash[79803]: debug 2024-04-07T17:52:28.549+0000 7f894d91f700 -1 osd.6 2716 heartbeat_check: no reply from 172.21.15.100:6822 osd.2 since back 2024-04-07T17:52:02.757914+0000 front 2024-04-07T17:52:02.757770+0000 (oldest deadline 2024-04-07T17:52:26.257434+0000) 2024-04-07T17:52:29.114 INFO:journalctl@ceph.osd.7.smithi144.stdout:Apr 07 17:52:28 smithi144 bash[84215]: debug 2024-04-07T17:52:28.737+0000 7f786bcc4700 -1 osd.7 2716 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-04-07T17:51:46.828238+0000 front 2024-04-07T17:51:46.828037+0000 (oldest deadline 2024-04-07T17:52:12.727582+0000) 2024-04-07T17:52:29.115 INFO:journalctl@ceph.osd.7.smithi144.stdout:Apr 07 17:52:28 smithi144 bash[84215]: debug 2024-04-07T17:52:28.737+0000 7f786bcc4700 -1 osd.7 2716 heartbeat_check: no reply from 172.21.15.100:6814 osd.1 since back 2024-04-07T17:51:55.029147+0000 front 2024-04-07T17:51:55.029247+0000 (oldest deadline 2024-04-07T17:52:17.928736+0000) 2024-04-07T17:52:29.115 INFO:journalctl@ceph.osd.7.smithi144.stdout:Apr 07 17:52:28 smithi144 bash[84215]: debug 2024-04-07T17:52:28.737+0000 7f786bcc4700 -1 osd.7 2716 heartbeat_check: no reply from 172.21.15.100:6822 osd.2 since back 2024-04-07T17:52:03.131185+0000 front 2024-04-07T17:52:03.131256+0000 (oldest deadline 2024-04-07T17:52:27.830667+0000) 2024-04-07T17:52:29.587 INFO:journalctl@ceph.osd.5.smithi144.stdout:Apr 07 17:52:29 smithi144 bash[75374]: debug 2024-04-07T17:52:29.329+0000 7f4af7be3700 -1 osd.5 2716 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-04-07T17:51:46.166431+0000 front 2024-04-07T17:51:46.166680+0000 (oldest deadline 2024-04-07T17:52:08.466253+0000) 2024-04-07T17:52:29.587 INFO:journalctl@ceph.osd.5.smithi144.stdout:Apr 07 17:52:29 smithi144 bash[75374]: debug 2024-04-07T17:52:29.329+0000 7f4af7be3700 -1 osd.5 2716 heartbeat_check: no reply from 172.21.15.100:6814 osd.1 since back 2024-04-07T17:51:51.269051+0000 front 2024-04-07T17:51:51.268528+0000 (oldest deadline 2024-04-07T17:52:20.069044+0000) 2024-04-07T17:52:29.864 INFO:journalctl@ceph.osd.6.smithi144.stdout:Apr 07 17:52:29 smithi144 bash[79803]: debug 2024-04-07T17:52:29.585+0000 7f894d91f700 -1 osd.6 2716 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-04-07T17:51:46.955007+0000 front 2024-04-07T17:51:46.955258+0000 (oldest deadline 2024-04-07T17:52:12.854871+0000) 2024-04-07T17:52:29.865 INFO:journalctl@ceph.osd.6.smithi144.stdout:Apr 07 17:52:29 smithi144 bash[79803]: debug 2024-04-07T17:52:29.585+0000 7f894d91f700 -1 osd.6 2716 heartbeat_check: no reply from 172.21.15.100:6814 osd.1 since back 2024-04-07T17:51:52.855832+0000 front 2024-04-07T17:51:52.855974+0000 (oldest deadline 2024-04-07T17:52:17.555534+0000) 2024-04-07T17:52:29.865 INFO:journalctl@ceph.osd.6.smithi144.stdout:Apr 07 17:52:29 smithi144 bash[79803]: debug 2024-04-07T17:52:29.585+0000 7f894d91f700 -1 osd.6 2716 heartbeat_check: no reply from 172.21.15.100:6822 osd.2 since back 2024-04-07T17:52:02.757914+0000 front 2024-04-07T17:52:02.757770+0000 (oldest deadline 2024-04-07T17:52:26.257434+0000) 2024-04-07T17:52:29.865 INFO:journalctl@ceph.osd.7.smithi144.stdout:Apr 07 17:52:29 smithi144 bash[84215]: debug 2024-04-07T17:52:29.765+0000 7f786bcc4700 -1 osd.7 2716 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-04-07T17:51:46.828238+0000 front 2024-04-07T17:51:46.828037+0000 (oldest deadline 2024-04-07T17:52:12.727582+0000) 2024-04-07T17:52:29.865 INFO:journalctl@ceph.osd.7.smithi144.stdout:Apr 07 17:52:29 smithi144 bash[84215]: debug 2024-04-07T17:52:29.765+0000 7f786bcc4700 -1 osd.7 2716 heartbeat_check: no reply from 172.21.15.100:6814 osd.1 since back 2024-04-07T17:51:55.029147+0000 front 2024-04-07T17:51:55.029247+0000 (oldest deadline 2024-04-07T17:52:17.928736+0000) 2024-04-07T17:52:29.865 INFO:journalctl@ceph.osd.7.smithi144.stdout:Apr 07 17:52:29 smithi144 bash[84215]: debug 2024-04-07T17:52:29.765+0000 7f786bcc4700 -1 osd.7 2716 heartbeat_check: no reply from 172.21.15.100:6822 osd.2 since back 2024-04-07T17:52:03.131185+0000 front 2024-04-07T17:52:03.131256+0000 (oldest deadline 2024-04-07T17:52:27.830667+0000) 2024-04-07T17:52:30.703 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Apr 07 17:52:30 smithi100 bash[105121]: ts=2024-04-07T17:52:30.248Z caller=dispatch.go:352 level=error component=dispatcher msg="Notify for alerts failed" num_alerts=1 err="ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://smithi100.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.100:8443: connect: connection refused" 2024-04-07T17:52:30.703 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Apr 07 17:52:30 smithi100 bash[105121]: ts=2024-04-07T17:52:30.249Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi100.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.100:8443: connect: connection refused" 2024-04-07T17:52:30.789 INFO:journalctl@ceph.osd.6.smithi144.stdout:Apr 07 17:52:30 smithi144 bash[79803]: debug 2024-04-07T17:52:30.537+0000 7f894d91f700 -1 osd.6 2716 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-04-07T17:51:46.955007+0000 front 2024-04-07T17:51:46.955258+0000 (oldest deadline 2024-04-07T17:52:12.854871+0000) 2024-04-07T17:52:30.789 INFO:journalctl@ceph.osd.6.smithi144.stdout:Apr 07 17:52:30 smithi144 bash[79803]: debug 2024-04-07T17:52:30.537+0000 7f894d91f700 -1 osd.6 2716 heartbeat_check: no reply from 172.21.15.100:6814 osd.1 since back 2024-04-07T17:51:52.855832+0000 front 2024-04-07T17:51:52.855974+0000 (oldest deadline 2024-04-07T17:52:17.555534+0000) 2024-04-07T17:52:30.789 INFO:journalctl@ceph.osd.6.smithi144.stdout:Apr 07 17:52:30 smithi144 bash[79803]: debug 2024-04-07T17:52:30.537+0000 7f894d91f700 -1 osd.6 2716 heartbeat_check: no reply from 172.21.15.100:6822 osd.2 since back 2024-04-07T17:52:02.757914+0000 front 2024-04-07T17:52:02.757770+0000 (oldest deadline 2024-04-07T17:52:26.257434+0000) 2024-04-07T17:52:30.790 INFO:journalctl@ceph.osd.7.smithi144.stdout:Apr 07 17:52:30 smithi144 bash[84215]: debug 2024-04-07T17:52:30.777+0000 7f786bcc4700 -1 osd.7 2716 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-04-07T17:51:46.828238+0000 front 2024-04-07T17:51:46.828037+0000 (oldest deadline 2024-04-07T17:52:12.727582+0000) 2024-04-07T17:52:30.790 INFO:journalctl@ceph.osd.7.smithi144.stdout:Apr 07 17:52:30 smithi144 bash[84215]: debug 2024-04-07T17:52:30.777+0000 7f786bcc4700 -1 osd.7 2716 heartbeat_check: no reply from 172.21.15.100:6814 osd.1 since back 2024-04-07T17:51:55.029147+0000 front 2024-04-07T17:51:55.029247+0000 (oldest deadline 2024-04-07T17:52:17.928736+0000) 2024-04-07T17:52:30.790 INFO:journalctl@ceph.osd.7.smithi144.stdout:Apr 07 17:52:30 smithi144 bash[84215]: debug 2024-04-07T17:52:30.777+0000 7f786bcc4700 -1 osd.7 2716 heartbeat_check: no reply from 172.21.15.100:6822 osd.2 since back 2024-04-07T17:52:03.131185+0000 front 2024-04-07T17:52:03.131256+0000 (oldest deadline 2024-04-07T17:52:27.830667+0000) 2024-04-07T17:52:31.114 INFO:journalctl@ceph.osd.5.smithi144.stdout:Apr 07 17:52:30 smithi144 bash[114005]: ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4-osd-5 2024-04-07T17:52:31.115 INFO:journalctl@ceph.osd.5.smithi144.stdout:Apr 07 17:52:31 smithi144 bash[114069]: Error response from daemon: No such container: ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4-osd-5 2024-04-07T17:52:31.799 INFO:journalctl@ceph.osd.6.smithi144.stdout:Apr 07 17:52:31 smithi144 bash[79803]: debug 2024-04-07T17:52:31.501+0000 7f894d91f700 -1 osd.6 2716 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-04-07T17:51:46.955007+0000 front 2024-04-07T17:51:46.955258+0000 (oldest deadline 2024-04-07T17:52:12.854871+0000) 2024-04-07T17:52:31.799 INFO:journalctl@ceph.osd.6.smithi144.stdout:Apr 07 17:52:31 smithi144 bash[79803]: debug 2024-04-07T17:52:31.501+0000 7f894d91f700 -1 osd.6 2716 heartbeat_check: no reply from 172.21.15.100:6814 osd.1 since back 2024-04-07T17:51:52.855832+0000 front 2024-04-07T17:51:52.855974+0000 (oldest deadline 2024-04-07T17:52:17.555534+0000) 2024-04-07T17:52:31.799 INFO:journalctl@ceph.osd.6.smithi144.stdout:Apr 07 17:52:31 smithi144 bash[79803]: debug 2024-04-07T17:52:31.501+0000 7f894d91f700 -1 osd.6 2716 heartbeat_check: no reply from 172.21.15.100:6822 osd.2 since back 2024-04-07T17:52:02.757914+0000 front 2024-04-07T17:52:02.757770+0000 (oldest deadline 2024-04-07T17:52:26.257434+0000) 2024-04-07T17:52:32.114 INFO:journalctl@ceph.osd.7.smithi144.stdout:Apr 07 17:52:31 smithi144 bash[84215]: debug 2024-04-07T17:52:31.805+0000 7f786bcc4700 -1 osd.7 2716 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-04-07T17:51:46.828238+0000 front 2024-04-07T17:51:46.828037+0000 (oldest deadline 2024-04-07T17:52:12.727582+0000) 2024-04-07T17:52:32.114 INFO:journalctl@ceph.osd.7.smithi144.stdout:Apr 07 17:52:31 smithi144 bash[84215]: debug 2024-04-07T17:52:31.805+0000 7f786bcc4700 -1 osd.7 2716 heartbeat_check: no reply from 172.21.15.100:6814 osd.1 since back 2024-04-07T17:51:55.029147+0000 front 2024-04-07T17:51:55.029247+0000 (oldest deadline 2024-04-07T17:52:17.928736+0000) 2024-04-07T17:52:32.114 INFO:journalctl@ceph.osd.7.smithi144.stdout:Apr 07 17:52:31 smithi144 bash[84215]: debug 2024-04-07T17:52:31.805+0000 7f786bcc4700 -1 osd.7 2716 heartbeat_check: no reply from 172.21.15.100:6822 osd.2 since back 2024-04-07T17:52:03.131185+0000 front 2024-04-07T17:52:03.131256+0000 (oldest deadline 2024-04-07T17:52:27.830667+0000) 2024-04-07T17:52:32.844 INFO:journalctl@ceph.osd.6.smithi144.stdout:Apr 07 17:52:32 smithi144 bash[79803]: debug 2024-04-07T17:52:32.537+0000 7f894d91f700 -1 osd.6 2716 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-04-07T17:51:46.955007+0000 front 2024-04-07T17:51:46.955258+0000 (oldest deadline 2024-04-07T17:52:12.854871+0000) 2024-04-07T17:52:32.844 INFO:journalctl@ceph.osd.6.smithi144.stdout:Apr 07 17:52:32 smithi144 bash[79803]: debug 2024-04-07T17:52:32.537+0000 7f894d91f700 -1 osd.6 2716 heartbeat_check: no reply from 172.21.15.100:6814 osd.1 since back 2024-04-07T17:51:52.855832+0000 front 2024-04-07T17:51:52.855974+0000 (oldest deadline 2024-04-07T17:52:17.555534+0000) 2024-04-07T17:52:32.844 INFO:journalctl@ceph.osd.6.smithi144.stdout:Apr 07 17:52:32 smithi144 bash[79803]: debug 2024-04-07T17:52:32.537+0000 7f894d91f700 -1 osd.6 2716 heartbeat_check: no reply from 172.21.15.100:6822 osd.2 since back 2024-04-07T17:52:02.757914+0000 front 2024-04-07T17:52:02.757770+0000 (oldest deadline 2024-04-07T17:52:26.257434+0000) 2024-04-07T17:52:32.902 DEBUG:teuthology.orchestra.run.smithi144:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@osd.5.service' 2024-04-07T17:52:32.929 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-07T17:52:32.929 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-04-07T17:52:32.929 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-04-07T17:52:32.929 DEBUG:teuthology.orchestra.run.smithi144:> sudo systemctl stop ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@osd.6 2024-04-07T17:52:33.114 INFO:journalctl@ceph.osd.6.smithi144.stdout:Apr 07 17:52:32 smithi144 systemd[1]: Stopping Ceph osd.6 for 0702e3a8-f4f8-11ee-b649-cb9ed24678a4... 2024-04-07T17:52:33.115 INFO:journalctl@ceph.osd.7.smithi144.stdout:Apr 07 17:52:32 smithi144 bash[84215]: debug 2024-04-07T17:52:32.841+0000 7f786bcc4700 -1 osd.7 2716 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-04-07T17:51:46.828238+0000 front 2024-04-07T17:51:46.828037+0000 (oldest deadline 2024-04-07T17:52:12.727582+0000) 2024-04-07T17:52:33.115 INFO:journalctl@ceph.osd.7.smithi144.stdout:Apr 07 17:52:32 smithi144 bash[84215]: debug 2024-04-07T17:52:32.841+0000 7f786bcc4700 -1 osd.7 2716 heartbeat_check: no reply from 172.21.15.100:6814 osd.1 since back 2024-04-07T17:51:55.029147+0000 front 2024-04-07T17:51:55.029247+0000 (oldest deadline 2024-04-07T17:52:17.928736+0000) 2024-04-07T17:52:33.115 INFO:journalctl@ceph.osd.7.smithi144.stdout:Apr 07 17:52:32 smithi144 bash[84215]: debug 2024-04-07T17:52:32.841+0000 7f786bcc4700 -1 osd.7 2716 heartbeat_check: no reply from 172.21.15.100:6822 osd.2 since back 2024-04-07T17:52:03.131185+0000 front 2024-04-07T17:52:03.131256+0000 (oldest deadline 2024-04-07T17:52:27.830667+0000) 2024-04-07T17:52:33.364 INFO:journalctl@ceph.osd.6.smithi144.stdout:Apr 07 17:52:33 smithi144 bash[79803]: debug 2024-04-07T17:52:33.113+0000 7f8955333700 -1 received signal: Terminated from /sbin/docker-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug (PID: 1) UID: 0 2024-04-07T17:52:33.364 INFO:journalctl@ceph.osd.6.smithi144.stdout:Apr 07 17:52:33 smithi144 bash[79803]: debug 2024-04-07T17:52:33.113+0000 7f8955333700 -1 osd.6 2716 *** Got signal Terminated *** 2024-04-07T17:52:33.365 INFO:journalctl@ceph.osd.6.smithi144.stdout:Apr 07 17:52:33 smithi144 bash[79803]: debug 2024-04-07T17:52:33.113+0000 7f8955333700 -1 osd.6 2716 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-07T17:52:33.844 INFO:journalctl@ceph.osd.6.smithi144.stdout:Apr 07 17:52:33 smithi144 bash[79803]: debug 2024-04-07T17:52:33.489+0000 7f894d91f700 -1 osd.6 2716 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-04-07T17:51:46.955007+0000 front 2024-04-07T17:51:46.955258+0000 (oldest deadline 2024-04-07T17:52:12.854871+0000) 2024-04-07T17:52:33.844 INFO:journalctl@ceph.osd.6.smithi144.stdout:Apr 07 17:52:33 smithi144 bash[79803]: debug 2024-04-07T17:52:33.489+0000 7f894d91f700 -1 osd.6 2716 heartbeat_check: no reply from 172.21.15.100:6814 osd.1 since back 2024-04-07T17:51:52.855832+0000 front 2024-04-07T17:51:52.855974+0000 (oldest deadline 2024-04-07T17:52:17.555534+0000) 2024-04-07T17:52:33.844 INFO:journalctl@ceph.osd.6.smithi144.stdout:Apr 07 17:52:33 smithi144 bash[79803]: debug 2024-04-07T17:52:33.489+0000 7f894d91f700 -1 osd.6 2716 heartbeat_check: no reply from 172.21.15.100:6822 osd.2 since back 2024-04-07T17:52:02.757914+0000 front 2024-04-07T17:52:02.757770+0000 (oldest deadline 2024-04-07T17:52:26.257434+0000) 2024-04-07T17:52:34.114 INFO:journalctl@ceph.osd.7.smithi144.stdout:Apr 07 17:52:33 smithi144 bash[84215]: debug 2024-04-07T17:52:33.841+0000 7f786bcc4700 -1 osd.7 2716 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-04-07T17:51:46.828238+0000 front 2024-04-07T17:51:46.828037+0000 (oldest deadline 2024-04-07T17:52:12.727582+0000) 2024-04-07T17:52:34.114 INFO:journalctl@ceph.osd.7.smithi144.stdout:Apr 07 17:52:33 smithi144 bash[84215]: debug 2024-04-07T17:52:33.841+0000 7f786bcc4700 -1 osd.7 2716 heartbeat_check: no reply from 172.21.15.100:6814 osd.1 since back 2024-04-07T17:51:55.029147+0000 front 2024-04-07T17:51:55.029247+0000 (oldest deadline 2024-04-07T17:52:17.928736+0000) 2024-04-07T17:52:34.115 INFO:journalctl@ceph.osd.7.smithi144.stdout:Apr 07 17:52:33 smithi144 bash[84215]: debug 2024-04-07T17:52:33.841+0000 7f786bcc4700 -1 osd.7 2716 heartbeat_check: no reply from 172.21.15.100:6822 osd.2 since back 2024-04-07T17:52:03.131185+0000 front 2024-04-07T17:52:03.131256+0000 (oldest deadline 2024-04-07T17:52:27.830667+0000) 2024-04-07T17:52:34.864 INFO:journalctl@ceph.osd.6.smithi144.stdout:Apr 07 17:52:34 smithi144 bash[79803]: debug 2024-04-07T17:52:34.465+0000 7f894d91f700 -1 osd.6 2716 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-04-07T17:51:46.955007+0000 front 2024-04-07T17:51:46.955258+0000 (oldest deadline 2024-04-07T17:52:12.854871+0000) 2024-04-07T17:52:34.864 INFO:journalctl@ceph.osd.6.smithi144.stdout:Apr 07 17:52:34 smithi144 bash[79803]: debug 2024-04-07T17:52:34.465+0000 7f894d91f700 -1 osd.6 2716 heartbeat_check: no reply from 172.21.15.100:6814 osd.1 since back 2024-04-07T17:51:52.855832+0000 front 2024-04-07T17:51:52.855974+0000 (oldest deadline 2024-04-07T17:52:17.555534+0000) 2024-04-07T17:52:34.864 INFO:journalctl@ceph.osd.6.smithi144.stdout:Apr 07 17:52:34 smithi144 bash[79803]: debug 2024-04-07T17:52:34.465+0000 7f894d91f700 -1 osd.6 2716 heartbeat_check: no reply from 172.21.15.100:6822 osd.2 since back 2024-04-07T17:52:02.757914+0000 front 2024-04-07T17:52:02.757770+0000 (oldest deadline 2024-04-07T17:52:26.257434+0000) 2024-04-07T17:52:35.364 INFO:journalctl@ceph.osd.7.smithi144.stdout:Apr 07 17:52:34 smithi144 bash[84215]: debug 2024-04-07T17:52:34.885+0000 7f786bcc4700 -1 osd.7 2716 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-04-07T17:51:46.828238+0000 front 2024-04-07T17:51:46.828037+0000 (oldest deadline 2024-04-07T17:52:12.727582+0000) 2024-04-07T17:52:35.365 INFO:journalctl@ceph.osd.7.smithi144.stdout:Apr 07 17:52:34 smithi144 bash[84215]: debug 2024-04-07T17:52:34.885+0000 7f786bcc4700 -1 osd.7 2716 heartbeat_check: no reply from 172.21.15.100:6814 osd.1 since back 2024-04-07T17:51:55.029147+0000 front 2024-04-07T17:51:55.029247+0000 (oldest deadline 2024-04-07T17:52:17.928736+0000) 2024-04-07T17:52:35.365 INFO:journalctl@ceph.osd.7.smithi144.stdout:Apr 07 17:52:34 smithi144 bash[84215]: debug 2024-04-07T17:52:34.885+0000 7f786bcc4700 -1 osd.7 2716 heartbeat_check: no reply from 172.21.15.100:6822 osd.2 since back 2024-04-07T17:52:03.131185+0000 front 2024-04-07T17:52:03.131256+0000 (oldest deadline 2024-04-07T17:52:27.830667+0000) 2024-04-07T17:52:35.365 INFO:journalctl@ceph.osd.7.smithi144.stdout:Apr 07 17:52:34 smithi144 bash[84215]: debug 2024-04-07T17:52:34.885+0000 7f786bcc4700 -1 osd.7 2716 heartbeat_check: no reply from 172.21.15.100:6830 osd.3 since back 2024-04-07T17:52:10.732559+0000 front 2024-04-07T17:52:10.732451+0000 (oldest deadline 2024-04-07T17:52:34.832010+0000) 2024-04-07T17:52:35.864 INFO:journalctl@ceph.osd.6.smithi144.stdout:Apr 07 17:52:35 smithi144 bash[79803]: debug 2024-04-07T17:52:35.505+0000 7f894d91f700 -1 osd.6 2716 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-04-07T17:51:46.955007+0000 front 2024-04-07T17:51:46.955258+0000 (oldest deadline 2024-04-07T17:52:12.854871+0000) 2024-04-07T17:52:35.864 INFO:journalctl@ceph.osd.6.smithi144.stdout:Apr 07 17:52:35 smithi144 bash[79803]: debug 2024-04-07T17:52:35.505+0000 7f894d91f700 -1 osd.6 2716 heartbeat_check: no reply from 172.21.15.100:6814 osd.1 since back 2024-04-07T17:51:52.855832+0000 front 2024-04-07T17:51:52.855974+0000 (oldest deadline 2024-04-07T17:52:17.555534+0000) 2024-04-07T17:52:35.865 INFO:journalctl@ceph.osd.6.smithi144.stdout:Apr 07 17:52:35 smithi144 bash[79803]: debug 2024-04-07T17:52:35.505+0000 7f894d91f700 -1 osd.6 2716 heartbeat_check: no reply from 172.21.15.100:6822 osd.2 since back 2024-04-07T17:52:02.757914+0000 front 2024-04-07T17:52:02.757770+0000 (oldest deadline 2024-04-07T17:52:26.257434+0000) 2024-04-07T17:52:36.364 INFO:journalctl@ceph.osd.7.smithi144.stdout:Apr 07 17:52:35 smithi144 bash[84215]: debug 2024-04-07T17:52:35.925+0000 7f786bcc4700 -1 osd.7 2716 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-04-07T17:51:46.828238+0000 front 2024-04-07T17:51:46.828037+0000 (oldest deadline 2024-04-07T17:52:12.727582+0000) 2024-04-07T17:52:36.364 INFO:journalctl@ceph.osd.7.smithi144.stdout:Apr 07 17:52:35 smithi144 bash[84215]: debug 2024-04-07T17:52:35.925+0000 7f786bcc4700 -1 osd.7 2716 heartbeat_check: no reply from 172.21.15.100:6814 osd.1 since back 2024-04-07T17:51:55.029147+0000 front 2024-04-07T17:51:55.029247+0000 (oldest deadline 2024-04-07T17:52:17.928736+0000) 2024-04-07T17:52:36.365 INFO:journalctl@ceph.osd.7.smithi144.stdout:Apr 07 17:52:35 smithi144 bash[84215]: debug 2024-04-07T17:52:35.925+0000 7f786bcc4700 -1 osd.7 2716 heartbeat_check: no reply from 172.21.15.100:6822 osd.2 since back 2024-04-07T17:52:03.131185+0000 front 2024-04-07T17:52:03.131256+0000 (oldest deadline 2024-04-07T17:52:27.830667+0000) 2024-04-07T17:52:36.365 INFO:journalctl@ceph.osd.7.smithi144.stdout:Apr 07 17:52:35 smithi144 bash[84215]: debug 2024-04-07T17:52:35.925+0000 7f786bcc4700 -1 osd.7 2716 heartbeat_check: no reply from 172.21.15.100:6830 osd.3 since back 2024-04-07T17:52:10.732559+0000 front 2024-04-07T17:52:10.732451+0000 (oldest deadline 2024-04-07T17:52:34.832010+0000) 2024-04-07T17:52:36.864 INFO:journalctl@ceph.osd.6.smithi144.stdout:Apr 07 17:52:36 smithi144 bash[79803]: debug 2024-04-07T17:52:36.485+0000 7f894d91f700 -1 osd.6 2716 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-04-07T17:51:46.955007+0000 front 2024-04-07T17:51:46.955258+0000 (oldest deadline 2024-04-07T17:52:12.854871+0000) 2024-04-07T17:52:36.864 INFO:journalctl@ceph.osd.6.smithi144.stdout:Apr 07 17:52:36 smithi144 bash[79803]: debug 2024-04-07T17:52:36.485+0000 7f894d91f700 -1 osd.6 2716 heartbeat_check: no reply from 172.21.15.100:6814 osd.1 since back 2024-04-07T17:51:52.855832+0000 front 2024-04-07T17:51:52.855974+0000 (oldest deadline 2024-04-07T17:52:17.555534+0000) 2024-04-07T17:52:36.865 INFO:journalctl@ceph.osd.6.smithi144.stdout:Apr 07 17:52:36 smithi144 bash[79803]: debug 2024-04-07T17:52:36.485+0000 7f894d91f700 -1 osd.6 2716 heartbeat_check: no reply from 172.21.15.100:6822 osd.2 since back 2024-04-07T17:52:02.757914+0000 front 2024-04-07T17:52:02.757770+0000 (oldest deadline 2024-04-07T17:52:26.257434+0000) 2024-04-07T17:52:37.364 INFO:journalctl@ceph.osd.7.smithi144.stdout:Apr 07 17:52:36 smithi144 bash[84215]: debug 2024-04-07T17:52:36.937+0000 7f786bcc4700 -1 osd.7 2716 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-04-07T17:51:46.828238+0000 front 2024-04-07T17:51:46.828037+0000 (oldest deadline 2024-04-07T17:52:12.727582+0000) 2024-04-07T17:52:37.365 INFO:journalctl@ceph.osd.7.smithi144.stdout:Apr 07 17:52:36 smithi144 bash[84215]: debug 2024-04-07T17:52:36.937+0000 7f786bcc4700 -1 osd.7 2716 heartbeat_check: no reply from 172.21.15.100:6814 osd.1 since back 2024-04-07T17:51:55.029147+0000 front 2024-04-07T17:51:55.029247+0000 (oldest deadline 2024-04-07T17:52:17.928736+0000) 2024-04-07T17:52:37.365 INFO:journalctl@ceph.osd.7.smithi144.stdout:Apr 07 17:52:36 smithi144 bash[84215]: debug 2024-04-07T17:52:36.937+0000 7f786bcc4700 -1 osd.7 2716 heartbeat_check: no reply from 172.21.15.100:6822 osd.2 since back 2024-04-07T17:52:03.131185+0000 front 2024-04-07T17:52:03.131256+0000 (oldest deadline 2024-04-07T17:52:27.830667+0000) 2024-04-07T17:52:37.365 INFO:journalctl@ceph.osd.7.smithi144.stdout:Apr 07 17:52:36 smithi144 bash[84215]: debug 2024-04-07T17:52:36.937+0000 7f786bcc4700 -1 osd.7 2716 heartbeat_check: no reply from 172.21.15.100:6830 osd.3 since back 2024-04-07T17:52:10.732559+0000 front 2024-04-07T17:52:10.732451+0000 (oldest deadline 2024-04-07T17:52:34.832010+0000) 2024-04-07T17:52:37.864 INFO:journalctl@ceph.osd.6.smithi144.stdout:Apr 07 17:52:37 smithi144 bash[79803]: debug 2024-04-07T17:52:37.481+0000 7f894d91f700 -1 osd.6 2716 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-04-07T17:51:46.955007+0000 front 2024-04-07T17:51:46.955258+0000 (oldest deadline 2024-04-07T17:52:12.854871+0000) 2024-04-07T17:52:37.864 INFO:journalctl@ceph.osd.6.smithi144.stdout:Apr 07 17:52:37 smithi144 bash[79803]: debug 2024-04-07T17:52:37.481+0000 7f894d91f700 -1 osd.6 2716 heartbeat_check: no reply from 172.21.15.100:6814 osd.1 since back 2024-04-07T17:51:52.855832+0000 front 2024-04-07T17:51:52.855974+0000 (oldest deadline 2024-04-07T17:52:17.555534+0000) 2024-04-07T17:52:37.864 INFO:journalctl@ceph.osd.6.smithi144.stdout:Apr 07 17:52:37 smithi144 bash[79803]: debug 2024-04-07T17:52:37.481+0000 7f894d91f700 -1 osd.6 2716 heartbeat_check: no reply from 172.21.15.100:6822 osd.2 since back 2024-04-07T17:52:02.757914+0000 front 2024-04-07T17:52:02.757770+0000 (oldest deadline 2024-04-07T17:52:26.257434+0000) 2024-04-07T17:52:38.364 INFO:journalctl@ceph.osd.7.smithi144.stdout:Apr 07 17:52:37 smithi144 bash[84215]: debug 2024-04-07T17:52:37.897+0000 7f786bcc4700 -1 osd.7 2716 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-04-07T17:51:46.828238+0000 front 2024-04-07T17:51:46.828037+0000 (oldest deadline 2024-04-07T17:52:12.727582+0000) 2024-04-07T17:52:38.365 INFO:journalctl@ceph.osd.7.smithi144.stdout:Apr 07 17:52:37 smithi144 bash[84215]: debug 2024-04-07T17:52:37.897+0000 7f786bcc4700 -1 osd.7 2716 heartbeat_check: no reply from 172.21.15.100:6814 osd.1 since back 2024-04-07T17:51:55.029147+0000 front 2024-04-07T17:51:55.029247+0000 (oldest deadline 2024-04-07T17:52:17.928736+0000) 2024-04-07T17:52:38.365 INFO:journalctl@ceph.osd.7.smithi144.stdout:Apr 07 17:52:37 smithi144 bash[84215]: debug 2024-04-07T17:52:37.897+0000 7f786bcc4700 -1 osd.7 2716 heartbeat_check: no reply from 172.21.15.100:6822 osd.2 since back 2024-04-07T17:52:03.131185+0000 front 2024-04-07T17:52:03.131256+0000 (oldest deadline 2024-04-07T17:52:27.830667+0000) 2024-04-07T17:52:38.365 INFO:journalctl@ceph.osd.7.smithi144.stdout:Apr 07 17:52:37 smithi144 bash[84215]: debug 2024-04-07T17:52:37.897+0000 7f786bcc4700 -1 osd.7 2716 heartbeat_check: no reply from 172.21.15.100:6830 osd.3 since back 2024-04-07T17:52:10.732559+0000 front 2024-04-07T17:52:10.732451+0000 (oldest deadline 2024-04-07T17:52:34.832010+0000) 2024-04-07T17:52:39.335 INFO:journalctl@ceph.osd.6.smithi144.stdout:Apr 07 17:52:39 smithi144 bash[114209]: ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4-osd-6 2024-04-07T17:52:39.335 INFO:journalctl@ceph.osd.7.smithi144.stdout:Apr 07 17:52:38 smithi144 bash[84215]: debug 2024-04-07T17:52:38.945+0000 7f786bcc4700 -1 osd.7 2716 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-04-07T17:51:46.828238+0000 front 2024-04-07T17:51:46.828037+0000 (oldest deadline 2024-04-07T17:52:12.727582+0000) 2024-04-07T17:52:39.335 INFO:journalctl@ceph.osd.7.smithi144.stdout:Apr 07 17:52:38 smithi144 bash[84215]: debug 2024-04-07T17:52:38.945+0000 7f786bcc4700 -1 osd.7 2716 heartbeat_check: no reply from 172.21.15.100:6814 osd.1 since back 2024-04-07T17:51:55.029147+0000 front 2024-04-07T17:51:55.029247+0000 (oldest deadline 2024-04-07T17:52:17.928736+0000) 2024-04-07T17:52:39.336 INFO:journalctl@ceph.osd.7.smithi144.stdout:Apr 07 17:52:38 smithi144 bash[84215]: debug 2024-04-07T17:52:38.945+0000 7f786bcc4700 -1 osd.7 2716 heartbeat_check: no reply from 172.21.15.100:6822 osd.2 since back 2024-04-07T17:52:03.131185+0000 front 2024-04-07T17:52:03.131256+0000 (oldest deadline 2024-04-07T17:52:27.830667+0000) 2024-04-07T17:52:39.336 INFO:journalctl@ceph.osd.7.smithi144.stdout:Apr 07 17:52:38 smithi144 bash[84215]: debug 2024-04-07T17:52:38.945+0000 7f786bcc4700 -1 osd.7 2716 heartbeat_check: no reply from 172.21.15.100:6830 osd.3 since back 2024-04-07T17:52:10.732559+0000 front 2024-04-07T17:52:10.732451+0000 (oldest deadline 2024-04-07T17:52:34.832010+0000) 2024-04-07T17:52:39.614 INFO:journalctl@ceph.osd.6.smithi144.stdout:Apr 07 17:52:39 smithi144 bash[114273]: Error response from daemon: No such container: ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4-osd-6 2024-04-07T17:52:40.189 INFO:journalctl@ceph.osd.7.smithi144.stdout:Apr 07 17:52:39 smithi144 bash[84215]: debug 2024-04-07T17:52:39.929+0000 7f786bcc4700 -1 osd.7 2716 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-04-07T17:51:46.828238+0000 front 2024-04-07T17:51:46.828037+0000 (oldest deadline 2024-04-07T17:52:12.727582+0000) 2024-04-07T17:52:40.189 INFO:journalctl@ceph.osd.7.smithi144.stdout:Apr 07 17:52:39 smithi144 bash[84215]: debug 2024-04-07T17:52:39.929+0000 7f786bcc4700 -1 osd.7 2716 heartbeat_check: no reply from 172.21.15.100:6814 osd.1 since back 2024-04-07T17:51:55.029147+0000 front 2024-04-07T17:51:55.029247+0000 (oldest deadline 2024-04-07T17:52:17.928736+0000) 2024-04-07T17:52:40.189 INFO:journalctl@ceph.osd.7.smithi144.stdout:Apr 07 17:52:39 smithi144 bash[84215]: debug 2024-04-07T17:52:39.929+0000 7f786bcc4700 -1 osd.7 2716 heartbeat_check: no reply from 172.21.15.100:6822 osd.2 since back 2024-04-07T17:52:03.131185+0000 front 2024-04-07T17:52:03.131256+0000 (oldest deadline 2024-04-07T17:52:27.830667+0000) 2024-04-07T17:52:40.189 INFO:journalctl@ceph.osd.7.smithi144.stdout:Apr 07 17:52:39 smithi144 bash[84215]: debug 2024-04-07T17:52:39.929+0000 7f786bcc4700 -1 osd.7 2716 heartbeat_check: no reply from 172.21.15.100:6830 osd.3 since back 2024-04-07T17:52:10.732559+0000 front 2024-04-07T17:52:10.732451+0000 (oldest deadline 2024-04-07T17:52:34.832010+0000) 2024-04-07T17:52:40.703 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Apr 07 17:52:40 smithi100 bash[105121]: ts=2024-04-07T17:52:40.248Z caller=dispatch.go:352 level=error component=dispatcher msg="Notify for alerts failed" num_alerts=1 err="ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://smithi100.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.100:8443: connect: connection refused" 2024-04-07T17:52:40.703 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Apr 07 17:52:40 smithi100 bash[105121]: ts=2024-04-07T17:52:40.249Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi100.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.100:8443: connect: connection refused" 2024-04-07T17:52:41.241 INFO:journalctl@ceph.osd.7.smithi144.stdout:Apr 07 17:52:40 smithi144 bash[84215]: debug 2024-04-07T17:52:40.909+0000 7f786bcc4700 -1 osd.7 2716 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-04-07T17:51:46.828238+0000 front 2024-04-07T17:51:46.828037+0000 (oldest deadline 2024-04-07T17:52:12.727582+0000) 2024-04-07T17:52:41.242 INFO:journalctl@ceph.osd.7.smithi144.stdout:Apr 07 17:52:40 smithi144 bash[84215]: debug 2024-04-07T17:52:40.909+0000 7f786bcc4700 -1 osd.7 2716 heartbeat_check: no reply from 172.21.15.100:6814 osd.1 since back 2024-04-07T17:51:55.029147+0000 front 2024-04-07T17:51:55.029247+0000 (oldest deadline 2024-04-07T17:52:17.928736+0000) 2024-04-07T17:52:41.242 INFO:journalctl@ceph.osd.7.smithi144.stdout:Apr 07 17:52:40 smithi144 bash[84215]: debug 2024-04-07T17:52:40.909+0000 7f786bcc4700 -1 osd.7 2716 heartbeat_check: no reply from 172.21.15.100:6822 osd.2 since back 2024-04-07T17:52:03.131185+0000 front 2024-04-07T17:52:03.131256+0000 (oldest deadline 2024-04-07T17:52:27.830667+0000) 2024-04-07T17:52:41.242 INFO:journalctl@ceph.osd.7.smithi144.stdout:Apr 07 17:52:40 smithi144 bash[84215]: debug 2024-04-07T17:52:40.909+0000 7f786bcc4700 -1 osd.7 2716 heartbeat_check: no reply from 172.21.15.100:6830 osd.3 since back 2024-04-07T17:52:10.732559+0000 front 2024-04-07T17:52:10.732451+0000 (oldest deadline 2024-04-07T17:52:34.832010+0000) 2024-04-07T17:52:41.245 DEBUG:teuthology.orchestra.run.smithi144:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@osd.6.service' 2024-04-07T17:52:41.272 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-07T17:52:41.272 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-04-07T17:52:41.272 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-04-07T17:52:41.272 DEBUG:teuthology.orchestra.run.smithi144:> sudo systemctl stop ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@osd.7 2024-04-07T17:52:41.614 INFO:journalctl@ceph.osd.7.smithi144.stdout:Apr 07 17:52:41 smithi144 systemd[1]: Stopping Ceph osd.7 for 0702e3a8-f4f8-11ee-b649-cb9ed24678a4... 2024-04-07T17:52:41.614 INFO:journalctl@ceph.osd.7.smithi144.stdout:Apr 07 17:52:41 smithi144 bash[84215]: debug 2024-04-07T17:52:41.457+0000 7f78736d8700 -1 received signal: Terminated from /sbin/docker-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug (PID: 1) UID: 0 2024-04-07T17:52:41.615 INFO:journalctl@ceph.osd.7.smithi144.stdout:Apr 07 17:52:41 smithi144 bash[84215]: debug 2024-04-07T17:52:41.457+0000 7f78736d8700 -1 osd.7 2716 *** Got signal Terminated *** 2024-04-07T17:52:41.615 INFO:journalctl@ceph.osd.7.smithi144.stdout:Apr 07 17:52:41 smithi144 bash[84215]: debug 2024-04-07T17:52:41.457+0000 7f78736d8700 -1 osd.7 2716 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-07T17:52:42.364 INFO:journalctl@ceph.osd.7.smithi144.stdout:Apr 07 17:52:41 smithi144 bash[84215]: debug 2024-04-07T17:52:41.869+0000 7f786bcc4700 -1 osd.7 2716 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-04-07T17:51:46.828238+0000 front 2024-04-07T17:51:46.828037+0000 (oldest deadline 2024-04-07T17:52:12.727582+0000) 2024-04-07T17:52:42.365 INFO:journalctl@ceph.osd.7.smithi144.stdout:Apr 07 17:52:41 smithi144 bash[84215]: debug 2024-04-07T17:52:41.869+0000 7f786bcc4700 -1 osd.7 2716 heartbeat_check: no reply from 172.21.15.100:6814 osd.1 since back 2024-04-07T17:51:55.029147+0000 front 2024-04-07T17:51:55.029247+0000 (oldest deadline 2024-04-07T17:52:17.928736+0000) 2024-04-07T17:52:42.365 INFO:journalctl@ceph.osd.7.smithi144.stdout:Apr 07 17:52:41 smithi144 bash[84215]: debug 2024-04-07T17:52:41.869+0000 7f786bcc4700 -1 osd.7 2716 heartbeat_check: no reply from 172.21.15.100:6822 osd.2 since back 2024-04-07T17:52:03.131185+0000 front 2024-04-07T17:52:03.131256+0000 (oldest deadline 2024-04-07T17:52:27.830667+0000) 2024-04-07T17:52:42.365 INFO:journalctl@ceph.osd.7.smithi144.stdout:Apr 07 17:52:41 smithi144 bash[84215]: debug 2024-04-07T17:52:41.869+0000 7f786bcc4700 -1 osd.7 2716 heartbeat_check: no reply from 172.21.15.100:6830 osd.3 since back 2024-04-07T17:52:10.732559+0000 front 2024-04-07T17:52:10.732451+0000 (oldest deadline 2024-04-07T17:52:34.832010+0000) 2024-04-07T17:52:43.114 INFO:journalctl@ceph.osd.7.smithi144.stdout:Apr 07 17:52:42 smithi144 bash[84215]: debug 2024-04-07T17:52:42.837+0000 7f786bcc4700 -1 osd.7 2716 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-04-07T17:51:46.828238+0000 front 2024-04-07T17:51:46.828037+0000 (oldest deadline 2024-04-07T17:52:12.727582+0000) 2024-04-07T17:52:43.115 INFO:journalctl@ceph.osd.7.smithi144.stdout:Apr 07 17:52:42 smithi144 bash[84215]: debug 2024-04-07T17:52:42.837+0000 7f786bcc4700 -1 osd.7 2716 heartbeat_check: no reply from 172.21.15.100:6814 osd.1 since back 2024-04-07T17:51:55.029147+0000 front 2024-04-07T17:51:55.029247+0000 (oldest deadline 2024-04-07T17:52:17.928736+0000) 2024-04-07T17:52:43.115 INFO:journalctl@ceph.osd.7.smithi144.stdout:Apr 07 17:52:42 smithi144 bash[84215]: debug 2024-04-07T17:52:42.837+0000 7f786bcc4700 -1 osd.7 2716 heartbeat_check: no reply from 172.21.15.100:6822 osd.2 since back 2024-04-07T17:52:03.131185+0000 front 2024-04-07T17:52:03.131256+0000 (oldest deadline 2024-04-07T17:52:27.830667+0000) 2024-04-07T17:52:43.115 INFO:journalctl@ceph.osd.7.smithi144.stdout:Apr 07 17:52:42 smithi144 bash[84215]: debug 2024-04-07T17:52:42.837+0000 7f786bcc4700 -1 osd.7 2716 heartbeat_check: no reply from 172.21.15.100:6830 osd.3 since back 2024-04-07T17:52:10.732559+0000 front 2024-04-07T17:52:10.732451+0000 (oldest deadline 2024-04-07T17:52:34.832010+0000) 2024-04-07T17:52:44.365 INFO:journalctl@ceph.osd.7.smithi144.stdout:Apr 07 17:52:43 smithi144 bash[84215]: debug 2024-04-07T17:52:43.861+0000 7f786bcc4700 -1 osd.7 2716 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-04-07T17:51:46.828238+0000 front 2024-04-07T17:51:46.828037+0000 (oldest deadline 2024-04-07T17:52:12.727582+0000) 2024-04-07T17:52:44.365 INFO:journalctl@ceph.osd.7.smithi144.stdout:Apr 07 17:52:43 smithi144 bash[84215]: debug 2024-04-07T17:52:43.861+0000 7f786bcc4700 -1 osd.7 2716 heartbeat_check: no reply from 172.21.15.100:6814 osd.1 since back 2024-04-07T17:51:55.029147+0000 front 2024-04-07T17:51:55.029247+0000 (oldest deadline 2024-04-07T17:52:17.928736+0000) 2024-04-07T17:52:44.365 INFO:journalctl@ceph.osd.7.smithi144.stdout:Apr 07 17:52:43 smithi144 bash[84215]: debug 2024-04-07T17:52:43.861+0000 7f786bcc4700 -1 osd.7 2716 heartbeat_check: no reply from 172.21.15.100:6822 osd.2 since back 2024-04-07T17:52:03.131185+0000 front 2024-04-07T17:52:03.131256+0000 (oldest deadline 2024-04-07T17:52:27.830667+0000) 2024-04-07T17:52:44.365 INFO:journalctl@ceph.osd.7.smithi144.stdout:Apr 07 17:52:43 smithi144 bash[84215]: debug 2024-04-07T17:52:43.861+0000 7f786bcc4700 -1 osd.7 2716 heartbeat_check: no reply from 172.21.15.100:6830 osd.3 since back 2024-04-07T17:52:10.732559+0000 front 2024-04-07T17:52:10.732451+0000 (oldest deadline 2024-04-07T17:52:34.832010+0000) 2024-04-07T17:52:44.365 INFO:journalctl@ceph.osd.7.smithi144.stdout:Apr 07 17:52:43 smithi144 bash[84215]: debug 2024-04-07T17:52:43.861+0000 7f786bcc4700 -1 osd.7 2716 heartbeat_check: no reply from 172.21.15.144:6804 osd.4 since back 2024-04-07T17:52:20.133498+0000 front 2024-04-07T17:52:20.133605+0000 (oldest deadline 2024-04-07T17:52:43.033084+0000) 2024-04-07T17:52:45.114 INFO:journalctl@ceph.osd.7.smithi144.stdout:Apr 07 17:52:44 smithi144 bash[84215]: debug 2024-04-07T17:52:44.841+0000 7f786bcc4700 -1 osd.7 2716 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-04-07T17:51:46.828238+0000 front 2024-04-07T17:51:46.828037+0000 (oldest deadline 2024-04-07T17:52:12.727582+0000) 2024-04-07T17:52:45.115 INFO:journalctl@ceph.osd.7.smithi144.stdout:Apr 07 17:52:44 smithi144 bash[84215]: debug 2024-04-07T17:52:44.841+0000 7f786bcc4700 -1 osd.7 2716 heartbeat_check: no reply from 172.21.15.100:6814 osd.1 since back 2024-04-07T17:51:55.029147+0000 front 2024-04-07T17:51:55.029247+0000 (oldest deadline 2024-04-07T17:52:17.928736+0000) 2024-04-07T17:52:45.115 INFO:journalctl@ceph.osd.7.smithi144.stdout:Apr 07 17:52:44 smithi144 bash[84215]: debug 2024-04-07T17:52:44.841+0000 7f786bcc4700 -1 osd.7 2716 heartbeat_check: no reply from 172.21.15.100:6822 osd.2 since back 2024-04-07T17:52:03.131185+0000 front 2024-04-07T17:52:03.131256+0000 (oldest deadline 2024-04-07T17:52:27.830667+0000) 2024-04-07T17:52:45.115 INFO:journalctl@ceph.osd.7.smithi144.stdout:Apr 07 17:52:44 smithi144 bash[84215]: debug 2024-04-07T17:52:44.841+0000 7f786bcc4700 -1 osd.7 2716 heartbeat_check: no reply from 172.21.15.100:6830 osd.3 since back 2024-04-07T17:52:10.732559+0000 front 2024-04-07T17:52:10.732451+0000 (oldest deadline 2024-04-07T17:52:34.832010+0000) 2024-04-07T17:52:45.115 INFO:journalctl@ceph.osd.7.smithi144.stdout:Apr 07 17:52:44 smithi144 bash[84215]: debug 2024-04-07T17:52:44.841+0000 7f786bcc4700 -1 osd.7 2716 heartbeat_check: no reply from 172.21.15.144:6804 osd.4 since back 2024-04-07T17:52:20.133498+0000 front 2024-04-07T17:52:20.133605+0000 (oldest deadline 2024-04-07T17:52:43.033084+0000) 2024-04-07T17:52:46.114 INFO:journalctl@ceph.osd.7.smithi144.stdout:Apr 07 17:52:45 smithi144 bash[84215]: debug 2024-04-07T17:52:45.841+0000 7f786bcc4700 -1 osd.7 2716 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-04-07T17:51:46.828238+0000 front 2024-04-07T17:51:46.828037+0000 (oldest deadline 2024-04-07T17:52:12.727582+0000) 2024-04-07T17:52:46.115 INFO:journalctl@ceph.osd.7.smithi144.stdout:Apr 07 17:52:45 smithi144 bash[84215]: debug 2024-04-07T17:52:45.841+0000 7f786bcc4700 -1 osd.7 2716 heartbeat_check: no reply from 172.21.15.100:6814 osd.1 since back 2024-04-07T17:51:55.029147+0000 front 2024-04-07T17:51:55.029247+0000 (oldest deadline 2024-04-07T17:52:17.928736+0000) 2024-04-07T17:52:46.115 INFO:journalctl@ceph.osd.7.smithi144.stdout:Apr 07 17:52:45 smithi144 bash[84215]: debug 2024-04-07T17:52:45.841+0000 7f786bcc4700 -1 osd.7 2716 heartbeat_check: no reply from 172.21.15.100:6822 osd.2 since back 2024-04-07T17:52:03.131185+0000 front 2024-04-07T17:52:03.131256+0000 (oldest deadline 2024-04-07T17:52:27.830667+0000) 2024-04-07T17:52:46.115 INFO:journalctl@ceph.osd.7.smithi144.stdout:Apr 07 17:52:45 smithi144 bash[84215]: debug 2024-04-07T17:52:45.841+0000 7f786bcc4700 -1 osd.7 2716 heartbeat_check: no reply from 172.21.15.100:6830 osd.3 since back 2024-04-07T17:52:10.732559+0000 front 2024-04-07T17:52:10.732451+0000 (oldest deadline 2024-04-07T17:52:34.832010+0000) 2024-04-07T17:52:46.115 INFO:journalctl@ceph.osd.7.smithi144.stdout:Apr 07 17:52:45 smithi144 bash[84215]: debug 2024-04-07T17:52:45.841+0000 7f786bcc4700 -1 osd.7 2716 heartbeat_check: no reply from 172.21.15.144:6804 osd.4 since back 2024-04-07T17:52:20.133498+0000 front 2024-04-07T17:52:20.133605+0000 (oldest deadline 2024-04-07T17:52:43.033084+0000) 2024-04-07T17:52:48.114 INFO:journalctl@ceph.osd.7.smithi144.stdout:Apr 07 17:52:47 smithi144 bash[114406]: ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4-osd-7 2024-04-07T17:52:48.115 INFO:journalctl@ceph.osd.7.smithi144.stdout:Apr 07 17:52:47 smithi144 bash[114468]: Error response from daemon: No such container: ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4-osd-7 2024-04-07T17:52:49.485 DEBUG:teuthology.orchestra.run.smithi144:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@osd.7.service' 2024-04-07T17:52:49.510 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-07T17:52:49.511 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-04-07T17:52:49.511 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-04-07T17:52:49.511 DEBUG:teuthology.orchestra.run.smithi100:> sudo systemctl stop ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@mds.a 2024-04-07T17:52:49.953 INFO:journalctl@ceph.mds.a.smithi100.stdout:Apr 07 17:52:49 smithi100 systemd[1]: Stopping Ceph mds.a for 0702e3a8-f4f8-11ee-b649-cb9ed24678a4... 2024-04-07T17:52:49.953 INFO:journalctl@ceph.mds.a.smithi100.stdout:Apr 07 17:52:49 smithi100 bash[98288]: debug 2024-04-07T17:52:49.664+0000 7fe521e93700 -1 received signal: Terminated from /sbin/docker-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug (PID: 1) UID: 0 2024-04-07T17:52:49.953 INFO:journalctl@ceph.mds.a.smithi100.stdout:Apr 07 17:52:49 smithi100 bash[98288]: debug 2024-04-07T17:52:49.664+0000 7fe521e93700 -1 mds.a *** got signal Terminated *** 2024-04-07T17:52:50.703 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Apr 07 17:52:50 smithi100 bash[105121]: ts=2024-04-07T17:52:50.249Z caller=dispatch.go:352 level=error component=dispatcher msg="Notify for alerts failed" num_alerts=1 err="ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://smithi100.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.100:8443: connect: connection refused" 2024-04-07T17:52:50.703 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Apr 07 17:52:50 smithi100 bash[105121]: ts=2024-04-07T17:52:50.250Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi100.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.100:8443: connect: connection refused" 2024-04-07T17:52:53.068 DEBUG:teuthology.orchestra.run.smithi100:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@mds.a.service' 2024-04-07T17:52:53.089 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-07T17:52:53.090 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-04-07T17:52:53.090 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-04-07T17:52:53.090 DEBUG:teuthology.orchestra.run.smithi144:> sudo systemctl stop ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@mds.b 2024-04-07T17:52:53.364 INFO:journalctl@ceph.mds.b.smithi144.stdout:Apr 07 17:52:53 smithi144 systemd[1]: Stopping Ceph mds.b for 0702e3a8-f4f8-11ee-b649-cb9ed24678a4... 2024-04-07T17:52:53.365 INFO:journalctl@ceph.mds.b.smithi144.stdout:Apr 07 17:52:53 smithi144 bash[88204]: debug 2024-04-07T17:52:53.236+0000 7ff362aea700 -1 received signal: Terminated from /sbin/docker-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug (PID: 1) UID: 0 2024-04-07T17:52:53.365 INFO:journalctl@ceph.mds.b.smithi144.stdout:Apr 07 17:52:53 smithi144 bash[88204]: debug 2024-04-07T17:52:53.236+0000 7ff362aea700 -1 mds.b *** got signal Terminated *** 2024-04-07T17:52:55.929 DEBUG:teuthology.orchestra.run.smithi144:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@mds.b.service' 2024-04-07T17:52:55.951 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-07T17:52:55.952 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-04-07T17:52:55.952 INFO:tasks.cephadm.prometheus.a:Stopping prometheus.a... 2024-04-07T17:52:55.952 DEBUG:teuthology.orchestra.run.smithi144:> sudo systemctl stop ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@prometheus.a 2024-04-07T17:52:56.181 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:52:56 smithi144 systemd[1]: Stopping Ceph prometheus.a for 0702e3a8-f4f8-11ee-b649-cb9ed24678a4... 2024-04-07T17:52:56.181 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:52:56 smithi144 bash[95233]: ts=2024-04-07T17:52:56.138Z caller=main.go:840 level=warn msg="Received SIGTERM, exiting gracefully..." 2024-04-07T17:52:56.181 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:52:56 smithi144 bash[95233]: ts=2024-04-07T17:52:56.138Z caller=main.go:864 level=info msg="Stopping scrape discovery manager..." 2024-04-07T17:52:56.181 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:52:56 smithi144 bash[95233]: ts=2024-04-07T17:52:56.138Z caller=main.go:878 level=info msg="Stopping notify discovery manager..." 2024-04-07T17:52:56.181 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:52:56 smithi144 bash[95233]: ts=2024-04-07T17:52:56.138Z caller=manager.go:988 level=info component="rule manager" msg="Stopping rule manager..." 2024-04-07T17:52:56.181 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:52:56 smithi144 bash[95233]: ts=2024-04-07T17:52:56.138Z caller=main.go:860 level=info msg="Scrape discovery manager stopped" 2024-04-07T17:52:56.181 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:52:56 smithi144 bash[95233]: ts=2024-04-07T17:52:56.138Z caller=main.go:874 level=info msg="Notify discovery manager stopped" 2024-04-07T17:52:56.181 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:52:56 smithi144 bash[95233]: ts=2024-04-07T17:52:56.138Z caller=manager.go:998 level=info component="rule manager" msg="Rule manager stopped" 2024-04-07T17:52:56.181 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:52:56 smithi144 bash[95233]: ts=2024-04-07T17:52:56.138Z caller=main.go:915 level=info msg="Stopping scrape manager..." 2024-04-07T17:52:56.181 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:52:56 smithi144 bash[95233]: ts=2024-04-07T17:52:56.138Z caller=main.go:907 level=info msg="Scrape manager stopped" 2024-04-07T17:52:56.182 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:52:56 smithi144 bash[95233]: ts=2024-04-07T17:52:56.141Z caller=notifier.go:606 level=info component=notifier msg="Stopping notification manager..." 2024-04-07T17:52:56.182 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:52:56 smithi144 bash[95233]: ts=2024-04-07T17:52:56.141Z caller=main.go:1135 level=info msg="Notifier manager stopped" 2024-04-07T17:52:56.182 INFO:journalctl@ceph.prometheus.a.smithi144.stdout:Apr 07 17:52:56 smithi144 bash[95233]: ts=2024-04-07T17:52:56.141Z caller=main.go:1147 level=info msg="See you next time!" 2024-04-07T17:52:56.843 DEBUG:teuthology.orchestra.run.smithi144:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@prometheus.a.service' 2024-04-07T17:52:56.867 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-07T17:52:56.867 INFO:tasks.cephadm.prometheus.a:Stopped prometheus.a 2024-04-07T17:52:56.867 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 0702e3a8-f4f8-11ee-b649-cb9ed24678a4 --force --keep-logs 2024-04-07T17:52:56.988 INFO:teuthology.orchestra.run.smithi100.stdout:Deleting cluster with fsid: 0702e3a8-f4f8-11ee-b649-cb9ed24678a4 2024-04-07T17:53:00.703 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Apr 07 17:53:00 smithi100 bash[105121]: ts=2024-04-07T17:53:00.250Z caller=dispatch.go:352 level=error component=dispatcher msg="Notify for alerts failed" num_alerts=1 err="ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://smithi100.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.100:8443: connect: connection refused" 2024-04-07T17:53:00.703 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Apr 07 17:53:00 smithi100 bash[105121]: ts=2024-04-07T17:53:00.251Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi100.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.100:8443: connect: connection refused" 2024-04-07T17:53:01.856 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 17:53:01 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T17:53:01.856 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Apr 07 17:53:01 smithi100 bash[105121]: ts=2024-04-07T17:53:01.624Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi100.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.100:8443: connect: connection refused" 2024-04-07T17:53:01.856 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Apr 07 17:53:01 smithi100 bash[105121]: ts=2024-04-07T17:53:01.626Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi144.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.144:8443: connect: connection refused" 2024-04-07T17:53:01.856 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Apr 07 17:53:01 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T17:53:02.203 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 17:53:01 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T17:53:02.203 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Apr 07 17:53:01 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T17:53:02.518 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Apr 07 17:53:02 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T17:53:02.518 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Apr 07 17:53:02 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T17:53:02.518 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 17:53:02 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T17:53:02.518 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 17:53:02 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T17:53:02.793 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 17:53:02 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T17:53:02.793 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Apr 07 17:53:02 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T17:53:03.129 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 17:53:02 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T17:53:03.130 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Apr 07 17:53:02 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T17:53:03.397 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Apr 07 17:53:03 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T17:53:03.398 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Apr 07 17:53:03 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T17:53:03.398 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 17:53:03 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T17:53:03.398 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 17:53:03 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T17:53:03.703 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 17:53:03 smithi100 systemd[1]: Stopping Ceph node-exporter.a for 0702e3a8-f4f8-11ee-b649-cb9ed24678a4... 2024-04-07T17:53:04.412 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 17:53:04 smithi100 bash[175842]: ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4-node-exporter-a 2024-04-07T17:53:04.412 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 17:53:04 smithi100 bash[175904]: ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4-node-exporter-a 2024-04-07T17:53:04.412 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 17:53:04 smithi100 systemd[1]: ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@node-exporter.a.service: Main process exited, code=exited, status=143/n/a 2024-04-07T17:53:04.412 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 17:53:04 smithi100 systemd[1]: ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@node-exporter.a.service: Failed with result 'exit-code'. 2024-04-07T17:53:04.412 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 17:53:04 smithi100 systemd[1]: Stopped Ceph node-exporter.a for 0702e3a8-f4f8-11ee-b649-cb9ed24678a4. 2024-04-07T17:53:04.703 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Apr 07 17:53:04 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T17:53:04.703 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Apr 07 17:53:04 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T17:53:04.703 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Apr 07 17:53:04 smithi100 systemd[1]: Stopping Ceph alertmanager.a for 0702e3a8-f4f8-11ee-b649-cb9ed24678a4... 2024-04-07T17:53:04.704 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Apr 07 17:53:04 smithi100 bash[105121]: ts=2024-04-07T17:53:04.601Z caller=main.go:583 level=info msg="Received SIGTERM, exiting gracefully..." 2024-04-07T17:53:05.452 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Apr 07 17:53:05 smithi100 bash[175969]: ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4-alertmanager-a 2024-04-07T17:53:05.453 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Apr 07 17:53:05 smithi100 bash[176030]: Error response from daemon: No such container: ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4-alertmanager-a 2024-04-07T17:53:05.453 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Apr 07 17:53:05 smithi100 systemd[1]: ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@alertmanager.a.service: Deactivated successfully. 2024-04-07T17:53:05.453 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Apr 07 17:53:05 smithi100 systemd[1]: Stopped Ceph alertmanager.a for 0702e3a8-f4f8-11ee-b649-cb9ed24678a4. 2024-04-07T17:53:05.806 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Apr 07 17:53:05 smithi100 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T17:53:06.094 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 0702e3a8-f4f8-11ee-b649-cb9ed24678a4 --force --keep-logs 2024-04-07T17:53:06.219 INFO:teuthology.orchestra.run.smithi144.stdout:Deleting cluster with fsid: 0702e3a8-f4f8-11ee-b649-cb9ed24678a4 2024-04-07T17:53:11.114 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 17:53:10 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T17:53:11.115 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 17:53:10 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T17:53:11.430 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 17:53:11 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T17:53:11.430 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 17:53:11 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T17:53:11.692 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 17:53:11 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T17:53:11.692 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 17:53:11 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T17:53:11.942 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 17:53:11 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T17:53:11.942 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 17:53:11 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T17:53:11.942 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 17:53:11 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T17:53:12.257 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 17:53:12 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T17:53:12.257 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 17:53:11 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T17:53:12.257 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 17:53:12 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T17:53:12.528 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 17:53:12 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T17:53:12.528 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 17:53:12 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T17:53:12.850 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 17:53:12 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T17:53:12.850 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 17:53:12 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T17:53:12.850 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 17:53:12 smithi144 systemd[1]: Stopping Ceph node-exporter.b for 0702e3a8-f4f8-11ee-b649-cb9ed24678a4... 2024-04-07T17:53:13.614 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 17:53:13 smithi144 bash[115057]: ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4-node-exporter-b 2024-04-07T17:53:13.614 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 17:53:13 smithi144 systemd[1]: ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@node-exporter.b.service: Main process exited, code=exited, status=143/n/a 2024-04-07T17:53:13.614 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 17:53:13 smithi144 bash[115120]: Error response from daemon: No such container: ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4-node-exporter-b 2024-04-07T17:53:13.615 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 17:53:13 smithi144 systemd[1]: ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@node-exporter.b.service: Failed with result 'exit-code'. 2024-04-07T17:53:13.615 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 17:53:13 smithi144 systemd[1]: Stopped Ceph node-exporter.b for 0702e3a8-f4f8-11ee-b649-cb9ed24678a4. 2024-04-07T17:53:13.966 INFO:journalctl@ceph.node-exporter.b.smithi144.stdout:Apr 07 17:53:13 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T17:53:13.966 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 17:53:13 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T17:53:13.966 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 17:53:13 smithi144 systemd[1]: Stopping Ceph grafana.a for 0702e3a8-f4f8-11ee-b649-cb9ed24678a4... 2024-04-07T17:53:13.966 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 17:53:13 smithi144 bash[98866]: logger=server t=2024-04-07T17:53:13.930568099Z level=info msg="Shutdown started" reason="System signal: terminated" 2024-04-07T17:53:13.966 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 17:53:13 smithi144 bash[98866]: logger=tracing t=2024-04-07T17:53:13.930637573Z level=info msg="Closing tracing" 2024-04-07T17:53:13.966 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 17:53:13 smithi144 bash[98866]: logger=ticker t=2024-04-07T17:53:13.930665287Z level=info msg=stopped last_tick=2024-04-07T17:53:10Z 2024-04-07T17:53:14.792 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 17:53:14 smithi144 bash[115182]: ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4-grafana-a 2024-04-07T17:53:14.792 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 17:53:14 smithi144 bash[115243]: Error response from daemon: No such container: ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4-grafana-a 2024-04-07T17:53:14.792 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 17:53:14 smithi144 systemd[1]: ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@grafana.a.service: Deactivated successfully. 2024-04-07T17:53:14.792 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 17:53:14 smithi144 systemd[1]: Stopped Ceph grafana.a for 0702e3a8-f4f8-11ee-b649-cb9ed24678a4. 2024-04-07T17:53:14.792 INFO:journalctl@ceph.grafana.a.smithi144.stdout:Apr 07 17:53:14 smithi144 systemd[1]: /etc/systemd/system/ceph-0702e3a8-f4f8-11ee-b649-cb9ed24678a4@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-04-07T17:53:15.315 DEBUG:teuthology.orchestra.run.smithi100:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-07T17:53:15.329 DEBUG:teuthology.orchestra.run.smithi144:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-07T17:53:15.341 INFO:tasks.cephadm:Archiving crash dumps... 2024-04-07T17:53:15.343 DEBUG:teuthology.misc:Transferring archived files from smithi100:/var/lib/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/crash to /home/teuthworker/archive/yuriw-2024-04-07_14:37:12-rados-wip-yuri3-testing-2024-04-05-0825-distro-default-smithi/7643840/remote/smithi100/crash 2024-04-07T17:53:15.344 DEBUG:teuthology.orchestra.run.smithi100:> sudo tar c -f - -C /var/lib/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/crash -- . 2024-04-07T17:53:15.382 INFO:teuthology.orchestra.run.smithi100.stderr:tar: /var/lib/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/crash: Cannot open: No such file or directory 2024-04-07T17:53:15.382 INFO:teuthology.orchestra.run.smithi100.stderr:tar: Error is not recoverable: exiting now 2024-04-07T17:53:15.384 DEBUG:teuthology.misc:Transferring archived files from smithi144:/var/lib/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/crash to /home/teuthworker/archive/yuriw-2024-04-07_14:37:12-rados-wip-yuri3-testing-2024-04-05-0825-distro-default-smithi/7643840/remote/smithi144/crash 2024-04-07T17:53:15.385 DEBUG:teuthology.orchestra.run.smithi144:> sudo tar c -f - -C /var/lib/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/crash -- . 2024-04-07T17:53:15.394 INFO:teuthology.orchestra.run.smithi144.stderr:tar: /var/lib/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/crash: Cannot open: No such file or directory 2024-04-07T17:53:15.394 INFO:teuthology.orchestra.run.smithi144.stderr:tar: Error is not recoverable: exiting now 2024-04-07T17:53:15.395 INFO:tasks.cephadm:Checking cluster log for badness... 2024-04-07T17:53:15.396 DEBUG:teuthology.orchestra.run.smithi100:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | head -n 1 2024-04-07T17:53:15.440 INFO:teuthology.orchestra.run.smithi100.stdout:2024-04-07T16:22:06.783529+0000 mon.a (mon.0) 359 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN) 2024-04-07T17:53:15.443 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-04-07T17:53:15.443 DEBUG:teuthology.orchestra.run.smithi100:> sudo egrep '\[SEC\]' /var/log/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | head -n 1 2024-04-07T17:53:15.502 DEBUG:teuthology.orchestra.run.smithi100:> sudo egrep '\[ERR\]' /var/log/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | head -n 1 2024-04-07T17:53:15.562 DEBUG:teuthology.orchestra.run.smithi100:> sudo egrep '\[WRN\]' /var/log/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | head -n 1 2024-04-07T17:53:15.614 INFO:teuthology.orchestra.run.smithi100.stdout:2024-04-07T16:22:06.783529+0000 mon.a (mon.0) 359 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN) 2024-04-07T17:53:15.616 INFO:tasks.cephadm:Compressing logs... 2024-04-07T17:53:15.616 DEBUG:teuthology.orchestra.run.smithi100:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-07T17:53:15.665 DEBUG:teuthology.orchestra.run.smithi144:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-07T17:53:15.674 INFO:teuthology.orchestra.run.smithi100.stderr:find: '/var/log/rbd-target-api': No such file or directory 2024-04-07T17:53:15.677 INFO:teuthology.orchestra.run.smithi144.stderr:find: '/var/log/rbd-target-api': No such file or directory 2024-04-07T17:53:15.686 INFO:teuthology.orchestra.run.smithi144.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-07T17:53:15.687 INFO:teuthology.orchestra.run.smithi144.stderr:gzip -5 --verbose -- /var/log/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/ceph-osd.7.log 2024-04-07T17:53:15.688 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-07T17:53:15.688 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/ceph-mgr.y.log 2024-04-07T17:53:15.688 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/ceph-osd.0.log 2024-04-07T17:53:15.689 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/ceph-osd.6.log 2024-04-07T17:53:15.689 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/ceph.cephadm.log 2024-04-07T17:53:15.689 INFO:teuthology.orchestra.run.smithi144.stderr:gzip -5 --verbose -- /var/log/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/ceph-osd.5.log 2024-04-07T17:53:15.690 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/ceph-osd.6.log: /var/log/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/ceph-mgr.x.log 2024-04-07T17:53:15.690 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/ceph.log 2024-04-07T17:53:15.690 INFO:teuthology.orchestra.run.smithi144.stderr: 84.1% -- replaced with /var/log/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/ceph.cephadm.log.gz 2024-04-07T17:53:15.690 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/ceph-volume.log 2024-04-07T17:53:15.692 INFO:teuthology.orchestra.run.smithi144.stderr:gzip/var/log/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/ceph.log: -5 --verbose -- /var/log/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/ceph.audit.log 2024-04-07T17:53:15.692 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/ceph-mds.b.log 2024-04-07T17:53:15.694 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/ceph-osd.2.log 2024-04-07T17:53:15.694 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/ceph-mon.a.log 2024-04-07T17:53:15.694 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/ceph.cephadm.log 2024-04-07T17:53:15.694 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/ceph-osd.3.log 2024-04-07T17:53:15.694 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/ceph-osd.1.log 2024-04-07T17:53:15.694 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/ceph.log 2024-04-07T17:53:15.694 INFO:teuthology.orchestra.run.smithi100.stderr: 90.6% -- replaced with /var/log/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/ceph.cephadm.log.gz 2024-04-07T17:53:15.694 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/ceph-volume.log 2024-04-07T17:53:15.696 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/ceph.log: 90.6% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-07T17:53:15.696 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/ceph-mon.c.log 2024-04-07T17:53:15.700 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/ceph.audit.log: 91.4% -- replaced with /var/log/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/ceph.audit.log.gz 2024-04-07T17:53:15.701 INFO:teuthology.orchestra.run.smithi144.stderr:gzip -5 --verbose -- /var/log/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/ceph-mon.b.log 2024-04-07T17:53:15.702 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/ceph-mds.b.log: 85.9% -- replaced with /var/log/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/ceph-mds.b.log.gz 2024-04-07T17:53:15.702 INFO:teuthology.orchestra.run.smithi144.stderr:gzip -5 --verbose -- /var/log/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/ceph-osd.4.log 2024-04-07T17:53:15.711 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/ceph.audit.log 2024-04-07T17:53:15.713 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/ceph-mon.b.log: /var/log/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/ceph-osd.4.log: 87.0% -- replaced with /var/log/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/ceph.log.gz 2024-04-07T17:53:15.713 INFO:teuthology.orchestra.run.smithi144.stderr: 89.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-07T17:53:15.716 INFO:teuthology.orchestra.run.smithi144.stderr: 91.2% -- replaced with /var/log/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/ceph-volume.log.gz 2024-04-07T17:53:15.721 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/ceph-mon.c.log: 92.8% -- replaced with /var/log/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/ceph.log.gz 2024-04-07T17:53:15.725 INFO:teuthology.orchestra.run.smithi100.stderr: 91.2% -- replaced with /var/log/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/ceph-volume.log.gz 2024-04-07T17:53:15.725 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/ceph-mds.a.log 2024-04-07T17:53:15.743 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/ceph.audit.log: 94.8% -- replaced with /var/log/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/ceph.audit.log.gz 2024-04-07T17:53:15.761 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/ceph-mds.a.log: 92.3% -- replaced with /var/log/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/ceph-mds.a.log.gz 2024-04-07T17:53:15.763 INFO:teuthology.orchestra.run.smithi144.stderr: 90.9% -- replaced with /var/log/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/ceph-mgr.x.log.gz 2024-04-07T17:53:16.899 INFO:teuthology.orchestra.run.smithi100.stderr: 89.9% -- replaced with /var/log/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/ceph-mgr.y.log.gz 2024-04-07T17:53:17.247 INFO:teuthology.orchestra.run.smithi144.stderr: 92.3% -- replaced with /var/log/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/ceph-mon.b.log.gz 2024-04-07T17:53:18.105 INFO:teuthology.orchestra.run.smithi100.stderr: 92.2% -- replaced with /var/log/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/ceph-mon.c.log.gz 2024-04-07T17:53:19.826 INFO:teuthology.orchestra.run.smithi100.stderr: 90.6% -- replaced with /var/log/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/ceph-mon.a.log.gz 2024-04-07T17:53:34.675 INFO:teuthology.orchestra.run.smithi144.stderr: 94.5% -- replaced with /var/log/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/ceph-osd.6.log.gz 2024-04-07T17:53:35.729 INFO:teuthology.orchestra.run.smithi144.stderr: 94.2% -- replaced with /var/log/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/ceph-osd.4.log.gz 2024-04-07T17:53:38.106 INFO:teuthology.orchestra.run.smithi144.stderr: 94.4% -- replaced with /var/log/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/ceph-osd.7.log.gz 2024-04-07T17:53:41.006 INFO:teuthology.orchestra.run.smithi100.stderr: 94.4% -- replaced with /var/log/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/ceph-osd.1.log.gz 2024-04-07T17:53:41.015 INFO:teuthology.orchestra.run.smithi144.stderr: 94.8% -- replaced with /var/log/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/ceph-osd.5.log.gz 2024-04-07T17:53:41.017 INFO:teuthology.orchestra.run.smithi144.stderr: 2024-04-07T17:53:41.017 INFO:teuthology.orchestra.run.smithi144.stderr:real 0m25.347s 2024-04-07T17:53:41.017 INFO:teuthology.orchestra.run.smithi144.stderr:user 1m25.316s 2024-04-07T17:53:41.017 INFO:teuthology.orchestra.run.smithi144.stderr:sys 0m3.152s 2024-04-07T17:53:41.133 INFO:teuthology.orchestra.run.smithi100.stderr: 94.6% -- replaced with /var/log/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/ceph-osd.3.log.gz 2024-04-07T17:53:41.741 INFO:teuthology.orchestra.run.smithi100.stderr: 94.4% -- replaced with /var/log/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/ceph-osd.0.log.gz 2024-04-07T17:53:42.303 INFO:teuthology.orchestra.run.smithi100.stderr: 94.0% -- replaced with /var/log/ceph/0702e3a8-f4f8-11ee-b649-cb9ed24678a4/ceph-osd.2.log.gz 2024-04-07T17:53:42.304 INFO:teuthology.orchestra.run.smithi100.stderr: 2024-04-07T17:53:42.304 INFO:teuthology.orchestra.run.smithi100.stderr:real 0m26.638s 2024-04-07T17:53:42.304 INFO:teuthology.orchestra.run.smithi100.stderr:user 1m47.272s 2024-04-07T17:53:42.304 INFO:teuthology.orchestra.run.smithi100.stderr:sys 0m3.946s 2024-04-07T17:53:42.305 INFO:tasks.cephadm:Archiving logs... 2024-04-07T17:53:42.305 DEBUG:teuthology.misc:Transferring archived files from smithi100:/var/log/ceph to /home/teuthworker/archive/yuriw-2024-04-07_14:37:12-rados-wip-yuri3-testing-2024-04-05-0825-distro-default-smithi/7643840/remote/smithi100/log 2024-04-07T17:53:42.307 DEBUG:teuthology.orchestra.run.smithi100:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-07T17:53:54.714 DEBUG:teuthology.misc:Transferring archived files from smithi144:/var/log/ceph to /home/teuthworker/archive/yuriw-2024-04-07_14:37:12-rados-wip-yuri3-testing-2024-04-05-0825-distro-default-smithi/7643840/remote/smithi144/log 2024-04-07T17:53:54.715 DEBUG:teuthology.orchestra.run.smithi144:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-07T17:54:04.495 INFO:tasks.cephadm:Removing cluster... 2024-04-07T17:54:04.495 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 0702e3a8-f4f8-11ee-b649-cb9ed24678a4 --force 2024-04-07T17:54:04.620 INFO:teuthology.orchestra.run.smithi100.stdout:Deleting cluster with fsid: 0702e3a8-f4f8-11ee-b649-cb9ed24678a4 2024-04-07T17:54:06.097 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 0702e3a8-f4f8-11ee-b649-cb9ed24678a4 --force 2024-04-07T17:54:06.218 INFO:teuthology.orchestra.run.smithi144.stdout:Deleting cluster with fsid: 0702e3a8-f4f8-11ee-b649-cb9ed24678a4 2024-04-07T17:54:07.705 INFO:tasks.cephadm:Removing cephadm ... 2024-04-07T17:54:07.706 DEBUG:teuthology.orchestra.run.smithi100:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-07T17:54:07.712 DEBUG:teuthology.orchestra.run.smithi144:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-07T17:54:07.718 INFO:tasks.cephadm:Teardown complete 2024-04-07T17:54:07.718 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-07T17:54:07.731 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-07T17:54:07.732 DEBUG:teuthology.orchestra.run.smithi100:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-07T17:54:07.756 DEBUG:teuthology.orchestra.run.smithi144:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-07T17:54:07.824 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-07T17:54:07.834 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-07T17:54:07.834 DEBUG:teuthology.orchestra.run.smithi100:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-07T17:54:07.837 DEBUG:teuthology.orchestra.run.smithi144:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-07T17:54:07.853 INFO:teuthology.orchestra.run.smithi144.stdout: remote refid st t when poll reach delay offset jitter 2024-04-07T17:54:07.853 INFO:teuthology.orchestra.run.smithi144.stdout:============================================================================== 2024-04-07T17:54:07.853 INFO:teuthology.orchestra.run.smithi144.stdout:*hv01.front.sepi 44.4.53.2 2 u 65 256 377 0.159 -0.750 1.291 2024-04-07T17:54:07.853 INFO:teuthology.orchestra.run.smithi144.stdout:+hv02.front.sepi 63.231.80.2 3 u 128 256 377 0.187 -2.510 0.606 2024-04-07T17:54:07.853 INFO:teuthology.orchestra.run.smithi144.stdout:+hv03.front.sepi 74.6.168.72 3 u 48 256 377 0.174 -3.092 1.056 2024-04-07T17:54:07.853 INFO:teuthology.orchestra.run.smithi144.stdout: hv04.front.sepi .INIT. 16 u - 1024 0 0.000 +0.000 0.000 2024-04-07T17:54:07.854 INFO:teuthology.orchestra.run.smithi100.stdout: remote refid st t when poll reach delay offset jitter 2024-04-07T17:54:07.854 INFO:teuthology.orchestra.run.smithi100.stdout:============================================================================== 2024-04-07T17:54:07.854 INFO:teuthology.orchestra.run.smithi100.stdout:*hv01.front.sepi 44.4.53.2 2 u 48 128 377 0.192 +0.584 0.626 2024-04-07T17:54:07.854 INFO:teuthology.orchestra.run.smithi100.stdout:+hv02.front.sepi 63.231.80.2 3 u 113 128 377 0.134 -0.537 0.623 2024-04-07T17:54:07.854 INFO:teuthology.orchestra.run.smithi100.stdout:+hv03.front.sepi 74.6.168.72 3 u 97 128 377 0.163 -0.556 0.514 2024-04-07T17:54:07.854 INFO:teuthology.orchestra.run.smithi100.stdout: hv04.front.sepi .INIT. 16 u - 1024 0 0.000 +0.000 0.000 2024-04-07T17:54:07.855 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-07T17:54:07.865 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-07T17:54:07.866 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-07T17:54:07.881 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-07T17:54:07.897 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-07T17:54:07.913 INFO:teuthology.task.internal:Duration was 7316.675312 seconds 2024-04-07T17:54:07.913 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-07T17:54:08.004 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-07T17:54:08.004 DEBUG:teuthology.orchestra.run.smithi100:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-07T17:54:08.007 DEBUG:teuthology.orchestra.run.smithi144:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-07T17:54:08.043 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-07T17:54:08.043 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi100.front.sepia.ceph.com 2024-04-07T17:54:08.043 DEBUG:teuthology.orchestra.run.smithi100:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-07T17:54:08.092 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi144.front.sepia.ceph.com 2024-04-07T17:54:08.092 DEBUG:teuthology.orchestra.run.smithi144:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-07T17:54:08.102 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-07T17:54:08.102 DEBUG:teuthology.orchestra.run.smithi100:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-07T17:54:08.136 DEBUG:teuthology.orchestra.run.smithi144:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-07T17:54:08.236 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-07T17:54:08.236 DEBUG:teuthology.orchestra.run.smithi100:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-07T17:54:08.239 DEBUG:teuthology.orchestra.run.smithi144:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-07T17:54:08.556 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-07T17:54:08.568 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-07T17:54:08.568 DEBUG:teuthology.orchestra.run.smithi100:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-07T17:54:08.580 DEBUG:teuthology.orchestra.run.smithi144:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-07T17:54:08.594 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-07T17:54:08.606 DEBUG:teuthology.orchestra.run.smithi100:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-07T17:54:08.624 DEBUG:teuthology.orchestra.run.smithi144:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-07T17:54:08.632 INFO:teuthology.orchestra.run.smithi100.stdout:kernel.core_pattern = core 2024-04-07T17:54:08.648 INFO:teuthology.orchestra.run.smithi144.stdout:kernel.core_pattern = core 2024-04-07T17:54:08.661 DEBUG:teuthology.orchestra.run.smithi100:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-07T17:54:08.694 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-07T17:54:08.695 DEBUG:teuthology.orchestra.run.smithi144:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-07T17:54:08.705 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-07T17:54:08.705 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-07T17:54:08.717 INFO:teuthology.task.internal:Transferring archived files... 2024-04-07T17:54:08.718 DEBUG:teuthology.misc:Transferring archived files from smithi100:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2024-04-07_14:37:12-rados-wip-yuri3-testing-2024-04-05-0825-distro-default-smithi/7643840/remote/smithi100 2024-04-07T17:54:08.718 DEBUG:teuthology.orchestra.run.smithi100:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-07T17:54:08.772 DEBUG:teuthology.misc:Transferring archived files from smithi144:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2024-04-07_14:37:12-rados-wip-yuri3-testing-2024-04-05-0825-distro-default-smithi/7643840/remote/smithi144 2024-04-07T17:54:08.772 DEBUG:teuthology.orchestra.run.smithi144:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-07T17:54:08.804 INFO:teuthology.task.internal:Removing archive directory... 2024-04-07T17:54:08.804 DEBUG:teuthology.orchestra.run.smithi100:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-07T17:54:08.806 DEBUG:teuthology.orchestra.run.smithi144:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-07T17:54:08.847 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-07T17:54:08.860 INFO:teuthology.task.internal:Not uploading archives. 2024-04-07T17:54:08.860 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-07T17:54:08.872 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-07T17:54:08.872 DEBUG:teuthology.orchestra.run.smithi100:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-07T17:54:08.875 DEBUG:teuthology.orchestra.run.smithi144:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-07T17:54:08.879 INFO:teuthology.orchestra.run.smithi100.stdout: 658624 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 7 17:54 /home/ubuntu/cephtest 2024-04-07T17:54:08.895 INFO:teuthology.orchestra.run.smithi144.stdout: 658623 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 7 17:54 /home/ubuntu/cephtest 2024-04-07T17:54:08.897 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-07T17:54:08.909 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-07T17:54:08.941 INFO:teuthology.nuke:Checking targets against current locks 2024-04-07T17:54:08.966 DEBUG:teuthology.nuke:shortname: smithi100 2024-04-07T17:54:08.966 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-07T17:54:08.991 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi100.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-04-07_14:37:12-rados-wip-yuri3-testing-2024-04-05-0825-distro-default-smithi/7643840', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-07 15:43:44.207867', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMvCZrXBXfjZD3eHBRmkPE6vfHXACCa9JHtdsxjkOCe1gSulfAtRRHBJXmkZyuhdWpF+PnJbuepLZr/Ept4h9yA='} 2024-04-07T17:54:08.992 DEBUG:teuthology.nuke:shortname: smithi144 2024-04-07T17:54:08.993 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-07T17:54:09.017 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi144.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-04-07_14:37:12-rados-wip-yuri3-testing-2024-04-05-0825-distro-default-smithi/7643840', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-07 15:43:44.206274', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMeJBXxAI0FxiausJKk6WgkEXzjIjP/zQyK16+mbO2O273fwK1Lu67a5kMJb58Wd1VkWo/ZQpRXn+gclCx3dev0='} 2024-04-07T17:54:09.046 INFO:teuthology.orchestra.console.smithi100:Power off 2024-04-07T17:54:09.046 DEBUG:teuthology.orchestra.console.smithi100:pexpect command: ipmitool -H smithi100.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-07T17:54:09.065 INFO:teuthology.orchestra.console.smithi144:Power off 2024-04-07T17:54:09.065 DEBUG:teuthology.orchestra.console.smithi144:pexpect command: ipmitool -H smithi144.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-07T17:54:09.075 DEBUG:teuthology.orchestra.console.smithi100:power off output: Chassis Power Control: Down/Off 2024-04-07T17:54:09.075 DEBUG:teuthology.orchestra.console.smithi100:pexpect command: ipmitool -H smithi100.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-07T17:54:09.089 DEBUG:teuthology.orchestra.console.smithi144:power off output: Chassis Power Control: Down/Off 2024-04-07T17:54:09.089 DEBUG:teuthology.orchestra.console.smithi144:pexpect command: ipmitool -H smithi144.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-07T17:54:09.098 DEBUG:teuthology.orchestra.console.smithi100:check power output: Chassis Power is on 2024-04-07T17:54:09.114 DEBUG:teuthology.orchestra.console.smithi144:check power output: Chassis Power is on 2024-04-07T17:54:13.099 DEBUG:teuthology.orchestra.console.smithi100:pexpect command: ipmitool -H smithi100.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-07T17:54:13.114 DEBUG:teuthology.orchestra.console.smithi144:pexpect command: ipmitool -H smithi144.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-07T17:54:13.209 DEBUG:teuthology.orchestra.console.smithi100:check power output: Chassis Power is on 2024-04-07T17:54:13.224 DEBUG:teuthology.orchestra.console.smithi144:check power output: Chassis Power is on 2024-04-07T17:54:17.214 DEBUG:teuthology.orchestra.console.smithi100:pexpect command: ipmitool -H smithi100.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-07T17:54:17.225 DEBUG:teuthology.orchestra.console.smithi144:pexpect command: ipmitool -H smithi144.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-07T17:54:17.325 DEBUG:teuthology.orchestra.console.smithi100:check power output: Chassis Power is on 2024-04-07T17:54:17.335 DEBUG:teuthology.orchestra.console.smithi144:check power output: Chassis Power is on 2024-04-07T17:54:21.329 DEBUG:teuthology.orchestra.console.smithi100:pexpect command: ipmitool -H smithi100.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-07T17:54:21.337 DEBUG:teuthology.orchestra.console.smithi144:pexpect command: ipmitool -H smithi144.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-07T17:54:21.440 DEBUG:teuthology.orchestra.console.smithi100:check power output: Chassis Power is off 2024-04-07T17:54:21.446 DEBUG:teuthology.orchestra.console.smithi144:check power output: Chassis Power is off 2024-04-07T17:54:21.541 INFO:teuthology.orchestra.console.smithi100:Power off completed 2024-04-07T17:54:21.547 INFO:teuthology.orchestra.console.smithi144:Power off completed 2024-04-07T17:54:21.651 INFO:teuthology.run:Summary data: description: rados/upgrade/parallel/{0-random-distro$/{ubuntu_22.04} 0-start 1-tasks mon_election/classic upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api test_rbd_python}} duration: 7316.675312280655 failure_reason: '"2024-04-07T16:22:06.783529+0000 mon.a (mon.0) 359 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log' flavor: default owner: scheduled_yuriw@teuthology success: false 2024-04-07T17:54:21.651 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-07T17:54:21.746 INFO:teuthology.run:FAIL